2024-04-17T09:36:27.506 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-17T09:36:27.506 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi --verbose --archive /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660029 --owner scheduled_rishabh@teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} -- /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660029/orig.config.yaml 2024-04-17T09:36:27.550 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T09:36:27.631 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660029 branch: wip-rishabh-testing-20240416.193735 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} email: null first_in_suite: false job_id: '7660029' kernel: flavor: default kdb: true sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-rishabh-testing-20240416.193735 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: a8c84f5997950fd7d008204839253291d52e6eda ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a8c84f5997950fd7d008204839253291d52e6eda subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240416.193735 sha1: a8c84f5997950fd7d008204839253291d52e6eda owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 3521 sha1: a8c84f5997950fd7d008204839253291d52e6eda sleep_before_teardown: 0 subset: 410/512 suite: fs suite_branch: wip-rishabh-testing-20240416.193735 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a8c84f5997950fd7d008204839253291d52e6eda syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi032.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAQNVG61I8ge12Sn5QATesek5EUC5OJJA2Lfl5bn1TFZolZGwJ5+ic2yXdRNKlEZ2eFM7ed+K/Dl/RatKvM26U8= smithi144.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDmzWqi31jF9Vw1PLRKqPj3ams9Rzq8QnAwo7r+CIq6CX4JoYwuACajkXfQHwmdK6OwV3MpfHFw9u4d/frZwoZg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-17_04:58:44 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-17T09:36:27.631 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa; will attempt to use it 2024-04-17T09:36:27.632 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks 2024-04-17T09:36:27.632 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-17T09:36:27.633 INFO:teuthology.task.internal:Checking packages... 2024-04-17T09:36:27.654 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'a8c84f5997950fd7d008204839253291d52e6eda' 2024-04-17T09:36:27.655 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-17T09:36:27.655 INFO:teuthology.packaging:ref: None 2024-04-17T09:36:27.655 INFO:teuthology.packaging:tag: None 2024-04-17T09:36:27.655 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240416.193735 2024-04-17T09:36:27.655 INFO:teuthology.packaging:sha1: a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T09:36:27.655 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-rishabh-testing-20240416.193735 2024-04-17T09:36:27.864 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3133-ga8c84f59-1jammy 2024-04-17T09:36:27.865 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-17T09:36:27.871 INFO:teuthology.task.internal:no buildpackages task found 2024-04-17T09:36:27.871 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-17T09:36:27.876 INFO:teuthology.task.internal:Saving configuration 2024-04-17T09:36:27.892 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-17T09:36:27.905 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-17T09:36:27.930 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660029', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 09:31:06.962818', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKeshKQymlXDOmafG60lXBtcdG549g0oAlTq0K3uWiJw'} 2024-04-17T09:36:27.950 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660029', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 09:31:06.964098', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJEdEH/F+UFujy/DTWwWkTT3nDaolsUUcKNXk6QfLrQW'} 2024-04-17T09:36:27.950 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-17T09:36:27.955 INFO:teuthology.task.internal:roles: ubuntu@smithi032.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-17T09:36:27.955 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-17T09:36:27.956 INFO:teuthology.run_tasks:Running task console_log... 2024-04-17T09:36:28.023 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7dd60b14c0>, signals=[15]) 2024-04-17T09:36:28.024 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-17T09:36:28.029 INFO:teuthology.task.internal:Opening connections... 2024-04-17T09:36:28.030 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T09:36:28.032 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T09:36:28.112 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T09:36:28.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T09:36:28.190 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-17T09:36:28.196 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2024-04-17T09:36:28.201 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2024-04-17T09:36:28.201 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2024-04-17T09:36:28.248 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-17T09:36:28.248 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="Ubuntu" 2024-04-17T09:36:28.248 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="22.04" 2024-04-17T09:36:28.248 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-17T09:36:28.248 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_CODENAME=jammy 2024-04-17T09:36:28.248 INFO:teuthology.orchestra.run.smithi032.stdout:ID=ubuntu 2024-04-17T09:36:28.248 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE=debian 2024-04-17T09:36:28.248 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-17T09:36:28.248 INFO:teuthology.orchestra.run.smithi032.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-17T09:36:28.248 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-17T09:36:28.248 INFO:teuthology.orchestra.run.smithi032.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-17T09:36:28.248 INFO:teuthology.orchestra.run.smithi032.stdout:UBUNTU_CODENAME=jammy 2024-04-17T09:36:28.249 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2024-04-17T09:36:28.272 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2024-04-17T09:36:28.277 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2024-04-17T09:36:28.277 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2024-04-17T09:36:28.324 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-17T09:36:28.324 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="Ubuntu" 2024-04-17T09:36:28.324 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="22.04" 2024-04-17T09:36:28.324 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-17T09:36:28.324 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_CODENAME=jammy 2024-04-17T09:36:28.324 INFO:teuthology.orchestra.run.smithi144.stdout:ID=ubuntu 2024-04-17T09:36:28.324 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE=debian 2024-04-17T09:36:28.324 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-17T09:36:28.324 INFO:teuthology.orchestra.run.smithi144.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-17T09:36:28.324 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-17T09:36:28.324 INFO:teuthology.orchestra.run.smithi144.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-17T09:36:28.325 INFO:teuthology.orchestra.run.smithi144.stdout:UBUNTU_CODENAME=jammy 2024-04-17T09:36:28.325 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2024-04-17T09:36:28.347 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-17T09:36:28.355 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-17T09:36:28.359 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-17T09:36:28.360 DEBUG:teuthology.orchestra.run.smithi032:> test '!' -e /home/ubuntu/cephtest 2024-04-17T09:36:28.362 DEBUG:teuthology.orchestra.run.smithi144:> test '!' -e /home/ubuntu/cephtest 2024-04-17T09:36:28.371 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-17T09:36:28.376 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-17T09:36:28.376 DEBUG:teuthology.orchestra.run.smithi032:> test -z $(ls -A /var/lib/ceph) 2024-04-17T09:36:28.409 DEBUG:teuthology.orchestra.run.smithi144:> test -z $(ls -A /var/lib/ceph) 2024-04-17T09:36:28.439 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-17T09:36:28.491 INFO:teuthology.run_tasks:Running task kernel... 2024-04-17T09:36:28.504 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'} 2024-04-17T09:36:28.504 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'}}, timeout 300 2024-04-17T09:36:28.504 DEBUG:teuthology.orchestra.run.smithi032:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-17T09:36:28.505 DEBUG:teuthology.orchestra.run.smithi144:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-17T09:36:28.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:36:28.510 DEBUG:teuthology.task.kernel:sha1 for mon.a is 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-17T09:36:28.510 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8"... 2024-04-17T09:36:28.510 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2024-04-17T09:36:28.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:36:28.511 DEBUG:teuthology.task.kernel:sha1 for mon.b is 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-17T09:36:28.511 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8"... 2024-04-17T09:36:28.512 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-04-17T09:36:28.556 INFO:teuthology.orchestra.run.smithi032.stdout:5.15.0-56-generic 2024-04-17T09:36:28.556 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-17T09:36:28.556 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-17T09:36:28.556 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-17T09:36:28.564 INFO:teuthology.orchestra.run.smithi144.stdout:5.15.0-56-generic 2024-04-17T09:36:28.564 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-17T09:36:28.564 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-17T09:36:28.564 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-17T09:36:28.694 INFO:teuthology.task.kernel:Installing linux-firmware on mon.a... 2024-04-17T09:36:28.694 DEBUG:teuthology.orchestra.run.smithi032:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-04-17T09:36:28.709 INFO:teuthology.task.kernel:Installing linux-firmware on mon.b... 2024-04-17T09:36:28.710 DEBUG:teuthology.orchestra.run.smithi144:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-04-17T09:36:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-04-17T09:36:28.727 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-04-17T09:36:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-04-17T09:36:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-04-17T09:36:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-04-17T09:36:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:hint: 2024-04-17T09:36:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:hint: git config --global init.defaultBranch 2024-04-17T09:36:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:hint: 2024-04-17T09:36:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-04-17T09:36:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-04-17T09:36:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:hint: 2024-04-17T09:36:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:hint: git branch -m 2024-04-17T09:36:28.828 INFO:teuthology.orchestra.run.smithi032.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-04-17T09:36:28.830 DEBUG:teuthology.orchestra.run.smithi032:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-04-17T09:36:28.891 INFO:teuthology.orchestra.run.smithi144.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-04-17T09:36:28.891 INFO:teuthology.orchestra.run.smithi144.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-04-17T09:36:28.891 INFO:teuthology.orchestra.run.smithi144.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-04-17T09:36:28.891 INFO:teuthology.orchestra.run.smithi144.stderr:hint: 2024-04-17T09:36:28.891 INFO:teuthology.orchestra.run.smithi144.stderr:hint: git config --global init.defaultBranch 2024-04-17T09:36:28.891 INFO:teuthology.orchestra.run.smithi144.stderr:hint: 2024-04-17T09:36:28.891 INFO:teuthology.orchestra.run.smithi144.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-04-17T09:36:28.892 INFO:teuthology.orchestra.run.smithi144.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-04-17T09:36:28.892 INFO:teuthology.orchestra.run.smithi144.stderr:hint: 2024-04-17T09:36:28.892 INFO:teuthology.orchestra.run.smithi144.stderr:hint: git branch -m 2024-04-17T09:36:28.893 INFO:teuthology.orchestra.run.smithi144.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-04-17T09:36:28.894 DEBUG:teuthology.orchestra.run.smithi144:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-04-17T09:36:28.896 DEBUG:teuthology.orchestra.run.smithi032:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-04-17T09:36:28.954 DEBUG:teuthology.orchestra.run.smithi032:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-04-17T09:36:28.958 DEBUG:teuthology.orchestra.run.smithi144:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-04-17T09:36:29.012 DEBUG:teuthology.orchestra.run.smithi144:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-04-17T09:38:41.005 INFO:teuthology.orchestra.run.smithi032.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-04-17T09:38:41.006 INFO:teuthology.orchestra.run.smithi032.stderr: * [new branch] main -> origin/main 2024-04-17T09:38:41.039 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190312 -> 20190312 2024-04-17T09:38:41.039 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190416 -> 20190416 2024-04-17T09:38:41.040 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190514 -> 20190514 2024-04-17T09:38:41.040 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190618 -> 20190618 2024-04-17T09:38:41.040 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190717 -> 20190717 2024-04-17T09:38:41.040 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190815 -> 20190815 2024-04-17T09:38:41.040 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190923 -> 20190923 2024-04-17T09:38:41.040 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20191022 -> 20191022 2024-04-17T09:38:41.040 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20191215 -> 20191215 2024-04-17T09:38:41.040 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200122 -> 20200122 2024-04-17T09:38:41.042 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200316 -> 20200316 2024-04-17T09:38:41.042 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200421 -> 20200421 2024-04-17T09:38:41.042 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200519 -> 20200519 2024-04-17T09:38:41.042 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200619 -> 20200619 2024-04-17T09:38:41.042 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200721 -> 20200721 2024-04-17T09:38:41.042 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200817 -> 20200817 2024-04-17T09:38:41.042 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200918 -> 20200918 2024-04-17T09:38:41.042 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20201022 -> 20201022 2024-04-17T09:38:41.043 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20201118 -> 20201118 2024-04-17T09:38:41.043 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20201218 -> 20201218 2024-04-17T09:38:41.043 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210208 -> 20210208 2024-04-17T09:38:41.043 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210315 -> 20210315 2024-04-17T09:38:41.043 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210511 -> 20210511 2024-04-17T09:38:41.043 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210716 -> 20210716 2024-04-17T09:38:41.043 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210818 -> 20210818 2024-04-17T09:38:41.043 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210919 -> 20210919 2024-04-17T09:38:41.043 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20211027 -> 20211027 2024-04-17T09:38:41.043 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20211216 -> 20211216 2024-04-17T09:38:41.045 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220209 -> 20220209 2024-04-17T09:38:41.045 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220310 -> 20220310 2024-04-17T09:38:41.045 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220411 -> 20220411 2024-04-17T09:38:41.045 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220509 -> 20220509 2024-04-17T09:38:41.045 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220610 -> 20220610 2024-04-17T09:38:41.045 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220708 -> 20220708 2024-04-17T09:38:41.046 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220815 -> 20220815 2024-04-17T09:38:41.046 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220913 -> 20220913 2024-04-17T09:38:41.046 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20221012 -> 20221012 2024-04-17T09:38:41.046 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20221109 -> 20221109 2024-04-17T09:38:41.046 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20221214 -> 20221214 2024-04-17T09:38:41.046 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230117 -> 20230117 2024-04-17T09:38:41.046 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230210 -> 20230210 2024-04-17T09:38:41.046 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230310 -> 20230310 2024-04-17T09:38:41.046 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230404 -> 20230404 2024-04-17T09:38:41.046 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230515 -> 20230515 2024-04-17T09:38:41.046 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230625 -> 20230625 2024-04-17T09:38:41.046 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230804 -> 20230804 2024-04-17T09:38:41.046 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230919 -> 20230919 2024-04-17T09:38:41.046 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20231030 -> 20231030 2024-04-17T09:38:41.046 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20231111 -> 20231111 2024-04-17T09:38:41.047 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20231211 -> 20231211 2024-04-17T09:38:41.047 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20240115 -> 20240115 2024-04-17T09:38:41.047 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20240220 -> 20240220 2024-04-17T09:38:41.047 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20240312 -> 20240312 2024-04-17T09:38:41.047 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20240410 -> 20240410 2024-04-17T09:38:45.441 INFO:teuthology.orchestra.run.smithi032.stderr:Updating files: 48% (1744/3602) Updating files: 49% (1765/3602) Updating files: 50% (1801/3602) Updating files: 51% (1838/3602) Updating files: 52% (1874/3602) Updating files: 53% (1910/3602) Updating files: 54% (1946/3602) Updating files: 55% (1982/3602) Updating files: 56% (2018/3602) Updating files: 57% (2054/3602) Updating files: 58% (2090/3602) Updating files: 59% (2126/3602) Updating files: 60% (2162/3602) Updating files: 61% (2198/3602) Updating files: 61% (2219/3602) Updating files: 62% (2234/3602) Updating files: 63% (2270/3602) Updating files: 64% (2306/3602) Updating files: 65% (2342/3602) Updating files: 66% (2378/3602) Updating files: 67% (2414/3602) Updating files: 68% (2450/3602) Updating files: 69% (2486/3602) Updating files: 70% (2522/3602) Updating files: 71% (2558/3602) Updating files: 71% (2584/3602) Updating files: 72% (2594/3602) Updating files: 73% (2630/3602) Updating files: 74% (2666/3602) Updating files: 75% (2702/3602) Updating files: 76% (2738/3602) Updating files: 77% (2774/3602) Updating files: 78% (2810/3602) Updating files: 79% (2846/3602) Updating files: 80% (2882/3602) Updating files: 81% (2918/3602) Updating files: 82% (2954/3602) Updating files: 83% (2990/3602) Updating files: 83% (3012/3602) Updating files: 84% (3026/3602) Updating files: 85% (3062/3602) Updating files: 86% (3098/3602) Updating files: 87% (3134/3602) Updating files: 88% (3170/3602) Updating files: 89% (3206/3602) Updating files: 90% (3242/3602) Updating files: 91% (3278/3602) Updating files: 92% (3314/3602) Updating files: 93% (3350/3602) Updating files: 94% (3386/3602) Updating files: 95% (3422/3602) Updating files: 96% (3458/3602) Updating files: 97% (3494/3602) Updating files: 98% (3530/3602) Updating files: 99% (3566/3602) Updating files: 100% (3602/3602) Updating files: 100% (3602/3602), done. 2024-04-17T09:38:45.447 INFO:teuthology.orchestra.run.smithi032.stdout:HEAD is now at 5da74b16 Merge branch 'robot/pr-0-1712895838' into 'main' 2024-04-17T09:38:45.469 INFO:teuthology.task.kernel:Downloading kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on mon.a... 2024-04-17T09:38:45.470 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-17T09:38:45.617 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g3f60ed14f7c1/ 2024-04-17T09:38:45.617 DEBUG:teuthology.orchestra.run.smithi032:> rm -f /tmp/linux-image.deb && echo linux-image-6.9.0-rc3-g3f60ed14f7c1_6.9.0-rc3-g3f60ed14f7c1-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://1.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g3f60ed14f7c1/ --input-file=- 2024-04-17T09:38:45.620 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi032.front.sepia.ceph.com to complete... 2024-04-17T09:38:47.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17 09:38:47 URL:https://1.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g3f60ed14f7c1/linux-image-6.9.0-rc3-g3f60ed14f7c1_6.9.0-rc3-g3f60ed14f7c1-1_amd64.deb [44839714/44839714] -> "/tmp/linux-image.deb" [1] 2024-04-17T09:38:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:FINISHED --2024-04-17 09:38:47-- 2024-04-17T09:38:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:Total wall clock time: 1.7s 2024-04-17T09:38:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:Downloaded: 1 files, 43M in 1.5s (27.6 MB/s) 2024-04-17T09:38:47.316 INFO:teuthology.task.kernel:Installing kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on mon.a... 2024-04-17T09:38:47.316 DEBUG:teuthology.orchestra.run.smithi032:> sudo dpkg -i /tmp/linux-image.deb 2024-04-17T09:38:47.566 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-image-6.9.0-rc3-g3f60ed14f7c1. 2024-04-17T09:38:49.640 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-04-17T09:38:49.644 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-04-17T09:38:49.752 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-image-6.9.0-rc3-g3f60ed14f7c1 (6.9.0-rc3-g3f60ed14f7c1-1) ... 2024-04-17T09:38:53.405 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-image-6.9.0-rc3-g3f60ed14f7c1 (6.9.0-rc3-g3f60ed14f7c1-1) ... 2024-04-17T09:38:53.513 INFO:teuthology.orchestra.run.smithi032.stderr:update-initramfs: Generating /boot/initrd.img-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:38:55.319 INFO:teuthology.orchestra.run.smithi144.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-04-17T09:38:55.320 INFO:teuthology.orchestra.run.smithi144.stderr: * [new branch] main -> origin/main 2024-04-17T09:38:55.353 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20190312 -> 20190312 2024-04-17T09:38:55.353 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20190416 -> 20190416 2024-04-17T09:38:55.353 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20190514 -> 20190514 2024-04-17T09:38:55.353 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20190618 -> 20190618 2024-04-17T09:38:55.355 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20190717 -> 20190717 2024-04-17T09:38:55.355 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20190815 -> 20190815 2024-04-17T09:38:55.355 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20190923 -> 20190923 2024-04-17T09:38:55.355 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20191022 -> 20191022 2024-04-17T09:38:55.355 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20191215 -> 20191215 2024-04-17T09:38:55.355 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20200122 -> 20200122 2024-04-17T09:38:55.355 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20200316 -> 20200316 2024-04-17T09:38:55.355 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20200421 -> 20200421 2024-04-17T09:38:55.355 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20200519 -> 20200519 2024-04-17T09:38:55.355 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20200619 -> 20200619 2024-04-17T09:38:55.355 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20200721 -> 20200721 2024-04-17T09:38:55.356 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20200817 -> 20200817 2024-04-17T09:38:55.358 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20200918 -> 20200918 2024-04-17T09:38:55.358 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20201022 -> 20201022 2024-04-17T09:38:55.358 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20201118 -> 20201118 2024-04-17T09:38:55.358 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20201218 -> 20201218 2024-04-17T09:38:55.358 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20210208 -> 20210208 2024-04-17T09:38:55.358 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20210315 -> 20210315 2024-04-17T09:38:55.358 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20210511 -> 20210511 2024-04-17T09:38:55.358 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20210716 -> 20210716 2024-04-17T09:38:55.358 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20210818 -> 20210818 2024-04-17T09:38:55.358 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20210919 -> 20210919 2024-04-17T09:38:55.358 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20211027 -> 20211027 2024-04-17T09:38:55.358 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20211216 -> 20211216 2024-04-17T09:38:55.358 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20220209 -> 20220209 2024-04-17T09:38:55.359 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20220310 -> 20220310 2024-04-17T09:38:55.359 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20220411 -> 20220411 2024-04-17T09:38:55.359 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20220509 -> 20220509 2024-04-17T09:38:55.359 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20220610 -> 20220610 2024-04-17T09:38:55.359 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20220708 -> 20220708 2024-04-17T09:38:55.359 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20220815 -> 20220815 2024-04-17T09:38:55.359 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20220913 -> 20220913 2024-04-17T09:38:55.361 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20221012 -> 20221012 2024-04-17T09:38:55.361 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20221109 -> 20221109 2024-04-17T09:38:55.361 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20221214 -> 20221214 2024-04-17T09:38:55.361 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20230117 -> 20230117 2024-04-17T09:38:55.361 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20230210 -> 20230210 2024-04-17T09:38:55.361 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20230310 -> 20230310 2024-04-17T09:38:55.361 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20230404 -> 20230404 2024-04-17T09:38:55.361 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20230515 -> 20230515 2024-04-17T09:38:55.361 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20230625 -> 20230625 2024-04-17T09:38:55.361 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20230804 -> 20230804 2024-04-17T09:38:55.361 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20230919 -> 20230919 2024-04-17T09:38:55.362 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20231030 -> 20231030 2024-04-17T09:38:55.362 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20231111 -> 20231111 2024-04-17T09:38:55.362 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20231211 -> 20231211 2024-04-17T09:38:55.362 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20240115 -> 20240115 2024-04-17T09:38:55.362 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20240220 -> 20240220 2024-04-17T09:38:55.362 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20240312 -> 20240312 2024-04-17T09:38:55.362 INFO:teuthology.orchestra.run.smithi144.stderr: * [new tag] 20240410 -> 20240410 2024-04-17T09:38:59.564 INFO:teuthology.orchestra.run.smithi144.stderr:Updating files: 48% (1764/3602) Updating files: 49% (1765/3602) Updating files: 50% (1801/3602) Updating files: 51% (1838/3602) Updating files: 52% (1874/3602) Updating files: 53% (1910/3602) Updating files: 54% (1946/3602) Updating files: 55% (1982/3602) Updating files: 56% (2018/3602) Updating files: 57% (2054/3602) Updating files: 58% (2090/3602) Updating files: 59% (2126/3602) Updating files: 60% (2162/3602) Updating files: 61% (2198/3602) Updating files: 62% (2234/3602) Updating files: 63% (2270/3602) Updating files: 63% (2292/3602) Updating files: 64% (2306/3602) Updating files: 65% (2342/3602) Updating files: 66% (2378/3602) Updating files: 67% (2414/3602) Updating files: 68% (2450/3602) Updating files: 69% (2486/3602) Updating files: 70% (2522/3602) Updating files: 71% (2558/3602) Updating files: 72% (2594/3602) Updating files: 73% (2630/3602) Updating files: 74% (2666/3602) Updating files: 75% (2702/3602) Updating files: 76% (2738/3602) Updating files: 77% (2774/3602) Updating files: 77% (2789/3602) Updating files: 78% (2810/3602) Updating files: 79% (2846/3602) Updating files: 80% (2882/3602) Updating files: 81% (2918/3602) Updating files: 82% (2954/3602) Updating files: 83% (2990/3602) Updating files: 84% (3026/3602) Updating files: 85% (3062/3602) Updating files: 86% (3098/3602) Updating files: 87% (3134/3602) Updating files: 88% (3170/3602) Updating files: 88% (3179/3602) Updating files: 89% (3206/3602) Updating files: 90% (3242/3602) Updating files: 91% (3278/3602) Updating files: 92% (3314/3602) Updating files: 93% (3350/3602) Updating files: 94% (3386/3602) Updating files: 95% (3422/3602) Updating files: 96% (3458/3602) Updating files: 97% (3494/3602) Updating files: 98% (3530/3602) Updating files: 99% (3566/3602) Updating files: 100% (3602/3602) Updating files: 100% (3602/3602), done. 2024-04-17T09:38:59.570 INFO:teuthology.orchestra.run.smithi144.stdout:HEAD is now at 5da74b16 Merge branch 'robot/pr-0-1712895838' into 'main' 2024-04-17T09:38:59.590 INFO:teuthology.task.kernel:Downloading kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on mon.b... 2024-04-17T09:38:59.591 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-17T09:38:59.743 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g3f60ed14f7c1/ 2024-04-17T09:38:59.743 DEBUG:teuthology.orchestra.run.smithi144:> rm -f /tmp/linux-image.deb && echo linux-image-6.9.0-rc3-g3f60ed14f7c1_6.9.0-rc3-g3f60ed14f7c1-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://1.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g3f60ed14f7c1/ --input-file=- 2024-04-17T09:38:59.746 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi144.front.sepia.ceph.com to complete... 2024-04-17T09:39:04.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17 09:39:04 URL:https://1.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g3f60ed14f7c1/linux-image-6.9.0-rc3-g3f60ed14f7c1_6.9.0-rc3-g3f60ed14f7c1-1_amd64.deb [44839714/44839714] -> "/tmp/linux-image.deb" [1] 2024-04-17T09:39:04.692 INFO:teuthology.orchestra.run.smithi144.stderr:FINISHED --2024-04-17 09:39:04-- 2024-04-17T09:39:04.692 INFO:teuthology.orchestra.run.smithi144.stderr:Total wall clock time: 4.9s 2024-04-17T09:39:04.692 INFO:teuthology.orchestra.run.smithi144.stderr:Downloaded: 1 files, 43M in 4.7s (9.11 MB/s) 2024-04-17T09:39:04.696 INFO:teuthology.task.kernel:Installing kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on mon.b... 2024-04-17T09:39:04.697 DEBUG:teuthology.orchestra.run.smithi144:> sudo dpkg -i /tmp/linux-image.deb 2024-04-17T09:39:04.974 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package linux-image-6.9.0-rc3-g3f60ed14f7c1. 2024-04-17T09:39:07.038 INFO:teuthology.orchestra.run.smithi144.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-04-17T09:39:07.042 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-04-17T09:39:07.313 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-image-6.9.0-rc3-g3f60ed14f7c1 (6.9.0-rc3-g3f60ed14f7c1-1) ... 2024-04-17T09:39:11.310 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-image-6.9.0-rc3-g3f60ed14f7c1 (6.9.0-rc3-g3f60ed14f7c1-1) ... 2024-04-17T09:39:11.425 INFO:teuthology.orchestra.run.smithi144.stderr:update-initramfs: Generating /boot/initrd.img-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:39:15.493 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub' 2024-04-17T09:39:15.508 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-17T09:39:15.527 INFO:teuthology.orchestra.run.smithi032.stderr:Generating grub configuration file ... 2024-04-17T09:39:15.820 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:39:15.836 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:39:16.141 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-17T09:39:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-17T09:39:16.215 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-17T09:39:16.220 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-17T09:39:16.352 INFO:teuthology.orchestra.run.smithi032.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-17T09:39:16.352 INFO:teuthology.orchestra.run.smithi032.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-17T09:39:16.352 INFO:teuthology.orchestra.run.smithi032.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-17T09:39:16.395 INFO:teuthology.orchestra.run.smithi032.stderr:done 2024-04-17T09:39:16.589 DEBUG:teuthology.orchestra.run.smithi032:> dpkg-deb -c /tmp/linux-image.deb 2024-04-17T09:39:16.598 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./ 2024-04-17T09:39:16.598 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./boot/ 2024-04-17T09:39:16.598 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7002012 2024-04-11 18:50 ./boot/System.map-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:39:16.617 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 172073 2024-04-11 18:50 ./boot/config-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:39:16.617 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13120512 2024-04-11 18:50 ./boot/vmlinuz-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:39:16.639 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./etc/ 2024-04-17T09:39:16.639 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./etc/kernel/ 2024-04-17T09:39:16.639 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./etc/kernel/postinst.d/ 2024-04-17T09:39:16.639 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./etc/kernel/postrm.d/ 2024-04-17T09:39:16.639 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./etc/kernel/preinst.d/ 2024-04-17T09:39:16.639 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./etc/kernel/prerm.d/ 2024-04-17T09:39:16.640 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/ 2024-04-17T09:39:16.640 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/ 2024-04-17T09:39:16.640 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/ 2024-04-17T09:39:16.640 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/ 2024-04-17T09:39:16.640 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/ 2024-04-17T09:39:16.640 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/ 2024-04-17T09:39:16.640 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/ 2024-04-17T09:39:16.640 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 507569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/aesni-intel.ko 2024-04-17T09:39:16.644 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-17T09:39:16.644 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-17T09:39:16.644 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-17T09:39:16.644 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 97953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-17T09:39:16.644 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 109905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-17T09:39:16.644 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 155249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-17T09:39:16.644 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-17T09:39:16.644 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-17T09:39:16.645 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 87009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-17T09:39:16.645 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-17T09:39:16.645 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-17T09:39:16.645 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-17T09:39:16.645 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-17T09:39:16.645 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44657 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-17T09:39:16.645 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-17T09:39:16.645 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-17T09:39:16.645 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-17T09:39:16.645 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-17T09:39:16.646 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14809 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-17T09:39:16.646 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/ 2024-04-17T09:39:16.646 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/ 2024-04-17T09:39:16.646 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce/ 2024-04-17T09:39:16.646 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-17T09:39:16.646 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpuid.ko 2024-04-17T09:39:16.646 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/msr.ko 2024-04-17T09:39:16.646 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/ 2024-04-17T09:39:16.646 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 444425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-17T09:39:16.646 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1035881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-17T09:39:16.647 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 2849609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm.ko 2024-04-17T09:39:16.670 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ 2024-04-17T09:39:16.670 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 65025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/af_alg.ko 2024-04-17T09:39:16.671 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/algif_hash.ko 2024-04-17T09:39:16.671 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/algif_skcipher.ko 2024-04-17T09:39:16.671 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ansi_cprng.ko 2024-04-17T09:39:16.671 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/anubis.ko 2024-04-17T09:39:16.671 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10929 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/arc4.ko 2024-04-17T09:39:16.671 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/asymmetric_keys/ 2024-04-17T09:39:16.671 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-04-17T09:39:16.671 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/ 2024-04-17T09:39:16.671 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_memcpy.ko 2024-04-17T09:39:16.671 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_pq.ko 2024-04-17T09:39:16.671 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-17T09:39:16.671 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_tx.ko 2024-04-17T09:39:16.672 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_xor.ko 2024-04-17T09:39:16.672 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/raid6test.ko 2024-04-17T09:39:16.672 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authenc.ko 2024-04-17T09:39:16.672 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authencesn.ko 2024-04-17T09:39:16.672 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blake2b_generic.ko 2024-04-17T09:39:16.672 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_common.ko 2024-04-17T09:39:16.672 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_generic.ko 2024-04-17T09:39:16.672 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/camellia_generic.ko 2024-04-17T09:39:16.672 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast5_generic.ko 2024-04-17T09:39:16.672 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast6_generic.ko 2024-04-17T09:39:16.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast_common.ko 2024-04-17T09:39:16.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ccm.ko 2024-04-17T09:39:16.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cmac.ko 2024-04-17T09:39:16.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crc32_generic.ko 2024-04-17T09:39:16.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cryptd.ko 2024-04-17T09:39:16.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_engine.ko 2024-04-17T09:39:16.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_simd.ko 2024-04-17T09:39:16.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_user.ko 2024-04-17T09:39:16.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cts.ko 2024-04-17T09:39:16.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/des_generic.ko 2024-04-17T09:39:16.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/dh_generic.ko 2024-04-17T09:39:16.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/echainiv.ko 2024-04-17T09:39:16.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/essiv.ko 2024-04-17T09:39:16.674 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/fcrypt.ko 2024-04-17T09:39:16.674 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/khazad.ko 2024-04-17T09:39:16.674 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lrw.ko 2024-04-17T09:39:16.674 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4.ko 2024-04-17T09:39:16.674 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4hc.ko 2024-04-17T09:39:16.674 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13393 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/md4.ko 2024-04-17T09:39:16.674 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/michael_mic.ko 2024-04-17T09:39:16.674 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcbc.ko 2024-04-17T09:39:16.674 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcrypt.ko 2024-04-17T09:39:16.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/rmd160.ko 2024-04-17T09:39:16.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/seed.ko 2024-04-17T09:39:16.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/serpent_generic.ko 2024-04-17T09:39:16.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 169065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tcrypt.ko 2024-04-17T09:39:16.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tea.ko 2024-04-17T09:39:16.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_common.ko 2024-04-17T09:39:16.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_generic.ko 2024-04-17T09:39:16.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/vmac.ko 2024-04-17T09:39:16.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/wp512.ko 2024-04-17T09:39:16.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xcbc.ko 2024-04-17T09:39:16.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xor.ko 2024-04-17T09:39:16.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xts.ko 2024-04-17T09:39:16.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xxhash_generic.ko 2024-04-17T09:39:16.675 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ 2024-04-17T09:39:16.675 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/ 2024-04-17T09:39:16.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_extlog.ko 2024-04-17T09:39:16.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-17T09:39:16.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_pad.ko 2024-04-17T09:39:16.676 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/apei/ 2024-04-17T09:39:16.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/apei/einj.ko 2024-04-17T09:39:16.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/ec_sys.ko 2024-04-17T09:39:16.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/platform_profile.ko 2024-04-17T09:39:16.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbs.ko 2024-04-17T09:39:16.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbshc.ko 2024-04-17T09:39:16.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 119033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/video.ko 2024-04-17T09:39:16.676 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ 2024-04-17T09:39:16.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/acard-ahci.ko 2024-04-17T09:39:16.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 96825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ahci.ko 2024-04-17T09:39:16.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ahci_platform.ko 2024-04-17T09:39:16.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 107737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/libahci.ko 2024-04-17T09:39:16.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/libahci_platform.ko 2024-04-17T09:39:16.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_acpi.ko 2024-04-17T09:39:16.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ali.ko 2024-04-17T09:39:16.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_amd.ko 2024-04-17T09:39:16.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_artop.ko 2024-04-17T09:39:16.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atiixp.ko 2024-04-17T09:39:16.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28785 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atp867x.ko 2024-04-17T09:39:16.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd640.ko 2024-04-17T09:39:16.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28929 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd64x.ko 2024-04-17T09:39:16.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cypress.ko 2024-04-17T09:39:16.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_efar.ko 2024-04-17T09:39:16.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt366.ko 2024-04-17T09:39:16.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt37x.ko 2024-04-17T09:39:16.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-17T09:39:16.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-17T09:39:16.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it8213.ko 2024-04-17T09:39:16.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it821x.ko 2024-04-17T09:39:16.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_jmicron.ko 2024-04-17T09:39:16.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_legacy.ko 2024-04-17T09:39:16.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_marvell.ko 2024-04-17T09:39:16.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_mpiix.ko 2024-04-17T09:39:16.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_netcell.ko 2024-04-17T09:39:16.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ninja32.ko 2024-04-17T09:39:16.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87410.ko 2024-04-17T09:39:16.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87415.ko 2024-04-17T09:39:16.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_oldpiix.ko 2024-04-17T09:39:16.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_opti.ko 2024-04-17T09:39:16.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_optidma.ko 2024-04-17T09:39:16.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pcmcia.ko 2024-04-17T09:39:16.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-17T09:39:16.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-17T09:39:16.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_piccolo.ko 2024-04-17T09:39:16.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_radisys.ko 2024-04-17T09:39:16.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_rdc.ko 2024-04-17T09:39:16.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_rz1000.ko 2024-04-17T09:39:16.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sch.ko 2024-04-17T09:39:16.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_serverworks.ko 2024-04-17T09:39:16.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sil680.ko 2024-04-17T09:39:16.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sl82c105.ko 2024-04-17T09:39:16.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_triflex.ko 2024-04-17T09:39:16.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_via.ko 2024-04-17T09:39:16.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pdc_adma.ko 2024-04-17T09:39:16.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_inic162x.ko 2024-04-17T09:39:16.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 102121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_mv.ko 2024-04-17T09:39:16.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 85409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_nv.ko 2024-04-17T09:39:16.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_promise.ko 2024-04-17T09:39:16.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_qstor.ko 2024-04-17T09:39:16.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil.ko 2024-04-17T09:39:16.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil24.ko 2024-04-17T09:39:16.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sis.ko 2024-04-17T09:39:16.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_svw.ko 2024-04-17T09:39:16.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sx4.ko 2024-04-17T09:39:16.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_uli.ko 2024-04-17T09:39:16.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_via.ko 2024-04-17T09:39:16.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_vsc.ko 2024-04-17T09:39:16.681 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay/ 2024-04-17T09:39:16.681 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay/ks0108.ko 2024-04-17T09:39:16.681 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma/ 2024-04-17T09:39:16.681 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 163969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma/bcma.ko 2024-04-17T09:39:16.681 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/ 2024-04-17T09:39:16.681 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe/ 2024-04-17T09:39:16.681 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 140865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe/aoe.ko 2024-04-17T09:39:16.681 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd/ 2024-04-17T09:39:16.681 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1241657 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd/drbd.ko 2024-04-17T09:39:16.681 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 203289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/floppy.ko 2024-04-17T09:39:16.681 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx/ 2024-04-17T09:39:16.681 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 115265 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-17T09:39:16.681 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 143433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/nbd.ko 2024-04-17T09:39:16.681 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk/ 2024-04-17T09:39:16.681 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 126617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk/null_blk.ko 2024-04-17T09:39:16.682 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 119873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/pktcdvd.ko 2024-04-17T09:39:16.682 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 313217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/rbd.ko 2024-04-17T09:39:16.682 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback/ 2024-04-17T09:39:16.682 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 117233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-17T09:39:16.682 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram/ 2024-04-17T09:39:16.682 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram/zram.ko 2024-04-17T09:39:16.682 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ 2024-04-17T09:39:16.682 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/applicom.ko 2024-04-17T09:39:16.682 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hangcheck-timer.ko 2024-04-17T09:39:16.682 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/ 2024-04-17T09:39:16.683 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-17T09:39:16.683 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-17T09:39:16.683 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-17T09:39:16.683 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/via-rng.ko 2024-04-17T09:39:16.683 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-17T09:39:16.683 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ 2024-04-17T09:39:16.683 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-17T09:39:16.683 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 186921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-17T09:39:16.683 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-17T09:39:16.683 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 179561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-17T09:39:16.683 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 85441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-17T09:39:16.683 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 65313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-17T09:39:16.683 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/lp.ko 2024-04-17T09:39:16.684 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave/ 2024-04-17T09:39:16.684 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 103409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave/mwave.ko 2024-04-17T09:39:16.684 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/nvram.ko 2024-04-17T09:39:16.684 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ppdev.ko 2024-04-17T09:39:16.684 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tlclk.ko 2024-04-17T09:39:16.684 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/ 2024-04-17T09:39:16.684 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-17T09:39:16.684 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-04-17T09:39:16.684 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-04-17T09:39:16.684 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39193 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-04-17T09:39:16.684 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-17T09:39:16.684 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-17T09:39:16.684 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-04-17T09:39:16.685 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/xillybus/ 2024-04-17T09:39:16.685 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/xillybus/xillybus_class.ko 2024-04-17T09:39:16.685 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62545 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/xillybus/xillybus_core.ko 2024-04-17T09:39:16.685 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13929 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-04-17T09:39:16.685 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/clk/ 2024-04-17T09:39:16.685 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/clk/clk-palmas.ko 2024-04-17T09:39:16.685 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/clk/clk-si5351.ko 2024-04-17T09:39:16.685 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/clk/clk-twl6040.ko 2024-04-17T09:39:16.685 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/clk/clk-wm831x.ko 2024-04-17T09:39:16.685 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/ 2024-04-17T09:39:16.685 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-17T09:39:16.685 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-17T09:39:16.685 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-17T09:39:16.685 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ 2024-04-17T09:39:16.685 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ 2024-04-17T09:39:16.686 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-17T09:39:16.686 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 333657 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp.ko 2024-04-17T09:39:16.686 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/ 2024-04-17T09:39:16.686 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/ 2024-04-17T09:39:16.686 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common/ 2024-04-17T09:39:16.686 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 751713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-17T09:39:16.749 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-04-17T09:39:16.749 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-17T09:39:16.749 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-aes.ko 2024-04-17T09:39:16.750 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-sha.ko 2024-04-17T09:39:16.750 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio/ 2024-04-17T09:39:16.750 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 90441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-17T09:39:16.750 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca/ 2024-04-17T09:39:16.750 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca/dca.ko 2024-04-17T09:39:16.750 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ 2024-04-17T09:39:16.750 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw/ 2024-04-17T09:39:16.750 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-17T09:39:16.750 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat/ 2024-04-17T09:39:16.750 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 146769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-17T09:39:16.750 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/ 2024-04-17T09:39:16.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 96145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/amd64_edac.ko 2024-04-17T09:39:16.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/e752x_edac.ko 2024-04-17T09:39:16.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/edac_mce_amd.ko 2024-04-17T09:39:16.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3000_edac.ko 2024-04-17T09:39:16.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3200_edac.ko 2024-04-17T09:39:16.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5100_edac.ko 2024-04-17T09:39:16.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5400_edac.ko 2024-04-17T09:39:16.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7300_edac.ko 2024-04-17T09:39:16.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 64161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7core_edac.ko 2024-04-17T09:39:16.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i82975x_edac.ko 2024-04-17T09:39:16.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/ie31200_edac.ko 2024-04-17T09:39:16.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 68697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/sb_edac.ko 2024-04-17T09:39:16.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/x38_edac.ko 2024-04-17T09:39:16.752 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/extcon/ 2024-04-17T09:39:16.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/extcon/extcon-gpio.ko 2024-04-17T09:39:16.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/extcon/extcon-max14577.ko 2024-04-17T09:39:16.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35601 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/extcon/extcon-max77693.ko 2024-04-17T09:39:16.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/extcon/extcon-max8997.ko 2024-04-17T09:39:16.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/extcon/extcon-palmas.ko 2024-04-17T09:39:16.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/extcon/extcon-rt8973a.ko 2024-04-17T09:39:16.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/extcon/extcon-sm5502.ko 2024-04-17T09:39:16.752 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/ 2024-04-17T09:39:16.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/dmi-sysfs.ko 2024-04-17T09:39:16.753 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/efi/ 2024-04-17T09:39:16.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/efi/efi-pstore.ko 2024-04-17T09:39:16.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-17T09:39:16.753 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/ 2024-04-17T09:39:16.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-adp5520.ko 2024-04-17T09:39:16.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-amd8111.ko 2024-04-17T09:39:16.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-arizona.ko 2024-04-17T09:39:16.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16745 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-da9052.ko 2024-04-17T09:39:16.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-da9055.ko 2024-04-17T09:39:16.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-dln2.ko 2024-04-17T09:39:16.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 81073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-f7188x.ko 2024-04-17T09:39:16.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-generic.ko 2024-04-17T09:39:16.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-ich.ko 2024-04-17T09:39:16.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-04-17T09:39:16.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-kempld.ko 2024-04-17T09:39:16.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-lp3943.ko 2024-04-17T09:39:16.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-max7300.ko 2024-04-17T09:39:16.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-max730x.ko 2024-04-17T09:39:16.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24745 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-max732x.ko 2024-04-17T09:39:16.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-04-17T09:39:16.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-pca953x.ko 2024-04-17T09:39:16.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-pcf857x.ko 2024-04-17T09:39:16.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-rdc321x.ko 2024-04-17T09:39:16.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-sch.ko 2024-04-17T09:39:16.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-sch311x.ko 2024-04-17T09:39:16.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14265 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-tps65912.ko 2024-04-17T09:39:16.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-twl4030.ko 2024-04-17T09:39:16.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-twl6040.ko 2024-04-17T09:39:16.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-17T09:39:16.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17897 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-vx855.ko 2024-04-17T09:39:16.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-wm831x.ko 2024-04-17T09:39:16.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-wm8350.ko 2024-04-17T09:39:16.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-wm8994.ko 2024-04-17T09:39:16.755 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/ 2024-04-17T09:39:16.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-a4tech.ko 2024-04-17T09:39:16.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43553 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-apple.ko 2024-04-17T09:39:16.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-appleir.ko 2024-04-17T09:39:16.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9393 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-aureal.ko 2024-04-17T09:39:16.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-axff.ko 2024-04-17T09:39:16.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-belkin.ko 2024-04-17T09:39:16.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-cherry.ko 2024-04-17T09:39:16.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-chicony.ko 2024-04-17T09:39:16.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 74393 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-cp2112.ko 2024-04-17T09:39:16.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-cypress.ko 2024-04-17T09:39:16.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-dr.ko 2024-04-17T09:39:16.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elecom.ko 2024-04-17T09:39:16.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elo.ko 2024-04-17T09:39:16.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-emsff.ko 2024-04-17T09:39:16.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-ezkey.ko 2024-04-17T09:39:16.757 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gaff.ko 2024-04-17T09:39:16.757 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-generic.ko 2024-04-17T09:39:16.757 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gt683r.ko 2024-04-17T09:39:16.757 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gyration.ko 2024-04-17T09:39:16.757 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-17T09:39:16.757 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-17T09:39:16.757 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtekff.ko 2024-04-17T09:39:16.757 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-hyperv.ko 2024-04-17T09:39:16.757 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-icade.ko 2024-04-17T09:39:16.757 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-kensington.ko 2024-04-17T09:39:16.757 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-keytouch.ko 2024-04-17T09:39:16.757 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-kye.ko 2024-04-17T09:39:16.757 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12265 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lcpower.ko 2024-04-17T09:39:16.757 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-led.ko 2024-04-17T09:39:16.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lenovo.ko 2024-04-17T09:39:16.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37553 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lg-g15.ko 2024-04-17T09:39:16.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 68081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-17T09:39:16.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 121369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-17T09:39:16.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 97953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech.ko 2024-04-17T09:39:16.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-magicmouse.ko 2024-04-17T09:39:16.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-microsoft.ko 2024-04-17T09:39:16.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-monterey.ko 2024-04-17T09:39:16.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-multitouch.ko 2024-04-17T09:39:16.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-ntrig.ko 2024-04-17T09:39:16.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-ortek.ko 2024-04-17T09:39:16.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-penmount.ko 2024-04-17T09:39:16.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-petalynx.ko 2024-04-17T09:39:16.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 91241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-picolcd.ko 2024-04-17T09:39:16.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-pl.ko 2024-04-17T09:39:16.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-plantronics.ko 2024-04-17T09:39:16.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-primax.ko 2024-04-17T09:39:16.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-rmi.ko 2024-04-17T09:39:16.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-17T09:39:16.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-common.ko 2024-04-17T09:39:16.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-17T09:39:16.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-17T09:39:16.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-17T09:39:16.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-17T09:39:16.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-17T09:39:16.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-17T09:39:16.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-17T09:39:16.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-17T09:39:16.760 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-17T09:39:16.760 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat.ko 2024-04-17T09:39:16.760 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-saitek.ko 2024-04-17T09:39:16.760 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-samsung.ko 2024-04-17T09:39:16.760 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-17T09:39:16.760 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sjoy.ko 2024-04-17T09:39:16.760 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sony.ko 2024-04-17T09:39:16.760 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-speedlink.ko 2024-04-17T09:39:16.760 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-steelseries.ko 2024-04-17T09:39:16.760 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sunplus.ko 2024-04-17T09:39:16.760 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-17T09:39:16.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tivo.ko 2024-04-17T09:39:16.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tmff.ko 2024-04-17T09:39:16.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-topseed.ko 2024-04-17T09:39:16.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-twinhan.ko 2024-04-17T09:39:16.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 81113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-uclogic.ko 2024-04-17T09:39:16.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-waltop.ko 2024-04-17T09:39:16.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 184745 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-wiimote.ko 2024-04-17T09:39:16.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-xinmo.ko 2024-04-17T09:39:16.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zpff.ko 2024-04-17T09:39:16.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zydacron.ko 2024-04-17T09:39:16.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 319921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid.ko 2024-04-17T09:39:16.762 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/uhid.ko 2024-04-17T09:39:16.762 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/usbhid/ 2024-04-17T09:39:16.762 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 153633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/usbhid/usbhid.ko 2024-04-17T09:39:16.762 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/usbhid/usbkbd.ko 2024-04-17T09:39:16.762 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/usbhid/usbmouse.ko 2024-04-17T09:39:16.762 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 261945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/wacom.ko 2024-04-17T09:39:16.762 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/ 2024-04-17T09:39:16.762 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 75065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_balloon.ko 2024-04-17T09:39:16.762 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 110681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_utils.ko 2024-04-17T09:39:16.762 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 382033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_vmbus.ko 2024-04-17T09:39:16.762 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ 2024-04-17T09:39:16.762 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 77153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru.ko 2024-04-17T09:39:16.763 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru3.ko 2024-04-17T09:39:16.763 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-17T09:39:16.763 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7414.ko 2024-04-17T09:39:16.763 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7418.ko 2024-04-17T09:39:16.763 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adc128d818.ko 2024-04-17T09:39:16.763 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1025.ko 2024-04-17T09:39:16.763 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 115057 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1026.ko 2024-04-17T09:39:16.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1029.ko 2024-04-17T09:39:16.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1031.ko 2024-04-17T09:39:16.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm9240.ko 2024-04-17T09:39:16.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ads7828.ko 2024-04-17T09:39:16.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7410.ko 2024-04-17T09:39:16.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7411.ko 2024-04-17T09:39:16.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7462.ko 2024-04-17T09:39:16.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7470.ko 2024-04-17T09:39:16.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 81473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7475.ko 2024-04-17T09:39:16.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7x10.ko 2024-04-17T09:39:16.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/amc6821.ko 2024-04-17T09:39:16.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 68865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/applesmc.ko 2024-04-17T09:39:16.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60193 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asb100.ko 2024-04-17T09:39:16.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asc7621.ko 2024-04-17T09:39:16.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-17T09:39:16.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/atxp1.ko 2024-04-17T09:39:16.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/coretemp.ko 2024-04-17T09:39:16.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/da9052-hwmon.ko 2024-04-17T09:39:16.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/da9055-hwmon.ko 2024-04-17T09:39:16.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-17T09:39:16.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 113201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dme1737.ko 2024-04-17T09:39:16.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds1621.ko 2024-04-17T09:39:16.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds620.ko 2024-04-17T09:39:16.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc1403.ko 2024-04-17T09:39:16.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc2103.ko 2024-04-17T09:39:16.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc6w201.ko 2024-04-17T09:39:16.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71805f.ko 2024-04-17T09:39:16.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 108961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71882fg.ko 2024-04-17T09:39:16.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f75375s.ko 2024-04-17T09:39:16.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fam15h_power.ko 2024-04-17T09:39:16.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 65769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fschmd.ko 2024-04-17T09:39:16.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g760a.ko 2024-04-17T09:39:16.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g762.ko 2024-04-17T09:39:16.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl518sm.ko 2024-04-17T09:39:16.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 56313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl520sm.ko 2024-04-17T09:39:16.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/hih6130.ko 2024-04-17T09:39:16.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-17T09:39:16.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5500_temp.ko 2024-04-17T09:39:16.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5k_amb.ko 2024-04-17T09:39:16.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmaem.ko 2024-04-17T09:39:16.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmpex.ko 2024-04-17T09:39:16.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina209.ko 2024-04-17T09:39:16.768 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina2xx.ko 2024-04-17T09:39:16.768 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 156465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/it87.ko 2024-04-17T09:39:16.768 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/jc42.ko 2024-04-17T09:39:16.768 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k10temp.ko 2024-04-17T09:39:16.768 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k8temp.ko 2024-04-17T09:39:16.768 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lineage-pem.ko 2024-04-17T09:39:16.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm63.ko 2024-04-17T09:39:16.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm73.ko 2024-04-17T09:39:16.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm75.ko 2024-04-17T09:39:16.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm77.ko 2024-04-17T09:39:16.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm78.ko 2024-04-17T09:39:16.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm80.ko 2024-04-17T09:39:16.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20545 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm83.ko 2024-04-17T09:39:16.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 86497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm85.ko 2024-04-17T09:39:16.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm87.ko 2024-04-17T09:39:16.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm90.ko 2024-04-17T09:39:16.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm92.ko 2024-04-17T09:39:16.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 131505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm93.ko 2024-04-17T09:39:16.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95234.ko 2024-04-17T09:39:16.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95241.ko 2024-04-17T09:39:16.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95245.ko 2024-04-17T09:39:16.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26393 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc2945.ko 2024-04-17T09:39:16.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4151.ko 2024-04-17T09:39:16.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4215.ko 2024-04-17T09:39:16.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4222.ko 2024-04-17T09:39:16.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4245.ko 2024-04-17T09:39:16.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4260.ko 2024-04-17T09:39:16.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4261.ko 2024-04-17T09:39:16.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max16065.ko 2024-04-17T09:39:16.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1619.ko 2024-04-17T09:39:16.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1668.ko 2024-04-17T09:39:16.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max197.ko 2024-04-17T09:39:16.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6639.ko 2024-04-17T09:39:16.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6650.ko 2024-04-17T09:39:16.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6697.ko 2024-04-17T09:39:16.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/mc13783-adc.ko 2024-04-17T09:39:16.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/mcp3021.ko 2024-04-17T09:39:16.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-04-17T09:39:16.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 56505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6683.ko 2024-04-17T09:39:16.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 141353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775-core.ko 2024-04-17T09:39:16.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775.ko 2024-04-17T09:39:16.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct7802.ko 2024-04-17T09:39:16.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 97777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87360.ko 2024-04-17T09:39:16.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87427.ko 2024-04-17T09:39:16.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pcf8591.ko 2024-04-17T09:39:16.772 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ 2024-04-17T09:39:16.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-17T09:39:16.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-17T09:39:16.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33265 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-17T09:39:16.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-17T09:39:16.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-17T09:39:16.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-17T09:39:16.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-17T09:39:16.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 101233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-17T09:39:16.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-17T09:39:16.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-17T09:39:16.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-17T09:39:16.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-17T09:39:16.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/powr1220.ko 2024-04-17T09:39:16.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5627.ko 2024-04-17T09:39:16.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5636.ko 2024-04-17T09:39:16.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-17T09:39:16.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40849 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht15.ko 2024-04-17T09:39:16.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht21.ko 2024-04-17T09:39:16.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/shtc1.ko 2024-04-17T09:39:16.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sis5595.ko 2024-04-17T09:39:16.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47b397.ko 2024-04-17T09:39:16.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m1.ko 2024-04-17T09:39:16.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m192.ko 2024-04-17T09:39:16.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/thmc50.ko 2024-04-17T09:39:16.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp102.ko 2024-04-17T09:39:16.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp103.ko 2024-04-17T09:39:16.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp401.ko 2024-04-17T09:39:16.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp421.ko 2024-04-17T09:39:16.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via-cputemp.ko 2024-04-17T09:39:16.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via686a.ko 2024-04-17T09:39:16.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt1211.ko 2024-04-17T09:39:16.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt8231.ko 2024-04-17T09:39:16.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 71169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627ehf.ko 2024-04-17T09:39:16.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 96577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627hf.ko 2024-04-17T09:39:16.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 85913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83781d.ko 2024-04-17T09:39:16.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 75129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83791d.ko 2024-04-17T09:39:16.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 83425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83792d.ko 2024-04-17T09:39:16.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 116777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83793.ko 2024-04-17T09:39:16.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 134441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83795.ko 2024-04-17T09:39:16.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l785ts.ko 2024-04-17T09:39:16.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l786ng.ko 2024-04-17T09:39:16.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-04-17T09:39:16.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-04-17T09:39:16.776 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/ 2024-04-17T09:39:16.776 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/ 2024-04-17T09:39:16.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-17T09:39:16.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-17T09:39:16.776 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/ 2024-04-17T09:39:16.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-04-17T09:39:16.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-04-17T09:39:16.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-04-17T09:39:16.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-17T09:39:16.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-17T09:39:16.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-17T09:39:16.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-04-17T09:39:16.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-17T09:39:16.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-04-17T09:39:16.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 64233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-17T09:39:16.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-17T09:39:16.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-17T09:39:16.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-17T09:39:16.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-04-17T09:39:16.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-04-17T09:39:16.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 76969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-17T09:39:16.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-17T09:39:16.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52937 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-17T09:39:16.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-04-17T09:39:16.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-17T09:39:16.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-17T09:39:16.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-04-17T09:39:16.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-17T09:39:16.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20057 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-17T09:39:16.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-17T09:39:16.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-04-17T09:39:16.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-17T09:39:16.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-17T09:39:16.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-04-17T09:39:16.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-04-17T09:39:16.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-17T09:39:16.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-04-17T09:39:16.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-17T09:39:16.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-17T09:39:16.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-17T09:39:16.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-17T09:39:16.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59193 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-04-17T09:39:16.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27849 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-mux.ko 2024-04-17T09:39:16.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-smbus.ko 2024-04-17T09:39:16.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-stub.ko 2024-04-17T09:39:16.780 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/muxes/ 2024-04-17T09:39:16.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-04-17T09:39:16.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-04-17T09:39:16.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-04-17T09:39:16.780 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ 2024-04-17T09:39:16.780 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ 2024-04-17T09:39:16.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 329697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-17T09:39:16.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1119913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_core.ko 2024-04-17T09:39:16.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 83905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-17T09:39:16.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 408433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-17T09:39:16.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 120921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-17T09:39:16.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 347217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-17T09:39:16.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 84945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-17T09:39:16.781 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/ 2024-04-17T09:39:16.781 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4/ 2024-04-17T09:39:16.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 590929 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-17T09:39:16.781 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4/ 2024-04-17T09:39:16.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 514641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-17T09:39:16.781 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5/ 2024-04-17T09:39:16.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 911665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-17T09:39:16.781 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca/ 2024-04-17T09:39:16.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 375545 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-17T09:39:16.781 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/ocrdma/ 2024-04-17T09:39:16.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 197121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-04-17T09:39:16.781 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/usnic/ 2024-04-17T09:39:16.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 212553 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-04-17T09:39:16.782 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ 2024-04-17T09:39:16.782 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib/ 2024-04-17T09:39:16.782 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 334401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-17T09:39:16.782 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser/ 2024-04-17T09:39:16.782 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 120673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-17T09:39:16.782 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert/ 2024-04-17T09:39:16.782 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 132585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-17T09:39:16.782 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp/ 2024-04-17T09:39:16.782 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 157937 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-17T09:39:16.782 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt/ 2024-04-17T09:39:16.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 156673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-17T09:39:16.783 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/ 2024-04-17T09:39:16.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/evbug.ko 2024-04-17T09:39:16.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/ff-memless.ko 2024-04-17T09:39:16.783 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/ 2024-04-17T09:39:16.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-17T09:39:16.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-17T09:39:16.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/gameport.ko 2024-04-17T09:39:16.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/joydev.ko 2024-04-17T09:39:16.783 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/ 2024-04-17T09:39:16.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-04-17T09:39:16.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-04-17T09:39:16.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-04-17T09:39:16.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-04-17T09:39:16.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-17T09:39:16.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-04-17T09:39:16.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/lkkbd.ko 2024-04-17T09:39:16.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/lm8323.ko 2024-04-17T09:39:16.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/lm8333.ko 2024-04-17T09:39:16.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-04-17T09:39:16.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-04-17T09:39:16.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-04-17T09:39:16.785 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-04-17T09:39:16.785 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/newtonkbd.ko 2024-04-17T09:39:16.785 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-04-17T09:39:16.785 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/qt1070.ko 2024-04-17T09:39:16.785 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/qt2160.ko 2024-04-17T09:39:16.785 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-04-17T09:39:16.785 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/stowaway.ko 2024-04-17T09:39:16.785 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/sunkbd.ko 2024-04-17T09:39:16.785 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-04-17T09:39:16.786 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-04-17T09:39:16.786 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-04-17T09:39:16.786 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/xtkbd.ko 2024-04-17T09:39:16.786 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/matrix-keymap.ko 2024-04-17T09:39:16.786 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/ 2024-04-17T09:39:16.786 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/appletouch.ko 2024-04-17T09:39:16.786 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39553 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/bcm5974.ko 2024-04-17T09:39:16.786 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 160017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/cyapatp.ko 2024-04-17T09:39:16.787 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 114273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-17T09:39:16.787 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/gpio_mouse.ko 2024-04-17T09:39:16.787 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 428185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/psmouse.ko 2024-04-17T09:39:16.787 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16937 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/sermouse.ko 2024-04-17T09:39:16.787 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-17T09:39:16.787 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27193 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-17T09:39:16.787 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18809 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-17T09:39:16.787 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4/ 2024-04-17T09:39:16.787 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 140969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-17T09:39:16.788 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/ 2024-04-17T09:39:16.788 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/altera_ps2.ko 2024-04-17T09:39:16.788 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/arc_ps2.ko 2024-04-17T09:39:16.788 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/ct82c710.ko 2024-04-17T09:39:16.788 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-17T09:39:16.788 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18393 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/parkbd.ko 2024-04-17T09:39:16.788 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/pcips2.ko 2024-04-17T09:39:16.788 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/ps2mult.ko 2024-04-17T09:39:16.788 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/serio_raw.ko 2024-04-17T09:39:16.788 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/serport.ko 2024-04-17T09:39:16.788 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/sparse-keymap.ko 2024-04-17T09:39:16.788 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/ 2024-04-17T09:39:16.789 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-88pm860x.ko 2024-04-17T09:39:16.789 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-adp5520.ko 2024-04-17T09:39:16.789 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 77265 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-bd2802.ko 2024-04-17T09:39:16.789 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-blinkm.ko 2024-04-17T09:39:16.789 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11897 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-da903x.ko 2024-04-17T09:39:16.789 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-da9052.ko 2024-04-17T09:39:16.789 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-gpio.ko 2024-04-17T09:39:16.789 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm3530.ko 2024-04-17T09:39:16.789 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm3533.ko 2024-04-17T09:39:16.789 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm355x.ko 2024-04-17T09:39:16.789 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm3642.ko 2024-04-17T09:39:16.789 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lp3944.ko 2024-04-17T09:39:16.790 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12809 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lp8788.ko 2024-04-17T09:39:16.790 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-max8997.ko 2024-04-17T09:39:16.790 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-mc13783.ko 2024-04-17T09:39:16.790 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-menf21bmc.ko 2024-04-17T09:39:16.790 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-pca9532.ko 2024-04-17T09:39:16.790 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23809 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-pca955x.ko 2024-04-17T09:39:16.790 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-pca963x.ko 2024-04-17T09:39:16.790 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-ss4200.ko 2024-04-17T09:39:16.790 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-tca6507.ko 2024-04-17T09:39:16.790 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-wm831x-status.ko 2024-04-17T09:39:16.790 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-wm8350.ko 2024-04-17T09:39:16.790 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ 2024-04-17T09:39:16.790 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-17T09:39:16.790 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-17T09:39:16.791 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-17T09:39:16.791 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-17T09:39:16.791 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-17T09:39:16.791 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16601 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-17T09:39:16.791 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-17T09:39:16.791 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-17T09:39:16.791 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-17T09:39:16.791 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/ 2024-04-17T09:39:16.791 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache/ 2024-04-17T09:39:16.791 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 790753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache/bcache.ko 2024-04-17T09:39:16.791 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-bio-prison.ko 2024-04-17T09:39:16.791 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 116825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-bufio.ko 2024-04-17T09:39:16.791 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache-smq.ko 2024-04-17T09:39:16.792 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 199041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache.ko 2024-04-17T09:39:16.792 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 134945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-crypt.ko 2024-04-17T09:39:16.792 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28897 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-delay.ko 2024-04-17T09:39:16.792 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-era.ko 2024-04-17T09:39:16.792 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-flakey.ko 2024-04-17T09:39:16.792 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-log-userspace.ko 2024-04-17T09:39:16.792 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-log.ko 2024-04-17T09:39:16.792 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-mirror.ko 2024-04-17T09:39:16.792 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 97209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-multipath.ko 2024-04-17T09:39:16.792 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19809 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-queue-length.ko 2024-04-17T09:39:16.792 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 85689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-raid.ko 2024-04-17T09:39:16.792 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-region-hash.ko 2024-04-17T09:39:16.792 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-round-robin.ko 2024-04-17T09:39:16.792 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-service-time.ko 2024-04-17T09:39:16.792 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 139817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-snapshot.ko 2024-04-17T09:39:16.793 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-switch.ko 2024-04-17T09:39:16.793 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 234457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-thin-pool.ko 2024-04-17T09:39:16.793 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-verity.ko 2024-04-17T09:39:16.793 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-zero.ko 2024-04-17T09:39:16.793 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data/ 2024-04-17T09:39:16.793 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 264329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-17T09:39:16.793 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid0.ko 2024-04-17T09:39:16.793 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 125009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid1.ko 2024-04-17T09:39:16.793 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 157681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid10.ko 2024-04-17T09:39:16.793 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 444665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid456.ko 2024-04-17T09:39:16.793 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/ 2024-04-17T09:39:16.793 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/ 2024-04-17T09:39:16.793 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 214841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptbase.ko 2024-04-17T09:39:16.793 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 77985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptctl.ko 2024-04-17T09:39:16.793 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54785 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptfc.ko 2024-04-17T09:39:16.793 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptlan.ko 2024-04-17T09:39:16.794 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 150761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptsas.ko 2024-04-17T09:39:16.794 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 97281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptscsih.ko 2024-04-17T09:39:16.794 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptspi.ko 2024-04-17T09:39:16.794 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/ 2024-04-17T09:39:16.794 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/88pm800.ko 2024-04-17T09:39:16.794 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/88pm805.ko 2024-04-17T09:39:16.794 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/88pm80x.ko 2024-04-17T09:39:16.794 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/arizona-i2c.ko 2024-04-17T09:39:16.794 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 148633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/arizona.ko 2024-04-17T09:39:16.794 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/bcm590xx.ko 2024-04-17T09:39:16.794 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/cros_ec_dev.ko 2024-04-17T09:39:16.794 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/dln2.ko 2024-04-17T09:39:16.795 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18601 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/janz-cmodio.ko 2024-04-17T09:39:16.795 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/kempld-core.ko 2024-04-17T09:39:16.795 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31897 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lm3533-core.ko 2024-04-17T09:39:16.795 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-04-17T09:39:16.795 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lp3943.ko 2024-04-17T09:39:16.795 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_ich.ko 2024-04-17T09:39:16.795 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_sch.ko 2024-04-17T09:39:16.795 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/max8907.ko 2024-04-17T09:39:16.795 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/mc13xxx-core.ko 2024-04-17T09:39:16.795 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12937 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-04-17T09:39:16.795 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/menf21bmc.ko 2024-04-17T09:39:16.795 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/pcf50633-adc.ko 2024-04-17T09:39:16.796 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/pcf50633-gpio.ko 2024-04-17T09:39:16.796 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/pcf50633.ko 2024-04-17T09:39:16.796 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-04-17T09:39:16.796 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/retu-mfd.ko 2024-04-17T09:39:16.796 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 99425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/si476x-core.ko 2024-04-17T09:39:16.796 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58057 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/sm501.ko 2024-04-17T09:39:16.796 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/tps65010.ko 2024-04-17T09:39:16.796 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11785 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/tps6507x.ko 2024-04-17T09:39:16.796 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/viperboard.ko 2024-04-17T09:39:16.796 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/vx855.ko 2024-04-17T09:39:16.796 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/wl1273-core.ko 2024-04-17T09:39:16.796 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/wm8994.ko 2024-04-17T09:39:16.797 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/ 2024-04-17T09:39:16.797 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-04-17T09:39:16.797 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 64265 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/ad525x_dpot.ko 2024-04-17T09:39:16.797 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl/ 2024-04-17T09:39:16.797 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-17T09:39:16.797 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds9802als.ko 2024-04-17T09:39:16.797 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds990x.ko 2024-04-17T09:39:16.797 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/bh1770glc.ko 2024-04-17T09:39:16.797 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/c2port/ 2024-04-17T09:39:16.797 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-04-17T09:39:16.797 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/c2port/core.ko 2024-04-17T09:39:16.797 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710/ 2024-04-17T09:39:16.797 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710/cb710.ko 2024-04-17T09:39:16.797 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/ds1682.ko 2024-04-17T09:39:16.797 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/dummy-irq.ko 2024-04-17T09:39:16.797 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/echo/ 2024-04-17T09:39:16.797 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/echo/echo.ko 2024-04-17T09:39:16.798 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/ 2024-04-17T09:39:16.798 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/at24.ko 2024-04-17T09:39:16.798 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-17T09:39:16.798 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/max6875.ko 2024-04-17T09:39:16.798 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/enclosure.ko 2024-04-17T09:39:16.798 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/genwqe/ 2024-04-17T09:39:16.798 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 191649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-04-17T09:39:16.798 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/hmc6352.ko 2024-04-17T09:39:16.798 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/hpilo.ko 2024-04-17T09:39:16.798 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/ibmasm/ 2024-04-17T09:39:16.798 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 92345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-04-17T09:39:16.798 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/ics932s401.ko 2024-04-17T09:39:16.798 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29003.ko 2024-04-17T09:39:16.798 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29020.ko 2024-04-17T09:39:16.799 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/ 2024-04-17T09:39:16.799 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-17T09:39:16.799 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-17T09:39:16.799 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/ 2024-04-17T09:39:16.799 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 154169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-me.ko 2024-04-17T09:39:16.799 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69809 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-txe.ko 2024-04-17T09:39:16.799 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 389033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei.ko 2024-04-17T09:39:16.799 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/phantom.ko 2024-04-17T09:39:16.799 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/ti-st/ 2024-04-17T09:39:16.799 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 91145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/ti-st/st_drv.ko 2024-04-17T09:39:16.799 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_7xx1.ko 2024-04-17T09:39:16.799 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_core.ko 2024-04-17T09:39:16.799 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tsl2550.ko 2024-04-17T09:39:16.799 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_balloon.ko 2024-04-17T09:39:16.799 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci/ 2024-04-17T09:39:16.800 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 234905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-17T09:39:16.800 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ 2024-04-17T09:39:16.800 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding/ 2024-04-17T09:39:16.800 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 620961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding/bonding.ko 2024-04-17T09:39:16.800 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/ 2024-04-17T09:39:16.800 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53/ 2024-04-17T09:39:16.800 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 139817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-17T09:39:16.800 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49545 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-17T09:39:16.800 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-17T09:39:16.800 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx/ 2024-04-17T09:39:16.801 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 587065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-17T09:39:16.801 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19553 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dummy.ko 2024-04-17T09:39:16.801 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/eql.ko 2024-04-17T09:39:16.801 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ 2024-04-17T09:39:16.801 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/ 2024-04-17T09:39:16.801 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-17T09:39:16.801 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-17T09:39:16.801 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 158649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-17T09:39:16.801 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 85953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-17T09:39:16.801 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/ 2024-04-17T09:39:16.801 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-17T09:39:16.801 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 73865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-17T09:39:16.802 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-17T09:39:16.802 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 109521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-17T09:39:16.802 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec/ 2024-04-17T09:39:16.802 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 64705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-17T09:39:16.802 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere/ 2024-04-17T09:39:16.802 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 84329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-17T09:39:16.802 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon/ 2024-04-17T09:39:16.802 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-17T09:39:16.802 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera/ 2024-04-17T09:39:16.802 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-17T09:39:16.802 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/ 2024-04-17T09:39:16.802 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 65857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-17T09:39:16.802 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 56057 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-17T09:39:16.803 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 115881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-17T09:39:16.803 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/ 2024-04-17T09:39:16.803 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx/ 2024-04-17T09:39:16.803 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 124497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-17T09:39:16.803 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-04-17T09:39:16.803 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 128409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-17T09:39:16.803 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-04-17T09:39:16.803 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 110345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-17T09:39:16.803 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/ 2024-04-17T09:39:16.803 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 101857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-17T09:39:16.803 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 88953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-17T09:39:16.803 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/ 2024-04-17T09:39:16.804 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 105633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-17T09:39:16.804 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 228985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-17T09:39:16.804 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-04-17T09:39:16.804 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1733353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-17T09:39:16.804 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 167185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-17T09:39:16.804 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet/ 2024-04-17T09:39:16.804 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 161417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-17T09:39:16.804 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 440625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-17T09:39:16.804 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/ 2024-04-17T09:39:16.804 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna/ 2024-04-17T09:39:16.804 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 432201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-17T09:39:16.804 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/ 2024-04-17T09:39:16.804 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-04-17T09:39:16.804 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 257201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-17T09:39:16.804 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-04-17T09:39:16.804 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 446177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-17T09:39:16.805 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-04-17T09:39:16.805 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1311433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-17T09:39:16.877 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-04-17T09:39:16.877 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 206689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-17T09:39:16.877 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-04-17T09:39:16.877 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-17T09:39:16.877 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/ 2024-04-17T09:39:16.878 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic/ 2024-04-17T09:39:16.878 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 269313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-17T09:39:16.878 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/ 2024-04-17T09:39:16.878 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/ 2024-04-17T09:39:16.878 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 85385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-17T09:39:16.878 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 77457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-17T09:39:16.878 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 169089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-17T09:39:16.878 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 65913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-17T09:39:16.878 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 76209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-17T09:39:16.878 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-17T09:39:16.878 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/ 2024-04-17T09:39:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 75945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-17T09:39:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-17T09:39:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dnet.ko 2024-04-17T09:39:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ec_bhf.ko 2024-04-17T09:39:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/ 2024-04-17T09:39:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet/ 2024-04-17T09:39:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 356377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-17T09:39:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ethoc.ko 2024-04-17T09:39:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/fealnx.ko 2024-04-17T09:39:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/fujitsu/ 2024-04-17T09:39:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-04-17T09:39:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ 2024-04-17T09:39:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 115569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-17T09:39:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000/ 2024-04-17T09:39:16.880 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 366217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-17T09:39:16.880 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e/ 2024-04-17T09:39:16.880 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 755793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-17T09:39:16.880 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k/ 2024-04-17T09:39:16.880 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 324577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-17T09:39:16.880 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e/ 2024-04-17T09:39:16.880 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1196297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-17T09:39:16.880 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf/ 2024-04-17T09:39:16.880 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 362593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-17T09:39:16.880 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb/ 2024-04-17T09:39:16.880 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 671801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-17T09:39:16.880 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf/ 2024-04-17T09:39:16.880 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 121537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-17T09:39:16.881 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-04-17T09:39:16.881 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 966705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-17T09:39:16.881 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-04-17T09:39:16.881 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 193585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-17T09:39:16.881 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 108633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/jme.ko 2024-04-17T09:39:16.881 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/ 2024-04-17T09:39:16.881 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-17T09:39:16.881 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 126881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-17T09:39:16.881 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 150777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-17T09:39:16.881 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/ 2024-04-17T09:39:16.881 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-04-17T09:39:16.881 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 905033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-17T09:39:16.882 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 344145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-17T09:39:16.882 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-04-17T09:39:16.882 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-04-17T09:39:16.882 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1412417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-17T09:39:16.882 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ 2024-04-17T09:39:16.882 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 65913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-04-17T09:39:16.882 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-04-17T09:39:16.882 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-04-17T09:39:16.882 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 117353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-17T09:39:16.882 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/ 2024-04-17T09:39:16.882 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-04-17T09:39:16.882 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 131425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-17T09:39:16.882 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/ 2024-04-17T09:39:16.883 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 91465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-17T09:39:16.883 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 56689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-17T09:39:16.883 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion/ 2024-04-17T09:39:16.883 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 181081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-17T09:39:16.883 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia/ 2024-04-17T09:39:16.883 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 171417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-17T09:39:16.883 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/ 2024-04-17T09:39:16.883 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-17T09:39:16.883 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-17T09:39:16.883 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/ 2024-04-17T09:39:16.883 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-04-17T09:39:16.883 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 300817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-17T09:39:16.884 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 105689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-17T09:39:16.884 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-04-17T09:39:16.884 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 885129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-17T09:39:16.884 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc/ 2024-04-17T09:39:16.884 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-17T09:39:16.884 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/ 2024-04-17T09:39:16.884 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 81641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-17T09:39:16.884 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 100689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-17T09:39:16.884 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-17T09:39:16.884 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 264161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-17T09:39:16.884 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/samsung/ 2024-04-17T09:39:16.884 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-04-17T09:39:16.884 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 142449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-04-17T09:39:16.884 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc/ 2024-04-17T09:39:16.885 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1350385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-17T09:39:16.885 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan/ 2024-04-17T09:39:16.885 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-17T09:39:16.885 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/ 2024-04-17T09:39:16.885 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-17T09:39:16.885 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 80065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-17T09:39:16.885 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/ 2024-04-17T09:39:16.885 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 74289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-17T09:39:16.885 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 79457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-17T09:39:16.885 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 100785 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-17T09:39:16.885 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 83441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-17T09:39:16.885 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/ 2024-04-17T09:39:16.885 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-04-17T09:39:16.885 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-04-17T09:39:16.886 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-17T09:39:16.886 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-04-17T09:39:16.886 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 644537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-17T09:39:16.886 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/ 2024-04-17T09:39:16.886 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 128497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-17T09:39:16.886 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 207993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-17T09:39:16.886 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 82633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-17T09:39:16.886 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 91353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-17T09:39:16.886 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti/ 2024-04-17T09:39:16.886 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-17T09:39:16.886 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti/ 2024-04-17T09:39:16.886 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 110729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-17T09:39:16.886 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/ 2024-04-17T09:39:16.887 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 98689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-17T09:39:16.887 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 102513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-17T09:39:16.887 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/ 2024-04-17T09:39:16.887 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-17T09:39:16.887 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-17T09:39:16.887 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom/ 2024-04-17T09:39:16.887 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-17T09:39:16.887 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/fddi/ 2024-04-17T09:39:16.887 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/fddi/defxx.ko 2024-04-17T09:39:16.887 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/fddi/skfp/ 2024-04-17T09:39:16.887 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 275801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/fddi/skfp/skfp.ko 2024-04-17T09:39:16.887 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 91105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/geneve.ko 2024-04-17T09:39:16.887 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv/ 2024-04-17T09:39:16.887 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 290441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-17T09:39:16.888 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154/ 2024-04-17T09:39:16.888 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-17T09:39:16.888 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ifb.ko 2024-04-17T09:39:16.888 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan/ 2024-04-17T09:39:16.888 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 108329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-17T09:39:16.888 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvlan.ko 2024-04-17T09:39:16.888 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvtap.ko 2024-04-17T09:39:16.888 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/ 2024-04-17T09:39:16.888 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-17T09:39:16.888 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-17T09:39:16.888 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-gpio.ko 2024-04-17T09:39:16.888 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-17T09:39:16.888 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio.ko 2024-04-17T09:39:16.889 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mii.ko 2024-04-17T09:39:16.889 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/netconsole.ko 2024-04-17T09:39:16.889 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/nlmon.ko 2024-04-17T09:39:16.889 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/ 2024-04-17T09:39:16.889 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-17T09:39:16.889 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-17T09:39:16.889 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/ 2024-04-17T09:39:16.889 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/amd.ko 2024-04-17T09:39:16.889 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-17T09:39:16.889 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-17T09:39:16.889 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm87xx.ko 2024-04-17T09:39:16.889 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/broadcom.ko 2024-04-17T09:39:16.889 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/cicada.ko 2024-04-17T09:39:16.890 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/davicom.ko 2024-04-17T09:39:16.890 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/et1011c.ko 2024-04-17T09:39:16.890 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/icplus.ko 2024-04-17T09:39:16.890 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/lxt.ko 2024-04-17T09:39:16.890 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 103377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/marvell.ko 2024-04-17T09:39:16.890 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 96025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/micrel.ko 2024-04-17T09:39:16.890 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/national.ko 2024-04-17T09:39:16.890 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 121105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/phylink.ko 2024-04-17T09:39:16.890 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/qsemi.ko 2024-04-17T09:39:16.890 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/realtek.ko 2024-04-17T09:39:16.890 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/smsc.ko 2024-04-17T09:39:16.890 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/ste10Xp.ko 2024-04-17T09:39:16.890 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/vitesse.ko 2024-04-17T09:39:16.891 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/plip/ 2024-04-17T09:39:16.891 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/plip/plip.ko 2024-04-17T09:39:16.891 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ 2024-04-17T09:39:16.891 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-17T09:39:16.891 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_async.ko 2024-04-17T09:39:16.891 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-17T09:39:16.891 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-17T09:39:16.891 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-17T09:39:16.891 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppoe.ko 2024-04-17T09:39:16.891 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppox.ko 2024-04-17T09:39:16.891 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pptp.ko 2024-04-17T09:39:16.891 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35937 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/sb1000.ko 2024-04-17T09:39:16.891 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/ 2024-04-17T09:39:16.892 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slip.ko 2024-04-17T09:39:16.892 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/sungem_phy.ko 2024-04-17T09:39:16.892 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 80161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tap.ko 2024-04-17T09:39:16.892 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/ 2024-04-17T09:39:16.892 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 137321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team.ko 2024-04-17T09:39:16.892 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-17T09:39:16.892 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-17T09:39:16.892 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-17T09:39:16.892 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_random.ko 2024-04-17T09:39:16.892 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-17T09:39:16.892 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 108089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/veth.ko 2024-04-17T09:39:16.892 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3/ 2024-04-17T09:39:16.892 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 176497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-17T09:39:16.893 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan/ 2024-04-17T09:39:16.893 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 383593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan/vxlan.ko 2024-04-17T09:39:16.893 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback/ 2024-04-17T09:39:16.893 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 175537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-17T09:39:16.893 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb/ 2024-04-17T09:39:16.893 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb/ntb.ko 2024-04-17T09:39:16.893 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/ 2024-04-17T09:39:16.893 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/ 2024-04-17T09:39:16.893 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 335857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme-core.ko 2024-04-17T09:39:16.893 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 126553 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme.ko 2024-04-17T09:39:16.893 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/ 2024-04-17T09:39:16.893 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 168145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport.ko 2024-04-17T09:39:16.893 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_cs.ko 2024-04-17T09:39:16.894 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 103889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_pc.ko 2024-04-17T09:39:16.894 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_serial.ko 2024-04-17T09:39:16.894 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/ 2024-04-17T09:39:16.894 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/ 2024-04-17T09:39:16.894 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-17T09:39:16.894 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-04-17T09:39:16.894 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-04-17T09:39:16.894 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pci-stub.ko 2024-04-17T09:39:16.894 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/xen-pcifront.ko 2024-04-17T09:39:16.894 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/ 2024-04-17T09:39:16.894 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/i82092.ko 2024-04-17T09:39:16.894 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 179961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pcmcia.ko 2024-04-17T09:39:16.895 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pcmcia_core.ko 2024-04-17T09:39:16.895 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-04-17T09:39:16.895 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pd6729.ko 2024-04-17T09:39:16.895 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 133753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-17T09:39:16.895 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/phy/ 2024-04-17T09:39:16.895 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/phy/broadcom/ 2024-04-17T09:39:16.895 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-04-17T09:39:16.895 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/phy/samsung/ 2024-04-17T09:39:16.895 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-04-17T09:39:16.895 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/ 2024-04-17T09:39:16.895 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel/ 2024-04-17T09:39:16.895 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78193 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-17T09:39:16.896 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/ 2024-04-17T09:39:16.896 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/chrome/ 2024-04-17T09:39:16.896 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-04-17T09:39:16.896 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29657 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/chrome/cros_ec.ko 2024-04-17T09:39:16.896 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-04-17T09:39:16.896 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-04-17T09:39:16.896 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-04-17T09:39:16.896 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-04-17T09:39:16.896 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-04-17T09:39:16.896 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/ 2024-04-17T09:39:16.897 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 87481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-17T09:39:16.897 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acerhdf.ko 2024-04-17T09:39:16.897 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-17T09:39:16.897 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 73865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-17T09:39:16.897 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-17T09:39:16.897 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 164769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-17T09:39:16.897 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-17T09:39:16.897 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-17T09:39:16.897 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-17T09:39:16.897 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-17T09:39:16.897 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-17T09:39:16.897 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/hdaps.ko 2024-04-17T09:39:16.897 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/ibm_rtl.ko 2024-04-17T09:39:16.897 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel/ 2024-04-17T09:39:16.897 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14193 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-17T09:39:16.898 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-04-17T09:39:16.898 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel_ips.ko 2024-04-17T09:39:16.898 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-17T09:39:16.898 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-17T09:39:16.898 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-17T09:39:16.898 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51897 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-17T09:39:16.898 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-17T09:39:16.898 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-17T09:39:16.898 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-17T09:39:16.898 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-17T09:39:16.898 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11657 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-17T09:39:16.898 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi.ko 2024-04-17T09:39:16.898 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/ 2024-04-17T09:39:16.899 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/ 2024-04-17T09:39:16.899 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/88pm860x_battery.ko 2024-04-17T09:39:16.899 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/88pm860x_charger.ko 2024-04-17T09:39:16.899 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 68833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/bq2415x_charger.ko 2024-04-17T09:39:16.899 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 73073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/bq24190_charger.ko 2024-04-17T09:39:16.899 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/bq24735-charger.ko 2024-04-17T09:39:16.899 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-04-17T09:39:16.899 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/da9030_battery.ko 2024-04-17T09:39:16.899 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/da9052-battery.ko 2024-04-17T09:39:16.899 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/ds2760_battery.ko 2024-04-17T09:39:16.899 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/ds2780_battery.ko 2024-04-17T09:39:16.899 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28809 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/ds2781_battery.ko 2024-04-17T09:39:16.899 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28657 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/ds2782_battery.ko 2024-04-17T09:39:16.900 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/gpio-charger.ko 2024-04-17T09:39:16.900 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/isp1704_charger.ko 2024-04-17T09:39:16.900 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/lp8727_charger.ko 2024-04-17T09:39:16.900 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/max14577_charger.ko 2024-04-17T09:39:16.900 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/max17042_battery.ko 2024-04-17T09:39:16.900 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/max8903_charger.ko 2024-04-17T09:39:16.900 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/max8925_power.ko 2024-04-17T09:39:16.900 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/pcf50633-charger.ko 2024-04-17T09:39:16.900 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/sbs-battery.ko 2024-04-17T09:39:16.900 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/test_power.ko 2024-04-17T09:39:16.900 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/tps65090-charger.ko 2024-04-17T09:39:16.901 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/wm831x_backup.ko 2024-04-17T09:39:16.901 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/wm831x_power.ko 2024-04-17T09:39:16.901 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/wm8350_power.ko 2024-04-17T09:39:16.901 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/ 2024-04-17T09:39:16.901 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 73297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-17T09:39:16.901 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-17T09:39:16.901 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/ 2024-04-17T09:39:16.901 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/ 2024-04-17T09:39:16.901 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-17T09:39:16.901 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-17T09:39:16.901 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps_parport.ko 2024-04-17T09:39:16.902 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/pps_core.ko 2024-04-17T09:39:16.902 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ 2024-04-17T09:39:16.902 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 91393 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp.ko 2024-04-17T09:39:16.902 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp_kvm.ko 2024-04-17T09:39:16.902 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/ 2024-04-17T09:39:16.902 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/ 2024-04-17T09:39:16.902 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl/ 2024-04-17T09:39:16.902 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-17T09:39:16.902 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/fmpm.ko 2024-04-17T09:39:16.902 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/ 2024-04-17T09:39:16.902 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-88pm80x.ko 2024-04-17T09:39:16.902 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-88pm860x.ko 2024-04-17T09:39:16.903 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-17T09:39:16.903 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-da9052.ko 2024-04-17T09:39:16.903 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-da9055.ko 2024-04-17T09:39:16.903 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-da9063.ko 2024-04-17T09:39:16.903 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-17T09:39:16.903 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 82225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-17T09:39:16.903 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-17T09:39:16.903 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-17T09:39:16.903 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-17T09:39:16.903 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16057 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-17T09:39:16.903 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-17T09:39:16.903 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-17T09:39:16.903 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-17T09:39:16.903 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-em3027.ko 2024-04-17T09:39:16.904 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-17T09:39:16.904 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-17T09:39:16.904 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-17T09:39:16.904 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-lp8788.ko 2024-04-17T09:39:16.904 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-17T09:39:16.904 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-17T09:39:16.904 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-17T09:39:16.904 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t86.ko 2024-04-17T09:39:16.904 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max6900.ko 2024-04-17T09:39:16.905 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max8907.ko 2024-04-17T09:39:16.905 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max8925.ko 2024-04-17T09:39:16.905 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max8997.ko 2024-04-17T09:39:16.905 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max8998.ko 2024-04-17T09:39:16.905 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-04-17T09:39:16.905 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-17T09:39:16.905 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-palmas.ko 2024-04-17T09:39:16.905 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 56777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-17T09:39:16.905 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21601 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf50633.ko 2024-04-17T09:39:16.905 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32657 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-17T09:39:16.905 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-17T09:39:16.906 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-17T09:39:16.906 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-17T09:39:16.906 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rc5t583.ko 2024-04-17T09:39:16.906 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-17T09:39:16.906 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-17T09:39:16.906 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-17T09:39:16.906 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-17T09:39:16.906 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-17T09:39:16.906 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-s35390a.ko 2024-04-17T09:39:16.906 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-17T09:39:16.906 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-tps6586x.ko 2024-04-17T09:39:16.906 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-tps65910.ko 2024-04-17T09:39:16.907 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-wm831x.ko 2024-04-17T09:39:16.907 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-wm8350.ko 2024-04-17T09:39:16.907 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-x1205.ko 2024-04-17T09:39:16.907 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ 2024-04-17T09:39:16.907 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 90377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-9xxx.ko 2024-04-17T09:39:16.907 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-sas.ko 2024-04-17T09:39:16.907 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 65945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-xxxx.ko 2024-04-17T09:39:16.907 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 164569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/BusLogic.ko 2024-04-17T09:39:16.907 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/a100u2w.ko 2024-04-17T09:39:16.907 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid/ 2024-04-17T09:39:16.907 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 340721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-17T09:39:16.907 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 120337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/advansys.ko 2024-04-17T09:39:16.908 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/ 2024-04-17T09:39:16.908 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 468081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-17T09:39:16.908 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 380673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-17T09:39:16.908 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic94xx/ 2024-04-17T09:39:16.908 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 243705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-04-17T09:39:16.908 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/am53c974.ko 2024-04-17T09:39:16.908 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr/ 2024-04-17T09:39:16.908 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 135617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-17T09:39:16.908 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/atp870u.ko 2024-04-17T09:39:16.908 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi/ 2024-04-17T09:39:16.908 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 304993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-17T09:39:16.908 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa/ 2024-04-17T09:39:16.908 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1527113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa/bfa.ko 2024-04-17T09:39:16.908 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc/ 2024-04-17T09:39:16.908 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 297801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-17T09:39:16.909 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i/ 2024-04-17T09:39:16.909 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 144401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-17T09:39:16.909 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ch.ko 2024-04-17T09:39:16.909 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor/ 2024-04-17T09:39:16.909 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 390713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-17T09:39:16.909 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/ 2024-04-17T09:39:16.909 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-04-17T09:39:16.909 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 88881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-17T09:39:16.909 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-04-17T09:39:16.909 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 117481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-17T09:39:16.909 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 173017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-17T09:39:16.909 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 91569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dc395x.ko 2024-04-17T09:39:16.909 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dmx3191d.ko 2024-04-17T09:39:16.910 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r/ 2024-04-17T09:39:16.910 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 267577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-17T09:39:16.910 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esp_scsi.ko 2024-04-17T09:39:16.910 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/ 2024-04-17T09:39:16.910 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 95329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-17T09:39:16.910 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 194017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-17T09:39:16.910 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fdomain.ko 2024-04-17T09:39:16.910 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic/ 2024-04-17T09:39:16.910 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 366529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic/fnic.ko 2024-04-17T09:39:16.910 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 251505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hpsa.ko 2024-04-17T09:39:16.910 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 65761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hptiop.ko 2024-04-17T09:39:16.910 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hv_storvsc.ko 2024-04-17T09:39:16.911 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/imm.ko 2024-04-17T09:39:16.911 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/initio.ko 2024-04-17T09:39:16.911 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 327841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ipr.ko 2024-04-17T09:39:16.911 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 129785 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ips.ko 2024-04-17T09:39:16.911 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci/ 2024-04-17T09:39:16.911 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 397785 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci/isci.ko 2024-04-17T09:39:16.911 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-17T09:39:16.911 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-17T09:39:16.911 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc/ 2024-04-17T09:39:16.911 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 386849 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc/libfc.ko 2024-04-17T09:39:16.911 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 178177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi.ko 2024-04-17T09:39:16.911 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 64809 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-17T09:39:16.911 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas/ 2024-04-17T09:39:16.911 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 274009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas/libsas.ko 2024-04-17T09:39:16.912 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc/ 2024-04-17T09:39:16.912 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 2736001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-17T09:39:16.912 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/ 2024-04-17T09:39:16.912 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 88529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-17T09:39:16.912 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-17T09:39:16.912 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 389177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-17T09:39:16.912 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 104769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid.ko 2024-04-17T09:39:16.912 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas/ 2024-04-17T09:39:16.912 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 836945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-17T09:39:16.912 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas/ 2024-04-17T09:39:16.912 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 158729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-17T09:39:16.912 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 96537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvumi.ko 2024-04-17T09:39:16.912 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pcmcia/ 2024-04-17T09:39:16.912 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 101625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-04-17T09:39:16.913 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-04-17T09:39:16.913 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-04-17T09:39:16.913 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28265 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-04-17T09:39:16.913 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001/ 2024-04-17T09:39:16.913 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 443745 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-17T09:39:16.913 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 152033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pmcraid.ko 2024-04-17T09:39:16.913 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ppa.ko 2024-04-17T09:39:16.913 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla1280.ko 2024-04-17T09:39:16.913 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/ 2024-04-17T09:39:16.913 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 2324809 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-17T09:39:16.913 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 103601 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-17T09:39:16.913 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx/ 2024-04-17T09:39:16.913 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 697337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-17T09:39:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qlogicfas408.ko 2024-04-17T09:39:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/raid_class.ko 2024-04-17T09:39:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 322865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_debug.ko 2024-04-17T09:39:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 219857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-17T09:39:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 357153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-17T09:39:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 121185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-17T09:39:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 93481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-17T09:39:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-17T09:39:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ses.ko 2024-04-17T09:39:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 138121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/st.ko 2024-04-17T09:39:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/stex.ko 2024-04-17T09:39:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2/ 2024-04-17T09:39:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 205025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-17T09:39:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/virtio_scsi.ko 2024-04-17T09:39:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-17T09:39:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/wd719x.ko 2024-04-17T09:39:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/xen-scsifront.ko 2024-04-17T09:39:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb/ 2024-04-17T09:39:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 174913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb/ssb.ko 2024-04-17T09:39:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/ 2024-04-17T09:39:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi/ 2024-04-17T09:39:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 920169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-17T09:39:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback/ 2024-04-17T09:39:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-17T09:39:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_file.ko 2024-04-17T09:39:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_iblock.ko 2024-04-17T09:39:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1170033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_mod.ko 2024-04-17T09:39:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51897 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_pscsi.ko 2024-04-17T09:39:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 149721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_user.ko 2024-04-17T09:39:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc/ 2024-04-17T09:39:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 90057 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-17T09:39:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/ 2024-04-17T09:39:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/ 2024-04-17T09:39:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/ 2024-04-17T09:39:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-17T09:39:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-17T09:39:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-17T09:39:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-17T09:39:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-17T09:39:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-17T09:39:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-17T09:39:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-17T09:39:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16745 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-17T09:39:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12929 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-17T09:39:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-17T09:39:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-17T09:39:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-17T09:39:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-17T09:39:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-17T09:39:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-04-17T09:39:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-17T09:39:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12849 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-17T09:39:16.918 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-17T09:39:16.918 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ 2024-04-17T09:39:16.918 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless/ 2024-04-17T09:39:16.918 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 99313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-17T09:39:16.918 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/moxa.ko 2024-04-17T09:39:16.918 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/mxser.ko 2024-04-17T09:39:16.918 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 134401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_gsm.ko 2024-04-17T09:39:16.918 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_hdlc.ko 2024-04-17T09:39:16.918 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/nozomi.ko 2024-04-17T09:39:16.918 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/ 2024-04-17T09:39:16.918 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250/ 2024-04-17T09:39:16.918 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44057 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-04-17T09:39:16.919 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 65121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-17T09:39:16.919 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-04-17T09:39:16.919 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/altera_uart.ko 2024-04-17T09:39:16.919 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/arc_uart.ko 2024-04-17T09:39:16.919 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 91665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-04-17T09:39:16.919 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm/ 2024-04-17T09:39:16.919 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 120577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-17T09:39:16.919 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/rp2.ko 2024-04-17T09:39:16.919 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 65641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/sc16is7xx.ko 2024-04-17T09:39:16.919 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/ 2024-04-17T09:39:16.919 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core/ 2024-04-17T09:39:16.920 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 693081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-17T09:39:16.920 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host/ 2024-04-17T09:39:16.920 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-17T09:39:16.920 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-04-17T09:39:16.920 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/ 2024-04-17T09:39:16.920 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio.ko 2024-04-17T09:39:16.920 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_aec.ko 2024-04-17T09:39:16.920 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_cif.ko 2024-04-17T09:39:16.920 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_dmem_genirq.ko 2024-04-17T09:39:16.920 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_mf624.ko 2024-04-17T09:39:16.920 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_netx.ko 2024-04-17T09:39:16.920 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_pci_generic.ko 2024-04-17T09:39:16.920 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-04-17T09:39:16.920 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_sercos3.ko 2024-04-17T09:39:16.921 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/ 2024-04-17T09:39:16.921 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/c67x00/ 2024-04-17T09:39:16.921 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 90321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/c67x00/c67x00.ko 2024-04-17T09:39:16.921 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/chipidea/ 2024-04-17T09:39:16.921 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 189209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-04-17T09:39:16.921 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-04-17T09:39:16.921 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-04-17T09:39:16.921 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-04-17T09:39:16.921 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-04-17T09:39:16.921 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/ 2024-04-17T09:39:16.922 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 101545 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-acm.ko 2024-04-17T09:39:16.922 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-17T09:39:16.922 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 56721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usblp.ko 2024-04-17T09:39:16.922 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 107417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usbtmc.ko 2024-04-17T09:39:16.922 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/common/ 2024-04-17T09:39:16.922 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/common/ulpi.ko 2024-04-17T09:39:16.922 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/dwc2/ 2024-04-17T09:39:16.922 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-04-17T09:39:16.922 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/dwc3/ 2024-04-17T09:39:16.922 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-04-17T09:39:16.922 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-04-17T09:39:16.922 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 574593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/dwc3/dwc3.ko 2024-04-17T09:39:16.923 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/ 2024-04-17T09:39:16.923 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/ 2024-04-17T09:39:16.923 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/u_ether.ko 2024-04-17T09:39:16.923 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/u_serial.ko 2024-04-17T09:39:16.923 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-04-17T09:39:16.923 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-04-17T09:39:16.923 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-04-17T09:39:16.923 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43553 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-04-17T09:39:16.923 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 131345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-04-17T09:39:16.923 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-04-17T09:39:16.923 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 146617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-04-17T09:39:16.923 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-04-17T09:39:16.923 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-04-17T09:39:16.924 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-04-17T09:39:16.924 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-04-17T09:39:16.924 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 101249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-04-17T09:39:16.924 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-04-17T09:39:16.924 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 71257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-04-17T09:39:16.924 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 84937 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-04-17T09:39:16.924 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/ 2024-04-17T09:39:16.924 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-04-17T09:39:16.924 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-04-17T09:39:16.924 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-04-17T09:39:16.924 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-04-17T09:39:16.924 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-04-17T09:39:16.924 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-04-17T09:39:16.924 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-04-17T09:39:16.925 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-04-17T09:39:16.925 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-04-17T09:39:16.925 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-04-17T09:39:16.925 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-04-17T09:39:16.925 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-04-17T09:39:16.925 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-04-17T09:39:16.925 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-04-17T09:39:16.925 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 204049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/libcomposite.ko 2024-04-17T09:39:16.925 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/ 2024-04-17T09:39:16.925 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-04-17T09:39:16.925 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/bdc/ 2024-04-17T09:39:16.925 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 182561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-04-17T09:39:16.925 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45545 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-04-17T09:39:16.925 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-04-17T09:39:16.926 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 85217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-04-17T09:39:16.926 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-04-17T09:39:16.926 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 90313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/net2272.ko 2024-04-17T09:39:16.926 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 90361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/net2280.ko 2024-04-17T09:39:16.926 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 94841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-04-17T09:39:16.926 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 80521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-04-17T09:39:16.926 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 87465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-04-17T09:39:16.926 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 95441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-04-17T09:39:16.927 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 195969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-04-17T09:39:16.927 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/ 2024-04-17T09:39:16.927 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/bcma-hcd.ko 2024-04-17T09:39:16.927 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 108321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/isp116x-hcd.ko 2024-04-17T09:39:16.927 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 104377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-04-17T09:39:16.927 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 102273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-04-17T09:39:16.927 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-17T09:39:16.927 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/sl811_cs.ko 2024-04-17T09:39:16.927 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/ssb-hcd.ko 2024-04-17T09:39:16.927 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32553 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-04-17T09:39:16.927 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/ 2024-04-17T09:39:16.928 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/mdc800.ko 2024-04-17T09:39:16.928 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/microtek.ko 2024-04-17T09:39:16.928 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ 2024-04-17T09:39:16.928 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/adutux.ko 2024-04-17T09:39:16.928 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/appledisplay.ko 2024-04-17T09:39:16.928 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-04-17T09:39:16.928 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/cytherm.ko 2024-04-17T09:39:16.928 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ehset.ko 2024-04-17T09:39:16.928 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi26.ko 2024-04-17T09:39:16.928 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi62.ko 2024-04-17T09:39:16.928 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13057 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ezusb.ko 2024-04-17T09:39:16.928 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/idmouse.ko 2024-04-17T09:39:16.928 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/iowarrior.ko 2024-04-17T09:39:16.929 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-17T09:39:16.929 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ldusb.ko 2024-04-17T09:39:16.929 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/legousbtower.ko 2024-04-17T09:39:16.929 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/lvstest.ko 2024-04-17T09:39:16.929 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga/ 2024-04-17T09:39:16.929 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 85313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-17T09:39:16.929 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14601 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-17T09:39:16.929 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usb3503.ko 2024-04-17T09:39:16.929 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usblcd.ko 2024-04-17T09:39:16.929 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27809 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-17T09:39:16.929 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 84265 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usbtest.ko 2024-04-17T09:39:16.929 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/uss720.ko 2024-04-17T09:39:16.929 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/yurex.ko 2024-04-17T09:39:16.930 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/mon/ 2024-04-17T09:39:16.930 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 91529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/mon/usbmon.ko 2024-04-17T09:39:16.930 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/musb/ 2024-04-17T09:39:16.930 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 396209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/musb/musb_hdrc.ko 2024-04-17T09:39:16.930 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/ 2024-04-17T09:39:16.930 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-generic.ko 2024-04-17T09:39:16.930 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-04-17T09:39:16.930 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-isp1301.ko 2024-04-17T09:39:16.930 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-tahvo.ko 2024-04-17T09:39:16.930 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ 2024-04-17T09:39:16.931 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/aircable.ko 2024-04-17T09:39:16.931 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ark3116.ko 2024-04-17T09:39:16.931 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-17T09:39:16.931 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ch341.ko 2024-04-17T09:39:16.931 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 79681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cp210x.ko 2024-04-17T09:39:16.931 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cyberjack.ko 2024-04-17T09:39:16.931 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-17T09:39:16.931 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-17T09:39:16.931 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/empeg.ko 2024-04-17T09:39:16.931 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/f81232.ko 2024-04-17T09:39:16.931 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 188609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-17T09:39:16.931 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-17T09:39:16.931 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 152305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-17T09:39:16.932 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 142833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_ti.ko 2024-04-17T09:39:16.932 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipaq.ko 2024-04-17T09:39:16.932 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipw.ko 2024-04-17T09:39:16.932 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ir-usb.ko 2024-04-17T09:39:16.932 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-17T09:39:16.932 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 122329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan.ko 2024-04-17T09:39:16.932 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-17T09:39:16.932 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-17T09:39:16.932 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-17T09:39:16.932 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mct_u232.ko 2024-04-17T09:39:16.932 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/metro-usb.ko 2024-04-17T09:39:16.932 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 79681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7720.ko 2024-04-17T09:39:16.933 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7840.ko 2024-04-17T09:39:16.933 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 65073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mxuport.ko 2024-04-17T09:39:16.933 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/navman.ko 2024-04-17T09:39:16.933 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/omninet.ko 2024-04-17T09:39:16.933 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/opticon.ko 2024-04-17T09:39:16.933 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 193857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/option.ko 2024-04-17T09:39:16.933 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/oti6858.ko 2024-04-17T09:39:16.933 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/pl2303.ko 2024-04-17T09:39:16.933 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcaux.ko 2024-04-17T09:39:16.933 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcserial.ko 2024-04-17T09:39:16.933 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/quatech2.ko 2024-04-17T09:39:16.933 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/safe_serial.ko 2024-04-17T09:39:16.934 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/sierra.ko 2024-04-17T09:39:16.934 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-17T09:39:16.934 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ssu100.ko 2024-04-17T09:39:16.934 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19601 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/symbolserial.ko 2024-04-17T09:39:16.934 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-17T09:39:16.934 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-17T09:39:16.934 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_debug.ko 2024-04-17T09:39:16.934 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-17T09:39:16.934 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 151137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usbserial.ko 2024-04-17T09:39:16.934 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/visor.ko 2024-04-17T09:39:16.934 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/whiteheat.ko 2024-04-17T09:39:16.934 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/wishbone-serial.ko 2024-04-17T09:39:16.935 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-17T09:39:16.935 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ 2024-04-17T09:39:16.935 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 64049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/uas.ko 2024-04-17T09:39:16.935 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-17T09:39:16.935 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-17T09:39:16.935 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-17T09:39:16.935 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-17T09:39:16.935 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-17T09:39:16.935 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-17T09:39:16.935 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-17T09:39:16.935 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-karma.ko 2024-04-17T09:39:16.936 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-17T09:39:16.936 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-17T09:39:16.936 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33849 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-17T09:39:16.936 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-17T09:39:16.936 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41785 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-17T09:39:16.936 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 201193 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/usb-storage.ko 2024-04-17T09:39:16.936 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/ 2024-04-17T09:39:16.936 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 76041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-17T09:39:16.936 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 85257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-17T09:39:16.936 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 122321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-17T09:39:16.936 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/ 2024-04-17T09:39:16.936 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/ 2024-04-17T09:39:16.937 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 206921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-17T09:39:16.937 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-17T09:39:16.937 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 134265 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio.ko 2024-04-17T09:39:16.937 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 93673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-17T09:39:16.937 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/ 2024-04-17T09:39:16.937 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 138721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost.ko 2024-04-17T09:39:16.937 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-17T09:39:16.937 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 71281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_net.ko 2024-04-17T09:39:16.937 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 107905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_scsi.ko 2024-04-17T09:39:16.937 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/ 2024-04-17T09:39:16.937 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/ 2024-04-17T09:39:16.938 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-04-17T09:39:16.938 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/aat2870_bl.ko 2024-04-17T09:39:16.938 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/adp5520_bl.ko 2024-04-17T09:39:16.938 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/adp8860_bl.ko 2024-04-17T09:39:16.938 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51745 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/adp8870_bl.ko 2024-04-17T09:39:16.938 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/apple_bl.ko 2024-04-17T09:39:16.938 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/as3711_bl.ko 2024-04-17T09:39:16.938 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/bd6107.ko 2024-04-17T09:39:16.938 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/da903x_bl.ko 2024-04-17T09:39:16.938 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/da9052_bl.ko 2024-04-17T09:39:16.938 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13785 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/gpio_backlight.ko 2024-04-17T09:39:16.938 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/kb3886_bl.ko 2024-04-17T09:39:16.939 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lcd.ko 2024-04-17T09:39:16.939 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lm3533_bl.ko 2024-04-17T09:39:16.939 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23393 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lm3639_bl.ko 2024-04-17T09:39:16.939 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lv5207lp.ko 2024-04-17T09:39:16.939 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/max8925_bl.ko 2024-04-17T09:39:16.939 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/pandora_bl.ko 2024-04-17T09:39:16.939 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13057 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-04-17T09:39:16.939 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-17T09:39:16.939 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/wm831x_bl.ko 2024-04-17T09:39:16.939 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/ 2024-04-17T09:39:16.939 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ 2024-04-17T09:39:16.939 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2482.ko 2024-04-17T09:39:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2490.ko 2024-04-17T09:39:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/matrox_w1.ko 2024-04-17T09:39:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/w1-gpio.ko 2024-04-17T09:39:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/ 2024-04-17T09:39:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-17T09:39:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-17T09:39:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-17T09:39:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-17T09:39:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-17T09:39:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-17T09:39:16.940 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-17T09:39:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-17T09:39:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-17T09:39:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-17T09:39:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-17T09:39:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 129433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/wire.ko 2024-04-17T09:39:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ 2024-04-17T09:39:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/acquirewdt.ko 2024-04-17T09:39:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/advantechwdt.ko 2024-04-17T09:39:16.941 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-17T09:39:16.942 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-17T09:39:16.942 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/cpu5wdt.ko 2024-04-17T09:39:16.942 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/da9052_wdt.ko 2024-04-17T09:39:16.942 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/da9055_wdt.ko 2024-04-17T09:39:16.942 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/da9063_wdt.ko 2024-04-17T09:39:16.942 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/dw_wdt.ko 2024-04-17T09:39:16.942 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/eurotechwdt.ko 2024-04-17T09:39:16.942 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-17T09:39:16.942 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/hpwdt.ko 2024-04-17T09:39:16.942 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/i6300esb.ko 2024-04-17T09:39:16.942 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-17T09:39:16.943 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-17T09:39:16.943 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ib700wdt.ko 2024-04-17T09:39:16.943 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ibmasr.ko 2024-04-17T09:39:16.943 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-17T09:39:16.943 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-17T09:39:16.943 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it87_wdt.ko 2024-04-17T09:39:16.943 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/kempld_wdt.ko 2024-04-17T09:39:16.943 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/machzwd.ko 2024-04-17T09:39:16.943 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/mena21_wdt.ko 2024-04-17T09:39:16.943 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-04-17T09:39:16.943 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/nv_tco.ko 2024-04-17T09:39:16.943 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-04-17T09:39:16.944 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pc87413_wdt.ko 2024-04-17T09:39:16.944 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-17T09:39:16.944 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37193 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-17T09:39:16.944 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/retu_wdt.ko 2024-04-17T09:39:16.944 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-04-17T09:39:16.944 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-04-17T09:39:16.944 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-17T09:39:16.944 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sc1200wdt.ko 2024-04-17T09:39:16.944 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-17T09:39:16.944 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-04-17T09:39:16.944 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/softdog.ko 2024-04-17T09:39:16.944 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-17T09:39:16.945 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/twl4030_wdt.ko 2024-04-17T09:39:16.945 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/via_wdt.ko 2024-04-17T09:39:16.945 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23393 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-17T09:39:16.945 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-17T09:39:16.945 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-17T09:39:16.945 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wafer5823wdt.ko 2024-04-17T09:39:16.945 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wdt_pci.ko 2024-04-17T09:39:16.945 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wm831x_wdt.ko 2024-04-17T09:39:16.945 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wm8350_wdt.ko 2024-04-17T09:39:16.945 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/xen_wdt.ko 2024-04-17T09:39:16.945 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/ 2024-04-17T09:39:16.946 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-evtchn.ko 2024-04-17T09:39:16.946 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntalloc.ko 2024-04-17T09:39:16.946 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntdev.ko 2024-04-17T09:39:16.946 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback/ 2024-04-17T09:39:16.946 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 213913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-17T09:39:16.946 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-privcmd.ko 2024-04-17T09:39:16.946 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 85713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-scsiback.ko 2024-04-17T09:39:16.946 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs/ 2024-04-17T09:39:16.946 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-17T09:39:16.946 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ 2024-04-17T09:39:16.946 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p/ 2024-04-17T09:39:16.947 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 192617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p/9p.ko 2024-04-17T09:39:16.947 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/adfs/ 2024-04-17T09:39:16.947 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/adfs/adfs.ko 2024-04-17T09:39:16.947 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs/ 2024-04-17T09:39:16.947 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 197849 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs/affs.ko 2024-04-17T09:39:16.947 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/afs/ 2024-04-17T09:39:16.947 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1261113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/afs/kafs.ko 2024-04-17T09:39:16.947 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/autofs/ 2024-04-17T09:39:16.947 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 127377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/autofs/autofs4.ko 2024-04-17T09:39:16.947 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs/ 2024-04-17T09:39:16.947 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 80521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs/befs.ko 2024-04-17T09:39:16.947 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/bfs/ 2024-04-17T09:39:16.947 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/bfs/bfs.ko 2024-04-17T09:39:16.948 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/binfmt_misc.ko 2024-04-17T09:39:16.948 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs/ 2024-04-17T09:39:16.948 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 4489593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs/btrfs.ko 2024-04-17T09:39:16.998 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles/ 2024-04-17T09:39:16.998 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 376857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles/cachefiles.ko 2024-04-17T09:39:16.998 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph/ 2024-04-17T09:39:16.998 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1586089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph/ceph.ko 2024-04-17T09:39:16.998 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda/ 2024-04-17T09:39:16.998 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 130073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda/coda.ko 2024-04-17T09:39:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/configfs/ 2024-04-17T09:39:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 139081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/configfs/configfs.ko 2024-04-17T09:39:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs/ 2024-04-17T09:39:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs/cramfs.ko 2024-04-17T09:39:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm/ 2024-04-17T09:39:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 813225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm/dlm.ko 2024-04-17T09:39:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/efs/ 2024-04-17T09:39:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/efs/efs.ko 2024-04-17T09:39:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs/ 2024-04-17T09:39:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1906817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs/f2fs.ko 2024-04-17T09:39:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/ 2024-04-17T09:39:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/msdos.ko 2024-04-17T09:39:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/freevxfs/ 2024-04-17T09:39:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/freevxfs/freevxfs.ko 2024-04-17T09:39:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/ 2024-04-17T09:39:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/cuse.ko 2024-04-17T09:39:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2/ 2024-04-17T09:39:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1486521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2/gfs2.ko 2024-04-17T09:39:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs/ 2024-04-17T09:39:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 183305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs/hfs.ko 2024-04-17T09:39:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus/ 2024-04-17T09:39:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 282097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus/hfsplus.ko 2024-04-17T09:39:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hpfs/ 2024-04-17T09:39:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 249009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hpfs/hpfs.ko 2024-04-17T09:39:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs/ 2024-04-17T09:39:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 93033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs/isofs.ko 2024-04-17T09:39:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs/ 2024-04-17T09:39:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 533073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs/jfs.ko 2024-04-17T09:39:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd/ 2024-04-17T09:39:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 361369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd/lockd.ko 2024-04-17T09:39:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix/ 2024-04-17T09:39:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 112609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix/minix.ko 2024-04-17T09:39:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs/ 2024-04-17T09:39:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 591585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs/netfs.ko 2024-04-17T09:39:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/ 2024-04-17T09:39:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout/ 2024-04-17T09:39:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 102561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-17T09:39:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout/ 2024-04-17T09:39:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 74353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-17T09:39:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout/ 2024-04-17T09:39:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 147689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-17T09:39:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1320249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfs.ko 2024-04-17T09:39:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 82105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv2.ko 2024-04-17T09:39:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 151641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv3.ko 2024-04-17T09:39:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 2518505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv4.ko 2024-04-17T09:39:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/ 2024-04-17T09:39:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19849 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/grace.ko 2024-04-17T09:39:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/nfs_acl.ko 2024-04-17T09:39:17.003 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd/ 2024-04-17T09:39:17.003 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 2024049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd/nfsd.ko 2024-04-17T09:39:17.003 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2/ 2024-04-17T09:39:17.003 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 734025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2/nilfs2.ko 2024-04-17T09:39:17.003 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/ 2024-04-17T09:39:17.003 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-celtic.ko 2024-04-17T09:39:17.003 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-centeuro.ko 2024-04-17T09:39:17.003 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-croatian.ko 2024-04-17T09:39:17.003 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-cyrillic.ko 2024-04-17T09:39:17.003 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-gaelic.ko 2024-04-17T09:39:17.003 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-greek.ko 2024-04-17T09:39:17.003 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-iceland.ko 2024-04-17T09:39:17.003 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-inuit.ko 2024-04-17T09:39:17.004 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-roman.ko 2024-04-17T09:39:17.004 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-romanian.ko 2024-04-17T09:39:17.004 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-turkish.ko 2024-04-17T09:39:17.004 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_ascii.ko 2024-04-17T09:39:17.004 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1250.ko 2024-04-17T09:39:17.004 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1251.ko 2024-04-17T09:39:17.004 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1255.ko 2024-04-17T09:39:17.004 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp737.ko 2024-04-17T09:39:17.004 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp775.ko 2024-04-17T09:39:17.004 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp850.ko 2024-04-17T09:39:17.004 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp852.ko 2024-04-17T09:39:17.004 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp855.ko 2024-04-17T09:39:17.005 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp857.ko 2024-04-17T09:39:17.005 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp860.ko 2024-04-17T09:39:17.005 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp861.ko 2024-04-17T09:39:17.005 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14657 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp862.ko 2024-04-17T09:39:17.005 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp863.ko 2024-04-17T09:39:17.005 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp864.ko 2024-04-17T09:39:17.005 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp865.ko 2024-04-17T09:39:17.005 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp866.ko 2024-04-17T09:39:17.005 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp869.ko 2024-04-17T09:39:17.005 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp874.ko 2024-04-17T09:39:17.005 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 95625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp932.ko 2024-04-17T09:39:17.005 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 144209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp936.ko 2024-04-17T09:39:17.005 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 167137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp949.ko 2024-04-17T09:39:17.006 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 119513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp950.ko 2024-04-17T09:39:17.006 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_euc-jp.ko 2024-04-17T09:39:17.006 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-1.ko 2024-04-17T09:39:17.006 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-13.ko 2024-04-17T09:39:17.006 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-14.ko 2024-04-17T09:39:17.006 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-15.ko 2024-04-17T09:39:17.006 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-2.ko 2024-04-17T09:39:17.006 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-3.ko 2024-04-17T09:39:17.006 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-4.ko 2024-04-17T09:39:17.006 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-5.ko 2024-04-17T09:39:17.006 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-6.ko 2024-04-17T09:39:17.007 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-7.ko 2024-04-17T09:39:17.007 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-9.ko 2024-04-17T09:39:17.007 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-r.ko 2024-04-17T09:39:17.007 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-ru.ko 2024-04-17T09:39:17.007 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-u.ko 2024-04-17T09:39:17.007 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-17T09:39:17.007 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9601 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_utf8.ko 2024-04-17T09:39:17.007 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ 2024-04-17T09:39:17.007 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster/ 2024-04-17T09:39:17.007 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 302073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-17T09:39:17.007 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm/ 2024-04-17T09:39:17.007 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 548169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-17T09:39:17.008 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs/ 2024-04-17T09:39:17.008 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-17T09:39:17.008 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 3245617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2.ko 2024-04-17T09:39:17.008 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-17T09:39:17.008 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-17T09:39:17.008 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-17T09:39:17.008 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/omfs/ 2024-04-17T09:39:17.008 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/omfs/omfs.ko 2024-04-17T09:39:17.008 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs/ 2024-04-17T09:39:17.008 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 462873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs/overlay.ko 2024-04-17T09:39:17.008 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore/ 2024-04-17T09:39:17.008 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore/ramoops.ko 2024-04-17T09:39:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/qnx4/ 2024-04-17T09:39:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/qnx4/qnx4.ko 2024-04-17T09:39:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/qnx6/ 2024-04-17T09:39:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 56097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/qnx6/qnx6.ko 2024-04-17T09:39:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/quota/ 2024-04-17T09:39:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/quota/quota_tree.ko 2024-04-17T09:39:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/quota/quota_v1.ko 2024-04-17T09:39:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/quota/quota_v2.ko 2024-04-17T09:39:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs/ 2024-04-17T09:39:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 684841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs/reiserfs.ko 2024-04-17T09:39:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs/ 2024-04-17T09:39:17.010 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs/romfs.ko 2024-04-17T09:39:17.010 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/ 2024-04-17T09:39:17.010 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client/ 2024-04-17T09:39:17.010 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 3319041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client/cifs.ko 2024-04-17T09:39:17.010 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/ 2024-04-17T09:39:17.010 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_arc4.ko 2024-04-17T09:39:17.010 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_md4.ko 2024-04-17T09:39:17.010 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs/ 2024-04-17T09:39:17.010 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 164185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs/squashfs.ko 2024-04-17T09:39:17.010 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv/ 2024-04-17T09:39:17.010 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 115977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv/sysv.ko 2024-04-17T09:39:17.011 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf/ 2024-04-17T09:39:17.011 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 310017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf/udf.ko 2024-04-17T09:39:17.011 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs/ 2024-04-17T09:39:17.011 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 198233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs/ufs.ko 2024-04-17T09:39:17.011 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs/ 2024-04-17T09:39:17.011 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 5192705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs/xfs.ko 2024-04-17T09:39:17.011 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/ 2024-04-17T09:39:17.011 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/locking/ 2024-04-17T09:39:17.011 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 97625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/locking/locktorture.ko 2024-04-17T09:39:17.011 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/time/ 2024-04-17T09:39:17.011 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/time/test_udelay.ko 2024-04-17T09:39:17.011 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/torture.ko 2024-04-17T09:39:17.012 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ 2024-04-17T09:39:17.012 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc-itu-t.ko 2024-04-17T09:39:17.012 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc7.ko 2024-04-17T09:39:17.012 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc8.ko 2024-04-17T09:39:17.012 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/ 2024-04-17T09:39:17.012 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libarc4.ko 2024-04-17T09:39:17.012 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libdes.ko 2024-04-17T09:39:17.012 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/interval_tree_test.ko 2024-04-17T09:39:17.012 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/libcrc32c.ko 2024-04-17T09:39:17.012 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lru_cache.ko 2024-04-17T09:39:17.012 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/ 2024-04-17T09:39:17.012 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4_compress.ko 2024-04-17T09:39:17.013 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4hc_compress.ko 2024-04-17T09:39:17.013 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math/ 2024-04-17T09:39:17.013 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math/cordic.ko 2024-04-17T09:39:17.013 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/memory-notifier-error-inject.ko 2024-04-17T09:39:17.013 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/notifier-error-inject.ko 2024-04-17T09:39:17.013 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/percpu_test.ko 2024-04-17T09:39:17.013 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/pm-notifier-error-inject.ko 2024-04-17T09:39:17.013 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6/ 2024-04-17T09:39:17.013 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 152025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6/raid6_pq.ko 2024-04-17T09:39:17.013 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/rbtree_test.ko 2024-04-17T09:39:17.013 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon/ 2024-04-17T09:39:17.014 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-17T09:39:17.014 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/test-kstrtox.ko 2024-04-17T09:39:17.014 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 4912441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/test_bpf.ko 2024-04-17T09:39:17.014 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/test_firmware.ko 2024-04-17T09:39:17.014 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/test_module.ko 2024-04-17T09:39:17.014 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24937 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/test_user_copy.ko 2024-04-17T09:39:17.014 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_bm.ko 2024-04-17T09:39:17.014 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_fsm.ko 2024-04-17T09:39:17.014 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_kmp.ko 2024-04-17T09:39:17.014 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/xz/ 2024-04-17T09:39:17.014 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/xz/xz_dec_test.ko 2024-04-17T09:39:17.014 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm/ 2024-04-17T09:39:17.015 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm/hwpoison-inject.ko 2024-04-17T09:39:17.015 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ 2024-04-17T09:39:17.015 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/ 2024-04-17T09:39:17.015 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/6lowpan.ko 2024-04-17T09:39:17.015 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_dest.ko 2024-04-17T09:39:17.015 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_fragment.ko 2024-04-17T09:39:17.015 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_hop.ko 2024-04-17T09:39:17.015 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-17T09:39:17.015 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_mobility.ko 2024-04-17T09:39:17.015 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_routing.ko 2024-04-17T09:39:17.015 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_udp.ko 2024-04-17T09:39:17.016 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/ 2024-04-17T09:39:17.016 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/garp.ko 2024-04-17T09:39:17.016 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/mrp.ko 2024-04-17T09:39:17.016 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9849 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/p8022.ko 2024-04-17T09:39:17.016 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/psnap.ko 2024-04-17T09:39:17.016 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/stp.ko 2024-04-17T09:39:17.016 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q/ 2024-04-17T09:39:17.016 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 106793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q/8021q.ko 2024-04-17T09:39:17.016 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/ 2024-04-17T09:39:17.016 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 236793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet.ko 2024-04-17T09:39:17.016 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_fd.ko 2024-04-17T09:39:17.016 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_rdma.ko 2024-04-17T09:39:17.016 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_virtio.ko 2024-04-17T09:39:17.017 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv/ 2024-04-17T09:39:17.017 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 879697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv/batman-adv.ko 2024-04-17T09:39:17.017 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/ 2024-04-17T09:39:17.017 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 84761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/br_netfilter.ko 2024-04-17T09:39:17.017 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1030873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/bridge.ko 2024-04-17T09:39:17.017 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ 2024-04-17T09:39:17.017 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-17T09:39:17.017 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-17T09:39:17.017 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-17T09:39:17.017 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-17T09:39:17.017 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-17T09:39:17.018 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-17T09:39:17.018 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-17T09:39:17.018 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-17T09:39:17.018 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-17T09:39:17.018 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-17T09:39:17.018 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-17T09:39:17.018 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-17T09:39:17.018 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-17T09:39:17.018 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-17T09:39:17.018 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-17T09:39:17.018 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-17T09:39:17.018 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-17T09:39:17.019 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-17T09:39:17.019 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-17T09:39:17.019 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-17T09:39:17.019 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtables.ko 2024-04-17T09:39:17.019 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-17T09:39:17.019 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-17T09:39:17.019 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph/ 2024-04-17T09:39:17.019 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1243505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph/libceph.ko 2024-04-17T09:39:17.019 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/ 2024-04-17T09:39:17.019 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 153801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/pktgen.ko 2024-04-17T09:39:17.019 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/ 2024-04-17T09:39:17.019 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 244889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp.ko 2024-04-17T09:39:17.020 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_diag.ko 2024-04-17T09:39:17.020 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 71033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv4.ko 2024-04-17T09:39:17.020 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 74993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv6.ko 2024-04-17T09:39:17.020 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/ 2024-04-17T09:39:17.020 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 507705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/dsa_core.ko 2024-04-17T09:39:17.020 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_brcm.ko 2024-04-17T09:39:17.020 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_dsa.ko 2024-04-17T09:39:17.020 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_none.ko 2024-04-17T09:39:17.020 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_trailer.ko 2024-04-17T09:39:17.020 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/hsr/ 2024-04-17T09:39:17.020 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 128873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/hsr/hsr.ko 2024-04-17T09:39:17.020 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ 2024-04-17T09:39:17.021 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan/ 2024-04-17T09:39:17.021 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-17T09:39:17.021 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 357425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154.ko 2024-04-17T09:39:17.021 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59393 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-17T09:39:17.021 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ 2024-04-17T09:39:17.021 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ah4.ko 2024-04-17T09:39:17.021 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/esp4.ko 2024-04-17T09:39:17.021 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/fou.ko 2024-04-17T09:39:17.021 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21937 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/gre.ko 2024-04-17T09:39:17.021 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 65665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/inet_diag.ko 2024-04-17T09:39:17.021 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 75489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_gre.ko 2024-04-17T09:39:17.021 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 77721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_tunnel.ko 2024-04-17T09:39:17.022 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_vti.ko 2024-04-17T09:39:17.022 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipcomp.ko 2024-04-17T09:39:17.022 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34745 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipip.ko 2024-04-17T09:39:17.022 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ 2024-04-17T09:39:17.022 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-17T09:39:17.022 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-17T09:39:17.022 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-17T09:39:17.022 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ip_tables.ko 2024-04-17T09:39:17.022 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-17T09:39:17.022 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-04-17T09:39:17.022 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-17T09:39:17.022 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-17T09:39:17.023 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-17T09:39:17.023 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-04-17T09:39:17.023 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-17T09:39:17.023 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-17T09:39:17.023 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-17T09:39:17.023 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-17T09:39:17.023 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-17T09:39:17.023 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-17T09:39:17.023 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-17T09:39:17.023 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-17T09:39:17.023 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-17T09:39:17.023 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-04-17T09:39:17.023 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-17T09:39:17.024 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-17T09:39:17.024 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-17T09:39:17.024 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-17T09:39:17.024 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-17T09:39:17.024 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_bic.ko 2024-04-17T09:39:17.024 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_dctcp.ko 2024-04-17T09:39:17.024 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_diag.ko 2024-04-17T09:39:17.024 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_highspeed.ko 2024-04-17T09:39:17.024 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_htcp.ko 2024-04-17T09:39:17.024 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_hybla.ko 2024-04-17T09:39:17.024 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_illinois.ko 2024-04-17T09:39:17.024 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_lp.ko 2024-04-17T09:39:17.025 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9601 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_scalable.ko 2024-04-17T09:39:17.025 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_vegas.ko 2024-04-17T09:39:17.025 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_veno.ko 2024-04-17T09:39:17.025 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_westwood.ko 2024-04-17T09:39:17.025 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11745 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_yeah.ko 2024-04-17T09:39:17.025 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tunnel4.ko 2024-04-17T09:39:17.025 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_diag.ko 2024-04-17T09:39:17.025 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_tunnel.ko 2024-04-17T09:39:17.025 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-17T09:39:17.025 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ 2024-04-17T09:39:17.025 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ah6.ko 2024-04-17T09:39:17.025 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/esp6.ko 2024-04-17T09:39:17.026 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/fou6.ko 2024-04-17T09:39:17.026 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 103177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_gre.ko 2024-04-17T09:39:17.026 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 118881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_tunnel.ko 2024-04-17T09:39:17.026 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-17T09:39:17.026 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 71089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_vti.ko 2024-04-17T09:39:17.026 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ipcomp6.ko 2024-04-17T09:39:17.026 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/mip6.ko 2024-04-17T09:39:17.026 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ 2024-04-17T09:39:17.026 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-17T09:39:17.026 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-04-17T09:39:17.026 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-17T09:39:17.027 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-17T09:39:17.027 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15929 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-17T09:39:17.027 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-17T09:39:17.027 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-17T09:39:17.027 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-17T09:39:17.027 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-17T09:39:17.027 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-17T09:39:17.027 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-17T09:39:17.027 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-17T09:39:17.027 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-17T09:39:17.027 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-17T09:39:17.028 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-17T09:39:17.028 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-17T09:39:17.028 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-17T09:39:17.028 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-17T09:39:17.028 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-17T09:39:17.028 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-17T09:39:17.028 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-17T09:39:17.028 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-17T09:39:17.028 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-17T09:39:17.028 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-17T09:39:17.028 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-17T09:39:17.029 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 94089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/sit.ko 2024-04-17T09:39:17.029 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/tunnel6.ko 2024-04-17T09:39:17.029 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-17T09:39:17.029 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key/ 2024-04-17T09:39:17.029 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 106369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key/af_key.ko 2024-04-17T09:39:17.029 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/ 2024-04-17T09:39:17.029 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 153217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_core.ko 2024-04-17T09:39:17.029 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-17T09:39:17.029 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_eth.ko 2024-04-17T09:39:17.029 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip.ko 2024-04-17T09:39:17.029 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip6.ko 2024-04-17T09:39:17.029 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_netlink.ko 2024-04-17T09:39:17.030 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ppp.ko 2024-04-17T09:39:17.030 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/lapb/ 2024-04-17T09:39:17.030 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/lapb/lapb.ko 2024-04-17T09:39:17.030 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc/ 2024-04-17T09:39:17.030 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc/llc.ko 2024-04-17T09:39:17.030 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 332889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc/llc2.ko 2024-04-17T09:39:17.030 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154/ 2024-04-17T09:39:17.030 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 388873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154/mac802154.ko 2024-04-17T09:39:17.030 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls/ 2024-04-17T09:39:17.030 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls/mpls_gso.ko 2024-04-17T09:39:17.030 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ 2024-04-17T09:39:17.031 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ 2024-04-17T09:39:17.031 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 147745 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set.ko 2024-04-17T09:39:17.031 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-17T09:39:17.031 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-17T09:39:17.031 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32553 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-17T09:39:17.031 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 124041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-17T09:39:17.031 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 124345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-17T09:39:17.031 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 126681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-17T09:39:17.031 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 126473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-17T09:39:17.031 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 137833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-17T09:39:17.031 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-17T09:39:17.031 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 133681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-17T09:39:17.032 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 139073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-17T09:39:17.032 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 138265 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-17T09:39:17.032 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 136177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-17T09:39:17.032 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 139897 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-17T09:39:17.032 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-17T09:39:17.032 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ 2024-04-17T09:39:17.032 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 534113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-17T09:39:17.032 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-17T09:39:17.032 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-17T09:39:17.032 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-17T09:39:17.032 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-17T09:39:17.033 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-17T09:39:17.033 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-17T09:39:17.033 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-17T09:39:17.033 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-17T09:39:17.033 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-17T09:39:17.033 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-17T09:39:17.033 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-17T09:39:17.033 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-17T09:39:17.033 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12809 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-17T09:39:17.033 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conncount.ko 2024-04-17T09:39:17.033 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 496817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack.ko 2024-04-17T09:39:17.033 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-17T09:39:17.034 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-17T09:39:17.034 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-17T09:39:17.034 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 134377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-17T09:39:17.034 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-17T09:39:17.034 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-17T09:39:17.034 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 140801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-17T09:39:17.034 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-17T09:39:17.034 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18937 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-17T09:39:17.034 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 79873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-17T09:39:17.034 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-17T09:39:17.034 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-17T09:39:17.035 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-17T09:39:17.035 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 117201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table.ko 2024-04-17T09:39:17.035 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-17T09:39:17.035 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_log_syslog.ko 2024-04-17T09:39:17.035 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 118369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat.ko 2024-04-17T09:39:17.035 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-17T09:39:17.035 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-17T09:39:17.035 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_irc.ko 2024-04-17T09:39:17.035 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_sip.ko 2024-04-17T09:39:17.035 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-17T09:39:17.035 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53393 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-17T09:39:17.036 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 807985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_tables.ko 2024-04-17T09:39:17.036 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink.ko 2024-04-17T09:39:17.036 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-17T09:39:17.036 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-04-17T09:39:17.036 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_log.ko 2024-04-17T09:39:17.036 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-17T09:39:17.036 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 65145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-17T09:39:17.036 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_chain_nat.ko 2024-04-17T09:39:17.036 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_compat.ko 2024-04-17T09:39:17.036 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50657 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_ct.ko 2024-04-17T09:39:17.036 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-17T09:39:17.036 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13745 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fib.ko 2024-04-17T09:39:17.037 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16545 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-17T09:39:17.037 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_hash.ko 2024-04-17T09:39:17.037 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_limit.ko 2024-04-17T09:39:17.037 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_log.ko 2024-04-17T09:39:17.037 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_masq.ko 2024-04-17T09:39:17.037 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_nat.ko 2024-04-17T09:39:17.037 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_numgen.ko 2024-04-17T09:39:17.037 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_osf.ko 2024-04-17T09:39:17.037 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_queue.ko 2024-04-17T09:39:17.037 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16657 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_redir.ko 2024-04-17T09:39:17.037 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject.ko 2024-04-17T09:39:17.037 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject_inet.ko 2024-04-17T09:39:17.038 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15057 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_socket.ko 2024-04-17T09:39:17.038 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_synproxy.ko 2024-04-17T09:39:17.038 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_tproxy.ko 2024-04-17T09:39:17.038 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 108393 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/x_tables.ko 2024-04-17T09:39:17.038 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14193 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_AUDIT.ko 2024-04-17T09:39:17.038 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-17T09:39:17.038 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-17T09:39:17.038 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-17T09:39:17.038 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CT.ko 2024-04-17T09:39:17.038 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_DSCP.ko 2024-04-17T09:39:17.038 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HL.ko 2024-04-17T09:39:17.039 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15745 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HMARK.ko 2024-04-17T09:39:17.039 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-17T09:39:17.039 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15937 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LED.ko 2024-04-17T09:39:17.039 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LOG.ko 2024-04-17T09:39:17.039 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-17T09:39:17.039 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NETMAP.ko 2024-04-17T09:39:17.039 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFLOG.ko 2024-04-17T09:39:17.039 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-17T09:39:17.039 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_RATEEST.ko 2024-04-17T09:39:17.039 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12553 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-17T09:39:17.039 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_SECMARK.ko 2024-04-17T09:39:17.039 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-17T09:39:17.040 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-17T09:39:17.040 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TEE.ko 2024-04-17T09:39:17.040 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TPROXY.ko 2024-04-17T09:39:17.040 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TRACE.ko 2024-04-17T09:39:17.040 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_addrtype.ko 2024-04-17T09:39:17.040 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_bpf.ko 2024-04-17T09:39:17.040 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cgroup.ko 2024-04-17T09:39:17.040 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cluster.ko 2024-04-17T09:39:17.040 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_comment.ko 2024-04-17T09:39:17.040 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connbytes.ko 2024-04-17T09:39:17.040 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlabel.ko 2024-04-17T09:39:17.041 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlimit.ko 2024-04-17T09:39:17.041 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connmark.ko 2024-04-17T09:39:17.041 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_conntrack.ko 2024-04-17T09:39:17.041 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cpu.ko 2024-04-17T09:39:17.041 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dccp.ko 2024-04-17T09:39:17.041 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_devgroup.ko 2024-04-17T09:39:17.041 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dscp.ko 2024-04-17T09:39:17.041 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12657 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ecn.ko 2024-04-17T09:39:17.041 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_esp.ko 2024-04-17T09:39:17.041 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hashlimit.ko 2024-04-17T09:39:17.041 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_helper.ko 2024-04-17T09:39:17.042 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hl.ko 2024-04-17T09:39:17.042 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipcomp.ko 2024-04-17T09:39:17.042 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_iprange.ko 2024-04-17T09:39:17.042 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipvs.ko 2024-04-17T09:39:17.042 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19193 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_l2tp.ko 2024-04-17T09:39:17.042 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_length.ko 2024-04-17T09:39:17.042 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_limit.ko 2024-04-17T09:39:17.042 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mac.ko 2024-04-17T09:39:17.042 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mark.ko 2024-04-17T09:39:17.042 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_multiport.ko 2024-04-17T09:39:17.042 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nat.ko 2024-04-17T09:39:17.043 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nfacct.ko 2024-04-17T09:39:17.043 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_osf.ko 2024-04-17T09:39:17.043 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_owner.ko 2024-04-17T09:39:17.043 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_physdev.ko 2024-04-17T09:39:17.043 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_pkttype.ko 2024-04-17T09:39:17.043 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13553 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_policy.ko 2024-04-17T09:39:17.043 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_quota.ko 2024-04-17T09:39:17.043 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_rateest.ko 2024-04-17T09:39:17.043 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_realm.ko 2024-04-17T09:39:17.043 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_recent.ko 2024-04-17T09:39:17.043 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_sctp.ko 2024-04-17T09:39:17.043 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_set.ko 2024-04-17T09:39:17.044 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_socket.ko 2024-04-17T09:39:17.044 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_state.ko 2024-04-17T09:39:17.044 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10849 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_statistic.ko 2024-04-17T09:39:17.044 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_string.ko 2024-04-17T09:39:17.044 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_tcpmss.ko 2024-04-17T09:39:17.044 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_tcpudp.ko 2024-04-17T09:39:17.044 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_time.ko 2024-04-17T09:39:17.044 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_u32.ko 2024-04-17T09:39:17.044 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink/ 2024-04-17T09:39:17.044 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink/netlink_diag.ko 2024-04-17T09:39:17.044 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh/ 2024-04-17T09:39:17.045 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh/nsh.ko 2024-04-17T09:39:17.045 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/ 2024-04-17T09:39:17.045 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 488825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/openvswitch.ko 2024-04-17T09:39:17.045 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-geneve.ko 2024-04-17T09:39:17.045 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-gre.ko 2024-04-17T09:39:17.045 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-vxlan.ko 2024-04-17T09:39:17.045 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet/ 2024-04-17T09:39:17.045 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15929 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet/af_packet_diag.ko 2024-04-17T09:39:17.045 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/phonet/ 2024-04-17T09:39:17.045 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 142025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/phonet/phonet.ko 2024-04-17T09:39:17.045 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 68817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/phonet/pn_pep.ko 2024-04-17T09:39:17.046 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/ 2024-04-17T09:39:17.046 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 318617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds.ko 2024-04-17T09:39:17.046 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 202305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_rdma.ko 2024-04-17T09:39:17.046 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 85105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_tcp.ko 2024-04-17T09:39:17.046 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rxrpc/ 2024-04-17T09:39:17.046 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1027897 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rxrpc/rxrpc.ko 2024-04-17T09:39:17.046 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/ 2024-04-17T09:39:17.046 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_csum.ko 2024-04-17T09:39:17.046 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_gact.ko 2024-04-17T09:39:17.046 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_mirred.ko 2024-04-17T09:39:17.046 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_nat.ko 2024-04-17T09:39:17.047 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_pedit.ko 2024-04-17T09:39:17.047 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_police.ko 2024-04-17T09:39:17.047 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_simple.ko 2024-04-17T09:39:17.047 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_skbedit.ko 2024-04-17T09:39:17.047 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_vlan.ko 2024-04-17T09:39:17.047 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_basic.ko 2024-04-17T09:39:17.047 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_bpf.ko 2024-04-17T09:39:17.047 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_cgroup.ko 2024-04-17T09:39:17.047 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_flow.ko 2024-04-17T09:39:17.047 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39601 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_fw.ko 2024-04-17T09:39:17.047 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_route.ko 2024-04-17T09:39:17.047 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 79145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_u32.ko 2024-04-17T09:39:17.048 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_cmp.ko 2024-04-17T09:39:17.048 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_ipset.ko 2024-04-17T09:39:17.048 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48785 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_meta.ko 2024-04-17T09:39:17.048 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_nbyte.ko 2024-04-17T09:39:17.048 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_text.ko 2024-04-17T09:39:17.048 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_u32.ko 2024-04-17T09:39:17.048 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_choke.ko 2024-04-17T09:39:17.048 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_codel.ko 2024-04-17T09:39:17.048 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_drr.ko 2024-04-17T09:39:17.048 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_fq.ko 2024-04-17T09:39:17.048 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_fq_codel.ko 2024-04-17T09:39:17.048 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_gred.ko 2024-04-17T09:39:17.049 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hfsc.ko 2024-04-17T09:39:17.049 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hhf.ko 2024-04-17T09:39:17.049 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 81833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_htb.ko 2024-04-17T09:39:17.049 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33545 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_ingress.ko 2024-04-17T09:39:17.049 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio.ko 2024-04-17T09:39:17.049 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio_lib.ko 2024-04-17T09:39:17.049 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_multiq.ko 2024-04-17T09:39:17.049 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_netem.ko 2024-04-17T09:39:17.049 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_pie.ko 2024-04-17T09:39:17.049 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_plug.ko 2024-04-17T09:39:17.049 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_prio.ko 2024-04-17T09:39:17.050 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_qfq.ko 2024-04-17T09:39:17.050 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_red.ko 2024-04-17T09:39:17.050 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfb.ko 2024-04-17T09:39:17.050 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfq.ko 2024-04-17T09:39:17.050 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_tbf.ko 2024-04-17T09:39:17.050 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_teql.ko 2024-04-17T09:39:17.050 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/ 2024-04-17T09:39:17.050 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1187313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp.ko 2024-04-17T09:39:17.050 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp_diag.ko 2024-04-17T09:39:17.050 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/ 2024-04-17T09:39:17.050 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/ 2024-04-17T09:39:17.051 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 432409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-17T09:39:17.051 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-17T09:39:17.051 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1940889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/sunrpc.ko 2024-04-17T09:39:17.051 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma/ 2024-04-17T09:39:17.051 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 890729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-17T09:39:17.051 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/ 2024-04-17T09:39:17.051 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1122337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc.ko 2024-04-17T09:39:17.051 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc_diag.ko 2024-04-17T09:39:17.051 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix/ 2024-04-17T09:39:17.051 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix/unix_diag.ko 2024-04-17T09:39:17.051 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/ 2024-04-17T09:39:17.051 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 105609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-17T09:39:17.052 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 90265 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-17T09:39:17.052 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 123769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock.ko 2024-04-17T09:39:17.052 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-17T09:39:17.052 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-17T09:39:17.052 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/x25/ 2024-04-17T09:39:17.052 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 165841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/x25/x25.ko 2024-04-17T09:39:17.052 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/ 2024-04-17T09:39:17.052 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/xfrm_algo.ko 2024-04-17T09:39:17.052 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-17T09:39:17.052 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 131865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/xfrm_user.ko 2024-04-17T09:39:17.052 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 644769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias 2024-04-17T09:39:17.052 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 628542 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias.bin 2024-04-17T09:39:17.052 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7706 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin 2024-04-17T09:39:17.053 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.alias.bin 2024-04-17T09:39:17.053 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9941 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.bin 2024-04-17T09:39:17.053 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70990 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.modinfo 2024-04-17T09:39:17.053 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 146959 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep 2024-04-17T09:39:17.053 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 232835 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep.bin 2024-04-17T09:39:17.053 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 210 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.devname 2024-04-17T09:39:17.053 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72602 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.order 2024-04-17T09:39:17.053 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 879 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.softdep 2024-04-17T09:39:17.053 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 258450 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols 2024-04-17T09:39:17.053 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 313019 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols.bin 2024-04-17T09:39:17.053 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./usr/ 2024-04-17T09:39:17.053 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./usr/share/ 2024-04-17T09:39:17.054 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./usr/share/doc/ 2024-04-17T09:39:17.054 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./usr/share/doc/linux-image-6.9.0-rc3-g3f60ed14f7c1/ 2024-04-17T09:39:17.054 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 174 2024-04-11 18:50 ./usr/share/doc/linux-image-6.9.0-rc3-g3f60ed14f7c1/changelog.Debian.gz 2024-04-17T09:39:17.054 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 691 2024-04-11 18:50 ./usr/share/doc/linux-image-6.9.0-rc3-g3f60ed14f7c1/copyright 2024-04-17T09:39:17.059 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:39:17.060 INFO:teuthology.task.kernel:searching for kernel 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:39:17.060 DEBUG:teuthology.orchestra.run.smithi032:> egrep '(submenu|menuentry.*6.9.0-rc3-g3f60ed14f7c1).*{' /boot/grub/grub.cfg 2024-04-17T09:39:17.066 INFO:teuthology.orchestra.run.smithi032.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-17T09:39:17.067 INFO:teuthology.orchestra.run.smithi032.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g3f60ed14f7c1' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g3f60ed14f7c1-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-17T09:39:17.067 INFO:teuthology.orchestra.run.smithi032.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g3f60ed14f7c1 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g3f60ed14f7c1-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-17T09:39:17.067 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-04-17T09:39:17.067 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:39:17.068 DEBUG:teuthology.orchestra.run.smithi032:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-04-17T09:39:17.111 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi032.front.sepia.ceph.com to complete... 2024-04-17T09:39:17.486 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub' 2024-04-17T09:39:17.487 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-17T09:39:17.493 INFO:teuthology.orchestra.run.smithi032.stderr:Generating grub configuration file ... 2024-04-17T09:39:17.662 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:39:17.670 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:39:17.888 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-17T09:39:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-17T09:39:17.933 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-17T09:39:17.939 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-17T09:39:18.077 INFO:teuthology.orchestra.run.smithi032.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-17T09:39:18.077 INFO:teuthology.orchestra.run.smithi032.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-17T09:39:18.077 INFO:teuthology.orchestra.run.smithi032.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-17T09:39:18.095 INFO:teuthology.orchestra.run.smithi032.stderr:done 2024-04-17T09:39:19.224 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2024-04-17T09:39:33.911 INFO:teuthology.orchestra.run.smithi144.stderr:Sourcing file `/etc/default/grub' 2024-04-17T09:39:33.929 INFO:teuthology.orchestra.run.smithi144.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-17T09:39:33.953 INFO:teuthology.orchestra.run.smithi144.stderr:Generating grub configuration file ... 2024-04-17T09:39:34.285 INFO:teuthology.orchestra.run.smithi144.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:39:34.303 INFO:teuthology.orchestra.run.smithi144.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:39:34.634 INFO:teuthology.orchestra.run.smithi144.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-17T09:39:34.641 INFO:teuthology.orchestra.run.smithi144.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-17T09:39:34.702 INFO:teuthology.orchestra.run.smithi144.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-17T09:39:34.708 INFO:teuthology.orchestra.run.smithi144.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-17T09:39:34.848 INFO:teuthology.orchestra.run.smithi144.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-17T09:39:34.849 INFO:teuthology.orchestra.run.smithi144.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-17T09:39:34.849 INFO:teuthology.orchestra.run.smithi144.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-17T09:39:34.900 INFO:teuthology.orchestra.run.smithi144.stderr:done 2024-04-17T09:39:35.126 DEBUG:teuthology.orchestra.run.smithi144:> dpkg-deb -c /tmp/linux-image.deb 2024-04-17T09:39:35.133 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./ 2024-04-17T09:39:35.133 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./boot/ 2024-04-17T09:39:35.133 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 7002012 2024-04-11 18:50 ./boot/System.map-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:39:35.147 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 172073 2024-04-11 18:50 ./boot/config-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:39:35.148 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13120512 2024-04-11 18:50 ./boot/vmlinuz-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:39:35.167 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./etc/ 2024-04-17T09:39:35.167 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./etc/kernel/ 2024-04-17T09:39:35.167 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./etc/kernel/postinst.d/ 2024-04-17T09:39:35.167 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./etc/kernel/postrm.d/ 2024-04-17T09:39:35.167 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./etc/kernel/preinst.d/ 2024-04-17T09:39:35.168 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./etc/kernel/prerm.d/ 2024-04-17T09:39:35.168 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/ 2024-04-17T09:39:35.168 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/ 2024-04-17T09:39:35.168 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/ 2024-04-17T09:39:35.168 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/ 2024-04-17T09:39:35.168 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/ 2024-04-17T09:39:35.168 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/ 2024-04-17T09:39:35.168 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/ 2024-04-17T09:39:35.168 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 507569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/aesni-intel.ko 2024-04-17T09:39:35.171 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-17T09:39:35.171 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 34313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-17T09:39:35.171 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 35009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-17T09:39:35.171 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 97953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-17T09:39:35.171 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 109905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-17T09:39:35.171 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 155249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-17T09:39:35.171 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-17T09:39:35.171 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-17T09:39:35.171 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 87009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-17T09:39:35.171 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-17T09:39:35.172 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 50305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-17T09:39:35.172 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 50377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-17T09:39:35.172 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 54513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-17T09:39:35.172 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 44657 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-17T09:39:35.172 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 48049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-17T09:39:35.172 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 62873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-17T09:39:35.172 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 53569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-17T09:39:35.172 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 34761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-17T09:39:35.172 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14809 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-17T09:39:35.172 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/ 2024-04-17T09:39:35.172 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/ 2024-04-17T09:39:35.172 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce/ 2024-04-17T09:39:35.172 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 49081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-17T09:39:35.172 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpuid.ko 2024-04-17T09:39:35.173 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/msr.ko 2024-04-17T09:39:35.173 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/ 2024-04-17T09:39:35.173 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 444425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-17T09:39:35.173 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 1035881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-17T09:39:35.173 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 2849609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm.ko 2024-04-17T09:39:35.250 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ 2024-04-17T09:39:35.250 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 65025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/af_alg.ko 2024-04-17T09:39:35.250 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 27961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/algif_hash.ko 2024-04-17T09:39:35.250 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/algif_skcipher.ko 2024-04-17T09:39:35.250 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ansi_cprng.ko 2024-04-17T09:39:35.250 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/anubis.ko 2024-04-17T09:39:35.250 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10929 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/arc4.ko 2024-04-17T09:39:35.251 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/asymmetric_keys/ 2024-04-17T09:39:35.251 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-04-17T09:39:35.251 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/ 2024-04-17T09:39:35.251 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_memcpy.ko 2024-04-17T09:39:35.251 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_pq.ko 2024-04-17T09:39:35.251 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-17T09:39:35.251 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_tx.ko 2024-04-17T09:39:35.251 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_xor.ko 2024-04-17T09:39:35.251 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/raid6test.ko 2024-04-17T09:39:35.251 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authenc.ko 2024-04-17T09:39:35.251 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 27953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authencesn.ko 2024-04-17T09:39:35.251 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blake2b_generic.ko 2024-04-17T09:39:35.251 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_common.ko 2024-04-17T09:39:35.251 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_generic.ko 2024-04-17T09:39:35.252 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 40889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/camellia_generic.ko 2024-04-17T09:39:35.252 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast5_generic.ko 2024-04-17T09:39:35.252 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast6_generic.ko 2024-04-17T09:39:35.252 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast_common.ko 2024-04-17T09:39:35.252 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 43281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ccm.ko 2024-04-17T09:39:35.252 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cmac.ko 2024-04-17T09:39:35.252 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crc32_generic.ko 2024-04-17T09:39:35.252 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 59129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cryptd.ko 2024-04-17T09:39:35.252 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 48113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_engine.ko 2024-04-17T09:39:35.252 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 32169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_simd.ko 2024-04-17T09:39:35.252 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 30881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_user.ko 2024-04-17T09:39:35.252 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cts.ko 2024-04-17T09:39:35.252 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/des_generic.ko 2024-04-17T09:39:35.252 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/dh_generic.ko 2024-04-17T09:39:35.252 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/echainiv.ko 2024-04-17T09:39:35.253 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 30689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/essiv.ko 2024-04-17T09:39:35.253 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/fcrypt.ko 2024-04-17T09:39:35.253 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/khazad.ko 2024-04-17T09:39:35.253 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lrw.ko 2024-04-17T09:39:35.253 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4.ko 2024-04-17T09:39:35.253 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4hc.ko 2024-04-17T09:39:35.253 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13393 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/md4.ko 2024-04-17T09:39:35.253 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/michael_mic.ko 2024-04-17T09:39:35.253 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcbc.ko 2024-04-17T09:39:35.253 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcrypt.ko 2024-04-17T09:39:35.253 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/rmd160.ko 2024-04-17T09:39:35.253 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 30673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/seed.ko 2024-04-17T09:39:35.253 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/serpent_generic.ko 2024-04-17T09:39:35.253 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 169065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tcrypt.ko 2024-04-17T09:39:35.253 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tea.ko 2024-04-17T09:39:35.254 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_common.ko 2024-04-17T09:39:35.254 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_generic.ko 2024-04-17T09:39:35.254 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/vmac.ko 2024-04-17T09:39:35.254 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 38209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/wp512.ko 2024-04-17T09:39:35.254 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xcbc.ko 2024-04-17T09:39:35.254 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 30345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xor.ko 2024-04-17T09:39:35.254 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xts.ko 2024-04-17T09:39:35.254 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xxhash_generic.ko 2024-04-17T09:39:35.254 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ 2024-04-17T09:39:35.254 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/ 2024-04-17T09:39:35.254 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_extlog.ko 2024-04-17T09:39:35.254 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 31921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-17T09:39:35.254 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 32281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_pad.ko 2024-04-17T09:39:35.255 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/apei/ 2024-04-17T09:39:35.255 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 44705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/apei/einj.ko 2024-04-17T09:39:35.255 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/ec_sys.ko 2024-04-17T09:39:35.255 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/platform_profile.ko 2024-04-17T09:39:35.255 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbs.ko 2024-04-17T09:39:35.255 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbshc.ko 2024-04-17T09:39:35.255 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 119033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/video.ko 2024-04-17T09:39:35.255 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ 2024-04-17T09:39:35.255 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/acard-ahci.ko 2024-04-17T09:39:35.255 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 96825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ahci.ko 2024-04-17T09:39:35.255 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ahci_platform.ko 2024-04-17T09:39:35.255 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 107737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/libahci.ko 2024-04-17T09:39:35.255 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 42001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/libahci_platform.ko 2024-04-17T09:39:35.255 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_acpi.ko 2024-04-17T09:39:35.256 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 35705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ali.ko 2024-04-17T09:39:35.256 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 38329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_amd.ko 2024-04-17T09:39:35.256 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_artop.ko 2024-04-17T09:39:35.256 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atiixp.ko 2024-04-17T09:39:35.256 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28785 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atp867x.ko 2024-04-17T09:39:35.256 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd640.ko 2024-04-17T09:39:35.256 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28929 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd64x.ko 2024-04-17T09:39:35.256 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cypress.ko 2024-04-17T09:39:35.256 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_efar.ko 2024-04-17T09:39:35.256 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt366.ko 2024-04-17T09:39:35.256 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 38913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt37x.ko 2024-04-17T09:39:35.256 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-17T09:39:35.256 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-17T09:39:35.256 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it8213.ko 2024-04-17T09:39:35.256 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 41289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it821x.ko 2024-04-17T09:39:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_jmicron.ko 2024-04-17T09:39:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 60257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_legacy.ko 2024-04-17T09:39:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_marvell.ko 2024-04-17T09:39:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_mpiix.ko 2024-04-17T09:39:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_netcell.ko 2024-04-17T09:39:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ninja32.ko 2024-04-17T09:39:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87410.ko 2024-04-17T09:39:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87415.ko 2024-04-17T09:39:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_oldpiix.ko 2024-04-17T09:39:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_opti.ko 2024-04-17T09:39:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_optidma.ko 2024-04-17T09:39:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 35145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pcmcia.ko 2024-04-17T09:39:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 43009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-17T09:39:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-17T09:39:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_piccolo.ko 2024-04-17T09:39:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_radisys.ko 2024-04-17T09:39:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_rdc.ko 2024-04-17T09:39:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_rz1000.ko 2024-04-17T09:39:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sch.ko 2024-04-17T09:39:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_serverworks.ko 2024-04-17T09:39:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 27305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sil680.ko 2024-04-17T09:39:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sl82c105.ko 2024-04-17T09:39:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_triflex.ko 2024-04-17T09:39:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 35025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_via.ko 2024-04-17T09:39:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pdc_adma.ko 2024-04-17T09:39:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 34889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_inic162x.ko 2024-04-17T09:39:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 102121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_mv.ko 2024-04-17T09:39:35.259 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 85409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_nv.ko 2024-04-17T09:39:35.259 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 44313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_promise.ko 2024-04-17T09:39:35.259 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_qstor.ko 2024-04-17T09:39:35.259 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 34025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil.ko 2024-04-17T09:39:35.259 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 45905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil24.ko 2024-04-17T09:39:35.259 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sis.ko 2024-04-17T09:39:35.259 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_svw.ko 2024-04-17T09:39:35.259 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 53353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sx4.ko 2024-04-17T09:39:35.259 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_uli.ko 2024-04-17T09:39:35.259 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 41049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_via.ko 2024-04-17T09:39:35.259 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_vsc.ko 2024-04-17T09:39:35.259 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay/ 2024-04-17T09:39:35.259 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay/ks0108.ko 2024-04-17T09:39:35.259 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma/ 2024-04-17T09:39:35.259 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 163969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma/bcma.ko 2024-04-17T09:39:35.260 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/ 2024-04-17T09:39:35.260 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe/ 2024-04-17T09:39:35.260 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 140865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe/aoe.ko 2024-04-17T09:39:35.260 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd/ 2024-04-17T09:39:35.260 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 1241657 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd/drbd.ko 2024-04-17T09:39:35.260 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 203289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/floppy.ko 2024-04-17T09:39:35.260 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx/ 2024-04-17T09:39:35.260 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 115265 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-17T09:39:35.260 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 143433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/nbd.ko 2024-04-17T09:39:35.260 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk/ 2024-04-17T09:39:35.260 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 126617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk/null_blk.ko 2024-04-17T09:39:35.260 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 119873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/pktcdvd.ko 2024-04-17T09:39:35.260 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 313217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/rbd.ko 2024-04-17T09:39:35.261 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback/ 2024-04-17T09:39:35.261 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 117233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-17T09:39:35.261 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram/ 2024-04-17T09:39:35.261 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 70385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram/zram.ko 2024-04-17T09:39:35.261 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ 2024-04-17T09:39:35.261 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 35377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/applicom.ko 2024-04-17T09:39:35.261 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hangcheck-timer.ko 2024-04-17T09:39:35.261 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/ 2024-04-17T09:39:35.261 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-17T09:39:35.261 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-17T09:39:35.261 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-17T09:39:35.261 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/via-rng.ko 2024-04-17T09:39:35.261 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-17T09:39:35.261 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ 2024-04-17T09:39:35.262 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 37257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-17T09:39:35.262 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 186921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-17T09:39:35.262 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 32521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-17T09:39:35.262 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 179561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-17T09:39:35.262 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 85441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-17T09:39:35.262 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 65313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-17T09:39:35.262 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 43857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/lp.ko 2024-04-17T09:39:35.262 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave/ 2024-04-17T09:39:35.262 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 103409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave/mwave.ko 2024-04-17T09:39:35.262 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 31833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/nvram.ko 2024-04-17T09:39:35.262 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 38977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ppdev.ko 2024-04-17T09:39:35.262 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 59241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tlclk.ko 2024-04-17T09:39:35.262 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/ 2024-04-17T09:39:35.263 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-17T09:39:35.263 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-04-17T09:39:35.263 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-04-17T09:39:35.263 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 39193 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-04-17T09:39:35.263 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 34169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-17T09:39:35.263 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-17T09:39:35.263 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-04-17T09:39:35.263 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/xillybus/ 2024-04-17T09:39:35.263 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/xillybus/xillybus_class.ko 2024-04-17T09:39:35.263 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 62545 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/xillybus/xillybus_core.ko 2024-04-17T09:39:35.263 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13929 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-04-17T09:39:35.263 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/clk/ 2024-04-17T09:39:35.263 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/clk/clk-palmas.ko 2024-04-17T09:39:35.263 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 57289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/clk/clk-si5351.ko 2024-04-17T09:39:35.263 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/clk/clk-twl6040.ko 2024-04-17T09:39:35.264 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/clk/clk-wm831x.ko 2024-04-17T09:39:35.264 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/ 2024-04-17T09:39:35.264 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-17T09:39:35.264 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-17T09:39:35.264 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-17T09:39:35.264 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ 2024-04-17T09:39:35.264 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ 2024-04-17T09:39:35.264 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 89257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-17T09:39:35.264 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 333657 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp.ko 2024-04-17T09:39:35.264 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/ 2024-04-17T09:39:35.264 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/ 2024-04-17T09:39:35.264 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common/ 2024-04-17T09:39:35.264 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 751713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-17T09:39:35.264 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-04-17T09:39:35.265 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-17T09:39:35.265 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-aes.ko 2024-04-17T09:39:35.265 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-sha.ko 2024-04-17T09:39:35.265 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio/ 2024-04-17T09:39:35.265 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 90441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-17T09:39:35.265 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca/ 2024-04-17T09:39:35.265 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 36105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca/dca.ko 2024-04-17T09:39:35.265 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ 2024-04-17T09:39:35.265 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw/ 2024-04-17T09:39:35.265 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-17T09:39:35.265 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat/ 2024-04-17T09:39:35.265 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 146769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-17T09:39:35.265 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/ 2024-04-17T09:39:35.265 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 96145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/amd64_edac.ko 2024-04-17T09:39:35.266 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 41993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/e752x_edac.ko 2024-04-17T09:39:35.266 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 51105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/edac_mce_amd.ko 2024-04-17T09:39:35.266 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3000_edac.ko 2024-04-17T09:39:35.266 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3200_edac.ko 2024-04-17T09:39:35.266 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5100_edac.ko 2024-04-17T09:39:35.266 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5400_edac.ko 2024-04-17T09:39:35.266 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7300_edac.ko 2024-04-17T09:39:35.266 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 64161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7core_edac.ko 2024-04-17T09:39:35.266 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i82975x_edac.ko 2024-04-17T09:39:35.266 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/ie31200_edac.ko 2024-04-17T09:39:35.266 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 68697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/sb_edac.ko 2024-04-17T09:39:35.266 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/x38_edac.ko 2024-04-17T09:39:35.266 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/extcon/ 2024-04-17T09:39:35.266 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/extcon/extcon-gpio.ko 2024-04-17T09:39:35.267 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 31369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/extcon/extcon-max14577.ko 2024-04-17T09:39:35.267 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 35601 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/extcon/extcon-max77693.ko 2024-04-17T09:39:35.267 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/extcon/extcon-max8997.ko 2024-04-17T09:39:35.267 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 30961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/extcon/extcon-palmas.ko 2024-04-17T09:39:35.267 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 27201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/extcon/extcon-rt8973a.ko 2024-04-17T09:39:35.267 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 31993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/extcon/extcon-sm5502.ko 2024-04-17T09:39:35.267 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/ 2024-04-17T09:39:35.267 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 42953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/dmi-sysfs.ko 2024-04-17T09:39:35.267 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/efi/ 2024-04-17T09:39:35.267 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/efi/efi-pstore.ko 2024-04-17T09:39:35.267 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-17T09:39:35.267 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/ 2024-04-17T09:39:35.267 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-adp5520.ko 2024-04-17T09:39:35.267 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-amd8111.ko 2024-04-17T09:39:35.267 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-arizona.ko 2024-04-17T09:39:35.267 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16745 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-da9052.ko 2024-04-17T09:39:35.268 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-da9055.ko 2024-04-17T09:39:35.268 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-dln2.ko 2024-04-17T09:39:35.268 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 81073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-f7188x.ko 2024-04-17T09:39:35.268 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 40761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-generic.ko 2024-04-17T09:39:35.268 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-ich.ko 2024-04-17T09:39:35.268 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-04-17T09:39:35.268 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-kempld.ko 2024-04-17T09:39:35.268 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-lp3943.ko 2024-04-17T09:39:35.268 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-max7300.ko 2024-04-17T09:39:35.268 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-max730x.ko 2024-04-17T09:39:35.268 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24745 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-max732x.ko 2024-04-17T09:39:35.268 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-04-17T09:39:35.268 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 55505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-pca953x.ko 2024-04-17T09:39:35.268 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 31881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-pcf857x.ko 2024-04-17T09:39:35.268 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-rdc321x.ko 2024-04-17T09:39:35.268 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-sch.ko 2024-04-17T09:39:35.269 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-sch311x.ko 2024-04-17T09:39:35.269 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14265 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-tps65912.ko 2024-04-17T09:39:35.269 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-twl4030.ko 2024-04-17T09:39:35.269 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-twl6040.ko 2024-04-17T09:39:35.269 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-17T09:39:35.269 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17897 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-vx855.ko 2024-04-17T09:39:35.269 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-wm831x.ko 2024-04-17T09:39:35.269 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-wm8350.ko 2024-04-17T09:39:35.269 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-wm8994.ko 2024-04-17T09:39:35.269 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/ 2024-04-17T09:39:35.269 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-a4tech.ko 2024-04-17T09:39:35.269 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 43553 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-apple.ko 2024-04-17T09:39:35.269 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-appleir.ko 2024-04-17T09:39:35.269 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9393 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-aureal.ko 2024-04-17T09:39:35.269 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-axff.ko 2024-04-17T09:39:35.269 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-belkin.ko 2024-04-17T09:39:35.270 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-cherry.ko 2024-04-17T09:39:35.270 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-chicony.ko 2024-04-17T09:39:35.270 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 74393 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-cp2112.ko 2024-04-17T09:39:35.270 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-cypress.ko 2024-04-17T09:39:35.270 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-dr.ko 2024-04-17T09:39:35.270 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elecom.ko 2024-04-17T09:39:35.270 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elo.ko 2024-04-17T09:39:35.270 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-emsff.ko 2024-04-17T09:39:35.270 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-ezkey.ko 2024-04-17T09:39:35.270 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gaff.ko 2024-04-17T09:39:35.270 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-generic.ko 2024-04-17T09:39:35.270 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gt683r.ko 2024-04-17T09:39:35.270 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gyration.ko 2024-04-17T09:39:35.270 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-17T09:39:35.271 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-17T09:39:35.271 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtekff.ko 2024-04-17T09:39:35.271 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-hyperv.ko 2024-04-17T09:39:35.271 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-icade.ko 2024-04-17T09:39:35.271 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-kensington.ko 2024-04-17T09:39:35.271 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-keytouch.ko 2024-04-17T09:39:35.271 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-kye.ko 2024-04-17T09:39:35.271 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12265 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lcpower.ko 2024-04-17T09:39:35.271 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-led.ko 2024-04-17T09:39:35.271 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 54217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lenovo.ko 2024-04-17T09:39:35.271 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 37553 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lg-g15.ko 2024-04-17T09:39:35.271 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 68081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-17T09:39:35.271 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 121369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-17T09:39:35.271 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 97953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech.ko 2024-04-17T09:39:35.271 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 36689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-magicmouse.ko 2024-04-17T09:39:35.272 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-microsoft.ko 2024-04-17T09:39:35.272 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-monterey.ko 2024-04-17T09:39:35.272 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 59001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-multitouch.ko 2024-04-17T09:39:35.272 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 39697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-ntrig.ko 2024-04-17T09:39:35.272 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-ortek.ko 2024-04-17T09:39:35.272 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-penmount.ko 2024-04-17T09:39:35.272 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-petalynx.ko 2024-04-17T09:39:35.272 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 91241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-picolcd.ko 2024-04-17T09:39:35.272 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-pl.ko 2024-04-17T09:39:35.272 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-plantronics.ko 2024-04-17T09:39:35.272 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-primax.ko 2024-04-17T09:39:35.272 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 42889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-rmi.ko 2024-04-17T09:39:35.272 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-17T09:39:35.272 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-common.ko 2024-04-17T09:39:35.272 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 38865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-17T09:39:35.273 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 36305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-17T09:39:35.273 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 38073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-17T09:39:35.273 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-17T09:39:35.273 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 35625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-17T09:39:35.273 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-17T09:39:35.273 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 35425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-17T09:39:35.273 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 35561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-17T09:39:35.273 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-17T09:39:35.273 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat.ko 2024-04-17T09:39:35.273 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-saitek.ko 2024-04-17T09:39:35.273 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 42305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-samsung.ko 2024-04-17T09:39:35.273 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 43609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-17T09:39:35.273 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sjoy.ko 2024-04-17T09:39:35.273 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 62129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sony.ko 2024-04-17T09:39:35.273 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-speedlink.ko 2024-04-17T09:39:35.273 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 30017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-steelseries.ko 2024-04-17T09:39:35.274 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sunplus.ko 2024-04-17T09:39:35.274 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-17T09:39:35.274 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tivo.ko 2024-04-17T09:39:35.274 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tmff.ko 2024-04-17T09:39:35.274 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-topseed.ko 2024-04-17T09:39:35.274 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-twinhan.ko 2024-04-17T09:39:35.274 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 81113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-uclogic.ko 2024-04-17T09:39:35.274 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-waltop.ko 2024-04-17T09:39:35.274 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 184745 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-wiimote.ko 2024-04-17T09:39:35.274 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-xinmo.ko 2024-04-17T09:39:35.274 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zpff.ko 2024-04-17T09:39:35.274 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zydacron.ko 2024-04-17T09:39:35.274 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 319921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid.ko 2024-04-17T09:39:35.274 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 39529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/uhid.ko 2024-04-17T09:39:35.274 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/usbhid/ 2024-04-17T09:39:35.275 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 153633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/usbhid/usbhid.ko 2024-04-17T09:39:35.275 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/usbhid/usbkbd.ko 2024-04-17T09:39:35.275 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/usbhid/usbmouse.ko 2024-04-17T09:39:35.275 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 261945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/wacom.ko 2024-04-17T09:39:35.275 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/ 2024-04-17T09:39:35.275 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 75065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_balloon.ko 2024-04-17T09:39:35.275 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 110681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_utils.ko 2024-04-17T09:39:35.275 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 382033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_vmbus.ko 2024-04-17T09:39:35.275 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ 2024-04-17T09:39:35.275 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 77153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru.ko 2024-04-17T09:39:35.275 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 67009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru3.ko 2024-04-17T09:39:35.275 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 42249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-17T09:39:35.275 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7414.ko 2024-04-17T09:39:35.275 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7418.ko 2024-04-17T09:39:35.275 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adc128d818.ko 2024-04-17T09:39:35.276 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 38097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1025.ko 2024-04-17T09:39:35.276 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 115057 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1026.ko 2024-04-17T09:39:35.276 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1029.ko 2024-04-17T09:39:35.276 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 55225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1031.ko 2024-04-17T09:39:35.276 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 31441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm9240.ko 2024-04-17T09:39:35.276 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ads7828.ko 2024-04-17T09:39:35.276 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7410.ko 2024-04-17T09:39:35.276 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7411.ko 2024-04-17T09:39:35.276 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 89025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7462.ko 2024-04-17T09:39:35.276 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 49153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7470.ko 2024-04-17T09:39:35.276 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 81473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7475.ko 2024-04-17T09:39:35.276 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7x10.ko 2024-04-17T09:39:35.276 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 48177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/amc6821.ko 2024-04-17T09:39:35.276 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 68865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/applesmc.ko 2024-04-17T09:39:35.276 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 60193 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asb100.ko 2024-04-17T09:39:35.276 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 69009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asc7621.ko 2024-04-17T09:39:35.277 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 47977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-17T09:39:35.277 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/atxp1.ko 2024-04-17T09:39:35.277 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/coretemp.ko 2024-04-17T09:39:35.277 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/da9052-hwmon.ko 2024-04-17T09:39:35.277 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/da9055-hwmon.ko 2024-04-17T09:39:35.277 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 55713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-17T09:39:35.277 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 113201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dme1737.ko 2024-04-17T09:39:35.277 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds1621.ko 2024-04-17T09:39:35.277 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds620.ko 2024-04-17T09:39:35.277 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc1403.ko 2024-04-17T09:39:35.277 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 40177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc2103.ko 2024-04-17T09:39:35.277 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 34649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc6w201.ko 2024-04-17T09:39:35.277 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 78921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71805f.ko 2024-04-17T09:39:35.277 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 108961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71882fg.ko 2024-04-17T09:39:35.277 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 50697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f75375s.ko 2024-04-17T09:39:35.277 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fam15h_power.ko 2024-04-17T09:39:35.278 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 65769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fschmd.ko 2024-04-17T09:39:35.278 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g760a.ko 2024-04-17T09:39:35.278 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 30585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g762.ko 2024-04-17T09:39:35.278 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 59761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl518sm.ko 2024-04-17T09:39:35.278 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 56313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl520sm.ko 2024-04-17T09:39:35.278 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/hih6130.ko 2024-04-17T09:39:35.278 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-17T09:39:35.278 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5500_temp.ko 2024-04-17T09:39:35.278 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 27633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5k_amb.ko 2024-04-17T09:39:35.278 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 42585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmaem.ko 2024-04-17T09:39:35.278 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmpex.ko 2024-04-17T09:39:35.278 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina209.ko 2024-04-17T09:39:35.278 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina2xx.ko 2024-04-17T09:39:35.278 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 156465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/it87.ko 2024-04-17T09:39:35.279 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/jc42.ko 2024-04-17T09:39:35.279 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k10temp.ko 2024-04-17T09:39:35.279 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k8temp.ko 2024-04-17T09:39:35.279 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lineage-pem.ko 2024-04-17T09:39:35.279 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 57769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm63.ko 2024-04-17T09:39:35.279 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm73.ko 2024-04-17T09:39:35.279 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 44121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm75.ko 2024-04-17T09:39:35.279 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm77.ko 2024-04-17T09:39:35.279 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 41473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm78.ko 2024-04-17T09:39:35.279 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 37049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm80.ko 2024-04-17T09:39:35.279 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20545 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm83.ko 2024-04-17T09:39:35.279 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 86497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm85.ko 2024-04-17T09:39:35.279 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 55641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm87.ko 2024-04-17T09:39:35.279 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 42249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm90.ko 2024-04-17T09:39:35.279 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm92.ko 2024-04-17T09:39:35.279 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 131505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm93.ko 2024-04-17T09:39:35.280 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 40225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95234.ko 2024-04-17T09:39:35.280 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95241.ko 2024-04-17T09:39:35.280 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95245.ko 2024-04-17T09:39:35.280 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26393 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc2945.ko 2024-04-17T09:39:35.280 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4151.ko 2024-04-17T09:39:35.280 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4215.ko 2024-04-17T09:39:35.280 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4222.ko 2024-04-17T09:39:35.280 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4245.ko 2024-04-17T09:39:35.280 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4260.ko 2024-04-17T09:39:35.280 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4261.ko 2024-04-17T09:39:35.280 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 36769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max16065.ko 2024-04-17T09:39:35.280 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1619.ko 2024-04-17T09:39:35.280 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1668.ko 2024-04-17T09:39:35.280 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max197.ko 2024-04-17T09:39:35.280 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 32505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6639.ko 2024-04-17T09:39:35.281 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 27305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6650.ko 2024-04-17T09:39:35.281 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 39729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6697.ko 2024-04-17T09:39:35.281 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/mc13783-adc.ko 2024-04-17T09:39:35.281 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/mcp3021.ko 2024-04-17T09:39:35.281 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-04-17T09:39:35.281 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 56505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6683.ko 2024-04-17T09:39:35.281 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 141353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775-core.ko 2024-04-17T09:39:35.281 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 72737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775.ko 2024-04-17T09:39:35.281 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 67841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct7802.ko 2024-04-17T09:39:35.281 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 97777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87360.ko 2024-04-17T09:39:35.281 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 61921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87427.ko 2024-04-17T09:39:35.281 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pcf8591.ko 2024-04-17T09:39:35.281 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ 2024-04-17T09:39:35.282 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-17T09:39:35.282 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-17T09:39:35.282 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33265 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-17T09:39:35.282 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-17T09:39:35.282 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-17T09:39:35.282 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-17T09:39:35.282 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-17T09:39:35.282 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 101233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-17T09:39:35.282 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-17T09:39:35.282 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 36177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-17T09:39:35.282 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-17T09:39:35.282 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-17T09:39:35.282 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/powr1220.ko 2024-04-17T09:39:35.282 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5627.ko 2024-04-17T09:39:35.283 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 32945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5636.ko 2024-04-17T09:39:35.283 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 34561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-17T09:39:35.283 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 40849 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht15.ko 2024-04-17T09:39:35.283 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht21.ko 2024-04-17T09:39:35.283 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/shtc1.ko 2024-04-17T09:39:35.283 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 48713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sis5595.ko 2024-04-17T09:39:35.283 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47b397.ko 2024-04-17T09:39:35.283 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 46473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m1.ko 2024-04-17T09:39:35.283 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 44313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m192.ko 2024-04-17T09:39:35.283 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 31417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/thmc50.ko 2024-04-17T09:39:35.283 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp102.ko 2024-04-17T09:39:35.283 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp103.ko 2024-04-17T09:39:35.283 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp401.ko 2024-04-17T09:39:35.283 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp421.ko 2024-04-17T09:39:35.283 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via-cputemp.ko 2024-04-17T09:39:35.283 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 50217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via686a.ko 2024-04-17T09:39:35.284 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 59665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt1211.ko 2024-04-17T09:39:35.284 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 62737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt8231.ko 2024-04-17T09:39:35.284 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 71169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627ehf.ko 2024-04-17T09:39:35.284 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 96577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627hf.ko 2024-04-17T09:39:35.284 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 85913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83781d.ko 2024-04-17T09:39:35.284 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 75129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83791d.ko 2024-04-17T09:39:35.284 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 83425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83792d.ko 2024-04-17T09:39:35.284 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 116777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83793.ko 2024-04-17T09:39:35.284 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 134441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83795.ko 2024-04-17T09:39:35.284 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l785ts.ko 2024-04-17T09:39:35.284 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 43385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l786ng.ko 2024-04-17T09:39:35.284 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-04-17T09:39:35.284 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-04-17T09:39:35.284 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/ 2024-04-17T09:39:35.284 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/ 2024-04-17T09:39:35.285 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 27353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-17T09:39:35.285 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-17T09:39:35.285 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/ 2024-04-17T09:39:35.285 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-04-17T09:39:35.285 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 35073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-04-17T09:39:35.285 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 32905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-04-17T09:39:35.285 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-17T09:39:35.285 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 30361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-17T09:39:35.285 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-17T09:39:35.285 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-04-17T09:39:35.285 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 7729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-17T09:39:35.285 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-04-17T09:39:35.285 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 64233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-17T09:39:35.285 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-17T09:39:35.285 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-17T09:39:35.285 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-17T09:39:35.286 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-04-17T09:39:35.286 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-04-17T09:39:35.286 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 76969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-17T09:39:35.286 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-17T09:39:35.286 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 52937 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-17T09:39:35.286 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-04-17T09:39:35.286 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-17T09:39:35.286 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-17T09:39:35.286 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-04-17T09:39:35.286 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 30049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-17T09:39:35.286 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20057 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-17T09:39:35.286 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 57217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-17T09:39:35.286 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-04-17T09:39:35.286 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-17T09:39:35.287 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-17T09:39:35.287 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-04-17T09:39:35.287 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 32585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-04-17T09:39:35.287 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-17T09:39:35.287 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-04-17T09:39:35.287 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-17T09:39:35.287 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-17T09:39:35.287 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-17T09:39:35.287 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-17T09:39:35.287 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 59193 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-04-17T09:39:35.287 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 27849 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-mux.ko 2024-04-17T09:39:35.287 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-smbus.ko 2024-04-17T09:39:35.287 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-stub.ko 2024-04-17T09:39:35.287 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/muxes/ 2024-04-17T09:39:35.288 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-04-17T09:39:35.288 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-04-17T09:39:35.288 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 36569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-04-17T09:39:35.288 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ 2024-04-17T09:39:35.288 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ 2024-04-17T09:39:35.288 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 329697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-17T09:39:35.288 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 1119913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_core.ko 2024-04-17T09:39:35.288 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 83905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-17T09:39:35.288 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 408433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-17T09:39:35.288 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 120921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-17T09:39:35.288 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 347217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-17T09:39:35.288 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 84945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-17T09:39:35.288 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/ 2024-04-17T09:39:35.288 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4/ 2024-04-17T09:39:35.289 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 590929 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-17T09:39:35.289 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4/ 2024-04-17T09:39:35.289 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 514641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-17T09:39:35.289 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5/ 2024-04-17T09:39:35.289 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 911665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-17T09:39:35.289 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca/ 2024-04-17T09:39:35.289 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 375545 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-17T09:39:35.289 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/ocrdma/ 2024-04-17T09:39:35.289 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 197121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-04-17T09:39:35.289 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/usnic/ 2024-04-17T09:39:35.289 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 212553 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-04-17T09:39:35.289 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ 2024-04-17T09:39:35.289 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib/ 2024-04-17T09:39:35.289 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 334401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-17T09:39:35.289 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser/ 2024-04-17T09:39:35.290 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 120673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-17T09:39:35.290 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert/ 2024-04-17T09:39:35.290 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 132585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-17T09:39:35.290 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp/ 2024-04-17T09:39:35.290 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 157937 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-17T09:39:35.290 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt/ 2024-04-17T09:39:35.290 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 156673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-17T09:39:35.290 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/ 2024-04-17T09:39:35.290 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/evbug.ko 2024-04-17T09:39:35.290 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 27609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/ff-memless.ko 2024-04-17T09:39:35.290 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/ 2024-04-17T09:39:35.290 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-17T09:39:35.291 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-17T09:39:35.291 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 50713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/gameport.ko 2024-04-17T09:39:35.291 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 49441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/joydev.ko 2024-04-17T09:39:35.291 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/ 2024-04-17T09:39:35.291 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-04-17T09:39:35.291 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-04-17T09:39:35.291 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 35665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-04-17T09:39:35.291 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-04-17T09:39:35.291 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 45473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-17T09:39:35.291 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-04-17T09:39:35.291 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/lkkbd.ko 2024-04-17T09:39:35.291 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 31857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/lm8323.ko 2024-04-17T09:39:35.291 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/lm8333.ko 2024-04-17T09:39:35.291 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-04-17T09:39:35.292 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-04-17T09:39:35.292 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-04-17T09:39:35.292 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-04-17T09:39:35.292 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/newtonkbd.ko 2024-04-17T09:39:35.292 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-04-17T09:39:35.292 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/qt1070.ko 2024-04-17T09:39:35.292 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/qt2160.ko 2024-04-17T09:39:35.292 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-04-17T09:39:35.292 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/stowaway.ko 2024-04-17T09:39:35.292 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/sunkbd.ko 2024-04-17T09:39:35.292 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-04-17T09:39:35.292 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-04-17T09:39:35.292 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-04-17T09:39:35.293 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/xtkbd.ko 2024-04-17T09:39:35.293 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/matrix-keymap.ko 2024-04-17T09:39:35.293 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/ 2024-04-17T09:39:35.293 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 40617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/appletouch.ko 2024-04-17T09:39:35.293 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 39553 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/bcm5974.ko 2024-04-17T09:39:35.293 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 160017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/cyapatp.ko 2024-04-17T09:39:35.293 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 114273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-17T09:39:35.293 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/gpio_mouse.ko 2024-04-17T09:39:35.293 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 428185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/psmouse.ko 2024-04-17T09:39:35.293 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16937 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/sermouse.ko 2024-04-17T09:39:35.293 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 27865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-17T09:39:35.293 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 27193 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-17T09:39:35.293 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18809 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-17T09:39:35.293 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4/ 2024-04-17T09:39:35.294 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 140969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-17T09:39:35.294 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/ 2024-04-17T09:39:35.294 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/altera_ps2.ko 2024-04-17T09:39:35.294 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/arc_ps2.ko 2024-04-17T09:39:35.294 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/ct82c710.ko 2024-04-17T09:39:35.294 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-17T09:39:35.294 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18393 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/parkbd.ko 2024-04-17T09:39:35.294 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/pcips2.ko 2024-04-17T09:39:35.294 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/ps2mult.ko 2024-04-17T09:39:35.294 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/serio_raw.ko 2024-04-17T09:39:35.294 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/serport.ko 2024-04-17T09:39:35.294 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/sparse-keymap.ko 2024-04-17T09:39:35.294 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/ 2024-04-17T09:39:35.294 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-88pm860x.ko 2024-04-17T09:39:35.295 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-adp5520.ko 2024-04-17T09:39:35.295 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 77265 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-bd2802.ko 2024-04-17T09:39:35.295 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 31097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-blinkm.ko 2024-04-17T09:39:35.295 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11897 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-da903x.ko 2024-04-17T09:39:35.295 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-da9052.ko 2024-04-17T09:39:35.295 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-gpio.ko 2024-04-17T09:39:35.295 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm3530.ko 2024-04-17T09:39:35.295 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 36705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm3533.ko 2024-04-17T09:39:35.295 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm355x.ko 2024-04-17T09:39:35.295 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm3642.ko 2024-04-17T09:39:35.295 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lp3944.ko 2024-04-17T09:39:35.295 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12809 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lp8788.ko 2024-04-17T09:39:35.295 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-max8997.ko 2024-04-17T09:39:35.296 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-mc13783.ko 2024-04-17T09:39:35.296 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-menf21bmc.ko 2024-04-17T09:39:35.296 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-pca9532.ko 2024-04-17T09:39:35.296 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23809 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-pca955x.ko 2024-04-17T09:39:35.296 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-pca963x.ko 2024-04-17T09:39:35.296 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 38113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-ss4200.ko 2024-04-17T09:39:35.296 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-tca6507.ko 2024-04-17T09:39:35.296 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-wm831x-status.ko 2024-04-17T09:39:35.296 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-wm8350.ko 2024-04-17T09:39:35.296 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ 2024-04-17T09:39:35.296 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-17T09:39:35.296 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-17T09:39:35.296 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-17T09:39:35.296 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 8641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-17T09:39:35.297 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-17T09:39:35.297 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16601 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-17T09:39:35.297 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-17T09:39:35.297 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-17T09:39:35.297 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-17T09:39:35.297 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/ 2024-04-17T09:39:35.297 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache/ 2024-04-17T09:39:35.297 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 790753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache/bcache.ko 2024-04-17T09:39:35.297 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 49081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-bio-prison.ko 2024-04-17T09:39:35.297 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 116825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-bufio.ko 2024-04-17T09:39:35.297 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 58281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache-smq.ko 2024-04-17T09:39:35.297 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 199041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache.ko 2024-04-17T09:39:35.297 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 134945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-crypt.ko 2024-04-17T09:39:35.297 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28897 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-delay.ko 2024-04-17T09:39:35.298 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 67497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-era.ko 2024-04-17T09:39:35.298 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-flakey.ko 2024-04-17T09:39:35.298 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 50457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-log-userspace.ko 2024-04-17T09:39:35.298 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 41145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-log.ko 2024-04-17T09:39:35.298 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 60977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-mirror.ko 2024-04-17T09:39:35.298 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 97209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-multipath.ko 2024-04-17T09:39:35.298 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19809 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-queue-length.ko 2024-04-17T09:39:35.298 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 85689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-raid.ko 2024-04-17T09:39:35.298 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 44289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-region-hash.ko 2024-04-17T09:39:35.298 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-round-robin.ko 2024-04-17T09:39:35.298 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-service-time.ko 2024-04-17T09:39:35.298 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 139817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-snapshot.ko 2024-04-17T09:39:35.298 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-switch.ko 2024-04-17T09:39:35.299 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 234457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-thin-pool.ko 2024-04-17T09:39:35.299 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 66425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-verity.ko 2024-04-17T09:39:35.299 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-zero.ko 2024-04-17T09:39:35.299 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data/ 2024-04-17T09:39:35.299 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 264329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-17T09:39:35.299 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 43137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid0.ko 2024-04-17T09:39:35.299 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 125009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid1.ko 2024-04-17T09:39:35.299 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 157681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid10.ko 2024-04-17T09:39:35.299 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 444665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid456.ko 2024-04-17T09:39:35.299 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/ 2024-04-17T09:39:35.299 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/ 2024-04-17T09:39:35.299 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 214841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptbase.ko 2024-04-17T09:39:35.299 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 77985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptctl.ko 2024-04-17T09:39:35.299 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 54785 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptfc.ko 2024-04-17T09:39:35.300 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 44633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptlan.ko 2024-04-17T09:39:35.300 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 150761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptsas.ko 2024-04-17T09:39:35.300 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 97281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptscsih.ko 2024-04-17T09:39:35.300 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 63873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptspi.ko 2024-04-17T09:39:35.300 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/ 2024-04-17T09:39:35.300 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/88pm800.ko 2024-04-17T09:39:35.300 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/88pm805.ko 2024-04-17T09:39:35.300 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/88pm80x.ko 2024-04-17T09:39:35.300 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/arizona-i2c.ko 2024-04-17T09:39:35.300 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 148633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/arizona.ko 2024-04-17T09:39:35.300 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/bcm590xx.ko 2024-04-17T09:39:35.300 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/cros_ec_dev.ko 2024-04-17T09:39:35.300 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 41009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/dln2.ko 2024-04-17T09:39:35.301 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18601 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/janz-cmodio.ko 2024-04-17T09:39:35.301 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 59089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/kempld-core.ko 2024-04-17T09:39:35.301 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 31897 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lm3533-core.ko 2024-04-17T09:39:35.301 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-04-17T09:39:35.301 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lp3943.ko 2024-04-17T09:39:35.301 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 60497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_ich.ko 2024-04-17T09:39:35.301 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_sch.ko 2024-04-17T09:39:35.301 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/max8907.ko 2024-04-17T09:39:35.301 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 35537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/mc13xxx-core.ko 2024-04-17T09:39:35.301 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12937 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-04-17T09:39:35.301 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/menf21bmc.ko 2024-04-17T09:39:35.301 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/pcf50633-adc.ko 2024-04-17T09:39:35.301 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/pcf50633-gpio.ko 2024-04-17T09:39:35.301 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 42297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/pcf50633.ko 2024-04-17T09:39:35.302 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-04-17T09:39:35.302 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/retu-mfd.ko 2024-04-17T09:39:35.302 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 99425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/si476x-core.ko 2024-04-17T09:39:35.302 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 58057 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/sm501.ko 2024-04-17T09:39:35.302 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 60681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/tps65010.ko 2024-04-17T09:39:35.302 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11785 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/tps6507x.ko 2024-04-17T09:39:35.302 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/viperboard.ko 2024-04-17T09:39:35.302 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/vx855.ko 2024-04-17T09:39:35.302 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/wl1273-core.ko 2024-04-17T09:39:35.302 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 52361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/wm8994.ko 2024-04-17T09:39:35.302 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/ 2024-04-17T09:39:35.302 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-04-17T09:39:35.302 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 64265 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/ad525x_dpot.ko 2024-04-17T09:39:35.303 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl/ 2024-04-17T09:39:35.303 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 57521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-17T09:39:35.303 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds9802als.ko 2024-04-17T09:39:35.303 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 47689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds990x.ko 2024-04-17T09:39:35.303 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 58633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/bh1770glc.ko 2024-04-17T09:39:35.303 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/c2port/ 2024-04-17T09:39:35.303 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-04-17T09:39:35.303 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 47449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/c2port/core.ko 2024-04-17T09:39:35.303 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710/ 2024-04-17T09:39:35.303 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710/cb710.ko 2024-04-17T09:39:35.303 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/ds1682.ko 2024-04-17T09:39:35.303 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/dummy-irq.ko 2024-04-17T09:39:35.303 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/echo/ 2024-04-17T09:39:35.303 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/echo/echo.ko 2024-04-17T09:39:35.303 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/ 2024-04-17T09:39:35.304 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 43041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/at24.ko 2024-04-17T09:39:35.304 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-17T09:39:35.304 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/max6875.ko 2024-04-17T09:39:35.304 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 41185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/enclosure.ko 2024-04-17T09:39:35.304 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/genwqe/ 2024-04-17T09:39:35.304 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 191649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-04-17T09:39:35.304 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/hmc6352.ko 2024-04-17T09:39:35.304 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 37761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/hpilo.ko 2024-04-17T09:39:35.304 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/ibmasm/ 2024-04-17T09:39:35.304 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 92345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-04-17T09:39:35.304 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/ics932s401.ko 2024-04-17T09:39:35.304 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29003.ko 2024-04-17T09:39:35.304 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29020.ko 2024-04-17T09:39:35.304 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/ 2024-04-17T09:39:35.305 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 54865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-17T09:39:35.305 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-17T09:39:35.305 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/ 2024-04-17T09:39:35.305 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 154169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-me.ko 2024-04-17T09:39:35.305 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 69809 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-txe.ko 2024-04-17T09:39:35.305 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 389033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei.ko 2024-04-17T09:39:35.305 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 37961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/phantom.ko 2024-04-17T09:39:35.305 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/ti-st/ 2024-04-17T09:39:35.305 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 91145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/ti-st/st_drv.ko 2024-04-17T09:39:35.305 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_7xx1.ko 2024-04-17T09:39:35.305 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 34457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_core.ko 2024-04-17T09:39:35.305 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tsl2550.ko 2024-04-17T09:39:35.305 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 55793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_balloon.ko 2024-04-17T09:39:35.305 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci/ 2024-04-17T09:39:35.306 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 234905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-17T09:39:35.306 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ 2024-04-17T09:39:35.306 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding/ 2024-04-17T09:39:35.306 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 620961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding/bonding.ko 2024-04-17T09:39:35.306 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/ 2024-04-17T09:39:35.306 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53/ 2024-04-17T09:39:35.306 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 139817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-17T09:39:35.306 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 49545 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-17T09:39:35.306 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-17T09:39:35.306 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx/ 2024-04-17T09:39:35.306 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 587065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-17T09:39:35.306 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19553 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dummy.ko 2024-04-17T09:39:35.306 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/eql.ko 2024-04-17T09:39:35.307 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ 2024-04-17T09:39:35.307 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/ 2024-04-17T09:39:35.307 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 53753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-17T09:39:35.307 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 49201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-17T09:39:35.307 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 158649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-17T09:39:35.307 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 85953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-17T09:39:35.307 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/ 2024-04-17T09:39:35.307 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 44161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-17T09:39:35.307 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 73865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-17T09:39:35.307 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-17T09:39:35.307 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 109521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-17T09:39:35.307 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec/ 2024-04-17T09:39:35.307 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 64705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-17T09:39:35.307 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere/ 2024-04-17T09:39:35.308 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 84329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-17T09:39:35.308 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon/ 2024-04-17T09:39:35.308 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 70737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-17T09:39:35.308 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera/ 2024-04-17T09:39:35.308 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 52705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-17T09:39:35.308 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/ 2024-04-17T09:39:35.308 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 65857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-17T09:39:35.308 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 56057 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-17T09:39:35.308 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 115881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-17T09:39:35.308 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/ 2024-04-17T09:39:35.308 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx/ 2024-04-17T09:39:35.308 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 124497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-17T09:39:35.308 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-04-17T09:39:35.308 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 128409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-17T09:39:35.309 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-04-17T09:39:35.309 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 110345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-17T09:39:35.309 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/ 2024-04-17T09:39:35.309 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 101857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-17T09:39:35.309 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 88953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-17T09:39:35.309 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/ 2024-04-17T09:39:35.309 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 105633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-17T09:39:35.309 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 228985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-17T09:39:35.309 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-04-17T09:39:35.309 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 1733353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-17T09:39:35.309 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 167185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-17T09:39:35.309 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet/ 2024-04-17T09:39:35.309 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 161417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-17T09:39:35.309 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 440625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-17T09:39:35.310 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/ 2024-04-17T09:39:35.310 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna/ 2024-04-17T09:39:35.310 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 432201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-17T09:39:35.310 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/ 2024-04-17T09:39:35.310 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-04-17T09:39:35.310 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 257201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-17T09:39:35.376 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-04-17T09:39:35.376 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 446177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-17T09:39:35.376 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-04-17T09:39:35.377 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 1311433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-17T09:39:35.377 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-04-17T09:39:35.377 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 206689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-17T09:39:35.377 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-04-17T09:39:35.377 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 40161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-17T09:39:35.377 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/ 2024-04-17T09:39:35.377 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic/ 2024-04-17T09:39:35.377 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 269313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-17T09:39:35.377 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/ 2024-04-17T09:39:35.377 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/ 2024-04-17T09:39:35.377 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 85385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-17T09:39:35.377 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 77457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-17T09:39:35.377 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 169089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-17T09:39:35.377 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 65913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-17T09:39:35.378 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 76209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-17T09:39:35.378 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 38065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-17T09:39:35.378 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/ 2024-04-17T09:39:35.378 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 75945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-17T09:39:35.378 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 78705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-17T09:39:35.378 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 52313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dnet.ko 2024-04-17T09:39:35.378 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 30409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ec_bhf.ko 2024-04-17T09:39:35.378 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/ 2024-04-17T09:39:35.378 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet/ 2024-04-17T09:39:35.378 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 356377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-17T09:39:35.378 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 61737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ethoc.ko 2024-04-17T09:39:35.378 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 63185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/fealnx.ko 2024-04-17T09:39:35.378 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/fujitsu/ 2024-04-17T09:39:35.378 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 51513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-04-17T09:39:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ 2024-04-17T09:39:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 115569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-17T09:39:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000/ 2024-04-17T09:39:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 366217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-17T09:39:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e/ 2024-04-17T09:39:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 755793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-17T09:39:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k/ 2024-04-17T09:39:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 324577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-17T09:39:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e/ 2024-04-17T09:39:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 1196297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-17T09:39:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf/ 2024-04-17T09:39:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 362593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-17T09:39:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb/ 2024-04-17T09:39:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 671801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-17T09:39:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf/ 2024-04-17T09:39:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 121537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-17T09:39:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-04-17T09:39:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 966705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-17T09:39:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-04-17T09:39:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 193585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-17T09:39:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 108633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/jme.ko 2024-04-17T09:39:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/ 2024-04-17T09:39:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-17T09:39:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 126881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-17T09:39:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 150777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-17T09:39:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/ 2024-04-17T09:39:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-04-17T09:39:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 905033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-17T09:39:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 344145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-17T09:39:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-04-17T09:39:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-04-17T09:39:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 1412417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-17T09:39:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ 2024-04-17T09:39:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 65913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-04-17T09:39:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 54497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-04-17T09:39:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-04-17T09:39:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 117353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-17T09:39:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/ 2024-04-17T09:39:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-04-17T09:39:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 131425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-17T09:39:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/ 2024-04-17T09:39:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 91465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-17T09:39:35.382 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 56689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-17T09:39:35.382 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion/ 2024-04-17T09:39:35.382 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 181081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-17T09:39:35.382 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia/ 2024-04-17T09:39:35.382 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 171417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-17T09:39:35.382 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/ 2024-04-17T09:39:35.382 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 58249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-17T09:39:35.382 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 50121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-17T09:39:35.382 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/ 2024-04-17T09:39:35.382 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-04-17T09:39:35.382 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 300817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-17T09:39:35.382 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 105689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-17T09:39:35.382 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-04-17T09:39:35.383 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 885129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-17T09:39:35.383 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc/ 2024-04-17T09:39:35.383 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 52305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-17T09:39:35.383 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/ 2024-04-17T09:39:35.383 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 81641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-17T09:39:35.383 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 100689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-17T09:39:35.383 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 34073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-17T09:39:35.383 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 264161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-17T09:39:35.383 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/samsung/ 2024-04-17T09:39:35.383 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-04-17T09:39:35.383 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 142449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-04-17T09:39:35.383 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc/ 2024-04-17T09:39:35.383 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 1350385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-17T09:39:35.383 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan/ 2024-04-17T09:39:35.383 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 57953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-17T09:39:35.383 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/ 2024-04-17T09:39:35.384 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 66505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-17T09:39:35.384 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 80065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-17T09:39:35.384 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/ 2024-04-17T09:39:35.384 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 74289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-17T09:39:35.384 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 79457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-17T09:39:35.384 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 100785 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-17T09:39:35.384 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 83441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-17T09:39:35.384 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/ 2024-04-17T09:39:35.384 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-04-17T09:39:35.384 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-04-17T09:39:35.384 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 46985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-17T09:39:35.384 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-04-17T09:39:35.384 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 644537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-17T09:39:35.384 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/ 2024-04-17T09:39:35.384 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 128497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-17T09:39:35.385 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 207993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-17T09:39:35.385 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 82633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-17T09:39:35.385 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 91353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-17T09:39:35.385 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti/ 2024-04-17T09:39:35.385 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 59865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-17T09:39:35.385 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti/ 2024-04-17T09:39:35.385 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 110729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-17T09:39:35.385 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/ 2024-04-17T09:39:35.385 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 98689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-17T09:39:35.385 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 102513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-17T09:39:35.385 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/ 2024-04-17T09:39:35.385 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 61497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-17T09:39:35.385 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 40609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-17T09:39:35.385 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom/ 2024-04-17T09:39:35.385 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 69913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-17T09:39:35.386 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/fddi/ 2024-04-17T09:39:35.386 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 49449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/fddi/defxx.ko 2024-04-17T09:39:35.386 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/fddi/skfp/ 2024-04-17T09:39:35.386 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 275801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/fddi/skfp/skfp.ko 2024-04-17T09:39:35.386 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 91105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/geneve.ko 2024-04-17T09:39:35.386 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv/ 2024-04-17T09:39:35.386 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 290441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-17T09:39:35.386 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154/ 2024-04-17T09:39:35.386 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-17T09:39:35.386 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 30001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ifb.ko 2024-04-17T09:39:35.386 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan/ 2024-04-17T09:39:35.386 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 108329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-17T09:39:35.386 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 78065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvlan.ko 2024-04-17T09:39:35.387 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvtap.ko 2024-04-17T09:39:35.387 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/ 2024-04-17T09:39:35.387 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-17T09:39:35.387 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-17T09:39:35.387 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-gpio.ko 2024-04-17T09:39:35.387 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-17T09:39:35.387 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio.ko 2024-04-17T09:39:35.387 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mii.ko 2024-04-17T09:39:35.387 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 62681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/netconsole.ko 2024-04-17T09:39:35.387 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/nlmon.ko 2024-04-17T09:39:35.387 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/ 2024-04-17T09:39:35.387 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-17T09:39:35.387 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 48313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-17T09:39:35.387 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/ 2024-04-17T09:39:35.387 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/amd.ko 2024-04-17T09:39:35.388 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 61985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-17T09:39:35.388 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 52865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-17T09:39:35.388 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm87xx.ko 2024-04-17T09:39:35.388 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 53409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/broadcom.ko 2024-04-17T09:39:35.388 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/cicada.ko 2024-04-17T09:39:35.388 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/davicom.ko 2024-04-17T09:39:35.388 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/et1011c.ko 2024-04-17T09:39:35.388 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 31313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/icplus.ko 2024-04-17T09:39:35.388 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/lxt.ko 2024-04-17T09:39:35.388 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 103377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/marvell.ko 2024-04-17T09:39:35.388 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 96025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/micrel.ko 2024-04-17T09:39:35.388 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/national.ko 2024-04-17T09:39:35.388 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 121105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/phylink.ko 2024-04-17T09:39:35.388 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/qsemi.ko 2024-04-17T09:39:35.388 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 62113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/realtek.ko 2024-04-17T09:39:35.389 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 35513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/smsc.ko 2024-04-17T09:39:35.389 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/ste10Xp.ko 2024-04-17T09:39:35.389 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/vitesse.ko 2024-04-17T09:39:35.389 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/plip/ 2024-04-17T09:39:35.389 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 55353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/plip/plip.ko 2024-04-17T09:39:35.389 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ 2024-04-17T09:39:35.389 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-17T09:39:35.389 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 36273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_async.ko 2024-04-17T09:39:35.389 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-17T09:39:35.389 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-17T09:39:35.389 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 35369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-17T09:39:35.389 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 53233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppoe.ko 2024-04-17T09:39:35.389 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppox.ko 2024-04-17T09:39:35.389 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 40081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pptp.ko 2024-04-17T09:39:35.389 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 35937 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/sb1000.ko 2024-04-17T09:39:35.390 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/ 2024-04-17T09:39:35.390 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 52073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slip.ko 2024-04-17T09:39:35.390 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 42409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/sungem_phy.ko 2024-04-17T09:39:35.390 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 80161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tap.ko 2024-04-17T09:39:35.390 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/ 2024-04-17T09:39:35.390 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 137321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team.ko 2024-04-17T09:39:35.390 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-17T09:39:35.390 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-17T09:39:35.390 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 36209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-17T09:39:35.390 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_random.ko 2024-04-17T09:39:35.390 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-17T09:39:35.390 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 108089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/veth.ko 2024-04-17T09:39:35.390 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3/ 2024-04-17T09:39:35.391 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 176497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-17T09:39:35.391 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan/ 2024-04-17T09:39:35.391 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 383593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan/vxlan.ko 2024-04-17T09:39:35.391 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback/ 2024-04-17T09:39:35.391 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 175537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-17T09:39:35.391 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb/ 2024-04-17T09:39:35.391 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb/ntb.ko 2024-04-17T09:39:35.391 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/ 2024-04-17T09:39:35.391 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/ 2024-04-17T09:39:35.391 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 335857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme-core.ko 2024-04-17T09:39:35.391 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 126553 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme.ko 2024-04-17T09:39:35.391 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/ 2024-04-17T09:39:35.391 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 168145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport.ko 2024-04-17T09:39:35.391 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_cs.ko 2024-04-17T09:39:35.392 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 103889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_pc.ko 2024-04-17T09:39:35.392 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 30313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_serial.ko 2024-04-17T09:39:35.392 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/ 2024-04-17T09:39:35.392 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/ 2024-04-17T09:39:35.392 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-17T09:39:35.392 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-04-17T09:39:35.392 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-04-17T09:39:35.392 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pci-stub.ko 2024-04-17T09:39:35.392 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 55329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/xen-pcifront.ko 2024-04-17T09:39:35.392 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/ 2024-04-17T09:39:35.392 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 31529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/i82092.ko 2024-04-17T09:39:35.392 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 179961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pcmcia.ko 2024-04-17T09:39:35.392 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 78161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pcmcia_core.ko 2024-04-17T09:39:35.392 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 41945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-04-17T09:39:35.392 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 40457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pd6729.ko 2024-04-17T09:39:35.393 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 133753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-17T09:39:35.393 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/phy/ 2024-04-17T09:39:35.393 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/phy/broadcom/ 2024-04-17T09:39:35.393 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-04-17T09:39:35.393 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/phy/samsung/ 2024-04-17T09:39:35.393 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-04-17T09:39:35.393 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/ 2024-04-17T09:39:35.393 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel/ 2024-04-17T09:39:35.393 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 78193 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-17T09:39:35.393 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/ 2024-04-17T09:39:35.393 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/chrome/ 2024-04-17T09:39:35.393 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 61041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-04-17T09:39:35.393 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29657 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/chrome/cros_ec.ko 2024-04-17T09:39:35.393 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-04-17T09:39:35.394 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 30217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-04-17T09:39:35.394 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 31801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-04-17T09:39:35.394 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-04-17T09:39:35.394 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-04-17T09:39:35.394 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/ 2024-04-17T09:39:35.394 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 87481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-17T09:39:35.394 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 44105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acerhdf.ko 2024-04-17T09:39:35.394 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 48257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-17T09:39:35.394 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 73865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-17T09:39:35.394 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-17T09:39:35.394 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 164769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-17T09:39:35.394 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 46185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-17T09:39:35.394 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 62073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-17T09:39:35.394 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-17T09:39:35.394 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 51881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-17T09:39:35.395 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-17T09:39:35.395 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 42761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/hdaps.ko 2024-04-17T09:39:35.395 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/ibm_rtl.ko 2024-04-17T09:39:35.395 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel/ 2024-04-17T09:39:35.395 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14193 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-17T09:39:35.395 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-04-17T09:39:35.395 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 54001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel_ips.ko 2024-04-17T09:39:35.395 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-17T09:39:35.395 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-17T09:39:35.395 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 43753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-17T09:39:35.395 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 51897 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-17T09:39:35.395 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-17T09:39:35.395 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-17T09:39:35.395 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-17T09:39:35.395 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-17T09:39:35.396 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11657 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-17T09:39:35.396 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 61417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi.ko 2024-04-17T09:39:35.396 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/ 2024-04-17T09:39:35.396 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/ 2024-04-17T09:39:35.396 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 36513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/88pm860x_battery.ko 2024-04-17T09:39:35.396 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 40425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/88pm860x_charger.ko 2024-04-17T09:39:35.396 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 68833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/bq2415x_charger.ko 2024-04-17T09:39:35.396 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 73073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/bq24190_charger.ko 2024-04-17T09:39:35.396 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/bq24735-charger.ko 2024-04-17T09:39:35.396 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-04-17T09:39:35.396 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/da9030_battery.ko 2024-04-17T09:39:35.396 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/da9052-battery.ko 2024-04-17T09:39:35.396 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 34489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/ds2760_battery.ko 2024-04-17T09:39:35.396 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/ds2780_battery.ko 2024-04-17T09:39:35.397 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28809 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/ds2781_battery.ko 2024-04-17T09:39:35.397 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28657 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/ds2782_battery.ko 2024-04-17T09:39:35.397 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/gpio-charger.ko 2024-04-17T09:39:35.397 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/isp1704_charger.ko 2024-04-17T09:39:35.397 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/lp8727_charger.ko 2024-04-17T09:39:35.397 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/max14577_charger.ko 2024-04-17T09:39:35.397 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 37833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/max17042_battery.ko 2024-04-17T09:39:35.397 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/max8903_charger.ko 2024-04-17T09:39:35.397 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 27689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/max8925_power.ko 2024-04-17T09:39:35.397 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/pcf50633-charger.ko 2024-04-17T09:39:35.397 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 44841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/sbs-battery.ko 2024-04-17T09:39:35.397 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 34905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/test_power.ko 2024-04-17T09:39:35.397 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/tps65090-charger.ko 2024-04-17T09:39:35.397 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/wm831x_backup.ko 2024-04-17T09:39:35.397 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/wm831x_power.ko 2024-04-17T09:39:35.397 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 27441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/power/supply/wm8350_power.ko 2024-04-17T09:39:35.398 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/ 2024-04-17T09:39:35.398 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 73297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-17T09:39:35.398 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-17T09:39:35.398 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/ 2024-04-17T09:39:35.398 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/ 2024-04-17T09:39:35.398 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-17T09:39:35.398 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-17T09:39:35.398 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps_parport.ko 2024-04-17T09:39:35.398 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 49817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/pps_core.ko 2024-04-17T09:39:35.398 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ 2024-04-17T09:39:35.398 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 91393 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp.ko 2024-04-17T09:39:35.398 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp_kvm.ko 2024-04-17T09:39:35.398 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/ 2024-04-17T09:39:35.399 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/ 2024-04-17T09:39:35.399 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl/ 2024-04-17T09:39:35.399 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 67513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-17T09:39:35.399 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 47137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/fmpm.ko 2024-04-17T09:39:35.399 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/ 2024-04-17T09:39:35.399 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 27593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-88pm80x.ko 2024-04-17T09:39:35.399 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-88pm860x.ko 2024-04-17T09:39:35.399 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-17T09:39:35.399 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-da9052.ko 2024-04-17T09:39:35.399 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-da9055.ko 2024-04-17T09:39:35.399 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-da9063.ko 2024-04-17T09:39:35.399 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-17T09:39:35.399 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 82225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-17T09:39:35.399 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-17T09:39:35.399 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-17T09:39:35.399 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-17T09:39:35.400 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16057 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-17T09:39:35.400 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-17T09:39:35.400 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-17T09:39:35.400 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-17T09:39:35.400 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-em3027.ko 2024-04-17T09:39:35.400 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-17T09:39:35.400 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-17T09:39:35.400 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 44129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-17T09:39:35.400 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-lp8788.ko 2024-04-17T09:39:35.400 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 46913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-17T09:39:35.400 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-17T09:39:35.400 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-17T09:39:35.400 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t86.ko 2024-04-17T09:39:35.400 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max6900.ko 2024-04-17T09:39:35.400 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max8907.ko 2024-04-17T09:39:35.401 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max8925.ko 2024-04-17T09:39:35.401 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 31337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max8997.ko 2024-04-17T09:39:35.401 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max8998.ko 2024-04-17T09:39:35.401 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-04-17T09:39:35.401 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-17T09:39:35.401 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-palmas.ko 2024-04-17T09:39:35.401 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 56777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-17T09:39:35.401 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21601 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf50633.ko 2024-04-17T09:39:35.401 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 32657 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-17T09:39:35.401 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-17T09:39:35.401 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 36065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-17T09:39:35.401 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-17T09:39:35.401 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rc5t583.ko 2024-04-17T09:39:35.401 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-17T09:39:35.401 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 40081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-17T09:39:35.401 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 34529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-17T09:39:35.402 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-17T09:39:35.402 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-17T09:39:35.402 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-s35390a.ko 2024-04-17T09:39:35.402 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-17T09:39:35.402 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-tps6586x.ko 2024-04-17T09:39:35.402 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-tps65910.ko 2024-04-17T09:39:35.402 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-wm831x.ko 2024-04-17T09:39:35.402 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-wm8350.ko 2024-04-17T09:39:35.402 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-x1205.ko 2024-04-17T09:39:35.402 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ 2024-04-17T09:39:35.402 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 90377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-9xxx.ko 2024-04-17T09:39:35.402 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 69593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-sas.ko 2024-04-17T09:39:35.402 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 65945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-xxxx.ko 2024-04-17T09:39:35.402 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 164569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/BusLogic.ko 2024-04-17T09:39:35.403 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 31617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/a100u2w.ko 2024-04-17T09:39:35.403 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid/ 2024-04-17T09:39:35.403 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 340721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-17T09:39:35.403 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 120337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/advansys.ko 2024-04-17T09:39:35.403 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/ 2024-04-17T09:39:35.403 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 468081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-17T09:39:35.403 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 380673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-17T09:39:35.403 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic94xx/ 2024-04-17T09:39:35.403 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 243705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-04-17T09:39:35.403 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/am53c974.ko 2024-04-17T09:39:35.403 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr/ 2024-04-17T09:39:35.403 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 135617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-17T09:39:35.403 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 44289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/atp870u.ko 2024-04-17T09:39:35.403 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi/ 2024-04-17T09:39:35.403 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 304993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-17T09:39:35.403 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa/ 2024-04-17T09:39:35.404 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 1527113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa/bfa.ko 2024-04-17T09:39:35.404 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc/ 2024-04-17T09:39:35.404 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 297801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-17T09:39:35.404 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i/ 2024-04-17T09:39:35.404 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 144401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-17T09:39:35.404 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 47457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ch.ko 2024-04-17T09:39:35.404 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor/ 2024-04-17T09:39:35.404 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 390713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-17T09:39:35.404 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/ 2024-04-17T09:39:35.404 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-04-17T09:39:35.404 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 88881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-17T09:39:35.404 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-04-17T09:39:35.404 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 117481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-17T09:39:35.404 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 173017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-17T09:39:35.405 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 91569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dc395x.ko 2024-04-17T09:39:35.405 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 45401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dmx3191d.ko 2024-04-17T09:39:35.405 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r/ 2024-04-17T09:39:35.405 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 267577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-17T09:39:35.405 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 78969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esp_scsi.ko 2024-04-17T09:39:35.405 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/ 2024-04-17T09:39:35.405 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 95329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-17T09:39:35.405 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 194017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-17T09:39:35.405 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fdomain.ko 2024-04-17T09:39:35.405 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic/ 2024-04-17T09:39:35.405 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 366529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic/fnic.ko 2024-04-17T09:39:35.405 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 251505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hpsa.ko 2024-04-17T09:39:35.405 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 65761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hptiop.ko 2024-04-17T09:39:35.405 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 57097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hv_storvsc.ko 2024-04-17T09:39:35.405 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 44689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/imm.ko 2024-04-17T09:39:35.406 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 55153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/initio.ko 2024-04-17T09:39:35.406 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 327841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ipr.ko 2024-04-17T09:39:35.406 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 129785 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ips.ko 2024-04-17T09:39:35.406 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci/ 2024-04-17T09:39:35.406 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 397785 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci/isci.ko 2024-04-17T09:39:35.406 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 32017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-17T09:39:35.406 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 63121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-17T09:39:35.406 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc/ 2024-04-17T09:39:35.406 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 386849 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc/libfc.ko 2024-04-17T09:39:35.406 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 178177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi.ko 2024-04-17T09:39:35.406 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 64809 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-17T09:39:35.406 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas/ 2024-04-17T09:39:35.406 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 274009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas/libsas.ko 2024-04-17T09:39:35.406 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc/ 2024-04-17T09:39:35.407 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 2736001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-17T09:39:35.407 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/ 2024-04-17T09:39:35.407 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 88529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-17T09:39:35.407 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 40145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-17T09:39:35.407 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 389177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-17T09:39:35.407 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 104769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid.ko 2024-04-17T09:39:35.407 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas/ 2024-04-17T09:39:35.407 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 836945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-17T09:39:35.407 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas/ 2024-04-17T09:39:35.407 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 158729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-17T09:39:35.407 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 96537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvumi.ko 2024-04-17T09:39:35.407 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pcmcia/ 2024-04-17T09:39:35.407 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 101625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-04-17T09:39:35.407 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-04-17T09:39:35.407 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-04-17T09:39:35.407 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28265 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-04-17T09:39:35.408 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001/ 2024-04-17T09:39:35.408 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 443745 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-17T09:39:35.408 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 152033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pmcraid.ko 2024-04-17T09:39:35.408 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 41625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ppa.ko 2024-04-17T09:39:35.408 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 66977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla1280.ko 2024-04-17T09:39:35.408 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/ 2024-04-17T09:39:35.408 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 2324809 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-17T09:39:35.408 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 103601 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-17T09:39:35.408 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx/ 2024-04-17T09:39:35.408 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 697337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-17T09:39:35.408 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qlogicfas408.ko 2024-04-17T09:39:35.408 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/raid_class.ko 2024-04-17T09:39:35.408 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 322865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_debug.ko 2024-04-17T09:39:35.408 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 219857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-17T09:39:35.409 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 357153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-17T09:39:35.409 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 121185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-17T09:39:35.409 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 93481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-17T09:39:35.409 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 55401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-17T09:39:35.409 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ses.ko 2024-04-17T09:39:35.409 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 138121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/st.ko 2024-04-17T09:39:35.409 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 57401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/stex.ko 2024-04-17T09:39:35.409 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2/ 2024-04-17T09:39:35.409 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 205025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-17T09:39:35.409 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 55425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/virtio_scsi.ko 2024-04-17T09:39:35.409 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 59185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-17T09:39:35.409 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 48777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/wd719x.ko 2024-04-17T09:39:35.409 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 59417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/xen-scsifront.ko 2024-04-17T09:39:35.409 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb/ 2024-04-17T09:39:35.409 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 174913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb/ssb.ko 2024-04-17T09:39:35.409 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/ 2024-04-17T09:39:35.410 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi/ 2024-04-17T09:39:35.410 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 920169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-17T09:39:35.410 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback/ 2024-04-17T09:39:35.410 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 60321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-17T09:39:35.410 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 48129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_file.ko 2024-04-17T09:39:35.410 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 54233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_iblock.ko 2024-04-17T09:39:35.410 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 1170033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_mod.ko 2024-04-17T09:39:35.410 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 51897 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_pscsi.ko 2024-04-17T09:39:35.410 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 149721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_user.ko 2024-04-17T09:39:35.410 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc/ 2024-04-17T09:39:35.410 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 90057 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-17T09:39:35.410 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/ 2024-04-17T09:39:35.410 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/ 2024-04-17T09:39:35.410 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/ 2024-04-17T09:39:35.410 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-17T09:39:35.411 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 35209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-17T09:39:35.411 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-17T09:39:35.411 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-17T09:39:35.411 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-17T09:39:35.411 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-17T09:39:35.411 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 36729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-17T09:39:35.411 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-17T09:39:35.411 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16745 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-17T09:39:35.411 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12929 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-17T09:39:35.411 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-17T09:39:35.411 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-17T09:39:35.411 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 46065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-17T09:39:35.411 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-17T09:39:35.411 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-17T09:39:35.412 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 35913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-04-17T09:39:35.412 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-17T09:39:35.412 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12849 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-17T09:39:35.412 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 30577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-17T09:39:35.412 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ 2024-04-17T09:39:35.412 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless/ 2024-04-17T09:39:35.412 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 99313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-17T09:39:35.412 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 69729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/moxa.ko 2024-04-17T09:39:35.412 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 61649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/mxser.ko 2024-04-17T09:39:35.412 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 134401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_gsm.ko 2024-04-17T09:39:35.412 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 37905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_hdlc.ko 2024-04-17T09:39:35.412 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 78969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/nozomi.ko 2024-04-17T09:39:35.412 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/ 2024-04-17T09:39:35.412 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250/ 2024-04-17T09:39:35.412 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 44057 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-04-17T09:39:35.413 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 65121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-17T09:39:35.413 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-04-17T09:39:35.413 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/altera_uart.ko 2024-04-17T09:39:35.413 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/arc_uart.ko 2024-04-17T09:39:35.413 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 91665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-04-17T09:39:35.413 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm/ 2024-04-17T09:39:35.413 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 120577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-17T09:39:35.413 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 35161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/rp2.ko 2024-04-17T09:39:35.413 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 65641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/sc16is7xx.ko 2024-04-17T09:39:35.413 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/ 2024-04-17T09:39:35.413 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core/ 2024-04-17T09:39:35.413 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 693081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-17T09:39:35.413 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host/ 2024-04-17T09:39:35.413 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 34449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-17T09:39:35.414 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-04-17T09:39:35.414 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/ 2024-04-17T09:39:35.414 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 55617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio.ko 2024-04-17T09:39:35.414 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_aec.ko 2024-04-17T09:39:35.414 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_cif.ko 2024-04-17T09:39:35.414 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_dmem_genirq.ko 2024-04-17T09:39:35.414 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_mf624.ko 2024-04-17T09:39:35.414 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_netx.ko 2024-04-17T09:39:35.414 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_pci_generic.ko 2024-04-17T09:39:35.414 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-04-17T09:39:35.414 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_sercos3.ko 2024-04-17T09:39:35.414 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/ 2024-04-17T09:39:35.414 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/c67x00/ 2024-04-17T09:39:35.414 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 90321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/c67x00/c67x00.ko 2024-04-17T09:39:35.415 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/chipidea/ 2024-04-17T09:39:35.415 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 189209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-04-17T09:39:35.415 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-04-17T09:39:35.415 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-04-17T09:39:35.415 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-04-17T09:39:35.415 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-04-17T09:39:35.415 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/ 2024-04-17T09:39:35.415 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 101545 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-acm.ko 2024-04-17T09:39:35.415 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 63073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-17T09:39:35.415 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 56721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usblp.ko 2024-04-17T09:39:35.415 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 107417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usbtmc.ko 2024-04-17T09:39:35.415 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/common/ 2024-04-17T09:39:35.415 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/common/ulpi.ko 2024-04-17T09:39:35.415 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/dwc2/ 2024-04-17T09:39:35.415 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-04-17T09:39:35.415 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/dwc3/ 2024-04-17T09:39:35.416 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-04-17T09:39:35.416 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-04-17T09:39:35.416 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 574593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/dwc3/dwc3.ko 2024-04-17T09:39:35.416 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/ 2024-04-17T09:39:35.416 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/ 2024-04-17T09:39:35.416 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 72249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/u_ether.ko 2024-04-17T09:39:35.416 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 61953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/u_serial.ko 2024-04-17T09:39:35.416 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-04-17T09:39:35.416 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 49737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-04-17T09:39:35.416 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-04-17T09:39:35.416 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 43553 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-04-17T09:39:35.416 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 131345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-04-17T09:39:35.416 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 62321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-04-17T09:39:35.416 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 146617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-04-17T09:39:35.416 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 70689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-04-17T09:39:35.416 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-04-17T09:39:35.417 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 37649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-04-17T09:39:35.417 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 70017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-04-17T09:39:35.417 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 101249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-04-17T09:39:35.417 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-04-17T09:39:35.417 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 71257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-04-17T09:39:35.417 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 84937 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-04-17T09:39:35.417 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/ 2024-04-17T09:39:35.417 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-04-17T09:39:35.417 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-04-17T09:39:35.417 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-04-17T09:39:35.417 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-04-17T09:39:35.417 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-04-17T09:39:35.417 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-04-17T09:39:35.417 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-04-17T09:39:35.417 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-04-17T09:39:35.418 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 35689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-04-17T09:39:35.418 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-04-17T09:39:35.418 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-04-17T09:39:35.418 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 30345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-04-17T09:39:35.418 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 78073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-04-17T09:39:35.418 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-04-17T09:39:35.418 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 204049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/libcomposite.ko 2024-04-17T09:39:35.418 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/ 2024-04-17T09:39:35.418 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-04-17T09:39:35.418 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/bdc/ 2024-04-17T09:39:35.418 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 182561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-04-17T09:39:35.418 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 45545 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-04-17T09:39:35.418 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 66513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-04-17T09:39:35.418 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 85217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-04-17T09:39:35.418 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 69313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-04-17T09:39:35.419 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 90313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/net2272.ko 2024-04-17T09:39:35.419 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 90361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/net2280.ko 2024-04-17T09:39:35.419 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 94841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-04-17T09:39:35.419 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 80521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-04-17T09:39:35.419 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 87465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-04-17T09:39:35.419 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 95441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-04-17T09:39:35.419 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 195969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-04-17T09:39:35.419 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/ 2024-04-17T09:39:35.419 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/bcma-hcd.ko 2024-04-17T09:39:35.419 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 108321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/isp116x-hcd.ko 2024-04-17T09:39:35.419 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 104377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-04-17T09:39:35.419 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 102273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-04-17T09:39:35.419 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 60105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-17T09:39:35.419 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/sl811_cs.ko 2024-04-17T09:39:35.419 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/ssb-hcd.ko 2024-04-17T09:39:35.419 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 32553 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-04-17T09:39:35.420 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/ 2024-04-17T09:39:35.420 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 40409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/mdc800.ko 2024-04-17T09:39:35.420 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/microtek.ko 2024-04-17T09:39:35.420 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ 2024-04-17T09:39:35.420 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 47353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/adutux.ko 2024-04-17T09:39:35.420 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/appledisplay.ko 2024-04-17T09:39:35.420 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-04-17T09:39:35.420 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/cytherm.ko 2024-04-17T09:39:35.420 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ehset.ko 2024-04-17T09:39:35.420 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi26.ko 2024-04-17T09:39:35.420 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi62.ko 2024-04-17T09:39:35.420 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13057 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ezusb.ko 2024-04-17T09:39:35.420 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/idmouse.ko 2024-04-17T09:39:35.420 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 42729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/iowarrior.ko 2024-04-17T09:39:35.420 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-17T09:39:35.421 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 42241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ldusb.ko 2024-04-17T09:39:35.421 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 42233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/legousbtower.ko 2024-04-17T09:39:35.421 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/lvstest.ko 2024-04-17T09:39:35.421 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga/ 2024-04-17T09:39:35.421 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 85313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-17T09:39:35.421 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14601 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-17T09:39:35.421 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usb3503.ko 2024-04-17T09:39:35.421 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usblcd.ko 2024-04-17T09:39:35.421 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 27809 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-17T09:39:35.421 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 84265 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usbtest.ko 2024-04-17T09:39:35.421 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 52065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/uss720.ko 2024-04-17T09:39:35.421 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 32345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/yurex.ko 2024-04-17T09:39:35.421 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/mon/ 2024-04-17T09:39:35.421 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 91529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/mon/usbmon.ko 2024-04-17T09:39:35.422 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/musb/ 2024-04-17T09:39:35.422 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 396209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/musb/musb_hdrc.ko 2024-04-17T09:39:35.422 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/ 2024-04-17T09:39:35.422 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-generic.ko 2024-04-17T09:39:35.422 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-04-17T09:39:35.422 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-isp1301.ko 2024-04-17T09:39:35.422 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-tahvo.ko 2024-04-17T09:39:35.422 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ 2024-04-17T09:39:35.422 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/aircable.ko 2024-04-17T09:39:35.422 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 38209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ark3116.ko 2024-04-17T09:39:35.422 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 30681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-17T09:39:35.422 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 44329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ch341.ko 2024-04-17T09:39:35.422 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 79681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cp210x.ko 2024-04-17T09:39:35.422 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 35137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cyberjack.ko 2024-04-17T09:39:35.423 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 66873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-17T09:39:35.423 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 58329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-17T09:39:35.423 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/empeg.ko 2024-04-17T09:39:35.423 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 53009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/f81232.ko 2024-04-17T09:39:35.423 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 188609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-17T09:39:35.423 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 61489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-17T09:39:35.423 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 152305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-17T09:39:35.423 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 142833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_ti.ko 2024-04-17T09:39:35.423 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 69217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipaq.ko 2024-04-17T09:39:35.423 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipw.ko 2024-04-17T09:39:35.423 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ir-usb.ko 2024-04-17T09:39:35.423 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 66361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-17T09:39:35.423 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 122329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan.ko 2024-04-17T09:39:35.423 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 41297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-17T09:39:35.423 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-17T09:39:35.423 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 38049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-17T09:39:35.424 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 38041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mct_u232.ko 2024-04-17T09:39:35.424 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/metro-usb.ko 2024-04-17T09:39:35.424 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 79681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7720.ko 2024-04-17T09:39:35.424 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 89969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7840.ko 2024-04-17T09:39:35.424 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 65073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mxuport.ko 2024-04-17T09:39:35.424 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/navman.ko 2024-04-17T09:39:35.424 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/omninet.ko 2024-04-17T09:39:35.424 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/opticon.ko 2024-04-17T09:39:35.424 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 193857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/option.ko 2024-04-17T09:39:35.424 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 45337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/oti6858.ko 2024-04-17T09:39:35.424 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 61761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/pl2303.ko 2024-04-17T09:39:35.424 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcaux.ko 2024-04-17T09:39:35.424 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 39985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcserial.ko 2024-04-17T09:39:35.424 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 40529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/quatech2.ko 2024-04-17T09:39:35.424 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/safe_serial.ko 2024-04-17T09:39:35.424 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 59345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/sierra.ko 2024-04-17T09:39:35.425 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-17T09:39:35.425 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 34065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ssu100.ko 2024-04-17T09:39:35.425 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19601 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/symbolserial.ko 2024-04-17T09:39:35.425 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 70497 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-17T09:39:35.425 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-17T09:39:35.425 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_debug.ko 2024-04-17T09:39:35.425 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 39321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-17T09:39:35.425 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 151137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usbserial.ko 2024-04-17T09:39:35.425 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 32505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/visor.ko 2024-04-17T09:39:35.425 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 43577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/whiteheat.ko 2024-04-17T09:39:35.425 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/wishbone-serial.ko 2024-04-17T09:39:35.425 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-17T09:39:35.425 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ 2024-04-17T09:39:35.425 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 64049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/uas.ko 2024-04-17T09:39:35.425 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 30209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-17T09:39:35.426 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-17T09:39:35.426 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-17T09:39:35.426 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 41073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-17T09:39:35.426 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-17T09:39:35.426 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-17T09:39:35.426 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-17T09:39:35.426 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-karma.ko 2024-04-17T09:39:35.426 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-17T09:39:35.426 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-17T09:39:35.426 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33849 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-17T09:39:35.426 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-17T09:39:35.426 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 41785 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-17T09:39:35.426 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 201193 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/usb-storage.ko 2024-04-17T09:39:35.426 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/ 2024-04-17T09:39:35.426 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 76041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-17T09:39:35.427 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 85257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-17T09:39:35.427 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 122321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-17T09:39:35.427 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/ 2024-04-17T09:39:35.427 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/ 2024-04-17T09:39:35.427 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 206921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-17T09:39:35.427 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-17T09:39:35.427 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 134265 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio.ko 2024-04-17T09:39:35.427 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 93673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-17T09:39:35.427 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/ 2024-04-17T09:39:35.427 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 138721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost.ko 2024-04-17T09:39:35.427 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-17T09:39:35.427 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 71281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_net.ko 2024-04-17T09:39:35.427 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 107905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_scsi.ko 2024-04-17T09:39:35.427 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/ 2024-04-17T09:39:35.427 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/ 2024-04-17T09:39:35.427 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-04-17T09:39:35.428 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/aat2870_bl.ko 2024-04-17T09:39:35.428 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/adp5520_bl.ko 2024-04-17T09:39:35.428 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 41169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/adp8860_bl.ko 2024-04-17T09:39:35.428 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 51745 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/adp8870_bl.ko 2024-04-17T09:39:35.428 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/apple_bl.ko 2024-04-17T09:39:35.428 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/as3711_bl.ko 2024-04-17T09:39:35.428 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/bd6107.ko 2024-04-17T09:39:35.428 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/da903x_bl.ko 2024-04-17T09:39:35.428 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/da9052_bl.ko 2024-04-17T09:39:35.428 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13785 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/gpio_backlight.ko 2024-04-17T09:39:35.428 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/kb3886_bl.ko 2024-04-17T09:39:35.428 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lcd.ko 2024-04-17T09:39:35.428 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lm3533_bl.ko 2024-04-17T09:39:35.428 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23393 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lm3639_bl.ko 2024-04-17T09:39:35.428 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lv5207lp.ko 2024-04-17T09:39:35.428 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/max8925_bl.ko 2024-04-17T09:39:35.429 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/pandora_bl.ko 2024-04-17T09:39:35.429 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13057 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-04-17T09:39:35.429 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-17T09:39:35.429 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/wm831x_bl.ko 2024-04-17T09:39:35.429 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/ 2024-04-17T09:39:35.429 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ 2024-04-17T09:39:35.429 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2482.ko 2024-04-17T09:39:35.429 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 46801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2490.ko 2024-04-17T09:39:35.429 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/matrox_w1.ko 2024-04-17T09:39:35.429 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/w1-gpio.ko 2024-04-17T09:39:35.429 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/ 2024-04-17T09:39:35.429 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-17T09:39:35.429 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 27257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-17T09:39:35.429 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-17T09:39:35.430 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-17T09:39:35.430 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-17T09:39:35.430 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-17T09:39:35.430 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-17T09:39:35.430 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-17T09:39:35.430 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-17T09:39:35.430 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 8313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-17T09:39:35.430 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 70369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-17T09:39:35.430 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 129433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/wire.ko 2024-04-17T09:39:35.430 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ 2024-04-17T09:39:35.430 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/acquirewdt.ko 2024-04-17T09:39:35.430 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/advantechwdt.ko 2024-04-17T09:39:35.431 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-17T09:39:35.431 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 27401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-17T09:39:35.431 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/cpu5wdt.ko 2024-04-17T09:39:35.431 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/da9052_wdt.ko 2024-04-17T09:39:35.431 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/da9055_wdt.ko 2024-04-17T09:39:35.431 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/da9063_wdt.ko 2024-04-17T09:39:35.431 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/dw_wdt.ko 2024-04-17T09:39:35.431 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/eurotechwdt.ko 2024-04-17T09:39:35.431 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-17T09:39:35.431 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 32857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/hpwdt.ko 2024-04-17T09:39:35.431 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/i6300esb.ko 2024-04-17T09:39:35.431 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-17T09:39:35.431 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-17T09:39:35.431 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ib700wdt.ko 2024-04-17T09:39:35.431 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ibmasr.ko 2024-04-17T09:39:35.432 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-17T09:39:35.432 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-17T09:39:35.432 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it87_wdt.ko 2024-04-17T09:39:35.432 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/kempld_wdt.ko 2024-04-17T09:39:35.432 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/machzwd.ko 2024-04-17T09:39:35.432 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/mena21_wdt.ko 2024-04-17T09:39:35.432 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-04-17T09:39:35.432 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/nv_tco.ko 2024-04-17T09:39:35.432 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-04-17T09:39:35.432 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 27721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pc87413_wdt.ko 2024-04-17T09:39:35.432 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 46953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-17T09:39:35.432 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 37193 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-17T09:39:35.432 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/retu_wdt.ko 2024-04-17T09:39:35.432 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-04-17T09:39:35.432 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-04-17T09:39:35.432 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-17T09:39:35.433 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sc1200wdt.ko 2024-04-17T09:39:35.433 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 27489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-17T09:39:35.433 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-04-17T09:39:35.433 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/softdog.ko 2024-04-17T09:39:35.433 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-17T09:39:35.433 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/twl4030_wdt.ko 2024-04-17T09:39:35.433 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/via_wdt.ko 2024-04-17T09:39:35.433 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23393 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-17T09:39:35.433 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-17T09:39:35.433 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 32481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-17T09:39:35.433 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wafer5823wdt.ko 2024-04-17T09:39:35.433 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wdt_pci.ko 2024-04-17T09:39:35.433 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wm831x_wdt.ko 2024-04-17T09:39:35.433 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wm8350_wdt.ko 2024-04-17T09:39:35.433 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/xen_wdt.ko 2024-04-17T09:39:35.433 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/ 2024-04-17T09:39:35.434 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 30049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-evtchn.ko 2024-04-17T09:39:35.434 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 30649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntalloc.ko 2024-04-17T09:39:35.434 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 55713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntdev.ko 2024-04-17T09:39:35.434 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback/ 2024-04-17T09:39:35.434 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 213913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-17T09:39:35.434 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 47689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-privcmd.ko 2024-04-17T09:39:35.434 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 85713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-scsiback.ko 2024-04-17T09:39:35.434 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs/ 2024-04-17T09:39:35.434 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-17T09:39:35.434 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ 2024-04-17T09:39:35.434 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p/ 2024-04-17T09:39:35.434 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 192617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p/9p.ko 2024-04-17T09:39:35.434 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/adfs/ 2024-04-17T09:39:35.434 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 78153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/adfs/adfs.ko 2024-04-17T09:39:35.435 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs/ 2024-04-17T09:39:35.435 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 197849 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs/affs.ko 2024-04-17T09:39:35.435 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/afs/ 2024-04-17T09:39:35.435 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 1261113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/afs/kafs.ko 2024-04-17T09:39:35.435 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/autofs/ 2024-04-17T09:39:35.435 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 127377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/autofs/autofs4.ko 2024-04-17T09:39:35.435 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs/ 2024-04-17T09:39:35.435 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 80521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs/befs.ko 2024-04-17T09:39:35.435 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/bfs/ 2024-04-17T09:39:35.435 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 50473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/bfs/bfs.ko 2024-04-17T09:39:35.435 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 45561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/binfmt_misc.ko 2024-04-17T09:39:35.435 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs/ 2024-04-17T09:39:35.435 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 4489593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs/btrfs.ko 2024-04-17T09:39:35.435 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles/ 2024-04-17T09:39:35.435 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 376857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles/cachefiles.ko 2024-04-17T09:39:35.435 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph/ 2024-04-17T09:39:35.436 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 1586089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph/ceph.ko 2024-04-17T09:39:35.436 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda/ 2024-04-17T09:39:35.436 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 130073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda/coda.ko 2024-04-17T09:39:35.436 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/configfs/ 2024-04-17T09:39:35.436 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 139081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/configfs/configfs.ko 2024-04-17T09:39:35.436 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs/ 2024-04-17T09:39:35.436 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 35665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs/cramfs.ko 2024-04-17T09:39:35.436 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm/ 2024-04-17T09:39:35.436 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 813225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm/dlm.ko 2024-04-17T09:39:35.479 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/efs/ 2024-04-17T09:39:35.479 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 46409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/efs/efs.ko 2024-04-17T09:39:35.479 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs/ 2024-04-17T09:39:35.479 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 1906817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs/f2fs.ko 2024-04-17T09:39:35.479 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/ 2024-04-17T09:39:35.479 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 35137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/msdos.ko 2024-04-17T09:39:35.479 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/freevxfs/ 2024-04-17T09:39:35.479 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 52313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/freevxfs/freevxfs.ko 2024-04-17T09:39:35.479 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/ 2024-04-17T09:39:35.479 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/cuse.ko 2024-04-17T09:39:35.480 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2/ 2024-04-17T09:39:35.480 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 1486521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2/gfs2.ko 2024-04-17T09:39:35.480 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs/ 2024-04-17T09:39:35.480 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 183305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs/hfs.ko 2024-04-17T09:39:35.480 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus/ 2024-04-17T09:39:35.480 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 282097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus/hfsplus.ko 2024-04-17T09:39:35.480 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hpfs/ 2024-04-17T09:39:35.480 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 249009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hpfs/hpfs.ko 2024-04-17T09:39:35.480 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs/ 2024-04-17T09:39:35.480 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 93033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs/isofs.ko 2024-04-17T09:39:35.480 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs/ 2024-04-17T09:39:35.480 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 533073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs/jfs.ko 2024-04-17T09:39:35.480 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd/ 2024-04-17T09:39:35.480 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 361369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd/lockd.ko 2024-04-17T09:39:35.480 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix/ 2024-04-17T09:39:35.481 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 112609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix/minix.ko 2024-04-17T09:39:35.481 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs/ 2024-04-17T09:39:35.481 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 591585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs/netfs.ko 2024-04-17T09:39:35.481 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/ 2024-04-17T09:39:35.481 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout/ 2024-04-17T09:39:35.481 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 102561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-17T09:39:35.481 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout/ 2024-04-17T09:39:35.481 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 74353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-17T09:39:35.481 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout/ 2024-04-17T09:39:35.481 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 147689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-17T09:39:35.481 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 1320249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfs.ko 2024-04-17T09:39:35.481 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 82105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv2.ko 2024-04-17T09:39:35.481 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 151641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv3.ko 2024-04-17T09:39:35.481 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 2518505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv4.ko 2024-04-17T09:39:35.481 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/ 2024-04-17T09:39:35.482 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19849 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/grace.ko 2024-04-17T09:39:35.482 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/nfs_acl.ko 2024-04-17T09:39:35.482 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd/ 2024-04-17T09:39:35.482 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 2024049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd/nfsd.ko 2024-04-17T09:39:35.482 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2/ 2024-04-17T09:39:35.482 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 734025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2/nilfs2.ko 2024-04-17T09:39:35.482 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/ 2024-04-17T09:39:35.482 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-celtic.ko 2024-04-17T09:39:35.482 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-centeuro.ko 2024-04-17T09:39:35.482 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-croatian.ko 2024-04-17T09:39:35.482 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-cyrillic.ko 2024-04-17T09:39:35.482 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-gaelic.ko 2024-04-17T09:39:35.482 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-greek.ko 2024-04-17T09:39:35.482 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-iceland.ko 2024-04-17T09:39:35.483 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-inuit.ko 2024-04-17T09:39:35.483 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-roman.ko 2024-04-17T09:39:35.483 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-romanian.ko 2024-04-17T09:39:35.483 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-turkish.ko 2024-04-17T09:39:35.483 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_ascii.ko 2024-04-17T09:39:35.483 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1250.ko 2024-04-17T09:39:35.483 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1251.ko 2024-04-17T09:39:35.483 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1255.ko 2024-04-17T09:39:35.483 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp737.ko 2024-04-17T09:39:35.483 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp775.ko 2024-04-17T09:39:35.483 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp850.ko 2024-04-17T09:39:35.483 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp852.ko 2024-04-17T09:39:35.483 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp855.ko 2024-04-17T09:39:35.483 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp857.ko 2024-04-17T09:39:35.483 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp860.ko 2024-04-17T09:39:35.483 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp861.ko 2024-04-17T09:39:35.484 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14657 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp862.ko 2024-04-17T09:39:35.484 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp863.ko 2024-04-17T09:39:35.484 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp864.ko 2024-04-17T09:39:35.484 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp865.ko 2024-04-17T09:39:35.484 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp866.ko 2024-04-17T09:39:35.484 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp869.ko 2024-04-17T09:39:35.484 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp874.ko 2024-04-17T09:39:35.484 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 95625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp932.ko 2024-04-17T09:39:35.484 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 144209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp936.ko 2024-04-17T09:39:35.484 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 167137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp949.ko 2024-04-17T09:39:35.484 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 119513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp950.ko 2024-04-17T09:39:35.484 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_euc-jp.ko 2024-04-17T09:39:35.484 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-1.ko 2024-04-17T09:39:35.484 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-13.ko 2024-04-17T09:39:35.484 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-14.ko 2024-04-17T09:39:35.484 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-15.ko 2024-04-17T09:39:35.485 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-2.ko 2024-04-17T09:39:35.485 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-3.ko 2024-04-17T09:39:35.485 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-4.ko 2024-04-17T09:39:35.485 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-5.ko 2024-04-17T09:39:35.485 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-6.ko 2024-04-17T09:39:35.485 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-7.ko 2024-04-17T09:39:35.485 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-9.ko 2024-04-17T09:39:35.485 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-r.ko 2024-04-17T09:39:35.485 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-ru.ko 2024-04-17T09:39:35.485 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-u.ko 2024-04-17T09:39:35.485 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 7521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-17T09:39:35.485 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9601 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_utf8.ko 2024-04-17T09:39:35.485 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ 2024-04-17T09:39:35.485 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster/ 2024-04-17T09:39:35.485 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 302073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-17T09:39:35.485 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm/ 2024-04-17T09:39:35.486 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 548169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-17T09:39:35.486 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs/ 2024-04-17T09:39:35.486 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 66361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-17T09:39:35.486 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 3245617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2.ko 2024-04-17T09:39:35.486 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 22113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-17T09:39:35.486 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 39313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-17T09:39:35.486 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 43073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-17T09:39:35.486 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/omfs/ 2024-04-17T09:39:35.486 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 67489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/omfs/omfs.ko 2024-04-17T09:39:35.486 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs/ 2024-04-17T09:39:35.486 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 462873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs/overlay.ko 2024-04-17T09:39:35.486 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore/ 2024-04-17T09:39:35.486 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 67977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore/ramoops.ko 2024-04-17T09:39:35.487 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/qnx4/ 2024-04-17T09:39:35.487 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/qnx4/qnx4.ko 2024-04-17T09:39:35.487 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/qnx6/ 2024-04-17T09:39:35.487 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 56097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/qnx6/qnx6.ko 2024-04-17T09:39:35.487 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/quota/ 2024-04-17T09:39:35.487 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 40313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/quota/quota_tree.ko 2024-04-17T09:39:35.487 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/quota/quota_v1.ko 2024-04-17T09:39:35.487 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/quota/quota_v2.ko 2024-04-17T09:39:35.487 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs/ 2024-04-17T09:39:35.487 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 684841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs/reiserfs.ko 2024-04-17T09:39:35.487 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs/ 2024-04-17T09:39:35.487 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 34673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs/romfs.ko 2024-04-17T09:39:35.487 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/ 2024-04-17T09:39:35.487 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client/ 2024-04-17T09:39:35.487 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 3319041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client/cifs.ko 2024-04-17T09:39:35.487 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/ 2024-04-17T09:39:35.488 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 8073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_arc4.ko 2024-04-17T09:39:35.488 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_md4.ko 2024-04-17T09:39:35.488 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs/ 2024-04-17T09:39:35.488 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 164185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs/squashfs.ko 2024-04-17T09:39:35.488 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv/ 2024-04-17T09:39:35.488 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 115977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv/sysv.ko 2024-04-17T09:39:35.488 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf/ 2024-04-17T09:39:35.488 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 310017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf/udf.ko 2024-04-17T09:39:35.488 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs/ 2024-04-17T09:39:35.488 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 198233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs/ufs.ko 2024-04-17T09:39:35.488 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs/ 2024-04-17T09:39:35.488 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 5192705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs/xfs.ko 2024-04-17T09:39:35.488 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/ 2024-04-17T09:39:35.488 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/locking/ 2024-04-17T09:39:35.488 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 97625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/locking/locktorture.ko 2024-04-17T09:39:35.488 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/time/ 2024-04-17T09:39:35.489 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/time/test_udelay.ko 2024-04-17T09:39:35.489 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 72665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/torture.ko 2024-04-17T09:39:35.489 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ 2024-04-17T09:39:35.489 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 7313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc-itu-t.ko 2024-04-17T09:39:35.489 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 7081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc7.ko 2024-04-17T09:39:35.489 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 7953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc8.ko 2024-04-17T09:39:35.489 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/ 2024-04-17T09:39:35.489 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 8009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libarc4.ko 2024-04-17T09:39:35.489 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 38985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libdes.ko 2024-04-17T09:39:35.489 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/interval_tree_test.ko 2024-04-17T09:39:35.489 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/libcrc32c.ko 2024-04-17T09:39:35.489 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lru_cache.ko 2024-04-17T09:39:35.489 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/ 2024-04-17T09:39:35.489 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 36169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4_compress.ko 2024-04-17T09:39:35.489 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4hc_compress.ko 2024-04-17T09:39:35.490 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math/ 2024-04-17T09:39:35.490 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 7281 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math/cordic.ko 2024-04-17T09:39:35.490 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/memory-notifier-error-inject.ko 2024-04-17T09:39:35.490 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/notifier-error-inject.ko 2024-04-17T09:39:35.490 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/percpu_test.ko 2024-04-17T09:39:35.490 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/pm-notifier-error-inject.ko 2024-04-17T09:39:35.490 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6/ 2024-04-17T09:39:35.490 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 152025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6/raid6_pq.ko 2024-04-17T09:39:35.490 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/rbtree_test.ko 2024-04-17T09:39:35.490 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon/ 2024-04-17T09:39:35.490 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-17T09:39:35.490 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 40969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/test-kstrtox.ko 2024-04-17T09:39:35.490 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 4912441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/test_bpf.ko 2024-04-17T09:39:35.490 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 61169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/test_firmware.ko 2024-04-17T09:39:35.491 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 7121 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/test_module.ko 2024-04-17T09:39:35.491 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24937 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/test_user_copy.ko 2024-04-17T09:39:35.491 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_bm.ko 2024-04-17T09:39:35.491 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_fsm.ko 2024-04-17T09:39:35.491 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_kmp.ko 2024-04-17T09:39:35.491 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/xz/ 2024-04-17T09:39:35.491 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/xz/xz_dec_test.ko 2024-04-17T09:39:35.491 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm/ 2024-04-17T09:39:35.491 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm/hwpoison-inject.ko 2024-04-17T09:39:35.491 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ 2024-04-17T09:39:35.491 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/ 2024-04-17T09:39:35.491 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 66969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/6lowpan.ko 2024-04-17T09:39:35.491 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 7577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_dest.ko 2024-04-17T09:39:35.491 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 7529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_fragment.ko 2024-04-17T09:39:35.492 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 7513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_hop.ko 2024-04-17T09:39:35.492 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 7513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-17T09:39:35.492 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 7529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_mobility.ko 2024-04-17T09:39:35.492 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 7521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_routing.ko 2024-04-17T09:39:35.492 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_udp.ko 2024-04-17T09:39:35.492 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/ 2024-04-17T09:39:35.492 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 36729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/garp.ko 2024-04-17T09:39:35.492 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 42297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/mrp.ko 2024-04-17T09:39:35.492 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9849 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/p8022.ko 2024-04-17T09:39:35.492 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/psnap.ko 2024-04-17T09:39:35.492 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13537 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/stp.ko 2024-04-17T09:39:35.492 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q/ 2024-04-17T09:39:35.492 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 106793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q/8021q.ko 2024-04-17T09:39:35.492 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/ 2024-04-17T09:39:35.492 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 236793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet.ko 2024-04-17T09:39:35.492 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 44449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_fd.ko 2024-04-17T09:39:35.493 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 32209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_rdma.ko 2024-04-17T09:39:35.493 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 39513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_virtio.ko 2024-04-17T09:39:35.493 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv/ 2024-04-17T09:39:35.493 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 879697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv/batman-adv.ko 2024-04-17T09:39:35.493 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/ 2024-04-17T09:39:35.493 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 84761 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/br_netfilter.ko 2024-04-17T09:39:35.493 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 1030873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/bridge.ko 2024-04-17T09:39:35.493 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ 2024-04-17T09:39:35.493 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 8905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-17T09:39:35.493 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-17T09:39:35.493 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-17T09:39:35.493 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-17T09:39:35.493 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-17T09:39:35.493 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-17T09:39:35.493 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-17T09:39:35.493 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-17T09:39:35.494 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-17T09:39:35.494 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 8841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-17T09:39:35.494 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 8777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-17T09:39:35.494 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9569 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-17T09:39:35.494 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 8625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-17T09:39:35.494 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-17T09:39:35.494 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-17T09:39:35.494 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-17T09:39:35.494 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-17T09:39:35.494 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-17T09:39:35.494 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-17T09:39:35.494 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-17T09:39:35.494 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 70529 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtables.ko 2024-04-17T09:39:35.495 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-17T09:39:35.495 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-17T09:39:35.495 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph/ 2024-04-17T09:39:35.495 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 1243505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph/libceph.ko 2024-04-17T09:39:35.495 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/ 2024-04-17T09:39:35.495 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 153801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/pktgen.ko 2024-04-17T09:39:35.495 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/ 2024-04-17T09:39:35.495 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 244889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp.ko 2024-04-17T09:39:35.495 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_diag.ko 2024-04-17T09:39:35.495 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 71033 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv4.ko 2024-04-17T09:39:35.495 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 74993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv6.ko 2024-04-17T09:39:35.495 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/ 2024-04-17T09:39:35.495 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 507705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/dsa_core.ko 2024-04-17T09:39:35.495 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_brcm.ko 2024-04-17T09:39:35.495 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_dsa.ko 2024-04-17T09:39:35.496 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 8873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_none.ko 2024-04-17T09:39:35.496 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_trailer.ko 2024-04-17T09:39:35.496 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/hsr/ 2024-04-17T09:39:35.496 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 128873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/hsr/hsr.ko 2024-04-17T09:39:35.496 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ 2024-04-17T09:39:35.496 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan/ 2024-04-17T09:39:35.496 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 55505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-17T09:39:35.496 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 357425 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154.ko 2024-04-17T09:39:35.496 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 59393 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-17T09:39:35.496 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ 2024-04-17T09:39:35.496 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ah4.ko 2024-04-17T09:39:35.496 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 46417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/esp4.ko 2024-04-17T09:39:35.496 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 60233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/fou.ko 2024-04-17T09:39:35.496 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21937 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/gre.ko 2024-04-17T09:39:35.496 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 65665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/inet_diag.ko 2024-04-17T09:39:35.497 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 75489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_gre.ko 2024-04-17T09:39:35.497 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 77721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_tunnel.ko 2024-04-17T09:39:35.497 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 43705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_vti.ko 2024-04-17T09:39:35.497 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipcomp.ko 2024-04-17T09:39:35.497 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 34745 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipip.ko 2024-04-17T09:39:35.497 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ 2024-04-17T09:39:35.497 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 45521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-17T09:39:35.497 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-17T09:39:35.497 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-17T09:39:35.497 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 50169 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ip_tables.ko 2024-04-17T09:39:35.497 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-17T09:39:35.497 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-04-17T09:39:35.497 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-17T09:39:35.497 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-17T09:39:35.498 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-17T09:39:35.498 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-04-17T09:39:35.498 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-17T09:39:35.498 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-17T09:39:35.498 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-17T09:39:35.498 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-17T09:39:35.498 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-17T09:39:35.498 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-17T09:39:35.498 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 32473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-17T09:39:35.498 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-17T09:39:35.498 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-17T09:39:35.498 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-04-17T09:39:35.498 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-17T09:39:35.499 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-17T09:39:35.499 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-17T09:39:35.499 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-17T09:39:35.499 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-17T09:39:35.499 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_bic.ko 2024-04-17T09:39:35.499 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_dctcp.ko 2024-04-17T09:39:35.499 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_diag.ko 2024-04-17T09:39:35.499 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_highspeed.ko 2024-04-17T09:39:35.499 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_htcp.ko 2024-04-17T09:39:35.499 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11665 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_hybla.ko 2024-04-17T09:39:35.499 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_illinois.ko 2024-04-17T09:39:35.499 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11737 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_lp.ko 2024-04-17T09:39:35.499 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9601 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_scalable.ko 2024-04-17T09:39:35.499 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_vegas.ko 2024-04-17T09:39:35.500 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_veno.ko 2024-04-17T09:39:35.500 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_westwood.ko 2024-04-17T09:39:35.500 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11745 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_yeah.ko 2024-04-17T09:39:35.500 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tunnel4.ko 2024-04-17T09:39:35.500 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_diag.ko 2024-04-17T09:39:35.500 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 54441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_tunnel.ko 2024-04-17T09:39:35.500 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-17T09:39:35.500 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ 2024-04-17T09:39:35.500 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 31073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ah6.ko 2024-04-17T09:39:35.500 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 47433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/esp6.ko 2024-04-17T09:39:35.500 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/fou6.ko 2024-04-17T09:39:35.500 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 103177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_gre.ko 2024-04-17T09:39:35.500 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 118881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_tunnel.ko 2024-04-17T09:39:35.500 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-17T09:39:35.501 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 71089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_vti.ko 2024-04-17T09:39:35.501 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ipcomp6.ko 2024-04-17T09:39:35.501 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/mip6.ko 2024-04-17T09:39:35.501 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ 2024-04-17T09:39:35.501 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 50673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-17T09:39:35.501 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-04-17T09:39:35.501 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-17T09:39:35.501 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-17T09:39:35.501 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15929 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-17T09:39:35.501 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 8721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-17T09:39:35.501 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-17T09:39:35.501 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-17T09:39:35.501 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-17T09:39:35.502 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-17T09:39:35.502 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-17T09:39:35.502 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-17T09:39:35.502 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-17T09:39:35.502 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-17T09:39:35.502 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-17T09:39:35.502 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-17T09:39:35.502 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-17T09:39:35.502 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 46313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-17T09:39:35.502 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-17T09:39:35.502 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 29817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-17T09:39:35.502 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-17T09:39:35.502 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-17T09:39:35.502 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-17T09:39:35.503 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-17T09:39:35.503 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-17T09:39:35.503 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 94089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/sit.ko 2024-04-17T09:39:35.503 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 30297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/tunnel6.ko 2024-04-17T09:39:35.503 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-17T09:39:35.503 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key/ 2024-04-17T09:39:35.503 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 106369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key/af_key.ko 2024-04-17T09:39:35.503 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/ 2024-04-17T09:39:35.503 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 153217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_core.ko 2024-04-17T09:39:35.503 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-17T09:39:35.503 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 31985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_eth.ko 2024-04-17T09:39:35.503 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 43641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip.ko 2024-04-17T09:39:35.503 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 44993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip6.ko 2024-04-17T09:39:35.503 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 42049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_netlink.ko 2024-04-17T09:39:35.504 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 66241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ppp.ko 2024-04-17T09:39:35.504 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/lapb/ 2024-04-17T09:39:35.504 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 63345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/lapb/lapb.ko 2024-04-17T09:39:35.504 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc/ 2024-04-17T09:39:35.504 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 27385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc/llc.ko 2024-04-17T09:39:35.504 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 332889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc/llc2.ko 2024-04-17T09:39:35.504 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154/ 2024-04-17T09:39:35.504 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 388873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154/mac802154.ko 2024-04-17T09:39:35.504 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls/ 2024-04-17T09:39:35.504 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls/mpls_gso.ko 2024-04-17T09:39:35.504 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ 2024-04-17T09:39:35.504 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ 2024-04-17T09:39:35.504 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 147745 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set.ko 2024-04-17T09:39:35.505 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 34473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-17T09:39:35.505 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-17T09:39:35.505 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 32553 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-17T09:39:35.505 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 124041 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-17T09:39:35.505 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 124345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-17T09:39:35.505 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 126681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-17T09:39:35.505 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 126473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-17T09:39:35.505 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 137833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-17T09:39:35.505 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 72353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-17T09:39:35.505 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 133681 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-17T09:39:35.505 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 139073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-17T09:39:35.505 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 138265 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-17T09:39:35.505 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 136177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-17T09:39:35.505 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 139897 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-17T09:39:35.505 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 37129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-17T09:39:35.506 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ 2024-04-17T09:39:35.506 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 534113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-17T09:39:35.506 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-17T09:39:35.506 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 8801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-17T09:39:35.506 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-17T09:39:35.506 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20953 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-17T09:39:35.506 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24577 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-17T09:39:35.506 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 8801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-17T09:39:35.506 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 8865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-17T09:39:35.506 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12649 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-17T09:39:35.506 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-17T09:39:35.506 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 8921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-17T09:39:35.506 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15977 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-17T09:39:35.507 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 8921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-17T09:39:35.507 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12809 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-17T09:39:35.507 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 55217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conncount.ko 2024-04-17T09:39:35.507 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 496817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack.ko 2024-04-17T09:39:35.507 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-17T09:39:35.507 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-17T09:39:35.507 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 37865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-17T09:39:35.507 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 134377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-17T09:39:35.507 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24513 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-17T09:39:35.507 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-17T09:39:35.507 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 140801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-17T09:39:35.507 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 38305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-17T09:39:35.507 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18937 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-17T09:39:35.507 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 79873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-17T09:39:35.508 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-17T09:39:35.508 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-17T09:39:35.508 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-17T09:39:35.508 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 117201 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table.ko 2024-04-17T09:39:35.508 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-17T09:39:35.508 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 46105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_log_syslog.ko 2024-04-17T09:39:35.508 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 118369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat.ko 2024-04-17T09:39:35.508 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-17T09:39:35.508 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14185 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-17T09:39:35.508 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13345 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_irc.ko 2024-04-17T09:39:35.508 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 34609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_sip.ko 2024-04-17T09:39:35.508 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-17T09:39:35.508 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 53393 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-17T09:39:35.508 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 807985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_tables.ko 2024-04-17T09:39:35.509 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 53049 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink.ko 2024-04-17T09:39:35.509 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 36481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-17T09:39:35.509 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 37209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-04-17T09:39:35.509 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 49297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_log.ko 2024-04-17T09:39:35.509 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-17T09:39:35.509 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 65145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-17T09:39:35.509 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_chain_nat.ko 2024-04-17T09:39:35.509 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 41769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_compat.ko 2024-04-17T09:39:35.509 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 50657 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_ct.ko 2024-04-17T09:39:35.509 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11081 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-17T09:39:35.509 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13745 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fib.ko 2024-04-17T09:39:35.509 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16545 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-17T09:39:35.509 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_hash.ko 2024-04-17T09:39:35.510 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_limit.ko 2024-04-17T09:39:35.510 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17921 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_log.ko 2024-04-17T09:39:35.510 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_masq.ko 2024-04-17T09:39:35.510 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_nat.ko 2024-04-17T09:39:35.510 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16473 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_numgen.ko 2024-04-17T09:39:35.510 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13945 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_osf.ko 2024-04-17T09:39:35.510 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_queue.ko 2024-04-17T09:39:35.510 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16657 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_redir.ko 2024-04-17T09:39:35.510 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject.ko 2024-04-17T09:39:35.510 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject_inet.ko 2024-04-17T09:39:35.510 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15057 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_socket.ko 2024-04-17T09:39:35.510 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 24993 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_synproxy.ko 2024-04-17T09:39:35.510 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17641 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_tproxy.ko 2024-04-17T09:39:35.510 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 108393 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/x_tables.ko 2024-04-17T09:39:35.511 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14193 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_AUDIT.ko 2024-04-17T09:39:35.511 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-17T09:39:35.511 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 8625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-17T09:39:35.511 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-17T09:39:35.511 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 21865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CT.ko 2024-04-17T09:39:35.511 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_DSCP.ko 2024-04-17T09:39:35.511 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HL.ko 2024-04-17T09:39:35.511 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15745 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HMARK.ko 2024-04-17T09:39:35.511 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 45217 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-17T09:39:35.511 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15937 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LED.ko 2024-04-17T09:39:35.511 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13017 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LOG.ko 2024-04-17T09:39:35.511 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-17T09:39:35.511 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NETMAP.ko 2024-04-17T09:39:35.511 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10857 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFLOG.ko 2024-04-17T09:39:35.512 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15361 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-17T09:39:35.512 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23089 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_RATEEST.ko 2024-04-17T09:39:35.512 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12553 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-17T09:39:35.512 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16689 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_SECMARK.ko 2024-04-17T09:39:35.512 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18985 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-17T09:39:35.512 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-17T09:39:35.512 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20697 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TEE.ko 2024-04-17T09:39:35.512 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TPROXY.ko 2024-04-17T09:39:35.512 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TRACE.ko 2024-04-17T09:39:35.512 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15289 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_addrtype.ko 2024-04-17T09:39:35.512 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16633 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_bpf.ko 2024-04-17T09:39:35.512 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cgroup.ko 2024-04-17T09:39:35.512 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cluster.ko 2024-04-17T09:39:35.512 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 8465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_comment.ko 2024-04-17T09:39:35.512 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11961 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connbytes.ko 2024-04-17T09:39:35.513 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlabel.ko 2024-04-17T09:39:35.513 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlimit.ko 2024-04-17T09:39:35.513 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connmark.ko 2024-04-17T09:39:35.513 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14209 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_conntrack.ko 2024-04-17T09:39:35.513 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 8913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cpu.ko 2024-04-17T09:39:35.513 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dccp.ko 2024-04-17T09:39:35.513 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_devgroup.ko 2024-04-17T09:39:35.513 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10257 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dscp.ko 2024-04-17T09:39:35.513 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12657 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ecn.ko 2024-04-17T09:39:35.513 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_esp.ko 2024-04-17T09:39:35.513 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 52817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hashlimit.ko 2024-04-17T09:39:35.513 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_helper.ko 2024-04-17T09:39:35.513 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hl.ko 2024-04-17T09:39:35.513 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipcomp.ko 2024-04-17T09:39:35.513 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_iprange.ko 2024-04-17T09:39:35.514 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipvs.ko 2024-04-17T09:39:35.514 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19193 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_l2tp.ko 2024-04-17T09:39:35.514 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9137 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_length.ko 2024-04-17T09:39:35.514 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11521 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_limit.ko 2024-04-17T09:39:35.514 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 8449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mac.ko 2024-04-17T09:39:35.514 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mark.ko 2024-04-17T09:39:35.514 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_multiport.ko 2024-04-17T09:39:35.514 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 16161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nat.ko 2024-04-17T09:39:35.514 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nfacct.ko 2024-04-17T09:39:35.514 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_osf.ko 2024-04-17T09:39:35.514 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11561 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_owner.ko 2024-04-17T09:39:35.514 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_physdev.ko 2024-04-17T09:39:35.514 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 8625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_pkttype.ko 2024-04-17T09:39:35.514 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13553 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_policy.ko 2024-04-17T09:39:35.515 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_quota.ko 2024-04-17T09:39:35.515 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 11881 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_rateest.ko 2024-04-17T09:39:35.515 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9273 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_realm.ko 2024-04-17T09:39:35.515 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 43465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_recent.ko 2024-04-17T09:39:35.515 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_sctp.ko 2024-04-17T09:39:35.515 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 36489 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_set.ko 2024-04-17T09:39:35.515 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 19777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_socket.ko 2024-04-17T09:39:35.515 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10753 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_state.ko 2024-04-17T09:39:35.515 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10849 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_statistic.ko 2024-04-17T09:39:35.515 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_string.ko 2024-04-17T09:39:35.515 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_tcpmss.ko 2024-04-17T09:39:35.515 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18449 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_tcpudp.ko 2024-04-17T09:39:35.515 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12441 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_time.ko 2024-04-17T09:39:35.515 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_u32.ko 2024-04-17T09:39:35.515 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink/ 2024-04-17T09:39:35.515 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 17233 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink/netlink_diag.ko 2024-04-17T09:39:35.516 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh/ 2024-04-17T09:39:35.516 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 14073 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh/nsh.ko 2024-04-17T09:39:35.516 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/ 2024-04-17T09:39:35.516 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 488825 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/openvswitch.ko 2024-04-17T09:39:35.516 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12009 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-geneve.ko 2024-04-17T09:39:35.516 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 10873 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-gre.ko 2024-04-17T09:39:35.516 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-vxlan.ko 2024-04-17T09:39:35.516 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet/ 2024-04-17T09:39:35.516 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15929 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet/af_packet_diag.ko 2024-04-17T09:39:35.516 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/phonet/ 2024-04-17T09:39:35.516 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 142025 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/phonet/phonet.ko 2024-04-17T09:39:35.516 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 68817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/phonet/pn_pep.ko 2024-04-17T09:39:35.516 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/ 2024-04-17T09:39:35.516 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 318617 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds.ko 2024-04-17T09:39:35.517 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 202305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_rdma.ko 2024-04-17T09:39:35.517 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 85105 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_tcp.ko 2024-04-17T09:39:35.517 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rxrpc/ 2024-04-17T09:39:35.517 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 1027897 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rxrpc/rxrpc.ko 2024-04-17T09:39:35.517 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/ 2024-04-17T09:39:35.517 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 39001 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_csum.ko 2024-04-17T09:39:35.517 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 28721 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_gact.ko 2024-04-17T09:39:35.517 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 45409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_mirred.ko 2024-04-17T09:39:35.517 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 27593 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_nat.ko 2024-04-17T09:39:35.517 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 53465 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_pedit.ko 2024-04-17T09:39:35.517 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 38705 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_police.ko 2024-04-17T09:39:35.517 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23969 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_simple.ko 2024-04-17T09:39:35.517 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 43417 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_skbedit.ko 2024-04-17T09:39:35.517 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 45673 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_vlan.ko 2024-04-17T09:39:35.517 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 32129 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_basic.ko 2024-04-17T09:39:35.518 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 41457 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_bpf.ko 2024-04-17T09:39:35.518 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23225 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_cgroup.ko 2024-04-17T09:39:35.518 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 37305 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_flow.ko 2024-04-17T09:39:35.518 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 39601 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_fw.ko 2024-04-17T09:39:35.518 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 49321 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_route.ko 2024-04-17T09:39:35.518 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 79145 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_u32.ko 2024-04-17T09:39:35.518 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 8713 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_cmp.ko 2024-04-17T09:39:35.518 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 15385 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_ipset.ko 2024-04-17T09:39:35.518 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 48785 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_meta.ko 2024-04-17T09:39:35.518 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9241 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_nbyte.ko 2024-04-17T09:39:35.518 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13369 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_text.ko 2024-04-17T09:39:35.518 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 8377 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_u32.ko 2024-04-17T09:39:35.518 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_choke.ko 2024-04-17T09:39:35.518 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 25097 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_codel.ko 2024-04-17T09:39:35.518 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 43481 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_drr.ko 2024-04-17T09:39:35.519 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 43153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_fq.ko 2024-04-17T09:39:35.519 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 39177 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_fq_codel.ko 2024-04-17T09:39:35.519 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 42889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_gred.ko 2024-04-17T09:39:35.519 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 62817 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hfsc.ko 2024-04-17T09:39:35.519 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26585 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hhf.ko 2024-04-17T09:39:35.519 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 81833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_htb.ko 2024-04-17T09:39:35.519 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33545 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_ingress.ko 2024-04-17T09:39:35.519 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 36625 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio.ko 2024-04-17T09:39:35.519 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12297 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio_lib.ko 2024-04-17T09:39:35.519 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33065 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_multiq.ko 2024-04-17T09:39:35.519 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 44793 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_netem.ko 2024-04-17T09:39:35.519 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 32833 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_pie.ko 2024-04-17T09:39:35.519 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 13505 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_plug.ko 2024-04-17T09:39:35.519 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 31801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_prio.ko 2024-04-17T09:39:35.519 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 60249 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_qfq.ko 2024-04-17T09:39:35.520 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 40889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_red.ko 2024-04-17T09:39:35.520 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 33433 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfb.ko 2024-04-17T09:39:35.520 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 38329 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfq.ko 2024-04-17T09:39:35.520 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 35777 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_tbf.ko 2024-04-17T09:39:35.520 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 34161 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_teql.ko 2024-04-17T09:39:35.520 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/ 2024-04-17T09:39:35.520 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 1187313 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp.ko 2024-04-17T09:39:35.520 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23913 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp_diag.ko 2024-04-17T09:39:35.520 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/ 2024-04-17T09:39:35.520 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/ 2024-04-17T09:39:35.520 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 432409 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-17T09:39:35.520 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 70113 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-17T09:39:35.520 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 1940889 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/sunrpc.ko 2024-04-17T09:39:35.520 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma/ 2024-04-17T09:39:35.520 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 890729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-17T09:39:35.521 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/ 2024-04-17T09:39:35.521 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 1122337 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc.ko 2024-04-17T09:39:35.521 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12153 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc_diag.ko 2024-04-17T09:39:35.521 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix/ 2024-04-17T09:39:35.521 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 18801 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix/unix_diag.ko 2024-04-17T09:39:35.521 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/ 2024-04-17T09:39:35.521 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 105609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-17T09:39:35.521 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 90265 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-17T09:39:35.521 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 123769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock.ko 2024-04-17T09:39:35.521 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 12905 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-17T09:39:35.521 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 20401 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-17T09:39:35.521 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/x25/ 2024-04-17T09:39:35.521 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 165841 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/x25/x25.ko 2024-04-17T09:39:35.521 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/ 2024-04-17T09:39:35.522 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 26729 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/xfrm_algo.ko 2024-04-17T09:39:35.522 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23353 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-17T09:39:35.522 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 131865 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/xfrm_user.ko 2024-04-17T09:39:35.522 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 644769 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias 2024-04-17T09:39:35.522 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 628542 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias.bin 2024-04-17T09:39:35.522 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 7706 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin 2024-04-17T09:39:35.522 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 23609 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.alias.bin 2024-04-17T09:39:35.522 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 9941 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.bin 2024-04-17T09:39:35.522 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 70990 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.modinfo 2024-04-17T09:39:35.522 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 146959 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep 2024-04-17T09:39:35.522 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 232835 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep.bin 2024-04-17T09:39:35.522 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 210 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.devname 2024-04-17T09:39:35.522 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 72602 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.order 2024-04-17T09:39:35.522 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 879 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.softdep 2024-04-17T09:39:35.523 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 258450 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols 2024-04-17T09:39:35.523 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 313019 2024-04-11 18:50 ./lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols.bin 2024-04-17T09:39:35.523 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./usr/ 2024-04-17T09:39:35.523 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./usr/share/ 2024-04-17T09:39:35.523 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./usr/share/doc/ 2024-04-17T09:39:35.523 INFO:teuthology.orchestra.run.smithi144.stdout:drwxr-xr-x root/root 0 2024-04-11 18:50 ./usr/share/doc/linux-image-6.9.0-rc3-g3f60ed14f7c1/ 2024-04-17T09:39:35.523 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 174 2024-04-11 18:50 ./usr/share/doc/linux-image-6.9.0-rc3-g3f60ed14f7c1/changelog.Debian.gz 2024-04-17T09:39:35.523 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r-- root/root 691 2024-04-11 18:50 ./usr/share/doc/linux-image-6.9.0-rc3-g3f60ed14f7c1/copyright 2024-04-17T09:39:35.527 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:39:35.527 INFO:teuthology.task.kernel:searching for kernel 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:39:35.528 DEBUG:teuthology.orchestra.run.smithi144:> egrep '(submenu|menuentry.*6.9.0-rc3-g3f60ed14f7c1).*{' /boot/grub/grub.cfg 2024-04-17T09:39:35.534 INFO:teuthology.orchestra.run.smithi144.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-17T09:39:35.534 INFO:teuthology.orchestra.run.smithi144.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g3f60ed14f7c1' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g3f60ed14f7c1-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-17T09:39:35.534 INFO:teuthology.orchestra.run.smithi144.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g3f60ed14f7c1 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g3f60ed14f7c1-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-17T09:39:35.535 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-04-17T09:39:35.535 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:39:35.535 DEBUG:teuthology.orchestra.run.smithi144:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-04-17T09:39:35.580 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi144.front.sepia.ceph.com to complete... 2024-04-17T09:39:35.977 INFO:teuthology.orchestra.run.smithi144.stderr:Sourcing file `/etc/default/grub' 2024-04-17T09:39:35.979 INFO:teuthology.orchestra.run.smithi144.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-17T09:39:36.010 INFO:teuthology.orchestra.run.smithi144.stderr:Generating grub configuration file ... 2024-04-17T09:39:36.195 INFO:teuthology.orchestra.run.smithi144.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:39:36.201 INFO:teuthology.orchestra.run.smithi144.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:39:36.445 INFO:teuthology.orchestra.run.smithi144.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-17T09:39:36.450 INFO:teuthology.orchestra.run.smithi144.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-17T09:39:36.488 INFO:teuthology.orchestra.run.smithi144.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-17T09:39:36.492 INFO:teuthology.orchestra.run.smithi144.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-17T09:39:36.592 INFO:teuthology.orchestra.run.smithi144.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-17T09:39:36.592 INFO:teuthology.orchestra.run.smithi144.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-17T09:39:36.592 INFO:teuthology.orchestra.run.smithi144.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-17T09:39:36.609 INFO:teuthology.orchestra.run.smithi144.stderr:done 2024-04-17T09:39:49.234 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-17T09:39:49.235 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-17T09:39:49.235 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T09:40:07.587 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2024-04-17T09:40:07.731 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-17T09:40:07.732 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-04-17T09:40:07.732 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T09:40:16.630 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-17T09:40:16.631 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T09:40:34.975 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2024-04-17T09:40:46.987 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-17T09:40:46.988 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T09:40:50.050 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2024-04-17T09:41:05.062 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-17T09:41:05.063 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T09:41:05.284 DEBUG:teuthology.orchestra.run.smithi032:> true 2024-04-17T09:41:06.058 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-17T09:41:06.058 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-g3f60ed14f7c1"... 2024-04-17T09:41:06.059 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2024-04-17T09:41:06.109 INFO:teuthology.orchestra.run.smithi032.stdout:6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:41:06.109 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g3f60ed14f7c1 vs 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:41:06.109 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-17T09:41:06.109 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-17T09:41:07.110 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-17T09:41:07.110 DEBUG:teuthology.orchestra.run.smithi032:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-17T09:41:07.198 INFO:teuthology.orchestra.run.smithi032.stdout:ttyS1 2024-04-17T09:41:07.240 DEBUG:teuthology.parallel:result is None 2024-04-17T09:41:07.734 DEBUG:teuthology.orchestra.remote:timed out 2024-04-17T09:41:16.736 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-04-17T09:41:16.737 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T09:41:16.977 ERROR:teuthology.orchestra.connection:Error authenticating with smithi144.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 485, in connect self._auth( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 818, in _auth raise saved_exception File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 794, in _auth self._transport.auth_publickey(username, key) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 1664, in auth_publickey raise SSHException("No existing session") paramiko.ssh_exception.SSHException: No existing session 2024-04-17T09:41:21.062 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-04-17T09:41:22.188 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-04-17T09:41:22.188 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-g3f60ed14f7c1"... 2024-04-17T09:41:22.188 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-04-17T09:41:22.238 INFO:teuthology.orchestra.run.smithi144.stdout:6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:41:22.238 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g3f60ed14f7c1 vs 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T09:41:22.238 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-17T09:41:22.238 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-17T09:41:23.239 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-17T09:41:23.239 DEBUG:teuthology.orchestra.run.smithi144:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-17T09:41:23.312 INFO:teuthology.orchestra.run.smithi144.stdout:ttyS1 2024-04-17T09:41:23.376 DEBUG:teuthology.parallel:result is None 2024-04-17T09:41:23.376 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-17T09:41:23.384 INFO:teuthology.task.internal:Creating test directory... 2024-04-17T09:41:23.384 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-17T09:41:23.388 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-17T09:41:23.397 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-17T09:41:23.403 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-17T09:41:23.409 INFO:teuthology.task.internal:Creating archive directory... 2024-04-17T09:41:23.409 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-17T09:41:23.439 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-17T09:41:23.476 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-17T09:41:23.483 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-17T09:41:23.483 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-17T09:41:23.512 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-17T09:41:23.533 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T09:41:23.547 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T09:41:23.551 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T09:41:23.565 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T09:41:23.567 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-17T09:41:23.574 INFO:teuthology.task.internal:Configuring sudo... 2024-04-17T09:41:23.575 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-17T09:41:23.600 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-17T09:41:23.631 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-17T09:41:23.642 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-17T09:41:23.642 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-17T09:41:23.668 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-17T09:41:23.683 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T09:41:23.730 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T09:41:23.786 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:41:23.786 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-17T09:41:23.862 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T09:41:23.872 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T09:41:23.923 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:41:23.923 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-17T09:41:24.000 DEBUG:teuthology.orchestra.run.smithi032:> sudo service rsyslog restart 2024-04-17T09:41:24.004 DEBUG:teuthology.orchestra.run.smithi144:> sudo service rsyslog restart 2024-04-17T09:41:24.132 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-17T09:41:24.140 INFO:teuthology.task.internal:Starting timer... 2024-04-17T09:41:24.140 INFO:teuthology.run_tasks:Running task pcp... 2024-04-17T09:41:24.212 INFO:teuthology.run_tasks:Running task selinux... 2024-04-17T09:41:24.220 DEBUG:teuthology.task.selinux:Excluding smithi032: OS 'ubuntu' does not support SELinux 2024-04-17T09:41:24.221 DEBUG:teuthology.task.selinux:Excluding smithi144: OS 'ubuntu' does not support SELinux 2024-04-17T09:41:24.221 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-17T09:41:24.221 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-17T09:41:24.221 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-17T09:41:24.221 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-17T09:41:24.229 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-17T09:41:24.235 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-17T09:41:24.384 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-17T09:41:24.402 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-17T09:41:24.403 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi032.front.sepia.ceph.com,smithi144.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-17T09:49:08.389 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi032.front.sepia.ceph.com'), Remote(name='ubuntu@smithi144.front.sepia.ceph.com')] 2024-04-17T09:49:08.391 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-17T09:49:08.392 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T09:49:08.479 DEBUG:teuthology.orchestra.run.smithi032:> true 2024-04-17T09:49:08.601 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-04-17T09:49:08.602 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-04-17T09:49:08.602 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T09:49:08.682 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-04-17T09:49:08.793 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-04-17T09:49:08.793 INFO:teuthology.run_tasks:Running task clock... 2024-04-17T09:49:08.804 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-17T09:49:08.805 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-17T09:49:08.805 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T09:49:08.808 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-17T09:49:08.809 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T09:49:08.848 INFO:teuthology.orchestra.run.smithi032.stdout:17 Apr 09:49:08 ntpd[17108]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-17T09:49:08.848 INFO:teuthology.orchestra.run.smithi032.stdout:17 Apr 09:49:08 ntpd[17108]: Command line: ntpd -gq 2024-04-17T09:49:08.848 INFO:teuthology.orchestra.run.smithi032.stdout:17 Apr 09:49:08 ntpd[17108]: ---------------------------------------------------- 2024-04-17T09:49:08.848 INFO:teuthology.orchestra.run.smithi032.stdout:17 Apr 09:49:08 ntpd[17108]: ntp-4 is maintained by Network Time Foundation, 2024-04-17T09:49:08.848 INFO:teuthology.orchestra.run.smithi032.stdout:17 Apr 09:49:08 ntpd[17108]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-17T09:49:08.848 INFO:teuthology.orchestra.run.smithi032.stdout:17 Apr 09:49:08 ntpd[17108]: corporation. Support and training for ntp-4 are 2024-04-17T09:49:08.848 INFO:teuthology.orchestra.run.smithi032.stdout:17 Apr 09:49:08 ntpd[17108]: available at https://www.nwtime.org/support 2024-04-17T09:49:08.848 INFO:teuthology.orchestra.run.smithi032.stdout:17 Apr 09:49:08 ntpd[17108]: ---------------------------------------------------- 2024-04-17T09:49:08.849 INFO:teuthology.orchestra.run.smithi032.stdout:17 Apr 09:49:08 ntpd[17108]: proto: precision = 0.044 usec (-24) 2024-04-17T09:49:08.849 INFO:teuthology.orchestra.run.smithi032.stdout:17 Apr 09:49:08 ntpd[17108]: basedate set to 2022-02-04 2024-04-17T09:49:08.849 INFO:teuthology.orchestra.run.smithi032.stdout:17 Apr 09:49:08 ntpd[17108]: gps base set to 2022-02-06 (week 2196) 2024-04-17T09:49:08.850 INFO:teuthology.orchestra.run.smithi032.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-17T09:49:08.850 INFO:teuthology.orchestra.run.smithi032.stdout:17 Apr 09:49:08 ntpd[17108]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-17T09:49:08.850 INFO:teuthology.orchestra.run.smithi032.stdout:17 Apr 09:49:08 ntpd[17108]: restrict ::: KOD does nothing without LIMITED. 2024-04-17T09:49:08.850 INFO:teuthology.orchestra.run.smithi032.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-17T09:49:08.851 INFO:teuthology.orchestra.run.smithi032.stdout:17 Apr 09:49:08 ntpd[17108]: Listen and drop on 0 v6wildcard [::]:123 2024-04-17T09:49:08.852 INFO:teuthology.orchestra.run.smithi032.stdout:17 Apr 09:49:08 ntpd[17108]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-17T09:49:08.852 INFO:teuthology.orchestra.run.smithi032.stdout:17 Apr 09:49:08 ntpd[17108]: Listen normally on 2 lo 127.0.0.1:123 2024-04-17T09:49:08.852 INFO:teuthology.orchestra.run.smithi032.stdout:17 Apr 09:49:08 ntpd[17108]: Listen normally on 3 ens1f0 172.21.15.32:123 2024-04-17T09:49:08.852 INFO:teuthology.orchestra.run.smithi032.stdout:17 Apr 09:49:08 ntpd[17108]: Listen normally on 4 lo [::1]:123 2024-04-17T09:49:08.852 INFO:teuthology.orchestra.run.smithi032.stdout:17 Apr 09:49:08 ntpd[17108]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:2602%3]:123 2024-04-17T09:49:08.852 INFO:teuthology.orchestra.run.smithi032.stdout:17 Apr 09:49:08 ntpd[17108]: Listening on routing socket on fd #22 for interface updates 2024-04-17T09:49:08.878 INFO:teuthology.orchestra.run.smithi144.stdout:17 Apr 09:49:08 ntpd[17198]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-17T09:49:08.878 INFO:teuthology.orchestra.run.smithi144.stdout:17 Apr 09:49:08 ntpd[17198]: Command line: ntpd -gq 2024-04-17T09:49:08.878 INFO:teuthology.orchestra.run.smithi144.stdout:17 Apr 09:49:08 ntpd[17198]: ---------------------------------------------------- 2024-04-17T09:49:08.878 INFO:teuthology.orchestra.run.smithi144.stdout:17 Apr 09:49:08 ntpd[17198]: ntp-4 is maintained by Network Time Foundation, 2024-04-17T09:49:08.878 INFO:teuthology.orchestra.run.smithi144.stdout:17 Apr 09:49:08 ntpd[17198]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-17T09:49:08.878 INFO:teuthology.orchestra.run.smithi144.stdout:17 Apr 09:49:08 ntpd[17198]: corporation. Support and training for ntp-4 are 2024-04-17T09:49:08.878 INFO:teuthology.orchestra.run.smithi144.stdout:17 Apr 09:49:08 ntpd[17198]: available at https://www.nwtime.org/support 2024-04-17T09:49:08.878 INFO:teuthology.orchestra.run.smithi144.stdout:17 Apr 09:49:08 ntpd[17198]: ---------------------------------------------------- 2024-04-17T09:49:08.879 INFO:teuthology.orchestra.run.smithi144.stdout:17 Apr 09:49:08 ntpd[17198]: proto: precision = 0.042 usec (-24) 2024-04-17T09:49:08.879 INFO:teuthology.orchestra.run.smithi144.stdout:17 Apr 09:49:08 ntpd[17198]: basedate set to 2022-02-04 2024-04-17T09:49:08.879 INFO:teuthology.orchestra.run.smithi144.stdout:17 Apr 09:49:08 ntpd[17198]: gps base set to 2022-02-06 (week 2196) 2024-04-17T09:49:08.879 INFO:teuthology.orchestra.run.smithi144.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-17T09:49:08.880 INFO:teuthology.orchestra.run.smithi144.stdout:17 Apr 09:49:08 ntpd[17198]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-17T09:49:08.880 INFO:teuthology.orchestra.run.smithi144.stdout:17 Apr 09:49:08 ntpd[17198]: restrict ::: KOD does nothing without LIMITED. 2024-04-17T09:49:08.880 INFO:teuthology.orchestra.run.smithi144.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-17T09:49:08.881 INFO:teuthology.orchestra.run.smithi144.stdout:17 Apr 09:49:08 ntpd[17198]: Listen and drop on 0 v6wildcard [::]:123 2024-04-17T09:49:08.881 INFO:teuthology.orchestra.run.smithi144.stdout:17 Apr 09:49:08 ntpd[17198]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-17T09:49:08.881 INFO:teuthology.orchestra.run.smithi144.stdout:17 Apr 09:49:08 ntpd[17198]: Listen normally on 2 lo 127.0.0.1:123 2024-04-17T09:49:08.881 INFO:teuthology.orchestra.run.smithi144.stdout:17 Apr 09:49:08 ntpd[17198]: Listen normally on 3 enp3s0f1 172.21.15.144:123 2024-04-17T09:49:08.882 INFO:teuthology.orchestra.run.smithi144.stdout:17 Apr 09:49:08 ntpd[17198]: Listen normally on 4 lo [::1]:123 2024-04-17T09:49:08.882 INFO:teuthology.orchestra.run.smithi144.stdout:17 Apr 09:49:08 ntpd[17198]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f5f%5]:123 2024-04-17T09:49:08.882 INFO:teuthology.orchestra.run.smithi144.stdout:17 Apr 09:49:08 ntpd[17198]: Listening on routing socket on fd #22 for interface updates 2024-04-17T09:49:09.879 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:09 ntpd[17198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:09.879 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:09 ntpd[17198]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:09.879 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:09 ntpd[17198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:09.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:09 ntpd[17198]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:09.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:09 ntpd[17198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:09.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:09 ntpd[17198]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:09.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:09 ntpd[17198]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T09:49:09.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:09 ntpd[17198]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T09:49:09.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:09 ntpd[17198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:09.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:09 ntpd[17198]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:09.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:09 ntpd[17198]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T09:49:09.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:09 ntpd[17198]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T09:49:10.849 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:10 ntpd[17108]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:10.849 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:10 ntpd[17108]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:10.849 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:10 ntpd[17108]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:10.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:10 ntpd[17108]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:10.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:10 ntpd[17108]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T09:49:10.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:10 ntpd[17108]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T09:49:11.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:11 ntpd[17108]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:11.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:11 ntpd[17108]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:11.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:11 ntpd[17108]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:11.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:11 ntpd[17108]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:11.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:11 ntpd[17108]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T09:49:11.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:11 ntpd[17108]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T09:49:11.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:11 ntpd[17198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:11.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:11 ntpd[17198]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:11.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:11 ntpd[17198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:11.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:11 ntpd[17198]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:11.881 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:11 ntpd[17198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:11.881 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:11 ntpd[17198]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:11.881 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:11 ntpd[17198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:11.881 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:11 ntpd[17198]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:11.881 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:11 ntpd[17198]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T09:49:11.881 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:11 ntpd[17198]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T09:49:11.881 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:11 ntpd[17198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:11.882 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:11 ntpd[17198]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:11.882 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:11 ntpd[17198]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T09:49:11.882 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:11 ntpd[17198]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T09:49:12.849 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:12 ntpd[17108]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:12.849 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:12 ntpd[17108]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:12.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:12 ntpd[17108]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:12.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:12 ntpd[17108]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:12.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:12 ntpd[17108]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:12.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:12 ntpd[17108]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:12.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:12 ntpd[17108]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T09:49:12.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:12 ntpd[17108]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T09:49:12.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:12 ntpd[17108]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:12.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:12 ntpd[17108]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:12.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:12 ntpd[17108]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T09:49:12.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:12 ntpd[17108]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T09:49:13.849 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:13 ntpd[17108]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:13.849 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:13 ntpd[17108]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:13.849 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:13 ntpd[17108]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:13.849 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:13 ntpd[17108]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:13.849 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:13 ntpd[17108]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:13.849 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:13 ntpd[17108]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:13.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:13 ntpd[17108]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T09:49:13.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:13 ntpd[17108]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T09:49:13.879 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:13 ntpd[17198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:13.879 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:13 ntpd[17198]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:13.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:13 ntpd[17198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:13.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:13 ntpd[17198]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:13.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:13 ntpd[17198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:13.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:13 ntpd[17198]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:13.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:13 ntpd[17198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:13.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:13 ntpd[17198]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:13.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:13 ntpd[17198]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T09:49:13.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:13 ntpd[17198]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T09:49:13.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:13 ntpd[17198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:13.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:13 ntpd[17198]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:13.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:13 ntpd[17198]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T09:49:13.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:13 ntpd[17198]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T09:49:13.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:13 ntpd[17198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:13.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:13 ntpd[17198]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:13.881 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:13 ntpd[17198]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T09:49:13.881 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:13 ntpd[17198]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T09:49:14.849 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:14 ntpd[17108]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:14.849 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:14 ntpd[17108]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:14.849 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:14 ntpd[17108]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:14 ntpd[17108]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:14 ntpd[17108]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:14 ntpd[17108]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:14 ntpd[17108]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T09:49:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:14 ntpd[17108]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T09:49:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:14 ntpd[17108]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:14 ntpd[17108]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:14 ntpd[17108]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T09:49:14.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:14 ntpd[17108]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T09:49:15.849 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:15 ntpd[17108]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:15.849 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:15 ntpd[17108]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:15.849 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:15 ntpd[17108]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:15.849 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:15 ntpd[17108]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:15.849 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:15 ntpd[17108]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T09:49:15.849 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:15 ntpd[17108]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T09:49:15.879 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:15 ntpd[17198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:15.879 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:15 ntpd[17198]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:15.879 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:15 ntpd[17198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:15.879 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:15 ntpd[17198]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:15.879 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:15 ntpd[17198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:15.879 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:15 ntpd[17198]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:15.879 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:15 ntpd[17198]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:15.879 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:15 ntpd[17198]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:15.879 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:15 ntpd[17198]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T09:49:15.879 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:15 ntpd[17198]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T09:49:15.880 INFO:teuthology.orchestra.run.smithi144.stdout:17 Apr 09:49:15 ntpd[17198]: ntpd: time slew +0.000264 s 2024-04-17T09:49:15.880 INFO:teuthology.orchestra.run.smithi144.stdout:ntpd: time slew +0.000264s 2024-04-17T09:49:15.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:15 ntpd[17198]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-17T09:49:15.880 INFO:teuthology.orchestra.run.smithi144.stderr:17 Apr 09:49:15 ntpd[17198]: can't open /var/log/ntpstats/loopstats.20240417: Permission denied 2024-04-17T09:49:15.998 INFO:teuthology.orchestra.run.smithi144.stdout: remote refid st t when poll reach delay offset jitter 2024-04-17T09:49:15.998 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================== 2024-04-17T09:49:15.998 INFO:teuthology.orchestra.run.smithi144.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T09:49:15.998 INFO:teuthology.orchestra.run.smithi144.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T09:49:15.999 INFO:teuthology.orchestra.run.smithi144.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T09:49:15.999 INFO:teuthology.orchestra.run.smithi144.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T09:49:16.849 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:16 ntpd[17108]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:16.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:16 ntpd[17108]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:16.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:16 ntpd[17108]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:16.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:16 ntpd[17108]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:16.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:16 ntpd[17108]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T09:49:16.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:16 ntpd[17108]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T09:49:16.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:16 ntpd[17108]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T09:49:16.850 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:16 ntpd[17108]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T09:49:16.850 INFO:teuthology.orchestra.run.smithi032.stdout:17 Apr 09:49:16 ntpd[17108]: ntpd: time slew +0.000273 s 2024-04-17T09:49:16.851 INFO:teuthology.orchestra.run.smithi032.stdout:ntpd: time slew +0.000273s 2024-04-17T09:49:16.851 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:16 ntpd[17108]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-17T09:49:16.851 INFO:teuthology.orchestra.run.smithi032.stderr:17 Apr 09:49:16 ntpd[17108]: can't open /var/log/ntpstats/loopstats.20240417: Permission denied 2024-04-17T09:49:16.965 INFO:teuthology.orchestra.run.smithi032.stdout: remote refid st t when poll reach delay offset jitter 2024-04-17T09:49:16.965 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================== 2024-04-17T09:49:16.966 INFO:teuthology.orchestra.run.smithi032.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T09:49:16.966 INFO:teuthology.orchestra.run.smithi032.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T09:49:16.966 INFO:teuthology.orchestra.run.smithi032.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T09:49:16.966 INFO:teuthology.orchestra.run.smithi032.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T09:49:16.967 INFO:teuthology.run_tasks:Running task install... 2024-04-17T09:49:16.976 DEBUG:teuthology.task.install:project ceph 2024-04-17T09:49:16.977 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}} 2024-04-17T09:49:16.977 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'} 2024-04-17T09:49:16.977 INFO:teuthology.task.install:Using flavor: default 2024-04-17T09:49:16.982 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-17T09:49:16.983 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-17T09:49:16.983 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-key list | grep Ceph 2024-04-17T09:49:16.983 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-key list | grep Ceph 2024-04-17T09:49:17.087 INFO:teuthology.orchestra.run.smithi144.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-17T09:49:17.114 INFO:teuthology.orchestra.run.smithi032.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-17T09:49:17.139 INFO:teuthology.orchestra.run.smithi144.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-17T09:49:17.139 INFO:teuthology.orchestra.run.smithi144.stdout:uid [ unknown] Ceph.com (release key) 2024-04-17T09:49:17.140 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-17T09:49:17.140 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-17T09:49:17.141 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T09:49:17.166 INFO:teuthology.orchestra.run.smithi032.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-17T09:49:17.167 INFO:teuthology.orchestra.run.smithi032.stdout:uid [ unknown] Ceph.com (release key) 2024-04-17T09:49:17.167 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-17T09:49:17.168 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-17T09:49:17.168 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T09:49:17.296 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default/ 2024-04-17T09:49:17.297 INFO:teuthology.task.install.deb:Package version is 19.0.0-3133-ga8c84f59-1jammy 2024-04-17T09:49:17.306 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default/ 2024-04-17T09:49:17.306 INFO:teuthology.task.install.deb:Package version is 19.0.0-3133-ga8c84f59-1jammy 2024-04-17T09:49:17.418 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:49:17.418 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-17T09:49:17.438 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get update 2024-04-17T09:49:17.471 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:49:17.471 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-17T09:49:17.490 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get update 2024-04-17T09:49:17.653 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-17T09:49:17.676 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-17T09:49:17.685 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-17T09:49:17.705 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-17T09:49:17.720 INFO:teuthology.orchestra.run.smithi144.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy InRelease 2024-04-17T09:49:17.734 INFO:teuthology.orchestra.run.smithi032.stdout:Ign:2 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy InRelease 2024-04-17T09:49:17.741 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-17T09:49:17.741 INFO:teuthology.orchestra.run.smithi144.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-17T09:49:17.763 INFO:teuthology.orchestra.run.smithi144.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-17T09:49:17.770 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-17T09:49:17.784 INFO:teuthology.orchestra.run.smithi144.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,409 B] 2024-04-17T09:49:17.795 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-17T09:49:17.808 INFO:teuthology.orchestra.run.smithi032.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-17T09:49:17.816 INFO:teuthology.orchestra.run.smithi144.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-04-17T09:49:17.823 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-17T09:49:17.844 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,409 B] 2024-04-17T09:49:17.891 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-04-17T09:49:17.906 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-17T09:49:18.258 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 29.2 kB in 1s (49.2 kB/s) 2024-04-17T09:49:18.262 INFO:teuthology.orchestra.run.smithi144.stdout:Fetched 29.2 kB in 1s (52.8 kB/s) 2024-04-17T09:49:19.234 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-04-17T09:49:19.249 DEBUG:teuthology.orchestra.run.smithi144:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3133-ga8c84f59-1jammy cephadm=19.0.0-3133-ga8c84f59-1jammy ceph-mds=19.0.0-3133-ga8c84f59-1jammy ceph-mgr=19.0.0-3133-ga8c84f59-1jammy ceph-common=19.0.0-3133-ga8c84f59-1jammy ceph-fuse=19.0.0-3133-ga8c84f59-1jammy ceph-test=19.0.0-3133-ga8c84f59-1jammy ceph-volume=19.0.0-3133-ga8c84f59-1jammy radosgw=19.0.0-3133-ga8c84f59-1jammy python3-rados=19.0.0-3133-ga8c84f59-1jammy python3-rgw=19.0.0-3133-ga8c84f59-1jammy python3-cephfs=19.0.0-3133-ga8c84f59-1jammy python3-rbd=19.0.0-3133-ga8c84f59-1jammy libcephfs2=19.0.0-3133-ga8c84f59-1jammy libcephfs-dev=19.0.0-3133-ga8c84f59-1jammy librados2=19.0.0-3133-ga8c84f59-1jammy librbd1=19.0.0-3133-ga8c84f59-1jammy rbd-fuse=19.0.0-3133-ga8c84f59-1jammy python3-cephfs=19.0.0-3133-ga8c84f59-1jammy cephfs-shell=19.0.0-3133-ga8c84f59-1jammy cephfs-top=19.0.0-3133-ga8c84f59-1jammy cephfs-mirror=19.0.0-3133-ga8c84f59-1jammy 2024-04-17T09:49:19.266 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2024-04-17T09:49:19.281 DEBUG:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3133-ga8c84f59-1jammy cephadm=19.0.0-3133-ga8c84f59-1jammy ceph-mds=19.0.0-3133-ga8c84f59-1jammy ceph-mgr=19.0.0-3133-ga8c84f59-1jammy ceph-common=19.0.0-3133-ga8c84f59-1jammy ceph-fuse=19.0.0-3133-ga8c84f59-1jammy ceph-test=19.0.0-3133-ga8c84f59-1jammy ceph-volume=19.0.0-3133-ga8c84f59-1jammy radosgw=19.0.0-3133-ga8c84f59-1jammy python3-rados=19.0.0-3133-ga8c84f59-1jammy python3-rgw=19.0.0-3133-ga8c84f59-1jammy python3-cephfs=19.0.0-3133-ga8c84f59-1jammy python3-rbd=19.0.0-3133-ga8c84f59-1jammy libcephfs2=19.0.0-3133-ga8c84f59-1jammy libcephfs-dev=19.0.0-3133-ga8c84f59-1jammy librados2=19.0.0-3133-ga8c84f59-1jammy librbd1=19.0.0-3133-ga8c84f59-1jammy rbd-fuse=19.0.0-3133-ga8c84f59-1jammy python3-cephfs=19.0.0-3133-ga8c84f59-1jammy cephfs-shell=19.0.0-3133-ga8c84f59-1jammy cephfs-top=19.0.0-3133-ga8c84f59-1jammy cephfs-mirror=19.0.0-3133-ga8c84f59-1jammy 2024-04-17T09:49:19.322 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-04-17T09:49:19.357 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2024-04-17T09:49:19.494 INFO:teuthology.orchestra.run.smithi144.stdout:Building dependency tree... 2024-04-17T09:49:19.495 INFO:teuthology.orchestra.run.smithi144.stdout:Reading state information... 2024-04-17T09:49:19.534 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2024-04-17T09:49:19.534 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2024-04-17T09:49:19.666 INFO:teuthology.orchestra.run.smithi144.stdout:The following packages were automatically installed and are no longer required: 2024-04-17T09:49:19.666 INFO:teuthology.orchestra.run.smithi144.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-17T09:49:19.666 INFO:teuthology.orchestra.run.smithi144.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-17T09:49:19.666 INFO:teuthology.orchestra.run.smithi144.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-17T09:49:19.667 INFO:teuthology.orchestra.run.smithi144.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-17T09:49:19.667 INFO:teuthology.orchestra.run.smithi144.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-17T09:49:19.667 INFO:teuthology.orchestra.run.smithi144.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-17T09:49:19.668 INFO:teuthology.orchestra.run.smithi144.stdout:The following additional packages will be installed: 2024-04-17T09:49:19.668 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-17T09:49:19.668 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-17T09:49:19.668 INFO:teuthology.orchestra.run.smithi144.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-17T09:49:19.668 INFO:teuthology.orchestra.run.smithi144.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-17T09:49:19.669 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-17T09:49:19.669 INFO:teuthology.orchestra.run.smithi144.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-17T09:49:19.670 INFO:teuthology.orchestra.run.smithi144.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-17T09:49:19.670 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-17T09:49:19.670 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-17T09:49:19.670 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-17T09:49:19.670 INFO:teuthology.orchestra.run.smithi144.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-17T09:49:19.670 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-17T09:49:19.670 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-17T09:49:19.670 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-17T09:49:19.670 INFO:teuthology.orchestra.run.smithi144.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-17T09:49:19.670 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-17T09:49:19.670 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-17T09:49:19.670 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-17T09:49:19.670 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-17T09:49:19.670 INFO:teuthology.orchestra.run.smithi144.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-17T09:49:19.671 INFO:teuthology.orchestra.run.smithi144.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-17T09:49:19.671 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-17T09:49:19.671 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-17T09:49:19.671 INFO:teuthology.orchestra.run.smithi144.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-17T09:49:19.671 INFO:teuthology.orchestra.run.smithi144.stdout:Suggested packages: 2024-04-17T09:49:19.671 INFO:teuthology.orchestra.run.smithi144.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-17T09:49:19.671 INFO:teuthology.orchestra.run.smithi144.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-17T09:49:19.671 INFO:teuthology.orchestra.run.smithi144.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-17T09:49:19.671 INFO:teuthology.orchestra.run.smithi144.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-17T09:49:19.671 INFO:teuthology.orchestra.run.smithi144.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-17T09:49:19.672 INFO:teuthology.orchestra.run.smithi144.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-17T09:49:19.672 INFO:teuthology.orchestra.run.smithi144.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-17T09:49:19.672 INFO:teuthology.orchestra.run.smithi144.stdout:Recommended packages: 2024-04-17T09:49:19.672 INFO:teuthology.orchestra.run.smithi144.stdout: btrfs-tools 2024-04-17T09:49:19.714 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages were automatically installed and are no longer required: 2024-04-17T09:49:19.714 INFO:teuthology.orchestra.run.smithi032.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-17T09:49:19.714 INFO:teuthology.orchestra.run.smithi032.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-17T09:49:19.714 INFO:teuthology.orchestra.run.smithi032.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-17T09:49:19.714 INFO:teuthology.orchestra.run.smithi032.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-17T09:49:19.714 INFO:teuthology.orchestra.run.smithi032.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-17T09:49:19.714 INFO:teuthology.orchestra.run.smithi032.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-17T09:49:19.715 INFO:teuthology.orchestra.run.smithi032.stdout:The following additional packages will be installed: 2024-04-17T09:49:19.715 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-17T09:49:19.715 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-17T09:49:19.715 INFO:teuthology.orchestra.run.smithi032.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-17T09:49:19.715 INFO:teuthology.orchestra.run.smithi032.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-17T09:49:19.716 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-17T09:49:19.716 INFO:teuthology.orchestra.run.smithi032.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-17T09:49:19.716 INFO:teuthology.orchestra.run.smithi032.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-17T09:49:19.716 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-17T09:49:19.717 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-17T09:49:19.717 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-17T09:49:19.717 INFO:teuthology.orchestra.run.smithi032.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-17T09:49:19.717 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-17T09:49:19.717 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-17T09:49:19.717 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-17T09:49:19.717 INFO:teuthology.orchestra.run.smithi032.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-17T09:49:19.717 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-17T09:49:19.717 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-17T09:49:19.717 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-17T09:49:19.717 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-17T09:49:19.717 INFO:teuthology.orchestra.run.smithi032.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-17T09:49:19.717 INFO:teuthology.orchestra.run.smithi032.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-17T09:49:19.717 INFO:teuthology.orchestra.run.smithi032.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-17T09:49:19.717 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-17T09:49:19.717 INFO:teuthology.orchestra.run.smithi032.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-17T09:49:19.718 INFO:teuthology.orchestra.run.smithi032.stdout:Suggested packages: 2024-04-17T09:49:19.718 INFO:teuthology.orchestra.run.smithi032.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-17T09:49:19.718 INFO:teuthology.orchestra.run.smithi032.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-17T09:49:19.718 INFO:teuthology.orchestra.run.smithi032.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-17T09:49:19.718 INFO:teuthology.orchestra.run.smithi032.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-17T09:49:19.718 INFO:teuthology.orchestra.run.smithi032.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-17T09:49:19.718 INFO:teuthology.orchestra.run.smithi032.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-17T09:49:19.718 INFO:teuthology.orchestra.run.smithi032.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-17T09:49:19.718 INFO:teuthology.orchestra.run.smithi032.stdout:Recommended packages: 2024-04-17T09:49:19.718 INFO:teuthology.orchestra.run.smithi032.stdout: btrfs-tools 2024-04-17T09:49:19.823 INFO:teuthology.orchestra.run.smithi144.stdout:The following NEW packages will be installed: 2024-04-17T09:49:19.823 INFO:teuthology.orchestra.run.smithi144.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-17T09:49:19.824 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-17T09:49:19.824 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-17T09:49:19.824 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-17T09:49:19.824 INFO:teuthology.orchestra.run.smithi144.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-17T09:49:19.824 INFO:teuthology.orchestra.run.smithi144.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-17T09:49:19.824 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-17T09:49:19.824 INFO:teuthology.orchestra.run.smithi144.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-17T09:49:19.824 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-17T09:49:19.824 INFO:teuthology.orchestra.run.smithi144.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-17T09:49:19.824 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-17T09:49:19.824 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-17T09:49:19.824 INFO:teuthology.orchestra.run.smithi144.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-17T09:49:19.824 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-17T09:49:19.824 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-17T09:49:19.824 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-17T09:49:19.824 INFO:teuthology.orchestra.run.smithi144.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-17T09:49:19.825 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-17T09:49:19.825 INFO:teuthology.orchestra.run.smithi144.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-17T09:49:19.825 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-17T09:49:19.825 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-17T09:49:19.825 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-17T09:49:19.825 INFO:teuthology.orchestra.run.smithi144.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-17T09:49:19.825 INFO:teuthology.orchestra.run.smithi144.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-17T09:49:19.825 INFO:teuthology.orchestra.run.smithi144.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-17T09:49:19.825 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-17T09:49:19.825 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-17T09:49:19.825 INFO:teuthology.orchestra.run.smithi144.stdout: socat unzip xmlstarlet zip 2024-04-17T09:49:19.875 INFO:teuthology.orchestra.run.smithi032.stdout:The following NEW packages will be installed: 2024-04-17T09:49:19.875 INFO:teuthology.orchestra.run.smithi032.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-17T09:49:19.876 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-17T09:49:19.876 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-17T09:49:19.876 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-17T09:49:19.876 INFO:teuthology.orchestra.run.smithi032.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-17T09:49:19.876 INFO:teuthology.orchestra.run.smithi032.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-17T09:49:19.876 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-17T09:49:19.876 INFO:teuthology.orchestra.run.smithi032.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-17T09:49:19.876 INFO:teuthology.orchestra.run.smithi032.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-17T09:49:19.876 INFO:teuthology.orchestra.run.smithi032.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-17T09:49:19.876 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-17T09:49:19.876 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-17T09:49:19.876 INFO:teuthology.orchestra.run.smithi032.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-17T09:49:19.876 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-17T09:49:19.876 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-17T09:49:19.876 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-17T09:49:19.877 INFO:teuthology.orchestra.run.smithi032.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-17T09:49:19.877 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-17T09:49:19.877 INFO:teuthology.orchestra.run.smithi032.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-17T09:49:19.877 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-17T09:49:19.877 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-17T09:49:19.877 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-17T09:49:19.877 INFO:teuthology.orchestra.run.smithi032.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-17T09:49:19.877 INFO:teuthology.orchestra.run.smithi032.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-17T09:49:19.877 INFO:teuthology.orchestra.run.smithi032.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-17T09:49:19.877 INFO:teuthology.orchestra.run.smithi032.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-17T09:49:19.877 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-17T09:49:19.877 INFO:teuthology.orchestra.run.smithi032.stdout: socat unzip xmlstarlet zip 2024-04-17T09:49:19.900 INFO:teuthology.orchestra.run.smithi144.stdout:0 upgraded, 124 newly installed, 0 to remove and 305 not upgraded. 2024-04-17T09:49:19.900 INFO:teuthology.orchestra.run.smithi144.stdout:Need to get 182 MB of archives. 2024-04-17T09:49:19.900 INFO:teuthology.orchestra.run.smithi144.stdout:After this operation, 831 MB of additional disk space will be used. 2024-04-17T09:49:19.900 INFO:teuthology.orchestra.run.smithi144.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-17T09:49:19.921 INFO:teuthology.orchestra.run.smithi144.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-17T09:49:19.932 INFO:teuthology.orchestra.run.smithi144.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-17T09:49:19.952 INFO:teuthology.orchestra.run.smithi032.stdout:0 upgraded, 124 newly installed, 0 to remove and 305 not upgraded. 2024-04-17T09:49:19.952 INFO:teuthology.orchestra.run.smithi032.stdout:Need to get 182 MB of archives. 2024-04-17T09:49:19.952 INFO:teuthology.orchestra.run.smithi032.stdout:After this operation, 831 MB of additional disk space will be used. 2024-04-17T09:49:19.952 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-17T09:49:19.974 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-17T09:49:19.984 INFO:teuthology.orchestra.run.smithi144.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-17T09:49:19.984 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-17T09:49:20.013 INFO:teuthology.orchestra.run.smithi144.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-17T09:49:20.018 INFO:teuthology.orchestra.run.smithi144.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-17T09:49:20.027 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3133-ga8c84f59-1jammy [3,555 kB] 2024-04-17T09:49:20.035 INFO:teuthology.orchestra.run.smithi144.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-17T09:49:20.046 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-17T09:49:20.060 INFO:teuthology.orchestra.run.smithi144.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3133-ga8c84f59-1jammy [3,555 kB] 2024-04-17T09:49:20.076 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-17T09:49:20.082 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-17T09:49:20.103 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-17T09:49:20.120 INFO:teuthology.orchestra.run.smithi144.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-17T09:49:20.122 INFO:teuthology.orchestra.run.smithi144.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-17T09:49:20.129 INFO:teuthology.orchestra.run.smithi144.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-17T09:49:20.132 INFO:teuthology.orchestra.run.smithi144.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-17T09:49:20.132 INFO:teuthology.orchestra.run.smithi144.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-17T09:49:20.138 INFO:teuthology.orchestra.run.smithi144.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-17T09:49:20.140 INFO:teuthology.orchestra.run.smithi144.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-17T09:49:20.142 INFO:teuthology.orchestra.run.smithi144.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-17T09:49:20.145 INFO:teuthology.orchestra.run.smithi144.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-17T09:49:20.145 INFO:teuthology.orchestra.run.smithi144.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-17T09:49:20.162 INFO:teuthology.orchestra.run.smithi144.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-17T09:49:20.168 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-17T09:49:20.170 INFO:teuthology.orchestra.run.smithi032.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-17T09:49:20.179 INFO:teuthology.orchestra.run.smithi032.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-17T09:49:20.182 INFO:teuthology.orchestra.run.smithi032.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-17T09:49:20.182 INFO:teuthology.orchestra.run.smithi032.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-17T09:49:20.183 INFO:teuthology.orchestra.run.smithi144.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-17T09:49:20.185 INFO:teuthology.orchestra.run.smithi144.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-17T09:49:20.186 INFO:teuthology.orchestra.run.smithi144.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-17T09:49:20.187 INFO:teuthology.orchestra.run.smithi144.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-17T09:49:20.188 INFO:teuthology.orchestra.run.smithi144.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-17T09:49:20.188 INFO:teuthology.orchestra.run.smithi144.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-17T09:49:20.189 INFO:teuthology.orchestra.run.smithi144.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-17T09:49:20.189 INFO:teuthology.orchestra.run.smithi144.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-17T09:49:20.190 INFO:teuthology.orchestra.run.smithi144.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-17T09:49:20.190 INFO:teuthology.orchestra.run.smithi032.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-17T09:49:20.192 INFO:teuthology.orchestra.run.smithi032.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-17T09:49:20.194 INFO:teuthology.orchestra.run.smithi032.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-17T09:49:20.195 INFO:teuthology.orchestra.run.smithi032.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-17T09:49:20.195 INFO:teuthology.orchestra.run.smithi032.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-17T09:49:20.206 INFO:teuthology.orchestra.run.smithi144.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-17T09:49:20.210 INFO:teuthology.orchestra.run.smithi144.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-17T09:49:20.213 INFO:teuthology.orchestra.run.smithi144.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-17T09:49:20.214 INFO:teuthology.orchestra.run.smithi144.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-17T09:49:20.215 INFO:teuthology.orchestra.run.smithi144.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-17T09:49:20.215 INFO:teuthology.orchestra.run.smithi144.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-17T09:49:20.216 INFO:teuthology.orchestra.run.smithi032.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-17T09:49:20.217 INFO:teuthology.orchestra.run.smithi144.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-17T09:49:20.217 INFO:teuthology.orchestra.run.smithi144.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-17T09:49:20.217 INFO:teuthology.orchestra.run.smithi144.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-17T09:49:20.218 INFO:teuthology.orchestra.run.smithi032.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-17T09:49:20.220 INFO:teuthology.orchestra.run.smithi032.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-17T09:49:20.221 INFO:teuthology.orchestra.run.smithi032.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-17T09:49:20.222 INFO:teuthology.orchestra.run.smithi032.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-17T09:49:20.223 INFO:teuthology.orchestra.run.smithi032.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-17T09:49:20.226 INFO:teuthology.orchestra.run.smithi144.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-17T09:49:20.237 INFO:teuthology.orchestra.run.smithi032.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-17T09:49:20.237 INFO:teuthology.orchestra.run.smithi032.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-17T09:49:20.237 INFO:teuthology.orchestra.run.smithi032.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-17T09:49:20.238 INFO:teuthology.orchestra.run.smithi144.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-17T09:49:20.238 INFO:teuthology.orchestra.run.smithi032.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-17T09:49:20.238 INFO:teuthology.orchestra.run.smithi144.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-17T09:49:20.243 INFO:teuthology.orchestra.run.smithi144.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-17T09:49:20.244 INFO:teuthology.orchestra.run.smithi144.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-17T09:49:20.245 INFO:teuthology.orchestra.run.smithi144.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-17T09:49:20.246 INFO:teuthology.orchestra.run.smithi144.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-17T09:49:20.247 INFO:teuthology.orchestra.run.smithi144.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-17T09:49:20.249 INFO:teuthology.orchestra.run.smithi144.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-17T09:49:20.249 INFO:teuthology.orchestra.run.smithi144.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-17T09:49:20.257 INFO:teuthology.orchestra.run.smithi032.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-17T09:49:20.257 INFO:teuthology.orchestra.run.smithi032.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-17T09:49:20.260 INFO:teuthology.orchestra.run.smithi032.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-17T09:49:20.261 INFO:teuthology.orchestra.run.smithi032.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-17T09:49:20.261 INFO:teuthology.orchestra.run.smithi032.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-17T09:49:20.261 INFO:teuthology.orchestra.run.smithi032.stdout:Get:34 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3133-ga8c84f59-1jammy [3,224 kB] 2024-04-17T09:49:20.262 INFO:teuthology.orchestra.run.smithi032.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-17T09:49:20.267 INFO:teuthology.orchestra.run.smithi144.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-17T09:49:20.268 INFO:teuthology.orchestra.run.smithi144.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-17T09:49:20.270 INFO:teuthology.orchestra.run.smithi144.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-17T09:49:20.271 INFO:teuthology.orchestra.run.smithi144.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-17T09:49:20.272 INFO:teuthology.orchestra.run.smithi144.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-17T09:49:20.278 INFO:teuthology.orchestra.run.smithi032.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-17T09:49:20.278 INFO:teuthology.orchestra.run.smithi032.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-17T09:49:20.278 INFO:teuthology.orchestra.run.smithi032.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-17T09:49:20.280 INFO:teuthology.orchestra.run.smithi032.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-17T09:49:20.286 INFO:teuthology.orchestra.run.smithi144.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-17T09:49:20.309 INFO:teuthology.orchestra.run.smithi032.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-17T09:49:20.310 INFO:teuthology.orchestra.run.smithi032.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-17T09:49:20.318 INFO:teuthology.orchestra.run.smithi032.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-17T09:49:20.318 INFO:teuthology.orchestra.run.smithi032.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-17T09:49:20.318 INFO:teuthology.orchestra.run.smithi032.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-17T09:49:20.319 INFO:teuthology.orchestra.run.smithi032.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-17T09:49:20.345 INFO:teuthology.orchestra.run.smithi032.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-17T09:49:20.347 INFO:teuthology.orchestra.run.smithi032.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-17T09:49:20.348 INFO:teuthology.orchestra.run.smithi032.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-17T09:49:20.349 INFO:teuthology.orchestra.run.smithi032.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-17T09:49:20.373 INFO:teuthology.orchestra.run.smithi032.stdout:Get:50 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3133-ga8c84f59-1jammy [716 kB] 2024-04-17T09:49:20.382 INFO:teuthology.orchestra.run.smithi144.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-17T09:49:20.383 INFO:teuthology.orchestra.run.smithi144.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-17T09:49:20.385 INFO:teuthology.orchestra.run.smithi144.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-17T09:49:20.385 INFO:teuthology.orchestra.run.smithi144.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-17T09:49:20.386 INFO:teuthology.orchestra.run.smithi032.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-17T09:49:20.388 INFO:teuthology.orchestra.run.smithi032.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-17T09:49:20.389 INFO:teuthology.orchestra.run.smithi032.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-17T09:49:20.390 INFO:teuthology.orchestra.run.smithi032.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-17T09:49:20.392 INFO:teuthology.orchestra.run.smithi032.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-17T09:49:20.402 INFO:teuthology.orchestra.run.smithi032.stdout:Get:56 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3133-ga8c84f59-1jammy [356 kB] 2024-04-17T09:49:20.414 INFO:teuthology.orchestra.run.smithi032.stdout:Get:57 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3133-ga8c84f59-1jammy [32.8 kB] 2024-04-17T09:49:20.415 INFO:teuthology.orchestra.run.smithi032.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3133-ga8c84f59-1jammy [180 kB] 2024-04-17T09:49:20.423 INFO:teuthology.orchestra.run.smithi032.stdout:Get:59 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3133-ga8c84f59-1jammy [66.9 kB] 2024-04-17T09:49:20.426 INFO:teuthology.orchestra.run.smithi032.stdout:Get:60 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3133-ga8c84f59-1jammy [330 kB] 2024-04-17T09:49:20.437 INFO:teuthology.orchestra.run.smithi032.stdout:Get:61 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3133-ga8c84f59-1jammy [6,890 kB] 2024-04-17T09:49:20.443 INFO:teuthology.orchestra.run.smithi144.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-17T09:49:20.526 INFO:teuthology.orchestra.run.smithi032.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-17T09:49:20.527 INFO:teuthology.orchestra.run.smithi032.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-17T09:49:20.529 INFO:teuthology.orchestra.run.smithi032.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-17T09:49:20.530 INFO:teuthology.orchestra.run.smithi032.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-17T09:49:20.530 INFO:teuthology.orchestra.run.smithi144.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-17T09:49:20.531 INFO:teuthology.orchestra.run.smithi032.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-17T09:49:20.532 INFO:teuthology.orchestra.run.smithi144.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-17T09:49:20.533 INFO:teuthology.orchestra.run.smithi144.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-17T09:49:20.558 INFO:teuthology.orchestra.run.smithi144.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-17T09:49:20.559 INFO:teuthology.orchestra.run.smithi144.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-17T09:49:20.560 INFO:teuthology.orchestra.run.smithi144.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-17T09:49:20.560 INFO:teuthology.orchestra.run.smithi144.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-17T09:49:20.561 INFO:teuthology.orchestra.run.smithi144.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-17T09:49:20.562 INFO:teuthology.orchestra.run.smithi144.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-17T09:49:20.564 INFO:teuthology.orchestra.run.smithi032.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-17T09:49:20.567 INFO:teuthology.orchestra.run.smithi032.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-17T09:49:20.567 INFO:teuthology.orchestra.run.smithi032.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-17T09:49:20.573 INFO:teuthology.orchestra.run.smithi144.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-17T09:49:20.586 INFO:teuthology.orchestra.run.smithi032.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-17T09:49:20.588 INFO:teuthology.orchestra.run.smithi032.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-17T09:49:20.593 INFO:teuthology.orchestra.run.smithi144.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-17T09:49:20.597 INFO:teuthology.orchestra.run.smithi144.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-17T09:49:20.600 INFO:teuthology.orchestra.run.smithi144.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-17T09:49:20.601 INFO:teuthology.orchestra.run.smithi144.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-17T09:49:20.602 INFO:teuthology.orchestra.run.smithi144.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-17T09:49:20.605 INFO:teuthology.orchestra.run.smithi032.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-17T09:49:20.606 INFO:teuthology.orchestra.run.smithi144.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-17T09:49:20.606 INFO:teuthology.orchestra.run.smithi032.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-17T09:49:20.607 INFO:teuthology.orchestra.run.smithi032.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-17T09:49:20.608 INFO:teuthology.orchestra.run.smithi032.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-17T09:49:20.608 INFO:teuthology.orchestra.run.smithi032.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-17T09:49:20.609 INFO:teuthology.orchestra.run.smithi144.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-17T09:49:20.609 INFO:teuthology.orchestra.run.smithi032.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-17T09:49:20.610 INFO:teuthology.orchestra.run.smithi144.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-17T09:49:20.611 INFO:teuthology.orchestra.run.smithi144.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-17T09:49:20.626 INFO:teuthology.orchestra.run.smithi032.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-17T09:49:20.628 INFO:teuthology.orchestra.run.smithi032.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-17T09:49:20.630 INFO:teuthology.orchestra.run.smithi032.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-17T09:49:20.631 INFO:teuthology.orchestra.run.smithi032.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-17T09:49:20.634 INFO:teuthology.orchestra.run.smithi144.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-17T09:49:20.649 INFO:teuthology.orchestra.run.smithi032.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-17T09:49:20.651 INFO:teuthology.orchestra.run.smithi032.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-17T09:49:20.652 INFO:teuthology.orchestra.run.smithi032.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-17T09:49:20.653 INFO:teuthology.orchestra.run.smithi032.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-17T09:49:20.654 INFO:teuthology.orchestra.run.smithi032.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-17T09:49:20.659 INFO:teuthology.orchestra.run.smithi032.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-17T09:49:20.669 INFO:teuthology.orchestra.run.smithi144.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-17T09:49:20.670 INFO:teuthology.orchestra.run.smithi144.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-17T09:49:20.673 INFO:teuthology.orchestra.run.smithi144.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-17T09:49:20.674 INFO:teuthology.orchestra.run.smithi144.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-17T09:49:20.674 INFO:teuthology.orchestra.run.smithi032.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-17T09:49:20.674 INFO:teuthology.orchestra.run.smithi144.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-17T09:49:20.675 INFO:teuthology.orchestra.run.smithi144.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-17T09:49:20.676 INFO:teuthology.orchestra.run.smithi144.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-17T09:49:20.677 INFO:teuthology.orchestra.run.smithi032.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-17T09:49:20.677 INFO:teuthology.orchestra.run.smithi144.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-17T09:49:20.678 INFO:teuthology.orchestra.run.smithi032.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-17T09:49:20.678 INFO:teuthology.orchestra.run.smithi032.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-17T09:49:20.684 INFO:teuthology.orchestra.run.smithi144.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-17T09:49:20.688 INFO:teuthology.orchestra.run.smithi032.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3133-ga8c84f59-1jammy [112 kB] 2024-04-17T09:49:20.689 INFO:teuthology.orchestra.run.smithi032.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3133-ga8c84f59-1jammy [466 kB] 2024-04-17T09:49:20.691 INFO:teuthology.orchestra.run.smithi144.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-17T09:49:20.695 INFO:teuthology.orchestra.run.smithi032.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-17T09:49:20.697 INFO:teuthology.orchestra.run.smithi032.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-17T09:49:20.698 INFO:teuthology.orchestra.run.smithi032.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-17T09:49:20.705 INFO:teuthology.orchestra.run.smithi032.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3133-ga8c84f59-1jammy [25.8 MB] 2024-04-17T09:49:20.707 INFO:teuthology.orchestra.run.smithi032.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-17T09:49:20.708 INFO:teuthology.orchestra.run.smithi032.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-17T09:49:20.708 INFO:teuthology.orchestra.run.smithi032.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-17T09:49:20.718 INFO:teuthology.orchestra.run.smithi144.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-17T09:49:20.722 INFO:teuthology.orchestra.run.smithi144.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-17T09:49:20.722 INFO:teuthology.orchestra.run.smithi144.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-17T09:49:20.731 INFO:teuthology.orchestra.run.smithi032.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-17T09:49:20.732 INFO:teuthology.orchestra.run.smithi032.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-17T09:49:21.257 INFO:teuthology.orchestra.run.smithi144.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3133-ga8c84f59-1jammy [3,224 kB] 2024-04-17T09:49:21.672 INFO:teuthology.orchestra.run.smithi032.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3133-ga8c84f59-1jammy [5,092 kB] 2024-04-17T09:49:21.861 INFO:teuthology.orchestra.run.smithi032.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3133-ga8c84f59-1jammy [244 kB] 2024-04-17T09:49:21.868 INFO:teuthology.orchestra.run.smithi032.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3133-ga8c84f59-1jammy [124 kB] 2024-04-17T09:49:21.873 INFO:teuthology.orchestra.run.smithi032.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3133-ga8c84f59-1jammy [1,529 kB] 2024-04-17T09:49:21.932 INFO:teuthology.orchestra.run.smithi032.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3133-ga8c84f59-1jammy [6,259 kB] 2024-04-17T09:49:21.950 INFO:teuthology.orchestra.run.smithi144.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3133-ga8c84f59-1jammy [716 kB] 2024-04-17T09:49:22.107 INFO:teuthology.orchestra.run.smithi144.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3133-ga8c84f59-1jammy [356 kB] 2024-04-17T09:49:22.176 INFO:teuthology.orchestra.run.smithi032.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3133-ga8c84f59-1jammy [22.8 MB] 2024-04-17T09:49:22.187 INFO:teuthology.orchestra.run.smithi144.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3133-ga8c84f59-1jammy [32.8 kB] 2024-04-17T09:49:22.196 INFO:teuthology.orchestra.run.smithi144.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3133-ga8c84f59-1jammy [180 kB] 2024-04-17T09:49:22.234 INFO:teuthology.orchestra.run.smithi144.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3133-ga8c84f59-1jammy [66.9 kB] 2024-04-17T09:49:22.249 INFO:teuthology.orchestra.run.smithi144.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3133-ga8c84f59-1jammy [330 kB] 2024-04-17T09:49:22.324 INFO:teuthology.orchestra.run.smithi144.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3133-ga8c84f59-1jammy [6,890 kB] 2024-04-17T09:49:23.055 INFO:teuthology.orchestra.run.smithi032.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3133-ga8c84f59-1jammy [14.0 kB] 2024-04-17T09:49:23.055 INFO:teuthology.orchestra.run.smithi032.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3133-ga8c84f59-1jammy [873 kB] 2024-04-17T09:49:23.081 INFO:teuthology.orchestra.run.smithi032.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3133-ga8c84f59-1jammy [2,437 kB] 2024-04-17T09:49:23.174 INFO:teuthology.orchestra.run.smithi032.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3133-ga8c84f59-1jammy [791 kB] 2024-04-17T09:49:23.204 INFO:teuthology.orchestra.run.smithi032.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3133-ga8c84f59-1jammy [148 kB] 2024-04-17T09:49:23.210 INFO:teuthology.orchestra.run.smithi032.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3133-ga8c84f59-1jammy [2,297 kB] 2024-04-17T09:49:23.298 INFO:teuthology.orchestra.run.smithi032.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3133-ga8c84f59-1jammy [8,625 kB] 2024-04-17T09:49:23.628 INFO:teuthology.orchestra.run.smithi032.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3133-ga8c84f59-1jammy [14.2 kB] 2024-04-17T09:49:23.629 INFO:teuthology.orchestra.run.smithi032.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3133-ga8c84f59-1jammy [51.7 MB] 2024-04-17T09:49:23.922 INFO:teuthology.orchestra.run.smithi144.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3133-ga8c84f59-1jammy [112 kB] 2024-04-17T09:49:23.950 INFO:teuthology.orchestra.run.smithi144.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3133-ga8c84f59-1jammy [466 kB] 2024-04-17T09:49:24.062 INFO:teuthology.orchestra.run.smithi144.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3133-ga8c84f59-1jammy [25.8 MB] 2024-04-17T09:49:25.596 INFO:teuthology.orchestra.run.smithi032.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3133-ga8c84f59-1jammy [124 kB] 2024-04-17T09:49:25.598 INFO:teuthology.orchestra.run.smithi032.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-3133-ga8c84f59-1jammy [232 kB] 2024-04-17T09:49:25.600 INFO:teuthology.orchestra.run.smithi032.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-3133-ga8c84f59-1jammy [34.0 kB] 2024-04-17T09:49:25.601 INFO:teuthology.orchestra.run.smithi032.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-3133-ga8c84f59-1jammy [23.4 kB] 2024-04-17T09:49:25.601 INFO:teuthology.orchestra.run.smithi032.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3133-ga8c84f59-1jammy [38.9 kB] 2024-04-17T09:49:25.602 INFO:teuthology.orchestra.run.smithi032.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3133-ga8c84f59-1jammy [13.6 MB] 2024-04-17T09:49:26.104 INFO:teuthology.orchestra.run.smithi032.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3133-ga8c84f59-1jammy [91.4 kB] 2024-04-17T09:49:26.779 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 182 MB in 6s (29.2 MB/s) 2024-04-17T09:49:26.922 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-17T09:49:26.990 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132922 files and directories currently installed.) 2024-04-17T09:49:26.993 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-17T09:49:27.038 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-17T09:49:27.490 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-17T09:49:27.505 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-17T09:49:27.549 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-17T09:49:27.810 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cmd2. 2024-04-17T09:49:27.825 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-17T09:49:27.868 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-17T09:49:28.162 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-17T09:49:28.178 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-17T09:49:28.254 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-17T09:49:28.448 INFO:teuthology.orchestra.run.smithi144.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3133-ga8c84f59-1jammy [5,092 kB] 2024-04-17T09:49:28.591 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-17T09:49:28.606 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-17T09:49:28.649 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-17T09:49:29.006 INFO:teuthology.orchestra.run.smithi144.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3133-ga8c84f59-1jammy [244 kB] 2024-04-17T09:49:29.010 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-17T09:49:29.025 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-17T09:49:29.035 INFO:teuthology.orchestra.run.smithi144.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3133-ga8c84f59-1jammy [124 kB] 2024-04-17T09:49:29.044 INFO:teuthology.orchestra.run.smithi144.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3133-ga8c84f59-1jammy [1,529 kB] 2024-04-17T09:49:29.068 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-17T09:49:29.212 INFO:teuthology.orchestra.run.smithi144.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3133-ga8c84f59-1jammy [6,259 kB] 2024-04-17T09:49:29.464 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-17T09:49:29.479 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-17T09:49:29.522 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T09:49:29.873 INFO:teuthology.orchestra.run.smithi144.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3133-ga8c84f59-1jammy [22.8 MB] 2024-04-17T09:49:29.925 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-17T09:49:29.940 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-17T09:49:30.000 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T09:49:30.361 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-17T09:49:30.376 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-17T09:49:30.419 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T09:49:30.739 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-17T09:49:30.754 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-17T09:49:30.789 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-17T09:49:31.075 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librados2. 2024-04-17T09:49:31.091 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../010-librados2_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:31.125 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librados2 (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:31.536 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librbd1. 2024-04-17T09:49:31.551 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../011-librbd1_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:31.594 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librbd1 (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:32.005 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libcephfs2. 2024-04-17T09:49:32.020 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:32.040 INFO:teuthology.orchestra.run.smithi144.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3133-ga8c84f59-1jammy [14.0 kB] 2024-04-17T09:49:32.041 INFO:teuthology.orchestra.run.smithi144.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3133-ga8c84f59-1jammy [873 kB] 2024-04-17T09:49:32.063 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libcephfs2 (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:32.113 INFO:teuthology.orchestra.run.smithi144.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3133-ga8c84f59-1jammy [2,437 kB] 2024-04-17T09:49:32.301 INFO:teuthology.orchestra.run.smithi144.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3133-ga8c84f59-1jammy [791 kB] 2024-04-17T09:49:32.341 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rados. 2024-04-17T09:49:32.356 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../013-python3-rados_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:32.364 INFO:teuthology.orchestra.run.smithi144.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3133-ga8c84f59-1jammy [148 kB] 2024-04-17T09:49:32.373 INFO:teuthology.orchestra.run.smithi144.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3133-ga8c84f59-1jammy [2,297 kB] 2024-04-17T09:49:32.399 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rados (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:32.570 INFO:teuthology.orchestra.run.smithi144.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3133-ga8c84f59-1jammy [8,625 kB] 2024-04-17T09:49:32.702 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-17T09:49:32.717 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-3133-ga8c84f59-1jammy_all.deb ... 2024-04-17T09:49:32.752 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-ceph-argparse (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:32.979 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cephfs. 2024-04-17T09:49:32.994 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:33.037 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cephfs (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:33.223 INFO:teuthology.orchestra.run.smithi144.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3133-ga8c84f59-1jammy [14.2 kB] 2024-04-17T09:49:33.223 INFO:teuthology.orchestra.run.smithi144.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3133-ga8c84f59-1jammy [51.7 MB] 2024-04-17T09:49:33.348 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-17T09:49:33.363 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-3133-ga8c84f59-1jammy_all.deb ... 2024-04-17T09:49:33.406 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-ceph-common (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:33.760 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-prettytable. 2024-04-17T09:49:33.775 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-17T09:49:33.817 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-17T09:49:34.078 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rbd. 2024-04-17T09:49:34.094 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:34.137 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rbd (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:34.473 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-17T09:49:34.488 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-17T09:49:34.522 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-17T09:49:34.777 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-17T09:49:34.792 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-17T09:49:34.835 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-17T09:49:35.155 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-17T09:49:35.170 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-17T09:49:35.213 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-17T09:49:35.508 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package lua5.1. 2024-04-17T09:49:35.523 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-17T09:49:35.558 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-17T09:49:35.852 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package lua-any. 2024-04-17T09:49:35.868 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-17T09:49:35.911 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-17T09:49:36.138 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package zip. 2024-04-17T09:49:36.153 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-17T09:49:36.188 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking zip (3.0-12build2) ... 2024-04-17T09:49:36.252 INFO:teuthology.orchestra.run.smithi144.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3133-ga8c84f59-1jammy [124 kB] 2024-04-17T09:49:36.298 INFO:teuthology.orchestra.run.smithi144.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-3133-ga8c84f59-1jammy [232 kB] 2024-04-17T09:49:36.348 INFO:teuthology.orchestra.run.smithi144.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-3133-ga8c84f59-1jammy [34.0 kB] 2024-04-17T09:49:36.389 INFO:teuthology.orchestra.run.smithi144.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-3133-ga8c84f59-1jammy [23.4 kB] 2024-04-17T09:49:36.389 INFO:teuthology.orchestra.run.smithi144.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3133-ga8c84f59-1jammy [38.9 kB] 2024-04-17T09:49:36.390 INFO:teuthology.orchestra.run.smithi144.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3133-ga8c84f59-1jammy [13.6 MB] 2024-04-17T09:49:36.831 INFO:teuthology.orchestra.run.smithi144.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3133-ga8c84f59-1jammy [91.4 kB] 2024-04-17T09:49:36.908 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package unzip. 2024-04-17T09:49:36.924 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-17T09:49:36.957 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-17T09:49:37.328 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package luarocks. 2024-04-17T09:49:37.344 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-17T09:49:37.386 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-17T09:49:37.493 INFO:teuthology.orchestra.run.smithi144.stdout:Fetched 182 MB in 17s (10.7 MB/s) 2024-04-17T09:49:37.617 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-17T09:49:37.684 INFO:teuthology.orchestra.run.smithi144.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132922 files and directories currently installed.) 2024-04-17T09:49:37.687 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-17T09:49:37.749 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-17T09:49:37.785 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librgw2. 2024-04-17T09:49:37.800 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../027-librgw2_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:37.835 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librgw2 (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:38.027 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-17T09:49:38.041 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-17T09:49:38.085 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-17T09:49:38.321 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rgw. 2024-04-17T09:49:38.337 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:38.387 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rgw (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:38.405 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cmd2. 2024-04-17T09:49:38.418 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-17T09:49:38.463 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-17T09:49:38.757 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-17T09:49:38.772 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-17T09:49:38.815 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-17T09:49:38.816 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-17T09:49:38.830 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-17T09:49:38.916 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-17T09:49:39.118 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libradosstriper1. 2024-04-17T09:49:39.134 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:39.176 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libradosstriper1 (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:39.278 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-17T09:49:39.292 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-17T09:49:39.336 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-17T09:49:39.563 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-common. 2024-04-17T09:49:39.579 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../031-ceph-common_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:39.621 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-common (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:39.706 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-17T09:49:39.721 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-17T09:49:39.764 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-17T09:49:40.176 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-17T09:49:40.191 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-17T09:49:40.243 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T09:49:40.663 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-17T09:49:40.677 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-17T09:49:40.721 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T09:49:40.743 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-base. 2024-04-17T09:49:40.758 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../032-ceph-base_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:40.810 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-base (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:41.049 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-17T09:49:41.063 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-17T09:49:41.132 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T09:49:41.388 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-17T09:49:41.404 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-17T09:49:41.444 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-17T09:49:41.447 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-17T09:49:41.458 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-17T09:49:41.502 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-17T09:49:41.749 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cheroot. 2024-04-17T09:49:41.765 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-17T09:49:41.807 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-17T09:49:41.822 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package librados2. 2024-04-17T09:49:41.836 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../010-librados2_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:41.879 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librados2 (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:42.094 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-17T09:49:42.109 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-17T09:49:42.143 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-17T09:49:42.307 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package librbd1. 2024-04-17T09:49:42.322 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../011-librbd1_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:42.359 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librbd1 (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:42.413 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-17T09:49:42.429 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-17T09:49:42.496 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-17T09:49:42.799 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-17T09:49:42.814 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-17T09:49:42.857 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-17T09:49:42.954 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libcephfs2. 2024-04-17T09:49:42.968 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:43.012 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libcephfs2 (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:43.152 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-tempora. 2024-04-17T09:49:43.167 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-17T09:49:43.210 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-17T09:49:43.306 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rados. 2024-04-17T09:49:43.321 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../013-python3-rados_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:43.364 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rados (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:43.513 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-portend. 2024-04-17T09:49:43.528 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-17T09:49:43.571 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-17T09:49:43.710 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-17T09:49:43.724 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-3133-ga8c84f59-1jammy_all.deb ... 2024-04-17T09:49:43.767 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-ceph-argparse (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:43.907 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-17T09:49:43.922 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-17T09:49:43.965 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-17T09:49:43.978 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cephfs. 2024-04-17T09:49:43.993 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:44.037 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cephfs (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:44.210 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-17T09:49:44.225 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-17T09:49:44.268 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-17T09:49:44.381 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-17T09:49:44.396 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-3133-ga8c84f59-1jammy_all.deb ... 2024-04-17T09:49:44.439 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-ceph-common (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:44.621 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-natsort. 2024-04-17T09:49:44.637 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-17T09:49:44.679 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-17T09:49:44.784 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-prettytable. 2024-04-17T09:49:44.798 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-17T09:49:44.834 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-17T09:49:44.941 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-logutils. 2024-04-17T09:49:44.956 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-17T09:49:45.015 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-17T09:49:45.062 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rbd. 2024-04-17T09:49:45.077 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:45.120 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rbd (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:45.327 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-17T09:49:45.342 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-17T09:49:45.385 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-17T09:49:45.532 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-17T09:49:45.546 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-17T09:49:45.590 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-17T09:49:45.671 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-mako. 2024-04-17T09:49:45.686 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-17T09:49:45.729 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-17T09:49:45.909 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-17T09:49:45.924 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-17T09:49:45.967 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-17T09:49:46.066 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-17T09:49:46.081 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-17T09:49:46.124 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-17T09:49:46.245 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-17T09:49:46.260 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-17T09:49:46.320 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-17T09:49:46.435 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-17T09:49:46.451 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-17T09:49:46.493 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-17T09:49:46.640 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package lua5.1. 2024-04-17T09:49:46.655 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-17T09:49:46.698 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-17T09:49:46.754 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-webob. 2024-04-17T09:49:46.770 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-17T09:49:46.804 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-17T09:49:47.035 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package lua-any. 2024-04-17T09:49:47.050 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-17T09:49:47.093 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-17T09:49:47.098 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-waitress. 2024-04-17T09:49:47.114 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-17T09:49:47.151 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-17T09:49:47.354 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package zip. 2024-04-17T09:49:47.369 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-17T09:49:47.404 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking zip (3.0-12build2) ... 2024-04-17T09:49:47.426 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-tempita. 2024-04-17T09:49:47.442 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-17T09:49:47.509 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-17T09:49:47.699 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package unzip. 2024-04-17T09:49:47.714 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-17T09:49:47.757 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-17T09:49:47.820 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-paste. 2024-04-17T09:49:47.836 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-17T09:49:47.879 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-17T09:49:48.203 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package luarocks. 2024-04-17T09:49:48.215 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-17T09:49:48.218 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-17T09:49:48.231 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-17T09:49:48.261 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-17T09:49:48.273 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-17T09:49:48.576 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-17T09:49:48.591 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-17T09:49:48.626 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-17T09:49:48.652 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package librgw2. 2024-04-17T09:49:48.666 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../027-librgw2_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:48.711 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librgw2 (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:48.937 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-webtest. 2024-04-17T09:49:48.953 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-17T09:49:48.995 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-17T09:49:49.180 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rgw. 2024-04-17T09:49:49.194 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:49.238 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rgw (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:49.315 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pecan. 2024-04-17T09:49:49.331 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-17T09:49:49.373 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-17T09:49:49.583 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-17T09:49:49.597 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-17T09:49:49.640 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-17T09:49:49.711 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-certifi. 2024-04-17T09:49:49.726 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-17T09:49:49.769 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-17T09:49:49.986 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libradosstriper1. 2024-04-17T09:49:50.001 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:50.044 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libradosstriper1 (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:50.139 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-idna. 2024-04-17T09:49:50.154 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-17T09:49:50.196 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-17T09:49:50.489 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-common. 2024-04-17T09:49:50.504 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../031-ceph-common_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:50.508 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-urllib3. 2024-04-17T09:49:50.523 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-17T09:49:50.547 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-common (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:50.566 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-17T09:49:50.894 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-requests. 2024-04-17T09:49:50.909 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-17T09:49:50.944 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-17T09:49:51.230 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-17T09:49:51.246 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-17T09:49:51.280 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-17T09:49:51.541 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-17T09:49:51.556 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-3133-ga8c84f59-1jammy_all.deb ... 2024-04-17T09:49:51.599 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:51.719 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-base. 2024-04-17T09:49:51.734 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../032-ceph-base_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:51.786 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-base (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:52.196 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-17T09:49:52.211 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:52.254 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:52.590 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-17T09:49:52.606 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-17T09:49:52.648 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-17T09:49:53.182 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-17T09:49:53.197 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-17T09:49:53.232 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-17T09:49:53.501 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-openssl. 2024-04-17T09:49:53.502 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cheroot. 2024-04-17T09:49:53.516 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-17T09:49:53.517 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-17T09:49:53.594 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-17T09:49:53.684 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-17T09:49:53.913 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-17T09:49:53.928 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-17T09:49:53.971 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-17T09:49:54.037 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr. 2024-04-17T09:49:54.053 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:54.095 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:54.324 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-17T09:49:54.339 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-17T09:49:54.382 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-17T09:49:54.482 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mon. 2024-04-17T09:49:54.497 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:54.540 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mon (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:54.669 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-17T09:49:54.684 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-17T09:49:54.719 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-17T09:49:55.022 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-tempora. 2024-04-17T09:49:55.036 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-17T09:49:55.110 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-osd. 2024-04-17T09:49:55.113 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-17T09:49:55.126 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:55.160 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-osd (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:55.416 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-portend. 2024-04-17T09:49:55.431 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-17T09:49:55.474 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-17T09:49:55.777 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-17T09:49:55.792 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-17T09:49:55.827 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-17T09:49:55.905 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph. 2024-04-17T09:49:55.920 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../068-ceph_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:55.963 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:56.139 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-17T09:49:56.153 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-17T09:49:56.197 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-17T09:49:56.307 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-fuse. 2024-04-17T09:49:56.323 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:56.365 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-fuse (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:56.533 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-natsort. 2024-04-17T09:49:56.547 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-17T09:49:56.591 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-17T09:49:56.727 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mds. 2024-04-17T09:49:56.741 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:56.784 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mds (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:56.895 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-logutils. 2024-04-17T09:49:56.910 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-17T09:49:56.953 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-17T09:49:57.254 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package cephadm. 2024-04-17T09:49:57.270 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../071-cephadm_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:49:57.290 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-17T09:49:57.304 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking cephadm (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:57.304 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-17T09:49:57.348 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-17T09:49:57.648 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-17T09:49:57.664 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-17T09:49:57.675 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-mako. 2024-04-17T09:49:57.690 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-17T09:49:57.706 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-17T09:49:57.734 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-17T09:49:57.984 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-17T09:49:57.999 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-17T09:49:58.033 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-17T09:49:58.070 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-17T09:49:58.085 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-17T09:49:58.129 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-17T09:49:58.424 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-17T09:49:58.438 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-17T09:49:58.481 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-17T09:49:58.793 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-webob. 2024-04-17T09:49:58.807 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-17T09:49:58.842 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-17T09:49:59.005 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-babel. 2024-04-17T09:49:59.020 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-17T09:49:59.063 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-17T09:49:59.171 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-waitress. 2024-04-17T09:49:59.186 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-17T09:49:59.248 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-17T09:49:59.383 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jinja2. 2024-04-17T09:49:59.397 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-17T09:49:59.441 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-17T09:49:59.532 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-tempita. 2024-04-17T09:49:59.547 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-17T09:49:59.590 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-17T09:49:59.794 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-17T09:49:59.810 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-3133-ga8c84f59-1jammy_all.deb ... 2024-04-17T09:49:59.852 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:49:59.910 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-paste. 2024-04-17T09:49:59.925 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-17T09:49:59.968 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-17T09:50:00.214 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-17T09:50:00.228 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-17T09:50:00.272 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-17T09:50:00.338 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-17T09:50:00.352 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-17T09:50:00.396 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-17T09:50:00.566 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-routes. 2024-04-17T09:50:00.581 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-17T09:50:00.616 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-17T09:50:00.674 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-17T09:50:00.688 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-17T09:50:00.757 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-17T09:50:00.927 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-17T09:50:00.943 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-3133-ga8c84f59-1jammy_all.deb ... 2024-04-17T09:50:00.994 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:01.060 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-webtest. 2024-04-17T09:50:01.076 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-17T09:50:01.119 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-17T09:50:01.455 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pecan. 2024-04-17T09:50:01.470 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-17T09:50:01.513 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-17T09:50:01.901 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-certifi. 2024-04-17T09:50:01.916 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-17T09:50:01.959 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-17T09:50:02.165 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-17T09:50:02.180 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-17T09:50:02.223 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-17T09:50:02.296 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-idna. 2024-04-17T09:50:02.310 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-17T09:50:02.346 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-17T09:50:02.657 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-urllib3. 2024-04-17T09:50:02.672 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-17T09:50:02.715 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-17T09:50:02.743 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-joblib. 2024-04-17T09:50:02.758 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-17T09:50:02.801 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-17T09:50:03.044 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-requests. 2024-04-17T09:50:03.058 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-17T09:50:03.102 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-17T09:50:03.163 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-17T09:50:03.178 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-17T09:50:03.298 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-17T09:50:03.577 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-sklearn. 2024-04-17T09:50:03.592 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-17T09:50:03.605 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-17T09:50:03.620 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-17T09:50:03.634 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-17T09:50:03.663 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-17T09:50:03.974 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-17T09:50:03.986 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-3133-ga8c84f59-1jammy_all.deb ... 2024-04-17T09:50:04.024 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:04.343 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-17T09:50:04.358 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-3133-ga8c84f59-1jammy_all.deb ... 2024-04-17T09:50:04.401 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:04.463 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-17T09:50:04.478 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:50:04.521 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:04.832 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-17T09:50:04.845 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-17T09:50:04.915 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-17T09:50:05.193 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-openssl. 2024-04-17T09:50:05.207 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-17T09:50:05.251 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-17T09:50:05.354 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cachetools. 2024-04-17T09:50:05.370 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-17T09:50:05.413 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-17T09:50:05.605 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr. 2024-04-17T09:50:05.618 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:50:05.663 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:05.674 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-17T09:50:05.689 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-17T09:50:05.732 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-17T09:50:06.035 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-17T09:50:06.050 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-17T09:50:06.074 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mon. 2024-04-17T09:50:06.085 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-17T09:50:06.088 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:50:06.132 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mon (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:06.388 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rsa. 2024-04-17T09:50:06.403 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-17T09:50:06.446 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-17T09:50:06.745 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-osd. 2024-04-17T09:50:06.758 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:50:06.766 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-google-auth. 2024-04-17T09:50:06.781 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-17T09:50:06.803 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-osd (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:06.824 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-17T09:50:07.144 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-17T09:50:07.158 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-17T09:50:07.202 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-17T09:50:07.514 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-websocket. 2024-04-17T09:50:07.530 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-17T09:50:07.564 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph. 2024-04-17T09:50:07.576 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../068-ceph_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:50:07.597 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-17T09:50:07.614 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:07.917 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-17T09:50:07.932 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-17T09:50:07.933 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-fuse. 2024-04-17T09:50:07.947 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:50:07.966 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-17T09:50:08.017 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-fuse (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:08.378 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mds. 2024-04-17T09:50:08.392 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:50:08.436 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mds (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:08.664 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-17T09:50:08.678 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-3133-ga8c84f59-1jammy_all.deb ... 2024-04-17T09:50:08.722 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:08.798 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package cephadm. 2024-04-17T09:50:08.811 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../071-cephadm_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:50:08.848 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking cephadm (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:09.083 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-17T09:50:09.096 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-17T09:50:09.141 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-17T09:50:09.175 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-17T09:50:09.188 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-17T09:50:09.233 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-17T09:50:09.444 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-17T09:50:09.461 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-17T09:50:09.493 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-17T09:50:09.570 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-17T09:50:09.586 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-17T09:50:09.628 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-17T09:50:09.746 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package jq. 2024-04-17T09:50:09.762 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-17T09:50:09.805 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-17T09:50:10.082 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package socat. 2024-04-17T09:50:10.098 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-17T09:50:10.149 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-17T09:50:10.418 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package xmlstarlet. 2024-04-17T09:50:10.434 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-17T09:50:10.468 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-17T09:50:10.608 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-babel. 2024-04-17T09:50:10.621 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-17T09:50:10.666 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-17T09:50:10.797 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-test. 2024-04-17T09:50:10.812 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../099-ceph-test_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:50:10.847 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-test (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:11.002 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jinja2. 2024-04-17T09:50:11.015 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-17T09:50:11.052 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-17T09:50:11.405 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-17T09:50:11.421 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-3133-ga8c84f59-1jammy_all.deb ... 2024-04-17T09:50:11.455 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:11.809 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-17T09:50:11.822 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-17T09:50:11.858 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-17T09:50:12.170 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-routes. 2024-04-17T09:50:12.183 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-17T09:50:12.228 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-17T09:50:12.514 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-17T09:50:12.528 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-3133-ga8c84f59-1jammy_all.deb ... 2024-04-17T09:50:12.572 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:13.035 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-volume. 2024-04-17T09:50:13.050 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-3133-ga8c84f59-1jammy_all.deb ... 2024-04-17T09:50:13.084 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-volume (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:13.466 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-17T09:50:13.482 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:50:13.523 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking cephfs-mirror (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:13.835 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-colorama. 2024-04-17T09:50:13.851 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-17T09:50:13.893 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-17T09:50:13.919 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-17T09:50:13.934 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-17T09:50:14.016 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-17T09:50:14.154 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package cephfs-shell. 2024-04-17T09:50:14.170 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-3133-ga8c84f59-1jammy_all.deb ... 2024-04-17T09:50:14.203 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking cephfs-shell (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:14.456 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package cephfs-top. 2024-04-17T09:50:14.472 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-3133-ga8c84f59-1jammy_all.deb ... 2024-04-17T09:50:14.514 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking cephfs-top (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:14.523 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-joblib. 2024-04-17T09:50:14.533 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-17T09:50:14.573 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-17T09:50:14.767 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-17T09:50:14.783 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:50:14.825 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libcephfs-dev (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:14.934 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-17T09:50:14.945 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-17T09:50:15.087 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-17T09:50:15.111 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-17T09:50:15.127 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-17T09:50:15.169 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-17T09:50:15.373 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-sklearn. 2024-04-17T09:50:15.384 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-17T09:50:15.414 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-17T09:50:15.532 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-17T09:50:15.548 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-17T09:50:15.590 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-17T09:50:15.953 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package nvme-cli. 2024-04-17T09:50:15.969 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-17T09:50:16.011 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-17T09:50:16.115 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-17T09:50:16.125 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-3133-ga8c84f59-1jammy_all.deb ... 2024-04-17T09:50:16.165 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:16.416 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package pkg-config. 2024-04-17T09:50:16.432 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-17T09:50:16.491 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-17T09:50:16.727 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-17T09:50:16.743 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-17T09:50:16.785 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-17T09:50:17.110 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cachetools. 2024-04-17T09:50:17.121 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-17T09:50:17.188 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-17T09:50:17.193 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-17T09:50:17.197 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-17T09:50:17.229 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-17T09:50:17.505 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-17T09:50:17.516 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-17T09:50:17.541 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pastescript. 2024-04-17T09:50:17.549 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-17T09:50:17.554 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-17T09:50:17.590 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-17T09:50:17.916 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-17T09:50:17.927 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-17T09:50:17.952 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pluggy. 2024-04-17T09:50:17.964 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-17T09:50:17.966 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-17T09:50:18.002 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-17T09:50:18.313 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-psutil. 2024-04-17T09:50:18.323 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-17T09:50:18.363 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-17T09:50:18.436 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rsa. 2024-04-17T09:50:18.447 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-17T09:50:18.486 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-17T09:50:18.682 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-py. 2024-04-17T09:50:18.695 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-17T09:50:18.732 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-17T09:50:18.806 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-google-auth. 2024-04-17T09:50:18.817 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-17T09:50:18.856 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-17T09:50:19.077 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pygments. 2024-04-17T09:50:19.088 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-17T09:50:19.127 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-17T09:50:19.176 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-17T09:50:19.187 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-17T09:50:19.259 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-17T09:50:19.564 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-17T09:50:19.573 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-websocket. 2024-04-17T09:50:19.574 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-17T09:50:19.587 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-17T09:50:19.614 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-17T09:50:19.631 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-17T09:50:19.917 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-toml. 2024-04-17T09:50:19.927 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-17T09:50:19.967 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-17T09:50:19.976 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-17T09:50:19.988 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-17T09:50:20.017 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-17T09:50:20.311 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pytest. 2024-04-17T09:50:20.322 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-17T09:50:20.361 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-17T09:50:20.706 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-simplejson. 2024-04-17T09:50:20.715 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-17T09:50:20.717 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-17T09:50:20.725 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-3133-ga8c84f59-1jammy_all.deb ... 2024-04-17T09:50:20.756 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-17T09:50:20.798 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:21.034 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-17T09:50:21.047 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-17T09:50:21.092 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-17T09:50:21.117 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-17T09:50:21.129 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-17T09:50:21.167 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-17T09:50:21.512 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-17T09:50:21.524 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-17T09:50:21.562 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-17T09:50:21.771 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package radosgw. 2024-04-17T09:50:21.781 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../122-radosgw_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:50:21.800 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package jq. 2024-04-17T09:50:21.812 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-17T09:50:21.820 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking radosgw (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:21.856 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-17T09:50:22.109 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package socat. 2024-04-17T09:50:22.124 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-17T09:50:22.167 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-17T09:50:22.415 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package rbd-fuse. 2024-04-17T09:50:22.426 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:50:22.465 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking rbd-fuse (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:22.520 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package xmlstarlet. 2024-04-17T09:50:22.534 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-17T09:50:22.578 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-17T09:50:22.765 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-17T09:50:22.949 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-test. 2024-04-17T09:50:22.961 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../099-ceph-test_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:50:22.999 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-test (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:23.028 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-17T09:50:23.154 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-17T09:50:23.431 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-17T09:50:23.855 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-17T09:50:24.489 INFO:teuthology.orchestra.run.smithi032.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-17T09:50:24.505 INFO:teuthology.orchestra.run.smithi032.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-04-17T09:50:24.548 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up cephadm (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:25.058 INFO:teuthology.orchestra.run.smithi032.stdout:Adding system user cephadm....done 2024-04-17T09:50:25.120 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-volume. 2024-04-17T09:50:25.134 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-3133-ga8c84f59-1jammy_all.deb ... 2024-04-17T09:50:25.178 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-volume (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:25.187 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-17T09:50:25.555 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-17T09:50:25.574 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-17T09:50:25.587 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:50:25.632 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking cephfs-mirror (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:25.823 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-17T09:50:25.949 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-17T09:50:25.977 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-colorama. 2024-04-17T09:50:25.991 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-17T09:50:26.035 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-17T09:50:26.225 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-17T09:50:26.296 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package cephfs-shell. 2024-04-17T09:50:26.309 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-3133-ga8c84f59-1jammy_all.deb ... 2024-04-17T09:50:26.400 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking cephfs-shell (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:26.518 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-17T09:50:26.636 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-17T09:50:26.640 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package cephfs-top. 2024-04-17T09:50:26.654 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-3133-ga8c84f59-1jammy_all.deb ... 2024-04-17T09:50:26.690 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking cephfs-top (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:26.946 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-17T09:50:26.951 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-17T09:50:26.965 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:50:27.010 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libcephfs-dev (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:27.323 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-17T09:50:27.329 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-17T09:50:27.342 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-17T09:50:27.379 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-17T09:50:27.599 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-17T09:50:27.750 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-17T09:50:27.763 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-17T09:50:27.800 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-17T09:50:27.867 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-17T09:50:28.188 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package nvme-cli. 2024-04-17T09:50:28.200 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-17T09:50:28.245 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-17T09:50:28.268 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-17T09:50:28.495 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-17T09:50:28.684 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package pkg-config. 2024-04-17T09:50:28.697 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-17T09:50:28.742 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-17T09:50:28.754 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-17T09:50:29.019 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-17T09:50:29.030 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-17T09:50:29.032 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-17T09:50:29.070 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-17T09:50:29.306 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-ceph-argparse (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:29.465 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-17T09:50:29.478 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-17T09:50:29.548 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-17T09:50:29.582 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-17T09:50:29.708 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-17T09:50:29.801 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pastescript. 2024-04-17T09:50:29.814 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-17T09:50:29.834 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-17T09:50:29.851 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-17T09:50:30.110 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-17T09:50:30.213 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pluggy. 2024-04-17T09:50:30.226 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-17T09:50:30.228 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-17T09:50:30.262 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-17T09:50:30.479 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-17T09:50:30.524 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-psutil. 2024-04-17T09:50:30.537 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-17T09:50:30.607 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-17T09:50:30.872 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-17T09:50:30.901 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-17T09:50:30.951 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-py. 2024-04-17T09:50:30.965 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-17T09:50:30.982 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-17T09:50:31.001 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-17T09:50:31.108 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-17T09:50:31.371 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pygments. 2024-04-17T09:50:31.384 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-17T09:50:31.413 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-17T09:50:31.518 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-17T09:50:31.635 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-17T09:50:31.893 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-17T09:50:31.895 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-17T09:50:31.904 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-17T09:50:31.942 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-17T09:50:32.179 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-17T09:50:32.278 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-toml. 2024-04-17T09:50:32.291 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-17T09:50:32.336 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-17T09:50:32.497 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up zip (3.0-12build2) ... 2024-04-17T09:50:32.623 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-17T09:50:32.639 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pytest. 2024-04-17T09:50:32.652 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-17T09:50:32.722 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-17T09:50:33.043 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-simplejson. 2024-04-17T09:50:33.056 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-17T09:50:33.100 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-17T09:50:33.362 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-17T09:50:33.367 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-17T09:50:33.375 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-17T09:50:33.412 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-17T09:50:33.643 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-17T09:50:33.768 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-17T09:50:33.869 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-17T09:50:34.116 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package radosgw. 2024-04-17T09:50:34.127 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../122-radosgw_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:50:34.165 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking radosgw (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:34.212 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-17T09:50:34.622 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-17T09:50:34.890 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-17T09:50:35.011 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package rbd-fuse. 2024-04-17T09:50:35.023 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-3133-ga8c84f59-1jammy_amd64.deb ... 2024-04-17T09:50:35.069 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking rbd-fuse (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:35.300 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-17T09:50:35.394 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-17T09:50:35.609 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-17T09:50:35.665 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-17T09:50:35.766 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-17T09:50:35.935 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-17T09:50:36.038 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-17T09:50:36.279 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-17T09:50:36.448 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-17T09:50:36.538 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-17T09:50:36.664 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-ceph-common (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:36.982 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-idna (3.3-1) ... 2024-04-17T09:50:37.140 INFO:teuthology.orchestra.run.smithi144.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-17T09:50:37.154 INFO:teuthology.orchestra.run.smithi144.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-04-17T09:50:37.210 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up cephadm (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:37.333 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-17T09:50:37.755 INFO:teuthology.orchestra.run.smithi144.stdout:Adding system user cephadm....done 2024-04-17T09:50:37.899 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-17T09:50:38.201 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-17T09:50:38.509 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-17T09:50:38.628 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-17T09:50:38.780 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-17T09:50:38.871 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-17T09:50:39.057 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T09:50:39.164 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-17T09:50:39.191 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-17T09:50:39.290 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-17T09:50:39.484 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-17T09:50:39.600 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-17T09:50:39.610 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-17T09:50:39.719 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-17T09:50:40.002 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-17T09:50:40.012 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-17T09:50:40.278 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-17T09:50:40.288 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-17T09:50:40.547 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-17T09:50:40.573 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T09:50:40.699 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-17T09:50:40.831 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-17T09:50:41.017 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-17T09:50:41.091 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-17T09:50:41.285 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-17T09:50:41.425 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-17T09:50:41.552 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-17T09:50:41.718 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-17T09:50:41.820 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-17T09:50:41.986 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-ceph-argparse (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:42.080 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-17T09:50:42.287 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-17T09:50:42.355 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-17T09:50:42.413 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-17T09:50:42.490 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-17T09:50:42.564 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-17T09:50:42.783 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-17T09:50:42.824 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-17T09:50:42.900 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-17T09:50:42.949 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-17T09:50:43.193 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-17T09:50:43.242 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-17T09:50:43.644 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-17T09:50:43.646 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-17T09:50:43.706 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-17T09:50:43.770 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-17T09:50:43.904 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-17T09:50:43.905 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-17T09:50:44.071 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-17T09:50:44.197 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-17T09:50:44.273 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-17T09:50:44.286 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-17T09:50:44.399 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-17T09:50:44.549 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-17T09:50:44.666 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-17T09:50:44.675 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-17T09:50:44.934 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-17T09:50:45.068 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-17T09:50:45.253 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up zip (3.0-12build2) ... 2024-04-17T09:50:45.328 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-17T09:50:45.362 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-17T09:50:45.453 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-17T09:50:45.721 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T09:50:45.847 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-17T09:50:46.048 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-17T09:50:46.190 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-17T09:50:46.357 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-17T09:50:46.482 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-17T09:50:46.516 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-17T09:50:46.609 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-17T09:50:46.818 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-17T09:50:46.951 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-17T09:50:47.013 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-17T09:50:47.313 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-17T09:50:47.353 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-17T09:50:47.606 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-17T09:50:47.612 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-17T09:50:47.908 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-17T09:50:48.039 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-17T09:50:48.301 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-17T09:50:48.340 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-17T09:50:48.578 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-17T09:50:48.650 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-17T09:50:48.948 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-17T09:50:49.026 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-17T09:50:49.081 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librados2 (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:49.207 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:49.285 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-17T09:50:49.341 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-17T09:50:49.394 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-ceph-common (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:49.644 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libcephfs2 (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:49.680 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-idna (3.3-1) ... 2024-04-17T09:50:49.771 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libradosstriper1 (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:49.897 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-17T09:50:50.047 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-17T09:50:50.215 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librbd1 (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:50.340 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-modules-core (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:50.467 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-fuse (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:50.683 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-17T09:50:50.683 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-17T09:50:51.303 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-17T09:50:51.377 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libcephfs-dev (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:51.511 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rados (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:51.538 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T09:50:51.611 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librgw2 (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:51.673 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-17T09:50:51.729 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up cephfs-top (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:51.855 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-17T09:50:51.949 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-17T09:50:52.083 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-17T09:50:52.209 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-17T09:50:52.535 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-17T09:50:52.778 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-17T09:50:53.055 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T09:50:53.181 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-17T09:50:53.209 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rbd (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:53.318 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up rbd-fuse (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:53.428 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rgw (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:53.490 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-17T09:50:53.553 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cephfs (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:53.696 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up cephfs-shell (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:53.758 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-17T09:50:53.822 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-common (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:54.059 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-17T09:50:54.335 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-17T09:50:54.400 INFO:teuthology.orchestra.run.smithi032.stdout:Adding group ceph....done 2024-04-17T09:50:54.612 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-17T09:50:54.743 INFO:teuthology.orchestra.run.smithi032.stdout:Adding system user ceph....done 2024-04-17T09:50:54.915 INFO:teuthology.orchestra.run.smithi032.stdout:Setting system user ceph properties....done 2024-04-17T09:50:54.920 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-17T09:50:54.935 INFO:teuthology.orchestra.run.smithi032.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-17T09:50:55.055 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-17T09:50:55.088 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-17T09:50:55.340 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-17T09:50:55.491 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-17T09:50:55.509 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-17T09:50:55.792 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-17T09:50:56.110 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-17T09:50:56.215 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up cephfs-mirror (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:56.369 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-17T09:50:56.431 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-17T09:50:56.431 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-17T09:50:56.513 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-17T09:50:56.638 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-17T09:50:56.727 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-17T09:50:56.998 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-17T09:50:57.075 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-test (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:57.124 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-17T09:50:57.192 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up radosgw (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:57.526 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-17T09:50:57.749 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-17T09:50:57.749 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-17T09:50:57.802 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-17T09:50:57.911 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-17T09:50:58.195 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T09:50:58.330 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-17T09:50:58.386 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-base (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:58.656 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-17T09:50:58.753 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-17T09:50:58.965 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-17T09:50:59.275 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-17T09:50:59.456 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mds (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:50:59.478 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-17T09:50:59.688 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-17T09:50:59.688 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-17T09:50:59.763 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-17T09:51:00.073 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-17T09:51:00.341 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-17T09:51:00.344 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:00.547 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-17T09:51:00.547 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-17T09:51:00.726 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-17T09:51:01.011 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-17T09:51:01.195 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-osd (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:01.380 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-17T09:51:01.508 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-17T09:51:01.509 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-17T09:51:01.514 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librados2 (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:01.682 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:01.808 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-17T09:51:02.161 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libcephfs2 (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:02.188 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:02.262 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libradosstriper1 (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:02.314 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:02.388 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-17T09:51:02.473 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mon (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:02.690 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librbd1 (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:02.693 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-17T09:51:02.694 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-17T09:51:02.816 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-modules-core (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:02.942 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-fuse (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:03.153 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-17T09:51:03.153 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-17T09:51:03.349 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-cephadm (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:03.492 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:03.601 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-dashboard (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:03.736 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-volume (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:03.801 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libcephfs-dev (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:03.919 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rados (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:04.028 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librgw2 (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:04.105 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-17T09:51:04.154 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up cephfs-top (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:04.214 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-17T09:51:04.331 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-17T09:51:04.375 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-17T09:51:05.584 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-17T09:51:05.617 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rbd (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:05.760 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up rbd-fuse (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:05.861 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rgw (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:05.979 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cephfs (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:06.113 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up cephfs-shell (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:06.239 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-common (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:06.270 INFO:teuthology.orchestra.run.smithi032.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-17T09:51:06.275 DEBUG:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-17T09:51:06.352 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2024-04-17T09:51:06.529 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2024-04-17T09:51:06.529 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2024-04-17T09:51:06.709 INFO:teuthology.orchestra.run.smithi032.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-17T09:51:06.709 INFO:teuthology.orchestra.run.smithi032.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-17T09:51:06.709 INFO:teuthology.orchestra.run.smithi032.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-17T09:51:06.709 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages were automatically installed and are no longer required: 2024-04-17T09:51:06.709 INFO:teuthology.orchestra.run.smithi032.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-17T09:51:06.709 INFO:teuthology.orchestra.run.smithi032.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-17T09:51:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-17T09:51:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-17T09:51:06.710 INFO:teuthology.orchestra.run.smithi032.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-17T09:51:06.710 INFO:teuthology.orchestra.run.smithi032.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-17T09:51:06.711 INFO:teuthology.orchestra.run.smithi032.stdout:The following additional packages will be installed: 2024-04-17T09:51:06.711 INFO:teuthology.orchestra.run.smithi032.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-17T09:51:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-17T09:51:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-17T09:51:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-17T09:51:06.712 INFO:teuthology.orchestra.run.smithi032.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-17T09:51:06.713 INFO:teuthology.orchestra.run.smithi032.stdout:Suggested packages: 2024-04-17T09:51:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-17T09:51:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-17T09:51:06.713 INFO:teuthology.orchestra.run.smithi032.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-17T09:51:06.769 INFO:teuthology.orchestra.run.smithi032.stdout:The following NEW packages will be installed: 2024-04-17T09:51:06.769 INFO:teuthology.orchestra.run.smithi032.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-17T09:51:06.769 INFO:teuthology.orchestra.run.smithi032.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-17T09:51:06.770 INFO:teuthology.orchestra.run.smithi032.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-17T09:51:06.770 INFO:teuthology.orchestra.run.smithi032.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-17T09:51:06.770 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-17T09:51:06.770 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-17T09:51:06.770 INFO:teuthology.orchestra.run.smithi032.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-17T09:51:06.771 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages will be upgraded: 2024-04-17T09:51:06.772 INFO:teuthology.orchestra.run.smithi032.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-17T09:51:06.865 INFO:teuthology.orchestra.run.smithi032.stdout:4 upgraded, 39 newly installed, 0 to remove and 301 not upgraded. 2024-04-17T09:51:06.865 INFO:teuthology.orchestra.run.smithi032.stdout:Need to get 55.7 MB of archives. 2024-04-17T09:51:06.865 INFO:teuthology.orchestra.run.smithi032.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-17T09:51:06.865 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-17T09:51:07.031 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-17T09:51:07.035 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-17T09:51:07.068 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-17T09:51:07.068 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-17T09:51:07.069 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-17T09:51:07.076 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-17T09:51:07.077 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-17T09:51:07.079 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-17T09:51:07.080 INFO:teuthology.orchestra.run.smithi032.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-17T09:51:07.081 INFO:teuthology.orchestra.run.smithi032.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-17T09:51:07.082 INFO:teuthology.orchestra.run.smithi032.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-17T09:51:07.093 INFO:teuthology.orchestra.run.smithi144.stdout:Adding group ceph....done 2024-04-17T09:51:07.099 INFO:teuthology.orchestra.run.smithi032.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-17T09:51:07.099 INFO:teuthology.orchestra.run.smithi032.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-17T09:51:07.100 INFO:teuthology.orchestra.run.smithi032.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-17T09:51:07.101 INFO:teuthology.orchestra.run.smithi032.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-17T09:51:07.346 INFO:teuthology.orchestra.run.smithi032.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-17T09:51:07.348 INFO:teuthology.orchestra.run.smithi032.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-17T09:51:07.348 INFO:teuthology.orchestra.run.smithi032.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-17T09:51:07.351 INFO:teuthology.orchestra.run.smithi032.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-17T09:51:07.351 INFO:teuthology.orchestra.run.smithi032.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-17T09:51:07.356 INFO:teuthology.orchestra.run.smithi032.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-17T09:51:07.358 INFO:teuthology.orchestra.run.smithi032.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-17T09:51:07.365 INFO:teuthology.orchestra.run.smithi032.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-17T09:51:07.365 INFO:teuthology.orchestra.run.smithi032.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-17T09:51:07.388 INFO:teuthology.orchestra.run.smithi032.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-17T09:51:07.388 INFO:teuthology.orchestra.run.smithi032.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-17T09:51:07.398 INFO:teuthology.orchestra.run.smithi032.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-17T09:51:07.421 INFO:teuthology.orchestra.run.smithi032.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-17T09:51:07.425 INFO:teuthology.orchestra.run.smithi032.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-17T09:51:07.425 INFO:teuthology.orchestra.run.smithi032.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-17T09:51:07.426 INFO:teuthology.orchestra.run.smithi032.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-17T09:51:07.427 INFO:teuthology.orchestra.run.smithi032.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-17T09:51:07.438 INFO:teuthology.orchestra.run.smithi032.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-17T09:51:07.440 INFO:teuthology.orchestra.run.smithi032.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-17T09:51:07.504 INFO:teuthology.orchestra.run.smithi144.stdout:Adding system user ceph....done 2024-04-17T09:51:07.606 INFO:teuthology.orchestra.run.smithi032.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-17T09:51:07.606 INFO:teuthology.orchestra.run.smithi032.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-17T09:51:07.607 INFO:teuthology.orchestra.run.smithi032.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-17T09:51:07.607 INFO:teuthology.orchestra.run.smithi032.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-17T09:51:07.607 INFO:teuthology.orchestra.run.smithi032.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-17T09:51:07.607 INFO:teuthology.orchestra.run.smithi032.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-17T09:51:07.621 INFO:teuthology.orchestra.run.smithi032.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-17T09:51:07.621 INFO:teuthology.orchestra.run.smithi032.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-17T09:51:07.665 INFO:teuthology.orchestra.run.smithi144.stdout:Setting system user ceph properties....done 2024-04-17T09:51:07.683 INFO:teuthology.orchestra.run.smithi144.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-17T09:51:07.828 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-17T09:51:08.015 INFO:teuthology.orchestra.run.smithi032.stdout:Preconfiguring packages ... 2024-04-17T09:51:08.158 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 55.7 MB in 1s (66.3 MB/s) 2024-04-17T09:51:08.220 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-17T09:51:08.447 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140923 files and directories currently installed.) 2024-04-17T09:51:08.451 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-17T09:51:08.576 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-17T09:51:08.909 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up cephfs-mirror (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:09.144 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-17T09:51:09.144 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-17T09:51:09.244 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-17T09:51:09.743 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-test (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:09.844 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up radosgw (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:10.398 INFO:teuthology.orchestra.run.smithi032.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-17T09:51:10.433 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-17T09:51:10.433 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-17T09:51:10.657 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package flex. 2024-04-17T09:51:10.729 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140923 files and directories currently installed.) 2024-04-17T09:51:10.732 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-17T09:51:10.780 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-17T09:51:11.063 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-base (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:11.267 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-17T09:51:11.420 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-17T09:51:11.422 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-17T09:51:11.802 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-17T09:51:12.099 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mds (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:12.330 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-17T09:51:12.330 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-17T09:51:12.363 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140996 files and directories currently installed.) 2024-04-17T09:51:12.367 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-17T09:51:12.469 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-17T09:51:12.917 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-17T09:51:12.967 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:13.043 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-17T09:51:13.205 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-17T09:51:13.205 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-17T09:51:13.327 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-17T09:51:13.711 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package bison. 2024-04-17T09:51:13.791 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140996 files and directories currently installed.) 2024-04-17T09:51:13.796 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-17T09:51:13.818 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-osd (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:13.838 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-17T09:51:14.109 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-17T09:51:14.109 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-17T09:51:14.248 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-17T09:51:14.263 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-17T09:51:14.306 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-17T09:51:14.592 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package dump. 2024-04-17T09:51:14.608 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-17T09:51:14.659 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking dump (0.4b47-3) ... 2024-04-17T09:51:14.761 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:14.887 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:14.938 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package indent. 2024-04-17T09:51:14.954 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-17T09:51:14.996 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-17T09:51:15.031 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mon (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:15.264 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-17T09:51:15.264 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-17T09:51:15.282 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-17T09:51:15.299 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-17T09:51:15.340 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-17T09:51:15.645 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-17T09:51:15.662 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-17T09:51:15.695 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-17T09:51:15.882 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-cephadm (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:15.939 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-17T09:51:15.956 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-17T09:51:15.997 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-17T09:51:16.024 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:16.175 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-dashboard (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:16.301 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-volume (19.0.0-3133-ga8c84f59-1jammy) ... 2024-04-17T09:51:16.335 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-17T09:51:16.351 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-17T09:51:16.393 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-17T09:51:16.637 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-17T09:51:16.654 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-17T09:51:16.670 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-17T09:51:16.704 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-17T09:51:16.788 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-17T09:51:16.932 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libjson-perl. 2024-04-17T09:51:16.940 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-17T09:51:16.948 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-17T09:51:16.990 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-17T09:51:17.260 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-17T09:51:17.276 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-17T09:51:17.310 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-17T09:51:18.034 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-17T09:51:18.639 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-17T09:51:18.655 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-17T09:51:18.696 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-17T09:51:18.733 INFO:teuthology.orchestra.run.smithi144.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-17T09:51:18.738 DEBUG:teuthology.orchestra.run.smithi144:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-17T09:51:18.812 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-04-17T09:51:18.973 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-17T09:51:18.987 INFO:teuthology.orchestra.run.smithi144.stdout:Building dependency tree... 2024-04-17T09:51:18.988 INFO:teuthology.orchestra.run.smithi144.stdout:Reading state information... 2024-04-17T09:51:18.989 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-17T09:51:19.032 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-17T09:51:19.158 INFO:teuthology.orchestra.run.smithi144.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-17T09:51:19.158 INFO:teuthology.orchestra.run.smithi144.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-17T09:51:19.158 INFO:teuthology.orchestra.run.smithi144.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-17T09:51:19.158 INFO:teuthology.orchestra.run.smithi144.stdout:The following packages were automatically installed and are no longer required: 2024-04-17T09:51:19.158 INFO:teuthology.orchestra.run.smithi144.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-17T09:51:19.158 INFO:teuthology.orchestra.run.smithi144.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-17T09:51:19.158 INFO:teuthology.orchestra.run.smithi144.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-17T09:51:19.159 INFO:teuthology.orchestra.run.smithi144.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-17T09:51:19.159 INFO:teuthology.orchestra.run.smithi144.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-17T09:51:19.159 INFO:teuthology.orchestra.run.smithi144.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-17T09:51:19.159 INFO:teuthology.orchestra.run.smithi144.stdout:The following additional packages will be installed: 2024-04-17T09:51:19.159 INFO:teuthology.orchestra.run.smithi144.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-17T09:51:19.160 INFO:teuthology.orchestra.run.smithi144.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-17T09:51:19.160 INFO:teuthology.orchestra.run.smithi144.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-17T09:51:19.160 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-17T09:51:19.160 INFO:teuthology.orchestra.run.smithi144.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-17T09:51:19.161 INFO:teuthology.orchestra.run.smithi144.stdout:Suggested packages: 2024-04-17T09:51:19.161 INFO:teuthology.orchestra.run.smithi144.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-17T09:51:19.162 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-17T09:51:19.162 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-17T09:51:19.215 INFO:teuthology.orchestra.run.smithi144.stdout:The following NEW packages will be installed: 2024-04-17T09:51:19.216 INFO:teuthology.orchestra.run.smithi144.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-17T09:51:19.216 INFO:teuthology.orchestra.run.smithi144.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-17T09:51:19.217 INFO:teuthology.orchestra.run.smithi144.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-17T09:51:19.217 INFO:teuthology.orchestra.run.smithi144.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-17T09:51:19.217 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-17T09:51:19.217 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-17T09:51:19.217 INFO:teuthology.orchestra.run.smithi144.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-17T09:51:19.218 INFO:teuthology.orchestra.run.smithi144.stdout:The following packages will be upgraded: 2024-04-17T09:51:19.219 INFO:teuthology.orchestra.run.smithi144.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-17T09:51:19.319 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-17T09:51:19.336 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-17T09:51:19.377 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-17T09:51:19.432 INFO:teuthology.orchestra.run.smithi144.stdout:4 upgraded, 39 newly installed, 0 to remove and 301 not upgraded. 2024-04-17T09:51:19.432 INFO:teuthology.orchestra.run.smithi144.stdout:Need to get 55.7 MB of archives. 2024-04-17T09:51:19.432 INFO:teuthology.orchestra.run.smithi144.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-17T09:51:19.432 INFO:teuthology.orchestra.run.smithi144.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-17T09:51:19.722 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-17T09:51:19.738 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-17T09:51:19.780 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-17T09:51:20.076 INFO:teuthology.orchestra.run.smithi144.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-17T09:51:20.092 INFO:teuthology.orchestra.run.smithi144.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-17T09:51:20.100 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-17T09:51:20.116 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-17T09:51:20.158 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-17T09:51:20.218 INFO:teuthology.orchestra.run.smithi144.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-17T09:51:20.218 INFO:teuthology.orchestra.run.smithi144.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-17T09:51:20.220 INFO:teuthology.orchestra.run.smithi144.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-17T09:51:20.248 INFO:teuthology.orchestra.run.smithi144.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-17T09:51:20.249 INFO:teuthology.orchestra.run.smithi144.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-17T09:51:20.252 INFO:teuthology.orchestra.run.smithi144.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-17T09:51:20.255 INFO:teuthology.orchestra.run.smithi144.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-17T09:51:20.256 INFO:teuthology.orchestra.run.smithi144.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-17T09:51:20.422 INFO:teuthology.orchestra.run.smithi144.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-17T09:51:20.470 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-17T09:51:20.486 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-17T09:51:20.528 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T09:51:20.716 INFO:teuthology.orchestra.run.smithi144.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-17T09:51:20.725 INFO:teuthology.orchestra.run.smithi144.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-17T09:51:20.730 INFO:teuthology.orchestra.run.smithi144.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-17T09:51:20.822 INFO:teuthology.orchestra.run.smithi144.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-17T09:51:20.864 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-17T09:51:20.881 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-17T09:51:20.923 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-17T09:51:21.209 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-17T09:51:21.225 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-17T09:51:21.267 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-17T09:51:21.512 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-17T09:51:21.528 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-17T09:51:21.561 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-17T09:51:21.790 INFO:teuthology.orchestra.run.smithi144.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-17T09:51:21.814 INFO:teuthology.orchestra.run.smithi144.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-17T09:51:21.815 INFO:teuthology.orchestra.run.smithi144.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-17T09:51:21.818 INFO:teuthology.orchestra.run.smithi144.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-17T09:51:21.818 INFO:teuthology.orchestra.run.smithi144.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-17T09:51:21.823 INFO:teuthology.orchestra.run.smithi144.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-17T09:51:21.988 INFO:teuthology.orchestra.run.smithi144.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-17T09:51:22.049 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-17T09:51:22.065 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-17T09:51:22.107 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-17T09:51:22.452 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package modemmanager. 2024-04-17T09:51:22.469 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-17T09:51:22.510 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-17T09:51:22.593 INFO:teuthology.orchestra.run.smithi144.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-17T09:51:22.593 INFO:teuthology.orchestra.run.smithi144.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-17T09:51:22.773 INFO:teuthology.orchestra.run.smithi144.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-17T09:51:22.773 INFO:teuthology.orchestra.run.smithi144.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-17T09:51:22.812 INFO:teuthology.orchestra.run.smithi144.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-17T09:51:22.873 INFO:teuthology.orchestra.run.smithi144.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-17T09:51:22.878 INFO:teuthology.orchestra.run.smithi144.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-17T09:51:22.880 INFO:teuthology.orchestra.run.smithi144.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-17T09:51:22.880 INFO:teuthology.orchestra.run.smithi144.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-17T09:51:22.881 INFO:teuthology.orchestra.run.smithi144.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-17T09:51:23.086 INFO:teuthology.orchestra.run.smithi144.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-17T09:51:23.109 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package network-manager. 2024-04-17T09:51:23.126 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-17T09:51:23.175 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-17T09:51:23.501 INFO:teuthology.orchestra.run.smithi144.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-17T09:51:23.717 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ppp. 2024-04-17T09:51:23.733 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-17T09:51:24.092 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-17T09:51:24.248 INFO:teuthology.orchestra.run.smithi144.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-17T09:51:24.248 INFO:teuthology.orchestra.run.smithi144.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-17T09:51:24.248 INFO:teuthology.orchestra.run.smithi144.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-17T09:51:24.248 INFO:teuthology.orchestra.run.smithi144.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-17T09:51:24.249 INFO:teuthology.orchestra.run.smithi144.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-17T09:51:24.249 INFO:teuthology.orchestra.run.smithi144.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-17T09:51:24.278 INFO:teuthology.orchestra.run.smithi144.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-17T09:51:24.279 INFO:teuthology.orchestra.run.smithi144.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-17T09:51:24.414 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package pptp-linux. 2024-04-17T09:51:24.429 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-17T09:51:24.472 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-17T09:51:24.658 INFO:teuthology.orchestra.run.smithi144.stdout:Preconfiguring packages ... 2024-04-17T09:51:24.725 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-17T09:51:24.740 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-17T09:51:24.783 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-17T09:51:24.799 INFO:teuthology.orchestra.run.smithi144.stdout:Fetched 55.7 MB in 5s (11.0 MB/s) 2024-04-17T09:51:25.069 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-17T09:51:25.085 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-17T09:51:25.113 INFO:teuthology.orchestra.run.smithi144.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140923 files and directories currently installed.) 2024-04-17T09:51:25.117 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-17T09:51:25.119 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-client-common (238) ... 2024-04-17T09:51:25.258 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-17T09:51:25.440 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-17T09:51:25.457 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-17T09:51:25.498 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T09:51:25.958 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-17T09:51:26.062 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-common. 2024-04-17T09:51:26.079 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-17T09:51:26.139 INFO:teuthology.orchestra.run.smithi032.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-17T09:51:26.169 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-common (238) ... 2024-04-17T09:51:26.645 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-14. 2024-04-17T09:51:26.661 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-17T09:51:26.697 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T09:51:27.074 INFO:teuthology.orchestra.run.smithi144.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-17T09:51:27.337 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package flex. 2024-04-17T09:51:27.408 INFO:teuthology.orchestra.run.smithi144.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140923 files and directories currently installed.) 2024-04-17T09:51:27.411 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-17T09:51:27.494 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-17T09:51:27.773 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql. 2024-04-17T09:51:27.788 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-17T09:51:27.823 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql (14+238) ... 2024-04-17T09:51:27.965 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-17T09:51:28.050 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-client. 2024-04-17T09:51:28.065 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-17T09:51:28.109 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-client (14+238) ... 2024-04-17T09:51:28.134 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-17T09:51:28.352 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-17T09:51:28.368 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-17T09:51:28.411 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-17T09:51:28.548 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-17T09:51:28.730 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package tcl8.6. 2024-04-17T09:51:28.747 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-17T09:51:28.788 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-17T09:51:29.066 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package tcl. 2024-04-17T09:51:29.084 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-17T09:51:29.103 INFO:teuthology.orchestra.run.smithi144.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140996 files and directories currently installed.) 2024-04-17T09:51:29.106 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-17T09:51:29.121 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-17T09:51:29.208 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-17T09:51:29.461 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package wpasupplicant. 2024-04-17T09:51:29.478 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-17T09:51:29.527 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-17T09:51:29.628 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-17T09:51:29.756 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-17T09:51:30.105 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-17T09:51:30.282 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-17T09:51:30.299 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-17T09:51:30.332 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-17T09:51:30.440 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package bison. 2024-04-17T09:51:30.514 INFO:teuthology.orchestra.run.smithi144.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140996 files and directories currently installed.) 2024-04-17T09:51:30.518 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-17T09:51:30.584 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-17T09:51:30.677 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-17T09:51:30.693 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-17T09:51:30.744 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-17T09:51:30.951 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-17T09:51:30.966 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-17T09:51:31.010 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-17T09:51:31.039 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-client-common (238) ... 2024-04-17T09:51:31.233 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-17T09:51:31.343 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-17T09:51:31.379 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package dump. 2024-04-17T09:51:31.395 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-17T09:51:31.437 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking dump (0.4b47-3) ... 2024-04-17T09:51:31.494 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T09:51:31.612 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-17T09:51:31.729 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-17T09:51:31.758 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package indent. 2024-04-17T09:51:31.774 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-17T09:51:31.816 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-17T09:51:31.855 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T09:51:32.052 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-17T09:51:32.068 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-17T09:51:32.136 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-17T09:51:32.465 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-17T09:51:32.480 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-17T09:51:32.574 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-17T09:51:32.819 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-17T09:51:32.860 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-17T09:51:32.876 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-17T09:51:32.910 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-17T09:51:32.920 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-17T09:51:33.239 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-17T09:51:33.254 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-17T09:51:33.306 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-17T09:51:33.348 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-17T09:51:33.348 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-17T09:51:33.533 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-17T09:51:33.548 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-17T09:51:33.591 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-17T09:51:33.878 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libjson-perl. 2024-04-17T09:51:33.893 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-17T09:51:33.936 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-17T09:51:34.064 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-17T09:51:34.190 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-17T09:51:34.301 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-17T09:51:34.356 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-17T09:51:34.372 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-17T09:51:34.401 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-17T09:51:34.414 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-17T09:51:34.527 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-17T09:51:34.653 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-17T09:51:34.779 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-17T09:51:34.905 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-17T09:51:35.561 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-17T09:51:35.695 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up dump (0.4b47-3) ... 2024-04-17T09:51:35.784 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-17T09:51:35.790 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-17T09:51:35.799 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-17T09:51:35.842 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-17T09:51:35.847 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-17T09:51:35.973 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-17T09:51:36.099 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-17T09:51:36.153 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-17T09:51:36.168 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-17T09:51:36.211 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-17T09:51:36.217 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-17T09:51:36.318 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-17T09:51:36.399 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-17T09:51:36.486 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-17T09:51:36.539 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-17T09:51:36.555 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-17T09:51:36.589 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-17T09:51:36.612 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-17T09:51:36.789 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-17T09:51:36.915 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-17T09:51:36.934 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-17T09:51:36.949 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-17T09:51:37.025 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-17T09:51:37.041 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-17T09:51:37.201 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-client (14+238) ... 2024-04-17T09:51:37.302 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-17T09:51:37.353 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-17T09:51:37.369 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-17T09:51:37.411 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-17T09:51:37.892 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:51:37.892 INFO:teuthology.orchestra.run.smithi032.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-17T09:51:37.892 INFO:teuthology.orchestra.run.smithi032.stdout:which means they are currently configured by ifupdown: 2024-04-17T09:51:37.892 INFO:teuthology.orchestra.run.smithi032.stdout:- ens1f0 2024-04-17T09:51:37.892 INFO:teuthology.orchestra.run.smithi032.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-17T09:51:37.892 INFO:teuthology.orchestra.run.smithi032.stdout:remove their configuration from /etc/network/interfaces. 2024-04-17T09:51:37.892 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:51:38.032 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-17T09:51:38.232 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-17T09:51:38.248 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-17T09:51:38.290 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T09:51:38.441 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-17T09:51:38.610 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-17T09:51:38.626 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-17T09:51:38.693 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-17T09:51:38.885 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-17T09:51:38.971 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-17T09:51:38.987 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-17T09:51:39.029 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-17T09:51:39.324 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-17T09:51:39.340 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-17T09:51:39.382 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-17T09:51:39.613 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-17T09:51:39.764 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-17T09:51:39.860 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-17T09:51:39.875 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-17T09:51:39.890 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-17T09:51:39.910 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-17T09:51:40.033 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-common (238) ... 2024-04-17T09:51:40.288 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package modemmanager. 2024-04-17T09:51:40.303 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-17T09:51:40.346 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-17T09:51:40.978 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package network-manager. 2024-04-17T09:51:40.993 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-17T09:51:41.044 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-17T09:51:41.298 INFO:teuthology.orchestra.run.smithi032.stdout:Adding user postgres to group ssl-cert 2024-04-17T09:51:41.619 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ppp. 2024-04-17T09:51:41.634 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-17T09:51:41.641 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:51:41.641 INFO:teuthology.orchestra.run.smithi032.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-17T09:51:41.910 INFO:teuthology.orchestra.run.smithi032.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-17T09:51:41.910 INFO:teuthology.orchestra.run.smithi032.stdout:Removing obsolete dictionary files: 2024-04-17T09:51:41.991 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-17T09:51:42.324 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package pptp-linux. 2024-04-17T09:51:42.338 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-17T09:51:42.373 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-17T09:51:42.593 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-17T09:51:42.608 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-17T09:51:42.635 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-17T09:51:42.651 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-17T09:51:42.988 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-17T09:51:43.003 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-17T09:51:43.046 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking postgresql-client-common (238) ... 2024-04-17T09:51:43.093 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T09:51:43.367 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-17T09:51:43.381 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-17T09:51:43.425 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T09:51:43.431 INFO:teuthology.orchestra.run.smithi032.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-17T09:51:43.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-17T09:51:43.503 INFO:teuthology.orchestra.run.smithi032.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-17T09:51:43.504 INFO:teuthology.orchestra.run.smithi032.stdout:This user must also own the server process. 2024-04-17T09:51:43.504 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:51:43.504 INFO:teuthology.orchestra.run.smithi032.stdout:The database cluster will be initialized with locale "en_US". 2024-04-17T09:51:43.504 INFO:teuthology.orchestra.run.smithi032.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-17T09:51:43.504 INFO:teuthology.orchestra.run.smithi032.stdout:The default text search configuration will be set to "english". 2024-04-17T09:51:43.504 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:51:43.504 INFO:teuthology.orchestra.run.smithi032.stdout:Data page checksums are disabled. 2024-04-17T09:51:43.504 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:51:43.504 INFO:teuthology.orchestra.run.smithi032.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-17T09:51:43.505 INFO:teuthology.orchestra.run.smithi032.stdout:creating subdirectories ... ok 2024-04-17T09:51:43.505 INFO:teuthology.orchestra.run.smithi032.stdout:selecting dynamic shared memory implementation ... posix 2024-04-17T09:51:43.529 INFO:teuthology.orchestra.run.smithi032.stdout:selecting default max_connections ... 100 2024-04-17T09:51:43.564 INFO:teuthology.orchestra.run.smithi032.stdout:selecting default shared_buffers ... 128MB 2024-04-17T09:51:43.580 INFO:teuthology.orchestra.run.smithi032.stdout:selecting default time zone ... Etc/UTC 2024-04-17T09:51:43.581 INFO:teuthology.orchestra.run.smithi032.stdout:creating configuration files ... ok 2024-04-17T09:51:43.754 INFO:teuthology.orchestra.run.smithi032.stdout:running bootstrap script ... ok 2024-04-17T09:51:44.013 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package postgresql-common. 2024-04-17T09:51:44.027 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-17T09:51:44.089 INFO:teuthology.orchestra.run.smithi144.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-17T09:51:44.119 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking postgresql-common (238) ... 2024-04-17T09:51:44.386 INFO:teuthology.orchestra.run.smithi032.stdout:performing post-bootstrap initialization ... ok 2024-04-17T09:51:44.620 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package postgresql-14. 2024-04-17T09:51:44.635 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-17T09:51:44.680 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T09:51:44.938 INFO:teuthology.orchestra.run.smithi032.stdout:syncing data to disk ... ok 2024-04-17T09:51:45.469 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-17T09:51:45.934 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package postgresql. 2024-04-17T09:51:45.949 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-17T09:51:45.990 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking postgresql (14+238) ... 2024-04-17T09:51:46.200 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package postgresql-client. 2024-04-17T09:51:46.215 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-17T09:51:46.258 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking postgresql-client (14+238) ... 2024-04-17T09:51:46.527 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-17T09:51:46.540 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-17T09:51:46.577 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-17T09:51:46.879 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package tcl8.6. 2024-04-17T09:51:46.894 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-17T09:51:46.929 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-17T09:51:47.224 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package tcl. 2024-04-17T09:51:47.237 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-17T09:51:47.278 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-17T09:51:47.610 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package wpasupplicant. 2024-04-17T09:51:47.624 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-17T09:51:47.668 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-17T09:51:48.039 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-17T09:51:48.053 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-17T09:51:48.106 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-17T09:51:48.419 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-17T09:51:48.518 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-17T09:51:48.531 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-17T09:51:48.545 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-17T09:51:48.584 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-17T09:51:48.672 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql (14+238) ... 2024-04-17T09:51:48.798 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-17T09:51:48.868 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up postgresql-client-common (238) ... 2024-04-17T09:51:48.941 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-17T09:51:49.115 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-17T09:51:49.210 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-17T09:51:49.210 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-17T09:51:49.241 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-17T09:51:49.367 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T09:51:49.540 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-17T09:51:49.669 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-17T09:51:49.787 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T09:51:49.960 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-17T09:51:50.044 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-17T09:51:50.379 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-17T09:51:50.549 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-17T09:51:50.730 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-17T09:51:50.826 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-17T09:51:51.238 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-17T09:51:51.238 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-17T09:51:51.945 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-17T09:51:52.018 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-17T09:51:52.110 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-17T09:51:52.113 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-17T09:51:52.248 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-17T09:51:52.373 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-17T09:51:52.500 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-17T09:51:52.657 INFO:teuthology.orchestra.run.smithi032.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-17T09:51:52.659 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-17T09:51:52.664 DEBUG:teuthology.parallel:result is None 2024-04-17T09:51:52.868 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-17T09:51:52.986 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-17T09:51:54.025 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-17T09:51:54.184 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up dump (0.4b47-3) ... 2024-04-17T09:51:54.279 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-17T09:51:54.344 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-17T09:51:54.462 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-17T09:51:54.562 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-17T09:51:54.713 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-17T09:51:54.841 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-17T09:51:54.931 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-17T09:51:55.001 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-17T09:51:55.128 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-17T09:51:55.329 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-17T09:51:55.438 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-17T09:51:55.564 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-17T09:51:55.765 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up postgresql-client (14+238) ... 2024-04-17T09:51:55.891 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-17T09:51:56.497 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:51:56.497 INFO:teuthology.orchestra.run.smithi144.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-17T09:51:56.497 INFO:teuthology.orchestra.run.smithi144.stdout:which means they are currently configured by ifupdown: 2024-04-17T09:51:56.497 INFO:teuthology.orchestra.run.smithi144.stdout:- enp3s0f1 2024-04-17T09:51:56.497 INFO:teuthology.orchestra.run.smithi144.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-17T09:51:56.498 INFO:teuthology.orchestra.run.smithi144.stdout:remove their configuration from /etc/network/interfaces. 2024-04-17T09:51:56.498 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:51:56.631 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-17T09:51:57.019 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-17T09:51:57.430 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-17T09:51:58.152 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-17T09:51:58.306 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-17T09:51:58.438 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-17T09:51:58.564 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up postgresql-common (238) ... 2024-04-17T09:51:59.521 INFO:teuthology.orchestra.run.smithi144.stdout:Adding user postgres to group ssl-cert 2024-04-17T09:51:59.734 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:51:59.735 INFO:teuthology.orchestra.run.smithi144.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-17T09:51:59.983 INFO:teuthology.orchestra.run.smithi144.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-17T09:51:59.984 INFO:teuthology.orchestra.run.smithi144.stdout:Removing obsolete dictionary files: 2024-04-17T09:52:00.718 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-17T09:52:01.358 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-17T09:52:01.701 INFO:teuthology.orchestra.run.smithi144.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-17T09:52:01.718 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-17T09:52:01.758 INFO:teuthology.orchestra.run.smithi144.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-17T09:52:01.758 INFO:teuthology.orchestra.run.smithi144.stdout:This user must also own the server process. 2024-04-17T09:52:01.758 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:01.758 INFO:teuthology.orchestra.run.smithi144.stdout:The database cluster will be initialized with locale "en_US". 2024-04-17T09:52:01.759 INFO:teuthology.orchestra.run.smithi144.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-17T09:52:01.759 INFO:teuthology.orchestra.run.smithi144.stdout:The default text search configuration will be set to "english". 2024-04-17T09:52:01.759 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:01.759 INFO:teuthology.orchestra.run.smithi144.stdout:Data page checksums are disabled. 2024-04-17T09:52:01.759 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:01.759 INFO:teuthology.orchestra.run.smithi144.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-17T09:52:01.760 INFO:teuthology.orchestra.run.smithi144.stdout:creating subdirectories ... ok 2024-04-17T09:52:01.760 INFO:teuthology.orchestra.run.smithi144.stdout:selecting dynamic shared memory implementation ... posix 2024-04-17T09:52:01.783 INFO:teuthology.orchestra.run.smithi144.stdout:selecting default max_connections ... 100 2024-04-17T09:52:01.816 INFO:teuthology.orchestra.run.smithi144.stdout:selecting default shared_buffers ... 128MB 2024-04-17T09:52:01.831 INFO:teuthology.orchestra.run.smithi144.stdout:selecting default time zone ... Etc/UTC 2024-04-17T09:52:01.832 INFO:teuthology.orchestra.run.smithi144.stdout:creating configuration files ... ok 2024-04-17T09:52:01.995 INFO:teuthology.orchestra.run.smithi144.stdout:running bootstrap script ... ok 2024-04-17T09:52:02.614 INFO:teuthology.orchestra.run.smithi144.stdout:performing post-bootstrap initialization ... ok 2024-04-17T09:52:03.206 INFO:teuthology.orchestra.run.smithi144.stdout:syncing data to disk ... ok 2024-04-17T09:52:03.739 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-17T09:52:06.797 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-17T09:52:06.923 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-17T09:52:07.058 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up postgresql (14+238) ... 2024-04-17T09:52:07.209 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-17T09:52:07.334 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-17T09:52:07.597 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-17T09:52:07.597 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-17T09:52:08.362 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-17T09:52:08.437 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-17T09:52:09.098 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-17T09:52:09.258 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-17T09:52:10.694 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-17T09:52:10.786 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-17T09:52:11.311 INFO:teuthology.orchestra.run.smithi144.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-17T09:52:11.317 DEBUG:teuthology.parallel:result is None 2024-04-17T09:52:11.318 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T09:52:11.608 DEBUG:teuthology.orchestra.run.smithi032:> dpkg-query -W -f '${Version}' ceph 2024-04-17T09:52:11.634 INFO:teuthology.orchestra.run.smithi032.stdout:19.0.0-3133-ga8c84f59-1jammy 2024-04-17T09:52:11.634 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133-ga8c84f59-1jammy 2024-04-17T09:52:11.634 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133-ga8c84f59-1jammy is installed. 2024-04-17T09:52:11.636 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T09:52:11.769 DEBUG:teuthology.orchestra.run.smithi144:> dpkg-query -W -f '${Version}' ceph 2024-04-17T09:52:11.792 INFO:teuthology.orchestra.run.smithi144.stdout:19.0.0-3133-ga8c84f59-1jammy 2024-04-17T09:52:11.792 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133-ga8c84f59-1jammy 2024-04-17T09:52:11.792 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133-ga8c84f59-1jammy is installed. 2024-04-17T09:52:11.793 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-17T09:52:11.793 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:11.794 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-17T09:52:11.816 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:52:11.816 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-17T09:52:11.853 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-17T09:52:11.853 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:11.853 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/daemon-helper 2024-04-17T09:52:11.883 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-17T09:52:11.952 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:52:11.952 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/daemon-helper 2024-04-17T09:52:11.971 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-17T09:52:12.031 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-17T09:52:12.031 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:12.032 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-17T09:52:12.053 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-17T09:52:12.115 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:52:12.116 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-17T09:52:12.134 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-17T09:52:12.195 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-17T09:52:12.195 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:12.196 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/stdin-killer 2024-04-17T09:52:12.217 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-17T09:52:12.280 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:52:12.280 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/stdin-killer 2024-04-17T09:52:12.298 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-17T09:52:12.359 INFO:teuthology.run_tasks:Running task ceph... 2024-04-17T09:52:12.449 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-17T09:52:12.449 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 777 /var/log/ceph 2024-04-17T09:52:12.453 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 777 /var/log/ceph 2024-04-17T09:52:12.471 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-17T09:52:12.471 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-17T09:52:12.518 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-17T09:52:12.536 INFO:tasks.ceph:Creating extra log directories... 2024-04-17T09:52:12.536 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-17T09:52:12.581 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-17T09:52:12.607 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-17T09:52:12.607 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-17T09:52:12.607 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-17T09:52:12.607 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:12.608 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-17T09:52:12.651 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-17T09:52:12.750 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:52:12.750 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-17T09:52:12.758 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-17T09:52:12.850 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-17T09:52:12.850 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-17T09:52:12.850 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660029', 'branch': 'wip-rishabh-testing-20240416.193735', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub}', 'email': None, 'first_in_suite': False, 'job_id': '7660029', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240416.193735'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240416.193735', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 3521, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'sleep_before_teardown': 0, 'subset': '410/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240416.193735', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi032.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAQNVG61I8ge12Sn5QATesek5EUC5OJJA2Lfl5bn1TFZolZGwJ5+ic2yXdRNKlEZ2eFM7ed+K/Dl/RatKvM26U8=', 'smithi144.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDmzWqi31jF9Vw1PLRKqPj3ams9Rzq8QnAwo7r+CIq6CX4JoYwuACajkXfQHwmdK6OwV3MpfHFw9u4d/frZwoZg='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '6c637841c215537a4502385240412f1966e0faab', 'timestamp': '2024-04-17_04:58:44', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-17T09:52:12.851 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-17T09:52:12.854 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-17T09:52:12.893 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/run/ceph 2024-04-17T09:52:12.907 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m0777 -- /var/run/ceph 2024-04-17T09:52:12.956 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:12.956 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/scratch_devs of=/dev/stdout 2024-04-17T09:52:12.975 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-17T09:52:12.975 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_1 2024-04-17T09:52:13.027 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-17T09:52:13.027 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T09:52:13.027 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 902 Links: 1 2024-04-17T09:52:13.027 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T09:52:13.027 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-04-17 09:45:37.202514653 +0000 2024-04-17T09:52:13.027 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-04-17 09:45:36.926517119 +0000 2024-04-17T09:52:13.027 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-04-17 09:45:36.926517119 +0000 2024-04-17T09:52:13.027 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-04-17 09:45:36.926517119 +0000 2024-04-17T09:52:13.028 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-17T09:52:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-04-17T09:52:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-04-17T09:52:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.00027669 s, 1.9 MB/s 2024-04-17T09:52:13.092 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-17T09:52:13.145 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_2 2024-04-17T09:52:13.195 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-17T09:52:13.195 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T09:52:13.195 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 920 Links: 1 2024-04-17T09:52:13.196 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T09:52:13.196 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-04-17 09:45:37.786509437 +0000 2024-04-17T09:52:13.196 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-04-17 09:45:37.502511974 +0000 2024-04-17T09:52:13.196 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-04-17 09:45:37.502511974 +0000 2024-04-17T09:52:13.196 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-04-17 09:45:37.502511974 +0000 2024-04-17T09:52:13.196 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-17T09:52:13.258 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-04-17T09:52:13.258 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-04-17T09:52:13.258 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000264326 s, 1.9 MB/s 2024-04-17T09:52:13.260 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-17T09:52:13.317 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_3 2024-04-17T09:52:13.368 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-17T09:52:13.368 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T09:52:13.368 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 938 Links: 1 2024-04-17T09:52:13.368 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T09:52:13.368 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-04-17 09:45:38.334504542 +0000 2024-04-17T09:52:13.368 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-04-17 09:45:38.062506971 +0000 2024-04-17T09:52:13.368 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-04-17 09:45:38.062506971 +0000 2024-04-17T09:52:13.368 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-04-17 09:45:38.062506971 +0000 2024-04-17T09:52:13.368 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-17T09:52:13.430 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-04-17T09:52:13.430 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-04-17T09:52:13.430 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000260939 s, 2.0 MB/s 2024-04-17T09:52:13.432 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-17T09:52:13.485 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_4 2024-04-17T09:52:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-17T09:52:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T09:52:13.540 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 952 Links: 1 2024-04-17T09:52:13.540 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T09:52:13.540 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-04-17 09:45:38.922499290 +0000 2024-04-17T09:52:13.540 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-04-17 09:45:38.646501756 +0000 2024-04-17T09:52:13.540 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-04-17 09:45:38.646501756 +0000 2024-04-17T09:52:13.540 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-04-17 09:45:38.646501756 +0000 2024-04-17T09:52:13.540 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-17T09:52:13.605 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-04-17T09:52:13.605 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-04-17T09:52:13.605 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000259984 s, 2.0 MB/s 2024-04-17T09:52:13.607 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-17T09:52:13.665 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T09:52:13.665 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:52:13.665 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/scratch_devs of=/dev/stdout 2024-04-17T09:52:13.674 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-17T09:52:13.674 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_1 2024-04-17T09:52:13.725 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-17T09:52:13.726 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T09:52:13.726 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 884 Links: 1 2024-04-17T09:52:13.726 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T09:52:13.726 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-04-17 09:45:35.113599833 +0000 2024-04-17T09:52:13.726 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-04-17 09:45:34.809602549 +0000 2024-04-17T09:52:13.726 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-04-17 09:45:34.809602549 +0000 2024-04-17T09:52:13.726 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-04-17 09:45:34.809602549 +0000 2024-04-17T09:52:13.726 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-17T09:52:13.787 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-04-17T09:52:13.787 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-04-17T09:52:13.787 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000374017 s, 1.4 MB/s 2024-04-17T09:52:13.789 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-17T09:52:13.839 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_2 2024-04-17T09:52:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-17T09:52:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T09:52:13.889 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 899 Links: 1 2024-04-17T09:52:13.889 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T09:52:13.889 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-04-17 09:45:35.729594332 +0000 2024-04-17T09:52:13.889 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-04-17 09:45:35.457596762 +0000 2024-04-17T09:52:13.890 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-04-17 09:45:35.457596762 +0000 2024-04-17T09:52:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-04-17 09:45:35.457596762 +0000 2024-04-17T09:52:13.890 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-17T09:52:13.947 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-04-17T09:52:13.947 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-04-17T09:52:13.947 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000359667 s, 1.4 MB/s 2024-04-17T09:52:13.949 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-17T09:52:13.999 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_3 2024-04-17T09:52:14.049 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-17T09:52:14.049 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T09:52:14.049 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 918 Links: 1 2024-04-17T09:52:14.049 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T09:52:14.049 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-04-17 09:45:36.337588902 +0000 2024-04-17T09:52:14.049 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-04-17 09:45:36.065591330 +0000 2024-04-17T09:52:14.049 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-04-17 09:45:36.065591330 +0000 2024-04-17T09:52:14.049 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-04-17 09:45:36.065591330 +0000 2024-04-17T09:52:14.050 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-17T09:52:14.106 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-04-17T09:52:14.107 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-04-17T09:52:14.107 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000622652 s, 822 kB/s 2024-04-17T09:52:14.108 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-17T09:52:14.159 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_4 2024-04-17T09:52:14.209 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-17T09:52:14.209 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T09:52:14.209 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 936 Links: 1 2024-04-17T09:52:14.209 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T09:52:14.209 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-04-17 09:45:36.925583650 +0000 2024-04-17T09:52:14.209 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-04-17 09:45:36.637586222 +0000 2024-04-17T09:52:14.210 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-04-17 09:45:36.637586222 +0000 2024-04-17T09:52:14.210 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-04-17 09:45:36.637586222 +0000 2024-04-17T09:52:14.210 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-17T09:52:14.270 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-04-17T09:52:14.271 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-04-17T09:52:14.271 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000637907 s, 803 kB/s 2024-04-17T09:52:14.272 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-17T09:52:14.323 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-17T09:52:14.323 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi144.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-17T09:52:14.323 INFO:tasks.ceph:Generating config... 2024-04-17T09:52:14.324 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-17T09:52:14.324 INFO:tasks.ceph:[client] debug client = 20 2024-04-17T09:52:14.324 INFO:tasks.ceph:[client] debug ms = 1 2024-04-17T09:52:14.325 INFO:tasks.ceph:[client] pid file = 2024-04-17T09:52:14.325 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-17T09:52:14.325 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-17T09:52:14.325 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-17T09:52:14.325 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-17T09:52:14.325 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-17T09:52:14.325 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-17T09:52:14.325 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-17T09:52:14.325 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-17T09:52:14.325 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-17T09:52:14.325 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-17T09:52:14.325 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-17T09:52:14.325 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-17T09:52:14.325 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-17T09:52:14.325 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-17T09:52:14.326 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-17T09:52:14.326 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-17T09:52:14.326 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-17T09:52:14.326 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-17T09:52:14.326 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-17T09:52:14.326 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-17T09:52:14.326 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-17T09:52:14.326 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-17T09:52:14.326 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-17T09:52:14.326 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-17T09:52:14.326 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-17T09:52:14.326 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-17T09:52:14.326 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-17T09:52:14.327 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-17T09:52:14.327 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-17T09:52:14.327 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-17T09:52:14.327 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-17T09:52:14.327 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-17T09:52:14.327 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-17T09:52:14.327 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-17T09:52:14.327 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-17T09:52:14.327 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-17T09:52:14.327 INFO:tasks.ceph:Setting up mon.a... 2024-04-17T09:52:14.327 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-17T09:52:14.374 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.keyring 2024-04-17T09:52:14.389 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-17T09:52:14.479 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-17T09:52:14.499 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.32'), ('mon.b', '172.21.15.144'), ('mon.c', '[v2:172.21.15.144:3301,v1:172.21.15.144:6790]')] 2024-04-17T09:52:14.499 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.32,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-17T09:52:14.500 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:14.500 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-17T09:52:14.551 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.32 --add b 172.21.15.144 --addv c '[v2:172.21.15.144:3301,v1:172.21.15.144:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-17T09:52:14.647 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-17T09:52:14.647 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: generated fsid ff178df8-4c06-4367-a1e9-2e3a66d870d0 2024-04-17T09:52:14.647 INFO:teuthology.orchestra.run.smithi032.stdout:setting min_mon_release = quincy 2024-04-17T09:52:14.647 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 0 2024-04-17T09:52:14.647 INFO:teuthology.orchestra.run.smithi032.stdout:fsid ff178df8-4c06-4367-a1e9-2e3a66d870d0 2024-04-17T09:52:14.647 INFO:teuthology.orchestra.run.smithi032.stdout:last_changed 2024-04-17T09:52:14.650495+0000 2024-04-17T09:52:14.647 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-04-17T09:52:14.650495+0000 2024-04-17T09:52:14.647 INFO:teuthology.orchestra.run.smithi032.stdout:min_mon_release 17 (quincy) 2024-04-17T09:52:14.647 INFO:teuthology.orchestra.run.smithi032.stdout:election_strategy: 1 2024-04-17T09:52:14.647 INFO:teuthology.orchestra.run.smithi032.stdout:0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-04-17T09:52:14.647 INFO:teuthology.orchestra.run.smithi032.stdout:1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-04-17T09:52:14.648 INFO:teuthology.orchestra.run.smithi032.stdout:2: [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] mon.c 2024-04-17T09:52:14.648 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-17T09:52:14.650 DEBUG:teuthology.orchestra.run.smithi032:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-17T09:52:14.658 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID ff178df8-4c06-4367-a1e9-2e3a66d870d0... 2024-04-17T09:52:14.659 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-17T09:52:14.701 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-17T09:52:14.742 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2024-04-17T09:52:14.742 INFO:teuthology.orchestra.run.smithi032.stdout: chdir = "" 2024-04-17T09:52:14.742 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-17T09:52:14.742 INFO:teuthology.orchestra.run.smithi032.stdout: auth supported = cephx 2024-04-17T09:52:14.742 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.742 INFO:teuthology.orchestra.run.smithi032.stdout: filestore xattr use omap = true 2024-04-17T09:52:14.742 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.742 INFO:teuthology.orchestra.run.smithi032.stdout: mon clock drift allowed = 1.000 2024-04-17T09:52:14.742 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.742 INFO:teuthology.orchestra.run.smithi032.stdout: osd crush chooseleaf type = 0 2024-04-17T09:52:14.743 INFO:teuthology.orchestra.run.smithi032.stdout: auth debug = true 2024-04-17T09:52:14.743 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.743 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on old message = true 2024-04-17T09:52:14.743 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on bug = true 2024-04-17T09:52:14.743 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.743 INFO:teuthology.orchestra.run.smithi032.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-17T09:52:14.743 INFO:teuthology.orchestra.run.smithi032.stdout: mon pg warn max object skew = 0 2024-04-17T09:52:14.743 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.743 INFO:teuthology.orchestra.run.smithi032.stdout: # disable pg_autoscaler by default for new pools 2024-04-17T09:52:14.743 INFO:teuthology.orchestra.run.smithi032.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-17T09:52:14.743 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.743 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default size = 2 2024-04-17T09:52:14.743 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.743 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow primary affinity = true 2024-04-17T09:52:14.743 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow pg remap = true 2024-04-17T09:52:14.743 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on legacy crush tunables = false 2024-04-17T09:52:14.744 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on crush straw calc version zero = false 2024-04-17T09:52:14.744 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on no sortbitwise = false 2024-04-17T09:52:14.744 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on osd down out interval zero = false 2024-04-17T09:52:14.744 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on too few osds = false 2024-04-17T09:52:14.744 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-17T09:52:14.744 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-17T09:52:14.744 INFO:teuthology.orchestra.run.smithi032.stdout: mon_allow_pool_size_one = true 2024-04-17T09:52:14.744 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.744 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-17T09:52:14.744 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.744 INFO:teuthology.orchestra.run.smithi032.stdout: osd default data pool replay window = 5 2024-04-17T09:52:14.744 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.744 INFO:teuthology.orchestra.run.smithi032.stdout: mon allow pool delete = true 2024-04-17T09:52:14.744 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.744 INFO:teuthology.orchestra.run.smithi032.stdout: mon cluster log file level = debug 2024-04-17T09:52:14.745 INFO:teuthology.orchestra.run.smithi032.stdout: debug asserts on shutdown = true 2024-04-17T09:52:14.745 INFO:teuthology.orchestra.run.smithi032.stdout: mon health detail to clog = false 2024-04-17T09:52:14.745 INFO:teuthology.orchestra.run.smithi032.stdout: mon host = "172.21.15.32,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2024-04-17T09:52:14.745 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = ff178df8-4c06-4367-a1e9-2e3a66d870d0 2024-04-17T09:52:14.745 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.745 INFO:teuthology.orchestra.run.smithi032.stdout:[osd] 2024-04-17T09:52:14.745 INFO:teuthology.orchestra.run.smithi032.stdout: osd journal size = 100 2024-04-17T09:52:14.745 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.745 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub load threshold = 5.0 2024-04-17T09:52:14.745 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub max interval = 600 2024-04-17T09:52:14.745 INFO:teuthology.orchestra.run.smithi032.stdout: osd mclock profile = high_recovery_ops 2024-04-17T09:52:14.745 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.745 INFO:teuthology.orchestra.run.smithi032.stdout: osd recover clone overlap = true 2024-04-17T09:52:14.746 INFO:teuthology.orchestra.run.smithi032.stdout: osd recovery max chunk = 1048576 2024-04-17T09:52:14.746 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.746 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug shutdown = true 2024-04-17T09:52:14.746 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug op order = true 2024-04-17T09:52:14.746 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug verify stray on activate = true 2024-04-17T09:52:14.746 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug trim objects = true 2024-04-17T09:52:14.746 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.746 INFO:teuthology.orchestra.run.smithi032.stdout: osd open classes on start = true 2024-04-17T09:52:14.746 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug pg log writeout = true 2024-04-17T09:52:14.746 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.746 INFO:teuthology.orchestra.run.smithi032.stdout: osd deep scrub update digest min age = 30 2024-04-17T09:52:14.746 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.746 INFO:teuthology.orchestra.run.smithi032.stdout: osd map max advance = 10 2024-04-17T09:52:14.747 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.747 INFO:teuthology.orchestra.run.smithi032.stdout: journal zero on create = true 2024-04-17T09:52:14.747 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.747 INFO:teuthology.orchestra.run.smithi032.stdout: filestore ondisk finisher threads = 3 2024-04-17T09:52:14.747 INFO:teuthology.orchestra.run.smithi032.stdout: filestore apply finisher threads = 3 2024-04-17T09:52:14.747 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.747 INFO:teuthology.orchestra.run.smithi032.stdout: bdev debug aio = true 2024-04-17T09:52:14.747 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug misdirected ops = true 2024-04-17T09:52:14.747 INFO:teuthology.orchestra.run.smithi032.stdout: bdev async discard = True 2024-04-17T09:52:14.747 INFO:teuthology.orchestra.run.smithi032.stdout: bdev enable discard = True 2024-04-17T09:52:14.747 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore allocator = bitmap 2024-04-17T09:52:14.747 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore block size = 96636764160 2024-04-17T09:52:14.747 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore fsck on mount = True 2024-04-17T09:52:14.747 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluefs = 1/20 2024-04-17T09:52:14.748 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluestore = 1/20 2024-04-17T09:52:14.748 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T09:52:14.748 INFO:teuthology.orchestra.run.smithi032.stdout: debug osd = 20 2024-04-17T09:52:14.748 INFO:teuthology.orchestra.run.smithi032.stdout: debug rocksdb = 4/10 2024-04-17T09:52:14.748 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd backfillfull_ratio = 0.85 2024-04-17T09:52:14.748 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd full ratio = 0.9 2024-04-17T09:52:14.749 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd nearfull ratio = 0.8 2024-04-17T09:52:14.749 INFO:teuthology.orchestra.run.smithi032.stdout: osd failsafe full ratio = 0.95 2024-04-17T09:52:14.749 INFO:teuthology.orchestra.run.smithi032.stdout: osd objectstore = bluestore 2024-04-17T09:52:14.749 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2024-04-17T09:52:14.749 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.749 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr] 2024-04-17T09:52:14.749 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T09:52:14.749 INFO:teuthology.orchestra.run.smithi032.stdout: debug mgr = 20 2024-04-17T09:52:14.749 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2024-04-17T09:52:14.749 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2024-04-17T09:52:14.750 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2024-04-17T09:52:14.750 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2024-04-17T09:52:14.750 INFO:teuthology.orchestra.run.smithi032.stdout: mgr/telemetry/nag = false 2024-04-17T09:52:14.750 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2024-04-17T09:52:14.750 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.750 INFO:teuthology.orchestra.run.smithi032.stdout:[mon] 2024-04-17T09:52:14.750 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T09:52:14.750 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2024-04-17T09:52:14.750 INFO:teuthology.orchestra.run.smithi032.stdout: debug paxos = 20 2024-04-17T09:52:14.750 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2024-04-17T09:52:14.750 INFO:teuthology.orchestra.run.smithi032.stdout: mon data avail warn = 5 2024-04-17T09:52:14.750 INFO:teuthology.orchestra.run.smithi032.stdout: mon mgr mkfs grace = 240 2024-04-17T09:52:14.750 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2024-04-17T09:52:14.751 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd reporter subtree level = osd 2024-04-17T09:52:14.751 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd prime pg temp = true 2024-04-17T09:52:14.751 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2024-04-17T09:52:14.751 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.751 INFO:teuthology.orchestra.run.smithi032.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-17T09:52:14.751 INFO:teuthology.orchestra.run.smithi032.stdout: auth mon ticket ttl = 660 # 11m 2024-04-17T09:52:14.751 INFO:teuthology.orchestra.run.smithi032.stdout: auth service ticket ttl = 240 # 4m 2024-04-17T09:52:14.751 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.751 INFO:teuthology.orchestra.run.smithi032.stdout: # don't complain about insecure global_id in the test suite 2024-04-17T09:52:14.751 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-17T09:52:14.751 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-17T09:52:14.751 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.751 INFO:teuthology.orchestra.run.smithi032.stdout: # 1m isn't quite enough 2024-04-17T09:52:14.751 INFO:teuthology.orchestra.run.smithi032.stdout: mon_down_mkfs_grace = 2m 2024-04-17T09:52:14.751 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.752 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_filestore_osds = false 2024-04-17T09:52:14.752 INFO:teuthology.orchestra.run.smithi032.stdout: mon down mkfs grace = 300 2024-04-17T09:52:14.752 INFO:teuthology.orchestra.run.smithi032.stdout: mon op complaint time = 120 2024-04-17T09:52:14.752 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:14.752 INFO:teuthology.orchestra.run.smithi032.stdout:[client] 2024-04-17T09:52:14.752 INFO:teuthology.orchestra.run.smithi032.stdout: rgw cache enabled = true 2024-04-17T09:52:14.752 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable ops log = true 2024-04-17T09:52:14.752 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable usage log = true 2024-04-17T09:52:14.752 INFO:teuthology.orchestra.run.smithi032.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-17T09:52:14.752 INFO:teuthology.orchestra.run.smithi032.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-17T09:52:14.752 INFO:teuthology.orchestra.run.smithi032.stdout: client mount timeout = 600 2024-04-17T09:52:14.752 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2024-04-17T09:52:14.752 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T09:52:14.752 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = "" 2024-04-17T09:52:14.752 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2024-04-17T09:52:14.752 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2024-04-17T09:52:14.752 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.a] 2024-04-17T09:52:14.753 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.b] 2024-04-17T09:52:14.753 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.c] 2024-04-17T09:52:14.753 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2024-04-17T09:52:14.753 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c] 2024-04-17T09:52:14.753 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2024-04-17T09:52:14.753 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.d] 2024-04-17T09:52:14.753 INFO:teuthology.orchestra.run.smithi032.stdout:[mds] 2024-04-17T09:52:14.753 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds = 20 2024-04-17T09:52:14.753 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds balancer = 20 2024-04-17T09:52:14.753 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T09:52:14.753 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug frag = True 2024-04-17T09:52:14.753 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug scatterstat = True 2024-04-17T09:52:14.753 INFO:teuthology.orchestra.run.smithi032.stdout: mds op complaint time = 180 2024-04-17T09:52:14.753 INFO:teuthology.orchestra.run.smithi032.stdout: mds verify scatter = True 2024-04-17T09:52:14.753 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2024-04-17T09:52:14.754 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2024-04-17T09:52:14.754 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2024-04-17T09:52:14.754 INFO:teuthology.orchestra.run.smithi144.stdout:[global] 2024-04-17T09:52:14.754 INFO:teuthology.orchestra.run.smithi144.stdout: chdir = "" 2024-04-17T09:52:14.754 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-17T09:52:14.754 INFO:teuthology.orchestra.run.smithi144.stdout: auth supported = cephx 2024-04-17T09:52:14.754 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.755 INFO:teuthology.orchestra.run.smithi144.stdout: filestore xattr use omap = true 2024-04-17T09:52:14.755 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.755 INFO:teuthology.orchestra.run.smithi144.stdout: mon clock drift allowed = 1.000 2024-04-17T09:52:14.755 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.755 INFO:teuthology.orchestra.run.smithi144.stdout: osd crush chooseleaf type = 0 2024-04-17T09:52:14.755 INFO:teuthology.orchestra.run.smithi144.stdout: auth debug = true 2024-04-17T09:52:14.755 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.755 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on old message = true 2024-04-17T09:52:14.755 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on bug = true 2024-04-17T09:52:14.755 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.755 INFO:teuthology.orchestra.run.smithi144.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-17T09:52:14.755 INFO:teuthology.orchestra.run.smithi144.stdout: mon pg warn max object skew = 0 2024-04-17T09:52:14.755 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.755 INFO:teuthology.orchestra.run.smithi144.stdout: # disable pg_autoscaler by default for new pools 2024-04-17T09:52:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-17T09:52:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default size = 2 2024-04-17T09:52:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow primary affinity = true 2024-04-17T09:52:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow pg remap = true 2024-04-17T09:52:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on legacy crush tunables = false 2024-04-17T09:52:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on crush straw calc version zero = false 2024-04-17T09:52:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on no sortbitwise = false 2024-04-17T09:52:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on osd down out interval zero = false 2024-04-17T09:52:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on too few osds = false 2024-04-17T09:52:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-17T09:52:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-17T09:52:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: mon_allow_pool_size_one = true 2024-04-17T09:52:14.756 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-17T09:52:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: osd default data pool replay window = 5 2024-04-17T09:52:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: mon allow pool delete = true 2024-04-17T09:52:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: mon cluster log file level = debug 2024-04-17T09:52:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: debug asserts on shutdown = true 2024-04-17T09:52:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: mon health detail to clog = false 2024-04-17T09:52:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: mon host = "172.21.15.32,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2024-04-17T09:52:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: fsid = ff178df8-4c06-4367-a1e9-2e3a66d870d0 2024-04-17T09:52:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.757 INFO:teuthology.orchestra.run.smithi144.stdout:[osd] 2024-04-17T09:52:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: osd journal size = 100 2024-04-17T09:52:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.757 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub load threshold = 5.0 2024-04-17T09:52:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub max interval = 600 2024-04-17T09:52:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: osd mclock profile = high_recovery_ops 2024-04-17T09:52:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: osd recover clone overlap = true 2024-04-17T09:52:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: osd recovery max chunk = 1048576 2024-04-17T09:52:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug shutdown = true 2024-04-17T09:52:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug op order = true 2024-04-17T09:52:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug verify stray on activate = true 2024-04-17T09:52:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug trim objects = true 2024-04-17T09:52:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: osd open classes on start = true 2024-04-17T09:52:14.758 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug pg log writeout = true 2024-04-17T09:52:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: osd deep scrub update digest min age = 30 2024-04-17T09:52:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: osd map max advance = 10 2024-04-17T09:52:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: journal zero on create = true 2024-04-17T09:52:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: filestore ondisk finisher threads = 3 2024-04-17T09:52:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: filestore apply finisher threads = 3 2024-04-17T09:52:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: bdev debug aio = true 2024-04-17T09:52:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug misdirected ops = true 2024-04-17T09:52:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: bdev async discard = True 2024-04-17T09:52:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: bdev enable discard = True 2024-04-17T09:52:14.759 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore allocator = bitmap 2024-04-17T09:52:14.760 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore block size = 96636764160 2024-04-17T09:52:14.760 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore fsck on mount = True 2024-04-17T09:52:14.760 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluefs = 1/20 2024-04-17T09:52:14.760 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluestore = 1/20 2024-04-17T09:52:14.760 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T09:52:14.760 INFO:teuthology.orchestra.run.smithi144.stdout: debug osd = 20 2024-04-17T09:52:14.760 INFO:teuthology.orchestra.run.smithi144.stdout: debug rocksdb = 4/10 2024-04-17T09:52:14.760 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd backfillfull_ratio = 0.85 2024-04-17T09:52:14.760 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd full ratio = 0.9 2024-04-17T09:52:14.760 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd nearfull ratio = 0.8 2024-04-17T09:52:14.760 INFO:teuthology.orchestra.run.smithi144.stdout: osd failsafe full ratio = 0.95 2024-04-17T09:52:14.760 INFO:teuthology.orchestra.run.smithi144.stdout: osd objectstore = bluestore 2024-04-17T09:52:14.760 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2024-04-17T09:52:14.761 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.761 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr] 2024-04-17T09:52:14.761 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T09:52:14.761 INFO:teuthology.orchestra.run.smithi144.stdout: debug mgr = 20 2024-04-17T09:52:14.761 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-04-17T09:52:14.761 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-04-17T09:52:14.761 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-04-17T09:52:14.761 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-04-17T09:52:14.761 INFO:teuthology.orchestra.run.smithi144.stdout: mgr/telemetry/nag = false 2024-04-17T09:52:14.761 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2024-04-17T09:52:14.761 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.761 INFO:teuthology.orchestra.run.smithi144.stdout:[mon] 2024-04-17T09:52:14.761 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T09:52:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-04-17T09:52:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: debug paxos = 20 2024-04-17T09:52:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-04-17T09:52:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: mon data avail warn = 5 2024-04-17T09:52:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: mon mgr mkfs grace = 240 2024-04-17T09:52:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-04-17T09:52:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd reporter subtree level = osd 2024-04-17T09:52:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd prime pg temp = true 2024-04-17T09:52:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-04-17T09:52:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-17T09:52:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: auth mon ticket ttl = 660 # 11m 2024-04-17T09:52:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: auth service ticket ttl = 240 # 4m 2024-04-17T09:52:14.762 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: # don't complain about insecure global_id in the test suite 2024-04-17T09:52:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-17T09:52:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-17T09:52:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: # 1m isn't quite enough 2024-04-17T09:52:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: mon_down_mkfs_grace = 2m 2024-04-17T09:52:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_filestore_osds = false 2024-04-17T09:52:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: mon down mkfs grace = 300 2024-04-17T09:52:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: mon op complaint time = 120 2024-04-17T09:52:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T09:52:14.763 INFO:teuthology.orchestra.run.smithi144.stdout:[client] 2024-04-17T09:52:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: rgw cache enabled = true 2024-04-17T09:52:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable ops log = true 2024-04-17T09:52:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable usage log = true 2024-04-17T09:52:14.763 INFO:teuthology.orchestra.run.smithi144.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-17T09:52:14.764 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-17T09:52:14.764 INFO:teuthology.orchestra.run.smithi144.stdout: client mount timeout = 600 2024-04-17T09:52:14.764 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2024-04-17T09:52:14.764 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T09:52:14.764 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = "" 2024-04-17T09:52:14.764 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2024-04-17T09:52:14.764 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2024-04-17T09:52:14.764 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.a] 2024-04-17T09:52:14.764 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.b] 2024-04-17T09:52:14.764 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.c] 2024-04-17T09:52:14.764 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.a] 2024-04-17T09:52:14.764 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.c] 2024-04-17T09:52:14.764 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.b] 2024-04-17T09:52:14.765 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.d] 2024-04-17T09:52:14.765 INFO:teuthology.orchestra.run.smithi144.stdout:[mds] 2024-04-17T09:52:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds = 20 2024-04-17T09:52:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds balancer = 20 2024-04-17T09:52:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T09:52:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug frag = True 2024-04-17T09:52:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug scatterstat = True 2024-04-17T09:52:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: mds op complaint time = 180 2024-04-17T09:52:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: mds verify scatter = True 2024-04-17T09:52:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2024-04-17T09:52:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2024-04-17T09:52:14.765 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2024-04-17T09:52:14.767 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-17T09:52:14.767 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-17T09:52:14.868 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-17T09:52:14.868 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:14.868 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-17T09:52:14.876 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:14.876 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-17T09:52:14.926 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T09:52:14.926 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:14.927 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-17T09:52:14.927 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-17T09:52:14.998 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:14.998 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-17T09:52:15.046 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T09:52:15.047 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:52:15.047 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-17T09:52:15.047 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-17T09:52:15.080 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:52:15.080 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-17T09:52:15.133 INFO:tasks.ceph:Setting up mon nodes... 2024-04-17T09:52:15.133 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-17T09:52:15.133 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-17T09:52:15.189 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-17T09:52:15.195 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-17T09:52:15.251 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-17T09:52:15.264 INFO:tasks.ceph:Setting up mds nodes... 2024-04-17T09:52:15.265 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-17T09:52:15.320 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-17T09:52:15.324 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-17T09:52:15.345 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-17T09:52:15.444 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-17T09:52:15.449 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-17T09:52:15.469 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-17T09:52:15.525 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-17T09:52:15.530 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-17T09:52:15.549 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-17T09:52:15.651 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-17T09:52:15.656 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-17T09:52:15.675 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-17T09:52:15.675 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-17T09:52:15.718 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-17T09:52:15.735 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-17T09:52:15.821 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-17T09:52:15.838 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-17T09:52:15.878 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-17T09:52:15.895 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-17T09:52:15.981 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-17T09:52:15.998 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-17T09:52:15.998 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi144.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-17T09:52:15.998 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-17T09:52:16.018 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T09:52:16.018 INFO:tasks.ceph:role: osd.0 2024-04-17T09:52:16.018 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T09:52:16.018 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-17T09:52:16.078 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T09:52:16.079 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T09:52:16.079 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T09:52:16.079 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T09:52:16.079 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T09:52:16.079 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2024-04-17T09:52:16.079 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T09:52:16.079 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T09:52:16.079 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T09:52:16.079 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T09:52:16.102 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2024-04-17T09:52:16.105 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2024-04-17T09:52:16.105 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-17T09:52:16.177 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-17T09:52:16.238 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T09:52:16.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:52:16.239 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-17T09:52:16.305 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T09:52:16.306 INFO:tasks.ceph:role: osd.1 2024-04-17T09:52:16.306 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T09:52:16.306 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-17T09:52:16.365 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T09:52:16.365 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T09:52:16.365 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T09:52:16.365 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T09:52:16.366 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T09:52:16.366 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2024-04-17T09:52:16.366 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T09:52:16.366 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T09:52:16.366 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T09:52:16.366 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T09:52:16.390 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2024-04-17T09:52:16.393 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2024-04-17T09:52:16.393 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-17T09:52:16.459 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-17T09:52:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T09:52:16.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:52:16.519 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-17T09:52:16.582 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T09:52:16.582 INFO:tasks.ceph:role: osd.2 2024-04-17T09:52:16.582 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T09:52:16.582 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-17T09:52:16.646 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T09:52:16.646 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T09:52:16.646 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T09:52:16.646 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T09:52:16.646 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T09:52:16.646 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2024-04-17T09:52:16.646 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T09:52:16.646 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T09:52:16.647 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T09:52:16.647 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T09:52:16.670 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2024-04-17T09:52:16.674 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2024-04-17T09:52:16.674 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-17T09:52:16.740 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-17T09:52:16.798 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T09:52:16.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:52:16.799 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-17T09:52:16.862 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T09:52:16.862 INFO:tasks.ceph:role: osd.3 2024-04-17T09:52:16.862 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T09:52:16.862 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-17T09:52:16.921 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T09:52:16.921 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T09:52:16.921 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T09:52:16.922 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T09:52:16.922 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T09:52:16.922 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2024-04-17T09:52:16.922 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T09:52:16.922 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T09:52:16.922 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T09:52:16.922 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T09:52:16.945 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2024-04-17T09:52:16.949 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2024-04-17T09:52:16.949 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-17T09:52:17.016 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-17T09:52:17.074 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T09:52:17.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:52:17.075 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T09:52:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:17.201+0000 7f5fa4c8a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-17T09:52:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:17.201+0000 7f5fa4c8a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-17T09:52:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:17.201+0000 7f5fa4c8a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-17T09:52:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:17.201+0000 7f5fa4c8a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-17T09:52:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:17.201+0000 7f5fa4c8a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-17T09:52:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:17.205+0000 7f5fa4c8a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-17T09:52:20.289 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-17T09:52:20.311 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T09:52:20.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:20.409+0000 7f80032877c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-17T09:52:20.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:20.409+0000 7f80032877c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-17T09:52:20.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:20.409+0000 7f80032877c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-17T09:52:20.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:20.409+0000 7f80032877c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-17T09:52:20.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:20.409+0000 7f80032877c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-17T09:52:20.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:20.413+0000 7f80032877c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-17T09:52:23.434 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-17T09:52:23.456 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T09:52:23.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:23.553+0000 7f2e180127c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-17T09:52:23.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:23.553+0000 7f2e180127c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-17T09:52:23.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:23.553+0000 7f2e180127c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-17T09:52:23.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:23.553+0000 7f2e180127c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-17T09:52:23.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:23.553+0000 7f2e180127c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-17T09:52:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:23.557+0000 7f2e180127c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-17T09:52:26.610 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-17T09:52:26.633 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T09:52:26.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:26.733+0000 7f59ab5037c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-17T09:52:26.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:26.733+0000 7f59ab5037c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-17T09:52:26.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:26.733+0000 7f59ab5037c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-17T09:52:26.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:26.733+0000 7f59ab5037c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-17T09:52:26.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:26.733+0000 7f59ab5037c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-17T09:52:26.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:26.733+0000 7f59ab5037c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-17T09:52:29.817 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-17T09:52:29.839 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-17T09:52:29.861 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-17T09:52:29.861 INFO:tasks.ceph:role: osd.4 2024-04-17T09:52:29.861 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T09:52:29.861 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-17T09:52:29.933 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T09:52:29.933 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T09:52:29.933 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T09:52:29.933 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T09:52:29.933 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T09:52:29.933 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2024-04-17T09:52:29.933 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T09:52:29.933 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T09:52:29.933 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T09:52:29.933 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T09:52:30.348 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2024-04-17T09:52:30.351 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2024-04-17T09:52:30.351 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-17T09:52:30.494 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-17T09:52:30.511 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T09:52:30.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:52:30.512 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-17T09:52:30.578 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-17T09:52:30.578 INFO:tasks.ceph:role: osd.5 2024-04-17T09:52:30.578 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T09:52:30.578 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-17T09:52:30.648 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T09:52:30.648 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T09:52:30.648 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T09:52:30.648 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T09:52:30.649 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T09:52:30.649 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2024-04-17T09:52:30.649 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T09:52:30.649 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T09:52:30.649 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T09:52:30.649 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T09:52:31.008 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2024-04-17T09:52:31.011 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2024-04-17T09:52:31.011 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-17T09:52:31.089 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-17T09:52:31.150 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T09:52:31.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:52:31.151 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-17T09:52:31.213 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-17T09:52:31.213 INFO:tasks.ceph:role: osd.6 2024-04-17T09:52:31.213 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T09:52:31.214 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-17T09:52:31.285 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T09:52:31.285 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T09:52:31.285 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T09:52:31.286 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T09:52:31.286 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T09:52:31.286 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2024-04-17T09:52:31.286 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T09:52:31.286 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T09:52:31.286 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T09:52:31.286 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T09:52:31.680 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2024-04-17T09:52:31.682 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2024-04-17T09:52:31.683 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-17T09:52:31.794 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-17T09:52:31.811 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T09:52:31.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:52:31.812 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-17T09:52:31.877 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-17T09:52:31.877 INFO:tasks.ceph:role: osd.7 2024-04-17T09:52:31.878 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T09:52:31.878 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-17T09:52:31.949 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T09:52:31.949 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T09:52:31.950 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T09:52:31.950 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T09:52:31.950 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T09:52:31.950 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2024-04-17T09:52:31.950 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T09:52:31.950 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T09:52:31.950 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T09:52:31.950 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T09:52:32.376 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2024-04-17T09:52:32.380 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2024-04-17T09:52:32.381 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-17T09:52:32.504 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-17T09:52:32.521 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T09:52:32.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:52:32.522 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T09:52:32.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:32.653+0000 7fb9cbc2b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-17T09:52:32.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:32.657+0000 7fb9cbc2b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-17T09:52:32.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:32.657+0000 7fb9cbc2b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-17T09:52:32.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:32.657+0000 7fb9cbc2b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-17T09:52:32.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:32.657+0000 7fb9cbc2b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-17T09:52:32.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:32.665+0000 7fb9cbc2b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-17T09:52:35.736 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-17T09:52:35.757 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T09:52:35.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:35.845+0000 7f812210b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-17T09:52:35.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:35.845+0000 7f812210b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-17T09:52:35.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:35.845+0000 7f812210b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-17T09:52:35.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:35.845+0000 7f812210b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-17T09:52:35.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:35.845+0000 7f812210b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-17T09:52:35.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:35.845+0000 7f812210b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-17T09:52:38.883 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-17T09:52:38.903 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T09:52:38.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:38.997+0000 7feafebb47c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-17T09:52:39.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:38.997+0000 7feafebb47c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-17T09:52:39.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:38.997+0000 7feafebb47c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-17T09:52:39.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:38.997+0000 7feafebb47c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-17T09:52:39.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:38.997+0000 7feafebb47c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-17T09:52:39.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:39.001+0000 7feafebb47c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-17T09:52:42.031 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-17T09:52:42.051 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T09:52:42.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:42.141+0000 7f28441bd7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-17T09:52:42.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:42.141+0000 7f28441bd7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-17T09:52:42.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:42.141+0000 7f28441bd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-17T09:52:42.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:42.141+0000 7f28441bd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-17T09:52:42.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:42.141+0000 7f28441bd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-17T09:52:42.144 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:42.141+0000 7f28441bd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-17T09:52:42.852 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:52:42.855 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:52:45.180 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-17T09:52:45.200 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-17T09:52:45.200 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:45.200 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-17T09:52:45.222 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:45.222 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-17T09:52:45.284 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:45.284 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-17T09:52:45.348 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:45.392 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-17T09:52:45.413 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:45.413 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-17T09:52:45.481 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:45.481 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-17T09:52:45.549 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:45.549 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-17T09:52:45.611 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:52:45.611 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-17T09:52:45.631 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:52:45.632 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-17T09:52:45.693 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:52:45.693 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-17T09:52:45.753 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:52:45.753 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-17T09:52:45.812 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:52:45.813 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-17T09:52:45.872 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:52:45.872 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-17T09:52:45.936 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:52:45.936 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-17T09:52:45.996 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:45.996 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-17T09:52:46.005 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:46.005 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-17T09:52:46.056 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:52:46.056 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-17T09:52:46.064 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:52:46.064 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-17T09:52:46.113 INFO:tasks.ceph:Adding keys to all mons... 2024-04-17T09:52:46.113 DEBUG:teuthology.orchestra.run.smithi032:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-17T09:52:46.117 DEBUG:teuthology.orchestra.run.smithi144:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-17T09:52:46.157 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.y] 2024-04-17T09:52:46.157 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDPmx9m3FmACxAApri5AI5PeR0+/HLev8DKcw== 2024-04-17T09:52:46.157 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2024-04-17T09:52:46.158 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDPmx9mzM1KExAAn0WH+Gf6gGV/dJonXzAgPQ== 2024-04-17T09:52:46.158 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c] 2024-04-17T09:52:46.158 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDPmx9mWWi5GhAA3RAdRxK6GRwuVCH4DHoMeA== 2024-04-17T09:52:46.158 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.0] 2024-04-17T09:52:46.158 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDRmx9mnPwyDBAAxD10jhxptVH5zhupengElg== 2024-04-17T09:52:46.158 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.1] 2024-04-17T09:52:46.158 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDUmx9mh3iuGBAAqvWSRyNX3u/TjEH6TJpqdA== 2024-04-17T09:52:46.158 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.2] 2024-04-17T09:52:46.158 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDXmx9mbe1CIRAA1pmzAl04tGkTu6QLpmCkag== 2024-04-17T09:52:46.158 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.3] 2024-04-17T09:52:46.158 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDamx9mIwXoKxAAWPTFaYdp4lIZhuOU7F0x8g== 2024-04-17T09:52:46.158 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.x] 2024-04-17T09:52:46.158 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDPmx9myy0oDxAAPD2BTjIP09hiU5L7i6gUMg== 2024-04-17T09:52:46.158 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2024-04-17T09:52:46.159 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDPmx9mQnJ4HxAAT0rFHhdAj7PMxZGiqEyWbg== 2024-04-17T09:52:46.159 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.d] 2024-04-17T09:52:46.159 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDPmx9m9TMAJxAAbmUp05GHRE1AqCmq83scVQ== 2024-04-17T09:52:46.159 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.4] 2024-04-17T09:52:46.159 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDgmx9mp0s/JxAAv+usQqwrxgKINRlkDhhydQ== 2024-04-17T09:52:46.159 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.5] 2024-04-17T09:52:46.159 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDjmx9mMVStMhAA/n8jk+LVDosWhziXk/lLRQ== 2024-04-17T09:52:46.159 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.6] 2024-04-17T09:52:46.159 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDnmx9m/YQlABAAOqfWzVtgonD7b+WlR95mnA== 2024-04-17T09:52:46.159 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.7] 2024-04-17T09:52:46.159 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDqmx9m572mCBAAqBib/yeQOBno+0B1K1dLxA== 2024-04-17T09:52:46.159 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-17T09:52:46.159 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDPmx9mUXoLKxAANV7zhLvL0DqHngk4TtsdBQ== 2024-04-17T09:52:46.159 INFO:teuthology.orchestra.run.smithi032.stdout:[client.2] 2024-04-17T09:52:46.159 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDPmx9mWKstMRAAp18Hbnk02w+hT0sYZ3tSLA== 2024-04-17T09:52:46.159 INFO:teuthology.orchestra.run.smithi032.stdout:[client.1] 2024-04-17T09:52:46.159 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDPmx9mZBeRNBAAQg5K950mp6+XND7NCABAKQ== 2024-04-17T09:52:46.160 INFO:teuthology.orchestra.run.smithi032.stdout:[client.3] 2024-04-17T09:52:46.160 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDPmx9mRQ2wOhAAzlWXFtP0WaXWmneqxnZ5/w== 2024-04-17T09:52:46.171 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr.y] 2024-04-17T09:52:46.171 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDPmx9m3FmACxAApri5AI5PeR0+/HLev8DKcw== 2024-04-17T09:52:46.171 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.a] 2024-04-17T09:52:46.171 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDPmx9mzM1KExAAn0WH+Gf6gGV/dJonXzAgPQ== 2024-04-17T09:52:46.171 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.c] 2024-04-17T09:52:46.171 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDPmx9mWWi5GhAA3RAdRxK6GRwuVCH4DHoMeA== 2024-04-17T09:52:46.171 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.0] 2024-04-17T09:52:46.171 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDRmx9mnPwyDBAAxD10jhxptVH5zhupengElg== 2024-04-17T09:52:46.171 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.1] 2024-04-17T09:52:46.171 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDUmx9mh3iuGBAAqvWSRyNX3u/TjEH6TJpqdA== 2024-04-17T09:52:46.171 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.2] 2024-04-17T09:52:46.171 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDXmx9mbe1CIRAA1pmzAl04tGkTu6QLpmCkag== 2024-04-17T09:52:46.172 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.3] 2024-04-17T09:52:46.172 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDamx9mIwXoKxAAWPTFaYdp4lIZhuOU7F0x8g== 2024-04-17T09:52:46.172 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr.x] 2024-04-17T09:52:46.172 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDPmx9myy0oDxAAPD2BTjIP09hiU5L7i6gUMg== 2024-04-17T09:52:46.172 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.b] 2024-04-17T09:52:46.172 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDPmx9mQnJ4HxAAT0rFHhdAj7PMxZGiqEyWbg== 2024-04-17T09:52:46.172 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.d] 2024-04-17T09:52:46.172 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDPmx9m9TMAJxAAbmUp05GHRE1AqCmq83scVQ== 2024-04-17T09:52:46.172 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.4] 2024-04-17T09:52:46.172 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDgmx9mp0s/JxAAv+usQqwrxgKINRlkDhhydQ== 2024-04-17T09:52:46.172 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.5] 2024-04-17T09:52:46.172 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDjmx9mMVStMhAA/n8jk+LVDosWhziXk/lLRQ== 2024-04-17T09:52:46.172 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.6] 2024-04-17T09:52:46.172 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDnmx9m/YQlABAAOqfWzVtgonD7b+WlR95mnA== 2024-04-17T09:52:46.173 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.7] 2024-04-17T09:52:46.173 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDqmx9m572mCBAAqBib/yeQOBno+0B1K1dLxA== 2024-04-17T09:52:46.173 INFO:teuthology.orchestra.run.smithi144.stdout:[client.0] 2024-04-17T09:52:46.173 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDPmx9mUXoLKxAANV7zhLvL0DqHngk4TtsdBQ== 2024-04-17T09:52:46.173 INFO:teuthology.orchestra.run.smithi144.stdout:[client.2] 2024-04-17T09:52:46.173 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDPmx9mWKstMRAAp18Hbnk02w+hT0sYZ3tSLA== 2024-04-17T09:52:46.173 INFO:teuthology.orchestra.run.smithi144.stdout:[client.1] 2024-04-17T09:52:46.173 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDPmx9mZBeRNBAAQg5K950mp6+XND7NCABAKQ== 2024-04-17T09:52:46.173 INFO:teuthology.orchestra.run.smithi144.stdout:[client.3] 2024-04-17T09:52:46.173 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDPmx9mRQ2wOhAAzlWXFtP0WaXWmneqxnZ5/w== 2024-04-17T09:52:46.174 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-17T09:52:46.208 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-17T09:52:46.271 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T09:52:46.275 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T09:52:46.332 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T09:52:46.335 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T09:52:46.390 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T09:52:46.393 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T09:52:46.446 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T09:52:46.449 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T09:52:46.502 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T09:52:46.505 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T09:52:46.557 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T09:52:46.560 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T09:52:46.613 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-17T09:52:46.616 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-17T09:52:46.669 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T09:52:46.673 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T09:52:46.729 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T09:52:46.732 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T09:52:46.789 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T09:52:46.792 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T09:52:46.845 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T09:52:46.849 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T09:52:46.902 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T09:52:46.905 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T09:52:46.957 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T09:52:46.960 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T09:52:47.013 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T09:52:47.016 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T09:52:47.072 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T09:52:47.075 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T09:52:47.130 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T09:52:47.133 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T09:52:47.189 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T09:52:47.192 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T09:52:47.247 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-17T09:52:47.247 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-17T09:52:47.267 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-17T09:52:47.375 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-17T09:52:47.395 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-17T09:52:47.414 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-17T09:52:47.528 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-17T09:52:47.547 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-17T09:52:47.612 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-17T09:52:47.722 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-17T09:52:47.741 DEBUG:teuthology.orchestra.run.smithi032:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-17T09:52:47.744 DEBUG:teuthology.orchestra.run.smithi144:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-17T09:52:47.789 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-17T09:52:47.789 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-17T09:52:47.790 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-17T09:52:47.798 INFO:tasks.ceph.mon.a:Started 2024-04-17T09:52:47.798 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-17T09:52:47.798 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-17T09:52:47.832 INFO:tasks.ceph.mon.b:Started 2024-04-17T09:52:47.832 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-17T09:52:47.832 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-17T09:52:47.835 INFO:tasks.ceph.mon.c:Started 2024-04-17T09:52:47.835 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-17T09:52:47.835 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-17T09:52:47.836 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-17T09:52:47.839 INFO:tasks.ceph.mgr.y:Started 2024-04-17T09:52:47.839 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-17T09:52:47.839 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-17T09:52:47.842 INFO:tasks.ceph.mgr.x:Started 2024-04-17T09:52:47.842 INFO:tasks.ceph:Setting crush tunables to default 2024-04-17T09:52:47.842 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd crush tunables default 2024-04-17T09:52:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:47.980+0000 7fc7ac600640 1 Processor -- start 2024-04-17T09:52:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:47.980+0000 7fc7ac600640 1 -- start start 2024-04-17T09:52:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:47.984+0000 7fc7ac600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7a40d0650 0x7fc7a413fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:47.984+0000 7fc7ac600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7a415d070 con 0x7fc7a40af490 2024-04-17T09:52:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:47.984+0000 7fc7ac600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7a415d350 con 0x7fc7a406d150 2024-04-17T09:52:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:47.984+0000 7fc7ac600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc7a415dc80 con 0x7fc7a40d0650 2024-04-17T09:52:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:47.984+0000 7fc7a9c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7a40d0650 0x7fc7a413fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:47.984+0000 7fc7a9c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7a40d0650 0x7fc7a413fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54330/0 (socket says 172.21.15.32:54330) 2024-04-17T09:52:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:47.984+0000 7fc7a9200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc7a40af490 0x7fc7a406ca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56996/0 (socket says 172.21.15.32:56996) 2024-04-17T09:52:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:47.984+0000 7fc7a9c00640 1 -- 172.21.15.32:0/3234672159 learned_addr learned my addr 172.21.15.32:0/3234672159 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:52:47.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:47.984+0000 7fc7a9c00640 1 -- 172.21.15.32:0/3234672159 >> v2:172.21.15.32:3300/0 conn(0x7fc7a40d0650 msgr2=0x7fc7a413fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-17T09:52:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:47.984+0000 7fc7a9c00640 1 -- 172.21.15.32:0/3234672159 >> v2:172.21.15.32:3300/0 conn(0x7fc7a40d0650 msgr2=0x7fc7a413fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T09:52:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:47.984+0000 7fc7a9c00640 1 --2- 172.21.15.32:0/3234672159 >> v2:172.21.15.32:3300/0 conn(0x7fc7a40d0650 0x7fc7a413fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T09:52:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:47.984+0000 7fc7a9c00640 1 --2- 172.21.15.32:0/3234672159 >> v2:172.21.15.32:3300/0 conn(0x7fc7a40d0650 0x7fc7a413fd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T09:52:48.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7a3e00640 1 -- 172.21.15.32:0/3234672159 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 893385800 0 0) 0x7fc7a415d070 con 0x7fc7a40af490 2024-04-17T09:52:48.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7a3e00640 1 -- 172.21.15.32:0/3234672159 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc78c003530 con 0x7fc7a40af490 2024-04-17T09:52:48.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7a3e00640 1 -- 172.21.15.32:0/3234672159 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2735724775 0 0) 0x7fc78c003530 con 0x7fc7a40af490 2024-04-17T09:52:48.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7a3e00640 1 -- 172.21.15.32:0/3234672159 >> v2:172.21.15.32:3300/0 conn(0x7fc7a40d0650 msgr2=0x7fc7a413fd80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T09:52:48.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7a3e00640 1 --2- 172.21.15.32:0/3234672159 >> v2:172.21.15.32:3300/0 conn(0x7fc7a40d0650 0x7fc7a413fd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:48.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7a3e00640 1 -- 172.21.15.32:0/3234672159 >> v1:172.21.15.144:6789/0 conn(0x7fc7a406d150 legacy=0x7fc7a4160560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:48.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7a3e00640 1 -- 172.21.15.32:0/3234672159 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a415d630 con 0x7fc7a40af490 2024-04-17T09:52:48.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7a3e00640 1 -- 172.21.15.32:0/3234672159 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7fc790001680 con 0x7fc7a40af490 2024-04-17T09:52:48.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7a3e00640 1 -- 172.21.15.32:0/3234672159 >> v1:172.21.15.32:6789/0 conn(0x7fc7a40af490 legacy=0x7fc7a406ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:48.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7a3e00640 1 --2- 172.21.15.32:0/3234672159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc78c0039c0 0x7fc78c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:48.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7a3e00640 1 --2- 172.21.15.32:0/3234672159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc78c004810 0x7fc78c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7a3e00640 1 --2- 172.21.15.32:0/3234672159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc78c0051d0 0x7fc78c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7a3e00640 1 -- 172.21.15.32:0/3234672159 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a415dc80 con 0x7fc78c0051d0 2024-04-17T09:52:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7a3e00640 1 -- 172.21.15.32:0/3234672159 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c00c120 con 0x7fc78c004810 2024-04-17T09:52:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7a3e00640 1 -- 172.21.15.32:0/3234672159 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc78c00c320 con 0x7fc78c0039c0 2024-04-17T09:52:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7aa600640 1 --2- 172.21.15.32:0/3234672159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc78c0051d0 0x7fc78c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7a9200640 1 --2- 172.21.15.32:0/3234672159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc78c004810 0x7fc78c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7a9c00640 1 --2- 172.21.15.32:0/3234672159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc78c0039c0 0x7fc78c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7aa600640 1 -- 172.21.15.32:0/3234672159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc78c0039c0 msgr2=0x7fc78c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:52:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7aa600640 1 --2- 172.21.15.32:0/3234672159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc78c0039c0 0x7fc78c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7aa600640 1 -- 172.21.15.32:0/3234672159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc78c004810 msgr2=0x7fc78c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:52:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7aa600640 1 --2- 172.21.15.32:0/3234672159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc78c004810 0x7fc78c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7aa600640 1 -- 172.21.15.32:0/3234672159 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78c003530 con 0x7fc78c0051d0 2024-04-17T09:52:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.132+0000 7fc7a9200640 1 --2- 172.21.15.32:0/3234672159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc78c004810 0x7fc78c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:52:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.136+0000 7fc7aa600640 1 -- 172.21.15.32:0/3234672159 reap_dead start 2024-04-17T09:52:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.136+0000 7fc7aa600640 1 --2- 172.21.15.32:0/3234672159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc78c0051d0 0x7fc78c00b9d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc79c001e40 tx=0x7fc79c01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=664d08377d7a77b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.136+0000 7fc7a3e00640 1 -- 172.21.15.32:0/3234672159 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c001a90 con 0x7fc78c0051d0 2024-04-17T09:52:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.136+0000 7fc7a3e00640 1 -- 172.21.15.32:0/3234672159 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc79c003b80 con 0x7fc78c0051d0 2024-04-17T09:52:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.136+0000 7fc7a3e00640 1 -- 172.21.15.32:0/3234672159 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c005020 con 0x7fc78c0051d0 2024-04-17T09:52:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.136+0000 7fc7ac600640 1 -- 172.21.15.32:0/3234672159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc78c0051d0 msgr2=0x7fc78c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.136+0000 7fc7ac600640 1 --2- 172.21.15.32:0/3234672159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc78c0051d0 0x7fc78c00b9d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc79c001e40 tx=0x7fc79c01c760 comp rx=0 tx=0).stop 2024-04-17T09:52:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.136+0000 7fc7ac600640 1 -- 172.21.15.32:0/3234672159 shutdown_connections 2024-04-17T09:52:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.136+0000 7fc7ac600640 1 --2- 172.21.15.32:0/3234672159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc78c0051d0 0x7fc78c00b9d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.136+0000 7fc7ac600640 1 -- 172.21.15.32:0/3234672159 >> 172.21.15.32:0/3234672159 conn(0x7fc7a4072670 msgr2=0x7fc7a4072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.136+0000 7fc7ac600640 1 -- 172.21.15.32:0/3234672159 shutdown_connections 2024-04-17T09:52:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.136+0000 7fc7ac600640 1 -- 172.21.15.32:0/3234672159 wait complete. 2024-04-17T09:52:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.140+0000 7fc7ac600640 1 Processor -- start 2024-04-17T09:52:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.140+0000 7fc7ac600640 1 -- start start 2024-04-17T09:52:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.140+0000 7fc7ac600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc78c0051d0 0x7fc7a40b2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.140+0000 7fc7ac600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7a413cfc0 0x7fc7a40b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.140+0000 7fc7ac600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7a415ed30 0x7fc7a414a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.140+0000 7fc7ac600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a40aefa0 con 0x7fc7a415ed30 2024-04-17T09:52:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.140+0000 7fc7ac600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a40d0a30 con 0x7fc78c0051d0 2024-04-17T09:52:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.140+0000 7fc7ac600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a40af120 con 0x7fc7a413cfc0 2024-04-17T09:52:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.140+0000 7fc7a9c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc78c0051d0 0x7fc7a40b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.140+0000 7fc7aa600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7a415ed30 0x7fc7a414a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.140+0000 7fc7a9200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7a413cfc0 0x7fc7a40b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.140+0000 7fc7a9c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc78c0051d0 0x7fc7a40b2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:32912/0 (socket says 172.21.15.32:32912) 2024-04-17T09:52:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.140+0000 7fc7a9c00640 1 -- 172.21.15.32:0/1877067056 learned_addr learned my addr 172.21.15.32:0/1877067056 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:52:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.140+0000 7fc7aa600640 1 -- 172.21.15.32:0/1877067056 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7a413cfc0 msgr2=0x7fc7a40b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.140+0000 7fc7aa600640 1 --2- 172.21.15.32:0/1877067056 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7a413cfc0 0x7fc7a40b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.140+0000 7fc7aa600640 1 -- 172.21.15.32:0/1877067056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc78c0051d0 msgr2=0x7fc7a40b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.140+0000 7fc7aa600640 1 --2- 172.21.15.32:0/1877067056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc78c0051d0 0x7fc7a40b2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.140+0000 7fc7aa600640 1 -- 172.21.15.32:0/1877067056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a40cdea0 con 0x7fc7a415ed30 2024-04-17T09:52:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.144+0000 7fc7aa600640 1 --2- 172.21.15.32:0/1877067056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7a415ed30 0x7fc7a414a300 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40b0140 tx=0x7fc78c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.144+0000 7fc7a1600640 1 -- 172.21.15.32:0/1877067056 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c007dc0 con 0x7fc7a415ed30 2024-04-17T09:52:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.144+0000 7fc7a1600640 1 -- 172.21.15.32:0/1877067056 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc79c0ad070 con 0x7fc7a415ed30 2024-04-17T09:52:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.144+0000 7fc7a1600640 1 -- 172.21.15.32:0/1877067056 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c006d80 con 0x7fc7a415ed30 2024-04-17T09:52:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.144+0000 7fc7ac600640 1 -- 172.21.15.32:0/1877067056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a40ce180 con 0x7fc7a415ed30 2024-04-17T09:52:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.144+0000 7fc7ac600640 1 -- 172.21.15.32:0/1877067056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a40ce740 con 0x7fc7a415ed30 2024-04-17T09:52:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.144+0000 7fc7a1600640 1 -- 172.21.15.32:0/1877067056 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc79c0b8060 con 0x7fc7a415ed30 2024-04-17T09:52:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.144+0000 7fc7a1600640 1 -- 172.21.15.32:0/1877067056 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc79c0a68e0 con 0x7fc7a415ed30 2024-04-17T09:52:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.144+0000 7fc7ac600640 1 -- 172.21.15.32:0/1877067056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a4068a90 con 0x7fc7a415ed30 2024-04-17T09:52:48.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.156+0000 7fc7a1600640 1 -- 172.21.15.32:0/1877067056 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fc79c016470 con 0x7fc7a415ed30 2024-04-17T09:52:48.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.220+0000 7fc7ac600640 1 -- 172.21.15.32:0/1877067056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fc7a4156590 con 0x7fc7a415ed30 2024-04-17T09:52:48.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.224+0000 7fc7a1600640 1 -- 172.21.15.32:0/1877067056 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fc79c0a6b30 con 0x7fc7a415ed30 2024-04-17T09:52:48.225 INFO:teuthology.orchestra.run.smithi032.stderr:adjusted tunables profile to default 2024-04-17T09:52:48.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.224+0000 7fc7ac600640 1 -- 172.21.15.32:0/1877067056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7a415ed30 msgr2=0x7fc7a414a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:48.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.224+0000 7fc7ac600640 1 --2- 172.21.15.32:0/1877067056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7a415ed30 0x7fc7a414a300 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40b0140 tx=0x7fc78c005840 comp rx=0 tx=0).stop 2024-04-17T09:52:48.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.224+0000 7fc7ac600640 1 -- 172.21.15.32:0/1877067056 shutdown_connections 2024-04-17T09:52:48.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.224+0000 7fc7ac600640 1 --2- 172.21.15.32:0/1877067056 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7a413cfc0 0x7fc7a40b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:48.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.224+0000 7fc7ac600640 1 --2- 172.21.15.32:0/1877067056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7a415ed30 0x7fc7a414a300 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:48.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.224+0000 7fc7ac600640 1 --2- 172.21.15.32:0/1877067056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc78c0051d0 0x7fc7a40b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:48.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.224+0000 7fc7ac600640 1 -- 172.21.15.32:0/1877067056 >> 172.21.15.32:0/1877067056 conn(0x7fc7a4072670 msgr2=0x7fc7a4156ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:48.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.224+0000 7fc7ac600640 1 -- 172.21.15.32:0/1877067056 shutdown_connections 2024-04-17T09:52:48.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:48.224+0000 7fc7ac600640 1 -- 172.21.15.32:0/1877067056 wait complete. 2024-04-17T09:52:48.244 INFO:tasks.ceph:check_enable_crimson: False 2024-04-17T09:52:48.244 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-17T09:52:48.244 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:48.245 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-17T09:52:48.264 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:48.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-17T09:52:48.327 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:48.327 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-17T09:52:48.352 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-17T09:52:48.349+0000 7f32a8109180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T09:52:48.356 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:48.356+0000 7f63c8272180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T09:52:48.375 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:52:48.375 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-17T09:52:48.439 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:52:48.439 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-17T09:52:48.458 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:52:48.459 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-17T09:52:48.521 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:52:48.521 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-17T09:52:48.585 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:52:48.585 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-17T09:52:48.644 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 09a0453d-54e1-4672-89dc-466a9dc6a049 0 2024-04-17T09:52:48.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.817+0000 7f428c400640 1 Processor -- start 2024-04-17T09:52:48.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.817+0000 7f428c400640 1 -- start start 2024-04-17T09:52:48.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.817+0000 7f428c400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f428415c890 0x7f42840d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:48.822 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.817+0000 7f428c400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f428409b100 0x7f428415f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:48.822 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.817+0000 7f428c400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42840d0b60 con 0x7f428415dcd0 2024-04-17T09:52:48.822 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.817+0000 7f428c400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f42840d16e0 con 0x7f428415c890 2024-04-17T09:52:48.822 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.817+0000 7f428c400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f42840cd1a0 con 0x7f428409b100 2024-04-17T09:52:48.822 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.817+0000 7f4289000640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f428415dcd0 0x7f428409a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52652/0 (socket says 172.21.15.144:52652) 2024-04-17T09:52:48.822 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.817+0000 7f4289000640 1 -- 172.21.15.144:0/3862465352 learned_addr learned my addr 172.21.15.144:0/3862465352 (peer_addr_for_me v1:172.21.15.144:0/0) 2024-04-17T09:52:48.822 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.817+0000 7f428a400640 1 --2- 172.21.15.144:0/3862465352 >> v2:172.21.15.144:3300/0 conn(0x7f428409b100 0x7f428415f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:48.822 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.817+0000 7f4289a00640 1 --2- 172.21.15.144:0/3862465352 >> v2:172.21.15.32:3300/0 conn(0x7f428415c890 0x7f42840d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:48.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.817+0000 7f4289a00640 1 -- 172.21.15.144:0/3862465352 >> v2:172.21.15.144:3300/0 conn(0x7f428409b100 msgr2=0x7f428415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:52:48.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.817+0000 7f4289a00640 1 --2- 172.21.15.144:0/3862465352 >> v2:172.21.15.144:3300/0 conn(0x7f428409b100 0x7f428415f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:48.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.817+0000 7f4289a00640 1 -- 172.21.15.144:0/3862465352 >> v1:172.21.15.144:6789/0 conn(0x7f428415dcd0 legacy=0x7f428409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:48.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.817+0000 7f4289a00640 1 -- 172.21.15.144:0/3862465352 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42840d0e40 con 0x7f428415c890 2024-04-17T09:52:48.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.821+0000 7f4289a00640 1 --2- 172.21.15.144:0/3862465352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f428415c890 0x7f42840d01f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f42840b1ec0 tx=0x7f427801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a926a4d7ab88a77d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:48.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.821+0000 7f4283e00640 1 -- 172.21.15.144:0/3862465352 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42780ab650 con 0x7f428415c890 2024-04-17T09:52:48.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.821+0000 7f4283e00640 1 -- 172.21.15.144:0/3862465352 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42780b0070 con 0x7f428415c890 2024-04-17T09:52:48.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.821+0000 7f4283e00640 1 -- 172.21.15.144:0/3862465352 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42780aba10 con 0x7f428415c890 2024-04-17T09:52:48.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.821+0000 7f428c400640 1 -- 172.21.15.144:0/3862465352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f428415c890 msgr2=0x7f42840d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:48.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.821+0000 7f428c400640 1 --2- 172.21.15.144:0/3862465352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f428415c890 0x7f42840d01f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f42840b1ec0 tx=0x7f427801c450 comp rx=0 tx=0).stop 2024-04-17T09:52:48.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.821+0000 7f428c400640 1 -- 172.21.15.144:0/3862465352 shutdown_connections 2024-04-17T09:52:48.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.821+0000 7f428c400640 1 --2- 172.21.15.144:0/3862465352 >> v2:172.21.15.144:3300/0 conn(0x7f428409b100 0x7f428415f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:48.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.821+0000 7f428c400640 1 --2- 172.21.15.144:0/3862465352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f428415c890 0x7f42840d01f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:48.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.821+0000 7f428c400640 1 -- 172.21.15.144:0/3862465352 >> 172.21.15.144:0/3862465352 conn(0x7f42840b4110 msgr2=0x7f42840b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:48.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.825+0000 7f428c400640 1 -- 172.21.15.144:0/3862465352 shutdown_connections 2024-04-17T09:52:48.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.825+0000 7f428c400640 1 -- 172.21.15.144:0/3862465352 wait complete. 2024-04-17T09:52:48.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.825+0000 7f428c400640 1 Processor -- start 2024-04-17T09:52:48.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.825+0000 7f428c400640 1 -- start start 2024-04-17T09:52:48.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.825+0000 7f428c400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f428409b100 0x7f42840c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:48.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.825+0000 7f428c400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f428415c890 0x7f42840b6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:48.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.825+0000 7f428c400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f428415dcd0 0x7f42840ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:48.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.825+0000 7f428c400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f428415cdf0 con 0x7f428415dcd0 2024-04-17T09:52:48.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.825+0000 7f428c400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f428415cc70 con 0x7f428415c890 2024-04-17T09:52:48.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.825+0000 7f428c400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f42840d0730 con 0x7f428409b100 2024-04-17T09:52:48.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.825+0000 7f4289a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f428409b100 0x7f42840c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:48.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.825+0000 7f4289a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f428409b100 0x7f42840c96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:60494/0 (socket says 172.21.15.144:60494) 2024-04-17T09:52:48.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.825+0000 7f4289000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f428415c890 0x7f42840b6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:48.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.825+0000 7f4289a00640 1 -- 172.21.15.144:0/2692733586 learned_addr learned my addr 172.21.15.144:0/2692733586 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-17T09:52:48.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.825+0000 7f428a400640 1 --2- 172.21.15.144:0/2692733586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f428415dcd0 0x7f42840ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:48.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.825+0000 7f4289a00640 1 -- 172.21.15.144:0/2692733586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f428415c890 msgr2=0x7f42840b6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:48.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.825+0000 7f4289a00640 1 --2- 172.21.15.144:0/2692733586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f428415c890 0x7f42840b6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:48.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.825+0000 7f4289a00640 1 -- 172.21.15.144:0/2692733586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f428415dcd0 msgr2=0x7f42840ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:48.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.825+0000 7f4289a00640 1 --2- 172.21.15.144:0/2692733586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f428415dcd0 0x7f42840ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:48.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.825+0000 7f4289a00640 1 -- 172.21.15.144:0/2692733586 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42840cf030 con 0x7f428409b100 2024-04-17T09:52:48.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.829+0000 7f4289a00640 1 --2- 172.21.15.144:0/2692733586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f428409b100 0x7f42840c96f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4278019060 tx=0x7f427801ac90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:48.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.829+0000 7f4281600640 1 -- 172.21.15.144:0/2692733586 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42780bf610 con 0x7f428409b100 2024-04-17T09:52:48.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.829+0000 7f4281600640 1 -- 172.21.15.144:0/2692733586 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42841602e0 con 0x7f428409b100 2024-04-17T09:52:48.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.829+0000 7f428c400640 1 -- 172.21.15.144:0/2692733586 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42840b6ee0 con 0x7f428409b100 2024-04-17T09:52:48.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.829+0000 7f4281600640 1 -- 172.21.15.144:0/2692733586 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42780bfa10 con 0x7f428409b100 2024-04-17T09:52:48.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.829+0000 7f428c400640 1 -- 172.21.15.144:0/2692733586 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f42841fb950 con 0x7f428409b100 2024-04-17T09:52:48.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.829+0000 7f4281600640 1 -- 172.21.15.144:0/2692733586 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f42780bd020 con 0x7f428409b100 2024-04-17T09:52:48.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.829+0000 7f4281600640 1 -- 172.21.15.144:0/2692733586 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f42780b5d60 con 0x7f428409b100 2024-04-17T09:52:48.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.829+0000 7f428c400640 1 -- 172.21.15.144:0/2692733586 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4284068a90 con 0x7f428409b100 2024-04-17T09:52:48.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.833+0000 7f4281600640 1 -- 172.21.15.144:0/2692733586 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f4278016070 con 0x7f428409b100 2024-04-17T09:52:48.863 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-17T09:52:48.861+0000 7f32a8109180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T09:52:48.873 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:48.872+0000 7f63c8272180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T09:52:48.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.897+0000 7f428c400640 1 -- 172.21.15.144:0/2692733586 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "09a0453d-54e1-4672-89dc-466a9dc6a049", "id": 0} v 0) -- 0x7f428409b520 con 0x7f428409b100 2024-04-17T09:52:48.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.905+0000 7f4281600640 1 -- 172.21.15.144:0/2692733586 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "09a0453d-54e1-4672-89dc-466a9dc6a049", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f42780059a0 con 0x7f428409b100 2024-04-17T09:52:48.906 INFO:teuthology.orchestra.run.smithi144.stdout:0 2024-04-17T09:52:48.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.905+0000 7f428c400640 1 -- 172.21.15.144:0/2692733586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f428409b100 msgr2=0x7f42840c96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:48.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.905+0000 7f428c400640 1 --2- 172.21.15.144:0/2692733586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f428409b100 0x7f42840c96f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4278019060 tx=0x7f427801ac90 comp rx=0 tx=0).stop 2024-04-17T09:52:48.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.905+0000 7f428c400640 1 -- 172.21.15.144:0/2692733586 shutdown_connections 2024-04-17T09:52:48.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.905+0000 7f428c400640 1 --2- 172.21.15.144:0/2692733586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f428415dcd0 0x7f42840ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:48.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.905+0000 7f428c400640 1 --2- 172.21.15.144:0/2692733586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f428415c890 0x7f42840b6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:48.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.905+0000 7f428c400640 1 --2- 172.21.15.144:0/2692733586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f428409b100 0x7f42840c96f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:48.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.905+0000 7f428c400640 1 -- 172.21.15.144:0/2692733586 >> 172.21.15.144:0/2692733586 conn(0x7f42840b4110 msgr2=0x7f42840b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:48.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.905+0000 7f428c400640 1 -- 172.21.15.144:0/2692733586 shutdown_connections 2024-04-17T09:52:48.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:48.905+0000 7f428c400640 1 -- 172.21.15.144:0/2692733586 wait complete. 2024-04-17T09:52:48.931 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 0fb07290-ef1f-4ab6-99f8-12e4b399579f 1 2024-04-17T09:52:49.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.065+0000 7fd366200640 1 Processor -- start 2024-04-17T09:52:49.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.065+0000 7fd366200640 1 -- start start 2024-04-17T09:52:49.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.065+0000 7fd366200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd360161ec0 0x7fd3601626d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:49.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.065+0000 7fd366200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd36006cdc0 0x7fd36009a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:49.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.065+0000 7fd366200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd3601604b0 0x7fd36009af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:49.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.065+0000 7fd366200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd3600d0b30 con 0x7fd36006cdc0 2024-04-17T09:52:49.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.065+0000 7fd366200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd360162fb0 con 0x7fd3601604b0 2024-04-17T09:52:49.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.065+0000 7fd366200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd360163290 con 0x7fd360161ec0 2024-04-17T09:52:49.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.065+0000 7fd35fe00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd3601604b0 0x7fd36009af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:49.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.065+0000 7fd35f400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd360161ec0 0x7fd3601626d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:49.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.065+0000 7fd35fe00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd3601604b0 0x7fd36009af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41236/0 (socket says 172.21.15.144:41236) 2024-04-17T09:52:49.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.065+0000 7fd35f400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd360161ec0 0x7fd3601626d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:60506/0 (socket says 172.21.15.144:60506) 2024-04-17T09:52:49.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.065+0000 7fd35fe00640 1 -- 172.21.15.144:0/253134915 learned_addr learned my addr 172.21.15.144:0/253134915 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-17T09:52:49.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.065+0000 7fd35ea00640 1 --2- 172.21.15.144:0/253134915 >> v2:172.21.15.32:3300/0 conn(0x7fd36006cdc0 0x7fd36009a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:49.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.065+0000 7fd35fe00640 1 -- 172.21.15.144:0/253134915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd360161ec0 msgr2=0x7fd3601626d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:52:49.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.065+0000 7fd35fe00640 1 --2- 172.21.15.144:0/253134915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd360161ec0 0x7fd3601626d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.065+0000 7fd35fe00640 1 -- 172.21.15.144:0/253134915 >> v2:172.21.15.32:3300/0 conn(0x7fd36006cdc0 msgr2=0x7fd36009a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:52:49.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.065+0000 7fd35fe00640 1 --2- 172.21.15.144:0/253134915 >> v2:172.21.15.32:3300/0 conn(0x7fd36006cdc0 0x7fd36009a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.065+0000 7fd35fe00640 1 -- 172.21.15.144:0/253134915 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3600d0da0 con 0x7fd3601604b0 2024-04-17T09:52:49.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.069+0000 7fd35fe00640 1 --2- 172.21.15.144:0/253134915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3601604b0 0x7fd36009af30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd3600758d0 tx=0x7fd35801c9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d648ee1442ecbbb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:49.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.069+0000 7fd35e000640 1 -- 172.21.15.144:0/253134915 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3580a57a0 con 0x7fd3601604b0 2024-04-17T09:52:49.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.069+0000 7fd35e000640 1 -- 172.21.15.144:0/253134915 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3580a5940 con 0x7fd3601604b0 2024-04-17T09:52:49.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.069+0000 7fd35e000640 1 -- 172.21.15.144:0/253134915 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3580a5d10 con 0x7fd3601604b0 2024-04-17T09:52:49.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.069+0000 7fd366200640 1 -- 172.21.15.144:0/253134915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3601604b0 msgr2=0x7fd36009af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:49.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.069+0000 7fd366200640 1 --2- 172.21.15.144:0/253134915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3601604b0 0x7fd36009af30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd3600758d0 tx=0x7fd35801c9f0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.069+0000 7fd366200640 1 -- 172.21.15.144:0/253134915 shutdown_connections 2024-04-17T09:52:49.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.069+0000 7fd366200640 1 --2- 172.21.15.144:0/253134915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3601604b0 0x7fd36009af30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.074 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.069+0000 7fd366200640 1 --2- 172.21.15.144:0/253134915 >> v2:172.21.15.32:3300/0 conn(0x7fd36006cdc0 0x7fd36009a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.074 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.069+0000 7fd366200640 1 --2- 172.21.15.144:0/253134915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd360161ec0 0x7fd3601626d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.074 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.069+0000 7fd366200640 1 -- 172.21.15.144:0/253134915 >> 172.21.15.144:0/253134915 conn(0x7fd3600b4110 msgr2=0x7fd3600b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:49.074 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.069+0000 7fd366200640 1 -- 172.21.15.144:0/253134915 shutdown_connections 2024-04-17T09:52:49.074 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.069+0000 7fd366200640 1 -- 172.21.15.144:0/253134915 wait complete. 2024-04-17T09:52:49.074 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd366200640 1 Processor -- start 2024-04-17T09:52:49.074 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd366200640 1 -- start start 2024-04-17T09:52:49.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd366200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd36006cdc0 0x7fd360152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:49.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd366200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3601604b0 0x7fd3600b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:49.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd366200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd360161ec0 0x7fd3600b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:49.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd366200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd3600d1c50 con 0x7fd3601604b0 2024-04-17T09:52:49.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd366200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd3600d1ad0 con 0x7fd36006cdc0 2024-04-17T09:52:49.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd366200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd3600d1dd0 con 0x7fd360161ec0 2024-04-17T09:52:49.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd35f400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd36006cdc0 0x7fd360152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:49.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd35f400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd36006cdc0 0x7fd360152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41252/0 (socket says 172.21.15.144:41252) 2024-04-17T09:52:49.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd35f400640 1 -- 172.21.15.144:0/444399097 learned_addr learned my addr 172.21.15.144:0/444399097 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-17T09:52:49.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd35fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd360161ec0 0x7fd3600b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:49.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd35ea00640 1 --2- 172.21.15.144:0/444399097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3601604b0 0x7fd3600b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:49.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd35f400640 1 -- 172.21.15.144:0/444399097 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd360161ec0 msgr2=0x7fd3600b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:49.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd35f400640 1 --2- 172.21.15.144:0/444399097 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd360161ec0 0x7fd3600b7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd35f400640 1 -- 172.21.15.144:0/444399097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3601604b0 msgr2=0x7fd3600b5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:49.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd35f400640 1 --2- 172.21.15.144:0/444399097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3601604b0 0x7fd3600b5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd35f400640 1 -- 172.21.15.144:0/444399097 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3600ca4d0 con 0x7fd36006cdc0 2024-04-17T09:52:49.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd35f400640 1 --2- 172.21.15.144:0/444399097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd36006cdc0 0x7fd360152f90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd35000e8a0 tx=0x7fd35000eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:49.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd357400640 1 -- 172.21.15.144:0/444399097 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350017070 con 0x7fd36006cdc0 2024-04-17T09:52:49.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd357400640 1 -- 172.21.15.144:0/444399097 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd35000c940 con 0x7fd36006cdc0 2024-04-17T09:52:49.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd357400640 1 -- 172.21.15.144:0/444399097 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35000ccf0 con 0x7fd36006cdc0 2024-04-17T09:52:49.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd366200640 1 -- 172.21.15.144:0/444399097 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3600ca7b0 con 0x7fd36006cdc0 2024-04-17T09:52:49.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd366200640 1 -- 172.21.15.144:0/444399097 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3600cad70 con 0x7fd36006cdc0 2024-04-17T09:52:49.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd357400640 1 -- 172.21.15.144:0/444399097 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd35001a070 con 0x7fd36006cdc0 2024-04-17T09:52:49.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd357400640 1 -- 172.21.15.144:0/444399097 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fd35001c840 con 0x7fd36006cdc0 2024-04-17T09:52:49.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.073+0000 7fd366200640 1 -- 172.21.15.144:0/444399097 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd360068a90 con 0x7fd36006cdc0 2024-04-17T09:52:49.093 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:49.092+0000 7f63c8272180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T09:52:49.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.113+0000 7fd357400640 1 -- 172.21.15.144:0/444399097 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fd35001ca40 con 0x7fd36006cdc0 2024-04-17T09:52:49.162 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-17T09:52:49.157+0000 7f32a8109180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T09:52:49.185 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.181+0000 7fd366200640 1 -- 172.21.15.144:0/444399097 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0fb07290-ef1f-4ab6-99f8-12e4b399579f", "id": 1} v 0) -- 0x7fd36009b520 con 0x7fd36006cdc0 2024-04-17T09:52:49.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.185+0000 7fd357400640 1 -- 172.21.15.144:0/444399097 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0fb07290-ef1f-4ab6-99f8-12e4b399579f", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fd35001cc20 con 0x7fd36006cdc0 2024-04-17T09:52:49.190 INFO:teuthology.orchestra.run.smithi144.stdout:1 2024-04-17T09:52:49.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.189+0000 7fd366200640 1 -- 172.21.15.144:0/444399097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd36006cdc0 msgr2=0x7fd360152f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:49.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.189+0000 7fd366200640 1 --2- 172.21.15.144:0/444399097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd36006cdc0 0x7fd360152f90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd35000e8a0 tx=0x7fd35000eca0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.189+0000 7fd366200640 1 -- 172.21.15.144:0/444399097 shutdown_connections 2024-04-17T09:52:49.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.189+0000 7fd366200640 1 --2- 172.21.15.144:0/444399097 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd360161ec0 0x7fd3600b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.189+0000 7fd366200640 1 --2- 172.21.15.144:0/444399097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3601604b0 0x7fd3600b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.189+0000 7fd366200640 1 --2- 172.21.15.144:0/444399097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd36006cdc0 0x7fd360152f90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.189+0000 7fd366200640 1 -- 172.21.15.144:0/444399097 >> 172.21.15.144:0/444399097 conn(0x7fd3600b4110 msgr2=0x7fd3600cf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:49.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.189+0000 7fd366200640 1 -- 172.21.15.144:0/444399097 shutdown_connections 2024-04-17T09:52:49.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.189+0000 7fd366200640 1 -- 172.21.15.144:0/444399097 wait complete. 2024-04-17T09:52:49.209 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 3e5c6a47-f005-4854-b351-a83b0bf1525a 2 2024-04-17T09:52:49.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.341+0000 7fec5de00640 1 Processor -- start 2024-04-17T09:52:49.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.341+0000 7fec5de00640 1 -- start start 2024-04-17T09:52:49.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.341+0000 7fec5de00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fec5815a410 0x7fec580cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:49.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.341+0000 7fec5de00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec5806c6f0 0x7fec580b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:49.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.341+0000 7fec5de00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec581603c0 con 0x7fec580cd6d0 2024-04-17T09:52:49.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.341+0000 7fec5de00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fec581606a0 con 0x7fec5815a410 2024-04-17T09:52:49.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.341+0000 7fec5de00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fec58160980 con 0x7fec5806c6f0 2024-04-17T09:52:49.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.341+0000 7fec56a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fec5815a410 0x7fec580cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:49.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.341+0000 7fec4fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec5806c6f0 0x7fec580b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:49.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.341+0000 7fec56a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fec5815a410 0x7fec580cd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41262/0 (socket says 172.21.15.144:41262) 2024-04-17T09:52:49.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.341+0000 7fec56a00640 1 -- 172.21.15.144:0/2311566657 learned_addr learned my addr 172.21.15.144:0/2311566657 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-17T09:52:49.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.341+0000 7fec56a00640 1 -- 172.21.15.144:0/2311566657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec5806c6f0 msgr2=0x7fec580b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:52:49.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.341+0000 7fec56a00640 1 --2- 172.21.15.144:0/2311566657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec5806c6f0 0x7fec580b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.341+0000 7fec56a00640 1 -- 172.21.15.144:0/2311566657 >> v1:172.21.15.32:6789/0 conn(0x7fec580cd6d0 legacy=0x7fec5815ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:49.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.341+0000 7fec56a00640 1 -- 172.21.15.144:0/2311566657 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec58160c60 con 0x7fec5815a410 2024-04-17T09:52:49.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.345+0000 7fec56a00640 1 --2- 172.21.15.144:0/2311566657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec5815a410 0x7fec580cd190 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fec48009870 tx=0x7fec4801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff11d78f5c7a95c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:49.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.345+0000 7fec56000640 1 -- 172.21.15.144:0/2311566657 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec480a7070 con 0x7fec5815a410 2024-04-17T09:52:49.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.345+0000 7fec56000640 1 -- 172.21.15.144:0/2311566657 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec480b0070 con 0x7fec5815a410 2024-04-17T09:52:49.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.345+0000 7fec56000640 1 -- 172.21.15.144:0/2311566657 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec480ab890 con 0x7fec5815a410 2024-04-17T09:52:49.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.345+0000 7fec5de00640 1 -- 172.21.15.144:0/2311566657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec5815a410 msgr2=0x7fec580cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:49.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.345+0000 7fec5de00640 1 --2- 172.21.15.144:0/2311566657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec5815a410 0x7fec580cd190 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fec48009870 tx=0x7fec4801c560 comp rx=0 tx=0).stop 2024-04-17T09:52:49.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.345+0000 7fec5de00640 1 -- 172.21.15.144:0/2311566657 shutdown_connections 2024-04-17T09:52:49.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.345+0000 7fec5de00640 1 --2- 172.21.15.144:0/2311566657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec5806c6f0 0x7fec580b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.345+0000 7fec5de00640 1 --2- 172.21.15.144:0/2311566657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec5815a410 0x7fec580cd190 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.345+0000 7fec5de00640 1 -- 172.21.15.144:0/2311566657 >> 172.21.15.144:0/2311566657 conn(0x7fec580b3000 msgr2=0x7fec580b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:49.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.345+0000 7fec5de00640 1 -- 172.21.15.144:0/2311566657 shutdown_connections 2024-04-17T09:52:49.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.345+0000 7fec5de00640 1 -- 172.21.15.144:0/2311566657 wait complete. 2024-04-17T09:52:49.350 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec5de00640 1 Processor -- start 2024-04-17T09:52:49.350 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec5de00640 1 -- start start 2024-04-17T09:52:49.352 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec5de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec5806c6f0 0x7fec581f9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:49.352 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec5de00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec580cd6d0 0x7fec5815df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:49.352 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec5de00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec5815a410 0x7fec5806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:49.352 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec5de00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fec58169840 con 0x7fec5806c6f0 2024-04-17T09:52:49.352 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec5de00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec581696c0 con 0x7fec5815a410 2024-04-17T09:52:49.352 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec5de00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fec581699c0 con 0x7fec580cd6d0 2024-04-17T09:52:49.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec4fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec580cd6d0 0x7fec5815df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:49.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec4fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec580cd6d0 0x7fec5815df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:60528/0 (socket says 172.21.15.144:60528) 2024-04-17T09:52:49.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec4fe00640 1 -- 172.21.15.144:0/2895975344 learned_addr learned my addr 172.21.15.144:0/2895975344 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-17T09:52:49.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec57400640 1 --2- 172.21.15.144:0/2895975344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec5815a410 0x7fec5806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:49.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec56a00640 1 --2- 172.21.15.144:0/2895975344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec5806c6f0 0x7fec581f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:49.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec4fe00640 1 -- 172.21.15.144:0/2895975344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec5815a410 msgr2=0x7fec5806fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:49.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec4fe00640 1 --2- 172.21.15.144:0/2895975344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec5815a410 0x7fec5806fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec4fe00640 1 -- 172.21.15.144:0/2895975344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec5806c6f0 msgr2=0x7fec581f9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:49.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec4fe00640 1 --2- 172.21.15.144:0/2895975344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec5806c6f0 0x7fec581f9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec4fe00640 1 -- 172.21.15.144:0/2895975344 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec581548c0 con 0x7fec580cd6d0 2024-04-17T09:52:49.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec4fe00640 1 --2- 172.21.15.144:0/2895975344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec580cd6d0 0x7fec5815df60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fec4000e930 tx=0x7fec4000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:49.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec4f400640 1 -- 172.21.15.144:0/2895975344 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec40017070 con 0x7fec580cd6d0 2024-04-17T09:52:49.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec4f400640 1 -- 172.21.15.144:0/2895975344 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec4000ca00 con 0x7fec580cd6d0 2024-04-17T09:52:49.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec4f400640 1 -- 172.21.15.144:0/2895975344 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4000cdf0 con 0x7fec580cd6d0 2024-04-17T09:52:49.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec5de00640 1 -- 172.21.15.144:0/2895975344 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec5815e4a0 con 0x7fec580cd6d0 2024-04-17T09:52:49.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec5de00640 1 -- 172.21.15.144:0/2895975344 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fec5815ea50 con 0x7fec580cd6d0 2024-04-17T09:52:49.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec4f400640 1 -- 172.21.15.144:0/2895975344 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fec4001a070 con 0x7fec580cd6d0 2024-04-17T09:52:49.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec4f400640 1 -- 172.21.15.144:0/2895975344 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fec40005dc0 con 0x7fec580cd6d0 2024-04-17T09:52:49.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.349+0000 7fec5de00640 1 -- 172.21.15.144:0/2895975344 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec58068a90 con 0x7fec580cd6d0 2024-04-17T09:52:49.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.353+0000 7fec4f400640 1 -- 172.21.15.144:0/2895975344 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fec40006660 con 0x7fec580cd6d0 2024-04-17T09:52:49.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.421+0000 7fec5de00640 1 -- 172.21.15.144:0/2895975344 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3e5c6a47-f005-4854-b351-a83b0bf1525a", "id": 2} v 0) -- 0x7fec580b7470 con 0x7fec580cd6d0 2024-04-17T09:52:49.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.425+0000 7fec4f400640 1 -- 172.21.15.144:0/2895975344 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3e5c6a47-f005-4854-b351-a83b0bf1525a", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fec40006020 con 0x7fec580cd6d0 2024-04-17T09:52:49.427 INFO:teuthology.orchestra.run.smithi144.stdout:2 2024-04-17T09:52:49.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.425+0000 7fec5de00640 1 -- 172.21.15.144:0/2895975344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec580cd6d0 msgr2=0x7fec5815df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:49.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.425+0000 7fec5de00640 1 --2- 172.21.15.144:0/2895975344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec580cd6d0 0x7fec5815df60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fec4000e930 tx=0x7fec4000edf0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.425+0000 7fec5de00640 1 -- 172.21.15.144:0/2895975344 shutdown_connections 2024-04-17T09:52:49.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.425+0000 7fec5de00640 1 --2- 172.21.15.144:0/2895975344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec580cd6d0 0x7fec5815df60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.425+0000 7fec5de00640 1 --2- 172.21.15.144:0/2895975344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec5815a410 0x7fec5806fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.425+0000 7fec5de00640 1 --2- 172.21.15.144:0/2895975344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec5806c6f0 0x7fec581f9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.425+0000 7fec5de00640 1 -- 172.21.15.144:0/2895975344 >> 172.21.15.144:0/2895975344 conn(0x7fec580b3000 msgr2=0x7fec58159220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:49.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.425+0000 7fec5de00640 1 -- 172.21.15.144:0/2895975344 shutdown_connections 2024-04-17T09:52:49.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.425+0000 7fec5de00640 1 -- 172.21.15.144:0/2895975344 wait complete. 2024-04-17T09:52:49.445 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 0d5b4f04-83ef-4444-bb10-477c868e58a8 3 2024-04-17T09:52:49.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.577+0000 7f8d2aa00640 1 Processor -- start 2024-04-17T09:52:49.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.577+0000 7f8d2aa00640 1 -- start start 2024-04-17T09:52:49.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.577+0000 7f8d2aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d2415c890 0x7f8d240d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:49.582 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.577+0000 7f8d2aa00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d240d0b60 con 0x7f8d2415dcd0 2024-04-17T09:52:49.582 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.577+0000 7f8d2aa00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d240d0e40 con 0x7f8d2409b100 2024-04-17T09:52:49.582 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.577+0000 7f8d2aa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8d240d16e0 con 0x7f8d2415c890 2024-04-17T09:52:49.582 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.577+0000 7f8d23e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d2415c890 0x7f8d240d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:49.582 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.577+0000 7f8d28a00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f8d2409b100 0x7f8d2415f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52668/0 (socket says 172.21.15.144:52668) 2024-04-17T09:52:49.582 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.577+0000 7f8d28a00640 1 -- 172.21.15.144:0/3030469459 learned_addr learned my addr 172.21.15.144:0/3030469459 (peer_addr_for_me v1:172.21.15.144:0/0) 2024-04-17T09:52:49.582 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.577+0000 7f8d23e00640 1 -- 172.21.15.144:0/3030469459 >> v1:172.21.15.144:6789/0 conn(0x7f8d2409b100 legacy=0x7f8d2415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:49.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.577+0000 7f8d23400640 1 -- 172.21.15.144:0/3030469459 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2627106439 0 0) 0x7f8d240d0e40 con 0x7f8d2409b100 2024-04-17T09:52:49.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.577+0000 7f8d23e00640 1 -- 172.21.15.144:0/3030469459 >> v1:172.21.15.32:6789/0 conn(0x7f8d2415dcd0 legacy=0x7f8d2409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:49.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.581+0000 7f8d23e00640 1 -- 172.21.15.144:0/3030469459 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d240d1120 con 0x7f8d2415c890 2024-04-17T09:52:49.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.585+0000 7f8d23e00640 1 --2- 172.21.15.144:0/3030469459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d2415c890 0x7f8d240d01f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8d140097f0 tx=0x7f8d1401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=608ca0725ec2623b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:49.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.585+0000 7f8d23400640 1 -- 172.21.15.144:0/3030469459 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d140b1070 con 0x7f8d2415c890 2024-04-17T09:52:49.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.585+0000 7f8d23400640 1 -- 172.21.15.144:0/3030469459 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d140ac630 con 0x7f8d2415c890 2024-04-17T09:52:49.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.585+0000 7f8d23400640 1 -- 172.21.15.144:0/3030469459 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d140a8050 con 0x7f8d2415c890 2024-04-17T09:52:49.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.585+0000 7f8d2aa00640 1 -- 172.21.15.144:0/3030469459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d2415c890 msgr2=0x7f8d240d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:49.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.585+0000 7f8d2aa00640 1 --2- 172.21.15.144:0/3030469459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d2415c890 0x7f8d240d01f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8d140097f0 tx=0x7f8d1401c4e0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.585+0000 7f8d2aa00640 1 -- 172.21.15.144:0/3030469459 shutdown_connections 2024-04-17T09:52:49.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.585+0000 7f8d2aa00640 1 --2- 172.21.15.144:0/3030469459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d2415c890 0x7f8d240d01f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.585+0000 7f8d2aa00640 1 -- 172.21.15.144:0/3030469459 >> 172.21.15.144:0/3030469459 conn(0x7f8d240b4110 msgr2=0x7f8d240b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:49.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.585+0000 7f8d2aa00640 1 -- 172.21.15.144:0/3030469459 shutdown_connections 2024-04-17T09:52:49.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.585+0000 7f8d2aa00640 1 -- 172.21.15.144:0/3030469459 wait complete. 2024-04-17T09:52:49.589 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.585+0000 7f8d2aa00640 1 Processor -- start 2024-04-17T09:52:49.589 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.585+0000 7f8d2aa00640 1 -- start start 2024-04-17T09:52:49.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d2aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d2409b100 0x7f8d240af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:49.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d2aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d2415c890 0x7f8d240cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:49.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d2aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d2415dcd0 0x7f8d240c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:49.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d2aa00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2415cdf0 con 0x7f8d2415dcd0 2024-04-17T09:52:49.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d2aa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2415cc70 con 0x7f8d2409b100 2024-04-17T09:52:49.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d2aa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8d240d0730 con 0x7f8d2415c890 2024-04-17T09:52:49.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d23e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d2409b100 0x7f8d240af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:49.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d23e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d2409b100 0x7f8d240af270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41280/0 (socket says 172.21.15.144:41280) 2024-04-17T09:52:49.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d23e00640 1 -- 172.21.15.144:0/1365419945 learned_addr learned my addr 172.21.15.144:0/1365419945 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-17T09:52:49.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d1be00640 1 --2- 172.21.15.144:0/1365419945 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d2415c890 0x7f8d240cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:49.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d28a00640 1 --2- 172.21.15.144:0/1365419945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d2415dcd0 0x7f8d240c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:49.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d23e00640 1 -- 172.21.15.144:0/1365419945 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d2415c890 msgr2=0x7f8d240cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:49.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d23e00640 1 --2- 172.21.15.144:0/1365419945 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d2415c890 0x7f8d240cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d23e00640 1 -- 172.21.15.144:0/1365419945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d2415dcd0 msgr2=0x7f8d240c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:49.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d23e00640 1 --2- 172.21.15.144:0/1365419945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d2415dcd0 0x7f8d240c96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d23e00640 1 -- 172.21.15.144:0/1365419945 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2406e250 con 0x7f8d2409b100 2024-04-17T09:52:49.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d1be00640 1 --2- 172.21.15.144:0/1365419945 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d2415c890 0x7f8d240cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:52:49.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d28a00640 1 --2- 172.21.15.144:0/1365419945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d2415dcd0 0x7f8d240c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:52:49.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d23e00640 1 --2- 172.21.15.144:0/1365419945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d2409b100 0x7f8d240af270 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8d140097f0 tx=0x7f8d1401ad20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:49.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d20c00640 1 -- 172.21.15.144:0/1365419945 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d140b1050 con 0x7f8d2409b100 2024-04-17T09:52:49.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d20c00640 1 -- 172.21.15.144:0/1365419945 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d140bd070 con 0x7f8d2409b100 2024-04-17T09:52:49.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d20c00640 1 -- 172.21.15.144:0/1365419945 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d140b6a70 con 0x7f8d2409b100 2024-04-17T09:52:49.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d2aa00640 1 -- 172.21.15.144:0/1365419945 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d2406e530 con 0x7f8d2409b100 2024-04-17T09:52:49.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d2aa00640 1 -- 172.21.15.144:0/1365419945 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d2406eaf0 con 0x7f8d2409b100 2024-04-17T09:52:49.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d20c00640 1 -- 172.21.15.144:0/1365419945 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8d140b6c10 con 0x7f8d2409b100 2024-04-17T09:52:49.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d20c00640 1 -- 172.21.15.144:0/1365419945 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f8d140ac4b0 con 0x7f8d2409b100 2024-04-17T09:52:49.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.589+0000 7f8d2aa00640 1 -- 172.21.15.144:0/1365419945 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d24068a90 con 0x7f8d2409b100 2024-04-17T09:52:49.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.593+0000 7f8d20c00640 1 -- 172.21.15.144:0/1365419945 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f8d140160d0 con 0x7f8d2409b100 2024-04-17T09:52:49.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.681+0000 7f8d2aa00640 1 -- 172.21.15.144:0/1365419945 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0d5b4f04-83ef-4444-bb10-477c868e58a8", "id": 3} v 0) -- 0x7f8d2409b520 con 0x7f8d2409b100 2024-04-17T09:52:49.687 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.685+0000 7f8d20c00640 1 -- 172.21.15.144:0/1365419945 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0d5b4f04-83ef-4444-bb10-477c868e58a8", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f8d140b8450 con 0x7f8d2409b100 2024-04-17T09:52:49.687 INFO:teuthology.orchestra.run.smithi144.stdout:3 2024-04-17T09:52:49.688 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.685+0000 7f8d2aa00640 1 -- 172.21.15.144:0/1365419945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d2409b100 msgr2=0x7f8d240af270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:49.688 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.685+0000 7f8d2aa00640 1 --2- 172.21.15.144:0/1365419945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d2409b100 0x7f8d240af270 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8d140097f0 tx=0x7f8d1401ad20 comp rx=0 tx=0).stop 2024-04-17T09:52:49.688 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.685+0000 7f8d2aa00640 1 -- 172.21.15.144:0/1365419945 shutdown_connections 2024-04-17T09:52:49.688 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.685+0000 7f8d2aa00640 1 --2- 172.21.15.144:0/1365419945 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d2415c890 0x7f8d240cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.688 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.685+0000 7f8d2aa00640 1 --2- 172.21.15.144:0/1365419945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d2415dcd0 0x7f8d240c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.688 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.685+0000 7f8d2aa00640 1 --2- 172.21.15.144:0/1365419945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d2409b100 0x7f8d240af270 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.688 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.685+0000 7f8d2aa00640 1 -- 172.21.15.144:0/1365419945 >> 172.21.15.144:0/1365419945 conn(0x7f8d240b4110 msgr2=0x7f8d240b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:49.689 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.685+0000 7f8d2aa00640 1 -- 172.21.15.144:0/1365419945 shutdown_connections 2024-04-17T09:52:49.689 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.685+0000 7f8d2aa00640 1 -- 172.21.15.144:0/1365419945 wait complete. 2024-04-17T09:52:49.706 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new db4af439-9a47-4c14-a6dc-6c4eef51ea45 4 2024-04-17T09:52:49.739 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:49.736+0000 7f63c8272180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T09:52:49.804 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:49.804+0000 7f63c8272180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T09:52:49.827 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-17T09:52:49.825+0000 7f32a8109180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T09:52:49.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.841+0000 7f9394e00640 1 Processor -- start 2024-04-17T09:52:49.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.841+0000 7f9394e00640 1 -- start start 2024-04-17T09:52:49.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.841+0000 7f9394e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f939015c890 0x7f93900d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:49.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.841+0000 7f9394e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f939009b100 0x7f939015f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:49.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.841+0000 7f9394e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93900d0b60 con 0x7f939015dcd0 2024-04-17T09:52:49.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.841+0000 7f9394e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f93900d16e0 con 0x7f939009b100 2024-04-17T09:52:49.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.841+0000 7f9394e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f93900cd1a0 con 0x7f939015c890 2024-04-17T09:52:49.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.841+0000 7f938de00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f939015c890 0x7f93900d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:49.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.841+0000 7f938de00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f939015c890 0x7f93900d01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41292/0 (socket says 172.21.15.144:41292) 2024-04-17T09:52:49.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.841+0000 7f938de00640 1 -- 172.21.15.144:0/3333118043 learned_addr learned my addr 172.21.15.144:0/3333118043 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-17T09:52:49.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.841+0000 7f938e800640 1 --2- 172.21.15.144:0/3333118043 >> v2:172.21.15.32:3300/0 conn(0x7f939009b100 0x7f939015f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:49.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.841+0000 7f938de00640 1 -- 172.21.15.144:0/3333118043 >> v2:172.21.15.32:3300/0 conn(0x7f939009b100 msgr2=0x7f939015f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:52:49.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.841+0000 7f938de00640 1 --2- 172.21.15.144:0/3333118043 >> v2:172.21.15.32:3300/0 conn(0x7f939009b100 0x7f939015f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.841+0000 7f938de00640 1 -- 172.21.15.144:0/3333118043 >> v1:172.21.15.32:6789/0 conn(0x7f939015dcd0 legacy=0x7f939009a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:49.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.841+0000 7f938de00640 1 -- 172.21.15.144:0/3333118043 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93900d0e40 con 0x7f939015c890 2024-04-17T09:52:49.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.845+0000 7f938de00640 1 --2- 172.21.15.144:0/3333118043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f939015c890 0x7f93900d01f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f93900735a0 tx=0x7f938001c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9f7f3a9035ea563 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:49.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.845+0000 7f938ca00640 1 -- 172.21.15.144:0/3333118043 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93800aa650 con 0x7f939015c890 2024-04-17T09:52:49.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.845+0000 7f938ca00640 1 -- 172.21.15.144:0/3333118043 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93800af070 con 0x7f939015c890 2024-04-17T09:52:49.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.845+0000 7f938ca00640 1 -- 172.21.15.144:0/3333118043 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93800aaa10 con 0x7f939015c890 2024-04-17T09:52:49.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.845+0000 7f9394e00640 1 -- 172.21.15.144:0/3333118043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f939015c890 msgr2=0x7f93900d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:49.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.845+0000 7f9394e00640 1 --2- 172.21.15.144:0/3333118043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f939015c890 0x7f93900d01f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f93900735a0 tx=0x7f938001c4f0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.845+0000 7f9394e00640 1 -- 172.21.15.144:0/3333118043 shutdown_connections 2024-04-17T09:52:49.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.845+0000 7f9394e00640 1 --2- 172.21.15.144:0/3333118043 >> v2:172.21.15.32:3300/0 conn(0x7f939009b100 0x7f939015f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.845+0000 7f9394e00640 1 --2- 172.21.15.144:0/3333118043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f939015c890 0x7f93900d01f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.845+0000 7f9394e00640 1 -- 172.21.15.144:0/3333118043 >> 172.21.15.144:0/3333118043 conn(0x7f93900b4110 msgr2=0x7f93900b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:49.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.845+0000 7f9394e00640 1 -- 172.21.15.144:0/3333118043 shutdown_connections 2024-04-17T09:52:49.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.845+0000 7f9394e00640 1 -- 172.21.15.144:0/3333118043 wait complete. 2024-04-17T09:52:49.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.849+0000 7f9394e00640 1 Processor -- start 2024-04-17T09:52:49.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.849+0000 7f9394e00640 1 -- start start 2024-04-17T09:52:49.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.849+0000 7f9394e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f939009b100 0x7f9390153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:49.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.849+0000 7f9394e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f939015c890 0x7f939006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.849+0000 7f9394e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f939015dcd0 0x7f9390152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.849+0000 7f9394e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f939015cdf0 con 0x7f939015dcd0 2024-04-17T09:52:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.849+0000 7f9394e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f939015cc70 con 0x7f939009b100 2024-04-17T09:52:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.849+0000 7f9394e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f93900d0730 con 0x7f939015c890 2024-04-17T09:52:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.849+0000 7f938de00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f939009b100 0x7f9390153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.849+0000 7f938d400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f939015c890 0x7f939006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.849+0000 7f938de00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f939009b100 0x7f9390153630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41294/0 (socket says 172.21.15.144:41294) 2024-04-17T09:52:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.849+0000 7f938de00640 1 -- 172.21.15.144:0/3137503211 learned_addr learned my addr 172.21.15.144:0/3137503211 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-17T09:52:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.853+0000 7f938e800640 1 --2- 172.21.15.144:0/3137503211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f939015dcd0 0x7f9390152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.853+0000 7f938d400640 1 -- 172.21.15.144:0/3137503211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f939009b100 msgr2=0x7f9390153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.853+0000 7f938d400640 1 --2- 172.21.15.144:0/3137503211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f939009b100 0x7f9390153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.853+0000 7f938d400640 1 -- 172.21.15.144:0/3137503211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f939015dcd0 msgr2=0x7f9390152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.853+0000 7f938d400640 1 --2- 172.21.15.144:0/3137503211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f939015dcd0 0x7f9390152e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.853+0000 7f938d400640 1 -- 172.21.15.144:0/3137503211 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939006e880 con 0x7f939015c890 2024-04-17T09:52:49.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.853+0000 7f938d400640 1 --2- 172.21.15.144:0/3137503211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f939015c890 0x7f939006e340 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f937c00c3c0 tx=0x7f937c00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:49.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.853+0000 7f9386000640 1 -- 172.21.15.144:0/3137503211 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937c0052d0 con 0x7f939015c890 2024-04-17T09:52:49.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.853+0000 7f9386000640 1 -- 172.21.15.144:0/3137503211 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f937c005470 con 0x7f939015c890 2024-04-17T09:52:49.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.853+0000 7f9394e00640 1 -- 172.21.15.144:0/3137503211 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93900cb520 con 0x7f939015c890 2024-04-17T09:52:49.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.853+0000 7f9386000640 1 -- 172.21.15.144:0/3137503211 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937c007260 con 0x7f939015c890 2024-04-17T09:52:49.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.853+0000 7f9394e00640 1 -- 172.21.15.144:0/3137503211 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f93900cbae0 con 0x7f939015c890 2024-04-17T09:52:49.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.853+0000 7f9386000640 1 -- 172.21.15.144:0/3137503211 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f937c007400 con 0x7f939015c890 2024-04-17T09:52:49.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.853+0000 7f9386000640 1 -- 172.21.15.144:0/3137503211 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f937c0067e0 con 0x7f939015c890 2024-04-17T09:52:49.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.853+0000 7f9394e00640 1 -- 172.21.15.144:0/3137503211 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9390068a90 con 0x7f939015c890 2024-04-17T09:52:49.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.857+0000 7f9386000640 1 -- 172.21.15.144:0/3137503211 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f939006e880 con 0x7f939015c890 2024-04-17T09:52:49.869 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:49.868+0000 7f63c8272180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T09:52:49.896 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-17T09:52:49.893+0000 7f32a8109180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T09:52:49.925 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.921+0000 7f9394e00640 1 -- 172.21.15.144:0/3137503211 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "db4af439-9a47-4c14-a6dc-6c4eef51ea45", "id": 4} v 0) -- 0x7f939009b520 con 0x7f939015c890 2024-04-17T09:52:49.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.925+0000 7f9386000640 1 -- 172.21.15.144:0/3137503211 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "db4af439-9a47-4c14-a6dc-6c4eef51ea45", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f937c003600 con 0x7f939015c890 2024-04-17T09:52:49.929 INFO:teuthology.orchestra.run.smithi144.stdout:4 2024-04-17T09:52:49.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.929+0000 7f9394e00640 1 -- 172.21.15.144:0/3137503211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f939015c890 msgr2=0x7f939006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:49.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.929+0000 7f9394e00640 1 --2- 172.21.15.144:0/3137503211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f939015c890 0x7f939006e340 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f937c00c3c0 tx=0x7f937c00c880 comp rx=0 tx=0).stop 2024-04-17T09:52:49.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.929+0000 7f9394e00640 1 -- 172.21.15.144:0/3137503211 shutdown_connections 2024-04-17T09:52:49.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.929+0000 7f9394e00640 1 --2- 172.21.15.144:0/3137503211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f939015c890 0x7f939006e340 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.929+0000 7f9394e00640 1 --2- 172.21.15.144:0/3137503211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f939015dcd0 0x7f9390152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.929+0000 7f9394e00640 1 --2- 172.21.15.144:0/3137503211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f939009b100 0x7f9390153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:49.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.929+0000 7f9394e00640 1 -- 172.21.15.144:0/3137503211 >> 172.21.15.144:0/3137503211 conn(0x7f93900b4110 msgr2=0x7f9390070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:49.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.929+0000 7f9394e00640 1 -- 172.21.15.144:0/3137503211 shutdown_connections 2024-04-17T09:52:49.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:49.929+0000 7f9394e00640 1 -- 172.21.15.144:0/3137503211 wait complete. 2024-04-17T09:52:49.948 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new c6b386a6-ee3e-4830-b841-8a0eccd10f52 5 2024-04-17T09:52:49.961 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-17T09:52:49.957+0000 7f32a8109180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T09:52:49.962 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:49.960+0000 7f63c8272180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T09:52:50.026 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:50.024+0000 7f63c8272180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T09:52:50.053 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-17T09:52:50.049+0000 7f32a8109180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T09:52:50.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.081+0000 7f015c200640 1 Processor -- start 2024-04-17T09:52:50.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.081+0000 7f015c200640 1 -- start start 2024-04-17T09:52:50.086 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.081+0000 7f015c200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f015415c890 0x7f01540d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:50.086 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.081+0000 7f015c200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f015415dcd0 0x7f015409a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:50.086 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.081+0000 7f015c200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f015409af30 0x7f015415f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:50.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.081+0000 7f015c200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f01540d16e0 con 0x7f015409af30 2024-04-17T09:52:50.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.081+0000 7f015c200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f01540cd1a0 con 0x7f015415dcd0 2024-04-17T09:52:50.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.081+0000 7f015c200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f01540d0b60 con 0x7f015415c890 2024-04-17T09:52:50.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.081+0000 7f0159800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f015415c890 0x7f01540d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:50.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.081+0000 7f0158e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f015415dcd0 0x7f015409a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:50.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.081+0000 7f0158e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f015415dcd0 0x7f015409a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41302/0 (socket says 172.21.15.144:41302) 2024-04-17T09:52:50.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.081+0000 7f0158e00640 1 -- 172.21.15.144:0/1894072818 learned_addr learned my addr 172.21.15.144:0/1894072818 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-17T09:52:50.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.081+0000 7f0159800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f015415c890 0x7f01540d01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:60560/0 (socket says 172.21.15.144:60560) 2024-04-17T09:52:50.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.081+0000 7f015a200640 1 --2- 172.21.15.144:0/1894072818 >> v2:172.21.15.32:3300/0 conn(0x7f015409af30 0x7f015415f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:50.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.085+0000 7f0159800640 1 -- 172.21.15.144:0/1894072818 >> v2:172.21.15.144:3300/0 conn(0x7f015415dcd0 msgr2=0x7f015409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:52:50.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.085+0000 7f0159800640 1 --2- 172.21.15.144:0/1894072818 >> v2:172.21.15.144:3300/0 conn(0x7f015415dcd0 0x7f015409a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.085+0000 7f0159800640 1 -- 172.21.15.144:0/1894072818 >> v2:172.21.15.32:3300/0 conn(0x7f015409af30 msgr2=0x7f015415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:52:50.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.085+0000 7f0159800640 1 --2- 172.21.15.144:0/1894072818 >> v2:172.21.15.32:3300/0 conn(0x7f015409af30 0x7f015415f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.085+0000 7f0159800640 1 -- 172.21.15.144:0/1894072818 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01540d0e40 con 0x7f015415c890 2024-04-17T09:52:50.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.085+0000 7f0158e00640 1 --2- 172.21.15.144:0/1894072818 >> v2:172.21.15.144:3300/0 conn(0x7f015415dcd0 0x7f015409a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:52:50.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.085+0000 7f015a200640 1 --2- 172.21.15.144:0/1894072818 >> v2:172.21.15.32:3300/0 conn(0x7f015409af30 0x7f015415f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:52:50.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.085+0000 7f0159800640 1 --2- 172.21.15.144:0/1894072818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f015415c890 0x7f01540d01f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f015406ca10 tx=0x7f014801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dcc3a599ee652d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:50.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.085+0000 7f0153e00640 1 -- 172.21.15.144:0/1894072818 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01480b0070 con 0x7f015415c890 2024-04-17T09:52:50.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.085+0000 7f0153e00640 1 -- 172.21.15.144:0/1894072818 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01541602e0 con 0x7f015415c890 2024-04-17T09:52:50.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.085+0000 7f0153e00640 1 -- 172.21.15.144:0/1894072818 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01480ab820 con 0x7f015415c890 2024-04-17T09:52:50.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.085+0000 7f015c200640 1 -- 172.21.15.144:0/1894072818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f015415c890 msgr2=0x7f01540d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:50.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.085+0000 7f015c200640 1 --2- 172.21.15.144:0/1894072818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f015415c890 0x7f01540d01f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f015406ca10 tx=0x7f014801c480 comp rx=0 tx=0).stop 2024-04-17T09:52:50.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f015c200640 1 -- 172.21.15.144:0/1894072818 shutdown_connections 2024-04-17T09:52:50.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f015c200640 1 --2- 172.21.15.144:0/1894072818 >> v2:172.21.15.32:3300/0 conn(0x7f015409af30 0x7f015415f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f015c200640 1 --2- 172.21.15.144:0/1894072818 >> v2:172.21.15.144:3300/0 conn(0x7f015415dcd0 0x7f015409a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f015c200640 1 --2- 172.21.15.144:0/1894072818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f015415c890 0x7f01540d01f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f015c200640 1 -- 172.21.15.144:0/1894072818 >> 172.21.15.144:0/1894072818 conn(0x7f01540b4110 msgr2=0x7f01540b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:50.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f015c200640 1 -- 172.21.15.144:0/1894072818 shutdown_connections 2024-04-17T09:52:50.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f015c200640 1 -- 172.21.15.144:0/1894072818 wait complete. 2024-04-17T09:52:50.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f015c200640 1 Processor -- start 2024-04-17T09:52:50.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f015c200640 1 -- start start 2024-04-17T09:52:50.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f015c200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f015409af30 0x7f015415bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:50.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f015c200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f015415c890 0x7f01540ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:50.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f015c200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f015415dcd0 0x7f01540ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:50.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f015c200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f015415cdf0 con 0x7f015409af30 2024-04-17T09:52:50.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f015c200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f015415cc70 con 0x7f015415c890 2024-04-17T09:52:50.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f015c200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f01540d0730 con 0x7f015415dcd0 2024-04-17T09:52:50.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f0158e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f015415c890 0x7f01540ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:50.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f015a200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f015415dcd0 0x7f01540ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:50.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f0158e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f015415c890 0x7f01540ca3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41304/0 (socket says 172.21.15.144:41304) 2024-04-17T09:52:50.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f0158e00640 1 -- 172.21.15.144:0/1528651499 learned_addr learned my addr 172.21.15.144:0/1528651499 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-17T09:52:50.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f0159800640 1 --2- 172.21.15.144:0/1528651499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f015409af30 0x7f015415bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:50.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f0158e00640 1 -- 172.21.15.144:0/1528651499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f015415dcd0 msgr2=0x7f01540ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:50.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f0158e00640 1 --2- 172.21.15.144:0/1528651499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f015415dcd0 0x7f01540ca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f0158e00640 1 -- 172.21.15.144:0/1528651499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f015409af30 msgr2=0x7f015415bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:50.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f0158e00640 1 --2- 172.21.15.144:0/1528651499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f015409af30 0x7f015415bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f0158e00640 1 -- 172.21.15.144:0/1528651499 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015415c0f0 con 0x7f015415c890 2024-04-17T09:52:50.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.089+0000 7f0158e00640 1 --2- 172.21.15.144:0/1528651499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f015415c890 0x7f01540ca3a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f014000e9d0 tx=0x7f014000ee90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:50.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.093+0000 7f0151600640 1 -- 172.21.15.144:0/1528651499 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0140017070 con 0x7f015415c890 2024-04-17T09:52:50.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.093+0000 7f0151600640 1 -- 172.21.15.144:0/1528651499 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f014000ca60 con 0x7f015415c890 2024-04-17T09:52:50.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.093+0000 7f015c200640 1 -- 172.21.15.144:0/1528651499 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f015406d6f0 con 0x7f015415c890 2024-04-17T09:52:50.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.093+0000 7f0151600640 1 -- 172.21.15.144:0/1528651499 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014000ce10 con 0x7f015415c890 2024-04-17T09:52:50.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.093+0000 7f015c200640 1 -- 172.21.15.144:0/1528651499 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f015406dcb0 con 0x7f015415c890 2024-04-17T09:52:50.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.093+0000 7f0151600640 1 -- 172.21.15.144:0/1528651499 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f014001a070 con 0x7f015415c890 2024-04-17T09:52:50.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.093+0000 7f0151600640 1 -- 172.21.15.144:0/1528651499 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f0140006570 con 0x7f015415c890 2024-04-17T09:52:50.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.093+0000 7f015c200640 1 -- 172.21.15.144:0/1528651499 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0154068a90 con 0x7f015415c890 2024-04-17T09:52:50.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.093+0000 7f0151600640 1 -- 172.21.15.144:0/1528651499 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f0140006850 con 0x7f015415c890 2024-04-17T09:52:50.118 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-17T09:52:50.113+0000 7f32a8109180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T09:52:50.119 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:50.116+0000 7f63c8272180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T09:52:50.167 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.165+0000 7f015c200640 1 -- 172.21.15.144:0/1528651499 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c6b386a6-ee3e-4830-b841-8a0eccd10f52", "id": 5} v 0) -- 0x7f01540b3a10 con 0x7f015415c890 2024-04-17T09:52:50.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.173+0000 7f0151600640 1 -- 172.21.15.144:0/1528651499 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c6b386a6-ee3e-4830-b841-8a0eccd10f52", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f0140006130 con 0x7f015415c890 2024-04-17T09:52:50.175 INFO:teuthology.orchestra.run.smithi144.stdout:5 2024-04-17T09:52:50.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.173+0000 7f015c200640 1 -- 172.21.15.144:0/1528651499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f015415c890 msgr2=0x7f01540ca3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:50.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.173+0000 7f015c200640 1 --2- 172.21.15.144:0/1528651499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f015415c890 0x7f01540ca3a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f014000e9d0 tx=0x7f014000ee90 comp rx=0 tx=0).stop 2024-04-17T09:52:50.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.173+0000 7f015c200640 1 -- 172.21.15.144:0/1528651499 shutdown_connections 2024-04-17T09:52:50.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.173+0000 7f015c200640 1 --2- 172.21.15.144:0/1528651499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f015415dcd0 0x7f01540ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.173+0000 7f015c200640 1 --2- 172.21.15.144:0/1528651499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f015415c890 0x7f01540ca3a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.173+0000 7f015c200640 1 --2- 172.21.15.144:0/1528651499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f015409af30 0x7f015415bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.173+0000 7f015c200640 1 -- 172.21.15.144:0/1528651499 >> 172.21.15.144:0/1528651499 conn(0x7f01540b4110 msgr2=0x7f0154070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:50.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.173+0000 7f015c200640 1 -- 172.21.15.144:0/1528651499 shutdown_connections 2024-04-17T09:52:50.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.173+0000 7f015c200640 1 -- 172.21.15.144:0/1528651499 wait complete. 2024-04-17T09:52:50.186 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:50.184+0000 7f63c8272180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T09:52:50.193 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new ed3e4cf6-cfa2-472b-a061-d9a5734f4169 6 2024-04-17T09:52:50.212 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-17T09:52:50.209+0000 7f32a8109180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T09:52:50.277 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-17T09:52:50.273+0000 7f32a8109180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T09:52:50.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.341+0000 7f9e9de00640 1 Processor -- start 2024-04-17T09:52:50.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.341+0000 7f9e9de00640 1 -- start start 2024-04-17T09:52:50.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.345+0000 7f9e9de00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e9815c890 0x7f9e980d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:50.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.345+0000 7f9e9de00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e9815dcd0 0x7f9e9809a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:50.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.345+0000 7f9e9de00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e980d0b60 con 0x7f9e9809af30 2024-04-17T09:52:50.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.345+0000 7f9e9de00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9e980d16e0 con 0x7f9e9815dcd0 2024-04-17T09:52:50.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.345+0000 7f9e9de00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9e980cd1a0 con 0x7f9e9815c890 2024-04-17T09:52:50.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.345+0000 7f9e97400640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f9e9809af30 0x7f9e9815f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52678/0 (socket says 172.21.15.144:52678) 2024-04-17T09:52:50.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.345+0000 7f9e97400640 1 -- 172.21.15.144:0/2115303675 learned_addr learned my addr 172.21.15.144:0/2115303675 (peer_addr_for_me v1:172.21.15.144:0/0) 2024-04-17T09:52:50.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.345+0000 7f9e96a00640 1 --2- 172.21.15.144:0/2115303675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e9815c890 0x7f9e980d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:50.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.345+0000 7f9e8fe00640 1 --2- 172.21.15.144:0/2115303675 >> v2:172.21.15.32:3300/0 conn(0x7f9e9815dcd0 0x7f9e9809a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:50.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.345+0000 7f9e96000640 1 -- 172.21.15.144:0/2115303675 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3123198584 0 0) 0x7f9e980d0b60 con 0x7f9e9809af30 2024-04-17T09:52:50.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.345+0000 7f9e96000640 1 -- 172.21.15.144:0/2115303675 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e7c003530 con 0x7f9e9809af30 2024-04-17T09:52:50.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.345+0000 7f9e96a00640 1 -- 172.21.15.144:0/2115303675 >> v2:172.21.15.32:3300/0 conn(0x7f9e9815dcd0 msgr2=0x7f9e9809a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:52:50.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.345+0000 7f9e96a00640 1 --2- 172.21.15.144:0/2115303675 >> v2:172.21.15.32:3300/0 conn(0x7f9e9815dcd0 0x7f9e9809a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.345+0000 7f9e96a00640 1 -- 172.21.15.144:0/2115303675 >> v1:172.21.15.144:6789/0 conn(0x7f9e9809af30 legacy=0x7f9e9815f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:50.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.345+0000 7f9e96a00640 1 -- 172.21.15.144:0/2115303675 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e980d0e40 con 0x7f9e9815c890 2024-04-17T09:52:50.350 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.345+0000 7f9e96a00640 1 --2- 172.21.15.144:0/2115303675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e9815c890 0x7f9e980d01f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9e880097c0 tx=0x7f9e8801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eeaa0d3c01fbd1e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:50.350 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e96000640 1 -- 172.21.15.144:0/2115303675 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e880b2070 con 0x7f9e9815c890 2024-04-17T09:52:50.351 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e96000640 1 -- 172.21.15.144:0/2115303675 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e880ad640 con 0x7f9e9815c890 2024-04-17T09:52:50.351 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e96000640 1 -- 172.21.15.144:0/2115303675 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e880ada10 con 0x7f9e9815c890 2024-04-17T09:52:50.351 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e9de00640 1 -- 172.21.15.144:0/2115303675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e9815c890 msgr2=0x7f9e980d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:50.351 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e9de00640 1 --2- 172.21.15.144:0/2115303675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e9815c890 0x7f9e980d01f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9e880097c0 tx=0x7f9e8801c4a0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.351 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e9de00640 1 -- 172.21.15.144:0/2115303675 shutdown_connections 2024-04-17T09:52:50.351 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e9de00640 1 --2- 172.21.15.144:0/2115303675 >> v2:172.21.15.32:3300/0 conn(0x7f9e9815dcd0 0x7f9e9809a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.352 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e9de00640 1 --2- 172.21.15.144:0/2115303675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e9815c890 0x7f9e980d01f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.352 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e9de00640 1 -- 172.21.15.144:0/2115303675 >> 172.21.15.144:0/2115303675 conn(0x7f9e980b4110 msgr2=0x7f9e980b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:50.352 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e9de00640 1 -- 172.21.15.144:0/2115303675 shutdown_connections 2024-04-17T09:52:50.352 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e9de00640 1 -- 172.21.15.144:0/2115303675 wait complete. 2024-04-17T09:52:50.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e9de00640 1 Processor -- start 2024-04-17T09:52:50.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e9de00640 1 -- start start 2024-04-17T09:52:50.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e9de00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e9809af30 0x7f9e980c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:50.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e9de00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e9815c890 0x7f9e980cf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:50.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e9de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e9815dcd0 0x7f9e980cf550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:50.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e9de00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9815cdf0 con 0x7f9e9815dcd0 2024-04-17T09:52:50.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e9de00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9815cc70 con 0x7f9e9815c890 2024-04-17T09:52:50.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e9de00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9e980d0730 con 0x7f9e9809af30 2024-04-17T09:52:50.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e8fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e9815c890 0x7f9e980cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:50.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e96a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e9809af30 0x7f9e980c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:50.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e96a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e9809af30 0x7f9e980c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:60572/0 (socket says 172.21.15.144:60572) 2024-04-17T09:52:50.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e8fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e9815c890 0x7f9e980cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41312/0 (socket says 172.21.15.144:41312) 2024-04-17T09:52:50.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e96a00640 1 -- 172.21.15.144:0/666231778 learned_addr learned my addr 172.21.15.144:0/666231778 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-17T09:52:50.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.349+0000 7f9e97400640 1 --2- 172.21.15.144:0/666231778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e9815dcd0 0x7f9e980cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:50.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.353+0000 7f9e96a00640 1 -- 172.21.15.144:0/666231778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e9815c890 msgr2=0x7f9e980cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:50.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.353+0000 7f9e96a00640 1 --2- 172.21.15.144:0/666231778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e9815c890 0x7f9e980cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.353+0000 7f9e96a00640 1 -- 172.21.15.144:0/666231778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e9815dcd0 msgr2=0x7f9e980cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:50.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.353+0000 7f9e96a00640 1 --2- 172.21.15.144:0/666231778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e9815dcd0 0x7f9e980cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.353+0000 7f9e96a00640 1 -- 172.21.15.144:0/666231778 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e98159f40 con 0x7f9e9809af30 2024-04-17T09:52:50.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.353+0000 7f9e96a00640 1 --2- 172.21.15.144:0/666231778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e9809af30 0x7f9e980c8190 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9e8801a0d0 tx=0x7f9e88019240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:50.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.353+0000 7f9e8f400640 1 -- 172.21.15.144:0/666231778 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e880b2040 con 0x7f9e9809af30 2024-04-17T09:52:50.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.353+0000 7f9e8f400640 1 -- 172.21.15.144:0/666231778 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e880bd070 con 0x7f9e9809af30 2024-04-17T09:52:50.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.353+0000 7f9e8f400640 1 -- 172.21.15.144:0/666231778 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e88006e20 con 0x7f9e9809af30 2024-04-17T09:52:50.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.353+0000 7f9e9de00640 1 -- 172.21.15.144:0/666231778 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e981596a0 con 0x7f9e9809af30 2024-04-17T09:52:50.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.353+0000 7f9e9de00640 1 -- 172.21.15.144:0/666231778 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e98159c60 con 0x7f9e9809af30 2024-04-17T09:52:50.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.353+0000 7f9e8f400640 1 -- 172.21.15.144:0/666231778 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9e880b6670 con 0x7f9e9809af30 2024-04-17T09:52:50.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.353+0000 7f9e8f400640 1 -- 172.21.15.144:0/666231778 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f9e880c13c0 con 0x7f9e9809af30 2024-04-17T09:52:50.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.353+0000 7f9e9de00640 1 -- 172.21.15.144:0/666231778 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e98068a90 con 0x7f9e9809af30 2024-04-17T09:52:50.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.357+0000 7f9e8f400640 1 -- 172.21.15.144:0/666231778 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f9e88016090 con 0x7f9e9809af30 2024-04-17T09:52:50.360 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:50.360+0000 7f63c8272180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T09:52:50.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.421+0000 7f9e9de00640 1 -- 172.21.15.144:0/666231778 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ed3e4cf6-cfa2-472b-a061-d9a5734f4169", "id": 6} v 0) -- 0x7f9e9809b520 con 0x7f9e9809af30 2024-04-17T09:52:50.433 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.429+0000 7f9e8f400640 1 -- 172.21.15.144:0/666231778 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ed3e4cf6-cfa2-472b-a061-d9a5734f4169", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f9e880b6960 con 0x7f9e9809af30 2024-04-17T09:52:50.433 INFO:teuthology.orchestra.run.smithi144.stdout:6 2024-04-17T09:52:50.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.433+0000 7f9e9de00640 1 -- 172.21.15.144:0/666231778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e9809af30 msgr2=0x7f9e980c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:50.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.433+0000 7f9e9de00640 1 --2- 172.21.15.144:0/666231778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e9809af30 0x7f9e980c8190 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9e8801a0d0 tx=0x7f9e88019240 comp rx=0 tx=0).stop 2024-04-17T09:52:50.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.433+0000 7f9e9de00640 1 -- 172.21.15.144:0/666231778 shutdown_connections 2024-04-17T09:52:50.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.433+0000 7f9e9de00640 1 --2- 172.21.15.144:0/666231778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e9815dcd0 0x7f9e980cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.433+0000 7f9e9de00640 1 --2- 172.21.15.144:0/666231778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e9815c890 0x7f9e980cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.433+0000 7f9e9de00640 1 --2- 172.21.15.144:0/666231778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e9809af30 0x7f9e980c8190 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.433+0000 7f9e9de00640 1 -- 172.21.15.144:0/666231778 >> 172.21.15.144:0/666231778 conn(0x7f9e980b4110 msgr2=0x7f9e980b1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:50.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.433+0000 7f9e9de00640 1 -- 172.21.15.144:0/666231778 shutdown_connections 2024-04-17T09:52:50.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.433+0000 7f9e9de00640 1 -- 172.21.15.144:0/666231778 wait complete. 2024-04-17T09:52:50.436 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:50.432+0000 7f63c8272180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T09:52:50.453 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new bc3f5d1f-f576-4fa5-85fe-c957d278349f 7 2024-04-17T09:52:50.453 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-17T09:52:50.449+0000 7f32a8109180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T09:52:50.527 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-17T09:52:50.525+0000 7f32a8109180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T09:52:50.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.585+0000 7f0faf600640 1 Processor -- start 2024-04-17T09:52:50.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.585+0000 7f0faf600640 1 -- start start 2024-04-17T09:52:50.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.585+0000 7f0faf600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa80b57a0 0x7f0fa80aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:50.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.585+0000 7f0faf600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0fa80ae440 0x7f0fa80b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:50.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.585+0000 7f0faf600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fa806c210 con 0x7f0fa806ba30 2024-04-17T09:52:50.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.585+0000 7f0faf600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0fa806ce20 con 0x7f0fa80ae440 2024-04-17T09:52:50.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.585+0000 7f0faf600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0fa806c4f0 con 0x7f0fa80b57a0 2024-04-17T09:52:50.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.585+0000 7f0facc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa80b57a0 0x7f0fa80aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:50.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.585+0000 7f0facc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa80b57a0 0x7f0fa80aff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:60586/0 (socket says 172.21.15.144:60586) 2024-04-17T09:52:50.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.585+0000 7f0facc00640 1 -- 172.21.15.144:0/2851471836 learned_addr learned my addr 172.21.15.144:0/2851471836 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-17T09:52:50.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.585+0000 7f0fad600640 1 --2- 172.21.15.144:0/2851471836 >> v2:172.21.15.32:3300/0 conn(0x7f0fa80ae440 0x7f0fa80b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:50.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.585+0000 7f0facc00640 1 -- 172.21.15.144:0/2851471836 >> v2:172.21.15.32:3300/0 conn(0x7f0fa80ae440 msgr2=0x7f0fa80b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:52:50.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.585+0000 7f0facc00640 1 --2- 172.21.15.144:0/2851471836 >> v2:172.21.15.32:3300/0 conn(0x7f0fa80ae440 0x7f0fa80b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.585+0000 7f0facc00640 1 -- 172.21.15.144:0/2851471836 >> v1:172.21.15.32:6789/0 conn(0x7f0fa806ba30 legacy=0x7f0fa80add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:52:50.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.585+0000 7f0facc00640 1 -- 172.21.15.144:0/2851471836 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa806c7d0 con 0x7f0fa80b57a0 2024-04-17T09:52:50.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.585+0000 7f0fad600640 1 --2- 172.21.15.144:0/2851471836 >> v2:172.21.15.32:3300/0 conn(0x7f0fa80ae440 0x7f0fa80b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:52:50.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.589+0000 7f0facc00640 1 --2- 172.21.15.144:0/2851471836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa80b57a0 0x7f0fa80aff80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c009870 tx=0x7f0f9c01c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b92bf9fb07bb931 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:50.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.589+0000 7f0fa7400640 1 -- 172.21.15.144:0/2851471836 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c0aa430 con 0x7f0fa80b57a0 2024-04-17T09:52:50.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.589+0000 7f0fa7400640 1 -- 172.21.15.144:0/2851471836 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f9c0af070 con 0x7f0fa80b57a0 2024-04-17T09:52:50.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.589+0000 7f0fa7400640 1 -- 172.21.15.144:0/2851471836 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c0aa7f0 con 0x7f0fa80b57a0 2024-04-17T09:52:50.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.589+0000 7f0faf600640 1 -- 172.21.15.144:0/2851471836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa80b57a0 msgr2=0x7f0fa80aff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:50.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.589+0000 7f0faf600640 1 --2- 172.21.15.144:0/2851471836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa80b57a0 0x7f0fa80aff80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c009870 tx=0x7f0f9c01c4f0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.589+0000 7f0faf600640 1 -- 172.21.15.144:0/2851471836 shutdown_connections 2024-04-17T09:52:50.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.589+0000 7f0faf600640 1 --2- 172.21.15.144:0/2851471836 >> v2:172.21.15.32:3300/0 conn(0x7f0fa80ae440 0x7f0fa80b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.589+0000 7f0faf600640 1 --2- 172.21.15.144:0/2851471836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa80b57a0 0x7f0fa80aff80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.589+0000 7f0faf600640 1 -- 172.21.15.144:0/2851471836 >> 172.21.15.144:0/2851471836 conn(0x7f0fa80d0d00 msgr2=0x7f0fa80d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:50.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.589+0000 7f0faf600640 1 -- 172.21.15.144:0/2851471836 shutdown_connections 2024-04-17T09:52:50.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.589+0000 7f0faf600640 1 -- 172.21.15.144:0/2851471836 wait complete. 2024-04-17T09:52:50.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0faf600640 1 Processor -- start 2024-04-17T09:52:50.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0faf600640 1 -- start start 2024-04-17T09:52:50.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0faf600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fa806ba30 0x7f0fa8145570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:50.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0faf600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa80ae440 0x7f0fa815b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:50.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0faf600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa80b57a0 0x7f0fa81facf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:50.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0faf600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa80b49a0 con 0x7f0fa806ba30 2024-04-17T09:52:50.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0faf600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa806be90 con 0x7f0fa80b57a0 2024-04-17T09:52:50.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0faf600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0fa80b4b20 con 0x7f0fa80ae440 2024-04-17T09:52:50.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0fa7e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa80ae440 0x7f0fa815b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:50.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0fad600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa80b57a0 0x7f0fa81facf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:50.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0fa7e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa80ae440 0x7f0fa815b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:60590/0 (socket says 172.21.15.144:60590) 2024-04-17T09:52:50.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0fa7e00640 1 -- 172.21.15.144:0/496324720 learned_addr learned my addr 172.21.15.144:0/496324720 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-17T09:52:50.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0facc00640 1 --2- 172.21.15.144:0/496324720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fa806ba30 0x7f0fa8145570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:50.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0fa7e00640 1 -- 172.21.15.144:0/496324720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa80b57a0 msgr2=0x7f0fa81facf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:50.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0fa7e00640 1 --2- 172.21.15.144:0/496324720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa80b57a0 0x7f0fa81facf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0fa7e00640 1 -- 172.21.15.144:0/496324720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fa806ba30 msgr2=0x7f0fa8145570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:50.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0fa7e00640 1 --2- 172.21.15.144:0/496324720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fa806ba30 0x7f0fa8145570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0fa7e00640 1 -- 172.21.15.144:0/496324720 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa815bb20 con 0x7f0fa80ae440 2024-04-17T09:52:50.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0fa7e00640 1 --2- 172.21.15.144:0/496324720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa80ae440 0x7f0fa815b490 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0f9400c480 tx=0x7f0f9400c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:50.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0fa4c00640 1 -- 172.21.15.144:0/496324720 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f94012830 con 0x7f0fa80ae440 2024-04-17T09:52:50.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0fa4c00640 1 -- 172.21.15.144:0/496324720 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f94017070 con 0x7f0fa80ae440 2024-04-17T09:52:50.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0fa4c00640 1 -- 172.21.15.144:0/496324720 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f94012c30 con 0x7f0fa80ae440 2024-04-17T09:52:50.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0faf600640 1 -- 172.21.15.144:0/496324720 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa815be00 con 0x7f0fa80ae440 2024-04-17T09:52:50.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0faf600640 1 -- 172.21.15.144:0/496324720 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa81fa4f0 con 0x7f0fa80ae440 2024-04-17T09:52:50.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.593+0000 7f0fa4c00640 1 -- 172.21.15.144:0/496324720 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0f9401c460 con 0x7f0fa80ae440 2024-04-17T09:52:50.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.597+0000 7f0fa4c00640 1 -- 172.21.15.144:0/496324720 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f0f94012dd0 con 0x7f0fa80ae440 2024-04-17T09:52:50.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.597+0000 7f0faf600640 1 -- 172.21.15.144:0/496324720 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa80691a0 con 0x7f0fa80ae440 2024-04-17T09:52:50.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.597+0000 7f0fa4c00640 1 -- 172.21.15.144:0/496324720 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f0f9401b730 con 0x7f0fa80ae440 2024-04-17T09:52:50.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.665+0000 7f0faf600640 1 -- 172.21.15.144:0/496324720 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "bc3f5d1f-f576-4fa5-85fe-c957d278349f", "id": 7} v 0) -- 0x7f0fa813ee00 con 0x7f0fa80ae440 2024-04-17T09:52:50.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.669+0000 7f0fa4c00640 1 -- 172.21.15.144:0/496324720 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bc3f5d1f-f576-4fa5-85fe-c957d278349f", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f0f9401b9e0 con 0x7f0fa80ae440 2024-04-17T09:52:50.673 INFO:teuthology.orchestra.run.smithi144.stdout:7 2024-04-17T09:52:50.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.673+0000 7f0faf600640 1 -- 172.21.15.144:0/496324720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa80ae440 msgr2=0x7f0fa815b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:50.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.673+0000 7f0faf600640 1 --2- 172.21.15.144:0/496324720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa80ae440 0x7f0fa815b490 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0f9400c480 tx=0x7f0f9400c940 comp rx=0 tx=0).stop 2024-04-17T09:52:50.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.673+0000 7f0faf600640 1 -- 172.21.15.144:0/496324720 shutdown_connections 2024-04-17T09:52:50.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.673+0000 7f0faf600640 1 --2- 172.21.15.144:0/496324720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa80ae440 0x7f0fa815b490 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.673+0000 7f0faf600640 1 --2- 172.21.15.144:0/496324720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa80b57a0 0x7f0fa81facf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.673+0000 7f0faf600640 1 --2- 172.21.15.144:0/496324720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fa806ba30 0x7f0fa8145570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.673+0000 7f0faf600640 1 -- 172.21.15.144:0/496324720 >> 172.21.15.144:0/496324720 conn(0x7f0fa80d0d00 msgr2=0x7f0fa8142850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:50.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.673+0000 7f0faf600640 1 -- 172.21.15.144:0/496324720 shutdown_connections 2024-04-17T09:52:50.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:52:50.673+0000 7f0faf600640 1 -- 172.21.15.144:0/496324720 wait complete. 2024-04-17T09:52:50.693 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-17T09:52:50.693 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-17T09:52:50.697 INFO:tasks.ceph.osd.0:Started 2024-04-17T09:52:50.697 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-17T09:52:50.697 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-17T09:52:50.700 INFO:tasks.ceph.osd.1:Started 2024-04-17T09:52:50.700 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-17T09:52:50.700 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-17T09:52:50.703 INFO:tasks.ceph.osd.2:Started 2024-04-17T09:52:50.703 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-17T09:52:50.703 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-17T09:52:50.707 INFO:tasks.ceph.osd.3:Started 2024-04-17T09:52:50.708 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-17T09:52:50.708 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-17T09:52:50.711 INFO:tasks.ceph.osd.4:Started 2024-04-17T09:52:50.711 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-17T09:52:50.713 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-17T09:52:50.716 INFO:tasks.ceph.osd.5:Started 2024-04-17T09:52:50.716 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-17T09:52:50.716 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-17T09:52:50.719 INFO:tasks.ceph.osd.6:Started 2024-04-17T09:52:50.719 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-17T09:52:50.719 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-17T09:52:50.723 INFO:tasks.ceph.osd.7:Started 2024-04-17T09:52:50.723 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:52:50.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.912+0000 7fa76d000640 1 Processor -- start 2024-04-17T09:52:50.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.912+0000 7fa76d000640 1 -- start start 2024-04-17T09:52:50.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.912+0000 7fa76d000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa76815eb30 0x7fa76809b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:50.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.912+0000 7fa76d000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa76809b100 0x7fa7680ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:50.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.912+0000 7fa76d000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa76809bcb0 con 0x7fa7680d0890 2024-04-17T09:52:50.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.912+0000 7fa76d000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa76806dd60 con 0x7fa76809b100 2024-04-17T09:52:50.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.912+0000 7fa76d000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa76815ef10 con 0x7fa76815eb30 2024-04-17T09:52:50.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.912+0000 7fa766800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa76809b100 0x7fa7680ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:50.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.916+0000 7fa766800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa76809b100 0x7fa7680ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55770/0 (socket says 172.21.15.32:55770) 2024-04-17T09:52:50.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.916+0000 7fa766800640 1 -- 172.21.15.32:0/1025957105 learned_addr learned my addr 172.21.15.32:0/1025957105 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:52:50.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.916+0000 7fa765e00640 1 --2- 172.21.15.32:0/1025957105 >> v2:172.21.15.144:3300/0 conn(0x7fa76815eb30 0x7fa76809b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:50.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.916+0000 7fa766800640 1 -- 172.21.15.32:0/1025957105 >> v2:172.21.15.144:3300/0 conn(0x7fa76815eb30 msgr2=0x7fa76809b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:52:50.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.916+0000 7fa766800640 1 --2- 172.21.15.32:0/1025957105 >> v2:172.21.15.144:3300/0 conn(0x7fa76815eb30 0x7fa76809b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.916+0000 7fa766800640 1 -- 172.21.15.32:0/1025957105 >> v1:172.21.15.144:6789/0 conn(0x7fa7680d0890 legacy=0x7fa76809a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:52:50.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.916+0000 7fa766800640 1 -- 172.21.15.32:0/1025957105 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7680d0db0 con 0x7fa76809b100 2024-04-17T09:52:50.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.916+0000 7fa766800640 1 --2- 172.21.15.32:0/1025957105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa76809b100 0x7fa7680ccb30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa768073350 tx=0x7fa76001ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68e8e3e58023414 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:50.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.916+0000 7fa765400640 1 -- 172.21.15.32:0/1025957105 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7600b5070 con 0x7fa76809b100 2024-04-17T09:52:50.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.916+0000 7fa765400640 1 -- 172.21.15.32:0/1025957105 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7600a89d0 con 0x7fa76809b100 2024-04-17T09:52:50.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.916+0000 7fa765400640 1 -- 172.21.15.32:0/1025957105 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7600a8da0 con 0x7fa76809b100 2024-04-17T09:52:50.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa76d000640 1 -- 172.21.15.32:0/1025957105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa76809b100 msgr2=0x7fa7680ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:50.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa76d000640 1 --2- 172.21.15.32:0/1025957105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa76809b100 0x7fa7680ccb30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa768073350 tx=0x7fa76001ecd0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa76d000640 1 -- 172.21.15.32:0/1025957105 shutdown_connections 2024-04-17T09:52:50.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa76d000640 1 --2- 172.21.15.32:0/1025957105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa76809b100 0x7fa7680ccb30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa76d000640 1 --2- 172.21.15.32:0/1025957105 >> v2:172.21.15.144:3300/0 conn(0x7fa76815eb30 0x7fa76809b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa76d000640 1 -- 172.21.15.32:0/1025957105 >> 172.21.15.32:0/1025957105 conn(0x7fa7680b3fd0 msgr2=0x7fa7680b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:50.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa76d000640 1 -- 172.21.15.32:0/1025957105 shutdown_connections 2024-04-17T09:52:50.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa76d000640 1 -- 172.21.15.32:0/1025957105 wait complete. 2024-04-17T09:52:50.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa76d000640 1 Processor -- start 2024-04-17T09:52:50.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa76d000640 1 -- start start 2024-04-17T09:52:50.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa76d000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa76809b100 0x7fa76809dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:50.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa76d000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7680d0890 0x7fa76809f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:50.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa76d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa76815eb30 0x7fa76809fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:50.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa76d000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa7680d1650 con 0x7fa76815eb30 2024-04-17T09:52:50.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa76d000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa7680d1370 con 0x7fa76809b100 2024-04-17T09:52:50.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa76d000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa7680d1930 con 0x7fa7680d0890 2024-04-17T09:52:50.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa766800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa76815eb30 0x7fa76809fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:50.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa766800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa76815eb30 0x7fa76809fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55776/0 (socket says 172.21.15.32:55776) 2024-04-17T09:52:50.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa766800640 1 -- 172.21.15.32:0/1176238446 learned_addr learned my addr 172.21.15.32:0/1176238446 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:52:50.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa75d400640 1 --2- 172.21.15.32:0/1176238446 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7680d0890 0x7fa76809f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:50.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa765e00640 1 --2- 172.21.15.32:0/1176238446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa76809b100 0x7fa76809dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:50.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa766800640 1 -- 172.21.15.32:0/1176238446 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7680d0890 msgr2=0x7fa76809f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:50.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa766800640 1 --2- 172.21.15.32:0/1176238446 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7680d0890 0x7fa76809f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa766800640 1 -- 172.21.15.32:0/1176238446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa76809b100 msgr2=0x7fa76809dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:50.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa766800640 1 --2- 172.21.15.32:0/1176238446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa76809b100 0x7fa76809dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.920+0000 7fa766800640 1 -- 172.21.15.32:0/1176238446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76809e1a0 con 0x7fa76815eb30 2024-04-17T09:52:50.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.924+0000 7fa766800640 1 --2- 172.21.15.32:0/1176238446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa76815eb30 0x7fa76809fb40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa7600b3040 tx=0x7fa76001ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:50.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.924+0000 7fa75ea00640 1 -- 172.21.15.32:0/1176238446 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7600c0070 con 0x7fa76815eb30 2024-04-17T09:52:50.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.924+0000 7fa75ea00640 1 -- 172.21.15.32:0/1176238446 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7600b09f0 con 0x7fa76815eb30 2024-04-17T09:52:50.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.924+0000 7fa75ea00640 1 -- 172.21.15.32:0/1176238446 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7600b5070 con 0x7fa76815eb30 2024-04-17T09:52:50.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.924+0000 7fa76d000640 1 -- 172.21.15.32:0/1176238446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7680a0180 con 0x7fa76815eb30 2024-04-17T09:52:50.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.924+0000 7fa76d000640 1 -- 172.21.15.32:0/1176238446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7681504c0 con 0x7fa76815eb30 2024-04-17T09:52:50.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.924+0000 7fa75ea00640 1 -- 172.21.15.32:0/1176238446 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa7600bb440 con 0x7fa76815eb30 2024-04-17T09:52:50.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.924+0000 7fa75ea00640 1 -- 172.21.15.32:0/1176238446 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fa7600c43f0 con 0x7fa76815eb30 2024-04-17T09:52:50.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.924+0000 7fa76d000640 1 -- 172.21.15.32:0/1176238446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa768068a90 con 0x7fa76815eb30 2024-04-17T09:52:50.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.924+0000 7fa75ea00640 1 -- 172.21.15.32:0/1176238446 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fa7600ca080 con 0x7fa76815eb30 2024-04-17T09:52:50.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.984+0000 7fa76d000640 1 -- 172.21.15.32:0/1176238446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7680ce9a0 con 0x7fa76815eb30 2024-04-17T09:52:50.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.984+0000 7fa75ea00640 1 -- 172.21.15.32:0/1176238446 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fa76809e1a0 con 0x7fa76815eb30 2024-04-17T09:52:50.987 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:52:50.987 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":10,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:52:50.671360+0000","last_up_change":"0.000000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:52:50.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.984+0000 7fa76d000640 1 -- 172.21.15.32:0/1176238446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa76815eb30 msgr2=0x7fa76809fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:50.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.984+0000 7fa76d000640 1 --2- 172.21.15.32:0/1176238446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa76815eb30 0x7fa76809fb40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa7600b3040 tx=0x7fa76001ba70 comp rx=0 tx=0).stop 2024-04-17T09:52:50.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.988+0000 7fa76d000640 1 -- 172.21.15.32:0/1176238446 shutdown_connections 2024-04-17T09:52:50.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.988+0000 7fa76d000640 1 --2- 172.21.15.32:0/1176238446 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7680d0890 0x7fa76809f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.988+0000 7fa76d000640 1 --2- 172.21.15.32:0/1176238446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa76815eb30 0x7fa76809fb40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.988+0000 7fa76d000640 1 --2- 172.21.15.32:0/1176238446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa76809b100 0x7fa76809dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:50.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.988+0000 7fa76d000640 1 -- 172.21.15.32:0/1176238446 >> 172.21.15.32:0/1176238446 conn(0x7fa7680b3fd0 msgr2=0x7fa76815d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:50.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.988+0000 7fa76d000640 1 -- 172.21.15.32:0/1176238446 shutdown_connections 2024-04-17T09:52:50.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:52:50.988+0000 7fa76d000640 1 -- 172.21.15.32:0/1176238446 wait complete. 2024-04-17T09:52:51.006 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-17T09:52:51.006 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-17T09:52:51.098 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-17T09:52:51.098 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-17T09:52:51.098 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2024-04-17T09:52:51.140 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:51.140+0000 7f63c8272180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T09:52:51.193 INFO:tasks.ceph.mgr.x.smithi144.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-17T09:52:51.194 INFO:tasks.ceph.mgr.x.smithi144.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-17T09:52:51.194 INFO:tasks.ceph.mgr.x.smithi144.stderr: from numpy import show_config as show_numpy_config 2024-04-17T09:52:51.243 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-17T09:52:51.244+0000 7f32a8109180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T09:52:51.296 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:51.296+0000 7f63c8272180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T09:52:51.347 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-04-17T09:52:51.344+0000 7f6ba00957c0 -1 Falling back to public interface 2024-04-17T09:52:51.369 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-04-17T09:52:51.368+0000 7f726a86b7c0 -1 Falling back to public interface 2024-04-17T09:52:51.379 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-04-17T09:52:51.376+0000 7ff10f3207c0 -1 Falling back to public interface 2024-04-17T09:52:51.379 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-04-17T09:52:51.376+0000 7f815302f7c0 -1 Falling back to public interface 2024-04-17T09:52:51.393 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-17T09:52:51.388+0000 7f32a8109180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T09:52:51.395 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-04-17T09:52:51.392+0000 7f173e4807c0 -1 Falling back to public interface 2024-04-17T09:52:51.417 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-04-17T09:52:51.416+0000 7f06b842b7c0 -1 Falling back to public interface 2024-04-17T09:52:51.418 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-04-17T09:52:51.416+0000 7faae10047c0 -1 Falling back to public interface 2024-04-17T09:52:51.418 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-04-17T09:52:51.416+0000 7faadfa00640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6808/0: (98) Address already in use 2024-04-17T09:52:51.453 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-04-17T09:52:51.452+0000 7f47e22d97c0 -1 Falling back to public interface 2024-04-17T09:52:51.492 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:51.488+0000 7f63c8272180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T09:52:51.569 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:51.568+0000 7f63c8272180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T09:52:51.578 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-17T09:52:51.572+0000 7f32a8109180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T09:52:51.651 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-17T09:52:51.652+0000 7f32a8109180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T09:52:51.731 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:51.728+0000 7f63c8272180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T09:52:51.802 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-17T09:52:51.804+0000 7f32a8109180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T09:52:51.812 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-17T09:52:51.944 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.940+0000 7f78a4a00640 1 Processor -- start 2024-04-17T09:52:51.944 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.940+0000 7f78a4a00640 1 -- start start 2024-04-17T09:52:51.946 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.944+0000 7f78a4a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f78a015c770 0x7f78a00d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:51.946 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.944+0000 7f78a4a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78a00cdb90 0x7f78a00cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:51.946 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.944+0000 7f78a4a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78a015cb50 con 0x7f78a0160270 2024-04-17T09:52:51.946 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.944+0000 7f78a4a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f78a0161050 con 0x7f78a015c770 2024-04-17T09:52:51.946 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.944+0000 7f78a4a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f78a00d00e0 con 0x7f78a00cdb90 2024-04-17T09:52:51.946 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.944+0000 7f789de00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f78a015c770 0x7f78a00d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:51.946 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.944+0000 7f789e800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f78a0160270 0x7f78a006fdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51364/0 (socket says 172.21.15.32:51364) 2024-04-17T09:52:51.946 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.944+0000 7f789e800640 1 -- 172.21.15.32:0/39650830 learned_addr learned my addr 172.21.15.32:0/39650830 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:52:51.946 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.944+0000 7f7897e00640 1 --2- 172.21.15.32:0/39650830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78a00cdb90 0x7f78a00cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:51.946 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.944+0000 7f789de00640 1 -- 172.21.15.32:0/39650830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78a00cdb90 msgr2=0x7f78a00cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:52:51.946 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.944+0000 7f789de00640 1 --2- 172.21.15.32:0/39650830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78a00cdb90 0x7f78a00cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:51.947 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.944+0000 7f789de00640 1 -- 172.21.15.32:0/39650830 >> v1:172.21.15.32:6789/0 conn(0x7f78a0160270 legacy=0x7f78a006fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:51.947 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.944+0000 7f789d400640 1 -- 172.21.15.32:0/39650830 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3285692521 0 0) 0x7f78a015cb50 con 0x7f78a0160270 2024-04-17T09:52:51.947 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.944+0000 7f789de00640 1 -- 172.21.15.32:0/39650830 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a00d03c0 con 0x7f78a015c770 2024-04-17T09:52:51.951 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.948+0000 7f789de00640 1 --2- 172.21.15.32:0/39650830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a015c770 0x7f78a00d1180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f78900097c0 tx=0x7f789001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba1946cabac2049 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:51.951 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.948+0000 7f789d400640 1 -- 172.21.15.32:0/39650830 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78900ab640 con 0x7f78a015c770 2024-04-17T09:52:51.951 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.948+0000 7f789d400640 1 -- 172.21.15.32:0/39650830 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78900b0070 con 0x7f78a015c770 2024-04-17T09:52:51.952 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.948+0000 7f789d400640 1 -- 172.21.15.32:0/39650830 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78900aba00 con 0x7f78a015c770 2024-04-17T09:52:51.952 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f78a4a00640 1 -- 172.21.15.32:0/39650830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a015c770 msgr2=0x7f78a00d1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:51.952 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f78a4a00640 1 --2- 172.21.15.32:0/39650830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a015c770 0x7f78a00d1180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f78900097c0 tx=0x7f789001c470 comp rx=0 tx=0).stop 2024-04-17T09:52:51.952 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f78a4a00640 1 -- 172.21.15.32:0/39650830 shutdown_connections 2024-04-17T09:52:51.953 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f78a4a00640 1 --2- 172.21.15.32:0/39650830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78a00cdb90 0x7f78a00cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:51.953 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f78a4a00640 1 --2- 172.21.15.32:0/39650830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a015c770 0x7f78a00d1180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:51.953 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f78a4a00640 1 -- 172.21.15.32:0/39650830 >> 172.21.15.32:0/39650830 conn(0x7f78a00d2500 msgr2=0x7f78a00d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:51.953 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f78a4a00640 1 -- 172.21.15.32:0/39650830 shutdown_connections 2024-04-17T09:52:51.953 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f78a4a00640 1 -- 172.21.15.32:0/39650830 wait complete. 2024-04-17T09:52:51.954 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f78a4a00640 1 Processor -- start 2024-04-17T09:52:51.954 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f78a4a00640 1 -- start start 2024-04-17T09:52:51.956 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f78a4a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78a00cdb90 0x7f78a015a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:51.956 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f78a4a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a015c770 0x7f78a0154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:51.956 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f78a4a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78a0160270 0x7f78a015b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:51.956 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f78a4a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f78a01695b0 con 0x7f78a015c770 2024-04-17T09:52:51.956 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f78a4a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f78a0160ad0 con 0x7f78a0160270 2024-04-17T09:52:51.956 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f78a4a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f78a0169730 con 0x7f78a00cdb90 2024-04-17T09:52:51.956 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f7897e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a015c770 0x7f78a0154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:51.956 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f7897e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a015c770 0x7f78a0154950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55796/0 (socket says 172.21.15.32:55796) 2024-04-17T09:52:51.956 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f7897e00640 1 -- 172.21.15.32:0/3530073480 learned_addr learned my addr 172.21.15.32:0/3530073480 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:52:51.956 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f789de00640 1 --2- 172.21.15.32:0/3530073480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78a00cdb90 0x7f78a015a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:51.957 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f789e800640 1 --2- 172.21.15.32:0/3530073480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78a0160270 0x7f78a015b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:51.957 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f7897e00640 1 -- 172.21.15.32:0/3530073480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78a00cdb90 msgr2=0x7f78a015a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:51.957 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f7897e00640 1 --2- 172.21.15.32:0/3530073480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78a00cdb90 0x7f78a015a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:51.957 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f7897e00640 1 -- 172.21.15.32:0/3530073480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78a0160270 msgr2=0x7f78a015b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:51.957 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f7897e00640 1 --2- 172.21.15.32:0/3530073480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78a0160270 0x7f78a015b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:51.957 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f7897e00640 1 -- 172.21.15.32:0/3530073480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a0154e90 con 0x7f78a015c770 2024-04-17T09:52:51.957 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f7897e00640 1 --2- 172.21.15.32:0/3530073480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a015c770 0x7f78a0154950 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f788800e870 tx=0x7f788800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:51.957 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.952+0000 7f7896000640 1 -- 172.21.15.32:0/3530073480 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7888017070 con 0x7f78a015c770 2024-04-17T09:52:51.957 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.956+0000 7f7896000640 1 -- 172.21.15.32:0/3530073480 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f788800c940 con 0x7f78a015c770 2024-04-17T09:52:51.957 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.956+0000 7f7896000640 1 -- 172.21.15.32:0/3530073480 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788800ccf0 con 0x7f78a015c770 2024-04-17T09:52:51.957 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.956+0000 7f78a4a00640 1 -- 172.21.15.32:0/3530073480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a015a5f0 con 0x7f78a015c770 2024-04-17T09:52:51.957 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.956+0000 7f78a4a00640 1 -- 172.21.15.32:0/3530073480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a015abb0 con 0x7f78a015c770 2024-04-17T09:52:51.958 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.956+0000 7f7896000640 1 -- 172.21.15.32:0/3530073480 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f788801a070 con 0x7f78a015c770 2024-04-17T09:52:51.958 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.956+0000 7f7896000640 1 -- 172.21.15.32:0/3530073480 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f7888006020 con 0x7f78a015c770 2024-04-17T09:52:51.958 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.956+0000 7f78a4a00640 1 -- 172.21.15.32:0/3530073480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78a00a0470 con 0x7f78a015c770 2024-04-17T09:52:51.959 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:51.956+0000 7f7896000640 1 -- 172.21.15.32:0/3530073480 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f7888006480 con 0x7f78a015c770 2024-04-17T09:52:52.018 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:52.016+0000 7f78a4a00640 1 -- 172.21.15.32:0/3530073480 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78a00ce980 con 0x7f78a015c770 2024-04-17T09:52:52.019 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:52.016+0000 7f7896000640 1 -- 172.21.15.32:0/3530073480 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f7888006660 con 0x7f78a015c770 2024-04-17T09:52:52.019 INFO:teuthology.misc.health.smithi032.stdout: 2024-04-17T09:52:52.019 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":10,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:52:50.671360+0000","last_up_change":"0.000000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:52:52.020 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:52.016+0000 7f78a4a00640 1 -- 172.21.15.32:0/3530073480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a015c770 msgr2=0x7f78a0154950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:52.020 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:52.016+0000 7f78a4a00640 1 --2- 172.21.15.32:0/3530073480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a015c770 0x7f78a0154950 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f788800e870 tx=0x7f788800ec70 comp rx=0 tx=0).stop 2024-04-17T09:52:52.020 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:52.016+0000 7f78a4a00640 1 -- 172.21.15.32:0/3530073480 shutdown_connections 2024-04-17T09:52:52.020 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:52.016+0000 7f78a4a00640 1 --2- 172.21.15.32:0/3530073480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78a0160270 0x7f78a015b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:52.020 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:52.016+0000 7f78a4a00640 1 --2- 172.21.15.32:0/3530073480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a015c770 0x7f78a0154950 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:52.021 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:52.016+0000 7f78a4a00640 1 --2- 172.21.15.32:0/3530073480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78a00cdb90 0x7f78a015a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:52.021 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:52.020+0000 7f78a4a00640 1 -- 172.21.15.32:0/3530073480 >> 172.21.15.32:0/3530073480 conn(0x7f78a00d2500 msgr2=0x7f78a006fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:52.021 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:52.020+0000 7f78a4a00640 1 -- 172.21.15.32:0/3530073480 shutdown_connections 2024-04-17T09:52:52.021 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:52.020+0000 7f78a4a00640 1 -- 172.21.15.32:0/3530073480 wait complete. 2024-04-17T09:52:52.037 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-17T09:52:52.450 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:52.448+0000 7f63c8272180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T09:52:52.484 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-17T09:52:52.484+0000 7f32a8109180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T09:52:52.541 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:52.540+0000 7f63c8272180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T09:52:52.572 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-17T09:52:52.572+0000 7f32a8109180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T09:52:52.755 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:52.752+0000 7f63c8272180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T09:52:52.776 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-17T09:52:52.776+0000 7f32a8109180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T09:52:52.844 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:52.844+0000 7f63c8272180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T09:52:52.862 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-17T09:52:52.860+0000 7f32a8109180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T09:52:53.192 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:53.192+0000 7f63c8272180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T09:52:53.197 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-17T09:52:53.196+0000 7f32a8109180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T09:52:53.266 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:53.264+0000 7f63c8272180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T09:52:53.269 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-17T09:52:53.268+0000 7f32a8109180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T09:52:54.287 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:54.284+0000 7f636e400640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-17T09:52:54.289 INFO:tasks.ceph.mgr.y.smithi032.stderr:2024-04-17T09:52:54.288+0000 7f636e400640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-17T09:52:55.236 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-04-17T09:52:55.236+0000 7f726a86b7c0 -1 osd.0 0 log_to_monitors true 2024-04-17T09:52:55.251 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-04-17T09:52:55.252+0000 7f815302f7c0 -1 osd.4 0 log_to_monitors true 2024-04-17T09:52:55.322 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-04-17T09:52:55.320+0000 7ff10f3207c0 -1 osd.6 0 log_to_monitors true 2024-04-17T09:52:55.354 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-04-17T09:52:55.352+0000 7f06b842b7c0 -1 osd.1 0 log_to_monitors true 2024-04-17T09:52:55.426 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-04-17T09:52:55.424+0000 7faae10047c0 -1 osd.2 0 log_to_monitors true 2024-04-17T09:52:55.450 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-04-17T09:52:55.448+0000 7f47e22d97c0 -1 osd.3 0 log_to_monitors true 2024-04-17T09:52:55.541 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-04-17T09:52:55.540+0000 7f6ba00957c0 -1 osd.5 0 log_to_monitors true 2024-04-17T09:52:55.615 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-04-17T09:52:55.616+0000 7f173e4807c0 -1 osd.7 0 log_to_monitors true 2024-04-17T09:52:57.303 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-04-17T09:52:57.300+0000 7f7265800640 -1 osd.0 0 waiting for initial osdmap 2024-04-17T09:52:57.345 INFO:tasks.ceph.osd.0.smithi032.stderr:2024-04-17T09:52:57.344+0000 7f725b800640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T09:52:57.389 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-04-17T09:52:57.388+0000 7f814de00640 -1 osd.4 0 waiting for initial osdmap 2024-04-17T09:52:57.447 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-04-17T09:52:57.448+0000 7f8143e00640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T09:52:58.675 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-04-17T09:52:58.671+0000 7faadbe00640 -1 osd.2 0 waiting for initial osdmap 2024-04-17T09:52:58.694 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-04-17T09:52:58.691+0000 7f06b2800640 -1 osd.1 0 waiting for initial osdmap 2024-04-17T09:52:58.701 INFO:tasks.ceph.osd.2.smithi032.stderr:2024-04-17T09:52:58.699+0000 7faad1e00640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T09:52:58.713 INFO:tasks.ceph.osd.1.smithi032.stderr:2024-04-17T09:52:58.711+0000 7f06a8800640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T09:52:58.728 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-04-17T09:52:58.727+0000 7f47dd200640 -1 osd.3 0 waiting for initial osdmap 2024-04-17T09:52:58.753 INFO:tasks.ceph.osd.3.smithi032.stderr:2024-04-17T09:52:58.751+0000 7f47d3200640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T09:52:58.844 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-17T09:52:58.973 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.971+0000 7f2823400640 1 Processor -- start 2024-04-17T09:52:58.973 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.971+0000 7f2823400640 1 -- start start 2024-04-17T09:52:58.975 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.971+0000 7f2823400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f281c0717d0 0x7f281c0b4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:58.975 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.971+0000 7f2823400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f281c0d0870 0x7f281c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:58.975 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.971+0000 7f2823400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f281c06cde0 0x7f281c06da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:58.975 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.971+0000 7f2823400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f281c0d1920 con 0x7f281c0717d0 2024-04-17T09:52:58.975 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.971+0000 7f2823400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f281c0d0d10 con 0x7f281c06cde0 2024-04-17T09:52:58.975 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.971+0000 7f2823400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f281c0d0ff0 con 0x7f281c0d0870 2024-04-17T09:52:58.975 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.971+0000 7f2820a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f281c0717d0 0x7f281c0b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:58.975 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.971+0000 7f2820a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f281c0717d0 0x7f281c0b4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55906/0 (socket says 172.21.15.32:55906) 2024-04-17T09:52:58.975 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.971+0000 7f2820a00640 1 -- 172.21.15.32:0/495166736 learned_addr learned my addr 172.21.15.32:0/495166736 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:52:58.975 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.971+0000 7f281be00640 1 --2- 172.21.15.32:0/495166736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f281c0d0870 0x7f281c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:58.976 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.971+0000 7f2821400640 1 --2- 172.21.15.32:0/495166736 >> v2:172.21.15.144:3300/0 conn(0x7f281c06cde0 0x7f281c06da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:58.976 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.971+0000 7f2820a00640 1 -- 172.21.15.32:0/495166736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f281c0d0870 msgr2=0x7f281c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:52:58.976 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.971+0000 7f2820a00640 1 --2- 172.21.15.32:0/495166736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f281c0d0870 0x7f281c06d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:58.976 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.971+0000 7f2820a00640 1 -- 172.21.15.32:0/495166736 >> v2:172.21.15.144:3300/0 conn(0x7f281c06cde0 msgr2=0x7f281c06da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:52:58.976 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.971+0000 7f2820a00640 1 --2- 172.21.15.32:0/495166736 >> v2:172.21.15.144:3300/0 conn(0x7f281c06cde0 0x7f281c06da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:58.976 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.971+0000 7f2820a00640 1 -- 172.21.15.32:0/495166736 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281c0d12d0 con 0x7f281c0717d0 2024-04-17T09:52:58.976 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.971+0000 7f281be00640 1 --2- 172.21.15.32:0/495166736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f281c0d0870 0x7f281c06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:52:58.978 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.975+0000 7f2820a00640 1 --2- 172.21.15.32:0/495166736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281c0717d0 0x7f281c0b4960 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f281c0cef30 tx=0x7f281001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a57b6071f81e859 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:58.979 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.975+0000 7f281b400640 1 -- 172.21.15.32:0/495166736 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28100aa660 con 0x7f281c0717d0 2024-04-17T09:52:58.979 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.975+0000 7f281b400640 1 -- 172.21.15.32:0/495166736 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28100af070 con 0x7f281c0717d0 2024-04-17T09:52:58.979 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.975+0000 7f281b400640 1 -- 172.21.15.32:0/495166736 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28100aaa20 con 0x7f281c0717d0 2024-04-17T09:52:58.980 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.975+0000 7f2823400640 1 -- 172.21.15.32:0/495166736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281c0717d0 msgr2=0x7f281c0b4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:58.980 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.975+0000 7f2823400640 1 --2- 172.21.15.32:0/495166736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281c0717d0 0x7f281c0b4960 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f281c0cef30 tx=0x7f281001c520 comp rx=0 tx=0).stop 2024-04-17T09:52:58.980 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.975+0000 7f2823400640 1 -- 172.21.15.32:0/495166736 shutdown_connections 2024-04-17T09:52:58.980 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.975+0000 7f2823400640 1 --2- 172.21.15.32:0/495166736 >> v2:172.21.15.144:3300/0 conn(0x7f281c06cde0 0x7f281c06da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:58.980 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.975+0000 7f2823400640 1 --2- 172.21.15.32:0/495166736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f281c0d0870 0x7f281c06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:58.980 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.975+0000 7f2823400640 1 --2- 172.21.15.32:0/495166736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281c0717d0 0x7f281c0b4960 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:58.980 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.975+0000 7f2823400640 1 -- 172.21.15.32:0/495166736 >> 172.21.15.32:0/495166736 conn(0x7f281c0d2500 msgr2=0x7f281c0d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:58.980 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.975+0000 7f2823400640 1 -- 172.21.15.32:0/495166736 shutdown_connections 2024-04-17T09:52:58.980 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.975+0000 7f2823400640 1 -- 172.21.15.32:0/495166736 wait complete. 2024-04-17T09:52:58.981 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f2823400640 1 Processor -- start 2024-04-17T09:52:58.981 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f2823400640 1 -- start start 2024-04-17T09:52:58.983 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f2823400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f281c06cde0 0x7f281c1556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:58.983 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f2823400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281c0717d0 0x7f281c15a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:58.983 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f2823400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f281c0d0870 0x7f281c15aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:58.983 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f2823400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f281c15ed10 con 0x7f281c0717d0 2024-04-17T09:52:58.984 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f2823400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f281c15eb90 con 0x7f281c06cde0 2024-04-17T09:52:58.984 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f2823400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f281c15ee90 con 0x7f281c0d0870 2024-04-17T09:52:58.984 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f281be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281c0717d0 0x7f281c15a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:58.984 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f281be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281c0717d0 0x7f281c15a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55922/0 (socket says 172.21.15.32:55922) 2024-04-17T09:52:58.984 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f281be00640 1 -- 172.21.15.32:0/2581738748 learned_addr learned my addr 172.21.15.32:0/2581738748 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:52:58.984 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f2820a00640 1 --2- 172.21.15.32:0/2581738748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f281c06cde0 0x7f281c1556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:58.984 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f2821400640 1 --2- 172.21.15.32:0/2581738748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f281c0d0870 0x7f281c15aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:58.984 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f281be00640 1 -- 172.21.15.32:0/2581738748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f281c0d0870 msgr2=0x7f281c15aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:58.984 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f281be00640 1 --2- 172.21.15.32:0/2581738748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f281c0d0870 0x7f281c15aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:58.984 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f281be00640 1 -- 172.21.15.32:0/2581738748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f281c06cde0 msgr2=0x7f281c1556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:58.984 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f281be00640 1 --2- 172.21.15.32:0/2581738748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f281c06cde0 0x7f281c1556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:58.984 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f281be00640 1 -- 172.21.15.32:0/2581738748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281c15b510 con 0x7f281c0717d0 2024-04-17T09:52:58.984 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f281be00640 1 --2- 172.21.15.32:0/2581738748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281c0717d0 0x7f281c15a9a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f280800e810 tx=0x7f280800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:58.984 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f2818c00640 1 -- 172.21.15.32:0/2581738748 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2808017070 con 0x7f281c0717d0 2024-04-17T09:52:58.985 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f2818c00640 1 -- 172.21.15.32:0/2581738748 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f280800c8d0 con 0x7f281c0717d0 2024-04-17T09:52:58.985 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f2818c00640 1 -- 172.21.15.32:0/2581738748 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280800f040 con 0x7f281c0717d0 2024-04-17T09:52:58.985 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f2823400640 1 -- 172.21.15.32:0/2581738748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281c15b7f0 con 0x7f281c0717d0 2024-04-17T09:52:58.985 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f2823400640 1 -- 172.21.15.32:0/2581738748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f281c15bdb0 con 0x7f281c0717d0 2024-04-17T09:52:58.985 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.979+0000 7f2823400640 1 -- 172.21.15.32:0/2581738748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f281c0a0470 con 0x7f281c0717d0 2024-04-17T09:52:58.985 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.983+0000 7f2818c00640 1 -- 172.21.15.32:0/2581738748 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) ==== 87931+0+0 (secure 0 0 0) 0x7f280801a070 con 0x7f281c0717d0 2024-04-17T09:52:58.987 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.983+0000 7f2818c00640 1 --2- 172.21.15.32:0/2581738748 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f27fc0682f0 0x7f27fc06a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:52:58.987 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.983+0000 7f2818c00640 1 -- 172.21.15.32:0/2581738748 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2791+0+0 (secure 0 0 0) 0x7f28080859e0 con 0x7f281c0717d0 2024-04-17T09:52:58.987 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.983+0000 7f2820a00640 1 --2- 172.21.15.32:0/2581738748 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f27fc0682f0 0x7f27fc06a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:52:58.987 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.983+0000 7f2820a00640 1 --2- 172.21.15.32:0/2581738748 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f27fc0682f0 0x7f27fc06a7b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f28100ae000 tx=0x7f28100a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:52:58.992 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:58.987+0000 7f2818c00640 1 -- 172.21.15.32:0/2581738748 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2808056b40 con 0x7f281c0717d0 2024-04-17T09:52:59.123 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-04-17T09:52:59.124+0000 7f6b9b000640 -1 osd.5 0 waiting for initial osdmap 2024-04-17T09:52:59.137 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-04-17T09:52:59.136+0000 7ff10a200640 -1 osd.6 0 waiting for initial osdmap 2024-04-17T09:52:59.147 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-04-17T09:52:59.148+0000 7f1739400640 -1 osd.7 0 waiting for initial osdmap 2024-04-17T09:52:59.152 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-04-17T09:52:59.152+0000 7f6b91000640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T09:52:59.159 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:59.155+0000 7f2823400640 1 -- 172.21.15.32:0/2581738748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f281c15a3c0 con 0x7f281c0717d0 2024-04-17T09:52:59.160 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:59.159+0000 7f2818c00640 1 -- 172.21.15.32:0/2581738748 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+8041 (secure 0 0 0) 0x7f2808056440 con 0x7f281c0717d0 2024-04-17T09:52:59.160 INFO:teuthology.misc.health.smithi032.stdout: 2024-04-17T09:52:59.160 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":14,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:52:58.304542+0000","last_up_change":"2024-04-17T09:52:58.304542+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:52:59.162 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:59.159+0000 7f2823400640 1 -- 172.21.15.32:0/2581738748 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f27fc0682f0 msgr2=0x7f27fc06a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:59.162 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:59.159+0000 7f2823400640 1 --2- 172.21.15.32:0/2581738748 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f27fc0682f0 0x7f27fc06a7b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f28100ae000 tx=0x7f28100a7000 comp rx=0 tx=0).stop 2024-04-17T09:52:59.163 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:59.159+0000 7f2823400640 1 -- 172.21.15.32:0/2581738748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281c0717d0 msgr2=0x7f281c15a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:52:59.163 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:59.159+0000 7f2823400640 1 --2- 172.21.15.32:0/2581738748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281c0717d0 0x7f281c15a9a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f280800e810 tx=0x7f280800ecd0 comp rx=0 tx=0).stop 2024-04-17T09:52:59.163 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:59.159+0000 7f2823400640 1 -- 172.21.15.32:0/2581738748 shutdown_connections 2024-04-17T09:52:59.163 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:59.159+0000 7f2823400640 1 --2- 172.21.15.32:0/2581738748 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f27fc0682f0 0x7f27fc06a7b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:59.163 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:59.159+0000 7f2823400640 1 --2- 172.21.15.32:0/2581738748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f281c0d0870 0x7f281c15aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:59.163 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:59.159+0000 7f2823400640 1 --2- 172.21.15.32:0/2581738748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281c0717d0 0x7f281c15a9a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:59.163 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:59.159+0000 7f2823400640 1 --2- 172.21.15.32:0/2581738748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f281c06cde0 0x7f281c1556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:52:59.163 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:59.159+0000 7f2823400640 1 -- 172.21.15.32:0/2581738748 >> 172.21.15.32:0/2581738748 conn(0x7f281c0d2500 msgr2=0x7f281c0b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:52:59.163 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:59.159+0000 7f2823400640 1 -- 172.21.15.32:0/2581738748 shutdown_connections 2024-04-17T09:52:59.163 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:52:59.159+0000 7f2823400640 1 -- 172.21.15.32:0/2581738748 wait complete. 2024-04-17T09:52:59.164 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-04-17T09:52:59.164+0000 7ff100200640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T09:52:59.179 DEBUG:teuthology.misc:2 of 8 OSDs are up 2024-04-17T09:52:59.179 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-04-17T09:52:59.180+0000 7f172f400640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T09:53:05.986 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-17T09:53:06.129 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.127+0000 7f7e4fa00640 1 Processor -- start 2024-04-17T09:53:06.129 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.127+0000 7f7e4fa00640 1 -- start start 2024-04-17T09:53:06.130 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.127+0000 7f7e4fa00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7e480d0f10 0x7f7e4809aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:06.130 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.127+0000 7f7e4fa00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e48160230 con 0x7f7e480b05e0 2024-04-17T09:53:06.130 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.127+0000 7f7e4fa00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e480cdcc0 con 0x7f7e4815fe50 2024-04-17T09:53:06.131 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.127+0000 7f7e4fa00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7e480cdfa0 con 0x7f7e480d0f10 2024-04-17T09:53:06.131 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.127+0000 7f7e47e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7e480d0f10 0x7f7e4809aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:06.131 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.127+0000 7f7e47e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7e480d0f10 0x7f7e4809aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37332/0 (socket says 172.21.15.32:37332) 2024-04-17T09:53:06.131 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.127+0000 7f7e47e00640 1 -- 172.21.15.32:0/2549859233 learned_addr learned my addr 172.21.15.32:0/2549859233 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:06.131 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.127+0000 7f7e47e00640 1 -- 172.21.15.32:0/2549859233 >> v1:172.21.15.144:6789/0 conn(0x7f7e4815fe50 legacy=0x7f7e480d0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:06.131 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.127+0000 7f7e47e00640 1 -- 172.21.15.32:0/2549859233 >> v1:172.21.15.32:6789/0 conn(0x7f7e480b05e0 legacy=0x7f7e4809af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:06.131 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.127+0000 7f7e47e00640 1 -- 172.21.15.32:0/2549859233 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e480ce280 con 0x7f7e480d0f10 2024-04-17T09:53:06.131 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.127+0000 7f7e47400640 1 -- 172.21.15.32:0/2549859233 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2965659941 0 0) 0x7f7e48160230 con 0x7f7e480b05e0 2024-04-17T09:53:06.134 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.131+0000 7f7e47e00640 1 --2- 172.21.15.32:0/2549859233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e480d0f10 0x7f7e4809aa30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7e34009870 tx=0x7f7e3401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6795b8e48ee6120c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:06.134 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.131+0000 7f7e47400640 1 -- 172.21.15.32:0/2549859233 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e340aa620 con 0x7f7e480d0f10 2024-04-17T09:53:06.134 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.131+0000 7f7e47400640 1 -- 172.21.15.32:0/2549859233 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e340af070 con 0x7f7e480d0f10 2024-04-17T09:53:06.134 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.131+0000 7f7e47400640 1 -- 172.21.15.32:0/2549859233 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e340aa9e0 con 0x7f7e480d0f10 2024-04-17T09:53:06.135 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.131+0000 7f7e4fa00640 1 -- 172.21.15.32:0/2549859233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e480d0f10 msgr2=0x7f7e4809aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:06.135 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.131+0000 7f7e4fa00640 1 --2- 172.21.15.32:0/2549859233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e480d0f10 0x7f7e4809aa30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7e34009870 tx=0x7f7e3401c4a0 comp rx=0 tx=0).stop 2024-04-17T09:53:06.135 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.131+0000 7f7e4fa00640 1 -- 172.21.15.32:0/2549859233 shutdown_connections 2024-04-17T09:53:06.135 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.131+0000 7f7e4fa00640 1 --2- 172.21.15.32:0/2549859233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e480d0f10 0x7f7e4809aa30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:06.135 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.131+0000 7f7e4fa00640 1 -- 172.21.15.32:0/2549859233 >> 172.21.15.32:0/2549859233 conn(0x7f7e480b4030 msgr2=0x7f7e480b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:06.135 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.131+0000 7f7e4fa00640 1 -- 172.21.15.32:0/2549859233 shutdown_connections 2024-04-17T09:53:06.135 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.131+0000 7f7e4fa00640 1 -- 172.21.15.32:0/2549859233 wait complete. 2024-04-17T09:53:06.136 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e4fa00640 1 Processor -- start 2024-04-17T09:53:06.136 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e4fa00640 1 -- start start 2024-04-17T09:53:06.139 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e4fa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e480b05e0 0x7f7e48156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:06.139 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e4fa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e480d0f10 0x7f7e4814ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:06.139 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e4fa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e4815fe50 0x7f7e48151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:06.139 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e4fa00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4816a8b0 con 0x7f7e4815fe50 2024-04-17T09:53:06.139 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e4fa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4816a730 con 0x7f7e480d0f10 2024-04-17T09:53:06.139 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e4fa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7e4816aa30 con 0x7f7e480b05e0 2024-04-17T09:53:06.139 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e4da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e4815fe50 0x7f7e48151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:06.139 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e4da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e4815fe50 0x7f7e48151f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37338/0 (socket says 172.21.15.32:37338) 2024-04-17T09:53:06.139 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e4da00640 1 -- 172.21.15.32:0/2176921465 learned_addr learned my addr 172.21.15.32:0/2176921465 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:06.139 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e47e00640 1 --2- 172.21.15.32:0/2176921465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e480d0f10 0x7f7e4814ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:06.140 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e4d000640 1 --2- 172.21.15.32:0/2176921465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e480b05e0 0x7f7e48156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:06.140 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e4da00640 1 -- 172.21.15.32:0/2176921465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e480b05e0 msgr2=0x7f7e48156e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:06.140 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e4da00640 1 --2- 172.21.15.32:0/2176921465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e480b05e0 0x7f7e48156e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:06.140 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e4da00640 1 -- 172.21.15.32:0/2176921465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e480d0f10 msgr2=0x7f7e4814ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:06.140 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e4da00640 1 --2- 172.21.15.32:0/2176921465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e480d0f10 0x7f7e4814ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:06.140 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e4da00640 1 -- 172.21.15.32:0/2176921465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e481573a0 con 0x7f7e4815fe50 2024-04-17T09:53:06.140 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e47e00640 1 --2- 172.21.15.32:0/2176921465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e480d0f10 0x7f7e4814ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:06.140 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e4da00640 1 --2- 172.21.15.32:0/2176921465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e4815fe50 0x7f7e48151f80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000c710 tx=0x7f7e4000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:06.140 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e44c00640 1 -- 172.21.15.32:0/2176921465 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40003b80 con 0x7f7e4815fe50 2024-04-17T09:53:06.140 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e44c00640 1 -- 172.21.15.32:0/2176921465 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e40003d40 con 0x7f7e4815fe50 2024-04-17T09:53:06.140 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e44c00640 1 -- 172.21.15.32:0/2176921465 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4000aa90 con 0x7f7e4815fe50 2024-04-17T09:53:06.140 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e4fa00640 1 -- 172.21.15.32:0/2176921465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e48158010 con 0x7f7e4815fe50 2024-04-17T09:53:06.140 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e4fa00640 1 -- 172.21.15.32:0/2176921465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e481585d0 con 0x7f7e4815fe50 2024-04-17T09:53:06.141 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.135+0000 7f7e4fa00640 1 -- 172.21.15.32:0/2176921465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e480a0590 con 0x7f7e4815fe50 2024-04-17T09:53:06.141 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.139+0000 7f7e44c00640 1 -- 172.21.15.32:0/2176921465 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7e4000ac30 con 0x7f7e4815fe50 2024-04-17T09:53:06.142 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.139+0000 7f7e44c00640 1 --2- 172.21.15.32:0/2176921465 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7e28068380 0x7f7e2806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:06.142 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.139+0000 7f7e44c00640 1 -- 172.21.15.32:0/2176921465 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f7e40086640 con 0x7f7e4815fe50 2024-04-17T09:53:06.142 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.139+0000 7f7e4d000640 1 --2- 172.21.15.32:0/2176921465 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7e28068380 0x7f7e2806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:06.142 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.139+0000 7f7e4d000640 1 --2- 172.21.15.32:0/2176921465 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7e28068380 0x7f7e2806a840 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c000f70 tx=0x7f7e3c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:06.147 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.143+0000 7f7e44c00640 1 -- 172.21.15.32:0/2176921465 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f7e4000ce20 con 0x7f7e4815fe50 2024-04-17T09:53:06.315 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.311+0000 7f7e4fa00640 1 -- 172.21.15.32:0/2176921465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e480b0d20 con 0x7f7e4815fe50 2024-04-17T09:53:06.316 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.311+0000 7f7e44c00640 1 -- 172.21.15.32:0/2176921465 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12896 (secure 0 0 0) 0x7f7e40056eb0 con 0x7f7e4815fe50 2024-04-17T09:53:06.316 INFO:teuthology.misc.health.smithi032.stdout: 2024-04-17T09:53:06.317 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":17,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:01.709523+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:53:06.319 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.315+0000 7f7e4fa00640 1 -- 172.21.15.32:0/2176921465 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7e28068380 msgr2=0x7f7e2806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:06.319 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.315+0000 7f7e4fa00640 1 --2- 172.21.15.32:0/2176921465 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7e28068380 0x7f7e2806a840 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c000f70 tx=0x7f7e3c009040 comp rx=0 tx=0).stop 2024-04-17T09:53:06.319 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.315+0000 7f7e4fa00640 1 -- 172.21.15.32:0/2176921465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e4815fe50 msgr2=0x7f7e48151f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:06.319 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.315+0000 7f7e4fa00640 1 --2- 172.21.15.32:0/2176921465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e4815fe50 0x7f7e48151f80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000c710 tx=0x7f7e4000cbd0 comp rx=0 tx=0).stop 2024-04-17T09:53:06.319 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.315+0000 7f7e4fa00640 1 -- 172.21.15.32:0/2176921465 shutdown_connections 2024-04-17T09:53:06.320 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.315+0000 7f7e4fa00640 1 --2- 172.21.15.32:0/2176921465 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7e28068380 0x7f7e2806a840 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:06.320 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.315+0000 7f7e4fa00640 1 --2- 172.21.15.32:0/2176921465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e4815fe50 0x7f7e48151f80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:06.320 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.315+0000 7f7e4fa00640 1 --2- 172.21.15.32:0/2176921465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e480d0f10 0x7f7e4814ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:06.320 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.315+0000 7f7e4fa00640 1 --2- 172.21.15.32:0/2176921465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e480b05e0 0x7f7e48156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:06.320 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.315+0000 7f7e4fa00640 1 -- 172.21.15.32:0/2176921465 >> 172.21.15.32:0/2176921465 conn(0x7f7e480b4030 msgr2=0x7f7e480cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:06.320 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.315+0000 7f7e4fa00640 1 -- 172.21.15.32:0/2176921465 shutdown_connections 2024-04-17T09:53:06.320 INFO:teuthology.misc.health.smithi032.stderr:2024-04-17T09:53:06.315+0000 7f7e4fa00640 1 -- 172.21.15.32:0/2176921465 wait complete. 2024-04-17T09:53:06.336 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-17T09:53:06.336 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-17T09:53:06.336 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-17T09:53:06.336 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-17T09:53:06.339 INFO:tasks.ceph.mds.a:Started 2024-04-17T09:53:06.339 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-17T09:53:06.339 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-17T09:53:06.342 INFO:tasks.ceph.mds.c:Started 2024-04-17T09:53:06.342 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-17T09:53:06.342 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-17T09:53:06.346 INFO:tasks.ceph.mds.b:Started 2024-04-17T09:53:06.346 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-17T09:53:06.346 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-17T09:53:06.349 INFO:tasks.ceph.mds.d:Started 2024-04-17T09:53:06.349 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-17T09:53:06.350 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:53:06.447 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-17T09:53:06.447 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-17T09:53:06.466 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2024-04-17T09:53:06.466 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2024-04-17T09:53:06.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.491+0000 7f5fb4000640 1 Processor -- start 2024-04-17T09:53:06.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.491+0000 7f5fb4000640 1 -- start start 2024-04-17T09:53:06.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.491+0000 7f5fb4000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5fac1604f0 0x7f5fac09ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:06.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.491+0000 7f5fb4000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5fac15e3a0 0x7f5fac15e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:06.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.491+0000 7f5fb4000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fac09a9f0 con 0x7f5fac1608d0 2024-04-17T09:53:06.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.491+0000 7f5fb4000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5fac09b600 con 0x7f5fac1604f0 2024-04-17T09:53:06.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.491+0000 7f5fb4000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f5fac09acd0 con 0x7f5fac15e3a0 2024-04-17T09:53:06.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.495+0000 7f5fb0c00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5fac1608d0 0x7f5fac0cdb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58636/0 (socket says 172.21.15.32:58636) 2024-04-17T09:53:06.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.495+0000 7f5fb0c00640 1 -- 172.21.15.32:0/3895487985 learned_addr learned my addr 172.21.15.32:0/3895487985 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:06.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.495+0000 7f5fb1600640 1 --2- 172.21.15.32:0/3895487985 >> v2:172.21.15.32:3300/0 conn(0x7f5fac1604f0 0x7f5fac09ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:06.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.495+0000 7f5fb2000640 1 --2- 172.21.15.32:0/3895487985 >> v2:172.21.15.144:3300/0 conn(0x7f5fac15e3a0 0x7f5fac15e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:06.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.495+0000 7f5fb1600640 1 -- 172.21.15.32:0/3895487985 >> v2:172.21.15.144:3300/0 conn(0x7f5fac15e3a0 msgr2=0x7f5fac15e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:06.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.495+0000 7f5fb1600640 1 --2- 172.21.15.32:0/3895487985 >> v2:172.21.15.144:3300/0 conn(0x7f5fac15e3a0 0x7f5fac15e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:06.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.495+0000 7f5fb1600640 1 -- 172.21.15.32:0/3895487985 >> v1:172.21.15.32:6789/0 conn(0x7f5fac1608d0 legacy=0x7f5fac0cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:06.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.495+0000 7f5fb1600640 1 -- 172.21.15.32:0/3895487985 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac09afb0 con 0x7f5fac1604f0 2024-04-17T09:53:06.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.495+0000 7f5fabe00640 1 -- 172.21.15.32:0/3895487985 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2485465639 0 0) 0x7f5fac09a9f0 con 0x7f5fac1608d0 2024-04-17T09:53:06.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.495+0000 7f5fb1600640 1 --2- 172.21.15.32:0/3895487985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fac1604f0 0x7f5fac09ba90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00097c0 tx=0x7f5fa001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4785d1a42a9a39 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:06.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.495+0000 7f5fabe00640 1 -- 172.21.15.32:0/3895487985 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa00b0070 con 0x7f5fac1604f0 2024-04-17T09:53:06.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.495+0000 7f5fabe00640 1 -- 172.21.15.32:0/3895487985 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa00ab600 con 0x7f5fac1604f0 2024-04-17T09:53:06.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.495+0000 7f5fabe00640 1 -- 172.21.15.32:0/3895487985 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa00ab9d0 con 0x7f5fac1604f0 2024-04-17T09:53:06.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb4000640 1 -- 172.21.15.32:0/3895487985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fac1604f0 msgr2=0x7f5fac09ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:06.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb4000640 1 --2- 172.21.15.32:0/3895487985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fac1604f0 0x7f5fac09ba90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00097c0 tx=0x7f5fa001c430 comp rx=0 tx=0).stop 2024-04-17T09:53:06.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb4000640 1 -- 172.21.15.32:0/3895487985 shutdown_connections 2024-04-17T09:53:06.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb4000640 1 --2- 172.21.15.32:0/3895487985 >> v2:172.21.15.144:3300/0 conn(0x7f5fac15e3a0 0x7f5fac15e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:06.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb4000640 1 --2- 172.21.15.32:0/3895487985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fac1604f0 0x7f5fac09ba90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:06.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb4000640 1 -- 172.21.15.32:0/3895487985 >> 172.21.15.32:0/3895487985 conn(0x7f5fac0b4010 msgr2=0x7f5fac0b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:06.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb4000640 1 -- 172.21.15.32:0/3895487985 shutdown_connections 2024-04-17T09:53:06.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb4000640 1 -- 172.21.15.32:0/3895487985 wait complete. 2024-04-17T09:53:06.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb4000640 1 Processor -- start 2024-04-17T09:53:06.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb4000640 1 -- start start 2024-04-17T09:53:06.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb4000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fac15e3a0 0x7f5fac0c7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:06.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb4000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fac1604f0 0x7f5fac0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:06.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb4000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fac1608d0 0x7f5fac14f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:06.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb4000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac0ce410 con 0x7f5fac15e3a0 2024-04-17T09:53:06.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb4000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac0ce290 con 0x7f5fac1604f0 2024-04-17T09:53:06.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb4000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5fac160ce0 con 0x7f5fac1608d0 2024-04-17T09:53:06.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb1600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fac15e3a0 0x7f5fac0c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:06.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb1600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fac15e3a0 0x7f5fac0c7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37392/0 (socket says 172.21.15.32:37392) 2024-04-17T09:53:06.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb1600640 1 -- 172.21.15.32:0/2030535775 learned_addr learned my addr 172.21.15.32:0/2030535775 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:06.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb1600640 1 -- 172.21.15.32:0/2030535775 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fac1608d0 msgr2=0x7f5fac14f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:06.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb1600640 1 --2- 172.21.15.32:0/2030535775 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fac1608d0 0x7f5fac14f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:06.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb1600640 1 -- 172.21.15.32:0/2030535775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fac1604f0 msgr2=0x7f5fac0b71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:06.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb1600640 1 --2- 172.21.15.32:0/2030535775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fac1604f0 0x7f5fac0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:06.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.499+0000 7f5fb1600640 1 -- 172.21.15.32:0/2030535775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac0c7f70 con 0x7f5fac15e3a0 2024-04-17T09:53:06.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.503+0000 7f5fb1600640 1 --2- 172.21.15.32:0/2030535775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fac15e3a0 0x7f5fac0c7720 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5fa001a060 tx=0x7f5fa00191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:06.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.503+0000 7f5fa9600640 1 -- 172.21.15.32:0/2030535775 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa00b0040 con 0x7f5fac15e3a0 2024-04-17T09:53:06.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.503+0000 7f5fa9600640 1 -- 172.21.15.32:0/2030535775 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa00bb070 con 0x7f5fac15e3a0 2024-04-17T09:53:06.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.503+0000 7f5fb4000640 1 -- 172.21.15.32:0/2030535775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fac0c8250 con 0x7f5fac15e3a0 2024-04-17T09:53:06.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.503+0000 7f5fa9600640 1 -- 172.21.15.32:0/2030535775 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0006e20 con 0x7f5fac15e3a0 2024-04-17T09:53:06.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.503+0000 7f5fb4000640 1 -- 172.21.15.32:0/2030535775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fac14ec80 con 0x7f5fac15e3a0 2024-04-17T09:53:06.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.503+0000 7f5fb4000640 1 -- 172.21.15.32:0/2030535775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fac068a90 con 0x7f5fac15e3a0 2024-04-17T09:53:06.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.503+0000 7f5fa9600640 1 -- 172.21.15.32:0/2030535775 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5fa00b3070 con 0x7f5fac15e3a0 2024-04-17T09:53:06.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.503+0000 7f5fa9600640 1 --2- 172.21.15.32:0/2030535775 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5f8c068380 0x7f5f8c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:06.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.503+0000 7f5fa9600640 1 -- 172.21.15.32:0/2030535775 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f5fa01203f0 con 0x7f5fac15e3a0 2024-04-17T09:53:06.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.503+0000 7f5fb0c00640 1 --2- 172.21.15.32:0/2030535775 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5f8c068380 0x7f5f8c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:06.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.507+0000 7f5fb0c00640 1 --2- 172.21.15.32:0/2030535775 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5f8c068380 0x7f5f8c06a840 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5f980011f0 tx=0x7f5f98007930 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:06.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.507+0000 7f5fa9600640 1 -- 172.21.15.32:0/2030535775 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5fa0016050 con 0x7f5fac15e3a0 2024-04-17T09:53:06.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.675+0000 7f5fb4000640 1 -- 172.21.15.32:0/2030535775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5fac0b4c70 con 0x7f5fac15e3a0 2024-04-17T09:53:06.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.675+0000 7f5fa9600640 1 -- 172.21.15.32:0/2030535775 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12896 (secure 0 0 0) 0x7f5fa00f0c60 con 0x7f5fac15e3a0 2024-04-17T09:53:06.678 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:06.679 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:01.709523+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:53:06.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.675+0000 7f5fb4000640 1 -- 172.21.15.32:0/2030535775 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5f8c068380 msgr2=0x7f5f8c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:06.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.675+0000 7f5fb4000640 1 --2- 172.21.15.32:0/2030535775 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5f8c068380 0x7f5f8c06a840 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5f980011f0 tx=0x7f5f98007930 comp rx=0 tx=0).stop 2024-04-17T09:53:06.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.675+0000 7f5fb4000640 1 -- 172.21.15.32:0/2030535775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fac15e3a0 msgr2=0x7f5fac0c7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:06.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.675+0000 7f5fb4000640 1 --2- 172.21.15.32:0/2030535775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fac15e3a0 0x7f5fac0c7720 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5fa001a060 tx=0x7f5fa00191d0 comp rx=0 tx=0).stop 2024-04-17T09:53:06.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.679+0000 7f5fb4000640 1 -- 172.21.15.32:0/2030535775 shutdown_connections 2024-04-17T09:53:06.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.679+0000 7f5fb4000640 1 --2- 172.21.15.32:0/2030535775 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5f8c068380 0x7f5f8c06a840 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:06.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.679+0000 7f5fb4000640 1 --2- 172.21.15.32:0/2030535775 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fac1608d0 0x7f5fac14f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:06.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.679+0000 7f5fb4000640 1 --2- 172.21.15.32:0/2030535775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fac1604f0 0x7f5fac0b71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:06.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.679+0000 7f5fb4000640 1 --2- 172.21.15.32:0/2030535775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fac15e3a0 0x7f5fac0c7720 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:06.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.679+0000 7f5fb4000640 1 -- 172.21.15.32:0/2030535775 >> 172.21.15.32:0/2030535775 conn(0x7f5fac0b4010 msgr2=0x7f5fac0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:06.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.679+0000 7f5fb4000640 1 -- 172.21.15.32:0/2030535775 shutdown_connections 2024-04-17T09:53:06.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.679+0000 7f5fb4000640 1 -- 172.21.15.32:0/2030535775 wait complete. 2024-04-17T09:53:06.699 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:53:06.699 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:53:06.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.839+0000 7f7bd0000640 1 Processor -- start 2024-04-17T09:53:06.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.839+0000 7f7bd0000640 1 -- start start 2024-04-17T09:53:06.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.839+0000 7f7bd0000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7bc809a9f0 0x7f7bc809add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:06.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.839+0000 7f7bd0000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bc80d1b10 con 0x7f7bc8162cd0 2024-04-17T09:53:06.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.839+0000 7f7bd0000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bc809b310 con 0x7f7bc806da00 2024-04-17T09:53:06.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.839+0000 7f7bd0000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f7bc80d1680 con 0x7f7bc809a9f0 2024-04-17T09:53:06.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.843+0000 7f7bce000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7bc8162cd0 0x7f7bc81698f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58642/0 (socket says 172.21.15.32:58642) 2024-04-17T09:53:06.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.843+0000 7f7bce000640 1 -- 172.21.15.32:0/2130524049 learned_addr learned my addr 172.21.15.32:0/2130524049 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:06.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.843+0000 7f7bcd600640 1 --2- 172.21.15.32:0/2130524049 >> v2:172.21.15.144:3300/0 conn(0x7f7bc809a9f0 0x7f7bc809add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:06.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.843+0000 7f7bc7e00640 1 -- 172.21.15.32:0/2130524049 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 153374216 0 0) 0x7f7bc00014a0 con 0x7f7bc8162cd0 2024-04-17T09:53:06.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.843+0000 7f7bc7e00640 1 -- 172.21.15.32:0/2130524049 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7bb0003530 con 0x7f7bc8162cd0 2024-04-17T09:53:06.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.843+0000 7f7bcd600640 1 -- 172.21.15.32:0/2130524049 >> v1:172.21.15.144:6789/0 conn(0x7f7bc806da00 legacy=0x7f7bc81625c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:06.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.843+0000 7f7bcd600640 1 -- 172.21.15.32:0/2130524049 >> v1:172.21.15.32:6789/0 conn(0x7f7bc8162cd0 legacy=0x7f7bc81698f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:06.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.843+0000 7f7bcd600640 1 -- 172.21.15.32:0/2130524049 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc8162310 con 0x7f7bc809a9f0 2024-04-17T09:53:06.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.843+0000 7f7bcd600640 1 --2- 172.21.15.32:0/2130524049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc809a9f0 0x7f7bc809add0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0097c0 tx=0x7f7bbc01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17239e9eea779059 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:06.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bc7e00640 1 -- 172.21.15.32:0/2130524049 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc0b1070 con 0x7f7bc809a9f0 2024-04-17T09:53:06.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bc7e00640 1 -- 172.21.15.32:0/2130524049 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bbc0a8050 con 0x7f7bc809a9f0 2024-04-17T09:53:06.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bc7e00640 1 -- 172.21.15.32:0/2130524049 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc0ac860 con 0x7f7bc809a9f0 2024-04-17T09:53:06.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bd0000640 1 -- 172.21.15.32:0/2130524049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc809a9f0 msgr2=0x7f7bc809add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:06.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bd0000640 1 --2- 172.21.15.32:0/2130524049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc809a9f0 0x7f7bc809add0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0097c0 tx=0x7f7bbc01c3e0 comp rx=0 tx=0).stop 2024-04-17T09:53:06.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bd0000640 1 -- 172.21.15.32:0/2130524049 shutdown_connections 2024-04-17T09:53:06.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bd0000640 1 --2- 172.21.15.32:0/2130524049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc809a9f0 0x7f7bc809add0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:06.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bd0000640 1 -- 172.21.15.32:0/2130524049 >> 172.21.15.32:0/2130524049 conn(0x7f7bc80b4080 msgr2=0x7f7bc80b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:06.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bd0000640 1 -- 172.21.15.32:0/2130524049 shutdown_connections 2024-04-17T09:53:06.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bd0000640 1 -- 172.21.15.32:0/2130524049 wait complete. 2024-04-17T09:53:06.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bd0000640 1 Processor -- start 2024-04-17T09:53:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bd0000640 1 -- start start 2024-04-17T09:53:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bd0000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc806da00 0x7f7bc80a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bd0000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bc809a9f0 0x7f7bc81f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bd0000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bc8162cd0 0x7f7bc815cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bd0000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc80d0720 con 0x7f7bc8162cd0 2024-04-17T09:53:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bd0000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc80d1310 con 0x7f7bc806da00 2024-04-17T09:53:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bd0000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7bc8161440 con 0x7f7bc809a9f0 2024-04-17T09:53:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bce000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bc8162cd0 0x7f7bc815cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bce000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bc8162cd0 0x7f7bc815cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37406/0 (socket says 172.21.15.32:37406) 2024-04-17T09:53:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bce000640 1 -- 172.21.15.32:0/2003298454 learned_addr learned my addr 172.21.15.32:0/2003298454 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bccc00640 1 --2- 172.21.15.32:0/2003298454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bc809a9f0 0x7f7bc81f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bcd600640 1 --2- 172.21.15.32:0/2003298454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc806da00 0x7f7bc80a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bce000640 1 -- 172.21.15.32:0/2003298454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bc809a9f0 msgr2=0x7f7bc81f7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bce000640 1 --2- 172.21.15.32:0/2003298454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bc809a9f0 0x7f7bc81f7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bce000640 1 -- 172.21.15.32:0/2003298454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc806da00 msgr2=0x7f7bc80a0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bce000640 1 --2- 172.21.15.32:0/2003298454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc806da00 0x7f7bc80a0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bce000640 1 -- 172.21.15.32:0/2003298454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc809f640 con 0x7f7bc8162cd0 2024-04-17T09:53:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.847+0000 7f7bcd600640 1 --2- 172.21.15.32:0/2003298454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc806da00 0x7f7bc80a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.851+0000 7f7bce000640 1 --2- 172.21.15.32:0/2003298454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bc8162cd0 0x7f7bc815cac0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000cb80 tx=0x7f7bc000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.851+0000 7f7bc5600640 1 -- 172.21.15.32:0/2003298454 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0017070 con 0x7f7bc8162cd0 2024-04-17T09:53:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.851+0000 7f7bc5600640 1 -- 172.21.15.32:0/2003298454 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bc000ad40 con 0x7f7bc8162cd0 2024-04-17T09:53:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.851+0000 7f7bd0000640 1 -- 172.21.15.32:0/2003298454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc81f8470 con 0x7f7bc8162cd0 2024-04-17T09:53:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.851+0000 7f7bc5600640 1 -- 172.21.15.32:0/2003298454 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0012620 con 0x7f7bc8162cd0 2024-04-17T09:53:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.851+0000 7f7bd0000640 1 -- 172.21.15.32:0/2003298454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc81f8a30 con 0x7f7bc8162cd0 2024-04-17T09:53:06.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.851+0000 7f7bd0000640 1 -- 172.21.15.32:0/2003298454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bc8068a90 con 0x7f7bc8162cd0 2024-04-17T09:53:06.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.851+0000 7f7bc5600640 1 -- 172.21.15.32:0/2003298454 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7bc001a070 con 0x7f7bc8162cd0 2024-04-17T09:53:06.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.851+0000 7f7bc5600640 1 --2- 172.21.15.32:0/2003298454 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7ba8068380 0x7f7ba806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:06.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.851+0000 7f7bc5600640 1 -- 172.21.15.32:0/2003298454 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f7bc0086490 con 0x7f7bc8162cd0 2024-04-17T09:53:06.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.851+0000 7f7bcd600640 1 --2- 172.21.15.32:0/2003298454 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7ba8068380 0x7f7ba806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:06.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.855+0000 7f7bcd600640 1 --2- 172.21.15.32:0/2003298454 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7ba8068380 0x7f7ba806a840 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0ae000 tx=0x7f7bbc0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:06.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:06.855+0000 7f7bc5600640 1 -- 172.21.15.32:0/2003298454 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f7bc80d1b10 con 0x7f7bc8162cd0 2024-04-17T09:53:07.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.019+0000 7f7bd0000640 1 -- 172.21.15.32:0/2003298454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7bc80b4de0 con 0x7f7bc8162cd0 2024-04-17T09:53:07.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.019+0000 7f7bc5600640 1 -- 172.21.15.32:0/2003298454 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7f7bc0056d90 con 0x7f7bc8162cd0 2024-04-17T09:53:07.023 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:53:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.023+0000 7f7bd0000640 1 -- 172.21.15.32:0/2003298454 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7ba8068380 msgr2=0x7f7ba806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.023+0000 7f7bd0000640 1 --2- 172.21.15.32:0/2003298454 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7ba8068380 0x7f7ba806a840 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0ae000 tx=0x7f7bbc0a7000 comp rx=0 tx=0).stop 2024-04-17T09:53:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.023+0000 7f7bd0000640 1 -- 172.21.15.32:0/2003298454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bc8162cd0 msgr2=0x7f7bc815cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.023+0000 7f7bd0000640 1 --2- 172.21.15.32:0/2003298454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bc8162cd0 0x7f7bc815cac0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000cb80 tx=0x7f7bc000a430 comp rx=0 tx=0).stop 2024-04-17T09:53:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.023+0000 7f7bd0000640 1 -- 172.21.15.32:0/2003298454 shutdown_connections 2024-04-17T09:53:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.023+0000 7f7bd0000640 1 --2- 172.21.15.32:0/2003298454 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7ba8068380 0x7f7ba806a840 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.023+0000 7f7bd0000640 1 --2- 172.21.15.32:0/2003298454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bc809a9f0 0x7f7bc81f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.023+0000 7f7bd0000640 1 --2- 172.21.15.32:0/2003298454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bc8162cd0 0x7f7bc815cac0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.023+0000 7f7bd0000640 1 --2- 172.21.15.32:0/2003298454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc806da00 0x7f7bc80a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.023+0000 7f7bd0000640 1 -- 172.21.15.32:0/2003298454 >> 172.21.15.32:0/2003298454 conn(0x7f7bc80b4080 msgr2=0x7f7bc80b73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.023+0000 7f7bd0000640 1 -- 172.21.15.32:0/2003298454 shutdown_connections 2024-04-17T09:53:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.023+0000 7f7bd0000640 1 -- 172.21.15.32:0/2003298454 wait complete. 2024-04-17T09:53:07.044 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:53:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.183+0000 7ff552000640 1 Processor -- start 2024-04-17T09:53:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.183+0000 7ff552000640 1 -- start start 2024-04-17T09:53:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.183+0000 7ff552000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff54c0b2430 0x7ff54c0b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.183+0000 7ff552000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff54c0b3900 0x7ff54c06ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.183+0000 7ff552000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff54c06c800 con 0x7ff54c15d670 2024-04-17T09:53:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.183+0000 7ff552000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff54c0b2d50 con 0x7ff54c0b2430 2024-04-17T09:53:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.183+0000 7ff552000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff54c06bf70 con 0x7ff54c0b3900 2024-04-17T09:53:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.183+0000 7ff54b400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff54c0b2430 0x7ff54c0b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.183+0000 7ff54b400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff54c0b2430 0x7ff54c0b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37420/0 (socket says 172.21.15.32:37420) 2024-04-17T09:53:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.183+0000 7ff54b400640 1 -- 172.21.15.32:0/3319538450 learned_addr learned my addr 172.21.15.32:0/3319538450 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.183+0000 7ff54b400640 1 -- 172.21.15.32:0/3319538450 >> v2:172.21.15.144:3300/0 conn(0x7ff54c0b3900 msgr2=0x7ff54c06ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.183+0000 7ff543e00640 1 --2- 172.21.15.32:0/3319538450 >> v2:172.21.15.144:3300/0 conn(0x7ff54c0b3900 0x7ff54c06ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.183+0000 7ff54b400640 1 --2- 172.21.15.32:0/3319538450 >> v2:172.21.15.144:3300/0 conn(0x7ff54c0b3900 0x7ff54c06ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.183+0000 7ff54b400640 1 -- 172.21.15.32:0/3319538450 >> v1:172.21.15.144:6789/0 conn(0x7ff54c15d670 legacy=0x7ff54c1602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.183+0000 7ff54b400640 1 -- 172.21.15.32:0/3319538450 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54c06c240 con 0x7ff54c0b2430 2024-04-17T09:53:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.183+0000 7ff543e00640 1 --2- 172.21.15.32:0/3319538450 >> v2:172.21.15.144:3300/0 conn(0x7ff54c0b3900 0x7ff54c06ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.187+0000 7ff54b400640 1 --2- 172.21.15.32:0/3319538450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff54c0b2430 0x7ff54c0b2810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009870 tx=0x7ff53c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844ae1c38f2ea041 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:07.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.187+0000 7ff54aa00640 1 -- 172.21.15.32:0/3319538450 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c0b0070 con 0x7ff54c0b2430 2024-04-17T09:53:07.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.187+0000 7ff54aa00640 1 -- 172.21.15.32:0/3319538450 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff53c0ab600 con 0x7ff54c0b2430 2024-04-17T09:53:07.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.187+0000 7ff54aa00640 1 -- 172.21.15.32:0/3319538450 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c0ab9d0 con 0x7ff54c0b2430 2024-04-17T09:53:07.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.187+0000 7ff552000640 1 -- 172.21.15.32:0/3319538450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff54c0b2430 msgr2=0x7ff54c0b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:07.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.187+0000 7ff552000640 1 --2- 172.21.15.32:0/3319538450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff54c0b2430 0x7ff54c0b2810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009870 tx=0x7ff53c01c4e0 comp rx=0 tx=0).stop 2024-04-17T09:53:07.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.187+0000 7ff552000640 1 -- 172.21.15.32:0/3319538450 shutdown_connections 2024-04-17T09:53:07.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.187+0000 7ff552000640 1 --2- 172.21.15.32:0/3319538450 >> v2:172.21.15.144:3300/0 conn(0x7ff54c0b3900 0x7ff54c06ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:07.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.187+0000 7ff552000640 1 --2- 172.21.15.32:0/3319538450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff54c0b2430 0x7ff54c0b2810 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:07.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.187+0000 7ff552000640 1 -- 172.21.15.32:0/3319538450 >> 172.21.15.32:0/3319538450 conn(0x7ff54c09a9f0 msgr2=0x7ff54c0b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:07.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.187+0000 7ff552000640 1 -- 172.21.15.32:0/3319538450 shutdown_connections 2024-04-17T09:53:07.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.187+0000 7ff552000640 1 -- 172.21.15.32:0/3319538450 wait complete. 2024-04-17T09:53:07.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.187+0000 7ff552000640 1 Processor -- start 2024-04-17T09:53:07.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff552000640 1 -- start start 2024-04-17T09:53:07.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff552000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff54c0b2430 0x7ff54c155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:07.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff552000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff54c0b3900 0x7ff54c0af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:07.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff552000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff54c15d670 0x7ff54c0af850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:07.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff552000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff54c0cdcb0 con 0x7ff54c0b2430 2024-04-17T09:53:07.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff552000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff54c0cdb30 con 0x7ff54c0b3900 2024-04-17T09:53:07.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff552000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff54c15dae0 con 0x7ff54c15d670 2024-04-17T09:53:07.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff54b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff54c0b2430 0x7ff54c155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:07.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff54b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff54c0b2430 0x7ff54c155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37424/0 (socket says 172.21.15.32:37424) 2024-04-17T09:53:07.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff54b400640 1 -- 172.21.15.32:0/2208965917 learned_addr learned my addr 172.21.15.32:0/2208965917 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:07.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff54b400640 1 -- 172.21.15.32:0/2208965917 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff54c15d670 msgr2=0x7ff54c0af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:07.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff54b400640 1 --2- 172.21.15.32:0/2208965917 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff54c15d670 0x7ff54c0af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:07.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff54b400640 1 -- 172.21.15.32:0/2208965917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff54c0b3900 msgr2=0x7ff54c0af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:07.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff543e00640 1 --2- 172.21.15.32:0/2208965917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff54c0b3900 0x7ff54c0af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:07.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff54b400640 1 --2- 172.21.15.32:0/2208965917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff54c0b3900 0x7ff54c0af310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:07.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff54b400640 1 -- 172.21.15.32:0/2208965917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54c0c9ae0 con 0x7ff54c0b2430 2024-04-17T09:53:07.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff543e00640 1 --2- 172.21.15.32:0/2208965917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff54c0b3900 0x7ff54c0af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:07.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff54b400640 1 --2- 172.21.15.32:0/2208965917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff54c0b2430 0x7ff54c155a10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff53c0ae040 tx=0x7ff53c01ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:07.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff543400640 1 -- 172.21.15.32:0/2208965917 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c0b0040 con 0x7ff54c0b2430 2024-04-17T09:53:07.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff543400640 1 -- 172.21.15.32:0/2208965917 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff53c0bc070 con 0x7ff54c0b2430 2024-04-17T09:53:07.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff552000640 1 -- 172.21.15.32:0/2208965917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff54c0c9800 con 0x7ff54c0b2430 2024-04-17T09:53:07.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff543400640 1 -- 172.21.15.32:0/2208965917 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c0b6880 con 0x7ff54c0b2430 2024-04-17T09:53:07.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff552000640 1 -- 172.21.15.32:0/2208965917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff54c155f50 con 0x7ff54c0b2430 2024-04-17T09:53:07.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.191+0000 7ff552000640 1 -- 172.21.15.32:0/2208965917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff54c068a90 con 0x7ff54c0b2430 2024-04-17T09:53:07.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.195+0000 7ff543400640 1 -- 172.21.15.32:0/2208965917 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff53c0ab4c0 con 0x7ff54c0b2430 2024-04-17T09:53:07.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.195+0000 7ff543400640 1 --2- 172.21.15.32:0/2208965917 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff528068380 0x7ff52806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:07.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.195+0000 7ff543400640 1 -- 172.21.15.32:0/2208965917 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7ff53c120b80 con 0x7ff54c0b2430 2024-04-17T09:53:07.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.195+0000 7ff543e00640 1 --2- 172.21.15.32:0/2208965917 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff528068380 0x7ff52806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:07.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.195+0000 7ff543e00640 1 --2- 172.21.15.32:0/2208965917 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff528068380 0x7ff52806a840 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff54c1fb090 tx=0x7ff534008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:07.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.199+0000 7ff543400640 1 -- 172.21.15.32:0/2208965917 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff53c0160d0 con 0x7ff54c0b2430 2024-04-17T09:53:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.399+0000 7ff552000640 1 -- 172.21.15.32:0/2208965917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff54c0b7450 con 0x7ff54c0b2430 2024-04-17T09:53:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.399+0000 7ff543400640 1 -- 172.21.15.32:0/2208965917 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7ff53c0f13f0 con 0x7ff54c0b2430 2024-04-17T09:53:07.404 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:07.404 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1066277472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1066277472},{"type":"v1","addr":"172.21.15.32:6836","nonce":1066277472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-17T09:53:07.405 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 2 2024-04-17T09:53:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.403+0000 7ff552000640 1 -- 172.21.15.32:0/2208965917 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff528068380 msgr2=0x7ff52806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.403+0000 7ff552000640 1 --2- 172.21.15.32:0/2208965917 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff528068380 0x7ff52806a840 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff54c1fb090 tx=0x7ff534008040 comp rx=0 tx=0).stop 2024-04-17T09:53:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.403+0000 7ff552000640 1 -- 172.21.15.32:0/2208965917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff54c0b2430 msgr2=0x7ff54c155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.403+0000 7ff552000640 1 --2- 172.21.15.32:0/2208965917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff54c0b2430 0x7ff54c155a10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff53c0ae040 tx=0x7ff53c01ad20 comp rx=0 tx=0).stop 2024-04-17T09:53:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.403+0000 7ff552000640 1 -- 172.21.15.32:0/2208965917 shutdown_connections 2024-04-17T09:53:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.403+0000 7ff552000640 1 --2- 172.21.15.32:0/2208965917 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff528068380 0x7ff52806a840 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.403+0000 7ff552000640 1 --2- 172.21.15.32:0/2208965917 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff54c15d670 0x7ff54c0af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.403+0000 7ff552000640 1 --2- 172.21.15.32:0/2208965917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff54c0b3900 0x7ff54c0af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.403+0000 7ff552000640 1 --2- 172.21.15.32:0/2208965917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff54c0b2430 0x7ff54c155a10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.403+0000 7ff552000640 1 -- 172.21.15.32:0/2208965917 >> 172.21.15.32:0/2208965917 conn(0x7ff54c09a9f0 msgr2=0x7ff54c0cde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.403+0000 7ff552000640 1 -- 172.21.15.32:0/2208965917 shutdown_connections 2024-04-17T09:53:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:07.403+0000 7ff552000640 1 -- 172.21.15.32:0/2208965917 wait complete. 2024-04-17T09:53:09.425 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:53:09.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.575+0000 7fab94600640 1 Processor -- start 2024-04-17T09:53:09.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.575+0000 7fab94600640 1 -- start start 2024-04-17T09:53:09.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.575+0000 7fab94600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab8c145f30 0x7fab8c071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:09.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.575+0000 7fab94600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fab8c15d8d0 0x7fab8c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:09.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.575+0000 7fab94600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab8c0cce70 0x7fab8c0cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:09.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.575+0000 7fab94600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fab8c06f400 con 0x7fab8c0cce70 2024-04-17T09:53:09.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.575+0000 7fab94600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fab8c146310 con 0x7fab8c15d8d0 2024-04-17T09:53:09.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.575+0000 7fab94600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fab8c15d440 con 0x7fab8c145f30 2024-04-17T09:53:09.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.575+0000 7fab92600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab8c0cce70 0x7fab8c0cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:09.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.575+0000 7fab92600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab8c0cce70 0x7fab8c0cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37434/0 (socket says 172.21.15.32:37434) 2024-04-17T09:53:09.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.575+0000 7fab92600640 1 -- 172.21.15.32:0/1814307317 learned_addr learned my addr 172.21.15.32:0/1814307317 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:09.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.575+0000 7fab91c00640 1 --2- 172.21.15.32:0/1814307317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab8c145f30 0x7fab8c071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:09.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.575+0000 7fab91200640 1 --2- 172.21.15.32:0/1814307317 >> v2:172.21.15.144:3300/0 conn(0x7fab8c15d8d0 0x7fab8c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:09.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.575+0000 7fab92600640 1 -- 172.21.15.32:0/1814307317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab8c145f30 msgr2=0x7fab8c071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:09.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.575+0000 7fab92600640 1 --2- 172.21.15.32:0/1814307317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab8c145f30 0x7fab8c071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:09.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.575+0000 7fab92600640 1 -- 172.21.15.32:0/1814307317 >> v2:172.21.15.144:3300/0 conn(0x7fab8c15d8d0 msgr2=0x7fab8c072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:09.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.575+0000 7fab92600640 1 --2- 172.21.15.32:0/1814307317 >> v2:172.21.15.144:3300/0 conn(0x7fab8c15d8d0 0x7fab8c072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:09.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.575+0000 7fab92600640 1 -- 172.21.15.32:0/1814307317 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c0cd840 con 0x7fab8c0cce70 2024-04-17T09:53:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.579+0000 7fab92600640 1 --2- 172.21.15.32:0/1814307317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab8c0cce70 0x7fab8c0cd300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0b58b0 tx=0x7fab8401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927bb3e224d10720 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.579+0000 7fab8be00640 1 -- 172.21.15.32:0/1814307317 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab840b4070 con 0x7fab8c0cce70 2024-04-17T09:53:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.579+0000 7fab8be00640 1 -- 172.21.15.32:0/1814307317 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab840af400 con 0x7fab8c0cce70 2024-04-17T09:53:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.579+0000 7fab8be00640 1 -- 172.21.15.32:0/1814307317 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab840af7d0 con 0x7fab8c0cce70 2024-04-17T09:53:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.579+0000 7fab94600640 1 -- 172.21.15.32:0/1814307317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab8c0cce70 msgr2=0x7fab8c0cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.579+0000 7fab94600640 1 --2- 172.21.15.32:0/1814307317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab8c0cce70 0x7fab8c0cd300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0b58b0 tx=0x7fab8401f1e0 comp rx=0 tx=0).stop 2024-04-17T09:53:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.579+0000 7fab94600640 1 -- 172.21.15.32:0/1814307317 shutdown_connections 2024-04-17T09:53:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.579+0000 7fab94600640 1 --2- 172.21.15.32:0/1814307317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab8c0cce70 0x7fab8c0cd300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.579+0000 7fab94600640 1 --2- 172.21.15.32:0/1814307317 >> v2:172.21.15.144:3300/0 conn(0x7fab8c15d8d0 0x7fab8c072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.579+0000 7fab94600640 1 --2- 172.21.15.32:0/1814307317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab8c145f30 0x7fab8c071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.579+0000 7fab94600640 1 -- 172.21.15.32:0/1814307317 >> 172.21.15.32:0/1814307317 conn(0x7fab8c06ba30 msgr2=0x7fab8c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.579+0000 7fab94600640 1 -- 172.21.15.32:0/1814307317 shutdown_connections 2024-04-17T09:53:09.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.579+0000 7fab94600640 1 -- 172.21.15.32:0/1814307317 wait complete. 2024-04-17T09:53:09.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab94600640 1 Processor -- start 2024-04-17T09:53:09.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab94600640 1 -- start start 2024-04-17T09:53:09.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab94600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab8c0cce70 0x7fab8c0ac910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:09.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab94600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab8c145f30 0x7fab8c0af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:09.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab94600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab8c15d8d0 0x7fab8c0af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:09.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab94600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c15cdf0 con 0x7fab8c145f30 2024-04-17T09:53:09.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab94600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c15cb90 con 0x7fab8c15d8d0 2024-04-17T09:53:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab94600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fab8c15d0d0 con 0x7fab8c0cce70 2024-04-17T09:53:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab91200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab8c145f30 0x7fab8c0af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab91200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab8c145f30 0x7fab8c0af250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37442/0 (socket says 172.21.15.32:37442) 2024-04-17T09:53:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab91200640 1 -- 172.21.15.32:0/668684836 learned_addr learned my addr 172.21.15.32:0/668684836 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab91c00640 1 --2- 172.21.15.32:0/668684836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab8c0cce70 0x7fab8c0ac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab92600640 1 --2- 172.21.15.32:0/668684836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab8c15d8d0 0x7fab8c0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab91200640 1 -- 172.21.15.32:0/668684836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab8c0cce70 msgr2=0x7fab8c0ac910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab91200640 1 --2- 172.21.15.32:0/668684836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab8c0cce70 0x7fab8c0ac910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab91200640 1 -- 172.21.15.32:0/668684836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab8c15d8d0 msgr2=0x7fab8c0af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab91200640 1 --2- 172.21.15.32:0/668684836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab8c15d8d0 0x7fab8c0af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab91200640 1 -- 172.21.15.32:0/668684836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c0ace50 con 0x7fab8c145f30 2024-04-17T09:53:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab92600640 1 --2- 172.21.15.32:0/668684836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab8c15d8d0 0x7fab8c0af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab91200640 1 --2- 172.21.15.32:0/668684836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab8c145f30 0x7fab8c0af250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fab7800e870 tx=0x7fab7800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab89600640 1 -- 172.21.15.32:0/668684836 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab78017070 con 0x7fab8c145f30 2024-04-17T09:53:09.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab89600640 1 -- 172.21.15.32:0/668684836 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab7800c940 con 0x7fab8c145f30 2024-04-17T09:53:09.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab94600640 1 -- 172.21.15.32:0/668684836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8c0ad340 con 0x7fab8c145f30 2024-04-17T09:53:09.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab89600640 1 -- 172.21.15.32:0/668684836 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7800ccf0 con 0x7fab8c145f30 2024-04-17T09:53:09.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab94600640 1 -- 172.21.15.32:0/668684836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8c0ad900 con 0x7fab8c145f30 2024-04-17T09:53:09.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.583+0000 7fab94600640 1 -- 172.21.15.32:0/668684836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab8c068a90 con 0x7fab8c145f30 2024-04-17T09:53:09.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.587+0000 7fab89600640 1 -- 172.21.15.32:0/668684836 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fab7801a070 con 0x7fab8c145f30 2024-04-17T09:53:09.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.587+0000 7fab89600640 1 --2- 172.21.15.32:0/668684836 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab6c068380 0x7fab6c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:09.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.587+0000 7fab89600640 1 -- 172.21.15.32:0/668684836 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fab78086510 con 0x7fab8c145f30 2024-04-17T09:53:09.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.587+0000 7fab91c00640 1 --2- 172.21.15.32:0/668684836 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab6c068380 0x7fab6c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:09.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.587+0000 7fab91c00640 1 --2- 172.21.15.32:0/668684836 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab6c068380 0x7fab6c06a840 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fab80009ae0 tx=0x7fab80009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:09.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.591+0000 7fab89600640 1 -- 172.21.15.32:0/668684836 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fab78056730 con 0x7fab8c145f30 2024-04-17T09:53:09.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.791+0000 7fab94600640 1 -- 172.21.15.32:0/668684836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab8c0cd390 con 0x7fab8c145f30 2024-04-17T09:53:09.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.791+0000 7fab89600640 1 -- 172.21.15.32:0/668684836 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3864 (secure 0 0 0) 0x7fab7805aa30 con 0x7fab8c145f30 2024-04-17T09:53:09.794 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:09.794 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/441201886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":441201886},{"type":"v1","addr":"172.21.15.144:6834","nonce":441201886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1066277472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1066277472},{"type":"v1","addr":"172.21.15.32:6836","nonce":1066277472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/201410023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":201410023},{"type":"v1","addr":"172.21.15.32:6837","nonce":201410023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3332924005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3332924005},{"type":"v1","addr":"172.21.15.144:6835","nonce":3332924005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-17T09:53:09.794 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 3 2024-04-17T09:53:09.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.791+0000 7fab94600640 1 -- 172.21.15.32:0/668684836 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab6c068380 msgr2=0x7fab6c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:09.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.791+0000 7fab94600640 1 --2- 172.21.15.32:0/668684836 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab6c068380 0x7fab6c06a840 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fab80009ae0 tx=0x7fab80009190 comp rx=0 tx=0).stop 2024-04-17T09:53:09.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.791+0000 7fab94600640 1 -- 172.21.15.32:0/668684836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab8c145f30 msgr2=0x7fab8c0af250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:09.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.791+0000 7fab94600640 1 --2- 172.21.15.32:0/668684836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab8c145f30 0x7fab8c0af250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fab7800e870 tx=0x7fab7800ec70 comp rx=0 tx=0).stop 2024-04-17T09:53:09.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.791+0000 7fab94600640 1 -- 172.21.15.32:0/668684836 shutdown_connections 2024-04-17T09:53:09.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.791+0000 7fab94600640 1 --2- 172.21.15.32:0/668684836 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab6c068380 0x7fab6c06a840 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:09.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.791+0000 7fab94600640 1 --2- 172.21.15.32:0/668684836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab8c15d8d0 0x7fab8c0af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:09.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.791+0000 7fab94600640 1 --2- 172.21.15.32:0/668684836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab8c145f30 0x7fab8c0af250 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:09.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.791+0000 7fab94600640 1 --2- 172.21.15.32:0/668684836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab8c0cce70 0x7fab8c0ac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:09.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.791+0000 7fab94600640 1 -- 172.21.15.32:0/668684836 >> 172.21.15.32:0/668684836 conn(0x7fab8c06ba30 msgr2=0x7fab8c15ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:09.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.791+0000 7fab94600640 1 -- 172.21.15.32:0/668684836 shutdown_connections 2024-04-17T09:53:09.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.791+0000 7fab94600640 1 -- 172.21.15.32:0/668684836 wait complete. 2024-04-17T09:53:09.815 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:53:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.951+0000 7f7187c00640 1 Processor -- start 2024-04-17T09:53:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.951+0000 7f7187c00640 1 -- start start 2024-04-17T09:53:09.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.951+0000 7f7187c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7180144e90 0x7f71800b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:09.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.955+0000 7f7187c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f718015c1b0 0x7f718006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:09.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.955+0000 7f7187c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7180072bb0 con 0x7f71800b4de0 2024-04-17T09:53:09.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.955+0000 7f7187c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7180145270 con 0x7f7180144e90 2024-04-17T09:53:09.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.955+0000 7f7187c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f71800b3dc0 con 0x7f718015c1b0 2024-04-17T09:53:09.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.955+0000 7f7185200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7180144e90 0x7f71800b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:09.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.955+0000 7f7185200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7180144e90 0x7f71800b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37446/0 (socket says 172.21.15.32:37446) 2024-04-17T09:53:09.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.955+0000 7f7185200640 1 -- 172.21.15.32:0/2144622685 learned_addr learned my addr 172.21.15.32:0/2144622685 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:09.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.955+0000 7f7185c00640 1 --2- 172.21.15.32:0/2144622685 >> v2:172.21.15.144:3300/0 conn(0x7f718015c1b0 0x7f718006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:09.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.955+0000 7f7185200640 1 -- 172.21.15.32:0/2144622685 >> v2:172.21.15.144:3300/0 conn(0x7f718015c1b0 msgr2=0x7f718006ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:09.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.955+0000 7f7185200640 1 --2- 172.21.15.32:0/2144622685 >> v2:172.21.15.144:3300/0 conn(0x7f718015c1b0 0x7f718006ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:09.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.955+0000 7f7185200640 1 -- 172.21.15.32:0/2144622685 >> v1:172.21.15.144:6789/0 conn(0x7f71800b4de0 legacy=0x7f718015baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:09.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.955+0000 7f7185200640 1 -- 172.21.15.32:0/2144622685 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71800b40a0 con 0x7f7180144e90 2024-04-17T09:53:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.955+0000 7f7185200640 1 --2- 172.21.15.32:0/2144622685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7180144e90 0x7f71800b48a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f718006ff30 tx=0x7f717001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b027c6ccc63d2024 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.959+0000 7f717fe00640 1 -- 172.21.15.32:0/2144622685 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71700b2070 con 0x7f7180144e90 2024-04-17T09:53:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.959+0000 7f717fe00640 1 -- 172.21.15.32:0/2144622685 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71700ad680 con 0x7f7180144e90 2024-04-17T09:53:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.959+0000 7f717fe00640 1 -- 172.21.15.32:0/2144622685 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71700ada50 con 0x7f7180144e90 2024-04-17T09:53:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.959+0000 7f7187c00640 1 -- 172.21.15.32:0/2144622685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7180144e90 msgr2=0x7f71800b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.959+0000 7f7187c00640 1 --2- 172.21.15.32:0/2144622685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7180144e90 0x7f71800b48a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f718006ff30 tx=0x7f717001c470 comp rx=0 tx=0).stop 2024-04-17T09:53:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.959+0000 7f7187c00640 1 -- 172.21.15.32:0/2144622685 shutdown_connections 2024-04-17T09:53:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.959+0000 7f7187c00640 1 --2- 172.21.15.32:0/2144622685 >> v2:172.21.15.144:3300/0 conn(0x7f718015c1b0 0x7f718006ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.959+0000 7f7187c00640 1 --2- 172.21.15.32:0/2144622685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7180144e90 0x7f71800b48a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.959+0000 7f7187c00640 1 -- 172.21.15.32:0/2144622685 >> 172.21.15.32:0/2144622685 conn(0x7f718006ba30 msgr2=0x7f718006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.959+0000 7f7187c00640 1 -- 172.21.15.32:0/2144622685 shutdown_connections 2024-04-17T09:53:09.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.959+0000 7f7187c00640 1 -- 172.21.15.32:0/2144622685 wait complete. 2024-04-17T09:53:09.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.959+0000 7f7187c00640 1 Processor -- start 2024-04-17T09:53:09.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.959+0000 7f7187c00640 1 -- start start 2024-04-17T09:53:09.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.959+0000 7f7187c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71800b4de0 0x7f7180156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:09.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.959+0000 7f7187c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7180144e90 0x7f718013a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:09.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.959+0000 7f7187c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f718015c1b0 0x7f718013ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:09.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.959+0000 7f7187c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7180166100 con 0x7f7180144e90 2024-04-17T09:53:09.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.959+0000 7f7187c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7180072880 con 0x7f718015c1b0 2024-04-17T09:53:09.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.959+0000 7f7187c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7180166280 con 0x7f71800b4de0 2024-04-17T09:53:09.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.959+0000 7f7177e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7180144e90 0x7f718013a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:09.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.959+0000 7f7177e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7180144e90 0x7f718013a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37456/0 (socket says 172.21.15.32:37456) 2024-04-17T09:53:09.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.959+0000 7f7177e00640 1 -- 172.21.15.32:0/805288355 learned_addr learned my addr 172.21.15.32:0/805288355 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:09.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.959+0000 7f7185200640 1 --2- 172.21.15.32:0/805288355 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71800b4de0 0x7f7180156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:09.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.963+0000 7f7185c00640 1 --2- 172.21.15.32:0/805288355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f718015c1b0 0x7f718013ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:09.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.963+0000 7f7177e00640 1 -- 172.21.15.32:0/805288355 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71800b4de0 msgr2=0x7f7180156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:09.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.963+0000 7f7177e00640 1 --2- 172.21.15.32:0/805288355 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71800b4de0 0x7f7180156f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:09.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.963+0000 7f7177e00640 1 -- 172.21.15.32:0/805288355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f718015c1b0 msgr2=0x7f718013ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:09.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.963+0000 7f7177e00640 1 --2- 172.21.15.32:0/805288355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f718015c1b0 0x7f718013ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:09.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.963+0000 7f7177e00640 1 -- 172.21.15.32:0/805288355 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718015b200 con 0x7f7180144e90 2024-04-17T09:53:09.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.963+0000 7f7177e00640 1 --2- 172.21.15.32:0/805288355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7180144e90 0x7f718013a650 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f716800c450 tx=0x7f716800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:09.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.963+0000 7f717d600640 1 -- 172.21.15.32:0/805288355 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7168012830 con 0x7f7180144e90 2024-04-17T09:53:09.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.963+0000 7f717d600640 1 -- 172.21.15.32:0/805288355 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7168017070 con 0x7f7180144e90 2024-04-17T09:53:09.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.963+0000 7f717d600640 1 -- 172.21.15.32:0/805288355 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7168012c30 con 0x7f7180144e90 2024-04-17T09:53:09.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.963+0000 7f7187c00640 1 -- 172.21.15.32:0/805288355 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f718015b4e0 con 0x7f7180144e90 2024-04-17T09:53:09.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.963+0000 7f7187c00640 1 -- 172.21.15.32:0/805288355 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71801673f0 con 0x7f7180144e90 2024-04-17T09:53:09.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.963+0000 7f7187c00640 1 -- 172.21.15.32:0/805288355 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7180068a90 con 0x7f7180144e90 2024-04-17T09:53:09.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.963+0000 7f717d600640 1 -- 172.21.15.32:0/805288355 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f716801c460 con 0x7f7180144e90 2024-04-17T09:53:09.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.963+0000 7f717d600640 1 --2- 172.21.15.32:0/805288355 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f715c068380 0x7f715c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:09.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.963+0000 7f717d600640 1 -- 172.21.15.32:0/805288355 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f7168086350 con 0x7f7180144e90 2024-04-17T09:53:09.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.967+0000 7f7185200640 1 --2- 172.21.15.32:0/805288355 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f715c068380 0x7f715c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:09.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.967+0000 7f7185200640 1 --2- 172.21.15.32:0/805288355 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f715c068380 0x7f715c06a840 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f71700af000 tx=0x7f71700a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:09.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:09.971+0000 7f717d600640 1 -- 172.21.15.32:0/805288355 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f7168056570 con 0x7f7180144e90 2024-04-17T09:53:10.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.135+0000 7f7187c00640 1 -- 172.21.15.32:0/805288355 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7180157490 con 0x7f7180144e90 2024-04-17T09:53:10.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.139+0000 7f717d600640 1 -- 172.21.15.32:0/805288355 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12896 (secure 0 0 0) 0x7f716805a870 con 0x7f7180144e90 2024-04-17T09:53:10.141 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:10.141 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:01.709523+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:53:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.139+0000 7f7187c00640 1 -- 172.21.15.32:0/805288355 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f715c068380 msgr2=0x7f715c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.139+0000 7f7187c00640 1 --2- 172.21.15.32:0/805288355 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f715c068380 0x7f715c06a840 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f71700af000 tx=0x7f71700a7000 comp rx=0 tx=0).stop 2024-04-17T09:53:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.139+0000 7f7187c00640 1 -- 172.21.15.32:0/805288355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7180144e90 msgr2=0x7f718013a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.139+0000 7f7187c00640 1 --2- 172.21.15.32:0/805288355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7180144e90 0x7f718013a650 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f716800c450 tx=0x7f716800c910 comp rx=0 tx=0).stop 2024-04-17T09:53:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.139+0000 7f7187c00640 1 -- 172.21.15.32:0/805288355 shutdown_connections 2024-04-17T09:53:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.139+0000 7f7187c00640 1 --2- 172.21.15.32:0/805288355 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f715c068380 0x7f715c06a840 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.139+0000 7f7187c00640 1 --2- 172.21.15.32:0/805288355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f718015c1b0 0x7f718013ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.139+0000 7f7187c00640 1 --2- 172.21.15.32:0/805288355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7180144e90 0x7f718013a650 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.139+0000 7f7187c00640 1 --2- 172.21.15.32:0/805288355 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71800b4de0 0x7f7180156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.139+0000 7f7187c00640 1 -- 172.21.15.32:0/805288355 >> 172.21.15.32:0/805288355 conn(0x7f718006ba30 msgr2=0x7f718013eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.139+0000 7f7187c00640 1 -- 172.21.15.32:0/805288355 shutdown_connections 2024-04-17T09:53:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.139+0000 7f7187c00640 1 -- 172.21.15.32:0/805288355 wait complete. 2024-04-17T09:53:10.163 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:53:10.163 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:53:10.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.303+0000 7f5691a00640 1 Processor -- start 2024-04-17T09:53:10.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.303+0000 7f5691a00640 1 -- start start 2024-04-17T09:53:10.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.307+0000 7f5691a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f568c06def0 0x7f568c0d43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:10.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.307+0000 7f5691a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f568c0d4da0 con 0x7f568c15eb00 2024-04-17T09:53:10.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.307+0000 7f5691a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f568c0d2470 con 0x7f568c0d48f0 2024-04-17T09:53:10.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.307+0000 7f5691a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f568c0d2750 con 0x7f568c06def0 2024-04-17T09:53:10.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.307+0000 7f568be00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f568c15eb00 0x7f568c06d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39136/0 (socket says 172.21.15.32:39136) 2024-04-17T09:53:10.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.307+0000 7f568be00640 1 -- 172.21.15.32:0/3662778174 learned_addr learned my addr 172.21.15.32:0/3662778174 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:10.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.307+0000 7f5683e00640 1 --2- 172.21.15.32:0/3662778174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f568c06def0 0x7f568c0d43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:10.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.307+0000 7f568b400640 1 -- 172.21.15.32:0/3662778174 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3564200842 0 0) 0x7f567c001120 con 0x7f568c15eb00 2024-04-17T09:53:10.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.307+0000 7f568b400640 1 -- 172.21.15.32:0/3662778174 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5670003530 con 0x7f568c15eb00 2024-04-17T09:53:10.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.307+0000 7f568b400640 1 -- 172.21.15.32:0/3662778174 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3090962132 0 0) 0x7f568c0d2470 con 0x7f568c0d48f0 2024-04-17T09:53:10.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.307+0000 7f568b400640 1 -- 172.21.15.32:0/3662778174 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f567c001120 con 0x7f568c0d48f0 2024-04-17T09:53:10.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.307+0000 7f5683e00640 1 -- 172.21.15.32:0/3662778174 >> v1:172.21.15.144:6789/0 conn(0x7f568c0d48f0 legacy=0x7f568c0cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:10.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.307+0000 7f5683e00640 1 -- 172.21.15.32:0/3662778174 >> v1:172.21.15.32:6789/0 conn(0x7f568c15eb00 legacy=0x7f568c06d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:10.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.307+0000 7f5683e00640 1 -- 172.21.15.32:0/3662778174 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568c0d2a30 con 0x7f568c06def0 2024-04-17T09:53:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.311+0000 7f5683e00640 1 --2- 172.21.15.32:0/3662778174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f568c06def0 0x7f568c0d43b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f56740097f0 tx=0x7f567401c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7527779b0f9ea438 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.311+0000 7f568b400640 1 -- 172.21.15.32:0/3662778174 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56740aa610 con 0x7f568c06def0 2024-04-17T09:53:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.311+0000 7f568b400640 1 -- 172.21.15.32:0/3662778174 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56740af070 con 0x7f568c06def0 2024-04-17T09:53:10.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.311+0000 7f568b400640 1 -- 172.21.15.32:0/3662778174 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56740aa9d0 con 0x7f568c06def0 2024-04-17T09:53:10.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.311+0000 7f5691a00640 1 -- 172.21.15.32:0/3662778174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f568c06def0 msgr2=0x7f568c0d43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:10.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.311+0000 7f5691a00640 1 --2- 172.21.15.32:0/3662778174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f568c06def0 0x7f568c0d43b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f56740097f0 tx=0x7f567401c4b0 comp rx=0 tx=0).stop 2024-04-17T09:53:10.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.311+0000 7f5691a00640 1 -- 172.21.15.32:0/3662778174 shutdown_connections 2024-04-17T09:53:10.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.311+0000 7f5691a00640 1 --2- 172.21.15.32:0/3662778174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f568c06def0 0x7f568c0d43b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:10.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.311+0000 7f5691a00640 1 -- 172.21.15.32:0/3662778174 >> 172.21.15.32:0/3662778174 conn(0x7f568c09d9e0 msgr2=0x7f568c08bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:10.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.311+0000 7f5691a00640 1 -- 172.21.15.32:0/3662778174 shutdown_connections 2024-04-17T09:53:10.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.311+0000 7f5691a00640 1 -- 172.21.15.32:0/3662778174 wait complete. 2024-04-17T09:53:10.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.311+0000 7f5691a00640 1 Processor -- start 2024-04-17T09:53:10.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.311+0000 7f5691a00640 1 -- start start 2024-04-17T09:53:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f5691a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f568c06def0 0x7f568c153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f5691a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f568c0d48f0 0x7f568c1583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f5691a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f568c15eb00 0x7f568c158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f5691a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f568c16c6c0 con 0x7f568c06def0 2024-04-17T09:53:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f5691a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f568c16c540 con 0x7f568c0d48f0 2024-04-17T09:53:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f5691a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f568c16c840 con 0x7f568c15eb00 2024-04-17T09:53:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f568be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f568c06def0 0x7f568c153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f568be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f568c06def0 0x7f568c153da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46608/0 (socket says 172.21.15.32:46608) 2024-04-17T09:53:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f568be00640 1 -- 172.21.15.32:0/3043559764 learned_addr learned my addr 172.21.15.32:0/3043559764 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f5683e00640 1 --2- 172.21.15.32:0/3043559764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f568c0d48f0 0x7f568c1583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f5691000640 1 --2- 172.21.15.32:0/3043559764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f568c15eb00 0x7f568c158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f568be00640 1 -- 172.21.15.32:0/3043559764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f568c15eb00 msgr2=0x7f568c158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:10.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f568be00640 1 --2- 172.21.15.32:0/3043559764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f568c15eb00 0x7f568c158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:10.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f568be00640 1 -- 172.21.15.32:0/3043559764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f568c0d48f0 msgr2=0x7f568c1583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:10.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f568be00640 1 --2- 172.21.15.32:0/3043559764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f568c0d48f0 0x7f568c1583c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:10.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f568be00640 1 -- 172.21.15.32:0/3043559764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568c159570 con 0x7f568c06def0 2024-04-17T09:53:10.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f5691000640 1 --2- 172.21.15.32:0/3043559764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f568c15eb00 0x7f568c158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:10.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f568be00640 1 --2- 172.21.15.32:0/3043559764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f568c06def0 0x7f568c153da0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f567c00c700 tx=0x7f567c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:10.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f5688c00640 1 -- 172.21.15.32:0/3043559764 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c017070 con 0x7f568c06def0 2024-04-17T09:53:10.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f5688c00640 1 -- 172.21.15.32:0/3043559764 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f567c005ac0 con 0x7f568c06def0 2024-04-17T09:53:10.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f5691a00640 1 -- 172.21.15.32:0/3043559764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568c159850 con 0x7f568c06def0 2024-04-17T09:53:10.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f5688c00640 1 -- 172.21.15.32:0/3043559764 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c007070 con 0x7f568c06def0 2024-04-17T09:53:10.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f5691a00640 1 -- 172.21.15.32:0/3043559764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f568c0b4400 con 0x7f568c06def0 2024-04-17T09:53:10.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f5691a00640 1 -- 172.21.15.32:0/3043559764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f568c04fda0 con 0x7f568c06def0 2024-04-17T09:53:10.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.315+0000 7f5688c00640 1 -- 172.21.15.32:0/3043559764 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f567c01a070 con 0x7f568c06def0 2024-04-17T09:53:10.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.319+0000 7f5688c00640 1 --2- 172.21.15.32:0/3043559764 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5668068380 0x7f566806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:10.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.319+0000 7f5688c00640 1 -- 172.21.15.32:0/3043559764 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f567c085c70 con 0x7f568c06def0 2024-04-17T09:53:10.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.319+0000 7f5683e00640 1 --2- 172.21.15.32:0/3043559764 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5668068380 0x7f566806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:10.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.319+0000 7f5683e00640 1 --2- 172.21.15.32:0/3043559764 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5668068380 0x7f566806a840 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f56740ae000 tx=0x7f56740a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:10.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.323+0000 7f5688c00640 1 -- 172.21.15.32:0/3043559764 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f568c0d4da0 con 0x7f568c06def0 2024-04-17T09:53:10.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.487+0000 7f5691a00640 1 -- 172.21.15.32:0/3043559764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f568c0cf5a0 con 0x7f568c06def0 2024-04-17T09:53:10.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.487+0000 7f5688c00640 1 -- 172.21.15.32:0/3043559764 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7f567c0564e0 con 0x7f568c06def0 2024-04-17T09:53:10.491 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:53:10.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.491+0000 7f5691a00640 1 -- 172.21.15.32:0/3043559764 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5668068380 msgr2=0x7f566806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:10.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.491+0000 7f5691a00640 1 --2- 172.21.15.32:0/3043559764 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5668068380 0x7f566806a840 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f56740ae000 tx=0x7f56740a7000 comp rx=0 tx=0).stop 2024-04-17T09:53:10.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.491+0000 7f5691a00640 1 -- 172.21.15.32:0/3043559764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f568c06def0 msgr2=0x7f568c153da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:10.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.491+0000 7f5691a00640 1 --2- 172.21.15.32:0/3043559764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f568c06def0 0x7f568c153da0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f567c00c700 tx=0x7f567c00cbc0 comp rx=0 tx=0).stop 2024-04-17T09:53:10.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.491+0000 7f5691a00640 1 -- 172.21.15.32:0/3043559764 shutdown_connections 2024-04-17T09:53:10.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.491+0000 7f5691a00640 1 --2- 172.21.15.32:0/3043559764 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5668068380 0x7f566806a840 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:10.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.491+0000 7f5691a00640 1 --2- 172.21.15.32:0/3043559764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f568c15eb00 0x7f568c158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:10.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.491+0000 7f5691a00640 1 --2- 172.21.15.32:0/3043559764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f568c0d48f0 0x7f568c1583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:10.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.491+0000 7f5691a00640 1 --2- 172.21.15.32:0/3043559764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f568c06def0 0x7f568c153da0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:10.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.491+0000 7f5691a00640 1 -- 172.21.15.32:0/3043559764 >> 172.21.15.32:0/3043559764 conn(0x7f568c09d9e0 msgr2=0x7f568c0c8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:10.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.491+0000 7f5691a00640 1 -- 172.21.15.32:0/3043559764 shutdown_connections 2024-04-17T09:53:10.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.491+0000 7f5691a00640 1 -- 172.21.15.32:0/3043559764 wait complete. 2024-04-17T09:53:10.512 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T09:53:10.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.647+0000 7f3861800640 1 Processor -- start 2024-04-17T09:53:10.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.647+0000 7f3861800640 1 -- start start 2024-04-17T09:53:10.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.651+0000 7f3861800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f385c145f30 0x7f385c071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:10.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.651+0000 7f3861800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f385c15d8d0 0x7f385c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:10.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.651+0000 7f3861800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f385c0cdac0 con 0x7f385c0cce70 2024-04-17T09:53:10.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.651+0000 7f3861800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f385c06f400 con 0x7f385c15d8d0 2024-04-17T09:53:10.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.651+0000 7f3861800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f385c146310 con 0x7f385c145f30 2024-04-17T09:53:10.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.651+0000 7f385b400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f385c0cce70 0x7f385c0cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39138/0 (socket says 172.21.15.32:39138) 2024-04-17T09:53:10.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.651+0000 7f385b400640 1 -- 172.21.15.32:0/2476534513 learned_addr learned my addr 172.21.15.32:0/2476534513 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:10.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.651+0000 7f3853e00640 1 --2- 172.21.15.32:0/2476534513 >> v2:172.21.15.144:3300/0 conn(0x7f385c15d8d0 0x7f385c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:10.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.651+0000 7f385aa00640 1 --2- 172.21.15.32:0/2476534513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f385c145f30 0x7f385c071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:10.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.651+0000 7f385a000640 1 -- 172.21.15.32:0/2476534513 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1366130375 0 0) 0x7f385c0cdac0 con 0x7f385c0cce70 2024-04-17T09:53:10.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.651+0000 7f385a000640 1 -- 172.21.15.32:0/2476534513 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3840003530 con 0x7f385c0cce70 2024-04-17T09:53:10.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.651+0000 7f3853e00640 1 -- 172.21.15.32:0/2476534513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f385c145f30 msgr2=0x7f385c071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:10.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.651+0000 7f3853e00640 1 --2- 172.21.15.32:0/2476534513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f385c145f30 0x7f385c071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:10.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.651+0000 7f3853e00640 1 -- 172.21.15.32:0/2476534513 >> v1:172.21.15.32:6789/0 conn(0x7f385c0cce70 legacy=0x7f385c0cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:10.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.651+0000 7f3853e00640 1 -- 172.21.15.32:0/2476534513 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385c15cdf0 con 0x7f385c15d8d0 2024-04-17T09:53:10.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.655+0000 7f3853e00640 1 --2- 172.21.15.32:0/2476534513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f385c15d8d0 0x7f385c072330 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f38440097c0 tx=0x7f384401ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=365bde8db971ba1b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:10.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.655+0000 7f385a000640 1 -- 172.21.15.32:0/2476534513 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38440b5070 con 0x7f385c15d8d0 2024-04-17T09:53:10.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.655+0000 7f385a000640 1 -- 172.21.15.32:0/2476534513 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38440a8990 con 0x7f385c15d8d0 2024-04-17T09:53:10.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.655+0000 7f385a000640 1 -- 172.21.15.32:0/2476534513 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38440a8d60 con 0x7f385c15d8d0 2024-04-17T09:53:10.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.655+0000 7f3861800640 1 -- 172.21.15.32:0/2476534513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f385c15d8d0 msgr2=0x7f385c072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:10.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.655+0000 7f3861800640 1 --2- 172.21.15.32:0/2476534513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f385c15d8d0 0x7f385c072330 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f38440097c0 tx=0x7f384401ec90 comp rx=0 tx=0).stop 2024-04-17T09:53:10.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.655+0000 7f3861800640 1 -- 172.21.15.32:0/2476534513 shutdown_connections 2024-04-17T09:53:10.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.655+0000 7f3861800640 1 --2- 172.21.15.32:0/2476534513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f385c15d8d0 0x7f385c072330 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:10.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.655+0000 7f3861800640 1 --2- 172.21.15.32:0/2476534513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f385c145f30 0x7f385c071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:10.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.655+0000 7f3861800640 1 -- 172.21.15.32:0/2476534513 >> 172.21.15.32:0/2476534513 conn(0x7f385c06ba30 msgr2=0x7f385c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:10.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.655+0000 7f3861800640 1 -- 172.21.15.32:0/2476534513 shutdown_connections 2024-04-17T09:53:10.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.655+0000 7f3861800640 1 -- 172.21.15.32:0/2476534513 wait complete. 2024-04-17T09:53:10.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.655+0000 7f3861800640 1 Processor -- start 2024-04-17T09:53:10.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.655+0000 7f3861800640 1 -- start start 2024-04-17T09:53:10.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f3861800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f385c0cce70 0x7f385c0b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:10.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f3861800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f385c145f30 0x7f385c1f47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:10.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f3861800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f385c15d8d0 0x7f385c1f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:10.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f3861800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f385c15cb90 con 0x7f385c15d8d0 2024-04-17T09:53:10.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f3861800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f385c15d440 con 0x7f385c145f30 2024-04-17T09:53:10.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f3861800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f385c1601d0 con 0x7f385c0cce70 2024-04-17T09:53:10.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f385b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f385c15d8d0 0x7f385c1f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:10.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f385b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f385c15d8d0 0x7f385c1f4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46618/0 (socket says 172.21.15.32:46618) 2024-04-17T09:53:10.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f385b400640 1 -- 172.21.15.32:0/2911276451 learned_addr learned my addr 172.21.15.32:0/2911276451 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:10.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f385aa00640 1 --2- 172.21.15.32:0/2911276451 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f385c0cce70 0x7f385c0b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:10.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f3853e00640 1 --2- 172.21.15.32:0/2911276451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f385c145f30 0x7f385c1f47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:10.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f385b400640 1 -- 172.21.15.32:0/2911276451 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f385c0cce70 msgr2=0x7f385c0b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:10.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f385b400640 1 --2- 172.21.15.32:0/2911276451 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f385c0cce70 0x7f385c0b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:10.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f385b400640 1 -- 172.21.15.32:0/2911276451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f385c145f30 msgr2=0x7f385c1f47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:10.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f385b400640 1 --2- 172.21.15.32:0/2911276451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f385c145f30 0x7f385c1f47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:10.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f385b400640 1 -- 172.21.15.32:0/2911276451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385c0cf4c0 con 0x7f385c15d8d0 2024-04-17T09:53:10.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f385b400640 1 --2- 172.21.15.32:0/2911276451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f385c15d8d0 0x7f385c1f4cf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f385400cb40 tx=0x7f385400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:10.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f3852a00640 1 -- 172.21.15.32:0/2911276451 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3854017070 con 0x7f385c15d8d0 2024-04-17T09:53:10.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f3852a00640 1 -- 172.21.15.32:0/2911276451 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f385400aca0 con 0x7f385c15d8d0 2024-04-17T09:53:10.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f3852a00640 1 -- 172.21.15.32:0/2911276451 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3854012620 con 0x7f385c15d8d0 2024-04-17T09:53:10.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f3861800640 1 -- 172.21.15.32:0/2911276451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f385c1f52e0 con 0x7f385c15d8d0 2024-04-17T09:53:10.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f3861800640 1 -- 172.21.15.32:0/2911276451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f385c1f58a0 con 0x7f385c15d8d0 2024-04-17T09:53:10.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f3861800640 1 -- 172.21.15.32:0/2911276451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f385c068a90 con 0x7f385c15d8d0 2024-04-17T09:53:10.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.659+0000 7f3852a00640 1 -- 172.21.15.32:0/2911276451 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f385401a070 con 0x7f385c15d8d0 2024-04-17T09:53:10.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.663+0000 7f3852a00640 1 --2- 172.21.15.32:0/2911276451 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3838068380 0x7f383806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:10.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.663+0000 7f3852a00640 1 -- 172.21.15.32:0/2911276451 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f38540864a0 con 0x7f385c15d8d0 2024-04-17T09:53:10.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.663+0000 7f385aa00640 1 --2- 172.21.15.32:0/2911276451 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3838068380 0x7f383806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:10.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.663+0000 7f385aa00640 1 --2- 172.21.15.32:0/2911276451 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3838068380 0x7f383806a840 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f384c009b60 tx=0x7f384c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:10.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.667+0000 7f3852a00640 1 -- 172.21.15.32:0/2911276451 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3854056630 con 0x7f385c15d8d0 2024-04-17T09:53:10.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.835+0000 7f3861800640 1 -- 172.21.15.32:0/2911276451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f385c072f90 con 0x7f385c15d8d0 2024-04-17T09:53:10.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.835+0000 7f3852a00640 1 -- 172.21.15.32:0/2911276451 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f385405a930 con 0x7f385c15d8d0 2024-04-17T09:53:10.839 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:10.839 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-17T09:53:10.839 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:53:10.839 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-04-17T09:53:10.839 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-04-17T09:53:10.839 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:53:10.839 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-17T09:53:10.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.839+0000 7f3861800640 1 -- 172.21.15.32:0/2911276451 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3838068380 msgr2=0x7f383806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:10.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.839+0000 7f3861800640 1 --2- 172.21.15.32:0/2911276451 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3838068380 0x7f383806a840 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f384c009b60 tx=0x7f384c009190 comp rx=0 tx=0).stop 2024-04-17T09:53:10.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.839+0000 7f3861800640 1 -- 172.21.15.32:0/2911276451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f385c15d8d0 msgr2=0x7f385c1f4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:10.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.839+0000 7f3861800640 1 --2- 172.21.15.32:0/2911276451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f385c15d8d0 0x7f385c1f4cf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f385400cb40 tx=0x7f385400a060 comp rx=0 tx=0).stop 2024-04-17T09:53:10.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.839+0000 7f3861800640 1 -- 172.21.15.32:0/2911276451 shutdown_connections 2024-04-17T09:53:10.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.839+0000 7f3861800640 1 --2- 172.21.15.32:0/2911276451 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3838068380 0x7f383806a840 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:10.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.839+0000 7f3861800640 1 --2- 172.21.15.32:0/2911276451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f385c15d8d0 0x7f385c1f4cf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:10.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.839+0000 7f3861800640 1 --2- 172.21.15.32:0/2911276451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f385c145f30 0x7f385c1f47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:10.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.839+0000 7f3861800640 1 --2- 172.21.15.32:0/2911276451 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f385c0cce70 0x7f385c0b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:10.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.839+0000 7f3861800640 1 -- 172.21.15.32:0/2911276451 >> 172.21.15.32:0/2911276451 conn(0x7f385c06ba30 msgr2=0x7f385c0b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:10.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.839+0000 7f3861800640 1 -- 172.21.15.32:0/2911276451 shutdown_connections 2024-04-17T09:53:10.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:10.839+0000 7f3861800640 1 -- 172.21.15.32:0/2911276451 wait complete. 2024-04-17T09:53:10.860 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T09:53:10.860 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T09:53:11.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feebb200640 1 Processor -- start 2024-04-17T09:53:11.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feebb200640 1 -- start start 2024-04-17T09:53:11.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feebb200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feebc0d0600 0x7feebc0d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:11.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feebb200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feebc0ad380 con 0x7feebc15c9b0 2024-04-17T09:53:11.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feebb200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feebc15d670 con 0x7feebc156ba0 2024-04-17T09:53:11.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feebb200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7feebc06dc80 con 0x7feebc0d0600 2024-04-17T09:53:11.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeba800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7feebc15c9b0 0x7feebc15ce40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39140/0 (socket says 172.21.15.32:39140) 2024-04-17T09:53:11.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeba800640 1 -- 172.21.15.32:0/45517612 learned_addr learned my addr 172.21.15.32:0/45517612 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:11.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeb3e00640 1 --2- 172.21.15.32:0/45517612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feebc0d0600 0x7feebc0d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:11.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeb9400640 1 -- 172.21.15.32:0/45517612 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2777954914 0 0) 0x7feeb40014a0 con 0x7feebc15c9b0 2024-04-17T09:53:11.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeb9400640 1 -- 172.21.15.32:0/45517612 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feea0003530 con 0x7feebc15c9b0 2024-04-17T09:53:11.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeb9400640 1 -- 172.21.15.32:0/45517612 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2527898597 0 0) 0x7feebc15d670 con 0x7feebc156ba0 2024-04-17T09:53:11.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeb9400640 1 -- 172.21.15.32:0/45517612 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feeb40014a0 con 0x7feebc156ba0 2024-04-17T09:53:11.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeb9400640 1 -- 172.21.15.32:0/45517612 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4097318388 0 0) 0x7feea0003530 con 0x7feebc15c9b0 2024-04-17T09:53:11.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeb9400640 1 -- 172.21.15.32:0/45517612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feebc0d0600 msgr2=0x7feebc0d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:11.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeb9400640 1 --2- 172.21.15.32:0/45517612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feebc0d0600 0x7feebc0d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:11.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeb9400640 1 -- 172.21.15.32:0/45517612 >> v1:172.21.15.144:6789/0 conn(0x7feebc156ba0 legacy=0x7feebc06e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:11.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeb9400640 1 -- 172.21.15.32:0/45517612 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feebc06df60 con 0x7feebc15c9b0 2024-04-17T09:53:11.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeb9400640 1 -- 172.21.15.32:0/45517612 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7feeb4001bd0 con 0x7feebc15c9b0 2024-04-17T09:53:11.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeb9400640 1 -- 172.21.15.32:0/45517612 >> v1:172.21.15.32:6789/0 conn(0x7feebc15c9b0 legacy=0x7feebc15ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:11.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeb9400640 1 --2- 172.21.15.32:0/45517612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feea0003b80 0x7feea0003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:11.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeb9400640 1 --2- 172.21.15.32:0/45517612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feebc0d0600 0x7feea0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:11.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeb9400640 1 --2- 172.21.15.32:0/45517612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feea0004d50 0x7feea00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:11.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeb9400640 1 -- 172.21.15.32:0/45517612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7feebc06dc80 con 0x7feea0004d50 2024-04-17T09:53:11.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeb9400640 1 -- 172.21.15.32:0/45517612 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feea000bab0 con 0x7feea0003b80 2024-04-17T09:53:11.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeb9400640 1 -- 172.21.15.32:0/45517612 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7feea000bcb0 con 0x7feebc0d0600 2024-04-17T09:53:11.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeba800640 1 --2- 172.21.15.32:0/45517612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feea0004d50 0x7feea00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:11.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeb9e00640 1 --2- 172.21.15.32:0/45517612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feea0003b80 0x7feea0003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:11.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeb3e00640 1 --2- 172.21.15.32:0/45517612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feebc0d0600 0x7feea0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:11.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeba800640 1 -- 172.21.15.32:0/45517612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feebc0d0600 msgr2=0x7feea0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:11.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeba800640 1 --2- 172.21.15.32:0/45517612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feebc0d0600 0x7feea0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:11.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeba800640 1 -- 172.21.15.32:0/45517612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feea0003b80 msgr2=0x7feea0003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:11.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeba800640 1 --2- 172.21.15.32:0/45517612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feea0003b80 0x7feea0003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:11.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.003+0000 7feeba800640 1 -- 172.21.15.32:0/45517612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea0003530 con 0x7feea0004d50 2024-04-17T09:53:11.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.007+0000 7feeba800640 1 --2- 172.21.15.32:0/45517612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feea0004d50 0x7feea00051d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feeb400a910 tx=0x7feeb401f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba572e4279f8828 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:11.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.007+0000 7feeb9400640 1 -- 172.21.15.32:0/45517612 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb400a530 con 0x7feea0004d50 2024-04-17T09:53:11.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.007+0000 7feeb9400640 1 -- 172.21.15.32:0/45517612 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb40b1070 con 0x7feea0004d50 2024-04-17T09:53:11.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.007+0000 7feeb9400640 1 -- 172.21.15.32:0/45517612 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb40ac890 con 0x7feea0004d50 2024-04-17T09:53:11.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.007+0000 7feebb200640 1 -- 172.21.15.32:0/45517612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feea0004d50 msgr2=0x7feea00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:11.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.007+0000 7feebb200640 1 --2- 172.21.15.32:0/45517612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feea0004d50 0x7feea00051d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feeb400a910 tx=0x7feeb401f9a0 comp rx=0 tx=0).stop 2024-04-17T09:53:11.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.007+0000 7feeba800640 1 -- 172.21.15.32:0/45517612 reap_dead start 2024-04-17T09:53:11.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.007+0000 7feebb200640 1 -- 172.21.15.32:0/45517612 shutdown_connections 2024-04-17T09:53:11.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.007+0000 7feebb200640 1 -- 172.21.15.32:0/45517612 >> 172.21.15.32:0/45517612 conn(0x7feebc0b2170 msgr2=0x7feebc0b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:11.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.007+0000 7feebb200640 1 -- 172.21.15.32:0/45517612 shutdown_connections 2024-04-17T09:53:11.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.007+0000 7feebb200640 1 -- 172.21.15.32:0/45517612 wait complete. 2024-04-17T09:53:11.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feebb200640 1 Processor -- start 2024-04-17T09:53:11.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feebb200640 1 -- start start 2024-04-17T09:53:11.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feebb200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feea0004d50 0x7feebc14e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:11.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feebb200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feebc099530 0x7feebc0999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:11.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feebb200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feebc157c50 0x7feebc1580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:11.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feebb200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7feebc0d0e10 con 0x7feebc099530 2024-04-17T09:53:11.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feebb200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feebc0d0c90 con 0x7feea0004d50 2024-04-17T09:53:11.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feebb200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7feebc0d0f90 con 0x7feebc157c50 2024-04-17T09:53:11.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feeb3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feebc099530 0x7feebc0999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:11.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feeb9e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feea0004d50 0x7feebc14e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:11.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feeba800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feebc157c50 0x7feebc1580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:11.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feeb3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feebc099530 0x7feebc0999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46628/0 (socket says 172.21.15.32:46628) 2024-04-17T09:53:11.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feeba800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feebc157c50 0x7feebc1580d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:56168/0 (socket says 172.21.15.32:56168) 2024-04-17T09:53:11.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feeb3e00640 1 -- 172.21.15.32:0/793705380 learned_addr learned my addr 172.21.15.32:0/793705380 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:11.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feeb3e00640 1 -- 172.21.15.32:0/793705380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feebc157c50 msgr2=0x7feebc1580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:11.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feeb3e00640 1 --2- 172.21.15.32:0/793705380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feebc157c50 0x7feebc1580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:11.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feeb3e00640 1 -- 172.21.15.32:0/793705380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feea0004d50 msgr2=0x7feebc14e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:11.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feeb3e00640 1 --2- 172.21.15.32:0/793705380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feea0004d50 0x7feebc14e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:11.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feeb3e00640 1 -- 172.21.15.32:0/793705380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feebc158c60 con 0x7feebc099530 2024-04-17T09:53:11.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feeb3e00640 1 --2- 172.21.15.32:0/793705380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feebc099530 0x7feebc0999b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feea40119d0 tx=0x7feea4011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:11.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feeb2000640 1 -- 172.21.15.32:0/793705380 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea400faf0 con 0x7feebc099530 2024-04-17T09:53:11.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feeb2000640 1 -- 172.21.15.32:0/793705380 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feea400fc90 con 0x7feebc099530 2024-04-17T09:53:11.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feeb2000640 1 -- 172.21.15.32:0/793705380 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea4014440 con 0x7feebc099530 2024-04-17T09:53:11.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feebb200640 1 -- 172.21.15.32:0/793705380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feebc1536b0 con 0x7feebc099530 2024-04-17T09:53:11.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.011+0000 7feebb200640 1 -- 172.21.15.32:0/793705380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feebc153950 con 0x7feebc099530 2024-04-17T09:53:11.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.015+0000 7feebb200640 1 -- 172.21.15.32:0/793705380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feebc04fd00 con 0x7feebc099530 2024-04-17T09:53:11.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.015+0000 7feeb2000640 1 -- 172.21.15.32:0/793705380 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7feea4012040 con 0x7feebc099530 2024-04-17T09:53:11.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.015+0000 7feeb2000640 1 --2- 172.21.15.32:0/793705380 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee98068380 0x7fee9806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:11.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.015+0000 7feeb2000640 1 -- 172.21.15.32:0/793705380 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7feea4086bf0 con 0x7feebc099530 2024-04-17T09:53:11.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.015+0000 7feeb9e00640 1 --2- 172.21.15.32:0/793705380 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee98068380 0x7fee9806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:11.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.015+0000 7feeb9e00640 1 --2- 172.21.15.32:0/793705380 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee98068380 0x7fee9806a840 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feeac007fd0 tx=0x7feeac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:11.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.019+0000 7feeb2000640 1 -- 172.21.15.32:0/793705380 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7feea4019030 con 0x7feebc099530 2024-04-17T09:53:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.183+0000 7feebb200640 1 -- 172.21.15.32:0/793705380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7feebc06f6c0 con 0x7feebc099530 2024-04-17T09:53:11.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.743+0000 7feeb2000640 1 -- 172.21.15.32:0/793705380 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7feea4057520 con 0x7feebc099530 2024-04-17T09:53:11.746 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-04-17T09:53:11.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.747+0000 7feebb200640 1 -- 172.21.15.32:0/793705380 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee98068380 msgr2=0x7fee9806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:11.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.747+0000 7feebb200640 1 --2- 172.21.15.32:0/793705380 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee98068380 0x7fee9806a840 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feeac007fd0 tx=0x7feeac009040 comp rx=0 tx=0).stop 2024-04-17T09:53:11.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.747+0000 7feebb200640 1 -- 172.21.15.32:0/793705380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feebc099530 msgr2=0x7feebc0999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:11.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.747+0000 7feebb200640 1 --2- 172.21.15.32:0/793705380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feebc099530 0x7feebc0999b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feea40119d0 tx=0x7feea4011dd0 comp rx=0 tx=0).stop 2024-04-17T09:53:11.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.747+0000 7feebb200640 1 -- 172.21.15.32:0/793705380 shutdown_connections 2024-04-17T09:53:11.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.747+0000 7feebb200640 1 --2- 172.21.15.32:0/793705380 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee98068380 0x7fee9806a840 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:11.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.747+0000 7feebb200640 1 --2- 172.21.15.32:0/793705380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feebc157c50 0x7feebc1580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:11.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.747+0000 7feebb200640 1 --2- 172.21.15.32:0/793705380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feebc099530 0x7feebc0999b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:11.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.747+0000 7feebb200640 1 --2- 172.21.15.32:0/793705380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feea0004d50 0x7feebc14e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:11.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.747+0000 7feebb200640 1 -- 172.21.15.32:0/793705380 >> 172.21.15.32:0/793705380 conn(0x7feebc0b2170 msgr2=0x7feebc0b14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:11.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.747+0000 7feebb200640 1 -- 172.21.15.32:0/793705380 shutdown_connections 2024-04-17T09:53:11.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.747+0000 7feebb200640 1 -- 172.21.15.32:0/793705380 wait complete. 2024-04-17T09:53:11.777 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-17T09:53:11.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.923+0000 7f0b6b800640 1 Processor -- start 2024-04-17T09:53:11.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.923+0000 7f0b6b800640 1 -- start start 2024-04-17T09:53:11.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.923+0000 7f0b6b800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b640cf370 0x7f0b640cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:11.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.923+0000 7f0b6b800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b64071e60 0x7f0b640722d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:11.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.923+0000 7f0b6b800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b640d0130 con 0x7f0b6406ba30 2024-04-17T09:53:11.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.923+0000 7f0b6b800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0b6415d090 con 0x7f0b64071e60 2024-04-17T09:53:11.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.923+0000 7f0b6b800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0b6415d370 con 0x7f0b640cf370 2024-04-17T09:53:11.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.923+0000 7f0b69800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b64071e60 0x7f0b640722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:11.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.923+0000 7f0b69800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b64071e60 0x7f0b640722d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46644/0 (socket says 172.21.15.32:46644) 2024-04-17T09:53:11.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.923+0000 7f0b69800640 1 -- 172.21.15.32:0/906724978 learned_addr learned my addr 172.21.15.32:0/906724978 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:11.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.923+0000 7f0b5be00640 1 --2- 172.21.15.32:0/906724978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b640cf370 0x7f0b640cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:11.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.927+0000 7f0b69800640 1 -- 172.21.15.32:0/906724978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b640cf370 msgr2=0x7f0b640cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:11.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.927+0000 7f0b69800640 1 --2- 172.21.15.32:0/906724978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b640cf370 0x7f0b640cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:11.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.927+0000 7f0b69800640 1 -- 172.21.15.32:0/906724978 >> v1:172.21.15.144:6789/0 conn(0x7f0b6406ba30 legacy=0x7f0b6406be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:11.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.927+0000 7f0b69800640 1 -- 172.21.15.32:0/906724978 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6415d650 con 0x7f0b64071e60 2024-04-17T09:53:11.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.927+0000 7f0b69800640 1 --2- 172.21.15.32:0/906724978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b64071e60 0x7f0b640722d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009e50 tx=0x7f0b5c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87be678dac092af server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:11.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.927+0000 7f0b63e00640 1 -- 172.21.15.32:0/906724978 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c0a5920 con 0x7f0b64071e60 2024-04-17T09:53:11.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.927+0000 7f0b63e00640 1 -- 172.21.15.32:0/906724978 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b5c0a5ac0 con 0x7f0b64071e60 2024-04-17T09:53:11.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.927+0000 7f0b63e00640 1 -- 172.21.15.32:0/906724978 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c0ad3f0 con 0x7f0b64071e60 2024-04-17T09:53:11.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.927+0000 7f0b6b800640 1 -- 172.21.15.32:0/906724978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b64071e60 msgr2=0x7f0b640722d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:11.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.927+0000 7f0b6b800640 1 --2- 172.21.15.32:0/906724978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b64071e60 0x7f0b640722d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009e50 tx=0x7f0b5c01ca30 comp rx=0 tx=0).stop 2024-04-17T09:53:11.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.927+0000 7f0b6b800640 1 -- 172.21.15.32:0/906724978 shutdown_connections 2024-04-17T09:53:11.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.927+0000 7f0b6b800640 1 --2- 172.21.15.32:0/906724978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b64071e60 0x7f0b640722d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:11.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.927+0000 7f0b6b800640 1 --2- 172.21.15.32:0/906724978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b640cf370 0x7f0b640cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:11.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.927+0000 7f0b6b800640 1 -- 172.21.15.32:0/906724978 >> 172.21.15.32:0/906724978 conn(0x7f0b640b2120 msgr2=0x7f0b640b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:11.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.927+0000 7f0b6b800640 1 -- 172.21.15.32:0/906724978 shutdown_connections 2024-04-17T09:53:11.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.927+0000 7f0b6b800640 1 -- 172.21.15.32:0/906724978 wait complete. 2024-04-17T09:53:11.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.931+0000 7f0b6b800640 1 Processor -- start 2024-04-17T09:53:11.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.931+0000 7f0b6b800640 1 -- start start 2024-04-17T09:53:11.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.931+0000 7f0b6b800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b6406ba30 0x7f0b64145580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:11.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.931+0000 7f0b6b800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b64071e60 0x7f0b6413f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:11.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.931+0000 7f0b6b800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b640cf370 0x7f0b640c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:11.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.931+0000 7f0b6b800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0b64146230 con 0x7f0b6406ba30 2024-04-17T09:53:11.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.931+0000 7f0b6b800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6415dd80 con 0x7f0b64071e60 2024-04-17T09:53:11.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.931+0000 7f0b6b800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0b641463b0 con 0x7f0b640cf370 2024-04-17T09:53:11.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.931+0000 7f0b68e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b6406ba30 0x7f0b64145580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:11.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.931+0000 7f0b68e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b6406ba30 0x7f0b64145580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46648/0 (socket says 172.21.15.32:46648) 2024-04-17T09:53:11.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.931+0000 7f0b68e00640 1 -- 172.21.15.32:0/1358849862 learned_addr learned my addr 172.21.15.32:0/1358849862 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:11.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.931+0000 7f0b5be00640 1 --2- 172.21.15.32:0/1358849862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b64071e60 0x7f0b6413f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:11.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.931+0000 7f0b69800640 1 --2- 172.21.15.32:0/1358849862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b640cf370 0x7f0b640c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:11.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.931+0000 7f0b68e00640 1 -- 172.21.15.32:0/1358849862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b640cf370 msgr2=0x7f0b640c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:11.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.931+0000 7f0b68e00640 1 --2- 172.21.15.32:0/1358849862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b640cf370 0x7f0b640c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:11.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.931+0000 7f0b68e00640 1 -- 172.21.15.32:0/1358849862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b64071e60 msgr2=0x7f0b6413f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:11.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.931+0000 7f0b68e00640 1 --2- 172.21.15.32:0/1358849862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b64071e60 0x7f0b6413f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:11.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.931+0000 7f0b68e00640 1 -- 172.21.15.32:0/1358849862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b64144240 con 0x7f0b6406ba30 2024-04-17T09:53:11.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.931+0000 7f0b68e00640 1 --2- 172.21.15.32:0/1358849862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b6406ba30 0x7f0b64145580 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400c420 tx=0x7f0b5400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:11.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.931+0000 7f0b61600640 1 -- 172.21.15.32:0/1358849862 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b540052d0 con 0x7f0b6406ba30 2024-04-17T09:53:11.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.931+0000 7f0b61600640 1 -- 172.21.15.32:0/1358849862 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b54005470 con 0x7f0b6406ba30 2024-04-17T09:53:11.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.931+0000 7f0b61600640 1 -- 172.21.15.32:0/1358849862 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b54007260 con 0x7f0b6406ba30 2024-04-17T09:53:11.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.931+0000 7f0b6b800640 1 -- 172.21.15.32:0/1358849862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b641444e0 con 0x7f0b6406ba30 2024-04-17T09:53:11.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.935+0000 7f0b6b800640 1 -- 172.21.15.32:0/1358849862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b64144aa0 con 0x7f0b6406ba30 2024-04-17T09:53:11.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.935+0000 7f0b6b800640 1 -- 172.21.15.32:0/1358849862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b64068a90 con 0x7f0b6406ba30 2024-04-17T09:53:11.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.935+0000 7f0b61600640 1 -- 172.21.15.32:0/1358849862 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0b54007400 con 0x7f0b6406ba30 2024-04-17T09:53:11.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.935+0000 7f0b61600640 1 --2- 172.21.15.32:0/1358849862 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0b40068380 0x7f0b4006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:11.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.935+0000 7f0b61600640 1 -- 172.21.15.32:0/1358849862 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7f0b54085c50 con 0x7f0b6406ba30 2024-04-17T09:53:11.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.935+0000 7f0b5be00640 1 --2- 172.21.15.32:0/1358849862 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0b40068380 0x7f0b4006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:11.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.935+0000 7f0b5be00640 1 --2- 172.21.15.32:0/1358849862 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0b40068380 0x7f0b4006a840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0b641676b0 tx=0x7f0b4c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:11.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:11.939+0000 7f0b61600640 1 -- 172.21.15.32:0/1358849862 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0b5408b020 con 0x7f0b6406ba30 2024-04-17T09:53:12.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.103+0000 7f0b6b800640 1 -- 172.21.15.32:0/1358849862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0b6406c850 con 0x7f0b6406ba30 2024-04-17T09:53:12.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.747+0000 7f0b61600640 1 -- 172.21.15.32:0/1358849862 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) ==== 170+0+0 (secure 0 0 0) 0x7f0b54055da0 con 0x7f0b6406ba30 2024-04-17T09:53:12.750 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-04-17T09:53:12.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.751+0000 7f0b6b800640 1 -- 172.21.15.32:0/1358849862 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0b40068380 msgr2=0x7f0b4006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:12.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.751+0000 7f0b6b800640 1 --2- 172.21.15.32:0/1358849862 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0b40068380 0x7f0b4006a840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0b641676b0 tx=0x7f0b4c009190 comp rx=0 tx=0).stop 2024-04-17T09:53:12.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.751+0000 7f0b6b800640 1 -- 172.21.15.32:0/1358849862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b6406ba30 msgr2=0x7f0b64145580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:12.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.751+0000 7f0b6b800640 1 --2- 172.21.15.32:0/1358849862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b6406ba30 0x7f0b64145580 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400c420 tx=0x7f0b5400c8e0 comp rx=0 tx=0).stop 2024-04-17T09:53:12.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.751+0000 7f0b6b800640 1 -- 172.21.15.32:0/1358849862 shutdown_connections 2024-04-17T09:53:12.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.751+0000 7f0b6b800640 1 --2- 172.21.15.32:0/1358849862 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0b40068380 0x7f0b4006a840 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:12.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.751+0000 7f0b6b800640 1 --2- 172.21.15.32:0/1358849862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b640cf370 0x7f0b640c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:12.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.751+0000 7f0b6b800640 1 --2- 172.21.15.32:0/1358849862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b64071e60 0x7f0b6413f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:12.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.751+0000 7f0b6b800640 1 --2- 172.21.15.32:0/1358849862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b6406ba30 0x7f0b64145580 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400c420 tx=0x7f0b5400c8e0 comp rx=0 tx=0).stop 2024-04-17T09:53:12.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.751+0000 7f0b6b800640 1 -- 172.21.15.32:0/1358849862 >> 172.21.15.32:0/1358849862 conn(0x7f0b640b2120 msgr2=0x7f0b6415c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:12.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.751+0000 7f0b6b800640 1 -- 172.21.15.32:0/1358849862 shutdown_connections 2024-04-17T09:53:12.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.751+0000 7f0b6b800640 1 -- 172.21.15.32:0/1358849862 wait complete. 2024-04-17T09:53:12.795 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T09:53:12.930 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:53:12.933 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:53:12.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.939+0000 7f1fcf600640 1 Processor -- start 2024-04-17T09:53:12.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.939+0000 7f1fcf600640 1 -- start start 2024-04-17T09:53:12.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.939+0000 7f1fcf600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1fc80ce740 0x7f1fc809a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:12.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.939+0000 7f1fcf600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1fc80ceba0 0x7f1fc809af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:12.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.939+0000 7f1fcf600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fc815cc40 con 0x7f1fc815c860 2024-04-17T09:53:12.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.939+0000 7f1fcf600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1fc80ce2b0 con 0x7f1fc80ce740 2024-04-17T09:53:12.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.939+0000 7f1fcf600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1fc80cd6a0 con 0x7f1fc80ceba0 2024-04-17T09:53:12.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.939+0000 7f1fc7e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1fc80ce740 0x7f1fc809a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:12.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.939+0000 7f1fccc00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1fc815c860 0x7f1fc80d1050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39144/0 (socket says 172.21.15.32:39144) 2024-04-17T09:53:12.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.939+0000 7f1fccc00640 1 -- 172.21.15.32:0/705962915 learned_addr learned my addr 172.21.15.32:0/705962915 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:12.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.943+0000 7f1fcd600640 1 --2- 172.21.15.32:0/705962915 >> v2:172.21.15.144:3300/0 conn(0x7f1fc80ceba0 0x7f1fc809af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:12.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.943+0000 7f1fc7400640 1 -- 172.21.15.32:0/705962915 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4133736331 0 0) 0x7f1fc815cc40 con 0x7f1fc815c860 2024-04-17T09:53:12.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.943+0000 7f1fc7400640 1 -- 172.21.15.32:0/705962915 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fb0003530 con 0x7f1fc815c860 2024-04-17T09:53:12.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.943+0000 7f1fc7e00640 1 -- 172.21.15.32:0/705962915 >> v2:172.21.15.144:3300/0 conn(0x7f1fc80ceba0 msgr2=0x7f1fc809af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:12.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.943+0000 7f1fc7e00640 1 --2- 172.21.15.32:0/705962915 >> v2:172.21.15.144:3300/0 conn(0x7f1fc80ceba0 0x7f1fc809af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:12.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.943+0000 7f1fc7e00640 1 -- 172.21.15.32:0/705962915 >> v1:172.21.15.32:6789/0 conn(0x7f1fc815c860 legacy=0x7f1fc80d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:12.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.943+0000 7f1fc7e00640 1 -- 172.21.15.32:0/705962915 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc80cd980 con 0x7f1fc80ce740 2024-04-17T09:53:12.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.943+0000 7f1fc7e00640 1 --2- 172.21.15.32:0/705962915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fc80ce740 0x7f1fc809a9f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1fc80695c0 tx=0x7f1fb401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d003003c2a002b33 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:12.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.943+0000 7f1fc7400640 1 -- 172.21.15.32:0/705962915 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb40b0070 con 0x7f1fc80ce740 2024-04-17T09:53:12.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.943+0000 7f1fc7400640 1 -- 172.21.15.32:0/705962915 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fb40ab600 con 0x7f1fc80ce740 2024-04-17T09:53:12.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.943+0000 7f1fc7400640 1 -- 172.21.15.32:0/705962915 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb40ab9d0 con 0x7f1fc80ce740 2024-04-17T09:53:12.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fcf600640 1 -- 172.21.15.32:0/705962915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fc80ce740 msgr2=0x7f1fc809a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:12.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fcf600640 1 --2- 172.21.15.32:0/705962915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fc80ce740 0x7f1fc809a9f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1fc80695c0 tx=0x7f1fb401c480 comp rx=0 tx=0).stop 2024-04-17T09:53:12.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fcf600640 1 -- 172.21.15.32:0/705962915 shutdown_connections 2024-04-17T09:53:12.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fcf600640 1 --2- 172.21.15.32:0/705962915 >> v2:172.21.15.144:3300/0 conn(0x7f1fc80ceba0 0x7f1fc809af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:12.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fcf600640 1 --2- 172.21.15.32:0/705962915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fc80ce740 0x7f1fc809a9f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:12.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fcf600640 1 -- 172.21.15.32:0/705962915 >> 172.21.15.32:0/705962915 conn(0x7f1fc80b4050 msgr2=0x7f1fc80b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:12.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fcf600640 1 -- 172.21.15.32:0/705962915 shutdown_connections 2024-04-17T09:53:12.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fcf600640 1 -- 172.21.15.32:0/705962915 wait complete. 2024-04-17T09:53:12.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fcf600640 1 Processor -- start 2024-04-17T09:53:12.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fcf600640 1 -- start start 2024-04-17T09:53:12.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fcf600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fc80ce740 0x7f1fc80cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:12.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fcf600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1fc80ceba0 0x7f1fc80b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:12.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fcf600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc815c860 0x7f1fc80c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:12.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fcf600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc80b5bc0 con 0x7f1fc80ce740 2024-04-17T09:53:12.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fcf600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc80cd350 con 0x7f1fc815c860 2024-04-17T09:53:12.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fcf600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1fc80b5d40 con 0x7f1fc80ceba0 2024-04-17T09:53:12.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fccc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fc80ce740 0x7f1fc80cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:12.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fc7e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1fc80ceba0 0x7f1fc80b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:12.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fccc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fc80ce740 0x7f1fc80cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46654/0 (socket says 172.21.15.32:46654) 2024-04-17T09:53:12.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fccc00640 1 -- 172.21.15.32:0/2818975858 learned_addr learned my addr 172.21.15.32:0/2818975858 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:12.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fcd600640 1 --2- 172.21.15.32:0/2818975858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc815c860 0x7f1fc80c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:12.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fccc00640 1 -- 172.21.15.32:0/2818975858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1fc80ceba0 msgr2=0x7f1fc80b6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:12.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fccc00640 1 --2- 172.21.15.32:0/2818975858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1fc80ceba0 0x7f1fc80b6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:12.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fccc00640 1 -- 172.21.15.32:0/2818975858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc815c860 msgr2=0x7f1fc80c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:12.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fccc00640 1 --2- 172.21.15.32:0/2818975858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc815c860 0x7f1fc80c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:12.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.947+0000 7f1fccc00640 1 -- 172.21.15.32:0/2818975858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc80c9a10 con 0x7f1fc80ce740 2024-04-17T09:53:12.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.951+0000 7f1fccc00640 1 --2- 172.21.15.32:0/2818975858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fc80ce740 0x7f1fc80cb1e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00c480 tx=0x7f1fbc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:12.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.951+0000 7f1fc4c00640 1 -- 172.21.15.32:0/2818975858 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc017070 con 0x7f1fc80ce740 2024-04-17T09:53:12.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.951+0000 7f1fc4c00640 1 -- 172.21.15.32:0/2818975858 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fbc005cf0 con 0x7f1fc80ce740 2024-04-17T09:53:12.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.951+0000 7f1fc4c00640 1 -- 172.21.15.32:0/2818975858 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc007290 con 0x7f1fc80ce740 2024-04-17T09:53:12.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.951+0000 7f1fcf600640 1 -- 172.21.15.32:0/2818975858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc80c9cf0 con 0x7f1fc80ce740 2024-04-17T09:53:12.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.951+0000 7f1fcf600640 1 -- 172.21.15.32:0/2818975858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc80ca2b0 con 0x7f1fc80ce740 2024-04-17T09:53:12.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.951+0000 7f1fcf600640 1 -- 172.21.15.32:0/2818975858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fc8068a90 con 0x7f1fc80ce740 2024-04-17T09:53:12.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.951+0000 7f1fc4c00640 1 -- 172.21.15.32:0/2818975858 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1fbc01a070 con 0x7f1fc80ce740 2024-04-17T09:53:12.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.951+0000 7f1fc4c00640 1 --2- 172.21.15.32:0/2818975858 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1fa8068380 0x7f1fa806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:12.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.951+0000 7f1fc4c00640 1 -- 172.21.15.32:0/2818975858 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f1fbc085d10 con 0x7f1fc80ce740 2024-04-17T09:53:12.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.951+0000 7f1fc7e00640 1 --2- 172.21.15.32:0/2818975858 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1fa8068380 0x7f1fa806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:12.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.955+0000 7f1fc7e00640 1 --2- 172.21.15.32:0/2818975858 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1fa8068380 0x7f1fa806a840 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40ad000 tx=0x7f1fb4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:12.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:12.959+0000 7f1fc4c00640 1 -- 172.21.15.32:0/2818975858 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f1fbc055ce0 con 0x7f1fc80ce740 2024-04-17T09:53:13.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.167+0000 7f1fcf600640 1 -- 172.21.15.32:0/2818975858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1fc80d09c0 con 0x7f1fc80ce740 2024-04-17T09:53:13.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.775+0000 7f1fc4c00640 1 -- 172.21.15.32:0/2818975858 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f1fbc059fe0 con 0x7f1fc80ce740 2024-04-17T09:53:13.786 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 2 and data pool 3 2024-04-17T09:53:13.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.787+0000 7f1fcf600640 1 -- 172.21.15.32:0/2818975858 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1fa8068380 msgr2=0x7f1fa806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.787+0000 7f1fcf600640 1 --2- 172.21.15.32:0/2818975858 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1fa8068380 0x7f1fa806a840 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40ad000 tx=0x7f1fb4008040 comp rx=0 tx=0).stop 2024-04-17T09:53:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.787+0000 7f1fcf600640 1 -- 172.21.15.32:0/2818975858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fc80ce740 msgr2=0x7f1fc80cb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.787+0000 7f1fcf600640 1 --2- 172.21.15.32:0/2818975858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fc80ce740 0x7f1fc80cb1e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00c480 tx=0x7f1fbc00c940 comp rx=0 tx=0).stop 2024-04-17T09:53:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.787+0000 7f1fcf600640 1 -- 172.21.15.32:0/2818975858 shutdown_connections 2024-04-17T09:53:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.787+0000 7f1fcf600640 1 --2- 172.21.15.32:0/2818975858 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1fa8068380 0x7f1fa806a840 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.787+0000 7f1fcf600640 1 --2- 172.21.15.32:0/2818975858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1fc80ceba0 0x7f1fc80b6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.787+0000 7f1fcf600640 1 --2- 172.21.15.32:0/2818975858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc815c860 0x7f1fc80c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.787+0000 7f1fcf600640 1 --2- 172.21.15.32:0/2818975858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fc80ce740 0x7f1fc80cb1e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.787+0000 7f1fcf600640 1 -- 172.21.15.32:0/2818975858 >> 172.21.15.32:0/2818975858 conn(0x7f1fc80b4050 msgr2=0x7f1fc80b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.787+0000 7f1fcf600640 1 -- 172.21.15.32:0/2818975858 shutdown_connections 2024-04-17T09:53:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.787+0000 7f1fcf600640 1 -- 172.21.15.32:0/2818975858 wait complete. 2024-04-17T09:53:13.827 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:53:13.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.967+0000 7f7f35000640 1 Processor -- start 2024-04-17T09:53:13.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.967+0000 7f7f35000640 1 -- start start 2024-04-17T09:53:13.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.967+0000 7f7f35000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f300cd580 0x7f7f30167560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:13.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.967+0000 7f7f35000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f300cdac0 con 0x7f7f30072500 2024-04-17T09:53:13.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.967+0000 7f7f35000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f3015cdf0 con 0x7f7f30145f30 2024-04-17T09:53:13.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.967+0000 7f7f35000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7f3006f400 con 0x7f7f300cd580 2024-04-17T09:53:13.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.967+0000 7f7f25400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7f30072500 0x7f7f300cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39150/0 (socket says 172.21.15.32:39150) 2024-04-17T09:53:13.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.967+0000 7f7f25400640 1 -- 172.21.15.32:0/4025280096 learned_addr learned my addr 172.21.15.32:0/4025280096 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:13.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.967+0000 7f7f2e800640 1 --2- 172.21.15.32:0/4025280096 >> v2:172.21.15.32:3300/0 conn(0x7f7f300cd580 0x7f7f30167560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:13.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.967+0000 7f7f2d400640 1 -- 172.21.15.32:0/4025280096 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1807935594 0 0) 0x7f7f300cdac0 con 0x7f7f30072500 2024-04-17T09:53:13.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.967+0000 7f7f2d400640 1 -- 172.21.15.32:0/4025280096 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f14003530 con 0x7f7f30072500 2024-04-17T09:53:13.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.967+0000 7f7f2e800640 1 -- 172.21.15.32:0/4025280096 >> v1:172.21.15.144:6789/0 conn(0x7f7f30145f30 legacy=0x7f7f30071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:13.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.967+0000 7f7f2e800640 1 -- 172.21.15.32:0/4025280096 >> v1:172.21.15.32:6789/0 conn(0x7f7f30072500 legacy=0x7f7f300cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:13.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.967+0000 7f7f2e800640 1 -- 172.21.15.32:0/4025280096 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3015d0d0 con 0x7f7f300cd580 2024-04-17T09:53:13.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.971+0000 7f7f2e800640 1 --2- 172.21.15.32:0/4025280096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f300cd580 0x7f7f30167560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7f28009d90 tx=0x7f7f2801f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467f57db8ac606dc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:13.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.971+0000 7f7f2d400640 1 -- 172.21.15.32:0/4025280096 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f280b4070 con 0x7f7f300cd580 2024-04-17T09:53:13.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.971+0000 7f7f2d400640 1 -- 172.21.15.32:0/4025280096 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f280af600 con 0x7f7f300cd580 2024-04-17T09:53:13.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.971+0000 7f7f2d400640 1 -- 172.21.15.32:0/4025280096 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f280af9d0 con 0x7f7f300cd580 2024-04-17T09:53:13.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.971+0000 7f7f35000640 1 -- 172.21.15.32:0/4025280096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f300cd580 msgr2=0x7f7f30167560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:13.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.971+0000 7f7f35000640 1 --2- 172.21.15.32:0/4025280096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f300cd580 0x7f7f30167560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7f28009d90 tx=0x7f7f2801f150 comp rx=0 tx=0).stop 2024-04-17T09:53:13.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.971+0000 7f7f35000640 1 -- 172.21.15.32:0/4025280096 shutdown_connections 2024-04-17T09:53:13.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.971+0000 7f7f35000640 1 --2- 172.21.15.32:0/4025280096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f300cd580 0x7f7f30167560 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:13.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.971+0000 7f7f35000640 1 -- 172.21.15.32:0/4025280096 >> 172.21.15.32:0/4025280096 conn(0x7f7f3006ba30 msgr2=0x7f7f3006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:13.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.971+0000 7f7f35000640 1 -- 172.21.15.32:0/4025280096 shutdown_connections 2024-04-17T09:53:13.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.971+0000 7f7f35000640 1 -- 172.21.15.32:0/4025280096 wait complete. 2024-04-17T09:53:13.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f35000640 1 Processor -- start 2024-04-17T09:53:13.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f35000640 1 -- start start 2024-04-17T09:53:13.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f35000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f30072500 0x7f7f300c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:13.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f35000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f300cd580 0x7f7f301f1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:13.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f35000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f30145f30 0x7f7f300c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:13.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f35000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7f30146310 con 0x7f7f300cd580 2024-04-17T09:53:13.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f35000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3015dba0 con 0x7f7f30145f30 2024-04-17T09:53:13.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f35000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7f3015d440 con 0x7f7f30072500 2024-04-17T09:53:13.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f25400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f300cd580 0x7f7f301f1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:13.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f25400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f300cd580 0x7f7f301f1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46662/0 (socket says 172.21.15.32:46662) 2024-04-17T09:53:13.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f25400640 1 -- 172.21.15.32:0/3953330545 learned_addr learned my addr 172.21.15.32:0/3953330545 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:13.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f2de00640 1 --2- 172.21.15.32:0/3953330545 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f30072500 0x7f7f300c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:13.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f2e800640 1 --2- 172.21.15.32:0/3953330545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f30145f30 0x7f7f300c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:13.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f25400640 1 -- 172.21.15.32:0/3953330545 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f30072500 msgr2=0x7f7f300c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:13.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f25400640 1 --2- 172.21.15.32:0/3953330545 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f30072500 0x7f7f300c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:13.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f25400640 1 -- 172.21.15.32:0/3953330545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f30145f30 msgr2=0x7f7f300c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:13.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f25400640 1 --2- 172.21.15.32:0/3953330545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f30145f30 0x7f7f300c8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:13.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f25400640 1 -- 172.21.15.32:0/3953330545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3012d950 con 0x7f7f300cd580 2024-04-17T09:53:13.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f2de00640 1 --2- 172.21.15.32:0/3953330545 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f30072500 0x7f7f300c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:13.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f25400640 1 --2- 172.21.15.32:0/3953330545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f300cd580 0x7f7f301f1c50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000c500 tx=0x7f7f2000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:13.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f26a00640 1 -- 172.21.15.32:0/3953330545 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20017070 con 0x7f7f300cd580 2024-04-17T09:53:13.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f26a00640 1 -- 172.21.15.32:0/3953330545 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f20005cf0 con 0x7f7f300cd580 2024-04-17T09:53:13.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f35000640 1 -- 172.21.15.32:0/3953330545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f3012dbf0 con 0x7f7f300cd580 2024-04-17T09:53:13.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f26a00640 1 -- 172.21.15.32:0/3953330545 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f200072f0 con 0x7f7f300cd580 2024-04-17T09:53:13.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f35000640 1 -- 172.21.15.32:0/3953330545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f3012e1b0 con 0x7f7f300cd580 2024-04-17T09:53:13.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.975+0000 7f7f35000640 1 -- 172.21.15.32:0/3953330545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f30068a90 con 0x7f7f300cd580 2024-04-17T09:53:13.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.979+0000 7f7f26a00640 1 -- 172.21.15.32:0/3953330545 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7f2001a070 con 0x7f7f300cd580 2024-04-17T09:53:13.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.979+0000 7f7f26a00640 1 --2- 172.21.15.32:0/3953330545 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7f0c068380 0x7f7f0c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:13.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.979+0000 7f7f26a00640 1 -- 172.21.15.32:0/3953330545 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7f20085d30 con 0x7f7f300cd580 2024-04-17T09:53:13.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.979+0000 7f7f2de00640 1 --2- 172.21.15.32:0/3953330545 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7f0c068380 0x7f7f0c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:13.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.979+0000 7f7f2de00640 1 --2- 172.21.15.32:0/3953330545 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7f0c068380 0x7f7f0c06a840 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7f18000f70 tx=0x7f7f18009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:13.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:13.983+0000 7f7f26a00640 1 -- 172.21.15.32:0/3953330545 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f7f20055c90 con 0x7f7f300cd580 2024-04-17T09:53:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.147+0000 7f7f35000640 1 -- 172.21.15.32:0/3953330545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f30072f90 con 0x7f7f300cd580 2024-04-17T09:53:14.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.147+0000 7f7f26a00640 1 -- 172.21.15.32:0/3953330545 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16630 (secure 0 0 0) 0x7f7f20059f90 con 0x7f7f300cd580 2024-04-17T09:53:14.152 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:14.152 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":20,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:13.172220+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:53:14.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.151+0000 7f7f35000640 1 -- 172.21.15.32:0/3953330545 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7f0c068380 msgr2=0x7f7f0c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:14.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.151+0000 7f7f35000640 1 --2- 172.21.15.32:0/3953330545 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7f0c068380 0x7f7f0c06a840 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7f18000f70 tx=0x7f7f18009040 comp rx=0 tx=0).stop 2024-04-17T09:53:14.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.151+0000 7f7f35000640 1 -- 172.21.15.32:0/3953330545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f300cd580 msgr2=0x7f7f301f1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:14.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.151+0000 7f7f35000640 1 --2- 172.21.15.32:0/3953330545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f300cd580 0x7f7f301f1c50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000c500 tx=0x7f7f2000c9c0 comp rx=0 tx=0).stop 2024-04-17T09:53:14.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.151+0000 7f7f35000640 1 -- 172.21.15.32:0/3953330545 shutdown_connections 2024-04-17T09:53:14.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.151+0000 7f7f35000640 1 --2- 172.21.15.32:0/3953330545 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7f0c068380 0x7f7f0c06a840 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:14.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.151+0000 7f7f35000640 1 --2- 172.21.15.32:0/3953330545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f30145f30 0x7f7f300c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:14.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.151+0000 7f7f35000640 1 --2- 172.21.15.32:0/3953330545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f300cd580 0x7f7f301f1c50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:14.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.151+0000 7f7f35000640 1 --2- 172.21.15.32:0/3953330545 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f30072500 0x7f7f300c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:14.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.151+0000 7f7f35000640 1 -- 172.21.15.32:0/3953330545 >> 172.21.15.32:0/3953330545 conn(0x7f7f3006ba30 msgr2=0x7f7f300b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:14.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.151+0000 7f7f35000640 1 -- 172.21.15.32:0/3953330545 shutdown_connections 2024-04-17T09:53:14.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.151+0000 7f7f35000640 1 -- 172.21.15.32:0/3953330545 wait complete. 2024-04-17T09:53:14.174 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:53:14.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.311+0000 7fc23a800640 1 Processor -- start 2024-04-17T09:53:14.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.311+0000 7fc23a800640 1 -- start start 2024-04-17T09:53:14.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.311+0000 7fc23a800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2340b0320 0x7fc2340d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:14.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.311+0000 7fc23a800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc2340d0780 0x7fc23406f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:14.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.311+0000 7fc23a800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc23415ec50 con 0x7fc23415e870 2024-04-17T09:53:14.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.311+0000 7fc23a800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc2340cf810 con 0x7fc2340b0320 2024-04-17T09:53:14.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.311+0000 7fc23a800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc234067dd0 con 0x7fc2340d0780 2024-04-17T09:53:14.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.311+0000 7fc22be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2340b0320 0x7fc2340d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:14.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.311+0000 7fc22be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2340b0320 0x7fc2340d0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46670/0 (socket says 172.21.15.32:46670) 2024-04-17T09:53:14.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.311+0000 7fc22be00640 1 -- 172.21.15.32:0/1450430056 learned_addr learned my addr 172.21.15.32:0/1450430056 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:14.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.311+0000 7fc233e00640 1 --2- 172.21.15.32:0/1450430056 >> v2:172.21.15.144:3300/0 conn(0x7fc2340d0780 0x7fc23406f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:14.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.315+0000 7fc22be00640 1 -- 172.21.15.32:0/1450430056 >> v2:172.21.15.144:3300/0 conn(0x7fc2340d0780 msgr2=0x7fc23406f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:14.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.315+0000 7fc22be00640 1 --2- 172.21.15.32:0/1450430056 >> v2:172.21.15.144:3300/0 conn(0x7fc2340d0780 0x7fc23406f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:14.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.315+0000 7fc22be00640 1 -- 172.21.15.32:0/1450430056 >> v1:172.21.15.144:6789/0 conn(0x7fc23415e870 legacy=0x7fc234162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:14.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.315+0000 7fc22be00640 1 -- 172.21.15.32:0/1450430056 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc234067810 con 0x7fc2340b0320 2024-04-17T09:53:14.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.315+0000 7fc22be00640 1 --2- 172.21.15.32:0/1450430056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2340b0320 0x7fc2340d0240 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc234067f50 tx=0x7fc21c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122da1251aa6d7fe server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:14.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.315+0000 7fc232a00640 1 -- 172.21.15.32:0/1450430056 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c0b0070 con 0x7fc2340b0320 2024-04-17T09:53:14.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.315+0000 7fc232a00640 1 -- 172.21.15.32:0/1450430056 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc21c0ab600 con 0x7fc2340b0320 2024-04-17T09:53:14.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.315+0000 7fc232a00640 1 -- 172.21.15.32:0/1450430056 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c0ab9d0 con 0x7fc2340b0320 2024-04-17T09:53:14.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.315+0000 7fc23a800640 1 -- 172.21.15.32:0/1450430056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2340b0320 msgr2=0x7fc2340d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:14.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.315+0000 7fc23a800640 1 --2- 172.21.15.32:0/1450430056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2340b0320 0x7fc2340d0240 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc234067f50 tx=0x7fc21c01c520 comp rx=0 tx=0).stop 2024-04-17T09:53:14.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.315+0000 7fc23a800640 1 -- 172.21.15.32:0/1450430056 shutdown_connections 2024-04-17T09:53:14.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.315+0000 7fc23a800640 1 --2- 172.21.15.32:0/1450430056 >> v2:172.21.15.144:3300/0 conn(0x7fc2340d0780 0x7fc23406f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:14.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.315+0000 7fc23a800640 1 --2- 172.21.15.32:0/1450430056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2340b0320 0x7fc2340d0240 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:14.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.315+0000 7fc23a800640 1 -- 172.21.15.32:0/1450430056 >> 172.21.15.32:0/1450430056 conn(0x7fc2340d0c00 msgr2=0x7fc2340d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:14.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.315+0000 7fc23a800640 1 -- 172.21.15.32:0/1450430056 shutdown_connections 2024-04-17T09:53:14.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc23a800640 1 -- 172.21.15.32:0/1450430056 wait complete. 2024-04-17T09:53:14.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc23a800640 1 Processor -- start 2024-04-17T09:53:14.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc23a800640 1 -- start start 2024-04-17T09:53:14.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc23a800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2340b0320 0x7fc23415cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:14.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc23a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc23415e870 0x7fc2340af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:14.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc23a800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2340afae0 0x7fc23414f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:14.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc23a800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc23416bb20 con 0x7fc23415e870 2024-04-17T09:53:14.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc23a800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc23416b9a0 con 0x7fc2340b0320 2024-04-17T09:53:14.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc23a800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc23416bca0 con 0x7fc2340afae0 2024-04-17T09:53:14.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc22be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc23415e870 0x7fc2340af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:14.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc22be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc23415e870 0x7fc2340af5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46682/0 (socket says 172.21.15.32:46682) 2024-04-17T09:53:14.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc22be00640 1 -- 172.21.15.32:0/2591509346 learned_addr learned my addr 172.21.15.32:0/2591509346 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:14.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc233400640 1 --2- 172.21.15.32:0/2591509346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2340b0320 0x7fc23415cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:14.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc233e00640 1 --2- 172.21.15.32:0/2591509346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2340afae0 0x7fc23414f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:14.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc22be00640 1 -- 172.21.15.32:0/2591509346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2340afae0 msgr2=0x7fc23414f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:14.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc22be00640 1 --2- 172.21.15.32:0/2591509346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2340afae0 0x7fc23414f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:14.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc22be00640 1 -- 172.21.15.32:0/2591509346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2340b0320 msgr2=0x7fc23415cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:14.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc22be00640 1 --2- 172.21.15.32:0/2591509346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2340b0320 0x7fc23415cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:14.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc22be00640 1 -- 172.21.15.32:0/2591509346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc234150690 con 0x7fc23415e870 2024-04-17T09:53:14.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc233e00640 1 --2- 172.21.15.32:0/2591509346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2340afae0 0x7fc23414f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:14.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc22be00640 1 --2- 172.21.15.32:0/2591509346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc23415e870 0x7fc2340af5a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc21c01abd0 tx=0x7fc21c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:14.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc22b400640 1 -- 172.21.15.32:0/2591509346 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c0bb070 con 0x7fc23415e870 2024-04-17T09:53:14.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc22b400640 1 -- 172.21.15.32:0/2591509346 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc21c0b6430 con 0x7fc23415e870 2024-04-17T09:53:14.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc22b400640 1 -- 172.21.15.32:0/2591509346 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c0b0040 con 0x7fc23415e870 2024-04-17T09:53:14.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.319+0000 7fc23a800640 1 -- 172.21.15.32:0/2591509346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc234150970 con 0x7fc23415e870 2024-04-17T09:53:14.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.323+0000 7fc23a800640 1 -- 172.21.15.32:0/2591509346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc234150f30 con 0x7fc23415e870 2024-04-17T09:53:14.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.323+0000 7fc23a800640 1 -- 172.21.15.32:0/2591509346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc234068a90 con 0x7fc23415e870 2024-04-17T09:53:14.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.323+0000 7fc22b400640 1 -- 172.21.15.32:0/2591509346 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc21c0b53d0 con 0x7fc23415e870 2024-04-17T09:53:14.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.323+0000 7fc22b400640 1 --2- 172.21.15.32:0/2591509346 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc2100683b0 0x7fc21006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:14.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.323+0000 7fc22b400640 1 -- 172.21.15.32:0/2591509346 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc21c120020 con 0x7fc23415e870 2024-04-17T09:53:14.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.323+0000 7fc233400640 1 --2- 172.21.15.32:0/2591509346 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc2100683b0 0x7fc21006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:14.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.323+0000 7fc233400640 1 --2- 172.21.15.32:0/2591509346 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc2100683b0 0x7fc21006a870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc224001810 tx=0x7fc224009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:14.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.327+0000 7fc22b400640 1 -- 172.21.15.32:0/2591509346 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc21c016110 con 0x7fc23415e870 2024-04-17T09:53:14.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.495+0000 7fc23a800640 1 -- 172.21.15.32:0/2591509346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc2340d1180 con 0x7fc23415e870 2024-04-17T09:53:14.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.495+0000 7fc22b400640 1 -- 172.21.15.32:0/2591509346 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16630 (secure 0 0 0) 0x7fc21c0efe60 con 0x7fc23415e870 2024-04-17T09:53:14.499 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:14.499 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":20,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:13.172220+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:53:14.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.499+0000 7fc23a800640 1 -- 172.21.15.32:0/2591509346 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc2100683b0 msgr2=0x7fc21006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.499+0000 7fc23a800640 1 --2- 172.21.15.32:0/2591509346 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc2100683b0 0x7fc21006a870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc224001810 tx=0x7fc224009040 comp rx=0 tx=0).stop 2024-04-17T09:53:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.499+0000 7fc23a800640 1 -- 172.21.15.32:0/2591509346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc23415e870 msgr2=0x7fc2340af5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.499+0000 7fc23a800640 1 --2- 172.21.15.32:0/2591509346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc23415e870 0x7fc2340af5a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc21c01abd0 tx=0x7fc21c0192c0 comp rx=0 tx=0).stop 2024-04-17T09:53:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.499+0000 7fc23a800640 1 -- 172.21.15.32:0/2591509346 shutdown_connections 2024-04-17T09:53:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.499+0000 7fc23a800640 1 --2- 172.21.15.32:0/2591509346 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc2100683b0 0x7fc21006a870 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.499+0000 7fc23a800640 1 --2- 172.21.15.32:0/2591509346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2340afae0 0x7fc23414f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.499+0000 7fc23a800640 1 --2- 172.21.15.32:0/2591509346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc23415e870 0x7fc2340af5a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.499+0000 7fc23a800640 1 --2- 172.21.15.32:0/2591509346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2340b0320 0x7fc23415cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.499+0000 7fc23a800640 1 -- 172.21.15.32:0/2591509346 >> 172.21.15.32:0/2591509346 conn(0x7fc2340d0c00 msgr2=0x7fc2340b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.499+0000 7fc23a800640 1 -- 172.21.15.32:0/2591509346 shutdown_connections 2024-04-17T09:53:14.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.499+0000 7fc23a800640 1 -- 172.21.15.32:0/2591509346 wait complete. 2024-04-17T09:53:14.522 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T09:53:14.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.663+0000 7f59b4400640 1 Processor -- start 2024-04-17T09:53:14.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.663+0000 7f59b4400640 1 -- start start 2024-04-17T09:53:14.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.667+0000 7f59b4400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ac0d0650 0x7f59ac13fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.667+0000 7f59b4400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f59ac06d150 0x7f59ac160560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.667+0000 7f59b4400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59ac15d070 con 0x7f59ac0af490 2024-04-17T09:53:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.667+0000 7f59b4400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f59ac15dc80 con 0x7f59ac06d150 2024-04-17T09:53:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.667+0000 7f59b4400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f59ac15d350 con 0x7f59ac0d0650 2024-04-17T09:53:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.667+0000 7f59b1000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f59ac0af490 0x7f59ac06ca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39166/0 (socket says 172.21.15.32:39166) 2024-04-17T09:53:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.667+0000 7f59b1000640 1 -- 172.21.15.32:0/23357410 learned_addr learned my addr 172.21.15.32:0/23357410 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.667+0000 7f59b1a00640 1 --2- 172.21.15.32:0/23357410 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ac0d0650 0x7f59ac13fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.667+0000 7f59b2400640 1 --2- 172.21.15.32:0/23357410 >> v2:172.21.15.144:3300/0 conn(0x7f59ac06d150 0x7f59ac160560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.667+0000 7f59abe00640 1 -- 172.21.15.32:0/23357410 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1719930386 0 0) 0x7f59ac15d070 con 0x7f59ac0af490 2024-04-17T09:53:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.667+0000 7f59abe00640 1 -- 172.21.15.32:0/23357410 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5994003530 con 0x7f59ac0af490 2024-04-17T09:53:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.667+0000 7f59b2400640 1 -- 172.21.15.32:0/23357410 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ac0d0650 msgr2=0x7f59ac13fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.667+0000 7f59b2400640 1 --2- 172.21.15.32:0/23357410 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ac0d0650 0x7f59ac13fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.667+0000 7f59b2400640 1 -- 172.21.15.32:0/23357410 >> v1:172.21.15.32:6789/0 conn(0x7f59ac0af490 legacy=0x7f59ac06ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.667+0000 7f59b2400640 1 -- 172.21.15.32:0/23357410 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ac15d630 con 0x7f59ac06d150 2024-04-17T09:53:14.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.667+0000 7f59b1a00640 1 --2- 172.21.15.32:0/23357410 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ac0d0650 0x7f59ac13fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:53:14.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.671+0000 7f59b2400640 1 --2- 172.21.15.32:0/23357410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ac06d150 0x7f59ac160560 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009d90 tx=0x7f59a401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1c847a7209292f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:14.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.671+0000 7f59abe00640 1 -- 172.21.15.32:0/23357410 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a40b2070 con 0x7f59ac06d150 2024-04-17T09:53:14.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.671+0000 7f59abe00640 1 -- 172.21.15.32:0/23357410 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a40a59f0 con 0x7f59ac06d150 2024-04-17T09:53:14.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.671+0000 7f59abe00640 1 -- 172.21.15.32:0/23357410 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a40a5dc0 con 0x7f59ac06d150 2024-04-17T09:53:14.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.671+0000 7f59b4400640 1 -- 172.21.15.32:0/23357410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ac06d150 msgr2=0x7f59ac160560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:14.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.671+0000 7f59b4400640 1 --2- 172.21.15.32:0/23357410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ac06d150 0x7f59ac160560 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009d90 tx=0x7f59a401c970 comp rx=0 tx=0).stop 2024-04-17T09:53:14.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.671+0000 7f59b4400640 1 -- 172.21.15.32:0/23357410 shutdown_connections 2024-04-17T09:53:14.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.671+0000 7f59b4400640 1 --2- 172.21.15.32:0/23357410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ac06d150 0x7f59ac160560 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:14.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.671+0000 7f59b4400640 1 --2- 172.21.15.32:0/23357410 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ac0d0650 0x7f59ac13fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:14.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.671+0000 7f59b4400640 1 -- 172.21.15.32:0/23357410 >> 172.21.15.32:0/23357410 conn(0x7f59ac072670 msgr2=0x7f59ac072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:14.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.671+0000 7f59b4400640 1 -- 172.21.15.32:0/23357410 shutdown_connections 2024-04-17T09:53:14.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.671+0000 7f59b4400640 1 -- 172.21.15.32:0/23357410 wait complete. 2024-04-17T09:53:14.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.671+0000 7f59b4400640 1 Processor -- start 2024-04-17T09:53:14.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.671+0000 7f59b4400640 1 -- start start 2024-04-17T09:53:14.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.671+0000 7f59b4400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59ac06d150 0x7f59ac0cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:14.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.671+0000 7f59b4400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ac0af490 0x7f59ac13fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:14.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.675+0000 7f59b4400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ac0d0650 0x7f59ac15e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:14.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.675+0000 7f59b4400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac15ce10 con 0x7f59ac06d150 2024-04-17T09:53:14.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.675+0000 7f59b4400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac15cc90 con 0x7f59ac0d0650 2024-04-17T09:53:14.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.675+0000 7f59b4400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f59ac169620 con 0x7f59ac0af490 2024-04-17T09:53:14.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.675+0000 7f59b1a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59ac06d150 0x7f59ac0cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:14.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.675+0000 7f59b1a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59ac06d150 0x7f59ac0cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46696/0 (socket says 172.21.15.32:46696) 2024-04-17T09:53:14.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.675+0000 7f59b1a00640 1 -- 172.21.15.32:0/3440698205 learned_addr learned my addr 172.21.15.32:0/3440698205 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:14.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.675+0000 7f59b2400640 1 --2- 172.21.15.32:0/3440698205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ac0d0650 0x7f59ac15e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:14.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.675+0000 7f59b1000640 1 --2- 172.21.15.32:0/3440698205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ac0af490 0x7f59ac13fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:14.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.675+0000 7f59b1a00640 1 -- 172.21.15.32:0/3440698205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ac0af490 msgr2=0x7f59ac13fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:14.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.675+0000 7f59b1a00640 1 --2- 172.21.15.32:0/3440698205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ac0af490 0x7f59ac13fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:14.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.675+0000 7f59b1a00640 1 -- 172.21.15.32:0/3440698205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ac0d0650 msgr2=0x7f59ac15e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:14.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.675+0000 7f59b1a00640 1 --2- 172.21.15.32:0/3440698205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ac0d0650 0x7f59ac15e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:14.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.675+0000 7f59b1a00640 1 -- 172.21.15.32:0/3440698205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ac0b5110 con 0x7f59ac06d150 2024-04-17T09:53:14.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.675+0000 7f59b1a00640 1 --2- 172.21.15.32:0/3440698205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59ac06d150 0x7f59ac0cf7c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f59a000e980 tx=0x7f59a000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:14.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.675+0000 7f59a9600640 1 -- 172.21.15.32:0/3440698205 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0017070 con 0x7f59ac06d150 2024-04-17T09:53:14.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.675+0000 7f59a9600640 1 -- 172.21.15.32:0/3440698205 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a000ca00 con 0x7f59ac06d150 2024-04-17T09:53:14.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.675+0000 7f59b4400640 1 -- 172.21.15.32:0/3440698205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59ac0cdea0 con 0x7f59ac06d150 2024-04-17T09:53:14.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.675+0000 7f59a9600640 1 -- 172.21.15.32:0/3440698205 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a000cdd0 con 0x7f59ac06d150 2024-04-17T09:53:14.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.675+0000 7f59b4400640 1 -- 172.21.15.32:0/3440698205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59ac0b5660 con 0x7f59ac06d150 2024-04-17T09:53:14.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.675+0000 7f59b4400640 1 -- 172.21.15.32:0/3440698205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59ac068a90 con 0x7f59ac06d150 2024-04-17T09:53:14.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.675+0000 7f59a9600640 1 -- 172.21.15.32:0/3440698205 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f59a001a070 con 0x7f59ac06d150 2024-04-17T09:53:14.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.675+0000 7f59a9600640 1 --2- 172.21.15.32:0/3440698205 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f598c068380 0x7f598c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:14.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.679+0000 7f59a9600640 1 -- 172.21.15.32:0/3440698205 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f59a0086900 con 0x7f59ac06d150 2024-04-17T09:53:14.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.679+0000 7f59b1000640 1 --2- 172.21.15.32:0/3440698205 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f598c068380 0x7f598c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:14.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.679+0000 7f59b1000640 1 --2- 172.21.15.32:0/3440698205 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f598c068380 0x7f598c06a840 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f59ac156940 tx=0x7f5998009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:14.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.679+0000 7f59a9600640 1 -- 172.21.15.32:0/3440698205 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f59a0056ec0 con 0x7f59ac06d150 2024-04-17T09:53:14.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:14.883+0000 7f59b4400640 1 -- 172.21.15.32:0/3440698205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f59ac0b27c0 con 0x7f59ac06d150 2024-04-17T09:53:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.787+0000 7f59a9600640 1 -- 172.21.15.32:0/3440698205 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f59a00567c0 con 0x7f59ac06d150 2024-04-17T09:53:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.791+0000 7f59b4400640 1 -- 172.21.15.32:0/3440698205 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f598c068380 msgr2=0x7f598c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.791+0000 7f59b4400640 1 --2- 172.21.15.32:0/3440698205 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f598c068380 0x7f598c06a840 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f59ac156940 tx=0x7f5998009040 comp rx=0 tx=0).stop 2024-04-17T09:53:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.791+0000 7f59b4400640 1 -- 172.21.15.32:0/3440698205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59ac06d150 msgr2=0x7f59ac0cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.791+0000 7f59b4400640 1 --2- 172.21.15.32:0/3440698205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59ac06d150 0x7f59ac0cf7c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f59a000e980 tx=0x7f59a000ee40 comp rx=0 tx=0).stop 2024-04-17T09:53:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.791+0000 7f59b4400640 1 -- 172.21.15.32:0/3440698205 shutdown_connections 2024-04-17T09:53:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.791+0000 7f59b4400640 1 --2- 172.21.15.32:0/3440698205 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f598c068380 0x7f598c06a840 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.791+0000 7f59b4400640 1 --2- 172.21.15.32:0/3440698205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ac0af490 0x7f59ac13fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.791+0000 7f59b4400640 1 --2- 172.21.15.32:0/3440698205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ac0d0650 0x7f59ac15e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.791+0000 7f59b4400640 1 --2- 172.21.15.32:0/3440698205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59ac06d150 0x7f59ac0cf7c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.791+0000 7f59b4400640 1 -- 172.21.15.32:0/3440698205 >> 172.21.15.32:0/3440698205 conn(0x7f59ac072670 msgr2=0x7f59ac1392d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.791+0000 7f59b4400640 1 -- 172.21.15.32:0/3440698205 shutdown_connections 2024-04-17T09:53:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.791+0000 7f59b4400640 1 -- 172.21.15.32:0/3440698205 wait complete. 2024-04-17T09:53:15.818 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-17T09:53:15.819 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-17T09:53:15.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.963+0000 7fddb8c00640 1 Processor -- start 2024-04-17T09:53:15.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.963+0000 7fddb8c00640 1 -- start start 2024-04-17T09:53:15.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.963+0000 7fddb8c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddb4141700 0x7fddb406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:15.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.963+0000 7fddb8c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fddb406cf80 0x7fddb40af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:15.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.963+0000 7fddb8c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fddb40afce0 0x7fddb4162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:15.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.963+0000 7fddb8c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fddb406d3e0 con 0x7fddb40afce0 2024-04-17T09:53:15.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.963+0000 7fddb8c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fddb40b0820 con 0x7fddb406cf80 2024-04-17T09:53:15.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.963+0000 7fddb8c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fddb40b0540 con 0x7fddb4141700 2024-04-17T09:53:15.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.963+0000 7fddb2800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fddb40afce0 0x7fddb4162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:15.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.963+0000 7fddb2800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fddb40afce0 0x7fddb4162320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46698/0 (socket says 172.21.15.32:46698) 2024-04-17T09:53:15.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.963+0000 7fddb2800640 1 -- 172.21.15.32:0/3046803224 learned_addr learned my addr 172.21.15.32:0/3046803224 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:15.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.963+0000 7fdda9400640 1 --2- 172.21.15.32:0/3046803224 >> v2:172.21.15.144:3300/0 conn(0x7fddb406cf80 0x7fddb40af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:15.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.963+0000 7fddb1e00640 1 --2- 172.21.15.32:0/3046803224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddb4141700 0x7fddb406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:15.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.963+0000 7fddb2800640 1 -- 172.21.15.32:0/3046803224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddb4141700 msgr2=0x7fddb406ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:15.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.963+0000 7fddb2800640 1 --2- 172.21.15.32:0/3046803224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddb4141700 0x7fddb406ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:15.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.963+0000 7fddb2800640 1 -- 172.21.15.32:0/3046803224 >> v2:172.21.15.144:3300/0 conn(0x7fddb406cf80 msgr2=0x7fddb40af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:15.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.963+0000 7fddb2800640 1 --2- 172.21.15.32:0/3046803224 >> v2:172.21.15.144:3300/0 conn(0x7fddb406cf80 0x7fddb40af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:15.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.963+0000 7fddb2800640 1 -- 172.21.15.32:0/3046803224 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb40b3ef0 con 0x7fddb40afce0 2024-04-17T09:53:15.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.967+0000 7fddb2800640 1 --2- 172.21.15.32:0/3046803224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40afce0 0x7fddb4162320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fddb409ee70 tx=0x7fddac01c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764f0aad25515e9b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:15.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.967+0000 7fddb1400640 1 -- 172.21.15.32:0/3046803224 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac0b0070 con 0x7fddb40afce0 2024-04-17T09:53:15.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.967+0000 7fddb1400640 1 -- 172.21.15.32:0/3046803224 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddac0ab600 con 0x7fddb40afce0 2024-04-17T09:53:15.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.967+0000 7fddb1400640 1 -- 172.21.15.32:0/3046803224 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac0ab9d0 con 0x7fddb40afce0 2024-04-17T09:53:15.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.967+0000 7fddb8c00640 1 -- 172.21.15.32:0/3046803224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40afce0 msgr2=0x7fddb4162320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:15.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.967+0000 7fddb8c00640 1 --2- 172.21.15.32:0/3046803224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40afce0 0x7fddb4162320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fddb409ee70 tx=0x7fddac01c230 comp rx=0 tx=0).stop 2024-04-17T09:53:15.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.967+0000 7fddb8c00640 1 -- 172.21.15.32:0/3046803224 shutdown_connections 2024-04-17T09:53:15.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.967+0000 7fddb8c00640 1 --2- 172.21.15.32:0/3046803224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40afce0 0x7fddb4162320 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:15.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.967+0000 7fddb8c00640 1 --2- 172.21.15.32:0/3046803224 >> v2:172.21.15.144:3300/0 conn(0x7fddb406cf80 0x7fddb40af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:15.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.967+0000 7fddb8c00640 1 --2- 172.21.15.32:0/3046803224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddb4141700 0x7fddb406ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:15.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.967+0000 7fddb8c00640 1 -- 172.21.15.32:0/3046803224 >> 172.21.15.32:0/3046803224 conn(0x7fddb4072670 msgr2=0x7fddb4072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:15.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.967+0000 7fddb8c00640 1 -- 172.21.15.32:0/3046803224 shutdown_connections 2024-04-17T09:53:15.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.967+0000 7fddb8c00640 1 -- 172.21.15.32:0/3046803224 wait complete. 2024-04-17T09:53:15.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fddb8c00640 1 Processor -- start 2024-04-17T09:53:15.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fddb8c00640 1 -- start start 2024-04-17T09:53:15.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fddb8c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddb406cf80 0x7fddb4160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fddb8c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40afce0 0x7fddb40b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fddb8c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddb4141700 0x7fddb40b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fddb8c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fddb4141c60 con 0x7fddb40afce0 2024-04-17T09:53:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fddb8c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fddb4141ae0 con 0x7fddb406cf80 2024-04-17T09:53:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fddb8c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fddb40b49d0 con 0x7fddb4141700 2024-04-17T09:53:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fdda9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40afce0 0x7fddb40b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fdda9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40afce0 0x7fddb40b19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46704/0 (socket says 172.21.15.32:46704) 2024-04-17T09:53:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fdda9400640 1 -- 172.21.15.32:0/3009282066 learned_addr learned my addr 172.21.15.32:0/3009282066 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fddb1e00640 1 --2- 172.21.15.32:0/3009282066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddb406cf80 0x7fddb4160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fddb2800640 1 --2- 172.21.15.32:0/3009282066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddb4141700 0x7fddb40b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fdda9400640 1 -- 172.21.15.32:0/3009282066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddb4141700 msgr2=0x7fddb40b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fdda9400640 1 --2- 172.21.15.32:0/3009282066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddb4141700 0x7fddb40b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fdda9400640 1 -- 172.21.15.32:0/3009282066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddb406cf80 msgr2=0x7fddb4160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:15.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fdda9400640 1 --2- 172.21.15.32:0/3009282066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddb406cf80 0x7fddb4160bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:15.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fdda9400640 1 -- 172.21.15.32:0/3009282066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb4140680 con 0x7fddb40afce0 2024-04-17T09:53:15.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fdda9400640 1 --2- 172.21.15.32:0/3009282066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40afce0 0x7fddb40b19f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c00d830 tx=0x7fdd9c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:15.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fddaaa00640 1 -- 172.21.15.32:0/3009282066 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c00b6c0 con 0x7fddb40afce0 2024-04-17T09:53:15.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fddaaa00640 1 -- 172.21.15.32:0/3009282066 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd9c019070 con 0x7fddb40afce0 2024-04-17T09:53:15.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fddb8c00640 1 -- 172.21.15.32:0/3009282066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb413e7f0 con 0x7fddb40afce0 2024-04-17T09:53:15.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fddaaa00640 1 -- 172.21.15.32:0/3009282066 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c00ba60 con 0x7fddb40afce0 2024-04-17T09:53:15.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fddb8c00640 1 -- 172.21.15.32:0/3009282066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb413f2d0 con 0x7fddb40afce0 2024-04-17T09:53:15.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.971+0000 7fddb8c00640 1 -- 172.21.15.32:0/3009282066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb413e5e0 con 0x7fddb40afce0 2024-04-17T09:53:15.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.975+0000 7fddaaa00640 1 -- 172.21.15.32:0/3009282066 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdd9c01e460 con 0x7fddb40afce0 2024-04-17T09:53:15.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.975+0000 7fddaaa00640 1 --2- 172.21.15.32:0/3009282066 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdd90068380 0x7fdd9006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:15.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.975+0000 7fddaaa00640 1 -- 172.21.15.32:0/3009282066 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdd9c087090 con 0x7fddb40afce0 2024-04-17T09:53:15.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.975+0000 7fddb1e00640 1 --2- 172.21.15.32:0/3009282066 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdd90068380 0x7fdd9006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:15.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.975+0000 7fddb1e00640 1 --2- 172.21.15.32:0/3009282066 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdd90068380 0x7fdd9006a840 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdda4003fd0 tx=0x7fdda4003ce0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:15.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:15.979+0000 7fddaaa00640 1 -- 172.21.15.32:0/3009282066 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fdd9c057640 con 0x7fddb40afce0 2024-04-17T09:53:16.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.183+0000 7fddb8c00640 1 -- 172.21.15.32:0/3009282066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fddb4068a90 con 0x7fddb40afce0 2024-04-17T09:53:16.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.791+0000 7fddaaa00640 1 -- 172.21.15.32:0/3009282066 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7fdd9c056f40 con 0x7fddb40afce0 2024-04-17T09:53:16.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.791+0000 7fddb8c00640 1 -- 172.21.15.32:0/3009282066 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdd90068380 msgr2=0x7fdd9006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.791+0000 7fddb8c00640 1 --2- 172.21.15.32:0/3009282066 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdd90068380 0x7fdd9006a840 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdda4003fd0 tx=0x7fdda4003ce0 comp rx=0 tx=0).stop 2024-04-17T09:53:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.791+0000 7fddb8c00640 1 -- 172.21.15.32:0/3009282066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40afce0 msgr2=0x7fddb40b19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.791+0000 7fddb8c00640 1 --2- 172.21.15.32:0/3009282066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40afce0 0x7fddb40b19f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c00d830 tx=0x7fdd9c00dcf0 comp rx=0 tx=0).stop 2024-04-17T09:53:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.791+0000 7fddb8c00640 1 -- 172.21.15.32:0/3009282066 shutdown_connections 2024-04-17T09:53:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.791+0000 7fddb8c00640 1 --2- 172.21.15.32:0/3009282066 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdd90068380 0x7fdd9006a840 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.791+0000 7fddb8c00640 1 --2- 172.21.15.32:0/3009282066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddb4141700 0x7fddb40b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.791+0000 7fddb8c00640 1 --2- 172.21.15.32:0/3009282066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40afce0 0x7fddb40b19f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.791+0000 7fddb8c00640 1 --2- 172.21.15.32:0/3009282066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddb406cf80 0x7fddb4160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.791+0000 7fddb8c00640 1 -- 172.21.15.32:0/3009282066 >> 172.21.15.32:0/3009282066 conn(0x7fddb4072670 msgr2=0x7fddb406ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.791+0000 7fddb8c00640 1 -- 172.21.15.32:0/3009282066 shutdown_connections 2024-04-17T09:53:16.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.791+0000 7fddb8c00640 1 -- 172.21.15.32:0/3009282066 wait complete. 2024-04-17T09:53:16.817 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-17T09:53:16.818 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:53:16.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.955+0000 7f856c200640 1 Processor -- start 2024-04-17T09:53:16.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.955+0000 7f856c200640 1 -- start start 2024-04-17T09:53:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f856c200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856415baa0 0x7f856415bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f856c200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8564072bb0 con 0x7f8564144e90 2024-04-17T09:53:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f856c200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85640b3dc0 con 0x7f856406ef60 2024-04-17T09:53:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f856c200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f856415c440 con 0x7f856415baa0 2024-04-17T09:53:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8569800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8564144e90 0x7f85640b48a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39172/0 (socket says 172.21.15.32:39172) 2024-04-17T09:53:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8569800640 1 -- 172.21.15.32:0/1810982449 learned_addr learned my addr 172.21.15.32:0/1810982449 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8568e00640 1 --2- 172.21.15.32:0/1810982449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856415baa0 0x7f856415bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8563e00640 1 -- 172.21.15.32:0/1810982449 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 976629626 0 0) 0x7f8564072bb0 con 0x7f8564144e90 2024-04-17T09:53:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8563e00640 1 -- 172.21.15.32:0/1810982449 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f854c003530 con 0x7f8564144e90 2024-04-17T09:53:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8563e00640 1 -- 172.21.15.32:0/1810982449 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4289969505 0 0) 0x7f854c003530 con 0x7f8564144e90 2024-04-17T09:53:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8563e00640 1 -- 172.21.15.32:0/1810982449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856415baa0 msgr2=0x7f856415bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8563e00640 1 --2- 172.21.15.32:0/1810982449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856415baa0 0x7f856415bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8563e00640 1 -- 172.21.15.32:0/1810982449 >> v1:172.21.15.144:6789/0 conn(0x7f856406ef60 legacy=0x7f8564071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8563e00640 1 -- 172.21.15.32:0/1810982449 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85640b40a0 con 0x7f8564144e90 2024-04-17T09:53:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8563e00640 1 -- 172.21.15.32:0/1810982449 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f8558001680 con 0x7f8564144e90 2024-04-17T09:53:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8563e00640 1 -- 172.21.15.32:0/1810982449 >> v1:172.21.15.32:6789/0 conn(0x7f8564144e90 legacy=0x7f85640b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8563e00640 1 --2- 172.21.15.32:0/1810982449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f854c0039f0 0x7f854c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8563e00640 1 --2- 172.21.15.32:0/1810982449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f854c004810 0x7f854c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8563e00640 1 --2- 172.21.15.32:0/1810982449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856415baa0 0x7f854c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8563e00640 1 -- 172.21.15.32:0/1810982449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f854c00ba80 con 0x7f854c004810 2024-04-17T09:53:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8563e00640 1 -- 172.21.15.32:0/1810982449 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f854c00bc00 con 0x7f854c0039f0 2024-04-17T09:53:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8563e00640 1 -- 172.21.15.32:0/1810982449 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f854c00e010 con 0x7f856415baa0 2024-04-17T09:53:16.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8568e00640 1 --2- 172.21.15.32:0/1810982449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f854c004810 0x7f854c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:16.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8569800640 1 --2- 172.21.15.32:0/1810982449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f854c0039f0 0x7f854c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:16.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f856a200640 1 --2- 172.21.15.32:0/1810982449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856415baa0 0x7f854c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:16.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8568e00640 1 -- 172.21.15.32:0/1810982449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856415baa0 msgr2=0x7f854c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:16.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8568e00640 1 --2- 172.21.15.32:0/1810982449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856415baa0 0x7f854c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:16.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8568e00640 1 -- 172.21.15.32:0/1810982449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f854c0039f0 msgr2=0x7f854c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:16.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8568e00640 1 --2- 172.21.15.32:0/1810982449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f854c0039f0 0x7f854c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:16.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.959+0000 7f8568e00640 1 -- 172.21.15.32:0/1810982449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854c003530 con 0x7f854c004810 2024-04-17T09:53:16.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.963+0000 7f8568e00640 1 --2- 172.21.15.32:0/1810982449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f854c004810 0x7f854c004c90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8550009ab0 tx=0x7f855001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8303f36464b2d8d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:16.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.963+0000 7f8563e00640 1 -- 172.21.15.32:0/1810982449 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85500b2070 con 0x7f854c004810 2024-04-17T09:53:16.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.963+0000 7f8563e00640 1 -- 172.21.15.32:0/1810982449 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85500a5510 con 0x7f854c004810 2024-04-17T09:53:16.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.963+0000 7f8563e00640 1 -- 172.21.15.32:0/1810982449 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85500096b0 con 0x7f854c004810 2024-04-17T09:53:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.963+0000 7f856c200640 1 -- 172.21.15.32:0/1810982449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f854c004810 msgr2=0x7f854c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.963+0000 7f856c200640 1 --2- 172.21.15.32:0/1810982449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f854c004810 0x7f854c004c90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8550009ab0 tx=0x7f855001ca30 comp rx=0 tx=0).stop 2024-04-17T09:53:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.963+0000 7f856a200640 1 -- 172.21.15.32:0/1810982449 reap_dead start 2024-04-17T09:53:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.963+0000 7f856c200640 1 -- 172.21.15.32:0/1810982449 shutdown_connections 2024-04-17T09:53:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.963+0000 7f856c200640 1 -- 172.21.15.32:0/1810982449 >> 172.21.15.32:0/1810982449 conn(0x7f856406ba30 msgr2=0x7f856406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.963+0000 7f856c200640 1 -- 172.21.15.32:0/1810982449 shutdown_connections 2024-04-17T09:53:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.963+0000 7f856c200640 1 -- 172.21.15.32:0/1810982449 wait complete. 2024-04-17T09:53:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.963+0000 7f856c200640 1 Processor -- start 2024-04-17T09:53:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.963+0000 7f856c200640 1 -- start start 2024-04-17T09:53:16.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.967+0000 7f856c200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f854c004810 0x7f856415ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:16.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.967+0000 7f856c200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85640ae8b0 0x7f85640aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:16.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.967+0000 7f856c200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85640cb990 0x7f85640cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:16.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.967+0000 7f856c200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f85640b4fb0 con 0x7f85640cb990 2024-04-17T09:53:16.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.967+0000 7f856c200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8564145270 con 0x7f85640ae8b0 2024-04-17T09:53:16.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.967+0000 7f856c200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f85640b5130 con 0x7f854c004810 2024-04-17T09:53:16.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.967+0000 7f856a200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85640cb990 0x7f85640cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:16.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.967+0000 7f8569800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f854c004810 0x7f856415ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:16.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.967+0000 7f856a200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85640cb990 0x7f85640cbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46724/0 (socket says 172.21.15.32:46724) 2024-04-17T09:53:16.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.967+0000 7f856a200640 1 -- 172.21.15.32:0/617875754 learned_addr learned my addr 172.21.15.32:0/617875754 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:16.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.967+0000 7f8568e00640 1 --2- 172.21.15.32:0/617875754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85640ae8b0 0x7f85640aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:16.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.967+0000 7f856a200640 1 -- 172.21.15.32:0/617875754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f854c004810 msgr2=0x7f856415ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:16.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.967+0000 7f856a200640 1 --2- 172.21.15.32:0/617875754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f854c004810 0x7f856415ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:16.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.967+0000 7f856a200640 1 -- 172.21.15.32:0/617875754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85640ae8b0 msgr2=0x7f85640aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:16.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.967+0000 7f856a200640 1 --2- 172.21.15.32:0/617875754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85640ae8b0 0x7f85640aed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:16.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.967+0000 7f856a200640 1 -- 172.21.15.32:0/617875754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85640cc350 con 0x7f85640cb990 2024-04-17T09:53:16.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.967+0000 7f856a200640 1 --2- 172.21.15.32:0/617875754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85640cb990 0x7f85640cbe10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f856406fb40 tx=0x7f855c00fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:16.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.967+0000 7f8561600640 1 -- 172.21.15.32:0/617875754 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c01a070 con 0x7f85640cb990 2024-04-17T09:53:16.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.967+0000 7f8561600640 1 -- 172.21.15.32:0/617875754 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f855c00fe60 con 0x7f85640cb990 2024-04-17T09:53:16.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.967+0000 7f856c200640 1 -- 172.21.15.32:0/617875754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85640cc610 con 0x7f85640cb990 2024-04-17T09:53:16.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.967+0000 7f8561600640 1 -- 172.21.15.32:0/617875754 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c003980 con 0x7f85640cb990 2024-04-17T09:53:16.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.967+0000 7f856c200640 1 -- 172.21.15.32:0/617875754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8564135790 con 0x7f85640cb990 2024-04-17T09:53:16.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.967+0000 7f856c200640 1 -- 172.21.15.32:0/617875754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85640af270 con 0x7f85640cb990 2024-04-17T09:53:16.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.971+0000 7f8561600640 1 -- 172.21.15.32:0/617875754 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f855c01d070 con 0x7f85640cb990 2024-04-17T09:53:16.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.971+0000 7f8561600640 1 --2- 172.21.15.32:0/617875754 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f85440683b0 0x7f854406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:16.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.971+0000 7f8561600640 1 -- 172.21.15.32:0/617875754 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f855c085ef0 con 0x7f85640cb990 2024-04-17T09:53:16.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.971+0000 7f8569800640 1 --2- 172.21.15.32:0/617875754 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f85440683b0 0x7f854406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:16.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.971+0000 7f8569800640 1 --2- 172.21.15.32:0/617875754 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f85440683b0 0x7f854406a870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8558000c10 tx=0x7f8558007c00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:16.975+0000 7f8561600640 1 -- 172.21.15.32:0/617875754 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f855c055ef0 con 0x7f85640cb990 2024-04-17T09:53:17.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.175+0000 7f856c200640 1 -- 172.21.15.32:0/617875754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85640b57d0 con 0x7f85640cb990 2024-04-17T09:53:17.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.175+0000 7f8561600640 1 -- 172.21.15.32:0/617875754 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5193 (secure 0 0 0) 0x7f855c05a1f0 con 0x7f85640cb990 2024-04-17T09:53:17.178 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:17.178 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/441201886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":441201886},{"type":"v1","addr":"172.21.15.144:6834","nonce":441201886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1066277472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1066277472},{"type":"v1","addr":"172.21.15.32:6836","nonce":1066277472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/201410023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":201410023},{"type":"v1","addr":"172.21.15.32:6837","nonce":201410023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:53:13.172187+0000","modified":"2024-04-17T09:53:16.188325+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/3332924005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3332924005},{"type":"v1","addr":"172.21.15.144:6835","nonce":3332924005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-17T09:53:17.178 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2024-04-17T09:53:17.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.175+0000 7f856c200640 1 -- 172.21.15.32:0/617875754 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f85440683b0 msgr2=0x7f854406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:17.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.175+0000 7f856c200640 1 --2- 172.21.15.32:0/617875754 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f85440683b0 0x7f854406a870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8558000c10 tx=0x7f8558007c00 comp rx=0 tx=0).stop 2024-04-17T09:53:17.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.175+0000 7f856c200640 1 -- 172.21.15.32:0/617875754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85640cb990 msgr2=0x7f85640cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:17.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.175+0000 7f856c200640 1 --2- 172.21.15.32:0/617875754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85640cb990 0x7f85640cbe10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f856406fb40 tx=0x7f855c00fc10 comp rx=0 tx=0).stop 2024-04-17T09:53:17.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.175+0000 7f856c200640 1 -- 172.21.15.32:0/617875754 shutdown_connections 2024-04-17T09:53:17.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.175+0000 7f856c200640 1 --2- 172.21.15.32:0/617875754 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f85440683b0 0x7f854406a870 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:17.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.175+0000 7f856c200640 1 --2- 172.21.15.32:0/617875754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85640cb990 0x7f85640cbe10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:17.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.175+0000 7f856c200640 1 --2- 172.21.15.32:0/617875754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85640ae8b0 0x7f85640aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:17.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.175+0000 7f856c200640 1 --2- 172.21.15.32:0/617875754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f854c004810 0x7f856415ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:17.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.175+0000 7f856c200640 1 -- 172.21.15.32:0/617875754 >> 172.21.15.32:0/617875754 conn(0x7f856406ba30 msgr2=0x7f856415bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:17.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.175+0000 7f856c200640 1 -- 172.21.15.32:0/617875754 shutdown_connections 2024-04-17T09:53:17.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.175+0000 7f856c200640 1 -- 172.21.15.32:0/617875754 wait complete. 2024-04-17T09:53:17.200 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:53:17.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.339+0000 7f0e16000640 1 Processor -- start 2024-04-17T09:53:17.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.339+0000 7f0e16000640 1 -- start start 2024-04-17T09:53:17.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.339+0000 7f0e16000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e10145f30 0x7f0e10071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:17.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.339+0000 7f0e16000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0e1015d8d0 0x7f0e10072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:17.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.339+0000 7f0e16000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e100cdac0 con 0x7f0e100cce70 2024-04-17T09:53:17.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.339+0000 7f0e16000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0e1006f400 con 0x7f0e1015d8d0 2024-04-17T09:53:17.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.339+0000 7f0e16000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0e10146310 con 0x7f0e10145f30 2024-04-17T09:53:17.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.339+0000 7f0e07e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0e1015d8d0 0x7f0e10072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:17.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.339+0000 7f0e0f400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e10145f30 0x7f0e10071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:17.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.339+0000 7f0e07e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0e1015d8d0 0x7f0e10072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:40672/0 (socket says 172.21.15.32:40672) 2024-04-17T09:53:17.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.339+0000 7f0e07e00640 1 -- 172.21.15.32:0/2415164984 learned_addr learned my addr 172.21.15.32:0/2415164984 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:17.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.339+0000 7f0e07e00640 1 -- 172.21.15.32:0/2415164984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e10145f30 msgr2=0x7f0e10071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:17.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.339+0000 7f0e07e00640 1 --2- 172.21.15.32:0/2415164984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e10145f30 0x7f0e10071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:17.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.339+0000 7f0e07e00640 1 -- 172.21.15.32:0/2415164984 >> v1:172.21.15.144:6789/0 conn(0x7f0e100cce70 legacy=0x7f0e100cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:17.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.339+0000 7f0e07e00640 1 -- 172.21.15.32:0/2415164984 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1015cdf0 con 0x7f0e1015d8d0 2024-04-17T09:53:17.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.343+0000 7f0e07e00640 1 --2- 172.21.15.32:0/2415164984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e1015d8d0 0x7f0e10072330 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009870 tx=0x7f0df801ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78f674b114f6127c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:17.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.343+0000 7f0e0ea00640 1 -- 172.21.15.32:0/2415164984 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df80b5070 con 0x7f0e1015d8d0 2024-04-17T09:53:17.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.343+0000 7f0e0ea00640 1 -- 172.21.15.32:0/2415164984 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df80a8a50 con 0x7f0e1015d8d0 2024-04-17T09:53:17.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.343+0000 7f0e0ea00640 1 -- 172.21.15.32:0/2415164984 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df80a8e20 con 0x7f0e1015d8d0 2024-04-17T09:53:17.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.343+0000 7f0e16000640 1 -- 172.21.15.32:0/2415164984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e1015d8d0 msgr2=0x7f0e10072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:17.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.343+0000 7f0e16000640 1 --2- 172.21.15.32:0/2415164984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e1015d8d0 0x7f0e10072330 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009870 tx=0x7f0df801ed50 comp rx=0 tx=0).stop 2024-04-17T09:53:17.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.343+0000 7f0e16000640 1 -- 172.21.15.32:0/2415164984 shutdown_connections 2024-04-17T09:53:17.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.343+0000 7f0e16000640 1 --2- 172.21.15.32:0/2415164984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e1015d8d0 0x7f0e10072330 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:17.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.343+0000 7f0e16000640 1 --2- 172.21.15.32:0/2415164984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e10145f30 0x7f0e10071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:17.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.343+0000 7f0e16000640 1 -- 172.21.15.32:0/2415164984 >> 172.21.15.32:0/2415164984 conn(0x7f0e1006ba30 msgr2=0x7f0e1006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:17.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.343+0000 7f0e16000640 1 -- 172.21.15.32:0/2415164984 shutdown_connections 2024-04-17T09:53:17.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.343+0000 7f0e16000640 1 -- 172.21.15.32:0/2415164984 wait complete. 2024-04-17T09:53:17.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e16000640 1 Processor -- start 2024-04-17T09:53:17.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e16000640 1 -- start start 2024-04-17T09:53:17.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e16000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e100cce70 0x7f0e1015bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:17.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e16000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e10145f30 0x7f0e10160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:17.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e16000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e1015d8d0 0x7f0e10160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:17.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e16000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1015cb90 con 0x7f0e10145f30 2024-04-17T09:53:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e16000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1015d440 con 0x7f0e1015d8d0 2024-04-17T09:53:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e16000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0e101601d0 con 0x7f0e100cce70 2024-04-17T09:53:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e07e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e10145f30 0x7f0e10160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e07e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e10145f30 0x7f0e10160600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46740/0 (socket says 172.21.15.32:46740) 2024-04-17T09:53:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e07e00640 1 -- 172.21.15.32:0/1861999279 learned_addr learned my addr 172.21.15.32:0/1861999279 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e0f400640 1 --2- 172.21.15.32:0/1861999279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e100cce70 0x7f0e1015bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e0fe00640 1 --2- 172.21.15.32:0/1861999279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e1015d8d0 0x7f0e10160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e07e00640 1 -- 172.21.15.32:0/1861999279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e100cce70 msgr2=0x7f0e1015bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e07e00640 1 --2- 172.21.15.32:0/1861999279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e100cce70 0x7f0e1015bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e07e00640 1 -- 172.21.15.32:0/1861999279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e1015d8d0 msgr2=0x7f0e10160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e07e00640 1 --2- 172.21.15.32:0/1861999279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e1015d8d0 0x7f0e10160b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e07e00640 1 -- 172.21.15.32:0/1861999279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1015c470 con 0x7f0e10145f30 2024-04-17T09:53:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e07e00640 1 --2- 172.21.15.32:0/1861999279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e10145f30 0x7f0e10160600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0df801d400 tx=0x7f0df801baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e07400640 1 -- 172.21.15.32:0/1861999279 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df80b5070 con 0x7f0e10145f30 2024-04-17T09:53:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e07400640 1 -- 172.21.15.32:0/1861999279 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df80b0de0 con 0x7f0e10145f30 2024-04-17T09:53:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e16000640 1 -- 172.21.15.32:0/1861999279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1012dea0 con 0x7f0e10145f30 2024-04-17T09:53:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e07400640 1 -- 172.21.15.32:0/1861999279 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df80bb670 con 0x7f0e10145f30 2024-04-17T09:53:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e16000640 1 -- 172.21.15.32:0/1861999279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e1012e460 con 0x7f0e10145f30 2024-04-17T09:53:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.347+0000 7f0e16000640 1 -- 172.21.15.32:0/1861999279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e10068a90 con 0x7f0e10145f30 2024-04-17T09:53:17.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.351+0000 7f0e07400640 1 -- 172.21.15.32:0/1861999279 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0df80c2060 con 0x7f0e10145f30 2024-04-17T09:53:17.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.351+0000 7f0e07400640 1 --2- 172.21.15.32:0/1861999279 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0dec068380 0x7f0dec06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:17.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.351+0000 7f0e07400640 1 -- 172.21.15.32:0/1861999279 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0df8123e50 con 0x7f0e10145f30 2024-04-17T09:53:17.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.351+0000 7f0e0f400640 1 --2- 172.21.15.32:0/1861999279 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0dec068380 0x7f0dec06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:17.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.351+0000 7f0e0f400640 1 --2- 172.21.15.32:0/1861999279 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0dec068380 0x7f0dec06a840 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009af0 tx=0x7f0e00008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:17.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.355+0000 7f0e07400640 1 -- 172.21.15.32:0/1861999279 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0df80188b0 con 0x7f0e10145f30 2024-04-17T09:53:17.466 INFO:tasks.ceph.mds.b.smithi144.stderr:2024-04-17T09:53:17.463+0000 7f57c4000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T09:53:17.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.519+0000 7f0e16000640 1 -- 172.21.15.32:0/1861999279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e10072f90 con 0x7f0e10145f30 2024-04-17T09:53:17.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.523+0000 7f0e07400640 1 -- 172.21.15.32:0/1861999279 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f0df80f4400 con 0x7f0e10145f30 2024-04-17T09:53:17.525 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:17.525 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:14.779535+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:53:17.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.523+0000 7f0e16000640 1 -- 172.21.15.32:0/1861999279 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0dec068380 msgr2=0x7f0dec06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:17.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.523+0000 7f0e16000640 1 --2- 172.21.15.32:0/1861999279 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0dec068380 0x7f0dec06a840 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009af0 tx=0x7f0e00008040 comp rx=0 tx=0).stop 2024-04-17T09:53:17.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.523+0000 7f0e16000640 1 -- 172.21.15.32:0/1861999279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e10145f30 msgr2=0x7f0e10160600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:17.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.523+0000 7f0e16000640 1 --2- 172.21.15.32:0/1861999279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e10145f30 0x7f0e10160600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0df801d400 tx=0x7f0df801baf0 comp rx=0 tx=0).stop 2024-04-17T09:53:17.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.523+0000 7f0e16000640 1 -- 172.21.15.32:0/1861999279 shutdown_connections 2024-04-17T09:53:17.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.523+0000 7f0e16000640 1 --2- 172.21.15.32:0/1861999279 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0dec068380 0x7f0dec06a840 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:17.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.523+0000 7f0e16000640 1 --2- 172.21.15.32:0/1861999279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e1015d8d0 0x7f0e10160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:17.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.523+0000 7f0e16000640 1 --2- 172.21.15.32:0/1861999279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e10145f30 0x7f0e10160600 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:17.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.523+0000 7f0e16000640 1 --2- 172.21.15.32:0/1861999279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e100cce70 0x7f0e1015bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:17.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.523+0000 7f0e16000640 1 -- 172.21.15.32:0/1861999279 >> 172.21.15.32:0/1861999279 conn(0x7f0e1006ba30 msgr2=0x7f0e10144830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:17.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.523+0000 7f0e16000640 1 -- 172.21.15.32:0/1861999279 shutdown_connections 2024-04-17T09:53:17.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.523+0000 7f0e16000640 1 -- 172.21.15.32:0/1861999279 wait complete. 2024-04-17T09:53:17.547 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T09:53:17.547 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T09:53:17.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.683+0000 7f343ae00640 1 Processor -- start 2024-04-17T09:53:17.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.683+0000 7f343ae00640 1 -- start start 2024-04-17T09:53:17.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.683+0000 7f343ae00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f343406f7c0 0x7f343406fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:17.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.683+0000 7f343ae00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f343415ec50 con 0x7f343415e870 2024-04-17T09:53:17.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.683+0000 7f343ae00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34340d0950 con 0x7f34340b0320 2024-04-17T09:53:17.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.683+0000 7f343ae00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f34340cf810 con 0x7f343406f7c0 2024-04-17T09:53:17.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.683+0000 7f3438e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f343406f7c0 0x7f343406fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:17.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.683+0000 7f3433e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f343415e870 0x7f3434162e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39176/0 (socket says 172.21.15.32:39176) 2024-04-17T09:53:17.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.683+0000 7f3433e00640 1 -- 172.21.15.32:0/1007444629 learned_addr learned my addr 172.21.15.32:0/1007444629 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:17.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.687+0000 7f3438e00640 1 -- 172.21.15.32:0/1007444629 >> v1:172.21.15.144:6789/0 conn(0x7f34340b0320 legacy=0x7f34340d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:17.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.687+0000 7f3438e00640 1 -- 172.21.15.32:0/1007444629 >> v1:172.21.15.32:6789/0 conn(0x7f343415e870 legacy=0x7f3434162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:17.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.687+0000 7f3438e00640 1 -- 172.21.15.32:0/1007444629 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3434067810 con 0x7f343406f7c0 2024-04-17T09:53:17.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.687+0000 7f3433400640 1 -- 172.21.15.32:0/1007444629 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4119266541 0 0) 0x7f343415ec50 con 0x7f343415e870 2024-04-17T09:53:17.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.687+0000 7f3438e00640 1 --2- 172.21.15.32:0/1007444629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f343406f7c0 0x7f343406fc50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f342c009d90 tx=0x7f342c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28245f1e3731088 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:17.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.687+0000 7f3433400640 1 -- 172.21.15.32:0/1007444629 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c0b2070 con 0x7f343406f7c0 2024-04-17T09:53:17.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.687+0000 7f3433400640 1 -- 172.21.15.32:0/1007444629 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f342c0a5830 con 0x7f343406f7c0 2024-04-17T09:53:17.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.687+0000 7f3433400640 1 -- 172.21.15.32:0/1007444629 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c0a5c00 con 0x7f343406f7c0 2024-04-17T09:53:17.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.687+0000 7f343ae00640 1 -- 172.21.15.32:0/1007444629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f343406f7c0 msgr2=0x7f343406fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:17.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.687+0000 7f343ae00640 1 --2- 172.21.15.32:0/1007444629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f343406f7c0 0x7f343406fc50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f342c009d90 tx=0x7f342c01c970 comp rx=0 tx=0).stop 2024-04-17T09:53:17.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f343ae00640 1 -- 172.21.15.32:0/1007444629 shutdown_connections 2024-04-17T09:53:17.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f343ae00640 1 --2- 172.21.15.32:0/1007444629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f343406f7c0 0x7f343406fc50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:17.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f343ae00640 1 -- 172.21.15.32:0/1007444629 >> 172.21.15.32:0/1007444629 conn(0x7f34340d0c00 msgr2=0x7f34340d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:17.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f343ae00640 1 -- 172.21.15.32:0/1007444629 shutdown_connections 2024-04-17T09:53:17.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f343ae00640 1 -- 172.21.15.32:0/1007444629 wait complete. 2024-04-17T09:53:17.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f343ae00640 1 Processor -- start 2024-04-17T09:53:17.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f343ae00640 1 -- start start 2024-04-17T09:53:17.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f343ae00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f343406f7c0 0x7f3434152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:17.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f343ae00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34340b0320 0x7f34340b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:17.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f343ae00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f343415e870 0x7f34340b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:17.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f343ae00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f343416c9b0 con 0x7f343415e870 2024-04-17T09:53:17.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f343ae00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f34340cfdb0 con 0x7f34340b0320 2024-04-17T09:53:17.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f343ae00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f343416cb30 con 0x7f343406f7c0 2024-04-17T09:53:17.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f3438e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f343415e870 0x7f34340b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:17.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f3438e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f343415e870 0x7f34340b5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46760/0 (socket says 172.21.15.32:46760) 2024-04-17T09:53:17.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f3438e00640 1 -- 172.21.15.32:0/655606376 learned_addr learned my addr 172.21.15.32:0/655606376 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:17.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f342b400640 1 --2- 172.21.15.32:0/655606376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34340b0320 0x7f34340b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:17.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f3433e00640 1 --2- 172.21.15.32:0/655606376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f343406f7c0 0x7f3434152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:17.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f3438e00640 1 -- 172.21.15.32:0/655606376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f343406f7c0 msgr2=0x7f3434152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:17.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f3438e00640 1 --2- 172.21.15.32:0/655606376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f343406f7c0 0x7f3434152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:17.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f3438e00640 1 -- 172.21.15.32:0/655606376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34340b0320 msgr2=0x7f34340b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:17.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f3438e00640 1 --2- 172.21.15.32:0/655606376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34340b0320 0x7f34340b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:17.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f3438e00640 1 -- 172.21.15.32:0/655606376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343415c2f0 con 0x7f343415e870 2024-04-17T09:53:17.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.691+0000 7f3438e00640 1 --2- 172.21.15.32:0/655606376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f343415e870 0x7f34340b5440 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f342c0b0040 tx=0x7f342c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:17.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.695+0000 7f3430c00640 1 -- 172.21.15.32:0/655606376 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c0b2040 con 0x7f343415e870 2024-04-17T09:53:17.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.695+0000 7f3430c00640 1 -- 172.21.15.32:0/655606376 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f342c0bf070 con 0x7f343415e870 2024-04-17T09:53:17.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.695+0000 7f3430c00640 1 -- 172.21.15.32:0/655606376 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c0ad900 con 0x7f343415e870 2024-04-17T09:53:17.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.695+0000 7f343ae00640 1 -- 172.21.15.32:0/655606376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f343415c5d0 con 0x7f343415e870 2024-04-17T09:53:17.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.695+0000 7f343ae00640 1 -- 172.21.15.32:0/655606376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f343415cb90 con 0x7f343415e870 2024-04-17T09:53:17.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.695+0000 7f343ae00640 1 -- 172.21.15.32:0/655606376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3434068a90 con 0x7f343415e870 2024-04-17T09:53:17.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.695+0000 7f3430c00640 1 -- 172.21.15.32:0/655606376 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f342c0adaa0 con 0x7f343415e870 2024-04-17T09:53:17.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.695+0000 7f3430c00640 1 --2- 172.21.15.32:0/655606376 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3410068380 0x7f341006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:17.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.695+0000 7f3430c00640 1 -- 172.21.15.32:0/655606376 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f342c121b10 con 0x7f343415e870 2024-04-17T09:53:17.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.695+0000 7f3433e00640 1 --2- 172.21.15.32:0/655606376 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3410068380 0x7f341006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:17.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.695+0000 7f3433e00640 1 --2- 172.21.15.32:0/655606376 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3410068380 0x7f341006a840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3424007bc0 tx=0x7f3424007b10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:17.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.699+0000 7f3430c00640 1 -- 172.21.15.32:0/655606376 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f342c016560 con 0x7f343415e870 2024-04-17T09:53:17.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.863+0000 7f343ae00640 1 -- 172.21.15.32:0/655606376 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3434073690 con 0x7f3410068380 2024-04-17T09:53:17.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.871+0000 7f3430c00640 1 -- 172.21.15.32:0/655606376 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251927 (secure 0 0 0) 0x7f3434073690 con 0x7f3410068380 2024-04-17T09:53:17.875 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:17.878 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T09:53:17.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.875+0000 7f343ae00640 1 -- 172.21.15.32:0/655606376 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3410068380 msgr2=0x7f341006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.875+0000 7f343ae00640 1 --2- 172.21.15.32:0/655606376 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3410068380 0x7f341006a840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3424007bc0 tx=0x7f3424007b10 comp rx=0 tx=0).stop 2024-04-17T09:53:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.875+0000 7f343ae00640 1 -- 172.21.15.32:0/655606376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f343415e870 msgr2=0x7f34340b5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.875+0000 7f343ae00640 1 --2- 172.21.15.32:0/655606376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f343415e870 0x7f34340b5440 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f342c0b0040 tx=0x7f342c019710 comp rx=0 tx=0).stop 2024-04-17T09:53:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.875+0000 7f343ae00640 1 -- 172.21.15.32:0/655606376 shutdown_connections 2024-04-17T09:53:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.875+0000 7f343ae00640 1 --2- 172.21.15.32:0/655606376 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3410068380 0x7f341006a840 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.875+0000 7f343ae00640 1 --2- 172.21.15.32:0/655606376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f343415e870 0x7f34340b5440 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.875+0000 7f343ae00640 1 --2- 172.21.15.32:0/655606376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34340b0320 0x7f34340b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.875+0000 7f343ae00640 1 --2- 172.21.15.32:0/655606376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f343406f7c0 0x7f3434152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.875+0000 7f343ae00640 1 -- 172.21.15.32:0/655606376 >> 172.21.15.32:0/655606376 conn(0x7f34340d0c00 msgr2=0x7f34340d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.875+0000 7f343ae00640 1 -- 172.21.15.32:0/655606376 shutdown_connections 2024-04-17T09:53:17.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:17.875+0000 7f343ae00640 1 -- 172.21.15.32:0/655606376 wait complete. 2024-04-17T09:53:17.896 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-17T09:53:17.285975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239744,"kb_used_data":2760,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734976,"statfs":{"total":773094113280,"available":772848615424,"internally_reserved":0,"allocated":2826240,"data_stored":1717839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.536773"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067861+0000","last_change":"2024-04-17T09:53:13.801310+0000","last_active":"2024-04-17T09:53:15.067861+0000","last_peered":"2024-04-17T09:53:15.067861+0000","last_clean":"2024-04-17T09:53:15.067861+0000","last_became_active":"2024-04-17T09:53:13.800622+0000","last_became_peered":"2024-04-17T09:53:13.800622+0000","last_unstale":"2024-04-17T09:53:15.067861+0000","last_undegraded":"2024-04-17T09:53:15.067861+0000","last_fullsized":"2024-04-17T09:53:15.067861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:10:45.479198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787738+0000","last_change":"2024-04-17T09:53:13.791349+0000","last_active":"2024-04-17T09:53:14.787738+0000","last_peered":"2024-04-17T09:53:14.787738+0000","last_clean":"2024-04-17T09:53:14.787738+0000","last_became_active":"2024-04-17T09:53:13.790726+0000","last_became_peered":"2024-04-17T09:53:13.790726+0000","last_unstale":"2024-04-17T09:53:14.787738+0000","last_undegraded":"2024-04-17T09:53:14.787738+0000","last_fullsized":"2024-04-17T09:53:14.787738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:51:42.871563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.792315+0000","last_change":"2024-04-17T09:53:13.784861+0000","last_active":"2024-04-17T09:53:13.792315+0000","last_peered":"2024-04-17T09:53:13.792315+0000","last_clean":"2024-04-17T09:53:13.792315+0000","last_became_active":"2024-04-17T09:53:13.784550+0000","last_became_peered":"2024-04-17T09:53:13.784550+0000","last_unstale":"2024-04-17T09:53:13.792315+0000","last_undegraded":"2024-04-17T09:53:13.792315+0000","last_fullsized":"2024-04-17T09:53:13.792315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:08.725420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066735+0000","last_change":"2024-04-17T09:53:13.793610+0000","last_active":"2024-04-17T09:53:15.066735+0000","last_peered":"2024-04-17T09:53:15.066735+0000","last_clean":"2024-04-17T09:53:15.066735+0000","last_became_active":"2024-04-17T09:53:13.793176+0000","last_became_peered":"2024-04-17T09:53:13.793176+0000","last_unstale":"2024-04-17T09:53:15.066735+0000","last_undegraded":"2024-04-17T09:53:15.066735+0000","last_fullsized":"2024-04-17T09:53:15.066735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:16:29.674656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067415+0000","last_change":"2024-04-17T09:53:13.799934+0000","last_active":"2024-04-17T09:53:15.067415+0000","last_peered":"2024-04-17T09:53:15.067415+0000","last_clean":"2024-04-17T09:53:15.067415+0000","last_became_active":"2024-04-17T09:53:13.799557+0000","last_became_peered":"2024-04-17T09:53:13.799557+0000","last_unstale":"2024-04-17T09:53:15.067415+0000","last_undegraded":"2024-04-17T09:53:15.067415+0000","last_fullsized":"2024-04-17T09:53:15.067415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:43:23.411109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066691+0000","last_change":"2024-04-17T09:53:13.806419+0000","last_active":"2024-04-17T09:53:15.066691+0000","last_peered":"2024-04-17T09:53:15.066691+0000","last_clean":"2024-04-17T09:53:15.066691+0000","last_became_active":"2024-04-17T09:53:13.805216+0000","last_became_peered":"2024-04-17T09:53:13.805216+0000","last_unstale":"2024-04-17T09:53:15.066691+0000","last_undegraded":"2024-04-17T09:53:15.066691+0000","last_fullsized":"2024-04-17T09:53:15.066691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:58:50.680456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066488+0000","last_change":"2024-04-17T09:53:13.799827+0000","last_active":"2024-04-17T09:53:15.066488+0000","last_peered":"2024-04-17T09:53:15.066488+0000","last_clean":"2024-04-17T09:53:15.066488+0000","last_became_active":"2024-04-17T09:53:13.799483+0000","last_became_peered":"2024-04-17T09:53:13.799483+0000","last_unstale":"2024-04-17T09:53:15.066488+0000","last_undegraded":"2024-04-17T09:53:15.066488+0000","last_fullsized":"2024-04-17T09:53:15.066488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:16:29.674656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066098+0000","last_change":"2024-04-17T09:53:13.805644+0000","last_active":"2024-04-17T09:53:15.066098+0000","last_peered":"2024-04-17T09:53:15.066098+0000","last_clean":"2024-04-17T09:53:15.066098+0000","last_became_active":"2024-04-17T09:53:13.805343+0000","last_became_peered":"2024-04-17T09:53:13.805343+0000","last_unstale":"2024-04-17T09:53:15.066098+0000","last_undegraded":"2024-04-17T09:53:15.066098+0000","last_fullsized":"2024-04-17T09:53:15.066098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:43:18.222168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067921+0000","last_change":"2024-04-17T09:53:13.815771+0000","last_active":"2024-04-17T09:53:15.067921+0000","last_peered":"2024-04-17T09:53:15.067921+0000","last_clean":"2024-04-17T09:53:15.067921+0000","last_became_active":"2024-04-17T09:53:13.815042+0000","last_became_peered":"2024-04-17T09:53:13.815042+0000","last_unstale":"2024-04-17T09:53:15.067921+0000","last_undegraded":"2024-04-17T09:53:15.067921+0000","last_fullsized":"2024-04-17T09:53:15.067921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:26:59.317045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.789174+0000","last_change":"2024-04-17T09:53:13.810712+0000","last_active":"2024-04-17T09:53:14.789174+0000","last_peered":"2024-04-17T09:53:14.789174+0000","last_clean":"2024-04-17T09:53:14.789174+0000","last_became_active":"2024-04-17T09:53:13.810369+0000","last_became_peered":"2024-04-17T09:53:13.810369+0000","last_unstale":"2024-04-17T09:53:14.789174+0000","last_undegraded":"2024-04-17T09:53:14.789174+0000","last_fullsized":"2024-04-17T09:53:14.789174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:14.645740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788592+0000","last_change":"2024-04-17T09:53:13.800154+0000","last_active":"2024-04-17T09:53:14.788592+0000","last_peered":"2024-04-17T09:53:14.788592+0000","last_clean":"2024-04-17T09:53:14.788592+0000","last_became_active":"2024-04-17T09:53:13.799871+0000","last_became_peered":"2024-04-17T09:53:13.799871+0000","last_unstale":"2024-04-17T09:53:14.788592+0000","last_undegraded":"2024-04-17T09:53:14.788592+0000","last_fullsized":"2024-04-17T09:53:14.788592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:02.320431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.809676+0000","last_change":"2024-04-17T09:53:13.805171+0000","last_active":"2024-04-17T09:53:13.809676+0000","last_peered":"2024-04-17T09:53:13.809676+0000","last_clean":"2024-04-17T09:53:13.809676+0000","last_became_active":"2024-04-17T09:53:13.804823+0000","last_became_peered":"2024-04-17T09:53:13.804823+0000","last_unstale":"2024-04-17T09:53:13.809676+0000","last_undegraded":"2024-04-17T09:53:13.809676+0000","last_fullsized":"2024-04-17T09:53:13.809676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:42:55.357531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066684+0000","last_change":"2024-04-17T09:53:13.789478+0000","last_active":"2024-04-17T09:53:15.066684+0000","last_peered":"2024-04-17T09:53:15.066684+0000","last_clean":"2024-04-17T09:53:15.066684+0000","last_became_active":"2024-04-17T09:53:13.789168+0000","last_became_peered":"2024-04-17T09:53:13.789168+0000","last_unstale":"2024-04-17T09:53:15.066684+0000","last_undegraded":"2024-04-17T09:53:15.066684+0000","last_fullsized":"2024-04-17T09:53:15.066684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:14.645740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.797363+0000","last_change":"2024-04-17T09:53:13.794592+0000","last_active":"2024-04-17T09:53:13.797363+0000","last_peered":"2024-04-17T09:53:13.797363+0000","last_clean":"2024-04-17T09:53:13.797363+0000","last_became_active":"2024-04-17T09:53:13.794323+0000","last_became_peered":"2024-04-17T09:53:13.794323+0000","last_unstale":"2024-04-17T09:53:13.797363+0000","last_undegraded":"2024-04-17T09:53:13.797363+0000","last_fullsized":"2024-04-17T09:53:13.797363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:51:44.795158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787454+0000","last_change":"2024-04-17T09:53:13.792566+0000","last_active":"2024-04-17T09:53:14.787454+0000","last_peered":"2024-04-17T09:53:14.787454+0000","last_clean":"2024-04-17T09:53:14.787454+0000","last_became_active":"2024-04-17T09:53:13.791848+0000","last_became_peered":"2024-04-17T09:53:13.791848+0000","last_unstale":"2024-04-17T09:53:14.787454+0000","last_undegraded":"2024-04-17T09:53:14.787454+0000","last_fullsized":"2024-04-17T09:53:14.787454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:17:54.590392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066342+0000","last_change":"2024-04-17T09:53:13.815067+0000","last_active":"2024-04-17T09:53:15.066342+0000","last_peered":"2024-04-17T09:53:15.066342+0000","last_clean":"2024-04-17T09:53:15.066342+0000","last_became_active":"2024-04-17T09:53:13.814781+0000","last_became_peered":"2024-04-17T09:53:13.814781+0000","last_unstale":"2024-04-17T09:53:15.066342+0000","last_undegraded":"2024-04-17T09:53:15.066342+0000","last_fullsized":"2024-04-17T09:53:15.066342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:14.645740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067458+0000","last_change":"2024-04-17T09:53:13.814757+0000","last_active":"2024-04-17T09:53:15.067458+0000","last_peered":"2024-04-17T09:53:15.067458+0000","last_clean":"2024-04-17T09:53:15.067458+0000","last_became_active":"2024-04-17T09:53:13.814433+0000","last_became_peered":"2024-04-17T09:53:13.814433+0000","last_unstale":"2024-04-17T09:53:15.067458+0000","last_undegraded":"2024-04-17T09:53:15.067458+0000","last_fullsized":"2024-04-17T09:53:15.067458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:34:37.237156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067453+0000","last_change":"2024-04-17T09:53:13.799374+0000","last_active":"2024-04-17T09:53:15.067453+0000","last_peered":"2024-04-17T09:53:15.067453+0000","last_clean":"2024-04-17T09:53:15.067453+0000","last_became_active":"2024-04-17T09:53:13.799117+0000","last_became_peered":"2024-04-17T09:53:13.799117+0000","last_unstale":"2024-04-17T09:53:15.067453+0000","last_undegraded":"2024-04-17T09:53:15.067453+0000","last_fullsized":"2024-04-17T09:53:15.067453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:54.146581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067542+0000","last_change":"2024-04-17T09:53:13.798276+0000","last_active":"2024-04-17T09:53:15.067542+0000","last_peered":"2024-04-17T09:53:15.067542+0000","last_clean":"2024-04-17T09:53:15.067542+0000","last_became_active":"2024-04-17T09:53:13.796681+0000","last_became_peered":"2024-04-17T09:53:13.796681+0000","last_unstale":"2024-04-17T09:53:15.067542+0000","last_undegraded":"2024-04-17T09:53:15.067542+0000","last_fullsized":"2024-04-17T09:53:15.067542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:27:15.142091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788909+0000","last_change":"2024-04-17T09:53:13.804880+0000","last_active":"2024-04-17T09:53:14.788909+0000","last_peered":"2024-04-17T09:53:14.788909+0000","last_clean":"2024-04-17T09:53:14.788909+0000","last_became_active":"2024-04-17T09:53:13.804615+0000","last_became_peered":"2024-04-17T09:53:13.804615+0000","last_unstale":"2024-04-17T09:53:14.788909+0000","last_undegraded":"2024-04-17T09:53:14.788909+0000","last_fullsized":"2024-04-17T09:53:14.788909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:17.663523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.802173+0000","last_change":"2024-04-17T09:53:13.800703+0000","last_active":"2024-04-17T09:53:13.802173+0000","last_peered":"2024-04-17T09:53:13.802173+0000","last_clean":"2024-04-17T09:53:13.802173+0000","last_became_active":"2024-04-17T09:53:13.800353+0000","last_became_peered":"2024-04-17T09:53:13.800353+0000","last_unstale":"2024-04-17T09:53:13.802173+0000","last_undegraded":"2024-04-17T09:53:13.802173+0000","last_fullsized":"2024-04-17T09:53:13.802173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:26.904134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066652+0000","last_change":"2024-04-17T09:53:13.809406+0000","last_active":"2024-04-17T09:53:15.066652+0000","last_peered":"2024-04-17T09:53:15.066652+0000","last_clean":"2024-04-17T09:53:15.066652+0000","last_became_active":"2024-04-17T09:53:13.809152+0000","last_became_peered":"2024-04-17T09:53:13.809152+0000","last_unstale":"2024-04-17T09:53:15.066652+0000","last_undegraded":"2024-04-17T09:53:15.066652+0000","last_fullsized":"2024-04-17T09:53:15.066652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:55:01.176803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066430+0000","last_change":"2024-04-17T09:53:13.785293+0000","last_active":"2024-04-17T09:53:15.066430+0000","last_peered":"2024-04-17T09:53:15.066430+0000","last_clean":"2024-04-17T09:53:15.066430+0000","last_became_active":"2024-04-17T09:53:13.785004+0000","last_became_peered":"2024-04-17T09:53:13.785004+0000","last_unstale":"2024-04-17T09:53:15.066430+0000","last_undegraded":"2024-04-17T09:53:15.066430+0000","last_fullsized":"2024-04-17T09:53:15.066430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:17.663523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066443+0000","last_change":"2024-04-17T09:53:13.803100+0000","last_active":"2024-04-17T09:53:15.066443+0000","last_peered":"2024-04-17T09:53:15.066443+0000","last_clean":"2024-04-17T09:53:15.066443+0000","last_became_active":"2024-04-17T09:53:13.802743+0000","last_became_peered":"2024-04-17T09:53:13.802743+0000","last_unstale":"2024-04-17T09:53:15.066443+0000","last_undegraded":"2024-04-17T09:53:15.066443+0000","last_fullsized":"2024-04-17T09:53:15.066443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:51:42.871563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067978+0000","last_change":"2024-04-17T09:53:13.789615+0000","last_active":"2024-04-17T09:53:15.067978+0000","last_peered":"2024-04-17T09:53:15.067978+0000","last_clean":"2024-04-17T09:53:15.067978+0000","last_became_active":"2024-04-17T09:53:13.787774+0000","last_became_peered":"2024-04-17T09:53:13.787774+0000","last_unstale":"2024-04-17T09:53:15.067978+0000","last_undegraded":"2024-04-17T09:53:15.067978+0000","last_fullsized":"2024-04-17T09:53:15.067978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:36:37.128859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.789301+0000","last_change":"2024-04-17T09:53:13.800946+0000","last_active":"2024-04-17T09:53:14.789301+0000","last_peered":"2024-04-17T09:53:14.789301+0000","last_clean":"2024-04-17T09:53:14.789301+0000","last_became_active":"2024-04-17T09:53:13.798598+0000","last_became_peered":"2024-04-17T09:53:13.798598+0000","last_unstale":"2024-04-17T09:53:14.789301+0000","last_undegraded":"2024-04-17T09:53:14.789301+0000","last_fullsized":"2024-04-17T09:53:14.789301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:51:42.871563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787682+0000","last_change":"2024-04-17T09:53:13.792719+0000","last_active":"2024-04-17T09:53:14.787682+0000","last_peered":"2024-04-17T09:53:14.787682+0000","last_clean":"2024-04-17T09:53:14.787682+0000","last_became_active":"2024-04-17T09:53:13.792390+0000","last_became_peered":"2024-04-17T09:53:13.792390+0000","last_unstale":"2024-04-17T09:53:14.787682+0000","last_undegraded":"2024-04-17T09:53:14.787682+0000","last_fullsized":"2024-04-17T09:53:14.787682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:14.645740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067164+0000","last_change":"2024-04-17T09:53:13.793602+0000","last_active":"2024-04-17T09:53:15.067164+0000","last_peered":"2024-04-17T09:53:15.067164+0000","last_clean":"2024-04-17T09:53:15.067164+0000","last_became_active":"2024-04-17T09:53:13.793277+0000","last_became_peered":"2024-04-17T09:53:13.793277+0000","last_unstale":"2024-04-17T09:53:15.067164+0000","last_undegraded":"2024-04-17T09:53:15.067164+0000","last_fullsized":"2024-04-17T09:53:15.067164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:58:50.680456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.787234+0000","last_change":"2024-04-17T09:53:13.781047+0000","last_active":"2024-04-17T09:53:13.787234+0000","last_peered":"2024-04-17T09:53:13.787234+0000","last_clean":"2024-04-17T09:53:13.787234+0000","last_became_active":"2024-04-17T09:53:13.780726+0000","last_became_peered":"2024-04-17T09:53:13.780726+0000","last_unstale":"2024-04-17T09:53:13.787234+0000","last_undegraded":"2024-04-17T09:53:13.787234+0000","last_fullsized":"2024-04-17T09:53:13.787234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:16.046896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066900+0000","last_change":"2024-04-17T09:53:13.806423+0000","last_active":"2024-04-17T09:53:15.066900+0000","last_peered":"2024-04-17T09:53:15.066900+0000","last_clean":"2024-04-17T09:53:15.066900+0000","last_became_active":"2024-04-17T09:53:13.804648+0000","last_became_peered":"2024-04-17T09:53:13.804648+0000","last_unstale":"2024-04-17T09:53:15.066900+0000","last_undegraded":"2024-04-17T09:53:15.066900+0000","last_fullsized":"2024-04-17T09:53:15.066900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:30:17.695383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066152+0000","last_change":"2024-04-17T09:53:13.803405+0000","last_active":"2024-04-17T09:53:15.066152+0000","last_peered":"2024-04-17T09:53:15.066152+0000","last_clean":"2024-04-17T09:53:15.066152+0000","last_became_active":"2024-04-17T09:53:13.803135+0000","last_became_peered":"2024-04-17T09:53:13.803135+0000","last_unstale":"2024-04-17T09:53:15.066152+0000","last_undegraded":"2024-04-17T09:53:15.066152+0000","last_fullsized":"2024-04-17T09:53:15.066152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:12.936111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787899+0000","last_change":"2024-04-17T09:53:13.786719+0000","last_active":"2024-04-17T09:53:14.787899+0000","last_peered":"2024-04-17T09:53:14.787899+0000","last_clean":"2024-04-17T09:53:14.787899+0000","last_became_active":"2024-04-17T09:53:13.786401+0000","last_became_peered":"2024-04-17T09:53:13.786401+0000","last_unstale":"2024-04-17T09:53:14.787899+0000","last_undegraded":"2024-04-17T09:53:14.787899+0000","last_fullsized":"2024-04-17T09:53:14.787899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:55:00.842846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.811507+0000","last_change":"2024-04-17T09:53:13.797469+0000","last_active":"2024-04-17T09:53:13.811507+0000","last_peered":"2024-04-17T09:53:13.811507+0000","last_clean":"2024-04-17T09:53:13.811507+0000","last_became_active":"2024-04-17T09:53:13.797180+0000","last_became_peered":"2024-04-17T09:53:13.797180+0000","last_unstale":"2024-04-17T09:53:13.811507+0000","last_undegraded":"2024-04-17T09:53:13.811507+0000","last_fullsized":"2024-04-17T09:53:13.811507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:49.307371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066953+0000","last_change":"2024-04-17T09:53:12.783002+0000","last_active":"2024-04-17T09:53:15.066953+0000","last_peered":"2024-04-17T09:53:15.066953+0000","last_clean":"2024-04-17T09:53:15.066953+0000","last_became_active":"2024-04-17T09:53:12.782606+0000","last_became_peered":"2024-04-17T09:53:12.782606+0000","last_unstale":"2024-04-17T09:53:15.066953+0000","last_undegraded":"2024-04-17T09:53:15.066953+0000","last_fullsized":"2024-04-17T09:53:15.066953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:52:55.917109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.787506+0000","last_change":"2024-04-17T09:53:13.784184+0000","last_active":"2024-04-17T09:53:13.787506+0000","last_peered":"2024-04-17T09:53:13.787506+0000","last_clean":"2024-04-17T09:53:13.787506+0000","last_became_active":"2024-04-17T09:53:13.783922+0000","last_became_peered":"2024-04-17T09:53:13.783922+0000","last_unstale":"2024-04-17T09:53:13.787506+0000","last_undegraded":"2024-04-17T09:53:13.787506+0000","last_fullsized":"2024-04-17T09:53:13.787506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:42:55.357531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788344+0000","last_change":"2024-04-17T09:53:12.768842+0000","last_active":"2024-04-17T09:53:14.788344+0000","last_peered":"2024-04-17T09:53:14.788344+0000","last_clean":"2024-04-17T09:53:14.788344+0000","last_became_active":"2024-04-17T09:53:12.768227+0000","last_became_peered":"2024-04-17T09:53:12.768227+0000","last_unstale":"2024-04-17T09:53:14.788344+0000","last_undegraded":"2024-04-17T09:53:14.788344+0000","last_fullsized":"2024-04-17T09:53:14.788344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:32:56.010181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066590+0000","last_change":"2024-04-17T09:53:12.773398+0000","last_active":"2024-04-17T09:53:15.066590+0000","last_peered":"2024-04-17T09:53:15.066590+0000","last_clean":"2024-04-17T09:53:15.066590+0000","last_became_active":"2024-04-17T09:53:12.771487+0000","last_became_peered":"2024-04-17T09:53:12.771487+0000","last_unstale":"2024-04-17T09:53:15.066590+0000","last_undegraded":"2024-04-17T09:53:15.066590+0000","last_fullsized":"2024-04-17T09:53:15.066590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:24.555970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066239+0000","last_change":"2024-04-17T09:53:13.782988+0000","last_active":"2024-04-17T09:53:15.066239+0000","last_peered":"2024-04-17T09:53:15.066239+0000","last_clean":"2024-04-17T09:53:15.066239+0000","last_became_active":"2024-04-17T09:53:13.782711+0000","last_became_peered":"2024-04-17T09:53:13.782711+0000","last_unstale":"2024-04-17T09:53:15.066239+0000","last_undegraded":"2024-04-17T09:53:15.066239+0000","last_fullsized":"2024-04-17T09:53:15.066239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:02.320431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066522+0000","last_change":"2024-04-17T09:53:12.778661+0000","last_active":"2024-04-17T09:53:15.066522+0000","last_peered":"2024-04-17T09:53:15.066522+0000","last_clean":"2024-04-17T09:53:15.066522+0000","last_became_active":"2024-04-17T09:53:12.778088+0000","last_became_peered":"2024-04-17T09:53:12.778088+0000","last_unstale":"2024-04-17T09:53:15.066522+0000","last_undegraded":"2024-04-17T09:53:15.066522+0000","last_fullsized":"2024-04-17T09:53:15.066522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:50.549367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787383+0000","last_change":"2024-04-17T09:53:13.792711+0000","last_active":"2024-04-17T09:53:14.787383+0000","last_peered":"2024-04-17T09:53:14.787383+0000","last_clean":"2024-04-17T09:53:14.787383+0000","last_became_active":"2024-04-17T09:53:13.792257+0000","last_became_peered":"2024-04-17T09:53:13.792257+0000","last_unstale":"2024-04-17T09:53:14.787383+0000","last_undegraded":"2024-04-17T09:53:14.787383+0000","last_fullsized":"2024-04-17T09:53:14.787383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:43:18.222168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.789124+0000","last_change":"2024-04-17T09:53:12.779227+0000","last_active":"2024-04-17T09:53:14.789124+0000","last_peered":"2024-04-17T09:53:14.789124+0000","last_clean":"2024-04-17T09:53:14.789124+0000","last_became_active":"2024-04-17T09:53:12.778942+0000","last_became_peered":"2024-04-17T09:53:12.778942+0000","last_unstale":"2024-04-17T09:53:14.789124+0000","last_undegraded":"2024-04-17T09:53:14.789124+0000","last_fullsized":"2024-04-17T09:53:14.789124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:07.296217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066798+0000","last_change":"2024-04-17T09:53:13.812353+0000","last_active":"2024-04-17T09:53:15.066798+0000","last_peered":"2024-04-17T09:53:15.066798+0000","last_clean":"2024-04-17T09:53:15.066798+0000","last_became_active":"2024-04-17T09:53:13.812007+0000","last_became_peered":"2024-04-17T09:53:13.812007+0000","last_unstale":"2024-04-17T09:53:15.066798+0000","last_undegraded":"2024-04-17T09:53:15.066798+0000","last_fullsized":"2024-04-17T09:53:15.066798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:36:37.128859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066937+0000","last_change":"2024-04-17T09:53:12.776576+0000","last_active":"2024-04-17T09:53:15.066937+0000","last_peered":"2024-04-17T09:53:15.066937+0000","last_clean":"2024-04-17T09:53:15.066937+0000","last_became_active":"2024-04-17T09:53:12.772424+0000","last_became_peered":"2024-04-17T09:53:12.772424+0000","last_unstale":"2024-04-17T09:53:15.066937+0000","last_undegraded":"2024-04-17T09:53:15.066937+0000","last_fullsized":"2024-04-17T09:53:15.066937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:53.140146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.818630+0000","last_change":"2024-04-17T09:53:13.815907+0000","last_active":"2024-04-17T09:53:13.818630+0000","last_peered":"2024-04-17T09:53:13.818630+0000","last_clean":"2024-04-17T09:53:13.818630+0000","last_became_active":"2024-04-17T09:53:13.815616+0000","last_became_peered":"2024-04-17T09:53:13.815616+0000","last_unstale":"2024-04-17T09:53:13.818630+0000","last_undegraded":"2024-04-17T09:53:13.818630+0000","last_fullsized":"2024-04-17T09:53:13.818630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:20:06.508805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787638+0000","last_change":"2024-04-17T09:53:12.791926+0000","last_active":"2024-04-17T09:53:14.787638+0000","last_peered":"2024-04-17T09:53:14.787638+0000","last_clean":"2024-04-17T09:53:14.787638+0000","last_became_active":"2024-04-17T09:53:12.791644+0000","last_became_peered":"2024-04-17T09:53:12.791644+0000","last_unstale":"2024-04-17T09:53:14.787638+0000","last_undegraded":"2024-04-17T09:53:14.787638+0000","last_fullsized":"2024-04-17T09:53:14.787638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:14.881628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.810932+0000","last_change":"2024-04-17T09:53:13.808620+0000","last_active":"2024-04-17T09:53:13.810932+0000","last_peered":"2024-04-17T09:53:13.810932+0000","last_clean":"2024-04-17T09:53:13.810932+0000","last_became_active":"2024-04-17T09:53:13.807344+0000","last_became_peered":"2024-04-17T09:53:13.807344+0000","last_unstale":"2024-04-17T09:53:13.810932+0000","last_undegraded":"2024-04-17T09:53:13.810932+0000","last_fullsized":"2024-04-17T09:53:13.810932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:52.635851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.789133+0000","last_change":"2024-04-17T09:53:12.771523+0000","last_active":"2024-04-17T09:53:14.789133+0000","last_peered":"2024-04-17T09:53:14.789133+0000","last_clean":"2024-04-17T09:53:14.789133+0000","last_became_active":"2024-04-17T09:53:12.771201+0000","last_became_peered":"2024-04-17T09:53:12.771201+0000","last_unstale":"2024-04-17T09:53:14.789133+0000","last_undegraded":"2024-04-17T09:53:14.789133+0000","last_fullsized":"2024-04-17T09:53:14.789133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:50.549367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067157+0000","last_change":"2024-04-17T09:53:13.797071+0000","last_active":"2024-04-17T09:53:15.067157+0000","last_peered":"2024-04-17T09:53:15.067157+0000","last_clean":"2024-04-17T09:53:15.067157+0000","last_became_active":"2024-04-17T09:53:13.796572+0000","last_became_peered":"2024-04-17T09:53:13.796572+0000","last_unstale":"2024-04-17T09:53:15.067157+0000","last_undegraded":"2024-04-17T09:53:15.067157+0000","last_fullsized":"2024-04-17T09:53:15.067157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:03:55.116831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.812275+0000","last_change":"2024-04-17T09:53:12.778367+0000","last_active":"2024-04-17T09:53:13.812275+0000","last_peered":"2024-04-17T09:53:13.812275+0000","last_clean":"2024-04-17T09:53:13.812275+0000","last_became_active":"2024-04-17T09:53:12.778084+0000","last_became_peered":"2024-04-17T09:53:12.778084+0000","last_unstale":"2024-04-17T09:53:13.812275+0000","last_undegraded":"2024-04-17T09:53:13.812275+0000","last_fullsized":"2024-04-17T09:53:13.812275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:19:28.217974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.800938+0000","last_change":"2024-04-17T09:53:13.798230+0000","last_active":"2024-04-17T09:53:13.800938+0000","last_peered":"2024-04-17T09:53:13.800938+0000","last_clean":"2024-04-17T09:53:13.800938+0000","last_became_active":"2024-04-17T09:53:13.797873+0000","last_became_peered":"2024-04-17T09:53:13.797873+0000","last_unstale":"2024-04-17T09:53:13.800938+0000","last_undegraded":"2024-04-17T09:53:13.800938+0000","last_fullsized":"2024-04-17T09:53:13.800938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:59:22.224091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067037+0000","last_change":"2024-04-17T09:53:12.784831+0000","last_active":"2024-04-17T09:53:15.067037+0000","last_peered":"2024-04-17T09:53:15.067037+0000","last_clean":"2024-04-17T09:53:15.067037+0000","last_became_active":"2024-04-17T09:53:12.784012+0000","last_became_peered":"2024-04-17T09:53:12.784012+0000","last_unstale":"2024-04-17T09:53:15.067037+0000","last_undegraded":"2024-04-17T09:53:15.067037+0000","last_fullsized":"2024-04-17T09:53:15.067037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:09.944062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067415+0000","last_change":"2024-04-17T09:53:13.809261+0000","last_active":"2024-04-17T09:53:15.067415+0000","last_peered":"2024-04-17T09:53:15.067415+0000","last_clean":"2024-04-17T09:53:15.067415+0000","last_became_active":"2024-04-17T09:53:13.809004+0000","last_became_peered":"2024-04-17T09:53:13.809004+0000","last_unstale":"2024-04-17T09:53:15.067415+0000","last_undegraded":"2024-04-17T09:53:15.067415+0000","last_fullsized":"2024-04-17T09:53:15.067415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:33:32.665364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788374+0000","last_change":"2024-04-17T09:53:12.768643+0000","last_active":"2024-04-17T09:53:14.788374+0000","last_peered":"2024-04-17T09:53:14.788374+0000","last_clean":"2024-04-17T09:53:14.788374+0000","last_became_active":"2024-04-17T09:53:12.768221+0000","last_became_peered":"2024-04-17T09:53:12.768221+0000","last_unstale":"2024-04-17T09:53:14.788374+0000","last_undegraded":"2024-04-17T09:53:14.788374+0000","last_fullsized":"2024-04-17T09:53:14.788374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:14:18.407034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788658+0000","last_change":"2024-04-17T09:53:13.788193+0000","last_active":"2024-04-17T09:53:14.788658+0000","last_peered":"2024-04-17T09:53:14.788658+0000","last_clean":"2024-04-17T09:53:14.788658+0000","last_became_active":"2024-04-17T09:53:13.787942+0000","last_became_peered":"2024-04-17T09:53:13.787942+0000","last_unstale":"2024-04-17T09:53:14.788658+0000","last_undegraded":"2024-04-17T09:53:14.788658+0000","last_fullsized":"2024-04-17T09:53:14.788658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:17:54.590392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066252+0000","last_change":"2024-04-17T09:53:12.767452+0000","last_active":"2024-04-17T09:53:15.066252+0000","last_peered":"2024-04-17T09:53:15.066252+0000","last_clean":"2024-04-17T09:53:15.066252+0000","last_became_active":"2024-04-17T09:53:12.766628+0000","last_became_peered":"2024-04-17T09:53:12.766628+0000","last_unstale":"2024-04-17T09:53:15.066252+0000","last_undegraded":"2024-04-17T09:53:15.066252+0000","last_fullsized":"2024-04-17T09:53:15.066252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:07.296217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067067+0000","last_change":"2024-04-17T09:53:13.803929+0000","last_active":"2024-04-17T09:53:15.067067+0000","last_peered":"2024-04-17T09:53:15.067067+0000","last_clean":"2024-04-17T09:53:15.067067+0000","last_became_active":"2024-04-17T09:53:13.803651+0000","last_became_peered":"2024-04-17T09:53:13.803651+0000","last_unstale":"2024-04-17T09:53:15.067067+0000","last_undegraded":"2024-04-17T09:53:15.067067+0000","last_fullsized":"2024-04-17T09:53:15.067067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:36:37.128859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.818058+0000","last_change":"2024-04-17T09:53:12.781091+0000","last_active":"2024-04-17T09:53:13.818058+0000","last_peered":"2024-04-17T09:53:13.818058+0000","last_clean":"2024-04-17T09:53:13.818058+0000","last_became_active":"2024-04-17T09:53:12.780752+0000","last_became_peered":"2024-04-17T09:53:12.780752+0000","last_unstale":"2024-04-17T09:53:13.818058+0000","last_undegraded":"2024-04-17T09:53:13.818058+0000","last_fullsized":"2024-04-17T09:53:13.818058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:31:08.049836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067755+0000","last_change":"2024-04-17T09:53:13.810509+0000","last_active":"2024-04-17T09:53:15.067755+0000","last_peered":"2024-04-17T09:53:15.067755+0000","last_clean":"2024-04-17T09:53:15.067755+0000","last_became_active":"2024-04-17T09:53:13.809888+0000","last_became_peered":"2024-04-17T09:53:13.809888+0000","last_unstale":"2024-04-17T09:53:15.067755+0000","last_undegraded":"2024-04-17T09:53:15.067755+0000","last_fullsized":"2024-04-17T09:53:15.067755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:34:37.237156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067459+0000","last_change":"2024-04-17T09:53:12.778623+0000","last_active":"2024-04-17T09:53:15.067459+0000","last_peered":"2024-04-17T09:53:15.067459+0000","last_clean":"2024-04-17T09:53:15.067459+0000","last_became_active":"2024-04-17T09:53:12.774044+0000","last_became_peered":"2024-04-17T09:53:12.774044+0000","last_unstale":"2024-04-17T09:53:15.067459+0000","last_undegraded":"2024-04-17T09:53:15.067459+0000","last_fullsized":"2024-04-17T09:53:15.067459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:09.944062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.068004+0000","last_change":"2024-04-17T09:53:13.809464+0000","last_active":"2024-04-17T09:53:15.068004+0000","last_peered":"2024-04-17T09:53:15.068004+0000","last_clean":"2024-04-17T09:53:15.068004+0000","last_became_active":"2024-04-17T09:53:13.808615+0000","last_became_peered":"2024-04-17T09:53:13.808615+0000","last_unstale":"2024-04-17T09:53:15.068004+0000","last_undegraded":"2024-04-17T09:53:15.068004+0000","last_fullsized":"2024-04-17T09:53:15.068004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:52:32.791236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787694+0000","last_change":"2024-04-17T09:53:12.776880+0000","last_active":"2024-04-17T09:53:14.787694+0000","last_peered":"2024-04-17T09:53:14.787694+0000","last_clean":"2024-04-17T09:53:14.787694+0000","last_became_active":"2024-04-17T09:53:12.776002+0000","last_became_peered":"2024-04-17T09:53:12.776002+0000","last_unstale":"2024-04-17T09:53:14.787694+0000","last_undegraded":"2024-04-17T09:53:14.787694+0000","last_fullsized":"2024-04-17T09:53:14.787694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:09.944062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066950+0000","last_change":"2024-04-17T09:53:13.793405+0000","last_active":"2024-04-17T09:53:15.066950+0000","last_peered":"2024-04-17T09:53:15.066950+0000","last_clean":"2024-04-17T09:53:15.066950+0000","last_became_active":"2024-04-17T09:53:13.792553+0000","last_became_peered":"2024-04-17T09:53:13.792553+0000","last_unstale":"2024-04-17T09:53:15.066950+0000","last_undegraded":"2024-04-17T09:53:15.066950+0000","last_fullsized":"2024-04-17T09:53:15.066950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:43:23.411109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066128+0000","last_change":"2024-04-17T09:53:12.780453+0000","last_active":"2024-04-17T09:53:15.066128+0000","last_peered":"2024-04-17T09:53:15.066128+0000","last_clean":"2024-04-17T09:53:15.066128+0000","last_became_active":"2024-04-17T09:53:12.780130+0000","last_became_peered":"2024-04-17T09:53:12.780130+0000","last_unstale":"2024-04-17T09:53:15.066128+0000","last_undegraded":"2024-04-17T09:53:15.066128+0000","last_fullsized":"2024-04-17T09:53:15.066128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:14:18.407034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067313+0000","last_change":"2024-04-17T09:53:13.802442+0000","last_active":"2024-04-17T09:53:15.067313+0000","last_peered":"2024-04-17T09:53:15.067313+0000","last_clean":"2024-04-17T09:53:15.067313+0000","last_became_active":"2024-04-17T09:53:13.802168+0000","last_became_peered":"2024-04-17T09:53:13.802168+0000","last_unstale":"2024-04-17T09:53:15.067313+0000","last_undegraded":"2024-04-17T09:53:15.067313+0000","last_fullsized":"2024-04-17T09:53:15.067313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:30:17.695383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066329+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T09:53:15.066329+0000","last_peered":"2024-04-17T09:53:15.066329+0000","last_clean":"2024-04-17T09:53:15.066329+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T09:53:15.066329+0000","last_undegraded":"2024-04-17T09:53:15.066329+0000","last_fullsized":"2024-04-17T09:53:15.066329+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066649+0000","last_change":"2024-04-17T09:53:12.772905+0000","last_active":"2024-04-17T09:53:15.066649+0000","last_peered":"2024-04-17T09:53:15.066649+0000","last_clean":"2024-04-17T09:53:15.066649+0000","last_became_active":"2024-04-17T09:53:12.772635+0000","last_became_peered":"2024-04-17T09:53:12.772635+0000","last_unstale":"2024-04-17T09:53:15.066649+0000","last_undegraded":"2024-04-17T09:53:15.066649+0000","last_fullsized":"2024-04-17T09:53:15.066649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:11.929676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066741+0000","last_change":"2024-04-17T09:53:13.815993+0000","last_active":"2024-04-17T09:53:15.066741+0000","last_peered":"2024-04-17T09:53:15.066741+0000","last_clean":"2024-04-17T09:53:15.066741+0000","last_became_active":"2024-04-17T09:53:13.815689+0000","last_became_peered":"2024-04-17T09:53:13.815689+0000","last_unstale":"2024-04-17T09:53:15.066741+0000","last_undegraded":"2024-04-17T09:53:15.066741+0000","last_fullsized":"2024-04-17T09:53:15.066741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:51:42.871563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066279+0000","last_change":"2024-04-17T09:53:12.790967+0000","last_active":"2024-04-17T09:53:15.066279+0000","last_peered":"2024-04-17T09:53:15.066279+0000","last_clean":"2024-04-17T09:53:15.066279+0000","last_became_active":"2024-04-17T09:53:12.790637+0000","last_became_peered":"2024-04-17T09:53:12.790637+0000","last_unstale":"2024-04-17T09:53:15.066279+0000","last_undegraded":"2024-04-17T09:53:15.066279+0000","last_fullsized":"2024-04-17T09:53:15.066279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:14.881628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.789966+0000","last_change":"2024-04-17T09:53:13.788385+0000","last_active":"2024-04-17T09:53:13.789966+0000","last_peered":"2024-04-17T09:53:13.789966+0000","last_clean":"2024-04-17T09:53:13.789966+0000","last_became_active":"2024-04-17T09:53:13.784112+0000","last_became_peered":"2024-04-17T09:53:13.784112+0000","last_unstale":"2024-04-17T09:53:13.789966+0000","last_undegraded":"2024-04-17T09:53:13.789966+0000","last_fullsized":"2024-04-17T09:53:13.789966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:52.635851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067778+0000","last_change":"2024-04-17T09:53:12.775859+0000","last_active":"2024-04-17T09:53:15.067778+0000","last_peered":"2024-04-17T09:53:15.067778+0000","last_clean":"2024-04-17T09:53:15.067778+0000","last_became_active":"2024-04-17T09:53:12.774051+0000","last_became_peered":"2024-04-17T09:53:12.774051+0000","last_unstale":"2024-04-17T09:53:15.067778+0000","last_undegraded":"2024-04-17T09:53:15.067778+0000","last_fullsized":"2024-04-17T09:53:15.067778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:32:56.010181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787431+0000","last_change":"2024-04-17T09:53:13.792329+0000","last_active":"2024-04-17T09:53:14.787431+0000","last_peered":"2024-04-17T09:53:14.787431+0000","last_clean":"2024-04-17T09:53:14.787431+0000","last_became_active":"2024-04-17T09:53:13.790615+0000","last_became_peered":"2024-04-17T09:53:13.790615+0000","last_unstale":"2024-04-17T09:53:14.787431+0000","last_undegraded":"2024-04-17T09:53:14.787431+0000","last_fullsized":"2024-04-17T09:53:14.787431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:02.320431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067057+0000","last_change":"2024-04-17T09:53:12.787408+0000","last_active":"2024-04-17T09:53:15.067057+0000","last_peered":"2024-04-17T09:53:15.067057+0000","last_clean":"2024-04-17T09:53:15.067057+0000","last_became_active":"2024-04-17T09:53:12.786322+0000","last_became_peered":"2024-04-17T09:53:12.786322+0000","last_unstale":"2024-04-17T09:53:15.067057+0000","last_undegraded":"2024-04-17T09:53:15.067057+0000","last_fullsized":"2024-04-17T09:53:15.067057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:14.881628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.806643+0000","last_change":"2024-04-17T09:53:13.786023+0000","last_active":"2024-04-17T09:53:13.806643+0000","last_peered":"2024-04-17T09:53:13.806643+0000","last_clean":"2024-04-17T09:53:13.806643+0000","last_became_active":"2024-04-17T09:53:13.785528+0000","last_became_peered":"2024-04-17T09:53:13.785528+0000","last_unstale":"2024-04-17T09:53:13.806643+0000","last_undegraded":"2024-04-17T09:53:13.806643+0000","last_fullsized":"2024-04-17T09:53:13.806643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:19:29.224408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787601+0000","last_change":"2024-04-17T09:53:12.786776+0000","last_active":"2024-04-17T09:53:14.787601+0000","last_peered":"2024-04-17T09:53:14.787601+0000","last_clean":"2024-04-17T09:53:14.787601+0000","last_became_active":"2024-04-17T09:53:12.786511+0000","last_became_peered":"2024-04-17T09:53:12.786511+0000","last_unstale":"2024-04-17T09:53:14.787601+0000","last_undegraded":"2024-04-17T09:53:14.787601+0000","last_fullsized":"2024-04-17T09:53:14.787601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:32:56.010181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067174+0000","last_change":"2024-04-17T09:53:13.795421+0000","last_active":"2024-04-17T09:53:15.067174+0000","last_peered":"2024-04-17T09:53:15.067174+0000","last_clean":"2024-04-17T09:53:15.067174+0000","last_became_active":"2024-04-17T09:53:13.793404+0000","last_became_peered":"2024-04-17T09:53:13.793404+0000","last_unstale":"2024-04-17T09:53:15.067174+0000","last_undegraded":"2024-04-17T09:53:15.067174+0000","last_fullsized":"2024-04-17T09:53:15.067174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:27:15.142091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067608+0000","last_change":"2024-04-17T09:53:12.786549+0000","last_active":"2024-04-17T09:53:15.067608+0000","last_peered":"2024-04-17T09:53:15.067608+0000","last_clean":"2024-04-17T09:53:15.067608+0000","last_became_active":"2024-04-17T09:53:12.784745+0000","last_became_peered":"2024-04-17T09:53:12.784745+0000","last_unstale":"2024-04-17T09:53:15.067608+0000","last_undegraded":"2024-04-17T09:53:15.067608+0000","last_fullsized":"2024-04-17T09:53:15.067608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:57:23.703708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787659+0000","last_change":"2024-04-17T09:53:13.792711+0000","last_active":"2024-04-17T09:53:14.787659+0000","last_peered":"2024-04-17T09:53:14.787659+0000","last_clean":"2024-04-17T09:53:14.787659+0000","last_became_active":"2024-04-17T09:53:13.791107+0000","last_became_peered":"2024-04-17T09:53:13.791107+0000","last_unstale":"2024-04-17T09:53:14.787659+0000","last_undegraded":"2024-04-17T09:53:14.787659+0000","last_fullsized":"2024-04-17T09:53:14.787659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:17.663523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.795170+0000","last_change":"2024-04-17T09:53:12.770038+0000","last_active":"2024-04-17T09:53:14.795170+0000","last_peered":"2024-04-17T09:53:14.795170+0000","last_clean":"2024-04-17T09:53:14.795170+0000","last_became_active":"2024-04-17T09:53:12.769750+0000","last_became_peered":"2024-04-17T09:53:12.769750+0000","last_unstale":"2024-04-17T09:53:14.795170+0000","last_undegraded":"2024-04-17T09:53:14.795170+0000","last_fullsized":"2024-04-17T09:53:14.795170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:50.549367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067988+0000","last_change":"2024-04-17T09:53:13.801993+0000","last_active":"2024-04-17T09:53:15.067988+0000","last_peered":"2024-04-17T09:53:15.067988+0000","last_clean":"2024-04-17T09:53:15.067988+0000","last_became_active":"2024-04-17T09:53:13.801328+0000","last_became_peered":"2024-04-17T09:53:13.801328+0000","last_unstale":"2024-04-17T09:53:15.067988+0000","last_undegraded":"2024-04-17T09:53:15.067988+0000","last_fullsized":"2024-04-17T09:53:15.067988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:03:55.116831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.789207+0000","last_change":"2024-04-17T09:53:12.781559+0000","last_active":"2024-04-17T09:53:14.789207+0000","last_peered":"2024-04-17T09:53:14.789207+0000","last_clean":"2024-04-17T09:53:14.789207+0000","last_became_active":"2024-04-17T09:53:12.780702+0000","last_became_peered":"2024-04-17T09:53:12.780702+0000","last_unstale":"2024-04-17T09:53:14.789207+0000","last_undegraded":"2024-04-17T09:53:14.789207+0000","last_fullsized":"2024-04-17T09:53:14.789207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:09.944062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.797479+0000","last_change":"2024-04-17T09:53:13.795015+0000","last_active":"2024-04-17T09:53:13.797479+0000","last_peered":"2024-04-17T09:53:13.797479+0000","last_clean":"2024-04-17T09:53:13.797479+0000","last_became_active":"2024-04-17T09:53:13.794609+0000","last_became_peered":"2024-04-17T09:53:13.794609+0000","last_unstale":"2024-04-17T09:53:13.797479+0000","last_undegraded":"2024-04-17T09:53:13.797479+0000","last_fullsized":"2024-04-17T09:53:13.797479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:09:58.265496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066266+0000","last_change":"2024-04-17T09:53:12.781463+0000","last_active":"2024-04-17T09:53:15.066266+0000","last_peered":"2024-04-17T09:53:15.066266+0000","last_clean":"2024-04-17T09:53:15.066266+0000","last_became_active":"2024-04-17T09:53:12.781138+0000","last_became_peered":"2024-04-17T09:53:12.781138+0000","last_unstale":"2024-04-17T09:53:15.066266+0000","last_undegraded":"2024-04-17T09:53:15.066266+0000","last_fullsized":"2024-04-17T09:53:15.066266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:09.944062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.810444+0000","last_change":"2024-04-17T09:53:13.797478+0000","last_active":"2024-04-17T09:53:13.810444+0000","last_peered":"2024-04-17T09:53:13.810444+0000","last_clean":"2024-04-17T09:53:13.810444+0000","last_became_active":"2024-04-17T09:53:13.797152+0000","last_became_peered":"2024-04-17T09:53:13.797152+0000","last_unstale":"2024-04-17T09:53:13.810444+0000","last_undegraded":"2024-04-17T09:53:13.810444+0000","last_fullsized":"2024-04-17T09:53:13.810444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:16.046896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787486+0000","last_change":"2024-04-17T09:53:12.785848+0000","last_active":"2024-04-17T09:53:14.787486+0000","last_peered":"2024-04-17T09:53:14.787486+0000","last_clean":"2024-04-17T09:53:14.787486+0000","last_became_active":"2024-04-17T09:53:12.784980+0000","last_became_peered":"2024-04-17T09:53:12.784980+0000","last_unstale":"2024-04-17T09:53:14.787486+0000","last_undegraded":"2024-04-17T09:53:14.787486+0000","last_fullsized":"2024-04-17T09:53:14.787486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:11.929676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066406+0000","last_change":"2024-04-17T09:53:13.796607+0000","last_active":"2024-04-17T09:53:15.066406+0000","last_peered":"2024-04-17T09:53:15.066406+0000","last_clean":"2024-04-17T09:53:15.066406+0000","last_became_active":"2024-04-17T09:53:13.796351+0000","last_became_peered":"2024-04-17T09:53:13.796351+0000","last_unstale":"2024-04-17T09:53:15.066406+0000","last_undegraded":"2024-04-17T09:53:15.066406+0000","last_fullsized":"2024-04-17T09:53:15.066406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:17:54.590392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787463+0000","last_change":"2024-04-17T09:53:12.788441+0000","last_active":"2024-04-17T09:53:14.787463+0000","last_peered":"2024-04-17T09:53:14.787463+0000","last_clean":"2024-04-17T09:53:14.787463+0000","last_became_active":"2024-04-17T09:53:12.787431+0000","last_became_peered":"2024-04-17T09:53:12.787431+0000","last_unstale":"2024-04-17T09:53:14.787463+0000","last_undegraded":"2024-04-17T09:53:14.787463+0000","last_fullsized":"2024-04-17T09:53:14.787463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:14:18.407034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066772+0000","last_change":"2024-04-17T09:53:13.783282+0000","last_active":"2024-04-17T09:53:15.066772+0000","last_peered":"2024-04-17T09:53:15.066772+0000","last_clean":"2024-04-17T09:53:15.066772+0000","last_became_active":"2024-04-17T09:53:13.782978+0000","last_became_peered":"2024-04-17T09:53:13.782978+0000","last_unstale":"2024-04-17T09:53:15.066772+0000","last_undegraded":"2024-04-17T09:53:15.066772+0000","last_fullsized":"2024-04-17T09:53:15.066772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:55:01.176803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067538+0000","last_change":"2024-04-17T09:53:12.785555+0000","last_active":"2024-04-17T09:53:15.067538+0000","last_peered":"2024-04-17T09:53:15.067538+0000","last_clean":"2024-04-17T09:53:15.067538+0000","last_became_active":"2024-04-17T09:53:12.785233+0000","last_became_peered":"2024-04-17T09:53:12.785233+0000","last_unstale":"2024-04-17T09:53:15.067538+0000","last_undegraded":"2024-04-17T09:53:15.067538+0000","last_fullsized":"2024-04-17T09:53:15.067538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:50.549367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788225+0000","last_change":"2024-04-17T09:53:13.800230+0000","last_active":"2024-04-17T09:53:14.788225+0000","last_peered":"2024-04-17T09:53:14.788225+0000","last_clean":"2024-04-17T09:53:14.788225+0000","last_became_active":"2024-04-17T09:53:13.799917+0000","last_became_peered":"2024-04-17T09:53:13.799917+0000","last_unstale":"2024-04-17T09:53:14.788225+0000","last_undegraded":"2024-04-17T09:53:14.788225+0000","last_fullsized":"2024-04-17T09:53:14.788225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:43:18.222168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067005+0000","last_change":"2024-04-17T09:53:13.814147+0000","last_active":"2024-04-17T09:53:15.067005+0000","last_peered":"2024-04-17T09:53:15.067005+0000","last_clean":"2024-04-17T09:53:15.067005+0000","last_became_active":"2024-04-17T09:53:13.813807+0000","last_became_peered":"2024-04-17T09:53:13.813807+0000","last_unstale":"2024-04-17T09:53:15.067005+0000","last_undegraded":"2024-04-17T09:53:15.067005+0000","last_fullsized":"2024-04-17T09:53:15.067005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:33:32.665364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067456+0000","last_change":"2024-04-17T09:53:12.788509+0000","last_active":"2024-04-17T09:53:15.067456+0000","last_peered":"2024-04-17T09:53:15.067456+0000","last_clean":"2024-04-17T09:53:15.067456+0000","last_became_active":"2024-04-17T09:53:12.788161+0000","last_became_peered":"2024-04-17T09:53:12.788161+0000","last_unstale":"2024-04-17T09:53:15.067456+0000","last_undegraded":"2024-04-17T09:53:15.067456+0000","last_fullsized":"2024-04-17T09:53:15.067456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:11.929676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066835+0000","last_change":"2024-04-17T09:53:13.812181+0000","last_active":"2024-04-17T09:53:15.066835+0000","last_peered":"2024-04-17T09:53:15.066835+0000","last_clean":"2024-04-17T09:53:15.066835+0000","last_became_active":"2024-04-17T09:53:13.811836+0000","last_became_peered":"2024-04-17T09:53:13.811836+0000","last_unstale":"2024-04-17T09:53:15.066835+0000","last_undegraded":"2024-04-17T09:53:15.066835+0000","last_fullsized":"2024-04-17T09:53:15.066835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:03:55.116831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067392+0000","last_change":"2024-04-17T09:53:12.786015+0000","last_active":"2024-04-17T09:53:15.067392+0000","last_peered":"2024-04-17T09:53:15.067392+0000","last_clean":"2024-04-17T09:53:15.067392+0000","last_became_active":"2024-04-17T09:53:12.784577+0000","last_became_peered":"2024-04-17T09:53:12.784577+0000","last_unstale":"2024-04-17T09:53:15.067392+0000","last_undegraded":"2024-04-17T09:53:15.067392+0000","last_fullsized":"2024-04-17T09:53:15.067392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:53.140146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.821119+0000","last_change":"2024-04-17T09:53:13.819959+0000","last_active":"2024-04-17T09:53:13.821119+0000","last_peered":"2024-04-17T09:53:13.821119+0000","last_clean":"2024-04-17T09:53:13.821119+0000","last_became_active":"2024-04-17T09:53:13.819509+0000","last_became_peered":"2024-04-17T09:53:13.819509+0000","last_unstale":"2024-04-17T09:53:13.821119+0000","last_undegraded":"2024-04-17T09:53:13.821119+0000","last_fullsized":"2024-04-17T09:53:13.821119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:31:53.376511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067396+0000","last_change":"2024-04-17T09:53:12.784607+0000","last_active":"2024-04-17T09:53:15.067396+0000","last_peered":"2024-04-17T09:53:15.067396+0000","last_clean":"2024-04-17T09:53:15.067396+0000","last_became_active":"2024-04-17T09:53:12.784265+0000","last_became_peered":"2024-04-17T09:53:12.784265+0000","last_unstale":"2024-04-17T09:53:15.067396+0000","last_undegraded":"2024-04-17T09:53:15.067396+0000","last_fullsized":"2024-04-17T09:53:15.067396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:07.296217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067248+0000","last_change":"2024-04-17T09:53:13.795050+0000","last_active":"2024-04-17T09:53:15.067248+0000","last_peered":"2024-04-17T09:53:15.067248+0000","last_clean":"2024-04-17T09:53:15.067248+0000","last_became_active":"2024-04-17T09:53:13.794002+0000","last_became_peered":"2024-04-17T09:53:13.794002+0000","last_unstale":"2024-04-17T09:53:15.067248+0000","last_undegraded":"2024-04-17T09:53:15.067248+0000","last_fullsized":"2024-04-17T09:53:15.067248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:16:29.674656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.817942+0000","last_change":"2024-04-17T09:53:12.782069+0000","last_active":"2024-04-17T09:53:13.817942+0000","last_peered":"2024-04-17T09:53:13.817942+0000","last_clean":"2024-04-17T09:53:13.817942+0000","last_became_active":"2024-04-17T09:53:12.781795+0000","last_became_peered":"2024-04-17T09:53:12.781795+0000","last_unstale":"2024-04-17T09:53:13.817942+0000","last_undegraded":"2024-04-17T09:53:13.817942+0000","last_fullsized":"2024-04-17T09:53:13.817942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:48.300937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":794624,"data_stored":656837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":200704,"data_stored":65977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":208896,"data_stored":67499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":196608,"data_stored":65887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":72347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":208896,"data_stored":66013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":778240,"data_stored":649480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T09:53:17.901 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T09:53:18.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.075+0000 7fcad2e00640 1 Processor -- start 2024-04-17T09:53:18.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.075+0000 7fcad2e00640 1 -- start start 2024-04-17T09:53:18.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.079+0000 7fcad2e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcacc144f20 0x7fcacc06e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:18.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.079+0000 7fcad2e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcacc06eda0 0x7fcacc071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:18.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.079+0000 7fcad2e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcacc1481e0 0x7fcacc072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:18.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.079+0000 7fcad2e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fcacc0aed20 con 0x7fcacc1481e0 2024-04-17T09:53:18.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.079+0000 7fcad2e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fcacc0af000 con 0x7fcacc144f20 2024-04-17T09:53:18.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.079+0000 7fcad2e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcacc0af2e0 con 0x7fcacc06eda0 2024-04-17T09:53:18.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.079+0000 7fcad0e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcacc1481e0 0x7fcacc072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:18.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.079+0000 7fcad0e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcacc1481e0 0x7fcacc072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46774/0 (socket says 172.21.15.32:46774) 2024-04-17T09:53:18.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.079+0000 7fcad0e00640 1 -- 172.21.15.32:0/3443749763 learned_addr learned my addr 172.21.15.32:0/3443749763 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:18.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.079+0000 7fcac3e00640 1 --2- 172.21.15.32:0/3443749763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcacc06eda0 0x7fcacc071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:18.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.079+0000 7fcacbe00640 1 --2- 172.21.15.32:0/3443749763 >> v2:172.21.15.144:3300/0 conn(0x7fcacc144f20 0x7fcacc06e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:18.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.079+0000 7fcad0e00640 1 -- 172.21.15.32:0/3443749763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcacc06eda0 msgr2=0x7fcacc071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:18.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.079+0000 7fcad0e00640 1 --2- 172.21.15.32:0/3443749763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcacc06eda0 0x7fcacc071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.079+0000 7fcad0e00640 1 -- 172.21.15.32:0/3443749763 >> v2:172.21.15.144:3300/0 conn(0x7fcacc144f20 msgr2=0x7fcacc06e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:18.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.079+0000 7fcad0e00640 1 --2- 172.21.15.32:0/3443749763 >> v2:172.21.15.144:3300/0 conn(0x7fcacc144f20 0x7fcacc06e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.079+0000 7fcad0e00640 1 -- 172.21.15.32:0/3443749763 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcacc0af5c0 con 0x7fcacc1481e0 2024-04-17T09:53:18.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.083+0000 7fcad0e00640 1 --2- 172.21.15.32:0/3443749763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcacc1481e0 0x7fcacc072330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcacc06ca90 tx=0x7fcac401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9e76b8b2927ea7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:18.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.083+0000 7fcacb400640 1 -- 172.21.15.32:0/3443749763 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac40a5680 con 0x7fcacc1481e0 2024-04-17T09:53:18.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.083+0000 7fcacb400640 1 -- 172.21.15.32:0/3443749763 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcac40a5820 con 0x7fcacc1481e0 2024-04-17T09:53:18.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.083+0000 7fcacb400640 1 -- 172.21.15.32:0/3443749763 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac40a5bf0 con 0x7fcacc1481e0 2024-04-17T09:53:18.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.083+0000 7fcad2e00640 1 -- 172.21.15.32:0/3443749763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcacc1481e0 msgr2=0x7fcacc072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:18.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.083+0000 7fcad2e00640 1 --2- 172.21.15.32:0/3443749763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcacc1481e0 0x7fcacc072330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcacc06ca90 tx=0x7fcac401c9f0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.083+0000 7fcad2e00640 1 -- 172.21.15.32:0/3443749763 shutdown_connections 2024-04-17T09:53:18.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.083+0000 7fcad2e00640 1 --2- 172.21.15.32:0/3443749763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcacc1481e0 0x7fcacc072330 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.083+0000 7fcad2e00640 1 --2- 172.21.15.32:0/3443749763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcacc06eda0 0x7fcacc071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.083+0000 7fcad2e00640 1 --2- 172.21.15.32:0/3443749763 >> v2:172.21.15.144:3300/0 conn(0x7fcacc144f20 0x7fcacc06e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.083+0000 7fcad2e00640 1 -- 172.21.15.32:0/3443749763 >> 172.21.15.32:0/3443749763 conn(0x7fcacc06ba30 msgr2=0x7fcacc06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:18.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.083+0000 7fcad2e00640 1 -- 172.21.15.32:0/3443749763 shutdown_connections 2024-04-17T09:53:18.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.083+0000 7fcad2e00640 1 -- 172.21.15.32:0/3443749763 wait complete. 2024-04-17T09:53:18.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.083+0000 7fcad2e00640 1 Processor -- start 2024-04-17T09:53:18.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.083+0000 7fcad2e00640 1 -- start start 2024-04-17T09:53:18.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcad2e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcacc06eda0 0x7fcacc13a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:18.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcad2e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcacc144f20 0x7fcacc15cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:18.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcad2e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcacc1481e0 0x7fcacc15d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:18.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcad2e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc162fe0 con 0x7fcacc06eda0 2024-04-17T09:53:18.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcad2e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc162e60 con 0x7fcacc1481e0 2024-04-17T09:53:18.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcad2e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcacc163160 con 0x7fcacc144f20 2024-04-17T09:53:18.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcacbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcacc06eda0 0x7fcacc13a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:18.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcacbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcacc06eda0 0x7fcacc13a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46784/0 (socket says 172.21.15.32:46784) 2024-04-17T09:53:18.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcacbe00640 1 -- 172.21.15.32:0/302793395 learned_addr learned my addr 172.21.15.32:0/302793395 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:18.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcac3e00640 1 --2- 172.21.15.32:0/302793395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcacc144f20 0x7fcacc15cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:18.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcad0e00640 1 --2- 172.21.15.32:0/302793395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcacc1481e0 0x7fcacc15d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:18.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcacbe00640 1 -- 172.21.15.32:0/302793395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcacc144f20 msgr2=0x7fcacc15cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcacbe00640 1 --2- 172.21.15.32:0/302793395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcacc144f20 0x7fcacc15cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcacbe00640 1 -- 172.21.15.32:0/302793395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcacc1481e0 msgr2=0x7fcacc15d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcacbe00640 1 --2- 172.21.15.32:0/302793395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcacc1481e0 0x7fcacc15d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcacbe00640 1 -- 172.21.15.32:0/302793395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcacc161690 con 0x7fcacc06eda0 2024-04-17T09:53:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcacbe00640 1 --2- 172.21.15.32:0/302793395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcacc06eda0 0x7fcacc13a330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00d800 tx=0x7fcabc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcac8c00640 1 -- 172.21.15.32:0/302793395 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc00b6c0 con 0x7fcacc06eda0 2024-04-17T09:53:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcac8c00640 1 -- 172.21.15.32:0/302793395 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcabc019070 con 0x7fcacc06eda0 2024-04-17T09:53:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcac8c00640 1 -- 172.21.15.32:0/302793395 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc00ba60 con 0x7fcacc06eda0 2024-04-17T09:53:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcad2e00640 1 -- 172.21.15.32:0/302793395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcacc161970 con 0x7fcacc06eda0 2024-04-17T09:53:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcad2e00640 1 -- 172.21.15.32:0/302793395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcacc13a870 con 0x7fcacc06eda0 2024-04-17T09:53:18.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcad2e00640 1 -- 172.21.15.32:0/302793395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcacc068a90 con 0x7fcacc06eda0 2024-04-17T09:53:18.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.087+0000 7fcac8c00640 1 -- 172.21.15.32:0/302793395 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcabc00bc00 con 0x7fcacc06eda0 2024-04-17T09:53:18.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.091+0000 7fcac8c00640 1 --2- 172.21.15.32:0/302793395 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcaa8068380 0x7fcaa806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:18.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.091+0000 7fcac8c00640 1 -- 172.21.15.32:0/302793395 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcabc086d00 con 0x7fcacc06eda0 2024-04-17T09:53:18.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.091+0000 7fcac3e00640 1 --2- 172.21.15.32:0/302793395 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcaa8068380 0x7fcaa806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:18.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.091+0000 7fcac3e00640 1 --2- 172.21.15.32:0/302793395 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcaa8068380 0x7fcaa806a840 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcacc15ea60 tx=0x7fcab4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.095+0000 7fcac8c00640 1 -- 172.21.15.32:0/302793395 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fcabc056c60 con 0x7fcacc06eda0 2024-04-17T09:53:18.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.259+0000 7fcad2e00640 1 -- 172.21.15.32:0/302793395 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcacc06c150 con 0x7fcaa8068380 2024-04-17T09:53:18.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.263+0000 7fcac8c00640 1 -- 172.21.15.32:0/302793395 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251927 (secure 0 0 0) 0x7fcacc06c150 con 0x7fcaa8068380 2024-04-17T09:53:18.272 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:18.273 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T09:53:18.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.267+0000 7fcad2e00640 1 -- 172.21.15.32:0/302793395 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcaa8068380 msgr2=0x7fcaa806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:18.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.267+0000 7fcad2e00640 1 --2- 172.21.15.32:0/302793395 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcaa8068380 0x7fcaa806a840 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcacc15ea60 tx=0x7fcab4008040 comp rx=0 tx=0).stop 2024-04-17T09:53:18.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.267+0000 7fcad2e00640 1 -- 172.21.15.32:0/302793395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcacc06eda0 msgr2=0x7fcacc13a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:18.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.267+0000 7fcad2e00640 1 --2- 172.21.15.32:0/302793395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcacc06eda0 0x7fcacc13a330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00d800 tx=0x7fcabc00dcc0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.271+0000 7fcad2e00640 1 -- 172.21.15.32:0/302793395 shutdown_connections 2024-04-17T09:53:18.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.271+0000 7fcad2e00640 1 --2- 172.21.15.32:0/302793395 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcaa8068380 0x7fcaa806a840 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.271+0000 7fcad2e00640 1 --2- 172.21.15.32:0/302793395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcacc144f20 0x7fcacc15cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.271+0000 7fcad2e00640 1 --2- 172.21.15.32:0/302793395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcacc1481e0 0x7fcacc15d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.271+0000 7fcad2e00640 1 --2- 172.21.15.32:0/302793395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcacc06eda0 0x7fcacc13a330 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.271+0000 7fcad2e00640 1 -- 172.21.15.32:0/302793395 >> 172.21.15.32:0/302793395 conn(0x7fcacc06ba30 msgr2=0x7fcacc143010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:18.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.271+0000 7fcad2e00640 1 -- 172.21.15.32:0/302793395 shutdown_connections 2024-04-17T09:53:18.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.271+0000 7fcad2e00640 1 -- 172.21.15.32:0/302793395 wait complete. 2024-04-17T09:53:18.293 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-17T09:53:17.285975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239744,"kb_used_data":2760,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734976,"statfs":{"total":773094113280,"available":772848615424,"internally_reserved":0,"allocated":2826240,"data_stored":1717839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.536773"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067861+0000","last_change":"2024-04-17T09:53:13.801310+0000","last_active":"2024-04-17T09:53:15.067861+0000","last_peered":"2024-04-17T09:53:15.067861+0000","last_clean":"2024-04-17T09:53:15.067861+0000","last_became_active":"2024-04-17T09:53:13.800622+0000","last_became_peered":"2024-04-17T09:53:13.800622+0000","last_unstale":"2024-04-17T09:53:15.067861+0000","last_undegraded":"2024-04-17T09:53:15.067861+0000","last_fullsized":"2024-04-17T09:53:15.067861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:10:45.479198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787738+0000","last_change":"2024-04-17T09:53:13.791349+0000","last_active":"2024-04-17T09:53:14.787738+0000","last_peered":"2024-04-17T09:53:14.787738+0000","last_clean":"2024-04-17T09:53:14.787738+0000","last_became_active":"2024-04-17T09:53:13.790726+0000","last_became_peered":"2024-04-17T09:53:13.790726+0000","last_unstale":"2024-04-17T09:53:14.787738+0000","last_undegraded":"2024-04-17T09:53:14.787738+0000","last_fullsized":"2024-04-17T09:53:14.787738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:51:42.871563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.792315+0000","last_change":"2024-04-17T09:53:13.784861+0000","last_active":"2024-04-17T09:53:13.792315+0000","last_peered":"2024-04-17T09:53:13.792315+0000","last_clean":"2024-04-17T09:53:13.792315+0000","last_became_active":"2024-04-17T09:53:13.784550+0000","last_became_peered":"2024-04-17T09:53:13.784550+0000","last_unstale":"2024-04-17T09:53:13.792315+0000","last_undegraded":"2024-04-17T09:53:13.792315+0000","last_fullsized":"2024-04-17T09:53:13.792315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:08.725420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066735+0000","last_change":"2024-04-17T09:53:13.793610+0000","last_active":"2024-04-17T09:53:15.066735+0000","last_peered":"2024-04-17T09:53:15.066735+0000","last_clean":"2024-04-17T09:53:15.066735+0000","last_became_active":"2024-04-17T09:53:13.793176+0000","last_became_peered":"2024-04-17T09:53:13.793176+0000","last_unstale":"2024-04-17T09:53:15.066735+0000","last_undegraded":"2024-04-17T09:53:15.066735+0000","last_fullsized":"2024-04-17T09:53:15.066735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:16:29.674656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067415+0000","last_change":"2024-04-17T09:53:13.799934+0000","last_active":"2024-04-17T09:53:15.067415+0000","last_peered":"2024-04-17T09:53:15.067415+0000","last_clean":"2024-04-17T09:53:15.067415+0000","last_became_active":"2024-04-17T09:53:13.799557+0000","last_became_peered":"2024-04-17T09:53:13.799557+0000","last_unstale":"2024-04-17T09:53:15.067415+0000","last_undegraded":"2024-04-17T09:53:15.067415+0000","last_fullsized":"2024-04-17T09:53:15.067415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:43:23.411109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066691+0000","last_change":"2024-04-17T09:53:13.806419+0000","last_active":"2024-04-17T09:53:15.066691+0000","last_peered":"2024-04-17T09:53:15.066691+0000","last_clean":"2024-04-17T09:53:15.066691+0000","last_became_active":"2024-04-17T09:53:13.805216+0000","last_became_peered":"2024-04-17T09:53:13.805216+0000","last_unstale":"2024-04-17T09:53:15.066691+0000","last_undegraded":"2024-04-17T09:53:15.066691+0000","last_fullsized":"2024-04-17T09:53:15.066691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:58:50.680456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066488+0000","last_change":"2024-04-17T09:53:13.799827+0000","last_active":"2024-04-17T09:53:15.066488+0000","last_peered":"2024-04-17T09:53:15.066488+0000","last_clean":"2024-04-17T09:53:15.066488+0000","last_became_active":"2024-04-17T09:53:13.799483+0000","last_became_peered":"2024-04-17T09:53:13.799483+0000","last_unstale":"2024-04-17T09:53:15.066488+0000","last_undegraded":"2024-04-17T09:53:15.066488+0000","last_fullsized":"2024-04-17T09:53:15.066488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:16:29.674656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066098+0000","last_change":"2024-04-17T09:53:13.805644+0000","last_active":"2024-04-17T09:53:15.066098+0000","last_peered":"2024-04-17T09:53:15.066098+0000","last_clean":"2024-04-17T09:53:15.066098+0000","last_became_active":"2024-04-17T09:53:13.805343+0000","last_became_peered":"2024-04-17T09:53:13.805343+0000","last_unstale":"2024-04-17T09:53:15.066098+0000","last_undegraded":"2024-04-17T09:53:15.066098+0000","last_fullsized":"2024-04-17T09:53:15.066098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:43:18.222168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067921+0000","last_change":"2024-04-17T09:53:13.815771+0000","last_active":"2024-04-17T09:53:15.067921+0000","last_peered":"2024-04-17T09:53:15.067921+0000","last_clean":"2024-04-17T09:53:15.067921+0000","last_became_active":"2024-04-17T09:53:13.815042+0000","last_became_peered":"2024-04-17T09:53:13.815042+0000","last_unstale":"2024-04-17T09:53:15.067921+0000","last_undegraded":"2024-04-17T09:53:15.067921+0000","last_fullsized":"2024-04-17T09:53:15.067921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:26:59.317045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.789174+0000","last_change":"2024-04-17T09:53:13.810712+0000","last_active":"2024-04-17T09:53:14.789174+0000","last_peered":"2024-04-17T09:53:14.789174+0000","last_clean":"2024-04-17T09:53:14.789174+0000","last_became_active":"2024-04-17T09:53:13.810369+0000","last_became_peered":"2024-04-17T09:53:13.810369+0000","last_unstale":"2024-04-17T09:53:14.789174+0000","last_undegraded":"2024-04-17T09:53:14.789174+0000","last_fullsized":"2024-04-17T09:53:14.789174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:14.645740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788592+0000","last_change":"2024-04-17T09:53:13.800154+0000","last_active":"2024-04-17T09:53:14.788592+0000","last_peered":"2024-04-17T09:53:14.788592+0000","last_clean":"2024-04-17T09:53:14.788592+0000","last_became_active":"2024-04-17T09:53:13.799871+0000","last_became_peered":"2024-04-17T09:53:13.799871+0000","last_unstale":"2024-04-17T09:53:14.788592+0000","last_undegraded":"2024-04-17T09:53:14.788592+0000","last_fullsized":"2024-04-17T09:53:14.788592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:02.320431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.809676+0000","last_change":"2024-04-17T09:53:13.805171+0000","last_active":"2024-04-17T09:53:13.809676+0000","last_peered":"2024-04-17T09:53:13.809676+0000","last_clean":"2024-04-17T09:53:13.809676+0000","last_became_active":"2024-04-17T09:53:13.804823+0000","last_became_peered":"2024-04-17T09:53:13.804823+0000","last_unstale":"2024-04-17T09:53:13.809676+0000","last_undegraded":"2024-04-17T09:53:13.809676+0000","last_fullsized":"2024-04-17T09:53:13.809676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:42:55.357531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066684+0000","last_change":"2024-04-17T09:53:13.789478+0000","last_active":"2024-04-17T09:53:15.066684+0000","last_peered":"2024-04-17T09:53:15.066684+0000","last_clean":"2024-04-17T09:53:15.066684+0000","last_became_active":"2024-04-17T09:53:13.789168+0000","last_became_peered":"2024-04-17T09:53:13.789168+0000","last_unstale":"2024-04-17T09:53:15.066684+0000","last_undegraded":"2024-04-17T09:53:15.066684+0000","last_fullsized":"2024-04-17T09:53:15.066684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:14.645740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.797363+0000","last_change":"2024-04-17T09:53:13.794592+0000","last_active":"2024-04-17T09:53:13.797363+0000","last_peered":"2024-04-17T09:53:13.797363+0000","last_clean":"2024-04-17T09:53:13.797363+0000","last_became_active":"2024-04-17T09:53:13.794323+0000","last_became_peered":"2024-04-17T09:53:13.794323+0000","last_unstale":"2024-04-17T09:53:13.797363+0000","last_undegraded":"2024-04-17T09:53:13.797363+0000","last_fullsized":"2024-04-17T09:53:13.797363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:51:44.795158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787454+0000","last_change":"2024-04-17T09:53:13.792566+0000","last_active":"2024-04-17T09:53:14.787454+0000","last_peered":"2024-04-17T09:53:14.787454+0000","last_clean":"2024-04-17T09:53:14.787454+0000","last_became_active":"2024-04-17T09:53:13.791848+0000","last_became_peered":"2024-04-17T09:53:13.791848+0000","last_unstale":"2024-04-17T09:53:14.787454+0000","last_undegraded":"2024-04-17T09:53:14.787454+0000","last_fullsized":"2024-04-17T09:53:14.787454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:17:54.590392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066342+0000","last_change":"2024-04-17T09:53:13.815067+0000","last_active":"2024-04-17T09:53:15.066342+0000","last_peered":"2024-04-17T09:53:15.066342+0000","last_clean":"2024-04-17T09:53:15.066342+0000","last_became_active":"2024-04-17T09:53:13.814781+0000","last_became_peered":"2024-04-17T09:53:13.814781+0000","last_unstale":"2024-04-17T09:53:15.066342+0000","last_undegraded":"2024-04-17T09:53:15.066342+0000","last_fullsized":"2024-04-17T09:53:15.066342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:14.645740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067458+0000","last_change":"2024-04-17T09:53:13.814757+0000","last_active":"2024-04-17T09:53:15.067458+0000","last_peered":"2024-04-17T09:53:15.067458+0000","last_clean":"2024-04-17T09:53:15.067458+0000","last_became_active":"2024-04-17T09:53:13.814433+0000","last_became_peered":"2024-04-17T09:53:13.814433+0000","last_unstale":"2024-04-17T09:53:15.067458+0000","last_undegraded":"2024-04-17T09:53:15.067458+0000","last_fullsized":"2024-04-17T09:53:15.067458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:34:37.237156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067453+0000","last_change":"2024-04-17T09:53:13.799374+0000","last_active":"2024-04-17T09:53:15.067453+0000","last_peered":"2024-04-17T09:53:15.067453+0000","last_clean":"2024-04-17T09:53:15.067453+0000","last_became_active":"2024-04-17T09:53:13.799117+0000","last_became_peered":"2024-04-17T09:53:13.799117+0000","last_unstale":"2024-04-17T09:53:15.067453+0000","last_undegraded":"2024-04-17T09:53:15.067453+0000","last_fullsized":"2024-04-17T09:53:15.067453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:54.146581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067542+0000","last_change":"2024-04-17T09:53:13.798276+0000","last_active":"2024-04-17T09:53:15.067542+0000","last_peered":"2024-04-17T09:53:15.067542+0000","last_clean":"2024-04-17T09:53:15.067542+0000","last_became_active":"2024-04-17T09:53:13.796681+0000","last_became_peered":"2024-04-17T09:53:13.796681+0000","last_unstale":"2024-04-17T09:53:15.067542+0000","last_undegraded":"2024-04-17T09:53:15.067542+0000","last_fullsized":"2024-04-17T09:53:15.067542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:27:15.142091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788909+0000","last_change":"2024-04-17T09:53:13.804880+0000","last_active":"2024-04-17T09:53:14.788909+0000","last_peered":"2024-04-17T09:53:14.788909+0000","last_clean":"2024-04-17T09:53:14.788909+0000","last_became_active":"2024-04-17T09:53:13.804615+0000","last_became_peered":"2024-04-17T09:53:13.804615+0000","last_unstale":"2024-04-17T09:53:14.788909+0000","last_undegraded":"2024-04-17T09:53:14.788909+0000","last_fullsized":"2024-04-17T09:53:14.788909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:17.663523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.802173+0000","last_change":"2024-04-17T09:53:13.800703+0000","last_active":"2024-04-17T09:53:13.802173+0000","last_peered":"2024-04-17T09:53:13.802173+0000","last_clean":"2024-04-17T09:53:13.802173+0000","last_became_active":"2024-04-17T09:53:13.800353+0000","last_became_peered":"2024-04-17T09:53:13.800353+0000","last_unstale":"2024-04-17T09:53:13.802173+0000","last_undegraded":"2024-04-17T09:53:13.802173+0000","last_fullsized":"2024-04-17T09:53:13.802173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:26.904134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066652+0000","last_change":"2024-04-17T09:53:13.809406+0000","last_active":"2024-04-17T09:53:15.066652+0000","last_peered":"2024-04-17T09:53:15.066652+0000","last_clean":"2024-04-17T09:53:15.066652+0000","last_became_active":"2024-04-17T09:53:13.809152+0000","last_became_peered":"2024-04-17T09:53:13.809152+0000","last_unstale":"2024-04-17T09:53:15.066652+0000","last_undegraded":"2024-04-17T09:53:15.066652+0000","last_fullsized":"2024-04-17T09:53:15.066652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:55:01.176803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066430+0000","last_change":"2024-04-17T09:53:13.785293+0000","last_active":"2024-04-17T09:53:15.066430+0000","last_peered":"2024-04-17T09:53:15.066430+0000","last_clean":"2024-04-17T09:53:15.066430+0000","last_became_active":"2024-04-17T09:53:13.785004+0000","last_became_peered":"2024-04-17T09:53:13.785004+0000","last_unstale":"2024-04-17T09:53:15.066430+0000","last_undegraded":"2024-04-17T09:53:15.066430+0000","last_fullsized":"2024-04-17T09:53:15.066430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:17.663523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066443+0000","last_change":"2024-04-17T09:53:13.803100+0000","last_active":"2024-04-17T09:53:15.066443+0000","last_peered":"2024-04-17T09:53:15.066443+0000","last_clean":"2024-04-17T09:53:15.066443+0000","last_became_active":"2024-04-17T09:53:13.802743+0000","last_became_peered":"2024-04-17T09:53:13.802743+0000","last_unstale":"2024-04-17T09:53:15.066443+0000","last_undegraded":"2024-04-17T09:53:15.066443+0000","last_fullsized":"2024-04-17T09:53:15.066443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:51:42.871563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067978+0000","last_change":"2024-04-17T09:53:13.789615+0000","last_active":"2024-04-17T09:53:15.067978+0000","last_peered":"2024-04-17T09:53:15.067978+0000","last_clean":"2024-04-17T09:53:15.067978+0000","last_became_active":"2024-04-17T09:53:13.787774+0000","last_became_peered":"2024-04-17T09:53:13.787774+0000","last_unstale":"2024-04-17T09:53:15.067978+0000","last_undegraded":"2024-04-17T09:53:15.067978+0000","last_fullsized":"2024-04-17T09:53:15.067978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:36:37.128859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.789301+0000","last_change":"2024-04-17T09:53:13.800946+0000","last_active":"2024-04-17T09:53:14.789301+0000","last_peered":"2024-04-17T09:53:14.789301+0000","last_clean":"2024-04-17T09:53:14.789301+0000","last_became_active":"2024-04-17T09:53:13.798598+0000","last_became_peered":"2024-04-17T09:53:13.798598+0000","last_unstale":"2024-04-17T09:53:14.789301+0000","last_undegraded":"2024-04-17T09:53:14.789301+0000","last_fullsized":"2024-04-17T09:53:14.789301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:51:42.871563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787682+0000","last_change":"2024-04-17T09:53:13.792719+0000","last_active":"2024-04-17T09:53:14.787682+0000","last_peered":"2024-04-17T09:53:14.787682+0000","last_clean":"2024-04-17T09:53:14.787682+0000","last_became_active":"2024-04-17T09:53:13.792390+0000","last_became_peered":"2024-04-17T09:53:13.792390+0000","last_unstale":"2024-04-17T09:53:14.787682+0000","last_undegraded":"2024-04-17T09:53:14.787682+0000","last_fullsized":"2024-04-17T09:53:14.787682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:14.645740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067164+0000","last_change":"2024-04-17T09:53:13.793602+0000","last_active":"2024-04-17T09:53:15.067164+0000","last_peered":"2024-04-17T09:53:15.067164+0000","last_clean":"2024-04-17T09:53:15.067164+0000","last_became_active":"2024-04-17T09:53:13.793277+0000","last_became_peered":"2024-04-17T09:53:13.793277+0000","last_unstale":"2024-04-17T09:53:15.067164+0000","last_undegraded":"2024-04-17T09:53:15.067164+0000","last_fullsized":"2024-04-17T09:53:15.067164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:58:50.680456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.787234+0000","last_change":"2024-04-17T09:53:13.781047+0000","last_active":"2024-04-17T09:53:13.787234+0000","last_peered":"2024-04-17T09:53:13.787234+0000","last_clean":"2024-04-17T09:53:13.787234+0000","last_became_active":"2024-04-17T09:53:13.780726+0000","last_became_peered":"2024-04-17T09:53:13.780726+0000","last_unstale":"2024-04-17T09:53:13.787234+0000","last_undegraded":"2024-04-17T09:53:13.787234+0000","last_fullsized":"2024-04-17T09:53:13.787234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:16.046896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066900+0000","last_change":"2024-04-17T09:53:13.806423+0000","last_active":"2024-04-17T09:53:15.066900+0000","last_peered":"2024-04-17T09:53:15.066900+0000","last_clean":"2024-04-17T09:53:15.066900+0000","last_became_active":"2024-04-17T09:53:13.804648+0000","last_became_peered":"2024-04-17T09:53:13.804648+0000","last_unstale":"2024-04-17T09:53:15.066900+0000","last_undegraded":"2024-04-17T09:53:15.066900+0000","last_fullsized":"2024-04-17T09:53:15.066900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:30:17.695383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066152+0000","last_change":"2024-04-17T09:53:13.803405+0000","last_active":"2024-04-17T09:53:15.066152+0000","last_peered":"2024-04-17T09:53:15.066152+0000","last_clean":"2024-04-17T09:53:15.066152+0000","last_became_active":"2024-04-17T09:53:13.803135+0000","last_became_peered":"2024-04-17T09:53:13.803135+0000","last_unstale":"2024-04-17T09:53:15.066152+0000","last_undegraded":"2024-04-17T09:53:15.066152+0000","last_fullsized":"2024-04-17T09:53:15.066152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:12.936111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787899+0000","last_change":"2024-04-17T09:53:13.786719+0000","last_active":"2024-04-17T09:53:14.787899+0000","last_peered":"2024-04-17T09:53:14.787899+0000","last_clean":"2024-04-17T09:53:14.787899+0000","last_became_active":"2024-04-17T09:53:13.786401+0000","last_became_peered":"2024-04-17T09:53:13.786401+0000","last_unstale":"2024-04-17T09:53:14.787899+0000","last_undegraded":"2024-04-17T09:53:14.787899+0000","last_fullsized":"2024-04-17T09:53:14.787899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:55:00.842846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.811507+0000","last_change":"2024-04-17T09:53:13.797469+0000","last_active":"2024-04-17T09:53:13.811507+0000","last_peered":"2024-04-17T09:53:13.811507+0000","last_clean":"2024-04-17T09:53:13.811507+0000","last_became_active":"2024-04-17T09:53:13.797180+0000","last_became_peered":"2024-04-17T09:53:13.797180+0000","last_unstale":"2024-04-17T09:53:13.811507+0000","last_undegraded":"2024-04-17T09:53:13.811507+0000","last_fullsized":"2024-04-17T09:53:13.811507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:49.307371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066953+0000","last_change":"2024-04-17T09:53:12.783002+0000","last_active":"2024-04-17T09:53:15.066953+0000","last_peered":"2024-04-17T09:53:15.066953+0000","last_clean":"2024-04-17T09:53:15.066953+0000","last_became_active":"2024-04-17T09:53:12.782606+0000","last_became_peered":"2024-04-17T09:53:12.782606+0000","last_unstale":"2024-04-17T09:53:15.066953+0000","last_undegraded":"2024-04-17T09:53:15.066953+0000","last_fullsized":"2024-04-17T09:53:15.066953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:52:55.917109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.787506+0000","last_change":"2024-04-17T09:53:13.784184+0000","last_active":"2024-04-17T09:53:13.787506+0000","last_peered":"2024-04-17T09:53:13.787506+0000","last_clean":"2024-04-17T09:53:13.787506+0000","last_became_active":"2024-04-17T09:53:13.783922+0000","last_became_peered":"2024-04-17T09:53:13.783922+0000","last_unstale":"2024-04-17T09:53:13.787506+0000","last_undegraded":"2024-04-17T09:53:13.787506+0000","last_fullsized":"2024-04-17T09:53:13.787506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:42:55.357531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788344+0000","last_change":"2024-04-17T09:53:12.768842+0000","last_active":"2024-04-17T09:53:14.788344+0000","last_peered":"2024-04-17T09:53:14.788344+0000","last_clean":"2024-04-17T09:53:14.788344+0000","last_became_active":"2024-04-17T09:53:12.768227+0000","last_became_peered":"2024-04-17T09:53:12.768227+0000","last_unstale":"2024-04-17T09:53:14.788344+0000","last_undegraded":"2024-04-17T09:53:14.788344+0000","last_fullsized":"2024-04-17T09:53:14.788344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:32:56.010181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066590+0000","last_change":"2024-04-17T09:53:12.773398+0000","last_active":"2024-04-17T09:53:15.066590+0000","last_peered":"2024-04-17T09:53:15.066590+0000","last_clean":"2024-04-17T09:53:15.066590+0000","last_became_active":"2024-04-17T09:53:12.771487+0000","last_became_peered":"2024-04-17T09:53:12.771487+0000","last_unstale":"2024-04-17T09:53:15.066590+0000","last_undegraded":"2024-04-17T09:53:15.066590+0000","last_fullsized":"2024-04-17T09:53:15.066590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:24.555970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066239+0000","last_change":"2024-04-17T09:53:13.782988+0000","last_active":"2024-04-17T09:53:15.066239+0000","last_peered":"2024-04-17T09:53:15.066239+0000","last_clean":"2024-04-17T09:53:15.066239+0000","last_became_active":"2024-04-17T09:53:13.782711+0000","last_became_peered":"2024-04-17T09:53:13.782711+0000","last_unstale":"2024-04-17T09:53:15.066239+0000","last_undegraded":"2024-04-17T09:53:15.066239+0000","last_fullsized":"2024-04-17T09:53:15.066239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:02.320431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066522+0000","last_change":"2024-04-17T09:53:12.778661+0000","last_active":"2024-04-17T09:53:15.066522+0000","last_peered":"2024-04-17T09:53:15.066522+0000","last_clean":"2024-04-17T09:53:15.066522+0000","last_became_active":"2024-04-17T09:53:12.778088+0000","last_became_peered":"2024-04-17T09:53:12.778088+0000","last_unstale":"2024-04-17T09:53:15.066522+0000","last_undegraded":"2024-04-17T09:53:15.066522+0000","last_fullsized":"2024-04-17T09:53:15.066522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:50.549367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787383+0000","last_change":"2024-04-17T09:53:13.792711+0000","last_active":"2024-04-17T09:53:14.787383+0000","last_peered":"2024-04-17T09:53:14.787383+0000","last_clean":"2024-04-17T09:53:14.787383+0000","last_became_active":"2024-04-17T09:53:13.792257+0000","last_became_peered":"2024-04-17T09:53:13.792257+0000","last_unstale":"2024-04-17T09:53:14.787383+0000","last_undegraded":"2024-04-17T09:53:14.787383+0000","last_fullsized":"2024-04-17T09:53:14.787383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:43:18.222168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.789124+0000","last_change":"2024-04-17T09:53:12.779227+0000","last_active":"2024-04-17T09:53:14.789124+0000","last_peered":"2024-04-17T09:53:14.789124+0000","last_clean":"2024-04-17T09:53:14.789124+0000","last_became_active":"2024-04-17T09:53:12.778942+0000","last_became_peered":"2024-04-17T09:53:12.778942+0000","last_unstale":"2024-04-17T09:53:14.789124+0000","last_undegraded":"2024-04-17T09:53:14.789124+0000","last_fullsized":"2024-04-17T09:53:14.789124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:07.296217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066798+0000","last_change":"2024-04-17T09:53:13.812353+0000","last_active":"2024-04-17T09:53:15.066798+0000","last_peered":"2024-04-17T09:53:15.066798+0000","last_clean":"2024-04-17T09:53:15.066798+0000","last_became_active":"2024-04-17T09:53:13.812007+0000","last_became_peered":"2024-04-17T09:53:13.812007+0000","last_unstale":"2024-04-17T09:53:15.066798+0000","last_undegraded":"2024-04-17T09:53:15.066798+0000","last_fullsized":"2024-04-17T09:53:15.066798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:36:37.128859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066937+0000","last_change":"2024-04-17T09:53:12.776576+0000","last_active":"2024-04-17T09:53:15.066937+0000","last_peered":"2024-04-17T09:53:15.066937+0000","last_clean":"2024-04-17T09:53:15.066937+0000","last_became_active":"2024-04-17T09:53:12.772424+0000","last_became_peered":"2024-04-17T09:53:12.772424+0000","last_unstale":"2024-04-17T09:53:15.066937+0000","last_undegraded":"2024-04-17T09:53:15.066937+0000","last_fullsized":"2024-04-17T09:53:15.066937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:53.140146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.818630+0000","last_change":"2024-04-17T09:53:13.815907+0000","last_active":"2024-04-17T09:53:13.818630+0000","last_peered":"2024-04-17T09:53:13.818630+0000","last_clean":"2024-04-17T09:53:13.818630+0000","last_became_active":"2024-04-17T09:53:13.815616+0000","last_became_peered":"2024-04-17T09:53:13.815616+0000","last_unstale":"2024-04-17T09:53:13.818630+0000","last_undegraded":"2024-04-17T09:53:13.818630+0000","last_fullsized":"2024-04-17T09:53:13.818630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:20:06.508805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787638+0000","last_change":"2024-04-17T09:53:12.791926+0000","last_active":"2024-04-17T09:53:14.787638+0000","last_peered":"2024-04-17T09:53:14.787638+0000","last_clean":"2024-04-17T09:53:14.787638+0000","last_became_active":"2024-04-17T09:53:12.791644+0000","last_became_peered":"2024-04-17T09:53:12.791644+0000","last_unstale":"2024-04-17T09:53:14.787638+0000","last_undegraded":"2024-04-17T09:53:14.787638+0000","last_fullsized":"2024-04-17T09:53:14.787638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:14.881628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.810932+0000","last_change":"2024-04-17T09:53:13.808620+0000","last_active":"2024-04-17T09:53:13.810932+0000","last_peered":"2024-04-17T09:53:13.810932+0000","last_clean":"2024-04-17T09:53:13.810932+0000","last_became_active":"2024-04-17T09:53:13.807344+0000","last_became_peered":"2024-04-17T09:53:13.807344+0000","last_unstale":"2024-04-17T09:53:13.810932+0000","last_undegraded":"2024-04-17T09:53:13.810932+0000","last_fullsized":"2024-04-17T09:53:13.810932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:52.635851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.789133+0000","last_change":"2024-04-17T09:53:12.771523+0000","last_active":"2024-04-17T09:53:14.789133+0000","last_peered":"2024-04-17T09:53:14.789133+0000","last_clean":"2024-04-17T09:53:14.789133+0000","last_became_active":"2024-04-17T09:53:12.771201+0000","last_became_peered":"2024-04-17T09:53:12.771201+0000","last_unstale":"2024-04-17T09:53:14.789133+0000","last_undegraded":"2024-04-17T09:53:14.789133+0000","last_fullsized":"2024-04-17T09:53:14.789133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:50.549367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067157+0000","last_change":"2024-04-17T09:53:13.797071+0000","last_active":"2024-04-17T09:53:15.067157+0000","last_peered":"2024-04-17T09:53:15.067157+0000","last_clean":"2024-04-17T09:53:15.067157+0000","last_became_active":"2024-04-17T09:53:13.796572+0000","last_became_peered":"2024-04-17T09:53:13.796572+0000","last_unstale":"2024-04-17T09:53:15.067157+0000","last_undegraded":"2024-04-17T09:53:15.067157+0000","last_fullsized":"2024-04-17T09:53:15.067157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:03:55.116831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.812275+0000","last_change":"2024-04-17T09:53:12.778367+0000","last_active":"2024-04-17T09:53:13.812275+0000","last_peered":"2024-04-17T09:53:13.812275+0000","last_clean":"2024-04-17T09:53:13.812275+0000","last_became_active":"2024-04-17T09:53:12.778084+0000","last_became_peered":"2024-04-17T09:53:12.778084+0000","last_unstale":"2024-04-17T09:53:13.812275+0000","last_undegraded":"2024-04-17T09:53:13.812275+0000","last_fullsized":"2024-04-17T09:53:13.812275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:19:28.217974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.800938+0000","last_change":"2024-04-17T09:53:13.798230+0000","last_active":"2024-04-17T09:53:13.800938+0000","last_peered":"2024-04-17T09:53:13.800938+0000","last_clean":"2024-04-17T09:53:13.800938+0000","last_became_active":"2024-04-17T09:53:13.797873+0000","last_became_peered":"2024-04-17T09:53:13.797873+0000","last_unstale":"2024-04-17T09:53:13.800938+0000","last_undegraded":"2024-04-17T09:53:13.800938+0000","last_fullsized":"2024-04-17T09:53:13.800938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:59:22.224091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067037+0000","last_change":"2024-04-17T09:53:12.784831+0000","last_active":"2024-04-17T09:53:15.067037+0000","last_peered":"2024-04-17T09:53:15.067037+0000","last_clean":"2024-04-17T09:53:15.067037+0000","last_became_active":"2024-04-17T09:53:12.784012+0000","last_became_peered":"2024-04-17T09:53:12.784012+0000","last_unstale":"2024-04-17T09:53:15.067037+0000","last_undegraded":"2024-04-17T09:53:15.067037+0000","last_fullsized":"2024-04-17T09:53:15.067037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:09.944062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067415+0000","last_change":"2024-04-17T09:53:13.809261+0000","last_active":"2024-04-17T09:53:15.067415+0000","last_peered":"2024-04-17T09:53:15.067415+0000","last_clean":"2024-04-17T09:53:15.067415+0000","last_became_active":"2024-04-17T09:53:13.809004+0000","last_became_peered":"2024-04-17T09:53:13.809004+0000","last_unstale":"2024-04-17T09:53:15.067415+0000","last_undegraded":"2024-04-17T09:53:15.067415+0000","last_fullsized":"2024-04-17T09:53:15.067415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:33:32.665364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788374+0000","last_change":"2024-04-17T09:53:12.768643+0000","last_active":"2024-04-17T09:53:14.788374+0000","last_peered":"2024-04-17T09:53:14.788374+0000","last_clean":"2024-04-17T09:53:14.788374+0000","last_became_active":"2024-04-17T09:53:12.768221+0000","last_became_peered":"2024-04-17T09:53:12.768221+0000","last_unstale":"2024-04-17T09:53:14.788374+0000","last_undegraded":"2024-04-17T09:53:14.788374+0000","last_fullsized":"2024-04-17T09:53:14.788374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:14:18.407034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788658+0000","last_change":"2024-04-17T09:53:13.788193+0000","last_active":"2024-04-17T09:53:14.788658+0000","last_peered":"2024-04-17T09:53:14.788658+0000","last_clean":"2024-04-17T09:53:14.788658+0000","last_became_active":"2024-04-17T09:53:13.787942+0000","last_became_peered":"2024-04-17T09:53:13.787942+0000","last_unstale":"2024-04-17T09:53:14.788658+0000","last_undegraded":"2024-04-17T09:53:14.788658+0000","last_fullsized":"2024-04-17T09:53:14.788658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:17:54.590392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066252+0000","last_change":"2024-04-17T09:53:12.767452+0000","last_active":"2024-04-17T09:53:15.066252+0000","last_peered":"2024-04-17T09:53:15.066252+0000","last_clean":"2024-04-17T09:53:15.066252+0000","last_became_active":"2024-04-17T09:53:12.766628+0000","last_became_peered":"2024-04-17T09:53:12.766628+0000","last_unstale":"2024-04-17T09:53:15.066252+0000","last_undegraded":"2024-04-17T09:53:15.066252+0000","last_fullsized":"2024-04-17T09:53:15.066252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:07.296217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067067+0000","last_change":"2024-04-17T09:53:13.803929+0000","last_active":"2024-04-17T09:53:15.067067+0000","last_peered":"2024-04-17T09:53:15.067067+0000","last_clean":"2024-04-17T09:53:15.067067+0000","last_became_active":"2024-04-17T09:53:13.803651+0000","last_became_peered":"2024-04-17T09:53:13.803651+0000","last_unstale":"2024-04-17T09:53:15.067067+0000","last_undegraded":"2024-04-17T09:53:15.067067+0000","last_fullsized":"2024-04-17T09:53:15.067067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:36:37.128859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.818058+0000","last_change":"2024-04-17T09:53:12.781091+0000","last_active":"2024-04-17T09:53:13.818058+0000","last_peered":"2024-04-17T09:53:13.818058+0000","last_clean":"2024-04-17T09:53:13.818058+0000","last_became_active":"2024-04-17T09:53:12.780752+0000","last_became_peered":"2024-04-17T09:53:12.780752+0000","last_unstale":"2024-04-17T09:53:13.818058+0000","last_undegraded":"2024-04-17T09:53:13.818058+0000","last_fullsized":"2024-04-17T09:53:13.818058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:31:08.049836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067755+0000","last_change":"2024-04-17T09:53:13.810509+0000","last_active":"2024-04-17T09:53:15.067755+0000","last_peered":"2024-04-17T09:53:15.067755+0000","last_clean":"2024-04-17T09:53:15.067755+0000","last_became_active":"2024-04-17T09:53:13.809888+0000","last_became_peered":"2024-04-17T09:53:13.809888+0000","last_unstale":"2024-04-17T09:53:15.067755+0000","last_undegraded":"2024-04-17T09:53:15.067755+0000","last_fullsized":"2024-04-17T09:53:15.067755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:34:37.237156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067459+0000","last_change":"2024-04-17T09:53:12.778623+0000","last_active":"2024-04-17T09:53:15.067459+0000","last_peered":"2024-04-17T09:53:15.067459+0000","last_clean":"2024-04-17T09:53:15.067459+0000","last_became_active":"2024-04-17T09:53:12.774044+0000","last_became_peered":"2024-04-17T09:53:12.774044+0000","last_unstale":"2024-04-17T09:53:15.067459+0000","last_undegraded":"2024-04-17T09:53:15.067459+0000","last_fullsized":"2024-04-17T09:53:15.067459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:09.944062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.068004+0000","last_change":"2024-04-17T09:53:13.809464+0000","last_active":"2024-04-17T09:53:15.068004+0000","last_peered":"2024-04-17T09:53:15.068004+0000","last_clean":"2024-04-17T09:53:15.068004+0000","last_became_active":"2024-04-17T09:53:13.808615+0000","last_became_peered":"2024-04-17T09:53:13.808615+0000","last_unstale":"2024-04-17T09:53:15.068004+0000","last_undegraded":"2024-04-17T09:53:15.068004+0000","last_fullsized":"2024-04-17T09:53:15.068004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:52:32.791236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787694+0000","last_change":"2024-04-17T09:53:12.776880+0000","last_active":"2024-04-17T09:53:14.787694+0000","last_peered":"2024-04-17T09:53:14.787694+0000","last_clean":"2024-04-17T09:53:14.787694+0000","last_became_active":"2024-04-17T09:53:12.776002+0000","last_became_peered":"2024-04-17T09:53:12.776002+0000","last_unstale":"2024-04-17T09:53:14.787694+0000","last_undegraded":"2024-04-17T09:53:14.787694+0000","last_fullsized":"2024-04-17T09:53:14.787694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:09.944062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066950+0000","last_change":"2024-04-17T09:53:13.793405+0000","last_active":"2024-04-17T09:53:15.066950+0000","last_peered":"2024-04-17T09:53:15.066950+0000","last_clean":"2024-04-17T09:53:15.066950+0000","last_became_active":"2024-04-17T09:53:13.792553+0000","last_became_peered":"2024-04-17T09:53:13.792553+0000","last_unstale":"2024-04-17T09:53:15.066950+0000","last_undegraded":"2024-04-17T09:53:15.066950+0000","last_fullsized":"2024-04-17T09:53:15.066950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:43:23.411109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066128+0000","last_change":"2024-04-17T09:53:12.780453+0000","last_active":"2024-04-17T09:53:15.066128+0000","last_peered":"2024-04-17T09:53:15.066128+0000","last_clean":"2024-04-17T09:53:15.066128+0000","last_became_active":"2024-04-17T09:53:12.780130+0000","last_became_peered":"2024-04-17T09:53:12.780130+0000","last_unstale":"2024-04-17T09:53:15.066128+0000","last_undegraded":"2024-04-17T09:53:15.066128+0000","last_fullsized":"2024-04-17T09:53:15.066128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:14:18.407034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067313+0000","last_change":"2024-04-17T09:53:13.802442+0000","last_active":"2024-04-17T09:53:15.067313+0000","last_peered":"2024-04-17T09:53:15.067313+0000","last_clean":"2024-04-17T09:53:15.067313+0000","last_became_active":"2024-04-17T09:53:13.802168+0000","last_became_peered":"2024-04-17T09:53:13.802168+0000","last_unstale":"2024-04-17T09:53:15.067313+0000","last_undegraded":"2024-04-17T09:53:15.067313+0000","last_fullsized":"2024-04-17T09:53:15.067313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:30:17.695383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066329+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T09:53:15.066329+0000","last_peered":"2024-04-17T09:53:15.066329+0000","last_clean":"2024-04-17T09:53:15.066329+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T09:53:15.066329+0000","last_undegraded":"2024-04-17T09:53:15.066329+0000","last_fullsized":"2024-04-17T09:53:15.066329+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066649+0000","last_change":"2024-04-17T09:53:12.772905+0000","last_active":"2024-04-17T09:53:15.066649+0000","last_peered":"2024-04-17T09:53:15.066649+0000","last_clean":"2024-04-17T09:53:15.066649+0000","last_became_active":"2024-04-17T09:53:12.772635+0000","last_became_peered":"2024-04-17T09:53:12.772635+0000","last_unstale":"2024-04-17T09:53:15.066649+0000","last_undegraded":"2024-04-17T09:53:15.066649+0000","last_fullsized":"2024-04-17T09:53:15.066649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:11.929676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066741+0000","last_change":"2024-04-17T09:53:13.815993+0000","last_active":"2024-04-17T09:53:15.066741+0000","last_peered":"2024-04-17T09:53:15.066741+0000","last_clean":"2024-04-17T09:53:15.066741+0000","last_became_active":"2024-04-17T09:53:13.815689+0000","last_became_peered":"2024-04-17T09:53:13.815689+0000","last_unstale":"2024-04-17T09:53:15.066741+0000","last_undegraded":"2024-04-17T09:53:15.066741+0000","last_fullsized":"2024-04-17T09:53:15.066741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:51:42.871563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066279+0000","last_change":"2024-04-17T09:53:12.790967+0000","last_active":"2024-04-17T09:53:15.066279+0000","last_peered":"2024-04-17T09:53:15.066279+0000","last_clean":"2024-04-17T09:53:15.066279+0000","last_became_active":"2024-04-17T09:53:12.790637+0000","last_became_peered":"2024-04-17T09:53:12.790637+0000","last_unstale":"2024-04-17T09:53:15.066279+0000","last_undegraded":"2024-04-17T09:53:15.066279+0000","last_fullsized":"2024-04-17T09:53:15.066279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:14.881628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.789966+0000","last_change":"2024-04-17T09:53:13.788385+0000","last_active":"2024-04-17T09:53:13.789966+0000","last_peered":"2024-04-17T09:53:13.789966+0000","last_clean":"2024-04-17T09:53:13.789966+0000","last_became_active":"2024-04-17T09:53:13.784112+0000","last_became_peered":"2024-04-17T09:53:13.784112+0000","last_unstale":"2024-04-17T09:53:13.789966+0000","last_undegraded":"2024-04-17T09:53:13.789966+0000","last_fullsized":"2024-04-17T09:53:13.789966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:52.635851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067778+0000","last_change":"2024-04-17T09:53:12.775859+0000","last_active":"2024-04-17T09:53:15.067778+0000","last_peered":"2024-04-17T09:53:15.067778+0000","last_clean":"2024-04-17T09:53:15.067778+0000","last_became_active":"2024-04-17T09:53:12.774051+0000","last_became_peered":"2024-04-17T09:53:12.774051+0000","last_unstale":"2024-04-17T09:53:15.067778+0000","last_undegraded":"2024-04-17T09:53:15.067778+0000","last_fullsized":"2024-04-17T09:53:15.067778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:32:56.010181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787431+0000","last_change":"2024-04-17T09:53:13.792329+0000","last_active":"2024-04-17T09:53:14.787431+0000","last_peered":"2024-04-17T09:53:14.787431+0000","last_clean":"2024-04-17T09:53:14.787431+0000","last_became_active":"2024-04-17T09:53:13.790615+0000","last_became_peered":"2024-04-17T09:53:13.790615+0000","last_unstale":"2024-04-17T09:53:14.787431+0000","last_undegraded":"2024-04-17T09:53:14.787431+0000","last_fullsized":"2024-04-17T09:53:14.787431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:02.320431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067057+0000","last_change":"2024-04-17T09:53:12.787408+0000","last_active":"2024-04-17T09:53:15.067057+0000","last_peered":"2024-04-17T09:53:15.067057+0000","last_clean":"2024-04-17T09:53:15.067057+0000","last_became_active":"2024-04-17T09:53:12.786322+0000","last_became_peered":"2024-04-17T09:53:12.786322+0000","last_unstale":"2024-04-17T09:53:15.067057+0000","last_undegraded":"2024-04-17T09:53:15.067057+0000","last_fullsized":"2024-04-17T09:53:15.067057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:14.881628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.806643+0000","last_change":"2024-04-17T09:53:13.786023+0000","last_active":"2024-04-17T09:53:13.806643+0000","last_peered":"2024-04-17T09:53:13.806643+0000","last_clean":"2024-04-17T09:53:13.806643+0000","last_became_active":"2024-04-17T09:53:13.785528+0000","last_became_peered":"2024-04-17T09:53:13.785528+0000","last_unstale":"2024-04-17T09:53:13.806643+0000","last_undegraded":"2024-04-17T09:53:13.806643+0000","last_fullsized":"2024-04-17T09:53:13.806643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:19:29.224408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787601+0000","last_change":"2024-04-17T09:53:12.786776+0000","last_active":"2024-04-17T09:53:14.787601+0000","last_peered":"2024-04-17T09:53:14.787601+0000","last_clean":"2024-04-17T09:53:14.787601+0000","last_became_active":"2024-04-17T09:53:12.786511+0000","last_became_peered":"2024-04-17T09:53:12.786511+0000","last_unstale":"2024-04-17T09:53:14.787601+0000","last_undegraded":"2024-04-17T09:53:14.787601+0000","last_fullsized":"2024-04-17T09:53:14.787601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:32:56.010181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067174+0000","last_change":"2024-04-17T09:53:13.795421+0000","last_active":"2024-04-17T09:53:15.067174+0000","last_peered":"2024-04-17T09:53:15.067174+0000","last_clean":"2024-04-17T09:53:15.067174+0000","last_became_active":"2024-04-17T09:53:13.793404+0000","last_became_peered":"2024-04-17T09:53:13.793404+0000","last_unstale":"2024-04-17T09:53:15.067174+0000","last_undegraded":"2024-04-17T09:53:15.067174+0000","last_fullsized":"2024-04-17T09:53:15.067174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:27:15.142091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067608+0000","last_change":"2024-04-17T09:53:12.786549+0000","last_active":"2024-04-17T09:53:15.067608+0000","last_peered":"2024-04-17T09:53:15.067608+0000","last_clean":"2024-04-17T09:53:15.067608+0000","last_became_active":"2024-04-17T09:53:12.784745+0000","last_became_peered":"2024-04-17T09:53:12.784745+0000","last_unstale":"2024-04-17T09:53:15.067608+0000","last_undegraded":"2024-04-17T09:53:15.067608+0000","last_fullsized":"2024-04-17T09:53:15.067608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:57:23.703708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787659+0000","last_change":"2024-04-17T09:53:13.792711+0000","last_active":"2024-04-17T09:53:14.787659+0000","last_peered":"2024-04-17T09:53:14.787659+0000","last_clean":"2024-04-17T09:53:14.787659+0000","last_became_active":"2024-04-17T09:53:13.791107+0000","last_became_peered":"2024-04-17T09:53:13.791107+0000","last_unstale":"2024-04-17T09:53:14.787659+0000","last_undegraded":"2024-04-17T09:53:14.787659+0000","last_fullsized":"2024-04-17T09:53:14.787659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:17.663523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.795170+0000","last_change":"2024-04-17T09:53:12.770038+0000","last_active":"2024-04-17T09:53:14.795170+0000","last_peered":"2024-04-17T09:53:14.795170+0000","last_clean":"2024-04-17T09:53:14.795170+0000","last_became_active":"2024-04-17T09:53:12.769750+0000","last_became_peered":"2024-04-17T09:53:12.769750+0000","last_unstale":"2024-04-17T09:53:14.795170+0000","last_undegraded":"2024-04-17T09:53:14.795170+0000","last_fullsized":"2024-04-17T09:53:14.795170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:50.549367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067988+0000","last_change":"2024-04-17T09:53:13.801993+0000","last_active":"2024-04-17T09:53:15.067988+0000","last_peered":"2024-04-17T09:53:15.067988+0000","last_clean":"2024-04-17T09:53:15.067988+0000","last_became_active":"2024-04-17T09:53:13.801328+0000","last_became_peered":"2024-04-17T09:53:13.801328+0000","last_unstale":"2024-04-17T09:53:15.067988+0000","last_undegraded":"2024-04-17T09:53:15.067988+0000","last_fullsized":"2024-04-17T09:53:15.067988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:03:55.116831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.789207+0000","last_change":"2024-04-17T09:53:12.781559+0000","last_active":"2024-04-17T09:53:14.789207+0000","last_peered":"2024-04-17T09:53:14.789207+0000","last_clean":"2024-04-17T09:53:14.789207+0000","last_became_active":"2024-04-17T09:53:12.780702+0000","last_became_peered":"2024-04-17T09:53:12.780702+0000","last_unstale":"2024-04-17T09:53:14.789207+0000","last_undegraded":"2024-04-17T09:53:14.789207+0000","last_fullsized":"2024-04-17T09:53:14.789207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:09.944062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.797479+0000","last_change":"2024-04-17T09:53:13.795015+0000","last_active":"2024-04-17T09:53:13.797479+0000","last_peered":"2024-04-17T09:53:13.797479+0000","last_clean":"2024-04-17T09:53:13.797479+0000","last_became_active":"2024-04-17T09:53:13.794609+0000","last_became_peered":"2024-04-17T09:53:13.794609+0000","last_unstale":"2024-04-17T09:53:13.797479+0000","last_undegraded":"2024-04-17T09:53:13.797479+0000","last_fullsized":"2024-04-17T09:53:13.797479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:09:58.265496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066266+0000","last_change":"2024-04-17T09:53:12.781463+0000","last_active":"2024-04-17T09:53:15.066266+0000","last_peered":"2024-04-17T09:53:15.066266+0000","last_clean":"2024-04-17T09:53:15.066266+0000","last_became_active":"2024-04-17T09:53:12.781138+0000","last_became_peered":"2024-04-17T09:53:12.781138+0000","last_unstale":"2024-04-17T09:53:15.066266+0000","last_undegraded":"2024-04-17T09:53:15.066266+0000","last_fullsized":"2024-04-17T09:53:15.066266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:09.944062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.810444+0000","last_change":"2024-04-17T09:53:13.797478+0000","last_active":"2024-04-17T09:53:13.810444+0000","last_peered":"2024-04-17T09:53:13.810444+0000","last_clean":"2024-04-17T09:53:13.810444+0000","last_became_active":"2024-04-17T09:53:13.797152+0000","last_became_peered":"2024-04-17T09:53:13.797152+0000","last_unstale":"2024-04-17T09:53:13.810444+0000","last_undegraded":"2024-04-17T09:53:13.810444+0000","last_fullsized":"2024-04-17T09:53:13.810444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:16.046896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787486+0000","last_change":"2024-04-17T09:53:12.785848+0000","last_active":"2024-04-17T09:53:14.787486+0000","last_peered":"2024-04-17T09:53:14.787486+0000","last_clean":"2024-04-17T09:53:14.787486+0000","last_became_active":"2024-04-17T09:53:12.784980+0000","last_became_peered":"2024-04-17T09:53:12.784980+0000","last_unstale":"2024-04-17T09:53:14.787486+0000","last_undegraded":"2024-04-17T09:53:14.787486+0000","last_fullsized":"2024-04-17T09:53:14.787486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:11.929676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066406+0000","last_change":"2024-04-17T09:53:13.796607+0000","last_active":"2024-04-17T09:53:15.066406+0000","last_peered":"2024-04-17T09:53:15.066406+0000","last_clean":"2024-04-17T09:53:15.066406+0000","last_became_active":"2024-04-17T09:53:13.796351+0000","last_became_peered":"2024-04-17T09:53:13.796351+0000","last_unstale":"2024-04-17T09:53:15.066406+0000","last_undegraded":"2024-04-17T09:53:15.066406+0000","last_fullsized":"2024-04-17T09:53:15.066406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:17:54.590392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787463+0000","last_change":"2024-04-17T09:53:12.788441+0000","last_active":"2024-04-17T09:53:14.787463+0000","last_peered":"2024-04-17T09:53:14.787463+0000","last_clean":"2024-04-17T09:53:14.787463+0000","last_became_active":"2024-04-17T09:53:12.787431+0000","last_became_peered":"2024-04-17T09:53:12.787431+0000","last_unstale":"2024-04-17T09:53:14.787463+0000","last_undegraded":"2024-04-17T09:53:14.787463+0000","last_fullsized":"2024-04-17T09:53:14.787463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:14:18.407034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066772+0000","last_change":"2024-04-17T09:53:13.783282+0000","last_active":"2024-04-17T09:53:15.066772+0000","last_peered":"2024-04-17T09:53:15.066772+0000","last_clean":"2024-04-17T09:53:15.066772+0000","last_became_active":"2024-04-17T09:53:13.782978+0000","last_became_peered":"2024-04-17T09:53:13.782978+0000","last_unstale":"2024-04-17T09:53:15.066772+0000","last_undegraded":"2024-04-17T09:53:15.066772+0000","last_fullsized":"2024-04-17T09:53:15.066772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:55:01.176803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067538+0000","last_change":"2024-04-17T09:53:12.785555+0000","last_active":"2024-04-17T09:53:15.067538+0000","last_peered":"2024-04-17T09:53:15.067538+0000","last_clean":"2024-04-17T09:53:15.067538+0000","last_became_active":"2024-04-17T09:53:12.785233+0000","last_became_peered":"2024-04-17T09:53:12.785233+0000","last_unstale":"2024-04-17T09:53:15.067538+0000","last_undegraded":"2024-04-17T09:53:15.067538+0000","last_fullsized":"2024-04-17T09:53:15.067538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:50.549367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788225+0000","last_change":"2024-04-17T09:53:13.800230+0000","last_active":"2024-04-17T09:53:14.788225+0000","last_peered":"2024-04-17T09:53:14.788225+0000","last_clean":"2024-04-17T09:53:14.788225+0000","last_became_active":"2024-04-17T09:53:13.799917+0000","last_became_peered":"2024-04-17T09:53:13.799917+0000","last_unstale":"2024-04-17T09:53:14.788225+0000","last_undegraded":"2024-04-17T09:53:14.788225+0000","last_fullsized":"2024-04-17T09:53:14.788225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:43:18.222168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067005+0000","last_change":"2024-04-17T09:53:13.814147+0000","last_active":"2024-04-17T09:53:15.067005+0000","last_peered":"2024-04-17T09:53:15.067005+0000","last_clean":"2024-04-17T09:53:15.067005+0000","last_became_active":"2024-04-17T09:53:13.813807+0000","last_became_peered":"2024-04-17T09:53:13.813807+0000","last_unstale":"2024-04-17T09:53:15.067005+0000","last_undegraded":"2024-04-17T09:53:15.067005+0000","last_fullsized":"2024-04-17T09:53:15.067005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:33:32.665364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067456+0000","last_change":"2024-04-17T09:53:12.788509+0000","last_active":"2024-04-17T09:53:15.067456+0000","last_peered":"2024-04-17T09:53:15.067456+0000","last_clean":"2024-04-17T09:53:15.067456+0000","last_became_active":"2024-04-17T09:53:12.788161+0000","last_became_peered":"2024-04-17T09:53:12.788161+0000","last_unstale":"2024-04-17T09:53:15.067456+0000","last_undegraded":"2024-04-17T09:53:15.067456+0000","last_fullsized":"2024-04-17T09:53:15.067456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:11.929676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066835+0000","last_change":"2024-04-17T09:53:13.812181+0000","last_active":"2024-04-17T09:53:15.066835+0000","last_peered":"2024-04-17T09:53:15.066835+0000","last_clean":"2024-04-17T09:53:15.066835+0000","last_became_active":"2024-04-17T09:53:13.811836+0000","last_became_peered":"2024-04-17T09:53:13.811836+0000","last_unstale":"2024-04-17T09:53:15.066835+0000","last_undegraded":"2024-04-17T09:53:15.066835+0000","last_fullsized":"2024-04-17T09:53:15.066835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:03:55.116831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067392+0000","last_change":"2024-04-17T09:53:12.786015+0000","last_active":"2024-04-17T09:53:15.067392+0000","last_peered":"2024-04-17T09:53:15.067392+0000","last_clean":"2024-04-17T09:53:15.067392+0000","last_became_active":"2024-04-17T09:53:12.784577+0000","last_became_peered":"2024-04-17T09:53:12.784577+0000","last_unstale":"2024-04-17T09:53:15.067392+0000","last_undegraded":"2024-04-17T09:53:15.067392+0000","last_fullsized":"2024-04-17T09:53:15.067392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:53.140146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.821119+0000","last_change":"2024-04-17T09:53:13.819959+0000","last_active":"2024-04-17T09:53:13.821119+0000","last_peered":"2024-04-17T09:53:13.821119+0000","last_clean":"2024-04-17T09:53:13.821119+0000","last_became_active":"2024-04-17T09:53:13.819509+0000","last_became_peered":"2024-04-17T09:53:13.819509+0000","last_unstale":"2024-04-17T09:53:13.821119+0000","last_undegraded":"2024-04-17T09:53:13.821119+0000","last_fullsized":"2024-04-17T09:53:13.821119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:31:53.376511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067396+0000","last_change":"2024-04-17T09:53:12.784607+0000","last_active":"2024-04-17T09:53:15.067396+0000","last_peered":"2024-04-17T09:53:15.067396+0000","last_clean":"2024-04-17T09:53:15.067396+0000","last_became_active":"2024-04-17T09:53:12.784265+0000","last_became_peered":"2024-04-17T09:53:12.784265+0000","last_unstale":"2024-04-17T09:53:15.067396+0000","last_undegraded":"2024-04-17T09:53:15.067396+0000","last_fullsized":"2024-04-17T09:53:15.067396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:07.296217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067248+0000","last_change":"2024-04-17T09:53:13.795050+0000","last_active":"2024-04-17T09:53:15.067248+0000","last_peered":"2024-04-17T09:53:15.067248+0000","last_clean":"2024-04-17T09:53:15.067248+0000","last_became_active":"2024-04-17T09:53:13.794002+0000","last_became_peered":"2024-04-17T09:53:13.794002+0000","last_unstale":"2024-04-17T09:53:15.067248+0000","last_undegraded":"2024-04-17T09:53:15.067248+0000","last_fullsized":"2024-04-17T09:53:15.067248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:16:29.674656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T09:53:13.817942+0000","last_change":"2024-04-17T09:53:12.782069+0000","last_active":"2024-04-17T09:53:13.817942+0000","last_peered":"2024-04-17T09:53:13.817942+0000","last_clean":"2024-04-17T09:53:13.817942+0000","last_became_active":"2024-04-17T09:53:12.781795+0000","last_became_peered":"2024-04-17T09:53:12.781795+0000","last_unstale":"2024-04-17T09:53:13.817942+0000","last_undegraded":"2024-04-17T09:53:13.817942+0000","last_fullsized":"2024-04-17T09:53:13.817942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:48.300937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":794624,"data_stored":656837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":200704,"data_stored":65977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":208896,"data_stored":67499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":196608,"data_stored":65887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":72347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":208896,"data_stored":66013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":778240,"data_stored":649480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T09:53:18.296 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T09:53:18.296 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-17T09:53:18.296 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-17T09:53:18.297 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-17T09:53:18.297 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-17T09:53:18.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.475+0000 7f716cc00640 1 Processor -- start 2024-04-17T09:53:18.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.475+0000 7f716cc00640 1 -- start start 2024-04-17T09:53:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.475+0000 7f716cc00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f716815eb10 0x7f716815f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.475+0000 7f716cc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71680d0f20 0x7f71681603c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.475+0000 7f716cc00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71680b2ac0 con 0x7f71680b2550 2024-04-17T09:53:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.475+0000 7f716cc00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f716815ff10 con 0x7f716815eb10 2024-04-17T09:53:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.475+0000 7f716cc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f71680b2da0 con 0x7f71680d0f20 2024-04-17T09:53:18.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.479+0000 7f7166800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71680d0f20 0x7f71681603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:18.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.479+0000 7f7165e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f716815eb10 0x7f716815f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:18.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.479+0000 7f7165e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f716815eb10 0x7f716815f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:40722/0 (socket says 172.21.15.32:40722) 2024-04-17T09:53:18.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.479+0000 7f7166800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71680d0f20 0x7f71681603c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:56304/0 (socket says 172.21.15.32:56304) 2024-04-17T09:53:18.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.479+0000 7f7165e00640 1 -- 172.21.15.32:0/3859232661 learned_addr learned my addr 172.21.15.32:0/3859232661 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:18.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.479+0000 7f715fe00640 1 --1- 172.21.15.32:0/3859232661 >> v1:172.21.15.144:6789/0 conn(0x7f71680b2550 0x7f71680d0810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:34400/0 (socket says 172.21.15.32:34400) 2024-04-17T09:53:18.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.479+0000 7f7165400640 1 -- 172.21.15.32:0/3859232661 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1184061101 0 0) 0x7f71680b2ac0 con 0x7f71680b2550 2024-04-17T09:53:18.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.479+0000 7f7165e00640 1 -- 172.21.15.32:0/3859232661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71680d0f20 msgr2=0x7f71681603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:18.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.479+0000 7f7165e00640 1 --2- 172.21.15.32:0/3859232661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71680d0f20 0x7f71681603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.479+0000 7f7165e00640 1 -- 172.21.15.32:0/3859232661 >> v1:172.21.15.144:6789/0 conn(0x7f71680b2550 legacy=0x7f71680d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:18.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.479+0000 7f7165e00640 1 -- 172.21.15.32:0/3859232661 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716806bb60 con 0x7f716815eb10 2024-04-17T09:53:18.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.479+0000 7f7166800640 1 --2- 172.21.15.32:0/3859232661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71680d0f20 0x7f71681603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:53:18.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.479+0000 7f7165e00640 1 --2- 172.21.15.32:0/3859232661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f716815eb10 0x7f716815f9d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7158009870 tx=0x7f715801c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eede8c60ee4f963 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:18.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f7165400640 1 -- 172.21.15.32:0/3859232661 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71580b1070 con 0x7f716815eb10 2024-04-17T09:53:18.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f7165400640 1 -- 172.21.15.32:0/3859232661 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71580a8050 con 0x7f716815eb10 2024-04-17T09:53:18.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f7165400640 1 -- 172.21.15.32:0/3859232661 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71580ac860 con 0x7f716815eb10 2024-04-17T09:53:18.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f716cc00640 1 -- 172.21.15.32:0/3859232661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f716815eb10 msgr2=0x7f716815f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:18.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f716cc00640 1 --2- 172.21.15.32:0/3859232661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f716815eb10 0x7f716815f9d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7158009870 tx=0x7f715801c4d0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f716cc00640 1 -- 172.21.15.32:0/3859232661 shutdown_connections 2024-04-17T09:53:18.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f716cc00640 1 --2- 172.21.15.32:0/3859232661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71680d0f20 0x7f71681603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f716cc00640 1 --2- 172.21.15.32:0/3859232661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f716815eb10 0x7f716815f9d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f716cc00640 1 -- 172.21.15.32:0/3859232661 >> 172.21.15.32:0/3859232661 conn(0x7f716809a9f0 msgr2=0x7f71680b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:18.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f716cc00640 1 -- 172.21.15.32:0/3859232661 shutdown_connections 2024-04-17T09:53:18.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f716cc00640 1 -- 172.21.15.32:0/3859232661 wait complete. 2024-04-17T09:53:18.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f716cc00640 1 Processor -- start 2024-04-17T09:53:18.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f716cc00640 1 -- start start 2024-04-17T09:53:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f716cc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71680b2550 0x7f71681f8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f716cc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71680d0f20 0x7f716809c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f716cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f716815eb10 0x7f7168158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f716cc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f716816bbf0 con 0x7f716815eb10 2024-04-17T09:53:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f716cc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f716816ba70 con 0x7f71680d0f20 2024-04-17T09:53:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f716cc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f716816bd70 con 0x7f71680b2550 2024-04-17T09:53:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f7166800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f716815eb10 0x7f7168158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f7166800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f716815eb10 0x7f7168158320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46798/0 (socket says 172.21.15.32:46798) 2024-04-17T09:53:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f7166800640 1 -- 172.21.15.32:0/1946523438 learned_addr learned my addr 172.21.15.32:0/1946523438 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f715fe00640 1 --2- 172.21.15.32:0/1946523438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71680d0f20 0x7f716809c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f7165e00640 1 --2- 172.21.15.32:0/1946523438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71680b2550 0x7f71681f8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f7166800640 1 -- 172.21.15.32:0/1946523438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71680b2550 msgr2=0x7f71681f8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f7166800640 1 --2- 172.21.15.32:0/1946523438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71680b2550 0x7f71681f8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.483+0000 7f7166800640 1 -- 172.21.15.32:0/1946523438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71680d0f20 msgr2=0x7f716809c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.487+0000 7f7166800640 1 --2- 172.21.15.32:0/1946523438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71680d0f20 0x7f716809c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.487+0000 7f7166800640 1 -- 172.21.15.32:0/1946523438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71680b7280 con 0x7f716815eb10 2024-04-17T09:53:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.487+0000 7f7166800640 1 --2- 172.21.15.32:0/1946523438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f716815eb10 0x7f7168158320 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f716000efc0 tx=0x7f7160009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.487+0000 7f715e000640 1 -- 172.21.15.32:0/1946523438 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160018030 con 0x7f716815eb10 2024-04-17T09:53:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.487+0000 7f715e000640 1 -- 172.21.15.32:0/1946523438 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f716000cde0 con 0x7f716815eb10 2024-04-17T09:53:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.487+0000 7f716cc00640 1 -- 172.21.15.32:0/1946523438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71680d2310 con 0x7f716815eb10 2024-04-17T09:53:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.487+0000 7f715e000640 1 -- 172.21.15.32:0/1946523438 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160003bc0 con 0x7f716815eb10 2024-04-17T09:53:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.487+0000 7f716cc00640 1 -- 172.21.15.32:0/1946523438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f716815ac00 con 0x7f716815eb10 2024-04-17T09:53:18.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.487+0000 7f716cc00640 1 -- 172.21.15.32:0/1946523438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7168068a90 con 0x7f716815eb10 2024-04-17T09:53:18.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.487+0000 7f715e000640 1 -- 172.21.15.32:0/1946523438 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f716001b070 con 0x7f716815eb10 2024-04-17T09:53:18.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.487+0000 7f715e000640 1 --2- 172.21.15.32:0/1946523438 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7144068380 0x7f714406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:18.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.487+0000 7f715e000640 1 -- 172.21.15.32:0/1946523438 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7160087490 con 0x7f716815eb10 2024-04-17T09:53:18.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.487+0000 7f7165e00640 1 --2- 172.21.15.32:0/1946523438 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7144068380 0x7f714406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:18.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.491+0000 7f7165e00640 1 --2- 172.21.15.32:0/1946523438 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7144068380 0x7f714406a840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f71580ae000 tx=0x7f71580a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:18.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.491+0000 7f715e000640 1 -- 172.21.15.32:0/1946523438 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f7160022070 con 0x7f716815eb10 2024-04-17T09:53:18.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.695+0000 7f716cc00640 1 -- 172.21.15.32:0/1946523438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f71680cecf0 con 0x7f716815eb10 2024-04-17T09:53:18.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.699+0000 7f715e000640 1 -- 172.21.15.32:0/1946523438 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+168081 (secure 0 0 0) 0x7f7160010040 con 0x7f716815eb10 2024-04-17T09:53:18.704 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:18.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.703+0000 7f716cc00640 1 -- 172.21.15.32:0/1946523438 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7144068380 msgr2=0x7f714406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:18.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.703+0000 7f716cc00640 1 --2- 172.21.15.32:0/1946523438 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7144068380 0x7f714406a840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f71580ae000 tx=0x7f71580a7000 comp rx=0 tx=0).stop 2024-04-17T09:53:18.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.703+0000 7f716cc00640 1 -- 172.21.15.32:0/1946523438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f716815eb10 msgr2=0x7f7168158320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:18.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.703+0000 7f716cc00640 1 --2- 172.21.15.32:0/1946523438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f716815eb10 0x7f7168158320 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f716000efc0 tx=0x7f7160009f90 comp rx=0 tx=0).stop 2024-04-17T09:53:18.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.703+0000 7f716cc00640 1 -- 172.21.15.32:0/1946523438 shutdown_connections 2024-04-17T09:53:18.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.703+0000 7f716cc00640 1 --2- 172.21.15.32:0/1946523438 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7144068380 0x7f714406a840 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.703+0000 7f716cc00640 1 --2- 172.21.15.32:0/1946523438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f716815eb10 0x7f7168158320 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.703+0000 7f716cc00640 1 --2- 172.21.15.32:0/1946523438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71680d0f20 0x7f716809c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.703+0000 7f716cc00640 1 --2- 172.21.15.32:0/1946523438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71680b2550 0x7f71681f8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.703+0000 7f716cc00640 1 -- 172.21.15.32:0/1946523438 >> 172.21.15.32:0/1946523438 conn(0x7f716809a9f0 msgr2=0x7f71680d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:18.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.703+0000 7f716cc00640 1 -- 172.21.15.32:0/1946523438 shutdown_connections 2024-04-17T09:53:18.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.703+0000 7f716cc00640 1 -- 172.21.15.32:0/1946523438 wait complete. 2024-04-17T09:53:18.724 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"flags":0,"active_gid":4106,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":1154261516},{"type":"v1","addr":"172.21.15.32:6833","nonce":1154261516}]},"active_addr":"172.21.15.32:6833/1154261516","active_change":"2024-04-17T09:52:53.271725+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4103,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":3104263988}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":84681559}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":4170153525}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":21374092}]}]} 2024-04-17T09:53:18.727 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-17T09:53:18.727 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-17T09:53:18.727 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:53:18.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.863+0000 7f49baa00640 1 Processor -- start 2024-04-17T09:53:18.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.863+0000 7f49baa00640 1 -- start start 2024-04-17T09:53:18.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49baa00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f49b4151db0 0x7f49b40b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:18.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49baa00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49b40b3490 con 0x7f49b406f630 2024-04-17T09:53:18.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49baa00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49b40cea40 con 0x7f49b4072d90 2024-04-17T09:53:18.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49baa00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f49b40b44f0 con 0x7f49b4151db0 2024-04-17T09:53:18.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49ab400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f49b406f630 0x7f49b4072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39178/0 (socket says 172.21.15.32:39178) 2024-04-17T09:53:18.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49ab400640 1 -- 172.21.15.32:0/3054518812 learned_addr learned my addr 172.21.15.32:0/3054518812 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:18.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49b3e00640 1 --2- 172.21.15.32:0/3054518812 >> v2:172.21.15.144:3300/0 conn(0x7f49b4151db0 0x7f49b40b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:18.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49b3400640 1 -- 172.21.15.32:0/3054518812 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1796721208 0 0) 0x7f49b40b3490 con 0x7f49b406f630 2024-04-17T09:53:18.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49b3400640 1 -- 172.21.15.32:0/3054518812 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4998003530 con 0x7f49b406f630 2024-04-17T09:53:18.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49b3400640 1 -- 172.21.15.32:0/3054518812 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2031121218 0 0) 0x7f4998003530 con 0x7f49b406f630 2024-04-17T09:53:18.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49b3400640 1 -- 172.21.15.32:0/3054518812 >> v2:172.21.15.144:3300/0 conn(0x7f49b4151db0 msgr2=0x7f49b40b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:18.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49b3400640 1 --2- 172.21.15.32:0/3054518812 >> v2:172.21.15.144:3300/0 conn(0x7f49b4151db0 0x7f49b40b51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49b3400640 1 -- 172.21.15.32:0/3054518812 >> v1:172.21.15.144:6789/0 conn(0x7f49b4072d90 legacy=0x7f49b406de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:18.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49b3400640 1 -- 172.21.15.32:0/3054518812 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b40b3730 con 0x7f49b406f630 2024-04-17T09:53:18.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49b3400640 1 -- 172.21.15.32:0/3054518812 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f499c0016a0 con 0x7f49b406f630 2024-04-17T09:53:18.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49b3400640 1 -- 172.21.15.32:0/3054518812 >> v1:172.21.15.32:6789/0 conn(0x7f49b406f630 legacy=0x7f49b4072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:18.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49b3400640 1 --2- 172.21.15.32:0/3054518812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49980039c0 0x7f4998003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:18.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49b3400640 1 --2- 172.21.15.32:0/3054518812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49980043c0 0x7f4998004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:18.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49b3400640 1 --2- 172.21.15.32:0/3054518812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4998005090 0x7f499800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:18.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49b3400640 1 -- 172.21.15.32:0/3054518812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f49b40b44f0 con 0x7f4998005090 2024-04-17T09:53:18.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49b3400640 1 -- 172.21.15.32:0/3054518812 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f499800bff0 con 0x7f49980039c0 2024-04-17T09:53:18.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49b3400640 1 -- 172.21.15.32:0/3054518812 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f499800c1f0 con 0x7f49980043c0 2024-04-17T09:53:18.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49b8a00640 1 --2- 172.21.15.32:0/3054518812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4998005090 0x7f499800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:18.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49ab400640 1 --2- 172.21.15.32:0/3054518812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49980043c0 0x7f4998004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:18.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49b3e00640 1 --2- 172.21.15.32:0/3054518812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49980039c0 0x7f4998003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:18.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49b8a00640 1 -- 172.21.15.32:0/3054518812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49980043c0 msgr2=0x7f4998004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:18.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49b8a00640 1 --2- 172.21.15.32:0/3054518812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49980043c0 0x7f4998004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49b8a00640 1 -- 172.21.15.32:0/3054518812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49980039c0 msgr2=0x7f4998003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:18.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49b8a00640 1 --2- 172.21.15.32:0/3054518812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49980039c0 0x7f4998003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.867+0000 7f49b8a00640 1 -- 172.21.15.32:0/3054518812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4998003530 con 0x7f4998005090 2024-04-17T09:53:18.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.871+0000 7f49b8a00640 1 -- 172.21.15.32:0/3054518812 reap_dead start 2024-04-17T09:53:18.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.871+0000 7f49b8a00640 1 --2- 172.21.15.32:0/3054518812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4998005090 0x7f499800b9d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f49ac001f60 tx=0x7f49ac01c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70588d7b4a00d3af server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:18.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.871+0000 7f49b3400640 1 -- 172.21.15.32:0/3054518812 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac001b60 con 0x7f4998005090 2024-04-17T09:53:18.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.871+0000 7f49b3400640 1 -- 172.21.15.32:0/3054518812 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49ac003b80 con 0x7f4998005090 2024-04-17T09:53:18.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.871+0000 7f49b3400640 1 -- 172.21.15.32:0/3054518812 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac0050d0 con 0x7f4998005090 2024-04-17T09:53:18.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.871+0000 7f49baa00640 1 -- 172.21.15.32:0/3054518812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4998005090 msgr2=0x7f499800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:18.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.871+0000 7f49baa00640 1 --2- 172.21.15.32:0/3054518812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4998005090 0x7f499800b9d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f49ac001f60 tx=0x7f49ac01c880 comp rx=0 tx=0).stop 2024-04-17T09:53:18.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.871+0000 7f49baa00640 1 -- 172.21.15.32:0/3054518812 shutdown_connections 2024-04-17T09:53:18.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.871+0000 7f49baa00640 1 --2- 172.21.15.32:0/3054518812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4998005090 0x7f499800b9d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.871+0000 7f49baa00640 1 -- 172.21.15.32:0/3054518812 >> 172.21.15.32:0/3054518812 conn(0x7f49b406ba30 msgr2=0x7f49b406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:18.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.871+0000 7f49baa00640 1 -- 172.21.15.32:0/3054518812 shutdown_connections 2024-04-17T09:53:18.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.871+0000 7f49baa00640 1 -- 172.21.15.32:0/3054518812 wait complete. 2024-04-17T09:53:18.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.871+0000 7f49baa00640 1 Processor -- start 2024-04-17T09:53:18.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.871+0000 7f49baa00640 1 -- start start 2024-04-17T09:53:18.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.875+0000 7f49baa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4998005090 0x7f49b41375d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:18.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.875+0000 7f49baa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49b413acf0 0x7f49b415d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:18.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.875+0000 7f49baa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49b413b170 0x7f49b415d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:18.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.875+0000 7f49baa00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f49b40b3fd0 con 0x7f49b413acf0 2024-04-17T09:53:18.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.875+0000 7f49baa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f49b40b3cf0 con 0x7f49b413b170 2024-04-17T09:53:18.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.875+0000 7f49baa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f49b40b51c0 con 0x7f4998005090 2024-04-17T09:53:18.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.875+0000 7f49ab400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49b413acf0 0x7f49b415d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:18.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.875+0000 7f49ab400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49b413acf0 0x7f49b415d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46814/0 (socket says 172.21.15.32:46814) 2024-04-17T09:53:18.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.875+0000 7f49ab400640 1 -- 172.21.15.32:0/1772176875 learned_addr learned my addr 172.21.15.32:0/1772176875 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:18.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.875+0000 7f49b3e00640 1 --2- 172.21.15.32:0/1772176875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4998005090 0x7f49b41375d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:18.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.875+0000 7f49b8a00640 1 --2- 172.21.15.32:0/1772176875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49b413b170 0x7f49b415d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:18.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.875+0000 7f49ab400640 1 -- 172.21.15.32:0/1772176875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4998005090 msgr2=0x7f49b41375d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:18.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.875+0000 7f49ab400640 1 --2- 172.21.15.32:0/1772176875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4998005090 0x7f49b41375d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.875+0000 7f49ab400640 1 -- 172.21.15.32:0/1772176875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49b413b170 msgr2=0x7f49b415d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:18.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.875+0000 7f49ab400640 1 --2- 172.21.15.32:0/1772176875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49b413b170 0x7f49b415d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:18.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.875+0000 7f49ab400640 1 -- 172.21.15.32:0/1772176875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b40b4670 con 0x7f49b413acf0 2024-04-17T09:53:18.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.875+0000 7f49ab400640 1 --2- 172.21.15.32:0/1772176875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49b413acf0 0x7f49b415d450 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f499c001ca0 tx=0x7f499c012630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:18.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.875+0000 7f49b0c00640 1 -- 172.21.15.32:0/1772176875 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c01c070 con 0x7f49b413acf0 2024-04-17T09:53:18.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.875+0000 7f49b0c00640 1 -- 172.21.15.32:0/1772176875 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f499c014750 con 0x7f49b413acf0 2024-04-17T09:53:18.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.875+0000 7f49b0c00640 1 -- 172.21.15.32:0/1772176875 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c014b40 con 0x7f49b413acf0 2024-04-17T09:53:18.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.875+0000 7f49baa00640 1 -- 172.21.15.32:0/1772176875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b4137b10 con 0x7f49b413acf0 2024-04-17T09:53:18.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.875+0000 7f49baa00640 1 -- 172.21.15.32:0/1772176875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b4136810 con 0x7f49b413acf0 2024-04-17T09:53:18.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.875+0000 7f49baa00640 1 -- 172.21.15.32:0/1772176875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49b4068a90 con 0x7f49b413acf0 2024-04-17T09:53:18.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.879+0000 7f49b0c00640 1 -- 172.21.15.32:0/1772176875 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f499c01f070 con 0x7f49b413acf0 2024-04-17T09:53:18.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.879+0000 7f49b0c00640 1 --2- 172.21.15.32:0/1772176875 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4990068380 0x7f499006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:18.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.879+0000 7f49b0c00640 1 -- 172.21.15.32:0/1772176875 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f499c089170 con 0x7f49b413acf0 2024-04-17T09:53:18.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.879+0000 7f49b3e00640 1 --2- 172.21.15.32:0/1772176875 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4990068380 0x7f499006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:18.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.879+0000 7f49b3e00640 1 --2- 172.21.15.32:0/1772176875 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4990068380 0x7f499006a840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f49a4009b60 tx=0x7f49a4009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:18.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:18.883+0000 7f49b0c00640 1 -- 172.21.15.32:0/1772176875 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f499c012e60 con 0x7f49b413acf0 2024-04-17T09:53:19.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.047+0000 7f49baa00640 1 -- 172.21.15.32:0/1772176875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49b40cea40 con 0x7f49b413acf0 2024-04-17T09:53:19.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.047+0000 7f49b0c00640 1 -- 172.21.15.32:0/1772176875 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f499c0590d0 con 0x7f49b413acf0 2024-04-17T09:53:19.051 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:19.051 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:14.779535+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:53:19.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.047+0000 7f49baa00640 1 -- 172.21.15.32:0/1772176875 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4990068380 msgr2=0x7f499006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.047+0000 7f49baa00640 1 --2- 172.21.15.32:0/1772176875 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4990068380 0x7f499006a840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f49a4009b60 tx=0x7f49a4009190 comp rx=0 tx=0).stop 2024-04-17T09:53:19.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.047+0000 7f49baa00640 1 -- 172.21.15.32:0/1772176875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49b413acf0 msgr2=0x7f49b415d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.047+0000 7f49baa00640 1 --2- 172.21.15.32:0/1772176875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49b413acf0 0x7f49b415d450 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f499c001ca0 tx=0x7f499c012630 comp rx=0 tx=0).stop 2024-04-17T09:53:19.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.051+0000 7f49baa00640 1 -- 172.21.15.32:0/1772176875 shutdown_connections 2024-04-17T09:53:19.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.051+0000 7f49baa00640 1 --2- 172.21.15.32:0/1772176875 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4990068380 0x7f499006a840 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.051+0000 7f49baa00640 1 --2- 172.21.15.32:0/1772176875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49b413b170 0x7f49b415d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.051+0000 7f49baa00640 1 --2- 172.21.15.32:0/1772176875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49b413acf0 0x7f49b415d450 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.051+0000 7f49baa00640 1 --2- 172.21.15.32:0/1772176875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4998005090 0x7f49b41375d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.051+0000 7f49baa00640 1 -- 172.21.15.32:0/1772176875 >> 172.21.15.32:0/1772176875 conn(0x7f49b406ba30 msgr2=0x7f49b4145d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:19.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.051+0000 7f49baa00640 1 -- 172.21.15.32:0/1772176875 shutdown_connections 2024-04-17T09:53:19.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.051+0000 7f49baa00640 1 -- 172.21.15.32:0/1772176875 wait complete. 2024-04-17T09:53:19.072 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-17T09:53:19.072 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:53:19.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.207+0000 7fb1b7400640 1 Processor -- start 2024-04-17T09:53:19.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.207+0000 7fb1b7400640 1 -- start start 2024-04-17T09:53:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.211+0000 7fb1b7400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb1b0144e90 0x7fb1b00b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.211+0000 7fb1b7400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1b00b4de0 0x7fb1b015baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.211+0000 7fb1b7400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1b015bfe0 0x7fb1b006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.211+0000 7fb1b7400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb1b015c430 con 0x7fb1b015bfe0 2024-04-17T09:53:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.211+0000 7fb1b7400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb1b0145270 con 0x7fb1b0144e90 2024-04-17T09:53:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.211+0000 7fb1b7400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b0072bb0 con 0x7fb1b00b4de0 2024-04-17T09:53:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.211+0000 7fb1b5400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1b015bfe0 0x7fb1b006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.211+0000 7fb1b5400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1b015bfe0 0x7fb1b006ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46818/0 (socket says 172.21.15.32:46818) 2024-04-17T09:53:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.211+0000 7fb1b5400640 1 -- 172.21.15.32:0/1859083774 learned_addr learned my addr 172.21.15.32:0/1859083774 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.211+0000 7fb1afe00640 1 --2- 172.21.15.32:0/1859083774 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1b00b4de0 0x7fb1b015baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.211+0000 7fb1b4a00640 1 --2- 172.21.15.32:0/1859083774 >> v2:172.21.15.144:3300/0 conn(0x7fb1b0144e90 0x7fb1b00b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.211+0000 7fb1b5400640 1 -- 172.21.15.32:0/1859083774 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1b00b4de0 msgr2=0x7fb1b015baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.211+0000 7fb1b5400640 1 --2- 172.21.15.32:0/1859083774 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1b00b4de0 0x7fb1b015baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.211+0000 7fb1b5400640 1 -- 172.21.15.32:0/1859083774 >> v2:172.21.15.144:3300/0 conn(0x7fb1b0144e90 msgr2=0x7fb1b00b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.211+0000 7fb1b5400640 1 --2- 172.21.15.32:0/1859083774 >> v2:172.21.15.144:3300/0 conn(0x7fb1b0144e90 0x7fb1b00b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.211+0000 7fb1b5400640 1 -- 172.21.15.32:0/1859083774 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b00b3dc0 con 0x7fb1b015bfe0 2024-04-17T09:53:19.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1b5400640 1 --2- 172.21.15.32:0/1859083774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b015bfe0 0x7fb1b006ef60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb1b006ff30 tx=0x7fb1a801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b537c5dfedd2d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1af400640 1 -- 172.21.15.32:0/1859083774 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a80a5860 con 0x7fb1b015bfe0 2024-04-17T09:53:19.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1af400640 1 -- 172.21.15.32:0/1859083774 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a80a5a00 con 0x7fb1b015bfe0 2024-04-17T09:53:19.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1af400640 1 -- 172.21.15.32:0/1859083774 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a80a5dd0 con 0x7fb1b015bfe0 2024-04-17T09:53:19.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1b7400640 1 -- 172.21.15.32:0/1859083774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b015bfe0 msgr2=0x7fb1b006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1b7400640 1 --2- 172.21.15.32:0/1859083774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b015bfe0 0x7fb1b006ef60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb1b006ff30 tx=0x7fb1a801c9c0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1b7400640 1 -- 172.21.15.32:0/1859083774 shutdown_connections 2024-04-17T09:53:19.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1b7400640 1 --2- 172.21.15.32:0/1859083774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b015bfe0 0x7fb1b006ef60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1b7400640 1 --2- 172.21.15.32:0/1859083774 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1b00b4de0 0x7fb1b015baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1b7400640 1 --2- 172.21.15.32:0/1859083774 >> v2:172.21.15.144:3300/0 conn(0x7fb1b0144e90 0x7fb1b00b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1b7400640 1 -- 172.21.15.32:0/1859083774 >> 172.21.15.32:0/1859083774 conn(0x7fb1b006ba30 msgr2=0x7fb1b006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:19.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1b7400640 1 -- 172.21.15.32:0/1859083774 shutdown_connections 2024-04-17T09:53:19.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1b7400640 1 -- 172.21.15.32:0/1859083774 wait complete. 2024-04-17T09:53:19.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1b7400640 1 Processor -- start 2024-04-17T09:53:19.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1b7400640 1 -- start start 2024-04-17T09:53:19.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1b7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b00b4de0 0x7fb1b01418b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1b7400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1b0144e90 0x7fb1b0140610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1b7400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1b015bfe0 0x7fb1b015acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1b7400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b006f8b0 con 0x7fb1b00b4de0 2024-04-17T09:53:19.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1b7400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b00728c0 con 0x7fb1b015bfe0 2024-04-17T09:53:19.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1b7400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b006de50 con 0x7fb1b0144e90 2024-04-17T09:53:19.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1b4a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b00b4de0 0x7fb1b01418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1b4a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b00b4de0 0x7fb1b01418b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46834/0 (socket says 172.21.15.32:46834) 2024-04-17T09:53:19.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.215+0000 7fb1b4a00640 1 -- 172.21.15.32:0/3288506007 learned_addr learned my addr 172.21.15.32:0/3288506007 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:19.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.219+0000 7fb1b5400640 1 --2- 172.21.15.32:0/3288506007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1b015bfe0 0x7fb1b015acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.219+0000 7fb1afe00640 1 --2- 172.21.15.32:0/3288506007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1b0144e90 0x7fb1b0140610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.219+0000 7fb1b4a00640 1 -- 172.21.15.32:0/3288506007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1b0144e90 msgr2=0x7fb1b0140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.219+0000 7fb1b4a00640 1 --2- 172.21.15.32:0/3288506007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1b0144e90 0x7fb1b0140610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.219+0000 7fb1b4a00640 1 -- 172.21.15.32:0/3288506007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1b015bfe0 msgr2=0x7fb1b015acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.219+0000 7fb1b4a00640 1 --2- 172.21.15.32:0/3288506007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1b015bfe0 0x7fb1b015acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.219+0000 7fb1b4a00640 1 -- 172.21.15.32:0/3288506007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b0144640 con 0x7fb1b00b4de0 2024-04-17T09:53:19.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.219+0000 7fb1b4a00640 1 --2- 172.21.15.32:0/3288506007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b00b4de0 0x7fb1b01418b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400e930 tx=0x7fb1a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.219+0000 7fb1acc00640 1 -- 172.21.15.32:0/3288506007 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4017070 con 0x7fb1b00b4de0 2024-04-17T09:53:19.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.219+0000 7fb1b7400640 1 -- 172.21.15.32:0/3288506007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b0140b50 con 0x7fb1b00b4de0 2024-04-17T09:53:19.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.219+0000 7fb1acc00640 1 -- 172.21.15.32:0/3288506007 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a400ca00 con 0x7fb1b00b4de0 2024-04-17T09:53:19.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.219+0000 7fb1acc00640 1 -- 172.21.15.32:0/3288506007 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a400cdb0 con 0x7fb1b00b4de0 2024-04-17T09:53:19.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.219+0000 7fb1b7400640 1 -- 172.21.15.32:0/3288506007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b0141f00 con 0x7fb1b00b4de0 2024-04-17T09:53:19.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.219+0000 7fb1b7400640 1 -- 172.21.15.32:0/3288506007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b0068a90 con 0x7fb1b00b4de0 2024-04-17T09:53:19.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.219+0000 7fb1acc00640 1 -- 172.21.15.32:0/3288506007 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb1a401a070 con 0x7fb1b00b4de0 2024-04-17T09:53:19.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.219+0000 7fb1acc00640 1 --2- 172.21.15.32:0/3288506007 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1900683b0 0x7fb19006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.219+0000 7fb1acc00640 1 -- 172.21.15.32:0/3288506007 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb1a4086990 con 0x7fb1b00b4de0 2024-04-17T09:53:19.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.223+0000 7fb1afe00640 1 --2- 172.21.15.32:0/3288506007 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1900683b0 0x7fb19006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.223+0000 7fb1afe00640 1 --2- 172.21.15.32:0/3288506007 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1900683b0 0x7fb19006a870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb1b00b3160 tx=0x7fb19c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.227+0000 7fb1acc00640 1 -- 172.21.15.32:0/3288506007 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fb1a4057020 con 0x7fb1b00b4de0 2024-04-17T09:53:19.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.391+0000 7fb1b7400640 1 -- 172.21.15.32:0/3288506007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1b0072df0 con 0x7fb1b00b4de0 2024-04-17T09:53:19.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.395+0000 7fb1acc00640 1 -- 172.21.15.32:0/3288506007 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7fb1a4056920 con 0x7fb1b00b4de0 2024-04-17T09:53:19.397 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:19.397 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:14.779535+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:53:19.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.395+0000 7fb1b7400640 1 -- 172.21.15.32:0/3288506007 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1900683b0 msgr2=0x7fb19006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.395+0000 7fb1b7400640 1 --2- 172.21.15.32:0/3288506007 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1900683b0 0x7fb19006a870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb1b00b3160 tx=0x7fb19c008040 comp rx=0 tx=0).stop 2024-04-17T09:53:19.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.395+0000 7fb1b7400640 1 -- 172.21.15.32:0/3288506007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b00b4de0 msgr2=0x7fb1b01418b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.395+0000 7fb1b7400640 1 --2- 172.21.15.32:0/3288506007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b00b4de0 0x7fb1b01418b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400e930 tx=0x7fb1a400edf0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.395+0000 7fb1b7400640 1 -- 172.21.15.32:0/3288506007 shutdown_connections 2024-04-17T09:53:19.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.395+0000 7fb1b7400640 1 --2- 172.21.15.32:0/3288506007 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1900683b0 0x7fb19006a870 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.395+0000 7fb1b7400640 1 --2- 172.21.15.32:0/3288506007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1b0144e90 0x7fb1b0140610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.395+0000 7fb1b7400640 1 --2- 172.21.15.32:0/3288506007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1b015bfe0 0x7fb1b015acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.395+0000 7fb1b7400640 1 --2- 172.21.15.32:0/3288506007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b00b4de0 0x7fb1b01418b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.395+0000 7fb1b7400640 1 -- 172.21.15.32:0/3288506007 >> 172.21.15.32:0/3288506007 conn(0x7fb1b006ba30 msgr2=0x7fb1b006be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:19.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.395+0000 7fb1b7400640 1 -- 172.21.15.32:0/3288506007 shutdown_connections 2024-04-17T09:53:19.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.395+0000 7fb1b7400640 1 -- 172.21.15.32:0/3288506007 wait complete. 2024-04-17T09:53:19.418 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-17T09:53:19.419 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-17T09:53:19.419 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-17T09:53:19.419 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-17T09:53:19.420 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-17T09:53:19.420 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-17T09:53:19.420 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-17T09:53:19.421 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-17T09:53:19.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.659+0000 7f017de00640 1 Processor -- start 2024-04-17T09:53:19.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.659+0000 7f017de00640 1 -- start start 2024-04-17T09:53:19.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.659+0000 7f5759e00640 1 Processor -- start 2024-04-17T09:53:19.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.659+0000 7f5759e00640 1 -- start start 2024-04-17T09:53:19.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.659+0000 7f017de00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f017815c8e0 0x7f01780acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.659+0000 7f017de00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01780cd230 0x7f01780b1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.659+0000 7f017de00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01780cf9c0 con 0x7f01780c37e0 2024-04-17T09:53:19.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.659+0000 7f017de00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f01780b29c0 con 0x7f017815c8e0 2024-04-17T09:53:19.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.659+0000 7f017de00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f01780cfc60 con 0x7f01780cd230 2024-04-17T09:53:19.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.659+0000 7f017d400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01780cd230 0x7f01780b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.659+0000 7f017d400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01780cd230 0x7f01780b1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:56364/0 (socket says 172.21.15.32:56364) 2024-04-17T09:53:19.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.659+0000 7f017ca00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f017815c8e0 0x7f01780acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.663+0000 7f017d400640 1 -- 172.21.15.32:0/3185228830 learned_addr learned my addr 172.21.15.32:0/3185228830 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:19.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.663+0000 7f5759e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f57540d0cb0 0x7f57540b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.663+0000 7f5759e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57540b2d20 con 0x7f57540ce2f0 2024-04-17T09:53:19.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.663+0000 7f5759e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57540b3000 con 0x7f575415fc90 2024-04-17T09:53:19.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.663+0000 7f5759e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f57540b3930 con 0x7f57540d0cb0 2024-04-17T09:53:19.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.663+0000 7f5753400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f57540ce2f0 0x7f575409a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39188/0 (socket says 172.21.15.32:39188) 2024-04-17T09:53:19.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.663+0000 7f5752000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f57540d0cb0 0x7f57540b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.663+0000 7f5752000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f57540d0cb0 0x7f57540b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:40792/0 (socket says 172.21.15.32:40792) 2024-04-17T09:53:19.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.663+0000 7f5752000640 1 -- 172.21.15.32:0/2356053640 learned_addr learned my addr 172.21.15.32:0/2356053640 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:19.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.663+0000 7f5752000640 1 -- 172.21.15.32:0/2356053640 >> v1:172.21.15.144:6789/0 conn(0x7f575415fc90 legacy=0x7f57540d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.663+0000 7f5752000640 1 -- 172.21.15.32:0/2356053640 >> v1:172.21.15.32:6789/0 conn(0x7f57540ce2f0 legacy=0x7f575409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:19.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.663+0000 7f5752000640 1 -- 172.21.15.32:0/2356053640 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57540b32e0 con 0x7f57540d0cb0 2024-04-17T09:53:19.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.663+0000 7f017d400640 1 -- 172.21.15.32:0/3185228830 >> v2:172.21.15.144:3300/0 conn(0x7f017815c8e0 msgr2=0x7f01780acfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:19.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.663+0000 7f017d400640 1 --2- 172.21.15.32:0/3185228830 >> v2:172.21.15.144:3300/0 conn(0x7f017815c8e0 0x7f01780acfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.663+0000 7f017d400640 1 -- 172.21.15.32:0/3185228830 >> v1:172.21.15.144:6789/0 conn(0x7f01780c37e0 legacy=0x7f01780ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.663+0000 7f017d400640 1 -- 172.21.15.32:0/3185228830 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01780ce1d0 con 0x7f01780cd230 2024-04-17T09:53:19.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.663+0000 7fe2b1200640 1 Processor -- start 2024-04-17T09:53:19.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.663+0000 7fe2b1200640 1 -- start start 2024-04-17T09:53:19.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.663+0000 7fe2b1200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac15eea0 0x7fe2ac0cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.663+0000 7fe2b1200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2ac070c30 0x7fe2ac06ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.663+0000 7fe2b1200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2ac06fb30 con 0x7fe2ac070190 2024-04-17T09:53:19.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.663+0000 7fe2b1200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe2ac0cf9d0 con 0x7fe2ac070c30 2024-04-17T09:53:19.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.663+0000 7fe2b1200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ac06fe10 con 0x7fe2ac15eea0 2024-04-17T09:53:19.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2aa800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe2ac070190 0x7fe2ac0afc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39190/0 (socket says 172.21.15.32:39190) 2024-04-17T09:53:19.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2aa800640 1 -- 172.21.15.32:0/20568354 learned_addr learned my addr 172.21.15.32:0/20568354 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:19.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2a9400640 1 --2- 172.21.15.32:0/20568354 >> v2:172.21.15.32:3300/0 conn(0x7fe2ac070c30 0x7fe2ac06ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2a8a00640 1 -- 172.21.15.32:0/20568354 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3880416507 0 0) 0x7fe2ac06fb30 con 0x7fe2ac070190 2024-04-17T09:53:19.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2a8a00640 1 -- 172.21.15.32:0/20568354 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe290003530 con 0x7fe2ac070190 2024-04-17T09:53:19.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7f017ca00640 1 --2- 172.21.15.32:0/3185228830 >> v2:172.21.15.144:3300/0 conn(0x7f017815c8e0 0x7f01780acfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:53:19.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2a8a00640 1 -- 172.21.15.32:0/20568354 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3439175824 0 0) 0x7fe290003530 con 0x7fe2ac070190 2024-04-17T09:53:19.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2a8a00640 1 -- 172.21.15.32:0/20568354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac15eea0 msgr2=0x7fe2ac0cbe70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T09:53:19.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2a8a00640 1 --2- 172.21.15.32:0/20568354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac15eea0 0x7fe2ac0cbe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2a8a00640 1 -- 172.21.15.32:0/20568354 >> v2:172.21.15.32:3300/0 conn(0x7fe2ac070c30 msgr2=0x7fe2ac06ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:19.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2a8a00640 1 --2- 172.21.15.32:0/20568354 >> v2:172.21.15.32:3300/0 conn(0x7fe2ac070c30 0x7fe2ac06ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2a8a00640 1 -- 172.21.15.32:0/20568354 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ac0cfc20 con 0x7fe2ac070190 2024-04-17T09:53:19.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7f017d400640 1 --2- 172.21.15.32:0/3185228830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01780cd230 0x7f01780b1130 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0170009e50 tx=0x7f017001ca70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59a957f512f72e63 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2a8a00640 1 -- 172.21.15.32:0/20568354 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7fe2a40019e0 con 0x7fe2ac070190 2024-04-17T09:53:19.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2a8a00640 1 -- 172.21.15.32:0/20568354 >> v1:172.21.15.32:6789/0 conn(0x7fe2ac070190 legacy=0x7fe2ac0afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2a8a00640 1 --2- 172.21.15.32:0/20568354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe290003a20 0x7fe290003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2a8a00640 1 --2- 172.21.15.32:0/20568354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe290004810 0x7fe290004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2a8a00640 1 --2- 172.21.15.32:0/20568354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac15eea0 0x7fe29000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2a8a00640 1 -- 172.21.15.32:0/20568354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac0cf9d0 con 0x7fe290003a20 2024-04-17T09:53:19.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2a8a00640 1 -- 172.21.15.32:0/20568354 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac06fe10 con 0x7fe290004810 2024-04-17T09:53:19.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2a8a00640 1 -- 172.21.15.32:0/20568354 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe290005620 con 0x7fe2ac15eea0 2024-04-17T09:53:19.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7f0177400640 1 -- 172.21.15.32:0/3185228830 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01700a5960 con 0x7f01780cd230 2024-04-17T09:53:19.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7f0177400640 1 -- 172.21.15.32:0/3185228830 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01700a5b00 con 0x7f01780cd230 2024-04-17T09:53:19.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7f0177400640 1 -- 172.21.15.32:0/3185228830 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01700ad3f0 con 0x7f01780cd230 2024-04-17T09:53:19.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7f5752000640 1 --2- 172.21.15.32:0/2356053640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57540d0cb0 0x7f57540b26d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f574c009e50 tx=0x7f574c01c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccd48f4ffacabfb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2aa800640 1 --2- 172.21.15.32:0/20568354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac15eea0 0x7fe29000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2a9e00640 1 --2- 172.21.15.32:0/20568354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe290003a20 0x7fe290003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7f5751600640 1 -- 172.21.15.32:0/2356053640 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c0a58a0 con 0x7f57540d0cb0 2024-04-17T09:53:19.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7f5751600640 1 -- 172.21.15.32:0/2356053640 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f574c0a5a40 con 0x7f57540d0cb0 2024-04-17T09:53:19.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7f5751600640 1 -- 172.21.15.32:0/2356053640 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c0a5e10 con 0x7f57540d0cb0 2024-04-17T09:53:19.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2a9e00640 1 -- 172.21.15.32:0/20568354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac15eea0 msgr2=0x7fe29000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:19.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2a9e00640 1 --2- 172.21.15.32:0/20568354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac15eea0 0x7fe29000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2a9e00640 1 -- 172.21.15.32:0/20568354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe290004810 msgr2=0x7fe290004c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T09:53:19.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2a9e00640 1 --2- 172.21.15.32:0/20568354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe290004810 0x7fe290004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.667+0000 7fe2a9e00640 1 -- 172.21.15.32:0/20568354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29000bfd0 con 0x7fe290003a20 2024-04-17T09:53:19.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.671+0000 7f017de00640 1 -- 172.21.15.32:0/3185228830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01780cd230 msgr2=0x7f01780b1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.671+0000 7f017de00640 1 --2- 172.21.15.32:0/3185228830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01780cd230 0x7f01780b1130 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0170009e50 tx=0x7f017001ca70 comp rx=0 tx=0).stop 2024-04-17T09:53:19.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.671+0000 7f5759e00640 1 -- 172.21.15.32:0/2356053640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57540d0cb0 msgr2=0x7f57540b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.671+0000 7f5759e00640 1 --2- 172.21.15.32:0/2356053640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57540d0cb0 0x7f57540b26d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f574c009e50 tx=0x7f574c01c9c0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.671+0000 7f017de00640 1 -- 172.21.15.32:0/3185228830 shutdown_connections 2024-04-17T09:53:19.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.671+0000 7f017de00640 1 --2- 172.21.15.32:0/3185228830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01780cd230 0x7f01780b1130 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.671+0000 7f017de00640 1 --2- 172.21.15.32:0/3185228830 >> v2:172.21.15.144:3300/0 conn(0x7f017815c8e0 0x7f01780acfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.671+0000 7f017de00640 1 -- 172.21.15.32:0/3185228830 >> 172.21.15.32:0/3185228830 conn(0x7f01780b1d00 msgr2=0x7f01780b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:19.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.671+0000 7f5759e00640 1 -- 172.21.15.32:0/2356053640 shutdown_connections 2024-04-17T09:53:19.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.671+0000 7f5759e00640 1 --2- 172.21.15.32:0/2356053640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57540d0cb0 0x7f57540b26d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.671+0000 7f5759e00640 1 -- 172.21.15.32:0/2356053640 >> 172.21.15.32:0/2356053640 conn(0x7f57540b4120 msgr2=0x7f57540b4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:19.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.671+0000 7f017de00640 1 -- 172.21.15.32:0/3185228830 shutdown_connections 2024-04-17T09:53:19.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.671+0000 7f017de00640 1 -- 172.21.15.32:0/3185228830 wait complete. 2024-04-17T09:53:19.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.671+0000 7fc2f4e00640 1 Processor -- start 2024-04-17T09:53:19.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.671+0000 7f5759e00640 1 -- 172.21.15.32:0/2356053640 shutdown_connections 2024-04-17T09:53:19.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.671+0000 7fc2f4e00640 1 -- start start 2024-04-17T09:53:19.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.671+0000 7f5759e00640 1 -- 172.21.15.32:0/2356053640 wait complete. 2024-04-17T09:53:19.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.671+0000 7fc2f4e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc2f00d0270 0x7fc2f015c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7f017de00640 1 Processor -- start 2024-04-17T09:53:19.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7fc2ede00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc2f00d0750 0x7fc2f00d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39202/0 (socket says 172.21.15.32:39202) 2024-04-17T09:53:19.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7fc2ee800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc2f00d0270 0x7fc2f015c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7fc2ee800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc2f00d0270 0x7fc2f015c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:40804/0 (socket says 172.21.15.32:40804) 2024-04-17T09:53:19.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7fc2ed400640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fc2f00b3a20 0x7fc2f00cd020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:34440/0 (socket says 172.21.15.32:34440) 2024-04-17T09:53:19.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7f017de00640 1 -- start start 2024-04-17T09:53:19.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7fc2ede00640 1 -- 172.21.15.32:0/1250703951 learned_addr learned my addr 172.21.15.32:0/1250703951 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:19.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7fe2a9e00640 1 --2- 172.21.15.32:0/20568354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe290003a20 0x7fe290003e60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0097e0 tx=0x7fe29c01c2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8203925d68eff53 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7fc2f4e00640 1 -- 172.21.15.32:0/1250703951 --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2f015cfa0 con 0x7fc2f00d0750 2024-04-17T09:53:19.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7fe2a8a00640 1 -- 172.21.15.32:0/20568354 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c0b0070 con 0x7fe290003a20 2024-04-17T09:53:19.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7fe2a8a00640 1 -- 172.21.15.32:0/20568354 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe29c0ab620 con 0x7fe290003a20 2024-04-17T09:53:19.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7fe2a8a00640 1 -- 172.21.15.32:0/20568354 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c0093b0 con 0x7fe290003a20 2024-04-17T09:53:19.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7fc2f4e00640 1 -- 172.21.15.32:0/1250703951 --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2f015d280 con 0x7fc2f00b3a20 2024-04-17T09:53:19.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7fc2f4e00640 1 -- 172.21.15.32:0/1250703951 --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc2f015dd60 con 0x7fc2f00d0270 2024-04-17T09:53:19.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7f5759e00640 1 Processor -- start 2024-04-17T09:53:19.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7f83b5a00640 1 Processor -- start 2024-04-17T09:53:19.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7f5759e00640 1 -- start start 2024-04-17T09:53:19.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7fc2ee800640 1 -- 172.21.15.32:0/1250703951 >> v1:172.21.15.144:6789/0 conn(0x7fc2f00b3a20 legacy=0x7fc2f00cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7fe2b1200640 1 -- 172.21.15.32:0/20568354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe290003a20 msgr2=0x7fe290003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7fe2b1200640 1 --2- 172.21.15.32:0/20568354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe290003a20 0x7fe290003e60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0097e0 tx=0x7fe29c01c2e0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7fc2ee800640 1 -- 172.21.15.32:0/1250703951 >> v1:172.21.15.32:6789/0 conn(0x7fc2f00d0750 legacy=0x7fc2f00d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7fc2ee800640 1 -- 172.21.15.32:0/1250703951 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f015d560 con 0x7fc2f00d0270 2024-04-17T09:53:19.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7f83b5a00640 1 -- start start 2024-04-17T09:53:19.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7f017de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01780c37e0 0x7f01780c4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7fe2aa800640 1 -- 172.21.15.32:0/20568354 reap_dead start 2024-04-17T09:53:19.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7fe2b1200640 1 -- 172.21.15.32:0/20568354 shutdown_connections 2024-04-17T09:53:19.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.675+0000 7fe2b1200640 1 -- 172.21.15.32:0/20568354 >> 172.21.15.32:0/20568354 conn(0x7fe2ac0b2f00 msgr2=0x7fe2ac0b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:19.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f135da00640 1 Processor -- start 2024-04-17T09:53:19.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f017de00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01780cd230 0x7f01780c18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f135da00640 1 -- start start 2024-04-17T09:53:19.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f017de00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f017815c8e0 0x7f01780992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f017ca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01780c37e0 0x7f01780c4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f017ca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01780c37e0 0x7f01780c4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46858/0 (socket says 172.21.15.32:46858) 2024-04-17T09:53:19.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f017de00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f01780ad6a0 con 0x7f01780c37e0 2024-04-17T09:53:19.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f017de00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f01780ad520 con 0x7f01780cd230 2024-04-17T09:53:19.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f017de00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f01780ca780 con 0x7f017815c8e0 2024-04-17T09:53:19.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f1e2a200640 1 Processor -- start 2024-04-17T09:53:19.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f017ca00640 1 -- 172.21.15.32:0/2244979789 learned_addr learned my addr 172.21.15.32:0/2244979789 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:19.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f0177e00640 1 --2- 172.21.15.32:0/2244979789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01780cd230 0x7f01780c18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f017d400640 1 --2- 172.21.15.32:0/2244979789 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f017815c8e0 0x7f01780992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f017ca00640 1 -- 172.21.15.32:0/2244979789 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f017815c8e0 msgr2=0x7f01780992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f017ca00640 1 --2- 172.21.15.32:0/2244979789 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f017815c8e0 0x7f01780992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f017ca00640 1 -- 172.21.15.32:0/2244979789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01780cd230 msgr2=0x7f01780c18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f017ca00640 1 --2- 172.21.15.32:0/2244979789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01780cd230 0x7f01780c18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f017ca00640 1 -- 172.21.15.32:0/2244979789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0178157a10 con 0x7f01780c37e0 2024-04-17T09:53:19.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f017d400640 1 --2- 172.21.15.32:0/2244979789 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f017815c8e0 0x7f01780992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:19.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f017ca00640 1 --2- 172.21.15.32:0/2244979789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01780c37e0 0x7f01780c4080 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f016800e9b0 tx=0x7f016800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f0177e00640 1 --2- 172.21.15.32:0/2244979789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01780cd230 0x7f01780c18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:19.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7fe2b1200640 1 -- 172.21.15.32:0/20568354 shutdown_connections 2024-04-17T09:53:19.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7fe2b1200640 1 -- 172.21.15.32:0/20568354 wait complete. 2024-04-17T09:53:19.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f83b5a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83b014ca40 0x7f83b006d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f83b5a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b009b400 0x7f83b00c5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f83b5a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f83b00c5ac0 0x7f83b0166b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f83b5a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f83b00c5f30 con 0x7f83b014ca40 2024-04-17T09:53:19.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f83b5a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f83b00b1ef0 con 0x7f83b00c5ac0 2024-04-17T09:53:19.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f83b5a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f83b009be50 con 0x7f83b009b400 2024-04-17T09:53:19.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f83af400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f83b00c5ac0 0x7f83b0166b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f83ae000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b009b400 0x7f83b00c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f83ae000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b009b400 0x7f83b00c5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:56398/0 (socket says 172.21.15.32:56398) 2024-04-17T09:53:19.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f83ae000640 1 -- 172.21.15.32:0/4269712655 learned_addr learned my addr 172.21.15.32:0/4269712655 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:19.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f1e2a200640 1 -- start start 2024-04-17T09:53:19.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f135da00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13580b2820 0x7f13580afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f135da00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f13580ccb00 0x7f135815a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f135da00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13580cda70 con 0x7f13580ce030 2024-04-17T09:53:19.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f135da00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f13580caa00 con 0x7f13580ccb00 2024-04-17T09:53:19.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f135da00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f13580cace0 con 0x7f13580b2820 2024-04-17T09:53:19.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f1e2a200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1e24166320 0x7f1e240cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f1e2a200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e240cc080 0x7f1e2406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f1e2a200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e2406f550 con 0x7f1e24070430 2024-04-17T09:53:19.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f1e2a200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1e2406fb60 con 0x7f1e240cc080 2024-04-17T09:53:19.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f135d000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f13580ce030 0x7f13580cbaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39218/0 (socket says 172.21.15.32:39218) 2024-04-17T09:53:19.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f135d000640 1 -- 172.21.15.32:0/2768075511 learned_addr learned my addr 172.21.15.32:0/2768075511 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:19.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f1e2a200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1e240c3f80 con 0x7f1e24166320 2024-04-17T09:53:19.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f5759e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57540ce2f0 0x7f5754152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f5759e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57540d0cb0 0x7f57540cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f5759e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f575415fc90 0x7f57540ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f5759e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f575409bd20 con 0x7f57540d0cb0 2024-04-17T09:53:19.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f5759e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f575409bba0 con 0x7f575415fc90 2024-04-17T09:53:19.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f5759e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f575409bea0 con 0x7f57540ce2f0 2024-04-17T09:53:19.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f5752000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57540d0cb0 0x7f57540cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f5752000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57540d0cb0 0x7f57540cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46874/0 (socket says 172.21.15.32:46874) 2024-04-17T09:53:19.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f5752000640 1 -- 172.21.15.32:0/536377384 learned_addr learned my addr 172.21.15.32:0/536377384 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:19.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f1357400640 1 --2- 172.21.15.32:0/2768075511 >> v2:172.21.15.32:3300/0 conn(0x7f13580ccb00 0x7f135815a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f1357e00640 1 --2- 172.21.15.32:0/2768075511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13580b2820 0x7f13580afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f5752a00640 1 --2- 172.21.15.32:0/536377384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57540ce2f0 0x7f5754152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f5753400640 1 --2- 172.21.15.32:0/536377384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f575415fc90 0x7f57540ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f83af400640 1 -- 172.21.15.32:0/4269712655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b009b400 msgr2=0x7f83b00c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:19.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f83af400640 1 --2- 172.21.15.32:0/4269712655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b009b400 0x7f83b00c5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f83af400640 1 -- 172.21.15.32:0/4269712655 >> v2:172.21.15.32:3300/0 conn(0x7f83b014ca40 msgr2=0x7f83b006d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:19.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f83af400640 1 --2- 172.21.15.32:0/4269712655 >> v2:172.21.15.32:3300/0 conn(0x7f83b014ca40 0x7f83b006d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f83af400640 1 -- 172.21.15.32:0/4269712655 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b014ce20 con 0x7f83b00c5ac0 2024-04-17T09:53:19.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f1356a00640 1 -- 172.21.15.32:0/2768075511 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 770111247 0 0) 0x7f13580cda70 con 0x7f13580ce030 2024-04-17T09:53:19.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f1356a00640 1 -- 172.21.15.32:0/2768075511 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f133c003530 con 0x7f13580ce030 2024-04-17T09:53:19.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f0174c00640 1 -- 172.21.15.32:0/2244979789 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0168017070 con 0x7f01780c37e0 2024-04-17T09:53:19.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f0174c00640 1 -- 172.21.15.32:0/2244979789 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f016800ca00 con 0x7f01780c37e0 2024-04-17T09:53:19.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f0174c00640 1 -- 172.21.15.32:0/2244979789 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016800cdb0 con 0x7f01780c37e0 2024-04-17T09:53:19.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.679+0000 7f1e29800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e240cc080 0x7f1e2406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f1e29800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e240cc080 0x7f1e2406ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46890/0 (socket says 172.21.15.32:46890) 2024-04-17T09:53:19.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f1e29800640 1 -- 172.21.15.32:0/869297753 learned_addr learned my addr 172.21.15.32:0/869297753 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:19.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f017de00640 1 -- 172.21.15.32:0/2244979789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0178157cf0 con 0x7f01780c37e0 2024-04-17T09:53:19.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f017de00640 1 -- 172.21.15.32:0/2244979789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0178099d90 con 0x7f01780c37e0 2024-04-17T09:53:19.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f1e28e00640 1 --2- 172.21.15.32:0/869297753 >> v2:172.21.15.144:3300/0 conn(0x7f1e24166320 0x7f1e240cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f5752a00640 1 -- 172.21.15.32:0/536377384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f575415fc90 msgr2=0x7f57540ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f5752a00640 1 --2- 172.21.15.32:0/536377384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f575415fc90 0x7f57540ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f5752a00640 1 -- 172.21.15.32:0/536377384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57540d0cb0 msgr2=0x7f57540cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f5752a00640 1 --2- 172.21.15.32:0/536377384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57540d0cb0 0x7f57540cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f5752a00640 1 -- 172.21.15.32:0/536377384 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57540b55a0 con 0x7f57540ce2f0 2024-04-17T09:53:19.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f1356a00640 1 -- 172.21.15.32:0/2768075511 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 83336466 0 0) 0x7f133c003530 con 0x7f13580ce030 2024-04-17T09:53:19.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f1356a00640 1 -- 172.21.15.32:0/2768075511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13580b2820 msgr2=0x7f13580afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:19.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f1e28e00640 1 -- 172.21.15.32:0/869297753 >> v2:172.21.15.32:3300/0 conn(0x7f1e240cc080 msgr2=0x7f1e2406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:19.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f1e28e00640 1 --2- 172.21.15.32:0/869297753 >> v2:172.21.15.32:3300/0 conn(0x7f1e240cc080 0x7f1e2406ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f1e28e00640 1 -- 172.21.15.32:0/869297753 >> v1:172.21.15.32:6789/0 conn(0x7f1e24070430 legacy=0x7f1e240cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f1e28e00640 1 -- 172.21.15.32:0/869297753 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2406f7f0 con 0x7f1e24166320 2024-04-17T09:53:19.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7fc2ee800640 1 --2- 172.21.15.32:0/1250703951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2f00d0270 0x7fc2f015c9c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009740 tx=0x7fc2e001c160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dab35048fa0fda9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f5752a00640 1 --2- 172.21.15.32:0/536377384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57540ce2f0 0x7f5754152090 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5740000f70 tx=0x7f574000c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f1356a00640 1 --2- 172.21.15.32:0/2768075511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13580b2820 0x7f13580afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f1356a00640 1 -- 172.21.15.32:0/2768075511 >> v2:172.21.15.32:3300/0 conn(0x7f13580ccb00 msgr2=0x7f135815a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:19.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f1356a00640 1 --2- 172.21.15.32:0/2768075511 >> v2:172.21.15.32:3300/0 conn(0x7f13580ccb00 0x7f135815a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f1356a00640 1 -- 172.21.15.32:0/2768075511 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13580cafc0 con 0x7f13580ce030 2024-04-17T09:53:19.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f1356a00640 1 -- 172.21.15.32:0/2768075511 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f1348001680 con 0x7f13580ce030 2024-04-17T09:53:19.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f1356a00640 1 -- 172.21.15.32:0/2768075511 >> v1:172.21.15.32:6789/0 conn(0x7f13580ce030 legacy=0x7f13580cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f574b400640 1 -- 172.21.15.32:0/536377384 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740017070 con 0x7f57540ce2f0 2024-04-17T09:53:19.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7fc2eca00640 1 -- 172.21.15.32:0/1250703951 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00b0070 con 0x7fc2f00d0270 2024-04-17T09:53:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7fc2eca00640 1 -- 172.21.15.32:0/1250703951 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e00ab600 con 0x7fc2f00d0270 2024-04-17T09:53:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7fc2eca00640 1 -- 172.21.15.32:0/1250703951 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00ab9d0 con 0x7fc2f00d0270 2024-04-17T09:53:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f574b400640 1 -- 172.21.15.32:0/536377384 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57400052d0 con 0x7f57540ce2f0 2024-04-17T09:53:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f574b400640 1 -- 172.21.15.32:0/536377384 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57400054c0 con 0x7f57540ce2f0 2024-04-17T09:53:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f1356a00640 1 --2- 172.21.15.32:0/2768075511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13580b2820 0x7f133c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f5759e00640 1 -- 172.21.15.32:0/536377384 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57540cafe0 con 0x7f57540ce2f0 2024-04-17T09:53:19.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f5759e00640 1 -- 172.21.15.32:0/536377384 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5754154e70 con 0x7f57540ce2f0 2024-04-17T09:53:19.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7fc2f4e00640 1 -- 172.21.15.32:0/1250703951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2f00d0270 msgr2=0x7fc2f015c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7fc2f4e00640 1 --2- 172.21.15.32:0/1250703951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2f00d0270 0x7fc2f015c9c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009740 tx=0x7fc2e001c160 comp rx=0 tx=0).stop 2024-04-17T09:53:19.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7fe2b1200640 1 Processor -- start 2024-04-17T09:53:19.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7fc2f4e00640 1 -- 172.21.15.32:0/1250703951 shutdown_connections 2024-04-17T09:53:19.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7fc0d3400640 1 Processor -- start 2024-04-17T09:53:19.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7fc2f4e00640 1 --2- 172.21.15.32:0/1250703951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2f00d0270 0x7fc2f015c9c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fc2f4e00640 1 -- 172.21.15.32:0/1250703951 >> 172.21.15.32:0/1250703951 conn(0x7fc2f00b2f00 msgr2=0x7fc2f00b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:19.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f017de00640 1 -- 172.21.15.32:0/2244979789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0154001040 con 0x7f01780c37e0 2024-04-17T09:53:19.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fe2b1200640 1 -- start start 2024-04-17T09:53:19.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f83af400640 1 --2- 172.21.15.32:0/4269712655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b00c5ac0 0x7f83b0166b60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f83b0073580 tx=0x7f839801ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d185f18e1b6ae51 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fc0d3400640 1 -- start start 2024-04-17T09:53:19.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f574b400640 1 -- 172.21.15.32:0/536377384 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f574001a070 con 0x7f57540ce2f0 2024-04-17T09:53:19.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f83ad600640 1 -- 172.21.15.32:0/4269712655 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83980b5070 con 0x7f83b00c5ac0 2024-04-17T09:53:19.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f1356a00640 1 --2- 172.21.15.32:0/2768075511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f133c003f90 0x7f133c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f1356a00640 1 --2- 172.21.15.32:0/2768075511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f133c004d50 0x7f133c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f1356a00640 1 -- 172.21.15.32:0/2768075511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f13580caa00 con 0x7f133c004d50 2024-04-17T09:53:19.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f1356a00640 1 -- 172.21.15.32:0/2768075511 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f13580cace0 con 0x7f133c003f90 2024-04-17T09:53:19.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f83ad600640 1 -- 172.21.15.32:0/4269712655 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83980a8930 con 0x7f83b00c5ac0 2024-04-17T09:53:19.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f83ad600640 1 -- 172.21.15.32:0/4269712655 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83980a8d00 con 0x7f83b00c5ac0 2024-04-17T09:53:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.683+0000 7f1356a00640 1 -- 172.21.15.32:0/2768075511 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f133c00bb30 con 0x7f13580b2820 2024-04-17T09:53:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f1357400640 1 --2- 172.21.15.32:0/2768075511 >> v2:172.21.15.32:3300/0 conn(0x7f13580ccb00 0x7f135815a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f1357400640 1 --2- 172.21.15.32:0/2768075511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f133c003f90 0x7f133c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f1357e00640 1 --2- 172.21.15.32:0/2768075511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13580b2820 0x7f133c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fe2b1200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe290003a20 0x7fe2ac0b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fe2b1200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2ac1f8a00 0x7fe2ac0c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f1357400640 1 -- 172.21.15.32:0/2768075511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13580b2820 msgr2=0x7f133c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f1357400640 1 --2- 172.21.15.32:0/2768075511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13580b2820 0x7f133c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f1357400640 1 -- 172.21.15.32:0/2768075511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f133c004d50 msgr2=0x7f133c0051d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T09:53:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fe2b1200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac157590 0x7fe2ac0c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fc0d3400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc0cc06c550 0x7fc0cc06e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fc0d3400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0cc06eab0 0x7fc0cc15a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fc0d3400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0cc15ab70 0x7fc0cc0b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fc0d3400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc0cc15b210 con 0x7fc0cc15ab70 2024-04-17T09:53:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fc0d3400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc0cc161790 con 0x7fc0cc06c550 2024-04-17T09:53:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fc0d3400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc0cc06f310 con 0x7fc0cc06eab0 2024-04-17T09:53:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f574b400640 1 --2- 172.21.15.32:0/536377384 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f57340683d0 0x7f573406a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f574b400640 1 -- 172.21.15.32:0/536377384 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5740085cd0 con 0x7f57540ce2f0 2024-04-17T09:53:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fe2a9400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2ac1f8a00 0x7fe2ac0c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fe2a9400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2ac1f8a00 0x7fe2ac0c8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:40856/0 (socket says 172.21.15.32:40856) 2024-04-17T09:53:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fe2a9400640 1 -- 172.21.15.32:0/2041706341 learned_addr learned my addr 172.21.15.32:0/2041706341 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fe2b1200640 1 -- 172.21.15.32:0/2041706341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac0cbe70 con 0x7fe290003a20 2024-04-17T09:53:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fe2b1200640 1 -- 172.21.15.32:0/2041706341 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac0b3d10 con 0x7fe2ac1f8a00 2024-04-17T09:53:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fe2b1200640 1 -- 172.21.15.32:0/2041706341 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ac0cbff0 con 0x7fe2ac157590 2024-04-17T09:53:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fc0cbe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0cc06eab0 0x7fc0cc15a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fc0cbe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0cc06eab0 0x7fc0cc15a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:56422/0 (socket says 172.21.15.32:56422) 2024-04-17T09:53:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fc0cbe00640 1 -- 172.21.15.32:0/3730389240 learned_addr learned my addr 172.21.15.32:0/3730389240 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fe2a9e00640 1 --2- 172.21.15.32:0/2041706341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe290003a20 0x7fe2ac0b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f5752000640 1 --2- 172.21.15.32:0/536377384 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f57340683d0 0x7f573406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fe2aa800640 1 --2- 172.21.15.32:0/2041706341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac157590 0x7fe2ac0c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fc0d1400640 1 --2- 172.21.15.32:0/3730389240 >> v2:172.21.15.32:3300/0 conn(0x7fc0cc15ab70 0x7fc0cc0b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fc0d0a00640 1 --2- 172.21.15.32:0/3730389240 >> v2:172.21.15.144:3300/0 conn(0x7fc0cc06c550 0x7fc0cc06e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f0174c00640 1 -- 172.21.15.32:0/2244979789 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f016801a070 con 0x7f01780c37e0 2024-04-17T09:53:19.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f0174c00640 1 --2- 172.21.15.32:0/2244979789 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f015c0683d0 0x7f015c06a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f0174c00640 1 -- 172.21.15.32:0/2244979789 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0168086c00 con 0x7f01780c37e0 2024-04-17T09:53:19.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f0174c00640 1 --2- 172.21.15.32:0/2244979789 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x7f015c070570 0x7f015c072990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f0174c00640 1 -- 172.21.15.32:0/2244979789 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f015c073040 con 0x7f015c070570 2024-04-17T09:53:19.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f0174c00640 1 -- 172.21.15.32:0/2244979789 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f016808d050 con 0x7f01780c37e0 2024-04-17T09:53:19.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f5759e00640 1 --2- 172.21.15.32:0/536377384 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x7f57540ca870 0x7f57540b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f5759e00640 1 -- 172.21.15.32:0/536377384 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f57540b5880 con 0x7f57540ca870 2024-04-17T09:53:19.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fe2a9400640 1 -- 172.21.15.32:0/2041706341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac157590 msgr2=0x7fe2ac0c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fe2a9400640 1 --2- 172.21.15.32:0/2041706341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac157590 0x7fe2ac0c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fe2a9400640 1 -- 172.21.15.32:0/2041706341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe290003a20 msgr2=0x7fe2ac0b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fc0cbe00640 1 -- 172.21.15.32:0/3730389240 >> v2:172.21.15.144:3300/0 conn(0x7fc0cc06c550 msgr2=0x7fc0cc06e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:19.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fc0cbe00640 1 --2- 172.21.15.32:0/3730389240 >> v2:172.21.15.144:3300/0 conn(0x7fc0cc06c550 0x7fc0cc06e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fc0cbe00640 1 -- 172.21.15.32:0/3730389240 >> v2:172.21.15.32:3300/0 conn(0x7fc0cc15ab70 msgr2=0x7fc0cc0b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:19.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fc0cbe00640 1 --2- 172.21.15.32:0/3730389240 >> v2:172.21.15.32:3300/0 conn(0x7fc0cc15ab70 0x7fc0cc0b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fc0cbe00640 1 -- 172.21.15.32:0/3730389240 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0cc06f5f0 con 0x7fc0cc06eab0 2024-04-17T09:53:19.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fc0d0a00640 1 --2- 172.21.15.32:0/3730389240 >> v2:172.21.15.144:3300/0 conn(0x7fc0cc06c550 0x7fc0cc06e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:19.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f017d400640 1 --2- 172.21.15.32:0/2244979789 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x7f015c070570 0x7f015c072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fe2a9400640 1 --2- 172.21.15.32:0/2041706341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe290003a20 0x7fe2ac0b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fe2a9400640 1 -- 172.21.15.32:0/2041706341 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ac1f8f70 con 0x7fe2ac1f8a00 2024-04-17T09:53:19.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7fe2a9e00640 1 --2- 172.21.15.32:0/2041706341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe290003a20 0x7fe2ac0b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:53:19.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f0177e00640 1 --2- 172.21.15.32:0/2244979789 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f015c0683d0 0x7f015c06a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f5753400640 1 --2- 172.21.15.32:0/536377384 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x7f57540ca870 0x7f57540b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7fc2f4e00640 1 -- 172.21.15.32:0/1250703951 shutdown_connections 2024-04-17T09:53:19.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7fe2a9400640 1 --2- 172.21.15.32:0/2041706341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2ac1f8a00 0x7fe2ac0c8860 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe29800d5d0 tx=0x7fe29800da90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f1357400640 1 --2- 172.21.15.32:0/2768075511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f133c004d50 0x7f133c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f1357400640 1 -- 172.21.15.32:0/2768075511 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c003530 con 0x7f133c003f90 2024-04-17T09:53:19.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.687+0000 7f1357e00640 1 --2- 172.21.15.32:0/2768075511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13580b2820 0x7f133c0039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:19.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f5752000640 1 --2- 172.21.15.32:0/536377384 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f57340683d0 0x7f573406a890 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f574c009e50 tx=0x7f574c0a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f83b5a00640 1 -- 172.21.15.32:0/4269712655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b00c5ac0 msgr2=0x7f83b0166b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f83b5a00640 1 --2- 172.21.15.32:0/4269712655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b00c5ac0 0x7f83b0166b60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f83b0073580 tx=0x7f839801ec70 comp rx=0 tx=0).stop 2024-04-17T09:53:19.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f017d400640 1 --2- 172.21.15.32:0/2244979789 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x7f015c070570 0x7f015c072990 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f1e28e00640 1 --2- 172.21.15.32:0/869297753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e24166320 0x7f1e240cb000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1e2406fce0 tx=0x7f1e1801c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71011109b6088567 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7fe2a2000640 1 -- 172.21.15.32:0/2041706341 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2980125f0 con 0x7fe2ac1f8a00 2024-04-17T09:53:19.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7fe2b1200640 1 -- 172.21.15.32:0/2041706341 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2ac1f9250 con 0x7fe2ac1f8a00 2024-04-17T09:53:19.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7fe2b1200640 1 -- 172.21.15.32:0/2041706341 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2ac1f9810 con 0x7fe2ac1f8a00 2024-04-17T09:53:19.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7fe2a2000640 1 -- 172.21.15.32:0/2041706341 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe298019070 con 0x7fe2ac1f8a00 2024-04-17T09:53:19.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7fe2a2000640 1 -- 172.21.15.32:0/2041706341 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2980129d0 con 0x7fe2ac1f8a00 2024-04-17T09:53:19.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f0177e00640 1 --2- 172.21.15.32:0/2244979789 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f015c0683d0 0x7f015c06a890 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f01780cf200 tx=0x7f016c007b10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f1e23400640 1 -- 172.21.15.32:0/869297753 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e180b1070 con 0x7f1e24166320 2024-04-17T09:53:19.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f83b5a00640 1 -- 172.21.15.32:0/4269712655 shutdown_connections 2024-04-17T09:53:19.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f5753400640 1 --2- 172.21.15.32:0/536377384 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x7f57540ca870 0x7f57540b0300 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f1e23400640 1 -- 172.21.15.32:0/869297753 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e180ac620 con 0x7f1e24166320 2024-04-17T09:53:19.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f1e23400640 1 -- 172.21.15.32:0/869297753 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e180ac9f0 con 0x7f1e24166320 2024-04-17T09:53:19.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7fc2f4e00640 1 -- 172.21.15.32:0/1250703951 wait complete. 2024-04-17T09:53:19.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f83b5a00640 1 --2- 172.21.15.32:0/4269712655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b00c5ac0 0x7f83b0166b60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f83b5a00640 1 --2- 172.21.15.32:0/4269712655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b009b400 0x7f83b00c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f83b5a00640 1 --2- 172.21.15.32:0/4269712655 >> v2:172.21.15.32:3300/0 conn(0x7f83b014ca40 0x7f83b006d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f83b5a00640 1 -- 172.21.15.32:0/4269712655 >> 172.21.15.32:0/4269712655 conn(0x7f83b00b40a0 msgr2=0x7f83b00b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:19.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f83b5a00640 1 -- 172.21.15.32:0/4269712655 shutdown_connections 2024-04-17T09:53:19.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f83b5a00640 1 -- 172.21.15.32:0/4269712655 wait complete. 2024-04-17T09:53:19.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f0174c00640 1 -- 172.21.15.32:0/2244979789 <== osd.0 v2:172.21.15.32:6800/2392056658 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f015c073040 con 0x7f015c070570 2024-04-17T09:53:19.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f574b400640 1 -- 172.21.15.32:0/536377384 <== osd.1 v2:172.21.15.32:6808/2454993455 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57540b5880 con 0x7f57540ca870 2024-04-17T09:53:19.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7fe2b1200640 1 -- 172.21.15.32:0/2041706341 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe284001040 con 0x7fe2ac1f8a00 2024-04-17T09:53:19.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f1e2a200640 1 -- 172.21.15.32:0/869297753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e24166320 msgr2=0x7f1e240cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f1e2a200640 1 --2- 172.21.15.32:0/869297753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e24166320 0x7f1e240cb000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1e2406fce0 tx=0x7f1e1801c460 comp rx=0 tx=0).stop 2024-04-17T09:53:19.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f1e2a200640 1 -- 172.21.15.32:0/869297753 shutdown_connections 2024-04-17T09:53:19.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f1e2a200640 1 --2- 172.21.15.32:0/869297753 >> v2:172.21.15.32:3300/0 conn(0x7f1e240cc080 0x7f1e2406ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f1e2a200640 1 --2- 172.21.15.32:0/869297753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e24166320 0x7f1e240cb000 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.691+0000 7f1e2a200640 1 -- 172.21.15.32:0/869297753 >> 172.21.15.32:0/869297753 conn(0x7f1e240cd430 msgr2=0x7f1e240cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:19.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7fc2f4e00640 1 Processor -- start 2024-04-17T09:53:19.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7fc2f4e00640 1 -- start start 2024-04-17T09:53:19.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f1e2a200640 1 -- 172.21.15.32:0/869297753 shutdown_connections 2024-04-17T09:53:19.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f83b5a00640 1 Processor -- start 2024-04-17T09:53:19.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f83b5a00640 1 -- start start 2024-04-17T09:53:19.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7fc2f4e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2f00b3a20 0x7fc2f00aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7fc2f4e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2f00d0270 0x7fc2f00c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7fc2f4e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2f00d0750 0x7fc2f014b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7fc2f4e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f006ee60 con 0x7fc2f00d0750 2024-04-17T09:53:19.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7fc2f4e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f00ce190 con 0x7fc2f00b3a20 2024-04-17T09:53:19.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7fc2f4e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f006efe0 con 0x7fc2f00d0270 2024-04-17T09:53:19.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f1357400640 1 --2- 172.21.15.32:0/2768075511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f133c003f90 0x7f133c004810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f13440098d0 tx=0x7f134401c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1863524cdfd38bed server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f1e2a200640 1 -- 172.21.15.32:0/869297753 wait complete. 2024-04-17T09:53:19.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f83b5a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b009b400 0x7f83b00cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f83b5a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b00c5ac0 0x7f83b01fc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f83b5a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b014ca40 0x7f83b0162650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f1356a00640 1 -- 172.21.15.32:0/2768075511 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13440ac070 con 0x7f133c003f90 2024-04-17T09:53:19.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f1356a00640 1 -- 172.21.15.32:0/2768075511 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13440a6660 con 0x7f133c003f90 2024-04-17T09:53:19.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f1356a00640 1 -- 172.21.15.32:0/2768075511 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134400f040 con 0x7f133c003f90 2024-04-17T09:53:19.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f83b5a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f83b00ccda0 con 0x7f83b009b400 2024-04-17T09:53:19.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f83b5a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f83b00cd3f0 con 0x7f83b014ca40 2024-04-17T09:53:19.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f83b5a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f83b00cd080 con 0x7f83b00c5ac0 2024-04-17T09:53:19.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7fc2ede00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2f00b3a20 0x7fc2f00aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7fc2ede00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2f00b3a20 0x7fc2f00aef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:40874/0 (socket says 172.21.15.32:40874) 2024-04-17T09:53:19.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7fc2ede00640 1 -- 172.21.15.32:0/3432004505 learned_addr learned my addr 172.21.15.32:0/3432004505 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:19.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7fc2ed400640 1 --2- 172.21.15.32:0/3432004505 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2f00d0270 0x7fc2f00c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7fe2a2000640 1 -- 172.21.15.32:0/2041706341 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe298012b70 con 0x7fe2ac1f8a00 2024-04-17T09:53:19.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f83af400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b014ca40 0x7f83b0162650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f83af400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b014ca40 0x7f83b0162650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:40878/0 (socket says 172.21.15.32:40878) 2024-04-17T09:53:19.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f83aea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b009b400 0x7f83b00cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f83af400640 1 -- 172.21.15.32:0/203301817 learned_addr learned my addr 172.21.15.32:0/203301817 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:19.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f83af400640 1 -- 172.21.15.32:0/203301817 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b00c5ac0 msgr2=0x7f83b01fc250 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-17T09:53:19.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f83af400640 1 --2- 172.21.15.32:0/203301817 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b00c5ac0 0x7f83b01fc250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f83af400640 1 -- 172.21.15.32:0/203301817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b009b400 msgr2=0x7f83b00cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f83af400640 1 --2- 172.21.15.32:0/203301817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b009b400 0x7f83b00cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f83af400640 1 -- 172.21.15.32:0/203301817 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b00b64f0 con 0x7f83b014ca40 2024-04-17T09:53:19.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f83aea00640 1 --2- 172.21.15.32:0/203301817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b009b400 0x7f83b00cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:19.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f1e2a200640 1 Processor -- start 2024-04-17T09:53:19.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7f83af400640 1 --2- 172.21.15.32:0/203301817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b014ca40 0x7f83b0162650 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f839801d320 tx=0x7f839801ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7fe2a2000640 1 --2- 172.21.15.32:0/2041706341 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe28c068450 0x7fe28c06a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7fc2ede00640 1 -- 172.21.15.32:0/3432004505 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2f00d0270 msgr2=0x7fc2f00c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7fc2ede00640 1 --2- 172.21.15.32:0/3432004505 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2f00d0270 0x7fc2f00c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7fc2ede00640 1 -- 172.21.15.32:0/3432004505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2f00d0750 msgr2=0x7fc2f014b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7fc2ede00640 1 --2- 172.21.15.32:0/3432004505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2f00d0750 0x7fc2f014b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7fc2ede00640 1 -- 172.21.15.32:0/3432004505 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f00ca1d0 con 0x7fc2f00b3a20 2024-04-17T09:53:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.695+0000 7fc2ede00640 1 --2- 172.21.15.32:0/3432004505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2f00b3a20 0x7fc2f00aef30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800cca0 tx=0x7fc2e800a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7fc2e6000640 1 -- 172.21.15.32:0/3432004505 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8012400 con 0x7fc2f00b3a20 2024-04-17T09:53:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7fc2e6000640 1 -- 172.21.15.32:0/3432004505 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e80125a0 con 0x7fc2f00b3a20 2024-04-17T09:53:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7fc2e6000640 1 -- 172.21.15.32:0/3432004505 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8012990 con 0x7fc2f00b3a20 2024-04-17T09:53:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f83a6a00640 1 -- 172.21.15.32:0/203301817 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83980b5040 con 0x7f83b014ca40 2024-04-17T09:53:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7fc0cbe00640 1 --2- 172.21.15.32:0/3730389240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0cc06eab0 0x7fc0cc15a630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc06df70 tx=0x7fc0b801ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5fba7ad418eb674 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7fe2a2000640 1 -- 172.21.15.32:0/2041706341 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe298086c30 con 0x7fe2ac1f8a00 2024-04-17T09:53:19.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f135da00640 1 -- 172.21.15.32:0/2768075511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f133c003f90 msgr2=0x7f133c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f135da00640 1 --2- 172.21.15.32:0/2768075511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f133c003f90 0x7f133c004810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f13440098d0 tx=0x7f134401c650 comp rx=0 tx=0).stop 2024-04-17T09:53:19.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f135d000640 1 -- 172.21.15.32:0/2768075511 reap_dead start 2024-04-17T09:53:19.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f83a6a00640 1 -- 172.21.15.32:0/203301817 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83980be070 con 0x7f83b014ca40 2024-04-17T09:53:19.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f83b5a00640 1 -- 172.21.15.32:0/203301817 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b00b7680 con 0x7f83b014ca40 2024-04-17T09:53:19.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f83a6a00640 1 -- 172.21.15.32:0/203301817 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83980b0900 con 0x7f83b014ca40 2024-04-17T09:53:19.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f83b5a00640 1 -- 172.21.15.32:0/203301817 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b00b7c40 con 0x7f83b014ca40 2024-04-17T09:53:19.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7fc0cb400640 1 -- 172.21.15.32:0/3730389240 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b80b5070 con 0x7fc0cc06eab0 2024-04-17T09:53:19.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7fc0cb400640 1 -- 172.21.15.32:0/3730389240 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b80a89d0 con 0x7fc0cc06eab0 2024-04-17T09:53:19.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7fc0cb400640 1 -- 172.21.15.32:0/3730389240 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b80a8da0 con 0x7fc0cc06eab0 2024-04-17T09:53:19.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7fe2a2000640 1 --2- 172.21.15.32:0/2041706341 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x7fe28c070670 0x7fe28c072a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7fe2a9e00640 1 --2- 172.21.15.32:0/2041706341 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe28c068450 0x7fe28c06a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7fc2f4e00640 1 -- 172.21.15.32:0/3432004505 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f00b04d0 con 0x7fc2f00b3a20 2024-04-17T09:53:19.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7fc2f4e00640 1 -- 172.21.15.32:0/3432004505 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f00c91b0 con 0x7fc2f00b3a20 2024-04-17T09:53:19.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f1e2a200640 1 -- start start 2024-04-17T09:53:19.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f1e2a200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e24070430 0x7f1e2415e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f1e2a200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e240cc080 0x7f1e24164330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f1e2a200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e24166320 0x7f1e240cef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f1e2a200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1e24070170 con 0x7f1e24166320 2024-04-17T09:53:19.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f1e2a200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2406fff0 con 0x7f1e240cc080 2024-04-17T09:53:19.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f1e2a200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1e2406c780 con 0x7f1e24070430 2024-04-17T09:53:19.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f1e28e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e24070430 0x7f1e2415e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f1e28e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e24070430 0x7f1e2415e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:56448/0 (socket says 172.21.15.32:56448) 2024-04-17T09:53:19.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f1e28e00640 1 -- 172.21.15.32:0/3161455522 learned_addr learned my addr 172.21.15.32:0/3161455522 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7fc0d3400640 1 -- 172.21.15.32:0/3730389240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0cc06eab0 msgr2=0x7fc0cc15a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7fc0d3400640 1 --2- 172.21.15.32:0/3730389240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0cc06eab0 0x7fc0cc15a630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc06df70 tx=0x7fc0b801ed10 comp rx=0 tx=0).stop 2024-04-17T09:53:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f135da00640 1 -- 172.21.15.32:0/2768075511 shutdown_connections 2024-04-17T09:53:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f135da00640 1 -- 172.21.15.32:0/2768075511 >> 172.21.15.32:0/2768075511 conn(0x7f13580b1d00 msgr2=0x7f13580b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f83b5a00640 1 -- 172.21.15.32:0/203301817 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f83b0068a90 con 0x7f83b014ca40 2024-04-17T09:53:19.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f135da00640 1 -- 172.21.15.32:0/2768075511 shutdown_connections 2024-04-17T09:53:19.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f135da00640 1 -- 172.21.15.32:0/2768075511 wait complete. 2024-04-17T09:53:19.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f1e28e00640 1 -- 172.21.15.32:0/3161455522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e240cc080 msgr2=0x7f1e24164330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f1e28e00640 1 --2- 172.21.15.32:0/3161455522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e240cc080 0x7f1e24164330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f1e28e00640 1 -- 172.21.15.32:0/3161455522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e24166320 msgr2=0x7f1e240cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f1e28e00640 1 --2- 172.21.15.32:0/3161455522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e24166320 0x7f1e240cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7fe2a2000640 1 -- 172.21.15.32:0/2041706341 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe28c073140 con 0x7fe28c070670 2024-04-17T09:53:19.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f1e28e00640 1 -- 172.21.15.32:0/3161455522 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e24164bb0 con 0x7f1e24070430 2024-04-17T09:53:19.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7fc2f4e00640 1 -- 172.21.15.32:0/3432004505 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc2f00c9870 con 0x7fc2f00b3a20 2024-04-17T09:53:19.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7fe2a2000640 1 -- 172.21.15.32:0/2041706341 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe298089290 con 0x7fe2ac1f8a00 2024-04-17T09:53:19.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f1e28e00640 1 --2- 172.21.15.32:0/3161455522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e24070430 0x7f1e2415e150 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1e1801a090 tx=0x7f1e240c4470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.699+0000 7f83a6a00640 1 -- 172.21.15.32:0/203301817 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f83980deab0 con 0x7f83b014ca40 2024-04-17T09:53:19.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7f1e20c00640 1 -- 172.21.15.32:0/3161455522 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e180b9020 con 0x7f1e24070430 2024-04-17T09:53:19.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7fe2a9e00640 1 --2- 172.21.15.32:0/2041706341 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe28c068450 0x7fe28c06a910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0ad000 tx=0x7fe29c004780 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7f1e20c00640 1 -- 172.21.15.32:0/3161455522 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e180b5410 con 0x7f1e24070430 2024-04-17T09:53:19.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7f1e20c00640 1 -- 172.21.15.32:0/3161455522 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e180b1040 con 0x7f1e24070430 2024-04-17T09:53:19.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7f135da00640 1 Processor -- start 2024-04-17T09:53:19.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7f83a6a00640 1 --2- 172.21.15.32:0/203301817 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f838c068450 0x7f838c06a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7f83a6a00640 1 -- 172.21.15.32:0/203301817 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8398123de0 con 0x7f83b014ca40 2024-04-17T09:53:19.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7f83a6a00640 1 --2- 172.21.15.32:0/203301817 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x7f838c070670 0x7f838c072a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7f135da00640 1 -- start start 2024-04-17T09:53:19.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7fc0d3400640 1 -- 172.21.15.32:0/3730389240 shutdown_connections 2024-04-17T09:53:19.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7fc0d3400640 1 --2- 172.21.15.32:0/3730389240 >> v2:172.21.15.32:3300/0 conn(0x7fc0cc15ab70 0x7fc0cc0b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7fc0d3400640 1 --2- 172.21.15.32:0/3730389240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0cc06eab0 0x7fc0cc15a630 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7fc0d3400640 1 --2- 172.21.15.32:0/3730389240 >> v2:172.21.15.144:3300/0 conn(0x7fc0cc06c550 0x7fc0cc06e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7fc0d3400640 1 -- 172.21.15.32:0/3730389240 >> 172.21.15.32:0/3730389240 conn(0x7fc0cc06ba30 msgr2=0x7fc0cc06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:19.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7fe2aa800640 1 --2- 172.21.15.32:0/2041706341 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x7fe28c070670 0x7fe28c072a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7fc2e6000640 1 -- 172.21.15.32:0/3432004505 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc2e8012b30 con 0x7fc2f00b3a20 2024-04-17T09:53:19.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7fc2e6000640 1 --2- 172.21.15.32:0/3432004505 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc2d0070be0 0x7fc2d00730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7fc2e6000640 1 -- 172.21.15.32:0/3432004505 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc2e8086fb0 con 0x7fc2f00b3a20 2024-04-17T09:53:19.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7fc2e6000640 1 --2- 172.21.15.32:0/3432004505 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x7fc2d0078de0 0x7fc2d007b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7f83a6a00640 1 -- 172.21.15.32:0/203301817 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f838c073140 con 0x7f838c070670 2024-04-17T09:53:19.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7f83a6a00640 1 -- 172.21.15.32:0/203301817 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f83981241d0 con 0x7f83b014ca40 2024-04-17T09:53:19.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7f83ae000640 1 --2- 172.21.15.32:0/203301817 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x7f838c070670 0x7f838c072a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7f83ae000640 1 --2- 172.21.15.32:0/203301817 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x7f838c070670 0x7f838c072a90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7fc2e6000640 1 -- 172.21.15.32:0/3432004505 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc2e8011e50 con 0x7fc2d0078de0 2024-04-17T09:53:19.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7f135da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f133c003f90 0x7f13580b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7f135da00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13580b3550 0x7f13581fc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7f135da00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13580b39d0 0x7f13581fcc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7fc0d3400640 1 -- 172.21.15.32:0/3730389240 shutdown_connections 2024-04-17T09:53:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7f135da00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1358073e20 con 0x7f133c003f90 2024-04-17T09:53:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7f135da00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f13580cb580 con 0x7f13580b39d0 2024-04-17T09:53:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7f135da00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f135815af10 con 0x7f13580b3550 2024-04-17T09:53:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7fc0d3400640 1 -- 172.21.15.32:0/3730389240 wait complete. 2024-04-17T09:53:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7fe2aa800640 1 --2- 172.21.15.32:0/2041706341 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x7fe28c070670 0x7fe28c072a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7fc2ee800640 1 --2- 172.21.15.32:0/3432004505 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x7fc2d0078de0 0x7fc2d007b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7f1357400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13580b3550 0x7f13581fc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7f1357400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13580b3550 0x7f13581fc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:56454/0 (socket says 172.21.15.32:56454) 2024-04-17T09:53:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7f1357400640 1 -- 172.21.15.32:0/1751711679 learned_addr learned my addr 172.21.15.32:0/1751711679 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7f135d000640 1 --2- 172.21.15.32:0/1751711679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13580b39d0 0x7f13581fcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7f1e2a200640 1 -- 172.21.15.32:0/3161455522 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e24164e90 con 0x7f1e24070430 2024-04-17T09:53:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.703+0000 7fc2ed400640 1 --2- 172.21.15.32:0/3432004505 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc2d0070be0 0x7fc2d00730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7fc2e6000640 1 -- 172.21.15.32:0/3432004505 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc2e808e050 con 0x7fc2f00b3a20 2024-04-17T09:53:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7f83aea00640 1 --2- 172.21.15.32:0/203301817 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f838c068450 0x7f838c06a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7fc2ee800640 1 --2- 172.21.15.32:0/3432004505 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x7fc2d0078de0 0x7fc2d007b200 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7f1e2a200640 1 -- 172.21.15.32:0/3161455522 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e24165450 con 0x7f1e24070430 2024-04-17T09:53:19.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7f1357400640 1 -- 172.21.15.32:0/1751711679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13580b39d0 msgr2=0x7f13581fcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7f1357400640 1 --2- 172.21.15.32:0/1751711679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13580b39d0 0x7f13581fcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7f1357400640 1 -- 172.21.15.32:0/1751711679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f133c003f90 msgr2=0x7f13580b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7f1357400640 1 --2- 172.21.15.32:0/1751711679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f133c003f90 0x7f13580b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7f1357400640 1 -- 172.21.15.32:0/1751711679 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1358099b20 con 0x7f13580b3550 2024-04-17T09:53:19.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7f1357400640 1 --2- 172.21.15.32:0/1751711679 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13580b3550 0x7f13581fc6d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f134401cb30 tx=0x7f13440193f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7fe2a2000640 1 -- 172.21.15.32:0/2041706341 <== osd.2 v2:172.21.15.32:6809/1954833462 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe28c073140 con 0x7fe28c070670 2024-04-17T09:53:19.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7f134fe00640 1 -- 172.21.15.32:0/1751711679 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13440ac040 con 0x7f13580b3550 2024-04-17T09:53:19.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7f134fe00640 1 -- 172.21.15.32:0/1751711679 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f134400f000 con 0x7f13580b3550 2024-04-17T09:53:19.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7f134fe00640 1 -- 172.21.15.32:0/1751711679 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13440b18a0 con 0x7f13580b3550 2024-04-17T09:53:19.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7f83a6a00640 1 -- 172.21.15.32:0/203301817 <== osd.5 v2:172.21.15.144:6800/2379578648 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f838c073140 con 0x7f838c070670 2024-04-17T09:53:19.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7fc0d3400640 1 Processor -- start 2024-04-17T09:53:19.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7fc0d3400640 1 -- start start 2024-04-17T09:53:19.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7f135da00640 1 -- 172.21.15.32:0/1751711679 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13580b2600 con 0x7f13580b3550 2024-04-17T09:53:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7fc0d3400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0cc06c550 0x7fc0cc151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7fc0d3400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0cc06eab0 0x7fc0cc147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7fc2ed400640 1 --2- 172.21.15.32:0/3432004505 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc2d0070be0 0x7fc2d00730a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0155be0 tx=0x7fc2dc007e30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7f135da00640 1 -- 172.21.15.32:0/1751711679 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f135806f480 con 0x7f13580b3550 2024-04-17T09:53:19.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7fc0d3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0cc15ab70 0x7fc0cc14b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7fc0cbe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0cc06eab0 0x7fc0cc147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7fc0cbe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0cc06eab0 0x7fc0cc147730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:56458/0 (socket says 172.21.15.32:56458) 2024-04-17T09:53:19.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7fc0cbe00640 1 -- 172.21.15.32:0/1198816772 learned_addr learned my addr 172.21.15.32:0/1198816772 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:19.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7fc0d3400640 1 -- 172.21.15.32:0/1198816772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc0b4480 con 0x7fc0cc15ab70 2024-04-17T09:53:19.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7fc0d3400640 1 -- 172.21.15.32:0/1198816772 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc0b4300 con 0x7fc0cc06c550 2024-04-17T09:53:19.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7fc0d3400640 1 -- 172.21.15.32:0/1198816772 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc0cc0b4600 con 0x7fc0cc06eab0 2024-04-17T09:53:19.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7fc0d1400640 1 --2- 172.21.15.32:0/1198816772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0cc15ab70 0x7fc0cc14b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7f83aea00640 1 --2- 172.21.15.32:0/203301817 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f838c068450 0x7f838c06a910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f83a0006fd0 tx=0x7f83a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.707+0000 7f1e20c00640 1 -- 172.21.15.32:0/3161455522 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1e180bc020 con 0x7f1e24070430 2024-04-17T09:53:19.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7f1e2a200640 1 -- 172.21.15.32:0/3161455522 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1e24165850 con 0x7f1e24070430 2024-04-17T09:53:19.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7f1e20c00640 1 --2- 172.21.15.32:0/3161455522 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1e040683d0 0x7f1e0406a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7f1e20c00640 1 -- 172.21.15.32:0/3161455522 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1e181201c0 con 0x7f1e24070430 2024-04-17T09:53:19.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7fc0cbe00640 1 -- 172.21.15.32:0/1198816772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0cc06c550 msgr2=0x7fc0cc151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7fc0cbe00640 1 --2- 172.21.15.32:0/1198816772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0cc06c550 0x7fc0cc151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7fc0cbe00640 1 -- 172.21.15.32:0/1198816772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0cc15ab70 msgr2=0x7fc0cc14b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7fc0cbe00640 1 --2- 172.21.15.32:0/1198816772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0cc15ab70 0x7fc0cc14b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7fc0cbe00640 1 -- 172.21.15.32:0/1198816772 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0cc151500 con 0x7fc0cc06eab0 2024-04-17T09:53:19.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7f1e20c00640 1 --2- 172.21.15.32:0/3161455522 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x7f1e04070570 0x7f1e04072990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7f1e20c00640 1 -- 172.21.15.32:0/3161455522 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1e18006800 con 0x7f1e04070570 2024-04-17T09:53:19.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7fc0d1400640 1 --2- 172.21.15.32:0/1198816772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0cc15ab70 0x7fc0cc14b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:53:19.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7fc0cbe00640 1 --2- 172.21.15.32:0/1198816772 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0cc06eab0 0x7fc0cc147730 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc0b801c940 tx=0x7fc0b801bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7f134d600640 1 -- 172.21.15.32:0/1751711679 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f135804fd00 con 0x7f13580b3550 2024-04-17T09:53:19.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7f1e29800640 1 --2- 172.21.15.32:0/3161455522 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x7f1e04070570 0x7f1e04072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7fc0c8c00640 1 -- 172.21.15.32:0/1198816772 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b80b5050 con 0x7fc0cc06eab0 2024-04-17T09:53:19.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7fc0c8c00640 1 -- 172.21.15.32:0/1198816772 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b80bf070 con 0x7fc0cc06eab0 2024-04-17T09:53:19.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7fc0d3400640 1 -- 172.21.15.32:0/1198816772 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0cc0cf360 con 0x7fc0cc06eab0 2024-04-17T09:53:19.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7fc0c8c00640 1 -- 172.21.15.32:0/1198816772 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b80b0c70 con 0x7fc0cc06eab0 2024-04-17T09:53:19.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7fc0d3400640 1 -- 172.21.15.32:0/1198816772 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0cc151790 con 0x7fc0cc06eab0 2024-04-17T09:53:19.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.715+0000 7fc2e6000640 1 -- 172.21.15.32:0/3432004505 <== osd.4 v2:172.21.15.144:6809/4076371964 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc2e8011e50 con 0x7fc2d0078de0 2024-04-17T09:53:19.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7f1e23e00640 1 --2- 172.21.15.32:0/3161455522 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1e040683d0 0x7f1e0406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7f1e20c00640 1 -- 172.21.15.32:0/3161455522 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1e180ec390 con 0x7f1e24070430 2024-04-17T09:53:19.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.715+0000 7f1e29800640 1 --2- 172.21.15.32:0/3161455522 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x7f1e04070570 0x7f1e04072990 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.715+0000 7fc0d3400640 1 -- 172.21.15.32:0/1198816772 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc0cc068a90 con 0x7fc0cc06eab0 2024-04-17T09:53:19.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.711+0000 7f134fe00640 1 -- 172.21.15.32:0/1751711679 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f13440af070 con 0x7f13580b3550 2024-04-17T09:53:19.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.715+0000 7f134fe00640 1 --2- 172.21.15.32:0/1751711679 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1338068400 0x7f133806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.715+0000 7f134fe00640 1 -- 172.21.15.32:0/1751711679 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1344120310 con 0x7f13580b3550 2024-04-17T09:53:19.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.715+0000 7f134fe00640 1 --2- 172.21.15.32:0/1751711679 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x7f1338070680 0x7f1338072aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.715+0000 7f134fe00640 1 -- 172.21.15.32:0/1751711679 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f13440a6660 con 0x7f1338070680 2024-04-17T09:53:19.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.715+0000 7f134fe00640 1 -- 172.21.15.32:0/1751711679 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1344125060 con 0x7f13580b3550 2024-04-17T09:53:19.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.715+0000 7f1357e00640 1 --2- 172.21.15.32:0/1751711679 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1338068400 0x7f133806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.715+0000 7fc0c8c00640 1 -- 172.21.15.32:0/1198816772 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc0b80ba440 con 0x7fc0cc06eab0 2024-04-17T09:53:19.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.715+0000 7fc0c8c00640 1 --2- 172.21.15.32:0/1198816772 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc0ac068380 0x7fc0ac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.715+0000 7f1e20c00640 1 -- 172.21.15.32:0/3161455522 <== osd.6 v2:172.21.15.144:6808/3198738696 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1e18006800 con 0x7f1e04070570 2024-04-17T09:53:19.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.715+0000 7f135d000640 1 --2- 172.21.15.32:0/1751711679 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x7f1338070680 0x7f1338072aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.715+0000 7fc0d0a00640 1 --2- 172.21.15.32:0/1198816772 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc0ac068380 0x7fc0ac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.719+0000 7f1e23e00640 1 --2- 172.21.15.32:0/3161455522 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1e040683d0 0x7f1e0406a890 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1e10000f70 tx=0x7f1e100079c0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.719+0000 7fc0d0a00640 1 --2- 172.21.15.32:0/1198816772 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc0ac068380 0x7fc0ac06a840 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4007130 tx=0x7fc0c4007300 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.719+0000 7f1357e00640 1 --2- 172.21.15.32:0/1751711679 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1338068400 0x7f133806a8c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1350007240 tx=0x7f1350009750 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.719+0000 7f135d000640 1 --2- 172.21.15.32:0/1751711679 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x7f1338070680 0x7f1338072aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.719+0000 7fc0c8c00640 1 -- 172.21.15.32:0/1198816772 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc0b8124020 con 0x7fc0cc06eab0 2024-04-17T09:53:19.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.719+0000 7fc0c8c00640 1 --2- 172.21.15.32:0/1198816772 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x7fc0ac070520 0x7fc0ac072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:19.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.719+0000 7fc0c8c00640 1 -- 172.21.15.32:0/1198816772 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc0b80b2ec0 con 0x7fc0ac070520 2024-04-17T09:53:19.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.719+0000 7fc0c8c00640 1 -- 172.21.15.32:0/1198816772 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc0b812b050 con 0x7fc0cc06eab0 2024-04-17T09:53:19.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.719+0000 7fc0d1400640 1 --2- 172.21.15.32:0/1198816772 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x7fc0ac070520 0x7fc0ac072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:19.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.719+0000 7fc0d1400640 1 --2- 172.21.15.32:0/1198816772 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x7fc0ac070520 0x7fc0ac072940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:19.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.723+0000 7fc0c8c00640 1 -- 172.21.15.32:0/1198816772 <== osd.7 v2:172.21.15.144:6824/1746570140 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc0b80b2ec0 con 0x7fc0ac070520 2024-04-17T09:53:19.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.723+0000 7f017de00640 1 -- 172.21.15.32:0/2244979789 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0154002de0 con 0x7f015c070570 2024-04-17T09:53:19.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.723+0000 7f134fe00640 1 -- 172.21.15.32:0/1751711679 <== osd.3 v2:172.21.15.32:6824/2182899298 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f13440a6660 con 0x7f1338070680 2024-04-17T09:53:19.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.723+0000 7f0174c00640 1 -- 172.21.15.32:0/2244979789 <== osd.0 v2:172.21.15.32:6800/2392056658 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0154002de0 con 0x7f015c070570 2024-04-17T09:53:19.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.727+0000 7f017de00640 1 -- 172.21.15.32:0/2244979789 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x7f015c070570 msgr2=0x7f015c072990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.723+0000 7f5759e00640 1 -- 172.21.15.32:0/536377384 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f575406eb50 con 0x7f57540ca870 2024-04-17T09:53:19.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.727+0000 7f017de00640 1 --2- 172.21.15.32:0/2244979789 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x7f015c070570 0x7f015c072990 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.727+0000 7f574b400640 1 -- 172.21.15.32:0/536377384 <== osd.1 v2:172.21.15.32:6808/2454993455 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f575406eb50 con 0x7f57540ca870 2024-04-17T09:53:19.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.727+0000 7f017de00640 1 -- 172.21.15.32:0/2244979789 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f015c0683d0 msgr2=0x7f015c06a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.727+0000 7f5759e00640 1 -- 172.21.15.32:0/536377384 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x7f57540ca870 msgr2=0x7f57540b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.727+0000 7f017de00640 1 --2- 172.21.15.32:0/2244979789 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f015c0683d0 0x7f015c06a890 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f01780cf200 tx=0x7f016c007b10 comp rx=0 tx=0).stop 2024-04-17T09:53:19.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.727+0000 7f017de00640 1 -- 172.21.15.32:0/2244979789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01780c37e0 msgr2=0x7f01780c4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.727+0000 7f017de00640 1 --2- 172.21.15.32:0/2244979789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01780c37e0 0x7f01780c4080 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f016800e9b0 tx=0x7f016800ee70 comp rx=0 tx=0).stop 2024-04-17T09:53:19.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.727+0000 7f017d400640 1 -- 172.21.15.32:0/2244979789 reap_dead start 2024-04-17T09:53:19.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.727+0000 7f017de00640 1 -- 172.21.15.32:0/2244979789 shutdown_connections 2024-04-17T09:53:19.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.727+0000 7f017de00640 1 -- 172.21.15.32:0/2244979789 >> 172.21.15.32:0/2244979789 conn(0x7f01780b1d00 msgr2=0x7f01780d0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:19.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.727+0000 7f5759e00640 1 --2- 172.21.15.32:0/536377384 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x7f57540ca870 0x7f57540b0300 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.727+0000 7f5759e00640 1 -- 172.21.15.32:0/536377384 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f57340683d0 msgr2=0x7f573406a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.727+0000 7f5759e00640 1 --2- 172.21.15.32:0/536377384 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f57340683d0 0x7f573406a890 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f574c009e50 tx=0x7f574c0a7070 comp rx=0 tx=0).stop 2024-04-17T09:53:19.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.727+0000 7f5759e00640 1 -- 172.21.15.32:0/536377384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57540ce2f0 msgr2=0x7f5754152090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.727+0000 7f5759e00640 1 --2- 172.21.15.32:0/536377384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57540ce2f0 0x7f5754152090 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5740000f70 tx=0x7f574000c890 comp rx=0 tx=0).stop 2024-04-17T09:53:19.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.727+0000 7f5753400640 1 -- 172.21.15.32:0/536377384 reap_dead start 2024-04-17T09:53:19.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.727+0000 7f5759e00640 1 -- 172.21.15.32:0/536377384 shutdown_connections 2024-04-17T09:53:19.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.727+0000 7f5759e00640 1 -- 172.21.15.32:0/536377384 >> 172.21.15.32:0/536377384 conn(0x7f57540b4120 msgr2=0x7f57540b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:19.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.727+0000 7f017de00640 1 -- 172.21.15.32:0/2244979789 shutdown_connections 2024-04-17T09:53:19.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.727+0000 7f5759e00640 1 -- 172.21.15.32:0/536377384 shutdown_connections 2024-04-17T09:53:19.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.727+0000 7f5759e00640 1 -- 172.21.15.32:0/536377384 wait complete. 2024-04-17T09:53:19.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.731+0000 7f017de00640 1 -- 172.21.15.32:0/2244979789 wait complete. 2024-04-17T09:53:19.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.731+0000 7fe2b1200640 1 -- 172.21.15.32:0/2041706341 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe284002de0 con 0x7fe28c070670 2024-04-17T09:53:19.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.731+0000 7fe2a2000640 1 -- 172.21.15.32:0/2041706341 <== osd.2 v2:172.21.15.32:6809/1954833462 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe284002de0 con 0x7fe28c070670 2024-04-17T09:53:19.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.735+0000 7fe2b1200640 1 -- 172.21.15.32:0/2041706341 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x7fe28c070670 msgr2=0x7fe28c072a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.735+0000 7fe2b1200640 1 --2- 172.21.15.32:0/2041706341 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x7fe28c070670 0x7fe28c072a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.735+0000 7fe2b1200640 1 -- 172.21.15.32:0/2041706341 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe28c068450 msgr2=0x7fe28c06a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.735+0000 7fe2b1200640 1 --2- 172.21.15.32:0/2041706341 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe28c068450 0x7fe28c06a910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0ad000 tx=0x7fe29c004780 comp rx=0 tx=0).stop 2024-04-17T09:53:19.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.735+0000 7fe2b1200640 1 -- 172.21.15.32:0/2041706341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2ac1f8a00 msgr2=0x7fe2ac0c8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.735+0000 7fe2b1200640 1 --2- 172.21.15.32:0/2041706341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2ac1f8a00 0x7fe2ac0c8860 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe29800d5d0 tx=0x7fe29800da90 comp rx=0 tx=0).stop 2024-04-17T09:53:19.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.735+0000 7fe2aa800640 1 -- 172.21.15.32:0/2041706341 reap_dead start 2024-04-17T09:53:19.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.735+0000 7fe2b1200640 1 -- 172.21.15.32:0/2041706341 shutdown_connections 2024-04-17T09:53:19.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.735+0000 7fe2b1200640 1 -- 172.21.15.32:0/2041706341 >> 172.21.15.32:0/2041706341 conn(0x7fe2ac0b2f00 msgr2=0x7fe2ac0ae480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:19.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.735+0000 7fe2b1200640 1 -- 172.21.15.32:0/2041706341 shutdown_connections 2024-04-17T09:53:19.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.735+0000 7fe2b1200640 1 -- 172.21.15.32:0/2041706341 wait complete. 2024-04-17T09:53:19.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.743+0000 7f83b5a00640 1 -- 172.21.15.32:0/203301817 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f83b01678d0 con 0x7f838c070670 2024-04-17T09:53:19.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.743+0000 7fc2f4e00640 1 -- 172.21.15.32:0/3432004505 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc2f0068a90 con 0x7fc2d0078de0 2024-04-17T09:53:19.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.743+0000 7f1e2a200640 1 -- 172.21.15.32:0/3161455522 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1e2404fd00 con 0x7f1e04070570 2024-04-17T09:53:19.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.743+0000 7f83a6a00640 1 -- 172.21.15.32:0/203301817 <== osd.5 v2:172.21.15.144:6800/2379578648 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f83b01678d0 con 0x7f838c070670 2024-04-17T09:53:19.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.743+0000 7f1e20c00640 1 -- 172.21.15.32:0/3161455522 <== osd.6 v2:172.21.15.144:6808/3198738696 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1e2404fd00 con 0x7f1e04070570 2024-04-17T09:53:19.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.743+0000 7fc2e6000640 1 -- 172.21.15.32:0/3432004505 <== osd.4 v2:172.21.15.144:6809/4076371964 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc2f0068a90 con 0x7fc2d0078de0 2024-04-17T09:53:19.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.743+0000 7f83b5a00640 1 -- 172.21.15.32:0/203301817 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x7f838c070670 msgr2=0x7f838c072a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.743+0000 7f83b5a00640 1 --2- 172.21.15.32:0/203301817 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x7f838c070670 0x7f838c072a90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.743+0000 7f83b5a00640 1 -- 172.21.15.32:0/203301817 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f838c068450 msgr2=0x7f838c06a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.743+0000 7f83b5a00640 1 --2- 172.21.15.32:0/203301817 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f838c068450 0x7f838c06a910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f83a0006fd0 tx=0x7f83a0008040 comp rx=0 tx=0).stop 2024-04-17T09:53:19.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.743+0000 7f83b5a00640 1 -- 172.21.15.32:0/203301817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b014ca40 msgr2=0x7f83b0162650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.743+0000 7f83b5a00640 1 --2- 172.21.15.32:0/203301817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b014ca40 0x7f83b0162650 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f839801d320 tx=0x7f839801ba10 comp rx=0 tx=0).stop 2024-04-17T09:53:19.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.747+0000 7f83af400640 1 -- 172.21.15.32:0/203301817 reap_dead start 2024-04-17T09:53:19.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.747+0000 7f83b5a00640 1 -- 172.21.15.32:0/203301817 shutdown_connections 2024-04-17T09:53:19.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.747+0000 7f83b5a00640 1 -- 172.21.15.32:0/203301817 >> 172.21.15.32:0/203301817 conn(0x7f83b00b40a0 msgr2=0x7f83b0166af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:19.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.747+0000 7f83b5a00640 1 -- 172.21.15.32:0/203301817 shutdown_connections 2024-04-17T09:53:19.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.747+0000 7f83b5a00640 1 -- 172.21.15.32:0/203301817 wait complete. 2024-04-17T09:53:19.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.747+0000 7f1e2a200640 1 -- 172.21.15.32:0/3161455522 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x7f1e04070570 msgr2=0x7f1e04072990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.747+0000 7f1e2a200640 1 --2- 172.21.15.32:0/3161455522 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x7f1e04070570 0x7f1e04072990 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.747+0000 7fc2f4e00640 1 -- 172.21.15.32:0/3432004505 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x7fc2d0078de0 msgr2=0x7fc2d007b200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.747+0000 7fc2f4e00640 1 --2- 172.21.15.32:0/3432004505 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x7fc2d0078de0 0x7fc2d007b200 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.747+0000 7f1e2a200640 1 -- 172.21.15.32:0/3161455522 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1e040683d0 msgr2=0x7f1e0406a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.747+0000 7f1e2a200640 1 --2- 172.21.15.32:0/3161455522 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1e040683d0 0x7f1e0406a890 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1e10000f70 tx=0x7f1e100079c0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.747+0000 7f1e2a200640 1 -- 172.21.15.32:0/3161455522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e24070430 msgr2=0x7f1e2415e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.747+0000 7f1e2a200640 1 --2- 172.21.15.32:0/3161455522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e24070430 0x7f1e2415e150 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1e1801a090 tx=0x7f1e240c4470 comp rx=0 tx=0).stop 2024-04-17T09:53:19.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.747+0000 7f1e29800640 1 -- 172.21.15.32:0/3161455522 reap_dead start 2024-04-17T09:53:19.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.747+0000 7f1e2a200640 1 -- 172.21.15.32:0/3161455522 shutdown_connections 2024-04-17T09:53:19.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.747+0000 7f1e2a200640 1 -- 172.21.15.32:0/3161455522 >> 172.21.15.32:0/3161455522 conn(0x7f1e240cd430 msgr2=0x7f1e240cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:19.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.747+0000 7f1e2a200640 1 -- 172.21.15.32:0/3161455522 shutdown_connections 2024-04-17T09:53:19.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.747+0000 7f1e2a200640 1 -- 172.21.15.32:0/3161455522 wait complete. 2024-04-17T09:53:19.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.747+0000 7fc2f4e00640 1 -- 172.21.15.32:0/3432004505 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc2d0070be0 msgr2=0x7fc2d00730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.747+0000 7fc2f4e00640 1 --2- 172.21.15.32:0/3432004505 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc2d0070be0 0x7fc2d00730a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0155be0 tx=0x7fc2dc007e30 comp rx=0 tx=0).stop 2024-04-17T09:53:19.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.747+0000 7fc2f4e00640 1 -- 172.21.15.32:0/3432004505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2f00b3a20 msgr2=0x7fc2f00aef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.747+0000 7fc2f4e00640 1 --2- 172.21.15.32:0/3432004505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2f00b3a20 0x7fc2f00aef30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800cca0 tx=0x7fc2e800a430 comp rx=0 tx=0).stop 2024-04-17T09:53:19.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.747+0000 7fc2ee800640 1 -- 172.21.15.32:0/3432004505 reap_dead start 2024-04-17T09:53:19.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.755+0000 7fc2f4e00640 1 -- 172.21.15.32:0/3432004505 shutdown_connections 2024-04-17T09:53:19.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.755+0000 7fc2f4e00640 1 -- 172.21.15.32:0/3432004505 >> 172.21.15.32:0/3432004505 conn(0x7fc2f00b2f00 msgr2=0x7fc2f00b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:19.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.755+0000 7fc0d3400640 1 -- 172.21.15.32:0/1198816772 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc0cc15e190 con 0x7fc0ac070520 2024-04-17T09:53:19.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.755+0000 7fc2f4e00640 1 -- 172.21.15.32:0/3432004505 shutdown_connections 2024-04-17T09:53:19.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.755+0000 7fc2f4e00640 1 -- 172.21.15.32:0/3432004505 wait complete. 2024-04-17T09:53:19.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.755+0000 7fc0c8c00640 1 -- 172.21.15.32:0/1198816772 <== osd.7 v2:172.21.15.144:6824/1746570140 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc0cc15e190 con 0x7fc0ac070520 2024-04-17T09:53:19.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.755+0000 7f134d600640 1 -- 172.21.15.32:0/1751711679 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f135806d770 con 0x7f1338070680 2024-04-17T09:53:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.755+0000 7fc0d3400640 1 -- 172.21.15.32:0/1198816772 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x7fc0ac070520 msgr2=0x7fc0ac072940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.755+0000 7fc0d3400640 1 --2- 172.21.15.32:0/1198816772 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x7fc0ac070520 0x7fc0ac072940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.755+0000 7fc0d3400640 1 -- 172.21.15.32:0/1198816772 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc0ac068380 msgr2=0x7fc0ac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.755+0000 7fc0d3400640 1 --2- 172.21.15.32:0/1198816772 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc0ac068380 0x7fc0ac06a840 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4007130 tx=0x7fc0c4007300 comp rx=0 tx=0).stop 2024-04-17T09:53:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.755+0000 7f134fe00640 1 -- 172.21.15.32:0/1751711679 <== osd.3 v2:172.21.15.32:6824/2182899298 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f135806d770 con 0x7f1338070680 2024-04-17T09:53:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.755+0000 7fc0d3400640 1 -- 172.21.15.32:0/1198816772 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0cc06eab0 msgr2=0x7fc0cc147730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.755+0000 7fc0d3400640 1 --2- 172.21.15.32:0/1198816772 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0cc06eab0 0x7fc0cc147730 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc0b801c940 tx=0x7fc0b801bab0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.755+0000 7fc0d1400640 1 -- 172.21.15.32:0/1198816772 reap_dead start 2024-04-17T09:53:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.755+0000 7fc0d3400640 1 -- 172.21.15.32:0/1198816772 shutdown_connections 2024-04-17T09:53:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.755+0000 7fc0d3400640 1 -- 172.21.15.32:0/1198816772 >> 172.21.15.32:0/1198816772 conn(0x7fc0cc06ba30 msgr2=0x7fc0cc15d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.759+0000 7fc0d3400640 1 -- 172.21.15.32:0/1198816772 shutdown_connections 2024-04-17T09:53:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.759+0000 7fc0d3400640 1 -- 172.21.15.32:0/1198816772 wait complete. 2024-04-17T09:53:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.759+0000 7f134d600640 1 -- 172.21.15.32:0/1751711679 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x7f1338070680 msgr2=0x7f1338072aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.759+0000 7f134d600640 1 --2- 172.21.15.32:0/1751711679 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x7f1338070680 0x7f1338072aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.759+0000 7f134d600640 1 -- 172.21.15.32:0/1751711679 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1338068400 msgr2=0x7f133806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.759+0000 7f134d600640 1 --2- 172.21.15.32:0/1751711679 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1338068400 0x7f133806a8c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1350007240 tx=0x7f1350009750 comp rx=0 tx=0).stop 2024-04-17T09:53:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.759+0000 7f134d600640 1 -- 172.21.15.32:0/1751711679 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13580b3550 msgr2=0x7f13581fc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.759+0000 7f134d600640 1 --2- 172.21.15.32:0/1751711679 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13580b3550 0x7f13581fc6d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f134401cb30 tx=0x7f13440193f0 comp rx=0 tx=0).stop 2024-04-17T09:53:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.759+0000 7f135d000640 1 -- 172.21.15.32:0/1751711679 reap_dead start 2024-04-17T09:53:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.759+0000 7f134d600640 1 -- 172.21.15.32:0/1751711679 shutdown_connections 2024-04-17T09:53:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.759+0000 7f134d600640 1 -- 172.21.15.32:0/1751711679 >> 172.21.15.32:0/1751711679 conn(0x7f13580b1d00 msgr2=0x7f13580ad650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:19.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.759+0000 7f134d600640 1 -- 172.21.15.32:0/1751711679 shutdown_connections 2024-04-17T09:53:19.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:19.759+0000 7f134d600640 1 -- 172.21.15.32:0/1751711679 wait complete. 2024-04-17T09:53:19.769 INFO:teuthology.orchestra.run.smithi032.stdout:60129542151 2024-04-17T09:53:19.770 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-17T09:53:19.771 INFO:teuthology.orchestra.run.smithi032.stdout:64424509446 2024-04-17T09:53:19.771 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-17T09:53:19.780 INFO:teuthology.orchestra.run.smithi032.stdout:64424509447 2024-04-17T09:53:19.781 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-17T09:53:19.787 INFO:teuthology.orchestra.run.smithi032.stdout:64424509447 2024-04-17T09:53:19.788 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-17T09:53:19.791 INFO:teuthology.orchestra.run.smithi032.stdout:64424509446 2024-04-17T09:53:19.791 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-17T09:53:19.798 INFO:teuthology.orchestra.run.smithi032.stdout:60129542151 2024-04-17T09:53:19.798 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-17T09:53:19.802 INFO:teuthology.orchestra.run.smithi032.stdout:64424509446 2024-04-17T09:53:19.802 INFO:teuthology.orchestra.run.smithi032.stdout:64424509446 2024-04-17T09:53:19.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-17T09:53:19.803 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-17T09:53:20.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.003+0000 7f410c600640 1 Processor -- start 2024-04-17T09:53:20.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.003+0000 7f410c600640 1 -- start start 2024-04-17T09:53:20.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.003+0000 7f410c600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4104160560 0x7f41040b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.007+0000 7f410c600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f410406c140 0x7f4104071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.007+0000 7f410c600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41040b42f0 con 0x7f41040b3d70 2024-04-17T09:53:20.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.007+0000 7f410c600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f410406e370 con 0x7f410406c140 2024-04-17T09:53:20.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.007+0000 7f410c600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f410406e5c0 con 0x7f4104160560 2024-04-17T09:53:20.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.007+0000 7fa85d000640 1 Processor -- start 2024-04-17T09:53:20.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.007+0000 7f4109c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4104160560 0x7f41040b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.007+0000 7f4109c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4104160560 0x7f41040b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:56460/0 (socket says 172.21.15.32:56460) 2024-04-17T09:53:20.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.007+0000 7f4109c00640 1 -- 172.21.15.32:0/1483928522 learned_addr learned my addr 172.21.15.32:0/1483928522 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:20.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.007+0000 7fa85d000640 1 -- start start 2024-04-17T09:53:20.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.007+0000 7f410a600640 1 --2- 172.21.15.32:0/1483928522 >> v2:172.21.15.32:3300/0 conn(0x7f410406c140 0x7f4104071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.007+0000 7f410a600640 1 -- 172.21.15.32:0/1483928522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4104160560 msgr2=0x7f41040b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:20.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.007+0000 7f410a600640 1 --2- 172.21.15.32:0/1483928522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4104160560 0x7f41040b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.007+0000 7f410a600640 1 -- 172.21.15.32:0/1483928522 >> v1:172.21.15.144:6789/0 conn(0x7f41040b3d70 legacy=0x7f410406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:20.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.007+0000 7f410a600640 1 -- 172.21.15.32:0/1483928522 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f410406e8a0 con 0x7f410406c140 2024-04-17T09:53:20.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.011+0000 7fa85d000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa85815e750 0x7fa858071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.011+0000 7fa85d000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8580afa20 0x7fa8580723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.011+0000 7fa85d000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8580728e0 0x7fa858162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.011+0000 7fa855400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8580afa20 0x7fa8580723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.011+0000 7fa855e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa85815e750 0x7fa858071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.011+0000 7fa856800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8580728e0 0x7fa858162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.011+0000 7fa855e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa85815e750 0x7fa858071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:40916/0 (socket says 172.21.15.32:40916) 2024-04-17T09:53:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.011+0000 7fa856800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8580728e0 0x7fa858162350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46968/0 (socket says 172.21.15.32:46968) 2024-04-17T09:53:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.011+0000 7fa855400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8580afa20 0x7fa8580723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:56468/0 (socket says 172.21.15.32:56468) 2024-04-17T09:53:20.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.011+0000 7fa855e00640 1 -- 172.21.15.32:0/2953625364 learned_addr learned my addr 172.21.15.32:0/2953625364 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:20.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.011+0000 7fa85d000640 1 -- 172.21.15.32:0/2953625364 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa85806e390 con 0x7fa8580728e0 2024-04-17T09:53:20.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.011+0000 7fa85d000640 1 -- 172.21.15.32:0/2953625364 --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa85806c920 con 0x7fa85815e750 2024-04-17T09:53:20.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.011+0000 7fa85d000640 1 -- 172.21.15.32:0/2953625364 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa85806ba30 con 0x7fa8580afa20 2024-04-17T09:53:20.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.011+0000 7fa855e00640 1 -- 172.21.15.32:0/2953625364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8580afa20 msgr2=0x7fa8580723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:20.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.011+0000 7fa855e00640 1 --2- 172.21.15.32:0/2953625364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8580afa20 0x7fa8580723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.011+0000 7fa855e00640 1 -- 172.21.15.32:0/2953625364 >> v2:172.21.15.32:3300/0 conn(0x7fa8580728e0 msgr2=0x7fa858162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:20.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.011+0000 7fa855e00640 1 --2- 172.21.15.32:0/2953625364 >> v2:172.21.15.32:3300/0 conn(0x7fa8580728e0 0x7fa858162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.011+0000 7fa855e00640 1 -- 172.21.15.32:0/2953625364 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa858074070 con 0x7fa85815e750 2024-04-17T09:53:20.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.015+0000 7f410a600640 1 --2- 172.21.15.32:0/1483928522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f410406c140 0x7f4104071e60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009d90 tx=0x7f40fc01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa123c079e028d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.015+0000 7fc5ce800640 1 Processor -- start 2024-04-17T09:53:20.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.015+0000 7fc5ce800640 1 -- start start 2024-04-17T09:53:20.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.015+0000 7fc5ce800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc5c815c7a0 0x7fc5c80d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.015+0000 7f4103e00640 1 -- 172.21.15.32:0/1483928522 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc0ab430 con 0x7f410406c140 2024-04-17T09:53:20.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.015+0000 7f4103e00640 1 -- 172.21.15.32:0/1483928522 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40fc0b0070 con 0x7f410406c140 2024-04-17T09:53:20.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.015+0000 7f4103e00640 1 -- 172.21.15.32:0/1483928522 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc0ab7f0 con 0x7f410406c140 2024-04-17T09:53:20.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.015+0000 7fc5ce800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5c80cda50 0x7fc5c809b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.015+0000 7fc5ce800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5c80cdfc0 con 0x7fc5c809bab0 2024-04-17T09:53:20.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.015+0000 7fc5ce800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc5c815cb80 con 0x7fc5c80cda50 2024-04-17T09:53:20.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.015+0000 7fc5ce800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc5c80ce2a0 con 0x7fc5c815c7a0 2024-04-17T09:53:20.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.015+0000 7fc5c7400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc5c815c7a0 0x7fc5c80d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7fc5c6a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5c80cda50 0x7fc5c809b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7fa855e00640 1 --2- 172.21.15.32:0/2953625364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa85815e750 0x7fa858071e60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa858071690 tx=0x7fa85001c700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e40191fab62b1dec server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7fa854a00640 1 -- 172.21.15.32:0/2953625364 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8500ac020 con 0x7fa85815e750 2024-04-17T09:53:20.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7fa854a00640 1 -- 172.21.15.32:0/2953625364 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8500a64a0 con 0x7fa85815e750 2024-04-17T09:53:20.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7fa854a00640 1 -- 172.21.15.32:0/2953625364 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8500a6870 con 0x7fa85815e750 2024-04-17T09:53:20.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7f410c600640 1 -- 172.21.15.32:0/1483928522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f410406c140 msgr2=0x7f4104071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7f410c600640 1 --2- 172.21.15.32:0/1483928522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f410406c140 0x7f4104071e60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009d90 tx=0x7f40fc01c950 comp rx=0 tx=0).stop 2024-04-17T09:53:20.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7f410c600640 1 -- 172.21.15.32:0/1483928522 shutdown_connections 2024-04-17T09:53:20.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7f410c600640 1 --2- 172.21.15.32:0/1483928522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f410406c140 0x7f4104071e60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7f410c600640 1 --2- 172.21.15.32:0/1483928522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4104160560 0x7f41040b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7f410c600640 1 -- 172.21.15.32:0/1483928522 >> 172.21.15.32:0/1483928522 conn(0x7f41040b2120 msgr2=0x7f41040b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:20.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7fc5c6a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5c80cda50 0x7fc5c809b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46978/0 (socket says 172.21.15.32:46978) 2024-04-17T09:53:20.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7fc5c6a00640 1 -- 172.21.15.32:0/2154503630 learned_addr learned my addr 172.21.15.32:0/2154503630 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:20.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7fc5c7400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc5c815c7a0 0x7fc5c80d0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:40922/0 (socket says 172.21.15.32:40922) 2024-04-17T09:53:20.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7fa85d000640 1 -- 172.21.15.32:0/2953625364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa85815e750 msgr2=0x7fa858071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7fa85d000640 1 --2- 172.21.15.32:0/2953625364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa85815e750 0x7fa858071e60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa858071690 tx=0x7fa85001c700 comp rx=0 tx=0).stop 2024-04-17T09:53:20.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7fc5c6a00640 1 -- 172.21.15.32:0/2154503630 >> v2:172.21.15.144:3300/0 conn(0x7fc5c815c7a0 msgr2=0x7fc5c80d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:20.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7fc5c6a00640 1 --2- 172.21.15.32:0/2154503630 >> v2:172.21.15.144:3300/0 conn(0x7fc5c815c7a0 0x7fc5c80d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7fc5c6a00640 1 -- 172.21.15.32:0/2154503630 >> v1:172.21.15.32:6789/0 conn(0x7fc5c809bab0 legacy=0x7fc5c80b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7fc5c6000640 1 -- 172.21.15.32:0/2154503630 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1436478235 0 0) 0x7fc5c80cdfc0 con 0x7fc5c809bab0 2024-04-17T09:53:20.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7fc5c6a00640 1 -- 172.21.15.32:0/2154503630 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c809aa90 con 0x7fc5c80cda50 2024-04-17T09:53:20.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7f410c600640 1 -- 172.21.15.32:0/1483928522 shutdown_connections 2024-04-17T09:53:20.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7fa85d000640 1 -- 172.21.15.32:0/2953625364 shutdown_connections 2024-04-17T09:53:20.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7fa85d000640 1 --2- 172.21.15.32:0/2953625364 >> v2:172.21.15.32:3300/0 conn(0x7fa8580728e0 0x7fa858162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7fa85d000640 1 --2- 172.21.15.32:0/2953625364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8580afa20 0x7fa8580723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7fa85d000640 1 --2- 172.21.15.32:0/2953625364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa85815e750 0x7fa858071e60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7fa85d000640 1 -- 172.21.15.32:0/2953625364 >> 172.21.15.32:0/2953625364 conn(0x7fa8580b2120 msgr2=0x7fa8580b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:20.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7fa85d000640 1 -- 172.21.15.32:0/2953625364 shutdown_connections 2024-04-17T09:53:20.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7f410c600640 1 -- 172.21.15.32:0/1483928522 wait complete. 2024-04-17T09:53:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.019+0000 7fa85d000640 1 -- 172.21.15.32:0/2953625364 wait complete. 2024-04-17T09:53:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.023+0000 7fa85d000640 1 Processor -- start 2024-04-17T09:53:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.023+0000 7f410c600640 1 Processor -- start 2024-04-17T09:53:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.023+0000 7fa85d000640 1 -- start start 2024-04-17T09:53:20.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.023+0000 7f410c600640 1 -- start start 2024-04-17T09:53:20.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.023+0000 7fc5c6a00640 1 --2- 172.21.15.32:0/2154503630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5c80cda50 0x7fc5c809b570 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc5c806ca10 tx=0x7fc5c0020d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfcce6040af240db server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7fc5c6000640 1 -- 172.21.15.32:0/2154503630 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c00090d0 con 0x7fc5c80cda50 2024-04-17T09:53:20.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7fc5c6000640 1 -- 172.21.15.32:0/2154503630 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5c00a9c20 con 0x7fc5c80cda50 2024-04-17T09:53:20.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7fc5c6000640 1 -- 172.21.15.32:0/2154503630 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0009270 con 0x7fc5c80cda50 2024-04-17T09:53:20.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.023+0000 7fa85d000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8580728e0 0x7fa858207740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f410c600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f410406c140 0x7f410415d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7fa85d000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8580afa20 0x7fa858151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7fa855e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8580728e0 0x7fa858207740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7fa85d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa85815e750 0x7fa8580ac560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7fa855e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8580728e0 0x7fa858207740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:56476/0 (socket says 172.21.15.32:56476) 2024-04-17T09:53:20.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7fa855400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8580afa20 0x7fa858151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7fa855400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8580afa20 0x7fa858151c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:40924/0 (socket says 172.21.15.32:40924) 2024-04-17T09:53:20.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7fa855e00640 1 -- 172.21.15.32:0/2786213255 learned_addr learned my addr 172.21.15.32:0/2786213255 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:20.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7fa85d000640 1 -- 172.21.15.32:0/2786213255 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa85806c5b0 con 0x7fa85815e750 2024-04-17T09:53:20.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7fa85d000640 1 -- 172.21.15.32:0/2786213255 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa858072cf0 con 0x7fa8580afa20 2024-04-17T09:53:20.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7fa85d000640 1 -- 172.21.15.32:0/2786213255 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa85815eb30 con 0x7fa8580728e0 2024-04-17T09:53:20.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f410c600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41040b3d70 0x7f410415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f4109200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41040b3d70 0x7f410415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f3c20800640 1 Processor -- start 2024-04-17T09:53:20.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f4109c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f410406c140 0x7f410415d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f4109c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f410406c140 0x7f410415d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:56488/0 (socket says 172.21.15.32:56488) 2024-04-17T09:53:20.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f3c20800640 1 -- start start 2024-04-17T09:53:20.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f4109c00640 1 -- 172.21.15.32:0/2629817355 learned_addr learned my addr 172.21.15.32:0/2629817355 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:20.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f410c600640 1 --2- 172.21.15.32:0/2629817355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4104160560 0x7f410414e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7fa855e00640 1 -- 172.21.15.32:0/2786213255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8580afa20 msgr2=0x7fa858151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7fa855e00640 1 --2- 172.21.15.32:0/2786213255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8580afa20 0x7fa858151c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7fa855e00640 1 -- 172.21.15.32:0/2786213255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa85815e750 msgr2=0x7fa8580ac560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7fa855e00640 1 --2- 172.21.15.32:0/2786213255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa85815e750 0x7fa8580ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f410c600640 1 -- 172.21.15.32:0/2629817355 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f410416fa40 con 0x7f41040b3d70 2024-04-17T09:53:20.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f410c600640 1 -- 172.21.15.32:0/2629817355 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f410406f2d0 con 0x7f4104160560 2024-04-17T09:53:20.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f410c600640 1 -- 172.21.15.32:0/2629817355 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f410416fbc0 con 0x7f410406c140 2024-04-17T09:53:20.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f410a600640 1 --2- 172.21.15.32:0/2629817355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4104160560 0x7f410414e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f4109c00640 1 -- 172.21.15.32:0/2629817355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4104160560 msgr2=0x7f410414e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f4109c00640 1 --2- 172.21.15.32:0/2629817355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4104160560 0x7f410414e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f4109c00640 1 -- 172.21.15.32:0/2629817355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41040b3d70 msgr2=0x7f410415a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f4109c00640 1 --2- 172.21.15.32:0/2629817355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41040b3d70 0x7f410415a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f4109c00640 1 -- 172.21.15.32:0/2629817355 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f410415aaf0 con 0x7f410406c140 2024-04-17T09:53:20.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f410a600640 1 --2- 172.21.15.32:0/2629817355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4104160560 0x7f410414e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:20.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f4109c00640 1 --2- 172.21.15.32:0/2629817355 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f410406c140 0x7f410415d230 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f40f800d7d0 tx=0x7f40f800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f4101600640 1 -- 172.21.15.32:0/2629817355 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f8018020 con 0x7f410406c140 2024-04-17T09:53:20.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f4101600640 1 -- 172.21.15.32:0/2629817355 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f80096a0 con 0x7f410406c140 2024-04-17T09:53:20.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f410c600640 1 -- 172.21.15.32:0/2629817355 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f410415add0 con 0x7f410406c140 2024-04-17T09:53:20.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7fa855e00640 1 -- 172.21.15.32:0/2786213255 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa858152560 con 0x7fa8580728e0 2024-04-17T09:53:20.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7fa855400640 1 --2- 172.21.15.32:0/2786213255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8580afa20 0x7fa858151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:20.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f3c20800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3c1815e750 0x7f3c18071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f3c20800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c18074070 con 0x7f3c180afa20 2024-04-17T09:53:20.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f3c20800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c1806ba30 con 0x7f3c180addf0 2024-04-17T09:53:20.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f3c20800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3c1806e390 con 0x7f3c1815e750 2024-04-17T09:53:20.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7fc5ce800640 1 -- 172.21.15.32:0/2154503630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5c80cda50 msgr2=0x7fc5c809b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7fc5ce800640 1 --2- 172.21.15.32:0/2154503630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5c80cda50 0x7fc5c809b570 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc5c806ca10 tx=0x7fc5c0020d30 comp rx=0 tx=0).stop 2024-04-17T09:53:20.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f4101600640 1 -- 172.21.15.32:0/2629817355 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f8009a70 con 0x7f410406c140 2024-04-17T09:53:20.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.027+0000 7f410c600640 1 -- 172.21.15.32:0/2629817355 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f410415b390 con 0x7f410406c140 2024-04-17T09:53:20.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.031+0000 7fa855e00640 1 --2- 172.21.15.32:0/2786213255 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8580728e0 0x7fa858207740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa8500ab040 tx=0x7fa8500194a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.031+0000 7f3c1e800640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f3c180addf0 0x7f3c180ae280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:34444/0 (socket says 172.21.15.32:34444) 2024-04-17T09:53:20.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.031+0000 7f3c1e800640 1 -- 172.21.15.32:0/2973134403 learned_addr learned my addr 172.21.15.32:0/2973134403 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:20.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.031+0000 7f3c1de00640 1 --2- 172.21.15.32:0/2973134403 >> v2:172.21.15.32:3300/0 conn(0x7f3c1815e750 0x7f3c18071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.031+0000 7f410c600640 1 -- 172.21.15.32:0/2629817355 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f410404fd00 con 0x7f410406c140 2024-04-17T09:53:20.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.031+0000 7f3c1ca00640 1 -- 172.21.15.32:0/2973134403 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1744357078 0 0) 0x7f3c1806ba30 con 0x7f3c180addf0 2024-04-17T09:53:20.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.031+0000 7f3c1ca00640 1 -- 172.21.15.32:0/2973134403 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c00003530 con 0x7f3c180addf0 2024-04-17T09:53:20.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.031+0000 7fa84e000640 1 -- 172.21.15.32:0/2786213255 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8500b7070 con 0x7fa8580728e0 2024-04-17T09:53:20.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.031+0000 7fa84e000640 1 -- 172.21.15.32:0/2786213255 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8500a76a0 con 0x7fa8580728e0 2024-04-17T09:53:20.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.031+0000 7fa84e000640 1 -- 172.21.15.32:0/2786213255 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8500a7a90 con 0x7fa8580728e0 2024-04-17T09:53:20.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.031+0000 7fa85d000640 1 -- 172.21.15.32:0/2786213255 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa858152840 con 0x7fa8580728e0 2024-04-17T09:53:20.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.031+0000 7fa85d000640 1 -- 172.21.15.32:0/2786213255 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa858152e00 con 0x7fa8580728e0 2024-04-17T09:53:20.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.031+0000 7f3c1de00640 1 -- 172.21.15.32:0/2973134403 >> v1:172.21.15.144:6789/0 conn(0x7f3c180addf0 legacy=0x7f3c180ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.031+0000 7f3c1de00640 1 -- 172.21.15.32:0/2973134403 >> v1:172.21.15.32:6789/0 conn(0x7f3c180afa20 legacy=0x7f3c180723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.031+0000 7f3c1de00640 1 -- 172.21.15.32:0/2973134403 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1806bd10 con 0x7f3c1815e750 2024-04-17T09:53:20.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.031+0000 7f4101600640 1 -- 172.21.15.32:0/2629817355 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f40f801b020 con 0x7f410406c140 2024-04-17T09:53:20.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.031+0000 7fa85d000640 1 -- 172.21.15.32:0/2786213255 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8300052a0 con 0x7fa8580728e0 2024-04-17T09:53:20.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.035+0000 7f4101600640 1 --2- 172.21.15.32:0/2629817355 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40e4068450 0x7f40e406a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.035+0000 7fa84e000640 1 -- 172.21.15.32:0/2786213255 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa8500c2060 con 0x7fa8580728e0 2024-04-17T09:53:20.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.035+0000 7fa84e000640 1 --2- 172.21.15.32:0/2786213255 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa8380683d0 0x7fa83806a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.035+0000 7fa84e000640 1 -- 172.21.15.32:0/2786213255 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa850120b00 con 0x7fa8580728e0 2024-04-17T09:53:20.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.035+0000 7f4101600640 1 -- 172.21.15.32:0/2629817355 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f40f8086f60 con 0x7f410406c140 2024-04-17T09:53:20.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.035+0000 7fa855400640 1 --2- 172.21.15.32:0/2786213255 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa8380683d0 0x7fa83806a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.035+0000 7f4109200640 1 --2- 172.21.15.32:0/2629817355 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40e4068450 0x7f40e406a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.035+0000 7f3c1de00640 1 --2- 172.21.15.32:0/2973134403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c1815e750 0x7f3c18071e60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0097c0 tx=0x7f3c0c01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffa605d5db015fb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.035+0000 7fc5ce800640 1 -- 172.21.15.32:0/2154503630 shutdown_connections 2024-04-17T09:53:20.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.035+0000 7f3c1ca00640 1 -- 172.21.15.32:0/2973134403 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c0b1070 con 0x7f3c1815e750 2024-04-17T09:53:20.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.035+0000 7f3c1ca00640 1 -- 172.21.15.32:0/2973134403 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c0c0ac410 con 0x7f3c1815e750 2024-04-17T09:53:20.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.035+0000 7f3c1ca00640 1 -- 172.21.15.32:0/2973134403 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c0ac7e0 con 0x7f3c1815e750 2024-04-17T09:53:20.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.035+0000 7f4109200640 1 --2- 172.21.15.32:0/2629817355 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40e4068450 0x7f40e406a910 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f41041676b0 tx=0x7f40f0007e40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.035+0000 7fa855400640 1 --2- 172.21.15.32:0/2786213255 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa8380683d0 0x7fa83806a890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa858150cc0 tx=0x7fa848008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.035+0000 7fc5ce800640 1 --2- 172.21.15.32:0/2154503630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5c80cda50 0x7fc5c809b570 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.035+0000 7fc5ce800640 1 --2- 172.21.15.32:0/2154503630 >> v2:172.21.15.144:3300/0 conn(0x7fc5c815c7a0 0x7fc5c80d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.035+0000 7fc5ce800640 1 -- 172.21.15.32:0/2154503630 >> 172.21.15.32:0/2154503630 conn(0x7fc5c80b3f90 msgr2=0x7fc5c80b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.035+0000 7fc5ce800640 1 -- 172.21.15.32:0/2154503630 shutdown_connections 2024-04-17T09:53:20.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.035+0000 7fc5ce800640 1 -- 172.21.15.32:0/2154503630 wait complete. 2024-04-17T09:53:20.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.039+0000 7f3c20800640 1 -- 172.21.15.32:0/2973134403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c1815e750 msgr2=0x7f3c18071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.039+0000 7f3c20800640 1 --2- 172.21.15.32:0/2973134403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c1815e750 0x7f3c18071e60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0097c0 tx=0x7f3c0c01c4c0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.039+0000 7f3c20800640 1 -- 172.21.15.32:0/2973134403 shutdown_connections 2024-04-17T09:53:20.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.039+0000 7f3c20800640 1 --2- 172.21.15.32:0/2973134403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c1815e750 0x7f3c18071e60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.039+0000 7f3c20800640 1 -- 172.21.15.32:0/2973134403 >> 172.21.15.32:0/2973134403 conn(0x7f3c180b2120 msgr2=0x7f3c180b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:20.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.039+0000 7f3c20800640 1 -- 172.21.15.32:0/2973134403 shutdown_connections 2024-04-17T09:53:20.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.039+0000 7f3c20800640 1 -- 172.21.15.32:0/2973134403 wait complete. 2024-04-17T09:53:20.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.039+0000 7f4101600640 1 -- 172.21.15.32:0/2629817355 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f40f8022020 con 0x7f410406c140 2024-04-17T09:53:20.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.039+0000 7fc5ce800640 1 Processor -- start 2024-04-17T09:53:20.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.039+0000 7f3c20800640 1 Processor -- start 2024-04-17T09:53:20.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.039+0000 7f3c20800640 1 -- start start 2024-04-17T09:53:20.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7f3c20800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c180addf0 0x7f3c1813dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7f3c20800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c180afa20 0x7f3c18157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7f3c20800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c1814de00 0x7f3c1814fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7f3c20800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1815ecb0 con 0x7f3c180addf0 2024-04-17T09:53:20.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7f3c20800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1815eb30 con 0x7f3c180afa20 2024-04-17T09:53:20.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7f3c20800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3c180aeda0 con 0x7f3c1814de00 2024-04-17T09:53:20.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7f3c1d400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c180afa20 0x7f3c18157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7f3c1de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c180addf0 0x7f3c1813dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7f3c1d400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c180afa20 0x7f3c18157490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:40936/0 (socket says 172.21.15.32:40936) 2024-04-17T09:53:20.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7f3c1d400640 1 -- 172.21.15.32:0/4191672350 learned_addr learned my addr 172.21.15.32:0/4191672350 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:20.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7f3c1e800640 1 --2- 172.21.15.32:0/4191672350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c1814de00 0x7f3c1814fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.039+0000 7fc5ce800640 1 -- start start 2024-04-17T09:53:20.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7f3c1d400640 1 -- 172.21.15.32:0/4191672350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c1814de00 msgr2=0x7f3c1814fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7f3c1d400640 1 --2- 172.21.15.32:0/4191672350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c1814de00 0x7f3c1814fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7f3c1d400640 1 -- 172.21.15.32:0/4191672350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c180addf0 msgr2=0x7f3c1813dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7f3c1d400640 1 --2- 172.21.15.32:0/4191672350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c180addf0 0x7f3c1813dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7f3c1d400640 1 -- 172.21.15.32:0/4191672350 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1814cd60 con 0x7f3c180afa20 2024-04-17T09:53:20.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7f3c1d400640 1 --2- 172.21.15.32:0/4191672350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c180afa20 0x7f3c18157490 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800c3a0 tx=0x7f3c0800c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7fa84e000640 1 -- 172.21.15.32:0/2786213255 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa8500165e0 con 0x7fa8580728e0 2024-04-17T09:53:20.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7f3c12000640 1 -- 172.21.15.32:0/4191672350 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c080052d0 con 0x7f3c180afa20 2024-04-17T09:53:20.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7f3c12000640 1 -- 172.21.15.32:0/4191672350 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c08005470 con 0x7f3c180afa20 2024-04-17T09:53:20.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7f3c20800640 1 -- 172.21.15.32:0/4191672350 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1814d040 con 0x7f3c180afa20 2024-04-17T09:53:20.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7f3c20800640 1 -- 172.21.15.32:0/4191672350 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c1814d600 con 0x7f3c180afa20 2024-04-17T09:53:20.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7fc5ce800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5c809bab0 0x7fc5c814ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7fc5ce800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5c80cda50 0x7fc5c815cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7fc5ce800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5c815c7a0 0x7fc5c8153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7fc5ce800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c80b5b80 con 0x7fc5c80cda50 2024-04-17T09:53:20.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7fc5ce800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c80b5a00 con 0x7fc5c809bab0 2024-04-17T09:53:20.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7fc5ce800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c81675c0 con 0x7fc5c815c7a0 2024-04-17T09:53:20.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7fc5c7400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5c809bab0 0x7fc5c814ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7fc5c7400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5c809bab0 0x7fc5c814ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:40948/0 (socket says 172.21.15.32:40948) 2024-04-17T09:53:20.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7fc5c7400640 1 -- 172.21.15.32:0/2062583824 learned_addr learned my addr 172.21.15.32:0/2062583824 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:20.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.043+0000 7f3c12000640 1 -- 172.21.15.32:0/4191672350 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08007240 con 0x7f3c180afa20 2024-04-17T09:53:20.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.047+0000 7fc5c7e00640 1 --2- 172.21.15.32:0/2062583824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5c815c7a0 0x7fc5c8153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.047+0000 7fc5c7400640 1 -- 172.21.15.32:0/2062583824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5c815c7a0 msgr2=0x7fc5c8153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.047+0000 7fc5c7400640 1 --2- 172.21.15.32:0/2062583824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5c815c7a0 0x7fc5c8153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.047+0000 7fc5c6a00640 1 --2- 172.21.15.32:0/2062583824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5c80cda50 0x7fc5c815cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.047+0000 7fc5c7400640 1 -- 172.21.15.32:0/2062583824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5c80cda50 msgr2=0x7fc5c815cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.047+0000 7fc5c7400640 1 --2- 172.21.15.32:0/2062583824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5c80cda50 0x7fc5c815cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.047+0000 7fc5c7400640 1 -- 172.21.15.32:0/2062583824 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c815d5d0 con 0x7fc5c809bab0 2024-04-17T09:53:20.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.047+0000 7f3c07400640 1 -- 172.21.15.32:0/4191672350 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c1804fd00 con 0x7f3c180afa20 2024-04-17T09:53:20.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.047+0000 7fc5c7e00640 1 --2- 172.21.15.32:0/2062583824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5c815c7a0 0x7fc5c8153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:20.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.047+0000 7fc5c7400640 1 --2- 172.21.15.32:0/2062583824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5c809bab0 0x7fc5c814ea40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00d830 tx=0x7fc5bc00dcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.047+0000 7fc5c6a00640 1 --2- 172.21.15.32:0/2062583824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5c80cda50 0x7fc5c815cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:20.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.047+0000 7fc5bb400640 1 -- 172.21.15.32:0/2062583824 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc00b6c0 con 0x7fc5c809bab0 2024-04-17T09:53:20.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.047+0000 7fc5bb400640 1 -- 172.21.15.32:0/2062583824 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5bc019070 con 0x7fc5c809bab0 2024-04-17T09:53:20.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.047+0000 7fc5bb400640 1 -- 172.21.15.32:0/2062583824 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc00ba60 con 0x7fc5c809bab0 2024-04-17T09:53:20.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.047+0000 7f3c12000640 1 -- 172.21.15.32:0/4191672350 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3c08007420 con 0x7f3c180afa20 2024-04-17T09:53:20.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.047+0000 7f3c12000640 1 --2- 172.21.15.32:0/4191672350 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3bfc0683d0 0x7f3bfc06a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.047+0000 7f3c12000640 1 -- 172.21.15.32:0/4191672350 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3c08085dc0 con 0x7f3c180afa20 2024-04-17T09:53:20.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.051+0000 7fc5ce800640 1 -- 172.21.15.32:0/2062583824 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c815d8b0 con 0x7fc5c809bab0 2024-04-17T09:53:20.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.051+0000 7fc5ce800640 1 -- 172.21.15.32:0/2062583824 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c815de70 con 0x7fc5c809bab0 2024-04-17T09:53:20.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.051+0000 7fc5ce800640 1 -- 172.21.15.32:0/2062583824 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c804fd00 con 0x7fc5c809bab0 2024-04-17T09:53:20.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.051+0000 7f3c1de00640 1 --2- 172.21.15.32:0/4191672350 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3bfc0683d0 0x7f3bfc06a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.051+0000 7f3c1de00640 1 --2- 172.21.15.32:0/4191672350 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3bfc0683d0 0x7f3bfc06a890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0ae000 tx=0x7f3c0c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.051+0000 7f83c3e00640 1 Processor -- start 2024-04-17T09:53:20.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.051+0000 7fd623000640 1 Processor -- start 2024-04-17T09:53:20.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.051+0000 7f83c3e00640 1 -- start start 2024-04-17T09:53:20.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7f83c3e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f83c4149110 0x7f83c4149840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7f83c3e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83c40b1c60 0x7f83c40b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7f83c3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83c40b2830 0x7f83c40b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7f83c3e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f83c40b2600 con 0x7f83c40b2830 2024-04-17T09:53:20.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7f83c3e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f83c40cdcf0 con 0x7f83c4149110 2024-04-17T09:53:20.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7f83c3e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f83c40cd120 con 0x7f83c40b1c60 2024-04-17T09:53:20.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7f3c12000640 1 -- 172.21.15.32:0/4191672350 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3c08055d10 con 0x7f3c180afa20 2024-04-17T09:53:20.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7fc5bb400640 1 -- 172.21.15.32:0/2062583824 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc5bc01e460 con 0x7fc5c809bab0 2024-04-17T09:53:20.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.051+0000 7fd623000640 1 -- start start 2024-04-17T09:53:20.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7fd623000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd61c162da0 0x7fd61c06f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7fd623000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd61c163fa0 0x7fd61c0c4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7fd623000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd61c06fdf0 0x7fd61c0c4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7fd623000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd61c163b10 con 0x7fd61c162da0 2024-04-17T09:53:20.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7fd623000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd61c070350 con 0x7fd61c06fdf0 2024-04-17T09:53:20.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7fd623000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd61c1634c0 con 0x7fd61c163fa0 2024-04-17T09:53:20.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7f83c2000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83c40b1c60 0x7f83c40b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7f83c2000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83c40b1c60 0x7f83c40b20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:56496/0 (socket says 172.21.15.32:56496) 2024-04-17T09:53:20.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7f83c2000640 1 -- 172.21.15.32:0/458746695 learned_addr learned my addr 172.21.15.32:0/458746695 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:20.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7f83c2a00640 1 --2- 172.21.15.32:0/458746695 >> v2:172.21.15.144:3300/0 conn(0x7f83c4149110 0x7f83c4149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7f83c2000640 1 -- 172.21.15.32:0/458746695 >> v2:172.21.15.144:3300/0 conn(0x7f83c4149110 msgr2=0x7f83c4149840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:20.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7f83c2000640 1 --2- 172.21.15.32:0/458746695 >> v2:172.21.15.144:3300/0 conn(0x7f83c4149110 0x7f83c4149840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7f83c2000640 1 -- 172.21.15.32:0/458746695 >> v2:172.21.15.32:3300/0 conn(0x7f83c40b2830 msgr2=0x7f83c40b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:20.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7f83c2000640 1 --2- 172.21.15.32:0/458746695 >> v2:172.21.15.32:3300/0 conn(0x7f83c40b2830 0x7f83c40b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7f83c2000640 1 -- 172.21.15.32:0/458746695 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c40cd3c0 con 0x7f83c40b1c60 2024-04-17T09:53:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7fd621c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd61c162da0 0x7fd61c06f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7fd622600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd61c06fdf0 0x7fd61c0c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7fd622600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd61c06fdf0 0x7fd61c0c4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:40968/0 (socket says 172.21.15.32:40968) 2024-04-17T09:53:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7fd622600640 1 -- 172.21.15.32:0/1707186975 learned_addr learned my addr 172.21.15.32:0/1707186975 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7fd621200640 1 --2- 172.21.15.32:0/1707186975 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd61c163fa0 0x7fd61c0c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.059+0000 7fd622600640 1 -- 172.21.15.32:0/1707186975 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd61c163fa0 msgr2=0x7fd61c0c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.059+0000 7fd622600640 1 --2- 172.21.15.32:0/1707186975 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd61c163fa0 0x7fd61c0c4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.059+0000 7fd622600640 1 -- 172.21.15.32:0/1707186975 >> v2:172.21.15.32:3300/0 conn(0x7fd61c162da0 msgr2=0x7fd61c06f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.059+0000 7fd622600640 1 --2- 172.21.15.32:0/1707186975 >> v2:172.21.15.32:3300/0 conn(0x7fd61c162da0 0x7fd61c06f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.059+0000 7fd622600640 1 -- 172.21.15.32:0/1707186975 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61c1637a0 con 0x7fd61c06fdf0 2024-04-17T09:53:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7fc5bb400640 1 --2- 172.21.15.32:0/2062583824 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc5a8068380 0x7fc5a806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.055+0000 7fc5bb400640 1 -- 172.21.15.32:0/2062583824 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc5bc087ba0 con 0x7fc5c809bab0 2024-04-17T09:53:20.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.059+0000 7fc5c6a00640 1 --2- 172.21.15.32:0/2062583824 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc5a8068380 0x7fc5a806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.059+0000 7fc5c6a00640 1 --2- 172.21.15.32:0/2062583824 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc5a8068380 0x7fc5a806a840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00b2000 tx=0x7fc5c00ab070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.059+0000 7fc5bb400640 1 -- 172.21.15.32:0/2062583824 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc5bc087e20 con 0x7fc5c809bab0 2024-04-17T09:53:20.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.059+0000 7f83c2000640 1 --2- 172.21.15.32:0/458746695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83c40b1c60 0x7f83c40b20c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f83c40ce0e0 tx=0x7f83b401c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a76e16bebb78c77f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.063+0000 7f83c1600640 1 -- 172.21.15.32:0/458746695 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b40aa610 con 0x7f83c40b1c60 2024-04-17T09:53:20.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.063+0000 7f83c1600640 1 -- 172.21.15.32:0/458746695 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b40af070 con 0x7f83c40b1c60 2024-04-17T09:53:20.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.063+0000 7f83c1600640 1 -- 172.21.15.32:0/458746695 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b40aa9d0 con 0x7f83c40b1c60 2024-04-17T09:53:20.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.063+0000 7fd622600640 1 --2- 172.21.15.32:0/1707186975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd61c06fdf0 0x7fd61c0c4750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd61c06daf0 tx=0x7fd61401c980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=becb4ada2e14438f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.063+0000 7f83c3e00640 1 -- 172.21.15.32:0/458746695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83c40b1c60 msgr2=0x7f83c40b20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.063+0000 7f83c3e00640 1 --2- 172.21.15.32:0/458746695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83c40b1c60 0x7f83c40b20c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f83c40ce0e0 tx=0x7f83b401c4f0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.063+0000 7f83c3e00640 1 -- 172.21.15.32:0/458746695 shutdown_connections 2024-04-17T09:53:20.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.063+0000 7f83c3e00640 1 --2- 172.21.15.32:0/458746695 >> v2:172.21.15.32:3300/0 conn(0x7f83c40b2830 0x7f83c40b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.063+0000 7f83c3e00640 1 --2- 172.21.15.32:0/458746695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83c40b1c60 0x7f83c40b20c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.063+0000 7f83c3e00640 1 --2- 172.21.15.32:0/458746695 >> v2:172.21.15.144:3300/0 conn(0x7f83c4149110 0x7f83c4149840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.063+0000 7f83c3e00640 1 -- 172.21.15.32:0/458746695 >> 172.21.15.32:0/458746695 conn(0x7f83c4067940 msgr2=0x7f83c40b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.063+0000 7f83c3e00640 1 -- 172.21.15.32:0/458746695 shutdown_connections 2024-04-17T09:53:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.063+0000 7fd61be00640 1 -- 172.21.15.32:0/1707186975 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6140a5600 con 0x7fd61c06fdf0 2024-04-17T09:53:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.063+0000 7fd61be00640 1 -- 172.21.15.32:0/1707186975 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6140a57a0 con 0x7fd61c06fdf0 2024-04-17T09:53:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.063+0000 7fd61be00640 1 -- 172.21.15.32:0/1707186975 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6140a5600 con 0x7fd61c06fdf0 2024-04-17T09:53:20.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.067+0000 7f83c3e00640 1 -- 172.21.15.32:0/458746695 wait complete. 2024-04-17T09:53:20.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.067+0000 7fd623000640 1 -- 172.21.15.32:0/1707186975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd61c06fdf0 msgr2=0x7fd61c0c4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.067+0000 7fd623000640 1 --2- 172.21.15.32:0/1707186975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd61c06fdf0 0x7fd61c0c4750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd61c06daf0 tx=0x7fd61401c980 comp rx=0 tx=0).stop 2024-04-17T09:53:20.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.067+0000 7fd623000640 1 -- 172.21.15.32:0/1707186975 shutdown_connections 2024-04-17T09:53:20.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.067+0000 7fd623000640 1 --2- 172.21.15.32:0/1707186975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd61c06fdf0 0x7fd61c0c4750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.067+0000 7fd623000640 1 --2- 172.21.15.32:0/1707186975 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd61c163fa0 0x7fd61c0c4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.067+0000 7fd623000640 1 --2- 172.21.15.32:0/1707186975 >> v2:172.21.15.32:3300/0 conn(0x7fd61c162da0 0x7fd61c06f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.067+0000 7fd623000640 1 -- 172.21.15.32:0/1707186975 >> 172.21.15.32:0/1707186975 conn(0x7fd61c0d1240 msgr2=0x7fd61c0d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:20.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.067+0000 7fd623000640 1 -- 172.21.15.32:0/1707186975 shutdown_connections 2024-04-17T09:53:20.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.067+0000 7f83c3e00640 1 Processor -- start 2024-04-17T09:53:20.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.067+0000 7f83c3e00640 1 -- start start 2024-04-17T09:53:20.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.067+0000 7fd623000640 1 -- 172.21.15.32:0/1707186975 wait complete. 2024-04-17T09:53:20.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7fd623000640 1 Processor -- start 2024-04-17T09:53:20.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7f83c3e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83c40b1c60 0x7f83c415f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7f83c3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83c40b2830 0x7f83c415e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7f83c3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c4149110 0x7f83c415b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7f83c3e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f83c40b5680 con 0x7f83c4149110 2024-04-17T09:53:20.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7f83c3e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f83c40b3200 con 0x7f83c40b2830 2024-04-17T09:53:20.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7f83c3e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f83c40b5800 con 0x7f83c40b1c60 2024-04-17T09:53:20.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7fd623000640 1 -- start start 2024-04-17T09:53:20.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7f83c2000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83c40b2830 0x7f83c415e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7f83c2000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83c40b2830 0x7f83c415e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:40978/0 (socket says 172.21.15.32:40978) 2024-04-17T09:53:20.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7f83c3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c4149110 0x7f83c415b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7f83c2000640 1 -- 172.21.15.32:0/3656965216 learned_addr learned my addr 172.21.15.32:0/3656965216 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:20.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7f83c2a00640 1 --2- 172.21.15.32:0/3656965216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83c40b1c60 0x7f83c415f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7f83c3400640 1 -- 172.21.15.32:0/3656965216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83c40b1c60 msgr2=0x7f83c415f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7f83c3400640 1 --2- 172.21.15.32:0/3656965216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83c40b1c60 0x7f83c415f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7f83c3400640 1 -- 172.21.15.32:0/3656965216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83c40b2830 msgr2=0x7f83c415e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7f83c3400640 1 --2- 172.21.15.32:0/3656965216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83c40b2830 0x7f83c415e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7f83c3400640 1 -- 172.21.15.32:0/3656965216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c415f8e0 con 0x7f83c4149110 2024-04-17T09:53:20.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7fd623000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd61c06fdf0 0x7fd61c146e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7f83c3400640 1 --2- 172.21.15.32:0/3656965216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c4149110 0x7f83c415b4e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f83bc00be20 tx=0x7f83bc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7fd623000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd61c162da0 0x7fd61c149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7fd623000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd61c163fa0 0x7fd61c0c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7f83baa00640 1 -- 172.21.15.32:0/3656965216 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc019070 con 0x7f83c4149110 2024-04-17T09:53:20.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7f83baa00640 1 -- 172.21.15.32:0/3656965216 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83bc014440 con 0x7f83c4149110 2024-04-17T09:53:20.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7f83baa00640 1 -- 172.21.15.32:0/3656965216 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc014830 con 0x7f83c4149110 2024-04-17T09:53:20.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7f83c3e00640 1 -- 172.21.15.32:0/3656965216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c415eab0 con 0x7f83c4149110 2024-04-17T09:53:20.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7fd623000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c06c5d0 con 0x7fd61c06fdf0 2024-04-17T09:53:20.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7fd623000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c06c450 con 0x7fd61c162da0 2024-04-17T09:53:20.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7fd623000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd61c06c750 con 0x7fd61c163fa0 2024-04-17T09:53:20.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7f83c3e00640 1 -- 172.21.15.32:0/3656965216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c4133dc0 con 0x7f83c4149110 2024-04-17T09:53:20.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7fd622600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd61c163fa0 0x7fd61c0c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7fd621200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd61c162da0 0x7fd61c149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7fd622600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd61c163fa0 0x7fd61c0c9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:56518/0 (socket says 172.21.15.32:56518) 2024-04-17T09:53:20.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7fd622600640 1 -- 172.21.15.32:0/913280655 learned_addr learned my addr 172.21.15.32:0/913280655 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:20.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7fd622600640 1 -- 172.21.15.32:0/913280655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd61c162da0 msgr2=0x7fd61c149cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7fd622600640 1 --2- 172.21.15.32:0/913280655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd61c162da0 0x7fd61c149cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7fd622600640 1 -- 172.21.15.32:0/913280655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd61c06fdf0 msgr2=0x7fd61c146e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.071+0000 7fd622600640 1 --2- 172.21.15.32:0/913280655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd61c06fdf0 0x7fd61c146e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.075+0000 7fd622600640 1 -- 172.21.15.32:0/913280655 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61c144100 con 0x7fd61c163fa0 2024-04-17T09:53:20.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.075+0000 7fd622600640 1 --2- 172.21.15.32:0/913280655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd61c163fa0 0x7fd61c0c9bc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd61401d220 tx=0x7fd614019720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.075+0000 7fd619600640 1 -- 172.21.15.32:0/913280655 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614009510 con 0x7fd61c163fa0 2024-04-17T09:53:20.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.075+0000 7fd623000640 1 -- 172.21.15.32:0/913280655 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61c1443e0 con 0x7fd61c163fa0 2024-04-17T09:53:20.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.075+0000 7fd623000640 1 -- 172.21.15.32:0/913280655 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd61c1449a0 con 0x7fd61c163fa0 2024-04-17T09:53:20.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.075+0000 7fd619600640 1 -- 172.21.15.32:0/913280655 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6140bc070 con 0x7fd61c163fa0 2024-04-17T09:53:20.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.075+0000 7fd619600640 1 -- 172.21.15.32:0/913280655 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614009510 con 0x7fd61c163fa0 2024-04-17T09:53:20.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.075+0000 7fd623000640 1 -- 172.21.15.32:0/913280655 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5f80052a0 con 0x7fd61c163fa0 2024-04-17T09:53:20.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.075+0000 7f83baa00640 1 -- 172.21.15.32:0/3656965216 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f83bc01c070 con 0x7f83c4149110 2024-04-17T09:53:20.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.079+0000 7f83c3e00640 1 -- 172.21.15.32:0/3656965216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83c404fd00 con 0x7f83c4149110 2024-04-17T09:53:20.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.079+0000 7f83baa00640 1 --2- 172.21.15.32:0/3656965216 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f83a40683b0 0x7f83a406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.079+0000 7f83c2a00640 1 --2- 172.21.15.32:0/3656965216 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f83a40683b0 0x7f83a406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.079+0000 7f83baa00640 1 -- 172.21.15.32:0/3656965216 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f83bc0869e0 con 0x7f83c4149110 2024-04-17T09:53:20.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.079+0000 7f83c2a00640 1 --2- 172.21.15.32:0/3656965216 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f83a40683b0 0x7f83a406a870 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f83b00079c0 tx=0x7f83b0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.079+0000 7fd619600640 1 -- 172.21.15.32:0/913280655 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd6140a5b50 con 0x7fd61c163fa0 2024-04-17T09:53:20.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.083+0000 7fd619600640 1 --2- 172.21.15.32:0/913280655 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5fc068780 0x7fd5fc06ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.083+0000 7fd621c00640 1 --2- 172.21.15.32:0/913280655 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5fc068780 0x7fd5fc06ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.083+0000 7fd619600640 1 -- 172.21.15.32:0/913280655 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd6140a5950 con 0x7fd61c163fa0 2024-04-17T09:53:20.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.083+0000 7fd621c00640 1 --2- 172.21.15.32:0/913280655 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5fc068780 0x7fd5fc06ac40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd6100099c0 tx=0x7fd610009480 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.087+0000 7fd619600640 1 -- 172.21.15.32:0/913280655 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd614016600 con 0x7fd61c163fa0 2024-04-17T09:53:20.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.087+0000 7f83baa00640 1 -- 172.21.15.32:0/3656965216 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f83bc056820 con 0x7f83c4149110 2024-04-17T09:53:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.099+0000 7ff5ab200640 1 Processor -- start 2024-04-17T09:53:20.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.099+0000 7ff5ab200640 1 -- start start 2024-04-17T09:53:20.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.103+0000 7ff5ab200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff5ac0b66c0 0x7ff5ac0b7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.103+0000 7ff5ab200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5ac09d430 con 0x7ff5ac0d3f10 2024-04-17T09:53:20.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.103+0000 7ff5ab200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5ac09cb90 con 0x7ff5ac0cf470 2024-04-17T09:53:20.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.103+0000 7ff5ab200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff5ac09ce70 con 0x7ff5ac0b66c0 2024-04-17T09:53:20.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.103+0000 7ff5aa800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff5ac0d3f10 0x7ff5ac0d2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39248/0 (socket says 172.21.15.32:39248) 2024-04-17T09:53:20.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.103+0000 7ff5aa800640 1 -- 172.21.15.32:0/2454824251 learned_addr learned my addr 172.21.15.32:0/2454824251 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:20.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.103+0000 7ff5a9400640 1 --2- 172.21.15.32:0/2454824251 >> v2:172.21.15.144:3300/0 conn(0x7ff5ac0b66c0 0x7ff5ac0b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.103+0000 7ff5a8a00640 1 -- 172.21.15.32:0/2454824251 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1575509205 0 0) 0x7ff5ac09cb90 con 0x7ff5ac0cf470 2024-04-17T09:53:20.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.103+0000 7ff5a8a00640 1 -- 172.21.15.32:0/2454824251 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff590003530 con 0x7ff5ac0cf470 2024-04-17T09:53:20.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.103+0000 7ff5a9400640 1 -- 172.21.15.32:0/2454824251 >> v1:172.21.15.144:6789/0 conn(0x7ff5ac0cf470 legacy=0x7ff5ac0b5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.103+0000 7ff5a8a00640 1 -- 172.21.15.32:0/2454824251 <== mon.1 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3580953867 0 0) 0x7ff590003530 con 0x7ff5ac0cf470 2024-04-17T09:53:20.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.103+0000 7ff5a9400640 1 -- 172.21.15.32:0/2454824251 >> v1:172.21.15.32:6789/0 conn(0x7ff5ac0d3f10 legacy=0x7ff5ac0d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.103+0000 7ff5a9400640 1 -- 172.21.15.32:0/2454824251 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ac09d150 con 0x7ff5ac0b66c0 2024-04-17T09:53:20.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.107+0000 7ff5a9400640 1 --2- 172.21.15.32:0/2454824251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ac0b66c0 0x7ff5ac0b7380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4009d90 tx=0x7ff5a401c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=960fbe376ee5c803 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.111+0000 7ff5a8a00640 1 -- 172.21.15.32:0/2454824251 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a40b2070 con 0x7ff5ac0b66c0 2024-04-17T09:53:20.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.111+0000 7ff5a8a00640 1 -- 172.21.15.32:0/2454824251 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5a40a5530 con 0x7ff5ac0b66c0 2024-04-17T09:53:20.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.111+0000 7ff5a8a00640 1 -- 172.21.15.32:0/2454824251 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c001680 con 0x7ff5ac0b66c0 2024-04-17T09:53:20.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.111+0000 7ff5ab200640 1 -- 172.21.15.32:0/2454824251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ac0b66c0 msgr2=0x7ff5ac0b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.111+0000 7ff5ab200640 1 --2- 172.21.15.32:0/2454824251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ac0b66c0 0x7ff5ac0b7380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4009d90 tx=0x7ff5a401c820 comp rx=0 tx=0).stop 2024-04-17T09:53:20.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.111+0000 7ff5ab200640 1 -- 172.21.15.32:0/2454824251 shutdown_connections 2024-04-17T09:53:20.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.111+0000 7ff5ab200640 1 --2- 172.21.15.32:0/2454824251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ac0b66c0 0x7ff5ac0b7380 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.111+0000 7ff5ab200640 1 -- 172.21.15.32:0/2454824251 >> 172.21.15.32:0/2454824251 conn(0x7ff5ac0d4530 msgr2=0x7ff5ac0d4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:20.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.111+0000 7ff5ab200640 1 -- 172.21.15.32:0/2454824251 shutdown_connections 2024-04-17T09:53:20.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.111+0000 7ff5ab200640 1 -- 172.21.15.32:0/2454824251 wait complete. 2024-04-17T09:53:20.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.115+0000 7ff5ab200640 1 Processor -- start 2024-04-17T09:53:20.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.115+0000 7ff5ab200640 1 -- start start 2024-04-17T09:53:20.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.115+0000 7ff5ab200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5ac0b66c0 0x7ff5ac161990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.115+0000 7ff5ab200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5ac0cf470 0x7ff5ac155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.115+0000 7ff5ab200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ac0d3f10 0x7ff5ac0cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.115+0000 7ff5ab200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac0d32d0 con 0x7ff5ac0b66c0 2024-04-17T09:53:20.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.115+0000 7ff5ab200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac0d3150 con 0x7ff5ac0d3f10 2024-04-17T09:53:20.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.115+0000 7ff5ab200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff5ac171280 con 0x7ff5ac0cf470 2024-04-17T09:53:20.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.115+0000 7ff5a9400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5ac0cf470 0x7ff5ac155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.115+0000 7ff5a9400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5ac0cf470 0x7ff5ac155ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:56528/0 (socket says 172.21.15.32:56528) 2024-04-17T09:53:20.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.115+0000 7ff5a9400640 1 -- 172.21.15.32:0/1172867371 learned_addr learned my addr 172.21.15.32:0/1172867371 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:20.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.115+0000 7ff5aa800640 1 --2- 172.21.15.32:0/1172867371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ac0d3f10 0x7ff5ac0cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.115+0000 7ff5aa800640 1 -- 172.21.15.32:0/1172867371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5ac0cf470 msgr2=0x7ff5ac155ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.115+0000 7ff5aa800640 1 --2- 172.21.15.32:0/1172867371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5ac0cf470 0x7ff5ac155ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.115+0000 7ff5aa800640 1 -- 172.21.15.32:0/1172867371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5ac0b66c0 msgr2=0x7ff5ac161990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.115+0000 7ff5a9e00640 1 --2- 172.21.15.32:0/1172867371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5ac0b66c0 0x7ff5ac161990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.119+0000 7ff5aa800640 1 --2- 172.21.15.32:0/1172867371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5ac0b66c0 0x7ff5ac161990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.119+0000 7ff5aa800640 1 -- 172.21.15.32:0/1172867371 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ac0ca3e0 con 0x7ff5ac0d3f10 2024-04-17T09:53:20.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.119+0000 7ff5a9e00640 1 --2- 172.21.15.32:0/1172867371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5ac0b66c0 0x7ff5ac161990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:20.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.119+0000 7ff5aa800640 1 --2- 172.21.15.32:0/1172867371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ac0d3f10 0x7ff5ac0cdc10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff59800c3f0 tx=0x7ff59800c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.119+0000 7ff5a2000640 1 -- 172.21.15.32:0/1172867371 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598017070 con 0x7ff5ac0d3f10 2024-04-17T09:53:20.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.119+0000 7ff5a2000640 1 -- 172.21.15.32:0/1172867371 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff598005cf0 con 0x7ff5ac0d3f10 2024-04-17T09:53:20.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.119+0000 7ff5a2000640 1 -- 172.21.15.32:0/1172867371 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5980072f0 con 0x7ff5ac0d3f10 2024-04-17T09:53:20.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.119+0000 7ff5ab200640 1 -- 172.21.15.32:0/1172867371 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5ac164f10 con 0x7ff5ac0d3f10 2024-04-17T09:53:20.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.119+0000 7ff5ab200640 1 -- 172.21.15.32:0/1172867371 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5ac1654d0 con 0x7ff5ac0d3f10 2024-04-17T09:53:20.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.123+0000 7ff5a2000640 1 -- 172.21.15.32:0/1172867371 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff59801a070 con 0x7ff5ac0d3f10 2024-04-17T09:53:20.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.123+0000 7ff5ab200640 1 -- 172.21.15.32:0/1172867371 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5ac04fd00 con 0x7ff5ac0d3f10 2024-04-17T09:53:20.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.123+0000 7ff5a2000640 1 --2- 172.21.15.32:0/1172867371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff58c068380 0x7ff58c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.123+0000 7ff5a2000640 1 -- 172.21.15.32:0/1172867371 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff598085d10 con 0x7ff5ac0d3f10 2024-04-17T09:53:20.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.127+0000 7ff5a9e00640 1 --2- 172.21.15.32:0/1172867371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff58c068380 0x7ff58c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.127+0000 7ff5a9e00640 1 --2- 172.21.15.32:0/1172867371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff58c068380 0x7ff58c06a840 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff59c0079a0 tx=0x7ff59c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.127+0000 7f5b95000640 1 Processor -- start 2024-04-17T09:53:20.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.127+0000 7f5b95000640 1 -- start start 2024-04-17T09:53:20.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.131+0000 7ff5a2000640 1 -- 172.21.15.32:0/1172867371 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff598056230 con 0x7ff5ac0d3f10 2024-04-17T09:53:20.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.131+0000 7f5b95000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b880e3290 0x7f5b880e6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.131+0000 7f5b95000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5b880383a0 0x7f5b880e73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.131+0000 7f5b95000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b880e78f0 0x7f5b88053780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.131+0000 7f5b95000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5b88058590 con 0x7f5b880e3290 2024-04-17T09:53:20.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.131+0000 7f5b95000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f5b88057a10 con 0x7f5b880383a0 2024-04-17T09:53:20.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.131+0000 7f5b95000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5b88057cf0 con 0x7f5b880e78f0 2024-04-17T09:53:20.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.131+0000 7f5b8de00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b880e3290 0x7f5b880e6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.131+0000 7f5b8de00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b880e3290 0x7f5b880e6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47102/0 (socket says 172.21.15.32:47102) 2024-04-17T09:53:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.131+0000 7f5b8de00640 1 -- 172.21.15.32:0/4032576162 learned_addr learned my addr 172.21.15.32:0/4032576162 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.131+0000 7f5b8e800640 1 --2- 172.21.15.32:0/4032576162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b880e78f0 0x7f5b88053780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.131+0000 7f5b8d400640 1 --2- 172.21.15.32:0/4032576162 >> v2:172.21.15.144:3300/0 conn(0x7f5b880383a0 0x7f5b880e73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.131+0000 7f5b8e800640 1 -- 172.21.15.32:0/4032576162 >> v2:172.21.15.144:3300/0 conn(0x7f5b880383a0 msgr2=0x7f5b880e73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.131+0000 7f5b8e800640 1 --2- 172.21.15.32:0/4032576162 >> v2:172.21.15.144:3300/0 conn(0x7f5b880383a0 0x7f5b880e73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.131+0000 7f5b8e800640 1 -- 172.21.15.32:0/4032576162 >> v2:172.21.15.32:3300/0 conn(0x7f5b880e3290 msgr2=0x7f5b880e6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:20.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.131+0000 7f5b8e800640 1 --2- 172.21.15.32:0/4032576162 >> v2:172.21.15.32:3300/0 conn(0x7f5b880e3290 0x7f5b880e6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.131+0000 7f5b8e800640 1 -- 172.21.15.32:0/4032576162 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8800f840 con 0x7f5b880e78f0 2024-04-17T09:53:20.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.135+0000 7f5b8e800640 1 --2- 172.21.15.32:0/4032576162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b880e78f0 0x7f5b88053780 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800e220 tx=0x7f5b7c01c0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e92679ac156a4d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.143+0000 7f5b8ca00640 1 -- 172.21.15.32:0/4032576162 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c0b0070 con 0x7f5b880e78f0 2024-04-17T09:53:20.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.143+0000 7f5b8ca00640 1 -- 172.21.15.32:0/4032576162 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b7c0ab600 con 0x7f5b880e78f0 2024-04-17T09:53:20.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.143+0000 7f5b8ca00640 1 -- 172.21.15.32:0/4032576162 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c0ab9d0 con 0x7f5b880e78f0 2024-04-17T09:53:20.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.143+0000 7f5b95000640 1 -- 172.21.15.32:0/4032576162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b880e78f0 msgr2=0x7f5b88053780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.143+0000 7f5b95000640 1 --2- 172.21.15.32:0/4032576162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b880e78f0 0x7f5b88053780 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800e220 tx=0x7f5b7c01c0a0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.143+0000 7f5b95000640 1 -- 172.21.15.32:0/4032576162 shutdown_connections 2024-04-17T09:53:20.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.143+0000 7f5b95000640 1 --2- 172.21.15.32:0/4032576162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b880e78f0 0x7f5b88053780 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.143+0000 7f5b95000640 1 --2- 172.21.15.32:0/4032576162 >> v2:172.21.15.144:3300/0 conn(0x7f5b880383a0 0x7f5b880e73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.143+0000 7f5b95000640 1 --2- 172.21.15.32:0/4032576162 >> v2:172.21.15.32:3300/0 conn(0x7f5b880e3290 0x7f5b880e6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.143+0000 7f5b95000640 1 -- 172.21.15.32:0/4032576162 >> 172.21.15.32:0/4032576162 conn(0x7f5b880559a0 msgr2=0x7f5b88055db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:20.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.143+0000 7f5b95000640 1 -- 172.21.15.32:0/4032576162 shutdown_connections 2024-04-17T09:53:20.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.143+0000 7f5b95000640 1 -- 172.21.15.32:0/4032576162 wait complete. 2024-04-17T09:53:20.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.147+0000 7f5b95000640 1 Processor -- start 2024-04-17T09:53:20.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.147+0000 7f5b95000640 1 -- start start 2024-04-17T09:53:20.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.147+0000 7f5b95000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b880383a0 0x7f5b88052990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.147+0000 7f5b95000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b880e3290 0x7f5b8804ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.147+0000 7f5b95000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b880e78f0 0x7f5b8818c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.147+0000 7f5b8de00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b880383a0 0x7f5b88052990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.151+0000 7f5b8de00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b880383a0 0x7f5b88052990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:56546/0 (socket says 172.21.15.32:56546) 2024-04-17T09:53:20.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.151+0000 7f5b8de00640 1 -- 172.21.15.32:0/1336741048 learned_addr learned my addr 172.21.15.32:0/1336741048 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:20.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.151+0000 7f5b95000640 1 -- 172.21.15.32:0/1336741048 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5b880e37f0 con 0x7f5b880e3290 2024-04-17T09:53:20.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.151+0000 7f5b95000640 1 -- 172.21.15.32:0/1336741048 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5b880e3670 con 0x7f5b880e78f0 2024-04-17T09:53:20.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.151+0000 7f5b95000640 1 -- 172.21.15.32:0/1336741048 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5b88006890 con 0x7f5b880383a0 2024-04-17T09:53:20.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.151+0000 7f5b8d400640 1 --2- 172.21.15.32:0/1336741048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b880e3290 0x7f5b8804ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.151+0000 7f5b8e800640 1 --2- 172.21.15.32:0/1336741048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b880e78f0 0x7f5b8818c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.151+0000 7f5b8e800640 1 -- 172.21.15.32:0/1336741048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b880383a0 msgr2=0x7f5b88052990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.151+0000 7f5b8e800640 1 --2- 172.21.15.32:0/1336741048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b880383a0 0x7f5b88052990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.151+0000 7f5b8e800640 1 -- 172.21.15.32:0/1336741048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b880e3290 msgr2=0x7f5b8804ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.151+0000 7f5b8e800640 1 --2- 172.21.15.32:0/1336741048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b880e3290 0x7f5b8804ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.151+0000 7f5b8e800640 1 -- 172.21.15.32:0/1336741048 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b88051bd0 con 0x7f5b880e78f0 2024-04-17T09:53:20.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.151+0000 7f5b8e800640 1 --2- 172.21.15.32:0/1336741048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b880e78f0 0x7f5b8818c140 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0ab000 tx=0x7f5b7c018e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.151+0000 7f5b86000640 1 -- 172.21.15.32:0/1336741048 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c0b8020 con 0x7f5b880e78f0 2024-04-17T09:53:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.151+0000 7f5b86000640 1 -- 172.21.15.32:0/1336741048 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b7c0b4620 con 0x7f5b880e78f0 2024-04-17T09:53:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.151+0000 7f5b86000640 1 -- 172.21.15.32:0/1336741048 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c0b0040 con 0x7f5b880e78f0 2024-04-17T09:53:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.151+0000 7f5b95000640 1 -- 172.21.15.32:0/1336741048 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b88051eb0 con 0x7f5b880e78f0 2024-04-17T09:53:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.151+0000 7f5b95000640 1 -- 172.21.15.32:0/1336741048 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b880e3ca0 con 0x7f5b880e78f0 2024-04-17T09:53:20.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.155+0000 7f5b86000640 1 -- 172.21.15.32:0/1336741048 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5b7c0bb020 con 0x7f5b880e78f0 2024-04-17T09:53:20.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.155+0000 7f5b86000640 1 --2- 172.21.15.32:0/1336741048 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5b70068380 0x7f5b7006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:20.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.159+0000 7f5b86000640 1 -- 172.21.15.32:0/1336741048 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5b7c120270 con 0x7f5b880e78f0 2024-04-17T09:53:20.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.159+0000 7f5b95000640 1 -- 172.21.15.32:0/1336741048 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b680052a0 con 0x7f5b880e78f0 2024-04-17T09:53:20.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.159+0000 7f5b8de00640 1 --2- 172.21.15.32:0/1336741048 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5b70068380 0x7f5b7006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:20.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.159+0000 7f5b8de00640 1 --2- 172.21.15.32:0/1336741048 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5b70068380 0x7f5b7006a840 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5b9006a920 tx=0x7f5b9008d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:20.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.163+0000 7f5b86000640 1 -- 172.21.15.32:0/1336741048 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5b7c0c2030 con 0x7f5b880e78f0 2024-04-17T09:53:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.399+0000 7f410c600640 1 -- 172.21.15.32:0/2629817355 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f410416d810 con 0x7f410406c140 2024-04-17T09:53:20.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.403+0000 7f4101600640 1 -- 172.21.15.32:0/2629817355 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f40f8057590 con 0x7f410406c140 2024-04-17T09:53:20.408 INFO:teuthology.orchestra.run.smithi032.stdout:60129542149 2024-04-17T09:53:20.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.407+0000 7f3c07400640 1 -- 172.21.15.32:0/4191672350 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f3c18156680 con 0x7f3c180afa20 2024-04-17T09:53:20.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.407+0000 7f3c12000640 1 -- 172.21.15.32:0/4191672350 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3c0805a010 con 0x7f3c180afa20 2024-04-17T09:53:20.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.407+0000 7f410c600640 1 -- 172.21.15.32:0/2629817355 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40e4068450 msgr2=0x7f40e406a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.407+0000 7f410c600640 1 --2- 172.21.15.32:0/2629817355 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40e4068450 0x7f40e406a910 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f41041676b0 tx=0x7f40f0007e40 comp rx=0 tx=0).stop 2024-04-17T09:53:20.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.407+0000 7f410c600640 1 -- 172.21.15.32:0/2629817355 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f410406c140 msgr2=0x7f410415d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.407+0000 7f410c600640 1 --2- 172.21.15.32:0/2629817355 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f410406c140 0x7f410415d230 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f40f800d7d0 tx=0x7f40f800dc90 comp rx=0 tx=0).stop 2024-04-17T09:53:20.412 INFO:teuthology.orchestra.run.smithi032.stdout:64424509446 2024-04-17T09:53:20.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.411+0000 7f410c600640 1 -- 172.21.15.32:0/2629817355 shutdown_connections 2024-04-17T09:53:20.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.411+0000 7f410c600640 1 --2- 172.21.15.32:0/2629817355 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40e4068450 0x7f40e406a910 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.411+0000 7f410c600640 1 --2- 172.21.15.32:0/2629817355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4104160560 0x7f410414e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.411+0000 7f410c600640 1 --2- 172.21.15.32:0/2629817355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41040b3d70 0x7f410415a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.411+0000 7f410c600640 1 --2- 172.21.15.32:0/2629817355 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f410406c140 0x7f410415d230 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.411+0000 7f410c600640 1 -- 172.21.15.32:0/2629817355 >> 172.21.15.32:0/2629817355 conn(0x7f41040b2120 msgr2=0x7f4104164350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:20.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.411+0000 7f410c600640 1 -- 172.21.15.32:0/2629817355 shutdown_connections 2024-04-17T09:53:20.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.411+0000 7f410c600640 1 -- 172.21.15.32:0/2629817355 wait complete. 2024-04-17T09:53:20.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.411+0000 7fa85d000640 1 -- 172.21.15.32:0/2786213255 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa830005590 con 0x7fa8580728e0 2024-04-17T09:53:20.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.411+0000 7f3c07400640 1 -- 172.21.15.32:0/4191672350 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3bfc0683d0 msgr2=0x7f3bfc06a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.411+0000 7f3c07400640 1 --2- 172.21.15.32:0/4191672350 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3bfc0683d0 0x7f3bfc06a890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0ae000 tx=0x7f3c0c008040 comp rx=0 tx=0).stop 2024-04-17T09:53:20.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.411+0000 7f3c07400640 1 -- 172.21.15.32:0/4191672350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c180afa20 msgr2=0x7f3c18157490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.411+0000 7f3c07400640 1 --2- 172.21.15.32:0/4191672350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c180afa20 0x7f3c18157490 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800c3a0 tx=0x7f3c0800c860 comp rx=0 tx=0).stop 2024-04-17T09:53:20.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.411+0000 7fa84e000640 1 -- 172.21.15.32:0/2786213255 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa8500f1160 con 0x7fa8580728e0 2024-04-17T09:53:20.416 INFO:teuthology.orchestra.run.smithi032.stdout:64424509445 2024-04-17T09:53:20.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.415+0000 7f3c07400640 1 -- 172.21.15.32:0/4191672350 shutdown_connections 2024-04-17T09:53:20.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.415+0000 7f3c07400640 1 --2- 172.21.15.32:0/4191672350 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3bfc0683d0 0x7f3bfc06a890 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.415+0000 7f3c07400640 1 --2- 172.21.15.32:0/4191672350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c1814de00 0x7f3c1814fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.415+0000 7f3c07400640 1 --2- 172.21.15.32:0/4191672350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c180afa20 0x7f3c18157490 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.415+0000 7f3c07400640 1 --2- 172.21.15.32:0/4191672350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c180addf0 0x7f3c1813dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.415+0000 7f3c07400640 1 -- 172.21.15.32:0/4191672350 >> 172.21.15.32:0/4191672350 conn(0x7f3c180b2120 msgr2=0x7f3c18160e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:20.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.415+0000 7f3c07400640 1 -- 172.21.15.32:0/4191672350 shutdown_connections 2024-04-17T09:53:20.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.415+0000 7f3c07400640 1 -- 172.21.15.32:0/4191672350 wait complete. 2024-04-17T09:53:20.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.415+0000 7fa85d000640 1 -- 172.21.15.32:0/2786213255 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa8380683d0 msgr2=0x7fa83806a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.415+0000 7fa85d000640 1 --2- 172.21.15.32:0/2786213255 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa8380683d0 0x7fa83806a890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa858150cc0 tx=0x7fa848008040 comp rx=0 tx=0).stop 2024-04-17T09:53:20.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.415+0000 7fa85d000640 1 -- 172.21.15.32:0/2786213255 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8580728e0 msgr2=0x7fa858207740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.415+0000 7fa85d000640 1 --2- 172.21.15.32:0/2786213255 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8580728e0 0x7fa858207740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa8500ab040 tx=0x7fa8500194a0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.415+0000 7fa85d000640 1 -- 172.21.15.32:0/2786213255 shutdown_connections 2024-04-17T09:53:20.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.415+0000 7fa85d000640 1 --2- 172.21.15.32:0/2786213255 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa8380683d0 0x7fa83806a890 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.415+0000 7fa85d000640 1 --2- 172.21.15.32:0/2786213255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa85815e750 0x7fa8580ac560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.415+0000 7fa85d000640 1 --2- 172.21.15.32:0/2786213255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8580afa20 0x7fa858151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.419+0000 7fa85d000640 1 --2- 172.21.15.32:0/2786213255 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8580728e0 0x7fa858207740 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.419+0000 7fa85d000640 1 -- 172.21.15.32:0/2786213255 >> 172.21.15.32:0/2786213255 conn(0x7fa8580b2120 msgr2=0x7fa858156ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:20.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.419+0000 7fa85d000640 1 -- 172.21.15.32:0/2786213255 shutdown_connections 2024-04-17T09:53:20.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.419+0000 7fa85d000640 1 -- 172.21.15.32:0/2786213255 wait complete. 2024-04-17T09:53:20.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.427+0000 7fc5ce800640 1 -- 172.21.15.32:0/2062583824 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fc5c806ff10 con 0x7fc5c809bab0 2024-04-17T09:53:20.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.427+0000 7fc5bb400640 1 -- 172.21.15.32:0/2062583824 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc5bc058140 con 0x7fc5c809bab0 2024-04-17T09:53:20.434 INFO:teuthology.orchestra.run.smithi032.stdout:64424509446 2024-04-17T09:53:20.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.431+0000 7f83c3e00640 1 -- 172.21.15.32:0/3656965216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f83c40ce750 con 0x7f83c4149110 2024-04-17T09:53:20.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.431+0000 7f83baa00640 1 -- 172.21.15.32:0/3656965216 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f83bc05ab20 con 0x7f83c4149110 2024-04-17T09:53:20.436 INFO:teuthology.orchestra.run.smithi032.stdout:60129542149 2024-04-17T09:53:20.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.431+0000 7fc5ce800640 1 -- 172.21.15.32:0/2062583824 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc5a8068380 msgr2=0x7fc5a806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.431+0000 7fc5ce800640 1 --2- 172.21.15.32:0/2062583824 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc5a8068380 0x7fc5a806a840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00b2000 tx=0x7fc5c00ab070 comp rx=0 tx=0).stop 2024-04-17T09:53:20.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.435+0000 7fc5ce800640 1 -- 172.21.15.32:0/2062583824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5c809bab0 msgr2=0x7fc5c814ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.435+0000 7fc5ce800640 1 --2- 172.21.15.32:0/2062583824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5c809bab0 0x7fc5c814ea40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00d830 tx=0x7fc5bc00dcf0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.435+0000 7fc5ce800640 1 -- 172.21.15.32:0/2062583824 shutdown_connections 2024-04-17T09:53:20.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.435+0000 7fc5ce800640 1 --2- 172.21.15.32:0/2062583824 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc5a8068380 0x7fc5a806a840 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.435+0000 7fc5ce800640 1 --2- 172.21.15.32:0/2062583824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5c815c7a0 0x7fc5c8153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.435+0000 7fc5ce800640 1 --2- 172.21.15.32:0/2062583824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5c80cda50 0x7fc5c815cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.435+0000 7fc5ce800640 1 --2- 172.21.15.32:0/2062583824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5c809bab0 0x7fc5c814ea40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.435+0000 7fc5ce800640 1 -- 172.21.15.32:0/2062583824 >> 172.21.15.32:0/2062583824 conn(0x7fc5c80b3f90 msgr2=0x7fc5c80b5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:20.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.435+0000 7fd623000640 1 -- 172.21.15.32:0/913280655 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd5f8005590 con 0x7fd61c163fa0 2024-04-17T09:53:20.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.435+0000 7fc5ce800640 1 -- 172.21.15.32:0/2062583824 shutdown_connections 2024-04-17T09:53:20.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.435+0000 7fc5ce800640 1 -- 172.21.15.32:0/2062583824 wait complete. 2024-04-17T09:53:20.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.435+0000 7fd619600640 1 -- 172.21.15.32:0/913280655 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd6140f1300 con 0x7fd61c163fa0 2024-04-17T09:53:20.441 INFO:teuthology.orchestra.run.smithi032.stdout:64424509445 2024-04-17T09:53:20.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7f83c3e00640 1 -- 172.21.15.32:0/3656965216 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f83a40683b0 msgr2=0x7f83a406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7f83c3e00640 1 --2- 172.21.15.32:0/3656965216 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f83a40683b0 0x7f83a406a870 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f83b00079c0 tx=0x7f83b0008040 comp rx=0 tx=0).stop 2024-04-17T09:53:20.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7f83c3e00640 1 -- 172.21.15.32:0/3656965216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c4149110 msgr2=0x7f83c415b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7f83c3e00640 1 --2- 172.21.15.32:0/3656965216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c4149110 0x7f83c415b4e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f83bc00be20 tx=0x7f83bc00bf10 comp rx=0 tx=0).stop 2024-04-17T09:53:20.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7f83c3e00640 1 -- 172.21.15.32:0/3656965216 shutdown_connections 2024-04-17T09:53:20.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7f83c3e00640 1 --2- 172.21.15.32:0/3656965216 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f83a40683b0 0x7f83a406a870 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f83b00079c0 tx=0x7f83b0008040 comp rx=0 tx=0).stop 2024-04-17T09:53:20.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7f83c3e00640 1 --2- 172.21.15.32:0/3656965216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c4149110 0x7f83c415b4e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7f83c3e00640 1 --2- 172.21.15.32:0/3656965216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83c40b2830 0x7f83c415e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7f83c3e00640 1 --2- 172.21.15.32:0/3656965216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83c40b1c60 0x7f83c415f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7f83c3e00640 1 -- 172.21.15.32:0/3656965216 >> 172.21.15.32:0/3656965216 conn(0x7f83c4067940 msgr2=0x7f83c413f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:20.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7fd623000640 1 -- 172.21.15.32:0/913280655 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5fc068780 msgr2=0x7fd5fc06ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7fd623000640 1 --2- 172.21.15.32:0/913280655 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5fc068780 0x7fd5fc06ac40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd6100099c0 tx=0x7fd610009480 comp rx=0 tx=0).stop 2024-04-17T09:53:20.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7fd623000640 1 -- 172.21.15.32:0/913280655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd61c163fa0 msgr2=0x7fd61c0c9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7fd623000640 1 --2- 172.21.15.32:0/913280655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd61c163fa0 0x7fd61c0c9bc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd61401d220 tx=0x7fd614019720 comp rx=0 tx=0).stop 2024-04-17T09:53:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7fd623000640 1 -- 172.21.15.32:0/913280655 shutdown_connections 2024-04-17T09:53:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7fd623000640 1 --2- 172.21.15.32:0/913280655 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5fc068780 0x7fd5fc06ac40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7f83c3e00640 1 -- 172.21.15.32:0/3656965216 shutdown_connections 2024-04-17T09:53:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7f83c3e00640 1 -- 172.21.15.32:0/3656965216 wait complete. 2024-04-17T09:53:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7fd623000640 1 --2- 172.21.15.32:0/913280655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd61c163fa0 0x7fd61c0c9bc0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7fd623000640 1 --2- 172.21.15.32:0/913280655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd61c162da0 0x7fd61c149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7fd623000640 1 --2- 172.21.15.32:0/913280655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd61c06fdf0 0x7fd61c146e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7fd623000640 1 -- 172.21.15.32:0/913280655 >> 172.21.15.32:0/913280655 conn(0x7fd61c0d1240 msgr2=0x7fd61c0cc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7fd623000640 1 -- 172.21.15.32:0/913280655 shutdown_connections 2024-04-17T09:53:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.439+0000 7fd623000640 1 -- 172.21.15.32:0/913280655 wait complete. 2024-04-17T09:53:20.457 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.0 2024-04-17T09:53:20.460 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.2 2024-04-17T09:53:20.462 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.1 2024-04-17T09:53:20.475 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.5 2024-04-17T09:53:20.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.471+0000 7ff5ab200640 1 -- 172.21.15.32:0/1172867371 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ff5ac09db60 con 0x7ff5ac0d3f10 2024-04-17T09:53:20.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.471+0000 7ff5a2000640 1 -- 172.21.15.32:0/1172867371 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff598055b30 con 0x7ff5ac0d3f10 2024-04-17T09:53:20.476 INFO:teuthology.orchestra.run.smithi032.stdout:64424509445 2024-04-17T09:53:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.475+0000 7ff5ab200640 1 -- 172.21.15.32:0/1172867371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff58c068380 msgr2=0x7ff58c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.475+0000 7ff5ab200640 1 --2- 172.21.15.32:0/1172867371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff58c068380 0x7ff58c06a840 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff59c0079a0 tx=0x7ff59c009040 comp rx=0 tx=0).stop 2024-04-17T09:53:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.475+0000 7ff5ab200640 1 -- 172.21.15.32:0/1172867371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ac0d3f10 msgr2=0x7ff5ac0cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.475+0000 7ff5ab200640 1 --2- 172.21.15.32:0/1172867371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ac0d3f10 0x7ff5ac0cdc10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff59800c3f0 tx=0x7ff59800c8b0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.475+0000 7ff5ab200640 1 -- 172.21.15.32:0/1172867371 shutdown_connections 2024-04-17T09:53:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.475+0000 7ff5ab200640 1 --2- 172.21.15.32:0/1172867371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff58c068380 0x7ff58c06a840 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.475+0000 7ff5ab200640 1 --2- 172.21.15.32:0/1172867371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5ac0cf470 0x7ff5ac155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.475+0000 7ff5ab200640 1 --2- 172.21.15.32:0/1172867371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ac0d3f10 0x7ff5ac0cdc10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.475+0000 7ff5ab200640 1 --2- 172.21.15.32:0/1172867371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5ac0b66c0 0x7ff5ac161990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.475+0000 7ff5ab200640 1 -- 172.21.15.32:0/1172867371 >> 172.21.15.32:0/1172867371 conn(0x7ff5ac0d4530 msgr2=0x7ff5ac0b5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.475+0000 7ff5ab200640 1 -- 172.21.15.32:0/1172867371 shutdown_connections 2024-04-17T09:53:20.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.475+0000 7ff5ab200640 1 -- 172.21.15.32:0/1172867371 wait complete. 2024-04-17T09:53:20.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.6 2024-04-17T09:53:20.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.4 2024-04-17T09:53:20.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.483+0000 7f5b95000640 1 -- 172.21.15.32:0/1336741048 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f5b68005590 con 0x7f5b880e78f0 2024-04-17T09:53:20.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.487+0000 7f5b86000640 1 -- 172.21.15.32:0/1336741048 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5b7c0c2030 con 0x7f5b880e78f0 2024-04-17T09:53:20.489 INFO:teuthology.orchestra.run.smithi032.stdout:64424509445 2024-04-17T09:53:20.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.487+0000 7f5b95000640 1 -- 172.21.15.32:0/1336741048 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5b70068380 msgr2=0x7f5b7006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.487+0000 7f5b95000640 1 --2- 172.21.15.32:0/1336741048 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5b70068380 0x7f5b7006a840 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5b9006a920 tx=0x7f5b9008d040 comp rx=0 tx=0).stop 2024-04-17T09:53:20.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.487+0000 7f5b95000640 1 -- 172.21.15.32:0/1336741048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b880e78f0 msgr2=0x7f5b8818c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:20.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.487+0000 7f5b95000640 1 --2- 172.21.15.32:0/1336741048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b880e78f0 0x7f5b8818c140 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0ab000 tx=0x7f5b7c018e40 comp rx=0 tx=0).stop 2024-04-17T09:53:20.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.487+0000 7f5b95000640 1 -- 172.21.15.32:0/1336741048 shutdown_connections 2024-04-17T09:53:20.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.487+0000 7f5b95000640 1 --2- 172.21.15.32:0/1336741048 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5b70068380 0x7f5b7006a840 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.487+0000 7f5b95000640 1 --2- 172.21.15.32:0/1336741048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b880e78f0 0x7f5b8818c140 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.487+0000 7f5b95000640 1 --2- 172.21.15.32:0/1336741048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b880e3290 0x7f5b8804ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.487+0000 7f5b95000640 1 --2- 172.21.15.32:0/1336741048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b880383a0 0x7f5b88052990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:20.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.487+0000 7f5b95000640 1 -- 172.21.15.32:0/1336741048 >> 172.21.15.32:0/1336741048 conn(0x7f5b880559a0 msgr2=0x7f5b880d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:20.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.487+0000 7f5b95000640 1 -- 172.21.15.32:0/1336741048 shutdown_connections 2024-04-17T09:53:20.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:20.487+0000 7f5b95000640 1 -- 172.21.15.32:0/1336741048 wait complete. 2024-04-17T09:53:20.500 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.3 2024-04-17T09:53:20.511 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.7 2024-04-17T09:53:21.458 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-17T09:53:21.461 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-17T09:53:21.462 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-17T09:53:21.475 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-17T09:53:21.482 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-17T09:53:21.482 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-17T09:53:21.501 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-17T09:53:21.511 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-17T09:53:21.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.679+0000 7f0296800640 1 Processor -- start 2024-04-17T09:53:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.679+0000 7f9db0400640 1 Processor -- start 2024-04-17T09:53:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.679+0000 7f9db0400640 1 -- start start 2024-04-17T09:53:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.679+0000 7f9db0400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9da80d6810 0x7f9da8163030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.679+0000 7f9db0400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9da80d6bf0 0x7f9da8166620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.679+0000 7f9db0400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9da80b57f0 con 0x7f9da8073970 2024-04-17T09:53:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.679+0000 7f9db0400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9da80b5b20 con 0x7f9da80d6bf0 2024-04-17T09:53:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.679+0000 7f9db0400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9da8160ad0 con 0x7f9da80d6810 2024-04-17T09:53:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.679+0000 7f0296800640 1 -- start start 2024-04-17T09:53:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.679+0000 7f0296800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0288033e40 0x7f02880eb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.679+0000 7f0296800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0288005ea0 0x7f02880062b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.679+0000 7f0296800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0288034220 con 0x7f0288047c20 2024-04-17T09:53:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.679+0000 7f0296800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0288050210 con 0x7f0288005ea0 2024-04-17T09:53:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.679+0000 7f0296800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0288037bc0 con 0x7f0288033e40 2024-04-17T09:53:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.679+0000 7f9dada00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9da80d6810 0x7f9da8163030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.683+0000 7f9dada00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9da80d6810 0x7f9da8163030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:43880/0 (socket says 172.21.15.32:43880) 2024-04-17T09:53:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.683+0000 7f9dada00640 1 -- 172.21.15.32:0/2476972181 learned_addr learned my addr 172.21.15.32:0/2476972181 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.679+0000 7f0295400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0288033e40 0x7f02880eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.683+0000 7f0295400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0288033e40 0x7f02880eb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:55298/0 (socket says 172.21.15.32:55298) 2024-04-17T09:53:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.683+0000 7f0295400640 1 -- 172.21.15.32:0/3555468297 learned_addr learned my addr 172.21.15.32:0/3555468297 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.683+0000 7f9dad000640 1 --2- 172.21.15.32:0/2476972181 >> v2:172.21.15.32:3300/0 conn(0x7f9da80d6bf0 0x7f9da8166620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.683+0000 7f0295400640 1 -- 172.21.15.32:0/3555468297 >> v2:172.21.15.32:3300/0 conn(0x7f0288005ea0 msgr2=0x7f02880062b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.683+0000 7f0295400640 1 --2- 172.21.15.32:0/3555468297 >> v2:172.21.15.32:3300/0 conn(0x7f0288005ea0 0x7f02880062b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.683+0000 7f0295400640 1 -- 172.21.15.32:0/3555468297 >> v1:172.21.15.144:6789/0 conn(0x7f0288047c20 legacy=0x7f02880eb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.683+0000 7f0295400640 1 -- 172.21.15.32:0/3555468297 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028804fa30 con 0x7f0288033e40 2024-04-17T09:53:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.683+0000 7f028fe00640 1 -- 172.21.15.32:0/3555468297 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 877721959 0 0) 0x7f0288034220 con 0x7f0288047c20 2024-04-17T09:53:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.683+0000 7f9da7e00640 1 -- 172.21.15.32:0/2476972181 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 242967639 0 0) 0x7f9da80b57f0 con 0x7f9da8073970 2024-04-17T09:53:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.683+0000 7f9dada00640 1 -- 172.21.15.32:0/2476972181 >> v2:172.21.15.32:3300/0 conn(0x7f9da80d6bf0 msgr2=0x7f9da8166620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.683+0000 7f9dada00640 1 --2- 172.21.15.32:0/2476972181 >> v2:172.21.15.32:3300/0 conn(0x7f9da80d6bf0 0x7f9da8166620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.683+0000 7f9dada00640 1 -- 172.21.15.32:0/2476972181 >> v1:172.21.15.32:6789/0 conn(0x7f9da8073970 legacy=0x7f9da8166b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.683+0000 7f9dada00640 1 -- 172.21.15.32:0/2476972181 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da81638f0 con 0x7f9da80d6810 2024-04-17T09:53:21.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.687+0000 7efdc7c00640 1 Processor -- start 2024-04-17T09:53:21.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.687+0000 7f9dada00640 1 --2- 172.21.15.32:0/2476972181 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9da80d6810 0x7f9da8163030 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009e50 tx=0x7f9d9801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9378a492dd192e4e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.687+0000 7efdc7c00640 1 -- start start 2024-04-17T09:53:21.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.687+0000 7f9da7e00640 1 -- 172.21.15.32:0/2476972181 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d980a5740 con 0x7f9da80d6810 2024-04-17T09:53:21.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.687+0000 7f9da7e00640 1 -- 172.21.15.32:0/2476972181 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d980a58e0 con 0x7f9da80d6810 2024-04-17T09:53:21.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.687+0000 7f9da7e00640 1 -- 172.21.15.32:0/2476972181 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d980a5cb0 con 0x7f9da80d6810 2024-04-17T09:53:21.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7f0295400640 1 --2- 172.21.15.32:0/3555468297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0288033e40 0x7f02880eb180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0288054a30 tx=0x7f027c01ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c20c9c5b0bab1ad7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7f9db0400640 1 -- 172.21.15.32:0/2476972181 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9da80d6810 msgr2=0x7f9da8163030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7f9db0400640 1 --2- 172.21.15.32:0/2476972181 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9da80d6810 0x7f9da8163030 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009e50 tx=0x7f9d9801c970 comp rx=0 tx=0).stop 2024-04-17T09:53:21.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7f028fe00640 1 -- 172.21.15.32:0/3555468297 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c0b5070 con 0x7f0288033e40 2024-04-17T09:53:21.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7f9db0400640 1 -- 172.21.15.32:0/2476972181 shutdown_connections 2024-04-17T09:53:21.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7f9db0400640 1 --2- 172.21.15.32:0/2476972181 >> v2:172.21.15.32:3300/0 conn(0x7f9da80d6bf0 0x7f9da8166620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7f9db0400640 1 --2- 172.21.15.32:0/2476972181 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9da80d6810 0x7f9da8163030 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7f9db0400640 1 -- 172.21.15.32:0/2476972181 >> 172.21.15.32:0/2476972181 conn(0x7f9da8072050 msgr2=0x7f9da8072f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:21.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7efdc7c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efdc0072b00 0x7efdc00b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7f028fe00640 1 -- 172.21.15.32:0/3555468297 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f027c0a8a30 con 0x7f0288033e40 2024-04-17T09:53:21.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7f9db0400640 1 -- 172.21.15.32:0/2476972181 shutdown_connections 2024-04-17T09:53:21.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7f9db0400640 1 -- 172.21.15.32:0/2476972181 wait complete. 2024-04-17T09:53:21.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7efdbfe00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7efdc006f8f0 0x7efdc00d02e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:45708/0 (socket says 172.21.15.32:45708) 2024-04-17T09:53:21.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7efdbfe00640 1 -- 172.21.15.32:0/3742253799 learned_addr learned my addr 172.21.15.32:0/3742253799 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:21.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7efdc7c00640 1 -- 172.21.15.32:0/3742253799 --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdc0072ee0 con 0x7efdc00d09f0 2024-04-17T09:53:21.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7efdc7c00640 1 -- 172.21.15.32:0/3742253799 --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdc0166900 con 0x7efdc006f8f0 2024-04-17T09:53:21.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7efdc7c00640 1 -- 172.21.15.32:0/3742253799 --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7efdc0166be0 con 0x7efdc0072b00 2024-04-17T09:53:21.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7efdc5200640 1 --2- 172.21.15.32:0/3742253799 >> v2:172.21.15.144:3300/0 conn(0x7efdc0072b00 0x7efdc00b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7f9db0400640 1 Processor -- start 2024-04-17T09:53:21.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7efdbf400640 1 -- 172.21.15.32:0/3742253799 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1281989793 0 0) 0x7efdc0166900 con 0x7efdc006f8f0 2024-04-17T09:53:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7efdbf400640 1 -- 172.21.15.32:0/3742253799 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efda8003530 con 0x7efdc006f8f0 2024-04-17T09:53:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7efdbf400640 1 -- 172.21.15.32:0/3742253799 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2807261062 0 0) 0x7efdc0072ee0 con 0x7efdc00d09f0 2024-04-17T09:53:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7efdbf400640 1 -- 172.21.15.32:0/3742253799 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdc0166900 con 0x7efdc00d09f0 2024-04-17T09:53:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7efdc5200640 1 -- 172.21.15.32:0/3742253799 >> v1:172.21.15.144:6789/0 conn(0x7efdc006f8f0 legacy=0x7efdc00d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7efdc5200640 1 -- 172.21.15.32:0/3742253799 >> v1:172.21.15.32:6789/0 conn(0x7efdc00d09f0 legacy=0x7efdc0163200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7efdc5200640 1 -- 172.21.15.32:0/3742253799 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc0166ec0 con 0x7efdc0072b00 2024-04-17T09:53:21.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7f0296800640 1 -- 172.21.15.32:0/3555468297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0288033e40 msgr2=0x7f02880eb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7f028fe00640 1 -- 172.21.15.32:0/3555468297 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c0b5070 con 0x7f0288033e40 2024-04-17T09:53:21.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7f0296800640 1 --2- 172.21.15.32:0/3555468297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0288033e40 0x7f02880eb180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0288054a30 tx=0x7f027c01ed30 comp rx=0 tx=0).stop 2024-04-17T09:53:21.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7f0296800640 1 -- 172.21.15.32:0/3555468297 shutdown_connections 2024-04-17T09:53:21.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7f0296800640 1 --2- 172.21.15.32:0/3555468297 >> v2:172.21.15.32:3300/0 conn(0x7f0288005ea0 0x7f02880062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7f0296800640 1 --2- 172.21.15.32:0/3555468297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0288033e40 0x7f02880eb180 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7f0296800640 1 -- 172.21.15.32:0/3555468297 >> 172.21.15.32:0/3555468297 conn(0x7f0288009fc0 msgr2=0x7f028800a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:21.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7f0296800640 1 -- 172.21.15.32:0/3555468297 shutdown_connections 2024-04-17T09:53:21.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.691+0000 7f0296800640 1 -- 172.21.15.32:0/3555468297 wait complete. 2024-04-17T09:53:21.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7f0296800640 1 Processor -- start 2024-04-17T09:53:21.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7f9db0400640 1 -- start start 2024-04-17T09:53:21.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7f9db0400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da8073970 0x7f9da80bd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7f9db0400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9da80d6810 0x7f9da80ba7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7f9db0400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80d6bf0 0x7f9da80957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7f9db0400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81673f0 con 0x7f9da80d6bf0 2024-04-17T09:53:21.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7f9db0400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9da8167270 con 0x7f9da8073970 2024-04-17T09:53:21.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7f9db0400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9da816e1c0 con 0x7f9da80d6810 2024-04-17T09:53:21.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7f9dae400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80d6bf0 0x7f9da80957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7f9dae400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80d6bf0 0x7f9da80957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49038/0 (socket says 172.21.15.32:49038) 2024-04-17T09:53:21.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7f9dae400640 1 -- 172.21.15.32:0/1295795737 learned_addr learned my addr 172.21.15.32:0/1295795737 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:21.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7f9dada00640 1 --2- 172.21.15.32:0/1295795737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da8073970 0x7f9da80bd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7f9dad000640 1 --2- 172.21.15.32:0/1295795737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9da80d6810 0x7f9da80ba7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7f0296800640 1 -- start start 2024-04-17T09:53:21.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7f0296800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0288005ea0 0x7f02880d8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7f0296800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0288033e40 0x7f028804bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.695+0000 7f0296800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0288047c20 0x7f0288039130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f9dad000640 1 -- 172.21.15.32:0/1295795737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da8073970 msgr2=0x7f9da80bd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f9dad000640 1 --2- 172.21.15.32:0/1295795737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da8073970 0x7f9da80bd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f0294a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0288033e40 0x7f028804bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f0294a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0288033e40 0x7f028804bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:43902/0 (socket says 172.21.15.32:43902) 2024-04-17T09:53:21.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f0294a00640 1 -- 172.21.15.32:0/1235402588 learned_addr learned my addr 172.21.15.32:0/1235402588 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:21.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f0295e00640 1 --2- 172.21.15.32:0/1235402588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0288047c20 0x7f0288039130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f0296800640 1 -- 172.21.15.32:0/1235402588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f028804fd80 con 0x7f0288005ea0 2024-04-17T09:53:21.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f0296800640 1 -- 172.21.15.32:0/1235402588 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f028804f090 con 0x7f0288047c20 2024-04-17T09:53:21.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f0296800640 1 -- 172.21.15.32:0/1235402588 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f028804f5c0 con 0x7f0288033e40 2024-04-17T09:53:21.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f9dad000640 1 -- 172.21.15.32:0/1295795737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80d6bf0 msgr2=0x7f9da80957b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f9dad000640 1 --2- 172.21.15.32:0/1295795737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80d6bf0 0x7f9da80957b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f9dad000640 1 -- 172.21.15.32:0/1295795737 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da816d7e0 con 0x7f9da80d6810 2024-04-17T09:53:21.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f9dad000640 1 --2- 172.21.15.32:0/1295795737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9da80d6810 0x7f9da80ba7d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400d600 tx=0x7f9d9400dac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7efdc5200640 1 --2- 172.21.15.32:0/3742253799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdc0072b00 0x7efdc00b3de0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efdb40097c0 tx=0x7efdb401c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34c0c89a6661c592 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7efdbf400640 1 -- 172.21.15.32:0/3742253799 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb40b0070 con 0x7efdc0072b00 2024-04-17T09:53:21.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7efdbf400640 1 -- 172.21.15.32:0/3742253799 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb40ab600 con 0x7efdc0072b00 2024-04-17T09:53:21.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7efdbf400640 1 -- 172.21.15.32:0/3742253799 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb40ab9d0 con 0x7efdc0072b00 2024-04-17T09:53:21.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f0295400640 1 --2- 172.21.15.32:0/1235402588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0288005ea0 0x7f02880d8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f0295e00640 1 -- 172.21.15.32:0/1235402588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0288033e40 msgr2=0x7f028804bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f0295e00640 1 --2- 172.21.15.32:0/1235402588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0288033e40 0x7f028804bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f0295e00640 1 -- 172.21.15.32:0/1235402588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0288005ea0 msgr2=0x7f02880d8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f0295e00640 1 --2- 172.21.15.32:0/1235402588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0288005ea0 0x7f02880d8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f0295e00640 1 -- 172.21.15.32:0/1235402588 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028804c280 con 0x7f0288047c20 2024-04-17T09:53:21.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f0295400640 1 --2- 172.21.15.32:0/1235402588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0288005ea0 0x7f02880d8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:21.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f0295e00640 1 --2- 172.21.15.32:0/1235402588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0288047c20 0x7f0288039130 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0290050220 tx=0x7f0290069570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f028d600640 1 -- 172.21.15.32:0/1235402588 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0290068020 con 0x7f0288047c20 2024-04-17T09:53:21.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f028d600640 1 -- 172.21.15.32:0/1235402588 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0290068220 con 0x7f0288047c20 2024-04-17T09:53:21.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f028d600640 1 -- 172.21.15.32:0/1235402588 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0290067770 con 0x7f0288047c20 2024-04-17T09:53:21.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f9da5600640 1 -- 172.21.15.32:0/1295795737 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d940125f0 con 0x7f9da80d6810 2024-04-17T09:53:21.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f9da5600640 1 -- 172.21.15.32:0/1295795737 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d94019070 con 0x7f9da80d6810 2024-04-17T09:53:21.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f9da5600640 1 -- 172.21.15.32:0/1295795737 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94012990 con 0x7f9da80d6810 2024-04-17T09:53:21.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f9db0400640 1 -- 172.21.15.32:0/1295795737 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da80bba80 con 0x7f9da80d6810 2024-04-17T09:53:21.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.699+0000 7f9db0400640 1 -- 172.21.15.32:0/1295795737 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da80bc040 con 0x7f9da80d6810 2024-04-17T09:53:21.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.703+0000 7f0296800640 1 -- 172.21.15.32:0/1235402588 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028804c560 con 0x7f0288047c20 2024-04-17T09:53:21.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.703+0000 7f0296800640 1 -- 172.21.15.32:0/1235402588 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f028804c840 con 0x7f0288047c20 2024-04-17T09:53:21.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.703+0000 7f9db0400640 1 -- 172.21.15.32:0/1295795737 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9da804fd00 con 0x7f9da80d6810 2024-04-17T09:53:21.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.703+0000 7f9da5600640 1 -- 172.21.15.32:0/1295795737 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9d94012b30 con 0x7f9da80d6810 2024-04-17T09:53:21.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.703+0000 7efdc7c00640 1 -- 172.21.15.32:0/3742253799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdc0072b00 msgr2=0x7efdc00b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.703+0000 7efdc7c00640 1 --2- 172.21.15.32:0/3742253799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdc0072b00 0x7efdc00b3de0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efdb40097c0 tx=0x7efdb401c420 comp rx=0 tx=0).stop 2024-04-17T09:53:21.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.703+0000 7efdc7c00640 1 -- 172.21.15.32:0/3742253799 shutdown_connections 2024-04-17T09:53:21.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.703+0000 7f0296800640 1 -- 172.21.15.32:0/1235402588 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02880053a0 con 0x7f0288047c20 2024-04-17T09:53:21.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.703+0000 7efdc7c00640 1 --2- 172.21.15.32:0/3742253799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdc0072b00 0x7efdc00b3de0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.703+0000 7efdc7c00640 1 -- 172.21.15.32:0/3742253799 >> 172.21.15.32:0/3742253799 conn(0x7efdc006ba30 msgr2=0x7efdc006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:21.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.703+0000 7f9da5600640 1 --2- 172.21.15.32:0/1295795737 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9d88068380 0x7f9d8806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.703+0000 7f9da5600640 1 -- 172.21.15.32:0/1295795737 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9d94086c30 con 0x7f9da80d6810 2024-04-17T09:53:21.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.707+0000 7efdc7c00640 1 -- 172.21.15.32:0/3742253799 shutdown_connections 2024-04-17T09:53:21.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.703+0000 7f028d600640 1 -- 172.21.15.32:0/1235402588 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0290067910 con 0x7f0288047c20 2024-04-17T09:53:21.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.707+0000 7f028d600640 1 --2- 172.21.15.32:0/1235402588 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0270068400 0x7f027006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.707+0000 7f028d600640 1 -- 172.21.15.32:0/1235402588 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0290101500 con 0x7f0288047c20 2024-04-17T09:53:21.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.707+0000 7f9dada00640 1 --2- 172.21.15.32:0/1295795737 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9d88068380 0x7f9d8806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.707+0000 7f9dada00640 1 --2- 172.21.15.32:0/1295795737 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9d88068380 0x7f9d8806a840 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9d980ae000 tx=0x7f9d980a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.707+0000 7efdc7c00640 1 -- 172.21.15.32:0/3742253799 wait complete. 2024-04-17T09:53:21.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.707+0000 7f0295400640 1 --2- 172.21.15.32:0/1235402588 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0270068400 0x7f027006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.707+0000 7efdc7c00640 1 Processor -- start 2024-04-17T09:53:21.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.707+0000 7f0295400640 1 --2- 172.21.15.32:0/1235402588 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0270068400 0x7f027006a8c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f027c0b2000 tx=0x7f027c0aa070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.707+0000 7f9da5600640 1 -- 172.21.15.32:0/1295795737 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f9d94056b10 con 0x7f9da80d6810 2024-04-17T09:53:21.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.707+0000 7efdc7c00640 1 -- start start 2024-04-17T09:53:21.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.711+0000 7efdc7c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc006f8f0 0x7efdc00cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.711+0000 7efdc5200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc006f8f0 0x7efdc00cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.711+0000 7f028d600640 1 -- 172.21.15.32:0/1235402588 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0290098030 con 0x7f0288047c20 2024-04-17T09:53:21.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.711+0000 7efdc5200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc006f8f0 0x7efdc00cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49060/0 (socket says 172.21.15.32:49060) 2024-04-17T09:53:21.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.711+0000 7efdc5200640 1 -- 172.21.15.32:0/3139016043 learned_addr learned my addr 172.21.15.32:0/3139016043 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.711+0000 7efdc7c00640 1 --2- 172.21.15.32:0/3139016043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdc0072b00 0x7efdc0161900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.711+0000 7efdc7c00640 1 --2- 172.21.15.32:0/3139016043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdc00d09f0 0x7efdc0161e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.711+0000 7efdc7c00640 1 -- 172.21.15.32:0/3139016043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efdc0170d90 con 0x7efdc006f8f0 2024-04-17T09:53:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.711+0000 7efdc7c00640 1 -- 172.21.15.32:0/3139016043 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efdc0170c10 con 0x7efdc0072b00 2024-04-17T09:53:21.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.711+0000 7efdc7c00640 1 -- 172.21.15.32:0/3139016043 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efdc0170f10 con 0x7efdc00d09f0 2024-04-17T09:53:21.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.711+0000 7efdc5c00640 1 --2- 172.21.15.32:0/3139016043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdc00d09f0 0x7efdc0161e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.711+0000 7efdbfe00640 1 --2- 172.21.15.32:0/3139016043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdc0072b00 0x7efdc0161900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.715+0000 7efdc5200640 1 -- 172.21.15.32:0/3139016043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdc00d09f0 msgr2=0x7efdc0161e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.715+0000 7efdc5200640 1 --2- 172.21.15.32:0/3139016043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdc00d09f0 0x7efdc0161e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.715+0000 7efdc5200640 1 -- 172.21.15.32:0/3139016043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdc0072b00 msgr2=0x7efdc0161900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.715+0000 7efdc5200640 1 --2- 172.21.15.32:0/3139016043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdc0072b00 0x7efdc0161900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.715+0000 7efdc5200640 1 -- 172.21.15.32:0/3139016043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc015d6e0 con 0x7efdc006f8f0 2024-04-17T09:53:21.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.715+0000 7efdc5200640 1 --2- 172.21.15.32:0/3139016043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc006f8f0 0x7efdc00cd1a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efdb401aad0 tx=0x7efdb40191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.719+0000 7efdbcc00640 1 -- 172.21.15.32:0/3139016043 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb40b9070 con 0x7efdc006f8f0 2024-04-17T09:53:21.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.719+0000 7efdc7c00640 1 -- 172.21.15.32:0/3139016043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc013b690 con 0x7efdc006f8f0 2024-04-17T09:53:21.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.719+0000 7efdc7c00640 1 -- 172.21.15.32:0/3139016043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc015d120 con 0x7efdc006f8f0 2024-04-17T09:53:21.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.719+0000 7efdbcc00640 1 -- 172.21.15.32:0/3139016043 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb40b5480 con 0x7efdc006f8f0 2024-04-17T09:53:21.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.719+0000 7efdbcc00640 1 -- 172.21.15.32:0/3139016043 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb40b0040 con 0x7efdc006f8f0 2024-04-17T09:53:21.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.723+0000 7efdbcc00640 1 -- 172.21.15.32:0/3139016043 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7efdb40bc070 con 0x7efdc006f8f0 2024-04-17T09:53:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.723+0000 7efdbcc00640 1 --2- 172.21.15.32:0/3139016043 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efda0068450 0x7efda006a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.723+0000 7efdbcc00640 1 -- 172.21.15.32:0/3139016043 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efdb4120900 con 0x7efdc006f8f0 2024-04-17T09:53:21.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.727+0000 7efdbfe00640 1 --2- 172.21.15.32:0/3139016043 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efda0068450 0x7efda006a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.727+0000 7efdc7c00640 1 -- 172.21.15.32:0/3139016043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd9c0052a0 con 0x7efdc006f8f0 2024-04-17T09:53:21.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.731+0000 7efdbfe00640 1 --2- 172.21.15.32:0/3139016043 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efda0068450 0x7efda006a910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efdc015b450 tx=0x7efdac007f30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.735+0000 7efdbcc00640 1 -- 172.21.15.32:0/3139016043 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7efdb40c3030 con 0x7efdc006f8f0 2024-04-17T09:53:21.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.763+0000 7f8ff2200640 1 Processor -- start 2024-04-17T09:53:21.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.763+0000 7f7993600640 1 Processor -- start 2024-04-17T09:53:21.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f7993600640 1 -- start start 2024-04-17T09:53:21.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.763+0000 7f8ff2200640 1 -- start start 2024-04-17T09:53:21.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f8ff2200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8fec15c820 0x7f8fec0d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f8ff2200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fec0cdbd0 0x7f8fec09b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f8ff2200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8fec09bc60 0x7f8fec0d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f8ff2200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8fec09a9f0 con 0x7f8fec09bc60 2024-04-17T09:53:21.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f8ff2200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f8fec15cc00 con 0x7f8fec15c820 2024-04-17T09:53:21.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f8ff2200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8fec0ce340 con 0x7f8fec0cdbd0 2024-04-17T09:53:21.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f7993600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f79840e7850 0x7f79840382c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f8febe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8fec09bc60 0x7f8fec0d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f8febe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8fec09bc60 0x7f8fec0d0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49066/0 (socket says 172.21.15.32:49066) 2024-04-17T09:53:21.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7fabc5600640 1 Processor -- start 2024-04-17T09:53:21.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f8febe00640 1 -- 172.21.15.32:0/3310227351 learned_addr learned my addr 172.21.15.32:0/3310227351 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:21.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f8feb400640 1 --2- 172.21.15.32:0/3310227351 >> v2:172.21.15.144:3300/0 conn(0x7f8fec15c820 0x7f8fec0d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f7993600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79840061e0 0x7f7984006670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7fabc5600640 1 -- start start 2024-04-17T09:53:21.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f7990c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f79840e7850 0x7f79840382c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f8feb400640 1 -- 172.21.15.32:0/3310227351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fec0cdbd0 msgr2=0x7f8fec09b720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T09:53:21.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f8feb400640 1 --2- 172.21.15.32:0/3310227351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fec0cdbd0 0x7f8fec09b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f8feb400640 1 -- 172.21.15.32:0/3310227351 >> v2:172.21.15.32:3300/0 conn(0x7f8fec09bc60 msgr2=0x7f8fec0d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:21.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f8feb400640 1 --2- 172.21.15.32:0/3310227351 >> v2:172.21.15.32:3300/0 conn(0x7f8fec09bc60 0x7f8fec0d0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f7993600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79840e7c30 con 0x7f79840eb340 2024-04-17T09:53:21.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f8feb400640 1 -- 172.21.15.32:0/3310227351 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fec09ac40 con 0x7f8fec15c820 2024-04-17T09:53:21.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f7993600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f7984057f60 con 0x7f79840e7850 2024-04-17T09:53:21.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f7993600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7984058240 con 0x7f79840061e0 2024-04-17T09:53:21.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f7990c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f79840e7850 0x7f79840382c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:55350/0 (socket says 172.21.15.32:55350) 2024-04-17T09:53:21.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f7990c00640 1 -- 172.21.15.32:0/4167781409 learned_addr learned my addr 172.21.15.32:0/4167781409 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:21.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.767+0000 7f798be00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f79840eb340 0x7f79840eb7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:45714/0 (socket says 172.21.15.32:45714) 2024-04-17T09:53:21.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.771+0000 7f7991600640 1 --2- 172.21.15.32:0/4167781409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79840061e0 0x7f7984006670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.771+0000 7f798b400640 1 -- 172.21.15.32:0/4167781409 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1763778226 0 0) 0x7f79840e7c30 con 0x7f79840eb340 2024-04-17T09:53:21.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.771+0000 7f798b400640 1 -- 172.21.15.32:0/4167781409 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7974003530 con 0x7f79840eb340 2024-04-17T09:53:21.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.771+0000 7fabc5600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fabc015a940 0x7fabc00ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.771+0000 7fabc5600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabc00afb70 0x7fabc00ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.771+0000 7fabc5600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc00ca780 con 0x7fabc00ccb00 2024-04-17T09:53:21.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.771+0000 7fabc5600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fabc00b00d0 con 0x7fabc015a940 2024-04-17T09:53:21.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.771+0000 7fabc5600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fabc00caa00 con 0x7fabc00afb70 2024-04-17T09:53:21.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.771+0000 7fabb7400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabc00afb70 0x7fabc00ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.771+0000 7fabb7400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabc00afb70 0x7fabc00ce570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:43924/0 (socket says 172.21.15.32:43924) 2024-04-17T09:53:21.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.771+0000 7fabb7400640 1 -- 172.21.15.32:0/1077054199 learned_addr learned my addr 172.21.15.32:0/1077054199 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:21.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.771+0000 7f7990c00640 1 -- 172.21.15.32:0/4167781409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79840061e0 msgr2=0x7f7984006670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:21.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.771+0000 7f7990c00640 1 --2- 172.21.15.32:0/4167781409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79840061e0 0x7f7984006670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.771+0000 7f7990c00640 1 -- 172.21.15.32:0/4167781409 >> v1:172.21.15.144:6789/0 conn(0x7f79840eb340 legacy=0x7f79840eb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.771+0000 7f7990c00640 1 -- 172.21.15.32:0/4167781409 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7984058520 con 0x7f79840e7850 2024-04-17T09:53:21.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.775+0000 7fabbfe00640 1 --2- 172.21.15.32:0/1077054199 >> v2:172.21.15.32:3300/0 conn(0x7fabc015a940 0x7fabc00ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.775+0000 7f8feb400640 1 --2- 172.21.15.32:0/3310227351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fec15c820 0x7f8fec0d0540 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0d36b0 tx=0x7f8fd8020880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22d8d15905cdb5c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.775+0000 7fabb7400640 1 -- 172.21.15.32:0/1077054199 >> v2:172.21.15.32:3300/0 conn(0x7fabc015a940 msgr2=0x7fabc00ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:21.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.775+0000 7fabb7400640 1 --2- 172.21.15.32:0/1077054199 >> v2:172.21.15.32:3300/0 conn(0x7fabc015a940 0x7fabc00ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.775+0000 7fabb7400640 1 -- 172.21.15.32:0/1077054199 >> v1:172.21.15.32:6789/0 conn(0x7fabc00ccb00 legacy=0x7fabc00cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.775+0000 7f8fea000640 1 -- 172.21.15.32:0/3310227351 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd80b6070 con 0x7f8fec15c820 2024-04-17T09:53:21.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.775+0000 7f8fea000640 1 -- 172.21.15.32:0/3310227351 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fd80a9710 con 0x7f8fec15c820 2024-04-17T09:53:21.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.775+0000 7f8fea000640 1 -- 172.21.15.32:0/3310227351 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd80a9ae0 con 0x7f8fec15c820 2024-04-17T09:53:21.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.775+0000 7fabb7400640 1 -- 172.21.15.32:0/1077054199 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc00cace0 con 0x7fabc00afb70 2024-04-17T09:53:21.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.775+0000 7fabbf400640 1 -- 172.21.15.32:0/1077054199 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 39358402 0 0) 0x7fabb8001120 con 0x7fabc00ccb00 2024-04-17T09:53:21.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.775+0000 7f7990c00640 1 --2- 172.21.15.32:0/4167781409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79840e7850 0x7f79840382c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7980009870 tx=0x7f798001c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f601f8d61f041e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.775+0000 7f8ff2200640 1 -- 172.21.15.32:0/3310227351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fec15c820 msgr2=0x7f8fec0d0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.775+0000 7f8ff2200640 1 --2- 172.21.15.32:0/3310227351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fec15c820 0x7f8fec0d0540 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0d36b0 tx=0x7f8fd8020880 comp rx=0 tx=0).stop 2024-04-17T09:53:21.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.775+0000 7f8ff2200640 1 -- 172.21.15.32:0/3310227351 shutdown_connections 2024-04-17T09:53:21.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.775+0000 7f8ff2200640 1 --2- 172.21.15.32:0/3310227351 >> v2:172.21.15.32:3300/0 conn(0x7f8fec09bc60 0x7f8fec0d0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.775+0000 7f8ff2200640 1 --2- 172.21.15.32:0/3310227351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fec0cdbd0 0x7f8fec09b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.775+0000 7f8ff2200640 1 --2- 172.21.15.32:0/3310227351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fec15c820 0x7f8fec0d0540 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0d36b0 tx=0x7f8fd8020880 comp rx=0 tx=0).stop 2024-04-17T09:53:21.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.775+0000 7f8ff2200640 1 -- 172.21.15.32:0/3310227351 >> 172.21.15.32:0/3310227351 conn(0x7f8fec0b40a0 msgr2=0x7f8fec0b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:21.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.775+0000 7f8ff2200640 1 -- 172.21.15.32:0/3310227351 shutdown_connections 2024-04-17T09:53:21.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.775+0000 7f798b400640 1 -- 172.21.15.32:0/4167781409 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79800b0070 con 0x7f79840e7850 2024-04-17T09:53:21.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.775+0000 7f8f2ce00640 1 Processor -- start 2024-04-17T09:53:21.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.775+0000 7f8f2ce00640 1 -- start start 2024-04-17T09:53:21.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.779+0000 7f798b400640 1 -- 172.21.15.32:0/4167781409 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79800ab600 con 0x7f79840e7850 2024-04-17T09:53:21.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.779+0000 7f7993600640 1 -- 172.21.15.32:0/4167781409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79840e7850 msgr2=0x7f79840382c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.779+0000 7f7993600640 1 --2- 172.21.15.32:0/4167781409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79840e7850 0x7f79840382c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7980009870 tx=0x7f798001c3f0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.779+0000 7f7993600640 1 -- 172.21.15.32:0/4167781409 shutdown_connections 2024-04-17T09:53:21.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.779+0000 7f7993600640 1 --2- 172.21.15.32:0/4167781409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79840061e0 0x7f7984006670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.779+0000 7f7993600640 1 --2- 172.21.15.32:0/4167781409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79840e7850 0x7f79840382c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.779+0000 7f7993600640 1 -- 172.21.15.32:0/4167781409 >> 172.21.15.32:0/4167781409 conn(0x7f7984055a20 msgr2=0x7f7984055e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:21.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.779+0000 7f8f2ce00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f2806f880 0x7f8f2816f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.779+0000 7f7993600640 1 -- 172.21.15.32:0/4167781409 shutdown_connections 2024-04-17T09:53:21.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.779+0000 7f7993600640 1 -- 172.21.15.32:0/4167781409 wait complete. 2024-04-17T09:53:21.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.779+0000 7f8f2ce00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f280b3f60 con 0x7f8f2816e7d0 2024-04-17T09:53:21.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.779+0000 7f8f2ce00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f280b4240 con 0x7f8f2816f620 2024-04-17T09:53:21.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.779+0000 7f8f2ce00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8f280728c0 con 0x7f8f2806f880 2024-04-17T09:53:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.779+0000 7f8f25400640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f8f2816f620 0x7f8f2816e0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:45720/0 (socket says 172.21.15.32:45720) 2024-04-17T09:53:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.779+0000 7f8f25400640 1 -- 172.21.15.32:0/4266784098 learned_addr learned my addr 172.21.15.32:0/4266784098 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.779+0000 7f8ff2200640 1 -- 172.21.15.32:0/3310227351 wait complete. 2024-04-17T09:53:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.779+0000 7f8f24a00640 1 -- 172.21.15.32:0/4266784098 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1189229087 0 0) 0x7f8f280b4240 con 0x7f8f2816f620 2024-04-17T09:53:21.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8f24a00640 1 -- 172.21.15.32:0/4266784098 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f0c003530 con 0x7f8f2816f620 2024-04-17T09:53:21.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7fd604600640 1 Processor -- start 2024-04-17T09:53:21.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8f24a00640 1 -- 172.21.15.32:0/4266784098 <== mon.1 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4174561554 0 0) 0x7f8f0c003530 con 0x7f8f2816f620 2024-04-17T09:53:21.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8f24a00640 1 -- 172.21.15.32:0/4266784098 >> v2:172.21.15.32:3300/0 conn(0x7f8f2806f880 msgr2=0x7f8f2816f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:21.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8f24a00640 1 --2- 172.21.15.32:0/4266784098 >> v2:172.21.15.32:3300/0 conn(0x7f8f2806f880 0x7f8f2816f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8f24a00640 1 -- 172.21.15.32:0/4266784098 >> v1:172.21.15.32:6789/0 conn(0x7f8f2816e7d0 legacy=0x7f8f2816b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:21.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8f24a00640 1 -- 172.21.15.32:0/4266784098 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f28072ba0 con 0x7f8f2816f620 2024-04-17T09:53:21.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8f24a00640 1 -- 172.21.15.32:0/4266784098 <== mon.1 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f8f14001680 con 0x7f8f2816f620 2024-04-17T09:53:21.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8f24a00640 1 -- 172.21.15.32:0/4266784098 >> v1:172.21.15.144:6789/0 conn(0x7f8f2816f620 legacy=0x7f8f2816e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8f24a00640 1 --2- 172.21.15.32:0/4266784098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f0c0039c0 0x7f8f0c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8f24a00640 1 --2- 172.21.15.32:0/4266784098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f0c004810 0x7f8f0c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8f24a00640 1 --2- 172.21.15.32:0/4266784098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f0c0053d0 0x7f8f0c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8f24a00640 1 -- 172.21.15.32:0/4266784098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8f280728c0 con 0x7f8f0c0039c0 2024-04-17T09:53:21.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8f24a00640 1 -- 172.21.15.32:0/4266784098 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c00c2b0 con 0x7f8f0c0053d0 2024-04-17T09:53:21.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8f24a00640 1 -- 172.21.15.32:0/4266784098 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8f0c00c4b0 con 0x7f8f0c004810 2024-04-17T09:53:21.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f7993600640 1 Processor -- start 2024-04-17T09:53:21.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8ff2200640 1 Processor -- start 2024-04-17T09:53:21.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f7993600640 1 -- start start 2024-04-17T09:53:21.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7fd604600640 1 -- start start 2024-04-17T09:53:21.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8f26800640 1 --2- 172.21.15.32:0/4266784098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f0c0053d0 0x7f8f0c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8f26800640 1 -- 172.21.15.32:0/4266784098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f0c004810 msgr2=0x7f8f0c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:21.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8f25400640 1 --2- 172.21.15.32:0/4266784098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f0c004810 0x7f8f0c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8f26800640 1 --2- 172.21.15.32:0/4266784098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f0c004810 0x7f8f0c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8f26800640 1 -- 172.21.15.32:0/4266784098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f0c0039c0 msgr2=0x7f8f0c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:21.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8f26800640 1 --2- 172.21.15.32:0/4266784098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f0c0039c0 0x7f8f0c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8f26800640 1 -- 172.21.15.32:0/4266784098 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f280b3f60 con 0x7f8f0c0053d0 2024-04-17T09:53:21.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f7993600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79840061e0 0x7f798418b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f7993600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79840e7850 0x7f79840deea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f7993600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79840eb340 0x7f79840e5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f7993600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7984054730 con 0x7f79840eb340 2024-04-17T09:53:21.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f7993600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7984058d20 con 0x7f79840e7850 2024-04-17T09:53:21.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f7993600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f798403ed80 con 0x7f79840061e0 2024-04-17T09:53:21.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7fd604600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd5fc15a630 0x7fd5fc15aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7fd604600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0b3dc0 0x7fd5fc0b4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7fd604600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5fc06f310 con 0x7fd5fc06c550 2024-04-17T09:53:21.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7fd604600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd5fc15b210 con 0x7fd5fc15a630 2024-04-17T09:53:21.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7fd604600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd5fc06f5f0 con 0x7fd5fc0b3dc0 2024-04-17T09:53:21.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8f25400640 1 --2- 172.21.15.32:0/4266784098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f0c004810 0x7f8f0c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:21.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8ff2200640 1 -- start start 2024-04-17T09:53:21.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8ff2200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fec09bc60 0x7f8fec09fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8ff2200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fec0cdbd0 0x7f8fec15bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f8ff2200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fec1fcef0 0x7f8fec0cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.779+0000 7fabb7400640 1 --2- 172.21.15.32:0/1077054199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabc00afb70 0x7fabc00ce570 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fabb000d370 tx=0x7fabb001eda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72acec7722fefd45 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7fabbf400640 1 -- 172.21.15.32:0/1077054199 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb00ad070 con 0x7fabc00afb70 2024-04-17T09:53:21.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7fabbf400640 1 -- 172.21.15.32:0/1077054199 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabb00b1400 con 0x7fabc00afb70 2024-04-17T09:53:21.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7fabbf400640 1 -- 172.21.15.32:0/1077054199 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb000be60 con 0x7fabc00afb70 2024-04-17T09:53:21.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7fabc5600640 1 -- 172.21.15.32:0/1077054199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabc00afb70 msgr2=0x7fabc00ce570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7fabc5600640 1 --2- 172.21.15.32:0/1077054199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabc00afb70 0x7fabc00ce570 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fabb000d370 tx=0x7fabb001eda0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7fabc5600640 1 -- 172.21.15.32:0/1077054199 shutdown_connections 2024-04-17T09:53:21.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7fabc5600640 1 --2- 172.21.15.32:0/1077054199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabc00afb70 0x7fabc00ce570 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fabb000d370 tx=0x7fabb001eda0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7fabc5600640 1 --2- 172.21.15.32:0/1077054199 >> v2:172.21.15.32:3300/0 conn(0x7fabc015a940 0x7fabc00ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7fabc5600640 1 -- 172.21.15.32:0/1077054199 >> 172.21.15.32:0/1077054199 conn(0x7fabc00b1d00 msgr2=0x7fabc00b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:21.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7fd601200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd5fc15a630 0x7fd5fc15aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7fd602600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0b3dc0 0x7fd5fc0b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f8feb400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fec09bc60 0x7f8fec09fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f8feb400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fec09bc60 0x7f8fec09fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:43958/0 (socket says 172.21.15.32:43958) 2024-04-17T09:53:21.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f7990c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79840061e0 0x7f798418b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f7990c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79840061e0 0x7f798418b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:43950/0 (socket says 172.21.15.32:43950) 2024-04-17T09:53:21.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f7990c00640 1 -- 172.21.15.32:0/2355096577 learned_addr learned my addr 172.21.15.32:0/2355096577 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:21.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7fd602600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0b3dc0 0x7fd5fc0b4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:43964/0 (socket says 172.21.15.32:43964) 2024-04-17T09:53:21.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f8feb400640 1 -- 172.21.15.32:0/3588017059 learned_addr learned my addr 172.21.15.32:0/3588017059 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f798be00640 1 --2- 172.21.15.32:0/2355096577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79840e7850 0x7f79840deea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f7990c00640 1 -- 172.21.15.32:0/2355096577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79840e7850 msgr2=0x7f79840deea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.783+0000 7f7990c00640 1 --2- 172.21.15.32:0/2355096577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79840e7850 0x7f79840deea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7fabc5600640 1 -- 172.21.15.32:0/1077054199 shutdown_connections 2024-04-17T09:53:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7fd601200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd5fc15a630 0x7fd5fc15aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:55378/0 (socket says 172.21.15.32:55378) 2024-04-17T09:53:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7fd601200640 1 -- 172.21.15.32:0/2217683442 learned_addr learned my addr 172.21.15.32:0/2217683442 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f7990c00640 1 -- 172.21.15.32:0/2355096577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79840eb340 msgr2=0x7f79840e5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f7990c00640 1 --2- 172.21.15.32:0/2355096577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79840eb340 0x7f79840e5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f7990c00640 1 -- 172.21.15.32:0/2355096577 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79840df6f0 con 0x7f79840061e0 2024-04-17T09:53:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f8febe00640 1 --2- 172.21.15.32:0/3588017059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fec1fcef0 0x7f8fec0cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f8ff2200640 1 -- 172.21.15.32:0/3588017059 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec160320 con 0x7f8fec0cdbd0 2024-04-17T09:53:21.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f8ff2200640 1 -- 172.21.15.32:0/3588017059 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec06eb50 con 0x7f8fec1fcef0 2024-04-17T09:53:21.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f8ff2200640 1 -- 172.21.15.32:0/3588017059 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8fec1604a0 con 0x7f8fec09bc60 2024-04-17T09:53:21.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f7990c00640 1 --2- 172.21.15.32:0/2355096577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79840061e0 0x7f798418b4d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f798001aaa0 tx=0x7f7980019190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f8feaa00640 1 --2- 172.21.15.32:0/3588017059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fec0cdbd0 0x7f8fec15bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f7988c00640 1 -- 172.21.15.32:0/2355096577 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79800b0050 con 0x7f79840061e0 2024-04-17T09:53:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f7988c00640 1 -- 172.21.15.32:0/2355096577 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79800ba070 con 0x7f79840061e0 2024-04-17T09:53:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f7988c00640 1 -- 172.21.15.32:0/2355096577 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79800b5440 con 0x7f79840061e0 2024-04-17T09:53:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f7993600640 1 -- 172.21.15.32:0/2355096577 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79840df9d0 con 0x7f79840061e0 2024-04-17T09:53:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7fabc5600640 1 -- 172.21.15.32:0/1077054199 wait complete. 2024-04-17T09:53:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.791+0000 7fabc5600640 1 Processor -- start 2024-04-17T09:53:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7fd601200640 1 -- 172.21.15.32:0/2217683442 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0b3dc0 msgr2=0x7fd5fc0b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7fd601200640 1 --2- 172.21.15.32:0/2217683442 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0b3dc0 0x7fd5fc0b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7fd601200640 1 -- 172.21.15.32:0/2217683442 >> v1:172.21.15.144:6789/0 conn(0x7fd5fc06c550 legacy=0x7fd5fc06e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7fd601200640 1 -- 172.21.15.32:0/2217683442 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5fc15e620 con 0x7fd5fc15a630 2024-04-17T09:53:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f8febe00640 1 -- 172.21.15.32:0/3588017059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fec09bc60 msgr2=0x7f8fec09fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f8febe00640 1 --2- 172.21.15.32:0/3588017059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fec09bc60 0x7f8fec09fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f8febe00640 1 -- 172.21.15.32:0/3588017059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fec0cdbd0 msgr2=0x7f8fec15bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f8febe00640 1 --2- 172.21.15.32:0/3588017059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fec0cdbd0 0x7f8fec15bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f8febe00640 1 -- 172.21.15.32:0/3588017059 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fec09c420 con 0x7f8fec1fcef0 2024-04-17T09:53:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f8feaa00640 1 --2- 172.21.15.32:0/3588017059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fec0cdbd0 0x7f8fec15bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f8febe00640 1 --2- 172.21.15.32:0/3588017059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fec1fcef0 0x7f8fec0cb7e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400ef90 tx=0x7f8fe400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f8fe3400640 1 -- 172.21.15.32:0/3588017059 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe400cd20 con 0x7f8fec1fcef0 2024-04-17T09:53:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f8fe3400640 1 -- 172.21.15.32:0/3588017059 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe40039a0 con 0x7f8fec1fcef0 2024-04-17T09:53:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f8fe3400640 1 -- 172.21.15.32:0/3588017059 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4003d70 con 0x7f8fec1fcef0 2024-04-17T09:53:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f8ff2200640 1 -- 172.21.15.32:0/3588017059 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fec09c700 con 0x7f8fec1fcef0 2024-04-17T09:53:21.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f8ff2200640 1 -- 172.21.15.32:0/3588017059 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fec09ccc0 con 0x7f8fec1fcef0 2024-04-17T09:53:21.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f7993600640 1 -- 172.21.15.32:0/2355096577 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f798418ba10 con 0x7f79840061e0 2024-04-17T09:53:21.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.787+0000 7f7993600640 1 -- 172.21.15.32:0/2355096577 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79840dfc10 con 0x7f79840061e0 2024-04-17T09:53:21.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.791+0000 7f8ff2200640 1 -- 172.21.15.32:0/3588017059 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fec068a90 con 0x7f8fec1fcef0 2024-04-17T09:53:21.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.791+0000 7fabc5600640 1 -- start start 2024-04-17T09:53:21.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.791+0000 7f8f26800640 1 -- 172.21.15.32:0/4266784098 reap_dead start 2024-04-17T09:53:21.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.791+0000 7f8f26800640 1 --2- 172.21.15.32:0/4266784098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f0c0053d0 0x7f8f0c00b9d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8f18001650 tx=0x7f8f1801c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7920dd6ed4d4e91 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fabc5600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc00ccb00 0x7fabc0098cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fabc5600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabc015a940 0x7fabc015a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fabc5600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabc015a560 0x7fabc0158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fd601200640 1 --2- 172.21.15.32:0/2217683442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5fc15a630 0x7fd5fc15aa90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009870 tx=0x7fd5e801ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=549353a7d66997c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fd5fbe00640 1 -- 172.21.15.32:0/2217683442 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e80b5070 con 0x7fd5fc15a630 2024-04-17T09:53:21.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fd5fbe00640 1 -- 172.21.15.32:0/2217683442 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5e80a8a50 con 0x7fd5fc15a630 2024-04-17T09:53:21.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fd5fbe00640 1 -- 172.21.15.32:0/2217683442 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e80a8e20 con 0x7fd5fc15a630 2024-04-17T09:53:21.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.791+0000 7f8f24a00640 1 -- 172.21.15.32:0/4266784098 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18001250 con 0x7f8f0c0053d0 2024-04-17T09:53:21.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.791+0000 7f8f24a00640 1 -- 172.21.15.32:0/4266784098 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f18005270 con 0x7f8f0c0053d0 2024-04-17T09:53:21.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.791+0000 7f8f24a00640 1 -- 172.21.15.32:0/4266784098 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18001250 con 0x7f8f0c0053d0 2024-04-17T09:53:21.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7f8f2ce00640 1 -- 172.21.15.32:0/4266784098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f0c0053d0 msgr2=0x7f8f0c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7f8f2ce00640 1 --2- 172.21.15.32:0/4266784098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f0c0053d0 0x7f8f0c00b9d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8f18001650 tx=0x7f8f1801c300 comp rx=0 tx=0).stop 2024-04-17T09:53:21.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.791+0000 7f8fe3400640 1 -- 172.21.15.32:0/3588017059 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8fe4012050 con 0x7f8fec1fcef0 2024-04-17T09:53:21.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7f8fe3400640 1 --2- 172.21.15.32:0/3588017059 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8fcc068400 0x7f8fcc06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fabbfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc00ccb00 0x7fabc0098cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fabbfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc00ccb00 0x7fabc0098cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49116/0 (socket says 172.21.15.32:49116) 2024-04-17T09:53:21.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7f7988c00640 1 -- 172.21.15.32:0/2355096577 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f79800ab600 con 0x7f79840061e0 2024-04-17T09:53:21.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7f8fe3400640 1 -- 172.21.15.32:0/3588017059 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8fe4054d90 con 0x7f8fec1fcef0 2024-04-17T09:53:21.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fabb7400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabc015a940 0x7fabc015a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fabb7400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabc015a940 0x7fabc015a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:43968/0 (socket says 172.21.15.32:43968) 2024-04-17T09:53:21.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fabb7400640 1 -- 172.21.15.32:0/469080588 learned_addr learned my addr 172.21.15.32:0/469080588 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:21.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fabc5600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fabc006c7a0 con 0x7fabc00ccb00 2024-04-17T09:53:21.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fabc5600640 1 -- 172.21.15.32:0/469080588 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fabc0073e20 con 0x7fabc015a560 2024-04-17T09:53:21.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fabc5600640 1 -- 172.21.15.32:0/469080588 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fabc006c920 con 0x7fabc015a940 2024-04-17T09:53:21.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fabc4c00640 1 --2- 172.21.15.32:0/469080588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabc015a560 0x7fabc0158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7f8f2ce00640 1 -- 172.21.15.32:0/4266784098 shutdown_connections 2024-04-17T09:53:21.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7f8f2ce00640 1 --2- 172.21.15.32:0/4266784098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f0c0053d0 0x7f8f0c00b9d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7f8f2ce00640 1 -- 172.21.15.32:0/4266784098 >> 172.21.15.32:0/4266784098 conn(0x7f8f2806ba30 msgr2=0x7f8f2806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:21.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7f7988c00640 1 --2- 172.21.15.32:0/2355096577 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f796c068380 0x7f796c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fabb7400640 1 -- 172.21.15.32:0/469080588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabc015a560 msgr2=0x7fabc0158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fabb7400640 1 --2- 172.21.15.32:0/469080588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabc015a560 0x7fabc0158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fabb7400640 1 -- 172.21.15.32:0/469080588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc00ccb00 msgr2=0x7fabc0098cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fabb7400640 1 --2- 172.21.15.32:0/469080588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc00ccb00 0x7fabc0098cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fabb7400640 1 -- 172.21.15.32:0/469080588 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc00c5f70 con 0x7fabc015a940 2024-04-17T09:53:21.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7f8f2ce00640 1 -- 172.21.15.32:0/4266784098 shutdown_connections 2024-04-17T09:53:21.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7f7988c00640 1 -- 172.21.15.32:0/2355096577 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7980121500 con 0x7f79840061e0 2024-04-17T09:53:21.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fd604600640 1 -- 172.21.15.32:0/2217683442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5fc15a630 msgr2=0x7fd5fc15aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fd604600640 1 --2- 172.21.15.32:0/2217683442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5fc15a630 0x7fd5fc15aa90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009870 tx=0x7fd5e801ed90 comp rx=0 tx=0).stop 2024-04-17T09:53:21.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fabb7400640 1 --2- 172.21.15.32:0/469080588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabc015a940 0x7fabc015a020 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fabb00b3040 tx=0x7fabb001bb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fd604600640 1 -- 172.21.15.32:0/2217683442 shutdown_connections 2024-04-17T09:53:21.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fd604600640 1 --2- 172.21.15.32:0/2217683442 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0b3dc0 0x7fd5fc0b4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fd604600640 1 --2- 172.21.15.32:0/2217683442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5fc15a630 0x7fd5fc15aa90 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009870 tx=0x7fd5e801ed90 comp rx=0 tx=0).stop 2024-04-17T09:53:21.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fd604600640 1 -- 172.21.15.32:0/2217683442 >> 172.21.15.32:0/2217683442 conn(0x7fd5fc06ba30 msgr2=0x7fd5fc06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:21.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fd604600640 1 -- 172.21.15.32:0/2217683442 shutdown_connections 2024-04-17T09:53:21.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7fd604600640 1 -- 172.21.15.32:0/2217683442 wait complete. 2024-04-17T09:53:21.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7f8f2ce00640 1 -- 172.21.15.32:0/4266784098 wait complete. 2024-04-17T09:53:21.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7f8feb400640 1 --2- 172.21.15.32:0/3588017059 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8fcc068400 0x7f8fcc06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.795+0000 7f798be00640 1 --2- 172.21.15.32:0/2355096577 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f796c068380 0x7f796c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.799+0000 7fabbcc00640 1 -- 172.21.15.32:0/469080588 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb00c0070 con 0x7fabc015a940 2024-04-17T09:53:21.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.799+0000 7fabc5600640 1 -- 172.21.15.32:0/469080588 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc00c6210 con 0x7fabc015a940 2024-04-17T09:53:21.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.799+0000 7fabc5600640 1 -- 172.21.15.32:0/469080588 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc01584d0 con 0x7fabc015a940 2024-04-17T09:53:21.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.799+0000 7f7988c00640 1 -- 172.21.15.32:0/2355096577 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f79800160d0 con 0x7f79840061e0 2024-04-17T09:53:21.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.799+0000 7f8f2ce00640 1 Processor -- start 2024-04-17T09:53:21.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.799+0000 7f8f2ce00640 1 -- start start 2024-04-17T09:53:21.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.799+0000 7fabc5600640 1 -- 172.21.15.32:0/469080588 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc004fd00 con 0x7fabc015a940 2024-04-17T09:53:21.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.799+0000 7fabbcc00640 1 -- 172.21.15.32:0/469080588 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabb0007cb0 con 0x7fabc015a940 2024-04-17T09:53:21.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.799+0000 7fabbcc00640 1 -- 172.21.15.32:0/469080588 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb00c0070 con 0x7fabc015a940 2024-04-17T09:53:21.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.799+0000 7fd604600640 1 Processor -- start 2024-04-17T09:53:21.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.799+0000 7fd604600640 1 -- start start 2024-04-17T09:53:21.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.799+0000 7f798be00640 1 --2- 172.21.15.32:0/2355096577 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f796c068380 0x7f796c06a840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f79840dee30 tx=0x7f7978007d00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.799+0000 7f8feb400640 1 --2- 172.21.15.32:0/3588017059 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8fcc068400 0x7f8fcc06a8c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8fd80208c0 tx=0x7f8fd8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.799+0000 7fd604600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5fc06c550 0x7fd5fc0a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.799+0000 7fd604600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc0b3dc0 0x7fd5fc0cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.799+0000 7fd604600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0aacd0 0x7fd5fc13fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7fd604600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc15b820 con 0x7fd5fc0b3dc0 2024-04-17T09:53:21.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7fd604600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc15b6a0 con 0x7fd5fc06c550 2024-04-17T09:53:21.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7fd604600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd5fc15b9a0 con 0x7fd5fc0aacd0 2024-04-17T09:53:21.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7fd602600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0aacd0 0x7fd5fc13fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7fd602600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0aacd0 0x7fd5fc13fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:43978/0 (socket says 172.21.15.32:43978) 2024-04-17T09:53:21.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7fd602600640 1 -- 172.21.15.32:0/3022243384 learned_addr learned my addr 172.21.15.32:0/3022243384 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:21.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7f8fe3400640 1 -- 172.21.15.32:0/3588017059 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f8fe4010040 con 0x7f8fec1fcef0 2024-04-17T09:53:21.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7fd601c00640 1 --2- 172.21.15.32:0/3022243384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5fc06c550 0x7fd5fc0a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7fd602600640 1 -- 172.21.15.32:0/3022243384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5fc06c550 msgr2=0x7fd5fc0a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7fd602600640 1 --2- 172.21.15.32:0/3022243384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5fc06c550 0x7fd5fc0a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7fd602600640 1 -- 172.21.15.32:0/3022243384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc0b3dc0 msgr2=0x7fd5fc0cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7fd602600640 1 --2- 172.21.15.32:0/3022243384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc0b3dc0 0x7fd5fc0cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7fd602600640 1 -- 172.21.15.32:0/3022243384 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5fc0a9990 con 0x7fd5fc0aacd0 2024-04-17T09:53:21.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7fd601c00640 1 --2- 172.21.15.32:0/3022243384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5fc06c550 0x7fd5fc0a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:21.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7f8f2ce00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f0c0053d0 0x7f8f2808b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7f8f2ce00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f2816afb0 0x7f8f2816b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7f8f2ce00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f2816bf00 0x7f8f281698d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7f8f25e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f0c0053d0 0x7f8f2808b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7fd602600640 1 --2- 172.21.15.32:0/3022243384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0aacd0 0x7fd5fc13fc60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400e7f0 tx=0x7fd5f400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7f8f25e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f0c0053d0 0x7f8f2808b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49144/0 (socket says 172.21.15.32:49144) 2024-04-17T09:53:21.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7f8f2ce00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2816f470 con 0x7f8f0c0053d0 2024-04-17T09:53:21.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7f8f2ce00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2816f2f0 con 0x7f8f2816bf00 2024-04-17T09:53:21.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7f8f2ce00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8f280b3dc0 con 0x7f8f2816afb0 2024-04-17T09:53:21.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7f8f26800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f2816bf00 0x7f8f281698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7f8f25e00640 1 -- 172.21.15.32:0/2230527620 learned_addr learned my addr 172.21.15.32:0/2230527620 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:21.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7f8f25400640 1 --2- 172.21.15.32:0/2230527620 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f2816afb0 0x7f8f2816b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7f8f25e00640 1 -- 172.21.15.32:0/2230527620 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f2816afb0 msgr2=0x7f8f2816b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7f8f25e00640 1 --2- 172.21.15.32:0/2230527620 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f2816afb0 0x7f8f2816b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7f8f25e00640 1 -- 172.21.15.32:0/2230527620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f2816bf00 msgr2=0x7f8f281698d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:21.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7f8f25e00640 1 --2- 172.21.15.32:0/2230527620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f2816bf00 0x7f8f281698d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:21.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7f8f25e00640 1 -- 172.21.15.32:0/2230527620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2808b760 con 0x7f8f0c0053d0 2024-04-17T09:53:21.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7f8f25400640 1 --2- 172.21.15.32:0/2230527620 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f2816afb0 0x7f8f2816b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:53:21.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7fabbcc00640 1 -- 172.21.15.32:0/469080588 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fabb0008040 con 0x7fabc015a940 2024-04-17T09:53:21.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7f8f25e00640 1 --2- 172.21.15.32:0/2230527620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f0c0053d0 0x7f8f2808b220 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000b970 tx=0x7f8f2000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7f8f1e000640 1 -- 172.21.15.32:0/2230527620 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20019070 con 0x7f8f0c0053d0 2024-04-17T09:53:21.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7f8f1e000640 1 -- 172.21.15.32:0/2230527620 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f2000cb60 con 0x7f8f0c0053d0 2024-04-17T09:53:21.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.803+0000 7f8f1e000640 1 -- 172.21.15.32:0/2230527620 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20014440 con 0x7f8f0c0053d0 2024-04-17T09:53:21.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.807+0000 7f8f2ce00640 1 -- 172.21.15.32:0/2230527620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f280cd2e0 con 0x7f8f0c0053d0 2024-04-17T09:53:21.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.807+0000 7f8f2ce00640 1 -- 172.21.15.32:0/2230527620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f280cd8a0 con 0x7f8f0c0053d0 2024-04-17T09:53:21.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.807+0000 7fd5f9600640 1 -- 172.21.15.32:0/3022243384 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4017070 con 0x7fd5fc0aacd0 2024-04-17T09:53:21.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.807+0000 7fd5f9600640 1 -- 172.21.15.32:0/3022243384 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5f400c940 con 0x7fd5fc0aacd0 2024-04-17T09:53:21.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.807+0000 7fd5f9600640 1 -- 172.21.15.32:0/3022243384 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f400eea0 con 0x7fd5fc0aacd0 2024-04-17T09:53:21.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.807+0000 7fd604600640 1 -- 172.21.15.32:0/3022243384 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5fc0a9c30 con 0x7fd5fc0aacd0 2024-04-17T09:53:21.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.807+0000 7fd604600640 1 -- 172.21.15.32:0/3022243384 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5fc0aa1f0 con 0x7fd5fc0aacd0 2024-04-17T09:53:21.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.807+0000 7fabbcc00640 1 --2- 172.21.15.32:0/469080588 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab9c0685b0 0x7fab9c06aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.807+0000 7fabbcc00640 1 -- 172.21.15.32:0/469080588 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fabb01209c0 con 0x7fabc015a940 2024-04-17T09:53:21.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.807+0000 7fabbfe00640 1 --2- 172.21.15.32:0/469080588 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab9c0685b0 0x7fab9c06aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.807+0000 7fabbcc00640 1 -- 172.21.15.32:0/469080588 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fabb00189c0 con 0x7fabc015a940 2024-04-17T09:53:21.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.807+0000 7f8f13400640 1 -- 172.21.15.32:0/2230527620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f28068a90 con 0x7f8f0c0053d0 2024-04-17T09:53:21.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.807+0000 7fd604600640 1 -- 172.21.15.32:0/3022243384 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d40052a0 con 0x7fd5fc0aacd0 2024-04-17T09:53:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.811+0000 7fabbfe00640 1 --2- 172.21.15.32:0/469080588 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab9c0685b0 0x7fab9c06aa70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faba8009b60 tx=0x7faba8009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.811+0000 7f8f1e000640 1 -- 172.21.15.32:0/2230527620 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8f2001c070 con 0x7f8f0c0053d0 2024-04-17T09:53:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.811+0000 7f8f1e000640 1 --2- 172.21.15.32:0/2230527620 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8f080683d0 0x7f8f0806a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.811+0000 7f8f1e000640 1 -- 172.21.15.32:0/2230527620 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8f200866a0 con 0x7f8f0c0053d0 2024-04-17T09:53:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.811+0000 7fd5f9600640 1 -- 172.21.15.32:0/3022243384 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd5f401a070 con 0x7fd5fc0aacd0 2024-04-17T09:53:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.811+0000 7fd5f9600640 1 --2- 172.21.15.32:0/3022243384 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5dc068450 0x7fd5dc06a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.811+0000 7fd5f9600640 1 -- 172.21.15.32:0/3022243384 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd5f4086890 con 0x7fd5fc0aacd0 2024-04-17T09:53:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.815+0000 7f8f25400640 1 --2- 172.21.15.32:0/2230527620 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8f080683d0 0x7f8f0806a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.815+0000 7f8f25400640 1 --2- 172.21.15.32:0/2230527620 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8f080683d0 0x7f8f0806a890 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8f2816d140 tx=0x7f8f14009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.819+0000 7fd601c00640 1 --2- 172.21.15.32:0/3022243384 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5dc068450 0x7fd5dc06a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:21.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.819+0000 7fd601c00640 1 --2- 172.21.15.32:0/3022243384 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5dc068450 0x7fd5dc06a910 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec007fd0 tx=0x7fd5ec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:21.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.819+0000 7fd5f9600640 1 -- 172.21.15.32:0/3022243384 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd5f4056e30 con 0x7fd5fc0aacd0 2024-04-17T09:53:21.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:21.823+0000 7f8f1e000640 1 -- 172.21.15.32:0/2230527620 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f8f200565f0 con 0x7f8f0c0053d0 2024-04-17T09:53:22.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.059+0000 7f9db0400640 1 -- 172.21.15.32:0/1295795737 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9da8072720 con 0x7f9da80d6810 2024-04-17T09:53:22.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.063+0000 7f9da5600640 1 -- 172.21.15.32:0/1295795737 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9d9405ae10 con 0x7f9da80d6810 2024-04-17T09:53:22.066 INFO:teuthology.orchestra.run.smithi032.stdout:64424509447 2024-04-17T09:53:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.079+0000 7f9db0400640 1 -- 172.21.15.32:0/1295795737 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9d88068380 msgr2=0x7f9d8806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.079+0000 7f9db0400640 1 --2- 172.21.15.32:0/1295795737 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9d88068380 0x7f9d8806a840 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9d980ae000 tx=0x7f9d980a7070 comp rx=0 tx=0).stop 2024-04-17T09:53:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.079+0000 7f9db0400640 1 -- 172.21.15.32:0/1295795737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9da80d6810 msgr2=0x7f9da80ba7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.079+0000 7f9db0400640 1 --2- 172.21.15.32:0/1295795737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9da80d6810 0x7f9da80ba7d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400d600 tx=0x7f9d9400dac0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.079+0000 7f9db0400640 1 -- 172.21.15.32:0/1295795737 shutdown_connections 2024-04-17T09:53:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.079+0000 7f9db0400640 1 --2- 172.21.15.32:0/1295795737 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9d88068380 0x7f9d8806a840 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.079+0000 7f9db0400640 1 --2- 172.21.15.32:0/1295795737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9da80d6810 0x7f9da80ba7d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.079+0000 7f9db0400640 1 --2- 172.21.15.32:0/1295795737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80d6bf0 0x7f9da80957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.079+0000 7f9db0400640 1 --2- 172.21.15.32:0/1295795737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da8073970 0x7f9da80bd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.079+0000 7f9db0400640 1 -- 172.21.15.32:0/1295795737 >> 172.21.15.32:0/1295795737 conn(0x7f9da8072050 msgr2=0x7f9da80d8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:22.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.079+0000 7f0296800640 1 -- 172.21.15.32:0/1235402588 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f0288052830 con 0x7f0288047c20 2024-04-17T09:53:22.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.079+0000 7f9db0400640 1 -- 172.21.15.32:0/1295795737 shutdown_connections 2024-04-17T09:53:22.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.079+0000 7f9db0400640 1 -- 172.21.15.32:0/1295795737 wait complete. 2024-04-17T09:53:22.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.079+0000 7f028d600640 1 -- 172.21.15.32:0/1235402588 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f029008f030 con 0x7f0288047c20 2024-04-17T09:53:22.087 INFO:teuthology.orchestra.run.smithi032.stdout:64424509448 2024-04-17T09:53:22.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.087+0000 7f0296800640 1 -- 172.21.15.32:0/1235402588 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0270068400 msgr2=0x7f027006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.087+0000 7f0296800640 1 --2- 172.21.15.32:0/1235402588 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0270068400 0x7f027006a8c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f027c0b2000 tx=0x7f027c0aa070 comp rx=0 tx=0).stop 2024-04-17T09:53:22.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.087+0000 7f0296800640 1 -- 172.21.15.32:0/1235402588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0288047c20 msgr2=0x7f0288039130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.087+0000 7f0296800640 1 --2- 172.21.15.32:0/1235402588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0288047c20 0x7f0288039130 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0290050220 tx=0x7f0290069570 comp rx=0 tx=0).stop 2024-04-17T09:53:22.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.087+0000 7f0296800640 1 -- 172.21.15.32:0/1235402588 shutdown_connections 2024-04-17T09:53:22.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.087+0000 7f0296800640 1 --2- 172.21.15.32:0/1235402588 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0270068400 0x7f027006a8c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.087+0000 7f0296800640 1 --2- 172.21.15.32:0/1235402588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0288033e40 0x7f028804bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.087+0000 7f0296800640 1 --2- 172.21.15.32:0/1235402588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0288047c20 0x7f0288039130 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.087+0000 7f0296800640 1 --2- 172.21.15.32:0/1235402588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0288005ea0 0x7f02880d8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.087+0000 7f0296800640 1 -- 172.21.15.32:0/1235402588 >> 172.21.15.32:0/1235402588 conn(0x7f0288009fc0 msgr2=0x7f0288006280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:22.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.087+0000 7f0296800640 1 -- 172.21.15.32:0/1235402588 shutdown_connections 2024-04-17T09:53:22.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.087+0000 7f0296800640 1 -- 172.21.15.32:0/1235402588 wait complete. 2024-04-17T09:53:22.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.095+0000 7efdc7c00640 1 -- 172.21.15.32:0/3139016043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7efd9c005590 con 0x7efdc006f8f0 2024-04-17T09:53:22.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.099+0000 7efdbcc00640 1 -- 172.21.15.32:0/3139016043 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efdb40c3030 con 0x7efdc006f8f0 2024-04-17T09:53:22.101 INFO:teuthology.orchestra.run.smithi032.stdout:60129542151 2024-04-17T09:53:22.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.103+0000 7efdc7c00640 1 -- 172.21.15.32:0/3139016043 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efda0068450 msgr2=0x7efda006a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.103+0000 7efdc7c00640 1 --2- 172.21.15.32:0/3139016043 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efda0068450 0x7efda006a910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efdc015b450 tx=0x7efdac007f30 comp rx=0 tx=0).stop 2024-04-17T09:53:22.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.103+0000 7efdc7c00640 1 -- 172.21.15.32:0/3139016043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc006f8f0 msgr2=0x7efdc00cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.103+0000 7efdc7c00640 1 --2- 172.21.15.32:0/3139016043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc006f8f0 0x7efdc00cd1a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efdb401aad0 tx=0x7efdb40191c0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.103+0000 7efdc7c00640 1 -- 172.21.15.32:0/3139016043 shutdown_connections 2024-04-17T09:53:22.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.103+0000 7efdc7c00640 1 --2- 172.21.15.32:0/3139016043 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efda0068450 0x7efda006a910 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.103+0000 7efdc7c00640 1 --2- 172.21.15.32:0/3139016043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdc00d09f0 0x7efdc0161e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.103+0000 7efdc7c00640 1 --2- 172.21.15.32:0/3139016043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdc0072b00 0x7efdc0161900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.103+0000 7efdc7c00640 1 --2- 172.21.15.32:0/3139016043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc006f8f0 0x7efdc00cd1a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.103+0000 7efdc7c00640 1 -- 172.21.15.32:0/3139016043 >> 172.21.15.32:0/3139016043 conn(0x7efdc006ba30 msgr2=0x7efdc0164840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:22.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.103+0000 7efdc7c00640 1 -- 172.21.15.32:0/3139016043 shutdown_connections 2024-04-17T09:53:22.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.103+0000 7efdc7c00640 1 -- 172.21.15.32:0/3139016043 wait complete. 2024-04-17T09:53:22.117 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.1 2024-04-17T09:53:22.117 DEBUG:teuthology.parallel:result is None 2024-04-17T09:53:22.128 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509448 for osd.2 2024-04-17T09:53:22.128 DEBUG:teuthology.parallel:result is None 2024-04-17T09:53:22.140 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.0 2024-04-17T09:53:22.140 DEBUG:teuthology.parallel:result is None 2024-04-17T09:53:22.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.143+0000 7f8f13400640 1 -- 172.21.15.32:0/2230527620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f8f2816c2e0 con 0x7f8f0c0053d0 2024-04-17T09:53:22.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.143+0000 7f8ff2200640 1 -- 172.21.15.32:0/3588017059 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f8fec0b4c90 con 0x7f8fec1fcef0 2024-04-17T09:53:22.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.143+0000 7f8f1e000640 1 -- 172.21.15.32:0/2230527620 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8f2005a8f0 con 0x7f8f0c0053d0 2024-04-17T09:53:22.147 INFO:teuthology.orchestra.run.smithi032.stdout:64424509447 2024-04-17T09:53:22.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.143+0000 7f8fe3400640 1 -- 172.21.15.32:0/3588017059 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8fe40573f0 con 0x7f8fec1fcef0 2024-04-17T09:53:22.147 INFO:teuthology.orchestra.run.smithi032.stdout:64424509447 2024-04-17T09:53:22.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.147+0000 7f8ff2200640 1 -- 172.21.15.32:0/3588017059 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8fcc068400 msgr2=0x7f8fcc06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.147+0000 7f8ff2200640 1 --2- 172.21.15.32:0/3588017059 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8fcc068400 0x7f8fcc06a8c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8fd80208c0 tx=0x7f8fd8008040 comp rx=0 tx=0).stop 2024-04-17T09:53:22.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.147+0000 7f8ff2200640 1 -- 172.21.15.32:0/3588017059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fec1fcef0 msgr2=0x7f8fec0cb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.147+0000 7f8ff2200640 1 --2- 172.21.15.32:0/3588017059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fec1fcef0 0x7f8fec0cb7e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400ef90 tx=0x7f8fe400c490 comp rx=0 tx=0).stop 2024-04-17T09:53:22.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.147+0000 7f8ff2200640 1 -- 172.21.15.32:0/3588017059 shutdown_connections 2024-04-17T09:53:22.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.147+0000 7f8ff2200640 1 --2- 172.21.15.32:0/3588017059 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8fcc068400 0x7f8fcc06a8c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.147+0000 7f8ff2200640 1 --2- 172.21.15.32:0/3588017059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fec1fcef0 0x7f8fec0cb7e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.143+0000 7f8f13400640 1 -- 172.21.15.32:0/2230527620 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8f080683d0 msgr2=0x7f8f0806a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.147+0000 7f8ff2200640 1 --2- 172.21.15.32:0/3588017059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fec0cdbd0 0x7f8fec15bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.147+0000 7f8ff2200640 1 --2- 172.21.15.32:0/3588017059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fec09bc60 0x7f8fec09fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.147+0000 7f8ff2200640 1 -- 172.21.15.32:0/3588017059 >> 172.21.15.32:0/3588017059 conn(0x7f8fec0b40a0 msgr2=0x7f8fec0b7120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:22.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.147+0000 7f8f13400640 1 --2- 172.21.15.32:0/2230527620 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8f080683d0 0x7f8f0806a890 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8f2816d140 tx=0x7f8f14009040 comp rx=0 tx=0).stop 2024-04-17T09:53:22.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.147+0000 7f8f13400640 1 -- 172.21.15.32:0/2230527620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f0c0053d0 msgr2=0x7f8f2808b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.147+0000 7f8f13400640 1 --2- 172.21.15.32:0/2230527620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f0c0053d0 0x7f8f2808b220 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000b970 tx=0x7f8f2000be30 comp rx=0 tx=0).stop 2024-04-17T09:53:22.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.147+0000 7f8f13400640 1 -- 172.21.15.32:0/2230527620 shutdown_connections 2024-04-17T09:53:22.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.147+0000 7f8f13400640 1 --2- 172.21.15.32:0/2230527620 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8f080683d0 0x7f8f0806a890 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.147+0000 7f8f13400640 1 --2- 172.21.15.32:0/2230527620 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f2816afb0 0x7f8f2816b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.147+0000 7f8f13400640 1 --2- 172.21.15.32:0/2230527620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f2816bf00 0x7f8f281698d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.147+0000 7f8f13400640 1 --2- 172.21.15.32:0/2230527620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f0c0053d0 0x7f8f2808b220 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000b970 tx=0x7f8f2000be30 comp rx=0 tx=0).stop 2024-04-17T09:53:22.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.147+0000 7f8f13400640 1 -- 172.21.15.32:0/2230527620 >> 172.21.15.32:0/2230527620 conn(0x7f8f2806ba30 msgr2=0x7f8f2817b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:22.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.147+0000 7f8ff2200640 1 -- 172.21.15.32:0/3588017059 shutdown_connections 2024-04-17T09:53:22.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.147+0000 7f8f13400640 1 -- 172.21.15.32:0/2230527620 shutdown_connections 2024-04-17T09:53:22.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.147+0000 7f8ff2200640 1 -- 172.21.15.32:0/3588017059 wait complete. 2024-04-17T09:53:22.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.147+0000 7f8f13400640 1 -- 172.21.15.32:0/2230527620 wait complete. 2024-04-17T09:53:22.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.151+0000 7fd604600640 1 -- 172.21.15.32:0/3022243384 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fd5d4005590 con 0x7fd5fc0aacd0 2024-04-17T09:53:22.157 INFO:teuthology.orchestra.run.smithi032.stdout:60129542151 2024-04-17T09:53:22.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.155+0000 7fd5f9600640 1 -- 172.21.15.32:0/3022243384 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd5f4056730 con 0x7fd5fc0aacd0 2024-04-17T09:53:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.155+0000 7fd604600640 1 -- 172.21.15.32:0/3022243384 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5dc068450 msgr2=0x7fd5dc06a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.155+0000 7fd604600640 1 --2- 172.21.15.32:0/3022243384 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5dc068450 0x7fd5dc06a910 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec007fd0 tx=0x7fd5ec009040 comp rx=0 tx=0).stop 2024-04-17T09:53:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.155+0000 7fd604600640 1 -- 172.21.15.32:0/3022243384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0aacd0 msgr2=0x7fd5fc13fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.155+0000 7fd604600640 1 --2- 172.21.15.32:0/3022243384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0aacd0 0x7fd5fc13fc60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400e7f0 tx=0x7fd5f400ecb0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.155+0000 7fd604600640 1 -- 172.21.15.32:0/3022243384 shutdown_connections 2024-04-17T09:53:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.155+0000 7fd604600640 1 --2- 172.21.15.32:0/3022243384 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5dc068450 0x7fd5dc06a910 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.155+0000 7fd604600640 1 --2- 172.21.15.32:0/3022243384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5fc0aacd0 0x7fd5fc13fc60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.155+0000 7fd604600640 1 --2- 172.21.15.32:0/3022243384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5fc0b3dc0 0x7fd5fc0cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.155+0000 7fd604600640 1 --2- 172.21.15.32:0/3022243384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5fc06c550 0x7fd5fc0a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.155+0000 7fd604600640 1 -- 172.21.15.32:0/3022243384 >> 172.21.15.32:0/3022243384 conn(0x7fd5fc06ba30 msgr2=0x7fd5fc15aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.155+0000 7fd604600640 1 -- 172.21.15.32:0/3022243384 shutdown_connections 2024-04-17T09:53:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.155+0000 7fd604600640 1 -- 172.21.15.32:0/3022243384 wait complete. 2024-04-17T09:53:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.159+0000 7f7993600640 1 -- 172.21.15.32:0/2355096577 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f7984005820 con 0x7f79840061e0 2024-04-17T09:53:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.159+0000 7f7988c00640 1 -- 172.21.15.32:0/2355096577 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f79800f1b30 con 0x7f79840061e0 2024-04-17T09:53:22.163 INFO:teuthology.orchestra.run.smithi032.stdout:64424509447 2024-04-17T09:53:22.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.163+0000 7f7993600640 1 -- 172.21.15.32:0/2355096577 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f796c068380 msgr2=0x7f796c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.163+0000 7f7993600640 1 --2- 172.21.15.32:0/2355096577 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f796c068380 0x7f796c06a840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f79840dee30 tx=0x7f7978007d00 comp rx=0 tx=0).stop 2024-04-17T09:53:22.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.163+0000 7f7993600640 1 -- 172.21.15.32:0/2355096577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79840061e0 msgr2=0x7f798418b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.163+0000 7f7993600640 1 --2- 172.21.15.32:0/2355096577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79840061e0 0x7f798418b4d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f798001aaa0 tx=0x7f7980019190 comp rx=0 tx=0).stop 2024-04-17T09:53:22.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.163+0000 7f7993600640 1 -- 172.21.15.32:0/2355096577 shutdown_connections 2024-04-17T09:53:22.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.163+0000 7f7993600640 1 --2- 172.21.15.32:0/2355096577 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f796c068380 0x7f796c06a840 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.163+0000 7f7993600640 1 --2- 172.21.15.32:0/2355096577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79840eb340 0x7f79840e5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.163+0000 7f7993600640 1 --2- 172.21.15.32:0/2355096577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79840e7850 0x7f79840deea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.163+0000 7f7993600640 1 --2- 172.21.15.32:0/2355096577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79840061e0 0x7f798418b4d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.163+0000 7f7993600640 1 -- 172.21.15.32:0/2355096577 >> 172.21.15.32:0/2355096577 conn(0x7f7984055a20 msgr2=0x7f7984055e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:22.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.163+0000 7f7993600640 1 -- 172.21.15.32:0/2355096577 shutdown_connections 2024-04-17T09:53:22.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.163+0000 7f7993600640 1 -- 172.21.15.32:0/2355096577 wait complete. 2024-04-17T09:53:22.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.163+0000 7fabc5600640 1 -- 172.21.15.32:0/469080588 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fabc006e6e0 con 0x7fabc015a940 2024-04-17T09:53:22.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.163+0000 7fabbcc00640 1 -- 172.21.15.32:0/469080588 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fabb00f0a10 con 0x7fabc015a940 2024-04-17T09:53:22.168 INFO:teuthology.orchestra.run.smithi032.stdout:64424509448 2024-04-17T09:53:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.167+0000 7fabc5600640 1 -- 172.21.15.32:0/469080588 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab9c0685b0 msgr2=0x7fab9c06aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.167+0000 7fabc5600640 1 --2- 172.21.15.32:0/469080588 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab9c0685b0 0x7fab9c06aa70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faba8009b60 tx=0x7faba8009190 comp rx=0 tx=0).stop 2024-04-17T09:53:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.167+0000 7fabc5600640 1 -- 172.21.15.32:0/469080588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabc015a940 msgr2=0x7fabc015a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.167+0000 7fabc5600640 1 --2- 172.21.15.32:0/469080588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabc015a940 0x7fabc015a020 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fabb00b3040 tx=0x7fabb001bb40 comp rx=0 tx=0).stop 2024-04-17T09:53:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.167+0000 7fabc5600640 1 -- 172.21.15.32:0/469080588 shutdown_connections 2024-04-17T09:53:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.167+0000 7fabc5600640 1 --2- 172.21.15.32:0/469080588 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab9c0685b0 0x7fab9c06aa70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.167+0000 7fabc5600640 1 --2- 172.21.15.32:0/469080588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabc015a940 0x7fabc015a020 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.167+0000 7fabc5600640 1 --2- 172.21.15.32:0/469080588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabc015a560 0x7fabc0158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.167+0000 7fabc5600640 1 --2- 172.21.15.32:0/469080588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc00ccb00 0x7fabc0098cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.167+0000 7fabc5600640 1 -- 172.21.15.32:0/469080588 >> 172.21.15.32:0/469080588 conn(0x7fabc00b1d00 msgr2=0x7fabc006ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.167+0000 7fabc5600640 1 -- 172.21.15.32:0/469080588 shutdown_connections 2024-04-17T09:53:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.171+0000 7fabc5600640 1 -- 172.21.15.32:0/469080588 wait complete. 2024-04-17T09:53:22.176 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.7 2024-04-17T09:53:22.177 DEBUG:teuthology.parallel:result is None 2024-04-17T09:53:22.177 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.3 2024-04-17T09:53:22.177 DEBUG:teuthology.parallel:result is None 2024-04-17T09:53:22.190 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.5 2024-04-17T09:53:22.190 DEBUG:teuthology.parallel:result is None 2024-04-17T09:53:22.191 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.4 2024-04-17T09:53:22.191 DEBUG:teuthology.parallel:result is None 2024-04-17T09:53:22.199 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509448 for osd.6 2024-04-17T09:53:22.199 DEBUG:teuthology.parallel:result is None 2024-04-17T09:53:22.200 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-17T09:53:22.200 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T09:53:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.375+0000 7ffbbb400640 1 Processor -- start 2024-04-17T09:53:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.375+0000 7ffbbb400640 1 -- start start 2024-04-17T09:53:22.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.379+0000 7ffbbb400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ffbb415c750 0x7ffbb415cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:22.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.379+0000 7ffbbb400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffbb4160ce0 0x7ffbb40b2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.379+0000 7ffbbb400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbb40b26b0 0x7ffbb406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.379+0000 7ffbbb400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ffbb40cd100 con 0x7ffbb4160ce0 2024-04-17T09:53:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.379+0000 7ffbbb400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ffbb40b3c40 con 0x7ffbb415c750 2024-04-17T09:53:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.379+0000 7ffbbb400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb415d1a0 con 0x7ffbb40b26b0 2024-04-17T09:53:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.379+0000 7ffbabe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffbb4160ce0 0x7ffbb40b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.379+0000 7ffbabe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffbb4160ce0 0x7ffbb40b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49156/0 (socket says 172.21.15.32:49156) 2024-04-17T09:53:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.379+0000 7ffbabe00640 1 -- 172.21.15.32:0/2331900350 learned_addr learned my addr 172.21.15.32:0/2331900350 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.379+0000 7ffbb8a00640 1 --2- 172.21.15.32:0/2331900350 >> v2:172.21.15.144:3300/0 conn(0x7ffbb415c750 0x7ffbb415cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.379+0000 7ffbb9400640 1 --2- 172.21.15.32:0/2331900350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbb40b26b0 0x7ffbb406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.379+0000 7ffbabe00640 1 -- 172.21.15.32:0/2331900350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbb40b26b0 msgr2=0x7ffbb406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.379+0000 7ffbabe00640 1 --2- 172.21.15.32:0/2331900350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbb40b26b0 0x7ffbb406ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.379+0000 7ffbabe00640 1 -- 172.21.15.32:0/2331900350 >> v2:172.21.15.144:3300/0 conn(0x7ffbb415c750 msgr2=0x7ffbb415cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.379+0000 7ffbabe00640 1 --2- 172.21.15.32:0/2331900350 >> v2:172.21.15.144:3300/0 conn(0x7ffbb415c750 0x7ffbb415cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.379+0000 7ffbabe00640 1 -- 172.21.15.32:0/2331900350 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb415d480 con 0x7ffbb4160ce0 2024-04-17T09:53:22.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.379+0000 7ffbb9400640 1 --2- 172.21.15.32:0/2331900350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbb40b26b0 0x7ffbb406ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:22.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.383+0000 7ffbabe00640 1 --2- 172.21.15.32:0/2331900350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbb4160ce0 0x7ffbb40b2170 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40d1410 tx=0x7ffb9c01c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d101beb6cf1c2cc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:22.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.383+0000 7ffbb3e00640 1 -- 172.21.15.32:0/2331900350 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c0b0070 con 0x7ffbb4160ce0 2024-04-17T09:53:22.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.383+0000 7ffbb3e00640 1 -- 172.21.15.32:0/2331900350 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9c0ab600 con 0x7ffbb4160ce0 2024-04-17T09:53:22.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.383+0000 7ffbb3e00640 1 -- 172.21.15.32:0/2331900350 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c0ab9d0 con 0x7ffbb4160ce0 2024-04-17T09:53:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.383+0000 7ffbbb400640 1 -- 172.21.15.32:0/2331900350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbb4160ce0 msgr2=0x7ffbb40b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.383+0000 7ffbbb400640 1 --2- 172.21.15.32:0/2331900350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbb4160ce0 0x7ffbb40b2170 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40d1410 tx=0x7ffb9c01c1d0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.383+0000 7ffbbb400640 1 -- 172.21.15.32:0/2331900350 shutdown_connections 2024-04-17T09:53:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.383+0000 7ffbbb400640 1 --2- 172.21.15.32:0/2331900350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbb40b26b0 0x7ffbb406ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.383+0000 7ffbbb400640 1 --2- 172.21.15.32:0/2331900350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbb4160ce0 0x7ffbb40b2170 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.383+0000 7ffbbb400640 1 --2- 172.21.15.32:0/2331900350 >> v2:172.21.15.144:3300/0 conn(0x7ffbb415c750 0x7ffbb415cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.383+0000 7ffbbb400640 1 -- 172.21.15.32:0/2331900350 >> 172.21.15.32:0/2331900350 conn(0x7ffbb409a9f0 msgr2=0x7ffbb40b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:22.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.383+0000 7ffbbb400640 1 -- 172.21.15.32:0/2331900350 shutdown_connections 2024-04-17T09:53:22.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.383+0000 7ffbbb400640 1 -- 172.21.15.32:0/2331900350 wait complete. 2024-04-17T09:53:22.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.383+0000 7ffbbb400640 1 Processor -- start 2024-04-17T09:53:22.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.383+0000 7ffbbb400640 1 -- start start 2024-04-17T09:53:22.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbbb400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbb40b26b0 0x7ffbb4154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:22.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbbb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbb415c750 0x7ffbb40c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbbb400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb4160ce0 0x7ffbb415b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbbb400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb40cd700 con 0x7ffbb415c750 2024-04-17T09:53:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbbb400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb40cd580 con 0x7ffbb4160ce0 2024-04-17T09:53:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbbb400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb40cd880 con 0x7ffbb40b26b0 2024-04-17T09:53:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbabe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbb415c750 0x7ffbb40c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbabe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbb415c750 0x7ffbb40c6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49170/0 (socket says 172.21.15.32:49170) 2024-04-17T09:53:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbabe00640 1 -- 172.21.15.32:0/3816615555 learned_addr learned my addr 172.21.15.32:0/3816615555 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbb9400640 1 --2- 172.21.15.32:0/3816615555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb4160ce0 0x7ffbb415b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbb8a00640 1 --2- 172.21.15.32:0/3816615555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbb40b26b0 0x7ffbb4154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbabe00640 1 -- 172.21.15.32:0/3816615555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbb40b26b0 msgr2=0x7ffbb4154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbabe00640 1 --2- 172.21.15.32:0/3816615555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbb40b26b0 0x7ffbb4154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbabe00640 1 -- 172.21.15.32:0/3816615555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb4160ce0 msgr2=0x7ffbb415b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbabe00640 1 --2- 172.21.15.32:0/3816615555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb4160ce0 0x7ffbb415b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbabe00640 1 -- 172.21.15.32:0/3816615555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb4155fd0 con 0x7ffbb415c750 2024-04-17T09:53:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbb9400640 1 --2- 172.21.15.32:0/3816615555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb4160ce0 0x7ffbb415b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:22.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbabe00640 1 --2- 172.21.15.32:0/3816615555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbb415c750 0x7ffbb40c6ba0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c0ae040 tx=0x7ffb9c018f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:22.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbb1600640 1 -- 172.21.15.32:0/3816615555 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c0ba020 con 0x7ffbb415c750 2024-04-17T09:53:22.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbb1600640 1 -- 172.21.15.32:0/3816615555 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9c0b6620 con 0x7ffbb415c750 2024-04-17T09:53:22.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbb1600640 1 -- 172.21.15.32:0/3816615555 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c0b0040 con 0x7ffbb415c750 2024-04-17T09:53:22.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbbb400640 1 -- 172.21.15.32:0/3816615555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb41562b0 con 0x7ffbb415c750 2024-04-17T09:53:22.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbbb400640 1 -- 172.21.15.32:0/3816615555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb4154da0 con 0x7ffbb415c750 2024-04-17T09:53:22.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbbb400640 1 -- 172.21.15.32:0/3816615555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb4068a90 con 0x7ffbb415c750 2024-04-17T09:53:22.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.387+0000 7ffbb1600640 1 -- 172.21.15.32:0/3816615555 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ffb9c0bd020 con 0x7ffbb415c750 2024-04-17T09:53:22.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.391+0000 7ffbb1600640 1 --2- 172.21.15.32:0/3816615555 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffb90068380 0x7ffb9006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:22.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.391+0000 7ffbb1600640 1 -- 172.21.15.32:0/3816615555 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ffb9c120960 con 0x7ffbb415c750 2024-04-17T09:53:22.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.391+0000 7ffbb8a00640 1 --2- 172.21.15.32:0/3816615555 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffb90068380 0x7ffb9006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:22.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.391+0000 7ffbb8a00640 1 --2- 172.21.15.32:0/3816615555 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffb90068380 0x7ffb9006a840 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffba400b9d0 tx=0x7ffba4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:22.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.395+0000 7ffbb1600640 1 -- 172.21.15.32:0/3816615555 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ffb9c0c4070 con 0x7ffbb415c750 2024-04-17T09:53:22.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.555+0000 7ffbbb400640 1 -- 172.21.15.32:0/3816615555 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffbb40b7450 con 0x7ffb90068380 2024-04-17T09:53:22.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.563+0000 7ffbb1600640 1 -- 172.21.15.32:0/3816615555 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7ffbb40b7450 con 0x7ffb90068380 2024-04-17T09:53:22.571 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:22.573 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T09:53:22.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.567+0000 7ffbbb400640 1 -- 172.21.15.32:0/3816615555 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffb90068380 msgr2=0x7ffb9006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.567+0000 7ffbbb400640 1 --2- 172.21.15.32:0/3816615555 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffb90068380 0x7ffb9006a840 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffba400b9d0 tx=0x7ffba4008040 comp rx=0 tx=0).stop 2024-04-17T09:53:22.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.567+0000 7ffbbb400640 1 -- 172.21.15.32:0/3816615555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbb415c750 msgr2=0x7ffbb40c6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.567+0000 7ffbbb400640 1 --2- 172.21.15.32:0/3816615555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbb415c750 0x7ffbb40c6ba0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c0ae040 tx=0x7ffb9c018f70 comp rx=0 tx=0).stop 2024-04-17T09:53:22.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.567+0000 7ffbbb400640 1 -- 172.21.15.32:0/3816615555 shutdown_connections 2024-04-17T09:53:22.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.567+0000 7ffbbb400640 1 --2- 172.21.15.32:0/3816615555 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffb90068380 0x7ffb9006a840 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.567+0000 7ffbbb400640 1 --2- 172.21.15.32:0/3816615555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb4160ce0 0x7ffbb415b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.567+0000 7ffbbb400640 1 --2- 172.21.15.32:0/3816615555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbb415c750 0x7ffbb40c6ba0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.567+0000 7ffbbb400640 1 --2- 172.21.15.32:0/3816615555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbb40b26b0 0x7ffbb4154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.567+0000 7ffbbb400640 1 -- 172.21.15.32:0/3816615555 >> 172.21.15.32:0/3816615555 conn(0x7ffbb409a9f0 msgr2=0x7ffbb415ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:22.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.567+0000 7ffbbb400640 1 -- 172.21.15.32:0/3816615555 shutdown_connections 2024-04-17T09:53:22.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.567+0000 7ffbbb400640 1 -- 172.21.15.32:0/3816615555 wait complete. 2024-04-17T09:53:22.589 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-17T09:53:21.286797+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239844,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734876,"statfs":{"total":773094113280,"available":772848513024,"internally_reserved":0,"allocated":2924544,"data_stored":1763052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001594"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067861+0000","last_change":"2024-04-17T09:53:13.801310+0000","last_active":"2024-04-17T09:53:15.067861+0000","last_peered":"2024-04-17T09:53:15.067861+0000","last_clean":"2024-04-17T09:53:15.067861+0000","last_became_active":"2024-04-17T09:53:13.800622+0000","last_became_peered":"2024-04-17T09:53:13.800622+0000","last_unstale":"2024-04-17T09:53:15.067861+0000","last_undegraded":"2024-04-17T09:53:15.067861+0000","last_fullsized":"2024-04-17T09:53:15.067861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:10:45.479198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787738+0000","last_change":"2024-04-17T09:53:13.791349+0000","last_active":"2024-04-17T09:53:14.787738+0000","last_peered":"2024-04-17T09:53:14.787738+0000","last_clean":"2024-04-17T09:53:14.787738+0000","last_became_active":"2024-04-17T09:53:13.790726+0000","last_became_peered":"2024-04-17T09:53:13.790726+0000","last_unstale":"2024-04-17T09:53:14.787738+0000","last_undegraded":"2024-04-17T09:53:14.787738+0000","last_fullsized":"2024-04-17T09:53:14.787738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:51:42.871563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067454+0000","last_change":"2024-04-17T09:53:13.784861+0000","last_active":"2024-04-17T09:53:15.067454+0000","last_peered":"2024-04-17T09:53:15.067454+0000","last_clean":"2024-04-17T09:53:15.067454+0000","last_became_active":"2024-04-17T09:53:13.784550+0000","last_became_peered":"2024-04-17T09:53:13.784550+0000","last_unstale":"2024-04-17T09:53:15.067454+0000","last_undegraded":"2024-04-17T09:53:15.067454+0000","last_fullsized":"2024-04-17T09:53:15.067454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:03.582853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066735+0000","last_change":"2024-04-17T09:53:13.793610+0000","last_active":"2024-04-17T09:53:15.066735+0000","last_peered":"2024-04-17T09:53:15.066735+0000","last_clean":"2024-04-17T09:53:15.066735+0000","last_became_active":"2024-04-17T09:53:13.793176+0000","last_became_peered":"2024-04-17T09:53:13.793176+0000","last_unstale":"2024-04-17T09:53:15.066735+0000","last_undegraded":"2024-04-17T09:53:15.066735+0000","last_fullsized":"2024-04-17T09:53:15.066735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:16:29.674656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067415+0000","last_change":"2024-04-17T09:53:13.799934+0000","last_active":"2024-04-17T09:53:15.067415+0000","last_peered":"2024-04-17T09:53:15.067415+0000","last_clean":"2024-04-17T09:53:15.067415+0000","last_became_active":"2024-04-17T09:53:13.799557+0000","last_became_peered":"2024-04-17T09:53:13.799557+0000","last_unstale":"2024-04-17T09:53:15.067415+0000","last_undegraded":"2024-04-17T09:53:15.067415+0000","last_fullsized":"2024-04-17T09:53:15.067415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:43:23.411109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066691+0000","last_change":"2024-04-17T09:53:13.806419+0000","last_active":"2024-04-17T09:53:15.066691+0000","last_peered":"2024-04-17T09:53:15.066691+0000","last_clean":"2024-04-17T09:53:15.066691+0000","last_became_active":"2024-04-17T09:53:13.805216+0000","last_became_peered":"2024-04-17T09:53:13.805216+0000","last_unstale":"2024-04-17T09:53:15.066691+0000","last_undegraded":"2024-04-17T09:53:15.066691+0000","last_fullsized":"2024-04-17T09:53:15.066691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:58:50.680456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066488+0000","last_change":"2024-04-17T09:53:13.799827+0000","last_active":"2024-04-17T09:53:15.066488+0000","last_peered":"2024-04-17T09:53:15.066488+0000","last_clean":"2024-04-17T09:53:15.066488+0000","last_became_active":"2024-04-17T09:53:13.799483+0000","last_became_peered":"2024-04-17T09:53:13.799483+0000","last_unstale":"2024-04-17T09:53:15.066488+0000","last_undegraded":"2024-04-17T09:53:15.066488+0000","last_fullsized":"2024-04-17T09:53:15.066488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:16:29.674656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066098+0000","last_change":"2024-04-17T09:53:13.805644+0000","last_active":"2024-04-17T09:53:15.066098+0000","last_peered":"2024-04-17T09:53:15.066098+0000","last_clean":"2024-04-17T09:53:15.066098+0000","last_became_active":"2024-04-17T09:53:13.805343+0000","last_became_peered":"2024-04-17T09:53:13.805343+0000","last_unstale":"2024-04-17T09:53:15.066098+0000","last_undegraded":"2024-04-17T09:53:15.066098+0000","last_fullsized":"2024-04-17T09:53:15.066098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:43:18.222168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067921+0000","last_change":"2024-04-17T09:53:13.815771+0000","last_active":"2024-04-17T09:53:15.067921+0000","last_peered":"2024-04-17T09:53:15.067921+0000","last_clean":"2024-04-17T09:53:15.067921+0000","last_became_active":"2024-04-17T09:53:13.815042+0000","last_became_peered":"2024-04-17T09:53:13.815042+0000","last_unstale":"2024-04-17T09:53:15.067921+0000","last_undegraded":"2024-04-17T09:53:15.067921+0000","last_fullsized":"2024-04-17T09:53:15.067921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:26:59.317045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.789174+0000","last_change":"2024-04-17T09:53:13.810712+0000","last_active":"2024-04-17T09:53:14.789174+0000","last_peered":"2024-04-17T09:53:14.789174+0000","last_clean":"2024-04-17T09:53:14.789174+0000","last_became_active":"2024-04-17T09:53:13.810369+0000","last_became_peered":"2024-04-17T09:53:13.810369+0000","last_unstale":"2024-04-17T09:53:14.789174+0000","last_undegraded":"2024-04-17T09:53:14.789174+0000","last_fullsized":"2024-04-17T09:53:14.789174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:14.645740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788592+0000","last_change":"2024-04-17T09:53:13.800154+0000","last_active":"2024-04-17T09:53:14.788592+0000","last_peered":"2024-04-17T09:53:14.788592+0000","last_clean":"2024-04-17T09:53:14.788592+0000","last_became_active":"2024-04-17T09:53:13.799871+0000","last_became_peered":"2024-04-17T09:53:13.799871+0000","last_unstale":"2024-04-17T09:53:14.788592+0000","last_undegraded":"2024-04-17T09:53:14.788592+0000","last_fullsized":"2024-04-17T09:53:14.788592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:02.320431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788631+0000","last_change":"2024-04-17T09:53:13.805171+0000","last_active":"2024-04-17T09:53:14.788631+0000","last_peered":"2024-04-17T09:53:14.788631+0000","last_clean":"2024-04-17T09:53:14.788631+0000","last_became_active":"2024-04-17T09:53:13.804823+0000","last_became_peered":"2024-04-17T09:53:13.804823+0000","last_unstale":"2024-04-17T09:53:14.788631+0000","last_undegraded":"2024-04-17T09:53:14.788631+0000","last_fullsized":"2024-04-17T09:53:14.788631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:26.904134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066684+0000","last_change":"2024-04-17T09:53:13.789478+0000","last_active":"2024-04-17T09:53:15.066684+0000","last_peered":"2024-04-17T09:53:15.066684+0000","last_clean":"2024-04-17T09:53:15.066684+0000","last_became_active":"2024-04-17T09:53:13.789168+0000","last_became_peered":"2024-04-17T09:53:13.789168+0000","last_unstale":"2024-04-17T09:53:15.066684+0000","last_undegraded":"2024-04-17T09:53:15.066684+0000","last_fullsized":"2024-04-17T09:53:15.066684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:14.645740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066671+0000","last_change":"2024-04-17T09:53:13.794592+0000","last_active":"2024-04-17T09:53:15.066671+0000","last_peered":"2024-04-17T09:53:15.066671+0000","last_clean":"2024-04-17T09:53:15.066671+0000","last_became_active":"2024-04-17T09:53:13.794323+0000","last_became_peered":"2024-04-17T09:53:13.794323+0000","last_unstale":"2024-04-17T09:53:15.066671+0000","last_undegraded":"2024-04-17T09:53:15.066671+0000","last_fullsized":"2024-04-17T09:53:15.066671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:40:23.720233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787454+0000","last_change":"2024-04-17T09:53:13.792566+0000","last_active":"2024-04-17T09:53:14.787454+0000","last_peered":"2024-04-17T09:53:14.787454+0000","last_clean":"2024-04-17T09:53:14.787454+0000","last_became_active":"2024-04-17T09:53:13.791848+0000","last_became_peered":"2024-04-17T09:53:13.791848+0000","last_unstale":"2024-04-17T09:53:14.787454+0000","last_undegraded":"2024-04-17T09:53:14.787454+0000","last_fullsized":"2024-04-17T09:53:14.787454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:17:54.590392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066342+0000","last_change":"2024-04-17T09:53:13.815067+0000","last_active":"2024-04-17T09:53:15.066342+0000","last_peered":"2024-04-17T09:53:15.066342+0000","last_clean":"2024-04-17T09:53:15.066342+0000","last_became_active":"2024-04-17T09:53:13.814781+0000","last_became_peered":"2024-04-17T09:53:13.814781+0000","last_unstale":"2024-04-17T09:53:15.066342+0000","last_undegraded":"2024-04-17T09:53:15.066342+0000","last_fullsized":"2024-04-17T09:53:15.066342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:14.645740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067458+0000","last_change":"2024-04-17T09:53:13.814757+0000","last_active":"2024-04-17T09:53:15.067458+0000","last_peered":"2024-04-17T09:53:15.067458+0000","last_clean":"2024-04-17T09:53:15.067458+0000","last_became_active":"2024-04-17T09:53:13.814433+0000","last_became_peered":"2024-04-17T09:53:13.814433+0000","last_unstale":"2024-04-17T09:53:15.067458+0000","last_undegraded":"2024-04-17T09:53:15.067458+0000","last_fullsized":"2024-04-17T09:53:15.067458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:34:37.237156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067453+0000","last_change":"2024-04-17T09:53:13.799374+0000","last_active":"2024-04-17T09:53:15.067453+0000","last_peered":"2024-04-17T09:53:15.067453+0000","last_clean":"2024-04-17T09:53:15.067453+0000","last_became_active":"2024-04-17T09:53:13.799117+0000","last_became_peered":"2024-04-17T09:53:13.799117+0000","last_unstale":"2024-04-17T09:53:15.067453+0000","last_undegraded":"2024-04-17T09:53:15.067453+0000","last_fullsized":"2024-04-17T09:53:15.067453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:54.146581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067542+0000","last_change":"2024-04-17T09:53:13.798276+0000","last_active":"2024-04-17T09:53:15.067542+0000","last_peered":"2024-04-17T09:53:15.067542+0000","last_clean":"2024-04-17T09:53:15.067542+0000","last_became_active":"2024-04-17T09:53:13.796681+0000","last_became_peered":"2024-04-17T09:53:13.796681+0000","last_unstale":"2024-04-17T09:53:15.067542+0000","last_undegraded":"2024-04-17T09:53:15.067542+0000","last_fullsized":"2024-04-17T09:53:15.067542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:27:15.142091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788909+0000","last_change":"2024-04-17T09:53:13.804880+0000","last_active":"2024-04-17T09:53:14.788909+0000","last_peered":"2024-04-17T09:53:14.788909+0000","last_clean":"2024-04-17T09:53:14.788909+0000","last_became_active":"2024-04-17T09:53:13.804615+0000","last_became_peered":"2024-04-17T09:53:13.804615+0000","last_unstale":"2024-04-17T09:53:14.788909+0000","last_undegraded":"2024-04-17T09:53:14.788909+0000","last_fullsized":"2024-04-17T09:53:14.788909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:17.663523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066926+0000","last_change":"2024-04-17T09:53:13.800703+0000","last_active":"2024-04-17T09:53:15.066926+0000","last_peered":"2024-04-17T09:53:15.066926+0000","last_clean":"2024-04-17T09:53:15.066926+0000","last_became_active":"2024-04-17T09:53:13.800353+0000","last_became_peered":"2024-04-17T09:53:13.800353+0000","last_unstale":"2024-04-17T09:53:15.066926+0000","last_undegraded":"2024-04-17T09:53:15.066926+0000","last_fullsized":"2024-04-17T09:53:15.066926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:55:22.260565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066652+0000","last_change":"2024-04-17T09:53:13.809406+0000","last_active":"2024-04-17T09:53:15.066652+0000","last_peered":"2024-04-17T09:53:15.066652+0000","last_clean":"2024-04-17T09:53:15.066652+0000","last_became_active":"2024-04-17T09:53:13.809152+0000","last_became_peered":"2024-04-17T09:53:13.809152+0000","last_unstale":"2024-04-17T09:53:15.066652+0000","last_undegraded":"2024-04-17T09:53:15.066652+0000","last_fullsized":"2024-04-17T09:53:15.066652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:55:01.176803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066430+0000","last_change":"2024-04-17T09:53:13.785293+0000","last_active":"2024-04-17T09:53:15.066430+0000","last_peered":"2024-04-17T09:53:15.066430+0000","last_clean":"2024-04-17T09:53:15.066430+0000","last_became_active":"2024-04-17T09:53:13.785004+0000","last_became_peered":"2024-04-17T09:53:13.785004+0000","last_unstale":"2024-04-17T09:53:15.066430+0000","last_undegraded":"2024-04-17T09:53:15.066430+0000","last_fullsized":"2024-04-17T09:53:15.066430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:17.663523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066443+0000","last_change":"2024-04-17T09:53:13.803100+0000","last_active":"2024-04-17T09:53:15.066443+0000","last_peered":"2024-04-17T09:53:15.066443+0000","last_clean":"2024-04-17T09:53:15.066443+0000","last_became_active":"2024-04-17T09:53:13.802743+0000","last_became_peered":"2024-04-17T09:53:13.802743+0000","last_unstale":"2024-04-17T09:53:15.066443+0000","last_undegraded":"2024-04-17T09:53:15.066443+0000","last_fullsized":"2024-04-17T09:53:15.066443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:51:42.871563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067978+0000","last_change":"2024-04-17T09:53:13.789615+0000","last_active":"2024-04-17T09:53:15.067978+0000","last_peered":"2024-04-17T09:53:15.067978+0000","last_clean":"2024-04-17T09:53:15.067978+0000","last_became_active":"2024-04-17T09:53:13.787774+0000","last_became_peered":"2024-04-17T09:53:13.787774+0000","last_unstale":"2024-04-17T09:53:15.067978+0000","last_undegraded":"2024-04-17T09:53:15.067978+0000","last_fullsized":"2024-04-17T09:53:15.067978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:36:37.128859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.789301+0000","last_change":"2024-04-17T09:53:13.800946+0000","last_active":"2024-04-17T09:53:14.789301+0000","last_peered":"2024-04-17T09:53:14.789301+0000","last_clean":"2024-04-17T09:53:14.789301+0000","last_became_active":"2024-04-17T09:53:13.798598+0000","last_became_peered":"2024-04-17T09:53:13.798598+0000","last_unstale":"2024-04-17T09:53:14.789301+0000","last_undegraded":"2024-04-17T09:53:14.789301+0000","last_fullsized":"2024-04-17T09:53:14.789301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:51:42.871563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787682+0000","last_change":"2024-04-17T09:53:13.792719+0000","last_active":"2024-04-17T09:53:14.787682+0000","last_peered":"2024-04-17T09:53:14.787682+0000","last_clean":"2024-04-17T09:53:14.787682+0000","last_became_active":"2024-04-17T09:53:13.792390+0000","last_became_peered":"2024-04-17T09:53:13.792390+0000","last_unstale":"2024-04-17T09:53:14.787682+0000","last_undegraded":"2024-04-17T09:53:14.787682+0000","last_fullsized":"2024-04-17T09:53:14.787682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:14.645740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067164+0000","last_change":"2024-04-17T09:53:13.793602+0000","last_active":"2024-04-17T09:53:15.067164+0000","last_peered":"2024-04-17T09:53:15.067164+0000","last_clean":"2024-04-17T09:53:15.067164+0000","last_became_active":"2024-04-17T09:53:13.793277+0000","last_became_peered":"2024-04-17T09:53:13.793277+0000","last_unstale":"2024-04-17T09:53:15.067164+0000","last_undegraded":"2024-04-17T09:53:15.067164+0000","last_fullsized":"2024-04-17T09:53:15.067164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:58:50.680456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066987+0000","last_change":"2024-04-17T09:53:13.781047+0000","last_active":"2024-04-17T09:53:15.066987+0000","last_peered":"2024-04-17T09:53:15.066987+0000","last_clean":"2024-04-17T09:53:15.066987+0000","last_became_active":"2024-04-17T09:53:13.780726+0000","last_became_peered":"2024-04-17T09:53:13.780726+0000","last_unstale":"2024-04-17T09:53:15.066987+0000","last_undegraded":"2024-04-17T09:53:15.066987+0000","last_fullsized":"2024-04-17T09:53:15.066987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:39:52.213103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066900+0000","last_change":"2024-04-17T09:53:13.806423+0000","last_active":"2024-04-17T09:53:15.066900+0000","last_peered":"2024-04-17T09:53:15.066900+0000","last_clean":"2024-04-17T09:53:15.066900+0000","last_became_active":"2024-04-17T09:53:13.804648+0000","last_became_peered":"2024-04-17T09:53:13.804648+0000","last_unstale":"2024-04-17T09:53:15.066900+0000","last_undegraded":"2024-04-17T09:53:15.066900+0000","last_fullsized":"2024-04-17T09:53:15.066900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:30:17.695383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066152+0000","last_change":"2024-04-17T09:53:13.803405+0000","last_active":"2024-04-17T09:53:15.066152+0000","last_peered":"2024-04-17T09:53:15.066152+0000","last_clean":"2024-04-17T09:53:15.066152+0000","last_became_active":"2024-04-17T09:53:13.803135+0000","last_became_peered":"2024-04-17T09:53:13.803135+0000","last_unstale":"2024-04-17T09:53:15.066152+0000","last_undegraded":"2024-04-17T09:53:15.066152+0000","last_fullsized":"2024-04-17T09:53:15.066152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:12.936111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787899+0000","last_change":"2024-04-17T09:53:13.786719+0000","last_active":"2024-04-17T09:53:14.787899+0000","last_peered":"2024-04-17T09:53:14.787899+0000","last_clean":"2024-04-17T09:53:14.787899+0000","last_became_active":"2024-04-17T09:53:13.786401+0000","last_became_peered":"2024-04-17T09:53:13.786401+0000","last_unstale":"2024-04-17T09:53:14.787899+0000","last_undegraded":"2024-04-17T09:53:14.787899+0000","last_fullsized":"2024-04-17T09:53:14.787899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:55:00.842846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787911+0000","last_change":"2024-04-17T09:53:13.797469+0000","last_active":"2024-04-17T09:53:14.787911+0000","last_peered":"2024-04-17T09:53:14.787911+0000","last_clean":"2024-04-17T09:53:14.787911+0000","last_became_active":"2024-04-17T09:53:13.797180+0000","last_became_peered":"2024-04-17T09:53:13.797180+0000","last_unstale":"2024-04-17T09:53:14.787911+0000","last_undegraded":"2024-04-17T09:53:14.787911+0000","last_fullsized":"2024-04-17T09:53:14.787911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:51:44.795158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066953+0000","last_change":"2024-04-17T09:53:12.783002+0000","last_active":"2024-04-17T09:53:15.066953+0000","last_peered":"2024-04-17T09:53:15.066953+0000","last_clean":"2024-04-17T09:53:15.066953+0000","last_became_active":"2024-04-17T09:53:12.782606+0000","last_became_peered":"2024-04-17T09:53:12.782606+0000","last_unstale":"2024-04-17T09:53:15.066953+0000","last_undegraded":"2024-04-17T09:53:15.066953+0000","last_fullsized":"2024-04-17T09:53:15.066953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:52:55.917109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067085+0000","last_change":"2024-04-17T09:53:13.784184+0000","last_active":"2024-04-17T09:53:15.067085+0000","last_peered":"2024-04-17T09:53:15.067085+0000","last_clean":"2024-04-17T09:53:15.067085+0000","last_became_active":"2024-04-17T09:53:13.783922+0000","last_became_peered":"2024-04-17T09:53:13.783922+0000","last_unstale":"2024-04-17T09:53:15.067085+0000","last_undegraded":"2024-04-17T09:53:15.067085+0000","last_fullsized":"2024-04-17T09:53:15.067085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:57:05.882921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788344+0000","last_change":"2024-04-17T09:53:12.768842+0000","last_active":"2024-04-17T09:53:14.788344+0000","last_peered":"2024-04-17T09:53:14.788344+0000","last_clean":"2024-04-17T09:53:14.788344+0000","last_became_active":"2024-04-17T09:53:12.768227+0000","last_became_peered":"2024-04-17T09:53:12.768227+0000","last_unstale":"2024-04-17T09:53:14.788344+0000","last_undegraded":"2024-04-17T09:53:14.788344+0000","last_fullsized":"2024-04-17T09:53:14.788344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:32:56.010181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066590+0000","last_change":"2024-04-17T09:53:12.773398+0000","last_active":"2024-04-17T09:53:15.066590+0000","last_peered":"2024-04-17T09:53:15.066590+0000","last_clean":"2024-04-17T09:53:15.066590+0000","last_became_active":"2024-04-17T09:53:12.771487+0000","last_became_peered":"2024-04-17T09:53:12.771487+0000","last_unstale":"2024-04-17T09:53:15.066590+0000","last_undegraded":"2024-04-17T09:53:15.066590+0000","last_fullsized":"2024-04-17T09:53:15.066590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:24.555970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066239+0000","last_change":"2024-04-17T09:53:13.782988+0000","last_active":"2024-04-17T09:53:15.066239+0000","last_peered":"2024-04-17T09:53:15.066239+0000","last_clean":"2024-04-17T09:53:15.066239+0000","last_became_active":"2024-04-17T09:53:13.782711+0000","last_became_peered":"2024-04-17T09:53:13.782711+0000","last_unstale":"2024-04-17T09:53:15.066239+0000","last_undegraded":"2024-04-17T09:53:15.066239+0000","last_fullsized":"2024-04-17T09:53:15.066239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:02.320431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066522+0000","last_change":"2024-04-17T09:53:12.778661+0000","last_active":"2024-04-17T09:53:15.066522+0000","last_peered":"2024-04-17T09:53:15.066522+0000","last_clean":"2024-04-17T09:53:15.066522+0000","last_became_active":"2024-04-17T09:53:12.778088+0000","last_became_peered":"2024-04-17T09:53:12.778088+0000","last_unstale":"2024-04-17T09:53:15.066522+0000","last_undegraded":"2024-04-17T09:53:15.066522+0000","last_fullsized":"2024-04-17T09:53:15.066522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:50.549367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787383+0000","last_change":"2024-04-17T09:53:13.792711+0000","last_active":"2024-04-17T09:53:14.787383+0000","last_peered":"2024-04-17T09:53:14.787383+0000","last_clean":"2024-04-17T09:53:14.787383+0000","last_became_active":"2024-04-17T09:53:13.792257+0000","last_became_peered":"2024-04-17T09:53:13.792257+0000","last_unstale":"2024-04-17T09:53:14.787383+0000","last_undegraded":"2024-04-17T09:53:14.787383+0000","last_fullsized":"2024-04-17T09:53:14.787383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:43:18.222168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.789124+0000","last_change":"2024-04-17T09:53:12.779227+0000","last_active":"2024-04-17T09:53:14.789124+0000","last_peered":"2024-04-17T09:53:14.789124+0000","last_clean":"2024-04-17T09:53:14.789124+0000","last_became_active":"2024-04-17T09:53:12.778942+0000","last_became_peered":"2024-04-17T09:53:12.778942+0000","last_unstale":"2024-04-17T09:53:14.789124+0000","last_undegraded":"2024-04-17T09:53:14.789124+0000","last_fullsized":"2024-04-17T09:53:14.789124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:07.296217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066798+0000","last_change":"2024-04-17T09:53:13.812353+0000","last_active":"2024-04-17T09:53:15.066798+0000","last_peered":"2024-04-17T09:53:15.066798+0000","last_clean":"2024-04-17T09:53:15.066798+0000","last_became_active":"2024-04-17T09:53:13.812007+0000","last_became_peered":"2024-04-17T09:53:13.812007+0000","last_unstale":"2024-04-17T09:53:15.066798+0000","last_undegraded":"2024-04-17T09:53:15.066798+0000","last_fullsized":"2024-04-17T09:53:15.066798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:36:37.128859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066937+0000","last_change":"2024-04-17T09:53:12.776576+0000","last_active":"2024-04-17T09:53:15.066937+0000","last_peered":"2024-04-17T09:53:15.066937+0000","last_clean":"2024-04-17T09:53:15.066937+0000","last_became_active":"2024-04-17T09:53:12.772424+0000","last_became_peered":"2024-04-17T09:53:12.772424+0000","last_unstale":"2024-04-17T09:53:15.066937+0000","last_undegraded":"2024-04-17T09:53:15.066937+0000","last_fullsized":"2024-04-17T09:53:15.066937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:53.140146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067737+0000","last_change":"2024-04-17T09:53:13.815907+0000","last_active":"2024-04-17T09:53:15.067737+0000","last_peered":"2024-04-17T09:53:15.067737+0000","last_clean":"2024-04-17T09:53:15.067737+0000","last_became_active":"2024-04-17T09:53:13.815616+0000","last_became_peered":"2024-04-17T09:53:13.815616+0000","last_unstale":"2024-04-17T09:53:15.067737+0000","last_undegraded":"2024-04-17T09:53:15.067737+0000","last_fullsized":"2024-04-17T09:53:15.067737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:33:33.232581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787638+0000","last_change":"2024-04-17T09:53:12.791926+0000","last_active":"2024-04-17T09:53:14.787638+0000","last_peered":"2024-04-17T09:53:14.787638+0000","last_clean":"2024-04-17T09:53:14.787638+0000","last_became_active":"2024-04-17T09:53:12.791644+0000","last_became_peered":"2024-04-17T09:53:12.791644+0000","last_unstale":"2024-04-17T09:53:14.787638+0000","last_undegraded":"2024-04-17T09:53:14.787638+0000","last_fullsized":"2024-04-17T09:53:14.787638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:14.881628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788745+0000","last_change":"2024-04-17T09:53:13.808620+0000","last_active":"2024-04-17T09:53:14.788745+0000","last_peered":"2024-04-17T09:53:14.788745+0000","last_clean":"2024-04-17T09:53:14.788745+0000","last_became_active":"2024-04-17T09:53:13.807344+0000","last_became_peered":"2024-04-17T09:53:13.807344+0000","last_unstale":"2024-04-17T09:53:14.788745+0000","last_undegraded":"2024-04-17T09:53:14.788745+0000","last_fullsized":"2024-04-17T09:53:14.788745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:20:06.508805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.789133+0000","last_change":"2024-04-17T09:53:12.771523+0000","last_active":"2024-04-17T09:53:14.789133+0000","last_peered":"2024-04-17T09:53:14.789133+0000","last_clean":"2024-04-17T09:53:14.789133+0000","last_became_active":"2024-04-17T09:53:12.771201+0000","last_became_peered":"2024-04-17T09:53:12.771201+0000","last_unstale":"2024-04-17T09:53:14.789133+0000","last_undegraded":"2024-04-17T09:53:14.789133+0000","last_fullsized":"2024-04-17T09:53:14.789133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:50.549367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067157+0000","last_change":"2024-04-17T09:53:13.797071+0000","last_active":"2024-04-17T09:53:15.067157+0000","last_peered":"2024-04-17T09:53:15.067157+0000","last_clean":"2024-04-17T09:53:15.067157+0000","last_became_active":"2024-04-17T09:53:13.796572+0000","last_became_peered":"2024-04-17T09:53:13.796572+0000","last_unstale":"2024-04-17T09:53:15.067157+0000","last_undegraded":"2024-04-17T09:53:15.067157+0000","last_fullsized":"2024-04-17T09:53:15.067157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:03:55.116831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067069+0000","last_change":"2024-04-17T09:53:12.778367+0000","last_active":"2024-04-17T09:53:15.067069+0000","last_peered":"2024-04-17T09:53:15.067069+0000","last_clean":"2024-04-17T09:53:15.067069+0000","last_became_active":"2024-04-17T09:53:12.778084+0000","last_became_peered":"2024-04-17T09:53:12.778084+0000","last_unstale":"2024-04-17T09:53:15.067069+0000","last_undegraded":"2024-04-17T09:53:15.067069+0000","last_fullsized":"2024-04-17T09:53:15.067069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:19:28.217974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066747+0000","last_change":"2024-04-17T09:53:13.798230+0000","last_active":"2024-04-17T09:53:15.066747+0000","last_peered":"2024-04-17T09:53:15.066747+0000","last_clean":"2024-04-17T09:53:15.066747+0000","last_became_active":"2024-04-17T09:53:13.797873+0000","last_became_peered":"2024-04-17T09:53:13.797873+0000","last_unstale":"2024-04-17T09:53:15.066747+0000","last_undegraded":"2024-04-17T09:53:15.066747+0000","last_fullsized":"2024-04-17T09:53:15.066747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:43:55.704203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067037+0000","last_change":"2024-04-17T09:53:12.784831+0000","last_active":"2024-04-17T09:53:15.067037+0000","last_peered":"2024-04-17T09:53:15.067037+0000","last_clean":"2024-04-17T09:53:15.067037+0000","last_became_active":"2024-04-17T09:53:12.784012+0000","last_became_peered":"2024-04-17T09:53:12.784012+0000","last_unstale":"2024-04-17T09:53:15.067037+0000","last_undegraded":"2024-04-17T09:53:15.067037+0000","last_fullsized":"2024-04-17T09:53:15.067037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:09.944062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067415+0000","last_change":"2024-04-17T09:53:13.809261+0000","last_active":"2024-04-17T09:53:15.067415+0000","last_peered":"2024-04-17T09:53:15.067415+0000","last_clean":"2024-04-17T09:53:15.067415+0000","last_became_active":"2024-04-17T09:53:13.809004+0000","last_became_peered":"2024-04-17T09:53:13.809004+0000","last_unstale":"2024-04-17T09:53:15.067415+0000","last_undegraded":"2024-04-17T09:53:15.067415+0000","last_fullsized":"2024-04-17T09:53:15.067415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:33:32.665364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788374+0000","last_change":"2024-04-17T09:53:12.768643+0000","last_active":"2024-04-17T09:53:14.788374+0000","last_peered":"2024-04-17T09:53:14.788374+0000","last_clean":"2024-04-17T09:53:14.788374+0000","last_became_active":"2024-04-17T09:53:12.768221+0000","last_became_peered":"2024-04-17T09:53:12.768221+0000","last_unstale":"2024-04-17T09:53:14.788374+0000","last_undegraded":"2024-04-17T09:53:14.788374+0000","last_fullsized":"2024-04-17T09:53:14.788374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:14:18.407034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788658+0000","last_change":"2024-04-17T09:53:13.788193+0000","last_active":"2024-04-17T09:53:14.788658+0000","last_peered":"2024-04-17T09:53:14.788658+0000","last_clean":"2024-04-17T09:53:14.788658+0000","last_became_active":"2024-04-17T09:53:13.787942+0000","last_became_peered":"2024-04-17T09:53:13.787942+0000","last_unstale":"2024-04-17T09:53:14.788658+0000","last_undegraded":"2024-04-17T09:53:14.788658+0000","last_fullsized":"2024-04-17T09:53:14.788658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:17:54.590392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066252+0000","last_change":"2024-04-17T09:53:12.767452+0000","last_active":"2024-04-17T09:53:15.066252+0000","last_peered":"2024-04-17T09:53:15.066252+0000","last_clean":"2024-04-17T09:53:15.066252+0000","last_became_active":"2024-04-17T09:53:12.766628+0000","last_became_peered":"2024-04-17T09:53:12.766628+0000","last_unstale":"2024-04-17T09:53:15.066252+0000","last_undegraded":"2024-04-17T09:53:15.066252+0000","last_fullsized":"2024-04-17T09:53:15.066252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:07.296217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067067+0000","last_change":"2024-04-17T09:53:13.803929+0000","last_active":"2024-04-17T09:53:15.067067+0000","last_peered":"2024-04-17T09:53:15.067067+0000","last_clean":"2024-04-17T09:53:15.067067+0000","last_became_active":"2024-04-17T09:53:13.803651+0000","last_became_peered":"2024-04-17T09:53:13.803651+0000","last_unstale":"2024-04-17T09:53:15.067067+0000","last_undegraded":"2024-04-17T09:53:15.067067+0000","last_fullsized":"2024-04-17T09:53:15.067067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:36:37.128859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.789205+0000","last_change":"2024-04-17T09:53:12.781091+0000","last_active":"2024-04-17T09:53:14.789205+0000","last_peered":"2024-04-17T09:53:14.789205+0000","last_clean":"2024-04-17T09:53:14.789205+0000","last_became_active":"2024-04-17T09:53:12.780752+0000","last_became_peered":"2024-04-17T09:53:12.780752+0000","last_unstale":"2024-04-17T09:53:14.789205+0000","last_undegraded":"2024-04-17T09:53:14.789205+0000","last_fullsized":"2024-04-17T09:53:14.789205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:31:08.049836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067755+0000","last_change":"2024-04-17T09:53:13.810509+0000","last_active":"2024-04-17T09:53:15.067755+0000","last_peered":"2024-04-17T09:53:15.067755+0000","last_clean":"2024-04-17T09:53:15.067755+0000","last_became_active":"2024-04-17T09:53:13.809888+0000","last_became_peered":"2024-04-17T09:53:13.809888+0000","last_unstale":"2024-04-17T09:53:15.067755+0000","last_undegraded":"2024-04-17T09:53:15.067755+0000","last_fullsized":"2024-04-17T09:53:15.067755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:34:37.237156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067459+0000","last_change":"2024-04-17T09:53:12.778623+0000","last_active":"2024-04-17T09:53:15.067459+0000","last_peered":"2024-04-17T09:53:15.067459+0000","last_clean":"2024-04-17T09:53:15.067459+0000","last_became_active":"2024-04-17T09:53:12.774044+0000","last_became_peered":"2024-04-17T09:53:12.774044+0000","last_unstale":"2024-04-17T09:53:15.067459+0000","last_undegraded":"2024-04-17T09:53:15.067459+0000","last_fullsized":"2024-04-17T09:53:15.067459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:09.944062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.068004+0000","last_change":"2024-04-17T09:53:13.809464+0000","last_active":"2024-04-17T09:53:15.068004+0000","last_peered":"2024-04-17T09:53:15.068004+0000","last_clean":"2024-04-17T09:53:15.068004+0000","last_became_active":"2024-04-17T09:53:13.808615+0000","last_became_peered":"2024-04-17T09:53:13.808615+0000","last_unstale":"2024-04-17T09:53:15.068004+0000","last_undegraded":"2024-04-17T09:53:15.068004+0000","last_fullsized":"2024-04-17T09:53:15.068004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:52:32.791236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787694+0000","last_change":"2024-04-17T09:53:12.776880+0000","last_active":"2024-04-17T09:53:14.787694+0000","last_peered":"2024-04-17T09:53:14.787694+0000","last_clean":"2024-04-17T09:53:14.787694+0000","last_became_active":"2024-04-17T09:53:12.776002+0000","last_became_peered":"2024-04-17T09:53:12.776002+0000","last_unstale":"2024-04-17T09:53:14.787694+0000","last_undegraded":"2024-04-17T09:53:14.787694+0000","last_fullsized":"2024-04-17T09:53:14.787694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:09.944062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066950+0000","last_change":"2024-04-17T09:53:13.793405+0000","last_active":"2024-04-17T09:53:15.066950+0000","last_peered":"2024-04-17T09:53:15.066950+0000","last_clean":"2024-04-17T09:53:15.066950+0000","last_became_active":"2024-04-17T09:53:13.792553+0000","last_became_peered":"2024-04-17T09:53:13.792553+0000","last_unstale":"2024-04-17T09:53:15.066950+0000","last_undegraded":"2024-04-17T09:53:15.066950+0000","last_fullsized":"2024-04-17T09:53:15.066950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:43:23.411109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066128+0000","last_change":"2024-04-17T09:53:12.780453+0000","last_active":"2024-04-17T09:53:15.066128+0000","last_peered":"2024-04-17T09:53:15.066128+0000","last_clean":"2024-04-17T09:53:15.066128+0000","last_became_active":"2024-04-17T09:53:12.780130+0000","last_became_peered":"2024-04-17T09:53:12.780130+0000","last_unstale":"2024-04-17T09:53:15.066128+0000","last_undegraded":"2024-04-17T09:53:15.066128+0000","last_fullsized":"2024-04-17T09:53:15.066128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:14:18.407034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067313+0000","last_change":"2024-04-17T09:53:13.802442+0000","last_active":"2024-04-17T09:53:15.067313+0000","last_peered":"2024-04-17T09:53:15.067313+0000","last_clean":"2024-04-17T09:53:15.067313+0000","last_became_active":"2024-04-17T09:53:13.802168+0000","last_became_peered":"2024-04-17T09:53:13.802168+0000","last_unstale":"2024-04-17T09:53:15.067313+0000","last_undegraded":"2024-04-17T09:53:15.067313+0000","last_fullsized":"2024-04-17T09:53:15.067313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:30:17.695383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066329+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T09:53:15.066329+0000","last_peered":"2024-04-17T09:53:15.066329+0000","last_clean":"2024-04-17T09:53:15.066329+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T09:53:15.066329+0000","last_undegraded":"2024-04-17T09:53:15.066329+0000","last_fullsized":"2024-04-17T09:53:15.066329+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066649+0000","last_change":"2024-04-17T09:53:12.772905+0000","last_active":"2024-04-17T09:53:15.066649+0000","last_peered":"2024-04-17T09:53:15.066649+0000","last_clean":"2024-04-17T09:53:15.066649+0000","last_became_active":"2024-04-17T09:53:12.772635+0000","last_became_peered":"2024-04-17T09:53:12.772635+0000","last_unstale":"2024-04-17T09:53:15.066649+0000","last_undegraded":"2024-04-17T09:53:15.066649+0000","last_fullsized":"2024-04-17T09:53:15.066649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:11.929676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066741+0000","last_change":"2024-04-17T09:53:13.815993+0000","last_active":"2024-04-17T09:53:15.066741+0000","last_peered":"2024-04-17T09:53:15.066741+0000","last_clean":"2024-04-17T09:53:15.066741+0000","last_became_active":"2024-04-17T09:53:13.815689+0000","last_became_peered":"2024-04-17T09:53:13.815689+0000","last_unstale":"2024-04-17T09:53:15.066741+0000","last_undegraded":"2024-04-17T09:53:15.066741+0000","last_fullsized":"2024-04-17T09:53:15.066741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:51:42.871563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066279+0000","last_change":"2024-04-17T09:53:12.790967+0000","last_active":"2024-04-17T09:53:15.066279+0000","last_peered":"2024-04-17T09:53:15.066279+0000","last_clean":"2024-04-17T09:53:15.066279+0000","last_became_active":"2024-04-17T09:53:12.790637+0000","last_became_peered":"2024-04-17T09:53:12.790637+0000","last_unstale":"2024-04-17T09:53:15.066279+0000","last_undegraded":"2024-04-17T09:53:15.066279+0000","last_fullsized":"2024-04-17T09:53:15.066279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:14.881628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066338+0000","last_change":"2024-04-17T09:53:13.788385+0000","last_active":"2024-04-17T09:53:15.066338+0000","last_peered":"2024-04-17T09:53:15.066338+0000","last_clean":"2024-04-17T09:53:15.066338+0000","last_became_active":"2024-04-17T09:53:13.784112+0000","last_became_peered":"2024-04-17T09:53:13.784112+0000","last_unstale":"2024-04-17T09:53:15.066338+0000","last_undegraded":"2024-04-17T09:53:15.066338+0000","last_fullsized":"2024-04-17T09:53:15.066338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:30:00.992434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067778+0000","last_change":"2024-04-17T09:53:12.775859+0000","last_active":"2024-04-17T09:53:15.067778+0000","last_peered":"2024-04-17T09:53:15.067778+0000","last_clean":"2024-04-17T09:53:15.067778+0000","last_became_active":"2024-04-17T09:53:12.774051+0000","last_became_peered":"2024-04-17T09:53:12.774051+0000","last_unstale":"2024-04-17T09:53:15.067778+0000","last_undegraded":"2024-04-17T09:53:15.067778+0000","last_fullsized":"2024-04-17T09:53:15.067778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:32:56.010181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787431+0000","last_change":"2024-04-17T09:53:13.792329+0000","last_active":"2024-04-17T09:53:14.787431+0000","last_peered":"2024-04-17T09:53:14.787431+0000","last_clean":"2024-04-17T09:53:14.787431+0000","last_became_active":"2024-04-17T09:53:13.790615+0000","last_became_peered":"2024-04-17T09:53:13.790615+0000","last_unstale":"2024-04-17T09:53:14.787431+0000","last_undegraded":"2024-04-17T09:53:14.787431+0000","last_fullsized":"2024-04-17T09:53:14.787431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:02.320431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067057+0000","last_change":"2024-04-17T09:53:12.787408+0000","last_active":"2024-04-17T09:53:15.067057+0000","last_peered":"2024-04-17T09:53:15.067057+0000","last_clean":"2024-04-17T09:53:15.067057+0000","last_became_active":"2024-04-17T09:53:12.786322+0000","last_became_peered":"2024-04-17T09:53:12.786322+0000","last_unstale":"2024-04-17T09:53:15.067057+0000","last_undegraded":"2024-04-17T09:53:15.067057+0000","last_fullsized":"2024-04-17T09:53:15.067057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:14.881628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788243+0000","last_change":"2024-04-17T09:53:13.786023+0000","last_active":"2024-04-17T09:53:14.788243+0000","last_peered":"2024-04-17T09:53:14.788243+0000","last_clean":"2024-04-17T09:53:14.788243+0000","last_became_active":"2024-04-17T09:53:13.785528+0000","last_became_peered":"2024-04-17T09:53:13.785528+0000","last_unstale":"2024-04-17T09:53:14.788243+0000","last_undegraded":"2024-04-17T09:53:14.788243+0000","last_fullsized":"2024-04-17T09:53:14.788243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:09:58.265496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787601+0000","last_change":"2024-04-17T09:53:12.786776+0000","last_active":"2024-04-17T09:53:14.787601+0000","last_peered":"2024-04-17T09:53:14.787601+0000","last_clean":"2024-04-17T09:53:14.787601+0000","last_became_active":"2024-04-17T09:53:12.786511+0000","last_became_peered":"2024-04-17T09:53:12.786511+0000","last_unstale":"2024-04-17T09:53:14.787601+0000","last_undegraded":"2024-04-17T09:53:14.787601+0000","last_fullsized":"2024-04-17T09:53:14.787601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:32:56.010181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067174+0000","last_change":"2024-04-17T09:53:13.795421+0000","last_active":"2024-04-17T09:53:15.067174+0000","last_peered":"2024-04-17T09:53:15.067174+0000","last_clean":"2024-04-17T09:53:15.067174+0000","last_became_active":"2024-04-17T09:53:13.793404+0000","last_became_peered":"2024-04-17T09:53:13.793404+0000","last_unstale":"2024-04-17T09:53:15.067174+0000","last_undegraded":"2024-04-17T09:53:15.067174+0000","last_fullsized":"2024-04-17T09:53:15.067174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:27:15.142091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067608+0000","last_change":"2024-04-17T09:53:12.786549+0000","last_active":"2024-04-17T09:53:15.067608+0000","last_peered":"2024-04-17T09:53:15.067608+0000","last_clean":"2024-04-17T09:53:15.067608+0000","last_became_active":"2024-04-17T09:53:12.784745+0000","last_became_peered":"2024-04-17T09:53:12.784745+0000","last_unstale":"2024-04-17T09:53:15.067608+0000","last_undegraded":"2024-04-17T09:53:15.067608+0000","last_fullsized":"2024-04-17T09:53:15.067608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:57:23.703708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787659+0000","last_change":"2024-04-17T09:53:13.792711+0000","last_active":"2024-04-17T09:53:14.787659+0000","last_peered":"2024-04-17T09:53:14.787659+0000","last_clean":"2024-04-17T09:53:14.787659+0000","last_became_active":"2024-04-17T09:53:13.791107+0000","last_became_peered":"2024-04-17T09:53:13.791107+0000","last_unstale":"2024-04-17T09:53:14.787659+0000","last_undegraded":"2024-04-17T09:53:14.787659+0000","last_fullsized":"2024-04-17T09:53:14.787659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:17.663523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.795170+0000","last_change":"2024-04-17T09:53:12.770038+0000","last_active":"2024-04-17T09:53:14.795170+0000","last_peered":"2024-04-17T09:53:14.795170+0000","last_clean":"2024-04-17T09:53:14.795170+0000","last_became_active":"2024-04-17T09:53:12.769750+0000","last_became_peered":"2024-04-17T09:53:12.769750+0000","last_unstale":"2024-04-17T09:53:14.795170+0000","last_undegraded":"2024-04-17T09:53:14.795170+0000","last_fullsized":"2024-04-17T09:53:14.795170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:50.549367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067988+0000","last_change":"2024-04-17T09:53:13.801993+0000","last_active":"2024-04-17T09:53:15.067988+0000","last_peered":"2024-04-17T09:53:15.067988+0000","last_clean":"2024-04-17T09:53:15.067988+0000","last_became_active":"2024-04-17T09:53:13.801328+0000","last_became_peered":"2024-04-17T09:53:13.801328+0000","last_unstale":"2024-04-17T09:53:15.067988+0000","last_undegraded":"2024-04-17T09:53:15.067988+0000","last_fullsized":"2024-04-17T09:53:15.067988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:03:55.116831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.789207+0000","last_change":"2024-04-17T09:53:12.781559+0000","last_active":"2024-04-17T09:53:14.789207+0000","last_peered":"2024-04-17T09:53:14.789207+0000","last_clean":"2024-04-17T09:53:14.789207+0000","last_became_active":"2024-04-17T09:53:12.780702+0000","last_became_peered":"2024-04-17T09:53:12.780702+0000","last_unstale":"2024-04-17T09:53:14.789207+0000","last_undegraded":"2024-04-17T09:53:14.789207+0000","last_fullsized":"2024-04-17T09:53:14.789207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:09.944062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067614+0000","last_change":"2024-04-17T09:53:13.795015+0000","last_active":"2024-04-17T09:53:15.067614+0000","last_peered":"2024-04-17T09:53:15.067614+0000","last_clean":"2024-04-17T09:53:15.067614+0000","last_became_active":"2024-04-17T09:53:13.794609+0000","last_became_peered":"2024-04-17T09:53:13.794609+0000","last_unstale":"2024-04-17T09:53:15.067614+0000","last_undegraded":"2024-04-17T09:53:15.067614+0000","last_fullsized":"2024-04-17T09:53:15.067614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:35.205605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066266+0000","last_change":"2024-04-17T09:53:12.781463+0000","last_active":"2024-04-17T09:53:15.066266+0000","last_peered":"2024-04-17T09:53:15.066266+0000","last_clean":"2024-04-17T09:53:15.066266+0000","last_became_active":"2024-04-17T09:53:12.781138+0000","last_became_peered":"2024-04-17T09:53:12.781138+0000","last_unstale":"2024-04-17T09:53:15.066266+0000","last_undegraded":"2024-04-17T09:53:15.066266+0000","last_fullsized":"2024-04-17T09:53:15.066266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:09.944062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788264+0000","last_change":"2024-04-17T09:53:13.797478+0000","last_active":"2024-04-17T09:53:14.788264+0000","last_peered":"2024-04-17T09:53:14.788264+0000","last_clean":"2024-04-17T09:53:14.788264+0000","last_became_active":"2024-04-17T09:53:13.797152+0000","last_became_peered":"2024-04-17T09:53:13.797152+0000","last_unstale":"2024-04-17T09:53:14.788264+0000","last_undegraded":"2024-04-17T09:53:14.788264+0000","last_fullsized":"2024-04-17T09:53:14.788264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:59:22.224091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787486+0000","last_change":"2024-04-17T09:53:12.785848+0000","last_active":"2024-04-17T09:53:14.787486+0000","last_peered":"2024-04-17T09:53:14.787486+0000","last_clean":"2024-04-17T09:53:14.787486+0000","last_became_active":"2024-04-17T09:53:12.784980+0000","last_became_peered":"2024-04-17T09:53:12.784980+0000","last_unstale":"2024-04-17T09:53:14.787486+0000","last_undegraded":"2024-04-17T09:53:14.787486+0000","last_fullsized":"2024-04-17T09:53:14.787486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:11.929676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066406+0000","last_change":"2024-04-17T09:53:13.796607+0000","last_active":"2024-04-17T09:53:15.066406+0000","last_peered":"2024-04-17T09:53:15.066406+0000","last_clean":"2024-04-17T09:53:15.066406+0000","last_became_active":"2024-04-17T09:53:13.796351+0000","last_became_peered":"2024-04-17T09:53:13.796351+0000","last_unstale":"2024-04-17T09:53:15.066406+0000","last_undegraded":"2024-04-17T09:53:15.066406+0000","last_fullsized":"2024-04-17T09:53:15.066406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:17:54.590392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787463+0000","last_change":"2024-04-17T09:53:12.788441+0000","last_active":"2024-04-17T09:53:14.787463+0000","last_peered":"2024-04-17T09:53:14.787463+0000","last_clean":"2024-04-17T09:53:14.787463+0000","last_became_active":"2024-04-17T09:53:12.787431+0000","last_became_peered":"2024-04-17T09:53:12.787431+0000","last_unstale":"2024-04-17T09:53:14.787463+0000","last_undegraded":"2024-04-17T09:53:14.787463+0000","last_fullsized":"2024-04-17T09:53:14.787463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:14:18.407034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066772+0000","last_change":"2024-04-17T09:53:13.783282+0000","last_active":"2024-04-17T09:53:15.066772+0000","last_peered":"2024-04-17T09:53:15.066772+0000","last_clean":"2024-04-17T09:53:15.066772+0000","last_became_active":"2024-04-17T09:53:13.782978+0000","last_became_peered":"2024-04-17T09:53:13.782978+0000","last_unstale":"2024-04-17T09:53:15.066772+0000","last_undegraded":"2024-04-17T09:53:15.066772+0000","last_fullsized":"2024-04-17T09:53:15.066772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:55:01.176803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067538+0000","last_change":"2024-04-17T09:53:12.785555+0000","last_active":"2024-04-17T09:53:15.067538+0000","last_peered":"2024-04-17T09:53:15.067538+0000","last_clean":"2024-04-17T09:53:15.067538+0000","last_became_active":"2024-04-17T09:53:12.785233+0000","last_became_peered":"2024-04-17T09:53:12.785233+0000","last_unstale":"2024-04-17T09:53:15.067538+0000","last_undegraded":"2024-04-17T09:53:15.067538+0000","last_fullsized":"2024-04-17T09:53:15.067538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:50.549367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788225+0000","last_change":"2024-04-17T09:53:13.800230+0000","last_active":"2024-04-17T09:53:14.788225+0000","last_peered":"2024-04-17T09:53:14.788225+0000","last_clean":"2024-04-17T09:53:14.788225+0000","last_became_active":"2024-04-17T09:53:13.799917+0000","last_became_peered":"2024-04-17T09:53:13.799917+0000","last_unstale":"2024-04-17T09:53:14.788225+0000","last_undegraded":"2024-04-17T09:53:14.788225+0000","last_fullsized":"2024-04-17T09:53:14.788225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:43:18.222168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067005+0000","last_change":"2024-04-17T09:53:13.814147+0000","last_active":"2024-04-17T09:53:15.067005+0000","last_peered":"2024-04-17T09:53:15.067005+0000","last_clean":"2024-04-17T09:53:15.067005+0000","last_became_active":"2024-04-17T09:53:13.813807+0000","last_became_peered":"2024-04-17T09:53:13.813807+0000","last_unstale":"2024-04-17T09:53:15.067005+0000","last_undegraded":"2024-04-17T09:53:15.067005+0000","last_fullsized":"2024-04-17T09:53:15.067005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:33:32.665364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067456+0000","last_change":"2024-04-17T09:53:12.788509+0000","last_active":"2024-04-17T09:53:15.067456+0000","last_peered":"2024-04-17T09:53:15.067456+0000","last_clean":"2024-04-17T09:53:15.067456+0000","last_became_active":"2024-04-17T09:53:12.788161+0000","last_became_peered":"2024-04-17T09:53:12.788161+0000","last_unstale":"2024-04-17T09:53:15.067456+0000","last_undegraded":"2024-04-17T09:53:15.067456+0000","last_fullsized":"2024-04-17T09:53:15.067456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:11.929676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066835+0000","last_change":"2024-04-17T09:53:13.812181+0000","last_active":"2024-04-17T09:53:15.066835+0000","last_peered":"2024-04-17T09:53:15.066835+0000","last_clean":"2024-04-17T09:53:15.066835+0000","last_became_active":"2024-04-17T09:53:13.811836+0000","last_became_peered":"2024-04-17T09:53:13.811836+0000","last_unstale":"2024-04-17T09:53:15.066835+0000","last_undegraded":"2024-04-17T09:53:15.066835+0000","last_fullsized":"2024-04-17T09:53:15.066835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:03:55.116831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067392+0000","last_change":"2024-04-17T09:53:12.786015+0000","last_active":"2024-04-17T09:53:15.067392+0000","last_peered":"2024-04-17T09:53:15.067392+0000","last_clean":"2024-04-17T09:53:15.067392+0000","last_became_active":"2024-04-17T09:53:12.784577+0000","last_became_peered":"2024-04-17T09:53:12.784577+0000","last_unstale":"2024-04-17T09:53:15.067392+0000","last_undegraded":"2024-04-17T09:53:15.067392+0000","last_fullsized":"2024-04-17T09:53:15.067392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:53.140146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067598+0000","last_change":"2024-04-17T09:53:13.819959+0000","last_active":"2024-04-17T09:53:15.067598+0000","last_peered":"2024-04-17T09:53:15.067598+0000","last_clean":"2024-04-17T09:53:15.067598+0000","last_became_active":"2024-04-17T09:53:13.819509+0000","last_became_peered":"2024-04-17T09:53:13.819509+0000","last_unstale":"2024-04-17T09:53:15.067598+0000","last_undegraded":"2024-04-17T09:53:15.067598+0000","last_fullsized":"2024-04-17T09:53:15.067598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:07:24.542013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067396+0000","last_change":"2024-04-17T09:53:12.784607+0000","last_active":"2024-04-17T09:53:15.067396+0000","last_peered":"2024-04-17T09:53:15.067396+0000","last_clean":"2024-04-17T09:53:15.067396+0000","last_became_active":"2024-04-17T09:53:12.784265+0000","last_became_peered":"2024-04-17T09:53:12.784265+0000","last_unstale":"2024-04-17T09:53:15.067396+0000","last_undegraded":"2024-04-17T09:53:15.067396+0000","last_fullsized":"2024-04-17T09:53:15.067396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:07.296217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067248+0000","last_change":"2024-04-17T09:53:13.795050+0000","last_active":"2024-04-17T09:53:15.067248+0000","last_peered":"2024-04-17T09:53:15.067248+0000","last_clean":"2024-04-17T09:53:15.067248+0000","last_became_active":"2024-04-17T09:53:13.794002+0000","last_became_peered":"2024-04-17T09:53:13.794002+0000","last_unstale":"2024-04-17T09:53:15.067248+0000","last_undegraded":"2024-04-17T09:53:15.067248+0000","last_fullsized":"2024-04-17T09:53:15.067248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:16:29.674656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067769+0000","last_change":"2024-04-17T09:53:12.782069+0000","last_active":"2024-04-17T09:53:15.067769+0000","last_peered":"2024-04-17T09:53:15.067769+0000","last_clean":"2024-04-17T09:53:15.067769+0000","last_became_active":"2024-04-17T09:53:12.781795+0000","last_became_peered":"2024-04-17T09:53:12.781795+0000","last_unstale":"2024-04-17T09:53:15.067769+0000","last_undegraded":"2024-04-17T09:53:15.067769+0000","last_fullsized":"2024-04-17T09:53:15.067769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:48.300937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":806912,"data_stored":663137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":72277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":208896,"data_stored":72187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":72347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":72313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":815104,"data_stored":663193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T09:53:22.593 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T09:53:22.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.771+0000 7f6151800640 1 Processor -- start 2024-04-17T09:53:22.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.771+0000 7f6151800640 1 -- start start 2024-04-17T09:53:22.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.771+0000 7f6151800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f614c145f30 0x7f614c071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:22.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.771+0000 7f6151800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f614c15d8d0 0x7f614c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:22.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.771+0000 7f6151800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f614c0cce70 0x7f614c0cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:22.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.771+0000 7f6151800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f614c06f400 con 0x7f614c0cce70 2024-04-17T09:53:22.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.771+0000 7f6151800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f614c146310 con 0x7f614c15d8d0 2024-04-17T09:53:22.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.771+0000 7f6151800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f614c15d440 con 0x7f614c145f30 2024-04-17T09:53:22.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.771+0000 7f614b400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f614c0cce70 0x7f614c0cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:22.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.771+0000 7f614b400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f614c0cce70 0x7f614c0cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49172/0 (socket says 172.21.15.32:49172) 2024-04-17T09:53:22.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.771+0000 7f614b400640 1 -- 172.21.15.32:0/1871268970 learned_addr learned my addr 172.21.15.32:0/1871268970 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:22.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.771+0000 7f6142000640 1 --2- 172.21.15.32:0/1871268970 >> v2:172.21.15.144:3300/0 conn(0x7f614c15d8d0 0x7f614c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:22.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.771+0000 7f614aa00640 1 --2- 172.21.15.32:0/1871268970 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f614c145f30 0x7f614c071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:22.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.771+0000 7f614b400640 1 -- 172.21.15.32:0/1871268970 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f614c145f30 msgr2=0x7f614c071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:22.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.771+0000 7f614b400640 1 --2- 172.21.15.32:0/1871268970 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f614c145f30 0x7f614c071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.771+0000 7f614b400640 1 -- 172.21.15.32:0/1871268970 >> v2:172.21.15.144:3300/0 conn(0x7f614c15d8d0 msgr2=0x7f614c072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:22.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.771+0000 7f614b400640 1 --2- 172.21.15.32:0/1871268970 >> v2:172.21.15.144:3300/0 conn(0x7f614c15d8d0 0x7f614c072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.771+0000 7f614b400640 1 -- 172.21.15.32:0/1871268970 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614c0cd840 con 0x7f614c0cce70 2024-04-17T09:53:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.775+0000 7f614b400640 1 --2- 172.21.15.32:0/1871268970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f614c0cce70 0x7f614c0cd300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f614c0b58b0 tx=0x7f614401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae90a70c7b8ece server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.775+0000 7f614a000640 1 -- 172.21.15.32:0/1871268970 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61440b4070 con 0x7f614c0cce70 2024-04-17T09:53:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.775+0000 7f614a000640 1 -- 172.21.15.32:0/1871268970 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61440af400 con 0x7f614c0cce70 2024-04-17T09:53:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.775+0000 7f614a000640 1 -- 172.21.15.32:0/1871268970 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61440af7d0 con 0x7f614c0cce70 2024-04-17T09:53:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.775+0000 7f6151800640 1 -- 172.21.15.32:0/1871268970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f614c0cce70 msgr2=0x7f614c0cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.775+0000 7f6151800640 1 --2- 172.21.15.32:0/1871268970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f614c0cce70 0x7f614c0cd300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f614c0b58b0 tx=0x7f614401f1e0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.775+0000 7f6151800640 1 -- 172.21.15.32:0/1871268970 shutdown_connections 2024-04-17T09:53:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.775+0000 7f6151800640 1 --2- 172.21.15.32:0/1871268970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f614c0cce70 0x7f614c0cd300 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.775+0000 7f6151800640 1 --2- 172.21.15.32:0/1871268970 >> v2:172.21.15.144:3300/0 conn(0x7f614c15d8d0 0x7f614c072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.775+0000 7f6151800640 1 --2- 172.21.15.32:0/1871268970 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f614c145f30 0x7f614c071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.775+0000 7f6151800640 1 -- 172.21.15.32:0/1871268970 >> 172.21.15.32:0/1871268970 conn(0x7f614c06ba30 msgr2=0x7f614c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:22.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.775+0000 7f6151800640 1 -- 172.21.15.32:0/1871268970 shutdown_connections 2024-04-17T09:53:22.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.775+0000 7f6151800640 1 -- 172.21.15.32:0/1871268970 wait complete. 2024-04-17T09:53:22.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f6151800640 1 Processor -- start 2024-04-17T09:53:22.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f6151800640 1 -- start start 2024-04-17T09:53:22.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f6151800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f614c0cce70 0x7f614c0b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:22.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f6151800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f614c145f30 0x7f614c15c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:22.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f6151800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f614c15d8d0 0x7f614c141860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:22.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f6151800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f614c15cdf0 con 0x7f614c0cce70 2024-04-17T09:53:22.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f6151800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f614c15cb90 con 0x7f614c145f30 2024-04-17T09:53:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f6151800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f614c15d0d0 con 0x7f614c15d8d0 2024-04-17T09:53:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f614aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f614c0cce70 0x7f614c0b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f614aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f614c0cce70 0x7f614c0b3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49176/0 (socket says 172.21.15.32:49176) 2024-04-17T09:53:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f614aa00640 1 -- 172.21.15.32:0/884641848 learned_addr learned my addr 172.21.15.32:0/884641848 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f6142000640 1 --2- 172.21.15.32:0/884641848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f614c145f30 0x7f614c15c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f614b400640 1 --2- 172.21.15.32:0/884641848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f614c15d8d0 0x7f614c141860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f614aa00640 1 -- 172.21.15.32:0/884641848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f614c15d8d0 msgr2=0x7f614c141860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f614aa00640 1 --2- 172.21.15.32:0/884641848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f614c15d8d0 0x7f614c141860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f614aa00640 1 -- 172.21.15.32:0/884641848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f614c145f30 msgr2=0x7f614c15c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f614aa00640 1 --2- 172.21.15.32:0/884641848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f614c145f30 0x7f614c15c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f614aa00640 1 -- 172.21.15.32:0/884641848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614c0c9ab0 con 0x7f614c0cce70 2024-04-17T09:53:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f614aa00640 1 --2- 172.21.15.32:0/884641848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f614c0cce70 0x7f614c0b3fa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f613c00d800 tx=0x7f613c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f6143400640 1 -- 172.21.15.32:0/884641848 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c018020 con 0x7f614c0cce70 2024-04-17T09:53:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f6143400640 1 -- 172.21.15.32:0/884641848 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f613c0096a0 con 0x7f614c0cce70 2024-04-17T09:53:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f6143400640 1 -- 172.21.15.32:0/884641848 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c009a50 con 0x7f614c0cce70 2024-04-17T09:53:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f6151800640 1 -- 172.21.15.32:0/884641848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f614c0ca3b0 con 0x7f614c0cce70 2024-04-17T09:53:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.779+0000 7f6151800640 1 -- 172.21.15.32:0/884641848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f614c0ca970 con 0x7f614c0cce70 2024-04-17T09:53:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.783+0000 7f6151800640 1 -- 172.21.15.32:0/884641848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f614c068a90 con 0x7f614c0cce70 2024-04-17T09:53:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.783+0000 7f6143400640 1 -- 172.21.15.32:0/884641848 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f613c01b020 con 0x7f614c0cce70 2024-04-17T09:53:22.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.783+0000 7f6143400640 1 --2- 172.21.15.32:0/884641848 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6128068380 0x7f612806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:22.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.783+0000 7f6143400640 1 -- 172.21.15.32:0/884641848 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f613c086fb0 con 0x7f614c0cce70 2024-04-17T09:53:22.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.783+0000 7f6142000640 1 --2- 172.21.15.32:0/884641848 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6128068380 0x7f612806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:22.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.783+0000 7f6142000640 1 --2- 172.21.15.32:0/884641848 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6128068380 0x7f612806a840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f614c0b2b80 tx=0x7f6134008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:22.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.787+0000 7f6143400640 1 -- 172.21.15.32:0/884641848 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f613c022020 con 0x7f614c0cce70 2024-04-17T09:53:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.951+0000 7f6151800640 1 -- 172.21.15.32:0/884641848 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f614c072f90 con 0x7f6128068380 2024-04-17T09:53:22.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.955+0000 7f6143400640 1 -- 172.21.15.32:0/884641848 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f614c072f90 con 0x7f6128068380 2024-04-17T09:53:22.962 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T09:53:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.959+0000 7f6151800640 1 -- 172.21.15.32:0/884641848 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6128068380 msgr2=0x7f612806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.959+0000 7f6151800640 1 --2- 172.21.15.32:0/884641848 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6128068380 0x7f612806a840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f614c0b2b80 tx=0x7f6134008040 comp rx=0 tx=0).stop 2024-04-17T09:53:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.959+0000 7f6151800640 1 -- 172.21.15.32:0/884641848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f614c0cce70 msgr2=0x7f614c0b3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.959+0000 7f6151800640 1 --2- 172.21.15.32:0/884641848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f614c0cce70 0x7f614c0b3fa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f613c00d800 tx=0x7f613c00dcc0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.959+0000 7f6151800640 1 -- 172.21.15.32:0/884641848 shutdown_connections 2024-04-17T09:53:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.959+0000 7f6151800640 1 --2- 172.21.15.32:0/884641848 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6128068380 0x7f612806a840 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.959+0000 7f6151800640 1 --2- 172.21.15.32:0/884641848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f614c15d8d0 0x7f614c141860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.959+0000 7f6151800640 1 --2- 172.21.15.32:0/884641848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f614c145f30 0x7f614c15c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.959+0000 7f6151800640 1 --2- 172.21.15.32:0/884641848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f614c0cce70 0x7f614c0b3fa0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.959+0000 7f6151800640 1 -- 172.21.15.32:0/884641848 >> 172.21.15.32:0/884641848 conn(0x7f614c06ba30 msgr2=0x7f614c0cdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:22.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.959+0000 7f6151800640 1 -- 172.21.15.32:0/884641848 shutdown_connections 2024-04-17T09:53:22.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:22.959+0000 7f6151800640 1 -- 172.21.15.32:0/884641848 wait complete. 2024-04-17T09:53:22.982 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-17T09:53:21.286797+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239844,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734876,"statfs":{"total":773094113280,"available":772848513024,"internally_reserved":0,"allocated":2924544,"data_stored":1763052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001594"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067861+0000","last_change":"2024-04-17T09:53:13.801310+0000","last_active":"2024-04-17T09:53:15.067861+0000","last_peered":"2024-04-17T09:53:15.067861+0000","last_clean":"2024-04-17T09:53:15.067861+0000","last_became_active":"2024-04-17T09:53:13.800622+0000","last_became_peered":"2024-04-17T09:53:13.800622+0000","last_unstale":"2024-04-17T09:53:15.067861+0000","last_undegraded":"2024-04-17T09:53:15.067861+0000","last_fullsized":"2024-04-17T09:53:15.067861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:10:45.479198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787738+0000","last_change":"2024-04-17T09:53:13.791349+0000","last_active":"2024-04-17T09:53:14.787738+0000","last_peered":"2024-04-17T09:53:14.787738+0000","last_clean":"2024-04-17T09:53:14.787738+0000","last_became_active":"2024-04-17T09:53:13.790726+0000","last_became_peered":"2024-04-17T09:53:13.790726+0000","last_unstale":"2024-04-17T09:53:14.787738+0000","last_undegraded":"2024-04-17T09:53:14.787738+0000","last_fullsized":"2024-04-17T09:53:14.787738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:51:42.871563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067454+0000","last_change":"2024-04-17T09:53:13.784861+0000","last_active":"2024-04-17T09:53:15.067454+0000","last_peered":"2024-04-17T09:53:15.067454+0000","last_clean":"2024-04-17T09:53:15.067454+0000","last_became_active":"2024-04-17T09:53:13.784550+0000","last_became_peered":"2024-04-17T09:53:13.784550+0000","last_unstale":"2024-04-17T09:53:15.067454+0000","last_undegraded":"2024-04-17T09:53:15.067454+0000","last_fullsized":"2024-04-17T09:53:15.067454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:03.582853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066735+0000","last_change":"2024-04-17T09:53:13.793610+0000","last_active":"2024-04-17T09:53:15.066735+0000","last_peered":"2024-04-17T09:53:15.066735+0000","last_clean":"2024-04-17T09:53:15.066735+0000","last_became_active":"2024-04-17T09:53:13.793176+0000","last_became_peered":"2024-04-17T09:53:13.793176+0000","last_unstale":"2024-04-17T09:53:15.066735+0000","last_undegraded":"2024-04-17T09:53:15.066735+0000","last_fullsized":"2024-04-17T09:53:15.066735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:16:29.674656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067415+0000","last_change":"2024-04-17T09:53:13.799934+0000","last_active":"2024-04-17T09:53:15.067415+0000","last_peered":"2024-04-17T09:53:15.067415+0000","last_clean":"2024-04-17T09:53:15.067415+0000","last_became_active":"2024-04-17T09:53:13.799557+0000","last_became_peered":"2024-04-17T09:53:13.799557+0000","last_unstale":"2024-04-17T09:53:15.067415+0000","last_undegraded":"2024-04-17T09:53:15.067415+0000","last_fullsized":"2024-04-17T09:53:15.067415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:43:23.411109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066691+0000","last_change":"2024-04-17T09:53:13.806419+0000","last_active":"2024-04-17T09:53:15.066691+0000","last_peered":"2024-04-17T09:53:15.066691+0000","last_clean":"2024-04-17T09:53:15.066691+0000","last_became_active":"2024-04-17T09:53:13.805216+0000","last_became_peered":"2024-04-17T09:53:13.805216+0000","last_unstale":"2024-04-17T09:53:15.066691+0000","last_undegraded":"2024-04-17T09:53:15.066691+0000","last_fullsized":"2024-04-17T09:53:15.066691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:58:50.680456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066488+0000","last_change":"2024-04-17T09:53:13.799827+0000","last_active":"2024-04-17T09:53:15.066488+0000","last_peered":"2024-04-17T09:53:15.066488+0000","last_clean":"2024-04-17T09:53:15.066488+0000","last_became_active":"2024-04-17T09:53:13.799483+0000","last_became_peered":"2024-04-17T09:53:13.799483+0000","last_unstale":"2024-04-17T09:53:15.066488+0000","last_undegraded":"2024-04-17T09:53:15.066488+0000","last_fullsized":"2024-04-17T09:53:15.066488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:16:29.674656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066098+0000","last_change":"2024-04-17T09:53:13.805644+0000","last_active":"2024-04-17T09:53:15.066098+0000","last_peered":"2024-04-17T09:53:15.066098+0000","last_clean":"2024-04-17T09:53:15.066098+0000","last_became_active":"2024-04-17T09:53:13.805343+0000","last_became_peered":"2024-04-17T09:53:13.805343+0000","last_unstale":"2024-04-17T09:53:15.066098+0000","last_undegraded":"2024-04-17T09:53:15.066098+0000","last_fullsized":"2024-04-17T09:53:15.066098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:43:18.222168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067921+0000","last_change":"2024-04-17T09:53:13.815771+0000","last_active":"2024-04-17T09:53:15.067921+0000","last_peered":"2024-04-17T09:53:15.067921+0000","last_clean":"2024-04-17T09:53:15.067921+0000","last_became_active":"2024-04-17T09:53:13.815042+0000","last_became_peered":"2024-04-17T09:53:13.815042+0000","last_unstale":"2024-04-17T09:53:15.067921+0000","last_undegraded":"2024-04-17T09:53:15.067921+0000","last_fullsized":"2024-04-17T09:53:15.067921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:26:59.317045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.789174+0000","last_change":"2024-04-17T09:53:13.810712+0000","last_active":"2024-04-17T09:53:14.789174+0000","last_peered":"2024-04-17T09:53:14.789174+0000","last_clean":"2024-04-17T09:53:14.789174+0000","last_became_active":"2024-04-17T09:53:13.810369+0000","last_became_peered":"2024-04-17T09:53:13.810369+0000","last_unstale":"2024-04-17T09:53:14.789174+0000","last_undegraded":"2024-04-17T09:53:14.789174+0000","last_fullsized":"2024-04-17T09:53:14.789174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:14.645740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788592+0000","last_change":"2024-04-17T09:53:13.800154+0000","last_active":"2024-04-17T09:53:14.788592+0000","last_peered":"2024-04-17T09:53:14.788592+0000","last_clean":"2024-04-17T09:53:14.788592+0000","last_became_active":"2024-04-17T09:53:13.799871+0000","last_became_peered":"2024-04-17T09:53:13.799871+0000","last_unstale":"2024-04-17T09:53:14.788592+0000","last_undegraded":"2024-04-17T09:53:14.788592+0000","last_fullsized":"2024-04-17T09:53:14.788592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:02.320431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788631+0000","last_change":"2024-04-17T09:53:13.805171+0000","last_active":"2024-04-17T09:53:14.788631+0000","last_peered":"2024-04-17T09:53:14.788631+0000","last_clean":"2024-04-17T09:53:14.788631+0000","last_became_active":"2024-04-17T09:53:13.804823+0000","last_became_peered":"2024-04-17T09:53:13.804823+0000","last_unstale":"2024-04-17T09:53:14.788631+0000","last_undegraded":"2024-04-17T09:53:14.788631+0000","last_fullsized":"2024-04-17T09:53:14.788631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:26.904134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066684+0000","last_change":"2024-04-17T09:53:13.789478+0000","last_active":"2024-04-17T09:53:15.066684+0000","last_peered":"2024-04-17T09:53:15.066684+0000","last_clean":"2024-04-17T09:53:15.066684+0000","last_became_active":"2024-04-17T09:53:13.789168+0000","last_became_peered":"2024-04-17T09:53:13.789168+0000","last_unstale":"2024-04-17T09:53:15.066684+0000","last_undegraded":"2024-04-17T09:53:15.066684+0000","last_fullsized":"2024-04-17T09:53:15.066684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:14.645740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066671+0000","last_change":"2024-04-17T09:53:13.794592+0000","last_active":"2024-04-17T09:53:15.066671+0000","last_peered":"2024-04-17T09:53:15.066671+0000","last_clean":"2024-04-17T09:53:15.066671+0000","last_became_active":"2024-04-17T09:53:13.794323+0000","last_became_peered":"2024-04-17T09:53:13.794323+0000","last_unstale":"2024-04-17T09:53:15.066671+0000","last_undegraded":"2024-04-17T09:53:15.066671+0000","last_fullsized":"2024-04-17T09:53:15.066671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:40:23.720233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787454+0000","last_change":"2024-04-17T09:53:13.792566+0000","last_active":"2024-04-17T09:53:14.787454+0000","last_peered":"2024-04-17T09:53:14.787454+0000","last_clean":"2024-04-17T09:53:14.787454+0000","last_became_active":"2024-04-17T09:53:13.791848+0000","last_became_peered":"2024-04-17T09:53:13.791848+0000","last_unstale":"2024-04-17T09:53:14.787454+0000","last_undegraded":"2024-04-17T09:53:14.787454+0000","last_fullsized":"2024-04-17T09:53:14.787454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:17:54.590392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066342+0000","last_change":"2024-04-17T09:53:13.815067+0000","last_active":"2024-04-17T09:53:15.066342+0000","last_peered":"2024-04-17T09:53:15.066342+0000","last_clean":"2024-04-17T09:53:15.066342+0000","last_became_active":"2024-04-17T09:53:13.814781+0000","last_became_peered":"2024-04-17T09:53:13.814781+0000","last_unstale":"2024-04-17T09:53:15.066342+0000","last_undegraded":"2024-04-17T09:53:15.066342+0000","last_fullsized":"2024-04-17T09:53:15.066342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:14.645740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067458+0000","last_change":"2024-04-17T09:53:13.814757+0000","last_active":"2024-04-17T09:53:15.067458+0000","last_peered":"2024-04-17T09:53:15.067458+0000","last_clean":"2024-04-17T09:53:15.067458+0000","last_became_active":"2024-04-17T09:53:13.814433+0000","last_became_peered":"2024-04-17T09:53:13.814433+0000","last_unstale":"2024-04-17T09:53:15.067458+0000","last_undegraded":"2024-04-17T09:53:15.067458+0000","last_fullsized":"2024-04-17T09:53:15.067458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:34:37.237156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067453+0000","last_change":"2024-04-17T09:53:13.799374+0000","last_active":"2024-04-17T09:53:15.067453+0000","last_peered":"2024-04-17T09:53:15.067453+0000","last_clean":"2024-04-17T09:53:15.067453+0000","last_became_active":"2024-04-17T09:53:13.799117+0000","last_became_peered":"2024-04-17T09:53:13.799117+0000","last_unstale":"2024-04-17T09:53:15.067453+0000","last_undegraded":"2024-04-17T09:53:15.067453+0000","last_fullsized":"2024-04-17T09:53:15.067453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:54.146581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067542+0000","last_change":"2024-04-17T09:53:13.798276+0000","last_active":"2024-04-17T09:53:15.067542+0000","last_peered":"2024-04-17T09:53:15.067542+0000","last_clean":"2024-04-17T09:53:15.067542+0000","last_became_active":"2024-04-17T09:53:13.796681+0000","last_became_peered":"2024-04-17T09:53:13.796681+0000","last_unstale":"2024-04-17T09:53:15.067542+0000","last_undegraded":"2024-04-17T09:53:15.067542+0000","last_fullsized":"2024-04-17T09:53:15.067542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:27:15.142091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788909+0000","last_change":"2024-04-17T09:53:13.804880+0000","last_active":"2024-04-17T09:53:14.788909+0000","last_peered":"2024-04-17T09:53:14.788909+0000","last_clean":"2024-04-17T09:53:14.788909+0000","last_became_active":"2024-04-17T09:53:13.804615+0000","last_became_peered":"2024-04-17T09:53:13.804615+0000","last_unstale":"2024-04-17T09:53:14.788909+0000","last_undegraded":"2024-04-17T09:53:14.788909+0000","last_fullsized":"2024-04-17T09:53:14.788909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:17.663523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066926+0000","last_change":"2024-04-17T09:53:13.800703+0000","last_active":"2024-04-17T09:53:15.066926+0000","last_peered":"2024-04-17T09:53:15.066926+0000","last_clean":"2024-04-17T09:53:15.066926+0000","last_became_active":"2024-04-17T09:53:13.800353+0000","last_became_peered":"2024-04-17T09:53:13.800353+0000","last_unstale":"2024-04-17T09:53:15.066926+0000","last_undegraded":"2024-04-17T09:53:15.066926+0000","last_fullsized":"2024-04-17T09:53:15.066926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:55:22.260565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066652+0000","last_change":"2024-04-17T09:53:13.809406+0000","last_active":"2024-04-17T09:53:15.066652+0000","last_peered":"2024-04-17T09:53:15.066652+0000","last_clean":"2024-04-17T09:53:15.066652+0000","last_became_active":"2024-04-17T09:53:13.809152+0000","last_became_peered":"2024-04-17T09:53:13.809152+0000","last_unstale":"2024-04-17T09:53:15.066652+0000","last_undegraded":"2024-04-17T09:53:15.066652+0000","last_fullsized":"2024-04-17T09:53:15.066652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:55:01.176803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066430+0000","last_change":"2024-04-17T09:53:13.785293+0000","last_active":"2024-04-17T09:53:15.066430+0000","last_peered":"2024-04-17T09:53:15.066430+0000","last_clean":"2024-04-17T09:53:15.066430+0000","last_became_active":"2024-04-17T09:53:13.785004+0000","last_became_peered":"2024-04-17T09:53:13.785004+0000","last_unstale":"2024-04-17T09:53:15.066430+0000","last_undegraded":"2024-04-17T09:53:15.066430+0000","last_fullsized":"2024-04-17T09:53:15.066430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:17.663523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066443+0000","last_change":"2024-04-17T09:53:13.803100+0000","last_active":"2024-04-17T09:53:15.066443+0000","last_peered":"2024-04-17T09:53:15.066443+0000","last_clean":"2024-04-17T09:53:15.066443+0000","last_became_active":"2024-04-17T09:53:13.802743+0000","last_became_peered":"2024-04-17T09:53:13.802743+0000","last_unstale":"2024-04-17T09:53:15.066443+0000","last_undegraded":"2024-04-17T09:53:15.066443+0000","last_fullsized":"2024-04-17T09:53:15.066443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:51:42.871563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067978+0000","last_change":"2024-04-17T09:53:13.789615+0000","last_active":"2024-04-17T09:53:15.067978+0000","last_peered":"2024-04-17T09:53:15.067978+0000","last_clean":"2024-04-17T09:53:15.067978+0000","last_became_active":"2024-04-17T09:53:13.787774+0000","last_became_peered":"2024-04-17T09:53:13.787774+0000","last_unstale":"2024-04-17T09:53:15.067978+0000","last_undegraded":"2024-04-17T09:53:15.067978+0000","last_fullsized":"2024-04-17T09:53:15.067978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:36:37.128859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.789301+0000","last_change":"2024-04-17T09:53:13.800946+0000","last_active":"2024-04-17T09:53:14.789301+0000","last_peered":"2024-04-17T09:53:14.789301+0000","last_clean":"2024-04-17T09:53:14.789301+0000","last_became_active":"2024-04-17T09:53:13.798598+0000","last_became_peered":"2024-04-17T09:53:13.798598+0000","last_unstale":"2024-04-17T09:53:14.789301+0000","last_undegraded":"2024-04-17T09:53:14.789301+0000","last_fullsized":"2024-04-17T09:53:14.789301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:51:42.871563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787682+0000","last_change":"2024-04-17T09:53:13.792719+0000","last_active":"2024-04-17T09:53:14.787682+0000","last_peered":"2024-04-17T09:53:14.787682+0000","last_clean":"2024-04-17T09:53:14.787682+0000","last_became_active":"2024-04-17T09:53:13.792390+0000","last_became_peered":"2024-04-17T09:53:13.792390+0000","last_unstale":"2024-04-17T09:53:14.787682+0000","last_undegraded":"2024-04-17T09:53:14.787682+0000","last_fullsized":"2024-04-17T09:53:14.787682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:14.645740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067164+0000","last_change":"2024-04-17T09:53:13.793602+0000","last_active":"2024-04-17T09:53:15.067164+0000","last_peered":"2024-04-17T09:53:15.067164+0000","last_clean":"2024-04-17T09:53:15.067164+0000","last_became_active":"2024-04-17T09:53:13.793277+0000","last_became_peered":"2024-04-17T09:53:13.793277+0000","last_unstale":"2024-04-17T09:53:15.067164+0000","last_undegraded":"2024-04-17T09:53:15.067164+0000","last_fullsized":"2024-04-17T09:53:15.067164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:58:50.680456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066987+0000","last_change":"2024-04-17T09:53:13.781047+0000","last_active":"2024-04-17T09:53:15.066987+0000","last_peered":"2024-04-17T09:53:15.066987+0000","last_clean":"2024-04-17T09:53:15.066987+0000","last_became_active":"2024-04-17T09:53:13.780726+0000","last_became_peered":"2024-04-17T09:53:13.780726+0000","last_unstale":"2024-04-17T09:53:15.066987+0000","last_undegraded":"2024-04-17T09:53:15.066987+0000","last_fullsized":"2024-04-17T09:53:15.066987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:39:52.213103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066900+0000","last_change":"2024-04-17T09:53:13.806423+0000","last_active":"2024-04-17T09:53:15.066900+0000","last_peered":"2024-04-17T09:53:15.066900+0000","last_clean":"2024-04-17T09:53:15.066900+0000","last_became_active":"2024-04-17T09:53:13.804648+0000","last_became_peered":"2024-04-17T09:53:13.804648+0000","last_unstale":"2024-04-17T09:53:15.066900+0000","last_undegraded":"2024-04-17T09:53:15.066900+0000","last_fullsized":"2024-04-17T09:53:15.066900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:30:17.695383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066152+0000","last_change":"2024-04-17T09:53:13.803405+0000","last_active":"2024-04-17T09:53:15.066152+0000","last_peered":"2024-04-17T09:53:15.066152+0000","last_clean":"2024-04-17T09:53:15.066152+0000","last_became_active":"2024-04-17T09:53:13.803135+0000","last_became_peered":"2024-04-17T09:53:13.803135+0000","last_unstale":"2024-04-17T09:53:15.066152+0000","last_undegraded":"2024-04-17T09:53:15.066152+0000","last_fullsized":"2024-04-17T09:53:15.066152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:12.936111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787899+0000","last_change":"2024-04-17T09:53:13.786719+0000","last_active":"2024-04-17T09:53:14.787899+0000","last_peered":"2024-04-17T09:53:14.787899+0000","last_clean":"2024-04-17T09:53:14.787899+0000","last_became_active":"2024-04-17T09:53:13.786401+0000","last_became_peered":"2024-04-17T09:53:13.786401+0000","last_unstale":"2024-04-17T09:53:14.787899+0000","last_undegraded":"2024-04-17T09:53:14.787899+0000","last_fullsized":"2024-04-17T09:53:14.787899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:55:00.842846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787911+0000","last_change":"2024-04-17T09:53:13.797469+0000","last_active":"2024-04-17T09:53:14.787911+0000","last_peered":"2024-04-17T09:53:14.787911+0000","last_clean":"2024-04-17T09:53:14.787911+0000","last_became_active":"2024-04-17T09:53:13.797180+0000","last_became_peered":"2024-04-17T09:53:13.797180+0000","last_unstale":"2024-04-17T09:53:14.787911+0000","last_undegraded":"2024-04-17T09:53:14.787911+0000","last_fullsized":"2024-04-17T09:53:14.787911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:51:44.795158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066953+0000","last_change":"2024-04-17T09:53:12.783002+0000","last_active":"2024-04-17T09:53:15.066953+0000","last_peered":"2024-04-17T09:53:15.066953+0000","last_clean":"2024-04-17T09:53:15.066953+0000","last_became_active":"2024-04-17T09:53:12.782606+0000","last_became_peered":"2024-04-17T09:53:12.782606+0000","last_unstale":"2024-04-17T09:53:15.066953+0000","last_undegraded":"2024-04-17T09:53:15.066953+0000","last_fullsized":"2024-04-17T09:53:15.066953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:52:55.917109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067085+0000","last_change":"2024-04-17T09:53:13.784184+0000","last_active":"2024-04-17T09:53:15.067085+0000","last_peered":"2024-04-17T09:53:15.067085+0000","last_clean":"2024-04-17T09:53:15.067085+0000","last_became_active":"2024-04-17T09:53:13.783922+0000","last_became_peered":"2024-04-17T09:53:13.783922+0000","last_unstale":"2024-04-17T09:53:15.067085+0000","last_undegraded":"2024-04-17T09:53:15.067085+0000","last_fullsized":"2024-04-17T09:53:15.067085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:57:05.882921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788344+0000","last_change":"2024-04-17T09:53:12.768842+0000","last_active":"2024-04-17T09:53:14.788344+0000","last_peered":"2024-04-17T09:53:14.788344+0000","last_clean":"2024-04-17T09:53:14.788344+0000","last_became_active":"2024-04-17T09:53:12.768227+0000","last_became_peered":"2024-04-17T09:53:12.768227+0000","last_unstale":"2024-04-17T09:53:14.788344+0000","last_undegraded":"2024-04-17T09:53:14.788344+0000","last_fullsized":"2024-04-17T09:53:14.788344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:32:56.010181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066590+0000","last_change":"2024-04-17T09:53:12.773398+0000","last_active":"2024-04-17T09:53:15.066590+0000","last_peered":"2024-04-17T09:53:15.066590+0000","last_clean":"2024-04-17T09:53:15.066590+0000","last_became_active":"2024-04-17T09:53:12.771487+0000","last_became_peered":"2024-04-17T09:53:12.771487+0000","last_unstale":"2024-04-17T09:53:15.066590+0000","last_undegraded":"2024-04-17T09:53:15.066590+0000","last_fullsized":"2024-04-17T09:53:15.066590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:24.555970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066239+0000","last_change":"2024-04-17T09:53:13.782988+0000","last_active":"2024-04-17T09:53:15.066239+0000","last_peered":"2024-04-17T09:53:15.066239+0000","last_clean":"2024-04-17T09:53:15.066239+0000","last_became_active":"2024-04-17T09:53:13.782711+0000","last_became_peered":"2024-04-17T09:53:13.782711+0000","last_unstale":"2024-04-17T09:53:15.066239+0000","last_undegraded":"2024-04-17T09:53:15.066239+0000","last_fullsized":"2024-04-17T09:53:15.066239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:02.320431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066522+0000","last_change":"2024-04-17T09:53:12.778661+0000","last_active":"2024-04-17T09:53:15.066522+0000","last_peered":"2024-04-17T09:53:15.066522+0000","last_clean":"2024-04-17T09:53:15.066522+0000","last_became_active":"2024-04-17T09:53:12.778088+0000","last_became_peered":"2024-04-17T09:53:12.778088+0000","last_unstale":"2024-04-17T09:53:15.066522+0000","last_undegraded":"2024-04-17T09:53:15.066522+0000","last_fullsized":"2024-04-17T09:53:15.066522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:50.549367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787383+0000","last_change":"2024-04-17T09:53:13.792711+0000","last_active":"2024-04-17T09:53:14.787383+0000","last_peered":"2024-04-17T09:53:14.787383+0000","last_clean":"2024-04-17T09:53:14.787383+0000","last_became_active":"2024-04-17T09:53:13.792257+0000","last_became_peered":"2024-04-17T09:53:13.792257+0000","last_unstale":"2024-04-17T09:53:14.787383+0000","last_undegraded":"2024-04-17T09:53:14.787383+0000","last_fullsized":"2024-04-17T09:53:14.787383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:43:18.222168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.789124+0000","last_change":"2024-04-17T09:53:12.779227+0000","last_active":"2024-04-17T09:53:14.789124+0000","last_peered":"2024-04-17T09:53:14.789124+0000","last_clean":"2024-04-17T09:53:14.789124+0000","last_became_active":"2024-04-17T09:53:12.778942+0000","last_became_peered":"2024-04-17T09:53:12.778942+0000","last_unstale":"2024-04-17T09:53:14.789124+0000","last_undegraded":"2024-04-17T09:53:14.789124+0000","last_fullsized":"2024-04-17T09:53:14.789124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:07.296217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066798+0000","last_change":"2024-04-17T09:53:13.812353+0000","last_active":"2024-04-17T09:53:15.066798+0000","last_peered":"2024-04-17T09:53:15.066798+0000","last_clean":"2024-04-17T09:53:15.066798+0000","last_became_active":"2024-04-17T09:53:13.812007+0000","last_became_peered":"2024-04-17T09:53:13.812007+0000","last_unstale":"2024-04-17T09:53:15.066798+0000","last_undegraded":"2024-04-17T09:53:15.066798+0000","last_fullsized":"2024-04-17T09:53:15.066798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:36:37.128859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066937+0000","last_change":"2024-04-17T09:53:12.776576+0000","last_active":"2024-04-17T09:53:15.066937+0000","last_peered":"2024-04-17T09:53:15.066937+0000","last_clean":"2024-04-17T09:53:15.066937+0000","last_became_active":"2024-04-17T09:53:12.772424+0000","last_became_peered":"2024-04-17T09:53:12.772424+0000","last_unstale":"2024-04-17T09:53:15.066937+0000","last_undegraded":"2024-04-17T09:53:15.066937+0000","last_fullsized":"2024-04-17T09:53:15.066937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:53.140146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067737+0000","last_change":"2024-04-17T09:53:13.815907+0000","last_active":"2024-04-17T09:53:15.067737+0000","last_peered":"2024-04-17T09:53:15.067737+0000","last_clean":"2024-04-17T09:53:15.067737+0000","last_became_active":"2024-04-17T09:53:13.815616+0000","last_became_peered":"2024-04-17T09:53:13.815616+0000","last_unstale":"2024-04-17T09:53:15.067737+0000","last_undegraded":"2024-04-17T09:53:15.067737+0000","last_fullsized":"2024-04-17T09:53:15.067737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:33:33.232581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787638+0000","last_change":"2024-04-17T09:53:12.791926+0000","last_active":"2024-04-17T09:53:14.787638+0000","last_peered":"2024-04-17T09:53:14.787638+0000","last_clean":"2024-04-17T09:53:14.787638+0000","last_became_active":"2024-04-17T09:53:12.791644+0000","last_became_peered":"2024-04-17T09:53:12.791644+0000","last_unstale":"2024-04-17T09:53:14.787638+0000","last_undegraded":"2024-04-17T09:53:14.787638+0000","last_fullsized":"2024-04-17T09:53:14.787638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:14.881628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788745+0000","last_change":"2024-04-17T09:53:13.808620+0000","last_active":"2024-04-17T09:53:14.788745+0000","last_peered":"2024-04-17T09:53:14.788745+0000","last_clean":"2024-04-17T09:53:14.788745+0000","last_became_active":"2024-04-17T09:53:13.807344+0000","last_became_peered":"2024-04-17T09:53:13.807344+0000","last_unstale":"2024-04-17T09:53:14.788745+0000","last_undegraded":"2024-04-17T09:53:14.788745+0000","last_fullsized":"2024-04-17T09:53:14.788745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:20:06.508805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.789133+0000","last_change":"2024-04-17T09:53:12.771523+0000","last_active":"2024-04-17T09:53:14.789133+0000","last_peered":"2024-04-17T09:53:14.789133+0000","last_clean":"2024-04-17T09:53:14.789133+0000","last_became_active":"2024-04-17T09:53:12.771201+0000","last_became_peered":"2024-04-17T09:53:12.771201+0000","last_unstale":"2024-04-17T09:53:14.789133+0000","last_undegraded":"2024-04-17T09:53:14.789133+0000","last_fullsized":"2024-04-17T09:53:14.789133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:50.549367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067157+0000","last_change":"2024-04-17T09:53:13.797071+0000","last_active":"2024-04-17T09:53:15.067157+0000","last_peered":"2024-04-17T09:53:15.067157+0000","last_clean":"2024-04-17T09:53:15.067157+0000","last_became_active":"2024-04-17T09:53:13.796572+0000","last_became_peered":"2024-04-17T09:53:13.796572+0000","last_unstale":"2024-04-17T09:53:15.067157+0000","last_undegraded":"2024-04-17T09:53:15.067157+0000","last_fullsized":"2024-04-17T09:53:15.067157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:03:55.116831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067069+0000","last_change":"2024-04-17T09:53:12.778367+0000","last_active":"2024-04-17T09:53:15.067069+0000","last_peered":"2024-04-17T09:53:15.067069+0000","last_clean":"2024-04-17T09:53:15.067069+0000","last_became_active":"2024-04-17T09:53:12.778084+0000","last_became_peered":"2024-04-17T09:53:12.778084+0000","last_unstale":"2024-04-17T09:53:15.067069+0000","last_undegraded":"2024-04-17T09:53:15.067069+0000","last_fullsized":"2024-04-17T09:53:15.067069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:19:28.217974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066747+0000","last_change":"2024-04-17T09:53:13.798230+0000","last_active":"2024-04-17T09:53:15.066747+0000","last_peered":"2024-04-17T09:53:15.066747+0000","last_clean":"2024-04-17T09:53:15.066747+0000","last_became_active":"2024-04-17T09:53:13.797873+0000","last_became_peered":"2024-04-17T09:53:13.797873+0000","last_unstale":"2024-04-17T09:53:15.066747+0000","last_undegraded":"2024-04-17T09:53:15.066747+0000","last_fullsized":"2024-04-17T09:53:15.066747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:43:55.704203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067037+0000","last_change":"2024-04-17T09:53:12.784831+0000","last_active":"2024-04-17T09:53:15.067037+0000","last_peered":"2024-04-17T09:53:15.067037+0000","last_clean":"2024-04-17T09:53:15.067037+0000","last_became_active":"2024-04-17T09:53:12.784012+0000","last_became_peered":"2024-04-17T09:53:12.784012+0000","last_unstale":"2024-04-17T09:53:15.067037+0000","last_undegraded":"2024-04-17T09:53:15.067037+0000","last_fullsized":"2024-04-17T09:53:15.067037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:09.944062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067415+0000","last_change":"2024-04-17T09:53:13.809261+0000","last_active":"2024-04-17T09:53:15.067415+0000","last_peered":"2024-04-17T09:53:15.067415+0000","last_clean":"2024-04-17T09:53:15.067415+0000","last_became_active":"2024-04-17T09:53:13.809004+0000","last_became_peered":"2024-04-17T09:53:13.809004+0000","last_unstale":"2024-04-17T09:53:15.067415+0000","last_undegraded":"2024-04-17T09:53:15.067415+0000","last_fullsized":"2024-04-17T09:53:15.067415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:33:32.665364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788374+0000","last_change":"2024-04-17T09:53:12.768643+0000","last_active":"2024-04-17T09:53:14.788374+0000","last_peered":"2024-04-17T09:53:14.788374+0000","last_clean":"2024-04-17T09:53:14.788374+0000","last_became_active":"2024-04-17T09:53:12.768221+0000","last_became_peered":"2024-04-17T09:53:12.768221+0000","last_unstale":"2024-04-17T09:53:14.788374+0000","last_undegraded":"2024-04-17T09:53:14.788374+0000","last_fullsized":"2024-04-17T09:53:14.788374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:14:18.407034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788658+0000","last_change":"2024-04-17T09:53:13.788193+0000","last_active":"2024-04-17T09:53:14.788658+0000","last_peered":"2024-04-17T09:53:14.788658+0000","last_clean":"2024-04-17T09:53:14.788658+0000","last_became_active":"2024-04-17T09:53:13.787942+0000","last_became_peered":"2024-04-17T09:53:13.787942+0000","last_unstale":"2024-04-17T09:53:14.788658+0000","last_undegraded":"2024-04-17T09:53:14.788658+0000","last_fullsized":"2024-04-17T09:53:14.788658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:17:54.590392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066252+0000","last_change":"2024-04-17T09:53:12.767452+0000","last_active":"2024-04-17T09:53:15.066252+0000","last_peered":"2024-04-17T09:53:15.066252+0000","last_clean":"2024-04-17T09:53:15.066252+0000","last_became_active":"2024-04-17T09:53:12.766628+0000","last_became_peered":"2024-04-17T09:53:12.766628+0000","last_unstale":"2024-04-17T09:53:15.066252+0000","last_undegraded":"2024-04-17T09:53:15.066252+0000","last_fullsized":"2024-04-17T09:53:15.066252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:07.296217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067067+0000","last_change":"2024-04-17T09:53:13.803929+0000","last_active":"2024-04-17T09:53:15.067067+0000","last_peered":"2024-04-17T09:53:15.067067+0000","last_clean":"2024-04-17T09:53:15.067067+0000","last_became_active":"2024-04-17T09:53:13.803651+0000","last_became_peered":"2024-04-17T09:53:13.803651+0000","last_unstale":"2024-04-17T09:53:15.067067+0000","last_undegraded":"2024-04-17T09:53:15.067067+0000","last_fullsized":"2024-04-17T09:53:15.067067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:36:37.128859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.789205+0000","last_change":"2024-04-17T09:53:12.781091+0000","last_active":"2024-04-17T09:53:14.789205+0000","last_peered":"2024-04-17T09:53:14.789205+0000","last_clean":"2024-04-17T09:53:14.789205+0000","last_became_active":"2024-04-17T09:53:12.780752+0000","last_became_peered":"2024-04-17T09:53:12.780752+0000","last_unstale":"2024-04-17T09:53:14.789205+0000","last_undegraded":"2024-04-17T09:53:14.789205+0000","last_fullsized":"2024-04-17T09:53:14.789205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:31:08.049836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067755+0000","last_change":"2024-04-17T09:53:13.810509+0000","last_active":"2024-04-17T09:53:15.067755+0000","last_peered":"2024-04-17T09:53:15.067755+0000","last_clean":"2024-04-17T09:53:15.067755+0000","last_became_active":"2024-04-17T09:53:13.809888+0000","last_became_peered":"2024-04-17T09:53:13.809888+0000","last_unstale":"2024-04-17T09:53:15.067755+0000","last_undegraded":"2024-04-17T09:53:15.067755+0000","last_fullsized":"2024-04-17T09:53:15.067755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:34:37.237156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067459+0000","last_change":"2024-04-17T09:53:12.778623+0000","last_active":"2024-04-17T09:53:15.067459+0000","last_peered":"2024-04-17T09:53:15.067459+0000","last_clean":"2024-04-17T09:53:15.067459+0000","last_became_active":"2024-04-17T09:53:12.774044+0000","last_became_peered":"2024-04-17T09:53:12.774044+0000","last_unstale":"2024-04-17T09:53:15.067459+0000","last_undegraded":"2024-04-17T09:53:15.067459+0000","last_fullsized":"2024-04-17T09:53:15.067459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:09.944062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.068004+0000","last_change":"2024-04-17T09:53:13.809464+0000","last_active":"2024-04-17T09:53:15.068004+0000","last_peered":"2024-04-17T09:53:15.068004+0000","last_clean":"2024-04-17T09:53:15.068004+0000","last_became_active":"2024-04-17T09:53:13.808615+0000","last_became_peered":"2024-04-17T09:53:13.808615+0000","last_unstale":"2024-04-17T09:53:15.068004+0000","last_undegraded":"2024-04-17T09:53:15.068004+0000","last_fullsized":"2024-04-17T09:53:15.068004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:52:32.791236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787694+0000","last_change":"2024-04-17T09:53:12.776880+0000","last_active":"2024-04-17T09:53:14.787694+0000","last_peered":"2024-04-17T09:53:14.787694+0000","last_clean":"2024-04-17T09:53:14.787694+0000","last_became_active":"2024-04-17T09:53:12.776002+0000","last_became_peered":"2024-04-17T09:53:12.776002+0000","last_unstale":"2024-04-17T09:53:14.787694+0000","last_undegraded":"2024-04-17T09:53:14.787694+0000","last_fullsized":"2024-04-17T09:53:14.787694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:09.944062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066950+0000","last_change":"2024-04-17T09:53:13.793405+0000","last_active":"2024-04-17T09:53:15.066950+0000","last_peered":"2024-04-17T09:53:15.066950+0000","last_clean":"2024-04-17T09:53:15.066950+0000","last_became_active":"2024-04-17T09:53:13.792553+0000","last_became_peered":"2024-04-17T09:53:13.792553+0000","last_unstale":"2024-04-17T09:53:15.066950+0000","last_undegraded":"2024-04-17T09:53:15.066950+0000","last_fullsized":"2024-04-17T09:53:15.066950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:43:23.411109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066128+0000","last_change":"2024-04-17T09:53:12.780453+0000","last_active":"2024-04-17T09:53:15.066128+0000","last_peered":"2024-04-17T09:53:15.066128+0000","last_clean":"2024-04-17T09:53:15.066128+0000","last_became_active":"2024-04-17T09:53:12.780130+0000","last_became_peered":"2024-04-17T09:53:12.780130+0000","last_unstale":"2024-04-17T09:53:15.066128+0000","last_undegraded":"2024-04-17T09:53:15.066128+0000","last_fullsized":"2024-04-17T09:53:15.066128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:14:18.407034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067313+0000","last_change":"2024-04-17T09:53:13.802442+0000","last_active":"2024-04-17T09:53:15.067313+0000","last_peered":"2024-04-17T09:53:15.067313+0000","last_clean":"2024-04-17T09:53:15.067313+0000","last_became_active":"2024-04-17T09:53:13.802168+0000","last_became_peered":"2024-04-17T09:53:13.802168+0000","last_unstale":"2024-04-17T09:53:15.067313+0000","last_undegraded":"2024-04-17T09:53:15.067313+0000","last_fullsized":"2024-04-17T09:53:15.067313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:30:17.695383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066329+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T09:53:15.066329+0000","last_peered":"2024-04-17T09:53:15.066329+0000","last_clean":"2024-04-17T09:53:15.066329+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T09:53:15.066329+0000","last_undegraded":"2024-04-17T09:53:15.066329+0000","last_fullsized":"2024-04-17T09:53:15.066329+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066649+0000","last_change":"2024-04-17T09:53:12.772905+0000","last_active":"2024-04-17T09:53:15.066649+0000","last_peered":"2024-04-17T09:53:15.066649+0000","last_clean":"2024-04-17T09:53:15.066649+0000","last_became_active":"2024-04-17T09:53:12.772635+0000","last_became_peered":"2024-04-17T09:53:12.772635+0000","last_unstale":"2024-04-17T09:53:15.066649+0000","last_undegraded":"2024-04-17T09:53:15.066649+0000","last_fullsized":"2024-04-17T09:53:15.066649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:11.929676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066741+0000","last_change":"2024-04-17T09:53:13.815993+0000","last_active":"2024-04-17T09:53:15.066741+0000","last_peered":"2024-04-17T09:53:15.066741+0000","last_clean":"2024-04-17T09:53:15.066741+0000","last_became_active":"2024-04-17T09:53:13.815689+0000","last_became_peered":"2024-04-17T09:53:13.815689+0000","last_unstale":"2024-04-17T09:53:15.066741+0000","last_undegraded":"2024-04-17T09:53:15.066741+0000","last_fullsized":"2024-04-17T09:53:15.066741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:51:42.871563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066279+0000","last_change":"2024-04-17T09:53:12.790967+0000","last_active":"2024-04-17T09:53:15.066279+0000","last_peered":"2024-04-17T09:53:15.066279+0000","last_clean":"2024-04-17T09:53:15.066279+0000","last_became_active":"2024-04-17T09:53:12.790637+0000","last_became_peered":"2024-04-17T09:53:12.790637+0000","last_unstale":"2024-04-17T09:53:15.066279+0000","last_undegraded":"2024-04-17T09:53:15.066279+0000","last_fullsized":"2024-04-17T09:53:15.066279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:14.881628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066338+0000","last_change":"2024-04-17T09:53:13.788385+0000","last_active":"2024-04-17T09:53:15.066338+0000","last_peered":"2024-04-17T09:53:15.066338+0000","last_clean":"2024-04-17T09:53:15.066338+0000","last_became_active":"2024-04-17T09:53:13.784112+0000","last_became_peered":"2024-04-17T09:53:13.784112+0000","last_unstale":"2024-04-17T09:53:15.066338+0000","last_undegraded":"2024-04-17T09:53:15.066338+0000","last_fullsized":"2024-04-17T09:53:15.066338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:30:00.992434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067778+0000","last_change":"2024-04-17T09:53:12.775859+0000","last_active":"2024-04-17T09:53:15.067778+0000","last_peered":"2024-04-17T09:53:15.067778+0000","last_clean":"2024-04-17T09:53:15.067778+0000","last_became_active":"2024-04-17T09:53:12.774051+0000","last_became_peered":"2024-04-17T09:53:12.774051+0000","last_unstale":"2024-04-17T09:53:15.067778+0000","last_undegraded":"2024-04-17T09:53:15.067778+0000","last_fullsized":"2024-04-17T09:53:15.067778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:32:56.010181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787431+0000","last_change":"2024-04-17T09:53:13.792329+0000","last_active":"2024-04-17T09:53:14.787431+0000","last_peered":"2024-04-17T09:53:14.787431+0000","last_clean":"2024-04-17T09:53:14.787431+0000","last_became_active":"2024-04-17T09:53:13.790615+0000","last_became_peered":"2024-04-17T09:53:13.790615+0000","last_unstale":"2024-04-17T09:53:14.787431+0000","last_undegraded":"2024-04-17T09:53:14.787431+0000","last_fullsized":"2024-04-17T09:53:14.787431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:02.320431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067057+0000","last_change":"2024-04-17T09:53:12.787408+0000","last_active":"2024-04-17T09:53:15.067057+0000","last_peered":"2024-04-17T09:53:15.067057+0000","last_clean":"2024-04-17T09:53:15.067057+0000","last_became_active":"2024-04-17T09:53:12.786322+0000","last_became_peered":"2024-04-17T09:53:12.786322+0000","last_unstale":"2024-04-17T09:53:15.067057+0000","last_undegraded":"2024-04-17T09:53:15.067057+0000","last_fullsized":"2024-04-17T09:53:15.067057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:14.881628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788243+0000","last_change":"2024-04-17T09:53:13.786023+0000","last_active":"2024-04-17T09:53:14.788243+0000","last_peered":"2024-04-17T09:53:14.788243+0000","last_clean":"2024-04-17T09:53:14.788243+0000","last_became_active":"2024-04-17T09:53:13.785528+0000","last_became_peered":"2024-04-17T09:53:13.785528+0000","last_unstale":"2024-04-17T09:53:14.788243+0000","last_undegraded":"2024-04-17T09:53:14.788243+0000","last_fullsized":"2024-04-17T09:53:14.788243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:09:58.265496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787601+0000","last_change":"2024-04-17T09:53:12.786776+0000","last_active":"2024-04-17T09:53:14.787601+0000","last_peered":"2024-04-17T09:53:14.787601+0000","last_clean":"2024-04-17T09:53:14.787601+0000","last_became_active":"2024-04-17T09:53:12.786511+0000","last_became_peered":"2024-04-17T09:53:12.786511+0000","last_unstale":"2024-04-17T09:53:14.787601+0000","last_undegraded":"2024-04-17T09:53:14.787601+0000","last_fullsized":"2024-04-17T09:53:14.787601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:32:56.010181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067174+0000","last_change":"2024-04-17T09:53:13.795421+0000","last_active":"2024-04-17T09:53:15.067174+0000","last_peered":"2024-04-17T09:53:15.067174+0000","last_clean":"2024-04-17T09:53:15.067174+0000","last_became_active":"2024-04-17T09:53:13.793404+0000","last_became_peered":"2024-04-17T09:53:13.793404+0000","last_unstale":"2024-04-17T09:53:15.067174+0000","last_undegraded":"2024-04-17T09:53:15.067174+0000","last_fullsized":"2024-04-17T09:53:15.067174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:27:15.142091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067608+0000","last_change":"2024-04-17T09:53:12.786549+0000","last_active":"2024-04-17T09:53:15.067608+0000","last_peered":"2024-04-17T09:53:15.067608+0000","last_clean":"2024-04-17T09:53:15.067608+0000","last_became_active":"2024-04-17T09:53:12.784745+0000","last_became_peered":"2024-04-17T09:53:12.784745+0000","last_unstale":"2024-04-17T09:53:15.067608+0000","last_undegraded":"2024-04-17T09:53:15.067608+0000","last_fullsized":"2024-04-17T09:53:15.067608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:57:23.703708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787659+0000","last_change":"2024-04-17T09:53:13.792711+0000","last_active":"2024-04-17T09:53:14.787659+0000","last_peered":"2024-04-17T09:53:14.787659+0000","last_clean":"2024-04-17T09:53:14.787659+0000","last_became_active":"2024-04-17T09:53:13.791107+0000","last_became_peered":"2024-04-17T09:53:13.791107+0000","last_unstale":"2024-04-17T09:53:14.787659+0000","last_undegraded":"2024-04-17T09:53:14.787659+0000","last_fullsized":"2024-04-17T09:53:14.787659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:17.663523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.795170+0000","last_change":"2024-04-17T09:53:12.770038+0000","last_active":"2024-04-17T09:53:14.795170+0000","last_peered":"2024-04-17T09:53:14.795170+0000","last_clean":"2024-04-17T09:53:14.795170+0000","last_became_active":"2024-04-17T09:53:12.769750+0000","last_became_peered":"2024-04-17T09:53:12.769750+0000","last_unstale":"2024-04-17T09:53:14.795170+0000","last_undegraded":"2024-04-17T09:53:14.795170+0000","last_fullsized":"2024-04-17T09:53:14.795170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:50.549367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067988+0000","last_change":"2024-04-17T09:53:13.801993+0000","last_active":"2024-04-17T09:53:15.067988+0000","last_peered":"2024-04-17T09:53:15.067988+0000","last_clean":"2024-04-17T09:53:15.067988+0000","last_became_active":"2024-04-17T09:53:13.801328+0000","last_became_peered":"2024-04-17T09:53:13.801328+0000","last_unstale":"2024-04-17T09:53:15.067988+0000","last_undegraded":"2024-04-17T09:53:15.067988+0000","last_fullsized":"2024-04-17T09:53:15.067988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:03:55.116831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.789207+0000","last_change":"2024-04-17T09:53:12.781559+0000","last_active":"2024-04-17T09:53:14.789207+0000","last_peered":"2024-04-17T09:53:14.789207+0000","last_clean":"2024-04-17T09:53:14.789207+0000","last_became_active":"2024-04-17T09:53:12.780702+0000","last_became_peered":"2024-04-17T09:53:12.780702+0000","last_unstale":"2024-04-17T09:53:14.789207+0000","last_undegraded":"2024-04-17T09:53:14.789207+0000","last_fullsized":"2024-04-17T09:53:14.789207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:09.944062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067614+0000","last_change":"2024-04-17T09:53:13.795015+0000","last_active":"2024-04-17T09:53:15.067614+0000","last_peered":"2024-04-17T09:53:15.067614+0000","last_clean":"2024-04-17T09:53:15.067614+0000","last_became_active":"2024-04-17T09:53:13.794609+0000","last_became_peered":"2024-04-17T09:53:13.794609+0000","last_unstale":"2024-04-17T09:53:15.067614+0000","last_undegraded":"2024-04-17T09:53:15.067614+0000","last_fullsized":"2024-04-17T09:53:15.067614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:35.205605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066266+0000","last_change":"2024-04-17T09:53:12.781463+0000","last_active":"2024-04-17T09:53:15.066266+0000","last_peered":"2024-04-17T09:53:15.066266+0000","last_clean":"2024-04-17T09:53:15.066266+0000","last_became_active":"2024-04-17T09:53:12.781138+0000","last_became_peered":"2024-04-17T09:53:12.781138+0000","last_unstale":"2024-04-17T09:53:15.066266+0000","last_undegraded":"2024-04-17T09:53:15.066266+0000","last_fullsized":"2024-04-17T09:53:15.066266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:09.944062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788264+0000","last_change":"2024-04-17T09:53:13.797478+0000","last_active":"2024-04-17T09:53:14.788264+0000","last_peered":"2024-04-17T09:53:14.788264+0000","last_clean":"2024-04-17T09:53:14.788264+0000","last_became_active":"2024-04-17T09:53:13.797152+0000","last_became_peered":"2024-04-17T09:53:13.797152+0000","last_unstale":"2024-04-17T09:53:14.788264+0000","last_undegraded":"2024-04-17T09:53:14.788264+0000","last_fullsized":"2024-04-17T09:53:14.788264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:59:22.224091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787486+0000","last_change":"2024-04-17T09:53:12.785848+0000","last_active":"2024-04-17T09:53:14.787486+0000","last_peered":"2024-04-17T09:53:14.787486+0000","last_clean":"2024-04-17T09:53:14.787486+0000","last_became_active":"2024-04-17T09:53:12.784980+0000","last_became_peered":"2024-04-17T09:53:12.784980+0000","last_unstale":"2024-04-17T09:53:14.787486+0000","last_undegraded":"2024-04-17T09:53:14.787486+0000","last_fullsized":"2024-04-17T09:53:14.787486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:11.929676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066406+0000","last_change":"2024-04-17T09:53:13.796607+0000","last_active":"2024-04-17T09:53:15.066406+0000","last_peered":"2024-04-17T09:53:15.066406+0000","last_clean":"2024-04-17T09:53:15.066406+0000","last_became_active":"2024-04-17T09:53:13.796351+0000","last_became_peered":"2024-04-17T09:53:13.796351+0000","last_unstale":"2024-04-17T09:53:15.066406+0000","last_undegraded":"2024-04-17T09:53:15.066406+0000","last_fullsized":"2024-04-17T09:53:15.066406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:17:54.590392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.787463+0000","last_change":"2024-04-17T09:53:12.788441+0000","last_active":"2024-04-17T09:53:14.787463+0000","last_peered":"2024-04-17T09:53:14.787463+0000","last_clean":"2024-04-17T09:53:14.787463+0000","last_became_active":"2024-04-17T09:53:12.787431+0000","last_became_peered":"2024-04-17T09:53:12.787431+0000","last_unstale":"2024-04-17T09:53:14.787463+0000","last_undegraded":"2024-04-17T09:53:14.787463+0000","last_fullsized":"2024-04-17T09:53:14.787463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:14:18.407034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066772+0000","last_change":"2024-04-17T09:53:13.783282+0000","last_active":"2024-04-17T09:53:15.066772+0000","last_peered":"2024-04-17T09:53:15.066772+0000","last_clean":"2024-04-17T09:53:15.066772+0000","last_became_active":"2024-04-17T09:53:13.782978+0000","last_became_peered":"2024-04-17T09:53:13.782978+0000","last_unstale":"2024-04-17T09:53:15.066772+0000","last_undegraded":"2024-04-17T09:53:15.066772+0000","last_fullsized":"2024-04-17T09:53:15.066772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:55:01.176803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067538+0000","last_change":"2024-04-17T09:53:12.785555+0000","last_active":"2024-04-17T09:53:15.067538+0000","last_peered":"2024-04-17T09:53:15.067538+0000","last_clean":"2024-04-17T09:53:15.067538+0000","last_became_active":"2024-04-17T09:53:12.785233+0000","last_became_peered":"2024-04-17T09:53:12.785233+0000","last_unstale":"2024-04-17T09:53:15.067538+0000","last_undegraded":"2024-04-17T09:53:15.067538+0000","last_fullsized":"2024-04-17T09:53:15.067538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:50.549367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:14.788225+0000","last_change":"2024-04-17T09:53:13.800230+0000","last_active":"2024-04-17T09:53:14.788225+0000","last_peered":"2024-04-17T09:53:14.788225+0000","last_clean":"2024-04-17T09:53:14.788225+0000","last_became_active":"2024-04-17T09:53:13.799917+0000","last_became_peered":"2024-04-17T09:53:13.799917+0000","last_unstale":"2024-04-17T09:53:14.788225+0000","last_undegraded":"2024-04-17T09:53:14.788225+0000","last_fullsized":"2024-04-17T09:53:14.788225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:43:18.222168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067005+0000","last_change":"2024-04-17T09:53:13.814147+0000","last_active":"2024-04-17T09:53:15.067005+0000","last_peered":"2024-04-17T09:53:15.067005+0000","last_clean":"2024-04-17T09:53:15.067005+0000","last_became_active":"2024-04-17T09:53:13.813807+0000","last_became_peered":"2024-04-17T09:53:13.813807+0000","last_unstale":"2024-04-17T09:53:15.067005+0000","last_undegraded":"2024-04-17T09:53:15.067005+0000","last_fullsized":"2024-04-17T09:53:15.067005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:33:32.665364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067456+0000","last_change":"2024-04-17T09:53:12.788509+0000","last_active":"2024-04-17T09:53:15.067456+0000","last_peered":"2024-04-17T09:53:15.067456+0000","last_clean":"2024-04-17T09:53:15.067456+0000","last_became_active":"2024-04-17T09:53:12.788161+0000","last_became_peered":"2024-04-17T09:53:12.788161+0000","last_unstale":"2024-04-17T09:53:15.067456+0000","last_undegraded":"2024-04-17T09:53:15.067456+0000","last_fullsized":"2024-04-17T09:53:15.067456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:11.929676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.066835+0000","last_change":"2024-04-17T09:53:13.812181+0000","last_active":"2024-04-17T09:53:15.066835+0000","last_peered":"2024-04-17T09:53:15.066835+0000","last_clean":"2024-04-17T09:53:15.066835+0000","last_became_active":"2024-04-17T09:53:13.811836+0000","last_became_peered":"2024-04-17T09:53:13.811836+0000","last_unstale":"2024-04-17T09:53:15.066835+0000","last_undegraded":"2024-04-17T09:53:15.066835+0000","last_fullsized":"2024-04-17T09:53:15.066835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:03:55.116831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067392+0000","last_change":"2024-04-17T09:53:12.786015+0000","last_active":"2024-04-17T09:53:15.067392+0000","last_peered":"2024-04-17T09:53:15.067392+0000","last_clean":"2024-04-17T09:53:15.067392+0000","last_became_active":"2024-04-17T09:53:12.784577+0000","last_became_peered":"2024-04-17T09:53:12.784577+0000","last_unstale":"2024-04-17T09:53:15.067392+0000","last_undegraded":"2024-04-17T09:53:15.067392+0000","last_fullsized":"2024-04-17T09:53:15.067392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:53.140146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067598+0000","last_change":"2024-04-17T09:53:13.819959+0000","last_active":"2024-04-17T09:53:15.067598+0000","last_peered":"2024-04-17T09:53:15.067598+0000","last_clean":"2024-04-17T09:53:15.067598+0000","last_became_active":"2024-04-17T09:53:13.819509+0000","last_became_peered":"2024-04-17T09:53:13.819509+0000","last_unstale":"2024-04-17T09:53:15.067598+0000","last_undegraded":"2024-04-17T09:53:15.067598+0000","last_fullsized":"2024-04-17T09:53:15.067598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:07:24.542013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067396+0000","last_change":"2024-04-17T09:53:12.784607+0000","last_active":"2024-04-17T09:53:15.067396+0000","last_peered":"2024-04-17T09:53:15.067396+0000","last_clean":"2024-04-17T09:53:15.067396+0000","last_became_active":"2024-04-17T09:53:12.784265+0000","last_became_peered":"2024-04-17T09:53:12.784265+0000","last_unstale":"2024-04-17T09:53:15.067396+0000","last_undegraded":"2024-04-17T09:53:15.067396+0000","last_fullsized":"2024-04-17T09:53:15.067396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:07.296217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067248+0000","last_change":"2024-04-17T09:53:13.795050+0000","last_active":"2024-04-17T09:53:15.067248+0000","last_peered":"2024-04-17T09:53:15.067248+0000","last_clean":"2024-04-17T09:53:15.067248+0000","last_became_active":"2024-04-17T09:53:13.794002+0000","last_became_peered":"2024-04-17T09:53:13.794002+0000","last_unstale":"2024-04-17T09:53:15.067248+0000","last_undegraded":"2024-04-17T09:53:15.067248+0000","last_fullsized":"2024-04-17T09:53:15.067248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:12.748570+0000","last_clean_scrub_stamp":"2024-04-17T09:53:12.748570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:16:29.674656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-17T09:53:15.067769+0000","last_change":"2024-04-17T09:53:12.782069+0000","last_active":"2024-04-17T09:53:15.067769+0000","last_peered":"2024-04-17T09:53:15.067769+0000","last_clean":"2024-04-17T09:53:15.067769+0000","last_became_active":"2024-04-17T09:53:12.781795+0000","last_became_peered":"2024-04-17T09:53:12.781795+0000","last_unstale":"2024-04-17T09:53:15.067769+0000","last_undegraded":"2024-04-17T09:53:15.067769+0000","last_fullsized":"2024-04-17T09:53:15.067769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:53:11.742136+0000","last_clean_scrub_stamp":"2024-04-17T09:53:11.742136+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:48.300937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":806912,"data_stored":663137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":72277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":208896,"data_stored":72187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":72347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":72313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":815104,"data_stored":663193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T09:53:22.986 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-17T09:53:22.986 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-17T09:53:22.986 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-17T09:53:22.986 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-17T09:53:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.163+0000 7f07d5200640 1 Processor -- start 2024-04-17T09:53:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.163+0000 7f07d5200640 1 -- start start 2024-04-17T09:53:23.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.167+0000 7f07d5200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07d0145f30 0x7f07d0071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.167+0000 7f07d5200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07d015d8d0 0x7f07d0072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.167+0000 7f07d5200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f07d00cce70 0x7f07d00cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.167+0000 7f07d5200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f07d006f400 con 0x7f07d015d8d0 2024-04-17T09:53:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.167+0000 7f07d5200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f07d0146310 con 0x7f07d00cce70 2024-04-17T09:53:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.167+0000 7f07d5200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f07d015d440 con 0x7f07d0145f30 2024-04-17T09:53:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.167+0000 7f07c5400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07d015d8d0 0x7f07d0072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.167+0000 7f07c5400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07d015d8d0 0x7f07d0072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49178/0 (socket says 172.21.15.32:49178) 2024-04-17T09:53:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.167+0000 7f07c5400640 1 -- 172.21.15.32:0/36770360 learned_addr learned my addr 172.21.15.32:0/36770360 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.167+0000 7f07cde00640 1 --2- 172.21.15.32:0/36770360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07d0145f30 0x7f07d0071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.167+0000 7f07ce800640 1 --2- 172.21.15.32:0/36770360 >> v2:172.21.15.144:3300/0 conn(0x7f07d00cce70 0x7f07d00cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.167+0000 7f07c5400640 1 -- 172.21.15.32:0/36770360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07d0145f30 msgr2=0x7f07d0071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.167+0000 7f07c5400640 1 --2- 172.21.15.32:0/36770360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07d0145f30 0x7f07d0071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.167+0000 7f07c5400640 1 -- 172.21.15.32:0/36770360 >> v2:172.21.15.144:3300/0 conn(0x7f07d00cce70 msgr2=0x7f07d00cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:23.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.167+0000 7f07c5400640 1 --2- 172.21.15.32:0/36770360 >> v2:172.21.15.144:3300/0 conn(0x7f07d00cce70 0x7f07d00cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.167+0000 7f07c5400640 1 -- 172.21.15.32:0/36770360 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d00cd840 con 0x7f07d015d8d0 2024-04-17T09:53:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.171+0000 7f07c5400640 1 --2- 172.21.15.32:0/36770360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d015d8d0 0x7f07d0072330 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f07d00b5a10 tx=0x7f07b801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13953f9888afc152 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.171+0000 7f07cd400640 1 -- 172.21.15.32:0/36770360 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b80b5070 con 0x7f07d015d8d0 2024-04-17T09:53:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.171+0000 7f07cd400640 1 -- 172.21.15.32:0/36770360 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07b80a89d0 con 0x7f07d015d8d0 2024-04-17T09:53:23.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.171+0000 7f07cd400640 1 -- 172.21.15.32:0/36770360 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b80a8da0 con 0x7f07d015d8d0 2024-04-17T09:53:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.171+0000 7f07d5200640 1 -- 172.21.15.32:0/36770360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d015d8d0 msgr2=0x7f07d0072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.171+0000 7f07d5200640 1 --2- 172.21.15.32:0/36770360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d015d8d0 0x7f07d0072330 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f07d00b5a10 tx=0x7f07b801ed10 comp rx=0 tx=0).stop 2024-04-17T09:53:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.171+0000 7f07d5200640 1 -- 172.21.15.32:0/36770360 shutdown_connections 2024-04-17T09:53:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.171+0000 7f07d5200640 1 --2- 172.21.15.32:0/36770360 >> v2:172.21.15.144:3300/0 conn(0x7f07d00cce70 0x7f07d00cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.171+0000 7f07d5200640 1 --2- 172.21.15.32:0/36770360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d015d8d0 0x7f07d0072330 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.171+0000 7f07d5200640 1 --2- 172.21.15.32:0/36770360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07d0145f30 0x7f07d0071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.171+0000 7f07d5200640 1 -- 172.21.15.32:0/36770360 >> 172.21.15.32:0/36770360 conn(0x7f07d006ba30 msgr2=0x7f07d006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.171+0000 7f07d5200640 1 -- 172.21.15.32:0/36770360 shutdown_connections 2024-04-17T09:53:23.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.171+0000 7f07d5200640 1 -- 172.21.15.32:0/36770360 wait complete. 2024-04-17T09:53:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.171+0000 7f07d5200640 1 Processor -- start 2024-04-17T09:53:23.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.171+0000 7f07d5200640 1 -- start start 2024-04-17T09:53:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.171+0000 7f07d5200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07d00cce70 0x7f07d01426c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.171+0000 7f07d5200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07d0145f30 0x7f07d015ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.171+0000 7f07d5200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d015d8d0 0x7f07d015ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.171+0000 7f07d5200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f07d015cdf0 con 0x7f07d015d8d0 2024-04-17T09:53:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.171+0000 7f07d5200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f07d015cb90 con 0x7f07d00cce70 2024-04-17T09:53:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.171+0000 7f07d5200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f07d015d0d0 con 0x7f07d0145f30 2024-04-17T09:53:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.171+0000 7f07ce800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d015d8d0 0x7f07d015ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.171+0000 7f07ce800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d015d8d0 0x7f07d015ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49190/0 (socket says 172.21.15.32:49190) 2024-04-17T09:53:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.175+0000 7f07ce800640 1 -- 172.21.15.32:0/1964044445 learned_addr learned my addr 172.21.15.32:0/1964044445 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.175+0000 7f07cde00640 1 --2- 172.21.15.32:0/1964044445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07d00cce70 0x7f07d01426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.175+0000 7f07c5400640 1 --2- 172.21.15.32:0/1964044445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07d0145f30 0x7f07d015ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.175+0000 7f07ce800640 1 -- 172.21.15.32:0/1964044445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07d0145f30 msgr2=0x7f07d015ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.175+0000 7f07ce800640 1 --2- 172.21.15.32:0/1964044445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07d0145f30 0x7f07d015ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.175+0000 7f07ce800640 1 -- 172.21.15.32:0/1964044445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07d00cce70 msgr2=0x7f07d01426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:23.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.175+0000 7f07ce800640 1 --2- 172.21.15.32:0/1964044445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07d00cce70 0x7f07d01426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.175+0000 7f07ce800640 1 -- 172.21.15.32:0/1964044445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d01e6030 con 0x7f07d015d8d0 2024-04-17T09:53:23.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.175+0000 7f07ce800640 1 --2- 172.21.15.32:0/1964044445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d015d8d0 0x7f07d015ed70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f07c8007e60 tx=0x7f07c800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:23.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.175+0000 7f07c6a00640 1 -- 172.21.15.32:0/1964044445 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c8014620 con 0x7f07d015d8d0 2024-04-17T09:53:23.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.175+0000 7f07c6a00640 1 -- 172.21.15.32:0/1964044445 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07c8019070 con 0x7f07d015d8d0 2024-04-17T09:53:23.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.175+0000 7f07c6a00640 1 -- 172.21.15.32:0/1964044445 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c8014a20 con 0x7f07d015d8d0 2024-04-17T09:53:23.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.175+0000 7f07d5200640 1 -- 172.21.15.32:0/1964044445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d01e6310 con 0x7f07d015d8d0 2024-04-17T09:53:23.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.175+0000 7f07d5200640 1 -- 172.21.15.32:0/1964044445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d01e68d0 con 0x7f07d015d8d0 2024-04-17T09:53:23.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.175+0000 7f07d5200640 1 -- 172.21.15.32:0/1964044445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07d0068a90 con 0x7f07d015d8d0 2024-04-17T09:53:23.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.175+0000 7f07c6a00640 1 -- 172.21.15.32:0/1964044445 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f07c8014bc0 con 0x7f07d015d8d0 2024-04-17T09:53:23.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.175+0000 7f07c6a00640 1 --2- 172.21.15.32:0/1964044445 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07ac068380 0x7f07ac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:23.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.175+0000 7f07c6a00640 1 -- 172.21.15.32:0/1964044445 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f07c80870f0 con 0x7f07d015d8d0 2024-04-17T09:53:23.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.179+0000 7f07cde00640 1 --2- 172.21.15.32:0/1964044445 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07ac068380 0x7f07ac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:23.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.179+0000 7f07cde00640 1 --2- 172.21.15.32:0/1964044445 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07ac068380 0x7f07ac06a840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f07d006f350 tx=0x7f07c0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:23.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.183+0000 7f07c6a00640 1 -- 172.21.15.32:0/1964044445 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f07c8057050 con 0x7f07d015d8d0 2024-04-17T09:53:23.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.383+0000 7f07d5200640 1 -- 172.21.15.32:0/1964044445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f07d0072f90 con 0x7f07d015d8d0 2024-04-17T09:53:23.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.383+0000 7f07c6a00640 1 -- 172.21.15.32:0/1964044445 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f07c805b350 con 0x7f07d015d8d0 2024-04-17T09:53:23.386 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:23.386 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-17T09:53:23.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.383+0000 7f07d5200640 1 -- 172.21.15.32:0/1964044445 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07ac068380 msgr2=0x7f07ac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:23.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.383+0000 7f07d5200640 1 --2- 172.21.15.32:0/1964044445 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07ac068380 0x7f07ac06a840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f07d006f350 tx=0x7f07c0008040 comp rx=0 tx=0).stop 2024-04-17T09:53:23.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.383+0000 7f07d5200640 1 -- 172.21.15.32:0/1964044445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d015d8d0 msgr2=0x7f07d015ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:23.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.383+0000 7f07d5200640 1 --2- 172.21.15.32:0/1964044445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d015d8d0 0x7f07d015ed70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f07c8007e60 tx=0x7f07c800e4b0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.383+0000 7f07d5200640 1 -- 172.21.15.32:0/1964044445 shutdown_connections 2024-04-17T09:53:23.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.383+0000 7f07d5200640 1 --2- 172.21.15.32:0/1964044445 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07ac068380 0x7f07ac06a840 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.383+0000 7f07d5200640 1 --2- 172.21.15.32:0/1964044445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07d0145f30 0x7f07d015ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.383+0000 7f07d5200640 1 --2- 172.21.15.32:0/1964044445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07d015d8d0 0x7f07d015ed70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.383+0000 7f07d5200640 1 --2- 172.21.15.32:0/1964044445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07d00cce70 0x7f07d01426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.383+0000 7f07d5200640 1 -- 172.21.15.32:0/1964044445 >> 172.21.15.32:0/1964044445 conn(0x7f07d006ba30 msgr2=0x7f07d00ce220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:23.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.383+0000 7f07d5200640 1 -- 172.21.15.32:0/1964044445 shutdown_connections 2024-04-17T09:53:23.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.383+0000 7f07d5200640 1 -- 172.21.15.32:0/1964044445 wait complete. 2024-04-17T09:53:23.406 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-17T09:53:23.407 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:53:23.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.543+0000 7f58bd000640 1 Processor -- start 2024-04-17T09:53:23.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.543+0000 7f58bd000640 1 -- start start 2024-04-17T09:53:23.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.543+0000 7f58bd000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58b8072500 0x7f58b80cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:23.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.543+0000 7f58bd000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58b80cdac0 con 0x7f58b815d8d0 2024-04-17T09:53:23.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.543+0000 7f58bd000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58b815cdf0 con 0x7f58b8145f30 2024-04-17T09:53:23.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.543+0000 7f58bd000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f58b806f400 con 0x7f58b8072500 2024-04-17T09:53:23.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.543+0000 7f58b6800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f58b815d8d0 0x7f58b81601d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48812/0 (socket says 172.21.15.32:48812) 2024-04-17T09:53:23.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.543+0000 7f58b6800640 1 -- 172.21.15.32:0/646325285 learned_addr learned my addr 172.21.15.32:0/646325285 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:23.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.543+0000 7f58afe00640 1 --2- 172.21.15.32:0/646325285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58b8072500 0x7f58b80cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:23.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.543+0000 7f58b5400640 1 -- 172.21.15.32:0/646325285 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2990601478 0 0) 0x7f58b80cdac0 con 0x7f58b815d8d0 2024-04-17T09:53:23.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.543+0000 7f58b5400640 1 -- 172.21.15.32:0/646325285 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f589c003530 con 0x7f58b815d8d0 2024-04-17T09:53:23.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.543+0000 7f58b5400640 1 -- 172.21.15.32:0/646325285 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 969329484 0 0) 0x7f58b815cdf0 con 0x7f58b8145f30 2024-04-17T09:53:23.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.543+0000 7f58b5400640 1 -- 172.21.15.32:0/646325285 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58b80cdac0 con 0x7f58b8145f30 2024-04-17T09:53:23.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.543+0000 7f58afe00640 1 -- 172.21.15.32:0/646325285 >> v1:172.21.15.144:6789/0 conn(0x7f58b8145f30 legacy=0x7f58b8071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:23.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.543+0000 7f58afe00640 1 -- 172.21.15.32:0/646325285 >> v1:172.21.15.32:6789/0 conn(0x7f58b815d8d0 legacy=0x7f58b81601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:23.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.543+0000 7f58afe00640 1 -- 172.21.15.32:0/646325285 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b815d0d0 con 0x7f58b8072500 2024-04-17T09:53:23.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.547+0000 7f58afe00640 1 --2- 172.21.15.32:0/646325285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58b8072500 0x7f58b80cce70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f58a00097f0 tx=0x7f58a001ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f95f454d560482d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:23.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.547+0000 7f58b5400640 1 -- 172.21.15.32:0/646325285 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a00b5070 con 0x7f58b8072500 2024-04-17T09:53:23.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.547+0000 7f58b5400640 1 -- 172.21.15.32:0/646325285 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a00a89d0 con 0x7f58b8072500 2024-04-17T09:53:23.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.547+0000 7f58b5400640 1 -- 172.21.15.32:0/646325285 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a00a8da0 con 0x7f58b8072500 2024-04-17T09:53:23.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.547+0000 7f58bd000640 1 -- 172.21.15.32:0/646325285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58b8072500 msgr2=0x7f58b80cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:23.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.547+0000 7f58bd000640 1 --2- 172.21.15.32:0/646325285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58b8072500 0x7f58b80cce70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f58a00097f0 tx=0x7f58a001ecd0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.547+0000 7f58bd000640 1 -- 172.21.15.32:0/646325285 shutdown_connections 2024-04-17T09:53:23.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.547+0000 7f58bd000640 1 --2- 172.21.15.32:0/646325285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58b8072500 0x7f58b80cce70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.547+0000 7f58bd000640 1 -- 172.21.15.32:0/646325285 >> 172.21.15.32:0/646325285 conn(0x7f58b806ba30 msgr2=0x7f58b806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:23.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58bd000640 1 -- 172.21.15.32:0/646325285 shutdown_connections 2024-04-17T09:53:23.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58bd000640 1 -- 172.21.15.32:0/646325285 wait complete. 2024-04-17T09:53:23.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58bd000640 1 Processor -- start 2024-04-17T09:53:23.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58bd000640 1 -- start start 2024-04-17T09:53:23.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58bd000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58b8072500 0x7f58b80c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58bd000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58b8145f30 0x7f58b80c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58bd000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58b815d8d0 0x7f58b80c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58bd000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f58b80cd7c0 con 0x7f58b8145f30 2024-04-17T09:53:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58bd000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f58b80cd640 con 0x7f58b815d8d0 2024-04-17T09:53:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58bd000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f58b8146310 con 0x7f58b8072500 2024-04-17T09:53:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58afe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58b8145f30 0x7f58b80c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58afe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58b8145f30 0x7f58b80c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49198/0 (socket says 172.21.15.32:49198) 2024-04-17T09:53:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58afe00640 1 -- 172.21.15.32:0/3366340342 learned_addr learned my addr 172.21.15.32:0/3366340342 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58b6800640 1 --2- 172.21.15.32:0/3366340342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58b815d8d0 0x7f58b80c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58b5e00640 1 --2- 172.21.15.32:0/3366340342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58b8072500 0x7f58b80c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58afe00640 1 -- 172.21.15.32:0/3366340342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58b8072500 msgr2=0x7f58b80c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58afe00640 1 --2- 172.21.15.32:0/3366340342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58b8072500 0x7f58b80c63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58afe00640 1 -- 172.21.15.32:0/3366340342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58b815d8d0 msgr2=0x7f58b80c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58afe00640 1 --2- 172.21.15.32:0/3366340342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58b815d8d0 0x7f58b80c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58afe00640 1 -- 172.21.15.32:0/3366340342 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b80c8260 con 0x7f58b8145f30 2024-04-17T09:53:23.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58afe00640 1 --2- 172.21.15.32:0/3366340342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58b8145f30 0x7f58b80c8a10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f58a001b8e0 tx=0x7f58a001ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:23.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58ae000640 1 -- 172.21.15.32:0/3366340342 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a00bf070 con 0x7f58b8145f30 2024-04-17T09:53:23.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58ae000640 1 -- 172.21.15.32:0/3366340342 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a00b0720 con 0x7f58b8145f30 2024-04-17T09:53:23.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.551+0000 7f58ae000640 1 -- 172.21.15.32:0/3366340342 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a00b5040 con 0x7f58b8145f30 2024-04-17T09:53:23.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.555+0000 7f58bd000640 1 -- 172.21.15.32:0/3366340342 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b80c6910 con 0x7f58b8145f30 2024-04-17T09:53:23.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.555+0000 7f58bd000640 1 -- 172.21.15.32:0/3366340342 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b80c7ca0 con 0x7f58b8145f30 2024-04-17T09:53:23.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.555+0000 7f58bd000640 1 -- 172.21.15.32:0/3366340342 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b8068a90 con 0x7f58b8145f30 2024-04-17T09:53:23.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.555+0000 7f58ae000640 1 -- 172.21.15.32:0/3366340342 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f58a00c2070 con 0x7f58b8145f30 2024-04-17T09:53:23.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.555+0000 7f58ae000640 1 --2- 172.21.15.32:0/3366340342 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5894068380 0x7f589406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:23.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.555+0000 7f58ae000640 1 -- 172.21.15.32:0/3366340342 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f58a0122f90 con 0x7f58b8145f30 2024-04-17T09:53:23.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.555+0000 7f58b5e00640 1 --2- 172.21.15.32:0/3366340342 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5894068380 0x7f589406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:23.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.555+0000 7f58b5e00640 1 --2- 172.21.15.32:0/3366340342 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5894068380 0x7f589406a840 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f58a80011f0 tx=0x7f58a8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:23.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.559+0000 7f58ae000640 1 -- 172.21.15.32:0/3366340342 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f58a0018830 con 0x7f58b8145f30 2024-04-17T09:53:23.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.727+0000 7f58bd000640 1 -- 172.21.15.32:0/3366340342 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58b80c6b40 con 0x7f58b8145f30 2024-04-17T09:53:23.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.727+0000 7f58ae000640 1 -- 172.21.15.32:0/3366340342 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f58a00f2ef0 con 0x7f58b8145f30 2024-04-17T09:53:23.730 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:23.730 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:14.779535+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:53:23.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.727+0000 7f58bd000640 1 -- 172.21.15.32:0/3366340342 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5894068380 msgr2=0x7f589406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:23.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.727+0000 7f58bd000640 1 --2- 172.21.15.32:0/3366340342 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5894068380 0x7f589406a840 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f58a80011f0 tx=0x7f58a8009040 comp rx=0 tx=0).stop 2024-04-17T09:53:23.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.727+0000 7f58bd000640 1 -- 172.21.15.32:0/3366340342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58b8145f30 msgr2=0x7f58b80c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:23.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.727+0000 7f58bd000640 1 --2- 172.21.15.32:0/3366340342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58b8145f30 0x7f58b80c8a10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f58a001b8e0 tx=0x7f58a001ba70 comp rx=0 tx=0).stop 2024-04-17T09:53:23.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.727+0000 7f58bd000640 1 -- 172.21.15.32:0/3366340342 shutdown_connections 2024-04-17T09:53:23.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.727+0000 7f58bd000640 1 --2- 172.21.15.32:0/3366340342 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5894068380 0x7f589406a840 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.727+0000 7f58bd000640 1 --2- 172.21.15.32:0/3366340342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58b815d8d0 0x7f58b80c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.727+0000 7f58bd000640 1 --2- 172.21.15.32:0/3366340342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58b8145f30 0x7f58b80c8a10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.727+0000 7f58bd000640 1 --2- 172.21.15.32:0/3366340342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58b8072500 0x7f58b80c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.727+0000 7f58bd000640 1 -- 172.21.15.32:0/3366340342 >> 172.21.15.32:0/3366340342 conn(0x7f58b806ba30 msgr2=0x7f58b806be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:23.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.731+0000 7f58bd000640 1 -- 172.21.15.32:0/3366340342 shutdown_connections 2024-04-17T09:53:23.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.731+0000 7f58bd000640 1 -- 172.21.15.32:0/3366340342 wait complete. 2024-04-17T09:53:23.751 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:53:11.198098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:53:12.118982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:53:23.752 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:53:23.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.891+0000 7f2716e00640 1 Processor -- start 2024-04-17T09:53:23.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.891+0000 7f2716e00640 1 -- start start 2024-04-17T09:53:23.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f2716e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f271006f7c0 0x7f271006ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:23.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f2716e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f271006f270 0x7f271006bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:23.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f2716e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27100cc940 con 0x7f271015b7d0 2024-04-17T09:53:23.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f2716e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f27100ccc20 con 0x7f271006f7c0 2024-04-17T09:53:23.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f2716e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f271006e0b0 con 0x7f271006f270 2024-04-17T09:53:23.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f270fe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f271015b7d0 0x7f271006e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48818/0 (socket says 172.21.15.32:48818) 2024-04-17T09:53:23.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f270fe00640 1 -- 172.21.15.32:0/665253298 learned_addr learned my addr 172.21.15.32:0/665253298 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:23.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f270f400640 1 -- 172.21.15.32:0/665253298 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3403161433 0 0) 0x7f27100cc940 con 0x7f271015b7d0 2024-04-17T09:53:23.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f270f400640 1 -- 172.21.15.32:0/665253298 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26f4003530 con 0x7f271015b7d0 2024-04-17T09:53:23.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f2714e00640 1 --2- 172.21.15.32:0/665253298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f271006f270 0x7f271006bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:23.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f2707400640 1 --2- 172.21.15.32:0/665253298 >> v2:172.21.15.144:3300/0 conn(0x7f271006f7c0 0x7f271006ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:23.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f270f400640 1 -- 172.21.15.32:0/665253298 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1611735689 0 0) 0x7f26f4003530 con 0x7f271015b7d0 2024-04-17T09:53:23.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f270f400640 1 -- 172.21.15.32:0/665253298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f271006f270 msgr2=0x7f271006bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:23.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f270f400640 1 --2- 172.21.15.32:0/665253298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f271006f270 0x7f271006bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f270f400640 1 -- 172.21.15.32:0/665253298 >> v2:172.21.15.144:3300/0 conn(0x7f271006f7c0 msgr2=0x7f271006ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:23.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f270f400640 1 --2- 172.21.15.32:0/665253298 >> v2:172.21.15.144:3300/0 conn(0x7f271006f7c0 0x7f271006ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f270f400640 1 -- 172.21.15.32:0/665253298 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271006e390 con 0x7f271015b7d0 2024-04-17T09:53:23.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f270f400640 1 -- 172.21.15.32:0/665253298 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f2700001680 con 0x7f271015b7d0 2024-04-17T09:53:23.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f2714e00640 1 --2- 172.21.15.32:0/665253298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f271006f270 0x7f271006bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:23.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f270f400640 1 -- 172.21.15.32:0/665253298 >> v1:172.21.15.32:6789/0 conn(0x7f271015b7d0 legacy=0x7f271006e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:23.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f270f400640 1 --2- 172.21.15.32:0/665253298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f271006f270 0x7f26f40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:23.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f270f400640 1 --2- 172.21.15.32:0/665253298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f4003f50 0x7f26f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:23.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f270f400640 1 --2- 172.21.15.32:0/665253298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26f4004d50 0x7f26f40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:23.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f270f400640 1 -- 172.21.15.32:0/665253298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f27100ccc20 con 0x7f26f4003f50 2024-04-17T09:53:23.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f270f400640 1 -- 172.21.15.32:0/665253298 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f271006e0b0 con 0x7f26f4004d50 2024-04-17T09:53:23.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f270f400640 1 -- 172.21.15.32:0/665253298 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f26f400dce0 con 0x7f271006f270 2024-04-17T09:53:23.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f2707400640 1 --2- 172.21.15.32:0/665253298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f4003f50 0x7f26f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:23.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f2714e00640 1 --2- 172.21.15.32:0/665253298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26f4004d50 0x7f26f40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:23.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f270fe00640 1 --2- 172.21.15.32:0/665253298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f271006f270 0x7f26f40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:23.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f2707400640 1 -- 172.21.15.32:0/665253298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f271006f270 msgr2=0x7f26f40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:23.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f2707400640 1 --2- 172.21.15.32:0/665253298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f271006f270 0x7f26f40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f2707400640 1 -- 172.21.15.32:0/665253298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26f4004d50 msgr2=0x7f26f40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:23.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f2707400640 1 --2- 172.21.15.32:0/665253298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26f4004d50 0x7f26f40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f2707400640 1 -- 172.21.15.32:0/665253298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f4003530 con 0x7f26f4003f50 2024-04-17T09:53:23.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.895+0000 7f270fe00640 1 --2- 172.21.15.32:0/665253298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f271006f270 0x7f26f40039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:23.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.899+0000 7f2707400640 1 --2- 172.21.15.32:0/665253298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f4003f50 0x7f26f4004810 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009ae0 tx=0x7f26f801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2616b1449f6e388 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:23.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.899+0000 7f270f400640 1 -- 172.21.15.32:0/665253298 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f80b2070 con 0x7f26f4003f50 2024-04-17T09:53:23.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.899+0000 7f270f400640 1 -- 172.21.15.32:0/665253298 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26f80a5590 con 0x7f26f4003f50 2024-04-17T09:53:23.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.899+0000 7f270f400640 1 -- 172.21.15.32:0/665253298 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f80096b0 con 0x7f26f4003f50 2024-04-17T09:53:23.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.899+0000 7f2716e00640 1 -- 172.21.15.32:0/665253298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f4003f50 msgr2=0x7f26f4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:23.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.899+0000 7f2716e00640 1 --2- 172.21.15.32:0/665253298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26f4003f50 0x7f26f4004810 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009ae0 tx=0x7f26f801cab0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.899+0000 7f2714e00640 1 -- 172.21.15.32:0/665253298 reap_dead start 2024-04-17T09:53:23.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.899+0000 7f2716e00640 1 -- 172.21.15.32:0/665253298 shutdown_connections 2024-04-17T09:53:23.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.899+0000 7f2716e00640 1 -- 172.21.15.32:0/665253298 >> 172.21.15.32:0/665253298 conn(0x7f27100b2120 msgr2=0x7f27100b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:23.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.899+0000 7f2716e00640 1 -- 172.21.15.32:0/665253298 shutdown_connections 2024-04-17T09:53:23.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.899+0000 7f2716e00640 1 -- 172.21.15.32:0/665253298 wait complete. 2024-04-17T09:53:23.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.899+0000 7f2716e00640 1 Processor -- start 2024-04-17T09:53:23.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.899+0000 7f2716e00640 1 -- start start 2024-04-17T09:53:23.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.903+0000 7f2716e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26f4003f50 0x7f271014f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:23.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.903+0000 7f2716e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f271014f350 0x7f271015e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:23.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.903+0000 7f2716e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2710071e60 0x7f27100722e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:23.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.903+0000 7f2716e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f27100cee30 con 0x7f271014f350 2024-04-17T09:53:23.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.903+0000 7f2716e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f27100cecb0 con 0x7f2710071e60 2024-04-17T09:53:23.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.903+0000 7f2716e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f271006c380 con 0x7f26f4003f50 2024-04-17T09:53:23.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.903+0000 7f2707400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f271014f350 0x7f271015e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:23.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.903+0000 7f2707400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f271014f350 0x7f271015e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49228/0 (socket says 172.21.15.32:49228) 2024-04-17T09:53:23.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.903+0000 7f2707400640 1 -- 172.21.15.32:0/4290938507 learned_addr learned my addr 172.21.15.32:0/4290938507 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:23.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.903+0000 7f2707400640 1 -- 172.21.15.32:0/4290938507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26f4003f50 msgr2=0x7f271014f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:23.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.903+0000 7f2707400640 1 --2- 172.21.15.32:0/4290938507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26f4003f50 0x7f271014f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.903+0000 7f2707400640 1 -- 172.21.15.32:0/4290938507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2710071e60 msgr2=0x7f27100722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:23.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.903+0000 7f2707400640 1 --2- 172.21.15.32:0/4290938507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2710071e60 0x7f27100722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:23.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.903+0000 7f2707400640 1 -- 172.21.15.32:0/4290938507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2710157950 con 0x7f271014f350 2024-04-17T09:53:23.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.903+0000 7f2707400640 1 --2- 172.21.15.32:0/4290938507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f271014f350 0x7f271015e220 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f26f801c8e0 tx=0x7f26f4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:23.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.903+0000 7f270cc00640 1 -- 172.21.15.32:0/4290938507 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f80b2040 con 0x7f271014f350 2024-04-17T09:53:23.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.903+0000 7f270cc00640 1 -- 172.21.15.32:0/4290938507 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26f80a5cc0 con 0x7f271014f350 2024-04-17T09:53:23.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.903+0000 7f270cc00640 1 -- 172.21.15.32:0/4290938507 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f8004220 con 0x7f271014f350 2024-04-17T09:53:23.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.903+0000 7f2716e00640 1 -- 172.21.15.32:0/4290938507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2710157c30 con 0x7f271014f350 2024-04-17T09:53:23.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.903+0000 7f2716e00640 1 -- 172.21.15.32:0/4290938507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27101581f0 con 0x7f271014f350 2024-04-17T09:53:23.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.903+0000 7f2716e00640 1 -- 172.21.15.32:0/4290938507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2710072820 con 0x7f271014f350 2024-04-17T09:53:23.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.903+0000 7f270cc00640 1 -- 172.21.15.32:0/4290938507 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f26f8003620 con 0x7f271014f350 2024-04-17T09:53:23.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.907+0000 7f270cc00640 1 --2- 172.21.15.32:0/4290938507 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f26ec0683b0 0x7f26ec06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:23.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.907+0000 7f270cc00640 1 -- 172.21.15.32:0/4290938507 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f26f8120550 con 0x7f271014f350 2024-04-17T09:53:23.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.907+0000 7f270fe00640 1 --2- 172.21.15.32:0/4290938507 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f26ec0683b0 0x7f26ec06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:23.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.907+0000 7f270fe00640 1 --2- 172.21.15.32:0/4290938507 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f26ec0683b0 0x7f26ec06a870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2700000c10 tx=0x7f2700001bd0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:23.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:23.911+0000 7f270cc00640 1 -- 172.21.15.32:0/4290938507 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f26f80167c0 con 0x7f271014f350 2024-04-17T09:53:24.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.079+0000 7f2716e00640 1 -- 172.21.15.32:0/4290938507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2710068a90 con 0x7f271014f350 2024-04-17T09:53:24.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.079+0000 7f270cc00640 1 -- 172.21.15.32:0/4290938507 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f26f80f04c0 con 0x7f271014f350 2024-04-17T09:53:24.084 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:53:24.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.083+0000 7f2716e00640 1 -- 172.21.15.32:0/4290938507 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f26ec0683b0 msgr2=0x7f26ec06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:24.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.083+0000 7f2716e00640 1 --2- 172.21.15.32:0/4290938507 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f26ec0683b0 0x7f26ec06a870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2700000c10 tx=0x7f2700001bd0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.083+0000 7f2716e00640 1 -- 172.21.15.32:0/4290938507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f271014f350 msgr2=0x7f271015e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:24.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.083+0000 7f2716e00640 1 --2- 172.21.15.32:0/4290938507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f271014f350 0x7f271015e220 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f26f801c8e0 tx=0x7f26f4005840 comp rx=0 tx=0).stop 2024-04-17T09:53:24.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.083+0000 7f2716e00640 1 -- 172.21.15.32:0/4290938507 shutdown_connections 2024-04-17T09:53:24.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.083+0000 7f2716e00640 1 --2- 172.21.15.32:0/4290938507 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f26ec0683b0 0x7f26ec06a870 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.083+0000 7f2716e00640 1 --2- 172.21.15.32:0/4290938507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2710071e60 0x7f27100722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.083+0000 7f2716e00640 1 --2- 172.21.15.32:0/4290938507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f271014f350 0x7f271015e220 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.083+0000 7f2716e00640 1 --2- 172.21.15.32:0/4290938507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26f4003f50 0x7f271014f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.083+0000 7f2716e00640 1 -- 172.21.15.32:0/4290938507 >> 172.21.15.32:0/4290938507 conn(0x7f27100b2120 msgr2=0x7f27100b1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:24.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.083+0000 7f2716e00640 1 -- 172.21.15.32:0/4290938507 shutdown_connections 2024-04-17T09:53:24.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.083+0000 7f2716e00640 1 -- 172.21.15.32:0/4290938507 wait complete. 2024-04-17T09:53:24.105 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:53:24.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.247+0000 7f699f400640 1 Processor -- start 2024-04-17T09:53:24.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.247+0000 7f699f400640 1 -- start start 2024-04-17T09:53:24.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.247+0000 7f699f400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f699806ba30 0x7f699806be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:24.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.247+0000 7f699f400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f699806c350 con 0x7f6998071e60 2024-04-17T09:53:24.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.247+0000 7f699f400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69980d0130 con 0x7f69980cf370 2024-04-17T09:53:24.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.247+0000 7f699f400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f69980cfef0 con 0x7f699806ba30 2024-04-17T09:53:24.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.247+0000 7f699ca00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f699806ba30 0x7f699806be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:24.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.247+0000 7f699d400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6998071e60 0x7f6998072240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48832/0 (socket says 172.21.15.32:48832) 2024-04-17T09:53:24.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.247+0000 7f699d400640 1 -- 172.21.15.32:0/304174953 learned_addr learned my addr 172.21.15.32:0/304174953 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:24.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.247+0000 7f699ca00640 1 -- 172.21.15.32:0/304174953 >> v1:172.21.15.144:6789/0 conn(0x7f69980cf370 legacy=0x7f69980cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:24.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.247+0000 7f699ca00640 1 -- 172.21.15.32:0/304174953 >> v1:172.21.15.32:6789/0 conn(0x7f6998071e60 legacy=0x7f6998072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:24.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.247+0000 7f699ca00640 1 -- 172.21.15.32:0/304174953 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699815d090 con 0x7f699806ba30 2024-04-17T09:53:24.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.247+0000 7f6997400640 1 -- 172.21.15.32:0/304174953 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 117407624 0 0) 0x7f699806c350 con 0x7f6998071e60 2024-04-17T09:53:24.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.251+0000 7f699ca00640 1 --2- 172.21.15.32:0/304174953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f699806ba30 0x7f699806be10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f698c0097c0 tx=0x7f698c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b569ac0d83b3523 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:24.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.251+0000 7f6997400640 1 -- 172.21.15.32:0/304174953 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c0b1070 con 0x7f699806ba30 2024-04-17T09:53:24.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.251+0000 7f6997400640 1 -- 172.21.15.32:0/304174953 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f698c0a8050 con 0x7f699806ba30 2024-04-17T09:53:24.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.251+0000 7f6997400640 1 -- 172.21.15.32:0/304174953 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c0ac860 con 0x7f699806ba30 2024-04-17T09:53:24.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.251+0000 7f699f400640 1 -- 172.21.15.32:0/304174953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f699806ba30 msgr2=0x7f699806be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:24.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.251+0000 7f699f400640 1 --2- 172.21.15.32:0/304174953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f699806ba30 0x7f699806be10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f698c0097c0 tx=0x7f698c01c3e0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.251+0000 7f699f400640 1 -- 172.21.15.32:0/304174953 shutdown_connections 2024-04-17T09:53:24.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.251+0000 7f699f400640 1 --2- 172.21.15.32:0/304174953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f699806ba30 0x7f699806be10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.251+0000 7f699f400640 1 -- 172.21.15.32:0/304174953 >> 172.21.15.32:0/304174953 conn(0x7f69980b2120 msgr2=0x7f69980b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:24.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.251+0000 7f699f400640 1 -- 172.21.15.32:0/304174953 shutdown_connections 2024-04-17T09:53:24.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.251+0000 7f699f400640 1 -- 172.21.15.32:0/304174953 wait complete. 2024-04-17T09:53:24.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.251+0000 7f699f400640 1 Processor -- start 2024-04-17T09:53:24.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.251+0000 7f699f400640 1 -- start start 2024-04-17T09:53:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f699f400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f699806ba30 0x7f6998159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f699f400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6998071e60 0x7f699806eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f699f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69980cf370 0x7f699806f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f699f400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f699815db70 con 0x7f69980cf370 2024-04-17T09:53:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f699f400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f699815cd00 con 0x7f699806ba30 2024-04-17T09:53:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f699f400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f699815dcf0 con 0x7f6998071e60 2024-04-17T09:53:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f699d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69980cf370 0x7f699806f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f699d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69980cf370 0x7f699806f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49236/0 (socket says 172.21.15.32:49236) 2024-04-17T09:53:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f699d400640 1 -- 172.21.15.32:0/3456629990 learned_addr learned my addr 172.21.15.32:0/3456629990 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f699ca00640 1 --2- 172.21.15.32:0/3456629990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f699806ba30 0x7f6998159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f6997e00640 1 --2- 172.21.15.32:0/3456629990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6998071e60 0x7f699806eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:24.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f699d400640 1 -- 172.21.15.32:0/3456629990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6998071e60 msgr2=0x7f699806eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:24.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f699d400640 1 --2- 172.21.15.32:0/3456629990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6998071e60 0x7f699806eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f699d400640 1 -- 172.21.15.32:0/3456629990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f699806ba30 msgr2=0x7f6998159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:24.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f699d400640 1 --2- 172.21.15.32:0/3456629990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f699806ba30 0x7f6998159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f699d400640 1 -- 172.21.15.32:0/3456629990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699813d4c0 con 0x7f69980cf370 2024-04-17T09:53:24.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f699d400640 1 --2- 172.21.15.32:0/3456629990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69980cf370 0x7f699806f030 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f699000c7a0 tx=0x7f699000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f6994c00640 1 -- 172.21.15.32:0/3456629990 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990018070 con 0x7f69980cf370 2024-04-17T09:53:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f6994c00640 1 -- 172.21.15.32:0/3456629990 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f699000a640 con 0x7f69980cf370 2024-04-17T09:53:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f6994c00640 1 -- 172.21.15.32:0/3456629990 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699000aab0 con 0x7f69980cf370 2024-04-17T09:53:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f699f400640 1 -- 172.21.15.32:0/3456629990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699813d7a0 con 0x7f69980cf370 2024-04-17T09:53:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f699f400640 1 -- 172.21.15.32:0/3456629990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69980b3e30 con 0x7f69980cf370 2024-04-17T09:53:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f699f400640 1 -- 172.21.15.32:0/3456629990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69980b4030 con 0x7f69980cf370 2024-04-17T09:53:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.255+0000 7f6994c00640 1 -- 172.21.15.32:0/3456629990 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f699000ac50 con 0x7f69980cf370 2024-04-17T09:53:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.259+0000 7f6994c00640 1 --2- 172.21.15.32:0/3456629990 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6978068380 0x7f697806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.259+0000 7f6994c00640 1 -- 172.21.15.32:0/3456629990 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6990086090 con 0x7f69980cf370 2024-04-17T09:53:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.259+0000 7f699ca00640 1 --2- 172.21.15.32:0/3456629990 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6978068380 0x7f697806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:24.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.259+0000 7f699ca00640 1 --2- 172.21.15.32:0/3456629990 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6978068380 0x7f697806a840 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f698c0ae000 tx=0x7f698c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:24.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.263+0000 7f6994c00640 1 -- 172.21.15.32:0/3456629990 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f69980b4030 con 0x7f69980cf370 2024-04-17T09:53:24.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.427+0000 7f699f400640 1 -- 172.21.15.32:0/3456629990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f699813dc50 con 0x7f69980cf370 2024-04-17T09:53:24.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.427+0000 7f6994c00640 1 -- 172.21.15.32:0/3456629990 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f699813dc50 con 0x7f69980cf370 2024-04-17T09:53:24.432 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:53:24.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.431+0000 7f699f400640 1 -- 172.21.15.32:0/3456629990 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6978068380 msgr2=0x7f697806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:24.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.431+0000 7f699f400640 1 --2- 172.21.15.32:0/3456629990 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6978068380 0x7f697806a840 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f698c0ae000 tx=0x7f698c008040 comp rx=0 tx=0).stop 2024-04-17T09:53:24.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.431+0000 7f699f400640 1 -- 172.21.15.32:0/3456629990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69980cf370 msgr2=0x7f699806f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:24.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.431+0000 7f699f400640 1 --2- 172.21.15.32:0/3456629990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69980cf370 0x7f699806f030 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f699000c7a0 tx=0x7f699000cc60 comp rx=0 tx=0).stop 2024-04-17T09:53:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.431+0000 7f699f400640 1 -- 172.21.15.32:0/3456629990 shutdown_connections 2024-04-17T09:53:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.431+0000 7f699f400640 1 --2- 172.21.15.32:0/3456629990 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6978068380 0x7f697806a840 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.431+0000 7f699f400640 1 --2- 172.21.15.32:0/3456629990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6998071e60 0x7f699806eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.431+0000 7f699f400640 1 --2- 172.21.15.32:0/3456629990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69980cf370 0x7f699806f030 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.431+0000 7f699f400640 1 --2- 172.21.15.32:0/3456629990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f699806ba30 0x7f6998159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.431+0000 7f699f400640 1 -- 172.21.15.32:0/3456629990 >> 172.21.15.32:0/3456629990 conn(0x7f69980b2120 msgr2=0x7f69980aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.431+0000 7f699f400640 1 -- 172.21.15.32:0/3456629990 shutdown_connections 2024-04-17T09:53:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.431+0000 7f699f400640 1 -- 172.21.15.32:0/3456629990 wait complete. 2024-04-17T09:53:24.453 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:53:24.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.591+0000 7ffb4be00640 1 Processor -- start 2024-04-17T09:53:24.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.591+0000 7ffb4be00640 1 -- start start 2024-04-17T09:53:24.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.595+0000 7ffb4be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb4406ba30 0x7ffb4406be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:24.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.595+0000 7ffb4be00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ffb440cf370 0x7ffb440cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:24.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.595+0000 7ffb4be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb44160350 0x7ffb44160730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:24.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.595+0000 7ffb4be00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ffb4406c350 con 0x7ffb4406ba30 2024-04-17T09:53:24.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.595+0000 7ffb4be00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ffb440d0130 con 0x7ffb440cf370 2024-04-17T09:53:24.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.595+0000 7ffb4be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffb4415d090 con 0x7ffb44160350 2024-04-17T09:53:24.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.595+0000 7ffb49400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb4406ba30 0x7ffb4406be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:24.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.595+0000 7ffb49400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb4406ba30 0x7ffb4406be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49246/0 (socket says 172.21.15.32:49246) 2024-04-17T09:53:24.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.595+0000 7ffb49400640 1 -- 172.21.15.32:0/2829955978 learned_addr learned my addr 172.21.15.32:0/2829955978 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:24.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.595+0000 7ffb48a00640 1 --2- 172.21.15.32:0/2829955978 >> v2:172.21.15.144:3300/0 conn(0x7ffb440cf370 0x7ffb440cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:24.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.595+0000 7ffb49e00640 1 --2- 172.21.15.32:0/2829955978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb44160350 0x7ffb44160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:24.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.595+0000 7ffb49400640 1 -- 172.21.15.32:0/2829955978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb44160350 msgr2=0x7ffb44160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:24.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.595+0000 7ffb49400640 1 --2- 172.21.15.32:0/2829955978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb44160350 0x7ffb44160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.595+0000 7ffb49400640 1 -- 172.21.15.32:0/2829955978 >> v2:172.21.15.144:3300/0 conn(0x7ffb440cf370 msgr2=0x7ffb440cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:24.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.595+0000 7ffb49400640 1 --2- 172.21.15.32:0/2829955978 >> v2:172.21.15.144:3300/0 conn(0x7ffb440cf370 0x7ffb440cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.595+0000 7ffb49400640 1 -- 172.21.15.32:0/2829955978 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4415d370 con 0x7ffb4406ba30 2024-04-17T09:53:24.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.599+0000 7ffb49400640 1 --2- 172.21.15.32:0/2829955978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb4406ba30 0x7ffb4406be10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffb440afec0 tx=0x7ffb3801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8c1df113d3aba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:24.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.599+0000 7ffb43e00640 1 -- 172.21.15.32:0/2829955978 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb380aa430 con 0x7ffb4406ba30 2024-04-17T09:53:24.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.599+0000 7ffb43e00640 1 -- 172.21.15.32:0/2829955978 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb380af070 con 0x7ffb4406ba30 2024-04-17T09:53:24.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.599+0000 7ffb43e00640 1 -- 172.21.15.32:0/2829955978 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb380aa7f0 con 0x7ffb4406ba30 2024-04-17T09:53:24.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.599+0000 7ffb4be00640 1 -- 172.21.15.32:0/2829955978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb4406ba30 msgr2=0x7ffb4406be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:24.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.599+0000 7ffb4be00640 1 --2- 172.21.15.32:0/2829955978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb4406ba30 0x7ffb4406be10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffb440afec0 tx=0x7ffb3801c4f0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.599+0000 7ffb4be00640 1 -- 172.21.15.32:0/2829955978 shutdown_connections 2024-04-17T09:53:24.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.599+0000 7ffb4be00640 1 --2- 172.21.15.32:0/2829955978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb44160350 0x7ffb44160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.599+0000 7ffb4be00640 1 --2- 172.21.15.32:0/2829955978 >> v2:172.21.15.144:3300/0 conn(0x7ffb440cf370 0x7ffb440cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.599+0000 7ffb4be00640 1 --2- 172.21.15.32:0/2829955978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb4406ba30 0x7ffb4406be10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.599+0000 7ffb4be00640 1 -- 172.21.15.32:0/2829955978 >> 172.21.15.32:0/2829955978 conn(0x7ffb440b2120 msgr2=0x7ffb440b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:24.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.599+0000 7ffb4be00640 1 -- 172.21.15.32:0/2829955978 shutdown_connections 2024-04-17T09:53:24.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.599+0000 7ffb4be00640 1 -- 172.21.15.32:0/2829955978 wait complete. 2024-04-17T09:53:24.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.599+0000 7ffb4be00640 1 Processor -- start 2024-04-17T09:53:24.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.599+0000 7ffb4be00640 1 -- start start 2024-04-17T09:53:24.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.599+0000 7ffb4be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4406ba30 0x7ffb4406f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:24.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.599+0000 7ffb4be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb440cf370 0x7ffb441408c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:24.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.599+0000 7ffb4be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb44160350 0x7ffb4413a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:24.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.599+0000 7ffb4be00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffb440cfea0 con 0x7ffb440cf370 2024-04-17T09:53:24.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.599+0000 7ffb4be00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffb440cfd20 con 0x7ffb44160350 2024-04-17T09:53:24.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.599+0000 7ffb4be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffb4415db70 con 0x7ffb4406ba30 2024-04-17T09:53:24.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.603+0000 7ffb48a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb440cf370 0x7ffb441408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:24.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.603+0000 7ffb48a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb440cf370 0x7ffb441408c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49260/0 (socket says 172.21.15.32:49260) 2024-04-17T09:53:24.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.603+0000 7ffb48a00640 1 -- 172.21.15.32:0/2296607408 learned_addr learned my addr 172.21.15.32:0/2296607408 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.603+0000 7ffb49e00640 1 --2- 172.21.15.32:0/2296607408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb44160350 0x7ffb4413a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.603+0000 7ffb49400640 1 --2- 172.21.15.32:0/2296607408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4406ba30 0x7ffb4406f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.603+0000 7ffb48a00640 1 -- 172.21.15.32:0/2296607408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4406ba30 msgr2=0x7ffb4406f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.603+0000 7ffb48a00640 1 --2- 172.21.15.32:0/2296607408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4406ba30 0x7ffb4406f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.603+0000 7ffb48a00640 1 -- 172.21.15.32:0/2296607408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb44160350 msgr2=0x7ffb4413a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.603+0000 7ffb48a00640 1 --2- 172.21.15.32:0/2296607408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb44160350 0x7ffb4413a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.603+0000 7ffb48a00640 1 -- 172.21.15.32:0/2296607408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4413eba0 con 0x7ffb440cf370 2024-04-17T09:53:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.603+0000 7ffb48a00640 1 --2- 172.21.15.32:0/2296607408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb440cf370 0x7ffb441408c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000e8b0 tx=0x7ffb3000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.603+0000 7ffb41600640 1 -- 172.21.15.32:0/2296607408 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30017070 con 0x7ffb440cf370 2024-04-17T09:53:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.603+0000 7ffb41600640 1 -- 172.21.15.32:0/2296607408 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb3000c940 con 0x7ffb440cf370 2024-04-17T09:53:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.603+0000 7ffb4be00640 1 -- 172.21.15.32:0/2296607408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb4413ee80 con 0x7ffb440cf370 2024-04-17T09:53:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.603+0000 7ffb41600640 1 -- 172.21.15.32:0/2296607408 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3000cd10 con 0x7ffb440cf370 2024-04-17T09:53:24.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.603+0000 7ffb4be00640 1 -- 172.21.15.32:0/2296607408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb4413f440 con 0x7ffb440cf370 2024-04-17T09:53:24.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.603+0000 7ffb4be00640 1 -- 172.21.15.32:0/2296607408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb44068a90 con 0x7ffb440cf370 2024-04-17T09:53:24.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.603+0000 7ffb41600640 1 -- 172.21.15.32:0/2296607408 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ffb3001a070 con 0x7ffb440cf370 2024-04-17T09:53:24.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.603+0000 7ffb41600640 1 --2- 172.21.15.32:0/2296607408 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffb24068380 0x7ffb2406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:24.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.603+0000 7ffb41600640 1 -- 172.21.15.32:0/2296607408 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ffb30086d70 con 0x7ffb440cf370 2024-04-17T09:53:24.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.607+0000 7ffb49400640 1 --2- 172.21.15.32:0/2296607408 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffb24068380 0x7ffb2406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:24.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.607+0000 7ffb49400640 1 --2- 172.21.15.32:0/2296607408 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffb24068380 0x7ffb2406a840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffb380ae000 tx=0x7ffb380a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:24.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.611+0000 7ffb41600640 1 -- 172.21.15.32:0/2296607408 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ffb30057320 con 0x7ffb440cf370 2024-04-17T09:53:24.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.771+0000 7ffb4be00640 1 -- 172.21.15.32:0/2296607408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffb4406c850 con 0x7ffb440cf370 2024-04-17T09:53:24.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.771+0000 7ffb41600640 1 -- 172.21.15.32:0/2296607408 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7ffb30056c20 con 0x7ffb440cf370 2024-04-17T09:53:24.776 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:53:24.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.775+0000 7ffb4be00640 1 -- 172.21.15.32:0/2296607408 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffb24068380 msgr2=0x7ffb2406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:24.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.775+0000 7ffb4be00640 1 --2- 172.21.15.32:0/2296607408 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffb24068380 0x7ffb2406a840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffb380ae000 tx=0x7ffb380a7000 comp rx=0 tx=0).stop 2024-04-17T09:53:24.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.775+0000 7ffb4be00640 1 -- 172.21.15.32:0/2296607408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb440cf370 msgr2=0x7ffb441408c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:24.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.775+0000 7ffb4be00640 1 --2- 172.21.15.32:0/2296607408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb440cf370 0x7ffb441408c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000e8b0 tx=0x7ffb3000ed70 comp rx=0 tx=0).stop 2024-04-17T09:53:24.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.775+0000 7ffb4be00640 1 -- 172.21.15.32:0/2296607408 shutdown_connections 2024-04-17T09:53:24.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.775+0000 7ffb4be00640 1 --2- 172.21.15.32:0/2296607408 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffb24068380 0x7ffb2406a840 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.775+0000 7ffb4be00640 1 --2- 172.21.15.32:0/2296607408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb44160350 0x7ffb4413a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.775+0000 7ffb4be00640 1 --2- 172.21.15.32:0/2296607408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb440cf370 0x7ffb441408c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.775+0000 7ffb4be00640 1 --2- 172.21.15.32:0/2296607408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4406ba30 0x7ffb4406f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.775+0000 7ffb4be00640 1 -- 172.21.15.32:0/2296607408 >> 172.21.15.32:0/2296607408 conn(0x7ffb440b2120 msgr2=0x7ffb440cdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:24.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.775+0000 7ffb4be00640 1 -- 172.21.15.32:0/2296607408 shutdown_connections 2024-04-17T09:53:24.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.775+0000 7ffb4be00640 1 -- 172.21.15.32:0/2296607408 wait complete. 2024-04-17T09:53:24.797 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:53:24.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.935+0000 7fafce200640 1 Processor -- start 2024-04-17T09:53:24.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.935+0000 7fafce200640 1 -- start start 2024-04-17T09:53:24.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.935+0000 7fafce200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fafc80ceb20 0x7fafc8169880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:24.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.935+0000 7fafce200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafc8162620 con 0x7fafc80b2430 2024-04-17T09:53:24.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.935+0000 7fafce200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafc8162900 con 0x7fafc80d0890 2024-04-17T09:53:24.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.935+0000 7fafce200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fafc8162be0 con 0x7fafc80ceb20 2024-04-17T09:53:24.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.935+0000 7fafc7400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fafc80b2430 0x7fafc80b2810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48838/0 (socket says 172.21.15.32:48838) 2024-04-17T09:53:24.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.935+0000 7fafc7400640 1 -- 172.21.15.32:0/233213420 learned_addr learned my addr 172.21.15.32:0/233213420 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:24.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.935+0000 7fafc7e00640 1 --2- 172.21.15.32:0/233213420 >> v2:172.21.15.32:3300/0 conn(0x7fafc80ceb20 0x7fafc8169880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:24.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.935+0000 7fafc6a00640 1 -- 172.21.15.32:0/233213420 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2215068406 0 0) 0x7fafc8162620 con 0x7fafc80b2430 2024-04-17T09:53:24.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.935+0000 7fafc7e00640 1 -- 172.21.15.32:0/233213420 >> v1:172.21.15.144:6789/0 conn(0x7fafc80d0890 legacy=0x7fafc80d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:24.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.935+0000 7fafc7e00640 1 -- 172.21.15.32:0/233213420 >> v1:172.21.15.32:6789/0 conn(0x7fafc80b2430 legacy=0x7fafc80b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:24.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.935+0000 7fafc7e00640 1 -- 172.21.15.32:0/233213420 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc8162ec0 con 0x7fafc80ceb20 2024-04-17T09:53:24.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.939+0000 7fafc7e00640 1 --2- 172.21.15.32:0/233213420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafc80ceb20 0x7fafc8169880 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fafc00097c0 tx=0x7fafc001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05fe19e4c2eb98c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:24.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.939+0000 7fafc6a00640 1 -- 172.21.15.32:0/233213420 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc00b0070 con 0x7fafc80ceb20 2024-04-17T09:53:24.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.939+0000 7fafc6a00640 1 -- 172.21.15.32:0/233213420 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc00ab600 con 0x7fafc80ceb20 2024-04-17T09:53:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.939+0000 7fafc6a00640 1 -- 172.21.15.32:0/233213420 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc00ab9d0 con 0x7fafc80ceb20 2024-04-17T09:53:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.939+0000 7fafce200640 1 -- 172.21.15.32:0/233213420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafc80ceb20 msgr2=0x7fafc8169880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.939+0000 7fafce200640 1 --2- 172.21.15.32:0/233213420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafc80ceb20 0x7fafc8169880 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fafc00097c0 tx=0x7fafc001c380 comp rx=0 tx=0).stop 2024-04-17T09:53:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.939+0000 7fafce200640 1 -- 172.21.15.32:0/233213420 shutdown_connections 2024-04-17T09:53:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.939+0000 7fafce200640 1 --2- 172.21.15.32:0/233213420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafc80ceb20 0x7fafc8169880 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.939+0000 7fafce200640 1 -- 172.21.15.32:0/233213420 >> 172.21.15.32:0/233213420 conn(0x7fafc809a9f0 msgr2=0x7fafc80b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:24.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.939+0000 7fafce200640 1 -- 172.21.15.32:0/233213420 shutdown_connections 2024-04-17T09:53:24.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.939+0000 7fafce200640 1 -- 172.21.15.32:0/233213420 wait complete. 2024-04-17T09:53:24.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafce200640 1 Processor -- start 2024-04-17T09:53:24.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafce200640 1 -- start start 2024-04-17T09:53:24.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafce200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafc80b2430 0x7fafc813ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:24.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafce200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafc80ceb20 0x7fafc81521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:24.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafce200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafc80d0890 0x7fafc8160880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:24.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafce200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fafc816cf90 con 0x7fafc80d0890 2024-04-17T09:53:24.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafce200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fafc816ce10 con 0x7fafc80ceb20 2024-04-17T09:53:24.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafce200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fafc816d110 con 0x7fafc80b2430 2024-04-17T09:53:24.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafc7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafc80d0890 0x7fafc8160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:24.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafc7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafc80d0890 0x7fafc8160880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49286/0 (socket says 172.21.15.32:49286) 2024-04-17T09:53:24.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafc7e00640 1 -- 172.21.15.32:0/3824305671 learned_addr learned my addr 172.21.15.32:0/3824305671 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:24.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafbea00640 1 --2- 172.21.15.32:0/3824305671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafc80ceb20 0x7fafc81521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:24.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafc7400640 1 --2- 172.21.15.32:0/3824305671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafc80b2430 0x7fafc813ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:24.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafc7e00640 1 -- 172.21.15.32:0/3824305671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafc80b2430 msgr2=0x7fafc813ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:24.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafc7e00640 1 --2- 172.21.15.32:0/3824305671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafc80b2430 0x7fafc813ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafc7e00640 1 -- 172.21.15.32:0/3824305671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafc80ceb20 msgr2=0x7fafc81521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:24.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafc7e00640 1 --2- 172.21.15.32:0/3824305671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafc80ceb20 0x7fafc81521c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:24.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafc7e00640 1 -- 172.21.15.32:0/3824305671 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc813f240 con 0x7fafc80d0890 2024-04-17T09:53:24.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafc7e00640 1 --2- 172.21.15.32:0/3824305671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafc80d0890 0x7fafc8160880 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fafc00ae040 tx=0x7fafc001abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:24.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafbfe00640 1 -- 172.21.15.32:0/3824305671 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc00b0050 con 0x7fafc80d0890 2024-04-17T09:53:24.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafbfe00640 1 -- 172.21.15.32:0/3824305671 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc00bc070 con 0x7fafc80d0890 2024-04-17T09:53:24.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafce200640 1 -- 172.21.15.32:0/3824305671 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc8152700 con 0x7fafc80d0890 2024-04-17T09:53:24.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafbfe00640 1 -- 172.21.15.32:0/3824305671 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc00b5830 con 0x7fafc80d0890 2024-04-17T09:53:24.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafce200640 1 -- 172.21.15.32:0/3824305671 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc8158110 con 0x7fafc80d0890 2024-04-17T09:53:24.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.943+0000 7fafce200640 1 -- 172.21.15.32:0/3824305671 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc8068a90 con 0x7fafc80d0890 2024-04-17T09:53:24.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.947+0000 7fafbfe00640 1 -- 172.21.15.32:0/3824305671 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fafc00b59d0 con 0x7fafc80d0890 2024-04-17T09:53:24.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.947+0000 7fafbfe00640 1 --2- 172.21.15.32:0/3824305671 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fafa4068380 0x7fafa406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:24.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.947+0000 7fafbfe00640 1 -- 172.21.15.32:0/3824305671 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fafc01210e0 con 0x7fafc80d0890 2024-04-17T09:53:24.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.947+0000 7fafc7400640 1 --2- 172.21.15.32:0/3824305671 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fafa4068380 0x7fafa406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:24.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.947+0000 7fafc7400640 1 --2- 172.21.15.32:0/3824305671 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fafa4068380 0x7fafa406a840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fafb8001750 tx=0x7fafb80016a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:24.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:24.951+0000 7fafbfe00640 1 -- 172.21.15.32:0/3824305671 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fafc8152700 con 0x7fafc80d0890 2024-04-17T09:53:25.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.151+0000 7fafce200640 1 -- 172.21.15.32:0/3824305671 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafc809af00 con 0x7fafc80d0890 2024-04-17T09:53:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.151+0000 7fafbfe00640 1 -- 172.21.15.32:0/3824305671 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5193 (secure 0 0 0) 0x7fafc813f240 con 0x7fafc80d0890 2024-04-17T09:53:25.156 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:25.156 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/441201886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":441201886},{"type":"v1","addr":"172.21.15.144:6834","nonce":441201886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1066277472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1066277472},{"type":"v1","addr":"172.21.15.32:6836","nonce":1066277472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/201410023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":201410023},{"type":"v1","addr":"172.21.15.32:6837","nonce":201410023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:53:13.172187+0000","modified":"2024-04-17T09:53:16.188325+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/3332924005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3332924005},{"type":"v1","addr":"172.21.15.144:6835","nonce":3332924005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-17T09:53:25.157 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2024-04-17T09:53:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.155+0000 7fafce200640 1 -- 172.21.15.32:0/3824305671 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fafa4068380 msgr2=0x7fafa406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.155+0000 7fafce200640 1 --2- 172.21.15.32:0/3824305671 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fafa4068380 0x7fafa406a840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fafb8001750 tx=0x7fafb80016a0 comp rx=0 tx=0).stop 2024-04-17T09:53:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.155+0000 7fafce200640 1 -- 172.21.15.32:0/3824305671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafc80d0890 msgr2=0x7fafc8160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.155+0000 7fafce200640 1 --2- 172.21.15.32:0/3824305671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafc80d0890 0x7fafc8160880 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fafc00ae040 tx=0x7fafc001abc0 comp rx=0 tx=0).stop 2024-04-17T09:53:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.155+0000 7fafce200640 1 -- 172.21.15.32:0/3824305671 shutdown_connections 2024-04-17T09:53:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.155+0000 7fafce200640 1 --2- 172.21.15.32:0/3824305671 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fafa4068380 0x7fafa406a840 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.155+0000 7fafce200640 1 --2- 172.21.15.32:0/3824305671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafc80d0890 0x7fafc8160880 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.155+0000 7fafce200640 1 --2- 172.21.15.32:0/3824305671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafc80ceb20 0x7fafc81521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.155+0000 7fafce200640 1 --2- 172.21.15.32:0/3824305671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafc80b2430 0x7fafc813ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.155+0000 7fafce200640 1 -- 172.21.15.32:0/3824305671 >> 172.21.15.32:0/3824305671 conn(0x7fafc809a9f0 msgr2=0x7fafc815cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.155+0000 7fafce200640 1 -- 172.21.15.32:0/3824305671 shutdown_connections 2024-04-17T09:53:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.155+0000 7fafce200640 1 -- 172.21.15.32:0/3824305671 wait complete. 2024-04-17T09:53:25.178 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:53:25.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.315+0000 7f469e800640 1 Processor -- start 2024-04-17T09:53:25.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.315+0000 7f469e800640 1 -- start start 2024-04-17T09:53:25.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.315+0000 7f469e800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46980d05a0 0x7f469815ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:25.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.315+0000 7f469e800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f469806e390 con 0x7f4698159010 2024-04-17T09:53:25.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.315+0000 7f469e800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f469806e670 con 0x7f46980b4200 2024-04-17T09:53:25.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.315+0000 7f469e800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f469806e950 con 0x7f46980d05a0 2024-04-17T09:53:25.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.315+0000 7f468ea00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46980d05a0 0x7f469815ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:25.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.315+0000 7f4697400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4698159010 0x7f46980d0bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48842/0 (socket says 172.21.15.32:48842) 2024-04-17T09:53:25.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.315+0000 7f4697400640 1 -- 172.21.15.32:0/3376793090 learned_addr learned my addr 172.21.15.32:0/3376793090 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:25.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.319+0000 7f468ea00640 1 -- 172.21.15.32:0/3376793090 >> v1:172.21.15.144:6789/0 conn(0x7f46980b4200 legacy=0x7f469806f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:25.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.319+0000 7f468ea00640 1 -- 172.21.15.32:0/3376793090 >> v1:172.21.15.32:6789/0 conn(0x7f4698159010 legacy=0x7f46980d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:25.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.319+0000 7f468ea00640 1 -- 172.21.15.32:0/3376793090 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469806ec30 con 0x7f46980d05a0 2024-04-17T09:53:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.319+0000 7f468ea00640 1 --2- 172.21.15.32:0/3376793090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46980d05a0 0x7f469815ff30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f46800097c0 tx=0x7f468001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc42f6686f762621 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.319+0000 7f4696a00640 1 -- 172.21.15.32:0/3376793090 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46800b0070 con 0x7f46980d05a0 2024-04-17T09:53:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.319+0000 7f4696a00640 1 -- 172.21.15.32:0/3376793090 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46800ab600 con 0x7f46980d05a0 2024-04-17T09:53:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.319+0000 7f4696a00640 1 -- 172.21.15.32:0/3376793090 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46800ab9d0 con 0x7f46980d05a0 2024-04-17T09:53:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.319+0000 7f469e800640 1 -- 172.21.15.32:0/3376793090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46980d05a0 msgr2=0x7f469815ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.319+0000 7f469e800640 1 --2- 172.21.15.32:0/3376793090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46980d05a0 0x7f469815ff30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f46800097c0 tx=0x7f468001c410 comp rx=0 tx=0).stop 2024-04-17T09:53:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f469e800640 1 -- 172.21.15.32:0/3376793090 shutdown_connections 2024-04-17T09:53:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f469e800640 1 --2- 172.21.15.32:0/3376793090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46980d05a0 0x7f469815ff30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f469e800640 1 -- 172.21.15.32:0/3376793090 >> 172.21.15.32:0/3376793090 conn(0x7f469806ba30 msgr2=0x7f469806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f469e800640 1 -- 172.21.15.32:0/3376793090 shutdown_connections 2024-04-17T09:53:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f469e800640 1 -- 172.21.15.32:0/3376793090 wait complete. 2024-04-17T09:53:25.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f469e800640 1 Processor -- start 2024-04-17T09:53:25.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f469e800640 1 -- start start 2024-04-17T09:53:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f469e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46980b4200 0x7f4698140a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f469e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46980d05a0 0x7f4698140fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f469e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4698159010 0x7f469813e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f469e800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f469816cd50 con 0x7f46980d05a0 2024-04-17T09:53:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f469e800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f469816cbd0 con 0x7f46980b4200 2024-04-17T09:53:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f469e800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f469816ced0 con 0x7f4698159010 2024-04-17T09:53:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f468ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46980d05a0 0x7f4698140fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f468ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46980d05a0 0x7f4698140fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49314/0 (socket says 172.21.15.32:49314) 2024-04-17T09:53:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f468ea00640 1 -- 172.21.15.32:0/3196735286 learned_addr learned my addr 172.21.15.32:0/3196735286 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f4697400640 1 --2- 172.21.15.32:0/3196735286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46980b4200 0x7f4698140a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f4697e00640 1 --2- 172.21.15.32:0/3196735286 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4698159010 0x7f469813e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f468ea00640 1 -- 172.21.15.32:0/3196735286 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4698159010 msgr2=0x7f469813e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f468ea00640 1 --2- 172.21.15.32:0/3196735286 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4698159010 0x7f469813e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f468ea00640 1 -- 172.21.15.32:0/3196735286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46980b4200 msgr2=0x7f4698140a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f468ea00640 1 --2- 172.21.15.32:0/3196735286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46980b4200 0x7f4698140a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f468ea00640 1 -- 172.21.15.32:0/3196735286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4698146490 con 0x7f46980d05a0 2024-04-17T09:53:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f468ea00640 1 --2- 172.21.15.32:0/3196735286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46980d05a0 0x7f4698140fd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f46800097c0 tx=0x7f468001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f468fe00640 1 -- 172.21.15.32:0/3196735286 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46800b0050 con 0x7f46980d05a0 2024-04-17T09:53:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f468fe00640 1 -- 172.21.15.32:0/3196735286 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46800bc070 con 0x7f46980d05a0 2024-04-17T09:53:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f468fe00640 1 -- 172.21.15.32:0/3196735286 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46800b5830 con 0x7f46980d05a0 2024-04-17T09:53:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.323+0000 7f469e800640 1 -- 172.21.15.32:0/3196735286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4698146770 con 0x7f46980d05a0 2024-04-17T09:53:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.327+0000 7f469e800640 1 -- 172.21.15.32:0/3196735286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4698146d30 con 0x7f46980d05a0 2024-04-17T09:53:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.327+0000 7f469e800640 1 -- 172.21.15.32:0/3196735286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4698068a90 con 0x7f46980d05a0 2024-04-17T09:53:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.327+0000 7f468fe00640 1 -- 172.21.15.32:0/3196735286 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f46800b59d0 con 0x7f46980d05a0 2024-04-17T09:53:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.327+0000 7f468fe00640 1 --2- 172.21.15.32:0/3196735286 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f46740683b0 0x7f467406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.327+0000 7f468fe00640 1 -- 172.21.15.32:0/3196735286 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4680121210 con 0x7f46980d05a0 2024-04-17T09:53:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.327+0000 7f4697400640 1 --2- 172.21.15.32:0/3196735286 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f46740683b0 0x7f467406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:25.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.327+0000 7f4697400640 1 --2- 172.21.15.32:0/3196735286 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f46740683b0 0x7f467406a870 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4688005890 tx=0x7f4688009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:25.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.331+0000 7f468fe00640 1 -- 172.21.15.32:0/3196735286 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f4680016090 con 0x7f46980d05a0 2024-04-17T09:53:25.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.495+0000 7f469e800640 1 -- 172.21.15.32:0/3196735286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f46980cfb40 con 0x7f46980d05a0 2024-04-17T09:53:25.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.495+0000 7f468fe00640 1 -- 172.21.15.32:0/3196735286 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f46800f1730 con 0x7f46980d05a0 2024-04-17T09:53:25.500 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:25.500 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:14.779535+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:53:25.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.499+0000 7f469e800640 1 -- 172.21.15.32:0/3196735286 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f46740683b0 msgr2=0x7f467406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:25.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.499+0000 7f469e800640 1 --2- 172.21.15.32:0/3196735286 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f46740683b0 0x7f467406a870 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4688005890 tx=0x7f4688009040 comp rx=0 tx=0).stop 2024-04-17T09:53:25.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.499+0000 7f469e800640 1 -- 172.21.15.32:0/3196735286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46980d05a0 msgr2=0x7f4698140fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:25.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.499+0000 7f469e800640 1 --2- 172.21.15.32:0/3196735286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46980d05a0 0x7f4698140fd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f46800097c0 tx=0x7f468001ac50 comp rx=0 tx=0).stop 2024-04-17T09:53:25.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.499+0000 7f469e800640 1 -- 172.21.15.32:0/3196735286 shutdown_connections 2024-04-17T09:53:25.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.499+0000 7f469e800640 1 --2- 172.21.15.32:0/3196735286 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f46740683b0 0x7f467406a870 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:25.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.499+0000 7f469e800640 1 --2- 172.21.15.32:0/3196735286 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4698159010 0x7f469813e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:25.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.499+0000 7f469e800640 1 --2- 172.21.15.32:0/3196735286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46980d05a0 0x7f4698140fd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:25.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.499+0000 7f469e800640 1 --2- 172.21.15.32:0/3196735286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46980b4200 0x7f4698140a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:25.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.499+0000 7f469e800640 1 -- 172.21.15.32:0/3196735286 >> 172.21.15.32:0/3196735286 conn(0x7f469806ba30 msgr2=0x7f46981486d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:25.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.499+0000 7f469e800640 1 -- 172.21.15.32:0/3196735286 shutdown_connections 2024-04-17T09:53:25.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.499+0000 7f469e800640 1 -- 172.21.15.32:0/3196735286 wait complete. 2024-04-17T09:53:25.522 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:53:11.198098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:53:12.118982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:53:25.522 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:53:25.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.663+0000 7f5bb2000640 1 Processor -- start 2024-04-17T09:53:25.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.663+0000 7f5bb2000640 1 -- start start 2024-04-17T09:53:25.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.663+0000 7f5bb2000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5bac15b7d0 0x7f5bac06e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:25.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.663+0000 7f5bb2000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bac06f7c0 0x7f5bac06ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:25.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.663+0000 7f5bb2000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bac0cc940 con 0x7f5bac06f270 2024-04-17T09:53:25.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.663+0000 7f5bb2000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5bac0ccc20 con 0x7f5bac15b7d0 2024-04-17T09:53:25.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.663+0000 7f5bb2000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5bac06e0b0 con 0x7f5bac06f7c0 2024-04-17T09:53:25.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.663+0000 7f5bab400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5bac15b7d0 0x7f5bac06e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:25.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.663+0000 7f5babe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5bac06f270 0x7f5bac0ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48854/0 (socket says 172.21.15.32:48854) 2024-04-17T09:53:25.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.663+0000 7f5babe00640 1 -- 172.21.15.32:0/2022757971 learned_addr learned my addr 172.21.15.32:0/2022757971 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:25.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.663+0000 7f5ba2a00640 1 --2- 172.21.15.32:0/2022757971 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bac06f7c0 0x7f5bac06ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:25.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.663+0000 7f5bab400640 1 -- 172.21.15.32:0/2022757971 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bac06f7c0 msgr2=0x7f5bac06ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:25.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.663+0000 7f5bab400640 1 --2- 172.21.15.32:0/2022757971 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bac06f7c0 0x7f5bac06ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:25.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.663+0000 7f5bab400640 1 -- 172.21.15.32:0/2022757971 >> v1:172.21.15.32:6789/0 conn(0x7f5bac06f270 legacy=0x7f5bac0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:25.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.663+0000 7f5bab400640 1 -- 172.21.15.32:0/2022757971 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bac06e390 con 0x7f5bac15b7d0 2024-04-17T09:53:25.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.667+0000 7f5bab400640 1 --2- 172.21.15.32:0/2022757971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bac15b7d0 0x7f5bac06e8b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c0097c0 tx=0x7f5b9c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc409d5afb36e230 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:25.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.667+0000 7f5baaa00640 1 -- 172.21.15.32:0/2022757971 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c0ab430 con 0x7f5bac15b7d0 2024-04-17T09:53:25.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.667+0000 7f5baaa00640 1 -- 172.21.15.32:0/2022757971 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b9c0b0070 con 0x7f5bac15b7d0 2024-04-17T09:53:25.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.667+0000 7f5baaa00640 1 -- 172.21.15.32:0/2022757971 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c0ab7f0 con 0x7f5bac15b7d0 2024-04-17T09:53:25.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.667+0000 7f5bb2000640 1 -- 172.21.15.32:0/2022757971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bac15b7d0 msgr2=0x7f5bac06e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:25.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.667+0000 7f5bb2000640 1 --2- 172.21.15.32:0/2022757971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bac15b7d0 0x7f5bac06e8b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c0097c0 tx=0x7f5b9c01c410 comp rx=0 tx=0).stop 2024-04-17T09:53:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.667+0000 7f5bb2000640 1 -- 172.21.15.32:0/2022757971 shutdown_connections 2024-04-17T09:53:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.667+0000 7f5bb2000640 1 --2- 172.21.15.32:0/2022757971 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bac06f7c0 0x7f5bac06ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.667+0000 7f5bb2000640 1 --2- 172.21.15.32:0/2022757971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bac15b7d0 0x7f5bac06e8b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.667+0000 7f5bb2000640 1 -- 172.21.15.32:0/2022757971 >> 172.21.15.32:0/2022757971 conn(0x7f5bac0b2120 msgr2=0x7f5bac0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.667+0000 7f5bb2000640 1 -- 172.21.15.32:0/2022757971 shutdown_connections 2024-04-17T09:53:25.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.667+0000 7f5bb2000640 1 -- 172.21.15.32:0/2022757971 wait complete. 2024-04-17T09:53:25.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5bb2000640 1 Processor -- start 2024-04-17T09:53:25.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5bb2000640 1 -- start start 2024-04-17T09:53:25.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5bb2000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bac06f270 0x7f5bac0ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:25.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5bb2000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bac06f7c0 0x7f5bac155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:25.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5bb2000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bac15b7d0 0x7f5bac15a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:25.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5bb2000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac168ab0 con 0x7f5bac06f7c0 2024-04-17T09:53:25.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5bb2000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac06c380 con 0x7f5bac06f270 2024-04-17T09:53:25.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5bb2000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5bac168c30 con 0x7f5bac15b7d0 2024-04-17T09:53:25.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5ba2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bac06f7c0 0x7f5bac155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:25.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5ba2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bac06f7c0 0x7f5bac155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49320/0 (socket says 172.21.15.32:49320) 2024-04-17T09:53:25.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5ba2a00640 1 -- 172.21.15.32:0/3911753500 learned_addr learned my addr 172.21.15.32:0/3911753500 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:25.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5bab400640 1 --2- 172.21.15.32:0/3911753500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bac06f270 0x7f5bac0ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:25.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5babe00640 1 --2- 172.21.15.32:0/3911753500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bac15b7d0 0x7f5bac15a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:25.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5ba2a00640 1 -- 172.21.15.32:0/3911753500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bac15b7d0 msgr2=0x7f5bac15a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:25.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5ba2a00640 1 --2- 172.21.15.32:0/3911753500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bac15b7d0 0x7f5bac15a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:25.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5ba2a00640 1 -- 172.21.15.32:0/3911753500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bac06f270 msgr2=0x7f5bac0ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:25.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5ba2a00640 1 --2- 172.21.15.32:0/3911753500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bac06f270 0x7f5bac0ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:25.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5ba2a00640 1 -- 172.21.15.32:0/3911753500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bac0ccf00 con 0x7f5bac06f7c0 2024-04-17T09:53:25.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5babe00640 1 --2- 172.21.15.32:0/3911753500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bac15b7d0 0x7f5bac15a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:25.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5ba2a00640 1 --2- 172.21.15.32:0/3911753500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bac06f7c0 0x7f5bac155600 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400d800 tx=0x7f5b9400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:25.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5ba3e00640 1 -- 172.21.15.32:0/3911753500 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9400b6c0 con 0x7f5bac06f7c0 2024-04-17T09:53:25.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5ba3e00640 1 -- 172.21.15.32:0/3911753500 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b94019070 con 0x7f5bac06f7c0 2024-04-17T09:53:25.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5ba3e00640 1 -- 172.21.15.32:0/3911753500 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9400ba60 con 0x7f5bac06f7c0 2024-04-17T09:53:25.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5bb2000640 1 -- 172.21.15.32:0/3911753500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bac158880 con 0x7f5bac06f7c0 2024-04-17T09:53:25.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5bb2000640 1 -- 172.21.15.32:0/3911753500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bac154480 con 0x7f5bac06f7c0 2024-04-17T09:53:25.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.671+0000 7f5bb2000640 1 -- 172.21.15.32:0/3911753500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bac068a90 con 0x7f5bac06f7c0 2024-04-17T09:53:25.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.675+0000 7f5ba3e00640 1 -- 172.21.15.32:0/3911753500 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5b9400bc00 con 0x7f5bac06f7c0 2024-04-17T09:53:25.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.675+0000 7f5ba3e00640 1 --2- 172.21.15.32:0/3911753500 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5b88068380 0x7f5b8806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:25.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.675+0000 7f5ba3e00640 1 -- 172.21.15.32:0/3911753500 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5b94086e00 con 0x7f5bac06f7c0 2024-04-17T09:53:25.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.675+0000 7f5bab400640 1 --2- 172.21.15.32:0/3911753500 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5b88068380 0x7f5b8806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:25.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.675+0000 7f5bab400640 1 --2- 172.21.15.32:0/3911753500 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5b88068380 0x7f5b8806a840 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c0af000 tx=0x7f5b9c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:25.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.679+0000 7f5ba3e00640 1 -- 172.21.15.32:0/3911753500 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5b940573b0 con 0x7f5bac06f7c0 2024-04-17T09:53:25.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.847+0000 7f5bb2000640 1 -- 172.21.15.32:0/3911753500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5bac0cd170 con 0x7f5bac06f7c0 2024-04-17T09:53:25.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.847+0000 7f5ba3e00640 1 -- 172.21.15.32:0/3911753500 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f5b94056cb0 con 0x7f5bac06f7c0 2024-04-17T09:53:25.850 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:53:25.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.847+0000 7f5bb2000640 1 -- 172.21.15.32:0/3911753500 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5b88068380 msgr2=0x7f5b8806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:25.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.847+0000 7f5bb2000640 1 --2- 172.21.15.32:0/3911753500 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5b88068380 0x7f5b8806a840 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c0af000 tx=0x7f5b9c008040 comp rx=0 tx=0).stop 2024-04-17T09:53:25.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.847+0000 7f5bb2000640 1 -- 172.21.15.32:0/3911753500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bac06f7c0 msgr2=0x7f5bac155600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:25.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.847+0000 7f5bb2000640 1 --2- 172.21.15.32:0/3911753500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bac06f7c0 0x7f5bac155600 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400d800 tx=0x7f5b9400dcc0 comp rx=0 tx=0).stop 2024-04-17T09:53:25.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.847+0000 7f5bb2000640 1 -- 172.21.15.32:0/3911753500 shutdown_connections 2024-04-17T09:53:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.847+0000 7f5bb2000640 1 --2- 172.21.15.32:0/3911753500 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5b88068380 0x7f5b8806a840 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.847+0000 7f5bb2000640 1 --2- 172.21.15.32:0/3911753500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bac15b7d0 0x7f5bac15a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.847+0000 7f5bb2000640 1 --2- 172.21.15.32:0/3911753500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bac06f7c0 0x7f5bac155600 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.847+0000 7f5bb2000640 1 --2- 172.21.15.32:0/3911753500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bac06f270 0x7f5bac0ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.847+0000 7f5bb2000640 1 -- 172.21.15.32:0/3911753500 >> 172.21.15.32:0/3911753500 conn(0x7f5bac0b2120 msgr2=0x7f5bac0b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.847+0000 7f5bb2000640 1 -- 172.21.15.32:0/3911753500 shutdown_connections 2024-04-17T09:53:25.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:25.847+0000 7f5bb2000640 1 -- 172.21.15.32:0/3911753500 wait complete. 2024-04-17T09:53:25.872 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:53:26.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.011+0000 7f89b9800640 1 Processor -- start 2024-04-17T09:53:26.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.011+0000 7f89b9800640 1 -- start start 2024-04-17T09:53:26.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.011+0000 7f89b9800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89b415e880 0x7f89b40b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:26.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.011+0000 7f89b9800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89b40b0a30 0x7f89b406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:26.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.011+0000 7f89b9800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f89b406bf70 0x7f89b406ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:26.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.011+0000 7f89b9800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f89b406c3e0 con 0x7f89b40b0a30 2024-04-17T09:53:26.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.011+0000 7f89b9800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f89b40d11d0 con 0x7f89b406bf70 2024-04-17T09:53:26.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.011+0000 7f89b9800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f89b40d0690 con 0x7f89b415e880 2024-04-17T09:53:26.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.011+0000 7f89abe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89b40b0a30 0x7f89b406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:26.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.011+0000 7f89abe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89b40b0a30 0x7f89b406ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49330/0 (socket says 172.21.15.32:49330) 2024-04-17T09:53:26.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.011+0000 7f89abe00640 1 -- 172.21.15.32:0/130740272 learned_addr learned my addr 172.21.15.32:0/130740272 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:26.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.015+0000 7f89b3400640 1 --2- 172.21.15.32:0/130740272 >> v2:172.21.15.144:3300/0 conn(0x7f89b406bf70 0x7f89b406ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:26.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.015+0000 7f89b2a00640 1 --2- 172.21.15.32:0/130740272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89b415e880 0x7f89b40b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:26.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.015+0000 7f89abe00640 1 -- 172.21.15.32:0/130740272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89b415e880 msgr2=0x7f89b40b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:26.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.015+0000 7f89abe00640 1 --2- 172.21.15.32:0/130740272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89b415e880 0x7f89b40b04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.015+0000 7f89abe00640 1 -- 172.21.15.32:0/130740272 >> v2:172.21.15.144:3300/0 conn(0x7f89b406bf70 msgr2=0x7f89b406ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:26.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.015+0000 7f89abe00640 1 --2- 172.21.15.32:0/130740272 >> v2:172.21.15.144:3300/0 conn(0x7f89b406bf70 0x7f89b406ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.015+0000 7f89abe00640 1 -- 172.21.15.32:0/130740272 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b40d0930 con 0x7f89b40b0a30 2024-04-17T09:53:26.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.015+0000 7f89b3400640 1 --2- 172.21.15.32:0/130740272 >> v2:172.21.15.144:3300/0 conn(0x7f89b406bf70 0x7f89b406ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:26.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.015+0000 7f89b2a00640 1 --2- 172.21.15.32:0/130740272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89b415e880 0x7f89b40b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:26.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.015+0000 7f89abe00640 1 --2- 172.21.15.32:0/130740272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89b40b0a30 0x7f89b406ba30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f89b40732c0 tx=0x7f899c01c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1d84d6518548b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:26.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.015+0000 7f89b2000640 1 -- 172.21.15.32:0/130740272 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c0b0070 con 0x7f89b40b0a30 2024-04-17T09:53:26.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.015+0000 7f89b2000640 1 -- 172.21.15.32:0/130740272 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f899c0ab600 con 0x7f89b40b0a30 2024-04-17T09:53:26.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.015+0000 7f89b2000640 1 -- 172.21.15.32:0/130740272 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c0ab9d0 con 0x7f89b40b0a30 2024-04-17T09:53:26.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.015+0000 7f89b9800640 1 -- 172.21.15.32:0/130740272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89b40b0a30 msgr2=0x7f89b406ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:26.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.015+0000 7f89b9800640 1 --2- 172.21.15.32:0/130740272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89b40b0a30 0x7f89b406ba30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f89b40732c0 tx=0x7f899c01c1a0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b9800640 1 -- 172.21.15.32:0/130740272 shutdown_connections 2024-04-17T09:53:26.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b9800640 1 --2- 172.21.15.32:0/130740272 >> v2:172.21.15.144:3300/0 conn(0x7f89b406bf70 0x7f89b406ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b9800640 1 --2- 172.21.15.32:0/130740272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89b40b0a30 0x7f89b406ba30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b9800640 1 --2- 172.21.15.32:0/130740272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89b415e880 0x7f89b40b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b9800640 1 -- 172.21.15.32:0/130740272 >> 172.21.15.32:0/130740272 conn(0x7f89b409a9f0 msgr2=0x7f89b40b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:26.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b9800640 1 -- 172.21.15.32:0/130740272 shutdown_connections 2024-04-17T09:53:26.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b9800640 1 -- 172.21.15.32:0/130740272 wait complete. 2024-04-17T09:53:26.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b9800640 1 Processor -- start 2024-04-17T09:53:26.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b9800640 1 -- start start 2024-04-17T09:53:26.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b9800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89b406bf70 0x7f89b40cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:26.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b9800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89b40b0a30 0x7f89b415b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:26.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b9800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89b415e880 0x7f89b4157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:26.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b9800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f89b40cbaa0 con 0x7f89b415e880 2024-04-17T09:53:26.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b9800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f89b415ec60 con 0x7f89b40b0a30 2024-04-17T09:53:26.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b9800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f89b40cbc20 con 0x7f89b406bf70 2024-04-17T09:53:26.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89b415e880 0x7f89b4157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:26.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89b415e880 0x7f89b4157a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49344/0 (socket says 172.21.15.32:49344) 2024-04-17T09:53:26.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b3400640 1 -- 172.21.15.32:0/3420556969 learned_addr learned my addr 172.21.15.32:0/3420556969 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:26.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b2a00640 1 --2- 172.21.15.32:0/3420556969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89b406bf70 0x7f89b40cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:26.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89abe00640 1 --2- 172.21.15.32:0/3420556969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89b40b0a30 0x7f89b415b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:26.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b3400640 1 -- 172.21.15.32:0/3420556969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89b406bf70 msgr2=0x7f89b40cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:26.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b3400640 1 --2- 172.21.15.32:0/3420556969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89b406bf70 0x7f89b40cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b3400640 1 -- 172.21.15.32:0/3420556969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89b40b0a30 msgr2=0x7f89b415b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:26.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b3400640 1 --2- 172.21.15.32:0/3420556969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89b40b0a30 0x7f89b415b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b3400640 1 -- 172.21.15.32:0/3420556969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b40cc3e0 con 0x7f89b415e880 2024-04-17T09:53:26.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.019+0000 7f89b3400640 1 --2- 172.21.15.32:0/3420556969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89b415e880 0x7f89b4157a00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00cd80 tx=0x7f89ac00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:26.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.023+0000 7f89aaa00640 1 -- 172.21.15.32:0/3420556969 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac00f680 con 0x7f89b415e880 2024-04-17T09:53:26.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.023+0000 7f89aaa00640 1 -- 172.21.15.32:0/3420556969 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89ac00f840 con 0x7f89b415e880 2024-04-17T09:53:26.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.023+0000 7f89aaa00640 1 -- 172.21.15.32:0/3420556969 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac00fc30 con 0x7f89b415e880 2024-04-17T09:53:26.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.023+0000 7f89b9800640 1 -- 172.21.15.32:0/3420556969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b40cc6c0 con 0x7f89b415e880 2024-04-17T09:53:26.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.023+0000 7f89b9800640 1 -- 172.21.15.32:0/3420556969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b40ccc80 con 0x7f89b415e880 2024-04-17T09:53:26.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.023+0000 7f89b9800640 1 -- 172.21.15.32:0/3420556969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89b4068a90 con 0x7f89b415e880 2024-04-17T09:53:26.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.023+0000 7f89aaa00640 1 -- 172.21.15.32:0/3420556969 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f89ac00e3e0 con 0x7f89b415e880 2024-04-17T09:53:26.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.023+0000 7f89aaa00640 1 --2- 172.21.15.32:0/3420556969 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8990068380 0x7f899006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:26.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.023+0000 7f89aaa00640 1 -- 172.21.15.32:0/3420556969 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f89ac0876d0 con 0x7f89b415e880 2024-04-17T09:53:26.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.023+0000 7f89b2a00640 1 --2- 172.21.15.32:0/3420556969 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8990068380 0x7f899006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:26.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.023+0000 7f89b2a00640 1 --2- 172.21.15.32:0/3420556969 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8990068380 0x7f899006a840 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f89b41fa850 tx=0x7f89a4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:26.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.027+0000 7f89aaa00640 1 -- 172.21.15.32:0/3420556969 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f89ac014030 con 0x7f89b415e880 2024-04-17T09:53:26.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.191+0000 7f89b9800640 1 -- 172.21.15.32:0/3420556969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f89b415bd00 con 0x7f89b415e880 2024-04-17T09:53:26.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.191+0000 7f89aaa00640 1 -- 172.21.15.32:0/3420556969 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f89ac0575a0 con 0x7f89b415e880 2024-04-17T09:53:26.196 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:53:26.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.195+0000 7f89b9800640 1 -- 172.21.15.32:0/3420556969 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8990068380 msgr2=0x7f899006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:26.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.195+0000 7f89b9800640 1 --2- 172.21.15.32:0/3420556969 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8990068380 0x7f899006a840 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f89b41fa850 tx=0x7f89a4008040 comp rx=0 tx=0).stop 2024-04-17T09:53:26.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.195+0000 7f89b9800640 1 -- 172.21.15.32:0/3420556969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89b415e880 msgr2=0x7f89b4157a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:26.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.195+0000 7f89b9800640 1 --2- 172.21.15.32:0/3420556969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89b415e880 0x7f89b4157a00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00cd80 tx=0x7f89ac00a650 comp rx=0 tx=0).stop 2024-04-17T09:53:26.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.195+0000 7f89b9800640 1 -- 172.21.15.32:0/3420556969 shutdown_connections 2024-04-17T09:53:26.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.195+0000 7f89b9800640 1 --2- 172.21.15.32:0/3420556969 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8990068380 0x7f899006a840 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.195+0000 7f89b9800640 1 --2- 172.21.15.32:0/3420556969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89b415e880 0x7f89b4157a00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.195+0000 7f89b9800640 1 --2- 172.21.15.32:0/3420556969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89b40b0a30 0x7f89b415b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.195+0000 7f89b9800640 1 --2- 172.21.15.32:0/3420556969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89b406bf70 0x7f89b40cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.195+0000 7f89b9800640 1 -- 172.21.15.32:0/3420556969 >> 172.21.15.32:0/3420556969 conn(0x7f89b409a9f0 msgr2=0x7f89b40b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:26.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.195+0000 7f89b9800640 1 -- 172.21.15.32:0/3420556969 shutdown_connections 2024-04-17T09:53:26.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.195+0000 7f89b9800640 1 -- 172.21.15.32:0/3420556969 wait complete. 2024-04-17T09:53:26.216 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:53:26.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f925f600640 1 Processor -- start 2024-04-17T09:53:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f925f600640 1 -- start start 2024-04-17T09:53:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f925f600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f925815e880 0x7f92580b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f925f600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f925806c140 0x7f92580cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f925f600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92580d0690 con 0x7f92580b0a30 2024-04-17T09:53:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f925f600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f92580d11d0 con 0x7f925815e880 2024-04-17T09:53:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f925f600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f92580d0930 con 0x7f925806c140 2024-04-17T09:53:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f9257e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f92580b0a30 0x7f925806ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48860/0 (socket says 172.21.15.32:48860) 2024-04-17T09:53:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f9257e00640 1 -- 172.21.15.32:0/309951887 learned_addr learned my addr 172.21.15.32:0/309951887 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f925cc00640 1 --2- 172.21.15.32:0/309951887 >> v2:172.21.15.144:3300/0 conn(0x7f925815e880 0x7f92580b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f925d600640 1 --2- 172.21.15.32:0/309951887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f925806c140 0x7f92580cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f9257400640 1 -- 172.21.15.32:0/309951887 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 738441716 0 0) 0x7f92580d0690 con 0x7f92580b0a30 2024-04-17T09:53:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f9257400640 1 -- 172.21.15.32:0/309951887 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9240003530 con 0x7f92580b0a30 2024-04-17T09:53:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f9257400640 1 -- 172.21.15.32:0/309951887 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 778656025 0 0) 0x7f9240003530 con 0x7f92580b0a30 2024-04-17T09:53:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f9257400640 1 -- 172.21.15.32:0/309951887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f925806c140 msgr2=0x7f92580cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f9257400640 1 --2- 172.21.15.32:0/309951887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f925806c140 0x7f92580cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f9257400640 1 -- 172.21.15.32:0/309951887 >> v2:172.21.15.144:3300/0 conn(0x7f925815e880 msgr2=0x7f92580b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f9257400640 1 --2- 172.21.15.32:0/309951887 >> v2:172.21.15.144:3300/0 conn(0x7f925815e880 0x7f92580b04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f9257400640 1 -- 172.21.15.32:0/309951887 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92580d0c10 con 0x7f92580b0a30 2024-04-17T09:53:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f9257400640 1 -- 172.21.15.32:0/309951887 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f92440016a0 con 0x7f92580b0a30 2024-04-17T09:53:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f9257400640 1 -- 172.21.15.32:0/309951887 >> v1:172.21.15.32:6789/0 conn(0x7f92580b0a30 legacy=0x7f925806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f9257400640 1 --2- 172.21.15.32:0/309951887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92400039c0 0x7f9240003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f9257400640 1 --2- 172.21.15.32:0/309951887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f925806c140 0x7f9240004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f9257400640 1 --2- 172.21.15.32:0/309951887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9240004d50 0x7f92400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f9257400640 1 -- 172.21.15.32:0/309951887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f92580d11d0 con 0x7f92400039c0 2024-04-17T09:53:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f9257400640 1 -- 172.21.15.32:0/309951887 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f92580d0930 con 0x7f9240004d50 2024-04-17T09:53:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f9257400640 1 -- 172.21.15.32:0/309951887 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f924000ba80 con 0x7f925806c140 2024-04-17T09:53:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f925cc00640 1 --2- 172.21.15.32:0/309951887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92400039c0 0x7f9240003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f9257e00640 1 --2- 172.21.15.32:0/309951887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f925806c140 0x7f9240004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f925d600640 1 --2- 172.21.15.32:0/309951887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9240004d50 0x7f92400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f925cc00640 1 -- 172.21.15.32:0/309951887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f925806c140 msgr2=0x7f9240004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f925cc00640 1 --2- 172.21.15.32:0/309951887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f925806c140 0x7f9240004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f925cc00640 1 -- 172.21.15.32:0/309951887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9240004d50 msgr2=0x7f92400051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f925cc00640 1 --2- 172.21.15.32:0/309951887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9240004d50 0x7f92400051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f925cc00640 1 -- 172.21.15.32:0/309951887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9240003530 con 0x7f92400039c0 2024-04-17T09:53:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f9257e00640 1 --2- 172.21.15.32:0/309951887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f925806c140 0x7f9240004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.359+0000 7f925d600640 1 --2- 172.21.15.32:0/309951887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9240004d50 0x7f92400051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.363+0000 7f925cc00640 1 --2- 172.21.15.32:0/309951887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92400039c0 0x7f9240003de0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f924c009ae0 tx=0x7f924c01ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb57fab0acc0c41 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.363+0000 7f9257400640 1 -- 172.21.15.32:0/309951887 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c0b2070 con 0x7f92400039c0 2024-04-17T09:53:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.363+0000 7f9257400640 1 -- 172.21.15.32:0/309951887 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f924c0a5610 con 0x7f92400039c0 2024-04-17T09:53:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.363+0000 7f9257400640 1 -- 172.21.15.32:0/309951887 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c0096b0 con 0x7f92400039c0 2024-04-17T09:53:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.363+0000 7f925f600640 1 -- 172.21.15.32:0/309951887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92400039c0 msgr2=0x7f9240003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.363+0000 7f925f600640 1 --2- 172.21.15.32:0/309951887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92400039c0 0x7f9240003de0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f924c009ae0 tx=0x7f924c01ca80 comp rx=0 tx=0).stop 2024-04-17T09:53:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.363+0000 7f925d600640 1 -- 172.21.15.32:0/309951887 reap_dead start 2024-04-17T09:53:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.363+0000 7f925f600640 1 -- 172.21.15.32:0/309951887 shutdown_connections 2024-04-17T09:53:26.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.363+0000 7f925f600640 1 -- 172.21.15.32:0/309951887 >> 172.21.15.32:0/309951887 conn(0x7f925809a9f0 msgr2=0x7f92580b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:26.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.363+0000 7f925f600640 1 -- 172.21.15.32:0/309951887 shutdown_connections 2024-04-17T09:53:26.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.363+0000 7f925f600640 1 -- 172.21.15.32:0/309951887 wait complete. 2024-04-17T09:53:26.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f925f600640 1 Processor -- start 2024-04-17T09:53:26.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f925f600640 1 -- start start 2024-04-17T09:53:26.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f925f600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92400039c0 0x7f92580b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:26.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f925f600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92581528b0 0x7f9258152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:26.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f925f600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9258153b80 0x7f9258154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:26.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f925f600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f92580cbc20 con 0x7f9258153b80 2024-04-17T09:53:26.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f925f600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f92580cbaa0 con 0x7f92581528b0 2024-04-17T09:53:26.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f925f600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f92580ce5b0 con 0x7f92400039c0 2024-04-17T09:53:26.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f925d600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9258153b80 0x7f9258154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:26.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f925d600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9258153b80 0x7f9258154030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49370/0 (socket says 172.21.15.32:49370) 2024-04-17T09:53:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f925d600640 1 -- 172.21.15.32:0/1424999123 learned_addr learned my addr 172.21.15.32:0/1424999123 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f9257e00640 1 --2- 172.21.15.32:0/1424999123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92581528b0 0x7f9258152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f925cc00640 1 --2- 172.21.15.32:0/1424999123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92400039c0 0x7f92580b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f925d600640 1 -- 172.21.15.32:0/1424999123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92400039c0 msgr2=0x7f92580b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f925d600640 1 --2- 172.21.15.32:0/1424999123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92400039c0 0x7f92580b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f925d600640 1 -- 172.21.15.32:0/1424999123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92581528b0 msgr2=0x7f9258152d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f925d600640 1 --2- 172.21.15.32:0/1424999123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92581528b0 0x7f9258152d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f925d600640 1 -- 172.21.15.32:0/1424999123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9258154870 con 0x7f9258153b80 2024-04-17T09:53:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f925cc00640 1 --2- 172.21.15.32:0/1424999123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92400039c0 0x7f92580b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f925d600640 1 --2- 172.21.15.32:0/1424999123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9258153b80 0x7f9258154030 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f92580d2530 tx=0x7f9250010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f9254c00640 1 -- 172.21.15.32:0/1424999123 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250010d20 con 0x7f9258153b80 2024-04-17T09:53:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f9254c00640 1 -- 172.21.15.32:0/1424999123 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92500039a0 con 0x7f9258153b80 2024-04-17T09:53:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f925f600640 1 -- 172.21.15.32:0/1424999123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9258154b50 con 0x7f9258153b80 2024-04-17T09:53:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f9254c00640 1 -- 172.21.15.32:0/1424999123 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250003dd0 con 0x7f9258153b80 2024-04-17T09:53:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.367+0000 7f925f600640 1 -- 172.21.15.32:0/1424999123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9258155110 con 0x7f9258153b80 2024-04-17T09:53:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.371+0000 7f925f600640 1 -- 172.21.15.32:0/1424999123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9258068a90 con 0x7f9258153b80 2024-04-17T09:53:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.371+0000 7f9254c00640 1 -- 172.21.15.32:0/1424999123 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f925001e020 con 0x7f9258153b80 2024-04-17T09:53:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.371+0000 7f9254c00640 1 --2- 172.21.15.32:0/1424999123 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f92380683b0 0x7f923806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.371+0000 7f9254c00640 1 -- 172.21.15.32:0/1424999123 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9250086fe0 con 0x7f9258153b80 2024-04-17T09:53:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.371+0000 7f925cc00640 1 --2- 172.21.15.32:0/1424999123 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f92380683b0 0x7f923806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.371+0000 7f925cc00640 1 --2- 172.21.15.32:0/1424999123 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f92380683b0 0x7f923806a870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f924c0af000 tx=0x7f924c0046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.375+0000 7f9254c00640 1 -- 172.21.15.32:0/1424999123 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f9250025020 con 0x7f9258153b80 2024-04-17T09:53:26.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.543+0000 7f925f600640 1 -- 172.21.15.32:0/1424999123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f92580b7080 con 0x7f9258153b80 2024-04-17T09:53:26.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.543+0000 7f9254c00640 1 -- 172.21.15.32:0/1424999123 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f9250057520 con 0x7f9258153b80 2024-04-17T09:53:26.547 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:53:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.547+0000 7f925f600640 1 -- 172.21.15.32:0/1424999123 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f92380683b0 msgr2=0x7f923806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.547+0000 7f925f600640 1 --2- 172.21.15.32:0/1424999123 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f92380683b0 0x7f923806a870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f924c0af000 tx=0x7f924c0046d0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.547+0000 7f925f600640 1 -- 172.21.15.32:0/1424999123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9258153b80 msgr2=0x7f9258154030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.547+0000 7f925f600640 1 --2- 172.21.15.32:0/1424999123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9258153b80 0x7f9258154030 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f92580d2530 tx=0x7f9250010590 comp rx=0 tx=0).stop 2024-04-17T09:53:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.547+0000 7f925f600640 1 -- 172.21.15.32:0/1424999123 shutdown_connections 2024-04-17T09:53:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.547+0000 7f925f600640 1 --2- 172.21.15.32:0/1424999123 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f92380683b0 0x7f923806a870 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.547+0000 7f925f600640 1 --2- 172.21.15.32:0/1424999123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9258153b80 0x7f9258154030 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.547+0000 7f925f600640 1 --2- 172.21.15.32:0/1424999123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92581528b0 0x7f9258152d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.547+0000 7f925f600640 1 --2- 172.21.15.32:0/1424999123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92400039c0 0x7f92580b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.547+0000 7f925f600640 1 -- 172.21.15.32:0/1424999123 >> 172.21.15.32:0/1424999123 conn(0x7f925809a9f0 msgr2=0x7f925806f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.547+0000 7f925f600640 1 -- 172.21.15.32:0/1424999123 shutdown_connections 2024-04-17T09:53:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.547+0000 7f925f600640 1 -- 172.21.15.32:0/1424999123 wait complete. 2024-04-17T09:53:26.568 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:53:26.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.703+0000 7fd8b5600640 1 Processor -- start 2024-04-17T09:53:26.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.703+0000 7fd8b5600640 1 -- start start 2024-04-17T09:53:26.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.707+0000 7fd8b5600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8b015e870 0x7fd8b0162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:26.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.707+0000 7fd8b5600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd8b00b0320 0x7fd8b00d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.707+0000 7fd8b5600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8b00d0780 0x7fd8b006f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.707+0000 7fd8b5600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd8b00cf810 con 0x7fd8b00d0780 2024-04-17T09:53:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.707+0000 7fd8b5600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd8b0067dd0 con 0x7fd8b00b0320 2024-04-17T09:53:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.707+0000 7fd8b5600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b00cfdb0 con 0x7fd8b015e870 2024-04-17T09:53:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.707+0000 7fd8af400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8b00d0780 0x7fd8b006f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.707+0000 7fd8af400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8b00d0780 0x7fd8b006f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49376/0 (socket says 172.21.15.32:49376) 2024-04-17T09:53:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.707+0000 7fd8af400640 1 -- 172.21.15.32:0/1368357281 learned_addr learned my addr 172.21.15.32:0/1368357281 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.707+0000 7fd8aea00640 1 --2- 172.21.15.32:0/1368357281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8b015e870 0x7fd8b0162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.707+0000 7fd8a6000640 1 --2- 172.21.15.32:0/1368357281 >> v2:172.21.15.144:3300/0 conn(0x7fd8b00b0320 0x7fd8b00d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.707+0000 7fd8af400640 1 -- 172.21.15.32:0/1368357281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8b015e870 msgr2=0x7fd8b0162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.707+0000 7fd8af400640 1 --2- 172.21.15.32:0/1368357281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8b015e870 0x7fd8b0162e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.707+0000 7fd8af400640 1 -- 172.21.15.32:0/1368357281 >> v2:172.21.15.144:3300/0 conn(0x7fd8b00b0320 msgr2=0x7fd8b00d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.707+0000 7fd8af400640 1 --2- 172.21.15.32:0/1368357281 >> v2:172.21.15.144:3300/0 conn(0x7fd8b00b0320 0x7fd8b00d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.707+0000 7fd8af400640 1 -- 172.21.15.32:0/1368357281 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b0163340 con 0x7fd8b00d0780 2024-04-17T09:53:26.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.707+0000 7fd8aea00640 1 --2- 172.21.15.32:0/1368357281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8b015e870 0x7fd8b0162e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:26.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.707+0000 7fd8a6000640 1 --2- 172.21.15.32:0/1368357281 >> v2:172.21.15.144:3300/0 conn(0x7fd8b00b0320 0x7fd8b00d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:26.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.707+0000 7fd8af400640 1 --2- 172.21.15.32:0/1368357281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b00d0780 0x7fd8b006f7c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd8b006e130 tx=0x7fd8a801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de052f00d4908d84 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:26.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.707+0000 7fd8ae000640 1 -- 172.21.15.32:0/1368357281 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a80a5780 con 0x7fd8b00d0780 2024-04-17T09:53:26.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.707+0000 7fd8ae000640 1 -- 172.21.15.32:0/1368357281 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a80a5920 con 0x7fd8b00d0780 2024-04-17T09:53:26.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.707+0000 7fd8ae000640 1 -- 172.21.15.32:0/1368357281 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a80a5cf0 con 0x7fd8b00d0780 2024-04-17T09:53:26.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8b5600640 1 -- 172.21.15.32:0/1368357281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b00d0780 msgr2=0x7fd8b006f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:26.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8b5600640 1 --2- 172.21.15.32:0/1368357281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b00d0780 0x7fd8b006f7c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd8b006e130 tx=0x7fd8a801c9f0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8b5600640 1 -- 172.21.15.32:0/1368357281 shutdown_connections 2024-04-17T09:53:26.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8b5600640 1 --2- 172.21.15.32:0/1368357281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b00d0780 0x7fd8b006f7c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8b5600640 1 --2- 172.21.15.32:0/1368357281 >> v2:172.21.15.144:3300/0 conn(0x7fd8b00b0320 0x7fd8b00d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8b5600640 1 --2- 172.21.15.32:0/1368357281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8b015e870 0x7fd8b0162e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8b5600640 1 -- 172.21.15.32:0/1368357281 >> 172.21.15.32:0/1368357281 conn(0x7fd8b00d0c00 msgr2=0x7fd8b00d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:26.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8b5600640 1 -- 172.21.15.32:0/1368357281 shutdown_connections 2024-04-17T09:53:26.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8b5600640 1 -- 172.21.15.32:0/1368357281 wait complete. 2024-04-17T09:53:26.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8b5600640 1 Processor -- start 2024-04-17T09:53:26.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8b5600640 1 -- start start 2024-04-17T09:53:26.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8b5600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b00b0320 0x7fd8b0156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:26.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8b5600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8b015e870 0x7fd8b015a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:26.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8b5600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b00b1f50 0x7fd8b014dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:26.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8b5600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b006e370 con 0x7fd8b00b1f50 2024-04-17T09:53:26.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8b5600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b00cfa40 con 0x7fd8b00b0320 2024-04-17T09:53:26.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8b5600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b006e4f0 con 0x7fd8b015e870 2024-04-17T09:53:26.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8af400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b00b1f50 0x7fd8b014dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:26.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8af400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b00b1f50 0x7fd8b014dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49380/0 (socket says 172.21.15.32:49380) 2024-04-17T09:53:26.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8af400640 1 -- 172.21.15.32:0/3456267803 learned_addr learned my addr 172.21.15.32:0/3456267803 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:26.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8aea00640 1 --2- 172.21.15.32:0/3456267803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b00b0320 0x7fd8b0156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:26.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8a6000640 1 --2- 172.21.15.32:0/3456267803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8b015e870 0x7fd8b015a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:26.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8af400640 1 -- 172.21.15.32:0/3456267803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8b015e870 msgr2=0x7fd8b015a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:26.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8af400640 1 --2- 172.21.15.32:0/3456267803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8b015e870 0x7fd8b015a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8af400640 1 -- 172.21.15.32:0/3456267803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b00b0320 msgr2=0x7fd8b0156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:26.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8af400640 1 --2- 172.21.15.32:0/3456267803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b00b0320 0x7fd8b0156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.711+0000 7fd8af400640 1 -- 172.21.15.32:0/3456267803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b0150490 con 0x7fd8b00b1f50 2024-04-17T09:53:26.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.715+0000 7fd8af400640 1 --2- 172.21.15.32:0/3456267803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b00b1f50 0x7fd8b014dd70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80af040 tx=0x7fd8a8019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:26.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.715+0000 7fd8a7400640 1 -- 172.21.15.32:0/3456267803 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a80bc020 con 0x7fd8b00b1f50 2024-04-17T09:53:26.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.715+0000 7fd8a7400640 1 -- 172.21.15.32:0/3456267803 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a80b8880 con 0x7fd8b00b1f50 2024-04-17T09:53:26.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.715+0000 7fd8a7400640 1 -- 172.21.15.32:0/3456267803 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a80b8c70 con 0x7fd8b00b1f50 2024-04-17T09:53:26.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.715+0000 7fd8b5600640 1 -- 172.21.15.32:0/3456267803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b01574d0 con 0x7fd8b00b1f50 2024-04-17T09:53:26.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.715+0000 7fd8b5600640 1 -- 172.21.15.32:0/3456267803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b0156400 con 0x7fd8b00b1f50 2024-04-17T09:53:26.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.715+0000 7fd8b5600640 1 -- 172.21.15.32:0/3456267803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b0068a90 con 0x7fd8b00b1f50 2024-04-17T09:53:26.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.715+0000 7fd8a7400640 1 -- 172.21.15.32:0/3456267803 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd8a80bf020 con 0x7fd8b00b1f50 2024-04-17T09:53:26.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.715+0000 7fd8a7400640 1 --2- 172.21.15.32:0/3456267803 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd88c0683b0 0x7fd88c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:26.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.715+0000 7fd8a7400640 1 -- 172.21.15.32:0/3456267803 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd8a8121e00 con 0x7fd8b00b1f50 2024-04-17T09:53:26.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.715+0000 7fd8aea00640 1 --2- 172.21.15.32:0/3456267803 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd88c0683b0 0x7fd88c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:26.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.719+0000 7fd8aea00640 1 --2- 172.21.15.32:0/3456267803 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd88c0683b0 0x7fd88c06a870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0006fd0 tx=0x7fd8a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:26.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.719+0000 7fd8a7400640 1 -- 172.21.15.32:0/3456267803 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd8a80c6030 con 0x7fd8b00b1f50 2024-04-17T09:53:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.919+0000 7fd8b5600640 1 -- 172.21.15.32:0/3456267803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8b0073690 con 0x7fd8b00b1f50 2024-04-17T09:53:26.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.919+0000 7fd8a7400640 1 -- 172.21.15.32:0/3456267803 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5193 (secure 0 0 0) 0x7fd8a80165e0 con 0x7fd8b00b1f50 2024-04-17T09:53:26.923 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:26.923 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/441201886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":441201886},{"type":"v1","addr":"172.21.15.144:6834","nonce":441201886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1066277472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1066277472},{"type":"v1","addr":"172.21.15.32:6836","nonce":1066277472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/201410023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":201410023},{"type":"v1","addr":"172.21.15.32:6837","nonce":201410023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:53:13.172187+0000","modified":"2024-04-17T09:53:16.188325+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/3332924005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3332924005},{"type":"v1","addr":"172.21.15.144:6835","nonce":3332924005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-17T09:53:26.924 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2024-04-17T09:53:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.923+0000 7fd8b5600640 1 -- 172.21.15.32:0/3456267803 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd88c0683b0 msgr2=0x7fd88c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.923+0000 7fd8b5600640 1 --2- 172.21.15.32:0/3456267803 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd88c0683b0 0x7fd88c06a870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0006fd0 tx=0x7fd8a0008040 comp rx=0 tx=0).stop 2024-04-17T09:53:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.923+0000 7fd8b5600640 1 -- 172.21.15.32:0/3456267803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b00b1f50 msgr2=0x7fd8b014dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.923+0000 7fd8b5600640 1 --2- 172.21.15.32:0/3456267803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b00b1f50 0x7fd8b014dd70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80af040 tx=0x7fd8a8019790 comp rx=0 tx=0).stop 2024-04-17T09:53:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.923+0000 7fd8b5600640 1 -- 172.21.15.32:0/3456267803 shutdown_connections 2024-04-17T09:53:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.923+0000 7fd8b5600640 1 --2- 172.21.15.32:0/3456267803 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd88c0683b0 0x7fd88c06a870 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.923+0000 7fd8b5600640 1 --2- 172.21.15.32:0/3456267803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8b015e870 0x7fd8b015a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.923+0000 7fd8b5600640 1 --2- 172.21.15.32:0/3456267803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b00b1f50 0x7fd8b014dd70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.923+0000 7fd8b5600640 1 --2- 172.21.15.32:0/3456267803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b00b0320 0x7fd8b0156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.923+0000 7fd8b5600640 1 -- 172.21.15.32:0/3456267803 >> 172.21.15.32:0/3456267803 conn(0x7fd8b00d0c00 msgr2=0x7fd8b00b39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.923+0000 7fd8b5600640 1 -- 172.21.15.32:0/3456267803 shutdown_connections 2024-04-17T09:53:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:26.923+0000 7fd8b5600640 1 -- 172.21.15.32:0/3456267803 wait complete. 2024-04-17T09:53:26.944 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:53:27.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.083+0000 7f398f800640 1 Processor -- start 2024-04-17T09:53:27.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.083+0000 7f398f800640 1 -- start start 2024-04-17T09:53:27.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.083+0000 7f398f800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3988144e90 0x7f39880b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:27.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.083+0000 7f398f800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f398815c1b0 0x7f398806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:27.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.083+0000 7f398f800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3988072bb0 con 0x7f39880b4de0 2024-04-17T09:53:27.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.083+0000 7f398f800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3988145270 con 0x7f3988144e90 2024-04-17T09:53:27.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.083+0000 7f398f800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f39880b3dc0 con 0x7f398815c1b0 2024-04-17T09:53:27.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.083+0000 7f398ce00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3988144e90 0x7f39880b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:27.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.083+0000 7f398ce00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3988144e90 0x7f39880b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49384/0 (socket says 172.21.15.32:49384) 2024-04-17T09:53:27.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.083+0000 7f398ce00640 1 -- 172.21.15.32:0/1757967366 learned_addr learned my addr 172.21.15.32:0/1757967366 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:27.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.083+0000 7f398ce00640 1 -- 172.21.15.32:0/1757967366 >> v2:172.21.15.144:3300/0 conn(0x7f398815c1b0 msgr2=0x7f398806ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:27.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.083+0000 7f398ce00640 1 --2- 172.21.15.32:0/1757967366 >> v2:172.21.15.144:3300/0 conn(0x7f398815c1b0 0x7f398806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:27.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.083+0000 7f398ce00640 1 -- 172.21.15.32:0/1757967366 >> v1:172.21.15.144:6789/0 conn(0x7f39880b4de0 legacy=0x7f398815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:27.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.083+0000 7f398ce00640 1 -- 172.21.15.32:0/1757967366 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39880b40a0 con 0x7f3988144e90 2024-04-17T09:53:27.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.087+0000 7f398ce00640 1 --2- 172.21.15.32:0/1757967366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3988144e90 0x7f39880b48a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f398806ff30 tx=0x7f397801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870f24894a1500ae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:27.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.087+0000 7f3987e00640 1 -- 172.21.15.32:0/1757967366 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39780b1070 con 0x7f3988144e90 2024-04-17T09:53:27.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.087+0000 7f3987e00640 1 -- 172.21.15.32:0/1757967366 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39780ac620 con 0x7f3988144e90 2024-04-17T09:53:27.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.087+0000 7f3987e00640 1 -- 172.21.15.32:0/1757967366 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39780ac9f0 con 0x7f3988144e90 2024-04-17T09:53:27.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.087+0000 7f398f800640 1 -- 172.21.15.32:0/1757967366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3988144e90 msgr2=0x7f39880b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:27.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.087+0000 7f398f800640 1 --2- 172.21.15.32:0/1757967366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3988144e90 0x7f39880b48a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f398806ff30 tx=0x7f397801c4b0 comp rx=0 tx=0).stop 2024-04-17T09:53:27.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.087+0000 7f398f800640 1 -- 172.21.15.32:0/1757967366 shutdown_connections 2024-04-17T09:53:27.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.087+0000 7f398f800640 1 --2- 172.21.15.32:0/1757967366 >> v2:172.21.15.144:3300/0 conn(0x7f398815c1b0 0x7f398806ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:27.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.087+0000 7f398f800640 1 --2- 172.21.15.32:0/1757967366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3988144e90 0x7f39880b48a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:27.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.087+0000 7f398f800640 1 -- 172.21.15.32:0/1757967366 >> 172.21.15.32:0/1757967366 conn(0x7f398806ba30 msgr2=0x7f398806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:27.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.087+0000 7f398f800640 1 -- 172.21.15.32:0/1757967366 shutdown_connections 2024-04-17T09:53:27.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.087+0000 7f398f800640 1 -- 172.21.15.32:0/1757967366 wait complete. 2024-04-17T09:53:27.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.087+0000 7f398f800640 1 Processor -- start 2024-04-17T09:53:27.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f398f800640 1 -- start start 2024-04-17T09:53:27.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f398f800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39880b4de0 0x7f39881f5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:27.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f398f800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3988144e90 0x7f398814bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:27.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f398f800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f398815c1b0 0x7f39880a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:27.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f398f800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3988168110 con 0x7f3988144e90 2024-04-17T09:53:27.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f398f800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3988072880 con 0x7f398815c1b0 2024-04-17T09:53:27.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f398f800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3988168290 con 0x7f39880b4de0 2024-04-17T09:53:27.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f397fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3988144e90 0x7f398814bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:27.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f397fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3988144e90 0x7f398814bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49400/0 (socket says 172.21.15.32:49400) 2024-04-17T09:53:27.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f397fe00640 1 -- 172.21.15.32:0/3185949314 learned_addr learned my addr 172.21.15.32:0/3185949314 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:27.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f398d800640 1 --2- 172.21.15.32:0/3185949314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f398815c1b0 0x7f39880a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:27.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f398ce00640 1 --2- 172.21.15.32:0/3185949314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39880b4de0 0x7f39881f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:27.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f397fe00640 1 -- 172.21.15.32:0/3185949314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39880b4de0 msgr2=0x7f39881f5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:27.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f397fe00640 1 --2- 172.21.15.32:0/3185949314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39880b4de0 0x7f39881f5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:27.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f397fe00640 1 -- 172.21.15.32:0/3185949314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f398815c1b0 msgr2=0x7f39880a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:27.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f397fe00640 1 --2- 172.21.15.32:0/3185949314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f398815c1b0 0x7f39880a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:27.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f397fe00640 1 -- 172.21.15.32:0/3185949314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39880b3800 con 0x7f3988144e90 2024-04-17T09:53:27.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f398d800640 1 --2- 172.21.15.32:0/3185949314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f398815c1b0 0x7f39880a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:27.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f397fe00640 1 --2- 172.21.15.32:0/3185949314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3988144e90 0x7f398814bae0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f397000c490 tx=0x7f3970001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:27.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f3985600640 1 -- 172.21.15.32:0/3185949314 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970017070 con 0x7f3988144e90 2024-04-17T09:53:27.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f3985600640 1 -- 172.21.15.32:0/3185949314 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3970003600 con 0x7f3988144e90 2024-04-17T09:53:27.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f3985600640 1 -- 172.21.15.32:0/3185949314 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970003a30 con 0x7f3988144e90 2024-04-17T09:53:27.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f398f800640 1 -- 172.21.15.32:0/3185949314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f398815cbc0 con 0x7f3988144e90 2024-04-17T09:53:27.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f398f800640 1 -- 172.21.15.32:0/3185949314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f398815d180 con 0x7f3988144e90 2024-04-17T09:53:27.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.091+0000 7f398f800640 1 -- 172.21.15.32:0/3185949314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3988068a90 con 0x7f3988144e90 2024-04-17T09:53:27.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.095+0000 7f3985600640 1 -- 172.21.15.32:0/3185949314 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f397001a070 con 0x7f3988144e90 2024-04-17T09:53:27.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.095+0000 7f3985600640 1 --2- 172.21.15.32:0/3185949314 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f39640683d0 0x7f396406a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:27.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.095+0000 7f3985600640 1 -- 172.21.15.32:0/3185949314 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3970085ec0 con 0x7f3988144e90 2024-04-17T09:53:27.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.095+0000 7f398ce00640 1 --2- 172.21.15.32:0/3185949314 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f39640683d0 0x7f396406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:27.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.095+0000 7f398ce00640 1 --2- 172.21.15.32:0/3185949314 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f39640683d0 0x7f396406a890 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f39780ae000 tx=0x7f39780a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:27.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.099+0000 7f3985600640 1 -- 172.21.15.32:0/3185949314 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3970056460 con 0x7f3988144e90 2024-04-17T09:53:27.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.267+0000 7f398f800640 1 -- 172.21.15.32:0/3185949314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39880b1cf0 con 0x7f3988144e90 2024-04-17T09:53:27.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.267+0000 7f3985600640 1 -- 172.21.15.32:0/3185949314 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f3970055d60 con 0x7f3988144e90 2024-04-17T09:53:27.270 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:27.270 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:14.779535+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:53:27.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.267+0000 7f398f800640 1 -- 172.21.15.32:0/3185949314 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f39640683d0 msgr2=0x7f396406a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:27.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.267+0000 7f398f800640 1 --2- 172.21.15.32:0/3185949314 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f39640683d0 0x7f396406a890 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f39780ae000 tx=0x7f39780a7000 comp rx=0 tx=0).stop 2024-04-17T09:53:27.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.267+0000 7f398f800640 1 -- 172.21.15.32:0/3185949314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3988144e90 msgr2=0x7f398814bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:27.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.267+0000 7f398f800640 1 --2- 172.21.15.32:0/3185949314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3988144e90 0x7f398814bae0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f397000c490 tx=0x7f3970001e80 comp rx=0 tx=0).stop 2024-04-17T09:53:27.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.267+0000 7f398f800640 1 -- 172.21.15.32:0/3185949314 shutdown_connections 2024-04-17T09:53:27.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.267+0000 7f398f800640 1 --2- 172.21.15.32:0/3185949314 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f39640683d0 0x7f396406a890 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:27.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.267+0000 7f398f800640 1 --2- 172.21.15.32:0/3185949314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f398815c1b0 0x7f39880a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:27.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.267+0000 7f398f800640 1 --2- 172.21.15.32:0/3185949314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3988144e90 0x7f398814bae0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:27.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.267+0000 7f398f800640 1 --2- 172.21.15.32:0/3185949314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39880b4de0 0x7f39881f5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:27.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.267+0000 7f398f800640 1 -- 172.21.15.32:0/3185949314 >> 172.21.15.32:0/3185949314 conn(0x7f398806ba30 msgr2=0x7f39880b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:27.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.267+0000 7f398f800640 1 -- 172.21.15.32:0/3185949314 shutdown_connections 2024-04-17T09:53:27.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.267+0000 7f398f800640 1 -- 172.21.15.32:0/3185949314 wait complete. 2024-04-17T09:53:27.292 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:53:27.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.427+0000 7ff6cde00640 1 Processor -- start 2024-04-17T09:53:27.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.427+0000 7ff6cde00640 1 -- start start 2024-04-17T09:53:27.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.431+0000 7ff6cde00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6c815eb60 0x7ff6c80b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:27.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.431+0000 7ff6cde00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6c80d0c70 con 0x7ff6c80b4970 2024-04-17T09:53:27.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.431+0000 7ff6cde00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6c80d0f50 con 0x7ff6c809b100 2024-04-17T09:53:27.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.431+0000 7ff6cde00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff6c815ef40 con 0x7ff6c815eb60 2024-04-17T09:53:27.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.431+0000 7ff6bfe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff6c80b4970 0x7ff6c809a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48868/0 (socket says 172.21.15.32:48868) 2024-04-17T09:53:27.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.431+0000 7ff6bfe00640 1 -- 172.21.15.32:0/1487911380 learned_addr learned my addr 172.21.15.32:0/1487911380 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:27.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.431+0000 7ff6cca00640 1 --2- 172.21.15.32:0/1487911380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6c815eb60 0x7ff6c80b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:27.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.431+0000 7ff6c7e00640 1 -- 172.21.15.32:0/1487911380 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1615245641 0 0) 0x7ff6c80d0c70 con 0x7ff6c80b4970 2024-04-17T09:53:27.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.431+0000 7ff6c7e00640 1 -- 172.21.15.32:0/1487911380 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6ac003530 con 0x7ff6c80b4970 2024-04-17T09:53:27.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.431+0000 7ff6cca00640 1 -- 172.21.15.32:0/1487911380 >> v1:172.21.15.144:6789/0 conn(0x7ff6c809b100 legacy=0x7ff6c806d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:27.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.431+0000 7ff6cca00640 1 -- 172.21.15.32:0/1487911380 >> v1:172.21.15.32:6789/0 conn(0x7ff6c80b4970 legacy=0x7ff6c809a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:27.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.431+0000 7ff6c7e00640 1 -- 172.21.15.32:0/1487911380 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1087091546 0 0) 0x7ff6c80d0f50 con 0x7ff6c809b100 2024-04-17T09:53:27.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.431+0000 7ff6cca00640 1 -- 172.21.15.32:0/1487911380 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c80d1230 con 0x7ff6c815eb60 2024-04-17T09:53:27.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.435+0000 7ff6cca00640 1 --2- 172.21.15.32:0/1487911380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6c815eb60 0x7ff6c80b13b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff6b80097f0 tx=0x7ff6b801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d7627425a40bd5f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:27.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.435+0000 7ff6c7e00640 1 -- 172.21.15.32:0/1487911380 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b80b0070 con 0x7ff6c815eb60 2024-04-17T09:53:27.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.435+0000 7ff6c7e00640 1 -- 172.21.15.32:0/1487911380 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b80ab600 con 0x7ff6c815eb60 2024-04-17T09:53:27.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.435+0000 7ff6c7e00640 1 -- 172.21.15.32:0/1487911380 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b80ab9d0 con 0x7ff6c815eb60 2024-04-17T09:53:27.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.435+0000 7ff6cde00640 1 -- 172.21.15.32:0/1487911380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6c815eb60 msgr2=0x7ff6c80b13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:27.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.435+0000 7ff6cde00640 1 --2- 172.21.15.32:0/1487911380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6c815eb60 0x7ff6c80b13b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff6b80097f0 tx=0x7ff6b801c4e0 comp rx=0 tx=0).stop 2024-04-17T09:53:27.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.435+0000 7ff6cde00640 1 -- 172.21.15.32:0/1487911380 shutdown_connections 2024-04-17T09:53:27.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.435+0000 7ff6cde00640 1 --2- 172.21.15.32:0/1487911380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6c815eb60 0x7ff6c80b13b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:27.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.435+0000 7ff6cde00640 1 -- 172.21.15.32:0/1487911380 >> 172.21.15.32:0/1487911380 conn(0x7ff6c80b3e50 msgr2=0x7ff6c80b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:27.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.435+0000 7ff6cde00640 1 -- 172.21.15.32:0/1487911380 shutdown_connections 2024-04-17T09:53:27.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.435+0000 7ff6cde00640 1 -- 172.21.15.32:0/1487911380 wait complete. 2024-04-17T09:53:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.435+0000 7ff6cde00640 1 Processor -- start 2024-04-17T09:53:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.435+0000 7ff6cde00640 1 -- start start 2024-04-17T09:53:27.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.435+0000 7ff6cde00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6c809b100 0x7ff6c814fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:27.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.435+0000 7ff6cde00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6c80b4970 0x7ff6c80b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:27.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.435+0000 7ff6cde00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c815eb60 0x7ff6c80b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.435+0000 7ff6cde00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c816cb20 con 0x7ff6c815eb60 2024-04-17T09:53:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.435+0000 7ff6cde00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c80ccdc0 con 0x7ff6c80b4970 2024-04-17T09:53:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.435+0000 7ff6cde00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff6c816cca0 con 0x7ff6c809b100 2024-04-17T09:53:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.435+0000 7ff6cd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c815eb60 0x7ff6c80b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.439+0000 7ff6cd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c815eb60 0x7ff6c80b0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49402/0 (socket says 172.21.15.32:49402) 2024-04-17T09:53:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.439+0000 7ff6cd400640 1 -- 172.21.15.32:0/2467296765 learned_addr learned my addr 172.21.15.32:0/2467296765 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.439+0000 7ff6cca00640 1 --2- 172.21.15.32:0/2467296765 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6c809b100 0x7ff6c814fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.439+0000 7ff6bfe00640 1 --2- 172.21.15.32:0/2467296765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6c80b4970 0x7ff6c80b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.439+0000 7ff6cd400640 1 -- 172.21.15.32:0/2467296765 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6c809b100 msgr2=0x7ff6c814fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.439+0000 7ff6cd400640 1 --2- 172.21.15.32:0/2467296765 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6c809b100 0x7ff6c814fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.439+0000 7ff6cd400640 1 -- 172.21.15.32:0/2467296765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6c80b4970 msgr2=0x7ff6c80b09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.439+0000 7ff6cd400640 1 --2- 172.21.15.32:0/2467296765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6c80b4970 0x7ff6c80b09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.439+0000 7ff6cd400640 1 -- 172.21.15.32:0/2467296765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c80b6df0 con 0x7ff6c815eb60 2024-04-17T09:53:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.439+0000 7ff6bfe00640 1 --2- 172.21.15.32:0/2467296765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6c80b4970 0x7ff6c80b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.439+0000 7ff6cd400640 1 --2- 172.21.15.32:0/2467296765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c815eb60 0x7ff6c80b0ef0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff6c000c780 tx=0x7ff6c000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.439+0000 7ff6c5600640 1 -- 172.21.15.32:0/2467296765 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0003db0 con 0x7ff6c815eb60 2024-04-17T09:53:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.439+0000 7ff6c5600640 1 -- 172.21.15.32:0/2467296765 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6c000ce30 con 0x7ff6c815eb60 2024-04-17T09:53:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.439+0000 7ff6cde00640 1 -- 172.21.15.32:0/2467296765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c80b70d0 con 0x7ff6c815eb60 2024-04-17T09:53:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.439+0000 7ff6c5600640 1 -- 172.21.15.32:0/2467296765 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0005290 con 0x7ff6c815eb60 2024-04-17T09:53:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.439+0000 7ff6cde00640 1 -- 172.21.15.32:0/2467296765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c80b7690 con 0x7ff6c815eb60 2024-04-17T09:53:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.439+0000 7ff6cde00640 1 -- 172.21.15.32:0/2467296765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6c804fd00 con 0x7ff6c815eb60 2024-04-17T09:53:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.439+0000 7ff6c5600640 1 -- 172.21.15.32:0/2467296765 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff6c0005430 con 0x7ff6c815eb60 2024-04-17T09:53:27.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.439+0000 7ff6c5600640 1 --2- 172.21.15.32:0/2467296765 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff6a4068380 0x7ff6a406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:27.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.439+0000 7ff6c5600640 1 -- 172.21.15.32:0/2467296765 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff6c0086920 con 0x7ff6c815eb60 2024-04-17T09:53:27.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.443+0000 7ff6cca00640 1 --2- 172.21.15.32:0/2467296765 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff6a4068380 0x7ff6a406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:27.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.443+0000 7ff6cca00640 1 --2- 172.21.15.32:0/2467296765 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff6a4068380 0x7ff6a406a840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff6b80ad000 tx=0x7ff6b80a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:27.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.447+0000 7ff6c5600640 1 -- 172.21.15.32:0/2467296765 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff6c0056e50 con 0x7ff6c815eb60 2024-04-17T09:53:27.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.643+0000 7ff6cde00640 1 -- 172.21.15.32:0/2467296765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6c80b1aa0 con 0x7ff6c815eb60 2024-04-17T09:53:27.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.643+0000 7ff6c5600640 1 -- 172.21.15.32:0/2467296765 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5193 (secure 0 0 0) 0x7ff6c0056750 con 0x7ff6c815eb60 2024-04-17T09:53:27.648 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:27.648 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/441201886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":441201886},{"type":"v1","addr":"172.21.15.144:6834","nonce":441201886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1066277472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1066277472},{"type":"v1","addr":"172.21.15.32:6836","nonce":1066277472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/201410023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":201410023},{"type":"v1","addr":"172.21.15.32:6837","nonce":201410023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:53:13.172187+0000","modified":"2024-04-17T09:53:16.188325+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/3332924005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3332924005},{"type":"v1","addr":"172.21.15.144:6835","nonce":3332924005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-17T09:53:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2024-04-17T09:53:27.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.647+0000 7ff6cde00640 1 -- 172.21.15.32:0/2467296765 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff6a4068380 msgr2=0x7ff6a406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:27.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.647+0000 7ff6cde00640 1 --2- 172.21.15.32:0/2467296765 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff6a4068380 0x7ff6a406a840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff6b80ad000 tx=0x7ff6b80a6000 comp rx=0 tx=0).stop 2024-04-17T09:53:27.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.647+0000 7ff6cde00640 1 -- 172.21.15.32:0/2467296765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c815eb60 msgr2=0x7ff6c80b0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:27.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.647+0000 7ff6cde00640 1 --2- 172.21.15.32:0/2467296765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c815eb60 0x7ff6c80b0ef0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff6c000c780 tx=0x7ff6c000cc40 comp rx=0 tx=0).stop 2024-04-17T09:53:27.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.647+0000 7ff6cde00640 1 -- 172.21.15.32:0/2467296765 shutdown_connections 2024-04-17T09:53:27.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.647+0000 7ff6cde00640 1 --2- 172.21.15.32:0/2467296765 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff6a4068380 0x7ff6a406a840 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:27.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.647+0000 7ff6cde00640 1 --2- 172.21.15.32:0/2467296765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c815eb60 0x7ff6c80b0ef0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:27.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.647+0000 7ff6cde00640 1 --2- 172.21.15.32:0/2467296765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6c80b4970 0x7ff6c80b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:27.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.647+0000 7ff6cde00640 1 --2- 172.21.15.32:0/2467296765 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6c809b100 0x7ff6c814fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:27.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.647+0000 7ff6cde00640 1 -- 172.21.15.32:0/2467296765 >> 172.21.15.32:0/2467296765 conn(0x7ff6c80b3e50 msgr2=0x7ff6c806fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:27.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.647+0000 7ff6cde00640 1 -- 172.21.15.32:0/2467296765 shutdown_connections 2024-04-17T09:53:27.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:27.647+0000 7ff6cde00640 1 -- 172.21.15.32:0/2467296765 wait complete. 2024-04-17T09:53:28.071 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T09:53:13.172187+0000', 'modified': '2024-04-17T09:53:16.188325+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4279}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4279': {'gid': 4279, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.144:6835/3332924005', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6833', 'nonce': 3332924005}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 3332924005}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4279, 'qdb_cluster': [4279]} 2024-04-17T09:53:28.071 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T09:53:28.071 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-17T09:53:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.211+0000 7fa207000640 1 Processor -- start 2024-04-17T09:53:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.211+0000 7fa207000640 1 -- start start 2024-04-17T09:53:28.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.215+0000 7fa207000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2000b3340 0x7fa20015f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.215+0000 7fa207000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa20015f870 con 0x7fa200155a70 2024-04-17T09:53:28.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.215+0000 7fa207000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa20015fb50 con 0x7fa200155e50 2024-04-17T09:53:28.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.215+0000 7fa207000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa2000b42a0 con 0x7fa2000b3340 2024-04-17T09:53:28.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.215+0000 7fa1ffe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa200155a70 0x7fa20015d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48878/0 (socket says 172.21.15.32:48878) 2024-04-17T09:53:28.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.215+0000 7fa1ffe00640 1 -- 172.21.15.32:0/4233533622 learned_addr learned my addr 172.21.15.32:0/4233533622 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:28.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.215+0000 7fa205000640 1 --2- 172.21.15.32:0/4233533622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2000b3340 0x7fa20015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:28.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.215+0000 7fa1ff400640 1 -- 172.21.15.32:0/4233533622 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3745461170 0 0) 0x7fa20015f870 con 0x7fa200155a70 2024-04-17T09:53:28.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.215+0000 7fa1ff400640 1 -- 172.21.15.32:0/4233533622 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1e4003530 con 0x7fa200155a70 2024-04-17T09:53:28.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.215+0000 7fa1ff400640 1 -- 172.21.15.32:0/4233533622 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2034604373 0 0) 0x7fa20015fb50 con 0x7fa200155e50 2024-04-17T09:53:28.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.215+0000 7fa1ff400640 1 -- 172.21.15.32:0/4233533622 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa20015f870 con 0x7fa200155e50 2024-04-17T09:53:28.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.215+0000 7fa205000640 1 -- 172.21.15.32:0/4233533622 >> v1:172.21.15.144:6789/0 conn(0x7fa200155e50 legacy=0x7fa20015c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:28.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.215+0000 7fa205000640 1 -- 172.21.15.32:0/4233533622 >> v1:172.21.15.32:6789/0 conn(0x7fa200155a70 legacy=0x7fa20015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:28.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.215+0000 7fa205000640 1 -- 172.21.15.32:0/4233533622 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20015fe30 con 0x7fa2000b3340 2024-04-17T09:53:28.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.219+0000 7fa205000640 1 --2- 172.21.15.32:0/4233533622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2000b3340 0x7fa20015f190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009dc0 tx=0x7fa1f801c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ab40af779a88125 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:28.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.219+0000 7fa1ff400640 1 -- 172.21.15.32:0/4233533622 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f80b2070 con 0x7fa2000b3340 2024-04-17T09:53:28.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.219+0000 7fa1ff400640 1 -- 172.21.15.32:0/4233533622 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f80a5850 con 0x7fa2000b3340 2024-04-17T09:53:28.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.219+0000 7fa1ff400640 1 -- 172.21.15.32:0/4233533622 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f80a5c20 con 0x7fa2000b3340 2024-04-17T09:53:28.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.219+0000 7fa207000640 1 -- 172.21.15.32:0/4233533622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2000b3340 msgr2=0x7fa20015f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:28.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.219+0000 7fa207000640 1 --2- 172.21.15.32:0/4233533622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2000b3340 0x7fa20015f190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009dc0 tx=0x7fa1f801c990 comp rx=0 tx=0).stop 2024-04-17T09:53:28.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.219+0000 7fa207000640 1 -- 172.21.15.32:0/4233533622 shutdown_connections 2024-04-17T09:53:28.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.219+0000 7fa207000640 1 --2- 172.21.15.32:0/4233533622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2000b3340 0x7fa20015f190 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.219+0000 7fa207000640 1 -- 172.21.15.32:0/4233533622 >> 172.21.15.32:0/4233533622 conn(0x7fa20006ba30 msgr2=0x7fa20006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:28.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.219+0000 7fa207000640 1 -- 172.21.15.32:0/4233533622 shutdown_connections 2024-04-17T09:53:28.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.219+0000 7fa207000640 1 -- 172.21.15.32:0/4233533622 wait complete. 2024-04-17T09:53:28.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.219+0000 7fa207000640 1 Processor -- start 2024-04-17T09:53:28.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.219+0000 7fa207000640 1 -- start start 2024-04-17T09:53:28.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.223+0000 7fa207000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2000b3340 0x7fa20014a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.223+0000 7fa207000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa200155a70 0x7fa200148df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.223+0000 7fa207000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa200155e50 0x7fa2001521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.223+0000 7fa207000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa2000d02f0 con 0x7fa2000b3340 2024-04-17T09:53:28.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.223+0000 7fa207000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa2000d0170 con 0x7fa200155a70 2024-04-17T09:53:28.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.223+0000 7fa207000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa2000d0470 con 0x7fa200155e50 2024-04-17T09:53:28.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.223+0000 7fa1ffe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2000b3340 0x7fa20014a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:28.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.223+0000 7fa1ffe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2000b3340 0x7fa20014a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49416/0 (socket says 172.21.15.32:49416) 2024-04-17T09:53:28.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.223+0000 7fa1ffe00640 1 -- 172.21.15.32:0/827062212 learned_addr learned my addr 172.21.15.32:0/827062212 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:28.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.223+0000 7fa205000640 1 --2- 172.21.15.32:0/827062212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa200155e50 0x7fa2001521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:28.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.223+0000 7fa1f7400640 1 --2- 172.21.15.32:0/827062212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa200155a70 0x7fa200148df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:28.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.223+0000 7fa1ffe00640 1 -- 172.21.15.32:0/827062212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa200155e50 msgr2=0x7fa2001521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:28.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.223+0000 7fa1ffe00640 1 --2- 172.21.15.32:0/827062212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa200155e50 0x7fa2001521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.223+0000 7fa1ffe00640 1 -- 172.21.15.32:0/827062212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa200155a70 msgr2=0x7fa200148df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:28.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.223+0000 7fa1ffe00640 1 --2- 172.21.15.32:0/827062212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa200155a70 0x7fa200148df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.223+0000 7fa1ffe00640 1 -- 172.21.15.32:0/827062212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa200152ad0 con 0x7fa2000b3340 2024-04-17T09:53:28.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.223+0000 7fa1ffe00640 1 --2- 172.21.15.32:0/827062212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2000b3340 0x7fa20014a180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000c4d0 tx=0x7fa1f000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:28.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.223+0000 7fa1fcc00640 1 -- 172.21.15.32:0/827062212 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0017070 con 0x7fa2000b3340 2024-04-17T09:53:28.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.223+0000 7fa1fcc00640 1 -- 172.21.15.32:0/827062212 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f0005cf0 con 0x7fa2000b3340 2024-04-17T09:53:28.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.223+0000 7fa1fcc00640 1 -- 172.21.15.32:0/827062212 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f00072b0 con 0x7fa2000b3340 2024-04-17T09:53:28.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.223+0000 7fa207000640 1 -- 172.21.15.32:0/827062212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2001496a0 con 0x7fa2000b3340 2024-04-17T09:53:28.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.223+0000 7fa207000640 1 -- 172.21.15.32:0/827062212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa200152730 con 0x7fa2000b3340 2024-04-17T09:53:28.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa1fcc00640 1 -- 172.21.15.32:0/827062212 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa1f001a070 con 0x7fa2000b3340 2024-04-17T09:53:28.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa1fcc00640 1 --2- 172.21.15.32:0/827062212 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa1dc0683b0 0x7fa1dc06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa1fcc00640 1 -- 172.21.15.32:0/827062212 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa1f0085d80 con 0x7fa2000b3340 2024-04-17T09:53:28.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa1f7400640 1 --2- 172.21.15.32:0/827062212 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa1dc0683b0 0x7fa1dc06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:28.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa20a538000 1 Processor -- start 2024-04-17T09:53:28.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa20a538000 1 -- start start 2024-04-17T09:53:28.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa1f7400640 1 --2- 172.21.15.32:0/827062212 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa1dc0683b0 0x7fa1dc06a870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa20014cd30 tx=0x7fa1e8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:28.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa20a538000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55996537bb50 0x55996537bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa20a538000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55996537c470 0x5599653317b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa20a538000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559965331cf0 0x5599653320d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa20a538000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55996517f4d0 con 0x55996537c470 2024-04-17T09:53:28.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa20a538000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559965119bf0 con 0x55996537bb50 2024-04-17T09:53:28.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa20a538000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55996517b490 con 0x559965331cf0 2024-04-17T09:53:28.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa1f7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55996537c470 0x5599653317b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:28.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa1f7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55996537c470 0x5599653317b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49430/0 (socket says 172.21.15.32:49430) 2024-04-17T09:53:28.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa1f7400640 1 -- 172.21.15.32:0/2618317800 learned_addr learned my addr 172.21.15.32:0/2618317800 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:28.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa205000640 1 --2- 172.21.15.32:0/2618317800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559965331cf0 0x5599653320d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:28.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa1ffe00640 1 --2- 172.21.15.32:0/2618317800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55996537bb50 0x55996537bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:28.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa1f7400640 1 -- 172.21.15.32:0/2618317800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559965331cf0 msgr2=0x5599653320d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:28.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa1f7400640 1 --2- 172.21.15.32:0/2618317800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559965331cf0 0x5599653320d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa1f7400640 1 -- 172.21.15.32:0/2618317800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55996537bb50 msgr2=0x55996537bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:28.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa1f7400640 1 --2- 172.21.15.32:0/2618317800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55996537bb50 0x55996537bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa1f7400640 1 -- 172.21.15.32:0/2618317800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559965105e80 con 0x55996537c470 2024-04-17T09:53:28.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa1f7400640 1 --2- 172.21.15.32:0/2618317800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55996537c470 0x5599653317b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8001ec0 tx=0x7fa1e8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48a59cc3bf5051e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:28.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa1f4c00640 1 -- 172.21.15.32:0/2618317800 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8025070 con 0x55996537c470 2024-04-17T09:53:28.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa1f4c00640 1 -- 172.21.15.32:0/2618317800 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e801b050 con 0x55996537c470 2024-04-17T09:53:28.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.227+0000 7fa1f4c00640 1 -- 172.21.15.32:0/2618317800 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8014e00 con 0x55996537c470 2024-04-17T09:53:28.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa20a538000 1 -- 172.21.15.32:0/2618317800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55996537c470 msgr2=0x5599653317b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:28.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa20a538000 1 --2- 172.21.15.32:0/2618317800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55996537c470 0x5599653317b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8001ec0 tx=0x7fa1e8012000 comp rx=0 tx=0).stop 2024-04-17T09:53:28.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa20a538000 1 -- 172.21.15.32:0/2618317800 shutdown_connections 2024-04-17T09:53:28.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa20a538000 1 --2- 172.21.15.32:0/2618317800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559965331cf0 0x5599653320d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa20a538000 1 --2- 172.21.15.32:0/2618317800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55996537c470 0x5599653317b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa20a538000 1 --2- 172.21.15.32:0/2618317800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55996537bb50 0x55996537bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa20a538000 1 -- 172.21.15.32:0/2618317800 >> 172.21.15.32:0/2618317800 conn(0x559965337470 msgr2=0x559965330090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:28.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa20a538000 1 -- 172.21.15.32:0/2618317800 shutdown_connections 2024-04-17T09:53:28.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa20a538000 1 -- 172.21.15.32:0/2618317800 wait complete. 2024-04-17T09:53:28.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa20a538000 1 -- start start 2024-04-17T09:53:28.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa20a538000 1 Processor -- start 2024-04-17T09:53:28.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa20a538000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559965331cf0 0x5599653367e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa20a538000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55996537bb50 0x559965369440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa20a538000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55996537c470 0x559965369980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa20a538000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55996513ca60 con 0x55996537bb50 2024-04-17T09:53:28.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa20a538000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559965123e80 con 0x559965331cf0 2024-04-17T09:53:28.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa20a538000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55996513acf0 con 0x55996537c470 2024-04-17T09:53:28.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa20a538000 10 client.0 _ll_register_callbacks cb 0x55996531ca50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:53:28.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa1f7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55996537bb50 0x559965369440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:28.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa1f7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55996537bb50 0x559965369440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49432/0 (socket says 172.21.15.32:49432) 2024-04-17T09:53:28.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa1f7400640 1 -- 172.21.15.32:0/1837321047 learned_addr learned my addr 172.21.15.32:0/1837321047 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:28.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa1ffe00640 1 --2- 172.21.15.32:0/1837321047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559965331cf0 0x5599653367e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:28.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa205000640 1 --2- 172.21.15.32:0/1837321047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55996537c470 0x559965369980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:28.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa1f7400640 1 -- 172.21.15.32:0/1837321047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55996537c470 msgr2=0x559965369980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:28.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa1f7400640 1 --2- 172.21.15.32:0/1837321047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55996537c470 0x559965369980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa1f7400640 1 -- 172.21.15.32:0/1837321047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559965331cf0 msgr2=0x5599653367e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:28.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa1f7400640 1 --2- 172.21.15.32:0/1837321047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559965331cf0 0x5599653367e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa1f7400640 1 -- 172.21.15.32:0/1837321047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559965119e40 con 0x55996537bb50 2024-04-17T09:53:28.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa1ffe00640 1 --2- 172.21.15.32:0/1837321047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559965331cf0 0x5599653367e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:53:28.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa1f7400640 1 --2- 172.21.15.32:0/1837321047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55996537bb50 0x559965369440 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8023040 tx=0x7fa1e8001160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb49817ea08f225 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:28.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa1ed600640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:53:28.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa1ed600640 1 -- 172.21.15.32:0/1837321047 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8025040 con 0x55996537bb50 2024-04-17T09:53:28.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa20a538000 1 -- 172.21.15.32:0/1837321047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1e801b050 con 0x55996537bb50 2024-04-17T09:53:28.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa1ed600640 1 -- 172.21.15.32:0/1837321047 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa1e8020a40 con 0x55996537bb50 2024-04-17T09:53:28.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa1ed600640 1 -- 172.21.15.32:0/1837321047 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fa1e8038050 con 0x55996537bb50 2024-04-17T09:53:28.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa20a538000 10 client.4620 fetch_fsmap learned FSMap version 8 2024-04-17T09:53:28.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.231+0000 7fa20a538000 1 -- 172.21.15.32:0/1837321047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55996514ae00 con 0x55996537bb50 2024-04-17T09:53:28.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa1ed600640 1 -- 172.21.15.32:0/1837321047 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e801b050 con 0x55996537bb50 2024-04-17T09:53:28.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa1ed600640 1 -- 172.21.15.32:0/1837321047 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 8) ==== 3495+0+0 (secure 0 0 0) 0x7fa1e803b020 con 0x55996537bb50 2024-04-17T09:53:28.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa20a538000 10 client.4620 fetch_fsmap finished waiting for FSMap version 8 2024-04-17T09:53:28.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa20a538000 10 client.4620 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-17T09:53:28.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa20a538000 20 client.4620 populate_metadata read hostname 'smithi032' 2024-04-17T09:53:28.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa20a538000 1 --2- 172.21.15.32:0/1837321047 >> [v2:172.21.15.144:6833/3332924005,v1:172.21.15.144:6835/3332924005] conn(0x55996538b0b0 0x55996538d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa20a538000 4 client.4620 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:53:28.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa20a538000 1 -- 172.21.15.32:0/1837321047 --> [v2:172.21.15.144:6833/3332924005,v1:172.21.15.144:6835/3332924005] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559965118bc0 con 0x55996538b0b0 2024-04-17T09:53:28.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa1ffe00640 1 --2- 172.21.15.32:0/1837321047 >> [v2:172.21.15.144:6833/3332924005,v1:172.21.15.144:6835/3332924005] conn(0x55996538b0b0 0x55996538d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:28.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa1ffe00640 1 --2- 172.21.15.32:0/1837321047 >> [v2:172.21.15.144:6833/3332924005,v1:172.21.15.144:6835/3332924005] conn(0x55996538b0b0 0x55996538d4a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15377944851772b7 server_cookie=659b92ee7b2a34e9 in_seq=0 out_seq=0 2024-04-17T09:53:28.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa1ed600640 10 client.4620 ms_handle_connect on v2:172.21.15.144:6833/3332924005 2024-04-17T09:53:28.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa1ed600640 1 -- 172.21.15.32:0/1837321047 <== mds.0 v2:172.21.15.144:6833/3332924005 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559965118bc0 con 0x55996538b0b0 2024-04-17T09:53:28.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa1ed600640 10 client.4620 handle_command_reply: tid=0 2024-04-17T09:53:28.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa20a538000 1 client.4620 shutdown 2024-04-17T09:53:28.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa20a538000 1 -- 172.21.15.32:0/1837321047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55996537bb50 msgr2=0x559965369440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:28.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa20a538000 1 --2- 172.21.15.32:0/1837321047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55996537bb50 0x559965369440 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8023040 tx=0x7fa1e8001160 comp rx=0 tx=0).stop 2024-04-17T09:53:28.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa20a538000 1 -- 172.21.15.32:0/1837321047 shutdown_connections 2024-04-17T09:53:28.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa20a538000 1 --2- 172.21.15.32:0/1837321047 >> [v2:172.21.15.144:6833/3332924005,v1:172.21.15.144:6835/3332924005] conn(0x55996538b0b0 0x55996538d4a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa20a538000 1 --2- 172.21.15.32:0/1837321047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55996537c470 0x559965369980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa20a538000 1 --2- 172.21.15.32:0/1837321047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55996537bb50 0x559965369440 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa1ed600640 0 client.4620 ms_handle_reset on v2:172.21.15.144:6833/3332924005 2024-04-17T09:53:28.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa20a538000 1 --2- 172.21.15.32:0/1837321047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559965331cf0 0x5599653367e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa20a538000 1 -- 172.21.15.32:0/1837321047 >> 172.21.15.32:0/1837321047 conn(0x559965337470 msgr2=0x5599653307a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:28.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa20a538000 1 -- 172.21.15.32:0/1837321047 shutdown_connections 2024-04-17T09:53:28.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa20a538000 1 -- 172.21.15.32:0/1837321047 wait complete. 2024-04-17T09:53:28.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.235+0000 7fa20a538000 20 client.4620 trim_cache size 0 max 16384 2024-04-17T09:53:28.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa20a538000 1 Processor -- start 2024-04-17T09:53:28.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa20a538000 1 -- start start 2024-04-17T09:53:28.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa20a538000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559965331cf0 0x559965369020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa20a538000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55996537bb50 0x559965369560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa20a538000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55996537c470 0x55996538c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa20a538000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa20015a1c0 con 0x559965331cf0 2024-04-17T09:53:28.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa20a538000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa2000ae450 con 0x55996537c470 2024-04-17T09:53:28.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa20a538000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa200159b70 con 0x55996537bb50 2024-04-17T09:53:28.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa1ffe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559965331cf0 0x559965369020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:28.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa1ffe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559965331cf0 0x559965369020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49446/0 (socket says 172.21.15.32:49446) 2024-04-17T09:53:28.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa1ffe00640 1 -- 172.21.15.32:0/2492195268 learned_addr learned my addr 172.21.15.32:0/2492195268 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:28.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa1ffe00640 1 -- 172.21.15.32:0/2492195268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55996537bb50 msgr2=0x559965369560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:28.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa1ffe00640 1 --2- 172.21.15.32:0/2492195268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55996537bb50 0x559965369560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa1ffe00640 1 -- 172.21.15.32:0/2492195268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55996537c470 msgr2=0x55996538c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:28.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa1ffe00640 1 --2- 172.21.15.32:0/2492195268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55996537c470 0x55996538c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa1ffe00640 1 -- 172.21.15.32:0/2492195268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559965200d30 con 0x559965331cf0 2024-04-17T09:53:28.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa1ffe00640 1 --2- 172.21.15.32:0/2492195268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559965331cf0 0x559965369020 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00525d0 tx=0x7fa1f00840e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8edcedc65d2af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:28.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa1efe00640 1 -- 172.21.15.32:0/2492195268 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0053f50 con 0x559965331cf0 2024-04-17T09:53:28.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa1efe00640 1 -- 172.21.15.32:0/2492195268 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f005e2f0 con 0x559965331cf0 2024-04-17T09:53:28.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa1efe00640 1 -- 172.21.15.32:0/2492195268 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f007db70 con 0x559965331cf0 2024-04-17T09:53:28.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa20a538000 1 -- 172.21.15.32:0/2492195268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559965331cf0 msgr2=0x559965369020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:28.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa20a538000 1 --2- 172.21.15.32:0/2492195268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559965331cf0 0x559965369020 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00525d0 tx=0x7fa1f00840e0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa20a538000 1 -- 172.21.15.32:0/2492195268 shutdown_connections 2024-04-17T09:53:28.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa20a538000 1 --2- 172.21.15.32:0/2492195268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55996537bb50 0x559965369560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa20a538000 1 --2- 172.21.15.32:0/2492195268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55996537c470 0x55996538c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa20a538000 1 --2- 172.21.15.32:0/2492195268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559965331cf0 0x559965369020 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.255+0000 7fa20a538000 1 -- 172.21.15.32:0/2492195268 >> 172.21.15.32:0/2492195268 conn(0x559965337470 msgr2=0x559965368910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:28.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa20a538000 1 -- 172.21.15.32:0/2492195268 shutdown_connections 2024-04-17T09:53:28.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa20a538000 1 -- 172.21.15.32:0/2492195268 wait complete. 2024-04-17T09:53:28.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa20a538000 1 -- start start 2024-04-17T09:53:28.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa20a538000 1 Processor -- start 2024-04-17T09:53:28.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa20a538000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559965331cf0 0x55996538f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa20a538000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55996537bb50 0x5599653a7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa20a538000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55996537c470 0x5599653a7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa20a538000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559965183b00 con 0x55996537c470 2024-04-17T09:53:28.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa20a538000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55996517e7e0 con 0x559965331cf0 2024-04-17T09:53:28.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa20a538000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55996518c6e0 con 0x55996537bb50 2024-04-17T09:53:28.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa20a538000 10 client.0 _ll_register_callbacks cb 0x7fa1e8023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:53:28.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa205000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55996537c470 0x5599653a7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:28.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa205000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55996537c470 0x5599653a7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49460/0 (socket says 172.21.15.32:49460) 2024-04-17T09:53:28.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa205000640 1 -- 172.21.15.32:0/2780282732 learned_addr learned my addr 172.21.15.32:0/2780282732 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:28.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa1ffe00640 1 --2- 172.21.15.32:0/2780282732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559965331cf0 0x55996538f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:28.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa1f7400640 1 --2- 172.21.15.32:0/2780282732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55996537bb50 0x5599653a7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:28.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa205000640 1 -- 172.21.15.32:0/2780282732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55996537bb50 msgr2=0x5599653a7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:28.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa205000640 1 --2- 172.21.15.32:0/2780282732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55996537bb50 0x5599653a7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa205000640 1 -- 172.21.15.32:0/2780282732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559965331cf0 msgr2=0x55996538f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:28.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa205000640 1 --2- 172.21.15.32:0/2780282732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559965331cf0 0x55996538f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa205000640 1 -- 172.21.15.32:0/2780282732 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599651117c0 con 0x55996537c470 2024-04-17T09:53:28.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa205000640 1 --2- 172.21.15.32:0/2780282732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55996537c470 0x5599653a7570 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa1f801cf00 tx=0x7fa1f8019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcfb5974bf729074 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:28.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa1ed600640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:53:28.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa1ed600640 1 -- 172.21.15.32:0/2780282732 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f80b2040 con 0x55996537c470 2024-04-17T09:53:28.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa1ed600640 1 -- 172.21.15.32:0/2780282732 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa1f80a5b30 con 0x55996537c470 2024-04-17T09:53:28.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa20a538000 1 -- 172.21.15.32:0/2780282732 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1f005e2f0 con 0x55996537c470 2024-04-17T09:53:28.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa1ed600640 1 -- 172.21.15.32:0/2780282732 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fa1f80a5d30 con 0x55996537c470 2024-04-17T09:53:28.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa20a538000 10 client.4626 fetch_fsmap learned FSMap version 8 2024-04-17T09:53:28.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa20a538000 1 -- 172.21.15.32:0/2780282732 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x559965124650 con 0x55996537c470 2024-04-17T09:53:28.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa1ed600640 1 -- 172.21.15.32:0/2780282732 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f005e2f0 con 0x55996537c470 2024-04-17T09:53:28.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa1ed600640 1 -- 172.21.15.32:0/2780282732 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 8) ==== 3495+0+0 (secure 0 0 0) 0x7fa2000b51a0 con 0x55996537c470 2024-04-17T09:53:28.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa20a538000 10 client.4626 fetch_fsmap finished waiting for FSMap version 8 2024-04-17T09:53:28.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa20a538000 10 client.4626 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-17T09:53:28.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa20a538000 20 client.4626 populate_metadata read hostname 'smithi032' 2024-04-17T09:53:28.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa20a538000 1 --2- 172.21.15.32:0/2780282732 >> [v2:172.21.15.144:6833/3332924005,v1:172.21.15.144:6835/3332924005] conn(0x5599653a7ab0 0x5599653ea460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa20a538000 4 client.4626 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T09:53:28.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.259+0000 7fa20a538000 1 -- 172.21.15.32:0/2780282732 --> [v2:172.21.15.144:6833/3332924005,v1:172.21.15.144:6835/3332924005] -- command(tid 0: {"prefix": "status"}) -- 0x55996518cef0 con 0x5599653a7ab0 2024-04-17T09:53:28.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa1ffe00640 1 --2- 172.21.15.32:0/2780282732 >> [v2:172.21.15.144:6833/3332924005,v1:172.21.15.144:6835/3332924005] conn(0x5599653a7ab0 0x5599653ea460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:28.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa1ffe00640 1 --2- 172.21.15.32:0/2780282732 >> [v2:172.21.15.144:6833/3332924005,v1:172.21.15.144:6835/3332924005] conn(0x5599653a7ab0 0x5599653ea460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=757984e724246084 server_cookie=5ea6f61ccc356312 in_seq=0 out_seq=0 2024-04-17T09:53:28.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa1ed600640 10 client.4626 ms_handle_connect on v2:172.21.15.144:6833/3332924005 2024-04-17T09:53:28.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa1ed600640 1 -- 172.21.15.32:0/2780282732 <== mds.0 v2:172.21.15.144:6833/3332924005 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55996518cef0 con 0x5599653a7ab0 2024-04-17T09:53:28.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa1ed600640 10 client.4626 handle_command_reply: tid=0 2024-04-17T09:53:28.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa20a538000 1 client.4626 shutdown 2024-04-17T09:53:28.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa20a538000 1 -- 172.21.15.32:0/2780282732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55996537c470 msgr2=0x5599653a7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:28.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa20a538000 1 --2- 172.21.15.32:0/2780282732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55996537c470 0x5599653a7570 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa1f801cf00 tx=0x7fa1f8019730 comp rx=0 tx=0).stop 2024-04-17T09:53:28.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa20a538000 1 -- 172.21.15.32:0/2780282732 shutdown_connections 2024-04-17T09:53:28.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa20a538000 1 --2- 172.21.15.32:0/2780282732 >> [v2:172.21.15.144:6833/3332924005,v1:172.21.15.144:6835/3332924005] conn(0x5599653a7ab0 0x5599653ea460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa20a538000 1 --2- 172.21.15.32:0/2780282732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55996537bb50 0x5599653a7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa20a538000 1 --2- 172.21.15.32:0/2780282732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55996537c470 0x5599653a7570 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa20a538000 1 --2- 172.21.15.32:0/2780282732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559965331cf0 0x55996538f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa20a538000 1 -- 172.21.15.32:0/2780282732 >> 172.21.15.32:0/2780282732 conn(0x559965337470 msgr2=0x559965333070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:28.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa1ed600640 0 client.4626 ms_handle_reset on v2:172.21.15.144:6833/3332924005 2024-04-17T09:53:28.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa20a538000 1 -- 172.21.15.32:0/2780282732 shutdown_connections 2024-04-17T09:53:28.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa20a538000 1 -- 172.21.15.32:0/2780282732 wait complete. 2024-04-17T09:53:28.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa20a538000 20 client.4626 trim_cache size 0 max 16384 2024-04-17T09:53:28.269 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:53:28.269 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "ff178df8-4c06-4367-a1e9-2e3a66d870d0", 2024-04-17T09:53:28.269 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-17T09:53:28.269 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 4279, 2024-04-17T09:53:28.270 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-17T09:53:28.270 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-17T09:53:28.270 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-17T09:53:28.270 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 14.485561499999999, 2024-04-17T09:53:28.270 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 8, 2024-04-17T09:53:28.270 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 21, 2024-04-17T09:53:28.270 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 21, 2024-04-17T09:53:28.270 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 21.793999671000002 2024-04-17T09:53:28.270 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:53:28.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa207000640 1 -- 172.21.15.32:0/827062212 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa1dc0683b0 msgr2=0x7fa1dc06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:28.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa207000640 1 --2- 172.21.15.32:0/827062212 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa1dc0683b0 0x7fa1dc06a870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa20014cd30 tx=0x7fa1e8009040 comp rx=0 tx=0).stop 2024-04-17T09:53:28.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa207000640 1 -- 172.21.15.32:0/827062212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2000b3340 msgr2=0x7fa20014a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:28.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa207000640 1 --2- 172.21.15.32:0/827062212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2000b3340 0x7fa20014a180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000c4d0 tx=0x7fa1f000c990 comp rx=0 tx=0).stop 2024-04-17T09:53:28.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa207000640 1 -- 172.21.15.32:0/827062212 shutdown_connections 2024-04-17T09:53:28.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa207000640 1 --2- 172.21.15.32:0/827062212 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa1dc0683b0 0x7fa1dc06a870 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa207000640 1 --2- 172.21.15.32:0/827062212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa200155e50 0x7fa2001521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa207000640 1 --2- 172.21.15.32:0/827062212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa200155a70 0x7fa200148df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa207000640 1 --2- 172.21.15.32:0/827062212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2000b3340 0x7fa20014a180 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa207000640 1 -- 172.21.15.32:0/827062212 >> 172.21.15.32:0/827062212 conn(0x7fa20006ba30 msgr2=0x7fa20015d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:28.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa207000640 1 -- 172.21.15.32:0/827062212 shutdown_connections 2024-04-17T09:53:28.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.263+0000 7fa207000640 1 -- 172.21.15.32:0/827062212 wait complete. 2024-04-17T09:53:28.286 INFO:teuthology.run_tasks:Running task sequential... 2024-04-17T09:53:28.296 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-17T09:53:28.296 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-17T09:53:28.296 INFO:teuthology.task.print:Enabling mgr modules 2024-04-17T09:53:28.296 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-17T09:53:28.353 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-17T09:53:28.353 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-17T09:53:28.353 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-17T09:53:28.353 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-17T09:53:28.353 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T09:53:28.353 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-17T09:53:28.353 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-17T09:53:28.354 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-17T09:53:28.354 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-17T09:53:28.354 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-17T09:53:28.354 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-17T09:53:28.354 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-17T09:53:28.354 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-17T09:53:28.354 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-17T09:53:28.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:28.354 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-17T09:53:28.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:28.364 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2024-04-17T09:53:28.426 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2024-04-17T09:53:28.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:53:28.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:28.428 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2024-04-17T09:53:28.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:28.439 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link delete ceph-brx 2024-04-17T09:53:28.507 INFO:teuthology.orchestra.run.smithi144.stderr:Cannot find device "ceph-brx" 2024-04-17T09:53:28.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:53:28.509 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-17T09:53:28.509 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T09:53:28.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.647+0000 7fdead200640 1 Processor -- start 2024-04-17T09:53:28.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.647+0000 7fdead200640 1 -- start start 2024-04-17T09:53:28.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.647+0000 7fdead200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdea809b100 0x7fdea80b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.647+0000 7fdead200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdea80d05e0 con 0x7fdea80d00b0 2024-04-17T09:53:28.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.647+0000 7fdead200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdea80d08c0 con 0x7fdea815c5c0 2024-04-17T09:53:28.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.647+0000 7fdead200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fdea80ce560 con 0x7fdea809b100 2024-04-17T09:53:28.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.647+0000 7fde9d400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdea80d00b0 0x7fdea809a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48892/0 (socket says 172.21.15.32:48892) 2024-04-17T09:53:28.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.647+0000 7fde9d400640 1 -- 172.21.15.32:0/3965822267 learned_addr learned my addr 172.21.15.32:0/3965822267 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:28.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.647+0000 7fdea6800640 1 --2- 172.21.15.32:0/3965822267 >> v2:172.21.15.32:3300/0 conn(0x7fdea809b100 0x7fdea80b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:28.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.647+0000 7fdea6800640 1 -- 172.21.15.32:0/3965822267 >> v1:172.21.15.144:6789/0 conn(0x7fdea815c5c0 legacy=0x7fdea80cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:28.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.647+0000 7fdea6800640 1 -- 172.21.15.32:0/3965822267 >> v1:172.21.15.32:6789/0 conn(0x7fdea80d00b0 legacy=0x7fdea809a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:28.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.647+0000 7fdea6800640 1 -- 172.21.15.32:0/3965822267 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea80d0ba0 con 0x7fdea809b100 2024-04-17T09:53:28.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.647+0000 7fdea5400640 1 -- 172.21.15.32:0/3965822267 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3061477131 0 0) 0x7fdea80d05e0 con 0x7fdea80d00b0 2024-04-17T09:53:28.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.651+0000 7fdea6800640 1 --2- 172.21.15.32:0/3965822267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdea809b100 0x7fdea80b31b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdea0009d90 tx=0x7fdea001f120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9bfe73ae4fc7a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:28.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.651+0000 7fdea5400640 1 -- 172.21.15.32:0/3965822267 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea00b4070 con 0x7fdea809b100 2024-04-17T09:53:28.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.651+0000 7fdea5400640 1 -- 172.21.15.32:0/3965822267 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea00af600 con 0x7fdea809b100 2024-04-17T09:53:28.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.651+0000 7fdea5400640 1 -- 172.21.15.32:0/3965822267 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea00af9d0 con 0x7fdea809b100 2024-04-17T09:53:28.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.651+0000 7fdead200640 1 -- 172.21.15.32:0/3965822267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdea809b100 msgr2=0x7fdea80b31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:28.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.651+0000 7fdead200640 1 --2- 172.21.15.32:0/3965822267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdea809b100 0x7fdea80b31b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdea0009d90 tx=0x7fdea001f120 comp rx=0 tx=0).stop 2024-04-17T09:53:28.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.651+0000 7fdead200640 1 -- 172.21.15.32:0/3965822267 shutdown_connections 2024-04-17T09:53:28.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.651+0000 7fdead200640 1 --2- 172.21.15.32:0/3965822267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdea809b100 0x7fdea80b31b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.651+0000 7fdead200640 1 -- 172.21.15.32:0/3965822267 >> 172.21.15.32:0/3965822267 conn(0x7fdea80b3fd0 msgr2=0x7fdea80b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:28.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.651+0000 7fdead200640 1 -- 172.21.15.32:0/3965822267 shutdown_connections 2024-04-17T09:53:28.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.651+0000 7fdead200640 1 -- 172.21.15.32:0/3965822267 wait complete. 2024-04-17T09:53:28.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fdead200640 1 Processor -- start 2024-04-17T09:53:28.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fdead200640 1 -- start start 2024-04-17T09:53:28.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fdead200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdea809b100 0x7fdea809cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fdead200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdea80d00b0 0x7fdea815d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fdead200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdea815c5c0 0x7fdea815df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fdead200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdea80b6340 con 0x7fdea80d00b0 2024-04-17T09:53:28.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fdead200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdea80d11f0 con 0x7fdea809b100 2024-04-17T09:53:28.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fdead200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdea80b64c0 con 0x7fdea815c5c0 2024-04-17T09:53:28.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fde9d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdea80d00b0 0x7fdea815d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:28.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fde9d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdea80d00b0 0x7fdea815d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49486/0 (socket says 172.21.15.32:49486) 2024-04-17T09:53:28.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fde9d400640 1 -- 172.21.15.32:0/2999640689 learned_addr learned my addr 172.21.15.32:0/2999640689 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fdea6800640 1 --2- 172.21.15.32:0/2999640689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdea815c5c0 0x7fdea815df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fdea5e00640 1 --2- 172.21.15.32:0/2999640689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdea809b100 0x7fdea809cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fde9d400640 1 -- 172.21.15.32:0/2999640689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdea815c5c0 msgr2=0x7fdea815df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fde9d400640 1 --2- 172.21.15.32:0/2999640689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdea815c5c0 0x7fdea815df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fde9d400640 1 -- 172.21.15.32:0/2999640689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdea809b100 msgr2=0x7fdea809cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fde9d400640 1 --2- 172.21.15.32:0/2999640689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdea809b100 0x7fdea809cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fde9d400640 1 -- 172.21.15.32:0/2999640689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea809f570 con 0x7fdea80d00b0 2024-04-17T09:53:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fdea6800640 1 --2- 172.21.15.32:0/2999640689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdea815c5c0 0x7fdea815df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fdea5e00640 1 --2- 172.21.15.32:0/2999640689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdea809b100 0x7fdea809cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fde9d400640 1 --2- 172.21.15.32:0/2999640689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdea80d00b0 0x7fdea815d890 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fde900011f0 tx=0x7fde9000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fde9ea00640 1 -- 172.21.15.32:0/2999640689 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde90018070 con 0x7fdea80d00b0 2024-04-17T09:53:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fde9ea00640 1 -- 172.21.15.32:0/2999640689 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde900074e0 con 0x7fdea80d00b0 2024-04-17T09:53:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fde9ea00640 1 -- 172.21.15.32:0/2999640689 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde90004a10 con 0x7fdea80d00b0 2024-04-17T09:53:28.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fdead200640 1 -- 172.21.15.32:0/2999640689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea809f850 con 0x7fdea80d00b0 2024-04-17T09:53:28.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fdead200640 1 -- 172.21.15.32:0/2999640689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea809fe10 con 0x7fdea80d00b0 2024-04-17T09:53:28.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.655+0000 7fdead200640 1 -- 172.21.15.32:0/2999640689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdea80c8400 con 0x7fdea80d00b0 2024-04-17T09:53:28.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.659+0000 7fde9ea00640 1 -- 172.21.15.32:0/2999640689 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fde90004bb0 con 0x7fdea80d00b0 2024-04-17T09:53:28.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.659+0000 7fde9ea00640 1 --2- 172.21.15.32:0/2999640689 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fde84068380 0x7fde8406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:28.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.659+0000 7fde9ea00640 1 -- 172.21.15.32:0/2999640689 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fde90085d60 con 0x7fdea80d00b0 2024-04-17T09:53:28.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.659+0000 7fdea5e00640 1 --2- 172.21.15.32:0/2999640689 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fde84068380 0x7fde8406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:28.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.659+0000 7fdea5e00640 1 --2- 172.21.15.32:0/2999640689 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fde84068380 0x7fde8406a840 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fde98003e40 tx=0x7fde98009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:28.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.663+0000 7fde9ea00640 1 -- 172.21.15.32:0/2999640689 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fde90056310 con 0x7fdea80d00b0 2024-04-17T09:53:28.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.867+0000 7fdead200640 1 -- 172.21.15.32:0/2999640689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdea809cae0 con 0x7fdea80d00b0 2024-04-17T09:53:28.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.867+0000 7fde9ea00640 1 -- 172.21.15.32:0/2999640689 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fde90055c10 con 0x7fdea80d00b0 2024-04-17T09:53:28.871 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T09:53:28.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.871+0000 7fdead200640 1 -- 172.21.15.32:0/2999640689 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fde84068380 msgr2=0x7fde8406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:28.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.871+0000 7fdead200640 1 --2- 172.21.15.32:0/2999640689 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fde84068380 0x7fde8406a840 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fde98003e40 tx=0x7fde98009040 comp rx=0 tx=0).stop 2024-04-17T09:53:28.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.871+0000 7fdead200640 1 -- 172.21.15.32:0/2999640689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdea80d00b0 msgr2=0x7fdea815d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:28.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.871+0000 7fdead200640 1 --2- 172.21.15.32:0/2999640689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdea80d00b0 0x7fdea815d890 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fde900011f0 tx=0x7fde9000c9b0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.871+0000 7fdead200640 1 -- 172.21.15.32:0/2999640689 shutdown_connections 2024-04-17T09:53:28.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.871+0000 7fdead200640 1 --2- 172.21.15.32:0/2999640689 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fde84068380 0x7fde8406a840 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.871+0000 7fdead200640 1 --2- 172.21.15.32:0/2999640689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdea815c5c0 0x7fdea815df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.871+0000 7fdead200640 1 --2- 172.21.15.32:0/2999640689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdea80d00b0 0x7fdea815d890 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.871+0000 7fdead200640 1 --2- 172.21.15.32:0/2999640689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdea809b100 0x7fdea809cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:28.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.871+0000 7fdead200640 1 -- 172.21.15.32:0/2999640689 >> 172.21.15.32:0/2999640689 conn(0x7fdea80b3fd0 msgr2=0x7fdea80b6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:28.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.871+0000 7fdead200640 1 -- 172.21.15.32:0/2999640689 shutdown_connections 2024-04-17T09:53:28.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28.871+0000 7fdead200640 1 -- 172.21.15.32:0/2999640689 wait complete. 2024-04-17T09:53:28.892 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T09:53:28.892 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T09:53:28.892 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-17T09:53:28.893 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T09:53:28.893 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T09:53:28.893 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T09:53:28.893 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T09:53:28.893 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T09:53:28.893 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:53:28.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:28.893 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-17T09:53:28.903 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T09:53:28.904 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T09:53:28.904 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T09:53:28.904 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:28.904 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-17T09:53:28.904 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:28.904 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:53:28.904 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-17T09:53:28.904 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-17T09:53:28.904 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T09:53:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:53:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-17T09:53:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-17T09:53:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42445sec preferred_lft 42445sec 2024-04-17T09:53:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2024-04-17T09:53:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:28.905 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:53:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-17T09:53:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-17T09:53:28.905 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:53:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:53:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-17T09:53:28.905 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T09:53:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:cd:f4:aa:5b brd ff:ff:ff:ff:ff:ff 2024-04-17T09:53:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T09:53:28.905 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:28.906 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-17T09:53:28.906 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:53:28.906 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T09:53:28.906 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add name ceph-brx type bridge 2024-04-17T09:53:28.906 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr flush dev ceph-brx 2024-04-17T09:53:28.906 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set ceph-brx up 2024-04-17T09:53:28.906 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-17T09:53:28.907 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T09:53:28.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:53:29.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:53:29.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:29.058 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T09:53:29.076 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-17T09:53:29.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:29.078 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-17T09:53:29.151 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-17T09:53:29.151 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T09:53:29.151 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-17T09:53:29.152 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T09:53:29.152 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-17T09:53:29.152 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T09:53:29.152 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:53:29.152 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T09:53:29.152 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-17T09:53:29.152 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-17T09:53:29.152 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-17T09:53:29.152 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T09:53:29.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:53:29.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:53:29.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:29.292 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-17T09:53:29.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:29.301 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2024-04-17T09:53:29.357 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:53:29.357 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T09:53:29.357 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:53:29.357 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-17T09:53:29.357 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T09:53:29.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:53:29.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:53:29.486 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-17T09:53:29.486 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:53:29.487 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T09:53:29.487 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-17T09:53:29.487 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-17T09:53:29.487 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-17T09:53:29.487 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-17T09:53:29.487 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-17T09:53:29.487 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T09:53:29.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:53:29.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:53:29.649 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:53:29.649 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T09:53:29.649 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-17T09:53:29.649 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.0 master ceph-brx 2024-04-17T09:53:29.649 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T09:53:29.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:53:29.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:53:29.735 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T09:53:29.735 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:53:29.735 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T09:53:29.785 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:53:29.785 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:53:29.785 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T09:53:29.840 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T09:53:29.900 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:53:29.953 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T09:53:29.954 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T09:53:29.954 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T09:53:29.954 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T09:53:29.954 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T09:53:29.954 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T09:53:29.954 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T09:53:29.954 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T09:53:29.954 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T09:53:29.954 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T09:53:29.954 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T09:53:29.955 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T09:53:29.955 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:53:29.955 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T09:53:29.955 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T09:53:29.955 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T09:53:29.955 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T09:53:29.955 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T09:53:29.955 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T09:53:29.955 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T09:53:29.955 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:53:29.955 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T09:53:29.955 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T09:53:29.955 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T09:53:29.955 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T09:53:29.955 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T09:53:29.955 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T09:53:29.956 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T09:53:29.956 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T09:53:29.956 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:53:29.956 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:53:29.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:29.957 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T09:53:30.008 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T09:53:30.008 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T09:53:30.051 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T09:53:30.071 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:53:30.121 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T09:53:30.122 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T09:53:30.122 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T09:53:30.122 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T09:53:30.122 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T09:53:30.122 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T09:53:30.122 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T09:53:30.122 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T09:53:30.122 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T09:53:30.122 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T09:53:30.122 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T09:53:30.122 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T09:53:30.122 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:53:30.122 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T09:53:30.123 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T09:53:30.123 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T09:53:30.123 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T09:53:30.123 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T09:53:30.123 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T09:53:30.123 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T09:53:30.123 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:53:30.123 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T09:53:30.123 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T09:53:30.123 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T09:53:30.123 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T09:53:30.123 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T09:53:30.123 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T09:53:30.123 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T09:53:30.123 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T09:53:30.124 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:53:30.124 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:53:30.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:30.126 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T09:53:30.147 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2024-04-17T09:53:30.142+0000 7f5468161580 -1 init, newargv = 0x55b1cc369de0 newargc=15 2024-04-17T09:53:30.148 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[33387]: starting ceph client 2024-04-17T09:53:30.156 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[33387]: starting fuse 2024-04-17T09:53:31.257 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T09:53:31.282 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:53:31.334 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T09:53:31.335 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T09:53:31.335 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T09:53:31.335 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T09:53:31.335 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T09:53:31.335 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T09:53:31.335 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T09:53:31.335 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T09:53:31.335 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T09:53:31.335 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T09:53:31.335 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T09:53:31.335 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T09:53:31.335 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:53:31.335 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T09:53:31.336 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T09:53:31.336 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T09:53:31.336 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T09:53:31.336 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T09:53:31.336 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T09:53:31.336 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T09:53:31.336 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:53:31.336 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T09:53:31.336 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T09:53:31.336 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T09:53:31.336 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T09:53:31.336 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T09:53:31.336 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T09:53:31.336 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T09:53:31.336 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T09:53:31.337 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:53:31.337 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:53:31.337 INFO:teuthology.orchestra.run.smithi032.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-17T09:53:31.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:31.338 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T09:53:31.389 INFO:teuthology.orchestra.run.smithi032.stdout:44 2024-04-17T09:53:31.389 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-17T09:53:31.389 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2024-04-17T09:53:31.389 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2024-04-17T09:53:31.389 DEBUG:teuthology.orchestra.run.smithi032:> import re 2024-04-17T09:53:31.389 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T09:53:31.389 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2024-04-17T09:53:31.389 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:53:31.390 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2024-04-17T09:53:31.390 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T09:53:31.390 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2024-04-17T09:53:31.390 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T09:53:31.390 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:53:31.390 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2024-04-17T09:53:31.390 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2024-04-17T09:53:31.390 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2024-04-17T09:53:31.390 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2024-04-17T09:53:31.390 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:53:31.390 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2024-04-17T09:53:31.390 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T09:53:31.390 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T09:53:31.390 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T09:53:31.390 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2024-04-17T09:53:31.391 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2024-04-17T09:53:31.391 DEBUG:teuthology.orchestra.run.smithi032:> return f 2024-04-17T09:53:31.391 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T09:53:31.391 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:53:31.391 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2024-04-17T09:53:31.391 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T09:53:31.524 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.33387.asok 2024-04-17T09:53:31.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:53:31.537 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.33387.asok 2024-04-17T09:53:31.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:31.537 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.33387.asok status 2024-04-17T09:53:31.714 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:53:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2024-04-17T09:53:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-17T09:53:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T09:53:31.714 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2024-04-17T09:53:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2024-04-17T09:53:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T09:53:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "33387", 2024-04-17T09:53:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2024-04-17T09:53:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:53:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2024-04-17T09:53:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2024-04-17T09:53:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 4644, 2024-04-17T09:53:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2024-04-17T09:53:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2024-04-17T09:53:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2024-04-17T09:53:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 4644 2024-04-17T09:53:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:53:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T09:53:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T09:53:31.715 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T09:53:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1737700019 2024-04-17T09:53:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:53:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:53:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T09:53:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T09:53:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T09:53:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1737700019 2024-04-17T09:53:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:53:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.4644 192.168.0.1:0/1737700019", 2024-04-17T09:53:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/1737700019", 2024-04-17T09:53:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2024-04-17T09:53:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 8, 2024-04-17T09:53:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 21, 2024-04-17T09:53:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2024-04-17T09:53:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2024-04-17T09:53:31.716 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs" 2024-04-17T09:53:31.716 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:53:31.730 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T09:53:31.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.866+0000 7faa27200640 1 Processor -- start 2024-04-17T09:53:31.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.866+0000 7faa27200640 1 -- start start 2024-04-17T09:53:31.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.866+0000 7faa27200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faa200cd580 0x7faa20167560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:31.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.866+0000 7faa27200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa200cdac0 con 0x7faa20145f30 2024-04-17T09:53:31.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.866+0000 7faa27200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa2015cdf0 con 0x7faa20072500 2024-04-17T09:53:31.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.866+0000 7faa27200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7faa2006f400 con 0x7faa200cd580 2024-04-17T09:53:31.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.866+0000 7faa1fe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7faa20145f30 0x7faa20071df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52660/0 (socket says 172.21.15.32:52660) 2024-04-17T09:53:31.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.866+0000 7faa1fe00640 1 -- 172.21.15.32:0/3864101244 learned_addr learned my addr 172.21.15.32:0/3864101244 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:31.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.866+0000 7faa25200640 1 --2- 172.21.15.32:0/3864101244 >> v2:172.21.15.144:3300/0 conn(0x7faa200cd580 0x7faa20167560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:31.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.866+0000 7faa1f400640 1 -- 172.21.15.32:0/3864101244 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2980335458 0 0) 0x7faa200cdac0 con 0x7faa20145f30 2024-04-17T09:53:31.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.866+0000 7faa1f400640 1 -- 172.21.15.32:0/3864101244 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa04003530 con 0x7faa20145f30 2024-04-17T09:53:31.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.866+0000 7faa25200640 1 -- 172.21.15.32:0/3864101244 >> v1:172.21.15.144:6789/0 conn(0x7faa20072500 legacy=0x7faa200cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:31.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.866+0000 7faa25200640 1 -- 172.21.15.32:0/3864101244 >> v1:172.21.15.32:6789/0 conn(0x7faa20145f30 legacy=0x7faa20071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:31.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.866+0000 7faa25200640 1 -- 172.21.15.32:0/3864101244 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa2015d0d0 con 0x7faa200cd580 2024-04-17T09:53:31.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.870+0000 7faa25200640 1 --2- 172.21.15.32:0/3864101244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa200cd580 0x7faa20167560 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faa18009d90 tx=0x7faa1801f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13a516d32866e7cb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:31.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.870+0000 7faa1f400640 1 -- 172.21.15.32:0/3864101244 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa180b4070 con 0x7faa200cd580 2024-04-17T09:53:31.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.870+0000 7faa1f400640 1 -- 172.21.15.32:0/3864101244 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa180af600 con 0x7faa200cd580 2024-04-17T09:53:31.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.870+0000 7faa1f400640 1 -- 172.21.15.32:0/3864101244 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa180af9d0 con 0x7faa200cd580 2024-04-17T09:53:31.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.870+0000 7faa27200640 1 -- 172.21.15.32:0/3864101244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa200cd580 msgr2=0x7faa20167560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:31.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.870+0000 7faa27200640 1 --2- 172.21.15.32:0/3864101244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa200cd580 0x7faa20167560 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faa18009d90 tx=0x7faa1801f150 comp rx=0 tx=0).stop 2024-04-17T09:53:31.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa27200640 1 -- 172.21.15.32:0/3864101244 shutdown_connections 2024-04-17T09:53:31.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa27200640 1 --2- 172.21.15.32:0/3864101244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa200cd580 0x7faa20167560 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:31.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa27200640 1 -- 172.21.15.32:0/3864101244 >> 172.21.15.32:0/3864101244 conn(0x7faa2006ba30 msgr2=0x7faa2006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:31.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa27200640 1 -- 172.21.15.32:0/3864101244 shutdown_connections 2024-04-17T09:53:31.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa27200640 1 -- 172.21.15.32:0/3864101244 wait complete. 2024-04-17T09:53:31.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa27200640 1 Processor -- start 2024-04-17T09:53:31.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa27200640 1 -- start start 2024-04-17T09:53:31.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa27200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa20072500 0x7faa2015bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:31.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa27200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa200cd580 0x7faa200c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:31.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa27200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa20145f30 0x7faa20133520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:31.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa27200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faa20146310 con 0x7faa20072500 2024-04-17T09:53:31.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa27200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faa2015dba0 con 0x7faa200cd580 2024-04-17T09:53:31.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa27200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7faa2015d440 con 0x7faa20145f30 2024-04-17T09:53:31.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa1fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa20072500 0x7faa2015bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:31.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa1fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa20072500 0x7faa2015bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46488/0 (socket says 172.21.15.32:46488) 2024-04-17T09:53:31.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa1fe00640 1 -- 172.21.15.32:0/4103402066 learned_addr learned my addr 172.21.15.32:0/4103402066 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:31.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa25200640 1 --2- 172.21.15.32:0/4103402066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa20145f30 0x7faa20133520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:31.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa1fe00640 1 -- 172.21.15.32:0/4103402066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa20145f30 msgr2=0x7faa20133520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:31.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa1fe00640 1 --2- 172.21.15.32:0/4103402066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa20145f30 0x7faa20133520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:31.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa1fe00640 1 -- 172.21.15.32:0/4103402066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa200cd580 msgr2=0x7faa200c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:31.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa1fe00640 1 --2- 172.21.15.32:0/4103402066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa200cd580 0x7faa200c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:31.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa1fe00640 1 -- 172.21.15.32:0/4103402066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa200c9cd0 con 0x7faa20072500 2024-04-17T09:53:31.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa25200640 1 --2- 172.21.15.32:0/4103402066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa20145f30 0x7faa20133520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:31.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa1fe00640 1 --2- 172.21.15.32:0/4103402066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa20072500 0x7faa2015bc20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7faa1000c420 tx=0x7faa1000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:31.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa1cc00640 1 -- 172.21.15.32:0/4103402066 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10017070 con 0x7faa20072500 2024-04-17T09:53:31.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa1cc00640 1 -- 172.21.15.32:0/4103402066 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa10005cf0 con 0x7faa20072500 2024-04-17T09:53:31.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa27200640 1 -- 172.21.15.32:0/4103402066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa200c9fb0 con 0x7faa20072500 2024-04-17T09:53:31.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa1cc00640 1 -- 172.21.15.32:0/4103402066 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa100072d0 con 0x7faa20072500 2024-04-17T09:53:31.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.874+0000 7faa27200640 1 -- 172.21.15.32:0/4103402066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa200ca570 con 0x7faa20072500 2024-04-17T09:53:31.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.878+0000 7faa27200640 1 -- 172.21.15.32:0/4103402066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa20068a90 con 0x7faa20072500 2024-04-17T09:53:31.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.878+0000 7faa1cc00640 1 -- 172.21.15.32:0/4103402066 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7faa1001a070 con 0x7faa20072500 2024-04-17T09:53:31.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.878+0000 7faa1cc00640 1 --2- 172.21.15.32:0/4103402066 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa9fc068380 0x7fa9fc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:31.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.878+0000 7faa1cc00640 1 -- 172.21.15.32:0/4103402066 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7faa10085df0 con 0x7faa20072500 2024-04-17T09:53:31.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.878+0000 7faa17e00640 1 --2- 172.21.15.32:0/4103402066 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa9fc068380 0x7fa9fc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:31.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.878+0000 7faa17e00640 1 --2- 172.21.15.32:0/4103402066 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa9fc068380 0x7fa9fc06a840 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faa08000f70 tx=0x7faa080079e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:31.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:31.882+0000 7faa1cc00640 1 -- 172.21.15.32:0/4103402066 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7faa10055c30 con 0x7faa20072500 2024-04-17T09:53:32.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:32.082+0000 7faa27200640 1 -- 172.21.15.32:0/4103402066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7faa20072f90 con 0x7faa20072500 2024-04-17T09:53:32.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:32.082+0000 7faa1cc00640 1 -- 172.21.15.32:0/4103402066 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7faa10059f30 con 0x7faa20072500 2024-04-17T09:53:32.086 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T09:53:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:32.082+0000 7faa27200640 1 -- 172.21.15.32:0/4103402066 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa9fc068380 msgr2=0x7fa9fc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:32.082+0000 7faa27200640 1 --2- 172.21.15.32:0/4103402066 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa9fc068380 0x7fa9fc06a840 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faa08000f70 tx=0x7faa080079e0 comp rx=0 tx=0).stop 2024-04-17T09:53:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:32.082+0000 7faa27200640 1 -- 172.21.15.32:0/4103402066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa20072500 msgr2=0x7faa2015bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:32.082+0000 7faa27200640 1 --2- 172.21.15.32:0/4103402066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa20072500 0x7faa2015bc20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7faa1000c420 tx=0x7faa1000c8e0 comp rx=0 tx=0).stop 2024-04-17T09:53:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:32.082+0000 7faa27200640 1 -- 172.21.15.32:0/4103402066 shutdown_connections 2024-04-17T09:53:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:32.082+0000 7faa27200640 1 --2- 172.21.15.32:0/4103402066 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa9fc068380 0x7fa9fc06a840 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:32.082+0000 7faa27200640 1 --2- 172.21.15.32:0/4103402066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa20145f30 0x7faa20133520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:32.082+0000 7faa27200640 1 --2- 172.21.15.32:0/4103402066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa200cd580 0x7faa200c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:32.082+0000 7faa27200640 1 --2- 172.21.15.32:0/4103402066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa20072500 0x7faa2015bc20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:32.082+0000 7faa27200640 1 -- 172.21.15.32:0/4103402066 >> 172.21.15.32:0/4103402066 conn(0x7faa2006ba30 msgr2=0x7faa2015f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:32.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:32.082+0000 7faa27200640 1 -- 172.21.15.32:0/4103402066 shutdown_connections 2024-04-17T09:53:32.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:32.082+0000 7faa27200640 1 -- 172.21.15.32:0/4103402066 wait complete. 2024-04-17T09:53:32.107 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T09:53:32.108 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T09:53:32.108 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-17T09:53:32.108 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-17T09:53:32.108 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-17T09:53:32.108 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T09:53:32.108 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T09:53:32.108 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T09:53:32.108 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-17T09:53:32.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:32.108 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-17T09:53:32.118 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T09:53:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T09:53:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T09:53:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-17T09:53:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:32.118 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:53:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-17T09:53:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-17T09:53:32.118 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T09:53:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:53:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-17T09:53:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-17T09:53:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42442sec preferred_lft 42442sec 2024-04-17T09:53:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2024-04-17T09:53:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:32.119 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:53:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-17T09:53:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-17T09:53:32.119 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:53:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:53:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-17T09:53:32.119 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T09:53:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:cd:f4:aa:5b brd ff:ff:ff:ff:ff:ff 2024-04-17T09:53:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T09:53:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:32.119 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T09:53:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T09:53:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T09:53:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T09:53:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:32.120 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T09:53:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:53:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T09:53:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:32.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:32.120 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T09:53:32.182 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-17T09:53:32.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:32.184 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-17T09:53:32.241 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-17T09:53:32.241 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T09:53:32.241 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-17T09:53:32.241 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T09:53:32.241 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-17T09:53:32.242 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T09:53:32.242 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:53:32.242 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T09:53:32.242 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-17T09:53:32.242 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-17T09:53:32.242 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-17T09:53:32.242 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T09:53:32.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:53:32.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:53:32.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:32.384 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-17T09:53:32.440 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T09:53:32.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:32.442 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2024-04-17T09:53:32.492 INFO:teuthology.orchestra.run.smithi032.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-17T09:53:32.493 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:53:32.493 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T09:53:32.493 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:53:32.493 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-17T09:53:32.493 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T09:53:32.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:53:32.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:53:32.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:32.625 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-17T09:53:32.646 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T09:53:32.646 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T09:53:32.646 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T09:53:32.646 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:32.646 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-17T09:53:32.647 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:32.647 INFO:teuthology.orchestra.run.smithi032.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T09:53:32.647 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether b2:e0:a2:21:d7:a9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-17T09:53:32.647 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-17T09:53:32.647 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:32.647 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::b0e0:a2ff:fe21:d7a9/64 scope link 2024-04-17T09:53:32.647 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:32.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:32.648 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-17T09:53:32.712 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T09:53:32.712 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T09:53:32.712 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T09:53:32.712 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-17T09:53:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:32.713 INFO:teuthology.orchestra.run.smithi032.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T09:53:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether b2:e0:a2:21:d7:a9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-17T09:53:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-17T09:53:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::b0e0:a2ff:fe21:d7a9/64 scope link 2024-04-17T09:53:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:32.714 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-17T09:53:32.714 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:53:32.715 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T09:53:32.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-17T09:53:32.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-17T09:53:32.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-17T09:53:32.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-17T09:53:32.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-17T09:53:32.715 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T09:53:32.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:53:32.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:53:32.901 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:53:32.901 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T09:53:32.901 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.1 up 2024-04-17T09:53:32.901 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.1 master ceph-brx 2024-04-17T09:53:32.901 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T09:53:32.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:53:32.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:53:32.986 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-17T09:53:32.986 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:53:32.987 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-17T09:53:32.995 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-17T09:53:32.995 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:53:32.995 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-17T09:53:33.048 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T09:53:33.113 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:53:33.166 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T09:53:33.166 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T09:53:33.166 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T09:53:33.166 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T09:53:33.166 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T09:53:33.166 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T09:53:33.166 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T09:53:33.166 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T09:53:33.166 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T09:53:33.166 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T09:53:33.166 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T09:53:33.166 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T09:53:33.166 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:53:33.166 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T09:53:33.167 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T09:53:33.167 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T09:53:33.167 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T09:53:33.167 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T09:53:33.167 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T09:53:33.167 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T09:53:33.167 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:53:33.167 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T09:53:33.167 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T09:53:33.167 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T09:53:33.167 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T09:53:33.167 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T09:53:33.167 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T09:53:33.167 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T09:53:33.167 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T09:53:33.167 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:53:33.168 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:53:33.168 INFO:teuthology.orchestra.run.smithi032.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-17T09:53:33.168 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:53:33.168 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:53:33.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:33.169 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T09:53:33.220 INFO:teuthology.orchestra.run.smithi032.stdout:44 2024-04-17T09:53:33.220 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [44] 2024-04-17T09:53:33.220 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-17T09:53:33.263 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T09:53:33.283 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:53:33.338 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T09:53:33.338 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T09:53:33.338 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T09:53:33.338 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T09:53:33.338 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T09:53:33.338 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T09:53:33.339 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T09:53:33.339 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T09:53:33.339 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T09:53:33.339 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T09:53:33.339 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T09:53:33.339 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T09:53:33.339 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:53:33.339 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T09:53:33.339 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T09:53:33.339 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T09:53:33.339 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T09:53:33.339 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T09:53:33.339 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T09:53:33.339 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T09:53:33.339 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:53:33.339 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T09:53:33.340 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T09:53:33.340 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T09:53:33.340 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T09:53:33.340 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T09:53:33.340 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T09:53:33.340 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T09:53:33.340 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T09:53:33.340 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:53:33.340 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:53:33.340 INFO:teuthology.orchestra.run.smithi032.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-17T09:53:33.340 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:53:33.340 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:53:33.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:33.341 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T09:53:33.359 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi032.stderr:2024-04-17T09:53:33.358+0000 7f70ee2d8580 -1 init, newargv = 0x5633820acb10 newargc=15 2024-04-17T09:53:33.359 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi032.stderr:ceph-fuse[33532]: starting ceph client 2024-04-17T09:53:33.367 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi032.stderr:ceph-fuse[33532]: starting fuse 2024-04-17T09:53:33.367 INFO:teuthology.orchestra.run.smithi032.stdout:44 2024-04-17T09:53:33.367 INFO:teuthology.orchestra.run.smithi032.stdout:45 2024-04-17T09:53:33.368 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44, 45] 2024-04-17T09:53:33.368 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2024-04-17T09:53:33.368 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2024-04-17T09:53:33.368 DEBUG:teuthology.orchestra.run.smithi032:> import re 2024-04-17T09:53:33.368 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T09:53:33.368 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2024-04-17T09:53:33.368 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:53:33.368 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2024-04-17T09:53:33.368 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-17T09:53:33.368 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2024-04-17T09:53:33.368 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-17T09:53:33.368 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:53:33.368 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2024-04-17T09:53:33.368 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2024-04-17T09:53:33.369 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2024-04-17T09:53:33.369 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2024-04-17T09:53:33.369 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:53:33.369 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2024-04-17T09:53:33.369 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T09:53:33.369 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T09:53:33.369 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T09:53:33.369 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2024-04-17T09:53:33.369 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2024-04-17T09:53:33.369 DEBUG:teuthology.orchestra.run.smithi032:> return f 2024-04-17T09:53:33.369 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T09:53:33.369 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:53:33.369 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.2")) 2024-04-17T09:53:33.369 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T09:53:33.494 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.2.33532.asok 2024-04-17T09:53:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:53:33.505 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.33532.asok 2024-04-17T09:53:33.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:33.506 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.33532.asok status 2024-04-17T09:53:33.679 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:53:33.680 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2024-04-17T09:53:33.680 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-17T09:53:33.680 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T09:53:33.680 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "2", 2024-04-17T09:53:33.680 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2024-04-17T09:53:33.680 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-17T09:53:33.680 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "33532", 2024-04-17T09:53:33.680 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2024-04-17T09:53:33.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:53:33.680 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2024-04-17T09:53:33.680 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2024-04-17T09:53:33.680 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 4659, 2024-04-17T09:53:33.680 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2024-04-17T09:53:33.680 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2024-04-17T09:53:33.680 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2024-04-17T09:53:33.681 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 4659 2024-04-17T09:53:33.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:53:33.681 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T09:53:33.681 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T09:53:33.681 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.2:0", 2024-04-17T09:53:33.681 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1085540853 2024-04-17T09:53:33.681 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:53:33.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:53:33.681 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T09:53:33.681 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T09:53:33.681 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.2:0", 2024-04-17T09:53:33.681 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1085540853 2024-04-17T09:53:33.681 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:53:33.681 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.4659 192.168.0.2:0/1085540853", 2024-04-17T09:53:33.681 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.2:0/1085540853", 2024-04-17T09:53:33.682 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2024-04-17T09:53:33.682 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 8, 2024-04-17T09:53:33.682 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 21, 2024-04-17T09:53:33.682 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2024-04-17T09:53:33.682 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2024-04-17T09:53:33.682 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs" 2024-04-17T09:53:33.682 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:53:33.694 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T09:53:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.834+0000 7f02b0800640 1 Processor -- start 2024-04-17T09:53:33.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.834+0000 7f02b0800640 1 -- start start 2024-04-17T09:53:33.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.834+0000 7f02b0800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02a806f670 0x7f02a815bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:33.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.834+0000 7f02b0800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02a8072bb0 con 0x7f02a815baa0 2024-04-17T09:53:33.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.834+0000 7f02b0800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02a80b3dc0 con 0x7f02a8144e90 2024-04-17T09:53:33.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.834+0000 7f02b0800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f02a8145270 con 0x7f02a806f670 2024-04-17T09:53:33.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.834+0000 7f02ad400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f02a815baa0 0x7f02a806ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52670/0 (socket says 172.21.15.32:52670) 2024-04-17T09:53:33.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.834+0000 7f02ad400640 1 -- 172.21.15.32:0/2255683863 learned_addr learned my addr 172.21.15.32:0/2255683863 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:33.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.834+0000 7f02ae800640 1 --2- 172.21.15.32:0/2255683863 >> v2:172.21.15.32:3300/0 conn(0x7f02a806f670 0x7f02a815bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:33.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.838+0000 7f02aca00640 1 -- 172.21.15.32:0/2255683863 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 423676518 0 0) 0x7f02a8072bb0 con 0x7f02a815baa0 2024-04-17T09:53:33.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.838+0000 7f02ae800640 1 -- 172.21.15.32:0/2255683863 >> v1:172.21.15.144:6789/0 conn(0x7f02a8144e90 legacy=0x7f02a80b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:33.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.838+0000 7f02ae800640 1 -- 172.21.15.32:0/2255683863 >> v1:172.21.15.32:6789/0 conn(0x7f02a815baa0 legacy=0x7f02a806ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:33.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.838+0000 7f02ae800640 1 -- 172.21.15.32:0/2255683863 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a80b40a0 con 0x7f02a806f670 2024-04-17T09:53:33.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.838+0000 7f02ae800640 1 --2- 172.21.15.32:0/2255683863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02a806f670 0x7f02a815bf30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f02a4009d90 tx=0x7f02a401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918a1f7140068c24 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:33.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.838+0000 7f02aca00640 1 -- 172.21.15.32:0/2255683863 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a40b2070 con 0x7f02a806f670 2024-04-17T09:53:33.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.838+0000 7f02aca00640 1 -- 172.21.15.32:0/2255683863 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02a40a5900 con 0x7f02a806f670 2024-04-17T09:53:33.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.838+0000 7f02aca00640 1 -- 172.21.15.32:0/2255683863 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a40a5cd0 con 0x7f02a806f670 2024-04-17T09:53:33.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.838+0000 7f02b0800640 1 -- 172.21.15.32:0/2255683863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02a806f670 msgr2=0x7f02a815bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:33.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.838+0000 7f02b0800640 1 --2- 172.21.15.32:0/2255683863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02a806f670 0x7f02a815bf30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f02a4009d90 tx=0x7f02a401c930 comp rx=0 tx=0).stop 2024-04-17T09:53:33.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.838+0000 7f02b0800640 1 -- 172.21.15.32:0/2255683863 shutdown_connections 2024-04-17T09:53:33.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.838+0000 7f02b0800640 1 --2- 172.21.15.32:0/2255683863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02a806f670 0x7f02a815bf30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:33.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.838+0000 7f02b0800640 1 -- 172.21.15.32:0/2255683863 >> 172.21.15.32:0/2255683863 conn(0x7f02a806ba30 msgr2=0x7f02a806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:33.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02b0800640 1 -- 172.21.15.32:0/2255683863 shutdown_connections 2024-04-17T09:53:33.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02b0800640 1 -- 172.21.15.32:0/2255683863 wait complete. 2024-04-17T09:53:33.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02b0800640 1 Processor -- start 2024-04-17T09:53:33.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02b0800640 1 -- start start 2024-04-17T09:53:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02b0800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02a806f670 0x7f02a815af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02b0800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02a8144e90 0x7f02a806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02b0800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02a815baa0 0x7f02a806e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02b0800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f02a8169120 con 0x7f02a806f670 2024-04-17T09:53:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02b0800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f02a8072880 con 0x7f02a815baa0 2024-04-17T09:53:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02b0800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f02a81692a0 con 0x7f02a8144e90 2024-04-17T09:53:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02ade00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02a806f670 0x7f02a815af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02ade00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02a806f670 0x7f02a815af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46506/0 (socket says 172.21.15.32:46506) 2024-04-17T09:53:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02ade00640 1 -- 172.21.15.32:0/3947177210 learned_addr learned my addr 172.21.15.32:0/3947177210 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02ade00640 1 -- 172.21.15.32:0/3947177210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02a8144e90 msgr2=0x7f02a806e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02ade00640 1 --2- 172.21.15.32:0/3947177210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02a8144e90 0x7f02a806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02ade00640 1 -- 172.21.15.32:0/3947177210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02a815baa0 msgr2=0x7f02a806e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02ade00640 1 --2- 172.21.15.32:0/3947177210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02a815baa0 0x7f02a806e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02ade00640 1 -- 172.21.15.32:0/3947177210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a8072f20 con 0x7f02a806f670 2024-04-17T09:53:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02ade00640 1 --2- 172.21.15.32:0/3947177210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02a806f670 0x7f02a815af50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f029c00c450 tx=0x7f029c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02a2000640 1 -- 172.21.15.32:0/3947177210 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c012830 con 0x7f02a806f670 2024-04-17T09:53:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02a2000640 1 -- 172.21.15.32:0/3947177210 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f029c017070 con 0x7f02a806f670 2024-04-17T09:53:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02a2000640 1 -- 172.21.15.32:0/3947177210 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c012c30 con 0x7f02a806f670 2024-04-17T09:53:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02b0800640 1 -- 172.21.15.32:0/3947177210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a8142170 con 0x7f02a806f670 2024-04-17T09:53:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.842+0000 7f02b0800640 1 -- 172.21.15.32:0/3947177210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a8141bb0 con 0x7f02a806f670 2024-04-17T09:53:33.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.846+0000 7f02b0800640 1 -- 172.21.15.32:0/3947177210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a8068a90 con 0x7f02a806f670 2024-04-17T09:53:33.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.846+0000 7f02a2000640 1 -- 172.21.15.32:0/3947177210 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f029c01c460 con 0x7f02a806f670 2024-04-17T09:53:33.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.846+0000 7f02a2000640 1 --2- 172.21.15.32:0/3947177210 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f028c068380 0x7f028c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:33.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.846+0000 7f02a2000640 1 -- 172.21.15.32:0/3947177210 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f029c086890 con 0x7f02a806f670 2024-04-17T09:53:33.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.846+0000 7f02ad400640 1 --2- 172.21.15.32:0/3947177210 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f028c068380 0x7f028c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:33.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.846+0000 7f02ad400640 1 --2- 172.21.15.32:0/3947177210 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f028c068380 0x7f028c06a840 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0298005890 tx=0x7f0298007a10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:33.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:33.850+0000 7f02a2000640 1 -- 172.21.15.32:0/3947177210 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f02a8142170 con 0x7f02a806f670 2024-04-17T09:53:34.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:34.054+0000 7f02b0800640 1 -- 172.21.15.32:0/3947177210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f02a80b2db0 con 0x7f02a806f670 2024-04-17T09:53:34.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:34.054+0000 7f02a2000640 1 -- 172.21.15.32:0/3947177210 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f029c056d20 con 0x7f02a806f670 2024-04-17T09:53:34.056 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T09:53:34.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:34.058+0000 7f02b0800640 1 -- 172.21.15.32:0/3947177210 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f028c068380 msgr2=0x7f028c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:34.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:34.058+0000 7f02b0800640 1 --2- 172.21.15.32:0/3947177210 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f028c068380 0x7f028c06a840 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0298005890 tx=0x7f0298007a10 comp rx=0 tx=0).stop 2024-04-17T09:53:34.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:34.058+0000 7f02b0800640 1 -- 172.21.15.32:0/3947177210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02a806f670 msgr2=0x7f02a815af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:34.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:34.058+0000 7f02b0800640 1 --2- 172.21.15.32:0/3947177210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02a806f670 0x7f02a815af50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f029c00c450 tx=0x7f029c00c910 comp rx=0 tx=0).stop 2024-04-17T09:53:34.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:34.058+0000 7f02b0800640 1 -- 172.21.15.32:0/3947177210 shutdown_connections 2024-04-17T09:53:34.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:34.058+0000 7f02b0800640 1 --2- 172.21.15.32:0/3947177210 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f028c068380 0x7f028c06a840 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:34.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:34.058+0000 7f02b0800640 1 --2- 172.21.15.32:0/3947177210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02a8144e90 0x7f02a806e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:34.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:34.058+0000 7f02b0800640 1 --2- 172.21.15.32:0/3947177210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02a815baa0 0x7f02a806e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:34.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:34.058+0000 7f02b0800640 1 --2- 172.21.15.32:0/3947177210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02a806f670 0x7f02a815af50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:34.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:34.058+0000 7f02b0800640 1 -- 172.21.15.32:0/3947177210 >> 172.21.15.32:0/3947177210 conn(0x7f02a806ba30 msgr2=0x7f02a815e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:34.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:34.058+0000 7f02b0800640 1 -- 172.21.15.32:0/3947177210 shutdown_connections 2024-04-17T09:53:34.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:34.058+0000 7f02b0800640 1 -- 172.21.15.32:0/3947177210 wait complete. 2024-04-17T09:53:34.077 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T09:53:34.077 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T09:53:34.077 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2024-04-17T09:53:34.077 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-17T09:53:34.077 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-17T09:53:34.077 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T09:53:34.077 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T09:53:34.077 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T09:53:34.077 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-17T09:53:34.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:34.078 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2024-04-17T09:53:34.087 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T09:53:34.087 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T09:53:34.087 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T09:53:34.087 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:34.087 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2024-04-17T09:53:34.087 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:34.087 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:53:34.088 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:53:34.088 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f0 2024-04-17T09:53:34.088 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:53:34.088 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:53:34.088 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f1 2024-04-17T09:53:34.088 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:53:34.088 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2024-04-17T09:53:34.088 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T09:53:34.088 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2024-04-17T09:53:34.088 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-17T09:53:34.088 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 42464sec preferred_lft 42464sec 2024-04-17T09:53:34.088 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link 2024-04-17T09:53:34.088 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:34.088 INFO:teuthology.orchestra.run.smithi144.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T09:53:34.088 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 02:42:2d:07:b3:c5 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:53:34.088 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T09:53:34.089 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:34.089 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-17T09:53:34.089 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:53:34.089 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-04-17T09:53:34.090 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link add name ceph-brx type bridge 2024-04-17T09:53:34.090 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip addr flush dev ceph-brx 2024-04-17T09:53:34.090 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set ceph-brx up 2024-04-17T09:53:34.090 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-17T09:53:34.090 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-04-17T09:53:34.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:53:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:53:34.244 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:53:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:53:34.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:34.251 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T09:53:34.269 INFO:teuthology.orchestra.run.smithi144.stdout:1 2024-04-17T09:53:34.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:34.271 DEBUG:teuthology.orchestra.run.smithi144:> route 2024-04-17T09:53:34.343 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2024-04-17T09:53:34.344 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T09:53:34.344 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-17T09:53:34.344 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T09:53:34.344 INFO:teuthology.orchestra.run.smithi144.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-17T09:53:34.344 INFO:teuthology.orchestra.run.smithi144.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T09:53:34.344 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T09:53:34.344 INFO:teuthology.orchestra.run.smithi144.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T09:53:34.344 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T09:53:34.344 INFO:teuthology.orchestra.run.smithi144.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T09:53:34.344 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T09:53:34.344 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:53:34.345 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-04-17T09:53:34.345 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T09:53:34.345 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T09:53:34.345 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T09:53:34.345 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-04-17T09:53:34.432 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:53:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:53:34.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:53:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:53:34.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:34.479 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2024-04-17T09:53:34.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:34.489 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list-id 2024-04-17T09:53:34.539 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:53:34.539 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-04-17T09:53:34.539 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T09:53:34.539 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-17T09:53:34.539 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-04-17T09:53:34.627 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:53:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:53:34.658 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:53:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:53:34.665 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-17T09:53:34.665 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:53:34.665 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-04-17T09:53:34.665 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-17T09:53:34.665 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-17T09:53:34.665 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-17T09:53:34.665 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-17T09:53:34.665 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-17T09:53:34.666 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-04-17T09:53:34.712 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:53:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:53:34.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:53:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:53:34.831 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:53:34.831 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-04-17T09:53:34.831 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.0 up 2024-04-17T09:53:34.832 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set dev brx.0 master ceph-brx 2024-04-17T09:53:34.832 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-04-17T09:53:34.879 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:53:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:53:34.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:53:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:53:34.916 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-17T09:53:34.916 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:53:34.917 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-17T09:53:34.925 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-17T09:53:34.925 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:53:34.925 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-17T09:53:34.974 DEBUG:teuthology.orchestra.run.smithi144:> sudo modprobe fuse 2024-04-17T09:53:35.034 DEBUG:teuthology.orchestra.run.smithi144:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:53:35.084 INFO:teuthology.orchestra.run.smithi144.stdout:/sys 2024-04-17T09:53:35.084 INFO:teuthology.orchestra.run.smithi144.stdout:/proc 2024-04-17T09:53:35.084 INFO:teuthology.orchestra.run.smithi144.stdout:/dev 2024-04-17T09:53:35.084 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/pts 2024-04-17T09:53:35.084 INFO:teuthology.orchestra.run.smithi144.stdout:/run 2024-04-17T09:53:35.084 INFO:teuthology.orchestra.run.smithi144.stdout:/ 2024-04-17T09:53:35.084 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/security 2024-04-17T09:53:35.084 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/shm 2024-04-17T09:53:35.084 INFO:teuthology.orchestra.run.smithi144.stdout:/run/lock 2024-04-17T09:53:35.085 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/cgroup 2024-04-17T09:53:35.085 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/pstore 2024-04-17T09:53:35.085 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/bpf 2024-04-17T09:53:35.085 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:53:35.085 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/hugepages 2024-04-17T09:53:35.085 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/mqueue 2024-04-17T09:53:35.085 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/debug 2024-04-17T09:53:35.085 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/tracing 2024-04-17T09:53:35.085 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/fs/nfsd 2024-04-17T09:53:35.085 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/fuse/connections 2024-04-17T09:53:35.085 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/config 2024-04-17T09:53:35.085 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:53:35.085 INFO:teuthology.orchestra.run.smithi144.stdout:/run/rpc_pipefs 2024-04-17T09:53:35.085 INFO:teuthology.orchestra.run.smithi144.stdout:/run/user/1000 2024-04-17T09:53:35.085 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph 2024-04-17T09:53:35.086 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-4 2024-04-17T09:53:35.086 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-5 2024-04-17T09:53:35.086 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-6 2024-04-17T09:53:35.086 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-7 2024-04-17T09:53:35.086 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns 2024-04-17T09:53:35.086 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T09:53:35.086 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T09:53:35.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:35.087 DEBUG:teuthology.orchestra.run.smithi144:> ls /sys/fs/fuse/connections 2024-04-17T09:53:35.142 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T09:53:35.142 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-17T09:53:35.185 DEBUG:teuthology.orchestra.run.smithi144:> sudo modprobe fuse 2024-04-17T09:53:35.205 DEBUG:teuthology.orchestra.run.smithi144:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:53:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:/sys 2024-04-17T09:53:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:/proc 2024-04-17T09:53:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:/dev 2024-04-17T09:53:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/pts 2024-04-17T09:53:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:/run 2024-04-17T09:53:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:/ 2024-04-17T09:53:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/security 2024-04-17T09:53:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/shm 2024-04-17T09:53:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:/run/lock 2024-04-17T09:53:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/cgroup 2024-04-17T09:53:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/pstore 2024-04-17T09:53:35.257 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/bpf 2024-04-17T09:53:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:53:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/hugepages 2024-04-17T09:53:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/mqueue 2024-04-17T09:53:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/debug 2024-04-17T09:53:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/tracing 2024-04-17T09:53:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/fs/nfsd 2024-04-17T09:53:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/fuse/connections 2024-04-17T09:53:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/config 2024-04-17T09:53:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:53:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:/run/rpc_pipefs 2024-04-17T09:53:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:/run/user/1000 2024-04-17T09:53:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph 2024-04-17T09:53:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-4 2024-04-17T09:53:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-5 2024-04-17T09:53:35.258 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-6 2024-04-17T09:53:35.259 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-7 2024-04-17T09:53:35.259 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns 2024-04-17T09:53:35.259 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T09:53:35.259 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T09:53:35.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:35.260 DEBUG:teuthology.orchestra.run.smithi144:> ls /sys/fs/fuse/connections 2024-04-17T09:53:35.280 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi144.stderr:2024-04-17T09:53:35.279+0000 7fcfa0709580 -1 init, newargv = 0x55933920ac10 newargc=15 2024-04-17T09:53:35.280 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi144.stderr:ceph-fuse[32735]: starting ceph client 2024-04-17T09:53:35.288 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi144.stderr:ceph-fuse[32735]: starting fuse 2024-04-17T09:53:36.389 DEBUG:teuthology.orchestra.run.smithi144:> sudo modprobe fuse 2024-04-17T09:53:36.413 DEBUG:teuthology.orchestra.run.smithi144:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:53:36.465 INFO:teuthology.orchestra.run.smithi144.stdout:/sys 2024-04-17T09:53:36.465 INFO:teuthology.orchestra.run.smithi144.stdout:/proc 2024-04-17T09:53:36.465 INFO:teuthology.orchestra.run.smithi144.stdout:/dev 2024-04-17T09:53:36.465 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/pts 2024-04-17T09:53:36.466 INFO:teuthology.orchestra.run.smithi144.stdout:/run 2024-04-17T09:53:36.466 INFO:teuthology.orchestra.run.smithi144.stdout:/ 2024-04-17T09:53:36.466 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/security 2024-04-17T09:53:36.466 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/shm 2024-04-17T09:53:36.466 INFO:teuthology.orchestra.run.smithi144.stdout:/run/lock 2024-04-17T09:53:36.466 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/cgroup 2024-04-17T09:53:36.466 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/pstore 2024-04-17T09:53:36.466 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/bpf 2024-04-17T09:53:36.466 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:53:36.466 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/hugepages 2024-04-17T09:53:36.466 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/mqueue 2024-04-17T09:53:36.466 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/debug 2024-04-17T09:53:36.466 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/tracing 2024-04-17T09:53:36.467 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/fs/nfsd 2024-04-17T09:53:36.467 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/fuse/connections 2024-04-17T09:53:36.467 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/config 2024-04-17T09:53:36.467 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:53:36.467 INFO:teuthology.orchestra.run.smithi144.stdout:/run/rpc_pipefs 2024-04-17T09:53:36.467 INFO:teuthology.orchestra.run.smithi144.stdout:/run/user/1000 2024-04-17T09:53:36.467 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph 2024-04-17T09:53:36.467 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-4 2024-04-17T09:53:36.467 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-5 2024-04-17T09:53:36.467 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-6 2024-04-17T09:53:36.467 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-7 2024-04-17T09:53:36.467 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns 2024-04-17T09:53:36.467 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T09:53:36.467 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T09:53:36.467 INFO:teuthology.orchestra.run.smithi144.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-17T09:53:36.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:36.468 DEBUG:teuthology.orchestra.run.smithi144:> ls /sys/fs/fuse/connections 2024-04-17T09:53:36.523 INFO:teuthology.orchestra.run.smithi144.stdout:44 2024-04-17T09:53:36.524 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-17T09:53:36.524 DEBUG:teuthology.orchestra.run.smithi144:> sudo stdin-killer -- python3 -c ' 2024-04-17T09:53:36.524 DEBUG:teuthology.orchestra.run.smithi144:> import glob 2024-04-17T09:53:36.524 DEBUG:teuthology.orchestra.run.smithi144:> import re 2024-04-17T09:53:36.524 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-04-17T09:53:36.524 DEBUG:teuthology.orchestra.run.smithi144:> import subprocess 2024-04-17T09:53:36.524 DEBUG:teuthology.orchestra.run.smithi144:> 2024-04-17T09:53:36.524 DEBUG:teuthology.orchestra.run.smithi144:> def _find_admin_socket(client_name): 2024-04-17T09:53:36.524 DEBUG:teuthology.orchestra.run.smithi144:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-17T09:53:36.524 DEBUG:teuthology.orchestra.run.smithi144:> files = glob.glob(asok_path) 2024-04-17T09:53:36.524 DEBUG:teuthology.orchestra.run.smithi144:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-17T09:53:36.524 DEBUG:teuthology.orchestra.run.smithi144:> 2024-04-17T09:53:36.524 DEBUG:teuthology.orchestra.run.smithi144:> # Given a non-glob path, it better be there 2024-04-17T09:53:36.525 DEBUG:teuthology.orchestra.run.smithi144:> if "*" not in asok_path: 2024-04-17T09:53:36.525 DEBUG:teuthology.orchestra.run.smithi144:> assert(len(files) == 1) 2024-04-17T09:53:36.525 DEBUG:teuthology.orchestra.run.smithi144:> return files[0] 2024-04-17T09:53:36.525 DEBUG:teuthology.orchestra.run.smithi144:> 2024-04-17T09:53:36.525 DEBUG:teuthology.orchestra.run.smithi144:> for f in files: 2024-04-17T09:53:36.525 DEBUG:teuthology.orchestra.run.smithi144:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T09:53:36.525 DEBUG:teuthology.orchestra.run.smithi144:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T09:53:36.525 DEBUG:teuthology.orchestra.run.smithi144:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T09:53:36.525 DEBUG:teuthology.orchestra.run.smithi144:> contents = proc_f.read() 2024-04-17T09:53:36.525 DEBUG:teuthology.orchestra.run.smithi144:> if mountpoint in contents: 2024-04-17T09:53:36.525 DEBUG:teuthology.orchestra.run.smithi144:> return f 2024-04-17T09:53:36.525 DEBUG:teuthology.orchestra.run.smithi144:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T09:53:36.525 DEBUG:teuthology.orchestra.run.smithi144:> 2024-04-17T09:53:36.525 DEBUG:teuthology.orchestra.run.smithi144:> print(_find_admin_socket("client.1")) 2024-04-17T09:53:36.525 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-04-17T09:53:36.658 INFO:teuthology.orchestra.run.smithi144.stdout:/var/run/ceph/ceph-client.1.32735.asok 2024-04-17T09:53:36.662 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:53:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:53:36.671 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.32735.asok 2024-04-17T09:53:36.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:36.671 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.32735.asok status 2024-04-17T09:53:36.845 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-04-17T09:53:36.845 INFO:teuthology.orchestra.run.smithi144.stdout: "metadata": { 2024-04-17T09:53:36.845 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-17T09:53:36.845 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T09:53:36.845 INFO:teuthology.orchestra.run.smithi144.stdout: "entity_id": "1", 2024-04-17T09:53:36.845 INFO:teuthology.orchestra.run.smithi144.stdout: "hostname": "smithi144", 2024-04-17T09:53:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-17T09:53:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: "pid": "32735", 2024-04-17T09:53:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: "root": "/" 2024-04-17T09:53:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-04-17T09:53:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: "dentry_count": 0, 2024-04-17T09:53:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: "dentry_pinned_count": 0, 2024-04-17T09:53:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: "id": 4648, 2024-04-17T09:53:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: "inst": { 2024-04-17T09:53:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: "name": { 2024-04-17T09:53:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "client", 2024-04-17T09:53:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: "num": 4648 2024-04-17T09:53:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-04-17T09:53:36.846 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": { 2024-04-17T09:53:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "v1", 2024-04-17T09:53:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": "192.168.0.1:0", 2024-04-17T09:53:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: "nonce": 1630885571 2024-04-17T09:53:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-04-17T09:53:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-04-17T09:53:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": { 2024-04-17T09:53:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "v1", 2024-04-17T09:53:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": "192.168.0.1:0", 2024-04-17T09:53:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: "nonce": 1630885571 2024-04-17T09:53:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-04-17T09:53:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: "inst_str": "client.4648 192.168.0.1:0/1630885571", 2024-04-17T09:53:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: "addr_str": "192.168.0.1:0/1630885571", 2024-04-17T09:53:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: "inode_count": 1, 2024-04-17T09:53:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_epoch": 8, 2024-04-17T09:53:36.847 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_epoch": 21, 2024-04-17T09:53:36.848 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_epoch_barrier": 0, 2024-04-17T09:53:36.848 INFO:teuthology.orchestra.run.smithi144.stdout: "blocklisted": false, 2024-04-17T09:53:36.848 INFO:teuthology.orchestra.run.smithi144.stdout: "fs_name": "cephfs" 2024-04-17T09:53:36.848 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-04-17T09:53:36.861 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T09:53:37.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.014+0000 7f092c600640 1 Processor -- start 2024-04-17T09:53:37.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.014+0000 7f092c600640 1 -- start start 2024-04-17T09:53:37.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.014+0000 7f092c600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f092415c8c0 0x7f0924160c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:37.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.014+0000 7f092c600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f09240d0100 0x7f092406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:37.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.014+0000 7f092c600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09240b4e20 con 0x7f09240cd040 2024-04-17T09:53:37.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.014+0000 7f092c600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f09240d1a50 con 0x7f092415c8c0 2024-04-17T09:53:37.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.014+0000 7f092c600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f09240cd4a0 con 0x7f09240d0100 2024-04-17T09:53:37.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.014+0000 7f0929c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f092415c8c0 0x7f0924160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:37.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.014+0000 7f0929200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f09240cd040 0x7f09240cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52676/0 (socket says 172.21.15.32:52676) 2024-04-17T09:53:37.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.014+0000 7f0929200640 1 -- 172.21.15.32:0/3286090312 learned_addr learned my addr 172.21.15.32:0/3286090312 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:37.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.014+0000 7f092a600640 1 --2- 172.21.15.32:0/3286090312 >> v2:172.21.15.144:3300/0 conn(0x7f09240d0100 0x7f092406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:37.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.014+0000 7f0929c00640 1 -- 172.21.15.32:0/3286090312 >> v2:172.21.15.144:3300/0 conn(0x7f09240d0100 msgr2=0x7f092406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:37.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.014+0000 7f0929c00640 1 --2- 172.21.15.32:0/3286090312 >> v2:172.21.15.144:3300/0 conn(0x7f09240d0100 0x7f092406ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:37.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.014+0000 7f0929c00640 1 -- 172.21.15.32:0/3286090312 >> v1:172.21.15.32:6789/0 conn(0x7f09240cd040 legacy=0x7f09240cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:37.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.014+0000 7f0929c00640 1 -- 172.21.15.32:0/3286090312 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09240b50a0 con 0x7f092415c8c0 2024-04-17T09:53:37.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.014+0000 7f0923e00640 1 -- 172.21.15.32:0/3286090312 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 234938477 0 0) 0x7f0910001120 con 0x7f09240cd040 2024-04-17T09:53:37.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.018+0000 7f0929c00640 1 --2- 172.21.15.32:0/3286090312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f092415c8c0 0x7f0924160c50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0924072210 tx=0x7f091801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930e13693602fb62 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:37.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.018+0000 7f0923e00640 1 -- 172.21.15.32:0/3286090312 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09180b0070 con 0x7f092415c8c0 2024-04-17T09:53:37.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.018+0000 7f0923e00640 1 -- 172.21.15.32:0/3286090312 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09180ab600 con 0x7f092415c8c0 2024-04-17T09:53:37.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.018+0000 7f0923e00640 1 -- 172.21.15.32:0/3286090312 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09180ab9d0 con 0x7f092415c8c0 2024-04-17T09:53:37.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.018+0000 7f092c600640 1 -- 172.21.15.32:0/3286090312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f092415c8c0 msgr2=0x7f0924160c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:37.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.018+0000 7f092c600640 1 --2- 172.21.15.32:0/3286090312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f092415c8c0 0x7f0924160c50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0924072210 tx=0x7f091801c480 comp rx=0 tx=0).stop 2024-04-17T09:53:37.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.018+0000 7f092c600640 1 -- 172.21.15.32:0/3286090312 shutdown_connections 2024-04-17T09:53:37.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.018+0000 7f092c600640 1 --2- 172.21.15.32:0/3286090312 >> v2:172.21.15.144:3300/0 conn(0x7f09240d0100 0x7f092406ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:37.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.018+0000 7f092c600640 1 --2- 172.21.15.32:0/3286090312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f092415c8c0 0x7f0924160c50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:37.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.018+0000 7f092c600640 1 -- 172.21.15.32:0/3286090312 >> 172.21.15.32:0/3286090312 conn(0x7f09240d0c00 msgr2=0x7f09240d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:37.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.018+0000 7f092c600640 1 -- 172.21.15.32:0/3286090312 shutdown_connections 2024-04-17T09:53:37.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.018+0000 7f092c600640 1 -- 172.21.15.32:0/3286090312 wait complete. 2024-04-17T09:53:37.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f092c600640 1 Processor -- start 2024-04-17T09:53:37.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f092c600640 1 -- start start 2024-04-17T09:53:37.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f092c600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09240cd040 0x7f092415cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:37.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f092c600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09240d0100 0x7f09240af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:37.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f092c600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f092415c8c0 0x7f09240af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:37.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f092c600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f092406c780 con 0x7f09240cd040 2024-04-17T09:53:37.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f092c600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f09240b5e60 con 0x7f092415c8c0 2024-04-17T09:53:37.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f092c600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f092406c900 con 0x7f09240d0100 2024-04-17T09:53:37.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f0929c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09240cd040 0x7f092415cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:37.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f0929c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09240cd040 0x7f092415cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46528/0 (socket says 172.21.15.32:46528) 2024-04-17T09:53:37.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f0929c00640 1 -- 172.21.15.32:0/1345736584 learned_addr learned my addr 172.21.15.32:0/1345736584 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:37.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f0929200640 1 --2- 172.21.15.32:0/1345736584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09240d0100 0x7f09240af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:37.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f092a600640 1 --2- 172.21.15.32:0/1345736584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f092415c8c0 0x7f09240af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:37.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f0929c00640 1 -- 172.21.15.32:0/1345736584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09240d0100 msgr2=0x7f09240af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:37.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f0929c00640 1 --2- 172.21.15.32:0/1345736584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09240d0100 0x7f09240af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:37.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f0929c00640 1 -- 172.21.15.32:0/1345736584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f092415c8c0 msgr2=0x7f09240af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:37.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f0929c00640 1 --2- 172.21.15.32:0/1345736584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f092415c8c0 0x7f09240af980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:37.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f0929c00640 1 -- 172.21.15.32:0/1345736584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f092415d410 con 0x7f09240cd040 2024-04-17T09:53:37.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f0929c00640 1 --2- 172.21.15.32:0/1345736584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09240cd040 0x7f092415cca0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f09180ae040 tx=0x7f091801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:37.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f0921600640 1 -- 172.21.15.32:0/1345736584 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09180b0050 con 0x7f09240cd040 2024-04-17T09:53:37.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f0921600640 1 -- 172.21.15.32:0/1345736584 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09180bc070 con 0x7f09240cd040 2024-04-17T09:53:37.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f092c600640 1 -- 172.21.15.32:0/1345736584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f092415d6f0 con 0x7f09240cd040 2024-04-17T09:53:37.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f0921600640 1 -- 172.21.15.32:0/1345736584 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09180b5870 con 0x7f09240cd040 2024-04-17T09:53:37.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.022+0000 7f092c600640 1 -- 172.21.15.32:0/1345736584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f092415dcb0 con 0x7f09240cd040 2024-04-17T09:53:37.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.026+0000 7f092c600640 1 -- 172.21.15.32:0/1345736584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0924068a90 con 0x7f09240cd040 2024-04-17T09:53:37.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.026+0000 7f0921600640 1 -- 172.21.15.32:0/1345736584 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f09180b5a10 con 0x7f09240cd040 2024-04-17T09:53:37.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.026+0000 7f0921600640 1 --2- 172.21.15.32:0/1345736584 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0904068380 0x7f090406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:37.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.026+0000 7f0921600640 1 -- 172.21.15.32:0/1345736584 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f09181211b0 con 0x7f09240cd040 2024-04-17T09:53:37.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.026+0000 7f0929200640 1 --2- 172.21.15.32:0/1345736584 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0904068380 0x7f090406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:37.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.026+0000 7f0929200640 1 --2- 172.21.15.32:0/1345736584 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0904068380 0x7f090406a840 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f09240b0190 tx=0x7f0910007df0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:37.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.030+0000 7f0921600640 1 -- 172.21.15.32:0/1345736584 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0918016070 con 0x7f09240cd040 2024-04-17T09:53:37.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.234+0000 7f092c600640 1 -- 172.21.15.32:0/1345736584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0924069370 con 0x7f09240cd040 2024-04-17T09:53:37.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.234+0000 7f0921600640 1 -- 172.21.15.32:0/1345736584 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f09180f1080 con 0x7f09240cd040 2024-04-17T09:53:37.234 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T09:53:37.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.234+0000 7f092c600640 1 -- 172.21.15.32:0/1345736584 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0904068380 msgr2=0x7f090406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:37.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.234+0000 7f092c600640 1 --2- 172.21.15.32:0/1345736584 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0904068380 0x7f090406a840 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f09240b0190 tx=0x7f0910007df0 comp rx=0 tx=0).stop 2024-04-17T09:53:37.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.234+0000 7f092c600640 1 -- 172.21.15.32:0/1345736584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09240cd040 msgr2=0x7f092415cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:37.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.234+0000 7f092c600640 1 --2- 172.21.15.32:0/1345736584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09240cd040 0x7f092415cca0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f09180ae040 tx=0x7f091801acc0 comp rx=0 tx=0).stop 2024-04-17T09:53:37.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.234+0000 7f092c600640 1 -- 172.21.15.32:0/1345736584 shutdown_connections 2024-04-17T09:53:37.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.234+0000 7f092c600640 1 --2- 172.21.15.32:0/1345736584 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0904068380 0x7f090406a840 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:37.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.234+0000 7f092c600640 1 --2- 172.21.15.32:0/1345736584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09240d0100 0x7f09240af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:37.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.234+0000 7f092c600640 1 --2- 172.21.15.32:0/1345736584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f092415c8c0 0x7f09240af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:37.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.234+0000 7f092c600640 1 --2- 172.21.15.32:0/1345736584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09240cd040 0x7f092415cca0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:37.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.234+0000 7f092c600640 1 -- 172.21.15.32:0/1345736584 >> 172.21.15.32:0/1345736584 conn(0x7f09240d0c00 msgr2=0x7f09240b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:37.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.234+0000 7f092c600640 1 -- 172.21.15.32:0/1345736584 shutdown_connections 2024-04-17T09:53:37.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:37.234+0000 7f092c600640 1 -- 172.21.15.32:0/1345736584 wait complete. 2024-04-17T09:53:37.255 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T09:53:37.255 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T09:53:37.255 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2024-04-17T09:53:37.255 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-17T09:53:37.255 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-17T09:53:37.255 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T09:53:37.255 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T09:53:37.255 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T09:53:37.255 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-17T09:53:37.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:37.255 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2024-04-17T09:53:37.265 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T09:53:37.265 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T09:53:37.265 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T09:53:37.265 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:37.266 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2024-04-17T09:53:37.266 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:37.266 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:53:37.266 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:53:37.266 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f0 2024-04-17T09:53:37.266 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:53:37.266 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:53:37.266 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f1 2024-04-17T09:53:37.266 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:53:37.266 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2024-04-17T09:53:37.266 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T09:53:37.266 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2024-04-17T09:53:37.266 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-17T09:53:37.267 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 42460sec preferred_lft 42460sec 2024-04-17T09:53:37.267 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link 2024-04-17T09:53:37.267 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:37.267 INFO:teuthology.orchestra.run.smithi144.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T09:53:37.267 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 02:42:2d:07:b3:c5 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:53:37.267 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T09:53:37.267 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:37.267 INFO:teuthology.orchestra.run.smithi144.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T09:53:37.267 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T09:53:37.267 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T09:53:37.267 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:37.267 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T09:53:37.267 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:37.267 INFO:teuthology.orchestra.run.smithi144.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T09:53:37.268 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T09:53:37.268 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T09:53:37.268 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:37.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:37.268 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T09:53:37.324 INFO:teuthology.orchestra.run.smithi144.stdout:1 2024-04-17T09:53:37.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:37.327 DEBUG:teuthology.orchestra.run.smithi144:> route 2024-04-17T09:53:37.383 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2024-04-17T09:53:37.383 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T09:53:37.383 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-17T09:53:37.383 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T09:53:37.383 INFO:teuthology.orchestra.run.smithi144.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-17T09:53:37.383 INFO:teuthology.orchestra.run.smithi144.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T09:53:37.383 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T09:53:37.383 INFO:teuthology.orchestra.run.smithi144.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T09:53:37.383 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T09:53:37.384 INFO:teuthology.orchestra.run.smithi144.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-17T09:53:37.384 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T09:53:37.384 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:53:37.384 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-04-17T09:53:37.384 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T09:53:37.384 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T09:53:37.384 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T09:53:37.384 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-04-17T09:53:37.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:53:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:53:37.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:53:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:53:37.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:37.522 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2024-04-17T09:53:37.571 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-17T09:53:37.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:37.572 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list-id 2024-04-17T09:53:37.622 INFO:teuthology.orchestra.run.smithi144.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-17T09:53:37.623 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:53:37.623 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-04-17T09:53:37.623 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-17T09:53:37.623 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-17T09:53:37.623 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-04-17T09:53:37.712 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:53:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:53:37.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:53:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:53:37.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:37.749 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-17T09:53:37.769 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T09:53:37.769 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T09:53:37.769 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T09:53:37.769 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:37.769 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2024-04-17T09:53:37.769 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:37.769 INFO:teuthology.orchestra.run.smithi144.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T09:53:37.769 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 7e:18:7f:3a:b4:e3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-17T09:53:37.770 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-17T09:53:37.770 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:37.770 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::7c18:7fff:fe3a:b4e3/64 scope link 2024-04-17T09:53:37.770 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:37.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:37.771 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-17T09:53:37.834 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T09:53:37.834 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T09:53:37.834 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T09:53:37.834 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:37.835 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2024-04-17T09:53:37.835 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:37.835 INFO:teuthology.orchestra.run.smithi144.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T09:53:37.835 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 7e:18:7f:3a:b4:e3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-17T09:53:37.835 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-17T09:53:37.835 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:37.835 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::7c18:7fff:fe3a:b4e3/64 scope link 2024-04-17T09:53:37.835 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:53:37.836 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-17T09:53:37.836 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:53:37.836 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-04-17T09:53:37.837 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-17T09:53:37.837 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-17T09:53:37.837 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-17T09:53:37.837 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-17T09:53:37.837 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-17T09:53:37.837 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-04-17T09:53:37.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:53:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:53:38.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:53:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:53:38.016 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:53:38.016 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-04-17T09:53:38.016 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.1 up 2024-04-17T09:53:38.017 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set dev brx.1 master ceph-brx 2024-04-17T09:53:38.017 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-04-17T09:53:38.064 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:53:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:53:38.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:53:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:53:38.100 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-17T09:53:38.100 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:53:38.100 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-17T09:53:38.154 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-17T09:53:38.155 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:53:38.155 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-17T09:53:38.206 DEBUG:teuthology.orchestra.run.smithi144:> sudo modprobe fuse 2024-04-17T09:53:38.266 DEBUG:teuthology.orchestra.run.smithi144:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:53:38.320 INFO:teuthology.orchestra.run.smithi144.stdout:/sys 2024-04-17T09:53:38.320 INFO:teuthology.orchestra.run.smithi144.stdout:/proc 2024-04-17T09:53:38.320 INFO:teuthology.orchestra.run.smithi144.stdout:/dev 2024-04-17T09:53:38.320 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/pts 2024-04-17T09:53:38.320 INFO:teuthology.orchestra.run.smithi144.stdout:/run 2024-04-17T09:53:38.320 INFO:teuthology.orchestra.run.smithi144.stdout:/ 2024-04-17T09:53:38.320 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/security 2024-04-17T09:53:38.320 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/shm 2024-04-17T09:53:38.320 INFO:teuthology.orchestra.run.smithi144.stdout:/run/lock 2024-04-17T09:53:38.320 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/cgroup 2024-04-17T09:53:38.321 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/pstore 2024-04-17T09:53:38.321 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/bpf 2024-04-17T09:53:38.321 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:53:38.321 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/hugepages 2024-04-17T09:53:38.321 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/mqueue 2024-04-17T09:53:38.321 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/debug 2024-04-17T09:53:38.321 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/tracing 2024-04-17T09:53:38.321 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/fs/nfsd 2024-04-17T09:53:38.321 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/fuse/connections 2024-04-17T09:53:38.321 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/config 2024-04-17T09:53:38.321 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:53:38.321 INFO:teuthology.orchestra.run.smithi144.stdout:/run/rpc_pipefs 2024-04-17T09:53:38.321 INFO:teuthology.orchestra.run.smithi144.stdout:/run/user/1000 2024-04-17T09:53:38.321 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph 2024-04-17T09:53:38.321 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-4 2024-04-17T09:53:38.322 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-5 2024-04-17T09:53:38.322 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-6 2024-04-17T09:53:38.322 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-7 2024-04-17T09:53:38.322 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns 2024-04-17T09:53:38.322 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T09:53:38.322 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T09:53:38.322 INFO:teuthology.orchestra.run.smithi144.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-17T09:53:38.322 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-17T09:53:38.322 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-17T09:53:38.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:38.323 DEBUG:teuthology.orchestra.run.smithi144:> ls /sys/fs/fuse/connections 2024-04-17T09:53:38.374 INFO:teuthology.orchestra.run.smithi144.stdout:44 2024-04-17T09:53:38.374 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [44] 2024-04-17T09:53:38.374 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-17T09:53:38.417 DEBUG:teuthology.orchestra.run.smithi144:> sudo modprobe fuse 2024-04-17T09:53:38.437 DEBUG:teuthology.orchestra.run.smithi144:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:53:38.488 INFO:teuthology.orchestra.run.smithi144.stdout:/sys 2024-04-17T09:53:38.488 INFO:teuthology.orchestra.run.smithi144.stdout:/proc 2024-04-17T09:53:38.488 INFO:teuthology.orchestra.run.smithi144.stdout:/dev 2024-04-17T09:53:38.488 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/pts 2024-04-17T09:53:38.489 INFO:teuthology.orchestra.run.smithi144.stdout:/run 2024-04-17T09:53:38.489 INFO:teuthology.orchestra.run.smithi144.stdout:/ 2024-04-17T09:53:38.489 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/security 2024-04-17T09:53:38.489 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/shm 2024-04-17T09:53:38.489 INFO:teuthology.orchestra.run.smithi144.stdout:/run/lock 2024-04-17T09:53:38.489 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/cgroup 2024-04-17T09:53:38.489 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/pstore 2024-04-17T09:53:38.489 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/bpf 2024-04-17T09:53:38.489 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:53:38.489 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/hugepages 2024-04-17T09:53:38.489 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/mqueue 2024-04-17T09:53:38.489 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/debug 2024-04-17T09:53:38.489 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/tracing 2024-04-17T09:53:38.489 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/fs/nfsd 2024-04-17T09:53:38.489 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/fuse/connections 2024-04-17T09:53:38.490 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/config 2024-04-17T09:53:38.490 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:53:38.490 INFO:teuthology.orchestra.run.smithi144.stdout:/run/rpc_pipefs 2024-04-17T09:53:38.490 INFO:teuthology.orchestra.run.smithi144.stdout:/run/user/1000 2024-04-17T09:53:38.490 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph 2024-04-17T09:53:38.490 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-4 2024-04-17T09:53:38.490 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-5 2024-04-17T09:53:38.490 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-6 2024-04-17T09:53:38.490 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-7 2024-04-17T09:53:38.490 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns 2024-04-17T09:53:38.490 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T09:53:38.490 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T09:53:38.490 INFO:teuthology.orchestra.run.smithi144.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-17T09:53:38.490 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-17T09:53:38.491 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-17T09:53:38.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:38.492 DEBUG:teuthology.orchestra.run.smithi144:> ls /sys/fs/fuse/connections 2024-04-17T09:53:38.511 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi144.stderr:2024-04-17T09:53:38.507+0000 7fcba420d580 -1 init, newargv = 0x5634d5473f20 newargc=15 2024-04-17T09:53:38.511 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi144.stderr:ceph-fuse[32854]: starting ceph client 2024-04-17T09:53:38.518 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi144.stderr:ceph-fuse[32854]: starting fuse 2024-04-17T09:53:38.520 INFO:teuthology.orchestra.run.smithi144.stdout:44 2024-04-17T09:53:38.520 INFO:teuthology.orchestra.run.smithi144.stdout:45 2024-04-17T09:53:38.520 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44, 45] 2024-04-17T09:53:38.520 DEBUG:teuthology.orchestra.run.smithi144:> sudo stdin-killer -- python3 -c ' 2024-04-17T09:53:38.520 DEBUG:teuthology.orchestra.run.smithi144:> import glob 2024-04-17T09:53:38.520 DEBUG:teuthology.orchestra.run.smithi144:> import re 2024-04-17T09:53:38.520 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-04-17T09:53:38.520 DEBUG:teuthology.orchestra.run.smithi144:> import subprocess 2024-04-17T09:53:38.520 DEBUG:teuthology.orchestra.run.smithi144:> 2024-04-17T09:53:38.520 DEBUG:teuthology.orchestra.run.smithi144:> def _find_admin_socket(client_name): 2024-04-17T09:53:38.520 DEBUG:teuthology.orchestra.run.smithi144:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-17T09:53:38.521 DEBUG:teuthology.orchestra.run.smithi144:> files = glob.glob(asok_path) 2024-04-17T09:53:38.521 DEBUG:teuthology.orchestra.run.smithi144:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-17T09:53:38.521 DEBUG:teuthology.orchestra.run.smithi144:> 2024-04-17T09:53:38.521 DEBUG:teuthology.orchestra.run.smithi144:> # Given a non-glob path, it better be there 2024-04-17T09:53:38.521 DEBUG:teuthology.orchestra.run.smithi144:> if "*" not in asok_path: 2024-04-17T09:53:38.521 DEBUG:teuthology.orchestra.run.smithi144:> assert(len(files) == 1) 2024-04-17T09:53:38.521 DEBUG:teuthology.orchestra.run.smithi144:> return files[0] 2024-04-17T09:53:38.521 DEBUG:teuthology.orchestra.run.smithi144:> 2024-04-17T09:53:38.521 DEBUG:teuthology.orchestra.run.smithi144:> for f in files: 2024-04-17T09:53:38.521 DEBUG:teuthology.orchestra.run.smithi144:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T09:53:38.521 DEBUG:teuthology.orchestra.run.smithi144:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T09:53:38.521 DEBUG:teuthology.orchestra.run.smithi144:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T09:53:38.521 DEBUG:teuthology.orchestra.run.smithi144:> contents = proc_f.read() 2024-04-17T09:53:38.521 DEBUG:teuthology.orchestra.run.smithi144:> if mountpoint in contents: 2024-04-17T09:53:38.521 DEBUG:teuthology.orchestra.run.smithi144:> return f 2024-04-17T09:53:38.521 DEBUG:teuthology.orchestra.run.smithi144:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T09:53:38.521 DEBUG:teuthology.orchestra.run.smithi144:> 2024-04-17T09:53:38.522 DEBUG:teuthology.orchestra.run.smithi144:> print(_find_admin_socket("client.3")) 2024-04-17T09:53:38.522 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-04-17T09:53:38.642 INFO:teuthology.orchestra.run.smithi144.stdout:/var/run/ceph/ceph-client.3.32854.asok 2024-04-17T09:53:38.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-17T09:53:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:53:38.653 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.32854.asok 2024-04-17T09:53:38.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:38.654 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.32854.asok status 2024-04-17T09:53:38.821 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-04-17T09:53:38.821 INFO:teuthology.orchestra.run.smithi144.stdout: "metadata": { 2024-04-17T09:53:38.822 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-17T09:53:38.822 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T09:53:38.822 INFO:teuthology.orchestra.run.smithi144.stdout: "entity_id": "3", 2024-04-17T09:53:38.822 INFO:teuthology.orchestra.run.smithi144.stdout: "hostname": "smithi144", 2024-04-17T09:53:38.822 INFO:teuthology.orchestra.run.smithi144.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-17T09:53:38.822 INFO:teuthology.orchestra.run.smithi144.stdout: "pid": "32854", 2024-04-17T09:53:38.822 INFO:teuthology.orchestra.run.smithi144.stdout: "root": "/" 2024-04-17T09:53:38.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-04-17T09:53:38.822 INFO:teuthology.orchestra.run.smithi144.stdout: "dentry_count": 0, 2024-04-17T09:53:38.822 INFO:teuthology.orchestra.run.smithi144.stdout: "dentry_pinned_count": 0, 2024-04-17T09:53:38.822 INFO:teuthology.orchestra.run.smithi144.stdout: "id": 4547, 2024-04-17T09:53:38.822 INFO:teuthology.orchestra.run.smithi144.stdout: "inst": { 2024-04-17T09:53:38.822 INFO:teuthology.orchestra.run.smithi144.stdout: "name": { 2024-04-17T09:53:38.823 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "client", 2024-04-17T09:53:38.823 INFO:teuthology.orchestra.run.smithi144.stdout: "num": 4547 2024-04-17T09:53:38.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-04-17T09:53:38.823 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": { 2024-04-17T09:53:38.823 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "v1", 2024-04-17T09:53:38.823 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": "192.168.0.2:0", 2024-04-17T09:53:38.823 INFO:teuthology.orchestra.run.smithi144.stdout: "nonce": 3297623950 2024-04-17T09:53:38.823 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-04-17T09:53:38.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-04-17T09:53:38.823 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": { 2024-04-17T09:53:38.823 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "v1", 2024-04-17T09:53:38.823 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": "192.168.0.2:0", 2024-04-17T09:53:38.823 INFO:teuthology.orchestra.run.smithi144.stdout: "nonce": 3297623950 2024-04-17T09:53:38.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-04-17T09:53:38.823 INFO:teuthology.orchestra.run.smithi144.stdout: "inst_str": "client.4547 192.168.0.2:0/3297623950", 2024-04-17T09:53:38.824 INFO:teuthology.orchestra.run.smithi144.stdout: "addr_str": "192.168.0.2:0/3297623950", 2024-04-17T09:53:38.824 INFO:teuthology.orchestra.run.smithi144.stdout: "inode_count": 1, 2024-04-17T09:53:38.824 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_epoch": 8, 2024-04-17T09:53:38.824 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_epoch": 21, 2024-04-17T09:53:38.824 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_epoch_barrier": 0, 2024-04-17T09:53:38.824 INFO:teuthology.orchestra.run.smithi144.stdout: "blocklisted": false, 2024-04-17T09:53:38.824 INFO:teuthology.orchestra.run.smithi144.stdout: "fs_name": "cephfs" 2024-04-17T09:53:38.824 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-04-17T09:53:38.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:38.836 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2024-04-17T09:53:38.836 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T09:53:38.847 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2024-04-17T09:53:38.847 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T09:53:38.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:38.847 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T09:53:38.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:38.917 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2024-04-17T09:53:38.918 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-17T09:53:38.970 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2024-04-17T09:53:38.970 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-17T09:53:38.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:38.970 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-17T09:53:39.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:39.038 DEBUG:teuthology.orchestra.run.smithi144:> stat --file-system '--printf=%T 2024-04-17T09:53:39.038 DEBUG:teuthology.orchestra.run.smithi144:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-17T09:53:39.047 INFO:teuthology.orchestra.run.smithi144.stdout:fuseblk 2024-04-17T09:53:39.047 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-17T09:53:39.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:39.047 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-17T09:53:39.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:39.110 DEBUG:teuthology.orchestra.run.smithi144:> stat --file-system '--printf=%T 2024-04-17T09:53:39.110 DEBUG:teuthology.orchestra.run.smithi144:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-17T09:53:39.159 INFO:teuthology.orchestra.run.smithi144.stdout:fuseblk 2024-04-17T09:53:39.159 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-17T09:53:39.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:39.159 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-17T09:53:39.225 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-17T09:53:39.241 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:53:39.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.394+0000 7fb552000640 1 Processor -- start 2024-04-17T09:53:39.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.394+0000 7fb552000640 1 -- start start 2024-04-17T09:53:39.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.394+0000 7fb552000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb54c0d0890 0x7fb54c0d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:39.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.394+0000 7fb552000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb54c0cdb10 0x7fb54c0cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:39.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.394+0000 7fb552000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb54c162620 con 0x7fb54c0b2430 2024-04-17T09:53:39.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.394+0000 7fb552000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb54c162900 con 0x7fb54c0d0890 2024-04-17T09:53:39.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.394+0000 7fb552000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb54c162be0 con 0x7fb54c0cdb10 2024-04-17T09:53:39.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.394+0000 7fb54b400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb54c0b2430 0x7fb54c0b2810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52686/0 (socket says 172.21.15.32:52686) 2024-04-17T09:53:39.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.394+0000 7fb54b400640 1 -- 172.21.15.32:0/3781895783 learned_addr learned my addr 172.21.15.32:0/3781895783 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:39.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.394+0000 7fb54be00640 1 --2- 172.21.15.32:0/3781895783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb54c0cdb10 0x7fb54c0cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:39.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.394+0000 7fb542a00640 1 --2- 172.21.15.32:0/3781895783 >> v2:172.21.15.144:3300/0 conn(0x7fb54c0d0890 0x7fb54c0d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:39.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.394+0000 7fb54aa00640 1 -- 172.21.15.32:0/3781895783 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2044258264 0 0) 0x7fb54c162620 con 0x7fb54c0b2430 2024-04-17T09:53:39.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.394+0000 7fb54aa00640 1 -- 172.21.15.32:0/3781895783 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb530003530 con 0x7fb54c0b2430 2024-04-17T09:53:39.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.394+0000 7fb54be00640 1 -- 172.21.15.32:0/3781895783 >> v2:172.21.15.144:3300/0 conn(0x7fb54c0d0890 msgr2=0x7fb54c0d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:39.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.394+0000 7fb54be00640 1 --2- 172.21.15.32:0/3781895783 >> v2:172.21.15.144:3300/0 conn(0x7fb54c0d0890 0x7fb54c0d0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:39.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.394+0000 7fb54be00640 1 -- 172.21.15.32:0/3781895783 >> v1:172.21.15.32:6789/0 conn(0x7fb54c0b2430 legacy=0x7fb54c0b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:39.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.394+0000 7fb54be00640 1 -- 172.21.15.32:0/3781895783 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54c162ec0 con 0x7fb54c0cdb10 2024-04-17T09:53:39.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.394+0000 7fb542a00640 1 --2- 172.21.15.32:0/3781895783 >> v2:172.21.15.144:3300/0 conn(0x7fb54c0d0890 0x7fb54c0d0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:53:39.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.398+0000 7fb54be00640 1 --2- 172.21.15.32:0/3781895783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb54c0cdb10 0x7fb54c0cdf60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb5440097c0 tx=0x7fb54401c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e36696073a70ae6a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:39.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.398+0000 7fb54aa00640 1 -- 172.21.15.32:0/3781895783 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5440b0070 con 0x7fb54c0cdb10 2024-04-17T09:53:39.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.398+0000 7fb54aa00640 1 -- 172.21.15.32:0/3781895783 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5440ab600 con 0x7fb54c0cdb10 2024-04-17T09:53:39.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.398+0000 7fb54aa00640 1 -- 172.21.15.32:0/3781895783 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5440ab9d0 con 0x7fb54c0cdb10 2024-04-17T09:53:39.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.398+0000 7fb552000640 1 -- 172.21.15.32:0/3781895783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb54c0cdb10 msgr2=0x7fb54c0cdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:39.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.398+0000 7fb552000640 1 --2- 172.21.15.32:0/3781895783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb54c0cdb10 0x7fb54c0cdf60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb5440097c0 tx=0x7fb54401c380 comp rx=0 tx=0).stop 2024-04-17T09:53:39.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.398+0000 7fb552000640 1 -- 172.21.15.32:0/3781895783 shutdown_connections 2024-04-17T09:53:39.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.398+0000 7fb552000640 1 --2- 172.21.15.32:0/3781895783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb54c0cdb10 0x7fb54c0cdf60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:39.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.398+0000 7fb552000640 1 --2- 172.21.15.32:0/3781895783 >> v2:172.21.15.144:3300/0 conn(0x7fb54c0d0890 0x7fb54c0d0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:39.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.398+0000 7fb552000640 1 -- 172.21.15.32:0/3781895783 >> 172.21.15.32:0/3781895783 conn(0x7fb54c09a9f0 msgr2=0x7fb54c0b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:39.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.398+0000 7fb552000640 1 -- 172.21.15.32:0/3781895783 shutdown_connections 2024-04-17T09:53:39.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.398+0000 7fb552000640 1 -- 172.21.15.32:0/3781895783 wait complete. 2024-04-17T09:53:39.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb552000640 1 Processor -- start 2024-04-17T09:53:39.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb552000640 1 -- start start 2024-04-17T09:53:39.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb552000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb54c0b2430 0x7fb54c156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:39.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb552000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb54c0cdb10 0x7fb54c0c7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:39.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb552000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb54c0d0890 0x7fb54c0c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:39.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb552000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c16bc20 con 0x7fb54c0cdb10 2024-04-17T09:53:39.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb552000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c16baa0 con 0x7fb54c0d0890 2024-04-17T09:53:39.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb552000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb54c16bda0 con 0x7fb54c0b2430 2024-04-17T09:53:39.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb542a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb54c0cdb10 0x7fb54c0c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:39.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb542a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb54c0cdb10 0x7fb54c0c7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46530/0 (socket says 172.21.15.32:46530) 2024-04-17T09:53:39.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb542a00640 1 -- 172.21.15.32:0/2861554629 learned_addr learned my addr 172.21.15.32:0/2861554629 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:39.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb54be00640 1 --2- 172.21.15.32:0/2861554629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb54c0d0890 0x7fb54c0c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:39.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb54b400640 1 --2- 172.21.15.32:0/2861554629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb54c0b2430 0x7fb54c156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:39.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb542a00640 1 -- 172.21.15.32:0/2861554629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb54c0b2430 msgr2=0x7fb54c156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:39.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb542a00640 1 --2- 172.21.15.32:0/2861554629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb54c0b2430 0x7fb54c156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:39.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb542a00640 1 -- 172.21.15.32:0/2861554629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb54c0d0890 msgr2=0x7fb54c0c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:39.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb542a00640 1 --2- 172.21.15.32:0/2861554629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb54c0d0890 0x7fb54c0c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:39.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb542a00640 1 -- 172.21.15.32:0/2861554629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54c0c7cb0 con 0x7fb54c0cdb10 2024-04-17T09:53:39.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb54be00640 1 --2- 172.21.15.32:0/2861554629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb54c0d0890 0x7fb54c0c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:39.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb542a00640 1 --2- 172.21.15.32:0/2861554629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb54c0cdb10 0x7fb54c0c7140 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb53400ebe0 tx=0x7fb53400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:39.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb543e00640 1 -- 172.21.15.32:0/2861554629 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb534017070 con 0x7fb54c0cdb10 2024-04-17T09:53:39.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb543e00640 1 -- 172.21.15.32:0/2861554629 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb53400cc30 con 0x7fb54c0cdb10 2024-04-17T09:53:39.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb552000640 1 -- 172.21.15.32:0/2861554629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb54c0c87f0 con 0x7fb54c0cdb10 2024-04-17T09:53:39.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb543e00640 1 -- 172.21.15.32:0/2861554629 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb534013470 con 0x7fb54c0cdb10 2024-04-17T09:53:39.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.402+0000 7fb552000640 1 -- 172.21.15.32:0/2861554629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb54c156c90 con 0x7fb54c0cdb10 2024-04-17T09:53:39.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.406+0000 7fb552000640 1 -- 172.21.15.32:0/2861554629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb54c0c8600 con 0x7fb54c0cdb10 2024-04-17T09:53:39.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.406+0000 7fb543e00640 1 -- 172.21.15.32:0/2861554629 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb53401a070 con 0x7fb54c0cdb10 2024-04-17T09:53:39.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.406+0000 7fb543e00640 1 --2- 172.21.15.32:0/2861554629 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb528068380 0x7fb52806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:39.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.406+0000 7fb543e00640 1 -- 172.21.15.32:0/2861554629 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb534086d40 con 0x7fb54c0cdb10 2024-04-17T09:53:39.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.406+0000 7fb54b400640 1 --2- 172.21.15.32:0/2861554629 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb528068380 0x7fb52806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:39.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.406+0000 7fb54b400640 1 --2- 172.21.15.32:0/2861554629 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb528068380 0x7fb52806a840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb53c005c10 tx=0x7fb53c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:39.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.410+0000 7fb543e00640 1 -- 172.21.15.32:0/2861554629 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fb534057260 con 0x7fb54c0cdb10 2024-04-17T09:53:39.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.578+0000 7fb552000640 1 -- 172.21.15.32:0/2861554629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb54c09af00 con 0x7fb54c0cdb10 2024-04-17T09:53:39.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.578+0000 7fb543e00640 1 -- 172.21.15.32:0/2861554629 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7fb534056b60 con 0x7fb54c0cdb10 2024-04-17T09:53:39.579 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:39.579 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:14.779535+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:53:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.578+0000 7fb552000640 1 -- 172.21.15.32:0/2861554629 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb528068380 msgr2=0x7fb52806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.578+0000 7fb552000640 1 --2- 172.21.15.32:0/2861554629 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb528068380 0x7fb52806a840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb53c005c10 tx=0x7fb53c009040 comp rx=0 tx=0).stop 2024-04-17T09:53:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.578+0000 7fb552000640 1 -- 172.21.15.32:0/2861554629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb54c0cdb10 msgr2=0x7fb54c0c7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.578+0000 7fb552000640 1 --2- 172.21.15.32:0/2861554629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb54c0cdb10 0x7fb54c0c7140 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb53400ebe0 tx=0x7fb53400c520 comp rx=0 tx=0).stop 2024-04-17T09:53:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.578+0000 7fb552000640 1 -- 172.21.15.32:0/2861554629 shutdown_connections 2024-04-17T09:53:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.578+0000 7fb552000640 1 --2- 172.21.15.32:0/2861554629 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb528068380 0x7fb52806a840 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.578+0000 7fb552000640 1 --2- 172.21.15.32:0/2861554629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb54c0d0890 0x7fb54c0c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.578+0000 7fb552000640 1 --2- 172.21.15.32:0/2861554629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb54c0cdb10 0x7fb54c0c7140 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.578+0000 7fb552000640 1 --2- 172.21.15.32:0/2861554629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb54c0b2430 0x7fb54c156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.578+0000 7fb552000640 1 -- 172.21.15.32:0/2861554629 >> 172.21.15.32:0/2861554629 conn(0x7fb54c09a9f0 msgr2=0x7fb54c06ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:39.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.582+0000 7fb552000640 1 -- 172.21.15.32:0/2861554629 shutdown_connections 2024-04-17T09:53:39.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.582+0000 7fb552000640 1 -- 172.21.15.32:0/2861554629 wait complete. 2024-04-17T09:53:39.601 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:53:11.198098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:53:12.118982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:53:39.601 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:53:39.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.742+0000 7fe037e00640 1 Processor -- start 2024-04-17T09:53:39.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.746+0000 7fe037e00640 1 -- start start 2024-04-17T09:53:39.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.746+0000 7fe037e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe03006ba30 0x7fe03006be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:39.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.746+0000 7fe037e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe030071e60 0x7fe0300722b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:39.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.746+0000 7fe037e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe03006c350 con 0x7fe0300cf370 2024-04-17T09:53:39.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.746+0000 7fe037e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe0300cfef0 con 0x7fe030071e60 2024-04-17T09:53:39.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.746+0000 7fe037e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe0300d0130 con 0x7fe03006ba30 2024-04-17T09:53:39.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.746+0000 7fe034a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe0300cf370 0x7fe0300cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52700/0 (socket says 172.21.15.32:52700) 2024-04-17T09:53:39.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.746+0000 7fe034a00640 1 -- 172.21.15.32:0/1155143733 learned_addr learned my addr 172.21.15.32:0/1155143733 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:39.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.746+0000 7fe035400640 1 --2- 172.21.15.32:0/1155143733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe03006ba30 0x7fe03006be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:39.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.746+0000 7fe035e00640 1 --2- 172.21.15.32:0/1155143733 >> v2:172.21.15.144:3300/0 conn(0x7fe030071e60 0x7fe0300722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:39.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.746+0000 7fe02fe00640 1 -- 172.21.15.32:0/1155143733 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2236768572 0 0) 0x7fe03006c350 con 0x7fe0300cf370 2024-04-17T09:53:39.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.746+0000 7fe02fe00640 1 -- 172.21.15.32:0/1155143733 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe018003530 con 0x7fe0300cf370 2024-04-17T09:53:39.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.746+0000 7fe035400640 1 -- 172.21.15.32:0/1155143733 >> v2:172.21.15.144:3300/0 conn(0x7fe030071e60 msgr2=0x7fe0300722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:39.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.746+0000 7fe035400640 1 --2- 172.21.15.32:0/1155143733 >> v2:172.21.15.144:3300/0 conn(0x7fe030071e60 0x7fe0300722b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:39.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.746+0000 7fe035400640 1 -- 172.21.15.32:0/1155143733 >> v1:172.21.15.32:6789/0 conn(0x7fe0300cf370 legacy=0x7fe0300cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:39.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.746+0000 7fe035400640 1 -- 172.21.15.32:0/1155143733 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03015d090 con 0x7fe03006ba30 2024-04-17T09:53:39.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.746+0000 7fe035e00640 1 --2- 172.21.15.32:0/1155143733 >> v2:172.21.15.144:3300/0 conn(0x7fe030071e60 0x7fe0300722b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:53:39.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.750+0000 7fe035400640 1 --2- 172.21.15.32:0/1155143733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe03006ba30 0x7fe03006be10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe0240097c0 tx=0x7fe02401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eca1d5e13749c86 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:39.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.750+0000 7fe02fe00640 1 -- 172.21.15.32:0/1155143733 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0240a7070 con 0x7fe03006ba30 2024-04-17T09:53:39.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.750+0000 7fe02fe00640 1 -- 172.21.15.32:0/1155143733 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0240b0070 con 0x7fe03006ba30 2024-04-17T09:53:39.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.750+0000 7fe02fe00640 1 -- 172.21.15.32:0/1155143733 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0240ab890 con 0x7fe03006ba30 2024-04-17T09:53:39.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.750+0000 7fe037e00640 1 -- 172.21.15.32:0/1155143733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe03006ba30 msgr2=0x7fe03006be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:39.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.750+0000 7fe037e00640 1 --2- 172.21.15.32:0/1155143733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe03006ba30 0x7fe03006be10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe0240097c0 tx=0x7fe02401c470 comp rx=0 tx=0).stop 2024-04-17T09:53:39.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.750+0000 7fe037e00640 1 -- 172.21.15.32:0/1155143733 shutdown_connections 2024-04-17T09:53:39.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.750+0000 7fe037e00640 1 --2- 172.21.15.32:0/1155143733 >> v2:172.21.15.144:3300/0 conn(0x7fe030071e60 0x7fe0300722b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:39.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.750+0000 7fe037e00640 1 --2- 172.21.15.32:0/1155143733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe03006ba30 0x7fe03006be10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:39.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.750+0000 7fe037e00640 1 -- 172.21.15.32:0/1155143733 >> 172.21.15.32:0/1155143733 conn(0x7fe0300b2120 msgr2=0x7fe0300b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:39.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.750+0000 7fe037e00640 1 -- 172.21.15.32:0/1155143733 shutdown_connections 2024-04-17T09:53:39.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.750+0000 7fe037e00640 1 -- 172.21.15.32:0/1155143733 wait complete. 2024-04-17T09:53:39.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.750+0000 7fe037e00640 1 Processor -- start 2024-04-17T09:53:39.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.750+0000 7fe037e00640 1 -- start start 2024-04-17T09:53:39.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.754+0000 7fe037e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe03006ba30 0x7fe03013ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:39.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.754+0000 7fe037e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe030071e60 0x7fe03013da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:39.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.754+0000 7fe037e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0300cf370 0x7fe03015a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:39.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.754+0000 7fe037e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe03015dcf0 con 0x7fe03006ba30 2024-04-17T09:53:39.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.754+0000 7fe037e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe03015db70 con 0x7fe030071e60 2024-04-17T09:53:39.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.754+0000 7fe037e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe030146230 con 0x7fe0300cf370 2024-04-17T09:53:39.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.754+0000 7fe035400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe03006ba30 0x7fe03013ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:39.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.754+0000 7fe035400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe03006ba30 0x7fe03013ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46532/0 (socket says 172.21.15.32:46532) 2024-04-17T09:53:39.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.754+0000 7fe035400640 1 -- 172.21.15.32:0/1716769542 learned_addr learned my addr 172.21.15.32:0/1716769542 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:39.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.754+0000 7fe034a00640 1 --2- 172.21.15.32:0/1716769542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe030071e60 0x7fe03013da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:39.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.754+0000 7fe035e00640 1 --2- 172.21.15.32:0/1716769542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0300cf370 0x7fe03015a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:39.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.754+0000 7fe035400640 1 -- 172.21.15.32:0/1716769542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0300cf370 msgr2=0x7fe03015a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:39.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.754+0000 7fe035400640 1 --2- 172.21.15.32:0/1716769542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0300cf370 0x7fe03015a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:39.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.754+0000 7fe035400640 1 -- 172.21.15.32:0/1716769542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe030071e60 msgr2=0x7fe03013da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:39.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.754+0000 7fe035400640 1 --2- 172.21.15.32:0/1716769542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe030071e60 0x7fe03013da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:39.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.754+0000 7fe035400640 1 -- 172.21.15.32:0/1716769542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0300b4060 con 0x7fe03006ba30 2024-04-17T09:53:39.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.754+0000 7fe035400640 1 --2- 172.21.15.32:0/1716769542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe03006ba30 0x7fe03013ea70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe0240ad040 tx=0x7fe02401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:39.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.754+0000 7fe02d600640 1 -- 172.21.15.32:0/1716769542 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe024006e20 con 0x7fe03006ba30 2024-04-17T09:53:39.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.754+0000 7fe02d600640 1 -- 172.21.15.32:0/1716769542 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0240abe50 con 0x7fe03006ba30 2024-04-17T09:53:39.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.754+0000 7fe02d600640 1 -- 172.21.15.32:0/1716769542 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0240bf870 con 0x7fe03006ba30 2024-04-17T09:53:39.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.754+0000 7fe037e00640 1 -- 172.21.15.32:0/1716769542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0300b4300 con 0x7fe03006ba30 2024-04-17T09:53:39.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.754+0000 7fe037e00640 1 -- 172.21.15.32:0/1716769542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0300b48c0 con 0x7fe03006ba30 2024-04-17T09:53:39.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.754+0000 7fe037e00640 1 -- 172.21.15.32:0/1716769542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe030068a90 con 0x7fe03006ba30 2024-04-17T09:53:39.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.758+0000 7fe02d600640 1 -- 172.21.15.32:0/1716769542 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe0240bd020 con 0x7fe03006ba30 2024-04-17T09:53:39.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.758+0000 7fe02d600640 1 --2- 172.21.15.32:0/1716769542 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0100683b0 0x7fe01006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:39.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.758+0000 7fe02d600640 1 -- 172.21.15.32:0/1716769542 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe024120e60 con 0x7fe03006ba30 2024-04-17T09:53:39.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.758+0000 7fe034a00640 1 --2- 172.21.15.32:0/1716769542 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0100683b0 0x7fe01006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:39.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.758+0000 7fe034a00640 1 --2- 172.21.15.32:0/1716769542 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0100683b0 0x7fe01006a870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe01c007fd0 tx=0x7fe01c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:39.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.762+0000 7fe02d600640 1 -- 172.21.15.32:0/1716769542 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fe024016090 con 0x7fe03006ba30 2024-04-17T09:53:39.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.926+0000 7fe037e00640 1 -- 172.21.15.32:0/1716769542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe0300ce5e0 con 0x7fe03006ba30 2024-04-17T09:53:39.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.926+0000 7fe02d600640 1 -- 172.21.15.32:0/1716769542 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fe0240f0d30 con 0x7fe03006ba30 2024-04-17T09:53:39.927 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:53:39.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.930+0000 7fe037e00640 1 -- 172.21.15.32:0/1716769542 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0100683b0 msgr2=0x7fe01006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:39.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.930+0000 7fe037e00640 1 --2- 172.21.15.32:0/1716769542 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0100683b0 0x7fe01006a870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe01c007fd0 tx=0x7fe01c009040 comp rx=0 tx=0).stop 2024-04-17T09:53:39.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.930+0000 7fe037e00640 1 -- 172.21.15.32:0/1716769542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe03006ba30 msgr2=0x7fe03013ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:39.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.930+0000 7fe037e00640 1 --2- 172.21.15.32:0/1716769542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe03006ba30 0x7fe03013ea70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe0240ad040 tx=0x7fe02401acb0 comp rx=0 tx=0).stop 2024-04-17T09:53:39.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.930+0000 7fe037e00640 1 -- 172.21.15.32:0/1716769542 shutdown_connections 2024-04-17T09:53:39.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.930+0000 7fe037e00640 1 --2- 172.21.15.32:0/1716769542 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0100683b0 0x7fe01006a870 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:39.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.930+0000 7fe037e00640 1 --2- 172.21.15.32:0/1716769542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0300cf370 0x7fe03015a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:39.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.930+0000 7fe037e00640 1 --2- 172.21.15.32:0/1716769542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe030071e60 0x7fe03013da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:39.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.930+0000 7fe037e00640 1 --2- 172.21.15.32:0/1716769542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe03006ba30 0x7fe03013ea70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:39.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.930+0000 7fe037e00640 1 -- 172.21.15.32:0/1716769542 >> 172.21.15.32:0/1716769542 conn(0x7fe0300b2120 msgr2=0x7fe0300b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:39.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.930+0000 7fe037e00640 1 -- 172.21.15.32:0/1716769542 shutdown_connections 2024-04-17T09:53:39.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:39.930+0000 7fe037e00640 1 -- 172.21.15.32:0/1716769542 wait complete. 2024-04-17T09:53:39.949 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:53:40.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f4502000640 1 Processor -- start 2024-04-17T09:53:40.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f4502000640 1 -- start start 2024-04-17T09:53:40.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f4502000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44fc141700 0x7f44fc06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:40.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f4502000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44fc0b0540 con 0x7f44fc0afeb0 2024-04-17T09:53:40.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f4502000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44fc0b3ef0 con 0x7f44fc06cf80 2024-04-17T09:53:40.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f4502000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f44fc06d3e0 con 0x7f44fc141700 2024-04-17T09:53:40.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44fbe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f44fc0afeb0 0x7f44fc0b49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52704/0 (socket says 172.21.15.32:52704) 2024-04-17T09:53:40.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44fbe00640 1 -- 172.21.15.32:0/309164635 learned_addr learned my addr 172.21.15.32:0/309164635 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:40.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44fb400640 1 --2- 172.21.15.32:0/309164635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44fc141700 0x7f44fc06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:40.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44faa00640 1 -- 172.21.15.32:0/309164635 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3358823315 0 0) 0x7f44fc0b0540 con 0x7f44fc0afeb0 2024-04-17T09:53:40.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44faa00640 1 -- 172.21.15.32:0/309164635 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44e0003530 con 0x7f44fc0afeb0 2024-04-17T09:53:40.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44faa00640 1 -- 172.21.15.32:0/309164635 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3665396483 0 0) 0x7f44e0003530 con 0x7f44fc0afeb0 2024-04-17T09:53:40.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44faa00640 1 -- 172.21.15.32:0/309164635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44fc141700 msgr2=0x7f44fc06ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:40.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44faa00640 1 --2- 172.21.15.32:0/309164635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44fc141700 0x7f44fc06ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44faa00640 1 -- 172.21.15.32:0/309164635 >> v1:172.21.15.144:6789/0 conn(0x7f44fc06cf80 legacy=0x7f44fc0af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:40.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44faa00640 1 -- 172.21.15.32:0/309164635 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44fc0b41d0 con 0x7f44fc0afeb0 2024-04-17T09:53:40.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44faa00640 1 -- 172.21.15.32:0/309164635 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f44f4001a00 con 0x7f44fc0afeb0 2024-04-17T09:53:40.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44faa00640 1 -- 172.21.15.32:0/309164635 >> v1:172.21.15.32:6789/0 conn(0x7f44fc0afeb0 legacy=0x7f44fc0b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:40.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44faa00640 1 --2- 172.21.15.32:0/309164635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44fc141700 0x7f44e00039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:40.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44faa00640 1 --2- 172.21.15.32:0/309164635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e0003fc0 0x7f44e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:40.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44faa00640 1 --2- 172.21.15.32:0/309164635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e0004d50 0x7f44e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:40.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44faa00640 1 -- 172.21.15.32:0/309164635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f44e000bae0 con 0x7f44e0004d50 2024-04-17T09:53:40.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44faa00640 1 -- 172.21.15.32:0/309164635 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f44e000bc90 con 0x7f44e0003fc0 2024-04-17T09:53:40.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44faa00640 1 -- 172.21.15.32:0/309164635 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f44e000be90 con 0x7f44fc141700 2024-04-17T09:53:40.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44fbe00640 1 --2- 172.21.15.32:0/309164635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e0004d50 0x7f44e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:40.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44f2a00640 1 --2- 172.21.15.32:0/309164635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e0003fc0 0x7f44e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:40.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44fb400640 1 --2- 172.21.15.32:0/309164635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44fc141700 0x7f44e00039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:40.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44fbe00640 1 -- 172.21.15.32:0/309164635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44fc141700 msgr2=0x7f44e00039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:40.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44fbe00640 1 --2- 172.21.15.32:0/309164635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44fc141700 0x7f44e00039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44fbe00640 1 -- 172.21.15.32:0/309164635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e0003fc0 msgr2=0x7f44e0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:40.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44fbe00640 1 --2- 172.21.15.32:0/309164635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e0003fc0 0x7f44e0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.094+0000 7f44fbe00640 1 -- 172.21.15.32:0/309164635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e0003530 con 0x7f44e0004d50 2024-04-17T09:53:40.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.098+0000 7f44fbe00640 1 --2- 172.21.15.32:0/309164635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e0004d50 0x7f44e00051d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f44f400a790 tx=0x7f44f401d2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e67f89a9e6fef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:40.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.098+0000 7f44faa00640 1 -- 172.21.15.32:0/309164635 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f400a3b0 con 0x7f44e0004d50 2024-04-17T09:53:40.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.098+0000 7f44faa00640 1 -- 172.21.15.32:0/309164635 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44f40ad650 con 0x7f44e0004d50 2024-04-17T09:53:40.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.098+0000 7f44faa00640 1 -- 172.21.15.32:0/309164635 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f40ada80 con 0x7f44e0004d50 2024-04-17T09:53:40.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.098+0000 7f4502000640 1 -- 172.21.15.32:0/309164635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e0004d50 msgr2=0x7f44e00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:40.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.098+0000 7f4502000640 1 --2- 172.21.15.32:0/309164635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e0004d50 0x7f44e00051d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f44f400a790 tx=0x7f44f401d2c0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.098+0000 7f44fbe00640 1 -- 172.21.15.32:0/309164635 reap_dead start 2024-04-17T09:53:40.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.098+0000 7f4502000640 1 -- 172.21.15.32:0/309164635 shutdown_connections 2024-04-17T09:53:40.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.098+0000 7f4502000640 1 -- 172.21.15.32:0/309164635 >> 172.21.15.32:0/309164635 conn(0x7f44fc072670 msgr2=0x7f44fc072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:40.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.098+0000 7f4502000640 1 -- 172.21.15.32:0/309164635 shutdown_connections 2024-04-17T09:53:40.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.098+0000 7f4502000640 1 -- 172.21.15.32:0/309164635 wait complete. 2024-04-17T09:53:40.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f4502000640 1 Processor -- start 2024-04-17T09:53:40.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f4502000640 1 -- start start 2024-04-17T09:53:40.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f4502000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44fc153ce0 0x7f44fc1617a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f4502000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44fc141e90 0x7f44fc13f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f4502000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44fc13fd20 0x7f44fc06ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f4502000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc141ae0 con 0x7f44fc141e90 2024-04-17T09:53:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f4502000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc0b0820 con 0x7f44fc153ce0 2024-04-17T09:53:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f4502000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f44fc141c60 con 0x7f44fc13fd20 2024-04-17T09:53:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f44f2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44fc141e90 0x7f44fc13f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f44f2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44fc141e90 0x7f44fc13f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46546/0 (socket says 172.21.15.32:46546) 2024-04-17T09:53:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f44f2a00640 1 -- 172.21.15.32:0/1852114967 learned_addr learned my addr 172.21.15.32:0/1852114967 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f44fb400640 1 --2- 172.21.15.32:0/1852114967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44fc153ce0 0x7f44fc1617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f44fbe00640 1 --2- 172.21.15.32:0/1852114967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44fc13fd20 0x7f44fc06ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f44f2a00640 1 -- 172.21.15.32:0/1852114967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44fc13fd20 msgr2=0x7f44fc06ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f44f2a00640 1 --2- 172.21.15.32:0/1852114967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44fc13fd20 0x7f44fc06ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f44f2a00640 1 -- 172.21.15.32:0/1852114967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44fc153ce0 msgr2=0x7f44fc1617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f44f2a00640 1 --2- 172.21.15.32:0/1852114967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44fc153ce0 0x7f44fc1617a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f44f2a00640 1 -- 172.21.15.32:0/1852114967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44fc142450 con 0x7f44fc141e90 2024-04-17T09:53:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f44fb400640 1 --2- 172.21.15.32:0/1852114967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44fc153ce0 0x7f44fc1617a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f44fbe00640 1 --2- 172.21.15.32:0/1852114967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44fc13fd20 0x7f44fc06ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f44f2a00640 1 --2- 172.21.15.32:0/1852114967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44fc141e90 0x7f44fc13f7e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f44e400f5f0 tx=0x7f44e400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f44f3e00640 1 -- 172.21.15.32:0/1852114967 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e40052e0 con 0x7f44fc141e90 2024-04-17T09:53:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f44f3e00640 1 -- 172.21.15.32:0/1852114967 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e4018070 con 0x7f44fc141e90 2024-04-17T09:53:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f4502000640 1 -- 172.21.15.32:0/1852114967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44fc142730 con 0x7f44fc141e90 2024-04-17T09:53:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f44f3e00640 1 -- 172.21.15.32:0/1852114967 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e4003840 con 0x7f44fc141e90 2024-04-17T09:53:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f4502000640 1 -- 172.21.15.32:0/1852114967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44fc142cf0 con 0x7f44fc141e90 2024-04-17T09:53:40.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.102+0000 7f4502000640 1 -- 172.21.15.32:0/1852114967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44fc068a90 con 0x7f44fc141e90 2024-04-17T09:53:40.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.106+0000 7f44f3e00640 1 -- 172.21.15.32:0/1852114967 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f44e40039e0 con 0x7f44fc141e90 2024-04-17T09:53:40.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.106+0000 7f44f3e00640 1 --2- 172.21.15.32:0/1852114967 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f44d80683b0 0x7f44d806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:40.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.106+0000 7f44f3e00640 1 -- 172.21.15.32:0/1852114967 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f44e40866e0 con 0x7f44fc141e90 2024-04-17T09:53:40.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.106+0000 7f44fb400640 1 --2- 172.21.15.32:0/1852114967 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f44d80683b0 0x7f44d806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:40.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.106+0000 7f44fb400640 1 --2- 172.21.15.32:0/1852114967 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f44d80683b0 0x7f44d806a870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009be0 tx=0x7f44ec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:40.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.110+0000 7f44f3e00640 1 -- 172.21.15.32:0/1852114967 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f44e4056c90 con 0x7f44fc141e90 2024-04-17T09:53:40.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.278+0000 7f4502000640 1 -- 172.21.15.32:0/1852114967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f44fc140100 con 0x7f44fc141e90 2024-04-17T09:53:40.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.278+0000 7f44f3e00640 1 -- 172.21.15.32:0/1852114967 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f44e4005480 con 0x7f44fc141e90 2024-04-17T09:53:40.279 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:53:40.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.278+0000 7f4502000640 1 -- 172.21.15.32:0/1852114967 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f44d80683b0 msgr2=0x7f44d806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:40.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.278+0000 7f4502000640 1 --2- 172.21.15.32:0/1852114967 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f44d80683b0 0x7f44d806a870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009be0 tx=0x7f44ec008040 comp rx=0 tx=0).stop 2024-04-17T09:53:40.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.278+0000 7f4502000640 1 -- 172.21.15.32:0/1852114967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44fc141e90 msgr2=0x7f44fc13f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:40.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.278+0000 7f4502000640 1 --2- 172.21.15.32:0/1852114967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44fc141e90 0x7f44fc13f7e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f44e400f5f0 tx=0x7f44e400fab0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.278+0000 7f4502000640 1 -- 172.21.15.32:0/1852114967 shutdown_connections 2024-04-17T09:53:40.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.278+0000 7f4502000640 1 --2- 172.21.15.32:0/1852114967 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f44d80683b0 0x7f44d806a870 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.278+0000 7f4502000640 1 --2- 172.21.15.32:0/1852114967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44fc13fd20 0x7f44fc06ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.278+0000 7f4502000640 1 --2- 172.21.15.32:0/1852114967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44fc141e90 0x7f44fc13f7e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.278+0000 7f4502000640 1 --2- 172.21.15.32:0/1852114967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44fc153ce0 0x7f44fc1617a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.278+0000 7f4502000640 1 -- 172.21.15.32:0/1852114967 >> 172.21.15.32:0/1852114967 conn(0x7f44fc072670 msgr2=0x7f44fc0b5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:40.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.278+0000 7f4502000640 1 -- 172.21.15.32:0/1852114967 shutdown_connections 2024-04-17T09:53:40.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.282+0000 7f4502000640 1 -- 172.21.15.32:0/1852114967 wait complete. 2024-04-17T09:53:40.300 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:53:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.442+0000 7f7d2cc00640 1 Processor -- start 2024-04-17T09:53:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.442+0000 7f7d2cc00640 1 -- start start 2024-04-17T09:53:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.446+0000 7f7d2cc00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7d28145e70 0x7f7d2806ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.446+0000 7f7d2cc00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d280b0eb0 0x7f7d280b1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.446+0000 7f7d2cc00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d281611b0 con 0x7f7d28160c50 2024-04-17T09:53:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.446+0000 7f7d2cc00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7d280afba0 con 0x7f7d280b0eb0 2024-04-17T09:53:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.446+0000 7f7d2cc00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f7d280afdd0 con 0x7f7d28145e70 2024-04-17T09:53:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.446+0000 7f7d27e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d280b0eb0 0x7f7d280b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.446+0000 7f7d27e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d280b0eb0 0x7f7d280b1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38870/0 (socket says 172.21.15.32:38870) 2024-04-17T09:53:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.446+0000 7f7d27e00640 1 -- 172.21.15.32:0/1477470098 learned_addr learned my addr 172.21.15.32:0/1477470098 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:40.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.446+0000 7f7d27400640 1 --2- 172.21.15.32:0/1477470098 >> v2:172.21.15.144:3300/0 conn(0x7f7d28145e70 0x7f7d2806ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:40.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.446+0000 7f7d27e00640 1 -- 172.21.15.32:0/1477470098 >> v2:172.21.15.144:3300/0 conn(0x7f7d28145e70 msgr2=0x7f7d2806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:40.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.446+0000 7f7d27e00640 1 --2- 172.21.15.32:0/1477470098 >> v2:172.21.15.144:3300/0 conn(0x7f7d28145e70 0x7f7d2806ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.446+0000 7f7d27e00640 1 -- 172.21.15.32:0/1477470098 >> v1:172.21.15.144:6789/0 conn(0x7f7d28160c50 legacy=0x7f7d2806bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.446+0000 7f7d27e00640 1 -- 172.21.15.32:0/1477470098 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d280b0070 con 0x7f7d280b0eb0 2024-04-17T09:53:40.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.446+0000 7f7d27400640 1 --2- 172.21.15.32:0/1477470098 >> v2:172.21.15.144:3300/0 conn(0x7f7d28145e70 0x7f7d2806ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:40.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.446+0000 7f7d27e00640 1 --2- 172.21.15.32:0/1477470098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d280b0eb0 0x7f7d280b1300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7d280d24c0 tx=0x7f7d2001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212bc7e3fdf2193b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:40.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.450+0000 7f7d26a00640 1 -- 172.21.15.32:0/1477470098 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d200a5720 con 0x7f7d280b0eb0 2024-04-17T09:53:40.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.450+0000 7f7d26a00640 1 -- 172.21.15.32:0/1477470098 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d200a58c0 con 0x7f7d280b0eb0 2024-04-17T09:53:40.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.450+0000 7f7d26a00640 1 -- 172.21.15.32:0/1477470098 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d200a5c90 con 0x7f7d280b0eb0 2024-04-17T09:53:40.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.450+0000 7f7d2cc00640 1 -- 172.21.15.32:0/1477470098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d280b0eb0 msgr2=0x7f7d280b1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:40.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.450+0000 7f7d2cc00640 1 --2- 172.21.15.32:0/1477470098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d280b0eb0 0x7f7d280b1300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7d280d24c0 tx=0x7f7d2001c9b0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.450+0000 7f7d2cc00640 1 -- 172.21.15.32:0/1477470098 shutdown_connections 2024-04-17T09:53:40.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.450+0000 7f7d2cc00640 1 --2- 172.21.15.32:0/1477470098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d280b0eb0 0x7f7d280b1300 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.450+0000 7f7d2cc00640 1 --2- 172.21.15.32:0/1477470098 >> v2:172.21.15.144:3300/0 conn(0x7f7d28145e70 0x7f7d2806ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.450+0000 7f7d2cc00640 1 -- 172.21.15.32:0/1477470098 >> 172.21.15.32:0/1477470098 conn(0x7f7d280732d0 msgr2=0x7f7d2809a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:40.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.450+0000 7f7d2cc00640 1 -- 172.21.15.32:0/1477470098 shutdown_connections 2024-04-17T09:53:40.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.450+0000 7f7d2cc00640 1 -- 172.21.15.32:0/1477470098 wait complete. 2024-04-17T09:53:40.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.450+0000 7f7d2cc00640 1 Processor -- start 2024-04-17T09:53:40.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.450+0000 7f7d2cc00640 1 -- start start 2024-04-17T09:53:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.450+0000 7f7d2cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d280b0eb0 0x7f7d280cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.450+0000 7f7d2cc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d28145e70 0x7f7d28144b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.450+0000 7f7d2cc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d28160c50 0x7f7d2815fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.450+0000 7f7d2cc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7d280b0b50 con 0x7f7d280b0eb0 2024-04-17T09:53:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.450+0000 7f7d2cc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7d280ae570 con 0x7f7d28145e70 2024-04-17T09:53:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.450+0000 7f7d2cc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7d281400a0 con 0x7f7d28160c50 2024-04-17T09:53:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.450+0000 7f7d27400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d280b0eb0 0x7f7d280cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.450+0000 7f7d27400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d280b0eb0 0x7f7d280cfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38884/0 (socket says 172.21.15.32:38884) 2024-04-17T09:53:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.450+0000 7f7d27400640 1 -- 172.21.15.32:0/2359520782 learned_addr learned my addr 172.21.15.32:0/2359520782 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.454+0000 7f7d27e00640 1 --2- 172.21.15.32:0/2359520782 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d28160c50 0x7f7d2815fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.454+0000 7f7d27400640 1 -- 172.21.15.32:0/2359520782 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d28160c50 msgr2=0x7f7d2815fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.454+0000 7f7d27400640 1 --2- 172.21.15.32:0/2359520782 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d28160c50 0x7f7d2815fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.454+0000 7f7d27400640 1 -- 172.21.15.32:0/2359520782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d28145e70 msgr2=0x7f7d28144b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.454+0000 7f7d1ea00640 1 --2- 172.21.15.32:0/2359520782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d28145e70 0x7f7d28144b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.454+0000 7f7d27400640 1 --2- 172.21.15.32:0/2359520782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d28145e70 0x7f7d28144b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.454+0000 7f7d27400640 1 -- 172.21.15.32:0/2359520782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2813f800 con 0x7f7d280b0eb0 2024-04-17T09:53:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.454+0000 7f7d27e00640 1 --2- 172.21.15.32:0/2359520782 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d28160c50 0x7f7d2815fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.454+0000 7f7d1ea00640 1 --2- 172.21.15.32:0/2359520782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d28145e70 0x7f7d28144b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.454+0000 7f7d27400640 1 --2- 172.21.15.32:0/2359520782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d280b0eb0 0x7f7d280cfbb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800c7c0 tx=0x7f7d1800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.454+0000 7f7d1fe00640 1 -- 172.21.15.32:0/2359520782 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d18007a30 con 0x7f7d280b0eb0 2024-04-17T09:53:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.454+0000 7f7d1fe00640 1 -- 172.21.15.32:0/2359520782 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d18007bd0 con 0x7f7d280b0eb0 2024-04-17T09:53:40.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.454+0000 7f7d1fe00640 1 -- 172.21.15.32:0/2359520782 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d180113f0 con 0x7f7d280b0eb0 2024-04-17T09:53:40.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.454+0000 7f7d2cc00640 1 -- 172.21.15.32:0/2359520782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2813fae0 con 0x7f7d280b0eb0 2024-04-17T09:53:40.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.454+0000 7f7d2cc00640 1 -- 172.21.15.32:0/2359520782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d2813f460 con 0x7f7d280b0eb0 2024-04-17T09:53:40.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.454+0000 7f7d2cc00640 1 -- 172.21.15.32:0/2359520782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d2804fda0 con 0x7f7d280b0eb0 2024-04-17T09:53:40.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.454+0000 7f7d1fe00640 1 -- 172.21.15.32:0/2359520782 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7d18011590 con 0x7f7d280b0eb0 2024-04-17T09:53:40.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.458+0000 7f7d1fe00640 1 --2- 172.21.15.32:0/2359520782 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7d04068380 0x7f7d0406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:40.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.458+0000 7f7d1fe00640 1 -- 172.21.15.32:0/2359520782 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7d18086bd0 con 0x7f7d280b0eb0 2024-04-17T09:53:40.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.458+0000 7f7d1ea00640 1 --2- 172.21.15.32:0/2359520782 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7d04068380 0x7f7d0406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:40.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.458+0000 7f7d1ea00640 1 --2- 172.21.15.32:0/2359520782 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7d04068380 0x7f7d0406a840 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7d2813ec90 tx=0x7f7d10008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:40.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.462+0000 7f7d1fe00640 1 -- 172.21.15.32:0/2359520782 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f7d18015020 con 0x7f7d280b0eb0 2024-04-17T09:53:40.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.622+0000 7f7d2cc00640 1 -- 172.21.15.32:0/2359520782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7d280cd680 con 0x7f7d280b0eb0 2024-04-17T09:53:40.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.626+0000 7f7d1fe00640 1 -- 172.21.15.32:0/2359520782 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f7d18057190 con 0x7f7d280b0eb0 2024-04-17T09:53:40.625 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:53:40.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.626+0000 7f7d2cc00640 1 -- 172.21.15.32:0/2359520782 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7d04068380 msgr2=0x7f7d0406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:40.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.626+0000 7f7d2cc00640 1 --2- 172.21.15.32:0/2359520782 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7d04068380 0x7f7d0406a840 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7d2813ec90 tx=0x7f7d10008040 comp rx=0 tx=0).stop 2024-04-17T09:53:40.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.626+0000 7f7d2cc00640 1 -- 172.21.15.32:0/2359520782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d280b0eb0 msgr2=0x7f7d280cfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:40.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.626+0000 7f7d2cc00640 1 --2- 172.21.15.32:0/2359520782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d280b0eb0 0x7f7d280cfbb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800c7c0 tx=0x7f7d1800cc80 comp rx=0 tx=0).stop 2024-04-17T09:53:40.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.626+0000 7f7d2cc00640 1 -- 172.21.15.32:0/2359520782 shutdown_connections 2024-04-17T09:53:40.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.626+0000 7f7d2cc00640 1 --2- 172.21.15.32:0/2359520782 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7d04068380 0x7f7d0406a840 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.626+0000 7f7d2cc00640 1 --2- 172.21.15.32:0/2359520782 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d28160c50 0x7f7d2815fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.626+0000 7f7d2cc00640 1 --2- 172.21.15.32:0/2359520782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d28145e70 0x7f7d28144b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.626+0000 7f7d2cc00640 1 --2- 172.21.15.32:0/2359520782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d280b0eb0 0x7f7d280cfbb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.626+0000 7f7d2cc00640 1 -- 172.21.15.32:0/2359520782 >> 172.21.15.32:0/2359520782 conn(0x7f7d280732d0 msgr2=0x7f7d281418d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:40.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.626+0000 7f7d2cc00640 1 -- 172.21.15.32:0/2359520782 shutdown_connections 2024-04-17T09:53:40.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.626+0000 7f7d2cc00640 1 -- 172.21.15.32:0/2359520782 wait complete. 2024-04-17T09:53:40.645 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:53:40.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.786+0000 7fb1e3c00640 1 Processor -- start 2024-04-17T09:53:40.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.786+0000 7fb1e3c00640 1 -- start start 2024-04-17T09:53:40.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.786+0000 7fb1e3c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1dc1481e0 0x7fb1dc071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:40.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.786+0000 7fb1e3c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1dc072330 0x7fb1dc0b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:40.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.786+0000 7fb1e3c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1dc06ef70 con 0x7fb1dc144f20 2024-04-17T09:53:40.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.786+0000 7fb1e3c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb1dc0aed20 con 0x7fb1dc072330 2024-04-17T09:53:40.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.786+0000 7fb1e3c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb1dc0af000 con 0x7fb1dc1481e0 2024-04-17T09:53:40.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.786+0000 7fb1e1c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1dc072330 0x7fb1dc0b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:40.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.786+0000 7fb1e1c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1dc072330 0x7fb1dc0b3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38898/0 (socket says 172.21.15.32:38898) 2024-04-17T09:53:40.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.786+0000 7fb1e1c00640 1 -- 172.21.15.32:0/2991756807 learned_addr learned my addr 172.21.15.32:0/2991756807 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:40.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.786+0000 7fb1d3e00640 1 --2- 172.21.15.32:0/2991756807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1dc1481e0 0x7fb1dc071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:40.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.790+0000 7fb1e1c00640 1 -- 172.21.15.32:0/2991756807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1dc1481e0 msgr2=0x7fb1dc071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:40.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.790+0000 7fb1e1c00640 1 --2- 172.21.15.32:0/2991756807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1dc1481e0 0x7fb1dc071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.790+0000 7fb1e1c00640 1 -- 172.21.15.32:0/2991756807 >> v1:172.21.15.144:6789/0 conn(0x7fb1dc144f20 legacy=0x7fb1dc06e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:40.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.790+0000 7fb1e1c00640 1 -- 172.21.15.32:0/2991756807 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1dc0af2e0 con 0x7fb1dc072330 2024-04-17T09:53:40.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.790+0000 7fb1dbe00640 1 -- 172.21.15.32:0/2991756807 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2292886695 0 0) 0x7fb1dc06ef70 con 0x7fb1dc144f20 2024-04-17T09:53:40.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.790+0000 7fb1e1c00640 1 --2- 172.21.15.32:0/2991756807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1dc072330 0x7fb1dc0b3490 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009e50 tx=0x7fb1d401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ae38960354e766 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:40.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.790+0000 7fb1dbe00640 1 -- 172.21.15.32:0/2991756807 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d40a5700 con 0x7fb1dc072330 2024-04-17T09:53:40.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.790+0000 7fb1dbe00640 1 -- 172.21.15.32:0/2991756807 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1d40a58a0 con 0x7fb1dc072330 2024-04-17T09:53:40.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.790+0000 7fb1dbe00640 1 -- 172.21.15.32:0/2991756807 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d40a5c70 con 0x7fb1dc072330 2024-04-17T09:53:40.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.790+0000 7fb1e3c00640 1 -- 172.21.15.32:0/2991756807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1dc072330 msgr2=0x7fb1dc0b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:40.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.790+0000 7fb1e3c00640 1 --2- 172.21.15.32:0/2991756807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1dc072330 0x7fb1dc0b3490 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009e50 tx=0x7fb1d401ca30 comp rx=0 tx=0).stop 2024-04-17T09:53:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.790+0000 7fb1e3c00640 1 -- 172.21.15.32:0/2991756807 shutdown_connections 2024-04-17T09:53:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.790+0000 7fb1e3c00640 1 --2- 172.21.15.32:0/2991756807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1dc072330 0x7fb1dc0b3490 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.790+0000 7fb1e3c00640 1 --2- 172.21.15.32:0/2991756807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1dc1481e0 0x7fb1dc071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.790+0000 7fb1e3c00640 1 -- 172.21.15.32:0/2991756807 >> 172.21.15.32:0/2991756807 conn(0x7fb1dc06ba30 msgr2=0x7fb1dc06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.790+0000 7fb1e3c00640 1 -- 172.21.15.32:0/2991756807 shutdown_connections 2024-04-17T09:53:40.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1e3c00640 1 -- 172.21.15.32:0/2991756807 wait complete. 2024-04-17T09:53:40.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1e3c00640 1 Processor -- start 2024-04-17T09:53:40.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1e3c00640 1 -- start start 2024-04-17T09:53:40.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1e3c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1dc072330 0x7fb1dc161470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:40.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1e3c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1dc144f20 0x7fb1dc0cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:40.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1e3c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1dc1481e0 0x7fb1dc0b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:40.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1e3c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc0b4360 con 0x7fb1dc072330 2024-04-17T09:53:40.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1e3c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc0b41e0 con 0x7fb1dc1481e0 2024-04-17T09:53:40.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1e3c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb1dc16b5d0 con 0x7fb1dc144f20 2024-04-17T09:53:40.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1e1200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1dc072330 0x7fb1dc161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:40.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1e1c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1dc1481e0 0x7fb1dc0b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:40.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1d3e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1dc144f20 0x7fb1dc0cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:40.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1e1200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1dc072330 0x7fb1dc161470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38902/0 (socket says 172.21.15.32:38902) 2024-04-17T09:53:40.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1e1200640 1 -- 172.21.15.32:0/2131879354 learned_addr learned my addr 172.21.15.32:0/2131879354 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:40.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1e1200640 1 -- 172.21.15.32:0/2131879354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1dc144f20 msgr2=0x7fb1dc0cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:40.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1e1200640 1 --2- 172.21.15.32:0/2131879354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1dc144f20 0x7fb1dc0cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1e1200640 1 -- 172.21.15.32:0/2131879354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1dc1481e0 msgr2=0x7fb1dc0b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:40.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1e1200640 1 --2- 172.21.15.32:0/2131879354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1dc1481e0 0x7fb1dc0b20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1e1200640 1 -- 172.21.15.32:0/2131879354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1dc168070 con 0x7fb1dc072330 2024-04-17T09:53:40.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1e1200640 1 --2- 172.21.15.32:0/2131879354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1dc072330 0x7fb1dc161470 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00c410 tx=0x7fb1cc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:40.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1d9600640 1 -- 172.21.15.32:0/2131879354 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc017070 con 0x7fb1dc072330 2024-04-17T09:53:40.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1d9600640 1 -- 172.21.15.32:0/2131879354 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1cc003600 con 0x7fb1dc072330 2024-04-17T09:53:40.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1d9600640 1 -- 172.21.15.32:0/2131879354 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc0039f0 con 0x7fb1dc072330 2024-04-17T09:53:40.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.794+0000 7fb1e3c00640 1 -- 172.21.15.32:0/2131879354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1dc15f8a0 con 0x7fb1dc072330 2024-04-17T09:53:40.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.798+0000 7fb1e3c00640 1 -- 172.21.15.32:0/2131879354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1dc167d00 con 0x7fb1dc072330 2024-04-17T09:53:40.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.798+0000 7fb1e3c00640 1 -- 172.21.15.32:0/2131879354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1dc068a90 con 0x7fb1dc072330 2024-04-17T09:53:40.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.798+0000 7fb1d9600640 1 -- 172.21.15.32:0/2131879354 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb1cc01a070 con 0x7fb1dc072330 2024-04-17T09:53:40.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.798+0000 7fb1d9600640 1 --2- 172.21.15.32:0/2131879354 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1b8068380 0x7fb1b806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:40.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.798+0000 7fb1d9600640 1 -- 172.21.15.32:0/2131879354 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb1cc085d20 con 0x7fb1dc072330 2024-04-17T09:53:40.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.798+0000 7fb1d3e00640 1 --2- 172.21.15.32:0/2131879354 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1b8068380 0x7fb1b806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:40.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.798+0000 7fb1d3e00640 1 --2- 172.21.15.32:0/2131879354 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1b8068380 0x7fb1b806a840 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc0cfd00 tx=0x7fb1c4009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:40.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.802+0000 7fb1d9600640 1 -- 172.21.15.32:0/2131879354 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fb1cc001090 con 0x7fb1dc072330 2024-04-17T09:53:40.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.970+0000 7fb1e3c00640 1 -- 172.21.15.32:0/2131879354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1dc146b20 con 0x7fb1dc072330 2024-04-17T09:53:40.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.970+0000 7fb1d9600640 1 -- 172.21.15.32:0/2131879354 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7fb1cc055c80 con 0x7fb1dc072330 2024-04-17T09:53:40.972 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:40.972 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:14.779535+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:53:40.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.974+0000 7fb1e3c00640 1 -- 172.21.15.32:0/2131879354 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1b8068380 msgr2=0x7fb1b806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:40.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.974+0000 7fb1e3c00640 1 --2- 172.21.15.32:0/2131879354 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1b8068380 0x7fb1b806a840 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc0cfd00 tx=0x7fb1c4009190 comp rx=0 tx=0).stop 2024-04-17T09:53:40.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.974+0000 7fb1e3c00640 1 -- 172.21.15.32:0/2131879354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1dc072330 msgr2=0x7fb1dc161470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:40.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.974+0000 7fb1e3c00640 1 --2- 172.21.15.32:0/2131879354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1dc072330 0x7fb1dc161470 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00c410 tx=0x7fb1cc00c8d0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.974+0000 7fb1e3c00640 1 -- 172.21.15.32:0/2131879354 shutdown_connections 2024-04-17T09:53:40.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.974+0000 7fb1e3c00640 1 --2- 172.21.15.32:0/2131879354 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1b8068380 0x7fb1b806a840 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.974+0000 7fb1e3c00640 1 --2- 172.21.15.32:0/2131879354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1dc144f20 0x7fb1dc0cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.974+0000 7fb1e3c00640 1 --2- 172.21.15.32:0/2131879354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1dc1481e0 0x7fb1dc0b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.974+0000 7fb1e3c00640 1 --2- 172.21.15.32:0/2131879354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1dc072330 0x7fb1dc161470 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:40.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.974+0000 7fb1e3c00640 1 -- 172.21.15.32:0/2131879354 >> 172.21.15.32:0/2131879354 conn(0x7fb1dc06ba30 msgr2=0x7fb1dc0b4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:40.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.974+0000 7fb1e3c00640 1 -- 172.21.15.32:0/2131879354 shutdown_connections 2024-04-17T09:53:40.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:40.974+0000 7fb1e3c00640 1 -- 172.21.15.32:0/2131879354 wait complete. 2024-04-17T09:53:40.993 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:53:11.198098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:53:12.118982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:53:40.994 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:53:41.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.138+0000 7f757ca00640 1 Processor -- start 2024-04-17T09:53:41.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.138+0000 7f757ca00640 1 -- start start 2024-04-17T09:53:41.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.138+0000 7f757ca00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f757815b7d0 0x7f757806e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:41.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.138+0000 7f757ca00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f757806f7c0 0x7f757806ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:41.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.138+0000 7f757ca00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f757806f270 0x7f757806bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:41.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.138+0000 7f757ca00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f75780cc940 con 0x7f757806f270 2024-04-17T09:53:41.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.138+0000 7f757ca00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f75780ccc20 con 0x7f757806f7c0 2024-04-17T09:53:41.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.138+0000 7f757ca00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f757806e0b0 con 0x7f757815b7d0 2024-04-17T09:53:41.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.138+0000 7f7576800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f757806f270 0x7f757806bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:41.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.138+0000 7f7576800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f757806f270 0x7f757806bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38916/0 (socket says 172.21.15.32:38916) 2024-04-17T09:53:41.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.138+0000 7f7576800640 1 -- 172.21.15.32:0/692607377 learned_addr learned my addr 172.21.15.32:0/692607377 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:41.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.138+0000 7f756d400640 1 --2- 172.21.15.32:0/692607377 >> v2:172.21.15.144:3300/0 conn(0x7f757806f7c0 0x7f757806ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:41.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.138+0000 7f7575e00640 1 --2- 172.21.15.32:0/692607377 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f757815b7d0 0x7f757806e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:41.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.138+0000 7f7576800640 1 -- 172.21.15.32:0/692607377 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f757815b7d0 msgr2=0x7f757806e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:41.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.138+0000 7f7576800640 1 --2- 172.21.15.32:0/692607377 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f757815b7d0 0x7f757806e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.138+0000 7f7576800640 1 -- 172.21.15.32:0/692607377 >> v2:172.21.15.144:3300/0 conn(0x7f757806f7c0 msgr2=0x7f757806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:41.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.138+0000 7f7576800640 1 --2- 172.21.15.32:0/692607377 >> v2:172.21.15.144:3300/0 conn(0x7f757806f7c0 0x7f757806ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.138+0000 7f7576800640 1 -- 172.21.15.32:0/692607377 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757806e390 con 0x7f757806f270 2024-04-17T09:53:41.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.142+0000 7f7576800640 1 --2- 172.21.15.32:0/692607377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757806f270 0x7f757806bf70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7578072ff0 tx=0x7f757001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97844ae97e958af5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:41.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.142+0000 7f7575400640 1 -- 172.21.15.32:0/692607377 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75700b2070 con 0x7f757806f270 2024-04-17T09:53:41.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.142+0000 7f7575400640 1 -- 172.21.15.32:0/692607377 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75700a9070 con 0x7f757806f270 2024-04-17T09:53:41.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.142+0000 7f7575400640 1 -- 172.21.15.32:0/692607377 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75700ad880 con 0x7f757806f270 2024-04-17T09:53:41.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.142+0000 7f757ca00640 1 -- 172.21.15.32:0/692607377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757806f270 msgr2=0x7f757806bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:41.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.142+0000 7f757ca00640 1 --2- 172.21.15.32:0/692607377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757806f270 0x7f757806bf70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7578072ff0 tx=0x7f757001c490 comp rx=0 tx=0).stop 2024-04-17T09:53:41.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.142+0000 7f757ca00640 1 -- 172.21.15.32:0/692607377 shutdown_connections 2024-04-17T09:53:41.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.142+0000 7f757ca00640 1 --2- 172.21.15.32:0/692607377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757806f270 0x7f757806bf70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.142+0000 7f757ca00640 1 --2- 172.21.15.32:0/692607377 >> v2:172.21.15.144:3300/0 conn(0x7f757806f7c0 0x7f757806ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.142+0000 7f757ca00640 1 --2- 172.21.15.32:0/692607377 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f757815b7d0 0x7f757806e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.142+0000 7f757ca00640 1 -- 172.21.15.32:0/692607377 >> 172.21.15.32:0/692607377 conn(0x7f75780b2120 msgr2=0x7f75780b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:41.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.142+0000 7f757ca00640 1 -- 172.21.15.32:0/692607377 shutdown_connections 2024-04-17T09:53:41.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.142+0000 7f757ca00640 1 -- 172.21.15.32:0/692607377 wait complete. 2024-04-17T09:53:41.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f757ca00640 1 Processor -- start 2024-04-17T09:53:41.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f757ca00640 1 -- start start 2024-04-17T09:53:41.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f757ca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757806f270 0x7f7578144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:41.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f757ca00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f757806f7c0 0x7f75781458e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:41.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f757ca00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f757815b7d0 0x7f7578146940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:41.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f757ca00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f75780cee30 con 0x7f757806f270 2024-04-17T09:53:41.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f757ca00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f75780cecb0 con 0x7f757806f7c0 2024-04-17T09:53:41.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f757ca00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f75780b3420 con 0x7f757815b7d0 2024-04-17T09:53:41.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f7575e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757806f270 0x7f7578144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:41.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f7575e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757806f270 0x7f7578144000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38928/0 (socket says 172.21.15.32:38928) 2024-04-17T09:53:41.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f7575e00640 1 -- 172.21.15.32:0/1019443210 learned_addr learned my addr 172.21.15.32:0/1019443210 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:41.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f756d400640 1 --2- 172.21.15.32:0/1019443210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f757806f7c0 0x7f75781458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:41.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f7576800640 1 --2- 172.21.15.32:0/1019443210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f757815b7d0 0x7f7578146940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:41.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f7575e00640 1 -- 172.21.15.32:0/1019443210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f757815b7d0 msgr2=0x7f7578146940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:41.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f7575e00640 1 --2- 172.21.15.32:0/1019443210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f757815b7d0 0x7f7578146940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f7575e00640 1 -- 172.21.15.32:0/1019443210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f757806f7c0 msgr2=0x7f75781458e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:41.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f7575e00640 1 --2- 172.21.15.32:0/1019443210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f757806f7c0 0x7f75781458e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f7575e00640 1 -- 172.21.15.32:0/1019443210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7578144840 con 0x7f757806f270 2024-04-17T09:53:41.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f756d400640 1 --2- 172.21.15.32:0/1019443210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f757806f7c0 0x7f75781458e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:41.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f7576800640 1 --2- 172.21.15.32:0/1019443210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f757815b7d0 0x7f7578146940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:41.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f7575e00640 1 --2- 172.21.15.32:0/1019443210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757806f270 0x7f7578144000 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f756800d5f0 tx=0x7f756800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:41.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f756ea00640 1 -- 172.21.15.32:0/1019443210 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75680125f0 con 0x7f757806f270 2024-04-17T09:53:41.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f756ea00640 1 -- 172.21.15.32:0/1019443210 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7568019070 con 0x7f757806f270 2024-04-17T09:53:41.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f756ea00640 1 -- 172.21.15.32:0/1019443210 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75680129b0 con 0x7f757806f270 2024-04-17T09:53:41.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f757ca00640 1 -- 172.21.15.32:0/1019443210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7578144b20 con 0x7f757806f270 2024-04-17T09:53:41.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.146+0000 7f757ca00640 1 -- 172.21.15.32:0/1019443210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75781450e0 con 0x7f757806f270 2024-04-17T09:53:41.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.150+0000 7f757ca00640 1 -- 172.21.15.32:0/1019443210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7578068a90 con 0x7f757806f270 2024-04-17T09:53:41.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.150+0000 7f756ea00640 1 -- 172.21.15.32:0/1019443210 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7568012b50 con 0x7f757806f270 2024-04-17T09:53:41.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.150+0000 7f756ea00640 1 --2- 172.21.15.32:0/1019443210 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7554068380 0x7f755406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:41.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.150+0000 7f756ea00640 1 -- 172.21.15.32:0/1019443210 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7568086ea0 con 0x7f757806f270 2024-04-17T09:53:41.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.150+0000 7f756d400640 1 --2- 172.21.15.32:0/1019443210 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7554068380 0x7f755406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:41.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.150+0000 7f756d400640 1 --2- 172.21.15.32:0/1019443210 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7554068380 0x7f755406a840 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f757806fce0 tx=0x7f756000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:41.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.154+0000 7f756ea00640 1 -- 172.21.15.32:0/1019443210 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f75680573c0 con 0x7f757806f270 2024-04-17T09:53:41.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.322+0000 7f757ca00640 1 -- 172.21.15.32:0/1019443210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7578144540 con 0x7f757806f270 2024-04-17T09:53:41.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.322+0000 7f756ea00640 1 -- 172.21.15.32:0/1019443210 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f7568056cc0 con 0x7f757806f270 2024-04-17T09:53:41.322 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:53:41.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.322+0000 7f757ca00640 1 -- 172.21.15.32:0/1019443210 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7554068380 msgr2=0x7f755406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:41.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.322+0000 7f757ca00640 1 --2- 172.21.15.32:0/1019443210 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7554068380 0x7f755406a840 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f757806fce0 tx=0x7f756000b040 comp rx=0 tx=0).stop 2024-04-17T09:53:41.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.322+0000 7f757ca00640 1 -- 172.21.15.32:0/1019443210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757806f270 msgr2=0x7f7578144000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:41.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.322+0000 7f757ca00640 1 --2- 172.21.15.32:0/1019443210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757806f270 0x7f7578144000 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f756800d5f0 tx=0x7f756800dab0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.322+0000 7f757ca00640 1 -- 172.21.15.32:0/1019443210 shutdown_connections 2024-04-17T09:53:41.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.322+0000 7f757ca00640 1 --2- 172.21.15.32:0/1019443210 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7554068380 0x7f755406a840 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.322+0000 7f757ca00640 1 --2- 172.21.15.32:0/1019443210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f757815b7d0 0x7f7578146940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.322+0000 7f757ca00640 1 --2- 172.21.15.32:0/1019443210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f757806f7c0 0x7f75781458e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.322+0000 7f757ca00640 1 --2- 172.21.15.32:0/1019443210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757806f270 0x7f7578144000 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.322+0000 7f757ca00640 1 -- 172.21.15.32:0/1019443210 >> 172.21.15.32:0/1019443210 conn(0x7f75780b2120 msgr2=0x7f7578158df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:41.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.322+0000 7f757ca00640 1 -- 172.21.15.32:0/1019443210 shutdown_connections 2024-04-17T09:53:41.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.322+0000 7f757ca00640 1 -- 172.21.15.32:0/1019443210 wait complete. 2024-04-17T09:53:41.343 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:53:41.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.486+0000 7f45c3c00640 1 Processor -- start 2024-04-17T09:53:41.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.486+0000 7f45c3c00640 1 -- start start 2024-04-17T09:53:41.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.486+0000 7f45c3c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f45bc06f7c0 0x7f45bc06ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:41.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.486+0000 7f45c3c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45bc0cc940 con 0x7f45bc15b7d0 2024-04-17T09:53:41.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.486+0000 7f45c3c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45bc0ccc20 con 0x7f45bc06f270 2024-04-17T09:53:41.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.486+0000 7f45c3c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f45bc06e0b0 con 0x7f45bc06f7c0 2024-04-17T09:53:41.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.486+0000 7f45c1200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f45bc15b7d0 0x7f45bc06e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44310/0 (socket says 172.21.15.32:44310) 2024-04-17T09:53:41.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.486+0000 7f45c1200640 1 -- 172.21.15.32:0/3588882016 learned_addr learned my addr 172.21.15.32:0/3588882016 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:41.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45b3e00640 1 --2- 172.21.15.32:0/3588882016 >> v2:172.21.15.144:3300/0 conn(0x7f45bc06f7c0 0x7f45bc06ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:41.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45bbe00640 1 -- 172.21.15.32:0/3588882016 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 632674867 0 0) 0x7f45bc0cc940 con 0x7f45bc15b7d0 2024-04-17T09:53:41.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45bbe00640 1 -- 172.21.15.32:0/3588882016 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45a0003530 con 0x7f45bc15b7d0 2024-04-17T09:53:41.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45bbe00640 1 -- 172.21.15.32:0/3588882016 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 116922787 0 0) 0x7f45a0003530 con 0x7f45bc15b7d0 2024-04-17T09:53:41.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45bbe00640 1 -- 172.21.15.32:0/3588882016 >> v2:172.21.15.144:3300/0 conn(0x7f45bc06f7c0 msgr2=0x7f45bc06ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:41.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45bbe00640 1 --2- 172.21.15.32:0/3588882016 >> v2:172.21.15.144:3300/0 conn(0x7f45bc06f7c0 0x7f45bc06ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45bbe00640 1 -- 172.21.15.32:0/3588882016 >> v1:172.21.15.144:6789/0 conn(0x7f45bc06f270 legacy=0x7f45bc0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:41.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45bbe00640 1 -- 172.21.15.32:0/3588882016 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc06e390 con 0x7f45bc15b7d0 2024-04-17T09:53:41.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45bbe00640 1 -- 172.21.15.32:0/3588882016 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f45ac001680 con 0x7f45bc15b7d0 2024-04-17T09:53:41.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45bbe00640 1 -- 172.21.15.32:0/3588882016 >> v1:172.21.15.32:6789/0 conn(0x7f45bc15b7d0 legacy=0x7f45bc06e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:41.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45bbe00640 1 --2- 172.21.15.32:0/3588882016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a00039c0 0x7f45a0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:41.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45bbe00640 1 --2- 172.21.15.32:0/3588882016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45a0004810 0x7f45a0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:41.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45bbe00640 1 --2- 172.21.15.32:0/3588882016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45a00051d0 0x7f45a000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:41.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45bbe00640 1 -- 172.21.15.32:0/3588882016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc06e0b0 con 0x7f45a00039c0 2024-04-17T09:53:41.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45bbe00640 1 -- 172.21.15.32:0/3588882016 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f45a000c170 con 0x7f45a00051d0 2024-04-17T09:53:41.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45bbe00640 1 -- 172.21.15.32:0/3588882016 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f45a000c370 con 0x7f45a0004810 2024-04-17T09:53:41.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45c1200640 1 --2- 172.21.15.32:0/3588882016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a00039c0 0x7f45a0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:41.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45b3e00640 1 --2- 172.21.15.32:0/3588882016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45a0004810 0x7f45a0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:41.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45c1c00640 1 --2- 172.21.15.32:0/3588882016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45a00051d0 0x7f45a000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:41.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45c1200640 1 -- 172.21.15.32:0/3588882016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45a0004810 msgr2=0x7f45a0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:41.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45c1200640 1 --2- 172.21.15.32:0/3588882016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45a0004810 0x7f45a0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45c1200640 1 -- 172.21.15.32:0/3588882016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45a00051d0 msgr2=0x7f45a000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:41.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45c1200640 1 --2- 172.21.15.32:0/3588882016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45a00051d0 0x7f45a000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45c1200640 1 -- 172.21.15.32:0/3588882016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a0003530 con 0x7f45a00039c0 2024-04-17T09:53:41.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45c1c00640 1 --2- 172.21.15.32:0/3588882016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45a00051d0 0x7f45a000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:41.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.490+0000 7f45c1c00640 1 -- 172.21.15.32:0/3588882016 reap_dead start 2024-04-17T09:53:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.494+0000 7f45c1200640 1 --2- 172.21.15.32:0/3588882016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a00039c0 0x7f45a0003de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00c210 tx=0x7f45ac01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53844e575856c80f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.494+0000 7f45bbe00640 1 -- 172.21.15.32:0/3588882016 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac007d40 con 0x7f45a00039c0 2024-04-17T09:53:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.494+0000 7f45bbe00640 1 -- 172.21.15.32:0/3588882016 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45ac0a9070 con 0x7f45a00039c0 2024-04-17T09:53:41.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.494+0000 7f45bbe00640 1 -- 172.21.15.32:0/3588882016 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac005d80 con 0x7f45a00039c0 2024-04-17T09:53:41.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.494+0000 7f45c3c00640 1 -- 172.21.15.32:0/3588882016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a00039c0 msgr2=0x7f45a0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:41.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.494+0000 7f45c3c00640 1 --2- 172.21.15.32:0/3588882016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a00039c0 0x7f45a0003de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00c210 tx=0x7f45ac01cca0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.494+0000 7f45c3c00640 1 -- 172.21.15.32:0/3588882016 shutdown_connections 2024-04-17T09:53:41.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.494+0000 7f45c3c00640 1 --2- 172.21.15.32:0/3588882016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a00039c0 0x7f45a0003de0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.494+0000 7f45c3c00640 1 -- 172.21.15.32:0/3588882016 >> 172.21.15.32:0/3588882016 conn(0x7f45bc0b2120 msgr2=0x7f45bc0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:41.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.494+0000 7f45c3c00640 1 -- 172.21.15.32:0/3588882016 shutdown_connections 2024-04-17T09:53:41.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.494+0000 7f45c3c00640 1 -- 172.21.15.32:0/3588882016 wait complete. 2024-04-17T09:53:41.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.494+0000 7f45c3c00640 1 Processor -- start 2024-04-17T09:53:41.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.494+0000 7f45c3c00640 1 -- start start 2024-04-17T09:53:41.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.494+0000 7f45c3c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45a00039c0 0x7f45bc0ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.494+0000 7f45c3c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc1ec1d0 0x7f45bc1ec630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.494+0000 7f45c3c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45bc15d100 0x7f45bc15d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.494+0000 7f45c3c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc0ce5e0 con 0x7f45bc1ec1d0 2024-04-17T09:53:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.494+0000 7f45c3c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc0ce460 con 0x7f45a00039c0 2024-04-17T09:53:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.494+0000 7f45c3c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f45bc0ce760 con 0x7f45bc15d100 2024-04-17T09:53:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.498+0000 7f45b3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc1ec1d0 0x7f45bc1ec630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.498+0000 7f45b3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc1ec1d0 0x7f45bc1ec630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38938/0 (socket says 172.21.15.32:38938) 2024-04-17T09:53:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.498+0000 7f45b3e00640 1 -- 172.21.15.32:0/3764161800 learned_addr learned my addr 172.21.15.32:0/3764161800 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.498+0000 7f45c1200640 1 --2- 172.21.15.32:0/3764161800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45a00039c0 0x7f45bc0ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.498+0000 7f45c1c00640 1 --2- 172.21.15.32:0/3764161800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45bc15d100 0x7f45bc15d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.498+0000 7f45b3e00640 1 -- 172.21.15.32:0/3764161800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45bc15d100 msgr2=0x7f45bc15d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.498+0000 7f45b3e00640 1 --2- 172.21.15.32:0/3764161800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45bc15d100 0x7f45bc15d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.498+0000 7f45b3e00640 1 -- 172.21.15.32:0/3764161800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45a00039c0 msgr2=0x7f45bc0ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:41.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.498+0000 7f45b3e00640 1 --2- 172.21.15.32:0/3764161800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45a00039c0 0x7f45bc0ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.498+0000 7f45b3e00640 1 -- 172.21.15.32:0/3764161800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc15de30 con 0x7f45bc1ec1d0 2024-04-17T09:53:41.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.498+0000 7f45c1200640 1 --2- 172.21.15.32:0/3764161800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45a00039c0 0x7f45bc0ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:41.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.498+0000 7f45b3e00640 1 --2- 172.21.15.32:0/3764161800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc1ec1d0 0x7f45bc1ec630 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f45a4011a20 tx=0x7f45a4011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:41.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.498+0000 7f45b9600640 1 -- 172.21.15.32:0/3764161800 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a400fbb0 con 0x7f45bc1ec1d0 2024-04-17T09:53:41.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.498+0000 7f45b9600640 1 -- 172.21.15.32:0/3764161800 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45a400fd50 con 0x7f45bc1ec1d0 2024-04-17T09:53:41.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.498+0000 7f45c3c00640 1 -- 172.21.15.32:0/3764161800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45bc15e110 con 0x7f45bc1ec1d0 2024-04-17T09:53:41.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.498+0000 7f45b9600640 1 -- 172.21.15.32:0/3764161800 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a4014640 con 0x7f45bc1ec1d0 2024-04-17T09:53:41.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.498+0000 7f45c3c00640 1 -- 172.21.15.32:0/3764161800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45bc15e6d0 con 0x7f45bc1ec1d0 2024-04-17T09:53:41.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.498+0000 7f45c3c00640 1 -- 172.21.15.32:0/3764161800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45bc068a90 con 0x7f45bc1ec1d0 2024-04-17T09:53:41.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.498+0000 7f45b9600640 1 -- 172.21.15.32:0/3764161800 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f45a4012040 con 0x7f45bc1ec1d0 2024-04-17T09:53:41.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.502+0000 7f45b9600640 1 --2- 172.21.15.32:0/3764161800 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f45980683b0 0x7f459806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:41.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.502+0000 7f45b9600640 1 -- 172.21.15.32:0/3764161800 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f45a40871e0 con 0x7f45bc1ec1d0 2024-04-17T09:53:41.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.502+0000 7f45c1200640 1 --2- 172.21.15.32:0/3764161800 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f45980683b0 0x7f459806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:41.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.502+0000 7f45c1200640 1 --2- 172.21.15.32:0/3764161800 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f45980683b0 0x7f459806a870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0ab000 tx=0x7f45ac0b0040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:41.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.506+0000 7f45b9600640 1 -- 172.21.15.32:0/3764161800 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f45a4019030 con 0x7f45bc1ec1d0 2024-04-17T09:53:41.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.670+0000 7f45c3c00640 1 -- 172.21.15.32:0/3764161800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f45bc0cd4b0 con 0x7f45bc1ec1d0 2024-04-17T09:53:41.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.670+0000 7f45b9600640 1 -- 172.21.15.32:0/3764161800 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f45a4057670 con 0x7f45bc1ec1d0 2024-04-17T09:53:41.670 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:53:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.670+0000 7f45c3c00640 1 -- 172.21.15.32:0/3764161800 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f45980683b0 msgr2=0x7f459806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.670+0000 7f45c3c00640 1 --2- 172.21.15.32:0/3764161800 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f45980683b0 0x7f459806a870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0ab000 tx=0x7f45ac0b0040 comp rx=0 tx=0).stop 2024-04-17T09:53:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.670+0000 7f45c3c00640 1 -- 172.21.15.32:0/3764161800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc1ec1d0 msgr2=0x7f45bc1ec630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.670+0000 7f45c3c00640 1 --2- 172.21.15.32:0/3764161800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc1ec1d0 0x7f45bc1ec630 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f45a4011a20 tx=0x7f45a4011ee0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.670+0000 7f45c3c00640 1 -- 172.21.15.32:0/3764161800 shutdown_connections 2024-04-17T09:53:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.670+0000 7f45c3c00640 1 --2- 172.21.15.32:0/3764161800 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f45980683b0 0x7f459806a870 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.670+0000 7f45c3c00640 1 --2- 172.21.15.32:0/3764161800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45bc15d100 0x7f45bc15d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.670+0000 7f45c3c00640 1 --2- 172.21.15.32:0/3764161800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc1ec1d0 0x7f45bc1ec630 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.670+0000 7f45c3c00640 1 --2- 172.21.15.32:0/3764161800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45a00039c0 0x7f45bc0ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.670+0000 7f45c3c00640 1 -- 172.21.15.32:0/3764161800 >> 172.21.15.32:0/3764161800 conn(0x7f45bc0b2120 msgr2=0x7f45bc0b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:41.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.670+0000 7f45c3c00640 1 -- 172.21.15.32:0/3764161800 shutdown_connections 2024-04-17T09:53:41.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.670+0000 7f45c3c00640 1 -- 172.21.15.32:0/3764161800 wait complete. 2024-04-17T09:53:41.691 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:53:41.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.838+0000 7fe44de00640 1 Processor -- start 2024-04-17T09:53:41.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.838+0000 7fe44de00640 1 -- start start 2024-04-17T09:53:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.838+0000 7fe44de00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe44809b800 0x7fe4480d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.838+0000 7fe44de00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe44806ff80 0x7fe4480d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.838+0000 7fe44de00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4480cd410 con 0x7fe44809a9f0 2024-04-17T09:53:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.838+0000 7fe44de00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe4480cd6f0 con 0x7fe44806ff80 2024-04-17T09:53:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.838+0000 7fe44de00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe4480cd9d0 con 0x7fe44809b800 2024-04-17T09:53:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.838+0000 7fe44d400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe44806ff80 0x7fe4480d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.838+0000 7fe447e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe44809b800 0x7fe4480d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.838+0000 7fe44d400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe44806ff80 0x7fe4480d08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:49372/0 (socket says 172.21.15.32:49372) 2024-04-17T09:53:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.838+0000 7fe44ca00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fe44809a9f0 0x7fe44809add0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:52096/0 (socket says 172.21.15.32:52096) 2024-04-17T09:53:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.838+0000 7fe44d400640 1 -- 172.21.15.32:0/2462856962 learned_addr learned my addr 172.21.15.32:0/2462856962 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.838+0000 7fe447e00640 1 -- 172.21.15.32:0/2462856962 >> v2:172.21.15.144:3300/0 conn(0x7fe44806ff80 msgr2=0x7fe4480d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.838+0000 7fe447e00640 1 --2- 172.21.15.32:0/2462856962 >> v2:172.21.15.144:3300/0 conn(0x7fe44806ff80 0x7fe4480d08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.838+0000 7fe447e00640 1 -- 172.21.15.32:0/2462856962 >> v1:172.21.15.144:6789/0 conn(0x7fe44809a9f0 legacy=0x7fe44809add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.838+0000 7fe447e00640 1 -- 172.21.15.32:0/2462856962 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4480cdcb0 con 0x7fe44809b800 2024-04-17T09:53:41.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.842+0000 7fe447e00640 1 --2- 172.21.15.32:0/2462856962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe44809b800 0x7fe4480d03b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe4340097c0 tx=0x7fe43401c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=117c8a3748baae22 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:41.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.842+0000 7fe447400640 1 -- 172.21.15.32:0/2462856962 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4340b0070 con 0x7fe44809b800 2024-04-17T09:53:41.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.842+0000 7fe447400640 1 -- 172.21.15.32:0/2462856962 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4340ab600 con 0x7fe44809b800 2024-04-17T09:53:41.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.842+0000 7fe447400640 1 -- 172.21.15.32:0/2462856962 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4340ab9d0 con 0x7fe44809b800 2024-04-17T09:53:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.842+0000 7fe44de00640 1 -- 172.21.15.32:0/2462856962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe44809b800 msgr2=0x7fe4480d03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.842+0000 7fe44de00640 1 --2- 172.21.15.32:0/2462856962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe44809b800 0x7fe4480d03b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe4340097c0 tx=0x7fe43401c350 comp rx=0 tx=0).stop 2024-04-17T09:53:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.842+0000 7fe44de00640 1 -- 172.21.15.32:0/2462856962 shutdown_connections 2024-04-17T09:53:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.842+0000 7fe44de00640 1 --2- 172.21.15.32:0/2462856962 >> v2:172.21.15.144:3300/0 conn(0x7fe44806ff80 0x7fe4480d08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.842+0000 7fe44de00640 1 --2- 172.21.15.32:0/2462856962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe44809b800 0x7fe4480d03b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.842+0000 7fe44de00640 1 -- 172.21.15.32:0/2462856962 >> 172.21.15.32:0/2462856962 conn(0x7fe4480b3fa0 msgr2=0x7fe4480b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.842+0000 7fe44de00640 1 -- 172.21.15.32:0/2462856962 shutdown_connections 2024-04-17T09:53:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.842+0000 7fe44de00640 1 -- 172.21.15.32:0/2462856962 wait complete. 2024-04-17T09:53:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe44de00640 1 Processor -- start 2024-04-17T09:53:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe44de00640 1 -- start start 2024-04-17T09:53:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe44de00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe44806ff80 0x7fe44815d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe44de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe44809a9f0 0x7fe448161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe44de00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe44809b800 0x7fe4481583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe44de00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe4480d13c0 con 0x7fe44809a9f0 2024-04-17T09:53:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe44de00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4480d1240 con 0x7fe44809b800 2024-04-17T09:53:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe44de00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe448071060 con 0x7fe44806ff80 2024-04-17T09:53:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe447e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe44809a9f0 0x7fe448161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe44ca00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe44806ff80 0x7fe44815d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe44d400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe44809b800 0x7fe4481583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe447e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe44809a9f0 0x7fe448161aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38948/0 (socket says 172.21.15.32:38948) 2024-04-17T09:53:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe447e00640 1 -- 172.21.15.32:0/2105891541 learned_addr learned my addr 172.21.15.32:0/2105891541 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe447e00640 1 -- 172.21.15.32:0/2105891541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe44806ff80 msgr2=0x7fe44815d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe447e00640 1 --2- 172.21.15.32:0/2105891541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe44806ff80 0x7fe44815d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe447e00640 1 -- 172.21.15.32:0/2105891541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe44809b800 msgr2=0x7fe4481583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe447e00640 1 --2- 172.21.15.32:0/2105891541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe44809b800 0x7fe4481583c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe447e00640 1 -- 172.21.15.32:0/2105891541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe448161fe0 con 0x7fe44809a9f0 2024-04-17T09:53:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe44ca00640 1 --2- 172.21.15.32:0/2105891541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe44806ff80 0x7fe44815d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:53:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe447e00640 1 --2- 172.21.15.32:0/2105891541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe44809a9f0 0x7fe448161aa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe434019f80 tx=0x7fe4340190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe444c00640 1 -- 172.21.15.32:0/2105891541 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4340ba020 con 0x7fe44809a9f0 2024-04-17T09:53:41.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe444c00640 1 -- 172.21.15.32:0/2105891541 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4340b5460 con 0x7fe44809a9f0 2024-04-17T09:53:41.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe44de00640 1 -- 172.21.15.32:0/2105891541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44815dea0 con 0x7fe44809a9f0 2024-04-17T09:53:41.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe444c00640 1 -- 172.21.15.32:0/2105891541 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4340b0040 con 0x7fe44809a9f0 2024-04-17T09:53:41.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.846+0000 7fe44de00640 1 -- 172.21.15.32:0/2105891541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe44815e460 con 0x7fe44809a9f0 2024-04-17T09:53:41.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.850+0000 7fe44de00640 1 -- 172.21.15.32:0/2105891541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe44804fda0 con 0x7fe44809a9f0 2024-04-17T09:53:41.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.850+0000 7fe444c00640 1 -- 172.21.15.32:0/2105891541 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe4340bd020 con 0x7fe44809a9f0 2024-04-17T09:53:41.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.850+0000 7fe444c00640 1 --2- 172.21.15.32:0/2105891541 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe428068380 0x7fe42806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:41.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.850+0000 7fe444c00640 1 -- 172.21.15.32:0/2105891541 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe434120ca0 con 0x7fe44809a9f0 2024-04-17T09:53:41.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.850+0000 7fe44ca00640 1 --2- 172.21.15.32:0/2105891541 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe428068380 0x7fe42806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:41.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.850+0000 7fe44ca00640 1 --2- 172.21.15.32:0/2105891541 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe428068380 0x7fe42806a840 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe43c000f70 tx=0x7fe43c007e90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:41.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:41.854+0000 7fe444c00640 1 -- 172.21.15.32:0/2105891541 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fe4340c4030 con 0x7fe44809a9f0 2024-04-17T09:53:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.018+0000 7fe44de00640 1 -- 172.21.15.32:0/2105891541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe4480b4ce0 con 0x7fe44809a9f0 2024-04-17T09:53:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.018+0000 7fe444c00640 1 -- 172.21.15.32:0/2105891541 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fe434016090 con 0x7fe44809a9f0 2024-04-17T09:53:42.020 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:53:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.022+0000 7fe44de00640 1 -- 172.21.15.32:0/2105891541 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe428068380 msgr2=0x7fe42806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.022+0000 7fe44de00640 1 --2- 172.21.15.32:0/2105891541 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe428068380 0x7fe42806a840 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe43c000f70 tx=0x7fe43c007e90 comp rx=0 tx=0).stop 2024-04-17T09:53:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.022+0000 7fe44de00640 1 -- 172.21.15.32:0/2105891541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe44809a9f0 msgr2=0x7fe448161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.022+0000 7fe44de00640 1 --2- 172.21.15.32:0/2105891541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe44809a9f0 0x7fe448161aa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe434019f80 tx=0x7fe4340190f0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.022+0000 7fe44de00640 1 -- 172.21.15.32:0/2105891541 shutdown_connections 2024-04-17T09:53:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.022+0000 7fe44de00640 1 --2- 172.21.15.32:0/2105891541 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe428068380 0x7fe42806a840 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.022+0000 7fe44de00640 1 --2- 172.21.15.32:0/2105891541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe44809b800 0x7fe4481583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.022+0000 7fe44de00640 1 --2- 172.21.15.32:0/2105891541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe44809a9f0 0x7fe448161aa0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.022+0000 7fe44de00640 1 --2- 172.21.15.32:0/2105891541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe44806ff80 0x7fe44815d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.022+0000 7fe44de00640 1 -- 172.21.15.32:0/2105891541 >> 172.21.15.32:0/2105891541 conn(0x7fe4480b3fa0 msgr2=0x7fe4480b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.022+0000 7fe44de00640 1 -- 172.21.15.32:0/2105891541 shutdown_connections 2024-04-17T09:53:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.022+0000 7fe44de00640 1 -- 172.21.15.32:0/2105891541 wait complete. 2024-04-17T09:53:42.041 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:53:42.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.182+0000 7fc90a200640 1 Processor -- start 2024-04-17T09:53:42.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.182+0000 7fc90a200640 1 -- start start 2024-04-17T09:53:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.182+0000 7fc90a200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc90415eb30 0x7fc90409b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.182+0000 7fc90a200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc90409b100 0x7fc9040ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.182+0000 7fc90a200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc90409bcb0 con 0x7fc9040d0890 2024-04-17T09:53:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.182+0000 7fc90a200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc90406dd60 con 0x7fc90409b100 2024-04-17T09:53:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.182+0000 7fc90a200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc90415ef10 con 0x7fc90415eb30 2024-04-17T09:53:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.182+0000 7fc903e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc90409b100 0x7fc9040ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.182+0000 7fc903e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc90409b100 0x7fc9040ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38956/0 (socket says 172.21.15.32:38956) 2024-04-17T09:53:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.182+0000 7fc903e00640 1 -- 172.21.15.32:0/2907251729 learned_addr learned my addr 172.21.15.32:0/2907251729 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.182+0000 7fc903400640 1 --2- 172.21.15.32:0/2907251729 >> v2:172.21.15.144:3300/0 conn(0x7fc90415eb30 0x7fc90409b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:42.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.182+0000 7fc903e00640 1 -- 172.21.15.32:0/2907251729 >> v2:172.21.15.144:3300/0 conn(0x7fc90415eb30 msgr2=0x7fc90409b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:42.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.182+0000 7fc903e00640 1 --2- 172.21.15.32:0/2907251729 >> v2:172.21.15.144:3300/0 conn(0x7fc90415eb30 0x7fc90409b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.182+0000 7fc903e00640 1 -- 172.21.15.32:0/2907251729 >> v1:172.21.15.144:6789/0 conn(0x7fc9040d0890 legacy=0x7fc90409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:42.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.182+0000 7fc903e00640 1 -- 172.21.15.32:0/2907251729 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9040d0db0 con 0x7fc90409b100 2024-04-17T09:53:42.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.186+0000 7fc903e00640 1 --2- 172.21.15.32:0/2907251729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc90409b100 0x7fc9040ccb30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc904073350 tx=0x7fc8fc01f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b35725176f38017 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:42.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.186+0000 7fc902a00640 1 -- 172.21.15.32:0/2907251729 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc0b4070 con 0x7fc90409b100 2024-04-17T09:53:42.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.186+0000 7fc902a00640 1 -- 172.21.15.32:0/2907251729 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8fc0af400 con 0x7fc90409b100 2024-04-17T09:53:42.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.186+0000 7fc902a00640 1 -- 172.21.15.32:0/2907251729 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc0af7d0 con 0x7fc90409b100 2024-04-17T09:53:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.186+0000 7fc90a200640 1 -- 172.21.15.32:0/2907251729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc90409b100 msgr2=0x7fc9040ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.186+0000 7fc90a200640 1 --2- 172.21.15.32:0/2907251729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc90409b100 0x7fc9040ccb30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc904073350 tx=0x7fc8fc01f1e0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.186+0000 7fc90a200640 1 -- 172.21.15.32:0/2907251729 shutdown_connections 2024-04-17T09:53:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.186+0000 7fc90a200640 1 --2- 172.21.15.32:0/2907251729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc90409b100 0x7fc9040ccb30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.186+0000 7fc90a200640 1 --2- 172.21.15.32:0/2907251729 >> v2:172.21.15.144:3300/0 conn(0x7fc90415eb30 0x7fc90409b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.186+0000 7fc90a200640 1 -- 172.21.15.32:0/2907251729 >> 172.21.15.32:0/2907251729 conn(0x7fc9040b3fd0 msgr2=0x7fc9040b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.186+0000 7fc90a200640 1 -- 172.21.15.32:0/2907251729 shutdown_connections 2024-04-17T09:53:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.186+0000 7fc90a200640 1 -- 172.21.15.32:0/2907251729 wait complete. 2024-04-17T09:53:42.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc90a200640 1 Processor -- start 2024-04-17T09:53:42.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc90a200640 1 -- start start 2024-04-17T09:53:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc90a200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc90409b100 0x7fc904153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc90a200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9040d0890 0x7fc90409c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc90a200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc90415eb30 0x7fc90409e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc90a200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc9040d1650 con 0x7fc9040d0890 2024-04-17T09:53:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc90a200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc9040d1370 con 0x7fc90415eb30 2024-04-17T09:53:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc90a200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc9040d1930 con 0x7fc90409b100 2024-04-17T09:53:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc8fbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9040d0890 0x7fc90409c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc8fbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9040d0890 0x7fc90409c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38968/0 (socket says 172.21.15.32:38968) 2024-04-17T09:53:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc8fbe00640 1 -- 172.21.15.32:0/2895387464 learned_addr learned my addr 172.21.15.32:0/2895387464 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc903400640 1 --2- 172.21.15.32:0/2895387464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc90409b100 0x7fc904153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc903e00640 1 --2- 172.21.15.32:0/2895387464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc90415eb30 0x7fc90409e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc8fbe00640 1 -- 172.21.15.32:0/2895387464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc90409b100 msgr2=0x7fc904153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc8fbe00640 1 --2- 172.21.15.32:0/2895387464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc90409b100 0x7fc904153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc8fbe00640 1 -- 172.21.15.32:0/2895387464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc90415eb30 msgr2=0x7fc90409e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc8fbe00640 1 --2- 172.21.15.32:0/2895387464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc90415eb30 0x7fc90409e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc8fbe00640 1 -- 172.21.15.32:0/2895387464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90409ccb0 con 0x7fc9040d0890 2024-04-17T09:53:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc8fbe00640 1 --2- 172.21.15.32:0/2895387464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9040d0890 0x7fc90409c2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec00c430 tx=0x7fc8ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc8fb400640 1 -- 172.21.15.32:0/2895387464 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec0052d0 con 0x7fc9040d0890 2024-04-17T09:53:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc8fb400640 1 -- 172.21.15.32:0/2895387464 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8ec005470 con 0x7fc9040d0890 2024-04-17T09:53:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc8fb400640 1 -- 172.21.15.32:0/2895387464 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec007260 con 0x7fc9040d0890 2024-04-17T09:53:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc90a200640 1 -- 172.21.15.32:0/2895387464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90409cf90 con 0x7fc9040d0890 2024-04-17T09:53:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc90a200640 1 -- 172.21.15.32:0/2895387464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc90409d550 con 0x7fc9040d0890 2024-04-17T09:53:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.190+0000 7fc90a200640 1 -- 172.21.15.32:0/2895387464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc904068a90 con 0x7fc9040d0890 2024-04-17T09:53:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.194+0000 7fc8fb400640 1 -- 172.21.15.32:0/2895387464 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc8ec007400 con 0x7fc9040d0890 2024-04-17T09:53:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.194+0000 7fc8fb400640 1 --2- 172.21.15.32:0/2895387464 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc8e0068380 0x7fc8e006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.194+0000 7fc8fb400640 1 -- 172.21.15.32:0/2895387464 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc8ec085f00 con 0x7fc9040d0890 2024-04-17T09:53:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.194+0000 7fc903400640 1 --2- 172.21.15.32:0/2895387464 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc8e0068380 0x7fc8e006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.194+0000 7fc903400640 1 --2- 172.21.15.32:0/2895387464 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc8e0068380 0x7fc8e006a840 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009ac0 tx=0x7fc8f4009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:42.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.198+0000 7fc8fb400640 1 -- 172.21.15.32:0/2895387464 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc8ec055dd0 con 0x7fc9040d0890 2024-04-17T09:53:42.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.362+0000 7fc90a200640 1 -- 172.21.15.32:0/2895387464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc90415f090 con 0x7fc9040d0890 2024-04-17T09:53:42.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.362+0000 7fc8fb400640 1 -- 172.21.15.32:0/2895387464 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7fc8ec05a0d0 con 0x7fc9040d0890 2024-04-17T09:53:42.364 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:42.364 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:14.779535+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:53:42.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.366+0000 7fc90a200640 1 -- 172.21.15.32:0/2895387464 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc8e0068380 msgr2=0x7fc8e006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:42.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.366+0000 7fc90a200640 1 --2- 172.21.15.32:0/2895387464 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc8e0068380 0x7fc8e006a840 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009ac0 tx=0x7fc8f4009190 comp rx=0 tx=0).stop 2024-04-17T09:53:42.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.366+0000 7fc90a200640 1 -- 172.21.15.32:0/2895387464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9040d0890 msgr2=0x7fc90409c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:42.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.366+0000 7fc90a200640 1 --2- 172.21.15.32:0/2895387464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9040d0890 0x7fc90409c2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec00c430 tx=0x7fc8ec00c8f0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.366+0000 7fc90a200640 1 -- 172.21.15.32:0/2895387464 shutdown_connections 2024-04-17T09:53:42.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.366+0000 7fc90a200640 1 --2- 172.21.15.32:0/2895387464 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc8e0068380 0x7fc8e006a840 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.366+0000 7fc90a200640 1 --2- 172.21.15.32:0/2895387464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc90415eb30 0x7fc90409e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.366+0000 7fc90a200640 1 --2- 172.21.15.32:0/2895387464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9040d0890 0x7fc90409c2f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.366+0000 7fc90a200640 1 --2- 172.21.15.32:0/2895387464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc90409b100 0x7fc904153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.366+0000 7fc90a200640 1 -- 172.21.15.32:0/2895387464 >> 172.21.15.32:0/2895387464 conn(0x7fc9040b3fd0 msgr2=0x7fc9040b7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:42.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.366+0000 7fc90a200640 1 -- 172.21.15.32:0/2895387464 shutdown_connections 2024-04-17T09:53:42.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.366+0000 7fc90a200640 1 -- 172.21.15.32:0/2895387464 wait complete. 2024-04-17T09:53:42.385 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:53:11.198098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:53:12.118982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:53:42.386 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:53:42.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.530+0000 7fd52c200640 1 Processor -- start 2024-04-17T09:53:42.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.530+0000 7fd52c200640 1 -- start start 2024-04-17T09:53:42.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.530+0000 7fd52c200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd52415c4c0 0x7fd5240adb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:42.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.530+0000 7fd52c200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd52406c140 0x7fd5240b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:42.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.530+0000 7fd52c200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd52415fe50 con 0x7fd52415f9c0 2024-04-17T09:53:42.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.530+0000 7fd52c200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd5241629a0 con 0x7fd52415c4c0 2024-04-17T09:53:42.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.530+0000 7fd52c200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd524162c80 con 0x7fd52406c140 2024-04-17T09:53:42.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.530+0000 7fd529800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd52415c4c0 0x7fd5240adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:42.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.530+0000 7fd529800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd52415c4c0 0x7fd5240adb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38972/0 (socket says 172.21.15.32:38972) 2024-04-17T09:53:42.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.530+0000 7fd529800640 1 -- 172.21.15.32:0/3109306030 learned_addr learned my addr 172.21.15.32:0/3109306030 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:42.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.530+0000 7fd529800640 1 -- 172.21.15.32:0/3109306030 >> v2:172.21.15.144:3300/0 conn(0x7fd52406c140 msgr2=0x7fd5240b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:42.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.530+0000 7fd529800640 1 --2- 172.21.15.32:0/3109306030 >> v2:172.21.15.144:3300/0 conn(0x7fd52406c140 0x7fd5240b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.530+0000 7fd529800640 1 -- 172.21.15.32:0/3109306030 >> v1:172.21.15.144:6789/0 conn(0x7fd52415f9c0 legacy=0x7fd52406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:42.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.530+0000 7fd529800640 1 -- 172.21.15.32:0/3109306030 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd524162f60 con 0x7fd52415c4c0 2024-04-17T09:53:42.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.534+0000 7fd529800640 1 --2- 172.21.15.32:0/3109306030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52415c4c0 0x7fd5240adb40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd52406cb30 tx=0x7fd51801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a767c2b87fdee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:42.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.534+0000 7fd523e00640 1 -- 172.21.15.32:0/3109306030 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5180ab5f0 con 0x7fd52415c4c0 2024-04-17T09:53:42.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.534+0000 7fd523e00640 1 -- 172.21.15.32:0/3109306030 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5180b0070 con 0x7fd52415c4c0 2024-04-17T09:53:42.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.534+0000 7fd523e00640 1 -- 172.21.15.32:0/3109306030 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5180ab9b0 con 0x7fd52415c4c0 2024-04-17T09:53:42.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.534+0000 7fd52c200640 1 -- 172.21.15.32:0/3109306030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52415c4c0 msgr2=0x7fd5240adb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:42.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.534+0000 7fd52c200640 1 --2- 172.21.15.32:0/3109306030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52415c4c0 0x7fd5240adb40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd52406cb30 tx=0x7fd51801c4e0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.534+0000 7fd52c200640 1 -- 172.21.15.32:0/3109306030 shutdown_connections 2024-04-17T09:53:42.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.534+0000 7fd52c200640 1 --2- 172.21.15.32:0/3109306030 >> v2:172.21.15.144:3300/0 conn(0x7fd52406c140 0x7fd5240b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.534+0000 7fd52c200640 1 --2- 172.21.15.32:0/3109306030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52415c4c0 0x7fd5240adb40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.534+0000 7fd52c200640 1 -- 172.21.15.32:0/3109306030 >> 172.21.15.32:0/3109306030 conn(0x7fd5240b2120 msgr2=0x7fd5240b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:42.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.534+0000 7fd52c200640 1 -- 172.21.15.32:0/3109306030 shutdown_connections 2024-04-17T09:53:42.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.534+0000 7fd52c200640 1 -- 172.21.15.32:0/3109306030 wait complete. 2024-04-17T09:53:42.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd52c200640 1 Processor -- start 2024-04-17T09:53:42.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd52c200640 1 -- start start 2024-04-17T09:53:42.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd52c200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd52406c140 0x7fd5240cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:42.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd52c200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd52415c4c0 0x7fd524140de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:42.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd52c200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52415f9c0 0x7fd5241509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:42.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd52c200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd5241626b0 con 0x7fd52415f9c0 2024-04-17T09:53:42.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd52c200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd524162530 con 0x7fd52415c4c0 2024-04-17T09:53:42.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd52c200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd52416b6a0 con 0x7fd52406c140 2024-04-17T09:53:42.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd52a200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52415f9c0 0x7fd5241509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:42.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd52a200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52415f9c0 0x7fd5241509c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38988/0 (socket says 172.21.15.32:38988) 2024-04-17T09:53:42.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd52a200640 1 -- 172.21.15.32:0/2720925818 learned_addr learned my addr 172.21.15.32:0/2720925818 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:42.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd529800640 1 --2- 172.21.15.32:0/2720925818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd52406c140 0x7fd5240cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:42.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd528e00640 1 --2- 172.21.15.32:0/2720925818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd52415c4c0 0x7fd524140de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:42.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd52a200640 1 -- 172.21.15.32:0/2720925818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd52406c140 msgr2=0x7fd5240cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:42.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd52a200640 1 --2- 172.21.15.32:0/2720925818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd52406c140 0x7fd5240cc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd52a200640 1 -- 172.21.15.32:0/2720925818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd52415c4c0 msgr2=0x7fd524140de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:42.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd52a200640 1 --2- 172.21.15.32:0/2720925818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd52415c4c0 0x7fd524140de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd52a200640 1 -- 172.21.15.32:0/2720925818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5240ccf20 con 0x7fd52415f9c0 2024-04-17T09:53:42.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd52a200640 1 --2- 172.21.15.32:0/2720925818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52415f9c0 0x7fd5241509c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00b970 tx=0x7fd51c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:42.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd521600640 1 -- 172.21.15.32:0/2720925818 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c019070 con 0x7fd52415f9c0 2024-04-17T09:53:42.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd521600640 1 -- 172.21.15.32:0/2720925818 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd51c00cb60 con 0x7fd52415f9c0 2024-04-17T09:53:42.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd52c200640 1 -- 172.21.15.32:0/2720925818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd524141320 con 0x7fd52415f9c0 2024-04-17T09:53:42.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd521600640 1 -- 172.21.15.32:0/2720925818 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c014440 con 0x7fd52415f9c0 2024-04-17T09:53:42.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd52c200640 1 -- 172.21.15.32:0/2720925818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5240cd4a0 con 0x7fd52415f9c0 2024-04-17T09:53:42.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.538+0000 7fd52c200640 1 -- 172.21.15.32:0/2720925818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd524068a90 con 0x7fd52415f9c0 2024-04-17T09:53:42.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.542+0000 7fd521600640 1 -- 172.21.15.32:0/2720925818 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd51c01c070 con 0x7fd52415f9c0 2024-04-17T09:53:42.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.542+0000 7fd521600640 1 --2- 172.21.15.32:0/2720925818 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd504068380 0x7fd50406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:42.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.542+0000 7fd521600640 1 -- 172.21.15.32:0/2720925818 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd51c086740 con 0x7fd52415f9c0 2024-04-17T09:53:42.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.542+0000 7fd529800640 1 --2- 172.21.15.32:0/2720925818 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd504068380 0x7fd50406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:42.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.542+0000 7fd529800640 1 --2- 172.21.15.32:0/2720925818 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd504068380 0x7fd50406a840 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd5180af000 tx=0x7fd5180a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:42.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.546+0000 7fd521600640 1 -- 172.21.15.32:0/2720925818 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd51c056610 con 0x7fd52415f9c0 2024-04-17T09:53:42.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.710+0000 7fd52c200640 1 -- 172.21.15.32:0/2720925818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd524162080 con 0x7fd52415f9c0 2024-04-17T09:53:42.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.710+0000 7fd521600640 1 -- 172.21.15.32:0/2720925818 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fd51c05a910 con 0x7fd52415f9c0 2024-04-17T09:53:42.712 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:53:42.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.714+0000 7fd52c200640 1 -- 172.21.15.32:0/2720925818 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd504068380 msgr2=0x7fd50406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:42.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.714+0000 7fd52c200640 1 --2- 172.21.15.32:0/2720925818 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd504068380 0x7fd50406a840 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd5180af000 tx=0x7fd5180a7000 comp rx=0 tx=0).stop 2024-04-17T09:53:42.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.714+0000 7fd52c200640 1 -- 172.21.15.32:0/2720925818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52415f9c0 msgr2=0x7fd5241509c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:42.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.714+0000 7fd52c200640 1 --2- 172.21.15.32:0/2720925818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52415f9c0 0x7fd5241509c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00b970 tx=0x7fd51c00be30 comp rx=0 tx=0).stop 2024-04-17T09:53:42.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.714+0000 7fd52c200640 1 -- 172.21.15.32:0/2720925818 shutdown_connections 2024-04-17T09:53:42.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.714+0000 7fd52c200640 1 --2- 172.21.15.32:0/2720925818 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd504068380 0x7fd50406a840 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.714+0000 7fd52c200640 1 --2- 172.21.15.32:0/2720925818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52415f9c0 0x7fd5241509c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.714+0000 7fd52c200640 1 --2- 172.21.15.32:0/2720925818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd52415c4c0 0x7fd524140de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.714+0000 7fd52c200640 1 --2- 172.21.15.32:0/2720925818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd52406c140 0x7fd5240cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.714+0000 7fd52c200640 1 -- 172.21.15.32:0/2720925818 >> 172.21.15.32:0/2720925818 conn(0x7fd5240b2120 msgr2=0x7fd5240b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:42.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.714+0000 7fd52c200640 1 -- 172.21.15.32:0/2720925818 shutdown_connections 2024-04-17T09:53:42.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.714+0000 7fd52c200640 1 -- 172.21.15.32:0/2720925818 wait complete. 2024-04-17T09:53:42.734 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:53:42.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.886+0000 7f65a4000640 1 Processor -- start 2024-04-17T09:53:42.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.886+0000 7f65a4000640 1 -- start start 2024-04-17T09:53:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.886+0000 7f65a4000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f659c0cf370 0x7f659c0cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.886+0000 7f65a4000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f659c071e60 0x7f659c072270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.886+0000 7f65a4000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f659c0d0130 con 0x7f659c06ba30 2024-04-17T09:53:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.886+0000 7f65a4000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f659c15d090 con 0x7f659c0cf370 2024-04-17T09:53:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.886+0000 7f65a4000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f659c15d370 con 0x7f659c071e60 2024-04-17T09:53:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.886+0000 7f65a0c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f659c0cf370 0x7f659c0cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.886+0000 7f65a1600640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f659c06ba30 0x7f659c06be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44320/0 (socket says 172.21.15.32:44320) 2024-04-17T09:53:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.886+0000 7f65a0c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f659c0cf370 0x7f659c0cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38998/0 (socket says 172.21.15.32:38998) 2024-04-17T09:53:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.886+0000 7f65a1600640 1 -- 172.21.15.32:0/425227093 learned_addr learned my addr 172.21.15.32:0/425227093 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.886+0000 7f65a2000640 1 --2- 172.21.15.32:0/425227093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f659c071e60 0x7f659c072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.886+0000 7f65a0c00640 1 -- 172.21.15.32:0/425227093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f659c071e60 msgr2=0x7f659c072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.886+0000 7f65a0c00640 1 --2- 172.21.15.32:0/425227093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f659c071e60 0x7f659c072270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.886+0000 7f65a0c00640 1 -- 172.21.15.32:0/425227093 >> v1:172.21.15.32:6789/0 conn(0x7f659c06ba30 legacy=0x7f659c06be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.886+0000 7f65a0c00640 1 -- 172.21.15.32:0/425227093 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659c15d650 con 0x7f659c0cf370 2024-04-17T09:53:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.886+0000 7f65a2000640 1 --2- 172.21.15.32:0/425227093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f659c071e60 0x7f659c072270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.886+0000 7f659be00640 1 -- 172.21.15.32:0/425227093 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4215090203 0 0) 0x7f659c0d0130 con 0x7f659c06ba30 2024-04-17T09:53:42.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.890+0000 7f65a0c00640 1 --2- 172.21.15.32:0/425227093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659c0cf370 0x7f659c0cf7e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f65880097c0 tx=0x7f658801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f7b980538bcb96 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:42.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.890+0000 7f659be00640 1 -- 172.21.15.32:0/425227093 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65880aa620 con 0x7f659c0cf370 2024-04-17T09:53:42.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.890+0000 7f659be00640 1 -- 172.21.15.32:0/425227093 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65880af070 con 0x7f659c0cf370 2024-04-17T09:53:42.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.890+0000 7f659be00640 1 -- 172.21.15.32:0/425227093 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65880aa9e0 con 0x7f659c0cf370 2024-04-17T09:53:42.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a4000640 1 -- 172.21.15.32:0/425227093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659c0cf370 msgr2=0x7f659c0cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:42.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a4000640 1 --2- 172.21.15.32:0/425227093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659c0cf370 0x7f659c0cf7e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f65880097c0 tx=0x7f658801c4a0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a4000640 1 -- 172.21.15.32:0/425227093 shutdown_connections 2024-04-17T09:53:42.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a4000640 1 --2- 172.21.15.32:0/425227093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f659c071e60 0x7f659c072270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a4000640 1 --2- 172.21.15.32:0/425227093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659c0cf370 0x7f659c0cf7e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a4000640 1 -- 172.21.15.32:0/425227093 >> 172.21.15.32:0/425227093 conn(0x7f659c0b2120 msgr2=0x7f659c0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:42.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a4000640 1 -- 172.21.15.32:0/425227093 shutdown_connections 2024-04-17T09:53:42.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a4000640 1 -- 172.21.15.32:0/425227093 wait complete. 2024-04-17T09:53:42.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a4000640 1 Processor -- start 2024-04-17T09:53:42.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a4000640 1 -- start start 2024-04-17T09:53:42.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a4000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659c06ba30 0x7f659c159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:42.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a4000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f659c071e60 0x7f659c141f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:42.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a4000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f659c0cf370 0x7f659c142450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:42.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a4000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f659c146230 con 0x7f659c06ba30 2024-04-17T09:53:42.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a4000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f659c15dd80 con 0x7f659c0cf370 2024-04-17T09:53:42.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a4000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f659c1463b0 con 0x7f659c071e60 2024-04-17T09:53:42.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a1600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659c06ba30 0x7f659c159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:42.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a1600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659c06ba30 0x7f659c159860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39008/0 (socket says 172.21.15.32:39008) 2024-04-17T09:53:42.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a1600640 1 -- 172.21.15.32:0/325722251 learned_addr learned my addr 172.21.15.32:0/325722251 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:42.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a2000640 1 --2- 172.21.15.32:0/325722251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f659c0cf370 0x7f659c142450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:42.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a0c00640 1 --2- 172.21.15.32:0/325722251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f659c071e60 0x7f659c141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:42.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a1600640 1 -- 172.21.15.32:0/325722251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f659c071e60 msgr2=0x7f659c141f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:42.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a1600640 1 --2- 172.21.15.32:0/325722251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f659c071e60 0x7f659c141f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a1600640 1 -- 172.21.15.32:0/325722251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f659c0cf370 msgr2=0x7f659c142450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:42.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a1600640 1 --2- 172.21.15.32:0/325722251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f659c0cf370 0x7f659c142450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:42.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.894+0000 7f65a1600640 1 -- 172.21.15.32:0/325722251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659c0ade20 con 0x7f659c06ba30 2024-04-17T09:53:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.898+0000 7f65a1600640 1 --2- 172.21.15.32:0/325722251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659c06ba30 0x7f659c159860 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f659000c550 tx=0x7f659000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.898+0000 7f6599600640 1 -- 172.21.15.32:0/325722251 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590017070 con 0x7f659c06ba30 2024-04-17T09:53:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.898+0000 7f6599600640 1 -- 172.21.15.32:0/325722251 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6590005d10 con 0x7f659c06ba30 2024-04-17T09:53:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.898+0000 7f6599600640 1 -- 172.21.15.32:0/325722251 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590007330 con 0x7f659c06ba30 2024-04-17T09:53:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.898+0000 7f65a4000640 1 -- 172.21.15.32:0/325722251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659c0ae100 con 0x7f659c06ba30 2024-04-17T09:53:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.898+0000 7f65a4000640 1 -- 172.21.15.32:0/325722251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f659c0ae6c0 con 0x7f659c06ba30 2024-04-17T09:53:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.898+0000 7f65a4000640 1 -- 172.21.15.32:0/325722251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f659c068a90 con 0x7f659c06ba30 2024-04-17T09:53:42.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.898+0000 7f6599600640 1 -- 172.21.15.32:0/325722251 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f659001a070 con 0x7f659c06ba30 2024-04-17T09:53:42.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.898+0000 7f6599600640 1 --2- 172.21.15.32:0/325722251 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f657c068380 0x7f657c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:42.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.898+0000 7f6599600640 1 -- 172.21.15.32:0/325722251 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6590085d90 con 0x7f659c06ba30 2024-04-17T09:53:42.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.898+0000 7f65a0c00640 1 --2- 172.21.15.32:0/325722251 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f657c068380 0x7f657c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:42.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.902+0000 7f65a0c00640 1 --2- 172.21.15.32:0/325722251 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f657c068380 0x7f657c06a840 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f65880ae000 tx=0x7f65880b5040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:42.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:42.902+0000 7f6599600640 1 -- 172.21.15.32:0/325722251 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f6590055cf0 con 0x7f659c06ba30 2024-04-17T09:53:43.012 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:53:43.017 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:53:43.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.070+0000 7f65a4000640 1 -- 172.21.15.32:0/325722251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f659c06c850 con 0x7f659c06ba30 2024-04-17T09:53:43.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.070+0000 7f6599600640 1 -- 172.21.15.32:0/325722251 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f6590059ff0 con 0x7f659c06ba30 2024-04-17T09:53:43.071 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:53:43.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.074+0000 7f65a4000640 1 -- 172.21.15.32:0/325722251 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f657c068380 msgr2=0x7f657c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:43.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.074+0000 7f65a4000640 1 --2- 172.21.15.32:0/325722251 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f657c068380 0x7f657c06a840 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f65880ae000 tx=0x7f65880b5040 comp rx=0 tx=0).stop 2024-04-17T09:53:43.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.074+0000 7f65a4000640 1 -- 172.21.15.32:0/325722251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659c06ba30 msgr2=0x7f659c159860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:43.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.074+0000 7f65a4000640 1 --2- 172.21.15.32:0/325722251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659c06ba30 0x7f659c159860 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f659000c550 tx=0x7f659000ca10 comp rx=0 tx=0).stop 2024-04-17T09:53:43.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.074+0000 7f65a4000640 1 -- 172.21.15.32:0/325722251 shutdown_connections 2024-04-17T09:53:43.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.074+0000 7f65a4000640 1 --2- 172.21.15.32:0/325722251 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f657c068380 0x7f657c06a840 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.074+0000 7f65a4000640 1 --2- 172.21.15.32:0/325722251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f659c071e60 0x7f659c141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.074+0000 7f65a4000640 1 --2- 172.21.15.32:0/325722251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f659c0cf370 0x7f659c142450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.074+0000 7f65a4000640 1 --2- 172.21.15.32:0/325722251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659c06ba30 0x7f659c159860 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.074+0000 7f65a4000640 1 -- 172.21.15.32:0/325722251 >> 172.21.15.32:0/325722251 conn(0x7f659c0b2120 msgr2=0x7f659c06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:43.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.074+0000 7f65a4000640 1 -- 172.21.15.32:0/325722251 shutdown_connections 2024-04-17T09:53:43.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.074+0000 7f65a4000640 1 -- 172.21.15.32:0/325722251 wait complete. 2024-04-17T09:53:43.092 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:53:43.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.282+0000 7fb8e6e00640 1 Processor -- start 2024-04-17T09:53:43.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.282+0000 7fb8e6e00640 1 -- start start 2024-04-17T09:53:43.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.286+0000 7fb8e6e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8e006f7c0 0x7fb8e006ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:43.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.286+0000 7fb8e6e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8e006f270 0x7fb8e006bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:43.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.286+0000 7fb8e6e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8e00cc940 con 0x7fb8e015b7d0 2024-04-17T09:53:43.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.286+0000 7fb8e6e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb8e00ccc20 con 0x7fb8e006f270 2024-04-17T09:53:43.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.286+0000 7fb8e6e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e006e0b0 con 0x7fb8e006f7c0 2024-04-17T09:53:43.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.286+0000 7fb8e4e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8e006f270 0x7fb8e006bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:43.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.286+0000 7fb8e4e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8e006f270 0x7fb8e006bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39014/0 (socket says 172.21.15.32:39014) 2024-04-17T09:53:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.286+0000 7fb8e4e00640 1 -- 172.21.15.32:0/1679709682 learned_addr learned my addr 172.21.15.32:0/1679709682 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.286+0000 7fb8df400640 1 --2- 172.21.15.32:0/1679709682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8e006f7c0 0x7fb8e006ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.286+0000 7fb8e4e00640 1 -- 172.21.15.32:0/1679709682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8e006f7c0 msgr2=0x7fb8e006ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.286+0000 7fb8e4e00640 1 --2- 172.21.15.32:0/1679709682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8e006f7c0 0x7fb8e006ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.286+0000 7fb8e4e00640 1 -- 172.21.15.32:0/1679709682 >> v1:172.21.15.144:6789/0 conn(0x7fb8e015b7d0 legacy=0x7fb8e006e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.286+0000 7fb8e4e00640 1 -- 172.21.15.32:0/1679709682 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e006e390 con 0x7fb8e006f270 2024-04-17T09:53:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.286+0000 7fb8df400640 1 --2- 172.21.15.32:0/1679709682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8e006f7c0 0x7fb8e006ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.286+0000 7fb8e4e00640 1 --2- 172.21.15.32:0/1679709682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8e006f270 0x7fb8e006bf70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800d150 tx=0x7fb8d801eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6491a3089888295c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.286+0000 7fb8dea00640 1 -- 172.21.15.32:0/1679709682 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d800b2d0 con 0x7fb8e006f270 2024-04-17T09:53:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.286+0000 7fb8dea00640 1 -- 172.21.15.32:0/1679709682 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8d800b470 con 0x7fb8e006f270 2024-04-17T09:53:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.286+0000 7fb8dea00640 1 -- 172.21.15.32:0/1679709682 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d80a7ba0 con 0x7fb8e006f270 2024-04-17T09:53:43.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.290+0000 7fb8e6e00640 1 -- 172.21.15.32:0/1679709682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8e006f270 msgr2=0x7fb8e006bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:43.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.290+0000 7fb8e6e00640 1 --2- 172.21.15.32:0/1679709682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8e006f270 0x7fb8e006bf70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800d150 tx=0x7fb8d801eba0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.290+0000 7fb8e6e00640 1 -- 172.21.15.32:0/1679709682 shutdown_connections 2024-04-17T09:53:43.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.290+0000 7fb8e6e00640 1 --2- 172.21.15.32:0/1679709682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8e006f270 0x7fb8e006bf70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.290+0000 7fb8e6e00640 1 --2- 172.21.15.32:0/1679709682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8e006f7c0 0x7fb8e006ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.290+0000 7fb8e6e00640 1 -- 172.21.15.32:0/1679709682 >> 172.21.15.32:0/1679709682 conn(0x7fb8e00b2120 msgr2=0x7fb8e00b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:43.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.290+0000 7fb8e6e00640 1 -- 172.21.15.32:0/1679709682 shutdown_connections 2024-04-17T09:53:43.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.290+0000 7fb8e6e00640 1 -- 172.21.15.32:0/1679709682 wait complete. 2024-04-17T09:53:43.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.290+0000 7fb8e6e00640 1 Processor -- start 2024-04-17T09:53:43.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.290+0000 7fb8e6e00640 1 -- start start 2024-04-17T09:53:43.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.290+0000 7fb8e6e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8e006f270 0x7fb8e015bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:43.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.290+0000 7fb8e6e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8e006f7c0 0x7fb8e0152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:43.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.290+0000 7fb8e6e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8e015b7d0 0x7fb8e013bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:43.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.290+0000 7fb8e6e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e00cee70 con 0x7fb8e006f270 2024-04-17T09:53:43.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.290+0000 7fb8e6e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e00cecf0 con 0x7fb8e006f7c0 2024-04-17T09:53:43.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.290+0000 7fb8e6e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e0071e60 con 0x7fb8e015b7d0 2024-04-17T09:53:43.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.290+0000 7fb8dfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8e006f270 0x7fb8e015bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:43.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.290+0000 7fb8dfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8e006f270 0x7fb8e015bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39016/0 (socket says 172.21.15.32:39016) 2024-04-17T09:53:43.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.290+0000 7fb8dfe00640 1 -- 172.21.15.32:0/4044253510 learned_addr learned my addr 172.21.15.32:0/4044253510 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:43.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.290+0000 7fb8df400640 1 --2- 172.21.15.32:0/4044253510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8e006f7c0 0x7fb8e0152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:43.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.290+0000 7fb8e4e00640 1 --2- 172.21.15.32:0/4044253510 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8e015b7d0 0x7fb8e013bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:43.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.294+0000 7fb8dfe00640 1 -- 172.21.15.32:0/4044253510 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8e015b7d0 msgr2=0x7fb8e013bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:43.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.294+0000 7fb8dfe00640 1 --2- 172.21.15.32:0/4044253510 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8e015b7d0 0x7fb8e013bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.294+0000 7fb8dfe00640 1 -- 172.21.15.32:0/4044253510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8e006f7c0 msgr2=0x7fb8e0152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:43.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.294+0000 7fb8dfe00640 1 --2- 172.21.15.32:0/4044253510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8e006f7c0 0x7fb8e0152110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.294+0000 7fb8dfe00640 1 -- 172.21.15.32:0/4044253510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e0150c80 con 0x7fb8e006f270 2024-04-17T09:53:43.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.294+0000 7fb8df400640 1 --2- 172.21.15.32:0/4044253510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8e006f7c0 0x7fb8e0152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:43.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.294+0000 7fb8dfe00640 1 --2- 172.21.15.32:0/4044253510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8e006f270 0x7fb8e015bea0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000c3a0 tx=0x7fb8d000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:43.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.294+0000 7fb8d7e00640 1 -- 172.21.15.32:0/4044253510 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d00052d0 con 0x7fb8e006f270 2024-04-17T09:53:43.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.294+0000 7fb8d7e00640 1 -- 172.21.15.32:0/4044253510 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8d0005470 con 0x7fb8e006f270 2024-04-17T09:53:43.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.294+0000 7fb8e6e00640 1 -- 172.21.15.32:0/4044253510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e0150f60 con 0x7fb8e006f270 2024-04-17T09:53:43.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.294+0000 7fb8d7e00640 1 -- 172.21.15.32:0/4044253510 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0007260 con 0x7fb8e006f270 2024-04-17T09:53:43.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.294+0000 7fb8e6e00640 1 -- 172.21.15.32:0/4044253510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e015d840 con 0x7fb8e006f270 2024-04-17T09:53:43.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.294+0000 7fb8e6e00640 1 -- 172.21.15.32:0/4044253510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e0068a90 con 0x7fb8e006f270 2024-04-17T09:53:43.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.294+0000 7fb8d7e00640 1 -- 172.21.15.32:0/4044253510 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb8d0007400 con 0x7fb8e006f270 2024-04-17T09:53:43.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.294+0000 7fb8d7e00640 1 --2- 172.21.15.32:0/4044253510 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb8c0068380 0x7fb8c006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:43.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.298+0000 7fb8d7e00640 1 -- 172.21.15.32:0/4044253510 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb8d0085dd0 con 0x7fb8e006f270 2024-04-17T09:53:43.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.298+0000 7fb8df400640 1 --2- 172.21.15.32:0/4044253510 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb8c0068380 0x7fb8c006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:43.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.298+0000 7fb8df400640 1 --2- 172.21.15.32:0/4044253510 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb8c0068380 0x7fb8c006a840 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb8e01514a0 tx=0x7fb8cc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:43.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.302+0000 7fb8d7e00640 1 -- 172.21.15.32:0/4044253510 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fb8d0055d30 con 0x7fb8e006f270 2024-04-17T09:53:43.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.466+0000 7fb8e6e00640 1 -- 172.21.15.32:0/4044253510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb8e0156450 con 0x7fb8e006f270 2024-04-17T09:53:43.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.466+0000 7fb8d7e00640 1 -- 172.21.15.32:0/4044253510 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fb8d005a030 con 0x7fb8e006f270 2024-04-17T09:53:43.468 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:53:43.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.470+0000 7fb8e6e00640 1 -- 172.21.15.32:0/4044253510 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb8c0068380 msgr2=0x7fb8c006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:43.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.470+0000 7fb8e6e00640 1 --2- 172.21.15.32:0/4044253510 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb8c0068380 0x7fb8c006a840 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb8e01514a0 tx=0x7fb8cc008040 comp rx=0 tx=0).stop 2024-04-17T09:53:43.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.470+0000 7fb8e6e00640 1 -- 172.21.15.32:0/4044253510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8e006f270 msgr2=0x7fb8e015bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:43.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.470+0000 7fb8e6e00640 1 --2- 172.21.15.32:0/4044253510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8e006f270 0x7fb8e015bea0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000c3a0 tx=0x7fb8d000c860 comp rx=0 tx=0).stop 2024-04-17T09:53:43.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.470+0000 7fb8e6e00640 1 -- 172.21.15.32:0/4044253510 shutdown_connections 2024-04-17T09:53:43.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.470+0000 7fb8e6e00640 1 --2- 172.21.15.32:0/4044253510 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb8c0068380 0x7fb8c006a840 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.470+0000 7fb8e6e00640 1 --2- 172.21.15.32:0/4044253510 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8e015b7d0 0x7fb8e013bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.470+0000 7fb8e6e00640 1 --2- 172.21.15.32:0/4044253510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8e006f7c0 0x7fb8e0152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.470+0000 7fb8e6e00640 1 --2- 172.21.15.32:0/4044253510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8e006f270 0x7fb8e015bea0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.470+0000 7fb8e6e00640 1 -- 172.21.15.32:0/4044253510 >> 172.21.15.32:0/4044253510 conn(0x7fb8e00b2120 msgr2=0x7fb8e015aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:43.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.470+0000 7fb8e6e00640 1 -- 172.21.15.32:0/4044253510 shutdown_connections 2024-04-17T09:53:43.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.470+0000 7fb8e6e00640 1 -- 172.21.15.32:0/4044253510 wait complete. 2024-04-17T09:53:43.490 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:53:43.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.630+0000 7f0690600640 1 Processor -- start 2024-04-17T09:53:43.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.630+0000 7f0690600640 1 -- start start 2024-04-17T09:53:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.630+0000 7f0690600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06880d0890 0x7f068809a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.630+0000 7f0690600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f068809af30 0x7f06880ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.630+0000 7f0690600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f068815ef10 con 0x7f068815eb30 2024-04-17T09:53:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.630+0000 7f0690600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f068806dd60 con 0x7f068809af30 2024-04-17T09:53:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.630+0000 7f0690600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f06880d0db0 con 0x7f06880d0890 2024-04-17T09:53:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.630+0000 7f068e600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f068809af30 0x7f06880ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.630+0000 7f068e600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f068809af30 0x7f06880ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39024/0 (socket says 172.21.15.32:39024) 2024-04-17T09:53:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.630+0000 7f068e600640 1 -- 172.21.15.32:0/3819084373 learned_addr learned my addr 172.21.15.32:0/3819084373 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.630+0000 7f068d200640 1 --2- 172.21.15.32:0/3819084373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06880d0890 0x7f068809a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.630+0000 7f068e600640 1 -- 172.21.15.32:0/3819084373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06880d0890 msgr2=0x7f068809a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.630+0000 7f068e600640 1 --2- 172.21.15.32:0/3819084373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06880d0890 0x7f068809a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.630+0000 7f068e600640 1 -- 172.21.15.32:0/3819084373 >> v1:172.21.15.144:6789/0 conn(0x7f068815eb30 legacy=0x7f068809b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.630+0000 7f068e600640 1 -- 172.21.15.32:0/3819084373 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06880d1090 con 0x7f068809af30 2024-04-17T09:53:43.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.634+0000 7f068e600640 1 --2- 172.21.15.32:0/3819084373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f068809af30 0x7f06880ccb30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0680009e50 tx=0x7f068001f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16110d391ab5ab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:43.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.634+0000 7f0687e00640 1 -- 172.21.15.32:0/3819084373 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06800b4070 con 0x7f068809af30 2024-04-17T09:53:43.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.634+0000 7f0687e00640 1 -- 172.21.15.32:0/3819084373 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06800af400 con 0x7f068809af30 2024-04-17T09:53:43.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.634+0000 7f0687e00640 1 -- 172.21.15.32:0/3819084373 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06800af7d0 con 0x7f068809af30 2024-04-17T09:53:43.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.634+0000 7f0690600640 1 -- 172.21.15.32:0/3819084373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f068809af30 msgr2=0x7f06880ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:43.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.634+0000 7f0690600640 1 --2- 172.21.15.32:0/3819084373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f068809af30 0x7f06880ccb30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0680009e50 tx=0x7f068001f210 comp rx=0 tx=0).stop 2024-04-17T09:53:43.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.634+0000 7f0690600640 1 -- 172.21.15.32:0/3819084373 shutdown_connections 2024-04-17T09:53:43.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.634+0000 7f0690600640 1 --2- 172.21.15.32:0/3819084373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f068809af30 0x7f06880ccb30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.634+0000 7f0690600640 1 --2- 172.21.15.32:0/3819084373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06880d0890 0x7f068809a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.634+0000 7f0690600640 1 -- 172.21.15.32:0/3819084373 >> 172.21.15.32:0/3819084373 conn(0x7f06880b3fd0 msgr2=0x7f06880b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:43.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.634+0000 7f0690600640 1 -- 172.21.15.32:0/3819084373 shutdown_connections 2024-04-17T09:53:43.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.634+0000 7f0690600640 1 -- 172.21.15.32:0/3819084373 wait complete. 2024-04-17T09:53:43.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f0690600640 1 Processor -- start 2024-04-17T09:53:43.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f0690600640 1 -- start start 2024-04-17T09:53:43.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f0690600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f068809af30 0x7f068815ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:43.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f0690600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06880d0890 0x7f068815b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:43.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f0690600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f068815eb30 0x7f06880cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:43.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f0690600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f06880d1930 con 0x7f06880d0890 2024-04-17T09:53:43.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f0690600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f06880d1650 con 0x7f068809af30 2024-04-17T09:53:43.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f0690600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f06880cd070 con 0x7f068815eb30 2024-04-17T09:53:43.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f068d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06880d0890 0x7f068815b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:43.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f068d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06880d0890 0x7f068815b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39026/0 (socket says 172.21.15.32:39026) 2024-04-17T09:53:43.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f068d200640 1 -- 172.21.15.32:0/3424438063 learned_addr learned my addr 172.21.15.32:0/3424438063 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:43.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f068dc00640 1 --2- 172.21.15.32:0/3424438063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f068809af30 0x7f068815ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:43.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f068e600640 1 --2- 172.21.15.32:0/3424438063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f068815eb30 0x7f06880cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:43.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f068d200640 1 -- 172.21.15.32:0/3424438063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f068815eb30 msgr2=0x7f06880cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:43.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f068d200640 1 --2- 172.21.15.32:0/3424438063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f068815eb30 0x7f06880cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f068d200640 1 -- 172.21.15.32:0/3424438063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f068809af30 msgr2=0x7f068815ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:43.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f068d200640 1 --2- 172.21.15.32:0/3424438063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f068809af30 0x7f068815ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f068d200640 1 -- 172.21.15.32:0/3424438063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06880c9a20 con 0x7f06880d0890 2024-04-17T09:53:43.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f068d200640 1 --2- 172.21.15.32:0/3424438063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06880d0890 0x7f068815b380 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f067400e930 tx=0x7f067400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:43.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f0685600640 1 -- 172.21.15.32:0/3424438063 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0674017070 con 0x7f06880d0890 2024-04-17T09:53:43.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f0685600640 1 -- 172.21.15.32:0/3424438063 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f067400ca00 con 0x7f06880d0890 2024-04-17T09:53:43.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f0690600640 1 -- 172.21.15.32:0/3424438063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06880c9d00 con 0x7f06880d0890 2024-04-17T09:53:43.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f0685600640 1 -- 172.21.15.32:0/3424438063 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067400cdb0 con 0x7f06880d0890 2024-04-17T09:53:43.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.638+0000 7f0690600640 1 -- 172.21.15.32:0/3424438063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06880ca2c0 con 0x7f06880d0890 2024-04-17T09:53:43.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.642+0000 7f0690600640 1 -- 172.21.15.32:0/3424438063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0688068a90 con 0x7f06880d0890 2024-04-17T09:53:43.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.642+0000 7f0685600640 1 -- 172.21.15.32:0/3424438063 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f067401a070 con 0x7f06880d0890 2024-04-17T09:53:43.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.642+0000 7f0685600640 1 --2- 172.21.15.32:0/3424438063 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0668068380 0x7f066806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:43.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.642+0000 7f0685600640 1 -- 172.21.15.32:0/3424438063 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0674086cf0 con 0x7f06880d0890 2024-04-17T09:53:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.642+0000 7f068dc00640 1 --2- 172.21.15.32:0/3424438063 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0668068380 0x7f066806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.642+0000 7f068dc00640 1 --2- 172.21.15.32:0/3424438063 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0668068380 0x7f066806a840 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f067c001770 tx=0x7f067c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:43.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.646+0000 7f0685600640 1 -- 172.21.15.32:0/3424438063 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0674056d10 con 0x7f06880d0890 2024-04-17T09:53:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.810+0000 7f0690600640 1 -- 172.21.15.32:0/3424438063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06880ce9a0 con 0x7f06880d0890 2024-04-17T09:53:43.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.814+0000 7f0685600640 1 -- 172.21.15.32:0/3424438063 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f067405b010 con 0x7f06880d0890 2024-04-17T09:53:43.813 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:43.813 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:14.779535+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:53:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.814+0000 7f0690600640 1 -- 172.21.15.32:0/3424438063 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0668068380 msgr2=0x7f066806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.814+0000 7f0690600640 1 --2- 172.21.15.32:0/3424438063 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0668068380 0x7f066806a840 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f067c001770 tx=0x7f067c009040 comp rx=0 tx=0).stop 2024-04-17T09:53:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.814+0000 7f0690600640 1 -- 172.21.15.32:0/3424438063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06880d0890 msgr2=0x7f068815b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.814+0000 7f0690600640 1 --2- 172.21.15.32:0/3424438063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06880d0890 0x7f068815b380 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f067400e930 tx=0x7f067400edf0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.814+0000 7f0690600640 1 -- 172.21.15.32:0/3424438063 shutdown_connections 2024-04-17T09:53:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.814+0000 7f0690600640 1 --2- 172.21.15.32:0/3424438063 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0668068380 0x7f066806a840 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.814+0000 7f0690600640 1 --2- 172.21.15.32:0/3424438063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f068815eb30 0x7f06880cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.814+0000 7f0690600640 1 --2- 172.21.15.32:0/3424438063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06880d0890 0x7f068815b380 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.814+0000 7f0690600640 1 --2- 172.21.15.32:0/3424438063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f068809af30 0x7f068815ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.814+0000 7f0690600640 1 -- 172.21.15.32:0/3424438063 >> 172.21.15.32:0/3424438063 conn(0x7f06880b3fd0 msgr2=0x7f068815f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.814+0000 7f0690600640 1 -- 172.21.15.32:0/3424438063 shutdown_connections 2024-04-17T09:53:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.814+0000 7f0690600640 1 -- 172.21.15.32:0/3424438063 wait complete. 2024-04-17T09:53:43.833 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:53:11.198098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:53:12.118982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:53:43.834 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:53:43.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.978+0000 7f0820e00640 1 Processor -- start 2024-04-17T09:53:43.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.978+0000 7f0820e00640 1 -- start start 2024-04-17T09:53:43.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.978+0000 7f0820e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f081c0ce740 0x7f081c09a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:43.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.978+0000 7f0820e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f081c15cc40 con 0x7f081c15c860 2024-04-17T09:53:43.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.978+0000 7f0820e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f081c0cd6a0 con 0x7f081c0ceba0 2024-04-17T09:53:43.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.978+0000 7f0820e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f081c0ce2b0 con 0x7f081c0ce740 2024-04-17T09:53:43.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.978+0000 7f0811400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f081c0ce740 0x7f081c09a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:43.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.978+0000 7f0819e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f081c15c860 0x7f081c0d1050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44328/0 (socket says 172.21.15.32:44328) 2024-04-17T09:53:43.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.978+0000 7f0819e00640 1 -- 172.21.15.32:0/2272571026 learned_addr learned my addr 172.21.15.32:0/2272571026 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:43.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.978+0000 7f0811400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f081c0ce740 0x7f081c09a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39034/0 (socket says 172.21.15.32:39034) 2024-04-17T09:53:43.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.978+0000 7f0811400640 1 -- 172.21.15.32:0/2272571026 >> v1:172.21.15.144:6789/0 conn(0x7f081c0ceba0 legacy=0x7f081c0b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:43.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.978+0000 7f0811400640 1 -- 172.21.15.32:0/2272571026 >> v1:172.21.15.32:6789/0 conn(0x7f081c15c860 legacy=0x7f081c0d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:43.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.978+0000 7f0811400640 1 -- 172.21.15.32:0/2272571026 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081c0cd980 con 0x7f081c0ce740 2024-04-17T09:53:43.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.978+0000 7f0819400640 1 -- 172.21.15.32:0/2272571026 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2758706429 0 0) 0x7f081c15cc40 con 0x7f081c15c860 2024-04-17T09:53:43.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.982+0000 7f0811400640 1 --2- 172.21.15.32:0/2272571026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f081c0ce740 0x7f081c09a9f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0804009d90 tx=0x7f080401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb03d80306bed4b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:43.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.982+0000 7f0819400640 1 -- 172.21.15.32:0/2272571026 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08040b2070 con 0x7f081c0ce740 2024-04-17T09:53:43.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.982+0000 7f0819400640 1 -- 172.21.15.32:0/2272571026 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08040a57f0 con 0x7f081c0ce740 2024-04-17T09:53:43.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.982+0000 7f0819400640 1 -- 172.21.15.32:0/2272571026 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08040a5bc0 con 0x7f081c0ce740 2024-04-17T09:53:43.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.982+0000 7f0820e00640 1 -- 172.21.15.32:0/2272571026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f081c0ce740 msgr2=0x7f081c09a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:43.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.982+0000 7f0820e00640 1 --2- 172.21.15.32:0/2272571026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f081c0ce740 0x7f081c09a9f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0804009d90 tx=0x7f080401c930 comp rx=0 tx=0).stop 2024-04-17T09:53:43.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.982+0000 7f0820e00640 1 -- 172.21.15.32:0/2272571026 shutdown_connections 2024-04-17T09:53:43.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.982+0000 7f0820e00640 1 --2- 172.21.15.32:0/2272571026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f081c0ce740 0x7f081c09a9f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.982+0000 7f0820e00640 1 -- 172.21.15.32:0/2272571026 >> 172.21.15.32:0/2272571026 conn(0x7f081c0b4050 msgr2=0x7f081c0b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:43.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.982+0000 7f0820e00640 1 -- 172.21.15.32:0/2272571026 shutdown_connections 2024-04-17T09:53:43.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.982+0000 7f0820e00640 1 -- 172.21.15.32:0/2272571026 wait complete. 2024-04-17T09:53:43.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f0820e00640 1 Processor -- start 2024-04-17T09:53:43.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f0820e00640 1 -- start start 2024-04-17T09:53:43.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f0820e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f081c0ce740 0x7f081c06e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:43.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f0820e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f081c0ceba0 0x7f081c15f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f0820e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f081c15c860 0x7f081c06cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f0820e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f081c0cd350 con 0x7f081c0ce740 2024-04-17T09:53:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f0820e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f081c09b340 con 0x7f081c0ceba0 2024-04-17T09:53:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f0820e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f081c16af40 con 0x7f081c15c860 2024-04-17T09:53:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f0819e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f081c0ce740 0x7f081c06e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f0819e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f081c0ce740 0x7f081c06e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39048/0 (socket says 172.21.15.32:39048) 2024-04-17T09:53:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f0819e00640 1 -- 172.21.15.32:0/4247180216 learned_addr learned my addr 172.21.15.32:0/4247180216 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f0811400640 1 --2- 172.21.15.32:0/4247180216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f081c0ceba0 0x7f081c15f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f081a800640 1 --2- 172.21.15.32:0/4247180216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f081c15c860 0x7f081c06cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f0819e00640 1 -- 172.21.15.32:0/4247180216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f081c15c860 msgr2=0x7f081c06cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f0819e00640 1 --2- 172.21.15.32:0/4247180216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f081c15c860 0x7f081c06cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f0819e00640 1 -- 172.21.15.32:0/4247180216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f081c0ceba0 msgr2=0x7f081c15f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f0819e00640 1 --2- 172.21.15.32:0/4247180216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f081c0ceba0 0x7f081c15f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f0819e00640 1 -- 172.21.15.32:0/4247180216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081c158b30 con 0x7f081c0ce740 2024-04-17T09:53:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f0819e00640 1 --2- 172.21.15.32:0/4247180216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f081c0ce740 0x7f081c06e550 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f080c0011f0 tx=0x7f080c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:43.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f0812a00640 1 -- 172.21.15.32:0/4247180216 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c018070 con 0x7f081c0ce740 2024-04-17T09:53:43.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f0812a00640 1 -- 172.21.15.32:0/4247180216 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f080c0074e0 con 0x7f081c0ce740 2024-04-17T09:53:43.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f0812a00640 1 -- 172.21.15.32:0/4247180216 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c004a10 con 0x7f081c0ce740 2024-04-17T09:53:43.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f0820e00640 1 -- 172.21.15.32:0/4247180216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f081c158e10 con 0x7f081c0ce740 2024-04-17T09:53:43.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.986+0000 7f0820e00640 1 -- 172.21.15.32:0/4247180216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f081c1593d0 con 0x7f081c0ce740 2024-04-17T09:53:43.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.990+0000 7f0820e00640 1 -- 172.21.15.32:0/4247180216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f081c068a90 con 0x7f081c0ce740 2024-04-17T09:53:43.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.990+0000 7f0812a00640 1 -- 172.21.15.32:0/4247180216 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f080c004bb0 con 0x7f081c0ce740 2024-04-17T09:53:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.990+0000 7f0812a00640 1 --2- 172.21.15.32:0/4247180216 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07f80683b0 0x7f07f806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.990+0000 7f0812a00640 1 -- 172.21.15.32:0/4247180216 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f080c085c30 con 0x7f081c0ce740 2024-04-17T09:53:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.990+0000 7f0811400640 1 --2- 172.21.15.32:0/4247180216 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07f80683b0 0x7f07f806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.990+0000 7f0811400640 1 --2- 172.21.15.32:0/4247180216 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07f80683b0 0x7f07f806a870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f08040af000 tx=0x7f08040a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:43.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:43.994+0000 7f0812a00640 1 -- 172.21.15.32:0/4247180216 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f080c0561e0 con 0x7f081c0ce740 2024-04-17T09:53:44.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.158+0000 7f0820e00640 1 -- 172.21.15.32:0/4247180216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f081c168500 con 0x7f081c0ce740 2024-04-17T09:53:44.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.158+0000 7f0812a00640 1 -- 172.21.15.32:0/4247180216 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f080c055ae0 con 0x7f081c0ce740 2024-04-17T09:53:44.160 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:53:44.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.162+0000 7f0820e00640 1 -- 172.21.15.32:0/4247180216 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07f80683b0 msgr2=0x7f07f806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:44.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.162+0000 7f0820e00640 1 --2- 172.21.15.32:0/4247180216 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07f80683b0 0x7f07f806a870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f08040af000 tx=0x7f08040a7070 comp rx=0 tx=0).stop 2024-04-17T09:53:44.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.162+0000 7f0820e00640 1 -- 172.21.15.32:0/4247180216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f081c0ce740 msgr2=0x7f081c06e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:44.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.162+0000 7f0820e00640 1 --2- 172.21.15.32:0/4247180216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f081c0ce740 0x7f081c06e550 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f080c0011f0 tx=0x7f080c00c9b0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.162+0000 7f0820e00640 1 -- 172.21.15.32:0/4247180216 shutdown_connections 2024-04-17T09:53:44.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.162+0000 7f0820e00640 1 --2- 172.21.15.32:0/4247180216 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07f80683b0 0x7f07f806a870 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.162+0000 7f0820e00640 1 --2- 172.21.15.32:0/4247180216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f081c15c860 0x7f081c06cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.162+0000 7f0820e00640 1 --2- 172.21.15.32:0/4247180216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f081c0ceba0 0x7f081c15f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.162+0000 7f0820e00640 1 --2- 172.21.15.32:0/4247180216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f081c0ce740 0x7f081c06e550 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.162+0000 7f0820e00640 1 -- 172.21.15.32:0/4247180216 >> 172.21.15.32:0/4247180216 conn(0x7f081c0b4050 msgr2=0x7f081c0d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:44.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.162+0000 7f0820e00640 1 -- 172.21.15.32:0/4247180216 shutdown_connections 2024-04-17T09:53:44.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.162+0000 7f0820e00640 1 -- 172.21.15.32:0/4247180216 wait complete. 2024-04-17T09:53:44.182 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:53:44.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.326+0000 7fdb9fc00640 1 Processor -- start 2024-04-17T09:53:44.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.326+0000 7fdb9fc00640 1 -- start start 2024-04-17T09:53:44.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.326+0000 7fdb9fc00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdb9815c750 0x7fdb980ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:44.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.326+0000 7fdb9fc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb980b3a90 0x7fdb980c5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:44.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.326+0000 7fdb9fc00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb980ceab0 0x7fdb980c5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:44.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.326+0000 7fdb9fc00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fdb980cd4d0 con 0x7fdb980ceab0 2024-04-17T09:53:44.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.326+0000 7fdb9fc00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fdb980cd7b0 con 0x7fdb9815c750 2024-04-17T09:53:44.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.326+0000 7fdb9fc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdb980cda90 con 0x7fdb980b3a90 2024-04-17T09:53:44.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.326+0000 7fdb9dc00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb980ceab0 0x7fdb980c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:44.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.326+0000 7fdb9dc00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb980ceab0 0x7fdb980c5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39054/0 (socket says 172.21.15.32:39054) 2024-04-17T09:53:44.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.326+0000 7fdb9dc00640 1 -- 172.21.15.32:0/2264000996 learned_addr learned my addr 172.21.15.32:0/2264000996 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:44.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.326+0000 7fdb97e00640 1 --2- 172.21.15.32:0/2264000996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb980b3a90 0x7fdb980c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:44.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.326+0000 7fdb9d200640 1 --2- 172.21.15.32:0/2264000996 >> v2:172.21.15.144:3300/0 conn(0x7fdb9815c750 0x7fdb980ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:44.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.326+0000 7fdb9dc00640 1 -- 172.21.15.32:0/2264000996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb980b3a90 msgr2=0x7fdb980c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:44.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.326+0000 7fdb9dc00640 1 --2- 172.21.15.32:0/2264000996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb980b3a90 0x7fdb980c5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.326+0000 7fdb9dc00640 1 -- 172.21.15.32:0/2264000996 >> v2:172.21.15.144:3300/0 conn(0x7fdb9815c750 msgr2=0x7fdb980ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:44.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.326+0000 7fdb9dc00640 1 --2- 172.21.15.32:0/2264000996 >> v2:172.21.15.144:3300/0 conn(0x7fdb9815c750 0x7fdb980ce570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.326+0000 7fdb9dc00640 1 -- 172.21.15.32:0/2264000996 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb980cdd70 con 0x7fdb980ceab0 2024-04-17T09:53:44.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.326+0000 7fdb9d200640 1 --2- 172.21.15.32:0/2264000996 >> v2:172.21.15.144:3300/0 conn(0x7fdb9815c750 0x7fdb980ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:44.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.330+0000 7fdb9dc00640 1 --2- 172.21.15.32:0/2264000996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb980ceab0 0x7fdb980c5560 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdb98073e40 tx=0x7fdb9001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ba9c8ff211b72c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:44.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.330+0000 7fdb97400640 1 -- 172.21.15.32:0/2264000996 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb900a57d0 con 0x7fdb980ceab0 2024-04-17T09:53:44.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.330+0000 7fdb97400640 1 -- 172.21.15.32:0/2264000996 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb900a5970 con 0x7fdb980ceab0 2024-04-17T09:53:44.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.330+0000 7fdb97400640 1 -- 172.21.15.32:0/2264000996 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb900a5d40 con 0x7fdb980ceab0 2024-04-17T09:53:44.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.330+0000 7fdb9fc00640 1 -- 172.21.15.32:0/2264000996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb980ceab0 msgr2=0x7fdb980c5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:44.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.330+0000 7fdb9fc00640 1 --2- 172.21.15.32:0/2264000996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb980ceab0 0x7fdb980c5560 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdb98073e40 tx=0x7fdb9001c930 comp rx=0 tx=0).stop 2024-04-17T09:53:44.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.330+0000 7fdb9fc00640 1 -- 172.21.15.32:0/2264000996 shutdown_connections 2024-04-17T09:53:44.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.330+0000 7fdb9fc00640 1 --2- 172.21.15.32:0/2264000996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb980ceab0 0x7fdb980c5560 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.330+0000 7fdb9fc00640 1 --2- 172.21.15.32:0/2264000996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb980b3a90 0x7fdb980c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.330+0000 7fdb9fc00640 1 --2- 172.21.15.32:0/2264000996 >> v2:172.21.15.144:3300/0 conn(0x7fdb9815c750 0x7fdb980ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.330+0000 7fdb9fc00640 1 -- 172.21.15.32:0/2264000996 >> 172.21.15.32:0/2264000996 conn(0x7fdb9809a9f0 msgr2=0x7fdb980b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:44.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.330+0000 7fdb9fc00640 1 -- 172.21.15.32:0/2264000996 shutdown_connections 2024-04-17T09:53:44.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.330+0000 7fdb9fc00640 1 -- 172.21.15.32:0/2264000996 wait complete. 2024-04-17T09:53:44.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb9fc00640 1 Processor -- start 2024-04-17T09:53:44.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb9fc00640 1 -- start start 2024-04-17T09:53:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb9fc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb980b3a90 0x7fdb980d0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb9fc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb980ceab0 0x7fdb98146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb9fc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9815c750 0x7fdb98146a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb9fc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdb98160400 con 0x7fdb9815c750 2024-04-17T09:53:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb9fc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdb980cd270 con 0x7fdb980ceab0 2024-04-17T09:53:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb9fc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdb98160580 con 0x7fdb980b3a90 2024-04-17T09:53:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb9dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9815c750 0x7fdb98146a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb9dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9815c750 0x7fdb98146a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39068/0 (socket says 172.21.15.32:39068) 2024-04-17T09:53:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb9dc00640 1 -- 172.21.15.32:0/1547901045 learned_addr learned my addr 172.21.15.32:0/1547901045 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb97e00640 1 --2- 172.21.15.32:0/1547901045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb980ceab0 0x7fdb98146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb9d200640 1 --2- 172.21.15.32:0/1547901045 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb980b3a90 0x7fdb980d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb9dc00640 1 -- 172.21.15.32:0/1547901045 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb980b3a90 msgr2=0x7fdb980d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb9dc00640 1 --2- 172.21.15.32:0/1547901045 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb980b3a90 0x7fdb980d0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb9dc00640 1 -- 172.21.15.32:0/1547901045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb980ceab0 msgr2=0x7fdb98146520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb9dc00640 1 --2- 172.21.15.32:0/1547901045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb980ceab0 0x7fdb98146520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb9dc00640 1 -- 172.21.15.32:0/1547901045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9815d020 con 0x7fdb9815c750 2024-04-17T09:53:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb9dc00640 1 --2- 172.21.15.32:0/1547901045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9815c750 0x7fdb98146a60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdb9001d1d0 tx=0x7fdb900196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:44.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb94c00640 1 -- 172.21.15.32:0/1547901045 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb900bc020 con 0x7fdb9815c750 2024-04-17T09:53:44.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb94c00640 1 -- 172.21.15.32:0/1547901045 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb900b8670 con 0x7fdb9815c750 2024-04-17T09:53:44.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb94c00640 1 -- 172.21.15.32:0/1547901045 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb900b8a60 con 0x7fdb9815c750 2024-04-17T09:53:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb9fc00640 1 -- 172.21.15.32:0/1547901045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9815d300 con 0x7fdb9815c750 2024-04-17T09:53:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb9fc00640 1 -- 172.21.15.32:0/1547901045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb9815d8c0 con 0x7fdb9815c750 2024-04-17T09:53:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.334+0000 7fdb9fc00640 1 -- 172.21.15.32:0/1547901045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb98068a90 con 0x7fdb9815c750 2024-04-17T09:53:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.338+0000 7fdb94c00640 1 -- 172.21.15.32:0/1547901045 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdb900bf020 con 0x7fdb9815c750 2024-04-17T09:53:44.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.338+0000 7fdb94c00640 1 --2- 172.21.15.32:0/1547901045 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdb78068380 0x7fdb7806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:44.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.338+0000 7fdb94c00640 1 -- 172.21.15.32:0/1547901045 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdb90121980 con 0x7fdb9815c750 2024-04-17T09:53:44.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.338+0000 7fdb9d200640 1 --2- 172.21.15.32:0/1547901045 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdb78068380 0x7fdb7806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:44.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.338+0000 7fdb9d200640 1 --2- 172.21.15.32:0/1547901045 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdb78068380 0x7fdb7806a840 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0045f0 tx=0x7fdb8c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.342+0000 7fdb94c00640 1 -- 172.21.15.32:0/1547901045 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fdb900c6030 con 0x7fdb9815c750 2024-04-17T09:53:44.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.510+0000 7fdb9fc00640 1 -- 172.21.15.32:0/1547901045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdb9815da90 con 0x7fdb9815c750 2024-04-17T09:53:44.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.510+0000 7fdb94c00640 1 -- 172.21.15.32:0/1547901045 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fdb900165e0 con 0x7fdb9815c750 2024-04-17T09:53:44.509 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:53:44.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.510+0000 7fdb9fc00640 1 -- 172.21.15.32:0/1547901045 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdb78068380 msgr2=0x7fdb7806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:44.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.510+0000 7fdb9fc00640 1 --2- 172.21.15.32:0/1547901045 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdb78068380 0x7fdb7806a840 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0045f0 tx=0x7fdb8c008040 comp rx=0 tx=0).stop 2024-04-17T09:53:44.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.510+0000 7fdb9fc00640 1 -- 172.21.15.32:0/1547901045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9815c750 msgr2=0x7fdb98146a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:44.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.510+0000 7fdb9fc00640 1 --2- 172.21.15.32:0/1547901045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9815c750 0x7fdb98146a60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdb9001d1d0 tx=0x7fdb900196d0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.510+0000 7fdb9fc00640 1 -- 172.21.15.32:0/1547901045 shutdown_connections 2024-04-17T09:53:44.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.510+0000 7fdb9fc00640 1 --2- 172.21.15.32:0/1547901045 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdb78068380 0x7fdb7806a840 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.510+0000 7fdb9fc00640 1 --2- 172.21.15.32:0/1547901045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9815c750 0x7fdb98146a60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.510+0000 7fdb9fc00640 1 --2- 172.21.15.32:0/1547901045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb980ceab0 0x7fdb98146520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.510+0000 7fdb9fc00640 1 --2- 172.21.15.32:0/1547901045 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb980b3a90 0x7fdb980d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.510+0000 7fdb9fc00640 1 -- 172.21.15.32:0/1547901045 >> 172.21.15.32:0/1547901045 conn(0x7fdb9809a9f0 msgr2=0x7fdb9806f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:44.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.510+0000 7fdb9fc00640 1 -- 172.21.15.32:0/1547901045 shutdown_connections 2024-04-17T09:53:44.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.510+0000 7fdb9fc00640 1 -- 172.21.15.32:0/1547901045 wait complete. 2024-04-17T09:53:44.530 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:53:44.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.674+0000 7f14a1000640 1 Processor -- start 2024-04-17T09:53:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.674+0000 7f14a1000640 1 -- start start 2024-04-17T09:53:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.674+0000 7f14a1000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f149c06e160 0x7f149c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.674+0000 7f14a1000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f149c0cd8d0 0x7f149c09a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.674+0000 7f14a1000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f149c0cdd90 con 0x7f149c0d1520 2024-04-17T09:53:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.674+0000 7f14a1000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f149c0ce070 con 0x7f149c0cd8d0 2024-04-17T09:53:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.674+0000 7f14a1000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f149c0ce350 con 0x7f149c06e160 2024-04-17T09:53:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.674+0000 7f1492a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f149c0cd8d0 0x7f149c09a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.674+0000 7f1492a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f149c0cd8d0 0x7f149c09a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39074/0 (socket says 172.21.15.32:39074) 2024-04-17T09:53:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.674+0000 7f1492a00640 1 -- 172.21.15.32:0/308439306 learned_addr learned my addr 172.21.15.32:0/308439306 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.674+0000 7f149b400640 1 --2- 172.21.15.32:0/308439306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f149c06e160 0x7f149c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.674+0000 7f1492a00640 1 -- 172.21.15.32:0/308439306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f149c06e160 msgr2=0x7f149c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.674+0000 7f1492a00640 1 --2- 172.21.15.32:0/308439306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f149c06e160 0x7f149c0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.674+0000 7f1492a00640 1 -- 172.21.15.32:0/308439306 >> v1:172.21.15.144:6789/0 conn(0x7f149c0d1520 legacy=0x7f149c15e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.674+0000 7f1492a00640 1 -- 172.21.15.32:0/308439306 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149c0ce630 con 0x7f149c0cd8d0 2024-04-17T09:53:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.678+0000 7f1492a00640 1 --2- 172.21.15.32:0/308439306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f149c0cd8d0 0x7f149c09a9f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1484009870 tx=0x7f148401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9f382b53c6ed53 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:44.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.678+0000 7f149aa00640 1 -- 172.21.15.32:0/308439306 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14840b2070 con 0x7f149c0cd8d0 2024-04-17T09:53:44.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.678+0000 7f149aa00640 1 -- 172.21.15.32:0/308439306 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14840a9070 con 0x7f149c0cd8d0 2024-04-17T09:53:44.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.678+0000 7f149aa00640 1 -- 172.21.15.32:0/308439306 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14840ad880 con 0x7f149c0cd8d0 2024-04-17T09:53:44.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.678+0000 7f14a1000640 1 -- 172.21.15.32:0/308439306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f149c0cd8d0 msgr2=0x7f149c09a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:44.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.678+0000 7f14a1000640 1 --2- 172.21.15.32:0/308439306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f149c0cd8d0 0x7f149c09a9f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1484009870 tx=0x7f148401c560 comp rx=0 tx=0).stop 2024-04-17T09:53:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.678+0000 7f14a1000640 1 -- 172.21.15.32:0/308439306 shutdown_connections 2024-04-17T09:53:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.678+0000 7f14a1000640 1 --2- 172.21.15.32:0/308439306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f149c0cd8d0 0x7f149c09a9f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.678+0000 7f14a1000640 1 --2- 172.21.15.32:0/308439306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f149c06e160 0x7f149c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.678+0000 7f14a1000640 1 -- 172.21.15.32:0/308439306 >> 172.21.15.32:0/308439306 conn(0x7f149c0b4000 msgr2=0x7f149c0b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.678+0000 7f14a1000640 1 -- 172.21.15.32:0/308439306 shutdown_connections 2024-04-17T09:53:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.678+0000 7f14a1000640 1 -- 172.21.15.32:0/308439306 wait complete. 2024-04-17T09:53:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f14a1000640 1 Processor -- start 2024-04-17T09:53:44.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f14a1000640 1 -- start start 2024-04-17T09:53:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f14a1000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f149c06e160 0x7f149c1568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f14a1000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f149c0cd8d0 0x7f149c0b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f14a1000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f149c0d1520 0x7f149c0b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f14a1000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f149c06d210 con 0x7f149c0cd8d0 2024-04-17T09:53:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f14a1000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f149c06d090 con 0x7f149c06e160 2024-04-17T09:53:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f14a1000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f149c06d390 con 0x7f149c0d1520 2024-04-17T09:53:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f1492a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f149c0cd8d0 0x7f149c0b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f1492a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f149c0cd8d0 0x7f149c0b5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39088/0 (socket says 172.21.15.32:39088) 2024-04-17T09:53:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f1492a00640 1 -- 172.21.15.32:0/2931043002 learned_addr learned my addr 172.21.15.32:0/2931043002 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f149b400640 1 --2- 172.21.15.32:0/2931043002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f149c06e160 0x7f149c1568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f149be00640 1 --2- 172.21.15.32:0/2931043002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f149c0d1520 0x7f149c0b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f1492a00640 1 -- 172.21.15.32:0/2931043002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f149c0d1520 msgr2=0x7f149c0b6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f1492a00640 1 --2- 172.21.15.32:0/2931043002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f149c0d1520 0x7f149c0b6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f1492a00640 1 -- 172.21.15.32:0/2931043002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f149c06e160 msgr2=0x7f149c1568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f1492a00640 1 --2- 172.21.15.32:0/2931043002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f149c06e160 0x7f149c1568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f1492a00640 1 -- 172.21.15.32:0/2931043002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149c0b5c50 con 0x7f149c0cd8d0 2024-04-17T09:53:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f1492a00640 1 --2- 172.21.15.32:0/2931043002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f149c0cd8d0 0x7f149c0b5710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f148401a190 tx=0x7f1484019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f1493e00640 1 -- 172.21.15.32:0/2931043002 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14840bb070 con 0x7f149c0cd8d0 2024-04-17T09:53:44.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f1493e00640 1 -- 172.21.15.32:0/2931043002 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14840ad9d0 con 0x7f149c0cd8d0 2024-04-17T09:53:44.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f1493e00640 1 -- 172.21.15.32:0/2931043002 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14840b2040 con 0x7f149c0cd8d0 2024-04-17T09:53:44.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f14a1000640 1 -- 172.21.15.32:0/2931043002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f149c158500 con 0x7f149c0cd8d0 2024-04-17T09:53:44.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f14a1000640 1 -- 172.21.15.32:0/2931043002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f149c0b6b10 con 0x7f149c0cd8d0 2024-04-17T09:53:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.682+0000 7f14a1000640 1 -- 172.21.15.32:0/2931043002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f149c0b6ef0 con 0x7f149c0cd8d0 2024-04-17T09:53:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.686+0000 7f1493e00640 1 -- 172.21.15.32:0/2931043002 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f14840ab070 con 0x7f149c0cd8d0 2024-04-17T09:53:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.686+0000 7f1493e00640 1 --2- 172.21.15.32:0/2931043002 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1478068380 0x7f147806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.686+0000 7f1493e00640 1 -- 172.21.15.32:0/2931043002 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f148411f380 con 0x7f149c0cd8d0 2024-04-17T09:53:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.686+0000 7f149b400640 1 --2- 172.21.15.32:0/2931043002 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1478068380 0x7f147806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:44.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.686+0000 7f149b400640 1 --2- 172.21.15.32:0/2931043002 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1478068380 0x7f147806a840 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f149c0c9540 tx=0x7f148c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:44.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.690+0000 7f1493e00640 1 -- 172.21.15.32:0/2931043002 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f149c0b6ef0 con 0x7f149c0cd8d0 2024-04-17T09:53:44.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.854+0000 7f14a1000640 1 -- 172.21.15.32:0/2931043002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f149c04fda0 con 0x7f149c0cd8d0 2024-04-17T09:53:44.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.854+0000 7f1493e00640 1 -- 172.21.15.32:0/2931043002 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f1484016150 con 0x7f149c0cd8d0 2024-04-17T09:53:44.857 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:53:44.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.858+0000 7f14a1000640 1 -- 172.21.15.32:0/2931043002 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1478068380 msgr2=0x7f147806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:44.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.858+0000 7f14a1000640 1 --2- 172.21.15.32:0/2931043002 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1478068380 0x7f147806a840 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f149c0c9540 tx=0x7f148c009190 comp rx=0 tx=0).stop 2024-04-17T09:53:44.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.858+0000 7f14a1000640 1 -- 172.21.15.32:0/2931043002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f149c0cd8d0 msgr2=0x7f149c0b5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:44.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.858+0000 7f14a1000640 1 --2- 172.21.15.32:0/2931043002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f149c0cd8d0 0x7f149c0b5710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f148401a190 tx=0x7f1484019300 comp rx=0 tx=0).stop 2024-04-17T09:53:44.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.858+0000 7f14a1000640 1 -- 172.21.15.32:0/2931043002 shutdown_connections 2024-04-17T09:53:44.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.858+0000 7f14a1000640 1 --2- 172.21.15.32:0/2931043002 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1478068380 0x7f147806a840 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.858+0000 7f14a1000640 1 --2- 172.21.15.32:0/2931043002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f149c0d1520 0x7f149c0b6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.858+0000 7f14a1000640 1 --2- 172.21.15.32:0/2931043002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f149c0cd8d0 0x7f149c0b5710 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.858+0000 7f14a1000640 1 --2- 172.21.15.32:0/2931043002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f149c06e160 0x7f149c1568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:44.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.858+0000 7f14a1000640 1 -- 172.21.15.32:0/2931043002 >> 172.21.15.32:0/2931043002 conn(0x7f149c0b4000 msgr2=0x7f149c160920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:44.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.858+0000 7f14a1000640 1 -- 172.21.15.32:0/2931043002 shutdown_connections 2024-04-17T09:53:44.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:44.858+0000 7f14a1000640 1 -- 172.21.15.32:0/2931043002 wait complete. 2024-04-17T09:53:44.890 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-17T09:53:44.890 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:53:45.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.030+0000 7f578c600640 1 Processor -- start 2024-04-17T09:53:45.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.030+0000 7f578c600640 1 -- start start 2024-04-17T09:53:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.034+0000 7f578c600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5784145f30 0x7f5784071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.034+0000 7f578c600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f578415d8d0 0x7f5784072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.034+0000 7f578c600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57840cdac0 con 0x7f57840cce70 2024-04-17T09:53:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.034+0000 7f578c600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f578406f400 con 0x7f5784145f30 2024-04-17T09:53:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.034+0000 7f578c600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5784146310 con 0x7f578415d8d0 2024-04-17T09:53:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.034+0000 7f578a600640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f57840cce70 0x7f57840cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44338/0 (socket says 172.21.15.32:44338) 2024-04-17T09:53:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.034+0000 7f578a600640 1 -- 172.21.15.32:0/4191826488 learned_addr learned my addr 172.21.15.32:0/4191826488 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.034+0000 7f5789200640 1 --2- 172.21.15.32:0/4191826488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f578415d8d0 0x7f5784072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.034+0000 7f5789c00640 1 --2- 172.21.15.32:0/4191826488 >> v2:172.21.15.144:3300/0 conn(0x7f5784145f30 0x7f5784071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:45.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.034+0000 7f5783e00640 1 -- 172.21.15.32:0/4191826488 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 315530687 0 0) 0x7f57840cdac0 con 0x7f57840cce70 2024-04-17T09:53:45.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.034+0000 7f5783e00640 1 -- 172.21.15.32:0/4191826488 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f576c003530 con 0x7f57840cce70 2024-04-17T09:53:45.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.034+0000 7f5789200640 1 -- 172.21.15.32:0/4191826488 >> v2:172.21.15.144:3300/0 conn(0x7f5784145f30 msgr2=0x7f5784071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:45.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.034+0000 7f5789200640 1 --2- 172.21.15.32:0/4191826488 >> v2:172.21.15.144:3300/0 conn(0x7f5784145f30 0x7f5784071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.034+0000 7f5789200640 1 -- 172.21.15.32:0/4191826488 >> v1:172.21.15.32:6789/0 conn(0x7f57840cce70 legacy=0x7f57840cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:45.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.034+0000 7f5789200640 1 -- 172.21.15.32:0/4191826488 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578415cdf0 con 0x7f578415d8d0 2024-04-17T09:53:45.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.038+0000 7f5789200640 1 --2- 172.21.15.32:0/4191826488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f578415d8d0 0x7f5784072330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f57700097c0 tx=0x7f577001ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de6a46842cd1a46f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:45.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.038+0000 7f5783e00640 1 -- 172.21.15.32:0/4191826488 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57700b5070 con 0x7f578415d8d0 2024-04-17T09:53:45.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.038+0000 7f5783e00640 1 -- 172.21.15.32:0/4191826488 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57700a8990 con 0x7f578415d8d0 2024-04-17T09:53:45.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.038+0000 7f5783e00640 1 -- 172.21.15.32:0/4191826488 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57700a8d60 con 0x7f578415d8d0 2024-04-17T09:53:45.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.038+0000 7f578c600640 1 -- 172.21.15.32:0/4191826488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f578415d8d0 msgr2=0x7f5784072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:45.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.038+0000 7f578c600640 1 --2- 172.21.15.32:0/4191826488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f578415d8d0 0x7f5784072330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f57700097c0 tx=0x7f577001ec90 comp rx=0 tx=0).stop 2024-04-17T09:53:45.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.038+0000 7f578c600640 1 -- 172.21.15.32:0/4191826488 shutdown_connections 2024-04-17T09:53:45.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.038+0000 7f578c600640 1 --2- 172.21.15.32:0/4191826488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f578415d8d0 0x7f5784072330 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.038+0000 7f578c600640 1 --2- 172.21.15.32:0/4191826488 >> v2:172.21.15.144:3300/0 conn(0x7f5784145f30 0x7f5784071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.038+0000 7f578c600640 1 -- 172.21.15.32:0/4191826488 >> 172.21.15.32:0/4191826488 conn(0x7f578406ba30 msgr2=0x7f578406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:45.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.038+0000 7f578c600640 1 -- 172.21.15.32:0/4191826488 shutdown_connections 2024-04-17T09:53:45.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.038+0000 7f578c600640 1 -- 172.21.15.32:0/4191826488 wait complete. 2024-04-17T09:53:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.038+0000 7f578c600640 1 Processor -- start 2024-04-17T09:53:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.038+0000 7f578c600640 1 -- start start 2024-04-17T09:53:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f578c600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57840cce70 0x7f57841e8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f578c600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5784145f30 0x7f57841443b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f578c600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f578415d8d0 0x7f57840b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f578c600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f578415cb90 con 0x7f5784145f30 2024-04-17T09:53:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f578c600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f578415d440 con 0x7f578415d8d0 2024-04-17T09:53:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f578c600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f57841601d0 con 0x7f57840cce70 2024-04-17T09:53:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f5789200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5784145f30 0x7f57841443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f5789200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5784145f30 0x7f57841443b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39096/0 (socket says 172.21.15.32:39096) 2024-04-17T09:53:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f5789200640 1 -- 172.21.15.32:0/989512395 learned_addr learned my addr 172.21.15.32:0/989512395 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f5789c00640 1 --2- 172.21.15.32:0/989512395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57840cce70 0x7f57841e8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f578a600640 1 --2- 172.21.15.32:0/989512395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f578415d8d0 0x7f57840b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f5789200640 1 -- 172.21.15.32:0/989512395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57840cce70 msgr2=0x7f57841e8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f5789200640 1 --2- 172.21.15.32:0/989512395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57840cce70 0x7f57841e8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f5789200640 1 -- 172.21.15.32:0/989512395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f578415d8d0 msgr2=0x7f57840b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f5789200640 1 --2- 172.21.15.32:0/989512395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f578415d8d0 0x7f57840b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f5789200640 1 -- 172.21.15.32:0/989512395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5784142c40 con 0x7f5784145f30 2024-04-17T09:53:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f5789c00640 1 --2- 172.21.15.32:0/989512395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57840cce70 0x7f57841e8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f578a600640 1 --2- 172.21.15.32:0/989512395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f578415d8d0 0x7f57840b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f5789200640 1 --2- 172.21.15.32:0/989512395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5784145f30 0x7f57841443b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f577001c8c0 tx=0x7f577001ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f5781600640 1 -- 172.21.15.32:0/989512395 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57700bf070 con 0x7f5784145f30 2024-04-17T09:53:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f5781600640 1 -- 172.21.15.32:0/989512395 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57700b0660 con 0x7f5784145f30 2024-04-17T09:53:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f5781600640 1 -- 172.21.15.32:0/989512395 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57700b5040 con 0x7f5784145f30 2024-04-17T09:53:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f578c600640 1 -- 172.21.15.32:0/989512395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5784142f20 con 0x7f5784145f30 2024-04-17T09:53:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f578c600640 1 -- 172.21.15.32:0/989512395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57841434e0 con 0x7f5784145f30 2024-04-17T09:53:45.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.042+0000 7f578c600640 1 -- 172.21.15.32:0/989512395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5784143720 con 0x7f5784145f30 2024-04-17T09:53:45.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.046+0000 7f5781600640 1 -- 172.21.15.32:0/989512395 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f57700c2070 con 0x7f5784145f30 2024-04-17T09:53:45.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.046+0000 7f5781600640 1 --2- 172.21.15.32:0/989512395 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5764068380 0x7f576406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:45.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.046+0000 7f5781600640 1 -- 172.21.15.32:0/989512395 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5770122f90 con 0x7f5784145f30 2024-04-17T09:53:45.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.046+0000 7f5789c00640 1 --2- 172.21.15.32:0/989512395 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5764068380 0x7f576406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:45.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.046+0000 7f5789c00640 1 --2- 172.21.15.32:0/989512395 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5764068380 0x7f576406a840 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5778009ae0 tx=0x7f5778008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:45.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.050+0000 7f5781600640 1 -- 172.21.15.32:0/989512395 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5784143720 con 0x7f5784145f30 2024-04-17T09:53:45.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.222+0000 7f578c600640 1 -- 172.21.15.32:0/989512395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5784068a90 con 0x7f5784145f30 2024-04-17T09:53:45.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.222+0000 7f5781600640 1 -- 172.21.15.32:0/989512395 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f57700187f0 con 0x7f5784145f30 2024-04-17T09:53:45.222 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:45.222 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:14.779535+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:53:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.222+0000 7f578c600640 1 -- 172.21.15.32:0/989512395 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5764068380 msgr2=0x7f576406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.222+0000 7f578c600640 1 --2- 172.21.15.32:0/989512395 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5764068380 0x7f576406a840 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5778009ae0 tx=0x7f5778008040 comp rx=0 tx=0).stop 2024-04-17T09:53:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.222+0000 7f578c600640 1 -- 172.21.15.32:0/989512395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5784145f30 msgr2=0x7f57841443b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.222+0000 7f578c600640 1 --2- 172.21.15.32:0/989512395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5784145f30 0x7f57841443b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f577001c8c0 tx=0x7f577001ba30 comp rx=0 tx=0).stop 2024-04-17T09:53:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.222+0000 7f578c600640 1 -- 172.21.15.32:0/989512395 shutdown_connections 2024-04-17T09:53:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.222+0000 7f578c600640 1 --2- 172.21.15.32:0/989512395 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5764068380 0x7f576406a840 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.222+0000 7f578c600640 1 --2- 172.21.15.32:0/989512395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f578415d8d0 0x7f57840b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.222+0000 7f578c600640 1 --2- 172.21.15.32:0/989512395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5784145f30 0x7f57841443b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.222+0000 7f578c600640 1 --2- 172.21.15.32:0/989512395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57840cce70 0x7f57841e8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.222+0000 7f578c600640 1 -- 172.21.15.32:0/989512395 >> 172.21.15.32:0/989512395 conn(0x7f578406ba30 msgr2=0x7f578415f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.222+0000 7f578c600640 1 -- 172.21.15.32:0/989512395 shutdown_connections 2024-04-17T09:53:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.222+0000 7f578c600640 1 -- 172.21.15.32:0/989512395 wait complete. 2024-04-17T09:53:45.243 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:53:11.198098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:53:12.118982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:53:45.244 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:53:45.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.386+0000 7fc939600640 1 Processor -- start 2024-04-17T09:53:45.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.386+0000 7fc939600640 1 -- start start 2024-04-17T09:53:45.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.386+0000 7fc939600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc93406f7c0 0x7fc93406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:45.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.386+0000 7fc939600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc93406f270 0x7fc93406bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:45.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.386+0000 7fc939600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9340cc940 con 0x7fc93415b7d0 2024-04-17T09:53:45.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.386+0000 7fc939600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc9340ccc20 con 0x7fc93406f270 2024-04-17T09:53:45.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.386+0000 7fc939600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc93406e0b0 con 0x7fc93406f7c0 2024-04-17T09:53:45.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.386+0000 7fc933400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc93406f270 0x7fc93406bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:45.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.386+0000 7fc933400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc93406f270 0x7fc93406bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39110/0 (socket says 172.21.15.32:39110) 2024-04-17T09:53:45.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.386+0000 7fc933400640 1 -- 172.21.15.32:0/292984863 learned_addr learned my addr 172.21.15.32:0/292984863 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:45.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.390+0000 7fc92a000640 1 --2- 172.21.15.32:0/292984863 >> v2:172.21.15.144:3300/0 conn(0x7fc93406f7c0 0x7fc93406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:45.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.390+0000 7fc933400640 1 -- 172.21.15.32:0/292984863 >> v2:172.21.15.144:3300/0 conn(0x7fc93406f7c0 msgr2=0x7fc93406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:45.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.390+0000 7fc933400640 1 --2- 172.21.15.32:0/292984863 >> v2:172.21.15.144:3300/0 conn(0x7fc93406f7c0 0x7fc93406ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.390+0000 7fc933400640 1 -- 172.21.15.32:0/292984863 >> v1:172.21.15.32:6789/0 conn(0x7fc93415b7d0 legacy=0x7fc93406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:45.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.390+0000 7fc933400640 1 -- 172.21.15.32:0/292984863 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93406e390 con 0x7fc93406f270 2024-04-17T09:53:45.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.390+0000 7fc933400640 1 --2- 172.21.15.32:0/292984863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc93406f270 0x7fc93406bf70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc9340b58f0 tx=0x7fc92c01eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de69ed187f17a28 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:45.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.390+0000 7fc932000640 1 -- 172.21.15.32:0/292984863 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c00b2d0 con 0x7fc93406f270 2024-04-17T09:53:45.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.390+0000 7fc932000640 1 -- 172.21.15.32:0/292984863 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc92c00b470 con 0x7fc93406f270 2024-04-17T09:53:45.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.390+0000 7fc932000640 1 -- 172.21.15.32:0/292984863 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c0a7b60 con 0x7fc93406f270 2024-04-17T09:53:45.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.390+0000 7fc939600640 1 -- 172.21.15.32:0/292984863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc93406f270 msgr2=0x7fc93406bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:45.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.390+0000 7fc939600640 1 --2- 172.21.15.32:0/292984863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc93406f270 0x7fc93406bf70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc9340b58f0 tx=0x7fc92c01eb60 comp rx=0 tx=0).stop 2024-04-17T09:53:45.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.390+0000 7fc939600640 1 -- 172.21.15.32:0/292984863 shutdown_connections 2024-04-17T09:53:45.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.390+0000 7fc939600640 1 --2- 172.21.15.32:0/292984863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc93406f270 0x7fc93406bf70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.390+0000 7fc939600640 1 --2- 172.21.15.32:0/292984863 >> v2:172.21.15.144:3300/0 conn(0x7fc93406f7c0 0x7fc93406ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc939600640 1 -- 172.21.15.32:0/292984863 >> 172.21.15.32:0/292984863 conn(0x7fc9340b2120 msgr2=0x7fc9340b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:45.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc939600640 1 -- 172.21.15.32:0/292984863 shutdown_connections 2024-04-17T09:53:45.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc939600640 1 -- 172.21.15.32:0/292984863 wait complete. 2024-04-17T09:53:45.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc939600640 1 Processor -- start 2024-04-17T09:53:45.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc939600640 1 -- start start 2024-04-17T09:53:45.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc939600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc93406f270 0x7fc93415c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:45.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc939600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc93406f7c0 0x7fc934152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:45.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc939600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc93415b7d0 0x7fc934144970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc939600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc9340cee30 con 0x7fc93406f7c0 2024-04-17T09:53:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc939600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc9340cecb0 con 0x7fc93406f270 2024-04-17T09:53:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc939600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc934071e60 con 0x7fc93415b7d0 2024-04-17T09:53:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc92a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc93406f7c0 0x7fc934152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc92a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc93406f7c0 0x7fc934152c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39118/0 (socket says 172.21.15.32:39118) 2024-04-17T09:53:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc92a000640 1 -- 172.21.15.32:0/2109931643 learned_addr learned my addr 172.21.15.32:0/2109931643 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc933400640 1 --2- 172.21.15.32:0/2109931643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc93415b7d0 0x7fc934144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc932a00640 1 --2- 172.21.15.32:0/2109931643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc93406f270 0x7fc93415c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc92a000640 1 -- 172.21.15.32:0/2109931643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc93415b7d0 msgr2=0x7fc934144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc92a000640 1 --2- 172.21.15.32:0/2109931643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc93415b7d0 0x7fc934144970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc92a000640 1 -- 172.21.15.32:0/2109931643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc93406f270 msgr2=0x7fc93415c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc92a000640 1 --2- 172.21.15.32:0/2109931643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc93406f270 0x7fc93415c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc92a000640 1 -- 172.21.15.32:0/2109931643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9340cd8c0 con 0x7fc93406f7c0 2024-04-17T09:53:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc92a000640 1 --2- 172.21.15.32:0/2109931643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc93406f7c0 0x7fc934152c10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc91c00d800 tx=0x7fc91c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:45.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc92b400640 1 -- 172.21.15.32:0/2109931643 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c00b6c0 con 0x7fc93406f7c0 2024-04-17T09:53:45.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc92b400640 1 -- 172.21.15.32:0/2109931643 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc91c019070 con 0x7fc93406f7c0 2024-04-17T09:53:45.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.394+0000 7fc92b400640 1 -- 172.21.15.32:0/2109931643 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c00ba60 con 0x7fc93406f7c0 2024-04-17T09:53:45.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.398+0000 7fc939600640 1 -- 172.21.15.32:0/2109931643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93415dc30 con 0x7fc93406f7c0 2024-04-17T09:53:45.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.398+0000 7fc939600640 1 -- 172.21.15.32:0/2109931643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc93415e1f0 con 0x7fc93406f7c0 2024-04-17T09:53:45.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.398+0000 7fc939600640 1 -- 172.21.15.32:0/2109931643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc934068a90 con 0x7fc93406f7c0 2024-04-17T09:53:45.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.398+0000 7fc92b400640 1 -- 172.21.15.32:0/2109931643 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc91c00bc00 con 0x7fc93406f7c0 2024-04-17T09:53:45.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.398+0000 7fc92b400640 1 --2- 172.21.15.32:0/2109931643 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc910068380 0x7fc91006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:45.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.398+0000 7fc92b400640 1 -- 172.21.15.32:0/2109931643 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc91c086d50 con 0x7fc93406f7c0 2024-04-17T09:53:45.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.398+0000 7fc932a00640 1 --2- 172.21.15.32:0/2109931643 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc910068380 0x7fc91006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:45.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.398+0000 7fc932a00640 1 --2- 172.21.15.32:0/2109931643 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc910068380 0x7fc91006a840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc924001770 tx=0x7fc924009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:45.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.402+0000 7fc92b400640 1 -- 172.21.15.32:0/2109931643 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc91c08a030 con 0x7fc93406f7c0 2024-04-17T09:53:45.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.566+0000 7fc939600640 1 -- 172.21.15.32:0/2109931643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc934072ae0 con 0x7fc93406f7c0 2024-04-17T09:53:45.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.566+0000 7fc92b400640 1 -- 172.21.15.32:0/2109931643 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fc91c056cc0 con 0x7fc93406f7c0 2024-04-17T09:53:45.568 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:53:45.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.570+0000 7fc939600640 1 -- 172.21.15.32:0/2109931643 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc910068380 msgr2=0x7fc91006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:45.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.570+0000 7fc939600640 1 --2- 172.21.15.32:0/2109931643 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc910068380 0x7fc91006a840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc924001770 tx=0x7fc924009040 comp rx=0 tx=0).stop 2024-04-17T09:53:45.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.570+0000 7fc939600640 1 -- 172.21.15.32:0/2109931643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc93406f7c0 msgr2=0x7fc934152c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:45.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.570+0000 7fc939600640 1 --2- 172.21.15.32:0/2109931643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc93406f7c0 0x7fc934152c10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc91c00d800 tx=0x7fc91c00dcc0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.570+0000 7fc939600640 1 -- 172.21.15.32:0/2109931643 shutdown_connections 2024-04-17T09:53:45.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.570+0000 7fc939600640 1 --2- 172.21.15.32:0/2109931643 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc910068380 0x7fc91006a840 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.570+0000 7fc939600640 1 --2- 172.21.15.32:0/2109931643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc93415b7d0 0x7fc934144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.570+0000 7fc939600640 1 --2- 172.21.15.32:0/2109931643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc93406f7c0 0x7fc934152c10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.570+0000 7fc939600640 1 --2- 172.21.15.32:0/2109931643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc93406f270 0x7fc93415c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.570+0000 7fc939600640 1 -- 172.21.15.32:0/2109931643 >> 172.21.15.32:0/2109931643 conn(0x7fc9340b2120 msgr2=0x7fc934151b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:45.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.570+0000 7fc939600640 1 -- 172.21.15.32:0/2109931643 shutdown_connections 2024-04-17T09:53:45.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.570+0000 7fc939600640 1 -- 172.21.15.32:0/2109931643 wait complete. 2024-04-17T09:53:45.588 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:53:45.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.730+0000 7f2397200640 1 Processor -- start 2024-04-17T09:53:45.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.730+0000 7f2397200640 1 -- start start 2024-04-17T09:53:45.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.734+0000 7f2397200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f23900cf370 0x7f23900cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:45.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.734+0000 7f2397200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23900d0130 con 0x7f2390071e60 2024-04-17T09:53:45.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.734+0000 7f2397200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f239015d090 con 0x7f239006ba30 2024-04-17T09:53:45.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.734+0000 7f2397200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f239015d370 con 0x7f23900cf370 2024-04-17T09:53:45.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.734+0000 7f2395200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2390071e60 0x7f2390072270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44352/0 (socket says 172.21.15.32:44352) 2024-04-17T09:53:45.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.734+0000 7f2395200640 1 -- 172.21.15.32:0/3599007904 learned_addr learned my addr 172.21.15.32:0/3599007904 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:45.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.734+0000 7f2387400640 1 --2- 172.21.15.32:0/3599007904 >> v2:172.21.15.144:3300/0 conn(0x7f23900cf370 0x7f23900cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:45.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.734+0000 7f238f400640 1 -- 172.21.15.32:0/3599007904 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2170581886 0 0) 0x7f23900d0130 con 0x7f2390071e60 2024-04-17T09:53:45.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.734+0000 7f238f400640 1 -- 172.21.15.32:0/3599007904 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2374003530 con 0x7f2390071e60 2024-04-17T09:53:45.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.734+0000 7f238f400640 1 -- 172.21.15.32:0/3599007904 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3737112566 0 0) 0x7f239015d090 con 0x7f239006ba30 2024-04-17T09:53:45.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.734+0000 7f238f400640 1 -- 172.21.15.32:0/3599007904 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23900d0130 con 0x7f239006ba30 2024-04-17T09:53:45.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.734+0000 7f2387400640 1 -- 172.21.15.32:0/3599007904 >> v1:172.21.15.144:6789/0 conn(0x7f239006ba30 legacy=0x7f239006be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:45.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.734+0000 7f2387400640 1 -- 172.21.15.32:0/3599007904 >> v1:172.21.15.32:6789/0 conn(0x7f2390071e60 legacy=0x7f2390072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:45.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.734+0000 7f238f400640 1 -- 172.21.15.32:0/3599007904 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3908051676 0 0) 0x7f2374003530 con 0x7f2390071e60 2024-04-17T09:53:45.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.734+0000 7f2387400640 1 -- 172.21.15.32:0/3599007904 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239015d650 con 0x7f23900cf370 2024-04-17T09:53:45.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.738+0000 7f2387400640 1 --2- 172.21.15.32:0/3599007904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23900cf370 0x7f23900cf7e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f23780097c0 tx=0x7f237801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63dd1831014bbfd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:45.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.738+0000 7f238f400640 1 -- 172.21.15.32:0/3599007904 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23780b0070 con 0x7f23900cf370 2024-04-17T09:53:45.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.738+0000 7f238f400640 1 -- 172.21.15.32:0/3599007904 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23780ab400 con 0x7f23900cf370 2024-04-17T09:53:45.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.738+0000 7f238f400640 1 -- 172.21.15.32:0/3599007904 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388001680 con 0x7f23900cf370 2024-04-17T09:53:45.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.738+0000 7f2397200640 1 -- 172.21.15.32:0/3599007904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23900cf370 msgr2=0x7f23900cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:45.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.738+0000 7f2397200640 1 --2- 172.21.15.32:0/3599007904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23900cf370 0x7f23900cf7e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f23780097c0 tx=0x7f237801c3c0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.738+0000 7f2397200640 1 -- 172.21.15.32:0/3599007904 shutdown_connections 2024-04-17T09:53:45.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.738+0000 7f2397200640 1 --2- 172.21.15.32:0/3599007904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23900cf370 0x7f23900cf7e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.738+0000 7f2397200640 1 -- 172.21.15.32:0/3599007904 >> 172.21.15.32:0/3599007904 conn(0x7f23900b2120 msgr2=0x7f23900b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:45.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.738+0000 7f2397200640 1 -- 172.21.15.32:0/3599007904 shutdown_connections 2024-04-17T09:53:45.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.738+0000 7f2397200640 1 -- 172.21.15.32:0/3599007904 wait complete. 2024-04-17T09:53:45.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.738+0000 7f2397200640 1 Processor -- start 2024-04-17T09:53:45.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f2397200640 1 -- start start 2024-04-17T09:53:45.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f2397200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f239006ba30 0x7f2390159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:45.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f2397200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2390071e60 0x7f2390136da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:45.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f2397200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23900cf370 0x7f239013b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:45.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f2397200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f239015dcf0 con 0x7f239006ba30 2024-04-17T09:53:45.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f2397200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f239015db70 con 0x7f2390071e60 2024-04-17T09:53:45.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f2397200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2390146230 con 0x7f23900cf370 2024-04-17T09:53:45.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f238fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f239006ba30 0x7f2390159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:45.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f238fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f239006ba30 0x7f2390159f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39132/0 (socket says 172.21.15.32:39132) 2024-04-17T09:53:45.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f238fe00640 1 -- 172.21.15.32:0/1229934044 learned_addr learned my addr 172.21.15.32:0/1229934044 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:45.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f2387400640 1 --2- 172.21.15.32:0/1229934044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2390071e60 0x7f2390136da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f2395200640 1 --2- 172.21.15.32:0/1229934044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23900cf370 0x7f239013b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f238fe00640 1 -- 172.21.15.32:0/1229934044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23900cf370 msgr2=0x7f239013b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f238fe00640 1 --2- 172.21.15.32:0/1229934044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23900cf370 0x7f239013b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f238fe00640 1 -- 172.21.15.32:0/1229934044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2390071e60 msgr2=0x7f2390136da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f238fe00640 1 --2- 172.21.15.32:0/1229934044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2390071e60 0x7f2390136da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f238fe00640 1 -- 172.21.15.32:0/1229934044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23900b4250 con 0x7f239006ba30 2024-04-17T09:53:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f238fe00640 1 --2- 172.21.15.32:0/1229934044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f239006ba30 0x7f2390159f50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f238000c8a0 tx=0x7f238000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f238cc00640 1 -- 172.21.15.32:0/1229934044 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380003b80 con 0x7f239006ba30 2024-04-17T09:53:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f238cc00640 1 -- 172.21.15.32:0/1229934044 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2380003d20 con 0x7f239006ba30 2024-04-17T09:53:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f2397200640 1 -- 172.21.15.32:0/1229934044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23901362c0 con 0x7f239006ba30 2024-04-17T09:53:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f238cc00640 1 -- 172.21.15.32:0/1229934044 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238000aa30 con 0x7f239006ba30 2024-04-17T09:53:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f2397200640 1 -- 172.21.15.32:0/1229934044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f239015a490 con 0x7f239006ba30 2024-04-17T09:53:45.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.742+0000 7f2397200640 1 -- 172.21.15.32:0/1229934044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2390068a90 con 0x7f239006ba30 2024-04-17T09:53:45.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.746+0000 7f238cc00640 1 -- 172.21.15.32:0/1229934044 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f238000abd0 con 0x7f239006ba30 2024-04-17T09:53:45.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.746+0000 7f238cc00640 1 --2- 172.21.15.32:0/1229934044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f236c068380 0x7f236c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:45.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.746+0000 7f238cc00640 1 -- 172.21.15.32:0/1229934044 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2380086a60 con 0x7f239006ba30 2024-04-17T09:53:45.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.746+0000 7f2387400640 1 --2- 172.21.15.32:0/1229934044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f236c068380 0x7f236c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:45.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.746+0000 7f2387400640 1 --2- 172.21.15.32:0/1229934044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f236c068380 0x7f236c06a840 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f239015a950 tx=0x7f23780a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:45.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.750+0000 7f238cc00640 1 -- 172.21.15.32:0/1229934044 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2380056f80 con 0x7f239006ba30 2024-04-17T09:53:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.918+0000 7f2397200640 1 -- 172.21.15.32:0/1229934044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f239015a730 con 0x7f239006ba30 2024-04-17T09:53:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.918+0000 7f238cc00640 1 -- 172.21.15.32:0/1229934044 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f2380056880 con 0x7f239006ba30 2024-04-17T09:53:45.919 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:53:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.922+0000 7f2397200640 1 -- 172.21.15.32:0/1229934044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f236c068380 msgr2=0x7f236c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.922+0000 7f2397200640 1 --2- 172.21.15.32:0/1229934044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f236c068380 0x7f236c06a840 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f239015a950 tx=0x7f23780a6000 comp rx=0 tx=0).stop 2024-04-17T09:53:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.922+0000 7f2397200640 1 -- 172.21.15.32:0/1229934044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f239006ba30 msgr2=0x7f2390159f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.922+0000 7f2397200640 1 --2- 172.21.15.32:0/1229934044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f239006ba30 0x7f2390159f50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f238000c8a0 tx=0x7f238000cd60 comp rx=0 tx=0).stop 2024-04-17T09:53:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.922+0000 7f2397200640 1 -- 172.21.15.32:0/1229934044 shutdown_connections 2024-04-17T09:53:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.922+0000 7f2397200640 1 --2- 172.21.15.32:0/1229934044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f236c068380 0x7f236c06a840 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.922+0000 7f2397200640 1 --2- 172.21.15.32:0/1229934044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23900cf370 0x7f239013b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.922+0000 7f2397200640 1 --2- 172.21.15.32:0/1229934044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2390071e60 0x7f2390136da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.922+0000 7f2397200640 1 --2- 172.21.15.32:0/1229934044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f239006ba30 0x7f2390159f50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.922+0000 7f2397200640 1 -- 172.21.15.32:0/1229934044 >> 172.21.15.32:0/1229934044 conn(0x7f23900b2120 msgr2=0x7f23900ce1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.922+0000 7f2397200640 1 -- 172.21.15.32:0/1229934044 shutdown_connections 2024-04-17T09:53:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:45.922+0000 7f2397200640 1 -- 172.21.15.32:0/1229934044 wait complete. 2024-04-17T09:53:45.940 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:53:46.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.082+0000 7f3e41a00640 1 Processor -- start 2024-04-17T09:53:46.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.082+0000 7f3e41a00640 1 -- start start 2024-04-17T09:53:46.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.086+0000 7f3e41a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3e3c06f7c0 0x7f3e3c06ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:46.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.086+0000 7f3e41a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e3c0cc940 con 0x7f3e3c15b7d0 2024-04-17T09:53:46.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.086+0000 7f3e41a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e3c0ccc20 con 0x7f3e3c06f270 2024-04-17T09:53:46.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.086+0000 7f3e41a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3e3c06e0b0 con 0x7f3e3c06f7c0 2024-04-17T09:53:46.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.086+0000 7f3e32000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3e3c06f7c0 0x7f3e3c06ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:46.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.086+0000 7f3e32000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3e3c06f7c0 0x7f3e3c06ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39148/0 (socket says 172.21.15.32:39148) 2024-04-17T09:53:46.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.086+0000 7f3e3aa00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3e3c15b7d0 0x7f3e3c06e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44356/0 (socket says 172.21.15.32:44356) 2024-04-17T09:53:46.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.086+0000 7f3e32000640 1 -- 172.21.15.32:0/2890133443 learned_addr learned my addr 172.21.15.32:0/2890133443 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:46.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.086+0000 7f3e32000640 1 -- 172.21.15.32:0/2890133443 >> v1:172.21.15.144:6789/0 conn(0x7f3e3c06f270 legacy=0x7f3e3c0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:46.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.086+0000 7f3e32000640 1 -- 172.21.15.32:0/2890133443 >> v1:172.21.15.32:6789/0 conn(0x7f3e3c15b7d0 legacy=0x7f3e3c06e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:46.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.086+0000 7f3e32000640 1 -- 172.21.15.32:0/2890133443 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3c06e390 con 0x7f3e3c06f7c0 2024-04-17T09:53:46.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.086+0000 7f3e3a000640 1 -- 172.21.15.32:0/2890133443 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1258197057 0 0) 0x7f3e3c0cc940 con 0x7f3e3c15b7d0 2024-04-17T09:53:46.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.086+0000 7f3e32000640 1 --2- 172.21.15.32:0/2890133443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e3c06f7c0 0x7f3e3c06ba30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3e2400bb10 tx=0x7f3e2401eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a408f4f4026831 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:46.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.086+0000 7f3e3a000640 1 -- 172.21.15.32:0/2890133443 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e240b4070 con 0x7f3e3c06f7c0 2024-04-17T09:53:46.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.090+0000 7f3e3a000640 1 -- 172.21.15.32:0/2890133443 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e240a7b90 con 0x7f3e3c06f7c0 2024-04-17T09:53:46.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.090+0000 7f3e3a000640 1 -- 172.21.15.32:0/2890133443 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e240af440 con 0x7f3e3c06f7c0 2024-04-17T09:53:46.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.090+0000 7f3e41a00640 1 -- 172.21.15.32:0/2890133443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e3c06f7c0 msgr2=0x7f3e3c06ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:46.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.090+0000 7f3e41a00640 1 --2- 172.21.15.32:0/2890133443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e3c06f7c0 0x7f3e3c06ba30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3e2400bb10 tx=0x7f3e2401eba0 comp rx=0 tx=0).stop 2024-04-17T09:53:46.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.090+0000 7f3e41a00640 1 -- 172.21.15.32:0/2890133443 shutdown_connections 2024-04-17T09:53:46.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.090+0000 7f3e41a00640 1 --2- 172.21.15.32:0/2890133443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e3c06f7c0 0x7f3e3c06ba30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:46.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.090+0000 7f3e41a00640 1 -- 172.21.15.32:0/2890133443 >> 172.21.15.32:0/2890133443 conn(0x7f3e3c0b2120 msgr2=0x7f3e3c0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:46.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.090+0000 7f3e41a00640 1 -- 172.21.15.32:0/2890133443 shutdown_connections 2024-04-17T09:53:46.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.090+0000 7f3e41a00640 1 -- 172.21.15.32:0/2890133443 wait complete. 2024-04-17T09:53:46.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.090+0000 7f3e41a00640 1 Processor -- start 2024-04-17T09:53:46.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.090+0000 7f3e41a00640 1 -- start start 2024-04-17T09:53:46.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.090+0000 7f3e41a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e3c06f270 0x7f3e3c0ae450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:46.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.090+0000 7f3e41a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c06f7c0 0x7f3e3c158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:46.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.090+0000 7f3e41a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e3c15b7d0 0x7f3e3c15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:46.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.090+0000 7f3e41a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c072a40 con 0x7f3e3c15b7d0 2024-04-17T09:53:46.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.090+0000 7f3e41a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c0728c0 con 0x7f3e3c06f270 2024-04-17T09:53:46.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.090+0000 7f3e41a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3e3c072bc0 con 0x7f3e3c06f7c0 2024-04-17T09:53:46.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.090+0000 7f3e3b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e3c15b7d0 0x7f3e3c15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:46.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.090+0000 7f3e3b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e3c15b7d0 0x7f3e3c15a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39154/0 (socket says 172.21.15.32:39154) 2024-04-17T09:53:46.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.090+0000 7f3e3b400640 1 -- 172.21.15.32:0/1875708980 learned_addr learned my addr 172.21.15.32:0/1875708980 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:46.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.090+0000 7f3e3aa00640 1 --2- 172.21.15.32:0/1875708980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e3c06f270 0x7f3e3c0ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:46.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.090+0000 7f3e32000640 1 --2- 172.21.15.32:0/1875708980 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c06f7c0 0x7f3e3c158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:46.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.094+0000 7f3e3b400640 1 -- 172.21.15.32:0/1875708980 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c06f7c0 msgr2=0x7f3e3c158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:46.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.094+0000 7f3e3b400640 1 --2- 172.21.15.32:0/1875708980 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c06f7c0 0x7f3e3c158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:46.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.094+0000 7f3e3b400640 1 -- 172.21.15.32:0/1875708980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e3c06f270 msgr2=0x7f3e3c0ae450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:46.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.094+0000 7f3e3b400640 1 --2- 172.21.15.32:0/1875708980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e3c06f270 0x7f3e3c0ae450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:46.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.094+0000 7f3e3b400640 1 -- 172.21.15.32:0/1875708980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3c0af4d0 con 0x7f3e3c15b7d0 2024-04-17T09:53:46.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.094+0000 7f3e3b400640 1 --2- 172.21.15.32:0/1875708980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e3c15b7d0 0x7f3e3c15a5e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400c730 tx=0x7f3e3400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:46.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.094+0000 7f3e33400640 1 -- 172.21.15.32:0/1875708980 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34003b80 con 0x7f3e3c15b7d0 2024-04-17T09:53:46.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.094+0000 7f3e33400640 1 -- 172.21.15.32:0/1875708980 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e34003d20 con 0x7f3e3c15b7d0 2024-04-17T09:53:46.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.094+0000 7f3e41a00640 1 -- 172.21.15.32:0/1875708980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e3c0ad990 con 0x7f3e3c15b7d0 2024-04-17T09:53:46.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.094+0000 7f3e33400640 1 -- 172.21.15.32:0/1875708980 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3400aa30 con 0x7f3e3c15b7d0 2024-04-17T09:53:46.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.094+0000 7f3e41a00640 1 -- 172.21.15.32:0/1875708980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e3c0adf50 con 0x7f3e3c15b7d0 2024-04-17T09:53:46.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.094+0000 7f3e41a00640 1 -- 172.21.15.32:0/1875708980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e3c068a90 con 0x7f3e3c15b7d0 2024-04-17T09:53:46.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.094+0000 7f3e33400640 1 -- 172.21.15.32:0/1875708980 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3e3400abd0 con 0x7f3e3c15b7d0 2024-04-17T09:53:46.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.094+0000 7f3e33400640 1 --2- 172.21.15.32:0/1875708980 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3e18068380 0x7f3e1806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:46.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.094+0000 7f3e33400640 1 -- 172.21.15.32:0/1875708980 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3e34086730 con 0x7f3e3c15b7d0 2024-04-17T09:53:46.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.094+0000 7f3e3aa00640 1 --2- 172.21.15.32:0/1875708980 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3e18068380 0x7f3e1806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:46.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.098+0000 7f3e3aa00640 1 --2- 172.21.15.32:0/1875708980 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3e18068380 0x7f3e1806a840 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c000f70 tx=0x7f3e2c007240 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:46.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.098+0000 7f3e33400640 1 -- 172.21.15.32:0/1875708980 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3e34056690 con 0x7f3e3c15b7d0 2024-04-17T09:53:46.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.262+0000 7f3e41a00640 1 -- 172.21.15.32:0/1875708980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3e3c070100 con 0x7f3e3c15b7d0 2024-04-17T09:53:46.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.262+0000 7f3e33400640 1 -- 172.21.15.32:0/1875708980 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f3e3405a990 con 0x7f3e3c15b7d0 2024-04-17T09:53:46.264 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:53:46.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.266+0000 7f3e41a00640 1 -- 172.21.15.32:0/1875708980 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3e18068380 msgr2=0x7f3e1806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:46.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.266+0000 7f3e41a00640 1 --2- 172.21.15.32:0/1875708980 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3e18068380 0x7f3e1806a840 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c000f70 tx=0x7f3e2c007240 comp rx=0 tx=0).stop 2024-04-17T09:53:46.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.266+0000 7f3e41a00640 1 -- 172.21.15.32:0/1875708980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e3c15b7d0 msgr2=0x7f3e3c15a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:46.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.266+0000 7f3e41a00640 1 --2- 172.21.15.32:0/1875708980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e3c15b7d0 0x7f3e3c15a5e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400c730 tx=0x7f3e3400cbf0 comp rx=0 tx=0).stop 2024-04-17T09:53:46.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.266+0000 7f3e41a00640 1 -- 172.21.15.32:0/1875708980 shutdown_connections 2024-04-17T09:53:46.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.266+0000 7f3e41a00640 1 --2- 172.21.15.32:0/1875708980 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3e18068380 0x7f3e1806a840 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:46.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.266+0000 7f3e41a00640 1 --2- 172.21.15.32:0/1875708980 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c06f7c0 0x7f3e3c158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:46.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.266+0000 7f3e41a00640 1 --2- 172.21.15.32:0/1875708980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e3c15b7d0 0x7f3e3c15a5e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:46.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.266+0000 7f3e41a00640 1 --2- 172.21.15.32:0/1875708980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e3c06f270 0x7f3e3c0ae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:46.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.266+0000 7f3e41a00640 1 -- 172.21.15.32:0/1875708980 >> 172.21.15.32:0/1875708980 conn(0x7f3e3c0b2120 msgr2=0x7f3e3c156080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:46.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.266+0000 7f3e41a00640 1 -- 172.21.15.32:0/1875708980 shutdown_connections 2024-04-17T09:53:46.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.266+0000 7f3e41a00640 1 -- 172.21.15.32:0/1875708980 wait complete. 2024-04-17T09:53:46.286 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-17T09:53:46.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.426+0000 7f3d22000640 1 Processor -- start 2024-04-17T09:53:46.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.426+0000 7f3d22000640 1 -- start start 2024-04-17T09:53:46.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.426+0000 7f3d22000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3d1c15c5c0 0x7f3d1c0cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:46.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.426+0000 7f3d22000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d1c0d05e0 con 0x7f3d1c09b100 2024-04-17T09:53:46.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.426+0000 7f3d22000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d1c0d08c0 con 0x7f3d1c0d00b0 2024-04-17T09:53:46.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.426+0000 7f3d22000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3d1c0d11f0 con 0x7f3d1c15c5c0 2024-04-17T09:53:46.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.426+0000 7f3d1b400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3d1c15c5c0 0x7f3d1c0cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:46.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.426+0000 7f3d1be00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3d1c09b100 0x7f3d1c0b31b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44370/0 (socket says 172.21.15.32:44370) 2024-04-17T09:53:46.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.426+0000 7f3d1be00640 1 -- 172.21.15.32:0/1301500085 learned_addr learned my addr 172.21.15.32:0/1301500085 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:46.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.426+0000 7f3d1b400640 1 -- 172.21.15.32:0/1301500085 >> v1:172.21.15.144:6789/0 conn(0x7f3d1c0d00b0 legacy=0x7f3d1c09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:46.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.426+0000 7f3d1b400640 1 -- 172.21.15.32:0/1301500085 >> v1:172.21.15.32:6789/0 conn(0x7f3d1c09b100 legacy=0x7f3d1c0b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:46.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.426+0000 7f3d1aa00640 1 -- 172.21.15.32:0/1301500085 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 220307757 0 0) 0x7f3d1c0d05e0 con 0x7f3d1c09b100 2024-04-17T09:53:46.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.426+0000 7f3d1b400640 1 -- 172.21.15.32:0/1301500085 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1c0d0ba0 con 0x7f3d1c15c5c0 2024-04-17T09:53:46.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.430+0000 7f3d1b400640 1 --2- 172.21.15.32:0/1301500085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d1c15c5c0 0x7f3d1c0cde50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0097c0 tx=0x7f3d0c01ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5a533a9eb3b141 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:46.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.430+0000 7f3d1aa00640 1 -- 172.21.15.32:0/1301500085 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c0b5070 con 0x7f3d1c15c5c0 2024-04-17T09:53:46.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.430+0000 7f3d1aa00640 1 -- 172.21.15.32:0/1301500085 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d0c0a8990 con 0x7f3d1c15c5c0 2024-04-17T09:53:46.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.430+0000 7f3d1aa00640 1 -- 172.21.15.32:0/1301500085 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c0a8d60 con 0x7f3d1c15c5c0 2024-04-17T09:53:46.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.430+0000 7f3d22000640 1 -- 172.21.15.32:0/1301500085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d1c15c5c0 msgr2=0x7f3d1c0cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:46.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.430+0000 7f3d22000640 1 --2- 172.21.15.32:0/1301500085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d1c15c5c0 0x7f3d1c0cde50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0097c0 tx=0x7f3d0c01ec90 comp rx=0 tx=0).stop 2024-04-17T09:53:46.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.430+0000 7f3d22000640 1 -- 172.21.15.32:0/1301500085 shutdown_connections 2024-04-17T09:53:46.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.430+0000 7f3d22000640 1 --2- 172.21.15.32:0/1301500085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d1c15c5c0 0x7f3d1c0cde50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:46.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.430+0000 7f3d22000640 1 -- 172.21.15.32:0/1301500085 >> 172.21.15.32:0/1301500085 conn(0x7f3d1c0b3fd0 msgr2=0x7f3d1c0b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:46.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.430+0000 7f3d22000640 1 -- 172.21.15.32:0/1301500085 shutdown_connections 2024-04-17T09:53:46.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.430+0000 7f3d22000640 1 -- 172.21.15.32:0/1301500085 wait complete. 2024-04-17T09:53:46.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d22000640 1 Processor -- start 2024-04-17T09:53:46.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d22000640 1 -- start start 2024-04-17T09:53:46.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d22000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d1c09b100 0x7f3d1c15f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:46.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d22000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d1c0d00b0 0x7f3d1c0cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:46.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d22000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d1c15c5c0 0x7f3d1c0cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:46.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d22000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c0ce510 con 0x7f3d1c0d00b0 2024-04-17T09:53:46.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d22000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c0ce390 con 0x7f3d1c15c5c0 2024-04-17T09:53:46.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d22000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3d1c0b6340 con 0x7f3d1c09b100 2024-04-17T09:53:46.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d12a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d1c0d00b0 0x7f3d1c0cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:46.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d12a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d1c0d00b0 0x7f3d1c0cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39168/0 (socket says 172.21.15.32:39168) 2024-04-17T09:53:46.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d12a00640 1 -- 172.21.15.32:0/1041129329 learned_addr learned my addr 172.21.15.32:0/1041129329 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:46.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d1b400640 1 --2- 172.21.15.32:0/1041129329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d1c09b100 0x7f3d1c15f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d1be00640 1 --2- 172.21.15.32:0/1041129329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d1c15c5c0 0x7f3d1c0cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d12a00640 1 -- 172.21.15.32:0/1041129329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d1c09b100 msgr2=0x7f3d1c15f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d12a00640 1 --2- 172.21.15.32:0/1041129329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d1c09b100 0x7f3d1c15f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d12a00640 1 -- 172.21.15.32:0/1041129329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d1c15c5c0 msgr2=0x7f3d1c0cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d12a00640 1 --2- 172.21.15.32:0/1041129329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d1c15c5c0 0x7f3d1c0cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d12a00640 1 -- 172.21.15.32:0/1041129329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1c15d780 con 0x7f3d1c0d00b0 2024-04-17T09:53:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d12a00640 1 --2- 172.21.15.32:0/1041129329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d1c0d00b0 0x7f3d1c0cadb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400c500 tx=0x7f3d0400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d13e00640 1 -- 172.21.15.32:0/1041129329 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04016020 con 0x7f3d1c0d00b0 2024-04-17T09:53:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d13e00640 1 -- 172.21.15.32:0/1041129329 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d04005220 con 0x7f3d1c0d00b0 2024-04-17T09:53:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d13e00640 1 -- 172.21.15.32:0/1041129329 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04007030 con 0x7f3d1c0d00b0 2024-04-17T09:53:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d22000640 1 -- 172.21.15.32:0/1041129329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d1c15da60 con 0x7f3d1c0d00b0 2024-04-17T09:53:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.434+0000 7f3d22000640 1 -- 172.21.15.32:0/1041129329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d1c15e020 con 0x7f3d1c0d00b0 2024-04-17T09:53:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.438+0000 7f3d22000640 1 -- 172.21.15.32:0/1041129329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d1c068a90 con 0x7f3d1c0d00b0 2024-04-17T09:53:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.438+0000 7f3d13e00640 1 -- 172.21.15.32:0/1041129329 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3d040071f0 con 0x7f3d1c0d00b0 2024-04-17T09:53:46.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.438+0000 7f3d13e00640 1 --2- 172.21.15.32:0/1041129329 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3cf8068380 0x7f3cf806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:46.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.438+0000 7f3d13e00640 1 -- 172.21.15.32:0/1041129329 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3d040862d0 con 0x7f3d1c0d00b0 2024-04-17T09:53:46.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.438+0000 7f3d1b400640 1 --2- 172.21.15.32:0/1041129329 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3cf8068380 0x7f3cf806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:46.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.438+0000 7f3d1b400640 1 --2- 172.21.15.32:0/1041129329 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3cf8068380 0x7f3cf806a840 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0b2000 tx=0x7f3d0c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:46.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.442+0000 7f3d13e00640 1 -- 172.21.15.32:0/1041129329 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3d04020020 con 0x7f3d1c0d00b0 2024-04-17T09:53:46.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.666+0000 7f3d22000640 1 -- 172.21.15.32:0/1041129329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7f3d1c0b4c30 con 0x7f3d1c0d00b0 2024-04-17T09:53:46.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.906+0000 7f3d13e00640 1 -- 172.21.15.32:0/1041129329 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v52) ==== 145+0+0 (secure 0 0 0) 0x7f3d04056250 con 0x7f3d1c0d00b0 2024-04-17T09:53:46.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.906+0000 7f3d22000640 1 -- 172.21.15.32:0/1041129329 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3cf8068380 msgr2=0x7f3cf806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:46.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.906+0000 7f3d22000640 1 --2- 172.21.15.32:0/1041129329 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3cf8068380 0x7f3cf806a840 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0b2000 tx=0x7f3d0c008040 comp rx=0 tx=0).stop 2024-04-17T09:53:46.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.906+0000 7f3d22000640 1 -- 172.21.15.32:0/1041129329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d1c0d00b0 msgr2=0x7f3d1c0cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:46.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.906+0000 7f3d22000640 1 --2- 172.21.15.32:0/1041129329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d1c0d00b0 0x7f3d1c0cadb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400c500 tx=0x7f3d0400c9c0 comp rx=0 tx=0).stop 2024-04-17T09:53:46.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.906+0000 7f3d22000640 1 -- 172.21.15.32:0/1041129329 shutdown_connections 2024-04-17T09:53:46.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.906+0000 7f3d22000640 1 --2- 172.21.15.32:0/1041129329 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3cf8068380 0x7f3cf806a840 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:46.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.906+0000 7f3d22000640 1 --2- 172.21.15.32:0/1041129329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d1c15c5c0 0x7f3d1c0cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:46.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.906+0000 7f3d22000640 1 --2- 172.21.15.32:0/1041129329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d1c0d00b0 0x7f3d1c0cadb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:46.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.906+0000 7f3d22000640 1 --2- 172.21.15.32:0/1041129329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d1c09b100 0x7f3d1c15f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:46.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.906+0000 7f3d22000640 1 -- 172.21.15.32:0/1041129329 >> 172.21.15.32:0/1041129329 conn(0x7f3d1c0b3fd0 msgr2=0x7f3d1c0ccfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:46.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.906+0000 7f3d22000640 1 -- 172.21.15.32:0/1041129329 shutdown_connections 2024-04-17T09:53:46.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:46.906+0000 7f3d22000640 1 -- 172.21.15.32:0/1041129329 wait complete. 2024-04-17T09:53:46.927 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-17T09:53:47.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.070+0000 7f9e79400640 1 Processor -- start 2024-04-17T09:53:47.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.070+0000 7f9e79400640 1 -- start start 2024-04-17T09:53:47.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.070+0000 7f9e79400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9e7415b7d0 0x7f9e7406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:47.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.070+0000 7f9e79400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e7406f7c0 0x7f9e7406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:47.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.070+0000 7f9e79400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e7406f270 0x7f9e7406bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:47.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.070+0000 7f9e79400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9e740cc940 con 0x7f9e7406f7c0 2024-04-17T09:53:47.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.070+0000 7f9e79400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f9e740ccc20 con 0x7f9e7415b7d0 2024-04-17T09:53:47.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.070+0000 7f9e79400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9e7406e0b0 con 0x7f9e7406f270 2024-04-17T09:53:47.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.070+0000 7f9e6be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e7406f7c0 0x7f9e7406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:47.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.070+0000 7f9e6be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e7406f7c0 0x7f9e7406ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39176/0 (socket says 172.21.15.32:39176) 2024-04-17T09:53:47.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.070+0000 7f9e6be00640 1 -- 172.21.15.32:0/2555341738 learned_addr learned my addr 172.21.15.32:0/2555341738 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:47.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.074+0000 7f9e72800640 1 --2- 172.21.15.32:0/2555341738 >> v2:172.21.15.144:3300/0 conn(0x7f9e7415b7d0 0x7f9e7406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:47.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.074+0000 7f9e73200640 1 --2- 172.21.15.32:0/2555341738 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e7406f270 0x7f9e7406bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:47.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.074+0000 7f9e6be00640 1 -- 172.21.15.32:0/2555341738 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e7406f270 msgr2=0x7f9e7406bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:47.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.074+0000 7f9e6be00640 1 --2- 172.21.15.32:0/2555341738 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e7406f270 0x7f9e7406bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:47.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.074+0000 7f9e6be00640 1 -- 172.21.15.32:0/2555341738 >> v2:172.21.15.144:3300/0 conn(0x7f9e7415b7d0 msgr2=0x7f9e7406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:47.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.074+0000 7f9e6be00640 1 --2- 172.21.15.32:0/2555341738 >> v2:172.21.15.144:3300/0 conn(0x7f9e7415b7d0 0x7f9e7406e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:47.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.074+0000 7f9e6be00640 1 -- 172.21.15.32:0/2555341738 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7406e390 con 0x7f9e7406f7c0 2024-04-17T09:53:47.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.074+0000 7f9e6be00640 1 --2- 172.21.15.32:0/2555341738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e7406f7c0 0x7f9e7406ba30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9e740d1530 tx=0x7f9e5c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59147d313bfde6c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:47.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.074+0000 7f9e71e00640 1 -- 172.21.15.32:0/2555341738 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c0a7070 con 0x7f9e7406f7c0 2024-04-17T09:53:47.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.074+0000 7f9e71e00640 1 -- 172.21.15.32:0/2555341738 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e5c0b0070 con 0x7f9e7406f7c0 2024-04-17T09:53:47.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.074+0000 7f9e71e00640 1 -- 172.21.15.32:0/2555341738 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c0ab890 con 0x7f9e7406f7c0 2024-04-17T09:53:47.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.074+0000 7f9e79400640 1 -- 172.21.15.32:0/2555341738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e7406f7c0 msgr2=0x7f9e7406ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:47.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.074+0000 7f9e79400640 1 --2- 172.21.15.32:0/2555341738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e7406f7c0 0x7f9e7406ba30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9e740d1530 tx=0x7f9e5c01c490 comp rx=0 tx=0).stop 2024-04-17T09:53:47.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.074+0000 7f9e79400640 1 -- 172.21.15.32:0/2555341738 shutdown_connections 2024-04-17T09:53:47.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.074+0000 7f9e79400640 1 --2- 172.21.15.32:0/2555341738 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e7406f270 0x7f9e7406bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:47.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.074+0000 7f9e79400640 1 --2- 172.21.15.32:0/2555341738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e7406f7c0 0x7f9e7406ba30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:47.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.074+0000 7f9e79400640 1 --2- 172.21.15.32:0/2555341738 >> v2:172.21.15.144:3300/0 conn(0x7f9e7415b7d0 0x7f9e7406e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:47.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.074+0000 7f9e79400640 1 -- 172.21.15.32:0/2555341738 >> 172.21.15.32:0/2555341738 conn(0x7f9e740b2120 msgr2=0x7f9e740b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:47.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e79400640 1 -- 172.21.15.32:0/2555341738 shutdown_connections 2024-04-17T09:53:47.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e79400640 1 -- 172.21.15.32:0/2555341738 wait complete. 2024-04-17T09:53:47.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e79400640 1 Processor -- start 2024-04-17T09:53:47.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e79400640 1 -- start start 2024-04-17T09:53:47.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e79400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e7406f270 0x7f9e7414d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:47.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e79400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e7406f7c0 0x7f9e74147550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e79400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e7415b7d0 0x7f9e740ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e79400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9e740cee70 con 0x7f9e7406f7c0 2024-04-17T09:53:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e79400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9e740cecf0 con 0x7f9e7406f270 2024-04-17T09:53:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e79400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9e740b3420 con 0x7f9e7415b7d0 2024-04-17T09:53:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e6be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e7406f7c0 0x7f9e74147550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e6be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e7406f7c0 0x7f9e74147550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39188/0 (socket says 172.21.15.32:39188) 2024-04-17T09:53:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e6be00640 1 -- 172.21.15.32:0/2561179873 learned_addr learned my addr 172.21.15.32:0/2561179873 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e72800640 1 --2- 172.21.15.32:0/2561179873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e7406f270 0x7f9e7414d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e73200640 1 --2- 172.21.15.32:0/2561179873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e7415b7d0 0x7f9e740ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e6be00640 1 -- 172.21.15.32:0/2561179873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e7415b7d0 msgr2=0x7f9e740ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e6be00640 1 --2- 172.21.15.32:0/2561179873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e7415b7d0 0x7f9e740ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e6be00640 1 -- 172.21.15.32:0/2561179873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e7406f270 msgr2=0x7f9e7414d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e6be00640 1 --2- 172.21.15.32:0/2561179873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e7406f270 0x7f9e7414d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e6be00640 1 -- 172.21.15.32:0/2561179873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e740adfd0 con 0x7f9e7406f7c0 2024-04-17T09:53:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e73200640 1 --2- 172.21.15.32:0/2561179873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e7415b7d0 0x7f9e740ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e6be00640 1 --2- 172.21.15.32:0/2561179873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e7406f7c0 0x7f9e74147550 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c01c970 tx=0x7f9e5c01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e6aa00640 1 -- 172.21.15.32:0/2561179873 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c0be5f0 con 0x7f9e7406f7c0 2024-04-17T09:53:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e6aa00640 1 -- 172.21.15.32:0/2561179873 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e5c0be7b0 con 0x7f9e7406f7c0 2024-04-17T09:53:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e79400640 1 -- 172.21.15.32:0/2561179873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e740ae2b0 con 0x7f9e7406f7c0 2024-04-17T09:53:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.078+0000 7f9e6aa00640 1 -- 172.21.15.32:0/2561179873 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c0beba0 con 0x7f9e7406f7c0 2024-04-17T09:53:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.082+0000 7f9e79400640 1 -- 172.21.15.32:0/2561179873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e740ae870 con 0x7f9e7406f7c0 2024-04-17T09:53:47.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.082+0000 7f9e79400640 1 -- 172.21.15.32:0/2561179873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e74068a90 con 0x7f9e7406f7c0 2024-04-17T09:53:47.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.082+0000 7f9e6aa00640 1 -- 172.21.15.32:0/2561179873 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9e5c0c7440 con 0x7f9e7406f7c0 2024-04-17T09:53:47.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.082+0000 7f9e6aa00640 1 --2- 172.21.15.32:0/2561179873 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9e50068380 0x7f9e5006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:47.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.082+0000 7f9e6aa00640 1 -- 172.21.15.32:0/2561179873 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9e5c121510 con 0x7f9e7406f7c0 2024-04-17T09:53:47.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.082+0000 7f9e72800640 1 --2- 172.21.15.32:0/2561179873 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9e50068380 0x7f9e5006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:47.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.082+0000 7f9e72800640 1 --2- 172.21.15.32:0/2561179873 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9e50068380 0x7f9e5006a840 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9e7406e080 tx=0x7f9e64008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:47.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.086+0000 7f9e6aa00640 1 -- 172.21.15.32:0/2561179873 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f9e5c016110 con 0x7f9e7406f7c0 2024-04-17T09:53:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.254+0000 7f9e79400640 1 -- 172.21.15.32:0/2561179873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f9e740afe30 con 0x7f9e7406f7c0 2024-04-17T09:53:47.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.258+0000 7f9e6aa00640 1 -- 172.21.15.32:0/2561179873 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f9e5c0f1590 con 0x7f9e7406f7c0 2024-04-17T09:53:47.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.258+0000 7f9e79400640 1 -- 172.21.15.32:0/2561179873 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9e50068380 msgr2=0x7f9e5006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:47.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.258+0000 7f9e79400640 1 --2- 172.21.15.32:0/2561179873 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9e50068380 0x7f9e5006a840 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9e7406e080 tx=0x7f9e64008040 comp rx=0 tx=0).stop 2024-04-17T09:53:47.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.258+0000 7f9e79400640 1 -- 172.21.15.32:0/2561179873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e7406f7c0 msgr2=0x7f9e74147550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:47.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.258+0000 7f9e79400640 1 --2- 172.21.15.32:0/2561179873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e7406f7c0 0x7f9e74147550 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c01c970 tx=0x7f9e5c01acd0 comp rx=0 tx=0).stop 2024-04-17T09:53:47.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.258+0000 7f9e79400640 1 -- 172.21.15.32:0/2561179873 shutdown_connections 2024-04-17T09:53:47.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.258+0000 7f9e79400640 1 --2- 172.21.15.32:0/2561179873 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9e50068380 0x7f9e5006a840 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:47.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.258+0000 7f9e79400640 1 --2- 172.21.15.32:0/2561179873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e7415b7d0 0x7f9e740ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:47.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.258+0000 7f9e79400640 1 --2- 172.21.15.32:0/2561179873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e7406f7c0 0x7f9e74147550 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:47.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.258+0000 7f9e79400640 1 --2- 172.21.15.32:0/2561179873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e7406f270 0x7f9e7414d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:47.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.258+0000 7f9e79400640 1 -- 172.21.15.32:0/2561179873 >> 172.21.15.32:0/2561179873 conn(0x7f9e740b2120 msgr2=0x7f9e740b4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:47.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.258+0000 7f9e79400640 1 -- 172.21.15.32:0/2561179873 shutdown_connections 2024-04-17T09:53:47.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.258+0000 7f9e79400640 1 -- 172.21.15.32:0/2561179873 wait complete. 2024-04-17T09:53:47.278 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T09:53:47.279 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T09:53:47.363 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T09:53:47.426 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T09:53:47.426 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:53:47.426 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:53:47.435 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:53:47.435 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:53:47.484 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:53:47.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.666+0000 7fbc0da00640 1 Processor -- start 2024-04-17T09:53:47.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.666+0000 7fbc0da00640 1 -- start start 2024-04-17T09:53:47.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.666+0000 7fbc0da00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbc0815c5c0 0x7fbc080cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:47.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.666+0000 7fbc0da00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc080d05e0 con 0x7fbc0809b100 2024-04-17T09:53:47.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.666+0000 7fbc0da00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc080d08c0 con 0x7fbc080d00b0 2024-04-17T09:53:47.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.666+0000 7fbc0da00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fbc080d11f0 con 0x7fbc0815c5c0 2024-04-17T09:53:47.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.666+0000 7fbc07400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbc0809b100 0x7fbc080b31b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44378/0 (socket says 172.21.15.32:44378) 2024-04-17T09:53:47.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.666+0000 7fbc07400640 1 -- 172.21.15.32:0/568943007 learned_addr learned my addr 172.21.15.32:0/568943007 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:47.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.666+0000 7fbc06a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbc0815c5c0 0x7fbc080cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:47.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.666+0000 7fbc06000640 1 -- 172.21.15.32:0/568943007 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4152838077 0 0) 0x7fbc080d05e0 con 0x7fbc0809b100 2024-04-17T09:53:47.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.666+0000 7fbc06000640 1 -- 172.21.15.32:0/568943007 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbec003530 con 0x7fbc0809b100 2024-04-17T09:53:47.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.666+0000 7fbc06a00640 1 -- 172.21.15.32:0/568943007 >> v1:172.21.15.144:6789/0 conn(0x7fbc080d00b0 legacy=0x7fbc0809a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:47.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.666+0000 7fbc06a00640 1 -- 172.21.15.32:0/568943007 >> v1:172.21.15.32:6789/0 conn(0x7fbc0809b100 legacy=0x7fbc080b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:47.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.666+0000 7fbc06a00640 1 -- 172.21.15.32:0/568943007 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc080d0ba0 con 0x7fbc0815c5c0 2024-04-17T09:53:47.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.670+0000 7fbc06a00640 1 --2- 172.21.15.32:0/568943007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc0815c5c0 0x7fbc080cde50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80097c0 tx=0x7fbbf801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b82c1109c9948da server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:47.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.670+0000 7fbc06000640 1 -- 172.21.15.32:0/568943007 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf80b5070 con 0x7fbc0815c5c0 2024-04-17T09:53:47.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.670+0000 7fbc06000640 1 -- 172.21.15.32:0/568943007 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbf80a8990 con 0x7fbc0815c5c0 2024-04-17T09:53:47.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.670+0000 7fbc06000640 1 -- 172.21.15.32:0/568943007 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf80a8d60 con 0x7fbc0815c5c0 2024-04-17T09:53:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.670+0000 7fbc0da00640 1 -- 172.21.15.32:0/568943007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc0815c5c0 msgr2=0x7fbc080cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.670+0000 7fbc0da00640 1 --2- 172.21.15.32:0/568943007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc0815c5c0 0x7fbc080cde50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80097c0 tx=0x7fbbf801ec90 comp rx=0 tx=0).stop 2024-04-17T09:53:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.670+0000 7fbc0da00640 1 -- 172.21.15.32:0/568943007 shutdown_connections 2024-04-17T09:53:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.670+0000 7fbc0da00640 1 --2- 172.21.15.32:0/568943007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc0815c5c0 0x7fbc080cde50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.670+0000 7fbc0da00640 1 -- 172.21.15.32:0/568943007 >> 172.21.15.32:0/568943007 conn(0x7fbc080b3fd0 msgr2=0x7fbc080b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.670+0000 7fbc0da00640 1 -- 172.21.15.32:0/568943007 shutdown_connections 2024-04-17T09:53:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.670+0000 7fbc0da00640 1 -- 172.21.15.32:0/568943007 wait complete. 2024-04-17T09:53:47.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.670+0000 7fbc0da00640 1 Processor -- start 2024-04-17T09:53:47.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbc0da00640 1 -- start start 2024-04-17T09:53:47.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbc0da00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc0809b100 0x7fbc080c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:47.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbc0da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc080d00b0 0x7fbc081f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:47.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbc0da00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc0815c5c0 0x7fbc080c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:47.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbc0da00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbc080ce510 con 0x7fbc080d00b0 2024-04-17T09:53:47.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbc0da00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc080ce390 con 0x7fbc0815c5c0 2024-04-17T09:53:47.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbc0da00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbc080b6340 con 0x7fbc0809b100 2024-04-17T09:53:47.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbbfe000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc080d00b0 0x7fbc081f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:47.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbbfe000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc080d00b0 0x7fbc081f1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39202/0 (socket says 172.21.15.32:39202) 2024-04-17T09:53:47.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbbfe000640 1 -- 172.21.15.32:0/2346885880 learned_addr learned my addr 172.21.15.32:0/2346885880 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:47.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbc07400640 1 --2- 172.21.15.32:0/2346885880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc0815c5c0 0x7fbc080c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:47.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbc06a00640 1 --2- 172.21.15.32:0/2346885880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc0809b100 0x7fbc080c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:47.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbbfe000640 1 -- 172.21.15.32:0/2346885880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc0809b100 msgr2=0x7fbc080c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:47.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbbfe000640 1 --2- 172.21.15.32:0/2346885880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc0809b100 0x7fbc080c7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:47.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbbfe000640 1 -- 172.21.15.32:0/2346885880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc0815c5c0 msgr2=0x7fbc080c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:47.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbbfe000640 1 --2- 172.21.15.32:0/2346885880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc0815c5c0 0x7fbc080c6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:47.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbbfe000640 1 -- 172.21.15.32:0/2346885880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc080c80b0 con 0x7fbc080d00b0 2024-04-17T09:53:47.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbbfe000640 1 --2- 172.21.15.32:0/2346885880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc080d00b0 0x7fbc081f1d30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000c420 tx=0x7fbbf000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:47.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbbff400640 1 -- 172.21.15.32:0/2346885880 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf00052d0 con 0x7fbc080d00b0 2024-04-17T09:53:47.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbbff400640 1 -- 172.21.15.32:0/2346885880 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbf0005470 con 0x7fbc080d00b0 2024-04-17T09:53:47.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbc0da00640 1 -- 172.21.15.32:0/2346885880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc080c8390 con 0x7fbc080d00b0 2024-04-17T09:53:47.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbbff400640 1 -- 172.21.15.32:0/2346885880 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0007260 con 0x7fbc080d00b0 2024-04-17T09:53:47.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbc0da00640 1 -- 172.21.15.32:0/2346885880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc080c8950 con 0x7fbc080d00b0 2024-04-17T09:53:47.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.674+0000 7fbc0da00640 1 -- 172.21.15.32:0/2346885880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc080ca8d0 con 0x7fbc080d00b0 2024-04-17T09:53:47.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.678+0000 7fbbff400640 1 -- 172.21.15.32:0/2346885880 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbbf0007400 con 0x7fbc080d00b0 2024-04-17T09:53:47.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.678+0000 7fbbff400640 1 --2- 172.21.15.32:0/2346885880 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbbe4068380 0x7fbbe406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:47.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.678+0000 7fbbff400640 1 -- 172.21.15.32:0/2346885880 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbbf0085e40 con 0x7fbc080d00b0 2024-04-17T09:53:47.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.678+0000 7fbc06a00640 1 --2- 172.21.15.32:0/2346885880 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbbe4068380 0x7fbbe406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:47.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.678+0000 7fbc06a00640 1 --2- 172.21.15.32:0/2346885880 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbbe4068380 0x7fbbe406a840 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80b2000 tx=0x7fbbf8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:47.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.682+0000 7fbbff400640 1 -- 172.21.15.32:0/2346885880 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fbc080ca8d0 con 0x7fbc080d00b0 2024-04-17T09:53:47.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.850+0000 7fbc0da00640 1 -- 172.21.15.32:0/2346885880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc08068a90 con 0x7fbc080d00b0 2024-04-17T09:53:47.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.850+0000 7fbbff400640 1 -- 172.21.15.32:0/2346885880 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7fbbf0055da0 con 0x7fbc080d00b0 2024-04-17T09:53:47.851 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:47.851 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:14.779535+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:53:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.850+0000 7fbc0da00640 1 -- 172.21.15.32:0/2346885880 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbbe4068380 msgr2=0x7fbbe406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.850+0000 7fbc0da00640 1 --2- 172.21.15.32:0/2346885880 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbbe4068380 0x7fbbe406a840 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80b2000 tx=0x7fbbf8008040 comp rx=0 tx=0).stop 2024-04-17T09:53:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.850+0000 7fbc0da00640 1 -- 172.21.15.32:0/2346885880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc080d00b0 msgr2=0x7fbc081f1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.850+0000 7fbc0da00640 1 --2- 172.21.15.32:0/2346885880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc080d00b0 0x7fbc081f1d30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000c420 tx=0x7fbbf000c8e0 comp rx=0 tx=0).stop 2024-04-17T09:53:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.850+0000 7fbc0da00640 1 -- 172.21.15.32:0/2346885880 shutdown_connections 2024-04-17T09:53:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.850+0000 7fbc0da00640 1 --2- 172.21.15.32:0/2346885880 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbbe4068380 0x7fbbe406a840 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.850+0000 7fbc0da00640 1 --2- 172.21.15.32:0/2346885880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc0815c5c0 0x7fbc080c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.850+0000 7fbc0da00640 1 --2- 172.21.15.32:0/2346885880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc080d00b0 0x7fbc081f1d30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.850+0000 7fbc0da00640 1 --2- 172.21.15.32:0/2346885880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc0809b100 0x7fbc080c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.850+0000 7fbc0da00640 1 -- 172.21.15.32:0/2346885880 >> 172.21.15.32:0/2346885880 conn(0x7fbc080b3fd0 msgr2=0x7fbc080b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.854+0000 7fbc0da00640 1 -- 172.21.15.32:0/2346885880 shutdown_connections 2024-04-17T09:53:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:47.854+0000 7fbc0da00640 1 -- 172.21.15.32:0/2346885880 wait complete. 2024-04-17T09:53:47.873 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:53:11.198098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:53:12.118982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:53:47.873 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:53:48.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.018+0000 7feffe600640 1 Processor -- start 2024-04-17T09:53:48.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.018+0000 7feffe600640 1 -- start start 2024-04-17T09:53:48.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.018+0000 7feffe600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7feff806f7c0 0x7feff806ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:48.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.018+0000 7feffe600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feff806f270 0x7feff806bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:48.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.018+0000 7feffe600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feff80cc940 con 0x7feff815b7d0 2024-04-17T09:53:48.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.018+0000 7feffe600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7feff80ccc20 con 0x7feff806f270 2024-04-17T09:53:48.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.018+0000 7feffe600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7feff806e0b0 con 0x7feff806f7c0 2024-04-17T09:53:48.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.018+0000 7feff7400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7feff815b7d0 0x7feff806e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44386/0 (socket says 172.21.15.32:44386) 2024-04-17T09:53:48.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.018+0000 7feff7400640 1 -- 172.21.15.32:0/1969271876 learned_addr learned my addr 172.21.15.32:0/1969271876 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:48.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.018+0000 7feff7e00640 1 --2- 172.21.15.32:0/1969271876 >> v2:172.21.15.32:3300/0 conn(0x7feff806f270 0x7feff806bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:48.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.018+0000 7fefeea00640 1 --2- 172.21.15.32:0/1969271876 >> v2:172.21.15.144:3300/0 conn(0x7feff806f7c0 0x7feff806ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:48.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.018+0000 7feff6a00640 1 -- 172.21.15.32:0/1969271876 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 364743265 0 0) 0x7feff80cc940 con 0x7feff815b7d0 2024-04-17T09:53:48.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.018+0000 7feff7e00640 1 -- 172.21.15.32:0/1969271876 >> v2:172.21.15.144:3300/0 conn(0x7feff806f7c0 msgr2=0x7feff806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:48.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.018+0000 7feff7e00640 1 --2- 172.21.15.32:0/1969271876 >> v2:172.21.15.144:3300/0 conn(0x7feff806f7c0 0x7feff806ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.018+0000 7feff7e00640 1 -- 172.21.15.32:0/1969271876 >> v1:172.21.15.32:6789/0 conn(0x7feff815b7d0 legacy=0x7feff806e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:48.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.018+0000 7feff7e00640 1 -- 172.21.15.32:0/1969271876 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff806e390 con 0x7feff806f270 2024-04-17T09:53:48.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.022+0000 7feff7e00640 1 --2- 172.21.15.32:0/1969271876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feff806f270 0x7feff806bf70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7feff80b58f0 tx=0x7feff001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15bd4181213a4a3d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:48.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.022+0000 7feff6a00640 1 -- 172.21.15.32:0/1969271876 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff00b2070 con 0x7feff806f270 2024-04-17T09:53:48.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.022+0000 7feff6a00640 1 -- 172.21.15.32:0/1969271876 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff00a5870 con 0x7feff806f270 2024-04-17T09:53:48.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.022+0000 7feff6a00640 1 -- 172.21.15.32:0/1969271876 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff00a5c40 con 0x7feff806f270 2024-04-17T09:53:48.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.022+0000 7feffe600640 1 -- 172.21.15.32:0/1969271876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feff806f270 msgr2=0x7feff806bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:48.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.022+0000 7feffe600640 1 --2- 172.21.15.32:0/1969271876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feff806f270 0x7feff806bf70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7feff80b58f0 tx=0x7feff001c8a0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.022+0000 7feffe600640 1 -- 172.21.15.32:0/1969271876 shutdown_connections 2024-04-17T09:53:48.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.022+0000 7feffe600640 1 --2- 172.21.15.32:0/1969271876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feff806f270 0x7feff806bf70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.022+0000 7feffe600640 1 --2- 172.21.15.32:0/1969271876 >> v2:172.21.15.144:3300/0 conn(0x7feff806f7c0 0x7feff806ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.022+0000 7feffe600640 1 -- 172.21.15.32:0/1969271876 >> 172.21.15.32:0/1969271876 conn(0x7feff80b2120 msgr2=0x7feff80b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:48.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.022+0000 7feffe600640 1 -- 172.21.15.32:0/1969271876 shutdown_connections 2024-04-17T09:53:48.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.022+0000 7feffe600640 1 -- 172.21.15.32:0/1969271876 wait complete. 2024-04-17T09:53:48.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7feffe600640 1 Processor -- start 2024-04-17T09:53:48.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7feffe600640 1 -- start start 2024-04-17T09:53:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7feffe600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feff806f270 0x7feff8149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7feffe600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feff806f7c0 0x7feff81491f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7feffe600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feff815b7d0 0x7feff815d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7feffe600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7feff80cee30 con 0x7feff815b7d0 2024-04-17T09:53:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7feffe600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feff80cecb0 con 0x7feff806f7c0 2024-04-17T09:53:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7feffe600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7feff81682a0 con 0x7feff806f270 2024-04-17T09:53:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7feff7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feff815b7d0 0x7feff815d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7feff7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feff815b7d0 0x7feff815d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39210/0 (socket says 172.21.15.32:39210) 2024-04-17T09:53:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7feff7e00640 1 -- 172.21.15.32:0/1141877901 learned_addr learned my addr 172.21.15.32:0/1141877901 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7fefeea00640 1 --2- 172.21.15.32:0/1141877901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feff806f7c0 0x7feff81491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7feff7400640 1 --2- 172.21.15.32:0/1141877901 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feff806f270 0x7feff8149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7feff7e00640 1 -- 172.21.15.32:0/1141877901 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feff806f270 msgr2=0x7feff8149a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7feff7e00640 1 --2- 172.21.15.32:0/1141877901 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feff806f270 0x7feff8149a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7feff7e00640 1 -- 172.21.15.32:0/1141877901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feff806f7c0 msgr2=0x7feff81491f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7feff7e00640 1 --2- 172.21.15.32:0/1141877901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feff806f7c0 0x7feff81491f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7feff7e00640 1 -- 172.21.15.32:0/1141877901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff815be50 con 0x7feff815b7d0 2024-04-17T09:53:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7feff7e00640 1 --2- 172.21.15.32:0/1141877901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feff815b7d0 0x7feff815d8d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7feff001a4d0 tx=0x7feff0019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7fefefe00640 1 -- 172.21.15.32:0/1141877901 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff00b2050 con 0x7feff815b7d0 2024-04-17T09:53:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7fefefe00640 1 -- 172.21.15.32:0/1141877901 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff00be070 con 0x7feff815b7d0 2024-04-17T09:53:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7fefefe00640 1 -- 172.21.15.32:0/1141877901 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff00adc40 con 0x7feff815b7d0 2024-04-17T09:53:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7feffe600640 1 -- 172.21.15.32:0/1141877901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff815c130 con 0x7feff815b7d0 2024-04-17T09:53:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7feffe600640 1 -- 172.21.15.32:0/1141877901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff8151690 con 0x7feff815b7d0 2024-04-17T09:53:48.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.026+0000 7feffe600640 1 -- 172.21.15.32:0/1141877901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff8068a90 con 0x7feff815b7d0 2024-04-17T09:53:48.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.030+0000 7fefefe00640 1 -- 172.21.15.32:0/1141877901 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7feff00c3460 con 0x7feff815b7d0 2024-04-17T09:53:48.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.030+0000 7fefefe00640 1 --2- 172.21.15.32:0/1141877901 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fefd4068380 0x7fefd406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:48.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.030+0000 7fefefe00640 1 -- 172.21.15.32:0/1141877901 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7feff0121c40 con 0x7feff815b7d0 2024-04-17T09:53:48.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.030+0000 7feff7400640 1 --2- 172.21.15.32:0/1141877901 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fefd4068380 0x7fefd406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:48.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.030+0000 7feff7400640 1 --2- 172.21.15.32:0/1141877901 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fefd4068380 0x7fefd406a840 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fefe8000f70 tx=0x7fefe8007df0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:48.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.034+0000 7fefefe00640 1 -- 172.21.15.32:0/1141877901 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7feff0016520 con 0x7feff815b7d0 2024-04-17T09:53:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.198+0000 7feffe600640 1 -- 172.21.15.32:0/1141877901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feff8154280 con 0x7feff815b7d0 2024-04-17T09:53:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.198+0000 7fefefe00640 1 -- 172.21.15.32:0/1141877901 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7feff00f1b10 con 0x7feff815b7d0 2024-04-17T09:53:48.199 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:53:48.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.202+0000 7feffe600640 1 -- 172.21.15.32:0/1141877901 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fefd4068380 msgr2=0x7fefd406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:48.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.202+0000 7feffe600640 1 --2- 172.21.15.32:0/1141877901 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fefd4068380 0x7fefd406a840 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fefe8000f70 tx=0x7fefe8007df0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.202+0000 7feffe600640 1 -- 172.21.15.32:0/1141877901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feff815b7d0 msgr2=0x7feff815d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:48.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.202+0000 7feffe600640 1 --2- 172.21.15.32:0/1141877901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feff815b7d0 0x7feff815d8d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7feff001a4d0 tx=0x7feff0019640 comp rx=0 tx=0).stop 2024-04-17T09:53:48.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.202+0000 7feffe600640 1 -- 172.21.15.32:0/1141877901 shutdown_connections 2024-04-17T09:53:48.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.202+0000 7feffe600640 1 --2- 172.21.15.32:0/1141877901 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fefd4068380 0x7fefd406a840 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.202+0000 7feffe600640 1 --2- 172.21.15.32:0/1141877901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feff815b7d0 0x7feff815d8d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.202+0000 7feffe600640 1 --2- 172.21.15.32:0/1141877901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feff806f7c0 0x7feff81491f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.202+0000 7feffe600640 1 --2- 172.21.15.32:0/1141877901 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feff806f270 0x7feff8149a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.202+0000 7feffe600640 1 -- 172.21.15.32:0/1141877901 >> 172.21.15.32:0/1141877901 conn(0x7feff80b2120 msgr2=0x7feff8159f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:48.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.202+0000 7feffe600640 1 -- 172.21.15.32:0/1141877901 shutdown_connections 2024-04-17T09:53:48.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.202+0000 7feffe600640 1 -- 172.21.15.32:0/1141877901 wait complete. 2024-04-17T09:53:48.220 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:53:48.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.366+0000 7ff92f000640 1 Processor -- start 2024-04-17T09:53:48.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.366+0000 7ff92f000640 1 -- start start 2024-04-17T09:53:48.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.366+0000 7ff92f000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff92815b7d0 0x7ff92806e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:48.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.366+0000 7ff92f000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff92806f7c0 0x7ff92806ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:48.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.366+0000 7ff92f000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9280cc940 con 0x7ff92806f270 2024-04-17T09:53:48.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.366+0000 7ff92f000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff9280ccc20 con 0x7ff92815b7d0 2024-04-17T09:53:48.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.366+0000 7ff92f000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff92806e0b0 con 0x7ff92806f7c0 2024-04-17T09:53:48.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.366+0000 7ff92d000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff92806f270 0x7ff9280ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44402/0 (socket says 172.21.15.32:44402) 2024-04-17T09:53:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.366+0000 7ff92d000640 1 -- 172.21.15.32:0/2092569748 learned_addr learned my addr 172.21.15.32:0/2092569748 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.366+0000 7ff927400640 1 --2- 172.21.15.32:0/2092569748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff92806f7c0 0x7ff92806ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.366+0000 7ff927e00640 1 --2- 172.21.15.32:0/2092569748 >> v2:172.21.15.144:3300/0 conn(0x7ff92815b7d0 0x7ff92806e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.366+0000 7ff926a00640 1 -- 172.21.15.32:0/2092569748 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1508017821 0 0) 0x7ff9280cc940 con 0x7ff92806f270 2024-04-17T09:53:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.366+0000 7ff926a00640 1 -- 172.21.15.32:0/2092569748 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff90c003530 con 0x7ff92806f270 2024-04-17T09:53:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.366+0000 7ff927400640 1 -- 172.21.15.32:0/2092569748 >> v2:172.21.15.144:3300/0 conn(0x7ff92815b7d0 msgr2=0x7ff92806e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.366+0000 7ff927400640 1 --2- 172.21.15.32:0/2092569748 >> v2:172.21.15.144:3300/0 conn(0x7ff92815b7d0 0x7ff92806e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.366+0000 7ff927400640 1 -- 172.21.15.32:0/2092569748 >> v1:172.21.15.32:6789/0 conn(0x7ff92806f270 legacy=0x7ff9280ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.366+0000 7ff927400640 1 -- 172.21.15.32:0/2092569748 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92806e390 con 0x7ff92806f7c0 2024-04-17T09:53:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.366+0000 7ff927e00640 1 --2- 172.21.15.32:0/2092569748 >> v2:172.21.15.144:3300/0 conn(0x7ff92815b7d0 0x7ff92806e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:53:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.370+0000 7ff927400640 1 --2- 172.21.15.32:0/2092569748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff92806f7c0 0x7ff92806ba30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff9140097c0 tx=0x7ff91401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=668a2734d600e419 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:48.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.370+0000 7ff926a00640 1 -- 172.21.15.32:0/2092569748 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9140b0070 con 0x7ff92806f7c0 2024-04-17T09:53:48.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.370+0000 7ff926a00640 1 -- 172.21.15.32:0/2092569748 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9140ab600 con 0x7ff92806f7c0 2024-04-17T09:53:48.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.370+0000 7ff926a00640 1 -- 172.21.15.32:0/2092569748 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9140ab9d0 con 0x7ff92806f7c0 2024-04-17T09:53:48.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.370+0000 7ff92f000640 1 -- 172.21.15.32:0/2092569748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff92806f7c0 msgr2=0x7ff92806ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:48.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.370+0000 7ff92f000640 1 --2- 172.21.15.32:0/2092569748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff92806f7c0 0x7ff92806ba30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff9140097c0 tx=0x7ff91401c410 comp rx=0 tx=0).stop 2024-04-17T09:53:48.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.370+0000 7ff92f000640 1 -- 172.21.15.32:0/2092569748 shutdown_connections 2024-04-17T09:53:48.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.370+0000 7ff92f000640 1 --2- 172.21.15.32:0/2092569748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff92806f7c0 0x7ff92806ba30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.370+0000 7ff92f000640 1 --2- 172.21.15.32:0/2092569748 >> v2:172.21.15.144:3300/0 conn(0x7ff92815b7d0 0x7ff92806e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.370+0000 7ff92f000640 1 -- 172.21.15.32:0/2092569748 >> 172.21.15.32:0/2092569748 conn(0x7ff9280b2120 msgr2=0x7ff9280b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:48.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.370+0000 7ff92f000640 1 -- 172.21.15.32:0/2092569748 shutdown_connections 2024-04-17T09:53:48.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.370+0000 7ff92f000640 1 -- 172.21.15.32:0/2092569748 wait complete. 2024-04-17T09:53:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff92f000640 1 Processor -- start 2024-04-17T09:53:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff92f000640 1 -- start start 2024-04-17T09:53:48.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff92f000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff92806f270 0x7ff9281455f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:48.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff92f000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff92806f7c0 0x7ff928132100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:48.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff92f000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff92815b7d0 0x7ff9281523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:48.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff92f000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff928072670 con 0x7ff92815b7d0 2024-04-17T09:53:48.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff92f000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff92806c380 con 0x7ff92806f270 2024-04-17T09:53:48.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff92f000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff9280727f0 con 0x7ff92806f7c0 2024-04-17T09:53:48.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff92d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff92815b7d0 0x7ff9281523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:48.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff92d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff92815b7d0 0x7ff9281523a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39226/0 (socket says 172.21.15.32:39226) 2024-04-17T09:53:48.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff92d000640 1 -- 172.21.15.32:0/10454943 learned_addr learned my addr 172.21.15.32:0/10454943 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:48.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff927400640 1 --2- 172.21.15.32:0/10454943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff92806f7c0 0x7ff928132100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:48.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff927e00640 1 --2- 172.21.15.32:0/10454943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff92806f270 0x7ff9281455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff92d000640 1 -- 172.21.15.32:0/10454943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff92806f7c0 msgr2=0x7ff928132100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff92d000640 1 --2- 172.21.15.32:0/10454943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff92806f7c0 0x7ff928132100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff92d000640 1 -- 172.21.15.32:0/10454943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff92806f270 msgr2=0x7ff9281455f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff92d000640 1 --2- 172.21.15.32:0/10454943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff92806f270 0x7ff9281455f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff92d000640 1 -- 172.21.15.32:0/10454943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff928132640 con 0x7ff92815b7d0 2024-04-17T09:53:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff927e00640 1 --2- 172.21.15.32:0/10454943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff92806f270 0x7ff9281455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff92d000640 1 --2- 172.21.15.32:0/10454943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff92815b7d0 0x7ff9281523a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff92000cb40 tx=0x7ff92000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff91fe00640 1 -- 172.21.15.32:0/10454943 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920017070 con 0x7ff92815b7d0 2024-04-17T09:53:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff91fe00640 1 -- 172.21.15.32:0/10454943 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff92000aca0 con 0x7ff92815b7d0 2024-04-17T09:53:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff91fe00640 1 -- 172.21.15.32:0/10454943 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920012620 con 0x7ff92815b7d0 2024-04-17T09:53:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff92f000640 1 -- 172.21.15.32:0/10454943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff928156620 con 0x7ff92815b7d0 2024-04-17T09:53:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff92f000640 1 -- 172.21.15.32:0/10454943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff928155d80 con 0x7ff92815b7d0 2024-04-17T09:53:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.374+0000 7ff92f000640 1 -- 172.21.15.32:0/10454943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff928068a90 con 0x7ff92815b7d0 2024-04-17T09:53:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.378+0000 7ff91fe00640 1 -- 172.21.15.32:0/10454943 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff92001a070 con 0x7ff92815b7d0 2024-04-17T09:53:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.378+0000 7ff91fe00640 1 --2- 172.21.15.32:0/10454943 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff908068380 0x7ff90806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:48.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.378+0000 7ff91fe00640 1 -- 172.21.15.32:0/10454943 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff920086890 con 0x7ff92815b7d0 2024-04-17T09:53:48.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.378+0000 7ff927e00640 1 --2- 172.21.15.32:0/10454943 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff908068380 0x7ff90806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:48.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.378+0000 7ff927e00640 1 --2- 172.21.15.32:0/10454943 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff908068380 0x7ff90806a840 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff91800bce0 tx=0x7ff91800b3a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.382+0000 7ff91fe00640 1 -- 172.21.15.32:0/10454943 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff920056db0 con 0x7ff92815b7d0 2024-04-17T09:53:48.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.546+0000 7ff92f000640 1 -- 172.21.15.32:0/10454943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff92814eeb0 con 0x7ff92815b7d0 2024-04-17T09:53:48.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.546+0000 7ff91fe00640 1 -- 172.21.15.32:0/10454943 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7ff9200566b0 con 0x7ff92815b7d0 2024-04-17T09:53:48.548 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:53:48.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.550+0000 7ff92f000640 1 -- 172.21.15.32:0/10454943 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff908068380 msgr2=0x7ff90806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.550+0000 7ff92f000640 1 --2- 172.21.15.32:0/10454943 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff908068380 0x7ff90806a840 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff91800bce0 tx=0x7ff91800b3a0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.550+0000 7ff92f000640 1 -- 172.21.15.32:0/10454943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff92815b7d0 msgr2=0x7ff9281523a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.550+0000 7ff92f000640 1 --2- 172.21.15.32:0/10454943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff92815b7d0 0x7ff9281523a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff92000cb40 tx=0x7ff92000a060 comp rx=0 tx=0).stop 2024-04-17T09:53:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.550+0000 7ff92f000640 1 -- 172.21.15.32:0/10454943 shutdown_connections 2024-04-17T09:53:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.550+0000 7ff92f000640 1 --2- 172.21.15.32:0/10454943 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff908068380 0x7ff90806a840 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.550+0000 7ff92f000640 1 --2- 172.21.15.32:0/10454943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff92806f7c0 0x7ff928132100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.550+0000 7ff92f000640 1 --2- 172.21.15.32:0/10454943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff92815b7d0 0x7ff9281523a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.550+0000 7ff92f000640 1 --2- 172.21.15.32:0/10454943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff92806f270 0x7ff9281455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.550+0000 7ff92f000640 1 -- 172.21.15.32:0/10454943 >> 172.21.15.32:0/10454943 conn(0x7ff9280b2120 msgr2=0x7ff9280b3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.550+0000 7ff92f000640 1 -- 172.21.15.32:0/10454943 shutdown_connections 2024-04-17T09:53:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.550+0000 7ff92f000640 1 -- 172.21.15.32:0/10454943 wait complete. 2024-04-17T09:53:48.568 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:53:48.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.710+0000 7fdb08a00640 1 Processor -- start 2024-04-17T09:53:48.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.710+0000 7fdb08a00640 1 -- start start 2024-04-17T09:53:48.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.710+0000 7fdb08a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdb040cf370 0x7fdb040cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:48.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.710+0000 7fdb08a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb04071e60 0x7fdb04072270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:48.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.710+0000 7fdb08a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb040d0130 con 0x7fdb0406ba30 2024-04-17T09:53:48.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.710+0000 7fdb08a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fdb0415d090 con 0x7fdb040cf370 2024-04-17T09:53:48.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.710+0000 7fdb08a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdb0415d370 con 0x7fdb04071e60 2024-04-17T09:53:48.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.714+0000 7fdb02800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb04071e60 0x7fdb04072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:48.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.714+0000 7fdaf9400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdb040cf370 0x7fdb040cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:48.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.714+0000 7fdb01e00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fdb0406ba30 0x7fdb0406be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:52170/0 (socket says 172.21.15.32:52170) 2024-04-17T09:53:48.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.714+0000 7fdaf9400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdb040cf370 0x7fdb040cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:49638/0 (socket says 172.21.15.32:49638) 2024-04-17T09:53:48.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.714+0000 7fdb02800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb04071e60 0x7fdb04072270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:49566/0 (socket says 172.21.15.32:49566) 2024-04-17T09:53:48.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.714+0000 7fdb01e00640 1 -- 172.21.15.32:0/2528522898 learned_addr learned my addr 172.21.15.32:0/2528522898 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:48.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.714+0000 7fdaf9400640 1 -- 172.21.15.32:0/2528522898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb04071e60 msgr2=0x7fdb04072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:48.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.714+0000 7fdaf9400640 1 --2- 172.21.15.32:0/2528522898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb04071e60 0x7fdb04072270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.714+0000 7fdaf9400640 1 -- 172.21.15.32:0/2528522898 >> v1:172.21.15.144:6789/0 conn(0x7fdb0406ba30 legacy=0x7fdb0406be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:48.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.714+0000 7fdaf9400640 1 -- 172.21.15.32:0/2528522898 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0415d650 con 0x7fdb040cf370 2024-04-17T09:53:48.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.714+0000 7fdb02800640 1 --2- 172.21.15.32:0/2528522898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb04071e60 0x7fdb04072270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:53:48.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.714+0000 7fdb01400640 1 -- 172.21.15.32:0/2528522898 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 954365031 0 0) 0x7fdb040d0130 con 0x7fdb0406ba30 2024-04-17T09:53:48.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.718+0000 7fdaf9400640 1 --2- 172.21.15.32:0/2528522898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb040cf370 0x7fdb040cf7e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0097c0 tx=0x7fdaec01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f00d3e97aadf66a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:48.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.718+0000 7fdb01400640 1 -- 172.21.15.32:0/2528522898 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec0aa620 con 0x7fdb040cf370 2024-04-17T09:53:48.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.718+0000 7fdb01400640 1 -- 172.21.15.32:0/2528522898 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaec0af070 con 0x7fdb040cf370 2024-04-17T09:53:48.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.718+0000 7fdb01400640 1 -- 172.21.15.32:0/2528522898 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec0aa9e0 con 0x7fdb040cf370 2024-04-17T09:53:48.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.718+0000 7fdb08a00640 1 -- 172.21.15.32:0/2528522898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb040cf370 msgr2=0x7fdb040cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:48.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.718+0000 7fdb08a00640 1 --2- 172.21.15.32:0/2528522898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb040cf370 0x7fdb040cf7e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0097c0 tx=0x7fdaec01c4a0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.718+0000 7fdb08a00640 1 -- 172.21.15.32:0/2528522898 shutdown_connections 2024-04-17T09:53:48.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.718+0000 7fdb08a00640 1 --2- 172.21.15.32:0/2528522898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb04071e60 0x7fdb04072270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.718+0000 7fdb08a00640 1 --2- 172.21.15.32:0/2528522898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb040cf370 0x7fdb040cf7e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.718+0000 7fdb08a00640 1 -- 172.21.15.32:0/2528522898 >> 172.21.15.32:0/2528522898 conn(0x7fdb040b2120 msgr2=0x7fdb040b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:48.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.718+0000 7fdb08a00640 1 -- 172.21.15.32:0/2528522898 shutdown_connections 2024-04-17T09:53:48.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.718+0000 7fdb08a00640 1 -- 172.21.15.32:0/2528522898 wait complete. 2024-04-17T09:53:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.718+0000 7fdb08a00640 1 Processor -- start 2024-04-17T09:53:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.718+0000 7fdb08a00640 1 -- start start 2024-04-17T09:53:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.718+0000 7fdb08a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0406ba30 0x7fdb04159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.718+0000 7fdb08a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb04071e60 0x7fdb04141f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.718+0000 7fdb08a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb040cf370 0x7fdb04142450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.718+0000 7fdb08a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdb04146230 con 0x7fdb04071e60 2024-04-17T09:53:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.718+0000 7fdb08a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0415dd80 con 0x7fdb0406ba30 2024-04-17T09:53:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.718+0000 7fdb08a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdb041463b0 con 0x7fdb040cf370 2024-04-17T09:53:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.722+0000 7fdaf9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb04071e60 0x7fdb04141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.722+0000 7fdaf9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb04071e60 0x7fdb04141f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39232/0 (socket says 172.21.15.32:39232) 2024-04-17T09:53:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.722+0000 7fdaf9400640 1 -- 172.21.15.32:0/1660433864 learned_addr learned my addr 172.21.15.32:0/1660433864 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.722+0000 7fdb02800640 1 --2- 172.21.15.32:0/1660433864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb040cf370 0x7fdb04142450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.722+0000 7fdb01e00640 1 --2- 172.21.15.32:0/1660433864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0406ba30 0x7fdb04159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.722+0000 7fdaf9400640 1 -- 172.21.15.32:0/1660433864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb040cf370 msgr2=0x7fdb04142450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.722+0000 7fdaf9400640 1 --2- 172.21.15.32:0/1660433864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb040cf370 0x7fdb04142450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.722+0000 7fdaf9400640 1 -- 172.21.15.32:0/1660433864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0406ba30 msgr2=0x7fdb04159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.722+0000 7fdaf9400640 1 --2- 172.21.15.32:0/1660433864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0406ba30 0x7fdb04159860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.722+0000 7fdaf9400640 1 -- 172.21.15.32:0/1660433864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb040ade20 con 0x7fdb04071e60 2024-04-17T09:53:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.722+0000 7fdaf9400640 1 --2- 172.21.15.32:0/1660433864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb04071e60 0x7fdb04141f10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdaec01ab50 tx=0x7fdaec019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.722+0000 7fdafaa00640 1 -- 172.21.15.32:0/1660433864 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec0aac10 con 0x7fdb04071e60 2024-04-17T09:53:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.722+0000 7fdafaa00640 1 -- 172.21.15.32:0/1660433864 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaec0aadb0 con 0x7fdb04071e60 2024-04-17T09:53:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.722+0000 7fdafaa00640 1 -- 172.21.15.32:0/1660433864 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec0be690 con 0x7fdb04071e60 2024-04-17T09:53:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.722+0000 7fdb08a00640 1 -- 172.21.15.32:0/1660433864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb040ae100 con 0x7fdb04071e60 2024-04-17T09:53:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.722+0000 7fdb08a00640 1 -- 172.21.15.32:0/1660433864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb040ae6c0 con 0x7fdb04071e60 2024-04-17T09:53:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.722+0000 7fdb08a00640 1 -- 172.21.15.32:0/1660433864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb04068a90 con 0x7fdb04071e60 2024-04-17T09:53:48.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.722+0000 7fdafaa00640 1 -- 172.21.15.32:0/1660433864 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdaec0be830 con 0x7fdb04071e60 2024-04-17T09:53:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.726+0000 7fdafaa00640 1 --2- 172.21.15.32:0/1660433864 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdae0068380 0x7fdae006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.726+0000 7fdafaa00640 1 -- 172.21.15.32:0/1660433864 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdaec120cb0 con 0x7fdb04071e60 2024-04-17T09:53:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.726+0000 7fdb01e00640 1 --2- 172.21.15.32:0/1660433864 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdae0068380 0x7fdae006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.726+0000 7fdb01e00640 1 --2- 172.21.15.32:0/1660433864 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdae0068380 0x7fdae006a840 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4007fd0 tx=0x7fdaf4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:48.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.730+0000 7fdafaa00640 1 -- 172.21.15.32:0/1660433864 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fdaec0c5030 con 0x7fdb04071e60 2024-04-17T09:53:48.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.890+0000 7fdb08a00640 1 -- 172.21.15.32:0/1660433864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdb0406c850 con 0x7fdb04071e60 2024-04-17T09:53:48.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.890+0000 7fdafaa00640 1 -- 172.21.15.32:0/1660433864 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fdaec016090 con 0x7fdb04071e60 2024-04-17T09:53:48.893 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:53:48.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.894+0000 7fdb08a00640 1 -- 172.21.15.32:0/1660433864 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdae0068380 msgr2=0x7fdae006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:48.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.894+0000 7fdb08a00640 1 --2- 172.21.15.32:0/1660433864 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdae0068380 0x7fdae006a840 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4007fd0 tx=0x7fdaf4009040 comp rx=0 tx=0).stop 2024-04-17T09:53:48.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.894+0000 7fdb08a00640 1 -- 172.21.15.32:0/1660433864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb04071e60 msgr2=0x7fdb04141f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:48.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.894+0000 7fdb08a00640 1 --2- 172.21.15.32:0/1660433864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb04071e60 0x7fdb04141f10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdaec01ab50 tx=0x7fdaec019240 comp rx=0 tx=0).stop 2024-04-17T09:53:48.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.894+0000 7fdb08a00640 1 -- 172.21.15.32:0/1660433864 shutdown_connections 2024-04-17T09:53:48.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.894+0000 7fdb08a00640 1 --2- 172.21.15.32:0/1660433864 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdae0068380 0x7fdae006a840 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.894+0000 7fdb08a00640 1 --2- 172.21.15.32:0/1660433864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb040cf370 0x7fdb04142450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.894+0000 7fdb08a00640 1 --2- 172.21.15.32:0/1660433864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb04071e60 0x7fdb04141f10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.894+0000 7fdb08a00640 1 --2- 172.21.15.32:0/1660433864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0406ba30 0x7fdb04159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:48.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.894+0000 7fdb08a00640 1 -- 172.21.15.32:0/1660433864 >> 172.21.15.32:0/1660433864 conn(0x7fdb040b2120 msgr2=0x7fdb0406e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:48.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.894+0000 7fdb08a00640 1 -- 172.21.15.32:0/1660433864 shutdown_connections 2024-04-17T09:53:48.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:48.894+0000 7fdb08a00640 1 -- 172.21.15.32:0/1660433864 wait complete. 2024-04-17T09:53:48.913 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T09:53:49.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.054+0000 7fd988800640 1 Processor -- start 2024-04-17T09:53:49.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.054+0000 7fd988800640 1 -- start start 2024-04-17T09:53:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.054+0000 7fd988800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd980155a70 0x7fd98015f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.054+0000 7fd988800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd9800b3340 0x7fd98015c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.054+0000 7fd988800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd98015f870 con 0x7fd980155e50 2024-04-17T09:53:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.054+0000 7fd988800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd9800b42a0 con 0x7fd9800b3340 2024-04-17T09:53:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.054+0000 7fd988800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd9800d1250 con 0x7fd980155a70 2024-04-17T09:53:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.054+0000 7fd986800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd9800b3340 0x7fd98015c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.054+0000 7fd986800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd9800b3340 0x7fd98015c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39246/0 (socket says 172.21.15.32:39246) 2024-04-17T09:53:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.054+0000 7fd986800640 1 -- 172.21.15.32:0/618930713 learned_addr learned my addr 172.21.15.32:0/618930713 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:49.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.054+0000 7fd985e00640 1 --2- 172.21.15.32:0/618930713 >> v2:172.21.15.144:3300/0 conn(0x7fd980155a70 0x7fd98015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:49.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.054+0000 7fd986800640 1 -- 172.21.15.32:0/618930713 >> v2:172.21.15.144:3300/0 conn(0x7fd980155a70 msgr2=0x7fd98015f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:49.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.054+0000 7fd986800640 1 --2- 172.21.15.32:0/618930713 >> v2:172.21.15.144:3300/0 conn(0x7fd980155a70 0x7fd98015f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.054+0000 7fd986800640 1 -- 172.21.15.32:0/618930713 >> v1:172.21.15.144:6789/0 conn(0x7fd980155e50 legacy=0x7fd98015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:49.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.054+0000 7fd986800640 1 -- 172.21.15.32:0/618930713 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd98015fb50 con 0x7fd9800b3340 2024-04-17T09:53:49.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.054+0000 7fd985e00640 1 --2- 172.21.15.32:0/618930713 >> v2:172.21.15.144:3300/0 conn(0x7fd980155a70 0x7fd98015f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:49.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.058+0000 7fd986800640 1 --2- 172.21.15.32:0/618930713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9800b3340 0x7fd98015c260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009e50 tx=0x7fd97c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2403e8bc6962a71 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:49.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.058+0000 7fd984a00640 1 -- 172.21.15.32:0/618930713 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0a5670 con 0x7fd9800b3340 2024-04-17T09:53:49.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.058+0000 7fd984a00640 1 -- 172.21.15.32:0/618930713 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd97c0a5810 con 0x7fd9800b3340 2024-04-17T09:53:49.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.058+0000 7fd984a00640 1 -- 172.21.15.32:0/618930713 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0a5be0 con 0x7fd9800b3340 2024-04-17T09:53:49.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.058+0000 7fd988800640 1 -- 172.21.15.32:0/618930713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9800b3340 msgr2=0x7fd98015c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:49.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.058+0000 7fd988800640 1 --2- 172.21.15.32:0/618930713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9800b3340 0x7fd98015c260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009e50 tx=0x7fd97c01c9b0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.058+0000 7fd988800640 1 -- 172.21.15.32:0/618930713 shutdown_connections 2024-04-17T09:53:49.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.058+0000 7fd988800640 1 --2- 172.21.15.32:0/618930713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9800b3340 0x7fd98015c260 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.058+0000 7fd988800640 1 --2- 172.21.15.32:0/618930713 >> v2:172.21.15.144:3300/0 conn(0x7fd980155a70 0x7fd98015f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.058+0000 7fd988800640 1 -- 172.21.15.32:0/618930713 >> 172.21.15.32:0/618930713 conn(0x7fd98006ba30 msgr2=0x7fd98006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:49.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.058+0000 7fd988800640 1 -- 172.21.15.32:0/618930713 shutdown_connections 2024-04-17T09:53:49.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.058+0000 7fd988800640 1 -- 172.21.15.32:0/618930713 wait complete. 2024-04-17T09:53:49.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd988800640 1 Processor -- start 2024-04-17T09:53:49.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd988800640 1 -- start start 2024-04-17T09:53:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd988800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9800b3340 0x7fd9801fbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd988800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd980155a70 0x7fd980154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd988800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd980155e50 0x7fd9800cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd988800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd9800d02f0 con 0x7fd9800b3340 2024-04-17T09:53:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd988800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd9800d0170 con 0x7fd980155a70 2024-04-17T09:53:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd988800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd9800d0470 con 0x7fd980155e50 2024-04-17T09:53:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd985e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9800b3340 0x7fd9801fbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd985e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9800b3340 0x7fd9801fbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39262/0 (socket says 172.21.15.32:39262) 2024-04-17T09:53:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd985e00640 1 -- 172.21.15.32:0/647586573 learned_addr learned my addr 172.21.15.32:0/647586573 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd985400640 1 --2- 172.21.15.32:0/647586573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd980155a70 0x7fd980154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd986800640 1 --2- 172.21.15.32:0/647586573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd980155e50 0x7fd9800cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd985e00640 1 -- 172.21.15.32:0/647586573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd980155e50 msgr2=0x7fd9800cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd985e00640 1 --2- 172.21.15.32:0/647586573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd980155e50 0x7fd9800cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd985e00640 1 -- 172.21.15.32:0/647586573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd980155a70 msgr2=0x7fd980154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd985e00640 1 --2- 172.21.15.32:0/647586573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd980155a70 0x7fd980154d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd985e00640 1 -- 172.21.15.32:0/647586573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd980156260 con 0x7fd9800b3340 2024-04-17T09:53:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd985400640 1 --2- 172.21.15.32:0/647586573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd980155a70 0x7fd980154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd985e00640 1 --2- 172.21.15.32:0/647586573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9800b3340 0x7fd9801fbe40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd97400c7c0 tx=0x7fd97400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd97a000640 1 -- 172.21.15.32:0/647586573 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd974007a30 con 0x7fd9800b3340 2024-04-17T09:53:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd97a000640 1 -- 172.21.15.32:0/647586573 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd974007bd0 con 0x7fd9800b3340 2024-04-17T09:53:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd97a000640 1 -- 172.21.15.32:0/647586573 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9740113f0 con 0x7fd9800b3340 2024-04-17T09:53:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd988800640 1 -- 172.21.15.32:0/647586573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd980156540 con 0x7fd9800b3340 2024-04-17T09:53:49.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd988800640 1 -- 172.21.15.32:0/647586573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd980156b00 con 0x7fd9800b3340 2024-04-17T09:53:49.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.062+0000 7fd988800640 1 -- 172.21.15.32:0/647586573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd980068a90 con 0x7fd9800b3340 2024-04-17T09:53:49.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.066+0000 7fd97a000640 1 -- 172.21.15.32:0/647586573 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd974011590 con 0x7fd9800b3340 2024-04-17T09:53:49.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.066+0000 7fd97a000640 1 --2- 172.21.15.32:0/647586573 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd964068380 0x7fd96406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:49.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.066+0000 7fd97a000640 1 -- 172.21.15.32:0/647586573 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd974086be0 con 0x7fd9800b3340 2024-04-17T09:53:49.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.066+0000 7fd985400640 1 --2- 172.21.15.32:0/647586573 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd964068380 0x7fd96406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:49.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.066+0000 7fd985400640 1 --2- 172.21.15.32:0/647586573 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd964068380 0x7fd96406a840 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd9800b4050 tx=0x7fd970008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.070+0000 7fd97a000640 1 -- 172.21.15.32:0/647586573 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd974015020 con 0x7fd9800b3340 2024-04-17T09:53:49.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.250+0000 7fd988800640 1 -- 172.21.15.32:0/647586573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd9800b1db0 con 0x7fd9800b3340 2024-04-17T09:53:49.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.250+0000 7fd97a000640 1 -- 172.21.15.32:0/647586573 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7fd974057190 con 0x7fd9800b3340 2024-04-17T09:53:49.249 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T09:53:49.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.250+0000 7fd988800640 1 -- 172.21.15.32:0/647586573 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd964068380 msgr2=0x7fd96406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.250+0000 7fd988800640 1 --2- 172.21.15.32:0/647586573 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd964068380 0x7fd96406a840 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd9800b4050 tx=0x7fd970008040 comp rx=0 tx=0).stop 2024-04-17T09:53:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.250+0000 7fd988800640 1 -- 172.21.15.32:0/647586573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9800b3340 msgr2=0x7fd9801fbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.250+0000 7fd988800640 1 --2- 172.21.15.32:0/647586573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9800b3340 0x7fd9801fbe40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd97400c7c0 tx=0x7fd97400cc80 comp rx=0 tx=0).stop 2024-04-17T09:53:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.250+0000 7fd988800640 1 -- 172.21.15.32:0/647586573 shutdown_connections 2024-04-17T09:53:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.250+0000 7fd988800640 1 --2- 172.21.15.32:0/647586573 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd964068380 0x7fd96406a840 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.250+0000 7fd988800640 1 --2- 172.21.15.32:0/647586573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd980155e50 0x7fd9800cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.250+0000 7fd988800640 1 --2- 172.21.15.32:0/647586573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd980155a70 0x7fd980154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.250+0000 7fd988800640 1 --2- 172.21.15.32:0/647586573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9800b3340 0x7fd9801fbe40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.250+0000 7fd988800640 1 -- 172.21.15.32:0/647586573 >> 172.21.15.32:0/647586573 conn(0x7fd98006ba30 msgr2=0x7fd98015d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.250+0000 7fd988800640 1 -- 172.21.15.32:0/647586573 shutdown_connections 2024-04-17T09:53:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.250+0000 7fd988800640 1 -- 172.21.15.32:0/647586573 wait complete. 2024-04-17T09:53:49.271 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:53:49.271 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:53:49.281 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:53:49.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.462+0000 7f67fe400640 1 Processor -- start 2024-04-17T09:53:49.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.462+0000 7f67fe400640 1 -- start start 2024-04-17T09:53:49.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.462+0000 7f67fe400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67f8144e90 0x7f67f80b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:49.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.462+0000 7f67fe400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f67f80b4de0 0x7f67f815baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:49.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.462+0000 7f67fe400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67f815bfe0 0x7f67f806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:49.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.462+0000 7f67fe400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f67f815c430 con 0x7f67f815bfe0 2024-04-17T09:53:49.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.462+0000 7f67fe400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f67f8145270 con 0x7f67f80b4de0 2024-04-17T09:53:49.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.462+0000 7f67fe400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f67f8072bb0 con 0x7f67f8144e90 2024-04-17T09:53:49.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.462+0000 7f67f7e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67f815bfe0 0x7f67f806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:49.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.466+0000 7f67f7e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67f815bfe0 0x7f67f806ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39276/0 (socket says 172.21.15.32:39276) 2024-04-17T09:53:49.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.466+0000 7f67f7e00640 1 -- 172.21.15.32:0/2172052713 learned_addr learned my addr 172.21.15.32:0/2172052713 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:49.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.466+0000 7f67f7400640 1 --2- 172.21.15.32:0/2172052713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67f8144e90 0x7f67f80b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:49.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.466+0000 7f67f6a00640 1 --2- 172.21.15.32:0/2172052713 >> v2:172.21.15.144:3300/0 conn(0x7f67f80b4de0 0x7f67f815baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:49.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.466+0000 7f67f7e00640 1 -- 172.21.15.32:0/2172052713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67f8144e90 msgr2=0x7f67f80b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:49.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.466+0000 7f67f7e00640 1 --2- 172.21.15.32:0/2172052713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67f8144e90 0x7f67f80b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.466+0000 7f67f7e00640 1 -- 172.21.15.32:0/2172052713 >> v2:172.21.15.144:3300/0 conn(0x7f67f80b4de0 msgr2=0x7f67f815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:49.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.466+0000 7f67f7e00640 1 --2- 172.21.15.32:0/2172052713 >> v2:172.21.15.144:3300/0 conn(0x7f67f80b4de0 0x7f67f815baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.466+0000 7f67f7e00640 1 -- 172.21.15.32:0/2172052713 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f80b3dc0 con 0x7f67f815bfe0 2024-04-17T09:53:49.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.466+0000 7f67f7400640 1 --2- 172.21.15.32:0/2172052713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67f8144e90 0x7f67f80b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:49.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.466+0000 7f67f6a00640 1 --2- 172.21.15.32:0/2172052713 >> v2:172.21.15.144:3300/0 conn(0x7f67f80b4de0 0x7f67f815baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:49.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.466+0000 7f67f7e00640 1 --2- 172.21.15.32:0/2172052713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67f815bfe0 0x7f67f806ef60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f67f806fb70 tx=0x7f67f001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ec4aeb02a4900d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:49.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.466+0000 7f67f6000640 1 -- 172.21.15.32:0/2172052713 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f00aa430 con 0x7f67f815bfe0 2024-04-17T09:53:49.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.466+0000 7f67f6000640 1 -- 172.21.15.32:0/2172052713 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67f00af070 con 0x7f67f815bfe0 2024-04-17T09:53:49.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.466+0000 7f67f6000640 1 -- 172.21.15.32:0/2172052713 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f00aa7f0 con 0x7f67f815bfe0 2024-04-17T09:53:49.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.466+0000 7f67fe400640 1 -- 172.21.15.32:0/2172052713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67f815bfe0 msgr2=0x7f67f806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:49.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67fe400640 1 --2- 172.21.15.32:0/2172052713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67f815bfe0 0x7f67f806ef60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f67f806fb70 tx=0x7f67f001c4f0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67fe400640 1 -- 172.21.15.32:0/2172052713 shutdown_connections 2024-04-17T09:53:49.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67fe400640 1 --2- 172.21.15.32:0/2172052713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67f815bfe0 0x7f67f806ef60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67fe400640 1 --2- 172.21.15.32:0/2172052713 >> v2:172.21.15.144:3300/0 conn(0x7f67f80b4de0 0x7f67f815baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67fe400640 1 --2- 172.21.15.32:0/2172052713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67f8144e90 0x7f67f80b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67fe400640 1 -- 172.21.15.32:0/2172052713 >> 172.21.15.32:0/2172052713 conn(0x7f67f806ba30 msgr2=0x7f67f806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:49.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67fe400640 1 -- 172.21.15.32:0/2172052713 shutdown_connections 2024-04-17T09:53:49.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67fe400640 1 -- 172.21.15.32:0/2172052713 wait complete. 2024-04-17T09:53:49.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67fe400640 1 Processor -- start 2024-04-17T09:53:49.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67fe400640 1 -- start start 2024-04-17T09:53:49.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67fe400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67f80b4de0 0x7f67f8142670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:49.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67fe400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67f8144e90 0x7f67f813cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:49.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67fe400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67f815bfe0 0x7f67f8140d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:49.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67fe400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f67f806f8b0 con 0x7f67f80b4de0 2024-04-17T09:53:49.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67fe400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f67f80728c0 con 0x7f67f8144e90 2024-04-17T09:53:49.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67fe400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f67f80cf2b0 con 0x7f67f815bfe0 2024-04-17T09:53:49.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67f7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67f80b4de0 0x7f67f8142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:49.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67f7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67f80b4de0 0x7f67f8142670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39280/0 (socket says 172.21.15.32:39280) 2024-04-17T09:53:49.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67f7400640 1 -- 172.21.15.32:0/2905723158 learned_addr learned my addr 172.21.15.32:0/2905723158 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:49.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67f6a00640 1 --2- 172.21.15.32:0/2905723158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67f8144e90 0x7f67f813cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:49.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67f7400640 1 -- 172.21.15.32:0/2905723158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67f815bfe0 msgr2=0x7f67f8140d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:49.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67f7400640 1 --2- 172.21.15.32:0/2905723158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67f815bfe0 0x7f67f8140d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67f7400640 1 -- 172.21.15.32:0/2905723158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67f8144e90 msgr2=0x7f67f813cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:49.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67f7400640 1 --2- 172.21.15.32:0/2905723158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67f8144e90 0x7f67f813cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67f7400640 1 -- 172.21.15.32:0/2905723158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f81415d0 con 0x7f67f80b4de0 2024-04-17T09:53:49.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67f6a00640 1 --2- 172.21.15.32:0/2905723158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67f8144e90 0x7f67f813cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:49.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.470+0000 7f67f7400640 1 --2- 172.21.15.32:0/2905723158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67f80b4de0 0x7f67f8142670 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f67e800cf60 tx=0x7f67e800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:49.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.474+0000 7f67ef400640 1 -- 172.21.15.32:0/2905723158 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e801a070 con 0x7f67f80b4de0 2024-04-17T09:53:49.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.474+0000 7f67ef400640 1 -- 172.21.15.32:0/2905723158 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67e800d840 con 0x7f67f80b4de0 2024-04-17T09:53:49.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.474+0000 7f67ef400640 1 -- 172.21.15.32:0/2905723158 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e800dc10 con 0x7f67f80b4de0 2024-04-17T09:53:49.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.474+0000 7f67fe400640 1 -- 172.21.15.32:0/2905723158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f81418b0 con 0x7f67f80b4de0 2024-04-17T09:53:49.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.474+0000 7f67fe400640 1 -- 172.21.15.32:0/2905723158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f8141e70 con 0x7f67f80b4de0 2024-04-17T09:53:49.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.474+0000 7f67fe400640 1 -- 172.21.15.32:0/2905723158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f8068a90 con 0x7f67f80b4de0 2024-04-17T09:53:49.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.474+0000 7f67ef400640 1 -- 172.21.15.32:0/2905723158 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f67e801d070 con 0x7f67f80b4de0 2024-04-17T09:53:49.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.474+0000 7f67ef400640 1 --2- 172.21.15.32:0/2905723158 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f67d8068380 0x7f67d806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:49.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.474+0000 7f67ef400640 1 -- 172.21.15.32:0/2905723158 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f67e80871a0 con 0x7f67f80b4de0 2024-04-17T09:53:49.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.474+0000 7f67f6a00640 1 --2- 172.21.15.32:0/2905723158 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f67d8068380 0x7f67d806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:49.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.474+0000 7f67f6a00640 1 --2- 172.21.15.32:0/2905723158 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f67d8068380 0x7f67d806a840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f67f81659a0 tx=0x7f67e4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:49.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.478+0000 7f67ef400640 1 -- 172.21.15.32:0/2905723158 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f67e8057070 con 0x7f67f80b4de0 2024-04-17T09:53:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.646+0000 7f67fe400640 1 -- 172.21.15.32:0/2905723158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67f80cd080 con 0x7f67f80b4de0 2024-04-17T09:53:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.646+0000 7f67ef400640 1 -- 172.21.15.32:0/2905723158 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f67e805b370 con 0x7f67f80b4de0 2024-04-17T09:53:49.648 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:49.648 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:14.779535+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:53:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.650+0000 7f67fe400640 1 -- 172.21.15.32:0/2905723158 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f67d8068380 msgr2=0x7f67d806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.650+0000 7f67fe400640 1 --2- 172.21.15.32:0/2905723158 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f67d8068380 0x7f67d806a840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f67f81659a0 tx=0x7f67e4008040 comp rx=0 tx=0).stop 2024-04-17T09:53:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.650+0000 7f67fe400640 1 -- 172.21.15.32:0/2905723158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67f80b4de0 msgr2=0x7f67f8142670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.650+0000 7f67fe400640 1 --2- 172.21.15.32:0/2905723158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67f80b4de0 0x7f67f8142670 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f67e800cf60 tx=0x7f67e800cf90 comp rx=0 tx=0).stop 2024-04-17T09:53:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.650+0000 7f67fe400640 1 -- 172.21.15.32:0/2905723158 shutdown_connections 2024-04-17T09:53:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.650+0000 7f67fe400640 1 --2- 172.21.15.32:0/2905723158 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f67d8068380 0x7f67d806a840 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.650+0000 7f67fe400640 1 --2- 172.21.15.32:0/2905723158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67f815bfe0 0x7f67f8140d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.650+0000 7f67fe400640 1 --2- 172.21.15.32:0/2905723158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67f8144e90 0x7f67f813cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.650+0000 7f67fe400640 1 --2- 172.21.15.32:0/2905723158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67f80b4de0 0x7f67f8142670 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.650+0000 7f67fe400640 1 -- 172.21.15.32:0/2905723158 >> 172.21.15.32:0/2905723158 conn(0x7f67f806ba30 msgr2=0x7f67f815b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.650+0000 7f67fe400640 1 -- 172.21.15.32:0/2905723158 shutdown_connections 2024-04-17T09:53:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.650+0000 7f67fe400640 1 -- 172.21.15.32:0/2905723158 wait complete. 2024-04-17T09:53:49.670 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:53:11.198098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:53:12.118982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:53:49.670 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:53:49.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.814+0000 7f3f4aa00640 1 Processor -- start 2024-04-17T09:53:49.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.814+0000 7f3f4aa00640 1 -- start start 2024-04-17T09:53:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.814+0000 7f3f4aa00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3f4406f7c0 0x7f3f4406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.814+0000 7f3f4aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f4406f270 0x7f3f4406bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.814+0000 7f3f4aa00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f440cc940 con 0x7f3f4415b7d0 2024-04-17T09:53:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.814+0000 7f3f4aa00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f3f440ccc20 con 0x7f3f4406f7c0 2024-04-17T09:53:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.814+0000 7f3f4aa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3f4406e0b0 con 0x7f3f4406f270 2024-04-17T09:53:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.814+0000 7f3f48a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f4406f270 0x7f3f4406bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.814+0000 7f3f43400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3f4406f7c0 0x7f3f4406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.814+0000 7f3f43400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3f4406f7c0 0x7f3f4406ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:49688/0 (socket says 172.21.15.32:49688) 2024-04-17T09:53:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.814+0000 7f3f43e00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f3f4415b7d0 0x7f3f4406e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:52192/0 (socket says 172.21.15.32:52192) 2024-04-17T09:53:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.814+0000 7f3f43e00640 1 -- 172.21.15.32:0/574151968 learned_addr learned my addr 172.21.15.32:0/574151968 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.814+0000 7f3f48a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f4406f270 0x7f3f4406bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:49616/0 (socket says 172.21.15.32:49616) 2024-04-17T09:53:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.814+0000 7f3f43400640 1 -- 172.21.15.32:0/574151968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f4406f270 msgr2=0x7f3f4406bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.814+0000 7f3f43400640 1 --2- 172.21.15.32:0/574151968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f4406f270 0x7f3f4406bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.814+0000 7f3f43400640 1 -- 172.21.15.32:0/574151968 >> v1:172.21.15.144:6789/0 conn(0x7f3f4415b7d0 legacy=0x7f3f4406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.814+0000 7f3f43400640 1 -- 172.21.15.32:0/574151968 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4406e390 con 0x7f3f4406f7c0 2024-04-17T09:53:49.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.818+0000 7f3f43400640 1 --2- 172.21.15.32:0/574151968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f4406f7c0 0x7f3f4406ba30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3f300097c0 tx=0x7f3f3001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8576aacdcdb6dc58 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:49.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.818+0000 7f3f42a00640 1 -- 172.21.15.32:0/574151968 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f300ab430 con 0x7f3f4406f7c0 2024-04-17T09:53:49.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.818+0000 7f3f42a00640 1 -- 172.21.15.32:0/574151968 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f300b0070 con 0x7f3f4406f7c0 2024-04-17T09:53:49.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.818+0000 7f3f42a00640 1 -- 172.21.15.32:0/574151968 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f300ab7f0 con 0x7f3f4406f7c0 2024-04-17T09:53:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.818+0000 7f3f4aa00640 1 -- 172.21.15.32:0/574151968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f4406f7c0 msgr2=0x7f3f4406ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.818+0000 7f3f4aa00640 1 --2- 172.21.15.32:0/574151968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f4406f7c0 0x7f3f4406ba30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3f300097c0 tx=0x7f3f3001c410 comp rx=0 tx=0).stop 2024-04-17T09:53:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.818+0000 7f3f4aa00640 1 -- 172.21.15.32:0/574151968 shutdown_connections 2024-04-17T09:53:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.818+0000 7f3f4aa00640 1 --2- 172.21.15.32:0/574151968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f4406f270 0x7f3f4406bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.818+0000 7f3f4aa00640 1 --2- 172.21.15.32:0/574151968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f4406f7c0 0x7f3f4406ba30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.818+0000 7f3f4aa00640 1 -- 172.21.15.32:0/574151968 >> 172.21.15.32:0/574151968 conn(0x7f3f440b2120 msgr2=0x7f3f440b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.818+0000 7f3f4aa00640 1 -- 172.21.15.32:0/574151968 shutdown_connections 2024-04-17T09:53:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.818+0000 7f3f4aa00640 1 -- 172.21.15.32:0/574151968 wait complete. 2024-04-17T09:53:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f4aa00640 1 Processor -- start 2024-04-17T09:53:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f4aa00640 1 -- start start 2024-04-17T09:53:49.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f4aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f4406f270 0x7f3f44150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:49.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f4aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f4406f7c0 0x7f3f4415e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:49.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f4aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f4415b7d0 0x7f3f44151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:49.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f4aa00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3f440ceeb0 con 0x7f3f4406f7c0 2024-04-17T09:53:49.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f4aa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3f440ced30 con 0x7f3f4415b7d0 2024-04-17T09:53:49.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f4aa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3f44071e60 con 0x7f3f4406f270 2024-04-17T09:53:49.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f43400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f4406f7c0 0x7f3f4415e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:49.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f43400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f4406f7c0 0x7f3f4415e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39296/0 (socket says 172.21.15.32:39296) 2024-04-17T09:53:49.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f43400640 1 -- 172.21.15.32:0/2654119010 learned_addr learned my addr 172.21.15.32:0/2654119010 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:49.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f48a00640 1 --2- 172.21.15.32:0/2654119010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f4415b7d0 0x7f3f44151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:49.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f43e00640 1 --2- 172.21.15.32:0/2654119010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f4406f270 0x7f3f44150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:49.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f43400640 1 -- 172.21.15.32:0/2654119010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f4406f270 msgr2=0x7f3f44150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:49.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f43400640 1 --2- 172.21.15.32:0/2654119010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f4406f270 0x7f3f44150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f43400640 1 -- 172.21.15.32:0/2654119010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f4415b7d0 msgr2=0x7f3f44151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:49.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f43400640 1 --2- 172.21.15.32:0/2654119010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f4415b7d0 0x7f3f44151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:49.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f43400640 1 -- 172.21.15.32:0/2654119010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f44150dd0 con 0x7f3f4406f7c0 2024-04-17T09:53:49.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f48a00640 1 --2- 172.21.15.32:0/2654119010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f4415b7d0 0x7f3f44151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:49.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f43400640 1 --2- 172.21.15.32:0/2654119010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f4406f7c0 0x7f3f4415e130 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3f300ad040 tx=0x7f3f300191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:49.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f3be00640 1 -- 172.21.15.32:0/2654119010 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f30006e20 con 0x7f3f4406f7c0 2024-04-17T09:53:49.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f3be00640 1 -- 172.21.15.32:0/2654119010 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f300b4600 con 0x7f3f4406f7c0 2024-04-17T09:53:49.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f4aa00640 1 -- 172.21.15.32:0/2654119010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f44152d10 con 0x7f3f4406f7c0 2024-04-17T09:53:49.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f3be00640 1 -- 172.21.15.32:0/2654119010 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f300b49f0 con 0x7f3f4406f7c0 2024-04-17T09:53:49.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.822+0000 7f3f4aa00640 1 -- 172.21.15.32:0/2654119010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f441532d0 con 0x7f3f4406f7c0 2024-04-17T09:53:49.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.826+0000 7f3f4aa00640 1 -- 172.21.15.32:0/2654119010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f44068a90 con 0x7f3f4406f7c0 2024-04-17T09:53:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.826+0000 7f3f3be00640 1 -- 172.21.15.32:0/2654119010 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3f300b4b90 con 0x7f3f4406f7c0 2024-04-17T09:53:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.826+0000 7f3f3be00640 1 --2- 172.21.15.32:0/2654119010 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3f24068380 0x7f3f2406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.826+0000 7f3f3be00640 1 -- 172.21.15.32:0/2654119010 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3f301211c0 con 0x7f3f4406f7c0 2024-04-17T09:53:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.826+0000 7f3f43e00640 1 --2- 172.21.15.32:0/2654119010 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3f24068380 0x7f3f2406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.826+0000 7f3f43e00640 1 --2- 172.21.15.32:0/2654119010 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3f24068380 0x7f3f2406a840 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3f34005890 tx=0x7f3f34009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:49.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.830+0000 7f3f3be00640 1 -- 172.21.15.32:0/2654119010 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3f30016090 con 0x7f3f4406f7c0 2024-04-17T09:53:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.994+0000 7f3f4aa00640 1 -- 172.21.15.32:0/2654119010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3f44159a30 con 0x7f3f4406f7c0 2024-04-17T09:53:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.998+0000 7f3f3be00640 1 -- 172.21.15.32:0/2654119010 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f3f300c5350 con 0x7f3f4406f7c0 2024-04-17T09:53:49.997 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:53:49.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.998+0000 7f3f4aa00640 1 -- 172.21.15.32:0/2654119010 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3f24068380 msgr2=0x7f3f2406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:49.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.998+0000 7f3f4aa00640 1 --2- 172.21.15.32:0/2654119010 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3f24068380 0x7f3f2406a840 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3f34005890 tx=0x7f3f34009040 comp rx=0 tx=0).stop 2024-04-17T09:53:49.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.998+0000 7f3f4aa00640 1 -- 172.21.15.32:0/2654119010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f4406f7c0 msgr2=0x7f3f4415e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:50.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.998+0000 7f3f4aa00640 1 --2- 172.21.15.32:0/2654119010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f4406f7c0 0x7f3f4415e130 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3f300ad040 tx=0x7f3f300191b0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.998+0000 7f3f4aa00640 1 -- 172.21.15.32:0/2654119010 shutdown_connections 2024-04-17T09:53:50.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.998+0000 7f3f4aa00640 1 --2- 172.21.15.32:0/2654119010 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3f24068380 0x7f3f2406a840 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.998+0000 7f3f4aa00640 1 --2- 172.21.15.32:0/2654119010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f4415b7d0 0x7f3f44151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.998+0000 7f3f4aa00640 1 --2- 172.21.15.32:0/2654119010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f4406f7c0 0x7f3f4415e130 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.998+0000 7f3f4aa00640 1 --2- 172.21.15.32:0/2654119010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f4406f270 0x7f3f44150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.998+0000 7f3f4aa00640 1 -- 172.21.15.32:0/2654119010 >> 172.21.15.32:0/2654119010 conn(0x7f3f440b2120 msgr2=0x7f3f4406ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:50.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.998+0000 7f3f4aa00640 1 -- 172.21.15.32:0/2654119010 shutdown_connections 2024-04-17T09:53:50.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:49.998+0000 7f3f4aa00640 1 -- 172.21.15.32:0/2654119010 wait complete. 2024-04-17T09:53:50.018 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:53:50.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.162+0000 7f9008000640 1 Processor -- start 2024-04-17T09:53:50.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.162+0000 7f9008000640 1 -- start start 2024-04-17T09:53:50.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.162+0000 7f9008000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f900015b7d0 0x7f900006e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:50.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.162+0000 7f9008000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f900006f7c0 0x7f900006ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:50.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.162+0000 7f9008000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90000cc940 con 0x7f900006f270 2024-04-17T09:53:50.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.162+0000 7f9008000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f90000ccc20 con 0x7f900006f7c0 2024-04-17T09:53:50.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.162+0000 7f9008000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f900006e0b0 con 0x7f900015b7d0 2024-04-17T09:53:50.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.162+0000 7f9004c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f900006f7c0 0x7f900006ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:50.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.162+0000 7f9004c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f900006f7c0 0x7f900006ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39304/0 (socket says 172.21.15.32:39304) 2024-04-17T09:53:50.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.162+0000 7f9004c00640 1 -- 172.21.15.32:0/2433303760 learned_addr learned my addr 172.21.15.32:0/2433303760 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:50.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.162+0000 7f9005600640 1 --2- 172.21.15.32:0/2433303760 >> v2:172.21.15.144:3300/0 conn(0x7f900015b7d0 0x7f900006e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:50.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.162+0000 7f9004c00640 1 -- 172.21.15.32:0/2433303760 >> v2:172.21.15.144:3300/0 conn(0x7f900015b7d0 msgr2=0x7f900006e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:50.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.162+0000 7f9004c00640 1 --2- 172.21.15.32:0/2433303760 >> v2:172.21.15.144:3300/0 conn(0x7f900015b7d0 0x7f900006e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.162+0000 7f9004c00640 1 -- 172.21.15.32:0/2433303760 >> v1:172.21.15.144:6789/0 conn(0x7f900006f270 legacy=0x7f90000ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:50.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.162+0000 7f9004c00640 1 -- 172.21.15.32:0/2433303760 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900006e390 con 0x7f900006f7c0 2024-04-17T09:53:50.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.166+0000 7f9004c00640 1 --2- 172.21.15.32:0/2433303760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f900006f7c0 0x7f900006ba30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f90000d1530 tx=0x7f8fec01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0c13bb35970312 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:50.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.166+0000 7f8fffe00640 1 -- 172.21.15.32:0/2433303760 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec0aa610 con 0x7f900006f7c0 2024-04-17T09:53:50.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.166+0000 7f8fffe00640 1 -- 172.21.15.32:0/2433303760 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fec0af070 con 0x7f900006f7c0 2024-04-17T09:53:50.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.166+0000 7f8fffe00640 1 -- 172.21.15.32:0/2433303760 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec0aa9d0 con 0x7f900006f7c0 2024-04-17T09:53:50.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.166+0000 7f9008000640 1 -- 172.21.15.32:0/2433303760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f900006f7c0 msgr2=0x7f900006ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:50.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.166+0000 7f9008000640 1 --2- 172.21.15.32:0/2433303760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f900006f7c0 0x7f900006ba30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f90000d1530 tx=0x7f8fec01c490 comp rx=0 tx=0).stop 2024-04-17T09:53:50.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.166+0000 7f9008000640 1 -- 172.21.15.32:0/2433303760 shutdown_connections 2024-04-17T09:53:50.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.166+0000 7f9008000640 1 --2- 172.21.15.32:0/2433303760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f900006f7c0 0x7f900006ba30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.166+0000 7f9008000640 1 --2- 172.21.15.32:0/2433303760 >> v2:172.21.15.144:3300/0 conn(0x7f900015b7d0 0x7f900006e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.166+0000 7f9008000640 1 -- 172.21.15.32:0/2433303760 >> 172.21.15.32:0/2433303760 conn(0x7f90000b2120 msgr2=0x7f90000b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:50.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.166+0000 7f9008000640 1 -- 172.21.15.32:0/2433303760 shutdown_connections 2024-04-17T09:53:50.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.166+0000 7f9008000640 1 -- 172.21.15.32:0/2433303760 wait complete. 2024-04-17T09:53:50.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f9008000640 1 Processor -- start 2024-04-17T09:53:50.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f9008000640 1 -- start start 2024-04-17T09:53:50.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f9008000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f900006f270 0x7f900014b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:50.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f9008000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f900006f7c0 0x7f90001f74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:50.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f9008000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f900015b7d0 0x7f9000152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:50.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f9008000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9000168ab0 con 0x7f900006f7c0 2024-04-17T09:53:50.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f9008000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f900006c380 con 0x7f900006f270 2024-04-17T09:53:50.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f9008000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9000168c30 con 0x7f900015b7d0 2024-04-17T09:53:50.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f9004c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f900006f7c0 0x7f90001f74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:50.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f9004c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f900006f7c0 0x7f90001f74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39318/0 (socket says 172.21.15.32:39318) 2024-04-17T09:53:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f9004c00640 1 -- 172.21.15.32:0/4153898162 learned_addr learned my addr 172.21.15.32:0/4153898162 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f9005600640 1 --2- 172.21.15.32:0/4153898162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f900006f270 0x7f900014b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f9004c00640 1 -- 172.21.15.32:0/4153898162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f900015b7d0 msgr2=0x7f9000152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f9006000640 1 --2- 172.21.15.32:0/4153898162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f900015b7d0 0x7f9000152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f9004c00640 1 --2- 172.21.15.32:0/4153898162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f900015b7d0 0x7f9000152100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f9004c00640 1 -- 172.21.15.32:0/4153898162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f900006f270 msgr2=0x7f900014b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f9004c00640 1 --2- 172.21.15.32:0/4153898162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f900006f270 0x7f900014b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f9004c00640 1 -- 172.21.15.32:0/4153898162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9000152640 con 0x7f900006f7c0 2024-04-17T09:53:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f9005600640 1 --2- 172.21.15.32:0/4153898162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f900006f270 0x7f900014b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f9006000640 1 --2- 172.21.15.32:0/4153898162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f900015b7d0 0x7f9000152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f9004c00640 1 --2- 172.21.15.32:0/4153898162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f900006f7c0 0x7f90001f74a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0190a0 tx=0x7f8fec01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f8ffd600640 1 -- 172.21.15.32:0/4153898162 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec0bd5f0 con 0x7f900006f7c0 2024-04-17T09:53:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f8ffd600640 1 -- 172.21.15.32:0/4153898162 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fec0bd790 con 0x7f900006f7c0 2024-04-17T09:53:50.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f8ffd600640 1 -- 172.21.15.32:0/4153898162 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec0bdb60 con 0x7f900006f7c0 2024-04-17T09:53:50.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f9008000640 1 -- 172.21.15.32:0/4153898162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f900014ed60 con 0x7f900006f7c0 2024-04-17T09:53:50.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.170+0000 7f9008000640 1 -- 172.21.15.32:0/4153898162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f900014f320 con 0x7f900006f7c0 2024-04-17T09:53:50.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.174+0000 7f9008000640 1 -- 172.21.15.32:0/4153898162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9000068a90 con 0x7f900006f7c0 2024-04-17T09:53:50.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.174+0000 7f8ffd600640 1 -- 172.21.15.32:0/4153898162 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8fec0b33f0 con 0x7f900006f7c0 2024-04-17T09:53:50.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.174+0000 7f8ffd600640 1 --2- 172.21.15.32:0/4153898162 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8fe00683b0 0x7f8fe006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:50.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.174+0000 7f8ffd600640 1 -- 172.21.15.32:0/4153898162 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8fec121370 con 0x7f900006f7c0 2024-04-17T09:53:50.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.174+0000 7f9005600640 1 --2- 172.21.15.32:0/4153898162 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8fe00683b0 0x7f8fe006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:50.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.174+0000 7f9005600640 1 --2- 172.21.15.32:0/4153898162 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8fe00683b0 0x7f8fe006a870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4009840 tx=0x7f8ff4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:50.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.178+0000 7f8ffd600640 1 -- 172.21.15.32:0/4153898162 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f8fec0a7070 con 0x7f900006f7c0 2024-04-17T09:53:50.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.342+0000 7f9008000640 1 -- 172.21.15.32:0/4153898162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f900006e7b0 con 0x7f900006f7c0 2024-04-17T09:53:50.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.342+0000 7f8ffd600640 1 -- 172.21.15.32:0/4153898162 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f8fec016110 con 0x7f900006f7c0 2024-04-17T09:53:50.344 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:53:50.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.346+0000 7f9008000640 1 -- 172.21.15.32:0/4153898162 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8fe00683b0 msgr2=0x7f8fe006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:50.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.346+0000 7f9008000640 1 --2- 172.21.15.32:0/4153898162 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8fe00683b0 0x7f8fe006a870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4009840 tx=0x7f8ff4008040 comp rx=0 tx=0).stop 2024-04-17T09:53:50.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.346+0000 7f9008000640 1 -- 172.21.15.32:0/4153898162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f900006f7c0 msgr2=0x7f90001f74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:50.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.346+0000 7f9008000640 1 --2- 172.21.15.32:0/4153898162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f900006f7c0 0x7f90001f74a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0190a0 tx=0x7f8fec01acd0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.346+0000 7f9008000640 1 -- 172.21.15.32:0/4153898162 shutdown_connections 2024-04-17T09:53:50.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.346+0000 7f9008000640 1 --2- 172.21.15.32:0/4153898162 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8fe00683b0 0x7f8fe006a870 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.346+0000 7f9008000640 1 --2- 172.21.15.32:0/4153898162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f900015b7d0 0x7f9000152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.346+0000 7f9008000640 1 --2- 172.21.15.32:0/4153898162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f900006f7c0 0x7f90001f74a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.346+0000 7f9008000640 1 --2- 172.21.15.32:0/4153898162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f900006f270 0x7f900014b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.346+0000 7f9008000640 1 -- 172.21.15.32:0/4153898162 >> 172.21.15.32:0/4153898162 conn(0x7f90000b2120 msgr2=0x7f90000b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:50.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.346+0000 7f9008000640 1 -- 172.21.15.32:0/4153898162 shutdown_connections 2024-04-17T09:53:50.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.346+0000 7f9008000640 1 -- 172.21.15.32:0/4153898162 wait complete. 2024-04-17T09:53:50.365 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:53:50.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.506+0000 7f6adb200640 1 Processor -- start 2024-04-17T09:53:50.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.506+0000 7f6adb200640 1 -- start start 2024-04-17T09:53:50.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6adb200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6ad409b100 0x7f6ad40cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:50.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6adb200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ad415f920 0x7f6ad40cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:50.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6adb200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ad40d0f00 con 0x7f6ad415dff0 2024-04-17T09:53:50.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6adb200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f6ad40ce940 con 0x7f6ad409b100 2024-04-17T09:53:50.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6adb200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6ad40d11a0 con 0x7f6ad415f920 2024-04-17T09:53:50.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6ad415dff0 0x7f6ad409a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54054/0 (socket says 172.21.15.32:54054) 2024-04-17T09:53:50.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3e00640 1 -- 172.21.15.32:0/2709029504 learned_addr learned my addr 172.21.15.32:0/2709029504 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:50.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6acb400640 1 --2- 172.21.15.32:0/2709029504 >> v2:172.21.15.144:3300/0 conn(0x7f6ad409b100 0x7f6ad40cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:50.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3400640 1 -- 172.21.15.32:0/2709029504 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 509154157 0 0) 0x7f6ad40d0f00 con 0x7f6ad415dff0 2024-04-17T09:53:50.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3400640 1 -- 172.21.15.32:0/2709029504 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ab8003530 con 0x7f6ad415dff0 2024-04-17T09:53:50.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad9200640 1 --2- 172.21.15.32:0/2709029504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ad415f920 0x7f6ad40cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:50.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3400640 1 -- 172.21.15.32:0/2709029504 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1134296722 0 0) 0x7f6ab8003530 con 0x7f6ad415dff0 2024-04-17T09:53:50.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3400640 1 -- 172.21.15.32:0/2709029504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ad415f920 msgr2=0x7f6ad40cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:50.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3400640 1 --2- 172.21.15.32:0/2709029504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ad415f920 0x7f6ad40cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3400640 1 -- 172.21.15.32:0/2709029504 >> v2:172.21.15.144:3300/0 conn(0x7f6ad409b100 msgr2=0x7f6ad40cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:50.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3400640 1 --2- 172.21.15.32:0/2709029504 >> v2:172.21.15.144:3300/0 conn(0x7f6ad409b100 0x7f6ad40cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3400640 1 -- 172.21.15.32:0/2709029504 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad40d1480 con 0x7f6ad415dff0 2024-04-17T09:53:50.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3400640 1 -- 172.21.15.32:0/2709029504 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f6ac40016a0 con 0x7f6ad415dff0 2024-04-17T09:53:50.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3400640 1 -- 172.21.15.32:0/2709029504 >> v1:172.21.15.32:6789/0 conn(0x7f6ad415dff0 legacy=0x7f6ad409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:50.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3400640 1 --2- 172.21.15.32:0/2709029504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ab80039c0 0x7f6ab8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:50.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3400640 1 --2- 172.21.15.32:0/2709029504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ad415f920 0x7f6ab8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:50.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3400640 1 --2- 172.21.15.32:0/2709029504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab8004d50 0x7f6ab80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:50.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3400640 1 -- 172.21.15.32:0/2709029504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad40ce940 con 0x7f6ab80039c0 2024-04-17T09:53:50.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3400640 1 -- 172.21.15.32:0/2709029504 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad40d11a0 con 0x7f6ab8004d50 2024-04-17T09:53:50.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3400640 1 -- 172.21.15.32:0/2709029504 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6ab800b9d0 con 0x7f6ad415f920 2024-04-17T09:53:50.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3e00640 1 --2- 172.21.15.32:0/2709029504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ab80039c0 0x7f6ab8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:50.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6acb400640 1 --2- 172.21.15.32:0/2709029504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ad415f920 0x7f6ab8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:50.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad9200640 1 --2- 172.21.15.32:0/2709029504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab8004d50 0x7f6ab80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:50.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3e00640 1 -- 172.21.15.32:0/2709029504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ad415f920 msgr2=0x7f6ab8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:50.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3e00640 1 --2- 172.21.15.32:0/2709029504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ad415f920 0x7f6ab8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3e00640 1 -- 172.21.15.32:0/2709029504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab8004d50 msgr2=0x7f6ab80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:50.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3e00640 1 --2- 172.21.15.32:0/2709029504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab8004d50 0x7f6ab80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.510+0000 7f6ad3e00640 1 -- 172.21.15.32:0/2709029504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab8003530 con 0x7f6ab80039c0 2024-04-17T09:53:50.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.514+0000 7f6ad3e00640 1 --2- 172.21.15.32:0/2709029504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ab80039c0 0x7f6ab8003de0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400c210 tx=0x7f6ac401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a8e0ba6bd2bd22 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:50.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.514+0000 7f6ad3400640 1 -- 172.21.15.32:0/2709029504 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4007d80 con 0x7f6ab80039c0 2024-04-17T09:53:50.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.514+0000 7f6ad3400640 1 -- 172.21.15.32:0/2709029504 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ac40a9020 con 0x7f6ab80039c0 2024-04-17T09:53:50.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.514+0000 7f6ad3400640 1 -- 172.21.15.32:0/2709029504 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4005ae0 con 0x7f6ab80039c0 2024-04-17T09:53:50.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.514+0000 7f6adb200640 1 -- 172.21.15.32:0/2709029504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ab80039c0 msgr2=0x7f6ab8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:50.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.514+0000 7f6adb200640 1 --2- 172.21.15.32:0/2709029504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ab80039c0 0x7f6ab8003de0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400c210 tx=0x7f6ac401cca0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.514+0000 7f6ad9200640 1 -- 172.21.15.32:0/2709029504 reap_dead start 2024-04-17T09:53:50.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.514+0000 7f6adb200640 1 -- 172.21.15.32:0/2709029504 shutdown_connections 2024-04-17T09:53:50.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.514+0000 7f6adb200640 1 -- 172.21.15.32:0/2709029504 >> 172.21.15.32:0/2709029504 conn(0x7f6ad40b40e0 msgr2=0x7f6ad40b44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:50.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.514+0000 7f6adb200640 1 -- 172.21.15.32:0/2709029504 shutdown_connections 2024-04-17T09:53:50.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.514+0000 7f6adb200640 1 -- 172.21.15.32:0/2709029504 wait complete. 2024-04-17T09:53:50.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6adb200640 1 Processor -- start 2024-04-17T09:53:50.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6adb200640 1 -- start start 2024-04-17T09:53:50.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6adb200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ab80039c0 0x7f6ad4159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:50.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6adb200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad40c6d60 0x7f6ad4156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:50.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6adb200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ad4159f00 0x7f6ad415a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:50.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6adb200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad40ce390 con 0x7f6ad40c6d60 2024-04-17T09:53:50.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6adb200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad40ce210 con 0x7f6ad4159f00 2024-04-17T09:53:50.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6adb200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6ad40ce510 con 0x7f6ab80039c0 2024-04-17T09:53:50.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6acb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad40c6d60 0x7f6ad4156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:50.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6acb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad40c6d60 0x7f6ad4156670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44626/0 (socket says 172.21.15.32:44626) 2024-04-17T09:53:50.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6acb400640 1 -- 172.21.15.32:0/1968241356 learned_addr learned my addr 172.21.15.32:0/1968241356 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:50.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6ad3e00640 1 --2- 172.21.15.32:0/1968241356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ab80039c0 0x7f6ad4159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6acb400640 1 -- 172.21.15.32:0/1968241356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ab80039c0 msgr2=0x7f6ad4159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6acb400640 1 --2- 172.21.15.32:0/1968241356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ab80039c0 0x7f6ad4159220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6acb400640 1 -- 172.21.15.32:0/1968241356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ad4159f00 msgr2=0x7f6ad415a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6ad9200640 1 --2- 172.21.15.32:0/1968241356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ad4159f00 0x7f6ad415a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6acb400640 1 --2- 172.21.15.32:0/1968241356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ad4159f00 0x7f6ad415a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6acb400640 1 -- 172.21.15.32:0/1968241356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad4069330 con 0x7f6ad40c6d60 2024-04-17T09:53:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6ad3e00640 1 --2- 172.21.15.32:0/1968241356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ab80039c0 0x7f6ad4159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6ad9200640 1 --2- 172.21.15.32:0/1968241356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ad4159f00 0x7f6ad415a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6acb400640 1 --2- 172.21.15.32:0/1968241356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad40c6d60 0x7f6ad4156670 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6abc0115e0 tx=0x7f6abc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6ad0c00640 1 -- 172.21.15.32:0/1968241356 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6abc00f8f0 con 0x7f6ad40c6d60 2024-04-17T09:53:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6ad0c00640 1 -- 172.21.15.32:0/1968241356 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6abc00fa90 con 0x7f6ad40c6d60 2024-04-17T09:53:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6adb200640 1 -- 172.21.15.32:0/1968241356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad4159760 con 0x7f6ad40c6d60 2024-04-17T09:53:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6ad0c00640 1 -- 172.21.15.32:0/1968241356 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6abc00fe40 con 0x7f6ad40c6d60 2024-04-17T09:53:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6adb200640 1 -- 172.21.15.32:0/1968241356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad40c7500 con 0x7f6ad40c6d60 2024-04-17T09:53:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.518+0000 7f6adb200640 1 -- 172.21.15.32:0/1968241356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad4068a90 con 0x7f6ad40c6d60 2024-04-17T09:53:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.522+0000 7f6ad0c00640 1 -- 172.21.15.32:0/1968241356 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6abc015460 con 0x7f6ad40c6d60 2024-04-17T09:53:50.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.522+0000 7f6ad0c00640 1 --2- 172.21.15.32:0/1968241356 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6ab0068380 0x7f6ab006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:50.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.522+0000 7f6ad0c00640 1 -- 172.21.15.32:0/1968241356 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6abc087280 con 0x7f6ad40c6d60 2024-04-17T09:53:50.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.522+0000 7f6ad3e00640 1 --2- 172.21.15.32:0/1968241356 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6ab0068380 0x7f6ab006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:50.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.522+0000 7f6ad3e00640 1 --2- 172.21.15.32:0/1968241356 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6ab0068380 0x7f6ab006a840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6ac40aa000 tx=0x7f6ac4001ae0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:50.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.526+0000 7f6ad0c00640 1 -- 172.21.15.32:0/1968241356 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f6abc019030 con 0x7f6ad40c6d60 2024-04-17T09:53:50.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.690+0000 7f6adb200640 1 -- 172.21.15.32:0/1968241356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6ad40c5c10 con 0x7f6ad40c6d60 2024-04-17T09:53:50.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.690+0000 7f6ad0c00640 1 -- 172.21.15.32:0/1968241356 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f6abc057830 con 0x7f6ad40c6d60 2024-04-17T09:53:50.691 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:53:50.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.690+0000 7f6adb200640 1 -- 172.21.15.32:0/1968241356 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6ab0068380 msgr2=0x7f6ab006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:50.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.690+0000 7f6adb200640 1 --2- 172.21.15.32:0/1968241356 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6ab0068380 0x7f6ab006a840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6ac40aa000 tx=0x7f6ac4001ae0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.690+0000 7f6adb200640 1 -- 172.21.15.32:0/1968241356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad40c6d60 msgr2=0x7f6ad4156670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:50.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.690+0000 7f6adb200640 1 --2- 172.21.15.32:0/1968241356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad40c6d60 0x7f6ad4156670 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6abc0115e0 tx=0x7f6abc011aa0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.690+0000 7f6adb200640 1 -- 172.21.15.32:0/1968241356 shutdown_connections 2024-04-17T09:53:50.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.690+0000 7f6adb200640 1 --2- 172.21.15.32:0/1968241356 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6ab0068380 0x7f6ab006a840 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.690+0000 7f6adb200640 1 --2- 172.21.15.32:0/1968241356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ad4159f00 0x7f6ad415a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.690+0000 7f6adb200640 1 --2- 172.21.15.32:0/1968241356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad40c6d60 0x7f6ad4156670 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.690+0000 7f6adb200640 1 --2- 172.21.15.32:0/1968241356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ab80039c0 0x7f6ad4159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.690+0000 7f6adb200640 1 -- 172.21.15.32:0/1968241356 >> 172.21.15.32:0/1968241356 conn(0x7f6ad40b40e0 msgr2=0x7f6ad409b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:50.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.690+0000 7f6adb200640 1 -- 172.21.15.32:0/1968241356 shutdown_connections 2024-04-17T09:53:50.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.694+0000 7f6adb200640 1 -- 172.21.15.32:0/1968241356 wait complete. 2024-04-17T09:53:50.711 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T09:53:50.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.854+0000 7fab97200640 1 Processor -- start 2024-04-17T09:53:50.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.854+0000 7fab97200640 1 -- start start 2024-04-17T09:53:50.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.854+0000 7fab97200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab9013f6e0 0x7fab900cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:50.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.854+0000 7fab97200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab900cf370 0x7fab900d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:50.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.854+0000 7fab97200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab901601e0 con 0x7fab900d0820 2024-04-17T09:53:50.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.854+0000 7fab97200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fab9013fac0 con 0x7fab900cf370 2024-04-17T09:53:50.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.854+0000 7fab97200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fab90160480 con 0x7fab9013f6e0 2024-04-17T09:53:50.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.854+0000 7fab87400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab900cf370 0x7fab900d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:50.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.854+0000 7fab95200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fab900d0820 0x7fab90141d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54066/0 (socket says 172.21.15.32:54066) 2024-04-17T09:53:50.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.854+0000 7fab95200640 1 -- 172.21.15.32:0/887284038 learned_addr learned my addr 172.21.15.32:0/887284038 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:50.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.854+0000 7fab8fe00640 1 --2- 172.21.15.32:0/887284038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab9013f6e0 0x7fab900cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:50.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.854+0000 7fab87400640 1 -- 172.21.15.32:0/887284038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab9013f6e0 msgr2=0x7fab900cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:50.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.854+0000 7fab87400640 1 --2- 172.21.15.32:0/887284038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab9013f6e0 0x7fab900cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.854+0000 7fab87400640 1 -- 172.21.15.32:0/887284038 >> v1:172.21.15.32:6789/0 conn(0x7fab900d0820 legacy=0x7fab90141d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:50.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.854+0000 7fab87400640 1 -- 172.21.15.32:0/887284038 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab90160760 con 0x7fab900cf370 2024-04-17T09:53:50.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.854+0000 7fab8f400640 1 -- 172.21.15.32:0/887284038 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3048256745 0 0) 0x7fab901601e0 con 0x7fab900d0820 2024-04-17T09:53:50.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.854+0000 7fab8fe00640 1 --2- 172.21.15.32:0/887284038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab9013f6e0 0x7fab900cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:50.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.858+0000 7fab87400640 1 --2- 172.21.15.32:0/887284038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab900cf370 0x7fab900d02e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fab780097c0 tx=0x7fab7801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af975a0534d355bf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:50.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.858+0000 7fab8f400640 1 -- 172.21.15.32:0/887284038 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab780b0070 con 0x7fab900cf370 2024-04-17T09:53:50.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.858+0000 7fab8f400640 1 -- 172.21.15.32:0/887284038 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab780ab600 con 0x7fab900cf370 2024-04-17T09:53:50.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.858+0000 7fab8f400640 1 -- 172.21.15.32:0/887284038 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab780ab9d0 con 0x7fab900cf370 2024-04-17T09:53:50.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.858+0000 7fab97200640 1 -- 172.21.15.32:0/887284038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab900cf370 msgr2=0x7fab900d02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:50.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.858+0000 7fab97200640 1 --2- 172.21.15.32:0/887284038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab900cf370 0x7fab900d02e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fab780097c0 tx=0x7fab7801c4a0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.858+0000 7fab97200640 1 -- 172.21.15.32:0/887284038 shutdown_connections 2024-04-17T09:53:50.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.858+0000 7fab97200640 1 --2- 172.21.15.32:0/887284038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab900cf370 0x7fab900d02e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.858+0000 7fab97200640 1 --2- 172.21.15.32:0/887284038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab9013f6e0 0x7fab900cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.858+0000 7fab97200640 1 -- 172.21.15.32:0/887284038 >> 172.21.15.32:0/887284038 conn(0x7fab9006ba30 msgr2=0x7fab9006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:50.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.858+0000 7fab97200640 1 -- 172.21.15.32:0/887284038 shutdown_connections 2024-04-17T09:53:50.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.858+0000 7fab97200640 1 -- 172.21.15.32:0/887284038 wait complete. 2024-04-17T09:53:50.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab97200640 1 Processor -- start 2024-04-17T09:53:50.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab97200640 1 -- start start 2024-04-17T09:53:50.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab97200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab900cf370 0x7fab90156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:50.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab97200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab900d0820 0x7fab9013e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:50.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab97200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab9013f6e0 0x7fab90155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:50.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab97200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fab900cff10 con 0x7fab9013f6e0 2024-04-17T09:53:50.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab97200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fab900cfd90 con 0x7fab900d0820 2024-04-17T09:53:50.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab97200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fab90169580 con 0x7fab900cf370 2024-04-17T09:53:50.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab95200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab9013f6e0 0x7fab90155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:50.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab95200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab9013f6e0 0x7fab90155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44636/0 (socket says 172.21.15.32:44636) 2024-04-17T09:53:50.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab95200640 1 -- 172.21.15.32:0/1837058208 learned_addr learned my addr 172.21.15.32:0/1837058208 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:50.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab8fe00640 1 --2- 172.21.15.32:0/1837058208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab900cf370 0x7fab90156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:50.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab87400640 1 --2- 172.21.15.32:0/1837058208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab900d0820 0x7fab9013e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:50.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab95200640 1 -- 172.21.15.32:0/1837058208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab900cf370 msgr2=0x7fab90156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:50.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab95200640 1 --2- 172.21.15.32:0/1837058208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab900cf370 0x7fab90156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab95200640 1 -- 172.21.15.32:0/1837058208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab900d0820 msgr2=0x7fab9013e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:50.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab95200640 1 --2- 172.21.15.32:0/1837058208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab900d0820 0x7fab9013e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:50.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab95200640 1 -- 172.21.15.32:0/1837058208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab9015e920 con 0x7fab9013f6e0 2024-04-17T09:53:50.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab95200640 1 --2- 172.21.15.32:0/1837058208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab9013f6e0 0x7fab90155d60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fab8800c7a0 tx=0x7fab8800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:50.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab8cc00640 1 -- 172.21.15.32:0/1837058208 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab88018070 con 0x7fab9013f6e0 2024-04-17T09:53:50.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab8cc00640 1 -- 172.21.15.32:0/1837058208 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab8800a640 con 0x7fab9013f6e0 2024-04-17T09:53:50.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab97200640 1 -- 172.21.15.32:0/1837058208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab9015ec00 con 0x7fab9013f6e0 2024-04-17T09:53:50.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab8cc00640 1 -- 172.21.15.32:0/1837058208 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8800aab0 con 0x7fab9013f6e0 2024-04-17T09:53:50.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab97200640 1 -- 172.21.15.32:0/1837058208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab9015f1c0 con 0x7fab9013f6e0 2024-04-17T09:53:50.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.862+0000 7fab97200640 1 -- 172.21.15.32:0/1837058208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab90068a90 con 0x7fab9013f6e0 2024-04-17T09:53:50.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.866+0000 7fab8cc00640 1 -- 172.21.15.32:0/1837058208 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fab8801b070 con 0x7fab9013f6e0 2024-04-17T09:53:50.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.866+0000 7fab8cc00640 1 --2- 172.21.15.32:0/1837058208 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab6c0683b0 0x7fab6c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:50.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.866+0000 7fab8cc00640 1 -- 172.21.15.32:0/1837058208 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fab88086050 con 0x7fab9013f6e0 2024-04-17T09:53:50.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.866+0000 7fab8fe00640 1 --2- 172.21.15.32:0/1837058208 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab6c0683b0 0x7fab6c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:50.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.866+0000 7fab8fe00640 1 --2- 172.21.15.32:0/1837058208 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab6c0683b0 0x7fab6c06a870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fab9015c2e0 tx=0x7fab80008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:50.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:50.870+0000 7fab8cc00640 1 -- 172.21.15.32:0/1837058208 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fab88056570 con 0x7fab9013f6e0 2024-04-17T09:53:51.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.050+0000 7fab97200640 1 -- 172.21.15.32:0/1837058208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fab900d1030 con 0x7fab9013f6e0 2024-04-17T09:53:51.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.050+0000 7fab8cc00640 1 -- 172.21.15.32:0/1837058208 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7fab88055e70 con 0x7fab9013f6e0 2024-04-17T09:53:51.049 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T09:53:51.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.050+0000 7fab97200640 1 -- 172.21.15.32:0/1837058208 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab6c0683b0 msgr2=0x7fab6c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:51.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.050+0000 7fab97200640 1 --2- 172.21.15.32:0/1837058208 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab6c0683b0 0x7fab6c06a870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fab9015c2e0 tx=0x7fab80008040 comp rx=0 tx=0).stop 2024-04-17T09:53:51.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.050+0000 7fab97200640 1 -- 172.21.15.32:0/1837058208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab9013f6e0 msgr2=0x7fab90155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:51.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.050+0000 7fab97200640 1 --2- 172.21.15.32:0/1837058208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab9013f6e0 0x7fab90155d60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fab8800c7a0 tx=0x7fab8800cc60 comp rx=0 tx=0).stop 2024-04-17T09:53:51.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.050+0000 7fab97200640 1 -- 172.21.15.32:0/1837058208 shutdown_connections 2024-04-17T09:53:51.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.050+0000 7fab97200640 1 --2- 172.21.15.32:0/1837058208 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab6c0683b0 0x7fab6c06a870 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:51.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.050+0000 7fab97200640 1 --2- 172.21.15.32:0/1837058208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab9013f6e0 0x7fab90155d60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:51.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.050+0000 7fab97200640 1 --2- 172.21.15.32:0/1837058208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab900d0820 0x7fab9013e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:51.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.050+0000 7fab97200640 1 --2- 172.21.15.32:0/1837058208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab900cf370 0x7fab90156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:51.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.050+0000 7fab97200640 1 -- 172.21.15.32:0/1837058208 >> 172.21.15.32:0/1837058208 conn(0x7fab9006ba30 msgr2=0x7fab90166d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:51.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.050+0000 7fab97200640 1 -- 172.21.15.32:0/1837058208 shutdown_connections 2024-04-17T09:53:51.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.050+0000 7fab97200640 1 -- 172.21.15.32:0/1837058208 wait complete. 2024-04-17T09:53:51.071 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2024-04-17T09:53:51.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:51.071 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T09:53:51.102 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:53:51.102 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T09:53:51.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:51.103 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T09:53:51.171 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:53:51.171 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:53:51.180 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:53:51.181 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:53:51.231 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:53:51.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.370+0000 7f2231c00640 1 Processor -- start 2024-04-17T09:53:51.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.370+0000 7f2231c00640 1 -- start start 2024-04-17T09:53:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.374+0000 7f2231c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f222c145f30 0x7f222c071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.374+0000 7f2231c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f222c15d8d0 0x7f222c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.374+0000 7f2231c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f222c0cce70 0x7f222c0cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.374+0000 7f2231c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f222c06f400 con 0x7f222c15d8d0 2024-04-17T09:53:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.374+0000 7f2231c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f222c146310 con 0x7f222c145f30 2024-04-17T09:53:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.374+0000 7f2231c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f222c15d440 con 0x7f222c0cce70 2024-04-17T09:53:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.374+0000 7f2222000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f222c15d8d0 0x7f222c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.374+0000 7f2222000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f222c15d8d0 0x7f222c072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44644/0 (socket says 172.21.15.32:44644) 2024-04-17T09:53:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.374+0000 7f2222000640 1 -- 172.21.15.32:0/4202377797 learned_addr learned my addr 172.21.15.32:0/4202377797 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.374+0000 7f222aa00640 1 --2- 172.21.15.32:0/4202377797 >> v2:172.21.15.144:3300/0 conn(0x7f222c145f30 0x7f222c071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.374+0000 7f222b400640 1 --2- 172.21.15.32:0/4202377797 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f222c0cce70 0x7f222c0cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.374+0000 7f2222000640 1 -- 172.21.15.32:0/4202377797 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f222c0cce70 msgr2=0x7f222c0cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.374+0000 7f2222000640 1 --2- 172.21.15.32:0/4202377797 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f222c0cce70 0x7f222c0cd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.374+0000 7f2222000640 1 -- 172.21.15.32:0/4202377797 >> v2:172.21.15.144:3300/0 conn(0x7f222c145f30 msgr2=0x7f222c071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.374+0000 7f2222000640 1 --2- 172.21.15.32:0/4202377797 >> v2:172.21.15.144:3300/0 conn(0x7f222c145f30 0x7f222c071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:51.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.374+0000 7f2222000640 1 -- 172.21.15.32:0/4202377797 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222c0cd820 con 0x7f222c15d8d0 2024-04-17T09:53:51.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.374+0000 7f222aa00640 1 --2- 172.21.15.32:0/4202377797 >> v2:172.21.15.144:3300/0 conn(0x7f222c145f30 0x7f222c071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:51.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f2222000640 1 --2- 172.21.15.32:0/4202377797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f222c15d8d0 0x7f222c072330 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f222c06ff00 tx=0x7f221401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1ef82e37f8fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:51.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f222a000640 1 -- 172.21.15.32:0/4202377797 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22140b4070 con 0x7f222c15d8d0 2024-04-17T09:53:51.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f222a000640 1 -- 172.21.15.32:0/4202377797 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22140af400 con 0x7f222c15d8d0 2024-04-17T09:53:51.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f222a000640 1 -- 172.21.15.32:0/4202377797 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22140af7d0 con 0x7f222c15d8d0 2024-04-17T09:53:51.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f2231c00640 1 -- 172.21.15.32:0/4202377797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f222c15d8d0 msgr2=0x7f222c072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:51.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f2231c00640 1 --2- 172.21.15.32:0/4202377797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f222c15d8d0 0x7f222c072330 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f222c06ff00 tx=0x7f221401f1e0 comp rx=0 tx=0).stop 2024-04-17T09:53:51.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f2231c00640 1 -- 172.21.15.32:0/4202377797 shutdown_connections 2024-04-17T09:53:51.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f2231c00640 1 --2- 172.21.15.32:0/4202377797 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f222c0cce70 0x7f222c0cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:51.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f2231c00640 1 --2- 172.21.15.32:0/4202377797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f222c15d8d0 0x7f222c072330 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:51.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f2231c00640 1 --2- 172.21.15.32:0/4202377797 >> v2:172.21.15.144:3300/0 conn(0x7f222c145f30 0x7f222c071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:51.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f2231c00640 1 -- 172.21.15.32:0/4202377797 >> 172.21.15.32:0/4202377797 conn(0x7f222c06ba30 msgr2=0x7f222c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:51.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f2231c00640 1 -- 172.21.15.32:0/4202377797 shutdown_connections 2024-04-17T09:53:51.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f2231c00640 1 -- 172.21.15.32:0/4202377797 wait complete. 2024-04-17T09:53:51.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f2231c00640 1 Processor -- start 2024-04-17T09:53:51.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f2231c00640 1 -- start start 2024-04-17T09:53:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f2231c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f222c0cce70 0x7f222c0a94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f2231c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f222c145f30 0x7f222c1e98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f2231c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f222c15d8d0 0x7f222c0aa060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f2231c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f222c15cdf0 con 0x7f222c145f30 2024-04-17T09:53:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f2231c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f222c15cb90 con 0x7f222c0cce70 2024-04-17T09:53:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f2231c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f222c15d0d0 con 0x7f222c15d8d0 2024-04-17T09:53:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f2222000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f222c145f30 0x7f222c1e98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f2222000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f222c145f30 0x7f222c1e98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44656/0 (socket says 172.21.15.32:44656) 2024-04-17T09:53:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.378+0000 7f2222000640 1 -- 172.21.15.32:0/2047323102 learned_addr learned my addr 172.21.15.32:0/2047323102 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.382+0000 7f222b400640 1 --2- 172.21.15.32:0/2047323102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f222c15d8d0 0x7f222c0aa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.382+0000 7f222aa00640 1 --2- 172.21.15.32:0/2047323102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f222c0cce70 0x7f222c0a94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.382+0000 7f2222000640 1 -- 172.21.15.32:0/2047323102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f222c15d8d0 msgr2=0x7f222c0aa060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.382+0000 7f2222000640 1 --2- 172.21.15.32:0/2047323102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f222c15d8d0 0x7f222c0aa060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.382+0000 7f2222000640 1 -- 172.21.15.32:0/2047323102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f222c0cce70 msgr2=0x7f222c0a94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.382+0000 7f2222000640 1 --2- 172.21.15.32:0/2047323102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f222c0cce70 0x7f222c0a94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:51.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.382+0000 7f2222000640 1 -- 172.21.15.32:0/2047323102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222c1e7a90 con 0x7f222c145f30 2024-04-17T09:53:51.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.382+0000 7f2222000640 1 --2- 172.21.15.32:0/2047323102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f222c145f30 0x7f222c1e98f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2214009040 tx=0x7f221401bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:51.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.382+0000 7f2223400640 1 -- 172.21.15.32:0/2047323102 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22140b4040 con 0x7f222c145f30 2024-04-17T09:53:51.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.382+0000 7f2223400640 1 -- 172.21.15.32:0/2047323102 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22140afe30 con 0x7f222c145f30 2024-04-17T09:53:51.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.382+0000 7f2223400640 1 -- 172.21.15.32:0/2047323102 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214009510 con 0x7f222c145f30 2024-04-17T09:53:51.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.382+0000 7f2231c00640 1 -- 172.21.15.32:0/2047323102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f222c1e7d50 con 0x7f222c145f30 2024-04-17T09:53:51.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.382+0000 7f2231c00640 1 -- 172.21.15.32:0/2047323102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f222c1e8310 con 0x7f222c145f30 2024-04-17T09:53:51.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.382+0000 7f2231c00640 1 -- 172.21.15.32:0/2047323102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f222c068a90 con 0x7f222c145f30 2024-04-17T09:53:51.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.382+0000 7f2223400640 1 -- 172.21.15.32:0/2047323102 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f22140c3020 con 0x7f222c145f30 2024-04-17T09:53:51.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.382+0000 7f2223400640 1 --2- 172.21.15.32:0/2047323102 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2208068380 0x7f220806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:51.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.382+0000 7f2223400640 1 -- 172.21.15.32:0/2047323102 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2214124470 con 0x7f222c145f30 2024-04-17T09:53:51.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.386+0000 7f222aa00640 1 --2- 172.21.15.32:0/2047323102 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2208068380 0x7f220806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:51.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.386+0000 7f222aa00640 1 --2- 172.21.15.32:0/2047323102 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2208068380 0x7f220806a840 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f221c00c900 tx=0x7f221c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.386+0000 7f2223400640 1 -- 172.21.15.32:0/2047323102 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f22140ca020 con 0x7f222c145f30 2024-04-17T09:53:51.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.558+0000 7f2231c00640 1 -- 172.21.15.32:0/2047323102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f222c072f90 con 0x7f222c145f30 2024-04-17T09:53:51.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.558+0000 7f2223400640 1 -- 172.21.15.32:0/2047323102 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f2214018d40 con 0x7f222c145f30 2024-04-17T09:53:51.558 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:51.559 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:14.779535+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:53:51.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.558+0000 7f2231c00640 1 -- 172.21.15.32:0/2047323102 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2208068380 msgr2=0x7f220806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:51.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.558+0000 7f2231c00640 1 --2- 172.21.15.32:0/2047323102 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2208068380 0x7f220806a840 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f221c00c900 tx=0x7f221c008040 comp rx=0 tx=0).stop 2024-04-17T09:53:51.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.558+0000 7f2231c00640 1 -- 172.21.15.32:0/2047323102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f222c145f30 msgr2=0x7f222c1e98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:51.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.558+0000 7f2231c00640 1 --2- 172.21.15.32:0/2047323102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f222c145f30 0x7f222c1e98f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2214009040 tx=0x7f221401bf80 comp rx=0 tx=0).stop 2024-04-17T09:53:51.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.558+0000 7f2231c00640 1 -- 172.21.15.32:0/2047323102 shutdown_connections 2024-04-17T09:53:51.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.558+0000 7f2231c00640 1 --2- 172.21.15.32:0/2047323102 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2208068380 0x7f220806a840 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:51.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.558+0000 7f2231c00640 1 --2- 172.21.15.32:0/2047323102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f222c15d8d0 0x7f222c0aa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:51.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.558+0000 7f2231c00640 1 --2- 172.21.15.32:0/2047323102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f222c145f30 0x7f222c1e98f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:51.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.558+0000 7f2231c00640 1 --2- 172.21.15.32:0/2047323102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f222c0cce70 0x7f222c0a94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:51.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.558+0000 7f2231c00640 1 -- 172.21.15.32:0/2047323102 >> 172.21.15.32:0/2047323102 conn(0x7f222c06ba30 msgr2=0x7f222c1601a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:51.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.558+0000 7f2231c00640 1 -- 172.21.15.32:0/2047323102 shutdown_connections 2024-04-17T09:53:51.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.558+0000 7f2231c00640 1 -- 172.21.15.32:0/2047323102 wait complete. 2024-04-17T09:53:51.580 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:53:11.198098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:53:12.118982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:53:51.580 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:53:51.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.722+0000 7f5cd2400640 1 Processor -- start 2024-04-17T09:53:51.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.722+0000 7f5cd2400640 1 -- start start 2024-04-17T09:53:51.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.722+0000 7f5cd2400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5ccc06f7c0 0x7f5ccc06ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.722+0000 7f5cd2400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ccc0cc940 con 0x7f5ccc06f270 2024-04-17T09:53:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.722+0000 7f5cd2400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ccc0ccc20 con 0x7f5ccc15b7d0 2024-04-17T09:53:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.722+0000 7f5cd2400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f5ccc06e0b0 con 0x7f5ccc06f7c0 2024-04-17T09:53:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.726+0000 7f5ccbe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5ccc06f270 0x7f5ccc0ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54072/0 (socket says 172.21.15.32:54072) 2024-04-17T09:53:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.726+0000 7f5ccbe00640 1 -- 172.21.15.32:0/1449732485 learned_addr learned my addr 172.21.15.32:0/1449732485 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.726+0000 7f5cc3e00640 1 --2- 172.21.15.32:0/1449732485 >> v2:172.21.15.144:3300/0 conn(0x7f5ccc06f7c0 0x7f5ccc06ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:51.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.726+0000 7f5ccaa00640 1 -- 172.21.15.32:0/1449732485 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1643811083 0 0) 0x7f5ccc0cc940 con 0x7f5ccc06f270 2024-04-17T09:53:51.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.726+0000 7f5ccaa00640 1 -- 172.21.15.32:0/1449732485 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5cb0005740 con 0x7f5ccc06f270 2024-04-17T09:53:51.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.726+0000 7f5cc3e00640 1 -- 172.21.15.32:0/1449732485 >> v1:172.21.15.144:6789/0 conn(0x7f5ccc15b7d0 legacy=0x7f5ccc06e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:51.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.726+0000 7f5cc3e00640 1 -- 172.21.15.32:0/1449732485 >> v1:172.21.15.32:6789/0 conn(0x7f5ccc06f270 legacy=0x7f5ccc0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:51.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.726+0000 7f5cc3e00640 1 -- 172.21.15.32:0/1449732485 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ccc06e390 con 0x7f5ccc06f7c0 2024-04-17T09:53:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.726+0000 7f5cc3e00640 1 --2- 172.21.15.32:0/1449732485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ccc06f7c0 0x7f5ccc06ba30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40097c0 tx=0x7f5cb401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=568d94bf780c5d3c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.730+0000 7f5ccaa00640 1 -- 172.21.15.32:0/1449732485 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40b0070 con 0x7f5ccc06f7c0 2024-04-17T09:53:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.730+0000 7f5ccaa00640 1 -- 172.21.15.32:0/1449732485 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cb40ab600 con 0x7f5ccc06f7c0 2024-04-17T09:53:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.730+0000 7f5ccaa00640 1 -- 172.21.15.32:0/1449732485 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40ab9b0 con 0x7f5ccc06f7c0 2024-04-17T09:53:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.730+0000 7f5cd2400640 1 -- 172.21.15.32:0/1449732485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ccc06f7c0 msgr2=0x7f5ccc06ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.730+0000 7f5cd2400640 1 --2- 172.21.15.32:0/1449732485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ccc06f7c0 0x7f5ccc06ba30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40097c0 tx=0x7f5cb401c410 comp rx=0 tx=0).stop 2024-04-17T09:53:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.730+0000 7f5cd2400640 1 -- 172.21.15.32:0/1449732485 shutdown_connections 2024-04-17T09:53:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.730+0000 7f5cd2400640 1 --2- 172.21.15.32:0/1449732485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ccc06f7c0 0x7f5ccc06ba30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.730+0000 7f5cd2400640 1 -- 172.21.15.32:0/1449732485 >> 172.21.15.32:0/1449732485 conn(0x7f5ccc0b2120 msgr2=0x7f5ccc0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.730+0000 7f5cd2400640 1 -- 172.21.15.32:0/1449732485 shutdown_connections 2024-04-17T09:53:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.730+0000 7f5cd2400640 1 -- 172.21.15.32:0/1449732485 wait complete. 2024-04-17T09:53:51.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.730+0000 7f5cd2400640 1 Processor -- start 2024-04-17T09:53:51.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.730+0000 7f5cd2400640 1 -- start start 2024-04-17T09:53:51.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.730+0000 7f5cd2400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ccc06f270 0x7f5ccc151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:51.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.730+0000 7f5cd2400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ccc06f7c0 0x7f5ccc0ada20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:51.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.730+0000 7f5cd2400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ccc15b7d0 0x7f5ccc159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:51.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.730+0000 7f5cd2400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc072a40 con 0x7f5ccc15b7d0 2024-04-17T09:53:51.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.730+0000 7f5cd2400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc0728c0 con 0x7f5ccc06f270 2024-04-17T09:53:51.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.730+0000 7f5cd2400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5ccc072bc0 con 0x7f5ccc06f7c0 2024-04-17T09:53:51.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.730+0000 7f5ccbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ccc15b7d0 0x7f5ccc159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:51.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.730+0000 7f5ccbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ccc15b7d0 0x7f5ccc159de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44658/0 (socket says 172.21.15.32:44658) 2024-04-17T09:53:51.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.730+0000 7f5ccbe00640 1 -- 172.21.15.32:0/3915991475 learned_addr learned my addr 172.21.15.32:0/3915991475 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:51.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.730+0000 7f5ccb400640 1 --2- 172.21.15.32:0/3915991475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ccc06f270 0x7f5ccc151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:51.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.730+0000 7f5cc3e00640 1 --2- 172.21.15.32:0/3915991475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ccc06f7c0 0x7f5ccc0ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:51.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.734+0000 7f5ccbe00640 1 -- 172.21.15.32:0/3915991475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ccc06f7c0 msgr2=0x7f5ccc0ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:51.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.734+0000 7f5ccbe00640 1 --2- 172.21.15.32:0/3915991475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ccc06f7c0 0x7f5ccc0ada20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:51.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.734+0000 7f5ccbe00640 1 -- 172.21.15.32:0/3915991475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ccc06f270 msgr2=0x7f5ccc151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:51.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.734+0000 7f5ccbe00640 1 --2- 172.21.15.32:0/3915991475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ccc06f270 0x7f5ccc151b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:51.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.734+0000 7f5ccbe00640 1 -- 172.21.15.32:0/3915991475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ccc135e80 con 0x7f5ccc15b7d0 2024-04-17T09:53:51.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.734+0000 7f5ccbe00640 1 --2- 172.21.15.32:0/3915991475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ccc15b7d0 0x7f5ccc159de0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400cad0 tx=0x7f5cc400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:51.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.734+0000 7f5cc3400640 1 -- 172.21.15.32:0/3915991475 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4017070 con 0x7f5ccc15b7d0 2024-04-17T09:53:51.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.734+0000 7f5cc3400640 1 -- 172.21.15.32:0/3915991475 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cc400ac30 con 0x7f5ccc15b7d0 2024-04-17T09:53:51.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.734+0000 7f5cc3400640 1 -- 172.21.15.32:0/3915991475 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4012400 con 0x7f5ccc15b7d0 2024-04-17T09:53:51.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.734+0000 7f5cd2400640 1 -- 172.21.15.32:0/3915991475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ccc136160 con 0x7f5ccc15b7d0 2024-04-17T09:53:51.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.734+0000 7f5cd2400640 1 -- 172.21.15.32:0/3915991475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ccc136720 con 0x7f5ccc15b7d0 2024-04-17T09:53:51.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.734+0000 7f5cd2400640 1 -- 172.21.15.32:0/3915991475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ccc068a90 con 0x7f5ccc15b7d0 2024-04-17T09:53:51.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.734+0000 7f5cc3400640 1 -- 172.21.15.32:0/3915991475 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5cc401a070 con 0x7f5ccc15b7d0 2024-04-17T09:53:51.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.734+0000 7f5cc3400640 1 --2- 172.21.15.32:0/3915991475 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ca8068380 0x7f5ca806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:51.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.734+0000 7f5cc3400640 1 -- 172.21.15.32:0/3915991475 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5cc4086770 con 0x7f5ccc15b7d0 2024-04-17T09:53:51.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.738+0000 7f5ccb400640 1 --2- 172.21.15.32:0/3915991475 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ca8068380 0x7f5ca806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:51.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.738+0000 7f5ccb400640 1 --2- 172.21.15.32:0/3915991475 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ca8068380 0x7f5ca806a840 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc007fd0 tx=0x7f5cbc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:51.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.738+0000 7f5cc3400640 1 -- 172.21.15.32:0/3915991475 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5cc4056650 con 0x7f5ccc15b7d0 2024-04-17T09:53:51.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.902+0000 7f5cd2400640 1 -- 172.21.15.32:0/3915991475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5ccc0b2980 con 0x7f5ccc15b7d0 2024-04-17T09:53:51.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.906+0000 7f5cc3400640 1 -- 172.21.15.32:0/3915991475 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f5cc405a950 con 0x7f5ccc15b7d0 2024-04-17T09:53:51.905 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:53:51.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.906+0000 7f5cd2400640 1 -- 172.21.15.32:0/3915991475 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ca8068380 msgr2=0x7f5ca806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:51.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.906+0000 7f5cd2400640 1 --2- 172.21.15.32:0/3915991475 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ca8068380 0x7f5ca806a840 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc007fd0 tx=0x7f5cbc009040 comp rx=0 tx=0).stop 2024-04-17T09:53:51.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.906+0000 7f5cd2400640 1 -- 172.21.15.32:0/3915991475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ccc15b7d0 msgr2=0x7f5ccc159de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:51.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.906+0000 7f5cd2400640 1 --2- 172.21.15.32:0/3915991475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ccc15b7d0 0x7f5ccc159de0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400cad0 tx=0x7f5cc400cf90 comp rx=0 tx=0).stop 2024-04-17T09:53:51.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.906+0000 7f5cd2400640 1 -- 172.21.15.32:0/3915991475 shutdown_connections 2024-04-17T09:53:51.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.906+0000 7f5cd2400640 1 --2- 172.21.15.32:0/3915991475 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ca8068380 0x7f5ca806a840 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:51.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.906+0000 7f5cd2400640 1 --2- 172.21.15.32:0/3915991475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ccc06f7c0 0x7f5ccc0ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:51.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.906+0000 7f5cd2400640 1 --2- 172.21.15.32:0/3915991475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ccc15b7d0 0x7f5ccc159de0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:51.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.906+0000 7f5cd2400640 1 --2- 172.21.15.32:0/3915991475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ccc06f270 0x7f5ccc151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:51.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.906+0000 7f5cd2400640 1 -- 172.21.15.32:0/3915991475 >> 172.21.15.32:0/3915991475 conn(0x7f5ccc0b2120 msgr2=0x7f5ccc155da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:51.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.906+0000 7f5cd2400640 1 -- 172.21.15.32:0/3915991475 shutdown_connections 2024-04-17T09:53:51.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:51.906+0000 7f5cd2400640 1 -- 172.21.15.32:0/3915991475 wait complete. 2024-04-17T09:53:51.927 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:53:52.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.070+0000 7f44e6400640 1 Processor -- start 2024-04-17T09:53:52.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.070+0000 7f44e6400640 1 -- start start 2024-04-17T09:53:52.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.070+0000 7f44e6400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e015c860 0x7f44e00d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:52.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.070+0000 7f44e6400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f44e009b100 0x7f44e00b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:52.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.070+0000 7f44e6400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44e015cc40 con 0x7f44e00ce740 2024-04-17T09:53:52.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.070+0000 7f44e6400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f44e00ce2b0 con 0x7f44e009b100 2024-04-17T09:53:52.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.070+0000 7f44e6400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f44e00cd6a0 con 0x7f44e015c860 2024-04-17T09:53:52.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.070+0000 7f44dfe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f44e009b100 0x7f44e00b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:52.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.070+0000 7f44dfe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f44e009b100 0x7f44e00b53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44670/0 (socket says 172.21.15.32:44670) 2024-04-17T09:53:52.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.070+0000 7f44dfe00640 1 -- 172.21.15.32:0/3218618769 learned_addr learned my addr 172.21.15.32:0/3218618769 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:52.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.070+0000 7f44df400640 1 --2- 172.21.15.32:0/3218618769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e015c860 0x7f44e00d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:52.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.070+0000 7f44dfe00640 1 -- 172.21.15.32:0/3218618769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e015c860 msgr2=0x7f44e00d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:52.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.070+0000 7f44dfe00640 1 --2- 172.21.15.32:0/3218618769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e015c860 0x7f44e00d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.070+0000 7f44dfe00640 1 -- 172.21.15.32:0/3218618769 >> v1:172.21.15.144:6789/0 conn(0x7f44e00ce740 legacy=0x7f44e009a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:52.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.070+0000 7f44dfe00640 1 -- 172.21.15.32:0/3218618769 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e00cd980 con 0x7f44e009b100 2024-04-17T09:53:52.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.074+0000 7f44dfe00640 1 --2- 172.21.15.32:0/3218618769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e009b100 0x7f44e00b53b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009e50 tx=0x7f44d801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c7f12f7a37614d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:52.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.074+0000 7f44dea00640 1 -- 172.21.15.32:0/3218618769 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d80a58d0 con 0x7f44e009b100 2024-04-17T09:53:52.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.074+0000 7f44dea00640 1 -- 172.21.15.32:0/3218618769 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44d80a5a70 con 0x7f44e009b100 2024-04-17T09:53:52.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.074+0000 7f44dea00640 1 -- 172.21.15.32:0/3218618769 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d80a5e40 con 0x7f44e009b100 2024-04-17T09:53:52.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.074+0000 7f44e6400640 1 -- 172.21.15.32:0/3218618769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e009b100 msgr2=0x7f44e00b53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:52.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.074+0000 7f44e6400640 1 --2- 172.21.15.32:0/3218618769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e009b100 0x7f44e00b53b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009e50 tx=0x7f44d801ca00 comp rx=0 tx=0).stop 2024-04-17T09:53:52.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.074+0000 7f44e6400640 1 -- 172.21.15.32:0/3218618769 shutdown_connections 2024-04-17T09:53:52.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.074+0000 7f44e6400640 1 --2- 172.21.15.32:0/3218618769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e009b100 0x7f44e00b53b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.074+0000 7f44e6400640 1 --2- 172.21.15.32:0/3218618769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e015c860 0x7f44e00d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.074+0000 7f44e6400640 1 -- 172.21.15.32:0/3218618769 >> 172.21.15.32:0/3218618769 conn(0x7f44e00b4050 msgr2=0x7f44e00b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:52.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.074+0000 7f44e6400640 1 -- 172.21.15.32:0/3218618769 shutdown_connections 2024-04-17T09:53:52.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.074+0000 7f44e6400640 1 -- 172.21.15.32:0/3218618769 wait complete. 2024-04-17T09:53:52.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44e6400640 1 Processor -- start 2024-04-17T09:53:52.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44e6400640 1 -- start start 2024-04-17T09:53:52.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44e6400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e009b100 0x7f44e01561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:52.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44e6400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e00ce740 0x7f44e015d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:52.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44e6400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e015c860 0x7f44e006e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:52.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44e6400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f44e00d1590 con 0x7f44e015c860 2024-04-17T09:53:52.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44e6400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f44e00cd350 con 0x7f44e00ce740 2024-04-17T09:53:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44e6400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f44e00cada0 con 0x7f44e009b100 2024-04-17T09:53:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44dfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e015c860 0x7f44e006e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44dfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e015c860 0x7f44e006e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44672/0 (socket says 172.21.15.32:44672) 2024-04-17T09:53:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44dfe00640 1 -- 172.21.15.32:0/1993688704 learned_addr learned my addr 172.21.15.32:0/1993688704 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44df400640 1 --2- 172.21.15.32:0/1993688704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e009b100 0x7f44e01561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44d6a00640 1 --2- 172.21.15.32:0/1993688704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e00ce740 0x7f44e015d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44dfe00640 1 -- 172.21.15.32:0/1993688704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e009b100 msgr2=0x7f44e01561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44dfe00640 1 --2- 172.21.15.32:0/1993688704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e009b100 0x7f44e01561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44dfe00640 1 -- 172.21.15.32:0/1993688704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e00ce740 msgr2=0x7f44e015d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44dfe00640 1 --2- 172.21.15.32:0/1993688704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e00ce740 0x7f44e015d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44dfe00640 1 -- 172.21.15.32:0/1993688704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e006d0b0 con 0x7f44e015c860 2024-04-17T09:53:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44df400640 1 --2- 172.21.15.32:0/1993688704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e009b100 0x7f44e01561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44d6a00640 1 --2- 172.21.15.32:0/1993688704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e00ce740 0x7f44e015d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44dfe00640 1 --2- 172.21.15.32:0/1993688704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e015c860 0x7f44e006e190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f44d801a630 tx=0x7f44d80197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44d7e00640 1 -- 172.21.15.32:0/1993688704 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d80a58d0 con 0x7f44e015c860 2024-04-17T09:53:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44d7e00640 1 -- 172.21.15.32:0/1993688704 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44d80be070 con 0x7f44e015c860 2024-04-17T09:53:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44d7e00640 1 -- 172.21.15.32:0/1993688704 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d8009510 con 0x7f44e015c860 2024-04-17T09:53:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44e6400640 1 -- 172.21.15.32:0/1993688704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e006d390 con 0x7f44e015c860 2024-04-17T09:53:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.078+0000 7f44e6400640 1 -- 172.21.15.32:0/1993688704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e006d950 con 0x7f44e015c860 2024-04-17T09:53:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.082+0000 7f44e6400640 1 -- 172.21.15.32:0/1993688704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44e0068a90 con 0x7f44e015c860 2024-04-17T09:53:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.082+0000 7f44d7e00640 1 -- 172.21.15.32:0/1993688704 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f44d80b86b0 con 0x7f44e015c860 2024-04-17T09:53:52.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.082+0000 7f44d7e00640 1 --2- 172.21.15.32:0/1993688704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f44bc068380 0x7f44bc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:52.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.082+0000 7f44d7e00640 1 -- 172.21.15.32:0/1993688704 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f44d8122080 con 0x7f44e015c860 2024-04-17T09:53:52.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.082+0000 7f44df400640 1 --2- 172.21.15.32:0/1993688704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f44bc068380 0x7f44bc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:52.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.082+0000 7f44df400640 1 --2- 172.21.15.32:0/1993688704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f44bc068380 0x7f44bc06a840 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f44d0007a70 tx=0x7f44d0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:52.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.086+0000 7f44d7e00640 1 -- 172.21.15.32:0/1993688704 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f44d8016620 con 0x7f44e015c860 2024-04-17T09:53:52.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.254+0000 7f44e6400640 1 -- 172.21.15.32:0/1993688704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f44e0158030 con 0x7f44e015c860 2024-04-17T09:53:52.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.254+0000 7f44d7e00640 1 -- 172.21.15.32:0/1993688704 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f44d80f2660 con 0x7f44e015c860 2024-04-17T09:53:52.255 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:53:52.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.258+0000 7f44e6400640 1 -- 172.21.15.32:0/1993688704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f44bc068380 msgr2=0x7f44bc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:52.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.258+0000 7f44e6400640 1 --2- 172.21.15.32:0/1993688704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f44bc068380 0x7f44bc06a840 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f44d0007a70 tx=0x7f44d0008040 comp rx=0 tx=0).stop 2024-04-17T09:53:52.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.258+0000 7f44e6400640 1 -- 172.21.15.32:0/1993688704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e015c860 msgr2=0x7f44e006e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:52.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.258+0000 7f44e6400640 1 --2- 172.21.15.32:0/1993688704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e015c860 0x7f44e006e190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f44d801a630 tx=0x7f44d80197a0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.258+0000 7f44e6400640 1 -- 172.21.15.32:0/1993688704 shutdown_connections 2024-04-17T09:53:52.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.258+0000 7f44e6400640 1 --2- 172.21.15.32:0/1993688704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f44bc068380 0x7f44bc06a840 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.258+0000 7f44e6400640 1 --2- 172.21.15.32:0/1993688704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44e015c860 0x7f44e006e190 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.258+0000 7f44e6400640 1 --2- 172.21.15.32:0/1993688704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e00ce740 0x7f44e015d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.258+0000 7f44e6400640 1 --2- 172.21.15.32:0/1993688704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e009b100 0x7f44e01561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.258+0000 7f44e6400640 1 -- 172.21.15.32:0/1993688704 >> 172.21.15.32:0/1993688704 conn(0x7f44e00b4050 msgr2=0x7f44e00b6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:52.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.258+0000 7f44e6400640 1 -- 172.21.15.32:0/1993688704 shutdown_connections 2024-04-17T09:53:52.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.258+0000 7f44e6400640 1 -- 172.21.15.32:0/1993688704 wait complete. 2024-04-17T09:53:52.276 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:53:52.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.418+0000 7f23e7200640 1 Processor -- start 2024-04-17T09:53:52.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.418+0000 7f23e7200640 1 -- start start 2024-04-17T09:53:52.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.422+0000 7f23e7200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f23e80721b0 0x7f23e8072660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:52.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.422+0000 7f23e7200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23e80aefa0 0x7f23e80af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:52.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.422+0000 7f23e7200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23e80ae520 con 0x7f23e80b5370 2024-04-17T09:53:52.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.422+0000 7f23e7200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f23e80b5750 con 0x7f23e80721b0 2024-04-17T09:53:52.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.422+0000 7f23e7200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f23e8162dc0 con 0x7f23e80aefa0 2024-04-17T09:53:52.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.422+0000 7f23e5e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f23e80b5370 0x7f23e8162340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54084/0 (socket says 172.21.15.32:54084) 2024-04-17T09:53:52.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.422+0000 7f23e5e00640 1 -- 172.21.15.32:0/3756702113 learned_addr learned my addr 172.21.15.32:0/3756702113 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:52.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.422+0000 7f23e6800640 1 --2- 172.21.15.32:0/3756702113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23e80aefa0 0x7f23e80af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:52.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.422+0000 7f23dd400640 1 --2- 172.21.15.32:0/3756702113 >> v2:172.21.15.144:3300/0 conn(0x7f23e80721b0 0x7f23e8072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:52.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.422+0000 7f23e5400640 1 -- 172.21.15.32:0/3756702113 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3000085831 0 0) 0x7f23d80014a0 con 0x7f23e80b5370 2024-04-17T09:53:52.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.422+0000 7f23e5400640 1 -- 172.21.15.32:0/3756702113 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23cc003530 con 0x7f23e80b5370 2024-04-17T09:53:52.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.422+0000 7f23e6800640 1 -- 172.21.15.32:0/3756702113 >> v2:172.21.15.144:3300/0 conn(0x7f23e80721b0 msgr2=0x7f23e8072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:52.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.422+0000 7f23e6800640 1 --2- 172.21.15.32:0/3756702113 >> v2:172.21.15.144:3300/0 conn(0x7f23e80721b0 0x7f23e8072660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.422+0000 7f23e6800640 1 -- 172.21.15.32:0/3756702113 >> v1:172.21.15.32:6789/0 conn(0x7f23e80b5370 legacy=0x7f23e8162340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:52.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.422+0000 7f23e5400640 1 -- 172.21.15.32:0/3756702113 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 424386667 0 0) 0x7f23cc003530 con 0x7f23e80b5370 2024-04-17T09:53:52.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.422+0000 7f23e6800640 1 -- 172.21.15.32:0/3756702113 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e81630a0 con 0x7f23e80aefa0 2024-04-17T09:53:52.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.422+0000 7f23dd400640 1 --2- 172.21.15.32:0/3756702113 >> v2:172.21.15.144:3300/0 conn(0x7f23e80721b0 0x7f23e8072660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:53:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.426+0000 7f23e6800640 1 --2- 172.21.15.32:0/3756702113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23e80aefa0 0x7f23e80af3f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f23e00097c0 tx=0x7f23e001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e38e2fb80988aebd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:52.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.426+0000 7f23e5400640 1 -- 172.21.15.32:0/3756702113 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e00b0070 con 0x7f23e80aefa0 2024-04-17T09:53:52.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.426+0000 7f23e5400640 1 -- 172.21.15.32:0/3756702113 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e00ab600 con 0x7f23e80aefa0 2024-04-17T09:53:52.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.426+0000 7f23e5400640 1 -- 172.21.15.32:0/3756702113 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e00ab9d0 con 0x7f23e80aefa0 2024-04-17T09:53:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.426+0000 7f23e7200640 1 -- 172.21.15.32:0/3756702113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23e80aefa0 msgr2=0x7f23e80af3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.426+0000 7f23e7200640 1 --2- 172.21.15.32:0/3756702113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23e80aefa0 0x7f23e80af3f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f23e00097c0 tx=0x7f23e001c4a0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.426+0000 7f23e7200640 1 -- 172.21.15.32:0/3756702113 shutdown_connections 2024-04-17T09:53:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.426+0000 7f23e7200640 1 --2- 172.21.15.32:0/3756702113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23e80aefa0 0x7f23e80af3f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.426+0000 7f23e7200640 1 --2- 172.21.15.32:0/3756702113 >> v2:172.21.15.144:3300/0 conn(0x7f23e80721b0 0x7f23e8072660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.426+0000 7f23e7200640 1 -- 172.21.15.32:0/3756702113 >> 172.21.15.32:0/3756702113 conn(0x7f23e806c1c0 msgr2=0x7f23e80b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.426+0000 7f23e7200640 1 -- 172.21.15.32:0/3756702113 shutdown_connections 2024-04-17T09:53:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.426+0000 7f23e7200640 1 -- 172.21.15.32:0/3756702113 wait complete. 2024-04-17T09:53:52.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.426+0000 7f23e7200640 1 Processor -- start 2024-04-17T09:53:52.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.426+0000 7f23e7200640 1 -- start start 2024-04-17T09:53:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.430+0000 7f23e7200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23e80721b0 0x7f23e8146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.430+0000 7f23e7200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23e80aefa0 0x7f23e8159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.430+0000 7f23e7200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e80b5370 0x7f23e8156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.430+0000 7f23e7200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f23e816b6e0 con 0x7f23e80b5370 2024-04-17T09:53:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.430+0000 7f23e7200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f23e815f000 con 0x7f23e80aefa0 2024-04-17T09:53:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.430+0000 7f23e7200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f23e816b860 con 0x7f23e80721b0 2024-04-17T09:53:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.430+0000 7f23e6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e80b5370 0x7f23e8156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.430+0000 7f23e6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e80b5370 0x7f23e8156be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44676/0 (socket says 172.21.15.32:44676) 2024-04-17T09:53:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.430+0000 7f23e6800640 1 -- 172.21.15.32:0/370373466 learned_addr learned my addr 172.21.15.32:0/370373466 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:52.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.430+0000 7f23e5e00640 1 --2- 172.21.15.32:0/370373466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23e80721b0 0x7f23e8146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:52.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.430+0000 7f23dd400640 1 --2- 172.21.15.32:0/370373466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23e80aefa0 0x7f23e8159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:52.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.430+0000 7f23e6800640 1 -- 172.21.15.32:0/370373466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23e80721b0 msgr2=0x7f23e8146100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:52.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.430+0000 7f23e6800640 1 --2- 172.21.15.32:0/370373466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23e80721b0 0x7f23e8146100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.430+0000 7f23e6800640 1 -- 172.21.15.32:0/370373466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23e80aefa0 msgr2=0x7f23e8159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:52.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.430+0000 7f23e6800640 1 --2- 172.21.15.32:0/370373466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23e80aefa0 0x7f23e8159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.430+0000 7f23e6800640 1 -- 172.21.15.32:0/370373466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e8159d90 con 0x7f23e80b5370 2024-04-17T09:53:52.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.430+0000 7f23e6800640 1 --2- 172.21.15.32:0/370373466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e80b5370 0x7f23e8156be0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f23e00097c0 tx=0x7f23e001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:52.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.430+0000 7f23dea00640 1 -- 172.21.15.32:0/370373466 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e00b0050 con 0x7f23e80b5370 2024-04-17T09:53:52.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.430+0000 7f23dea00640 1 -- 172.21.15.32:0/370373466 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e00bc070 con 0x7f23e80b5370 2024-04-17T09:53:52.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.430+0000 7f23e7200640 1 -- 172.21.15.32:0/370373466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23e815a070 con 0x7f23e80b5370 2024-04-17T09:53:52.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.430+0000 7f23dea00640 1 -- 172.21.15.32:0/370373466 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e00b7650 con 0x7f23e80b5370 2024-04-17T09:53:52.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.430+0000 7f23e7200640 1 -- 172.21.15.32:0/370373466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23e815a630 con 0x7f23e80b5370 2024-04-17T09:53:52.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.430+0000 7f23e7200640 1 -- 172.21.15.32:0/370373466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23e804fda0 con 0x7f23e80b5370 2024-04-17T09:53:52.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.434+0000 7f23dea00640 1 -- 172.21.15.32:0/370373466 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f23e00b77f0 con 0x7f23e80b5370 2024-04-17T09:53:52.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.434+0000 7f23dea00640 1 --2- 172.21.15.32:0/370373466 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f23c4068380 0x7f23c406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:52.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.434+0000 7f23dea00640 1 -- 172.21.15.32:0/370373466 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f23e01210b0 con 0x7f23e80b5370 2024-04-17T09:53:52.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.434+0000 7f23e5e00640 1 --2- 172.21.15.32:0/370373466 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f23c4068380 0x7f23c406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:52.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.434+0000 7f23e5e00640 1 --2- 172.21.15.32:0/370373466 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f23c4068380 0x7f23c406a840 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f23d8004fd0 tx=0x7f23d80012a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:52.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.438+0000 7f23dea00640 1 -- 172.21.15.32:0/370373466 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f23e0016090 con 0x7f23e80b5370 2024-04-17T09:53:52.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.606+0000 7f23e7200640 1 -- 172.21.15.32:0/370373466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f23e81587b0 con 0x7f23e80b5370 2024-04-17T09:53:52.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.606+0000 7f23dea00640 1 -- 172.21.15.32:0/370373466 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f23e00f1150 con 0x7f23e80b5370 2024-04-17T09:53:52.607 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:53:52.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.610+0000 7f23e7200640 1 -- 172.21.15.32:0/370373466 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f23c4068380 msgr2=0x7f23c406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:52.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.610+0000 7f23e7200640 1 --2- 172.21.15.32:0/370373466 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f23c4068380 0x7f23c406a840 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f23d8004fd0 tx=0x7f23d80012a0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.610+0000 7f23e7200640 1 -- 172.21.15.32:0/370373466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e80b5370 msgr2=0x7f23e8156be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:52.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.610+0000 7f23e7200640 1 --2- 172.21.15.32:0/370373466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e80b5370 0x7f23e8156be0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f23e00097c0 tx=0x7f23e001ace0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.610+0000 7f23e7200640 1 -- 172.21.15.32:0/370373466 shutdown_connections 2024-04-17T09:53:52.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.610+0000 7f23e7200640 1 --2- 172.21.15.32:0/370373466 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f23c4068380 0x7f23c406a840 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.610+0000 7f23e7200640 1 --2- 172.21.15.32:0/370373466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e80b5370 0x7f23e8156be0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.610+0000 7f23e7200640 1 --2- 172.21.15.32:0/370373466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23e80aefa0 0x7f23e8159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.610+0000 7f23e7200640 1 --2- 172.21.15.32:0/370373466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23e80721b0 0x7f23e8146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.610+0000 7f23e7200640 1 -- 172.21.15.32:0/370373466 >> 172.21.15.32:0/370373466 conn(0x7f23e806c1c0 msgr2=0x7f23e815ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:52.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.610+0000 7f23e7200640 1 -- 172.21.15.32:0/370373466 shutdown_connections 2024-04-17T09:53:52.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.610+0000 7f23e7200640 1 -- 172.21.15.32:0/370373466 wait complete. 2024-04-17T09:53:52.629 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T09:53:52.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.770+0000 7f2e39a00640 1 Processor -- start 2024-04-17T09:53:52.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.770+0000 7f2e39a00640 1 -- start start 2024-04-17T09:53:52.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.770+0000 7f2e39a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e3415b420 0x7f2e3406e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:52.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.770+0000 7f2e39a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e340b40f0 con 0x7f2e3415bd90 2024-04-17T09:53:52.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.770+0000 7f2e39a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e340b43d0 con 0x7f2e34072490 2024-04-17T09:53:52.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.770+0000 7f2e39a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2e340b46b0 con 0x7f2e3415b420 2024-04-17T09:53:52.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.770+0000 7f2e32a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2e3415bd90 0x7f2e34071d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54096/0 (socket says 172.21.15.32:54096) 2024-04-17T09:53:52.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.770+0000 7f2e32a00640 1 -- 172.21.15.32:0/812374558 learned_addr learned my addr 172.21.15.32:0/812374558 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:52.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.770+0000 7f2e33400640 1 --2- 172.21.15.32:0/812374558 >> v2:172.21.15.32:3300/0 conn(0x7f2e3415b420 0x7f2e3406e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.770+0000 7f2e32000640 1 -- 172.21.15.32:0/812374558 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2491780451 0 0) 0x7f2e340b40f0 con 0x7f2e3415bd90 2024-04-17T09:53:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.770+0000 7f2e32000640 1 -- 172.21.15.32:0/812374558 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e18003450 con 0x7f2e3415bd90 2024-04-17T09:53:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.770+0000 7f2e32000640 1 -- 172.21.15.32:0/812374558 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1771974324 0 0) 0x7f2e340b43d0 con 0x7f2e34072490 2024-04-17T09:53:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.770+0000 7f2e33400640 1 -- 172.21.15.32:0/812374558 >> v1:172.21.15.144:6789/0 conn(0x7f2e34072490 legacy=0x7f2e3406e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.770+0000 7f2e33400640 1 -- 172.21.15.32:0/812374558 >> v1:172.21.15.32:6789/0 conn(0x7f2e3415bd90 legacy=0x7f2e34071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.770+0000 7f2e33400640 1 -- 172.21.15.32:0/812374558 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e340b4990 con 0x7f2e3415b420 2024-04-17T09:53:52.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.774+0000 7f2e33400640 1 --2- 172.21.15.32:0/812374558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3415b420 0x7f2e3406e7b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0097c0 tx=0x7f2e3406f500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d4d2745939dbed server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:52.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.774+0000 7f2e32000640 1 -- 172.21.15.32:0/812374558 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c0b0070 con 0x7f2e3415b420 2024-04-17T09:53:52.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.774+0000 7f2e32000640 1 -- 172.21.15.32:0/812374558 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e2c0ab600 con 0x7f2e3415b420 2024-04-17T09:53:52.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.774+0000 7f2e32000640 1 -- 172.21.15.32:0/812374558 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c0ab9d0 con 0x7f2e3415b420 2024-04-17T09:53:52.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.774+0000 7f2e39a00640 1 -- 172.21.15.32:0/812374558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3415b420 msgr2=0x7f2e3406e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:52.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.774+0000 7f2e39a00640 1 --2- 172.21.15.32:0/812374558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3415b420 0x7f2e3406e7b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0097c0 tx=0x7f2e3406f500 comp rx=0 tx=0).stop 2024-04-17T09:53:52.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.774+0000 7f2e39a00640 1 -- 172.21.15.32:0/812374558 shutdown_connections 2024-04-17T09:53:52.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.774+0000 7f2e39a00640 1 --2- 172.21.15.32:0/812374558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3415b420 0x7f2e3406e7b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.774+0000 7f2e39a00640 1 -- 172.21.15.32:0/812374558 >> 172.21.15.32:0/812374558 conn(0x7f2e3406ba30 msgr2=0x7f2e3406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:52.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.774+0000 7f2e39a00640 1 -- 172.21.15.32:0/812374558 shutdown_connections 2024-04-17T09:53:52.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.774+0000 7f2e39a00640 1 -- 172.21.15.32:0/812374558 wait complete. 2024-04-17T09:53:52.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e39a00640 1 Processor -- start 2024-04-17T09:53:52.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e39a00640 1 -- start start 2024-04-17T09:53:52.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e39a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e34072490 0x7f2e34164510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:52.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e39a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3415b420 0x7f2e3414e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:52.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e39a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e3415bd90 0x7f2e340b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:52.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e39a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2e340d04d0 con 0x7f2e3415b420 2024-04-17T09:53:52.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e39a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2e340d0350 con 0x7f2e3415bd90 2024-04-17T09:53:52.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e39a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2e340d0650 con 0x7f2e34072490 2024-04-17T09:53:52.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e2a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3415b420 0x7f2e3414e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:52.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e2a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3415b420 0x7f2e3414e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44700/0 (socket says 172.21.15.32:44700) 2024-04-17T09:53:52.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e2a000640 1 -- 172.21.15.32:0/2280978111 learned_addr learned my addr 172.21.15.32:0/2280978111 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:52.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e33400640 1 --2- 172.21.15.32:0/2280978111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e3415bd90 0x7f2e340b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:52.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e32a00640 1 --2- 172.21.15.32:0/2280978111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e34072490 0x7f2e34164510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:52.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e2a000640 1 -- 172.21.15.32:0/2280978111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e34072490 msgr2=0x7f2e34164510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:52.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e2a000640 1 --2- 172.21.15.32:0/2280978111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e34072490 0x7f2e34164510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e2a000640 1 -- 172.21.15.32:0/2280978111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e3415bd90 msgr2=0x7f2e340b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:52.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e2a000640 1 --2- 172.21.15.32:0/2280978111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e3415bd90 0x7f2e340b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e2a000640 1 -- 172.21.15.32:0/2280978111 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e34153c50 con 0x7f2e3415b420 2024-04-17T09:53:52.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e2a000640 1 --2- 172.21.15.32:0/2280978111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3415b420 0x7f2e3414e7d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00c480 tx=0x7f2e1c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:52.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e2b400640 1 -- 172.21.15.32:0/2280978111 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c005480 con 0x7f2e3415b420 2024-04-17T09:53:52.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e2b400640 1 -- 172.21.15.32:0/2280978111 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e1c003600 con 0x7f2e3415b420 2024-04-17T09:53:52.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e39a00640 1 -- 172.21.15.32:0/2280978111 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e34153f30 con 0x7f2e3415b420 2024-04-17T09:53:52.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e2b400640 1 -- 172.21.15.32:0/2280978111 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0039f0 con 0x7f2e3415b420 2024-04-17T09:53:52.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.778+0000 7f2e39a00640 1 -- 172.21.15.32:0/2280978111 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e341544f0 con 0x7f2e3415b420 2024-04-17T09:53:52.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.782+0000 7f2e39a00640 1 -- 172.21.15.32:0/2280978111 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e34068a90 con 0x7f2e3415b420 2024-04-17T09:53:52.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.782+0000 7f2e2b400640 1 -- 172.21.15.32:0/2280978111 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2e1c003b90 con 0x7f2e3415b420 2024-04-17T09:53:52.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.782+0000 7f2e2b400640 1 --2- 172.21.15.32:0/2280978111 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e10068380 0x7f2e1006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:52.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.782+0000 7f2e2b400640 1 -- 172.21.15.32:0/2280978111 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2e1c086660 con 0x7f2e3415b420 2024-04-17T09:53:52.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.782+0000 7f2e32a00640 1 --2- 172.21.15.32:0/2280978111 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e10068380 0x7f2e1006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:52.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.782+0000 7f2e32a00640 1 --2- 172.21.15.32:0/2280978111 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e10068380 0x7f2e1006a840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2e24005c10 tx=0x7f2e24009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:52.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.786+0000 7f2e2b400640 1 -- 172.21.15.32:0/2280978111 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2e1c056c10 con 0x7f2e3415b420 2024-04-17T09:53:52.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.966+0000 7f2e39a00640 1 -- 172.21.15.32:0/2280978111 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2e340cc850 con 0x7f2e3415b420 2024-04-17T09:53:52.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.966+0000 7f2e2b400640 1 -- 172.21.15.32:0/2280978111 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f2e1c056510 con 0x7f2e3415b420 2024-04-17T09:53:52.968 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T09:53:52.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.970+0000 7f2e39a00640 1 -- 172.21.15.32:0/2280978111 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e10068380 msgr2=0x7f2e1006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:52.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.970+0000 7f2e39a00640 1 --2- 172.21.15.32:0/2280978111 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e10068380 0x7f2e1006a840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2e24005c10 tx=0x7f2e24009040 comp rx=0 tx=0).stop 2024-04-17T09:53:52.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.970+0000 7f2e39a00640 1 -- 172.21.15.32:0/2280978111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3415b420 msgr2=0x7f2e3414e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:52.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.970+0000 7f2e39a00640 1 --2- 172.21.15.32:0/2280978111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3415b420 0x7f2e3414e7d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00c480 tx=0x7f2e1c00c940 comp rx=0 tx=0).stop 2024-04-17T09:53:52.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.970+0000 7f2e39a00640 1 -- 172.21.15.32:0/2280978111 shutdown_connections 2024-04-17T09:53:52.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.970+0000 7f2e39a00640 1 --2- 172.21.15.32:0/2280978111 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e10068380 0x7f2e1006a840 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.970+0000 7f2e39a00640 1 --2- 172.21.15.32:0/2280978111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e3415bd90 0x7f2e340b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.970+0000 7f2e39a00640 1 --2- 172.21.15.32:0/2280978111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3415b420 0x7f2e3414e7d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.970+0000 7f2e39a00640 1 --2- 172.21.15.32:0/2280978111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e34072490 0x7f2e34164510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:52.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.970+0000 7f2e39a00640 1 -- 172.21.15.32:0/2280978111 >> 172.21.15.32:0/2280978111 conn(0x7f2e3406ba30 msgr2=0x7f2e34154ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:52.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.970+0000 7f2e39a00640 1 -- 172.21.15.32:0/2280978111 shutdown_connections 2024-04-17T09:53:52.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:52.970+0000 7f2e39a00640 1 -- 172.21.15.32:0/2280978111 wait complete. 2024-04-17T09:53:52.991 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:53:52.991 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:53:53.000 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:53:53.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.142+0000 7fb6cc200640 1 Processor -- start 2024-04-17T09:53:53.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.142+0000 7fb6cc200640 1 -- start start 2024-04-17T09:53:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.142+0000 7fb6cc200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6c4144e90 0x7fb6c40b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.142+0000 7fb6cc200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb6c40b4de0 0x7fb6c415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.142+0000 7fb6cc200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6c4072bb0 con 0x7fb6c415bfe0 2024-04-17T09:53:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.142+0000 7fb6cc200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb6c415c430 con 0x7fb6c40b4de0 2024-04-17T09:53:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.142+0000 7fb6cc200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb6c4145270 con 0x7fb6c4144e90 2024-04-17T09:53:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.142+0000 7fb6ca200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb6c415bfe0 0x7fb6c406ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54102/0 (socket says 172.21.15.32:54102) 2024-04-17T09:53:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.142+0000 7fb6ca200640 1 -- 172.21.15.32:0/3078304582 learned_addr learned my addr 172.21.15.32:0/3078304582 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.142+0000 7fb6c9800640 1 --2- 172.21.15.32:0/3078304582 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6c4144e90 0x7fb6c40b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.142+0000 7fb6c8e00640 1 --2- 172.21.15.32:0/3078304582 >> v2:172.21.15.144:3300/0 conn(0x7fb6c40b4de0 0x7fb6c415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.142+0000 7fb6c3e00640 1 -- 172.21.15.32:0/3078304582 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 542398489 0 0) 0x7fb6c4072bb0 con 0x7fb6c415bfe0 2024-04-17T09:53:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.142+0000 7fb6c3e00640 1 -- 172.21.15.32:0/3078304582 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6ac005740 con 0x7fb6c415bfe0 2024-04-17T09:53:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.142+0000 7fb6c9800640 1 -- 172.21.15.32:0/3078304582 >> v2:172.21.15.144:3300/0 conn(0x7fb6c40b4de0 msgr2=0x7fb6c415baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.142+0000 7fb6c9800640 1 --2- 172.21.15.32:0/3078304582 >> v2:172.21.15.144:3300/0 conn(0x7fb6c40b4de0 0x7fb6c415baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.142+0000 7fb6c9800640 1 -- 172.21.15.32:0/3078304582 >> v1:172.21.15.32:6789/0 conn(0x7fb6c415bfe0 legacy=0x7fb6c406ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.142+0000 7fb6c9800640 1 -- 172.21.15.32:0/3078304582 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c40b3dc0 con 0x7fb6c4144e90 2024-04-17T09:53:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.142+0000 7fb6c8e00640 1 --2- 172.21.15.32:0/3078304582 >> v2:172.21.15.144:3300/0 conn(0x7fb6c40b4de0 0x7fb6c415baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:53:53.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.146+0000 7fb6c9800640 1 --2- 172.21.15.32:0/3078304582 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6c4144e90 0x7fb6c40b48a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb6b80097c0 tx=0x7fb6b801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e02764d5f4001e46 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:53.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.146+0000 7fb6c3e00640 1 -- 172.21.15.32:0/3078304582 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b80aa5f0 con 0x7fb6c4144e90 2024-04-17T09:53:53.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.146+0000 7fb6c3e00640 1 -- 172.21.15.32:0/3078304582 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6b80af070 con 0x7fb6c4144e90 2024-04-17T09:53:53.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.146+0000 7fb6c3e00640 1 -- 172.21.15.32:0/3078304582 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b80aa9b0 con 0x7fb6c4144e90 2024-04-17T09:53:53.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.146+0000 7fb6cc200640 1 -- 172.21.15.32:0/3078304582 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6c4144e90 msgr2=0x7fb6c40b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:53.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.146+0000 7fb6cc200640 1 --2- 172.21.15.32:0/3078304582 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6c4144e90 0x7fb6c40b48a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb6b80097c0 tx=0x7fb6b801c470 comp rx=0 tx=0).stop 2024-04-17T09:53:53.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.146+0000 7fb6cc200640 1 -- 172.21.15.32:0/3078304582 shutdown_connections 2024-04-17T09:53:53.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.146+0000 7fb6cc200640 1 --2- 172.21.15.32:0/3078304582 >> v2:172.21.15.144:3300/0 conn(0x7fb6c40b4de0 0x7fb6c415baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.146+0000 7fb6cc200640 1 --2- 172.21.15.32:0/3078304582 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6c4144e90 0x7fb6c40b48a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.146+0000 7fb6cc200640 1 -- 172.21.15.32:0/3078304582 >> 172.21.15.32:0/3078304582 conn(0x7fb6c406ba30 msgr2=0x7fb6c406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:53.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.146+0000 7fb6cc200640 1 -- 172.21.15.32:0/3078304582 shutdown_connections 2024-04-17T09:53:53.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.146+0000 7fb6cc200640 1 -- 172.21.15.32:0/3078304582 wait complete. 2024-04-17T09:53:53.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6cc200640 1 Processor -- start 2024-04-17T09:53:53.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6cc200640 1 -- start start 2024-04-17T09:53:53.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6cc200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6c40b4de0 0x7fb6c4141240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:53.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6cc200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6c4144e90 0x7fb6c4160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:53.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6cc200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6c415bfe0 0x7fb6c4161960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:53.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6cc200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c4072670 con 0x7fb6c415bfe0 2024-04-17T09:53:53.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6cc200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c406f8c0 con 0x7fb6c4144e90 2024-04-17T09:53:53.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6cc200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb6c40727f0 con 0x7fb6c40b4de0 2024-04-17T09:53:53.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6ca200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6c415bfe0 0x7fb6c4161960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:53.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6ca200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6c415bfe0 0x7fb6c4161960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44706/0 (socket says 172.21.15.32:44706) 2024-04-17T09:53:53.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6ca200640 1 -- 172.21.15.32:0/397066598 learned_addr learned my addr 172.21.15.32:0/397066598 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:53.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6c9800640 1 --2- 172.21.15.32:0/397066598 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6c40b4de0 0x7fb6c4141240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:53.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6c8e00640 1 --2- 172.21.15.32:0/397066598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6c4144e90 0x7fb6c4160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:53.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6ca200640 1 -- 172.21.15.32:0/397066598 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6c40b4de0 msgr2=0x7fb6c4141240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:53.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6ca200640 1 --2- 172.21.15.32:0/397066598 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6c40b4de0 0x7fb6c4141240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6ca200640 1 -- 172.21.15.32:0/397066598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6c4144e90 msgr2=0x7fb6c4160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:53.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6ca200640 1 --2- 172.21.15.32:0/397066598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6c4144e90 0x7fb6c4160230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6ca200640 1 -- 172.21.15.32:0/397066598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c4160770 con 0x7fb6c415bfe0 2024-04-17T09:53:53.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6ca200640 1 --2- 172.21.15.32:0/397066598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6c415bfe0 0x7fb6c4161960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc00cad0 tx=0x7fb6bc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:53.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6c1600640 1 -- 172.21.15.32:0/397066598 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc017070 con 0x7fb6c415bfe0 2024-04-17T09:53:53.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6c1600640 1 -- 172.21.15.32:0/397066598 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6bc00ac30 con 0x7fb6c415bfe0 2024-04-17T09:53:53.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6c1600640 1 -- 172.21.15.32:0/397066598 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc012400 con 0x7fb6c415bfe0 2024-04-17T09:53:53.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6cc200640 1 -- 172.21.15.32:0/397066598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c4160fd0 con 0x7fb6c415bfe0 2024-04-17T09:53:53.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6cc200640 1 -- 172.21.15.32:0/397066598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c4160cf0 con 0x7fb6c415bfe0 2024-04-17T09:53:53.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.150+0000 7fb6cc200640 1 -- 172.21.15.32:0/397066598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c4068a90 con 0x7fb6c415bfe0 2024-04-17T09:53:53.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.154+0000 7fb6c1600640 1 -- 172.21.15.32:0/397066598 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb6bc01a070 con 0x7fb6c415bfe0 2024-04-17T09:53:53.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.154+0000 7fb6c1600640 1 --2- 172.21.15.32:0/397066598 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb6a4068380 0x7fb6a406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:53.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.154+0000 7fb6c1600640 1 -- 172.21.15.32:0/397066598 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb6bc086790 con 0x7fb6c415bfe0 2024-04-17T09:53:53.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.154+0000 7fb6c9800640 1 --2- 172.21.15.32:0/397066598 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb6a4068380 0x7fb6a406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:53.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.154+0000 7fb6c9800640 1 --2- 172.21.15.32:0/397066598 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb6a4068380 0x7fb6a406a840 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb6b80097c0 tx=0x7fb6b80a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:53.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.158+0000 7fb6c1600640 1 -- 172.21.15.32:0/397066598 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fb6bc056d40 con 0x7fb6c415bfe0 2024-04-17T09:53:53.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.326+0000 7fb6cc200640 1 -- 172.21.15.32:0/397066598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6c40af930 con 0x7fb6c415bfe0 2024-04-17T09:53:53.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.326+0000 7fb6c1600640 1 -- 172.21.15.32:0/397066598 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7fb6bc056640 con 0x7fb6c415bfe0 2024-04-17T09:53:53.327 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:53.328 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:14.779535+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:53:53.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.330+0000 7fb6cc200640 1 -- 172.21.15.32:0/397066598 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb6a4068380 msgr2=0x7fb6a406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:53.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.330+0000 7fb6cc200640 1 --2- 172.21.15.32:0/397066598 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb6a4068380 0x7fb6a406a840 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb6b80097c0 tx=0x7fb6b80a7000 comp rx=0 tx=0).stop 2024-04-17T09:53:53.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.330+0000 7fb6cc200640 1 -- 172.21.15.32:0/397066598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6c415bfe0 msgr2=0x7fb6c4161960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:53.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.330+0000 7fb6cc200640 1 --2- 172.21.15.32:0/397066598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6c415bfe0 0x7fb6c4161960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc00cad0 tx=0x7fb6bc00cf90 comp rx=0 tx=0).stop 2024-04-17T09:53:53.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.330+0000 7fb6cc200640 1 -- 172.21.15.32:0/397066598 shutdown_connections 2024-04-17T09:53:53.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.330+0000 7fb6cc200640 1 --2- 172.21.15.32:0/397066598 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb6a4068380 0x7fb6a406a840 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.330+0000 7fb6cc200640 1 --2- 172.21.15.32:0/397066598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6c415bfe0 0x7fb6c4161960 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.330+0000 7fb6cc200640 1 --2- 172.21.15.32:0/397066598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6c4144e90 0x7fb6c4160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.330+0000 7fb6cc200640 1 --2- 172.21.15.32:0/397066598 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6c40b4de0 0x7fb6c4141240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.330+0000 7fb6cc200640 1 -- 172.21.15.32:0/397066598 >> 172.21.15.32:0/397066598 conn(0x7fb6c406ba30 msgr2=0x7fb6c40cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:53.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.330+0000 7fb6cc200640 1 -- 172.21.15.32:0/397066598 shutdown_connections 2024-04-17T09:53:53.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.330+0000 7fb6cc200640 1 -- 172.21.15.32:0/397066598 wait complete. 2024-04-17T09:53:53.348 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:53:11.198098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:53:12.118982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:53:53.349 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:53:53.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.490+0000 7f91ef800640 1 Processor -- start 2024-04-17T09:53:53.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.490+0000 7f91ef800640 1 -- start start 2024-04-17T09:53:53.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.490+0000 7f91ef800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f91e806ba30 0x7f91e806be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:53.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.490+0000 7f91ef800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91e80cf370 0x7f91e80cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:53.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.490+0000 7f91ef800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91e806c350 con 0x7f91e8160350 2024-04-17T09:53:53.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.490+0000 7f91ef800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f91e80d0130 con 0x7f91e806ba30 2024-04-17T09:53:53.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.490+0000 7f91ef800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f91e815d090 con 0x7f91e80cf370 2024-04-17T09:53:53.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.490+0000 7f91ece00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f91e806ba30 0x7f91e806be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:53.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.490+0000 7f91ece00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f91e806ba30 0x7f91e806be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44710/0 (socket says 172.21.15.32:44710) 2024-04-17T09:53:53.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.490+0000 7f91ece00640 1 -- 172.21.15.32:0/1684833961 learned_addr learned my addr 172.21.15.32:0/1684833961 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:53.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.494+0000 7f91e7e00640 1 --2- 172.21.15.32:0/1684833961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91e80cf370 0x7f91e80cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:53.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.494+0000 7f91ece00640 1 -- 172.21.15.32:0/1684833961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91e80cf370 msgr2=0x7f91e80cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:53.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.494+0000 7f91ece00640 1 --2- 172.21.15.32:0/1684833961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91e80cf370 0x7f91e80cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.494+0000 7f91ece00640 1 -- 172.21.15.32:0/1684833961 >> v1:172.21.15.144:6789/0 conn(0x7f91e8160350 legacy=0x7f91e81607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:53.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.494+0000 7f91ece00640 1 -- 172.21.15.32:0/1684833961 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e815d370 con 0x7f91e806ba30 2024-04-17T09:53:53.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.494+0000 7f91e7e00640 1 --2- 172.21.15.32:0/1684833961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91e80cf370 0x7f91e80cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:53.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.494+0000 7f91ece00640 1 --2- 172.21.15.32:0/1684833961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e806ba30 0x7f91e806be10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f91dc009870 tx=0x7f91dc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b775a525bb86f85c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:53.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.494+0000 7f91e7400640 1 -- 172.21.15.32:0/1684833961 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91dc0ab5f0 con 0x7f91e806ba30 2024-04-17T09:53:53.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.494+0000 7f91e7400640 1 -- 172.21.15.32:0/1684833961 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91dc0b0070 con 0x7f91e806ba30 2024-04-17T09:53:53.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.494+0000 7f91e7400640 1 -- 172.21.15.32:0/1684833961 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91dc0ab9b0 con 0x7f91e806ba30 2024-04-17T09:53:53.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.498+0000 7f91ef800640 1 -- 172.21.15.32:0/1684833961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e806ba30 msgr2=0x7f91e806be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:53.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.498+0000 7f91ef800640 1 --2- 172.21.15.32:0/1684833961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e806ba30 0x7f91e806be10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f91dc009870 tx=0x7f91dc01c520 comp rx=0 tx=0).stop 2024-04-17T09:53:53.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.498+0000 7f91ef800640 1 -- 172.21.15.32:0/1684833961 shutdown_connections 2024-04-17T09:53:53.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.498+0000 7f91ef800640 1 --2- 172.21.15.32:0/1684833961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91e80cf370 0x7f91e80cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.498+0000 7f91ef800640 1 --2- 172.21.15.32:0/1684833961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e806ba30 0x7f91e806be10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.498+0000 7f91ef800640 1 -- 172.21.15.32:0/1684833961 >> 172.21.15.32:0/1684833961 conn(0x7f91e80b2120 msgr2=0x7f91e80b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:53.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.498+0000 7f91ef800640 1 -- 172.21.15.32:0/1684833961 shutdown_connections 2024-04-17T09:53:53.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.498+0000 7f91ef800640 1 -- 172.21.15.32:0/1684833961 wait complete. 2024-04-17T09:53:53.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.498+0000 7f91ef800640 1 Processor -- start 2024-04-17T09:53:53.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.498+0000 7f91ef800640 1 -- start start 2024-04-17T09:53:53.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.498+0000 7f91ef800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91e806ba30 0x7f91e81440e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:53.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.498+0000 7f91ef800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91e80cf370 0x7f91e8144620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:53.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.498+0000 7f91ef800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e8160350 0x7f91e80b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:53.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.498+0000 7f91ef800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f91e815dcf0 con 0x7f91e8160350 2024-04-17T09:53:53.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.498+0000 7f91ef800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f91e815db70 con 0x7f91e80cf370 2024-04-17T09:53:53.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.498+0000 7f91ef800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f91e8146230 con 0x7f91e806ba30 2024-04-17T09:53:53.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.498+0000 7f91ed800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e8160350 0x7f91e80b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:53.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.498+0000 7f91ed800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e8160350 0x7f91e80b37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44714/0 (socket says 172.21.15.32:44714) 2024-04-17T09:53:53.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.498+0000 7f91ed800640 1 -- 172.21.15.32:0/1363395504 learned_addr learned my addr 172.21.15.32:0/1363395504 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:53.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.498+0000 7f91e7e00640 1 --2- 172.21.15.32:0/1363395504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91e80cf370 0x7f91e8144620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:53.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.498+0000 7f91ece00640 1 --2- 172.21.15.32:0/1363395504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91e806ba30 0x7f91e81440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:53.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.502+0000 7f91ed800640 1 -- 172.21.15.32:0/1363395504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91e806ba30 msgr2=0x7f91e81440e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:53.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.502+0000 7f91ed800640 1 --2- 172.21.15.32:0/1363395504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91e806ba30 0x7f91e81440e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.502+0000 7f91ed800640 1 -- 172.21.15.32:0/1363395504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91e80cf370 msgr2=0x7f91e8144620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:53.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.502+0000 7f91ed800640 1 --2- 172.21.15.32:0/1363395504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91e80cf370 0x7f91e8144620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.502+0000 7f91ed800640 1 -- 172.21.15.32:0/1363395504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e80b4560 con 0x7f91e8160350 2024-04-17T09:53:53.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.502+0000 7f91ed800640 1 --2- 172.21.15.32:0/1363395504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e8160350 0x7f91e80b37d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f91e000c780 tx=0x7f91e000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:53.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.502+0000 7f91e4c00640 1 -- 172.21.15.32:0/1363395504 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e0004b70 con 0x7f91e8160350 2024-04-17T09:53:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.502+0000 7f91e4c00640 1 -- 172.21.15.32:0/1363395504 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91e0004d10 con 0x7f91e8160350 2024-04-17T09:53:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.502+0000 7f91ef800640 1 -- 172.21.15.32:0/1363395504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e80b4840 con 0x7f91e8160350 2024-04-17T09:53:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.502+0000 7f91e4c00640 1 -- 172.21.15.32:0/1363395504 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e000aa90 con 0x7f91e8160350 2024-04-17T09:53:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.502+0000 7f91ef800640 1 -- 172.21.15.32:0/1363395504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e80b4e00 con 0x7f91e8160350 2024-04-17T09:53:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.502+0000 7f91ef800640 1 -- 172.21.15.32:0/1363395504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91e8068a90 con 0x7f91e8160350 2024-04-17T09:53:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.502+0000 7f91e4c00640 1 -- 172.21.15.32:0/1363395504 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f91e000ac30 con 0x7f91e8160350 2024-04-17T09:53:53.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.502+0000 7f91e4c00640 1 --2- 172.21.15.32:0/1363395504 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f91c8068380 0x7f91c806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:53.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.502+0000 7f91e4c00640 1 -- 172.21.15.32:0/1363395504 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f91e0086ce0 con 0x7f91e8160350 2024-04-17T09:53:53.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.502+0000 7f91ece00640 1 --2- 172.21.15.32:0/1363395504 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f91c8068380 0x7f91c806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:53.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.506+0000 7f91ece00640 1 --2- 172.21.15.32:0/1363395504 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f91c8068380 0x7f91c806a840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f91dc0af000 tx=0x7f91dc0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:53.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.506+0000 7f91e4c00640 1 -- 172.21.15.32:0/1363395504 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f91e0020070 con 0x7f91e8160350 2024-04-17T09:53:53.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.674+0000 7f91ef800640 1 -- 172.21.15.32:0/1363395504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f91e806c850 con 0x7f91e8160350 2024-04-17T09:53:53.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.674+0000 7f91e4c00640 1 -- 172.21.15.32:0/1363395504 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f91e0057200 con 0x7f91e8160350 2024-04-17T09:53:53.673 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:53:53.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.674+0000 7f91ef800640 1 -- 172.21.15.32:0/1363395504 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f91c8068380 msgr2=0x7f91c806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:53.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.674+0000 7f91ef800640 1 --2- 172.21.15.32:0/1363395504 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f91c8068380 0x7f91c806a840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f91dc0af000 tx=0x7f91dc0a7000 comp rx=0 tx=0).stop 2024-04-17T09:53:53.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.674+0000 7f91ef800640 1 -- 172.21.15.32:0/1363395504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e8160350 msgr2=0x7f91e80b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:53.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.674+0000 7f91ef800640 1 --2- 172.21.15.32:0/1363395504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e8160350 0x7f91e80b37d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f91e000c780 tx=0x7f91e000cc40 comp rx=0 tx=0).stop 2024-04-17T09:53:53.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.674+0000 7f91ef800640 1 -- 172.21.15.32:0/1363395504 shutdown_connections 2024-04-17T09:53:53.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.674+0000 7f91ef800640 1 --2- 172.21.15.32:0/1363395504 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f91c8068380 0x7f91c806a840 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.674+0000 7f91ef800640 1 --2- 172.21.15.32:0/1363395504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91e8160350 0x7f91e80b37d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.674+0000 7f91ef800640 1 --2- 172.21.15.32:0/1363395504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91e80cf370 0x7f91e8144620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.674+0000 7f91ef800640 1 --2- 172.21.15.32:0/1363395504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91e806ba30 0x7f91e81440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.674+0000 7f91ef800640 1 -- 172.21.15.32:0/1363395504 >> 172.21.15.32:0/1363395504 conn(0x7f91e80b2120 msgr2=0x7f91e80d0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:53.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.674+0000 7f91ef800640 1 -- 172.21.15.32:0/1363395504 shutdown_connections 2024-04-17T09:53:53.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.674+0000 7f91ef800640 1 -- 172.21.15.32:0/1363395504 wait complete. 2024-04-17T09:53:53.696 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:53:53.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5180a00640 1 Processor -- start 2024-04-17T09:53:53.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5180a00640 1 -- start start 2024-04-17T09:53:53.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5180a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f517c15e7f0 0x7f517c0d1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:53.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5180a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f517c06dc60 0x7f517c0cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:53.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5180a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f517c0b52b0 con 0x7f517c06d080 2024-04-17T09:53:53.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5180a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f517c0d1aa0 con 0x7f517c06dc60 2024-04-17T09:53:53.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5180a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f517c0ccb50 con 0x7f517c15e7f0 2024-04-17T09:53:53.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5171e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f517c06d080 0x7f517c06d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54108/0 (socket says 172.21.15.32:54108) 2024-04-17T09:53:53.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5171e00640 1 -- 172.21.15.32:0/520448457 learned_addr learned my addr 172.21.15.32:0/520448457 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:53.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f517a800640 1 --2- 172.21.15.32:0/520448457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f517c15e7f0 0x7f517c0d1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:53.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f517b200640 1 --2- 172.21.15.32:0/520448457 >> v2:172.21.15.144:3300/0 conn(0x7f517c06dc60 0x7f517c0cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:53.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5179e00640 1 -- 172.21.15.32:0/520448457 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3533196571 0 0) 0x7f517c0b52b0 con 0x7f517c06d080 2024-04-17T09:53:53.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5179e00640 1 -- 172.21.15.32:0/520448457 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5160003530 con 0x7f517c06d080 2024-04-17T09:53:53.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5179e00640 1 -- 172.21.15.32:0/520448457 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3961887288 0 0) 0x7f5160003530 con 0x7f517c06d080 2024-04-17T09:53:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5179e00640 1 -- 172.21.15.32:0/520448457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f517c15e7f0 msgr2=0x7f517c0d1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5179e00640 1 --2- 172.21.15.32:0/520448457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f517c15e7f0 0x7f517c0d1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5179e00640 1 -- 172.21.15.32:0/520448457 >> v2:172.21.15.144:3300/0 conn(0x7f517c06dc60 msgr2=0x7f517c0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5179e00640 1 --2- 172.21.15.32:0/520448457 >> v2:172.21.15.144:3300/0 conn(0x7f517c06dc60 0x7f517c0cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5179e00640 1 -- 172.21.15.32:0/520448457 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f517c0cce30 con 0x7f517c06d080 2024-04-17T09:53:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5179e00640 1 -- 172.21.15.32:0/520448457 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f5164001680 con 0x7f517c06d080 2024-04-17T09:53:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f517a800640 1 --2- 172.21.15.32:0/520448457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f517c15e7f0 0x7f517c0d1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:53:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f517b200640 1 --2- 172.21.15.32:0/520448457 >> v2:172.21.15.144:3300/0 conn(0x7f517c06dc60 0x7f517c0cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:53:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5179e00640 1 -- 172.21.15.32:0/520448457 >> v1:172.21.15.32:6789/0 conn(0x7f517c06d080 legacy=0x7f517c06d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5179e00640 1 --2- 172.21.15.32:0/520448457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5160003a30 0x7f5160003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5179e00640 1 --2- 172.21.15.32:0/520448457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5160004810 0x7f5160004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5179e00640 1 --2- 172.21.15.32:0/520448457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f517c15e7f0 0x7f51600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:53.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5179e00640 1 -- 172.21.15.32:0/520448457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f517c0d1aa0 con 0x7f5160003a30 2024-04-17T09:53:53.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5179e00640 1 -- 172.21.15.32:0/520448457 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f517c0ccb50 con 0x7f5160004810 2024-04-17T09:53:53.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.842+0000 7f5179e00640 1 -- 172.21.15.32:0/520448457 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f516000b9d0 con 0x7f517c15e7f0 2024-04-17T09:53:53.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.846+0000 7f517a800640 1 --2- 172.21.15.32:0/520448457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5160003a30 0x7f5160003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:53.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.846+0000 7f5171e00640 1 --2- 172.21.15.32:0/520448457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5160004810 0x7f5160004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:53.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.846+0000 7f517b200640 1 --2- 172.21.15.32:0/520448457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f517c15e7f0 0x7f51600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:53.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.846+0000 7f517a800640 1 -- 172.21.15.32:0/520448457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f517c15e7f0 msgr2=0x7f51600051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:53.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.846+0000 7f517a800640 1 --2- 172.21.15.32:0/520448457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f517c15e7f0 0x7f51600051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.846+0000 7f517a800640 1 -- 172.21.15.32:0/520448457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5160004810 msgr2=0x7f5160004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:53.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.846+0000 7f517a800640 1 --2- 172.21.15.32:0/520448457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5160004810 0x7f5160004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.846+0000 7f517a800640 1 -- 172.21.15.32:0/520448457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5160003530 con 0x7f5160003a30 2024-04-17T09:53:53.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.846+0000 7f5171e00640 1 --2- 172.21.15.32:0/520448457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5160004810 0x7f5160004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:53.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.846+0000 7f517a800640 1 --2- 172.21.15.32:0/520448457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5160003a30 0x7f5160003e50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f516c009cb0 tx=0x7f516c01cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6963afe479d4fd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:53.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.846+0000 7f5179e00640 1 -- 172.21.15.32:0/520448457 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c0b2070 con 0x7f5160003a30 2024-04-17T09:53:53.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.846+0000 7f5179e00640 1 -- 172.21.15.32:0/520448457 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f516c0a57a0 con 0x7f5160003a30 2024-04-17T09:53:53.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.846+0000 7f5179e00640 1 -- 172.21.15.32:0/520448457 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c009880 con 0x7f5160003a30 2024-04-17T09:53:53.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.846+0000 7f5180a00640 1 -- 172.21.15.32:0/520448457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5160003a30 msgr2=0x7f5160003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:53.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.846+0000 7f5180a00640 1 --2- 172.21.15.32:0/520448457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5160003a30 0x7f5160003e50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f516c009cb0 tx=0x7f516c01cbe0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.846+0000 7f517b200640 1 -- 172.21.15.32:0/520448457 reap_dead start 2024-04-17T09:53:53.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.850+0000 7f5180a00640 1 -- 172.21.15.32:0/520448457 shutdown_connections 2024-04-17T09:53:53.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.850+0000 7f5180a00640 1 -- 172.21.15.32:0/520448457 >> 172.21.15.32:0/520448457 conn(0x7f517c0b3f50 msgr2=0x7f517c0b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:53.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.850+0000 7f5180a00640 1 -- 172.21.15.32:0/520448457 shutdown_connections 2024-04-17T09:53:53.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.850+0000 7f5180a00640 1 -- 172.21.15.32:0/520448457 wait complete. 2024-04-17T09:53:53.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.850+0000 7f5180a00640 1 Processor -- start 2024-04-17T09:53:53.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.850+0000 7f5180a00640 1 -- start start 2024-04-17T09:53:53.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.850+0000 7f5180a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5160003a30 0x7f517c15fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:53.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.850+0000 7f5180a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f517c15d550 0x7f517c15d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:53.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.850+0000 7f5180a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f517c09b1b0 0x7f517c09b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:53.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.850+0000 7f5180a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f517c0cdfd0 con 0x7f517c09b1b0 2024-04-17T09:53:53.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.850+0000 7f5180a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f517c0cde50 con 0x7f5160003a30 2024-04-17T09:53:53.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.850+0000 7f5180a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f517c0ce150 con 0x7f517c15d550 2024-04-17T09:53:53.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.850+0000 7f517b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f517c09b1b0 0x7f517c09b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.850+0000 7f517b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f517c09b1b0 0x7f517c09b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44730/0 (socket says 172.21.15.32:44730) 2024-04-17T09:53:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.850+0000 7f517b200640 1 -- 172.21.15.32:0/3277451444 learned_addr learned my addr 172.21.15.32:0/3277451444 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.850+0000 7f517b200640 1 -- 172.21.15.32:0/3277451444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f517c15d550 msgr2=0x7f517c15d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.850+0000 7f517b200640 1 --2- 172.21.15.32:0/3277451444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f517c15d550 0x7f517c15d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.850+0000 7f517b200640 1 -- 172.21.15.32:0/3277451444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5160003a30 msgr2=0x7f517c15fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.850+0000 7f517a800640 1 --2- 172.21.15.32:0/3277451444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5160003a30 0x7f517c15fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.850+0000 7f517b200640 1 --2- 172.21.15.32:0/3277451444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5160003a30 0x7f517c15fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.850+0000 7f517b200640 1 -- 172.21.15.32:0/3277451444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f517c1605a0 con 0x7f517c09b1b0 2024-04-17T09:53:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.850+0000 7f517a800640 1 --2- 172.21.15.32:0/3277451444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5160003a30 0x7f517c15fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.850+0000 7f517b200640 1 --2- 172.21.15.32:0/3277451444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f517c09b1b0 0x7f517c09b630 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f517c06e0f0 tx=0x7f51740115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.854+0000 7f5173400640 1 -- 172.21.15.32:0/3277451444 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51740039a0 con 0x7f517c09b1b0 2024-04-17T09:53:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.854+0000 7f5173400640 1 -- 172.21.15.32:0/3277451444 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f517401b070 con 0x7f517c09b1b0 2024-04-17T09:53:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.854+0000 7f5173400640 1 -- 172.21.15.32:0/3277451444 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5174003b60 con 0x7f517c09b1b0 2024-04-17T09:53:53.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.854+0000 7f5180a00640 1 -- 172.21.15.32:0/3277451444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f517c160880 con 0x7f517c09b1b0 2024-04-17T09:53:53.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.854+0000 7f5180a00640 1 -- 172.21.15.32:0/3277451444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f517c160e40 con 0x7f517c09b1b0 2024-04-17T09:53:53.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.854+0000 7f5180a00640 1 -- 172.21.15.32:0/3277451444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f517c04fda0 con 0x7f517c09b1b0 2024-04-17T09:53:53.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.854+0000 7f5173400640 1 -- 172.21.15.32:0/3277451444 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5174003d00 con 0x7f517c09b1b0 2024-04-17T09:53:53.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.854+0000 7f5173400640 1 --2- 172.21.15.32:0/3277451444 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5158068380 0x7f515806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:53.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.854+0000 7f5173400640 1 -- 172.21.15.32:0/3277451444 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f51740873b0 con 0x7f517c09b1b0 2024-04-17T09:53:53.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.854+0000 7f517a800640 1 --2- 172.21.15.32:0/3277451444 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5158068380 0x7f515806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:53.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.854+0000 7f517a800640 1 --2- 172.21.15.32:0/3277451444 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5158068380 0x7f515806a840 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f516c0af000 tx=0x7f516c0046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:53.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:53.858+0000 7f5173400640 1 -- 172.21.15.32:0/3277451444 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5174025020 con 0x7f517c09b1b0 2024-04-17T09:53:54.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.030+0000 7f5180a00640 1 -- 172.21.15.32:0/3277451444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f517c161cf0 con 0x7f517c09b1b0 2024-04-17T09:53:54.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.030+0000 7f5173400640 1 -- 172.21.15.32:0/3277451444 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f5174057a00 con 0x7f517c09b1b0 2024-04-17T09:53:54.031 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:53:54.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.030+0000 7f5180a00640 1 -- 172.21.15.32:0/3277451444 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5158068380 msgr2=0x7f515806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:54.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.030+0000 7f5180a00640 1 --2- 172.21.15.32:0/3277451444 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5158068380 0x7f515806a840 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f516c0af000 tx=0x7f516c0046d0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.030+0000 7f5180a00640 1 -- 172.21.15.32:0/3277451444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f517c09b1b0 msgr2=0x7f517c09b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:54.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.030+0000 7f5180a00640 1 --2- 172.21.15.32:0/3277451444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f517c09b1b0 0x7f517c09b630 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f517c06e0f0 tx=0x7f51740115a0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.030+0000 7f5180a00640 1 -- 172.21.15.32:0/3277451444 shutdown_connections 2024-04-17T09:53:54.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.030+0000 7f5180a00640 1 --2- 172.21.15.32:0/3277451444 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5158068380 0x7f515806a840 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.030+0000 7f5180a00640 1 --2- 172.21.15.32:0/3277451444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f517c15d550 0x7f517c15d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.030+0000 7f5180a00640 1 --2- 172.21.15.32:0/3277451444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f517c09b1b0 0x7f517c09b630 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.030+0000 7f5180a00640 1 --2- 172.21.15.32:0/3277451444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5160003a30 0x7f517c15fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.030+0000 7f5180a00640 1 -- 172.21.15.32:0/3277451444 >> 172.21.15.32:0/3277451444 conn(0x7f517c0b3f50 msgr2=0x7f517c071430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:54.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.034+0000 7f5180a00640 1 -- 172.21.15.32:0/3277451444 shutdown_connections 2024-04-17T09:53:54.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.034+0000 7f5180a00640 1 -- 172.21.15.32:0/3277451444 wait complete. 2024-04-17T09:53:54.053 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:53:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.194+0000 7f4e09400640 1 Processor -- start 2024-04-17T09:53:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.194+0000 7f4e09400640 1 -- start start 2024-04-17T09:53:54.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e09400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4e0415b7d0 0x7f4e0406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e09400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e0406f7c0 0x7f4e0406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e09400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e040cc940 con 0x7f4e0406f270 2024-04-17T09:53:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e09400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f4e040ccc20 con 0x7f4e0415b7d0 2024-04-17T09:53:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e09400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4e0406e0b0 con 0x7f4e0406f7c0 2024-04-17T09:53:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e03200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4e0406f270 0x7f4e040ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54112/0 (socket says 172.21.15.32:54112) 2024-04-17T09:53:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e03200640 1 -- 172.21.15.32:0/1122907343 learned_addr learned my addr 172.21.15.32:0/1122907343 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e02800640 1 --2- 172.21.15.32:0/1122907343 >> v2:172.21.15.144:3300/0 conn(0x7f4e0415b7d0 0x7f4e0406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4dfbe00640 1 --2- 172.21.15.32:0/1122907343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e0406f7c0 0x7f4e0406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e01e00640 1 -- 172.21.15.32:0/1122907343 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3007311548 0 0) 0x7f4e040cc940 con 0x7f4e0406f270 2024-04-17T09:53:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e01e00640 1 -- 172.21.15.32:0/1122907343 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4de8003530 con 0x7f4e0406f270 2024-04-17T09:53:54.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e01e00640 1 -- 172.21.15.32:0/1122907343 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4014052699 0 0) 0x7f4de8003530 con 0x7f4e0406f270 2024-04-17T09:53:54.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e01e00640 1 -- 172.21.15.32:0/1122907343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e0406f7c0 msgr2=0x7f4e0406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:54.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e01e00640 1 --2- 172.21.15.32:0/1122907343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e0406f7c0 0x7f4e0406ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e01e00640 1 -- 172.21.15.32:0/1122907343 >> v2:172.21.15.144:3300/0 conn(0x7f4e0415b7d0 msgr2=0x7f4e0406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:54.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e01e00640 1 --2- 172.21.15.32:0/1122907343 >> v2:172.21.15.144:3300/0 conn(0x7f4e0415b7d0 0x7f4e0406e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e01e00640 1 -- 172.21.15.32:0/1122907343 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0406e390 con 0x7f4e0406f270 2024-04-17T09:53:54.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e01e00640 1 -- 172.21.15.32:0/1122907343 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f4dfc001a00 con 0x7f4e0406f270 2024-04-17T09:53:54.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e01e00640 1 -- 172.21.15.32:0/1122907343 >> v1:172.21.15.32:6789/0 conn(0x7f4e0406f270 legacy=0x7f4e040ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:54.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e01e00640 1 --2- 172.21.15.32:0/1122907343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e0406f7c0 0x7f4de80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:54.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e01e00640 1 --2- 172.21.15.32:0/1122907343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de8003f90 0x7f4de8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:54.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e01e00640 1 --2- 172.21.15.32:0/1122907343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4de8004d50 0x7f4de80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:54.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e01e00640 1 -- 172.21.15.32:0/1122907343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4e040ccc20 con 0x7f4de8003f90 2024-04-17T09:53:54.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e01e00640 1 -- 172.21.15.32:0/1122907343 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0406e0b0 con 0x7f4de8004d50 2024-04-17T09:53:54.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e01e00640 1 -- 172.21.15.32:0/1122907343 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4de800dc90 con 0x7f4e0406f7c0 2024-04-17T09:53:54.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4dfbe00640 1 --2- 172.21.15.32:0/1122907343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de8003f90 0x7f4de8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:54.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e02800640 1 --2- 172.21.15.32:0/1122907343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e0406f7c0 0x7f4de80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:54.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4e03200640 1 --2- 172.21.15.32:0/1122907343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4de8004d50 0x7f4de80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:54.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4dfbe00640 1 -- 172.21.15.32:0/1122907343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e0406f7c0 msgr2=0x7f4de80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:54.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4dfbe00640 1 --2- 172.21.15.32:0/1122907343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e0406f7c0 0x7f4de80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4dfbe00640 1 -- 172.21.15.32:0/1122907343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4de8004d50 msgr2=0x7f4de80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:54.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4dfbe00640 1 --2- 172.21.15.32:0/1122907343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4de8004d50 0x7f4de80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.198+0000 7f4dfbe00640 1 -- 172.21.15.32:0/1122907343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de8003530 con 0x7f4de8003f90 2024-04-17T09:53:54.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.202+0000 7f4dfbe00640 1 --2- 172.21.15.32:0/1122907343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de8003f90 0x7f4de8004810 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4dec009ae0 tx=0x7f4dec01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35adf0c8867eedd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:54.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.202+0000 7f4e01e00640 1 -- 172.21.15.32:0/1122907343 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec0b2070 con 0x7f4de8003f90 2024-04-17T09:53:54.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.202+0000 7f4e01e00640 1 -- 172.21.15.32:0/1122907343 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dec0a5660 con 0x7f4de8003f90 2024-04-17T09:53:54.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.202+0000 7f4e01e00640 1 -- 172.21.15.32:0/1122907343 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec0096b0 con 0x7f4de8003f90 2024-04-17T09:53:54.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.202+0000 7f4e09400640 1 -- 172.21.15.32:0/1122907343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de8003f90 msgr2=0x7f4de8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:54.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.202+0000 7f4e09400640 1 --2- 172.21.15.32:0/1122907343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de8003f90 0x7f4de8004810 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4dec009ae0 tx=0x7f4dec01ca70 comp rx=0 tx=0).stop 2024-04-17T09:53:54.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.202+0000 7f4e03200640 1 -- 172.21.15.32:0/1122907343 reap_dead start 2024-04-17T09:53:54.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.202+0000 7f4e09400640 1 -- 172.21.15.32:0/1122907343 shutdown_connections 2024-04-17T09:53:54.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.202+0000 7f4e09400640 1 -- 172.21.15.32:0/1122907343 >> 172.21.15.32:0/1122907343 conn(0x7f4e040b2120 msgr2=0x7f4e040b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:54.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.202+0000 7f4e09400640 1 -- 172.21.15.32:0/1122907343 shutdown_connections 2024-04-17T09:53:54.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.202+0000 7f4e09400640 1 -- 172.21.15.32:0/1122907343 wait complete. 2024-04-17T09:53:54.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.202+0000 7f4e09400640 1 Processor -- start 2024-04-17T09:53:54.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.202+0000 7f4e09400640 1 -- start start 2024-04-17T09:53:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4e09400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de8003f90 0x7f4e04146f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4e09400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e04157cd0 0x7f4e04158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4e09400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e041f78f0 0x7f4e041f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4e09400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4e04071fe0 con 0x7f4de8003f90 2024-04-17T09:53:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4e09400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4e04071e60 con 0x7f4e041f78f0 2024-04-17T09:53:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4e09400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4e04072160 con 0x7f4e04157cd0 2024-04-17T09:53:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4e02800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de8003f90 0x7f4e04146f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4e02800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de8003f90 0x7f4e04146f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44754/0 (socket says 172.21.15.32:44754) 2024-04-17T09:53:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4e02800640 1 -- 172.21.15.32:0/936018812 learned_addr learned my addr 172.21.15.32:0/936018812 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4dfbe00640 1 --2- 172.21.15.32:0/936018812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e04157cd0 0x7f4e04158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4e03200640 1 --2- 172.21.15.32:0/936018812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e041f78f0 0x7f4e041f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4e02800640 1 -- 172.21.15.32:0/936018812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e04157cd0 msgr2=0x7f4e04158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:54.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4e02800640 1 --2- 172.21.15.32:0/936018812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e04157cd0 0x7f4e04158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4e02800640 1 -- 172.21.15.32:0/936018812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e041f78f0 msgr2=0x7f4e041f7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:54.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4e02800640 1 --2- 172.21.15.32:0/936018812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e041f78f0 0x7f4e041f7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4e02800640 1 -- 172.21.15.32:0/936018812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0415c7d0 con 0x7f4de8003f90 2024-04-17T09:53:54.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4dfbe00640 1 --2- 172.21.15.32:0/936018812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e04157cd0 0x7f4e04158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:54.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4e02800640 1 --2- 172.21.15.32:0/936018812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de8003f90 0x7f4e04146f20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4df4011660 tx=0x7f4df4011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:54.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4dfaa00640 1 -- 172.21.15.32:0/936018812 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df400f960 con 0x7f4de8003f90 2024-04-17T09:53:54.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4dfaa00640 1 -- 172.21.15.32:0/936018812 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4df400fb00 con 0x7f4de8003f90 2024-04-17T09:53:54.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4dfaa00640 1 -- 172.21.15.32:0/936018812 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df4015460 con 0x7f4de8003f90 2024-04-17T09:53:54.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4e09400640 1 -- 172.21.15.32:0/936018812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e041f8860 con 0x7f4de8003f90 2024-04-17T09:53:54.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4e09400640 1 -- 172.21.15.32:0/936018812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e041f8580 con 0x7f4de8003f90 2024-04-17T09:53:54.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4e09400640 1 -- 172.21.15.32:0/936018812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e04068a90 con 0x7f4de8003f90 2024-04-17T09:53:54.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.206+0000 7f4dfaa00640 1 -- 172.21.15.32:0/936018812 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4df4015600 con 0x7f4de8003f90 2024-04-17T09:53:54.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.210+0000 7f4dfaa00640 1 --2- 172.21.15.32:0/936018812 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4de0068380 0x7f4de006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:54.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.210+0000 7f4dfaa00640 1 -- 172.21.15.32:0/936018812 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f4df40873c0 con 0x7f4de8003f90 2024-04-17T09:53:54.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.210+0000 7f4dfbe00640 1 --2- 172.21.15.32:0/936018812 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4de0068380 0x7f4de006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:54.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.210+0000 7f4dfbe00640 1 --2- 172.21.15.32:0/936018812 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4de0068380 0x7f4de006a840 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4dec0af000 tx=0x7f4dec0046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:54.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.214+0000 7f4dfaa00640 1 -- 172.21.15.32:0/936018812 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f4e0415c7d0 con 0x7f4de8003f90 2024-04-17T09:53:54.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.378+0000 7f4e09400640 1 -- 172.21.15.32:0/936018812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4e04159f60 con 0x7f4de8003f90 2024-04-17T09:53:54.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.378+0000 7f4dfaa00640 1 -- 172.21.15.32:0/936018812 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f4df4019030 con 0x7f4de8003f90 2024-04-17T09:53:54.378 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:53:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.378+0000 7f4e09400640 1 -- 172.21.15.32:0/936018812 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4de0068380 msgr2=0x7f4de006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.378+0000 7f4e09400640 1 --2- 172.21.15.32:0/936018812 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4de0068380 0x7f4de006a840 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4dec0af000 tx=0x7f4dec0046d0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.378+0000 7f4e09400640 1 -- 172.21.15.32:0/936018812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de8003f90 msgr2=0x7f4e04146f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.378+0000 7f4e09400640 1 --2- 172.21.15.32:0/936018812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de8003f90 0x7f4e04146f20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4df4011660 tx=0x7f4df4011b20 comp rx=0 tx=0).stop 2024-04-17T09:53:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.378+0000 7f4e09400640 1 -- 172.21.15.32:0/936018812 shutdown_connections 2024-04-17T09:53:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.378+0000 7f4e09400640 1 --2- 172.21.15.32:0/936018812 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4de0068380 0x7f4de006a840 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.378+0000 7f4e09400640 1 --2- 172.21.15.32:0/936018812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e04157cd0 0x7f4e04158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.378+0000 7f4e09400640 1 --2- 172.21.15.32:0/936018812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e041f78f0 0x7f4e041f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.378+0000 7f4e09400640 1 --2- 172.21.15.32:0/936018812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4de8003f90 0x7f4e04146f20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.378+0000 7f4e09400640 1 -- 172.21.15.32:0/936018812 >> 172.21.15.32:0/936018812 conn(0x7f4e040b2120 msgr2=0x7f4e040b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.378+0000 7f4e09400640 1 -- 172.21.15.32:0/936018812 shutdown_connections 2024-04-17T09:53:54.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.378+0000 7f4e09400640 1 -- 172.21.15.32:0/936018812 wait complete. 2024-04-17T09:53:54.400 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T09:53:54.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefe6800640 1 Processor -- start 2024-04-17T09:53:54.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefe6800640 1 -- start start 2024-04-17T09:53:54.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefe6800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fefe006ecc0 0x7fefe00cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:54.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefe6800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fefe006fc70 0x7fefe00afe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:54.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefe6800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefe00700e0 con 0x7fefe0070ae0 2024-04-17T09:53:54.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefe6800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fefe0160090 con 0x7fefe006ecc0 2024-04-17T09:53:54.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefe6800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fefe006f990 con 0x7fefe006fc70 2024-04-17T09:53:54.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefe4a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fefe0070ae0 0x7fefe006f0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54116/0 (socket says 172.21.15.32:54116) 2024-04-17T09:53:54.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefe4a00640 1 -- 172.21.15.32:0/1990806925 learned_addr learned my addr 172.21.15.32:0/1990806925 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:54.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefe5400640 1 --2- 172.21.15.32:0/1990806925 >> v2:172.21.15.144:3300/0 conn(0x7fefe006ecc0 0x7fefe00cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:54.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefe5e00640 1 --2- 172.21.15.32:0/1990806925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fefe006fc70 0x7fefe00afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:54.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefdfe00640 1 -- 172.21.15.32:0/1990806925 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2671525667 0 0) 0x7fefe00700e0 con 0x7fefe0070ae0 2024-04-17T09:53:54.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefdfe00640 1 -- 172.21.15.32:0/1990806925 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefc8003530 con 0x7fefe0070ae0 2024-04-17T09:53:54.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefdfe00640 1 -- 172.21.15.32:0/1990806925 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 368848036 0 0) 0x7fefc8003530 con 0x7fefe0070ae0 2024-04-17T09:53:54.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefdfe00640 1 -- 172.21.15.32:0/1990806925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fefe006fc70 msgr2=0x7fefe00afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:54.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefdfe00640 1 --2- 172.21.15.32:0/1990806925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fefe006fc70 0x7fefe00afe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefdfe00640 1 -- 172.21.15.32:0/1990806925 >> v2:172.21.15.144:3300/0 conn(0x7fefe006ecc0 msgr2=0x7fefe00cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:54.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefdfe00640 1 --2- 172.21.15.32:0/1990806925 >> v2:172.21.15.144:3300/0 conn(0x7fefe006ecc0 0x7fefe00cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefdfe00640 1 -- 172.21.15.32:0/1990806925 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe00cf9a0 con 0x7fefe0070ae0 2024-04-17T09:53:54.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefdfe00640 1 -- 172.21.15.32:0/1990806925 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7fefcc001680 con 0x7fefe0070ae0 2024-04-17T09:53:54.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefe5400640 1 --2- 172.21.15.32:0/1990806925 >> v2:172.21.15.144:3300/0 conn(0x7fefe006ecc0 0x7fefe00cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:53:54.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefdfe00640 1 -- 172.21.15.32:0/1990806925 >> v1:172.21.15.32:6789/0 conn(0x7fefe0070ae0 legacy=0x7fefe006f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:54.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefdfe00640 1 --2- 172.21.15.32:0/1990806925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fefe006fc70 0x7fefc80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:54.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefdfe00640 1 --2- 172.21.15.32:0/1990806925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefc8003f40 0x7fefc8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:54.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefdfe00640 1 --2- 172.21.15.32:0/1990806925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefc8004d50 0x7fefc80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:54.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefdfe00640 1 -- 172.21.15.32:0/1990806925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fefe0160090 con 0x7fefc8003f40 2024-04-17T09:53:54.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefdfe00640 1 -- 172.21.15.32:0/1990806925 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fefe006f990 con 0x7fefc8004d50 2024-04-17T09:53:54.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.542+0000 7fefdfe00640 1 -- 172.21.15.32:0/1990806925 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fefc800b9d0 con 0x7fefe006fc70 2024-04-17T09:53:54.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.546+0000 7fefe4a00640 1 --2- 172.21.15.32:0/1990806925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefc8003f40 0x7fefc8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:54.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.546+0000 7fefe5400640 1 --2- 172.21.15.32:0/1990806925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fefe006fc70 0x7fefc80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:54.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.546+0000 7fefe5e00640 1 --2- 172.21.15.32:0/1990806925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefc8004d50 0x7fefc80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:54.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.546+0000 7fefe4a00640 1 -- 172.21.15.32:0/1990806925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fefe006fc70 msgr2=0x7fefc80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:54.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.546+0000 7fefe4a00640 1 --2- 172.21.15.32:0/1990806925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fefe006fc70 0x7fefc80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.546+0000 7fefe4a00640 1 -- 172.21.15.32:0/1990806925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefc8004d50 msgr2=0x7fefc80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:54.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.546+0000 7fefe4a00640 1 --2- 172.21.15.32:0/1990806925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefc8004d50 0x7fefc80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.546+0000 7fefe4a00640 1 -- 172.21.15.32:0/1990806925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc8003530 con 0x7fefc8003f40 2024-04-17T09:53:54.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.546+0000 7fefe4a00640 1 --2- 172.21.15.32:0/1990806925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefc8003f40 0x7fefc8004810 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fefcc001060 tx=0x7fefcc01f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6bf9b54c00df7b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:54.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.546+0000 7fefdfe00640 1 -- 172.21.15.32:0/1990806925 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc007d80 con 0x7fefc8003f40 2024-04-17T09:53:54.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.546+0000 7fefdfe00640 1 -- 172.21.15.32:0/1990806925 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefcc005b00 con 0x7fefc8003f40 2024-04-17T09:53:54.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.546+0000 7fefdfe00640 1 -- 172.21.15.32:0/1990806925 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc007070 con 0x7fefc8003f40 2024-04-17T09:53:54.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe6800640 1 -- 172.21.15.32:0/1990806925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefc8003f40 msgr2=0x7fefc8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:54.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe6800640 1 --2- 172.21.15.32:0/1990806925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefc8003f40 0x7fefc8004810 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fefcc001060 tx=0x7fefcc01f3b0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe5e00640 1 -- 172.21.15.32:0/1990806925 reap_dead start 2024-04-17T09:53:54.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe6800640 1 -- 172.21.15.32:0/1990806925 shutdown_connections 2024-04-17T09:53:54.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe6800640 1 -- 172.21.15.32:0/1990806925 >> 172.21.15.32:0/1990806925 conn(0x7fefe009a9f0 msgr2=0x7fefe00b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:54.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe6800640 1 -- 172.21.15.32:0/1990806925 shutdown_connections 2024-04-17T09:53:54.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe6800640 1 -- 172.21.15.32:0/1990806925 wait complete. 2024-04-17T09:53:54.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe6800640 1 Processor -- start 2024-04-17T09:53:54.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe6800640 1 -- start start 2024-04-17T09:53:54.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe6800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefc8003f40 0x7fefe00c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:54.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefe00b2920 0x7fefe00cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:54.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe6800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fefe00b2d80 0x7fefe00cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:54.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe6800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fefe0070500 con 0x7fefe00b2920 2024-04-17T09:53:54.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe6800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fefe00cff60 con 0x7fefc8003f40 2024-04-17T09:53:54.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe6800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fefe00afe00 con 0x7fefe00b2d80 2024-04-17T09:53:54.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe4a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefe00b2920 0x7fefe00cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:54.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe4a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefe00b2920 0x7fefe00cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44778/0 (socket says 172.21.15.32:44778) 2024-04-17T09:53:54.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe4a00640 1 -- 172.21.15.32:0/118033986 learned_addr learned my addr 172.21.15.32:0/118033986 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:54.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe5e00640 1 --2- 172.21.15.32:0/118033986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fefe00b2d80 0x7fefe00cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:54.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe5400640 1 --2- 172.21.15.32:0/118033986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefc8003f40 0x7fefe00c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:54.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe4a00640 1 -- 172.21.15.32:0/118033986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fefe00b2d80 msgr2=0x7fefe00cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:54.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe4a00640 1 --2- 172.21.15.32:0/118033986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fefe00b2d80 0x7fefe00cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe4a00640 1 -- 172.21.15.32:0/118033986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefc8003f40 msgr2=0x7fefe00c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:54.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe4a00640 1 --2- 172.21.15.32:0/118033986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefc8003f40 0x7fefe00c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.550+0000 7fefe4a00640 1 -- 172.21.15.32:0/118033986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe00b1880 con 0x7fefe00b2920 2024-04-17T09:53:54.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.554+0000 7fefe4a00640 1 --2- 172.21.15.32:0/118033986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefe00b2920 0x7fefe00cd8a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fefcc01fc50 tx=0x7fefc8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:54.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.554+0000 7fefdd600640 1 -- 172.21.15.32:0/118033986 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc005cb0 con 0x7fefe00b2920 2024-04-17T09:53:54.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.554+0000 7fefdd600640 1 -- 172.21.15.32:0/118033986 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefcc005e50 con 0x7fefe00b2920 2024-04-17T09:53:54.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.554+0000 7fefdd600640 1 -- 172.21.15.32:0/118033986 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc004a10 con 0x7fefe00b2920 2024-04-17T09:53:54.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.554+0000 7fefe6800640 1 -- 172.21.15.32:0/118033986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe00b1b60 con 0x7fefe00b2920 2024-04-17T09:53:54.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.554+0000 7fefe6800640 1 -- 172.21.15.32:0/118033986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe00b2120 con 0x7fefe00b2920 2024-04-17T09:53:54.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.554+0000 7fefe6800640 1 -- 172.21.15.32:0/118033986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe004fd00 con 0x7fefe00b2920 2024-04-17T09:53:54.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.554+0000 7fefdd600640 1 -- 172.21.15.32:0/118033986 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fefcc007040 con 0x7fefe00b2920 2024-04-17T09:53:54.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.554+0000 7fefdd600640 1 --2- 172.21.15.32:0/118033986 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fefc00683b0 0x7fefc006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:54.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.554+0000 7fefdd600640 1 -- 172.21.15.32:0/118033986 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fefcc123bc0 con 0x7fefe00b2920 2024-04-17T09:53:54.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.554+0000 7fefe5400640 1 --2- 172.21.15.32:0/118033986 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fefc00683b0 0x7fefc006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:54.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.558+0000 7fefe5400640 1 --2- 172.21.15.32:0/118033986 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fefc00683b0 0x7fefc006a870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009cb0 tx=0x7fefd4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:54.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.558+0000 7fefdd600640 1 -- 172.21.15.32:0/118033986 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fefcc019120 con 0x7fefe00b2920 2024-04-17T09:53:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.738+0000 7fefe6800640 1 -- 172.21.15.32:0/118033986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fefe006c8e0 con 0x7fefe00b2920 2024-04-17T09:53:54.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.738+0000 7fefdd600640 1 -- 172.21.15.32:0/118033986 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fefcc0f3fd0 con 0x7fefe00b2920 2024-04-17T09:53:54.739 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T09:53:54.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.738+0000 7fefe6800640 1 -- 172.21.15.32:0/118033986 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fefc00683b0 msgr2=0x7fefc006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:54.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.738+0000 7fefe6800640 1 --2- 172.21.15.32:0/118033986 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fefc00683b0 0x7fefc006a870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009cb0 tx=0x7fefd4008040 comp rx=0 tx=0).stop 2024-04-17T09:53:54.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.738+0000 7fefe6800640 1 -- 172.21.15.32:0/118033986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefe00b2920 msgr2=0x7fefe00cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:54.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.738+0000 7fefe6800640 1 --2- 172.21.15.32:0/118033986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefe00b2920 0x7fefe00cd8a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fefcc01fc50 tx=0x7fefc8005840 comp rx=0 tx=0).stop 2024-04-17T09:53:54.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.738+0000 7fefe6800640 1 -- 172.21.15.32:0/118033986 shutdown_connections 2024-04-17T09:53:54.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.738+0000 7fefe6800640 1 --2- 172.21.15.32:0/118033986 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fefc00683b0 0x7fefc006a870 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.742+0000 7fefe6800640 1 --2- 172.21.15.32:0/118033986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fefe00b2d80 0x7fefe00cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.742+0000 7fefe6800640 1 --2- 172.21.15.32:0/118033986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefe00b2920 0x7fefe00cd8a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.742+0000 7fefe6800640 1 --2- 172.21.15.32:0/118033986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefc8003f40 0x7fefe00c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:54.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.742+0000 7fefe6800640 1 -- 172.21.15.32:0/118033986 >> 172.21.15.32:0/118033986 conn(0x7fefe009a9f0 msgr2=0x7fefe00af0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:54.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.742+0000 7fefe6800640 1 -- 172.21.15.32:0/118033986 shutdown_connections 2024-04-17T09:53:54.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:54.742+0000 7fefe6800640 1 -- 172.21.15.32:0/118033986 wait complete. 2024-04-17T09:53:54.761 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi144.front.sepia.ceph.com... 2024-04-17T09:53:54.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:54.761 DEBUG:teuthology.orchestra.run.smithi144:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-17T09:53:54.792 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:53:54.793 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T09:53:54.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:54.793 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-17T09:53:54.850 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:53:54.850 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:53:54.859 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:53:54.859 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:53:54.909 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:53:55.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.094+0000 7fced0800640 1 Processor -- start 2024-04-17T09:53:55.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.094+0000 7fced0800640 1 -- start start 2024-04-17T09:53:55.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.094+0000 7fced0800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcec8145f30 0x7fcec8071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:55.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.094+0000 7fced0800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcec80cd580 0x7fcec8072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:55.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.094+0000 7fced0800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcec80cdac0 con 0x7fcec815d8d0 2024-04-17T09:53:55.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.094+0000 7fced0800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fcec806f400 con 0x7fcec80cd580 2024-04-17T09:53:55.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.094+0000 7fced0800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcec8146310 con 0x7fcec8145f30 2024-04-17T09:53:55.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.094+0000 7fcece800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcec80cd580 0x7fcec8072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:55.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.094+0000 7fcece800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcec80cd580 0x7fcec8072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44780/0 (socket says 172.21.15.32:44780) 2024-04-17T09:53:55.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.094+0000 7fcece800640 1 -- 172.21.15.32:0/2687908922 learned_addr learned my addr 172.21.15.32:0/2687908922 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:55.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.094+0000 7fcecde00640 1 --2- 172.21.15.32:0/2687908922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcec8145f30 0x7fcec8071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:55.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.094+0000 7fcece800640 1 -- 172.21.15.32:0/2687908922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcec8145f30 msgr2=0x7fcec8071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:55.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.094+0000 7fcece800640 1 --2- 172.21.15.32:0/2687908922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcec8145f30 0x7fcec8071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.094+0000 7fcece800640 1 -- 172.21.15.32:0/2687908922 >> v1:172.21.15.144:6789/0 conn(0x7fcec815d8d0 legacy=0x7fcec80cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:55.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.094+0000 7fcece800640 1 -- 172.21.15.32:0/2687908922 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec815cdf0 con 0x7fcec80cd580 2024-04-17T09:53:55.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.098+0000 7fcece800640 1 --2- 172.21.15.32:0/2687908922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcec80cd580 0x7fcec8072330 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009e50 tx=0x7fcec401f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905616e4c69c2ccb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:55.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.098+0000 7fcecca00640 1 -- 172.21.15.32:0/2687908922 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec40b4070 con 0x7fcec80cd580 2024-04-17T09:53:55.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.098+0000 7fcecca00640 1 -- 172.21.15.32:0/2687908922 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec40af400 con 0x7fcec80cd580 2024-04-17T09:53:55.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.098+0000 7fcecca00640 1 -- 172.21.15.32:0/2687908922 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec40af7d0 con 0x7fcec80cd580 2024-04-17T09:53:55.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.098+0000 7fced0800640 1 -- 172.21.15.32:0/2687908922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcec80cd580 msgr2=0x7fcec8072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:55.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.098+0000 7fced0800640 1 --2- 172.21.15.32:0/2687908922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcec80cd580 0x7fcec8072330 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009e50 tx=0x7fcec401f210 comp rx=0 tx=0).stop 2024-04-17T09:53:55.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.098+0000 7fced0800640 1 -- 172.21.15.32:0/2687908922 shutdown_connections 2024-04-17T09:53:55.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.098+0000 7fced0800640 1 --2- 172.21.15.32:0/2687908922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcec80cd580 0x7fcec8072330 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.098+0000 7fced0800640 1 --2- 172.21.15.32:0/2687908922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcec8145f30 0x7fcec8071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.098+0000 7fced0800640 1 -- 172.21.15.32:0/2687908922 >> 172.21.15.32:0/2687908922 conn(0x7fcec806ba30 msgr2=0x7fcec806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:55.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.098+0000 7fced0800640 1 -- 172.21.15.32:0/2687908922 shutdown_connections 2024-04-17T09:53:55.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.098+0000 7fced0800640 1 -- 172.21.15.32:0/2687908922 wait complete. 2024-04-17T09:53:55.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fced0800640 1 Processor -- start 2024-04-17T09:53:55.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fced0800640 1 -- start start 2024-04-17T09:53:55.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fced0800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcec80cd580 0x7fcec80ad460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:55.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fced0800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcec8145f30 0x7fcec80cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fced0800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcec815d8d0 0x7fcec80cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fced0800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcec815cb90 con 0x7fcec815d8d0 2024-04-17T09:53:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fced0800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcec815d440 con 0x7fcec8145f30 2024-04-17T09:53:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fced0800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcec81601d0 con 0x7fcec80cd580 2024-04-17T09:53:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fcece800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcec815d8d0 0x7fcec80cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fcece800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcec815d8d0 0x7fcec80cc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44786/0 (socket says 172.21.15.32:44786) 2024-04-17T09:53:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fcece800640 1 -- 172.21.15.32:0/1609490003 learned_addr learned my addr 172.21.15.32:0/1609490003 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fcecd400640 1 --2- 172.21.15.32:0/1609490003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcec8145f30 0x7fcec80cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fcecde00640 1 --2- 172.21.15.32:0/1609490003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcec80cd580 0x7fcec80ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fcece800640 1 -- 172.21.15.32:0/1609490003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcec80cd580 msgr2=0x7fcec80ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fcece800640 1 --2- 172.21.15.32:0/1609490003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcec80cd580 0x7fcec80ad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fcece800640 1 -- 172.21.15.32:0/1609490003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcec8145f30 msgr2=0x7fcec80cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fcece800640 1 --2- 172.21.15.32:0/1609490003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcec8145f30 0x7fcec80cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fcece800640 1 -- 172.21.15.32:0/1609490003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec8137ef0 con 0x7fcec815d8d0 2024-04-17T09:53:55.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fcece800640 1 --2- 172.21.15.32:0/1609490003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcec815d8d0 0x7fcec80cc020 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcec401ce40 tx=0x7fcec401bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:55.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fcec2000640 1 -- 172.21.15.32:0/1609490003 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec40b4040 con 0x7fcec815d8d0 2024-04-17T09:53:55.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fcec2000640 1 -- 172.21.15.32:0/1609490003 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec40afd30 con 0x7fcec815d8d0 2024-04-17T09:53:55.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fced0800640 1 -- 172.21.15.32:0/1609490003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec8135010 con 0x7fcec815d8d0 2024-04-17T09:53:55.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fcec2000640 1 -- 172.21.15.32:0/1609490003 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4009510 con 0x7fcec815d8d0 2024-04-17T09:53:55.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fced0800640 1 -- 172.21.15.32:0/1609490003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec81355d0 con 0x7fcec815d8d0 2024-04-17T09:53:55.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.102+0000 7fced0800640 1 -- 172.21.15.32:0/1609490003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcec8068a90 con 0x7fcec815d8d0 2024-04-17T09:53:55.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.106+0000 7fcec2000640 1 -- 172.21.15.32:0/1609490003 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcec40c2020 con 0x7fcec815d8d0 2024-04-17T09:53:55.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.106+0000 7fcec2000640 1 --2- 172.21.15.32:0/1609490003 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fceac068380 0x7fceac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:55.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.106+0000 7fcec2000640 1 -- 172.21.15.32:0/1609490003 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fcec4124350 con 0x7fcec815d8d0 2024-04-17T09:53:55.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.106+0000 7fcecde00640 1 --2- 172.21.15.32:0/1609490003 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fceac068380 0x7fceac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:55.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.106+0000 7fcecde00640 1 --2- 172.21.15.32:0/1609490003 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fceac068380 0x7fceac06a840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcebc007a60 tx=0x7fcebc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:55.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.110+0000 7fcec2000640 1 -- 172.21.15.32:0/1609490003 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fcec40c9020 con 0x7fcec815d8d0 2024-04-17T09:53:55.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.278+0000 7fced0800640 1 -- 172.21.15.32:0/1609490003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcec80ced10 con 0x7fcec815d8d0 2024-04-17T09:53:55.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.278+0000 7fcec2000640 1 -- 172.21.15.32:0/1609490003 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16866 (secure 0 0 0) 0x7fcec4018d70 con 0x7fcec815d8d0 2024-04-17T09:53:55.278 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:55.278 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:54.932571+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:53:55.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.278+0000 7fced0800640 1 -- 172.21.15.32:0/1609490003 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fceac068380 msgr2=0x7fceac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:55.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.278+0000 7fced0800640 1 --2- 172.21.15.32:0/1609490003 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fceac068380 0x7fceac06a840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcebc007a60 tx=0x7fcebc008040 comp rx=0 tx=0).stop 2024-04-17T09:53:55.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.278+0000 7fced0800640 1 -- 172.21.15.32:0/1609490003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcec815d8d0 msgr2=0x7fcec80cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:55.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.278+0000 7fced0800640 1 --2- 172.21.15.32:0/1609490003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcec815d8d0 0x7fcec80cc020 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcec401ce40 tx=0x7fcec401bfb0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.278+0000 7fced0800640 1 -- 172.21.15.32:0/1609490003 shutdown_connections 2024-04-17T09:53:55.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.278+0000 7fced0800640 1 --2- 172.21.15.32:0/1609490003 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fceac068380 0x7fceac06a840 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.278+0000 7fced0800640 1 --2- 172.21.15.32:0/1609490003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcec815d8d0 0x7fcec80cc020 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.278+0000 7fced0800640 1 --2- 172.21.15.32:0/1609490003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcec8145f30 0x7fcec80cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.278+0000 7fced0800640 1 --2- 172.21.15.32:0/1609490003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcec80cd580 0x7fcec80ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.278+0000 7fced0800640 1 -- 172.21.15.32:0/1609490003 >> 172.21.15.32:0/1609490003 conn(0x7fcec806ba30 msgr2=0x7fcec8144c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:55.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.278+0000 7fced0800640 1 -- 172.21.15.32:0/1609490003 shutdown_connections 2024-04-17T09:53:55.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.278+0000 7fced0800640 1 -- 172.21.15.32:0/1609490003 wait complete. 2024-04-17T09:53:55.300 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:53:11.198098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:53:12.118982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:53:55.301 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:53:55.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.446+0000 7f5383e00640 1 Processor -- start 2024-04-17T09:53:55.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.446+0000 7f5383e00640 1 -- start start 2024-04-17T09:53:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.446+0000 7f5383e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f538413f630 0x7f5384071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.446+0000 7f5383e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5384160a30 0x7f53840ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.446+0000 7f5383e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f538415db00 con 0x7f5384072330 2024-04-17T09:53:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.446+0000 7f5383e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f538413fa10 con 0x7f5384160a30 2024-04-17T09:53:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.446+0000 7f5383e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5384072770 con 0x7f538413f630 2024-04-17T09:53:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.446+0000 7f537a000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5384072330 0x7f5384160320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54124/0 (socket says 172.21.15.32:54124) 2024-04-17T09:53:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.446+0000 7f537a000640 1 -- 172.21.15.32:0/2136090249 learned_addr learned my addr 172.21.15.32:0/2136090249 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.446+0000 7f5382a00640 1 --2- 172.21.15.32:0/2136090249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f538413f630 0x7f5384071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:55.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.446+0000 7f5383400640 1 --2- 172.21.15.32:0/2136090249 >> v2:172.21.15.144:3300/0 conn(0x7f5384160a30 0x7f53840ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:55.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.446+0000 7f5382000640 1 -- 172.21.15.32:0/2136090249 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 193275294 0 0) 0x7f538415db00 con 0x7f5384072330 2024-04-17T09:53:55.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.446+0000 7f5382000640 1 -- 172.21.15.32:0/2136090249 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5368003530 con 0x7f5384072330 2024-04-17T09:53:55.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.446+0000 7f5382a00640 1 -- 172.21.15.32:0/2136090249 >> v2:172.21.15.144:3300/0 conn(0x7f5384160a30 msgr2=0x7f53840ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:55.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.446+0000 7f5382a00640 1 --2- 172.21.15.32:0/2136090249 >> v2:172.21.15.144:3300/0 conn(0x7f5384160a30 0x7f53840ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.446+0000 7f5382a00640 1 -- 172.21.15.32:0/2136090249 >> v1:172.21.15.32:6789/0 conn(0x7f5384072330 legacy=0x7f5384160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:55.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.446+0000 7f5382a00640 1 -- 172.21.15.32:0/2136090249 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53841610c0 con 0x7f538413f630 2024-04-17T09:53:55.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.446+0000 7f5383400640 1 --2- 172.21.15.32:0/2136090249 >> v2:172.21.15.144:3300/0 conn(0x7f5384160a30 0x7f53840ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:53:55.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.450+0000 7f5382a00640 1 --2- 172.21.15.32:0/2136090249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f538413f630 0x7f5384071df0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f53740097c0 tx=0x7f537401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8580c89dfde52c28 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:55.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.450+0000 7f5382000640 1 -- 172.21.15.32:0/2136090249 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53740b2070 con 0x7f538413f630 2024-04-17T09:53:55.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.450+0000 7f5382000640 1 -- 172.21.15.32:0/2136090249 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53740ad640 con 0x7f538413f630 2024-04-17T09:53:55.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.450+0000 7f5382000640 1 -- 172.21.15.32:0/2136090249 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53740ada10 con 0x7f538413f630 2024-04-17T09:53:55.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.450+0000 7f5383e00640 1 -- 172.21.15.32:0/2136090249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f538413f630 msgr2=0x7f5384071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:55.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.450+0000 7f5383e00640 1 --2- 172.21.15.32:0/2136090249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f538413f630 0x7f5384071df0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f53740097c0 tx=0x7f537401c4a0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.450+0000 7f5383e00640 1 -- 172.21.15.32:0/2136090249 shutdown_connections 2024-04-17T09:53:55.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.450+0000 7f5383e00640 1 --2- 172.21.15.32:0/2136090249 >> v2:172.21.15.144:3300/0 conn(0x7f5384160a30 0x7f53840ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.450+0000 7f5383e00640 1 --2- 172.21.15.32:0/2136090249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f538413f630 0x7f5384071df0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.450+0000 7f5383e00640 1 -- 172.21.15.32:0/2136090249 >> 172.21.15.32:0/2136090249 conn(0x7f5384067d70 msgr2=0x7f538406ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:55.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.450+0000 7f5383e00640 1 -- 172.21.15.32:0/2136090249 shutdown_connections 2024-04-17T09:53:55.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.450+0000 7f5383e00640 1 -- 172.21.15.32:0/2136090249 wait complete. 2024-04-17T09:53:55.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f5383e00640 1 Processor -- start 2024-04-17T09:53:55.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f5383e00640 1 -- start start 2024-04-17T09:53:55.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f5383e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5384072330 0x7f538413ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:55.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f5383e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f538413f630 0x7f538413a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:55.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f5383e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5384160a30 0x7f538413a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:55.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f5383e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f538413fc90 con 0x7f5384072330 2024-04-17T09:53:55.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f5383e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f538415d5e0 con 0x7f5384160a30 2024-04-17T09:53:55.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f5383e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f538413fe10 con 0x7f538413f630 2024-04-17T09:53:55.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f5382a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5384072330 0x7f538413ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:55.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f5382a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5384072330 0x7f538413ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44800/0 (socket says 172.21.15.32:44800) 2024-04-17T09:53:55.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f5382a00640 1 -- 172.21.15.32:0/1120246044 learned_addr learned my addr 172.21.15.32:0/1120246044 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:55.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f537a000640 1 --2- 172.21.15.32:0/1120246044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f538413f630 0x7f538413a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:55.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f5383400640 1 --2- 172.21.15.32:0/1120246044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5384160a30 0x7f538413a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:55.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f5382a00640 1 -- 172.21.15.32:0/1120246044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f538413f630 msgr2=0x7f538413a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:55.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f5382a00640 1 --2- 172.21.15.32:0/1120246044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f538413f630 0x7f538413a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f5382a00640 1 -- 172.21.15.32:0/1120246044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5384160a30 msgr2=0x7f538413a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:55.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f5382a00640 1 --2- 172.21.15.32:0/1120246044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5384160a30 0x7f538413a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f5382a00640 1 -- 172.21.15.32:0/1120246044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5384138a70 con 0x7f5384072330 2024-04-17T09:53:55.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f5382a00640 1 --2- 172.21.15.32:0/1120246044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5384072330 0x7f538413ae60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f537401a0d0 tx=0x7f5374019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:55.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f537b400640 1 -- 172.21.15.32:0/1120246044 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53740ba020 con 0x7f5384072330 2024-04-17T09:53:55.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f537b400640 1 -- 172.21.15.32:0/1120246044 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53740b6410 con 0x7f5384072330 2024-04-17T09:53:55.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f537b400640 1 -- 172.21.15.32:0/1120246044 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53740b2040 con 0x7f5384072330 2024-04-17T09:53:55.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f5383e00640 1 -- 172.21.15.32:0/1120246044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5384138d50 con 0x7f5384072330 2024-04-17T09:53:55.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f5383e00640 1 -- 172.21.15.32:0/1120246044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5384139310 con 0x7f5384072330 2024-04-17T09:53:55.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.454+0000 7f5383e00640 1 -- 172.21.15.32:0/1120246044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f538404fda0 con 0x7f5384072330 2024-04-17T09:53:55.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.458+0000 7f537b400640 1 -- 172.21.15.32:0/1120246044 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f53740bd020 con 0x7f5384072330 2024-04-17T09:53:55.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.458+0000 7f537b400640 1 --2- 172.21.15.32:0/1120246044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5360068380 0x7f536006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:55.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.458+0000 7f537b400640 1 -- 172.21.15.32:0/1120246044 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f5374120710 con 0x7f5384072330 2024-04-17T09:53:55.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.458+0000 7f537a000640 1 --2- 172.21.15.32:0/1120246044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5360068380 0x7f536006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:55.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.458+0000 7f537a000640 1 --2- 172.21.15.32:0/1120246044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5360068380 0x7f536006a840 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f536c007fd0 tx=0x7f536c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:55.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.462+0000 7f537b400640 1 -- 172.21.15.32:0/1120246044 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f53740c4030 con 0x7f5384072330 2024-04-17T09:53:55.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.630+0000 7f5383e00640 1 -- 172.21.15.32:0/1120246044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5384072f70 con 0x7f5384072330 2024-04-17T09:53:55.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.630+0000 7f537b400640 1 -- 172.21.15.32:0/1120246044 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f5374016090 con 0x7f5384072330 2024-04-17T09:53:55.630 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:53:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.630+0000 7f5383e00640 1 -- 172.21.15.32:0/1120246044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5360068380 msgr2=0x7f536006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.630+0000 7f5383e00640 1 --2- 172.21.15.32:0/1120246044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5360068380 0x7f536006a840 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f536c007fd0 tx=0x7f536c009040 comp rx=0 tx=0).stop 2024-04-17T09:53:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.630+0000 7f5383e00640 1 -- 172.21.15.32:0/1120246044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5384072330 msgr2=0x7f538413ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.630+0000 7f5383e00640 1 --2- 172.21.15.32:0/1120246044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5384072330 0x7f538413ae60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f537401a0d0 tx=0x7f5374019240 comp rx=0 tx=0).stop 2024-04-17T09:53:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.630+0000 7f5383e00640 1 -- 172.21.15.32:0/1120246044 shutdown_connections 2024-04-17T09:53:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.630+0000 7f5383e00640 1 --2- 172.21.15.32:0/1120246044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5360068380 0x7f536006a840 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.630+0000 7f5383e00640 1 --2- 172.21.15.32:0/1120246044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f538413f630 0x7f538413a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.630+0000 7f5383e00640 1 --2- 172.21.15.32:0/1120246044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5384160a30 0x7f538413a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.630+0000 7f5383e00640 1 --2- 172.21.15.32:0/1120246044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5384072330 0x7f538413ae60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.630+0000 7f5383e00640 1 -- 172.21.15.32:0/1120246044 >> 172.21.15.32:0/1120246044 conn(0x7f5384067d70 msgr2=0x7f53841534f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.634+0000 7f5383e00640 1 -- 172.21.15.32:0/1120246044 shutdown_connections 2024-04-17T09:53:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.634+0000 7f5383e00640 1 -- 172.21.15.32:0/1120246044 wait complete. 2024-04-17T09:53:55.652 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:53:55.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.794+0000 7f438a200640 1 Processor -- start 2024-04-17T09:53:55.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.794+0000 7f438a200640 1 -- start start 2024-04-17T09:53:55.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.794+0000 7f438a200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f438415b7d0 0x7f438406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:55.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.794+0000 7f438a200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f438406f7c0 0x7f438406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:55.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.794+0000 7f438a200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43840cc940 con 0x7f438406f270 2024-04-17T09:53:55.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.794+0000 7f438a200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f43840ccc20 con 0x7f438415b7d0 2024-04-17T09:53:55.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.794+0000 7f438a200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f438406e0b0 con 0x7f438406f7c0 2024-04-17T09:53:55.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.794+0000 7f4383e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f438406f270 0x7f43840ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54136/0 (socket says 172.21.15.32:54136) 2024-04-17T09:53:55.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.794+0000 7f4383400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f438415b7d0 0x7f438406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:55.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.794+0000 7f4383e00640 1 -- 172.21.15.32:0/4248104282 learned_addr learned my addr 172.21.15.32:0/4248104282 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:55.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.794+0000 7f437aa00640 1 --2- 172.21.15.32:0/4248104282 >> v2:172.21.15.144:3300/0 conn(0x7f438406f7c0 0x7f438406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:55.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.794+0000 7f4383400640 1 -- 172.21.15.32:0/4248104282 >> v2:172.21.15.144:3300/0 conn(0x7f438406f7c0 msgr2=0x7f438406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:55.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.794+0000 7f4383400640 1 --2- 172.21.15.32:0/4248104282 >> v2:172.21.15.144:3300/0 conn(0x7f438406f7c0 0x7f438406ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.794+0000 7f4383400640 1 -- 172.21.15.32:0/4248104282 >> v1:172.21.15.32:6789/0 conn(0x7f438406f270 legacy=0x7f43840ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:55.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.794+0000 7f4383400640 1 -- 172.21.15.32:0/4248104282 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f438406e390 con 0x7f438415b7d0 2024-04-17T09:53:55.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.794+0000 7f4382a00640 1 -- 172.21.15.32:0/4248104282 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2860948590 0 0) 0x7f43840cc940 con 0x7f438406f270 2024-04-17T09:53:55.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.794+0000 7f437aa00640 1 --2- 172.21.15.32:0/4248104282 >> v2:172.21.15.144:3300/0 conn(0x7f438406f7c0 0x7f438406ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:55.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.798+0000 7f4383400640 1 --2- 172.21.15.32:0/4248104282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f438415b7d0 0x7f438406e8b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f43840b58f0 tx=0x7f437401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa64bf96cf45843 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:55.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.798+0000 7f4382a00640 1 -- 172.21.15.32:0/4248104282 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43740b2070 con 0x7f438415b7d0 2024-04-17T09:53:55.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.798+0000 7f4382a00640 1 -- 172.21.15.32:0/4248104282 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43740a5870 con 0x7f438415b7d0 2024-04-17T09:53:55.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.798+0000 7f4382a00640 1 -- 172.21.15.32:0/4248104282 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43740a5c40 con 0x7f438415b7d0 2024-04-17T09:53:55.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f438a200640 1 -- 172.21.15.32:0/4248104282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f438415b7d0 msgr2=0x7f438406e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:55.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f438a200640 1 --2- 172.21.15.32:0/4248104282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f438415b7d0 0x7f438406e8b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f43840b58f0 tx=0x7f437401c8a0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f438a200640 1 -- 172.21.15.32:0/4248104282 shutdown_connections 2024-04-17T09:53:55.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f438a200640 1 --2- 172.21.15.32:0/4248104282 >> v2:172.21.15.144:3300/0 conn(0x7f438406f7c0 0x7f438406ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f438a200640 1 --2- 172.21.15.32:0/4248104282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f438415b7d0 0x7f438406e8b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f438a200640 1 -- 172.21.15.32:0/4248104282 >> 172.21.15.32:0/4248104282 conn(0x7f43840b2120 msgr2=0x7f43840b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:55.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f438a200640 1 -- 172.21.15.32:0/4248104282 shutdown_connections 2024-04-17T09:53:55.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f438a200640 1 -- 172.21.15.32:0/4248104282 wait complete. 2024-04-17T09:53:55.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f438a200640 1 Processor -- start 2024-04-17T09:53:55.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f438a200640 1 -- start start 2024-04-17T09:53:55.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f438a200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f438406f270 0x7f4384144700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:55.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f438a200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f438406f7c0 0x7f43840af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:55.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f438a200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f438415b7d0 0x7f43841599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:55.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f438a200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4384168ab0 con 0x7f438406f7c0 2024-04-17T09:53:55.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f438a200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f438406c380 con 0x7f438415b7d0 2024-04-17T09:53:55.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f438a200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4384168c30 con 0x7f438406f270 2024-04-17T09:53:55.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f437aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f438406f7c0 0x7f43840af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:55.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f437aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f438406f7c0 0x7f43840af6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44828/0 (socket says 172.21.15.32:44828) 2024-04-17T09:53:55.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f437aa00640 1 -- 172.21.15.32:0/449786618 learned_addr learned my addr 172.21.15.32:0/449786618 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:55.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f4383400640 1 --2- 172.21.15.32:0/449786618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f438406f270 0x7f4384144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:55.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f4383e00640 1 --2- 172.21.15.32:0/449786618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f438415b7d0 0x7f43841599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:55.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f437aa00640 1 -- 172.21.15.32:0/449786618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f438406f270 msgr2=0x7f4384144700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:55.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f437aa00640 1 --2- 172.21.15.32:0/449786618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f438406f270 0x7f4384144700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f437aa00640 1 -- 172.21.15.32:0/449786618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f438415b7d0 msgr2=0x7f43841599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:55.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f437aa00640 1 --2- 172.21.15.32:0/449786618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f438415b7d0 0x7f43841599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f437aa00640 1 -- 172.21.15.32:0/449786618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43840b1ee0 con 0x7f438406f7c0 2024-04-17T09:53:55.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.802+0000 7f4383e00640 1 --2- 172.21.15.32:0/449786618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f438415b7d0 0x7f43841599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:55.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.806+0000 7f437aa00640 1 --2- 172.21.15.32:0/449786618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f438406f7c0 0x7f43840af6d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f436c00e810 tx=0x7f436c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:55.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.806+0000 7f437be00640 1 -- 172.21.15.32:0/449786618 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c017070 con 0x7f438406f7c0 2024-04-17T09:53:55.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.806+0000 7f437be00640 1 -- 172.21.15.32:0/449786618 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f436c00c8d0 con 0x7f438406f7c0 2024-04-17T09:53:55.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.806+0000 7f437be00640 1 -- 172.21.15.32:0/449786618 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c00f040 con 0x7f438406f7c0 2024-04-17T09:53:55.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.806+0000 7f438a200640 1 -- 172.21.15.32:0/449786618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43840afde0 con 0x7f438406f7c0 2024-04-17T09:53:55.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.806+0000 7f438a200640 1 -- 172.21.15.32:0/449786618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4384141f00 con 0x7f438406f7c0 2024-04-17T09:53:55.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.806+0000 7f438a200640 1 -- 172.21.15.32:0/449786618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4384068a90 con 0x7f438406f7c0 2024-04-17T09:53:55.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.806+0000 7f437be00640 1 -- 172.21.15.32:0/449786618 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f436c01a070 con 0x7f438406f7c0 2024-04-17T09:53:55.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.806+0000 7f437be00640 1 --2- 172.21.15.32:0/449786618 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4360068380 0x7f436006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:55.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.806+0000 7f437be00640 1 -- 172.21.15.32:0/449786618 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f436c086630 con 0x7f438406f7c0 2024-04-17T09:53:55.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.806+0000 7f4383400640 1 --2- 172.21.15.32:0/449786618 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4360068380 0x7f436006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:55.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.810+0000 7f4383400640 1 --2- 172.21.15.32:0/449786618 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4360068380 0x7f436006a840 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f43740af000 tx=0x7f43740a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:55.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.810+0000 7f437be00640 1 -- 172.21.15.32:0/449786618 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f436c056480 con 0x7f438406f7c0 2024-04-17T09:53:55.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.978+0000 7f438a200640 1 -- 172.21.15.32:0/449786618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f43840b4eb0 con 0x7f438406f7c0 2024-04-17T09:53:55.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.978+0000 7f437be00640 1 -- 172.21.15.32:0/449786618 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f436c05a780 con 0x7f438406f7c0 2024-04-17T09:53:55.978 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:53:55.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.978+0000 7f438a200640 1 -- 172.21.15.32:0/449786618 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4360068380 msgr2=0x7f436006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:55.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.978+0000 7f438a200640 1 --2- 172.21.15.32:0/449786618 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4360068380 0x7f436006a840 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f43740af000 tx=0x7f43740a7070 comp rx=0 tx=0).stop 2024-04-17T09:53:55.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.978+0000 7f438a200640 1 -- 172.21.15.32:0/449786618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f438406f7c0 msgr2=0x7f43840af6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:55.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.978+0000 7f438a200640 1 --2- 172.21.15.32:0/449786618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f438406f7c0 0x7f43840af6d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f436c00e810 tx=0x7f436c00ecd0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.978+0000 7f438a200640 1 -- 172.21.15.32:0/449786618 shutdown_connections 2024-04-17T09:53:55.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.978+0000 7f438a200640 1 --2- 172.21.15.32:0/449786618 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4360068380 0x7f436006a840 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.978+0000 7f438a200640 1 --2- 172.21.15.32:0/449786618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f438415b7d0 0x7f43841599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.978+0000 7f438a200640 1 --2- 172.21.15.32:0/449786618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f438406f7c0 0x7f43840af6d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.978+0000 7f438a200640 1 --2- 172.21.15.32:0/449786618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f438406f270 0x7f4384144700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:55.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.978+0000 7f438a200640 1 -- 172.21.15.32:0/449786618 >> 172.21.15.32:0/449786618 conn(0x7f43840b2120 msgr2=0x7f43840b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:55.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.978+0000 7f438a200640 1 -- 172.21.15.32:0/449786618 shutdown_connections 2024-04-17T09:53:55.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:55.978+0000 7f438a200640 1 -- 172.21.15.32:0/449786618 wait complete. 2024-04-17T09:53:56.000 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:53:56.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.142+0000 7f1658c00640 1 Processor -- start 2024-04-17T09:53:56.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.142+0000 7f1658c00640 1 -- start start 2024-04-17T09:53:56.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.142+0000 7f1658c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f16540ce740 0x7f165409a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:56.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.142+0000 7f1658c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f16540ceba0 0x7f165409af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:56.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.146+0000 7f1658c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f165415cc40 con 0x7f165415c860 2024-04-17T09:53:56.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.146+0000 7f1658c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f16540ce2b0 con 0x7f16540ceba0 2024-04-17T09:53:56.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.146+0000 7f1658c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f16540cd6a0 con 0x7f16540ce740 2024-04-17T09:53:56.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.146+0000 7f1652800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f16540ceba0 0x7f165409af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:56.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.146+0000 7f1651e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f165415c860 0x7f16540d1050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54152/0 (socket says 172.21.15.32:54152) 2024-04-17T09:53:56.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.146+0000 7f1651e00640 1 -- 172.21.15.32:0/3207562360 learned_addr learned my addr 172.21.15.32:0/3207562360 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:56.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.146+0000 7f1649400640 1 --2- 172.21.15.32:0/3207562360 >> v2:172.21.15.144:3300/0 conn(0x7f16540ce740 0x7f165409a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:56.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.146+0000 7f1652800640 1 -- 172.21.15.32:0/3207562360 >> v2:172.21.15.144:3300/0 conn(0x7f16540ce740 msgr2=0x7f165409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:56.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.146+0000 7f1652800640 1 --2- 172.21.15.32:0/3207562360 >> v2:172.21.15.144:3300/0 conn(0x7f16540ce740 0x7f165409a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:56.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.146+0000 7f1652800640 1 -- 172.21.15.32:0/3207562360 >> v1:172.21.15.32:6789/0 conn(0x7f165415c860 legacy=0x7f16540d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:56.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.146+0000 7f1652800640 1 -- 172.21.15.32:0/3207562360 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16540cd980 con 0x7f16540ceba0 2024-04-17T09:53:56.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.146+0000 7f1652800640 1 --2- 172.21.15.32:0/3207562360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16540ceba0 0x7f165409af30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f165409bbe0 tx=0x7f164c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4fd5659348aa45 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:56.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.150+0000 7f1651400640 1 -- 172.21.15.32:0/3207562360 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c0b2070 con 0x7f16540ceba0 2024-04-17T09:53:56.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.150+0000 7f1651400640 1 -- 172.21.15.32:0/3207562360 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f164c0a58e0 con 0x7f16540ceba0 2024-04-17T09:53:56.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.150+0000 7f1651400640 1 -- 172.21.15.32:0/3207562360 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c0a5cb0 con 0x7f16540ceba0 2024-04-17T09:53:56.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.150+0000 7f1658c00640 1 -- 172.21.15.32:0/3207562360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16540ceba0 msgr2=0x7f165409af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:56.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.150+0000 7f1658c00640 1 --2- 172.21.15.32:0/3207562360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16540ceba0 0x7f165409af30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f165409bbe0 tx=0x7f164c01c930 comp rx=0 tx=0).stop 2024-04-17T09:53:56.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.150+0000 7f1658c00640 1 -- 172.21.15.32:0/3207562360 shutdown_connections 2024-04-17T09:53:56.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.150+0000 7f1658c00640 1 --2- 172.21.15.32:0/3207562360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16540ceba0 0x7f165409af30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:56.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.150+0000 7f1658c00640 1 --2- 172.21.15.32:0/3207562360 >> v2:172.21.15.144:3300/0 conn(0x7f16540ce740 0x7f165409a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:56.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.150+0000 7f1658c00640 1 -- 172.21.15.32:0/3207562360 >> 172.21.15.32:0/3207562360 conn(0x7f16540b4050 msgr2=0x7f16540b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:56.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.150+0000 7f1658c00640 1 -- 172.21.15.32:0/3207562360 shutdown_connections 2024-04-17T09:53:56.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.150+0000 7f1658c00640 1 -- 172.21.15.32:0/3207562360 wait complete. 2024-04-17T09:53:56.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.150+0000 7f1658c00640 1 Processor -- start 2024-04-17T09:53:56.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.150+0000 7f1658c00640 1 -- start start 2024-04-17T09:53:56.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.150+0000 7f1658c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16540ce740 0x7f16541525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:56.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.150+0000 7f1658c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16540ceba0 0x7f165415a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:56.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.150+0000 7f1658c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f165415c860 0x7f16540cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:56.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.150+0000 7f1658c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f16540b5bc0 con 0x7f16540ce740 2024-04-17T09:53:56.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.150+0000 7f1658c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f16540cd350 con 0x7f165415c860 2024-04-17T09:53:56.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.150+0000 7f1658c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f16540b5d40 con 0x7f16540ceba0 2024-04-17T09:53:56.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.150+0000 7f1651e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16540ce740 0x7f16541525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:56.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.150+0000 7f1651e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16540ce740 0x7f16541525a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44840/0 (socket says 172.21.15.32:44840) 2024-04-17T09:53:56.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.150+0000 7f1651e00640 1 -- 172.21.15.32:0/1954181460 learned_addr learned my addr 172.21.15.32:0/1954181460 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:56.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.154+0000 7f1651e00640 1 -- 172.21.15.32:0/1954181460 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16540ceba0 msgr2=0x7f165415a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:56.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.154+0000 7f1651e00640 1 --2- 172.21.15.32:0/1954181460 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16540ceba0 0x7f165415a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:56.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.154+0000 7f1652800640 1 --2- 172.21.15.32:0/1954181460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f165415c860 0x7f16540cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:56.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.154+0000 7f1651e00640 1 -- 172.21.15.32:0/1954181460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f165415c860 msgr2=0x7f16540cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:56.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.154+0000 7f1651e00640 1 --2- 172.21.15.32:0/1954181460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f165415c860 0x7f16540cbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:56.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.154+0000 7f1651e00640 1 -- 172.21.15.32:0/1954181460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16540c97d0 con 0x7f16540ce740 2024-04-17T09:53:56.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.154+0000 7f1652800640 1 --2- 172.21.15.32:0/1954181460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f165415c860 0x7f16540cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:56.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.154+0000 7f1651e00640 1 --2- 172.21.15.32:0/1954181460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16540ce740 0x7f16541525a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f164400c450 tx=0x7f164400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:56.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.154+0000 7f164aa00640 1 -- 172.21.15.32:0/1954181460 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1644017070 con 0x7f16540ce740 2024-04-17T09:53:56.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.154+0000 7f164aa00640 1 -- 172.21.15.32:0/1954181460 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1644005cf0 con 0x7f16540ce740 2024-04-17T09:53:56.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.154+0000 7f164aa00640 1 -- 172.21.15.32:0/1954181460 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1644007270 con 0x7f16540ce740 2024-04-17T09:53:56.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.154+0000 7f1658c00640 1 -- 172.21.15.32:0/1954181460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16540c9ab0 con 0x7f16540ce740 2024-04-17T09:53:56.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.154+0000 7f1658c00640 1 -- 172.21.15.32:0/1954181460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1654152ae0 con 0x7f16540ce740 2024-04-17T09:53:56.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.154+0000 7f1658c00640 1 -- 172.21.15.32:0/1954181460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1654068a90 con 0x7f16540ce740 2024-04-17T09:53:56.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.154+0000 7f164aa00640 1 -- 172.21.15.32:0/1954181460 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f164401a070 con 0x7f16540ce740 2024-04-17T09:53:56.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.154+0000 7f164aa00640 1 --2- 172.21.15.32:0/1954181460 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1630068380 0x7f163006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:56.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.154+0000 7f164aa00640 1 -- 172.21.15.32:0/1954181460 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f1644085ca0 con 0x7f16540ce740 2024-04-17T09:53:56.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.154+0000 7f1649400640 1 --2- 172.21.15.32:0/1954181460 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1630068380 0x7f163006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:56.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.158+0000 7f1649400640 1 --2- 172.21.15.32:0/1954181460 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1630068380 0x7f163006a840 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f16541676e0 tx=0x7f163c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.158+0000 7f164aa00640 1 -- 172.21.15.32:0/1954181460 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f1644055af0 con 0x7f16540ce740 2024-04-17T09:53:56.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.322+0000 7f1658c00640 1 -- 172.21.15.32:0/1954181460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f16540b3900 con 0x7f16540ce740 2024-04-17T09:53:56.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.322+0000 7f164aa00640 1 -- 172.21.15.32:0/1954181460 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f1644059df0 con 0x7f16540ce740 2024-04-17T09:53:56.324 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:53:56.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.326+0000 7f1658c00640 1 -- 172.21.15.32:0/1954181460 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1630068380 msgr2=0x7f163006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.326+0000 7f1658c00640 1 --2- 172.21.15.32:0/1954181460 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1630068380 0x7f163006a840 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f16541676e0 tx=0x7f163c008040 comp rx=0 tx=0).stop 2024-04-17T09:53:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.326+0000 7f1658c00640 1 -- 172.21.15.32:0/1954181460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16540ce740 msgr2=0x7f16541525a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.326+0000 7f1658c00640 1 --2- 172.21.15.32:0/1954181460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16540ce740 0x7f16541525a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f164400c450 tx=0x7f164400c910 comp rx=0 tx=0).stop 2024-04-17T09:53:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.326+0000 7f1658c00640 1 -- 172.21.15.32:0/1954181460 shutdown_connections 2024-04-17T09:53:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.326+0000 7f1658c00640 1 --2- 172.21.15.32:0/1954181460 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1630068380 0x7f163006a840 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.326+0000 7f1658c00640 1 --2- 172.21.15.32:0/1954181460 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16540ceba0 0x7f165415a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.326+0000 7f1658c00640 1 --2- 172.21.15.32:0/1954181460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f165415c860 0x7f16540cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.326+0000 7f1658c00640 1 --2- 172.21.15.32:0/1954181460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16540ce740 0x7f16541525a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.326+0000 7f1658c00640 1 -- 172.21.15.32:0/1954181460 >> 172.21.15.32:0/1954181460 conn(0x7f16540b4050 msgr2=0x7f1654157500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.326+0000 7f1658c00640 1 -- 172.21.15.32:0/1954181460 shutdown_connections 2024-04-17T09:53:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.326+0000 7f1658c00640 1 -- 172.21.15.32:0/1954181460 wait complete. 2024-04-17T09:53:56.346 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T09:53:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.486+0000 7fead9000640 1 Processor -- start 2024-04-17T09:53:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.486+0000 7fead9000640 1 -- start start 2024-04-17T09:53:56.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.486+0000 7fead9000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead40b3340 0x7fead415f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:56.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.486+0000 7fead9000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fead415f870 con 0x7fead4155e50 2024-04-17T09:53:56.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.486+0000 7fead9000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fead415fb50 con 0x7fead4155a70 2024-04-17T09:53:56.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.486+0000 7fead9000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fead40b42a0 con 0x7fead40b3340 2024-04-17T09:53:56.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.486+0000 7feac9400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fead4155e50 0x7fead415c260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54168/0 (socket says 172.21.15.32:54168) 2024-04-17T09:53:56.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.486+0000 7feac9400640 1 -- 172.21.15.32:0/191534387 learned_addr learned my addr 172.21.15.32:0/191534387 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:56.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.486+0000 7fead2800640 1 --2- 172.21.15.32:0/191534387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead40b3340 0x7fead415f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:56.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.486+0000 7fead1400640 1 -- 172.21.15.32:0/191534387 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1385144417 0 0) 0x7fead415f870 con 0x7fead4155e50 2024-04-17T09:53:56.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.486+0000 7fead1400640 1 -- 172.21.15.32:0/191534387 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feab8003530 con 0x7fead4155e50 2024-04-17T09:53:56.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.490+0000 7fead1400640 1 -- 172.21.15.32:0/191534387 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1088688950 0 0) 0x7fead415fb50 con 0x7fead4155a70 2024-04-17T09:53:56.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.490+0000 7fead1400640 1 -- 172.21.15.32:0/191534387 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fead415f870 con 0x7fead4155a70 2024-04-17T09:53:56.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.490+0000 7fead2800640 1 -- 172.21.15.32:0/191534387 >> v1:172.21.15.144:6789/0 conn(0x7fead4155a70 legacy=0x7fead415d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:56.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.490+0000 7fead2800640 1 -- 172.21.15.32:0/191534387 >> v1:172.21.15.32:6789/0 conn(0x7fead4155e50 legacy=0x7fead415c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:56.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.490+0000 7fead1400640 1 -- 172.21.15.32:0/191534387 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1744090471 0 0) 0x7feab8003530 con 0x7fead4155e50 2024-04-17T09:53:56.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.490+0000 7fead2800640 1 -- 172.21.15.32:0/191534387 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead415fe30 con 0x7fead40b3340 2024-04-17T09:53:56.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.490+0000 7fead2800640 1 --2- 172.21.15.32:0/191534387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead40b3340 0x7fead415f190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feacc009dc0 tx=0x7feacc01c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=893f554694f41234 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:56.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.490+0000 7fead1400640 1 -- 172.21.15.32:0/191534387 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc001680 con 0x7fead40b3340 2024-04-17T09:53:56.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.490+0000 7fead1400640 1 -- 172.21.15.32:0/191534387 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feacc0a6500 con 0x7fead40b3340 2024-04-17T09:53:56.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.490+0000 7fead1400640 1 -- 172.21.15.32:0/191534387 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc0a68d0 con 0x7fead40b3340 2024-04-17T09:53:56.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.490+0000 7fead9000640 1 -- 172.21.15.32:0/191534387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead40b3340 msgr2=0x7fead415f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:56.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.490+0000 7fead9000640 1 --2- 172.21.15.32:0/191534387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead40b3340 0x7fead415f190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feacc009dc0 tx=0x7feacc01c8b0 comp rx=0 tx=0).stop 2024-04-17T09:53:56.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead9000640 1 -- 172.21.15.32:0/191534387 shutdown_connections 2024-04-17T09:53:56.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead9000640 1 --2- 172.21.15.32:0/191534387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead40b3340 0x7fead415f190 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:56.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead9000640 1 -- 172.21.15.32:0/191534387 >> 172.21.15.32:0/191534387 conn(0x7fead406ba30 msgr2=0x7fead406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:56.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead9000640 1 -- 172.21.15.32:0/191534387 shutdown_connections 2024-04-17T09:53:56.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead9000640 1 -- 172.21.15.32:0/191534387 wait complete. 2024-04-17T09:53:56.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead9000640 1 Processor -- start 2024-04-17T09:53:56.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead9000640 1 -- start start 2024-04-17T09:53:56.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead9000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead40b3340 0x7fead40cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:56.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead9000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead4155a70 0x7fead4145950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:56.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead9000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fead4155e50 0x7fead40cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:56.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead9000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fead40d02f0 con 0x7fead40b3340 2024-04-17T09:53:56.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead9000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fead40d0170 con 0x7fead4155e50 2024-04-17T09:53:56.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead9000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fead40d0470 con 0x7fead4155a70 2024-04-17T09:53:56.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead40b3340 0x7fead40cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:56.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead40b3340 0x7fead40cfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44842/0 (socket says 172.21.15.32:44842) 2024-04-17T09:53:56.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead1e00640 1 -- 172.21.15.32:0/3627856960 learned_addr learned my addr 172.21.15.32:0/3627856960 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:56.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7feac9400640 1 --2- 172.21.15.32:0/3627856960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead4155a70 0x7fead4145950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:56.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead2800640 1 --2- 172.21.15.32:0/3627856960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fead4155e50 0x7fead40cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:56.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead1e00640 1 -- 172.21.15.32:0/3627856960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead4155a70 msgr2=0x7fead4145950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:56.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead1e00640 1 --2- 172.21.15.32:0/3627856960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead4155a70 0x7fead4145950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:56.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead1e00640 1 -- 172.21.15.32:0/3627856960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fead4155e50 msgr2=0x7fead40cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:56.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead1e00640 1 --2- 172.21.15.32:0/3627856960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fead4155e50 0x7fead40cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:56.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead1e00640 1 -- 172.21.15.32:0/3627856960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead4145e90 con 0x7fead40b3340 2024-04-17T09:53:56.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7feac9400640 1 --2- 172.21.15.32:0/3627856960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead4155a70 0x7fead4145950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:56.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead1e00640 1 --2- 172.21.15.32:0/3627856960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead40b3340 0x7fead40cfb50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feac400c500 tx=0x7feac400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:56.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7feacaa00640 1 -- 172.21.15.32:0/3627856960 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac40052d0 con 0x7fead40b3340 2024-04-17T09:53:56.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7feacaa00640 1 -- 172.21.15.32:0/3627856960 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feac4005490 con 0x7fead40b3340 2024-04-17T09:53:56.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7fead9000640 1 -- 172.21.15.32:0/3627856960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fead40cf5e0 con 0x7fead40b3340 2024-04-17T09:53:56.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.494+0000 7feacaa00640 1 -- 172.21.15.32:0/3627856960 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4012850 con 0x7fead40b3340 2024-04-17T09:53:56.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.498+0000 7fead9000640 1 -- 172.21.15.32:0/3627856960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fead4156790 con 0x7fead40b3340 2024-04-17T09:53:56.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.498+0000 7fead9000640 1 -- 172.21.15.32:0/3627856960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fead4068a90 con 0x7fead40b3340 2024-04-17T09:53:56.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.498+0000 7feacaa00640 1 -- 172.21.15.32:0/3627856960 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7feac40129f0 con 0x7fead40b3340 2024-04-17T09:53:56.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.498+0000 7feacaa00640 1 --2- 172.21.15.32:0/3627856960 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7feab0068380 0x7feab006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:56.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.498+0000 7feacaa00640 1 -- 172.21.15.32:0/3627856960 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7feac4086720 con 0x7fead40b3340 2024-04-17T09:53:56.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.498+0000 7feac9400640 1 --2- 172.21.15.32:0/3627856960 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7feab0068380 0x7feab006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:56.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.498+0000 7feac9400640 1 --2- 172.21.15.32:0/3627856960 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7feab0068380 0x7feab006a840 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fead41faae0 tx=0x7feabc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:56.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.502+0000 7feacaa00640 1 -- 172.21.15.32:0/3627856960 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fead4145e90 con 0x7fead40b3340 2024-04-17T09:53:56.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.678+0000 7fead9000640 1 -- 172.21.15.32:0/3627856960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fead415c550 con 0x7fead40b3340 2024-04-17T09:53:56.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.678+0000 7feacaa00640 1 -- 172.21.15.32:0/3627856960 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7feac4056620 con 0x7fead40b3340 2024-04-17T09:53:56.679 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T09:53:56.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.682+0000 7fead9000640 1 -- 172.21.15.32:0/3627856960 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7feab0068380 msgr2=0x7feab006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.682+0000 7fead9000640 1 --2- 172.21.15.32:0/3627856960 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7feab0068380 0x7feab006a840 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fead41faae0 tx=0x7feabc009040 comp rx=0 tx=0).stop 2024-04-17T09:53:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.682+0000 7fead9000640 1 -- 172.21.15.32:0/3627856960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead40b3340 msgr2=0x7fead40cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.682+0000 7fead9000640 1 --2- 172.21.15.32:0/3627856960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead40b3340 0x7fead40cfb50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feac400c500 tx=0x7feac400c9c0 comp rx=0 tx=0).stop 2024-04-17T09:53:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.682+0000 7fead9000640 1 -- 172.21.15.32:0/3627856960 shutdown_connections 2024-04-17T09:53:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.682+0000 7fead9000640 1 --2- 172.21.15.32:0/3627856960 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7feab0068380 0x7feab006a840 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.682+0000 7fead9000640 1 --2- 172.21.15.32:0/3627856960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fead4155a70 0x7fead4145950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.682+0000 7fead9000640 1 --2- 172.21.15.32:0/3627856960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fead4155e50 0x7fead40cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.682+0000 7fead9000640 1 --2- 172.21.15.32:0/3627856960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead40b3340 0x7fead40cfb50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.682+0000 7fead9000640 1 -- 172.21.15.32:0/3627856960 >> 172.21.15.32:0/3627856960 conn(0x7fead406ba30 msgr2=0x7fead40d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.682+0000 7fead9000640 1 -- 172.21.15.32:0/3627856960 shutdown_connections 2024-04-17T09:53:56.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.682+0000 7fead9000640 1 -- 172.21.15.32:0/3627856960 wait complete. 2024-04-17T09:53:56.702 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:53:56.702 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:53:56.711 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:53:56.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.894+0000 7f4177000640 1 Processor -- start 2024-04-17T09:53:56.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.894+0000 7f4177000640 1 -- start start 2024-04-17T09:53:56.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.898+0000 7f4177000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f417015baa0 0x7f417015bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:56.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.898+0000 7f4177000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4170072bb0 con 0x7f417006ef60 2024-04-17T09:53:56.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.898+0000 7f4177000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41700b3dc0 con 0x7f4170144e90 2024-04-17T09:53:56.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.898+0000 7f4177000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f4170145270 con 0x7f417015baa0 2024-04-17T09:53:56.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.898+0000 7f4175000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f417006ef60 0x7f4170071e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54182/0 (socket says 172.21.15.32:54182) 2024-04-17T09:53:56.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.898+0000 7f4175000640 1 -- 172.21.15.32:0/2018737987 learned_addr learned my addr 172.21.15.32:0/2018737987 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:56.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.898+0000 7f4167400640 1 --2- 172.21.15.32:0/2018737987 >> v2:172.21.15.144:3300/0 conn(0x7f417015baa0 0x7f417015bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:56.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.898+0000 7f416f400640 1 -- 172.21.15.32:0/2018737987 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 294999171 0 0) 0x7f4170072bb0 con 0x7f417006ef60 2024-04-17T09:53:56.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.898+0000 7f416f400640 1 -- 172.21.15.32:0/2018737987 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4154005740 con 0x7f417006ef60 2024-04-17T09:53:56.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.898+0000 7f416f400640 1 -- 172.21.15.32:0/2018737987 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1550647250 0 0) 0x7f41700b3dc0 con 0x7f4170144e90 2024-04-17T09:53:56.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.898+0000 7f416f400640 1 -- 172.21.15.32:0/2018737987 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4170072bb0 con 0x7f4170144e90 2024-04-17T09:53:56.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.898+0000 7f416f400640 1 -- 172.21.15.32:0/2018737987 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3762875848 0 0) 0x7f4154005740 con 0x7f417006ef60 2024-04-17T09:53:56.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.898+0000 7f4167400640 1 -- 172.21.15.32:0/2018737987 >> v1:172.21.15.144:6789/0 conn(0x7f4170144e90 legacy=0x7f41700b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:56.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.898+0000 7f4167400640 1 -- 172.21.15.32:0/2018737987 >> v1:172.21.15.32:6789/0 conn(0x7f417006ef60 legacy=0x7f4170071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:56.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.898+0000 7f4167400640 1 -- 172.21.15.32:0/2018737987 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41700b40a0 con 0x7f417015baa0 2024-04-17T09:53:56.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.902+0000 7f4167400640 1 --2- 172.21.15.32:0/2018737987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f417015baa0 0x7f417015bf30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f41580097c0 tx=0x7f415801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c29e03131d42f49c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:56.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.902+0000 7f416f400640 1 -- 172.21.15.32:0/2018737987 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41580b0070 con 0x7f417015baa0 2024-04-17T09:53:56.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.902+0000 7f416f400640 1 -- 172.21.15.32:0/2018737987 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41580ab400 con 0x7f417015baa0 2024-04-17T09:53:56.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.902+0000 7f416f400640 1 -- 172.21.15.32:0/2018737987 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168001a00 con 0x7f417015baa0 2024-04-17T09:53:56.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.902+0000 7f4177000640 1 -- 172.21.15.32:0/2018737987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f417015baa0 msgr2=0x7f417015bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:56.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.902+0000 7f4177000640 1 --2- 172.21.15.32:0/2018737987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f417015baa0 0x7f417015bf30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f41580097c0 tx=0x7f415801c3c0 comp rx=0 tx=0).stop 2024-04-17T09:53:56.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.902+0000 7f4177000640 1 -- 172.21.15.32:0/2018737987 shutdown_connections 2024-04-17T09:53:56.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.902+0000 7f4177000640 1 --2- 172.21.15.32:0/2018737987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f417015baa0 0x7f417015bf30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:56.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.902+0000 7f4177000640 1 -- 172.21.15.32:0/2018737987 >> 172.21.15.32:0/2018737987 conn(0x7f417006ba30 msgr2=0x7f417006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:56.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.902+0000 7f4177000640 1 -- 172.21.15.32:0/2018737987 shutdown_connections 2024-04-17T09:53:56.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.902+0000 7f4177000640 1 -- 172.21.15.32:0/2018737987 wait complete. 2024-04-17T09:53:56.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.902+0000 7f4177000640 1 Processor -- start 2024-04-17T09:53:56.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.902+0000 7f4177000640 1 -- start start 2024-04-17T09:53:56.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.902+0000 7f4177000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f417006ef60 0x7f41701fb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:56.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.902+0000 7f4177000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4170144e90 0x7f417015af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:56.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.902+0000 7f4177000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f417015baa0 0x7f41700ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:56.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.902+0000 7f4177000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f417006e7e0 con 0x7f417006ef60 2024-04-17T09:53:56.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.902+0000 7f4177000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f417006e660 con 0x7f4170144e90 2024-04-17T09:53:56.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.902+0000 7f4177000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f417006e960 con 0x7f417015baa0 2024-04-17T09:53:56.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.902+0000 7f416fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f417006ef60 0x7f41701fb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:56.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.902+0000 7f416fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f417006ef60 0x7f41701fb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44850/0 (socket says 172.21.15.32:44850) 2024-04-17T09:53:56.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.902+0000 7f416fe00640 1 -- 172.21.15.32:0/1729262384 learned_addr learned my addr 172.21.15.32:0/1729262384 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:56.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.906+0000 7f4167400640 1 --2- 172.21.15.32:0/1729262384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4170144e90 0x7f417015af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:56.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.906+0000 7f4175000640 1 --2- 172.21.15.32:0/1729262384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f417015baa0 0x7f41700ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:56.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.906+0000 7f416fe00640 1 -- 172.21.15.32:0/1729262384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f417015baa0 msgr2=0x7f41700ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:56.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.906+0000 7f416fe00640 1 --2- 172.21.15.32:0/1729262384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f417015baa0 0x7f41700ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:56.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.906+0000 7f416fe00640 1 -- 172.21.15.32:0/1729262384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4170144e90 msgr2=0x7f417015af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:56.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.906+0000 7f416fe00640 1 --2- 172.21.15.32:0/1729262384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4170144e90 0x7f417015af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:56.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.906+0000 7f416fe00640 1 -- 172.21.15.32:0/1729262384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f417013f930 con 0x7f417006ef60 2024-04-17T09:53:56.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.906+0000 7f416fe00640 1 --2- 172.21.15.32:0/1729262384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f417006ef60 0x7f41701fb970 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f416000c490 tx=0x7f416000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:56.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.906+0000 7f416cc00640 1 -- 172.21.15.32:0/1729262384 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41600052d0 con 0x7f417006ef60 2024-04-17T09:53:56.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.906+0000 7f416cc00640 1 -- 172.21.15.32:0/1729262384 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4160005470 con 0x7f417006ef60 2024-04-17T09:53:56.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.906+0000 7f4177000640 1 -- 172.21.15.32:0/1729262384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41700c5e10 con 0x7f417006ef60 2024-04-17T09:53:56.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.906+0000 7f416cc00640 1 -- 172.21.15.32:0/1729262384 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160012870 con 0x7f417006ef60 2024-04-17T09:53:56.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.906+0000 7f4177000640 1 -- 172.21.15.32:0/1729262384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41700c63d0 con 0x7f417006ef60 2024-04-17T09:53:56.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.906+0000 7f4177000640 1 -- 172.21.15.32:0/1729262384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4170068a90 con 0x7f417006ef60 2024-04-17T09:53:56.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.906+0000 7f416cc00640 1 -- 172.21.15.32:0/1729262384 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4160012a10 con 0x7f417006ef60 2024-04-17T09:53:56.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.906+0000 7f416cc00640 1 --2- 172.21.15.32:0/1729262384 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f414c068380 0x7f414c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:56.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.906+0000 7f416cc00640 1 -- 172.21.15.32:0/1729262384 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f41600867a0 con 0x7f417006ef60 2024-04-17T09:53:56.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.910+0000 7f4167400640 1 --2- 172.21.15.32:0/1729262384 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f414c068380 0x7f414c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:56.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.910+0000 7f4167400640 1 --2- 172.21.15.32:0/1729262384 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f414c068380 0x7f414c06a840 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f41580ad000 tx=0x7f4158008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:56.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:56.910+0000 7f416cc00640 1 -- 172.21.15.32:0/1729262384 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f4160056c40 con 0x7f417006ef60 2024-04-17T09:53:57.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.078+0000 7f4177000640 1 -- 172.21.15.32:0/1729262384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f417015b480 con 0x7f417006ef60 2024-04-17T09:53:57.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.078+0000 7f416cc00640 1 -- 172.21.15.32:0/1729262384 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16866 (secure 0 0 0) 0x7f4160056540 con 0x7f417006ef60 2024-04-17T09:53:57.079 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:57.079 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:54.932571+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:53:57.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.078+0000 7f4177000640 1 -- 172.21.15.32:0/1729262384 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f414c068380 msgr2=0x7f414c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:57.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.078+0000 7f4177000640 1 --2- 172.21.15.32:0/1729262384 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f414c068380 0x7f414c06a840 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f41580ad000 tx=0x7f4158008040 comp rx=0 tx=0).stop 2024-04-17T09:53:57.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.078+0000 7f4177000640 1 -- 172.21.15.32:0/1729262384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f417006ef60 msgr2=0x7f41701fb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:57.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.078+0000 7f4177000640 1 --2- 172.21.15.32:0/1729262384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f417006ef60 0x7f41701fb970 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f416000c490 tx=0x7f416000c950 comp rx=0 tx=0).stop 2024-04-17T09:53:57.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.078+0000 7f4177000640 1 -- 172.21.15.32:0/1729262384 shutdown_connections 2024-04-17T09:53:57.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.078+0000 7f4177000640 1 --2- 172.21.15.32:0/1729262384 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f414c068380 0x7f414c06a840 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.078+0000 7f4177000640 1 --2- 172.21.15.32:0/1729262384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f417015baa0 0x7f41700ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.078+0000 7f4177000640 1 --2- 172.21.15.32:0/1729262384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4170144e90 0x7f417015af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.078+0000 7f4177000640 1 --2- 172.21.15.32:0/1729262384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f417006ef60 0x7f41701fb970 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.078+0000 7f4177000640 1 -- 172.21.15.32:0/1729262384 >> 172.21.15.32:0/1729262384 conn(0x7f417006ba30 msgr2=0x7f417006bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:57.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.082+0000 7f4177000640 1 -- 172.21.15.32:0/1729262384 shutdown_connections 2024-04-17T09:53:57.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.082+0000 7f4177000640 1 -- 172.21.15.32:0/1729262384 wait complete. 2024-04-17T09:53:57.100 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:53:11.198098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:53:12.118982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:53:57.101 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:53:57.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.242+0000 7f778ac00640 1 Processor -- start 2024-04-17T09:53:57.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.242+0000 7f778ac00640 1 -- start start 2024-04-17T09:53:57.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.246+0000 7f778ac00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f77840cf370 0x7f77840cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:57.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.246+0000 7f778ac00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77840d0130 con 0x7f778406ba30 2024-04-17T09:53:57.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.246+0000 7f778ac00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f778415d090 con 0x7f7784071e60 2024-04-17T09:53:57.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.246+0000 7f778ac00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f778415d370 con 0x7f77840cf370 2024-04-17T09:53:57.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.246+0000 7f7783e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f778406ba30 0x7f778406be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54184/0 (socket says 172.21.15.32:54184) 2024-04-17T09:53:57.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.246+0000 7f7783e00640 1 -- 172.21.15.32:0/2305211391 learned_addr learned my addr 172.21.15.32:0/2305211391 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:57.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.246+0000 7f777b400640 1 --2- 172.21.15.32:0/2305211391 >> v2:172.21.15.144:3300/0 conn(0x7f77840cf370 0x7f77840cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:57.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.246+0000 7f7783400640 1 -- 172.21.15.32:0/2305211391 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 31825943 0 0) 0x7f77840d0130 con 0x7f778406ba30 2024-04-17T09:53:57.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.246+0000 7f7783400640 1 -- 172.21.15.32:0/2305211391 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7768003530 con 0x7f778406ba30 2024-04-17T09:53:57.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.246+0000 7f777b400640 1 -- 172.21.15.32:0/2305211391 >> v1:172.21.15.144:6789/0 conn(0x7f7784071e60 legacy=0x7f7784072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:57.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.246+0000 7f777b400640 1 -- 172.21.15.32:0/2305211391 >> v1:172.21.15.32:6789/0 conn(0x7f778406ba30 legacy=0x7f778406be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:57.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.246+0000 7f777b400640 1 -- 172.21.15.32:0/2305211391 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778415d650 con 0x7f77840cf370 2024-04-17T09:53:57.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.250+0000 7f777b400640 1 --2- 172.21.15.32:0/2305211391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77840cf370 0x7f77840cf7e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f776c0097c0 tx=0x7f776c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d43737af19780fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:57.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.250+0000 7f7783400640 1 -- 172.21.15.32:0/2305211391 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c0aa600 con 0x7f77840cf370 2024-04-17T09:53:57.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.250+0000 7f7783400640 1 -- 172.21.15.32:0/2305211391 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f776c0af070 con 0x7f77840cf370 2024-04-17T09:53:57.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.250+0000 7f7783400640 1 -- 172.21.15.32:0/2305211391 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c0aa9a0 con 0x7f77840cf370 2024-04-17T09:53:57.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.250+0000 7f778ac00640 1 -- 172.21.15.32:0/2305211391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77840cf370 msgr2=0x7f77840cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:57.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.250+0000 7f778ac00640 1 --2- 172.21.15.32:0/2305211391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77840cf370 0x7f77840cf7e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f776c0097c0 tx=0x7f776c01c4a0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.250+0000 7f778ac00640 1 -- 172.21.15.32:0/2305211391 shutdown_connections 2024-04-17T09:53:57.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.250+0000 7f778ac00640 1 --2- 172.21.15.32:0/2305211391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77840cf370 0x7f77840cf7e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.250+0000 7f778ac00640 1 -- 172.21.15.32:0/2305211391 >> 172.21.15.32:0/2305211391 conn(0x7f77840b2120 msgr2=0x7f77840b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:57.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.250+0000 7f778ac00640 1 -- 172.21.15.32:0/2305211391 shutdown_connections 2024-04-17T09:53:57.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.250+0000 7f778ac00640 1 -- 172.21.15.32:0/2305211391 wait complete. 2024-04-17T09:53:57.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.250+0000 7f778ac00640 1 Processor -- start 2024-04-17T09:53:57.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.250+0000 7f778ac00640 1 -- start start 2024-04-17T09:53:57.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.254+0000 7f778ac00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f778406ba30 0x7f7784141360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:57.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.254+0000 7f778ac00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7784071e60 0x7f778406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:57.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.254+0000 7f778ac00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f77840cf370 0x7f7784159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:57.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.254+0000 7f778ac00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f778415dcf0 con 0x7f7784071e60 2024-04-17T09:53:57.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.254+0000 7f778ac00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f778415db70 con 0x7f778406ba30 2024-04-17T09:53:57.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.254+0000 7f778ac00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7784146230 con 0x7f77840cf370 2024-04-17T09:53:57.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.254+0000 7f777b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7784071e60 0x7f778406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:57.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.254+0000 7f777b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7784071e60 0x7f778406e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44864/0 (socket says 172.21.15.32:44864) 2024-04-17T09:53:57.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.254+0000 7f777b400640 1 -- 172.21.15.32:0/3379482647 learned_addr learned my addr 172.21.15.32:0/3379482647 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:57.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.254+0000 7f7783e00640 1 --2- 172.21.15.32:0/3379482647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f778406ba30 0x7f7784141360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:57.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.254+0000 7f7788c00640 1 --2- 172.21.15.32:0/3379482647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f77840cf370 0x7f7784159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:57.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.254+0000 7f777b400640 1 -- 172.21.15.32:0/3379482647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f77840cf370 msgr2=0x7f7784159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:57.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.254+0000 7f777b400640 1 --2- 172.21.15.32:0/3379482647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f77840cf370 0x7f7784159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.254+0000 7f777b400640 1 -- 172.21.15.32:0/3379482647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f778406ba30 msgr2=0x7f7784141360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:57.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.254+0000 7f777b400640 1 --2- 172.21.15.32:0/3379482647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f778406ba30 0x7f7784141360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.254+0000 7f777b400640 1 -- 172.21.15.32:0/3379482647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77841418a0 con 0x7f7784071e60 2024-04-17T09:53:57.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.254+0000 7f777b400640 1 --2- 172.21.15.32:0/3379482647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7784071e60 0x7f778406e5d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f776c0097c0 tx=0x7f776c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:57.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.254+0000 7f7780c00640 1 -- 172.21.15.32:0/3379482647 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c006e20 con 0x7f7784071e60 2024-04-17T09:53:57.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.254+0000 7f7780c00640 1 -- 172.21.15.32:0/3379482647 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f776c004220 con 0x7f7784071e60 2024-04-17T09:53:57.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.254+0000 7f778ac00640 1 -- 172.21.15.32:0/3379482647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f778413fba0 con 0x7f7784071e60 2024-04-17T09:53:57.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.254+0000 7f7780c00640 1 -- 172.21.15.32:0/3379482647 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c003620 con 0x7f7784071e60 2024-04-17T09:53:57.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.254+0000 7f778ac00640 1 -- 172.21.15.32:0/3379482647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77840b3930 con 0x7f7784071e60 2024-04-17T09:53:57.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.254+0000 7f778ac00640 1 -- 172.21.15.32:0/3379482647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7784068a90 con 0x7f7784071e60 2024-04-17T09:53:57.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.258+0000 7f7780c00640 1 -- 172.21.15.32:0/3379482647 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f776c0a9070 con 0x7f7784071e60 2024-04-17T09:53:57.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.258+0000 7f7780c00640 1 --2- 172.21.15.32:0/3379482647 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f77600683b0 0x7f776006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:57.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.258+0000 7f7780c00640 1 -- 172.21.15.32:0/3379482647 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f776c120d30 con 0x7f7784071e60 2024-04-17T09:53:57.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.258+0000 7f7783e00640 1 --2- 172.21.15.32:0/3379482647 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f77600683b0 0x7f776006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:57.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.258+0000 7f7783e00640 1 --2- 172.21.15.32:0/3379482647 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f77600683b0 0x7f776006a870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f77740011f0 tx=0x7f7774007d40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:57.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.262+0000 7f7780c00640 1 -- 172.21.15.32:0/3379482647 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f776c016090 con 0x7f7784071e60 2024-04-17T09:53:57.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.426+0000 7f778ac00640 1 -- 172.21.15.32:0/3379482647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f778406c850 con 0x7f7784071e60 2024-04-17T09:53:57.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.430+0000 7f7780c00640 1 -- 172.21.15.32:0/3379482647 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f776c0f0c30 con 0x7f7784071e60 2024-04-17T09:53:57.429 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:53:57.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.430+0000 7f778ac00640 1 -- 172.21.15.32:0/3379482647 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f77600683b0 msgr2=0x7f776006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:57.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.430+0000 7f778ac00640 1 --2- 172.21.15.32:0/3379482647 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f77600683b0 0x7f776006a870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f77740011f0 tx=0x7f7774007d40 comp rx=0 tx=0).stop 2024-04-17T09:53:57.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.430+0000 7f778ac00640 1 -- 172.21.15.32:0/3379482647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7784071e60 msgr2=0x7f778406e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:57.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.430+0000 7f778ac00640 1 --2- 172.21.15.32:0/3379482647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7784071e60 0x7f778406e5d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f776c0097c0 tx=0x7f776c01ace0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.430+0000 7f778ac00640 1 -- 172.21.15.32:0/3379482647 shutdown_connections 2024-04-17T09:53:57.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.430+0000 7f778ac00640 1 --2- 172.21.15.32:0/3379482647 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f77600683b0 0x7f776006a870 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.430+0000 7f778ac00640 1 --2- 172.21.15.32:0/3379482647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f77840cf370 0x7f7784159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.430+0000 7f778ac00640 1 --2- 172.21.15.32:0/3379482647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7784071e60 0x7f778406e5d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.430+0000 7f778ac00640 1 --2- 172.21.15.32:0/3379482647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f778406ba30 0x7f7784141360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.430+0000 7f778ac00640 1 -- 172.21.15.32:0/3379482647 >> 172.21.15.32:0/3379482647 conn(0x7f77840b2120 msgr2=0x7f778415b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:57.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.430+0000 7f778ac00640 1 -- 172.21.15.32:0/3379482647 shutdown_connections 2024-04-17T09:53:57.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.430+0000 7f778ac00640 1 -- 172.21.15.32:0/3379482647 wait complete. 2024-04-17T09:53:57.451 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:53:57.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.594+0000 7fbf42e00640 1 Processor -- start 2024-04-17T09:53:57.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.594+0000 7fbf42e00640 1 -- start start 2024-04-17T09:53:57.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.594+0000 7fbf42e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf3c072970 0x7fbf3c15eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:57.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.594+0000 7fbf42e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbf3c0b4490 0x7fbf3c0b5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:57.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.594+0000 7fbf42e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf3c0b56d0 con 0x7fbf3c15e7c0 2024-04-17T09:53:57.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.594+0000 7fbf42e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fbf3c0b39b0 con 0x7fbf3c0b4490 2024-04-17T09:53:57.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.594+0000 7fbf42e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbf3c0b3c90 con 0x7fbf3c072970 2024-04-17T09:53:57.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.594+0000 7fbf3be00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbf3c15e7c0 0x7fbf3c072260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54188/0 (socket says 172.21.15.32:54188) 2024-04-17T09:53:57.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.594+0000 7fbf3be00640 1 -- 172.21.15.32:0/901773546 learned_addr learned my addr 172.21.15.32:0/901773546 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:57.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.594+0000 7fbf33e00640 1 --2- 172.21.15.32:0/901773546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf3c072970 0x7fbf3c15eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:57.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.594+0000 7fbf40e00640 1 --2- 172.21.15.32:0/901773546 >> v2:172.21.15.144:3300/0 conn(0x7fbf3c0b4490 0x7fbf3c0b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:57.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.594+0000 7fbf3b400640 1 -- 172.21.15.32:0/901773546 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3533017030 0 0) 0x7fbf3c0b56d0 con 0x7fbf3c15e7c0 2024-04-17T09:53:57.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.594+0000 7fbf3b400640 1 -- 172.21.15.32:0/901773546 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf20003530 con 0x7fbf3c15e7c0 2024-04-17T09:53:57.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.594+0000 7fbf40e00640 1 -- 172.21.15.32:0/901773546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf3c072970 msgr2=0x7fbf3c15eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:57.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.594+0000 7fbf40e00640 1 --2- 172.21.15.32:0/901773546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf3c072970 0x7fbf3c15eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.594+0000 7fbf40e00640 1 -- 172.21.15.32:0/901773546 >> v1:172.21.15.32:6789/0 conn(0x7fbf3c15e7c0 legacy=0x7fbf3c072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:57.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.594+0000 7fbf40e00640 1 -- 172.21.15.32:0/901773546 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3c0b3f70 con 0x7fbf3c0b4490 2024-04-17T09:53:57.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.598+0000 7fbf40e00640 1 --2- 172.21.15.32:0/901773546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf3c0b4490 0x7fbf3c0b5040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009d90 tx=0x7fbf3401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d33cc7b729ebafe server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:57.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.598+0000 7fbf3b400640 1 -- 172.21.15.32:0/901773546 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf340b2070 con 0x7fbf3c0b4490 2024-04-17T09:53:57.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.598+0000 7fbf3b400640 1 -- 172.21.15.32:0/901773546 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf340a59c0 con 0x7fbf3c0b4490 2024-04-17T09:53:57.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.598+0000 7fbf3b400640 1 -- 172.21.15.32:0/901773546 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf340a5d90 con 0x7fbf3c0b4490 2024-04-17T09:53:57.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.598+0000 7fbf42e00640 1 -- 172.21.15.32:0/901773546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf3c0b4490 msgr2=0x7fbf3c0b5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:57.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.598+0000 7fbf42e00640 1 --2- 172.21.15.32:0/901773546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf3c0b4490 0x7fbf3c0b5040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009d90 tx=0x7fbf3401c8e0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.598+0000 7fbf42e00640 1 -- 172.21.15.32:0/901773546 shutdown_connections 2024-04-17T09:53:57.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.598+0000 7fbf42e00640 1 --2- 172.21.15.32:0/901773546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf3c0b4490 0x7fbf3c0b5040 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.598+0000 7fbf42e00640 1 --2- 172.21.15.32:0/901773546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf3c072970 0x7fbf3c15eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.598+0000 7fbf42e00640 1 -- 172.21.15.32:0/901773546 >> 172.21.15.32:0/901773546 conn(0x7fbf3c0b2510 msgr2=0x7fbf3c0b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:57.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.598+0000 7fbf42e00640 1 -- 172.21.15.32:0/901773546 shutdown_connections 2024-04-17T09:53:57.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.598+0000 7fbf42e00640 1 -- 172.21.15.32:0/901773546 wait complete. 2024-04-17T09:53:57.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf42e00640 1 Processor -- start 2024-04-17T09:53:57.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf42e00640 1 -- start start 2024-04-17T09:53:57.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf42e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf3c072970 0x7fbf3c154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:57.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf42e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf3c0b4490 0x7fbf3c160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:57.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf42e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf3c15e7c0 0x7fbf3c160d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:57.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf42e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c06f1b0 con 0x7fbf3c072970 2024-04-17T09:53:57.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf42e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c06f030 con 0x7fbf3c0b4490 2024-04-17T09:53:57.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf42e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbf3c06f330 con 0x7fbf3c15e7c0 2024-04-17T09:53:57.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf3be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf3c072970 0x7fbf3c154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:57.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf3be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf3c072970 0x7fbf3c154f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44866/0 (socket says 172.21.15.32:44866) 2024-04-17T09:53:57.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf3be00640 1 -- 172.21.15.32:0/3088527704 learned_addr learned my addr 172.21.15.32:0/3088527704 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:57.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf33e00640 1 --2- 172.21.15.32:0/3088527704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf3c0b4490 0x7fbf3c160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:57.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf40e00640 1 --2- 172.21.15.32:0/3088527704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf3c15e7c0 0x7fbf3c160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:57.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf3be00640 1 -- 172.21.15.32:0/3088527704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf3c15e7c0 msgr2=0x7fbf3c160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:57.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf3be00640 1 --2- 172.21.15.32:0/3088527704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf3c15e7c0 0x7fbf3c160d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf3be00640 1 -- 172.21.15.32:0/3088527704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf3c0b4490 msgr2=0x7fbf3c160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:57.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf3be00640 1 --2- 172.21.15.32:0/3088527704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf3c0b4490 0x7fbf3c160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf3be00640 1 -- 172.21.15.32:0/3088527704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3c14f930 con 0x7fbf3c072970 2024-04-17T09:53:57.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf3be00640 1 --2- 172.21.15.32:0/3088527704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf3c072970 0x7fbf3c154f00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c00c450 tx=0x7fbf2c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:57.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf38c00640 1 -- 172.21.15.32:0/3088527704 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c017070 con 0x7fbf3c072970 2024-04-17T09:53:57.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf38c00640 1 -- 172.21.15.32:0/3088527704 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf2c005cf0 con 0x7fbf3c072970 2024-04-17T09:53:57.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf38c00640 1 -- 172.21.15.32:0/3088527704 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c007290 con 0x7fbf3c072970 2024-04-17T09:53:57.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf42e00640 1 -- 172.21.15.32:0/3088527704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf3c14fc10 con 0x7fbf3c072970 2024-04-17T09:53:57.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.602+0000 7fbf42e00640 1 -- 172.21.15.32:0/3088527704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf3c1504b0 con 0x7fbf3c072970 2024-04-17T09:53:57.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.606+0000 7fbf42e00640 1 -- 172.21.15.32:0/3088527704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf3c14fde0 con 0x7fbf3c072970 2024-04-17T09:53:57.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.606+0000 7fbf38c00640 1 -- 172.21.15.32:0/3088527704 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbf2c01a070 con 0x7fbf3c072970 2024-04-17T09:53:57.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.606+0000 7fbf38c00640 1 --2- 172.21.15.32:0/3088527704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbf18068380 0x7fbf1806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:57.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.606+0000 7fbf38c00640 1 -- 172.21.15.32:0/3088527704 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fbf2c085e90 con 0x7fbf3c072970 2024-04-17T09:53:57.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.606+0000 7fbf33e00640 1 --2- 172.21.15.32:0/3088527704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbf18068380 0x7fbf1806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:57.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.606+0000 7fbf33e00640 1 --2- 172.21.15.32:0/3088527704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbf18068380 0x7fbf1806a840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c160560 tx=0x7fbf24008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:57.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.610+0000 7fbf38c00640 1 -- 172.21.15.32:0/3088527704 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fbf2c056330 con 0x7fbf3c072970 2024-04-17T09:53:57.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.774+0000 7fbf42e00640 1 -- 172.21.15.32:0/3088527704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbf3c068a90 con 0x7fbf3c072970 2024-04-17T09:53:57.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.774+0000 7fbf38c00640 1 -- 172.21.15.32:0/3088527704 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7fbf2c0870a0 con 0x7fbf3c072970 2024-04-17T09:53:57.776 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:53:57.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.778+0000 7fbf42e00640 1 -- 172.21.15.32:0/3088527704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbf18068380 msgr2=0x7fbf1806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:57.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.778+0000 7fbf42e00640 1 --2- 172.21.15.32:0/3088527704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbf18068380 0x7fbf1806a840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c160560 tx=0x7fbf24008040 comp rx=0 tx=0).stop 2024-04-17T09:53:57.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.778+0000 7fbf42e00640 1 -- 172.21.15.32:0/3088527704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf3c072970 msgr2=0x7fbf3c154f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:57.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.778+0000 7fbf42e00640 1 --2- 172.21.15.32:0/3088527704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf3c072970 0x7fbf3c154f00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c00c450 tx=0x7fbf2c00c910 comp rx=0 tx=0).stop 2024-04-17T09:53:57.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.778+0000 7fbf42e00640 1 -- 172.21.15.32:0/3088527704 shutdown_connections 2024-04-17T09:53:57.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.778+0000 7fbf42e00640 1 --2- 172.21.15.32:0/3088527704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbf18068380 0x7fbf1806a840 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.778+0000 7fbf42e00640 1 --2- 172.21.15.32:0/3088527704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf3c15e7c0 0x7fbf3c160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.778+0000 7fbf42e00640 1 --2- 172.21.15.32:0/3088527704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf3c0b4490 0x7fbf3c160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.778+0000 7fbf42e00640 1 --2- 172.21.15.32:0/3088527704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf3c072970 0x7fbf3c154f00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.778+0000 7fbf42e00640 1 -- 172.21.15.32:0/3088527704 >> 172.21.15.32:0/3088527704 conn(0x7fbf3c0b2510 msgr2=0x7fbf3c0b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:57.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.778+0000 7fbf42e00640 1 -- 172.21.15.32:0/3088527704 shutdown_connections 2024-04-17T09:53:57.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.778+0000 7fbf42e00640 1 -- 172.21.15.32:0/3088527704 wait complete. 2024-04-17T09:53:57.797 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:53:57.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.938+0000 7f892ae00640 1 Processor -- start 2024-04-17T09:53:57.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.938+0000 7f892ae00640 1 -- start start 2024-04-17T09:53:57.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.942+0000 7f892ae00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f892406ba30 0x7f892406be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:57.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.942+0000 7f892ae00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f892406c350 con 0x7f89240cf370 2024-04-17T09:53:57.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.942+0000 7f892ae00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89240d0130 con 0x7f8924071e60 2024-04-17T09:53:57.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.942+0000 7f892ae00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f89240cfef0 con 0x7f892406ba30 2024-04-17T09:53:57.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.942+0000 7f891be00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f89240cf370 0x7f89240cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54194/0 (socket says 172.21.15.32:54194) 2024-04-17T09:53:57.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.942+0000 7f8923e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f892406ba30 0x7f892406be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:57.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.942+0000 7f891be00640 1 -- 172.21.15.32:0/2316815626 learned_addr learned my addr 172.21.15.32:0/2316815626 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:57.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.942+0000 7f8923e00640 1 -- 172.21.15.32:0/2316815626 >> v1:172.21.15.144:6789/0 conn(0x7f8924071e60 legacy=0x7f8924072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:57.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.942+0000 7f8923e00640 1 -- 172.21.15.32:0/2316815626 >> v1:172.21.15.32:6789/0 conn(0x7f89240cf370 legacy=0x7f89240cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:57.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.942+0000 7f8923e00640 1 -- 172.21.15.32:0/2316815626 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892415d090 con 0x7f892406ba30 2024-04-17T09:53:57.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.946+0000 7f8923e00640 1 --2- 172.21.15.32:0/2316815626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892406ba30 0x7f892406be10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f89140097c0 tx=0x7f891401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b480f8dc2fc883 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:57.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.946+0000 7f8923400640 1 -- 172.21.15.32:0/2316815626 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89140b1070 con 0x7f892406ba30 2024-04-17T09:53:57.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.946+0000 7f8923400640 1 -- 172.21.15.32:0/2316815626 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89140a8050 con 0x7f892406ba30 2024-04-17T09:53:57.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.946+0000 7f8923400640 1 -- 172.21.15.32:0/2316815626 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89140ac860 con 0x7f892406ba30 2024-04-17T09:53:57.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.946+0000 7f892ae00640 1 -- 172.21.15.32:0/2316815626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892406ba30 msgr2=0x7f892406be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:57.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.946+0000 7f892ae00640 1 --2- 172.21.15.32:0/2316815626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892406ba30 0x7f892406be10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f89140097c0 tx=0x7f891401c3e0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.946+0000 7f892ae00640 1 -- 172.21.15.32:0/2316815626 shutdown_connections 2024-04-17T09:53:57.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.946+0000 7f892ae00640 1 --2- 172.21.15.32:0/2316815626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892406ba30 0x7f892406be10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.946+0000 7f892ae00640 1 -- 172.21.15.32:0/2316815626 >> 172.21.15.32:0/2316815626 conn(0x7f89240b2120 msgr2=0x7f89240b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:57.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.946+0000 7f892ae00640 1 -- 172.21.15.32:0/2316815626 shutdown_connections 2024-04-17T09:53:57.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.946+0000 7f892ae00640 1 -- 172.21.15.32:0/2316815626 wait complete. 2024-04-17T09:53:57.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.946+0000 7f892ae00640 1 Processor -- start 2024-04-17T09:53:57.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.946+0000 7f892ae00640 1 -- start start 2024-04-17T09:53:57.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.950+0000 7f892ae00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f892406ba30 0x7f892415a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:57.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.950+0000 7f892ae00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8924071e60 0x7f892406e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:57.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.950+0000 7f892ae00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89240cf370 0x7f892406ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:57.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.950+0000 7f892ae00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f892415db70 con 0x7f8924071e60 2024-04-17T09:53:57.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.950+0000 7f892ae00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f892415cd00 con 0x7f892406ba30 2024-04-17T09:53:57.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.950+0000 7f892ae00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f892415dcf0 con 0x7f89240cf370 2024-04-17T09:53:57.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.950+0000 7f891be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8924071e60 0x7f892406e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:57.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.950+0000 7f8923e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f892406ba30 0x7f892415a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:57.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.950+0000 7f891be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8924071e60 0x7f892406e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44872/0 (socket says 172.21.15.32:44872) 2024-04-17T09:53:57.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.950+0000 7f8928e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89240cf370 0x7f892406ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:57.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.950+0000 7f891be00640 1 -- 172.21.15.32:0/3858868865 learned_addr learned my addr 172.21.15.32:0/3858868865 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:57.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.950+0000 7f891be00640 1 -- 172.21.15.32:0/3858868865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89240cf370 msgr2=0x7f892406ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:57.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.950+0000 7f891be00640 1 --2- 172.21.15.32:0/3858868865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89240cf370 0x7f892406ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.950+0000 7f891be00640 1 -- 172.21.15.32:0/3858868865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f892406ba30 msgr2=0x7f892415a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:57.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.950+0000 7f891be00640 1 --2- 172.21.15.32:0/3858868865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f892406ba30 0x7f892415a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:57.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.950+0000 7f891be00640 1 -- 172.21.15.32:0/3858868865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892413ce60 con 0x7f8924071e60 2024-04-17T09:53:57.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.950+0000 7f891be00640 1 --2- 172.21.15.32:0/3858868865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8924071e60 0x7f892406e910 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f890c00c430 tx=0x7f890c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:57.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.950+0000 7f8920c00640 1 -- 172.21.15.32:0/3858868865 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c017070 con 0x7f8924071e60 2024-04-17T09:53:57.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.950+0000 7f8920c00640 1 -- 172.21.15.32:0/3858868865 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f890c005cf0 con 0x7f8924071e60 2024-04-17T09:53:57.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.950+0000 7f892ae00640 1 -- 172.21.15.32:0/3858868865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f892413d140 con 0x7f8924071e60 2024-04-17T09:53:57.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.950+0000 7f8920c00640 1 -- 172.21.15.32:0/3858868865 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c0072d0 con 0x7f8924071e60 2024-04-17T09:53:57.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.950+0000 7f892ae00640 1 -- 172.21.15.32:0/3858868865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f892413d700 con 0x7f8924071e60 2024-04-17T09:53:57.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.954+0000 7f892ae00640 1 -- 172.21.15.32:0/3858868865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8924068a90 con 0x7f8924071e60 2024-04-17T09:53:57.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.954+0000 7f8920c00640 1 -- 172.21.15.32:0/3858868865 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f890c01a070 con 0x7f8924071e60 2024-04-17T09:53:57.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.954+0000 7f8920c00640 1 --2- 172.21.15.32:0/3858868865 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8900068380 0x7f890006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:57.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.954+0000 7f8920c00640 1 -- 172.21.15.32:0/3858868865 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f890c085d10 con 0x7f8924071e60 2024-04-17T09:53:57.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.954+0000 7f8923e00640 1 --2- 172.21.15.32:0/3858868865 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8900068380 0x7f890006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:57.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.954+0000 7f8923e00640 1 --2- 172.21.15.32:0/3858868865 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8900068380 0x7f890006a840 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f89140ae000 tx=0x7f8914008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:57.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:57.958+0000 7f8920c00640 1 -- 172.21.15.32:0/3858868865 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f890c08b020 con 0x7f8924071e60 2024-04-17T09:53:58.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.126+0000 7f892ae00640 1 -- 172.21.15.32:0/3858868865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f892406c850 con 0x7f8924071e60 2024-04-17T09:53:58.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.126+0000 7f8920c00640 1 -- 172.21.15.32:0/3858868865 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f890c055b60 con 0x7f8924071e60 2024-04-17T09:53:58.127 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:53:58.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.130+0000 7f892ae00640 1 -- 172.21.15.32:0/3858868865 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8900068380 msgr2=0x7f890006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:58.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.130+0000 7f892ae00640 1 --2- 172.21.15.32:0/3858868865 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8900068380 0x7f890006a840 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f89140ae000 tx=0x7f8914008040 comp rx=0 tx=0).stop 2024-04-17T09:53:58.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.130+0000 7f892ae00640 1 -- 172.21.15.32:0/3858868865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8924071e60 msgr2=0x7f892406e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:58.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.130+0000 7f892ae00640 1 --2- 172.21.15.32:0/3858868865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8924071e60 0x7f892406e910 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f890c00c430 tx=0x7f890c00c8f0 comp rx=0 tx=0).stop 2024-04-17T09:53:58.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.130+0000 7f892ae00640 1 -- 172.21.15.32:0/3858868865 shutdown_connections 2024-04-17T09:53:58.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.130+0000 7f892ae00640 1 --2- 172.21.15.32:0/3858868865 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8900068380 0x7f890006a840 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:58.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.130+0000 7f892ae00640 1 --2- 172.21.15.32:0/3858868865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89240cf370 0x7f892406ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:58.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.130+0000 7f892ae00640 1 --2- 172.21.15.32:0/3858868865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8924071e60 0x7f892406e910 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:58.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.130+0000 7f892ae00640 1 --2- 172.21.15.32:0/3858868865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f892406ba30 0x7f892415a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:58.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.130+0000 7f892ae00640 1 -- 172.21.15.32:0/3858868865 >> 172.21.15.32:0/3858868865 conn(0x7f89240b2120 msgr2=0x7f892415c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:58.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.130+0000 7f892ae00640 1 -- 172.21.15.32:0/3858868865 shutdown_connections 2024-04-17T09:53:58.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.130+0000 7f892ae00640 1 -- 172.21.15.32:0/3858868865 wait complete. 2024-04-17T09:53:58.148 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T09:53:58.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.290+0000 7f483d400640 1 Processor -- start 2024-04-17T09:53:58.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.290+0000 7f483d400640 1 -- start start 2024-04-17T09:53:58.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.290+0000 7f483d400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4838072490 0x7f483806e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:58.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.290+0000 7f483d400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f483815b420 0x7f48380d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:58.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.290+0000 7f483d400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48380b40f0 con 0x7f483815bd90 2024-04-17T09:53:58.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.290+0000 7f483d400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f48380b43d0 con 0x7f483815b420 2024-04-17T09:53:58.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.290+0000 7f483d400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f48380b46b0 con 0x7f4838072490 2024-04-17T09:53:58.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.290+0000 7f4837200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f483815b420 0x7f48380d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:58.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.290+0000 7f4837200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f483815b420 0x7f48380d0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44876/0 (socket says 172.21.15.32:44876) 2024-04-17T09:53:58.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.290+0000 7f4837200640 1 -- 172.21.15.32:0/2019205486 learned_addr learned my addr 172.21.15.32:0/2019205486 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:58.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.290+0000 7f482fe00640 1 --2- 172.21.15.32:0/2019205486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4838072490 0x7f483806e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:58.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.290+0000 7f4837200640 1 -- 172.21.15.32:0/2019205486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4838072490 msgr2=0x7f483806e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:58.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.290+0000 7f4837200640 1 --2- 172.21.15.32:0/2019205486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4838072490 0x7f483806e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:58.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.290+0000 7f4837200640 1 -- 172.21.15.32:0/2019205486 >> v1:172.21.15.144:6789/0 conn(0x7f483815bd90 legacy=0x7f4838071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:58.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.290+0000 7f4837200640 1 -- 172.21.15.32:0/2019205486 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48380b4990 con 0x7f483815b420 2024-04-17T09:53:58.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.290+0000 7f482fe00640 1 --2- 172.21.15.32:0/2019205486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4838072490 0x7f483806e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:58.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.294+0000 7f4837200640 1 --2- 172.21.15.32:0/2019205486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f483815b420 0x7f48380d0350 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4830009e50 tx=0x7f483001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367733abbde6aee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:58.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.294+0000 7f4835e00640 1 -- 172.21.15.32:0/2019205486 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48300a57e0 con 0x7f483815b420 2024-04-17T09:53:58.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.294+0000 7f4835e00640 1 -- 172.21.15.32:0/2019205486 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48300a5980 con 0x7f483815b420 2024-04-17T09:53:58.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.294+0000 7f4835e00640 1 -- 172.21.15.32:0/2019205486 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48300a5d50 con 0x7f483815b420 2024-04-17T09:53:58.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.294+0000 7f483d400640 1 -- 172.21.15.32:0/2019205486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f483815b420 msgr2=0x7f48380d0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:58.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.294+0000 7f483d400640 1 --2- 172.21.15.32:0/2019205486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f483815b420 0x7f48380d0350 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4830009e50 tx=0x7f483001ca00 comp rx=0 tx=0).stop 2024-04-17T09:53:58.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.294+0000 7f483d400640 1 -- 172.21.15.32:0/2019205486 shutdown_connections 2024-04-17T09:53:58.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.294+0000 7f483d400640 1 --2- 172.21.15.32:0/2019205486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f483815b420 0x7f48380d0350 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:58.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.294+0000 7f483d400640 1 --2- 172.21.15.32:0/2019205486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4838072490 0x7f483806e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:58.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.294+0000 7f483d400640 1 -- 172.21.15.32:0/2019205486 >> 172.21.15.32:0/2019205486 conn(0x7f483806ba30 msgr2=0x7f483806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:58.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.294+0000 7f483d400640 1 -- 172.21.15.32:0/2019205486 shutdown_connections 2024-04-17T09:53:58.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.294+0000 7f483d400640 1 -- 172.21.15.32:0/2019205486 wait complete. 2024-04-17T09:53:58.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f483d400640 1 Processor -- start 2024-04-17T09:53:58.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f483d400640 1 -- start start 2024-04-17T09:53:58.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f483d400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4838072490 0x7f4838144e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:58.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f483d400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f483815b420 0x7f48381f7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:58.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f483d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f483815bd90 0x7f48380b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:58.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f483d400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f483815c2f0 con 0x7f483815bd90 2024-04-17T09:53:58.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f483d400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f483815c170 con 0x7f4838072490 2024-04-17T09:53:58.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f483d400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f483806edf0 con 0x7f483815b420 2024-04-17T09:53:58.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f4837200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f483815bd90 0x7f48380b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:58.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f4836800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4838072490 0x7f4838144e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:58.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f4837200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f483815bd90 0x7f48380b2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44890/0 (socket says 172.21.15.32:44890) 2024-04-17T09:53:58.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f4837200640 1 -- 172.21.15.32:0/3350472154 learned_addr learned my addr 172.21.15.32:0/3350472154 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:58.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f482fe00640 1 --2- 172.21.15.32:0/3350472154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f483815b420 0x7f48381f7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:58.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f4837200640 1 -- 172.21.15.32:0/3350472154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f483815b420 msgr2=0x7f48381f7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:58.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f4837200640 1 --2- 172.21.15.32:0/3350472154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f483815b420 0x7f48381f7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:58.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f4837200640 1 -- 172.21.15.32:0/3350472154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4838072490 msgr2=0x7f4838144e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:58.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f4837200640 1 --2- 172.21.15.32:0/3350472154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4838072490 0x7f4838144e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:58.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f4837200640 1 -- 172.21.15.32:0/3350472154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48380d0aa0 con 0x7f483815bd90 2024-04-17T09:53:58.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f4837200640 1 --2- 172.21.15.32:0/3350472154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f483815bd90 0x7f48380b2c70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f483001a630 tx=0x7f48300197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:58.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f482ea00640 1 -- 172.21.15.32:0/3350472154 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48300b8670 con 0x7f483815bd90 2024-04-17T09:53:58.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f482ea00640 1 -- 172.21.15.32:0/3350472154 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48300be070 con 0x7f483815bd90 2024-04-17T09:53:58.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f482ea00640 1 -- 172.21.15.32:0/3350472154 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48300b8a70 con 0x7f483815bd90 2024-04-17T09:53:58.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f483d400640 1 -- 172.21.15.32:0/3350472154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48381f85e0 con 0x7f483815bd90 2024-04-17T09:53:58.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f483d400640 1 -- 172.21.15.32:0/3350472154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f483815c850 con 0x7f483815bd90 2024-04-17T09:53:58.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.298+0000 7f483d400640 1 -- 172.21.15.32:0/3350472154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4838068a90 con 0x7f483815bd90 2024-04-17T09:53:58.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.302+0000 7f482ea00640 1 -- 172.21.15.32:0/3350472154 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f48300b8c10 con 0x7f483815bd90 2024-04-17T09:53:58.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.302+0000 7f482ea00640 1 --2- 172.21.15.32:0/3350472154 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f48140683b0 0x7f481406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:58.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.302+0000 7f482ea00640 1 -- 172.21.15.32:0/3350472154 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f4830121b00 con 0x7f483815bd90 2024-04-17T09:53:58.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.302+0000 7f4836800640 1 --2- 172.21.15.32:0/3350472154 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f48140683b0 0x7f481406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:58.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.302+0000 7f4836800640 1 --2- 172.21.15.32:0/3350472154 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f48140683b0 0x7f481406a870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f48380cc740 tx=0x7f4828009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:58.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.306+0000 7f482ea00640 1 -- 172.21.15.32:0/3350472154 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f4830016620 con 0x7f483815bd90 2024-04-17T09:53:58.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.482+0000 7f483d400640 1 -- 172.21.15.32:0/3350472154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f48380b19b0 con 0x7f483815bd90 2024-04-17T09:53:58.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.486+0000 7f482ea00640 1 -- 172.21.15.32:0/3350472154 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f48300f1a10 con 0x7f483815bd90 2024-04-17T09:53:58.485 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T09:53:58.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.486+0000 7f483d400640 1 -- 172.21.15.32:0/3350472154 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f48140683b0 msgr2=0x7f481406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:58.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.486+0000 7f483d400640 1 --2- 172.21.15.32:0/3350472154 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f48140683b0 0x7f481406a870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f48380cc740 tx=0x7f4828009040 comp rx=0 tx=0).stop 2024-04-17T09:53:58.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.486+0000 7f483d400640 1 -- 172.21.15.32:0/3350472154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f483815bd90 msgr2=0x7f48380b2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:58.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.486+0000 7f483d400640 1 --2- 172.21.15.32:0/3350472154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f483815bd90 0x7f48380b2c70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f483001a630 tx=0x7f48300197a0 comp rx=0 tx=0).stop 2024-04-17T09:53:58.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.486+0000 7f483d400640 1 -- 172.21.15.32:0/3350472154 shutdown_connections 2024-04-17T09:53:58.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.486+0000 7f483d400640 1 --2- 172.21.15.32:0/3350472154 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f48140683b0 0x7f481406a870 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:58.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.486+0000 7f483d400640 1 --2- 172.21.15.32:0/3350472154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f483815b420 0x7f48381f7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:58.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.486+0000 7f483d400640 1 --2- 172.21.15.32:0/3350472154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f483815bd90 0x7f48380b2c70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:58.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.486+0000 7f483d400640 1 --2- 172.21.15.32:0/3350472154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4838072490 0x7f4838144e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:58.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.486+0000 7f483d400640 1 -- 172.21.15.32:0/3350472154 >> 172.21.15.32:0/3350472154 conn(0x7f483806ba30 msgr2=0x7f4838154f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:58.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.486+0000 7f483d400640 1 -- 172.21.15.32:0/3350472154 shutdown_connections 2024-04-17T09:53:58.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.486+0000 7f483d400640 1 -- 172.21.15.32:0/3350472154 wait complete. 2024-04-17T09:53:58.506 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2024-04-17T09:53:58.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:58.507 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-17T09:53:58.528 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:53:58.528 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T09:53:58.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:53:58.528 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-17T09:53:58.581 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:53:58.581 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:53:58.591 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:53:58.591 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:53:58.644 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:53:58.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.782+0000 7fd937800640 1 Processor -- start 2024-04-17T09:53:58.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.782+0000 7fd937800640 1 -- start start 2024-04-17T09:53:58.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.786+0000 7fd937800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd930145f30 0x7fd930071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:58.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.786+0000 7fd937800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9300cdac0 con 0x7fd93015d8d0 2024-04-17T09:53:58.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.786+0000 7fd937800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd93015cdf0 con 0x7fd9300cd580 2024-04-17T09:53:58.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.786+0000 7fd937800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd93006f400 con 0x7fd930145f30 2024-04-17T09:53:58.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.786+0000 7fd927e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd93015d8d0 0x7fd9300cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54208/0 (socket says 172.21.15.32:54208) 2024-04-17T09:53:58.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.786+0000 7fd927e00640 1 -- 172.21.15.32:0/3510068314 learned_addr learned my addr 172.21.15.32:0/3510068314 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:58.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.786+0000 7fd934e00640 1 --2- 172.21.15.32:0/3510068314 >> v2:172.21.15.144:3300/0 conn(0x7fd930145f30 0x7fd930071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:58.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.786+0000 7fd92fe00640 1 -- 172.21.15.32:0/3510068314 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 504855438 0 0) 0x7fd9300cdac0 con 0x7fd93015d8d0 2024-04-17T09:53:58.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.786+0000 7fd92fe00640 1 -- 172.21.15.32:0/3510068314 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd914003530 con 0x7fd93015d8d0 2024-04-17T09:53:58.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.786+0000 7fd92fe00640 1 -- 172.21.15.32:0/3510068314 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3758497091 0 0) 0x7fd93015cdf0 con 0x7fd9300cd580 2024-04-17T09:53:58.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.786+0000 7fd92fe00640 1 -- 172.21.15.32:0/3510068314 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9300cdac0 con 0x7fd9300cd580 2024-04-17T09:53:58.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.786+0000 7fd934e00640 1 -- 172.21.15.32:0/3510068314 >> v1:172.21.15.144:6789/0 conn(0x7fd9300cd580 legacy=0x7fd9301601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:58.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.786+0000 7fd934e00640 1 -- 172.21.15.32:0/3510068314 >> v1:172.21.15.32:6789/0 conn(0x7fd93015d8d0 legacy=0x7fd9300cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:58.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.786+0000 7fd934e00640 1 -- 172.21.15.32:0/3510068314 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93015d0d0 con 0x7fd930145f30 2024-04-17T09:53:58.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.790+0000 7fd934e00640 1 --2- 172.21.15.32:0/3510068314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd930145f30 0x7fd930071df0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd9200097c0 tx=0x7fd92001ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a6e428d6feef859 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:58.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.790+0000 7fd92fe00640 1 -- 172.21.15.32:0/3510068314 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9200b5070 con 0x7fd930145f30 2024-04-17T09:53:58.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.790+0000 7fd92fe00640 1 -- 172.21.15.32:0/3510068314 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9200a8990 con 0x7fd930145f30 2024-04-17T09:53:58.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.790+0000 7fd92fe00640 1 -- 172.21.15.32:0/3510068314 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9200a8d40 con 0x7fd930145f30 2024-04-17T09:53:58.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.790+0000 7fd937800640 1 -- 172.21.15.32:0/3510068314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd930145f30 msgr2=0x7fd930071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:58.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.790+0000 7fd937800640 1 --2- 172.21.15.32:0/3510068314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd930145f30 0x7fd930071df0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd9200097c0 tx=0x7fd92001ec90 comp rx=0 tx=0).stop 2024-04-17T09:53:58.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.790+0000 7fd937800640 1 -- 172.21.15.32:0/3510068314 shutdown_connections 2024-04-17T09:53:58.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.790+0000 7fd937800640 1 --2- 172.21.15.32:0/3510068314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd930145f30 0x7fd930071df0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:58.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.790+0000 7fd937800640 1 -- 172.21.15.32:0/3510068314 >> 172.21.15.32:0/3510068314 conn(0x7fd93006ba30 msgr2=0x7fd93006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:58.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.790+0000 7fd937800640 1 -- 172.21.15.32:0/3510068314 shutdown_connections 2024-04-17T09:53:58.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.790+0000 7fd937800640 1 -- 172.21.15.32:0/3510068314 wait complete. 2024-04-17T09:53:58.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.790+0000 7fd937800640 1 Processor -- start 2024-04-17T09:53:58.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.790+0000 7fd937800640 1 -- start start 2024-04-17T09:53:58.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.790+0000 7fd937800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9300cd580 0x7fd93013e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:58.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.790+0000 7fd937800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd930145f30 0x7fd9301e80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:58.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.790+0000 7fd937800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd93015d8d0 0x7fd9301e8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:58.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.790+0000 7fd937800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd930146310 con 0x7fd930145f30 2024-04-17T09:53:58.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.790+0000 7fd937800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd930072600 con 0x7fd93015d8d0 2024-04-17T09:53:58.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.790+0000 7fd937800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd930072780 con 0x7fd9300cd580 2024-04-17T09:53:58.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.794+0000 7fd927e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd930145f30 0x7fd9301e80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:58.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.794+0000 7fd927e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd930145f30 0x7fd9301e80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44894/0 (socket says 172.21.15.32:44894) 2024-04-17T09:53:58.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.794+0000 7fd927e00640 1 -- 172.21.15.32:0/2227605855 learned_addr learned my addr 172.21.15.32:0/2227605855 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:58.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.794+0000 7fd934e00640 1 --2- 172.21.15.32:0/2227605855 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9300cd580 0x7fd93013e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:58.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.794+0000 7fd935800640 1 --2- 172.21.15.32:0/2227605855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd93015d8d0 0x7fd9301e8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:58.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.794+0000 7fd927e00640 1 -- 172.21.15.32:0/2227605855 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9300cd580 msgr2=0x7fd93013e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:58.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.794+0000 7fd927e00640 1 --2- 172.21.15.32:0/2227605855 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9300cd580 0x7fd93013e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:58.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.794+0000 7fd927e00640 1 -- 172.21.15.32:0/2227605855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd93015d8d0 msgr2=0x7fd9301e8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:58.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.794+0000 7fd927e00640 1 --2- 172.21.15.32:0/2227605855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd93015d8d0 0x7fd9301e8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:58.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.794+0000 7fd927e00640 1 -- 172.21.15.32:0/2227605855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd930142a70 con 0x7fd930145f30 2024-04-17T09:53:58.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.794+0000 7fd927e00640 1 --2- 172.21.15.32:0/2227605855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd930145f30 0x7fd9301e80f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd91800c420 tx=0x7fd91800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:58.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.794+0000 7fd92d600640 1 -- 172.21.15.32:0/2227605855 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd918017070 con 0x7fd930145f30 2024-04-17T09:53:58.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.794+0000 7fd92d600640 1 -- 172.21.15.32:0/2227605855 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd918005cf0 con 0x7fd930145f30 2024-04-17T09:53:58.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.794+0000 7fd937800640 1 -- 172.21.15.32:0/2227605855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd930142d50 con 0x7fd930145f30 2024-04-17T09:53:58.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.794+0000 7fd92d600640 1 -- 172.21.15.32:0/2227605855 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9180072d0 con 0x7fd930145f30 2024-04-17T09:53:58.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.794+0000 7fd937800640 1 -- 172.21.15.32:0/2227605855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd930143310 con 0x7fd930145f30 2024-04-17T09:53:58.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.794+0000 7fd937800640 1 -- 172.21.15.32:0/2227605855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd930068a90 con 0x7fd930145f30 2024-04-17T09:53:58.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.794+0000 7fd92d600640 1 -- 172.21.15.32:0/2227605855 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd91801a070 con 0x7fd930145f30 2024-04-17T09:53:58.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.798+0000 7fd92d600640 1 --2- 172.21.15.32:0/2227605855 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd90c068380 0x7fd90c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:58.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.798+0000 7fd92d600640 1 -- 172.21.15.32:0/2227605855 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fd918085d70 con 0x7fd930145f30 2024-04-17T09:53:58.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.798+0000 7fd934e00640 1 --2- 172.21.15.32:0/2227605855 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd90c068380 0x7fd90c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:58.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.798+0000 7fd934e00640 1 --2- 172.21.15.32:0/2227605855 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd90c068380 0x7fd90c06a840 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd9200b2000 tx=0x7fd9200aa070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:58.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.802+0000 7fd92d600640 1 -- 172.21.15.32:0/2227605855 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd918055c60 con 0x7fd930145f30 2024-04-17T09:53:58.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.970+0000 7fd937800640 1 -- 172.21.15.32:0/2227605855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd930072f90 con 0x7fd930145f30 2024-04-17T09:53:58.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.970+0000 7fd92d600640 1 -- 172.21.15.32:0/2227605855 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16866 (secure 0 0 0) 0x7fd918059f60 con 0x7fd930145f30 2024-04-17T09:53:58.971 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:53:58.971 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:54.932571+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:53:58.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.970+0000 7fd937800640 1 -- 172.21.15.32:0/2227605855 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd90c068380 msgr2=0x7fd90c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:58.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.970+0000 7fd937800640 1 --2- 172.21.15.32:0/2227605855 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd90c068380 0x7fd90c06a840 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd9200b2000 tx=0x7fd9200aa070 comp rx=0 tx=0).stop 2024-04-17T09:53:58.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.970+0000 7fd937800640 1 -- 172.21.15.32:0/2227605855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd930145f30 msgr2=0x7fd9301e80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:58.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.970+0000 7fd937800640 1 --2- 172.21.15.32:0/2227605855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd930145f30 0x7fd9301e80f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd91800c420 tx=0x7fd91800c8e0 comp rx=0 tx=0).stop 2024-04-17T09:53:58.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.970+0000 7fd937800640 1 -- 172.21.15.32:0/2227605855 shutdown_connections 2024-04-17T09:53:58.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.970+0000 7fd937800640 1 --2- 172.21.15.32:0/2227605855 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd90c068380 0x7fd90c06a840 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:58.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.970+0000 7fd937800640 1 --2- 172.21.15.32:0/2227605855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd93015d8d0 0x7fd9301e8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:58.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.970+0000 7fd937800640 1 --2- 172.21.15.32:0/2227605855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd930145f30 0x7fd9301e80f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:58.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.970+0000 7fd937800640 1 --2- 172.21.15.32:0/2227605855 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9300cd580 0x7fd93013e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:58.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.970+0000 7fd937800640 1 -- 172.21.15.32:0/2227605855 >> 172.21.15.32:0/2227605855 conn(0x7fd93006ba30 msgr2=0x7fd9300b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:58.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.974+0000 7fd937800640 1 -- 172.21.15.32:0/2227605855 shutdown_connections 2024-04-17T09:53:58.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:58.974+0000 7fd937800640 1 -- 172.21.15.32:0/2227605855 wait complete. 2024-04-17T09:53:58.992 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:53:11.198098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:53:12.118982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:53:58.992 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:53:59.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.134+0000 7f61d1a00640 1 Processor -- start 2024-04-17T09:53:59.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.134+0000 7f61d1a00640 1 -- start start 2024-04-17T09:53:59.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61d1a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61cc072970 0x7f61cc15e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:59.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61d1a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61cc0cd890 con 0x7f61cc15c780 2024-04-17T09:53:59.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61d1a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61cc0cdb70 con 0x7f61cc15cb60 2024-04-17T09:53:59.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61d1a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f61cc070ba0 con 0x7f61cc072970 2024-04-17T09:53:59.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61caa00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f61cc15c780 0x7f61cc070490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54216/0 (socket says 172.21.15.32:54216) 2024-04-17T09:53:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61caa00640 1 -- 172.21.15.32:0/1008675328 learned_addr learned my addr 172.21.15.32:0/1008675328 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:53:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61cb400640 1 --2- 172.21.15.32:0/1008675328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61cc072970 0x7f61cc15e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61ca000640 1 -- 172.21.15.32:0/1008675328 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2386815310 0 0) 0x7f61cc0cd890 con 0x7f61cc15c780 2024-04-17T09:53:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61ca000640 1 -- 172.21.15.32:0/1008675328 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61b0003530 con 0x7f61cc15c780 2024-04-17T09:53:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61ca000640 1 -- 172.21.15.32:0/1008675328 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4172481684 0 0) 0x7f61cc0cdb70 con 0x7f61cc15cb60 2024-04-17T09:53:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61ca000640 1 -- 172.21.15.32:0/1008675328 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61cc0cd890 con 0x7f61cc15cb60 2024-04-17T09:53:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61ca000640 1 -- 172.21.15.32:0/1008675328 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4231576548 0 0) 0x7f61b0003530 con 0x7f61cc15c780 2024-04-17T09:53:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61ca000640 1 -- 172.21.15.32:0/1008675328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61cc072970 msgr2=0x7f61cc15e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61ca000640 1 --2- 172.21.15.32:0/1008675328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61cc072970 0x7f61cc15e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61ca000640 1 -- 172.21.15.32:0/1008675328 >> v1:172.21.15.144:6789/0 conn(0x7f61cc15cb60 legacy=0x7f61cc072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61ca000640 1 -- 172.21.15.32:0/1008675328 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61cc0cde50 con 0x7f61cc15c780 2024-04-17T09:53:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61ca000640 1 -- 172.21.15.32:0/1008675328 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f61bc001680 con 0x7f61cc15c780 2024-04-17T09:53:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61ca000640 1 -- 172.21.15.32:0/1008675328 >> v1:172.21.15.32:6789/0 conn(0x7f61cc15c780 legacy=0x7f61cc070490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61ca000640 1 --2- 172.21.15.32:0/1008675328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61b0003b80 0x7f61b0003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:59.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61ca000640 1 --2- 172.21.15.32:0/1008675328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61b0004810 0x7f61b0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:59.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61ca000640 1 --2- 172.21.15.32:0/1008675328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61cc072970 0x7f61b00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:59.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61ca000640 1 -- 172.21.15.32:0/1008675328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f61b000ba80 con 0x7f61b0004810 2024-04-17T09:53:59.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61ca000640 1 -- 172.21.15.32:0/1008675328 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f61b000bc30 con 0x7f61b0003b80 2024-04-17T09:53:59.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61ca000640 1 -- 172.21.15.32:0/1008675328 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f61b000be30 con 0x7f61cc072970 2024-04-17T09:53:59.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61c2000640 1 --2- 172.21.15.32:0/1008675328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61b0004810 0x7f61b0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:59.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61caa00640 1 --2- 172.21.15.32:0/1008675328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61b0003b80 0x7f61b0003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:59.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61cb400640 1 --2- 172.21.15.32:0/1008675328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61cc072970 0x7f61b00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:59.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61c2000640 1 -- 172.21.15.32:0/1008675328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61cc072970 msgr2=0x7f61b00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:59.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61c2000640 1 --2- 172.21.15.32:0/1008675328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61cc072970 0x7f61b00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61c2000640 1 -- 172.21.15.32:0/1008675328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61b0003b80 msgr2=0x7f61b0003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:59.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61c2000640 1 --2- 172.21.15.32:0/1008675328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61b0003b80 0x7f61b0003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.138+0000 7f61c2000640 1 -- 172.21.15.32:0/1008675328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b0003530 con 0x7f61b0004810 2024-04-17T09:53:59.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.142+0000 7f61c2000640 1 --2- 172.21.15.32:0/1008675328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61b0004810 0x7f61b0004c90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f61b40018d0 tx=0x7f61b401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd81992bd8db3e6b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:59.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.142+0000 7f61ca000640 1 -- 172.21.15.32:0/1008675328 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4001520 con 0x7f61b0004810 2024-04-17T09:53:59.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.142+0000 7f61ca000640 1 -- 172.21.15.32:0/1008675328 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b40052f0 con 0x7f61b0004810 2024-04-17T09:53:59.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.142+0000 7f61ca000640 1 -- 172.21.15.32:0/1008675328 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b40a8950 con 0x7f61b0004810 2024-04-17T09:53:59.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.142+0000 7f61d1a00640 1 -- 172.21.15.32:0/1008675328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61b0004810 msgr2=0x7f61b0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:59.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.142+0000 7f61d1a00640 1 --2- 172.21.15.32:0/1008675328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61b0004810 0x7f61b0004c90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f61b40018d0 tx=0x7f61b401ecf0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.142+0000 7f61cb400640 1 -- 172.21.15.32:0/1008675328 reap_dead start 2024-04-17T09:53:59.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.142+0000 7f61d1a00640 1 -- 172.21.15.32:0/1008675328 shutdown_connections 2024-04-17T09:53:59.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.142+0000 7f61d1a00640 1 -- 172.21.15.32:0/1008675328 >> 172.21.15.32:0/1008675328 conn(0x7f61cc0b2510 msgr2=0x7f61cc0b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:59.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.142+0000 7f61d1a00640 1 -- 172.21.15.32:0/1008675328 shutdown_connections 2024-04-17T09:53:59.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.142+0000 7f61d1a00640 1 -- 172.21.15.32:0/1008675328 wait complete. 2024-04-17T09:53:59.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.142+0000 7f61d1a00640 1 Processor -- start 2024-04-17T09:53:59.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61d1a00640 1 -- start start 2024-04-17T09:53:59.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61d1a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61b0004810 0x7f61cc15a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:59.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61d1a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61cc0ad860 0x7f61cc06e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:59.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61d1a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61cc15ec90 0x7f61cc14cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:59.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61d1a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc0cd630 con 0x7f61cc15ec90 2024-04-17T09:53:59.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61d1a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc0cd4b0 con 0x7f61cc0ad860 2024-04-17T09:53:59.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61d1a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f61cc15e180 con 0x7f61b0004810 2024-04-17T09:53:59.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61cb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61cc15ec90 0x7f61cc14cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:59.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61cb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61cc15ec90 0x7f61cc14cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44910/0 (socket says 172.21.15.32:44910) 2024-04-17T09:53:59.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61cb400640 1 -- 172.21.15.32:0/4159256762 learned_addr learned my addr 172.21.15.32:0/4159256762 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:59.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61c2000640 1 --2- 172.21.15.32:0/4159256762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61cc0ad860 0x7f61cc06e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:59.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61caa00640 1 --2- 172.21.15.32:0/4159256762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61b0004810 0x7f61cc15a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:59.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61cb400640 1 -- 172.21.15.32:0/4159256762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61b0004810 msgr2=0x7f61cc15a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:59.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61cb400640 1 --2- 172.21.15.32:0/4159256762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61b0004810 0x7f61cc15a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61cb400640 1 -- 172.21.15.32:0/4159256762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61cc0ad860 msgr2=0x7f61cc06e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:59.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61cb400640 1 --2- 172.21.15.32:0/4159256762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61cc0ad860 0x7f61cc06e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61cb400640 1 -- 172.21.15.32:0/4159256762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61cc070670 con 0x7f61cc15ec90 2024-04-17T09:53:59.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61cb400640 1 --2- 172.21.15.32:0/4159256762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61cc15ec90 0x7f61cc14cee0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f61cc072210 tx=0x7f61c40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:59.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61c3400640 1 -- 172.21.15.32:0/4159256762 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c40039a0 con 0x7f61cc15ec90 2024-04-17T09:53:59.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61c3400640 1 -- 172.21.15.32:0/4159256762 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61c401b070 con 0x7f61cc15ec90 2024-04-17T09:53:59.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61d1a00640 1 -- 172.21.15.32:0/4159256762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61cc06ec00 con 0x7f61cc15ec90 2024-04-17T09:53:59.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61c3400640 1 -- 172.21.15.32:0/4159256762 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c4003b80 con 0x7f61cc15ec90 2024-04-17T09:53:59.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61d1a00640 1 -- 172.21.15.32:0/4159256762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61cc06f1c0 con 0x7f61cc15ec90 2024-04-17T09:53:59.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.146+0000 7f61d1a00640 1 -- 172.21.15.32:0/4159256762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61cc068a90 con 0x7f61cc15ec90 2024-04-17T09:53:59.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.150+0000 7f61c3400640 1 -- 172.21.15.32:0/4159256762 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f61c4004030 con 0x7f61cc15ec90 2024-04-17T09:53:59.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.150+0000 7f61c3400640 1 --2- 172.21.15.32:0/4159256762 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f61a80683b0 0x7f61a806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:59.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.150+0000 7f61c3400640 1 -- 172.21.15.32:0/4159256762 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f61c4086fe0 con 0x7f61cc15ec90 2024-04-17T09:53:59.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.150+0000 7f61caa00640 1 --2- 172.21.15.32:0/4159256762 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f61a80683b0 0x7f61a806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:59.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.150+0000 7f61caa00640 1 --2- 172.21.15.32:0/4159256762 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f61a80683b0 0x7f61a806a870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f61bc001060 tx=0x7f61bc001d80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:59.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.154+0000 7f61c3400640 1 -- 172.21.15.32:0/4159256762 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f61c4056e90 con 0x7f61cc15ec90 2024-04-17T09:53:59.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.318+0000 7f61d1a00640 1 -- 172.21.15.32:0/4159256762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f61cc15bea0 con 0x7f61cc15ec90 2024-04-17T09:53:59.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.322+0000 7f61c3400640 1 -- 172.21.15.32:0/4159256762 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f61c405b190 con 0x7f61cc15ec90 2024-04-17T09:53:59.321 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:53:59.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.322+0000 7f61d1a00640 1 -- 172.21.15.32:0/4159256762 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f61a80683b0 msgr2=0x7f61a806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:59.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.322+0000 7f61d1a00640 1 --2- 172.21.15.32:0/4159256762 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f61a80683b0 0x7f61a806a870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f61bc001060 tx=0x7f61bc001d80 comp rx=0 tx=0).stop 2024-04-17T09:53:59.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.322+0000 7f61d1a00640 1 -- 172.21.15.32:0/4159256762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61cc15ec90 msgr2=0x7f61cc14cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:59.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.322+0000 7f61d1a00640 1 --2- 172.21.15.32:0/4159256762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61cc15ec90 0x7f61cc14cee0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f61cc072210 tx=0x7f61c40115a0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.322+0000 7f61d1a00640 1 -- 172.21.15.32:0/4159256762 shutdown_connections 2024-04-17T09:53:59.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.322+0000 7f61d1a00640 1 --2- 172.21.15.32:0/4159256762 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f61a80683b0 0x7f61a806a870 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.322+0000 7f61d1a00640 1 --2- 172.21.15.32:0/4159256762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61cc15ec90 0x7f61cc14cee0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.322+0000 7f61d1a00640 1 --2- 172.21.15.32:0/4159256762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61cc0ad860 0x7f61cc06e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.322+0000 7f61d1a00640 1 --2- 172.21.15.32:0/4159256762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61b0004810 0x7f61cc15a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.322+0000 7f61d1a00640 1 -- 172.21.15.32:0/4159256762 >> 172.21.15.32:0/4159256762 conn(0x7f61cc0b2510 msgr2=0x7f61cc0d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:59.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.322+0000 7f61d1a00640 1 -- 172.21.15.32:0/4159256762 shutdown_connections 2024-04-17T09:53:59.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.322+0000 7f61d1a00640 1 -- 172.21.15.32:0/4159256762 wait complete. 2024-04-17T09:53:59.343 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:53:59.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.486+0000 7f0235800640 1 Processor -- start 2024-04-17T09:53:59.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.486+0000 7f0235800640 1 -- start start 2024-04-17T09:53:59.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.486+0000 7f0235800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f023015ac30 0x7f023006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:59.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.486+0000 7f0235800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f023015b010 0x7f023006dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:59.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.486+0000 7f0235800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f023006fcd0 con 0x7f02300b3d80 2024-04-17T09:53:59.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.486+0000 7f0235800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f023006f300 con 0x7f023015b010 2024-04-17T09:53:59.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.486+0000 7f0235800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f023006e6f0 con 0x7f023015ac30 2024-04-17T09:53:59.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.486+0000 7f0227e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f023015b010 0x7f023006dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:59.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.486+0000 7f0227e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f023015b010 0x7f023006dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44916/0 (socket says 172.21.15.32:44916) 2024-04-17T09:53:59.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.486+0000 7f0227e00640 1 -- 172.21.15.32:0/525764485 learned_addr learned my addr 172.21.15.32:0/525764485 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:59.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.490+0000 7f0227e00640 1 -- 172.21.15.32:0/525764485 >> v2:172.21.15.144:3300/0 conn(0x7f023015ac30 msgr2=0x7f023006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:59.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.490+0000 7f0227e00640 1 --2- 172.21.15.32:0/525764485 >> v2:172.21.15.144:3300/0 conn(0x7f023015ac30 0x7f023006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.490+0000 7f0227e00640 1 -- 172.21.15.32:0/525764485 >> v1:172.21.15.144:6789/0 conn(0x7f02300b3d80 legacy=0x7f0230071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:59.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.490+0000 7f0227e00640 1 -- 172.21.15.32:0/525764485 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023006e9d0 con 0x7f023015b010 2024-04-17T09:53:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.490+0000 7f0227e00640 1 --2- 172.21.15.32:0/525764485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f023015b010 0x7f023006dea0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f02300d1530 tx=0x7f021801e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be32c7a8f6dc0223 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.490+0000 7f022e000640 1 -- 172.21.15.32:0/525764485 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02180ad070 con 0x7f023015b010 2024-04-17T09:53:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.490+0000 7f022e000640 1 -- 172.21.15.32:0/525764485 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0218004230 con 0x7f023015b010 2024-04-17T09:53:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.490+0000 7f022e000640 1 -- 172.21.15.32:0/525764485 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02180a7860 con 0x7f023015b010 2024-04-17T09:53:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.490+0000 7f0235800640 1 -- 172.21.15.32:0/525764485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f023015b010 msgr2=0x7f023006dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.490+0000 7f0235800640 1 --2- 172.21.15.32:0/525764485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f023015b010 0x7f023006dea0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f02300d1530 tx=0x7f021801e990 comp rx=0 tx=0).stop 2024-04-17T09:53:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.490+0000 7f0235800640 1 -- 172.21.15.32:0/525764485 shutdown_connections 2024-04-17T09:53:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.490+0000 7f0235800640 1 --2- 172.21.15.32:0/525764485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f023015b010 0x7f023006dea0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.490+0000 7f0235800640 1 --2- 172.21.15.32:0/525764485 >> v2:172.21.15.144:3300/0 conn(0x7f023015ac30 0x7f023006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.490+0000 7f0235800640 1 -- 172.21.15.32:0/525764485 >> 172.21.15.32:0/525764485 conn(0x7f02300b2120 msgr2=0x7f02300b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f0235800640 1 -- 172.21.15.32:0/525764485 shutdown_connections 2024-04-17T09:53:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f0235800640 1 -- 172.21.15.32:0/525764485 wait complete. 2024-04-17T09:53:59.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f0235800640 1 Processor -- start 2024-04-17T09:53:59.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f0235800640 1 -- start start 2024-04-17T09:53:59.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f0235800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02300b3d80 0x7f0230147b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:59.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f0235800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f023015ac30 0x7f02301498c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:59.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f0235800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f023015b010 0x7f023015c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:59.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f0235800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f02300726f0 con 0x7f023015b010 2024-04-17T09:53:59.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f0235800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0230072570 con 0x7f023015ac30 2024-04-17T09:53:59.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f0235800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f02300cd9c0 con 0x7f02300b3d80 2024-04-17T09:53:59.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f022f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f023015b010 0x7f023015c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:59.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f022f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f023015b010 0x7f023015c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44920/0 (socket says 172.21.15.32:44920) 2024-04-17T09:53:59.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f022f400640 1 -- 172.21.15.32:0/1157591823 learned_addr learned my addr 172.21.15.32:0/1157591823 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:59.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f0227e00640 1 --2- 172.21.15.32:0/1157591823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f023015ac30 0x7f02301498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:59.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f022ea00640 1 --2- 172.21.15.32:0/1157591823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02300b3d80 0x7f0230147b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:59.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f022f400640 1 -- 172.21.15.32:0/1157591823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02300b3d80 msgr2=0x7f0230147b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:59.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f022f400640 1 --2- 172.21.15.32:0/1157591823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02300b3d80 0x7f0230147b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f022f400640 1 -- 172.21.15.32:0/1157591823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f023015ac30 msgr2=0x7f02301498c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:59.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f022f400640 1 --2- 172.21.15.32:0/1157591823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f023015ac30 0x7f02301498c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f022f400640 1 -- 172.21.15.32:0/1157591823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023015a2e0 con 0x7f023015b010 2024-04-17T09:53:59.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f022ea00640 1 --2- 172.21.15.32:0/1157591823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02300b3d80 0x7f0230147b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:53:59.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f022f400640 1 --2- 172.21.15.32:0/1157591823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f023015b010 0x7f023015c300 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f022800c7b0 tx=0x7f022800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:59.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f0226a00640 1 -- 172.21.15.32:0/1157591823 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228017070 con 0x7f023015b010 2024-04-17T09:53:59.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f0226a00640 1 -- 172.21.15.32:0/1157591823 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0228003db0 con 0x7f023015b010 2024-04-17T09:53:59.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f0235800640 1 -- 172.21.15.32:0/1157591823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f023015b540 con 0x7f023015b010 2024-04-17T09:53:59.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f0226a00640 1 -- 172.21.15.32:0/1157591823 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02280052b0 con 0x7f023015b010 2024-04-17T09:53:59.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.494+0000 7f0235800640 1 -- 172.21.15.32:0/1157591823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f023015bb00 con 0x7f023015b010 2024-04-17T09:53:59.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.498+0000 7f0235800640 1 -- 172.21.15.32:0/1157591823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0230068a90 con 0x7f023015b010 2024-04-17T09:53:59.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.498+0000 7f0226a00640 1 -- 172.21.15.32:0/1157591823 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f022801a070 con 0x7f023015b010 2024-04-17T09:53:59.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.498+0000 7f0226a00640 1 --2- 172.21.15.32:0/1157591823 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f020c068380 0x7f020c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:59.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.498+0000 7f0226a00640 1 -- 172.21.15.32:0/1157591823 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f0228086160 con 0x7f023015b010 2024-04-17T09:53:59.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.498+0000 7f022ea00640 1 --2- 172.21.15.32:0/1157591823 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f020c068380 0x7f020c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:59.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.498+0000 7f022ea00640 1 --2- 172.21.15.32:0/1157591823 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f020c068380 0x7f020c06a840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0230152610 tx=0x7f0220008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:59.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.502+0000 7f0226a00640 1 -- 172.21.15.32:0/1157591823 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0228056600 con 0x7f023015b010 2024-04-17T09:53:59.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.666+0000 7f0235800640 1 -- 172.21.15.32:0/1157591823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0230072a60 con 0x7f023015b010 2024-04-17T09:53:59.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.666+0000 7f0226a00640 1 -- 172.21.15.32:0/1157591823 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f0228055f00 con 0x7f023015b010 2024-04-17T09:53:59.668 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:53:59.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.670+0000 7f0235800640 1 -- 172.21.15.32:0/1157591823 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f020c068380 msgr2=0x7f020c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:59.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.670+0000 7f0235800640 1 --2- 172.21.15.32:0/1157591823 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f020c068380 0x7f020c06a840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0230152610 tx=0x7f0220008040 comp rx=0 tx=0).stop 2024-04-17T09:53:59.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.670+0000 7f0235800640 1 -- 172.21.15.32:0/1157591823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f023015b010 msgr2=0x7f023015c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:59.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.670+0000 7f0235800640 1 --2- 172.21.15.32:0/1157591823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f023015b010 0x7f023015c300 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f022800c7b0 tx=0x7f022800cc70 comp rx=0 tx=0).stop 2024-04-17T09:53:59.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.670+0000 7f0235800640 1 -- 172.21.15.32:0/1157591823 shutdown_connections 2024-04-17T09:53:59.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.670+0000 7f0235800640 1 --2- 172.21.15.32:0/1157591823 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f020c068380 0x7f020c06a840 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.670+0000 7f0235800640 1 --2- 172.21.15.32:0/1157591823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f023015b010 0x7f023015c300 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.670+0000 7f0235800640 1 --2- 172.21.15.32:0/1157591823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f023015ac30 0x7f02301498c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.670+0000 7f0235800640 1 --2- 172.21.15.32:0/1157591823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02300b3d80 0x7f0230147b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.670+0000 7f0235800640 1 -- 172.21.15.32:0/1157591823 >> 172.21.15.32:0/1157591823 conn(0x7f02300b2120 msgr2=0x7f02300cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:59.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.670+0000 7f0235800640 1 -- 172.21.15.32:0/1157591823 shutdown_connections 2024-04-17T09:53:59.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.670+0000 7f0235800640 1 -- 172.21.15.32:0/1157591823 wait complete. 2024-04-17T09:53:59.689 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:53:59.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.830+0000 7fccd6200640 1 Processor -- start 2024-04-17T09:53:59.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.830+0000 7fccd6200640 1 -- start start 2024-04-17T09:53:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.834+0000 7fccd6200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fccd015b7d0 0x7fccd006e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.834+0000 7fccd6200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fccd006f7c0 0x7fccd006ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.834+0000 7fccd6200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fccd006f270 0x7fccd006bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.834+0000 7fccd6200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fccd00cc940 con 0x7fccd006f270 2024-04-17T09:53:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.834+0000 7fccd6200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fccd00ccc20 con 0x7fccd015b7d0 2024-04-17T09:53:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.834+0000 7fccd6200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fccd006e0b0 con 0x7fccd006f7c0 2024-04-17T09:53:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.834+0000 7fcccfe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fccd006f270 0x7fccd006bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.834+0000 7fcccfe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fccd006f270 0x7fccd006bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44924/0 (socket says 172.21.15.32:44924) 2024-04-17T09:53:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.834+0000 7fcccfe00640 1 -- 172.21.15.32:0/421800591 learned_addr learned my addr 172.21.15.32:0/421800591 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.834+0000 7fccc7e00640 1 --2- 172.21.15.32:0/421800591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fccd006f7c0 0x7fccd006ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.834+0000 7fcccf400640 1 --2- 172.21.15.32:0/421800591 >> v2:172.21.15.144:3300/0 conn(0x7fccd015b7d0 0x7fccd006e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.834+0000 7fcccfe00640 1 -- 172.21.15.32:0/421800591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fccd006f7c0 msgr2=0x7fccd006ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.834+0000 7fcccfe00640 1 --2- 172.21.15.32:0/421800591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fccd006f7c0 0x7fccd006ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.834+0000 7fcccfe00640 1 -- 172.21.15.32:0/421800591 >> v2:172.21.15.144:3300/0 conn(0x7fccd015b7d0 msgr2=0x7fccd006e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:53:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.834+0000 7fcccfe00640 1 --2- 172.21.15.32:0/421800591 >> v2:172.21.15.144:3300/0 conn(0x7fccd015b7d0 0x7fccd006e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.834+0000 7fcccfe00640 1 -- 172.21.15.32:0/421800591 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd006e390 con 0x7fccd006f270 2024-04-17T09:53:59.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fcccfe00640 1 --2- 172.21.15.32:0/421800591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd006f270 0x7fccd006bf70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fccd00d1530 tx=0x7fccc801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3666e2855fa88631 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:59.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fcccea00640 1 -- 172.21.15.32:0/421800591 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc80a56f0 con 0x7fccd006f270 2024-04-17T09:53:59.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fcccea00640 1 -- 172.21.15.32:0/421800591 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccc80a5890 con 0x7fccd006f270 2024-04-17T09:53:59.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fcccea00640 1 -- 172.21.15.32:0/421800591 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc80a5c60 con 0x7fccd006f270 2024-04-17T09:53:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fccd6200640 1 -- 172.21.15.32:0/421800591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd006f270 msgr2=0x7fccd006bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fccd6200640 1 --2- 172.21.15.32:0/421800591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd006f270 0x7fccd006bf70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fccd00d1530 tx=0x7fccc801c960 comp rx=0 tx=0).stop 2024-04-17T09:53:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fccd6200640 1 -- 172.21.15.32:0/421800591 shutdown_connections 2024-04-17T09:53:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fccd6200640 1 --2- 172.21.15.32:0/421800591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd006f270 0x7fccd006bf70 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fccd6200640 1 --2- 172.21.15.32:0/421800591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fccd006f7c0 0x7fccd006ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fccd6200640 1 --2- 172.21.15.32:0/421800591 >> v2:172.21.15.144:3300/0 conn(0x7fccd015b7d0 0x7fccd006e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fccd6200640 1 -- 172.21.15.32:0/421800591 >> 172.21.15.32:0/421800591 conn(0x7fccd00b2120 msgr2=0x7fccd00b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:53:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fccd6200640 1 -- 172.21.15.32:0/421800591 shutdown_connections 2024-04-17T09:53:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fccd6200640 1 -- 172.21.15.32:0/421800591 wait complete. 2024-04-17T09:53:59.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fccd6200640 1 Processor -- start 2024-04-17T09:53:59.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fccd6200640 1 -- start start 2024-04-17T09:53:59.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fccd6200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fccd006f270 0x7fccd00ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:59.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fccd6200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccd006f7c0 0x7fccd0151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:59.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fccd6200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd015b7d0 0x7fccd0144810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:59.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fccd6200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fccd00cee30 con 0x7fccd015b7d0 2024-04-17T09:53:59.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fccd6200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fccd00cecb0 con 0x7fccd006f7c0 2024-04-17T09:53:59.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fccd6200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fccd00b3420 con 0x7fccd006f270 2024-04-17T09:53:59.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fcccfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd015b7d0 0x7fccd0144810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:59.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fcccfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd015b7d0 0x7fccd0144810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44938/0 (socket says 172.21.15.32:44938) 2024-04-17T09:53:59.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.838+0000 7fcccfe00640 1 -- 172.21.15.32:0/1951111754 learned_addr learned my addr 172.21.15.32:0/1951111754 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:53:59.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.842+0000 7fcccf400640 1 --2- 172.21.15.32:0/1951111754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fccd006f270 0x7fccd00ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:59.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.842+0000 7fccc7e00640 1 --2- 172.21.15.32:0/1951111754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccd006f7c0 0x7fccd0151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:59.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.842+0000 7fcccfe00640 1 -- 172.21.15.32:0/1951111754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fccd006f270 msgr2=0x7fccd00ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:59.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.842+0000 7fcccfe00640 1 --2- 172.21.15.32:0/1951111754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fccd006f270 0x7fccd00ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.842+0000 7fcccfe00640 1 -- 172.21.15.32:0/1951111754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccd006f7c0 msgr2=0x7fccd0151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:53:59.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.842+0000 7fcccfe00640 1 --2- 172.21.15.32:0/1951111754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccd006f7c0 0x7fccd0151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:53:59.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.842+0000 7fcccfe00640 1 -- 172.21.15.32:0/1951111754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd0151840 con 0x7fccd015b7d0 2024-04-17T09:53:59.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.842+0000 7fccc7e00640 1 --2- 172.21.15.32:0/1951111754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccd006f7c0 0x7fccd0151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:59.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.842+0000 7fcccf400640 1 --2- 172.21.15.32:0/1951111754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fccd006f270 0x7fccd00ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:53:59.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.842+0000 7fcccfe00640 1 --2- 172.21.15.32:0/1951111754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd015b7d0 0x7fccd0144810 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fccc801d200 tx=0x7fccc8019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:59.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.842+0000 7fccc7400640 1 -- 172.21.15.32:0/1951111754 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc80b8670 con 0x7fccd015b7d0 2024-04-17T09:53:59.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.842+0000 7fccc7400640 1 -- 172.21.15.32:0/1951111754 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccc80be070 con 0x7fccd015b7d0 2024-04-17T09:53:59.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.842+0000 7fccc7400640 1 -- 172.21.15.32:0/1951111754 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc80b8a90 con 0x7fccd015b7d0 2024-04-17T09:53:59.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.842+0000 7fccd6200640 1 -- 172.21.15.32:0/1951111754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd00af590 con 0x7fccd015b7d0 2024-04-17T09:53:59.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.842+0000 7fccd6200640 1 -- 172.21.15.32:0/1951111754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd00ae7d0 con 0x7fccd015b7d0 2024-04-17T09:53:59.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.842+0000 7fccd6200640 1 -- 172.21.15.32:0/1951111754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd0068a90 con 0x7fccd015b7d0 2024-04-17T09:53:59.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.842+0000 7fccc7400640 1 -- 172.21.15.32:0/1951111754 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fccc80b8c30 con 0x7fccd015b7d0 2024-04-17T09:53:59.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.846+0000 7fccc7400640 1 --2- 172.21.15.32:0/1951111754 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fccac068380 0x7fccac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:53:59.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.846+0000 7fccc7400640 1 -- 172.21.15.32:0/1951111754 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fccc8121b10 con 0x7fccd015b7d0 2024-04-17T09:53:59.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.846+0000 7fcccf400640 1 --2- 172.21.15.32:0/1951111754 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fccac068380 0x7fccac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:53:59.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.846+0000 7fcccf400640 1 --2- 172.21.15.32:0/1951111754 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fccac068380 0x7fccac06a840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fccc000b9f0 tx=0x7fccc0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:53:59.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:53:59.850+0000 7fccc7400640 1 -- 172.21.15.32:0/1951111754 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fccd0151840 con 0x7fccd015b7d0 2024-04-17T09:54:00.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.014+0000 7fccd6200640 1 -- 172.21.15.32:0/1951111754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fccd01526a0 con 0x7fccd015b7d0 2024-04-17T09:54:00.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.014+0000 7fccc7400640 1 -- 172.21.15.32:0/1951111754 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fccc80165e0 con 0x7fccd015b7d0 2024-04-17T09:54:00.016 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:54:00.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.018+0000 7fccd6200640 1 -- 172.21.15.32:0/1951111754 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fccac068380 msgr2=0x7fccac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:00.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.018+0000 7fccd6200640 1 --2- 172.21.15.32:0/1951111754 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fccac068380 0x7fccac06a840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fccc000b9f0 tx=0x7fccc0008040 comp rx=0 tx=0).stop 2024-04-17T09:54:00.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.018+0000 7fccd6200640 1 -- 172.21.15.32:0/1951111754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd015b7d0 msgr2=0x7fccd0144810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:00.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.018+0000 7fccd6200640 1 --2- 172.21.15.32:0/1951111754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd015b7d0 0x7fccd0144810 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fccc801d200 tx=0x7fccc8019700 comp rx=0 tx=0).stop 2024-04-17T09:54:00.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.018+0000 7fccd6200640 1 -- 172.21.15.32:0/1951111754 shutdown_connections 2024-04-17T09:54:00.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.018+0000 7fccd6200640 1 --2- 172.21.15.32:0/1951111754 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fccac068380 0x7fccac06a840 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.018+0000 7fccd6200640 1 --2- 172.21.15.32:0/1951111754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd015b7d0 0x7fccd0144810 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.018+0000 7fccd6200640 1 --2- 172.21.15.32:0/1951111754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccd006f7c0 0x7fccd0151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.018+0000 7fccd6200640 1 --2- 172.21.15.32:0/1951111754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fccd006f270 0x7fccd00ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.018+0000 7fccd6200640 1 -- 172.21.15.32:0/1951111754 >> 172.21.15.32:0/1951111754 conn(0x7fccd00b2120 msgr2=0x7fccd006ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:00.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.018+0000 7fccd6200640 1 -- 172.21.15.32:0/1951111754 shutdown_connections 2024-04-17T09:54:00.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.018+0000 7fccd6200640 1 -- 172.21.15.32:0/1951111754 wait complete. 2024-04-17T09:54:00.037 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T09:54:00.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.178+0000 7fa94be00640 1 Processor -- start 2024-04-17T09:54:00.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.178+0000 7fa94be00640 1 -- start start 2024-04-17T09:54:00.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.178+0000 7fa94be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa94415c820 0x7fa9440d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:00.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.178+0000 7fa94be00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa94409be30 con 0x7fa9440d0b90 2024-04-17T09:54:00.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.178+0000 7fa94be00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa94415cc00 con 0x7fa9440cdbd0 2024-04-17T09:54:00.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.178+0000 7fa94be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa94409a9f0 con 0x7fa94415c820 2024-04-17T09:54:00.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.178+0000 7fa949400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa94415c820 0x7fa9440d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:00.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.178+0000 7fa948a00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fa9440cdbd0 0x7fa94409b720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:46584/0 (socket says 172.21.15.32:46584) 2024-04-17T09:54:00.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.178+0000 7fa948a00640 1 -- 172.21.15.32:0/3981987946 learned_addr learned my addr 172.21.15.32:0/3981987946 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:00.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.178+0000 7fa943e00640 1 -- 172.21.15.32:0/3981987946 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1984026792 0 0) 0x7fa94415cc00 con 0x7fa9440cdbd0 2024-04-17T09:54:00.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.178+0000 7fa949400640 1 -- 172.21.15.32:0/3981987946 >> v1:172.21.15.144:6789/0 conn(0x7fa9440cdbd0 legacy=0x7fa94409b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:00.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.178+0000 7fa949400640 1 -- 172.21.15.32:0/3981987946 >> v1:172.21.15.32:6789/0 conn(0x7fa9440d0b90 legacy=0x7fa9440c6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:00.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.178+0000 7fa949400640 1 -- 172.21.15.32:0/3981987946 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9440ce340 con 0x7fa94415c820 2024-04-17T09:54:00.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.186+0000 7fa949400640 1 --2- 172.21.15.32:0/3981987946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa94415c820 0x7fa9440d0540 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa9380097f0 tx=0x7fa93801c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2011e9086fa91a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:00.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.186+0000 7fa943e00640 1 -- 172.21.15.32:0/3981987946 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9380b1070 con 0x7fa94415c820 2024-04-17T09:54:00.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.186+0000 7fa943e00640 1 -- 172.21.15.32:0/3981987946 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9380ac640 con 0x7fa94415c820 2024-04-17T09:54:00.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.186+0000 7fa943e00640 1 -- 172.21.15.32:0/3981987946 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9380aca10 con 0x7fa94415c820 2024-04-17T09:54:00.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.186+0000 7fa94be00640 1 -- 172.21.15.32:0/3981987946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa94415c820 msgr2=0x7fa9440d0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:00.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.186+0000 7fa94be00640 1 --2- 172.21.15.32:0/3981987946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa94415c820 0x7fa9440d0540 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa9380097f0 tx=0x7fa93801c370 comp rx=0 tx=0).stop 2024-04-17T09:54:00.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.186+0000 7fa94be00640 1 -- 172.21.15.32:0/3981987946 shutdown_connections 2024-04-17T09:54:00.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.186+0000 7fa94be00640 1 --2- 172.21.15.32:0/3981987946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa94415c820 0x7fa9440d0540 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.186+0000 7fa94be00640 1 -- 172.21.15.32:0/3981987946 >> 172.21.15.32:0/3981987946 conn(0x7fa9440b40a0 msgr2=0x7fa9440b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:00.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.186+0000 7fa94be00640 1 -- 172.21.15.32:0/3981987946 shutdown_connections 2024-04-17T09:54:00.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.186+0000 7fa94be00640 1 -- 172.21.15.32:0/3981987946 wait complete. 2024-04-17T09:54:00.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.186+0000 7fa94be00640 1 Processor -- start 2024-04-17T09:54:00.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.186+0000 7fa94be00640 1 -- start start 2024-04-17T09:54:00.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa94be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9440cdbd0 0x7fa94415e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:00.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa94be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9440d0b90 0x7fa9440b73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:00.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa94be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa94415c820 0x7fa9440b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:00.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa94be00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa94409b200 con 0x7fa9440d0b90 2024-04-17T09:54:00.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa94be00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa94409af20 con 0x7fa9440cdbd0 2024-04-17T09:54:00.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa94be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa9440ce030 con 0x7fa94415c820 2024-04-17T09:54:00.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa949400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9440cdbd0 0x7fa94415e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:00.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa949e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa94415c820 0x7fa9440b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:00.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa948a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9440d0b90 0x7fa9440b73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:00.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa949400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9440cdbd0 0x7fa94415e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:33598/0 (socket says 172.21.15.32:33598) 2024-04-17T09:54:00.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa949400640 1 -- 172.21.15.32:0/816981466 learned_addr learned my addr 172.21.15.32:0/816981466 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:00.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa949e00640 1 -- 172.21.15.32:0/816981466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9440cdbd0 msgr2=0x7fa94415e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:00.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa949e00640 1 --2- 172.21.15.32:0/816981466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9440cdbd0 0x7fa94415e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa949e00640 1 -- 172.21.15.32:0/816981466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9440d0b90 msgr2=0x7fa9440b73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:00.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa949e00640 1 --2- 172.21.15.32:0/816981466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9440d0b90 0x7fa9440b73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa949e00640 1 -- 172.21.15.32:0/816981466 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94415cfc0 con 0x7fa94415c820 2024-04-17T09:54:00.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa949400640 1 --2- 172.21.15.32:0/816981466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9440cdbd0 0x7fa94415e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:54:00.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa949e00640 1 --2- 172.21.15.32:0/816981466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa94415c820 0x7fa9440b78e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00c090 tx=0x7fa93c00c550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:00.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa941600640 1 -- 172.21.15.32:0/816981466 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c018030 con 0x7fa94415c820 2024-04-17T09:54:00.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa941600640 1 -- 172.21.15.32:0/816981466 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa93c003980 con 0x7fa94415c820 2024-04-17T09:54:00.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa941600640 1 -- 172.21.15.32:0/816981466 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c003dd0 con 0x7fa94415c820 2024-04-17T09:54:00.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa94be00640 1 -- 172.21.15.32:0/816981466 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9440c7160 con 0x7fa94415c820 2024-04-17T09:54:00.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa94be00640 1 -- 172.21.15.32:0/816981466 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa94415d260 con 0x7fa94415c820 2024-04-17T09:54:00.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa94be00640 1 -- 172.21.15.32:0/816981466 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa944068a90 con 0x7fa94415c820 2024-04-17T09:54:00.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.190+0000 7fa941600640 1 -- 172.21.15.32:0/816981466 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa93c01b070 con 0x7fa94415c820 2024-04-17T09:54:00.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.194+0000 7fa941600640 1 --2- 172.21.15.32:0/816981466 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa924068380 0x7fa92406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:00.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.194+0000 7fa941600640 1 -- 172.21.15.32:0/816981466 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fa93c089320 con 0x7fa94415c820 2024-04-17T09:54:00.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.194+0000 7fa949400640 1 --2- 172.21.15.32:0/816981466 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa924068380 0x7fa92406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:00.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.194+0000 7fa949400640 1 --2- 172.21.15.32:0/816981466 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa924068380 0x7fa92406a840 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa93801aa20 tx=0x7fa9380a6070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:00.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.202+0000 7fa941600640 1 -- 172.21.15.32:0/816981466 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa93c0591f0 con 0x7fa94415c820 2024-04-17T09:54:00.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.378+0000 7fa94be00640 1 -- 172.21.15.32:0/816981466 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa9440b4c90 con 0x7fa94415c820 2024-04-17T09:54:00.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.378+0000 7fa941600640 1 -- 172.21.15.32:0/816981466 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fa93c05d4f0 con 0x7fa94415c820 2024-04-17T09:54:00.380 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T09:54:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.382+0000 7fa94be00640 1 -- 172.21.15.32:0/816981466 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa924068380 msgr2=0x7fa92406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.382+0000 7fa94be00640 1 --2- 172.21.15.32:0/816981466 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa924068380 0x7fa92406a840 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa93801aa20 tx=0x7fa9380a6070 comp rx=0 tx=0).stop 2024-04-17T09:54:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.382+0000 7fa94be00640 1 -- 172.21.15.32:0/816981466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa94415c820 msgr2=0x7fa9440b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.382+0000 7fa94be00640 1 --2- 172.21.15.32:0/816981466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa94415c820 0x7fa9440b78e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00c090 tx=0x7fa93c00c550 comp rx=0 tx=0).stop 2024-04-17T09:54:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.382+0000 7fa94be00640 1 -- 172.21.15.32:0/816981466 shutdown_connections 2024-04-17T09:54:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.382+0000 7fa94be00640 1 --2- 172.21.15.32:0/816981466 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa924068380 0x7fa92406a840 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.382+0000 7fa94be00640 1 --2- 172.21.15.32:0/816981466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa94415c820 0x7fa9440b78e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.382+0000 7fa94be00640 1 --2- 172.21.15.32:0/816981466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9440d0b90 0x7fa9440b73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.382+0000 7fa94be00640 1 --2- 172.21.15.32:0/816981466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9440cdbd0 0x7fa94415e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.382+0000 7fa94be00640 1 -- 172.21.15.32:0/816981466 >> 172.21.15.32:0/816981466 conn(0x7fa9440b40a0 msgr2=0x7fa9440ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.382+0000 7fa94be00640 1 -- 172.21.15.32:0/816981466 shutdown_connections 2024-04-17T09:54:00.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.382+0000 7fa94be00640 1 -- 172.21.15.32:0/816981466 wait complete. 2024-04-17T09:54:00.401 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:54:00.402 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:54:00.411 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:54:00.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.550+0000 7f09ee200640 1 Processor -- start 2024-04-17T09:54:00.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.550+0000 7f09ee200640 1 -- start start 2024-04-17T09:54:00.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.550+0000 7f09ee200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09e8144e90 0x7f09e80b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:00.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.550+0000 7f09ee200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f09e80b4de0 0x7f09e815baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:00.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.554+0000 7f09ee200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09e8072bb0 con 0x7f09e815bfe0 2024-04-17T09:54:00.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.554+0000 7f09ee200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f09e815c430 con 0x7f09e80b4de0 2024-04-17T09:54:00.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.554+0000 7f09ee200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f09e8145270 con 0x7f09e8144e90 2024-04-17T09:54:00.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.554+0000 7f09e7e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f09e815bfe0 0x7f09e806ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48878/0 (socket says 172.21.15.32:48878) 2024-04-17T09:54:00.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.554+0000 7f09e7e00640 1 -- 172.21.15.32:0/1379020721 learned_addr learned my addr 172.21.15.32:0/1379020721 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:00.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.554+0000 7f09e7400640 1 --2- 172.21.15.32:0/1379020721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09e8144e90 0x7f09e80b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:00.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.554+0000 7f09dea00640 1 --2- 172.21.15.32:0/1379020721 >> v2:172.21.15.144:3300/0 conn(0x7f09e80b4de0 0x7f09e815baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:00.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.554+0000 7f09e7400640 1 -- 172.21.15.32:0/1379020721 >> v2:172.21.15.144:3300/0 conn(0x7f09e80b4de0 msgr2=0x7f09e815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:00.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.554+0000 7f09e7400640 1 --2- 172.21.15.32:0/1379020721 >> v2:172.21.15.144:3300/0 conn(0x7f09e80b4de0 0x7f09e815baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.554+0000 7f09e7400640 1 -- 172.21.15.32:0/1379020721 >> v1:172.21.15.32:6789/0 conn(0x7f09e815bfe0 legacy=0x7f09e806ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:00.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.554+0000 7f09e6a00640 1 -- 172.21.15.32:0/1379020721 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 64775329 0 0) 0x7f09e8072bb0 con 0x7f09e815bfe0 2024-04-17T09:54:00.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.554+0000 7f09e7400640 1 -- 172.21.15.32:0/1379020721 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e80b3dc0 con 0x7f09e8144e90 2024-04-17T09:54:00.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.554+0000 7f09e7400640 1 --2- 172.21.15.32:0/1379020721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09e8144e90 0x7f09e80b48a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f09d80097c0 tx=0x7f09d801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c307ef6b6fee008 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:00.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.554+0000 7f09e6a00640 1 -- 172.21.15.32:0/1379020721 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d80aa5f0 con 0x7f09e8144e90 2024-04-17T09:54:00.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.554+0000 7f09e6a00640 1 -- 172.21.15.32:0/1379020721 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d80af070 con 0x7f09e8144e90 2024-04-17T09:54:00.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.554+0000 7f09e6a00640 1 -- 172.21.15.32:0/1379020721 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d80aa9b0 con 0x7f09e8144e90 2024-04-17T09:54:00.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09ee200640 1 -- 172.21.15.32:0/1379020721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09e8144e90 msgr2=0x7f09e80b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:00.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09ee200640 1 --2- 172.21.15.32:0/1379020721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09e8144e90 0x7f09e80b48a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f09d80097c0 tx=0x7f09d801c470 comp rx=0 tx=0).stop 2024-04-17T09:54:00.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09ee200640 1 -- 172.21.15.32:0/1379020721 shutdown_connections 2024-04-17T09:54:00.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09ee200640 1 --2- 172.21.15.32:0/1379020721 >> v2:172.21.15.144:3300/0 conn(0x7f09e80b4de0 0x7f09e815baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09ee200640 1 --2- 172.21.15.32:0/1379020721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09e8144e90 0x7f09e80b48a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09ee200640 1 -- 172.21.15.32:0/1379020721 >> 172.21.15.32:0/1379020721 conn(0x7f09e806ba30 msgr2=0x7f09e806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:00.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09ee200640 1 -- 172.21.15.32:0/1379020721 shutdown_connections 2024-04-17T09:54:00.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09ee200640 1 -- 172.21.15.32:0/1379020721 wait complete. 2024-04-17T09:54:00.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09ee200640 1 Processor -- start 2024-04-17T09:54:00.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09ee200640 1 -- start start 2024-04-17T09:54:00.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09ee200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09e80b4de0 0x7f09e815afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:00.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09ee200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09e8144e90 0x7f09e813b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:00.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09ee200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09e815bfe0 0x7f09e813ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:00.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09ee200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f09e8072670 con 0x7f09e815bfe0 2024-04-17T09:54:00.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09ee200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f09e806f8c0 con 0x7f09e8144e90 2024-04-17T09:54:00.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09ee200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f09e80727f0 con 0x7f09e80b4de0 2024-04-17T09:54:00.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09e7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09e815bfe0 0x7f09e813ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:00.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09e7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09e815bfe0 0x7f09e813ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33002/0 (socket says 172.21.15.32:33002) 2024-04-17T09:54:00.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09e7e00640 1 -- 172.21.15.32:0/864504880 learned_addr learned my addr 172.21.15.32:0/864504880 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:00.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09e7400640 1 --2- 172.21.15.32:0/864504880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09e80b4de0 0x7f09e815afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:00.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09dea00640 1 --2- 172.21.15.32:0/864504880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09e8144e90 0x7f09e813b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:00.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09e7e00640 1 -- 172.21.15.32:0/864504880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09e80b4de0 msgr2=0x7f09e815afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:00.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09e7e00640 1 --2- 172.21.15.32:0/864504880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09e80b4de0 0x7f09e815afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09e7e00640 1 -- 172.21.15.32:0/864504880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09e8144e90 msgr2=0x7f09e813b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:00.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09e7e00640 1 --2- 172.21.15.32:0/864504880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09e8144e90 0x7f09e813b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09e7e00640 1 -- 172.21.15.32:0/864504880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e813b9f0 con 0x7f09e815bfe0 2024-04-17T09:54:00.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.558+0000 7f09dea00640 1 --2- 172.21.15.32:0/864504880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09e8144e90 0x7f09e813b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:00.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.562+0000 7f09e7e00640 1 --2- 172.21.15.32:0/864504880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09e815bfe0 0x7f09e813ac50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f09e000c700 tx=0x7f09e000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:00.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.562+0000 7f09dfe00640 1 -- 172.21.15.32:0/864504880 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0018070 con 0x7f09e815bfe0 2024-04-17T09:54:00.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.562+0000 7f09dfe00640 1 -- 172.21.15.32:0/864504880 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09e000ce10 con 0x7f09e815bfe0 2024-04-17T09:54:00.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.562+0000 7f09dfe00640 1 -- 172.21.15.32:0/864504880 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e000a860 con 0x7f09e815bfe0 2024-04-17T09:54:00.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.562+0000 7f09ee200640 1 -- 172.21.15.32:0/864504880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e813b190 con 0x7f09e815bfe0 2024-04-17T09:54:00.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.562+0000 7f09ee200640 1 -- 172.21.15.32:0/864504880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e813bf70 con 0x7f09e815bfe0 2024-04-17T09:54:00.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.562+0000 7f09ee200640 1 -- 172.21.15.32:0/864504880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e8068a90 con 0x7f09e815bfe0 2024-04-17T09:54:00.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.562+0000 7f09dfe00640 1 -- 172.21.15.32:0/864504880 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f09e000aa00 con 0x7f09e815bfe0 2024-04-17T09:54:00.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.562+0000 7f09dfe00640 1 --2- 172.21.15.32:0/864504880 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f09c40683b0 0x7f09c406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:00.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.562+0000 7f09dfe00640 1 -- 172.21.15.32:0/864504880 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f09e0085e30 con 0x7f09e815bfe0 2024-04-17T09:54:00.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.566+0000 7f09e7400640 1 --2- 172.21.15.32:0/864504880 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f09c40683b0 0x7f09c406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:00.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.566+0000 7f09e7400640 1 --2- 172.21.15.32:0/864504880 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f09c40683b0 0x7f09c406a870 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f09d80097c0 tx=0x7f09d80a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:00.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.570+0000 7f09dfe00640 1 -- 172.21.15.32:0/864504880 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f09e0056400 con 0x7f09e815bfe0 2024-04-17T09:54:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.738+0000 7f09ee200640 1 -- 172.21.15.32:0/864504880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09e806cdd0 con 0x7f09e815bfe0 2024-04-17T09:54:00.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.738+0000 7f09dfe00640 1 -- 172.21.15.32:0/864504880 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16866 (secure 0 0 0) 0x7f09e0055d00 con 0x7f09e815bfe0 2024-04-17T09:54:00.738 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:00.738 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:54.932571+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:54:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.738+0000 7f09ee200640 1 -- 172.21.15.32:0/864504880 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f09c40683b0 msgr2=0x7f09c406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.738+0000 7f09ee200640 1 --2- 172.21.15.32:0/864504880 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f09c40683b0 0x7f09c406a870 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f09d80097c0 tx=0x7f09d80a7000 comp rx=0 tx=0).stop 2024-04-17T09:54:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.738+0000 7f09ee200640 1 -- 172.21.15.32:0/864504880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09e815bfe0 msgr2=0x7f09e813ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.738+0000 7f09ee200640 1 --2- 172.21.15.32:0/864504880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09e815bfe0 0x7f09e813ac50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f09e000c700 tx=0x7f09e000cbc0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.738+0000 7f09ee200640 1 -- 172.21.15.32:0/864504880 shutdown_connections 2024-04-17T09:54:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.738+0000 7f09ee200640 1 --2- 172.21.15.32:0/864504880 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f09c40683b0 0x7f09c406a870 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.738+0000 7f09ee200640 1 --2- 172.21.15.32:0/864504880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09e815bfe0 0x7f09e813ac50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.738+0000 7f09ee200640 1 --2- 172.21.15.32:0/864504880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09e8144e90 0x7f09e813b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.738+0000 7f09ee200640 1 --2- 172.21.15.32:0/864504880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09e80b4de0 0x7f09e815afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.738+0000 7f09ee200640 1 -- 172.21.15.32:0/864504880 >> 172.21.15.32:0/864504880 conn(0x7f09e806ba30 msgr2=0x7f09e806ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.738+0000 7f09ee200640 1 -- 172.21.15.32:0/864504880 shutdown_connections 2024-04-17T09:54:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.738+0000 7f09ee200640 1 -- 172.21.15.32:0/864504880 wait complete. 2024-04-17T09:54:00.759 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:53:11.198098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:53:12.118982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:54:00.760 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:54:00.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.902+0000 7f74bb800640 1 Processor -- start 2024-04-17T09:54:00.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.902+0000 7f74bb800640 1 -- start start 2024-04-17T09:54:00.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.906+0000 7f74bb800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74b406ba30 0x7f74b406be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:00.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.906+0000 7f74bb800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74b4071e60 0x7f74b4072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:00.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.906+0000 7f74bb800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74b406c350 con 0x7f74b40cf370 2024-04-17T09:54:00.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.906+0000 7f74bb800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f74b40cfef0 con 0x7f74b406ba30 2024-04-17T09:54:00.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.906+0000 7f74bb800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f74b40d0130 con 0x7f74b4071e60 2024-04-17T09:54:00.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.906+0000 7f74b3e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f74b40cf370 0x7f74b40cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48890/0 (socket says 172.21.15.32:48890) 2024-04-17T09:54:00.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.906+0000 7f74b3e00640 1 -- 172.21.15.32:0/3757878927 learned_addr learned my addr 172.21.15.32:0/3757878927 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:00.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.906+0000 7f74b8e00640 1 --2- 172.21.15.32:0/3757878927 >> v2:172.21.15.32:3300/0 conn(0x7f74b406ba30 0x7f74b406be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:00.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.906+0000 7f74b9800640 1 --2- 172.21.15.32:0/3757878927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74b4071e60 0x7f74b4072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:00.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.906+0000 7f74b3400640 1 -- 172.21.15.32:0/3757878927 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3302360475 0 0) 0x7f74b406c350 con 0x7f74b40cf370 2024-04-17T09:54:00.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.906+0000 7f74b8e00640 1 -- 172.21.15.32:0/3757878927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74b4071e60 msgr2=0x7f74b4072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:00.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.906+0000 7f74b8e00640 1 --2- 172.21.15.32:0/3757878927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74b4071e60 0x7f74b4072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.906+0000 7f74b8e00640 1 -- 172.21.15.32:0/3757878927 >> v1:172.21.15.32:6789/0 conn(0x7f74b40cf370 legacy=0x7f74b40cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:00.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.906+0000 7f74b8e00640 1 -- 172.21.15.32:0/3757878927 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b415d090 con 0x7f74b406ba30 2024-04-17T09:54:00.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.910+0000 7f74b8e00640 1 --2- 172.21.15.32:0/3757878927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74b406ba30 0x7f74b406be10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f74a80097c0 tx=0x7f74a801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7481bf99928eae9f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:00.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.910+0000 7f74b3400640 1 -- 172.21.15.32:0/3757878927 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a80b1070 con 0x7f74b406ba30 2024-04-17T09:54:00.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.910+0000 7f74b3400640 1 -- 172.21.15.32:0/3757878927 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74a80ac410 con 0x7f74b406ba30 2024-04-17T09:54:00.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.910+0000 7f74b3400640 1 -- 172.21.15.32:0/3757878927 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a80ac7e0 con 0x7f74b406ba30 2024-04-17T09:54:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.910+0000 7f74bb800640 1 -- 172.21.15.32:0/3757878927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74b406ba30 msgr2=0x7f74b406be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.910+0000 7f74bb800640 1 --2- 172.21.15.32:0/3757878927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74b406ba30 0x7f74b406be10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f74a80097c0 tx=0x7f74a801c440 comp rx=0 tx=0).stop 2024-04-17T09:54:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.910+0000 7f74bb800640 1 -- 172.21.15.32:0/3757878927 shutdown_connections 2024-04-17T09:54:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.910+0000 7f74bb800640 1 --2- 172.21.15.32:0/3757878927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74b4071e60 0x7f74b4072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.910+0000 7f74bb800640 1 --2- 172.21.15.32:0/3757878927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74b406ba30 0x7f74b406be10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.910+0000 7f74bb800640 1 -- 172.21.15.32:0/3757878927 >> 172.21.15.32:0/3757878927 conn(0x7f74b40b2120 msgr2=0x7f74b40b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.910+0000 7f74bb800640 1 -- 172.21.15.32:0/3757878927 shutdown_connections 2024-04-17T09:54:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.910+0000 7f74bb800640 1 -- 172.21.15.32:0/3757878927 wait complete. 2024-04-17T09:54:00.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.910+0000 7f74bb800640 1 Processor -- start 2024-04-17T09:54:00.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.910+0000 7f74bb800640 1 -- start start 2024-04-17T09:54:00.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.910+0000 7f74bb800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74b406ba30 0x7f74b415bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:00.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.910+0000 7f74bb800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74b4071e60 0x7f74b40ae750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:00.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.910+0000 7f74bb800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74b40cf370 0x7f74b41443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:00.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.910+0000 7f74bb800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f74b415dcf0 con 0x7f74b406ba30 2024-04-17T09:54:00.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.910+0000 7f74bb800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f74b415db70 con 0x7f74b4071e60 2024-04-17T09:54:00.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.910+0000 7f74bb800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f74b4146230 con 0x7f74b40cf370 2024-04-17T09:54:00.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.910+0000 7f74b8e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74b406ba30 0x7f74b415bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:00.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.914+0000 7f74b8e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74b406ba30 0x7f74b415bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33014/0 (socket says 172.21.15.32:33014) 2024-04-17T09:54:00.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.914+0000 7f74b8e00640 1 -- 172.21.15.32:0/318914352 learned_addr learned my addr 172.21.15.32:0/318914352 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:00.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.914+0000 7f74b9800640 1 --2- 172.21.15.32:0/318914352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74b40cf370 0x7f74b41443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:00.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.914+0000 7f74b3e00640 1 --2- 172.21.15.32:0/318914352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74b4071e60 0x7f74b40ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:00.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.914+0000 7f74b8e00640 1 -- 172.21.15.32:0/318914352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74b40cf370 msgr2=0x7f74b41443a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:00.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.914+0000 7f74b8e00640 1 --2- 172.21.15.32:0/318914352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74b40cf370 0x7f74b41443a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.914+0000 7f74b8e00640 1 -- 172.21.15.32:0/318914352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74b4071e60 msgr2=0x7f74b40ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:00.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.914+0000 7f74b8e00640 1 --2- 172.21.15.32:0/318914352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74b4071e60 0x7f74b40ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:00.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.914+0000 7f74b8e00640 1 -- 172.21.15.32:0/318914352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b415c080 con 0x7f74b406ba30 2024-04-17T09:54:00.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.914+0000 7f74b8e00640 1 --2- 172.21.15.32:0/318914352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74b406ba30 0x7f74b415bb40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f74a80097c0 tx=0x7f74a80191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:00.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.914+0000 7f74b0c00640 1 -- 172.21.15.32:0/318914352 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a80b1040 con 0x7f74b406ba30 2024-04-17T09:54:00.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.914+0000 7f74b0c00640 1 -- 172.21.15.32:0/318914352 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74a80acd70 con 0x7f74b406ba30 2024-04-17T09:54:00.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.914+0000 7f74b0c00640 1 -- 172.21.15.32:0/318914352 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a8006e20 con 0x7f74b406ba30 2024-04-17T09:54:00.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.914+0000 7f74bb800640 1 -- 172.21.15.32:0/318914352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74b40aefd0 con 0x7f74b406ba30 2024-04-17T09:54:00.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.914+0000 7f74bb800640 1 -- 172.21.15.32:0/318914352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74b4142290 con 0x7f74b406ba30 2024-04-17T09:54:00.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.914+0000 7f74bb800640 1 -- 172.21.15.32:0/318914352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74b4068a90 con 0x7f74b406ba30 2024-04-17T09:54:00.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.914+0000 7f74b0c00640 1 -- 172.21.15.32:0/318914352 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f74a8004230 con 0x7f74b406ba30 2024-04-17T09:54:00.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.914+0000 7f74b0c00640 1 --2- 172.21.15.32:0/318914352 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7494068380 0x7f749406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:00.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.914+0000 7f74b0c00640 1 -- 172.21.15.32:0/318914352 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f74a8120590 con 0x7f74b406ba30 2024-04-17T09:54:00.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.918+0000 7f74b3e00640 1 --2- 172.21.15.32:0/318914352 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7494068380 0x7f749406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:00.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.918+0000 7f74b3e00640 1 --2- 172.21.15.32:0/318914352 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7494068380 0x7f749406a840 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f74b413ce60 tx=0x7f74a0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:00.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:00.918+0000 7f74b0c00640 1 -- 172.21.15.32:0/318914352 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f74a80b9020 con 0x7f74b406ba30 2024-04-17T09:54:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.086+0000 7f74bb800640 1 -- 172.21.15.32:0/318914352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f74b413d420 con 0x7f74b406ba30 2024-04-17T09:54:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.086+0000 7f74b0c00640 1 -- 172.21.15.32:0/318914352 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f74a8016090 con 0x7f74b406ba30 2024-04-17T09:54:01.085 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:54:01.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.086+0000 7f74bb800640 1 -- 172.21.15.32:0/318914352 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7494068380 msgr2=0x7f749406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:01.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.086+0000 7f74bb800640 1 --2- 172.21.15.32:0/318914352 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7494068380 0x7f749406a840 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f74b413ce60 tx=0x7f74a0009040 comp rx=0 tx=0).stop 2024-04-17T09:54:01.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.086+0000 7f74bb800640 1 -- 172.21.15.32:0/318914352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74b406ba30 msgr2=0x7f74b415bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:01.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.086+0000 7f74bb800640 1 --2- 172.21.15.32:0/318914352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74b406ba30 0x7f74b415bb40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f74a80097c0 tx=0x7f74a80191e0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.086+0000 7f74bb800640 1 -- 172.21.15.32:0/318914352 shutdown_connections 2024-04-17T09:54:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.086+0000 7f74bb800640 1 --2- 172.21.15.32:0/318914352 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7494068380 0x7f749406a840 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.086+0000 7f74bb800640 1 --2- 172.21.15.32:0/318914352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74b40cf370 0x7f74b41443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.086+0000 7f74bb800640 1 --2- 172.21.15.32:0/318914352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74b4071e60 0x7f74b40ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.086+0000 7f74bb800640 1 --2- 172.21.15.32:0/318914352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74b406ba30 0x7f74b415bb40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.086+0000 7f74bb800640 1 -- 172.21.15.32:0/318914352 >> 172.21.15.32:0/318914352 conn(0x7f74b40b2120 msgr2=0x7f74b413d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.086+0000 7f74bb800640 1 -- 172.21.15.32:0/318914352 shutdown_connections 2024-04-17T09:54:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.086+0000 7f74bb800640 1 -- 172.21.15.32:0/318914352 wait complete. 2024-04-17T09:54:01.106 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:54:01.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.250+0000 7f3aa9a00640 1 Processor -- start 2024-04-17T09:54:01.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.250+0000 7f3aa9a00640 1 -- start start 2024-04-17T09:54:01.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.250+0000 7f3aa9a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3aa406d080 0x7f3aa406d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:01.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.250+0000 7f3aa9a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3aa406da90 0x7f3aa40cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:01.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.250+0000 7f3aa9a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aa40b52b0 con 0x7f3aa415e7f0 2024-04-17T09:54:01.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.250+0000 7f3aa9a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3aa40d1880 con 0x7f3aa406d080 2024-04-17T09:54:01.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.250+0000 7f3aa9a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3aa40d1aa0 con 0x7f3aa406da90 2024-04-17T09:54:01.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.250+0000 7f3aa3e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3aa415e7f0 0x7f3aa40d1170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48902/0 (socket says 172.21.15.32:48902) 2024-04-17T09:54:01.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.250+0000 7f3aa3e00640 1 -- 172.21.15.32:0/1912349321 learned_addr learned my addr 172.21.15.32:0/1912349321 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:01.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.250+0000 7f3a9be00640 1 --2- 172.21.15.32:0/1912349321 >> v2:172.21.15.32:3300/0 conn(0x7f3aa406d080 0x7f3aa406d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:01.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.250+0000 7f3aa9000640 1 --2- 172.21.15.32:0/1912349321 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3aa406da90 0x7f3aa40cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:01.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.254+0000 7f3a9be00640 1 -- 172.21.15.32:0/1912349321 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3aa406da90 msgr2=0x7f3aa40cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:01.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.254+0000 7f3a9be00640 1 --2- 172.21.15.32:0/1912349321 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3aa406da90 0x7f3aa40cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.254+0000 7f3a9be00640 1 -- 172.21.15.32:0/1912349321 >> v1:172.21.15.32:6789/0 conn(0x7f3aa415e7f0 legacy=0x7f3aa40d1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:01.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.254+0000 7f3a9be00640 1 -- 172.21.15.32:0/1912349321 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa40ccb50 con 0x7f3aa406d080 2024-04-17T09:54:01.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.254+0000 7f3aa3400640 1 -- 172.21.15.32:0/1912349321 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1358080469 0 0) 0x7f3aa40b52b0 con 0x7f3aa415e7f0 2024-04-17T09:54:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.254+0000 7f3a9be00640 1 --2- 172.21.15.32:0/1912349321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa406d080 0x7f3aa406d550 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0097c0 tx=0x7f3a8c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccde16c8ddded3bc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.254+0000 7f3aa3400640 1 -- 172.21.15.32:0/1912349321 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c0a7070 con 0x7f3aa406d080 2024-04-17T09:54:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.254+0000 7f3aa3400640 1 -- 172.21.15.32:0/1912349321 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a8c0b0070 con 0x7f3aa406d080 2024-04-17T09:54:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.254+0000 7f3aa3400640 1 -- 172.21.15.32:0/1912349321 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c0ab890 con 0x7f3aa406d080 2024-04-17T09:54:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.254+0000 7f3aa9a00640 1 -- 172.21.15.32:0/1912349321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa406d080 msgr2=0x7f3aa406d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.254+0000 7f3aa9a00640 1 --2- 172.21.15.32:0/1912349321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa406d080 0x7f3aa406d550 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0097c0 tx=0x7f3a8c01c470 comp rx=0 tx=0).stop 2024-04-17T09:54:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.254+0000 7f3aa9a00640 1 -- 172.21.15.32:0/1912349321 shutdown_connections 2024-04-17T09:54:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.254+0000 7f3aa9a00640 1 --2- 172.21.15.32:0/1912349321 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3aa406da90 0x7f3aa40cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.254+0000 7f3aa9a00640 1 --2- 172.21.15.32:0/1912349321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa406d080 0x7f3aa406d550 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.254+0000 7f3aa9a00640 1 -- 172.21.15.32:0/1912349321 >> 172.21.15.32:0/1912349321 conn(0x7f3aa40b3f50 msgr2=0x7f3aa40b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.258+0000 7f3aa9a00640 1 -- 172.21.15.32:0/1912349321 shutdown_connections 2024-04-17T09:54:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.258+0000 7f3aa9a00640 1 -- 172.21.15.32:0/1912349321 wait complete. 2024-04-17T09:54:01.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.258+0000 7f3aa9a00640 1 Processor -- start 2024-04-17T09:54:01.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.258+0000 7f3aa9a00640 1 -- start start 2024-04-17T09:54:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.258+0000 7f3aa9a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3aa406d080 0x7f3aa40cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.258+0000 7f3aa9a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aa406da90 0x7f3aa415d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.258+0000 7f3aa9a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa415e7f0 0x7f3aa415db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.258+0000 7f3aa9a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa415ebd0 con 0x7f3aa415e7f0 2024-04-17T09:54:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.258+0000 7f3aa9a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa406dea0 con 0x7f3aa406da90 2024-04-17T09:54:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.258+0000 7f3aa9a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3aa415ed50 con 0x7f3aa406d080 2024-04-17T09:54:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.258+0000 7f3aa9000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa415e7f0 0x7f3aa415db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.258+0000 7f3aa9000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa415e7f0 0x7f3aa415db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33032/0 (socket says 172.21.15.32:33032) 2024-04-17T09:54:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.258+0000 7f3aa9000640 1 -- 172.21.15.32:0/3241794489 learned_addr learned my addr 172.21.15.32:0/3241794489 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.258+0000 7f3aa3e00640 1 --2- 172.21.15.32:0/3241794489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3aa406d080 0x7f3aa40cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.258+0000 7f3a9be00640 1 --2- 172.21.15.32:0/3241794489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aa406da90 0x7f3aa415d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.258+0000 7f3aa9000640 1 -- 172.21.15.32:0/3241794489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3aa406d080 msgr2=0x7f3aa40cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.258+0000 7f3aa9000640 1 --2- 172.21.15.32:0/3241794489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3aa406d080 0x7f3aa40cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.258+0000 7f3aa9000640 1 -- 172.21.15.32:0/3241794489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aa406da90 msgr2=0x7f3aa415d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.258+0000 7f3aa9000640 1 --2- 172.21.15.32:0/3241794489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aa406da90 0x7f3aa415d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.258+0000 7f3aa9000640 1 -- 172.21.15.32:0/3241794489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa41569b0 con 0x7f3aa415e7f0 2024-04-17T09:54:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.258+0000 7f3aa9000640 1 --2- 172.21.15.32:0/3241794489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa415e7f0 0x7f3aa415db20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c007e40 tx=0x7f3a9c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.262+0000 7f3aa0c00640 1 -- 172.21.15.32:0/3241794489 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c014620 con 0x7f3aa415e7f0 2024-04-17T09:54:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.262+0000 7f3aa0c00640 1 -- 172.21.15.32:0/3241794489 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a9c019070 con 0x7f3aa415e7f0 2024-04-17T09:54:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.262+0000 7f3aa9a00640 1 -- 172.21.15.32:0/3241794489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa4155490 con 0x7f3aa415e7f0 2024-04-17T09:54:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.262+0000 7f3aa0c00640 1 -- 172.21.15.32:0/3241794489 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c014a20 con 0x7f3aa415e7f0 2024-04-17T09:54:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.262+0000 7f3aa9a00640 1 -- 172.21.15.32:0/3241794489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa4155a50 con 0x7f3aa415e7f0 2024-04-17T09:54:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.262+0000 7f3aa9a00640 1 -- 172.21.15.32:0/3241794489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa404fda0 con 0x7f3aa415e7f0 2024-04-17T09:54:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.262+0000 7f3aa0c00640 1 -- 172.21.15.32:0/3241794489 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3a9c014bc0 con 0x7f3aa415e7f0 2024-04-17T09:54:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.262+0000 7f3aa0c00640 1 --2- 172.21.15.32:0/3241794489 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3a80068380 0x7f3a8006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.262+0000 7f3aa0c00640 1 -- 172.21.15.32:0/3241794489 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f3a9c087290 con 0x7f3aa415e7f0 2024-04-17T09:54:01.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.262+0000 7f3aa3e00640 1 --2- 172.21.15.32:0/3241794489 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3a80068380 0x7f3a8006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:01.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.262+0000 7f3aa3e00640 1 --2- 172.21.15.32:0/3241794489 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3a80068380 0x7f3a8006a840 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3a94000f70 tx=0x7f3a94007df0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:01.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.266+0000 7f3aa0c00640 1 -- 172.21.15.32:0/3241794489 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3aa41569b0 con 0x7f3aa415e7f0 2024-04-17T09:54:01.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.438+0000 7f3aa9a00640 1 -- 172.21.15.32:0/3241794489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3aa40b2f90 con 0x7f3aa415e7f0 2024-04-17T09:54:01.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.438+0000 7f3aa0c00640 1 -- 172.21.15.32:0/3241794489 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f3a9c0570e0 con 0x7f3aa415e7f0 2024-04-17T09:54:01.438 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:54:01.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.438+0000 7f3aa9a00640 1 -- 172.21.15.32:0/3241794489 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3a80068380 msgr2=0x7f3a8006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:01.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.438+0000 7f3aa9a00640 1 --2- 172.21.15.32:0/3241794489 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3a80068380 0x7f3a8006a840 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3a94000f70 tx=0x7f3a94007df0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.438+0000 7f3aa9a00640 1 -- 172.21.15.32:0/3241794489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa415e7f0 msgr2=0x7f3aa415db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:01.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.438+0000 7f3aa9a00640 1 --2- 172.21.15.32:0/3241794489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa415e7f0 0x7f3aa415db20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c007e40 tx=0x7f3a9c00e4b0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.438+0000 7f3aa9a00640 1 -- 172.21.15.32:0/3241794489 shutdown_connections 2024-04-17T09:54:01.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.438+0000 7f3aa9a00640 1 --2- 172.21.15.32:0/3241794489 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3a80068380 0x7f3a8006a840 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.438+0000 7f3aa9a00640 1 --2- 172.21.15.32:0/3241794489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa415e7f0 0x7f3aa415db20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.438+0000 7f3aa9a00640 1 --2- 172.21.15.32:0/3241794489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aa406da90 0x7f3aa415d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.438+0000 7f3aa9a00640 1 --2- 172.21.15.32:0/3241794489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3aa406d080 0x7f3aa40cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.438+0000 7f3aa9a00640 1 -- 172.21.15.32:0/3241794489 >> 172.21.15.32:0/3241794489 conn(0x7f3aa40b3f50 msgr2=0x7f3aa40d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:01.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.438+0000 7f3aa9a00640 1 -- 172.21.15.32:0/3241794489 shutdown_connections 2024-04-17T09:54:01.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.438+0000 7f3aa9a00640 1 -- 172.21.15.32:0/3241794489 wait complete. 2024-04-17T09:54:01.458 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:54:01.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.602+0000 7f2bc3a00640 1 Processor -- start 2024-04-17T09:54:01.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.602+0000 7f2bc3a00640 1 -- start start 2024-04-17T09:54:01.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.602+0000 7f2bc3a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2bbc15e7c0 0x7f2bbc15eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:01.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.602+0000 7f2bc3a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bbc0b56d0 con 0x7f2bbc0b4490 2024-04-17T09:54:01.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.602+0000 7f2bc3a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bbc0b39b0 con 0x7f2bbc072970 2024-04-17T09:54:01.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.602+0000 7f2bc3a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2bbc0b3c90 con 0x7f2bbc15e7c0 2024-04-17T09:54:01.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.602+0000 7f2bc1000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2bbc15e7c0 0x7f2bbc15eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:01.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.602+0000 7f2bb3e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2bbc0b4490 0x7f2bbc072260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48914/0 (socket says 172.21.15.32:48914) 2024-04-17T09:54:01.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.602+0000 7f2bb3e00640 1 -- 172.21.15.32:0/3533108131 learned_addr learned my addr 172.21.15.32:0/3533108131 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:01.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.602+0000 7f2bc1000640 1 -- 172.21.15.32:0/3533108131 >> v1:172.21.15.144:6789/0 conn(0x7f2bbc072970 legacy=0x7f2bbc06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:01.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.602+0000 7f2bc1000640 1 -- 172.21.15.32:0/3533108131 >> v1:172.21.15.32:6789/0 conn(0x7f2bbc0b4490 legacy=0x7f2bbc072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:01.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.602+0000 7f2bc1000640 1 -- 172.21.15.32:0/3533108131 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bbc0b3f70 con 0x7f2bbc15e7c0 2024-04-17T09:54:01.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.602+0000 7f2bbbe00640 1 -- 172.21.15.32:0/3533108131 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1517800391 0 0) 0x7f2bbc0b56d0 con 0x7f2bbc0b4490 2024-04-17T09:54:01.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.606+0000 7f2bc1000640 1 --2- 172.21.15.32:0/3533108131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bbc15e7c0 0x7f2bbc15eba0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0097c0 tx=0x7f2bac01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fee9cc4d54996c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:01.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.606+0000 7f2bbbe00640 1 -- 172.21.15.32:0/3533108131 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac0ab5f0 con 0x7f2bbc15e7c0 2024-04-17T09:54:01.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.606+0000 7f2bbbe00640 1 -- 172.21.15.32:0/3533108131 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bac0b0070 con 0x7f2bbc15e7c0 2024-04-17T09:54:01.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.606+0000 7f2bbbe00640 1 -- 172.21.15.32:0/3533108131 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac0ab9b0 con 0x7f2bbc15e7c0 2024-04-17T09:54:01.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.606+0000 7f2bc3a00640 1 -- 172.21.15.32:0/3533108131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bbc15e7c0 msgr2=0x7f2bbc15eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:01.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.606+0000 7f2bc3a00640 1 --2- 172.21.15.32:0/3533108131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bbc15e7c0 0x7f2bbc15eba0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0097c0 tx=0x7f2bac01c3d0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.606+0000 7f2bc3a00640 1 -- 172.21.15.32:0/3533108131 shutdown_connections 2024-04-17T09:54:01.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.606+0000 7f2bc3a00640 1 --2- 172.21.15.32:0/3533108131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bbc15e7c0 0x7f2bbc15eba0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.606+0000 7f2bc3a00640 1 -- 172.21.15.32:0/3533108131 >> 172.21.15.32:0/3533108131 conn(0x7f2bbc0b2510 msgr2=0x7f2bbc0b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:01.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.606+0000 7f2bc3a00640 1 -- 172.21.15.32:0/3533108131 shutdown_connections 2024-04-17T09:54:01.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.606+0000 7f2bc3a00640 1 -- 172.21.15.32:0/3533108131 wait complete. 2024-04-17T09:54:01.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bc3a00640 1 Processor -- start 2024-04-17T09:54:01.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bc3a00640 1 -- start start 2024-04-17T09:54:01.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bc3a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bbc072970 0x7f2bbc152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:01.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bc3a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bbc0b4490 0x7f2bbc160d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:01.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bc3a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bbc15e7c0 0x7f2bbc161270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:01.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bc3a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc16ccd0 con 0x7f2bbc15e7c0 2024-04-17T09:54:01.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bc3a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc16cb50 con 0x7f2bbc072970 2024-04-17T09:54:01.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bc3a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2bbc16ce50 con 0x7f2bbc0b4490 2024-04-17T09:54:01.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bc1a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bbc15e7c0 0x7f2bbc161270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:01.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bc1a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bbc15e7c0 0x7f2bbc161270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33040/0 (socket says 172.21.15.32:33040) 2024-04-17T09:54:01.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bc1a00640 1 -- 172.21.15.32:0/826586667 learned_addr learned my addr 172.21.15.32:0/826586667 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:01.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bb3e00640 1 --2- 172.21.15.32:0/826586667 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bbc0b4490 0x7f2bbc160d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:01.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bc1000640 1 --2- 172.21.15.32:0/826586667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bbc072970 0x7f2bbc152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:01.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bc1a00640 1 -- 172.21.15.32:0/826586667 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bbc0b4490 msgr2=0x7f2bbc160d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:01.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bc1a00640 1 --2- 172.21.15.32:0/826586667 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bbc0b4490 0x7f2bbc160d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bc1a00640 1 -- 172.21.15.32:0/826586667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bbc072970 msgr2=0x7f2bbc152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:01.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bc1a00640 1 --2- 172.21.15.32:0/826586667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bbc072970 0x7f2bbc152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bc1a00640 1 -- 172.21.15.32:0/826586667 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bbc158dd0 con 0x7f2bbc15e7c0 2024-04-17T09:54:01.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bc1a00640 1 --2- 172.21.15.32:0/826586667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bbc15e7c0 0x7f2bbc161270 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400c780 tx=0x7f2bb400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:01.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bb9600640 1 -- 172.21.15.32:0/826586667 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4004b70 con 0x7f2bbc15e7c0 2024-04-17T09:54:01.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bb9600640 1 -- 172.21.15.32:0/826586667 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bb4004d10 con 0x7f2bbc15e7c0 2024-04-17T09:54:01.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bb9600640 1 -- 172.21.15.32:0/826586667 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb400aa90 con 0x7f2bbc15e7c0 2024-04-17T09:54:01.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bc3a00640 1 -- 172.21.15.32:0/826586667 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bbc1593e0 con 0x7f2bbc15e7c0 2024-04-17T09:54:01.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.610+0000 7f2bc3a00640 1 -- 172.21.15.32:0/826586667 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bbc153200 con 0x7f2bbc15e7c0 2024-04-17T09:54:01.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.614+0000 7f2bc3a00640 1 -- 172.21.15.32:0/826586667 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bbc068a90 con 0x7f2bbc15e7c0 2024-04-17T09:54:01.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.614+0000 7f2bb9600640 1 -- 172.21.15.32:0/826586667 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2bb400ac30 con 0x7f2bbc15e7c0 2024-04-17T09:54:01.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.614+0000 7f2bb9600640 1 --2- 172.21.15.32:0/826586667 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2b98068380 0x7f2b9806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:01.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.614+0000 7f2bb9600640 1 -- 172.21.15.32:0/826586667 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f2bb4086c40 con 0x7f2bbc15e7c0 2024-04-17T09:54:01.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.614+0000 7f2bc1000640 1 --2- 172.21.15.32:0/826586667 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2b98068380 0x7f2b9806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:01.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.614+0000 7f2bc1000640 1 --2- 172.21.15.32:0/826586667 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2b98068380 0x7f2b9806a840 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0af000 tx=0x7f2bac0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:01.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.618+0000 7f2bb9600640 1 -- 172.21.15.32:0/826586667 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2bb4020070 con 0x7f2bbc15e7c0 2024-04-17T09:54:01.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.782+0000 7f2bc3a00640 1 -- 172.21.15.32:0/826586667 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2bbc073060 con 0x7f2bbc15e7c0 2024-04-17T09:54:01.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.786+0000 7f2bb9600640 1 -- 172.21.15.32:0/826586667 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f2bb4056a90 con 0x7f2bbc15e7c0 2024-04-17T09:54:01.785 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:54:01.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.786+0000 7f2bc3a00640 1 -- 172.21.15.32:0/826586667 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2b98068380 msgr2=0x7f2b9806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:01.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.786+0000 7f2bc3a00640 1 --2- 172.21.15.32:0/826586667 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2b98068380 0x7f2b9806a840 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0af000 tx=0x7f2bac0a7000 comp rx=0 tx=0).stop 2024-04-17T09:54:01.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.786+0000 7f2bc3a00640 1 -- 172.21.15.32:0/826586667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bbc15e7c0 msgr2=0x7f2bbc161270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:01.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.786+0000 7f2bc3a00640 1 --2- 172.21.15.32:0/826586667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bbc15e7c0 0x7f2bbc161270 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400c780 tx=0x7f2bb400cc40 comp rx=0 tx=0).stop 2024-04-17T09:54:01.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.786+0000 7f2bc3a00640 1 -- 172.21.15.32:0/826586667 shutdown_connections 2024-04-17T09:54:01.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.786+0000 7f2bc3a00640 1 --2- 172.21.15.32:0/826586667 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2b98068380 0x7f2b9806a840 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.786+0000 7f2bc3a00640 1 --2- 172.21.15.32:0/826586667 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bbc0b4490 0x7f2bbc160d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.786+0000 7f2bc3a00640 1 --2- 172.21.15.32:0/826586667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bbc15e7c0 0x7f2bbc161270 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.786+0000 7f2bc3a00640 1 --2- 172.21.15.32:0/826586667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bbc072970 0x7f2bbc152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.786+0000 7f2bc3a00640 1 -- 172.21.15.32:0/826586667 >> 172.21.15.32:0/826586667 conn(0x7f2bbc0b2510 msgr2=0x7f2bbc0707b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:01.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.786+0000 7f2bc3a00640 1 -- 172.21.15.32:0/826586667 shutdown_connections 2024-04-17T09:54:01.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.786+0000 7f2bc3a00640 1 -- 172.21.15.32:0/826586667 wait complete. 2024-04-17T09:54:01.807 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T09:54:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.950+0000 7f65b6e00640 1 Processor -- start 2024-04-17T09:54:01.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.950+0000 7f65b6e00640 1 -- start start 2024-04-17T09:54:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.950+0000 7f65b6e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f65b013f6e0 0x7f65b00cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.950+0000 7f65b6e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f65b00d09f0 0x7f65b0071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.950+0000 7f65b6e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65b01601e0 con 0x7f65b00cf370 2024-04-17T09:54:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.950+0000 7f65b6e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f65b013fac0 con 0x7f65b00d09f0 2024-04-17T09:54:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.950+0000 7f65b6e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f65b0160480 con 0x7f65b013f6e0 2024-04-17T09:54:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.950+0000 7f65b4e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f65b00d09f0 0x7f65b0071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.950+0000 7f65b4e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f65b00d09f0 0x7f65b0071d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33052/0 (socket says 172.21.15.32:33052) 2024-04-17T09:54:01.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.950+0000 7f65b4e00640 1 -- 172.21.15.32:0/2520492114 learned_addr learned my addr 172.21.15.32:0/2520492114 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:01.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.950+0000 7f65afe00640 1 --2- 172.21.15.32:0/2520492114 >> v2:172.21.15.144:3300/0 conn(0x7f65b013f6e0 0x7f65b00cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:01.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.950+0000 7f65b4e00640 1 -- 172.21.15.32:0/2520492114 >> v2:172.21.15.144:3300/0 conn(0x7f65b013f6e0 msgr2=0x7f65b00cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:01.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.950+0000 7f65b4e00640 1 --2- 172.21.15.32:0/2520492114 >> v2:172.21.15.144:3300/0 conn(0x7f65b013f6e0 0x7f65b00cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.950+0000 7f65b4e00640 1 -- 172.21.15.32:0/2520492114 >> v1:172.21.15.144:6789/0 conn(0x7f65b00cf370 legacy=0x7f65b00d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:01.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.950+0000 7f65b4e00640 1 -- 172.21.15.32:0/2520492114 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b0160760 con 0x7f65b00d09f0 2024-04-17T09:54:01.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.954+0000 7f65b4e00640 1 --2- 172.21.15.32:0/2520492114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b00d09f0 0x7f65b0071d80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f65b00b3100 tx=0x7f65a801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c5fae3f29e8c62 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:01.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.954+0000 7f65af400640 1 -- 172.21.15.32:0/2520492114 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a80a5880 con 0x7f65b00d09f0 2024-04-17T09:54:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.954+0000 7f65af400640 1 -- 172.21.15.32:0/2520492114 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65a80a5a20 con 0x7f65b00d09f0 2024-04-17T09:54:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.954+0000 7f65af400640 1 -- 172.21.15.32:0/2520492114 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a80a5df0 con 0x7f65b00d09f0 2024-04-17T09:54:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.954+0000 7f65b6e00640 1 -- 172.21.15.32:0/2520492114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b00d09f0 msgr2=0x7f65b0071d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.954+0000 7f65b6e00640 1 --2- 172.21.15.32:0/2520492114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b00d09f0 0x7f65b0071d80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f65b00b3100 tx=0x7f65a801c9c0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.954+0000 7f65b6e00640 1 -- 172.21.15.32:0/2520492114 shutdown_connections 2024-04-17T09:54:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.954+0000 7f65b6e00640 1 --2- 172.21.15.32:0/2520492114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b00d09f0 0x7f65b0071d80 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.954+0000 7f65b6e00640 1 --2- 172.21.15.32:0/2520492114 >> v2:172.21.15.144:3300/0 conn(0x7f65b013f6e0 0x7f65b00cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.954+0000 7f65b6e00640 1 -- 172.21.15.32:0/2520492114 >> 172.21.15.32:0/2520492114 conn(0x7f65b006ba30 msgr2=0x7f65b006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.954+0000 7f65b6e00640 1 -- 172.21.15.32:0/2520492114 shutdown_connections 2024-04-17T09:54:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.954+0000 7f65b6e00640 1 -- 172.21.15.32:0/2520492114 wait complete. 2024-04-17T09:54:01.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65b6e00640 1 Processor -- start 2024-04-17T09:54:01.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65b6e00640 1 -- start start 2024-04-17T09:54:01.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65b6e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65b00cf370 0x7f65b013c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:01.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65b6e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b00d09f0 0x7f65b01fa590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:01.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65b6e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65b013f6e0 0x7f65b00b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:01.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65b6e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f65b00cff10 con 0x7f65b00d09f0 2024-04-17T09:54:01.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65b6e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65b00cfd90 con 0x7f65b013f6e0 2024-04-17T09:54:01.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65b6e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f65b0072ad0 con 0x7f65b00cf370 2024-04-17T09:54:01.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65a7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b00d09f0 0x7f65b01fa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:01.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65a7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b00d09f0 0x7f65b01fa590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33066/0 (socket says 172.21.15.32:33066) 2024-04-17T09:54:01.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65a7400640 1 -- 172.21.15.32:0/1860986000 learned_addr learned my addr 172.21.15.32:0/1860986000 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:01.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65afe00640 1 --2- 172.21.15.32:0/1860986000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65b00cf370 0x7f65b013c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:01.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65b4e00640 1 --2- 172.21.15.32:0/1860986000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65b013f6e0 0x7f65b00b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:01.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65a7400640 1 -- 172.21.15.32:0/1860986000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65b00cf370 msgr2=0x7f65b013c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:01.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65a7400640 1 --2- 172.21.15.32:0/1860986000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65b00cf370 0x7f65b013c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65a7400640 1 -- 172.21.15.32:0/1860986000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65b013f6e0 msgr2=0x7f65b00b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:01.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65a7400640 1 --2- 172.21.15.32:0/1860986000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65b013f6e0 0x7f65b00b2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:01.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65a7400640 1 -- 172.21.15.32:0/1860986000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b01f9210 con 0x7f65b00d09f0 2024-04-17T09:54:01.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65b4e00640 1 --2- 172.21.15.32:0/1860986000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65b013f6e0 0x7f65b00b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:01.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65afe00640 1 --2- 172.21.15.32:0/1860986000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65b00cf370 0x7f65b013c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:01.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65a7400640 1 --2- 172.21.15.32:0/1860986000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b00d09f0 0x7f65b01fa590 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f659800c3c0 tx=0x7f659800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:01.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65acc00640 1 -- 172.21.15.32:0/1860986000 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598017070 con 0x7f65b00d09f0 2024-04-17T09:54:01.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65acc00640 1 -- 172.21.15.32:0/1860986000 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65980052f0 con 0x7f65b00d09f0 2024-04-17T09:54:01.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65acc00640 1 -- 172.21.15.32:0/1860986000 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598003860 con 0x7f65b00d09f0 2024-04-17T09:54:01.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65b6e00640 1 -- 172.21.15.32:0/1860986000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b01f94f0 con 0x7f65b00d09f0 2024-04-17T09:54:01.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65b6e00640 1 -- 172.21.15.32:0/1860986000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b01f9ab0 con 0x7f65b00d09f0 2024-04-17T09:54:01.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.958+0000 7f65b6e00640 1 -- 172.21.15.32:0/1860986000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b0068a90 con 0x7f65b00d09f0 2024-04-17T09:54:01.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.962+0000 7f65acc00640 1 -- 172.21.15.32:0/1860986000 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f659801a070 con 0x7f65b00d09f0 2024-04-17T09:54:01.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.962+0000 7f65acc00640 1 --2- 172.21.15.32:0/1860986000 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f658c068380 0x7f658c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:01.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.962+0000 7f65acc00640 1 -- 172.21.15.32:0/1860986000 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f6598085cb0 con 0x7f65b00d09f0 2024-04-17T09:54:01.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.962+0000 7f65afe00640 1 --2- 172.21.15.32:0/1860986000 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f658c068380 0x7f658c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:01.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.962+0000 7f65afe00640 1 --2- 172.21.15.32:0/1860986000 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f658c068380 0x7f658c06a840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f65a000b9f0 tx=0x7f65a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:01.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:01.966+0000 7f65acc00640 1 -- 172.21.15.32:0/1860986000 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f6598056280 con 0x7f65b00d09f0 2024-04-17T09:54:02.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.142+0000 7f65b6e00640 1 -- 172.21.15.32:0/1860986000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f65b00b48c0 con 0x7f65b00d09f0 2024-04-17T09:54:02.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.142+0000 7f65acc00640 1 -- 172.21.15.32:0/1860986000 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f6598055b80 con 0x7f65b00d09f0 2024-04-17T09:54:02.145 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T09:54:02.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.146+0000 7f65b6e00640 1 -- 172.21.15.32:0/1860986000 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f658c068380 msgr2=0x7f658c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:02.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.146+0000 7f65b6e00640 1 --2- 172.21.15.32:0/1860986000 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f658c068380 0x7f658c06a840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f65a000b9f0 tx=0x7f65a0008040 comp rx=0 tx=0).stop 2024-04-17T09:54:02.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.146+0000 7f65b6e00640 1 -- 172.21.15.32:0/1860986000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b00d09f0 msgr2=0x7f65b01fa590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:02.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.146+0000 7f65b6e00640 1 --2- 172.21.15.32:0/1860986000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b00d09f0 0x7f65b01fa590 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f659800c3c0 tx=0x7f659800c880 comp rx=0 tx=0).stop 2024-04-17T09:54:02.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.146+0000 7f65b6e00640 1 -- 172.21.15.32:0/1860986000 shutdown_connections 2024-04-17T09:54:02.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.146+0000 7f65b6e00640 1 --2- 172.21.15.32:0/1860986000 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f658c068380 0x7f658c06a840 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:02.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.146+0000 7f65b6e00640 1 --2- 172.21.15.32:0/1860986000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65b013f6e0 0x7f65b00b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:02.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.146+0000 7f65b6e00640 1 --2- 172.21.15.32:0/1860986000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b00d09f0 0x7f65b01fa590 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:02.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.146+0000 7f65b6e00640 1 --2- 172.21.15.32:0/1860986000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65b00cf370 0x7f65b013c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:02.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.146+0000 7f65b6e00640 1 -- 172.21.15.32:0/1860986000 >> 172.21.15.32:0/1860986000 conn(0x7f65b006ba30 msgr2=0x7f65b015f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:02.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.146+0000 7f65b6e00640 1 -- 172.21.15.32:0/1860986000 shutdown_connections 2024-04-17T09:54:02.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.146+0000 7f65b6e00640 1 -- 172.21.15.32:0/1860986000 wait complete. 2024-04-17T09:54:02.262 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi144.front.sepia.ceph.com... 2024-04-17T09:54:02.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:02.262 DEBUG:teuthology.orchestra.run.smithi144:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-17T09:54:02.286 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:54:02.286 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T09:54:02.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:02.286 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-17T09:54:02.336 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-17T09:54:02.336 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:54:02.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.478+0000 7f0184e00640 1 Processor -- start 2024-04-17T09:54:02.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.478+0000 7f0184e00640 1 -- start start 2024-04-17T09:54:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.478+0000 7f0184e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0180072500 0x7f01800cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.478+0000 7f0184e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f018015d8d0 0x7f01800cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.478+0000 7f0184e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01800cdac0 con 0x7f0180145f30 2024-04-17T09:54:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.478+0000 7f0184e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f018006f400 con 0x7f018015d8d0 2024-04-17T09:54:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.478+0000 7f0184e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0180146310 con 0x7f0180072500 2024-04-17T09:54:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.478+0000 7f0177e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0180072500 0x7f01800cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.478+0000 7f017e800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f018015d8d0 0x7f01800cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.478+0000 7f0177e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0180072500 0x7f01800cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:42122/0 (socket says 172.21.15.32:42122) 2024-04-17T09:54:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.478+0000 7f0177e00640 1 -- 172.21.15.32:0/2497624206 learned_addr learned my addr 172.21.15.32:0/2497624206 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.478+0000 7f0177e00640 1 -- 172.21.15.32:0/2497624206 >> v2:172.21.15.144:3300/0 conn(0x7f018015d8d0 msgr2=0x7f01800cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.478+0000 7f0177e00640 1 --2- 172.21.15.32:0/2497624206 >> v2:172.21.15.144:3300/0 conn(0x7f018015d8d0 0x7f01800cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.478+0000 7f0177e00640 1 -- 172.21.15.32:0/2497624206 >> v1:172.21.15.144:6789/0 conn(0x7f0180145f30 legacy=0x7f0180071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.478+0000 7f0177e00640 1 -- 172.21.15.32:0/2497624206 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f018015cdf0 con 0x7f0180072500 2024-04-17T09:54:02.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.482+0000 7f0177e00640 1 --2- 172.21.15.32:0/2497624206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0180072500 0x7f01800cce70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0168009870 tx=0x7f016801ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aea2de8bd2f8b8cb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:02.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.482+0000 7f017d400640 1 -- 172.21.15.32:0/2497624206 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01680b5070 con 0x7f0180072500 2024-04-17T09:54:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.482+0000 7f017d400640 1 -- 172.21.15.32:0/2497624206 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01680a8a50 con 0x7f0180072500 2024-04-17T09:54:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.482+0000 7f017d400640 1 -- 172.21.15.32:0/2497624206 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01680a8e20 con 0x7f0180072500 2024-04-17T09:54:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.482+0000 7f0184e00640 1 -- 172.21.15.32:0/2497624206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0180072500 msgr2=0x7f01800cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.482+0000 7f0184e00640 1 --2- 172.21.15.32:0/2497624206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0180072500 0x7f01800cce70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0168009870 tx=0x7f016801ed50 comp rx=0 tx=0).stop 2024-04-17T09:54:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.482+0000 7f0184e00640 1 -- 172.21.15.32:0/2497624206 shutdown_connections 2024-04-17T09:54:02.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.482+0000 7f0184e00640 1 --2- 172.21.15.32:0/2497624206 >> v2:172.21.15.144:3300/0 conn(0x7f018015d8d0 0x7f01800cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:02.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.482+0000 7f0184e00640 1 --2- 172.21.15.32:0/2497624206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0180072500 0x7f01800cce70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:02.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.482+0000 7f0184e00640 1 -- 172.21.15.32:0/2497624206 >> 172.21.15.32:0/2497624206 conn(0x7f018006ba30 msgr2=0x7f018006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:02.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.482+0000 7f0184e00640 1 -- 172.21.15.32:0/2497624206 shutdown_connections 2024-04-17T09:54:02.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.482+0000 7f0184e00640 1 -- 172.21.15.32:0/2497624206 wait complete. 2024-04-17T09:54:02.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f0184e00640 1 Processor -- start 2024-04-17T09:54:02.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f0184e00640 1 -- start start 2024-04-17T09:54:02.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f0184e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0180072500 0x7f01800c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:02.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f0184e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0180145f30 0x7f0180140b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:02.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f0184e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018015d8d0 0x7f018013dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:02.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f0184e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f018015cb90 con 0x7f0180145f30 2024-04-17T09:54:02.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f0184e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f018015d440 con 0x7f018015d8d0 2024-04-17T09:54:02.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f0184e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f01801601d0 con 0x7f0180072500 2024-04-17T09:54:02.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f0177e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0180145f30 0x7f0180140b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:02.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f0177e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0180145f30 0x7f0180140b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33068/0 (socket says 172.21.15.32:33068) 2024-04-17T09:54:02.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f0177e00640 1 -- 172.21.15.32:0/3355795844 learned_addr learned my addr 172.21.15.32:0/3355795844 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:02.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f017de00640 1 --2- 172.21.15.32:0/3355795844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0180072500 0x7f01800c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:02.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f017e800640 1 --2- 172.21.15.32:0/3355795844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018015d8d0 0x7f018013dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:02.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f0177e00640 1 -- 172.21.15.32:0/3355795844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0180072500 msgr2=0x7f01800c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:02.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f0177e00640 1 --2- 172.21.15.32:0/3355795844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0180072500 0x7f01800c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:02.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f0177e00640 1 -- 172.21.15.32:0/3355795844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018015d8d0 msgr2=0x7f018013dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:02.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f0177e00640 1 --2- 172.21.15.32:0/3355795844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018015d8d0 0x7f018013dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:02.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f0177e00640 1 -- 172.21.15.32:0/3355795844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01800c8cf0 con 0x7f0180145f30 2024-04-17T09:54:02.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f017e800640 1 --2- 172.21.15.32:0/3355795844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018015d8d0 0x7f018013dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:02.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f017de00640 1 --2- 172.21.15.32:0/3355795844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0180072500 0x7f01800c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:02.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f0177e00640 1 --2- 172.21.15.32:0/3355795844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0180145f30 0x7f0180140b10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f01680b3040 tx=0x7f016801baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:02.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f0176000640 1 -- 172.21.15.32:0/3355795844 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01680b5070 con 0x7f0180145f30 2024-04-17T09:54:02.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f0176000640 1 -- 172.21.15.32:0/3355795844 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01680b0de0 con 0x7f0180145f30 2024-04-17T09:54:02.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f0184e00640 1 -- 172.21.15.32:0/3355795844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01800c8fd0 con 0x7f0180145f30 2024-04-17T09:54:02.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f0176000640 1 -- 172.21.15.32:0/3355795844 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01680bb670 con 0x7f0180145f30 2024-04-17T09:54:02.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f0184e00640 1 -- 172.21.15.32:0/3355795844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01800c9590 con 0x7f0180145f30 2024-04-17T09:54:02.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.486+0000 7f0184e00640 1 -- 172.21.15.32:0/3355795844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0180068a90 con 0x7f0180145f30 2024-04-17T09:54:02.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.490+0000 7f0176000640 1 -- 172.21.15.32:0/3355795844 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f01680c2060 con 0x7f0180145f30 2024-04-17T09:54:02.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.490+0000 7f0176000640 1 --2- 172.21.15.32:0/3355795844 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f015c0683b0 0x7f015c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:02.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.490+0000 7f0176000640 1 -- 172.21.15.32:0/3355795844 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f0168123f90 con 0x7f0180145f30 2024-04-17T09:54:02.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.490+0000 7f017de00640 1 --2- 172.21.15.32:0/3355795844 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f015c0683b0 0x7f015c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:02.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.490+0000 7f017de00640 1 --2- 172.21.15.32:0/3355795844 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f015c0683b0 0x7f015c06a870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0170001710 tx=0x7f0170009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:02.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.494+0000 7f0176000640 1 -- 172.21.15.32:0/3355795844 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f01680188b0 con 0x7f0180145f30 2024-04-17T09:54:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.694+0000 7f0184e00640 1 -- 172.21.15.32:0/3355795844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0180072f90 con 0x7f0180145f30 2024-04-17T09:54:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.694+0000 7f0176000640 1 -- 172.21.15.32:0/3355795844 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5193 (secure 0 0 0) 0x7f01680f4430 con 0x7f0180145f30 2024-04-17T09:54:02.694 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:02.694 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/441201886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":441201886},{"type":"v1","addr":"172.21.15.144:6834","nonce":441201886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1066277472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1066277472},{"type":"v1","addr":"172.21.15.32:6836","nonce":1066277472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/201410023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":201410023},{"type":"v1","addr":"172.21.15.32:6837","nonce":201410023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:53:13.172187+0000","modified":"2024-04-17T09:53:16.188325+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/3332924005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3332924005},{"type":"v1","addr":"172.21.15.144:6835","nonce":3332924005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-17T09:54:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2024-04-17T09:54:02.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.694+0000 7f0184e00640 1 -- 172.21.15.32:0/3355795844 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f015c0683b0 msgr2=0x7f015c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:02.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.694+0000 7f0184e00640 1 --2- 172.21.15.32:0/3355795844 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f015c0683b0 0x7f015c06a870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0170001710 tx=0x7f0170009040 comp rx=0 tx=0).stop 2024-04-17T09:54:02.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.694+0000 7f0184e00640 1 -- 172.21.15.32:0/3355795844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0180145f30 msgr2=0x7f0180140b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:02.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.694+0000 7f0184e00640 1 --2- 172.21.15.32:0/3355795844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0180145f30 0x7f0180140b10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f01680b3040 tx=0x7f016801baf0 comp rx=0 tx=0).stop 2024-04-17T09:54:02.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.694+0000 7f0184e00640 1 -- 172.21.15.32:0/3355795844 shutdown_connections 2024-04-17T09:54:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.694+0000 7f0184e00640 1 --2- 172.21.15.32:0/3355795844 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f015c0683b0 0x7f015c06a870 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.694+0000 7f0184e00640 1 --2- 172.21.15.32:0/3355795844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018015d8d0 0x7f018013dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.694+0000 7f0184e00640 1 --2- 172.21.15.32:0/3355795844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0180145f30 0x7f0180140b10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.694+0000 7f0184e00640 1 --2- 172.21.15.32:0/3355795844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0180072500 0x7f01800c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.694+0000 7f0184e00640 1 -- 172.21.15.32:0/3355795844 >> 172.21.15.32:0/3355795844 conn(0x7f018006ba30 msgr2=0x7f018015f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.694+0000 7f0184e00640 1 -- 172.21.15.32:0/3355795844 shutdown_connections 2024-04-17T09:54:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.694+0000 7f0184e00640 1 -- 172.21.15.32:0/3355795844 wait complete. 2024-04-17T09:54:02.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:54:02.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.854+0000 7efece000640 1 Processor -- start 2024-04-17T09:54:02.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.854+0000 7efece000640 1 -- start start 2024-04-17T09:54:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.854+0000 7efece000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efec806f250 0x7efec806dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.854+0000 7efece000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec806e1b0 con 0x7efec80cf5b0 2024-04-17T09:54:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.854+0000 7efece000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec806e490 con 0x7efec80cf990 2024-04-17T09:54:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.854+0000 7efece000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7efec806edc0 con 0x7efec806f250 2024-04-17T09:54:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.854+0000 7efeccc00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efec80cf5b0 0x7efec815d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48916/0 (socket says 172.21.15.32:48916) 2024-04-17T09:54:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.854+0000 7efeccc00640 1 -- 172.21.15.32:0/826354145 learned_addr learned my addr 172.21.15.32:0/826354145 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.854+0000 7efecd600640 1 --2- 172.21.15.32:0/826354145 >> v2:172.21.15.32:3300/0 conn(0x7efec806f250 0x7efec806dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.858+0000 7efec7e00640 1 -- 172.21.15.32:0/826354145 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2410139541 0 0) 0x7efec806e1b0 con 0x7efec80cf5b0 2024-04-17T09:54:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.858+0000 7efec7e00640 1 -- 172.21.15.32:0/826354145 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efeac003530 con 0x7efec80cf5b0 2024-04-17T09:54:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.858+0000 7efecd600640 1 -- 172.21.15.32:0/826354145 >> v1:172.21.15.144:6789/0 conn(0x7efec80cf990 legacy=0x7efec80aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.858+0000 7efec7e00640 1 -- 172.21.15.32:0/826354145 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1267192306 0 0) 0x7efec806e490 con 0x7efec80cf990 2024-04-17T09:54:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.858+0000 7efecd600640 1 -- 172.21.15.32:0/826354145 >> v1:172.21.15.32:6789/0 conn(0x7efec80cf5b0 legacy=0x7efec815d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.858+0000 7efecd600640 1 -- 172.21.15.32:0/826354145 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec806e770 con 0x7efec806f250 2024-04-17T09:54:02.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.858+0000 7efecd600640 1 --2- 172.21.15.32:0/826354145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec806f250 0x7efec806dbd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efec806ef40 tx=0x7efec001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ccf5f5031b88072 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:02.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.858+0000 7efec7e00640 1 -- 172.21.15.32:0/826354145 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec00b2070 con 0x7efec806f250 2024-04-17T09:54:02.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.858+0000 7efec7e00640 1 -- 172.21.15.32:0/826354145 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec00a5860 con 0x7efec806f250 2024-04-17T09:54:02.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.858+0000 7efec7e00640 1 -- 172.21.15.32:0/826354145 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec00a5c30 con 0x7efec806f250 2024-04-17T09:54:02.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.858+0000 7efece000640 1 -- 172.21.15.32:0/826354145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec806f250 msgr2=0x7efec806dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:02.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.858+0000 7efece000640 1 --2- 172.21.15.32:0/826354145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec806f250 0x7efec806dbd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efec806ef40 tx=0x7efec001c890 comp rx=0 tx=0).stop 2024-04-17T09:54:02.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.858+0000 7efece000640 1 -- 172.21.15.32:0/826354145 shutdown_connections 2024-04-17T09:54:02.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.858+0000 7efece000640 1 --2- 172.21.15.32:0/826354145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec806f250 0x7efec806dbd0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:02.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.858+0000 7efece000640 1 -- 172.21.15.32:0/826354145 >> 172.21.15.32:0/826354145 conn(0x7efec80d0180 msgr2=0x7efec80d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:02.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.858+0000 7efece000640 1 -- 172.21.15.32:0/826354145 shutdown_connections 2024-04-17T09:54:02.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.858+0000 7efece000640 1 -- 172.21.15.32:0/826354145 wait complete. 2024-04-17T09:54:02.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.862+0000 7efece000640 1 Processor -- start 2024-04-17T09:54:02.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.862+0000 7efece000640 1 -- start start 2024-04-17T09:54:02.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.862+0000 7efece000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec806f250 0x7efec8144300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:02.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.862+0000 7efece000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efec80cf5b0 0x7efec81406f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:02.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.862+0000 7efece000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efec80cf990 0x7efec8140c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:02.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.862+0000 7efece000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efec815e720 con 0x7efec806f250 2024-04-17T09:54:02.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.862+0000 7efece000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efec815e5a0 con 0x7efec80cf5b0 2024-04-17T09:54:02.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.862+0000 7efece000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efec815e8a0 con 0x7efec80cf990 2024-04-17T09:54:02.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.862+0000 7efeccc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec806f250 0x7efec8144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:02.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.862+0000 7efeccc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec806f250 0x7efec8144300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33092/0 (socket says 172.21.15.32:33092) 2024-04-17T09:54:02.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.862+0000 7efeccc00640 1 -- 172.21.15.32:0/2629209084 learned_addr learned my addr 172.21.15.32:0/2629209084 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:02.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.862+0000 7efeccc00640 1 -- 172.21.15.32:0/2629209084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efec80cf990 msgr2=0x7efec8140c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:02.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.862+0000 7efeccc00640 1 --2- 172.21.15.32:0/2629209084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efec80cf990 0x7efec8140c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:02.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.862+0000 7efeccc00640 1 -- 172.21.15.32:0/2629209084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efec80cf5b0 msgr2=0x7efec81406f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:02.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.862+0000 7efeccc00640 1 --2- 172.21.15.32:0/2629209084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efec80cf5b0 0x7efec81406f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:02.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.862+0000 7efeccc00640 1 -- 172.21.15.32:0/2629209084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec81568a0 con 0x7efec806f250 2024-04-17T09:54:02.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.862+0000 7efeccc00640 1 --2- 172.21.15.32:0/2629209084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec806f250 0x7efec8144300 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efeb800c450 tx=0x7efeb800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:02.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.862+0000 7efec5600640 1 -- 172.21.15.32:0/2629209084 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8017070 con 0x7efec806f250 2024-04-17T09:54:02.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.862+0000 7efec5600640 1 -- 172.21.15.32:0/2629209084 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeb8005db0 con 0x7efec806f250 2024-04-17T09:54:02.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.862+0000 7efece000640 1 -- 172.21.15.32:0/2629209084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec8155800 con 0x7efec806f250 2024-04-17T09:54:02.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.862+0000 7efec5600640 1 -- 172.21.15.32:0/2629209084 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8007290 con 0x7efec806f250 2024-04-17T09:54:02.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.862+0000 7efece000640 1 -- 172.21.15.32:0/2629209084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec8155dc0 con 0x7efec806f250 2024-04-17T09:54:02.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.866+0000 7efece000640 1 -- 172.21.15.32:0/2629209084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec804fd00 con 0x7efec806f250 2024-04-17T09:54:02.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.866+0000 7efec5600640 1 -- 172.21.15.32:0/2629209084 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7efeb801a070 con 0x7efec806f250 2024-04-17T09:54:02.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.866+0000 7efec5600640 1 --2- 172.21.15.32:0/2629209084 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efea40683d0 0x7efea406a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:02.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.866+0000 7efec5600640 1 -- 172.21.15.32:0/2629209084 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7efeb8085e90 con 0x7efec806f250 2024-04-17T09:54:02.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.866+0000 7efebfe00640 1 --2- 172.21.15.32:0/2629209084 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efea40683d0 0x7efea406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:02.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.866+0000 7efebfe00640 1 --2- 172.21.15.32:0/2629209084 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efea40683d0 0x7efea406a890 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efeb0007fd0 tx=0x7efeb0007920 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:02.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:02.870+0000 7efec5600640 1 -- 172.21.15.32:0/2629209084 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7efeb80563b0 con 0x7efec806f250 2024-04-17T09:54:03.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.038+0000 7efece000640 1 -- 172.21.15.32:0/2629209084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efec8073a40 con 0x7efec806f250 2024-04-17T09:54:03.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.038+0000 7efec5600640 1 -- 172.21.15.32:0/2629209084 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16866 (secure 0 0 0) 0x7efeb8087020 con 0x7efec806f250 2024-04-17T09:54:03.039 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:03.039 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:54.932571+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:54:03.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.042+0000 7efece000640 1 -- 172.21.15.32:0/2629209084 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efea40683d0 msgr2=0x7efea406a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:03.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.042+0000 7efece000640 1 --2- 172.21.15.32:0/2629209084 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efea40683d0 0x7efea406a890 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efeb0007fd0 tx=0x7efeb0007920 comp rx=0 tx=0).stop 2024-04-17T09:54:03.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.042+0000 7efece000640 1 -- 172.21.15.32:0/2629209084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec806f250 msgr2=0x7efec8144300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:03.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.042+0000 7efece000640 1 --2- 172.21.15.32:0/2629209084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec806f250 0x7efec8144300 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efeb800c450 tx=0x7efeb800c910 comp rx=0 tx=0).stop 2024-04-17T09:54:03.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.042+0000 7efece000640 1 -- 172.21.15.32:0/2629209084 shutdown_connections 2024-04-17T09:54:03.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.042+0000 7efece000640 1 --2- 172.21.15.32:0/2629209084 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efea40683d0 0x7efea406a890 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.042+0000 7efece000640 1 --2- 172.21.15.32:0/2629209084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efec80cf990 0x7efec8140c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.042+0000 7efece000640 1 --2- 172.21.15.32:0/2629209084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efec80cf5b0 0x7efec81406f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.042+0000 7efece000640 1 --2- 172.21.15.32:0/2629209084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efec806f250 0x7efec8144300 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.042+0000 7efece000640 1 -- 172.21.15.32:0/2629209084 >> 172.21.15.32:0/2629209084 conn(0x7efec80d0180 msgr2=0x7efec815ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:03.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.042+0000 7efece000640 1 -- 172.21.15.32:0/2629209084 shutdown_connections 2024-04-17T09:54:03.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.042+0000 7efece000640 1 -- 172.21.15.32:0/2629209084 wait complete. 2024-04-17T09:54:03.061 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:53:11.198098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:53:12.118982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:54:03.061 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:54:03.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.202+0000 7efe3a600640 1 Processor -- start 2024-04-17T09:54:03.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.202+0000 7efe3a600640 1 -- start start 2024-04-17T09:54:03.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.206+0000 7efe3a600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe3415b7d0 0x7efe3406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:03.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.206+0000 7efe3a600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efe3406c140 0x7efe340ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:03.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.206+0000 7efe3a600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe340cc940 con 0x7efe3406f7c0 2024-04-17T09:54:03.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.206+0000 7efe3a600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7efe340ccc20 con 0x7efe3406c140 2024-04-17T09:54:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.206+0000 7efe3a600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efe3406e0b0 con 0x7efe3415b7d0 2024-04-17T09:54:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.206+0000 7efe33e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efe3406c140 0x7efe340ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.206+0000 7efe33e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efe3406c140 0x7efe340ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33094/0 (socket says 172.21.15.32:33094) 2024-04-17T09:54:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.206+0000 7efe33e00640 1 -- 172.21.15.32:0/962216048 learned_addr learned my addr 172.21.15.32:0/962216048 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.206+0000 7efe33400640 1 --2- 172.21.15.32:0/962216048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe3415b7d0 0x7efe3406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.206+0000 7efe33e00640 1 -- 172.21.15.32:0/962216048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe3415b7d0 msgr2=0x7efe3406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:03.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.206+0000 7efe33e00640 1 --2- 172.21.15.32:0/962216048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe3415b7d0 0x7efe3406e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.206+0000 7efe33e00640 1 -- 172.21.15.32:0/962216048 >> v1:172.21.15.144:6789/0 conn(0x7efe3406f7c0 legacy=0x7efe3406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:03.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.206+0000 7efe33e00640 1 -- 172.21.15.32:0/962216048 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3406e390 con 0x7efe3406c140 2024-04-17T09:54:03.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.206+0000 7efe33400640 1 --2- 172.21.15.32:0/962216048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe3415b7d0 0x7efe3406e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:03.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.210+0000 7efe33e00640 1 --2- 172.21.15.32:0/962216048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe3406c140 0x7efe340ce290 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009870 tx=0x7efe2c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafc282fd5779efd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:03.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.210+0000 7efe32a00640 1 -- 172.21.15.32:0/962216048 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c0aa650 con 0x7efe3406c140 2024-04-17T09:54:03.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.210+0000 7efe32a00640 1 -- 172.21.15.32:0/962216048 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe2c0af070 con 0x7efe3406c140 2024-04-17T09:54:03.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.210+0000 7efe32a00640 1 -- 172.21.15.32:0/962216048 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c0aaa10 con 0x7efe3406c140 2024-04-17T09:54:03.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.210+0000 7efe3a600640 1 -- 172.21.15.32:0/962216048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe3406c140 msgr2=0x7efe340ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:03.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.210+0000 7efe3a600640 1 --2- 172.21.15.32:0/962216048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe3406c140 0x7efe340ce290 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009870 tx=0x7efe2c01c4d0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.210+0000 7efe3a600640 1 -- 172.21.15.32:0/962216048 shutdown_connections 2024-04-17T09:54:03.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.210+0000 7efe3a600640 1 --2- 172.21.15.32:0/962216048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe3406c140 0x7efe340ce290 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.210+0000 7efe3a600640 1 --2- 172.21.15.32:0/962216048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe3415b7d0 0x7efe3406e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.210+0000 7efe3a600640 1 -- 172.21.15.32:0/962216048 >> 172.21.15.32:0/962216048 conn(0x7efe340b2120 msgr2=0x7efe340b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:03.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.210+0000 7efe3a600640 1 -- 172.21.15.32:0/962216048 shutdown_connections 2024-04-17T09:54:03.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.210+0000 7efe3a600640 1 -- 172.21.15.32:0/962216048 wait complete. 2024-04-17T09:54:03.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.210+0000 7efe3a600640 1 Processor -- start 2024-04-17T09:54:03.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.210+0000 7efe3a600640 1 -- start start 2024-04-17T09:54:03.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.210+0000 7efe3a600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe3406c140 0x7efe3415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:03.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.210+0000 7efe3a600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe3406f7c0 0x7efe34155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:03.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.210+0000 7efe3a600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe3415b7d0 0x7efe34146580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:03.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.210+0000 7efe3a600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efe34168630 con 0x7efe3406f7c0 2024-04-17T09:54:03.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.210+0000 7efe3a600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efe341684b0 con 0x7efe3406c140 2024-04-17T09:54:03.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.210+0000 7efe3a600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efe341687b0 con 0x7efe3415b7d0 2024-04-17T09:54:03.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.210+0000 7efe2aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe3406f7c0 0x7efe34155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:03.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.214+0000 7efe2aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe3406f7c0 0x7efe34155710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33098/0 (socket says 172.21.15.32:33098) 2024-04-17T09:54:03.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.214+0000 7efe2aa00640 1 -- 172.21.15.32:0/1774198612 learned_addr learned my addr 172.21.15.32:0/1774198612 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:03.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.214+0000 7efe33400640 1 --2- 172.21.15.32:0/1774198612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe3406c140 0x7efe3415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:03.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.214+0000 7efe33e00640 1 --2- 172.21.15.32:0/1774198612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe3415b7d0 0x7efe34146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:03.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.214+0000 7efe2aa00640 1 -- 172.21.15.32:0/1774198612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe3415b7d0 msgr2=0x7efe34146580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:03.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.214+0000 7efe2aa00640 1 --2- 172.21.15.32:0/1774198612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe3415b7d0 0x7efe34146580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.214+0000 7efe2aa00640 1 -- 172.21.15.32:0/1774198612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe3406c140 msgr2=0x7efe3415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:03.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.214+0000 7efe2aa00640 1 --2- 172.21.15.32:0/1774198612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe3406c140 0x7efe3415bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.214+0000 7efe2aa00640 1 -- 172.21.15.32:0/1774198612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe340ad860 con 0x7efe3406f7c0 2024-04-17T09:54:03.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.214+0000 7efe2aa00640 1 --2- 172.21.15.32:0/1774198612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe3406f7c0 0x7efe34155710 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00c370 tx=0x7efe1c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:03.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.214+0000 7efe2be00640 1 -- 172.21.15.32:0/1774198612 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c0052d0 con 0x7efe3406f7c0 2024-04-17T09:54:03.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.214+0000 7efe2be00640 1 -- 172.21.15.32:0/1774198612 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe1c005470 con 0x7efe3406f7c0 2024-04-17T09:54:03.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.214+0000 7efe3a600640 1 -- 172.21.15.32:0/1774198612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe340adb40 con 0x7efe3406f7c0 2024-04-17T09:54:03.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.214+0000 7efe2be00640 1 -- 172.21.15.32:0/1774198612 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c007260 con 0x7efe3406f7c0 2024-04-17T09:54:03.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.214+0000 7efe3a600640 1 -- 172.21.15.32:0/1774198612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe340ae100 con 0x7efe3406f7c0 2024-04-17T09:54:03.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.214+0000 7efe3a600640 1 -- 172.21.15.32:0/1774198612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe34068a90 con 0x7efe3406f7c0 2024-04-17T09:54:03.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.214+0000 7efe2be00640 1 -- 172.21.15.32:0/1774198612 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7efe1c007400 con 0x7efe3406f7c0 2024-04-17T09:54:03.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.214+0000 7efe2be00640 1 --2- 172.21.15.32:0/1774198612 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efe10068380 0x7efe1006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:03.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.214+0000 7efe2be00640 1 -- 172.21.15.32:0/1774198612 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7efe1c085ed0 con 0x7efe3406f7c0 2024-04-17T09:54:03.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.218+0000 7efe33400640 1 --2- 172.21.15.32:0/1774198612 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efe10068380 0x7efe1006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:03.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.218+0000 7efe33400640 1 --2- 172.21.15.32:0/1774198612 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efe10068380 0x7efe1006a840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efe24003b90 tx=0x7efe24009890 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:03.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.222+0000 7efe2be00640 1 -- 172.21.15.32:0/1774198612 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7efe1c055d20 con 0x7efe3406f7c0 2024-04-17T09:54:03.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.390+0000 7efe3a600640 1 -- 172.21.15.32:0/1774198612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efe3406e9e0 con 0x7efe3406f7c0 2024-04-17T09:54:03.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.390+0000 7efe2be00640 1 -- 172.21.15.32:0/1774198612 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7efe1c05c020 con 0x7efe3406f7c0 2024-04-17T09:54:03.390 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:54:03.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.390+0000 7efe3a600640 1 -- 172.21.15.32:0/1774198612 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efe10068380 msgr2=0x7efe1006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:03.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.390+0000 7efe3a600640 1 --2- 172.21.15.32:0/1774198612 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efe10068380 0x7efe1006a840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efe24003b90 tx=0x7efe24009890 comp rx=0 tx=0).stop 2024-04-17T09:54:03.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.390+0000 7efe3a600640 1 -- 172.21.15.32:0/1774198612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe3406f7c0 msgr2=0x7efe34155710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:03.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.390+0000 7efe3a600640 1 --2- 172.21.15.32:0/1774198612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe3406f7c0 0x7efe34155710 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00c370 tx=0x7efe1c00c830 comp rx=0 tx=0).stop 2024-04-17T09:54:03.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.390+0000 7efe3a600640 1 -- 172.21.15.32:0/1774198612 shutdown_connections 2024-04-17T09:54:03.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.390+0000 7efe3a600640 1 --2- 172.21.15.32:0/1774198612 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efe10068380 0x7efe1006a840 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.390+0000 7efe3a600640 1 --2- 172.21.15.32:0/1774198612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe3415b7d0 0x7efe34146580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.390+0000 7efe3a600640 1 --2- 172.21.15.32:0/1774198612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe3406f7c0 0x7efe34155710 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.390+0000 7efe3a600640 1 --2- 172.21.15.32:0/1774198612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe3406c140 0x7efe3415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.390+0000 7efe3a600640 1 -- 172.21.15.32:0/1774198612 >> 172.21.15.32:0/1774198612 conn(0x7efe340b2120 msgr2=0x7efe34159de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:03.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.390+0000 7efe3a600640 1 -- 172.21.15.32:0/1774198612 shutdown_connections 2024-04-17T09:54:03.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.390+0000 7efe3a600640 1 -- 172.21.15.32:0/1774198612 wait complete. 2024-04-17T09:54:03.411 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:54:03.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.554+0000 7f5f33c00640 1 Processor -- start 2024-04-17T09:54:03.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.554+0000 7f5f33c00640 1 -- start start 2024-04-17T09:54:03.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.554+0000 7f5f33c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f2c141700 0x7f5f2c06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:03.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.554+0000 7f5f33c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5f2c0afeb0 0x7f5f2c0b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:03.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.554+0000 7f5f33c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f2c0b0540 con 0x7f5f2c06cf80 2024-04-17T09:54:03.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.554+0000 7f5f33c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f5f2c06d3e0 con 0x7f5f2c0afeb0 2024-04-17T09:54:03.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.554+0000 7f5f33c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5f2c0b0820 con 0x7f5f2c141700 2024-04-17T09:54:03.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.554+0000 7f5f31200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f2c141700 0x7f5f2c06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:03.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.554+0000 7f5f31c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5f2c0afeb0 0x7f5f2c0b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:03.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.554+0000 7f5f31200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f2c141700 0x7f5f2c06ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:42156/0 (socket says 172.21.15.32:42156) 2024-04-17T09:54:03.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.554+0000 7f5f31200640 1 -- 172.21.15.32:0/4198817551 learned_addr learned my addr 172.21.15.32:0/4198817551 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:03.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.554+0000 7f5f31200640 1 -- 172.21.15.32:0/4198817551 >> v2:172.21.15.144:3300/0 conn(0x7f5f2c0afeb0 msgr2=0x7f5f2c0b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:03.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.554+0000 7f5f31200640 1 --2- 172.21.15.32:0/4198817551 >> v2:172.21.15.144:3300/0 conn(0x7f5f2c0afeb0 0x7f5f2c0b49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.554+0000 7f5f31200640 1 -- 172.21.15.32:0/4198817551 >> v1:172.21.15.144:6789/0 conn(0x7f5f2c06cf80 legacy=0x7f5f2c0af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:03.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.554+0000 7f5f31200640 1 -- 172.21.15.32:0/4198817551 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2c0b3ef0 con 0x7f5f2c141700 2024-04-17T09:54:03.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.554+0000 7f5f2be00640 1 -- 172.21.15.32:0/4198817551 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2104093330 0 0) 0x7f5f2c0b0540 con 0x7f5f2c06cf80 2024-04-17T09:54:03.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.554+0000 7f5f31c00640 1 --2- 172.21.15.32:0/4198817551 >> v2:172.21.15.144:3300/0 conn(0x7f5f2c0afeb0 0x7f5f2c0b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:54:03.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.558+0000 7f5f31200640 1 --2- 172.21.15.32:0/4198817551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f2c141700 0x7f5f2c06ca40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009870 tx=0x7f5f1c01c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db96cfb5a5e82ee server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:03.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.558+0000 7f5f2be00640 1 -- 172.21.15.32:0/4198817551 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c0ab430 con 0x7f5f2c141700 2024-04-17T09:54:03.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.558+0000 7f5f2be00640 1 -- 172.21.15.32:0/4198817551 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f1c0b0070 con 0x7f5f2c141700 2024-04-17T09:54:03.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.558+0000 7f5f2be00640 1 -- 172.21.15.32:0/4198817551 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c0ab7f0 con 0x7f5f2c141700 2024-04-17T09:54:03.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.558+0000 7f5f33c00640 1 -- 172.21.15.32:0/4198817551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f2c141700 msgr2=0x7f5f2c06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:03.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.558+0000 7f5f33c00640 1 --2- 172.21.15.32:0/4198817551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f2c141700 0x7f5f2c06ca40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009870 tx=0x7f5f1c01c540 comp rx=0 tx=0).stop 2024-04-17T09:54:03.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.558+0000 7f5f33c00640 1 -- 172.21.15.32:0/4198817551 shutdown_connections 2024-04-17T09:54:03.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.558+0000 7f5f33c00640 1 --2- 172.21.15.32:0/4198817551 >> v2:172.21.15.144:3300/0 conn(0x7f5f2c0afeb0 0x7f5f2c0b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.558+0000 7f5f33c00640 1 --2- 172.21.15.32:0/4198817551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f2c141700 0x7f5f2c06ca40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f33c00640 1 -- 172.21.15.32:0/4198817551 >> 172.21.15.32:0/4198817551 conn(0x7f5f2c072670 msgr2=0x7f5f2c072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:03.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f33c00640 1 -- 172.21.15.32:0/4198817551 shutdown_connections 2024-04-17T09:54:03.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f33c00640 1 -- 172.21.15.32:0/4198817551 wait complete. 2024-04-17T09:54:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f33c00640 1 Processor -- start 2024-04-17T09:54:03.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f33c00640 1 -- start start 2024-04-17T09:54:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f33c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f2c06cf80 0x7f5f2c15c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f33c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f2c0afeb0 0x7f5f2c144150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f33c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f2c141700 0x7f5f2c144690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f33c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c0b5720 con 0x7f5f2c0afeb0 2024-04-17T09:54:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f33c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c141cf0 con 0x7f5f2c06cf80 2024-04-17T09:54:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f33c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5f2c0b58a0 con 0x7f5f2c141700 2024-04-17T09:54:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f23e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f2c0afeb0 0x7f5f2c144150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:03.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f31200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f2c06cf80 0x7f5f2c15c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f23e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f2c0afeb0 0x7f5f2c144150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33114/0 (socket says 172.21.15.32:33114) 2024-04-17T09:54:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f31200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f2c06cf80 0x7f5f2c15c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:44794/0 (socket says 172.21.15.32:44794) 2024-04-17T09:54:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f23e00640 1 -- 172.21.15.32:0/2994132288 learned_addr learned my addr 172.21.15.32:0/2994132288 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f31c00640 1 --2- 172.21.15.32:0/2994132288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f2c141700 0x7f5f2c144690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f31200640 1 -- 172.21.15.32:0/2994132288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f2c141700 msgr2=0x7f5f2c144690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f31200640 1 --2- 172.21.15.32:0/2994132288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f2c141700 0x7f5f2c144690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f31200640 1 -- 172.21.15.32:0/2994132288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f2c0afeb0 msgr2=0x7f5f2c144150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f31200640 1 --2- 172.21.15.32:0/2994132288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f2c0afeb0 0x7f5f2c144150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f31200640 1 -- 172.21.15.32:0/2994132288 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2c13d7c0 con 0x7f5f2c06cf80 2024-04-17T09:54:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f23e00640 1 --2- 172.21.15.32:0/2994132288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f2c0afeb0 0x7f5f2c144150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:54:03.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.562+0000 7f5f31200640 1 --2- 172.21.15.32:0/2994132288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f2c06cf80 0x7f5f2c15c0a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0ac000 tx=0x7f5f1c0192e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:03.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.566+0000 7f5f29600640 1 -- 172.21.15.32:0/2994132288 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c0be410 con 0x7f5f2c06cf80 2024-04-17T09:54:03.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.566+0000 7f5f29600640 1 -- 172.21.15.32:0/2994132288 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f1c0be5d0 con 0x7f5f2c06cf80 2024-04-17T09:54:03.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.566+0000 7f5f33c00640 1 -- 172.21.15.32:0/2994132288 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f2c142a00 con 0x7f5f2c06cf80 2024-04-17T09:54:03.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.566+0000 7f5f29600640 1 -- 172.21.15.32:0/2994132288 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c0be9c0 con 0x7f5f2c06cf80 2024-04-17T09:54:03.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.566+0000 7f5f33c00640 1 -- 172.21.15.32:0/2994132288 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f2c15dc80 con 0x7f5f2c06cf80 2024-04-17T09:54:03.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.566+0000 7f5f33c00640 1 -- 172.21.15.32:0/2994132288 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f2c068a90 con 0x7f5f2c06cf80 2024-04-17T09:54:03.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.566+0000 7f5f29600640 1 -- 172.21.15.32:0/2994132288 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5f1c0bc020 con 0x7f5f2c06cf80 2024-04-17T09:54:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.566+0000 7f5f29600640 1 --2- 172.21.15.32:0/2994132288 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5f080683b0 0x7f5f0806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.566+0000 7f5f29600640 1 -- 172.21.15.32:0/2994132288 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f5f1c120c90 con 0x7f5f2c06cf80 2024-04-17T09:54:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.566+0000 7f5f23e00640 1 --2- 172.21.15.32:0/2994132288 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5f080683b0 0x7f5f0806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.570+0000 7f5f23e00640 1 --2- 172.21.15.32:0/2994132288 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5f080683b0 0x7f5f0806a870 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c15e6d0 tx=0x7f5f14009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:03.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.570+0000 7f5f29600640 1 -- 172.21.15.32:0/2994132288 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5f1c016130 con 0x7f5f2c06cf80 2024-04-17T09:54:03.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.738+0000 7f5f33c00640 1 -- 172.21.15.32:0/2994132288 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5f2c073930 con 0x7f5f2c06cf80 2024-04-17T09:54:03.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.738+0000 7f5f29600640 1 -- 172.21.15.32:0/2994132288 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f5f1c0f0bf0 con 0x7f5f2c06cf80 2024-04-17T09:54:03.740 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:54:03.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.742+0000 7f5f33c00640 1 -- 172.21.15.32:0/2994132288 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5f080683b0 msgr2=0x7f5f0806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:03.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.742+0000 7f5f33c00640 1 --2- 172.21.15.32:0/2994132288 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5f080683b0 0x7f5f0806a870 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c15e6d0 tx=0x7f5f14009040 comp rx=0 tx=0).stop 2024-04-17T09:54:03.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.742+0000 7f5f33c00640 1 -- 172.21.15.32:0/2994132288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f2c06cf80 msgr2=0x7f5f2c15c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:03.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.742+0000 7f5f33c00640 1 --2- 172.21.15.32:0/2994132288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f2c06cf80 0x7f5f2c15c0a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0ac000 tx=0x7f5f1c0192e0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.742+0000 7f5f33c00640 1 -- 172.21.15.32:0/2994132288 shutdown_connections 2024-04-17T09:54:03.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.742+0000 7f5f33c00640 1 --2- 172.21.15.32:0/2994132288 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5f080683b0 0x7f5f0806a870 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.742+0000 7f5f33c00640 1 --2- 172.21.15.32:0/2994132288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f2c141700 0x7f5f2c144690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.742+0000 7f5f33c00640 1 --2- 172.21.15.32:0/2994132288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f2c0afeb0 0x7f5f2c144150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.742+0000 7f5f33c00640 1 --2- 172.21.15.32:0/2994132288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f2c06cf80 0x7f5f2c15c0a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.742+0000 7f5f33c00640 1 -- 172.21.15.32:0/2994132288 >> 172.21.15.32:0/2994132288 conn(0x7f5f2c072670 msgr2=0x7f5f2c0b09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:03.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.742+0000 7f5f33c00640 1 -- 172.21.15.32:0/2994132288 shutdown_connections 2024-04-17T09:54:03.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.742+0000 7f5f33c00640 1 -- 172.21.15.32:0/2994132288 wait complete. 2024-04-17T09:54:03.762 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:54:03.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.902+0000 7fe17a000640 1 Processor -- start 2024-04-17T09:54:03.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.902+0000 7fe17a000640 1 -- start start 2024-04-17T09:54:03.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.906+0000 7fe17a000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe17406ba30 0x7fe17406be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:03.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.906+0000 7fe17a000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1740cf370 0x7fe1740cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:03.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.906+0000 7fe17a000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe17406c350 con 0x7fe174160350 2024-04-17T09:54:03.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.906+0000 7fe17a000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe1740d0130 con 0x7fe17406ba30 2024-04-17T09:54:03.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.906+0000 7fe17a000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe17415d090 con 0x7fe1740cf370 2024-04-17T09:54:03.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.906+0000 7fe173e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe174160350 0x7fe1741607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48930/0 (socket says 172.21.15.32:48930) 2024-04-17T09:54:03.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.906+0000 7fe173e00640 1 -- 172.21.15.32:0/8284383 learned_addr learned my addr 172.21.15.32:0/8284383 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:03.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.906+0000 7fe173400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe17406ba30 0x7fe17406be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:03.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.906+0000 7fe16aa00640 1 --2- 172.21.15.32:0/8284383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1740cf370 0x7fe1740cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:03.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.906+0000 7fe172a00640 1 -- 172.21.15.32:0/8284383 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1602662589 0 0) 0x7fe17406c350 con 0x7fe174160350 2024-04-17T09:54:03.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.906+0000 7fe172a00640 1 -- 172.21.15.32:0/8284383 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe158003530 con 0x7fe174160350 2024-04-17T09:54:03.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.906+0000 7fe173400640 1 -- 172.21.15.32:0/8284383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1740cf370 msgr2=0x7fe1740cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:03.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.906+0000 7fe173400640 1 --2- 172.21.15.32:0/8284383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1740cf370 0x7fe1740cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.906+0000 7fe173400640 1 -- 172.21.15.32:0/8284383 >> v1:172.21.15.32:6789/0 conn(0x7fe174160350 legacy=0x7fe1741607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:03.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.906+0000 7fe173400640 1 -- 172.21.15.32:0/8284383 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17415d370 con 0x7fe17406ba30 2024-04-17T09:54:03.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.910+0000 7fe173400640 1 --2- 172.21.15.32:0/8284383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe17406ba30 0x7fe17406be10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe1640097c0 tx=0x7fe16401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0abb5d82fb37ef8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:03.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.910+0000 7fe172a00640 1 -- 172.21.15.32:0/8284383 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1640b1070 con 0x7fe17406ba30 2024-04-17T09:54:03.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.910+0000 7fe172a00640 1 -- 172.21.15.32:0/8284383 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1640ac410 con 0x7fe17406ba30 2024-04-17T09:54:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.910+0000 7fe172a00640 1 -- 172.21.15.32:0/8284383 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1640ac7c0 con 0x7fe17406ba30 2024-04-17T09:54:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.910+0000 7fe17a000640 1 -- 172.21.15.32:0/8284383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe17406ba30 msgr2=0x7fe17406be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.910+0000 7fe17a000640 1 --2- 172.21.15.32:0/8284383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe17406ba30 0x7fe17406be10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe1640097c0 tx=0x7fe16401c4a0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.910+0000 7fe17a000640 1 -- 172.21.15.32:0/8284383 shutdown_connections 2024-04-17T09:54:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.910+0000 7fe17a000640 1 --2- 172.21.15.32:0/8284383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1740cf370 0x7fe1740cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.910+0000 7fe17a000640 1 --2- 172.21.15.32:0/8284383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe17406ba30 0x7fe17406be10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.910+0000 7fe17a000640 1 -- 172.21.15.32:0/8284383 >> 172.21.15.32:0/8284383 conn(0x7fe1740b2120 msgr2=0x7fe1740b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:03.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.910+0000 7fe17a000640 1 -- 172.21.15.32:0/8284383 shutdown_connections 2024-04-17T09:54:03.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.910+0000 7fe17a000640 1 -- 172.21.15.32:0/8284383 wait complete. 2024-04-17T09:54:03.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.910+0000 7fe17a000640 1 Processor -- start 2024-04-17T09:54:03.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.910+0000 7fe17a000640 1 -- start start 2024-04-17T09:54:03.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe17a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe17406ba30 0x7fe17415a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:03.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe17a000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1740cf370 0x7fe17413b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:03.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe17a000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe174160350 0x7fe1740b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:03.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe17a000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe17415dcf0 con 0x7fe17406ba30 2024-04-17T09:54:03.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe17a000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe17415db70 con 0x7fe174160350 2024-04-17T09:54:03.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe17a000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe174146230 con 0x7fe1740cf370 2024-04-17T09:54:03.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe173400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe17406ba30 0x7fe17415a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:03.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe173400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe17406ba30 0x7fe17415a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33134/0 (socket says 172.21.15.32:33134) 2024-04-17T09:54:03.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe173400640 1 -- 172.21.15.32:0/2976053741 learned_addr learned my addr 172.21.15.32:0/2976053741 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:03.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe16aa00640 1 --2- 172.21.15.32:0/2976053741 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1740cf370 0x7fe17413b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:03.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe173e00640 1 --2- 172.21.15.32:0/2976053741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe174160350 0x7fe1740b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:03.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe173400640 1 -- 172.21.15.32:0/2976053741 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1740cf370 msgr2=0x7fe17413b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:03.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe173400640 1 --2- 172.21.15.32:0/2976053741 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1740cf370 0x7fe17413b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe173400640 1 -- 172.21.15.32:0/2976053741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe174160350 msgr2=0x7fe1740b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:03.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe173400640 1 --2- 172.21.15.32:0/2976053741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe174160350 0x7fe1740b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:03.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe173400640 1 -- 172.21.15.32:0/2976053741 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17413e840 con 0x7fe17406ba30 2024-04-17T09:54:03.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe173e00640 1 --2- 172.21.15.32:0/2976053741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe174160350 0x7fe1740b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:54:03.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe173400640 1 --2- 172.21.15.32:0/2976053741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe17406ba30 0x7fe17415a110 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe16401a0d0 tx=0x7fe164019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:03.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe16be00640 1 -- 172.21.15.32:0/2976053741 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1640b9070 con 0x7fe17406ba30 2024-04-17T09:54:03.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe16be00640 1 -- 172.21.15.32:0/2976053741 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1640accc0 con 0x7fe17406ba30 2024-04-17T09:54:03.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe16be00640 1 -- 172.21.15.32:0/2976053741 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1640b1040 con 0x7fe17406ba30 2024-04-17T09:54:03.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe17a000640 1 -- 172.21.15.32:0/2976053741 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe17413eb20 con 0x7fe17406ba30 2024-04-17T09:54:03.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe17a000640 1 -- 172.21.15.32:0/2976053741 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe17413f0e0 con 0x7fe17406ba30 2024-04-17T09:54:03.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe17a000640 1 -- 172.21.15.32:0/2976053741 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe174068a90 con 0x7fe17406ba30 2024-04-17T09:54:03.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.914+0000 7fe16be00640 1 -- 172.21.15.32:0/2976053741 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe1640bc070 con 0x7fe17406ba30 2024-04-17T09:54:03.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.918+0000 7fe16be00640 1 --2- 172.21.15.32:0/2976053741 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe1500683b0 0x7fe15006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:03.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.918+0000 7fe16be00640 1 -- 172.21.15.32:0/2976053741 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fe164120510 con 0x7fe17406ba30 2024-04-17T09:54:03.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.918+0000 7fe16aa00640 1 --2- 172.21.15.32:0/2976053741 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe1500683b0 0x7fe15006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:03.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.918+0000 7fe16aa00640 1 --2- 172.21.15.32:0/2976053741 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe1500683b0 0x7fe15006a870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe1740b1d30 tx=0x7fe15c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:03.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:03.922+0000 7fe16be00640 1 -- 172.21.15.32:0/2976053741 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fe164016090 con 0x7fe17406ba30 2024-04-17T09:54:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.086+0000 7fe17a000640 1 -- 172.21.15.32:0/2976053741 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe174145a60 con 0x7fe17406ba30 2024-04-17T09:54:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.086+0000 7fe16be00640 1 -- 172.21.15.32:0/2976053741 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fe1640f0a40 con 0x7fe17406ba30 2024-04-17T09:54:04.087 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:54:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.086+0000 7fe17a000640 1 -- 172.21.15.32:0/2976053741 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe1500683b0 msgr2=0x7fe15006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.086+0000 7fe17a000640 1 --2- 172.21.15.32:0/2976053741 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe1500683b0 0x7fe15006a870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe1740b1d30 tx=0x7fe15c008040 comp rx=0 tx=0).stop 2024-04-17T09:54:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.086+0000 7fe17a000640 1 -- 172.21.15.32:0/2976053741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe17406ba30 msgr2=0x7fe17415a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.086+0000 7fe17a000640 1 --2- 172.21.15.32:0/2976053741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe17406ba30 0x7fe17415a110 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe16401a0d0 tx=0x7fe164019240 comp rx=0 tx=0).stop 2024-04-17T09:54:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.086+0000 7fe17a000640 1 -- 172.21.15.32:0/2976053741 shutdown_connections 2024-04-17T09:54:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.086+0000 7fe17a000640 1 --2- 172.21.15.32:0/2976053741 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe1500683b0 0x7fe15006a870 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.086+0000 7fe17a000640 1 --2- 172.21.15.32:0/2976053741 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1740cf370 0x7fe17413b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.086+0000 7fe17a000640 1 --2- 172.21.15.32:0/2976053741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe174160350 0x7fe1740b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.086+0000 7fe17a000640 1 --2- 172.21.15.32:0/2976053741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe17406ba30 0x7fe17415a110 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.086+0000 7fe17a000640 1 -- 172.21.15.32:0/2976053741 >> 172.21.15.32:0/2976053741 conn(0x7fe1740b2120 msgr2=0x7fe17415b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.086+0000 7fe17a000640 1 -- 172.21.15.32:0/2976053741 shutdown_connections 2024-04-17T09:54:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.090+0000 7fe17a000640 1 -- 172.21.15.32:0/2976053741 wait complete. 2024-04-17T09:54:04.108 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:54:04.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.246+0000 7fa9ba800640 1 Processor -- start 2024-04-17T09:54:04.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.246+0000 7fa9ba800640 1 -- start start 2024-04-17T09:54:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.250+0000 7fa9ba800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa9b4145f30 0x7fa9b4071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.250+0000 7fa9ba800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9b40cd580 0x7fa9b4072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.250+0000 7fa9ba800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9b40cdac0 con 0x7fa9b415d8d0 2024-04-17T09:54:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.250+0000 7fa9ba800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa9b406f400 con 0x7fa9b40cd580 2024-04-17T09:54:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.250+0000 7fa9ba800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa9b4146310 con 0x7fa9b4145f30 2024-04-17T09:54:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.250+0000 7fa9b3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9b40cd580 0x7fa9b4072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.250+0000 7fa9aaa00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa9b415d8d0 0x7fa9b40cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48944/0 (socket says 172.21.15.32:48944) 2024-04-17T09:54:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.250+0000 7fa9aaa00640 1 -- 172.21.15.32:0/3560618956 learned_addr learned my addr 172.21.15.32:0/3560618956 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.250+0000 7fa9b3400640 1 --2- 172.21.15.32:0/3560618956 >> v2:172.21.15.144:3300/0 conn(0x7fa9b4145f30 0x7fa9b4071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:04.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.250+0000 7fa9b3e00640 1 -- 172.21.15.32:0/3560618956 >> v2:172.21.15.144:3300/0 conn(0x7fa9b4145f30 msgr2=0x7fa9b4071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:04.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.250+0000 7fa9b3e00640 1 --2- 172.21.15.32:0/3560618956 >> v2:172.21.15.144:3300/0 conn(0x7fa9b4145f30 0x7fa9b4071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.250+0000 7fa9b3e00640 1 -- 172.21.15.32:0/3560618956 >> v1:172.21.15.32:6789/0 conn(0x7fa9b415d8d0 legacy=0x7fa9b40cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:04.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.250+0000 7fa9b3e00640 1 -- 172.21.15.32:0/3560618956 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b415cdf0 con 0x7fa9b40cd580 2024-04-17T09:54:04.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.254+0000 7fa9b3e00640 1 --2- 172.21.15.32:0/3560618956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b40cd580 0x7fa9b4072330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa9b406ff00 tx=0x7fa9ac01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16722d5dea388fd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:04.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.254+0000 7fa9b2a00640 1 -- 172.21.15.32:0/3560618956 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac0b5070 con 0x7fa9b40cd580 2024-04-17T09:54:04.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.254+0000 7fa9b2a00640 1 -- 172.21.15.32:0/3560618956 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9ac0a8970 con 0x7fa9b40cd580 2024-04-17T09:54:04.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.254+0000 7fa9b2a00640 1 -- 172.21.15.32:0/3560618956 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac0a8d40 con 0x7fa9b40cd580 2024-04-17T09:54:04.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.254+0000 7fa9ba800640 1 -- 172.21.15.32:0/3560618956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b40cd580 msgr2=0x7fa9b4072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:04.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.254+0000 7fa9ba800640 1 --2- 172.21.15.32:0/3560618956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b40cd580 0x7fa9b4072330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa9b406ff00 tx=0x7fa9ac01ec70 comp rx=0 tx=0).stop 2024-04-17T09:54:04.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.254+0000 7fa9ba800640 1 -- 172.21.15.32:0/3560618956 shutdown_connections 2024-04-17T09:54:04.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.254+0000 7fa9ba800640 1 --2- 172.21.15.32:0/3560618956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b40cd580 0x7fa9b4072330 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.254+0000 7fa9ba800640 1 --2- 172.21.15.32:0/3560618956 >> v2:172.21.15.144:3300/0 conn(0x7fa9b4145f30 0x7fa9b4071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.254+0000 7fa9ba800640 1 -- 172.21.15.32:0/3560618956 >> 172.21.15.32:0/3560618956 conn(0x7fa9b406ba30 msgr2=0x7fa9b406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:04.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.254+0000 7fa9ba800640 1 -- 172.21.15.32:0/3560618956 shutdown_connections 2024-04-17T09:54:04.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.254+0000 7fa9ba800640 1 -- 172.21.15.32:0/3560618956 wait complete. 2024-04-17T09:54:04.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.254+0000 7fa9ba800640 1 Processor -- start 2024-04-17T09:54:04.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.254+0000 7fa9ba800640 1 -- start start 2024-04-17T09:54:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.254+0000 7fa9ba800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b40cd580 0x7fa9b41443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.254+0000 7fa9ba800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b4145f30 0x7fa9b40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.254+0000 7fa9ba800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9b415d8d0 0x7fa9b40b5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.254+0000 7fa9ba800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b415cb90 con 0x7fa9b40cd580 2024-04-17T09:54:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.254+0000 7fa9ba800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b415d440 con 0x7fa9b4145f30 2024-04-17T09:54:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.254+0000 7fa9ba800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa9b41601d0 con 0x7fa9b415d8d0 2024-04-17T09:54:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.254+0000 7fa9b3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b40cd580 0x7fa9b41443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.254+0000 7fa9b3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b40cd580 0x7fa9b41443a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33144/0 (socket says 172.21.15.32:33144) 2024-04-17T09:54:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.254+0000 7fa9b3400640 1 -- 172.21.15.32:0/2422453916 learned_addr learned my addr 172.21.15.32:0/2422453916 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.258+0000 7fa9aaa00640 1 --2- 172.21.15.32:0/2422453916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b4145f30 0x7fa9b40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.258+0000 7fa9b3e00640 1 --2- 172.21.15.32:0/2422453916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9b415d8d0 0x7fa9b40b5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.258+0000 7fa9b3400640 1 -- 172.21.15.32:0/2422453916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9b415d8d0 msgr2=0x7fa9b40b5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.258+0000 7fa9b3400640 1 --2- 172.21.15.32:0/2422453916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9b415d8d0 0x7fa9b40b5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.258+0000 7fa9b3400640 1 -- 172.21.15.32:0/2422453916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b4145f30 msgr2=0x7fa9b40b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.258+0000 7fa9b3400640 1 --2- 172.21.15.32:0/2422453916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b4145f30 0x7fa9b40b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.258+0000 7fa9b3400640 1 -- 172.21.15.32:0/2422453916 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b40c9c10 con 0x7fa9b40cd580 2024-04-17T09:54:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.258+0000 7fa9b3400640 1 --2- 172.21.15.32:0/2422453916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b40cd580 0x7fa9b41443a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400d800 tx=0x7fa9a400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.258+0000 7fa9abe00640 1 -- 172.21.15.32:0/2422453916 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a400b6c0 con 0x7fa9b40cd580 2024-04-17T09:54:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.258+0000 7fa9abe00640 1 -- 172.21.15.32:0/2422453916 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a4019070 con 0x7fa9b40cd580 2024-04-17T09:54:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.258+0000 7fa9ba800640 1 -- 172.21.15.32:0/2422453916 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b40c9ef0 con 0x7fa9b40cd580 2024-04-17T09:54:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.258+0000 7fa9abe00640 1 -- 172.21.15.32:0/2422453916 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a400ba60 con 0x7fa9b40cd580 2024-04-17T09:54:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.258+0000 7fa9ba800640 1 -- 172.21.15.32:0/2422453916 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b40ca4b0 con 0x7fa9b40cd580 2024-04-17T09:54:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.258+0000 7fa9ba800640 1 -- 172.21.15.32:0/2422453916 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b40ca8e0 con 0x7fa9b40cd580 2024-04-17T09:54:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.258+0000 7fa9abe00640 1 -- 172.21.15.32:0/2422453916 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa9a400bc00 con 0x7fa9b40cd580 2024-04-17T09:54:04.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.258+0000 7fa9abe00640 1 --2- 172.21.15.32:0/2422453916 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa990068380 0x7fa99006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:04.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.258+0000 7fa9abe00640 1 -- 172.21.15.32:0/2422453916 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fa9a4086f30 con 0x7fa9b40cd580 2024-04-17T09:54:04.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.258+0000 7fa9aaa00640 1 --2- 172.21.15.32:0/2422453916 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa990068380 0x7fa99006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:04.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.262+0000 7fa9aaa00640 1 --2- 172.21.15.32:0/2422453916 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa990068380 0x7fa99006a840 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4136310 tx=0x7fa99c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:04.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.262+0000 7fa9abe00640 1 -- 172.21.15.32:0/2422453916 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa9a4057470 con 0x7fa9b40cd580 2024-04-17T09:54:04.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.466+0000 7fa9ba800640 1 -- 172.21.15.32:0/2422453916 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9b40c9960 con 0x7fa9b40cd580 2024-04-17T09:54:04.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.466+0000 7fa9abe00640 1 -- 172.21.15.32:0/2422453916 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5193 (secure 0 0 0) 0x7fa9a4056d70 con 0x7fa9b40cd580 2024-04-17T09:54:04.468 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:04.468 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/441201886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":441201886},{"type":"v1","addr":"172.21.15.144:6834","nonce":441201886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1066277472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1066277472},{"type":"v1","addr":"172.21.15.32:6836","nonce":1066277472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/201410023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":201410023},{"type":"v1","addr":"172.21.15.32:6837","nonce":201410023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:53:13.172187+0000","modified":"2024-04-17T09:53:16.188325+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/3332924005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3332924005},{"type":"v1","addr":"172.21.15.144:6835","nonce":3332924005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-17T09:54:04.469 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2024-04-17T09:54:04.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.470+0000 7fa9ba800640 1 -- 172.21.15.32:0/2422453916 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa990068380 msgr2=0x7fa99006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:04.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.470+0000 7fa9ba800640 1 --2- 172.21.15.32:0/2422453916 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa990068380 0x7fa99006a840 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4136310 tx=0x7fa99c009040 comp rx=0 tx=0).stop 2024-04-17T09:54:04.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.470+0000 7fa9ba800640 1 -- 172.21.15.32:0/2422453916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b40cd580 msgr2=0x7fa9b41443a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:04.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.470+0000 7fa9ba800640 1 --2- 172.21.15.32:0/2422453916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b40cd580 0x7fa9b41443a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400d800 tx=0x7fa9a400dcc0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.470+0000 7fa9ba800640 1 -- 172.21.15.32:0/2422453916 shutdown_connections 2024-04-17T09:54:04.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.470+0000 7fa9ba800640 1 --2- 172.21.15.32:0/2422453916 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa990068380 0x7fa99006a840 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.470+0000 7fa9ba800640 1 --2- 172.21.15.32:0/2422453916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9b415d8d0 0x7fa9b40b5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.470+0000 7fa9ba800640 1 --2- 172.21.15.32:0/2422453916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b4145f30 0x7fa9b40b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.470+0000 7fa9ba800640 1 --2- 172.21.15.32:0/2422453916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b40cd580 0x7fa9b41443a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.470+0000 7fa9ba800640 1 -- 172.21.15.32:0/2422453916 >> 172.21.15.32:0/2422453916 conn(0x7fa9b406ba30 msgr2=0x7fa9b415f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:04.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.470+0000 7fa9ba800640 1 -- 172.21.15.32:0/2422453916 shutdown_connections 2024-04-17T09:54:04.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.470+0000 7fa9ba800640 1 -- 172.21.15.32:0/2422453916 wait complete. 2024-04-17T09:54:04.491 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:54:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.630+0000 7f8f95800640 1 Processor -- start 2024-04-17T09:54:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.630+0000 7f8f95800640 1 -- start start 2024-04-17T09:54:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.634+0000 7f8f95800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f90072500 0x7f8f900cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.634+0000 7f8f95800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f9015d8d0 0x7f8f900cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.634+0000 7f8f95800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f900cdac0 con 0x7f8f90145f30 2024-04-17T09:54:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.634+0000 7f8f95800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8f9006f400 con 0x7f8f9015d8d0 2024-04-17T09:54:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.634+0000 7f8f95800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8f90146310 con 0x7f8f90072500 2024-04-17T09:54:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.634+0000 7f8f8f400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f9015d8d0 0x7f8f900cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.634+0000 7f8f8ea00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8f90145f30 0x7f8f90071df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48948/0 (socket says 172.21.15.32:48948) 2024-04-17T09:54:04.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.634+0000 7f8f8ea00640 1 -- 172.21.15.32:0/2360712236 learned_addr learned my addr 172.21.15.32:0/2360712236 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:04.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.634+0000 7f8f87e00640 1 --2- 172.21.15.32:0/2360712236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f90072500 0x7f8f900cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:04.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.634+0000 7f8f8f400640 1 -- 172.21.15.32:0/2360712236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f90072500 msgr2=0x7f8f900cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:04.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.634+0000 7f8f8f400640 1 --2- 172.21.15.32:0/2360712236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f90072500 0x7f8f900cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.634+0000 7f8f8f400640 1 -- 172.21.15.32:0/2360712236 >> v1:172.21.15.32:6789/0 conn(0x7f8f90145f30 legacy=0x7f8f90071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:04.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.634+0000 7f8f8f400640 1 -- 172.21.15.32:0/2360712236 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9015cdf0 con 0x7f8f9015d8d0 2024-04-17T09:54:04.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.634+0000 7f8f8e000640 1 -- 172.21.15.32:0/2360712236 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1316533241 0 0) 0x7f8f900cdac0 con 0x7f8f90145f30 2024-04-17T09:54:04.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.638+0000 7f8f8f400640 1 --2- 172.21.15.32:0/2360712236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f9015d8d0 0x7f8f900cd3b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009d90 tx=0x7f8f8801f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337f525a89f2d5c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:04.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.638+0000 7f8f8e000640 1 -- 172.21.15.32:0/2360712236 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f880b4070 con 0x7f8f9015d8d0 2024-04-17T09:54:04.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.638+0000 7f8f8e000640 1 -- 172.21.15.32:0/2360712236 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f880af600 con 0x7f8f9015d8d0 2024-04-17T09:54:04.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.638+0000 7f8f8e000640 1 -- 172.21.15.32:0/2360712236 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f880af9d0 con 0x7f8f9015d8d0 2024-04-17T09:54:04.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.638+0000 7f8f95800640 1 -- 172.21.15.32:0/2360712236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f9015d8d0 msgr2=0x7f8f900cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:04.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.638+0000 7f8f95800640 1 --2- 172.21.15.32:0/2360712236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f9015d8d0 0x7f8f900cd3b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009d90 tx=0x7f8f8801f150 comp rx=0 tx=0).stop 2024-04-17T09:54:04.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.638+0000 7f8f95800640 1 -- 172.21.15.32:0/2360712236 shutdown_connections 2024-04-17T09:54:04.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.638+0000 7f8f95800640 1 --2- 172.21.15.32:0/2360712236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f9015d8d0 0x7f8f900cd3b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.638+0000 7f8f95800640 1 --2- 172.21.15.32:0/2360712236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f90072500 0x7f8f900cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.638+0000 7f8f95800640 1 -- 172.21.15.32:0/2360712236 >> 172.21.15.32:0/2360712236 conn(0x7f8f9006ba30 msgr2=0x7f8f9006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:04.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.638+0000 7f8f95800640 1 -- 172.21.15.32:0/2360712236 shutdown_connections 2024-04-17T09:54:04.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.638+0000 7f8f95800640 1 -- 172.21.15.32:0/2360712236 wait complete. 2024-04-17T09:54:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.638+0000 7f8f95800640 1 Processor -- start 2024-04-17T09:54:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.638+0000 7f8f95800640 1 -- start start 2024-04-17T09:54:04.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.638+0000 7f8f95800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f90072500 0x7f8f9013dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.638+0000 7f8f95800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f90145f30 0x7f8f900b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.638+0000 7f8f95800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f9015d8d0 0x7f8f900b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.638+0000 7f8f95800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9015cb90 con 0x7f8f90145f30 2024-04-17T09:54:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.638+0000 7f8f95800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9015d440 con 0x7f8f9015d8d0 2024-04-17T09:54:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.638+0000 7f8f95800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8f901601d0 con 0x7f8f90072500 2024-04-17T09:54:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.638+0000 7f8f87e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f90145f30 0x7f8f900b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.642+0000 7f8f87e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f90145f30 0x7f8f900b4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33160/0 (socket says 172.21.15.32:33160) 2024-04-17T09:54:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.642+0000 7f8f87e00640 1 -- 172.21.15.32:0/244948019 learned_addr learned my addr 172.21.15.32:0/244948019 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.642+0000 7f8f8ea00640 1 --2- 172.21.15.32:0/244948019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f90072500 0x7f8f9013dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.642+0000 7f8f8f400640 1 --2- 172.21.15.32:0/244948019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f9015d8d0 0x7f8f900b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.642+0000 7f8f87e00640 1 -- 172.21.15.32:0/244948019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f90072500 msgr2=0x7f8f9013dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.642+0000 7f8f87e00640 1 --2- 172.21.15.32:0/244948019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f90072500 0x7f8f9013dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.642+0000 7f8f87e00640 1 -- 172.21.15.32:0/244948019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f9015d8d0 msgr2=0x7f8f900b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:04.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.642+0000 7f8f87e00640 1 --2- 172.21.15.32:0/244948019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f9015d8d0 0x7f8f900b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.642+0000 7f8f87e00640 1 -- 172.21.15.32:0/244948019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f90140240 con 0x7f8f90145f30 2024-04-17T09:54:04.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.642+0000 7f8f87e00640 1 --2- 172.21.15.32:0/244948019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f90145f30 0x7f8f900b4070 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800d800 tx=0x7f8f7800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:04.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.642+0000 7f8f86a00640 1 -- 172.21.15.32:0/244948019 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7800b6c0 con 0x7f8f90145f30 2024-04-17T09:54:04.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.642+0000 7f8f86a00640 1 -- 172.21.15.32:0/244948019 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f78019070 con 0x7f8f90145f30 2024-04-17T09:54:04.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.642+0000 7f8f86a00640 1 -- 172.21.15.32:0/244948019 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7800ba60 con 0x7f8f90145f30 2024-04-17T09:54:04.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.642+0000 7f8f95800640 1 -- 172.21.15.32:0/244948019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f90140520 con 0x7f8f90145f30 2024-04-17T09:54:04.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.642+0000 7f8f95800640 1 -- 172.21.15.32:0/244948019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f90140ae0 con 0x7f8f90145f30 2024-04-17T09:54:04.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.642+0000 7f8f95800640 1 -- 172.21.15.32:0/244948019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f90068a90 con 0x7f8f90145f30 2024-04-17T09:54:04.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.642+0000 7f8f86a00640 1 -- 172.21.15.32:0/244948019 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8f7800bc00 con 0x7f8f90145f30 2024-04-17T09:54:04.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.642+0000 7f8f86a00640 1 --2- 172.21.15.32:0/244948019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8f6c068380 0x7f8f6c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:04.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.646+0000 7f8f86a00640 1 -- 172.21.15.32:0/244948019 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f8f78086f10 con 0x7f8f90145f30 2024-04-17T09:54:04.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.646+0000 7f8f8ea00640 1 --2- 172.21.15.32:0/244948019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8f6c068380 0x7f8f6c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:04.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.646+0000 7f8f8ea00640 1 --2- 172.21.15.32:0/244948019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8f6c068380 0x7f8f6c06a840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8f80007fd0 tx=0x7f8f80007d60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:04.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.650+0000 7f8f86a00640 1 -- 172.21.15.32:0/244948019 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f8f780573b0 con 0x7f8f90145f30 2024-04-17T09:54:04.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.814+0000 7f8f95800640 1 -- 172.21.15.32:0/244948019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f9006cdd0 con 0x7f8f90145f30 2024-04-17T09:54:04.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.814+0000 7f8f86a00640 1 -- 172.21.15.32:0/244948019 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16866 (secure 0 0 0) 0x7f8f78056cb0 con 0x7f8f90145f30 2024-04-17T09:54:04.815 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:04.815 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:54.932571+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:54:04.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.818+0000 7f8f95800640 1 -- 172.21.15.32:0/244948019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8f6c068380 msgr2=0x7f8f6c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:04.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.818+0000 7f8f95800640 1 --2- 172.21.15.32:0/244948019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8f6c068380 0x7f8f6c06a840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8f80007fd0 tx=0x7f8f80007d60 comp rx=0 tx=0).stop 2024-04-17T09:54:04.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.818+0000 7f8f95800640 1 -- 172.21.15.32:0/244948019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f90145f30 msgr2=0x7f8f900b4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:04.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.818+0000 7f8f95800640 1 --2- 172.21.15.32:0/244948019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f90145f30 0x7f8f900b4070 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800d800 tx=0x7f8f7800dcc0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.818+0000 7f8f95800640 1 -- 172.21.15.32:0/244948019 shutdown_connections 2024-04-17T09:54:04.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.818+0000 7f8f95800640 1 --2- 172.21.15.32:0/244948019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8f6c068380 0x7f8f6c06a840 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.818+0000 7f8f95800640 1 --2- 172.21.15.32:0/244948019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f9015d8d0 0x7f8f900b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.818+0000 7f8f95800640 1 --2- 172.21.15.32:0/244948019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f90145f30 0x7f8f900b4070 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.818+0000 7f8f95800640 1 --2- 172.21.15.32:0/244948019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f90072500 0x7f8f9013dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.818+0000 7f8f95800640 1 -- 172.21.15.32:0/244948019 >> 172.21.15.32:0/244948019 conn(0x7f8f9006ba30 msgr2=0x7f8f900b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:04.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.818+0000 7f8f95800640 1 -- 172.21.15.32:0/244948019 shutdown_connections 2024-04-17T09:54:04.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.818+0000 7f8f95800640 1 -- 172.21.15.32:0/244948019 wait complete. 2024-04-17T09:54:04.836 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T09:54:04.836 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:54:04.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.974+0000 7f0ca6a00640 1 Processor -- start 2024-04-17T09:54:04.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.974+0000 7f0ca6a00640 1 -- start start 2024-04-17T09:54:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.978+0000 7f0ca6a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ca015c5c0 0x7f0ca00cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.978+0000 7f0ca6a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0ca009b100 0x7f0ca00b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.978+0000 7f0ca6a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ca00d05e0 con 0x7f0ca00d00b0 2024-04-17T09:54:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.978+0000 7f0ca6a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0ca00d11f0 con 0x7f0ca015c5c0 2024-04-17T09:54:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.978+0000 7f0ca6a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0ca00d08c0 con 0x7f0ca009b100 2024-04-17T09:54:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.978+0000 7f0c9fe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ca015c5c0 0x7f0ca00cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.978+0000 7f0c9fe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ca015c5c0 0x7f0ca00cde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33170/0 (socket says 172.21.15.32:33170) 2024-04-17T09:54:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.978+0000 7f0c9fe00640 1 -- 172.21.15.32:0/2102710617 learned_addr learned my addr 172.21.15.32:0/2102710617 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.978+0000 7f0ca4a00640 1 --2- 172.21.15.32:0/2102710617 >> v2:172.21.15.144:3300/0 conn(0x7f0ca009b100 0x7f0ca00b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.978+0000 7f0c9fe00640 1 -- 172.21.15.32:0/2102710617 >> v2:172.21.15.144:3300/0 conn(0x7f0ca009b100 msgr2=0x7f0ca00b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:04.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.978+0000 7f0c9fe00640 1 --2- 172.21.15.32:0/2102710617 >> v2:172.21.15.144:3300/0 conn(0x7f0ca009b100 0x7f0ca00b31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.978+0000 7f0c9fe00640 1 -- 172.21.15.32:0/2102710617 >> v1:172.21.15.144:6789/0 conn(0x7f0ca00d00b0 legacy=0x7f0ca009a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:04.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.978+0000 7f0c9fe00640 1 -- 172.21.15.32:0/2102710617 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca00d0ba0 con 0x7f0ca015c5c0 2024-04-17T09:54:04.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.982+0000 7f0c9fe00640 1 --2- 172.21.15.32:0/2102710617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ca015c5c0 0x7f0ca00cde50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0161460 tx=0x7f0c9001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ce646fed0f61da server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:04.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.982+0000 7f0c9ea00640 1 -- 172.21.15.32:0/2102710617 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c900b5070 con 0x7f0ca015c5c0 2024-04-17T09:54:04.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.982+0000 7f0c9ea00640 1 -- 172.21.15.32:0/2102710617 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c900a8a10 con 0x7f0ca015c5c0 2024-04-17T09:54:04.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.982+0000 7f0c9ea00640 1 -- 172.21.15.32:0/2102710617 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c900a8de0 con 0x7f0ca015c5c0 2024-04-17T09:54:04.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.982+0000 7f0ca6a00640 1 -- 172.21.15.32:0/2102710617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ca015c5c0 msgr2=0x7f0ca00cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:04.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.982+0000 7f0ca6a00640 1 --2- 172.21.15.32:0/2102710617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ca015c5c0 0x7f0ca00cde50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0161460 tx=0x7f0c9001ed10 comp rx=0 tx=0).stop 2024-04-17T09:54:04.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.982+0000 7f0ca6a00640 1 -- 172.21.15.32:0/2102710617 shutdown_connections 2024-04-17T09:54:04.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.982+0000 7f0ca6a00640 1 --2- 172.21.15.32:0/2102710617 >> v2:172.21.15.144:3300/0 conn(0x7f0ca009b100 0x7f0ca00b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.982+0000 7f0ca6a00640 1 --2- 172.21.15.32:0/2102710617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ca015c5c0 0x7f0ca00cde50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.982+0000 7f0ca6a00640 1 -- 172.21.15.32:0/2102710617 >> 172.21.15.32:0/2102710617 conn(0x7f0ca00b3fd0 msgr2=0x7f0ca00b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:04.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.982+0000 7f0ca6a00640 1 -- 172.21.15.32:0/2102710617 shutdown_connections 2024-04-17T09:54:04.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.982+0000 7f0ca6a00640 1 -- 172.21.15.32:0/2102710617 wait complete. 2024-04-17T09:54:04.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.982+0000 7f0ca6a00640 1 Processor -- start 2024-04-17T09:54:04.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.982+0000 7f0ca6a00640 1 -- start start 2024-04-17T09:54:04.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0ca6a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ca009b100 0x7f0ca01508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:04.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0ca6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ca00d00b0 0x7f0ca006cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:04.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0ca6a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ca015c5c0 0x7f0ca006d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:04.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0ca6a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca00ce510 con 0x7f0ca00d00b0 2024-04-17T09:54:04.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0ca6a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca00ce390 con 0x7f0ca015c5c0 2024-04-17T09:54:04.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0ca6a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0ca015c9a0 con 0x7f0ca009b100 2024-04-17T09:54:04.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0c9f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ca00d00b0 0x7f0ca006cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:04.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0c9f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ca00d00b0 0x7f0ca006cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33184/0 (socket says 172.21.15.32:33184) 2024-04-17T09:54:04.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0c9f400640 1 -- 172.21.15.32:0/1996179612 learned_addr learned my addr 172.21.15.32:0/1996179612 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:04.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0c9fe00640 1 --2- 172.21.15.32:0/1996179612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ca009b100 0x7f0ca01508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:04.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0ca4a00640 1 --2- 172.21.15.32:0/1996179612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ca015c5c0 0x7f0ca006d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:04.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0c9f400640 1 -- 172.21.15.32:0/1996179612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ca009b100 msgr2=0x7f0ca01508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:04.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0c9f400640 1 --2- 172.21.15.32:0/1996179612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ca009b100 0x7f0ca01508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0c9f400640 1 -- 172.21.15.32:0/1996179612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ca015c5c0 msgr2=0x7f0ca006d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:04.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0c9f400640 1 --2- 172.21.15.32:0/1996179612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ca015c5c0 0x7f0ca006d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:04.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0c9f400640 1 -- 172.21.15.32:0/1996179612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca00c6080 con 0x7f0ca00d00b0 2024-04-17T09:54:04.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0ca4a00640 1 --2- 172.21.15.32:0/1996179612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ca015c5c0 0x7f0ca006d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:04.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0c9f400640 1 --2- 172.21.15.32:0/1996179612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ca00d00b0 0x7f0ca006cb90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00c420 tx=0x7f0c8c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:04.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0c97e00640 1 -- 172.21.15.32:0/1996179612 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c0052d0 con 0x7f0ca00d00b0 2024-04-17T09:54:04.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0c97e00640 1 -- 172.21.15.32:0/1996179612 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c8c005470 con 0x7f0ca00d00b0 2024-04-17T09:54:04.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0c97e00640 1 -- 172.21.15.32:0/1996179612 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c007260 con 0x7f0ca00d00b0 2024-04-17T09:54:04.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0ca6a00640 1 -- 172.21.15.32:0/1996179612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca00c6360 con 0x7f0ca00d00b0 2024-04-17T09:54:04.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0ca6a00640 1 -- 172.21.15.32:0/1996179612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca00c6920 con 0x7f0ca00d00b0 2024-04-17T09:54:04.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0ca6a00640 1 -- 172.21.15.32:0/1996179612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ca0068a90 con 0x7f0ca00d00b0 2024-04-17T09:54:04.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.986+0000 7f0c97e00640 1 -- 172.21.15.32:0/1996179612 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0c8c007400 con 0x7f0ca00d00b0 2024-04-17T09:54:04.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.990+0000 7f0c97e00640 1 --2- 172.21.15.32:0/1996179612 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0c800683b0 0x7f0c8006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:04.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.990+0000 7f0c97e00640 1 -- 172.21.15.32:0/1996179612 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f0c8c085e60 con 0x7f0ca00d00b0 2024-04-17T09:54:04.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.990+0000 7f0c9fe00640 1 --2- 172.21.15.32:0/1996179612 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0c800683b0 0x7f0c8006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:04.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.990+0000 7f0c9fe00640 1 --2- 172.21.15.32:0/1996179612 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0c800683b0 0x7f0c8006a870 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0c900b2000 tx=0x7f0c90009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:04.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:04.994+0000 7f0c97e00640 1 -- 172.21.15.32:0/1996179612 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0c8c055d40 con 0x7f0ca00d00b0 2024-04-17T09:54:05.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.194+0000 7f0ca6a00640 1 -- 172.21.15.32:0/1996179612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ca00b8140 con 0x7f0ca00d00b0 2024-04-17T09:54:05.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.194+0000 7f0c97e00640 1 -- 172.21.15.32:0/1996179612 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5193 (secure 0 0 0) 0x7f0c8c05a040 con 0x7f0ca00d00b0 2024-04-17T09:54:05.194 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:05.194 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/441201886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":441201886},{"type":"v1","addr":"172.21.15.144:6834","nonce":441201886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1066277472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1066277472},{"type":"v1","addr":"172.21.15.32:6836","nonce":1066277472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/201410023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":201410023},{"type":"v1","addr":"172.21.15.32:6837","nonce":201410023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:53:13.172187+0000","modified":"2024-04-17T09:53:16.188325+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/3332924005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3332924005},{"type":"v1","addr":"172.21.15.144:6835","nonce":3332924005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-17T09:54:05.194 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2024-04-17T09:54:05.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.194+0000 7f0ca6a00640 1 -- 172.21.15.32:0/1996179612 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0c800683b0 msgr2=0x7f0c8006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:05.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.194+0000 7f0ca6a00640 1 --2- 172.21.15.32:0/1996179612 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0c800683b0 0x7f0c8006a870 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0c900b2000 tx=0x7f0c90009040 comp rx=0 tx=0).stop 2024-04-17T09:54:05.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.194+0000 7f0ca6a00640 1 -- 172.21.15.32:0/1996179612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ca00d00b0 msgr2=0x7f0ca006cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:05.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.194+0000 7f0ca6a00640 1 --2- 172.21.15.32:0/1996179612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ca00d00b0 0x7f0ca006cb90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00c420 tx=0x7f0c8c00c8e0 comp rx=0 tx=0).stop 2024-04-17T09:54:05.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.194+0000 7f0ca6a00640 1 -- 172.21.15.32:0/1996179612 shutdown_connections 2024-04-17T09:54:05.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.194+0000 7f0ca6a00640 1 --2- 172.21.15.32:0/1996179612 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0c800683b0 0x7f0c8006a870 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:05.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.194+0000 7f0ca6a00640 1 --2- 172.21.15.32:0/1996179612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ca015c5c0 0x7f0ca006d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:05.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.194+0000 7f0ca6a00640 1 --2- 172.21.15.32:0/1996179612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ca00d00b0 0x7f0ca006cb90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:05.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.194+0000 7f0ca6a00640 1 --2- 172.21.15.32:0/1996179612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ca009b100 0x7f0ca01508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:05.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.194+0000 7f0ca6a00640 1 -- 172.21.15.32:0/1996179612 >> 172.21.15.32:0/1996179612 conn(0x7f0ca00b3fd0 msgr2=0x7f0ca00cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:05.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.194+0000 7f0ca6a00640 1 -- 172.21.15.32:0/1996179612 shutdown_connections 2024-04-17T09:54:05.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.194+0000 7f0ca6a00640 1 -- 172.21.15.32:0/1996179612 wait complete. 2024-04-17T09:54:05.216 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:54:05.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.358+0000 7f52c3200640 1 Processor -- start 2024-04-17T09:54:05.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.358+0000 7f52c3200640 1 -- start start 2024-04-17T09:54:05.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.358+0000 7f52c3200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f52c40add30 0x7f52c40ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:05.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.358+0000 7f52c3200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52c40af6c0 con 0x7f52c406fd30 2024-04-17T09:54:05.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.358+0000 7f52c3200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52c40af9a0 con 0x7f52c4154d30 2024-04-17T09:54:05.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.358+0000 7f52c3200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f52c40b0740 con 0x7f52c40add30 2024-04-17T09:54:05.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.358+0000 7f52c1e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f52c406fd30 0x7f52c415f5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48960/0 (socket says 172.21.15.32:48960) 2024-04-17T09:54:05.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.358+0000 7f52c1e00640 1 -- 172.21.15.32:0/3220383924 learned_addr learned my addr 172.21.15.32:0/3220383924 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:05.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.358+0000 7f52c1400640 1 --2- 172.21.15.32:0/3220383924 >> v2:172.21.15.144:3300/0 conn(0x7f52c40add30 0x7f52c40ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:05.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.358+0000 7f52c0a00640 1 -- 172.21.15.32:0/3220383924 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3426920066 0 0) 0x7f52c40af6c0 con 0x7f52c406fd30 2024-04-17T09:54:05.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.358+0000 7f52c0a00640 1 -- 172.21.15.32:0/3220383924 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52a8003530 con 0x7f52c406fd30 2024-04-17T09:54:05.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.358+0000 7f52c0a00640 1 -- 172.21.15.32:0/3220383924 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2341755749 0 0) 0x7f52c40af9a0 con 0x7f52c4154d30 2024-04-17T09:54:05.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.358+0000 7f52c1400640 1 -- 172.21.15.32:0/3220383924 >> v1:172.21.15.144:6789/0 conn(0x7f52c4154d30 legacy=0x7f52c41551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:05.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.358+0000 7f52c1400640 1 -- 172.21.15.32:0/3220383924 >> v1:172.21.15.32:6789/0 conn(0x7f52c406fd30 legacy=0x7f52c415f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:05.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.358+0000 7f52c1400640 1 -- 172.21.15.32:0/3220383924 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c40afc80 con 0x7f52c40add30 2024-04-17T09:54:05.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.362+0000 7f52c1400640 1 --2- 172.21.15.32:0/3220383924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52c40add30 0x7f52c40ae1c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f52b00097c0 tx=0x7f52b001c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd80cd94b072823d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:05.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.362+0000 7f52c0a00640 1 -- 172.21.15.32:0/3220383924 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b00b0070 con 0x7f52c40add30 2024-04-17T09:54:05.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.362+0000 7f52c0a00640 1 -- 172.21.15.32:0/3220383924 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b00ab400 con 0x7f52c40add30 2024-04-17T09:54:05.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.362+0000 7f52c0a00640 1 -- 172.21.15.32:0/3220383924 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4001600 con 0x7f52c40add30 2024-04-17T09:54:05.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.362+0000 7f52c3200640 1 -- 172.21.15.32:0/3220383924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52c40add30 msgr2=0x7f52c40ae1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:05.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.362+0000 7f52c3200640 1 --2- 172.21.15.32:0/3220383924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52c40add30 0x7f52c40ae1c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f52b00097c0 tx=0x7f52b001c2e0 comp rx=0 tx=0).stop 2024-04-17T09:54:05.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.362+0000 7f52c3200640 1 -- 172.21.15.32:0/3220383924 shutdown_connections 2024-04-17T09:54:05.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.362+0000 7f52c3200640 1 --2- 172.21.15.32:0/3220383924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52c40add30 0x7f52c40ae1c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:05.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.362+0000 7f52c3200640 1 -- 172.21.15.32:0/3220383924 >> 172.21.15.32:0/3220383924 conn(0x7f52c4073380 msgr2=0x7f52c406ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:05.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.362+0000 7f52c3200640 1 -- 172.21.15.32:0/3220383924 shutdown_connections 2024-04-17T09:54:05.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.362+0000 7f52c3200640 1 -- 172.21.15.32:0/3220383924 wait complete. 2024-04-17T09:54:05.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52c3200640 1 Processor -- start 2024-04-17T09:54:05.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52c3200640 1 -- start start 2024-04-17T09:54:05.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52c3200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52c406fd30 0x7f52c41562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:05.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52c3200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52c40add30 0x7f52c4162180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:05.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52c3200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52c4154d30 0x7f52c414d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:05.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52c3200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f52c4155a70 con 0x7f52c40add30 2024-04-17T09:54:05.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52c3200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f52c41558f0 con 0x7f52c406fd30 2024-04-17T09:54:05.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52c3200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f52c40ae9b0 con 0x7f52c4154d30 2024-04-17T09:54:05.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52c1400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52c40add30 0x7f52c4162180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:05.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52c1400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52c40add30 0x7f52c4162180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33196/0 (socket says 172.21.15.32:33196) 2024-04-17T09:54:05.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52c1400640 1 -- 172.21.15.32:0/1996455801 learned_addr learned my addr 172.21.15.32:0/1996455801 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:05.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52c1e00640 1 --2- 172.21.15.32:0/1996455801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52c406fd30 0x7f52c41562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:05.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52c2800640 1 --2- 172.21.15.32:0/1996455801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52c4154d30 0x7f52c414d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:05.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52c1400640 1 -- 172.21.15.32:0/1996455801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52c4154d30 msgr2=0x7f52c414d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:05.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52c1400640 1 --2- 172.21.15.32:0/1996455801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52c4154d30 0x7f52c414d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:05.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52c1400640 1 -- 172.21.15.32:0/1996455801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52c406fd30 msgr2=0x7f52c41562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:05.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52c1400640 1 --2- 172.21.15.32:0/1996455801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52c406fd30 0x7f52c41562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:05.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52c1400640 1 -- 172.21.15.32:0/1996455801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c4156bc0 con 0x7f52c40add30 2024-04-17T09:54:05.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52c1400640 1 --2- 172.21.15.32:0/1996455801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52c40add30 0x7f52c4162180 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f52b00097c0 tx=0x7f52b0019080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:05.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52ba000640 1 -- 172.21.15.32:0/1996455801 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b00b0040 con 0x7f52c40add30 2024-04-17T09:54:05.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52ba000640 1 -- 172.21.15.32:0/1996455801 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b0004030 con 0x7f52c40add30 2024-04-17T09:54:05.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52ba000640 1 -- 172.21.15.32:0/1996455801 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b00043e0 con 0x7f52c40add30 2024-04-17T09:54:05.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52c3200640 1 -- 172.21.15.32:0/1996455801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c41540b0 con 0x7f52c40add30 2024-04-17T09:54:05.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52c3200640 1 -- 172.21.15.32:0/1996455801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c4154670 con 0x7f52c40add30 2024-04-17T09:54:05.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.366+0000 7f52c3200640 1 -- 172.21.15.32:0/1996455801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c404fd00 con 0x7f52c40add30 2024-04-17T09:54:05.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.370+0000 7f52ba000640 1 -- 172.21.15.32:0/1996455801 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f52b00b3070 con 0x7f52c40add30 2024-04-17T09:54:05.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.370+0000 7f52ba000640 1 --2- 172.21.15.32:0/1996455801 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f52a4068380 0x7f52a406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:05.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.370+0000 7f52ba000640 1 -- 172.21.15.32:0/1996455801 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f52b0120ab0 con 0x7f52c40add30 2024-04-17T09:54:05.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.370+0000 7f52c1e00640 1 --2- 172.21.15.32:0/1996455801 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f52a4068380 0x7f52a406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:05.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.370+0000 7f52c1e00640 1 --2- 172.21.15.32:0/1996455801 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f52a4068380 0x7f52a406a840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f52b4000fc0 tx=0x7f52b4001c40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:05.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.374+0000 7f52ba000640 1 -- 172.21.15.32:0/1996455801 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f52b00c2030 con 0x7f52c40add30 2024-04-17T09:54:05.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.574+0000 7f52c3200640 1 -- 172.21.15.32:0/1996455801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52c4161800 con 0x7f52c40add30 2024-04-17T09:54:05.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.574+0000 7f52ba000640 1 -- 172.21.15.32:0/1996455801 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5193 (secure 0 0 0) 0x7f52b00f5e20 con 0x7f52c40add30 2024-04-17T09:54:05.577 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:05.577 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/441201886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":441201886},{"type":"v1","addr":"172.21.15.144:6834","nonce":441201886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1066277472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1066277472},{"type":"v1","addr":"172.21.15.32:6836","nonce":1066277472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/201410023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":201410023},{"type":"v1","addr":"172.21.15.32:6837","nonce":201410023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:53:13.172187+0000","modified":"2024-04-17T09:53:16.188325+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/3332924005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3332924005},{"type":"v1","addr":"172.21.15.144:6835","nonce":3332924005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-17T09:54:05.577 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2024-04-17T09:54:05.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.578+0000 7f52c3200640 1 -- 172.21.15.32:0/1996455801 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f52a4068380 msgr2=0x7f52a406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:05.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.578+0000 7f52c3200640 1 --2- 172.21.15.32:0/1996455801 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f52a4068380 0x7f52a406a840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f52b4000fc0 tx=0x7f52b4001c40 comp rx=0 tx=0).stop 2024-04-17T09:54:05.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.578+0000 7f52c3200640 1 -- 172.21.15.32:0/1996455801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52c40add30 msgr2=0x7f52c4162180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:05.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.578+0000 7f52c3200640 1 --2- 172.21.15.32:0/1996455801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52c40add30 0x7f52c4162180 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f52b00097c0 tx=0x7f52b0019080 comp rx=0 tx=0).stop 2024-04-17T09:54:05.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.578+0000 7f52c3200640 1 -- 172.21.15.32:0/1996455801 shutdown_connections 2024-04-17T09:54:05.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.578+0000 7f52c3200640 1 --2- 172.21.15.32:0/1996455801 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f52a4068380 0x7f52a406a840 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:05.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.578+0000 7f52c3200640 1 --2- 172.21.15.32:0/1996455801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52c4154d30 0x7f52c414d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:05.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.578+0000 7f52c3200640 1 --2- 172.21.15.32:0/1996455801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52c40add30 0x7f52c4162180 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:05.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.578+0000 7f52c3200640 1 --2- 172.21.15.32:0/1996455801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52c406fd30 0x7f52c41562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:05.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.578+0000 7f52c3200640 1 -- 172.21.15.32:0/1996455801 >> 172.21.15.32:0/1996455801 conn(0x7f52c4073380 msgr2=0x7f52c41551e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:05.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.578+0000 7f52c3200640 1 -- 172.21.15.32:0/1996455801 shutdown_connections 2024-04-17T09:54:05.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.578+0000 7f52c3200640 1 -- 172.21.15.32:0/1996455801 wait complete. 2024-04-17T09:54:05.598 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:54:05.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.738+0000 7fab5d600640 1 Processor -- start 2024-04-17T09:54:05.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.738+0000 7fab5d600640 1 -- start start 2024-04-17T09:54:05.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.738+0000 7fab5d600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fab58145f30 0x7fab58071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:05.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.738+0000 7fab5d600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab5815d8d0 0x7fab58072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:05.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.738+0000 7fab5d600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab580cdac0 con 0x7fab580cce70 2024-04-17T09:54:05.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.738+0000 7fab5d600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fab5806f400 con 0x7fab58145f30 2024-04-17T09:54:05.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.738+0000 7fab5d600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fab58146310 con 0x7fab5815d8d0 2024-04-17T09:54:05.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.738+0000 7fab57400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fab580cce70 0x7fab580cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48962/0 (socket says 172.21.15.32:48962) 2024-04-17T09:54:05.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.738+0000 7fab57400640 1 -- 172.21.15.32:0/2592737463 learned_addr learned my addr 172.21.15.32:0/2592737463 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:05.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.738+0000 7fab4e000640 1 --2- 172.21.15.32:0/2592737463 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab5815d8d0 0x7fab58072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:05.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.738+0000 7fab56a00640 1 --2- 172.21.15.32:0/2592737463 >> v2:172.21.15.144:3300/0 conn(0x7fab58145f30 0x7fab58071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:05.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.738+0000 7fab56000640 1 -- 172.21.15.32:0/2592737463 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2222102966 0 0) 0x7fab580cdac0 con 0x7fab580cce70 2024-04-17T09:54:05.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.738+0000 7fab56000640 1 -- 172.21.15.32:0/2592737463 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab3c003530 con 0x7fab580cce70 2024-04-17T09:54:05.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.738+0000 7fab4e000640 1 -- 172.21.15.32:0/2592737463 >> v2:172.21.15.144:3300/0 conn(0x7fab58145f30 msgr2=0x7fab58071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:05.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.738+0000 7fab4e000640 1 --2- 172.21.15.32:0/2592737463 >> v2:172.21.15.144:3300/0 conn(0x7fab58145f30 0x7fab58071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:05.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.738+0000 7fab4e000640 1 -- 172.21.15.32:0/2592737463 >> v1:172.21.15.32:6789/0 conn(0x7fab580cce70 legacy=0x7fab580cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:05.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.738+0000 7fab4e000640 1 -- 172.21.15.32:0/2592737463 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5815cdf0 con 0x7fab5815d8d0 2024-04-17T09:54:05.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.742+0000 7fab4e000640 1 --2- 172.21.15.32:0/2592737463 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab5815d8d0 0x7fab58072330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fab400097c0 tx=0x7fab4001ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7cb7c9f419b74eb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:05.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.742+0000 7fab56000640 1 -- 172.21.15.32:0/2592737463 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab400b5070 con 0x7fab5815d8d0 2024-04-17T09:54:05.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.742+0000 7fab56000640 1 -- 172.21.15.32:0/2592737463 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab400a8990 con 0x7fab5815d8d0 2024-04-17T09:54:05.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.742+0000 7fab56000640 1 -- 172.21.15.32:0/2592737463 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab400a8d60 con 0x7fab5815d8d0 2024-04-17T09:54:05.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.742+0000 7fab5d600640 1 -- 172.21.15.32:0/2592737463 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab5815d8d0 msgr2=0x7fab58072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:05.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.742+0000 7fab5d600640 1 --2- 172.21.15.32:0/2592737463 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab5815d8d0 0x7fab58072330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fab400097c0 tx=0x7fab4001ec90 comp rx=0 tx=0).stop 2024-04-17T09:54:05.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.742+0000 7fab5d600640 1 -- 172.21.15.32:0/2592737463 shutdown_connections 2024-04-17T09:54:05.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.742+0000 7fab5d600640 1 --2- 172.21.15.32:0/2592737463 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab5815d8d0 0x7fab58072330 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:05.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.742+0000 7fab5d600640 1 --2- 172.21.15.32:0/2592737463 >> v2:172.21.15.144:3300/0 conn(0x7fab58145f30 0x7fab58071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:05.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.742+0000 7fab5d600640 1 -- 172.21.15.32:0/2592737463 >> 172.21.15.32:0/2592737463 conn(0x7fab5806ba30 msgr2=0x7fab5806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:05.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.742+0000 7fab5d600640 1 -- 172.21.15.32:0/2592737463 shutdown_connections 2024-04-17T09:54:05.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.742+0000 7fab5d600640 1 -- 172.21.15.32:0/2592737463 wait complete. 2024-04-17T09:54:05.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab5d600640 1 Processor -- start 2024-04-17T09:54:05.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab5d600640 1 -- start start 2024-04-17T09:54:05.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab5d600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab580cce70 0x7fab58140cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:05.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab5d600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab58145f30 0x7fab581f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:05.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab5d600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab5815d8d0 0x7fab581f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:05.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab5d600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fab5815cb90 con 0x7fab580cce70 2024-04-17T09:54:05.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab5d600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fab5815d440 con 0x7fab5815d8d0 2024-04-17T09:54:05.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab5d600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fab581601d0 con 0x7fab58145f30 2024-04-17T09:54:05.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab56a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab580cce70 0x7fab58140cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:05.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab56a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab580cce70 0x7fab58140cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33204/0 (socket says 172.21.15.32:33204) 2024-04-17T09:54:05.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab56a00640 1 -- 172.21.15.32:0/4078596731 learned_addr learned my addr 172.21.15.32:0/4078596731 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:05.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab57400640 1 --2- 172.21.15.32:0/4078596731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab5815d8d0 0x7fab581f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:05.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab4e000640 1 --2- 172.21.15.32:0/4078596731 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab58145f30 0x7fab581f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:05.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab56a00640 1 -- 172.21.15.32:0/4078596731 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab58145f30 msgr2=0x7fab581f31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:05.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab56a00640 1 --2- 172.21.15.32:0/4078596731 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab58145f30 0x7fab581f31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:05.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab56a00640 1 -- 172.21.15.32:0/4078596731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab5815d8d0 msgr2=0x7fab581f36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:05.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab56a00640 1 --2- 172.21.15.32:0/4078596731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab5815d8d0 0x7fab581f36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:05.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab56a00640 1 -- 172.21.15.32:0/4078596731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab580c9310 con 0x7fab580cce70 2024-04-17T09:54:05.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab56a00640 1 --2- 172.21.15.32:0/4078596731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab580cce70 0x7fab58140cc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fab4800e930 tx=0x7fab4800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:05.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab4f400640 1 -- 172.21.15.32:0/4078596731 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48017070 con 0x7fab580cce70 2024-04-17T09:54:05.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab4f400640 1 -- 172.21.15.32:0/4078596731 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab4800ca00 con 0x7fab580cce70 2024-04-17T09:54:05.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab4f400640 1 -- 172.21.15.32:0/4078596731 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4800cdb0 con 0x7fab580cce70 2024-04-17T09:54:05.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab5d600640 1 -- 172.21.15.32:0/4078596731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab580c95f0 con 0x7fab580cce70 2024-04-17T09:54:05.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.746+0000 7fab5d600640 1 -- 172.21.15.32:0/4078596731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab580c9bb0 con 0x7fab580cce70 2024-04-17T09:54:05.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.750+0000 7fab5d600640 1 -- 172.21.15.32:0/4078596731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab58068a90 con 0x7fab580cce70 2024-04-17T09:54:05.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.750+0000 7fab4f400640 1 -- 172.21.15.32:0/4078596731 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fab4801a070 con 0x7fab580cce70 2024-04-17T09:54:05.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.750+0000 7fab4f400640 1 --2- 172.21.15.32:0/4078596731 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab340683d0 0x7fab3406a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:05.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.750+0000 7fab4f400640 1 -- 172.21.15.32:0/4078596731 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fab48086dd0 con 0x7fab580cce70 2024-04-17T09:54:05.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.750+0000 7fab4e000640 1 --2- 172.21.15.32:0/4078596731 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab340683d0 0x7fab3406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:05.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.750+0000 7fab4e000640 1 --2- 172.21.15.32:0/4078596731 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab340683d0 0x7fab3406a890 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fab400b2000 tx=0x7fab40008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:05.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.754+0000 7fab4f400640 1 -- 172.21.15.32:0/4078596731 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fab48057440 con 0x7fab580cce70 2024-04-17T09:54:05.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.922+0000 7fab5d600640 1 -- 172.21.15.32:0/4078596731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab58072f90 con 0x7fab580cce70 2024-04-17T09:54:05.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.922+0000 7fab4f400640 1 -- 172.21.15.32:0/4078596731 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16866 (secure 0 0 0) 0x7fab48056d40 con 0x7fab580cce70 2024-04-17T09:54:05.923 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:05.923 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:53:54.932571+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:53:11.198098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T09:53:12.118982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:54:05.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.922+0000 7fab5d600640 1 -- 172.21.15.32:0/4078596731 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab340683d0 msgr2=0x7fab3406a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:05.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.922+0000 7fab5d600640 1 --2- 172.21.15.32:0/4078596731 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab340683d0 0x7fab3406a890 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fab400b2000 tx=0x7fab40008040 comp rx=0 tx=0).stop 2024-04-17T09:54:05.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.922+0000 7fab5d600640 1 -- 172.21.15.32:0/4078596731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab580cce70 msgr2=0x7fab58140cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:05.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.922+0000 7fab5d600640 1 --2- 172.21.15.32:0/4078596731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab580cce70 0x7fab58140cc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fab4800e930 tx=0x7fab4800edf0 comp rx=0 tx=0).stop 2024-04-17T09:54:05.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.926+0000 7fab5d600640 1 -- 172.21.15.32:0/4078596731 shutdown_connections 2024-04-17T09:54:05.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.926+0000 7fab5d600640 1 --2- 172.21.15.32:0/4078596731 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab340683d0 0x7fab3406a890 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:05.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.926+0000 7fab5d600640 1 --2- 172.21.15.32:0/4078596731 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab58145f30 0x7fab581f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:05.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.926+0000 7fab5d600640 1 --2- 172.21.15.32:0/4078596731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab5815d8d0 0x7fab581f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:05.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.926+0000 7fab5d600640 1 --2- 172.21.15.32:0/4078596731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab580cce70 0x7fab58140cc0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:05.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.926+0000 7fab5d600640 1 -- 172.21.15.32:0/4078596731 >> 172.21.15.32:0/4078596731 conn(0x7fab5806ba30 msgr2=0x7fab581451e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:05.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.926+0000 7fab5d600640 1 -- 172.21.15.32:0/4078596731 shutdown_connections 2024-04-17T09:54:05.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:05.926+0000 7fab5d600640 1 -- 172.21.15.32:0/4078596731 wait complete. 2024-04-17T09:54:05.945 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T09:54:06.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.086+0000 7f167b400640 1 Processor -- start 2024-04-17T09:54:06.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.086+0000 7f167b400640 1 -- start start 2024-04-17T09:54:06.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.090+0000 7f167b400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1674155a70 0x7f167415f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:06.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.090+0000 7f167b400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f16740b3340 0x7f167415c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:06.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.090+0000 7f167b400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f167415f870 con 0x7f1674155e50 2024-04-17T09:54:06.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.090+0000 7f167b400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f16740b42a0 con 0x7f1674155a70 2024-04-17T09:54:06.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.090+0000 7f167b400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f16740d1250 con 0x7f16740b3340 2024-04-17T09:54:06.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.090+0000 7f1678a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1674155a70 0x7f167415f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:06.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.090+0000 7f1678a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1674155a70 0x7f167415f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33220/0 (socket says 172.21.15.32:33220) 2024-04-17T09:54:06.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.090+0000 7f1678a00640 1 -- 172.21.15.32:0/352585008 learned_addr learned my addr 172.21.15.32:0/352585008 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:06.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.090+0000 7f1679400640 1 --2- 172.21.15.32:0/352585008 >> v2:172.21.15.144:3300/0 conn(0x7f16740b3340 0x7f167415c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:06.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.090+0000 7f1678a00640 1 -- 172.21.15.32:0/352585008 >> v2:172.21.15.144:3300/0 conn(0x7f16740b3340 msgr2=0x7f167415c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:06.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.090+0000 7f1678a00640 1 --2- 172.21.15.32:0/352585008 >> v2:172.21.15.144:3300/0 conn(0x7f16740b3340 0x7f167415c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:06.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.090+0000 7f1678a00640 1 -- 172.21.15.32:0/352585008 >> v1:172.21.15.144:6789/0 conn(0x7f1674155e50 legacy=0x7f167415d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:06.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.090+0000 7f1678a00640 1 -- 172.21.15.32:0/352585008 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167415fb50 con 0x7f1674155a70 2024-04-17T09:54:06.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.094+0000 7f1678a00640 1 --2- 172.21.15.32:0/352585008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1674155a70 0x7f167415f190 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1664009870 tx=0x7f166401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3ac6a4dca48b5d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:06.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.094+0000 7f1673e00640 1 -- 172.21.15.32:0/352585008 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16640b0070 con 0x7f1674155a70 2024-04-17T09:54:06.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.094+0000 7f1673e00640 1 -- 172.21.15.32:0/352585008 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16640ab600 con 0x7f1674155a70 2024-04-17T09:54:06.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.094+0000 7f1673e00640 1 -- 172.21.15.32:0/352585008 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16640ab9d0 con 0x7f1674155a70 2024-04-17T09:54:06.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.094+0000 7f167b400640 1 -- 172.21.15.32:0/352585008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1674155a70 msgr2=0x7f167415f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:06.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.094+0000 7f167b400640 1 --2- 172.21.15.32:0/352585008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1674155a70 0x7f167415f190 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1664009870 tx=0x7f166401c4e0 comp rx=0 tx=0).stop 2024-04-17T09:54:06.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.094+0000 7f167b400640 1 -- 172.21.15.32:0/352585008 shutdown_connections 2024-04-17T09:54:06.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.094+0000 7f167b400640 1 --2- 172.21.15.32:0/352585008 >> v2:172.21.15.144:3300/0 conn(0x7f16740b3340 0x7f167415c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:06.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.094+0000 7f167b400640 1 --2- 172.21.15.32:0/352585008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1674155a70 0x7f167415f190 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:06.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.094+0000 7f167b400640 1 -- 172.21.15.32:0/352585008 >> 172.21.15.32:0/352585008 conn(0x7f167406ba30 msgr2=0x7f167406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:06.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.094+0000 7f167b400640 1 -- 172.21.15.32:0/352585008 shutdown_connections 2024-04-17T09:54:06.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.094+0000 7f167b400640 1 -- 172.21.15.32:0/352585008 wait complete. 2024-04-17T09:54:06.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.094+0000 7f167b400640 1 Processor -- start 2024-04-17T09:54:06.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.094+0000 7f167b400640 1 -- start start 2024-04-17T09:54:06.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.094+0000 7f167b400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16740b3340 0x7f167414fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:06.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.094+0000 7f167b400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1674155a70 0x7f16741504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:06.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.094+0000 7f167b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1674155e50 0x7f167406dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:06.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.094+0000 7f167b400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f16740d02f0 con 0x7f1674155e50 2024-04-17T09:54:06.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.094+0000 7f167b400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f16740d0170 con 0x7f1674155a70 2024-04-17T09:54:06.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.094+0000 7f167b400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f16740d0470 con 0x7f16740b3340 2024-04-17T09:54:06.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.098+0000 7f1679400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1674155e50 0x7f167406dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:06.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.098+0000 7f1679400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1674155e50 0x7f167406dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33228/0 (socket says 172.21.15.32:33228) 2024-04-17T09:54:06.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.098+0000 7f1679400640 1 -- 172.21.15.32:0/1702850946 learned_addr learned my addr 172.21.15.32:0/1702850946 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:06.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.098+0000 7f1678a00640 1 --2- 172.21.15.32:0/1702850946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16740b3340 0x7f167414fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:06.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.098+0000 7f166be00640 1 --2- 172.21.15.32:0/1702850946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1674155a70 0x7f16741504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:06.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.098+0000 7f1679400640 1 -- 172.21.15.32:0/1702850946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16740b3340 msgr2=0x7f167414fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:06.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.098+0000 7f1679400640 1 --2- 172.21.15.32:0/1702850946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16740b3340 0x7f167414fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:06.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.098+0000 7f1679400640 1 -- 172.21.15.32:0/1702850946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1674155a70 msgr2=0x7f16741504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:06.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.098+0000 7f1679400640 1 --2- 172.21.15.32:0/1702850946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1674155a70 0x7f16741504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:06.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.098+0000 7f1679400640 1 -- 172.21.15.32:0/1702850946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16740af3a0 con 0x7f1674155e50 2024-04-17T09:54:06.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.098+0000 7f166be00640 1 --2- 172.21.15.32:0/1702850946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1674155a70 0x7f16741504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:54:06.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.098+0000 7f1679400640 1 --2- 172.21.15.32:0/1702850946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1674155e50 0x7f167406dc80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f166c007e40 tx=0x7f166c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:06.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.098+0000 7f1671600640 1 -- 172.21.15.32:0/1702850946 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c014620 con 0x7f1674155e50 2024-04-17T09:54:06.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.098+0000 7f1671600640 1 -- 172.21.15.32:0/1702850946 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f166c019070 con 0x7f1674155e50 2024-04-17T09:54:06.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.098+0000 7f1671600640 1 -- 172.21.15.32:0/1702850946 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c014a20 con 0x7f1674155e50 2024-04-17T09:54:06.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.098+0000 7f167b400640 1 -- 172.21.15.32:0/1702850946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16740a94f0 con 0x7f1674155e50 2024-04-17T09:54:06.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.098+0000 7f167b400640 1 -- 172.21.15.32:0/1702850946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16740aee80 con 0x7f1674155e50 2024-04-17T09:54:06.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.098+0000 7f167b400640 1 -- 172.21.15.32:0/1702850946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16740af080 con 0x7f1674155e50 2024-04-17T09:54:06.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.098+0000 7f1671600640 1 -- 172.21.15.32:0/1702850946 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f166c014bc0 con 0x7f1674155e50 2024-04-17T09:54:06.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.098+0000 7f1671600640 1 --2- 172.21.15.32:0/1702850946 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1650068380 0x7f165006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:06.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.098+0000 7f1671600640 1 -- 172.21.15.32:0/1702850946 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f166c0872f0 con 0x7f1674155e50 2024-04-17T09:54:06.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.102+0000 7f1678a00640 1 --2- 172.21.15.32:0/1702850946 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1650068380 0x7f165006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:06.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.102+0000 7f1678a00640 1 --2- 172.21.15.32:0/1702850946 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1650068380 0x7f165006a840 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f16640ad000 tx=0x7f16640a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:06.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.106+0000 7f1671600640 1 -- 172.21.15.32:0/1702850946 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f16740af080 con 0x7f1674155e50 2024-04-17T09:54:06.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.306+0000 7f167b400640 1 -- 172.21.15.32:0/1702850946 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1674068a90 con 0x7f1674155e50 2024-04-17T09:54:06.305 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-17T09:54:06.306+0000 7f93c9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T09:54:06.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.306+0000 7f1671600640 1 -- 172.21.15.32:0/1702850946 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f166c057820 con 0x7f1674155e50 2024-04-17T09:54:06.308 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T09:54:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.310+0000 7f167b400640 1 -- 172.21.15.32:0/1702850946 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1650068380 msgr2=0x7f165006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.310+0000 7f167b400640 1 --2- 172.21.15.32:0/1702850946 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1650068380 0x7f165006a840 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f16640ad000 tx=0x7f16640a6000 comp rx=0 tx=0).stop 2024-04-17T09:54:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.310+0000 7f167b400640 1 -- 172.21.15.32:0/1702850946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1674155e50 msgr2=0x7f167406dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.310+0000 7f167b400640 1 --2- 172.21.15.32:0/1702850946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1674155e50 0x7f167406dc80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f166c007e40 tx=0x7f166c00e4b0 comp rx=0 tx=0).stop 2024-04-17T09:54:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.310+0000 7f167b400640 1 -- 172.21.15.32:0/1702850946 shutdown_connections 2024-04-17T09:54:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.310+0000 7f167b400640 1 --2- 172.21.15.32:0/1702850946 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1650068380 0x7f165006a840 secure :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f16640ad000 tx=0x7f16640a6000 comp rx=0 tx=0).stop 2024-04-17T09:54:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.310+0000 7f167b400640 1 --2- 172.21.15.32:0/1702850946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1674155e50 0x7f167406dc80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.310+0000 7f167b400640 1 --2- 172.21.15.32:0/1702850946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1674155a70 0x7f16741504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.310+0000 7f167b400640 1 --2- 172.21.15.32:0/1702850946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16740b3340 0x7f167414fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.310+0000 7f167b400640 1 -- 172.21.15.32:0/1702850946 >> 172.21.15.32:0/1702850946 conn(0x7f167406ba30 msgr2=0x7f16740b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.310+0000 7f167b400640 1 -- 172.21.15.32:0/1702850946 shutdown_connections 2024-04-17T09:54:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.310+0000 7f167b400640 1 -- 172.21.15.32:0/1702850946 wait complete. 2024-04-17T09:54:06.321 INFO:tasks.ceph.mds.b.smithi144.stderr: -3334> 2024-04-17T09:53:17.463+0000 7f57c4000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T09:54:06.331 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-17T09:54:06.371 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2024-04-17T09:54:06.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.474+0000 7f7743600640 1 Processor -- start 2024-04-17T09:54:06.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.474+0000 7f7743600640 1 -- start start 2024-04-17T09:54:06.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.474+0000 7f7743600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f773c15b7d0 0x7f773c06e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:06.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.474+0000 7f7743600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f773c0cc940 con 0x7f773c06c140 2024-04-17T09:54:06.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.474+0000 7f7743600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f773c0ccc20 con 0x7f773c06f7c0 2024-04-17T09:54:06.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.474+0000 7f7743600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f773c06e0b0 con 0x7f773c15b7d0 2024-04-17T09:54:06.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.474+0000 7f7741600640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f773c06c140 0x7f773c0ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48978/0 (socket says 172.21.15.32:48978) 2024-04-17T09:54:06.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.474+0000 7f7741600640 1 -- 172.21.15.32:0/723101997 learned_addr learned my addr 172.21.15.32:0/723101997 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:06.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.474+0000 7f7740c00640 1 --2- 172.21.15.32:0/723101997 >> v2:172.21.15.32:3300/0 conn(0x7f773c15b7d0 0x7f773c06e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:06.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.478+0000 7f7740c00640 1 -- 172.21.15.32:0/723101997 >> v1:172.21.15.144:6789/0 conn(0x7f773c06f7c0 legacy=0x7f773c06ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:06.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.478+0000 7f7740c00640 1 -- 172.21.15.32:0/723101997 >> v1:172.21.15.32:6789/0 conn(0x7f773c06c140 legacy=0x7f773c0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:06.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.478+0000 7f7740c00640 1 -- 172.21.15.32:0/723101997 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773c06e390 con 0x7f773c15b7d0 2024-04-17T09:54:06.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.478+0000 7f7740c00640 1 --2- 172.21.15.32:0/723101997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f773c15b7d0 0x7f773c06e8b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f772c0097c0 tx=0x7f772c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef2dbf218c3ceb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:06.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.478+0000 7f773be00640 1 -- 172.21.15.32:0/723101997 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c0ab430 con 0x7f773c15b7d0 2024-04-17T09:54:06.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.478+0000 7f773be00640 1 -- 172.21.15.32:0/723101997 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f772c0b0070 con 0x7f773c15b7d0 2024-04-17T09:54:06.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.478+0000 7f773be00640 1 -- 172.21.15.32:0/723101997 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c0ab7f0 con 0x7f773c15b7d0 2024-04-17T09:54:06.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.478+0000 7f7743600640 1 -- 172.21.15.32:0/723101997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f773c15b7d0 msgr2=0x7f773c06e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:06.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.478+0000 7f7743600640 1 --2- 172.21.15.32:0/723101997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f773c15b7d0 0x7f773c06e8b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f772c0097c0 tx=0x7f772c01c410 comp rx=0 tx=0).stop 2024-04-17T09:54:06.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.478+0000 7f7743600640 1 -- 172.21.15.32:0/723101997 shutdown_connections 2024-04-17T09:54:06.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.478+0000 7f7743600640 1 --2- 172.21.15.32:0/723101997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f773c15b7d0 0x7f773c06e8b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:06.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.478+0000 7f7743600640 1 -- 172.21.15.32:0/723101997 >> 172.21.15.32:0/723101997 conn(0x7f773c0b2120 msgr2=0x7f773c0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:06.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7743600640 1 -- 172.21.15.32:0/723101997 shutdown_connections 2024-04-17T09:54:06.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7743600640 1 -- 172.21.15.32:0/723101997 wait complete. 2024-04-17T09:54:06.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7743600640 1 Processor -- start 2024-04-17T09:54:06.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7743600640 1 -- start start 2024-04-17T09:54:06.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7743600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f773c06c140 0x7f773c141f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:06.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7743600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f773c06f7c0 0x7f773c0b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:06.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7743600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f773c15b7d0 0x7f773c0b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:06.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7743600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f773c169c40 con 0x7f773c15b7d0 2024-04-17T09:54:06.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7743600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f773c169ac0 con 0x7f773c06f7c0 2024-04-17T09:54:06.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7743600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f773c169dc0 con 0x7f773c06c140 2024-04-17T09:54:06.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7741600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f773c15b7d0 0x7f773c0b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:06.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7741600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f773c15b7d0 0x7f773c0b14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33242/0 (socket says 172.21.15.32:33242) 2024-04-17T09:54:06.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7740c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f773c06c140 0x7f773c141f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:06.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7741600640 1 -- 172.21.15.32:0/287375611 learned_addr learned my addr 172.21.15.32:0/287375611 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:06.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7733e00640 1 --2- 172.21.15.32:0/287375611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f773c06f7c0 0x7f773c0b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:06.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7741600640 1 -- 172.21.15.32:0/287375611 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f773c06c140 msgr2=0x7f773c141f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:06.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7741600640 1 --2- 172.21.15.32:0/287375611 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f773c06c140 0x7f773c141f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:06.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7741600640 1 -- 172.21.15.32:0/287375611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f773c06f7c0 msgr2=0x7f773c0b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:06.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7741600640 1 --2- 172.21.15.32:0/287375611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f773c06f7c0 0x7f773c0b0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:06.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7741600640 1 -- 172.21.15.32:0/287375611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773c14f100 con 0x7f773c15b7d0 2024-04-17T09:54:06.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7741600640 1 --2- 172.21.15.32:0/287375611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f773c15b7d0 0x7f773c0b14c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f773400ca50 tx=0x7f773400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:06.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7739600640 1 -- 172.21.15.32:0/287375611 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734017070 con 0x7f773c15b7d0 2024-04-17T09:54:06.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7739600640 1 -- 172.21.15.32:0/287375611 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f773400abb0 con 0x7f773c15b7d0 2024-04-17T09:54:06.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7739600640 1 -- 172.21.15.32:0/287375611 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734012400 con 0x7f773c15b7d0 2024-04-17T09:54:06.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.482+0000 7f7743600640 1 -- 172.21.15.32:0/287375611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f773c14ee20 con 0x7f773c15b7d0 2024-04-17T09:54:06.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.486+0000 7f7743600640 1 -- 172.21.15.32:0/287375611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f773c0af0c0 con 0x7f773c15b7d0 2024-04-17T09:54:06.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.486+0000 7f7743600640 1 -- 172.21.15.32:0/287375611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f773c068a90 con 0x7f773c15b7d0 2024-04-17T09:54:06.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.486+0000 7f7739600640 1 -- 172.21.15.32:0/287375611 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f773401a070 con 0x7f773c15b7d0 2024-04-17T09:54:06.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.486+0000 7f7739600640 1 --2- 172.21.15.32:0/287375611 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7718068380 0x7f771806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:06.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.486+0000 7f7739600640 1 -- 172.21.15.32:0/287375611 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5911+0+0 (secure 0 0 0) 0x7f7734086580 con 0x7f773c15b7d0 2024-04-17T09:54:06.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.486+0000 7f7740c00640 1 --2- 172.21.15.32:0/287375611 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7718068380 0x7f771806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:06.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.486+0000 7f7740c00640 1 --2- 172.21.15.32:0/287375611 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7718068380 0x7f771806a840 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f772c0af000 tx=0x7f772c0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:06.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.490+0000 7f7739600640 1 -- 172.21.15.32:0/287375611 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f7734056420 con 0x7f773c15b7d0 2024-04-17T09:54:06.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:06.690+0000 7f7743600640 1 -- 172.21.15.32:0/287375611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f773c06be40 con 0x7f773c15b7d0 2024-04-17T09:54:07.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.310+0000 7f7739600640 1 -- 172.21.15.32:0/287375611 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f773405a720 con 0x7f773c15b7d0 2024-04-17T09:54:07.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.310+0000 7f7743600640 1 -- 172.21.15.32:0/287375611 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7718068380 msgr2=0x7f771806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:07.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.310+0000 7f7743600640 1 --2- 172.21.15.32:0/287375611 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7718068380 0x7f771806a840 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f772c0af000 tx=0x7f772c0a7000 comp rx=0 tx=0).stop 2024-04-17T09:54:07.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.310+0000 7f7743600640 1 -- 172.21.15.32:0/287375611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f773c15b7d0 msgr2=0x7f773c0b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:07.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.310+0000 7f7743600640 1 --2- 172.21.15.32:0/287375611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f773c15b7d0 0x7f773c0b14c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f773400ca50 tx=0x7f773400cf10 comp rx=0 tx=0).stop 2024-04-17T09:54:07.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.310+0000 7f7743600640 1 -- 172.21.15.32:0/287375611 shutdown_connections 2024-04-17T09:54:07.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.310+0000 7f7743600640 1 --2- 172.21.15.32:0/287375611 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7718068380 0x7f771806a840 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:07.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.310+0000 7f7743600640 1 --2- 172.21.15.32:0/287375611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f773c15b7d0 0x7f773c0b14c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:07.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.310+0000 7f7743600640 1 --2- 172.21.15.32:0/287375611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f773c06f7c0 0x7f773c0b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:07.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.310+0000 7f7743600640 1 --2- 172.21.15.32:0/287375611 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f773c06c140 0x7f773c141f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:07.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.310+0000 7f7743600640 1 -- 172.21.15.32:0/287375611 >> 172.21.15.32:0/287375611 conn(0x7f773c0b2120 msgr2=0x7f773c151b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:07.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.310+0000 7f7743600640 1 -- 172.21.15.32:0/287375611 shutdown_connections 2024-04-17T09:54:07.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.314+0000 7f7743600640 1 -- 172.21.15.32:0/287375611 wait complete. 2024-04-17T09:54:07.335 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-17T09:54:07.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.482+0000 7f0833c00640 1 Processor -- start 2024-04-17T09:54:07.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.482+0000 7f0833c00640 1 -- start start 2024-04-17T09:54:07.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.482+0000 7f0833c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f082c15acc0 0x7f082c0cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:07.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.482+0000 7f0833c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082c15b0a0 con 0x7f082c15e840 2024-04-17T09:54:07.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.482+0000 7f0833c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082c15f020 con 0x7f082c0ceb50 2024-04-17T09:54:07.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.482+0000 7f0833c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f082c15f300 con 0x7f082c15acc0 2024-04-17T09:54:07.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.482+0000 7f0823e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f082c15e840 0x7f082c06e410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48984/0 (socket says 172.21.15.32:48984) 2024-04-17T09:54:07.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.482+0000 7f0823e00640 1 -- 172.21.15.32:0/346629692 learned_addr learned my addr 172.21.15.32:0/346629692 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:07.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.482+0000 7f0831200640 1 --2- 172.21.15.32:0/346629692 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f082c15acc0 0x7f082c0cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:07.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.482+0000 7f082be00640 1 -- 172.21.15.32:0/346629692 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1669509474 0 0) 0x7f082c15b0a0 con 0x7f082c15e840 2024-04-17T09:54:07.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.482+0000 7f082be00640 1 -- 172.21.15.32:0/346629692 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0810003530 con 0x7f082c15e840 2024-04-17T09:54:07.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f082be00640 1 -- 172.21.15.32:0/346629692 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 325068548 0 0) 0x7f082c15f020 con 0x7f082c0ceb50 2024-04-17T09:54:07.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f082be00640 1 -- 172.21.15.32:0/346629692 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f082c15b0a0 con 0x7f082c0ceb50 2024-04-17T09:54:07.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f082be00640 1 -- 172.21.15.32:0/346629692 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3698912352 0 0) 0x7f0810003530 con 0x7f082c15e840 2024-04-17T09:54:07.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f082be00640 1 -- 172.21.15.32:0/346629692 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f082c15acc0 msgr2=0x7f082c0cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:07.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f082be00640 1 --2- 172.21.15.32:0/346629692 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f082c15acc0 0x7f082c0cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:07.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f082be00640 1 -- 172.21.15.32:0/346629692 >> v1:172.21.15.144:6789/0 conn(0x7f082c0ceb50 legacy=0x7f082c072640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:07.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f082be00640 1 -- 172.21.15.32:0/346629692 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082c15f5e0 con 0x7f082c15e840 2024-04-17T09:54:07.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f082be00640 1 -- 172.21.15.32:0/346629692 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f0814001800 con 0x7f082c15e840 2024-04-17T09:54:07.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f082be00640 1 -- 172.21.15.32:0/346629692 >> v1:172.21.15.32:6789/0 conn(0x7f082c15e840 legacy=0x7f082c06e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:07.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f082be00640 1 --2- 172.21.15.32:0/346629692 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f082c15acc0 0x7f0810003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:07.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f082be00640 1 --2- 172.21.15.32:0/346629692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08100040e0 0x7f0810004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:07.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f082be00640 1 --2- 172.21.15.32:0/346629692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0810004d50 0x7f08100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:07.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f082be00640 1 -- 172.21.15.32:0/346629692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f082c15f300 con 0x7f08100040e0 2024-04-17T09:54:07.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f082be00640 1 -- 172.21.15.32:0/346629692 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f081000bb60 con 0x7f0810004d50 2024-04-17T09:54:07.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f082be00640 1 -- 172.21.15.32:0/346629692 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f081000bd30 con 0x7f082c15acc0 2024-04-17T09:54:07.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f0823e00640 1 --2- 172.21.15.32:0/346629692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08100040e0 0x7f0810004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:07.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f0831200640 1 --2- 172.21.15.32:0/346629692 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f082c15acc0 0x7f0810003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:07.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f0831c00640 1 --2- 172.21.15.32:0/346629692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0810004d50 0x7f08100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:07.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f0823e00640 1 -- 172.21.15.32:0/346629692 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f082c15acc0 msgr2=0x7f0810003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:07.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f0823e00640 1 --2- 172.21.15.32:0/346629692 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f082c15acc0 0x7f0810003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:07.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f0823e00640 1 -- 172.21.15.32:0/346629692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0810004d50 msgr2=0x7f08100051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:07.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f0823e00640 1 --2- 172.21.15.32:0/346629692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0810004d50 0x7f08100051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:07.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f0823e00640 1 -- 172.21.15.32:0/346629692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0810003530 con 0x7f08100040e0 2024-04-17T09:54:07.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f0823e00640 1 --2- 172.21.15.32:0/346629692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08100040e0 0x7f0810004810 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f081400c190 tx=0x7f081401cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ddbd79e20ee9b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:07.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.486+0000 7f082be00640 1 -- 172.21.15.32:0/346629692 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814007cb0 con 0x7f08100040e0 2024-04-17T09:54:07.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.490+0000 7f082be00640 1 -- 172.21.15.32:0/346629692 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08140a9020 con 0x7f08100040e0 2024-04-17T09:54:07.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.490+0000 7f082be00640 1 -- 172.21.15.32:0/346629692 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08140072f0 con 0x7f08100040e0 2024-04-17T09:54:07.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.490+0000 7f0833c00640 1 -- 172.21.15.32:0/346629692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08100040e0 msgr2=0x7f0810004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:07.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.490+0000 7f0833c00640 1 --2- 172.21.15.32:0/346629692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08100040e0 0x7f0810004810 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f081400c190 tx=0x7f081401cc00 comp rx=0 tx=0).stop 2024-04-17T09:54:07.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.490+0000 7f0831c00640 1 -- 172.21.15.32:0/346629692 reap_dead start 2024-04-17T09:54:07.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.490+0000 7f0833c00640 1 -- 172.21.15.32:0/346629692 shutdown_connections 2024-04-17T09:54:07.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.490+0000 7f0833c00640 1 -- 172.21.15.32:0/346629692 >> 172.21.15.32:0/346629692 conn(0x7f082c0b2190 msgr2=0x7f082c0b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:07.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.490+0000 7f0833c00640 1 -- 172.21.15.32:0/346629692 shutdown_connections 2024-04-17T09:54:07.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.490+0000 7f0833c00640 1 -- 172.21.15.32:0/346629692 wait complete. 2024-04-17T09:54:07.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.490+0000 7f0833c00640 1 Processor -- start 2024-04-17T09:54:07.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.490+0000 7f0833c00640 1 -- start start 2024-04-17T09:54:07.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.490+0000 7f0833c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08100040e0 0x7f082c1594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:07.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.490+0000 7f0833c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f082c14b490 0x7f082c14b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:07.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.490+0000 7f0833c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f082c157260 0x7f082c1576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:07.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.490+0000 7f0833c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f082c072ed0 con 0x7f082c157260 2024-04-17T09:54:07.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.490+0000 7f0833c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f082c072d50 con 0x7f08100040e0 2024-04-17T09:54:07.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.490+0000 7f0833c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f082c0cd230 con 0x7f082c14b490 2024-04-17T09:54:07.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.490+0000 7f0831c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f082c157260 0x7f082c1576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:07.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.490+0000 7f0831c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f082c157260 0x7f082c1576c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33260/0 (socket says 172.21.15.32:33260) 2024-04-17T09:54:07.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.490+0000 7f0831c00640 1 -- 172.21.15.32:0/1705889258 learned_addr learned my addr 172.21.15.32:0/1705889258 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:07.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.490+0000 7f0831200640 1 --2- 172.21.15.32:0/1705889258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08100040e0 0x7f082c1594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:07.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.490+0000 7f0823e00640 1 --2- 172.21.15.32:0/1705889258 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f082c14b490 0x7f082c14b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:07.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.494+0000 7f0831c00640 1 -- 172.21.15.32:0/1705889258 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f082c14b490 msgr2=0x7f082c14b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:07.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.494+0000 7f0831c00640 1 --2- 172.21.15.32:0/1705889258 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f082c14b490 0x7f082c14b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:07.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.494+0000 7f0831c00640 1 -- 172.21.15.32:0/1705889258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08100040e0 msgr2=0x7f082c1594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:07.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.494+0000 7f0831c00640 1 --2- 172.21.15.32:0/1705889258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08100040e0 0x7f082c1594f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:07.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.494+0000 7f0831c00640 1 -- 172.21.15.32:0/1705889258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082c157f90 con 0x7f082c157260 2024-04-17T09:54:07.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.494+0000 7f0831200640 1 --2- 172.21.15.32:0/1705889258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08100040e0 0x7f082c1594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:07.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.494+0000 7f0831c00640 1 --2- 172.21.15.32:0/1705889258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f082c157260 0x7f082c1576c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0824007fd0 tx=0x7f082400ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:07.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.494+0000 7f0829600640 1 -- 172.21.15.32:0/1705889258 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824019070 con 0x7f082c157260 2024-04-17T09:54:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.494+0000 7f0829600640 1 -- 172.21.15.32:0/1705889258 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0824006e30 con 0x7f082c157260 2024-04-17T09:54:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.494+0000 7f0829600640 1 -- 172.21.15.32:0/1705889258 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824003b80 con 0x7f082c157260 2024-04-17T09:54:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.494+0000 7f0833c00640 1 -- 172.21.15.32:0/1705889258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f082c161c50 con 0x7f082c157260 2024-04-17T09:54:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.494+0000 7f0833c00640 1 -- 172.21.15.32:0/1705889258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f082c162210 con 0x7f082c157260 2024-04-17T09:54:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.494+0000 7f0833c00640 1 -- 172.21.15.32:0/1705889258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f082c068a90 con 0x7f082c157260 2024-04-17T09:54:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.494+0000 7f0829600640 1 -- 172.21.15.32:0/1705889258 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f082401c070 con 0x7f082c157260 2024-04-17T09:54:07.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.494+0000 7f0829600640 1 --2- 172.21.15.32:0/1705889258 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0808068380 0x7f080806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:07.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.494+0000 7f0829600640 1 -- 172.21.15.32:0/1705889258 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5911+0+0 (secure 0 0 0) 0x7f0824085fc0 con 0x7f082c157260 2024-04-17T09:54:07.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.494+0000 7f0831200640 1 --2- 172.21.15.32:0/1705889258 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0808068380 0x7f080806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:07.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.498+0000 7f0831200640 1 --2- 172.21.15.32:0/1705889258 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0808068380 0x7f080806a840 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f081c009b00 tx=0x7f081c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.498+0000 7f0829600640 1 -- 172.21.15.32:0/1705889258 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f08240564a0 con 0x7f082c157260 2024-04-17T09:54:07.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:07.666+0000 7f0833c00640 1 -- 172.21.15.32:0/1705889258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f082c168e50 con 0x7f082c157260 2024-04-17T09:54:08.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.310+0000 7f0829600640 1 -- 172.21.15.32:0/1705889258 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) ==== 181+0+0 (secure 0 0 0) 0x7f0824055da0 con 0x7f082c157260 2024-04-17T09:54:08.312 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-04-17T09:54:08.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.314+0000 7f0833c00640 1 -- 172.21.15.32:0/1705889258 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0808068380 msgr2=0x7f080806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:08.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.314+0000 7f0833c00640 1 --2- 172.21.15.32:0/1705889258 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0808068380 0x7f080806a840 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f081c009b00 tx=0x7f081c008040 comp rx=0 tx=0).stop 2024-04-17T09:54:08.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.314+0000 7f0833c00640 1 -- 172.21.15.32:0/1705889258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f082c157260 msgr2=0x7f082c1576c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:08.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.314+0000 7f0833c00640 1 --2- 172.21.15.32:0/1705889258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f082c157260 0x7f082c1576c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0824007fd0 tx=0x7f082400ff70 comp rx=0 tx=0).stop 2024-04-17T09:54:08.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.314+0000 7f0833c00640 1 -- 172.21.15.32:0/1705889258 shutdown_connections 2024-04-17T09:54:08.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.314+0000 7f0833c00640 1 --2- 172.21.15.32:0/1705889258 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0808068380 0x7f080806a840 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:08.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.314+0000 7f0833c00640 1 --2- 172.21.15.32:0/1705889258 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f082c14b490 0x7f082c14b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:08.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.314+0000 7f0833c00640 1 --2- 172.21.15.32:0/1705889258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f082c157260 0x7f082c1576c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:08.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.314+0000 7f0833c00640 1 --2- 172.21.15.32:0/1705889258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08100040e0 0x7f082c1594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:08.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.314+0000 7f0833c00640 1 -- 172.21.15.32:0/1705889258 >> 172.21.15.32:0/1705889258 conn(0x7f082c0b2190 msgr2=0x7f082c0cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:08.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.314+0000 7f0833c00640 1 -- 172.21.15.32:0/1705889258 shutdown_connections 2024-04-17T09:54:08.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.314+0000 7f0833c00640 1 -- 172.21.15.32:0/1705889258 wait complete. 2024-04-17T09:54:08.341 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-17T09:54:08.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.490+0000 7fce82400640 1 Processor -- start 2024-04-17T09:54:08.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.490+0000 7fce82400640 1 -- start start 2024-04-17T09:54:08.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.490+0000 7fce82400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce7c15fb90 0x7fce7c0d0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:08.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.490+0000 7fce82400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fce7c15ff70 0x7fce7c0d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:08.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.490+0000 7fce82400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce7c09b5a0 con 0x7fce7c09ba30 2024-04-17T09:54:08.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.490+0000 7fce82400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fce7c0b3b30 con 0x7fce7c15ff70 2024-04-17T09:54:08.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.490+0000 7fce82400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fce7c0b3570 con 0x7fce7c15fb90 2024-04-17T09:54:08.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.490+0000 7fce81a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fce7c15ff70 0x7fce7c0d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:08.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.490+0000 7fce81000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce7c15fb90 0x7fce7c0d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:08.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.490+0000 7fce81a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fce7c15ff70 0x7fce7c0d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33276/0 (socket says 172.21.15.32:33276) 2024-04-17T09:54:08.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.490+0000 7fce81a00640 1 -- 172.21.15.32:0/970730349 learned_addr learned my addr 172.21.15.32:0/970730349 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:08.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.490+0000 7fce73e00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fce7c09ba30 0x7fce7c0cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:42122/0 (socket says 172.21.15.32:42122) 2024-04-17T09:54:08.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.490+0000 7fce81a00640 1 -- 172.21.15.32:0/970730349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce7c15fb90 msgr2=0x7fce7c0d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:08.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.490+0000 7fce81a00640 1 --2- 172.21.15.32:0/970730349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce7c15fb90 0x7fce7c0d0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:08.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.490+0000 7fce81a00640 1 -- 172.21.15.32:0/970730349 >> v1:172.21.15.144:6789/0 conn(0x7fce7c09ba30 legacy=0x7fce7c0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:08.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.490+0000 7fce81a00640 1 -- 172.21.15.32:0/970730349 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7c0b3850 con 0x7fce7c15ff70 2024-04-17T09:54:08.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.494+0000 7fce81a00640 1 --2- 172.21.15.32:0/970730349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce7c15ff70 0x7fce7c0d0ea0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fce74009e50 tx=0x7fce7401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca49d5e841d0ed0c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:08.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.494+0000 7fce7be00640 1 -- 172.21.15.32:0/970730349 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce740a5600 con 0x7fce7c15ff70 2024-04-17T09:54:08.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.494+0000 7fce7be00640 1 -- 172.21.15.32:0/970730349 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce740a57a0 con 0x7fce7c15ff70 2024-04-17T09:54:08.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.494+0000 7fce7be00640 1 -- 172.21.15.32:0/970730349 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce740a5b70 con 0x7fce7c15ff70 2024-04-17T09:54:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.494+0000 7fce82400640 1 -- 172.21.15.32:0/970730349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce7c15ff70 msgr2=0x7fce7c0d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.494+0000 7fce82400640 1 --2- 172.21.15.32:0/970730349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce7c15ff70 0x7fce7c0d0ea0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fce74009e50 tx=0x7fce7401ca30 comp rx=0 tx=0).stop 2024-04-17T09:54:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.494+0000 7fce82400640 1 -- 172.21.15.32:0/970730349 shutdown_connections 2024-04-17T09:54:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.494+0000 7fce82400640 1 --2- 172.21.15.32:0/970730349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce7c15ff70 0x7fce7c0d0ea0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.494+0000 7fce82400640 1 --2- 172.21.15.32:0/970730349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce7c15fb90 0x7fce7c0d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.494+0000 7fce82400640 1 -- 172.21.15.32:0/970730349 >> 172.21.15.32:0/970730349 conn(0x7fce7c0b3e70 msgr2=0x7fce7c0b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.494+0000 7fce82400640 1 -- 172.21.15.32:0/970730349 shutdown_connections 2024-04-17T09:54:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.494+0000 7fce82400640 1 -- 172.21.15.32:0/970730349 wait complete. 2024-04-17T09:54:08.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce82400640 1 Processor -- start 2024-04-17T09:54:08.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce82400640 1 -- start start 2024-04-17T09:54:08.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce82400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce7c09ba30 0x7fce7c1553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:08.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce82400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce7c15fb90 0x7fce7c0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:08.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce82400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce7c15ff70 0x7fce7c0b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:08.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce82400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fce7c1696d0 con 0x7fce7c15fb90 2024-04-17T09:54:08.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce82400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fce7c0cdd90 con 0x7fce7c09ba30 2024-04-17T09:54:08.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce82400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fce7c169850 con 0x7fce7c15ff70 2024-04-17T09:54:08.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce73e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce7c15fb90 0x7fce7c0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:08.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce81000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce7c09ba30 0x7fce7c1553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:08.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce81000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce7c09ba30 0x7fce7c1553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:44926/0 (socket says 172.21.15.32:44926) 2024-04-17T09:54:08.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce81000640 1 -- 172.21.15.32:0/2786946005 learned_addr learned my addr 172.21.15.32:0/2786946005 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:08.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce81a00640 1 --2- 172.21.15.32:0/2786946005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce7c15ff70 0x7fce7c0b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:08.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce73e00640 1 -- 172.21.15.32:0/2786946005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce7c15ff70 msgr2=0x7fce7c0b58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:08.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce73e00640 1 --2- 172.21.15.32:0/2786946005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce7c15ff70 0x7fce7c0b58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:08.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce73e00640 1 -- 172.21.15.32:0/2786946005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce7c09ba30 msgr2=0x7fce7c1553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:08.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce73e00640 1 --2- 172.21.15.32:0/2786946005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce7c09ba30 0x7fce7c1553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:08.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce73e00640 1 -- 172.21.15.32:0/2786946005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7c1558f0 con 0x7fce7c15fb90 2024-04-17T09:54:08.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce73e00640 1 --2- 172.21.15.32:0/2786946005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce7c15fb90 0x7fce7c0b5390 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fce6400c390 tx=0x7fce6400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:08.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce79600640 1 -- 172.21.15.32:0/2786946005 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce640074e0 con 0x7fce7c15fb90 2024-04-17T09:54:08.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce79600640 1 -- 172.21.15.32:0/2786946005 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce64007680 con 0x7fce7c15fb90 2024-04-17T09:54:08.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce82400640 1 -- 172.21.15.32:0/2786946005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce7c06e960 con 0x7fce7c15fb90 2024-04-17T09:54:08.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce79600640 1 -- 172.21.15.32:0/2786946005 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64004a30 con 0x7fce7c15fb90 2024-04-17T09:54:08.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.498+0000 7fce82400640 1 -- 172.21.15.32:0/2786946005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce7c06cec0 con 0x7fce7c15fb90 2024-04-17T09:54:08.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.502+0000 7fce82400640 1 -- 172.21.15.32:0/2786946005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce7c04fd00 con 0x7fce7c15fb90 2024-04-17T09:54:08.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.502+0000 7fce79600640 1 -- 172.21.15.32:0/2786946005 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fce6401a020 con 0x7fce7c15fb90 2024-04-17T09:54:08.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.502+0000 7fce79600640 1 --2- 172.21.15.32:0/2786946005 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fce580683b0 0x7fce5806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:08.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.502+0000 7fce79600640 1 -- 172.21.15.32:0/2786946005 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5473+0+0 (secure 0 0 0) 0x7fce64085e50 con 0x7fce7c15fb90 2024-04-17T09:54:08.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.502+0000 7fce81000640 1 --2- 172.21.15.32:0/2786946005 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fce580683b0 0x7fce5806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:08.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.502+0000 7fce81000640 1 --2- 172.21.15.32:0/2786946005 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fce580683b0 0x7fce5806a870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0c8ef0 tx=0x7fce6c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.506+0000 7fce79600640 1 -- 172.21.15.32:0/2786946005 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fce64055ea0 con 0x7fce7c15fb90 2024-04-17T09:54:08.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:08.682+0000 7fce82400640 1 -- 172.21.15.32:0/2786946005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fce7c0b6b20 con 0x7fce7c15fb90 2024-04-17T09:54:09.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.314+0000 7fce79600640 1 -- 172.21.15.32:0/2786946005 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) ==== 169+0+0 (secure 0 0 0) 0x7fce6405a1a0 con 0x7fce7c15fb90 2024-04-17T09:54:09.314 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-04-17T09:54:09.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.318+0000 7fce82400640 1 -- 172.21.15.32:0/2786946005 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fce580683b0 msgr2=0x7fce5806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:09.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.318+0000 7fce82400640 1 --2- 172.21.15.32:0/2786946005 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fce580683b0 0x7fce5806a870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0c8ef0 tx=0x7fce6c009190 comp rx=0 tx=0).stop 2024-04-17T09:54:09.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.318+0000 7fce82400640 1 -- 172.21.15.32:0/2786946005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce7c15fb90 msgr2=0x7fce7c0b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:09.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.318+0000 7fce82400640 1 --2- 172.21.15.32:0/2786946005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce7c15fb90 0x7fce7c0b5390 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fce6400c390 tx=0x7fce6400c850 comp rx=0 tx=0).stop 2024-04-17T09:54:09.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.318+0000 7fce82400640 1 -- 172.21.15.32:0/2786946005 shutdown_connections 2024-04-17T09:54:09.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.318+0000 7fce82400640 1 --2- 172.21.15.32:0/2786946005 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fce580683b0 0x7fce5806a870 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:09.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.318+0000 7fce82400640 1 --2- 172.21.15.32:0/2786946005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce7c15ff70 0x7fce7c0b58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:09.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.318+0000 7fce82400640 1 --2- 172.21.15.32:0/2786946005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce7c15fb90 0x7fce7c0b5390 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:09.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.318+0000 7fce82400640 1 --2- 172.21.15.32:0/2786946005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce7c09ba30 0x7fce7c1553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:09.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.318+0000 7fce82400640 1 -- 172.21.15.32:0/2786946005 >> 172.21.15.32:0/2786946005 conn(0x7fce7c0b3e70 msgr2=0x7fce7c0b72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:09.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.318+0000 7fce82400640 1 -- 172.21.15.32:0/2786946005 shutdown_connections 2024-04-17T09:54:09.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.318+0000 7fce82400640 1 -- 172.21.15.32:0/2786946005 wait complete. 2024-04-17T09:54:09.345 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-17T09:54:09.345 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-17T09:54:09.345 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-17T09:54:09.345 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:54:09.345 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-17T09:54:09.345 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-17T09:54:09.346 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-17T09:54:09.346 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:54:09.346 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-17T09:54:09.346 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-17T09:54:09.346 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-17T09:54:09.346 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:54:09.347 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-17T09:54:09.347 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-17T09:54:09.347 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-17T09:54:09.347 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:54:09.358 INFO:tasks.ceph.mds.c:Stopped 2024-04-17T09:54:09.358 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-17T09:54:09.359 INFO:tasks.ceph.mds.a:Stopped 2024-04-17T09:54:09.359 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-17T09:54:09.360 INFO:tasks.ceph.mds.b:Stopped 2024-04-17T09:54:09.360 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-17T09:54:09.362 INFO:tasks.ceph.mds.d:Stopped 2024-04-17T09:54:09.362 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-17T09:54:09.407 INFO:tasks.ceph.mds.c:Started 2024-04-17T09:54:09.407 DEBUG:teuthology.parallel:result is None 2024-04-17T09:54:09.409 INFO:tasks.ceph.mds.a:Started 2024-04-17T09:54:09.409 DEBUG:teuthology.parallel:result is None 2024-04-17T09:54:09.409 INFO:tasks.ceph.mds.b:Started 2024-04-17T09:54:09.410 DEBUG:teuthology.parallel:result is None 2024-04-17T09:54:09.412 INFO:tasks.ceph.mds.d:Started 2024-04-17T09:54:09.412 DEBUG:teuthology.parallel:result is None 2024-04-17T09:54:09.412 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-17T09:54:09.497 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-17T09:54:09.497 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-17T09:54:09.500 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2024-04-17T09:54:09.500 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2024-04-17T09:54:09.501 INFO:tasks.ceph.mds.b.smithi144.stderr:2024-04-17T09:54:09.498+0000 7f3a82800640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-04-17T09:54:09.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.558+0000 7f9cec600640 1 Processor -- start 2024-04-17T09:54:09.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.558+0000 7f9cec600640 1 -- start start 2024-04-17T09:54:09.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.558+0000 7f9cec600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9ce413f6e0 0x7f9ce40cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:09.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.558+0000 7f9cec600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ce40cf370 0x7f9ce40d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:09.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.558+0000 7f9cec600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9ce40d0820 0x7f9ce4141d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:09.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.558+0000 7f9cec600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9ce413fac0 con 0x7f9ce40d0820 2024-04-17T09:54:09.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.558+0000 7f9cec600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f9ce41601e0 con 0x7f9ce413f6e0 2024-04-17T09:54:09.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.558+0000 7f9cec600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce4160480 con 0x7f9ce40cf370 2024-04-17T09:54:09.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.558+0000 7f9cea600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9ce40d0820 0x7f9ce4141d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:09.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.558+0000 7f9cea600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9ce40d0820 0x7f9ce4141d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33326/0 (socket says 172.21.15.32:33326) 2024-04-17T09:54:09.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.558+0000 7f9cea600640 1 -- 172.21.15.32:0/2547641193 learned_addr learned my addr 172.21.15.32:0/2547641193 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:09.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.558+0000 7f9ce9c00640 1 --2- 172.21.15.32:0/2547641193 >> v2:172.21.15.144:3300/0 conn(0x7f9ce413f6e0 0x7f9ce40cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:09.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.558+0000 7f9ce9200640 1 --2- 172.21.15.32:0/2547641193 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ce40cf370 0x7f9ce40d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:09.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.558+0000 7f9cea600640 1 -- 172.21.15.32:0/2547641193 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ce40cf370 msgr2=0x7f9ce40d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:09.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.558+0000 7f9cea600640 1 --2- 172.21.15.32:0/2547641193 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ce40cf370 0x7f9ce40d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:09.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.558+0000 7f9cea600640 1 -- 172.21.15.32:0/2547641193 >> v2:172.21.15.144:3300/0 conn(0x7f9ce413f6e0 msgr2=0x7f9ce40cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:09.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.558+0000 7f9cea600640 1 --2- 172.21.15.32:0/2547641193 >> v2:172.21.15.144:3300/0 conn(0x7f9ce413f6e0 0x7f9ce40cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:09.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.558+0000 7f9cea600640 1 -- 172.21.15.32:0/2547641193 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce4160760 con 0x7f9ce40d0820 2024-04-17T09:54:09.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.558+0000 7f9ce9200640 1 --2- 172.21.15.32:0/2547641193 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ce40cf370 0x7f9ce40d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:09.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.562+0000 7f9cea600640 1 --2- 172.21.15.32:0/2547641193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ce40d0820 0x7f9ce4141d70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40b3100 tx=0x7f9cdc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fc90eced2df56c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:09.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.562+0000 7f9ce3e00640 1 -- 172.21.15.32:0/2547641193 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc0a5760 con 0x7f9ce40d0820 2024-04-17T09:54:09.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.562+0000 7f9ce3e00640 1 -- 172.21.15.32:0/2547641193 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cdc0a5900 con 0x7f9ce40d0820 2024-04-17T09:54:09.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.562+0000 7f9ce3e00640 1 -- 172.21.15.32:0/2547641193 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc0a5cd0 con 0x7f9ce40d0820 2024-04-17T09:54:09.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.562+0000 7f9cec600640 1 -- 172.21.15.32:0/2547641193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ce40d0820 msgr2=0x7f9ce4141d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:09.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.562+0000 7f9cec600640 1 --2- 172.21.15.32:0/2547641193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ce40d0820 0x7f9ce4141d70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40b3100 tx=0x7f9cdc01c9f0 comp rx=0 tx=0).stop 2024-04-17T09:54:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.562+0000 7f9cec600640 1 -- 172.21.15.32:0/2547641193 shutdown_connections 2024-04-17T09:54:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.562+0000 7f9cec600640 1 --2- 172.21.15.32:0/2547641193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ce40d0820 0x7f9ce4141d70 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.562+0000 7f9cec600640 1 --2- 172.21.15.32:0/2547641193 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ce40cf370 0x7f9ce40d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.562+0000 7f9cec600640 1 --2- 172.21.15.32:0/2547641193 >> v2:172.21.15.144:3300/0 conn(0x7f9ce413f6e0 0x7f9ce40cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.562+0000 7f9cec600640 1 -- 172.21.15.32:0/2547641193 >> 172.21.15.32:0/2547641193 conn(0x7f9ce406ba30 msgr2=0x7f9ce406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.562+0000 7f9cec600640 1 -- 172.21.15.32:0/2547641193 shutdown_connections 2024-04-17T09:54:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.562+0000 7f9cec600640 1 -- 172.21.15.32:0/2547641193 wait complete. 2024-04-17T09:54:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9cec600640 1 Processor -- start 2024-04-17T09:54:09.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9cec600640 1 -- start start 2024-04-17T09:54:09.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9cec600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ce40cf370 0x7f9ce413cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:09.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9cec600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ce40d0820 0x7f9ce4157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:09.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9cec600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ce413f6e0 0x7f9ce41581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:09.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9cec600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce40cff10 con 0x7f9ce40cf370 2024-04-17T09:54:09.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9cec600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce40cfd90 con 0x7f9ce40d0820 2024-04-17T09:54:09.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9cec600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce4071d80 con 0x7f9ce413f6e0 2024-04-17T09:54:09.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9ce9c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ce40cf370 0x7f9ce413cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:09.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9ce9c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ce40cf370 0x7f9ce413cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33334/0 (socket says 172.21.15.32:33334) 2024-04-17T09:54:09.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9ce9c00640 1 -- 172.21.15.32:0/445147297 learned_addr learned my addr 172.21.15.32:0/445147297 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:09.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9ce9200640 1 --2- 172.21.15.32:0/445147297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ce40d0820 0x7f9ce4157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:09.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9cea600640 1 --2- 172.21.15.32:0/445147297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ce413f6e0 0x7f9ce41581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:09.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9ce9c00640 1 -- 172.21.15.32:0/445147297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ce413f6e0 msgr2=0x7f9ce41581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:09.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9ce9c00640 1 --2- 172.21.15.32:0/445147297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ce413f6e0 0x7f9ce41581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:09.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9ce9c00640 1 -- 172.21.15.32:0/445147297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ce40d0820 msgr2=0x7f9ce4157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:09.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9ce9c00640 1 --2- 172.21.15.32:0/445147297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ce40d0820 0x7f9ce4157cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:09.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9ce9c00640 1 -- 172.21.15.32:0/445147297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce413d8e0 con 0x7f9ce40cf370 2024-04-17T09:54:09.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9ce9c00640 1 --2- 172.21.15.32:0/445147297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ce40cf370 0x7f9ce413cf50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800e870 tx=0x7f9cd800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:09.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9ce1600640 1 -- 172.21.15.32:0/445147297 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8017070 con 0x7f9ce40cf370 2024-04-17T09:54:09.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9ce1600640 1 -- 172.21.15.32:0/445147297 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cd800c940 con 0x7f9ce40cf370 2024-04-17T09:54:09.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9ce1600640 1 -- 172.21.15.32:0/445147297 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd800ccf0 con 0x7f9ce40cf370 2024-04-17T09:54:09.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9cec600640 1 -- 172.21.15.32:0/445147297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce413dbc0 con 0x7f9ce40cf370 2024-04-17T09:54:09.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.566+0000 7f9cec600640 1 -- 172.21.15.32:0/445147297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce413e180 con 0x7f9ce40cf370 2024-04-17T09:54:09.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.570+0000 7f9cec600640 1 -- 172.21.15.32:0/445147297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ce4068a90 con 0x7f9ce40cf370 2024-04-17T09:54:09.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.570+0000 7f9ce1600640 1 -- 172.21.15.32:0/445147297 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9cd801a070 con 0x7f9ce40cf370 2024-04-17T09:54:09.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.570+0000 7f9ce1600640 1 --2- 172.21.15.32:0/445147297 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9cc4068380 0x7f9cc406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:09.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.570+0000 7f9ce1600640 1 -- 172.21.15.32:0/445147297 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4943+0+0 (secure 0 0 0) 0x7f9cd80865e0 con 0x7f9ce40cf370 2024-04-17T09:54:09.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.570+0000 7f9ce9200640 1 --2- 172.21.15.32:0/445147297 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9cc4068380 0x7f9cc406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:09.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.570+0000 7f9ce9200640 1 --2- 172.21.15.32:0/445147297 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9cc4068380 0x7f9cc406a840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4158b20 tx=0x7f9cd0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:09.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.574+0000 7f9ce1600640 1 -- 172.21.15.32:0/445147297 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f9cd8056ea0 con 0x7f9ce40cf370 2024-04-17T09:54:09.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:09.754+0000 7f9cec600640 1 -- 172.21.15.32:0/445147297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f9ce40d1030 con 0x7f9ce40cf370 2024-04-17T09:54:11.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.010+0000 7f9ce1600640 1 -- 172.21.15.32:0/445147297 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v31) ==== 97+0+0 (secure 0 0 0) 0x7f9cd80567a0 con 0x7f9ce40cf370 2024-04-17T09:54:11.012 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2024-04-17T09:54:11.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.014+0000 7f9cec600640 1 -- 172.21.15.32:0/445147297 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9cc4068380 msgr2=0x7f9cc406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:11.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.014+0000 7f9cec600640 1 --2- 172.21.15.32:0/445147297 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9cc4068380 0x7f9cc406a840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4158b20 tx=0x7f9cd0008040 comp rx=0 tx=0).stop 2024-04-17T09:54:11.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.014+0000 7f9cec600640 1 -- 172.21.15.32:0/445147297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ce40cf370 msgr2=0x7f9ce413cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:11.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.014+0000 7f9cec600640 1 --2- 172.21.15.32:0/445147297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ce40cf370 0x7f9ce413cf50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800e870 tx=0x7f9cd800ec70 comp rx=0 tx=0).stop 2024-04-17T09:54:11.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.014+0000 7f9cec600640 1 -- 172.21.15.32:0/445147297 shutdown_connections 2024-04-17T09:54:11.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.014+0000 7f9cec600640 1 --2- 172.21.15.32:0/445147297 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9cc4068380 0x7f9cc406a840 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.014+0000 7f9cec600640 1 --2- 172.21.15.32:0/445147297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ce413f6e0 0x7f9ce41581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.014+0000 7f9cec600640 1 --2- 172.21.15.32:0/445147297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ce40d0820 0x7f9ce4157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.014+0000 7f9cec600640 1 --2- 172.21.15.32:0/445147297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ce40cf370 0x7f9ce413cf50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.014+0000 7f9cec600640 1 -- 172.21.15.32:0/445147297 >> 172.21.15.32:0/445147297 conn(0x7f9ce406ba30 msgr2=0x7f9ce4158e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:11.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.014+0000 7f9cec600640 1 -- 172.21.15.32:0/445147297 shutdown_connections 2024-04-17T09:54:11.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.014+0000 7f9cec600640 1 -- 172.21.15.32:0/445147297 wait complete. 2024-04-17T09:54:11.039 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-17T09:54:11.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.182+0000 7f1470600640 1 Processor -- start 2024-04-17T09:54:11.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.182+0000 7f1470600640 1 -- start start 2024-04-17T09:54:11.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.186+0000 7f1470600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1468144e90 0x7f14680b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:11.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.186+0000 7f1470600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f14680b4de0 0x7f146815baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:11.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.186+0000 7f1470600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1468072bb0 con 0x7f146815bfe0 2024-04-17T09:54:11.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.186+0000 7f1470600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f146815c430 con 0x7f1468144e90 2024-04-17T09:54:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.186+0000 7f1470600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1468145270 con 0x7f14680b4de0 2024-04-17T09:54:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.186+0000 7f146dc00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1468144e90 0x7f14680b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.186+0000 7f146dc00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1468144e90 0x7f14680b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43092/0 (socket says 172.21.15.32:43092) 2024-04-17T09:54:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.186+0000 7f146dc00640 1 -- 172.21.15.32:0/2059351912 learned_addr learned my addr 172.21.15.32:0/2059351912 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.186+0000 7f146d200640 1 --2- 172.21.15.32:0/2059351912 >> v2:172.21.15.144:3300/0 conn(0x7f14680b4de0 0x7f146815baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.186+0000 7f146dc00640 1 -- 172.21.15.32:0/2059351912 >> v2:172.21.15.144:3300/0 conn(0x7f14680b4de0 msgr2=0x7f146815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.186+0000 7f146dc00640 1 --2- 172.21.15.32:0/2059351912 >> v2:172.21.15.144:3300/0 conn(0x7f14680b4de0 0x7f146815baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.186+0000 7f146dc00640 1 -- 172.21.15.32:0/2059351912 >> v1:172.21.15.144:6789/0 conn(0x7f146815bfe0 legacy=0x7f146806ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.186+0000 7f146dc00640 1 -- 172.21.15.32:0/2059351912 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14680b3dc0 con 0x7f1468144e90 2024-04-17T09:54:11.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.190+0000 7f146dc00640 1 --2- 172.21.15.32:0/2059351912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1468144e90 0x7f14680b48a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f146806fb40 tx=0x7f145c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d262218507d517 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:11.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.190+0000 7f1467e00640 1 -- 172.21.15.32:0/2059351912 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c0aa610 con 0x7f1468144e90 2024-04-17T09:54:11.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.190+0000 7f1467e00640 1 -- 172.21.15.32:0/2059351912 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f145c0af070 con 0x7f1468144e90 2024-04-17T09:54:11.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.190+0000 7f1467e00640 1 -- 172.21.15.32:0/2059351912 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c0aa9d0 con 0x7f1468144e90 2024-04-17T09:54:11.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.190+0000 7f1470600640 1 -- 172.21.15.32:0/2059351912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1468144e90 msgr2=0x7f14680b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:11.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.190+0000 7f1470600640 1 --2- 172.21.15.32:0/2059351912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1468144e90 0x7f14680b48a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f146806fb40 tx=0x7f145c01c4b0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.190+0000 7f1470600640 1 -- 172.21.15.32:0/2059351912 shutdown_connections 2024-04-17T09:54:11.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.190+0000 7f1470600640 1 --2- 172.21.15.32:0/2059351912 >> v2:172.21.15.144:3300/0 conn(0x7f14680b4de0 0x7f146815baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.190+0000 7f1470600640 1 --2- 172.21.15.32:0/2059351912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1468144e90 0x7f14680b48a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.190+0000 7f1470600640 1 -- 172.21.15.32:0/2059351912 >> 172.21.15.32:0/2059351912 conn(0x7f146806ba30 msgr2=0x7f146806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:11.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.190+0000 7f1470600640 1 -- 172.21.15.32:0/2059351912 shutdown_connections 2024-04-17T09:54:11.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.190+0000 7f1470600640 1 -- 172.21.15.32:0/2059351912 wait complete. 2024-04-17T09:54:11.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.190+0000 7f1470600640 1 Processor -- start 2024-04-17T09:54:11.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.190+0000 7f1470600640 1 -- start start 2024-04-17T09:54:11.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f1470600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14680b4de0 0x7f1468140f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:11.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f1470600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1468144e90 0x7f146806de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:11.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f1470600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f146815bfe0 0x7f146806e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:11.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f1470600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1468072670 con 0x7f146815bfe0 2024-04-17T09:54:11.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f1470600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f146806f880 con 0x7f14680b4de0 2024-04-17T09:54:11.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f1470600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f14680727f0 con 0x7f1468144e90 2024-04-17T09:54:11.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f146e600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f146815bfe0 0x7f146806e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:11.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f146e600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f146815bfe0 0x7f146806e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43096/0 (socket says 172.21.15.32:43096) 2024-04-17T09:54:11.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f146e600640 1 -- 172.21.15.32:0/1530944852 learned_addr learned my addr 172.21.15.32:0/1530944852 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:11.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f146dc00640 1 --2- 172.21.15.32:0/1530944852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14680b4de0 0x7f1468140f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:11.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f146d200640 1 --2- 172.21.15.32:0/1530944852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1468144e90 0x7f146806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:11.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f146e600640 1 -- 172.21.15.32:0/1530944852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1468144e90 msgr2=0x7f146806de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:11.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f146e600640 1 --2- 172.21.15.32:0/1530944852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1468144e90 0x7f146806de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f146e600640 1 -- 172.21.15.32:0/1530944852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14680b4de0 msgr2=0x7f1468140f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:11.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f146e600640 1 --2- 172.21.15.32:0/1530944852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14680b4de0 0x7f1468140f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f146e600640 1 -- 172.21.15.32:0/1530944852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14681414d0 con 0x7f146815bfe0 2024-04-17T09:54:11.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f146dc00640 1 --2- 172.21.15.32:0/1530944852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14680b4de0 0x7f1468140f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:54:11.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f146e600640 1 --2- 172.21.15.32:0/1530944852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f146815bfe0 0x7f146806e390 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f146000b970 tx=0x7f146000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:11.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f1465600640 1 -- 172.21.15.32:0/1530944852 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460019070 con 0x7f146815bfe0 2024-04-17T09:54:11.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f1465600640 1 -- 172.21.15.32:0/1530944852 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f146000cb40 con 0x7f146815bfe0 2024-04-17T09:54:11.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f1465600640 1 -- 172.21.15.32:0/1530944852 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460014440 con 0x7f146815bfe0 2024-04-17T09:54:11.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f1470600640 1 -- 172.21.15.32:0/1530944852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f146813dab0 con 0x7f146815bfe0 2024-04-17T09:54:11.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f1470600640 1 -- 172.21.15.32:0/1530944852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f146813e070 con 0x7f146815bfe0 2024-04-17T09:54:11.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.194+0000 7f1470600640 1 -- 172.21.15.32:0/1530944852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1468068a90 con 0x7f146815bfe0 2024-04-17T09:54:11.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.198+0000 7f1465600640 1 -- 172.21.15.32:0/1530944852 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f146001c070 con 0x7f146815bfe0 2024-04-17T09:54:11.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.198+0000 7f1465600640 1 --2- 172.21.15.32:0/1530944852 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1448068380 0x7f144806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:11.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.198+0000 7f1465600640 1 -- 172.21.15.32:0/1530944852 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f1460086370 con 0x7f146815bfe0 2024-04-17T09:54:11.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.198+0000 7f146dc00640 1 --2- 172.21.15.32:0/1530944852 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1448068380 0x7f144806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:11.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.198+0000 7f146dc00640 1 --2- 172.21.15.32:0/1530944852 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1448068380 0x7f144806a840 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f145c0ae000 tx=0x7f145c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:11.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.202+0000 7f1465600640 1 -- 172.21.15.32:0/1530944852 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f14681414d0 con 0x7f146815bfe0 2024-04-17T09:54:11.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.426+0000 7f1470600640 1 -- 172.21.15.32:0/1530944852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f14680b1d00 con 0x7f146815bfe0 2024-04-17T09:54:11.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.426+0000 7f1465600640 1 -- 172.21.15.32:0/1530944852 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) ==== 80+0+6645 (secure 0 0 0) 0x7f1460056be0 con 0x7f146815bfe0 2024-04-17T09:54:11.426 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:11.426 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:54:11.427 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2024-04-17T09:54:11.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.427 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2024-04-17T09:54:11.427 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mzM1KExAAn0WH+Gf6gGV/dJonXzAgPQ==", 2024-04-17T09:54:11.427 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.427 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:54:11.427 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:54:11.427 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:54:11.427 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:54:11.427 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.427 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.427 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2024-04-17T09:54:11.427 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mQnJ4HxAAT0rFHhdAj7PMxZGiqEyWbg==", 2024-04-17T09:54:11.428 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.428 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:54:11.428 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:54:11.428 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:54:11.428 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:54:11.428 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.428 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.428 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2024-04-17T09:54:11.428 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mWWi5GhAA3RAdRxK6GRwuVCH4DHoMeA==", 2024-04-17T09:54:11.428 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.428 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:54:11.428 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:54:11.428 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:54:11.428 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:54:11.429 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.429 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2024-04-17T09:54:11.429 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9m9TMAJxAAbmUp05GHRE1AqCmq83scVQ==", 2024-04-17T09:54:11.429 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.429 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:54:11.429 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:54:11.429 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:54:11.429 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:54:11.429 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.429 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.429 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2024-04-17T09:54:11.429 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDRmx9mnPwyDBAAxD10jhxptVH5zhupengElg==", 2024-04-17T09:54:11.430 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.430 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:54:11.430 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:54:11.430 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:54:11.430 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.430 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.430 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.430 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2024-04-17T09:54:11.430 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUmx9mh3iuGBAAqvWSRyNX3u/TjEH6TJpqdA==", 2024-04-17T09:54:11.430 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.430 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:54:11.430 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:54:11.430 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:54:11.430 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.431 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2024-04-17T09:54:11.431 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXmx9mbe1CIRAA1pmzAl04tGkTu6QLpmCkag==", 2024-04-17T09:54:11.431 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.431 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:54:11.431 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:54:11.431 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:54:11.431 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.431 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2024-04-17T09:54:11.431 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDamx9mIwXoKxAAWPTFaYdp4lIZhuOU7F0x8g==", 2024-04-17T09:54:11.432 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.432 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:54:11.432 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:54:11.432 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:54:11.432 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.432 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.432 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2024-04-17T09:54:11.432 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDgmx9mp0s/JxAAv+usQqwrxgKINRlkDhhydQ==", 2024-04-17T09:54:11.432 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.432 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:54:11.432 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:54:11.432 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:54:11.432 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.433 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2024-04-17T09:54:11.433 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDjmx9mMVStMhAA/n8jk+LVDosWhziXk/lLRQ==", 2024-04-17T09:54:11.433 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.433 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:54:11.433 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:54:11.433 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:54:11.433 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.433 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.433 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2024-04-17T09:54:11.433 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDnmx9m/YQlABAAOqfWzVtgonD7b+WlR95mnA==", 2024-04-17T09:54:11.433 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.433 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:54:11.433 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:54:11.434 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:54:11.434 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.434 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2024-04-17T09:54:11.434 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDqmx9m572mCBAAqBib/yeQOBno+0B1K1dLxA==", 2024-04-17T09:54:11.434 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.434 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:54:11.434 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:54:11.434 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:54:11.434 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.434 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.434 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2024-04-17T09:54:11.434 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mUXoLKxAANV7zhLvL0DqHngk4TtsdBQ==", 2024-04-17T09:54:11.434 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.435 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:54:11.435 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-17T09:54:11.435 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-17T09:54:11.435 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-17T09:54:11.435 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.435 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2024-04-17T09:54:11.435 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mZBeRNBAAQg5K950mp6+XND7NCABAKQ==", 2024-04-17T09:54:11.435 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.435 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:54:11.435 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-17T09:54:11.435 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-17T09:54:11.435 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-17T09:54:11.436 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.436 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2024-04-17T09:54:11.436 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mWKstMRAAp18Hbnk02w+hT0sYZ3tSLA==", 2024-04-17T09:54:11.436 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.436 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:54:11.436 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-17T09:54:11.436 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-17T09:54:11.436 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-17T09:54:11.436 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.436 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2024-04-17T09:54:11.436 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mRQ2wOhAAzlWXFtP0WaXWmneqxnZ5/w==", 2024-04-17T09:54:11.436 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.437 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:54:11.437 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-17T09:54:11.437 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-17T09:54:11.437 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-17T09:54:11.437 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.437 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.437 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2024-04-17T09:54:11.437 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDOmx9maDStMxAAMCa36HfycwV3WbLg6Voo9Q==", 2024-04-17T09:54:11.437 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.437 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-17T09:54:11.437 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:54:11.437 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:54:11.437 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:54:11.437 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.438 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-mds", 2024-04-17T09:54:11.438 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDwmx9mqLohBxAAK9nGu/VG9pW8LURaiW35tA==", 2024-04-17T09:54:11.438 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.438 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-mds" 2024-04-17T09:54:11.438 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.438 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.438 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-mgr", 2024-04-17T09:54:11.438 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDwmx9mVcQhBxAAHBqkMnLlKH0zt4i2YFRKvg==", 2024-04-17T09:54:11.438 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.438 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-17T09:54:11.439 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.439 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-osd", 2024-04-17T09:54:11.439 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDwmx9m1cwhBxAAfSHokuRDHxd0MU7PFAc3eg==", 2024-04-17T09:54:11.439 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.439 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-osd" 2024-04-17T09:54:11.439 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.439 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.439 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rbd", 2024-04-17T09:54:11.439 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDwmx9mIdUhBxAAGx3g5Xp/MlnghqSOIA3qag==", 2024-04-17T09:54:11.439 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.439 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-17T09:54:11.439 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.440 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-17T09:54:11.440 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDwmx9mkd0hBxAAxYzhnmJbxY5xmVvcqTkIOQ==", 2024-04-17T09:54:11.440 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.440 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-17T09:54:11.440 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.440 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.440 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rgw", 2024-04-17T09:54:11.440 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDwmx9mMOchBxAAtomzVwhTPqmFtH6dy/pPDg==", 2024-04-17T09:54:11.440 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.440 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-17T09:54:11.440 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.441 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2024-04-17T09:54:11.441 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9myy0oDxAAPD2BTjIP09hiU5L7i6gUMg==", 2024-04-17T09:54:11.441 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.441 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-17T09:54:11.441 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-17T09:54:11.441 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:54:11.441 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.441 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:11.441 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:11.441 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2024-04-17T09:54:11.441 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9m3FmACxAApri5AI5PeR0+/HLev8DKcw==", 2024-04-17T09:54:11.441 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:54:11.442 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-17T09:54:11.442 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-17T09:54:11.442 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:54:11.442 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.442 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:11.442 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-17T09:54:11.442 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:54:11.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.426+0000 7f1470600640 1 -- 172.21.15.32:0/1530944852 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1448068380 msgr2=0x7f144806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:11.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.426+0000 7f1470600640 1 --2- 172.21.15.32:0/1530944852 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1448068380 0x7f144806a840 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f145c0ae000 tx=0x7f145c008040 comp rx=0 tx=0).stop 2024-04-17T09:54:11.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.426+0000 7f1470600640 1 -- 172.21.15.32:0/1530944852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f146815bfe0 msgr2=0x7f146806e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:11.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.426+0000 7f1470600640 1 --2- 172.21.15.32:0/1530944852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f146815bfe0 0x7f146806e390 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f146000b970 tx=0x7f146000be30 comp rx=0 tx=0).stop 2024-04-17T09:54:11.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.426+0000 7f1470600640 1 -- 172.21.15.32:0/1530944852 shutdown_connections 2024-04-17T09:54:11.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.426+0000 7f1470600640 1 --2- 172.21.15.32:0/1530944852 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1448068380 0x7f144806a840 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.426+0000 7f1470600640 1 --2- 172.21.15.32:0/1530944852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1468144e90 0x7f146806de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.426+0000 7f1470600640 1 --2- 172.21.15.32:0/1530944852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f146815bfe0 0x7f146806e390 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.426+0000 7f1470600640 1 --2- 172.21.15.32:0/1530944852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14680b4de0 0x7f1468140f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.426+0000 7f1470600640 1 -- 172.21.15.32:0/1530944852 >> 172.21.15.32:0/1530944852 conn(0x7f146806ba30 msgr2=0x7f14680cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:11.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.426+0000 7f1470600640 1 -- 172.21.15.32:0/1530944852 shutdown_connections 2024-04-17T09:54:11.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.426+0000 7f1470600640 1 -- 172.21.15.32:0/1530944852 wait complete. 2024-04-17T09:54:11.448 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-17T09:54:11.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.590+0000 7f2adca00640 1 Processor -- start 2024-04-17T09:54:11.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.590+0000 7f2adca00640 1 -- start start 2024-04-17T09:54:11.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.590+0000 7f2adca00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ad8155e50 0x7f2ad815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:11.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.590+0000 7f2adca00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ad815f870 con 0x7f2ad8155a70 2024-04-17T09:54:11.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.590+0000 7f2adca00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ad815fb50 con 0x7f2ad80b3340 2024-04-17T09:54:11.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.590+0000 7f2adca00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2ad80b42a0 con 0x7f2ad8155e50 2024-04-17T09:54:11.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.590+0000 7f2ad5e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2ad8155a70 0x7f2ad815d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55554/0 (socket says 172.21.15.32:55554) 2024-04-17T09:54:11.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.590+0000 7f2ad5e00640 1 -- 172.21.15.32:0/2787795649 learned_addr learned my addr 172.21.15.32:0/2787795649 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:11.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.590+0000 7f2acfe00640 1 --2- 172.21.15.32:0/2787795649 >> v2:172.21.15.32:3300/0 conn(0x7f2ad8155e50 0x7f2ad815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:11.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.590+0000 7f2ad5400640 1 -- 172.21.15.32:0/2787795649 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 705083407 0 0) 0x7f2ad815f870 con 0x7f2ad8155a70 2024-04-17T09:54:11.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.590+0000 7f2acfe00640 1 -- 172.21.15.32:0/2787795649 >> v1:172.21.15.144:6789/0 conn(0x7f2ad80b3340 legacy=0x7f2ad815c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:11.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.590+0000 7f2acfe00640 1 -- 172.21.15.32:0/2787795649 >> v1:172.21.15.32:6789/0 conn(0x7f2ad8155a70 legacy=0x7f2ad815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:11.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.590+0000 7f2acfe00640 1 -- 172.21.15.32:0/2787795649 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad815fe30 con 0x7f2ad8155e50 2024-04-17T09:54:11.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.594+0000 7f2acfe00640 1 --2- 172.21.15.32:0/2787795649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ad8155e50 0x7f2ad815f190 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80b4420 tx=0x7f2ac001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f414aa092116e42 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:11.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.594+0000 7f2ad5400640 1 -- 172.21.15.32:0/2787795649 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00b0070 con 0x7f2ad8155e50 2024-04-17T09:54:11.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.594+0000 7f2ad5400640 1 -- 172.21.15.32:0/2787795649 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac00ab600 con 0x7f2ad8155e50 2024-04-17T09:54:11.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.594+0000 7f2ad5400640 1 -- 172.21.15.32:0/2787795649 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00ab9d0 con 0x7f2ad8155e50 2024-04-17T09:54:11.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.594+0000 7f2adca00640 1 -- 172.21.15.32:0/2787795649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ad8155e50 msgr2=0x7f2ad815f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:11.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.594+0000 7f2adca00640 1 --2- 172.21.15.32:0/2787795649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ad8155e50 0x7f2ad815f190 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80b4420 tx=0x7f2ac001c440 comp rx=0 tx=0).stop 2024-04-17T09:54:11.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.594+0000 7f2adca00640 1 -- 172.21.15.32:0/2787795649 shutdown_connections 2024-04-17T09:54:11.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.594+0000 7f2adca00640 1 --2- 172.21.15.32:0/2787795649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ad8155e50 0x7f2ad815f190 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.594+0000 7f2adca00640 1 -- 172.21.15.32:0/2787795649 >> 172.21.15.32:0/2787795649 conn(0x7f2ad806ba30 msgr2=0x7f2ad806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:11.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.594+0000 7f2adca00640 1 -- 172.21.15.32:0/2787795649 shutdown_connections 2024-04-17T09:54:11.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.594+0000 7f2adca00640 1 -- 172.21.15.32:0/2787795649 wait complete. 2024-04-17T09:54:11.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2adca00640 1 Processor -- start 2024-04-17T09:54:11.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2adca00640 1 -- start start 2024-04-17T09:54:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2adca00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ad80b3340 0x7f2ad80ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2adca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ad8155a70 0x7f2ad815e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2adca00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ad8155e50 0x7f2ad815e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2adca00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad80d0170 con 0x7f2ad8155a70 2024-04-17T09:54:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2adca00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad80d02f0 con 0x7f2ad80b3340 2024-04-17T09:54:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2adca00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2ad80d0470 con 0x7f2ad8155e50 2024-04-17T09:54:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2acfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ad8155a70 0x7f2ad815e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2acfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ad8155a70 0x7f2ad815e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43122/0 (socket says 172.21.15.32:43122) 2024-04-17T09:54:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2acfe00640 1 -- 172.21.15.32:0/3337480522 learned_addr learned my addr 172.21.15.32:0/3337480522 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2ad6800640 1 --2- 172.21.15.32:0/3337480522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ad8155e50 0x7f2ad815e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2ad5e00640 1 --2- 172.21.15.32:0/3337480522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ad80b3340 0x7f2ad80ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2acfe00640 1 -- 172.21.15.32:0/3337480522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ad8155e50 msgr2=0x7f2ad815e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:11.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2acfe00640 1 --2- 172.21.15.32:0/3337480522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ad8155e50 0x7f2ad815e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2acfe00640 1 -- 172.21.15.32:0/3337480522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ad80b3340 msgr2=0x7f2ad80ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:11.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2acfe00640 1 --2- 172.21.15.32:0/3337480522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ad80b3340 0x7f2ad80ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2acfe00640 1 -- 172.21.15.32:0/3337480522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad80ce630 con 0x7f2ad8155a70 2024-04-17T09:54:11.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2acfe00640 1 --2- 172.21.15.32:0/3337480522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ad8155a70 0x7f2ad815e1b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00ae040 tx=0x7f2ac001ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:11.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2ace000640 1 -- 172.21.15.32:0/3337480522 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00b0050 con 0x7f2ad8155a70 2024-04-17T09:54:11.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2ace000640 1 -- 172.21.15.32:0/3337480522 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac00bc070 con 0x7f2ad8155a70 2024-04-17T09:54:11.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2ace000640 1 -- 172.21.15.32:0/3337480522 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00b5a70 con 0x7f2ad8155a70 2024-04-17T09:54:11.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2adca00640 1 -- 172.21.15.32:0/3337480522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad80ce910 con 0x7f2ad8155a70 2024-04-17T09:54:11.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2adca00640 1 -- 172.21.15.32:0/3337480522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad80ceed0 con 0x7f2ad8155a70 2024-04-17T09:54:11.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.598+0000 7f2adca00640 1 -- 172.21.15.32:0/3337480522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ad8068a90 con 0x7f2ad8155a70 2024-04-17T09:54:11.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.602+0000 7f2ace000640 1 -- 172.21.15.32:0/3337480522 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2ac00ab4c0 con 0x7f2ad8155a70 2024-04-17T09:54:11.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.602+0000 7f2ace000640 1 --2- 172.21.15.32:0/3337480522 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2ab40683d0 0x7f2ab406a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:11.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.602+0000 7f2ace000640 1 -- 172.21.15.32:0/3337480522 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f2ac0120e10 con 0x7f2ad8155a70 2024-04-17T09:54:11.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.602+0000 7f2ad5e00640 1 --2- 172.21.15.32:0/3337480522 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2ab40683d0 0x7f2ab406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:11.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.602+0000 7f2ad5e00640 1 --2- 172.21.15.32:0/3337480522 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2ab40683d0 0x7f2ab406a890 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8007fd0 tx=0x7f2ac8007d60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:11.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.606+0000 7f2ace000640 1 -- 172.21.15.32:0/3337480522 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2ac0124030 con 0x7f2ad8155a70 2024-04-17T09:54:11.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.826+0000 7f2adca00640 1 -- 172.21.15.32:0/3337480522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f2ad815f310 con 0x7f2ad8155a70 2024-04-17T09:54:11.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.830+0000 7f2ace000640 1 -- 172.21.15.32:0/3337480522 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f2ac0016030 con 0x7f2ad8155a70 2024-04-17T09:54:11.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.830+0000 7f2adca00640 1 -- 172.21.15.32:0/3337480522 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2ab40683d0 msgr2=0x7f2ab406a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:11.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.830+0000 7f2adca00640 1 --2- 172.21.15.32:0/3337480522 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2ab40683d0 0x7f2ab406a890 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8007fd0 tx=0x7f2ac8007d60 comp rx=0 tx=0).stop 2024-04-17T09:54:11.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.830+0000 7f2adca00640 1 -- 172.21.15.32:0/3337480522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ad8155a70 msgr2=0x7f2ad815e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:11.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.830+0000 7f2adca00640 1 --2- 172.21.15.32:0/3337480522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ad8155a70 0x7f2ad815e1b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00ae040 tx=0x7f2ac001ac80 comp rx=0 tx=0).stop 2024-04-17T09:54:11.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.830+0000 7f2adca00640 1 -- 172.21.15.32:0/3337480522 shutdown_connections 2024-04-17T09:54:11.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.830+0000 7f2adca00640 1 --2- 172.21.15.32:0/3337480522 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2ab40683d0 0x7f2ab406a890 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.830+0000 7f2adca00640 1 --2- 172.21.15.32:0/3337480522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ad8155e50 0x7f2ad815e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.830+0000 7f2adca00640 1 --2- 172.21.15.32:0/3337480522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ad8155a70 0x7f2ad815e1b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.830+0000 7f2adca00640 1 --2- 172.21.15.32:0/3337480522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ad80b3340 0x7f2ad80ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.830+0000 7f2adca00640 1 -- 172.21.15.32:0/3337480522 >> 172.21.15.32:0/3337480522 conn(0x7f2ad806ba30 msgr2=0x7f2ad814d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:11.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.830+0000 7f2adca00640 1 -- 172.21.15.32:0/3337480522 shutdown_connections 2024-04-17T09:54:11.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.830+0000 7f2adca00640 1 -- 172.21.15.32:0/3337480522 wait complete. 2024-04-17T09:54:11.851 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-17T09:54:11.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.990+0000 7f59bb600640 1 Processor -- start 2024-04-17T09:54:11.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.990+0000 7f59bb600640 1 -- start start 2024-04-17T09:54:11.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.994+0000 7f59bb600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59b413f6e0 0x7f59b40cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:11.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.994+0000 7f59bb600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59b40d09f0 0x7f59b4071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:11.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.994+0000 7f59bb600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59b41601e0 con 0x7f59b40cf370 2024-04-17T09:54:11.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.994+0000 7f59bb600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f59b413fac0 con 0x7f59b40d09f0 2024-04-17T09:54:11.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.994+0000 7f59bb600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f59b4160480 con 0x7f59b413f6e0 2024-04-17T09:54:11.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.994+0000 7f59b9600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59b40d09f0 0x7f59b4071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:11.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.994+0000 7f59b9600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59b40d09f0 0x7f59b4071d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43126/0 (socket says 172.21.15.32:43126) 2024-04-17T09:54:11.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.994+0000 7f59b9600640 1 -- 172.21.15.32:0/3342815189 learned_addr learned my addr 172.21.15.32:0/3342815189 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:11.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.994+0000 7f59b8c00640 1 --2- 172.21.15.32:0/3342815189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59b413f6e0 0x7f59b40cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:11.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.994+0000 7f59b9600640 1 -- 172.21.15.32:0/3342815189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59b413f6e0 msgr2=0x7f59b40cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:11.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.994+0000 7f59b9600640 1 --2- 172.21.15.32:0/3342815189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59b413f6e0 0x7f59b40cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.994+0000 7f59b9600640 1 -- 172.21.15.32:0/3342815189 >> v1:172.21.15.144:6789/0 conn(0x7f59b40cf370 legacy=0x7f59b40d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:11.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.994+0000 7f59b9600640 1 -- 172.21.15.32:0/3342815189 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b4160760 con 0x7f59b40d09f0 2024-04-17T09:54:11.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.994+0000 7f59b8c00640 1 --2- 172.21.15.32:0/3342815189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59b413f6e0 0x7f59b40cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:54:11.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.998+0000 7f59b9600640 1 --2- 172.21.15.32:0/3342815189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59b40d09f0 0x7f59b4071d80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f59ac009e50 tx=0x7f59ac01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19324e0806b43ea server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.998+0000 7f59b3400640 1 -- 172.21.15.32:0/3342815189 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac0a5840 con 0x7f59b40d09f0 2024-04-17T09:54:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.998+0000 7f59b3400640 1 -- 172.21.15.32:0/3342815189 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59ac0a59e0 con 0x7f59b40d09f0 2024-04-17T09:54:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.998+0000 7f59b3400640 1 -- 172.21.15.32:0/3342815189 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac0a5db0 con 0x7f59b40d09f0 2024-04-17T09:54:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.998+0000 7f59bb600640 1 -- 172.21.15.32:0/3342815189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59b40d09f0 msgr2=0x7f59b4071d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.998+0000 7f59bb600640 1 --2- 172.21.15.32:0/3342815189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59b40d09f0 0x7f59b4071d80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f59ac009e50 tx=0x7f59ac01c9c0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.998+0000 7f59bb600640 1 -- 172.21.15.32:0/3342815189 shutdown_connections 2024-04-17T09:54:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.998+0000 7f59bb600640 1 --2- 172.21.15.32:0/3342815189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59b40d09f0 0x7f59b4071d80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.998+0000 7f59bb600640 1 --2- 172.21.15.32:0/3342815189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59b413f6e0 0x7f59b40cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.998+0000 7f59bb600640 1 -- 172.21.15.32:0/3342815189 >> 172.21.15.32:0/3342815189 conn(0x7f59b406ba30 msgr2=0x7f59b406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.998+0000 7f59bb600640 1 -- 172.21.15.32:0/3342815189 shutdown_connections 2024-04-17T09:54:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.998+0000 7f59bb600640 1 -- 172.21.15.32:0/3342815189 wait complete. 2024-04-17T09:54:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.998+0000 7f59bb600640 1 Processor -- start 2024-04-17T09:54:12.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:11.998+0000 7f59bb600640 1 -- start start 2024-04-17T09:54:12.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59bb600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59b40cf370 0x7f59b40b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:12.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59bb600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59b40d09f0 0x7f59b40affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:12.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59bb600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59b413f6e0 0x7f59b41551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:12.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59bb600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f59b40cff10 con 0x7f59b413f6e0 2024-04-17T09:54:12.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59bb600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f59b40cfd90 con 0x7f59b40d09f0 2024-04-17T09:54:12.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59bb600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f59b4072ad0 con 0x7f59b40cf370 2024-04-17T09:54:12.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59b9600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59b413f6e0 0x7f59b41551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:12.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59b9600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59b413f6e0 0x7f59b41551a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43136/0 (socket says 172.21.15.32:43136) 2024-04-17T09:54:12.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59b9600640 1 -- 172.21.15.32:0/1837241038 learned_addr learned my addr 172.21.15.32:0/1837241038 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:12.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59b3e00640 1 --2- 172.21.15.32:0/1837241038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59b40d09f0 0x7f59b40affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:12.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59b8c00640 1 --2- 172.21.15.32:0/1837241038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59b40cf370 0x7f59b40b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59b9600640 1 -- 172.21.15.32:0/1837241038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59b40cf370 msgr2=0x7f59b40b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59b9600640 1 --2- 172.21.15.32:0/1837241038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59b40cf370 0x7f59b40b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59b9600640 1 -- 172.21.15.32:0/1837241038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59b40d09f0 msgr2=0x7f59b40affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59b9600640 1 --2- 172.21.15.32:0/1837241038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59b40d09f0 0x7f59b40affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59b9600640 1 -- 172.21.15.32:0/1837241038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b4137650 con 0x7f59b413f6e0 2024-04-17T09:54:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59b9600640 1 --2- 172.21.15.32:0/1837241038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59b413f6e0 0x7f59b41551a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f59ac01b070 tx=0x7f59ac019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59b0c00640 1 -- 172.21.15.32:0/1837241038 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac009270 con 0x7f59b413f6e0 2024-04-17T09:54:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59b0c00640 1 -- 172.21.15.32:0/1837241038 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59ac0be070 con 0x7f59b413f6e0 2024-04-17T09:54:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59b0c00640 1 -- 172.21.15.32:0/1837241038 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac0b8b00 con 0x7f59b413f6e0 2024-04-17T09:54:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59bb600640 1 -- 172.21.15.32:0/1837241038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b41378f0 con 0x7f59b413f6e0 2024-04-17T09:54:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59bb600640 1 -- 172.21.15.32:0/1837241038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b40b3440 con 0x7f59b413f6e0 2024-04-17T09:54:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59bb600640 1 -- 172.21.15.32:0/1837241038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b4068a90 con 0x7f59b413f6e0 2024-04-17T09:54:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.002+0000 7f59b0c00640 1 -- 172.21.15.32:0/1837241038 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f59ac0b8ca0 con 0x7f59b413f6e0 2024-04-17T09:54:12.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.006+0000 7f59b0c00640 1 --2- 172.21.15.32:0/1837241038 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5994068380 0x7f599406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:12.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.006+0000 7f59b0c00640 1 -- 172.21.15.32:0/1837241038 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f59ac121480 con 0x7f59b413f6e0 2024-04-17T09:54:12.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.006+0000 7f59b8c00640 1 --2- 172.21.15.32:0/1837241038 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5994068380 0x7f599406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:12.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.006+0000 7f59b8c00640 1 --2- 172.21.15.32:0/1837241038 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5994068380 0x7f599406a840 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f59a8006fd0 tx=0x7f59a8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:12.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.010+0000 7f59b0c00640 1 -- 172.21.15.32:0/1837241038 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f59ac0165e0 con 0x7f59b413f6e0 2024-04-17T09:54:12.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.230+0000 7f59bb600640 1 -- 172.21.15.32:0/1837241038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f59b40d1030 con 0x7f59b413f6e0 2024-04-17T09:54:12.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.234+0000 7f59b0c00640 1 -- 172.21.15.32:0/1837241038 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f59ac0a59b0 con 0x7f59b413f6e0 2024-04-17T09:54:12.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.238+0000 7f59bb600640 1 -- 172.21.15.32:0/1837241038 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5994068380 msgr2=0x7f599406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:12.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.238+0000 7f59bb600640 1 --2- 172.21.15.32:0/1837241038 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5994068380 0x7f599406a840 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f59a8006fd0 tx=0x7f59a8008040 comp rx=0 tx=0).stop 2024-04-17T09:54:12.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.238+0000 7f59bb600640 1 -- 172.21.15.32:0/1837241038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59b413f6e0 msgr2=0x7f59b41551a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:12.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.238+0000 7f59bb600640 1 --2- 172.21.15.32:0/1837241038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59b413f6e0 0x7f59b41551a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f59ac01b070 tx=0x7f59ac019760 comp rx=0 tx=0).stop 2024-04-17T09:54:12.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.238+0000 7f59bb600640 1 -- 172.21.15.32:0/1837241038 shutdown_connections 2024-04-17T09:54:12.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.238+0000 7f59bb600640 1 --2- 172.21.15.32:0/1837241038 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5994068380 0x7f599406a840 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:12.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.238+0000 7f59bb600640 1 --2- 172.21.15.32:0/1837241038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59b413f6e0 0x7f59b41551a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:12.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.238+0000 7f59bb600640 1 --2- 172.21.15.32:0/1837241038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59b40d09f0 0x7f59b40affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:12.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.238+0000 7f59bb600640 1 --2- 172.21.15.32:0/1837241038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59b40cf370 0x7f59b40b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:12.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.238+0000 7f59bb600640 1 -- 172.21.15.32:0/1837241038 >> 172.21.15.32:0/1837241038 conn(0x7f59b406ba30 msgr2=0x7f59b40d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:12.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.238+0000 7f59bb600640 1 -- 172.21.15.32:0/1837241038 shutdown_connections 2024-04-17T09:54:12.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.238+0000 7f59bb600640 1 -- 172.21.15.32:0/1837241038 wait complete. 2024-04-17T09:54:12.256 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-17T09:54:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.398+0000 7fad55400640 1 Processor -- start 2024-04-17T09:54:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.398+0000 7fad55400640 1 -- start start 2024-04-17T09:54:12.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.398+0000 7fad55400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fad50158ef0 0x7fad500721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:12.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.398+0000 7fad55400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad501631c0 con 0x7fad50162ba0 2024-04-17T09:54:12.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.398+0000 7fad55400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad5015ff80 con 0x7fad50072710 2024-04-17T09:54:12.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.398+0000 7fad55400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fad501592d0 con 0x7fad50158ef0 2024-04-17T09:54:12.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.398+0000 7fad4f200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fad50162ba0 0x7fad50160d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55566/0 (socket says 172.21.15.32:55566) 2024-04-17T09:54:12.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.398+0000 7fad4f200640 1 -- 172.21.15.32:0/810212372 learned_addr learned my addr 172.21.15.32:0/810212372 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:12.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.398+0000 7fad4e800640 1 --2- 172.21.15.32:0/810212372 >> v2:172.21.15.144:3300/0 conn(0x7fad50158ef0 0x7fad500721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:12.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.402+0000 7fad4de00640 1 -- 172.21.15.32:0/810212372 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1669881353 0 0) 0x7fad501631c0 con 0x7fad50162ba0 2024-04-17T09:54:12.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.402+0000 7fad4de00640 1 -- 172.21.15.32:0/810212372 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad34003530 con 0x7fad50162ba0 2024-04-17T09:54:12.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.402+0000 7fad4e800640 1 -- 172.21.15.32:0/810212372 >> v1:172.21.15.144:6789/0 conn(0x7fad50072710 legacy=0x7fad50162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:12.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.402+0000 7fad4de00640 1 -- 172.21.15.32:0/810212372 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4203400151 0 0) 0x7fad5015ff80 con 0x7fad50072710 2024-04-17T09:54:12.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.402+0000 7fad4e800640 1 -- 172.21.15.32:0/810212372 >> v1:172.21.15.32:6789/0 conn(0x7fad50162ba0 legacy=0x7fad50160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:12.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.402+0000 7fad4e800640 1 -- 172.21.15.32:0/810212372 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad50160260 con 0x7fad50158ef0 2024-04-17T09:54:12.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.402+0000 7fad4e800640 1 --2- 172.21.15.32:0/810212372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad50158ef0 0x7fad500721d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fad50159450 tx=0x7fad4001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a52d0dfdcd55a2e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:12.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.402+0000 7fad4de00640 1 -- 172.21.15.32:0/810212372 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad400b0070 con 0x7fad50158ef0 2024-04-17T09:54:12.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.402+0000 7fad4de00640 1 -- 172.21.15.32:0/810212372 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad400ab600 con 0x7fad50158ef0 2024-04-17T09:54:12.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.402+0000 7fad4de00640 1 -- 172.21.15.32:0/810212372 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad400ab9d0 con 0x7fad50158ef0 2024-04-17T09:54:12.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad55400640 1 -- 172.21.15.32:0/810212372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad50158ef0 msgr2=0x7fad500721d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:12.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad55400640 1 --2- 172.21.15.32:0/810212372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad50158ef0 0x7fad500721d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fad50159450 tx=0x7fad4001c4a0 comp rx=0 tx=0).stop 2024-04-17T09:54:12.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad55400640 1 -- 172.21.15.32:0/810212372 shutdown_connections 2024-04-17T09:54:12.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad55400640 1 --2- 172.21.15.32:0/810212372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad50158ef0 0x7fad500721d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:12.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad55400640 1 -- 172.21.15.32:0/810212372 >> 172.21.15.32:0/810212372 conn(0x7fad5006ba30 msgr2=0x7fad5006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:12.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad55400640 1 -- 172.21.15.32:0/810212372 shutdown_connections 2024-04-17T09:54:12.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad55400640 1 -- 172.21.15.32:0/810212372 wait complete. 2024-04-17T09:54:12.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad55400640 1 Processor -- start 2024-04-17T09:54:12.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad55400640 1 -- start start 2024-04-17T09:54:12.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad55400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad50072710 0x7fad501486b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:12.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad55400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad50158ef0 0x7fad5014fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:12.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad55400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad50162ba0 0x7fad501408a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:12.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad55400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fad50162070 con 0x7fad50162ba0 2024-04-17T09:54:12.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad55400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fad50161ef0 con 0x7fad50072710 2024-04-17T09:54:12.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad55400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fad501608b0 con 0x7fad50158ef0 2024-04-17T09:54:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad4f200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad50162ba0 0x7fad501408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad4f200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad50162ba0 0x7fad501408a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43140/0 (socket says 172.21.15.32:43140) 2024-04-17T09:54:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad4f200640 1 -- 172.21.15.32:0/3941510604 learned_addr learned my addr 172.21.15.32:0/3941510604 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad4e800640 1 --2- 172.21.15.32:0/3941510604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad50072710 0x7fad501486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad47e00640 1 --2- 172.21.15.32:0/3941510604 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad50158ef0 0x7fad5014fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad4f200640 1 -- 172.21.15.32:0/3941510604 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad50158ef0 msgr2=0x7fad5014fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad4f200640 1 --2- 172.21.15.32:0/3941510604 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad50158ef0 0x7fad5014fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad4f200640 1 -- 172.21.15.32:0/3941510604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad50072710 msgr2=0x7fad501486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad4f200640 1 --2- 172.21.15.32:0/3941510604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad50072710 0x7fad501486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad4f200640 1 -- 172.21.15.32:0/3941510604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad500702b0 con 0x7fad50162ba0 2024-04-17T09:54:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.406+0000 7fad4f200640 1 --2- 172.21.15.32:0/3941510604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad50162ba0 0x7fad501408a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fad4800caa0 tx=0x7fad4800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:12.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.410+0000 7fad46a00640 1 -- 172.21.15.32:0/3941510604 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48017070 con 0x7fad50162ba0 2024-04-17T09:54:12.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.410+0000 7fad46a00640 1 -- 172.21.15.32:0/3941510604 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad4800aba0 con 0x7fad50162ba0 2024-04-17T09:54:12.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.410+0000 7fad46a00640 1 -- 172.21.15.32:0/3941510604 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48012400 con 0x7fad50162ba0 2024-04-17T09:54:12.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.410+0000 7fad55400640 1 -- 172.21.15.32:0/3941510604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad50147610 con 0x7fad50162ba0 2024-04-17T09:54:12.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.410+0000 7fad55400640 1 -- 172.21.15.32:0/3941510604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad50147bd0 con 0x7fad50162ba0 2024-04-17T09:54:12.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.410+0000 7fad55400640 1 -- 172.21.15.32:0/3941510604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad50068a90 con 0x7fad50162ba0 2024-04-17T09:54:12.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.410+0000 7fad46a00640 1 -- 172.21.15.32:0/3941510604 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fad4801a070 con 0x7fad50162ba0 2024-04-17T09:54:12.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.410+0000 7fad46a00640 1 --2- 172.21.15.32:0/3941510604 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fad2c068380 0x7fad2c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:12.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.410+0000 7fad46a00640 1 -- 172.21.15.32:0/3941510604 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fad48086440 con 0x7fad50162ba0 2024-04-17T09:54:12.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.410+0000 7fad4e800640 1 --2- 172.21.15.32:0/3941510604 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fad2c068380 0x7fad2c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:12.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.410+0000 7fad4e800640 1 --2- 172.21.15.32:0/3941510604 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fad2c068380 0x7fad2c06a840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fad400ad000 tx=0x7fad400a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:12.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.414+0000 7fad46a00640 1 -- 172.21.15.32:0/3941510604 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fad480565d0 con 0x7fad50162ba0 2024-04-17T09:54:12.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.638+0000 7fad55400640 1 -- 172.21.15.32:0/3941510604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fad50158720 con 0x7fad50162ba0 2024-04-17T09:54:12.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.638+0000 7fad46a00640 1 -- 172.21.15.32:0/3941510604 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) ==== 90+0+0 (secure 0 0 0) 0x7fad4805a8d0 con 0x7fad50162ba0 2024-04-17T09:54:12.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.642+0000 7fad55400640 1 -- 172.21.15.32:0/3941510604 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fad2c068380 msgr2=0x7fad2c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:12.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.642+0000 7fad55400640 1 --2- 172.21.15.32:0/3941510604 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fad2c068380 0x7fad2c06a840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fad400ad000 tx=0x7fad400a6000 comp rx=0 tx=0).stop 2024-04-17T09:54:12.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.642+0000 7fad55400640 1 -- 172.21.15.32:0/3941510604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad50162ba0 msgr2=0x7fad501408a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:12.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.642+0000 7fad55400640 1 --2- 172.21.15.32:0/3941510604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad50162ba0 0x7fad501408a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fad4800caa0 tx=0x7fad4800cf60 comp rx=0 tx=0).stop 2024-04-17T09:54:12.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.642+0000 7fad55400640 1 -- 172.21.15.32:0/3941510604 shutdown_connections 2024-04-17T09:54:12.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.642+0000 7fad55400640 1 --2- 172.21.15.32:0/3941510604 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fad2c068380 0x7fad2c06a840 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:12.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.642+0000 7fad55400640 1 --2- 172.21.15.32:0/3941510604 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad50158ef0 0x7fad5014fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:12.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.642+0000 7fad55400640 1 --2- 172.21.15.32:0/3941510604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad50162ba0 0x7fad501408a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:12.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.642+0000 7fad55400640 1 --2- 172.21.15.32:0/3941510604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad50072710 0x7fad501486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:12.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.642+0000 7fad55400640 1 -- 172.21.15.32:0/3941510604 >> 172.21.15.32:0/3941510604 conn(0x7fad5006ba30 msgr2=0x7fad5006eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:12.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.642+0000 7fad55400640 1 -- 172.21.15.32:0/3941510604 shutdown_connections 2024-04-17T09:54:12.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.642+0000 7fad55400640 1 -- 172.21.15.32:0/3941510604 wait complete. 2024-04-17T09:54:12.662 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-17T09:54:12.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.802+0000 7f94bfc00640 1 Processor -- start 2024-04-17T09:54:12.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.802+0000 7f94bfc00640 1 -- start start 2024-04-17T09:54:12.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.802+0000 7f94bfc00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f94b813f6e0 0x7f94b80cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:12.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.802+0000 7f94bfc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94b80cf370 0x7f94b80d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:12.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.802+0000 7f94bfc00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94b81601e0 con 0x7f94b80d0820 2024-04-17T09:54:12.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.802+0000 7f94bfc00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f94b813fac0 con 0x7f94b813f6e0 2024-04-17T09:54:12.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.802+0000 7f94bfc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f94b8160480 con 0x7f94b80cf370 2024-04-17T09:54:12.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.802+0000 7f94afe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94b80cf370 0x7f94b80d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:12.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.802+0000 7f94bd200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f94b813f6e0 0x7f94b80cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:12.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.802+0000 7f94bd200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f94b813f6e0 0x7f94b80cf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:43932/0 (socket says 172.21.15.32:43932) 2024-04-17T09:54:12.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.802+0000 7f94afe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94b80cf370 0x7f94b80d02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:38290/0 (socket says 172.21.15.32:38290) 2024-04-17T09:54:12.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.802+0000 7f94bd200640 1 -- 172.21.15.32:0/1747245710 learned_addr learned my addr 172.21.15.32:0/1747245710 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:12.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.806+0000 7f94bd200640 1 -- 172.21.15.32:0/1747245710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94b80cf370 msgr2=0x7f94b80d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:12.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.806+0000 7f94bd200640 1 --2- 172.21.15.32:0/1747245710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94b80cf370 0x7f94b80d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:12.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.806+0000 7f94bd200640 1 -- 172.21.15.32:0/1747245710 >> v1:172.21.15.144:6789/0 conn(0x7f94b80d0820 legacy=0x7f94b8141d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:12.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.806+0000 7f94bd200640 1 -- 172.21.15.32:0/1747245710 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b8160760 con 0x7f94b813f6e0 2024-04-17T09:54:12.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.806+0000 7f94bd200640 1 --2- 172.21.15.32:0/1747245710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94b813f6e0 0x7f94b80cf850 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009870 tx=0x7f94a801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16b8a5a80e253385 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:12.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.806+0000 7f94b7e00640 1 -- 172.21.15.32:0/1747245710 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a80b2070 con 0x7f94b813f6e0 2024-04-17T09:54:12.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.806+0000 7f94b7e00640 1 -- 172.21.15.32:0/1747245710 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a80a9070 con 0x7f94b813f6e0 2024-04-17T09:54:12.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.806+0000 7f94b7e00640 1 -- 172.21.15.32:0/1747245710 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a80ad880 con 0x7f94b813f6e0 2024-04-17T09:54:12.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.806+0000 7f94bfc00640 1 -- 172.21.15.32:0/1747245710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94b813f6e0 msgr2=0x7f94b80cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:12.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.806+0000 7f94bfc00640 1 --2- 172.21.15.32:0/1747245710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94b813f6e0 0x7f94b80cf850 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009870 tx=0x7f94a801c560 comp rx=0 tx=0).stop 2024-04-17T09:54:12.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.806+0000 7f94bfc00640 1 -- 172.21.15.32:0/1747245710 shutdown_connections 2024-04-17T09:54:12.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.806+0000 7f94bfc00640 1 --2- 172.21.15.32:0/1747245710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94b80cf370 0x7f94b80d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:12.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.806+0000 7f94bfc00640 1 --2- 172.21.15.32:0/1747245710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94b813f6e0 0x7f94b80cf850 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:12.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.806+0000 7f94bfc00640 1 -- 172.21.15.32:0/1747245710 >> 172.21.15.32:0/1747245710 conn(0x7f94b806ba30 msgr2=0x7f94b806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:12.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.810+0000 7f94bfc00640 1 -- 172.21.15.32:0/1747245710 shutdown_connections 2024-04-17T09:54:12.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.810+0000 7f94bfc00640 1 -- 172.21.15.32:0/1747245710 wait complete. 2024-04-17T09:54:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.810+0000 7f94bfc00640 1 Processor -- start 2024-04-17T09:54:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.810+0000 7f94bfc00640 1 -- start start 2024-04-17T09:54:12.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.810+0000 7f94bfc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b80cf370 0x7f94b80b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:12.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.810+0000 7f94bfc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94b80d0820 0x7f94b813d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:12.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.810+0000 7f94bfc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94b813f6e0 0x7f94b815a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:12.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.810+0000 7f94bfc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f94b80cff10 con 0x7f94b80cf370 2024-04-17T09:54:12.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.810+0000 7f94bfc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f94b80cfd90 con 0x7f94b813f6e0 2024-04-17T09:54:12.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.810+0000 7f94bfc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f94b8169580 con 0x7f94b80d0820 2024-04-17T09:54:12.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.810+0000 7f94bd200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b80cf370 0x7f94b80b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:12.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.810+0000 7f94bd200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b80cf370 0x7f94b80b3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43150/0 (socket says 172.21.15.32:43150) 2024-04-17T09:54:12.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.810+0000 7f94bd200640 1 -- 172.21.15.32:0/696557274 learned_addr learned my addr 172.21.15.32:0/696557274 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:12.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.810+0000 7f94afe00640 1 --2- 172.21.15.32:0/696557274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94b80d0820 0x7f94b813d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:12.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.810+0000 7f94bdc00640 1 --2- 172.21.15.32:0/696557274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94b813f6e0 0x7f94b815a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:12.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.810+0000 7f94bd200640 1 -- 172.21.15.32:0/696557274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94b80d0820 msgr2=0x7f94b813d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:12.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.810+0000 7f94bd200640 1 --2- 172.21.15.32:0/696557274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94b80d0820 0x7f94b813d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:12.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.810+0000 7f94bd200640 1 -- 172.21.15.32:0/696557274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94b813f6e0 msgr2=0x7f94b815a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:12.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.810+0000 7f94bd200640 1 --2- 172.21.15.32:0/696557274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94b813f6e0 0x7f94b815a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:12.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.810+0000 7f94bd200640 1 -- 172.21.15.32:0/696557274 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b8159ec0 con 0x7f94b80cf370 2024-04-17T09:54:12.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.810+0000 7f94afe00640 1 --2- 172.21.15.32:0/696557274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94b80d0820 0x7f94b813d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:12.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.810+0000 7f94bdc00640 1 --2- 172.21.15.32:0/696557274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94b813f6e0 0x7f94b815a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:12.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.814+0000 7f94bd200640 1 --2- 172.21.15.32:0/696557274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b80cf370 0x7f94b80b3e50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f94a80b0040 tx=0x7f94a801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:12.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.814+0000 7f94b5600640 1 -- 172.21.15.32:0/696557274 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a80b2040 con 0x7f94b80cf370 2024-04-17T09:54:12.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.814+0000 7f94b5600640 1 -- 172.21.15.32:0/696557274 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a80be070 con 0x7f94b80cf370 2024-04-17T09:54:12.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.814+0000 7f94b5600640 1 -- 172.21.15.32:0/696557274 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a80b8420 con 0x7f94b80cf370 2024-04-17T09:54:12.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.814+0000 7f94bfc00640 1 -- 172.21.15.32:0/696557274 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b80b4430 con 0x7f94b80cf370 2024-04-17T09:54:12.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.814+0000 7f94bfc00640 1 -- 172.21.15.32:0/696557274 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b80b49f0 con 0x7f94b80cf370 2024-04-17T09:54:12.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.814+0000 7f94bfc00640 1 -- 172.21.15.32:0/696557274 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94b8068a90 con 0x7f94b80cf370 2024-04-17T09:54:12.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.814+0000 7f94b5600640 1 -- 172.21.15.32:0/696557274 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f94a80ab070 con 0x7f94b80cf370 2024-04-17T09:54:12.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.814+0000 7f94b5600640 1 --2- 172.21.15.32:0/696557274 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f94940683b0 0x7f949406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:12.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.814+0000 7f94b5600640 1 -- 172.21.15.32:0/696557274 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f94a8120770 con 0x7f94b80cf370 2024-04-17T09:54:12.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.814+0000 7f94afe00640 1 --2- 172.21.15.32:0/696557274 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f94940683b0 0x7f949406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:12.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.818+0000 7f94afe00640 1 --2- 172.21.15.32:0/696557274 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f94940683b0 0x7f949406a870 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f94b806eef0 tx=0x7f94a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:12.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:12.818+0000 7f94b5600640 1 -- 172.21.15.32:0/696557274 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f94a8016150 con 0x7f94b80cf370 2024-04-17T09:54:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.042+0000 7f94bfc00640 1 -- 172.21.15.32:0/696557274 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f94b8141800 con 0x7f94b80cf370 2024-04-17T09:54:13.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.046+0000 7f94b5600640 1 -- 172.21.15.32:0/696557274 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f94a80f5e20 con 0x7f94b80cf370 2024-04-17T09:54:13.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.046+0000 7f94bfc00640 1 -- 172.21.15.32:0/696557274 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f94940683b0 msgr2=0x7f949406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:13.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.046+0000 7f94bfc00640 1 --2- 172.21.15.32:0/696557274 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f94940683b0 0x7f949406a870 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f94b806eef0 tx=0x7f94a0008040 comp rx=0 tx=0).stop 2024-04-17T09:54:13.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.046+0000 7f94bfc00640 1 -- 172.21.15.32:0/696557274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b80cf370 msgr2=0x7f94b80b3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.046+0000 7f94bfc00640 1 --2- 172.21.15.32:0/696557274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b80cf370 0x7f94b80b3e50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f94a80b0040 tx=0x7f94a801ada0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.046+0000 7f94bfc00640 1 -- 172.21.15.32:0/696557274 shutdown_connections 2024-04-17T09:54:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.046+0000 7f94bfc00640 1 --2- 172.21.15.32:0/696557274 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f94940683b0 0x7f949406a870 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.046+0000 7f94bfc00640 1 --2- 172.21.15.32:0/696557274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94b80d0820 0x7f94b813d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.046+0000 7f94bfc00640 1 --2- 172.21.15.32:0/696557274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94b813f6e0 0x7f94b815a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.046+0000 7f94bfc00640 1 --2- 172.21.15.32:0/696557274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94b80cf370 0x7f94b80b3e50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.046+0000 7f94bfc00640 1 -- 172.21.15.32:0/696557274 >> 172.21.15.32:0/696557274 conn(0x7f94b806ba30 msgr2=0x7f94b806f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.046+0000 7f94bfc00640 1 -- 172.21.15.32:0/696557274 shutdown_connections 2024-04-17T09:54:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.046+0000 7f94bfc00640 1 -- 172.21.15.32:0/696557274 wait complete. 2024-04-17T09:54:13.067 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-17T09:54:13.076 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:54:13.116 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:54:13.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.210+0000 7f0c2ca00640 1 Processor -- start 2024-04-17T09:54:13.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.210+0000 7f0c2ca00640 1 -- start start 2024-04-17T09:54:13.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.210+0000 7f0c2ca00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0c280b3340 0x7f0c2815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:13.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.210+0000 7f0c2ca00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c2815f870 con 0x7f0c28155a70 2024-04-17T09:54:13.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.210+0000 7f0c2ca00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c2815fb50 con 0x7f0c28155e50 2024-04-17T09:54:13.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.210+0000 7f0c2ca00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0c280b42a0 con 0x7f0c280b3340 2024-04-17T09:54:13.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.214+0000 7f0c26800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0c280b3340 0x7f0c2815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:13.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.214+0000 7f0c25e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0c28155a70 0x7f0c2815d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55580/0 (socket says 172.21.15.32:55580) 2024-04-17T09:54:13.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.214+0000 7f0c25e00640 1 -- 172.21.15.32:0/3904377867 learned_addr learned my addr 172.21.15.32:0/3904377867 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:13.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.214+0000 7f0c26800640 1 -- 172.21.15.32:0/3904377867 >> v1:172.21.15.144:6789/0 conn(0x7f0c28155e50 legacy=0x7f0c2815c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:13.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.214+0000 7f0c26800640 1 -- 172.21.15.32:0/3904377867 >> v1:172.21.15.32:6789/0 conn(0x7f0c28155a70 legacy=0x7f0c2815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:13.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.214+0000 7f0c26800640 1 -- 172.21.15.32:0/3904377867 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2815fe30 con 0x7f0c280b3340 2024-04-17T09:54:13.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.214+0000 7f0c26800640 1 --2- 172.21.15.32:0/3904377867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c280b3340 0x7f0c2815f190 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0c280b4420 tx=0x7f0c2001eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6efab88e6116907 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:13.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.214+0000 7f0c25400640 1 -- 172.21.15.32:0/3904377867 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c200b4070 con 0x7f0c280b3340 2024-04-17T09:54:13.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.214+0000 7f0c25400640 1 -- 172.21.15.32:0/3904377867 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c200a7a30 con 0x7f0c280b3340 2024-04-17T09:54:13.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.214+0000 7f0c25400640 1 -- 172.21.15.32:0/3904377867 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c200a7e00 con 0x7f0c280b3340 2024-04-17T09:54:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c2ca00640 1 -- 172.21.15.32:0/3904377867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c280b3340 msgr2=0x7f0c2815f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c2ca00640 1 --2- 172.21.15.32:0/3904377867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c280b3340 0x7f0c2815f190 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0c280b4420 tx=0x7f0c2001eb60 comp rx=0 tx=0).stop 2024-04-17T09:54:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c2ca00640 1 -- 172.21.15.32:0/3904377867 shutdown_connections 2024-04-17T09:54:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c2ca00640 1 --2- 172.21.15.32:0/3904377867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c280b3340 0x7f0c2815f190 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c2ca00640 1 -- 172.21.15.32:0/3904377867 >> 172.21.15.32:0/3904377867 conn(0x7f0c2806ba30 msgr2=0x7f0c2806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c2ca00640 1 -- 172.21.15.32:0/3904377867 shutdown_connections 2024-04-17T09:54:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c2ca00640 1 -- 172.21.15.32:0/3904377867 wait complete. 2024-04-17T09:54:13.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c2ca00640 1 Processor -- start 2024-04-17T09:54:13.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c2ca00640 1 -- start start 2024-04-17T09:54:13.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c2ca00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c280b3340 0x7f0c28158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:13.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c2ca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c28155a70 0x7f0c2814c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:13.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c2ca00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c28155e50 0x7f0c28157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:13.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c2ca00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0c280d0170 con 0x7f0c28155a70 2024-04-17T09:54:13.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c2ca00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0c280d02f0 con 0x7f0c28155e50 2024-04-17T09:54:13.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c2ca00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0c280d0470 con 0x7f0c280b3340 2024-04-17T09:54:13.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c1d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c28155a70 0x7f0c2814c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:13.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c1d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c28155a70 0x7f0c2814c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43166/0 (socket says 172.21.15.32:43166) 2024-04-17T09:54:13.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c1d400640 1 -- 172.21.15.32:0/4029655009 learned_addr learned my addr 172.21.15.32:0/4029655009 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:13.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c25e00640 1 --2- 172.21.15.32:0/4029655009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c280b3340 0x7f0c28158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:13.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c26800640 1 --2- 172.21.15.32:0/4029655009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c28155e50 0x7f0c28157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:13.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c1d400640 1 -- 172.21.15.32:0/4029655009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c280b3340 msgr2=0x7f0c28158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:13.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c1d400640 1 --2- 172.21.15.32:0/4029655009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c280b3340 0x7f0c28158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c1d400640 1 -- 172.21.15.32:0/4029655009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c28155e50 msgr2=0x7f0c28157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:13.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c1d400640 1 --2- 172.21.15.32:0/4029655009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c28155e50 0x7f0c28157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c1d400640 1 -- 172.21.15.32:0/4029655009 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2815aef0 con 0x7f0c28155a70 2024-04-17T09:54:13.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.218+0000 7f0c1d400640 1 --2- 172.21.15.32:0/4029655009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c28155a70 0x7f0c2814c620 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000c370 tx=0x7f0c1000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:13.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.222+0000 7f0c1ea00640 1 -- 172.21.15.32:0/4029655009 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c100052d0 con 0x7f0c28155a70 2024-04-17T09:54:13.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.222+0000 7f0c1ea00640 1 -- 172.21.15.32:0/4029655009 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c10005470 con 0x7f0c28155a70 2024-04-17T09:54:13.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.222+0000 7f0c2ca00640 1 -- 172.21.15.32:0/4029655009 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c280ae430 con 0x7f0c28155a70 2024-04-17T09:54:13.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.222+0000 7f0c1ea00640 1 -- 172.21.15.32:0/4029655009 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c10007260 con 0x7f0c28155a70 2024-04-17T09:54:13.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.222+0000 7f0c2ca00640 1 -- 172.21.15.32:0/4029655009 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c281573a0 con 0x7f0c28155a70 2024-04-17T09:54:13.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.222+0000 7f0c2ca00640 1 -- 172.21.15.32:0/4029655009 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c28068a90 con 0x7f0c28155a70 2024-04-17T09:54:13.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.222+0000 7f0c1ea00640 1 -- 172.21.15.32:0/4029655009 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0c10007400 con 0x7f0c28155a70 2024-04-17T09:54:13.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.222+0000 7f0c1ea00640 1 --2- 172.21.15.32:0/4029655009 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0c04068380 0x7f0c0406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:13.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.222+0000 7f0c1ea00640 1 -- 172.21.15.32:0/4029655009 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f0c10085aa0 con 0x7f0c28155a70 2024-04-17T09:54:13.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.222+0000 7f0c25e00640 1 --2- 172.21.15.32:0/4029655009 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0c04068380 0x7f0c0406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:13.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.222+0000 7f0c25e00640 1 --2- 172.21.15.32:0/4029655009 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0c04068380 0x7f0c0406a840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0c18005890 tx=0x7f0c18009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:13.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.226+0000 7f0c1ea00640 1 -- 172.21.15.32:0/4029655009 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0c10055cc0 con 0x7f0c28155a70 2024-04-17T09:54:13.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.454+0000 7f0c2ca00640 1 -- 172.21.15.32:0/4029655009 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f0c28157170 con 0x7f0c28155a70 2024-04-17T09:54:13.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.458+0000 7f0c1ea00640 1 -- 172.21.15.32:0/4029655009 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) ==== 97+0+0 (secure 0 0 0) 0x7f0c10059fc0 con 0x7f0c28155a70 2024-04-17T09:54:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.462+0000 7f0c2ca00640 1 -- 172.21.15.32:0/4029655009 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0c04068380 msgr2=0x7f0c0406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:13.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.462+0000 7f0c2ca00640 1 --2- 172.21.15.32:0/4029655009 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0c04068380 0x7f0c0406a840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0c18005890 tx=0x7f0c18009040 comp rx=0 tx=0).stop 2024-04-17T09:54:13.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.462+0000 7f0c2ca00640 1 -- 172.21.15.32:0/4029655009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c28155a70 msgr2=0x7f0c2814c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:13.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.462+0000 7f0c2ca00640 1 --2- 172.21.15.32:0/4029655009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c28155a70 0x7f0c2814c620 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000c370 tx=0x7f0c1000c830 comp rx=0 tx=0).stop 2024-04-17T09:54:13.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.462+0000 7f0c2ca00640 1 -- 172.21.15.32:0/4029655009 shutdown_connections 2024-04-17T09:54:13.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.462+0000 7f0c2ca00640 1 --2- 172.21.15.32:0/4029655009 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0c04068380 0x7f0c0406a840 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.462+0000 7f0c2ca00640 1 --2- 172.21.15.32:0/4029655009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c28155e50 0x7f0c28157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.462+0000 7f0c2ca00640 1 --2- 172.21.15.32:0/4029655009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c28155a70 0x7f0c2814c620 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.462+0000 7f0c2ca00640 1 --2- 172.21.15.32:0/4029655009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c280b3340 0x7f0c28158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.462+0000 7f0c2ca00640 1 -- 172.21.15.32:0/4029655009 >> 172.21.15.32:0/4029655009 conn(0x7f0c2806ba30 msgr2=0x7f0c2806dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:13.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.462+0000 7f0c2ca00640 1 -- 172.21.15.32:0/4029655009 shutdown_connections 2024-04-17T09:54:13.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.462+0000 7f0c2ca00640 1 -- 172.21.15.32:0/4029655009 wait complete. 2024-04-17T09:54:13.481 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-17T09:54:13.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.622+0000 7f8bd7800640 1 Processor -- start 2024-04-17T09:54:13.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.622+0000 7f8bd7800640 1 -- start start 2024-04-17T09:54:13.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.622+0000 7f8bd7800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8bd0155a70 0x7f8bd015f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:13.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.622+0000 7f8bd7800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bd0155e50 0x7f8bd015d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:13.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.622+0000 7f8bd7800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8bd015d5d0 0x7f8bd015c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:13.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.622+0000 7f8bd7800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8bd00b42a0 con 0x7f8bd0155a70 2024-04-17T09:54:13.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.622+0000 7f8bd7800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f8bd00d1250 con 0x7f8bd015d5d0 2024-04-17T09:54:13.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.622+0000 7f8bd7800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8bd0160110 con 0x7f8bd0155e50 2024-04-17T09:54:13.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.622+0000 7f8bd4e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8bd0155a70 0x7f8bd015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:13.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.622+0000 7f8bd4e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8bd0155a70 0x7f8bd015f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43172/0 (socket says 172.21.15.32:43172) 2024-04-17T09:54:13.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.622+0000 7f8bd4e00640 1 -- 172.21.15.32:0/1283945625 learned_addr learned my addr 172.21.15.32:0/1283945625 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:13.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.626+0000 7f8bc7e00640 1 --2- 172.21.15.32:0/1283945625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bd0155e50 0x7f8bd015d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:13.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.626+0000 7f8bd5800640 1 --2- 172.21.15.32:0/1283945625 >> v2:172.21.15.144:3300/0 conn(0x7f8bd015d5d0 0x7f8bd015c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:13.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.626+0000 7f8bd4e00640 1 -- 172.21.15.32:0/1283945625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bd0155e50 msgr2=0x7f8bd015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:13.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.626+0000 7f8bd4e00640 1 --2- 172.21.15.32:0/1283945625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bd0155e50 0x7f8bd015d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.626+0000 7f8bd4e00640 1 -- 172.21.15.32:0/1283945625 >> v2:172.21.15.144:3300/0 conn(0x7f8bd015d5d0 msgr2=0x7f8bd015c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:13.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.626+0000 7f8bd4e00640 1 --2- 172.21.15.32:0/1283945625 >> v2:172.21.15.144:3300/0 conn(0x7f8bd015d5d0 0x7f8bd015c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.626+0000 7f8bd4e00640 1 -- 172.21.15.32:0/1283945625 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd015f870 con 0x7f8bd0155a70 2024-04-17T09:54:13.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.626+0000 7f8bd4e00640 1 --2- 172.21.15.32:0/1283945625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd0155a70 0x7f8bd015f190 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0160290 tx=0x7f8bc001eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e845e512e91ab6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:13.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.626+0000 7f8bcfe00640 1 -- 172.21.15.32:0/1283945625 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc00b4070 con 0x7f8bd0155a70 2024-04-17T09:54:13.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.626+0000 7f8bcfe00640 1 -- 172.21.15.32:0/1283945625 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc00a7c30 con 0x7f8bd0155a70 2024-04-17T09:54:13.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.626+0000 7f8bcfe00640 1 -- 172.21.15.32:0/1283945625 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc00af440 con 0x7f8bd0155a70 2024-04-17T09:54:13.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.626+0000 7f8bd7800640 1 -- 172.21.15.32:0/1283945625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd0155a70 msgr2=0x7f8bd015f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:13.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.626+0000 7f8bd7800640 1 --2- 172.21.15.32:0/1283945625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd0155a70 0x7f8bd015f190 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0160290 tx=0x7f8bc001eba0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd7800640 1 -- 172.21.15.32:0/1283945625 shutdown_connections 2024-04-17T09:54:13.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd7800640 1 --2- 172.21.15.32:0/1283945625 >> v2:172.21.15.144:3300/0 conn(0x7f8bd015d5d0 0x7f8bd015c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd7800640 1 --2- 172.21.15.32:0/1283945625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bd0155e50 0x7f8bd015d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd7800640 1 --2- 172.21.15.32:0/1283945625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd0155a70 0x7f8bd015f190 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd7800640 1 -- 172.21.15.32:0/1283945625 >> 172.21.15.32:0/1283945625 conn(0x7f8bd006ba30 msgr2=0x7f8bd006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:13.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd7800640 1 -- 172.21.15.32:0/1283945625 shutdown_connections 2024-04-17T09:54:13.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd7800640 1 -- 172.21.15.32:0/1283945625 wait complete. 2024-04-17T09:54:13.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd7800640 1 Processor -- start 2024-04-17T09:54:13.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd7800640 1 -- start start 2024-04-17T09:54:13.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd7800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bd0155a70 0x7f8bd0145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:13.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd7800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bd0155e50 0x7f8bd014ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:13.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd7800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd015d5d0 0x7f8bd0150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:13.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd7800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd00b5320 con 0x7f8bd015d5d0 2024-04-17T09:54:13.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd7800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd00b51a0 con 0x7f8bd0155e50 2024-04-17T09:54:13.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd7800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8bd00d0170 con 0x7f8bd0155a70 2024-04-17T09:54:13.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd5800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd015d5d0 0x7f8bd0150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:13.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd4e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bd0155a70 0x7f8bd0145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:13.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bc7e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bd0155e50 0x7f8bd014ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:13.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd5800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd015d5d0 0x7f8bd0150690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43182/0 (socket says 172.21.15.32:43182) 2024-04-17T09:54:13.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd5800640 1 -- 172.21.15.32:0/1769066656 learned_addr learned my addr 172.21.15.32:0/1769066656 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:13.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd5800640 1 -- 172.21.15.32:0/1769066656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bd0155a70 msgr2=0x7f8bd0145970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:13.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd5800640 1 --2- 172.21.15.32:0/1769066656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bd0155a70 0x7f8bd0145970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd5800640 1 -- 172.21.15.32:0/1769066656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bd0155e50 msgr2=0x7f8bd014ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:13.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd5800640 1 --2- 172.21.15.32:0/1769066656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bd0155e50 0x7f8bd014ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd5800640 1 -- 172.21.15.32:0/1769066656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd014c110 con 0x7f8bd015d5d0 2024-04-17T09:54:13.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bc7e00640 1 --2- 172.21.15.32:0/1769066656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bd0155e50 0x7f8bd014ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:54:13.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.630+0000 7f8bd5800640 1 --2- 172.21.15.32:0/1769066656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd015d5d0 0x7f8bd0150690 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8009fd0 tx=0x7f8bc800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:13.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.634+0000 7f8bcd600640 1 -- 172.21.15.32:0/1769066656 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8014420 con 0x7f8bd015d5d0 2024-04-17T09:54:13.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.634+0000 7f8bcd600640 1 -- 172.21.15.32:0/1769066656 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc8019070 con 0x7f8bd015d5d0 2024-04-17T09:54:13.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.634+0000 7f8bcd600640 1 -- 172.21.15.32:0/1769066656 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8014800 con 0x7f8bd015d5d0 2024-04-17T09:54:13.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.634+0000 7f8bd7800640 1 -- 172.21.15.32:0/1769066656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd014c3f0 con 0x7f8bd015d5d0 2024-04-17T09:54:13.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.634+0000 7f8bd7800640 1 -- 172.21.15.32:0/1769066656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd00cc5c0 con 0x7f8bd015d5d0 2024-04-17T09:54:13.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.634+0000 7f8bd7800640 1 -- 172.21.15.32:0/1769066656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd0068a90 con 0x7f8bd015d5d0 2024-04-17T09:54:13.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.634+0000 7f8bcd600640 1 -- 172.21.15.32:0/1769066656 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8bc80149a0 con 0x7f8bd015d5d0 2024-04-17T09:54:13.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.634+0000 7f8bcd600640 1 --2- 172.21.15.32:0/1769066656 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8bac068380 0x7f8bac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:13.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.634+0000 7f8bcd600640 1 -- 172.21.15.32:0/1769066656 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f8bc8086ce0 con 0x7f8bd015d5d0 2024-04-17T09:54:13.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.634+0000 7f8bd4e00640 1 --2- 172.21.15.32:0/1769066656 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8bac068380 0x7f8bac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:13.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.638+0000 7f8bd4e00640 1 --2- 172.21.15.32:0/1769066656 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8bac068380 0x7f8bac06a840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8bc00b1000 tx=0x7f8bc00a9070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:13.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.638+0000 7f8bcd600640 1 -- 172.21.15.32:0/1769066656 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f8bc8056f90 con 0x7f8bd015d5d0 2024-04-17T09:54:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.862+0000 7f8bd7800640 1 -- 172.21.15.32:0/1769066656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f8bd006f770 con 0x7f8bd015d5d0 2024-04-17T09:54:13.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.862+0000 7f8bcd600640 1 -- 172.21.15.32:0/1769066656 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) ==== 90+0+0 (secure 0 0 0) 0x7f8bc805b290 con 0x7f8bd015d5d0 2024-04-17T09:54:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.866+0000 7f8bd7800640 1 -- 172.21.15.32:0/1769066656 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8bac068380 msgr2=0x7f8bac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.866+0000 7f8bd7800640 1 --2- 172.21.15.32:0/1769066656 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8bac068380 0x7f8bac06a840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8bc00b1000 tx=0x7f8bc00a9070 comp rx=0 tx=0).stop 2024-04-17T09:54:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.866+0000 7f8bd7800640 1 -- 172.21.15.32:0/1769066656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd015d5d0 msgr2=0x7f8bd0150690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.866+0000 7f8bd7800640 1 --2- 172.21.15.32:0/1769066656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd015d5d0 0x7f8bd0150690 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8009fd0 tx=0x7f8bc800e4b0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.866+0000 7f8bd7800640 1 -- 172.21.15.32:0/1769066656 shutdown_connections 2024-04-17T09:54:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.866+0000 7f8bd7800640 1 --2- 172.21.15.32:0/1769066656 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8bac068380 0x7f8bac06a840 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.866+0000 7f8bd7800640 1 --2- 172.21.15.32:0/1769066656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd015d5d0 0x7f8bd0150690 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.866+0000 7f8bd7800640 1 --2- 172.21.15.32:0/1769066656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bd0155e50 0x7f8bd014ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.866+0000 7f8bd7800640 1 --2- 172.21.15.32:0/1769066656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bd0155a70 0x7f8bd0145970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.866+0000 7f8bd7800640 1 -- 172.21.15.32:0/1769066656 >> 172.21.15.32:0/1769066656 conn(0x7f8bd006ba30 msgr2=0x7f8bd006eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.866+0000 7f8bd7800640 1 -- 172.21.15.32:0/1769066656 shutdown_connections 2024-04-17T09:54:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:13.866+0000 7f8bd7800640 1 -- 172.21.15.32:0/1769066656 wait complete. 2024-04-17T09:54:13.886 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:54:14.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.026+0000 7f119b800640 1 Processor -- start 2024-04-17T09:54:14.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.026+0000 7f119b800640 1 -- start start 2024-04-17T09:54:14.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.026+0000 7f119b800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1194144e90 0x7f11940b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:14.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.026+0000 7f119b800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11940b4de0 0x7f119415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:14.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.026+0000 7f119b800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1194072bb0 con 0x7f119415bfe0 2024-04-17T09:54:14.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.026+0000 7f119b800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f119415c430 con 0x7f1194144e90 2024-04-17T09:54:14.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.026+0000 7f119b800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1194145270 con 0x7f11940b4de0 2024-04-17T09:54:14.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.026+0000 7f1199800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f119415bfe0 0x7f119406ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55586/0 (socket says 172.21.15.32:55586) 2024-04-17T09:54:14.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.026+0000 7f1199800640 1 -- 172.21.15.32:0/2135669631 learned_addr learned my addr 172.21.15.32:0/2135669631 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:14.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.026+0000 7f1198e00640 1 --2- 172.21.15.32:0/2135669631 >> v2:172.21.15.32:3300/0 conn(0x7f1194144e90 0x7f11940b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:14.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.030+0000 7f118be00640 1 --2- 172.21.15.32:0/2135669631 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11940b4de0 0x7f119415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:14.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.030+0000 7f1198e00640 1 -- 172.21.15.32:0/2135669631 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11940b4de0 msgr2=0x7f119415baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:14.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.030+0000 7f1198e00640 1 --2- 172.21.15.32:0/2135669631 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11940b4de0 0x7f119415baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.030+0000 7f1198e00640 1 -- 172.21.15.32:0/2135669631 >> v1:172.21.15.32:6789/0 conn(0x7f119415bfe0 legacy=0x7f119406ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:14.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.030+0000 7f1198e00640 1 -- 172.21.15.32:0/2135669631 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11940b3dc0 con 0x7f1194144e90 2024-04-17T09:54:14.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.030+0000 7f118be00640 1 --2- 172.21.15.32:0/2135669631 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11940b4de0 0x7f119415baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:14.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.030+0000 7f1198e00640 1 --2- 172.21.15.32:0/2135669631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1194144e90 0x7f11940b48a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f11840097f0 tx=0x7f118401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1546ab7f503028d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:14.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.030+0000 7f1193e00640 1 -- 172.21.15.32:0/2135669631 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11840aa5f0 con 0x7f1194144e90 2024-04-17T09:54:14.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.030+0000 7f1193e00640 1 -- 172.21.15.32:0/2135669631 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11840af070 con 0x7f1194144e90 2024-04-17T09:54:14.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.030+0000 7f1193e00640 1 -- 172.21.15.32:0/2135669631 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11840aa9b0 con 0x7f1194144e90 2024-04-17T09:54:14.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.030+0000 7f119b800640 1 -- 172.21.15.32:0/2135669631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1194144e90 msgr2=0x7f11940b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:14.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.030+0000 7f119b800640 1 --2- 172.21.15.32:0/2135669631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1194144e90 0x7f11940b48a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f11840097f0 tx=0x7f118401c470 comp rx=0 tx=0).stop 2024-04-17T09:54:14.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.030+0000 7f119b800640 1 -- 172.21.15.32:0/2135669631 shutdown_connections 2024-04-17T09:54:14.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.030+0000 7f119b800640 1 --2- 172.21.15.32:0/2135669631 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11940b4de0 0x7f119415baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.030+0000 7f119b800640 1 --2- 172.21.15.32:0/2135669631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1194144e90 0x7f11940b48a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.030+0000 7f119b800640 1 -- 172.21.15.32:0/2135669631 >> 172.21.15.32:0/2135669631 conn(0x7f119406ba30 msgr2=0x7f119406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:14.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f119b800640 1 -- 172.21.15.32:0/2135669631 shutdown_connections 2024-04-17T09:54:14.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f119b800640 1 -- 172.21.15.32:0/2135669631 wait complete. 2024-04-17T09:54:14.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f119b800640 1 Processor -- start 2024-04-17T09:54:14.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f119b800640 1 -- start start 2024-04-17T09:54:14.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f119b800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11940b4de0 0x7f1194141580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:14.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f119b800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1194144e90 0x7f1194141ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:14.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f119b800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f119415bfe0 0x7f11941fa970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:14.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f119b800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1194072670 con 0x7f119415bfe0 2024-04-17T09:54:14.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f119b800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f119406f8c0 con 0x7f11940b4de0 2024-04-17T09:54:14.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f119b800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f11940727f0 con 0x7f1194144e90 2024-04-17T09:54:14.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f1199800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f119415bfe0 0x7f11941fa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f1199800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f119415bfe0 0x7f11941fa970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43200/0 (socket says 172.21.15.32:43200) 2024-04-17T09:54:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f1199800640 1 -- 172.21.15.32:0/2266514836 learned_addr learned my addr 172.21.15.32:0/2266514836 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f1198e00640 1 --2- 172.21.15.32:0/2266514836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11940b4de0 0x7f1194141580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f118be00640 1 --2- 172.21.15.32:0/2266514836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1194144e90 0x7f1194141ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f1199800640 1 -- 172.21.15.32:0/2266514836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1194144e90 msgr2=0x7f1194141ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f1199800640 1 --2- 172.21.15.32:0/2266514836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1194144e90 0x7f1194141ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f1199800640 1 -- 172.21.15.32:0/2266514836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11940b4de0 msgr2=0x7f1194141580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:14.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f1199800640 1 --2- 172.21.15.32:0/2266514836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11940b4de0 0x7f1194141580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f1199800640 1 -- 172.21.15.32:0/2266514836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f119413c280 con 0x7f119415bfe0 2024-04-17T09:54:14.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f1198e00640 1 --2- 172.21.15.32:0/2266514836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11940b4de0 0x7f1194141580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:54:14.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f118be00640 1 --2- 172.21.15.32:0/2266514836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1194144e90 0x7f1194141ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:54:14.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f1199800640 1 --2- 172.21.15.32:0/2266514836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f119415bfe0 0x7f11941fa970 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f118c00ca80 tx=0x7f118c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f1191600640 1 -- 172.21.15.32:0/2266514836 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c017070 con 0x7f119415bfe0 2024-04-17T09:54:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f1191600640 1 -- 172.21.15.32:0/2266514836 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f118c00ac30 con 0x7f119415bfe0 2024-04-17T09:54:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f1191600640 1 -- 172.21.15.32:0/2266514836 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c012600 con 0x7f119415bfe0 2024-04-17T09:54:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f119b800640 1 -- 172.21.15.32:0/2266514836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f119413c560 con 0x7f119415bfe0 2024-04-17T09:54:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.034+0000 7f119b800640 1 -- 172.21.15.32:0/2266514836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f119413ce00 con 0x7f119415bfe0 2024-04-17T09:54:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.038+0000 7f119b800640 1 -- 172.21.15.32:0/2266514836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f119413c950 con 0x7f119415bfe0 2024-04-17T09:54:14.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.038+0000 7f1191600640 1 -- 172.21.15.32:0/2266514836 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f118c01a070 con 0x7f119415bfe0 2024-04-17T09:54:14.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.038+0000 7f1191600640 1 --2- 172.21.15.32:0/2266514836 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1170068380 0x7f117006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:14.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.038+0000 7f1191600640 1 -- 172.21.15.32:0/2266514836 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f118c086240 con 0x7f119415bfe0 2024-04-17T09:54:14.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.038+0000 7f1198e00640 1 --2- 172.21.15.32:0/2266514836 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1170068380 0x7f117006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:14.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.038+0000 7f1198e00640 1 --2- 172.21.15.32:0/2266514836 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1170068380 0x7f117006a840 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f11840ae000 tx=0x7f1184008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:14.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.042+0000 7f1191600640 1 -- 172.21.15.32:0/2266514836 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f118c00ae20 con 0x7f119415bfe0 2024-04-17T09:54:14.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.210+0000 7f119b800640 1 -- 172.21.15.32:0/2266514836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1194068a90 con 0x7f119415bfe0 2024-04-17T09:54:14.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.210+0000 7f1191600640 1 -- 172.21.15.32:0/2266514836 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+12903 (secure 0 0 0) 0x7f118c0564f0 con 0x7f119415bfe0 2024-04-17T09:54:14.210 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:14.210 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":31,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:54:11.012121+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:54:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.210+0000 7f119b800640 1 -- 172.21.15.32:0/2266514836 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1170068380 msgr2=0x7f117006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.210+0000 7f119b800640 1 --2- 172.21.15.32:0/2266514836 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1170068380 0x7f117006a840 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f11840ae000 tx=0x7f1184008040 comp rx=0 tx=0).stop 2024-04-17T09:54:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.210+0000 7f119b800640 1 -- 172.21.15.32:0/2266514836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f119415bfe0 msgr2=0x7f11941fa970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.210+0000 7f119b800640 1 --2- 172.21.15.32:0/2266514836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f119415bfe0 0x7f11941fa970 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f118c00ca80 tx=0x7f118c00cf40 comp rx=0 tx=0).stop 2024-04-17T09:54:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.210+0000 7f119b800640 1 -- 172.21.15.32:0/2266514836 shutdown_connections 2024-04-17T09:54:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.210+0000 7f119b800640 1 --2- 172.21.15.32:0/2266514836 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1170068380 0x7f117006a840 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.210+0000 7f119b800640 1 --2- 172.21.15.32:0/2266514836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1194144e90 0x7f1194141ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.210+0000 7f119b800640 1 --2- 172.21.15.32:0/2266514836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f119415bfe0 0x7f11941fa970 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.210+0000 7f119b800640 1 --2- 172.21.15.32:0/2266514836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11940b4de0 0x7f1194141580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.210+0000 7f119b800640 1 -- 172.21.15.32:0/2266514836 >> 172.21.15.32:0/2266514836 conn(0x7f119406ba30 msgr2=0x7f119406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.210+0000 7f119b800640 1 -- 172.21.15.32:0/2266514836 shutdown_connections 2024-04-17T09:54:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.210+0000 7f119b800640 1 -- 172.21.15.32:0/2266514836 wait complete. 2024-04-17T09:54:14.231 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:54:14.231 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:54:14.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.373+0000 7f99da400640 1 Processor -- start 2024-04-17T09:54:14.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.373+0000 7f99da400640 1 -- start start 2024-04-17T09:54:14.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.373+0000 7f99da400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f99d406f7c0 0x7f99d406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:14.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.373+0000 7f99da400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99d406f270 0x7f99d406bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:14.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.373+0000 7f99da400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99d40cc940 con 0x7f99d415b7d0 2024-04-17T09:54:14.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.373+0000 7f99da400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f99d40ccc20 con 0x7f99d406f7c0 2024-04-17T09:54:14.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.373+0000 7f99da400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f99d406e0b0 con 0x7f99d406f270 2024-04-17T09:54:14.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.377+0000 7f99cbe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f99d406f7c0 0x7f99d406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:14.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.377+0000 7f99cbe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f99d406f7c0 0x7f99d406ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43206/0 (socket says 172.21.15.32:43206) 2024-04-17T09:54:14.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.377+0000 7f99d3400640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f99d415b7d0 0x7f99d406e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:51940/0 (socket says 172.21.15.32:51940) 2024-04-17T09:54:14.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.377+0000 7f99d3e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99d406f270 0x7f99d406bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:14.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.377+0000 7f99cbe00640 1 -- 172.21.15.32:0/2303609958 learned_addr learned my addr 172.21.15.32:0/2303609958 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:14.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.377+0000 7f99cbe00640 1 -- 172.21.15.32:0/2303609958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99d406f270 msgr2=0x7f99d406bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:14.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.377+0000 7f99cbe00640 1 --2- 172.21.15.32:0/2303609958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99d406f270 0x7f99d406bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.377+0000 7f99cbe00640 1 -- 172.21.15.32:0/2303609958 >> v1:172.21.15.144:6789/0 conn(0x7f99d415b7d0 legacy=0x7f99d406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:14.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.377+0000 7f99cbe00640 1 -- 172.21.15.32:0/2303609958 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d406e390 con 0x7f99d406f7c0 2024-04-17T09:54:14.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.377+0000 7f99d2a00640 1 -- 172.21.15.32:0/2303609958 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1014850256 0 0) 0x7f99d40cc940 con 0x7f99d415b7d0 2024-04-17T09:54:14.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.377+0000 7f99cbe00640 1 --2- 172.21.15.32:0/2303609958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99d406f7c0 0x7f99d406ba30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009870 tx=0x7f99bc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce33045db6087fd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:14.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.377+0000 7f99d2a00640 1 -- 172.21.15.32:0/2303609958 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc0aa650 con 0x7f99d406f7c0 2024-04-17T09:54:14.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.377+0000 7f99d2a00640 1 -- 172.21.15.32:0/2303609958 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99bc0af070 con 0x7f99d406f7c0 2024-04-17T09:54:14.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.377+0000 7f99d2a00640 1 -- 172.21.15.32:0/2303609958 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc0aaa10 con 0x7f99d406f7c0 2024-04-17T09:54:14.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99da400640 1 -- 172.21.15.32:0/2303609958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99d406f7c0 msgr2=0x7f99d406ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:14.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99da400640 1 --2- 172.21.15.32:0/2303609958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99d406f7c0 0x7f99d406ba30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009870 tx=0x7f99bc01c4d0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99da400640 1 -- 172.21.15.32:0/2303609958 shutdown_connections 2024-04-17T09:54:14.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99da400640 1 --2- 172.21.15.32:0/2303609958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99d406f270 0x7f99d406bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99da400640 1 --2- 172.21.15.32:0/2303609958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99d406f7c0 0x7f99d406ba30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99da400640 1 -- 172.21.15.32:0/2303609958 >> 172.21.15.32:0/2303609958 conn(0x7f99d40b2120 msgr2=0x7f99d40b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99da400640 1 -- 172.21.15.32:0/2303609958 shutdown_connections 2024-04-17T09:54:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99da400640 1 -- 172.21.15.32:0/2303609958 wait complete. 2024-04-17T09:54:14.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99da400640 1 Processor -- start 2024-04-17T09:54:14.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99da400640 1 -- start start 2024-04-17T09:54:14.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99da400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99d406f270 0x7f99d40ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:14.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99da400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99d406f7c0 0x7f99d41557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:14.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99da400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99d415b7d0 0x7f99d4140d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:14.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99da400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f99d40ceeb0 con 0x7f99d406f7c0 2024-04-17T09:54:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99da400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f99d40ced30 con 0x7f99d415b7d0 2024-04-17T09:54:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99da400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f99d41682a0 con 0x7f99d406f270 2024-04-17T09:54:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99cbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99d406f7c0 0x7f99d41557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99cbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99d406f7c0 0x7f99d41557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43208/0 (socket says 172.21.15.32:43208) 2024-04-17T09:54:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99cbe00640 1 -- 172.21.15.32:0/2083539622 learned_addr learned my addr 172.21.15.32:0/2083539622 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99d3400640 1 --2- 172.21.15.32:0/2083539622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99d406f270 0x7f99d40ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99d3e00640 1 --2- 172.21.15.32:0/2083539622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99d415b7d0 0x7f99d4140d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99cbe00640 1 -- 172.21.15.32:0/2083539622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99d406f270 msgr2=0x7f99d40ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99cbe00640 1 --2- 172.21.15.32:0/2083539622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99d406f270 0x7f99d40ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99cbe00640 1 -- 172.21.15.32:0/2083539622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99d415b7d0 msgr2=0x7f99d4140d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99cbe00640 1 --2- 172.21.15.32:0/2083539622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99d415b7d0 0x7f99d4140d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.381+0000 7f99cbe00640 1 -- 172.21.15.32:0/2083539622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d41583d0 con 0x7f99d406f7c0 2024-04-17T09:54:14.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.385+0000 7f99cbe00640 1 --2- 172.21.15.32:0/2083539622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99d406f7c0 0x7f99d41557d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0ac040 tx=0x7f99bc01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:14.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.385+0000 7f99cb400640 1 -- 172.21.15.32:0/2083539622 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc0b9070 con 0x7f99d406f7c0 2024-04-17T09:54:14.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.385+0000 7f99cb400640 1 -- 172.21.15.32:0/2083539622 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99bc0a6040 con 0x7f99d406f7c0 2024-04-17T09:54:14.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.385+0000 7f99cb400640 1 -- 172.21.15.32:0/2083539622 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc004020 con 0x7f99d406f7c0 2024-04-17T09:54:14.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.385+0000 7f99da400640 1 -- 172.21.15.32:0/2083539622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d41586b0 con 0x7f99d406f7c0 2024-04-17T09:54:14.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.385+0000 7f99da400640 1 -- 172.21.15.32:0/2083539622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d4153dd0 con 0x7f99d406f7c0 2024-04-17T09:54:14.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.385+0000 7f99da400640 1 -- 172.21.15.32:0/2083539622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99d4068a90 con 0x7f99d406f7c0 2024-04-17T09:54:14.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.385+0000 7f99cb400640 1 -- 172.21.15.32:0/2083539622 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f99bc0041c0 con 0x7f99d406f7c0 2024-04-17T09:54:14.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.385+0000 7f99cb400640 1 --2- 172.21.15.32:0/2083539622 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f99b0068380 0x7f99b006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:14.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.385+0000 7f99cb400640 1 -- 172.21.15.32:0/2083539622 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f99bc120310 con 0x7f99d406f7c0 2024-04-17T09:54:14.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.385+0000 7f99d3400640 1 --2- 172.21.15.32:0/2083539622 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f99b0068380 0x7f99b006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:14.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.389+0000 7f99d3400640 1 --2- 172.21.15.32:0/2083539622 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f99b0068380 0x7f99b006a840 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f99c4000f70 tx=0x7f99c4007240 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:14.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.389+0000 7f99cb400640 1 -- 172.21.15.32:0/2083539622 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f99bc0c4030 con 0x7f99d406f7c0 2024-04-17T09:54:14.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.553+0000 7f99da400640 1 -- 172.21.15.32:0/2083539622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f99d41530b0 con 0x7f99d406f7c0 2024-04-17T09:54:14.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.553+0000 7f99cb400640 1 -- 172.21.15.32:0/2083539622 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) ==== 93+0+10 (secure 0 0 0) 0x7f99bc016150 con 0x7f99d406f7c0 2024-04-17T09:54:14.556 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:54:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.557+0000 7f99da400640 1 -- 172.21.15.32:0/2083539622 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f99b0068380 msgr2=0x7f99b006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.557+0000 7f99da400640 1 --2- 172.21.15.32:0/2083539622 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f99b0068380 0x7f99b006a840 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f99c4000f70 tx=0x7f99c4007240 comp rx=0 tx=0).stop 2024-04-17T09:54:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.557+0000 7f99da400640 1 -- 172.21.15.32:0/2083539622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99d406f7c0 msgr2=0x7f99d41557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.557+0000 7f99da400640 1 --2- 172.21.15.32:0/2083539622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99d406f7c0 0x7f99d41557d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0ac040 tx=0x7f99bc01ad10 comp rx=0 tx=0).stop 2024-04-17T09:54:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.557+0000 7f99da400640 1 -- 172.21.15.32:0/2083539622 shutdown_connections 2024-04-17T09:54:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.557+0000 7f99da400640 1 --2- 172.21.15.32:0/2083539622 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f99b0068380 0x7f99b006a840 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.557+0000 7f99da400640 1 --2- 172.21.15.32:0/2083539622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99d415b7d0 0x7f99d4140d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.557+0000 7f99da400640 1 --2- 172.21.15.32:0/2083539622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99d406f7c0 0x7f99d41557d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.557+0000 7f99da400640 1 --2- 172.21.15.32:0/2083539622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99d406f270 0x7f99d40ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.557+0000 7f99da400640 1 -- 172.21.15.32:0/2083539622 >> 172.21.15.32:0/2083539622 conn(0x7f99d40b2120 msgr2=0x7f99d40b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:14.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.557+0000 7f99da400640 1 -- 172.21.15.32:0/2083539622 shutdown_connections 2024-04-17T09:54:14.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.557+0000 7f99da400640 1 -- 172.21.15.32:0/2083539622 wait complete. 2024-04-17T09:54:14.578 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T09:54:14.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.717+0000 7faed6a00640 1 Processor -- start 2024-04-17T09:54:14.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.717+0000 7faed6a00640 1 -- start start 2024-04-17T09:54:14.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.717+0000 7faed6a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faed006f670 0x7faed015bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:14.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.717+0000 7faed6a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faed0072bb0 con 0x7faed0144e90 2024-04-17T09:54:14.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.717+0000 7faed6a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faed00b3dc0 con 0x7faed015baa0 2024-04-17T09:54:14.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.717+0000 7faed6a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7faed0145270 con 0x7faed006f670 2024-04-17T09:54:14.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.717+0000 7faecfe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7faed0144e90 0x7faed00b48a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55592/0 (socket says 172.21.15.32:55592) 2024-04-17T09:54:14.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.717+0000 7faecfe00640 1 -- 172.21.15.32:0/1545316798 learned_addr learned my addr 172.21.15.32:0/1545316798 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:14.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.717+0000 7faed4a00640 1 --2- 172.21.15.32:0/1545316798 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faed006f670 0x7faed015bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:14.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.717+0000 7faecf400640 1 -- 172.21.15.32:0/1545316798 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3945799534 0 0) 0x7faed0072bb0 con 0x7faed0144e90 2024-04-17T09:54:14.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.717+0000 7faecf400640 1 -- 172.21.15.32:0/1545316798 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faeb4003530 con 0x7faed0144e90 2024-04-17T09:54:14.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.717+0000 7faecf400640 1 -- 172.21.15.32:0/1545316798 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 19028389 0 0) 0x7faed00b3dc0 con 0x7faed015baa0 2024-04-17T09:54:14.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.717+0000 7faecf400640 1 -- 172.21.15.32:0/1545316798 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faed0072bb0 con 0x7faed015baa0 2024-04-17T09:54:14.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.717+0000 7faed4a00640 1 -- 172.21.15.32:0/1545316798 >> v1:172.21.15.144:6789/0 conn(0x7faed015baa0 legacy=0x7faed006ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:14.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.717+0000 7faed4a00640 1 -- 172.21.15.32:0/1545316798 >> v1:172.21.15.32:6789/0 conn(0x7faed0144e90 legacy=0x7faed00b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:14.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.717+0000 7faed4a00640 1 -- 172.21.15.32:0/1545316798 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed00b40a0 con 0x7faed006f670 2024-04-17T09:54:14.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.721+0000 7faed4a00640 1 --2- 172.21.15.32:0/1545316798 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faed006f670 0x7faed015bf30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faec8009dc0 tx=0x7faec801c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7222735b9efee844 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:14.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.721+0000 7faecf400640 1 -- 172.21.15.32:0/1545316798 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec80b2070 con 0x7faed006f670 2024-04-17T09:54:14.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.721+0000 7faecf400640 1 -- 172.21.15.32:0/1545316798 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faec80a5960 con 0x7faed006f670 2024-04-17T09:54:14.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.721+0000 7faecf400640 1 -- 172.21.15.32:0/1545316798 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec80a5d30 con 0x7faed006f670 2024-04-17T09:54:14.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.721+0000 7faed6a00640 1 -- 172.21.15.32:0/1545316798 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faed006f670 msgr2=0x7faed015bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:14.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.721+0000 7faed6a00640 1 --2- 172.21.15.32:0/1545316798 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faed006f670 0x7faed015bf30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faec8009dc0 tx=0x7faec801c990 comp rx=0 tx=0).stop 2024-04-17T09:54:14.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.721+0000 7faed6a00640 1 -- 172.21.15.32:0/1545316798 shutdown_connections 2024-04-17T09:54:14.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.721+0000 7faed6a00640 1 --2- 172.21.15.32:0/1545316798 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faed006f670 0x7faed015bf30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.721+0000 7faed6a00640 1 -- 172.21.15.32:0/1545316798 >> 172.21.15.32:0/1545316798 conn(0x7faed006ba30 msgr2=0x7faed006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:14.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.721+0000 7faed6a00640 1 -- 172.21.15.32:0/1545316798 shutdown_connections 2024-04-17T09:54:14.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.721+0000 7faed6a00640 1 -- 172.21.15.32:0/1545316798 wait complete. 2024-04-17T09:54:14.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faed6a00640 1 Processor -- start 2024-04-17T09:54:14.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faed6a00640 1 -- start start 2024-04-17T09:54:14.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faed6a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faed006f670 0x7faed013d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:14.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faed6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed0144e90 0x7faed0157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:14.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faed6a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faed015baa0 0x7faed00b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:14.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faed6a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faed0169120 con 0x7faed0144e90 2024-04-17T09:54:14.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faed6a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faed0072900 con 0x7faed006f670 2024-04-17T09:54:14.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faed6a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7faed01692a0 con 0x7faed015baa0 2024-04-17T09:54:14.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faec7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed0144e90 0x7faed0157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:14.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faec7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed0144e90 0x7faed0157a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43222/0 (socket says 172.21.15.32:43222) 2024-04-17T09:54:14.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faec7e00640 1 -- 172.21.15.32:0/2218530900 learned_addr learned my addr 172.21.15.32:0/2218530900 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:14.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faecfe00640 1 --2- 172.21.15.32:0/2218530900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faed006f670 0x7faed013d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:14.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faed4a00640 1 --2- 172.21.15.32:0/2218530900 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faed015baa0 0x7faed00b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:14.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faec7e00640 1 -- 172.21.15.32:0/2218530900 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faed015baa0 msgr2=0x7faed00b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:14.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faec7e00640 1 --2- 172.21.15.32:0/2218530900 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faed015baa0 0x7faed00b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faec7e00640 1 -- 172.21.15.32:0/2218530900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faed006f670 msgr2=0x7faed013d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:14.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faec7e00640 1 --2- 172.21.15.32:0/2218530900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faed006f670 0x7faed013d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faec7e00640 1 -- 172.21.15.32:0/2218530900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed0144620 con 0x7faed0144e90 2024-04-17T09:54:14.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faed4a00640 1 --2- 172.21.15.32:0/2218530900 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faed015baa0 0x7faed00b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:14.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faec7e00640 1 --2- 172.21.15.32:0/2218530900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed0144e90 0x7faed0157a70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7faeb800c530 tx=0x7faeb800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:14.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faeccc00640 1 -- 172.21.15.32:0/2218530900 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb80052d0 con 0x7faed0144e90 2024-04-17T09:54:14.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faeccc00640 1 -- 172.21.15.32:0/2218530900 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeb8005490 con 0x7faed0144e90 2024-04-17T09:54:14.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faed6a00640 1 -- 172.21.15.32:0/2218530900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed00cb8e0 con 0x7faed0144e90 2024-04-17T09:54:14.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faeccc00640 1 -- 172.21.15.32:0/2218530900 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb8012850 con 0x7faed0144e90 2024-04-17T09:54:14.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.725+0000 7faed6a00640 1 -- 172.21.15.32:0/2218530900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed013fc30 con 0x7faed0144e90 2024-04-17T09:54:14.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.729+0000 7faed6a00640 1 -- 172.21.15.32:0/2218530900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faed0068a90 con 0x7faed0144e90 2024-04-17T09:54:14.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.729+0000 7faeccc00640 1 -- 172.21.15.32:0/2218530900 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7faeb80129f0 con 0x7faed0144e90 2024-04-17T09:54:14.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.729+0000 7faeccc00640 1 --2- 172.21.15.32:0/2218530900 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faeac0683b0 0x7faeac06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:14.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.729+0000 7faeccc00640 1 -- 172.21.15.32:0/2218530900 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7faeb8086280 con 0x7faed0144e90 2024-04-17T09:54:14.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.729+0000 7faecfe00640 1 --2- 172.21.15.32:0/2218530900 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faeac0683b0 0x7faeac06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:14.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.729+0000 7faecfe00640 1 --2- 172.21.15.32:0/2218530900 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faeac0683b0 0x7faeac06a870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faec00011f0 tx=0x7faec0007d40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:14.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.733+0000 7faeccc00640 1 -- 172.21.15.32:0/2218530900 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7faeb8056b00 con 0x7faed0144e90 2024-04-17T09:54:14.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.897+0000 7faed6a00640 1 -- 172.21.15.32:0/2218530900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7faed0142940 con 0x7faed0144e90 2024-04-17T09:54:14.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.901+0000 7faeccc00640 1 -- 172.21.15.32:0/2218530900 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) ==== 84+0+65 (secure 0 0 0) 0x7faeb8086080 con 0x7faed0144e90 2024-04-17T09:54:14.901 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:14.901 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-17T09:54:14.901 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:54:14.901 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-04-17T09:54:14.901 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-04-17T09:54:14.901 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:14.901 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-17T09:54:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.901+0000 7faed6a00640 1 -- 172.21.15.32:0/2218530900 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faeac0683b0 msgr2=0x7faeac06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.901+0000 7faed6a00640 1 --2- 172.21.15.32:0/2218530900 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faeac0683b0 0x7faeac06a870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faec00011f0 tx=0x7faec0007d40 comp rx=0 tx=0).stop 2024-04-17T09:54:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.901+0000 7faed6a00640 1 -- 172.21.15.32:0/2218530900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed0144e90 msgr2=0x7faed0157a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.901+0000 7faed6a00640 1 --2- 172.21.15.32:0/2218530900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed0144e90 0x7faed0157a70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7faeb800c530 tx=0x7faeb800c9f0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.901+0000 7faed6a00640 1 -- 172.21.15.32:0/2218530900 shutdown_connections 2024-04-17T09:54:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.901+0000 7faed6a00640 1 --2- 172.21.15.32:0/2218530900 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faeac0683b0 0x7faeac06a870 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.901+0000 7faed6a00640 1 --2- 172.21.15.32:0/2218530900 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faed015baa0 0x7faed00b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.901+0000 7faed6a00640 1 --2- 172.21.15.32:0/2218530900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed0144e90 0x7faed0157a70 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.901+0000 7faed6a00640 1 --2- 172.21.15.32:0/2218530900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faed006f670 0x7faed013d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.901+0000 7faed6a00640 1 -- 172.21.15.32:0/2218530900 >> 172.21.15.32:0/2218530900 conn(0x7faed006ba30 msgr2=0x7faed0142d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.901+0000 7faed6a00640 1 -- 172.21.15.32:0/2218530900 shutdown_connections 2024-04-17T09:54:14.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:14.901+0000 7faed6a00640 1 -- 172.21.15.32:0/2218530900 wait complete. 2024-04-17T09:54:14.921 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T09:54:14.922 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T09:54:15.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.065+0000 7f6a38a00640 1 Processor -- start 2024-04-17T09:54:15.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.065+0000 7f6a38a00640 1 -- start start 2024-04-17T09:54:15.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.065+0000 7f6a38a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a340cd870 0x7f6a341661a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:15.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.065+0000 7f6a38a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a3415f200 con 0x7f6a3406f500 2024-04-17T09:54:15.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.065+0000 7f6a38a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a3415f4e0 con 0x7f6a340cfc70 2024-04-17T09:54:15.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.065+0000 7f6a38a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6a3415fb30 con 0x7f6a340cd870 2024-04-17T09:54:15.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.065+0000 7f6a2be00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6a3406f500 0x7f6a340cd160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55598/0 (socket says 172.21.15.32:55598) 2024-04-17T09:54:15.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.065+0000 7f6a2be00640 1 -- 172.21.15.32:0/624915034 learned_addr learned my addr 172.21.15.32:0/624915034 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:15.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.065+0000 7f6a32800640 1 --2- 172.21.15.32:0/624915034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a340cd870 0x7f6a341661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:15.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.065+0000 7f6a31400640 1 -- 172.21.15.32:0/624915034 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1962539708 0 0) 0x7f6a3415f200 con 0x7f6a3406f500 2024-04-17T09:54:15.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.065+0000 7f6a31400640 1 -- 172.21.15.32:0/624915034 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a18003530 con 0x7f6a3406f500 2024-04-17T09:54:15.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.065+0000 7f6a31400640 1 -- 172.21.15.32:0/624915034 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3472023670 0 0) 0x7f6a18003530 con 0x7f6a3406f500 2024-04-17T09:54:15.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.065+0000 7f6a31400640 1 -- 172.21.15.32:0/624915034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a340cd870 msgr2=0x7f6a341661a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:15.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.065+0000 7f6a31400640 1 --2- 172.21.15.32:0/624915034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a340cd870 0x7f6a341661a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:15.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.065+0000 7f6a31400640 1 -- 172.21.15.32:0/624915034 >> v1:172.21.15.144:6789/0 conn(0x7f6a340cfc70 legacy=0x7f6a3406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:15.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.065+0000 7f6a31400640 1 -- 172.21.15.32:0/624915034 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3415f7c0 con 0x7f6a3406f500 2024-04-17T09:54:15.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.065+0000 7f6a31400640 1 -- 172.21.15.32:0/624915034 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f6a1c001800 con 0x7f6a3406f500 2024-04-17T09:54:15.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.065+0000 7f6a31400640 1 -- 172.21.15.32:0/624915034 >> v1:172.21.15.32:6789/0 conn(0x7f6a3406f500 legacy=0x7f6a340cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:15.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.065+0000 7f6a31400640 1 --2- 172.21.15.32:0/624915034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a180039f0 0x7f6a18003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:15.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.069+0000 7f6a31400640 1 --2- 172.21.15.32:0/624915034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a340cd870 0x7f6a18004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:15.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.069+0000 7f6a31400640 1 --2- 172.21.15.32:0/624915034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a18004d50 0x7f6a180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:15.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.069+0000 7f6a31400640 1 -- 172.21.15.32:0/624915034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3415fb30 con 0x7f6a180039f0 2024-04-17T09:54:15.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.069+0000 7f6a31400640 1 -- 172.21.15.32:0/624915034 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1800ba80 con 0x7f6a18004d50 2024-04-17T09:54:15.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.069+0000 7f6a31400640 1 -- 172.21.15.32:0/624915034 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6a1800de90 con 0x7f6a340cd870 2024-04-17T09:54:15.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.069+0000 7f6a31e00640 1 --2- 172.21.15.32:0/624915034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a180039f0 0x7f6a18003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:15.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.069+0000 7f6a2be00640 1 --2- 172.21.15.32:0/624915034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a340cd870 0x7f6a18004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:15.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.069+0000 7f6a32800640 1 --2- 172.21.15.32:0/624915034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a18004d50 0x7f6a180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:15.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.069+0000 7f6a31e00640 1 -- 172.21.15.32:0/624915034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a340cd870 msgr2=0x7f6a18004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:15.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.069+0000 7f6a31e00640 1 --2- 172.21.15.32:0/624915034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a340cd870 0x7f6a18004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:15.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.069+0000 7f6a31e00640 1 -- 172.21.15.32:0/624915034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a18004d50 msgr2=0x7f6a180051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:15.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.069+0000 7f6a31e00640 1 --2- 172.21.15.32:0/624915034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a18004d50 0x7f6a180051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:15.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.069+0000 7f6a31e00640 1 -- 172.21.15.32:0/624915034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a18003530 con 0x7f6a180039f0 2024-04-17T09:54:15.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.069+0000 7f6a32800640 1 --2- 172.21.15.32:0/624915034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a18004d50 0x7f6a180051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:54:15.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.069+0000 7f6a31e00640 1 --2- 172.21.15.32:0/624915034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a180039f0 0x7f6a18003e10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6a24001750 tx=0x7f6a2401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383fd3c46bd1814c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:15.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.069+0000 7f6a31400640 1 -- 172.21.15.32:0/624915034 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a240013a0 con 0x7f6a180039f0 2024-04-17T09:54:15.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.069+0000 7f6a31400640 1 -- 172.21.15.32:0/624915034 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a240052f0 con 0x7f6a180039f0 2024-04-17T09:54:15.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.069+0000 7f6a31400640 1 -- 172.21.15.32:0/624915034 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24003840 con 0x7f6a180039f0 2024-04-17T09:54:15.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.069+0000 7f6a38a00640 1 -- 172.21.15.32:0/624915034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a180039f0 msgr2=0x7f6a18003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:15.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.069+0000 7f6a38a00640 1 --2- 172.21.15.32:0/624915034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a180039f0 0x7f6a18003e10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6a24001750 tx=0x7f6a2401c3e0 comp rx=0 tx=0).stop 2024-04-17T09:54:15.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a32800640 1 -- 172.21.15.32:0/624915034 reap_dead start 2024-04-17T09:54:15.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a38a00640 1 -- 172.21.15.32:0/624915034 shutdown_connections 2024-04-17T09:54:15.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a38a00640 1 -- 172.21.15.32:0/624915034 >> 172.21.15.32:0/624915034 conn(0x7f6a340b2190 msgr2=0x7f6a340b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:15.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a38a00640 1 -- 172.21.15.32:0/624915034 shutdown_connections 2024-04-17T09:54:15.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a38a00640 1 -- 172.21.15.32:0/624915034 wait complete. 2024-04-17T09:54:15.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a38a00640 1 Processor -- start 2024-04-17T09:54:15.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a38a00640 1 -- start start 2024-04-17T09:54:15.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a38a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a180039f0 0x7f6a34154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:15.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a38a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a34150160 0x7f6a3415b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:15.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a38a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a3415bf20 0x7f6a34157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:15.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a38a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3406e740 con 0x7f6a3415bf20 2024-04-17T09:54:15.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a38a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6a340d0260 con 0x7f6a180039f0 2024-04-17T09:54:15.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a38a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6a3406e8c0 con 0x7f6a34150160 2024-04-17T09:54:15.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a32800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a3415bf20 0x7f6a34157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:15.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a32800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a3415bf20 0x7f6a34157470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43250/0 (socket says 172.21.15.32:43250) 2024-04-17T09:54:15.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a32800640 1 -- 172.21.15.32:0/1690544142 learned_addr learned my addr 172.21.15.32:0/1690544142 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:15.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a31e00640 1 --2- 172.21.15.32:0/1690544142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a180039f0 0x7f6a34154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:15.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a2be00640 1 --2- 172.21.15.32:0/1690544142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a34150160 0x7f6a3415b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:15.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a32800640 1 -- 172.21.15.32:0/1690544142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a34150160 msgr2=0x7f6a3415b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:15.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a32800640 1 --2- 172.21.15.32:0/1690544142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a34150160 0x7f6a3415b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:15.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a32800640 1 -- 172.21.15.32:0/1690544142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a180039f0 msgr2=0x7f6a34154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:15.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a32800640 1 --2- 172.21.15.32:0/1690544142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a180039f0 0x7f6a34154c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:15.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a32800640 1 -- 172.21.15.32:0/1690544142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3415a260 con 0x7f6a3415bf20 2024-04-17T09:54:15.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a2be00640 1 --2- 172.21.15.32:0/1690544142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a34150160 0x7f6a3415b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:15.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a32800640 1 --2- 172.21.15.32:0/1690544142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a3415bf20 0x7f6a34157470 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6a340702c0 tx=0x7f6a2c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:15.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.073+0000 7f6a2a000640 1 -- 172.21.15.32:0/1690544142 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c01c070 con 0x7f6a3415bf20 2024-04-17T09:54:15.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.077+0000 7f6a2a000640 1 -- 172.21.15.32:0/1690544142 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a2c017400 con 0x7f6a3415bf20 2024-04-17T09:54:15.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.077+0000 7f6a38a00640 1 -- 172.21.15.32:0/1690544142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a3415a540 con 0x7f6a3415bf20 2024-04-17T09:54:15.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.077+0000 7f6a2a000640 1 -- 172.21.15.32:0/1690544142 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c0177f0 con 0x7f6a3415bf20 2024-04-17T09:54:15.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.077+0000 7f6a38a00640 1 -- 172.21.15.32:0/1690544142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a34156c00 con 0x7f6a3415bf20 2024-04-17T09:54:15.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.077+0000 7f6a38a00640 1 -- 172.21.15.32:0/1690544142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a3415c3a0 con 0x7f6a3415bf20 2024-04-17T09:54:15.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.077+0000 7f6a2a000640 1 -- 172.21.15.32:0/1690544142 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6a2c01f070 con 0x7f6a3415bf20 2024-04-17T09:54:15.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.077+0000 7f6a2a000640 1 --2- 172.21.15.32:0/1690544142 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6a100683b0 0x7f6a1006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:15.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.077+0000 7f6a2a000640 1 -- 172.21.15.32:0/1690544142 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f6a2c086590 con 0x7f6a3415bf20 2024-04-17T09:54:15.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.077+0000 7f6a31e00640 1 --2- 172.21.15.32:0/1690544142 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6a100683b0 0x7f6a1006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:15.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.077+0000 7f6a31e00640 1 --2- 172.21.15.32:0/1690544142 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6a100683b0 0x7f6a1006a870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6a180057f0 tx=0x7f6a240011a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:15.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.081+0000 7f6a2a000640 1 -- 172.21.15.32:0/1690544142 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f6a2c012070 con 0x7f6a3415bf20 2024-04-17T09:54:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.249+0000 7f6a38a00640 1 -- 172.21.15.32:0/1690544142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6a34068a90 con 0x7f6a3415bf20 2024-04-17T09:54:15.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.333+0000 7f6a2a000640 1 -- 172.21.15.32:0/1690544142 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) ==== 138+0+0 (secure 0 0 0) 0x7f6a2c0567b0 con 0x7f6a3415bf20 2024-04-17T09:54:15.334 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-04-17T09:54:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.333+0000 7f6a38a00640 1 -- 172.21.15.32:0/1690544142 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6a100683b0 msgr2=0x7f6a1006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.333+0000 7f6a38a00640 1 --2- 172.21.15.32:0/1690544142 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6a100683b0 0x7f6a1006a870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6a180057f0 tx=0x7f6a240011a0 comp rx=0 tx=0).stop 2024-04-17T09:54:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.333+0000 7f6a38a00640 1 -- 172.21.15.32:0/1690544142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a3415bf20 msgr2=0x7f6a34157470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.333+0000 7f6a38a00640 1 --2- 172.21.15.32:0/1690544142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a3415bf20 0x7f6a34157470 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6a340702c0 tx=0x7f6a2c00f4a0 comp rx=0 tx=0).stop 2024-04-17T09:54:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.333+0000 7f6a38a00640 1 -- 172.21.15.32:0/1690544142 shutdown_connections 2024-04-17T09:54:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.333+0000 7f6a38a00640 1 --2- 172.21.15.32:0/1690544142 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6a100683b0 0x7f6a1006a870 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.333+0000 7f6a38a00640 1 --2- 172.21.15.32:0/1690544142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a34150160 0x7f6a3415b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.333+0000 7f6a38a00640 1 --2- 172.21.15.32:0/1690544142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a3415bf20 0x7f6a34157470 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.333+0000 7f6a38a00640 1 --2- 172.21.15.32:0/1690544142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a180039f0 0x7f6a34154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.333+0000 7f6a38a00640 1 -- 172.21.15.32:0/1690544142 >> 172.21.15.32:0/1690544142 conn(0x7f6a340b2190 msgr2=0x7f6a340b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.337+0000 7f6a38a00640 1 -- 172.21.15.32:0/1690544142 shutdown_connections 2024-04-17T09:54:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.337+0000 7f6a38a00640 1 -- 172.21.15.32:0/1690544142 wait complete. 2024-04-17T09:54:15.361 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-17T09:54:15.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.509+0000 7f4861c00640 1 Processor -- start 2024-04-17T09:54:15.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.509+0000 7f4861c00640 1 -- start start 2024-04-17T09:54:15.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.509+0000 7f4861c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f485c06c140 0x7f485c0adb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:15.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.509+0000 7f4861c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f485c1629a0 con 0x7f485c15c4c0 2024-04-17T09:54:15.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.509+0000 7f4861c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f485c162c80 con 0x7f485c15f9c0 2024-04-17T09:54:15.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.509+0000 7f4861c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f485c162f60 con 0x7f485c06c140 2024-04-17T09:54:15.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.509+0000 7f485aa00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f485c15c4c0 0x7f485c06ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55604/0 (socket says 172.21.15.32:55604) 2024-04-17T09:54:15.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.509+0000 7f485aa00640 1 -- 172.21.15.32:0/3218629476 learned_addr learned my addr 172.21.15.32:0/3218629476 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:15.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.509+0000 7f485a000640 1 --2- 172.21.15.32:0/3218629476 >> v2:172.21.15.144:3300/0 conn(0x7f485c06c140 0x7f485c0adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:15.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.509+0000 7f4859600640 1 -- 172.21.15.32:0/3218629476 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1268296302 0 0) 0x7f485c1629a0 con 0x7f485c15c4c0 2024-04-17T09:54:15.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.509+0000 7f4859600640 1 -- 172.21.15.32:0/3218629476 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4840003530 con 0x7f485c15c4c0 2024-04-17T09:54:15.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.509+0000 7f485a000640 1 -- 172.21.15.32:0/3218629476 >> v1:172.21.15.144:6789/0 conn(0x7f485c15f9c0 legacy=0x7f485c0b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:15.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.509+0000 7f4859600640 1 -- 172.21.15.32:0/3218629476 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 592052932 0 0) 0x7f485c162c80 con 0x7f485c15f9c0 2024-04-17T09:54:15.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.509+0000 7f485a000640 1 -- 172.21.15.32:0/3218629476 >> v1:172.21.15.32:6789/0 conn(0x7f485c15c4c0 legacy=0x7f485c06ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:15.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.509+0000 7f485a000640 1 -- 172.21.15.32:0/3218629476 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485c163240 con 0x7f485c06c140 2024-04-17T09:54:15.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.513+0000 7f485a000640 1 --2- 172.21.15.32:0/3218629476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f485c06c140 0x7f485c0adb40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f48480097c0 tx=0x7f484801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b89ebfbc1a036ae8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:15.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.513+0000 7f4859600640 1 -- 172.21.15.32:0/3218629476 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48480b1070 con 0x7f485c06c140 2024-04-17T09:54:15.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.513+0000 7f4859600640 1 -- 172.21.15.32:0/3218629476 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48480ac410 con 0x7f485c06c140 2024-04-17T09:54:15.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.513+0000 7f4859600640 1 -- 172.21.15.32:0/3218629476 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48480ac7e0 con 0x7f485c06c140 2024-04-17T09:54:15.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.513+0000 7f4861c00640 1 -- 172.21.15.32:0/3218629476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f485c06c140 msgr2=0x7f485c0adb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:15.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.513+0000 7f4861c00640 1 --2- 172.21.15.32:0/3218629476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f485c06c140 0x7f485c0adb40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f48480097c0 tx=0x7f484801c4a0 comp rx=0 tx=0).stop 2024-04-17T09:54:15.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.513+0000 7f4861c00640 1 -- 172.21.15.32:0/3218629476 shutdown_connections 2024-04-17T09:54:15.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.513+0000 7f4861c00640 1 --2- 172.21.15.32:0/3218629476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f485c06c140 0x7f485c0adb40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:15.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.513+0000 7f4861c00640 1 -- 172.21.15.32:0/3218629476 >> 172.21.15.32:0/3218629476 conn(0x7f485c0b2120 msgr2=0x7f485c0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.513+0000 7f4861c00640 1 -- 172.21.15.32:0/3218629476 shutdown_connections 2024-04-17T09:54:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.513+0000 7f4861c00640 1 -- 172.21.15.32:0/3218629476 wait complete. 2024-04-17T09:54:15.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f4861c00640 1 Processor -- start 2024-04-17T09:54:15.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f4861c00640 1 -- start start 2024-04-17T09:54:15.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f4861c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f485c06c140 0x7f485c1555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:15.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f4861c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f485c15c4c0 0x7f485c1619f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:15.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f4861c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f485c15f9c0 0x7f485c156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:15.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f4861c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f485c1626b0 con 0x7f485c15f9c0 2024-04-17T09:54:15.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f4861c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f485c162530 con 0x7f485c06c140 2024-04-17T09:54:15.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f4861c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f485c16c6b0 con 0x7f485c15c4c0 2024-04-17T09:54:15.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f485b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f485c15f9c0 0x7f485c156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:15.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f485b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f485c15f9c0 0x7f485c156380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43262/0 (socket says 172.21.15.32:43262) 2024-04-17T09:54:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f485b400640 1 -- 172.21.15.32:0/369683090 learned_addr learned my addr 172.21.15.32:0/369683090 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f485a000640 1 --2- 172.21.15.32:0/369683090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f485c15c4c0 0x7f485c1619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f485aa00640 1 --2- 172.21.15.32:0/369683090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f485c06c140 0x7f485c1555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f485b400640 1 -- 172.21.15.32:0/369683090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f485c15c4c0 msgr2=0x7f485c1619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f485b400640 1 --2- 172.21.15.32:0/369683090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f485c15c4c0 0x7f485c1619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f485b400640 1 -- 172.21.15.32:0/369683090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f485c06c140 msgr2=0x7f485c1555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f485b400640 1 --2- 172.21.15.32:0/369683090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f485c06c140 0x7f485c1555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f485b400640 1 -- 172.21.15.32:0/369683090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485c1576f0 con 0x7f485c15f9c0 2024-04-17T09:54:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f485b400640 1 --2- 172.21.15.32:0/369683090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f485c15f9c0 0x7f485c156380 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f485400c780 tx=0x7f485400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f4852a00640 1 -- 172.21.15.32:0/369683090 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854003db0 con 0x7f485c15f9c0 2024-04-17T09:54:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f4852a00640 1 -- 172.21.15.32:0/369683090 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4854005070 con 0x7f485c15f9c0 2024-04-17T09:54:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f4852a00640 1 -- 172.21.15.32:0/369683090 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854005460 con 0x7f485c15f9c0 2024-04-17T09:54:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f4861c00640 1 -- 172.21.15.32:0/369683090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f485c1579d0 con 0x7f485c15f9c0 2024-04-17T09:54:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f4861c00640 1 -- 172.21.15.32:0/369683090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f485c1614d0 con 0x7f485c15f9c0 2024-04-17T09:54:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.517+0000 7f4861c00640 1 -- 172.21.15.32:0/369683090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f485c068a90 con 0x7f485c15f9c0 2024-04-17T09:54:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.521+0000 7f4852a00640 1 -- 172.21.15.32:0/369683090 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4854005600 con 0x7f485c15f9c0 2024-04-17T09:54:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.521+0000 7f4852a00640 1 --2- 172.21.15.32:0/369683090 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f483c0683b0 0x7f483c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.521+0000 7f4852a00640 1 -- 172.21.15.32:0/369683090 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5227+0+0 (secure 0 0 0) 0x7f4854086620 con 0x7f485c15f9c0 2024-04-17T09:54:15.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.521+0000 7f485aa00640 1 --2- 172.21.15.32:0/369683090 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f483c0683b0 0x7f483c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:15.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.521+0000 7f485aa00640 1 --2- 172.21.15.32:0/369683090 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f483c0683b0 0x7f483c06a870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f484c007fd0 tx=0x7f484c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:15.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.525+0000 7f4852a00640 1 -- 172.21.15.32:0/369683090 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f4854056760 con 0x7f485c15f9c0 2024-04-17T09:54:15.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:15.693+0000 7f4861c00640 1 -- 172.21.15.32:0/369683090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f485c156e80 con 0x7f485c15f9c0 2024-04-17T09:54:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.337+0000 7f4852a00640 1 -- 172.21.15.32:0/369683090 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) ==== 170+0+0 (secure 0 0 0) 0x7f485405aa60 con 0x7f485c15f9c0 2024-04-17T09:54:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-04-17T09:54:16.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.337+0000 7f4861c00640 1 -- 172.21.15.32:0/369683090 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f483c0683b0 msgr2=0x7f483c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:16.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.337+0000 7f4861c00640 1 --2- 172.21.15.32:0/369683090 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f483c0683b0 0x7f483c06a870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f484c007fd0 tx=0x7f484c009040 comp rx=0 tx=0).stop 2024-04-17T09:54:16.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.337+0000 7f4861c00640 1 -- 172.21.15.32:0/369683090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f485c15f9c0 msgr2=0x7f485c156380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:16.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.337+0000 7f4861c00640 1 --2- 172.21.15.32:0/369683090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f485c15f9c0 0x7f485c156380 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f485400c780 tx=0x7f485400cc40 comp rx=0 tx=0).stop 2024-04-17T09:54:16.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.337+0000 7f4861c00640 1 -- 172.21.15.32:0/369683090 shutdown_connections 2024-04-17T09:54:16.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.337+0000 7f4861c00640 1 --2- 172.21.15.32:0/369683090 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f483c0683b0 0x7f483c06a870 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:16.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.337+0000 7f4861c00640 1 --2- 172.21.15.32:0/369683090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f485c15c4c0 0x7f485c1619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:16.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.337+0000 7f4861c00640 1 --2- 172.21.15.32:0/369683090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f485c15f9c0 0x7f485c156380 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:16.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.337+0000 7f4861c00640 1 --2- 172.21.15.32:0/369683090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f485c06c140 0x7f485c1555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:16.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.337+0000 7f4861c00640 1 -- 172.21.15.32:0/369683090 >> 172.21.15.32:0/369683090 conn(0x7f485c0b2120 msgr2=0x7f485c06e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:16.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.337+0000 7f4861c00640 1 -- 172.21.15.32:0/369683090 shutdown_connections 2024-04-17T09:54:16.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.337+0000 7f4861c00640 1 -- 172.21.15.32:0/369683090 wait complete. 2024-04-17T09:54:16.378 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T09:54:16.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.521+0000 7fed59a00640 1 Processor -- start 2024-04-17T09:54:16.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.521+0000 7fed59a00640 1 -- start start 2024-04-17T09:54:16.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.521+0000 7fed59a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fed5406f7c0 0x7fed5406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:16.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.521+0000 7fed59a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed540cc940 con 0x7fed5406f270 2024-04-17T09:54:16.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.521+0000 7fed59a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed540ccc20 con 0x7fed5415b7d0 2024-04-17T09:54:16.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.521+0000 7fed59a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fed5406e0b0 con 0x7fed5406f7c0 2024-04-17T09:54:16.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.521+0000 7fed4be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fed5406f7c0 0x7fed5406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:16.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.521+0000 7fed53400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fed5406f270 0x7fed540ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55606/0 (socket says 172.21.15.32:55606) 2024-04-17T09:54:16.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.521+0000 7fed53400640 1 -- 172.21.15.32:0/389543298 learned_addr learned my addr 172.21.15.32:0/389543298 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:16.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.521+0000 7fed4be00640 1 -- 172.21.15.32:0/389543298 >> v1:172.21.15.144:6789/0 conn(0x7fed5415b7d0 legacy=0x7fed5406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:16.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.521+0000 7fed4be00640 1 -- 172.21.15.32:0/389543298 >> v1:172.21.15.32:6789/0 conn(0x7fed5406f270 legacy=0x7fed540ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:16.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.521+0000 7fed4be00640 1 -- 172.21.15.32:0/389543298 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5406e390 con 0x7fed5406f7c0 2024-04-17T09:54:16.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.521+0000 7fed52000640 1 -- 172.21.15.32:0/389543298 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2457226789 0 0) 0x7fed540cc940 con 0x7fed5406f270 2024-04-17T09:54:16.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.525+0000 7fed4be00640 1 --2- 172.21.15.32:0/389543298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed5406f7c0 0x7fed5406ba30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fed3c0097c0 tx=0x7fed3c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185eb9fa848d4c8f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:16.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.525+0000 7fed52000640 1 -- 172.21.15.32:0/389543298 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c0b0070 con 0x7fed5406f7c0 2024-04-17T09:54:16.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.525+0000 7fed52000640 1 -- 172.21.15.32:0/389543298 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed3c0ab600 con 0x7fed5406f7c0 2024-04-17T09:54:16.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.525+0000 7fed52000640 1 -- 172.21.15.32:0/389543298 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c0ab9d0 con 0x7fed5406f7c0 2024-04-17T09:54:16.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.525+0000 7fed59a00640 1 -- 172.21.15.32:0/389543298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed5406f7c0 msgr2=0x7fed5406ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:16.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.525+0000 7fed59a00640 1 --2- 172.21.15.32:0/389543298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed5406f7c0 0x7fed5406ba30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fed3c0097c0 tx=0x7fed3c01c410 comp rx=0 tx=0).stop 2024-04-17T09:54:16.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.525+0000 7fed59a00640 1 -- 172.21.15.32:0/389543298 shutdown_connections 2024-04-17T09:54:16.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.525+0000 7fed59a00640 1 --2- 172.21.15.32:0/389543298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed5406f7c0 0x7fed5406ba30 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:16.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.525+0000 7fed59a00640 1 -- 172.21.15.32:0/389543298 >> 172.21.15.32:0/389543298 conn(0x7fed540b2120 msgr2=0x7fed540b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:16.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.525+0000 7fed59a00640 1 -- 172.21.15.32:0/389543298 shutdown_connections 2024-04-17T09:54:16.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.525+0000 7fed59a00640 1 -- 172.21.15.32:0/389543298 wait complete. 2024-04-17T09:54:16.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed59a00640 1 Processor -- start 2024-04-17T09:54:16.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed59a00640 1 -- start start 2024-04-17T09:54:16.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed59a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed5406f270 0x7fed54155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:16.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed59a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed5406f7c0 0x7fed54144f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:16.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed59a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed5415b7d0 0x7fed540ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:16.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed59a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fed54169cd0 con 0x7fed5406f7c0 2024-04-17T09:54:16.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed59a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fed5406c380 con 0x7fed5406f270 2024-04-17T09:54:16.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed59a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fed54169e50 con 0x7fed5415b7d0 2024-04-17T09:54:16.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed4be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed5406f7c0 0x7fed54144f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:16.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed4be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed5406f7c0 0x7fed54144f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43282/0 (socket says 172.21.15.32:43282) 2024-04-17T09:54:16.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed4be00640 1 -- 172.21.15.32:0/4042739541 learned_addr learned my addr 172.21.15.32:0/4042739541 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:16.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed52a00640 1 --2- 172.21.15.32:0/4042739541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed5406f270 0x7fed54155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:16.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed53400640 1 --2- 172.21.15.32:0/4042739541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed5415b7d0 0x7fed540ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:16.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed4be00640 1 -- 172.21.15.32:0/4042739541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed5415b7d0 msgr2=0x7fed540ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:16.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed4be00640 1 --2- 172.21.15.32:0/4042739541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed5415b7d0 0x7fed540ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:16.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed4be00640 1 -- 172.21.15.32:0/4042739541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed5406f270 msgr2=0x7fed54155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:16.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed4be00640 1 --2- 172.21.15.32:0/4042739541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed5406f270 0x7fed54155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:16.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed4be00640 1 -- 172.21.15.32:0/4042739541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed54154b90 con 0x7fed5406f7c0 2024-04-17T09:54:16.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed4be00640 1 --2- 172.21.15.32:0/4042739541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed5406f7c0 0x7fed54144f20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fed3c01a040 tx=0x7fed3c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:16.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed4aa00640 1 -- 172.21.15.32:0/4042739541 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c0b0040 con 0x7fed5406f7c0 2024-04-17T09:54:16.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed4aa00640 1 -- 172.21.15.32:0/4042739541 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed3c0b9070 con 0x7fed5406f7c0 2024-04-17T09:54:16.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed4aa00640 1 -- 172.21.15.32:0/4042739541 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c0aa9f0 con 0x7fed5406f7c0 2024-04-17T09:54:16.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed59a00640 1 -- 172.21.15.32:0/4042739541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed540af670 con 0x7fed5406f7c0 2024-04-17T09:54:16.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed59a00640 1 -- 172.21.15.32:0/4042739541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed54154e30 con 0x7fed5406f7c0 2024-04-17T09:54:16.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.529+0000 7fed59a00640 1 -- 172.21.15.32:0/4042739541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed54068a90 con 0x7fed5406f7c0 2024-04-17T09:54:16.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.533+0000 7fed4aa00640 1 -- 172.21.15.32:0/4042739541 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fed3c0aab90 con 0x7fed5406f7c0 2024-04-17T09:54:16.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.533+0000 7fed4aa00640 1 --2- 172.21.15.32:0/4042739541 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fed300683b0 0x7fed3006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:16.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.533+0000 7fed4aa00640 1 -- 172.21.15.32:0/4042739541 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5609+0+0 (secure 0 0 0) 0x7fed3c1214c0 con 0x7fed5406f7c0 2024-04-17T09:54:16.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.533+0000 7fed52a00640 1 --2- 172.21.15.32:0/4042739541 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fed300683b0 0x7fed3006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:16.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.533+0000 7fed52a00640 1 --2- 172.21.15.32:0/4042739541 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fed300683b0 0x7fed3006a870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fed44007fd0 tx=0x7fed44009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:16.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.537+0000 7fed4aa00640 1 -- 172.21.15.32:0/4042739541 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fed3c016090 con 0x7fed5406f7c0 2024-04-17T09:54:16.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:16.741+0000 7fed59a00640 1 -- 172.21.15.32:0/4042739541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fed54155000 con 0x7fed5406f7c0 2024-04-17T09:54:17.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.365+0000 7fed4aa00640 1 -- 172.21.15.32:0/4042739541 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7fed3c0f1ac0 con 0x7fed5406f7c0 2024-04-17T09:54:17.373 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 4 and data pool 5 2024-04-17T09:54:17.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.385+0000 7fed59a00640 1 -- 172.21.15.32:0/4042739541 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fed300683b0 msgr2=0x7fed3006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:17.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.385+0000 7fed59a00640 1 --2- 172.21.15.32:0/4042739541 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fed300683b0 0x7fed3006a870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fed44007fd0 tx=0x7fed44009040 comp rx=0 tx=0).stop 2024-04-17T09:54:17.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.385+0000 7fed59a00640 1 -- 172.21.15.32:0/4042739541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed5406f7c0 msgr2=0x7fed54144f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:17.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.385+0000 7fed59a00640 1 --2- 172.21.15.32:0/4042739541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed5406f7c0 0x7fed54144f20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fed3c01a040 tx=0x7fed3c0191b0 comp rx=0 tx=0).stop 2024-04-17T09:54:17.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.385+0000 7fed59a00640 1 -- 172.21.15.32:0/4042739541 shutdown_connections 2024-04-17T09:54:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.385+0000 7fed59a00640 1 --2- 172.21.15.32:0/4042739541 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fed300683b0 0x7fed3006a870 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.385+0000 7fed59a00640 1 --2- 172.21.15.32:0/4042739541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed5415b7d0 0x7fed540ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.385+0000 7fed59a00640 1 --2- 172.21.15.32:0/4042739541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed5406f7c0 0x7fed54144f20 secure :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fed3c01a040 tx=0x7fed3c0191b0 comp rx=0 tx=0).stop 2024-04-17T09:54:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.385+0000 7fed59a00640 1 --2- 172.21.15.32:0/4042739541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed5406f270 0x7fed54155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.385+0000 7fed59a00640 1 -- 172.21.15.32:0/4042739541 >> 172.21.15.32:0/4042739541 conn(0x7fed540b2120 msgr2=0x7fed54159ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.389+0000 7fed59a00640 1 -- 172.21.15.32:0/4042739541 shutdown_connections 2024-04-17T09:54:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.389+0000 7fed59a00640 1 -- 172.21.15.32:0/4042739541 wait complete. 2024-04-17T09:54:17.421 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:54:17.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.561+0000 7f566d000640 1 Processor -- start 2024-04-17T09:54:17.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.565+0000 7f566d000640 1 -- start start 2024-04-17T09:54:17.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.565+0000 7f566d000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56680b2430 0x7f56680b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:17.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.565+0000 7f566d000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f56680b3900 0x7f566806ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:17.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.565+0000 7f566d000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f566806c800 con 0x7f566815d670 2024-04-17T09:54:17.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.565+0000 7f566d000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f56680b2d50 con 0x7f56680b2430 2024-04-17T09:54:17.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.565+0000 7f566d000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f566806bf70 con 0x7f56680b3900 2024-04-17T09:54:17.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.565+0000 7f5665e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56680b2430 0x7f56680b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:17.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.565+0000 7f5665e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56680b2430 0x7f56680b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43298/0 (socket says 172.21.15.32:43298) 2024-04-17T09:54:17.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.565+0000 7f5665e00640 1 -- 172.21.15.32:0/1910063309 learned_addr learned my addr 172.21.15.32:0/1910063309 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:17.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.565+0000 7f565fe00640 1 --2- 172.21.15.32:0/1910063309 >> v2:172.21.15.144:3300/0 conn(0x7f56680b3900 0x7f566806ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:17.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.565+0000 7f5665e00640 1 -- 172.21.15.32:0/1910063309 >> v2:172.21.15.144:3300/0 conn(0x7f56680b3900 msgr2=0x7f566806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:17.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.565+0000 7f5665e00640 1 --2- 172.21.15.32:0/1910063309 >> v2:172.21.15.144:3300/0 conn(0x7f56680b3900 0x7f566806ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:17.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.565+0000 7f5665e00640 1 -- 172.21.15.32:0/1910063309 >> v1:172.21.15.144:6789/0 conn(0x7f566815d670 legacy=0x7f56681602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:17.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.565+0000 7f5665e00640 1 -- 172.21.15.32:0/1910063309 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566806c240 con 0x7f56680b2430 2024-04-17T09:54:17.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.569+0000 7f5665e00640 1 --2- 172.21.15.32:0/1910063309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56680b2430 0x7f56680b2810 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5658009870 tx=0x7f565801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9241fcf80df95b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:17.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.569+0000 7f5665400640 1 -- 172.21.15.32:0/1910063309 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56580b0070 con 0x7f56680b2430 2024-04-17T09:54:17.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.569+0000 7f5665400640 1 -- 172.21.15.32:0/1910063309 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56580ab600 con 0x7f56680b2430 2024-04-17T09:54:17.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.569+0000 7f5665400640 1 -- 172.21.15.32:0/1910063309 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56580ab9d0 con 0x7f56680b2430 2024-04-17T09:54:17.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.569+0000 7f566d000640 1 -- 172.21.15.32:0/1910063309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56680b2430 msgr2=0x7f56680b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:17.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.569+0000 7f566d000640 1 --2- 172.21.15.32:0/1910063309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56680b2430 0x7f56680b2810 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5658009870 tx=0x7f565801c4e0 comp rx=0 tx=0).stop 2024-04-17T09:54:17.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.569+0000 7f566d000640 1 -- 172.21.15.32:0/1910063309 shutdown_connections 2024-04-17T09:54:17.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.569+0000 7f566d000640 1 --2- 172.21.15.32:0/1910063309 >> v2:172.21.15.144:3300/0 conn(0x7f56680b3900 0x7f566806ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:17.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.569+0000 7f566d000640 1 --2- 172.21.15.32:0/1910063309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56680b2430 0x7f56680b2810 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:17.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.569+0000 7f566d000640 1 -- 172.21.15.32:0/1910063309 >> 172.21.15.32:0/1910063309 conn(0x7f566809a9f0 msgr2=0x7f56680b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:17.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.569+0000 7f566d000640 1 -- 172.21.15.32:0/1910063309 shutdown_connections 2024-04-17T09:54:17.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.569+0000 7f566d000640 1 -- 172.21.15.32:0/1910063309 wait complete. 2024-04-17T09:54:17.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.569+0000 7f566d000640 1 Processor -- start 2024-04-17T09:54:17.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.569+0000 7f566d000640 1 -- start start 2024-04-17T09:54:17.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f566d000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56680b2430 0x7f56681f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:17.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f566d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56680b3900 0x7f56680b6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:17.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f566d000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f566815d670 0x7f56680af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:17.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f566d000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f56680cdcb0 con 0x7f56680b3900 2024-04-17T09:54:17.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f566d000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f56680cdb30 con 0x7f56680b2430 2024-04-17T09:54:17.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f566d000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f566815dae0 con 0x7f566815d670 2024-04-17T09:54:17.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f565fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56680b3900 0x7f56680b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:17.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f565fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56680b3900 0x7f56680b6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43308/0 (socket says 172.21.15.32:43308) 2024-04-17T09:54:17.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f565fe00640 1 -- 172.21.15.32:0/1499368286 learned_addr learned my addr 172.21.15.32:0/1499368286 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:17.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f5665e00640 1 --2- 172.21.15.32:0/1499368286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56680b2430 0x7f56681f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:17.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f5666800640 1 --2- 172.21.15.32:0/1499368286 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f566815d670 0x7f56680af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:17.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f565fe00640 1 -- 172.21.15.32:0/1499368286 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f566815d670 msgr2=0x7f56680af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:17.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f565fe00640 1 --2- 172.21.15.32:0/1499368286 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f566815d670 0x7f56680af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:17.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f565fe00640 1 -- 172.21.15.32:0/1499368286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56680b2430 msgr2=0x7f56681f7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:17.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f565fe00640 1 --2- 172.21.15.32:0/1499368286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56680b2430 0x7f56681f7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:17.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f565fe00640 1 -- 172.21.15.32:0/1499368286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56680c98e0 con 0x7f56680b3900 2024-04-17T09:54:17.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f5666800640 1 --2- 172.21.15.32:0/1499368286 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f566815d670 0x7f56680af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:17.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f565fe00640 1 --2- 172.21.15.32:0/1499368286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56680b3900 0x7f56680b6890 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f565000d7d0 tx=0x7f565000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:17.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f565e000640 1 -- 172.21.15.32:0/1499368286 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650004020 con 0x7f56680b3900 2024-04-17T09:54:17.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f565e000640 1 -- 172.21.15.32:0/1499368286 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56500041c0 con 0x7f56680b3900 2024-04-17T09:54:17.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f565e000640 1 -- 172.21.15.32:0/1499368286 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56500043b0 con 0x7f56680b3900 2024-04-17T09:54:17.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f566d000640 1 -- 172.21.15.32:0/1499368286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56680c9bc0 con 0x7f56680b3900 2024-04-17T09:54:17.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f566d000640 1 -- 172.21.15.32:0/1499368286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56680ca180 con 0x7f56680b3900 2024-04-17T09:54:17.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f566d000640 1 -- 172.21.15.32:0/1499368286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5668068a90 con 0x7f56680b3900 2024-04-17T09:54:17.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.573+0000 7f565e000640 1 -- 172.21.15.32:0/1499368286 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f565001b020 con 0x7f56680b3900 2024-04-17T09:54:17.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.577+0000 7f565e000640 1 --2- 172.21.15.32:0/1499368286 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5644068380 0x7f564406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:17.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.577+0000 7f565e000640 1 -- 172.21.15.32:0/1499368286 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f56500867f0 con 0x7f56680b3900 2024-04-17T09:54:17.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.577+0000 7f5665e00640 1 --2- 172.21.15.32:0/1499368286 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5644068380 0x7f564406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:17.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.577+0000 7f5665e00640 1 --2- 172.21.15.32:0/1499368286 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5644068380 0x7f564406a840 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f56580ad000 tx=0x7f56580a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:17.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.581+0000 7f565e000640 1 -- 172.21.15.32:0/1499368286 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5650056d10 con 0x7f56680b3900 2024-04-17T09:54:17.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.745+0000 7f566d000640 1 -- 172.21.15.32:0/1499368286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5668074f20 con 0x7f56680b3900 2024-04-17T09:54:17.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.745+0000 7f565e000640 1 -- 172.21.15.32:0/1499368286 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16694 (secure 0 0 0) 0x7f5650056610 con 0x7f56680b3900 2024-04-17T09:54:17.748 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:17.748 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:54:16.745183+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:54:15.261471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-17T09:54:15.704958+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:54:17.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.749+0000 7f566d000640 1 -- 172.21.15.32:0/1499368286 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5644068380 msgr2=0x7f564406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:17.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.749+0000 7f566d000640 1 --2- 172.21.15.32:0/1499368286 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5644068380 0x7f564406a840 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f56580ad000 tx=0x7f56580a6000 comp rx=0 tx=0).stop 2024-04-17T09:54:17.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.749+0000 7f566d000640 1 -- 172.21.15.32:0/1499368286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56680b3900 msgr2=0x7f56680b6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:17.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.749+0000 7f566d000640 1 --2- 172.21.15.32:0/1499368286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56680b3900 0x7f56680b6890 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f565000d7d0 tx=0x7f565000dc90 comp rx=0 tx=0).stop 2024-04-17T09:54:17.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.749+0000 7f566d000640 1 -- 172.21.15.32:0/1499368286 shutdown_connections 2024-04-17T09:54:17.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.749+0000 7f566d000640 1 --2- 172.21.15.32:0/1499368286 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5644068380 0x7f564406a840 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:17.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.749+0000 7f566d000640 1 --2- 172.21.15.32:0/1499368286 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f566815d670 0x7f56680af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:17.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.749+0000 7f566d000640 1 --2- 172.21.15.32:0/1499368286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56680b3900 0x7f56680b6890 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:17.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.749+0000 7f566d000640 1 --2- 172.21.15.32:0/1499368286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56680b2430 0x7f56681f7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:17.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.749+0000 7f566d000640 1 -- 172.21.15.32:0/1499368286 >> 172.21.15.32:0/1499368286 conn(0x7f566809a9f0 msgr2=0x7f56680cde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:17.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.749+0000 7f566d000640 1 -- 172.21.15.32:0/1499368286 shutdown_connections 2024-04-17T09:54:17.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.749+0000 7f566d000640 1 -- 172.21.15.32:0/1499368286 wait complete. 2024-04-17T09:54:17.769 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:54:17.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.909+0000 7f1ec8400640 1 Processor -- start 2024-04-17T09:54:17.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.909+0000 7f1ec8400640 1 -- start start 2024-04-17T09:54:17.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.913+0000 7f1ec8400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1ec015eb30 0x7f1ec009b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:17.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.913+0000 7f1ec8400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ec00d0890 0x7f1ec009a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:17.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.913+0000 7f1ec8400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ec009bcb0 con 0x7f1ec009af30 2024-04-17T09:54:17.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.913+0000 7f1ec8400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1ec009b380 con 0x7f1ec00d0890 2024-04-17T09:54:17.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.913+0000 7f1ec8400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1ec006dd60 con 0x7f1ec015eb30 2024-04-17T09:54:17.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.913+0000 7f1ec5000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ec00d0890 0x7f1ec009a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:17.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.913+0000 7f1ec5000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ec00d0890 0x7f1ec009a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43314/0 (socket says 172.21.15.32:43314) 2024-04-17T09:54:17.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.913+0000 7f1ec5000640 1 -- 172.21.15.32:0/1047498940 learned_addr learned my addr 172.21.15.32:0/1047498940 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:17.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.913+0000 7f1ec5a00640 1 --2- 172.21.15.32:0/1047498940 >> v2:172.21.15.144:3300/0 conn(0x7f1ec015eb30 0x7f1ec009b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:17.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.913+0000 7f1ec5000640 1 -- 172.21.15.32:0/1047498940 >> v2:172.21.15.144:3300/0 conn(0x7f1ec015eb30 msgr2=0x7f1ec009b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:17.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.913+0000 7f1ec5000640 1 --2- 172.21.15.32:0/1047498940 >> v2:172.21.15.144:3300/0 conn(0x7f1ec015eb30 0x7f1ec009b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:17.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.913+0000 7f1ec5000640 1 -- 172.21.15.32:0/1047498940 >> v1:172.21.15.144:6789/0 conn(0x7f1ec009af30 legacy=0x7f1ec0162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:17.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.913+0000 7f1ec5000640 1 -- 172.21.15.32:0/1047498940 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec015ef10 con 0x7f1ec00d0890 2024-04-17T09:54:17.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.917+0000 7f1ec5000640 1 --2- 172.21.15.32:0/1047498940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ec00d0890 0x7f1ec009a9f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0073350 tx=0x7f1eac01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959998d9d7a02203 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:17.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.917+0000 7f1ebfe00640 1 -- 172.21.15.32:0/1047498940 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eac0b5070 con 0x7f1ec00d0890 2024-04-17T09:54:17.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.917+0000 7f1ebfe00640 1 -- 172.21.15.32:0/1047498940 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eac0a8a10 con 0x7f1ec00d0890 2024-04-17T09:54:17.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.917+0000 7f1ebfe00640 1 -- 172.21.15.32:0/1047498940 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eac0a8de0 con 0x7f1ec00d0890 2024-04-17T09:54:17.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.917+0000 7f1ec8400640 1 -- 172.21.15.32:0/1047498940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ec00d0890 msgr2=0x7f1ec009a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:17.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.917+0000 7f1ec8400640 1 --2- 172.21.15.32:0/1047498940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ec00d0890 0x7f1ec009a9f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0073350 tx=0x7f1eac01ed10 comp rx=0 tx=0).stop 2024-04-17T09:54:17.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.917+0000 7f1ec8400640 1 -- 172.21.15.32:0/1047498940 shutdown_connections 2024-04-17T09:54:17.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.917+0000 7f1ec8400640 1 --2- 172.21.15.32:0/1047498940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ec00d0890 0x7f1ec009a9f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:17.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.917+0000 7f1ec8400640 1 --2- 172.21.15.32:0/1047498940 >> v2:172.21.15.144:3300/0 conn(0x7f1ec015eb30 0x7f1ec009b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:17.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.917+0000 7f1ec8400640 1 -- 172.21.15.32:0/1047498940 >> 172.21.15.32:0/1047498940 conn(0x7f1ec00b3fd0 msgr2=0x7f1ec00b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:17.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.917+0000 7f1ec8400640 1 -- 172.21.15.32:0/1047498940 shutdown_connections 2024-04-17T09:54:17.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.917+0000 7f1ec8400640 1 -- 172.21.15.32:0/1047498940 wait complete. 2024-04-17T09:54:17.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.917+0000 7f1ec8400640 1 Processor -- start 2024-04-17T09:54:17.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.917+0000 7f1ec8400640 1 -- start start 2024-04-17T09:54:17.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.917+0000 7f1ec8400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ec009af30 0x7f1ec00cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:17.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.917+0000 7f1ec8400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ec00d0890 0x7f1ec00ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:17.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.917+0000 7f1ec8400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ec015eb30 0x7f1ec00af170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:17.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.917+0000 7f1ec8400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec00d1370 con 0x7f1ec015eb30 2024-04-17T09:54:17.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.917+0000 7f1ec8400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec00d1090 con 0x7f1ec009af30 2024-04-17T09:54:17.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.917+0000 7f1ec8400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1ec00d1650 con 0x7f1ec00d0890 2024-04-17T09:54:17.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.917+0000 7f1ec6400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ec015eb30 0x7f1ec00af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:17.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.921+0000 7f1ec6400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ec015eb30 0x7f1ec00af170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43322/0 (socket says 172.21.15.32:43322) 2024-04-17T09:54:17.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.921+0000 7f1ec6400640 1 -- 172.21.15.32:0/2699833371 learned_addr learned my addr 172.21.15.32:0/2699833371 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:17.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.921+0000 7f1ec5000640 1 --2- 172.21.15.32:0/2699833371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ec00d0890 0x7f1ec00ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:17.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.921+0000 7f1ec5a00640 1 --2- 172.21.15.32:0/2699833371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ec009af30 0x7f1ec00cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:17.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.921+0000 7f1ec6400640 1 -- 172.21.15.32:0/2699833371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ec00d0890 msgr2=0x7f1ec00ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:17.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.921+0000 7f1ec6400640 1 --2- 172.21.15.32:0/2699833371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ec00d0890 0x7f1ec00ca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:17.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.921+0000 7f1ec6400640 1 -- 172.21.15.32:0/2699833371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ec009af30 msgr2=0x7f1ec00cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:17.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.921+0000 7f1ec6400640 1 --2- 172.21.15.32:0/2699833371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ec009af30 0x7f1ec00cc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:17.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.921+0000 7f1ec6400640 1 -- 172.21.15.32:0/2699833371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec00b1820 con 0x7f1ec015eb30 2024-04-17T09:54:17.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.921+0000 7f1ec6400640 1 --2- 172.21.15.32:0/2699833371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ec015eb30 0x7f1ec00af170 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800c770 tx=0x7f1eb800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:17.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.921+0000 7f1ebd600640 1 -- 172.21.15.32:0/2699833371 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8003b80 con 0x7f1ec015eb30 2024-04-17T09:54:17.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.921+0000 7f1ebd600640 1 -- 172.21.15.32:0/2699833371 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eb8003d20 con 0x7f1ec015eb30 2024-04-17T09:54:17.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.921+0000 7f1ebd600640 1 -- 172.21.15.32:0/2699833371 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb800aa90 con 0x7f1ec015eb30 2024-04-17T09:54:17.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.921+0000 7f1ec8400640 1 -- 172.21.15.32:0/2699833371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec00b1b00 con 0x7f1ec015eb30 2024-04-17T09:54:17.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.921+0000 7f1ec8400640 1 -- 172.21.15.32:0/2699833371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec00cc7e0 con 0x7f1ec015eb30 2024-04-17T09:54:17.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.921+0000 7f1ec8400640 1 -- 172.21.15.32:0/2699833371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec0068a90 con 0x7f1ec015eb30 2024-04-17T09:54:17.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.921+0000 7f1ebd600640 1 -- 172.21.15.32:0/2699833371 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1eb800ac30 con 0x7f1ec015eb30 2024-04-17T09:54:17.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.921+0000 7f1ebd600640 1 --2- 172.21.15.32:0/2699833371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1ea0068380 0x7f1ea006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:17.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.921+0000 7f1ebd600640 1 -- 172.21.15.32:0/2699833371 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f1eb8086780 con 0x7f1ec015eb30 2024-04-17T09:54:17.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.925+0000 7f1ec5a00640 1 --2- 172.21.15.32:0/2699833371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1ea0068380 0x7f1ea006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:17.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.925+0000 7f1ec5a00640 1 --2- 172.21.15.32:0/2699833371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1ea0068380 0x7f1ea006a840 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009ac0 tx=0x7f1eb4009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:17.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:17.929+0000 7f1ebd600640 1 -- 172.21.15.32:0/2699833371 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f1eb80566e0 con 0x7f1ec015eb30 2024-04-17T09:54:18.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.093+0000 7f1ec8400640 1 -- 172.21.15.32:0/2699833371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ec00ce9a0 con 0x7f1ec015eb30 2024-04-17T09:54:18.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.093+0000 7f1ebd600640 1 -- 172.21.15.32:0/2699833371 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16694 (secure 0 0 0) 0x7f1eb805a9e0 con 0x7f1ec015eb30 2024-04-17T09:54:18.095 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:18.095 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:54:16.745183+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:54:15.261471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-17T09:54:15.704958+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:54:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.097+0000 7f1ec8400640 1 -- 172.21.15.32:0/2699833371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1ea0068380 msgr2=0x7f1ea006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.097+0000 7f1ec8400640 1 --2- 172.21.15.32:0/2699833371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1ea0068380 0x7f1ea006a840 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009ac0 tx=0x7f1eb4009190 comp rx=0 tx=0).stop 2024-04-17T09:54:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.097+0000 7f1ec8400640 1 -- 172.21.15.32:0/2699833371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ec015eb30 msgr2=0x7f1ec00af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.097+0000 7f1ec8400640 1 --2- 172.21.15.32:0/2699833371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ec015eb30 0x7f1ec00af170 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800c770 tx=0x7f1eb800cc30 comp rx=0 tx=0).stop 2024-04-17T09:54:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.097+0000 7f1ec8400640 1 -- 172.21.15.32:0/2699833371 shutdown_connections 2024-04-17T09:54:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.097+0000 7f1ec8400640 1 --2- 172.21.15.32:0/2699833371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1ea0068380 0x7f1ea006a840 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.097+0000 7f1ec8400640 1 --2- 172.21.15.32:0/2699833371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ec00d0890 0x7f1ec00ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.097+0000 7f1ec8400640 1 --2- 172.21.15.32:0/2699833371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ec015eb30 0x7f1ec00af170 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.097+0000 7f1ec8400640 1 --2- 172.21.15.32:0/2699833371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ec009af30 0x7f1ec00cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.097+0000 7f1ec8400640 1 -- 172.21.15.32:0/2699833371 >> 172.21.15.32:0/2699833371 conn(0x7f1ec00b3fd0 msgr2=0x7f1ec015da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.097+0000 7f1ec8400640 1 -- 172.21.15.32:0/2699833371 shutdown_connections 2024-04-17T09:54:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.097+0000 7f1ec8400640 1 -- 172.21.15.32:0/2699833371 wait complete. 2024-04-17T09:54:18.116 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T09:54:18.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.257+0000 7f03cc400640 1 Processor -- start 2024-04-17T09:54:18.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.257+0000 7f03cc400640 1 -- start start 2024-04-17T09:54:18.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.257+0000 7f03cc400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f03c415b7d0 0x7f03c406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:18.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.257+0000 7f03cc400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f03c406c140 0x7f03c40ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:18.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.257+0000 7f03cc400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03c40cc940 con 0x7f03c406f7c0 2024-04-17T09:54:18.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.257+0000 7f03cc400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f03c40ccc20 con 0x7f03c406c140 2024-04-17T09:54:18.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.257+0000 7f03cc400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f03c406e0b0 con 0x7f03c415b7d0 2024-04-17T09:54:18.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.257+0000 7f03c9000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f03c406f7c0 0x7f03c406ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55608/0 (socket says 172.21.15.32:55608) 2024-04-17T09:54:18.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.257+0000 7f03ca400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f03c406c140 0x7f03c40ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:18.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.261+0000 7f03c9000640 1 -- 172.21.15.32:0/1433506129 learned_addr learned my addr 172.21.15.32:0/1433506129 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:18.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.261+0000 7f03c9a00640 1 --2- 172.21.15.32:0/1433506129 >> v2:172.21.15.144:3300/0 conn(0x7f03c415b7d0 0x7f03c406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:18.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.261+0000 7f03ca400640 1 -- 172.21.15.32:0/1433506129 >> v2:172.21.15.144:3300/0 conn(0x7f03c415b7d0 msgr2=0x7f03c406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:18.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.261+0000 7f03ca400640 1 --2- 172.21.15.32:0/1433506129 >> v2:172.21.15.144:3300/0 conn(0x7f03c415b7d0 0x7f03c406e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:18.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.261+0000 7f03ca400640 1 -- 172.21.15.32:0/1433506129 >> v1:172.21.15.32:6789/0 conn(0x7f03c406f7c0 legacy=0x7f03c406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:18.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.261+0000 7f03ca400640 1 -- 172.21.15.32:0/1433506129 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c406e390 con 0x7f03c406c140 2024-04-17T09:54:18.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.261+0000 7f03c3e00640 1 -- 172.21.15.32:0/1433506129 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1404638693 0 0) 0x7f03c40cc940 con 0x7f03c406f7c0 2024-04-17T09:54:18.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.261+0000 7f03ca400640 1 --2- 172.21.15.32:0/1433506129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03c406c140 0x7f03c40ce290 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f03c40d1530 tx=0x7f03bc01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9116a8ac89c73a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:18.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.261+0000 7f03c3e00640 1 -- 172.21.15.32:0/1433506129 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc0b2070 con 0x7f03c406c140 2024-04-17T09:54:18.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.261+0000 7f03c3e00640 1 -- 172.21.15.32:0/1433506129 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03bc0a5980 con 0x7f03c406c140 2024-04-17T09:54:18.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.261+0000 7f03c3e00640 1 -- 172.21.15.32:0/1433506129 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc0a5d50 con 0x7f03c406c140 2024-04-17T09:54:18.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03cc400640 1 -- 172.21.15.32:0/1433506129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03c406c140 msgr2=0x7f03c40ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:18.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03cc400640 1 --2- 172.21.15.32:0/1433506129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03c406c140 0x7f03c40ce290 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f03c40d1530 tx=0x7f03bc01c8a0 comp rx=0 tx=0).stop 2024-04-17T09:54:18.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03cc400640 1 -- 172.21.15.32:0/1433506129 shutdown_connections 2024-04-17T09:54:18.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03cc400640 1 --2- 172.21.15.32:0/1433506129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03c406c140 0x7f03c40ce290 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:18.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03cc400640 1 --2- 172.21.15.32:0/1433506129 >> v2:172.21.15.144:3300/0 conn(0x7f03c415b7d0 0x7f03c406e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:18.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03cc400640 1 -- 172.21.15.32:0/1433506129 >> 172.21.15.32:0/1433506129 conn(0x7f03c40b2120 msgr2=0x7f03c40b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:18.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03cc400640 1 -- 172.21.15.32:0/1433506129 shutdown_connections 2024-04-17T09:54:18.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03cc400640 1 -- 172.21.15.32:0/1433506129 wait complete. 2024-04-17T09:54:18.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03cc400640 1 Processor -- start 2024-04-17T09:54:18.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03cc400640 1 -- start start 2024-04-17T09:54:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03cc400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03c406c140 0x7f03c4140fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03cc400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03c406f7c0 0x7f03c40ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03cc400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03c415b7d0 0x7f03c415dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03cc400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f03c4168630 con 0x7f03c406c140 2024-04-17T09:54:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03cc400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f03c41684b0 con 0x7f03c406f7c0 2024-04-17T09:54:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03cc400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f03c41687b0 con 0x7f03c415b7d0 2024-04-17T09:54:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03c9a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03c406c140 0x7f03c4140fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03c9a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03c406c140 0x7f03c4140fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43342/0 (socket says 172.21.15.32:43342) 2024-04-17T09:54:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03c9a00640 1 -- 172.21.15.32:0/1507536437 learned_addr learned my addr 172.21.15.32:0/1507536437 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03c9a00640 1 -- 172.21.15.32:0/1507536437 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03c415b7d0 msgr2=0x7f03c415dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03c9a00640 1 --2- 172.21.15.32:0/1507536437 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03c415b7d0 0x7f03c415dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03c9a00640 1 -- 172.21.15.32:0/1507536437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03c406f7c0 msgr2=0x7f03c40ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03c9a00640 1 --2- 172.21.15.32:0/1507536437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03c406f7c0 0x7f03c40ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03c9a00640 1 -- 172.21.15.32:0/1507536437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c41f7df0 con 0x7f03c406c140 2024-04-17T09:54:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.265+0000 7f03c9a00640 1 --2- 172.21.15.32:0/1507536437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03c406c140 0x7f03c4140fe0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f03b800e870 tx=0x7f03b800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.269+0000 7f03c1600640 1 -- 172.21.15.32:0/1507536437 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8017070 con 0x7f03c406c140 2024-04-17T09:54:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.269+0000 7f03c1600640 1 -- 172.21.15.32:0/1507536437 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03b800c940 con 0x7f03c406c140 2024-04-17T09:54:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.269+0000 7f03c1600640 1 -- 172.21.15.32:0/1507536437 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b800ccf0 con 0x7f03c406c140 2024-04-17T09:54:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.269+0000 7f03cc400640 1 -- 172.21.15.32:0/1507536437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c41f80d0 con 0x7f03c406c140 2024-04-17T09:54:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.269+0000 7f03cc400640 1 -- 172.21.15.32:0/1507536437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c41f8970 con 0x7f03c406c140 2024-04-17T09:54:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.269+0000 7f03cc400640 1 -- 172.21.15.32:0/1507536437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03c41f82a0 con 0x7f03c406c140 2024-04-17T09:54:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.269+0000 7f03c1600640 1 -- 172.21.15.32:0/1507536437 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f03b801a070 con 0x7f03c406c140 2024-04-17T09:54:18.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.269+0000 7f03c1600640 1 --2- 172.21.15.32:0/1507536437 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f03a40683b0 0x7f03a406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:18.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.269+0000 7f03c1600640 1 -- 172.21.15.32:0/1507536437 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f03b80868a0 con 0x7f03c406c140 2024-04-17T09:54:18.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.269+0000 7f03c9000640 1 --2- 172.21.15.32:0/1507536437 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f03a40683b0 0x7f03a406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:18.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.269+0000 7f03c9000640 1 --2- 172.21.15.32:0/1507536437 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f03a40683b0 0x7f03a406a870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f03b00011f0 tx=0x7f03b0007930 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:18.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.273+0000 7f03c1600640 1 -- 172.21.15.32:0/1507536437 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f03b8056e50 con 0x7f03c406c140 2024-04-17T09:54:18.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:18.481+0000 7f03cc400640 1 -- 172.21.15.32:0/1507536437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f03c4068a90 con 0x7f03c406c140 2024-04-17T09:54:19.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.369+0000 7f03c1600640 1 -- 172.21.15.32:0/1507536437 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) ==== 118+0+0 (secure 0 0 0) 0x7f03b8056750 con 0x7f03c406c140 2024-04-17T09:54:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.373+0000 7f03cc400640 1 -- 172.21.15.32:0/1507536437 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f03a40683b0 msgr2=0x7f03a406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.373+0000 7f03cc400640 1 --2- 172.21.15.32:0/1507536437 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f03a40683b0 0x7f03a406a870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f03b00011f0 tx=0x7f03b0007930 comp rx=0 tx=0).stop 2024-04-17T09:54:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.373+0000 7f03cc400640 1 -- 172.21.15.32:0/1507536437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03c406c140 msgr2=0x7f03c4140fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.373+0000 7f03cc400640 1 --2- 172.21.15.32:0/1507536437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03c406c140 0x7f03c4140fe0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f03b800e870 tx=0x7f03b800ec70 comp rx=0 tx=0).stop 2024-04-17T09:54:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.373+0000 7f03cc400640 1 -- 172.21.15.32:0/1507536437 shutdown_connections 2024-04-17T09:54:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.373+0000 7f03cc400640 1 --2- 172.21.15.32:0/1507536437 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f03a40683b0 0x7f03a406a870 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.373+0000 7f03cc400640 1 --2- 172.21.15.32:0/1507536437 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03c415b7d0 0x7f03c415dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.373+0000 7f03cc400640 1 --2- 172.21.15.32:0/1507536437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03c406f7c0 0x7f03c40ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.373+0000 7f03cc400640 1 --2- 172.21.15.32:0/1507536437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03c406c140 0x7f03c4140fe0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.373+0000 7f03cc400640 1 -- 172.21.15.32:0/1507536437 >> 172.21.15.32:0/1507536437 conn(0x7f03c40b2120 msgr2=0x7f03c4155d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.373+0000 7f03cc400640 1 -- 172.21.15.32:0/1507536437 shutdown_connections 2024-04-17T09:54:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.373+0000 7f03cc400640 1 -- 172.21.15.32:0/1507536437 wait complete. 2024-04-17T09:54:19.397 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-17T09:54:19.398 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-17T09:54:19.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.541+0000 7f3cae800640 1 Processor -- start 2024-04-17T09:54:19.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.541+0000 7f3cae800640 1 -- start start 2024-04-17T09:54:19.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.545+0000 7f3cae800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ca80b0430 0x7f3ca80b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:19.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.545+0000 7f3cae800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ca80b1f40 con 0x7f3ca806c140 2024-04-17T09:54:19.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.545+0000 7f3cae800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ca80b29a0 con 0x7f3ca815ea80 2024-04-17T09:54:19.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.545+0000 7f3cae800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3ca80b2c80 con 0x7f3ca80b0430 2024-04-17T09:54:19.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.545+0000 7f3caca00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3ca806c140 0x7f3ca80b0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55622/0 (socket says 172.21.15.32:55622) 2024-04-17T09:54:19.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.545+0000 7f3cade00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ca80b0430 0x7f3ca80b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:19.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.545+0000 7f3caca00640 1 -- 172.21.15.32:0/3953817909 learned_addr learned my addr 172.21.15.32:0/3953817909 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:19.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.545+0000 7f3cade00640 1 -- 172.21.15.32:0/3953817909 >> v1:172.21.15.144:6789/0 conn(0x7f3ca815ea80 legacy=0x7f3ca806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:19.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.545+0000 7f3cade00640 1 -- 172.21.15.32:0/3953817909 >> v1:172.21.15.32:6789/0 conn(0x7f3ca806c140 legacy=0x7f3ca80b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:19.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.545+0000 7f3cade00640 1 -- 172.21.15.32:0/3953817909 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca80b5dd0 con 0x7f3ca80b0430 2024-04-17T09:54:19.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.545+0000 7f3ca7e00640 1 -- 172.21.15.32:0/3953817909 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2599446990 0 0) 0x7f3ca80b1f40 con 0x7f3ca806c140 2024-04-17T09:54:19.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.545+0000 7f3cade00640 1 --2- 172.21.15.32:0/3953817909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca80b0430 0x7f3ca80b11b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0009d90 tx=0x7f3ca001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87583c9ac686ea88 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:19.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.549+0000 7f3ca7e00640 1 -- 172.21.15.32:0/3953817909 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca00b2070 con 0x7f3ca80b0430 2024-04-17T09:54:19.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.549+0000 7f3ca7e00640 1 -- 172.21.15.32:0/3953817909 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ca00a5870 con 0x7f3ca80b0430 2024-04-17T09:54:19.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.549+0000 7f3ca7e00640 1 -- 172.21.15.32:0/3953817909 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca00a5c40 con 0x7f3ca80b0430 2024-04-17T09:54:19.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.549+0000 7f3cae800640 1 -- 172.21.15.32:0/3953817909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca80b0430 msgr2=0x7f3ca80b11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:19.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.549+0000 7f3cae800640 1 --2- 172.21.15.32:0/3953817909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca80b0430 0x7f3ca80b11b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0009d90 tx=0x7f3ca001c8a0 comp rx=0 tx=0).stop 2024-04-17T09:54:19.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.549+0000 7f3cae800640 1 -- 172.21.15.32:0/3953817909 shutdown_connections 2024-04-17T09:54:19.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.549+0000 7f3cae800640 1 --2- 172.21.15.32:0/3953817909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca80b0430 0x7f3ca80b11b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:19.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.549+0000 7f3cae800640 1 -- 172.21.15.32:0/3953817909 >> 172.21.15.32:0/3953817909 conn(0x7f3ca80b6820 msgr2=0x7f3ca80b6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:19.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.549+0000 7f3cae800640 1 -- 172.21.15.32:0/3953817909 shutdown_connections 2024-04-17T09:54:19.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.549+0000 7f3cae800640 1 -- 172.21.15.32:0/3953817909 wait complete. 2024-04-17T09:54:19.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.549+0000 7f3cae800640 1 Processor -- start 2024-04-17T09:54:19.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.549+0000 7f3cae800640 1 -- start start 2024-04-17T09:54:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.549+0000 7f3cae800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ca806c140 0x7f3ca8157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.549+0000 7f3cae800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ca80b0430 0x7f3ca8148360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.549+0000 7f3cae800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca815ea80 0x7f3ca8150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.549+0000 7f3cae800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca80b1a80 con 0x7f3ca815ea80 2024-04-17T09:54:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.549+0000 7f3cae800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca80b1900 con 0x7f3ca806c140 2024-04-17T09:54:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.549+0000 7f3cae800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3ca80b1c00 con 0x7f3ca80b0430 2024-04-17T09:54:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.549+0000 7f3cade00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca815ea80 0x7f3ca8150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.549+0000 7f3cade00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca815ea80 0x7f3ca8150d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43366/0 (socket says 172.21.15.32:43366) 2024-04-17T09:54:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.549+0000 7f3cade00640 1 -- 172.21.15.32:0/1639864896 learned_addr learned my addr 172.21.15.32:0/1639864896 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.553+0000 7f3cad400640 1 --2- 172.21.15.32:0/1639864896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ca806c140 0x7f3ca8157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.553+0000 7f3caca00640 1 --2- 172.21.15.32:0/1639864896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ca80b0430 0x7f3ca8148360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.553+0000 7f3cade00640 1 -- 172.21.15.32:0/1639864896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ca80b0430 msgr2=0x7f3ca8148360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.553+0000 7f3cade00640 1 --2- 172.21.15.32:0/1639864896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ca80b0430 0x7f3ca8148360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:19.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.553+0000 7f3cade00640 1 -- 172.21.15.32:0/1639864896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ca806c140 msgr2=0x7f3ca8157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:19.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.553+0000 7f3cade00640 1 --2- 172.21.15.32:0/1639864896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ca806c140 0x7f3ca8157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:19.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.553+0000 7f3cade00640 1 -- 172.21.15.32:0/1639864896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca815a530 con 0x7f3ca815ea80 2024-04-17T09:54:19.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.553+0000 7f3caca00640 1 --2- 172.21.15.32:0/1639864896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ca80b0430 0x7f3ca8148360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:54:19.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.553+0000 7f3cade00640 1 --2- 172.21.15.32:0/1639864896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca815ea80 0x7f3ca8150d10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3ca00b0040 tx=0x7f3ca0019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:19.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.553+0000 7f3ca5600640 1 -- 172.21.15.32:0/1639864896 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca00bd020 con 0x7f3ca815ea80 2024-04-17T09:54:19.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.553+0000 7f3ca5600640 1 -- 172.21.15.32:0/1639864896 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ca00adaf0 con 0x7f3ca815ea80 2024-04-17T09:54:19.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.553+0000 7f3ca5600640 1 -- 172.21.15.32:0/1639864896 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca00b2040 con 0x7f3ca815ea80 2024-04-17T09:54:19.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.553+0000 7f3cae800640 1 -- 172.21.15.32:0/1639864896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca815a810 con 0x7f3ca815ea80 2024-04-17T09:54:19.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.553+0000 7f3cae800640 1 -- 172.21.15.32:0/1639864896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca81507f0 con 0x7f3ca815ea80 2024-04-17T09:54:19.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.553+0000 7f3cae800640 1 -- 172.21.15.32:0/1639864896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca8150340 con 0x7f3ca815ea80 2024-04-17T09:54:19.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.553+0000 7f3ca5600640 1 -- 172.21.15.32:0/1639864896 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3ca00c0020 con 0x7f3ca815ea80 2024-04-17T09:54:19.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.553+0000 7f3ca5600640 1 --2- 172.21.15.32:0/1639864896 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c880683b0 0x7f3c8806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:19.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.553+0000 7f3ca5600640 1 -- 172.21.15.32:0/1639864896 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f3ca0121760 con 0x7f3ca815ea80 2024-04-17T09:54:19.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.553+0000 7f3cad400640 1 --2- 172.21.15.32:0/1639864896 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c880683b0 0x7f3c8806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:19.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.557+0000 7f3cad400640 1 --2- 172.21.15.32:0/1639864896 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c880683b0 0x7f3c8806a870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c004e30 tx=0x7f3c9c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:19.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.557+0000 7f3ca5600640 1 -- 172.21.15.32:0/1639864896 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3ca00c8020 con 0x7f3ca815ea80 2024-04-17T09:54:19.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:19.757+0000 7f3cae800640 1 -- 172.21.15.32:0/1639864896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3ca804fda0 con 0x7f3ca815ea80 2024-04-17T09:54:20.324 INFO:tasks.ceph.mds.a.smithi032.stderr:2024-04-17T09:54:20.321+0000 7fe33da00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T09:54:20.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.373+0000 7f3ca5600640 1 -- 172.21.15.32:0/1639864896 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) ==== 122+0+0 (secure 0 0 0) 0x7f3ca0016520 con 0x7f3ca815ea80 2024-04-17T09:54:20.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.377+0000 7f3cae800640 1 -- 172.21.15.32:0/1639864896 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c880683b0 msgr2=0x7f3c8806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:20.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.377+0000 7f3cae800640 1 --2- 172.21.15.32:0/1639864896 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c880683b0 0x7f3c8806a870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c004e30 tx=0x7f3c9c009040 comp rx=0 tx=0).stop 2024-04-17T09:54:20.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.377+0000 7f3cae800640 1 -- 172.21.15.32:0/1639864896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca815ea80 msgr2=0x7f3ca8150d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:20.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.377+0000 7f3cae800640 1 --2- 172.21.15.32:0/1639864896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca815ea80 0x7f3ca8150d10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3ca00b0040 tx=0x7f3ca0019640 comp rx=0 tx=0).stop 2024-04-17T09:54:20.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.377+0000 7f3cae800640 1 -- 172.21.15.32:0/1639864896 shutdown_connections 2024-04-17T09:54:20.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.377+0000 7f3cae800640 1 --2- 172.21.15.32:0/1639864896 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c880683b0 0x7f3c8806a870 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:20.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.377+0000 7f3cae800640 1 --2- 172.21.15.32:0/1639864896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ca80b0430 0x7f3ca8148360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:20.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.377+0000 7f3cae800640 1 --2- 172.21.15.32:0/1639864896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca815ea80 0x7f3ca8150d10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:20.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.377+0000 7f3cae800640 1 --2- 172.21.15.32:0/1639864896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ca806c140 0x7f3ca8157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:20.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.377+0000 7f3cae800640 1 -- 172.21.15.32:0/1639864896 >> 172.21.15.32:0/1639864896 conn(0x7f3ca80b6820 msgr2=0x7f3ca80aec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:20.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.377+0000 7f3cae800640 1 -- 172.21.15.32:0/1639864896 shutdown_connections 2024-04-17T09:54:20.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.377+0000 7f3cae800640 1 -- 172.21.15.32:0/1639864896 wait complete. 2024-04-17T09:54:20.401 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-17T09:54:20.402 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:54:20.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.545+0000 7fdeefa00640 1 Processor -- start 2024-04-17T09:54:20.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.545+0000 7fdeefa00640 1 -- start start 2024-04-17T09:54:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.545+0000 7fdeefa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdee8144e90 0x7fdee80b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.545+0000 7fdeefa00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdee8072bb0 con 0x7fdee815c1b0 2024-04-17T09:54:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.545+0000 7fdeefa00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdee80b3dc0 con 0x7fdee80b4de0 2024-04-17T09:54:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.545+0000 7fdeefa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdee8145270 con 0x7fdee8144e90 2024-04-17T09:54:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.545+0000 7fdeeda00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdee815c1b0 0x7fdee806ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37072/0 (socket says 172.21.15.32:37072) 2024-04-17T09:54:20.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.545+0000 7fdeeda00640 1 -- 172.21.15.32:0/3225902207 learned_addr learned my addr 172.21.15.32:0/3225902207 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:20.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdee7e00640 1 -- 172.21.15.32:0/3225902207 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 611383012 0 0) 0x7fdee8072bb0 con 0x7fdee815c1b0 2024-04-17T09:54:20.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdeed000640 1 --2- 172.21.15.32:0/3225902207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdee8144e90 0x7fdee80b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdee7e00640 1 -- 172.21.15.32:0/3225902207 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdecc003530 con 0x7fdee815c1b0 2024-04-17T09:54:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdee7e00640 1 -- 172.21.15.32:0/3225902207 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4022670499 0 0) 0x7fdecc003530 con 0x7fdee815c1b0 2024-04-17T09:54:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdee7e00640 1 -- 172.21.15.32:0/3225902207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdee8144e90 msgr2=0x7fdee80b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdee7e00640 1 --2- 172.21.15.32:0/3225902207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdee8144e90 0x7fdee80b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdee7e00640 1 -- 172.21.15.32:0/3225902207 >> v1:172.21.15.144:6789/0 conn(0x7fdee80b4de0 legacy=0x7fdee815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdee7e00640 1 -- 172.21.15.32:0/3225902207 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee80b40a0 con 0x7fdee815c1b0 2024-04-17T09:54:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdee7e00640 1 -- 172.21.15.32:0/3225902207 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7fdee0001a20 con 0x7fdee815c1b0 2024-04-17T09:54:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdee7e00640 1 -- 172.21.15.32:0/3225902207 >> v1:172.21.15.32:6789/0 conn(0x7fdee815c1b0 legacy=0x7fdee806ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdee7e00640 1 --2- 172.21.15.32:0/3225902207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdecc0039f0 0x7fdecc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdee7e00640 1 --2- 172.21.15.32:0/3225902207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdee8144e90 0x7fdecc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdee7e00640 1 --2- 172.21.15.32:0/3225902207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdecc004d50 0x7fdecc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdee7e00640 1 -- 172.21.15.32:0/3225902207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdee8145270 con 0x7fdecc0039f0 2024-04-17T09:54:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdee7e00640 1 -- 172.21.15.32:0/3225902207 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc00bab0 con 0x7fdecc004d50 2024-04-17T09:54:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdee7e00640 1 -- 172.21.15.32:0/3225902207 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdecc00dec0 con 0x7fdee8144e90 2024-04-17T09:54:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdeed000640 1 --2- 172.21.15.32:0/3225902207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdecc0039f0 0x7fdecc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:20.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdedfe00640 1 --2- 172.21.15.32:0/3225902207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdee8144e90 0x7fdecc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:20.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdeeda00640 1 --2- 172.21.15.32:0/3225902207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdecc004d50 0x7fdecc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:20.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdeed000640 1 -- 172.21.15.32:0/3225902207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdee8144e90 msgr2=0x7fdecc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:20.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdeed000640 1 --2- 172.21.15.32:0/3225902207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdee8144e90 0x7fdecc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:20.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdeed000640 1 -- 172.21.15.32:0/3225902207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdecc004d50 msgr2=0x7fdecc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:20.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdeed000640 1 --2- 172.21.15.32:0/3225902207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdecc004d50 0x7fdecc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:20.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.549+0000 7fdeed000640 1 -- 172.21.15.32:0/3225902207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdecc003530 con 0x7fdecc0039f0 2024-04-17T09:54:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.553+0000 7fdeed000640 1 --2- 172.21.15.32:0/3225902207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdecc0039f0 0x7fdecc003e10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fded8009ab0 tx=0x7fded801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e71824ce2aff6f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.553+0000 7fdee7e00640 1 -- 172.21.15.32:0/3225902207 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded80b2070 con 0x7fdecc0039f0 2024-04-17T09:54:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.553+0000 7fdee7e00640 1 -- 172.21.15.32:0/3225902207 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded80a5610 con 0x7fdecc0039f0 2024-04-17T09:54:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.553+0000 7fdee7e00640 1 -- 172.21.15.32:0/3225902207 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded80096b0 con 0x7fdecc0039f0 2024-04-17T09:54:20.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.553+0000 7fdeefa00640 1 -- 172.21.15.32:0/3225902207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdecc0039f0 msgr2=0x7fdecc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:20.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.553+0000 7fdeefa00640 1 --2- 172.21.15.32:0/3225902207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdecc0039f0 0x7fdecc003e10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fded8009ab0 tx=0x7fded801ca20 comp rx=0 tx=0).stop 2024-04-17T09:54:20.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.553+0000 7fdeeda00640 1 -- 172.21.15.32:0/3225902207 reap_dead start 2024-04-17T09:54:20.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.553+0000 7fdeefa00640 1 -- 172.21.15.32:0/3225902207 shutdown_connections 2024-04-17T09:54:20.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.553+0000 7fdeefa00640 1 -- 172.21.15.32:0/3225902207 >> 172.21.15.32:0/3225902207 conn(0x7fdee806ba30 msgr2=0x7fdee806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:20.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.553+0000 7fdeefa00640 1 -- 172.21.15.32:0/3225902207 shutdown_connections 2024-04-17T09:54:20.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.553+0000 7fdeefa00640 1 -- 172.21.15.32:0/3225902207 wait complete. 2024-04-17T09:54:20.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.553+0000 7fdeefa00640 1 Processor -- start 2024-04-17T09:54:20.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.553+0000 7fdeefa00640 1 -- start start 2024-04-17T09:54:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.553+0000 7fdeefa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdecc0039f0 0x7fdee8139620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.553+0000 7fdeefa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdee806e5c0 0x7fdee8138dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.553+0000 7fdeefa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdee815e440 0x7fdee815e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.553+0000 7fdeefa00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdee8072670 con 0x7fdecc0039f0 2024-04-17T09:54:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.553+0000 7fdeefa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdee806f8c0 con 0x7fdee815e440 2024-04-17T09:54:20.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.553+0000 7fdeefa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdee80727f0 con 0x7fdee806e5c0 2024-04-17T09:54:20.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.557+0000 7fdeed000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdecc0039f0 0x7fdee8139620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:20.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.557+0000 7fdeed000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdecc0039f0 0x7fdee8139620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50994/0 (socket says 172.21.15.32:50994) 2024-04-17T09:54:20.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.557+0000 7fdeed000640 1 -- 172.21.15.32:0/334629375 learned_addr learned my addr 172.21.15.32:0/334629375 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:20.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.557+0000 7fdedfe00640 1 --2- 172.21.15.32:0/334629375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdee806e5c0 0x7fdee8138dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:20.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.557+0000 7fdeeda00640 1 --2- 172.21.15.32:0/334629375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdee815e440 0x7fdee815e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:20.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.557+0000 7fdeed000640 1 -- 172.21.15.32:0/334629375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdee806e5c0 msgr2=0x7fdee8138dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:20.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.557+0000 7fdeed000640 1 --2- 172.21.15.32:0/334629375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdee806e5c0 0x7fdee8138dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:20.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.557+0000 7fdeed000640 1 -- 172.21.15.32:0/334629375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdee815e440 msgr2=0x7fdee815e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:20.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.557+0000 7fdeed000640 1 --2- 172.21.15.32:0/334629375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdee815e440 0x7fdee815e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:20.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.557+0000 7fdeed000640 1 -- 172.21.15.32:0/334629375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee8139e50 con 0x7fdecc0039f0 2024-04-17T09:54:20.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.557+0000 7fdeeda00640 1 --2- 172.21.15.32:0/334629375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdee815e440 0x7fdee815e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:20.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.557+0000 7fdeed000640 1 --2- 172.21.15.32:0/334629375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdecc0039f0 0x7fdee8139620 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fded8009ab0 tx=0x7fdecc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:20.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.557+0000 7fdee5600640 1 -- 172.21.15.32:0/334629375 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded80b2040 con 0x7fdecc0039f0 2024-04-17T09:54:20.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.557+0000 7fdee5600640 1 -- 172.21.15.32:0/334629375 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded80a5e40 con 0x7fdecc0039f0 2024-04-17T09:54:20.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.557+0000 7fdeefa00640 1 -- 172.21.15.32:0/334629375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee813a130 con 0x7fdecc0039f0 2024-04-17T09:54:20.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.557+0000 7fdee5600640 1 -- 172.21.15.32:0/334629375 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8004220 con 0x7fdecc0039f0 2024-04-17T09:54:20.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.557+0000 7fdeefa00640 1 -- 172.21.15.32:0/334629375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee813a6f0 con 0x7fdecc0039f0 2024-04-17T09:54:20.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.557+0000 7fdeefa00640 1 -- 172.21.15.32:0/334629375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdee8068a90 con 0x7fdecc0039f0 2024-04-17T09:54:20.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.557+0000 7fdee5600640 1 -- 172.21.15.32:0/334629375 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fded80c0060 con 0x7fdecc0039f0 2024-04-17T09:54:20.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.561+0000 7fdee5600640 1 --2- 172.21.15.32:0/334629375 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdec4068380 0x7fdec406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:20.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.561+0000 7fdee5600640 1 -- 172.21.15.32:0/334629375 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fded81202a0 con 0x7fdecc0039f0 2024-04-17T09:54:20.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.561+0000 7fdedfe00640 1 --2- 172.21.15.32:0/334629375 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdec4068380 0x7fdec406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:20.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.561+0000 7fdedfe00640 1 --2- 172.21.15.32:0/334629375 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdec4068380 0x7fdec406a840 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdee813c620 tx=0x7fded0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:20.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.565+0000 7fdee5600640 1 -- 172.21.15.32:0/334629375 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fded8016760 con 0x7fdecc0039f0 2024-04-17T09:54:20.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.765+0000 7fdeefa00640 1 -- 172.21.15.32:0/334629375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdee806c2c0 con 0x7fdecc0039f0 2024-04-17T09:54:20.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.765+0000 7fdee5600640 1 -- 172.21.15.32:0/334629375 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5199 (secure 0 0 0) 0x7fded80f0170 con 0x7fdecc0039f0 2024-04-17T09:54:20.766 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:20.766 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2127207201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2127207201},{"type":"v1","addr":"172.21.15.144:6835","nonce":2127207201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5110,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3782699135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3782699135},{"type":"v1","addr":"172.21.15.144:6834","nonce":3782699135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/385811857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":385811857},{"type":"v1","addr":"172.21.15.32:6836","nonce":385811857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:54:16.745151+0000","modified":"2024-04-17T09:54:19.761481+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5217":{"gid":5217,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/925293029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":925293029},{"type":"v1","addr":"172.21.15.32:6837","nonce":925293029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5217,"qdb_cluster":[5217]},"id":2}]} 2024-04-17T09:54:20.767 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 15 2024-04-17T09:54:20.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.765+0000 7fdeefa00640 1 -- 172.21.15.32:0/334629375 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdec4068380 msgr2=0x7fdec406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:20.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.765+0000 7fdeefa00640 1 --2- 172.21.15.32:0/334629375 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdec4068380 0x7fdec406a840 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdee813c620 tx=0x7fded0009040 comp rx=0 tx=0).stop 2024-04-17T09:54:20.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.765+0000 7fdeefa00640 1 -- 172.21.15.32:0/334629375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdecc0039f0 msgr2=0x7fdee8139620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:20.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.765+0000 7fdeefa00640 1 --2- 172.21.15.32:0/334629375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdecc0039f0 0x7fdee8139620 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fded8009ab0 tx=0x7fdecc005840 comp rx=0 tx=0).stop 2024-04-17T09:54:20.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.765+0000 7fdeefa00640 1 -- 172.21.15.32:0/334629375 shutdown_connections 2024-04-17T09:54:20.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.765+0000 7fdeefa00640 1 --2- 172.21.15.32:0/334629375 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdec4068380 0x7fdec406a840 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:20.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.765+0000 7fdeefa00640 1 --2- 172.21.15.32:0/334629375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdee806e5c0 0x7fdee8138dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:20.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.765+0000 7fdeefa00640 1 --2- 172.21.15.32:0/334629375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdee815e440 0x7fdee815e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:20.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.765+0000 7fdeefa00640 1 --2- 172.21.15.32:0/334629375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdecc0039f0 0x7fdee8139620 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:20.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.765+0000 7fdeefa00640 1 -- 172.21.15.32:0/334629375 >> 172.21.15.32:0/334629375 conn(0x7fdee806ba30 msgr2=0x7fdee80af6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:20.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.765+0000 7fdeefa00640 1 -- 172.21.15.32:0/334629375 shutdown_connections 2024-04-17T09:54:20.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.765+0000 7fdeefa00640 1 -- 172.21.15.32:0/334629375 wait complete. 2024-04-17T09:54:20.788 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:54:20.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.925+0000 7f2e0d200640 1 Processor -- start 2024-04-17T09:54:20.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.925+0000 7f2e0d200640 1 -- start start 2024-04-17T09:54:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.929+0000 7f2e0d200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e08145f30 0x7f2e08071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.929+0000 7f2e0d200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e080cd580 0x7f2e08072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.929+0000 7f2e0d200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e080cdac0 con 0x7f2e0815d8d0 2024-04-17T09:54:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.929+0000 7f2e0d200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2e0806f400 con 0x7f2e080cd580 2024-04-17T09:54:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.929+0000 7f2e0d200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2e08146310 con 0x7f2e08145f30 2024-04-17T09:54:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.929+0000 7f2e06800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e080cd580 0x7f2e08072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.929+0000 7f2dffe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2e0815d8d0 0x7f2e080cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37086/0 (socket says 172.21.15.32:37086) 2024-04-17T09:54:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.929+0000 7f2dffe00640 1 -- 172.21.15.32:0/3518395557 learned_addr learned my addr 172.21.15.32:0/3518395557 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.929+0000 7f2e05e00640 1 --2- 172.21.15.32:0/3518395557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e08145f30 0x7f2e08071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.929+0000 7f2e06800640 1 -- 172.21.15.32:0/3518395557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e08145f30 msgr2=0x7f2e08071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.929+0000 7f2e06800640 1 --2- 172.21.15.32:0/3518395557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e08145f30 0x7f2e08071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.929+0000 7f2e06800640 1 -- 172.21.15.32:0/3518395557 >> v1:172.21.15.32:6789/0 conn(0x7f2e0815d8d0 legacy=0x7f2e080cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:20.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.929+0000 7f2e06800640 1 -- 172.21.15.32:0/3518395557 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0815cdf0 con 0x7f2e080cd580 2024-04-17T09:54:20.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.933+0000 7f2e06800640 1 --2- 172.21.15.32:0/3518395557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e080cd580 0x7f2e08072330 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009d90 tx=0x7f2e0001f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd2629313441275 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:20.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.933+0000 7f2e05400640 1 -- 172.21.15.32:0/3518395557 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e000b4070 con 0x7f2e080cd580 2024-04-17T09:54:20.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.933+0000 7f2e05400640 1 -- 172.21.15.32:0/3518395557 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e000af600 con 0x7f2e080cd580 2024-04-17T09:54:20.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.933+0000 7f2e05400640 1 -- 172.21.15.32:0/3518395557 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e000af9d0 con 0x7f2e080cd580 2024-04-17T09:54:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.933+0000 7f2e0d200640 1 -- 172.21.15.32:0/3518395557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e080cd580 msgr2=0x7f2e08072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.933+0000 7f2e0d200640 1 --2- 172.21.15.32:0/3518395557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e080cd580 0x7f2e08072330 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009d90 tx=0x7f2e0001f150 comp rx=0 tx=0).stop 2024-04-17T09:54:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.933+0000 7f2e0d200640 1 -- 172.21.15.32:0/3518395557 shutdown_connections 2024-04-17T09:54:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.933+0000 7f2e0d200640 1 --2- 172.21.15.32:0/3518395557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e080cd580 0x7f2e08072330 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.933+0000 7f2e0d200640 1 --2- 172.21.15.32:0/3518395557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e08145f30 0x7f2e08071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.933+0000 7f2e0d200640 1 -- 172.21.15.32:0/3518395557 >> 172.21.15.32:0/3518395557 conn(0x7f2e0806ba30 msgr2=0x7f2e0806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.933+0000 7f2e0d200640 1 -- 172.21.15.32:0/3518395557 shutdown_connections 2024-04-17T09:54:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.933+0000 7f2e0d200640 1 -- 172.21.15.32:0/3518395557 wait complete. 2024-04-17T09:54:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.933+0000 7f2e0d200640 1 Processor -- start 2024-04-17T09:54:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.933+0000 7f2e0d200640 1 -- start start 2024-04-17T09:54:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.933+0000 7f2e0d200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e080cd580 0x7f2e08144340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.933+0000 7f2e0d200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e08145f30 0x7f2e081e8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.933+0000 7f2e0d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e0815d8d0 0x7f2e081f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.933+0000 7f2e0d200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0815cb90 con 0x7f2e0815d8d0 2024-04-17T09:54:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.933+0000 7f2e0d200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0815d440 con 0x7f2e080cd580 2024-04-17T09:54:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.933+0000 7f2e0d200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2e081601d0 con 0x7f2e08145f30 2024-04-17T09:54:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.933+0000 7f2e06800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e0815d8d0 0x7f2e081f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.937+0000 7f2e06800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e0815d8d0 0x7f2e081f7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51004/0 (socket says 172.21.15.32:51004) 2024-04-17T09:54:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.937+0000 7f2e06800640 1 -- 172.21.15.32:0/2554952411 learned_addr learned my addr 172.21.15.32:0/2554952411 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.937+0000 7f2dffe00640 1 --2- 172.21.15.32:0/2554952411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e08145f30 0x7f2e081e8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.937+0000 7f2e05e00640 1 --2- 172.21.15.32:0/2554952411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e080cd580 0x7f2e08144340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.937+0000 7f2e06800640 1 -- 172.21.15.32:0/2554952411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e08145f30 msgr2=0x7f2e081e8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.937+0000 7f2e06800640 1 --2- 172.21.15.32:0/2554952411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e08145f30 0x7f2e081e8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.937+0000 7f2e06800640 1 -- 172.21.15.32:0/2554952411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e080cd580 msgr2=0x7f2e08144340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.937+0000 7f2e06800640 1 --2- 172.21.15.32:0/2554952411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e080cd580 0x7f2e08144340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.937+0000 7f2e06800640 1 -- 172.21.15.32:0/2554952411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e080b4c50 con 0x7f2e0815d8d0 2024-04-17T09:54:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.937+0000 7f2e06800640 1 --2- 172.21.15.32:0/2554952411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e0815d8d0 0x7f2e081f7e60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2e0001d800 tx=0x7f2e0001bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:20.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.937+0000 7f2dfe000640 1 -- 172.21.15.32:0/2554952411 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e000bf020 con 0x7f2e0815d8d0 2024-04-17T09:54:20.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.937+0000 7f2dfe000640 1 -- 172.21.15.32:0/2554952411 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e000ba470 con 0x7f2e0815d8d0 2024-04-17T09:54:20.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.937+0000 7f2dfe000640 1 -- 172.21.15.32:0/2554952411 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e000b4040 con 0x7f2e0815d8d0 2024-04-17T09:54:20.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.937+0000 7f2e0d200640 1 -- 172.21.15.32:0/2554952411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e081e93b0 con 0x7f2e0815d8d0 2024-04-17T09:54:20.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.937+0000 7f2e0d200640 1 -- 172.21.15.32:0/2554952411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e080aa210 con 0x7f2e0815d8d0 2024-04-17T09:54:20.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.937+0000 7f2e0d200640 1 -- 172.21.15.32:0/2554952411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e08068a90 con 0x7f2e0815d8d0 2024-04-17T09:54:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.937+0000 7f2dfe000640 1 -- 172.21.15.32:0/2554952411 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2e000c2020 con 0x7f2e0815d8d0 2024-04-17T09:54:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.937+0000 7f2dfe000640 1 --2- 172.21.15.32:0/2554952411 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2de40683b0 0x7f2de406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.937+0000 7f2dfe000640 1 -- 172.21.15.32:0/2554952411 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f2e001252b0 con 0x7f2e0815d8d0 2024-04-17T09:54:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.941+0000 7f2e05e00640 1 --2- 172.21.15.32:0/2554952411 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2de40683b0 0x7f2de406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:20.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.941+0000 7f2e05e00640 1 --2- 172.21.15.32:0/2554952411 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2de40683b0 0x7f2de406a870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2df800b9d0 tx=0x7f2df8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:20.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:20.941+0000 7f2dfe000640 1 -- 172.21.15.32:0/2554952411 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2e000c9030 con 0x7f2e0815d8d0 2024-04-17T09:54:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.109+0000 7f2e0d200640 1 -- 172.21.15.32:0/2554952411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e081e95e0 con 0x7f2e0815d8d0 2024-04-17T09:54:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.109+0000 7f2dfe000640 1 -- 172.21.15.32:0/2554952411 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16681 (secure 0 0 0) 0x7f2e00018cb0 con 0x7f2e0815d8d0 2024-04-17T09:54:21.112 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:21.113 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:54:18.351519+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:54:15.261471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-17T09:54:15.704958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:54:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.113+0000 7f2e0d200640 1 -- 172.21.15.32:0/2554952411 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2de40683b0 msgr2=0x7f2de406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.113+0000 7f2e0d200640 1 --2- 172.21.15.32:0/2554952411 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2de40683b0 0x7f2de406a870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2df800b9d0 tx=0x7f2df8008040 comp rx=0 tx=0).stop 2024-04-17T09:54:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.113+0000 7f2e0d200640 1 -- 172.21.15.32:0/2554952411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e0815d8d0 msgr2=0x7f2e081f7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.113+0000 7f2e0d200640 1 --2- 172.21.15.32:0/2554952411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e0815d8d0 0x7f2e081f7e60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2e0001d800 tx=0x7f2e0001bef0 comp rx=0 tx=0).stop 2024-04-17T09:54:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.113+0000 7f2e0d200640 1 -- 172.21.15.32:0/2554952411 shutdown_connections 2024-04-17T09:54:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.113+0000 7f2e0d200640 1 --2- 172.21.15.32:0/2554952411 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2de40683b0 0x7f2de406a870 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.113+0000 7f2e0d200640 1 --2- 172.21.15.32:0/2554952411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e08145f30 0x7f2e081e8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.113+0000 7f2e0d200640 1 --2- 172.21.15.32:0/2554952411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e0815d8d0 0x7f2e081f7e60 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:21.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.113+0000 7f2e0d200640 1 --2- 172.21.15.32:0/2554952411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e080cd580 0x7f2e08144340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:21.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.113+0000 7f2e0d200640 1 -- 172.21.15.32:0/2554952411 >> 172.21.15.32:0/2554952411 conn(0x7f2e0806ba30 msgr2=0x7f2e0815dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:21.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.113+0000 7f2e0d200640 1 -- 172.21.15.32:0/2554952411 shutdown_connections 2024-04-17T09:54:21.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.113+0000 7f2e0d200640 1 -- 172.21.15.32:0/2554952411 wait complete. 2024-04-17T09:54:21.134 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T09:54:21.135 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T09:54:21.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.273+0000 7fed0c200640 1 Processor -- start 2024-04-17T09:54:21.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.273+0000 7fed0c200640 1 -- start start 2024-04-17T09:54:21.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.273+0000 7fed0c200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fed04072500 0x7fed040cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:21.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.273+0000 7fed0c200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed0415d8d0 0x7fed040cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:21.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.273+0000 7fed0c200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed040cdac0 con 0x7fed04145f30 2024-04-17T09:54:21.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.273+0000 7fed0c200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fed0406f400 con 0x7fed04072500 2024-04-17T09:54:21.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.273+0000 7fed0c200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fed04146310 con 0x7fed0415d8d0 2024-04-17T09:54:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.277+0000 7fed0a200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed0415d8d0 0x7fed040cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.277+0000 7fed08e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fed04072500 0x7fed040cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.277+0000 7fed09800640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fed04145f30 0x7fed04071df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:52182/0 (socket says 172.21.15.32:52182) 2024-04-17T09:54:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.277+0000 7fed09800640 1 -- 172.21.15.32:0/2716554604 learned_addr learned my addr 172.21.15.32:0/2716554604 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.277+0000 7fed0a200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed0415d8d0 0x7fed040cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:55888/0 (socket says 172.21.15.32:55888) 2024-04-17T09:54:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.277+0000 7fed03e00640 1 -- 172.21.15.32:0/2716554604 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1850656051 0 0) 0x7fed040cdac0 con 0x7fed04145f30 2024-04-17T09:54:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.277+0000 7fed0a200640 1 -- 172.21.15.32:0/2716554604 >> v2:172.21.15.144:3300/0 conn(0x7fed04072500 msgr2=0x7fed040cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.277+0000 7fed0a200640 1 --2- 172.21.15.32:0/2716554604 >> v2:172.21.15.144:3300/0 conn(0x7fed04072500 0x7fed040cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.277+0000 7fed0a200640 1 -- 172.21.15.32:0/2716554604 >> v1:172.21.15.144:6789/0 conn(0x7fed04145f30 legacy=0x7fed04071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.277+0000 7fed0a200640 1 -- 172.21.15.32:0/2716554604 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0415cdf0 con 0x7fed0415d8d0 2024-04-17T09:54:21.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.277+0000 7fed08e00640 1 --2- 172.21.15.32:0/2716554604 >> v2:172.21.15.144:3300/0 conn(0x7fed04072500 0x7fed040cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:54:21.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.277+0000 7fed0a200640 1 --2- 172.21.15.32:0/2716554604 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed0415d8d0 0x7fed040cd3b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009d90 tx=0x7fecfc01f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cc1fee59585ccd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:21.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed03e00640 1 -- 172.21.15.32:0/2716554604 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc0b4070 con 0x7fed0415d8d0 2024-04-17T09:54:21.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed03e00640 1 -- 172.21.15.32:0/2716554604 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecfc0af600 con 0x7fed0415d8d0 2024-04-17T09:54:21.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed03e00640 1 -- 172.21.15.32:0/2716554604 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc0af9d0 con 0x7fed0415d8d0 2024-04-17T09:54:21.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed0c200640 1 -- 172.21.15.32:0/2716554604 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed0415d8d0 msgr2=0x7fed040cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:21.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed0c200640 1 --2- 172.21.15.32:0/2716554604 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed0415d8d0 0x7fed040cd3b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009d90 tx=0x7fecfc01f150 comp rx=0 tx=0).stop 2024-04-17T09:54:21.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed0c200640 1 -- 172.21.15.32:0/2716554604 shutdown_connections 2024-04-17T09:54:21.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed0c200640 1 --2- 172.21.15.32:0/2716554604 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed0415d8d0 0x7fed040cd3b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:21.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed0c200640 1 --2- 172.21.15.32:0/2716554604 >> v2:172.21.15.144:3300/0 conn(0x7fed04072500 0x7fed040cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:21.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed0c200640 1 -- 172.21.15.32:0/2716554604 >> 172.21.15.32:0/2716554604 conn(0x7fed0406ba30 msgr2=0x7fed0406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:21.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed0c200640 1 -- 172.21.15.32:0/2716554604 shutdown_connections 2024-04-17T09:54:21.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed0c200640 1 -- 172.21.15.32:0/2716554604 wait complete. 2024-04-17T09:54:21.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed0c200640 1 Processor -- start 2024-04-17T09:54:21.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed0c200640 1 -- start start 2024-04-17T09:54:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed0c200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed04072500 0x7fed04140d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed0c200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed04145f30 0x7fed04141680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed0c200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed0415d8d0 0x7fed0415c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed0c200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fed0415cb90 con 0x7fed0415d8d0 2024-04-17T09:54:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed0c200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fed0415d440 con 0x7fed04072500 2024-04-17T09:54:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed0c200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fed041601d0 con 0x7fed04145f30 2024-04-17T09:54:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed0a200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed0415d8d0 0x7fed0415c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed0a200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed0415d8d0 0x7fed0415c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51014/0 (socket says 172.21.15.32:51014) 2024-04-17T09:54:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed0a200640 1 -- 172.21.15.32:0/556590171 learned_addr learned my addr 172.21.15.32:0/556590171 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed09800640 1 --2- 172.21.15.32:0/556590171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed04072500 0x7fed04140d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.281+0000 7fed08e00640 1 --2- 172.21.15.32:0/556590171 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed04145f30 0x7fed04141680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.285+0000 7fed0a200640 1 -- 172.21.15.32:0/556590171 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed04145f30 msgr2=0x7fed04141680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.285+0000 7fed0a200640 1 --2- 172.21.15.32:0/556590171 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed04145f30 0x7fed04141680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:21.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.285+0000 7fed0a200640 1 -- 172.21.15.32:0/556590171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed04072500 msgr2=0x7fed04140d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:21.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.285+0000 7fed0a200640 1 --2- 172.21.15.32:0/556590171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed04072500 0x7fed04140d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:21.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.285+0000 7fed0a200640 1 -- 172.21.15.32:0/556590171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed04141bc0 con 0x7fed0415d8d0 2024-04-17T09:54:21.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.285+0000 7fed0a200640 1 --2- 172.21.15.32:0/556590171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed0415d8d0 0x7fed0415c0e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fecfc01cd80 tx=0x7fecfc01bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:21.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.285+0000 7fed01600640 1 -- 172.21.15.32:0/556590171 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc0b4040 con 0x7fed0415d8d0 2024-04-17T09:54:21.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.285+0000 7fed01600640 1 -- 172.21.15.32:0/556590171 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecfc0c1070 con 0x7fed0415d8d0 2024-04-17T09:54:21.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.285+0000 7fed01600640 1 -- 172.21.15.32:0/556590171 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc009450 con 0x7fed0415d8d0 2024-04-17T09:54:21.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.285+0000 7fed0c200640 1 -- 172.21.15.32:0/556590171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed04141e60 con 0x7fed0415d8d0 2024-04-17T09:54:21.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.285+0000 7fed0c200640 1 -- 172.21.15.32:0/556590171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed04142420 con 0x7fed0415d8d0 2024-04-17T09:54:21.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.285+0000 7fed0c200640 1 -- 172.21.15.32:0/556590171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed04068a90 con 0x7fed0415d8d0 2024-04-17T09:54:21.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.285+0000 7fed01600640 1 -- 172.21.15.32:0/556590171 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fecfc0ba6b0 con 0x7fed0415d8d0 2024-04-17T09:54:21.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.285+0000 7fed01600640 1 --2- 172.21.15.32:0/556590171 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fece4068380 0x7fece406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:21.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.285+0000 7fed01600640 1 -- 172.21.15.32:0/556590171 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fecfc125270 con 0x7fed0415d8d0 2024-04-17T09:54:21.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.285+0000 7fed09800640 1 --2- 172.21.15.32:0/556590171 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fece4068380 0x7fece406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:21.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.289+0000 7fed09800640 1 --2- 172.21.15.32:0/556590171 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fece4068380 0x7fece406a840 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fecf8007fd0 tx=0x7fecf8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:21.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.289+0000 7fed01600640 1 -- 172.21.15.32:0/556590171 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fecfc018cb0 con 0x7fed0415d8d0 2024-04-17T09:54:21.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.453+0000 7fed0c200640 1 -- 172.21.15.32:0/556590171 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fed04072f90 con 0x7fece4068380 2024-04-17T09:54:21.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.461+0000 7fed01600640 1 -- 172.21.15.32:0/556590171 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251950 (secure 0 0 0) 0x7fed04072f90 con 0x7fece4068380 2024-04-17T09:54:21.465 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:21.467 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T09:54:21.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.465+0000 7fed0c200640 1 -- 172.21.15.32:0/556590171 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fece4068380 msgr2=0x7fece406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:21.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.465+0000 7fed0c200640 1 --2- 172.21.15.32:0/556590171 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fece4068380 0x7fece406a840 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fecf8007fd0 tx=0x7fecf8009040 comp rx=0 tx=0).stop 2024-04-17T09:54:21.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.465+0000 7fed0c200640 1 -- 172.21.15.32:0/556590171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed0415d8d0 msgr2=0x7fed0415c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:21.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.465+0000 7fed0c200640 1 --2- 172.21.15.32:0/556590171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed0415d8d0 0x7fed0415c0e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fecfc01cd80 tx=0x7fecfc01bef0 comp rx=0 tx=0).stop 2024-04-17T09:54:21.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.465+0000 7fed0c200640 1 -- 172.21.15.32:0/556590171 shutdown_connections 2024-04-17T09:54:21.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.465+0000 7fed0c200640 1 --2- 172.21.15.32:0/556590171 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fece4068380 0x7fece406a840 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:21.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.465+0000 7fed0c200640 1 --2- 172.21.15.32:0/556590171 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed04145f30 0x7fed04141680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:21.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.465+0000 7fed0c200640 1 --2- 172.21.15.32:0/556590171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed0415d8d0 0x7fed0415c0e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:21.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.465+0000 7fed0c200640 1 --2- 172.21.15.32:0/556590171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed04072500 0x7fed04140d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:21.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.465+0000 7fed0c200640 1 -- 172.21.15.32:0/556590171 >> 172.21.15.32:0/556590171 conn(0x7fed0406ba30 msgr2=0x7fed04160600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:21.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.465+0000 7fed0c200640 1 -- 172.21.15.32:0/556590171 shutdown_connections 2024-04-17T09:54:21.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.465+0000 7fed0c200640 1 -- 172.21.15.32:0/556590171 wait complete. 2024-04-17T09:54:21.486 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-04-17T09:54:21.298196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241244,"kb_used_data":4264,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733476,"statfs":{"total":773094113280,"available":772847079424,"internally_reserved":0,"allocated":4366336,"data_stored":2410964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.961770"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355927+0000","last_change":"2024-04-17T09:54:17.376160+0000","last_active":"2024-04-17T09:54:18.355927+0000","last_peered":"2024-04-17T09:54:18.355927+0000","last_clean":"2024-04-17T09:54:18.355927+0000","last_became_active":"2024-04-17T09:54:17.374386+0000","last_became_peered":"2024-04-17T09:54:17.374386+0000","last_unstale":"2024-04-17T09:54:18.355927+0000","last_undegraded":"2024-04-17T09:54:18.355927+0000","last_fullsized":"2024-04-17T09:54:18.355927+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:34:22.500707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356387+0000","last_change":"2024-04-17T09:54:17.370783+0000","last_active":"2024-04-17T09:54:18.356387+0000","last_peered":"2024-04-17T09:54:18.356387+0000","last_clean":"2024-04-17T09:54:18.356387+0000","last_became_active":"2024-04-17T09:54:17.370388+0000","last_became_peered":"2024-04-17T09:54:17.370388+0000","last_unstale":"2024-04-17T09:54:18.356387+0000","last_undegraded":"2024-04-17T09:54:18.356387+0000","last_fullsized":"2024-04-17T09:54:18.356387+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:31:39.270271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948487+0000","last_change":"2024-04-17T09:54:17.383521+0000","last_active":"2024-04-17T09:54:18.948487+0000","last_peered":"2024-04-17T09:54:18.948487+0000","last_clean":"2024-04-17T09:54:18.948487+0000","last_became_active":"2024-04-17T09:54:17.383239+0000","last_became_peered":"2024-04-17T09:54:17.383239+0000","last_unstale":"2024-04-17T09:54:18.948487+0000","last_undegraded":"2024-04-17T09:54:18.948487+0000","last_fullsized":"2024-04-17T09:54:18.948487+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:04:35.939029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395162+0000","last_change":"2024-04-17T09:54:17.384642+0000","last_active":"2024-04-17T09:54:18.395162+0000","last_peered":"2024-04-17T09:54:18.395162+0000","last_clean":"2024-04-17T09:54:18.395162+0000","last_became_active":"2024-04-17T09:54:17.384356+0000","last_became_peered":"2024-04-17T09:54:17.384356+0000","last_unstale":"2024-04-17T09:54:18.395162+0000","last_undegraded":"2024-04-17T09:54:18.395162+0000","last_fullsized":"2024-04-17T09:54:18.395162+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:08.210894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.448200+0000","last_change":"2024-04-17T09:54:17.382868+0000","last_active":"2024-04-17T09:54:18.448200+0000","last_peered":"2024-04-17T09:54:18.448200+0000","last_clean":"2024-04-17T09:54:18.448200+0000","last_became_active":"2024-04-17T09:54:17.382623+0000","last_became_peered":"2024-04-17T09:54:17.382623+0000","last_unstale":"2024-04-17T09:54:18.448200+0000","last_undegraded":"2024-04-17T09:54:18.448200+0000","last_fullsized":"2024-04-17T09:54:18.448200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:38:46.982126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355647+0000","last_change":"2024-04-17T09:54:17.375486+0000","last_active":"2024-04-17T09:54:18.355647+0000","last_peered":"2024-04-17T09:54:18.355647+0000","last_clean":"2024-04-17T09:54:18.355647+0000","last_became_active":"2024-04-17T09:54:17.374997+0000","last_became_peered":"2024-04-17T09:54:17.374997+0000","last_unstale":"2024-04-17T09:54:18.355647+0000","last_undegraded":"2024-04-17T09:54:18.355647+0000","last_fullsized":"2024-04-17T09:54:18.355647+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:24:07.861301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356830+0000","last_change":"2024-04-17T09:54:17.378457+0000","last_active":"2024-04-17T09:54:18.356830+0000","last_peered":"2024-04-17T09:54:18.356830+0000","last_clean":"2024-04-17T09:54:18.356830+0000","last_became_active":"2024-04-17T09:54:17.377938+0000","last_became_peered":"2024-04-17T09:54:17.377938+0000","last_unstale":"2024-04-17T09:54:18.356830+0000","last_undegraded":"2024-04-17T09:54:18.356830+0000","last_fullsized":"2024-04-17T09:54:18.356830+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:31:39.270271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356002+0000","last_change":"2024-04-17T09:54:17.381751+0000","last_active":"2024-04-17T09:54:18.356002+0000","last_peered":"2024-04-17T09:54:18.356002+0000","last_clean":"2024-04-17T09:54:18.356002+0000","last_became_active":"2024-04-17T09:54:17.381321+0000","last_became_peered":"2024-04-17T09:54:17.381321+0000","last_unstale":"2024-04-17T09:54:18.356002+0000","last_undegraded":"2024-04-17T09:54:18.356002+0000","last_fullsized":"2024-04-17T09:54:18.356002+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:04:35.939029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949486+0000","last_change":"2024-04-17T09:54:17.378952+0000","last_active":"2024-04-17T09:54:18.949486+0000","last_peered":"2024-04-17T09:54:18.949486+0000","last_clean":"2024-04-17T09:54:18.949486+0000","last_became_active":"2024-04-17T09:54:17.378641+0000","last_became_peered":"2024-04-17T09:54:17.378641+0000","last_unstale":"2024-04-17T09:54:18.949486+0000","last_undegraded":"2024-04-17T09:54:18.949486+0000","last_fullsized":"2024-04-17T09:54:18.949486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:11:30.035005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395470+0000","last_change":"2024-04-17T09:54:17.384874+0000","last_active":"2024-04-17T09:54:18.395470+0000","last_peered":"2024-04-17T09:54:18.395470+0000","last_clean":"2024-04-17T09:54:18.395470+0000","last_became_active":"2024-04-17T09:54:17.384562+0000","last_became_peered":"2024-04-17T09:54:17.384562+0000","last_unstale":"2024-04-17T09:54:18.395470+0000","last_undegraded":"2024-04-17T09:54:18.395470+0000","last_fullsized":"2024-04-17T09:54:18.395470+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:10:54.779389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356548+0000","last_change":"2024-04-17T09:54:17.381734+0000","last_active":"2024-04-17T09:54:18.356548+0000","last_peered":"2024-04-17T09:54:18.356548+0000","last_clean":"2024-04-17T09:54:18.356548+0000","last_became_active":"2024-04-17T09:54:17.381425+0000","last_became_peered":"2024-04-17T09:54:17.381425+0000","last_unstale":"2024-04-17T09:54:18.356548+0000","last_undegraded":"2024-04-17T09:54:18.356548+0000","last_fullsized":"2024-04-17T09:54:18.356548+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:42:58.110213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948912+0000","last_change":"2024-04-17T09:54:17.383440+0000","last_active":"2024-04-17T09:54:18.948912+0000","last_peered":"2024-04-17T09:54:18.948912+0000","last_clean":"2024-04-17T09:54:18.948912+0000","last_became_active":"2024-04-17T09:54:17.383101+0000","last_became_peered":"2024-04-17T09:54:17.383101+0000","last_unstale":"2024-04-17T09:54:18.948912+0000","last_undegraded":"2024-04-17T09:54:18.948912+0000","last_fullsized":"2024-04-17T09:54:18.948912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:54.371142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.448087+0000","last_change":"2024-04-17T09:54:17.379410+0000","last_active":"2024-04-17T09:54:18.448087+0000","last_peered":"2024-04-17T09:54:18.448087+0000","last_clean":"2024-04-17T09:54:18.448087+0000","last_became_active":"2024-04-17T09:54:17.379141+0000","last_became_peered":"2024-04-17T09:54:17.379141+0000","last_unstale":"2024-04-17T09:54:18.448087+0000","last_undegraded":"2024-04-17T09:54:18.448087+0000","last_fullsized":"2024-04-17T09:54:18.448087+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:02:43.944515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355673+0000","last_change":"2024-04-17T09:54:17.367214+0000","last_active":"2024-04-17T09:54:18.355673+0000","last_peered":"2024-04-17T09:54:18.355673+0000","last_clean":"2024-04-17T09:54:18.355673+0000","last_became_active":"2024-04-17T09:54:17.366900+0000","last_became_peered":"2024-04-17T09:54:17.366900+0000","last_unstale":"2024-04-17T09:54:18.355673+0000","last_undegraded":"2024-04-17T09:54:18.355673+0000","last_fullsized":"2024-04-17T09:54:18.355673+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:08:56.864943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948391+0000","last_change":"2024-04-17T09:54:17.365335+0000","last_active":"2024-04-17T09:54:18.948391+0000","last_peered":"2024-04-17T09:54:18.948391+0000","last_clean":"2024-04-17T09:54:18.948391+0000","last_became_active":"2024-04-17T09:54:17.363002+0000","last_became_peered":"2024-04-17T09:54:17.363002+0000","last_unstale":"2024-04-17T09:54:18.948391+0000","last_undegraded":"2024-04-17T09:54:18.948391+0000","last_fullsized":"2024-04-17T09:54:18.948391+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:26:01.393314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356455+0000","last_change":"2024-04-17T09:54:17.386566+0000","last_active":"2024-04-17T09:54:18.356455+0000","last_peered":"2024-04-17T09:54:18.356455+0000","last_clean":"2024-04-17T09:54:18.356455+0000","last_became_active":"2024-04-17T09:54:17.386176+0000","last_became_peered":"2024-04-17T09:54:17.386176+0000","last_unstale":"2024-04-17T09:54:18.356455+0000","last_undegraded":"2024-04-17T09:54:18.356455+0000","last_fullsized":"2024-04-17T09:54:18.356455+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:32:50.828977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356728+0000","last_change":"2024-04-17T09:54:17.382008+0000","last_active":"2024-04-17T09:54:18.356728+0000","last_peered":"2024-04-17T09:54:18.356728+0000","last_clean":"2024-04-17T09:54:18.356728+0000","last_became_active":"2024-04-17T09:54:17.381686+0000","last_became_peered":"2024-04-17T09:54:17.381686+0000","last_unstale":"2024-04-17T09:54:18.356728+0000","last_undegraded":"2024-04-17T09:54:18.356728+0000","last_fullsized":"2024-04-17T09:54:18.356728+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:56:37.564448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356415+0000","last_change":"2024-04-17T09:54:17.384926+0000","last_active":"2024-04-17T09:54:18.356415+0000","last_peered":"2024-04-17T09:54:18.356415+0000","last_clean":"2024-04-17T09:54:18.356415+0000","last_became_active":"2024-04-17T09:54:17.381926+0000","last_became_peered":"2024-04-17T09:54:17.381926+0000","last_unstale":"2024-04-17T09:54:18.356415+0000","last_undegraded":"2024-04-17T09:54:18.356415+0000","last_fullsized":"2024-04-17T09:54:18.356415+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:04:48.927345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355942+0000","last_change":"2024-04-17T09:54:17.387372+0000","last_active":"2024-04-17T09:54:18.355942+0000","last_peered":"2024-04-17T09:54:18.355942+0000","last_clean":"2024-04-17T09:54:18.355942+0000","last_became_active":"2024-04-17T09:54:17.387086+0000","last_became_peered":"2024-04-17T09:54:17.387086+0000","last_unstale":"2024-04-17T09:54:18.355942+0000","last_undegraded":"2024-04-17T09:54:18.355942+0000","last_fullsized":"2024-04-17T09:54:18.355942+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:21.920667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.448202+0000","last_change":"2024-04-17T09:54:17.374765+0000","last_active":"2024-04-17T09:54:18.448202+0000","last_peered":"2024-04-17T09:54:18.448202+0000","last_clean":"2024-04-17T09:54:18.448202+0000","last_became_active":"2024-04-17T09:54:17.374409+0000","last_became_peered":"2024-04-17T09:54:17.374409+0000","last_unstale":"2024-04-17T09:54:18.448202+0000","last_undegraded":"2024-04-17T09:54:18.448202+0000","last_fullsized":"2024-04-17T09:54:18.448202+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:58.120418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948817+0000","last_change":"2024-04-17T09:54:17.368495+0000","last_active":"2024-04-17T09:54:18.948817+0000","last_peered":"2024-04-17T09:54:18.948817+0000","last_clean":"2024-04-17T09:54:18.948817+0000","last_became_active":"2024-04-17T09:54:17.368225+0000","last_became_peered":"2024-04-17T09:54:17.368225+0000","last_unstale":"2024-04-17T09:54:18.948817+0000","last_undegraded":"2024-04-17T09:54:18.948817+0000","last_fullsized":"2024-04-17T09:54:18.948817+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:48:46.480627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949116+0000","last_change":"2024-04-17T09:54:17.371677+0000","last_active":"2024-04-17T09:54:18.949116+0000","last_peered":"2024-04-17T09:54:18.949116+0000","last_clean":"2024-04-17T09:54:18.949116+0000","last_became_active":"2024-04-17T09:54:17.371346+0000","last_became_peered":"2024-04-17T09:54:17.371346+0000","last_unstale":"2024-04-17T09:54:18.949116+0000","last_undegraded":"2024-04-17T09:54:18.949116+0000","last_fullsized":"2024-04-17T09:54:18.949116+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:07.206960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.392506+0000","last_change":"2024-04-17T09:54:17.371801+0000","last_active":"2024-04-17T09:54:18.392506+0000","last_peered":"2024-04-17T09:54:18.392506+0000","last_clean":"2024-04-17T09:54:18.392506+0000","last_became_active":"2024-04-17T09:54:17.371472+0000","last_became_peered":"2024-04-17T09:54:17.371472+0000","last_unstale":"2024-04-17T09:54:18.392506+0000","last_undegraded":"2024-04-17T09:54:18.392506+0000","last_fullsized":"2024-04-17T09:54:18.392506+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:58.710579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948334+0000","last_change":"2024-04-17T09:54:17.368572+0000","last_active":"2024-04-17T09:54:18.948334+0000","last_peered":"2024-04-17T09:54:18.948334+0000","last_clean":"2024-04-17T09:54:18.948334+0000","last_became_active":"2024-04-17T09:54:17.367856+0000","last_became_peered":"2024-04-17T09:54:17.367856+0000","last_unstale":"2024-04-17T09:54:18.948334+0000","last_undegraded":"2024-04-17T09:54:18.948334+0000","last_fullsized":"2024-04-17T09:54:18.948334+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:05:17.694457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395514+0000","last_change":"2024-04-17T09:54:17.380161+0000","last_active":"2024-04-17T09:54:18.395514+0000","last_peered":"2024-04-17T09:54:18.395514+0000","last_clean":"2024-04-17T09:54:18.395514+0000","last_became_active":"2024-04-17T09:54:17.379809+0000","last_became_peered":"2024-04-17T09:54:17.379809+0000","last_unstale":"2024-04-17T09:54:18.395514+0000","last_undegraded":"2024-04-17T09:54:18.395514+0000","last_fullsized":"2024-04-17T09:54:18.395514+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:05:17.694457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356536+0000","last_change":"2024-04-17T09:54:17.373209+0000","last_active":"2024-04-17T09:54:18.356536+0000","last_peered":"2024-04-17T09:54:18.356536+0000","last_clean":"2024-04-17T09:54:18.356536+0000","last_became_active":"2024-04-17T09:54:17.372647+0000","last_became_peered":"2024-04-17T09:54:17.372647+0000","last_unstale":"2024-04-17T09:54:18.356536+0000","last_undegraded":"2024-04-17T09:54:18.356536+0000","last_fullsized":"2024-04-17T09:54:18.356536+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:35.050031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395523+0000","last_change":"2024-04-17T09:54:17.387523+0000","last_active":"2024-04-17T09:54:18.395523+0000","last_peered":"2024-04-17T09:54:18.395523+0000","last_clean":"2024-04-17T09:54:18.395523+0000","last_became_active":"2024-04-17T09:54:17.384217+0000","last_became_peered":"2024-04-17T09:54:17.384217+0000","last_unstale":"2024-04-17T09:54:18.395523+0000","last_undegraded":"2024-04-17T09:54:18.395523+0000","last_fullsized":"2024-04-17T09:54:18.395523+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:26:01.393314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949066+0000","last_change":"2024-04-17T09:54:17.372311+0000","last_active":"2024-04-17T09:54:18.949066+0000","last_peered":"2024-04-17T09:54:18.949066+0000","last_clean":"2024-04-17T09:54:18.949066+0000","last_became_active":"2024-04-17T09:54:17.367208+0000","last_became_peered":"2024-04-17T09:54:17.367208+0000","last_unstale":"2024-04-17T09:54:18.949066+0000","last_undegraded":"2024-04-17T09:54:18.949066+0000","last_fullsized":"2024-04-17T09:54:18.949066+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:08:56.864943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948921+0000","last_change":"2024-04-17T09:54:17.371697+0000","last_active":"2024-04-17T09:54:18.948921+0000","last_peered":"2024-04-17T09:54:18.948921+0000","last_clean":"2024-04-17T09:54:18.948921+0000","last_became_active":"2024-04-17T09:54:17.371423+0000","last_became_peered":"2024-04-17T09:54:17.371423+0000","last_unstale":"2024-04-17T09:54:18.948921+0000","last_undegraded":"2024-04-17T09:54:18.948921+0000","last_fullsized":"2024-04-17T09:54:18.948921+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:58.710579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356175+0000","last_change":"2024-04-17T09:54:17.382293+0000","last_active":"2024-04-17T09:54:18.356175+0000","last_peered":"2024-04-17T09:54:18.356175+0000","last_clean":"2024-04-17T09:54:18.356175+0000","last_became_active":"2024-04-17T09:54:17.382015+0000","last_became_peered":"2024-04-17T09:54:17.382015+0000","last_unstale":"2024-04-17T09:54:18.356175+0000","last_undegraded":"2024-04-17T09:54:18.356175+0000","last_fullsized":"2024-04-17T09:54:18.356175+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:44:35.259640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355950+0000","last_change":"2024-04-17T09:54:17.366475+0000","last_active":"2024-04-17T09:54:18.355950+0000","last_peered":"2024-04-17T09:54:18.355950+0000","last_clean":"2024-04-17T09:54:18.355950+0000","last_became_active":"2024-04-17T09:54:17.365805+0000","last_became_peered":"2024-04-17T09:54:17.365805+0000","last_unstale":"2024-04-17T09:54:18.355950+0000","last_undegraded":"2024-04-17T09:54:18.355950+0000","last_fullsized":"2024-04-17T09:54:18.355950+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:49:29.350387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356020+0000","last_change":"2024-04-17T09:54:17.376263+0000","last_active":"2024-04-17T09:54:18.356020+0000","last_peered":"2024-04-17T09:54:18.356020+0000","last_clean":"2024-04-17T09:54:18.356020+0000","last_became_active":"2024-04-17T09:54:17.375608+0000","last_became_peered":"2024-04-17T09:54:17.375608+0000","last_unstale":"2024-04-17T09:54:18.356020+0000","last_undegraded":"2024-04-17T09:54:18.356020+0000","last_fullsized":"2024-04-17T09:54:18.356020+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:46:43.928292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.392786+0000","last_change":"2024-04-17T09:54:17.373590+0000","last_active":"2024-04-17T09:54:18.392786+0000","last_peered":"2024-04-17T09:54:18.392786+0000","last_clean":"2024-04-17T09:54:18.392786+0000","last_became_active":"2024-04-17T09:54:17.373319+0000","last_became_peered":"2024-04-17T09:54:17.373319+0000","last_unstale":"2024-04-17T09:54:18.392786+0000","last_undegraded":"2024-04-17T09:54:18.392786+0000","last_fullsized":"2024-04-17T09:54:18.392786+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:22:28.141447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356089+0000","last_change":"2024-04-17T09:54:16.360332+0000","last_active":"2024-04-17T09:54:18.356089+0000","last_peered":"2024-04-17T09:54:18.356089+0000","last_clean":"2024-04-17T09:54:18.356089+0000","last_became_active":"2024-04-17T09:54:16.360073+0000","last_became_peered":"2024-04-17T09:54:16.360073+0000","last_unstale":"2024-04-17T09:54:18.356089+0000","last_undegraded":"2024-04-17T09:54:18.356089+0000","last_fullsized":"2024-04-17T09:54:18.356089+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:53:03.772098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356083+0000","last_change":"2024-04-17T09:54:17.364860+0000","last_active":"2024-04-17T09:54:18.356083+0000","last_peered":"2024-04-17T09:54:18.356083+0000","last_clean":"2024-04-17T09:54:18.356083+0000","last_became_active":"2024-04-17T09:54:17.364484+0000","last_became_peered":"2024-04-17T09:54:17.364484+0000","last_unstale":"2024-04-17T09:54:18.356083+0000","last_undegraded":"2024-04-17T09:54:18.356083+0000","last_fullsized":"2024-04-17T09:54:18.356083+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:03:05.817393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.392822+0000","last_change":"2024-04-17T09:54:16.363721+0000","last_active":"2024-04-17T09:54:18.392822+0000","last_peered":"2024-04-17T09:54:18.392822+0000","last_clean":"2024-04-17T09:54:18.392822+0000","last_became_active":"2024-04-17T09:54:16.363083+0000","last_became_peered":"2024-04-17T09:54:16.363083+0000","last_unstale":"2024-04-17T09:54:18.392822+0000","last_undegraded":"2024-04-17T09:54:18.392822+0000","last_fullsized":"2024-04-17T09:54:18.392822+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:55:39.879843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949177+0000","last_change":"2024-04-17T09:54:17.369252+0000","last_active":"2024-04-17T09:54:18.949177+0000","last_peered":"2024-04-17T09:54:18.949177+0000","last_clean":"2024-04-17T09:54:18.949177+0000","last_became_active":"2024-04-17T09:54:17.368808+0000","last_became_peered":"2024-04-17T09:54:17.368808+0000","last_unstale":"2024-04-17T09:54:18.949177+0000","last_undegraded":"2024-04-17T09:54:18.949177+0000","last_fullsized":"2024-04-17T09:54:18.949177+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:10.103965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356099+0000","last_change":"2024-04-17T09:54:16.358302+0000","last_active":"2024-04-17T09:54:18.356099+0000","last_peered":"2024-04-17T09:54:18.356099+0000","last_clean":"2024-04-17T09:54:18.356099+0000","last_became_active":"2024-04-17T09:54:16.358036+0000","last_became_peered":"2024-04-17T09:54:16.358036+0000","last_unstale":"2024-04-17T09:54:18.356099+0000","last_undegraded":"2024-04-17T09:54:18.356099+0000","last_fullsized":"2024-04-17T09:54:18.356099+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:34:21.495573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395656+0000","last_change":"2024-04-17T09:54:17.381113+0000","last_active":"2024-04-17T09:54:18.395656+0000","last_peered":"2024-04-17T09:54:18.395656+0000","last_clean":"2024-04-17T09:54:18.395656+0000","last_became_active":"2024-04-17T09:54:17.380867+0000","last_became_peered":"2024-04-17T09:54:17.380867+0000","last_unstale":"2024-04-17T09:54:18.395656+0000","last_undegraded":"2024-04-17T09:54:18.395656+0000","last_fullsized":"2024-04-17T09:54:18.395656+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:25.615888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.448228+0000","last_change":"2024-04-17T09:54:16.369646+0000","last_active":"2024-04-17T09:54:18.448228+0000","last_peered":"2024-04-17T09:54:18.448228+0000","last_clean":"2024-04-17T09:54:18.448228+0000","last_became_active":"2024-04-17T09:54:16.369382+0000","last_became_peered":"2024-04-17T09:54:16.369382+0000","last_unstale":"2024-04-17T09:54:18.448228+0000","last_undegraded":"2024-04-17T09:54:18.448228+0000","last_fullsized":"2024-04-17T09:54:18.448228+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:31:39.263793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356378+0000","last_change":"2024-04-17T09:54:17.380563+0000","last_active":"2024-04-17T09:54:18.356378+0000","last_peered":"2024-04-17T09:54:18.356378+0000","last_clean":"2024-04-17T09:54:18.356378+0000","last_became_active":"2024-04-17T09:54:17.380254+0000","last_became_peered":"2024-04-17T09:54:17.380254+0000","last_unstale":"2024-04-17T09:54:18.356378+0000","last_undegraded":"2024-04-17T09:54:18.356378+0000","last_fullsized":"2024-04-17T09:54:18.356378+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:08:56.864943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948938+0000","last_change":"2024-04-17T09:54:16.355434+0000","last_active":"2024-04-17T09:54:18.948938+0000","last_peered":"2024-04-17T09:54:18.948938+0000","last_clean":"2024-04-17T09:54:18.948938+0000","last_became_active":"2024-04-17T09:54:16.354665+0000","last_became_peered":"2024-04-17T09:54:16.354665+0000","last_unstale":"2024-04-17T09:54:18.948938+0000","last_undegraded":"2024-04-17T09:54:18.948938+0000","last_fullsized":"2024-04-17T09:54:18.948938+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:36:18.292446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948414+0000","last_change":"2024-04-17T09:54:17.377485+0000","last_active":"2024-04-17T09:54:18.948414+0000","last_peered":"2024-04-17T09:54:18.948414+0000","last_clean":"2024-04-17T09:54:18.948414+0000","last_became_active":"2024-04-17T09:54:17.377173+0000","last_became_peered":"2024-04-17T09:54:17.377173+0000","last_unstale":"2024-04-17T09:54:18.948414+0000","last_undegraded":"2024-04-17T09:54:18.948414+0000","last_fullsized":"2024-04-17T09:54:18.948414+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:25.615888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948835+0000","last_change":"2024-04-17T09:54:16.352471+0000","last_active":"2024-04-17T09:54:18.948835+0000","last_peered":"2024-04-17T09:54:18.948835+0000","last_clean":"2024-04-17T09:54:18.948835+0000","last_became_active":"2024-04-17T09:54:16.350287+0000","last_became_peered":"2024-04-17T09:54:16.350287+0000","last_unstale":"2024-04-17T09:54:18.948835+0000","last_undegraded":"2024-04-17T09:54:18.948835+0000","last_fullsized":"2024-04-17T09:54:18.948835+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:59:30.048582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948678+0000","last_change":"2024-04-17T09:54:17.364045+0000","last_active":"2024-04-17T09:54:18.948678+0000","last_peered":"2024-04-17T09:54:18.948678+0000","last_clean":"2024-04-17T09:54:18.948678+0000","last_became_active":"2024-04-17T09:54:17.363677+0000","last_became_peered":"2024-04-17T09:54:17.363677+0000","last_unstale":"2024-04-17T09:54:18.948678+0000","last_undegraded":"2024-04-17T09:54:18.948678+0000","last_fullsized":"2024-04-17T09:54:18.948678+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:38:58.355284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355761+0000","last_change":"2024-04-17T09:54:16.359267+0000","last_active":"2024-04-17T09:54:18.355761+0000","last_peered":"2024-04-17T09:54:18.355761+0000","last_clean":"2024-04-17T09:54:18.355761+0000","last_became_active":"2024-04-17T09:54:16.358963+0000","last_became_peered":"2024-04-17T09:54:16.358963+0000","last_unstale":"2024-04-17T09:54:18.355761+0000","last_undegraded":"2024-04-17T09:54:18.355761+0000","last_fullsized":"2024-04-17T09:54:18.355761+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:38:45.976992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395033+0000","last_change":"2024-04-17T09:54:17.376624+0000","last_active":"2024-04-17T09:54:18.395033+0000","last_peered":"2024-04-17T09:54:18.395033+0000","last_clean":"2024-04-17T09:54:18.395033+0000","last_became_active":"2024-04-17T09:54:17.376365+0000","last_became_peered":"2024-04-17T09:54:17.376365+0000","last_unstale":"2024-04-17T09:54:18.395033+0000","last_undegraded":"2024-04-17T09:54:18.395033+0000","last_fullsized":"2024-04-17T09:54:18.395033+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:29:49.267181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949227+0000","last_change":"2024-04-17T09:54:16.353761+0000","last_active":"2024-04-17T09:54:18.949227+0000","last_peered":"2024-04-17T09:54:18.949227+0000","last_clean":"2024-04-17T09:54:18.949227+0000","last_became_active":"2024-04-17T09:54:16.353426+0000","last_became_peered":"2024-04-17T09:54:16.353426+0000","last_unstale":"2024-04-17T09:54:18.949227+0000","last_undegraded":"2024-04-17T09:54:18.949227+0000","last_fullsized":"2024-04-17T09:54:18.949227+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:20:24.640480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948657+0000","last_change":"2024-04-17T09:54:17.373037+0000","last_active":"2024-04-17T09:54:18.948657+0000","last_peered":"2024-04-17T09:54:18.948657+0000","last_clean":"2024-04-17T09:54:18.948657+0000","last_became_active":"2024-04-17T09:54:17.372733+0000","last_became_peered":"2024-04-17T09:54:17.372733+0000","last_unstale":"2024-04-17T09:54:18.948657+0000","last_undegraded":"2024-04-17T09:54:18.948657+0000","last_fullsized":"2024-04-17T09:54:18.948657+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:44:35.259640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356319+0000","last_change":"2024-04-17T09:54:16.365789+0000","last_active":"2024-04-17T09:54:18.356319+0000","last_peered":"2024-04-17T09:54:18.356319+0000","last_clean":"2024-04-17T09:54:18.356319+0000","last_became_active":"2024-04-17T09:54:16.365521+0000","last_became_peered":"2024-04-17T09:54:16.365521+0000","last_unstale":"2024-04-17T09:54:18.356319+0000","last_undegraded":"2024-04-17T09:54:18.356319+0000","last_fullsized":"2024-04-17T09:54:18.356319+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:03:04.812259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356313+0000","last_change":"2024-04-17T09:54:17.387554+0000","last_active":"2024-04-17T09:54:18.356313+0000","last_peered":"2024-04-17T09:54:18.356313+0000","last_clean":"2024-04-17T09:54:18.356313+0000","last_became_active":"2024-04-17T09:54:17.387190+0000","last_became_peered":"2024-04-17T09:54:17.387190+0000","last_unstale":"2024-04-17T09:54:18.356313+0000","last_undegraded":"2024-04-17T09:54:18.356313+0000","last_fullsized":"2024-04-17T09:54:18.356313+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:56:37.564448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.392904+0000","last_change":"2024-04-17T09:54:16.369081+0000","last_active":"2024-04-17T09:54:18.392904+0000","last_peered":"2024-04-17T09:54:18.392904+0000","last_clean":"2024-04-17T09:54:18.392904+0000","last_became_active":"2024-04-17T09:54:16.368772+0000","last_became_peered":"2024-04-17T09:54:16.368772+0000","last_unstale":"2024-04-17T09:54:18.392904+0000","last_undegraded":"2024-04-17T09:54:18.392904+0000","last_fullsized":"2024-04-17T09:54:18.392904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:33:18.638703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356029+0000","last_change":"2024-04-17T09:54:17.381710+0000","last_active":"2024-04-17T09:54:18.356029+0000","last_peered":"2024-04-17T09:54:18.356029+0000","last_clean":"2024-04-17T09:54:18.356029+0000","last_became_active":"2024-04-17T09:54:17.381363+0000","last_became_peered":"2024-04-17T09:54:17.381363+0000","last_unstale":"2024-04-17T09:54:18.356029+0000","last_undegraded":"2024-04-17T09:54:18.356029+0000","last_fullsized":"2024-04-17T09:54:18.356029+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:32:50.828977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948633+0000","last_change":"2024-04-17T09:54:16.364352+0000","last_active":"2024-04-17T09:54:18.948633+0000","last_peered":"2024-04-17T09:54:18.948633+0000","last_clean":"2024-04-17T09:54:18.948633+0000","last_became_active":"2024-04-17T09:54:16.363940+0000","last_became_peered":"2024-04-17T09:54:16.363940+0000","last_unstale":"2024-04-17T09:54:18.948633+0000","last_undegraded":"2024-04-17T09:54:18.948633+0000","last_fullsized":"2024-04-17T09:54:18.948633+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:57.115283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948502+0000","last_change":"2024-04-17T09:54:17.386126+0000","last_active":"2024-04-17T09:54:18.948502+0000","last_peered":"2024-04-17T09:54:18.948502+0000","last_clean":"2024-04-17T09:54:18.948502+0000","last_became_active":"2024-04-17T09:54:17.384769+0000","last_became_peered":"2024-04-17T09:54:17.384769+0000","last_unstale":"2024-04-17T09:54:18.948502+0000","last_undegraded":"2024-04-17T09:54:18.948502+0000","last_fullsized":"2024-04-17T09:54:18.948502+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:46:43.928292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356202+0000","last_change":"2024-04-17T09:54:16.367115+0000","last_active":"2024-04-17T09:54:18.356202+0000","last_peered":"2024-04-17T09:54:18.356202+0000","last_clean":"2024-04-17T09:54:18.356202+0000","last_became_active":"2024-04-17T09:54:16.362616+0000","last_became_peered":"2024-04-17T09:54:16.362616+0000","last_unstale":"2024-04-17T09:54:18.356202+0000","last_undegraded":"2024-04-17T09:54:18.356202+0000","last_fullsized":"2024-04-17T09:54:18.356202+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:49:28.345253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395244+0000","last_change":"2024-04-17T09:54:17.377531+0000","last_active":"2024-04-17T09:54:18.395244+0000","last_peered":"2024-04-17T09:54:18.395244+0000","last_clean":"2024-04-17T09:54:18.395244+0000","last_became_active":"2024-04-17T09:54:17.377268+0000","last_became_peered":"2024-04-17T09:54:17.377268+0000","last_unstale":"2024-04-17T09:54:18.395244+0000","last_undegraded":"2024-04-17T09:54:18.395244+0000","last_fullsized":"2024-04-17T09:54:18.395244+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:05:57.651811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949061+0000","last_change":"2024-04-17T09:54:16.356729+0000","last_active":"2024-04-17T09:54:18.949061+0000","last_peered":"2024-04-17T09:54:18.949061+0000","last_clean":"2024-04-17T09:54:18.949061+0000","last_became_active":"2024-04-17T09:54:16.356458+0000","last_became_peered":"2024-04-17T09:54:16.356458+0000","last_unstale":"2024-04-17T09:54:18.949061+0000","last_undegraded":"2024-04-17T09:54:18.949061+0000","last_fullsized":"2024-04-17T09:54:18.949061+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:09:00.415290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356088+0000","last_change":"2024-04-17T09:54:17.381943+0000","last_active":"2024-04-17T09:54:18.356088+0000","last_peered":"2024-04-17T09:54:18.356088+0000","last_clean":"2024-04-17T09:54:18.356088+0000","last_became_active":"2024-04-17T09:54:17.381545+0000","last_became_peered":"2024-04-17T09:54:17.381545+0000","last_unstale":"2024-04-17T09:54:18.356088+0000","last_undegraded":"2024-04-17T09:54:18.356088+0000","last_fullsized":"2024-04-17T09:54:18.356088+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:42:58.110213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395221+0000","last_change":"2024-04-17T09:54:16.362555+0000","last_active":"2024-04-17T09:54:18.395221+0000","last_peered":"2024-04-17T09:54:18.395221+0000","last_clean":"2024-04-17T09:54:18.395221+0000","last_became_active":"2024-04-17T09:54:16.362271+0000","last_became_peered":"2024-04-17T09:54:16.362271+0000","last_unstale":"2024-04-17T09:54:18.395221+0000","last_undegraded":"2024-04-17T09:54:18.395221+0000","last_fullsized":"2024-04-17T09:54:18.395221+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:53:03.772098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356184+0000","last_change":"2024-04-17T09:54:17.376536+0000","last_active":"2024-04-17T09:54:18.356184+0000","last_peered":"2024-04-17T09:54:18.356184+0000","last_clean":"2024-04-17T09:54:18.356184+0000","last_became_active":"2024-04-17T09:54:17.375993+0000","last_became_peered":"2024-04-17T09:54:17.375993+0000","last_unstale":"2024-04-17T09:54:18.356184+0000","last_undegraded":"2024-04-17T09:54:18.356184+0000","last_fullsized":"2024-04-17T09:54:18.356184+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:54.371142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355616+0000","last_change":"2024-04-17T09:54:16.363275+0000","last_active":"2024-04-17T09:54:18.355616+0000","last_peered":"2024-04-17T09:54:18.355616+0000","last_clean":"2024-04-17T09:54:18.355616+0000","last_became_active":"2024-04-17T09:54:16.362917+0000","last_became_peered":"2024-04-17T09:54:16.362917+0000","last_unstale":"2024-04-17T09:54:18.355616+0000","last_undegraded":"2024-04-17T09:54:18.355616+0000","last_fullsized":"2024-04-17T09:54:18.355616+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:25:59.865466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356117+0000","last_change":"2024-04-17T09:54:17.361797+0000","last_active":"2024-04-17T09:54:18.356117+0000","last_peered":"2024-04-17T09:54:18.356117+0000","last_clean":"2024-04-17T09:54:18.356117+0000","last_became_active":"2024-04-17T09:54:17.361510+0000","last_became_peered":"2024-04-17T09:54:17.361510+0000","last_unstale":"2024-04-17T09:54:18.356117+0000","last_undegraded":"2024-04-17T09:54:18.356117+0000","last_fullsized":"2024-04-17T09:54:18.356117+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:24:33.163226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":22,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355686+0000","last_change":"2024-04-17T09:54:16.362386+0000","last_active":"2024-04-17T09:54:18.355686+0000","last_peered":"2024-04-17T09:54:18.355686+0000","last_clean":"2024-04-17T09:54:18.355686+0000","last_became_active":"2024-04-17T09:54:16.362098+0000","last_became_peered":"2024-04-17T09:54:16.362098+0000","last_unstale":"2024-04-17T09:54:18.355686+0000","last_undegraded":"2024-04-17T09:54:18.355686+0000","last_fullsized":"2024-04-17T09:54:18.355686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:24:32.158091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949418+0000","last_change":"2024-04-17T09:54:17.375249+0000","last_active":"2024-04-17T09:54:18.949418+0000","last_peered":"2024-04-17T09:54:18.949418+0000","last_clean":"2024-04-17T09:54:18.949418+0000","last_became_active":"2024-04-17T09:54:17.374928+0000","last_became_peered":"2024-04-17T09:54:17.374928+0000","last_unstale":"2024-04-17T09:54:18.949418+0000","last_undegraded":"2024-04-17T09:54:18.949418+0000","last_fullsized":"2024-04-17T09:54:18.949418+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:19:45.596601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395603+0000","last_change":"2024-04-17T09:54:16.368531+0000","last_active":"2024-04-17T09:54:18.395603+0000","last_peered":"2024-04-17T09:54:18.395603+0000","last_clean":"2024-04-17T09:54:18.395603+0000","last_became_active":"2024-04-17T09:54:16.366600+0000","last_became_peered":"2024-04-17T09:54:16.366600+0000","last_unstale":"2024-04-17T09:54:18.395603+0000","last_undegraded":"2024-04-17T09:54:18.395603+0000","last_fullsized":"2024-04-17T09:54:18.395603+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:49:08.687051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949000+0000","last_change":"2024-04-17T09:54:17.368031+0000","last_active":"2024-04-17T09:54:18.949000+0000","last_peered":"2024-04-17T09:54:18.949000+0000","last_clean":"2024-04-17T09:54:18.949000+0000","last_became_active":"2024-04-17T09:54:17.367652+0000","last_became_peered":"2024-04-17T09:54:17.367652+0000","last_unstale":"2024-04-17T09:54:18.949000+0000","last_undegraded":"2024-04-17T09:54:18.949000+0000","last_fullsized":"2024-04-17T09:54:18.949000+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:24:07.861301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949564+0000","last_change":"2024-04-17T09:54:16.355502+0000","last_active":"2024-04-17T09:54:18.949564+0000","last_peered":"2024-04-17T09:54:18.949564+0000","last_clean":"2024-04-17T09:54:18.949564+0000","last_became_active":"2024-04-17T09:54:16.354313+0000","last_became_peered":"2024-04-17T09:54:16.354313+0000","last_unstale":"2024-04-17T09:54:18.949564+0000","last_undegraded":"2024-04-17T09:54:18.949564+0000","last_fullsized":"2024-04-17T09:54:18.949564+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:49.443648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395309+0000","last_change":"2024-04-17T09:54:17.384520+0000","last_active":"2024-04-17T09:54:18.395309+0000","last_peered":"2024-04-17T09:54:18.395309+0000","last_clean":"2024-04-17T09:54:18.395309+0000","last_became_active":"2024-04-17T09:54:17.384220+0000","last_became_peered":"2024-04-17T09:54:17.384220+0000","last_unstale":"2024-04-17T09:54:18.395309+0000","last_undegraded":"2024-04-17T09:54:18.395309+0000","last_fullsized":"2024-04-17T09:54:18.395309+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:47:21.997789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949500+0000","last_change":"2024-04-17T09:54:16.359120+0000","last_active":"2024-04-17T09:54:18.949500+0000","last_peered":"2024-04-17T09:54:18.949500+0000","last_clean":"2024-04-17T09:54:18.949500+0000","last_became_active":"2024-04-17T09:54:16.358779+0000","last_became_peered":"2024-04-17T09:54:16.358779+0000","last_unstale":"2024-04-17T09:54:18.949500+0000","last_undegraded":"2024-04-17T09:54:18.949500+0000","last_fullsized":"2024-04-17T09:54:18.949500+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:15.893018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.448217+0000","last_change":"2024-04-17T09:54:17.369109+0000","last_active":"2024-04-17T09:54:18.448217+0000","last_peered":"2024-04-17T09:54:18.448217+0000","last_clean":"2024-04-17T09:54:18.448217+0000","last_became_active":"2024-04-17T09:54:17.368845+0000","last_became_peered":"2024-04-17T09:54:17.368845+0000","last_unstale":"2024-04-17T09:54:18.448217+0000","last_undegraded":"2024-04-17T09:54:18.448217+0000","last_fullsized":"2024-04-17T09:54:18.448217+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:00.870600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949456+0000","last_change":"2024-04-17T09:54:16.354365+0000","last_active":"2024-04-17T09:54:18.949456+0000","last_peered":"2024-04-17T09:54:18.949456+0000","last_clean":"2024-04-17T09:54:18.949456+0000","last_became_active":"2024-04-17T09:54:16.354093+0000","last_became_peered":"2024-04-17T09:54:16.354093+0000","last_unstale":"2024-04-17T09:54:18.949456+0000","last_undegraded":"2024-04-17T09:54:18.949456+0000","last_fullsized":"2024-04-17T09:54:18.949456+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:35:34.537059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.448058+0000","last_change":"2024-04-17T09:54:17.362613+0000","last_active":"2024-04-17T09:54:18.448058+0000","last_peered":"2024-04-17T09:54:18.448058+0000","last_clean":"2024-04-17T09:54:18.448058+0000","last_became_active":"2024-04-17T09:54:17.362336+0000","last_became_peered":"2024-04-17T09:54:17.362336+0000","last_unstale":"2024-04-17T09:54:18.448058+0000","last_undegraded":"2024-04-17T09:54:18.448058+0000","last_fullsized":"2024-04-17T09:54:18.448058+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:33:09.050240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.392664+0000","last_change":"2024-04-17T09:54:16.363705+0000","last_active":"2024-04-17T09:54:18.392664+0000","last_peered":"2024-04-17T09:54:18.392664+0000","last_clean":"2024-04-17T09:54:18.392664+0000","last_became_active":"2024-04-17T09:54:16.363405+0000","last_became_peered":"2024-04-17T09:54:16.363405+0000","last_unstale":"2024-04-17T09:54:18.392664+0000","last_undegraded":"2024-04-17T09:54:18.392664+0000","last_fullsized":"2024-04-17T09:54:18.392664+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:38:27.804441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.448273+0000","last_change":"2024-04-17T09:54:17.379977+0000","last_active":"2024-04-17T09:54:18.448273+0000","last_peered":"2024-04-17T09:54:18.448273+0000","last_clean":"2024-04-17T09:54:18.448273+0000","last_became_active":"2024-04-17T09:54:17.379723+0000","last_became_peered":"2024-04-17T09:54:17.379723+0000","last_unstale":"2024-04-17T09:54:18.448273+0000","last_undegraded":"2024-04-17T09:54:18.448273+0000","last_fullsized":"2024-04-17T09:54:18.448273+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:49:29.350387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355901+0000","last_change":"2024-04-17T09:54:16.361330+0000","last_active":"2024-04-17T09:54:18.355901+0000","last_peered":"2024-04-17T09:54:18.355901+0000","last_clean":"2024-04-17T09:54:18.355901+0000","last_became_active":"2024-04-17T09:54:16.361056+0000","last_became_peered":"2024-04-17T09:54:16.361056+0000","last_unstale":"2024-04-17T09:54:18.355901+0000","last_undegraded":"2024-04-17T09:54:18.355901+0000","last_fullsized":"2024-04-17T09:54:18.355901+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:39:30.974585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.447914+0000","last_change":"2024-04-17T09:54:17.366185+0000","last_active":"2024-04-17T09:54:18.447914+0000","last_peered":"2024-04-17T09:54:18.447914+0000","last_clean":"2024-04-17T09:54:18.447914+0000","last_became_active":"2024-04-17T09:54:17.365837+0000","last_became_peered":"2024-04-17T09:54:17.365837+0000","last_unstale":"2024-04-17T09:54:18.447914+0000","last_undegraded":"2024-04-17T09:54:18.447914+0000","last_fullsized":"2024-04-17T09:54:18.447914+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:49:09.692185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949026+0000","last_change":"2024-04-17T09:54:16.365581+0000","last_active":"2024-04-17T09:54:18.949026+0000","last_peered":"2024-04-17T09:54:18.949026+0000","last_clean":"2024-04-17T09:54:18.949026+0000","last_became_active":"2024-04-17T09:54:16.365308+0000","last_became_peered":"2024-04-17T09:54:16.365308+0000","last_unstale":"2024-04-17T09:54:18.949026+0000","last_undegraded":"2024-04-17T09:54:18.949026+0000","last_fullsized":"2024-04-17T09:54:18.949026+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:55.683434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356301+0000","last_change":"2024-04-17T09:54:17.378209+0000","last_active":"2024-04-17T09:54:18.356301+0000","last_peered":"2024-04-17T09:54:18.356301+0000","last_clean":"2024-04-17T09:54:18.356301+0000","last_became_active":"2024-04-17T09:54:17.377727+0000","last_became_peered":"2024-04-17T09:54:17.377727+0000","last_unstale":"2024-04-17T09:54:18.356301+0000","last_undegraded":"2024-04-17T09:54:18.356301+0000","last_fullsized":"2024-04-17T09:54:18.356301+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:24:07.861301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948281+0000","last_change":"2024-04-17T09:54:16.362646+0000","last_active":"2024-04-17T09:54:18.948281+0000","last_peered":"2024-04-17T09:54:18.948281+0000","last_clean":"2024-04-17T09:54:18.948281+0000","last_became_active":"2024-04-17T09:54:16.362306+0000","last_became_peered":"2024-04-17T09:54:16.362306+0000","last_unstale":"2024-04-17T09:54:18.948281+0000","last_undegraded":"2024-04-17T09:54:18.948281+0000","last_fullsized":"2024-04-17T09:54:18.948281+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:38:45.976992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.448239+0000","last_change":"2024-04-17T09:54:17.376712+0000","last_active":"2024-04-17T09:54:18.448239+0000","last_peered":"2024-04-17T09:54:18.448239+0000","last_clean":"2024-04-17T09:54:18.448239+0000","last_became_active":"2024-04-17T09:54:17.376382+0000","last_became_peered":"2024-04-17T09:54:17.376382+0000","last_unstale":"2024-04-17T09:54:18.448239+0000","last_undegraded":"2024-04-17T09:54:18.448239+0000","last_fullsized":"2024-04-17T09:54:18.448239+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:34:22.500707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395654+0000","last_change":"2024-04-17T09:54:16.366766+0000","last_active":"2024-04-17T09:54:18.395654+0000","last_peered":"2024-04-17T09:54:18.395654+0000","last_clean":"2024-04-17T09:54:18.395654+0000","last_became_active":"2024-04-17T09:54:16.366351+0000","last_became_peered":"2024-04-17T09:54:16.366351+0000","last_unstale":"2024-04-17T09:54:18.395654+0000","last_undegraded":"2024-04-17T09:54:18.395654+0000","last_fullsized":"2024-04-17T09:54:18.395654+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:33:08.045106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949142+0000","last_change":"2024-04-17T09:54:17.385290+0000","last_active":"2024-04-17T09:54:18.949142+0000","last_peered":"2024-04-17T09:54:18.949142+0000","last_clean":"2024-04-17T09:54:18.949142+0000","last_became_active":"2024-04-17T09:54:17.384972+0000","last_became_peered":"2024-04-17T09:54:17.384972+0000","last_unstale":"2024-04-17T09:54:18.949142+0000","last_undegraded":"2024-04-17T09:54:18.949142+0000","last_fullsized":"2024-04-17T09:54:18.949142+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:21.920667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.394428+0000","last_change":"2024-04-17T09:54:16.363895+0000","last_active":"2024-04-17T09:54:18.394428+0000","last_peered":"2024-04-17T09:54:18.394428+0000","last_clean":"2024-04-17T09:54:18.394428+0000","last_became_active":"2024-04-17T09:54:16.363461+0000","last_became_peered":"2024-04-17T09:54:16.363461+0000","last_unstale":"2024-04-17T09:54:18.394428+0000","last_undegraded":"2024-04-17T09:54:18.394428+0000","last_fullsized":"2024-04-17T09:54:18.394428+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:09:51.673285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.392687+0000","last_change":"2024-04-17T09:54:17.390013+0000","last_active":"2024-04-17T09:54:18.392687+0000","last_peered":"2024-04-17T09:54:18.392687+0000","last_clean":"2024-04-17T09:54:18.392687+0000","last_became_active":"2024-04-17T09:54:17.389724+0000","last_became_peered":"2024-04-17T09:54:17.389724+0000","last_unstale":"2024-04-17T09:54:18.392687+0000","last_undegraded":"2024-04-17T09:54:18.392687+0000","last_fullsized":"2024-04-17T09:54:18.392687+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:10.103965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395460+0000","last_change":"2024-04-17T09:54:16.364397+0000","last_active":"2024-04-17T09:54:18.395460+0000","last_peered":"2024-04-17T09:54:18.395460+0000","last_clean":"2024-04-17T09:54:18.395460+0000","last_became_active":"2024-04-17T09:54:16.364110+0000","last_became_peered":"2024-04-17T09:54:16.364110+0000","last_unstale":"2024-04-17T09:54:18.395460+0000","last_undegraded":"2024-04-17T09:54:18.395460+0000","last_fullsized":"2024-04-17T09:54:18.395460+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:12.816388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355784+0000","last_change":"2024-04-17T09:54:17.371227+0000","last_active":"2024-04-17T09:54:18.355784+0000","last_peered":"2024-04-17T09:54:18.355784+0000","last_clean":"2024-04-17T09:54:18.355784+0000","last_became_active":"2024-04-17T09:54:17.370965+0000","last_became_peered":"2024-04-17T09:54:17.370965+0000","last_unstale":"2024-04-17T09:54:18.355784+0000","last_undegraded":"2024-04-17T09:54:18.355784+0000","last_fullsized":"2024-04-17T09:54:18.355784+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:00.870600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.392758+0000","last_change":"2024-04-17T09:54:16.361087+0000","last_active":"2024-04-17T09:54:18.392758+0000","last_peered":"2024-04-17T09:54:18.392758+0000","last_clean":"2024-04-17T09:54:18.392758+0000","last_became_active":"2024-04-17T09:54:16.360811+0000","last_became_peered":"2024-04-17T09:54:16.360811+0000","last_unstale":"2024-04-17T09:54:18.392758+0000","last_undegraded":"2024-04-17T09:54:18.392758+0000","last_fullsized":"2024-04-17T09:54:18.392758+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:15:00.730444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.392642+0000","last_change":"2024-04-17T09:54:17.371753+0000","last_active":"2024-04-17T09:54:18.392642+0000","last_peered":"2024-04-17T09:54:18.392642+0000","last_clean":"2024-04-17T09:54:18.392642+0000","last_became_active":"2024-04-17T09:54:17.371448+0000","last_became_peered":"2024-04-17T09:54:17.371448+0000","last_unstale":"2024-04-17T09:54:18.392642+0000","last_undegraded":"2024-04-17T09:54:18.392642+0000","last_fullsized":"2024-04-17T09:54:18.392642+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:07.206960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395312+0000","last_change":"2024-04-17T09:54:16.367447+0000","last_active":"2024-04-17T09:54:18.395312+0000","last_peered":"2024-04-17T09:54:18.395312+0000","last_clean":"2024-04-17T09:54:18.395312+0000","last_became_active":"2024-04-17T09:54:16.367196+0000","last_became_peered":"2024-04-17T09:54:16.367196+0000","last_unstale":"2024-04-17T09:54:18.395312+0000","last_undegraded":"2024-04-17T09:54:18.395312+0000","last_fullsized":"2024-04-17T09:54:18.395312+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:16:06.147936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355576+0000","last_change":"2024-04-17T09:54:17.387333+0000","last_active":"2024-04-17T09:54:18.355576+0000","last_peered":"2024-04-17T09:54:18.355576+0000","last_clean":"2024-04-17T09:54:18.355576+0000","last_became_active":"2024-04-17T09:54:17.382030+0000","last_became_peered":"2024-04-17T09:54:17.382030+0000","last_unstale":"2024-04-17T09:54:18.355576+0000","last_undegraded":"2024-04-17T09:54:18.355576+0000","last_fullsized":"2024-04-17T09:54:18.355576+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:54.371142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949136+0000","last_change":"2024-04-17T09:54:16.361810+0000","last_active":"2024-04-17T09:54:18.949136+0000","last_peered":"2024-04-17T09:54:18.949136+0000","last_clean":"2024-04-17T09:54:18.949136+0000","last_became_active":"2024-04-17T09:54:16.361521+0000","last_became_peered":"2024-04-17T09:54:16.361521+0000","last_unstale":"2024-04-17T09:54:18.949136+0000","last_undegraded":"2024-04-17T09:54:18.949136+0000","last_fullsized":"2024-04-17T09:54:18.949136+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:44.012560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949338+0000","last_change":"2024-04-17T09:54:17.368893+0000","last_active":"2024-04-17T09:54:18.949338+0000","last_peered":"2024-04-17T09:54:18.949338+0000","last_clean":"2024-04-17T09:54:18.949338+0000","last_became_active":"2024-04-17T09:54:17.368627+0000","last_became_peered":"2024-04-17T09:54:17.368627+0000","last_unstale":"2024-04-17T09:54:18.949338+0000","last_undegraded":"2024-04-17T09:54:18.949338+0000","last_fullsized":"2024-04-17T09:54:18.949338+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:22:28.141447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355785+0000","last_change":"2024-04-17T09:54:16.365957+0000","last_active":"2024-04-17T09:54:18.355785+0000","last_peered":"2024-04-17T09:54:18.355785+0000","last_clean":"2024-04-17T09:54:18.355785+0000","last_became_active":"2024-04-17T09:54:16.365667+0000","last_became_peered":"2024-04-17T09:54:16.365667+0000","last_unstale":"2024-04-17T09:54:18.355785+0000","last_undegraded":"2024-04-17T09:54:18.355785+0000","last_fullsized":"2024-04-17T09:54:18.355785+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:07.205760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356429+0000","last_change":"2024-04-17T09:54:17.384255+0000","last_active":"2024-04-17T09:54:18.356429+0000","last_peered":"2024-04-17T09:54:18.356429+0000","last_clean":"2024-04-17T09:54:18.356429+0000","last_became_active":"2024-04-17T09:54:17.383920+0000","last_became_peered":"2024-04-17T09:54:17.383920+0000","last_unstale":"2024-04-17T09:54:18.356429+0000","last_undegraded":"2024-04-17T09:54:18.356429+0000","last_fullsized":"2024-04-17T09:54:18.356429+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:21.920667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355720+0000","last_change":"2024-04-17T09:54:16.354634+0000","last_active":"2024-04-17T09:54:18.355720+0000","last_peered":"2024-04-17T09:54:18.355720+0000","last_clean":"2024-04-17T09:54:18.355720+0000","last_became_active":"2024-04-17T09:54:16.354293+0000","last_became_peered":"2024-04-17T09:54:16.354293+0000","last_unstale":"2024-04-17T09:54:18.355720+0000","last_undegraded":"2024-04-17T09:54:18.355720+0000","last_fullsized":"2024-04-17T09:54:18.355720+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:28:16.576076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":196,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356427+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T09:54:18.356427+0000","last_peered":"2024-04-17T09:54:18.356427+0000","last_clean":"2024-04-17T09:54:18.356427+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T09:54:18.356427+0000","last_undegraded":"2024-04-17T09:54:18.356427+0000","last_fullsized":"2024-04-17T09:54:18.356427+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":987136,"data_stored":743634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":397312,"data_stored":153792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509459,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":397312,"data_stored":153848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509459,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":405504,"data_stored":154810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":393216,"data_stored":153246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":405504,"data_stored":154810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":397312,"data_stored":153280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":983040,"data_stored":743544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T09:54:21.490 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T09:54:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.673+0000 7f9ab3e00640 1 Processor -- start 2024-04-17T09:54:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.673+0000 7f9ab3e00640 1 -- start start 2024-04-17T09:54:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.673+0000 7f9ab3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9aac15baa0 0x7f9aac15bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.673+0000 7f9ab3e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aac072bb0 con 0x7f9aac144e90 2024-04-17T09:54:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.673+0000 7f9ab3e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aac0b3dc0 con 0x7f9aac06ef60 2024-04-17T09:54:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.673+0000 7f9ab3e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9aac145270 con 0x7f9aac15baa0 2024-04-17T09:54:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.673+0000 7f9ab0a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9aac15baa0 0x7f9aac15bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.673+0000 7f9ab1400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9aac144e90 0x7f9aac0b48a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37102/0 (socket says 172.21.15.32:37102) 2024-04-17T09:54:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.673+0000 7f9ab1400640 1 -- 172.21.15.32:0/2917367609 learned_addr learned my addr 172.21.15.32:0/2917367609 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.673+0000 7f9ab0a00640 1 -- 172.21.15.32:0/2917367609 >> v1:172.21.15.144:6789/0 conn(0x7f9aac06ef60 legacy=0x7f9aac071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.673+0000 7f9ab0a00640 1 -- 172.21.15.32:0/2917367609 >> v1:172.21.15.32:6789/0 conn(0x7f9aac144e90 legacy=0x7f9aac0b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.673+0000 7f9ab0a00640 1 -- 172.21.15.32:0/2917367609 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aac0b40a0 con 0x7f9aac15baa0 2024-04-17T09:54:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.677+0000 7f9ab0a00640 1 --2- 172.21.15.32:0/2917367609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9aac15baa0 0x7f9aac15bf30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9a980097c0 tx=0x7f9a9801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce25630ab8a69a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.677+0000 7f9aabe00640 1 -- 172.21.15.32:0/2917367609 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a980b0070 con 0x7f9aac15baa0 2024-04-17T09:54:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.677+0000 7f9aabe00640 1 -- 172.21.15.32:0/2917367609 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a980ab600 con 0x7f9aac15baa0 2024-04-17T09:54:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.677+0000 7f9aabe00640 1 -- 172.21.15.32:0/2917367609 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a980ab9d0 con 0x7f9aac15baa0 2024-04-17T09:54:21.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.677+0000 7f9ab3e00640 1 -- 172.21.15.32:0/2917367609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9aac15baa0 msgr2=0x7f9aac15bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:21.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.677+0000 7f9ab3e00640 1 --2- 172.21.15.32:0/2917367609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9aac15baa0 0x7f9aac15bf30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9a980097c0 tx=0x7f9a9801c460 comp rx=0 tx=0).stop 2024-04-17T09:54:21.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.677+0000 7f9ab3e00640 1 -- 172.21.15.32:0/2917367609 shutdown_connections 2024-04-17T09:54:21.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.677+0000 7f9ab3e00640 1 --2- 172.21.15.32:0/2917367609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9aac15baa0 0x7f9aac15bf30 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:21.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.677+0000 7f9ab3e00640 1 -- 172.21.15.32:0/2917367609 >> 172.21.15.32:0/2917367609 conn(0x7f9aac06ba30 msgr2=0x7f9aac06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:21.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.677+0000 7f9ab3e00640 1 -- 172.21.15.32:0/2917367609 shutdown_connections 2024-04-17T09:54:21.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.677+0000 7f9ab3e00640 1 -- 172.21.15.32:0/2917367609 wait complete. 2024-04-17T09:54:21.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab3e00640 1 Processor -- start 2024-04-17T09:54:21.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab3e00640 1 -- start start 2024-04-17T09:54:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab3e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9aac06ef60 0x7f9aac15b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9aac144e90 0x7f9aac156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9aac15baa0 0x7f9aac139370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab3e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac06dfd0 con 0x7f9aac15baa0 2024-04-17T09:54:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab3e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac06de50 con 0x7f9aac144e90 2024-04-17T09:54:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab3e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9aac06e150 con 0x7f9aac06ef60 2024-04-17T09:54:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9aac15baa0 0x7f9aac139370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab1400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9aac06ef60 0x7f9aac15b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab0a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9aac144e90 0x7f9aac156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab0a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9aac144e90 0x7f9aac156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:54936/0 (socket says 172.21.15.32:54936) 2024-04-17T09:54:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab0a00640 1 -- 172.21.15.32:0/3968850807 learned_addr learned my addr 172.21.15.32:0/3968850807 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab1e00640 1 -- 172.21.15.32:0/3968850807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9aac06ef60 msgr2=0x7f9aac15b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab1e00640 1 --2- 172.21.15.32:0/3968850807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9aac06ef60 0x7f9aac15b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab1e00640 1 -- 172.21.15.32:0/3968850807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9aac144e90 msgr2=0x7f9aac156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab1e00640 1 --2- 172.21.15.32:0/3968850807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9aac144e90 0x7f9aac156f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:21.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab1e00640 1 -- 172.21.15.32:0/3968850807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aac138740 con 0x7f9aac15baa0 2024-04-17T09:54:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab1400640 1 --2- 172.21.15.32:0/3968850807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9aac06ef60 0x7f9aac15b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:54:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab0a00640 1 --2- 172.21.15.32:0/3968850807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9aac144e90 0x7f9aac156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:54:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab1e00640 1 --2- 172.21.15.32:0/3968850807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9aac15baa0 0x7f9aac139370 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9aa400c780 tx=0x7f9aa400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9aa9600640 1 -- 172.21.15.32:0/3968850807 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4016020 con 0x7f9aac15baa0 2024-04-17T09:54:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9aa9600640 1 -- 172.21.15.32:0/3968850807 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa400aa50 con 0x7f9aac15baa0 2024-04-17T09:54:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9aa9600640 1 -- 172.21.15.32:0/3968850807 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa400ce30 con 0x7f9aac15baa0 2024-04-17T09:54:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab3e00640 1 -- 172.21.15.32:0/3968850807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aac15a060 con 0x7f9aac15baa0 2024-04-17T09:54:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab3e00640 1 -- 172.21.15.32:0/3968850807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aac15a620 con 0x7f9aac15baa0 2024-04-17T09:54:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.681+0000 7f9ab3e00640 1 -- 172.21.15.32:0/3968850807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aac068a90 con 0x7f9aac15baa0 2024-04-17T09:54:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.685+0000 7f9aa9600640 1 -- 172.21.15.32:0/3968850807 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9aa4019020 con 0x7f9aac15baa0 2024-04-17T09:54:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.685+0000 7f9aa9600640 1 --2- 172.21.15.32:0/3968850807 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9a8c068380 0x7f9a8c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.685+0000 7f9aa9600640 1 -- 172.21.15.32:0/3968850807 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f9aa4086420 con 0x7f9aac15baa0 2024-04-17T09:54:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.685+0000 7f9ab1400640 1 --2- 172.21.15.32:0/3968850807 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9a8c068380 0x7f9a8c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.685+0000 7f9ab1400640 1 --2- 172.21.15.32:0/3968850807 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9a8c068380 0x7f9a8c06a840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9aac1f6fc0 tx=0x7f9aa0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:21.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.689+0000 7f9aa9600640 1 -- 172.21.15.32:0/3968850807 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f9aa4020070 con 0x7f9aac15baa0 2024-04-17T09:54:21.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.849+0000 7f9ab3e00640 1 -- 172.21.15.32:0/3968850807 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9aac0b0070 con 0x7f9a8c068380 2024-04-17T09:54:21.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.857+0000 7f9aa9600640 1 -- 172.21.15.32:0/3968850807 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251950 (secure 0 0 0) 0x7f9aac0b0070 con 0x7f9a8c068380 2024-04-17T09:54:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T09:54:21.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.861+0000 7f9ab3e00640 1 -- 172.21.15.32:0/3968850807 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9a8c068380 msgr2=0x7f9a8c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:21.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.861+0000 7f9ab3e00640 1 --2- 172.21.15.32:0/3968850807 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9a8c068380 0x7f9a8c06a840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9aac1f6fc0 tx=0x7f9aa0009040 comp rx=0 tx=0).stop 2024-04-17T09:54:21.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.861+0000 7f9ab3e00640 1 -- 172.21.15.32:0/3968850807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9aac15baa0 msgr2=0x7f9aac139370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:21.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.861+0000 7f9ab3e00640 1 --2- 172.21.15.32:0/3968850807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9aac15baa0 0x7f9aac139370 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9aa400c780 tx=0x7f9aa400cc40 comp rx=0 tx=0).stop 2024-04-17T09:54:21.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.861+0000 7f9ab3e00640 1 -- 172.21.15.32:0/3968850807 shutdown_connections 2024-04-17T09:54:21.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.861+0000 7f9ab3e00640 1 --2- 172.21.15.32:0/3968850807 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9a8c068380 0x7f9a8c06a840 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:21.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.861+0000 7f9ab3e00640 1 --2- 172.21.15.32:0/3968850807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9aac15baa0 0x7f9aac139370 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:21.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.861+0000 7f9ab3e00640 1 --2- 172.21.15.32:0/3968850807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9aac144e90 0x7f9aac156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:21.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.861+0000 7f9ab3e00640 1 --2- 172.21.15.32:0/3968850807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9aac06ef60 0x7f9aac15b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:21.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.861+0000 7f9ab3e00640 1 -- 172.21.15.32:0/3968850807 >> 172.21.15.32:0/3968850807 conn(0x7f9aac06ba30 msgr2=0x7f9aac0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:21.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.861+0000 7f9ab3e00640 1 -- 172.21.15.32:0/3968850807 shutdown_connections 2024-04-17T09:54:21.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:21.861+0000 7f9ab3e00640 1 -- 172.21.15.32:0/3968850807 wait complete. 2024-04-17T09:54:21.881 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-04-17T09:54:21.298196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241244,"kb_used_data":4264,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733476,"statfs":{"total":773094113280,"available":772847079424,"internally_reserved":0,"allocated":4366336,"data_stored":2410964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.961770"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355927+0000","last_change":"2024-04-17T09:54:17.376160+0000","last_active":"2024-04-17T09:54:18.355927+0000","last_peered":"2024-04-17T09:54:18.355927+0000","last_clean":"2024-04-17T09:54:18.355927+0000","last_became_active":"2024-04-17T09:54:17.374386+0000","last_became_peered":"2024-04-17T09:54:17.374386+0000","last_unstale":"2024-04-17T09:54:18.355927+0000","last_undegraded":"2024-04-17T09:54:18.355927+0000","last_fullsized":"2024-04-17T09:54:18.355927+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:34:22.500707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356387+0000","last_change":"2024-04-17T09:54:17.370783+0000","last_active":"2024-04-17T09:54:18.356387+0000","last_peered":"2024-04-17T09:54:18.356387+0000","last_clean":"2024-04-17T09:54:18.356387+0000","last_became_active":"2024-04-17T09:54:17.370388+0000","last_became_peered":"2024-04-17T09:54:17.370388+0000","last_unstale":"2024-04-17T09:54:18.356387+0000","last_undegraded":"2024-04-17T09:54:18.356387+0000","last_fullsized":"2024-04-17T09:54:18.356387+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:31:39.270271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948487+0000","last_change":"2024-04-17T09:54:17.383521+0000","last_active":"2024-04-17T09:54:18.948487+0000","last_peered":"2024-04-17T09:54:18.948487+0000","last_clean":"2024-04-17T09:54:18.948487+0000","last_became_active":"2024-04-17T09:54:17.383239+0000","last_became_peered":"2024-04-17T09:54:17.383239+0000","last_unstale":"2024-04-17T09:54:18.948487+0000","last_undegraded":"2024-04-17T09:54:18.948487+0000","last_fullsized":"2024-04-17T09:54:18.948487+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:04:35.939029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395162+0000","last_change":"2024-04-17T09:54:17.384642+0000","last_active":"2024-04-17T09:54:18.395162+0000","last_peered":"2024-04-17T09:54:18.395162+0000","last_clean":"2024-04-17T09:54:18.395162+0000","last_became_active":"2024-04-17T09:54:17.384356+0000","last_became_peered":"2024-04-17T09:54:17.384356+0000","last_unstale":"2024-04-17T09:54:18.395162+0000","last_undegraded":"2024-04-17T09:54:18.395162+0000","last_fullsized":"2024-04-17T09:54:18.395162+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:08.210894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.448200+0000","last_change":"2024-04-17T09:54:17.382868+0000","last_active":"2024-04-17T09:54:18.448200+0000","last_peered":"2024-04-17T09:54:18.448200+0000","last_clean":"2024-04-17T09:54:18.448200+0000","last_became_active":"2024-04-17T09:54:17.382623+0000","last_became_peered":"2024-04-17T09:54:17.382623+0000","last_unstale":"2024-04-17T09:54:18.448200+0000","last_undegraded":"2024-04-17T09:54:18.448200+0000","last_fullsized":"2024-04-17T09:54:18.448200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:38:46.982126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355647+0000","last_change":"2024-04-17T09:54:17.375486+0000","last_active":"2024-04-17T09:54:18.355647+0000","last_peered":"2024-04-17T09:54:18.355647+0000","last_clean":"2024-04-17T09:54:18.355647+0000","last_became_active":"2024-04-17T09:54:17.374997+0000","last_became_peered":"2024-04-17T09:54:17.374997+0000","last_unstale":"2024-04-17T09:54:18.355647+0000","last_undegraded":"2024-04-17T09:54:18.355647+0000","last_fullsized":"2024-04-17T09:54:18.355647+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:24:07.861301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356830+0000","last_change":"2024-04-17T09:54:17.378457+0000","last_active":"2024-04-17T09:54:18.356830+0000","last_peered":"2024-04-17T09:54:18.356830+0000","last_clean":"2024-04-17T09:54:18.356830+0000","last_became_active":"2024-04-17T09:54:17.377938+0000","last_became_peered":"2024-04-17T09:54:17.377938+0000","last_unstale":"2024-04-17T09:54:18.356830+0000","last_undegraded":"2024-04-17T09:54:18.356830+0000","last_fullsized":"2024-04-17T09:54:18.356830+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:31:39.270271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356002+0000","last_change":"2024-04-17T09:54:17.381751+0000","last_active":"2024-04-17T09:54:18.356002+0000","last_peered":"2024-04-17T09:54:18.356002+0000","last_clean":"2024-04-17T09:54:18.356002+0000","last_became_active":"2024-04-17T09:54:17.381321+0000","last_became_peered":"2024-04-17T09:54:17.381321+0000","last_unstale":"2024-04-17T09:54:18.356002+0000","last_undegraded":"2024-04-17T09:54:18.356002+0000","last_fullsized":"2024-04-17T09:54:18.356002+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:04:35.939029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949486+0000","last_change":"2024-04-17T09:54:17.378952+0000","last_active":"2024-04-17T09:54:18.949486+0000","last_peered":"2024-04-17T09:54:18.949486+0000","last_clean":"2024-04-17T09:54:18.949486+0000","last_became_active":"2024-04-17T09:54:17.378641+0000","last_became_peered":"2024-04-17T09:54:17.378641+0000","last_unstale":"2024-04-17T09:54:18.949486+0000","last_undegraded":"2024-04-17T09:54:18.949486+0000","last_fullsized":"2024-04-17T09:54:18.949486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:11:30.035005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395470+0000","last_change":"2024-04-17T09:54:17.384874+0000","last_active":"2024-04-17T09:54:18.395470+0000","last_peered":"2024-04-17T09:54:18.395470+0000","last_clean":"2024-04-17T09:54:18.395470+0000","last_became_active":"2024-04-17T09:54:17.384562+0000","last_became_peered":"2024-04-17T09:54:17.384562+0000","last_unstale":"2024-04-17T09:54:18.395470+0000","last_undegraded":"2024-04-17T09:54:18.395470+0000","last_fullsized":"2024-04-17T09:54:18.395470+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:10:54.779389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356548+0000","last_change":"2024-04-17T09:54:17.381734+0000","last_active":"2024-04-17T09:54:18.356548+0000","last_peered":"2024-04-17T09:54:18.356548+0000","last_clean":"2024-04-17T09:54:18.356548+0000","last_became_active":"2024-04-17T09:54:17.381425+0000","last_became_peered":"2024-04-17T09:54:17.381425+0000","last_unstale":"2024-04-17T09:54:18.356548+0000","last_undegraded":"2024-04-17T09:54:18.356548+0000","last_fullsized":"2024-04-17T09:54:18.356548+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:42:58.110213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948912+0000","last_change":"2024-04-17T09:54:17.383440+0000","last_active":"2024-04-17T09:54:18.948912+0000","last_peered":"2024-04-17T09:54:18.948912+0000","last_clean":"2024-04-17T09:54:18.948912+0000","last_became_active":"2024-04-17T09:54:17.383101+0000","last_became_peered":"2024-04-17T09:54:17.383101+0000","last_unstale":"2024-04-17T09:54:18.948912+0000","last_undegraded":"2024-04-17T09:54:18.948912+0000","last_fullsized":"2024-04-17T09:54:18.948912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:54.371142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.448087+0000","last_change":"2024-04-17T09:54:17.379410+0000","last_active":"2024-04-17T09:54:18.448087+0000","last_peered":"2024-04-17T09:54:18.448087+0000","last_clean":"2024-04-17T09:54:18.448087+0000","last_became_active":"2024-04-17T09:54:17.379141+0000","last_became_peered":"2024-04-17T09:54:17.379141+0000","last_unstale":"2024-04-17T09:54:18.448087+0000","last_undegraded":"2024-04-17T09:54:18.448087+0000","last_fullsized":"2024-04-17T09:54:18.448087+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:02:43.944515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355673+0000","last_change":"2024-04-17T09:54:17.367214+0000","last_active":"2024-04-17T09:54:18.355673+0000","last_peered":"2024-04-17T09:54:18.355673+0000","last_clean":"2024-04-17T09:54:18.355673+0000","last_became_active":"2024-04-17T09:54:17.366900+0000","last_became_peered":"2024-04-17T09:54:17.366900+0000","last_unstale":"2024-04-17T09:54:18.355673+0000","last_undegraded":"2024-04-17T09:54:18.355673+0000","last_fullsized":"2024-04-17T09:54:18.355673+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:08:56.864943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948391+0000","last_change":"2024-04-17T09:54:17.365335+0000","last_active":"2024-04-17T09:54:18.948391+0000","last_peered":"2024-04-17T09:54:18.948391+0000","last_clean":"2024-04-17T09:54:18.948391+0000","last_became_active":"2024-04-17T09:54:17.363002+0000","last_became_peered":"2024-04-17T09:54:17.363002+0000","last_unstale":"2024-04-17T09:54:18.948391+0000","last_undegraded":"2024-04-17T09:54:18.948391+0000","last_fullsized":"2024-04-17T09:54:18.948391+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:26:01.393314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356455+0000","last_change":"2024-04-17T09:54:17.386566+0000","last_active":"2024-04-17T09:54:18.356455+0000","last_peered":"2024-04-17T09:54:18.356455+0000","last_clean":"2024-04-17T09:54:18.356455+0000","last_became_active":"2024-04-17T09:54:17.386176+0000","last_became_peered":"2024-04-17T09:54:17.386176+0000","last_unstale":"2024-04-17T09:54:18.356455+0000","last_undegraded":"2024-04-17T09:54:18.356455+0000","last_fullsized":"2024-04-17T09:54:18.356455+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:32:50.828977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356728+0000","last_change":"2024-04-17T09:54:17.382008+0000","last_active":"2024-04-17T09:54:18.356728+0000","last_peered":"2024-04-17T09:54:18.356728+0000","last_clean":"2024-04-17T09:54:18.356728+0000","last_became_active":"2024-04-17T09:54:17.381686+0000","last_became_peered":"2024-04-17T09:54:17.381686+0000","last_unstale":"2024-04-17T09:54:18.356728+0000","last_undegraded":"2024-04-17T09:54:18.356728+0000","last_fullsized":"2024-04-17T09:54:18.356728+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:56:37.564448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356415+0000","last_change":"2024-04-17T09:54:17.384926+0000","last_active":"2024-04-17T09:54:18.356415+0000","last_peered":"2024-04-17T09:54:18.356415+0000","last_clean":"2024-04-17T09:54:18.356415+0000","last_became_active":"2024-04-17T09:54:17.381926+0000","last_became_peered":"2024-04-17T09:54:17.381926+0000","last_unstale":"2024-04-17T09:54:18.356415+0000","last_undegraded":"2024-04-17T09:54:18.356415+0000","last_fullsized":"2024-04-17T09:54:18.356415+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:04:48.927345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355942+0000","last_change":"2024-04-17T09:54:17.387372+0000","last_active":"2024-04-17T09:54:18.355942+0000","last_peered":"2024-04-17T09:54:18.355942+0000","last_clean":"2024-04-17T09:54:18.355942+0000","last_became_active":"2024-04-17T09:54:17.387086+0000","last_became_peered":"2024-04-17T09:54:17.387086+0000","last_unstale":"2024-04-17T09:54:18.355942+0000","last_undegraded":"2024-04-17T09:54:18.355942+0000","last_fullsized":"2024-04-17T09:54:18.355942+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:21.920667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.448202+0000","last_change":"2024-04-17T09:54:17.374765+0000","last_active":"2024-04-17T09:54:18.448202+0000","last_peered":"2024-04-17T09:54:18.448202+0000","last_clean":"2024-04-17T09:54:18.448202+0000","last_became_active":"2024-04-17T09:54:17.374409+0000","last_became_peered":"2024-04-17T09:54:17.374409+0000","last_unstale":"2024-04-17T09:54:18.448202+0000","last_undegraded":"2024-04-17T09:54:18.448202+0000","last_fullsized":"2024-04-17T09:54:18.448202+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:58.120418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948817+0000","last_change":"2024-04-17T09:54:17.368495+0000","last_active":"2024-04-17T09:54:18.948817+0000","last_peered":"2024-04-17T09:54:18.948817+0000","last_clean":"2024-04-17T09:54:18.948817+0000","last_became_active":"2024-04-17T09:54:17.368225+0000","last_became_peered":"2024-04-17T09:54:17.368225+0000","last_unstale":"2024-04-17T09:54:18.948817+0000","last_undegraded":"2024-04-17T09:54:18.948817+0000","last_fullsized":"2024-04-17T09:54:18.948817+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:48:46.480627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949116+0000","last_change":"2024-04-17T09:54:17.371677+0000","last_active":"2024-04-17T09:54:18.949116+0000","last_peered":"2024-04-17T09:54:18.949116+0000","last_clean":"2024-04-17T09:54:18.949116+0000","last_became_active":"2024-04-17T09:54:17.371346+0000","last_became_peered":"2024-04-17T09:54:17.371346+0000","last_unstale":"2024-04-17T09:54:18.949116+0000","last_undegraded":"2024-04-17T09:54:18.949116+0000","last_fullsized":"2024-04-17T09:54:18.949116+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:07.206960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.392506+0000","last_change":"2024-04-17T09:54:17.371801+0000","last_active":"2024-04-17T09:54:18.392506+0000","last_peered":"2024-04-17T09:54:18.392506+0000","last_clean":"2024-04-17T09:54:18.392506+0000","last_became_active":"2024-04-17T09:54:17.371472+0000","last_became_peered":"2024-04-17T09:54:17.371472+0000","last_unstale":"2024-04-17T09:54:18.392506+0000","last_undegraded":"2024-04-17T09:54:18.392506+0000","last_fullsized":"2024-04-17T09:54:18.392506+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:58.710579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948334+0000","last_change":"2024-04-17T09:54:17.368572+0000","last_active":"2024-04-17T09:54:18.948334+0000","last_peered":"2024-04-17T09:54:18.948334+0000","last_clean":"2024-04-17T09:54:18.948334+0000","last_became_active":"2024-04-17T09:54:17.367856+0000","last_became_peered":"2024-04-17T09:54:17.367856+0000","last_unstale":"2024-04-17T09:54:18.948334+0000","last_undegraded":"2024-04-17T09:54:18.948334+0000","last_fullsized":"2024-04-17T09:54:18.948334+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:05:17.694457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395514+0000","last_change":"2024-04-17T09:54:17.380161+0000","last_active":"2024-04-17T09:54:18.395514+0000","last_peered":"2024-04-17T09:54:18.395514+0000","last_clean":"2024-04-17T09:54:18.395514+0000","last_became_active":"2024-04-17T09:54:17.379809+0000","last_became_peered":"2024-04-17T09:54:17.379809+0000","last_unstale":"2024-04-17T09:54:18.395514+0000","last_undegraded":"2024-04-17T09:54:18.395514+0000","last_fullsized":"2024-04-17T09:54:18.395514+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:05:17.694457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356536+0000","last_change":"2024-04-17T09:54:17.373209+0000","last_active":"2024-04-17T09:54:18.356536+0000","last_peered":"2024-04-17T09:54:18.356536+0000","last_clean":"2024-04-17T09:54:18.356536+0000","last_became_active":"2024-04-17T09:54:17.372647+0000","last_became_peered":"2024-04-17T09:54:17.372647+0000","last_unstale":"2024-04-17T09:54:18.356536+0000","last_undegraded":"2024-04-17T09:54:18.356536+0000","last_fullsized":"2024-04-17T09:54:18.356536+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:35.050031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395523+0000","last_change":"2024-04-17T09:54:17.387523+0000","last_active":"2024-04-17T09:54:18.395523+0000","last_peered":"2024-04-17T09:54:18.395523+0000","last_clean":"2024-04-17T09:54:18.395523+0000","last_became_active":"2024-04-17T09:54:17.384217+0000","last_became_peered":"2024-04-17T09:54:17.384217+0000","last_unstale":"2024-04-17T09:54:18.395523+0000","last_undegraded":"2024-04-17T09:54:18.395523+0000","last_fullsized":"2024-04-17T09:54:18.395523+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:26:01.393314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949066+0000","last_change":"2024-04-17T09:54:17.372311+0000","last_active":"2024-04-17T09:54:18.949066+0000","last_peered":"2024-04-17T09:54:18.949066+0000","last_clean":"2024-04-17T09:54:18.949066+0000","last_became_active":"2024-04-17T09:54:17.367208+0000","last_became_peered":"2024-04-17T09:54:17.367208+0000","last_unstale":"2024-04-17T09:54:18.949066+0000","last_undegraded":"2024-04-17T09:54:18.949066+0000","last_fullsized":"2024-04-17T09:54:18.949066+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:08:56.864943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948921+0000","last_change":"2024-04-17T09:54:17.371697+0000","last_active":"2024-04-17T09:54:18.948921+0000","last_peered":"2024-04-17T09:54:18.948921+0000","last_clean":"2024-04-17T09:54:18.948921+0000","last_became_active":"2024-04-17T09:54:17.371423+0000","last_became_peered":"2024-04-17T09:54:17.371423+0000","last_unstale":"2024-04-17T09:54:18.948921+0000","last_undegraded":"2024-04-17T09:54:18.948921+0000","last_fullsized":"2024-04-17T09:54:18.948921+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:58.710579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356175+0000","last_change":"2024-04-17T09:54:17.382293+0000","last_active":"2024-04-17T09:54:18.356175+0000","last_peered":"2024-04-17T09:54:18.356175+0000","last_clean":"2024-04-17T09:54:18.356175+0000","last_became_active":"2024-04-17T09:54:17.382015+0000","last_became_peered":"2024-04-17T09:54:17.382015+0000","last_unstale":"2024-04-17T09:54:18.356175+0000","last_undegraded":"2024-04-17T09:54:18.356175+0000","last_fullsized":"2024-04-17T09:54:18.356175+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:44:35.259640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355950+0000","last_change":"2024-04-17T09:54:17.366475+0000","last_active":"2024-04-17T09:54:18.355950+0000","last_peered":"2024-04-17T09:54:18.355950+0000","last_clean":"2024-04-17T09:54:18.355950+0000","last_became_active":"2024-04-17T09:54:17.365805+0000","last_became_peered":"2024-04-17T09:54:17.365805+0000","last_unstale":"2024-04-17T09:54:18.355950+0000","last_undegraded":"2024-04-17T09:54:18.355950+0000","last_fullsized":"2024-04-17T09:54:18.355950+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:49:29.350387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356020+0000","last_change":"2024-04-17T09:54:17.376263+0000","last_active":"2024-04-17T09:54:18.356020+0000","last_peered":"2024-04-17T09:54:18.356020+0000","last_clean":"2024-04-17T09:54:18.356020+0000","last_became_active":"2024-04-17T09:54:17.375608+0000","last_became_peered":"2024-04-17T09:54:17.375608+0000","last_unstale":"2024-04-17T09:54:18.356020+0000","last_undegraded":"2024-04-17T09:54:18.356020+0000","last_fullsized":"2024-04-17T09:54:18.356020+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:46:43.928292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.392786+0000","last_change":"2024-04-17T09:54:17.373590+0000","last_active":"2024-04-17T09:54:18.392786+0000","last_peered":"2024-04-17T09:54:18.392786+0000","last_clean":"2024-04-17T09:54:18.392786+0000","last_became_active":"2024-04-17T09:54:17.373319+0000","last_became_peered":"2024-04-17T09:54:17.373319+0000","last_unstale":"2024-04-17T09:54:18.392786+0000","last_undegraded":"2024-04-17T09:54:18.392786+0000","last_fullsized":"2024-04-17T09:54:18.392786+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:22:28.141447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356089+0000","last_change":"2024-04-17T09:54:16.360332+0000","last_active":"2024-04-17T09:54:18.356089+0000","last_peered":"2024-04-17T09:54:18.356089+0000","last_clean":"2024-04-17T09:54:18.356089+0000","last_became_active":"2024-04-17T09:54:16.360073+0000","last_became_peered":"2024-04-17T09:54:16.360073+0000","last_unstale":"2024-04-17T09:54:18.356089+0000","last_undegraded":"2024-04-17T09:54:18.356089+0000","last_fullsized":"2024-04-17T09:54:18.356089+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:53:03.772098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356083+0000","last_change":"2024-04-17T09:54:17.364860+0000","last_active":"2024-04-17T09:54:18.356083+0000","last_peered":"2024-04-17T09:54:18.356083+0000","last_clean":"2024-04-17T09:54:18.356083+0000","last_became_active":"2024-04-17T09:54:17.364484+0000","last_became_peered":"2024-04-17T09:54:17.364484+0000","last_unstale":"2024-04-17T09:54:18.356083+0000","last_undegraded":"2024-04-17T09:54:18.356083+0000","last_fullsized":"2024-04-17T09:54:18.356083+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:03:05.817393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.392822+0000","last_change":"2024-04-17T09:54:16.363721+0000","last_active":"2024-04-17T09:54:18.392822+0000","last_peered":"2024-04-17T09:54:18.392822+0000","last_clean":"2024-04-17T09:54:18.392822+0000","last_became_active":"2024-04-17T09:54:16.363083+0000","last_became_peered":"2024-04-17T09:54:16.363083+0000","last_unstale":"2024-04-17T09:54:18.392822+0000","last_undegraded":"2024-04-17T09:54:18.392822+0000","last_fullsized":"2024-04-17T09:54:18.392822+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:55:39.879843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949177+0000","last_change":"2024-04-17T09:54:17.369252+0000","last_active":"2024-04-17T09:54:18.949177+0000","last_peered":"2024-04-17T09:54:18.949177+0000","last_clean":"2024-04-17T09:54:18.949177+0000","last_became_active":"2024-04-17T09:54:17.368808+0000","last_became_peered":"2024-04-17T09:54:17.368808+0000","last_unstale":"2024-04-17T09:54:18.949177+0000","last_undegraded":"2024-04-17T09:54:18.949177+0000","last_fullsized":"2024-04-17T09:54:18.949177+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:10.103965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356099+0000","last_change":"2024-04-17T09:54:16.358302+0000","last_active":"2024-04-17T09:54:18.356099+0000","last_peered":"2024-04-17T09:54:18.356099+0000","last_clean":"2024-04-17T09:54:18.356099+0000","last_became_active":"2024-04-17T09:54:16.358036+0000","last_became_peered":"2024-04-17T09:54:16.358036+0000","last_unstale":"2024-04-17T09:54:18.356099+0000","last_undegraded":"2024-04-17T09:54:18.356099+0000","last_fullsized":"2024-04-17T09:54:18.356099+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:34:21.495573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395656+0000","last_change":"2024-04-17T09:54:17.381113+0000","last_active":"2024-04-17T09:54:18.395656+0000","last_peered":"2024-04-17T09:54:18.395656+0000","last_clean":"2024-04-17T09:54:18.395656+0000","last_became_active":"2024-04-17T09:54:17.380867+0000","last_became_peered":"2024-04-17T09:54:17.380867+0000","last_unstale":"2024-04-17T09:54:18.395656+0000","last_undegraded":"2024-04-17T09:54:18.395656+0000","last_fullsized":"2024-04-17T09:54:18.395656+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:25.615888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.448228+0000","last_change":"2024-04-17T09:54:16.369646+0000","last_active":"2024-04-17T09:54:18.448228+0000","last_peered":"2024-04-17T09:54:18.448228+0000","last_clean":"2024-04-17T09:54:18.448228+0000","last_became_active":"2024-04-17T09:54:16.369382+0000","last_became_peered":"2024-04-17T09:54:16.369382+0000","last_unstale":"2024-04-17T09:54:18.448228+0000","last_undegraded":"2024-04-17T09:54:18.448228+0000","last_fullsized":"2024-04-17T09:54:18.448228+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:31:39.263793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356378+0000","last_change":"2024-04-17T09:54:17.380563+0000","last_active":"2024-04-17T09:54:18.356378+0000","last_peered":"2024-04-17T09:54:18.356378+0000","last_clean":"2024-04-17T09:54:18.356378+0000","last_became_active":"2024-04-17T09:54:17.380254+0000","last_became_peered":"2024-04-17T09:54:17.380254+0000","last_unstale":"2024-04-17T09:54:18.356378+0000","last_undegraded":"2024-04-17T09:54:18.356378+0000","last_fullsized":"2024-04-17T09:54:18.356378+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:08:56.864943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948938+0000","last_change":"2024-04-17T09:54:16.355434+0000","last_active":"2024-04-17T09:54:18.948938+0000","last_peered":"2024-04-17T09:54:18.948938+0000","last_clean":"2024-04-17T09:54:18.948938+0000","last_became_active":"2024-04-17T09:54:16.354665+0000","last_became_peered":"2024-04-17T09:54:16.354665+0000","last_unstale":"2024-04-17T09:54:18.948938+0000","last_undegraded":"2024-04-17T09:54:18.948938+0000","last_fullsized":"2024-04-17T09:54:18.948938+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:36:18.292446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948414+0000","last_change":"2024-04-17T09:54:17.377485+0000","last_active":"2024-04-17T09:54:18.948414+0000","last_peered":"2024-04-17T09:54:18.948414+0000","last_clean":"2024-04-17T09:54:18.948414+0000","last_became_active":"2024-04-17T09:54:17.377173+0000","last_became_peered":"2024-04-17T09:54:17.377173+0000","last_unstale":"2024-04-17T09:54:18.948414+0000","last_undegraded":"2024-04-17T09:54:18.948414+0000","last_fullsized":"2024-04-17T09:54:18.948414+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:25.615888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948835+0000","last_change":"2024-04-17T09:54:16.352471+0000","last_active":"2024-04-17T09:54:18.948835+0000","last_peered":"2024-04-17T09:54:18.948835+0000","last_clean":"2024-04-17T09:54:18.948835+0000","last_became_active":"2024-04-17T09:54:16.350287+0000","last_became_peered":"2024-04-17T09:54:16.350287+0000","last_unstale":"2024-04-17T09:54:18.948835+0000","last_undegraded":"2024-04-17T09:54:18.948835+0000","last_fullsized":"2024-04-17T09:54:18.948835+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:59:30.048582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948678+0000","last_change":"2024-04-17T09:54:17.364045+0000","last_active":"2024-04-17T09:54:18.948678+0000","last_peered":"2024-04-17T09:54:18.948678+0000","last_clean":"2024-04-17T09:54:18.948678+0000","last_became_active":"2024-04-17T09:54:17.363677+0000","last_became_peered":"2024-04-17T09:54:17.363677+0000","last_unstale":"2024-04-17T09:54:18.948678+0000","last_undegraded":"2024-04-17T09:54:18.948678+0000","last_fullsized":"2024-04-17T09:54:18.948678+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:38:58.355284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355761+0000","last_change":"2024-04-17T09:54:16.359267+0000","last_active":"2024-04-17T09:54:18.355761+0000","last_peered":"2024-04-17T09:54:18.355761+0000","last_clean":"2024-04-17T09:54:18.355761+0000","last_became_active":"2024-04-17T09:54:16.358963+0000","last_became_peered":"2024-04-17T09:54:16.358963+0000","last_unstale":"2024-04-17T09:54:18.355761+0000","last_undegraded":"2024-04-17T09:54:18.355761+0000","last_fullsized":"2024-04-17T09:54:18.355761+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:38:45.976992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395033+0000","last_change":"2024-04-17T09:54:17.376624+0000","last_active":"2024-04-17T09:54:18.395033+0000","last_peered":"2024-04-17T09:54:18.395033+0000","last_clean":"2024-04-17T09:54:18.395033+0000","last_became_active":"2024-04-17T09:54:17.376365+0000","last_became_peered":"2024-04-17T09:54:17.376365+0000","last_unstale":"2024-04-17T09:54:18.395033+0000","last_undegraded":"2024-04-17T09:54:18.395033+0000","last_fullsized":"2024-04-17T09:54:18.395033+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:29:49.267181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949227+0000","last_change":"2024-04-17T09:54:16.353761+0000","last_active":"2024-04-17T09:54:18.949227+0000","last_peered":"2024-04-17T09:54:18.949227+0000","last_clean":"2024-04-17T09:54:18.949227+0000","last_became_active":"2024-04-17T09:54:16.353426+0000","last_became_peered":"2024-04-17T09:54:16.353426+0000","last_unstale":"2024-04-17T09:54:18.949227+0000","last_undegraded":"2024-04-17T09:54:18.949227+0000","last_fullsized":"2024-04-17T09:54:18.949227+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:20:24.640480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948657+0000","last_change":"2024-04-17T09:54:17.373037+0000","last_active":"2024-04-17T09:54:18.948657+0000","last_peered":"2024-04-17T09:54:18.948657+0000","last_clean":"2024-04-17T09:54:18.948657+0000","last_became_active":"2024-04-17T09:54:17.372733+0000","last_became_peered":"2024-04-17T09:54:17.372733+0000","last_unstale":"2024-04-17T09:54:18.948657+0000","last_undegraded":"2024-04-17T09:54:18.948657+0000","last_fullsized":"2024-04-17T09:54:18.948657+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:44:35.259640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356319+0000","last_change":"2024-04-17T09:54:16.365789+0000","last_active":"2024-04-17T09:54:18.356319+0000","last_peered":"2024-04-17T09:54:18.356319+0000","last_clean":"2024-04-17T09:54:18.356319+0000","last_became_active":"2024-04-17T09:54:16.365521+0000","last_became_peered":"2024-04-17T09:54:16.365521+0000","last_unstale":"2024-04-17T09:54:18.356319+0000","last_undegraded":"2024-04-17T09:54:18.356319+0000","last_fullsized":"2024-04-17T09:54:18.356319+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:03:04.812259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356313+0000","last_change":"2024-04-17T09:54:17.387554+0000","last_active":"2024-04-17T09:54:18.356313+0000","last_peered":"2024-04-17T09:54:18.356313+0000","last_clean":"2024-04-17T09:54:18.356313+0000","last_became_active":"2024-04-17T09:54:17.387190+0000","last_became_peered":"2024-04-17T09:54:17.387190+0000","last_unstale":"2024-04-17T09:54:18.356313+0000","last_undegraded":"2024-04-17T09:54:18.356313+0000","last_fullsized":"2024-04-17T09:54:18.356313+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:56:37.564448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.392904+0000","last_change":"2024-04-17T09:54:16.369081+0000","last_active":"2024-04-17T09:54:18.392904+0000","last_peered":"2024-04-17T09:54:18.392904+0000","last_clean":"2024-04-17T09:54:18.392904+0000","last_became_active":"2024-04-17T09:54:16.368772+0000","last_became_peered":"2024-04-17T09:54:16.368772+0000","last_unstale":"2024-04-17T09:54:18.392904+0000","last_undegraded":"2024-04-17T09:54:18.392904+0000","last_fullsized":"2024-04-17T09:54:18.392904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:33:18.638703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356029+0000","last_change":"2024-04-17T09:54:17.381710+0000","last_active":"2024-04-17T09:54:18.356029+0000","last_peered":"2024-04-17T09:54:18.356029+0000","last_clean":"2024-04-17T09:54:18.356029+0000","last_became_active":"2024-04-17T09:54:17.381363+0000","last_became_peered":"2024-04-17T09:54:17.381363+0000","last_unstale":"2024-04-17T09:54:18.356029+0000","last_undegraded":"2024-04-17T09:54:18.356029+0000","last_fullsized":"2024-04-17T09:54:18.356029+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:32:50.828977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948633+0000","last_change":"2024-04-17T09:54:16.364352+0000","last_active":"2024-04-17T09:54:18.948633+0000","last_peered":"2024-04-17T09:54:18.948633+0000","last_clean":"2024-04-17T09:54:18.948633+0000","last_became_active":"2024-04-17T09:54:16.363940+0000","last_became_peered":"2024-04-17T09:54:16.363940+0000","last_unstale":"2024-04-17T09:54:18.948633+0000","last_undegraded":"2024-04-17T09:54:18.948633+0000","last_fullsized":"2024-04-17T09:54:18.948633+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:57.115283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948502+0000","last_change":"2024-04-17T09:54:17.386126+0000","last_active":"2024-04-17T09:54:18.948502+0000","last_peered":"2024-04-17T09:54:18.948502+0000","last_clean":"2024-04-17T09:54:18.948502+0000","last_became_active":"2024-04-17T09:54:17.384769+0000","last_became_peered":"2024-04-17T09:54:17.384769+0000","last_unstale":"2024-04-17T09:54:18.948502+0000","last_undegraded":"2024-04-17T09:54:18.948502+0000","last_fullsized":"2024-04-17T09:54:18.948502+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:46:43.928292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356202+0000","last_change":"2024-04-17T09:54:16.367115+0000","last_active":"2024-04-17T09:54:18.356202+0000","last_peered":"2024-04-17T09:54:18.356202+0000","last_clean":"2024-04-17T09:54:18.356202+0000","last_became_active":"2024-04-17T09:54:16.362616+0000","last_became_peered":"2024-04-17T09:54:16.362616+0000","last_unstale":"2024-04-17T09:54:18.356202+0000","last_undegraded":"2024-04-17T09:54:18.356202+0000","last_fullsized":"2024-04-17T09:54:18.356202+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:49:28.345253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395244+0000","last_change":"2024-04-17T09:54:17.377531+0000","last_active":"2024-04-17T09:54:18.395244+0000","last_peered":"2024-04-17T09:54:18.395244+0000","last_clean":"2024-04-17T09:54:18.395244+0000","last_became_active":"2024-04-17T09:54:17.377268+0000","last_became_peered":"2024-04-17T09:54:17.377268+0000","last_unstale":"2024-04-17T09:54:18.395244+0000","last_undegraded":"2024-04-17T09:54:18.395244+0000","last_fullsized":"2024-04-17T09:54:18.395244+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:05:57.651811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949061+0000","last_change":"2024-04-17T09:54:16.356729+0000","last_active":"2024-04-17T09:54:18.949061+0000","last_peered":"2024-04-17T09:54:18.949061+0000","last_clean":"2024-04-17T09:54:18.949061+0000","last_became_active":"2024-04-17T09:54:16.356458+0000","last_became_peered":"2024-04-17T09:54:16.356458+0000","last_unstale":"2024-04-17T09:54:18.949061+0000","last_undegraded":"2024-04-17T09:54:18.949061+0000","last_fullsized":"2024-04-17T09:54:18.949061+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:09:00.415290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356088+0000","last_change":"2024-04-17T09:54:17.381943+0000","last_active":"2024-04-17T09:54:18.356088+0000","last_peered":"2024-04-17T09:54:18.356088+0000","last_clean":"2024-04-17T09:54:18.356088+0000","last_became_active":"2024-04-17T09:54:17.381545+0000","last_became_peered":"2024-04-17T09:54:17.381545+0000","last_unstale":"2024-04-17T09:54:18.356088+0000","last_undegraded":"2024-04-17T09:54:18.356088+0000","last_fullsized":"2024-04-17T09:54:18.356088+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:42:58.110213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395221+0000","last_change":"2024-04-17T09:54:16.362555+0000","last_active":"2024-04-17T09:54:18.395221+0000","last_peered":"2024-04-17T09:54:18.395221+0000","last_clean":"2024-04-17T09:54:18.395221+0000","last_became_active":"2024-04-17T09:54:16.362271+0000","last_became_peered":"2024-04-17T09:54:16.362271+0000","last_unstale":"2024-04-17T09:54:18.395221+0000","last_undegraded":"2024-04-17T09:54:18.395221+0000","last_fullsized":"2024-04-17T09:54:18.395221+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:53:03.772098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356184+0000","last_change":"2024-04-17T09:54:17.376536+0000","last_active":"2024-04-17T09:54:18.356184+0000","last_peered":"2024-04-17T09:54:18.356184+0000","last_clean":"2024-04-17T09:54:18.356184+0000","last_became_active":"2024-04-17T09:54:17.375993+0000","last_became_peered":"2024-04-17T09:54:17.375993+0000","last_unstale":"2024-04-17T09:54:18.356184+0000","last_undegraded":"2024-04-17T09:54:18.356184+0000","last_fullsized":"2024-04-17T09:54:18.356184+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:54.371142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355616+0000","last_change":"2024-04-17T09:54:16.363275+0000","last_active":"2024-04-17T09:54:18.355616+0000","last_peered":"2024-04-17T09:54:18.355616+0000","last_clean":"2024-04-17T09:54:18.355616+0000","last_became_active":"2024-04-17T09:54:16.362917+0000","last_became_peered":"2024-04-17T09:54:16.362917+0000","last_unstale":"2024-04-17T09:54:18.355616+0000","last_undegraded":"2024-04-17T09:54:18.355616+0000","last_fullsized":"2024-04-17T09:54:18.355616+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:25:59.865466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356117+0000","last_change":"2024-04-17T09:54:17.361797+0000","last_active":"2024-04-17T09:54:18.356117+0000","last_peered":"2024-04-17T09:54:18.356117+0000","last_clean":"2024-04-17T09:54:18.356117+0000","last_became_active":"2024-04-17T09:54:17.361510+0000","last_became_peered":"2024-04-17T09:54:17.361510+0000","last_unstale":"2024-04-17T09:54:18.356117+0000","last_undegraded":"2024-04-17T09:54:18.356117+0000","last_fullsized":"2024-04-17T09:54:18.356117+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:24:33.163226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":22,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355686+0000","last_change":"2024-04-17T09:54:16.362386+0000","last_active":"2024-04-17T09:54:18.355686+0000","last_peered":"2024-04-17T09:54:18.355686+0000","last_clean":"2024-04-17T09:54:18.355686+0000","last_became_active":"2024-04-17T09:54:16.362098+0000","last_became_peered":"2024-04-17T09:54:16.362098+0000","last_unstale":"2024-04-17T09:54:18.355686+0000","last_undegraded":"2024-04-17T09:54:18.355686+0000","last_fullsized":"2024-04-17T09:54:18.355686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:24:32.158091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949418+0000","last_change":"2024-04-17T09:54:17.375249+0000","last_active":"2024-04-17T09:54:18.949418+0000","last_peered":"2024-04-17T09:54:18.949418+0000","last_clean":"2024-04-17T09:54:18.949418+0000","last_became_active":"2024-04-17T09:54:17.374928+0000","last_became_peered":"2024-04-17T09:54:17.374928+0000","last_unstale":"2024-04-17T09:54:18.949418+0000","last_undegraded":"2024-04-17T09:54:18.949418+0000","last_fullsized":"2024-04-17T09:54:18.949418+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:19:45.596601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395603+0000","last_change":"2024-04-17T09:54:16.368531+0000","last_active":"2024-04-17T09:54:18.395603+0000","last_peered":"2024-04-17T09:54:18.395603+0000","last_clean":"2024-04-17T09:54:18.395603+0000","last_became_active":"2024-04-17T09:54:16.366600+0000","last_became_peered":"2024-04-17T09:54:16.366600+0000","last_unstale":"2024-04-17T09:54:18.395603+0000","last_undegraded":"2024-04-17T09:54:18.395603+0000","last_fullsized":"2024-04-17T09:54:18.395603+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:49:08.687051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949000+0000","last_change":"2024-04-17T09:54:17.368031+0000","last_active":"2024-04-17T09:54:18.949000+0000","last_peered":"2024-04-17T09:54:18.949000+0000","last_clean":"2024-04-17T09:54:18.949000+0000","last_became_active":"2024-04-17T09:54:17.367652+0000","last_became_peered":"2024-04-17T09:54:17.367652+0000","last_unstale":"2024-04-17T09:54:18.949000+0000","last_undegraded":"2024-04-17T09:54:18.949000+0000","last_fullsized":"2024-04-17T09:54:18.949000+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:24:07.861301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949564+0000","last_change":"2024-04-17T09:54:16.355502+0000","last_active":"2024-04-17T09:54:18.949564+0000","last_peered":"2024-04-17T09:54:18.949564+0000","last_clean":"2024-04-17T09:54:18.949564+0000","last_became_active":"2024-04-17T09:54:16.354313+0000","last_became_peered":"2024-04-17T09:54:16.354313+0000","last_unstale":"2024-04-17T09:54:18.949564+0000","last_undegraded":"2024-04-17T09:54:18.949564+0000","last_fullsized":"2024-04-17T09:54:18.949564+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:49.443648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395309+0000","last_change":"2024-04-17T09:54:17.384520+0000","last_active":"2024-04-17T09:54:18.395309+0000","last_peered":"2024-04-17T09:54:18.395309+0000","last_clean":"2024-04-17T09:54:18.395309+0000","last_became_active":"2024-04-17T09:54:17.384220+0000","last_became_peered":"2024-04-17T09:54:17.384220+0000","last_unstale":"2024-04-17T09:54:18.395309+0000","last_undegraded":"2024-04-17T09:54:18.395309+0000","last_fullsized":"2024-04-17T09:54:18.395309+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:47:21.997789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949500+0000","last_change":"2024-04-17T09:54:16.359120+0000","last_active":"2024-04-17T09:54:18.949500+0000","last_peered":"2024-04-17T09:54:18.949500+0000","last_clean":"2024-04-17T09:54:18.949500+0000","last_became_active":"2024-04-17T09:54:16.358779+0000","last_became_peered":"2024-04-17T09:54:16.358779+0000","last_unstale":"2024-04-17T09:54:18.949500+0000","last_undegraded":"2024-04-17T09:54:18.949500+0000","last_fullsized":"2024-04-17T09:54:18.949500+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:15.893018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.448217+0000","last_change":"2024-04-17T09:54:17.369109+0000","last_active":"2024-04-17T09:54:18.448217+0000","last_peered":"2024-04-17T09:54:18.448217+0000","last_clean":"2024-04-17T09:54:18.448217+0000","last_became_active":"2024-04-17T09:54:17.368845+0000","last_became_peered":"2024-04-17T09:54:17.368845+0000","last_unstale":"2024-04-17T09:54:18.448217+0000","last_undegraded":"2024-04-17T09:54:18.448217+0000","last_fullsized":"2024-04-17T09:54:18.448217+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:00.870600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949456+0000","last_change":"2024-04-17T09:54:16.354365+0000","last_active":"2024-04-17T09:54:18.949456+0000","last_peered":"2024-04-17T09:54:18.949456+0000","last_clean":"2024-04-17T09:54:18.949456+0000","last_became_active":"2024-04-17T09:54:16.354093+0000","last_became_peered":"2024-04-17T09:54:16.354093+0000","last_unstale":"2024-04-17T09:54:18.949456+0000","last_undegraded":"2024-04-17T09:54:18.949456+0000","last_fullsized":"2024-04-17T09:54:18.949456+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:35:34.537059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.448058+0000","last_change":"2024-04-17T09:54:17.362613+0000","last_active":"2024-04-17T09:54:18.448058+0000","last_peered":"2024-04-17T09:54:18.448058+0000","last_clean":"2024-04-17T09:54:18.448058+0000","last_became_active":"2024-04-17T09:54:17.362336+0000","last_became_peered":"2024-04-17T09:54:17.362336+0000","last_unstale":"2024-04-17T09:54:18.448058+0000","last_undegraded":"2024-04-17T09:54:18.448058+0000","last_fullsized":"2024-04-17T09:54:18.448058+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:33:09.050240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.392664+0000","last_change":"2024-04-17T09:54:16.363705+0000","last_active":"2024-04-17T09:54:18.392664+0000","last_peered":"2024-04-17T09:54:18.392664+0000","last_clean":"2024-04-17T09:54:18.392664+0000","last_became_active":"2024-04-17T09:54:16.363405+0000","last_became_peered":"2024-04-17T09:54:16.363405+0000","last_unstale":"2024-04-17T09:54:18.392664+0000","last_undegraded":"2024-04-17T09:54:18.392664+0000","last_fullsized":"2024-04-17T09:54:18.392664+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:38:27.804441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.448273+0000","last_change":"2024-04-17T09:54:17.379977+0000","last_active":"2024-04-17T09:54:18.448273+0000","last_peered":"2024-04-17T09:54:18.448273+0000","last_clean":"2024-04-17T09:54:18.448273+0000","last_became_active":"2024-04-17T09:54:17.379723+0000","last_became_peered":"2024-04-17T09:54:17.379723+0000","last_unstale":"2024-04-17T09:54:18.448273+0000","last_undegraded":"2024-04-17T09:54:18.448273+0000","last_fullsized":"2024-04-17T09:54:18.448273+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:49:29.350387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355901+0000","last_change":"2024-04-17T09:54:16.361330+0000","last_active":"2024-04-17T09:54:18.355901+0000","last_peered":"2024-04-17T09:54:18.355901+0000","last_clean":"2024-04-17T09:54:18.355901+0000","last_became_active":"2024-04-17T09:54:16.361056+0000","last_became_peered":"2024-04-17T09:54:16.361056+0000","last_unstale":"2024-04-17T09:54:18.355901+0000","last_undegraded":"2024-04-17T09:54:18.355901+0000","last_fullsized":"2024-04-17T09:54:18.355901+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:39:30.974585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.447914+0000","last_change":"2024-04-17T09:54:17.366185+0000","last_active":"2024-04-17T09:54:18.447914+0000","last_peered":"2024-04-17T09:54:18.447914+0000","last_clean":"2024-04-17T09:54:18.447914+0000","last_became_active":"2024-04-17T09:54:17.365837+0000","last_became_peered":"2024-04-17T09:54:17.365837+0000","last_unstale":"2024-04-17T09:54:18.447914+0000","last_undegraded":"2024-04-17T09:54:18.447914+0000","last_fullsized":"2024-04-17T09:54:18.447914+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:49:09.692185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949026+0000","last_change":"2024-04-17T09:54:16.365581+0000","last_active":"2024-04-17T09:54:18.949026+0000","last_peered":"2024-04-17T09:54:18.949026+0000","last_clean":"2024-04-17T09:54:18.949026+0000","last_became_active":"2024-04-17T09:54:16.365308+0000","last_became_peered":"2024-04-17T09:54:16.365308+0000","last_unstale":"2024-04-17T09:54:18.949026+0000","last_undegraded":"2024-04-17T09:54:18.949026+0000","last_fullsized":"2024-04-17T09:54:18.949026+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:55.683434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356301+0000","last_change":"2024-04-17T09:54:17.378209+0000","last_active":"2024-04-17T09:54:18.356301+0000","last_peered":"2024-04-17T09:54:18.356301+0000","last_clean":"2024-04-17T09:54:18.356301+0000","last_became_active":"2024-04-17T09:54:17.377727+0000","last_became_peered":"2024-04-17T09:54:17.377727+0000","last_unstale":"2024-04-17T09:54:18.356301+0000","last_undegraded":"2024-04-17T09:54:18.356301+0000","last_fullsized":"2024-04-17T09:54:18.356301+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:24:07.861301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.948281+0000","last_change":"2024-04-17T09:54:16.362646+0000","last_active":"2024-04-17T09:54:18.948281+0000","last_peered":"2024-04-17T09:54:18.948281+0000","last_clean":"2024-04-17T09:54:18.948281+0000","last_became_active":"2024-04-17T09:54:16.362306+0000","last_became_peered":"2024-04-17T09:54:16.362306+0000","last_unstale":"2024-04-17T09:54:18.948281+0000","last_undegraded":"2024-04-17T09:54:18.948281+0000","last_fullsized":"2024-04-17T09:54:18.948281+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:38:45.976992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.448239+0000","last_change":"2024-04-17T09:54:17.376712+0000","last_active":"2024-04-17T09:54:18.448239+0000","last_peered":"2024-04-17T09:54:18.448239+0000","last_clean":"2024-04-17T09:54:18.448239+0000","last_became_active":"2024-04-17T09:54:17.376382+0000","last_became_peered":"2024-04-17T09:54:17.376382+0000","last_unstale":"2024-04-17T09:54:18.448239+0000","last_undegraded":"2024-04-17T09:54:18.448239+0000","last_fullsized":"2024-04-17T09:54:18.448239+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:34:22.500707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395654+0000","last_change":"2024-04-17T09:54:16.366766+0000","last_active":"2024-04-17T09:54:18.395654+0000","last_peered":"2024-04-17T09:54:18.395654+0000","last_clean":"2024-04-17T09:54:18.395654+0000","last_became_active":"2024-04-17T09:54:16.366351+0000","last_became_peered":"2024-04-17T09:54:16.366351+0000","last_unstale":"2024-04-17T09:54:18.395654+0000","last_undegraded":"2024-04-17T09:54:18.395654+0000","last_fullsized":"2024-04-17T09:54:18.395654+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:33:08.045106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949142+0000","last_change":"2024-04-17T09:54:17.385290+0000","last_active":"2024-04-17T09:54:18.949142+0000","last_peered":"2024-04-17T09:54:18.949142+0000","last_clean":"2024-04-17T09:54:18.949142+0000","last_became_active":"2024-04-17T09:54:17.384972+0000","last_became_peered":"2024-04-17T09:54:17.384972+0000","last_unstale":"2024-04-17T09:54:18.949142+0000","last_undegraded":"2024-04-17T09:54:18.949142+0000","last_fullsized":"2024-04-17T09:54:18.949142+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:21.920667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.394428+0000","last_change":"2024-04-17T09:54:16.363895+0000","last_active":"2024-04-17T09:54:18.394428+0000","last_peered":"2024-04-17T09:54:18.394428+0000","last_clean":"2024-04-17T09:54:18.394428+0000","last_became_active":"2024-04-17T09:54:16.363461+0000","last_became_peered":"2024-04-17T09:54:16.363461+0000","last_unstale":"2024-04-17T09:54:18.394428+0000","last_undegraded":"2024-04-17T09:54:18.394428+0000","last_fullsized":"2024-04-17T09:54:18.394428+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:09:51.673285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.392687+0000","last_change":"2024-04-17T09:54:17.390013+0000","last_active":"2024-04-17T09:54:18.392687+0000","last_peered":"2024-04-17T09:54:18.392687+0000","last_clean":"2024-04-17T09:54:18.392687+0000","last_became_active":"2024-04-17T09:54:17.389724+0000","last_became_peered":"2024-04-17T09:54:17.389724+0000","last_unstale":"2024-04-17T09:54:18.392687+0000","last_undegraded":"2024-04-17T09:54:18.392687+0000","last_fullsized":"2024-04-17T09:54:18.392687+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:10.103965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395460+0000","last_change":"2024-04-17T09:54:16.364397+0000","last_active":"2024-04-17T09:54:18.395460+0000","last_peered":"2024-04-17T09:54:18.395460+0000","last_clean":"2024-04-17T09:54:18.395460+0000","last_became_active":"2024-04-17T09:54:16.364110+0000","last_became_peered":"2024-04-17T09:54:16.364110+0000","last_unstale":"2024-04-17T09:54:18.395460+0000","last_undegraded":"2024-04-17T09:54:18.395460+0000","last_fullsized":"2024-04-17T09:54:18.395460+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:12.816388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355784+0000","last_change":"2024-04-17T09:54:17.371227+0000","last_active":"2024-04-17T09:54:18.355784+0000","last_peered":"2024-04-17T09:54:18.355784+0000","last_clean":"2024-04-17T09:54:18.355784+0000","last_became_active":"2024-04-17T09:54:17.370965+0000","last_became_peered":"2024-04-17T09:54:17.370965+0000","last_unstale":"2024-04-17T09:54:18.355784+0000","last_undegraded":"2024-04-17T09:54:18.355784+0000","last_fullsized":"2024-04-17T09:54:18.355784+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:00.870600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.392758+0000","last_change":"2024-04-17T09:54:16.361087+0000","last_active":"2024-04-17T09:54:18.392758+0000","last_peered":"2024-04-17T09:54:18.392758+0000","last_clean":"2024-04-17T09:54:18.392758+0000","last_became_active":"2024-04-17T09:54:16.360811+0000","last_became_peered":"2024-04-17T09:54:16.360811+0000","last_unstale":"2024-04-17T09:54:18.392758+0000","last_undegraded":"2024-04-17T09:54:18.392758+0000","last_fullsized":"2024-04-17T09:54:18.392758+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:15:00.730444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.392642+0000","last_change":"2024-04-17T09:54:17.371753+0000","last_active":"2024-04-17T09:54:18.392642+0000","last_peered":"2024-04-17T09:54:18.392642+0000","last_clean":"2024-04-17T09:54:18.392642+0000","last_became_active":"2024-04-17T09:54:17.371448+0000","last_became_peered":"2024-04-17T09:54:17.371448+0000","last_unstale":"2024-04-17T09:54:18.392642+0000","last_undegraded":"2024-04-17T09:54:18.392642+0000","last_fullsized":"2024-04-17T09:54:18.392642+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:07.206960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.395312+0000","last_change":"2024-04-17T09:54:16.367447+0000","last_active":"2024-04-17T09:54:18.395312+0000","last_peered":"2024-04-17T09:54:18.395312+0000","last_clean":"2024-04-17T09:54:18.395312+0000","last_became_active":"2024-04-17T09:54:16.367196+0000","last_became_peered":"2024-04-17T09:54:16.367196+0000","last_unstale":"2024-04-17T09:54:18.395312+0000","last_undegraded":"2024-04-17T09:54:18.395312+0000","last_fullsized":"2024-04-17T09:54:18.395312+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:16:06.147936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355576+0000","last_change":"2024-04-17T09:54:17.387333+0000","last_active":"2024-04-17T09:54:18.355576+0000","last_peered":"2024-04-17T09:54:18.355576+0000","last_clean":"2024-04-17T09:54:18.355576+0000","last_became_active":"2024-04-17T09:54:17.382030+0000","last_became_peered":"2024-04-17T09:54:17.382030+0000","last_unstale":"2024-04-17T09:54:18.355576+0000","last_undegraded":"2024-04-17T09:54:18.355576+0000","last_fullsized":"2024-04-17T09:54:18.355576+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:54.371142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949136+0000","last_change":"2024-04-17T09:54:16.361810+0000","last_active":"2024-04-17T09:54:18.949136+0000","last_peered":"2024-04-17T09:54:18.949136+0000","last_clean":"2024-04-17T09:54:18.949136+0000","last_became_active":"2024-04-17T09:54:16.361521+0000","last_became_peered":"2024-04-17T09:54:16.361521+0000","last_unstale":"2024-04-17T09:54:18.949136+0000","last_undegraded":"2024-04-17T09:54:18.949136+0000","last_fullsized":"2024-04-17T09:54:18.949136+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:44.012560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.949338+0000","last_change":"2024-04-17T09:54:17.368893+0000","last_active":"2024-04-17T09:54:18.949338+0000","last_peered":"2024-04-17T09:54:18.949338+0000","last_clean":"2024-04-17T09:54:18.949338+0000","last_became_active":"2024-04-17T09:54:17.368627+0000","last_became_peered":"2024-04-17T09:54:17.368627+0000","last_unstale":"2024-04-17T09:54:18.949338+0000","last_undegraded":"2024-04-17T09:54:18.949338+0000","last_fullsized":"2024-04-17T09:54:18.949338+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:22:28.141447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355785+0000","last_change":"2024-04-17T09:54:16.365957+0000","last_active":"2024-04-17T09:54:18.355785+0000","last_peered":"2024-04-17T09:54:18.355785+0000","last_clean":"2024-04-17T09:54:18.355785+0000","last_became_active":"2024-04-17T09:54:16.365667+0000","last_became_peered":"2024-04-17T09:54:16.365667+0000","last_unstale":"2024-04-17T09:54:18.355785+0000","last_undegraded":"2024-04-17T09:54:18.355785+0000","last_fullsized":"2024-04-17T09:54:18.355785+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:07.205760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356429+0000","last_change":"2024-04-17T09:54:17.384255+0000","last_active":"2024-04-17T09:54:18.356429+0000","last_peered":"2024-04-17T09:54:18.356429+0000","last_clean":"2024-04-17T09:54:18.356429+0000","last_became_active":"2024-04-17T09:54:17.383920+0000","last_became_peered":"2024-04-17T09:54:17.383920+0000","last_unstale":"2024-04-17T09:54:18.356429+0000","last_undegraded":"2024-04-17T09:54:18.356429+0000","last_fullsized":"2024-04-17T09:54:18.356429+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:16.335939+0000","last_clean_scrub_stamp":"2024-04-17T09:54:16.335939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:21.920667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.355720+0000","last_change":"2024-04-17T09:54:16.354634+0000","last_active":"2024-04-17T09:54:18.355720+0000","last_peered":"2024-04-17T09:54:18.355720+0000","last_clean":"2024-04-17T09:54:18.355720+0000","last_became_active":"2024-04-17T09:54:16.354293+0000","last_became_peered":"2024-04-17T09:54:16.354293+0000","last_unstale":"2024-04-17T09:54:18.355720+0000","last_undegraded":"2024-04-17T09:54:18.355720+0000","last_fullsized":"2024-04-17T09:54:18.355720+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:54:15.330805+0000","last_clean_scrub_stamp":"2024-04-17T09:54:15.330805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:28:16.576076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":196,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-17T09:54:18.356427+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T09:54:18.356427+0000","last_peered":"2024-04-17T09:54:18.356427+0000","last_clean":"2024-04-17T09:54:18.356427+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T09:54:18.356427+0000","last_undegraded":"2024-04-17T09:54:18.356427+0000","last_fullsized":"2024-04-17T09:54:18.356427+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":987136,"data_stored":743634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":397312,"data_stored":153792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509459,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":397312,"data_stored":153848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509459,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":405504,"data_stored":154810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":393216,"data_stored":153246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":405504,"data_stored":154810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":397312,"data_stored":153280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":983040,"data_stored":743544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T09:54:21.885 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T09:54:21.885 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-17T09:54:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.069+0000 7fd0e4c00640 1 Processor -- start 2024-04-17T09:54:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.069+0000 7fd0e4c00640 1 -- start start 2024-04-17T09:54:22.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.069+0000 7fd0e4c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e006f7c0 0x7fd0e006ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:22.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.069+0000 7fd0e4c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd0e006f270 0x7fd0e006bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.073+0000 7fd0e4c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0e00cc940 con 0x7fd0e015b7d0 2024-04-17T09:54:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.073+0000 7fd0e4c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd0e00ccc20 con 0x7fd0e006f270 2024-04-17T09:54:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.073+0000 7fd0e4c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd0e006e0b0 con 0x7fd0e006f7c0 2024-04-17T09:54:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.073+0000 7fd0dde00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd0e015b7d0 0x7fd0e006e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37108/0 (socket says 172.21.15.32:37108) 2024-04-17T09:54:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.073+0000 7fd0dde00640 1 -- 172.21.15.32:0/2623394450 learned_addr learned my addr 172.21.15.32:0/2623394450 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.073+0000 7fd0dd400640 1 --2- 172.21.15.32:0/2623394450 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e006f7c0 0x7fd0e006ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.073+0000 7fd0de800640 1 --2- 172.21.15.32:0/2623394450 >> v2:172.21.15.144:3300/0 conn(0x7fd0e006f270 0x7fd0e006bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.073+0000 7fd0dca00640 1 -- 172.21.15.32:0/2623394450 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3820896827 0 0) 0x7fd0e00cc940 con 0x7fd0e015b7d0 2024-04-17T09:54:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.073+0000 7fd0dca00640 1 -- 172.21.15.32:0/2623394450 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0c4003530 con 0x7fd0e015b7d0 2024-04-17T09:54:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.073+0000 7fd0dd400640 1 -- 172.21.15.32:0/2623394450 >> v2:172.21.15.144:3300/0 conn(0x7fd0e006f270 msgr2=0x7fd0e006bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.073+0000 7fd0dd400640 1 --2- 172.21.15.32:0/2623394450 >> v2:172.21.15.144:3300/0 conn(0x7fd0e006f270 0x7fd0e006bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.073+0000 7fd0dd400640 1 -- 172.21.15.32:0/2623394450 >> v1:172.21.15.32:6789/0 conn(0x7fd0e015b7d0 legacy=0x7fd0e006e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.073+0000 7fd0dd400640 1 -- 172.21.15.32:0/2623394450 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e006e390 con 0x7fd0e006f7c0 2024-04-17T09:54:22.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.077+0000 7fd0dd400640 1 --2- 172.21.15.32:0/2623394450 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e006f7c0 0x7fd0e006ba30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0097c0 tx=0x7fd0cc01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6112bf68a75f7986 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:22.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.077+0000 7fd0dca00640 1 -- 172.21.15.32:0/2623394450 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc0ab430 con 0x7fd0e006f7c0 2024-04-17T09:54:22.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.077+0000 7fd0dca00640 1 -- 172.21.15.32:0/2623394450 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0cc0b0070 con 0x7fd0e006f7c0 2024-04-17T09:54:22.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.077+0000 7fd0dca00640 1 -- 172.21.15.32:0/2623394450 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc0ab7f0 con 0x7fd0e006f7c0 2024-04-17T09:54:22.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.077+0000 7fd0e4c00640 1 -- 172.21.15.32:0/2623394450 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e006f7c0 msgr2=0x7fd0e006ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:22.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.077+0000 7fd0e4c00640 1 --2- 172.21.15.32:0/2623394450 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e006f7c0 0x7fd0e006ba30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0097c0 tx=0x7fd0cc01c410 comp rx=0 tx=0).stop 2024-04-17T09:54:22.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.077+0000 7fd0e4c00640 1 -- 172.21.15.32:0/2623394450 shutdown_connections 2024-04-17T09:54:22.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.077+0000 7fd0e4c00640 1 --2- 172.21.15.32:0/2623394450 >> v2:172.21.15.144:3300/0 conn(0x7fd0e006f270 0x7fd0e006bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:22.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.077+0000 7fd0e4c00640 1 --2- 172.21.15.32:0/2623394450 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e006f7c0 0x7fd0e006ba30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:22.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.077+0000 7fd0e4c00640 1 -- 172.21.15.32:0/2623394450 >> 172.21.15.32:0/2623394450 conn(0x7fd0e00b2120 msgr2=0x7fd0e00b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:22.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.077+0000 7fd0e4c00640 1 -- 172.21.15.32:0/2623394450 shutdown_connections 2024-04-17T09:54:22.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.077+0000 7fd0e4c00640 1 -- 172.21.15.32:0/2623394450 wait complete. 2024-04-17T09:54:22.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.077+0000 7fd0e4c00640 1 Processor -- start 2024-04-17T09:54:22.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.077+0000 7fd0e4c00640 1 -- start start 2024-04-17T09:54:22.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.077+0000 7fd0e4c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0e006f270 0x7fd0e0145e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:22.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.077+0000 7fd0e4c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e006f7c0 0x7fd0e015ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:22.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.081+0000 7fd0e4c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e015b7d0 0x7fd0e0153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:22.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.081+0000 7fd0e4c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e00ceeb0 con 0x7fd0e015b7d0 2024-04-17T09:54:22.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.081+0000 7fd0e4c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e00ced30 con 0x7fd0e006f270 2024-04-17T09:54:22.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.081+0000 7fd0e4c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd0e0071e60 con 0x7fd0e006f7c0 2024-04-17T09:54:22.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.081+0000 7fd0de800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e015b7d0 0x7fd0e0153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:22.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.081+0000 7fd0de800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e015b7d0 0x7fd0e0153160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51048/0 (socket says 172.21.15.32:51048) 2024-04-17T09:54:22.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.081+0000 7fd0de800640 1 -- 172.21.15.32:0/269941576 learned_addr learned my addr 172.21.15.32:0/269941576 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:22.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.081+0000 7fd0dde00640 1 --2- 172.21.15.32:0/269941576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0e006f270 0x7fd0e0145e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:22.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.081+0000 7fd0dd400640 1 --2- 172.21.15.32:0/269941576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e006f7c0 0x7fd0e015ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:22.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.081+0000 7fd0de800640 1 -- 172.21.15.32:0/269941576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e006f7c0 msgr2=0x7fd0e015ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:22.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.081+0000 7fd0de800640 1 --2- 172.21.15.32:0/269941576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e006f7c0 0x7fd0e015ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:22.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.081+0000 7fd0de800640 1 -- 172.21.15.32:0/269941576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0e006f270 msgr2=0x7fd0e0145e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:22.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.081+0000 7fd0de800640 1 --2- 172.21.15.32:0/269941576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0e006f270 0x7fd0e0145e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:22.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.081+0000 7fd0de800640 1 -- 172.21.15.32:0/269941576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e015ce30 con 0x7fd0e015b7d0 2024-04-17T09:54:22.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.081+0000 7fd0dd400640 1 --2- 172.21.15.32:0/269941576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e006f7c0 0x7fd0e015ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:22.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.081+0000 7fd0de800640 1 --2- 172.21.15.32:0/269941576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e015b7d0 0x7fd0e0153160 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800efc0 tx=0x7fd0d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:22.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.081+0000 7fd0d6000640 1 -- 172.21.15.32:0/269941576 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8018020 con 0x7fd0e015b7d0 2024-04-17T09:54:22.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.081+0000 7fd0d6000640 1 -- 172.21.15.32:0/269941576 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d8014480 con 0x7fd0e015b7d0 2024-04-17T09:54:22.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.081+0000 7fd0d6000640 1 -- 172.21.15.32:0/269941576 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8014870 con 0x7fd0e015b7d0 2024-04-17T09:54:22.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.081+0000 7fd0e4c00640 1 -- 172.21.15.32:0/269941576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e015d110 con 0x7fd0e015b7d0 2024-04-17T09:54:22.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.081+0000 7fd0e4c00640 1 -- 172.21.15.32:0/269941576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e015d6d0 con 0x7fd0e015b7d0 2024-04-17T09:54:22.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.081+0000 7fd0e4c00640 1 -- 172.21.15.32:0/269941576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e0068a90 con 0x7fd0e015b7d0 2024-04-17T09:54:22.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.081+0000 7fd0d6000640 1 -- 172.21.15.32:0/269941576 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd0d801b020 con 0x7fd0e015b7d0 2024-04-17T09:54:22.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.085+0000 7fd0d6000640 1 --2- 172.21.15.32:0/269941576 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd0c0068380 0x7fd0c006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:22.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.085+0000 7fd0d6000640 1 -- 172.21.15.32:0/269941576 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fd0d8087560 con 0x7fd0e015b7d0 2024-04-17T09:54:22.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.085+0000 7fd0dde00640 1 --2- 172.21.15.32:0/269941576 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd0c0068380 0x7fd0c006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:22.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.085+0000 7fd0dde00640 1 --2- 172.21.15.32:0/269941576 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd0c0068380 0x7fd0c006a840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0007fd0 tx=0x7fd0d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:22.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.089+0000 7fd0d6000640 1 -- 172.21.15.32:0/269941576 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd0d8022070 con 0x7fd0e015b7d0 2024-04-17T09:54:22.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.313+0000 7fd0e4c00640 1 -- 172.21.15.32:0/269941576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd0e0075010 con 0x7fd0e015b7d0 2024-04-17T09:54:22.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.313+0000 7fd0d6000640 1 -- 172.21.15.32:0/269941576 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7fd0d8010040 con 0x7fd0e015b7d0 2024-04-17T09:54:22.316 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-17T09:54:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDPmx9mUXoLKxAANV7zhLvL0DqHngk4TtsdBQ== 2024-04-17T09:54:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2024-04-17T09:54:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2024-04-17T09:54:22.316 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-17T09:54:22.316 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2024-04-17T09:54:22.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.317+0000 7fd0e4c00640 1 -- 172.21.15.32:0/269941576 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd0c0068380 msgr2=0x7fd0c006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:22.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.317+0000 7fd0e4c00640 1 --2- 172.21.15.32:0/269941576 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd0c0068380 0x7fd0c006a840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0007fd0 tx=0x7fd0d0009040 comp rx=0 tx=0).stop 2024-04-17T09:54:22.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.317+0000 7fd0e4c00640 1 -- 172.21.15.32:0/269941576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e015b7d0 msgr2=0x7fd0e0153160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:22.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.317+0000 7fd0e4c00640 1 --2- 172.21.15.32:0/269941576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e015b7d0 0x7fd0e0153160 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800efc0 tx=0x7fd0d800c490 comp rx=0 tx=0).stop 2024-04-17T09:54:22.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.317+0000 7fd0e4c00640 1 -- 172.21.15.32:0/269941576 shutdown_connections 2024-04-17T09:54:22.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.317+0000 7fd0e4c00640 1 --2- 172.21.15.32:0/269941576 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd0c0068380 0x7fd0c006a840 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:22.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.317+0000 7fd0e4c00640 1 --2- 172.21.15.32:0/269941576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e006f7c0 0x7fd0e015ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:22.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.317+0000 7fd0e4c00640 1 --2- 172.21.15.32:0/269941576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e015b7d0 0x7fd0e0153160 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:22.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.317+0000 7fd0e4c00640 1 --2- 172.21.15.32:0/269941576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0e006f270 0x7fd0e0145e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:22.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.317+0000 7fd0e4c00640 1 -- 172.21.15.32:0/269941576 >> 172.21.15.32:0/269941576 conn(0x7fd0e00b2120 msgr2=0x7fd0e015a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:22.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.317+0000 7fd0e4c00640 1 -- 172.21.15.32:0/269941576 shutdown_connections 2024-04-17T09:54:22.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.317+0000 7fd0e4c00640 1 -- 172.21.15.32:0/269941576 wait complete. 2024-04-17T09:54:22.337 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:54:22.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb215a00640 1 Processor -- start 2024-04-17T09:54:22.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb215a00640 1 -- start start 2024-04-17T09:54:22.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb215a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb2100d00b0 0x7fb21009a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:22.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb215a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb21015c9a0 0x7fb21009af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:22.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb215a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2100d05e0 con 0x7fb21015c5c0 2024-04-17T09:54:22.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb215a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb2100ce560 con 0x7fb2100d00b0 2024-04-17T09:54:22.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb215a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb2100d11f0 con 0x7fb21015c9a0 2024-04-17T09:54:22.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb20ea00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb21015c5c0 0x7fb2100cde50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37120/0 (socket says 172.21.15.32:37120) 2024-04-17T09:54:22.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb20ea00640 1 -- 172.21.15.32:0/3573228218 learned_addr learned my addr 172.21.15.32:0/3573228218 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:22.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb206000640 1 --2- 172.21.15.32:0/3573228218 >> v2:172.21.15.144:3300/0 conn(0x7fb2100d00b0 0x7fb21009a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:22.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb20f400640 1 --2- 172.21.15.32:0/3573228218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb21015c9a0 0x7fb21009af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:22.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb20e000640 1 -- 172.21.15.32:0/3573228218 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1905136514 0 0) 0x7fb2100d05e0 con 0x7fb21015c5c0 2024-04-17T09:54:22.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb20e000640 1 -- 172.21.15.32:0/3573228218 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1f4003530 con 0x7fb21015c5c0 2024-04-17T09:54:22.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb20e000640 1 -- 172.21.15.32:0/3573228218 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 911932978 0 0) 0x7fb1f4003530 con 0x7fb21015c5c0 2024-04-17T09:54:22.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb20e000640 1 -- 172.21.15.32:0/3573228218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb21015c9a0 msgr2=0x7fb21009af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:22.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb20e000640 1 --2- 172.21.15.32:0/3573228218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb21015c9a0 0x7fb21009af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:22.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb20e000640 1 -- 172.21.15.32:0/3573228218 >> v2:172.21.15.144:3300/0 conn(0x7fb2100d00b0 msgr2=0x7fb21009a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:22.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb20e000640 1 --2- 172.21.15.32:0/3573228218 >> v2:172.21.15.144:3300/0 conn(0x7fb2100d00b0 0x7fb21009a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:22.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb20e000640 1 -- 172.21.15.32:0/3573228218 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2100d08c0 con 0x7fb21015c5c0 2024-04-17T09:54:22.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb20e000640 1 -- 172.21.15.32:0/3573228218 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7fb200001680 con 0x7fb21015c5c0 2024-04-17T09:54:22.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb20e000640 1 -- 172.21.15.32:0/3573228218 >> v1:172.21.15.32:6789/0 conn(0x7fb21015c5c0 legacy=0x7fb2100cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:22.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb20e000640 1 --2- 172.21.15.32:0/3573228218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1f40039c0 0x7fb1f4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:22.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb20e000640 1 --2- 172.21.15.32:0/3573228218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1f4004810 0x7fb1f4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:22.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb20e000640 1 --2- 172.21.15.32:0/3573228218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb21015c9a0 0x7fb1f40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:22.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb20e000640 1 -- 172.21.15.32:0/3573228218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb2100ce560 con 0x7fb1f40039c0 2024-04-17T09:54:22.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb20e000640 1 -- 172.21.15.32:0/3573228218 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2100d11f0 con 0x7fb1f4004810 2024-04-17T09:54:22.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.477+0000 7fb20e000640 1 -- 172.21.15.32:0/3573228218 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb1f400b9d0 con 0x7fb21015c9a0 2024-04-17T09:54:22.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.481+0000 7fb20ea00640 1 --2- 172.21.15.32:0/3573228218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1f40039c0 0x7fb1f4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:22.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.481+0000 7fb206000640 1 --2- 172.21.15.32:0/3573228218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1f4004810 0x7fb1f4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:22.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.481+0000 7fb20f400640 1 --2- 172.21.15.32:0/3573228218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb21015c9a0 0x7fb1f40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:22.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.481+0000 7fb206000640 1 -- 172.21.15.32:0/3573228218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb21015c9a0 msgr2=0x7fb1f40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:22.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.481+0000 7fb206000640 1 --2- 172.21.15.32:0/3573228218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb21015c9a0 0x7fb1f40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:22.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.481+0000 7fb206000640 1 -- 172.21.15.32:0/3573228218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1f40039c0 msgr2=0x7fb1f4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:22.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.481+0000 7fb206000640 1 --2- 172.21.15.32:0/3573228218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1f40039c0 0x7fb1f4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:22.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.481+0000 7fb206000640 1 -- 172.21.15.32:0/3573228218 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f4003530 con 0x7fb1f4004810 2024-04-17T09:54:22.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.481+0000 7fb206000640 1 --2- 172.21.15.32:0/3573228218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1f4004810 0x7fb1f4004c90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009ae0 tx=0x7fb1f801ca80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6f73743a4c132a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:22.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.481+0000 7fb20e000640 1 -- 172.21.15.32:0/3573228218 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f80096b0 con 0x7fb1f4004810 2024-04-17T09:54:22.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.481+0000 7fb20e000640 1 -- 172.21.15.32:0/3573228218 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1f80a6490 con 0x7fb1f4004810 2024-04-17T09:54:22.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.481+0000 7fb20e000640 1 -- 172.21.15.32:0/3573228218 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f80a6880 con 0x7fb1f4004810 2024-04-17T09:54:22.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb215a00640 1 -- 172.21.15.32:0/3573228218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1f4004810 msgr2=0x7fb1f4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:22.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb215a00640 1 --2- 172.21.15.32:0/3573228218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1f4004810 0x7fb1f4004c90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009ae0 tx=0x7fb1f801ca80 comp rx=0 tx=0).stop 2024-04-17T09:54:22.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb20f400640 1 -- 172.21.15.32:0/3573228218 reap_dead start 2024-04-17T09:54:22.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb215a00640 1 -- 172.21.15.32:0/3573228218 shutdown_connections 2024-04-17T09:54:22.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb215a00640 1 -- 172.21.15.32:0/3573228218 >> 172.21.15.32:0/3573228218 conn(0x7fb2100b3fd0 msgr2=0x7fb2100b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:22.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb215a00640 1 -- 172.21.15.32:0/3573228218 shutdown_connections 2024-04-17T09:54:22.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb215a00640 1 -- 172.21.15.32:0/3573228218 wait complete. 2024-04-17T09:54:22.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb215a00640 1 Processor -- start 2024-04-17T09:54:22.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb215a00640 1 -- start start 2024-04-17T09:54:22.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb215a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1f4004810 0x7fb210158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:22.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb215a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb21015cec0 0x7fb210156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:22.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb215a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb210157370 0x7fb2100c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:22.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb215a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb2100b3d90 con 0x7fb21015cec0 2024-04-17T09:54:22.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb215a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2100b3c10 con 0x7fb1f4004810 2024-04-17T09:54:22.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb215a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb2100b5330 con 0x7fb210157370 2024-04-17T09:54:22.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb206000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb21015cec0 0x7fb210156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:22.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb206000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb21015cec0 0x7fb210156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51076/0 (socket says 172.21.15.32:51076) 2024-04-17T09:54:22.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb206000640 1 -- 172.21.15.32:0/3444981663 learned_addr learned my addr 172.21.15.32:0/3444981663 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:22.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb20f400640 1 --2- 172.21.15.32:0/3444981663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb210157370 0x7fb2100c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:22.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb20ea00640 1 --2- 172.21.15.32:0/3444981663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1f4004810 0x7fb210158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:22.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb206000640 1 -- 172.21.15.32:0/3444981663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb210157370 msgr2=0x7fb2100c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:22.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb206000640 1 --2- 172.21.15.32:0/3444981663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb210157370 0x7fb2100c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:22.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb206000640 1 -- 172.21.15.32:0/3444981663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1f4004810 msgr2=0x7fb210158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:22.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb206000640 1 --2- 172.21.15.32:0/3444981663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1f4004810 0x7fb210158b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:22.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb206000640 1 -- 172.21.15.32:0/3444981663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb210157cc0 con 0x7fb21015cec0 2024-04-17T09:54:22.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.485+0000 7fb206000640 1 --2- 172.21.15.32:0/3444981663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb21015cec0 0x7fb210156e30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009ae0 tx=0x7fb1f4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:22.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.489+0000 7fb207400640 1 -- 172.21.15.32:0/3444981663 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f80096b0 con 0x7fb21015cec0 2024-04-17T09:54:22.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.489+0000 7fb207400640 1 -- 172.21.15.32:0/3444981663 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1f80a6c40 con 0x7fb21015cec0 2024-04-17T09:54:22.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.489+0000 7fb207400640 1 -- 172.21.15.32:0/3444981663 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f80a7470 con 0x7fb21015cec0 2024-04-17T09:54:22.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.489+0000 7fb215a00640 1 -- 172.21.15.32:0/3444981663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb21015a6b0 con 0x7fb21015cec0 2024-04-17T09:54:22.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.489+0000 7fb215a00640 1 -- 172.21.15.32:0/3444981663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb21015ac70 con 0x7fb21015cec0 2024-04-17T09:54:22.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.489+0000 7fb215a00640 1 -- 172.21.15.32:0/3444981663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb210068a90 con 0x7fb21015cec0 2024-04-17T09:54:22.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.489+0000 7fb207400640 1 -- 172.21.15.32:0/3444981663 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb1f80c2060 con 0x7fb21015cec0 2024-04-17T09:54:22.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.489+0000 7fb207400640 1 --2- 172.21.15.32:0/3444981663 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1ec0683b0 0x7fb1ec06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:22.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.489+0000 7fb207400640 1 -- 172.21.15.32:0/3444981663 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fb1f81207d0 con 0x7fb21015cec0 2024-04-17T09:54:22.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.489+0000 7fb20ea00640 1 --2- 172.21.15.32:0/3444981663 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1ec0683b0 0x7fb1ec06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:22.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.493+0000 7fb20ea00640 1 --2- 172.21.15.32:0/3444981663 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1ec0683b0 0x7fb1ec06a870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb210166510 tx=0x7fb200007900 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:22.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.493+0000 7fb207400640 1 -- 172.21.15.32:0/3444981663 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fb1f80b6070 con 0x7fb21015cec0 2024-04-17T09:54:22.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.693+0000 7fb215a00640 1 -- 172.21.15.32:0/3444981663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2100b4c30 con 0x7fb21015cec0 2024-04-17T09:54:22.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.693+0000 7fb207400640 1 -- 172.21.15.32:0/3444981663 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5199 (secure 0 0 0) 0x7fb1f80ac020 con 0x7fb21015cec0 2024-04-17T09:54:22.694 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:22.694 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2127207201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2127207201},{"type":"v1","addr":"172.21.15.144:6835","nonce":2127207201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5110,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3782699135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3782699135},{"type":"v1","addr":"172.21.15.144:6834","nonce":3782699135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/385811857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":385811857},{"type":"v1","addr":"172.21.15.32:6836","nonce":385811857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:54:16.745151+0000","modified":"2024-04-17T09:54:19.761481+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5217":{"gid":5217,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/925293029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":925293029},{"type":"v1","addr":"172.21.15.32:6837","nonce":925293029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5217,"qdb_cluster":[5217]},"id":2}]} 2024-04-17T09:54:22.694 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 15 2024-04-17T09:54:22.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.693+0000 7fb215a00640 1 -- 172.21.15.32:0/3444981663 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1ec0683b0 msgr2=0x7fb1ec06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:22.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.693+0000 7fb215a00640 1 --2- 172.21.15.32:0/3444981663 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1ec0683b0 0x7fb1ec06a870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb210166510 tx=0x7fb200007900 comp rx=0 tx=0).stop 2024-04-17T09:54:22.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.693+0000 7fb215a00640 1 -- 172.21.15.32:0/3444981663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb21015cec0 msgr2=0x7fb210156e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:22.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.693+0000 7fb215a00640 1 --2- 172.21.15.32:0/3444981663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb21015cec0 0x7fb210156e30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009ae0 tx=0x7fb1f4005840 comp rx=0 tx=0).stop 2024-04-17T09:54:22.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.693+0000 7fb215a00640 1 -- 172.21.15.32:0/3444981663 shutdown_connections 2024-04-17T09:54:22.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.693+0000 7fb215a00640 1 --2- 172.21.15.32:0/3444981663 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1ec0683b0 0x7fb1ec06a870 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:22.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.693+0000 7fb215a00640 1 --2- 172.21.15.32:0/3444981663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb210157370 0x7fb2100c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:22.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.693+0000 7fb215a00640 1 --2- 172.21.15.32:0/3444981663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb21015cec0 0x7fb210156e30 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:22.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.693+0000 7fb215a00640 1 --2- 172.21.15.32:0/3444981663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1f4004810 0x7fb210158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:22.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.693+0000 7fb215a00640 1 -- 172.21.15.32:0/3444981663 >> 172.21.15.32:0/3444981663 conn(0x7fb2100b3fd0 msgr2=0x7fb210070660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:22.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.693+0000 7fb215a00640 1 -- 172.21.15.32:0/3444981663 shutdown_connections 2024-04-17T09:54:22.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:22.693+0000 7fb215a00640 1 -- 172.21.15.32:0/3444981663 wait complete. 2024-04-17T09:54:23.119 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T09:54:16.745151+0000', 'modified': '2024-04-17T09:54:19.761481+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5217}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5217': {'gid': 5217, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6837/925293029', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6835', 'nonce': 925293029}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 925293029}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5217, 'qdb_cluster': [5217]} 2024-04-17T09:54:23.119 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T09:54:23.119 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-17T09:54:23.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.261+0000 7f0dae400640 1 Processor -- start 2024-04-17T09:54:23.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.261+0000 7f0dae400640 1 -- start start 2024-04-17T09:54:23.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.265+0000 7f0dae400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0da813f6e0 0x7f0da80cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.265+0000 7f0dae400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0da81601e0 con 0x7f0da80cf370 2024-04-17T09:54:23.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.265+0000 7f0dae400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0da8160480 con 0x7f0da80d09f0 2024-04-17T09:54:23.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.265+0000 7f0dae400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0da813fac0 con 0x7f0da813f6e0 2024-04-17T09:54:23.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.265+0000 7f0d9ea00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0da80cf370 0x7f0da80d02e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37132/0 (socket says 172.21.15.32:37132) 2024-04-17T09:54:23.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.265+0000 7f0da7400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0da813f6e0 0x7f0da80cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.265+0000 7f0d9ea00640 1 -- 172.21.15.32:0/1835349498 learned_addr learned my addr 172.21.15.32:0/1835349498 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:23.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.265+0000 7f0da7400640 1 -- 172.21.15.32:0/1835349498 >> v1:172.21.15.144:6789/0 conn(0x7f0da80d09f0 legacy=0x7f0da8071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:23.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.265+0000 7f0da7400640 1 -- 172.21.15.32:0/1835349498 >> v1:172.21.15.32:6789/0 conn(0x7f0da80cf370 legacy=0x7f0da80d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:23.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.265+0000 7f0da7400640 1 -- 172.21.15.32:0/1835349498 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da8160760 con 0x7f0da813f6e0 2024-04-17T09:54:23.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.265+0000 7f0da7400640 1 --2- 172.21.15.32:0/1835349498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da813f6e0 0x7f0da80cf850 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009d90 tx=0x7f0d9801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8723ef46789c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:23.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.265+0000 7f0da6a00640 1 -- 172.21.15.32:0/1835349498 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d980b2070 con 0x7f0da813f6e0 2024-04-17T09:54:23.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.265+0000 7f0da6a00640 1 -- 172.21.15.32:0/1835349498 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d980a59c0 con 0x7f0da813f6e0 2024-04-17T09:54:23.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.265+0000 7f0da6a00640 1 -- 172.21.15.32:0/1835349498 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d980a5d90 con 0x7f0da813f6e0 2024-04-17T09:54:23.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.269+0000 7f0dae400640 1 -- 172.21.15.32:0/1835349498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da813f6e0 msgr2=0x7f0da80cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:23.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.269+0000 7f0dae400640 1 --2- 172.21.15.32:0/1835349498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da813f6e0 0x7f0da80cf850 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009d90 tx=0x7f0d9801c900 comp rx=0 tx=0).stop 2024-04-17T09:54:23.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.269+0000 7f0dae400640 1 -- 172.21.15.32:0/1835349498 shutdown_connections 2024-04-17T09:54:23.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.269+0000 7f0dae400640 1 --2- 172.21.15.32:0/1835349498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da813f6e0 0x7f0da80cf850 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.269+0000 7f0dae400640 1 -- 172.21.15.32:0/1835349498 >> 172.21.15.32:0/1835349498 conn(0x7f0da806ba30 msgr2=0x7f0da806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:23.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.269+0000 7f0dae400640 1 -- 172.21.15.32:0/1835349498 shutdown_connections 2024-04-17T09:54:23.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.269+0000 7f0dae400640 1 -- 172.21.15.32:0/1835349498 wait complete. 2024-04-17T09:54:23.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.269+0000 7f0dae400640 1 Processor -- start 2024-04-17T09:54:23.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.269+0000 7f0dae400640 1 -- start start 2024-04-17T09:54:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.269+0000 7f0dae400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0da80cf370 0x7f0da815a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.269+0000 7f0dae400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da80d09f0 0x7f0da80b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.269+0000 7f0dae400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0da813f6e0 0x7f0da80b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.269+0000 7f0dae400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0da80cff10 con 0x7f0da80d09f0 2024-04-17T09:54:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.269+0000 7f0dae400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0da80cfd90 con 0x7f0da813f6e0 2024-04-17T09:54:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.269+0000 7f0dae400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0da816a590 con 0x7f0da80cf370 2024-04-17T09:54:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.269+0000 7f0d9ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da80d09f0 0x7f0da80b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.269+0000 7f0d9ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da80d09f0 0x7f0da80b19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51092/0 (socket says 172.21.15.32:51092) 2024-04-17T09:54:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.269+0000 7f0d9ea00640 1 -- 172.21.15.32:0/3158072434 learned_addr learned my addr 172.21.15.32:0/3158072434 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.269+0000 7f0da7400640 1 --2- 172.21.15.32:0/3158072434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0da80cf370 0x7f0da815a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.269+0000 7f0da7e00640 1 --2- 172.21.15.32:0/3158072434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0da813f6e0 0x7f0da80b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.273+0000 7f0d9ea00640 1 -- 172.21.15.32:0/3158072434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0da80cf370 msgr2=0x7f0da815a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.273+0000 7f0d9ea00640 1 --2- 172.21.15.32:0/3158072434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0da80cf370 0x7f0da815a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.273+0000 7f0d9ea00640 1 -- 172.21.15.32:0/3158072434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0da813f6e0 msgr2=0x7f0da80b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:23.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.273+0000 7f0d9ea00640 1 --2- 172.21.15.32:0/3158072434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0da813f6e0 0x7f0da80b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.273+0000 7f0d9ea00640 1 -- 172.21.15.32:0/3158072434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da813dfe0 con 0x7f0da80d09f0 2024-04-17T09:54:23.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.273+0000 7f0da7e00640 1 --2- 172.21.15.32:0/3158072434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0da813f6e0 0x7f0da80b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:23.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.273+0000 7f0d9ea00640 1 --2- 172.21.15.32:0/3158072434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da80d09f0 0x7f0da80b19c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000c430 tx=0x7f0d9000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:23.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.273+0000 7f0d9fe00640 1 -- 172.21.15.32:0/3158072434 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90017070 con 0x7f0da80d09f0 2024-04-17T09:54:23.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.273+0000 7f0d9fe00640 1 -- 172.21.15.32:0/3158072434 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d90005d10 con 0x7f0da80d09f0 2024-04-17T09:54:23.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.273+0000 7f0d9fe00640 1 -- 172.21.15.32:0/3158072434 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90007310 con 0x7f0da80d09f0 2024-04-17T09:54:23.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.273+0000 7f0dae400640 1 -- 172.21.15.32:0/3158072434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da80b2530 con 0x7f0da80d09f0 2024-04-17T09:54:23.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.273+0000 7f0dae400640 1 -- 172.21.15.32:0/3158072434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da81296e0 con 0x7f0da80d09f0 2024-04-17T09:54:23.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.273+0000 7f0d9fe00640 1 -- 172.21.15.32:0/3158072434 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0d9001a070 con 0x7f0da80d09f0 2024-04-17T09:54:23.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.273+0000 7f0d9fe00640 1 --2- 172.21.15.32:0/3158072434 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0d84068380 0x7f0d8406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.273+0000 7f0d9fe00640 1 -- 172.21.15.32:0/3158072434 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f0d90085d50 con 0x7f0da80d09f0 2024-04-17T09:54:23.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.273+0000 7f0da7400640 1 --2- 172.21.15.32:0/3158072434 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0d84068380 0x7f0d8406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0db199a000 1 Processor -- start 2024-04-17T09:54:23.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0db199a000 1 -- start start 2024-04-17T09:54:23.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0da7400640 1 --2- 172.21.15.32:0/3158072434 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0d84068380 0x7f0d8406a840 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0d980af000 tx=0x7f0d980a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:23.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0db199a000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55611bb91b50 0x55611bb91f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0db199a000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb92470 0x55611bb477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0db199a000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55611bb47cf0 0x55611bb480d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0db199a000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55611b9954d0 con 0x55611bb92470 2024-04-17T09:54:23.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0db199a000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55611b92fbf0 con 0x55611bb47cf0 2024-04-17T09:54:23.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0db199a000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55611b991490 con 0x55611bb91b50 2024-04-17T09:54:23.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0d9ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb92470 0x55611bb477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0d9ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb92470 0x55611bb477b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51100/0 (socket says 172.21.15.32:51100) 2024-04-17T09:54:23.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0d9ea00640 1 -- 172.21.15.32:0/2647246102 learned_addr learned my addr 172.21.15.32:0/2647246102 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:23.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0da7e00640 1 --2- 172.21.15.32:0/2647246102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55611bb47cf0 0x55611bb480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0da7400640 1 --2- 172.21.15.32:0/2647246102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55611bb91b50 0x55611bb91f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0d9ea00640 1 -- 172.21.15.32:0/2647246102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55611bb91b50 msgr2=0x55611bb91f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:23.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0d9ea00640 1 --2- 172.21.15.32:0/2647246102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55611bb91b50 0x55611bb91f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0d9ea00640 1 -- 172.21.15.32:0/2647246102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55611bb47cf0 msgr2=0x55611bb480d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:23.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0d9ea00640 1 --2- 172.21.15.32:0/2647246102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55611bb47cf0 0x55611bb480d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0d9ea00640 1 -- 172.21.15.32:0/2647246102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55611b91be80 con 0x55611bb92470 2024-04-17T09:54:23.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0da7e00640 1 --2- 172.21.15.32:0/2647246102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55611bb47cf0 0x55611bb480d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:54:23.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0da7400640 1 --2- 172.21.15.32:0/2647246102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55611bb91b50 0x55611bb91f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:54:23.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0d9ea00640 1 --2- 172.21.15.32:0/2647246102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb92470 0x55611bb477b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0d90052080 tx=0x7f0d90055d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f035ce4afa5094 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:23.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0d97e00640 1 -- 172.21.15.32:0/2647246102 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90052290 con 0x55611bb92470 2024-04-17T09:54:23.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0d97e00640 1 -- 172.21.15.32:0/2647246102 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d900074b0 con 0x55611bb92470 2024-04-17T09:54:23.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0d97e00640 1 -- 172.21.15.32:0/2647246102 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9007db40 con 0x55611bb92470 2024-04-17T09:54:23.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0db199a000 1 -- 172.21.15.32:0/2647246102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb92470 msgr2=0x55611bb477b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:23.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0db199a000 1 --2- 172.21.15.32:0/2647246102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb92470 0x55611bb477b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0d90052080 tx=0x7f0d90055d60 comp rx=0 tx=0).stop 2024-04-17T09:54:23.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0db199a000 1 -- 172.21.15.32:0/2647246102 shutdown_connections 2024-04-17T09:54:23.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0db199a000 1 --2- 172.21.15.32:0/2647246102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55611bb47cf0 0x55611bb480d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0db199a000 1 --2- 172.21.15.32:0/2647246102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb92470 0x55611bb477b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0db199a000 1 --2- 172.21.15.32:0/2647246102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55611bb91b50 0x55611bb91f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0db199a000 1 -- 172.21.15.32:0/2647246102 >> 172.21.15.32:0/2647246102 conn(0x55611bb4d470 msgr2=0x55611bb46090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:23.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0db199a000 1 -- 172.21.15.32:0/2647246102 shutdown_connections 2024-04-17T09:54:23.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0db199a000 1 -- 172.21.15.32:0/2647246102 wait complete. 2024-04-17T09:54:23.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0db199a000 1 -- start start 2024-04-17T09:54:23.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.277+0000 7f0db199a000 1 Processor -- start 2024-04-17T09:54:23.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0db199a000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55611bb47cf0 0x55611bb4c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0db199a000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55611bb91b50 0x55611bb7f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0db199a000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb92470 0x55611bb7f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0db199a000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55611b952a60 con 0x55611bb92470 2024-04-17T09:54:23.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0db199a000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55611b939e80 con 0x55611bb47cf0 2024-04-17T09:54:23.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0db199a000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55611b950cf0 con 0x55611bb91b50 2024-04-17T09:54:23.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0db199a000 10 client.0 _ll_register_callbacks cb 0x55611bb32a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:54:23.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0da7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb92470 0x55611bb7f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0da7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb92470 0x55611bb7f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51116/0 (socket says 172.21.15.32:51116) 2024-04-17T09:54:23.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0da7e00640 1 -- 172.21.15.32:0/4111082194 learned_addr learned my addr 172.21.15.32:0/4111082194 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:23.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0da7400640 1 --2- 172.21.15.32:0/4111082194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55611bb47cf0 0x55611bb4c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0d9ea00640 1 --2- 172.21.15.32:0/4111082194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55611bb91b50 0x55611bb7f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0da7e00640 1 -- 172.21.15.32:0/4111082194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55611bb91b50 msgr2=0x55611bb7f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:23.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0da7e00640 1 --2- 172.21.15.32:0/4111082194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55611bb91b50 0x55611bb7f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0da7e00640 1 -- 172.21.15.32:0/4111082194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55611bb47cf0 msgr2=0x55611bb4c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:23.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0da7e00640 1 --2- 172.21.15.32:0/4111082194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55611bb47cf0 0x55611bb4c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0da7e00640 1 -- 172.21.15.32:0/4111082194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55611b92fe40 con 0x55611bb92470 2024-04-17T09:54:23.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0da7e00640 1 --2- 172.21.15.32:0/4111082194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb92470 0x55611bb7f980 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0da00055f0 tx=0x7f0da000f600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b6863cf7745d95 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:23.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0d94c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:54:23.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0d94c00640 1 -- 172.21.15.32:0/4111082194 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0018020 con 0x55611bb92470 2024-04-17T09:54:23.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0d94c00640 1 -- 172.21.15.32:0/4111082194 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f0da0014410 con 0x55611bb92470 2024-04-17T09:54:23.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0db199a000 1 -- 172.21.15.32:0/4111082194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d900074b0 con 0x55611bb92470 2024-04-17T09:54:23.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0d94c00640 1 -- 172.21.15.32:0/4111082194 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7f0da0022050 con 0x55611bb92470 2024-04-17T09:54:23.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0db199a000 10 client.5400 fetch_fsmap learned FSMap version 15 2024-04-17T09:54:23.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0db199a000 1 -- 172.21.15.32:0/4111082194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x55611b960e00 con 0x55611bb92470 2024-04-17T09:54:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0d94c00640 1 -- 172.21.15.32:0/4111082194 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d900074b0 con 0x55611bb92470 2024-04-17T09:54:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0d94c00640 1 -- 172.21.15.32:0/4111082194 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 15) ==== 3495+0+0 (secure 0 0 0) 0x7f0da0025020 con 0x55611bb92470 2024-04-17T09:54:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0db199a000 10 client.5400 fetch_fsmap finished waiting for FSMap version 15 2024-04-17T09:54:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0db199a000 10 client.5400 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T09:54:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0db199a000 20 client.5400 populate_metadata read hostname 'smithi032' 2024-04-17T09:54:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0db199a000 1 --2- 172.21.15.32:0/4111082194 >> [v2:172.21.15.32:6835/925293029,v1:172.21.15.32:6837/925293029] conn(0x55611bba0be0 0x55611bba2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0db199a000 4 client.5400 mds_command: new command op to 5217 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:54:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0db199a000 1 -- 172.21.15.32:0/4111082194 --> [v2:172.21.15.32:6835/925293029,v1:172.21.15.32:6837/925293029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55611b92ebc0 con 0x55611bba0be0 2024-04-17T09:54:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0da7400640 1 --2- 172.21.15.32:0/4111082194 >> [v2:172.21.15.32:6835/925293029,v1:172.21.15.32:6837/925293029] conn(0x55611bba0be0 0x55611bba2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0da7400640 1 --2- 172.21.15.32:0/4111082194 >> [v2:172.21.15.32:6835/925293029,v1:172.21.15.32:6837/925293029] conn(0x55611bba0be0 0x55611bba2fd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15a41d96bc952a80 server_cookie=e5d0616810181dee in_seq=0 out_seq=0 2024-04-17T09:54:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.281+0000 7f0d94c00640 10 client.5400 ms_handle_connect on v2:172.21.15.32:6835/925293029 2024-04-17T09:54:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.285+0000 7f0d94c00640 1 -- 172.21.15.32:0/4111082194 <== mds.0 v2:172.21.15.32:6835/925293029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55611b92ebc0 con 0x55611bba0be0 2024-04-17T09:54:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.285+0000 7f0d94c00640 10 client.5400 handle_command_reply: tid=0 2024-04-17T09:54:23.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.285+0000 7f0db199a000 1 client.5400 shutdown 2024-04-17T09:54:23.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.285+0000 7f0db199a000 1 -- 172.21.15.32:0/4111082194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb92470 msgr2=0x55611bb7f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:23.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.285+0000 7f0db199a000 1 --2- 172.21.15.32:0/4111082194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb92470 0x55611bb7f980 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0da00055f0 tx=0x7f0da000f600 comp rx=0 tx=0).stop 2024-04-17T09:54:23.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.285+0000 7f0db199a000 1 -- 172.21.15.32:0/4111082194 shutdown_connections 2024-04-17T09:54:23.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.285+0000 7f0db199a000 1 --2- 172.21.15.32:0/4111082194 >> [v2:172.21.15.32:6835/925293029,v1:172.21.15.32:6837/925293029] conn(0x55611bba0be0 0x55611bba2fd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.285+0000 7f0db199a000 1 --2- 172.21.15.32:0/4111082194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55611bb91b50 0x55611bb7f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.285+0000 7f0db199a000 1 --2- 172.21.15.32:0/4111082194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb92470 0x55611bb7f980 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.285+0000 7f0d94c00640 0 client.5400 ms_handle_reset on v2:172.21.15.32:6835/925293029 2024-04-17T09:54:23.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.285+0000 7f0db199a000 1 --2- 172.21.15.32:0/4111082194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55611bb47cf0 0x55611bb4c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.285+0000 7f0db199a000 1 -- 172.21.15.32:0/4111082194 >> 172.21.15.32:0/4111082194 conn(0x55611bb4d470 msgr2=0x55611bb467a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:23.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.285+0000 7f0db199a000 1 -- 172.21.15.32:0/4111082194 shutdown_connections 2024-04-17T09:54:23.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.285+0000 7f0db199a000 1 -- 172.21.15.32:0/4111082194 wait complete. 2024-04-17T09:54:23.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.285+0000 7f0db199a000 20 client.5400 trim_cache size 0 max 16384 2024-04-17T09:54:23.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.301+0000 7f0db199a000 1 Processor -- start 2024-04-17T09:54:23.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.301+0000 7f0db199a000 1 -- start start 2024-04-17T09:54:23.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55611bb47cf0 0x55611bb49920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55611bb91b50 0x55611bb49e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb92470 0x55611bba2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0da8202f70 con 0x55611bb92470 2024-04-17T09:54:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0da812aad0 con 0x55611bb47cf0 2024-04-17T09:54:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0da812b120 con 0x55611bb91b50 2024-04-17T09:54:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0da7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb92470 0x55611bba2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0da7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb92470 0x55611bba2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51132/0 (socket says 172.21.15.32:51132) 2024-04-17T09:54:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0da7e00640 1 -- 172.21.15.32:0/1164728267 learned_addr learned my addr 172.21.15.32:0/1164728267 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0da7400640 1 --2- 172.21.15.32:0/1164728267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55611bb47cf0 0x55611bb49920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0d9ea00640 1 --2- 172.21.15.32:0/1164728267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55611bb91b50 0x55611bb49e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0da7e00640 1 -- 172.21.15.32:0/1164728267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55611bb91b50 msgr2=0x55611bb49e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0da7e00640 1 --2- 172.21.15.32:0/1164728267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55611bb91b50 0x55611bb49e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0da7e00640 1 -- 172.21.15.32:0/1164728267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55611bb47cf0 msgr2=0x55611bb49920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0da7e00640 1 --2- 172.21.15.32:0/1164728267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55611bb47cf0 0x55611bb49920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0da7e00640 1 -- 172.21.15.32:0/1164728267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55611ba16d30 con 0x55611bb92470 2024-04-17T09:54:23.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0da7e00640 1 --2- 172.21.15.32:0/1164728267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb92470 0x55611bba2fd0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0da0004eb0 tx=0x7f0da0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515363c7ae2b482f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:23.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0d97400640 1 -- 172.21.15.32:0/1164728267 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0018070 con 0x55611bb92470 2024-04-17T09:54:23.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0d97400640 1 -- 172.21.15.32:0/1164728267 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da0000e50 con 0x55611bb92470 2024-04-17T09:54:23.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0d97400640 1 -- 172.21.15.32:0/1164728267 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0010040 con 0x55611bb92470 2024-04-17T09:54:23.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 1 -- 172.21.15.32:0/1164728267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb92470 msgr2=0x55611bba2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:23.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 1 --2- 172.21.15.32:0/1164728267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb92470 0x55611bba2fd0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0da0004eb0 tx=0x7f0da0014000 comp rx=0 tx=0).stop 2024-04-17T09:54:23.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 1 -- 172.21.15.32:0/1164728267 shutdown_connections 2024-04-17T09:54:23.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 1 --2- 172.21.15.32:0/1164728267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55611bb91b50 0x55611bb49e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 1 --2- 172.21.15.32:0/1164728267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb92470 0x55611bba2fd0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 1 --2- 172.21.15.32:0/1164728267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55611bb47cf0 0x55611bb49920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 1 -- 172.21.15.32:0/1164728267 >> 172.21.15.32:0/1164728267 conn(0x55611bb4d470 msgr2=0x55611bb49210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:23.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 1 -- 172.21.15.32:0/1164728267 shutdown_connections 2024-04-17T09:54:23.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 1 -- 172.21.15.32:0/1164728267 wait complete. 2024-04-17T09:54:23.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 1 -- start start 2024-04-17T09:54:23.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 1 Processor -- start 2024-04-17T09:54:23.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55611bb47cf0 0x55611bba5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb91b50 0x55611bbbcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55611bb92470 0x55611bbbd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55611b999b00 con 0x55611bb91b50 2024-04-17T09:54:23.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55611b9947e0 con 0x55611bb92470 2024-04-17T09:54:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55611b9a26e0 con 0x55611bb47cf0 2024-04-17T09:54:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0db199a000 10 client.0 _ll_register_callbacks cb 0x7f0da0007de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:54:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.305+0000 7f0d9ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb91b50 0x55611bbbcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0d9ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb91b50 0x55611bbbcb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51148/0 (socket says 172.21.15.32:51148) 2024-04-17T09:54:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0d9ea00640 1 -- 172.21.15.32:0/3903655079 learned_addr learned my addr 172.21.15.32:0/3903655079 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0da7400640 1 --2- 172.21.15.32:0/3903655079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55611bb47cf0 0x55611bba5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0da7e00640 1 --2- 172.21.15.32:0/3903655079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55611bb92470 0x55611bbbd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0d9ea00640 1 -- 172.21.15.32:0/3903655079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55611bb47cf0 msgr2=0x55611bba5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0d9ea00640 1 --2- 172.21.15.32:0/3903655079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55611bb47cf0 0x55611bba5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0d9ea00640 1 -- 172.21.15.32:0/3903655079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55611bb92470 msgr2=0x55611bbbd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0d9ea00640 1 --2- 172.21.15.32:0/3903655079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55611bb92470 0x55611bbbd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0d9ea00640 1 -- 172.21.15.32:0/3903655079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55611b9277c0 con 0x55611bb91b50 2024-04-17T09:54:23.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0da7400640 1 --2- 172.21.15.32:0/3903655079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55611bb47cf0 0x55611bba5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:23.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0d9ea00640 1 --2- 172.21.15.32:0/3903655079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb91b50 0x55611bbbcb60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0d90052080 tx=0x7f0d90000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4fba6b3970ed138 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:23.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0d94c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:54:23.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0d94c00640 1 -- 172.21.15.32:0/3903655079 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90073070 con 0x55611bb91b50 2024-04-17T09:54:23.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0d94c00640 1 -- 172.21.15.32:0/3903655079 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f0d900076c0 con 0x55611bb91b50 2024-04-17T09:54:23.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0db199a000 1 -- 172.21.15.32:0/3903655079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0da0000e50 con 0x55611bb91b50 2024-04-17T09:54:23.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0d94c00640 1 -- 172.21.15.32:0/3903655079 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7f0d9007a050 con 0x55611bb91b50 2024-04-17T09:54:23.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0db199a000 10 client.5406 fetch_fsmap learned FSMap version 15 2024-04-17T09:54:23.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0db199a000 1 -- 172.21.15.32:0/3903655079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x55611b93a650 con 0x55611bb91b50 2024-04-17T09:54:23.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0d94c00640 1 -- 172.21.15.32:0/3903655079 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da0000e50 con 0x55611bb91b50 2024-04-17T09:54:23.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0d94c00640 1 -- 172.21.15.32:0/3903655079 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 15) ==== 3495+0+0 (secure 0 0 0) 0x7f0d9007d020 con 0x55611bb91b50 2024-04-17T09:54:23.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0db199a000 10 client.5406 fetch_fsmap finished waiting for FSMap version 15 2024-04-17T09:54:23.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0db199a000 10 client.5406 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T09:54:23.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0db199a000 20 client.5406 populate_metadata read hostname 'smithi032' 2024-04-17T09:54:23.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0db199a000 1 --2- 172.21.15.32:0/3903655079 >> [v2:172.21.15.32:6835/925293029,v1:172.21.15.32:6837/925293029] conn(0x55611bc007d0 0x55611bc00bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0db199a000 4 client.5406 mds_command: new command op to 5217 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T09:54:23.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0db199a000 1 -- 172.21.15.32:0/3903655079 --> [v2:172.21.15.32:6835/925293029,v1:172.21.15.32:6837/925293029] -- command(tid 0: {"prefix": "status"}) -- 0x55611b9a2ef0 con 0x55611bc007d0 2024-04-17T09:54:23.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0da7400640 1 --2- 172.21.15.32:0/3903655079 >> [v2:172.21.15.32:6835/925293029,v1:172.21.15.32:6837/925293029] conn(0x55611bc007d0 0x55611bc00bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0da7400640 1 --2- 172.21.15.32:0/3903655079 >> [v2:172.21.15.32:6835/925293029,v1:172.21.15.32:6837/925293029] conn(0x55611bc007d0 0x55611bc00bb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23a58377801ecee8 server_cookie=5fdc3719e5dba471 in_seq=0 out_seq=0 2024-04-17T09:54:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0d94c00640 10 client.5406 ms_handle_connect on v2:172.21.15.32:6835/925293029 2024-04-17T09:54:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0d94c00640 1 -- 172.21.15.32:0/3903655079 <== mds.0 v2:172.21.15.32:6835/925293029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55611b9a2ef0 con 0x55611bc007d0 2024-04-17T09:54:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0d94c00640 10 client.5406 handle_command_reply: tid=0 2024-04-17T09:54:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0db199a000 1 client.5406 shutdown 2024-04-17T09:54:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0db199a000 1 -- 172.21.15.32:0/3903655079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb91b50 msgr2=0x55611bbbcb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0db199a000 1 --2- 172.21.15.32:0/3903655079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb91b50 0x55611bbbcb60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0d90052080 tx=0x7f0d90000bd0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0db199a000 1 -- 172.21.15.32:0/3903655079 shutdown_connections 2024-04-17T09:54:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0db199a000 1 --2- 172.21.15.32:0/3903655079 >> [v2:172.21.15.32:6835/925293029,v1:172.21.15.32:6837/925293029] conn(0x55611bc007d0 0x55611bc00bb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0db199a000 1 --2- 172.21.15.32:0/3903655079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55611bb92470 0x55611bbbd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0db199a000 1 --2- 172.21.15.32:0/3903655079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55611bb91b50 0x55611bbbcb60 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0db199a000 1 --2- 172.21.15.32:0/3903655079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55611bb47cf0 0x55611bba5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0d94c00640 0 client.5406 ms_handle_reset on v2:172.21.15.32:6835/925293029 2024-04-17T09:54:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0db199a000 1 -- 172.21.15.32:0/3903655079 >> 172.21.15.32:0/3903655079 conn(0x55611bb4d470 msgr2=0x55611bb49920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0db199a000 1 -- 172.21.15.32:0/3903655079 shutdown_connections 2024-04-17T09:54:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0db199a000 1 -- 172.21.15.32:0/3903655079 wait complete. 2024-04-17T09:54:23.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0db199a000 20 client.5406 trim_cache size 0 max 16384 2024-04-17T09:54:23.315 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:54:23.315 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "ff178df8-4c06-4367-a1e9-2e3a66d870d0", 2024-04-17T09:54:23.315 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-17T09:54:23.315 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 5217, 2024-04-17T09:54:23.315 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-17T09:54:23.315 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-17T09:54:23.315 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-17T09:54:23.315 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.9445471650000004, 2024-04-17T09:54:23.315 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 15, 2024-04-17T09:54:23.315 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 35, 2024-04-17T09:54:23.315 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 35, 2024-04-17T09:54:23.316 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 13.808940936999999 2024-04-17T09:54:23.316 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:54:23.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0dae400640 1 -- 172.21.15.32:0/3158072434 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0d84068380 msgr2=0x7f0d8406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:23.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0dae400640 1 --2- 172.21.15.32:0/3158072434 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0d84068380 0x7f0d8406a840 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0d980af000 tx=0x7f0d980a7070 comp rx=0 tx=0).stop 2024-04-17T09:54:23.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0dae400640 1 -- 172.21.15.32:0/3158072434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da80d09f0 msgr2=0x7f0da80b19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:23.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0dae400640 1 --2- 172.21.15.32:0/3158072434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da80d09f0 0x7f0da80b19c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000c430 tx=0x7f0d9000c8f0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0dae400640 1 -- 172.21.15.32:0/3158072434 shutdown_connections 2024-04-17T09:54:23.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0dae400640 1 --2- 172.21.15.32:0/3158072434 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0d84068380 0x7f0d8406a840 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0dae400640 1 --2- 172.21.15.32:0/3158072434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0da813f6e0 0x7f0da80b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0dae400640 1 --2- 172.21.15.32:0/3158072434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0da80d09f0 0x7f0da80b19c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0dae400640 1 --2- 172.21.15.32:0/3158072434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0da80cf370 0x7f0da815a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0dae400640 1 -- 172.21.15.32:0/3158072434 >> 172.21.15.32:0/3158072434 conn(0x7f0da806ba30 msgr2=0x7f0da815f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:23.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0dae400640 1 -- 172.21.15.32:0/3158072434 shutdown_connections 2024-04-17T09:54:23.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.309+0000 7f0dae400640 1 -- 172.21.15.32:0/3158072434 wait complete. 2024-04-17T09:54:23.331 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T09:54:23.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.473+0000 7f9d1d200640 1 Processor -- start 2024-04-17T09:54:23.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.473+0000 7f9d1d200640 1 -- start start 2024-04-17T09:54:23.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.473+0000 7f9d1d200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d1815eb30 0x7f9d1809b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.473+0000 7f9d1d200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d1809b100 0x7f9d180ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.473+0000 7f9d1d200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d1809bcb0 con 0x7f9d180d0890 2024-04-17T09:54:23.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.473+0000 7f9d1d200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9d1806dd60 con 0x7f9d1815eb30 2024-04-17T09:54:23.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.473+0000 7f9d1d200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9d1815ef10 con 0x7f9d1809b100 2024-04-17T09:54:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.473+0000 7f9d15e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d1815eb30 0x7f9d1809b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.473+0000 7f9d0d400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9d180d0890 0x7f9d1809a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37140/0 (socket says 172.21.15.32:37140) 2024-04-17T09:54:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.473+0000 7f9d0d400640 1 -- 172.21.15.32:0/339653159 learned_addr learned my addr 172.21.15.32:0/339653159 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.473+0000 7f9d16800640 1 --2- 172.21.15.32:0/339653159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d1809b100 0x7f9d180ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.473+0000 7f9d15e00640 1 -- 172.21.15.32:0/339653159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d1809b100 msgr2=0x7f9d180ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.473+0000 7f9d15e00640 1 --2- 172.21.15.32:0/339653159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d1809b100 0x7f9d180ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.473+0000 7f9d15e00640 1 -- 172.21.15.32:0/339653159 >> v1:172.21.15.32:6789/0 conn(0x7f9d180d0890 legacy=0x7f9d1809a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.473+0000 7f9d15e00640 1 -- 172.21.15.32:0/339653159 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d180d0db0 con 0x7f9d1815eb30 2024-04-17T09:54:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.473+0000 7f9d15400640 1 -- 172.21.15.32:0/339653159 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1142980390 0 0) 0x7f9d1809bcb0 con 0x7f9d180d0890 2024-04-17T09:54:23.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.477+0000 7f9d15e00640 1 --2- 172.21.15.32:0/339653159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d1815eb30 0x7f9d1809b770 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9d080097c0 tx=0x7f9d0801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9f20c1dfc37d82 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:23.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.477+0000 7f9d15400640 1 -- 172.21.15.32:0/339653159 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d080b5070 con 0x7f9d1815eb30 2024-04-17T09:54:23.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.477+0000 7f9d15400640 1 -- 172.21.15.32:0/339653159 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d080a8990 con 0x7f9d1815eb30 2024-04-17T09:54:23.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.477+0000 7f9d15400640 1 -- 172.21.15.32:0/339653159 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d080a8d60 con 0x7f9d1815eb30 2024-04-17T09:54:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.477+0000 7f9d1d200640 1 -- 172.21.15.32:0/339653159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d1815eb30 msgr2=0x7f9d1809b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.477+0000 7f9d1d200640 1 --2- 172.21.15.32:0/339653159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d1815eb30 0x7f9d1809b770 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9d080097c0 tx=0x7f9d0801ec90 comp rx=0 tx=0).stop 2024-04-17T09:54:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.477+0000 7f9d1d200640 1 -- 172.21.15.32:0/339653159 shutdown_connections 2024-04-17T09:54:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.477+0000 7f9d1d200640 1 --2- 172.21.15.32:0/339653159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d1809b100 0x7f9d180ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.477+0000 7f9d1d200640 1 --2- 172.21.15.32:0/339653159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d1815eb30 0x7f9d1809b770 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.477+0000 7f9d1d200640 1 -- 172.21.15.32:0/339653159 >> 172.21.15.32:0/339653159 conn(0x7f9d180b3fd0 msgr2=0x7f9d180b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.477+0000 7f9d1d200640 1 -- 172.21.15.32:0/339653159 shutdown_connections 2024-04-17T09:54:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.477+0000 7f9d1d200640 1 -- 172.21.15.32:0/339653159 wait complete. 2024-04-17T09:54:23.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d1d200640 1 Processor -- start 2024-04-17T09:54:23.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d1d200640 1 -- start start 2024-04-17T09:54:23.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d1d200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d1809b100 0x7f9d180ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d1d200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d180d0890 0x7f9d180cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d1d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d1815eb30 0x7f9d18155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d1d200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9d180d1650 con 0x7f9d1815eb30 2024-04-17T09:54:23.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d1d200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9d180d1370 con 0x7f9d1809b100 2024-04-17T09:54:23.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d1d200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9d180d1930 con 0x7f9d180d0890 2024-04-17T09:54:23.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d16800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d1815eb30 0x7f9d18155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d16800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d1815eb30 0x7f9d18155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51168/0 (socket says 172.21.15.32:51168) 2024-04-17T09:54:23.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d16800640 1 -- 172.21.15.32:0/3952031239 learned_addr learned my addr 172.21.15.32:0/3952031239 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:23.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d15e00640 1 --2- 172.21.15.32:0/3952031239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d1809b100 0x7f9d180ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d0d400640 1 --2- 172.21.15.32:0/3952031239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d180d0890 0x7f9d180cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d16800640 1 -- 172.21.15.32:0/3952031239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d180d0890 msgr2=0x7f9d180cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:23.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d16800640 1 --2- 172.21.15.32:0/3952031239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d180d0890 0x7f9d180cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d16800640 1 -- 172.21.15.32:0/3952031239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d1809b100 msgr2=0x7f9d180ca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:23.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d16800640 1 --2- 172.21.15.32:0/3952031239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d1809b100 0x7f9d180ca5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d16800640 1 -- 172.21.15.32:0/3952031239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d180cab90 con 0x7f9d1815eb30 2024-04-17T09:54:23.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d16800640 1 --2- 172.21.15.32:0/3952031239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d1815eb30 0x7f9d18155ba0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000efc0 tx=0x7f9d1000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d0ea00640 1 -- 172.21.15.32:0/3952031239 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d10018020 con 0x7f9d1815eb30 2024-04-17T09:54:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d0ea00640 1 -- 172.21.15.32:0/3952031239 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d10014480 con 0x7f9d1815eb30 2024-04-17T09:54:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d1d200640 1 -- 172.21.15.32:0/3952031239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d180cae70 con 0x7f9d1815eb30 2024-04-17T09:54:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d0ea00640 1 -- 172.21.15.32:0/3952031239 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d10014870 con 0x7f9d1815eb30 2024-04-17T09:54:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.481+0000 7f9d1d200640 1 -- 172.21.15.32:0/3952031239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d180cb430 con 0x7f9d1815eb30 2024-04-17T09:54:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.485+0000 7f9d1d200640 1 -- 172.21.15.32:0/3952031239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d18068a90 con 0x7f9d1815eb30 2024-04-17T09:54:23.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.485+0000 7f9d0ea00640 1 -- 172.21.15.32:0/3952031239 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9d1001b020 con 0x7f9d1815eb30 2024-04-17T09:54:23.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.485+0000 7f9d0ea00640 1 --2- 172.21.15.32:0/3952031239 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9cf40683b0 0x7f9cf406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:23.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.485+0000 7f9d0ea00640 1 -- 172.21.15.32:0/3952031239 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f9d10087770 con 0x7f9d1815eb30 2024-04-17T09:54:23.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.485+0000 7f9d15e00640 1 --2- 172.21.15.32:0/3952031239 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9cf40683b0 0x7f9cf406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:23.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.485+0000 7f9d15e00640 1 --2- 172.21.15.32:0/3952031239 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9cf40683b0 0x7f9cf406a870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9d080b2000 tx=0x7f9d08008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:23.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.489+0000 7f9d0ea00640 1 -- 172.21.15.32:0/3952031239 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f9d10022030 con 0x7f9d1815eb30 2024-04-17T09:54:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.689+0000 7f9d1d200640 1 -- 172.21.15.32:0/3952031239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9d180ce9a0 con 0x7f9d1815eb30 2024-04-17T09:54:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.693+0000 7f9d0ea00640 1 -- 172.21.15.32:0/3952031239 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) ==== 53+0+73 (secure 0 0 0) 0x7f9d10010040 con 0x7f9d1815eb30 2024-04-17T09:54:23.693 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T09:54:23.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.693+0000 7f9d1d200640 1 -- 172.21.15.32:0/3952031239 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9cf40683b0 msgr2=0x7f9cf406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:23.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.693+0000 7f9d1d200640 1 --2- 172.21.15.32:0/3952031239 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9cf40683b0 0x7f9cf406a870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9d080b2000 tx=0x7f9d08008040 comp rx=0 tx=0).stop 2024-04-17T09:54:23.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.693+0000 7f9d1d200640 1 -- 172.21.15.32:0/3952031239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d1815eb30 msgr2=0x7f9d18155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:23.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.693+0000 7f9d1d200640 1 --2- 172.21.15.32:0/3952031239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d1815eb30 0x7f9d18155ba0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000efc0 tx=0x7f9d1000c490 comp rx=0 tx=0).stop 2024-04-17T09:54:23.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.693+0000 7f9d1d200640 1 -- 172.21.15.32:0/3952031239 shutdown_connections 2024-04-17T09:54:23.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.693+0000 7f9d1d200640 1 --2- 172.21.15.32:0/3952031239 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9cf40683b0 0x7f9cf406a870 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.693+0000 7f9d1d200640 1 --2- 172.21.15.32:0/3952031239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d180d0890 0x7f9d180cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.693+0000 7f9d1d200640 1 --2- 172.21.15.32:0/3952031239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d1815eb30 0x7f9d18155ba0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.693+0000 7f9d1d200640 1 --2- 172.21.15.32:0/3952031239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d1809b100 0x7f9d180ca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:23.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.693+0000 7f9d1d200640 1 -- 172.21.15.32:0/3952031239 >> 172.21.15.32:0/3952031239 conn(0x7f9d180b3fd0 msgr2=0x7f9d1806cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:23.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.693+0000 7f9d1d200640 1 -- 172.21.15.32:0/3952031239 shutdown_connections 2024-04-17T09:54:23.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23.693+0000 7f9d1d200640 1 -- 172.21.15.32:0/3952031239 wait complete. 2024-04-17T09:54:23.713 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T09:54:23.713 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T09:54:23.713 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-17T09:54:23.713 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T09:54:23.714 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T09:54:23.714 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T09:54:23.714 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T09:54:23.714 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T09:54:23.714 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:54:23.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:23.714 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-17T09:54:23.723 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T09:54:23.723 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T09:54:23.723 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T09:54:23.723 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:54:23.723 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-17T09:54:23.723 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:54:23.723 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:54:23.723 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-17T09:54:23.724 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-17T09:54:23.724 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T09:54:23.724 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:54:23.724 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-17T09:54:23.724 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-17T09:54:23.724 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42390sec preferred_lft 42390sec 2024-04-17T09:54:23.724 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2024-04-17T09:54:23.724 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:54:23.724 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:54:23.724 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-17T09:54:23.724 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-17T09:54:23.724 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:54:23.724 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:54:23.724 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-17T09:54:23.724 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T09:54:23.725 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:cd:f4:aa:5b brd ff:ff:ff:ff:ff:ff 2024-04-17T09:54:23.725 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T09:54:23.725 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:54:23.725 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T09:54:23.725 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T09:54:23.725 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T09:54:23.725 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:54:23.725 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T09:54:23.725 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:54:23.725 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T09:54:23.725 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:54:23.725 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T09:54:23.725 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:54:23.725 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T09:54:23.725 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:54:23.726 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T09:54:23.726 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:54:23.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:23.727 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T09:54:23.786 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-17T09:54:23.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:23.788 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-17T09:54:23.841 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-17T09:54:23.842 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T09:54:23.842 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-17T09:54:23.842 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T09:54:23.842 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-17T09:54:23.842 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T09:54:23.842 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:54:23.842 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T09:54:23.842 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-17T09:54:23.842 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-17T09:54:23.842 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-17T09:54:23.842 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T09:54:23.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:54:23.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:54:23.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:23.980 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-17T09:54:24.036 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T09:54:24.036 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T09:54:24.037 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:54:24.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:24.037 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-17T09:54:24.100 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T09:54:24.100 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:54:24.100 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T09:54:24.152 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:54:24.152 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:54:24.152 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T09:54:24.203 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T09:54:24.265 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:54:24.322 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T09:54:24.322 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T09:54:24.322 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T09:54:24.322 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T09:54:24.322 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T09:54:24.322 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T09:54:24.322 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T09:54:24.322 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T09:54:24.322 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T09:54:24.322 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T09:54:24.322 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T09:54:24.322 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T09:54:24.323 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:54:24.323 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T09:54:24.323 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T09:54:24.323 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T09:54:24.323 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T09:54:24.323 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T09:54:24.323 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T09:54:24.323 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T09:54:24.323 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:54:24.323 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T09:54:24.323 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T09:54:24.323 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T09:54:24.323 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T09:54:24.323 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T09:54:24.323 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T09:54:24.323 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T09:54:24.324 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T09:54:24.324 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:54:24.324 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:54:24.324 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:54:24.324 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:54:24.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:24.325 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T09:54:24.380 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T09:54:24.380 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T09:54:24.427 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T09:54:24.449 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:54:24.505 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T09:54:24.506 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T09:54:24.506 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T09:54:24.506 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T09:54:24.506 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T09:54:24.506 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T09:54:24.506 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T09:54:24.506 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T09:54:24.506 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T09:54:24.506 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T09:54:24.506 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T09:54:24.506 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T09:54:24.506 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:54:24.506 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T09:54:24.506 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T09:54:24.506 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T09:54:24.507 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T09:54:24.507 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T09:54:24.507 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T09:54:24.507 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T09:54:24.507 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:54:24.507 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T09:54:24.507 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T09:54:24.507 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T09:54:24.507 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T09:54:24.507 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T09:54:24.507 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T09:54:24.507 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T09:54:24.507 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T09:54:24.507 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:54:24.507 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:54:24.507 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:54:24.508 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:54:24.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:24.508 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T09:54:24.522 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2024-04-17T09:54:24.521+0000 7f9f2ef55580 -1 init, newargv = 0x5564c8593c30 newargc=15 2024-04-17T09:54:24.522 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[36429]: starting ceph client 2024-04-17T09:54:24.529 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[36429]: starting fuse 2024-04-17T09:54:24.532 INFO:teuthology.orchestra.run.smithi032.stdout:44 2024-04-17T09:54:24.532 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-17T09:54:24.532 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2024-04-17T09:54:24.532 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2024-04-17T09:54:24.532 DEBUG:teuthology.orchestra.run.smithi032:> import re 2024-04-17T09:54:24.533 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T09:54:24.533 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2024-04-17T09:54:24.533 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:54:24.533 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2024-04-17T09:54:24.533 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T09:54:24.533 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2024-04-17T09:54:24.533 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T09:54:24.533 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:54:24.533 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2024-04-17T09:54:24.533 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2024-04-17T09:54:24.533 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2024-04-17T09:54:24.533 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2024-04-17T09:54:24.533 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:54:24.533 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2024-04-17T09:54:24.533 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T09:54:24.533 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T09:54:24.533 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T09:54:24.534 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2024-04-17T09:54:24.534 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2024-04-17T09:54:24.534 DEBUG:teuthology.orchestra.run.smithi032:> return f 2024-04-17T09:54:24.534 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T09:54:24.534 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:54:24.534 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2024-04-17T09:54:24.534 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T09:54:24.658 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.36429.asok 2024-04-17T09:54:24.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:54:24.669 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36429.asok 2024-04-17T09:54:24.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:24.669 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36429.asok status 2024-04-17T09:54:24.844 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:54:24.844 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2024-04-17T09:54:24.844 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-17T09:54:24.844 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T09:54:24.844 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2024-04-17T09:54:24.845 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2024-04-17T09:54:24.845 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T09:54:24.845 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "36429", 2024-04-17T09:54:24.845 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2024-04-17T09:54:24.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:24.845 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2024-04-17T09:54:24.845 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2024-04-17T09:54:24.845 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 5424, 2024-04-17T09:54:24.845 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2024-04-17T09:54:24.845 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2024-04-17T09:54:24.845 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2024-04-17T09:54:24.845 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 5424 2024-04-17T09:54:24.845 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:24.845 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T09:54:24.845 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T09:54:24.846 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T09:54:24.846 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3917254433 2024-04-17T09:54:24.846 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:54:24.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:24.846 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T09:54:24.846 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T09:54:24.846 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T09:54:24.846 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3917254433 2024-04-17T09:54:24.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:54:24.846 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.5424 192.168.0.1:0/3917254433", 2024-04-17T09:54:24.846 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/3917254433", 2024-04-17T09:54:24.846 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2024-04-17T09:54:24.846 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 15, 2024-04-17T09:54:24.846 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 35, 2024-04-17T09:54:24.847 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2024-04-17T09:54:24.847 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2024-04-17T09:54:24.847 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs" 2024-04-17T09:54:24.847 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:54:24.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:24.859 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2024-04-17T09:54:24.859 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T09:54:24.868 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2024-04-17T09:54:24.868 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T09:54:24.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:24.868 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T09:54:24.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:24.932 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-04-17T09:54:24.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:24.992 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-04-17T09:54:25.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:25.048 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-04-17T09:54:25.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:25.104 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-04-17T09:54:25.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:25.163 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-04-17T09:54:25.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:25.219 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-17T09:54:25.285 INFO:teuthology.orchestra.run.smithi032.stdout:./ 2024-04-17T09:54:25.285 INFO:teuthology.orchestra.run.smithi032.stdout:./rfile 2024-04-17T09:54:25.285 INFO:teuthology.orchestra.run.smithi032.stdout:./parentdir 2024-04-17T09:54:25.285 INFO:teuthology.orchestra.run.smithi032.stdout:./parentdir/pfile 2024-04-17T09:54:25.286 INFO:teuthology.orchestra.run.smithi032.stdout:./parentdir/childdir 2024-04-17T09:54:25.286 INFO:teuthology.orchestra.run.smithi032.stdout:./parentdir/childdir/cfile 2024-04-17T09:54:25.287 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T09:54:25.287 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T09:54:25.287 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T09:54:25.287 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:54:25.287 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-17T09:54:25.287 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T09:54:25.396 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-17T09:54:25.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:54:25.406 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T09:54:25.406 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T09:54:25.406 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T09:54:25.406 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:54:25.406 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-04-17T09:54:25.406 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T09:54:25.514 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627778 2024-04-17T09:54:25.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:54:25.524 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T09:54:25.524 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T09:54:25.524 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T09:54:25.524 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:54:25.524 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-04-17T09:54:25.524 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T09:54:25.630 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627776 2024-04-17T09:54:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:54:25.640 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T09:54:25.640 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T09:54:25.640 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T09:54:25.640 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:54:25.640 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-04-17T09:54:25.640 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T09:54:25.750 INFO:teuthology.orchestra.run.smithi032.stdout:1099511628279 2024-04-17T09:54:25.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:54:25.759 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T09:54:25.759 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T09:54:25.759 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T09:54:25.759 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:54:25.759 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-04-17T09:54:25.759 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T09:54:25.866 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627777 2024-04-17T09:54:25.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:54:25.875 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T09:54:25.875 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T09:54:25.876 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T09:54:25.876 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:54:25.876 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-04-17T09:54:25.876 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T09:54:25.986 INFO:teuthology.orchestra.run.smithi032.stdout:1099511628280 2024-04-17T09:54:25.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:54:25.995 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:54:25.995 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:54:26.044 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:54:26.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.225+0000 7fecbe800640 1 Processor -- start 2024-04-17T09:54:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.225+0000 7fecbe800640 1 -- start start 2024-04-17T09:54:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.229+0000 7fecbe800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fecb80cd580 0x7fecb8167560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.229+0000 7fecbe800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecb80cdac0 con 0x7fecb8072500 2024-04-17T09:54:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.229+0000 7fecbe800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecb815cdf0 con 0x7fecb8145f30 2024-04-17T09:54:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.229+0000 7fecbe800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fecb806f400 con 0x7fecb80cd580 2024-04-17T09:54:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.229+0000 7fecb7e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fecb80cd580 0x7fecb8167560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.229+0000 7fecaea00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fecb8072500 0x7fecb80cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37154/0 (socket says 172.21.15.32:37154) 2024-04-17T09:54:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.229+0000 7fecb7e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fecb80cd580 0x7fecb8167560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51170/0 (socket says 172.21.15.32:51170) 2024-04-17T09:54:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.229+0000 7fecaea00640 1 -- 172.21.15.32:0/3325259510 learned_addr learned my addr 172.21.15.32:0/3325259510 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.229+0000 7fecb7e00640 1 -- 172.21.15.32:0/3325259510 >> v1:172.21.15.144:6789/0 conn(0x7fecb8145f30 legacy=0x7fecb8071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.229+0000 7fecb7e00640 1 -- 172.21.15.32:0/3325259510 >> v1:172.21.15.32:6789/0 conn(0x7fecb8072500 legacy=0x7fecb80cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.229+0000 7fecb7e00640 1 -- 172.21.15.32:0/3325259510 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb815d0d0 con 0x7fecb80cd580 2024-04-17T09:54:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.233+0000 7fecb7e00640 1 --2- 172.21.15.32:0/3325259510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecb80cd580 0x7fecb8167560 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fecb00097c0 tx=0x7fecb001ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561007911482c1ca server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.233+0000 7fecb6a00640 1 -- 172.21.15.32:0/3325259510 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb00b5070 con 0x7fecb80cd580 2024-04-17T09:54:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.233+0000 7fecb6a00640 1 -- 172.21.15.32:0/3325259510 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb00a8990 con 0x7fecb80cd580 2024-04-17T09:54:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.233+0000 7fecb6a00640 1 -- 172.21.15.32:0/3325259510 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb00a8d60 con 0x7fecb80cd580 2024-04-17T09:54:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.233+0000 7fecbe800640 1 -- 172.21.15.32:0/3325259510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecb80cd580 msgr2=0x7fecb8167560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.233+0000 7fecbe800640 1 --2- 172.21.15.32:0/3325259510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecb80cd580 0x7fecb8167560 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fecb00097c0 tx=0x7fecb001ec90 comp rx=0 tx=0).stop 2024-04-17T09:54:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.233+0000 7fecbe800640 1 -- 172.21.15.32:0/3325259510 shutdown_connections 2024-04-17T09:54:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.233+0000 7fecbe800640 1 --2- 172.21.15.32:0/3325259510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecb80cd580 0x7fecb8167560 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.233+0000 7fecbe800640 1 -- 172.21.15.32:0/3325259510 >> 172.21.15.32:0/3325259510 conn(0x7fecb806ba30 msgr2=0x7fecb806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.233+0000 7fecbe800640 1 -- 172.21.15.32:0/3325259510 shutdown_connections 2024-04-17T09:54:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.233+0000 7fecbe800640 1 -- 172.21.15.32:0/3325259510 wait complete. 2024-04-17T09:54:26.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.233+0000 7fecbe800640 1 Processor -- start 2024-04-17T09:54:26.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.233+0000 7fecbe800640 1 -- start start 2024-04-17T09:54:26.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.233+0000 7fecbe800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecb8072500 0x7fecb80cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:26.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.233+0000 7fecbe800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecb80cd580 0x7fecb815b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:26.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.233+0000 7fecbe800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecb8145f30 0x7fecb81f7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:26.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.233+0000 7fecbe800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fecb8146310 con 0x7fecb8072500 2024-04-17T09:54:26.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.233+0000 7fecbe800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fecb815dba0 con 0x7fecb80cd580 2024-04-17T09:54:26.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.233+0000 7fecbe800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fecb815d440 con 0x7fecb8145f30 2024-04-17T09:54:26.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.233+0000 7fecb7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecb8072500 0x7fecb80cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:26.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.233+0000 7fecb7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecb8072500 0x7fecb80cbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51184/0 (socket says 172.21.15.32:51184) 2024-04-17T09:54:26.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.233+0000 7fecb7400640 1 -- 172.21.15.32:0/742950226 learned_addr learned my addr 172.21.15.32:0/742950226 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:26.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.237+0000 7fecaea00640 1 --2- 172.21.15.32:0/742950226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecb80cd580 0x7fecb815b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:26.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.237+0000 7fecb7e00640 1 --2- 172.21.15.32:0/742950226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecb8145f30 0x7fecb81f7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:26.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.237+0000 7fecb7400640 1 -- 172.21.15.32:0/742950226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecb8145f30 msgr2=0x7fecb81f7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:26.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.237+0000 7fecb7400640 1 --2- 172.21.15.32:0/742950226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecb8145f30 0x7fecb81f7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:26.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.237+0000 7fecb7400640 1 -- 172.21.15.32:0/742950226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecb80cd580 msgr2=0x7fecb815b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:26.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.237+0000 7fecb7400640 1 --2- 172.21.15.32:0/742950226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecb80cd580 0x7fecb815b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:26.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.237+0000 7fecb7400640 1 -- 172.21.15.32:0/742950226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb81f54d0 con 0x7fecb8072500 2024-04-17T09:54:26.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.237+0000 7fecb7e00640 1 --2- 172.21.15.32:0/742950226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecb8145f30 0x7fecb81f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:26.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.237+0000 7fecb7400640 1 --2- 172.21.15.32:0/742950226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecb8072500 0x7fecb80cbce0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7feca800c730 tx=0x7feca800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:26.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.237+0000 7fecafe00640 1 -- 172.21.15.32:0/742950226 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca8016020 con 0x7fecb8072500 2024-04-17T09:54:26.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.237+0000 7fecafe00640 1 -- 172.21.15.32:0/742950226 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca8006d60 con 0x7fecb8072500 2024-04-17T09:54:26.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.237+0000 7fecafe00640 1 -- 172.21.15.32:0/742950226 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca8003bc0 con 0x7fecb8072500 2024-04-17T09:54:26.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.237+0000 7fecbe800640 1 -- 172.21.15.32:0/742950226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb80cac40 con 0x7fecb8072500 2024-04-17T09:54:26.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.237+0000 7fecbe800640 1 -- 172.21.15.32:0/742950226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb80cb200 con 0x7fecb8072500 2024-04-17T09:54:26.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.237+0000 7fecbe800640 1 -- 172.21.15.32:0/742950226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecb8068a90 con 0x7fecb8072500 2024-04-17T09:54:26.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.237+0000 7fecafe00640 1 -- 172.21.15.32:0/742950226 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7feca800a850 con 0x7fecb8072500 2024-04-17T09:54:26.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.237+0000 7fecafe00640 1 --2- 172.21.15.32:0/742950226 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fec94068380 0x7fec9406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:26.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.237+0000 7fecafe00640 1 -- 172.21.15.32:0/742950226 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7feca80866c0 con 0x7fecb8072500 2024-04-17T09:54:26.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.237+0000 7fecaea00640 1 --2- 172.21.15.32:0/742950226 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fec94068380 0x7fec9406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:26.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.241+0000 7fecaea00640 1 --2- 172.21.15.32:0/742950226 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fec94068380 0x7fec9406a840 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fecb80b5510 tx=0x7feca0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:26.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.245+0000 7fecafe00640 1 -- 172.21.15.32:0/742950226 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7feca8020020 con 0x7fecb8072500 2024-04-17T09:54:26.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.409+0000 7fecbe800640 1 -- 172.21.15.32:0/742950226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fecb8072f90 con 0x7fecb8072500 2024-04-17T09:54:26.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.413+0000 7fecafe00640 1 -- 172.21.15.32:0/742950226 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16681 (secure 0 0 0) 0x7feca8056c70 con 0x7fecb8072500 2024-04-17T09:54:26.413 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:26.413 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:54:18.351519+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:54:15.261471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-17T09:54:15.704958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:54:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.413+0000 7fecbe800640 1 -- 172.21.15.32:0/742950226 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fec94068380 msgr2=0x7fec9406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.413+0000 7fecbe800640 1 --2- 172.21.15.32:0/742950226 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fec94068380 0x7fec9406a840 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fecb80b5510 tx=0x7feca0009040 comp rx=0 tx=0).stop 2024-04-17T09:54:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.413+0000 7fecbe800640 1 -- 172.21.15.32:0/742950226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecb8072500 msgr2=0x7fecb80cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.413+0000 7fecbe800640 1 --2- 172.21.15.32:0/742950226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecb8072500 0x7fecb80cbce0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7feca800c730 tx=0x7feca800cbf0 comp rx=0 tx=0).stop 2024-04-17T09:54:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.413+0000 7fecbe800640 1 -- 172.21.15.32:0/742950226 shutdown_connections 2024-04-17T09:54:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.413+0000 7fecbe800640 1 --2- 172.21.15.32:0/742950226 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fec94068380 0x7fec9406a840 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.413+0000 7fecbe800640 1 --2- 172.21.15.32:0/742950226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecb8145f30 0x7fecb81f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.413+0000 7fecbe800640 1 --2- 172.21.15.32:0/742950226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecb80cd580 0x7fecb815b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.413+0000 7fecbe800640 1 --2- 172.21.15.32:0/742950226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecb8072500 0x7fecb80cbce0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.413+0000 7fecbe800640 1 -- 172.21.15.32:0/742950226 >> 172.21.15.32:0/742950226 conn(0x7fecb806ba30 msgr2=0x7fecb8071df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.413+0000 7fecbe800640 1 -- 172.21.15.32:0/742950226 shutdown_connections 2024-04-17T09:54:26.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.413+0000 7fecbe800640 1 -- 172.21.15.32:0/742950226 wait complete. 2024-04-17T09:54:26.435 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:54:15.261471+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:54:15.704958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:54:26.435 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:54:26.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.577+0000 7f85eae00640 1 Processor -- start 2024-04-17T09:54:26.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.577+0000 7f85eae00640 1 -- start start 2024-04-17T09:54:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.577+0000 7f85eae00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85e409b100 0x7f85e40cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.577+0000 7f85eae00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85e415f920 0x7f85e40cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.577+0000 7f85eae00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85e40d0f00 con 0x7f85e415dff0 2024-04-17T09:54:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.577+0000 7f85eae00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f85e40ce940 con 0x7f85e415f920 2024-04-17T09:54:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.577+0000 7f85eae00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f85e40d11a0 con 0x7f85e409b100 2024-04-17T09:54:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.577+0000 7f85e3e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f85e415dff0 0x7f85e409a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37162/0 (socket says 172.21.15.32:37162) 2024-04-17T09:54:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.577+0000 7f85e3e00640 1 -- 172.21.15.32:0/3827688083 learned_addr learned my addr 172.21.15.32:0/3827688083 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.577+0000 7f85e8e00640 1 --2- 172.21.15.32:0/3827688083 >> v2:172.21.15.32:3300/0 conn(0x7f85e415f920 0x7f85e40cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.577+0000 7f85db400640 1 --2- 172.21.15.32:0/3827688083 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85e409b100 0x7f85e40cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.577+0000 7f85e8e00640 1 -- 172.21.15.32:0/3827688083 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85e409b100 msgr2=0x7f85e40cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:26.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.577+0000 7f85e8e00640 1 --2- 172.21.15.32:0/3827688083 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85e409b100 0x7f85e40cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:26.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.577+0000 7f85e8e00640 1 -- 172.21.15.32:0/3827688083 >> v1:172.21.15.32:6789/0 conn(0x7f85e415dff0 legacy=0x7f85e409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:26.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.577+0000 7f85e8e00640 1 -- 172.21.15.32:0/3827688083 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e40d1480 con 0x7f85e415f920 2024-04-17T09:54:26.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.577+0000 7f85db400640 1 --2- 172.21.15.32:0/3827688083 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85e409b100 0x7f85e40cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:26.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.581+0000 7f85e8e00640 1 --2- 172.21.15.32:0/3827688083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e415f920 0x7f85e40cd7f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f85dc0097c0 tx=0x7f85dc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743de9644855a408 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:26.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.581+0000 7f85e3400640 1 -- 172.21.15.32:0/3827688083 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc0b0070 con 0x7f85e415f920 2024-04-17T09:54:26.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.581+0000 7f85e3400640 1 -- 172.21.15.32:0/3827688083 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85dc0ab600 con 0x7f85e415f920 2024-04-17T09:54:26.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.581+0000 7f85e3400640 1 -- 172.21.15.32:0/3827688083 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc0ab9d0 con 0x7f85e415f920 2024-04-17T09:54:26.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.581+0000 7f85eae00640 1 -- 172.21.15.32:0/3827688083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e415f920 msgr2=0x7f85e40cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.581+0000 7f85eae00640 1 --2- 172.21.15.32:0/3827688083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e415f920 0x7f85e40cd7f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f85dc0097c0 tx=0x7f85dc01c410 comp rx=0 tx=0).stop 2024-04-17T09:54:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.581+0000 7f85eae00640 1 -- 172.21.15.32:0/3827688083 shutdown_connections 2024-04-17T09:54:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.581+0000 7f85eae00640 1 --2- 172.21.15.32:0/3827688083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e415f920 0x7f85e40cd7f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.581+0000 7f85eae00640 1 --2- 172.21.15.32:0/3827688083 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85e409b100 0x7f85e40cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.581+0000 7f85eae00640 1 -- 172.21.15.32:0/3827688083 >> 172.21.15.32:0/3827688083 conn(0x7f85e40b40e0 msgr2=0x7f85e40b44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.581+0000 7f85eae00640 1 -- 172.21.15.32:0/3827688083 shutdown_connections 2024-04-17T09:54:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.581+0000 7f85eae00640 1 -- 172.21.15.32:0/3827688083 wait complete. 2024-04-17T09:54:26.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85eae00640 1 Processor -- start 2024-04-17T09:54:26.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85eae00640 1 -- start start 2024-04-17T09:54:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85eae00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85e409b100 0x7f85e40b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85eae00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e415dff0 0x7f85e406dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85eae00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85e415f920 0x7f85e406e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85eae00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f85e4169a50 con 0x7f85e415dff0 2024-04-17T09:54:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85eae00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f85e41698d0 con 0x7f85e415f920 2024-04-17T09:54:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85eae00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f85e4169bd0 con 0x7f85e409b100 2024-04-17T09:54:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85db400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e415dff0 0x7f85e406dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85db400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e415dff0 0x7f85e406dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51202/0 (socket says 172.21.15.32:51202) 2024-04-17T09:54:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85db400640 1 -- 172.21.15.32:0/4140492004 learned_addr learned my addr 172.21.15.32:0/4140492004 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85e3e00640 1 --2- 172.21.15.32:0/4140492004 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85e409b100 0x7f85e40b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85e8e00640 1 --2- 172.21.15.32:0/4140492004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85e415f920 0x7f85e406e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85db400640 1 -- 172.21.15.32:0/4140492004 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85e409b100 msgr2=0x7f85e40b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:26.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85db400640 1 --2- 172.21.15.32:0/4140492004 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85e409b100 0x7f85e40b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:26.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85db400640 1 -- 172.21.15.32:0/4140492004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85e415f920 msgr2=0x7f85e406e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:26.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85db400640 1 --2- 172.21.15.32:0/4140492004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85e415f920 0x7f85e406e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:26.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85db400640 1 -- 172.21.15.32:0/4140492004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e40c71b0 con 0x7f85e415dff0 2024-04-17T09:54:26.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85e8e00640 1 --2- 172.21.15.32:0/4140492004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85e415f920 0x7f85e406e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:54:26.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85db400640 1 --2- 172.21.15.32:0/4140492004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e415dff0 0x7f85e406dbe0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009fc0 tx=0x7f85cc00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:26.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85e0c00640 1 -- 172.21.15.32:0/4140492004 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc01a070 con 0x7f85e415dff0 2024-04-17T09:54:26.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85e0c00640 1 -- 172.21.15.32:0/4140492004 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85cc003bc0 con 0x7f85e415dff0 2024-04-17T09:54:26.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85eae00640 1 -- 172.21.15.32:0/4140492004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e40c7490 con 0x7f85e415dff0 2024-04-17T09:54:26.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85e0c00640 1 -- 172.21.15.32:0/4140492004 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc004030 con 0x7f85e415dff0 2024-04-17T09:54:26.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85eae00640 1 -- 172.21.15.32:0/4140492004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e40c7d30 con 0x7f85e415dff0 2024-04-17T09:54:26.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.585+0000 7f85eae00640 1 -- 172.21.15.32:0/4140492004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85e40c7660 con 0x7f85e415dff0 2024-04-17T09:54:26.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.589+0000 7f85e0c00640 1 -- 172.21.15.32:0/4140492004 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f85cc01d070 con 0x7f85e415dff0 2024-04-17T09:54:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.589+0000 7f85e0c00640 1 --2- 172.21.15.32:0/4140492004 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f85c0068380 0x7f85c006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.589+0000 7f85e0c00640 1 -- 172.21.15.32:0/4140492004 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f85cc086ce0 con 0x7f85e415dff0 2024-04-17T09:54:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.589+0000 7f85e3e00640 1 --2- 172.21.15.32:0/4140492004 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f85c0068380 0x7f85c006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:26.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.589+0000 7f85e3e00640 1 --2- 172.21.15.32:0/4140492004 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f85c0068380 0x7f85c006a840 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f85d4000f70 tx=0x7f85d4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:26.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.593+0000 7f85e0c00640 1 -- 172.21.15.32:0/4140492004 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f85cc010070 con 0x7f85e415dff0 2024-04-17T09:54:26.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.757+0000 7f85eae00640 1 -- 172.21.15.32:0/4140492004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f85e4068a90 con 0x7f85e415dff0 2024-04-17T09:54:26.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.757+0000 7f85e0c00640 1 -- 172.21.15.32:0/4140492004 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f85cc056bb0 con 0x7f85e415dff0 2024-04-17T09:54:26.759 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:54:26.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.761+0000 7f85eae00640 1 -- 172.21.15.32:0/4140492004 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f85c0068380 msgr2=0x7f85c006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:26.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.761+0000 7f85eae00640 1 --2- 172.21.15.32:0/4140492004 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f85c0068380 0x7f85c006a840 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f85d4000f70 tx=0x7f85d4009040 comp rx=0 tx=0).stop 2024-04-17T09:54:26.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.761+0000 7f85eae00640 1 -- 172.21.15.32:0/4140492004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e415dff0 msgr2=0x7f85e406dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:26.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.761+0000 7f85eae00640 1 --2- 172.21.15.32:0/4140492004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e415dff0 0x7f85e406dbe0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009fc0 tx=0x7f85cc00d560 comp rx=0 tx=0).stop 2024-04-17T09:54:26.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.761+0000 7f85eae00640 1 -- 172.21.15.32:0/4140492004 shutdown_connections 2024-04-17T09:54:26.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.761+0000 7f85eae00640 1 --2- 172.21.15.32:0/4140492004 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f85c0068380 0x7f85c006a840 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:26.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.761+0000 7f85eae00640 1 --2- 172.21.15.32:0/4140492004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85e415f920 0x7f85e406e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:26.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.761+0000 7f85eae00640 1 --2- 172.21.15.32:0/4140492004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85e415dff0 0x7f85e406dbe0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:26.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.761+0000 7f85eae00640 1 --2- 172.21.15.32:0/4140492004 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85e409b100 0x7f85e40b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:26.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.761+0000 7f85eae00640 1 -- 172.21.15.32:0/4140492004 >> 172.21.15.32:0/4140492004 conn(0x7f85e40b40e0 msgr2=0x7f85e406ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:26.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.761+0000 7f85eae00640 1 -- 172.21.15.32:0/4140492004 shutdown_connections 2024-04-17T09:54:26.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.761+0000 7f85eae00640 1 -- 172.21.15.32:0/4140492004 wait complete. 2024-04-17T09:54:26.780 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:54:26.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.921+0000 7fd649c00640 1 Processor -- start 2024-04-17T09:54:26.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.921+0000 7fd649c00640 1 -- start start 2024-04-17T09:54:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.925+0000 7fd649c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd64415e7c0 0x7fd64415eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.925+0000 7fd649c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd644072970 0x7fd6440b5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.925+0000 7fd649c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6440b56d0 con 0x7fd6440b4490 2024-04-17T09:54:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.925+0000 7fd649c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd6440b39b0 con 0x7fd644072970 2024-04-17T09:54:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.925+0000 7fd649c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd6440b3c90 con 0x7fd64415e7c0 2024-04-17T09:54:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.925+0000 7fd643400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd644072970 0x7fd6440b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.925+0000 7fd643400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd644072970 0x7fd6440b5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51206/0 (socket says 172.21.15.32:51206) 2024-04-17T09:54:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.925+0000 7fd643400640 1 -- 172.21.15.32:0/4216431116 learned_addr learned my addr 172.21.15.32:0/4216431116 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.925+0000 7fd642a00640 1 --2- 172.21.15.32:0/4216431116 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd64415e7c0 0x7fd64415eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.925+0000 7fd643400640 1 -- 172.21.15.32:0/4216431116 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd64415e7c0 msgr2=0x7fd64415eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.925+0000 7fd643400640 1 --2- 172.21.15.32:0/4216431116 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd64415e7c0 0x7fd64415eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.925+0000 7fd643400640 1 -- 172.21.15.32:0/4216431116 >> v1:172.21.15.144:6789/0 conn(0x7fd6440b4490 legacy=0x7fd644072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.925+0000 7fd643400640 1 -- 172.21.15.32:0/4216431116 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6440b3f70 con 0x7fd644072970 2024-04-17T09:54:26.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.925+0000 7fd643400640 1 --2- 172.21.15.32:0/4216431116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd644072970 0x7fd6440b5040 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009e50 tx=0x7fd63c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc7d15c11e84d32 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:26.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.925+0000 7fd642000640 1 -- 172.21.15.32:0/4216431116 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0a5880 con 0x7fd644072970 2024-04-17T09:54:26.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd642000640 1 -- 172.21.15.32:0/4216431116 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd63c0a5a20 con 0x7fd644072970 2024-04-17T09:54:26.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd642000640 1 -- 172.21.15.32:0/4216431116 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0a5df0 con 0x7fd644072970 2024-04-17T09:54:26.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd649c00640 1 -- 172.21.15.32:0/4216431116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd644072970 msgr2=0x7fd6440b5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:26.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd649c00640 1 --2- 172.21.15.32:0/4216431116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd644072970 0x7fd6440b5040 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009e50 tx=0x7fd63c01c9a0 comp rx=0 tx=0).stop 2024-04-17T09:54:26.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd649c00640 1 -- 172.21.15.32:0/4216431116 shutdown_connections 2024-04-17T09:54:26.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd649c00640 1 --2- 172.21.15.32:0/4216431116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd644072970 0x7fd6440b5040 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:26.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd649c00640 1 --2- 172.21.15.32:0/4216431116 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd64415e7c0 0x7fd64415eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:26.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd649c00640 1 -- 172.21.15.32:0/4216431116 >> 172.21.15.32:0/4216431116 conn(0x7fd6440b2510 msgr2=0x7fd6440b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:26.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd649c00640 1 -- 172.21.15.32:0/4216431116 shutdown_connections 2024-04-17T09:54:26.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd649c00640 1 -- 172.21.15.32:0/4216431116 wait complete. 2024-04-17T09:54:26.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd649c00640 1 Processor -- start 2024-04-17T09:54:26.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd649c00640 1 -- start start 2024-04-17T09:54:26.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd649c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd644072970 0x7fd644153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:26.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd649c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6440b4490 0x7fd644158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:26.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd649c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd64415e7c0 0x7fd644152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:26.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd649c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd64406f1b0 con 0x7fd6440b4490 2024-04-17T09:54:26.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd649c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd64406f030 con 0x7fd64415e7c0 2024-04-17T09:54:26.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd649c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd64406f330 con 0x7fd644072970 2024-04-17T09:54:26.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd63a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6440b4490 0x7fd644158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:26.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd63a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6440b4490 0x7fd644158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51212/0 (socket says 172.21.15.32:51212) 2024-04-17T09:54:26.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd63a000640 1 -- 172.21.15.32:0/1779530298 learned_addr learned my addr 172.21.15.32:0/1779530298 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:26.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd642a00640 1 --2- 172.21.15.32:0/1779530298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd644072970 0x7fd644153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:26.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd643400640 1 --2- 172.21.15.32:0/1779530298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd64415e7c0 0x7fd644152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:26.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd63a000640 1 -- 172.21.15.32:0/1779530298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd644072970 msgr2=0x7fd644153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:26.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd63a000640 1 --2- 172.21.15.32:0/1779530298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd644072970 0x7fd644153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:26.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd63a000640 1 -- 172.21.15.32:0/1779530298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd64415e7c0 msgr2=0x7fd644152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:26.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd63a000640 1 --2- 172.21.15.32:0/1779530298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd64415e7c0 0x7fd644152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:26.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.929+0000 7fd63a000640 1 -- 172.21.15.32:0/1779530298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6441546b0 con 0x7fd6440b4490 2024-04-17T09:54:26.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.933+0000 7fd63a000640 1 --2- 172.21.15.32:0/1779530298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6440b4490 0x7fd644158400 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00c340 tx=0x7fd62c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:26.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.933+0000 7fd63b400640 1 -- 172.21.15.32:0/1779530298 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c0052d0 con 0x7fd6440b4490 2024-04-17T09:54:26.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.933+0000 7fd63b400640 1 -- 172.21.15.32:0/1779530298 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd62c005470 con 0x7fd6440b4490 2024-04-17T09:54:26.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.933+0000 7fd63b400640 1 -- 172.21.15.32:0/1779530298 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c007260 con 0x7fd6440b4490 2024-04-17T09:54:26.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.933+0000 7fd649c00640 1 -- 172.21.15.32:0/1779530298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd644154c30 con 0x7fd6440b4490 2024-04-17T09:54:26.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.933+0000 7fd649c00640 1 -- 172.21.15.32:0/1779530298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd64415fd80 con 0x7fd6440b4490 2024-04-17T09:54:26.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.933+0000 7fd649c00640 1 -- 172.21.15.32:0/1779530298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd644068a90 con 0x7fd6440b4490 2024-04-17T09:54:26.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.933+0000 7fd63b400640 1 -- 172.21.15.32:0/1779530298 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd62c007400 con 0x7fd6440b4490 2024-04-17T09:54:26.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.933+0000 7fd63b400640 1 --2- 172.21.15.32:0/1779530298 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6200683b0 0x7fd62006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:26.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.933+0000 7fd63b400640 1 -- 172.21.15.32:0/1779530298 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fd62c085d50 con 0x7fd6440b4490 2024-04-17T09:54:26.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.933+0000 7fd642a00640 1 --2- 172.21.15.32:0/1779530298 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6200683b0 0x7fd62006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:26.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.937+0000 7fd642a00640 1 --2- 172.21.15.32:0/1779530298 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6200683b0 0x7fd62006a870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd64415ac40 tx=0x7fd634008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:26.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:26.937+0000 7fd63b400640 1 -- 172.21.15.32:0/1779530298 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd62c055cb0 con 0x7fd6440b4490 2024-04-17T09:54:27.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.101+0000 7fd649c00640 1 -- 172.21.15.32:0/1779530298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd644073060 con 0x7fd6440b4490 2024-04-17T09:54:27.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.101+0000 7fd63b400640 1 -- 172.21.15.32:0/1779530298 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) ==== 104+0+11 (secure 0 0 0) 0x7fd62c059fb0 con 0x7fd6440b4490 2024-04-17T09:54:27.104 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:54:27.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.105+0000 7fd649c00640 1 -- 172.21.15.32:0/1779530298 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6200683b0 msgr2=0x7fd62006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:27.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.105+0000 7fd649c00640 1 --2- 172.21.15.32:0/1779530298 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6200683b0 0x7fd62006a870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd64415ac40 tx=0x7fd634008040 comp rx=0 tx=0).stop 2024-04-17T09:54:27.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.105+0000 7fd649c00640 1 -- 172.21.15.32:0/1779530298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6440b4490 msgr2=0x7fd644158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:27.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.105+0000 7fd649c00640 1 --2- 172.21.15.32:0/1779530298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6440b4490 0x7fd644158400 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00c340 tx=0x7fd62c00c800 comp rx=0 tx=0).stop 2024-04-17T09:54:27.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.105+0000 7fd649c00640 1 -- 172.21.15.32:0/1779530298 shutdown_connections 2024-04-17T09:54:27.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.105+0000 7fd649c00640 1 --2- 172.21.15.32:0/1779530298 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6200683b0 0x7fd62006a870 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.105+0000 7fd649c00640 1 --2- 172.21.15.32:0/1779530298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd64415e7c0 0x7fd644152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.105+0000 7fd649c00640 1 --2- 172.21.15.32:0/1779530298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6440b4490 0x7fd644158400 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.105+0000 7fd649c00640 1 --2- 172.21.15.32:0/1779530298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd644072970 0x7fd644153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.105+0000 7fd649c00640 1 -- 172.21.15.32:0/1779530298 >> 172.21.15.32:0/1779530298 conn(0x7fd6440b2510 msgr2=0x7fd644157b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:27.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.105+0000 7fd649c00640 1 -- 172.21.15.32:0/1779530298 shutdown_connections 2024-04-17T09:54:27.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.105+0000 7fd649c00640 1 -- 172.21.15.32:0/1779530298 wait complete. 2024-04-17T09:54:27.125 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:54:27.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.265+0000 7f6c55a00640 1 Processor -- start 2024-04-17T09:54:27.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.265+0000 7f6c55a00640 1 -- start start 2024-04-17T09:54:27.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.269+0000 7f6c55a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c5006f7c0 0x7f6c5006ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:27.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.269+0000 7f6c55a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c5006f270 0x7f6c5006bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:27.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.269+0000 7f6c55a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c500cc940 con 0x7f6c5015b7d0 2024-04-17T09:54:27.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.269+0000 7f6c55a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6c500ccc20 con 0x7f6c5006f7c0 2024-04-17T09:54:27.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.269+0000 7f6c55a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6c5006e0b0 con 0x7f6c5006f270 2024-04-17T09:54:27.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.269+0000 7f6c46000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c5006f7c0 0x7f6c5006ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:27.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.269+0000 7f6c46000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c5006f7c0 0x7f6c5006ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51218/0 (socket says 172.21.15.32:51218) 2024-04-17T09:54:27.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.269+0000 7f6c46000640 1 -- 172.21.15.32:0/2671974006 learned_addr learned my addr 172.21.15.32:0/2671974006 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:27.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.269+0000 7f6c4ea00640 1 --1- 172.21.15.32:0/2671974006 >> v1:172.21.15.32:6789/0 conn(0x7f6c5015b7d0 0x7f6c5006e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37176/0 (socket says 172.21.15.32:37176) 2024-04-17T09:54:27.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.269+0000 7f6c4f400640 1 --2- 172.21.15.32:0/2671974006 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c5006f270 0x7f6c5006bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:27.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.269+0000 7f6c46000640 1 -- 172.21.15.32:0/2671974006 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c5006f270 msgr2=0x7f6c5006bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:27.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.269+0000 7f6c46000640 1 --2- 172.21.15.32:0/2671974006 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c5006f270 0x7f6c5006bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.269+0000 7f6c46000640 1 -- 172.21.15.32:0/2671974006 >> v1:172.21.15.32:6789/0 conn(0x7f6c5015b7d0 legacy=0x7f6c5006e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:27.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.269+0000 7f6c46000640 1 -- 172.21.15.32:0/2671974006 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5006e390 con 0x7f6c5006f7c0 2024-04-17T09:54:27.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.273+0000 7f6c46000640 1 --2- 172.21.15.32:0/2671974006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c5006f7c0 0x7f6c5006ba30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009870 tx=0x7f6c3801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c6c76cbb847b86 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:27.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.273+0000 7f6c4e000640 1 -- 172.21.15.32:0/2671974006 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c380aa650 con 0x7f6c5006f7c0 2024-04-17T09:54:27.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.273+0000 7f6c4e000640 1 -- 172.21.15.32:0/2671974006 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c380af070 con 0x7f6c5006f7c0 2024-04-17T09:54:27.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.273+0000 7f6c4e000640 1 -- 172.21.15.32:0/2671974006 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c380aaa10 con 0x7f6c5006f7c0 2024-04-17T09:54:27.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.273+0000 7f6c55a00640 1 -- 172.21.15.32:0/2671974006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c5006f7c0 msgr2=0x7f6c5006ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:27.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.273+0000 7f6c55a00640 1 --2- 172.21.15.32:0/2671974006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c5006f7c0 0x7f6c5006ba30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009870 tx=0x7f6c3801c4d0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.273+0000 7f6c55a00640 1 -- 172.21.15.32:0/2671974006 shutdown_connections 2024-04-17T09:54:27.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.273+0000 7f6c55a00640 1 --2- 172.21.15.32:0/2671974006 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c5006f270 0x7f6c5006bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.273+0000 7f6c55a00640 1 --2- 172.21.15.32:0/2671974006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c5006f7c0 0x7f6c5006ba30 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.273+0000 7f6c55a00640 1 -- 172.21.15.32:0/2671974006 >> 172.21.15.32:0/2671974006 conn(0x7f6c500b2120 msgr2=0x7f6c500b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:27.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.273+0000 7f6c55a00640 1 -- 172.21.15.32:0/2671974006 shutdown_connections 2024-04-17T09:54:27.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.273+0000 7f6c55a00640 1 -- 172.21.15.32:0/2671974006 wait complete. 2024-04-17T09:54:27.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.273+0000 7f6c55a00640 1 Processor -- start 2024-04-17T09:54:27.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.273+0000 7f6c55a00640 1 -- start start 2024-04-17T09:54:27.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.277+0000 7f6c55a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c5006f270 0x7f6c5015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:27.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.277+0000 7f6c55a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c5006f7c0 0x7f6c5015e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:27.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.277+0000 7f6c55a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c5015b7d0 0x7f6c5013fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:27.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.277+0000 7f6c55a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6c500ceeb0 con 0x7f6c5006f7c0 2024-04-17T09:54:27.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.277+0000 7f6c55a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6c500ced30 con 0x7f6c5015b7d0 2024-04-17T09:54:27.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.277+0000 7f6c55a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6c50071e60 con 0x7f6c5006f270 2024-04-17T09:54:27.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.277+0000 7f6c46000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c5006f7c0 0x7f6c5015e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:27.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.277+0000 7f6c46000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c5006f7c0 0x7f6c5015e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51228/0 (socket says 172.21.15.32:51228) 2024-04-17T09:54:27.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.277+0000 7f6c46000640 1 -- 172.21.15.32:0/3078563150 learned_addr learned my addr 172.21.15.32:0/3078563150 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:27.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.277+0000 7f6c46000640 1 -- 172.21.15.32:0/3078563150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c5006f270 msgr2=0x7f6c5015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:27.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.277+0000 7f6c46000640 1 --2- 172.21.15.32:0/3078563150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c5006f270 0x7f6c5015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.277+0000 7f6c46000640 1 -- 172.21.15.32:0/3078563150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c5015b7d0 msgr2=0x7f6c5013fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:27.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.277+0000 7f6c46000640 1 --2- 172.21.15.32:0/3078563150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c5015b7d0 0x7f6c5013fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.277+0000 7f6c46000640 1 -- 172.21.15.32:0/3078563150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c50150620 con 0x7f6c5006f7c0 2024-04-17T09:54:27.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.277+0000 7f6c46000640 1 --2- 172.21.15.32:0/3078563150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c5006f7c0 0x7f6c5015e5e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6c380ac040 tx=0x7f6c3801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:27.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.277+0000 7f6c47400640 1 -- 172.21.15.32:0/3078563150 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c380b3800 con 0x7f6c5006f7c0 2024-04-17T09:54:27.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.277+0000 7f6c47400640 1 -- 172.21.15.32:0/3078563150 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c380a6040 con 0x7f6c5006f7c0 2024-04-17T09:54:27.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.277+0000 7f6c47400640 1 -- 172.21.15.32:0/3078563150 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c380b3be0 con 0x7f6c5006f7c0 2024-04-17T09:54:27.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.277+0000 7f6c55a00640 1 -- 172.21.15.32:0/3078563150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c50150900 con 0x7f6c5006f7c0 2024-04-17T09:54:27.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.277+0000 7f6c55a00640 1 -- 172.21.15.32:0/3078563150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c50140410 con 0x7f6c5006f7c0 2024-04-17T09:54:27.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.277+0000 7f6c55a00640 1 -- 172.21.15.32:0/3078563150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c50068a90 con 0x7f6c5006f7c0 2024-04-17T09:54:27.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.277+0000 7f6c47400640 1 -- 172.21.15.32:0/3078563150 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6c380c6440 con 0x7f6c5006f7c0 2024-04-17T09:54:27.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.281+0000 7f6c47400640 1 --2- 172.21.15.32:0/3078563150 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6c2c0683d0 0x7f6c2c06a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:27.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.281+0000 7f6c47400640 1 -- 172.21.15.32:0/3078563150 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f6c38121220 con 0x7f6c5006f7c0 2024-04-17T09:54:27.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.281+0000 7f6c4ea00640 1 --2- 172.21.15.32:0/3078563150 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6c2c0683d0 0x7f6c2c06a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:27.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.281+0000 7f6c4ea00640 1 --2- 172.21.15.32:0/3078563150 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6c2c0683d0 0x7f6c2c06a890 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6c400012f0 tx=0x7f6c40007d90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:27.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.285+0000 7f6c47400640 1 -- 172.21.15.32:0/3078563150 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f6c38016150 con 0x7f6c5006f7c0 2024-04-17T09:54:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.449+0000 7f6c55a00640 1 -- 172.21.15.32:0/3078563150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6c501563d0 con 0x7f6c5006f7c0 2024-04-17T09:54:27.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.453+0000 7f6c47400640 1 -- 172.21.15.32:0/3078563150 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) ==== 100+0+11 (secure 0 0 0) 0x7f6c380f1970 con 0x7f6c5006f7c0 2024-04-17T09:54:27.453 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:54:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.453+0000 7f6c55a00640 1 -- 172.21.15.32:0/3078563150 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6c2c0683d0 msgr2=0x7f6c2c06a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.453+0000 7f6c55a00640 1 --2- 172.21.15.32:0/3078563150 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6c2c0683d0 0x7f6c2c06a890 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6c400012f0 tx=0x7f6c40007d90 comp rx=0 tx=0).stop 2024-04-17T09:54:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.453+0000 7f6c55a00640 1 -- 172.21.15.32:0/3078563150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c5006f7c0 msgr2=0x7f6c5015e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.453+0000 7f6c55a00640 1 --2- 172.21.15.32:0/3078563150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c5006f7c0 0x7f6c5015e5e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6c380ac040 tx=0x7f6c3801ad10 comp rx=0 tx=0).stop 2024-04-17T09:54:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.453+0000 7f6c55a00640 1 -- 172.21.15.32:0/3078563150 shutdown_connections 2024-04-17T09:54:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.453+0000 7f6c55a00640 1 --2- 172.21.15.32:0/3078563150 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6c2c0683d0 0x7f6c2c06a890 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.453+0000 7f6c55a00640 1 --2- 172.21.15.32:0/3078563150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c5015b7d0 0x7f6c5013fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.453+0000 7f6c55a00640 1 --2- 172.21.15.32:0/3078563150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c5006f7c0 0x7f6c5015e5e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.453+0000 7f6c55a00640 1 --2- 172.21.15.32:0/3078563150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c5006f270 0x7f6c5015bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.453+0000 7f6c55a00640 1 -- 172.21.15.32:0/3078563150 >> 172.21.15.32:0/3078563150 conn(0x7f6c500b2120 msgr2=0x7f6c501591e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:27.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.453+0000 7f6c55a00640 1 -- 172.21.15.32:0/3078563150 shutdown_connections 2024-04-17T09:54:27.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.453+0000 7f6c55a00640 1 -- 172.21.15.32:0/3078563150 wait complete. 2024-04-17T09:54:27.474 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T09:54:27.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.613+0000 7fe924000640 1 Processor -- start 2024-04-17T09:54:27.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.613+0000 7fe924000640 1 -- start start 2024-04-17T09:54:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.613+0000 7fe924000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe91c15c290 0x7fe91c0d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.613+0000 7fe924000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe91c0d00f0 0x7fe91c0af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.613+0000 7fe924000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe91c071eb0 con 0x7fe91c15c670 2024-04-17T09:54:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.613+0000 7fe924000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe91c072190 con 0x7fe91c15c290 2024-04-17T09:54:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.613+0000 7fe924000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe91c072470 con 0x7fe91c0d00f0 2024-04-17T09:54:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.613+0000 7fe922000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe91c15c670 0x7fe91c162450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37186/0 (socket says 172.21.15.32:37186) 2024-04-17T09:54:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.613+0000 7fe922000640 1 -- 172.21.15.32:0/3787688201 learned_addr learned my addr 172.21.15.32:0/3787688201 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.613+0000 7fe921600640 1 --2- 172.21.15.32:0/3787688201 >> v2:172.21.15.144:3300/0 conn(0x7fe91c15c290 0x7fe91c0d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.613+0000 7fe920c00640 1 --2- 172.21.15.32:0/3787688201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe91c0d00f0 0x7fe91c0af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.617+0000 7fe91be00640 1 -- 172.21.15.32:0/3787688201 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3570197047 0 0) 0x7fe91c071eb0 con 0x7fe91c15c670 2024-04-17T09:54:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.617+0000 7fe91be00640 1 -- 172.21.15.32:0/3787688201 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe904003530 con 0x7fe91c15c670 2024-04-17T09:54:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.617+0000 7fe91be00640 1 -- 172.21.15.32:0/3787688201 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1691610443 0 0) 0x7fe904003530 con 0x7fe91c15c670 2024-04-17T09:54:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.617+0000 7fe921600640 1 -- 172.21.15.32:0/3787688201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe91c0d00f0 msgr2=0x7fe91c0af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.617+0000 7fe921600640 1 --2- 172.21.15.32:0/3787688201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe91c0d00f0 0x7fe91c0af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.617+0000 7fe921600640 1 -- 172.21.15.32:0/3787688201 >> v1:172.21.15.32:6789/0 conn(0x7fe91c15c670 legacy=0x7fe91c162450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.617+0000 7fe921600640 1 -- 172.21.15.32:0/3787688201 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91c072750 con 0x7fe91c15c290 2024-04-17T09:54:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.617+0000 7fe920c00640 1 --2- 172.21.15.32:0/3787688201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe91c0d00f0 0x7fe91c0af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:54:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.617+0000 7fe921600640 1 --2- 172.21.15.32:0/3787688201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe91c15c290 0x7fe91c0d0720 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe9100097c0 tx=0x7fe91001c2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=673257b69771705a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.617+0000 7fe91be00640 1 -- 172.21.15.32:0/3787688201 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9100b0070 con 0x7fe91c15c290 2024-04-17T09:54:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.621+0000 7fe91be00640 1 -- 172.21.15.32:0/3787688201 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9100ab400 con 0x7fe91c15c290 2024-04-17T09:54:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.621+0000 7fe91be00640 1 -- 172.21.15.32:0/3787688201 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe914001a00 con 0x7fe91c15c290 2024-04-17T09:54:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.621+0000 7fe924000640 1 -- 172.21.15.32:0/3787688201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe91c15c290 msgr2=0x7fe91c0d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.621+0000 7fe924000640 1 --2- 172.21.15.32:0/3787688201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe91c15c290 0x7fe91c0d0720 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe9100097c0 tx=0x7fe91001c2a0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.621+0000 7fe924000640 1 -- 172.21.15.32:0/3787688201 shutdown_connections 2024-04-17T09:54:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.621+0000 7fe924000640 1 --2- 172.21.15.32:0/3787688201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe91c0d00f0 0x7fe91c0af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.621+0000 7fe924000640 1 --2- 172.21.15.32:0/3787688201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe91c15c290 0x7fe91c0d0720 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.621+0000 7fe924000640 1 -- 172.21.15.32:0/3787688201 >> 172.21.15.32:0/3787688201 conn(0x7fe91c06ba30 msgr2=0x7fe91c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:27.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.621+0000 7fe924000640 1 -- 172.21.15.32:0/3787688201 shutdown_connections 2024-04-17T09:54:27.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.621+0000 7fe924000640 1 -- 172.21.15.32:0/3787688201 wait complete. 2024-04-17T09:54:27.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.621+0000 7fe924000640 1 Processor -- start 2024-04-17T09:54:27.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.621+0000 7fe924000640 1 -- start start 2024-04-17T09:54:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.621+0000 7fe924000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe91c0d00f0 0x7fe91c13ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.621+0000 7fe924000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91c15c290 0x7fe91c163460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.621+0000 7fe924000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe91c15c670 0x7fe91c0b0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:27.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.621+0000 7fe924000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c16b970 con 0x7fe91c15c290 2024-04-17T09:54:27.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.621+0000 7fe924000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c16b7f0 con 0x7fe91c0d00f0 2024-04-17T09:54:27.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.621+0000 7fe924000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe91c16baf0 con 0x7fe91c15c670 2024-04-17T09:54:27.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.621+0000 7fe920c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91c15c290 0x7fe91c163460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:27.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.621+0000 7fe920c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91c15c290 0x7fe91c163460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51236/0 (socket says 172.21.15.32:51236) 2024-04-17T09:54:27.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.621+0000 7fe920c00640 1 -- 172.21.15.32:0/3165207695 learned_addr learned my addr 172.21.15.32:0/3165207695 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:27.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.625+0000 7fe921600640 1 --2- 172.21.15.32:0/3165207695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe91c0d00f0 0x7fe91c13ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:27.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.625+0000 7fe922000640 1 --2- 172.21.15.32:0/3165207695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe91c15c670 0x7fe91c0b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:27.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.625+0000 7fe920c00640 1 -- 172.21.15.32:0/3165207695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe91c15c670 msgr2=0x7fe91c0b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:27.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.625+0000 7fe920c00640 1 --2- 172.21.15.32:0/3165207695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe91c15c670 0x7fe91c0b0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.625+0000 7fe920c00640 1 -- 172.21.15.32:0/3165207695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe91c0d00f0 msgr2=0x7fe91c13ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:27.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.625+0000 7fe920c00640 1 --2- 172.21.15.32:0/3165207695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe91c0d00f0 0x7fe91c13ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.625+0000 7fe920c00640 1 -- 172.21.15.32:0/3165207695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91c163f20 con 0x7fe91c15c290 2024-04-17T09:54:27.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.625+0000 7fe920c00640 1 --2- 172.21.15.32:0/3165207695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91c15c290 0x7fe91c163460 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe90800e980 tx=0x7fe90800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:27.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.625+0000 7fe919600640 1 -- 172.21.15.32:0/3165207695 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe908017070 con 0x7fe91c15c290 2024-04-17T09:54:27.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.625+0000 7fe919600640 1 -- 172.21.15.32:0/3165207695 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe90800ca00 con 0x7fe91c15c290 2024-04-17T09:54:27.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.625+0000 7fe919600640 1 -- 172.21.15.32:0/3165207695 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90800cdd0 con 0x7fe91c15c290 2024-04-17T09:54:27.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.625+0000 7fe924000640 1 -- 172.21.15.32:0/3165207695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe91c164200 con 0x7fe91c15c290 2024-04-17T09:54:27.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.625+0000 7fe924000640 1 -- 172.21.15.32:0/3165207695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe91c1647c0 con 0x7fe91c15c290 2024-04-17T09:54:27.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.625+0000 7fe924000640 1 -- 172.21.15.32:0/3165207695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe91c068a90 con 0x7fe91c15c290 2024-04-17T09:54:27.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.625+0000 7fe919600640 1 -- 172.21.15.32:0/3165207695 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe90801a070 con 0x7fe91c15c290 2024-04-17T09:54:27.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.625+0000 7fe919600640 1 --2- 172.21.15.32:0/3165207695 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe8fc068380 0x7fe8fc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:27.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.625+0000 7fe919600640 1 -- 172.21.15.32:0/3165207695 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fe908086d20 con 0x7fe91c15c290 2024-04-17T09:54:27.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.625+0000 7fe921600640 1 --2- 172.21.15.32:0/3165207695 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe8fc068380 0x7fe8fc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:27.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.629+0000 7fe921600640 1 --2- 172.21.15.32:0/3165207695 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe8fc068380 0x7fe8fc06a840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe9100ad000 tx=0x7fe9100a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:27.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.629+0000 7fe919600640 1 -- 172.21.15.32:0/3165207695 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fe908057240 con 0x7fe91c15c290 2024-04-17T09:54:27.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.809+0000 7fe924000640 1 -- 172.21.15.32:0/3165207695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe91c06f120 con 0x7fe91c15c290 2024-04-17T09:54:27.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.809+0000 7fe919600640 1 -- 172.21.15.32:0/3165207695 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) ==== 80+0+0 (secure 0 0 0) 0x7fe908056b40 con 0x7fe91c15c290 2024-04-17T09:54:27.810 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T09:54:27.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.809+0000 7fe924000640 1 -- 172.21.15.32:0/3165207695 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe8fc068380 msgr2=0x7fe8fc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:27.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.809+0000 7fe924000640 1 --2- 172.21.15.32:0/3165207695 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe8fc068380 0x7fe8fc06a840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe9100ad000 tx=0x7fe9100a6000 comp rx=0 tx=0).stop 2024-04-17T09:54:27.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.809+0000 7fe924000640 1 -- 172.21.15.32:0/3165207695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91c15c290 msgr2=0x7fe91c163460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:27.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.809+0000 7fe924000640 1 --2- 172.21.15.32:0/3165207695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91c15c290 0x7fe91c163460 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe90800e980 tx=0x7fe90800ee40 comp rx=0 tx=0).stop 2024-04-17T09:54:27.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.809+0000 7fe924000640 1 -- 172.21.15.32:0/3165207695 shutdown_connections 2024-04-17T09:54:27.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.809+0000 7fe924000640 1 --2- 172.21.15.32:0/3165207695 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe8fc068380 0x7fe8fc06a840 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.809+0000 7fe924000640 1 --2- 172.21.15.32:0/3165207695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe91c15c670 0x7fe91c0b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.809+0000 7fe924000640 1 --2- 172.21.15.32:0/3165207695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91c15c290 0x7fe91c163460 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.809+0000 7fe924000640 1 --2- 172.21.15.32:0/3165207695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe91c0d00f0 0x7fe91c13ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:27.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.809+0000 7fe924000640 1 -- 172.21.15.32:0/3165207695 >> 172.21.15.32:0/3165207695 conn(0x7fe91c06ba30 msgr2=0x7fe91c15db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:27.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.809+0000 7fe924000640 1 -- 172.21.15.32:0/3165207695 shutdown_connections 2024-04-17T09:54:27.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:27.809+0000 7fe924000640 1 -- 172.21.15.32:0/3165207695 wait complete. 2024-04-17T09:54:27.831 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:54:27.831 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:54:27.840 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:54:28.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.021+0000 7f160e400640 1 Processor -- start 2024-04-17T09:54:28.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.021+0000 7f160e400640 1 -- start start 2024-04-17T09:54:28.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f160e400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f160815fb70 0x7f16080b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:28.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f160e400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16080b4d40 con 0x7f160815ff50 2024-04-17T09:54:28.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f160e400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16080d0ec0 con 0x7f160809b100 2024-04-17T09:54:28.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f160e400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f16080d0c70 con 0x7f160815fb70 2024-04-17T09:54:28.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f15ffe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f160815ff50 0x7f160809a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37202/0 (socket says 172.21.15.32:37202) 2024-04-17T09:54:28.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f15ffe00640 1 -- 172.21.15.32:0/1930044164 learned_addr learned my addr 172.21.15.32:0/1930044164 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:28.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f160d000640 1 --2- 172.21.15.32:0/1930044164 >> v2:172.21.15.144:3300/0 conn(0x7f160815fb70 0x7f16080b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:28.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f1607e00640 1 -- 172.21.15.32:0/1930044164 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2357901467 0 0) 0x7f16080b4d40 con 0x7f160815ff50 2024-04-17T09:54:28.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f1607e00640 1 -- 172.21.15.32:0/1930044164 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15ec003530 con 0x7f160815ff50 2024-04-17T09:54:28.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f1607e00640 1 -- 172.21.15.32:0/1930044164 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2576095533 0 0) 0x7f16080d0ec0 con 0x7f160809b100 2024-04-17T09:54:28.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f1607e00640 1 -- 172.21.15.32:0/1930044164 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16080b4d40 con 0x7f160809b100 2024-04-17T09:54:28.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f1607e00640 1 -- 172.21.15.32:0/1930044164 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4260117174 0 0) 0x7f15ec003530 con 0x7f160815ff50 2024-04-17T09:54:28.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f1607e00640 1 -- 172.21.15.32:0/1930044164 >> v2:172.21.15.144:3300/0 conn(0x7f160815fb70 msgr2=0x7f16080b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:28.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f1607e00640 1 --2- 172.21.15.32:0/1930044164 >> v2:172.21.15.144:3300/0 conn(0x7f160815fb70 0x7f16080b1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f1607e00640 1 -- 172.21.15.32:0/1930044164 >> v1:172.21.15.144:6789/0 conn(0x7f160809b100 legacy=0x7f16080cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:28.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f1607e00640 1 -- 172.21.15.32:0/1930044164 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16080d11a0 con 0x7f160815ff50 2024-04-17T09:54:28.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f1607e00640 1 -- 172.21.15.32:0/1930044164 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f15f0001680 con 0x7f160815ff50 2024-04-17T09:54:28.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f160d000640 1 --2- 172.21.15.32:0/1930044164 >> v2:172.21.15.144:3300/0 conn(0x7f160815fb70 0x7f16080b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:54:28.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f1607e00640 1 -- 172.21.15.32:0/1930044164 >> v1:172.21.15.32:6789/0 conn(0x7f160815ff50 legacy=0x7f160809a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:28.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f1607e00640 1 --2- 172.21.15.32:0/1930044164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15ec003b50 0x7f15ec003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:28.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f1607e00640 1 --2- 172.21.15.32:0/1930044164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15ec004810 0x7f15ec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:28.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f1607e00640 1 --2- 172.21.15.32:0/1930044164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec0051d0 0x7f15ec00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:28.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f1607e00640 1 -- 172.21.15.32:0/1930044164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f16080d0c70 con 0x7f15ec003b50 2024-04-17T09:54:28.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f1607e00640 1 -- 172.21.15.32:0/1930044164 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec00c100 con 0x7f15ec004810 2024-04-17T09:54:28.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f1607e00640 1 -- 172.21.15.32:0/1930044164 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f15ec00c300 con 0x7f15ec0051d0 2024-04-17T09:54:28.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f160d000640 1 --2- 172.21.15.32:0/1930044164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15ec003b50 0x7f15ec003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:28.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f15ffe00640 1 --2- 172.21.15.32:0/1930044164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15ec004810 0x7f15ec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:28.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f160da00640 1 --2- 172.21.15.32:0/1930044164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec0051d0 0x7f15ec00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:28.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f160d000640 1 -- 172.21.15.32:0/1930044164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec0051d0 msgr2=0x7f15ec00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:28.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f160d000640 1 --2- 172.21.15.32:0/1930044164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec0051d0 0x7f15ec00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f160d000640 1 -- 172.21.15.32:0/1930044164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15ec004810 msgr2=0x7f15ec004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:28.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f160d000640 1 --2- 172.21.15.32:0/1930044164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15ec004810 0x7f15ec004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f160d000640 1 -- 172.21.15.32:0/1930044164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15ec003530 con 0x7f15ec003b50 2024-04-17T09:54:28.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f15ffe00640 1 --2- 172.21.15.32:0/1930044164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15ec004810 0x7f15ec004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:28.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f160da00640 1 --2- 172.21.15.32:0/1930044164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec0051d0 0x7f15ec00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:28.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.025+0000 7f160da00640 1 -- 172.21.15.32:0/1930044164 reap_dead start 2024-04-17T09:54:28.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.029+0000 7f160d000640 1 --2- 172.21.15.32:0/1930044164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15ec003b50 0x7f15ec003f50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f15f8009cb0 tx=0x7f15f801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36da2d15002382a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:28.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.029+0000 7f1607e00640 1 -- 172.21.15.32:0/1930044164 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f80b2070 con 0x7f15ec003b50 2024-04-17T09:54:28.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.029+0000 7f1607e00640 1 -- 172.21.15.32:0/1930044164 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15f80a57c0 con 0x7f15ec003b50 2024-04-17T09:54:28.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.029+0000 7f1607e00640 1 -- 172.21.15.32:0/1930044164 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f8009880 con 0x7f15ec003b50 2024-04-17T09:54:28.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.029+0000 7f160e400640 1 -- 172.21.15.32:0/1930044164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15ec003b50 msgr2=0x7f15ec003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:28.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.029+0000 7f160e400640 1 --2- 172.21.15.32:0/1930044164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15ec003b50 0x7f15ec003f50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f15f8009cb0 tx=0x7f15f801cc10 comp rx=0 tx=0).stop 2024-04-17T09:54:28.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.029+0000 7f160e400640 1 -- 172.21.15.32:0/1930044164 shutdown_connections 2024-04-17T09:54:28.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.029+0000 7f160e400640 1 --2- 172.21.15.32:0/1930044164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15ec003b50 0x7f15ec003f50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.029+0000 7f160e400640 1 -- 172.21.15.32:0/1930044164 >> 172.21.15.32:0/1930044164 conn(0x7f16080b3e50 msgr2=0x7f16080b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:28.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.029+0000 7f160e400640 1 -- 172.21.15.32:0/1930044164 shutdown_connections 2024-04-17T09:54:28.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.029+0000 7f160e400640 1 -- 172.21.15.32:0/1930044164 wait complete. 2024-04-17T09:54:28.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.029+0000 7f160e400640 1 Processor -- start 2024-04-17T09:54:28.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.029+0000 7f160e400640 1 -- start start 2024-04-17T09:54:28.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f160e400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec003b50 0x7f16080cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:28.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f160e400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16080cf040 0x7f16081f82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:28.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f160e400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16080cf4c0 0x7f16081f8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f160e400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f16080b1f40 con 0x7f16080cf040 2024-04-17T09:54:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f160e400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f16080b1dc0 con 0x7f16080cf4c0 2024-04-17T09:54:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f160e400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f160809a9f0 con 0x7f15ec003b50 2024-04-17T09:54:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f15ffe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16080cf040 0x7f16081f82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f15ffe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16080cf040 0x7f16081f82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51246/0 (socket says 172.21.15.32:51246) 2024-04-17T09:54:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f15ffe00640 1 -- 172.21.15.32:0/3938077694 learned_addr learned my addr 172.21.15.32:0/3938077694 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f160d000640 1 --2- 172.21.15.32:0/3938077694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec003b50 0x7f16080cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f160da00640 1 --2- 172.21.15.32:0/3938077694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16080cf4c0 0x7f16081f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f15ffe00640 1 -- 172.21.15.32:0/3938077694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec003b50 msgr2=0x7f16080cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f15ffe00640 1 --2- 172.21.15.32:0/3938077694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec003b50 0x7f16080cc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f15ffe00640 1 -- 172.21.15.32:0/3938077694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16080cf4c0 msgr2=0x7f16081f8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f15ffe00640 1 --2- 172.21.15.32:0/3938077694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16080cf4c0 0x7f16081f8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f15ffe00640 1 -- 172.21.15.32:0/3938077694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16080ca970 con 0x7f16080cf040 2024-04-17T09:54:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f160da00640 1 --2- 172.21.15.32:0/3938077694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16080cf4c0 0x7f16081f8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f15ffe00640 1 --2- 172.21.15.32:0/3938077694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16080cf040 0x7f16081f82d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f15f000e780 tx=0x7f15f000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f1605600640 1 -- 172.21.15.32:0/3938077694 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f000c6c0 con 0x7f16080cf040 2024-04-17T09:54:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f1605600640 1 -- 172.21.15.32:0/3938077694 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15f0018070 con 0x7f16080cf040 2024-04-17T09:54:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f160e400640 1 -- 172.21.15.32:0/3938077694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16080cac50 con 0x7f16080cf040 2024-04-17T09:54:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f1605600640 1 -- 172.21.15.32:0/3938077694 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f000ca60 con 0x7f16080cf040 2024-04-17T09:54:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f160e400640 1 -- 172.21.15.32:0/3938077694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16080cb210 con 0x7f16080cf040 2024-04-17T09:54:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.033+0000 7f160e400640 1 -- 172.21.15.32:0/3938077694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f160804fd00 con 0x7f16080cf040 2024-04-17T09:54:28.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.037+0000 7f1605600640 1 -- 172.21.15.32:0/3938077694 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f15f000cc00 con 0x7f16080cf040 2024-04-17T09:54:28.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.037+0000 7f1605600640 1 --2- 172.21.15.32:0/3938077694 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f15e40683b0 0x7f15e406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:28.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.037+0000 7f1605600640 1 -- 172.21.15.32:0/3938077694 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f15f00866b0 con 0x7f16080cf040 2024-04-17T09:54:28.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.037+0000 7f160d000640 1 --2- 172.21.15.32:0/3938077694 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f15e40683b0 0x7f15e406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:28.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.037+0000 7f160d000640 1 --2- 172.21.15.32:0/3938077694 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f15e40683b0 0x7f15e406a870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f15f80af000 tx=0x7f15f80046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.041+0000 7f1605600640 1 -- 172.21.15.32:0/3938077694 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f15f0056c70 con 0x7f16080cf040 2024-04-17T09:54:28.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.209+0000 7f160e400640 1 -- 172.21.15.32:0/3938077694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16080b82e0 con 0x7f16080cf040 2024-04-17T09:54:28.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.213+0000 7f1605600640 1 -- 172.21.15.32:0/3938077694 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16681 (secure 0 0 0) 0x7f15f0056570 con 0x7f16080cf040 2024-04-17T09:54:28.213 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:28.213 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:54:18.351519+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:54:15.261471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-17T09:54:15.704958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:54:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.213+0000 7f160e400640 1 -- 172.21.15.32:0/3938077694 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f15e40683b0 msgr2=0x7f15e406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.213+0000 7f160e400640 1 --2- 172.21.15.32:0/3938077694 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f15e40683b0 0x7f15e406a870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f15f80af000 tx=0x7f15f80046d0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.213+0000 7f160e400640 1 -- 172.21.15.32:0/3938077694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16080cf040 msgr2=0x7f16081f82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.213+0000 7f160e400640 1 --2- 172.21.15.32:0/3938077694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16080cf040 0x7f16081f82d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f15f000e780 tx=0x7f15f000ec40 comp rx=0 tx=0).stop 2024-04-17T09:54:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.213+0000 7f160e400640 1 -- 172.21.15.32:0/3938077694 shutdown_connections 2024-04-17T09:54:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.213+0000 7f160e400640 1 --2- 172.21.15.32:0/3938077694 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f15e40683b0 0x7f15e406a870 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.213+0000 7f160e400640 1 --2- 172.21.15.32:0/3938077694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16080cf4c0 0x7f16081f8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.213+0000 7f160e400640 1 --2- 172.21.15.32:0/3938077694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16080cf040 0x7f16081f82d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.213+0000 7f160e400640 1 --2- 172.21.15.32:0/3938077694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec003b50 0x7f16080cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.213+0000 7f160e400640 1 -- 172.21.15.32:0/3938077694 >> 172.21.15.32:0/3938077694 conn(0x7f16080b3e50 msgr2=0x7f16080b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.213+0000 7f160e400640 1 -- 172.21.15.32:0/3938077694 shutdown_connections 2024-04-17T09:54:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.213+0000 7f160e400640 1 -- 172.21.15.32:0/3938077694 wait complete. 2024-04-17T09:54:28.234 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:54:15.261471+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:54:15.704958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:54:28.234 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:54:28.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.377+0000 7fb884a00640 1 Processor -- start 2024-04-17T09:54:28.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.377+0000 7fb884a00640 1 -- start start 2024-04-17T09:54:28.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.377+0000 7fb884a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb88006e160 0x7fb8800cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:28.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.377+0000 7fb884a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb8800cd8d0 0x7fb88009a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:28.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.377+0000 7fb884a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8800cdd90 con 0x7fb8800d1520 2024-04-17T09:54:28.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.377+0000 7fb884a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb8800ce070 con 0x7fb88006e160 2024-04-17T09:54:28.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.377+0000 7fb884a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb8800ce350 con 0x7fb8800cd8d0 2024-04-17T09:54:28.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.377+0000 7fb87f400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb88006e160 0x7fb8800cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:28.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.377+0000 7fb87fe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb8800d1520 0x7fb88015e180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37208/0 (socket says 172.21.15.32:37208) 2024-04-17T09:54:28.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.377+0000 7fb87fe00640 1 -- 172.21.15.32:0/2799222722 learned_addr learned my addr 172.21.15.32:0/2799222722 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:28.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.377+0000 7fb876a00640 1 --2- 172.21.15.32:0/2799222722 >> v2:172.21.15.144:3300/0 conn(0x7fb8800cd8d0 0x7fb88009a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:28.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.377+0000 7fb87ea00640 1 -- 172.21.15.32:0/2799222722 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3318548839 0 0) 0x7fb8800cdd90 con 0x7fb8800d1520 2024-04-17T09:54:28.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.377+0000 7fb87ea00640 1 -- 172.21.15.32:0/2799222722 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb864003530 con 0x7fb8800d1520 2024-04-17T09:54:28.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.377+0000 7fb87f400640 1 -- 172.21.15.32:0/2799222722 >> v2:172.21.15.144:3300/0 conn(0x7fb8800cd8d0 msgr2=0x7fb88009a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:28.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.377+0000 7fb87f400640 1 --2- 172.21.15.32:0/2799222722 >> v2:172.21.15.144:3300/0 conn(0x7fb8800cd8d0 0x7fb88009a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.377+0000 7fb87f400640 1 -- 172.21.15.32:0/2799222722 >> v1:172.21.15.32:6789/0 conn(0x7fb8800d1520 legacy=0x7fb88015e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:28.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.377+0000 7fb87f400640 1 -- 172.21.15.32:0/2799222722 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8800ce630 con 0x7fb88006e160 2024-04-17T09:54:28.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.381+0000 7fb87f400640 1 --2- 172.21.15.32:0/2799222722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb88006e160 0x7fb8800cd390 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb880075450 tx=0x7fb87001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6af434dde42042 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:28.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.381+0000 7fb87ea00640 1 -- 172.21.15.32:0/2799222722 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8700b0070 con 0x7fb88006e160 2024-04-17T09:54:28.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.381+0000 7fb87ea00640 1 -- 172.21.15.32:0/2799222722 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8700ab600 con 0x7fb88006e160 2024-04-17T09:54:28.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.381+0000 7fb87ea00640 1 -- 172.21.15.32:0/2799222722 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8700ab9d0 con 0x7fb88006e160 2024-04-17T09:54:28.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.381+0000 7fb884a00640 1 -- 172.21.15.32:0/2799222722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb88006e160 msgr2=0x7fb8800cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:28.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.381+0000 7fb884a00640 1 --2- 172.21.15.32:0/2799222722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb88006e160 0x7fb8800cd390 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb880075450 tx=0x7fb87001c480 comp rx=0 tx=0).stop 2024-04-17T09:54:28.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.381+0000 7fb884a00640 1 -- 172.21.15.32:0/2799222722 shutdown_connections 2024-04-17T09:54:28.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.381+0000 7fb884a00640 1 --2- 172.21.15.32:0/2799222722 >> v2:172.21.15.144:3300/0 conn(0x7fb8800cd8d0 0x7fb88009a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.381+0000 7fb884a00640 1 --2- 172.21.15.32:0/2799222722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb88006e160 0x7fb8800cd390 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.381+0000 7fb884a00640 1 -- 172.21.15.32:0/2799222722 >> 172.21.15.32:0/2799222722 conn(0x7fb8800b4000 msgr2=0x7fb8800b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:28.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.381+0000 7fb884a00640 1 -- 172.21.15.32:0/2799222722 shutdown_connections 2024-04-17T09:54:28.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.381+0000 7fb884a00640 1 -- 172.21.15.32:0/2799222722 wait complete. 2024-04-17T09:54:28.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb884a00640 1 Processor -- start 2024-04-17T09:54:28.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb884a00640 1 -- start start 2024-04-17T09:54:28.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb884a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb88006e160 0x7fb88014ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:28.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb884a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8800cd8d0 0x7fb880153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:28.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb884a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8800d1520 0x7fb880153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:28.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb884a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb88006d210 con 0x7fb8800d1520 2024-04-17T09:54:28.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb884a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb88006d090 con 0x7fb8800cd8d0 2024-04-17T09:54:28.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb884a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb88006d390 con 0x7fb88006e160 2024-04-17T09:54:28.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb87fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8800d1520 0x7fb880153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:28.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb87fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8800d1520 0x7fb880153570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51278/0 (socket says 172.21.15.32:51278) 2024-04-17T09:54:28.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb87fe00640 1 -- 172.21.15.32:0/2569281025 learned_addr learned my addr 172.21.15.32:0/2569281025 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:28.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb876a00640 1 --2- 172.21.15.32:0/2569281025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8800cd8d0 0x7fb880153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:28.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb87f400640 1 --2- 172.21.15.32:0/2569281025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb88006e160 0x7fb88014ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:28.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb87fe00640 1 -- 172.21.15.32:0/2569281025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb88006e160 msgr2=0x7fb88014ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:28.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb87fe00640 1 --2- 172.21.15.32:0/2569281025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb88006e160 0x7fb88014ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb87fe00640 1 -- 172.21.15.32:0/2569281025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8800cd8d0 msgr2=0x7fb880153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:28.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb87fe00640 1 --2- 172.21.15.32:0/2569281025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8800cd8d0 0x7fb880153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb87fe00640 1 -- 172.21.15.32:0/2569281025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8800b6290 con 0x7fb8800d1520 2024-04-17T09:54:28.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb876a00640 1 --2- 172.21.15.32:0/2569281025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8800cd8d0 0x7fb880153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:28.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb87fe00640 1 --2- 172.21.15.32:0/2569281025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8800d1520 0x7fb880153570 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb87800caa0 tx=0x7fb87800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:28.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb877e00640 1 -- 172.21.15.32:0/2569281025 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878017070 con 0x7fb8800d1520 2024-04-17T09:54:28.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb877e00640 1 -- 172.21.15.32:0/2569281025 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb87800ac00 con 0x7fb8800d1520 2024-04-17T09:54:28.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb877e00640 1 -- 172.21.15.32:0/2569281025 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878012400 con 0x7fb8800d1520 2024-04-17T09:54:28.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb884a00640 1 -- 172.21.15.32:0/2569281025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8800b6570 con 0x7fb8800d1520 2024-04-17T09:54:28.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.385+0000 7fb884a00640 1 -- 172.21.15.32:0/2569281025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8800b6b30 con 0x7fb8800d1520 2024-04-17T09:54:28.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.389+0000 7fb884a00640 1 -- 172.21.15.32:0/2569281025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb88004fda0 con 0x7fb8800d1520 2024-04-17T09:54:28.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.389+0000 7fb877e00640 1 -- 172.21.15.32:0/2569281025 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb87801a070 con 0x7fb8800d1520 2024-04-17T09:54:28.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.389+0000 7fb877e00640 1 --2- 172.21.15.32:0/2569281025 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb85c068380 0x7fb85c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:28.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.389+0000 7fb877e00640 1 -- 172.21.15.32:0/2569281025 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fb878086730 con 0x7fb8800d1520 2024-04-17T09:54:28.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.389+0000 7fb87f400640 1 --2- 172.21.15.32:0/2569281025 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb85c068380 0x7fb85c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:28.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.389+0000 7fb87f400640 1 --2- 172.21.15.32:0/2569281025 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb85c068380 0x7fb85c06a840 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb8700ad000 tx=0x7fb870008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:28.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.393+0000 7fb877e00640 1 -- 172.21.15.32:0/2569281025 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fb878056690 con 0x7fb8800d1520 2024-04-17T09:54:28.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.561+0000 7fb884a00640 1 -- 172.21.15.32:0/2569281025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb8800c5d60 con 0x7fb8800d1520 2024-04-17T09:54:28.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.561+0000 7fb877e00640 1 -- 172.21.15.32:0/2569281025 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7fb87805a990 con 0x7fb8800d1520 2024-04-17T09:54:28.564 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:54:28.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.565+0000 7fb884a00640 1 -- 172.21.15.32:0/2569281025 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb85c068380 msgr2=0x7fb85c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:28.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.565+0000 7fb884a00640 1 --2- 172.21.15.32:0/2569281025 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb85c068380 0x7fb85c06a840 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb8700ad000 tx=0x7fb870008040 comp rx=0 tx=0).stop 2024-04-17T09:54:28.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.565+0000 7fb884a00640 1 -- 172.21.15.32:0/2569281025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8800d1520 msgr2=0x7fb880153570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:28.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.565+0000 7fb884a00640 1 --2- 172.21.15.32:0/2569281025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8800d1520 0x7fb880153570 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb87800caa0 tx=0x7fb87800cf60 comp rx=0 tx=0).stop 2024-04-17T09:54:28.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.565+0000 7fb884a00640 1 -- 172.21.15.32:0/2569281025 shutdown_connections 2024-04-17T09:54:28.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.565+0000 7fb884a00640 1 --2- 172.21.15.32:0/2569281025 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb85c068380 0x7fb85c06a840 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.565+0000 7fb884a00640 1 --2- 172.21.15.32:0/2569281025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8800d1520 0x7fb880153570 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.565+0000 7fb884a00640 1 --2- 172.21.15.32:0/2569281025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8800cd8d0 0x7fb880153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.565+0000 7fb884a00640 1 --2- 172.21.15.32:0/2569281025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb88006e160 0x7fb88014ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.565+0000 7fb884a00640 1 -- 172.21.15.32:0/2569281025 >> 172.21.15.32:0/2569281025 conn(0x7fb8800b4000 msgr2=0x7fb88006fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:28.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.565+0000 7fb884a00640 1 -- 172.21.15.32:0/2569281025 shutdown_connections 2024-04-17T09:54:28.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.565+0000 7fb884a00640 1 -- 172.21.15.32:0/2569281025 wait complete. 2024-04-17T09:54:28.585 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:54:28.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.725+0000 7f0260600640 1 Processor -- start 2024-04-17T09:54:28.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.729+0000 7f0260600640 1 -- start start 2024-04-17T09:54:28.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.729+0000 7f0260600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f025815ac30 0x7f025806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:28.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.729+0000 7f0260600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f025815b010 0x7f025806dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:28.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.729+0000 7f0260600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f025806fcd0 con 0x7f02580b3d80 2024-04-17T09:54:28.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.729+0000 7f0260600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f025806f300 con 0x7f025815b010 2024-04-17T09:54:28.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.729+0000 7f0260600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f025806e6f0 con 0x7f025815ac30 2024-04-17T09:54:28.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.729+0000 7f025d200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f025815b010 0x7f025806dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:28.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.729+0000 7f025d200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f025815b010 0x7f025806dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51290/0 (socket says 172.21.15.32:51290) 2024-04-17T09:54:28.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.729+0000 7f025d200640 1 -- 172.21.15.32:0/2523095547 learned_addr learned my addr 172.21.15.32:0/2523095547 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:28.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.729+0000 7f025dc00640 1 --2- 172.21.15.32:0/2523095547 >> v2:172.21.15.144:3300/0 conn(0x7f025815ac30 0x7f025806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:28.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.729+0000 7f025d200640 1 -- 172.21.15.32:0/2523095547 >> v2:172.21.15.144:3300/0 conn(0x7f025815ac30 msgr2=0x7f025806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:28.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.729+0000 7f025d200640 1 --2- 172.21.15.32:0/2523095547 >> v2:172.21.15.144:3300/0 conn(0x7f025815ac30 0x7f025806f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.729+0000 7f025d200640 1 -- 172.21.15.32:0/2523095547 >> v1:172.21.15.144:6789/0 conn(0x7f02580b3d80 legacy=0x7f0258071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:28.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.729+0000 7f025d200640 1 -- 172.21.15.32:0/2523095547 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025806e9d0 con 0x7f025815b010 2024-04-17T09:54:28.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.733+0000 7f025d200640 1 --2- 172.21.15.32:0/2523095547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f025815b010 0x7f025806dea0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f02580d1530 tx=0x7f024401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad8c08a2d31f882 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:28.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.733+0000 7f0257e00640 1 -- 172.21.15.32:0/2523095547 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02440aa680 con 0x7f025815b010 2024-04-17T09:54:28.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.733+0000 7f0257e00640 1 -- 172.21.15.32:0/2523095547 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02440af070 con 0x7f025815b010 2024-04-17T09:54:28.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.733+0000 7f0257e00640 1 -- 172.21.15.32:0/2523095547 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02440aaa40 con 0x7f025815b010 2024-04-17T09:54:28.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.733+0000 7f0260600640 1 -- 172.21.15.32:0/2523095547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f025815b010 msgr2=0x7f025806dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:28.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.733+0000 7f0260600640 1 --2- 172.21.15.32:0/2523095547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f025815b010 0x7f025806dea0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f02580d1530 tx=0x7f024401c520 comp rx=0 tx=0).stop 2024-04-17T09:54:28.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.733+0000 7f0260600640 1 -- 172.21.15.32:0/2523095547 shutdown_connections 2024-04-17T09:54:28.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.733+0000 7f0260600640 1 --2- 172.21.15.32:0/2523095547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f025815b010 0x7f025806dea0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.733+0000 7f0260600640 1 --2- 172.21.15.32:0/2523095547 >> v2:172.21.15.144:3300/0 conn(0x7f025815ac30 0x7f025806f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.733+0000 7f0260600640 1 -- 172.21.15.32:0/2523095547 >> 172.21.15.32:0/2523095547 conn(0x7f02580b2120 msgr2=0x7f02580b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:28.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.733+0000 7f0260600640 1 -- 172.21.15.32:0/2523095547 shutdown_connections 2024-04-17T09:54:28.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.733+0000 7f0260600640 1 -- 172.21.15.32:0/2523095547 wait complete. 2024-04-17T09:54:28.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.733+0000 7f0260600640 1 Processor -- start 2024-04-17T09:54:28.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.733+0000 7f0260600640 1 -- start start 2024-04-17T09:54:28.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f0260600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02580b3d80 0x7f0258150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:28.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f0260600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f025815ac30 0x7f02580cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:28.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f0260600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f025815b010 0x7f02580ce230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:28.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f0260600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f02580726f0 con 0x7f02580b3d80 2024-04-17T09:54:28.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f0260600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0258072570 con 0x7f025815ac30 2024-04-17T09:54:28.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f0260600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f02580cd9c0 con 0x7f025815b010 2024-04-17T09:54:28.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f025dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02580b3d80 0x7f0258150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:28.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f025dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02580b3d80 0x7f0258150b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51306/0 (socket says 172.21.15.32:51306) 2024-04-17T09:54:28.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f025dc00640 1 -- 172.21.15.32:0/3658956208 learned_addr learned my addr 172.21.15.32:0/3658956208 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:28.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f025e600640 1 --2- 172.21.15.32:0/3658956208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f025815b010 0x7f02580ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:28.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f025d200640 1 --2- 172.21.15.32:0/3658956208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f025815ac30 0x7f02580cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:28.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f025dc00640 1 -- 172.21.15.32:0/3658956208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f025815b010 msgr2=0x7f02580ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:28.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f025dc00640 1 --2- 172.21.15.32:0/3658956208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f025815b010 0x7f02580ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f025dc00640 1 -- 172.21.15.32:0/3658956208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f025815ac30 msgr2=0x7f02580cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:28.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f025dc00640 1 --2- 172.21.15.32:0/3658956208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f025815ac30 0x7f02580cdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f025dc00640 1 -- 172.21.15.32:0/3658956208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02581512c0 con 0x7f02580b3d80 2024-04-17T09:54:28.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f025e600640 1 --2- 172.21.15.32:0/3658956208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f025815b010 0x7f02580ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:28.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f025d200640 1 --2- 172.21.15.32:0/3658956208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f025815ac30 0x7f02580cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:28.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f025dc00640 1 --2- 172.21.15.32:0/3658956208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02580b3d80 0x7f0258150b00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f024c00d6d0 tx=0x7f024c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:28.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f0255600640 1 -- 172.21.15.32:0/3658956208 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c015070 con 0x7f02580b3d80 2024-04-17T09:54:28.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f0255600640 1 -- 172.21.15.32:0/3658956208 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f024c007900 con 0x7f02580b3d80 2024-04-17T09:54:28.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f0255600640 1 -- 172.21.15.32:0/3658956208 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c00b040 con 0x7f02580b3d80 2024-04-17T09:54:28.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f0260600640 1 -- 172.21.15.32:0/3658956208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0258151560 con 0x7f02580b3d80 2024-04-17T09:54:28.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f0260600640 1 -- 172.21.15.32:0/3658956208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0258151b20 con 0x7f02580b3d80 2024-04-17T09:54:28.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f0260600640 1 -- 172.21.15.32:0/3658956208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0258068a90 con 0x7f02580b3d80 2024-04-17T09:54:28.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.737+0000 7f0255600640 1 -- 172.21.15.32:0/3658956208 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f024c018070 con 0x7f02580b3d80 2024-04-17T09:54:28.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.741+0000 7f0255600640 1 --2- 172.21.15.32:0/3658956208 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0238068380 0x7f023806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:28.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.741+0000 7f0255600640 1 -- 172.21.15.32:0/3658956208 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f024c086d20 con 0x7f02580b3d80 2024-04-17T09:54:28.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.741+0000 7f025d200640 1 --2- 172.21.15.32:0/3658956208 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0238068380 0x7f023806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:28.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.741+0000 7f025d200640 1 --2- 172.21.15.32:0/3658956208 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0238068380 0x7f023806a840 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f02440ae000 tx=0x7f02440a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:28.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.745+0000 7f0255600640 1 -- 172.21.15.32:0/3658956208 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f024c057240 con 0x7f02580b3d80 2024-04-17T09:54:28.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.905+0000 7f0260600640 1 -- 172.21.15.32:0/3658956208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0258072a60 con 0x7f02580b3d80 2024-04-17T09:54:28.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.905+0000 7f0255600640 1 -- 172.21.15.32:0/3658956208 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) ==== 104+0+11 (secure 0 0 0) 0x7f024c056b40 con 0x7f02580b3d80 2024-04-17T09:54:28.908 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:54:28.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.909+0000 7f0260600640 1 -- 172.21.15.32:0/3658956208 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0238068380 msgr2=0x7f023806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:28.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.909+0000 7f0260600640 1 --2- 172.21.15.32:0/3658956208 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0238068380 0x7f023806a840 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f02440ae000 tx=0x7f02440a7000 comp rx=0 tx=0).stop 2024-04-17T09:54:28.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.909+0000 7f0260600640 1 -- 172.21.15.32:0/3658956208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02580b3d80 msgr2=0x7f0258150b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:28.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.909+0000 7f0260600640 1 --2- 172.21.15.32:0/3658956208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02580b3d80 0x7f0258150b00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f024c00d6d0 tx=0x7f024c00db90 comp rx=0 tx=0).stop 2024-04-17T09:54:28.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.909+0000 7f0260600640 1 -- 172.21.15.32:0/3658956208 shutdown_connections 2024-04-17T09:54:28.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.909+0000 7f0260600640 1 --2- 172.21.15.32:0/3658956208 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0238068380 0x7f023806a840 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.909+0000 7f0260600640 1 --2- 172.21.15.32:0/3658956208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f025815b010 0x7f02580ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.909+0000 7f0260600640 1 --2- 172.21.15.32:0/3658956208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f025815ac30 0x7f02580cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.909+0000 7f0260600640 1 --2- 172.21.15.32:0/3658956208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02580b3d80 0x7f0258150b00 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:28.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.909+0000 7f0260600640 1 -- 172.21.15.32:0/3658956208 >> 172.21.15.32:0/3658956208 conn(0x7f02580b2120 msgr2=0x7f0258071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:28.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.909+0000 7f0260600640 1 -- 172.21.15.32:0/3658956208 shutdown_connections 2024-04-17T09:54:28.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:28.909+0000 7f0260600640 1 -- 172.21.15.32:0/3658956208 wait complete. 2024-04-17T09:54:28.929 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:54:29.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.069+0000 7f942da00640 1 Processor -- start 2024-04-17T09:54:29.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.073+0000 7f942da00640 1 -- start start 2024-04-17T09:54:29.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.073+0000 7f942da00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f942815de50 0x7f94280d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:29.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.073+0000 7f942da00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94280d0700 0x7f942809afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:29.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.073+0000 7f942da00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94280ce5b0 con 0x7f94280b3500 2024-04-17T09:54:29.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.073+0000 7f942da00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f94280cdd10 con 0x7f94280d0700 2024-04-17T09:54:29.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.073+0000 7f942da00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f94280cdff0 con 0x7f942815de50 2024-04-17T09:54:29.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.073+0000 7f9427400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94280d0700 0x7f942809afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:29.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.073+0000 7f9427400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94280d0700 0x7f942809afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51308/0 (socket says 172.21.15.32:51308) 2024-04-17T09:54:29.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.073+0000 7f9427400640 1 -- 172.21.15.32:0/2028997940 learned_addr learned my addr 172.21.15.32:0/2028997940 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:29.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.073+0000 7f9426a00640 1 --2- 172.21.15.32:0/2028997940 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f942815de50 0x7f94280d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:29.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.073+0000 7f9427400640 1 -- 172.21.15.32:0/2028997940 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f942815de50 msgr2=0x7f94280d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:29.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.073+0000 7f9427400640 1 --2- 172.21.15.32:0/2028997940 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f942815de50 0x7f94280d01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:29.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.073+0000 7f9427400640 1 -- 172.21.15.32:0/2028997940 >> v1:172.21.15.144:6789/0 conn(0x7f94280b3500 legacy=0x7f94280d0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:29.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.073+0000 7f9427400640 1 -- 172.21.15.32:0/2028997940 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94280ce2d0 con 0x7f94280d0700 2024-04-17T09:54:29.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.077+0000 7f9427400640 1 --2- 172.21.15.32:0/2028997940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94280d0700 0x7f942809afb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9420009e50 tx=0x7f942001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86814074055a530b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:29.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.077+0000 7f9426000640 1 -- 172.21.15.32:0/2028997940 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94200a5810 con 0x7f94280d0700 2024-04-17T09:54:29.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.077+0000 7f9426000640 1 -- 172.21.15.32:0/2028997940 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94200a59b0 con 0x7f94280d0700 2024-04-17T09:54:29.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.077+0000 7f9426000640 1 -- 172.21.15.32:0/2028997940 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94200a5d80 con 0x7f94280d0700 2024-04-17T09:54:29.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.077+0000 7f942da00640 1 -- 172.21.15.32:0/2028997940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94280d0700 msgr2=0x7f942809afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:29.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.077+0000 7f942da00640 1 --2- 172.21.15.32:0/2028997940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94280d0700 0x7f942809afb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9420009e50 tx=0x7f942001ca30 comp rx=0 tx=0).stop 2024-04-17T09:54:29.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.077+0000 7f942da00640 1 -- 172.21.15.32:0/2028997940 shutdown_connections 2024-04-17T09:54:29.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.077+0000 7f942da00640 1 --2- 172.21.15.32:0/2028997940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94280d0700 0x7f942809afb0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:29.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.077+0000 7f942da00640 1 --2- 172.21.15.32:0/2028997940 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f942815de50 0x7f94280d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:29.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.077+0000 7f942da00640 1 -- 172.21.15.32:0/2028997940 >> 172.21.15.32:0/2028997940 conn(0x7f94280b40e0 msgr2=0x7f94280b44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:29.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.077+0000 7f942da00640 1 -- 172.21.15.32:0/2028997940 shutdown_connections 2024-04-17T09:54:29.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.077+0000 7f942da00640 1 -- 172.21.15.32:0/2028997940 wait complete. 2024-04-17T09:54:29.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.077+0000 7f942da00640 1 Processor -- start 2024-04-17T09:54:29.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.077+0000 7f942da00640 1 -- start start 2024-04-17T09:54:29.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f942da00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94280b3500 0x7f94280cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:29.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f942da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94280d0700 0x7f94280c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:29.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f942da00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f942815de50 0x7f94280cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:29.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f942da00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9428169bd0 con 0x7f94280d0700 2024-04-17T09:54:29.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f942da00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9428169a50 con 0x7f942815de50 2024-04-17T09:54:29.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f942da00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9428169d50 con 0x7f94280b3500 2024-04-17T09:54:29.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f941e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94280d0700 0x7f94280c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:29.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f941e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94280d0700 0x7f94280c9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51314/0 (socket says 172.21.15.32:51314) 2024-04-17T09:54:29.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f941e000640 1 -- 172.21.15.32:0/3453763991 learned_addr learned my addr 172.21.15.32:0/3453763991 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:29.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f9426a00640 1 --2- 172.21.15.32:0/3453763991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94280b3500 0x7f94280cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:29.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f9427400640 1 --2- 172.21.15.32:0/3453763991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f942815de50 0x7f94280cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:29.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f941e000640 1 -- 172.21.15.32:0/3453763991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94280b3500 msgr2=0x7f94280cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:29.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f941e000640 1 --2- 172.21.15.32:0/3453763991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94280b3500 0x7f94280cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:29.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f941e000640 1 -- 172.21.15.32:0/3453763991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f942815de50 msgr2=0x7f94280cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:29.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f941e000640 1 --2- 172.21.15.32:0/3453763991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f942815de50 0x7f94280cb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:29.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f941e000640 1 -- 172.21.15.32:0/3453763991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94280cbfd0 con 0x7f94280d0700 2024-04-17T09:54:29.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f941e000640 1 --2- 172.21.15.32:0/3453763991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94280d0700 0x7f94280c9ea0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f941000c3a0 tx=0x7f941000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:29.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f941f400640 1 -- 172.21.15.32:0/3453763991 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94100052d0 con 0x7f94280d0700 2024-04-17T09:54:29.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f941f400640 1 -- 172.21.15.32:0/3453763991 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9410005470 con 0x7f94280d0700 2024-04-17T09:54:29.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f941f400640 1 -- 172.21.15.32:0/3453763991 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410007260 con 0x7f94280d0700 2024-04-17T09:54:29.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f942da00640 1 -- 172.21.15.32:0/3453763991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94280cc2b0 con 0x7f94280d0700 2024-04-17T09:54:29.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f942da00640 1 -- 172.21.15.32:0/3453763991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94280cbca0 con 0x7f94280d0700 2024-04-17T09:54:29.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f942da00640 1 -- 172.21.15.32:0/3453763991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9428068a90 con 0x7f94280d0700 2024-04-17T09:54:29.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.081+0000 7f941f400640 1 -- 172.21.15.32:0/3453763991 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9410007400 con 0x7f94280d0700 2024-04-17T09:54:29.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.085+0000 7f941f400640 1 --2- 172.21.15.32:0/3453763991 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9404068380 0x7f940406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:29.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.085+0000 7f941f400640 1 -- 172.21.15.32:0/3453763991 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f9410085dd0 con 0x7f94280d0700 2024-04-17T09:54:29.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.085+0000 7f9426a00640 1 --2- 172.21.15.32:0/3453763991 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9404068380 0x7f940406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:29.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.085+0000 7f9426a00640 1 --2- 172.21.15.32:0/3453763991 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9404068380 0x7f940406a840 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9418006aa0 tx=0x7f9418008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.089+0000 7f941f400640 1 -- 172.21.15.32:0/3453763991 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f9410055d30 con 0x7f94280d0700 2024-04-17T09:54:29.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.253+0000 7f942da00640 1 -- 172.21.15.32:0/3453763991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f94280b4ff0 con 0x7f94280d0700 2024-04-17T09:54:29.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.253+0000 7f941f400640 1 -- 172.21.15.32:0/3453763991 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) ==== 100+0+11 (secure 0 0 0) 0x7f941005a030 con 0x7f94280d0700 2024-04-17T09:54:29.255 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:54:29.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.253+0000 7f942da00640 1 -- 172.21.15.32:0/3453763991 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9404068380 msgr2=0x7f940406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:29.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.253+0000 7f942da00640 1 --2- 172.21.15.32:0/3453763991 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9404068380 0x7f940406a840 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9418006aa0 tx=0x7f9418008040 comp rx=0 tx=0).stop 2024-04-17T09:54:29.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.253+0000 7f942da00640 1 -- 172.21.15.32:0/3453763991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94280d0700 msgr2=0x7f94280c9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:29.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.253+0000 7f942da00640 1 --2- 172.21.15.32:0/3453763991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94280d0700 0x7f94280c9ea0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f941000c3a0 tx=0x7f941000c860 comp rx=0 tx=0).stop 2024-04-17T09:54:29.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.253+0000 7f942da00640 1 -- 172.21.15.32:0/3453763991 shutdown_connections 2024-04-17T09:54:29.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.253+0000 7f942da00640 1 --2- 172.21.15.32:0/3453763991 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9404068380 0x7f940406a840 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:29.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.253+0000 7f942da00640 1 --2- 172.21.15.32:0/3453763991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f942815de50 0x7f94280cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:29.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.253+0000 7f942da00640 1 --2- 172.21.15.32:0/3453763991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94280d0700 0x7f94280c9ea0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:29.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.253+0000 7f942da00640 1 --2- 172.21.15.32:0/3453763991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94280b3500 0x7f94280cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:29.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.253+0000 7f942da00640 1 -- 172.21.15.32:0/3453763991 >> 172.21.15.32:0/3453763991 conn(0x7f94280b40e0 msgr2=0x7f94280ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:29.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.257+0000 7f942da00640 1 -- 172.21.15.32:0/3453763991 shutdown_connections 2024-04-17T09:54:29.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.257+0000 7f942da00640 1 -- 172.21.15.32:0/3453763991 wait complete. 2024-04-17T09:54:29.277 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T09:54:29.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.417+0000 7f9eb3000640 1 Processor -- start 2024-04-17T09:54:29.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.417+0000 7f9eb3000640 1 -- start start 2024-04-17T09:54:29.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.421+0000 7f9eb3000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9eac09b100 0x7f9eac15fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:29.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.421+0000 7f9eb3000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9eac06ce70 con 0x7f9eac0b3340 2024-04-17T09:54:29.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.421+0000 7f9eb3000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9eac15f290 con 0x7f9eac15eeb0 2024-04-17T09:54:29.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.421+0000 7f9eb3000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9eac09b9a0 con 0x7f9eac09b100 2024-04-17T09:54:29.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.421+0000 7f9eb1000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9eac09b100 0x7f9eac15fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:29.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.421+0000 7f9eb1000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9eac09b100 0x7f9eac15fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51330/0 (socket says 172.21.15.32:51330) 2024-04-17T09:54:29.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.421+0000 7f9eb1000640 1 -- 172.21.15.32:0/1483790734 learned_addr learned my addr 172.21.15.32:0/1483790734 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:29.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.421+0000 7f9eab400640 1 --1- 172.21.15.32:0/1483790734 >> v1:172.21.15.32:6789/0 conn(0x7f9eac0b3340 0x7f9eac09a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37214/0 (socket says 172.21.15.32:37214) 2024-04-17T09:54:29.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.421+0000 7f9eb1000640 1 -- 172.21.15.32:0/1483790734 >> v1:172.21.15.144:6789/0 conn(0x7f9eac15eeb0 legacy=0x7f9eac06d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:29.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.421+0000 7f9eb1000640 1 -- 172.21.15.32:0/1483790734 >> v1:172.21.15.32:6789/0 conn(0x7f9eac0b3340 legacy=0x7f9eac09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:29.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.421+0000 7f9eb1000640 1 -- 172.21.15.32:0/1483790734 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eac09bc80 con 0x7f9eac09b100 2024-04-17T09:54:29.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.425+0000 7f9eb1000640 1 --2- 172.21.15.32:0/1483790734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9eac09b100 0x7f9eac15fec0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009e80 tx=0x7f9ea401f200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41e8914f0fcc924 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:29.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.425+0000 7f9eaaa00640 1 -- 172.21.15.32:0/1483790734 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea40b4070 con 0x7f9eac09b100 2024-04-17T09:54:29.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.425+0000 7f9eaaa00640 1 -- 172.21.15.32:0/1483790734 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea40af400 con 0x7f9eac09b100 2024-04-17T09:54:29.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.425+0000 7f9eaaa00640 1 -- 172.21.15.32:0/1483790734 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea40af7d0 con 0x7f9eac09b100 2024-04-17T09:54:29.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.425+0000 7f9eb3000640 1 -- 172.21.15.32:0/1483790734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9eac09b100 msgr2=0x7f9eac15fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:29.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.425+0000 7f9eb3000640 1 --2- 172.21.15.32:0/1483790734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9eac09b100 0x7f9eac15fec0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009e80 tx=0x7f9ea401f200 comp rx=0 tx=0).stop 2024-04-17T09:54:29.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.425+0000 7f9eb3000640 1 -- 172.21.15.32:0/1483790734 shutdown_connections 2024-04-17T09:54:29.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.425+0000 7f9eb3000640 1 --2- 172.21.15.32:0/1483790734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9eac09b100 0x7f9eac15fec0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:29.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.425+0000 7f9eb3000640 1 -- 172.21.15.32:0/1483790734 >> 172.21.15.32:0/1483790734 conn(0x7f9eac0b40a0 msgr2=0x7f9eac0b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:29.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.425+0000 7f9eb3000640 1 -- 172.21.15.32:0/1483790734 shutdown_connections 2024-04-17T09:54:29.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.425+0000 7f9eb3000640 1 -- 172.21.15.32:0/1483790734 wait complete. 2024-04-17T09:54:29.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.425+0000 7f9eb3000640 1 Processor -- start 2024-04-17T09:54:29.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.425+0000 7f9eb3000640 1 -- start start 2024-04-17T09:54:29.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.425+0000 7f9eb3000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9eac09b100 0x7f9eac1f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:29.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.425+0000 7f9eb3000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9eac0b3340 0x7f9eac0cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:29.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.425+0000 7f9eb3000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9eac15eeb0 0x7f9eac0cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:29.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.425+0000 7f9eb3000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac0d09c0 con 0x7f9eac0b3340 2024-04-17T09:54:29.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.425+0000 7f9eb3000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac06d3f0 con 0x7f9eac09b100 2024-04-17T09:54:29.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.425+0000 7f9eb3000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9eac0d0b40 con 0x7f9eac15eeb0 2024-04-17T09:54:29.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.425+0000 7f9eab400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9eac0b3340 0x7f9eac0cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.425+0000 7f9eab400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9eac0b3340 0x7f9eac0cb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51344/0 (socket says 172.21.15.32:51344) 2024-04-17T09:54:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.425+0000 7f9eab400640 1 -- 172.21.15.32:0/215699522 learned_addr learned my addr 172.21.15.32:0/215699522 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.429+0000 7f9eabe00640 1 --2- 172.21.15.32:0/215699522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9eac09b100 0x7f9eac1f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.429+0000 7f9eb1000640 1 --2- 172.21.15.32:0/215699522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9eac15eeb0 0x7f9eac0cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.429+0000 7f9eab400640 1 -- 172.21.15.32:0/215699522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9eac15eeb0 msgr2=0x7f9eac0cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.429+0000 7f9eab400640 1 --2- 172.21.15.32:0/215699522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9eac15eeb0 0x7f9eac0cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.429+0000 7f9eab400640 1 -- 172.21.15.32:0/215699522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9eac09b100 msgr2=0x7f9eac1f42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.429+0000 7f9eab400640 1 --2- 172.21.15.32:0/215699522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9eac09b100 0x7f9eac1f42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.429+0000 7f9eab400640 1 -- 172.21.15.32:0/215699522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eac0cbba0 con 0x7f9eac0b3340 2024-04-17T09:54:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.429+0000 7f9eab400640 1 --2- 172.21.15.32:0/215699522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9eac0b3340 0x7f9eac0cb050 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800c490 tx=0x7f9e9800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.429+0000 7f9ea3e00640 1 -- 172.21.15.32:0/215699522 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e98018070 con 0x7f9eac0b3340 2024-04-17T09:54:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.429+0000 7f9ea3e00640 1 -- 172.21.15.32:0/215699522 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e98007480 con 0x7f9eac0b3340 2024-04-17T09:54:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.429+0000 7f9ea3e00640 1 -- 172.21.15.32:0/215699522 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e980047f0 con 0x7f9eac0b3340 2024-04-17T09:54:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.429+0000 7f9eb3000640 1 -- 172.21.15.32:0/215699522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eac0cbe80 con 0x7f9eac0b3340 2024-04-17T09:54:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.429+0000 7f9eb3000640 1 -- 172.21.15.32:0/215699522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eac0cc440 con 0x7f9eac0b3340 2024-04-17T09:54:29.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.429+0000 7f9eb3000640 1 -- 172.21.15.32:0/215699522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eac068a90 con 0x7f9eac0b3340 2024-04-17T09:54:29.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.429+0000 7f9ea3e00640 1 -- 172.21.15.32:0/215699522 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9e9801b070 con 0x7f9eac0b3340 2024-04-17T09:54:29.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.429+0000 7f9ea3e00640 1 --2- 172.21.15.32:0/215699522 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9e8c068380 0x7f9e8c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:29.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.429+0000 7f9ea3e00640 1 -- 172.21.15.32:0/215699522 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f9e98085cd0 con 0x7f9eac0b3340 2024-04-17T09:54:29.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.429+0000 7f9eabe00640 1 --2- 172.21.15.32:0/215699522 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9e8c068380 0x7f9e8c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:29.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.433+0000 7f9eabe00640 1 --2- 172.21.15.32:0/215699522 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9e8c068380 0x7f9e8c06a840 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0015b0 tx=0x7f9e9c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:29.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.433+0000 7f9ea3e00640 1 -- 172.21.15.32:0/215699522 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f9e980561f0 con 0x7f9eac0b3340 2024-04-17T09:54:29.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.613+0000 7f9eb3000640 1 -- 172.21.15.32:0/215699522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9eac0701d0 con 0x7f9eac0b3340 2024-04-17T09:54:29.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.613+0000 7f9ea3e00640 1 -- 172.21.15.32:0/215699522 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) ==== 80+0+0 (secure 0 0 0) 0x7f9e98055af0 con 0x7f9eac0b3340 2024-04-17T09:54:29.616 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T09:54:29.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.617+0000 7f9eb3000640 1 -- 172.21.15.32:0/215699522 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9e8c068380 msgr2=0x7f9e8c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:29.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.617+0000 7f9eb3000640 1 --2- 172.21.15.32:0/215699522 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9e8c068380 0x7f9e8c06a840 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0015b0 tx=0x7f9e9c009040 comp rx=0 tx=0).stop 2024-04-17T09:54:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.617+0000 7f9eb3000640 1 -- 172.21.15.32:0/215699522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9eac0b3340 msgr2=0x7f9eac0cb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.617+0000 7f9eb3000640 1 --2- 172.21.15.32:0/215699522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9eac0b3340 0x7f9eac0cb050 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800c490 tx=0x7f9e9800c950 comp rx=0 tx=0).stop 2024-04-17T09:54:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.617+0000 7f9eb3000640 1 -- 172.21.15.32:0/215699522 shutdown_connections 2024-04-17T09:54:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.617+0000 7f9eb3000640 1 --2- 172.21.15.32:0/215699522 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9e8c068380 0x7f9e8c06a840 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.617+0000 7f9eb3000640 1 --2- 172.21.15.32:0/215699522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9eac15eeb0 0x7f9eac0cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.617+0000 7f9eb3000640 1 --2- 172.21.15.32:0/215699522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9eac0b3340 0x7f9eac0cb050 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.617+0000 7f9eb3000640 1 --2- 172.21.15.32:0/215699522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9eac09b100 0x7f9eac1f42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.617+0000 7f9eb3000640 1 -- 172.21.15.32:0/215699522 >> 172.21.15.32:0/215699522 conn(0x7f9eac0b40a0 msgr2=0x7f9eac0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.617+0000 7f9eb3000640 1 -- 172.21.15.32:0/215699522 shutdown_connections 2024-04-17T09:54:29.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:29.617+0000 7f9eb3000640 1 -- 172.21.15.32:0/215699522 wait complete. 2024-04-17T09:54:29.638 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2024-04-17T09:54:29.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:29.638 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T09:54:29.657 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[36429]: fuse finished with error 0 and tester_r 0 2024-04-17T09:54:29.660 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:54:35.762 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T09:54:35.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:35.762 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T09:54:35.773 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:54:35.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.969+0000 7f1a6e000640 1 Processor -- start 2024-04-17T09:54:35.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.969+0000 7f1a6e000640 1 -- start start 2024-04-17T09:54:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.969+0000 7f1a6e000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1a680b7fb0 0x7f1a680cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.969+0000 7f1a6e000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1a680ce040 0x7f1a6809a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.969+0000 7f1a6e000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a6809af30 0x7f1a680d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.969+0000 7f1a6e000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1a680b8390 con 0x7f1a680ce040 2024-04-17T09:54:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.969+0000 7f1a6e000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1a680b54d0 con 0x7f1a680b7fb0 2024-04-17T09:54:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.969+0000 7f1a6e000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1a680b57b0 con 0x7f1a6809af30 2024-04-17T09:54:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.969+0000 7f1a5ea00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1a680ce040 0x7f1a6809a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:35.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.969+0000 7f1a5ea00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1a680ce040 0x7f1a6809a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59854/0 (socket says 172.21.15.32:59854) 2024-04-17T09:54:35.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.969+0000 7f1a5ea00640 1 -- 172.21.15.32:0/592199162 learned_addr learned my addr 172.21.15.32:0/592199162 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:35.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.969+0000 7f1a67400640 1 --2- 172.21.15.32:0/592199162 >> v2:172.21.15.144:3300/0 conn(0x7f1a680b7fb0 0x7f1a680cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:35.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.969+0000 7f1a67e00640 1 --2- 172.21.15.32:0/592199162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a6809af30 0x7f1a680d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:35.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.969+0000 7f1a5ea00640 1 -- 172.21.15.32:0/592199162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a6809af30 msgr2=0x7f1a680d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:35.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.969+0000 7f1a5ea00640 1 --2- 172.21.15.32:0/592199162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a6809af30 0x7f1a680d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:35.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.969+0000 7f1a5ea00640 1 -- 172.21.15.32:0/592199162 >> v2:172.21.15.144:3300/0 conn(0x7f1a680b7fb0 msgr2=0x7f1a680cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:35.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.969+0000 7f1a5ea00640 1 --2- 172.21.15.32:0/592199162 >> v2:172.21.15.144:3300/0 conn(0x7f1a680b7fb0 0x7f1a680cdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:35.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.969+0000 7f1a5ea00640 1 -- 172.21.15.32:0/592199162 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a680b5a90 con 0x7f1a680ce040 2024-04-17T09:54:35.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.969+0000 7f1a67e00640 1 --2- 172.21.15.32:0/592199162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a6809af30 0x7f1a680d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:54:35.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.973+0000 7f1a5ea00640 1 --2- 172.21.15.32:0/592199162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a680ce040 0x7f1a6809a9f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1a680731e0 tx=0x7f1a5001c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e330bd5f9d6868 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.973+0000 7f1a66a00640 1 -- 172.21.15.32:0/592199162 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a500a9430 con 0x7f1a680ce040 2024-04-17T09:54:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.973+0000 7f1a66a00640 1 -- 172.21.15.32:0/592199162 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a500ae070 con 0x7f1a680ce040 2024-04-17T09:54:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.973+0000 7f1a66a00640 1 -- 172.21.15.32:0/592199162 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a500a97f0 con 0x7f1a680ce040 2024-04-17T09:54:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.973+0000 7f1a6e000640 1 -- 172.21.15.32:0/592199162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a680ce040 msgr2=0x7f1a6809a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.973+0000 7f1a6e000640 1 --2- 172.21.15.32:0/592199162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a680ce040 0x7f1a6809a9f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1a680731e0 tx=0x7f1a5001c6a0 comp rx=0 tx=0).stop 2024-04-17T09:54:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.973+0000 7f1a6e000640 1 -- 172.21.15.32:0/592199162 shutdown_connections 2024-04-17T09:54:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.973+0000 7f1a6e000640 1 --2- 172.21.15.32:0/592199162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a6809af30 0x7f1a680d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.973+0000 7f1a6e000640 1 --2- 172.21.15.32:0/592199162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a680ce040 0x7f1a6809a9f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.973+0000 7f1a6e000640 1 --2- 172.21.15.32:0/592199162 >> v2:172.21.15.144:3300/0 conn(0x7f1a680b7fb0 0x7f1a680cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.973+0000 7f1a6e000640 1 -- 172.21.15.32:0/592199162 >> 172.21.15.32:0/592199162 conn(0x7f1a680b3fd0 msgr2=0x7f1a680b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.973+0000 7f1a6e000640 1 -- 172.21.15.32:0/592199162 shutdown_connections 2024-04-17T09:54:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.973+0000 7f1a6e000640 1 -- 172.21.15.32:0/592199162 wait complete. 2024-04-17T09:54:35.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a6e000640 1 Processor -- start 2024-04-17T09:54:35.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a6e000640 1 -- start start 2024-04-17T09:54:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a6e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a6809af30 0x7f1a680c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a6e000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a680b7fb0 0x7f1a680af170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a6e000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a680ce040 0x7f1a68157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a6e000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1a680b6620 con 0x7f1a6809af30 2024-04-17T09:54:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a6e000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1a680b64a0 con 0x7f1a680b7fb0 2024-04-17T09:54:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a6e000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1a680b67a0 con 0x7f1a680ce040 2024-04-17T09:54:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a67400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a6809af30 0x7f1a680c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a67400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a6809af30 0x7f1a680c8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59866/0 (socket says 172.21.15.32:59866) 2024-04-17T09:54:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a67400640 1 -- 172.21.15.32:0/1779430865 learned_addr learned my addr 172.21.15.32:0/1779430865 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a67e00640 1 --2- 172.21.15.32:0/1779430865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a680ce040 0x7f1a68157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a5ea00640 1 --2- 172.21.15.32:0/1779430865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a680b7fb0 0x7f1a680af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a67400640 1 -- 172.21.15.32:0/1779430865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a680ce040 msgr2=0x7f1a68157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a67400640 1 --2- 172.21.15.32:0/1779430865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a680ce040 0x7f1a68157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:35.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a67400640 1 -- 172.21.15.32:0/1779430865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a680b7fb0 msgr2=0x7f1a680af170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a67400640 1 --2- 172.21.15.32:0/1779430865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a680b7fb0 0x7f1a680af170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a67400640 1 -- 172.21.15.32:0/1779430865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a680c9ee0 con 0x7f1a6809af30 2024-04-17T09:54:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a5ea00640 1 --2- 172.21.15.32:0/1779430865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a680b7fb0 0x7f1a680af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a67400640 1 --2- 172.21.15.32:0/1779430865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a6809af30 0x7f1a680c8400 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800d800 tx=0x7f1a5800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a5fe00640 1 -- 172.21.15.32:0/1779430865 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5800b6c0 con 0x7f1a6809af30 2024-04-17T09:54:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a5fe00640 1 -- 172.21.15.32:0/1779430865 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a58019070 con 0x7f1a6809af30 2024-04-17T09:54:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a5fe00640 1 -- 172.21.15.32:0/1779430865 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5800ba60 con 0x7f1a6809af30 2024-04-17T09:54:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a6e000640 1 -- 172.21.15.32:0/1779430865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a680ca1c0 con 0x7f1a6809af30 2024-04-17T09:54:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.977+0000 7f1a6e000640 1 -- 172.21.15.32:0/1779430865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a680ca780 con 0x7f1a6809af30 2024-04-17T09:54:35.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.981+0000 7f1a6e000640 1 -- 172.21.15.32:0/1779430865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a68068a90 con 0x7f1a6809af30 2024-04-17T09:54:35.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.981+0000 7f1a5fe00640 1 -- 172.21.15.32:0/1779430865 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1a5800bc00 con 0x7f1a6809af30 2024-04-17T09:54:35.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.981+0000 7f1a5fe00640 1 --2- 172.21.15.32:0/1779430865 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1a44068380 0x7f1a4406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:35.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.981+0000 7f1a5fe00640 1 -- 172.21.15.32:0/1779430865 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f1a58086da0 con 0x7f1a6809af30 2024-04-17T09:54:35.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.981+0000 7f1a5ea00640 1 --2- 172.21.15.32:0/1779430865 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1a44068380 0x7f1a4406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:35.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.981+0000 7f1a5ea00640 1 --2- 172.21.15.32:0/1779430865 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1a44068380 0x7f1a4406a840 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1a500ad000 tx=0x7f1a500b4040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:35.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:35.985+0000 7f1a5fe00640 1 -- 172.21.15.32:0/1779430865 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f1a58057350 con 0x7f1a6809af30 2024-04-17T09:54:36.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.185+0000 7f1a6e000640 1 -- 172.21.15.32:0/1779430865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a680c91d0 con 0x7f1a6809af30 2024-04-17T09:54:36.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.185+0000 7f1a5fe00640 1 -- 172.21.15.32:0/1779430865 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5199 (secure 0 0 0) 0x7f1a58056c50 con 0x7f1a6809af30 2024-04-17T09:54:36.187 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:36.187 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2127207201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2127207201},{"type":"v1","addr":"172.21.15.144:6835","nonce":2127207201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5110,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3782699135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3782699135},{"type":"v1","addr":"172.21.15.144:6834","nonce":3782699135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/385811857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":385811857},{"type":"v1","addr":"172.21.15.32:6836","nonce":385811857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:54:16.745151+0000","modified":"2024-04-17T09:54:19.761481+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5217":{"gid":5217,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/925293029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":925293029},{"type":"v1","addr":"172.21.15.32:6837","nonce":925293029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5217,"qdb_cluster":[5217]},"id":2}]} 2024-04-17T09:54:36.187 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 15 2024-04-17T09:54:36.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.189+0000 7f1a6e000640 1 -- 172.21.15.32:0/1779430865 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1a44068380 msgr2=0x7f1a4406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:36.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.189+0000 7f1a6e000640 1 --2- 172.21.15.32:0/1779430865 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1a44068380 0x7f1a4406a840 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1a500ad000 tx=0x7f1a500b4040 comp rx=0 tx=0).stop 2024-04-17T09:54:36.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.189+0000 7f1a6e000640 1 -- 172.21.15.32:0/1779430865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a6809af30 msgr2=0x7f1a680c8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:36.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.189+0000 7f1a6e000640 1 --2- 172.21.15.32:0/1779430865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a6809af30 0x7f1a680c8400 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800d800 tx=0x7f1a5800dcc0 comp rx=0 tx=0).stop 2024-04-17T09:54:36.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.189+0000 7f1a6e000640 1 -- 172.21.15.32:0/1779430865 shutdown_connections 2024-04-17T09:54:36.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.189+0000 7f1a6e000640 1 --2- 172.21.15.32:0/1779430865 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1a44068380 0x7f1a4406a840 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:36.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.189+0000 7f1a6e000640 1 --2- 172.21.15.32:0/1779430865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a680ce040 0x7f1a68157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:36.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.189+0000 7f1a6e000640 1 --2- 172.21.15.32:0/1779430865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a680b7fb0 0x7f1a680af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:36.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.189+0000 7f1a6e000640 1 --2- 172.21.15.32:0/1779430865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a6809af30 0x7f1a680c8400 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:36.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.189+0000 7f1a6e000640 1 -- 172.21.15.32:0/1779430865 >> 172.21.15.32:0/1779430865 conn(0x7f1a680b3fd0 msgr2=0x7f1a68071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:36.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.189+0000 7f1a6e000640 1 -- 172.21.15.32:0/1779430865 shutdown_connections 2024-04-17T09:54:36.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.189+0000 7f1a6e000640 1 -- 172.21.15.32:0/1779430865 wait complete. 2024-04-17T09:54:36.209 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-17T09:54:36.379 INFO:teuthology.orchestra.run.smithi032.stdout:{"message":"","return_code":0} 2024-04-17T09:54:36.380 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-17T09:54:36.380 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:54:36.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.521+0000 7fbffb200640 1 Processor -- start 2024-04-17T09:54:36.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.521+0000 7fbffb200640 1 -- start start 2024-04-17T09:54:36.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.521+0000 7fbffb200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbffc15fe10 0x7fbffc0b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:36.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.521+0000 7fbffb200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbffc09b100 0x7fbffc0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:36.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.521+0000 7fbffb200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbffc0cdb70 con 0x7fbffc0b1800 2024-04-17T09:54:36.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.521+0000 7fbffb200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fbffc0cde50 con 0x7fbffc15fe10 2024-04-17T09:54:36.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.521+0000 7fbffb200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbffc0ce130 con 0x7fbffc09b100 2024-04-17T09:54:36.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.521+0000 7fbffa800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbffc09b100 0x7fbffc0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:36.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.521+0000 7fbff9e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbffc15fe10 0x7fbffc0b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:36.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.521+0000 7fbffa800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbffc09b100 0x7fbffc0b2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:36388/0 (socket says 172.21.15.32:36388) 2024-04-17T09:54:36.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.521+0000 7fbffa800640 1 -- 172.21.15.32:0/1469587488 learned_addr learned my addr 172.21.15.32:0/1469587488 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:36.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.521+0000 7fbff8a00640 1 -- 172.21.15.32:0/1469587488 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2992709050 0 0) 0x7fbffc0cdb70 con 0x7fbffc0b1800 2024-04-17T09:54:36.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.521+0000 7fbff9e00640 1 -- 172.21.15.32:0/1469587488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbffc09b100 msgr2=0x7fbffc0b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:36.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.521+0000 7fbff9e00640 1 --2- 172.21.15.32:0/1469587488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbffc09b100 0x7fbffc0b2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:36.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.521+0000 7fbff9e00640 1 -- 172.21.15.32:0/1469587488 >> v1:172.21.15.144:6789/0 conn(0x7fbffc0b1800 legacy=0x7fbffc09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:36.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.521+0000 7fbff9e00640 1 -- 172.21.15.32:0/1469587488 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbffc0ce410 con 0x7fbffc15fe10 2024-04-17T09:54:36.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.521+0000 7fbffa800640 1 --2- 172.21.15.32:0/1469587488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbffc09b100 0x7fbffc0b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:54:36.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.525+0000 7fbff9e00640 1 --2- 172.21.15.32:0/1469587488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbffc15fe10 0x7fbffc0b12c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0097f0 tx=0x7fbfec01ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e52b1b914d94e81e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:36.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.525+0000 7fbff8a00640 1 -- 172.21.15.32:0/1469587488 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0b5070 con 0x7fbffc15fe10 2024-04-17T09:54:36.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.525+0000 7fbff8a00640 1 -- 172.21.15.32:0/1469587488 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfec0a8990 con 0x7fbffc15fe10 2024-04-17T09:54:36.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.525+0000 7fbff8a00640 1 -- 172.21.15.32:0/1469587488 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0a8d60 con 0x7fbffc15fe10 2024-04-17T09:54:36.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.525+0000 7fbffb200640 1 -- 172.21.15.32:0/1469587488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbffc15fe10 msgr2=0x7fbffc0b12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:36.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.525+0000 7fbffb200640 1 --2- 172.21.15.32:0/1469587488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbffc15fe10 0x7fbffc0b12c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0097f0 tx=0x7fbfec01ec90 comp rx=0 tx=0).stop 2024-04-17T09:54:36.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.525+0000 7fbffb200640 1 -- 172.21.15.32:0/1469587488 shutdown_connections 2024-04-17T09:54:36.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.525+0000 7fbffb200640 1 --2- 172.21.15.32:0/1469587488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbffc09b100 0x7fbffc0b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.525+0000 7fbffb200640 1 --2- 172.21.15.32:0/1469587488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbffc15fe10 0x7fbffc0b12c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.525+0000 7fbffb200640 1 -- 172.21.15.32:0/1469587488 >> 172.21.15.32:0/1469587488 conn(0x7fbffc0b3ec0 msgr2=0x7fbffc0b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.525+0000 7fbffb200640 1 -- 172.21.15.32:0/1469587488 shutdown_connections 2024-04-17T09:54:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.525+0000 7fbffb200640 1 -- 172.21.15.32:0/1469587488 wait complete. 2024-04-17T09:54:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbffb200640 1 Processor -- start 2024-04-17T09:54:36.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbffb200640 1 -- start start 2024-04-17T09:54:36.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbffb200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbffc09b100 0x7fbffc0b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:36.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbffb200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbffc0b1800 0x7fbffc09faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:36.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbffb200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbffc15fe10 0x7fbffc1f2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbffb200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc09bb90 con 0x7fbffc0b1800 2024-04-17T09:54:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbffb200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc09ba10 con 0x7fbffc09b100 2024-04-17T09:54:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbffb200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbffc0b2660 con 0x7fbffc15fe10 2024-04-17T09:54:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbff9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbffc0b1800 0x7fbffc09faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbff9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbffc0b1800 0x7fbffc09faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59876/0 (socket says 172.21.15.32:59876) 2024-04-17T09:54:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbff9400640 1 -- 172.21.15.32:0/3843087374 learned_addr learned my addr 172.21.15.32:0/3843087374 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbffa800640 1 --2- 172.21.15.32:0/3843087374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbffc15fe10 0x7fbffc1f2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbff9e00640 1 --2- 172.21.15.32:0/3843087374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbffc09b100 0x7fbffc0b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbff9400640 1 -- 172.21.15.32:0/3843087374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbffc15fe10 msgr2=0x7fbffc1f2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbff9400640 1 --2- 172.21.15.32:0/3843087374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbffc15fe10 0x7fbffc1f2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbff9400640 1 -- 172.21.15.32:0/3843087374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbffc09b100 msgr2=0x7fbffc0b78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbff9400640 1 --2- 172.21.15.32:0/3843087374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbffc09b100 0x7fbffc0b78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbff9400640 1 -- 172.21.15.32:0/3843087374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbffc1f2900 con 0x7fbffc0b1800 2024-04-17T09:54:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbff9e00640 1 --2- 172.21.15.32:0/3843087374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbffc09b100 0x7fbffc0b78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbffa800640 1 --2- 172.21.15.32:0/3843087374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbffc15fe10 0x7fbffc1f2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbff9400640 1 --2- 172.21.15.32:0/3843087374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbffc0b1800 0x7fbffc09faa0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800c480 tx=0x7fbfe800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbff2000640 1 -- 172.21.15.32:0/3843087374 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8005480 con 0x7fbffc0b1800 2024-04-17T09:54:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbff2000640 1 -- 172.21.15.32:0/3843087374 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfe8003600 con 0x7fbffc0b1800 2024-04-17T09:54:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbff2000640 1 -- 172.21.15.32:0/3843087374 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe80039f0 con 0x7fbffc0b1800 2024-04-17T09:54:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbffb200640 1 -- 172.21.15.32:0/3843087374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbffc1f2be0 con 0x7fbffc0b1800 2024-04-17T09:54:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbffb200640 1 -- 172.21.15.32:0/3843087374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbffc0ca770 con 0x7fbffc0b1800 2024-04-17T09:54:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.529+0000 7fbffb200640 1 -- 172.21.15.32:0/3843087374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbffc0ca560 con 0x7fbffc0b1800 2024-04-17T09:54:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.533+0000 7fbff2000640 1 -- 172.21.15.32:0/3843087374 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbfe8003b90 con 0x7fbffc0b1800 2024-04-17T09:54:36.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.533+0000 7fbff2000640 1 --2- 172.21.15.32:0/3843087374 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbfdc068380 0x7fbfdc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:36.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.533+0000 7fbff2000640 1 -- 172.21.15.32:0/3843087374 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fbfe8086720 con 0x7fbffc0b1800 2024-04-17T09:54:36.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.533+0000 7fbff9e00640 1 --2- 172.21.15.32:0/3843087374 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbfdc068380 0x7fbfdc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:36.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.533+0000 7fbff9e00640 1 --2- 172.21.15.32:0/3843087374 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbfdc068380 0x7fbfdc06a840 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0b2000 tx=0x7fbfec0aa070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:36.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.537+0000 7fbff2000640 1 -- 172.21.15.32:0/3843087374 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fbfe8056cd0 con 0x7fbffc0b1800 2024-04-17T09:54:36.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.737+0000 7fbffb200640 1 -- 172.21.15.32:0/3843087374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbffc04fd00 con 0x7fbffc0b1800 2024-04-17T09:54:36.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.737+0000 7fbff2000640 1 -- 172.21.15.32:0/3843087374 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5199 (secure 0 0 0) 0x7fbfe80565d0 con 0x7fbffc0b1800 2024-04-17T09:54:36.738 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:36.739 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2127207201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2127207201},{"type":"v1","addr":"172.21.15.144:6835","nonce":2127207201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5110,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3782699135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3782699135},{"type":"v1","addr":"172.21.15.144:6834","nonce":3782699135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/385811857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":385811857},{"type":"v1","addr":"172.21.15.32:6836","nonce":385811857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:54:16.745151+0000","modified":"2024-04-17T09:54:19.761481+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5217":{"gid":5217,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/925293029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":925293029},{"type":"v1","addr":"172.21.15.32:6837","nonce":925293029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5217,"qdb_cluster":[5217]},"id":2}]} 2024-04-17T09:54:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 15 2024-04-17T09:54:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.737+0000 7fbffb200640 1 -- 172.21.15.32:0/3843087374 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbfdc068380 msgr2=0x7fbfdc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.737+0000 7fbffb200640 1 --2- 172.21.15.32:0/3843087374 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbfdc068380 0x7fbfdc06a840 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0b2000 tx=0x7fbfec0aa070 comp rx=0 tx=0).stop 2024-04-17T09:54:36.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.737+0000 7fbffb200640 1 -- 172.21.15.32:0/3843087374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbffc0b1800 msgr2=0x7fbffc09faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:36.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.737+0000 7fbffb200640 1 --2- 172.21.15.32:0/3843087374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbffc0b1800 0x7fbffc09faa0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800c480 tx=0x7fbfe800c940 comp rx=0 tx=0).stop 2024-04-17T09:54:36.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.737+0000 7fbffb200640 1 -- 172.21.15.32:0/3843087374 shutdown_connections 2024-04-17T09:54:36.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.737+0000 7fbffb200640 1 --2- 172.21.15.32:0/3843087374 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbfdc068380 0x7fbfdc06a840 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:36.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.737+0000 7fbffb200640 1 --2- 172.21.15.32:0/3843087374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbffc15fe10 0x7fbffc1f2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:36.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.737+0000 7fbffb200640 1 --2- 172.21.15.32:0/3843087374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbffc0b1800 0x7fbffc09faa0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:36.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.737+0000 7fbffb200640 1 --2- 172.21.15.32:0/3843087374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbffc09b100 0x7fbffc0b78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:36.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.737+0000 7fbffb200640 1 -- 172.21.15.32:0/3843087374 >> 172.21.15.32:0/3843087374 conn(0x7fbffc0b3ec0 msgr2=0x7fbffc0ccdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:36.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.741+0000 7fbffb200640 1 -- 172.21.15.32:0/3843087374 shutdown_connections 2024-04-17T09:54:36.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:36.741+0000 7fbffb200640 1 -- 172.21.15.32:0/3843087374 wait complete. 2024-04-17T09:54:36.760 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parentdir mytag 2024-04-17T09:54:36.925 INFO:teuthology.orchestra.run.smithi032.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-04-17T09:54:36.926 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-04-17T09:54:43.173 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:54:43.177 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:54:46.927 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-04-17T09:54:46.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.977+0000 7f2a8442c080 1 Processor -- start 2024-04-17T09:54:46.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.977+0000 7f2a8442c080 1 -- start start 2024-04-17T09:54:46.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.977+0000 7f2a8442c080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5607682c4100 0x560768307060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:46.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a8442c080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56076830e230 0x560768397da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:46.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a8442c080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56076839b270 0x560768396ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:46.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a8442c080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56076826af90 con 0x56076830e230 2024-04-17T09:54:46.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a8442c080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56076826efd0 con 0x56076839b270 2024-04-17T09:54:46.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a8442c080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x56076839bd60 con 0x5607682c4100 2024-04-17T09:54:46.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a7fc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56076830e230 0x560768397da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:46.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a7fc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56076830e230 0x560768397da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51348/0 (socket says 172.21.15.32:51348) 2024-04-17T09:54:46.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a7fc00640 1 -- 172.21.15.32:0/75380831 learned_addr learned my addr 172.21.15.32:0/75380831 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:46.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a80600640 1 --2- 172.21.15.32:0/75380831 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5607682c4100 0x560768307060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:46.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a81000640 1 --2- 172.21.15.32:0/75380831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56076839b270 0x560768396ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:46.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a7fc00640 1 -- 172.21.15.32:0/75380831 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5607682c4100 msgr2=0x560768307060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:46.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a7fc00640 1 --2- 172.21.15.32:0/75380831 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5607682c4100 0x560768307060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:46.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a7fc00640 1 -- 172.21.15.32:0/75380831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56076839b270 msgr2=0x560768396ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:46.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a7fc00640 1 --2- 172.21.15.32:0/75380831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56076839b270 0x560768396ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:46.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a7fc00640 1 -- 172.21.15.32:0/75380831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560768395d80 con 0x56076830e230 2024-04-17T09:54:46.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a7fc00640 1 --2- 172.21.15.32:0/75380831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56076830e230 0x560768397da0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00b330 tx=0x7f2a6c00b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43602d60b106cb1a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a7e800640 1 -- 172.21.15.32:0/75380831 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c00d5d0 con 0x56076830e230 2024-04-17T09:54:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a7e800640 1 -- 172.21.15.32:0/75380831 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6c008070 con 0x56076830e230 2024-04-17T09:54:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a7e800640 1 -- 172.21.15.32:0/75380831 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c00d9b0 con 0x56076830e230 2024-04-17T09:54:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a8442c080 1 -- 172.21.15.32:0/75380831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56076830e230 msgr2=0x560768397da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a8442c080 1 --2- 172.21.15.32:0/75380831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56076830e230 0x560768397da0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00b330 tx=0x7f2a6c00b7f0 comp rx=0 tx=0).stop 2024-04-17T09:54:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a8442c080 1 -- 172.21.15.32:0/75380831 shutdown_connections 2024-04-17T09:54:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a8442c080 1 --2- 172.21.15.32:0/75380831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56076839b270 0x560768396ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a8442c080 1 --2- 172.21.15.32:0/75380831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56076830e230 0x560768397da0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a8442c080 1 --2- 172.21.15.32:0/75380831 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5607682c4100 0x560768307060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a8442c080 1 -- 172.21.15.32:0/75380831 >> 172.21.15.32:0/75380831 conn(0x5607682c4620 msgr2=0x5607682f56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a8442c080 1 -- 172.21.15.32:0/75380831 shutdown_connections 2024-04-17T09:54:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a8442c080 1 -- 172.21.15.32:0/75380831 wait complete. 2024-04-17T09:54:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a8442c080 1 Processor -- start 2024-04-17T09:54:46.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.981+0000 7f2a8442c080 1 -- start start 2024-04-17T09:54:46.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a8442c080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607682c4100 0x56076837d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a8442c080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56076830e230 0x56076830c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a8442c080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56076839b270 0x5607682f2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a8442c080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5607683962c0 con 0x56076839b270 2024-04-17T09:54:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a8442c080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560768396020 con 0x5607682c4100 2024-04-17T09:54:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a8442c080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x560768396560 con 0x56076830e230 2024-04-17T09:54:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a81000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56076839b270 0x5607682f2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a81000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56076839b270 0x5607682f2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51358/0 (socket says 172.21.15.32:51358) 2024-04-17T09:54:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a81000640 1 -- 172.21.15.32:0/3256894829 learned_addr learned my addr 172.21.15.32:0/3256894829 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a7fc00640 1 --2- 172.21.15.32:0/3256894829 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56076830e230 0x56076830c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a80600640 1 --2- 172.21.15.32:0/3256894829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607682c4100 0x56076837d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a81000640 1 -- 172.21.15.32:0/3256894829 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56076830e230 msgr2=0x56076830c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a81000640 1 --2- 172.21.15.32:0/3256894829 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56076830e230 0x56076830c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a81000640 1 -- 172.21.15.32:0/3256894829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607682c4100 msgr2=0x56076837d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a81000640 1 --2- 172.21.15.32:0/3256894829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607682c4100 0x56076837d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a81000640 1 -- 172.21.15.32:0/3256894829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607683073e0 con 0x56076839b270 2024-04-17T09:54:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a80600640 1 --2- 172.21.15.32:0/3256894829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607682c4100 0x56076837d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a81000640 1 --2- 172.21.15.32:0/3256894829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56076839b270 0x5607682f2e20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2a780ae040 tx=0x7f2a78017d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a7d400640 1 -- 172.21.15.32:0/3256894829 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a780b9070 con 0x56076839b270 2024-04-17T09:54:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a7d400640 1 -- 172.21.15.32:0/3256894829 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a780b5680 con 0x56076839b270 2024-04-17T09:54:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a7d400640 1 -- 172.21.15.32:0/3256894829 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a780b0040 con 0x56076839b270 2024-04-17T09:54:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a8442c080 1 -- 172.21.15.32:0/3256894829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560768307680 con 0x56076839b270 2024-04-17T09:54:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a8442c080 1 -- 172.21.15.32:0/3256894829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560768307bc0 con 0x56076839b270 2024-04-17T09:54:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.985+0000 7f2a7d400640 1 -- 172.21.15.32:0/3256894829 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2a780bc070 con 0x56076839b270 2024-04-17T09:54:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.989+0000 7f2a7d400640 1 --2- 172.21.15.32:0/3256894829 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2a60066d90 0x7f2a60069250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:46.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.989+0000 7f2a7d400640 1 -- 172.21.15.32:0/3256894829 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f2a78120220 con 0x56076839b270 2024-04-17T09:54:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.989+0000 7f2a80600640 1 --2- 172.21.15.32:0/3256894829 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2a60066d90 0x7f2a60069250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.989+0000 7f2a8442c080 1 --2- 172.21.15.32:0/3256894829 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x56076837d9d0 0x560768308300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.989+0000 7f2a8442c080 1 -- 172.21.15.32:0/3256894829 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) -- 0x560768308840 con 0x56076837d9d0 2024-04-17T09:54:46.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.989+0000 7f2a7fc00640 1 --2- 172.21.15.32:0/3256894829 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x56076837d9d0 0x560768308300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.989+0000 7f2a80600640 1 --2- 172.21.15.32:0/3256894829 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2a60066d90 0x7f2a60069250 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2a70003470 tx=0x7f2a70003050 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.989+0000 7f2a7fc00640 1 --2- 172.21.15.32:0/3256894829 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x56076837d9d0 0x560768308300 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.989+0000 7f2a7fc00640 1 -- 172.21.15.32:0/3256894829 <== osd.3 v2:172.21.15.32:6824/2182899298 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f2a6c00d750 con 0x56076837d9d0 2024-04-17T09:54:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.989+0000 7f2a8442c080 1 -- 172.21.15.32:0/3256894829 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x56076837d9d0 msgr2=0x560768308300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.989+0000 7f2a8442c080 1 --2- 172.21.15.32:0/3256894829 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x56076837d9d0 0x560768308300 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.989+0000 7f2a8442c080 1 -- 172.21.15.32:0/3256894829 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2a60066d90 msgr2=0x7f2a60069250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.989+0000 7f2a8442c080 1 --2- 172.21.15.32:0/3256894829 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2a60066d90 0x7f2a60069250 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2a70003470 tx=0x7f2a70003050 comp rx=0 tx=0).stop 2024-04-17T09:54:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.989+0000 7f2a8442c080 1 -- 172.21.15.32:0/3256894829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56076839b270 msgr2=0x5607682f2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.989+0000 7f2a8442c080 1 --2- 172.21.15.32:0/3256894829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56076839b270 0x5607682f2e20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2a780ae040 tx=0x7f2a78017d80 comp rx=0 tx=0).stop 2024-04-17T09:54:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.989+0000 7f2a81000640 1 -- 172.21.15.32:0/3256894829 reap_dead start 2024-04-17T09:54:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.989+0000 7f2a8442c080 1 -- 172.21.15.32:0/3256894829 shutdown_connections 2024-04-17T09:54:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.989+0000 7f2a8442c080 1 -- 172.21.15.32:0/3256894829 >> 172.21.15.32:0/3256894829 conn(0x5607682c4620 msgr2=0x56076830d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.989+0000 7f2a8442c080 1 -- 172.21.15.32:0/3256894829 shutdown_connections 2024-04-17T09:54:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:46.989+0000 7f2a8442c080 1 -- 172.21.15.32:0/3256894829 wait complete. 2024-04-17T09:54:46.997 INFO:teuthology.orchestra.run.smithi032.stdout:mytag 2024-04-17T09:54:46.997 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-04-17T09:54:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.045+0000 7f282692f080 1 Processor -- start 2024-04-17T09:54:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.045+0000 7f282692f080 1 -- start start 2024-04-17T09:54:47.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.045+0000 7f282692f080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7f0792700 0x55b7f0717600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.045+0000 7f282692f080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f0792ae0 0x55b7f07941e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.045+0000 7f282692f080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7f0795dc0 0x55b7f06d79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.045+0000 7f282692f080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b7f07adae0 con 0x55b7f0795dc0 2024-04-17T09:54:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.045+0000 7f282692f080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b7f067df90 con 0x55b7f0792ae0 2024-04-17T09:54:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.045+0000 7f282692f080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b7f07adc60 con 0x55b7f0792700 2024-04-17T09:54:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.045+0000 7f2824600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7f0795dc0 0x55b7f06d79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.045+0000 7f2823c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7f0792700 0x55b7f0717600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.045+0000 7f2824600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7f0795dc0 0x55b7f06d79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51374/0 (socket says 172.21.15.32:51374) 2024-04-17T09:54:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.045+0000 7f2823c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7f0792700 0x55b7f0717600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:37654/0 (socket says 172.21.15.32:37654) 2024-04-17T09:54:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.045+0000 7f2824600640 1 -- 172.21.15.32:0/3111382622 learned_addr learned my addr 172.21.15.32:0/3111382622 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.045+0000 7f281be00640 1 --2- 172.21.15.32:0/3111382622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f0792ae0 0x55b7f07941e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.045+0000 7f2824600640 1 -- 172.21.15.32:0/3111382622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7f0792700 msgr2=0x55b7f0717600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.045+0000 7f2824600640 1 --2- 172.21.15.32:0/3111382622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7f0792700 0x55b7f0717600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.045+0000 7f2824600640 1 -- 172.21.15.32:0/3111382622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f0792ae0 msgr2=0x55b7f07941e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.045+0000 7f2824600640 1 --2- 172.21.15.32:0/3111382622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f0792ae0 0x55b7f07941e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.045+0000 7f2824600640 1 -- 172.21.15.32:0/3111382622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7f0602950 con 0x55b7f0795dc0 2024-04-17T09:54:47.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.045+0000 7f2824600640 1 --2- 172.21.15.32:0/3111382622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7f0795dc0 0x55b7f06d79c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f281c00b790 tx=0x7f281c00bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a85294d0ea73e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:47.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.045+0000 7f2822800640 1 -- 172.21.15.32:0/3111382622 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c016020 con 0x55b7f0795dc0 2024-04-17T09:54:47.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.045+0000 7f2822800640 1 -- 172.21.15.32:0/3111382622 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f281c00d960 con 0x55b7f0795dc0 2024-04-17T09:54:47.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f2822800640 1 -- 172.21.15.32:0/3111382622 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c00bea0 con 0x55b7f0795dc0 2024-04-17T09:54:47.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f282692f080 1 -- 172.21.15.32:0/3111382622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7f0795dc0 msgr2=0x55b7f06d79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f282692f080 1 --2- 172.21.15.32:0/3111382622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7f0795dc0 0x55b7f06d79c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f281c00b790 tx=0x7f281c00bc50 comp rx=0 tx=0).stop 2024-04-17T09:54:47.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f282692f080 1 -- 172.21.15.32:0/3111382622 shutdown_connections 2024-04-17T09:54:47.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f282692f080 1 --2- 172.21.15.32:0/3111382622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7f0795dc0 0x55b7f06d79c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f282692f080 1 --2- 172.21.15.32:0/3111382622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f0792ae0 0x55b7f07941e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f282692f080 1 --2- 172.21.15.32:0/3111382622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7f0792700 0x55b7f0717600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f282692f080 1 -- 172.21.15.32:0/3111382622 >> 172.21.15.32:0/3111382622 conn(0x55b7f0703760 msgr2=0x55b7f0791920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:47.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f282692f080 1 -- 172.21.15.32:0/3111382622 shutdown_connections 2024-04-17T09:54:47.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f282692f080 1 -- 172.21.15.32:0/3111382622 wait complete. 2024-04-17T09:54:47.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f282692f080 1 Processor -- start 2024-04-17T09:54:47.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f282692f080 1 -- start start 2024-04-17T09:54:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f282692f080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7f0792700 0x55b7f079b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f282692f080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7f0792ae0 0x55b7f07a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f282692f080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f0795dc0 0x55b7f06d6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f282692f080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b7f06d9b00 con 0x55b7f0792700 2024-04-17T09:54:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f282692f080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b7f06d98a0 con 0x55b7f0795dc0 2024-04-17T09:54:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f282692f080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b7f06d58c0 con 0x55b7f0792ae0 2024-04-17T09:54:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f2823c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7f0792700 0x55b7f079b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f2823c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7f0792700 0x55b7f079b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51382/0 (socket says 172.21.15.32:51382) 2024-04-17T09:54:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f2823c00640 1 -- 172.21.15.32:0/2847135764 learned_addr learned my addr 172.21.15.32:0/2847135764 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f281be00640 1 --2- 172.21.15.32:0/2847135764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7f0792ae0 0x55b7f07a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f2824600640 1 --2- 172.21.15.32:0/2847135764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f0795dc0 0x55b7f06d6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f2823c00640 1 -- 172.21.15.32:0/2847135764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7f0792ae0 msgr2=0x55b7f07a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f2823c00640 1 --2- 172.21.15.32:0/2847135764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7f0792ae0 0x55b7f07a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f2823c00640 1 -- 172.21.15.32:0/2847135764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f0795dc0 msgr2=0x55b7f06d6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f2823c00640 1 --2- 172.21.15.32:0/2847135764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f0795dc0 0x55b7f06d6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f2823c00640 1 -- 172.21.15.32:0/2847135764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7f079c080 con 0x55b7f0792700 2024-04-17T09:54:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f2823c00640 1 --2- 172.21.15.32:0/2847135764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7f0792700 0x55b7f079b690 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f281401b790 tx=0x7f281401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f2821400640 1 -- 172.21.15.32:0/2847135764 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28140c7070 con 0x55b7f0792700 2024-04-17T09:54:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f2821400640 1 -- 172.21.15.32:0/2847135764 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28140b7780 con 0x55b7f0792700 2024-04-17T09:54:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f282692f080 1 -- 172.21.15.32:0/2847135764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b7f079c320 con 0x55b7f0792700 2024-04-17T09:54:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.049+0000 7f2821400640 1 -- 172.21.15.32:0/2847135764 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28140b7b30 con 0x55b7f0792700 2024-04-17T09:54:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.053+0000 7f282692f080 1 -- 172.21.15.32:0/2847135764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7f079c860 con 0x55b7f0792700 2024-04-17T09:54:47.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.053+0000 7f2821400640 1 -- 172.21.15.32:0/2847135764 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f28140ca070 con 0x55b7f0792700 2024-04-17T09:54:47.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.053+0000 7f2821400640 1 --2- 172.21.15.32:0/2847135764 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2800066d90 0x7f2800069250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.053+0000 7f2821400640 1 -- 172.21.15.32:0/2847135764 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f2814127990 con 0x55b7f0792700 2024-04-17T09:54:47.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.053+0000 7f281be00640 1 --2- 172.21.15.32:0/2847135764 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2800066d90 0x7f2800069250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.053+0000 7f282692f080 1 --2- 172.21.15.32:0/2847135764 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55b7f079bbd0 0x55b7f083ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.053+0000 7f282692f080 1 -- 172.21.15.32:0/2847135764 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:1 4.7 4:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) -- 0x55b7f0791300 con 0x55b7f079bbd0 2024-04-17T09:54:47.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.053+0000 7f2824600640 1 --2- 172.21.15.32:0/2847135764 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55b7f079bbd0 0x55b7f083ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.053+0000 7f281be00640 1 --2- 172.21.15.32:0/2847135764 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2800066d90 0x7f2800069250 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f280c000c10 tx=0x7f280c001800 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:47.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.053+0000 7f2824600640 1 --2- 172.21.15.32:0/2847135764 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55b7f079bbd0 0x55b7f083ec60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:47.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.057+0000 7f2824600640 1 -- 172.21.15.32:0/2847135764 <== osd.1 v2:172.21.15.32:6808/2454993455 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f281c00dc10 con 0x55b7f079bbd0 2024-04-17T09:54:47.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.057+0000 7f282692f080 1 -- 172.21.15.32:0/2847135764 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55b7f079bbd0 msgr2=0x55b7f083ec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.057+0000 7f282692f080 1 --2- 172.21.15.32:0/2847135764 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55b7f079bbd0 0x55b7f083ec60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.057+0000 7f282692f080 1 -- 172.21.15.32:0/2847135764 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2800066d90 msgr2=0x7f2800069250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.057+0000 7f282692f080 1 --2- 172.21.15.32:0/2847135764 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2800066d90 0x7f2800069250 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f280c000c10 tx=0x7f280c001800 comp rx=0 tx=0).stop 2024-04-17T09:54:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.057+0000 7f282692f080 1 -- 172.21.15.32:0/2847135764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7f0792700 msgr2=0x55b7f079b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.057+0000 7f282692f080 1 --2- 172.21.15.32:0/2847135764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7f0792700 0x55b7f079b690 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f281401b790 tx=0x7f281401c3a0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.057+0000 7f2824600640 1 -- 172.21.15.32:0/2847135764 reap_dead start 2024-04-17T09:54:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.057+0000 7f282692f080 1 -- 172.21.15.32:0/2847135764 shutdown_connections 2024-04-17T09:54:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.057+0000 7f282692f080 1 -- 172.21.15.32:0/2847135764 >> 172.21.15.32:0/2847135764 conn(0x55b7f0703760 msgr2=0x55b7f071d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.057+0000 7f282692f080 1 -- 172.21.15.32:0/2847135764 shutdown_connections 2024-04-17T09:54:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.057+0000 7f282692f080 1 -- 172.21.15.32:0/2847135764 wait complete. 2024-04-17T09:54:47.062 INFO:teuthology.orchestra.run.smithi032.stdout:mytag 2024-04-17T09:54:47.062 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-17T09:54:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.153+0000 7f08172a8080 1 Processor -- start 2024-04-17T09:54:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.153+0000 7f08172a8080 1 -- start start 2024-04-17T09:54:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.153+0000 7f08172a8080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48fa59100 0x55e48fb2a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.153+0000 7f08172a8080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e48faa3230 0x55e48fb2f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.153+0000 7f08172a8080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e48fb30270 0x55e48fa58240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.153+0000 7f08172a8080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e48f9fff90 con 0x55e48fa59100 2024-04-17T09:54:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.153+0000 7f08172a8080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e48fa03fd0 con 0x55e48faa3230 2024-04-17T09:54:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.153+0000 7f08172a8080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e48faa65f0 con 0x55e48fb30270 2024-04-17T09:54:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.153+0000 7f0814400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48fa59100 0x55e48fb2a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.153+0000 7f0814400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48fa59100 0x55e48fb2a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51402/0 (socket says 172.21.15.32:51402) 2024-04-17T09:54:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.153+0000 7f0814400640 1 -- 172.21.15.32:0/1865405088 learned_addr learned my addr 172.21.15.32:0/1865405088 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.153+0000 7f0814e00640 1 --2- 172.21.15.32:0/1865405088 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e48fb30270 0x55e48fa58240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.153+0000 7f0813a00640 1 --2- 172.21.15.32:0/1865405088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e48faa3230 0x55e48fb2f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f0814400640 1 -- 172.21.15.32:0/1865405088 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e48fb30270 msgr2=0x55e48fa58240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f0814400640 1 --2- 172.21.15.32:0/1865405088 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e48fb30270 0x55e48fa58240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f0814400640 1 -- 172.21.15.32:0/1865405088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e48faa3230 msgr2=0x55e48fb2f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f0814400640 1 --2- 172.21.15.32:0/1865405088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e48faa3230 0x55e48fb2f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f0814400640 1 -- 172.21.15.32:0/1865405088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e48fb2aad0 con 0x55e48fa59100 2024-04-17T09:54:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f0814400640 1 --2- 172.21.15.32:0/1865405088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48fa59100 0x55e48fb2a4a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f080400a500 tx=0x7f0804003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b2042d9f9c3852 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f0812600640 1 -- 172.21.15.32:0/1865405088 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08040af040 con 0x55e48fa59100 2024-04-17T09:54:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f0812600640 1 -- 172.21.15.32:0/1865405088 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08040b9070 con 0x55e48fa59100 2024-04-17T09:54:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f0812600640 1 -- 172.21.15.32:0/1865405088 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08040b4410 con 0x55e48fa59100 2024-04-17T09:54:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f08172a8080 1 -- 172.21.15.32:0/1865405088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48fa59100 msgr2=0x55e48fb2a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f08172a8080 1 --2- 172.21.15.32:0/1865405088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48fa59100 0x55e48fb2a4a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f080400a500 tx=0x7f0804003030 comp rx=0 tx=0).stop 2024-04-17T09:54:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f08172a8080 1 -- 172.21.15.32:0/1865405088 shutdown_connections 2024-04-17T09:54:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f08172a8080 1 --2- 172.21.15.32:0/1865405088 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e48fb30270 0x55e48fa58240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f08172a8080 1 --2- 172.21.15.32:0/1865405088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e48faa3230 0x55e48fb2f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f08172a8080 1 --2- 172.21.15.32:0/1865405088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48fa59100 0x55e48fb2a4a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f08172a8080 1 -- 172.21.15.32:0/1865405088 >> 172.21.15.32:0/1865405088 conn(0x55e48fa59620 msgr2=0x55e48fb2d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:47.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f08172a8080 1 -- 172.21.15.32:0/1865405088 shutdown_connections 2024-04-17T09:54:47.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f08172a8080 1 -- 172.21.15.32:0/1865405088 wait complete. 2024-04-17T09:54:47.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f08172a8080 1 Processor -- start 2024-04-17T09:54:47.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f08172a8080 1 -- start start 2024-04-17T09:54:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f08172a8080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e48fa59100 0x55e48fb0d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f08172a8080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e48faa3230 0x55e48fb0be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f08172a8080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48fb30270 0x55e48faa2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f08172a8080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e48fbc30a0 con 0x55e48fb30270 2024-04-17T09:54:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f08172a8080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e48fb2ad70 con 0x55e48fa59100 2024-04-17T09:54:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f08172a8080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e48fbc3340 con 0x55e48faa3230 2024-04-17T09:54:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f0814e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48fb30270 0x55e48faa2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f0814e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48fb30270 0x55e48faa2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51408/0 (socket says 172.21.15.32:51408) 2024-04-17T09:54:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f0814e00640 1 -- 172.21.15.32:0/2840755680 learned_addr learned my addr 172.21.15.32:0/2840755680 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f0814400640 1 --2- 172.21.15.32:0/2840755680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e48fa59100 0x55e48fb0d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f0813a00640 1 --2- 172.21.15.32:0/2840755680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e48faa3230 0x55e48fb0be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f0814e00640 1 -- 172.21.15.32:0/2840755680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e48faa3230 msgr2=0x55e48fb0be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f0814e00640 1 --2- 172.21.15.32:0/2840755680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e48faa3230 0x55e48fb0be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f0814e00640 1 -- 172.21.15.32:0/2840755680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e48fa59100 msgr2=0x55e48fb0d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f0814e00640 1 --2- 172.21.15.32:0/2840755680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e48fa59100 0x55e48fb0d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.157+0000 7f0814e00640 1 -- 172.21.15.32:0/2840755680 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e48fb0c760 con 0x55e48fb30270 2024-04-17T09:54:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.161+0000 7f0814e00640 1 --2- 172.21.15.32:0/2840755680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48fb30270 0x55e48faa2320 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f080c004290 tx=0x7f080c00d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.161+0000 7f0811200640 1 -- 172.21.15.32:0/2840755680 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c018070 con 0x55e48fb30270 2024-04-17T09:54:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.161+0000 7f0811200640 1 -- 172.21.15.32:0/2840755680 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f080c0030d0 con 0x55e48fb30270 2024-04-17T09:54:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.161+0000 7f0811200640 1 -- 172.21.15.32:0/2840755680 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c001550 con 0x55e48fb30270 2024-04-17T09:54:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.161+0000 7f08172a8080 1 -- 172.21.15.32:0/2840755680 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e48fb0ca00 con 0x55e48fb30270 2024-04-17T09:54:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.161+0000 7f08172a8080 1 -- 172.21.15.32:0/2840755680 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e48fb0cf40 con 0x55e48fb30270 2024-04-17T09:54:47.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.161+0000 7f0811200640 1 -- 172.21.15.32:0/2840755680 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f080c0016f0 con 0x55e48fb30270 2024-04-17T09:54:47.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.161+0000 7f0811200640 1 --2- 172.21.15.32:0/2840755680 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07f4066d60 0x7f07f4069220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.161+0000 7f0811200640 1 -- 172.21.15.32:0/2840755680 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f080c084e70 con 0x55e48fb30270 2024-04-17T09:54:47.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.161+0000 7f0814400640 1 --2- 172.21.15.32:0/2840755680 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07f4066d60 0x7f07f4069220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.161+0000 7f08172a8080 1 --2- 172.21.15.32:0/2840755680 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55e48fb0de90 0x55e48fb0e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.161+0000 7f08172a8080 1 -- 172.21.15.32:0/2840755680 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:1 5.36 5:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) -- 0x55e48fb26090 con 0x55e48fb0de90 2024-04-17T09:54:47.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.161+0000 7f0813a00640 1 --2- 172.21.15.32:0/2840755680 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55e48fb0de90 0x55e48fb0e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.165+0000 7f0814400640 1 --2- 172.21.15.32:0/2840755680 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07f4066d60 0x7f07f4069220 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f08040aa5d0 tx=0x7f08040a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.165+0000 7f0813a00640 1 --2- 172.21.15.32:0/2840755680 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55e48fb0de90 0x55e48fb0e270 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.165+0000 7f0813a00640 1 -- 172.21.15.32:0/2840755680 <== osd.1 v2:172.21.15.32:6808/2454993455 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f0800010610 con 0x55e48fb0de90 2024-04-17T09:54:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.165+0000 7f08172a8080 1 -- 172.21.15.32:0/2840755680 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55e48fb0de90 msgr2=0x55e48fb0e270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.165+0000 7f08172a8080 1 --2- 172.21.15.32:0/2840755680 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55e48fb0de90 0x55e48fb0e270 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.165+0000 7f08172a8080 1 -- 172.21.15.32:0/2840755680 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07f4066d60 msgr2=0x7f07f4069220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.165+0000 7f08172a8080 1 --2- 172.21.15.32:0/2840755680 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07f4066d60 0x7f07f4069220 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f08040aa5d0 tx=0x7f08040a7000 comp rx=0 tx=0).stop 2024-04-17T09:54:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.165+0000 7f08172a8080 1 -- 172.21.15.32:0/2840755680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48fb30270 msgr2=0x55e48faa2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.165+0000 7f08172a8080 1 --2- 172.21.15.32:0/2840755680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48fb30270 0x55e48faa2320 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f080c004290 tx=0x7f080c00d5f0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.165+0000 7f0814e00640 1 -- 172.21.15.32:0/2840755680 reap_dead start 2024-04-17T09:54:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.165+0000 7f08172a8080 1 -- 172.21.15.32:0/2840755680 shutdown_connections 2024-04-17T09:54:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.165+0000 7f08172a8080 1 -- 172.21.15.32:0/2840755680 >> 172.21.15.32:0/2840755680 conn(0x55e48fa59620 msgr2=0x55e48fb29a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.165+0000 7f08172a8080 1 -- 172.21.15.32:0/2840755680 shutdown_connections 2024-04-17T09:54:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.165+0000 7f08172a8080 1 -- 172.21.15.32:0/2840755680 wait complete. 2024-04-17T09:54:47.170 INFO:teuthology.orchestra.run.smithi032.stdout:mytag 2024-04-17T09:54:47.170 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-17T09:54:47.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f6548c77080 1 Processor -- start 2024-04-17T09:54:47.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f6548c77080 1 -- start start 2024-04-17T09:54:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f6548c77080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55787a0298b0 0x55787a04cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f6548c77080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55787a029c90 0x55787a0f8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f6548c77080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55787a0fc2a0 0x55787a0f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f6548c77080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55787a072630 con 0x55787a0298b0 2024-04-17T09:54:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f6548c77080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557879fcef90 con 0x55787a0fc2a0 2024-04-17T09:54:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f6548c77080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55787a0727b0 con 0x55787a029c90 2024-04-17T09:54:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f6545e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55787a0298b0 0x55787a04cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f6545e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55787a0298b0 0x55787a04cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51424/0 (socket says 172.21.15.32:51424) 2024-04-17T09:54:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f6545e00640 1 -- 172.21.15.32:0/1809027967 learned_addr learned my addr 172.21.15.32:0/1809027967 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f6545400640 1 --2- 172.21.15.32:0/1809027967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55787a029c90 0x55787a0f8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f6546800640 1 --2- 172.21.15.32:0/1809027967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55787a0fc2a0 0x55787a0f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f6545e00640 1 -- 172.21.15.32:0/1809027967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55787a029c90 msgr2=0x55787a0f8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:47.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f6545e00640 1 --2- 172.21.15.32:0/1809027967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55787a029c90 0x55787a0f8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f6545e00640 1 -- 172.21.15.32:0/1809027967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55787a0fc2a0 msgr2=0x55787a0f3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:47.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f6545e00640 1 --2- 172.21.15.32:0/1809027967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55787a0fc2a0 0x55787a0f3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f6545e00640 1 -- 172.21.15.32:0/1809027967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557879f53950 con 0x55787a0298b0 2024-04-17T09:54:47.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f6545400640 1 --2- 172.21.15.32:0/1809027967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55787a029c90 0x55787a0f8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:47.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f6545e00640 1 --2- 172.21.15.32:0/1809027967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55787a0298b0 0x55787a04cf30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f653800b350 tx=0x7f653800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8db473024f6109 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:47.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f653fe00640 1 -- 172.21.15.32:0/1809027967 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653800d5d0 con 0x55787a0298b0 2024-04-17T09:54:47.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f653fe00640 1 -- 172.21.15.32:0/1809027967 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6538008070 con 0x55787a0298b0 2024-04-17T09:54:47.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f653fe00640 1 -- 172.21.15.32:0/1809027967 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653800d970 con 0x55787a0298b0 2024-04-17T09:54:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.261+0000 7f6548c77080 1 -- 172.21.15.32:0/1809027967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55787a0298b0 msgr2=0x55787a04cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6548c77080 1 --2- 172.21.15.32:0/1809027967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55787a0298b0 0x55787a04cf30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f653800b350 tx=0x7f653800b810 comp rx=0 tx=0).stop 2024-04-17T09:54:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6548c77080 1 -- 172.21.15.32:0/1809027967 shutdown_connections 2024-04-17T09:54:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6548c77080 1 --2- 172.21.15.32:0/1809027967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55787a029c90 0x55787a0f8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6548c77080 1 --2- 172.21.15.32:0/1809027967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55787a0fc2a0 0x55787a0f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6548c77080 1 --2- 172.21.15.32:0/1809027967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55787a0298b0 0x55787a04cf30 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6548c77080 1 -- 172.21.15.32:0/1809027967 >> 172.21.15.32:0/1809027967 conn(0x55787a054760 msgr2=0x55787a057d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6548c77080 1 -- 172.21.15.32:0/1809027967 shutdown_connections 2024-04-17T09:54:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6548c77080 1 -- 172.21.15.32:0/1809027967 wait complete. 2024-04-17T09:54:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6548c77080 1 Processor -- start 2024-04-17T09:54:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6548c77080 1 -- start start 2024-04-17T09:54:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6548c77080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55787a0298b0 0x55787a0f7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6548c77080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55787a029c90 0x55787a057ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6548c77080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55787a0fc2a0 0x55787a068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6548c77080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55787a026e80 con 0x55787a029c90 2024-04-17T09:54:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6548c77080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55787a027120 con 0x55787a0298b0 2024-04-17T09:54:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6548c77080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55787a0273c0 con 0x55787a0fc2a0 2024-04-17T09:54:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6545400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55787a029c90 0x55787a057ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6545400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55787a029c90 0x55787a057ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51430/0 (socket says 172.21.15.32:51430) 2024-04-17T09:54:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6545e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55787a0298b0 0x55787a0f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6545400640 1 -- 172.21.15.32:0/2762493827 learned_addr learned my addr 172.21.15.32:0/2762493827 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6546800640 1 --2- 172.21.15.32:0/2762493827 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55787a0fc2a0 0x55787a068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6545400640 1 -- 172.21.15.32:0/2762493827 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55787a0fc2a0 msgr2=0x55787a068e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6545400640 1 --2- 172.21.15.32:0/2762493827 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55787a0fc2a0 0x55787a068e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6545400640 1 -- 172.21.15.32:0/2762493827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55787a0298b0 msgr2=0x55787a0f7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6545400640 1 --2- 172.21.15.32:0/2762493827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55787a0298b0 0x55787a0f7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6545400640 1 -- 172.21.15.32:0/2762493827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55787a067f30 con 0x55787a029c90 2024-04-17T09:54:47.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6545e00640 1 --2- 172.21.15.32:0/2762493827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55787a0298b0 0x55787a0f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:54:47.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6545400640 1 --2- 172.21.15.32:0/2762493827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55787a029c90 0x55787a057ac0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6534008880 tx=0x7f6534008d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:47.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f653ea00640 1 -- 172.21.15.32:0/2762493827 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6534006b80 con 0x55787a029c90 2024-04-17T09:54:47.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f653ea00640 1 -- 172.21.15.32:0/2762493827 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6534006d20 con 0x55787a029c90 2024-04-17T09:54:47.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f653ea00640 1 -- 172.21.15.32:0/2762493827 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6534015660 con 0x55787a029c90 2024-04-17T09:54:47.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6548c77080 1 -- 172.21.15.32:0/2762493827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55787a0681d0 con 0x55787a029c90 2024-04-17T09:54:47.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.265+0000 7f6548c77080 1 -- 172.21.15.32:0/2762493827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55787a068710 con 0x55787a029c90 2024-04-17T09:54:47.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.269+0000 7f653ea00640 1 -- 172.21.15.32:0/2762493827 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f653401c020 con 0x55787a029c90 2024-04-17T09:54:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.269+0000 7f653ea00640 1 --2- 172.21.15.32:0/2762493827 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6528066d90 0x7f6528069250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.269+0000 7f653ea00640 1 -- 172.21.15.32:0/2762493827 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f6534085d00 con 0x55787a029c90 2024-04-17T09:54:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.269+0000 7f6545e00640 1 --2- 172.21.15.32:0/2762493827 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6528066d90 0x7f6528069250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.269+0000 7f6548c77080 1 --2- 172.21.15.32:0/2762493827 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55787a058000 0x55787a069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.269+0000 7f6548c77080 1 -- 172.21.15.32:0/2762493827 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:1 5.19 5:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) -- 0x55787a0f9140 con 0x55787a058000 2024-04-17T09:54:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.269+0000 7f6546800640 1 --2- 172.21.15.32:0/2762493827 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55787a058000 0x55787a069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.269+0000 7f6545e00640 1 --2- 172.21.15.32:0/2762493827 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6528066d90 0x7f6528069250 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f653800b350 tx=0x7f6538008000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.269+0000 7f6546800640 1 --2- 172.21.15.32:0/2762493827 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55787a058000 0x55787a069390 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:47.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.269+0000 7f6546800640 1 -- 172.21.15.32:0/2762493827 <== osd.1 v2:172.21.15.32:6808/2454993455 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f65400b3040 con 0x55787a058000 2024-04-17T09:54:47.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.273+0000 7f6548c77080 1 -- 172.21.15.32:0/2762493827 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55787a058000 msgr2=0x55787a069390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.273+0000 7f6548c77080 1 --2- 172.21.15.32:0/2762493827 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55787a058000 0x55787a069390 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.273+0000 7f6548c77080 1 -- 172.21.15.32:0/2762493827 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6528066d90 msgr2=0x7f6528069250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.273+0000 7f6548c77080 1 --2- 172.21.15.32:0/2762493827 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6528066d90 0x7f6528069250 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f653800b350 tx=0x7f6538008000 comp rx=0 tx=0).stop 2024-04-17T09:54:47.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.273+0000 7f6548c77080 1 -- 172.21.15.32:0/2762493827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55787a029c90 msgr2=0x55787a057ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.273+0000 7f6548c77080 1 --2- 172.21.15.32:0/2762493827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55787a029c90 0x55787a057ac0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6534008880 tx=0x7f6534008d40 comp rx=0 tx=0).stop 2024-04-17T09:54:47.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.273+0000 7f6546800640 1 -- 172.21.15.32:0/2762493827 reap_dead start 2024-04-17T09:54:47.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.273+0000 7f6548c77080 1 -- 172.21.15.32:0/2762493827 shutdown_connections 2024-04-17T09:54:47.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.273+0000 7f6548c77080 1 -- 172.21.15.32:0/2762493827 >> 172.21.15.32:0/2762493827 conn(0x55787a054760 msgr2=0x55787a028240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:47.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.273+0000 7f6548c77080 1 -- 172.21.15.32:0/2762493827 shutdown_connections 2024-04-17T09:54:47.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.273+0000 7f6548c77080 1 -- 172.21.15.32:0/2762493827 wait complete. 2024-04-17T09:54:47.277 INFO:teuthology.orchestra.run.smithi032.stdout:mytag 2024-04-17T09:54:47.277 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-17T09:54:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.321+0000 7f9b27d2b080 1 Processor -- start 2024-04-17T09:54:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.321+0000 7f9b27d2b080 1 -- start start 2024-04-17T09:54:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.321+0000 7f9b27d2b080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564126e68b0 0x5564127b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.321+0000 7f9b27d2b080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564126e6c90 0x55641272d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.321+0000 7f9b27d2b080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5564127b92a0 0x55641272daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.321+0000 7f9b27d2b080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5564126e8c50 con 0x5564126e6c90 2024-04-17T09:54:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.321+0000 7f9b27d2b080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55641268bf90 con 0x5564126e68b0 2024-04-17T09:54:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.321+0000 7f9b27d2b080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55641272e490 con 0x5564127b92a0 2024-04-17T09:54:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b1fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564126e6c90 0x55641272d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b1fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564126e6c90 0x55641272d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51444/0 (socket says 172.21.15.32:51444) 2024-04-17T09:54:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b1fe00640 1 -- 172.21.15.32:0/3982741559 learned_addr learned my addr 172.21.15.32:0/3982741559 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b25a00640 1 --2- 172.21.15.32:0/3982741559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5564127b92a0 0x55641272daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b25000640 1 --2- 172.21.15.32:0/3982741559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564126e68b0 0x5564127b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b1fe00640 1 -- 172.21.15.32:0/3982741559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5564127b92a0 msgr2=0x55641272daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b1fe00640 1 --2- 172.21.15.32:0/3982741559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5564127b92a0 0x55641272daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b1fe00640 1 -- 172.21.15.32:0/3982741559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564126e68b0 msgr2=0x5564127b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b1fe00640 1 --2- 172.21.15.32:0/3982741559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564126e68b0 0x5564127b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b1fe00640 1 -- 172.21.15.32:0/3982741559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556412610950 con 0x5564126e6c90 2024-04-17T09:54:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b25a00640 1 --2- 172.21.15.32:0/3982741559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5564127b92a0 0x55641272daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:54:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b1fe00640 1 --2- 172.21.15.32:0/3982741559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564126e6c90 0x55641272d560 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9b100ad040 tx=0x7f9b10003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1841602e771df379 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b1ea00640 1 -- 172.21.15.32:0/3982741559 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b100af040 con 0x5564126e6c90 2024-04-17T09:54:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b1ea00640 1 -- 172.21.15.32:0/3982741559 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b100b9070 con 0x5564126e6c90 2024-04-17T09:54:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b1ea00640 1 -- 172.21.15.32:0/3982741559 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b100aae10 con 0x5564126e6c90 2024-04-17T09:54:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b27d2b080 1 -- 172.21.15.32:0/3982741559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564126e6c90 msgr2=0x55641272d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b27d2b080 1 --2- 172.21.15.32:0/3982741559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564126e6c90 0x55641272d560 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9b100ad040 tx=0x7f9b10003030 comp rx=0 tx=0).stop 2024-04-17T09:54:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b27d2b080 1 -- 172.21.15.32:0/3982741559 shutdown_connections 2024-04-17T09:54:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b27d2b080 1 --2- 172.21.15.32:0/3982741559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5564127b92a0 0x55641272daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b27d2b080 1 --2- 172.21.15.32:0/3982741559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564126e6c90 0x55641272d560 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b27d2b080 1 --2- 172.21.15.32:0/3982741559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564126e68b0 0x5564127b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b27d2b080 1 -- 172.21.15.32:0/3982741559 >> 172.21.15.32:0/3982741559 conn(0x556412711760 msgr2=0x5564127148a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b27d2b080 1 -- 172.21.15.32:0/3982741559 shutdown_connections 2024-04-17T09:54:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b27d2b080 1 -- 172.21.15.32:0/3982741559 wait complete. 2024-04-17T09:54:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b27d2b080 1 Processor -- start 2024-04-17T09:54:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b27d2b080 1 -- start start 2024-04-17T09:54:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b27d2b080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5564126e68b0 0x55641270e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b27d2b080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564127b92a0 0x55641272b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b27d2b080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55641270e7d0 0x5564127b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b27d2b080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556412714240 con 0x5564127b92a0 2024-04-17T09:54:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b27d2b080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556412713fa0 con 0x55641270e7d0 2024-04-17T09:54:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.325+0000 7f9b27d2b080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5564127144e0 con 0x5564126e68b0 2024-04-17T09:54:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.329+0000 7f9b1fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564127b92a0 0x55641272b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.329+0000 7f9b1fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564127b92a0 0x55641272b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51456/0 (socket says 172.21.15.32:51456) 2024-04-17T09:54:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.329+0000 7f9b1fe00640 1 -- 172.21.15.32:0/1376382337 learned_addr learned my addr 172.21.15.32:0/1376382337 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.329+0000 7f9b25000640 1 --2- 172.21.15.32:0/1376382337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5564126e68b0 0x55641270e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.329+0000 7f9b25a00640 1 --2- 172.21.15.32:0/1376382337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55641270e7d0 0x5564127b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.329+0000 7f9b1fe00640 1 -- 172.21.15.32:0/1376382337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5564126e68b0 msgr2=0x55641270e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.329+0000 7f9b1fe00640 1 --2- 172.21.15.32:0/1376382337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5564126e68b0 0x55641270e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.329+0000 7f9b1fe00640 1 -- 172.21.15.32:0/1376382337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55641270e7d0 msgr2=0x5564127b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.329+0000 7f9b1fe00640 1 --2- 172.21.15.32:0/1376382337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55641270e7d0 0x5564127b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.329+0000 7f9b1fe00640 1 -- 172.21.15.32:0/1376382337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55641270d430 con 0x5564127b92a0 2024-04-17T09:54:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.329+0000 7f9b1fe00640 1 --2- 172.21.15.32:0/1376382337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564127b92a0 0x55641272b0e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9b100b6040 tx=0x7f9b10018010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.329+0000 7f9b1d600640 1 -- 172.21.15.32:0/1376382337 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b100af040 con 0x5564127b92a0 2024-04-17T09:54:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.329+0000 7f9b1d600640 1 -- 172.21.15.32:0/1376382337 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b100c6680 con 0x5564127b92a0 2024-04-17T09:54:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.329+0000 7f9b27d2b080 1 -- 172.21.15.32:0/1376382337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55641270d6d0 con 0x5564127b92a0 2024-04-17T09:54:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.329+0000 7f9b1d600640 1 -- 172.21.15.32:0/1376382337 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b100c6a30 con 0x5564127b92a0 2024-04-17T09:54:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.329+0000 7f9b27d2b080 1 -- 172.21.15.32:0/1376382337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55641270dc10 con 0x5564127b92a0 2024-04-17T09:54:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.329+0000 7f9b1d600640 1 -- 172.21.15.32:0/1376382337 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9b100c6bd0 con 0x5564127b92a0 2024-04-17T09:54:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.329+0000 7f9b1d600640 1 --2- 172.21.15.32:0/1376382337 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9b04066d60 0x7f9b04069220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.333+0000 7f9b1d600640 1 -- 172.21.15.32:0/1376382337 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f9b101219d0 con 0x5564127b92a0 2024-04-17T09:54:47.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.333+0000 7f9b25000640 1 --2- 172.21.15.32:0/1376382337 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9b04066d60 0x7f9b04069220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.333+0000 7f9b27d2b080 1 --2- 172.21.15.32:0/1376382337 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x556412855330 0x556412855710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.333+0000 7f9b27d2b080 1 -- 172.21.15.32:0/1376382337 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:1 5.12 5:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) -- 0x5564127b0e20 con 0x556412855330 2024-04-17T09:54:47.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.333+0000 7f9b25a00640 1 --2- 172.21.15.32:0/1376382337 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x556412855330 0x556412855710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.333+0000 7f9b25000640 1 --2- 172.21.15.32:0/1376382337 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9b04066d60 0x7f9b04069220 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9b18005620 tx=0x7f9b1800c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:47.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.333+0000 7f9b25a00640 1 --2- 172.21.15.32:0/1376382337 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x556412855330 0x556412855710 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:47.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.333+0000 7f9b25a00640 1 -- 172.21.15.32:0/1376382337 <== osd.1 v2:172.21.15.32:6808/2454993455 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7f9b2000d690 con 0x556412855330 2024-04-17T09:54:47.335 INFO:teuthology.orchestra.run.smithi032.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-04-17T09:54:47.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.333+0000 7f9b27d2b080 1 -- 172.21.15.32:0/1376382337 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x556412855330 msgr2=0x556412855710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.333+0000 7f9b27d2b080 1 --2- 172.21.15.32:0/1376382337 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x556412855330 0x556412855710 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.333+0000 7f9b27d2b080 1 -- 172.21.15.32:0/1376382337 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9b04066d60 msgr2=0x7f9b04069220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.333+0000 7f9b27d2b080 1 --2- 172.21.15.32:0/1376382337 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9b04066d60 0x7f9b04069220 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9b18005620 tx=0x7f9b1800c040 comp rx=0 tx=0).stop 2024-04-17T09:54:47.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.333+0000 7f9b27d2b080 1 -- 172.21.15.32:0/1376382337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564127b92a0 msgr2=0x55641272b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.333+0000 7f9b27d2b080 1 --2- 172.21.15.32:0/1376382337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564127b92a0 0x55641272b0e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9b100b6040 tx=0x7f9b10018010 comp rx=0 tx=0).stop 2024-04-17T09:54:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.333+0000 7f9b25a00640 1 -- 172.21.15.32:0/1376382337 reap_dead start 2024-04-17T09:54:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.333+0000 7f9b27d2b080 1 -- 172.21.15.32:0/1376382337 shutdown_connections 2024-04-17T09:54:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.333+0000 7f9b27d2b080 1 -- 172.21.15.32:0/1376382337 >> 172.21.15.32:0/1376382337 conn(0x556412711760 msgr2=0x55641272d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.333+0000 7f9b27d2b080 1 -- 172.21.15.32:0/1376382337 shutdown_connections 2024-04-17T09:54:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.333+0000 7f9b27d2b080 1 -- 172.21.15.32:0/1376382337 wait complete. 2024-04-17T09:54:47.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:54:47.339 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T09:54:47.339 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T09:54:47.463 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T09:54:47.554 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T09:54:47.554 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T09:54:47.554 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:54:47.554 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:54:47.564 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T09:54:47.564 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T09:54:47.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:47.564 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T09:54:47.615 INFO:teuthology.orchestra.run.smithi032.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-17T09:54:47.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:54:47.616 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T09:54:47.617 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:54:47.617 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:54:47.626 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T09:54:47.626 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T09:54:47.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:47.626 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-17T09:54:47.679 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-17T09:54:47.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:54:47.680 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T09:54:47.680 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:54:47.680 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:54:47.689 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T09:54:47.689 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T09:54:47.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:47.689 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-17T09:54:47.740 INFO:teuthology.orchestra.run.smithi032.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-17T09:54:47.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:54:47.741 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T09:54:47.741 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:54:47.741 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:54:47.750 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T09:54:47.750 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T09:54:47.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:54:47.750 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-17T09:54:47.798 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-17T09:54:47.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:54:47.800 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:54:47.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.941+0000 7ff0dba00640 1 Processor -- start 2024-04-17T09:54:47.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.941+0000 7ff0dba00640 1 -- start start 2024-04-17T09:54:47.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.941+0000 7ff0dba00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff0d4144f20 0x7ff0d406e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.941+0000 7ff0dba00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0d41481e0 0x7ff0d40b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.941+0000 7ff0dba00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0d40aed20 con 0x7ff0d406eda0 2024-04-17T09:54:47.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.941+0000 7ff0dba00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff0d40af000 con 0x7ff0d41481e0 2024-04-17T09:54:47.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.941+0000 7ff0dba00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff0d40af2e0 con 0x7ff0d4144f20 2024-04-17T09:54:47.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.941+0000 7ff0d9a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0d41481e0 0x7ff0d40b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.941+0000 7ff0d9a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0d41481e0 0x7ff0d40b3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51464/0 (socket says 172.21.15.32:51464) 2024-04-17T09:54:47.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.941+0000 7ff0d9a00640 1 -- 172.21.15.32:0/1335124300 learned_addr learned my addr 172.21.15.32:0/1335124300 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:47.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.941+0000 7ff0d9000640 1 --2- 172.21.15.32:0/1335124300 >> v2:172.21.15.144:3300/0 conn(0x7ff0d4144f20 0x7ff0d406e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.941+0000 7ff0d9a00640 1 -- 172.21.15.32:0/1335124300 >> v2:172.21.15.144:3300/0 conn(0x7ff0d4144f20 msgr2=0x7ff0d406e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:47.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.941+0000 7ff0d9a00640 1 --2- 172.21.15.32:0/1335124300 >> v2:172.21.15.144:3300/0 conn(0x7ff0d4144f20 0x7ff0d406e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.941+0000 7ff0d9a00640 1 -- 172.21.15.32:0/1335124300 >> v1:172.21.15.144:6789/0 conn(0x7ff0d406eda0 legacy=0x7ff0d4071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:47.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.941+0000 7ff0d9a00640 1 -- 172.21.15.32:0/1335124300 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d40af5c0 con 0x7ff0d41481e0 2024-04-17T09:54:47.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.941+0000 7ff0d9000640 1 --2- 172.21.15.32:0/1335124300 >> v2:172.21.15.144:3300/0 conn(0x7ff0d4144f20 0x7ff0d406e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:54:47.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.945+0000 7ff0d9a00640 1 --2- 172.21.15.32:0/1335124300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0d41481e0 0x7ff0d40b3490 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff0d406ca90 tx=0x7ff0cc01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d638d92503b406c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:47.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.945+0000 7ff0d3400640 1 -- 172.21.15.32:0/1335124300 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc0a5680 con 0x7ff0d41481e0 2024-04-17T09:54:47.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.945+0000 7ff0d3400640 1 -- 172.21.15.32:0/1335124300 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0cc0a5820 con 0x7ff0d41481e0 2024-04-17T09:54:47.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.945+0000 7ff0d3400640 1 -- 172.21.15.32:0/1335124300 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc0a5bf0 con 0x7ff0d41481e0 2024-04-17T09:54:47.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.945+0000 7ff0dba00640 1 -- 172.21.15.32:0/1335124300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0d41481e0 msgr2=0x7ff0d40b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.945+0000 7ff0dba00640 1 --2- 172.21.15.32:0/1335124300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0d41481e0 0x7ff0d40b3490 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff0d406ca90 tx=0x7ff0cc01c9b0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.945+0000 7ff0dba00640 1 -- 172.21.15.32:0/1335124300 shutdown_connections 2024-04-17T09:54:47.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.945+0000 7ff0dba00640 1 --2- 172.21.15.32:0/1335124300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0d41481e0 0x7ff0d40b3490 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.945+0000 7ff0dba00640 1 --2- 172.21.15.32:0/1335124300 >> v2:172.21.15.144:3300/0 conn(0x7ff0d4144f20 0x7ff0d406e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.945+0000 7ff0dba00640 1 -- 172.21.15.32:0/1335124300 >> 172.21.15.32:0/1335124300 conn(0x7ff0d406ba30 msgr2=0x7ff0d406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:47.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.945+0000 7ff0dba00640 1 -- 172.21.15.32:0/1335124300 shutdown_connections 2024-04-17T09:54:47.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.945+0000 7ff0dba00640 1 -- 172.21.15.32:0/1335124300 wait complete. 2024-04-17T09:54:47.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0dba00640 1 Processor -- start 2024-04-17T09:54:47.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0dba00640 1 -- start start 2024-04-17T09:54:47.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0dba00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0d406eda0 0x7ff0d4140e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0dba00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0d4144f20 0x7ff0d40b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0dba00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0d41481e0 0x7ff0d40b1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0dba00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d40b41e0 con 0x7ff0d41481e0 2024-04-17T09:54:47.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0dba00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d40b4360 con 0x7ff0d406eda0 2024-04-17T09:54:47.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0dba00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff0d416b5d0 con 0x7ff0d4144f20 2024-04-17T09:54:47.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0d9a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0d41481e0 0x7ff0d40b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0d9a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0d41481e0 0x7ff0d40b1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51476/0 (socket says 172.21.15.32:51476) 2024-04-17T09:54:47.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0d9a00640 1 -- 172.21.15.32:0/1256672790 learned_addr learned my addr 172.21.15.32:0/1256672790 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:47.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0d9000640 1 --2- 172.21.15.32:0/1256672790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0d406eda0 0x7ff0d4140e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0d3e00640 1 --2- 172.21.15.32:0/1256672790 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0d4144f20 0x7ff0d40b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0d9a00640 1 -- 172.21.15.32:0/1256672790 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0d4144f20 msgr2=0x7ff0d40b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0d9a00640 1 --2- 172.21.15.32:0/1256672790 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0d4144f20 0x7ff0d40b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0d9a00640 1 -- 172.21.15.32:0/1256672790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0d406eda0 msgr2=0x7ff0d4140e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:47.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0d9a00640 1 --2- 172.21.15.32:0/1256672790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0d406eda0 0x7ff0d4140e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:47.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0d9a00640 1 -- 172.21.15.32:0/1256672790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d4141cb0 con 0x7ff0d41481e0 2024-04-17T09:54:47.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0d9a00640 1 --2- 172.21.15.32:0/1256672790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0d41481e0 0x7ff0d40b1650 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0af040 tx=0x7ff0cc019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:47.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0d0c00640 1 -- 172.21.15.32:0/1256672790 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc0b8670 con 0x7ff0d41481e0 2024-04-17T09:54:47.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0d0c00640 1 -- 172.21.15.32:0/1256672790 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0cc0be070 con 0x7ff0d41481e0 2024-04-17T09:54:47.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0d0c00640 1 -- 172.21.15.32:0/1256672790 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc0b8a90 con 0x7ff0d41481e0 2024-04-17T09:54:47.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0dba00640 1 -- 172.21.15.32:0/1256672790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d4141f90 con 0x7ff0d41481e0 2024-04-17T09:54:47.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.949+0000 7ff0dba00640 1 -- 172.21.15.32:0/1256672790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d4140960 con 0x7ff0d41481e0 2024-04-17T09:54:47.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.953+0000 7ff0dba00640 1 -- 172.21.15.32:0/1256672790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0d4140b60 con 0x7ff0d41481e0 2024-04-17T09:54:47.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.953+0000 7ff0d0c00640 1 -- 172.21.15.32:0/1256672790 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff0cc0b8c30 con 0x7ff0d41481e0 2024-04-17T09:54:47.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.953+0000 7ff0d0c00640 1 --2- 172.21.15.32:0/1256672790 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff0b4068380 0x7ff0b406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:47.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.953+0000 7ff0d0c00640 1 -- 172.21.15.32:0/1256672790 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7ff0cc121b30 con 0x7ff0d41481e0 2024-04-17T09:54:47.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.953+0000 7ff0d9000640 1 --2- 172.21.15.32:0/1256672790 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff0b4068380 0x7ff0b406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:47.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.953+0000 7ff0d9000640 1 --2- 172.21.15.32:0/1256672790 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff0b4068380 0x7ff0b406a840 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4169590 tx=0x7ff0c8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:47.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:47.957+0000 7ff0d0c00640 1 -- 172.21.15.32:0/1256672790 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff0d4140b60 con 0x7ff0d41481e0 2024-04-17T09:54:48.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.157+0000 7ff0dba00640 1 -- 172.21.15.32:0/1256672790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0d4068a90 con 0x7ff0d41481e0 2024-04-17T09:54:48.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.157+0000 7ff0d0c00640 1 -- 172.21.15.32:0/1256672790 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5199 (secure 0 0 0) 0x7ff0cc0165a0 con 0x7ff0d41481e0 2024-04-17T09:54:48.159 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:48.159 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2127207201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2127207201},{"type":"v1","addr":"172.21.15.144:6835","nonce":2127207201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5110,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3782699135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3782699135},{"type":"v1","addr":"172.21.15.144:6834","nonce":3782699135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/385811857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":385811857},{"type":"v1","addr":"172.21.15.32:6836","nonce":385811857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:54:16.745151+0000","modified":"2024-04-17T09:54:19.761481+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5217":{"gid":5217,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/925293029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":925293029},{"type":"v1","addr":"172.21.15.32:6837","nonce":925293029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5217,"qdb_cluster":[5217]},"id":2}]} 2024-04-17T09:54:48.159 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 15 2024-04-17T09:54:48.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.161+0000 7ff0dba00640 1 -- 172.21.15.32:0/1256672790 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff0b4068380 msgr2=0x7ff0b406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:48.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.161+0000 7ff0dba00640 1 --2- 172.21.15.32:0/1256672790 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff0b4068380 0x7ff0b406a840 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4169590 tx=0x7ff0c8008040 comp rx=0 tx=0).stop 2024-04-17T09:54:48.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.161+0000 7ff0dba00640 1 -- 172.21.15.32:0/1256672790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0d41481e0 msgr2=0x7ff0d40b1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:48.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.161+0000 7ff0dba00640 1 --2- 172.21.15.32:0/1256672790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0d41481e0 0x7ff0d40b1650 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0af040 tx=0x7ff0cc019750 comp rx=0 tx=0).stop 2024-04-17T09:54:48.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.161+0000 7ff0dba00640 1 -- 172.21.15.32:0/1256672790 shutdown_connections 2024-04-17T09:54:48.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.161+0000 7ff0dba00640 1 --2- 172.21.15.32:0/1256672790 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff0b4068380 0x7ff0b406a840 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.161+0000 7ff0dba00640 1 --2- 172.21.15.32:0/1256672790 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0d4144f20 0x7ff0d40b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.161+0000 7ff0dba00640 1 --2- 172.21.15.32:0/1256672790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0d41481e0 0x7ff0d40b1650 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.161+0000 7ff0dba00640 1 --2- 172.21.15.32:0/1256672790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0d406eda0 0x7ff0d4140e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.161+0000 7ff0dba00640 1 -- 172.21.15.32:0/1256672790 >> 172.21.15.32:0/1256672790 conn(0x7ff0d406ba30 msgr2=0x7ff0d413f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:48.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.161+0000 7ff0dba00640 1 -- 172.21.15.32:0/1256672790 shutdown_connections 2024-04-17T09:54:48.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.161+0000 7ff0dba00640 1 -- 172.21.15.32:0/1256672790 wait complete. 2024-04-17T09:54:48.181 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:54:48.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.321+0000 7f6746000640 1 Processor -- start 2024-04-17T09:54:48.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.321+0000 7f6746000640 1 -- start start 2024-04-17T09:54:48.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.321+0000 7f6746000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f674015eb30 0x7f67400d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:48.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.321+0000 7f6746000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f674009b100 0x7f67400cd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:48.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.321+0000 7f6746000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67400ccbd0 con 0x7f67400b4af0 2024-04-17T09:54:48.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.321+0000 7f6746000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f67400cceb0 con 0x7f674009b100 2024-04-17T09:54:48.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.321+0000 7f6746000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f67400cd190 con 0x7f674015eb30 2024-04-17T09:54:48.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.321+0000 7f6736a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f67400b4af0 0x7f674009a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48444/0 (socket says 172.21.15.32:48444) 2024-04-17T09:54:48.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.321+0000 7f6736a00640 1 -- 172.21.15.32:0/2820419615 learned_addr learned my addr 172.21.15.32:0/2820419615 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:48.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.321+0000 7f673fe00640 1 --2- 172.21.15.32:0/2820419615 >> v2:172.21.15.32:3300/0 conn(0x7f674009b100 0x7f67400cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:48.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.321+0000 7f673f400640 1 --2- 172.21.15.32:0/2820419615 >> v2:172.21.15.144:3300/0 conn(0x7f674015eb30 0x7f67400d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:48.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.321+0000 7f673ea00640 1 -- 172.21.15.32:0/2820419615 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1882644279 0 0) 0x7f67400ccbd0 con 0x7f67400b4af0 2024-04-17T09:54:48.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.321+0000 7f673ea00640 1 -- 172.21.15.32:0/2820419615 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6724003530 con 0x7f67400b4af0 2024-04-17T09:54:48.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.321+0000 7f673fe00640 1 -- 172.21.15.32:0/2820419615 >> v2:172.21.15.144:3300/0 conn(0x7f674015eb30 msgr2=0x7f67400d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:48.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.321+0000 7f673fe00640 1 --2- 172.21.15.32:0/2820419615 >> v2:172.21.15.144:3300/0 conn(0x7f674015eb30 0x7f67400d06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.321+0000 7f673fe00640 1 -- 172.21.15.32:0/2820419615 >> v1:172.21.15.32:6789/0 conn(0x7f67400b4af0 legacy=0x7f674009a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:48.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.321+0000 7f673fe00640 1 -- 172.21.15.32:0/2820419615 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67400cd470 con 0x7f674009b100 2024-04-17T09:54:48.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.321+0000 7f673f400640 1 --2- 172.21.15.32:0/2820419615 >> v2:172.21.15.144:3300/0 conn(0x7f674015eb30 0x7f67400d06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:48.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.325+0000 7f673fe00640 1 --2- 172.21.15.32:0/2820419615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f674009b100 0x7f67400cd990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f674006ed20 tx=0x7f673801f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65726749ddee280c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:48.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.325+0000 7f673ea00640 1 -- 172.21.15.32:0/2820419615 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67380b4070 con 0x7f674009b100 2024-04-17T09:54:48.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.325+0000 7f673ea00640 1 -- 172.21.15.32:0/2820419615 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67380af600 con 0x7f674009b100 2024-04-17T09:54:48.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.325+0000 7f673ea00640 1 -- 172.21.15.32:0/2820419615 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67380af9d0 con 0x7f674009b100 2024-04-17T09:54:48.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.325+0000 7f6746000640 1 -- 172.21.15.32:0/2820419615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f674009b100 msgr2=0x7f67400cd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:48.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.325+0000 7f6746000640 1 --2- 172.21.15.32:0/2820419615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f674009b100 0x7f67400cd990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f674006ed20 tx=0x7f673801f110 comp rx=0 tx=0).stop 2024-04-17T09:54:48.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.325+0000 7f6746000640 1 -- 172.21.15.32:0/2820419615 shutdown_connections 2024-04-17T09:54:48.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.325+0000 7f6746000640 1 --2- 172.21.15.32:0/2820419615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f674009b100 0x7f67400cd990 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.325+0000 7f6746000640 1 --2- 172.21.15.32:0/2820419615 >> v2:172.21.15.144:3300/0 conn(0x7f674015eb30 0x7f67400d06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.325+0000 7f6746000640 1 -- 172.21.15.32:0/2820419615 >> 172.21.15.32:0/2820419615 conn(0x7f67400b3fd0 msgr2=0x7f67400b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:48.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.325+0000 7f6746000640 1 -- 172.21.15.32:0/2820419615 shutdown_connections 2024-04-17T09:54:48.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.325+0000 7f6746000640 1 -- 172.21.15.32:0/2820419615 wait complete. 2024-04-17T09:54:48.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f6746000640 1 Processor -- start 2024-04-17T09:54:48.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f6746000640 1 -- start start 2024-04-17T09:54:48.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f6746000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f674009b100 0x7f67400b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:48.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f6746000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67400b4af0 0x7f67400afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:48.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f6746000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f674015eb30 0x7f67400b6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:48.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f6746000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f67400af170 con 0x7f67400b4af0 2024-04-17T09:54:48.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f6746000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f67400d1680 con 0x7f674015eb30 2024-04-17T09:54:48.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f6746000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f67400d19b0 con 0x7f674009b100 2024-04-17T09:54:48.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f6736a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67400b4af0 0x7f67400afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f6736a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67400b4af0 0x7f67400afb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51508/0 (socket says 172.21.15.32:51508) 2024-04-17T09:54:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f6736a00640 1 -- 172.21.15.32:0/1348682696 learned_addr learned my addr 172.21.15.32:0/1348682696 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f673fe00640 1 --2- 172.21.15.32:0/1348682696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f674015eb30 0x7f67400b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f673f400640 1 --2- 172.21.15.32:0/1348682696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f674009b100 0x7f67400b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f6736a00640 1 -- 172.21.15.32:0/1348682696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f674009b100 msgr2=0x7f67400b7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f6736a00640 1 --2- 172.21.15.32:0/1348682696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f674009b100 0x7f67400b7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f6736a00640 1 -- 172.21.15.32:0/1348682696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f674015eb30 msgr2=0x7f67400b6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f6736a00640 1 --2- 172.21.15.32:0/1348682696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f674015eb30 0x7f67400b6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f6736a00640 1 -- 172.21.15.32:0/1348682696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67400b1c10 con 0x7f67400b4af0 2024-04-17T09:54:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f6736a00640 1 --2- 172.21.15.32:0/1348682696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67400b4af0 0x7f67400afb70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f672800c420 tx=0x7f672800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f6737e00640 1 -- 172.21.15.32:0/1348682696 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728017070 con 0x7f67400b4af0 2024-04-17T09:54:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f6737e00640 1 -- 172.21.15.32:0/1348682696 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6728005d10 con 0x7f67400b4af0 2024-04-17T09:54:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f6737e00640 1 -- 172.21.15.32:0/1348682696 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728007310 con 0x7f67400b4af0 2024-04-17T09:54:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f6746000640 1 -- 172.21.15.32:0/1348682696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67400b0480 con 0x7f67400b4af0 2024-04-17T09:54:48.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f6746000640 1 -- 172.21.15.32:0/1348682696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67400b2190 con 0x7f67400b4af0 2024-04-17T09:54:48.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.329+0000 7f6746000640 1 -- 172.21.15.32:0/1348682696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6740068a90 con 0x7f67400b4af0 2024-04-17T09:54:48.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.333+0000 7f6737e00640 1 -- 172.21.15.32:0/1348682696 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f672801a070 con 0x7f67400b4af0 2024-04-17T09:54:48.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.333+0000 7f6737e00640 1 --2- 172.21.15.32:0/1348682696 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f671c068380 0x7f671c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:48.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.333+0000 7f6737e00640 1 -- 172.21.15.32:0/1348682696 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f6728085c70 con 0x7f67400b4af0 2024-04-17T09:54:48.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.333+0000 7f673f400640 1 --2- 172.21.15.32:0/1348682696 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f671c068380 0x7f671c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:48.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.333+0000 7f673f400640 1 --2- 172.21.15.32:0/1348682696 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f671c068380 0x7f671c06a840 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f67300045f0 tx=0x7f6730008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:48.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.337+0000 7f6737e00640 1 -- 172.21.15.32:0/1348682696 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f6728055d30 con 0x7f67400b4af0 2024-04-17T09:54:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.505+0000 7f6746000640 1 -- 172.21.15.32:0/1348682696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f674006ef10 con 0x7f67400b4af0 2024-04-17T09:54:48.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.505+0000 7f6737e00640 1 -- 172.21.15.32:0/1348682696 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16681 (secure 0 0 0) 0x7f672805a030 con 0x7f67400b4af0 2024-04-17T09:54:48.507 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:48.507 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:54:18.351519+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:54:15.261471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-17T09:54:15.704958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:54:48.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.509+0000 7f6746000640 1 -- 172.21.15.32:0/1348682696 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f671c068380 msgr2=0x7f671c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.509+0000 7f6746000640 1 --2- 172.21.15.32:0/1348682696 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f671c068380 0x7f671c06a840 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f67300045f0 tx=0x7f6730008040 comp rx=0 tx=0).stop 2024-04-17T09:54:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.509+0000 7f6746000640 1 -- 172.21.15.32:0/1348682696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67400b4af0 msgr2=0x7f67400afb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.509+0000 7f6746000640 1 --2- 172.21.15.32:0/1348682696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67400b4af0 0x7f67400afb70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f672800c420 tx=0x7f672800c8e0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.509+0000 7f6746000640 1 -- 172.21.15.32:0/1348682696 shutdown_connections 2024-04-17T09:54:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.509+0000 7f6746000640 1 --2- 172.21.15.32:0/1348682696 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f671c068380 0x7f671c06a840 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.509+0000 7f6746000640 1 --2- 172.21.15.32:0/1348682696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f674015eb30 0x7f67400b6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.509+0000 7f6746000640 1 --2- 172.21.15.32:0/1348682696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67400b4af0 0x7f67400afb70 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.509+0000 7f6746000640 1 --2- 172.21.15.32:0/1348682696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f674009b100 0x7f67400b7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.509+0000 7f6746000640 1 -- 172.21.15.32:0/1348682696 >> 172.21.15.32:0/1348682696 conn(0x7f67400b3fd0 msgr2=0x7f67400ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.509+0000 7f6746000640 1 -- 172.21.15.32:0/1348682696 shutdown_connections 2024-04-17T09:54:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.509+0000 7f6746000640 1 -- 172.21.15.32:0/1348682696 wait complete. 2024-04-17T09:54:48.529 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:54:15.261471+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:54:15.704958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:54:48.530 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:54:48.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.673+0000 7f94fb200640 1 Processor -- start 2024-04-17T09:54:48.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.673+0000 7f94fb200640 1 -- start start 2024-04-17T09:54:48.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.673+0000 7f94fb200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94f40721b0 0x7f94f4072660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:48.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.673+0000 7f94fb200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f94f40aefa0 0x7f94f40af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:48.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.673+0000 7f94fb200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94f40ae520 con 0x7f94f40b5370 2024-04-17T09:54:48.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.673+0000 7f94fb200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f94f40b5750 con 0x7f94f40721b0 2024-04-17T09:54:48.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.673+0000 7f94fb200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f94f4162dc0 con 0x7f94f40aefa0 2024-04-17T09:54:48.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.673+0000 7f94f9e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f94f40b5370 0x7f94f4162340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48458/0 (socket says 172.21.15.32:48458) 2024-04-17T09:54:48.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.673+0000 7f94f9400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94f40721b0 0x7f94f4072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:48.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.673+0000 7f94f9e00640 1 -- 172.21.15.32:0/400714832 learned_addr learned my addr 172.21.15.32:0/400714832 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:48.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.673+0000 7f94fa800640 1 --2- 172.21.15.32:0/400714832 >> v2:172.21.15.144:3300/0 conn(0x7f94f40aefa0 0x7f94f40af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:48.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.673+0000 7f94f9400640 1 -- 172.21.15.32:0/400714832 >> v2:172.21.15.144:3300/0 conn(0x7f94f40aefa0 msgr2=0x7f94f40af3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:48.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.673+0000 7f94f9400640 1 --2- 172.21.15.32:0/400714832 >> v2:172.21.15.144:3300/0 conn(0x7f94f40aefa0 0x7f94f40af3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.673+0000 7f94f9400640 1 -- 172.21.15.32:0/400714832 >> v1:172.21.15.32:6789/0 conn(0x7f94f40b5370 legacy=0x7f94f4162340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:48.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.673+0000 7f94f9400640 1 -- 172.21.15.32:0/400714832 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f41630a0 con 0x7f94f40721b0 2024-04-17T09:54:48.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.673+0000 7f94f8a00640 1 -- 172.21.15.32:0/400714832 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2027487074 0 0) 0x7f94e8001120 con 0x7f94f40b5370 2024-04-17T09:54:48.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.677+0000 7f94f9400640 1 --2- 172.21.15.32:0/400714832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94f40721b0 0x7f94f4072660 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f94f406ca60 tx=0x7f94e401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4e61ae08a7ab6e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:48.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.677+0000 7f94f8a00640 1 -- 172.21.15.32:0/400714832 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e40b0070 con 0x7f94f40721b0 2024-04-17T09:54:48.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.677+0000 7f94f8a00640 1 -- 172.21.15.32:0/400714832 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94e40ab600 con 0x7f94f40721b0 2024-04-17T09:54:48.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.677+0000 7f94f8a00640 1 -- 172.21.15.32:0/400714832 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e40ab9d0 con 0x7f94f40721b0 2024-04-17T09:54:48.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.677+0000 7f94fb200640 1 -- 172.21.15.32:0/400714832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94f40721b0 msgr2=0x7f94f4072660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:48.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.677+0000 7f94fb200640 1 --2- 172.21.15.32:0/400714832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94f40721b0 0x7f94f4072660 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f94f406ca60 tx=0x7f94e401c480 comp rx=0 tx=0).stop 2024-04-17T09:54:48.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.677+0000 7f94fb200640 1 -- 172.21.15.32:0/400714832 shutdown_connections 2024-04-17T09:54:48.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.677+0000 7f94fb200640 1 --2- 172.21.15.32:0/400714832 >> v2:172.21.15.144:3300/0 conn(0x7f94f40aefa0 0x7f94f40af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.677+0000 7f94fb200640 1 --2- 172.21.15.32:0/400714832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94f40721b0 0x7f94f4072660 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.677+0000 7f94fb200640 1 -- 172.21.15.32:0/400714832 >> 172.21.15.32:0/400714832 conn(0x7f94f406c1c0 msgr2=0x7f94f40b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:48.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.677+0000 7f94fb200640 1 -- 172.21.15.32:0/400714832 shutdown_connections 2024-04-17T09:54:48.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.677+0000 7f94fb200640 1 -- 172.21.15.32:0/400714832 wait complete. 2024-04-17T09:54:48.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94fb200640 1 Processor -- start 2024-04-17T09:54:48.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94fb200640 1 -- start start 2024-04-17T09:54:48.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94fb200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94f40721b0 0x7f94f4154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:48.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94fb200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94f40aefa0 0x7f94f4146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94fb200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94f40b5370 0x7f94f415fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94fb200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f94f416b6e0 con 0x7f94f40b5370 2024-04-17T09:54:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94fb200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f94f415f000 con 0x7f94f40aefa0 2024-04-17T09:54:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94fb200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f94f416b860 con 0x7f94f40721b0 2024-04-17T09:54:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94fa800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94f40b5370 0x7f94f415fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94fa800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94f40b5370 0x7f94f415fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51528/0 (socket says 172.21.15.32:51528) 2024-04-17T09:54:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94fa800640 1 -- 172.21.15.32:0/4120633343 learned_addr learned my addr 172.21.15.32:0/4120633343 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94f9400640 1 --2- 172.21.15.32:0/4120633343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94f40aefa0 0x7f94f4146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94f9e00640 1 --2- 172.21.15.32:0/4120633343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94f40721b0 0x7f94f4154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94fa800640 1 -- 172.21.15.32:0/4120633343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94f40721b0 msgr2=0x7f94f4154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94fa800640 1 --2- 172.21.15.32:0/4120633343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94f40721b0 0x7f94f4154ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94fa800640 1 -- 172.21.15.32:0/4120633343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94f40aefa0 msgr2=0x7f94f4146100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:48.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94fa800640 1 --2- 172.21.15.32:0/4120633343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94f40aefa0 0x7f94f4146100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94fa800640 1 -- 172.21.15.32:0/4120633343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f4160230 con 0x7f94f40b5370 2024-04-17T09:54:48.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94fa800640 1 --2- 172.21.15.32:0/4120633343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94f40b5370 0x7f94f415fac0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f94f0007e70 tx=0x7f94f000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:48.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94ee000640 1 -- 172.21.15.32:0/4120633343 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f0014620 con 0x7f94f40b5370 2024-04-17T09:54:48.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94ee000640 1 -- 172.21.15.32:0/4120633343 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94f0019070 con 0x7f94f40b5370 2024-04-17T09:54:48.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94ee000640 1 -- 172.21.15.32:0/4120633343 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f0014a20 con 0x7f94f40b5370 2024-04-17T09:54:48.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94fb200640 1 -- 172.21.15.32:0/4120633343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f4160510 con 0x7f94f40b5370 2024-04-17T09:54:48.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.681+0000 7f94fb200640 1 -- 172.21.15.32:0/4120633343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f4156d70 con 0x7f94f40b5370 2024-04-17T09:54:48.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.685+0000 7f94fb200640 1 -- 172.21.15.32:0/4120633343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94f404fda0 con 0x7f94f40b5370 2024-04-17T09:54:48.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.685+0000 7f94ee000640 1 -- 172.21.15.32:0/4120633343 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f94f0014bc0 con 0x7f94f40b5370 2024-04-17T09:54:48.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.685+0000 7f94ee000640 1 --2- 172.21.15.32:0/4120633343 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f94d8068380 0x7f94d806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:48.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.685+0000 7f94ee000640 1 -- 172.21.15.32:0/4120633343 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f94f0087210 con 0x7f94f40b5370 2024-04-17T09:54:48.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.685+0000 7f94f9e00640 1 --2- 172.21.15.32:0/4120633343 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f94d8068380 0x7f94d806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:48.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.685+0000 7f94f9e00640 1 --2- 172.21.15.32:0/4120633343 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f94d8068380 0x7f94d806a840 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f94e8000f70 tx=0x7f94e8007df0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:48.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.689+0000 7f94ee000640 1 -- 172.21.15.32:0/4120633343 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f94f00570f0 con 0x7f94f40b5370 2024-04-17T09:54:48.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.853+0000 7f94fb200640 1 -- 172.21.15.32:0/4120633343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f94f41606e0 con 0x7f94f40b5370 2024-04-17T09:54:48.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.853+0000 7f94ee000640 1 -- 172.21.15.32:0/4120633343 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f94f005b3f0 con 0x7f94f40b5370 2024-04-17T09:54:48.854 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:54:48.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.853+0000 7f94fb200640 1 -- 172.21.15.32:0/4120633343 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f94d8068380 msgr2=0x7f94d806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:48.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.853+0000 7f94fb200640 1 --2- 172.21.15.32:0/4120633343 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f94d8068380 0x7f94d806a840 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f94e8000f70 tx=0x7f94e8007df0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.853+0000 7f94fb200640 1 -- 172.21.15.32:0/4120633343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94f40b5370 msgr2=0x7f94f415fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:48.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.853+0000 7f94fb200640 1 --2- 172.21.15.32:0/4120633343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94f40b5370 0x7f94f415fac0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f94f0007e70 tx=0x7f94f000e4b0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.853+0000 7f94fb200640 1 -- 172.21.15.32:0/4120633343 shutdown_connections 2024-04-17T09:54:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.853+0000 7f94fb200640 1 --2- 172.21.15.32:0/4120633343 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f94d8068380 0x7f94d806a840 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.853+0000 7f94fb200640 1 --2- 172.21.15.32:0/4120633343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94f40b5370 0x7f94f415fac0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.853+0000 7f94fb200640 1 --2- 172.21.15.32:0/4120633343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94f40aefa0 0x7f94f4146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.853+0000 7f94fb200640 1 --2- 172.21.15.32:0/4120633343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94f40721b0 0x7f94f4154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.853+0000 7f94fb200640 1 -- 172.21.15.32:0/4120633343 >> 172.21.15.32:0/4120633343 conn(0x7f94f406c1c0 msgr2=0x7f94f415b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.853+0000 7f94fb200640 1 -- 172.21.15.32:0/4120633343 shutdown_connections 2024-04-17T09:54:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:48.853+0000 7f94fb200640 1 -- 172.21.15.32:0/4120633343 wait complete. 2024-04-17T09:54:48.875 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:54:49.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.017+0000 7f6a2b200640 1 Processor -- start 2024-04-17T09:54:49.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.017+0000 7f6a2b200640 1 -- start start 2024-04-17T09:54:49.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.017+0000 7f6a2b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2415b7d0 0x7f6a2406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:49.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.017+0000 7f6a2b200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6a2406f7c0 0x7f6a2406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:49.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.017+0000 7f6a2b200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a240cc940 con 0x7f6a2406f270 2024-04-17T09:54:49.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.017+0000 7f6a2b200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f6a240ccc20 con 0x7f6a2406f7c0 2024-04-17T09:54:49.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.017+0000 7f6a2b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6a2406e0b0 con 0x7f6a2415b7d0 2024-04-17T09:54:49.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.021+0000 7f6a23e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2415b7d0 0x7f6a2406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:49.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.021+0000 7f6a1b400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6a2406f7c0 0x7f6a2406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:49.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.021+0000 7f6a23e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2415b7d0 0x7f6a2406e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:37746/0 (socket says 172.21.15.32:37746) 2024-04-17T09:54:49.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.021+0000 7f6a1b400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6a2406f7c0 0x7f6a2406ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:59366/0 (socket says 172.21.15.32:59366) 2024-04-17T09:54:49.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.021+0000 7f6a23e00640 1 -- 172.21.15.32:0/3496572663 learned_addr learned my addr 172.21.15.32:0/3496572663 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:49.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.021+0000 7f6a29200640 1 --1- 172.21.15.32:0/3496572663 >> v1:172.21.15.144:6789/0 conn(0x7f6a2406f270 0x7f6a240ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:48638/0 (socket says 172.21.15.32:48638) 2024-04-17T09:54:49.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.021+0000 7f6a23e00640 1 -- 172.21.15.32:0/3496572663 >> v2:172.21.15.144:3300/0 conn(0x7f6a2406f7c0 msgr2=0x7f6a2406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:49.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.021+0000 7f6a23e00640 1 --2- 172.21.15.32:0/3496572663 >> v2:172.21.15.144:3300/0 conn(0x7f6a2406f7c0 0x7f6a2406ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.021+0000 7f6a23e00640 1 -- 172.21.15.32:0/3496572663 >> v1:172.21.15.144:6789/0 conn(0x7f6a2406f270 legacy=0x7f6a240ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:49.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.021+0000 7f6a23e00640 1 -- 172.21.15.32:0/3496572663 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2406e390 con 0x7f6a2415b7d0 2024-04-17T09:54:49.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.021+0000 7f6a23e00640 1 --2- 172.21.15.32:0/3496572663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2415b7d0 0x7f6a2406e8b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6a14009870 tx=0x7f6a1401c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18405936f041496d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:49.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.021+0000 7f6a23400640 1 -- 172.21.15.32:0/3496572663 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a140ab5f0 con 0x7f6a2415b7d0 2024-04-17T09:54:49.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.025+0000 7f6a23400640 1 -- 172.21.15.32:0/3496572663 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a140b0070 con 0x7f6a2415b7d0 2024-04-17T09:54:49.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.025+0000 7f6a23400640 1 -- 172.21.15.32:0/3496572663 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a140ab9b0 con 0x7f6a2415b7d0 2024-04-17T09:54:49.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.025+0000 7f6a2b200640 1 -- 172.21.15.32:0/3496572663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2415b7d0 msgr2=0x7f6a2406e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:49.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.025+0000 7f6a2b200640 1 --2- 172.21.15.32:0/3496572663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2415b7d0 0x7f6a2406e8b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6a14009870 tx=0x7f6a1401c4d0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.025+0000 7f6a2b200640 1 -- 172.21.15.32:0/3496572663 shutdown_connections 2024-04-17T09:54:49.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.025+0000 7f6a2b200640 1 --2- 172.21.15.32:0/3496572663 >> v2:172.21.15.144:3300/0 conn(0x7f6a2406f7c0 0x7f6a2406ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.025+0000 7f6a2b200640 1 --2- 172.21.15.32:0/3496572663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2415b7d0 0x7f6a2406e8b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.025+0000 7f6a2b200640 1 -- 172.21.15.32:0/3496572663 >> 172.21.15.32:0/3496572663 conn(0x7f6a240b2120 msgr2=0x7f6a240b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:49.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.025+0000 7f6a2b200640 1 -- 172.21.15.32:0/3496572663 shutdown_connections 2024-04-17T09:54:49.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.025+0000 7f6a2b200640 1 -- 172.21.15.32:0/3496572663 wait complete. 2024-04-17T09:54:49.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.025+0000 7f6a2b200640 1 Processor -- start 2024-04-17T09:54:49.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.025+0000 7f6a2b200640 1 -- start start 2024-04-17T09:54:49.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.025+0000 7f6a2b200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a2406f270 0x7f6a2413eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:49.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.025+0000 7f6a2b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a2406f7c0 0x7f6a2414d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:49.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.025+0000 7f6a2b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2415b7d0 0x7f6a2415e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:49.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.025+0000 7f6a2b200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6a24072670 con 0x7f6a2406f7c0 2024-04-17T09:54:49.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.025+0000 7f6a2b200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2406c380 con 0x7f6a2406f270 2024-04-17T09:54:49.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.025+0000 7f6a2b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6a240727f0 con 0x7f6a2415b7d0 2024-04-17T09:54:49.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.025+0000 7f6a1b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a2406f7c0 0x7f6a2414d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:49.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.025+0000 7f6a1b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a2406f7c0 0x7f6a2414d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51538/0 (socket says 172.21.15.32:51538) 2024-04-17T09:54:49.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.025+0000 7f6a1b400640 1 -- 172.21.15.32:0/676936426 learned_addr learned my addr 172.21.15.32:0/676936426 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:49.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.025+0000 7f6a23e00640 1 --2- 172.21.15.32:0/676936426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a2406f270 0x7f6a2413eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:49.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.025+0000 7f6a29200640 1 --2- 172.21.15.32:0/676936426 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2415b7d0 0x7f6a2415e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:49.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.029+0000 7f6a1b400640 1 -- 172.21.15.32:0/676936426 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2415b7d0 msgr2=0x7f6a2415e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:49.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.029+0000 7f6a1b400640 1 --2- 172.21.15.32:0/676936426 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2415b7d0 0x7f6a2415e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.029+0000 7f6a1b400640 1 -- 172.21.15.32:0/676936426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a2406f270 msgr2=0x7f6a2413eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:49.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.029+0000 7f6a1b400640 1 --2- 172.21.15.32:0/676936426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a2406f270 0x7f6a2413eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.029+0000 7f6a1b400640 1 -- 172.21.15.32:0/676936426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2414df20 con 0x7f6a2406f7c0 2024-04-17T09:54:49.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.029+0000 7f6a1b400640 1 --2- 172.21.15.32:0/676936426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a2406f7c0 0x7f6a2414d9e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00ed40 tx=0x7f6a0c00d720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:49.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.029+0000 7f6a20c00640 1 -- 172.21.15.32:0/676936426 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c00de30 con 0x7f6a2406f7c0 2024-04-17T09:54:49.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.029+0000 7f6a20c00640 1 -- 172.21.15.32:0/676936426 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a0c010650 con 0x7f6a2406f7c0 2024-04-17T09:54:49.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.029+0000 7f6a20c00640 1 -- 172.21.15.32:0/676936426 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c010a00 con 0x7f6a2406f7c0 2024-04-17T09:54:49.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.029+0000 7f6a2b200640 1 -- 172.21.15.32:0/676936426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2415d2d0 con 0x7f6a2406f7c0 2024-04-17T09:54:49.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.029+0000 7f6a2b200640 1 -- 172.21.15.32:0/676936426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a2415d890 con 0x7f6a2406f7c0 2024-04-17T09:54:49.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.029+0000 7f6a2b200640 1 -- 172.21.15.32:0/676936426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a24068a90 con 0x7f6a2406f7c0 2024-04-17T09:54:49.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.029+0000 7f6a20c00640 1 -- 172.21.15.32:0/676936426 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6a0c010ba0 con 0x7f6a2406f7c0 2024-04-17T09:54:49.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.029+0000 7f6a20c00640 1 --2- 172.21.15.32:0/676936426 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6a000683b0 0x7f6a0006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:49.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.029+0000 7f6a20c00640 1 -- 172.21.15.32:0/676936426 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f6a0c087290 con 0x7f6a2406f7c0 2024-04-17T09:54:49.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.029+0000 7f6a23e00640 1 --2- 172.21.15.32:0/676936426 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6a000683b0 0x7f6a0006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:49.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.033+0000 7f6a23e00640 1 --2- 172.21.15.32:0/676936426 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6a000683b0 0x7f6a0006a870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6a140af000 tx=0x7f6a140a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:49.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.033+0000 7f6a20c00640 1 -- 172.21.15.32:0/676936426 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f6a2414df20 con 0x7f6a2406f7c0 2024-04-17T09:54:49.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.197+0000 7f6a2b200640 1 -- 172.21.15.32:0/676936426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6a241552d0 con 0x7f6a2406f7c0 2024-04-17T09:54:49.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.197+0000 7f6a20c00640 1 -- 172.21.15.32:0/676936426 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) ==== 104+0+11 (secure 0 0 0) 0x7f6a0c014030 con 0x7f6a2406f7c0 2024-04-17T09:54:49.200 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:54:49.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.201+0000 7f6a2b200640 1 -- 172.21.15.32:0/676936426 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6a000683b0 msgr2=0x7f6a0006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:49.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.201+0000 7f6a2b200640 1 --2- 172.21.15.32:0/676936426 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6a000683b0 0x7f6a0006a870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6a140af000 tx=0x7f6a140a7000 comp rx=0 tx=0).stop 2024-04-17T09:54:49.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.201+0000 7f6a2b200640 1 -- 172.21.15.32:0/676936426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a2406f7c0 msgr2=0x7f6a2414d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:49.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.201+0000 7f6a2b200640 1 --2- 172.21.15.32:0/676936426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a2406f7c0 0x7f6a2414d9e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00ed40 tx=0x7f6a0c00d720 comp rx=0 tx=0).stop 2024-04-17T09:54:49.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.201+0000 7f6a2b200640 1 -- 172.21.15.32:0/676936426 shutdown_connections 2024-04-17T09:54:49.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.201+0000 7f6a2b200640 1 --2- 172.21.15.32:0/676936426 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6a000683b0 0x7f6a0006a870 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.201+0000 7f6a2b200640 1 --2- 172.21.15.32:0/676936426 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2415b7d0 0x7f6a2415e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.201+0000 7f6a2b200640 1 --2- 172.21.15.32:0/676936426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a2406f7c0 0x7f6a2414d9e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.201+0000 7f6a2b200640 1 --2- 172.21.15.32:0/676936426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a2406f270 0x7f6a2413eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.201+0000 7f6a2b200640 1 -- 172.21.15.32:0/676936426 >> 172.21.15.32:0/676936426 conn(0x7f6a240b2120 msgr2=0x7f6a240b0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:49.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.201+0000 7f6a2b200640 1 -- 172.21.15.32:0/676936426 shutdown_connections 2024-04-17T09:54:49.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.201+0000 7f6a2b200640 1 -- 172.21.15.32:0/676936426 wait complete. 2024-04-17T09:54:49.221 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:54:49.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.361+0000 7f21edc00640 1 Processor -- start 2024-04-17T09:54:49.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.365+0000 7f21edc00640 1 -- start start 2024-04-17T09:54:49.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.365+0000 7f21edc00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f21e815ffc0 0x7f21e80b3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:49.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.365+0000 7f21edc00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21e80cddb0 con 0x7f21e809b7c0 2024-04-17T09:54:49.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.365+0000 7f21edc00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21e80ce090 con 0x7f21e809b100 2024-04-17T09:54:49.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.365+0000 7f21edc00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f21e809bc20 con 0x7f21e815ffc0 2024-04-17T09:54:49.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.365+0000 7f21e7400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f21e809b7c0 0x7f21e809a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48464/0 (socket says 172.21.15.32:48464) 2024-04-17T09:54:49.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.365+0000 7f21e7400640 1 -- 172.21.15.32:0/2849914114 learned_addr learned my addr 172.21.15.32:0/2849914114 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:49.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.365+0000 7f21e7e00640 1 --2- 172.21.15.32:0/2849914114 >> v2:172.21.15.144:3300/0 conn(0x7f21e815ffc0 0x7f21e80b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:49.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.365+0000 7f21e6a00640 1 -- 172.21.15.32:0/2849914114 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 332526178 0 0) 0x7f21e80cddb0 con 0x7f21e809b7c0 2024-04-17T09:54:49.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.365+0000 7f21e6a00640 1 -- 172.21.15.32:0/2849914114 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21cc003530 con 0x7f21e809b7c0 2024-04-17T09:54:49.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.365+0000 7f21e7e00640 1 -- 172.21.15.32:0/2849914114 >> v1:172.21.15.144:6789/0 conn(0x7f21e809b100 legacy=0x7f21e80d0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:49.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.365+0000 7f21e7e00640 1 -- 172.21.15.32:0/2849914114 >> v1:172.21.15.32:6789/0 conn(0x7f21e809b7c0 legacy=0x7f21e809a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:49.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.365+0000 7f21e7e00640 1 -- 172.21.15.32:0/2849914114 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e80ce370 con 0x7f21e815ffc0 2024-04-17T09:54:49.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.369+0000 7f21e7e00640 1 --2- 172.21.15.32:0/2849914114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21e815ffc0 0x7f21e80b3360 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f21d80097c0 tx=0x7f21d801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4f9c2b6fed8a450 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:49.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.369+0000 7f21e6a00640 1 -- 172.21.15.32:0/2849914114 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d80b0070 con 0x7f21e815ffc0 2024-04-17T09:54:49.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.369+0000 7f21e6a00640 1 -- 172.21.15.32:0/2849914114 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21d80ab600 con 0x7f21e815ffc0 2024-04-17T09:54:49.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.369+0000 7f21e6a00640 1 -- 172.21.15.32:0/2849914114 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d80ab9d0 con 0x7f21e815ffc0 2024-04-17T09:54:49.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.369+0000 7f21edc00640 1 -- 172.21.15.32:0/2849914114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21e815ffc0 msgr2=0x7f21e80b3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:49.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.369+0000 7f21edc00640 1 --2- 172.21.15.32:0/2849914114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21e815ffc0 0x7f21e80b3360 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f21d80097c0 tx=0x7f21d801c4a0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.369+0000 7f21edc00640 1 -- 172.21.15.32:0/2849914114 shutdown_connections 2024-04-17T09:54:49.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.369+0000 7f21edc00640 1 --2- 172.21.15.32:0/2849914114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21e815ffc0 0x7f21e80b3360 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.369+0000 7f21edc00640 1 -- 172.21.15.32:0/2849914114 >> 172.21.15.32:0/2849914114 conn(0x7f21e80b4000 msgr2=0x7f21e80b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:49.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.369+0000 7f21edc00640 1 -- 172.21.15.32:0/2849914114 shutdown_connections 2024-04-17T09:54:49.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.369+0000 7f21edc00640 1 -- 172.21.15.32:0/2849914114 wait complete. 2024-04-17T09:54:49.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21edc00640 1 Processor -- start 2024-04-17T09:54:49.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21edc00640 1 -- start start 2024-04-17T09:54:49.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21edc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21e809b100 0x7f21e80af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:49.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21edc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21e809b7c0 0x7f21e80afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:49.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21edc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21e815ffc0 0x7f21e80c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:49.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21edc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f21e816a9f0 con 0x7f21e809b100 2024-04-17T09:54:49.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21edc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f21e816ab70 con 0x7f21e809b7c0 2024-04-17T09:54:49.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21edc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f21e816acf0 con 0x7f21e815ffc0 2024-04-17T09:54:49.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21e7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21e809b100 0x7f21e80af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:49.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21e7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21e809b100 0x7f21e80af160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51544/0 (socket says 172.21.15.32:51544) 2024-04-17T09:54:49.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21e7e00640 1 -- 172.21.15.32:0/54726713 learned_addr learned my addr 172.21.15.32:0/54726713 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:49.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21e7400640 1 --2- 172.21.15.32:0/54726713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21e809b7c0 0x7f21e80afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:49.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21ed200640 1 --2- 172.21.15.32:0/54726713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21e815ffc0 0x7f21e80c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:49.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21e7e00640 1 -- 172.21.15.32:0/54726713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21e815ffc0 msgr2=0x7f21e80c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:49.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21e7e00640 1 --2- 172.21.15.32:0/54726713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21e815ffc0 0x7f21e80c85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21e7e00640 1 -- 172.21.15.32:0/54726713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21e809b7c0 msgr2=0x7f21e80afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:49.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21e7e00640 1 --2- 172.21.15.32:0/54726713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21e809b7c0 0x7f21e80afe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21e7e00640 1 -- 172.21.15.32:0/54726713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e80b1330 con 0x7f21e809b100 2024-04-17T09:54:49.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21e7400640 1 --2- 172.21.15.32:0/54726713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21e809b7c0 0x7f21e80afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:49.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21e7e00640 1 --2- 172.21.15.32:0/54726713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21e809b100 0x7f21e80af160 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f21d80190b0 tx=0x7f21d8019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:49.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21dfe00640 1 -- 172.21.15.32:0/54726713 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d80b0040 con 0x7f21e809b100 2024-04-17T09:54:49.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21dfe00640 1 -- 172.21.15.32:0/54726713 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21d80ba070 con 0x7f21e809b100 2024-04-17T09:54:49.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21edc00640 1 -- 172.21.15.32:0/54726713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e81524d0 con 0x7f21e809b100 2024-04-17T09:54:49.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21dfe00640 1 -- 172.21.15.32:0/54726713 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d80b4630 con 0x7f21e809b100 2024-04-17T09:54:49.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.373+0000 7f21edc00640 1 -- 172.21.15.32:0/54726713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e8152a90 con 0x7f21e809b100 2024-04-17T09:54:49.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.377+0000 7f21edc00640 1 -- 172.21.15.32:0/54726713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21e804fda0 con 0x7f21e809b100 2024-04-17T09:54:49.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.377+0000 7f21dfe00640 1 -- 172.21.15.32:0/54726713 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f21d80b47d0 con 0x7f21e809b100 2024-04-17T09:54:49.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.377+0000 7f21dfe00640 1 --2- 172.21.15.32:0/54726713 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f21c8068380 0x7f21c806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:49.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.377+0000 7f21dfe00640 1 -- 172.21.15.32:0/54726713 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f21d8120670 con 0x7f21e809b100 2024-04-17T09:54:49.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.377+0000 7f21e7400640 1 --2- 172.21.15.32:0/54726713 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f21c8068380 0x7f21c806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:49.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.377+0000 7f21e7400640 1 --2- 172.21.15.32:0/54726713 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f21c8068380 0x7f21c806a840 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f21e80cc8c0 tx=0x7f21d4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:49.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.381+0000 7f21dfe00640 1 -- 172.21.15.32:0/54726713 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f21d8016090 con 0x7f21e809b100 2024-04-17T09:54:49.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.549+0000 7f21edc00640 1 -- 172.21.15.32:0/54726713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f21e8070750 con 0x7f21e809b100 2024-04-17T09:54:49.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.549+0000 7f21dfe00640 1 -- 172.21.15.32:0/54726713 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) ==== 100+0+11 (secure 0 0 0) 0x7f21d80f0b90 con 0x7f21e809b100 2024-04-17T09:54:49.551 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:54:49.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.553+0000 7f21edc00640 1 -- 172.21.15.32:0/54726713 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f21c8068380 msgr2=0x7f21c806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:49.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.553+0000 7f21edc00640 1 --2- 172.21.15.32:0/54726713 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f21c8068380 0x7f21c806a840 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f21e80cc8c0 tx=0x7f21d4009040 comp rx=0 tx=0).stop 2024-04-17T09:54:49.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.553+0000 7f21edc00640 1 -- 172.21.15.32:0/54726713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21e809b100 msgr2=0x7f21e80af160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:49.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.553+0000 7f21edc00640 1 --2- 172.21.15.32:0/54726713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21e809b100 0x7f21e80af160 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f21d80190b0 tx=0x7f21d8019240 comp rx=0 tx=0).stop 2024-04-17T09:54:49.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.553+0000 7f21edc00640 1 -- 172.21.15.32:0/54726713 shutdown_connections 2024-04-17T09:54:49.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.553+0000 7f21edc00640 1 --2- 172.21.15.32:0/54726713 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f21c8068380 0x7f21c806a840 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.553+0000 7f21edc00640 1 --2- 172.21.15.32:0/54726713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21e815ffc0 0x7f21e80c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.553+0000 7f21edc00640 1 --2- 172.21.15.32:0/54726713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21e809b7c0 0x7f21e80afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.553+0000 7f21edc00640 1 --2- 172.21.15.32:0/54726713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21e809b100 0x7f21e80af160 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.553+0000 7f21edc00640 1 -- 172.21.15.32:0/54726713 >> 172.21.15.32:0/54726713 conn(0x7f21e80b4000 msgr2=0x7f21e8157c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:49.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.553+0000 7f21edc00640 1 -- 172.21.15.32:0/54726713 shutdown_connections 2024-04-17T09:54:49.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.553+0000 7f21edc00640 1 -- 172.21.15.32:0/54726713 wait complete. 2024-04-17T09:54:49.573 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:54:49.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe7a3a00640 1 Processor -- start 2024-04-17T09:54:49.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe7a3a00640 1 -- start start 2024-04-17T09:54:49.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe7a3a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe79c144e90 0x7fe79c0b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:49.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe7a3a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe79c072bb0 con 0x7fe79c0b4de0 2024-04-17T09:54:49.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe7a3a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe79c0b3dc0 con 0x7fe79c15c1b0 2024-04-17T09:54:49.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe7a3a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe79c145270 con 0x7fe79c144e90 2024-04-17T09:54:49.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe79be00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe79c0b4de0 0x7fe79c15baa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48478/0 (socket says 172.21.15.32:48478) 2024-04-17T09:54:49.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe79be00640 1 -- 172.21.15.32:0/1398893261 learned_addr learned my addr 172.21.15.32:0/1398893261 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:49.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe7a1000640 1 --2- 172.21.15.32:0/1398893261 >> v2:172.21.15.144:3300/0 conn(0x7fe79c144e90 0x7fe79c0b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:49.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe79b400640 1 -- 172.21.15.32:0/1398893261 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3228826218 0 0) 0x7fe79c072bb0 con 0x7fe79c0b4de0 2024-04-17T09:54:49.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe79b400640 1 -- 172.21.15.32:0/1398893261 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe784003530 con 0x7fe79c0b4de0 2024-04-17T09:54:49.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe79b400640 1 -- 172.21.15.32:0/1398893261 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3335970263 0 0) 0x7fe784003530 con 0x7fe79c0b4de0 2024-04-17T09:54:49.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe79b400640 1 -- 172.21.15.32:0/1398893261 >> v2:172.21.15.144:3300/0 conn(0x7fe79c144e90 msgr2=0x7fe79c0b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:49.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe79b400640 1 --2- 172.21.15.32:0/1398893261 >> v2:172.21.15.144:3300/0 conn(0x7fe79c144e90 0x7fe79c0b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe79b400640 1 -- 172.21.15.32:0/1398893261 >> v1:172.21.15.144:6789/0 conn(0x7fe79c15c1b0 legacy=0x7fe79c06ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:49.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe79b400640 1 -- 172.21.15.32:0/1398893261 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79c0b40a0 con 0x7fe79c0b4de0 2024-04-17T09:54:49.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe79b400640 1 -- 172.21.15.32:0/1398893261 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7fe7880016a0 con 0x7fe79c0b4de0 2024-04-17T09:54:49.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe79b400640 1 -- 172.21.15.32:0/1398893261 >> v1:172.21.15.32:6789/0 conn(0x7fe79c0b4de0 legacy=0x7fe79c15baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:49.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe79b400640 1 --2- 172.21.15.32:0/1398893261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7840039c0 0x7fe784003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:49.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe79b400640 1 --2- 172.21.15.32:0/1398893261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe784004810 0x7fe784004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:49.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe79b400640 1 --2- 172.21.15.32:0/1398893261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7840051d0 0x7fe78400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:49.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe79b400640 1 -- 172.21.15.32:0/1398893261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c145270 con 0x7fe784004810 2024-04-17T09:54:49.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe79b400640 1 -- 172.21.15.32:0/1398893261 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe78400c070 con 0x7fe7840051d0 2024-04-17T09:54:49.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe79b400640 1 -- 172.21.15.32:0/1398893261 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe78400c270 con 0x7fe7840039c0 2024-04-17T09:54:49.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe79be00640 1 --2- 172.21.15.32:0/1398893261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe784004810 0x7fe784004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:49.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe7a1000640 1 --2- 172.21.15.32:0/1398893261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7840039c0 0x7fe784003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:49.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe7a1a00640 1 --2- 172.21.15.32:0/1398893261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7840051d0 0x7fe78400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:49.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe79be00640 1 -- 172.21.15.32:0/1398893261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7840039c0 msgr2=0x7fe784003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:49.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe79be00640 1 --2- 172.21.15.32:0/1398893261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7840039c0 0x7fe784003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe79be00640 1 -- 172.21.15.32:0/1398893261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7840051d0 msgr2=0x7fe78400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:49.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.713+0000 7fe79be00640 1 --2- 172.21.15.32:0/1398893261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7840051d0 0x7fe78400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.717+0000 7fe79be00640 1 -- 172.21.15.32:0/1398893261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe784003530 con 0x7fe784004810 2024-04-17T09:54:49.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.717+0000 7fe7a1a00640 1 -- 172.21.15.32:0/1398893261 reap_dead start 2024-04-17T09:54:49.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.717+0000 7fe79be00640 1 --2- 172.21.15.32:0/1398893261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe784004810 0x7fe784004c90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe788007fc0 tx=0x7fe7880215e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a727243558f05cc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:49.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.717+0000 7fe79b400640 1 -- 172.21.15.32:0/1398893261 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788007d40 con 0x7fe784004810 2024-04-17T09:54:49.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.717+0000 7fe79b400640 1 -- 172.21.15.32:0/1398893261 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe788005ae0 con 0x7fe784004810 2024-04-17T09:54:49.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.717+0000 7fe79b400640 1 -- 172.21.15.32:0/1398893261 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788007070 con 0x7fe784004810 2024-04-17T09:54:49.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.717+0000 7fe7a3a00640 1 -- 172.21.15.32:0/1398893261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe784004810 msgr2=0x7fe784004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:49.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.717+0000 7fe7a3a00640 1 --2- 172.21.15.32:0/1398893261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe784004810 0x7fe784004c90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe788007fc0 tx=0x7fe7880215e0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.717+0000 7fe7a3a00640 1 -- 172.21.15.32:0/1398893261 shutdown_connections 2024-04-17T09:54:49.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.717+0000 7fe7a3a00640 1 --2- 172.21.15.32:0/1398893261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe784004810 0x7fe784004c90 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.717+0000 7fe7a3a00640 1 -- 172.21.15.32:0/1398893261 >> 172.21.15.32:0/1398893261 conn(0x7fe79c06ba30 msgr2=0x7fe79c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:49.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.717+0000 7fe7a3a00640 1 -- 172.21.15.32:0/1398893261 shutdown_connections 2024-04-17T09:54:49.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.717+0000 7fe7a3a00640 1 -- 172.21.15.32:0/1398893261 wait complete. 2024-04-17T09:54:49.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe7a3a00640 1 Processor -- start 2024-04-17T09:54:49.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe7a3a00640 1 -- start start 2024-04-17T09:54:49.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe7a3a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe784004810 0x7fe79c0cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:49.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe7a3a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79c0c5e10 0x7fe79c0c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:49.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe7a3a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe79c0c8cf0 0x7fe79c0c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:49.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe7a3a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c0b4a20 con 0x7fe79c0c5e10 2024-04-17T09:54:49.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe7a3a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c0b48a0 con 0x7fe784004810 2024-04-17T09:54:49.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe7a3a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe79c0b4ba0 con 0x7fe79c0c8cf0 2024-04-17T09:54:49.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe79be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79c0c5e10 0x7fe79c0c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:49.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe79be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79c0c5e10 0x7fe79c0c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51552/0 (socket says 172.21.15.32:51552) 2024-04-17T09:54:49.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe79be00640 1 -- 172.21.15.32:0/4139381558 learned_addr learned my addr 172.21.15.32:0/4139381558 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:49.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe7a1000640 1 --2- 172.21.15.32:0/4139381558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe784004810 0x7fe79c0cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:49.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe7a1a00640 1 --2- 172.21.15.32:0/4139381558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe79c0c8cf0 0x7fe79c0c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:49.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe79be00640 1 -- 172.21.15.32:0/4139381558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe79c0c8cf0 msgr2=0x7fe79c0c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:49.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe79be00640 1 --2- 172.21.15.32:0/4139381558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe79c0c8cf0 0x7fe79c0c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe79be00640 1 -- 172.21.15.32:0/4139381558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe784004810 msgr2=0x7fe79c0cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:49.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe79be00640 1 --2- 172.21.15.32:0/4139381558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe784004810 0x7fe79c0cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe79be00640 1 -- 172.21.15.32:0/4139381558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79c0c96e0 con 0x7fe79c0c5e10 2024-04-17T09:54:49.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe7a1000640 1 --2- 172.21.15.32:0/4139381558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe784004810 0x7fe79c0cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:49.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe79be00640 1 --2- 172.21.15.32:0/4139381558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79c0c5e10 0x7fe79c0c9cd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe788028250 tx=0x7fe784005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:49.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe798c00640 1 -- 172.21.15.32:0/4139381558 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788007d40 con 0x7fe79c0c5e10 2024-04-17T09:54:49.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe798c00640 1 -- 172.21.15.32:0/4139381558 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe788001660 con 0x7fe79c0c5e10 2024-04-17T09:54:49.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe7a3a00640 1 -- 172.21.15.32:0/4139381558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79c0c62d0 con 0x7fe79c0c5e10 2024-04-17T09:54:49.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe798c00640 1 -- 172.21.15.32:0/4139381558 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7880049f0 con 0x7fe79c0c5e10 2024-04-17T09:54:49.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.721+0000 7fe7a3a00640 1 -- 172.21.15.32:0/4139381558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79c0c6890 con 0x7fe79c0c5e10 2024-04-17T09:54:49.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.725+0000 7fe7a3a00640 1 -- 172.21.15.32:0/4139381558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe79c068a90 con 0x7fe79c0c5e10 2024-04-17T09:54:49.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.725+0000 7fe798c00640 1 -- 172.21.15.32:0/4139381558 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe788004b90 con 0x7fe79c0c5e10 2024-04-17T09:54:49.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.725+0000 7fe798c00640 1 --2- 172.21.15.32:0/4139381558 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe77c0683b0 0x7fe77c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:49.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.725+0000 7fe798c00640 1 -- 172.21.15.32:0/4139381558 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fe7881244b0 con 0x7fe79c0c5e10 2024-04-17T09:54:49.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.725+0000 7fe7a1000640 1 --2- 172.21.15.32:0/4139381558 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe77c0683b0 0x7fe77c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:49.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.725+0000 7fe7a1000640 1 --2- 172.21.15.32:0/4139381558 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe77c0683b0 0x7fe77c06a870 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe790009c60 tx=0x7fe790008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:49.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.729+0000 7fe798c00640 1 -- 172.21.15.32:0/4139381558 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fe78801b2f0 con 0x7fe79c0c5e10 2024-04-17T09:54:49.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.929+0000 7fe7a3a00640 1 -- 172.21.15.32:0/4139381558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe79c15edd0 con 0x7fe79c0c5e10 2024-04-17T09:54:49.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.929+0000 7fe798c00640 1 -- 172.21.15.32:0/4139381558 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5199 (secure 0 0 0) 0x7fe7880f4940 con 0x7fe79c0c5e10 2024-04-17T09:54:49.931 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:49.932 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2127207201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2127207201},{"type":"v1","addr":"172.21.15.144:6835","nonce":2127207201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5110,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3782699135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3782699135},{"type":"v1","addr":"172.21.15.144:6834","nonce":3782699135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/385811857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":385811857},{"type":"v1","addr":"172.21.15.32:6836","nonce":385811857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:54:16.745151+0000","modified":"2024-04-17T09:54:19.761481+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5217":{"gid":5217,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/925293029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":925293029},{"type":"v1","addr":"172.21.15.32:6837","nonce":925293029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5217,"qdb_cluster":[5217]},"id":2}]} 2024-04-17T09:54:49.932 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 15 2024-04-17T09:54:49.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.933+0000 7fe7a3a00640 1 -- 172.21.15.32:0/4139381558 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe77c0683b0 msgr2=0x7fe77c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:49.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.933+0000 7fe7a3a00640 1 --2- 172.21.15.32:0/4139381558 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe77c0683b0 0x7fe77c06a870 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe790009c60 tx=0x7fe790008040 comp rx=0 tx=0).stop 2024-04-17T09:54:49.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.933+0000 7fe7a3a00640 1 -- 172.21.15.32:0/4139381558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79c0c5e10 msgr2=0x7fe79c0c9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:49.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.933+0000 7fe7a3a00640 1 --2- 172.21.15.32:0/4139381558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79c0c5e10 0x7fe79c0c9cd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe788028250 tx=0x7fe784005840 comp rx=0 tx=0).stop 2024-04-17T09:54:49.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.933+0000 7fe7a3a00640 1 -- 172.21.15.32:0/4139381558 shutdown_connections 2024-04-17T09:54:49.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.933+0000 7fe7a3a00640 1 --2- 172.21.15.32:0/4139381558 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe77c0683b0 0x7fe77c06a870 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.933+0000 7fe7a3a00640 1 --2- 172.21.15.32:0/4139381558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe79c0c8cf0 0x7fe79c0c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.933+0000 7fe7a3a00640 1 --2- 172.21.15.32:0/4139381558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79c0c5e10 0x7fe79c0c9cd0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.933+0000 7fe7a3a00640 1 --2- 172.21.15.32:0/4139381558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe784004810 0x7fe79c0cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:49.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.933+0000 7fe7a3a00640 1 -- 172.21.15.32:0/4139381558 >> 172.21.15.32:0/4139381558 conn(0x7fe79c06ba30 msgr2=0x7fe79c06ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:49.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.933+0000 7fe7a3a00640 1 -- 172.21.15.32:0/4139381558 shutdown_connections 2024-04-17T09:54:49.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:49.933+0000 7fe7a3a00640 1 -- 172.21.15.32:0/4139381558 wait complete. 2024-04-17T09:54:49.953 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:54:50.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.093+0000 7fd26f800640 1 Processor -- start 2024-04-17T09:54:50.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.093+0000 7fd26f800640 1 -- start start 2024-04-17T09:54:50.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.093+0000 7fd26f800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd26815eb10 0x7fd26815f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:50.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.093+0000 7fd26f800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2680d0f20 0x7fd2681603c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:50.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.093+0000 7fd26f800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2680b2ac0 con 0x7fd2680b2550 2024-04-17T09:54:50.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.093+0000 7fd26f800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd26815ff10 con 0x7fd26815eb10 2024-04-17T09:54:50.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.093+0000 7fd26f800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd2680b2da0 con 0x7fd2680d0f20 2024-04-17T09:54:50.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.093+0000 7fd26ce00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd26815eb10 0x7fd26815f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:50.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.093+0000 7fd26ce00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd26815eb10 0x7fd26815f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51562/0 (socket says 172.21.15.32:51562) 2024-04-17T09:54:50.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.093+0000 7fd26ce00640 1 -- 172.21.15.32:0/2084757457 learned_addr learned my addr 172.21.15.32:0/2084757457 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.093+0000 7fd26d800640 1 --2- 172.21.15.32:0/2084757457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2680d0f20 0x7fd2681603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.093+0000 7fd26ce00640 1 -- 172.21.15.32:0/2084757457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2680d0f20 msgr2=0x7fd2681603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.093+0000 7fd26ce00640 1 --2- 172.21.15.32:0/2084757457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2680d0f20 0x7fd2681603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.093+0000 7fd26ce00640 1 -- 172.21.15.32:0/2084757457 >> v1:172.21.15.144:6789/0 conn(0x7fd2680b2550 legacy=0x7fd2680d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.093+0000 7fd26ce00640 1 -- 172.21.15.32:0/2084757457 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26806bb60 con 0x7fd26815eb10 2024-04-17T09:54:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.093+0000 7fd26d800640 1 --2- 172.21.15.32:0/2084757457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2680d0f20 0x7fd2681603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:54:50.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.097+0000 7fd26ce00640 1 --2- 172.21.15.32:0/2084757457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd26815eb10 0x7fd26815f9d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009870 tx=0x7fd25c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f86232480f03ff server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:50.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.097+0000 7fd267400640 1 -- 172.21.15.32:0/2084757457 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c0b2070 con 0x7fd26815eb10 2024-04-17T09:54:50.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.097+0000 7fd267400640 1 -- 172.21.15.32:0/2084757457 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd25c0a9070 con 0x7fd26815eb10 2024-04-17T09:54:50.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.097+0000 7fd267400640 1 -- 172.21.15.32:0/2084757457 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c0ad880 con 0x7fd26815eb10 2024-04-17T09:54:50.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.097+0000 7fd26f800640 1 -- 172.21.15.32:0/2084757457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd26815eb10 msgr2=0x7fd26815f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:50.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.097+0000 7fd26f800640 1 --2- 172.21.15.32:0/2084757457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd26815eb10 0x7fd26815f9d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009870 tx=0x7fd25c01c490 comp rx=0 tx=0).stop 2024-04-17T09:54:50.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.097+0000 7fd26f800640 1 -- 172.21.15.32:0/2084757457 shutdown_connections 2024-04-17T09:54:50.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.097+0000 7fd26f800640 1 --2- 172.21.15.32:0/2084757457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2680d0f20 0x7fd2681603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:50.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.097+0000 7fd26f800640 1 --2- 172.21.15.32:0/2084757457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd26815eb10 0x7fd26815f9d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:50.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.097+0000 7fd26f800640 1 -- 172.21.15.32:0/2084757457 >> 172.21.15.32:0/2084757457 conn(0x7fd26809a9f0 msgr2=0x7fd2680b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:50.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.097+0000 7fd26f800640 1 -- 172.21.15.32:0/2084757457 shutdown_connections 2024-04-17T09:54:50.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.097+0000 7fd26f800640 1 -- 172.21.15.32:0/2084757457 wait complete. 2024-04-17T09:54:50.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd26f800640 1 Processor -- start 2024-04-17T09:54:50.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd26f800640 1 -- start start 2024-04-17T09:54:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd26f800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2680b2550 0x7fd2680b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd26f800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2680d0f20 0x7fd2681418b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd26f800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd26815eb10 0x7fd268141df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd26f800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd26816bbf0 con 0x7fd2680d0f20 2024-04-17T09:54:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd26f800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd26816ba70 con 0x7fd2680b2550 2024-04-17T09:54:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd26f800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd26816bd70 con 0x7fd26815eb10 2024-04-17T09:54:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd267e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2680d0f20 0x7fd2681418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd267e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2680d0f20 0x7fd2681418b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51566/0 (socket says 172.21.15.32:51566) 2024-04-17T09:54:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd267e00640 1 -- 172.21.15.32:0/942594799 learned_addr learned my addr 172.21.15.32:0/942594799 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd26ce00640 1 --2- 172.21.15.32:0/942594799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2680b2550 0x7fd2680b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd26d800640 1 --2- 172.21.15.32:0/942594799 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd26815eb10 0x7fd268141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:50.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd267e00640 1 -- 172.21.15.32:0/942594799 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd26815eb10 msgr2=0x7fd268141df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:50.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd267e00640 1 --2- 172.21.15.32:0/942594799 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd26815eb10 0x7fd268141df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:50.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd267e00640 1 -- 172.21.15.32:0/942594799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2680b2550 msgr2=0x7fd2680b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:50.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd267e00640 1 --2- 172.21.15.32:0/942594799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2680b2550 0x7fd2680b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:50.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd267e00640 1 -- 172.21.15.32:0/942594799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26815cb50 con 0x7fd2680d0f20 2024-04-17T09:54:50.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd267e00640 1 --2- 172.21.15.32:0/942594799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2680d0f20 0x7fd2681418b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd25400c480 tx=0x7fd25400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:50.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd264c00640 1 -- 172.21.15.32:0/942594799 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd254012830 con 0x7fd2680d0f20 2024-04-17T09:54:50.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd264c00640 1 -- 172.21.15.32:0/942594799 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd254017070 con 0x7fd2680d0f20 2024-04-17T09:54:50.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd26f800640 1 -- 172.21.15.32:0/942594799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd26815ce30 con 0x7fd2680d0f20 2024-04-17T09:54:50.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd264c00640 1 -- 172.21.15.32:0/942594799 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd254012c30 con 0x7fd2680d0f20 2024-04-17T09:54:50.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd26f800640 1 -- 172.21.15.32:0/942594799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd26815d3f0 con 0x7fd2680d0f20 2024-04-17T09:54:50.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.101+0000 7fd26f800640 1 -- 172.21.15.32:0/942594799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd268068a90 con 0x7fd2680d0f20 2024-04-17T09:54:50.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.105+0000 7fd264c00640 1 -- 172.21.15.32:0/942594799 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd25401c460 con 0x7fd2680d0f20 2024-04-17T09:54:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.105+0000 7fd264c00640 1 --2- 172.21.15.32:0/942594799 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd2480683b0 0x7fd24806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.105+0000 7fd264c00640 1 -- 172.21.15.32:0/942594799 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fd2540866f0 con 0x7fd2680d0f20 2024-04-17T09:54:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.105+0000 7fd26ce00640 1 --2- 172.21.15.32:0/942594799 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd2480683b0 0x7fd24806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.105+0000 7fd26ce00640 1 --2- 172.21.15.32:0/942594799 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd2480683b0 0x7fd24806a870 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0af000 tx=0x7fd25c0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:50.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.109+0000 7fd264c00640 1 -- 172.21.15.32:0/942594799 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd254056650 con 0x7fd2680d0f20 2024-04-17T09:54:50.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.277+0000 7fd26f800640 1 -- 172.21.15.32:0/942594799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2680731e0 con 0x7fd2680d0f20 2024-04-17T09:54:50.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.277+0000 7fd264c00640 1 -- 172.21.15.32:0/942594799 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16681 (secure 0 0 0) 0x7fd25405a950 con 0x7fd2680d0f20 2024-04-17T09:54:50.278 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:50.278 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:54:18.351519+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:54:15.261471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-17T09:54:15.704958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:54:50.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.277+0000 7fd26f800640 1 -- 172.21.15.32:0/942594799 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd2480683b0 msgr2=0x7fd24806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:50.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.277+0000 7fd26f800640 1 --2- 172.21.15.32:0/942594799 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd2480683b0 0x7fd24806a870 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0af000 tx=0x7fd25c0a7000 comp rx=0 tx=0).stop 2024-04-17T09:54:50.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.277+0000 7fd26f800640 1 -- 172.21.15.32:0/942594799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2680d0f20 msgr2=0x7fd2681418b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:50.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.277+0000 7fd26f800640 1 --2- 172.21.15.32:0/942594799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2680d0f20 0x7fd2681418b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd25400c480 tx=0x7fd25400c940 comp rx=0 tx=0).stop 2024-04-17T09:54:50.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.277+0000 7fd26f800640 1 -- 172.21.15.32:0/942594799 shutdown_connections 2024-04-17T09:54:50.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.277+0000 7fd26f800640 1 --2- 172.21.15.32:0/942594799 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd2480683b0 0x7fd24806a870 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:50.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.277+0000 7fd26f800640 1 --2- 172.21.15.32:0/942594799 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd26815eb10 0x7fd268141df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:50.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.277+0000 7fd26f800640 1 --2- 172.21.15.32:0/942594799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2680d0f20 0x7fd2681418b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:50.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.277+0000 7fd26f800640 1 --2- 172.21.15.32:0/942594799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2680b2550 0x7fd2680b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:50.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.277+0000 7fd26f800640 1 -- 172.21.15.32:0/942594799 >> 172.21.15.32:0/942594799 conn(0x7fd26809a9f0 msgr2=0x7fd2680ae420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:50.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.277+0000 7fd26f800640 1 -- 172.21.15.32:0/942594799 shutdown_connections 2024-04-17T09:54:50.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.277+0000 7fd26f800640 1 -- 172.21.15.32:0/942594799 wait complete. 2024-04-17T09:54:50.299 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T09:54:50.299 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:54:50.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.437+0000 7fd581200640 1 Processor -- start 2024-04-17T09:54:50.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.437+0000 7fd581200640 1 -- start start 2024-04-17T09:54:50.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.441+0000 7fd581200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd57c15baa0 0x7fd57c15bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:50.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.441+0000 7fd581200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd57c06ef60 0x7fd57c06f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:50.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.441+0000 7fd581200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd57c072bb0 con 0x7fd57c144e90 2024-04-17T09:54:50.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.441+0000 7fd581200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd57c06f8f0 con 0x7fd57c15baa0 2024-04-17T09:54:50.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.441+0000 7fd581200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd57c145270 con 0x7fd57c06ef60 2024-04-17T09:54:50.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.441+0000 7fd573e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd57c15baa0 0x7fd57c15bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:50.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.441+0000 7fd579e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd57c144e90 0x7fd57c0b48a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33200/0 (socket says 172.21.15.32:33200) 2024-04-17T09:54:50.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.441+0000 7fd579e00640 1 -- 172.21.15.32:0/1228614284 learned_addr learned my addr 172.21.15.32:0/1228614284 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:50.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.441+0000 7fd57a800640 1 --2- 172.21.15.32:0/1228614284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd57c06ef60 0x7fd57c06f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:50.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.441+0000 7fd573e00640 1 -- 172.21.15.32:0/1228614284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd57c06ef60 msgr2=0x7fd57c06f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:50.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.441+0000 7fd573e00640 1 --2- 172.21.15.32:0/1228614284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd57c06ef60 0x7fd57c06f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:50.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.441+0000 7fd573e00640 1 -- 172.21.15.32:0/1228614284 >> v1:172.21.15.32:6789/0 conn(0x7fd57c144e90 legacy=0x7fd57c0b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:50.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.441+0000 7fd573e00640 1 -- 172.21.15.32:0/1228614284 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57c0b3dc0 con 0x7fd57c15baa0 2024-04-17T09:54:50.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.441+0000 7fd57a800640 1 --2- 172.21.15.32:0/1228614284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd57c06ef60 0x7fd57c06f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:50.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.441+0000 7fd573e00640 1 --2- 172.21.15.32:0/1228614284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd57c15baa0 0x7fd57c15bf30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd5640097f0 tx=0x7fd56401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d7063c225bee34 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:50.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd579400640 1 -- 172.21.15.32:0/1228614284 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5640b2070 con 0x7fd57c15baa0 2024-04-17T09:54:50.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd579400640 1 -- 172.21.15.32:0/1228614284 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5640ad640 con 0x7fd57c15baa0 2024-04-17T09:54:50.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd579400640 1 -- 172.21.15.32:0/1228614284 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5640ada10 con 0x7fd57c15baa0 2024-04-17T09:54:50.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd581200640 1 -- 172.21.15.32:0/1228614284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd57c15baa0 msgr2=0x7fd57c15bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:50.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd581200640 1 --2- 172.21.15.32:0/1228614284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd57c15baa0 0x7fd57c15bf30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd5640097f0 tx=0x7fd56401c4a0 comp rx=0 tx=0).stop 2024-04-17T09:54:50.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd581200640 1 -- 172.21.15.32:0/1228614284 shutdown_connections 2024-04-17T09:54:50.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd581200640 1 --2- 172.21.15.32:0/1228614284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd57c06ef60 0x7fd57c06f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:50.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd581200640 1 --2- 172.21.15.32:0/1228614284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd57c15baa0 0x7fd57c15bf30 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:50.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd581200640 1 -- 172.21.15.32:0/1228614284 >> 172.21.15.32:0/1228614284 conn(0x7fd57c06ba30 msgr2=0x7fd57c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:50.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd581200640 1 -- 172.21.15.32:0/1228614284 shutdown_connections 2024-04-17T09:54:50.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd581200640 1 -- 172.21.15.32:0/1228614284 wait complete. 2024-04-17T09:54:50.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd581200640 1 Processor -- start 2024-04-17T09:54:50.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd581200640 1 -- start start 2024-04-17T09:54:50.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd581200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd57c06ef60 0x7fd57c15e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:50.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd581200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd57c144e90 0x7fd57c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:50.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd581200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd57c15baa0 0x7fd57c13e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:50.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd581200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c0727f0 con 0x7fd57c06ef60 2024-04-17T09:54:50.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd581200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c072670 con 0x7fd57c15baa0 2024-04-17T09:54:50.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd581200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd57c072970 con 0x7fd57c144e90 2024-04-17T09:54:50.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd579e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd57c06ef60 0x7fd57c15e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:50.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd579e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd57c06ef60 0x7fd57c15e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50836/0 (socket says 172.21.15.32:50836) 2024-04-17T09:54:50.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd579e00640 1 -- 172.21.15.32:0/819497210 learned_addr learned my addr 172.21.15.32:0/819497210 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:50.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd579e00640 1 -- 172.21.15.32:0/819497210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd57c144e90 msgr2=0x7fd57c15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:50.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.445+0000 7fd579e00640 1 --2- 172.21.15.32:0/819497210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd57c144e90 0x7fd57c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:50.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.449+0000 7fd579e00640 1 -- 172.21.15.32:0/819497210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd57c15baa0 msgr2=0x7fd57c13e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:50.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.449+0000 7fd579e00640 1 --2- 172.21.15.32:0/819497210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd57c15baa0 0x7fd57c13e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:50.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.449+0000 7fd579e00640 1 -- 172.21.15.32:0/819497210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57c15f410 con 0x7fd57c06ef60 2024-04-17T09:54:50.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.449+0000 7fd579e00640 1 --2- 172.21.15.32:0/819497210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd57c06ef60 0x7fd57c15e950 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00c420 tx=0x7fd56c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:50.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.449+0000 7fd572000640 1 -- 172.21.15.32:0/819497210 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c017070 con 0x7fd57c06ef60 2024-04-17T09:54:50.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.449+0000 7fd572000640 1 -- 172.21.15.32:0/819497210 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd56c005cf0 con 0x7fd57c06ef60 2024-04-17T09:54:50.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.449+0000 7fd572000640 1 -- 172.21.15.32:0/819497210 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c0072f0 con 0x7fd57c06ef60 2024-04-17T09:54:50.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.449+0000 7fd581200640 1 -- 172.21.15.32:0/819497210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd57c15d230 con 0x7fd57c06ef60 2024-04-17T09:54:50.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.449+0000 7fd581200640 1 -- 172.21.15.32:0/819497210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd57c15d7f0 con 0x7fd57c06ef60 2024-04-17T09:54:50.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.449+0000 7fd581200640 1 -- 172.21.15.32:0/819497210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd57c068a90 con 0x7fd57c06ef60 2024-04-17T09:54:50.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.449+0000 7fd572000640 1 -- 172.21.15.32:0/819497210 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd56c01a070 con 0x7fd57c06ef60 2024-04-17T09:54:50.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.449+0000 7fd572000640 1 --2- 172.21.15.32:0/819497210 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5580683b0 0x7fd55806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:50.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.449+0000 7fd572000640 1 -- 172.21.15.32:0/819497210 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fd56c085cf0 con 0x7fd57c06ef60 2024-04-17T09:54:50.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.449+0000 7fd573e00640 1 --2- 172.21.15.32:0/819497210 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5580683b0 0x7fd55806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:50.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.453+0000 7fd573e00640 1 --2- 172.21.15.32:0/819497210 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5580683b0 0x7fd55806a870 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd5640af000 tx=0x7fd564004740 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:50.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.453+0000 7fd572000640 1 -- 172.21.15.32:0/819497210 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd56c08a070 con 0x7fd57c06ef60 2024-04-17T09:54:50.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.653+0000 7fd581200640 1 -- 172.21.15.32:0/819497210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd57c15cfe0 con 0x7fd57c06ef60 2024-04-17T09:54:50.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.653+0000 7fd572000640 1 -- 172.21.15.32:0/819497210 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5199 (secure 0 0 0) 0x7fd56c0562b0 con 0x7fd57c06ef60 2024-04-17T09:54:50.656 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:50.656 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2127207201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2127207201},{"type":"v1","addr":"172.21.15.144:6835","nonce":2127207201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5110,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3782699135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3782699135},{"type":"v1","addr":"172.21.15.144:6834","nonce":3782699135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/385811857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":385811857},{"type":"v1","addr":"172.21.15.32:6836","nonce":385811857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:54:16.745151+0000","modified":"2024-04-17T09:54:19.761481+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5217":{"gid":5217,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/925293029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":925293029},{"type":"v1","addr":"172.21.15.32:6837","nonce":925293029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5217,"qdb_cluster":[5217]},"id":2}]} 2024-04-17T09:54:50.656 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 15 2024-04-17T09:54:50.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.657+0000 7fd581200640 1 -- 172.21.15.32:0/819497210 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5580683b0 msgr2=0x7fd55806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:50.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.657+0000 7fd581200640 1 --2- 172.21.15.32:0/819497210 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5580683b0 0x7fd55806a870 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd5640af000 tx=0x7fd564004740 comp rx=0 tx=0).stop 2024-04-17T09:54:50.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.657+0000 7fd581200640 1 -- 172.21.15.32:0/819497210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd57c06ef60 msgr2=0x7fd57c15e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:50.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.657+0000 7fd581200640 1 --2- 172.21.15.32:0/819497210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd57c06ef60 0x7fd57c15e950 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00c420 tx=0x7fd56c00c8e0 comp rx=0 tx=0).stop 2024-04-17T09:54:50.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.657+0000 7fd581200640 1 -- 172.21.15.32:0/819497210 shutdown_connections 2024-04-17T09:54:50.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.657+0000 7fd581200640 1 --2- 172.21.15.32:0/819497210 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5580683b0 0x7fd55806a870 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:50.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.657+0000 7fd581200640 1 --2- 172.21.15.32:0/819497210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd57c144e90 0x7fd57c15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:50.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.657+0000 7fd581200640 1 --2- 172.21.15.32:0/819497210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd57c15baa0 0x7fd57c13e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:50.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.657+0000 7fd581200640 1 --2- 172.21.15.32:0/819497210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd57c06ef60 0x7fd57c15e950 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:50.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.657+0000 7fd581200640 1 -- 172.21.15.32:0/819497210 >> 172.21.15.32:0/819497210 conn(0x7fd57c06ba30 msgr2=0x7fd57c06de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:50.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.657+0000 7fd581200640 1 -- 172.21.15.32:0/819497210 shutdown_connections 2024-04-17T09:54:50.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.657+0000 7fd581200640 1 -- 172.21.15.32:0/819497210 wait complete. 2024-04-17T09:54:50.677 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:54:50.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.817+0000 7f6aec800640 1 Processor -- start 2024-04-17T09:54:50.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.817+0000 7f6aec800640 1 -- start start 2024-04-17T09:54:50.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.817+0000 7f6aec800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ae40cd580 0x7f6ae4167560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:50.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.817+0000 7f6aec800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ae40cdac0 con 0x7f6ae4072500 2024-04-17T09:54:50.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.817+0000 7f6aec800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ae415cdf0 con 0x7f6ae4145f30 2024-04-17T09:54:50.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.817+0000 7f6aec800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6ae406f400 con 0x7f6ae40cd580 2024-04-17T09:54:50.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.817+0000 7f6ae9400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6ae4072500 0x7f6ae40cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33204/0 (socket says 172.21.15.32:33204) 2024-04-17T09:54:50.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.817+0000 7f6ae9400640 1 -- 172.21.15.32:0/1991127581 learned_addr learned my addr 172.21.15.32:0/1991127581 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:50.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.817+0000 7f6aea800640 1 --2- 172.21.15.32:0/1991127581 >> v2:172.21.15.32:3300/0 conn(0x7f6ae40cd580 0x7f6ae4167560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:50.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.817+0000 7f6ae8a00640 1 -- 172.21.15.32:0/1991127581 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2755367924 0 0) 0x7f6ae40cdac0 con 0x7f6ae4072500 2024-04-17T09:54:50.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.817+0000 7f6ae8a00640 1 -- 172.21.15.32:0/1991127581 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6acc003530 con 0x7f6ae4072500 2024-04-17T09:54:50.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.817+0000 7f6aea800640 1 -- 172.21.15.32:0/1991127581 >> v1:172.21.15.144:6789/0 conn(0x7f6ae4145f30 legacy=0x7f6ae4071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:50.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.817+0000 7f6aea800640 1 -- 172.21.15.32:0/1991127581 >> v1:172.21.15.32:6789/0 conn(0x7f6ae4072500 legacy=0x7f6ae40cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:50.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.817+0000 7f6aea800640 1 -- 172.21.15.32:0/1991127581 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae415d0d0 con 0x7f6ae40cd580 2024-04-17T09:54:50.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.821+0000 7f6aea800640 1 --2- 172.21.15.32:0/1991127581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ae40cd580 0x7f6ae4167560 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009d90 tx=0x7f6ae001f120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c044268e762b2c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:50.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.821+0000 7f6ae8a00640 1 -- 172.21.15.32:0/1991127581 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00b4070 con 0x7f6ae40cd580 2024-04-17T09:54:50.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.821+0000 7f6ae8a00640 1 -- 172.21.15.32:0/1991127581 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ae00af600 con 0x7f6ae40cd580 2024-04-17T09:54:50.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.821+0000 7f6ae8a00640 1 -- 172.21.15.32:0/1991127581 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00af9d0 con 0x7f6ae40cd580 2024-04-17T09:54:50.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.821+0000 7f6aec800640 1 -- 172.21.15.32:0/1991127581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ae40cd580 msgr2=0x7f6ae4167560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:50.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.821+0000 7f6aec800640 1 --2- 172.21.15.32:0/1991127581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ae40cd580 0x7f6ae4167560 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009d90 tx=0x7f6ae001f120 comp rx=0 tx=0).stop 2024-04-17T09:54:50.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.821+0000 7f6aec800640 1 -- 172.21.15.32:0/1991127581 shutdown_connections 2024-04-17T09:54:50.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.821+0000 7f6aec800640 1 --2- 172.21.15.32:0/1991127581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ae40cd580 0x7f6ae4167560 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:50.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.821+0000 7f6aec800640 1 -- 172.21.15.32:0/1991127581 >> 172.21.15.32:0/1991127581 conn(0x7f6ae406ba30 msgr2=0x7f6ae406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:50.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.821+0000 7f6aec800640 1 -- 172.21.15.32:0/1991127581 shutdown_connections 2024-04-17T09:54:50.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.821+0000 7f6aec800640 1 -- 172.21.15.32:0/1991127581 wait complete. 2024-04-17T09:54:50.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.821+0000 7f6aec800640 1 Processor -- start 2024-04-17T09:54:50.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6aec800640 1 -- start start 2024-04-17T09:54:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6aec800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ae4072500 0x7f6ae4140750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6aec800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ae40cd580 0x7f6ae4142a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6aec800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ae4145f30 0x7f6ae4142f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6aec800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae4146310 con 0x7f6ae4145f30 2024-04-17T09:54:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6aec800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae415dba0 con 0x7f6ae4072500 2024-04-17T09:54:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6aec800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6ae415d440 con 0x7f6ae40cd580 2024-04-17T09:54:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6aea800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ae4145f30 0x7f6ae4142f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6aea800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ae4145f30 0x7f6ae4142f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50864/0 (socket says 172.21.15.32:50864) 2024-04-17T09:54:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6aea800640 1 -- 172.21.15.32:0/1706063263 learned_addr learned my addr 172.21.15.32:0/1706063263 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6ae9e00640 1 --2- 172.21.15.32:0/1706063263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ae4072500 0x7f6ae4140750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:50.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6ae9400640 1 --2- 172.21.15.32:0/1706063263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ae40cd580 0x7f6ae4142a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:50.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6aea800640 1 -- 172.21.15.32:0/1706063263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ae40cd580 msgr2=0x7f6ae4142a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:50.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6aea800640 1 --2- 172.21.15.32:0/1706063263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ae40cd580 0x7f6ae4142a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:50.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6aea800640 1 -- 172.21.15.32:0/1706063263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ae4072500 msgr2=0x7f6ae4140750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:50.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6aea800640 1 --2- 172.21.15.32:0/1706063263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ae4072500 0x7f6ae4140750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:50.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6aea800640 1 -- 172.21.15.32:0/1706063263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae40b54c0 con 0x7f6ae4145f30 2024-04-17T09:54:50.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6ae9400640 1 --2- 172.21.15.32:0/1706063263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ae40cd580 0x7f6ae4142a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:50.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6ae9e00640 1 --2- 172.21.15.32:0/1706063263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ae4072500 0x7f6ae4140750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:50.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6aea800640 1 --2- 172.21.15.32:0/1706063263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ae4145f30 0x7f6ae4142f40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6ae001cd50 tx=0x7f6ae001bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:50.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6ade000640 1 -- 172.21.15.32:0/1706063263 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00b4040 con 0x7f6ae4145f30 2024-04-17T09:54:50.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6ade000640 1 -- 172.21.15.32:0/1706063263 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ae00c1070 con 0x7f6ae4145f30 2024-04-17T09:54:50.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6ade000640 1 -- 172.21.15.32:0/1706063263 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0009450 con 0x7f6ae4145f30 2024-04-17T09:54:50.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6aec800640 1 -- 172.21.15.32:0/1706063263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae40b4f00 con 0x7f6ae4145f30 2024-04-17T09:54:50.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6aec800640 1 -- 172.21.15.32:0/1706063263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae40c98b0 con 0x7f6ae4145f30 2024-04-17T09:54:50.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.825+0000 7f6aec800640 1 -- 172.21.15.32:0/1706063263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ae4068a90 con 0x7f6ae4145f30 2024-04-17T09:54:50.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.829+0000 7f6ade000640 1 -- 172.21.15.32:0/1706063263 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6ae00ba690 con 0x7f6ae4145f30 2024-04-17T09:54:50.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.829+0000 7f6ade000640 1 --2- 172.21.15.32:0/1706063263 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6ac8068380 0x7f6ac806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:50.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.829+0000 7f6ade000640 1 -- 172.21.15.32:0/1706063263 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f6ae0125310 con 0x7f6ae4145f30 2024-04-17T09:54:50.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.829+0000 7f6ae9e00640 1 --2- 172.21.15.32:0/1706063263 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6ac8068380 0x7f6ac806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:50.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.829+0000 7f6ae9e00640 1 --2- 172.21.15.32:0/1706063263 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6ac8068380 0x7f6ac806a840 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80098b0 tx=0x7f6ad8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:50.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:50.833+0000 7f6ade000640 1 -- 172.21.15.32:0/1706063263 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f6ae0018c80 con 0x7f6ae4145f30 2024-04-17T09:54:51.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.037+0000 7f6aec800640 1 -- 172.21.15.32:0/1706063263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ae4072f90 con 0x7f6ae4145f30 2024-04-17T09:54:51.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.037+0000 7f6ade000640 1 -- 172.21.15.32:0/1706063263 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5199 (secure 0 0 0) 0x7f6ae00f5960 con 0x7f6ae4145f30 2024-04-17T09:54:51.037 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:51.038 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2127207201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2127207201},{"type":"v1","addr":"172.21.15.144:6835","nonce":2127207201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5110,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3782699135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3782699135},{"type":"v1","addr":"172.21.15.144:6834","nonce":3782699135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/385811857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":385811857},{"type":"v1","addr":"172.21.15.32:6836","nonce":385811857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:54:16.745151+0000","modified":"2024-04-17T09:54:19.761481+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5217":{"gid":5217,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/925293029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":925293029},{"type":"v1","addr":"172.21.15.32:6837","nonce":925293029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5217,"qdb_cluster":[5217]},"id":2}]} 2024-04-17T09:54:51.038 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 15 2024-04-17T09:54:51.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.037+0000 7f6aec800640 1 -- 172.21.15.32:0/1706063263 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6ac8068380 msgr2=0x7f6ac806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:51.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.037+0000 7f6aec800640 1 --2- 172.21.15.32:0/1706063263 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6ac8068380 0x7f6ac806a840 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80098b0 tx=0x7f6ad8008040 comp rx=0 tx=0).stop 2024-04-17T09:54:51.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.037+0000 7f6aec800640 1 -- 172.21.15.32:0/1706063263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ae4145f30 msgr2=0x7f6ae4142f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:51.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.037+0000 7f6aec800640 1 --2- 172.21.15.32:0/1706063263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ae4145f30 0x7f6ae4142f40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6ae001cd50 tx=0x7f6ae001bec0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.037+0000 7f6aec800640 1 -- 172.21.15.32:0/1706063263 shutdown_connections 2024-04-17T09:54:51.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.037+0000 7f6aec800640 1 --2- 172.21.15.32:0/1706063263 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6ac8068380 0x7f6ac806a840 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.037+0000 7f6aec800640 1 --2- 172.21.15.32:0/1706063263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ae40cd580 0x7f6ae4142a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.037+0000 7f6aec800640 1 --2- 172.21.15.32:0/1706063263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ae4145f30 0x7f6ae4142f40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.037+0000 7f6aec800640 1 --2- 172.21.15.32:0/1706063263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ae4072500 0x7f6ae4140750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.037+0000 7f6aec800640 1 -- 172.21.15.32:0/1706063263 >> 172.21.15.32:0/1706063263 conn(0x7f6ae406ba30 msgr2=0x7f6ae4143e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:51.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.037+0000 7f6aec800640 1 -- 172.21.15.32:0/1706063263 shutdown_connections 2024-04-17T09:54:51.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.037+0000 7f6aec800640 1 -- 172.21.15.32:0/1706063263 wait complete. 2024-04-17T09:54:51.059 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:54:51.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40bb400640 1 Processor -- start 2024-04-17T09:54:51.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40bb400640 1 -- start start 2024-04-17T09:54:51.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40bb400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40b41481e0 0x7f40b4071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:51.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40bb400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40b406ef70 con 0x7f40b4072330 2024-04-17T09:54:51.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40bb400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40b40aed20 con 0x7f40b4144f20 2024-04-17T09:54:51.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40bb400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f40b40af000 con 0x7f40b41481e0 2024-04-17T09:54:51.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b9400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f40b4072330 0x7f40b40b3490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33220/0 (socket says 172.21.15.32:33220) 2024-04-17T09:54:51.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b9400640 1 -- 172.21.15.32:0/3318545542 learned_addr learned my addr 172.21.15.32:0/3318545542 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:51.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b3e00640 1 --2- 172.21.15.32:0/3318545542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40b41481e0 0x7f40b4071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:51.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b3400640 1 -- 172.21.15.32:0/3318545542 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1459830459 0 0) 0x7f40b406ef70 con 0x7f40b4072330 2024-04-17T09:54:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b3400640 1 -- 172.21.15.32:0/3318545542 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f409c003530 con 0x7f40b4072330 2024-04-17T09:54:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b3400640 1 -- 172.21.15.32:0/3318545542 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 191434307 0 0) 0x7f409c003530 con 0x7f40b4072330 2024-04-17T09:54:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b3400640 1 -- 172.21.15.32:0/3318545542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40b41481e0 msgr2=0x7f40b4071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b3400640 1 --2- 172.21.15.32:0/3318545542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40b41481e0 0x7f40b4071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b3400640 1 -- 172.21.15.32:0/3318545542 >> v1:172.21.15.144:6789/0 conn(0x7f40b4144f20 legacy=0x7f40b406e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b3400640 1 -- 172.21.15.32:0/3318545542 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b40af2e0 con 0x7f40b4072330 2024-04-17T09:54:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b3400640 1 -- 172.21.15.32:0/3318545542 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f40ac001a00 con 0x7f40b4072330 2024-04-17T09:54:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b3400640 1 -- 172.21.15.32:0/3318545542 >> v1:172.21.15.32:6789/0 conn(0x7f40b4072330 legacy=0x7f40b40b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b3400640 1 --2- 172.21.15.32:0/3318545542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40b41481e0 0x7f409c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b3400640 1 --2- 172.21.15.32:0/3318545542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f409c003fc0 0x7f409c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b3400640 1 --2- 172.21.15.32:0/3318545542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f409c004d50 0x7f409c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b3400640 1 -- 172.21.15.32:0/3318545542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f40b40af000 con 0x7f409c004d50 2024-04-17T09:54:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b3400640 1 -- 172.21.15.32:0/3318545542 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f409c00bb10 con 0x7f409c003fc0 2024-04-17T09:54:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b3400640 1 -- 172.21.15.32:0/3318545542 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f409c00bd10 con 0x7f40b41481e0 2024-04-17T09:54:51.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b9400640 1 --2- 172.21.15.32:0/3318545542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f409c004d50 0x7f409c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:51.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b3e00640 1 --2- 172.21.15.32:0/3318545542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f409c003fc0 0x7f409c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:51.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b8a00640 1 --2- 172.21.15.32:0/3318545542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40b41481e0 0x7f409c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:51.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b9400640 1 -- 172.21.15.32:0/3318545542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40b41481e0 msgr2=0x7f409c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:51.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b9400640 1 --2- 172.21.15.32:0/3318545542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40b41481e0 0x7f409c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b9400640 1 -- 172.21.15.32:0/3318545542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f409c003fc0 msgr2=0x7f409c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:51.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b9400640 1 --2- 172.21.15.32:0/3318545542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f409c003fc0 0x7f409c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.201+0000 7f40b9400640 1 -- 172.21.15.32:0/3318545542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409c003530 con 0x7f409c004d50 2024-04-17T09:54:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.205+0000 7f40b9400640 1 --2- 172.21.15.32:0/3318545542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f409c004d50 0x7f409c0051d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f40ac00a7b0 tx=0x7f40ac01d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595c3e7c08a5ecc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.205+0000 7f40b3400640 1 -- 172.21.15.32:0/3318545542 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac00a3d0 con 0x7f409c004d50 2024-04-17T09:54:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.205+0000 7f40b3400640 1 -- 172.21.15.32:0/3318545542 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40ac0ad630 con 0x7f409c004d50 2024-04-17T09:54:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.205+0000 7f40b3400640 1 -- 172.21.15.32:0/3318545542 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac0adaa0 con 0x7f409c004d50 2024-04-17T09:54:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.205+0000 7f40bb400640 1 -- 172.21.15.32:0/3318545542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f409c004d50 msgr2=0x7f409c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.205+0000 7f40bb400640 1 --2- 172.21.15.32:0/3318545542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f409c004d50 0x7f409c0051d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f40ac00a7b0 tx=0x7f40ac01d2e0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.205+0000 7f40b9400640 1 -- 172.21.15.32:0/3318545542 reap_dead start 2024-04-17T09:54:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.205+0000 7f40bb400640 1 -- 172.21.15.32:0/3318545542 shutdown_connections 2024-04-17T09:54:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.205+0000 7f40bb400640 1 -- 172.21.15.32:0/3318545542 >> 172.21.15.32:0/3318545542 conn(0x7f40b406ba30 msgr2=0x7f40b406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.205+0000 7f40bb400640 1 -- 172.21.15.32:0/3318545542 shutdown_connections 2024-04-17T09:54:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.205+0000 7f40bb400640 1 -- 172.21.15.32:0/3318545542 wait complete. 2024-04-17T09:54:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40bb400640 1 Processor -- start 2024-04-17T09:54:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40bb400640 1 -- start start 2024-04-17T09:54:51.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40bb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f409c004d50 0x7f40b413e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:51.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40bb400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40b40b13f0 0x7f40b40b1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:51.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40bb400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40b415da00 0x7f40b415de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:51.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40bb400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f40b40afdc0 con 0x7f409c004d50 2024-04-17T09:54:51.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40bb400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f40b40af8a0 con 0x7f40b40b13f0 2024-04-17T09:54:51.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40bb400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f40b40b2c30 con 0x7f40b415da00 2024-04-17T09:54:51.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40b8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f409c004d50 0x7f40b413e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:51.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40b8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f409c004d50 0x7f40b413e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50888/0 (socket says 172.21.15.32:50888) 2024-04-17T09:54:51.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40b8a00640 1 -- 172.21.15.32:0/2512862167 learned_addr learned my addr 172.21.15.32:0/2512862167 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:51.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40b3e00640 1 --2- 172.21.15.32:0/2512862167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40b40b13f0 0x7f40b40b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40b9400640 1 --2- 172.21.15.32:0/2512862167 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40b415da00 0x7f40b415de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40b8a00640 1 -- 172.21.15.32:0/2512862167 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40b415da00 msgr2=0x7f40b415de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40b8a00640 1 --2- 172.21.15.32:0/2512862167 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40b415da00 0x7f40b415de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40b8a00640 1 -- 172.21.15.32:0/2512862167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40b40b13f0 msgr2=0x7f40b40b1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40b8a00640 1 --2- 172.21.15.32:0/2512862167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40b40b13f0 0x7f40b40b1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40b8a00640 1 -- 172.21.15.32:0/2512862167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b40b1d90 con 0x7f409c004d50 2024-04-17T09:54:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40b3e00640 1 --2- 172.21.15.32:0/2512862167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40b40b13f0 0x7f40b40b1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40b8a00640 1 --2- 172.21.15.32:0/2512862167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f409c004d50 0x7f40b413e760 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f40a800e4d0 tx=0x7f40a800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40b0c00640 1 -- 172.21.15.32:0/2512862167 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a80053a0 con 0x7f409c004d50 2024-04-17T09:54:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40b0c00640 1 -- 172.21.15.32:0/2512862167 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40a8018070 con 0x7f409c004d50 2024-04-17T09:54:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40b0c00640 1 -- 172.21.15.32:0/2512862167 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8003820 con 0x7f409c004d50 2024-04-17T09:54:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40bb400640 1 -- 172.21.15.32:0/2512862167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b40b2020 con 0x7f409c004d50 2024-04-17T09:54:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.209+0000 7f40bb400640 1 -- 172.21.15.32:0/2512862167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b40b25e0 con 0x7f409c004d50 2024-04-17T09:54:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.213+0000 7f40bb400640 1 -- 172.21.15.32:0/2512862167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b4068a90 con 0x7f409c004d50 2024-04-17T09:54:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.213+0000 7f40b0c00640 1 -- 172.21.15.32:0/2512862167 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f40a80039c0 con 0x7f409c004d50 2024-04-17T09:54:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.213+0000 7f40b0c00640 1 --2- 172.21.15.32:0/2512862167 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40940683b0 0x7f409406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:51.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.213+0000 7f40b0c00640 1 -- 172.21.15.32:0/2512862167 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f40a8086460 con 0x7f409c004d50 2024-04-17T09:54:51.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.213+0000 7f40b3e00640 1 --2- 172.21.15.32:0/2512862167 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40940683b0 0x7f409406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:51.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.213+0000 7f40b3e00640 1 --2- 172.21.15.32:0/2512862167 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40940683b0 0x7f409406a870 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f40b4143e90 tx=0x7f40a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.217+0000 7f40b0c00640 1 -- 172.21.15.32:0/2512862167 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f40a80568f0 con 0x7f409c004d50 2024-04-17T09:54:51.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.381+0000 7f40bb400640 1 -- 172.21.15.32:0/2512862167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f40b4144300 con 0x7f409c004d50 2024-04-17T09:54:51.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.385+0000 7f40b0c00640 1 -- 172.21.15.32:0/2512862167 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16681 (secure 0 0 0) 0x7f40a80561f0 con 0x7f409c004d50 2024-04-17T09:54:51.386 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:51.386 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:54:18.351519+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:54:15.261471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-17T09:54:15.704958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:54:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.385+0000 7f40bb400640 1 -- 172.21.15.32:0/2512862167 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40940683b0 msgr2=0x7f409406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.385+0000 7f40bb400640 1 --2- 172.21.15.32:0/2512862167 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40940683b0 0x7f409406a870 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f40b4143e90 tx=0x7f40a0008040 comp rx=0 tx=0).stop 2024-04-17T09:54:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.385+0000 7f40bb400640 1 -- 172.21.15.32:0/2512862167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f409c004d50 msgr2=0x7f40b413e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.385+0000 7f40bb400640 1 --2- 172.21.15.32:0/2512862167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f409c004d50 0x7f40b413e760 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f40a800e4d0 tx=0x7f40a800e990 comp rx=0 tx=0).stop 2024-04-17T09:54:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.385+0000 7f40bb400640 1 -- 172.21.15.32:0/2512862167 shutdown_connections 2024-04-17T09:54:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.385+0000 7f40bb400640 1 --2- 172.21.15.32:0/2512862167 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40940683b0 0x7f409406a870 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.385+0000 7f40bb400640 1 --2- 172.21.15.32:0/2512862167 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40b415da00 0x7f40b415de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.385+0000 7f40bb400640 1 --2- 172.21.15.32:0/2512862167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40b40b13f0 0x7f40b40b1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.385+0000 7f40bb400640 1 --2- 172.21.15.32:0/2512862167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f409c004d50 0x7f40b413e760 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.385+0000 7f40bb400640 1 -- 172.21.15.32:0/2512862167 >> 172.21.15.32:0/2512862167 conn(0x7f40b406ba30 msgr2=0x7f40b40b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.385+0000 7f40bb400640 1 -- 172.21.15.32:0/2512862167 shutdown_connections 2024-04-17T09:54:51.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.385+0000 7f40bb400640 1 -- 172.21.15.32:0/2512862167 wait complete. 2024-04-17T09:54:51.407 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T09:54:51.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.549+0000 7fc125400640 1 Processor -- start 2024-04-17T09:54:51.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.549+0000 7fc125400640 1 -- start start 2024-04-17T09:54:51.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.549+0000 7fc125400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc120155e50 0x7fc12015f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:51.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.549+0000 7fc125400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc12015f870 con 0x7fc120155a70 2024-04-17T09:54:51.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.549+0000 7fc125400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc12015fb50 con 0x7fc1200b3340 2024-04-17T09:54:51.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.549+0000 7fc125400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc1200b42a0 con 0x7fc120155e50 2024-04-17T09:54:51.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.549+0000 7fc11de00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc120155e50 0x7fc12015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:51.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.549+0000 7fc11e800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc120155a70 0x7fc12015d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33236/0 (socket says 172.21.15.32:33236) 2024-04-17T09:54:51.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.549+0000 7fc11e800640 1 -- 172.21.15.32:0/1713313147 learned_addr learned my addr 172.21.15.32:0/1713313147 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:51.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.549+0000 7fc11de00640 1 -- 172.21.15.32:0/1713313147 >> v1:172.21.15.144:6789/0 conn(0x7fc1200b3340 legacy=0x7fc12015c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:51.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.549+0000 7fc11d400640 1 -- 172.21.15.32:0/1713313147 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3661889764 0 0) 0x7fc12015f870 con 0x7fc120155a70 2024-04-17T09:54:51.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.549+0000 7fc11de00640 1 -- 172.21.15.32:0/1713313147 >> v1:172.21.15.32:6789/0 conn(0x7fc120155a70 legacy=0x7fc12015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:51.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.549+0000 7fc11de00640 1 -- 172.21.15.32:0/1713313147 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12015fe30 con 0x7fc120155e50 2024-04-17T09:54:51.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.553+0000 7fc11de00640 1 --2- 172.21.15.32:0/1713313147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc120155e50 0x7fc12015f190 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc1200b4420 tx=0x7fc10c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d504d6950d41f30 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:51.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.553+0000 7fc11d400640 1 -- 172.21.15.32:0/1713313147 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c0b0070 con 0x7fc120155e50 2024-04-17T09:54:51.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.553+0000 7fc11d400640 1 -- 172.21.15.32:0/1713313147 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc10c0ab600 con 0x7fc120155e50 2024-04-17T09:54:51.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.553+0000 7fc11d400640 1 -- 172.21.15.32:0/1713313147 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c0ab9d0 con 0x7fc120155e50 2024-04-17T09:54:51.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.553+0000 7fc125400640 1 -- 172.21.15.32:0/1713313147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc120155e50 msgr2=0x7fc12015f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:51.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.553+0000 7fc125400640 1 --2- 172.21.15.32:0/1713313147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc120155e50 0x7fc12015f190 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc1200b4420 tx=0x7fc10c01c480 comp rx=0 tx=0).stop 2024-04-17T09:54:51.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.553+0000 7fc125400640 1 -- 172.21.15.32:0/1713313147 shutdown_connections 2024-04-17T09:54:51.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.553+0000 7fc125400640 1 --2- 172.21.15.32:0/1713313147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc120155e50 0x7fc12015f190 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.553+0000 7fc125400640 1 -- 172.21.15.32:0/1713313147 >> 172.21.15.32:0/1713313147 conn(0x7fc12006ba30 msgr2=0x7fc12006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:51.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.553+0000 7fc125400640 1 -- 172.21.15.32:0/1713313147 shutdown_connections 2024-04-17T09:54:51.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.553+0000 7fc125400640 1 -- 172.21.15.32:0/1713313147 wait complete. 2024-04-17T09:54:51.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.553+0000 7fc125400640 1 Processor -- start 2024-04-17T09:54:51.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.553+0000 7fc125400640 1 -- start start 2024-04-17T09:54:51.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc125400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1200b3340 0x7fc120149e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:51.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc125400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc120155a70 0x7fc1201544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:51.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc125400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc120155e50 0x7fc120154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:51.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc125400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc1200d02f0 con 0x7fc120155a70 2024-04-17T09:54:51.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc125400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc1200d0170 con 0x7fc120155e50 2024-04-17T09:54:51.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc125400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc1200d0470 con 0x7fc1200b3340 2024-04-17T09:54:51.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc11de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc120155a70 0x7fc1201544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:51.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc11de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc120155a70 0x7fc1201544c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50912/0 (socket says 172.21.15.32:50912) 2024-04-17T09:54:51.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc11de00640 1 -- 172.21.15.32:0/3859508789 learned_addr learned my addr 172.21.15.32:0/3859508789 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:51.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc11e800640 1 --2- 172.21.15.32:0/3859508789 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1200b3340 0x7fc120149e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:51.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc11f200640 1 --2- 172.21.15.32:0/3859508789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc120155e50 0x7fc120154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:51.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc11de00640 1 -- 172.21.15.32:0/3859508789 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1200b3340 msgr2=0x7fc120149e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:51.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc11de00640 1 --2- 172.21.15.32:0/3859508789 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1200b3340 0x7fc120149e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc11de00640 1 -- 172.21.15.32:0/3859508789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc120155e50 msgr2=0x7fc120154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:51.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc11de00640 1 --2- 172.21.15.32:0/3859508789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc120155e50 0x7fc120154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc11de00640 1 -- 172.21.15.32:0/3859508789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc120159e90 con 0x7fc120155a70 2024-04-17T09:54:51.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc11e800640 1 --2- 172.21.15.32:0/3859508789 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1200b3340 0x7fc120149e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:51.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc11de00640 1 --2- 172.21.15.32:0/3859508789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc120155a70 0x7fc1201544c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0ae040 tx=0x7fc10c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:51.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc116a00640 1 -- 172.21.15.32:0/3859508789 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c0b0050 con 0x7fc120155a70 2024-04-17T09:54:51.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc116a00640 1 -- 172.21.15.32:0/3859508789 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc10c0bc070 con 0x7fc120155a70 2024-04-17T09:54:51.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc116a00640 1 -- 172.21.15.32:0/3859508789 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c0b5a70 con 0x7fc120155a70 2024-04-17T09:54:51.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc125400640 1 -- 172.21.15.32:0/3859508789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc12015a170 con 0x7fc120155a70 2024-04-17T09:54:51.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc125400640 1 -- 172.21.15.32:0/3859508789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc120159ba0 con 0x7fc120155a70 2024-04-17T09:54:51.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.557+0000 7fc125400640 1 -- 172.21.15.32:0/3859508789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc120068a90 con 0x7fc120155a70 2024-04-17T09:54:51.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.561+0000 7fc116a00640 1 -- 172.21.15.32:0/3859508789 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc10c0ab4c0 con 0x7fc120155a70 2024-04-17T09:54:51.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.561+0000 7fc116a00640 1 --2- 172.21.15.32:0/3859508789 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc100068380 0x7fc10006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:51.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.561+0000 7fc116a00640 1 -- 172.21.15.32:0/3859508789 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fc10c1213b0 con 0x7fc120155a70 2024-04-17T09:54:51.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.561+0000 7fc11e800640 1 --2- 172.21.15.32:0/3859508789 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc100068380 0x7fc10006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:51.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.561+0000 7fc11e800640 1 --2- 172.21.15.32:0/3859508789 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc100068380 0x7fc10006a840 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc110000f70 tx=0x7fc110007c60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:51.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.565+0000 7fc116a00640 1 -- 172.21.15.32:0/3859508789 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc10c016070 con 0x7fc120155a70 2024-04-17T09:54:51.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.761+0000 7fc125400640 1 -- 172.21.15.32:0/3859508789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc12006cd60 con 0x7fc120155a70 2024-04-17T09:54:51.763 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-17T09:54:51.761+0000 7f93c9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T09:54:51.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.765+0000 7fc116a00640 1 -- 172.21.15.32:0/3859508789 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v16) ==== 161+0+0 (secure 0 0 0) 0x7fc10c0f18d0 con 0x7fc120155a70 2024-04-17T09:54:51.765 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T09:54:51.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.765+0000 7fc125400640 1 -- 172.21.15.32:0/3859508789 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc100068380 msgr2=0x7fc10006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:51.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.765+0000 7fc125400640 1 --2- 172.21.15.32:0/3859508789 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc100068380 0x7fc10006a840 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc110000f70 tx=0x7fc110007c60 comp rx=0 tx=0).stop 2024-04-17T09:54:51.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.765+0000 7fc125400640 1 -- 172.21.15.32:0/3859508789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc120155a70 msgr2=0x7fc1201544c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:51.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.765+0000 7fc125400640 1 --2- 172.21.15.32:0/3859508789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc120155a70 0x7fc1201544c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0ae040 tx=0x7fc10c01acc0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.765+0000 7fc125400640 1 -- 172.21.15.32:0/3859508789 shutdown_connections 2024-04-17T09:54:51.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.765+0000 7fc125400640 1 --2- 172.21.15.32:0/3859508789 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc100068380 0x7fc10006a840 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.765+0000 7fc125400640 1 --2- 172.21.15.32:0/3859508789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc120155e50 0x7fc120154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.765+0000 7fc125400640 1 --2- 172.21.15.32:0/3859508789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc120155a70 0x7fc1201544c0 secure :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0ae040 tx=0x7fc10c01acc0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.765+0000 7fc125400640 1 --2- 172.21.15.32:0/3859508789 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1200b3340 0x7fc120149e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.769+0000 7fc125400640 1 -- 172.21.15.32:0/3859508789 >> 172.21.15.32:0/3859508789 conn(0x7fc12006ba30 msgr2=0x7fc12014f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:51.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.769+0000 7fc125400640 1 -- 172.21.15.32:0/3859508789 shutdown_connections 2024-04-17T09:54:51.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.769+0000 7fc125400640 1 -- 172.21.15.32:0/3859508789 wait complete. 2024-04-17T09:54:51.779 INFO:tasks.ceph.mds.a.smithi032.stderr: -6080> 2024-04-17T09:54:20.321+0000 7fe33da00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T09:54:51.789 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-17T09:54:51.829 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-17T09:54:51.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.933+0000 7f45c2800640 1 Processor -- start 2024-04-17T09:54:51.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.933+0000 7f45c2800640 1 -- start start 2024-04-17T09:54:51.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.933+0000 7f45c2800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45bc0cd5d0 0x7f45bc0cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:51.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.933+0000 7f45c2800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45bc1603e0 con 0x7f45bc0d0e70 2024-04-17T09:54:51.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.933+0000 7f45c2800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45bc1606c0 con 0x7f45bc0b37d0 2024-04-17T09:54:51.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.933+0000 7f45c2800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f45bc160190 con 0x7f45bc0cd5d0 2024-04-17T09:54:51.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.933+0000 7f45bbe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45bc0cd5d0 0x7f45bc0cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:51.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.933+0000 7f45bb400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f45bc0d0e70 0x7f45bc0ccec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33242/0 (socket says 172.21.15.32:33242) 2024-04-17T09:54:51.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.933+0000 7f45bb400640 1 -- 172.21.15.32:0/1249689002 learned_addr learned my addr 172.21.15.32:0/1249689002 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:51.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.933+0000 7f45bbe00640 1 -- 172.21.15.32:0/1249689002 >> v1:172.21.15.144:6789/0 conn(0x7f45bc0b37d0 legacy=0x7f45bc06bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:51.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.933+0000 7f45bbe00640 1 -- 172.21.15.32:0/1249689002 >> v1:172.21.15.32:6789/0 conn(0x7f45bc0d0e70 legacy=0x7f45bc0ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:51.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.933+0000 7f45bbe00640 1 -- 172.21.15.32:0/1249689002 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc1609a0 con 0x7f45bc0cd5d0 2024-04-17T09:54:51.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.933+0000 7f45baa00640 1 -- 172.21.15.32:0/1249689002 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3654139744 0 0) 0x7f45bc1603e0 con 0x7f45bc0d0e70 2024-04-17T09:54:51.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.937+0000 7f45bbe00640 1 --2- 172.21.15.32:0/1249689002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc0cd5d0 0x7f45bc0cf020 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009d90 tx=0x7f45b401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7cae83275b94b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:51.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.937+0000 7f45baa00640 1 -- 172.21.15.32:0/1249689002 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b40b2070 con 0x7f45bc0cd5d0 2024-04-17T09:54:51.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.937+0000 7f45baa00640 1 -- 172.21.15.32:0/1249689002 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b40a5940 con 0x7f45bc0cd5d0 2024-04-17T09:54:51.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.937+0000 7f45baa00640 1 -- 172.21.15.32:0/1249689002 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b40a5d10 con 0x7f45bc0cd5d0 2024-04-17T09:54:51.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.937+0000 7f45c2800640 1 -- 172.21.15.32:0/1249689002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc0cd5d0 msgr2=0x7f45bc0cf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:51.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.937+0000 7f45c2800640 1 --2- 172.21.15.32:0/1249689002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc0cd5d0 0x7f45bc0cf020 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009d90 tx=0x7f45b401c970 comp rx=0 tx=0).stop 2024-04-17T09:54:51.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.937+0000 7f45c2800640 1 -- 172.21.15.32:0/1249689002 shutdown_connections 2024-04-17T09:54:51.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.937+0000 7f45c2800640 1 --2- 172.21.15.32:0/1249689002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc0cd5d0 0x7f45bc0cf020 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.937+0000 7f45c2800640 1 -- 172.21.15.32:0/1249689002 >> 172.21.15.32:0/1249689002 conn(0x7f45bc09a9f0 msgr2=0x7f45bc0b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:51.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.937+0000 7f45c2800640 1 -- 172.21.15.32:0/1249689002 shutdown_connections 2024-04-17T09:54:51.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.937+0000 7f45c2800640 1 -- 172.21.15.32:0/1249689002 wait complete. 2024-04-17T09:54:51.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45c2800640 1 Processor -- start 2024-04-17T09:54:51.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45c2800640 1 -- start start 2024-04-17T09:54:51.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45c2800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45bc0b37d0 0x7f45bc1526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:51.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45c2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc0cd5d0 0x7f45bc153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:51.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45c2800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45bc0d0e70 0x7f45bc154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:51.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45c2800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc16ab00 con 0x7f45bc0cd5d0 2024-04-17T09:54:51.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45c2800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc06c9a0 con 0x7f45bc0b37d0 2024-04-17T09:54:51.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45c2800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f45bc16ac80 con 0x7f45bc0d0e70 2024-04-17T09:54:51.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45b2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc0cd5d0 0x7f45bc153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:51.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45b2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc0cd5d0 0x7f45bc153d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50940/0 (socket says 172.21.15.32:50940) 2024-04-17T09:54:51.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45b2a00640 1 -- 172.21.15.32:0/3148570934 learned_addr learned my addr 172.21.15.32:0/3148570934 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:51.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45bb400640 1 --2- 172.21.15.32:0/3148570934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45bc0b37d0 0x7f45bc1526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:51.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45bbe00640 1 --2- 172.21.15.32:0/3148570934 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45bc0d0e70 0x7f45bc154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:51.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45b2a00640 1 -- 172.21.15.32:0/3148570934 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45bc0d0e70 msgr2=0x7f45bc154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:51.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45b2a00640 1 --2- 172.21.15.32:0/3148570934 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45bc0d0e70 0x7f45bc154290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45b2a00640 1 -- 172.21.15.32:0/3148570934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45bc0b37d0 msgr2=0x7f45bc1526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:51.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45b2a00640 1 --2- 172.21.15.32:0/3148570934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45bc0b37d0 0x7f45bc1526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:51.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45b2a00640 1 -- 172.21.15.32:0/3148570934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc06ef80 con 0x7f45bc0cd5d0 2024-04-17T09:54:51.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45bb400640 1 --2- 172.21.15.32:0/3148570934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45bc0b37d0 0x7f45bc1526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:51.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45b2a00640 1 --2- 172.21.15.32:0/3148570934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc0cd5d0 0x7f45bc153d50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f45a400c500 tx=0x7f45a400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:51.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45b3e00640 1 -- 172.21.15.32:0/3148570934 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a40052d0 con 0x7f45bc0cd5d0 2024-04-17T09:54:51.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45b3e00640 1 -- 172.21.15.32:0/3148570934 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45a4005470 con 0x7f45bc0cd5d0 2024-04-17T09:54:51.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45b3e00640 1 -- 172.21.15.32:0/3148570934 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a4012830 con 0x7f45bc0cd5d0 2024-04-17T09:54:51.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45c2800640 1 -- 172.21.15.32:0/3148570934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45bc1548c0 con 0x7f45bc0cd5d0 2024-04-17T09:54:51.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45c2800640 1 -- 172.21.15.32:0/3148570934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45bc154e80 con 0x7f45bc0cd5d0 2024-04-17T09:54:51.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.941+0000 7f45c2800640 1 -- 172.21.15.32:0/3148570934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45bc155050 con 0x7f45bc0cd5d0 2024-04-17T09:54:51.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.945+0000 7f45b3e00640 1 -- 172.21.15.32:0/3148570934 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f45a40129d0 con 0x7f45bc0cd5d0 2024-04-17T09:54:51.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.945+0000 7f45b3e00640 1 --2- 172.21.15.32:0/3148570934 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4598068380 0x7f459806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:51.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.945+0000 7f45b3e00640 1 -- 172.21.15.32:0/3148570934 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f45a4086560 con 0x7f45bc0cd5d0 2024-04-17T09:54:51.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.945+0000 7f45bb400640 1 --2- 172.21.15.32:0/3148570934 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4598068380 0x7f459806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:51.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.945+0000 7f45bb400640 1 --2- 172.21.15.32:0/3148570934 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4598068380 0x7f459806a840 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f45ac000f70 tx=0x7f45ac007dd0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:51.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:51.949+0000 7f45b3e00640 1 -- 172.21.15.32:0/3148570934 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f45a4056ad0 con 0x7f45bc0cd5d0 2024-04-17T09:54:52.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.149+0000 7f45c2800640 1 -- 172.21.15.32:0/3148570934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f45bc0cb230 con 0x7f45bc0cd5d0 2024-04-17T09:54:52.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.765+0000 7f45b3e00640 1 -- 172.21.15.32:0/3148570934 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f45a40563d0 con 0x7f45bc0cd5d0 2024-04-17T09:54:52.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.769+0000 7f45c2800640 1 -- 172.21.15.32:0/3148570934 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4598068380 msgr2=0x7f459806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:52.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.769+0000 7f45c2800640 1 --2- 172.21.15.32:0/3148570934 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4598068380 0x7f459806a840 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f45ac000f70 tx=0x7f45ac007dd0 comp rx=0 tx=0).stop 2024-04-17T09:54:52.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.769+0000 7f45c2800640 1 -- 172.21.15.32:0/3148570934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc0cd5d0 msgr2=0x7f45bc153d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:52.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.769+0000 7f45c2800640 1 --2- 172.21.15.32:0/3148570934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc0cd5d0 0x7f45bc153d50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f45a400c500 tx=0x7f45a400c9c0 comp rx=0 tx=0).stop 2024-04-17T09:54:52.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.769+0000 7f45c2800640 1 -- 172.21.15.32:0/3148570934 shutdown_connections 2024-04-17T09:54:52.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.769+0000 7f45c2800640 1 --2- 172.21.15.32:0/3148570934 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4598068380 0x7f459806a840 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:52.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.769+0000 7f45c2800640 1 --2- 172.21.15.32:0/3148570934 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45bc0d0e70 0x7f45bc154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:52.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.769+0000 7f45c2800640 1 --2- 172.21.15.32:0/3148570934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45bc0cd5d0 0x7f45bc153d50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:52.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.769+0000 7f45c2800640 1 --2- 172.21.15.32:0/3148570934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45bc0b37d0 0x7f45bc1526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:52.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.769+0000 7f45c2800640 1 -- 172.21.15.32:0/3148570934 >> 172.21.15.32:0/3148570934 conn(0x7f45bc09a9f0 msgr2=0x7f45bc15ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:52.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.769+0000 7f45c2800640 1 -- 172.21.15.32:0/3148570934 shutdown_connections 2024-04-17T09:54:52.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.769+0000 7f45c2800640 1 -- 172.21.15.32:0/3148570934 wait complete. 2024-04-17T09:54:52.795 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-17T09:54:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.985+0000 7fc2d7e00640 1 Processor -- start 2024-04-17T09:54:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.985+0000 7fc2d7e00640 1 -- start start 2024-04-17T09:54:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.985+0000 7fc2d7e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc2d00cfc70 0x7fc2d006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.985+0000 7fc2d7e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2d006f330 0x7fc2d00cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.985+0000 7fc2d7e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2d00cd6a0 0x7fc2d0071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.985+0000 7fc2d7e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc2d015fb30 con 0x7fc2d006f330 2024-04-17T09:54:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.985+0000 7fc2d7e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc2d006f770 con 0x7fc2d00cfc70 2024-04-17T09:54:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.985+0000 7fc2d7e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc2d015f200 con 0x7fc2d00cd6a0 2024-04-17T09:54:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.985+0000 7fc2d4a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2d006f330 0x7fc2d00cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.985+0000 7fc2d4a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2d006f330 0x7fc2d00cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50942/0 (socket says 172.21.15.32:50942) 2024-04-17T09:54:52.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.985+0000 7fc2d4a00640 1 -- 172.21.15.32:0/1145920482 learned_addr learned my addr 172.21.15.32:0/1145920482 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:52.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.985+0000 7fc2d5e00640 1 --2- 172.21.15.32:0/1145920482 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2d00cd6a0 0x7fc2d0071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:52.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.985+0000 7fc2d5400640 1 --2- 172.21.15.32:0/1145920482 >> v2:172.21.15.144:3300/0 conn(0x7fc2d00cfc70 0x7fc2d006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:52.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.985+0000 7fc2d4a00640 1 -- 172.21.15.32:0/1145920482 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2d00cd6a0 msgr2=0x7fc2d0071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:52.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.985+0000 7fc2d4a00640 1 --2- 172.21.15.32:0/1145920482 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2d00cd6a0 0x7fc2d0071db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:52.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.985+0000 7fc2d4a00640 1 -- 172.21.15.32:0/1145920482 >> v2:172.21.15.144:3300/0 conn(0x7fc2d00cfc70 msgr2=0x7fc2d006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:52.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.985+0000 7fc2d4a00640 1 --2- 172.21.15.32:0/1145920482 >> v2:172.21.15.144:3300/0 conn(0x7fc2d00cfc70 0x7fc2d006edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:52.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.985+0000 7fc2d4a00640 1 -- 172.21.15.32:0/1145920482 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d015f4e0 con 0x7fc2d006f330 2024-04-17T09:54:52.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.989+0000 7fc2d4a00640 1 --2- 172.21.15.32:0/1145920482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d006f330 0x7fc2d00cd160 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0070170 tx=0x7fc2bc01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66f30e4222b963a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:52.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.989+0000 7fc2cfe00640 1 -- 172.21.15.32:0/1145920482 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0b0070 con 0x7fc2d006f330 2024-04-17T09:54:52.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.989+0000 7fc2cfe00640 1 -- 172.21.15.32:0/1145920482 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2bc0ab600 con 0x7fc2d006f330 2024-04-17T09:54:52.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.989+0000 7fc2cfe00640 1 -- 172.21.15.32:0/1145920482 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0ab9d0 con 0x7fc2d006f330 2024-04-17T09:54:52.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.989+0000 7fc2d7e00640 1 -- 172.21.15.32:0/1145920482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d006f330 msgr2=0x7fc2d00cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:52.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.989+0000 7fc2d7e00640 1 --2- 172.21.15.32:0/1145920482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d006f330 0x7fc2d00cd160 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0070170 tx=0x7fc2bc01c260 comp rx=0 tx=0).stop 2024-04-17T09:54:52.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.989+0000 7fc2d7e00640 1 -- 172.21.15.32:0/1145920482 shutdown_connections 2024-04-17T09:54:52.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.989+0000 7fc2d7e00640 1 --2- 172.21.15.32:0/1145920482 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2d00cd6a0 0x7fc2d0071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:52.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.989+0000 7fc2d7e00640 1 --2- 172.21.15.32:0/1145920482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d006f330 0x7fc2d00cd160 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:52.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.989+0000 7fc2d7e00640 1 --2- 172.21.15.32:0/1145920482 >> v2:172.21.15.144:3300/0 conn(0x7fc2d00cfc70 0x7fc2d006edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:52.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.989+0000 7fc2d7e00640 1 -- 172.21.15.32:0/1145920482 >> 172.21.15.32:0/1145920482 conn(0x7fc2d00b2190 msgr2=0x7fc2d00b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:52.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.989+0000 7fc2d7e00640 1 -- 172.21.15.32:0/1145920482 shutdown_connections 2024-04-17T09:54:52.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.989+0000 7fc2d7e00640 1 -- 172.21.15.32:0/1145920482 wait complete. 2024-04-17T09:54:52.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2d7e00640 1 Processor -- start 2024-04-17T09:54:52.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2d7e00640 1 -- start start 2024-04-17T09:54:52.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2d7e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2d006f330 0x7fc2d013ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:52.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2d7e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2d00cd6a0 0x7fc2d0158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:52.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2d7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d00cfc70 0x7fc2d0158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:52.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2d7e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d00d01d0 con 0x7fc2d00cfc70 2024-04-17T09:54:52.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2d7e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d00d0050 con 0x7fc2d00cd6a0 2024-04-17T09:54:52.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2d7e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc2d015ba70 con 0x7fc2d006f330 2024-04-17T09:54:52.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2d5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d00cfc70 0x7fc2d0158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:52.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2d5e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d00cfc70 0x7fc2d0158790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50946/0 (socket says 172.21.15.32:50946) 2024-04-17T09:54:52.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2d5e00640 1 -- 172.21.15.32:0/3125543389 learned_addr learned my addr 172.21.15.32:0/3125543389 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:52.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2d4a00640 1 --2- 172.21.15.32:0/3125543389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2d00cd6a0 0x7fc2d0158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:52.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2d5400640 1 --2- 172.21.15.32:0/3125543389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2d006f330 0x7fc2d013ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:52.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2d5e00640 1 -- 172.21.15.32:0/3125543389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2d006f330 msgr2=0x7fc2d013ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:52.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2d5e00640 1 --2- 172.21.15.32:0/3125543389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2d006f330 0x7fc2d013ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:52.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2d5e00640 1 -- 172.21.15.32:0/3125543389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2d00cd6a0 msgr2=0x7fc2d0158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:52.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2d5e00640 1 --2- 172.21.15.32:0/3125543389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2d00cd6a0 0x7fc2d0158250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:52.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2d5e00640 1 -- 172.21.15.32:0/3125543389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d0153f50 con 0x7fc2d00cfc70 2024-04-17T09:54:52.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2d4a00640 1 --2- 172.21.15.32:0/3125543389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2d00cd6a0 0x7fc2d0158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:52.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2d5e00640 1 --2- 172.21.15.32:0/3125543389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d00cfc70 0x7fc2d0158790 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8007e40 tx=0x7fc2c800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:52.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2cd600640 1 -- 172.21.15.32:0/3125543389 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8014620 con 0x7fc2d00cfc70 2024-04-17T09:54:52.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2cd600640 1 -- 172.21.15.32:0/3125543389 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2c8019070 con 0x7fc2d00cfc70 2024-04-17T09:54:52.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2cd600640 1 -- 172.21.15.32:0/3125543389 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8014a20 con 0x7fc2d00cfc70 2024-04-17T09:54:52.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2d7e00640 1 -- 172.21.15.32:0/3125543389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d0154230 con 0x7fc2d00cfc70 2024-04-17T09:54:52.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.993+0000 7fc2d7e00640 1 -- 172.21.15.32:0/3125543389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d01547f0 con 0x7fc2d00cfc70 2024-04-17T09:54:52.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.997+0000 7fc2d7e00640 1 -- 172.21.15.32:0/3125543389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2d0068a90 con 0x7fc2d00cfc70 2024-04-17T09:54:52.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.997+0000 7fc2cd600640 1 -- 172.21.15.32:0/3125543389 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc2c8014bc0 con 0x7fc2d00cfc70 2024-04-17T09:54:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.997+0000 7fc2cd600640 1 --2- 172.21.15.32:0/3125543389 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc2b0068380 0x7fc2b006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.997+0000 7fc2cd600640 1 -- 172.21.15.32:0/3125543389 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fc2c80873b0 con 0x7fc2d00cfc70 2024-04-17T09:54:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.997+0000 7fc2d5400640 1 --2- 172.21.15.32:0/3125543389 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc2b0068380 0x7fc2b006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:52.997+0000 7fc2d5400640 1 --2- 172.21.15.32:0/3125543389 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc2b0068380 0x7fc2b006a840 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc2c400d8d0 tx=0x7fc2c4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:53.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.001+0000 7fc2cd600640 1 -- 172.21.15.32:0/3125543389 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc2c8057870 con 0x7fc2d00cfc70 2024-04-17T09:54:53.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.165+0000 7fc2d7e00640 1 -- 172.21.15.32:0/3125543389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc2d0072e50 con 0x7fc2d00cfc70 2024-04-17T09:54:53.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.773+0000 7fc2cd600640 1 -- 172.21.15.32:0/3125543389 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) ==== 181+0+0 (secure 0 0 0) 0x7fc2c8057170 con 0x7fc2d00cfc70 2024-04-17T09:54:53.774 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-04-17T09:54:53.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.773+0000 7fc2d7e00640 1 -- 172.21.15.32:0/3125543389 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc2b0068380 msgr2=0x7fc2b006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:53.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.773+0000 7fc2d7e00640 1 --2- 172.21.15.32:0/3125543389 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc2b0068380 0x7fc2b006a840 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc2c400d8d0 tx=0x7fc2c4008040 comp rx=0 tx=0).stop 2024-04-17T09:54:53.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.773+0000 7fc2d7e00640 1 -- 172.21.15.32:0/3125543389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d00cfc70 msgr2=0x7fc2d0158790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:53.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.773+0000 7fc2d7e00640 1 --2- 172.21.15.32:0/3125543389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d00cfc70 0x7fc2d0158790 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8007e40 tx=0x7fc2c800e4b0 comp rx=0 tx=0).stop 2024-04-17T09:54:53.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.777+0000 7fc2d7e00640 1 -- 172.21.15.32:0/3125543389 shutdown_connections 2024-04-17T09:54:53.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.777+0000 7fc2d7e00640 1 --2- 172.21.15.32:0/3125543389 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc2b0068380 0x7fc2b006a840 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:53.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.777+0000 7fc2d7e00640 1 --2- 172.21.15.32:0/3125543389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d00cfc70 0x7fc2d0158790 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:53.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.777+0000 7fc2d7e00640 1 --2- 172.21.15.32:0/3125543389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2d00cd6a0 0x7fc2d0158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:53.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.777+0000 7fc2d7e00640 1 --2- 172.21.15.32:0/3125543389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2d006f330 0x7fc2d013ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:53.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.777+0000 7fc2d7e00640 1 -- 172.21.15.32:0/3125543389 >> 172.21.15.32:0/3125543389 conn(0x7fc2d00b2190 msgr2=0x7fc2d0165710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:53.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.777+0000 7fc2d7e00640 1 -- 172.21.15.32:0/3125543389 shutdown_connections 2024-04-17T09:54:53.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.777+0000 7fc2d7e00640 1 -- 172.21.15.32:0/3125543389 wait complete. 2024-04-17T09:54:53.800 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-17T09:54:53.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.945+0000 7f976b400640 1 Processor -- start 2024-04-17T09:54:53.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.945+0000 7f976b400640 1 -- start start 2024-04-17T09:54:53.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.945+0000 7f976b400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f97640cfc70 0x7f976406edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:53.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.945+0000 7f976b400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f97640cd870 0x7f976415ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:53.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.945+0000 7f976b400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f976415f200 con 0x7f976406f330 2024-04-17T09:54:53.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.945+0000 7f976b400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f976415fb30 con 0x7f97640cd870 2024-04-17T09:54:53.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.945+0000 7f976b400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f976406f770 con 0x7f97640cfc70 2024-04-17T09:54:53.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.945+0000 7f975be00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f976406f330 0x7f97640cd160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33250/0 (socket says 172.21.15.32:33250) 2024-04-17T09:54:53.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.945+0000 7f9769400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f97640cd870 0x7f976415ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:53.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.945+0000 7f975be00640 1 -- 172.21.15.32:0/1429041386 learned_addr learned my addr 172.21.15.32:0/1429041386 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:53.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.945+0000 7f9768a00640 1 --2- 172.21.15.32:0/1429041386 >> v2:172.21.15.144:3300/0 conn(0x7f97640cfc70 0x7f976406edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:53.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.945+0000 7f9769400640 1 -- 172.21.15.32:0/1429041386 >> v2:172.21.15.144:3300/0 conn(0x7f97640cfc70 msgr2=0x7f976406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:53.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.945+0000 7f9769400640 1 --2- 172.21.15.32:0/1429041386 >> v2:172.21.15.144:3300/0 conn(0x7f97640cfc70 0x7f976406edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:53.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.945+0000 7f9769400640 1 -- 172.21.15.32:0/1429041386 >> v1:172.21.15.32:6789/0 conn(0x7f976406f330 legacy=0x7f97640cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:53.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.945+0000 7f9769400640 1 -- 172.21.15.32:0/1429041386 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976415f4e0 con 0x7f97640cd870 2024-04-17T09:54:53.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.945+0000 7f9763e00640 1 -- 172.21.15.32:0/1429041386 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1180475956 0 0) 0x7f976415f200 con 0x7f976406f330 2024-04-17T09:54:53.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.949+0000 7f9769400640 1 --2- 172.21.15.32:0/1429041386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97640cd870 0x7f976415ba70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9764070170 tx=0x7f975c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0b69d126764fb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:53.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.949+0000 7f9763e00640 1 -- 172.21.15.32:0/1429041386 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c0a66f0 con 0x7f97640cd870 2024-04-17T09:54:53.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.949+0000 7f9763e00640 1 -- 172.21.15.32:0/1429041386 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f975c0a6890 con 0x7f97640cd870 2024-04-17T09:54:53.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.949+0000 7f9763e00640 1 -- 172.21.15.32:0/1429041386 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c0a6c60 con 0x7f97640cd870 2024-04-17T09:54:53.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.949+0000 7f976b400640 1 -- 172.21.15.32:0/1429041386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97640cd870 msgr2=0x7f976415ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:53.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.949+0000 7f976b400640 1 --2- 172.21.15.32:0/1429041386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97640cd870 0x7f976415ba70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9764070170 tx=0x7f975c01c930 comp rx=0 tx=0).stop 2024-04-17T09:54:53.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.949+0000 7f976b400640 1 -- 172.21.15.32:0/1429041386 shutdown_connections 2024-04-17T09:54:53.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.949+0000 7f976b400640 1 --2- 172.21.15.32:0/1429041386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97640cd870 0x7f976415ba70 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:53.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.949+0000 7f976b400640 1 --2- 172.21.15.32:0/1429041386 >> v2:172.21.15.144:3300/0 conn(0x7f97640cfc70 0x7f976406edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:53.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.949+0000 7f976b400640 1 -- 172.21.15.32:0/1429041386 >> 172.21.15.32:0/1429041386 conn(0x7f97640b2190 msgr2=0x7f97640b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:53.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.949+0000 7f976b400640 1 -- 172.21.15.32:0/1429041386 shutdown_connections 2024-04-17T09:54:53.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.949+0000 7f976b400640 1 -- 172.21.15.32:0/1429041386 wait complete. 2024-04-17T09:54:53.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f976b400640 1 Processor -- start 2024-04-17T09:54:53.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f976b400640 1 -- start start 2024-04-17T09:54:53.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f976b400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f976406f330 0x7f9764158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:53.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f976b400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97640cd870 0x7f9764151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:53.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f976b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97640cfc70 0x7f976413f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:53.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f976b400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f97641681b0 con 0x7f97640cfc70 2024-04-17T09:54:53.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f976b400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f97640d0260 con 0x7f976406f330 2024-04-17T09:54:53.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f976b400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9764168330 con 0x7f97640cd870 2024-04-17T09:54:53.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f9769400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97640cfc70 0x7f976413f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:53.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f9768a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f976406f330 0x7f9764158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:53.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f9769400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97640cfc70 0x7f976413f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50962/0 (socket says 172.21.15.32:50962) 2024-04-17T09:54:53.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f9769400640 1 -- 172.21.15.32:0/2310043142 learned_addr learned my addr 172.21.15.32:0/2310043142 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:53.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f9768a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f976406f330 0x7f9764158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:59174/0 (socket says 172.21.15.32:59174) 2024-04-17T09:54:53.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f975be00640 1 --2- 172.21.15.32:0/2310043142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97640cd870 0x7f9764151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:53.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f9769400640 1 -- 172.21.15.32:0/2310043142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97640cd870 msgr2=0x7f9764151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:53.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f9769400640 1 --2- 172.21.15.32:0/2310043142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97640cd870 0x7f9764151f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:53.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f9769400640 1 -- 172.21.15.32:0/2310043142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f976406f330 msgr2=0x7f9764158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:53.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f9769400640 1 --2- 172.21.15.32:0/2310043142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f976406f330 0x7f9764158630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:53.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f9769400640 1 -- 172.21.15.32:0/2310043142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97641f5710 con 0x7f97640cfc70 2024-04-17T09:54:53.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f9769400640 1 --2- 172.21.15.32:0/2310043142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97640cfc70 0x7f976413f820 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f975c019540 tx=0x7f975c0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:53.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f9761600640 1 -- 172.21.15.32:0/2310043142 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c0b8430 con 0x7f97640cfc70 2024-04-17T09:54:53.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f9761600640 1 -- 172.21.15.32:0/2310043142 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f975c0b85d0 con 0x7f97640cfc70 2024-04-17T09:54:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f9761600640 1 -- 172.21.15.32:0/2310043142 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c0b89c0 con 0x7f97640cfc70 2024-04-17T09:54:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f976b400640 1 -- 172.21.15.32:0/2310043142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9764158b70 con 0x7f97640cfc70 2024-04-17T09:54:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.953+0000 7f976b400640 1 -- 172.21.15.32:0/2310043142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9764152e50 con 0x7f97640cfc70 2024-04-17T09:54:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.957+0000 7f976b400640 1 -- 172.21.15.32:0/2310043142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9764152c40 con 0x7f97640cfc70 2024-04-17T09:54:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.957+0000 7f9761600640 1 -- 172.21.15.32:0/2310043142 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f975c0c3020 con 0x7f97640cfc70 2024-04-17T09:54:53.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.957+0000 7f9761600640 1 --2- 172.21.15.32:0/2310043142 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9740068380 0x7f974006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:53.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.957+0000 7f9761600640 1 -- 172.21.15.32:0/2310043142 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5357+0+0 (secure 0 0 0) 0x7f975c121700 con 0x7f97640cfc70 2024-04-17T09:54:53.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.957+0000 7f9768a00640 1 --2- 172.21.15.32:0/2310043142 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9740068380 0x7f974006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:53.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.957+0000 7f9768a00640 1 --2- 172.21.15.32:0/2310043142 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9740068380 0x7f974006a840 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f97540079c0 tx=0x7f9754008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:53.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:53.961+0000 7f9761600640 1 -- 172.21.15.32:0/2310043142 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f975c016520 con 0x7f97640cfc70 2024-04-17T09:54:54.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.125+0000 7f976b400640 1 -- 172.21.15.32:0/2310043142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9764072e50 con 0x7f97640cfc70 2024-04-17T09:54:54.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.773+0000 7f9761600640 1 -- 172.21.15.32:0/2310043142 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) ==== 169+0+0 (secure 0 0 0) 0x7f975c0f16a0 con 0x7f97640cfc70 2024-04-17T09:54:54.776 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-04-17T09:54:54.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.777+0000 7f976b400640 1 -- 172.21.15.32:0/2310043142 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9740068380 msgr2=0x7f974006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:54.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.777+0000 7f976b400640 1 --2- 172.21.15.32:0/2310043142 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9740068380 0x7f974006a840 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f97540079c0 tx=0x7f9754008040 comp rx=0 tx=0).stop 2024-04-17T09:54:54.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.777+0000 7f976b400640 1 -- 172.21.15.32:0/2310043142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97640cfc70 msgr2=0x7f976413f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.777+0000 7f976b400640 1 --2- 172.21.15.32:0/2310043142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97640cfc70 0x7f976413f820 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f975c019540 tx=0x7f975c0196d0 comp rx=0 tx=0).stop 2024-04-17T09:54:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.777+0000 7f976b400640 1 -- 172.21.15.32:0/2310043142 shutdown_connections 2024-04-17T09:54:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.777+0000 7f976b400640 1 --2- 172.21.15.32:0/2310043142 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9740068380 0x7f974006a840 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.777+0000 7f976b400640 1 --2- 172.21.15.32:0/2310043142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97640cd870 0x7f9764151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.777+0000 7f976b400640 1 --2- 172.21.15.32:0/2310043142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97640cfc70 0x7f976413f820 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.777+0000 7f976b400640 1 --2- 172.21.15.32:0/2310043142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f976406f330 0x7f9764158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.777+0000 7f976b400640 1 -- 172.21.15.32:0/2310043142 >> 172.21.15.32:0/2310043142 conn(0x7f97640b2190 msgr2=0x7f97640b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:54.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.777+0000 7f976b400640 1 -- 172.21.15.32:0/2310043142 shutdown_connections 2024-04-17T09:54:54.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.777+0000 7f976b400640 1 -- 172.21.15.32:0/2310043142 wait complete. 2024-04-17T09:54:54.805 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-17T09:54:54.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.957+0000 7ff795600640 1 Processor -- start 2024-04-17T09:54:54.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.957+0000 7ff795600640 1 -- start start 2024-04-17T09:54:54.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.957+0000 7ff795600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7900cf370 0x7ff7900cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:54.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.957+0000 7ff795600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7900d0130 con 0x7ff790071e60 2024-04-17T09:54:54.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.957+0000 7ff795600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff79015d090 con 0x7ff79006ba30 2024-04-17T09:54:54.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.957+0000 7ff795600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff79015d370 con 0x7ff7900cf370 2024-04-17T09:54:54.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.957+0000 7ff78f400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff790071e60 0x7ff7900722d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33264/0 (socket says 172.21.15.32:33264) 2024-04-17T09:54:54.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.957+0000 7ff78f400640 1 -- 172.21.15.32:0/445145222 learned_addr learned my addr 172.21.15.32:0/445145222 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:54.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.957+0000 7ff786000640 1 --2- 172.21.15.32:0/445145222 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7900cf370 0x7ff7900cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:54.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.957+0000 7ff78e000640 1 -- 172.21.15.32:0/445145222 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4188968059 0 0) 0x7ff7900d0130 con 0x7ff790071e60 2024-04-17T09:54:54.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.957+0000 7ff78e000640 1 -- 172.21.15.32:0/445145222 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff774003530 con 0x7ff790071e60 2024-04-17T09:54:54.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.957+0000 7ff78e000640 1 -- 172.21.15.32:0/445145222 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4182051014 0 0) 0x7ff79015d090 con 0x7ff79006ba30 2024-04-17T09:54:54.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.957+0000 7ff78e000640 1 -- 172.21.15.32:0/445145222 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7900d0130 con 0x7ff79006ba30 2024-04-17T09:54:54.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.957+0000 7ff786000640 1 -- 172.21.15.32:0/445145222 >> v1:172.21.15.144:6789/0 conn(0x7ff79006ba30 legacy=0x7ff79006be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:54.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.957+0000 7ff786000640 1 -- 172.21.15.32:0/445145222 >> v1:172.21.15.32:6789/0 conn(0x7ff790071e60 legacy=0x7ff7900722d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:54.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.957+0000 7ff786000640 1 -- 172.21.15.32:0/445145222 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79015d650 con 0x7ff7900cf370 2024-04-17T09:54:54.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.961+0000 7ff786000640 1 --2- 172.21.15.32:0/445145222 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7900cf370 0x7ff7900cf7e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff7780097f0 tx=0x7ff77801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e9dc40834ef0c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:54.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.961+0000 7ff78e000640 1 -- 172.21.15.32:0/445145222 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7780ab5f0 con 0x7ff7900cf370 2024-04-17T09:54:54.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.961+0000 7ff78e000640 1 -- 172.21.15.32:0/445145222 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7780b0070 con 0x7ff7900cf370 2024-04-17T09:54:54.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.961+0000 7ff78e000640 1 -- 172.21.15.32:0/445145222 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7780ab9b0 con 0x7ff7900cf370 2024-04-17T09:54:54.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.961+0000 7ff795600640 1 -- 172.21.15.32:0/445145222 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7900cf370 msgr2=0x7ff7900cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:54.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.961+0000 7ff795600640 1 --2- 172.21.15.32:0/445145222 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7900cf370 0x7ff7900cf7e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff7780097f0 tx=0x7ff77801c4e0 comp rx=0 tx=0).stop 2024-04-17T09:54:54.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.961+0000 7ff795600640 1 -- 172.21.15.32:0/445145222 shutdown_connections 2024-04-17T09:54:54.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.961+0000 7ff795600640 1 --2- 172.21.15.32:0/445145222 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7900cf370 0x7ff7900cf7e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:54.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.961+0000 7ff795600640 1 -- 172.21.15.32:0/445145222 >> 172.21.15.32:0/445145222 conn(0x7ff7900b2120 msgr2=0x7ff7900b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:54.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.961+0000 7ff795600640 1 -- 172.21.15.32:0/445145222 shutdown_connections 2024-04-17T09:54:54.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.961+0000 7ff795600640 1 -- 172.21.15.32:0/445145222 wait complete. 2024-04-17T09:54:54.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff795600640 1 Processor -- start 2024-04-17T09:54:54.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff795600640 1 -- start start 2024-04-17T09:54:54.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff795600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff79006ba30 0x7ff790159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:54.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff795600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff790071e60 0x7ff7900adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:54.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff795600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7900cf370 0x7ff7900ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:54.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff795600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff7901463b0 con 0x7ff790071e60 2024-04-17T09:54:54.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff795600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff790146230 con 0x7ff7900cf370 2024-04-17T09:54:54.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff795600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff79016af10 con 0x7ff79006ba30 2024-04-17T09:54:54.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff786000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff790071e60 0x7ff7900adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:54.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff786000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff790071e60 0x7ff7900adcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50964/0 (socket says 172.21.15.32:50964) 2024-04-17T09:54:54.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff786000640 1 -- 172.21.15.32:0/294815728 learned_addr learned my addr 172.21.15.32:0/294815728 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:54.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff78ea00640 1 --2- 172.21.15.32:0/294815728 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff79006ba30 0x7ff790159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:54.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff78f400640 1 --2- 172.21.15.32:0/294815728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7900cf370 0x7ff7900ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:54.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff786000640 1 -- 172.21.15.32:0/294815728 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff79006ba30 msgr2=0x7ff790159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:54.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff786000640 1 --2- 172.21.15.32:0/294815728 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff79006ba30 0x7ff790159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:54.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff786000640 1 -- 172.21.15.32:0/294815728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7900cf370 msgr2=0x7ff7900ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:54.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff786000640 1 --2- 172.21.15.32:0/294815728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7900cf370 0x7ff7900ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:54.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff786000640 1 -- 172.21.15.32:0/294815728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7900af910 con 0x7ff790071e60 2024-04-17T09:54:54.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff786000640 1 --2- 172.21.15.32:0/294815728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff790071e60 0x7ff7900adcd0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff7780190f0 tx=0x7ff778019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:54.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff787400640 1 -- 172.21.15.32:0/294815728 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7780b4ae0 con 0x7ff790071e60 2024-04-17T09:54:54.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff787400640 1 -- 172.21.15.32:0/294815728 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7780b4c80 con 0x7ff790071e60 2024-04-17T09:54:54.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff787400640 1 -- 172.21.15.32:0/294815728 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7780bf690 con 0x7ff790071e60 2024-04-17T09:54:54.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff795600640 1 -- 172.21.15.32:0/294815728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7900ae870 con 0x7ff790071e60 2024-04-17T09:54:54.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.965+0000 7ff795600640 1 -- 172.21.15.32:0/294815728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7900aee30 con 0x7ff790071e60 2024-04-17T09:54:54.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.969+0000 7ff795600640 1 -- 172.21.15.32:0/294815728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff790068a90 con 0x7ff790071e60 2024-04-17T09:54:54.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.969+0000 7ff787400640 1 -- 172.21.15.32:0/294815728 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff7780bf830 con 0x7ff790071e60 2024-04-17T09:54:54.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.969+0000 7ff787400640 1 --2- 172.21.15.32:0/294815728 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff76c068380 0x7ff76c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:54.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.969+0000 7ff787400640 1 -- 172.21.15.32:0/294815728 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7ff7781207c0 con 0x7ff790071e60 2024-04-17T09:54:54.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.969+0000 7ff78ea00640 1 --2- 172.21.15.32:0/294815728 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff76c068380 0x7ff76c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:54.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.969+0000 7ff78ea00640 1 --2- 172.21.15.32:0/294815728 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff76c068380 0x7ff76c06a840 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff780001720 tx=0x7ff780009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:54.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:54.973+0000 7ff787400640 1 -- 172.21.15.32:0/294815728 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff7780c6030 con 0x7ff790071e60 2024-04-17T09:54:55.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.137+0000 7ff795600640 1 -- 172.21.15.32:0/294815728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7ff79006c850 con 0x7ff790071e60 2024-04-17T09:54:55.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.141+0000 7ff787400640 1 -- 172.21.15.32:0/294815728 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7ff7780f5e20 con 0x7ff790071e60 2024-04-17T09:54:55.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.145+0000 7ff795600640 1 -- 172.21.15.32:0/294815728 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff76c068380 msgr2=0x7ff76c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:55.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.145+0000 7ff795600640 1 --2- 172.21.15.32:0/294815728 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff76c068380 0x7ff76c06a840 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff780001720 tx=0x7ff780009040 comp rx=0 tx=0).stop 2024-04-17T09:54:55.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.145+0000 7ff795600640 1 -- 172.21.15.32:0/294815728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff790071e60 msgr2=0x7ff7900adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:55.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.145+0000 7ff795600640 1 --2- 172.21.15.32:0/294815728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff790071e60 0x7ff7900adcd0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff7780190f0 tx=0x7ff778019280 comp rx=0 tx=0).stop 2024-04-17T09:54:55.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.145+0000 7ff795600640 1 -- 172.21.15.32:0/294815728 shutdown_connections 2024-04-17T09:54:55.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.145+0000 7ff795600640 1 --2- 172.21.15.32:0/294815728 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff76c068380 0x7ff76c06a840 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:55.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.145+0000 7ff795600640 1 --2- 172.21.15.32:0/294815728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7900cf370 0x7ff7900ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:55.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.145+0000 7ff795600640 1 --2- 172.21.15.32:0/294815728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff790071e60 0x7ff7900adcd0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:55.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.145+0000 7ff795600640 1 --2- 172.21.15.32:0/294815728 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff79006ba30 0x7ff790159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:55.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.145+0000 7ff795600640 1 -- 172.21.15.32:0/294815728 >> 172.21.15.32:0/294815728 conn(0x7ff7900b2120 msgr2=0x7ff79006be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:55.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.145+0000 7ff795600640 1 -- 172.21.15.32:0/294815728 shutdown_connections 2024-04-17T09:54:55.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.145+0000 7ff795600640 1 -- 172.21.15.32:0/294815728 wait complete. 2024-04-17T09:54:55.165 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-17T09:54:55.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.305+0000 7fd478400640 1 Processor -- start 2024-04-17T09:54:55.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.305+0000 7fd478400640 1 -- start start 2024-04-17T09:54:55.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.305+0000 7fd478400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd4701481e0 0x7fd470071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:55.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.305+0000 7fd478400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd470072330 0x7fd4700b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:55.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.305+0000 7fd478400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd47006ef70 con 0x7fd470144f20 2024-04-17T09:54:55.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.305+0000 7fd478400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd4700aed20 con 0x7fd4701481e0 2024-04-17T09:54:55.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.305+0000 7fd478400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd4700af000 con 0x7fd470072330 2024-04-17T09:54:55.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.305+0000 7fd475000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd4701481e0 0x7fd470071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:55.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.305+0000 7fd476400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd470072330 0x7fd4700b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:55.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.305+0000 7fd475000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd4701481e0 0x7fd470071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:59186/0 (socket says 172.21.15.32:59186) 2024-04-17T09:54:55.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.305+0000 7fd475000640 1 -- 172.21.15.32:0/738599239 learned_addr learned my addr 172.21.15.32:0/738599239 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:55.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.305+0000 7fd475000640 1 -- 172.21.15.32:0/738599239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd470072330 msgr2=0x7fd4700b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:55.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.305+0000 7fd475000640 1 --2- 172.21.15.32:0/738599239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd470072330 0x7fd4700b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:55.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.305+0000 7fd475000640 1 -- 172.21.15.32:0/738599239 >> v1:172.21.15.144:6789/0 conn(0x7fd470144f20 legacy=0x7fd47006e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:55.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.305+0000 7fd475000640 1 -- 172.21.15.32:0/738599239 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4700af2e0 con 0x7fd4701481e0 2024-04-17T09:54:55.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.305+0000 7fd46fe00640 1 -- 172.21.15.32:0/738599239 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1759632383 0 0) 0x7fd47006ef70 con 0x7fd470144f20 2024-04-17T09:54:55.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.309+0000 7fd475000640 1 --2- 172.21.15.32:0/738599239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4701481e0 0x7fd470071df0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009870 tx=0x7fd45c01c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fec5ed26ed9e537 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:55.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.309+0000 7fd46fe00640 1 -- 172.21.15.32:0/738599239 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0b2070 con 0x7fd4701481e0 2024-04-17T09:54:55.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.309+0000 7fd46fe00640 1 -- 172.21.15.32:0/738599239 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd45c0a9070 con 0x7fd4701481e0 2024-04-17T09:54:55.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.309+0000 7fd46fe00640 1 -- 172.21.15.32:0/738599239 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0ad880 con 0x7fd4701481e0 2024-04-17T09:54:55.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.309+0000 7fd478400640 1 -- 172.21.15.32:0/738599239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4701481e0 msgr2=0x7fd470071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:55.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.309+0000 7fd478400640 1 --2- 172.21.15.32:0/738599239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4701481e0 0x7fd470071df0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009870 tx=0x7fd45c01c560 comp rx=0 tx=0).stop 2024-04-17T09:54:55.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.309+0000 7fd478400640 1 -- 172.21.15.32:0/738599239 shutdown_connections 2024-04-17T09:54:55.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.309+0000 7fd478400640 1 --2- 172.21.15.32:0/738599239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd470072330 0x7fd4700b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:55.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.309+0000 7fd478400640 1 --2- 172.21.15.32:0/738599239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4701481e0 0x7fd470071df0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:55.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.309+0000 7fd478400640 1 -- 172.21.15.32:0/738599239 >> 172.21.15.32:0/738599239 conn(0x7fd47006ba30 msgr2=0x7fd47006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:55.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.309+0000 7fd478400640 1 -- 172.21.15.32:0/738599239 shutdown_connections 2024-04-17T09:54:55.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.309+0000 7fd478400640 1 -- 172.21.15.32:0/738599239 wait complete. 2024-04-17T09:54:55.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd478400640 1 Processor -- start 2024-04-17T09:54:55.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd478400640 1 -- start start 2024-04-17T09:54:55.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd478400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd470072330 0x7fd470140750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:55.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd478400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd470144f20 0x7fd470139ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:55.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd478400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4701481e0 0x7fd47013a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:55.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd478400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd4700b4360 con 0x7fd470144f20 2024-04-17T09:54:55.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd478400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd4700b41e0 con 0x7fd4701481e0 2024-04-17T09:54:55.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd478400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd47016b5d0 con 0x7fd470072330 2024-04-17T09:54:55.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd475000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd470144f20 0x7fd470139ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:55.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd475000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd470144f20 0x7fd470139ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50968/0 (socket says 172.21.15.32:50968) 2024-04-17T09:54:55.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd475000640 1 -- 172.21.15.32:0/2961652343 learned_addr learned my addr 172.21.15.32:0/2961652343 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd475a00640 1 --2- 172.21.15.32:0/2961652343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd470072330 0x7fd470140750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd476400640 1 --2- 172.21.15.32:0/2961652343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4701481e0 0x7fd47013a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd475000640 1 -- 172.21.15.32:0/2961652343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd470072330 msgr2=0x7fd470140750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd475000640 1 --2- 172.21.15.32:0/2961652343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd470072330 0x7fd470140750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd475000640 1 -- 172.21.15.32:0/2961652343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4701481e0 msgr2=0x7fd47013a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd475000640 1 --2- 172.21.15.32:0/2961652343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4701481e0 0x7fd47013a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd475000640 1 -- 172.21.15.32:0/2961652343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4701402d0 con 0x7fd470144f20 2024-04-17T09:54:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd475000640 1 --2- 172.21.15.32:0/2961652343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd470144f20 0x7fd470139ec0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0b0040 tx=0x7fd45c019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd46d600640 1 -- 172.21.15.32:0/2961652343 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0b2050 con 0x7fd470144f20 2024-04-17T09:54:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd46d600640 1 -- 172.21.15.32:0/2961652343 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd45c0be070 con 0x7fd470144f20 2024-04-17T09:54:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd46d600640 1 -- 172.21.15.32:0/2961652343 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0b96b0 con 0x7fd470144f20 2024-04-17T09:54:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd478400640 1 -- 172.21.15.32:0/2961652343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd470137850 con 0x7fd470144f20 2024-04-17T09:54:55.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.313+0000 7fd478400640 1 -- 172.21.15.32:0/2961652343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd470137e10 con 0x7fd470144f20 2024-04-17T09:54:55.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.317+0000 7fd478400640 1 -- 172.21.15.32:0/2961652343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4701384d0 con 0x7fd470144f20 2024-04-17T09:54:55.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.317+0000 7fd46d600640 1 -- 172.21.15.32:0/2961652343 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd45c0ab070 con 0x7fd470144f20 2024-04-17T09:54:55.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.317+0000 7fd46d600640 1 --2- 172.21.15.32:0/2961652343 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd450068380 0x7fd45006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:55.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.317+0000 7fd46d600640 1 -- 172.21.15.32:0/2961652343 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fd45c1209f0 con 0x7fd470144f20 2024-04-17T09:54:55.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.317+0000 7fd475a00640 1 --2- 172.21.15.32:0/2961652343 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd450068380 0x7fd45006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:55.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.317+0000 7fd475a00640 1 --2- 172.21.15.32:0/2961652343 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd450068380 0x7fd45006a840 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd464000f70 tx=0x7fd464009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:55.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.321+0000 7fd46d600640 1 -- 172.21.15.32:0/2961652343 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd4701384d0 con 0x7fd470144f20 2024-04-17T09:54:55.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.545+0000 7fd478400640 1 -- 172.21.15.32:0/2961652343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7fd470068a90 con 0x7fd470144f20 2024-04-17T09:54:55.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.785+0000 7fd46d600640 1 -- 172.21.15.32:0/2961652343 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v112) ==== 142+0+0 (secure 0 0 0) 0x7fd45c016150 con 0x7fd470144f20 2024-04-17T09:54:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.789+0000 7fd478400640 1 -- 172.21.15.32:0/2961652343 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd450068380 msgr2=0x7fd45006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.789+0000 7fd478400640 1 --2- 172.21.15.32:0/2961652343 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd450068380 0x7fd45006a840 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd464000f70 tx=0x7fd464009040 comp rx=0 tx=0).stop 2024-04-17T09:54:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.789+0000 7fd478400640 1 -- 172.21.15.32:0/2961652343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd470144f20 msgr2=0x7fd470139ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.789+0000 7fd478400640 1 --2- 172.21.15.32:0/2961652343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd470144f20 0x7fd470139ec0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0b0040 tx=0x7fd45c019300 comp rx=0 tx=0).stop 2024-04-17T09:54:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.789+0000 7fd478400640 1 -- 172.21.15.32:0/2961652343 shutdown_connections 2024-04-17T09:54:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.789+0000 7fd478400640 1 --2- 172.21.15.32:0/2961652343 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd450068380 0x7fd45006a840 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.789+0000 7fd478400640 1 --2- 172.21.15.32:0/2961652343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4701481e0 0x7fd47013a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.789+0000 7fd478400640 1 --2- 172.21.15.32:0/2961652343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd470144f20 0x7fd470139ec0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.789+0000 7fd478400640 1 --2- 172.21.15.32:0/2961652343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd470072330 0x7fd470140750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.789+0000 7fd478400640 1 -- 172.21.15.32:0/2961652343 >> 172.21.15.32:0/2961652343 conn(0x7fd47006ba30 msgr2=0x7fd4701425b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.789+0000 7fd478400640 1 -- 172.21.15.32:0/2961652343 shutdown_connections 2024-04-17T09:54:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.789+0000 7fd478400640 1 -- 172.21.15.32:0/2961652343 wait complete. 2024-04-17T09:54:55.809 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-17T09:54:55.809 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-04-17T09:54:55.810 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-17T09:54:55.810 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:54:55.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.949+0000 7fd4e0a00640 1 Processor -- start 2024-04-17T09:54:55.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.949+0000 7fd4e0a00640 1 -- start start 2024-04-17T09:54:55.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.949+0000 7fd4e0a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4dc0af660 0x7fd4dc0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:55.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.953+0000 7fd4e0a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4dc15bf10 0x7fd4dc06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:55.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.953+0000 7fd4e0a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4dc15c6f0 con 0x7fd4dc06f570 2024-04-17T09:54:55.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.953+0000 7fd4e0a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd4dc06ff10 con 0x7fd4dc0af660 2024-04-17T09:54:55.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.953+0000 7fd4e0a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd4dc073a40 con 0x7fd4dc15bf10 2024-04-17T09:54:55.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.953+0000 7fd4da800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4dc0af660 0x7fd4dc0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:55.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.953+0000 7fd4da800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4dc0af660 0x7fd4dc0b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50976/0 (socket says 172.21.15.32:50976) 2024-04-17T09:54:55.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.953+0000 7fd4da800640 1 -- 172.21.15.32:0/2358797635 learned_addr learned my addr 172.21.15.32:0/2358797635 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:55.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.953+0000 7fd4db200640 1 --2- 172.21.15.32:0/2358797635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4dc15bf10 0x7fd4dc06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:55.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.953+0000 7fd4da800640 1 -- 172.21.15.32:0/2358797635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4dc15bf10 msgr2=0x7fd4dc06f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:55.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.953+0000 7fd4da800640 1 --2- 172.21.15.32:0/2358797635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4dc15bf10 0x7fd4dc06f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:55.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.953+0000 7fd4da800640 1 -- 172.21.15.32:0/2358797635 >> v1:172.21.15.144:6789/0 conn(0x7fd4dc06f570 legacy=0x7fd4dc15b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:55.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.953+0000 7fd4da800640 1 -- 172.21.15.32:0/2358797635 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4dc15c410 con 0x7fd4dc0af660 2024-04-17T09:54:55.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.953+0000 7fd4db200640 1 --2- 172.21.15.32:0/2358797635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4dc15bf10 0x7fd4dc06f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:54:55.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.953+0000 7fd4da800640 1 --2- 172.21.15.32:0/2358797635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4dc0af660 0x7fd4dc0b3c90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009870 tx=0x7fd4cc01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494b221022275a67 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:55.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.953+0000 7fd4d9e00640 1 -- 172.21.15.32:0/2358797635 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc0a95f0 con 0x7fd4dc0af660 2024-04-17T09:54:55.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.953+0000 7fd4d9e00640 1 -- 172.21.15.32:0/2358797635 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4cc0a9790 con 0x7fd4dc0af660 2024-04-17T09:54:55.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.953+0000 7fd4d9e00640 1 -- 172.21.15.32:0/2358797635 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc0a9b60 con 0x7fd4dc0af660 2024-04-17T09:54:55.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4e0a00640 1 -- 172.21.15.32:0/2358797635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4dc0af660 msgr2=0x7fd4dc0b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:55.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4e0a00640 1 --2- 172.21.15.32:0/2358797635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4dc0af660 0x7fd4dc0b3c90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009870 tx=0x7fd4cc01c530 comp rx=0 tx=0).stop 2024-04-17T09:54:55.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4e0a00640 1 -- 172.21.15.32:0/2358797635 shutdown_connections 2024-04-17T09:54:55.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4e0a00640 1 --2- 172.21.15.32:0/2358797635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4dc15bf10 0x7fd4dc06f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:55.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4e0a00640 1 --2- 172.21.15.32:0/2358797635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4dc0af660 0x7fd4dc0b3c90 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:55.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4e0a00640 1 -- 172.21.15.32:0/2358797635 >> 172.21.15.32:0/2358797635 conn(0x7fd4dc0d0180 msgr2=0x7fd4dc0d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:55.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4e0a00640 1 -- 172.21.15.32:0/2358797635 shutdown_connections 2024-04-17T09:54:55.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4e0a00640 1 -- 172.21.15.32:0/2358797635 wait complete. 2024-04-17T09:54:55.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4e0a00640 1 Processor -- start 2024-04-17T09:54:55.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4e0a00640 1 -- start start 2024-04-17T09:54:55.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4e0a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4dc06f570 0x7fd4dc15a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:55.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4e0a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4dc0af660 0x7fd4dc150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:55.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4e0a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4dc15bf10 0x7fd4dc150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:55.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4e0a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc0afbc0 con 0x7fd4dc15bf10 2024-04-17T09:54:55.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4e0a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc0afa40 con 0x7fd4dc0af660 2024-04-17T09:54:55.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4e0a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd4dc072430 con 0x7fd4dc06f570 2024-04-17T09:54:55.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4db200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4dc15bf10 0x7fd4dc150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:55.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4d1e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4dc0af660 0x7fd4dc150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:55.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4da800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4dc06f570 0x7fd4dc15a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:55.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4db200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4dc15bf10 0x7fd4dc150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50982/0 (socket says 172.21.15.32:50982) 2024-04-17T09:54:55.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4db200640 1 -- 172.21.15.32:0/973723753 learned_addr learned my addr 172.21.15.32:0/973723753 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:55.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4db200640 1 -- 172.21.15.32:0/973723753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4dc06f570 msgr2=0x7fd4dc15a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:55.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4db200640 1 --2- 172.21.15.32:0/973723753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4dc06f570 0x7fd4dc15a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:55.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4db200640 1 -- 172.21.15.32:0/973723753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4dc0af660 msgr2=0x7fd4dc150710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:55.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4db200640 1 --2- 172.21.15.32:0/973723753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4dc0af660 0x7fd4dc150710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:55.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.957+0000 7fd4db200640 1 -- 172.21.15.32:0/973723753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4dc151320 con 0x7fd4dc15bf10 2024-04-17T09:54:55.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.961+0000 7fd4db200640 1 --2- 172.21.15.32:0/973723753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4dc15bf10 0x7fd4dc150c50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400ccf0 tx=0x7fd4d400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:55.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.961+0000 7fd4d3400640 1 -- 172.21.15.32:0/973723753 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d400f680 con 0x7fd4dc15bf10 2024-04-17T09:54:55.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.961+0000 7fd4d3400640 1 -- 172.21.15.32:0/973723753 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4d400f840 con 0x7fd4dc15bf10 2024-04-17T09:54:55.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.961+0000 7fd4e0a00640 1 -- 172.21.15.32:0/973723753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4dc151600 con 0x7fd4dc15bf10 2024-04-17T09:54:55.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.961+0000 7fd4d3400640 1 -- 172.21.15.32:0/973723753 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d400fc30 con 0x7fd4dc15bf10 2024-04-17T09:54:55.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.961+0000 7fd4e0a00640 1 -- 172.21.15.32:0/973723753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4dc14cd70 con 0x7fd4dc15bf10 2024-04-17T09:54:55.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.961+0000 7fd4e0a00640 1 -- 172.21.15.32:0/973723753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4dc04fd00 con 0x7fd4dc15bf10 2024-04-17T09:54:55.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.961+0000 7fd4d3400640 1 -- 172.21.15.32:0/973723753 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd4d40213f0 con 0x7fd4dc15bf10 2024-04-17T09:54:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.961+0000 7fd4d3400640 1 --2- 172.21.15.32:0/973723753 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd4b8068380 0x7fd4b806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.961+0000 7fd4d3400640 1 -- 172.21.15.32:0/973723753 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fd4d40873b0 con 0x7fd4dc15bf10 2024-04-17T09:54:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.961+0000 7fd4da800640 1 --2- 172.21.15.32:0/973723753 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd4b8068380 0x7fd4b806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:55.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.965+0000 7fd4da800640 1 --2- 172.21.15.32:0/973723753 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd4b8068380 0x7fd4b806a840 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0ae000 tx=0x7fd4cc0b3040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:55.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:55.965+0000 7fd4d3400640 1 -- 172.21.15.32:0/973723753 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd4d4014030 con 0x7fd4dc15bf10 2024-04-17T09:54:56.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.133+0000 7fd4e0a00640 1 -- 172.21.15.32:0/973723753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4dc15eda0 con 0x7fd4dc15bf10 2024-04-17T09:54:56.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.133+0000 7fd4d3400640 1 -- 172.21.15.32:0/973723753 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+13030 (secure 0 0 0) 0x7fd4d4057580 con 0x7fd4dc15bf10 2024-04-17T09:54:56.135 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:56.135 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":38,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:54:54.776016+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6837/925293029":"2024-04-18T09:54:51.765805+0000","172.21.15.32:6835/925293029":"2024-04-18T09:54:51.765805+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:54:56.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.137+0000 7fd4e0a00640 1 -- 172.21.15.32:0/973723753 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd4b8068380 msgr2=0x7fd4b806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:56.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.137+0000 7fd4e0a00640 1 --2- 172.21.15.32:0/973723753 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd4b8068380 0x7fd4b806a840 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0ae000 tx=0x7fd4cc0b3040 comp rx=0 tx=0).stop 2024-04-17T09:54:56.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.137+0000 7fd4e0a00640 1 -- 172.21.15.32:0/973723753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4dc15bf10 msgr2=0x7fd4dc150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:56.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.137+0000 7fd4e0a00640 1 --2- 172.21.15.32:0/973723753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4dc15bf10 0x7fd4dc150c50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400ccf0 tx=0x7fd4d400a650 comp rx=0 tx=0).stop 2024-04-17T09:54:56.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.137+0000 7fd4e0a00640 1 -- 172.21.15.32:0/973723753 shutdown_connections 2024-04-17T09:54:56.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.137+0000 7fd4e0a00640 1 --2- 172.21.15.32:0/973723753 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd4b8068380 0x7fd4b806a840 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:56.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.137+0000 7fd4e0a00640 1 --2- 172.21.15.32:0/973723753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4dc15bf10 0x7fd4dc150c50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:56.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.137+0000 7fd4e0a00640 1 --2- 172.21.15.32:0/973723753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4dc0af660 0x7fd4dc150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:56.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.137+0000 7fd4e0a00640 1 --2- 172.21.15.32:0/973723753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4dc06f570 0x7fd4dc15a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:56.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.137+0000 7fd4e0a00640 1 -- 172.21.15.32:0/973723753 >> 172.21.15.32:0/973723753 conn(0x7fd4dc0d0180 msgr2=0x7fd4dc0d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:56.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.137+0000 7fd4e0a00640 1 -- 172.21.15.32:0/973723753 shutdown_connections 2024-04-17T09:54:56.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.137+0000 7fd4e0a00640 1 -- 172.21.15.32:0/973723753 wait complete. 2024-04-17T09:54:56.160 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:54:56.160 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:54:56.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.301+0000 7f75b7200640 1 Processor -- start 2024-04-17T09:54:56.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.301+0000 7f75b7200640 1 -- start start 2024-04-17T09:54:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.305+0000 7f75b7200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f75b015b010 0x7f75b006dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.305+0000 7f75b7200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75b00b3d80 0x7f75b0071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.305+0000 7f75b7200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75b006e6f0 con 0x7f75b015ac30 2024-04-17T09:54:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.305+0000 7f75b7200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f75b006f300 con 0x7f75b015b010 2024-04-17T09:54:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.305+0000 7f75b7200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f75b006e9d0 con 0x7f75b00b3d80 2024-04-17T09:54:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.305+0000 7f75a7e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f75b015b010 0x7f75b006dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.305+0000 7f75a7e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f75b015b010 0x7f75b006dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50992/0 (socket says 172.21.15.32:50992) 2024-04-17T09:54:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.305+0000 7f75a7e00640 1 -- 172.21.15.32:0/509173787 learned_addr learned my addr 172.21.15.32:0/509173787 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.305+0000 7f75b5200640 1 --2- 172.21.15.32:0/509173787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75b00b3d80 0x7f75b0071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.305+0000 7f75a7e00640 1 -- 172.21.15.32:0/509173787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75b00b3d80 msgr2=0x7f75b0071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.305+0000 7f75a7e00640 1 --2- 172.21.15.32:0/509173787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75b00b3d80 0x7f75b0071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:56.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.305+0000 7f75a7e00640 1 -- 172.21.15.32:0/509173787 >> v1:172.21.15.144:6789/0 conn(0x7f75b015ac30 legacy=0x7f75b006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:56.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.305+0000 7f75a7e00640 1 -- 172.21.15.32:0/509173787 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b006ecb0 con 0x7f75b015b010 2024-04-17T09:54:56.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.305+0000 7f75b5200640 1 --2- 172.21.15.32:0/509173787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75b00b3d80 0x7f75b0071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:56.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.309+0000 7f75a7e00640 1 --2- 172.21.15.32:0/509173787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b015b010 0x7f75b006dea0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f75b006f480 tx=0x7f759801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ac5de35fe1dd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:56.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.309+0000 7f75af400640 1 -- 172.21.15.32:0/509173787 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75980b1070 con 0x7f75b015b010 2024-04-17T09:54:56.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.309+0000 7f75af400640 1 -- 172.21.15.32:0/509173787 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75980ac630 con 0x7f75b015b010 2024-04-17T09:54:56.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.309+0000 7f75af400640 1 -- 172.21.15.32:0/509173787 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75980a8050 con 0x7f75b015b010 2024-04-17T09:54:56.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.309+0000 7f75b7200640 1 -- 172.21.15.32:0/509173787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b015b010 msgr2=0x7f75b006dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:56.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.309+0000 7f75b7200640 1 --2- 172.21.15.32:0/509173787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b015b010 0x7f75b006dea0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f75b006f480 tx=0x7f759801c540 comp rx=0 tx=0).stop 2024-04-17T09:54:56.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.309+0000 7f75b7200640 1 -- 172.21.15.32:0/509173787 shutdown_connections 2024-04-17T09:54:56.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.309+0000 7f75b7200640 1 --2- 172.21.15.32:0/509173787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75b00b3d80 0x7f75b0071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:56.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.309+0000 7f75b7200640 1 --2- 172.21.15.32:0/509173787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b015b010 0x7f75b006dea0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:56.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.309+0000 7f75b7200640 1 -- 172.21.15.32:0/509173787 >> 172.21.15.32:0/509173787 conn(0x7f75b00b2120 msgr2=0x7f75b00b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:56.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.309+0000 7f75b7200640 1 -- 172.21.15.32:0/509173787 shutdown_connections 2024-04-17T09:54:56.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.309+0000 7f75b7200640 1 -- 172.21.15.32:0/509173787 wait complete. 2024-04-17T09:54:56.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.309+0000 7f75b7200640 1 Processor -- start 2024-04-17T09:54:56.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.309+0000 7f75b7200640 1 -- start start 2024-04-17T09:54:56.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75b7200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75b00b3d80 0x7f75b0149d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:56.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75b7200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b015ac30 0x7f75b00ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:56.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75b7200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75b015b010 0x7f75b00ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:56.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75b7200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f75b00cddd0 con 0x7f75b015ac30 2024-04-17T09:54:56.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75b7200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f75b00cdc50 con 0x7f75b00b3d80 2024-04-17T09:54:56.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75b7200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f75b00cdf50 con 0x7f75b015b010 2024-04-17T09:54:56.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75a7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b015ac30 0x7f75b00ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:56.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75a7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b015ac30 0x7f75b00ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51006/0 (socket says 172.21.15.32:51006) 2024-04-17T09:54:56.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75a7e00640 1 -- 172.21.15.32:0/2389256499 learned_addr learned my addr 172.21.15.32:0/2389256499 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:56.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75b5200640 1 --2- 172.21.15.32:0/2389256499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75b015b010 0x7f75b00ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:56.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75afe00640 1 --2- 172.21.15.32:0/2389256499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75b00b3d80 0x7f75b0149d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:56.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75a7e00640 1 -- 172.21.15.32:0/2389256499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75b015b010 msgr2=0x7f75b00ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:56.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75a7e00640 1 --2- 172.21.15.32:0/2389256499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75b015b010 0x7f75b00ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:56.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75a7e00640 1 -- 172.21.15.32:0/2389256499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75b00b3d80 msgr2=0x7f75b0149d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:56.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75a7e00640 1 --2- 172.21.15.32:0/2389256499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75b00b3d80 0x7f75b0149d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:56.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75a7e00640 1 -- 172.21.15.32:0/2389256499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b00c6b60 con 0x7f75b015ac30 2024-04-17T09:54:56.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75b5200640 1 --2- 172.21.15.32:0/2389256499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75b015b010 0x7f75b00ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:54:56.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75afe00640 1 --2- 172.21.15.32:0/2389256499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75b00b3d80 0x7f75b0149d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:54:56.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75a7e00640 1 --2- 172.21.15.32:0/2389256499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b015ac30 0x7f75b00ce3b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f759801a170 tx=0x7f75980192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:56.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75acc00640 1 -- 172.21.15.32:0/2389256499 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75980bc070 con 0x7f75b015ac30 2024-04-17T09:54:56.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75acc00640 1 -- 172.21.15.32:0/2389256499 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75980ab5e0 con 0x7f75b015ac30 2024-04-17T09:54:56.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75acc00640 1 -- 172.21.15.32:0/2389256499 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75980b1070 con 0x7f75b015ac30 2024-04-17T09:54:56.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75b7200640 1 -- 172.21.15.32:0/2389256499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b00c5fe0 con 0x7f75b015ac30 2024-04-17T09:54:56.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75b7200640 1 -- 172.21.15.32:0/2389256499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b00c65a0 con 0x7f75b015ac30 2024-04-17T09:54:56.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75b7200640 1 -- 172.21.15.32:0/2389256499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75b0068a90 con 0x7f75b015ac30 2024-04-17T09:54:56.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.313+0000 7f75acc00640 1 -- 172.21.15.32:0/2389256499 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f75980bf070 con 0x7f75b015ac30 2024-04-17T09:54:56.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.317+0000 7f75acc00640 1 --2- 172.21.15.32:0/2389256499 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f758c068380 0x7f758c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:56.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.317+0000 7f75acc00640 1 -- 172.21.15.32:0/2389256499 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f759811ff50 con 0x7f75b015ac30 2024-04-17T09:54:56.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.317+0000 7f75afe00640 1 --2- 172.21.15.32:0/2389256499 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f758c068380 0x7f758c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:56.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.317+0000 7f75afe00640 1 --2- 172.21.15.32:0/2389256499 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f758c068380 0x7f758c06a840 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f75a00016f0 tx=0x7f75a0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:56.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.321+0000 7f75acc00640 1 -- 172.21.15.32:0/2389256499 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f7598016130 con 0x7f75b015ac30 2024-04-17T09:54:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.485+0000 7f75b7200640 1 -- 172.21.15.32:0/2389256499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f75b0072a60 con 0x7f75b015ac30 2024-04-17T09:54:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.485+0000 7f75acc00640 1 -- 172.21.15.32:0/2389256499 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7f75980f0800 con 0x7f75b015ac30 2024-04-17T09:54:56.487 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:54:56.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.489+0000 7f75b7200640 1 -- 172.21.15.32:0/2389256499 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f758c068380 msgr2=0x7f758c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:56.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.489+0000 7f75b7200640 1 --2- 172.21.15.32:0/2389256499 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f758c068380 0x7f758c06a840 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f75a00016f0 tx=0x7f75a0009040 comp rx=0 tx=0).stop 2024-04-17T09:54:56.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.489+0000 7f75b7200640 1 -- 172.21.15.32:0/2389256499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b015ac30 msgr2=0x7f75b00ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:56.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.489+0000 7f75b7200640 1 --2- 172.21.15.32:0/2389256499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b015ac30 0x7f75b00ce3b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f759801a170 tx=0x7f75980192e0 comp rx=0 tx=0).stop 2024-04-17T09:54:56.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.489+0000 7f75b7200640 1 -- 172.21.15.32:0/2389256499 shutdown_connections 2024-04-17T09:54:56.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.489+0000 7f75b7200640 1 --2- 172.21.15.32:0/2389256499 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f758c068380 0x7f758c06a840 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:56.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.489+0000 7f75b7200640 1 --2- 172.21.15.32:0/2389256499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75b015b010 0x7f75b00ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:56.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.489+0000 7f75b7200640 1 --2- 172.21.15.32:0/2389256499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b015ac30 0x7f75b00ce3b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:56.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.489+0000 7f75b7200640 1 --2- 172.21.15.32:0/2389256499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75b00b3d80 0x7f75b0149d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:56.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.489+0000 7f75b7200640 1 -- 172.21.15.32:0/2389256499 >> 172.21.15.32:0/2389256499 conn(0x7f75b00b2120 msgr2=0x7f75b015d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:56.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.489+0000 7f75b7200640 1 -- 172.21.15.32:0/2389256499 shutdown_connections 2024-04-17T09:54:56.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.489+0000 7f75b7200640 1 -- 172.21.15.32:0/2389256499 wait complete. 2024-04-17T09:54:56.508 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-17T09:54:56.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.649+0000 7f3ded800640 1 Processor -- start 2024-04-17T09:54:56.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.649+0000 7f3ded800640 1 -- start start 2024-04-17T09:54:56.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.649+0000 7f3ded800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3de815e870 0x7f3de8162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:56.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.649+0000 7f3ded800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3de806f7c0 0x7f3de806fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:56.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.649+0000 7f3ded800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3de8163340 con 0x7f3de80b0320 2024-04-17T09:54:56.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.649+0000 7f3ded800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3de80cf810 con 0x7f3de806f7c0 2024-04-17T09:54:56.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.649+0000 7f3ded800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3de8067dd0 con 0x7f3de815e870 2024-04-17T09:54:56.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.649+0000 7f3dde000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3de80b0320 0x7f3de80d0240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33274/0 (socket says 172.21.15.32:33274) 2024-04-17T09:54:56.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.649+0000 7f3de7400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3de806f7c0 0x7f3de806fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:56.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.649+0000 7f3dde000640 1 -- 172.21.15.32:0/228270750 learned_addr learned my addr 172.21.15.32:0/228270750 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:56.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.649+0000 7f3de6a00640 1 --2- 172.21.15.32:0/228270750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3de815e870 0x7f3de8162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:56.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.649+0000 7f3de7400640 1 -- 172.21.15.32:0/228270750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3de815e870 msgr2=0x7f3de8162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:56.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.649+0000 7f3de7400640 1 --2- 172.21.15.32:0/228270750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3de815e870 0x7f3de8162e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:56.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.649+0000 7f3de7400640 1 -- 172.21.15.32:0/228270750 >> v1:172.21.15.32:6789/0 conn(0x7f3de80b0320 legacy=0x7f3de80d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:56.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.649+0000 7f3de6000640 1 -- 172.21.15.32:0/228270750 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 975467492 0 0) 0x7f3de8163340 con 0x7f3de80b0320 2024-04-17T09:54:56.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.649+0000 7f3de7400640 1 -- 172.21.15.32:0/228270750 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de815ec50 con 0x7f3de806f7c0 2024-04-17T09:54:56.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.653+0000 7f3de7400640 1 --2- 172.21.15.32:0/228270750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3de806f7c0 0x7f3de806fc50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009d90 tx=0x7f3de001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fa8adca57c4697 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:56.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.653+0000 7f3de6000640 1 -- 172.21.15.32:0/228270750 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de00b2070 con 0x7f3de806f7c0 2024-04-17T09:54:56.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.653+0000 7f3de6000640 1 -- 172.21.15.32:0/228270750 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3de00a5830 con 0x7f3de806f7c0 2024-04-17T09:54:56.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.653+0000 7f3de6000640 1 -- 172.21.15.32:0/228270750 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de00a5c00 con 0x7f3de806f7c0 2024-04-17T09:54:56.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.653+0000 7f3ded800640 1 -- 172.21.15.32:0/228270750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3de806f7c0 msgr2=0x7f3de806fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:56.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.653+0000 7f3ded800640 1 --2- 172.21.15.32:0/228270750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3de806f7c0 0x7f3de806fc50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009d90 tx=0x7f3de001c970 comp rx=0 tx=0).stop 2024-04-17T09:54:56.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.653+0000 7f3ded800640 1 -- 172.21.15.32:0/228270750 shutdown_connections 2024-04-17T09:54:56.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.653+0000 7f3ded800640 1 --2- 172.21.15.32:0/228270750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3de806f7c0 0x7f3de806fc50 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:56.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.653+0000 7f3ded800640 1 --2- 172.21.15.32:0/228270750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3de815e870 0x7f3de8162e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:56.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.653+0000 7f3ded800640 1 -- 172.21.15.32:0/228270750 >> 172.21.15.32:0/228270750 conn(0x7f3de80d0c00 msgr2=0x7f3de80d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:56.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.653+0000 7f3ded800640 1 -- 172.21.15.32:0/228270750 shutdown_connections 2024-04-17T09:54:56.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.653+0000 7f3ded800640 1 -- 172.21.15.32:0/228270750 wait complete. 2024-04-17T09:54:56.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3ded800640 1 Processor -- start 2024-04-17T09:54:56.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3ded800640 1 -- start start 2024-04-17T09:54:56.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3ded800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3de806f7c0 0x7f3de80af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:56.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3ded800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3de80b0320 0x7f3de8159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:56.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3ded800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3de815e870 0x7f3de8160f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:56.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3ded800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3de816b790 con 0x7f3de80b0320 2024-04-17T09:54:56.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3ded800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3de80cfa40 con 0x7f3de815e870 2024-04-17T09:54:56.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3ded800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3de816b910 con 0x7f3de806f7c0 2024-04-17T09:54:56.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3dde000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3de80b0320 0x7f3de8159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:56.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3dde000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3de80b0320 0x7f3de8159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51018/0 (socket says 172.21.15.32:51018) 2024-04-17T09:54:56.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3dde000640 1 -- 172.21.15.32:0/2196827180 learned_addr learned my addr 172.21.15.32:0/2196827180 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:56.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3de6a00640 1 --2- 172.21.15.32:0/2196827180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3de806f7c0 0x7f3de80af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:56.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3de7400640 1 --2- 172.21.15.32:0/2196827180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3de815e870 0x7f3de8160f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3dde000640 1 -- 172.21.15.32:0/2196827180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3de806f7c0 msgr2=0x7f3de80af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3dde000640 1 --2- 172.21.15.32:0/2196827180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3de806f7c0 0x7f3de80af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3dde000640 1 -- 172.21.15.32:0/2196827180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3de815e870 msgr2=0x7f3de8160f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3dde000640 1 --2- 172.21.15.32:0/2196827180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3de815e870 0x7f3de8160f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3dde000640 1 -- 172.21.15.32:0/2196827180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de8157a80 con 0x7f3de80b0320 2024-04-17T09:54:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3dde000640 1 --2- 172.21.15.32:0/2196827180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3de80b0320 0x7f3de8159fb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3dd000c520 tx=0x7f3dd000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3ddf400640 1 -- 172.21.15.32:0/2196827180 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0018070 con 0x7f3de80b0320 2024-04-17T09:54:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3ddf400640 1 -- 172.21.15.32:0/2196827180 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dd00074e0 con 0x7f3de80b0320 2024-04-17T09:54:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3ded800640 1 -- 172.21.15.32:0/2196827180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de8157d60 con 0x7f3de80b0320 2024-04-17T09:54:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3ddf400640 1 -- 172.21.15.32:0/2196827180 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd00049f0 con 0x7f3de80b0320 2024-04-17T09:54:56.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3ded800640 1 -- 172.21.15.32:0/2196827180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de80aeac0 con 0x7f3de80b0320 2024-04-17T09:54:56.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.657+0000 7f3ded800640 1 -- 172.21.15.32:0/2196827180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3de8068a90 con 0x7f3de80b0320 2024-04-17T09:54:56.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.661+0000 7f3ddf400640 1 -- 172.21.15.32:0/2196827180 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3dd001b070 con 0x7f3de80b0320 2024-04-17T09:54:56.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.661+0000 7f3ddf400640 1 --2- 172.21.15.32:0/2196827180 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3dc4068380 0x7f3dc406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:56.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.661+0000 7f3ddf400640 1 -- 172.21.15.32:0/2196827180 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f3dd0085ae0 con 0x7f3de80b0320 2024-04-17T09:54:56.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.661+0000 7f3de6a00640 1 --2- 172.21.15.32:0/2196827180 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3dc4068380 0x7f3dc406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:56.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.661+0000 7f3de6a00640 1 --2- 172.21.15.32:0/2196827180 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3dc4068380 0x7f3dc406a840 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009ac0 tx=0x7f3dd8009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:56.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.665+0000 7f3ddf400640 1 -- 172.21.15.32:0/2196827180 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3dd0055cb0 con 0x7f3de80b0320 2024-04-17T09:54:56.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:56.889+0000 7f3ded800640 1 -- 172.21.15.32:0/2196827180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7f3de806cf70 con 0x7f3de80b0320 2024-04-17T09:54:57.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.793+0000 7f3ddf400640 1 -- 172.21.15.32:0/2196827180 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v114) ==== 152+0+0 (secure 0 0 0) 0x7f3dd0059fb0 con 0x7f3de80b0320 2024-04-17T09:54:57.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.797+0000 7f3ded800640 1 -- 172.21.15.32:0/2196827180 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3dc4068380 msgr2=0x7f3dc406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:57.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.797+0000 7f3ded800640 1 --2- 172.21.15.32:0/2196827180 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3dc4068380 0x7f3dc406a840 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009ac0 tx=0x7f3dd8009190 comp rx=0 tx=0).stop 2024-04-17T09:54:57.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.797+0000 7f3ded800640 1 -- 172.21.15.32:0/2196827180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3de80b0320 msgr2=0x7f3de8159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:57.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.797+0000 7f3ded800640 1 --2- 172.21.15.32:0/2196827180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3de80b0320 0x7f3de8159fb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3dd000c520 tx=0x7f3dd000c9e0 comp rx=0 tx=0).stop 2024-04-17T09:54:57.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.797+0000 7f3ded800640 1 -- 172.21.15.32:0/2196827180 shutdown_connections 2024-04-17T09:54:57.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.797+0000 7f3ded800640 1 --2- 172.21.15.32:0/2196827180 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3dc4068380 0x7f3dc406a840 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:57.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.797+0000 7f3ded800640 1 --2- 172.21.15.32:0/2196827180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3de815e870 0x7f3de8160f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:57.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.797+0000 7f3ded800640 1 --2- 172.21.15.32:0/2196827180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3de80b0320 0x7f3de8159fb0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:57.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.797+0000 7f3ded800640 1 --2- 172.21.15.32:0/2196827180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3de806f7c0 0x7f3de80af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:57.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.797+0000 7f3ded800640 1 -- 172.21.15.32:0/2196827180 >> 172.21.15.32:0/2196827180 conn(0x7f3de80d0c00 msgr2=0x7f3de815af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:57.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.797+0000 7f3ded800640 1 -- 172.21.15.32:0/2196827180 shutdown_connections 2024-04-17T09:54:57.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.797+0000 7f3ded800640 1 -- 172.21.15.32:0/2196827180 wait complete. 2024-04-17T09:54:57.821 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-17T09:54:57.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.969+0000 7f8642c00640 1 Processor -- start 2024-04-17T09:54:57.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.969+0000 7f8642c00640 1 -- start start 2024-04-17T09:54:57.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.969+0000 7f8642c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f863c0cf370 0x7f863c0cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:57.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.969+0000 7f8642c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f863c071e60 0x7f863c072270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:57.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.969+0000 7f8642c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f863c0d0130 con 0x7f863c06ba30 2024-04-17T09:54:57.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.969+0000 7f8642c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f863c15d090 con 0x7f863c0cf370 2024-04-17T09:54:57.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.969+0000 7f8642c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f863c15d370 con 0x7f863c071e60 2024-04-17T09:54:57.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.969+0000 7f863be00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f863c06ba30 0x7f863c06be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33284/0 (socket says 172.21.15.32:33284) 2024-04-17T09:54:57.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.969+0000 7f863be00640 1 -- 172.21.15.32:0/1507107567 learned_addr learned my addr 172.21.15.32:0/1507107567 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:57.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.969+0000 7f8633e00640 1 --2- 172.21.15.32:0/1507107567 >> v2:172.21.15.32:3300/0 conn(0x7f863c0cf370 0x7f863c0cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:57.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.969+0000 7f8640c00640 1 --2- 172.21.15.32:0/1507107567 >> v2:172.21.15.144:3300/0 conn(0x7f863c071e60 0x7f863c072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:57.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.969+0000 7f863b400640 1 -- 172.21.15.32:0/1507107567 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3208626022 0 0) 0x7f863c0d0130 con 0x7f863c06ba30 2024-04-17T09:54:57.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.969+0000 7f8633e00640 1 -- 172.21.15.32:0/1507107567 >> v2:172.21.15.144:3300/0 conn(0x7f863c071e60 msgr2=0x7f863c072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:57.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.969+0000 7f8633e00640 1 --2- 172.21.15.32:0/1507107567 >> v2:172.21.15.144:3300/0 conn(0x7f863c071e60 0x7f863c072270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:57.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.969+0000 7f8633e00640 1 -- 172.21.15.32:0/1507107567 >> v1:172.21.15.32:6789/0 conn(0x7f863c06ba30 legacy=0x7f863c06be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:57.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.969+0000 7f8633e00640 1 -- 172.21.15.32:0/1507107567 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863c15d650 con 0x7f863c0cf370 2024-04-17T09:54:57.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.969+0000 7f8640c00640 1 --2- 172.21.15.32:0/1507107567 >> v2:172.21.15.144:3300/0 conn(0x7f863c071e60 0x7f863c072270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:54:57.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.973+0000 7f8633e00640 1 --2- 172.21.15.32:0/1507107567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f863c0cf370 0x7f863c0cf7e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f863c15a950 tx=0x7f862401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=782370a7dce64c0a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:57.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.973+0000 7f863b400640 1 -- 172.21.15.32:0/1507107567 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86240aa600 con 0x7f863c0cf370 2024-04-17T09:54:57.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.973+0000 7f863b400640 1 -- 172.21.15.32:0/1507107567 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86240af070 con 0x7f863c0cf370 2024-04-17T09:54:57.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.973+0000 7f863b400640 1 -- 172.21.15.32:0/1507107567 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86240aa9c0 con 0x7f863c0cf370 2024-04-17T09:54:57.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.973+0000 7f8642c00640 1 -- 172.21.15.32:0/1507107567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f863c0cf370 msgr2=0x7f863c0cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:57.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.973+0000 7f8642c00640 1 --2- 172.21.15.32:0/1507107567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f863c0cf370 0x7f863c0cf7e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f863c15a950 tx=0x7f862401c480 comp rx=0 tx=0).stop 2024-04-17T09:54:57.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.973+0000 7f8642c00640 1 -- 172.21.15.32:0/1507107567 shutdown_connections 2024-04-17T09:54:57.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.973+0000 7f8642c00640 1 --2- 172.21.15.32:0/1507107567 >> v2:172.21.15.144:3300/0 conn(0x7f863c071e60 0x7f863c072270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:57.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.973+0000 7f8642c00640 1 --2- 172.21.15.32:0/1507107567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f863c0cf370 0x7f863c0cf7e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:57.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.973+0000 7f8642c00640 1 -- 172.21.15.32:0/1507107567 >> 172.21.15.32:0/1507107567 conn(0x7f863c0b2120 msgr2=0x7f863c0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:57.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.973+0000 7f8642c00640 1 -- 172.21.15.32:0/1507107567 shutdown_connections 2024-04-17T09:54:57.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.973+0000 7f8642c00640 1 -- 172.21.15.32:0/1507107567 wait complete. 2024-04-17T09:54:57.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8642c00640 1 Processor -- start 2024-04-17T09:54:57.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8642c00640 1 -- start start 2024-04-17T09:54:57.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8642c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f863c06ba30 0x7f863c0a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:57.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8642c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f863c071e60 0x7f863c159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:57.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8642c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f863c0cf370 0x7f863c13e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:57.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8642c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f863c146230 con 0x7f863c0cf370 2024-04-17T09:54:57.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8642c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f863c15dd80 con 0x7f863c06ba30 2024-04-17T09:54:57.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8642c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f863c1463b0 con 0x7f863c071e60 2024-04-17T09:54:57.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8640c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f863c0cf370 0x7f863c13e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:57.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8640c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f863c0cf370 0x7f863c13e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51044/0 (socket says 172.21.15.32:51044) 2024-04-17T09:54:57.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8640c00640 1 -- 172.21.15.32:0/2650847347 learned_addr learned my addr 172.21.15.32:0/2650847347 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:57.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8633e00640 1 --2- 172.21.15.32:0/2650847347 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f863c071e60 0x7f863c159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:57.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f863be00640 1 --2- 172.21.15.32:0/2650847347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f863c06ba30 0x7f863c0a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:57.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8640c00640 1 -- 172.21.15.32:0/2650847347 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f863c071e60 msgr2=0x7f863c159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:57.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8640c00640 1 --2- 172.21.15.32:0/2650847347 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f863c071e60 0x7f863c159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:57.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8640c00640 1 -- 172.21.15.32:0/2650847347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f863c06ba30 msgr2=0x7f863c0a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:57.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8640c00640 1 --2- 172.21.15.32:0/2650847347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f863c06ba30 0x7f863c0a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:57.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8640c00640 1 -- 172.21.15.32:0/2650847347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863c0a9990 con 0x7f863c0cf370 2024-04-17T09:54:57.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f863be00640 1 --2- 172.21.15.32:0/2650847347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f863c06ba30 0x7f863c0a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:54:57.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8640c00640 1 --2- 172.21.15.32:0/2650847347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f863c0cf370 0x7f863c13e7f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f863400cef0 tx=0x7f863400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:57.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8638c00640 1 -- 172.21.15.32:0/2650847347 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863401a070 con 0x7f863c0cf370 2024-04-17T09:54:57.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8638c00640 1 -- 172.21.15.32:0/2650847347 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f863400d860 con 0x7f863c0cf370 2024-04-17T09:54:57.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8638c00640 1 -- 172.21.15.32:0/2650847347 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863400dc50 con 0x7f863c0cf370 2024-04-17T09:54:57.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8642c00640 1 -- 172.21.15.32:0/2650847347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f863c0aa020 con 0x7f863c0cf370 2024-04-17T09:54:57.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8642c00640 1 -- 172.21.15.32:0/2650847347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f863c15a240 con 0x7f863c0cf370 2024-04-17T09:54:57.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.977+0000 7f8642c00640 1 -- 172.21.15.32:0/2650847347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f863c068a90 con 0x7f863c0cf370 2024-04-17T09:54:57.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.981+0000 7f8638c00640 1 -- 172.21.15.32:0/2650847347 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f863401d070 con 0x7f863c0cf370 2024-04-17T09:54:57.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.981+0000 7f8638c00640 1 --2- 172.21.15.32:0/2650847347 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8618068380 0x7f861806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:57.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.981+0000 7f8638c00640 1 -- 172.21.15.32:0/2650847347 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f8634086f20 con 0x7f863c0cf370 2024-04-17T09:54:57.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.981+0000 7f863be00640 1 --2- 172.21.15.32:0/2650847347 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8618068380 0x7f861806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:57.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.981+0000 7f863be00640 1 --2- 172.21.15.32:0/2650847347 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8618068380 0x7f861806a840 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f862c000f70 tx=0x7f862c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:57.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:57.985+0000 7f8638c00640 1 -- 172.21.15.32:0/2650847347 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f8634057890 con 0x7f863c0cf370 2024-04-17T09:54:58.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.165+0000 7f8642c00640 1 -- 172.21.15.32:0/2650847347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f863c0cc5d0 con 0x7f863c0cf370 2024-04-17T09:54:58.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.169+0000 7f8638c00640 1 -- 172.21.15.32:0/2650847347 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f8634057190 con 0x7f863c0cf370 2024-04-17T09:54:58.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.169+0000 7f8642c00640 1 -- 172.21.15.32:0/2650847347 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8618068380 msgr2=0x7f861806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:58.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.169+0000 7f8642c00640 1 --2- 172.21.15.32:0/2650847347 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8618068380 0x7f861806a840 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f862c000f70 tx=0x7f862c009040 comp rx=0 tx=0).stop 2024-04-17T09:54:58.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.169+0000 7f8642c00640 1 -- 172.21.15.32:0/2650847347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f863c0cf370 msgr2=0x7f863c13e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:58.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.169+0000 7f8642c00640 1 --2- 172.21.15.32:0/2650847347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f863c0cf370 0x7f863c13e7f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f863400cef0 tx=0x7f863400a710 comp rx=0 tx=0).stop 2024-04-17T09:54:58.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.169+0000 7f8642c00640 1 -- 172.21.15.32:0/2650847347 shutdown_connections 2024-04-17T09:54:58.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.169+0000 7f8642c00640 1 --2- 172.21.15.32:0/2650847347 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8618068380 0x7f861806a840 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:58.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.169+0000 7f8642c00640 1 --2- 172.21.15.32:0/2650847347 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f863c071e60 0x7f863c159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:58.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.169+0000 7f8642c00640 1 --2- 172.21.15.32:0/2650847347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f863c0cf370 0x7f863c13e7f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:58.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.169+0000 7f8642c00640 1 --2- 172.21.15.32:0/2650847347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f863c06ba30 0x7f863c0a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:58.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.169+0000 7f8642c00640 1 -- 172.21.15.32:0/2650847347 >> 172.21.15.32:0/2650847347 conn(0x7f863c0b2120 msgr2=0x7f863c0b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:58.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.169+0000 7f8642c00640 1 -- 172.21.15.32:0/2650847347 shutdown_connections 2024-04-17T09:54:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.169+0000 7f8642c00640 1 -- 172.21.15.32:0/2650847347 wait complete. 2024-04-17T09:54:58.191 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T09:54:58.191 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T09:54:58.295 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T09:54:58.386 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T09:54:58.386 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:54:58.386 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:54:58.395 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T09:54:58.395 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:54:58.395 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:54:58.405 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T09:54:58.405 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:54:58.405 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:54:58.448 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T09:54:58.448 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:54:58.448 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:54:58.457 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T09:54:58.457 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-17T09:54:58.457 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:54:58.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.629+0000 7fc690c00640 1 Processor -- start 2024-04-17T09:54:58.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.629+0000 7fc690c00640 1 -- start start 2024-04-17T09:54:58.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.633+0000 7fc690c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc68c151db0 0x7fc68c0b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:58.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.633+0000 7fc690c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc68c06f630 0x7fc68c072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:58.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.633+0000 7fc690c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc68c072bc0 0x7fc68c15e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:58.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.633+0000 7fc690c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc68c0b44f0 con 0x7fc68c151db0 2024-04-17T09:54:58.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.633+0000 7fc690c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc68c0b3490 con 0x7fc68c06f630 2024-04-17T09:54:58.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.633+0000 7fc690c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc68c0cea40 con 0x7fc68c072bc0 2024-04-17T09:54:58.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.633+0000 7fc689e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc68c151db0 0x7fc68c0b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:58.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.633+0000 7fc689e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc68c151db0 0x7fc68c0b51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51046/0 (socket says 172.21.15.32:51046) 2024-04-17T09:54:58.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.633+0000 7fc689e00640 1 -- 172.21.15.32:0/3408306233 learned_addr learned my addr 172.21.15.32:0/3408306233 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:58.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.633+0000 7fc683e00640 1 --2- 172.21.15.32:0/3408306233 >> v2:172.21.15.144:3300/0 conn(0x7fc68c06f630 0x7fc68c072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:58.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.633+0000 7fc68a800640 1 --2- 172.21.15.32:0/3408306233 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc68c072bc0 0x7fc68c15e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:58.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.633+0000 7fc689e00640 1 -- 172.21.15.32:0/3408306233 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc68c072bc0 msgr2=0x7fc68c15e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:58.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.633+0000 7fc689e00640 1 --2- 172.21.15.32:0/3408306233 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc68c072bc0 0x7fc68c15e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:58.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.633+0000 7fc689e00640 1 -- 172.21.15.32:0/3408306233 >> v2:172.21.15.144:3300/0 conn(0x7fc68c06f630 msgr2=0x7fc68c072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:58.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.633+0000 7fc689e00640 1 --2- 172.21.15.32:0/3408306233 >> v2:172.21.15.144:3300/0 conn(0x7fc68c06f630 0x7fc68c072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:58.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.633+0000 7fc689e00640 1 -- 172.21.15.32:0/3408306233 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68c0b3730 con 0x7fc68c151db0 2024-04-17T09:54:58.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.633+0000 7fc68a800640 1 --2- 172.21.15.32:0/3408306233 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc68c072bc0 0x7fc68c15e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:54:58.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.633+0000 7fc683e00640 1 --2- 172.21.15.32:0/3408306233 >> v2:172.21.15.144:3300/0 conn(0x7fc68c06f630 0x7fc68c072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:54:58.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.637+0000 7fc689e00640 1 --2- 172.21.15.32:0/3408306233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc68c151db0 0x7fc68c0b51c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc68c070300 tx=0x7fc67c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5955452f750eb076 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:58.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.637+0000 7fc689400640 1 -- 172.21.15.32:0/3408306233 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c0b0070 con 0x7fc68c151db0 2024-04-17T09:54:58.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.637+0000 7fc689400640 1 -- 172.21.15.32:0/3408306233 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc67c0ab600 con 0x7fc68c151db0 2024-04-17T09:54:58.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.637+0000 7fc689400640 1 -- 172.21.15.32:0/3408306233 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c0ab9d0 con 0x7fc68c151db0 2024-04-17T09:54:58.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.637+0000 7fc690c00640 1 -- 172.21.15.32:0/3408306233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc68c151db0 msgr2=0x7fc68c0b51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:58.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.637+0000 7fc690c00640 1 --2- 172.21.15.32:0/3408306233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc68c151db0 0x7fc68c0b51c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc68c070300 tx=0x7fc67c01c520 comp rx=0 tx=0).stop 2024-04-17T09:54:58.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.637+0000 7fc690c00640 1 -- 172.21.15.32:0/3408306233 shutdown_connections 2024-04-17T09:54:58.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.637+0000 7fc690c00640 1 --2- 172.21.15.32:0/3408306233 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc68c072bc0 0x7fc68c15e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:58.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.637+0000 7fc690c00640 1 --2- 172.21.15.32:0/3408306233 >> v2:172.21.15.144:3300/0 conn(0x7fc68c06f630 0x7fc68c072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:58.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.637+0000 7fc690c00640 1 --2- 172.21.15.32:0/3408306233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc68c151db0 0x7fc68c0b51c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:58.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.637+0000 7fc690c00640 1 -- 172.21.15.32:0/3408306233 >> 172.21.15.32:0/3408306233 conn(0x7fc68c06ba30 msgr2=0x7fc68c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:58.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.637+0000 7fc690c00640 1 -- 172.21.15.32:0/3408306233 shutdown_connections 2024-04-17T09:54:58.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.637+0000 7fc690c00640 1 -- 172.21.15.32:0/3408306233 wait complete. 2024-04-17T09:54:58.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.637+0000 7fc690c00640 1 Processor -- start 2024-04-17T09:54:58.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.637+0000 7fc690c00640 1 -- start start 2024-04-17T09:54:58.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.637+0000 7fc690c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc68c06f630 0x7fc68c0aa270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:58.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.637+0000 7fc690c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc68c072bc0 0x7fc68c164fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:58.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.637+0000 7fc690c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc68c151db0 0x7fc68c0aab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:58.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.637+0000 7fc690c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c0b5880 con 0x7fc68c072bc0 2024-04-17T09:54:58.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.637+0000 7fc690c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c0b5700 con 0x7fc68c151db0 2024-04-17T09:54:58.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.637+0000 7fc690c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc68c15ed00 con 0x7fc68c06f630 2024-04-17T09:54:58.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.641+0000 7fc683e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc68c072bc0 0x7fc68c164fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:58.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.641+0000 7fc683e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc68c072bc0 0x7fc68c164fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51050/0 (socket says 172.21.15.32:51050) 2024-04-17T09:54:58.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.641+0000 7fc683e00640 1 -- 172.21.15.32:0/2701622751 learned_addr learned my addr 172.21.15.32:0/2701622751 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:58.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.641+0000 7fc689e00640 1 --2- 172.21.15.32:0/2701622751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc68c06f630 0x7fc68c0aa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.641+0000 7fc68a800640 1 --2- 172.21.15.32:0/2701622751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc68c151db0 0x7fc68c0aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.641+0000 7fc683e00640 1 -- 172.21.15.32:0/2701622751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc68c06f630 msgr2=0x7fc68c0aa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.641+0000 7fc683e00640 1 --2- 172.21.15.32:0/2701622751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc68c06f630 0x7fc68c0aa270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.641+0000 7fc683e00640 1 -- 172.21.15.32:0/2701622751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc68c151db0 msgr2=0x7fc68c0aab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.641+0000 7fc683e00640 1 --2- 172.21.15.32:0/2701622751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc68c151db0 0x7fc68c0aab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.641+0000 7fc683e00640 1 -- 172.21.15.32:0/2701622751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68c0aa7b0 con 0x7fc68c072bc0 2024-04-17T09:54:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.641+0000 7fc68a800640 1 --2- 172.21.15.32:0/2701622751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc68c151db0 0x7fc68c0aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.641+0000 7fc689e00640 1 --2- 172.21.15.32:0/2701622751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc68c06f630 0x7fc68c0aa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:54:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.641+0000 7fc683e00640 1 --2- 172.21.15.32:0/2701622751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc68c072bc0 0x7fc68c164fe0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc67400c7c0 tx=0x7fc67400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.641+0000 7fc682000640 1 -- 172.21.15.32:0/2701622751 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc674015070 con 0x7fc68c072bc0 2024-04-17T09:54:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.641+0000 7fc682000640 1 -- 172.21.15.32:0/2701622751 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc674007a30 con 0x7fc68c072bc0 2024-04-17T09:54:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.641+0000 7fc690c00640 1 -- 172.21.15.32:0/2701622751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc68c0a94f0 con 0x7fc68c072bc0 2024-04-17T09:54:58.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.641+0000 7fc682000640 1 -- 172.21.15.32:0/2701622751 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc674007e00 con 0x7fc68c072bc0 2024-04-17T09:54:58.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.641+0000 7fc690c00640 1 -- 172.21.15.32:0/2701622751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc68c0a9d50 con 0x7fc68c072bc0 2024-04-17T09:54:58.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.641+0000 7fc690c00640 1 -- 172.21.15.32:0/2701622751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc68c068a90 con 0x7fc68c072bc0 2024-04-17T09:54:58.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.641+0000 7fc682000640 1 -- 172.21.15.32:0/2701622751 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc674018070 con 0x7fc68c072bc0 2024-04-17T09:54:58.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.645+0000 7fc682000640 1 --2- 172.21.15.32:0/2701622751 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc668068380 0x7fc66806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:58.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.645+0000 7fc682000640 1 -- 172.21.15.32:0/2701622751 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fc6740864d0 con 0x7fc68c072bc0 2024-04-17T09:54:58.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.645+0000 7fc689e00640 1 --2- 172.21.15.32:0/2701622751 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc668068380 0x7fc66806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:58.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.645+0000 7fc689e00640 1 --2- 172.21.15.32:0/2701622751 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc668068380 0x7fc66806a840 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0ad000 tx=0x7fc67c0a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:58.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.649+0000 7fc682000640 1 -- 172.21.15.32:0/2701622751 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc674056e20 con 0x7fc68c072bc0 2024-04-17T09:54:58.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.849+0000 7fc690c00640 1 -- 172.21.15.32:0/2701622751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc68c072800 con 0x7fc68c072bc0 2024-04-17T09:54:58.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.849+0000 7fc682000640 1 -- 172.21.15.32:0/2701622751 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+3872 (secure 0 0 0) 0x7fc674056720 con 0x7fc68c072bc0 2024-04-17T09:54:58.850 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:54:58.850 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2127207201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2127207201},{"type":"v1","addr":"172.21.15.144:6835","nonce":2127207201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5110,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3782699135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3782699135},{"type":"v1","addr":"172.21.15.144:6834","nonce":3782699135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/385811857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":385811857},{"type":"v1","addr":"172.21.15.32:6836","nonce":385811857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1369751702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1369751702},{"type":"v1","addr":"172.21.15.32:6837","nonce":1369751702}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[]} 2024-04-17T09:54:58.850 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 17 2024-04-17T09:54:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.849+0000 7fc690c00640 1 -- 172.21.15.32:0/2701622751 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc668068380 msgr2=0x7fc66806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.849+0000 7fc690c00640 1 --2- 172.21.15.32:0/2701622751 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc668068380 0x7fc66806a840 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0ad000 tx=0x7fc67c0a6000 comp rx=0 tx=0).stop 2024-04-17T09:54:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.849+0000 7fc690c00640 1 -- 172.21.15.32:0/2701622751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc68c072bc0 msgr2=0x7fc68c164fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.849+0000 7fc690c00640 1 --2- 172.21.15.32:0/2701622751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc68c072bc0 0x7fc68c164fe0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc67400c7c0 tx=0x7fc67400cc80 comp rx=0 tx=0).stop 2024-04-17T09:54:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.849+0000 7fc690c00640 1 -- 172.21.15.32:0/2701622751 shutdown_connections 2024-04-17T09:54:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.849+0000 7fc690c00640 1 --2- 172.21.15.32:0/2701622751 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc668068380 0x7fc66806a840 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.849+0000 7fc690c00640 1 --2- 172.21.15.32:0/2701622751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc68c151db0 0x7fc68c0aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.849+0000 7fc690c00640 1 --2- 172.21.15.32:0/2701622751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc68c072bc0 0x7fc68c164fe0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.849+0000 7fc690c00640 1 --2- 172.21.15.32:0/2701622751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc68c06f630 0x7fc68c0aa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:58.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.849+0000 7fc690c00640 1 -- 172.21.15.32:0/2701622751 >> 172.21.15.32:0/2701622751 conn(0x7fc68c06ba30 msgr2=0x7fc68c0b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:58.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.849+0000 7fc690c00640 1 -- 172.21.15.32:0/2701622751 shutdown_connections 2024-04-17T09:54:58.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:58.849+0000 7fc690c00640 1 -- 172.21.15.32:0/2701622751 wait complete. 2024-04-17T09:54:58.872 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-17T09:54:58.872 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-17T09:54:58.872 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-17T09:54:58.872 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:54:58.872 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-17T09:54:58.872 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-17T09:54:58.872 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-17T09:54:58.873 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:54:58.873 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-17T09:54:58.873 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-17T09:54:58.873 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-17T09:54:58.873 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:54:58.873 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-17T09:54:58.873 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-17T09:54:58.874 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-17T09:54:58.874 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:54:58.883 INFO:tasks.ceph.mds.a:Stopped 2024-04-17T09:54:58.883 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-17T09:54:58.886 INFO:tasks.ceph.mds.c:Stopped 2024-04-17T09:54:58.886 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-17T09:54:58.887 INFO:tasks.ceph.mds.b:Stopped 2024-04-17T09:54:58.887 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-17T09:54:58.888 INFO:tasks.ceph.mds.d:Stopped 2024-04-17T09:54:58.888 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-17T09:54:58.930 INFO:tasks.ceph.mds.a:Started 2024-04-17T09:54:58.930 DEBUG:teuthology.parallel:result is None 2024-04-17T09:54:58.932 INFO:tasks.ceph.mds.c:Started 2024-04-17T09:54:58.932 DEBUG:teuthology.parallel:result is None 2024-04-17T09:54:58.938 INFO:tasks.ceph.mds.b:Started 2024-04-17T09:54:58.938 DEBUG:teuthology.parallel:result is None 2024-04-17T09:54:58.939 INFO:tasks.ceph.mds.d:Started 2024-04-17T09:54:58.940 DEBUG:teuthology.parallel:result is None 2024-04-17T09:54:58.940 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-17T09:54:59.016 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-17T09:54:59.016 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-17T09:54:59.029 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2024-04-17T09:54:59.029 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2024-04-17T09:54:59.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.085+0000 7f33cdc00640 1 Processor -- start 2024-04-17T09:54:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.085+0000 7f33cdc00640 1 -- start start 2024-04-17T09:54:59.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.085+0000 7f33cdc00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f33c8072490 0x7f33c806e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:59.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.085+0000 7f33cdc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33c815b420 0x7f33c806e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:59.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.085+0000 7f33cdc00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33c80b40f0 con 0x7f33c815bd90 2024-04-17T09:54:59.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.085+0000 7f33cdc00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f33c80b43d0 con 0x7f33c8072490 2024-04-17T09:54:59.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.085+0000 7f33cdc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f33c80b46b0 con 0x7f33c815b420 2024-04-17T09:54:59.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.085+0000 7f33c6a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f33c815bd90 0x7f33c8071d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33292/0 (socket says 172.21.15.32:33292) 2024-04-17T09:54:59.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.085+0000 7f33c6a00640 1 -- 172.21.15.32:0/3646108877 learned_addr learned my addr 172.21.15.32:0/3646108877 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:59.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.085+0000 7f33c6000640 1 --2- 172.21.15.32:0/3646108877 >> v2:172.21.15.144:3300/0 conn(0x7f33c8072490 0x7f33c806e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:59.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.085+0000 7f33c7400640 1 --2- 172.21.15.32:0/3646108877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33c815b420 0x7f33c806e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:59.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.085+0000 7f33c5600640 1 -- 172.21.15.32:0/3646108877 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 385971301 0 0) 0x7f33c80b40f0 con 0x7f33c815bd90 2024-04-17T09:54:59.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.085+0000 7f33c5600640 1 -- 172.21.15.32:0/3646108877 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33ac003530 con 0x7f33c815bd90 2024-04-17T09:54:59.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.085+0000 7f33c7400640 1 -- 172.21.15.32:0/3646108877 >> v2:172.21.15.144:3300/0 conn(0x7f33c8072490 msgr2=0x7f33c806e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:54:59.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.085+0000 7f33c7400640 1 --2- 172.21.15.32:0/3646108877 >> v2:172.21.15.144:3300/0 conn(0x7f33c8072490 0x7f33c806e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:59.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.085+0000 7f33c7400640 1 -- 172.21.15.32:0/3646108877 >> v1:172.21.15.32:6789/0 conn(0x7f33c815bd90 legacy=0x7f33c8071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:59.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.085+0000 7f33c7400640 1 -- 172.21.15.32:0/3646108877 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c80b4990 con 0x7f33c815b420 2024-04-17T09:54:59.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.085+0000 7f33c6000640 1 --2- 172.21.15.32:0/3646108877 >> v2:172.21.15.144:3300/0 conn(0x7f33c8072490 0x7f33c806e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:54:59.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.089+0000 7f33c7400640 1 --2- 172.21.15.32:0/3646108877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33c815b420 0x7f33c806e5e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f33c0009d90 tx=0x7f33c001c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b1e2c39675ae4a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:59.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.089+0000 7f33c5600640 1 -- 172.21.15.32:0/3646108877 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c00b2070 con 0x7f33c815b420 2024-04-17T09:54:59.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.089+0000 7f33c5600640 1 -- 172.21.15.32:0/3646108877 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33c00a5800 con 0x7f33c815b420 2024-04-17T09:54:59.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.089+0000 7f33c5600640 1 -- 172.21.15.32:0/3646108877 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c00a5bd0 con 0x7f33c815b420 2024-04-17T09:54:59.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.089+0000 7f33cdc00640 1 -- 172.21.15.32:0/3646108877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33c815b420 msgr2=0x7f33c806e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:59.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.089+0000 7f33cdc00640 1 --2- 172.21.15.32:0/3646108877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33c815b420 0x7f33c806e5e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f33c0009d90 tx=0x7f33c001c8b0 comp rx=0 tx=0).stop 2024-04-17T09:54:59.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.089+0000 7f33cdc00640 1 -- 172.21.15.32:0/3646108877 shutdown_connections 2024-04-17T09:54:59.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.089+0000 7f33cdc00640 1 --2- 172.21.15.32:0/3646108877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33c815b420 0x7f33c806e5e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:59.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.089+0000 7f33cdc00640 1 --2- 172.21.15.32:0/3646108877 >> v2:172.21.15.144:3300/0 conn(0x7f33c8072490 0x7f33c806e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:59.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.089+0000 7f33cdc00640 1 -- 172.21.15.32:0/3646108877 >> 172.21.15.32:0/3646108877 conn(0x7f33c806ba30 msgr2=0x7f33c806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:59.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.089+0000 7f33cdc00640 1 -- 172.21.15.32:0/3646108877 shutdown_connections 2024-04-17T09:54:59.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.089+0000 7f33cdc00640 1 -- 172.21.15.32:0/3646108877 wait complete. 2024-04-17T09:54:59.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33cdc00640 1 Processor -- start 2024-04-17T09:54:59.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33cdc00640 1 -- start start 2024-04-17T09:54:59.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33cdc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33c8072490 0x7f33c8151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:59.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33cdc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33c815b420 0x7f33c80cded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:59.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33cdc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33c815bd90 0x7f33c81519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:59.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33cdc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f33c806eca0 con 0x7f33c815b420 2024-04-17T09:54:59.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33cdc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f33c806eb20 con 0x7f33c8072490 2024-04-17T09:54:59.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33cdc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f33c806ee20 con 0x7f33c815bd90 2024-04-17T09:54:59.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33c6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33c815b420 0x7f33c80cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:59.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33c6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33c815b420 0x7f33c80cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51088/0 (socket says 172.21.15.32:51088) 2024-04-17T09:54:59.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33c6000640 1 -- 172.21.15.32:0/970871420 learned_addr learned my addr 172.21.15.32:0/970871420 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:59.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33c6a00640 1 --2- 172.21.15.32:0/970871420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33c8072490 0x7f33c8151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:59.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33c7400640 1 --2- 172.21.15.32:0/970871420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33c815bd90 0x7f33c81519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:59.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33c6000640 1 -- 172.21.15.32:0/970871420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33c815bd90 msgr2=0x7f33c81519a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:59.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33c6000640 1 --2- 172.21.15.32:0/970871420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33c815bd90 0x7f33c81519a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:59.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33c6000640 1 -- 172.21.15.32:0/970871420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33c8072490 msgr2=0x7f33c8151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:59.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33c6000640 1 --2- 172.21.15.32:0/970871420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33c8072490 0x7f33c8151270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:59.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33c6000640 1 -- 172.21.15.32:0/970871420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c81517b0 con 0x7f33c815b420 2024-04-17T09:54:59.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33c6000640 1 --2- 172.21.15.32:0/970871420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33c815b420 0x7f33c80cded0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f33b400ebe0 tx=0x7f33b400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:59.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33bea00640 1 -- 172.21.15.32:0/970871420 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b4017070 con 0x7f33c815b420 2024-04-17T09:54:59.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33bea00640 1 -- 172.21.15.32:0/970871420 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b400cc30 con 0x7f33c815b420 2024-04-17T09:54:59.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33cdc00640 1 -- 172.21.15.32:0/970871420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c80cebd0 con 0x7f33c815b420 2024-04-17T09:54:59.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33bea00640 1 -- 172.21.15.32:0/970871420 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b4013470 con 0x7f33c815b420 2024-04-17T09:54:59.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.093+0000 7f33cdc00640 1 -- 172.21.15.32:0/970871420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c80cf190 con 0x7f33c815b420 2024-04-17T09:54:59.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.097+0000 7f33cdc00640 1 -- 172.21.15.32:0/970871420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c8068a90 con 0x7f33c815b420 2024-04-17T09:54:59.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.097+0000 7f33bea00640 1 -- 172.21.15.32:0/970871420 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f33b401a070 con 0x7f33c815b420 2024-04-17T09:54:59.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.097+0000 7f33bea00640 1 --2- 172.21.15.32:0/970871420 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f33a8068380 0x7f33a806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:59.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.097+0000 7f33bea00640 1 -- 172.21.15.32:0/970871420 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f33b40865d0 con 0x7f33c815b420 2024-04-17T09:54:59.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.097+0000 7f33c6a00640 1 --2- 172.21.15.32:0/970871420 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f33a8068380 0x7f33a806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:59.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.097+0000 7f33c6a00640 1 --2- 172.21.15.32:0/970871420 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f33a8068380 0x7f33a806a840 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f33c81fada0 tx=0x7f33b8000bd0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:59.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.101+0000 7f33bea00640 1 -- 172.21.15.32:0/970871420 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f33b40567a0 con 0x7f33c815b420 2024-04-17T09:54:59.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.277+0000 7f33cdc00640 1 -- 172.21.15.32:0/970871420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f33c806f480 con 0x7f33c815b420 2024-04-17T09:54:59.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.805+0000 7f33bea00640 1 -- 172.21.15.32:0/970871420 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v40) ==== 97+0+0 (secure 0 0 0) 0x7f33b405aaa0 con 0x7f33c815b420 2024-04-17T09:54:59.807 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2024-04-17T09:54:59.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.809+0000 7f33cdc00640 1 -- 172.21.15.32:0/970871420 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f33a8068380 msgr2=0x7f33a806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:59.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.809+0000 7f33cdc00640 1 --2- 172.21.15.32:0/970871420 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f33a8068380 0x7f33a806a840 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f33c81fada0 tx=0x7f33b8000bd0 comp rx=0 tx=0).stop 2024-04-17T09:54:59.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.809+0000 7f33cdc00640 1 -- 172.21.15.32:0/970871420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33c815b420 msgr2=0x7f33c80cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:59.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.809+0000 7f33cdc00640 1 --2- 172.21.15.32:0/970871420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33c815b420 0x7f33c80cded0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f33b400ebe0 tx=0x7f33b400c520 comp rx=0 tx=0).stop 2024-04-17T09:54:59.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.809+0000 7f33cdc00640 1 -- 172.21.15.32:0/970871420 shutdown_connections 2024-04-17T09:54:59.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.809+0000 7f33cdc00640 1 --2- 172.21.15.32:0/970871420 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f33a8068380 0x7f33a806a840 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:59.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.809+0000 7f33cdc00640 1 --2- 172.21.15.32:0/970871420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33c815bd90 0x7f33c81519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:59.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.809+0000 7f33cdc00640 1 --2- 172.21.15.32:0/970871420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33c815b420 0x7f33c80cded0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:59.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.809+0000 7f33cdc00640 1 --2- 172.21.15.32:0/970871420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33c8072490 0x7f33c8151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:59.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.809+0000 7f33cdc00640 1 -- 172.21.15.32:0/970871420 >> 172.21.15.32:0/970871420 conn(0x7f33c806ba30 msgr2=0x7f33c80b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:59.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.809+0000 7f33cdc00640 1 -- 172.21.15.32:0/970871420 shutdown_connections 2024-04-17T09:54:59.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.809+0000 7f33cdc00640 1 -- 172.21.15.32:0/970871420 wait complete. 2024-04-17T09:54:59.834 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-17T09:54:59.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.973+0000 7fab7ba00640 1 Processor -- start 2024-04-17T09:54:59.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.973+0000 7fab7ba00640 1 -- start start 2024-04-17T09:54:59.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.977+0000 7fab7ba00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab74162bb0 0x7fab7406e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:59.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.977+0000 7fab7ba00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab7406ef10 con 0x7fab74166ef0 2024-04-17T09:54:59.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.977+0000 7fab7ba00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab741673f0 con 0x7fab74071df0 2024-04-17T09:54:59.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.977+0000 7fab7ba00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fab740cfe60 con 0x7fab74162bb0 2024-04-17T09:54:59.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.977+0000 7fab79a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fab74166ef0 0x7fab740b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33298/0 (socket says 172.21.15.32:33298) 2024-04-17T09:54:59.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.977+0000 7fab79a00640 1 -- 172.21.15.32:0/3731996109 learned_addr learned my addr 172.21.15.32:0/3731996109 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:54:59.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.977+0000 7fab79000640 1 --2- 172.21.15.32:0/3731996109 >> v2:172.21.15.32:3300/0 conn(0x7fab74162bb0 0x7fab7406e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:59.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.977+0000 7fab79000640 1 -- 172.21.15.32:0/3731996109 >> v1:172.21.15.144:6789/0 conn(0x7fab74071df0 legacy=0x7fab741667e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:59.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.977+0000 7fab79000640 1 -- 172.21.15.32:0/3731996109 >> v1:172.21.15.32:6789/0 conn(0x7fab74166ef0 legacy=0x7fab740b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:59.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.977+0000 7fab79000640 1 -- 172.21.15.32:0/3731996109 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab741676d0 con 0x7fab74162bb0 2024-04-17T09:54:59.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.977+0000 7fab73400640 1 -- 172.21.15.32:0/3731996109 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3223421677 0 0) 0x7fab7406ef10 con 0x7fab74166ef0 2024-04-17T09:54:59.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.981+0000 7fab79000640 1 --2- 172.21.15.32:0/3731996109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab74162bb0 0x7fab7406e9d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fab680097c0 tx=0x7fab6801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6516ab93525b816d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:59.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.981+0000 7fab73400640 1 -- 172.21.15.32:0/3731996109 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab680b1070 con 0x7fab74162bb0 2024-04-17T09:54:59.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.981+0000 7fab73400640 1 -- 172.21.15.32:0/3731996109 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab680ac5f0 con 0x7fab74162bb0 2024-04-17T09:54:59.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.981+0000 7fab73400640 1 -- 172.21.15.32:0/3731996109 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab680ac9c0 con 0x7fab74162bb0 2024-04-17T09:54:59.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.981+0000 7fab7ba00640 1 -- 172.21.15.32:0/3731996109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab74162bb0 msgr2=0x7fab7406e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:59.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.981+0000 7fab7ba00640 1 --2- 172.21.15.32:0/3731996109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab74162bb0 0x7fab7406e9d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fab680097c0 tx=0x7fab6801c4a0 comp rx=0 tx=0).stop 2024-04-17T09:54:59.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.981+0000 7fab7ba00640 1 -- 172.21.15.32:0/3731996109 shutdown_connections 2024-04-17T09:54:59.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.981+0000 7fab7ba00640 1 --2- 172.21.15.32:0/3731996109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab74162bb0 0x7fab7406e9d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:59.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.981+0000 7fab7ba00640 1 -- 172.21.15.32:0/3731996109 >> 172.21.15.32:0/3731996109 conn(0x7fab7406ba30 msgr2=0x7fab7406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:54:59.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.981+0000 7fab7ba00640 1 -- 172.21.15.32:0/3731996109 shutdown_connections 2024-04-17T09:54:59.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.981+0000 7fab7ba00640 1 -- 172.21.15.32:0/3731996109 wait complete. 2024-04-17T09:54:59.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.981+0000 7fab7ba00640 1 Processor -- start 2024-04-17T09:54:59.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab7ba00640 1 -- start start 2024-04-17T09:54:59.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab7ba00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab74071df0 0x7fab741ff000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:59.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab7ba00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab74162bb0 0x7fab7416b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:59.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab7ba00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab74166ef0 0x7fab7416d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:59.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab7ba00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fab74072610 con 0x7fab74162bb0 2024-04-17T09:54:59.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab7ba00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fab74162f90 con 0x7fab74071df0 2024-04-17T09:54:59.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab7ba00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fab74169fb0 con 0x7fab74166ef0 2024-04-17T09:54:59.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab73e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab74162bb0 0x7fab7416b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:59.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab73e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab74162bb0 0x7fab7416b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51112/0 (socket says 172.21.15.32:51112) 2024-04-17T09:54:59.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab73e00640 1 -- 172.21.15.32:0/224926099 learned_addr learned my addr 172.21.15.32:0/224926099 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:54:59.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab79000640 1 --2- 172.21.15.32:0/224926099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab74071df0 0x7fab741ff000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:59.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab79a00640 1 --2- 172.21.15.32:0/224926099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab74166ef0 0x7fab7416d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:59.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab73e00640 1 -- 172.21.15.32:0/224926099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab74166ef0 msgr2=0x7fab7416d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:59.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab73e00640 1 --2- 172.21.15.32:0/224926099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab74166ef0 0x7fab7416d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:59.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab73e00640 1 -- 172.21.15.32:0/224926099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab74071df0 msgr2=0x7fab741ff000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:54:59.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab73e00640 1 --2- 172.21.15.32:0/224926099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab74071df0 0x7fab741ff000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:54:59.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab73e00640 1 -- 172.21.15.32:0/224926099 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab741ff540 con 0x7fab74162bb0 2024-04-17T09:54:59.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab73e00640 1 --2- 172.21.15.32:0/224926099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab74162bb0 0x7fab7416b260 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fab6000c3c0 tx=0x7fab6000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:59.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab70c00640 1 -- 172.21.15.32:0/224926099 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab60017070 con 0x7fab74162bb0 2024-04-17T09:54:59.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab70c00640 1 -- 172.21.15.32:0/224926099 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab600052d0 con 0x7fab74162bb0 2024-04-17T09:54:59.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab70c00640 1 -- 172.21.15.32:0/224926099 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab60003840 con 0x7fab74162bb0 2024-04-17T09:54:59.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab7ba00640 1 -- 172.21.15.32:0/224926099 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab741693a0 con 0x7fab74162bb0 2024-04-17T09:54:59.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab7ba00640 1 -- 172.21.15.32:0/224926099 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab74169960 con 0x7fab74162bb0 2024-04-17T09:54:59.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.985+0000 7fab7ba00640 1 -- 172.21.15.32:0/224926099 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab74068a90 con 0x7fab74162bb0 2024-04-17T09:54:59.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.989+0000 7fab70c00640 1 -- 172.21.15.32:0/224926099 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fab6001a070 con 0x7fab74162bb0 2024-04-17T09:54:59.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.989+0000 7fab70c00640 1 --2- 172.21.15.32:0/224926099 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab540683b0 0x7fab5406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:54:59.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.989+0000 7fab70c00640 1 -- 172.21.15.32:0/224926099 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7fab60085900 con 0x7fab74162bb0 2024-04-17T09:54:59.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.989+0000 7fab79000640 1 --2- 172.21.15.32:0/224926099 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab540683b0 0x7fab5406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:54:59.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.989+0000 7fab79000640 1 --2- 172.21.15.32:0/224926099 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab540683b0 0x7fab5406a870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fab680ae000 tx=0x7fab680a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:54:59.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:54:59.993+0000 7fab70c00640 1 -- 172.21.15.32:0/224926099 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fab60005480 con 0x7fab74162bb0 2024-04-17T09:55:00.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.213+0000 7fab7ba00640 1 -- 172.21.15.32:0/224926099 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fab740cc680 con 0x7fab74162bb0 2024-04-17T09:55:00.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.213+0000 7fab70c00640 1 -- 172.21.15.32:0/224926099 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7fab60055b20 con 0x7fab74162bb0 2024-04-17T09:55:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:00.216 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:55:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2024-04-17T09:55:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2024-04-17T09:55:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mzM1KExAAn0WH+Gf6gGV/dJonXzAgPQ==", 2024-04-17T09:55:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:55:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:55:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:55:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:00.216 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2024-04-17T09:55:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mQnJ4HxAAT0rFHhdAj7PMxZGiqEyWbg==", 2024-04-17T09:55:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:55:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:55:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:55:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2024-04-17T09:55:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mWWi5GhAA3RAdRxK6GRwuVCH4DHoMeA==", 2024-04-17T09:55:00.217 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:55:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:55:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:55:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2024-04-17T09:55:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9m9TMAJxAAbmUp05GHRE1AqCmq83scVQ==", 2024-04-17T09:55:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:55:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:55:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:55:00.218 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2024-04-17T09:55:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDRmx9mnPwyDBAAxD10jhxptVH5zhupengElg==", 2024-04-17T09:55:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:55:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:55:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2024-04-17T09:55:00.219 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUmx9mh3iuGBAAqvWSRyNX3u/TjEH6TJpqdA==", 2024-04-17T09:55:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:55:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:55:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2024-04-17T09:55:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXmx9mbe1CIRAA1pmzAl04tGkTu6QLpmCkag==", 2024-04-17T09:55:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:55:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:55:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:00.220 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2024-04-17T09:55:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDamx9mIwXoKxAAWPTFaYdp4lIZhuOU7F0x8g==", 2024-04-17T09:55:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:55:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:55:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2024-04-17T09:55:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDgmx9mp0s/JxAAv+usQqwrxgKINRlkDhhydQ==", 2024-04-17T09:55:00.221 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:55:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:55:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2024-04-17T09:55:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDjmx9mMVStMhAA/n8jk+LVDosWhziXk/lLRQ==", 2024-04-17T09:55:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:55:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:55:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:00.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2024-04-17T09:55:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDnmx9m/YQlABAAOqfWzVtgonD7b+WlR95mnA==", 2024-04-17T09:55:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:55:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:55:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2024-04-17T09:55:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDqmx9m572mCBAAqBib/yeQOBno+0B1K1dLxA==", 2024-04-17T09:55:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:55:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:55:00.223 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2024-04-17T09:55:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mUXoLKxAANV7zhLvL0DqHngk4TtsdBQ==", 2024-04-17T09:55:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:55:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2024-04-17T09:55:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-17T09:55:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2024-04-17T09:55:00.224 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mZBeRNBAAQg5K950mp6+XND7NCABAKQ==", 2024-04-17T09:55:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:55:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-17T09:55:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-17T09:55:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-17T09:55:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2024-04-17T09:55:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mWKstMRAAp18Hbnk02w+hT0sYZ3tSLA==", 2024-04-17T09:55:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:55:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-17T09:55:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-17T09:55:00.225 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-17T09:55:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2024-04-17T09:55:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mRQ2wOhAAzlWXFtP0WaXWmneqxnZ5/w==", 2024-04-17T09:55:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:55:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-17T09:55:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-17T09:55:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-17T09:55:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2024-04-17T09:55:00.226 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDOmx9maDStMxAAMCa36HfycwV3WbLg6Voo9Q==", 2024-04-17T09:55:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-17T09:55:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:55:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:55:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2024-04-17T09:55:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9myy0oDxAAPD2BTjIP09hiU5L7i6gUMg==", 2024-04-17T09:55:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-17T09:55:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-17T09:55:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:00.227 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2024-04-17T09:55:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9m3FmACxAApri5AI5PeR0+/HLev8DKcw==", 2024-04-17T09:55:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-17T09:55:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-17T09:55:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:00.228 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-17T09:55:00.228 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:55:00.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.217+0000 7fab7ba00640 1 -- 172.21.15.32:0/224926099 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab540683b0 msgr2=0x7fab5406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:00.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.217+0000 7fab7ba00640 1 --2- 172.21.15.32:0/224926099 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab540683b0 0x7fab5406a870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fab680ae000 tx=0x7fab680a7000 comp rx=0 tx=0).stop 2024-04-17T09:55:00.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.217+0000 7fab7ba00640 1 -- 172.21.15.32:0/224926099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab74162bb0 msgr2=0x7fab7416b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:00.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.217+0000 7fab7ba00640 1 --2- 172.21.15.32:0/224926099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab74162bb0 0x7fab7416b260 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fab6000c3c0 tx=0x7fab6000c880 comp rx=0 tx=0).stop 2024-04-17T09:55:00.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.217+0000 7fab7ba00640 1 -- 172.21.15.32:0/224926099 shutdown_connections 2024-04-17T09:55:00.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.217+0000 7fab7ba00640 1 --2- 172.21.15.32:0/224926099 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab540683b0 0x7fab5406a870 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.217+0000 7fab7ba00640 1 --2- 172.21.15.32:0/224926099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab74166ef0 0x7fab7416d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.217+0000 7fab7ba00640 1 --2- 172.21.15.32:0/224926099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab74162bb0 0x7fab7416b260 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.217+0000 7fab7ba00640 1 --2- 172.21.15.32:0/224926099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab74071df0 0x7fab741ff000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.217+0000 7fab7ba00640 1 -- 172.21.15.32:0/224926099 >> 172.21.15.32:0/224926099 conn(0x7fab7406ba30 msgr2=0x7fab7415f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:00.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.217+0000 7fab7ba00640 1 -- 172.21.15.32:0/224926099 shutdown_connections 2024-04-17T09:55:00.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.217+0000 7fab7ba00640 1 -- 172.21.15.32:0/224926099 wait complete. 2024-04-17T09:55:00.238 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:55:00.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.377+0000 7fb4d4600640 1 Processor -- start 2024-04-17T09:55:00.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.377+0000 7fb4d4600640 1 -- start start 2024-04-17T09:55:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.381+0000 7fb4d4600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4cc145f30 0x7fb4cc071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.381+0000 7fb4d4600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4cc15d8d0 0x7fb4cc072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.381+0000 7fb4d4600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb4cc0cce70 0x7fb4cc0cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.381+0000 7fb4d4600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb4cc06f400 con 0x7fb4cc145f30 2024-04-17T09:55:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.381+0000 7fb4d4600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb4cc146310 con 0x7fb4cc0cce70 2024-04-17T09:55:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.381+0000 7fb4d4600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb4cc15d440 con 0x7fb4cc15d8d0 2024-04-17T09:55:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.381+0000 7fb4d1c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4cc145f30 0x7fb4cc071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.381+0000 7fb4d1c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4cc145f30 0x7fb4cc071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53804/0 (socket says 172.21.15.32:53804) 2024-04-17T09:55:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.381+0000 7fb4d1c00640 1 -- 172.21.15.32:0/1188029912 learned_addr learned my addr 172.21.15.32:0/1188029912 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.381+0000 7fb4d1200640 1 --2- 172.21.15.32:0/1188029912 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4cc15d8d0 0x7fb4cc072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.381+0000 7fb4d2600640 1 --2- 172.21.15.32:0/1188029912 >> v2:172.21.15.144:3300/0 conn(0x7fb4cc0cce70 0x7fb4cc0cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:00.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.381+0000 7fb4d1c00640 1 -- 172.21.15.32:0/1188029912 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4cc15d8d0 msgr2=0x7fb4cc072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:00.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.381+0000 7fb4d1c00640 1 --2- 172.21.15.32:0/1188029912 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4cc15d8d0 0x7fb4cc072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.381+0000 7fb4d1c00640 1 -- 172.21.15.32:0/1188029912 >> v2:172.21.15.144:3300/0 conn(0x7fb4cc0cce70 msgr2=0x7fb4cc0cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:00.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.381+0000 7fb4d1c00640 1 --2- 172.21.15.32:0/1188029912 >> v2:172.21.15.144:3300/0 conn(0x7fb4cc0cce70 0x7fb4cc0cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.381+0000 7fb4d1c00640 1 -- 172.21.15.32:0/1188029912 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4cc0cd840 con 0x7fb4cc145f30 2024-04-17T09:55:00.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.381+0000 7fb4d2600640 1 --2- 172.21.15.32:0/1188029912 >> v2:172.21.15.144:3300/0 conn(0x7fb4cc0cce70 0x7fb4cc0cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:00.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.385+0000 7fb4d1c00640 1 --2- 172.21.15.32:0/1188029912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4cc145f30 0x7fb4cc071df0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0b5a10 tx=0x7fb4c001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15434c511581d9e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:00.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.385+0000 7fb4cbe00640 1 -- 172.21.15.32:0/1188029912 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c00b5070 con 0x7fb4cc145f30 2024-04-17T09:55:00.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.385+0000 7fb4cbe00640 1 -- 172.21.15.32:0/1188029912 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4c00a89d0 con 0x7fb4cc145f30 2024-04-17T09:55:00.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.385+0000 7fb4cbe00640 1 -- 172.21.15.32:0/1188029912 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c00a8da0 con 0x7fb4cc145f30 2024-04-17T09:55:00.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.385+0000 7fb4d4600640 1 -- 172.21.15.32:0/1188029912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4cc145f30 msgr2=0x7fb4cc071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:00.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.385+0000 7fb4d4600640 1 --2- 172.21.15.32:0/1188029912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4cc145f30 0x7fb4cc071df0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0b5a10 tx=0x7fb4c001ed10 comp rx=0 tx=0).stop 2024-04-17T09:55:00.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.385+0000 7fb4d4600640 1 -- 172.21.15.32:0/1188029912 shutdown_connections 2024-04-17T09:55:00.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.385+0000 7fb4d4600640 1 --2- 172.21.15.32:0/1188029912 >> v2:172.21.15.144:3300/0 conn(0x7fb4cc0cce70 0x7fb4cc0cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.385+0000 7fb4d4600640 1 --2- 172.21.15.32:0/1188029912 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4cc15d8d0 0x7fb4cc072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.385+0000 7fb4d4600640 1 --2- 172.21.15.32:0/1188029912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4cc145f30 0x7fb4cc071df0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.385+0000 7fb4d4600640 1 -- 172.21.15.32:0/1188029912 >> 172.21.15.32:0/1188029912 conn(0x7fb4cc06ba30 msgr2=0x7fb4cc06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:00.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.385+0000 7fb4d4600640 1 -- 172.21.15.32:0/1188029912 shutdown_connections 2024-04-17T09:55:00.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.385+0000 7fb4d4600640 1 -- 172.21.15.32:0/1188029912 wait complete. 2024-04-17T09:55:00.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.385+0000 7fb4d4600640 1 Processor -- start 2024-04-17T09:55:00.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.385+0000 7fb4d4600640 1 -- start start 2024-04-17T09:55:00.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.389+0000 7fb4d4600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4cc0cce70 0x7fb4cc151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.389+0000 7fb4d4600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4cc145f30 0x7fb4cc0b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.389+0000 7fb4d4600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4cc15d8d0 0x7fb4cc0b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.389+0000 7fb4d4600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc15cdf0 con 0x7fb4cc145f30 2024-04-17T09:55:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.389+0000 7fb4d4600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc15cb90 con 0x7fb4cc0cce70 2024-04-17T09:55:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.389+0000 7fb4d4600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb4cc15d0d0 con 0x7fb4cc15d8d0 2024-04-17T09:55:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.389+0000 7fb4d1200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4cc145f30 0x7fb4cc0b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.389+0000 7fb4d1200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4cc145f30 0x7fb4cc0b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53816/0 (socket says 172.21.15.32:53816) 2024-04-17T09:55:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.389+0000 7fb4d1c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4cc0cce70 0x7fb4cc151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.389+0000 7fb4d1200640 1 -- 172.21.15.32:0/684666968 learned_addr learned my addr 172.21.15.32:0/684666968 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.389+0000 7fb4d2600640 1 --2- 172.21.15.32:0/684666968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4cc15d8d0 0x7fb4cc0b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:00.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.389+0000 7fb4d1200640 1 -- 172.21.15.32:0/684666968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4cc15d8d0 msgr2=0x7fb4cc0b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:00.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.389+0000 7fb4d1200640 1 --2- 172.21.15.32:0/684666968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4cc15d8d0 0x7fb4cc0b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.389+0000 7fb4d1200640 1 -- 172.21.15.32:0/684666968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4cc0cce70 msgr2=0x7fb4cc151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:00.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.389+0000 7fb4d1200640 1 --2- 172.21.15.32:0/684666968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4cc0cce70 0x7fb4cc151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.389+0000 7fb4d1200640 1 -- 172.21.15.32:0/684666968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4cc0b4f30 con 0x7fb4cc145f30 2024-04-17T09:55:00.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.389+0000 7fb4d1200640 1 --2- 172.21.15.32:0/684666968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4cc145f30 0x7fb4cc0b44b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb4b800d620 tx=0x7fb4b800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:00.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.389+0000 7fb4c9600640 1 -- 172.21.15.32:0/684666968 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b800b620 con 0x7fb4cc145f30 2024-04-17T09:55:00.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.389+0000 7fb4c9600640 1 -- 172.21.15.32:0/684666968 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4b8019070 con 0x7fb4cc145f30 2024-04-17T09:55:00.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.389+0000 7fb4c9600640 1 -- 172.21.15.32:0/684666968 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b800b9e0 con 0x7fb4cc145f30 2024-04-17T09:55:00.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.389+0000 7fb4d4600640 1 -- 172.21.15.32:0/684666968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4cc0c98b0 con 0x7fb4cc145f30 2024-04-17T09:55:00.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.389+0000 7fb4d4600640 1 -- 172.21.15.32:0/684666968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4cc0c9e70 con 0x7fb4cc145f30 2024-04-17T09:55:00.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.389+0000 7fb4d4600640 1 -- 172.21.15.32:0/684666968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4cc068a90 con 0x7fb4cc145f30 2024-04-17T09:55:00.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.393+0000 7fb4c9600640 1 -- 172.21.15.32:0/684666968 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb4b800bb80 con 0x7fb4cc145f30 2024-04-17T09:55:00.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.393+0000 7fb4c9600640 1 --2- 172.21.15.32:0/684666968 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb4ac068380 0x7fb4ac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:00.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.393+0000 7fb4c9600640 1 -- 172.21.15.32:0/684666968 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7fb4b8086950 con 0x7fb4cc145f30 2024-04-17T09:55:00.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.393+0000 7fb4d1c00640 1 --2- 172.21.15.32:0/684666968 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb4ac068380 0x7fb4ac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:00.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.393+0000 7fb4d1c00640 1 --2- 172.21.15.32:0/684666968 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb4ac068380 0x7fb4ac06a840 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00b2000 tx=0x7fb4c0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:00.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.397+0000 7fb4c9600640 1 -- 172.21.15.32:0/684666968 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fb4b8056c00 con 0x7fb4cc145f30 2024-04-17T09:55:00.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.565+0000 7fb4d4600640 1 -- 172.21.15.32:0/684666968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4cc06f070 con 0x7fb4cc145f30 2024-04-17T09:55:00.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.565+0000 7fb4c9600640 1 -- 172.21.15.32:0/684666968 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+12903 (secure 0 0 0) 0x7fb4b805af00 con 0x7fb4cc145f30 2024-04-17T09:55:00.566 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:00.567 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":42,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:54:59.817509+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:55:00.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.565+0000 7fb4d4600640 1 -- 172.21.15.32:0/684666968 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb4ac068380 msgr2=0x7fb4ac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:00.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.565+0000 7fb4d4600640 1 --2- 172.21.15.32:0/684666968 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb4ac068380 0x7fb4ac06a840 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00b2000 tx=0x7fb4c0009040 comp rx=0 tx=0).stop 2024-04-17T09:55:00.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.565+0000 7fb4d4600640 1 -- 172.21.15.32:0/684666968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4cc145f30 msgr2=0x7fb4cc0b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:00.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.569+0000 7fb4d4600640 1 --2- 172.21.15.32:0/684666968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4cc145f30 0x7fb4cc0b44b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb4b800d620 tx=0x7fb4b800dae0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.569+0000 7fb4d4600640 1 -- 172.21.15.32:0/684666968 shutdown_connections 2024-04-17T09:55:00.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.569+0000 7fb4d4600640 1 --2- 172.21.15.32:0/684666968 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb4ac068380 0x7fb4ac06a840 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.569+0000 7fb4d4600640 1 --2- 172.21.15.32:0/684666968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4cc15d8d0 0x7fb4cc0b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.569+0000 7fb4d4600640 1 --2- 172.21.15.32:0/684666968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4cc145f30 0x7fb4cc0b44b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.569+0000 7fb4d4600640 1 --2- 172.21.15.32:0/684666968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4cc0cce70 0x7fb4cc151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.569+0000 7fb4d4600640 1 -- 172.21.15.32:0/684666968 >> 172.21.15.32:0/684666968 conn(0x7fb4cc06ba30 msgr2=0x7fb4cc0b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:00.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.569+0000 7fb4d4600640 1 -- 172.21.15.32:0/684666968 shutdown_connections 2024-04-17T09:55:00.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.569+0000 7fb4d4600640 1 -- 172.21.15.32:0/684666968 wait complete. 2024-04-17T09:55:00.588 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:55:00.589 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:55:00.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c57000640 1 Processor -- start 2024-04-17T09:55:00.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c57000640 1 -- start start 2024-04-17T09:55:00.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c57000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0c50160060 0x7f0c500d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:00.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c57000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c500b1a30 con 0x7f0c50070bc0 2024-04-17T09:55:00.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c57000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c5009b530 con 0x7f0c500d1430 2024-04-17T09:55:00.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c57000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0c500b1d10 con 0x7f0c50160060 2024-04-17T09:55:00.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c56600640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0c50070bc0 0x7f0c500cd0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33062/0 (socket says 172.21.15.32:33062) 2024-04-17T09:55:00.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c56600640 1 -- 172.21.15.32:0/3766977493 learned_addr learned my addr 172.21.15.32:0/3766977493 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:00.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c55c00640 1 --2- 172.21.15.32:0/3766977493 >> v2:172.21.15.144:3300/0 conn(0x7f0c50160060 0x7f0c500d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:00.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c4fe00640 1 -- 172.21.15.32:0/3766977493 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1448167718 0 0) 0x7f0c500b1a30 con 0x7f0c50070bc0 2024-04-17T09:55:00.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c4fe00640 1 -- 172.21.15.32:0/3766977493 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c38003530 con 0x7f0c50070bc0 2024-04-17T09:55:00.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c4fe00640 1 -- 172.21.15.32:0/3766977493 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1691379056 0 0) 0x7f0c5009b530 con 0x7f0c500d1430 2024-04-17T09:55:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c4fe00640 1 -- 172.21.15.32:0/3766977493 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c500b1a30 con 0x7f0c500d1430 2024-04-17T09:55:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c4fe00640 1 -- 172.21.15.32:0/3766977493 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4033937304 0 0) 0x7f0c38003530 con 0x7f0c50070bc0 2024-04-17T09:55:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c4fe00640 1 -- 172.21.15.32:0/3766977493 >> v2:172.21.15.144:3300/0 conn(0x7f0c50160060 msgr2=0x7f0c500d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c4fe00640 1 --2- 172.21.15.32:0/3766977493 >> v2:172.21.15.144:3300/0 conn(0x7f0c50160060 0x7f0c500d1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c4fe00640 1 -- 172.21.15.32:0/3766977493 >> v1:172.21.15.144:6789/0 conn(0x7f0c500d1430 legacy=0x7f0c500704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c4fe00640 1 -- 172.21.15.32:0/3766977493 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5009b810 con 0x7f0c50070bc0 2024-04-17T09:55:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c4fe00640 1 -- 172.21.15.32:0/3766977493 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f0c48001a00 con 0x7f0c50070bc0 2024-04-17T09:55:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c4fe00640 1 -- 172.21.15.32:0/3766977493 >> v1:172.21.15.32:6789/0 conn(0x7f0c50070bc0 legacy=0x7f0c500cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c4fe00640 1 --2- 172.21.15.32:0/3766977493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c38003bc0 0x7f0c38003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c4fe00640 1 --2- 172.21.15.32:0/3766977493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c38004810 0x7f0c38004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c4fe00640 1 --2- 172.21.15.32:0/3766977493 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c380052e0 0x7f0c3800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c4fe00640 1 -- 172.21.15.32:0/3766977493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0c500b1d10 con 0x7f0c38004810 2024-04-17T09:55:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c4fe00640 1 -- 172.21.15.32:0/3766977493 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3800c1f0 con 0x7f0c38003bc0 2024-04-17T09:55:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c4fe00640 1 -- 172.21.15.32:0/3766977493 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0c3800c3f0 con 0x7f0c380052e0 2024-04-17T09:55:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c55200640 1 --2- 172.21.15.32:0/3766977493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c38004810 0x7f0c38004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c55c00640 1 --2- 172.21.15.32:0/3766977493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c38003bc0 0x7f0c38003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:00.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c56600640 1 --2- 172.21.15.32:0/3766977493 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c380052e0 0x7f0c3800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:00.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c55200640 1 -- 172.21.15.32:0/3766977493 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c380052e0 msgr2=0x7f0c3800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:00.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c55200640 1 --2- 172.21.15.32:0/3766977493 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c380052e0 0x7f0c3800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c55200640 1 -- 172.21.15.32:0/3766977493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c38003bc0 msgr2=0x7f0c38003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:00.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c55200640 1 --2- 172.21.15.32:0/3766977493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c38003bc0 0x7f0c38003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c55200640 1 -- 172.21.15.32:0/3766977493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c38003530 con 0x7f0c38004810 2024-04-17T09:55:00.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.733+0000 7f0c56600640 1 -- 172.21.15.32:0/3766977493 reap_dead start 2024-04-17T09:55:00.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.737+0000 7f0c55200640 1 --2- 172.21.15.32:0/3766977493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c38004810 0x7f0c38004c90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c0018d0 tx=0x7f0c3c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba46912ff4c6cae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:00.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.737+0000 7f0c4fe00640 1 -- 172.21.15.32:0/3766977493 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c0b0070 con 0x7f0c38004810 2024-04-17T09:55:00.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.737+0000 7f0c4fe00640 1 -- 172.21.15.32:0/3766977493 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c3c0052f0 con 0x7f0c38004810 2024-04-17T09:55:00.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.737+0000 7f0c4fe00640 1 -- 172.21.15.32:0/3766977493 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c001520 con 0x7f0c38004810 2024-04-17T09:55:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.737+0000 7f0c57000640 1 -- 172.21.15.32:0/3766977493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c38004810 msgr2=0x7f0c38004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.737+0000 7f0c57000640 1 --2- 172.21.15.32:0/3766977493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c38004810 0x7f0c38004c90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c0018d0 tx=0x7f0c3c01c530 comp rx=0 tx=0).stop 2024-04-17T09:55:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.737+0000 7f0c57000640 1 -- 172.21.15.32:0/3766977493 shutdown_connections 2024-04-17T09:55:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.737+0000 7f0c57000640 1 --2- 172.21.15.32:0/3766977493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c38004810 0x7f0c38004c90 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.737+0000 7f0c57000640 1 -- 172.21.15.32:0/3766977493 >> 172.21.15.32:0/3766977493 conn(0x7f0c500b3fa0 msgr2=0x7f0c500b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.737+0000 7f0c57000640 1 -- 172.21.15.32:0/3766977493 shutdown_connections 2024-04-17T09:55:00.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.737+0000 7f0c57000640 1 -- 172.21.15.32:0/3766977493 wait complete. 2024-04-17T09:55:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c57000640 1 Processor -- start 2024-04-17T09:55:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c57000640 1 -- start start 2024-04-17T09:55:00.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c57000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c38004810 0x7f0c500b7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:00.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c57000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c500b5d30 0x7f0c500ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:00.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c57000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c50150390 0x7f0c50150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:00.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c57000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0c500d0bc0 con 0x7f0c38004810 2024-04-17T09:55:00.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c57000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0c500d2300 con 0x7f0c50150390 2024-04-17T09:55:00.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c57000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0c500d0d40 con 0x7f0c500b5d30 2024-04-17T09:55:00.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c55c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c38004810 0x7f0c500b7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:00.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c55c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c38004810 0x7f0c500b7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53830/0 (socket says 172.21.15.32:53830) 2024-04-17T09:55:00.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c55c00640 1 -- 172.21.15.32:0/1750528223 learned_addr learned my addr 172.21.15.32:0/1750528223 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:00.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c55200640 1 --2- 172.21.15.32:0/1750528223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c500b5d30 0x7f0c500ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:00.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c56600640 1 --2- 172.21.15.32:0/1750528223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c50150390 0x7f0c50150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:00.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c55c00640 1 -- 172.21.15.32:0/1750528223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c500b5d30 msgr2=0x7f0c500ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c55c00640 1 --2- 172.21.15.32:0/1750528223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c500b5d30 0x7f0c500ceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c55c00640 1 -- 172.21.15.32:0/1750528223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c50150390 msgr2=0x7f0c50150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c55c00640 1 --2- 172.21.15.32:0/1750528223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c50150390 0x7f0c50150820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c55c00640 1 -- 172.21.15.32:0/1750528223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c50150d60 con 0x7f0c38004810 2024-04-17T09:55:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c55c00640 1 --2- 172.21.15.32:0/1750528223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c38004810 0x7f0c500b7480 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0c44011930 tx=0x7f0c44011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c4d600640 1 -- 172.21.15.32:0/1750528223 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c44019070 con 0x7f0c38004810 2024-04-17T09:55:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c4d600640 1 -- 172.21.15.32:0/1750528223 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c44004240 con 0x7f0c38004810 2024-04-17T09:55:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c4d600640 1 -- 172.21.15.32:0/1750528223 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c440095b0 con 0x7f0c38004810 2024-04-17T09:55:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c57000640 1 -- 172.21.15.32:0/1750528223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c500b6210 con 0x7f0c38004810 2024-04-17T09:55:00.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.741+0000 7f0c57000640 1 -- 172.21.15.32:0/1750528223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c500b67d0 con 0x7f0c38004810 2024-04-17T09:55:00.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.745+0000 7f0c57000640 1 -- 172.21.15.32:0/1750528223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c5004fda0 con 0x7f0c38004810 2024-04-17T09:55:00.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.745+0000 7f0c4d600640 1 -- 172.21.15.32:0/1750528223 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0c4401c070 con 0x7f0c38004810 2024-04-17T09:55:00.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.745+0000 7f0c4d600640 1 --2- 172.21.15.32:0/1750528223 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0c300683b0 0x7f0c3006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:00.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.745+0000 7f0c4d600640 1 -- 172.21.15.32:0/1750528223 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f0c44086c60 con 0x7f0c38004810 2024-04-17T09:55:00.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.745+0000 7f0c55200640 1 --2- 172.21.15.32:0/1750528223 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0c300683b0 0x7f0c3006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:00.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.745+0000 7f0c55200640 1 --2- 172.21.15.32:0/1750528223 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0c300683b0 0x7f0c3006a870 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c0ad000 tx=0x7f0c3c0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:00.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.749+0000 7f0c4d600640 1 -- 172.21.15.32:0/1750528223 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0c44056f10 con 0x7f0c38004810 2024-04-17T09:55:00.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.921+0000 7f0c57000640 1 -- 172.21.15.32:0/1750528223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0c501ea0b0 con 0x7f0c38004810 2024-04-17T09:55:00.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.921+0000 7f0c4d600640 1 -- 172.21.15.32:0/1750528223 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) ==== 93+0+10 (secure 0 0 0) 0x7f0c4405b210 con 0x7f0c38004810 2024-04-17T09:55:00.921 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:55:00.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.921+0000 7f0c57000640 1 -- 172.21.15.32:0/1750528223 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0c300683b0 msgr2=0x7f0c3006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:00.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.921+0000 7f0c57000640 1 --2- 172.21.15.32:0/1750528223 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0c300683b0 0x7f0c3006a870 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c0ad000 tx=0x7f0c3c0a7000 comp rx=0 tx=0).stop 2024-04-17T09:55:00.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.921+0000 7f0c57000640 1 -- 172.21.15.32:0/1750528223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c38004810 msgr2=0x7f0c500b7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:00.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.921+0000 7f0c57000640 1 --2- 172.21.15.32:0/1750528223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c38004810 0x7f0c500b7480 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0c44011930 tx=0x7f0c44011df0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.921+0000 7f0c57000640 1 -- 172.21.15.32:0/1750528223 shutdown_connections 2024-04-17T09:55:00.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.921+0000 7f0c57000640 1 --2- 172.21.15.32:0/1750528223 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0c300683b0 0x7f0c3006a870 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.921+0000 7f0c57000640 1 --2- 172.21.15.32:0/1750528223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c500b5d30 0x7f0c500ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.921+0000 7f0c57000640 1 --2- 172.21.15.32:0/1750528223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c50150390 0x7f0c50150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.921+0000 7f0c57000640 1 --2- 172.21.15.32:0/1750528223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c38004810 0x7f0c500b7480 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:00.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.921+0000 7f0c57000640 1 -- 172.21.15.32:0/1750528223 >> 172.21.15.32:0/1750528223 conn(0x7f0c500b3fa0 msgr2=0x7f0c500b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:00.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.921+0000 7f0c57000640 1 -- 172.21.15.32:0/1750528223 shutdown_connections 2024-04-17T09:55:00.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:00.921+0000 7f0c57000640 1 -- 172.21.15.32:0/1750528223 wait complete. 2024-04-17T09:55:00.942 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T09:55:01.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.081+0000 7f6e20800640 1 Processor -- start 2024-04-17T09:55:01.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.081+0000 7f6e20800640 1 -- start start 2024-04-17T09:55:01.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.081+0000 7f6e20800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6e1815baa0 0x7f6e1815bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:01.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.081+0000 7f6e20800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6e1806ef60 0x7f6e1806f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:01.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.081+0000 7f6e20800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e18072bb0 con 0x7f6e18144e90 2024-04-17T09:55:01.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.081+0000 7f6e20800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6e1806f8f0 con 0x7f6e1806ef60 2024-04-17T09:55:01.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.081+0000 7f6e20800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f6e18145270 con 0x7f6e1815baa0 2024-04-17T09:55:01.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.081+0000 7f6e1de00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6e18144e90 0x7f6e180b48a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33078/0 (socket says 172.21.15.32:33078) 2024-04-17T09:55:01.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.081+0000 7f6e1de00640 1 -- 172.21.15.32:0/3179772494 learned_addr learned my addr 172.21.15.32:0/3179772494 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.081+0000 7f6e1e800640 1 --2- 172.21.15.32:0/3179772494 >> v2:172.21.15.32:3300/0 conn(0x7f6e1806ef60 0x7f6e1806f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.081+0000 7f6e1d400640 1 --2- 172.21.15.32:0/3179772494 >> v2:172.21.15.144:3300/0 conn(0x7f6e1815baa0 0x7f6e1815bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.081+0000 7f6e1ca00640 1 -- 172.21.15.32:0/3179772494 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4049898989 0 0) 0x7f6e18072bb0 con 0x7f6e18144e90 2024-04-17T09:55:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.081+0000 7f6e1ca00640 1 -- 172.21.15.32:0/3179772494 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e00005740 con 0x7f6e18144e90 2024-04-17T09:55:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.085+0000 7f6e1e800640 1 -- 172.21.15.32:0/3179772494 >> v2:172.21.15.144:3300/0 conn(0x7f6e1815baa0 msgr2=0x7f6e1815bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.085+0000 7f6e1e800640 1 --2- 172.21.15.32:0/3179772494 >> v2:172.21.15.144:3300/0 conn(0x7f6e1815baa0 0x7f6e1815bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.085+0000 7f6e1e800640 1 -- 172.21.15.32:0/3179772494 >> v1:172.21.15.32:6789/0 conn(0x7f6e18144e90 legacy=0x7f6e180b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.085+0000 7f6e1e800640 1 -- 172.21.15.32:0/3179772494 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e180b3dc0 con 0x7f6e1806ef60 2024-04-17T09:55:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.085+0000 7f6e1d400640 1 --2- 172.21.15.32:0/3179772494 >> v2:172.21.15.144:3300/0 conn(0x7f6e1815baa0 0x7f6e1815bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:01.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.085+0000 7f6e1e800640 1 --2- 172.21.15.32:0/3179772494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e1806ef60 0x7f6e1806f3b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6e1806fb40 tx=0x7f6e1401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd622b5a9a9ec50 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:01.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.085+0000 7f6e1ca00640 1 -- 172.21.15.32:0/3179772494 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e140b2070 con 0x7f6e1806ef60 2024-04-17T09:55:01.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.085+0000 7f6e1ca00640 1 -- 172.21.15.32:0/3179772494 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e140a58e0 con 0x7f6e1806ef60 2024-04-17T09:55:01.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.085+0000 7f6e1ca00640 1 -- 172.21.15.32:0/3179772494 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e140a5cb0 con 0x7f6e1806ef60 2024-04-17T09:55:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.085+0000 7f6e20800640 1 -- 172.21.15.32:0/3179772494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e1806ef60 msgr2=0x7f6e1806f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.085+0000 7f6e20800640 1 --2- 172.21.15.32:0/3179772494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e1806ef60 0x7f6e1806f3b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6e1806fb40 tx=0x7f6e1401c930 comp rx=0 tx=0).stop 2024-04-17T09:55:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.085+0000 7f6e20800640 1 -- 172.21.15.32:0/3179772494 shutdown_connections 2024-04-17T09:55:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.085+0000 7f6e20800640 1 --2- 172.21.15.32:0/3179772494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e1806ef60 0x7f6e1806f3b0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.085+0000 7f6e20800640 1 --2- 172.21.15.32:0/3179772494 >> v2:172.21.15.144:3300/0 conn(0x7f6e1815baa0 0x7f6e1815bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.085+0000 7f6e20800640 1 -- 172.21.15.32:0/3179772494 >> 172.21.15.32:0/3179772494 conn(0x7f6e1806ba30 msgr2=0x7f6e1806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.085+0000 7f6e20800640 1 -- 172.21.15.32:0/3179772494 shutdown_connections 2024-04-17T09:55:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.085+0000 7f6e20800640 1 -- 172.21.15.32:0/3179772494 wait complete. 2024-04-17T09:55:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e20800640 1 Processor -- start 2024-04-17T09:55:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e20800640 1 -- start start 2024-04-17T09:55:01.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e20800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e1806ef60 0x7f6e1815aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:01.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e20800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e18144e90 0x7f6e1813e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:01.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e20800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e1815baa0 0x7f6e1813e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:01.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e20800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6e180727f0 con 0x7f6e18144e90 2024-04-17T09:55:01.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e20800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6e18072670 con 0x7f6e1815baa0 2024-04-17T09:55:01.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e20800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6e18072970 con 0x7f6e1806ef60 2024-04-17T09:55:01.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e1d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e18144e90 0x7f6e1813e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:01.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e1d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e18144e90 0x7f6e1813e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53852/0 (socket says 172.21.15.32:53852) 2024-04-17T09:55:01.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e1d400640 1 -- 172.21.15.32:0/1125530151 learned_addr learned my addr 172.21.15.32:0/1125530151 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:01.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e1e800640 1 --2- 172.21.15.32:0/1125530151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e1815baa0 0x7f6e1813e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:01.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e1de00640 1 --2- 172.21.15.32:0/1125530151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e1806ef60 0x7f6e1815aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:01.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e1d400640 1 -- 172.21.15.32:0/1125530151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e1806ef60 msgr2=0x7f6e1815aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:01.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e1d400640 1 --2- 172.21.15.32:0/1125530151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e1806ef60 0x7f6e1815aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e1d400640 1 -- 172.21.15.32:0/1125530151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e1815baa0 msgr2=0x7f6e1813e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:01.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e1d400640 1 --2- 172.21.15.32:0/1125530151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e1815baa0 0x7f6e1813e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e1d400640 1 -- 172.21.15.32:0/1125530151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e18159a10 con 0x7f6e18144e90 2024-04-17T09:55:01.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e1d400640 1 --2- 172.21.15.32:0/1125530151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e18144e90 0x7f6e1813e0f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800d800 tx=0x7f6e0800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:01.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e12000640 1 -- 172.21.15.32:0/1125530151 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0800b6c0 con 0x7f6e18144e90 2024-04-17T09:55:01.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e12000640 1 -- 172.21.15.32:0/1125530151 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e08019070 con 0x7f6e18144e90 2024-04-17T09:55:01.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e12000640 1 -- 172.21.15.32:0/1125530151 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0800baa0 con 0x7f6e18144e90 2024-04-17T09:55:01.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e20800640 1 -- 172.21.15.32:0/1125530151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e18159cf0 con 0x7f6e18144e90 2024-04-17T09:55:01.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.089+0000 7f6e20800640 1 -- 172.21.15.32:0/1125530151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e1815a2b0 con 0x7f6e18144e90 2024-04-17T09:55:01.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.093+0000 7f6e20800640 1 -- 172.21.15.32:0/1125530151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e18068a90 con 0x7f6e18144e90 2024-04-17T09:55:01.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.093+0000 7f6e12000640 1 -- 172.21.15.32:0/1125530151 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6e0800bc40 con 0x7f6e18144e90 2024-04-17T09:55:01.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.093+0000 7f6e12000640 1 --2- 172.21.15.32:0/1125530151 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6dfc068380 0x7f6dfc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:01.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.093+0000 7f6e12000640 1 -- 172.21.15.32:0/1125530151 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f6e08086b10 con 0x7f6e18144e90 2024-04-17T09:55:01.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.093+0000 7f6e1de00640 1 --2- 172.21.15.32:0/1125530151 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6dfc068380 0x7f6dfc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:01.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.093+0000 7f6e1de00640 1 --2- 172.21.15.32:0/1125530151 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6dfc068380 0x7f6dfc06a840 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c005890 tx=0x7f6e0c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:01.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.097+0000 7f6e12000640 1 -- 172.21.15.32:0/1125530151 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f6e08057300 con 0x7f6e18144e90 2024-04-17T09:55:01.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.261+0000 7f6e20800640 1 -- 172.21.15.32:0/1125530151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6e180afc20 con 0x7f6e18144e90 2024-04-17T09:55:01.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.265+0000 7f6e12000640 1 -- 172.21.15.32:0/1125530151 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v42) ==== 84+0+65 (secure 0 0 0) 0x7f6e08056c00 con 0x7f6e18144e90 2024-04-17T09:55:01.265 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:01.265 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-17T09:55:01.265 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:01.265 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-04-17T09:55:01.265 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-04-17T09:55:01.265 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:01.265 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-17T09:55:01.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.265+0000 7f6e20800640 1 -- 172.21.15.32:0/1125530151 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6dfc068380 msgr2=0x7f6dfc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:01.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.265+0000 7f6e20800640 1 --2- 172.21.15.32:0/1125530151 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6dfc068380 0x7f6dfc06a840 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c005890 tx=0x7f6e0c009040 comp rx=0 tx=0).stop 2024-04-17T09:55:01.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.265+0000 7f6e20800640 1 -- 172.21.15.32:0/1125530151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e18144e90 msgr2=0x7f6e1813e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:01.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.265+0000 7f6e20800640 1 --2- 172.21.15.32:0/1125530151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e18144e90 0x7f6e1813e0f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800d800 tx=0x7f6e0800dcc0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.265+0000 7f6e20800640 1 -- 172.21.15.32:0/1125530151 shutdown_connections 2024-04-17T09:55:01.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.265+0000 7f6e20800640 1 --2- 172.21.15.32:0/1125530151 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6dfc068380 0x7f6dfc06a840 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.265+0000 7f6e20800640 1 --2- 172.21.15.32:0/1125530151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e1815baa0 0x7f6e1813e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.265+0000 7f6e20800640 1 --2- 172.21.15.32:0/1125530151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e18144e90 0x7f6e1813e0f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.265+0000 7f6e20800640 1 --2- 172.21.15.32:0/1125530151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e1806ef60 0x7f6e1815aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.265+0000 7f6e20800640 1 -- 172.21.15.32:0/1125530151 >> 172.21.15.32:0/1125530151 conn(0x7f6e1806ba30 msgr2=0x7f6e1806bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:01.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.265+0000 7f6e20800640 1 -- 172.21.15.32:0/1125530151 shutdown_connections 2024-04-17T09:55:01.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.265+0000 7f6e20800640 1 -- 172.21.15.32:0/1125530151 wait complete. 2024-04-17T09:55:01.285 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T09:55:01.286 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T09:55:01.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.429+0000 7f7ac3a00640 1 Processor -- start 2024-04-17T09:55:01.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.429+0000 7f7ac3a00640 1 -- start start 2024-04-17T09:55:01.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.433+0000 7f7ac3a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7abc0cfc70 0x7f7abc06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:01.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.433+0000 7f7ac3a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7abc0cd870 0x7f7abc15ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:01.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.433+0000 7f7ac3a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7abc15f200 con 0x7f7abc06f330 2024-04-17T09:55:01.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.433+0000 7f7ac3a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7abc15fb30 con 0x7f7abc0cfc70 2024-04-17T09:55:01.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.433+0000 7f7ac3a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f7abc06f770 con 0x7f7abc0cd870 2024-04-17T09:55:01.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.433+0000 7f7ac1000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7abc0cfc70 0x7f7abc06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:01.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.433+0000 7f7ac1000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7abc0cfc70 0x7f7abc06edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53862/0 (socket says 172.21.15.32:53862) 2024-04-17T09:55:01.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.433+0000 7f7ac1000640 1 -- 172.21.15.32:0/4258911703 learned_addr learned my addr 172.21.15.32:0/4258911703 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:01.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.433+0000 7f7ac1a00640 1 --2- 172.21.15.32:0/4258911703 >> v2:172.21.15.144:3300/0 conn(0x7f7abc0cd870 0x7f7abc15ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:01.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.433+0000 7f7ac1000640 1 -- 172.21.15.32:0/4258911703 >> v2:172.21.15.144:3300/0 conn(0x7f7abc0cd870 msgr2=0x7f7abc15ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:01.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.433+0000 7f7ac1000640 1 --2- 172.21.15.32:0/4258911703 >> v2:172.21.15.144:3300/0 conn(0x7f7abc0cd870 0x7f7abc15ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.433+0000 7f7ac1000640 1 -- 172.21.15.32:0/4258911703 >> v1:172.21.15.144:6789/0 conn(0x7f7abc06f330 legacy=0x7f7abc0cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:01.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.433+0000 7f7ac1000640 1 -- 172.21.15.32:0/4258911703 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7abc15f4e0 con 0x7f7abc0cfc70 2024-04-17T09:55:01.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.437+0000 7f7ac1000640 1 --2- 172.21.15.32:0/4258911703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7abc0cfc70 0x7f7abc06edf0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7abc0b2120 tx=0x7f7ab001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1311683b6b87f0b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:01.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.437+0000 7f7abb400640 1 -- 172.21.15.32:0/4258911703 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab00b0070 con 0x7f7abc0cfc70 2024-04-17T09:55:01.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.437+0000 7f7abb400640 1 -- 172.21.15.32:0/4258911703 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab00ab600 con 0x7f7abc0cfc70 2024-04-17T09:55:01.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.437+0000 7f7abb400640 1 -- 172.21.15.32:0/4258911703 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab00ab9d0 con 0x7f7abc0cfc70 2024-04-17T09:55:01.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.437+0000 7f7ac3a00640 1 -- 172.21.15.32:0/4258911703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7abc0cfc70 msgr2=0x7f7abc06edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:01.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.437+0000 7f7ac3a00640 1 --2- 172.21.15.32:0/4258911703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7abc0cfc70 0x7f7abc06edf0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7abc0b2120 tx=0x7f7ab001c520 comp rx=0 tx=0).stop 2024-04-17T09:55:01.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.437+0000 7f7ac3a00640 1 -- 172.21.15.32:0/4258911703 shutdown_connections 2024-04-17T09:55:01.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.437+0000 7f7ac3a00640 1 --2- 172.21.15.32:0/4258911703 >> v2:172.21.15.144:3300/0 conn(0x7f7abc0cd870 0x7f7abc15ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.437+0000 7f7ac3a00640 1 --2- 172.21.15.32:0/4258911703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7abc0cfc70 0x7f7abc06edf0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.437+0000 7f7ac3a00640 1 -- 172.21.15.32:0/4258911703 >> 172.21.15.32:0/4258911703 conn(0x7f7abc0b2190 msgr2=0x7f7abc0b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:01.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.437+0000 7f7ac3a00640 1 -- 172.21.15.32:0/4258911703 shutdown_connections 2024-04-17T09:55:01.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.437+0000 7f7ac3a00640 1 -- 172.21.15.32:0/4258911703 wait complete. 2024-04-17T09:55:01.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.437+0000 7f7ac3a00640 1 Processor -- start 2024-04-17T09:55:01.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.437+0000 7f7ac3a00640 1 -- start start 2024-04-17T09:55:01.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7ac3a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7abc06f330 0x7f7abc0af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:01.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7ac3a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7abc0cd870 0x7f7abc14f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:01.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7ac3a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7abc0cfc70 0x7f7abc151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:01.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7ac3a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc168330 con 0x7f7abc0cd870 2024-04-17T09:55:01.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7ac3a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc1681b0 con 0x7f7abc06f330 2024-04-17T09:55:01.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7ac3a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7abc1684b0 con 0x7f7abc0cfc70 2024-04-17T09:55:01.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7abbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7abc0cd870 0x7f7abc14f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:01.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7abbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7abc0cd870 0x7f7abc14f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53866/0 (socket says 172.21.15.32:53866) 2024-04-17T09:55:01.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7ac1000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7abc06f330 0x7f7abc0af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:01.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7abbe00640 1 -- 172.21.15.32:0/2649681538 learned_addr learned my addr 172.21.15.32:0/2649681538 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:01.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7ac1a00640 1 --2- 172.21.15.32:0/2649681538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7abc0cfc70 0x7f7abc151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:01.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7abbe00640 1 -- 172.21.15.32:0/2649681538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7abc0cfc70 msgr2=0x7f7abc151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:01.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7abbe00640 1 --2- 172.21.15.32:0/2649681538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7abc0cfc70 0x7f7abc151cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7abbe00640 1 -- 172.21.15.32:0/2649681538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7abc06f330 msgr2=0x7f7abc0af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:01.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7abbe00640 1 --2- 172.21.15.32:0/2649681538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7abc06f330 0x7f7abc0af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7abbe00640 1 -- 172.21.15.32:0/2649681538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7abc164400 con 0x7f7abc0cd870 2024-04-17T09:55:01.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7abbe00640 1 --2- 172.21.15.32:0/2649681538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7abc0cd870 0x7f7abc14f940 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800c3c0 tx=0x7f7aa800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:01.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7ab8c00640 1 -- 172.21.15.32:0/2649681538 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa80052d0 con 0x7f7abc0cd870 2024-04-17T09:55:01.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7ab8c00640 1 -- 172.21.15.32:0/2649681538 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa80054b0 con 0x7f7abc0cd870 2024-04-17T09:55:01.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7ac3a00640 1 -- 172.21.15.32:0/2649681538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7abc1646a0 con 0x7f7abc0cd870 2024-04-17T09:55:01.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7ab8c00640 1 -- 172.21.15.32:0/2649681538 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8006a50 con 0x7f7abc0cd870 2024-04-17T09:55:01.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7ac3a00640 1 -- 172.21.15.32:0/2649681538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7abc164c60 con 0x7f7abc0cd870 2024-04-17T09:55:01.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7ac3a00640 1 -- 172.21.15.32:0/2649681538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7abc164e60 con 0x7f7abc0cd870 2024-04-17T09:55:01.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.441+0000 7f7ab8c00640 1 -- 172.21.15.32:0/2649681538 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7aa8006cb0 con 0x7f7abc0cd870 2024-04-17T09:55:01.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.445+0000 7f7ab8c00640 1 --2- 172.21.15.32:0/2649681538 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7a9c068380 0x7f7a9c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:01.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.445+0000 7f7ab8c00640 1 -- 172.21.15.32:0/2649681538 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f7aa8085a70 con 0x7f7abc0cd870 2024-04-17T09:55:01.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.445+0000 7f7ac1000640 1 --2- 172.21.15.32:0/2649681538 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7a9c068380 0x7f7a9c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:01.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.445+0000 7f7ac1000640 1 --2- 172.21.15.32:0/2649681538 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7a9c068380 0x7f7a9c06a840 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00ad000 tx=0x7f7ab00a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:01.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.449+0000 7f7ab8c00640 1 -- 172.21.15.32:0/2649681538 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f7abc164e60 con 0x7f7abc0cd870 2024-04-17T09:55:01.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.613+0000 7f7ac3a00640 1 -- 172.21.15.32:0/2649681538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7abc14fee0 con 0x7f7abc0cd870 2024-04-17T09:55:01.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.813+0000 7f7ab8c00640 1 -- 172.21.15.32:0/2649681538 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v43) ==== 138+0+0 (secure 0 0 0) 0x7f7aa805c030 con 0x7f7abc0cd870 2024-04-17T09:55:01.814 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-04-17T09:55:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.817+0000 7f7ac3a00640 1 -- 172.21.15.32:0/2649681538 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7a9c068380 msgr2=0x7f7a9c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.817+0000 7f7ac3a00640 1 --2- 172.21.15.32:0/2649681538 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7a9c068380 0x7f7a9c06a840 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00ad000 tx=0x7f7ab00a6000 comp rx=0 tx=0).stop 2024-04-17T09:55:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.817+0000 7f7ac3a00640 1 -- 172.21.15.32:0/2649681538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7abc0cd870 msgr2=0x7f7abc14f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.817+0000 7f7ac3a00640 1 --2- 172.21.15.32:0/2649681538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7abc0cd870 0x7f7abc14f940 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800c3c0 tx=0x7f7aa800c880 comp rx=0 tx=0).stop 2024-04-17T09:55:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.817+0000 7f7ac3a00640 1 -- 172.21.15.32:0/2649681538 shutdown_connections 2024-04-17T09:55:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.817+0000 7f7ac3a00640 1 --2- 172.21.15.32:0/2649681538 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7a9c068380 0x7f7a9c06a840 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.817+0000 7f7ac3a00640 1 --2- 172.21.15.32:0/2649681538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7abc0cfc70 0x7f7abc151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.817+0000 7f7ac3a00640 1 --2- 172.21.15.32:0/2649681538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7abc0cd870 0x7f7abc14f940 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.817+0000 7f7ac3a00640 1 --2- 172.21.15.32:0/2649681538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7abc06f330 0x7f7abc0af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.817+0000 7f7ac3a00640 1 -- 172.21.15.32:0/2649681538 >> 172.21.15.32:0/2649681538 conn(0x7f7abc0b2190 msgr2=0x7f7abc06eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.817+0000 7f7ac3a00640 1 -- 172.21.15.32:0/2649681538 shutdown_connections 2024-04-17T09:55:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.817+0000 7f7ac3a00640 1 -- 172.21.15.32:0/2649681538 wait complete. 2024-04-17T09:55:01.842 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-17T09:55:01.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.989+0000 7f6597400640 1 Processor -- start 2024-04-17T09:55:01.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.989+0000 7f6597400640 1 -- start start 2024-04-17T09:55:01.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.989+0000 7f6597400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65900cf370 0x7f65900cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:01.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.989+0000 7f6597400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6590071e60 0x7f65900722d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:01.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.989+0000 7f6597400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65900d0130 con 0x7f659006ba30 2024-04-17T09:55:01.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.989+0000 7f6597400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f659015d090 con 0x7f6590071e60 2024-04-17T09:55:01.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.989+0000 7f6597400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f659015d370 con 0x7f65900cf370 2024-04-17T09:55:01.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.989+0000 7f6594a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f659006ba30 0x7f659006be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33092/0 (socket says 172.21.15.32:33092) 2024-04-17T09:55:01.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.989+0000 7f6594a00640 1 -- 172.21.15.32:0/444978937 learned_addr learned my addr 172.21.15.32:0/444978937 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:01.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f6595400640 1 --2- 172.21.15.32:0/444978937 >> v2:172.21.15.144:3300/0 conn(0x7f6590071e60 0x7f65900722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:01.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f6587e00640 1 --2- 172.21.15.32:0/444978937 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65900cf370 0x7f65900cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:01.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f658fe00640 1 -- 172.21.15.32:0/444978937 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1736490249 0 0) 0x7f65900d0130 con 0x7f659006ba30 2024-04-17T09:55:01.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f658fe00640 1 -- 172.21.15.32:0/444978937 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6574003530 con 0x7f659006ba30 2024-04-17T09:55:01.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f658fe00640 1 -- 172.21.15.32:0/444978937 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 949976774 0 0) 0x7f6574003530 con 0x7f659006ba30 2024-04-17T09:55:01.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f658fe00640 1 -- 172.21.15.32:0/444978937 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65900cf370 msgr2=0x7f65900cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:01.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f658fe00640 1 --2- 172.21.15.32:0/444978937 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65900cf370 0x7f65900cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f658fe00640 1 -- 172.21.15.32:0/444978937 >> v2:172.21.15.144:3300/0 conn(0x7f6590071e60 msgr2=0x7f65900722d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:01.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f658fe00640 1 --2- 172.21.15.32:0/444978937 >> v2:172.21.15.144:3300/0 conn(0x7f6590071e60 0x7f65900722d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f658fe00640 1 -- 172.21.15.32:0/444978937 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659015d650 con 0x7f659006ba30 2024-04-17T09:55:01.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f658fe00640 1 -- 172.21.15.32:0/444978937 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f6580001680 con 0x7f659006ba30 2024-04-17T09:55:01.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f6595400640 1 --2- 172.21.15.32:0/444978937 >> v2:172.21.15.144:3300/0 conn(0x7f6590071e60 0x7f65900722d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:55:01.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f658fe00640 1 -- 172.21.15.32:0/444978937 >> v1:172.21.15.32:6789/0 conn(0x7f659006ba30 legacy=0x7f659006be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:01.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f658fe00640 1 --2- 172.21.15.32:0/444978937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65740039c0 0x7f6574003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:01.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f658fe00640 1 --2- 172.21.15.32:0/444978937 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65900cf370 0x7f6574004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:01.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f658fe00640 1 --2- 172.21.15.32:0/444978937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6574004d50 0x7f65740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:01.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f658fe00640 1 -- 172.21.15.32:0/444978937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f659015d090 con 0x7f6574004d50 2024-04-17T09:55:01.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f658fe00640 1 -- 172.21.15.32:0/444978937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f659015d370 con 0x7f65740039c0 2024-04-17T09:55:01.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f658fe00640 1 -- 172.21.15.32:0/444978937 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f657400b9d0 con 0x7f65900cf370 2024-04-17T09:55:01.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f6595400640 1 --2- 172.21.15.32:0/444978937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6574004d50 0x7f65740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:01.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f6587e00640 1 --2- 172.21.15.32:0/444978937 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65900cf370 0x7f6574004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:01.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f6594a00640 1 --2- 172.21.15.32:0/444978937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65740039c0 0x7f6574003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:01.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f6587e00640 1 -- 172.21.15.32:0/444978937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65740039c0 msgr2=0x7f6574003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:01.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f6587e00640 1 --2- 172.21.15.32:0/444978937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65740039c0 0x7f6574003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f6587e00640 1 -- 172.21.15.32:0/444978937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6574004d50 msgr2=0x7f65740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:01.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f6587e00640 1 --2- 172.21.15.32:0/444978937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6574004d50 0x7f65740051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.993+0000 7f6587e00640 1 -- 172.21.15.32:0/444978937 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6574003530 con 0x7f65900cf370 2024-04-17T09:55:01.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.997+0000 7f6587e00640 1 --2- 172.21.15.32:0/444978937 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65900cf370 0x7f6574004810 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6578009ae0 tx=0x7f657801cab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d283b5693fead56c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:01.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.997+0000 7f658fe00640 1 -- 172.21.15.32:0/444978937 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65780b2070 con 0x7f65900cf370 2024-04-17T09:55:01.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.997+0000 7f658fe00640 1 -- 172.21.15.32:0/444978937 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65780a56a0 con 0x7f65900cf370 2024-04-17T09:55:01.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.997+0000 7f658fe00640 1 -- 172.21.15.32:0/444978937 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65780096b0 con 0x7f65900cf370 2024-04-17T09:55:01.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.997+0000 7f6597400640 1 -- 172.21.15.32:0/444978937 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65900cf370 msgr2=0x7f6574004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:01.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.997+0000 7f6597400640 1 --2- 172.21.15.32:0/444978937 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65900cf370 0x7f6574004810 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6578009ae0 tx=0x7f657801cab0 comp rx=0 tx=0).stop 2024-04-17T09:55:01.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.997+0000 7f6595400640 1 -- 172.21.15.32:0/444978937 reap_dead start 2024-04-17T09:55:01.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.997+0000 7f6597400640 1 -- 172.21.15.32:0/444978937 shutdown_connections 2024-04-17T09:55:01.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.997+0000 7f6597400640 1 -- 172.21.15.32:0/444978937 >> 172.21.15.32:0/444978937 conn(0x7f65900b2120 msgr2=0x7f65900b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:01.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.997+0000 7f6597400640 1 -- 172.21.15.32:0/444978937 shutdown_connections 2024-04-17T09:55:01.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.997+0000 7f6597400640 1 -- 172.21.15.32:0/444978937 wait complete. 2024-04-17T09:55:01.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.997+0000 7f6597400640 1 Processor -- start 2024-04-17T09:55:01.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:01.997+0000 7f6597400640 1 -- start start 2024-04-17T09:55:02.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f6597400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65900cf370 0x7f659013d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:02.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f6597400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6590159ab0 0x7f65901591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:02.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f6597400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659013ecd0 0x7f659013f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:02.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f6597400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f659015dcf0 con 0x7f659013ecd0 2024-04-17T09:55:02.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f6597400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f659015db70 con 0x7f6590159ab0 2024-04-17T09:55:02.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f6597400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6590146230 con 0x7f65900cf370 2024-04-17T09:55:02.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f6595400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659013ecd0 0x7f659013f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:02.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f6595400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659013ecd0 0x7f659013f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53878/0 (socket says 172.21.15.32:53878) 2024-04-17T09:55:02.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f6595400640 1 -- 172.21.15.32:0/3281905757 learned_addr learned my addr 172.21.15.32:0/3281905757 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:02.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f6594a00640 1 --2- 172.21.15.32:0/3281905757 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65900cf370 0x7f659013d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:02.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f6587e00640 1 --2- 172.21.15.32:0/3281905757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6590159ab0 0x7f65901591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:02.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f6595400640 1 -- 172.21.15.32:0/3281905757 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65900cf370 msgr2=0x7f659013d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:02.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f6595400640 1 --2- 172.21.15.32:0/3281905757 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65900cf370 0x7f659013d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:02.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f6595400640 1 -- 172.21.15.32:0/3281905757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6590159ab0 msgr2=0x7f65901591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:02.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f6595400640 1 --2- 172.21.15.32:0/3281905757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6590159ab0 0x7f65901591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:02.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f6595400640 1 -- 172.21.15.32:0/3281905757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659013de80 con 0x7f659013ecd0 2024-04-17T09:55:02.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f6595400640 1 --2- 172.21.15.32:0/3281905757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659013ecd0 0x7f659013f150 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f659006d7b0 tx=0x7f658800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:02.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f658d600640 1 -- 172.21.15.32:0/3281905757 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658800fdf0 con 0x7f659013ecd0 2024-04-17T09:55:02.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f658d600640 1 -- 172.21.15.32:0/3281905757 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65880039a0 con 0x7f659013ecd0 2024-04-17T09:55:02.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f658d600640 1 -- 172.21.15.32:0/3281905757 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588003d90 con 0x7f659013ecd0 2024-04-17T09:55:02.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f6597400640 1 -- 172.21.15.32:0/3281905757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659013e160 con 0x7f659013ecd0 2024-04-17T09:55:02.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f6597400640 1 -- 172.21.15.32:0/3281905757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f659013e720 con 0x7f659013ecd0 2024-04-17T09:55:02.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f6597400640 1 -- 172.21.15.32:0/3281905757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6590068a90 con 0x7f659013ecd0 2024-04-17T09:55:02.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.001+0000 7f658d600640 1 -- 172.21.15.32:0/3281905757 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f658801f020 con 0x7f659013ecd0 2024-04-17T09:55:02.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.005+0000 7f658d600640 1 --2- 172.21.15.32:0/3281905757 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f656c0683b0 0x7f656c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:02.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.005+0000 7f658d600640 1 -- 172.21.15.32:0/3281905757 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5227+0+0 (secure 0 0 0) 0x7f65880876a0 con 0x7f659013ecd0 2024-04-17T09:55:02.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.005+0000 7f6594a00640 1 --2- 172.21.15.32:0/3281905757 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f656c0683b0 0x7f656c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:02.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.005+0000 7f6594a00640 1 --2- 172.21.15.32:0/3281905757 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f656c0683b0 0x7f656c06a870 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6580001560 tx=0x7f6580007900 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:02.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.009+0000 7f658d600640 1 -- 172.21.15.32:0/3281905757 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f6588010040 con 0x7f659013ecd0 2024-04-17T09:55:02.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.173+0000 7f6597400640 1 -- 172.21.15.32:0/3281905757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f659006c850 con 0x7f659013ecd0 2024-04-17T09:55:02.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.821+0000 7f658d600640 1 -- 172.21.15.32:0/3281905757 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v44) ==== 170+0+0 (secure 0 0 0) 0x7f6588057e20 con 0x7f659013ecd0 2024-04-17T09:55:02.822 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-04-17T09:55:02.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.825+0000 7f6597400640 1 -- 172.21.15.32:0/3281905757 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f656c0683b0 msgr2=0x7f656c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:02.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.825+0000 7f6597400640 1 --2- 172.21.15.32:0/3281905757 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f656c0683b0 0x7f656c06a870 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6580001560 tx=0x7f6580007900 comp rx=0 tx=0).stop 2024-04-17T09:55:02.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.825+0000 7f6597400640 1 -- 172.21.15.32:0/3281905757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659013ecd0 msgr2=0x7f659013f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:02.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.825+0000 7f6597400640 1 --2- 172.21.15.32:0/3281905757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659013ecd0 0x7f659013f150 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f659006d7b0 tx=0x7f658800f580 comp rx=0 tx=0).stop 2024-04-17T09:55:02.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.825+0000 7f6597400640 1 -- 172.21.15.32:0/3281905757 shutdown_connections 2024-04-17T09:55:02.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.825+0000 7f6597400640 1 --2- 172.21.15.32:0/3281905757 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f656c0683b0 0x7f656c06a870 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:02.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.825+0000 7f6597400640 1 --2- 172.21.15.32:0/3281905757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f659013ecd0 0x7f659013f150 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:02.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.825+0000 7f6597400640 1 --2- 172.21.15.32:0/3281905757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6590159ab0 0x7f65901591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:02.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.825+0000 7f6597400640 1 --2- 172.21.15.32:0/3281905757 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65900cf370 0x7f659013d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:02.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.825+0000 7f6597400640 1 -- 172.21.15.32:0/3281905757 >> 172.21.15.32:0/3281905757 conn(0x7f65900b2120 msgr2=0x7f659006e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:02.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.825+0000 7f6597400640 1 -- 172.21.15.32:0/3281905757 shutdown_connections 2024-04-17T09:55:02.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:02.825+0000 7f6597400640 1 -- 172.21.15.32:0/3281905757 wait complete. 2024-04-17T09:55:02.866 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T09:55:03.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.009+0000 7fe197400640 1 Processor -- start 2024-04-17T09:55:03.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.009+0000 7fe197400640 1 -- start start 2024-04-17T09:55:03.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.009+0000 7fe197400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe19015eae0 0x7fe1900d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:03.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.013+0000 7fe197400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe1900d1750 0x7fe1900ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:03.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.013+0000 7fe197400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe19009ac40 con 0x7fe19006ce50 2024-04-17T09:55:03.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.013+0000 7fe197400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe19009a9f0 con 0x7fe1900d1750 2024-04-17T09:55:03.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.013+0000 7fe197400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe19009b570 con 0x7fe19015eae0 2024-04-17T09:55:03.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.013+0000 7fe195400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe1900d1750 0x7fe1900ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:03.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.013+0000 7fe187e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe19006ce50 0x7fe19006d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33104/0 (socket says 172.21.15.32:33104) 2024-04-17T09:55:03.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.013+0000 7fe187e00640 1 -- 172.21.15.32:0/2094590870 learned_addr learned my addr 172.21.15.32:0/2094590870 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:03.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.013+0000 7fe194a00640 1 --2- 172.21.15.32:0/2094590870 >> v2:172.21.15.144:3300/0 conn(0x7fe19015eae0 0x7fe1900d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:03.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.013+0000 7fe195400640 1 -- 172.21.15.32:0/2094590870 >> v2:172.21.15.144:3300/0 conn(0x7fe19015eae0 msgr2=0x7fe1900d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:03.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.013+0000 7fe195400640 1 --2- 172.21.15.32:0/2094590870 >> v2:172.21.15.144:3300/0 conn(0x7fe19015eae0 0x7fe1900d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:03.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.013+0000 7fe195400640 1 -- 172.21.15.32:0/2094590870 >> v1:172.21.15.32:6789/0 conn(0x7fe19006ce50 legacy=0x7fe19006d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:03.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.013+0000 7fe195400640 1 -- 172.21.15.32:0/2094590870 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19009af20 con 0x7fe1900d1750 2024-04-17T09:55:03.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.013+0000 7fe195400640 1 --2- 172.21.15.32:0/2094590870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1900d1750 0x7fe1900ccbe0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe1900732f0 tx=0x7fe18801c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8d1a99e169ef2f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:03.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.013+0000 7fe18fe00640 1 -- 172.21.15.32:0/2094590870 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1880b2070 con 0x7fe1900d1750 2024-04-17T09:55:03.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.017+0000 7fe18fe00640 1 -- 172.21.15.32:0/2094590870 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1880a58c0 con 0x7fe1900d1750 2024-04-17T09:55:03.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.017+0000 7fe18fe00640 1 -- 172.21.15.32:0/2094590870 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1880a5c90 con 0x7fe1900d1750 2024-04-17T09:55:03.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.017+0000 7fe197400640 1 -- 172.21.15.32:0/2094590870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1900d1750 msgr2=0x7fe1900ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:03.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.017+0000 7fe197400640 1 --2- 172.21.15.32:0/2094590870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1900d1750 0x7fe1900ccbe0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe1900732f0 tx=0x7fe18801c7e0 comp rx=0 tx=0).stop 2024-04-17T09:55:03.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.017+0000 7fe197400640 1 -- 172.21.15.32:0/2094590870 shutdown_connections 2024-04-17T09:55:03.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.017+0000 7fe197400640 1 --2- 172.21.15.32:0/2094590870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1900d1750 0x7fe1900ccbe0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:03.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.017+0000 7fe197400640 1 --2- 172.21.15.32:0/2094590870 >> v2:172.21.15.144:3300/0 conn(0x7fe19015eae0 0x7fe1900d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:03.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.017+0000 7fe197400640 1 -- 172.21.15.32:0/2094590870 >> 172.21.15.32:0/2094590870 conn(0x7fe1900b4080 msgr2=0x7fe1900b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:03.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.017+0000 7fe197400640 1 -- 172.21.15.32:0/2094590870 shutdown_connections 2024-04-17T09:55:03.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.017+0000 7fe197400640 1 -- 172.21.15.32:0/2094590870 wait complete. 2024-04-17T09:55:03.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.017+0000 7fe197400640 1 Processor -- start 2024-04-17T09:55:03.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.017+0000 7fe197400640 1 -- start start 2024-04-17T09:55:03.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.017+0000 7fe197400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe19006ce50 0x7fe190154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:03.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.017+0000 7fe197400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1900d1750 0x7fe1900c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:03.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.017+0000 7fe197400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe19015eae0 0x7fe1900cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:03.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.017+0000 7fe197400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe19009bb60 con 0x7fe19006ce50 2024-04-17T09:55:03.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.017+0000 7fe197400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe19009b9e0 con 0x7fe1900d1750 2024-04-17T09:55:03.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.017+0000 7fe197400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe19009bce0 con 0x7fe19015eae0 2024-04-17T09:55:03.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.017+0000 7fe194a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe19006ce50 0x7fe190154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:03.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.017+0000 7fe194a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe19006ce50 0x7fe190154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53896/0 (socket says 172.21.15.32:53896) 2024-04-17T09:55:03.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.017+0000 7fe194a00640 1 -- 172.21.15.32:0/1844785366 learned_addr learned my addr 172.21.15.32:0/1844785366 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:03.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.021+0000 7fe195400640 1 --2- 172.21.15.32:0/1844785366 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe19015eae0 0x7fe1900cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:03.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.021+0000 7fe194a00640 1 -- 172.21.15.32:0/1844785366 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe19015eae0 msgr2=0x7fe1900cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:03.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.021+0000 7fe187e00640 1 --2- 172.21.15.32:0/1844785366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1900d1750 0x7fe1900c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:03.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.021+0000 7fe194a00640 1 --2- 172.21.15.32:0/1844785366 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe19015eae0 0x7fe1900cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:03.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.021+0000 7fe194a00640 1 -- 172.21.15.32:0/1844785366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1900d1750 msgr2=0x7fe1900c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:03.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.021+0000 7fe194a00640 1 --2- 172.21.15.32:0/1844785366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1900d1750 0x7fe1900c9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:03.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.021+0000 7fe194a00640 1 -- 172.21.15.32:0/1844785366 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19015f6b0 con 0x7fe19006ce50 2024-04-17T09:55:03.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.021+0000 7fe195400640 1 --2- 172.21.15.32:0/1844785366 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe19015eae0 0x7fe1900cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:55:03.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.021+0000 7fe187e00640 1 --2- 172.21.15.32:0/1844785366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1900d1750 0x7fe1900c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:55:03.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.021+0000 7fe194a00640 1 --2- 172.21.15.32:0/1844785366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe19006ce50 0x7fe190154d60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe18000e930 tx=0x7fe18000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:03.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.021+0000 7fe18d600640 1 -- 172.21.15.32:0/1844785366 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180017070 con 0x7fe19006ce50 2024-04-17T09:55:03.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.021+0000 7fe197400640 1 -- 172.21.15.32:0/1844785366 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe190161ba0 con 0x7fe19006ce50 2024-04-17T09:55:03.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.021+0000 7fe18d600640 1 -- 172.21.15.32:0/1844785366 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe18000ca00 con 0x7fe19006ce50 2024-04-17T09:55:03.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.021+0000 7fe18d600640 1 -- 172.21.15.32:0/1844785366 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18000cdb0 con 0x7fe19006ce50 2024-04-17T09:55:03.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.021+0000 7fe197400640 1 -- 172.21.15.32:0/1844785366 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe19015f990 con 0x7fe19006ce50 2024-04-17T09:55:03.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.021+0000 7fe197400640 1 -- 172.21.15.32:0/1844785366 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe190068a90 con 0x7fe19006ce50 2024-04-17T09:55:03.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.021+0000 7fe18d600640 1 -- 172.21.15.32:0/1844785366 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe18001a070 con 0x7fe19006ce50 2024-04-17T09:55:03.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.021+0000 7fe18d600640 1 --2- 172.21.15.32:0/1844785366 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe16c068380 0x7fe16c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:03.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.021+0000 7fe18d600640 1 -- 172.21.15.32:0/1844785366 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5609+0+0 (secure 0 0 0) 0x7fe180087130 con 0x7fe19006ce50 2024-04-17T09:55:03.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.025+0000 7fe187e00640 1 --2- 172.21.15.32:0/1844785366 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe16c068380 0x7fe16c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:03.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.025+0000 7fe187e00640 1 --2- 172.21.15.32:0/1844785366 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe16c068380 0x7fe16c06a840 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe1901574f0 tx=0x7fe178009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:03.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.025+0000 7fe18d600640 1 -- 172.21.15.32:0/1844785366 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fe180057880 con 0x7fe19006ce50 2024-04-17T09:55:03.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.229+0000 7fe197400640 1 -- 172.21.15.32:0/1844785366 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe1900b8510 con 0x7fe19006ce50 2024-04-17T09:55:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 6 and data pool 7 2024-04-17T09:55:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.837+0000 7fe18d600640 1 -- 172.21.15.32:0/1844785366 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v20) ==== 172+0+0 (secure 0 0 0) 0x7fe180057180 con 0x7fe19006ce50 2024-04-17T09:55:03.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.841+0000 7fe197400640 1 -- 172.21.15.32:0/1844785366 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe16c068380 msgr2=0x7fe16c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:03.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.841+0000 7fe197400640 1 --2- 172.21.15.32:0/1844785366 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe16c068380 0x7fe16c06a840 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe1901574f0 tx=0x7fe178009040 comp rx=0 tx=0).stop 2024-04-17T09:55:03.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.845+0000 7fe197400640 1 -- 172.21.15.32:0/1844785366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe19006ce50 msgr2=0x7fe190154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:03.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.845+0000 7fe197400640 1 --2- 172.21.15.32:0/1844785366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe19006ce50 0x7fe190154d60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe18000e930 tx=0x7fe18000edf0 comp rx=0 tx=0).stop 2024-04-17T09:55:03.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.849+0000 7fe197400640 1 -- 172.21.15.32:0/1844785366 shutdown_connections 2024-04-17T09:55:03.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.849+0000 7fe197400640 1 --2- 172.21.15.32:0/1844785366 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe16c068380 0x7fe16c06a840 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:03.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.849+0000 7fe197400640 1 --2- 172.21.15.32:0/1844785366 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe19015eae0 0x7fe1900cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:03.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.849+0000 7fe197400640 1 --2- 172.21.15.32:0/1844785366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1900d1750 0x7fe1900c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:03.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.849+0000 7fe197400640 1 --2- 172.21.15.32:0/1844785366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe19006ce50 0x7fe190154d60 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:03.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.849+0000 7fe197400640 1 -- 172.21.15.32:0/1844785366 >> 172.21.15.32:0/1844785366 conn(0x7fe1900b4080 msgr2=0x7fe1900b5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:03.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.857+0000 7fe197400640 1 -- 172.21.15.32:0/1844785366 shutdown_connections 2024-04-17T09:55:03.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:03.857+0000 7fe197400640 1 -- 172.21.15.32:0/1844785366 wait complete. 2024-04-17T09:55:03.885 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:55:04.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.025+0000 7f3a16800640 1 Processor -- start 2024-04-17T09:55:04.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.025+0000 7f3a16800640 1 -- start start 2024-04-17T09:55:04.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.025+0000 7f3a16800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3a100d0890 0x7f3a1009a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.025+0000 7f3a16800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a1015ef10 con 0x7f3a1009af30 2024-04-17T09:55:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.025+0000 7f3a16800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a100d0db0 con 0x7f3a1015eb30 2024-04-17T09:55:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.025+0000 7f3a16800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f3a1009b3a0 con 0x7f3a100d0890 2024-04-17T09:55:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.025+0000 7f3a0fe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3a1009af30 0x7f3a100ccb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33110/0 (socket says 172.21.15.32:33110) 2024-04-17T09:55:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.025+0000 7f3a0fe00640 1 -- 172.21.15.32:0/1295091545 learned_addr learned my addr 172.21.15.32:0/1295091545 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.025+0000 7f3a07e00640 1 --2- 172.21.15.32:0/1295091545 >> v2:172.21.15.144:3300/0 conn(0x7f3a100d0890 0x7f3a1009a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.029+0000 7f3a0ea00640 1 -- 172.21.15.32:0/1295091545 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 908655538 0 0) 0x7f3a1015ef10 con 0x7f3a1009af30 2024-04-17T09:55:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.029+0000 7f3a0ea00640 1 -- 172.21.15.32:0/1295091545 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39f4003530 con 0x7f3a1009af30 2024-04-17T09:55:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.029+0000 7f3a07e00640 1 -- 172.21.15.32:0/1295091545 >> v1:172.21.15.144:6789/0 conn(0x7f3a1015eb30 legacy=0x7f3a1009b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.029+0000 7f3a07e00640 1 -- 172.21.15.32:0/1295091545 >> v1:172.21.15.32:6789/0 conn(0x7f3a1009af30 legacy=0x7f3a100ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.029+0000 7f3a07e00640 1 -- 172.21.15.32:0/1295091545 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a100d1090 con 0x7f3a100d0890 2024-04-17T09:55:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.029+0000 7f3a07e00640 1 --2- 172.21.15.32:0/1295091545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a100d0890 0x7f3a1009a9f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f39f80097c0 tx=0x7f39f801ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e685c55d16c6c5b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.029+0000 7f3a0ea00640 1 -- 172.21.15.32:0/1295091545 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f80b5070 con 0x7f3a100d0890 2024-04-17T09:55:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.029+0000 7f3a0ea00640 1 -- 172.21.15.32:0/1295091545 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39f80a8990 con 0x7f3a100d0890 2024-04-17T09:55:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.029+0000 7f3a0ea00640 1 -- 172.21.15.32:0/1295091545 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f80a8d40 con 0x7f3a100d0890 2024-04-17T09:55:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.029+0000 7f3a16800640 1 -- 172.21.15.32:0/1295091545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a100d0890 msgr2=0x7f3a1009a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.029+0000 7f3a16800640 1 --2- 172.21.15.32:0/1295091545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a100d0890 0x7f3a1009a9f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f39f80097c0 tx=0x7f39f801ec90 comp rx=0 tx=0).stop 2024-04-17T09:55:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.029+0000 7f3a16800640 1 -- 172.21.15.32:0/1295091545 shutdown_connections 2024-04-17T09:55:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.029+0000 7f3a16800640 1 --2- 172.21.15.32:0/1295091545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a100d0890 0x7f3a1009a9f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.029+0000 7f3a16800640 1 -- 172.21.15.32:0/1295091545 >> 172.21.15.32:0/1295091545 conn(0x7f3a100b3fd0 msgr2=0x7f3a100b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.033+0000 7f3a16800640 1 -- 172.21.15.32:0/1295091545 shutdown_connections 2024-04-17T09:55:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.033+0000 7f3a16800640 1 -- 172.21.15.32:0/1295091545 wait complete. 2024-04-17T09:55:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.033+0000 7f3a16800640 1 Processor -- start 2024-04-17T09:55:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.033+0000 7f3a16800640 1 -- start start 2024-04-17T09:55:04.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.033+0000 7f3a16800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a1009af30 0x7f3a100b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:04.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.033+0000 7f3a16800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a100d0890 0x7f3a101602a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:04.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.033+0000 7f3a16800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a1015eb30 0x7f3a101607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:04.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.033+0000 7f3a16800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3a100d1650 con 0x7f3a100d0890 2024-04-17T09:55:04.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.033+0000 7f3a16800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1006dd60 con 0x7f3a1015eb30 2024-04-17T09:55:04.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.033+0000 7f3a16800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3a100d1930 con 0x7f3a1009af30 2024-04-17T09:55:04.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.033+0000 7f3a07e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a100d0890 0x7f3a101602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:04.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.033+0000 7f3a07e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a100d0890 0x7f3a101602a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53910/0 (socket says 172.21.15.32:53910) 2024-04-17T09:55:04.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.033+0000 7f3a07e00640 1 -- 172.21.15.32:0/1284531181 learned_addr learned my addr 172.21.15.32:0/1284531181 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:04.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.033+0000 7f3a0f400640 1 --2- 172.21.15.32:0/1284531181 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a1009af30 0x7f3a100b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:04.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.033+0000 7f3a0fe00640 1 --2- 172.21.15.32:0/1284531181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a1015eb30 0x7f3a101607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:04.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.033+0000 7f3a07e00640 1 -- 172.21.15.32:0/1284531181 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a1009af30 msgr2=0x7f3a100b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:04.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.033+0000 7f3a07e00640 1 --2- 172.21.15.32:0/1284531181 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a1009af30 0x7f3a100b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.033+0000 7f3a07e00640 1 -- 172.21.15.32:0/1284531181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a1015eb30 msgr2=0x7f3a101607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:04.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.033+0000 7f3a07e00640 1 --2- 172.21.15.32:0/1284531181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a1015eb30 0x7f3a101607e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.033+0000 7f3a07e00640 1 -- 172.21.15.32:0/1284531181 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a10151040 con 0x7f3a100d0890 2024-04-17T09:55:04.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.033+0000 7f3a07e00640 1 --2- 172.21.15.32:0/1284531181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a100d0890 0x7f3a101602a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f39f80b3040 tx=0x7f39f801ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:04.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.037+0000 7f3a07400640 1 -- 172.21.15.32:0/1284531181 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f80c0070 con 0x7f3a100d0890 2024-04-17T09:55:04.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.037+0000 7f3a07400640 1 -- 172.21.15.32:0/1284531181 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39f8004230 con 0x7f3a100d0890 2024-04-17T09:55:04.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.037+0000 7f3a07400640 1 -- 172.21.15.32:0/1284531181 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f80b5040 con 0x7f3a100d0890 2024-04-17T09:55:04.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.037+0000 7f3a16800640 1 -- 172.21.15.32:0/1284531181 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a10150280 con 0x7f3a100d0890 2024-04-17T09:55:04.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.037+0000 7f3a16800640 1 -- 172.21.15.32:0/1284531181 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a10150840 con 0x7f3a100d0890 2024-04-17T09:55:04.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.037+0000 7f3a16800640 1 -- 172.21.15.32:0/1284531181 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a10068a90 con 0x7f3a100d0890 2024-04-17T09:55:04.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.037+0000 7f3a07400640 1 -- 172.21.15.32:0/1284531181 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f39f8003620 con 0x7f3a100d0890 2024-04-17T09:55:04.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.037+0000 7f3a07400640 1 --2- 172.21.15.32:0/1284531181 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f39ec068380 0x7f39ec06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:04.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.037+0000 7f3a07400640 1 -- 172.21.15.32:0/1284531181 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f39f81232a0 con 0x7f3a100d0890 2024-04-17T09:55:04.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.037+0000 7f3a0f400640 1 --2- 172.21.15.32:0/1284531181 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f39ec068380 0x7f39ec06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:04.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.037+0000 7f3a0f400640 1 --2- 172.21.15.32:0/1284531181 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f39ec068380 0x7f39ec06a840 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3a00001770 tx=0x7f3a00009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:04.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.041+0000 7f3a07400640 1 -- 172.21.15.32:0/1284531181 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f39f80187f0 con 0x7f3a100d0890 2024-04-17T09:55:04.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.209+0000 7f3a16800640 1 -- 172.21.15.32:0/1284531181 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a100ce9a0 con 0x7f3a100d0890 2024-04-17T09:55:04.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.209+0000 7f3a07400640 1 -- 172.21.15.32:0/1284531181 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16638 (secure 0 0 0) 0x7f39f80f3170 con 0x7f3a100d0890 2024-04-17T09:55:04.210 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:04.211 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":45,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:55:03.233586+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:55:01.624737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-17T09:55:02.185212+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:55:04.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.209+0000 7f3a16800640 1 -- 172.21.15.32:0/1284531181 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f39ec068380 msgr2=0x7f39ec06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:04.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.209+0000 7f3a16800640 1 --2- 172.21.15.32:0/1284531181 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f39ec068380 0x7f39ec06a840 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3a00001770 tx=0x7f3a00009040 comp rx=0 tx=0).stop 2024-04-17T09:55:04.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.209+0000 7f3a16800640 1 -- 172.21.15.32:0/1284531181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a100d0890 msgr2=0x7f3a101602a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:04.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.209+0000 7f3a16800640 1 --2- 172.21.15.32:0/1284531181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a100d0890 0x7f3a101602a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f39f80b3040 tx=0x7f39f801ba30 comp rx=0 tx=0).stop 2024-04-17T09:55:04.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.213+0000 7f3a16800640 1 -- 172.21.15.32:0/1284531181 shutdown_connections 2024-04-17T09:55:04.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.213+0000 7f3a16800640 1 --2- 172.21.15.32:0/1284531181 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f39ec068380 0x7f39ec06a840 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.213+0000 7f3a16800640 1 --2- 172.21.15.32:0/1284531181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a1015eb30 0x7f3a101607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.213+0000 7f3a16800640 1 --2- 172.21.15.32:0/1284531181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a100d0890 0x7f3a101602a0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.213+0000 7f3a16800640 1 --2- 172.21.15.32:0/1284531181 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a1009af30 0x7f3a100b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.213+0000 7f3a16800640 1 -- 172.21.15.32:0/1284531181 >> 172.21.15.32:0/1284531181 conn(0x7f3a100b3fd0 msgr2=0x7f3a1015d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:04.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.213+0000 7f3a16800640 1 -- 172.21.15.32:0/1284531181 shutdown_connections 2024-04-17T09:55:04.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.213+0000 7f3a16800640 1 -- 172.21.15.32:0/1284531181 wait complete. 2024-04-17T09:55:04.232 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:55:04.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.373+0000 7ff1f8a00640 1 Processor -- start 2024-04-17T09:55:04.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.373+0000 7ff1f8a00640 1 -- start start 2024-04-17T09:55:04.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.373+0000 7ff1f8a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff1f415eb30 0x7ff1f409b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:04.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.373+0000 7ff1f8a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1f40d0890 0x7ff1f409a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:04.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.373+0000 7ff1f8a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff1f409af30 0x7ff1f4162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:04.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.373+0000 7ff1f8a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff1f406dd60 con 0x7ff1f415eb30 2024-04-17T09:55:04.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.373+0000 7ff1f8a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff1f409bcb0 con 0x7ff1f409af30 2024-04-17T09:55:04.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.373+0000 7ff1f8a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f415ef10 con 0x7ff1f40d0890 2024-04-17T09:55:04.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.373+0000 7ff1f1e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff1f415eb30 0x7ff1f409b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:04.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.373+0000 7ff1f1e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff1f415eb30 0x7ff1f409b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53916/0 (socket says 172.21.15.32:53916) 2024-04-17T09:55:04.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.373+0000 7ff1f1e00640 1 -- 172.21.15.32:0/2615942044 learned_addr learned my addr 172.21.15.32:0/2615942044 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:04.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.373+0000 7ff1e9400640 1 --2- 172.21.15.32:0/2615942044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1f40d0890 0x7ff1f409a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:04.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.373+0000 7ff1f2800640 1 --2- 172.21.15.32:0/2615942044 >> v2:172.21.15.144:3300/0 conn(0x7ff1f409af30 0x7ff1f4162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:04.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.373+0000 7ff1f1e00640 1 -- 172.21.15.32:0/2615942044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1f40d0890 msgr2=0x7ff1f409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:04.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.373+0000 7ff1f1e00640 1 --2- 172.21.15.32:0/2615942044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1f40d0890 0x7ff1f409a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.373+0000 7ff1f1e00640 1 -- 172.21.15.32:0/2615942044 >> v2:172.21.15.144:3300/0 conn(0x7ff1f409af30 msgr2=0x7ff1f4162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:04.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.373+0000 7ff1f1e00640 1 --2- 172.21.15.32:0/2615942044 >> v2:172.21.15.144:3300/0 conn(0x7ff1f409af30 0x7ff1f4162630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.373+0000 7ff1f1e00640 1 -- 172.21.15.32:0/2615942044 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f40d0db0 con 0x7ff1f415eb30 2024-04-17T09:55:04.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.377+0000 7ff1f1e00640 1 --2- 172.21.15.32:0/2615942044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f415eb30 0x7ff1f409b770 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40c9540 tx=0x7ff1e401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb19967a2d75098 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:04.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.377+0000 7ff1f1400640 1 -- 172.21.15.32:0/2615942044 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e40b4070 con 0x7ff1f415eb30 2024-04-17T09:55:04.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.377+0000 7ff1f1400640 1 -- 172.21.15.32:0/2615942044 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e40af400 con 0x7ff1f415eb30 2024-04-17T09:55:04.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.377+0000 7ff1f1400640 1 -- 172.21.15.32:0/2615942044 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e40af7d0 con 0x7ff1f415eb30 2024-04-17T09:55:04.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.377+0000 7ff1f8a00640 1 -- 172.21.15.32:0/2615942044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f415eb30 msgr2=0x7ff1f409b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:04.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.377+0000 7ff1f8a00640 1 --2- 172.21.15.32:0/2615942044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f415eb30 0x7ff1f409b770 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40c9540 tx=0x7ff1e401f1e0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.377+0000 7ff1f8a00640 1 -- 172.21.15.32:0/2615942044 shutdown_connections 2024-04-17T09:55:04.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.377+0000 7ff1f8a00640 1 --2- 172.21.15.32:0/2615942044 >> v2:172.21.15.144:3300/0 conn(0x7ff1f409af30 0x7ff1f4162630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.377+0000 7ff1f8a00640 1 --2- 172.21.15.32:0/2615942044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1f40d0890 0x7ff1f409a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.377+0000 7ff1f8a00640 1 --2- 172.21.15.32:0/2615942044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f415eb30 0x7ff1f409b770 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.377+0000 7ff1f8a00640 1 -- 172.21.15.32:0/2615942044 >> 172.21.15.32:0/2615942044 conn(0x7ff1f40b3fd0 msgr2=0x7ff1f40b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:04.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.377+0000 7ff1f8a00640 1 -- 172.21.15.32:0/2615942044 shutdown_connections 2024-04-17T09:55:04.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.377+0000 7ff1f8a00640 1 -- 172.21.15.32:0/2615942044 wait complete. 2024-04-17T09:55:04.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1f8a00640 1 Processor -- start 2024-04-17T09:55:04.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1f8a00640 1 -- start start 2024-04-17T09:55:04.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1f8a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f409af30 0x7ff1f40c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:04.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1f8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f40d0890 0x7ff1f41f8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:04.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1f8a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1f415eb30 0x7ff1f40cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:04.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1f8a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f40d1650 con 0x7ff1f40d0890 2024-04-17T09:55:04.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1f8a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f40d1370 con 0x7ff1f409af30 2024-04-17T09:55:04.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1f8a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f40d1930 con 0x7ff1f415eb30 2024-04-17T09:55:04.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1e9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f40d0890 0x7ff1f41f8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:04.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1e9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f40d0890 0x7ff1f41f8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53922/0 (socket says 172.21.15.32:53922) 2024-04-17T09:55:04.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1e9400640 1 -- 172.21.15.32:0/1470061757 learned_addr learned my addr 172.21.15.32:0/1470061757 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:04.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1f1e00640 1 --2- 172.21.15.32:0/1470061757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f409af30 0x7ff1f40c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:04.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1f2800640 1 --2- 172.21.15.32:0/1470061757 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1f415eb30 0x7ff1f40cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:04.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1e9400640 1 -- 172.21.15.32:0/1470061757 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1f415eb30 msgr2=0x7ff1f40cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:04.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1e9400640 1 --2- 172.21.15.32:0/1470061757 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1f415eb30 0x7ff1f40cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1e9400640 1 -- 172.21.15.32:0/1470061757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f409af30 msgr2=0x7ff1f40c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:04.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1e9400640 1 --2- 172.21.15.32:0/1470061757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f409af30 0x7ff1f40c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1e9400640 1 -- 172.21.15.32:0/1470061757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f41f89a0 con 0x7ff1f40d0890 2024-04-17T09:55:04.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1f1e00640 1 --2- 172.21.15.32:0/1470061757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f409af30 0x7ff1f40c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:55:04.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1e9400640 1 --2- 172.21.15.32:0/1470061757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f40d0890 0x7ff1f41f8460 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00e930 tx=0x7ff1dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:04.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1eaa00640 1 -- 172.21.15.32:0/1470061757 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc017070 con 0x7ff1f40d0890 2024-04-17T09:55:04.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1eaa00640 1 -- 172.21.15.32:0/1470061757 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1dc00ca00 con 0x7ff1f40d0890 2024-04-17T09:55:04.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1eaa00640 1 -- 172.21.15.32:0/1470061757 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc00cdb0 con 0x7ff1f40d0890 2024-04-17T09:55:04.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1f8a00640 1 -- 172.21.15.32:0/1470061757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f41f8c40 con 0x7ff1f40d0890 2024-04-17T09:55:04.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.381+0000 7ff1f8a00640 1 -- 172.21.15.32:0/1470061757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f41f9200 con 0x7ff1f40d0890 2024-04-17T09:55:04.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.385+0000 7ff1f8a00640 1 -- 172.21.15.32:0/1470061757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f4068a90 con 0x7ff1f40d0890 2024-04-17T09:55:04.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.385+0000 7ff1eaa00640 1 -- 172.21.15.32:0/1470061757 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff1dc01a070 con 0x7ff1f40d0890 2024-04-17T09:55:04.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.385+0000 7ff1eaa00640 1 --2- 172.21.15.32:0/1470061757 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff1d0068380 0x7ff1d006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:04.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.385+0000 7ff1eaa00640 1 -- 172.21.15.32:0/1470061757 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7ff1dc086c50 con 0x7ff1f40d0890 2024-04-17T09:55:04.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.385+0000 7ff1f1e00640 1 --2- 172.21.15.32:0/1470061757 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff1d0068380 0x7ff1d006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:04.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.385+0000 7ff1f1e00640 1 --2- 172.21.15.32:0/1470061757 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff1d0068380 0x7ff1d006a840 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40b1000 tx=0x7ff1e40aa000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:04.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.389+0000 7ff1eaa00640 1 -- 172.21.15.32:0/1470061757 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff1dc08c020 con 0x7ff1f40d0890 2024-04-17T09:55:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.553+0000 7ff1f8a00640 1 -- 172.21.15.32:0/1470061757 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1f40ce9a0 con 0x7ff1f40d0890 2024-04-17T09:55:04.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.557+0000 7ff1eaa00640 1 -- 172.21.15.32:0/1470061757 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16638 (secure 0 0 0) 0x7ff1dc056d10 con 0x7ff1f40d0890 2024-04-17T09:55:04.557 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:04.557 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":45,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:55:03.233586+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:55:01.624737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-17T09:55:02.185212+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:55:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.557+0000 7ff1f8a00640 1 -- 172.21.15.32:0/1470061757 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff1d0068380 msgr2=0x7ff1d006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.557+0000 7ff1f8a00640 1 --2- 172.21.15.32:0/1470061757 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff1d0068380 0x7ff1d006a840 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40b1000 tx=0x7ff1e40aa000 comp rx=0 tx=0).stop 2024-04-17T09:55:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.557+0000 7ff1f8a00640 1 -- 172.21.15.32:0/1470061757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f40d0890 msgr2=0x7ff1f41f8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.557+0000 7ff1f8a00640 1 --2- 172.21.15.32:0/1470061757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f40d0890 0x7ff1f41f8460 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00e930 tx=0x7ff1dc00edf0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.557+0000 7ff1f8a00640 1 -- 172.21.15.32:0/1470061757 shutdown_connections 2024-04-17T09:55:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.557+0000 7ff1f8a00640 1 --2- 172.21.15.32:0/1470061757 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff1d0068380 0x7ff1d006a840 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.557+0000 7ff1f8a00640 1 --2- 172.21.15.32:0/1470061757 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1f415eb30 0x7ff1f40cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.557+0000 7ff1f8a00640 1 --2- 172.21.15.32:0/1470061757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f40d0890 0x7ff1f41f8460 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.557+0000 7ff1f8a00640 1 --2- 172.21.15.32:0/1470061757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f409af30 0x7ff1f40c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.557+0000 7ff1f8a00640 1 -- 172.21.15.32:0/1470061757 >> 172.21.15.32:0/1470061757 conn(0x7ff1f40b3fd0 msgr2=0x7ff1f415d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:04.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.557+0000 7ff1f8a00640 1 -- 172.21.15.32:0/1470061757 shutdown_connections 2024-04-17T09:55:04.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.557+0000 7ff1f8a00640 1 -- 172.21.15.32:0/1470061757 wait complete. 2024-04-17T09:55:04.579 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T09:55:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.721+0000 7fe984e00640 1 Processor -- start 2024-04-17T09:55:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.721+0000 7fe984e00640 1 -- start start 2024-04-17T09:55:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.721+0000 7fe984e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe98006ba30 0x7fe98006be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.721+0000 7fe984e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe9800cf370 0x7fe9800cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.721+0000 7fe984e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe98006c350 con 0x7fe980160350 2024-04-17T09:55:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.721+0000 7fe984e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe9800d0130 con 0x7fe9800cf370 2024-04-17T09:55:04.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.721+0000 7fe984e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe98015d090 con 0x7fe98006ba30 2024-04-17T09:55:04.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.721+0000 7fe975400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe9800cf370 0x7fe9800cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:04.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.721+0000 7fe97e800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe980160350 0x7fe9801607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33114/0 (socket says 172.21.15.32:33114) 2024-04-17T09:55:04.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.721+0000 7fe97e800640 1 -- 172.21.15.32:0/3476041161 learned_addr learned my addr 172.21.15.32:0/3476041161 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:04.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.721+0000 7fe97de00640 1 --2- 172.21.15.32:0/3476041161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe98006ba30 0x7fe98006be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:04.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.721+0000 7fe975400640 1 -- 172.21.15.32:0/3476041161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe98006ba30 msgr2=0x7fe98006be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:04.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.721+0000 7fe975400640 1 --2- 172.21.15.32:0/3476041161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe98006ba30 0x7fe98006be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.721+0000 7fe975400640 1 -- 172.21.15.32:0/3476041161 >> v1:172.21.15.32:6789/0 conn(0x7fe980160350 legacy=0x7fe9801607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:04.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.721+0000 7fe975400640 1 -- 172.21.15.32:0/3476041161 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98015d370 con 0x7fe9800cf370 2024-04-17T09:55:04.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.721+0000 7fe97d400640 1 -- 172.21.15.32:0/3476041161 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 214294800 0 0) 0x7fe98006c350 con 0x7fe980160350 2024-04-17T09:55:04.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.721+0000 7fe97de00640 1 --2- 172.21.15.32:0/3476041161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe98006ba30 0x7fe98006be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:55:04.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.725+0000 7fe975400640 1 --2- 172.21.15.32:0/3476041161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9800cf370 0x7fe9800cf7e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe9680097c0 tx=0x7fe96801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1582adb896abf88c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:04.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.725+0000 7fe97d400640 1 -- 172.21.15.32:0/3476041161 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9680b1070 con 0x7fe9800cf370 2024-04-17T09:55:04.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.725+0000 7fe97d400640 1 -- 172.21.15.32:0/3476041161 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9680ac410 con 0x7fe9800cf370 2024-04-17T09:55:04.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.725+0000 7fe97d400640 1 -- 172.21.15.32:0/3476041161 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9680ac7e0 con 0x7fe9800cf370 2024-04-17T09:55:04.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.725+0000 7fe984e00640 1 -- 172.21.15.32:0/3476041161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9800cf370 msgr2=0x7fe9800cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:04.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.725+0000 7fe984e00640 1 --2- 172.21.15.32:0/3476041161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9800cf370 0x7fe9800cf7e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe9680097c0 tx=0x7fe96801c4a0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.725+0000 7fe984e00640 1 -- 172.21.15.32:0/3476041161 shutdown_connections 2024-04-17T09:55:04.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.725+0000 7fe984e00640 1 --2- 172.21.15.32:0/3476041161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9800cf370 0x7fe9800cf7e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.725+0000 7fe984e00640 1 --2- 172.21.15.32:0/3476041161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe98006ba30 0x7fe98006be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.725+0000 7fe984e00640 1 -- 172.21.15.32:0/3476041161 >> 172.21.15.32:0/3476041161 conn(0x7fe9800b2120 msgr2=0x7fe9800b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:04.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.725+0000 7fe984e00640 1 -- 172.21.15.32:0/3476041161 shutdown_connections 2024-04-17T09:55:04.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.725+0000 7fe984e00640 1 -- 172.21.15.32:0/3476041161 wait complete. 2024-04-17T09:55:04.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe984e00640 1 Processor -- start 2024-04-17T09:55:04.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe984e00640 1 -- start start 2024-04-17T09:55:04.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe984e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe98006ba30 0x7fe9800b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:04.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe984e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9800cf370 0x7fe9800b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:04.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe984e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe980160350 0x7fe98015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:04.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe984e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe98015dcf0 con 0x7fe9800cf370 2024-04-17T09:55:04.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe984e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe98015db70 con 0x7fe980160350 2024-04-17T09:55:04.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe984e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe980146230 con 0x7fe98006ba30 2024-04-17T09:55:04.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe975400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9800cf370 0x7fe9800b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:04.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe975400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9800cf370 0x7fe9800b4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53940/0 (socket says 172.21.15.32:53940) 2024-04-17T09:55:04.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe975400640 1 -- 172.21.15.32:0/1856605158 learned_addr learned my addr 172.21.15.32:0/1856605158 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:04.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe97de00640 1 --2- 172.21.15.32:0/1856605158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe98006ba30 0x7fe9800b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:04.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe97e800640 1 --2- 172.21.15.32:0/1856605158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe980160350 0x7fe98015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:04.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe975400640 1 -- 172.21.15.32:0/1856605158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe98006ba30 msgr2=0x7fe9800b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:04.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe975400640 1 --2- 172.21.15.32:0/1856605158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe98006ba30 0x7fe9800b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe975400640 1 -- 172.21.15.32:0/1856605158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe980160350 msgr2=0x7fe98015ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:04.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe975400640 1 --2- 172.21.15.32:0/1856605158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe980160350 0x7fe98015ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:04.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe975400640 1 -- 172.21.15.32:0/1856605158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98015bab0 con 0x7fe9800cf370 2024-04-17T09:55:04.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe975400640 1 --2- 172.21.15.32:0/1856605158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9800cf370 0x7fe9800b4ac0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe96801a0d0 tx=0x7fe968019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:04.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe976a00640 1 -- 172.21.15.32:0/1856605158 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9680b9020 con 0x7fe9800cf370 2024-04-17T09:55:04.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe976a00640 1 -- 172.21.15.32:0/1856605158 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9680ace40 con 0x7fe9800cf370 2024-04-17T09:55:04.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe976a00640 1 -- 172.21.15.32:0/1856605158 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9680b1040 con 0x7fe9800cf370 2024-04-17T09:55:04.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe984e00640 1 -- 172.21.15.32:0/1856605158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98015b4f0 con 0x7fe9800cf370 2024-04-17T09:55:04.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe984e00640 1 -- 172.21.15.32:0/1856605158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9800b42c0 con 0x7fe9800cf370 2024-04-17T09:55:04.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.729+0000 7fe984e00640 1 -- 172.21.15.32:0/1856605158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe980068a90 con 0x7fe9800cf370 2024-04-17T09:55:04.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.733+0000 7fe976a00640 1 -- 172.21.15.32:0/1856605158 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe9680bc020 con 0x7fe9800cf370 2024-04-17T09:55:04.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.733+0000 7fe976a00640 1 --2- 172.21.15.32:0/1856605158 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe95c0683b0 0x7fe95c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:04.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.733+0000 7fe976a00640 1 -- 172.21.15.32:0/1856605158 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fe9681203e0 con 0x7fe9800cf370 2024-04-17T09:55:04.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.733+0000 7fe97de00640 1 --2- 172.21.15.32:0/1856605158 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe95c0683b0 0x7fe95c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:04.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.733+0000 7fe97de00640 1 --2- 172.21.15.32:0/1856605158 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe95c0683b0 0x7fe95c06a870 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe9800b03e0 tx=0x7fe970008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:04.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.737+0000 7fe976a00640 1 -- 172.21.15.32:0/1856605158 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fe9680c3030 con 0x7fe9800cf370 2024-04-17T09:55:04.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:04.945+0000 7fe984e00640 1 -- 172.21.15.32:0/1856605158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe98006c850 con 0x7fe9800cf370 2024-04-17T09:55:05.815 INFO:tasks.ceph.mds.d.smithi144.stderr:2024-04-17T09:55:05.813+0000 7fb02ea00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T09:55:05.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:05.845+0000 7fe976a00640 1 -- 172.21.15.32:0/1856605158 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) ==== 118+0+0 (secure 0 0 0) 0x7fe968016090 con 0x7fe9800cf370 2024-04-17T09:55:05.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:05.849+0000 7fe984e00640 1 -- 172.21.15.32:0/1856605158 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe95c0683b0 msgr2=0x7fe95c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:05.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:05.849+0000 7fe984e00640 1 --2- 172.21.15.32:0/1856605158 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe95c0683b0 0x7fe95c06a870 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe9800b03e0 tx=0x7fe970008040 comp rx=0 tx=0).stop 2024-04-17T09:55:05.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:05.849+0000 7fe984e00640 1 -- 172.21.15.32:0/1856605158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9800cf370 msgr2=0x7fe9800b4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:05.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:05.849+0000 7fe984e00640 1 --2- 172.21.15.32:0/1856605158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9800cf370 0x7fe9800b4ac0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe96801a0d0 tx=0x7fe968019240 comp rx=0 tx=0).stop 2024-04-17T09:55:05.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:05.849+0000 7fe984e00640 1 -- 172.21.15.32:0/1856605158 shutdown_connections 2024-04-17T09:55:05.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:05.849+0000 7fe984e00640 1 --2- 172.21.15.32:0/1856605158 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe95c0683b0 0x7fe95c06a870 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:05.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:05.849+0000 7fe984e00640 1 --2- 172.21.15.32:0/1856605158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe980160350 0x7fe98015ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:05.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:05.849+0000 7fe984e00640 1 --2- 172.21.15.32:0/1856605158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9800cf370 0x7fe9800b4ac0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:05.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:05.849+0000 7fe984e00640 1 --2- 172.21.15.32:0/1856605158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe98006ba30 0x7fe9800b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:05.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:05.849+0000 7fe984e00640 1 -- 172.21.15.32:0/1856605158 >> 172.21.15.32:0/1856605158 conn(0x7fe9800b2120 msgr2=0x7fe9800cc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:05.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:05.849+0000 7fe984e00640 1 -- 172.21.15.32:0/1856605158 shutdown_connections 2024-04-17T09:55:05.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:05.849+0000 7fe984e00640 1 -- 172.21.15.32:0/1856605158 wait complete. 2024-04-17T09:55:05.873 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-17T09:55:05.873 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-17T09:55:06.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.021+0000 7fbb19400640 1 Processor -- start 2024-04-17T09:55:06.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.021+0000 7fbb19400640 1 -- start start 2024-04-17T09:55:06.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.021+0000 7fbb19400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb1406ba30 0x7fbb1406be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:06.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.021+0000 7fbb19400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbb140cdda0 0x7fbb140ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:06.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.021+0000 7fbb19400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb140d0690 con 0x7fbb1415e880 2024-04-17T09:55:06.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.021+0000 7fbb19400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fbb1406c3d0 con 0x7fbb140cdda0 2024-04-17T09:55:06.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.021+0000 7fbb19400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbb140d11d0 con 0x7fbb1406ba30 2024-04-17T09:55:06.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.021+0000 7fbb13200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbb140cdda0 0x7fbb140ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:06.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.021+0000 7fbb13200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbb140cdda0 0x7fbb140ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53946/0 (socket says 172.21.15.32:53946) 2024-04-17T09:55:06.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.021+0000 7fbb13200640 1 -- 172.21.15.32:0/1008312802 learned_addr learned my addr 172.21.15.32:0/1008312802 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:06.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.021+0000 7fbb11e00640 1 --2- 172.21.15.32:0/1008312802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb1406ba30 0x7fbb1406be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:06.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.021+0000 7fbb13200640 1 -- 172.21.15.32:0/1008312802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb1406ba30 msgr2=0x7fbb1406be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:06.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.021+0000 7fbb13200640 1 --2- 172.21.15.32:0/1008312802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb1406ba30 0x7fbb1406be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:06.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.021+0000 7fbb13200640 1 -- 172.21.15.32:0/1008312802 >> v1:172.21.15.144:6789/0 conn(0x7fbb1415e880 legacy=0x7fbb140b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:06.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.021+0000 7fbb13200640 1 -- 172.21.15.32:0/1008312802 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb140d0930 con 0x7fbb140cdda0 2024-04-17T09:55:06.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.021+0000 7fbb11e00640 1 --2- 172.21.15.32:0/1008312802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb1406ba30 0x7fbb1406be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:06.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.025+0000 7fbb13200640 1 --2- 172.21.15.32:0/1008312802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb140cdda0 0x7fbb140ce210 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbb140d1350 tx=0x7fbb0c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb46dff20718f2f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:06.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.025+0000 7fbb11400640 1 -- 172.21.15.32:0/1008312802 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c0a5810 con 0x7fbb140cdda0 2024-04-17T09:55:06.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.025+0000 7fbb11400640 1 -- 172.21.15.32:0/1008312802 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb0c0a59b0 con 0x7fbb140cdda0 2024-04-17T09:55:06.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.025+0000 7fbb11400640 1 -- 172.21.15.32:0/1008312802 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c0a5d80 con 0x7fbb140cdda0 2024-04-17T09:55:06.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.025+0000 7fbb19400640 1 -- 172.21.15.32:0/1008312802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb140cdda0 msgr2=0x7fbb140ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:06.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.025+0000 7fbb19400640 1 --2- 172.21.15.32:0/1008312802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb140cdda0 0x7fbb140ce210 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbb140d1350 tx=0x7fbb0c01c950 comp rx=0 tx=0).stop 2024-04-17T09:55:06.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.025+0000 7fbb19400640 1 -- 172.21.15.32:0/1008312802 shutdown_connections 2024-04-17T09:55:06.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.025+0000 7fbb19400640 1 --2- 172.21.15.32:0/1008312802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb140cdda0 0x7fbb140ce210 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:06.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.025+0000 7fbb19400640 1 --2- 172.21.15.32:0/1008312802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb1406ba30 0x7fbb1406be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:06.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.025+0000 7fbb19400640 1 -- 172.21.15.32:0/1008312802 >> 172.21.15.32:0/1008312802 conn(0x7fbb1409a9f0 msgr2=0x7fbb140b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:06.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.025+0000 7fbb19400640 1 -- 172.21.15.32:0/1008312802 shutdown_connections 2024-04-17T09:55:06.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.025+0000 7fbb19400640 1 -- 172.21.15.32:0/1008312802 wait complete. 2024-04-17T09:55:06.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb19400640 1 Processor -- start 2024-04-17T09:55:06.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb19400640 1 -- start start 2024-04-17T09:55:06.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb19400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb1406ba30 0x7fbb14153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:06.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb19400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb140cdda0 0x7fbb1414fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:06.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb19400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb1415e880 0x7fbb140b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb19400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbb140ceb60 con 0x7fbb140cdda0 2024-04-17T09:55:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb19400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1415ec60 con 0x7fbb1415e880 2024-04-17T09:55:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb19400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbb140cece0 con 0x7fbb1406ba30 2024-04-17T09:55:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb11e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb140cdda0 0x7fbb1414fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb11e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb140cdda0 0x7fbb1414fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53954/0 (socket says 172.21.15.32:53954) 2024-04-17T09:55:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb11e00640 1 -- 172.21.15.32:0/252063268 learned_addr learned my addr 172.21.15.32:0/252063268 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb12800640 1 --2- 172.21.15.32:0/252063268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb1406ba30 0x7fbb14153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb13200640 1 --2- 172.21.15.32:0/252063268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb1415e880 0x7fbb140b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb11e00640 1 -- 172.21.15.32:0/252063268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb1406ba30 msgr2=0x7fbb14153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb11e00640 1 --2- 172.21.15.32:0/252063268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb1406ba30 0x7fbb14153670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb11e00640 1 -- 172.21.15.32:0/252063268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb1415e880 msgr2=0x7fbb140b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb11e00640 1 --2- 172.21.15.32:0/252063268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb1415e880 0x7fbb140b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb11e00640 1 -- 172.21.15.32:0/252063268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb14153c50 con 0x7fbb140cdda0 2024-04-17T09:55:06.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb11e00640 1 --2- 172.21.15.32:0/252063268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb140cdda0 0x7fbb1414fa50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000d800 tx=0x7fbb0000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:06.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb0aa00640 1 -- 172.21.15.32:0/252063268 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0000b6c0 con 0x7fbb140cdda0 2024-04-17T09:55:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb0aa00640 1 -- 172.21.15.32:0/252063268 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb00019070 con 0x7fbb140cdda0 2024-04-17T09:55:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb19400640 1 -- 172.21.15.32:0/252063268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb14153f30 con 0x7fbb140cdda0 2024-04-17T09:55:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb0aa00640 1 -- 172.21.15.32:0/252063268 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0000baa0 con 0x7fbb140cdda0 2024-04-17T09:55:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb19400640 1 -- 172.21.15.32:0/252063268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb141501e0 con 0x7fbb140cdda0 2024-04-17T09:55:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.029+0000 7fbb19400640 1 -- 172.21.15.32:0/252063268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb14068a90 con 0x7fbb140cdda0 2024-04-17T09:55:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.033+0000 7fbb0aa00640 1 -- 172.21.15.32:0/252063268 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbb0000bc40 con 0x7fbb140cdda0 2024-04-17T09:55:06.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.033+0000 7fbb0aa00640 1 --2- 172.21.15.32:0/252063268 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbaf40683b0 0x7fbaf406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:06.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.033+0000 7fbb0aa00640 1 -- 172.21.15.32:0/252063268 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fbb00086c90 con 0x7fbb140cdda0 2024-04-17T09:55:06.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.033+0000 7fbb12800640 1 --2- 172.21.15.32:0/252063268 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbaf40683b0 0x7fbaf406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:06.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.033+0000 7fbb12800640 1 --2- 172.21.15.32:0/252063268 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbaf40683b0 0x7fbaf406a870 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbb141552d0 tx=0x7fbb04009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:06.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.037+0000 7fbb0aa00640 1 -- 172.21.15.32:0/252063268 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fbb00056be0 con 0x7fbb140cdda0 2024-04-17T09:55:06.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.241+0000 7fbb19400640 1 -- 172.21.15.32:0/252063268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fbb14152060 con 0x7fbb140cdda0 2024-04-17T09:55:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.845+0000 7fbb0aa00640 1 -- 172.21.15.32:0/252063268 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7fbb0005aee0 con 0x7fbb140cdda0 2024-04-17T09:55:06.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.849+0000 7fbb19400640 1 -- 172.21.15.32:0/252063268 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbaf40683b0 msgr2=0x7fbaf406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:06.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.849+0000 7fbb19400640 1 --2- 172.21.15.32:0/252063268 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbaf40683b0 0x7fbaf406a870 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbb141552d0 tx=0x7fbb04009040 comp rx=0 tx=0).stop 2024-04-17T09:55:06.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.849+0000 7fbb19400640 1 -- 172.21.15.32:0/252063268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb140cdda0 msgr2=0x7fbb1414fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:06.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.849+0000 7fbb19400640 1 --2- 172.21.15.32:0/252063268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb140cdda0 0x7fbb1414fa50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000d800 tx=0x7fbb0000dcc0 comp rx=0 tx=0).stop 2024-04-17T09:55:06.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.849+0000 7fbb19400640 1 -- 172.21.15.32:0/252063268 shutdown_connections 2024-04-17T09:55:06.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.849+0000 7fbb19400640 1 --2- 172.21.15.32:0/252063268 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbaf40683b0 0x7fbaf406a870 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:06.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.849+0000 7fbb19400640 1 --2- 172.21.15.32:0/252063268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb1415e880 0x7fbb140b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:06.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.849+0000 7fbb19400640 1 --2- 172.21.15.32:0/252063268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb140cdda0 0x7fbb1414fa50 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:06.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.849+0000 7fbb19400640 1 --2- 172.21.15.32:0/252063268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb1406ba30 0x7fbb14153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:06.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.849+0000 7fbb19400640 1 -- 172.21.15.32:0/252063268 >> 172.21.15.32:0/252063268 conn(0x7fbb1409a9f0 msgr2=0x7fbb140b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:06.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.849+0000 7fbb19400640 1 -- 172.21.15.32:0/252063268 shutdown_connections 2024-04-17T09:55:06.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:06.849+0000 7fbb19400640 1 -- 172.21.15.32:0/252063268 wait complete. 2024-04-17T09:55:06.874 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-17T09:55:06.874 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:55:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.017+0000 7fa091600640 1 Processor -- start 2024-04-17T09:55:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.017+0000 7fa091600640 1 -- start start 2024-04-17T09:55:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.017+0000 7fa091600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa08c144e90 0x7fa08c0b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.017+0000 7fa091600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa08c0b4de0 0x7fa08c15baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.017+0000 7fa091600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa08c15bfe0 0x7fa08c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.017+0000 7fa091600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa08c15c430 con 0x7fa08c0b4de0 2024-04-17T09:55:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.017+0000 7fa091600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa08c145270 con 0x7fa08c15bfe0 2024-04-17T09:55:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.017+0000 7fa091600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa08c072bb0 con 0x7fa08c144e90 2024-04-17T09:55:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.017+0000 7fa082000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa08c0b4de0 0x7fa08c15baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.017+0000 7fa082000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa08c0b4de0 0x7fa08c15baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53958/0 (socket says 172.21.15.32:53958) 2024-04-17T09:55:07.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.017+0000 7fa082000640 1 -- 172.21.15.32:0/906948740 learned_addr learned my addr 172.21.15.32:0/906948740 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:07.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.017+0000 7fa08b400640 1 --2- 172.21.15.32:0/906948740 >> v2:172.21.15.144:3300/0 conn(0x7fa08c15bfe0 0x7fa08c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:07.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.017+0000 7fa08aa00640 1 --2- 172.21.15.32:0/906948740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa08c144e90 0x7fa08c0b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:07.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.021+0000 7fa082000640 1 -- 172.21.15.32:0/906948740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa08c144e90 msgr2=0x7fa08c0b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:07.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.021+0000 7fa082000640 1 --2- 172.21.15.32:0/906948740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa08c144e90 0x7fa08c0b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.021+0000 7fa082000640 1 -- 172.21.15.32:0/906948740 >> v2:172.21.15.144:3300/0 conn(0x7fa08c15bfe0 msgr2=0x7fa08c06ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:07.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.021+0000 7fa082000640 1 --2- 172.21.15.32:0/906948740 >> v2:172.21.15.144:3300/0 conn(0x7fa08c15bfe0 0x7fa08c06ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.021+0000 7fa082000640 1 -- 172.21.15.32:0/906948740 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08c0b3dc0 con 0x7fa08c0b4de0 2024-04-17T09:55:07.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.021+0000 7fa08b400640 1 --2- 172.21.15.32:0/906948740 >> v2:172.21.15.144:3300/0 conn(0x7fa08c15bfe0 0x7fa08c06ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:07.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.021+0000 7fa082000640 1 --2- 172.21.15.32:0/906948740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c0b4de0 0x7fa08c15baa0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa08c070150 tx=0x7fa07401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446992f2ccb7de5e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:07.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.021+0000 7fa08a000640 1 -- 172.21.15.32:0/906948740 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0740b1070 con 0x7fa08c0b4de0 2024-04-17T09:55:07.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.021+0000 7fa08a000640 1 -- 172.21.15.32:0/906948740 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0740ac620 con 0x7fa08c0b4de0 2024-04-17T09:55:07.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.021+0000 7fa08a000640 1 -- 172.21.15.32:0/906948740 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0740ac9f0 con 0x7fa08c0b4de0 2024-04-17T09:55:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.021+0000 7fa091600640 1 -- 172.21.15.32:0/906948740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c0b4de0 msgr2=0x7fa08c15baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.021+0000 7fa091600640 1 --2- 172.21.15.32:0/906948740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c0b4de0 0x7fa08c15baa0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa08c070150 tx=0x7fa07401c4f0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.021+0000 7fa091600640 1 -- 172.21.15.32:0/906948740 shutdown_connections 2024-04-17T09:55:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.021+0000 7fa091600640 1 --2- 172.21.15.32:0/906948740 >> v2:172.21.15.144:3300/0 conn(0x7fa08c15bfe0 0x7fa08c06ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.021+0000 7fa091600640 1 --2- 172.21.15.32:0/906948740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c0b4de0 0x7fa08c15baa0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.021+0000 7fa091600640 1 --2- 172.21.15.32:0/906948740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa08c144e90 0x7fa08c0b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.021+0000 7fa091600640 1 -- 172.21.15.32:0/906948740 >> 172.21.15.32:0/906948740 conn(0x7fa08c06ba30 msgr2=0x7fa08c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.021+0000 7fa091600640 1 -- 172.21.15.32:0/906948740 shutdown_connections 2024-04-17T09:55:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.021+0000 7fa091600640 1 -- 172.21.15.32:0/906948740 wait complete. 2024-04-17T09:55:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa091600640 1 Processor -- start 2024-04-17T09:55:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa091600640 1 -- start start 2024-04-17T09:55:07.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa091600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa08c0b4de0 0x7fa08c0ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:07.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa091600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa08c144e90 0x7fa08c15add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:07.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa091600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c15bfe0 0x7fa08c0cf860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:07.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa091600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c0cf2b0 con 0x7fa08c15bfe0 2024-04-17T09:55:07.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa091600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c072880 con 0x7fa08c144e90 2024-04-17T09:55:07.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa091600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa08c0cf430 con 0x7fa08c0b4de0 2024-04-17T09:55:07.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa08b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c15bfe0 0x7fa08c0cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:07.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa08b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c15bfe0 0x7fa08c0cf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53974/0 (socket says 172.21.15.32:53974) 2024-04-17T09:55:07.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa08b400640 1 -- 172.21.15.32:0/2413885031 learned_addr learned my addr 172.21.15.32:0/2413885031 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:07.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa082000640 1 --2- 172.21.15.32:0/2413885031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa08c144e90 0x7fa08c15add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:07.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa08aa00640 1 --2- 172.21.15.32:0/2413885031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa08c0b4de0 0x7fa08c0ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:07.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa08b400640 1 -- 172.21.15.32:0/2413885031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa08c0b4de0 msgr2=0x7fa08c0ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:07.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa08b400640 1 --2- 172.21.15.32:0/2413885031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa08c0b4de0 0x7fa08c0ae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa08b400640 1 -- 172.21.15.32:0/2413885031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa08c144e90 msgr2=0x7fa08c15add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa08b400640 1 --2- 172.21.15.32:0/2413885031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa08c144e90 0x7fa08c15add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa08b400640 1 -- 172.21.15.32:0/2413885031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08c1f7280 con 0x7fa08c15bfe0 2024-04-17T09:55:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa08b400640 1 --2- 172.21.15.32:0/2413885031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c15bfe0 0x7fa08c0cf860 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa084007e70 tx=0x7fa08400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa083400640 1 -- 172.21.15.32:0/2413885031 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08401a070 con 0x7fa08c15bfe0 2024-04-17T09:55:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa083400640 1 -- 172.21.15.32:0/2413885031 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa084003bc0 con 0x7fa08c15bfe0 2024-04-17T09:55:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa091600640 1 -- 172.21.15.32:0/2413885031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08c13f240 con 0x7fa08c15bfe0 2024-04-17T09:55:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa083400640 1 -- 172.21.15.32:0/2413885031 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084004030 con 0x7fa08c15bfe0 2024-04-17T09:55:07.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.025+0000 7fa091600640 1 -- 172.21.15.32:0/2413885031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa08c0aeb90 con 0x7fa08c15bfe0 2024-04-17T09:55:07.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.029+0000 7fa091600640 1 -- 172.21.15.32:0/2413885031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa08c068a90 con 0x7fa08c15bfe0 2024-04-17T09:55:07.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.029+0000 7fa083400640 1 -- 172.21.15.32:0/2413885031 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa08401d070 con 0x7fa08c15bfe0 2024-04-17T09:55:07.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.029+0000 7fa083400640 1 --2- 172.21.15.32:0/2413885031 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa068068380 0x7fa06806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:07.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.029+0000 7fa083400640 1 -- 172.21.15.32:0/2413885031 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fa084086b50 con 0x7fa08c15bfe0 2024-04-17T09:55:07.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.029+0000 7fa08aa00640 1 --2- 172.21.15.32:0/2413885031 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa068068380 0x7fa06806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:07.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.029+0000 7fa08aa00640 1 --2- 172.21.15.32:0/2413885031 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa068068380 0x7fa06806a840 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa07c007a60 tx=0x7fa07c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:07.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.033+0000 7fa083400640 1 -- 172.21.15.32:0/2413885031 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa084010070 con 0x7fa08c15bfe0 2024-04-17T09:55:07.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.233+0000 7fa091600640 1 -- 172.21.15.32:0/2413885031 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa08c140250 con 0x7fa08c15bfe0 2024-04-17T09:55:07.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.233+0000 7fa083400640 1 -- 172.21.15.32:0/2413885031 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5202 (secure 0 0 0) 0x7fa084056ab0 con 0x7fa08c15bfe0 2024-04-17T09:55:07.236 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:07.236 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2542578926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2542578926},{"type":"v1","addr":"172.21.15.144:6834","nonce":2542578926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1424948520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1424948520},{"type":"v1","addr":"172.21.15.32:6836","nonce":1424948520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/393813154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":393813154},{"type":"v1","addr":"172.21.15.32:6837","nonce":393813154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:55:03.233552+0000","modified":"2024-04-17T09:55:06.243321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5742":{"gid":5742,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/2177384675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2177384675},{"type":"v1","addr":"172.21.15.144:6835","nonce":2177384675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742]},"id":3}]} 2024-04-17T09:55:07.236 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2024-04-17T09:55:07.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.237+0000 7fa091600640 1 -- 172.21.15.32:0/2413885031 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa068068380 msgr2=0x7fa06806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:07.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.237+0000 7fa091600640 1 --2- 172.21.15.32:0/2413885031 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa068068380 0x7fa06806a840 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa07c007a60 tx=0x7fa07c008040 comp rx=0 tx=0).stop 2024-04-17T09:55:07.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.237+0000 7fa091600640 1 -- 172.21.15.32:0/2413885031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c15bfe0 msgr2=0x7fa08c0cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:07.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.237+0000 7fa091600640 1 --2- 172.21.15.32:0/2413885031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c15bfe0 0x7fa08c0cf860 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa084007e70 tx=0x7fa08400d4a0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.237+0000 7fa091600640 1 -- 172.21.15.32:0/2413885031 shutdown_connections 2024-04-17T09:55:07.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.237+0000 7fa091600640 1 --2- 172.21.15.32:0/2413885031 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa068068380 0x7fa06806a840 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.237+0000 7fa091600640 1 --2- 172.21.15.32:0/2413885031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08c15bfe0 0x7fa08c0cf860 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.237+0000 7fa091600640 1 --2- 172.21.15.32:0/2413885031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa08c144e90 0x7fa08c15add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.237+0000 7fa091600640 1 --2- 172.21.15.32:0/2413885031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa08c0b4de0 0x7fa08c0ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.237+0000 7fa091600640 1 -- 172.21.15.32:0/2413885031 >> 172.21.15.32:0/2413885031 conn(0x7fa08c06ba30 msgr2=0x7fa08c140770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:07.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.237+0000 7fa091600640 1 -- 172.21.15.32:0/2413885031 shutdown_connections 2024-04-17T09:55:07.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.237+0000 7fa091600640 1 -- 172.21.15.32:0/2413885031 wait complete. 2024-04-17T09:55:07.258 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:55:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.397+0000 7fa6d5a00640 1 Processor -- start 2024-04-17T09:55:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.397+0000 7fa6d5a00640 1 -- start start 2024-04-17T09:55:07.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.397+0000 7fa6d5a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa6d00cf990 0x7fa6d006dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:07.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.397+0000 7fa6d5a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa6d006f250 0x7fa6d00aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:07.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.397+0000 7fa6d5a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6d006e1b0 con 0x7fa6d00cf5b0 2024-04-17T09:55:07.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.397+0000 7fa6d5a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa6d006edc0 con 0x7fa6d00cf990 2024-04-17T09:55:07.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.397+0000 7fa6d5a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa6d006e490 con 0x7fa6d006f250 2024-04-17T09:55:07.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.397+0000 7fa6c7e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa6d00cf990 0x7fa6d006dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:07.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.397+0000 7fa6cfe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa6d00cf5b0 0x7fa6d015d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33130/0 (socket says 172.21.15.32:33130) 2024-04-17T09:55:07.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.397+0000 7fa6cfe00640 1 -- 172.21.15.32:0/2119107255 learned_addr learned my addr 172.21.15.32:0/2119107255 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.401+0000 7fa6d5000640 1 --2- 172.21.15.32:0/2119107255 >> v2:172.21.15.144:3300/0 conn(0x7fa6d006f250 0x7fa6d00aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.401+0000 7fa6c7e00640 1 -- 172.21.15.32:0/2119107255 >> v2:172.21.15.144:3300/0 conn(0x7fa6d006f250 msgr2=0x7fa6d00aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.401+0000 7fa6c7e00640 1 --2- 172.21.15.32:0/2119107255 >> v2:172.21.15.144:3300/0 conn(0x7fa6d006f250 0x7fa6d00aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.401+0000 7fa6c7e00640 1 -- 172.21.15.32:0/2119107255 >> v1:172.21.15.32:6789/0 conn(0x7fa6d00cf5b0 legacy=0x7fa6d015d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.401+0000 7fa6c7e00640 1 -- 172.21.15.32:0/2119107255 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d006e770 con 0x7fa6d00cf990 2024-04-17T09:55:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.401+0000 7fa6c7e00640 1 --2- 172.21.15.32:0/2119107255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d00cf990 0x7fa6d006dbd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa6d015a510 tx=0x7fa6b801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff69169b1af5f393 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.401+0000 7fa6cf400640 1 -- 172.21.15.32:0/2119107255 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b80b1070 con 0x7fa6d00cf990 2024-04-17T09:55:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.401+0000 7fa6cf400640 1 -- 172.21.15.32:0/2119107255 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b80a8050 con 0x7fa6d00cf990 2024-04-17T09:55:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.401+0000 7fa6cf400640 1 -- 172.21.15.32:0/2119107255 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b80ac860 con 0x7fa6d00cf990 2024-04-17T09:55:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.401+0000 7fa6d5a00640 1 -- 172.21.15.32:0/2119107255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d00cf990 msgr2=0x7fa6d006dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.401+0000 7fa6d5a00640 1 --2- 172.21.15.32:0/2119107255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d00cf990 0x7fa6d006dbd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa6d015a510 tx=0x7fa6b801c3c0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6d5a00640 1 -- 172.21.15.32:0/2119107255 shutdown_connections 2024-04-17T09:55:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6d5a00640 1 --2- 172.21.15.32:0/2119107255 >> v2:172.21.15.144:3300/0 conn(0x7fa6d006f250 0x7fa6d00aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6d5a00640 1 --2- 172.21.15.32:0/2119107255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d00cf990 0x7fa6d006dbd0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6d5a00640 1 -- 172.21.15.32:0/2119107255 >> 172.21.15.32:0/2119107255 conn(0x7fa6d00d0180 msgr2=0x7fa6d00d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:07.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6d5a00640 1 -- 172.21.15.32:0/2119107255 shutdown_connections 2024-04-17T09:55:07.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6d5a00640 1 -- 172.21.15.32:0/2119107255 wait complete. 2024-04-17T09:55:07.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6d5a00640 1 Processor -- start 2024-04-17T09:55:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6d5a00640 1 -- start start 2024-04-17T09:55:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6d5a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6d006f250 0x7fa6d0145b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6d5a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d00cf5b0 0x7fa6d00b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6d5a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d00cf990 0x7fa6d0143dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6d5a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d015e6e0 con 0x7fa6d00cf5b0 2024-04-17T09:55:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6d5a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d015e560 con 0x7fa6d00cf990 2024-04-17T09:55:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6d5a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa6d015e860 con 0x7fa6d006f250 2024-04-17T09:55:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6c7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d00cf5b0 0x7fa6d00b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6c7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d00cf5b0 0x7fa6d00b42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53994/0 (socket says 172.21.15.32:53994) 2024-04-17T09:55:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6c7e00640 1 -- 172.21.15.32:0/832524302 learned_addr learned my addr 172.21.15.32:0/832524302 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6cfe00640 1 --2- 172.21.15.32:0/832524302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6d006f250 0x7fa6d0145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6d5000640 1 --2- 172.21.15.32:0/832524302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d00cf990 0x7fa6d0143dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6c7e00640 1 -- 172.21.15.32:0/832524302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6d006f250 msgr2=0x7fa6d0145b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6c7e00640 1 --2- 172.21.15.32:0/832524302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6d006f250 0x7fa6d0145b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6c7e00640 1 -- 172.21.15.32:0/832524302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d00cf990 msgr2=0x7fa6d0143dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6c7e00640 1 --2- 172.21.15.32:0/832524302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d00cf990 0x7fa6d0143dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6c7e00640 1 -- 172.21.15.32:0/832524302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d015aab0 con 0x7fa6d00cf5b0 2024-04-17T09:55:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6d5000640 1 --2- 172.21.15.32:0/832524302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d00cf990 0x7fa6d0143dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6c7e00640 1 --2- 172.21.15.32:0/832524302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d00cf5b0 0x7fa6d00b42e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8019ff0 tx=0x7fa6b801ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6ccc00640 1 -- 172.21.15.32:0/832524302 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b80bc070 con 0x7fa6d00cf5b0 2024-04-17T09:55:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6ccc00640 1 -- 172.21.15.32:0/832524302 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b80b7630 con 0x7fa6d00cf5b0 2024-04-17T09:55:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6ccc00640 1 -- 172.21.15.32:0/832524302 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b80b1040 con 0x7fa6d00cf5b0 2024-04-17T09:55:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.405+0000 7fa6d5a00640 1 -- 172.21.15.32:0/832524302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d015ad90 con 0x7fa6d00cf5b0 2024-04-17T09:55:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.409+0000 7fa6d5a00640 1 -- 172.21.15.32:0/832524302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d015b350 con 0x7fa6d00cf5b0 2024-04-17T09:55:07.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.409+0000 7fa6d5a00640 1 -- 172.21.15.32:0/832524302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d004fd00 con 0x7fa6d00cf5b0 2024-04-17T09:55:07.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.409+0000 7fa6ccc00640 1 -- 172.21.15.32:0/832524302 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa6b80ab3e0 con 0x7fa6d00cf5b0 2024-04-17T09:55:07.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.409+0000 7fa6ccc00640 1 --2- 172.21.15.32:0/832524302 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa6ac068380 0x7fa6ac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:07.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.409+0000 7fa6ccc00640 1 -- 172.21.15.32:0/832524302 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fa6b81206b0 con 0x7fa6d00cf5b0 2024-04-17T09:55:07.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.409+0000 7fa6cfe00640 1 --2- 172.21.15.32:0/832524302 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa6ac068380 0x7fa6ac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:07.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.409+0000 7fa6cfe00640 1 --2- 172.21.15.32:0/832524302 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa6ac068380 0x7fa6ac06a840 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0000f70 tx=0x7fa6c0007e90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:07.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.413+0000 7fa6ccc00640 1 -- 172.21.15.32:0/832524302 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa6b8016070 con 0x7fa6d00cf5b0 2024-04-17T09:55:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.581+0000 7fa6d5a00640 1 -- 172.21.15.32:0/832524302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6d006dd70 con 0x7fa6d00cf5b0 2024-04-17T09:55:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.581+0000 7fa6ccc00640 1 -- 172.21.15.32:0/832524302 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16625 (secure 0 0 0) 0x7fa6b80f04f0 con 0x7fa6d00cf5b0 2024-04-17T09:55:07.584 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:07.584 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":46,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:55:04.834732+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:55:01.624737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-17T09:55:02.185212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:55:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.585+0000 7fa6d5a00640 1 -- 172.21.15.32:0/832524302 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa6ac068380 msgr2=0x7fa6ac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.585+0000 7fa6d5a00640 1 --2- 172.21.15.32:0/832524302 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa6ac068380 0x7fa6ac06a840 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0000f70 tx=0x7fa6c0007e90 comp rx=0 tx=0).stop 2024-04-17T09:55:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.585+0000 7fa6d5a00640 1 -- 172.21.15.32:0/832524302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d00cf5b0 msgr2=0x7fa6d00b42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.585+0000 7fa6d5a00640 1 --2- 172.21.15.32:0/832524302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d00cf5b0 0x7fa6d00b42e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8019ff0 tx=0x7fa6b801ac00 comp rx=0 tx=0).stop 2024-04-17T09:55:07.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.585+0000 7fa6d5a00640 1 -- 172.21.15.32:0/832524302 shutdown_connections 2024-04-17T09:55:07.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.585+0000 7fa6d5a00640 1 --2- 172.21.15.32:0/832524302 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa6ac068380 0x7fa6ac06a840 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.585+0000 7fa6d5a00640 1 --2- 172.21.15.32:0/832524302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d00cf990 0x7fa6d0143dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.585+0000 7fa6d5a00640 1 --2- 172.21.15.32:0/832524302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d00cf5b0 0x7fa6d00b42e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.585+0000 7fa6d5a00640 1 --2- 172.21.15.32:0/832524302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6d006f250 0x7fa6d0145b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.585+0000 7fa6d5a00640 1 -- 172.21.15.32:0/832524302 >> 172.21.15.32:0/832524302 conn(0x7fa6d00d0180 msgr2=0x7fa6d006c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:07.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.585+0000 7fa6d5a00640 1 -- 172.21.15.32:0/832524302 shutdown_connections 2024-04-17T09:55:07.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.585+0000 7fa6d5a00640 1 -- 172.21.15.32:0/832524302 wait complete. 2024-04-17T09:55:07.606 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T09:55:07.606 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T09:55:07.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92d9200640 1 Processor -- start 2024-04-17T09:55:07.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92d9200640 1 -- start start 2024-04-17T09:55:07.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92d9200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d40cd580 0x7f92d4167560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:07.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92d9200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92d40cdac0 con 0x7f92d4072500 2024-04-17T09:55:07.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92d9200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92d415cdf0 con 0x7f92d4145f30 2024-04-17T09:55:07.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92d9200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f92d406f400 con 0x7f92d40cd580 2024-04-17T09:55:07.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92c9400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f92d4072500 0x7f92d40cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33142/0 (socket says 172.21.15.32:33142) 2024-04-17T09:55:07.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92c9400640 1 -- 172.21.15.32:0/168375586 learned_addr learned my addr 172.21.15.32:0/168375586 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:07.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92d2800640 1 --2- 172.21.15.32:0/168375586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d40cd580 0x7f92d4167560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:07.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92d1400640 1 -- 172.21.15.32:0/168375586 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3835876157 0 0) 0x7f92d40cdac0 con 0x7f92d4072500 2024-04-17T09:55:07.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92d1400640 1 -- 172.21.15.32:0/168375586 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92b8003530 con 0x7f92d4072500 2024-04-17T09:55:07.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92d1400640 1 -- 172.21.15.32:0/168375586 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2809685541 0 0) 0x7f92b8003530 con 0x7f92d4072500 2024-04-17T09:55:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92d1400640 1 -- 172.21.15.32:0/168375586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d40cd580 msgr2=0x7f92d4167560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92d1400640 1 --2- 172.21.15.32:0/168375586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d40cd580 0x7f92d4167560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92d1400640 1 -- 172.21.15.32:0/168375586 >> v1:172.21.15.144:6789/0 conn(0x7f92d4145f30 legacy=0x7f92d4071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92d1400640 1 -- 172.21.15.32:0/168375586 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d415d0d0 con 0x7f92d4072500 2024-04-17T09:55:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92d1400640 1 -- 172.21.15.32:0/168375586 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f92bc001a20 con 0x7f92d4072500 2024-04-17T09:55:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92d1400640 1 -- 172.21.15.32:0/168375586 >> v1:172.21.15.32:6789/0 conn(0x7f92d4072500 legacy=0x7f92d40cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92d1400640 1 --2- 172.21.15.32:0/168375586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92b80039f0 0x7f92b8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92d1400640 1 --2- 172.21.15.32:0/168375586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d40cd580 0x7f92b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92d1400640 1 --2- 172.21.15.32:0/168375586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b8004d50 0x7f92b80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92d1400640 1 -- 172.21.15.32:0/168375586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f92d406f400 con 0x7f92b8004d50 2024-04-17T09:55:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92d1400640 1 -- 172.21.15.32:0/168375586 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f92b800bb40 con 0x7f92b80039f0 2024-04-17T09:55:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.745+0000 7f92d1400640 1 -- 172.21.15.32:0/168375586 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f92b800bd40 con 0x7f92d40cd580 2024-04-17T09:55:07.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.749+0000 7f92d2800640 1 --2- 172.21.15.32:0/168375586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b8004d50 0x7f92b80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:07.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.749+0000 7f92d1e00640 1 --2- 172.21.15.32:0/168375586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92b80039f0 0x7f92b8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:07.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.749+0000 7f92c9400640 1 --2- 172.21.15.32:0/168375586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d40cd580 0x7f92b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:07.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.749+0000 7f92d2800640 1 -- 172.21.15.32:0/168375586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d40cd580 msgr2=0x7f92b8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:07.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.749+0000 7f92d2800640 1 --2- 172.21.15.32:0/168375586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d40cd580 0x7f92b8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.749+0000 7f92d2800640 1 -- 172.21.15.32:0/168375586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92b80039f0 msgr2=0x7f92b8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:07.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.749+0000 7f92d2800640 1 --2- 172.21.15.32:0/168375586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92b80039f0 0x7f92b8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.749+0000 7f92d2800640 1 -- 172.21.15.32:0/168375586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b8003530 con 0x7f92b8004d50 2024-04-17T09:55:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.749+0000 7f92d2800640 1 --2- 172.21.15.32:0/168375586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b8004d50 0x7f92b80051d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009ab0 tx=0x7f92cc01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88758c0f98a8c27f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.749+0000 7f92d1400640 1 -- 172.21.15.32:0/168375586 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc0b2070 con 0x7f92b8004d50 2024-04-17T09:55:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.749+0000 7f92d1400640 1 -- 172.21.15.32:0/168375586 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92cc0a5640 con 0x7f92b8004d50 2024-04-17T09:55:07.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.749+0000 7f92d1400640 1 -- 172.21.15.32:0/168375586 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc0096b0 con 0x7f92b8004d50 2024-04-17T09:55:07.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d9200640 1 -- 172.21.15.32:0/168375586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b8004d50 msgr2=0x7f92b80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:07.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d9200640 1 --2- 172.21.15.32:0/168375586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b8004d50 0x7f92b80051d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009ab0 tx=0x7f92cc01ca50 comp rx=0 tx=0).stop 2024-04-17T09:55:07.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d2800640 1 -- 172.21.15.32:0/168375586 reap_dead start 2024-04-17T09:55:07.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d9200640 1 -- 172.21.15.32:0/168375586 shutdown_connections 2024-04-17T09:55:07.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d9200640 1 -- 172.21.15.32:0/168375586 >> 172.21.15.32:0/168375586 conn(0x7f92d406ba30 msgr2=0x7f92d406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:07.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d9200640 1 -- 172.21.15.32:0/168375586 shutdown_connections 2024-04-17T09:55:07.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d9200640 1 -- 172.21.15.32:0/168375586 wait complete. 2024-04-17T09:55:07.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d9200640 1 Processor -- start 2024-04-17T09:55:07.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d9200640 1 -- start start 2024-04-17T09:55:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d9200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b8004d50 0x7f92d40b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d9200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d40c9e60 0x7f92d40ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d9200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d40c5d10 0x7f92d40c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d9200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f92d4146310 con 0x7f92b8004d50 2024-04-17T09:55:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d9200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f92d415dba0 con 0x7f92d40c5d10 2024-04-17T09:55:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d9200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f92d415d440 con 0x7f92d40c9e60 2024-04-17T09:55:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b8004d50 0x7f92d40b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b8004d50 0x7f92d40b1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54014/0 (socket says 172.21.15.32:54014) 2024-04-17T09:55:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d1e00640 1 -- 172.21.15.32:0/806757202 learned_addr learned my addr 172.21.15.32:0/806757202 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92c9400640 1 --2- 172.21.15.32:0/806757202 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d40c9e60 0x7f92d40ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d2800640 1 --2- 172.21.15.32:0/806757202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d40c5d10 0x7f92d40c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d1e00640 1 -- 172.21.15.32:0/806757202 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d40c9e60 msgr2=0x7f92d40ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d1e00640 1 --2- 172.21.15.32:0/806757202 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d40c9e60 0x7f92d40ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d1e00640 1 -- 172.21.15.32:0/806757202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d40c5d10 msgr2=0x7f92d40c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d1e00640 1 --2- 172.21.15.32:0/806757202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d40c5d10 0x7f92d40c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.753+0000 7f92d1e00640 1 -- 172.21.15.32:0/806757202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d40ca820 con 0x7f92b8004d50 2024-04-17T09:55:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.757+0000 7f92d1e00640 1 --2- 172.21.15.32:0/806757202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b8004d50 0x7f92d40b1870 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f92c400f820 tx=0x7f92c400fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.757+0000 7f92caa00640 1 -- 172.21.15.32:0/806757202 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c40050d0 con 0x7f92b8004d50 2024-04-17T09:55:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.757+0000 7f92caa00640 1 -- 172.21.15.32:0/806757202 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92c4018070 con 0x7f92b8004d50 2024-04-17T09:55:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.757+0000 7f92caa00640 1 -- 172.21.15.32:0/806757202 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c400d730 con 0x7f92b8004d50 2024-04-17T09:55:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.757+0000 7f92d9200640 1 -- 172.21.15.32:0/806757202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d40c66d0 con 0x7f92b8004d50 2024-04-17T09:55:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.757+0000 7f92d9200640 1 -- 172.21.15.32:0/806757202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d40c6980 con 0x7f92b8004d50 2024-04-17T09:55:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.757+0000 7f92d9200640 1 -- 172.21.15.32:0/806757202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92d4068a90 con 0x7f92b8004d50 2024-04-17T09:55:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.757+0000 7f92caa00640 1 -- 172.21.15.32:0/806757202 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f92c400d8d0 con 0x7f92b8004d50 2024-04-17T09:55:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.757+0000 7f92caa00640 1 --2- 172.21.15.32:0/806757202 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f92b0068380 0x7f92b006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.757+0000 7f92caa00640 1 -- 172.21.15.32:0/806757202 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f92c4086660 con 0x7f92b8004d50 2024-04-17T09:55:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.757+0000 7f92c9400640 1 --2- 172.21.15.32:0/806757202 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f92b0068380 0x7f92b006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:07.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.761+0000 7f92c9400640 1 --2- 172.21.15.32:0/806757202 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f92b0068380 0x7f92b006a840 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f92d40cf730 tx=0x7f92bc006040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:07.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.761+0000 7f92caa00640 1 -- 172.21.15.32:0/806757202 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f92c4056660 con 0x7f92b8004d50 2024-04-17T09:55:07.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.929+0000 7f92d9200640 1 -- 172.21.15.32:0/806757202 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f92d4072f90 con 0x7f92b0068380 2024-04-17T09:55:07.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.933+0000 7f92caa00640 1 -- 172.21.15.32:0/806757202 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251917 (secure 0 0 0) 0x7f92d4072f90 con 0x7f92b0068380 2024-04-17T09:55:07.939 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:07.940 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T09:55:07.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.937+0000 7f92d9200640 1 -- 172.21.15.32:0/806757202 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f92b0068380 msgr2=0x7f92b006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:07.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.937+0000 7f92d9200640 1 --2- 172.21.15.32:0/806757202 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f92b0068380 0x7f92b006a840 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f92d40cf730 tx=0x7f92bc006040 comp rx=0 tx=0).stop 2024-04-17T09:55:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.937+0000 7f92d9200640 1 -- 172.21.15.32:0/806757202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b8004d50 msgr2=0x7f92d40b1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.937+0000 7f92d9200640 1 --2- 172.21.15.32:0/806757202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b8004d50 0x7f92d40b1870 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f92c400f820 tx=0x7f92c400fce0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.937+0000 7f92d9200640 1 -- 172.21.15.32:0/806757202 shutdown_connections 2024-04-17T09:55:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.937+0000 7f92d9200640 1 --2- 172.21.15.32:0/806757202 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f92b0068380 0x7f92b006a840 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.937+0000 7f92d9200640 1 --2- 172.21.15.32:0/806757202 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d40c9e60 0x7f92d40ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.937+0000 7f92d9200640 1 --2- 172.21.15.32:0/806757202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d40c5d10 0x7f92d40c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.937+0000 7f92d9200640 1 --2- 172.21.15.32:0/806757202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b8004d50 0x7f92d40b1870 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.937+0000 7f92d9200640 1 -- 172.21.15.32:0/806757202 >> 172.21.15.32:0/806757202 conn(0x7f92d406ba30 msgr2=0x7f92d415f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.937+0000 7f92d9200640 1 -- 172.21.15.32:0/806757202 shutdown_connections 2024-04-17T09:55:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:07.937+0000 7f92d9200640 1 -- 172.21.15.32:0/806757202 wait complete. 2024-04-17T09:55:07.959 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-04-17T09:55:07.307762+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242276,"kb_used_data":5296,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732444,"statfs":{"total":773094113280,"available":772846022656,"internally_reserved":0,"allocated":5423104,"data_stored":2871891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.490770"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.852797+0000","last_change":"2024-04-17T09:55:03.849325+0000","last_active":"2024-04-17T09:55:03.852797+0000","last_peered":"2024-04-17T09:55:03.852797+0000","last_clean":"2024-04-17T09:55:03.852797+0000","last_became_active":"2024-04-17T09:55:03.848939+0000","last_became_peered":"2024-04-17T09:55:03.848939+0000","last_unstale":"2024-04-17T09:55:03.852797+0000","last_undegraded":"2024-04-17T09:55:03.852797+0000","last_fullsized":"2024-04-17T09:55:03.852797+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:31.009742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.856176+0000","last_change":"2024-04-17T09:55:03.854965+0000","last_active":"2024-04-17T09:55:03.856176+0000","last_peered":"2024-04-17T09:55:03.856176+0000","last_clean":"2024-04-17T09:55:03.856176+0000","last_became_active":"2024-04-17T09:55:03.854644+0000","last_became_peered":"2024-04-17T09:55:03.854644+0000","last_unstale":"2024-04-17T09:55:03.856176+0000","last_undegraded":"2024-04-17T09:55:03.856176+0000","last_fullsized":"2024-04-17T09:55:03.856176+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:07:26.161323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.871640+0000","last_change":"2024-04-17T09:55:03.868111+0000","last_active":"2024-04-17T09:55:03.871640+0000","last_peered":"2024-04-17T09:55:03.871640+0000","last_clean":"2024-04-17T09:55:03.871640+0000","last_became_active":"2024-04-17T09:55:03.865910+0000","last_became_peered":"2024-04-17T09:55:03.865910+0000","last_unstale":"2024-04-17T09:55:03.871640+0000","last_undegraded":"2024-04-17T09:55:03.871640+0000","last_fullsized":"2024-04-17T09:55:03.871640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:38:04.471407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.862003+0000","last_change":"2024-04-17T09:55:03.856899+0000","last_active":"2024-04-17T09:55:03.862003+0000","last_peered":"2024-04-17T09:55:03.862003+0000","last_clean":"2024-04-17T09:55:03.862003+0000","last_became_active":"2024-04-17T09:55:03.855227+0000","last_became_peered":"2024-04-17T09:55:03.855227+0000","last_unstale":"2024-04-17T09:55:03.862003+0000","last_undegraded":"2024-04-17T09:55:03.862003+0000","last_fullsized":"2024-04-17T09:55:03.862003+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:08.824623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.857302+0000","last_change":"2024-04-17T09:55:03.855439+0000","last_active":"2024-04-17T09:55:03.857302+0000","last_peered":"2024-04-17T09:55:03.857302+0000","last_clean":"2024-04-17T09:55:03.857302+0000","last_became_active":"2024-04-17T09:55:03.855126+0000","last_became_peered":"2024-04-17T09:55:03.855126+0000","last_unstale":"2024-04-17T09:55:03.857302+0000","last_undegraded":"2024-04-17T09:55:03.857302+0000","last_fullsized":"2024-04-17T09:55:03.857302+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:17:03.025913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.855691+0000","last_change":"2024-04-17T09:55:03.851400+0000","last_active":"2024-04-17T09:55:04.855691+0000","last_peered":"2024-04-17T09:55:04.855691+0000","last_clean":"2024-04-17T09:55:04.855691+0000","last_became_active":"2024-04-17T09:55:03.851119+0000","last_became_peered":"2024-04-17T09:55:03.851119+0000","last_unstale":"2024-04-17T09:55:04.855691+0000","last_undegraded":"2024-04-17T09:55:04.855691+0000","last_fullsized":"2024-04-17T09:55:04.855691+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:13:10.584024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.844135+0000","last_change":"2024-04-17T09:55:03.841822+0000","last_active":"2024-04-17T09:55:03.844135+0000","last_peered":"2024-04-17T09:55:03.844135+0000","last_clean":"2024-04-17T09:55:03.844135+0000","last_became_active":"2024-04-17T09:55:03.841536+0000","last_became_peered":"2024-04-17T09:55:03.841536+0000","last_unstale":"2024-04-17T09:55:03.844135+0000","last_undegraded":"2024-04-17T09:55:03.844135+0000","last_fullsized":"2024-04-17T09:55:03.844135+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:46:54.551275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.862354+0000","last_change":"2024-04-17T09:55:03.860673+0000","last_active":"2024-04-17T09:55:03.862354+0000","last_peered":"2024-04-17T09:55:03.862354+0000","last_clean":"2024-04-17T09:55:03.862354+0000","last_became_active":"2024-04-17T09:55:03.855522+0000","last_became_peered":"2024-04-17T09:55:03.855522+0000","last_unstale":"2024-04-17T09:55:03.862354+0000","last_undegraded":"2024-04-17T09:55:03.862354+0000","last_fullsized":"2024-04-17T09:55:03.862354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:38:40.946652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.861315+0000","last_change":"2024-04-17T09:55:03.853563+0000","last_active":"2024-04-17T09:55:03.861315+0000","last_peered":"2024-04-17T09:55:03.861315+0000","last_clean":"2024-04-17T09:55:03.861315+0000","last_became_active":"2024-04-17T09:55:03.853314+0000","last_became_peered":"2024-04-17T09:55:03.853314+0000","last_unstale":"2024-04-17T09:55:03.861315+0000","last_undegraded":"2024-04-17T09:55:03.861315+0000","last_fullsized":"2024-04-17T09:55:03.861315+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:14:17.006620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.862452+0000","last_change":"2024-04-17T09:55:03.860578+0000","last_active":"2024-04-17T09:55:03.862452+0000","last_peered":"2024-04-17T09:55:03.862452+0000","last_clean":"2024-04-17T09:55:03.862452+0000","last_became_active":"2024-04-17T09:55:03.855286+0000","last_became_peered":"2024-04-17T09:55:03.855286+0000","last_unstale":"2024-04-17T09:55:03.862452+0000","last_undegraded":"2024-04-17T09:55:03.862452+0000","last_fullsized":"2024-04-17T09:55:03.862452+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:31:41.399858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.856676+0000","last_change":"2024-04-17T09:55:03.850950+0000","last_active":"2024-04-17T09:55:03.856676+0000","last_peered":"2024-04-17T09:55:03.856676+0000","last_clean":"2024-04-17T09:55:03.856676+0000","last_became_active":"2024-04-17T09:55:03.850432+0000","last_became_peered":"2024-04-17T09:55:03.850432+0000","last_unstale":"2024-04-17T09:55:03.856676+0000","last_undegraded":"2024-04-17T09:55:03.856676+0000","last_fullsized":"2024-04-17T09:55:03.856676+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:03:57.004315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.859804+0000","last_change":"2024-04-17T09:55:03.858317+0000","last_active":"2024-04-17T09:55:03.859804+0000","last_peered":"2024-04-17T09:55:03.859804+0000","last_clean":"2024-04-17T09:55:03.859804+0000","last_became_active":"2024-04-17T09:55:03.856743+0000","last_became_peered":"2024-04-17T09:55:03.856743+0000","last_unstale":"2024-04-17T09:55:03.859804+0000","last_undegraded":"2024-04-17T09:55:03.859804+0000","last_fullsized":"2024-04-17T09:55:03.859804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:21.677589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.842710+0000","last_change":"2024-04-17T09:55:03.846602+0000","last_active":"2024-04-17T09:55:04.842710+0000","last_peered":"2024-04-17T09:55:04.842710+0000","last_clean":"2024-04-17T09:55:04.842710+0000","last_became_active":"2024-04-17T09:55:03.846292+0000","last_became_peered":"2024-04-17T09:55:03.846292+0000","last_unstale":"2024-04-17T09:55:04.842710+0000","last_undegraded":"2024-04-17T09:55:04.842710+0000","last_fullsized":"2024-04-17T09:55:04.842710+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:00.891807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.856023+0000","last_change":"2024-04-17T09:55:03.844840+0000","last_active":"2024-04-17T09:55:03.856023+0000","last_peered":"2024-04-17T09:55:03.856023+0000","last_clean":"2024-04-17T09:55:03.856023+0000","last_became_active":"2024-04-17T09:55:03.844467+0000","last_became_peered":"2024-04-17T09:55:03.844467+0000","last_unstale":"2024-04-17T09:55:03.856023+0000","last_undegraded":"2024-04-17T09:55:03.856023+0000","last_fullsized":"2024-04-17T09:55:03.856023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:24:22.826472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.855748+0000","last_change":"2024-04-17T09:55:03.856578+0000","last_active":"2024-04-17T09:55:04.855748+0000","last_peered":"2024-04-17T09:55:04.855748+0000","last_clean":"2024-04-17T09:55:04.855748+0000","last_became_active":"2024-04-17T09:55:03.853231+0000","last_became_peered":"2024-04-17T09:55:03.853231+0000","last_unstale":"2024-04-17T09:55:04.855748+0000","last_undegraded":"2024-04-17T09:55:04.855748+0000","last_fullsized":"2024-04-17T09:55:04.855748+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:22:46.796157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.842765+0000","last_change":"2024-04-17T09:55:03.854152+0000","last_active":"2024-04-17T09:55:04.842765+0000","last_peered":"2024-04-17T09:55:04.842765+0000","last_clean":"2024-04-17T09:55:04.842765+0000","last_became_active":"2024-04-17T09:55:03.853866+0000","last_became_peered":"2024-04-17T09:55:03.853866+0000","last_unstale":"2024-04-17T09:55:04.842765+0000","last_undegraded":"2024-04-17T09:55:04.842765+0000","last_fullsized":"2024-04-17T09:55:04.842765+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:07.056555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.870064+0000","last_change":"2024-04-17T09:55:03.860757+0000","last_active":"2024-04-17T09:55:03.870064+0000","last_peered":"2024-04-17T09:55:03.870064+0000","last_clean":"2024-04-17T09:55:03.870064+0000","last_became_active":"2024-04-17T09:55:03.860460+0000","last_became_peered":"2024-04-17T09:55:03.860460+0000","last_unstale":"2024-04-17T09:55:03.870064+0000","last_undegraded":"2024-04-17T09:55:03.870064+0000","last_fullsized":"2024-04-17T09:55:03.870064+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:09:55.354856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.868722+0000","last_change":"2024-04-17T09:55:03.865594+0000","last_active":"2024-04-17T09:55:03.868722+0000","last_peered":"2024-04-17T09:55:03.868722+0000","last_clean":"2024-04-17T09:55:03.868722+0000","last_became_active":"2024-04-17T09:55:03.865109+0000","last_became_peered":"2024-04-17T09:55:03.865109+0000","last_unstale":"2024-04-17T09:55:03.868722+0000","last_undegraded":"2024-04-17T09:55:03.868722+0000","last_fullsized":"2024-04-17T09:55:03.868722+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:14:17.006620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.843851+0000","last_change":"2024-04-17T09:55:03.854194+0000","last_active":"2024-04-17T09:55:04.843851+0000","last_peered":"2024-04-17T09:55:04.843851+0000","last_clean":"2024-04-17T09:55:04.843851+0000","last_became_active":"2024-04-17T09:55:03.853132+0000","last_became_peered":"2024-04-17T09:55:03.853132+0000","last_unstale":"2024-04-17T09:55:04.843851+0000","last_undegraded":"2024-04-17T09:55:04.843851+0000","last_fullsized":"2024-04-17T09:55:04.843851+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:55:53.473939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.842847+0000","last_change":"2024-04-17T09:55:03.846415+0000","last_active":"2024-04-17T09:55:04.842847+0000","last_peered":"2024-04-17T09:55:04.842847+0000","last_clean":"2024-04-17T09:55:04.842847+0000","last_became_active":"2024-04-17T09:55:03.846116+0000","last_became_peered":"2024-04-17T09:55:03.846116+0000","last_unstale":"2024-04-17T09:55:04.842847+0000","last_undegraded":"2024-04-17T09:55:04.842847+0000","last_fullsized":"2024-04-17T09:55:04.842847+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:28:58.768665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.855769+0000","last_change":"2024-04-17T09:55:03.856297+0000","last_active":"2024-04-17T09:55:04.855769+0000","last_peered":"2024-04-17T09:55:04.855769+0000","last_clean":"2024-04-17T09:55:04.855769+0000","last_became_active":"2024-04-17T09:55:03.855956+0000","last_became_peered":"2024-04-17T09:55:03.855956+0000","last_unstale":"2024-04-17T09:55:04.855769+0000","last_undegraded":"2024-04-17T09:55:04.855769+0000","last_fullsized":"2024-04-17T09:55:04.855769+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:52:31.651115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.861282+0000","last_change":"2024-04-17T09:55:03.855335+0000","last_active":"2024-04-17T09:55:03.861282+0000","last_peered":"2024-04-17T09:55:03.861282+0000","last_clean":"2024-04-17T09:55:03.861282+0000","last_became_active":"2024-04-17T09:55:03.854983+0000","last_became_peered":"2024-04-17T09:55:03.854983+0000","last_unstale":"2024-04-17T09:55:03.861282+0000","last_undegraded":"2024-04-17T09:55:03.861282+0000","last_fullsized":"2024-04-17T09:55:03.861282+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:38:04.471407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.843242+0000","last_change":"2024-04-17T09:55:03.836067+0000","last_active":"2024-04-17T09:55:04.843242+0000","last_peered":"2024-04-17T09:55:04.843242+0000","last_clean":"2024-04-17T09:55:04.843242+0000","last_became_active":"2024-04-17T09:55:03.835767+0000","last_became_peered":"2024-04-17T09:55:03.835767+0000","last_unstale":"2024-04-17T09:55:04.843242+0000","last_undegraded":"2024-04-17T09:55:04.843242+0000","last_fullsized":"2024-04-17T09:55:04.843242+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:59:44.714355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.842777+0000","last_change":"2024-04-17T09:55:03.846631+0000","last_active":"2024-04-17T09:55:04.842777+0000","last_peered":"2024-04-17T09:55:04.842777+0000","last_clean":"2024-04-17T09:55:04.842777+0000","last_became_active":"2024-04-17T09:55:03.846395+0000","last_became_peered":"2024-04-17T09:55:03.846395+0000","last_unstale":"2024-04-17T09:55:04.842777+0000","last_undegraded":"2024-04-17T09:55:04.842777+0000","last_fullsized":"2024-04-17T09:55:04.842777+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:24:11.783113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.843925+0000","last_change":"2024-04-17T09:55:03.848777+0000","last_active":"2024-04-17T09:55:04.843925+0000","last_peered":"2024-04-17T09:55:04.843925+0000","last_clean":"2024-04-17T09:55:04.843925+0000","last_became_active":"2024-04-17T09:55:03.848529+0000","last_became_peered":"2024-04-17T09:55:03.848529+0000","last_unstale":"2024-04-17T09:55:04.843925+0000","last_undegraded":"2024-04-17T09:55:04.843925+0000","last_fullsized":"2024-04-17T09:55:04.843925+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:57:06.558212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.842903+0000","last_change":"2024-04-17T09:55:03.845536+0000","last_active":"2024-04-17T09:55:04.842903+0000","last_peered":"2024-04-17T09:55:04.842903+0000","last_clean":"2024-04-17T09:55:04.842903+0000","last_became_active":"2024-04-17T09:55:03.845265+0000","last_became_peered":"2024-04-17T09:55:03.845265+0000","last_unstale":"2024-04-17T09:55:04.842903+0000","last_undegraded":"2024-04-17T09:55:04.842903+0000","last_fullsized":"2024-04-17T09:55:04.842903+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:47:33.674100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.871661+0000","last_change":"2024-04-17T09:55:03.868089+0000","last_active":"2024-04-17T09:55:03.871661+0000","last_peered":"2024-04-17T09:55:03.871661+0000","last_clean":"2024-04-17T09:55:03.871661+0000","last_became_active":"2024-04-17T09:55:03.866411+0000","last_became_peered":"2024-04-17T09:55:03.866411+0000","last_unstale":"2024-04-17T09:55:03.871661+0000","last_undegraded":"2024-04-17T09:55:03.871661+0000","last_fullsized":"2024-04-17T09:55:03.871661+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:35:24.615538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.855955+0000","last_change":"2024-04-17T09:55:03.849541+0000","last_active":"2024-04-17T09:55:03.855955+0000","last_peered":"2024-04-17T09:55:03.855955+0000","last_clean":"2024-04-17T09:55:03.855955+0000","last_became_active":"2024-04-17T09:55:03.849049+0000","last_became_peered":"2024-04-17T09:55:03.849049+0000","last_unstale":"2024-04-17T09:55:03.855955+0000","last_undegraded":"2024-04-17T09:55:03.855955+0000","last_fullsized":"2024-04-17T09:55:03.855955+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:26:02.753909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.843084+0000","last_change":"2024-04-17T09:55:03.849935+0000","last_active":"2024-04-17T09:55:04.843084+0000","last_peered":"2024-04-17T09:55:04.843084+0000","last_clean":"2024-04-17T09:55:04.843084+0000","last_became_active":"2024-04-17T09:55:03.849205+0000","last_became_peered":"2024-04-17T09:55:03.849205+0000","last_unstale":"2024-04-17T09:55:04.843084+0000","last_undegraded":"2024-04-17T09:55:04.843084+0000","last_fullsized":"2024-04-17T09:55:04.843084+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:23.883842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.865766+0000","last_change":"2024-04-17T09:55:03.859935+0000","last_active":"2024-04-17T09:55:03.865766+0000","last_peered":"2024-04-17T09:55:03.865766+0000","last_clean":"2024-04-17T09:55:03.865766+0000","last_became_active":"2024-04-17T09:55:03.859643+0000","last_became_peered":"2024-04-17T09:55:03.859643+0000","last_unstale":"2024-04-17T09:55:03.865766+0000","last_undegraded":"2024-04-17T09:55:03.865766+0000","last_fullsized":"2024-04-17T09:55:03.865766+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:46:05.201092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.855684+0000","last_change":"2024-04-17T09:55:03.856787+0000","last_active":"2024-04-17T09:55:04.855684+0000","last_peered":"2024-04-17T09:55:04.855684+0000","last_clean":"2024-04-17T09:55:04.855684+0000","last_became_active":"2024-04-17T09:55:03.856343+0000","last_became_peered":"2024-04-17T09:55:03.856343+0000","last_unstale":"2024-04-17T09:55:04.855684+0000","last_undegraded":"2024-04-17T09:55:04.855684+0000","last_fullsized":"2024-04-17T09:55:04.855684+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:26:06.903831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.842686+0000","last_change":"2024-04-17T09:55:03.845403+0000","last_active":"2024-04-17T09:55:04.842686+0000","last_peered":"2024-04-17T09:55:04.842686+0000","last_clean":"2024-04-17T09:55:04.842686+0000","last_became_active":"2024-04-17T09:55:03.845135+0000","last_became_peered":"2024-04-17T09:55:03.845135+0000","last_unstale":"2024-04-17T09:55:04.842686+0000","last_undegraded":"2024-04-17T09:55:04.842686+0000","last_fullsized":"2024-04-17T09:55:04.842686+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:15:49.139439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.859065+0000","last_change":"2024-04-17T09:55:03.849511+0000","last_active":"2024-04-17T09:55:03.859065+0000","last_peered":"2024-04-17T09:55:03.859065+0000","last_clean":"2024-04-17T09:55:03.859065+0000","last_became_active":"2024-04-17T09:55:03.849205+0000","last_became_peered":"2024-04-17T09:55:03.849205+0000","last_unstale":"2024-04-17T09:55:03.859065+0000","last_undegraded":"2024-04-17T09:55:03.859065+0000","last_fullsized":"2024-04-17T09:55:03.859065+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:56:23.155554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.856230+0000","last_change":"2024-04-17T09:55:02.849881+0000","last_active":"2024-04-17T09:55:04.856230+0000","last_peered":"2024-04-17T09:55:04.856230+0000","last_clean":"2024-04-17T09:55:04.856230+0000","last_became_active":"2024-04-17T09:55:02.849598+0000","last_became_peered":"2024-04-17T09:55:02.849598+0000","last_unstale":"2024-04-17T09:55:04.856230+0000","last_undegraded":"2024-04-17T09:55:04.856230+0000","last_fullsized":"2024-04-17T09:55:04.856230+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:08.775226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.844068+0000","last_change":"2024-04-17T09:55:03.851114+0000","last_active":"2024-04-17T09:55:04.844068+0000","last_peered":"2024-04-17T09:55:04.844068+0000","last_clean":"2024-04-17T09:55:04.844068+0000","last_became_active":"2024-04-17T09:55:03.850826+0000","last_became_peered":"2024-04-17T09:55:03.850826+0000","last_unstale":"2024-04-17T09:55:04.844068+0000","last_undegraded":"2024-04-17T09:55:04.844068+0000","last_fullsized":"2024-04-17T09:55:04.844068+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:26:02.753909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.872369+0000","last_change":"2024-04-17T09:55:02.838094+0000","last_active":"2024-04-17T09:55:03.872369+0000","last_peered":"2024-04-17T09:55:03.872369+0000","last_clean":"2024-04-17T09:55:03.872369+0000","last_became_active":"2024-04-17T09:55:02.837812+0000","last_became_peered":"2024-04-17T09:55:02.837812+0000","last_unstale":"2024-04-17T09:55:03.872369+0000","last_undegraded":"2024-04-17T09:55:03.872369+0000","last_fullsized":"2024-04-17T09:55:03.872369+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:02.061750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.855933+0000","last_change":"2024-04-17T09:55:03.849327+0000","last_active":"2024-04-17T09:55:03.855933+0000","last_peered":"2024-04-17T09:55:03.855933+0000","last_clean":"2024-04-17T09:55:03.855933+0000","last_became_active":"2024-04-17T09:55:03.848666+0000","last_became_peered":"2024-04-17T09:55:03.848666+0000","last_unstale":"2024-04-17T09:55:03.855933+0000","last_undegraded":"2024-04-17T09:55:03.855933+0000","last_fullsized":"2024-04-17T09:55:03.855933+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:24:35.433580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.869372+0000","last_change":"2024-04-17T09:55:02.848104+0000","last_active":"2024-04-17T09:55:03.869372+0000","last_peered":"2024-04-17T09:55:03.869372+0000","last_clean":"2024-04-17T09:55:03.869372+0000","last_became_active":"2024-04-17T09:55:02.847846+0000","last_became_peered":"2024-04-17T09:55:02.847846+0000","last_unstale":"2024-04-17T09:55:03.869372+0000","last_undegraded":"2024-04-17T09:55:03.869372+0000","last_fullsized":"2024-04-17T09:55:03.869372+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:28:57.762658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.858920+0000","last_change":"2024-04-17T09:55:03.848552+0000","last_active":"2024-04-17T09:55:03.858920+0000","last_peered":"2024-04-17T09:55:03.858920+0000","last_clean":"2024-04-17T09:55:03.858920+0000","last_became_active":"2024-04-17T09:55:03.848247+0000","last_became_peered":"2024-04-17T09:55:03.848247+0000","last_unstale":"2024-04-17T09:55:03.858920+0000","last_undegraded":"2024-04-17T09:55:03.858920+0000","last_fullsized":"2024-04-17T09:55:03.858920+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:24:11.783113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.869339+0000","last_change":"2024-04-17T09:55:02.849355+0000","last_active":"2024-04-17T09:55:03.869339+0000","last_peered":"2024-04-17T09:55:03.869339+0000","last_clean":"2024-04-17T09:55:03.869339+0000","last_became_active":"2024-04-17T09:55:02.849056+0000","last_became_peered":"2024-04-17T09:55:02.849056+0000","last_unstale":"2024-04-17T09:55:03.869339+0000","last_undegraded":"2024-04-17T09:55:03.869339+0000","last_fullsized":"2024-04-17T09:55:03.869339+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:17:02.019906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.860149+0000","last_change":"2024-04-17T09:55:03.852360+0000","last_active":"2024-04-17T09:55:03.860149+0000","last_peered":"2024-04-17T09:55:03.860149+0000","last_clean":"2024-04-17T09:55:03.860149+0000","last_became_active":"2024-04-17T09:55:03.852055+0000","last_became_peered":"2024-04-17T09:55:03.852055+0000","last_unstale":"2024-04-17T09:55:03.860149+0000","last_undegraded":"2024-04-17T09:55:03.860149+0000","last_fullsized":"2024-04-17T09:55:03.860149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:03:57.004315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.855919+0000","last_change":"2024-04-17T09:55:02.851447+0000","last_active":"2024-04-17T09:55:04.855919+0000","last_peered":"2024-04-17T09:55:04.855919+0000","last_clean":"2024-04-17T09:55:04.855919+0000","last_became_active":"2024-04-17T09:55:02.851014+0000","last_became_peered":"2024-04-17T09:55:02.851014+0000","last_unstale":"2024-04-17T09:55:04.855919+0000","last_undegraded":"2024-04-17T09:55:04.855919+0000","last_fullsized":"2024-04-17T09:55:04.855919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:07:25.155315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.857602+0000","last_change":"2024-04-17T09:55:03.850790+0000","last_active":"2024-04-17T09:55:03.857602+0000","last_peered":"2024-04-17T09:55:03.857602+0000","last_clean":"2024-04-17T09:55:03.857602+0000","last_became_active":"2024-04-17T09:55:03.849904+0000","last_became_peered":"2024-04-17T09:55:03.849904+0000","last_unstale":"2024-04-17T09:55:03.857602+0000","last_undegraded":"2024-04-17T09:55:03.857602+0000","last_fullsized":"2024-04-17T09:55:03.857602+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:54:04.483854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.872208+0000","last_change":"2024-04-17T09:55:02.842980+0000","last_active":"2024-04-17T09:55:03.872208+0000","last_peered":"2024-04-17T09:55:03.872208+0000","last_clean":"2024-04-17T09:55:03.872208+0000","last_became_active":"2024-04-17T09:55:02.842754+0000","last_became_peered":"2024-04-17T09:55:02.842754+0000","last_unstale":"2024-04-17T09:55:03.872208+0000","last_undegraded":"2024-04-17T09:55:03.872208+0000","last_fullsized":"2024-04-17T09:55:03.872208+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:21:41.297029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.860714+0000","last_change":"2024-04-17T09:55:03.854463+0000","last_active":"2024-04-17T09:55:03.860714+0000","last_peered":"2024-04-17T09:55:03.860714+0000","last_clean":"2024-04-17T09:55:03.860714+0000","last_became_active":"2024-04-17T09:55:03.854272+0000","last_became_peered":"2024-04-17T09:55:03.854272+0000","last_unstale":"2024-04-17T09:55:03.860714+0000","last_undegraded":"2024-04-17T09:55:03.860714+0000","last_fullsized":"2024-04-17T09:55:03.860714+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:35:24.615538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.834582+0000","last_change":"2024-04-17T09:55:02.846242+0000","last_active":"2024-04-17T09:55:03.834582+0000","last_peered":"2024-04-17T09:55:03.834582+0000","last_clean":"2024-04-17T09:55:03.834582+0000","last_became_active":"2024-04-17T09:55:02.845569+0000","last_became_peered":"2024-04-17T09:55:02.845569+0000","last_unstale":"2024-04-17T09:55:03.834582+0000","last_undegraded":"2024-04-17T09:55:03.834582+0000","last_fullsized":"2024-04-17T09:55:03.834582+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:14.024482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.843024+0000","last_change":"2024-04-17T09:55:03.845639+0000","last_active":"2024-04-17T09:55:04.843024+0000","last_peered":"2024-04-17T09:55:04.843024+0000","last_clean":"2024-04-17T09:55:04.843024+0000","last_became_active":"2024-04-17T09:55:03.845365+0000","last_became_peered":"2024-04-17T09:55:03.845365+0000","last_unstale":"2024-04-17T09:55:04.843024+0000","last_undegraded":"2024-04-17T09:55:04.843024+0000","last_fullsized":"2024-04-17T09:55:04.843024+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:56:23.155554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.842321+0000","last_change":"2024-04-17T09:55:02.846320+0000","last_active":"2024-04-17T09:55:03.842321+0000","last_peered":"2024-04-17T09:55:03.842321+0000","last_clean":"2024-04-17T09:55:03.842321+0000","last_became_active":"2024-04-17T09:55:02.846048+0000","last_became_peered":"2024-04-17T09:55:02.846048+0000","last_unstale":"2024-04-17T09:55:03.842321+0000","last_undegraded":"2024-04-17T09:55:03.842321+0000","last_fullsized":"2024-04-17T09:55:03.842321+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:21:13.601828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.868710+0000","last_change":"2024-04-17T09:55:03.865357+0000","last_active":"2024-04-17T09:55:03.868710+0000","last_peered":"2024-04-17T09:55:03.868710+0000","last_clean":"2024-04-17T09:55:03.868710+0000","last_became_active":"2024-04-17T09:55:03.864736+0000","last_became_peered":"2024-04-17T09:55:03.864736+0000","last_unstale":"2024-04-17T09:55:03.868710+0000","last_undegraded":"2024-04-17T09:55:03.868710+0000","last_fullsized":"2024-04-17T09:55:03.868710+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:20:50.772361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.846627+0000","last_change":"2024-04-17T09:55:02.840739+0000","last_active":"2024-04-17T09:55:03.846627+0000","last_peered":"2024-04-17T09:55:03.846627+0000","last_clean":"2024-04-17T09:55:03.846627+0000","last_became_active":"2024-04-17T09:55:02.840461+0000","last_became_peered":"2024-04-17T09:55:02.840461+0000","last_unstale":"2024-04-17T09:55:03.846627+0000","last_undegraded":"2024-04-17T09:55:03.846627+0000","last_fullsized":"2024-04-17T09:55:03.846627+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:24:21.820465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.852776+0000","last_change":"2024-04-17T09:55:03.847707+0000","last_active":"2024-04-17T09:55:03.852776+0000","last_peered":"2024-04-17T09:55:03.852776+0000","last_clean":"2024-04-17T09:55:03.852776+0000","last_became_active":"2024-04-17T09:55:03.847396+0000","last_became_peered":"2024-04-17T09:55:03.847396+0000","last_unstale":"2024-04-17T09:55:03.852776+0000","last_undegraded":"2024-04-17T09:55:03.852776+0000","last_fullsized":"2024-04-17T09:55:03.852776+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:19:43.980847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.851587+0000","last_change":"2024-04-17T09:55:02.839995+0000","last_active":"2024-04-17T09:55:03.851587+0000","last_peered":"2024-04-17T09:55:03.851587+0000","last_clean":"2024-04-17T09:55:03.851587+0000","last_became_active":"2024-04-17T09:55:02.839611+0000","last_became_peered":"2024-04-17T09:55:02.839611+0000","last_unstale":"2024-04-17T09:55:03.851587+0000","last_undegraded":"2024-04-17T09:55:03.851587+0000","last_fullsized":"2024-04-17T09:55:03.851587+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:46:04.195085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.856011+0000","last_change":"2024-04-17T09:55:03.850140+0000","last_active":"2024-04-17T09:55:04.856011+0000","last_peered":"2024-04-17T09:55:04.856011+0000","last_clean":"2024-04-17T09:55:04.856011+0000","last_became_active":"2024-04-17T09:55:03.849625+0000","last_became_peered":"2024-04-17T09:55:03.849625+0000","last_unstale":"2024-04-17T09:55:04.856011+0000","last_undegraded":"2024-04-17T09:55:04.856011+0000","last_fullsized":"2024-04-17T09:55:04.856011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:43:34.034038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.871985+0000","last_change":"2024-04-17T09:55:02.846604+0000","last_active":"2024-04-17T09:55:03.871985+0000","last_peered":"2024-04-17T09:55:03.871985+0000","last_clean":"2024-04-17T09:55:03.871985+0000","last_became_active":"2024-04-17T09:55:02.846351+0000","last_became_peered":"2024-04-17T09:55:02.846351+0000","last_unstale":"2024-04-17T09:55:03.871985+0000","last_undegraded":"2024-04-17T09:55:03.871985+0000","last_fullsized":"2024-04-17T09:55:03.871985+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:20.671582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.849102+0000","last_change":"2024-04-17T09:55:03.846805+0000","last_active":"2024-04-17T09:55:03.849102+0000","last_peered":"2024-04-17T09:55:03.849102+0000","last_clean":"2024-04-17T09:55:03.849102+0000","last_became_active":"2024-04-17T09:55:03.846502+0000","last_became_peered":"2024-04-17T09:55:03.846502+0000","last_unstale":"2024-04-17T09:55:03.849102+0000","last_undegraded":"2024-04-17T09:55:03.849102+0000","last_fullsized":"2024-04-17T09:55:03.849102+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:02.206573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.842855+0000","last_change":"2024-04-17T09:55:02.838983+0000","last_active":"2024-04-17T09:55:04.842855+0000","last_peered":"2024-04-17T09:55:04.842855+0000","last_clean":"2024-04-17T09:55:04.842855+0000","last_became_active":"2024-04-17T09:55:02.838432+0000","last_became_peered":"2024-04-17T09:55:02.838432+0000","last_unstale":"2024-04-17T09:55:04.842855+0000","last_undegraded":"2024-04-17T09:55:04.842855+0000","last_fullsized":"2024-04-17T09:55:04.842855+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:12:07.395491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.843635+0000","last_change":"2024-04-17T09:55:03.843497+0000","last_active":"2024-04-17T09:55:04.843635+0000","last_peered":"2024-04-17T09:55:04.843635+0000","last_clean":"2024-04-17T09:55:04.843635+0000","last_became_active":"2024-04-17T09:55:03.843185+0000","last_became_peered":"2024-04-17T09:55:03.843185+0000","last_unstale":"2024-04-17T09:55:04.843635+0000","last_undegraded":"2024-04-17T09:55:04.843635+0000","last_fullsized":"2024-04-17T09:55:04.843635+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:46:05.199714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.856029+0000","last_change":"2024-04-17T09:55:02.851638+0000","last_active":"2024-04-17T09:55:04.856029+0000","last_peered":"2024-04-17T09:55:04.856029+0000","last_clean":"2024-04-17T09:55:04.856029+0000","last_became_active":"2024-04-17T09:55:02.851391+0000","last_became_peered":"2024-04-17T09:55:02.851391+0000","last_unstale":"2024-04-17T09:55:04.856029+0000","last_undegraded":"2024-04-17T09:55:04.856029+0000","last_fullsized":"2024-04-17T09:55:04.856029+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:17:02.019906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.861559+0000","last_change":"2024-04-17T09:55:03.854380+0000","last_active":"2024-04-17T09:55:03.861559+0000","last_peered":"2024-04-17T09:55:03.861559+0000","last_clean":"2024-04-17T09:55:03.861559+0000","last_became_active":"2024-04-17T09:55:03.854044+0000","last_became_peered":"2024-04-17T09:55:03.854044+0000","last_unstale":"2024-04-17T09:55:03.861559+0000","last_undegraded":"2024-04-17T09:55:03.861559+0000","last_fullsized":"2024-04-17T09:55:03.861559+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:15:14.718813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.831177+0000","last_change":"2024-04-17T09:55:02.853317+0000","last_active":"2024-04-17T09:55:03.831177+0000","last_peered":"2024-04-17T09:55:03.831177+0000","last_clean":"2024-04-17T09:55:03.831177+0000","last_became_active":"2024-04-17T09:55:02.853057+0000","last_became_peered":"2024-04-17T09:55:02.853057+0000","last_unstale":"2024-04-17T09:55:03.831177+0000","last_undegraded":"2024-04-17T09:55:03.831177+0000","last_fullsized":"2024-04-17T09:55:03.831177+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:26:01.747901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.855670+0000","last_change":"2024-04-17T09:55:03.858259+0000","last_active":"2024-04-17T09:55:04.855670+0000","last_peered":"2024-04-17T09:55:04.855670+0000","last_clean":"2024-04-17T09:55:04.855670+0000","last_became_active":"2024-04-17T09:55:03.857952+0000","last_became_peered":"2024-04-17T09:55:03.857952+0000","last_unstale":"2024-04-17T09:55:04.855670+0000","last_undegraded":"2024-04-17T09:55:04.855670+0000","last_fullsized":"2024-04-17T09:55:04.855670+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:37.829815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.855792+0000","last_change":"2024-04-17T09:55:02.845545+0000","last_active":"2024-04-17T09:55:04.855792+0000","last_peered":"2024-04-17T09:55:04.855792+0000","last_clean":"2024-04-17T09:55:04.855792+0000","last_became_active":"2024-04-17T09:55:02.845226+0000","last_became_peered":"2024-04-17T09:55:02.845226+0000","last_unstale":"2024-04-17T09:55:04.855792+0000","last_undegraded":"2024-04-17T09:55:04.855792+0000","last_fullsized":"2024-04-17T09:55:04.855792+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:01.200566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.850476+0000","last_change":"2024-04-17T09:55:03.847514+0000","last_active":"2024-04-17T09:55:03.850476+0000","last_peered":"2024-04-17T09:55:03.850476+0000","last_clean":"2024-04-17T09:55:03.850476+0000","last_became_active":"2024-04-17T09:55:03.846942+0000","last_became_peered":"2024-04-17T09:55:03.846942+0000","last_unstale":"2024-04-17T09:55:03.850476+0000","last_undegraded":"2024-04-17T09:55:03.850476+0000","last_fullsized":"2024-04-17T09:55:03.850476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:09.781233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.875558+0000","last_change":"2024-04-17T09:55:02.845741+0000","last_active":"2024-04-17T09:55:03.875558+0000","last_peered":"2024-04-17T09:55:03.875558+0000","last_clean":"2024-04-17T09:55:03.875558+0000","last_became_active":"2024-04-17T09:55:02.845453+0000","last_became_peered":"2024-04-17T09:55:02.845453+0000","last_unstale":"2024-04-17T09:55:03.875558+0000","last_undegraded":"2024-04-17T09:55:03.875558+0000","last_fullsized":"2024-04-17T09:55:03.875558+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:21:41.297029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.843049+0000","last_change":"2024-04-17T09:55:03.843826+0000","last_active":"2024-04-17T09:55:04.843049+0000","last_peered":"2024-04-17T09:55:04.843049+0000","last_clean":"2024-04-17T09:55:04.843049+0000","last_became_active":"2024-04-17T09:55:03.843556+0000","last_became_peered":"2024-04-17T09:55:03.843556+0000","last_unstale":"2024-04-17T09:55:04.843049+0000","last_undegraded":"2024-04-17T09:55:04.843049+0000","last_fullsized":"2024-04-17T09:55:04.843049+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:07.056555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.842964+0000","last_change":"2024-04-17T09:55:02.851610+0000","last_active":"2024-04-17T09:55:04.842964+0000","last_peered":"2024-04-17T09:55:04.842964+0000","last_clean":"2024-04-17T09:55:04.842964+0000","last_became_active":"2024-04-17T09:55:02.851166+0000","last_became_peered":"2024-04-17T09:55:02.851166+0000","last_unstale":"2024-04-17T09:55:04.842964+0000","last_undegraded":"2024-04-17T09:55:04.842964+0000","last_fullsized":"2024-04-17T09:55:04.842964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:33:36.303801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.861291+0000","last_change":"2024-04-17T09:55:03.853543+0000","last_active":"2024-04-17T09:55:03.861291+0000","last_peered":"2024-04-17T09:55:03.861291+0000","last_clean":"2024-04-17T09:55:03.861291+0000","last_became_active":"2024-04-17T09:55:03.853269+0000","last_became_peered":"2024-04-17T09:55:03.853269+0000","last_unstale":"2024-04-17T09:55:03.861291+0000","last_undegraded":"2024-04-17T09:55:03.861291+0000","last_fullsized":"2024-04-17T09:55:03.861291+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:20:50.772361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.872455+0000","last_change":"2024-04-17T09:55:02.838898+0000","last_active":"2024-04-17T09:55:03.872455+0000","last_peered":"2024-04-17T09:55:03.872455+0000","last_clean":"2024-04-17T09:55:03.872455+0000","last_became_active":"2024-04-17T09:55:02.838259+0000","last_became_peered":"2024-04-17T09:55:02.838259+0000","last_unstale":"2024-04-17T09:55:03.872455+0000","last_undegraded":"2024-04-17T09:55:03.872455+0000","last_fullsized":"2024-04-17T09:55:03.872455+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:09:25.853640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.843393+0000","last_change":"2024-04-17T09:55:03.849813+0000","last_active":"2024-04-17T09:55:04.843393+0000","last_peered":"2024-04-17T09:55:04.843393+0000","last_clean":"2024-04-17T09:55:04.843393+0000","last_became_active":"2024-04-17T09:55:03.848297+0000","last_became_peered":"2024-04-17T09:55:03.848297+0000","last_unstale":"2024-04-17T09:55:04.843393+0000","last_undegraded":"2024-04-17T09:55:04.843393+0000","last_fullsized":"2024-04-17T09:55:04.843393+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:15.030489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.867862+0000","last_change":"2024-04-17T09:55:02.845754+0000","last_active":"2024-04-17T09:55:03.867862+0000","last_peered":"2024-04-17T09:55:03.867862+0000","last_clean":"2024-04-17T09:55:03.867862+0000","last_became_active":"2024-04-17T09:55:02.845238+0000","last_became_peered":"2024-04-17T09:55:02.845238+0000","last_unstale":"2024-04-17T09:55:03.867862+0000","last_undegraded":"2024-04-17T09:55:03.867862+0000","last_fullsized":"2024-04-17T09:55:03.867862+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:55:52.467932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.843739+0000","last_change":"2024-04-17T09:55:03.850786+0000","last_active":"2024-04-17T09:55:04.843739+0000","last_peered":"2024-04-17T09:55:04.843739+0000","last_clean":"2024-04-17T09:55:04.843739+0000","last_became_active":"2024-04-17T09:55:03.849305+0000","last_became_peered":"2024-04-17T09:55:03.849305+0000","last_unstale":"2024-04-17T09:55:04.843739+0000","last_undegraded":"2024-04-17T09:55:04.843739+0000","last_fullsized":"2024-04-17T09:55:04.843739+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:52:50.376225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.855975+0000","last_change":"2024-04-17T09:55:02.850448+0000","last_active":"2024-04-17T09:55:04.855975+0000","last_peered":"2024-04-17T09:55:04.855975+0000","last_clean":"2024-04-17T09:55:04.855975+0000","last_became_active":"2024-04-17T09:55:02.849631+0000","last_became_peered":"2024-04-17T09:55:02.849631+0000","last_unstale":"2024-04-17T09:55:04.855975+0000","last_undegraded":"2024-04-17T09:55:04.855975+0000","last_fullsized":"2024-04-17T09:55:04.855975+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:46:53.545268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.855766+0000","last_change":"2024-04-17T09:55:03.855613+0000","last_active":"2024-04-17T09:55:04.855766+0000","last_peered":"2024-04-17T09:55:04.855766+0000","last_clean":"2024-04-17T09:55:04.855766+0000","last_became_active":"2024-04-17T09:55:03.852617+0000","last_became_peered":"2024-04-17T09:55:03.852617+0000","last_unstale":"2024-04-17T09:55:04.855766+0000","last_undegraded":"2024-04-17T09:55:04.855766+0000","last_fullsized":"2024-04-17T09:55:04.855766+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:27:27.242594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.856133+0000","last_change":"2024-04-17T09:55:02.849110+0000","last_active":"2024-04-17T09:55:04.856133+0000","last_peered":"2024-04-17T09:55:04.856133+0000","last_clean":"2024-04-17T09:55:04.856133+0000","last_became_active":"2024-04-17T09:55:02.847524+0000","last_became_peered":"2024-04-17T09:55:02.847524+0000","last_unstale":"2024-04-17T09:55:04.856133+0000","last_undegraded":"2024-04-17T09:55:04.856133+0000","last_fullsized":"2024-04-17T09:55:04.856133+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:20.671582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.848967+0000","last_change":"2024-04-17T09:55:03.843367+0000","last_active":"2024-04-17T09:55:03.848967+0000","last_peered":"2024-04-17T09:55:03.848967+0000","last_clean":"2024-04-17T09:55:03.848967+0000","last_became_active":"2024-04-17T09:55:03.843054+0000","last_became_peered":"2024-04-17T09:55:03.843054+0000","last_unstale":"2024-04-17T09:55:03.848967+0000","last_undegraded":"2024-04-17T09:55:03.848967+0000","last_fullsized":"2024-04-17T09:55:03.848967+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:07.056555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.835648+0000","last_change":"2024-04-17T09:55:02.845764+0000","last_active":"2024-04-17T09:55:03.835648+0000","last_peered":"2024-04-17T09:55:03.835648+0000","last_clean":"2024-04-17T09:55:03.835648+0000","last_became_active":"2024-04-17T09:55:02.845436+0000","last_became_peered":"2024-04-17T09:55:02.845436+0000","last_unstale":"2024-04-17T09:55:03.835648+0000","last_undegraded":"2024-04-17T09:55:03.835648+0000","last_fullsized":"2024-04-17T09:55:03.835648+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:52:49.370217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.842642+0000","last_change":"2024-04-17T09:55:03.835558+0000","last_active":"2024-04-17T09:55:04.842642+0000","last_peered":"2024-04-17T09:55:04.842642+0000","last_clean":"2024-04-17T09:55:04.842642+0000","last_became_active":"2024-04-17T09:55:03.835292+0000","last_became_peered":"2024-04-17T09:55:03.835292+0000","last_unstale":"2024-04-17T09:55:04.842642+0000","last_undegraded":"2024-04-17T09:55:04.842642+0000","last_fullsized":"2024-04-17T09:55:04.842642+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:44:28.122478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.842509+0000","last_change":"2024-04-17T09:55:02.836905+0000","last_active":"2024-04-17T09:55:03.842509+0000","last_peered":"2024-04-17T09:55:03.842509+0000","last_clean":"2024-04-17T09:55:03.842509+0000","last_became_active":"2024-04-17T09:55:02.836560+0000","last_became_peered":"2024-04-17T09:55:02.836560+0000","last_unstale":"2024-04-17T09:55:03.842509+0000","last_undegraded":"2024-04-17T09:55:03.842509+0000","last_fullsized":"2024-04-17T09:55:03.842509+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:02.061750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.849751+0000","last_change":"2024-04-17T09:55:03.843921+0000","last_active":"2024-04-17T09:55:03.849751+0000","last_peered":"2024-04-17T09:55:03.849751+0000","last_clean":"2024-04-17T09:55:03.849751+0000","last_became_active":"2024-04-17T09:55:03.843719+0000","last_became_peered":"2024-04-17T09:55:03.843719+0000","last_unstale":"2024-04-17T09:55:03.849751+0000","last_undegraded":"2024-04-17T09:55:03.849751+0000","last_fullsized":"2024-04-17T09:55:03.849751+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:09:26.859647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.830893+0000","last_change":"2024-04-17T09:55:02.852972+0000","last_active":"2024-04-17T09:55:03.830893+0000","last_peered":"2024-04-17T09:55:03.830893+0000","last_clean":"2024-04-17T09:55:03.830893+0000","last_became_active":"2024-04-17T09:55:02.852753+0000","last_became_peered":"2024-04-17T09:55:02.852753+0000","last_unstale":"2024-04-17T09:55:03.830893+0000","last_undegraded":"2024-04-17T09:55:03.830893+0000","last_fullsized":"2024-04-17T09:55:03.830893+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:24:34.427573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.844057+0000","last_change":"2024-04-17T09:55:03.856033+0000","last_active":"2024-04-17T09:55:04.844057+0000","last_peered":"2024-04-17T09:55:04.844057+0000","last_clean":"2024-04-17T09:55:04.844057+0000","last_became_active":"2024-04-17T09:55:03.852577+0000","last_became_peered":"2024-04-17T09:55:03.852577+0000","last_unstale":"2024-04-17T09:55:04.844057+0000","last_undegraded":"2024-04-17T09:55:04.844057+0000","last_fullsized":"2024-04-17T09:55:04.844057+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:24:35.433580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.857625+0000","last_change":"2024-04-17T09:55:02.847850+0000","last_active":"2024-04-17T09:55:04.857625+0000","last_peered":"2024-04-17T09:55:04.857625+0000","last_clean":"2024-04-17T09:55:04.857625+0000","last_became_active":"2024-04-17T09:55:02.847505+0000","last_became_peered":"2024-04-17T09:55:02.847505+0000","last_unstale":"2024-04-17T09:55:04.857625+0000","last_undegraded":"2024-04-17T09:55:04.857625+0000","last_fullsized":"2024-04-17T09:55:04.857625+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:03.190518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.867201+0000","last_change":"2024-04-17T09:55:03.863759+0000","last_active":"2024-04-17T09:55:03.867201+0000","last_peered":"2024-04-17T09:55:03.867201+0000","last_clean":"2024-04-17T09:55:03.867201+0000","last_became_active":"2024-04-17T09:55:03.863488+0000","last_became_peered":"2024-04-17T09:55:03.863488+0000","last_unstale":"2024-04-17T09:55:03.867201+0000","last_undegraded":"2024-04-17T09:55:03.867201+0000","last_fullsized":"2024-04-17T09:55:03.867201+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:44:47.869450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.831888+0000","last_change":"2024-04-17T09:55:02.847782+0000","last_active":"2024-04-17T09:55:03.831888+0000","last_peered":"2024-04-17T09:55:03.831888+0000","last_clean":"2024-04-17T09:55:03.831888+0000","last_became_active":"2024-04-17T09:55:02.847491+0000","last_became_peered":"2024-04-17T09:55:02.847491+0000","last_unstale":"2024-04-17T09:55:03.831888+0000","last_undegraded":"2024-04-17T09:55:03.831888+0000","last_fullsized":"2024-04-17T09:55:03.831888+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:03.190518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.850986+0000","last_change":"2024-04-17T09:55:03.847527+0000","last_active":"2024-04-17T09:55:03.850986+0000","last_peered":"2024-04-17T09:55:03.850986+0000","last_clean":"2024-04-17T09:55:03.850986+0000","last_became_active":"2024-04-17T09:55:03.847162+0000","last_became_peered":"2024-04-17T09:55:03.847162+0000","last_unstale":"2024-04-17T09:55:03.850986+0000","last_undegraded":"2024-04-17T09:55:03.850986+0000","last_fullsized":"2024-04-17T09:55:03.850986+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:25:29.044313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.832605+0000","last_change":"2024-04-17T09:55:02.846877+0000","last_active":"2024-04-17T09:55:03.832605+0000","last_peered":"2024-04-17T09:55:03.832605+0000","last_clean":"2024-04-17T09:55:03.832605+0000","last_became_active":"2024-04-17T09:55:02.846522+0000","last_became_peered":"2024-04-17T09:55:02.846522+0000","last_unstale":"2024-04-17T09:55:03.832605+0000","last_undegraded":"2024-04-17T09:55:03.832605+0000","last_fullsized":"2024-04-17T09:55:03.832605+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:23:59.885800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.855854+0000","last_change":"2024-04-17T09:55:03.843549+0000","last_active":"2024-04-17T09:55:04.855854+0000","last_peered":"2024-04-17T09:55:04.855854+0000","last_clean":"2024-04-17T09:55:04.855854+0000","last_became_active":"2024-04-17T09:55:03.843248+0000","last_became_peered":"2024-04-17T09:55:03.843248+0000","last_unstale":"2024-04-17T09:55:04.855854+0000","last_undegraded":"2024-04-17T09:55:04.855854+0000","last_fullsized":"2024-04-17T09:55:04.855854+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:27:28.693741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.831459+0000","last_change":"2024-04-17T09:55:02.847288+0000","last_active":"2024-04-17T09:55:03.831459+0000","last_peered":"2024-04-17T09:55:03.831459+0000","last_clean":"2024-04-17T09:55:03.831459+0000","last_became_active":"2024-04-17T09:55:02.847030+0000","last_became_peered":"2024-04-17T09:55:02.847030+0000","last_unstale":"2024-04-17T09:55:03.831459+0000","last_undegraded":"2024-04-17T09:55:03.831459+0000","last_fullsized":"2024-04-17T09:55:03.831459+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:07:25.155315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.855997+0000","last_change":"2024-04-17T09:55:03.851746+0000","last_active":"2024-04-17T09:55:04.855997+0000","last_peered":"2024-04-17T09:55:04.855997+0000","last_clean":"2024-04-17T09:55:04.855997+0000","last_became_active":"2024-04-17T09:55:03.849989+0000","last_became_peered":"2024-04-17T09:55:03.849989+0000","last_unstale":"2024-04-17T09:55:04.855997+0000","last_undegraded":"2024-04-17T09:55:04.855997+0000","last_fullsized":"2024-04-17T09:55:04.855997+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:39.456313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.842042+0000","last_change":"2024-04-17T09:55:02.846614+0000","last_active":"2024-04-17T09:55:03.842042+0000","last_peered":"2024-04-17T09:55:03.842042+0000","last_clean":"2024-04-17T09:55:03.842042+0000","last_became_active":"2024-04-17T09:55:02.846303+0000","last_became_peered":"2024-04-17T09:55:02.846303+0000","last_unstale":"2024-04-17T09:55:03.842042+0000","last_undegraded":"2024-04-17T09:55:03.842042+0000","last_fullsized":"2024-04-17T09:55:03.842042+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:09:21.458482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.848013+0000","last_change":"2024-04-17T09:55:03.844424+0000","last_active":"2024-04-17T09:55:03.848013+0000","last_peered":"2024-04-17T09:55:03.848013+0000","last_clean":"2024-04-17T09:55:03.848013+0000","last_became_active":"2024-04-17T09:55:03.844169+0000","last_became_peered":"2024-04-17T09:55:03.844169+0000","last_unstale":"2024-04-17T09:55:03.848013+0000","last_undegraded":"2024-04-17T09:55:03.848013+0000","last_fullsized":"2024-04-17T09:55:03.848013+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:57:06.558212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.833276+0000","last_change":"2024-04-17T09:55:02.844542+0000","last_active":"2024-04-17T09:55:03.833276+0000","last_peered":"2024-04-17T09:55:03.833276+0000","last_clean":"2024-04-17T09:55:03.833276+0000","last_became_active":"2024-04-17T09:55:02.841873+0000","last_became_peered":"2024-04-17T09:55:02.841873+0000","last_unstale":"2024-04-17T09:55:03.833276+0000","last_undegraded":"2024-04-17T09:55:03.833276+0000","last_fullsized":"2024-04-17T09:55:03.833276+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:47:32.668093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.852858+0000","last_change":"2024-04-17T09:55:03.850047+0000","last_active":"2024-04-17T09:55:03.852858+0000","last_peered":"2024-04-17T09:55:03.852858+0000","last_clean":"2024-04-17T09:55:03.852858+0000","last_became_active":"2024-04-17T09:55:03.849786+0000","last_became_peered":"2024-04-17T09:55:03.849786+0000","last_unstale":"2024-04-17T09:55:03.852858+0000","last_undegraded":"2024-04-17T09:55:03.852858+0000","last_fullsized":"2024-04-17T09:55:03.852858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:54:04.483854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.842820+0000","last_change":"2024-04-17T09:55:02.850181+0000","last_active":"2024-04-17T09:55:03.842820+0000","last_peered":"2024-04-17T09:55:03.842820+0000","last_clean":"2024-04-17T09:55:03.842820+0000","last_became_active":"2024-04-17T09:55:02.849909+0000","last_became_peered":"2024-04-17T09:55:02.849909+0000","last_unstale":"2024-04-17T09:55:03.842820+0000","last_undegraded":"2024-04-17T09:55:03.842820+0000","last_fullsized":"2024-04-17T09:55:03.842820+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:09:21.458482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.861332+0000","last_change":"2024-04-17T09:55:03.853216+0000","last_active":"2024-04-17T09:55:03.861332+0000","last_peered":"2024-04-17T09:55:03.861332+0000","last_clean":"2024-04-17T09:55:03.861332+0000","last_became_active":"2024-04-17T09:55:03.852844+0000","last_became_peered":"2024-04-17T09:55:03.852844+0000","last_unstale":"2024-04-17T09:55:03.861332+0000","last_undegraded":"2024-04-17T09:55:03.861332+0000","last_fullsized":"2024-04-17T09:55:03.861332+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:09:55.354856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.869028+0000","last_change":"2024-04-17T09:55:02.850319+0000","last_active":"2024-04-17T09:55:03.869028+0000","last_peered":"2024-04-17T09:55:03.869028+0000","last_clean":"2024-04-17T09:55:03.869028+0000","last_became_active":"2024-04-17T09:55:02.849973+0000","last_became_peered":"2024-04-17T09:55:02.849973+0000","last_unstale":"2024-04-17T09:55:03.869028+0000","last_undegraded":"2024-04-17T09:55:03.869028+0000","last_fullsized":"2024-04-17T09:55:03.869028+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:46:04.193707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":215,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.830955+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T09:55:03.830955+0000","last_peered":"2024-04-17T09:55:03.830955+0000","last_clean":"2024-04-17T09:55:03.830955+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T09:55:03.830955+0000","last_undegraded":"2024-04-17T09:55:03.830955+0000","last_fullsized":"2024-04-17T09:55:03.830955+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509468,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1118208,"data_stored":800526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":210740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509468,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":524288,"data_stored":210138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":544768,"data_stored":217006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":548864,"data_stored":218002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509468,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31700,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":524288,"data_stored":210138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542172,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31700,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":532480,"data_stored":211680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542172,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1101824,"data_stored":793661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T09:55:07.967 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T09:55:08.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79fc600640 1 Processor -- start 2024-04-17T09:55:08.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79fc600640 1 -- start start 2024-04-17T09:55:08.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79fc600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f4144e90 0x7f79f40b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:08.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79fc600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79f4072bb0 con 0x7f79f40b4de0 2024-04-17T09:55:08.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79fc600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79f40b3dc0 con 0x7f79f415c1b0 2024-04-17T09:55:08.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79fc600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f79f4145270 con 0x7f79f4144e90 2024-04-17T09:55:08.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f9200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f79f40b4de0 0x7f79f415baa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33154/0 (socket says 172.21.15.32:33154) 2024-04-17T09:55:08.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f9200640 1 -- 172.21.15.32:0/2997820169 learned_addr learned my addr 172.21.15.32:0/2997820169 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:08.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f9c00640 1 --2- 172.21.15.32:0/2997820169 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f4144e90 0x7f79f40b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:08.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f3e00640 1 -- 172.21.15.32:0/2997820169 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 200500200 0 0) 0x7f79f4072bb0 con 0x7f79f40b4de0 2024-04-17T09:55:08.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f3e00640 1 -- 172.21.15.32:0/2997820169 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79dc003530 con 0x7f79f40b4de0 2024-04-17T09:55:08.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f3e00640 1 -- 172.21.15.32:0/2997820169 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4258189999 0 0) 0x7f79f40b3dc0 con 0x7f79f415c1b0 2024-04-17T09:55:08.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f3e00640 1 -- 172.21.15.32:0/2997820169 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79f4072bb0 con 0x7f79f415c1b0 2024-04-17T09:55:08.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f3e00640 1 -- 172.21.15.32:0/2997820169 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2513275548 0 0) 0x7f79dc003530 con 0x7f79f40b4de0 2024-04-17T09:55:08.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f3e00640 1 -- 172.21.15.32:0/2997820169 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f4144e90 msgr2=0x7f79f40b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:08.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f3e00640 1 --2- 172.21.15.32:0/2997820169 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f4144e90 0x7f79f40b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:08.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f3e00640 1 -- 172.21.15.32:0/2997820169 >> v1:172.21.15.144:6789/0 conn(0x7f79f415c1b0 legacy=0x7f79f406ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:08.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f3e00640 1 -- 172.21.15.32:0/2997820169 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f40b40a0 con 0x7f79f40b4de0 2024-04-17T09:55:08.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f3e00640 1 -- 172.21.15.32:0/2997820169 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f79e00016a0 con 0x7f79f40b4de0 2024-04-17T09:55:08.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f9c00640 1 --2- 172.21.15.32:0/2997820169 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f4144e90 0x7f79f40b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:55:08.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f3e00640 1 -- 172.21.15.32:0/2997820169 >> v1:172.21.15.32:6789/0 conn(0x7f79f40b4de0 legacy=0x7f79f415baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:08.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f3e00640 1 --2- 172.21.15.32:0/2997820169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79dc003b80 0x7f79dc003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:08.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f3e00640 1 --2- 172.21.15.32:0/2997820169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc004810 0x7f79dc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:08.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f3e00640 1 --2- 172.21.15.32:0/2997820169 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f4144e90 0x7f79dc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:08.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f3e00640 1 -- 172.21.15.32:0/2997820169 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f79f4145270 con 0x7f79dc004810 2024-04-17T09:55:08.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f3e00640 1 -- 172.21.15.32:0/2997820169 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f79dc00ba00 con 0x7f79dc003b80 2024-04-17T09:55:08.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f3e00640 1 -- 172.21.15.32:0/2997820169 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f79dc00bc00 con 0x7f79f4144e90 2024-04-17T09:55:08.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f9200640 1 --2- 172.21.15.32:0/2997820169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc004810 0x7f79dc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:08.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f9c00640 1 --2- 172.21.15.32:0/2997820169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79dc003b80 0x7f79dc003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:08.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79fa600640 1 --2- 172.21.15.32:0/2997820169 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f4144e90 0x7f79dc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:08.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f9200640 1 -- 172.21.15.32:0/2997820169 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f4144e90 msgr2=0x7f79dc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:08.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f9200640 1 --2- 172.21.15.32:0/2997820169 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f4144e90 0x7f79dc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:08.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f9200640 1 -- 172.21.15.32:0/2997820169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79dc003b80 msgr2=0x7f79dc003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:08.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f9200640 1 --2- 172.21.15.32:0/2997820169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79dc003b80 0x7f79dc003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:08.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.145+0000 7f79f9200640 1 -- 172.21.15.32:0/2997820169 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79dc003530 con 0x7f79dc004810 2024-04-17T09:55:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.149+0000 7f79f9200640 1 --2- 172.21.15.32:0/2997820169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc004810 0x7f79dc004c90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f79f40cceb0 tx=0x7f79e001cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f445b240466054d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.149+0000 7f79f3e00640 1 -- 172.21.15.32:0/2997820169 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0007ce0 con 0x7f79dc004810 2024-04-17T09:55:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.149+0000 7f79f3e00640 1 -- 172.21.15.32:0/2997820169 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e00a9020 con 0x7f79dc004810 2024-04-17T09:55:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.149+0000 7f79f3e00640 1 -- 172.21.15.32:0/2997820169 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0005b00 con 0x7f79dc004810 2024-04-17T09:55:08.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.149+0000 7f79fc600640 1 -- 172.21.15.32:0/2997820169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc004810 msgr2=0x7f79dc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:08.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.149+0000 7f79fc600640 1 --2- 172.21.15.32:0/2997820169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc004810 0x7f79dc004c90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f79f40cceb0 tx=0x7f79e001cba0 comp rx=0 tx=0).stop 2024-04-17T09:55:08.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.149+0000 7f79fa600640 1 -- 172.21.15.32:0/2997820169 reap_dead start 2024-04-17T09:55:08.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.149+0000 7f79fc600640 1 -- 172.21.15.32:0/2997820169 shutdown_connections 2024-04-17T09:55:08.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.149+0000 7f79fc600640 1 -- 172.21.15.32:0/2997820169 >> 172.21.15.32:0/2997820169 conn(0x7f79f406ba30 msgr2=0x7f79f406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:08.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.149+0000 7f79fc600640 1 -- 172.21.15.32:0/2997820169 shutdown_connections 2024-04-17T09:55:08.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.149+0000 7f79fc600640 1 -- 172.21.15.32:0/2997820169 wait complete. 2024-04-17T09:55:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79fc600640 1 Processor -- start 2024-04-17T09:55:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79fc600640 1 -- start start 2024-04-17T09:55:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79fc600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79dc004810 0x7f79f41fb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79fc600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f40b15b0 0x7f79f413f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79fc600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f40cf270 0x7f79f40cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79fc600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f79f406f7f0 con 0x7f79f40b15b0 2024-04-17T09:55:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79fc600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f79f406f670 con 0x7f79dc004810 2024-04-17T09:55:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79fc600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f79f40b48a0 con 0x7f79f40cf270 2024-04-17T09:55:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79f9200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f40b15b0 0x7f79f413f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79f9200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f40b15b0 0x7f79f413f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54022/0 (socket says 172.21.15.32:54022) 2024-04-17T09:55:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79f9200640 1 -- 172.21.15.32:0/3463354468 learned_addr learned my addr 172.21.15.32:0/3463354468 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79f9c00640 1 --2- 172.21.15.32:0/3463354468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79dc004810 0x7f79f41fb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79fa600640 1 --2- 172.21.15.32:0/3463354468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f40cf270 0x7f79f40cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79f9200640 1 -- 172.21.15.32:0/3463354468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f40cf270 msgr2=0x7f79f40cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79f9200640 1 --2- 172.21.15.32:0/3463354468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f40cf270 0x7f79f40cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79f9200640 1 -- 172.21.15.32:0/3463354468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79dc004810 msgr2=0x7f79f41fb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79f9200640 1 --2- 172.21.15.32:0/3463354468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79dc004810 0x7f79f41fb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79f9200640 1 -- 172.21.15.32:0/3463354468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f40b1b20 con 0x7f79f40b15b0 2024-04-17T09:55:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79fa600640 1 --2- 172.21.15.32:0/3463354468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f40cf270 0x7f79f40cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79f9200640 1 --2- 172.21.15.32:0/3463354468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f40b15b0 0x7f79f413f9b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f79e001c9d0 tx=0x7f79dc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79f1600640 1 -- 172.21.15.32:0/3463354468 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0007ce0 con 0x7f79f40b15b0 2024-04-17T09:55:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79f1600640 1 -- 172.21.15.32:0/3463354468 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e0004a10 con 0x7f79f40b15b0 2024-04-17T09:55:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79f1600640 1 -- 172.21.15.32:0/3463354468 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0001c40 con 0x7f79f40b15b0 2024-04-17T09:55:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79fc600640 1 -- 172.21.15.32:0/3463354468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f40b1e00 con 0x7f79f40b15b0 2024-04-17T09:55:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.153+0000 7f79fc600640 1 -- 172.21.15.32:0/3463354468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f40b23c0 con 0x7f79f40b15b0 2024-04-17T09:55:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.157+0000 7f79fc600640 1 -- 172.21.15.32:0/3463354468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f40cfc30 con 0x7f79f40b15b0 2024-04-17T09:55:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.157+0000 7f79f1600640 1 -- 172.21.15.32:0/3463354468 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f79e0007070 con 0x7f79f40b15b0 2024-04-17T09:55:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.157+0000 7f79f1600640 1 --2- 172.21.15.32:0/3463354468 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f79d4068380 0x7f79d406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:08.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.157+0000 7f79f1600640 1 -- 172.21.15.32:0/3463354468 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f79e0120c30 con 0x7f79f40b15b0 2024-04-17T09:55:08.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.157+0000 7f79f9c00640 1 --2- 172.21.15.32:0/3463354468 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f79d4068380 0x7f79d406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:08.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.157+0000 7f79f9c00640 1 --2- 172.21.15.32:0/3463354468 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f79d4068380 0x7f79d406a840 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009a70 tx=0x7f79e8009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.161+0000 7f79f1600640 1 -- 172.21.15.32:0/3463354468 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f79e00168b0 con 0x7f79f40b15b0 2024-04-17T09:55:08.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.321+0000 7f79fc600640 1 -- 172.21.15.32:0/3463354468 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f79f4068a90 con 0x7f79d4068380 2024-04-17T09:55:08.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.329+0000 7f79f1600640 1 -- 172.21.15.32:0/3463354468 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251917 (secure 0 0 0) 0x7f79f4068a90 con 0x7f79d4068380 2024-04-17T09:55:08.333 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:08.334 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T09:55:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.333+0000 7f79fc600640 1 -- 172.21.15.32:0/3463354468 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f79d4068380 msgr2=0x7f79d406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.333+0000 7f79fc600640 1 --2- 172.21.15.32:0/3463354468 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f79d4068380 0x7f79d406a840 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009a70 tx=0x7f79e8009190 comp rx=0 tx=0).stop 2024-04-17T09:55:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.333+0000 7f79fc600640 1 -- 172.21.15.32:0/3463354468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f40b15b0 msgr2=0x7f79f413f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.333+0000 7f79fc600640 1 --2- 172.21.15.32:0/3463354468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f40b15b0 0x7f79f413f9b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f79e001c9d0 tx=0x7f79dc005840 comp rx=0 tx=0).stop 2024-04-17T09:55:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.333+0000 7f79fc600640 1 -- 172.21.15.32:0/3463354468 shutdown_connections 2024-04-17T09:55:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.333+0000 7f79fc600640 1 --2- 172.21.15.32:0/3463354468 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f79d4068380 0x7f79d406a840 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.333+0000 7f79fc600640 1 --2- 172.21.15.32:0/3463354468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f40cf270 0x7f79f40cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:08.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.333+0000 7f79fc600640 1 --2- 172.21.15.32:0/3463354468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f40b15b0 0x7f79f413f9b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:08.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.333+0000 7f79fc600640 1 --2- 172.21.15.32:0/3463354468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79dc004810 0x7f79f41fb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:08.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.333+0000 7f79fc600640 1 -- 172.21.15.32:0/3463354468 >> 172.21.15.32:0/3463354468 conn(0x7f79f406ba30 msgr2=0x7f79f40ae550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:08.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.333+0000 7f79fc600640 1 -- 172.21.15.32:0/3463354468 shutdown_connections 2024-04-17T09:55:08.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.333+0000 7f79fc600640 1 -- 172.21.15.32:0/3463354468 wait complete. 2024-04-17T09:55:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-04-17T09:55:07.307762+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242276,"kb_used_data":5296,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732444,"statfs":{"total":773094113280,"available":772846022656,"internally_reserved":0,"allocated":5423104,"data_stored":2871891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.490770"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.852797+0000","last_change":"2024-04-17T09:55:03.849325+0000","last_active":"2024-04-17T09:55:03.852797+0000","last_peered":"2024-04-17T09:55:03.852797+0000","last_clean":"2024-04-17T09:55:03.852797+0000","last_became_active":"2024-04-17T09:55:03.848939+0000","last_became_peered":"2024-04-17T09:55:03.848939+0000","last_unstale":"2024-04-17T09:55:03.852797+0000","last_undegraded":"2024-04-17T09:55:03.852797+0000","last_fullsized":"2024-04-17T09:55:03.852797+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:31.009742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.856176+0000","last_change":"2024-04-17T09:55:03.854965+0000","last_active":"2024-04-17T09:55:03.856176+0000","last_peered":"2024-04-17T09:55:03.856176+0000","last_clean":"2024-04-17T09:55:03.856176+0000","last_became_active":"2024-04-17T09:55:03.854644+0000","last_became_peered":"2024-04-17T09:55:03.854644+0000","last_unstale":"2024-04-17T09:55:03.856176+0000","last_undegraded":"2024-04-17T09:55:03.856176+0000","last_fullsized":"2024-04-17T09:55:03.856176+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:07:26.161323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.871640+0000","last_change":"2024-04-17T09:55:03.868111+0000","last_active":"2024-04-17T09:55:03.871640+0000","last_peered":"2024-04-17T09:55:03.871640+0000","last_clean":"2024-04-17T09:55:03.871640+0000","last_became_active":"2024-04-17T09:55:03.865910+0000","last_became_peered":"2024-04-17T09:55:03.865910+0000","last_unstale":"2024-04-17T09:55:03.871640+0000","last_undegraded":"2024-04-17T09:55:03.871640+0000","last_fullsized":"2024-04-17T09:55:03.871640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:38:04.471407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.862003+0000","last_change":"2024-04-17T09:55:03.856899+0000","last_active":"2024-04-17T09:55:03.862003+0000","last_peered":"2024-04-17T09:55:03.862003+0000","last_clean":"2024-04-17T09:55:03.862003+0000","last_became_active":"2024-04-17T09:55:03.855227+0000","last_became_peered":"2024-04-17T09:55:03.855227+0000","last_unstale":"2024-04-17T09:55:03.862003+0000","last_undegraded":"2024-04-17T09:55:03.862003+0000","last_fullsized":"2024-04-17T09:55:03.862003+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:08.824623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.857302+0000","last_change":"2024-04-17T09:55:03.855439+0000","last_active":"2024-04-17T09:55:03.857302+0000","last_peered":"2024-04-17T09:55:03.857302+0000","last_clean":"2024-04-17T09:55:03.857302+0000","last_became_active":"2024-04-17T09:55:03.855126+0000","last_became_peered":"2024-04-17T09:55:03.855126+0000","last_unstale":"2024-04-17T09:55:03.857302+0000","last_undegraded":"2024-04-17T09:55:03.857302+0000","last_fullsized":"2024-04-17T09:55:03.857302+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:17:03.025913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.855691+0000","last_change":"2024-04-17T09:55:03.851400+0000","last_active":"2024-04-17T09:55:04.855691+0000","last_peered":"2024-04-17T09:55:04.855691+0000","last_clean":"2024-04-17T09:55:04.855691+0000","last_became_active":"2024-04-17T09:55:03.851119+0000","last_became_peered":"2024-04-17T09:55:03.851119+0000","last_unstale":"2024-04-17T09:55:04.855691+0000","last_undegraded":"2024-04-17T09:55:04.855691+0000","last_fullsized":"2024-04-17T09:55:04.855691+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:13:10.584024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.844135+0000","last_change":"2024-04-17T09:55:03.841822+0000","last_active":"2024-04-17T09:55:03.844135+0000","last_peered":"2024-04-17T09:55:03.844135+0000","last_clean":"2024-04-17T09:55:03.844135+0000","last_became_active":"2024-04-17T09:55:03.841536+0000","last_became_peered":"2024-04-17T09:55:03.841536+0000","last_unstale":"2024-04-17T09:55:03.844135+0000","last_undegraded":"2024-04-17T09:55:03.844135+0000","last_fullsized":"2024-04-17T09:55:03.844135+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:46:54.551275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.862354+0000","last_change":"2024-04-17T09:55:03.860673+0000","last_active":"2024-04-17T09:55:03.862354+0000","last_peered":"2024-04-17T09:55:03.862354+0000","last_clean":"2024-04-17T09:55:03.862354+0000","last_became_active":"2024-04-17T09:55:03.855522+0000","last_became_peered":"2024-04-17T09:55:03.855522+0000","last_unstale":"2024-04-17T09:55:03.862354+0000","last_undegraded":"2024-04-17T09:55:03.862354+0000","last_fullsized":"2024-04-17T09:55:03.862354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:38:40.946652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.861315+0000","last_change":"2024-04-17T09:55:03.853563+0000","last_active":"2024-04-17T09:55:03.861315+0000","last_peered":"2024-04-17T09:55:03.861315+0000","last_clean":"2024-04-17T09:55:03.861315+0000","last_became_active":"2024-04-17T09:55:03.853314+0000","last_became_peered":"2024-04-17T09:55:03.853314+0000","last_unstale":"2024-04-17T09:55:03.861315+0000","last_undegraded":"2024-04-17T09:55:03.861315+0000","last_fullsized":"2024-04-17T09:55:03.861315+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:14:17.006620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.862452+0000","last_change":"2024-04-17T09:55:03.860578+0000","last_active":"2024-04-17T09:55:03.862452+0000","last_peered":"2024-04-17T09:55:03.862452+0000","last_clean":"2024-04-17T09:55:03.862452+0000","last_became_active":"2024-04-17T09:55:03.855286+0000","last_became_peered":"2024-04-17T09:55:03.855286+0000","last_unstale":"2024-04-17T09:55:03.862452+0000","last_undegraded":"2024-04-17T09:55:03.862452+0000","last_fullsized":"2024-04-17T09:55:03.862452+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:31:41.399858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.856676+0000","last_change":"2024-04-17T09:55:03.850950+0000","last_active":"2024-04-17T09:55:03.856676+0000","last_peered":"2024-04-17T09:55:03.856676+0000","last_clean":"2024-04-17T09:55:03.856676+0000","last_became_active":"2024-04-17T09:55:03.850432+0000","last_became_peered":"2024-04-17T09:55:03.850432+0000","last_unstale":"2024-04-17T09:55:03.856676+0000","last_undegraded":"2024-04-17T09:55:03.856676+0000","last_fullsized":"2024-04-17T09:55:03.856676+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:03:57.004315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.859804+0000","last_change":"2024-04-17T09:55:03.858317+0000","last_active":"2024-04-17T09:55:03.859804+0000","last_peered":"2024-04-17T09:55:03.859804+0000","last_clean":"2024-04-17T09:55:03.859804+0000","last_became_active":"2024-04-17T09:55:03.856743+0000","last_became_peered":"2024-04-17T09:55:03.856743+0000","last_unstale":"2024-04-17T09:55:03.859804+0000","last_undegraded":"2024-04-17T09:55:03.859804+0000","last_fullsized":"2024-04-17T09:55:03.859804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:21.677589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.842710+0000","last_change":"2024-04-17T09:55:03.846602+0000","last_active":"2024-04-17T09:55:04.842710+0000","last_peered":"2024-04-17T09:55:04.842710+0000","last_clean":"2024-04-17T09:55:04.842710+0000","last_became_active":"2024-04-17T09:55:03.846292+0000","last_became_peered":"2024-04-17T09:55:03.846292+0000","last_unstale":"2024-04-17T09:55:04.842710+0000","last_undegraded":"2024-04-17T09:55:04.842710+0000","last_fullsized":"2024-04-17T09:55:04.842710+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:00.891807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.856023+0000","last_change":"2024-04-17T09:55:03.844840+0000","last_active":"2024-04-17T09:55:03.856023+0000","last_peered":"2024-04-17T09:55:03.856023+0000","last_clean":"2024-04-17T09:55:03.856023+0000","last_became_active":"2024-04-17T09:55:03.844467+0000","last_became_peered":"2024-04-17T09:55:03.844467+0000","last_unstale":"2024-04-17T09:55:03.856023+0000","last_undegraded":"2024-04-17T09:55:03.856023+0000","last_fullsized":"2024-04-17T09:55:03.856023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:24:22.826472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.855748+0000","last_change":"2024-04-17T09:55:03.856578+0000","last_active":"2024-04-17T09:55:04.855748+0000","last_peered":"2024-04-17T09:55:04.855748+0000","last_clean":"2024-04-17T09:55:04.855748+0000","last_became_active":"2024-04-17T09:55:03.853231+0000","last_became_peered":"2024-04-17T09:55:03.853231+0000","last_unstale":"2024-04-17T09:55:04.855748+0000","last_undegraded":"2024-04-17T09:55:04.855748+0000","last_fullsized":"2024-04-17T09:55:04.855748+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:22:46.796157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.842765+0000","last_change":"2024-04-17T09:55:03.854152+0000","last_active":"2024-04-17T09:55:04.842765+0000","last_peered":"2024-04-17T09:55:04.842765+0000","last_clean":"2024-04-17T09:55:04.842765+0000","last_became_active":"2024-04-17T09:55:03.853866+0000","last_became_peered":"2024-04-17T09:55:03.853866+0000","last_unstale":"2024-04-17T09:55:04.842765+0000","last_undegraded":"2024-04-17T09:55:04.842765+0000","last_fullsized":"2024-04-17T09:55:04.842765+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:07.056555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.870064+0000","last_change":"2024-04-17T09:55:03.860757+0000","last_active":"2024-04-17T09:55:03.870064+0000","last_peered":"2024-04-17T09:55:03.870064+0000","last_clean":"2024-04-17T09:55:03.870064+0000","last_became_active":"2024-04-17T09:55:03.860460+0000","last_became_peered":"2024-04-17T09:55:03.860460+0000","last_unstale":"2024-04-17T09:55:03.870064+0000","last_undegraded":"2024-04-17T09:55:03.870064+0000","last_fullsized":"2024-04-17T09:55:03.870064+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:09:55.354856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.868722+0000","last_change":"2024-04-17T09:55:03.865594+0000","last_active":"2024-04-17T09:55:03.868722+0000","last_peered":"2024-04-17T09:55:03.868722+0000","last_clean":"2024-04-17T09:55:03.868722+0000","last_became_active":"2024-04-17T09:55:03.865109+0000","last_became_peered":"2024-04-17T09:55:03.865109+0000","last_unstale":"2024-04-17T09:55:03.868722+0000","last_undegraded":"2024-04-17T09:55:03.868722+0000","last_fullsized":"2024-04-17T09:55:03.868722+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:14:17.006620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.843851+0000","last_change":"2024-04-17T09:55:03.854194+0000","last_active":"2024-04-17T09:55:04.843851+0000","last_peered":"2024-04-17T09:55:04.843851+0000","last_clean":"2024-04-17T09:55:04.843851+0000","last_became_active":"2024-04-17T09:55:03.853132+0000","last_became_peered":"2024-04-17T09:55:03.853132+0000","last_unstale":"2024-04-17T09:55:04.843851+0000","last_undegraded":"2024-04-17T09:55:04.843851+0000","last_fullsized":"2024-04-17T09:55:04.843851+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:55:53.473939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.842847+0000","last_change":"2024-04-17T09:55:03.846415+0000","last_active":"2024-04-17T09:55:04.842847+0000","last_peered":"2024-04-17T09:55:04.842847+0000","last_clean":"2024-04-17T09:55:04.842847+0000","last_became_active":"2024-04-17T09:55:03.846116+0000","last_became_peered":"2024-04-17T09:55:03.846116+0000","last_unstale":"2024-04-17T09:55:04.842847+0000","last_undegraded":"2024-04-17T09:55:04.842847+0000","last_fullsized":"2024-04-17T09:55:04.842847+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:28:58.768665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.855769+0000","last_change":"2024-04-17T09:55:03.856297+0000","last_active":"2024-04-17T09:55:04.855769+0000","last_peered":"2024-04-17T09:55:04.855769+0000","last_clean":"2024-04-17T09:55:04.855769+0000","last_became_active":"2024-04-17T09:55:03.855956+0000","last_became_peered":"2024-04-17T09:55:03.855956+0000","last_unstale":"2024-04-17T09:55:04.855769+0000","last_undegraded":"2024-04-17T09:55:04.855769+0000","last_fullsized":"2024-04-17T09:55:04.855769+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:52:31.651115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.861282+0000","last_change":"2024-04-17T09:55:03.855335+0000","last_active":"2024-04-17T09:55:03.861282+0000","last_peered":"2024-04-17T09:55:03.861282+0000","last_clean":"2024-04-17T09:55:03.861282+0000","last_became_active":"2024-04-17T09:55:03.854983+0000","last_became_peered":"2024-04-17T09:55:03.854983+0000","last_unstale":"2024-04-17T09:55:03.861282+0000","last_undegraded":"2024-04-17T09:55:03.861282+0000","last_fullsized":"2024-04-17T09:55:03.861282+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:38:04.471407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.843242+0000","last_change":"2024-04-17T09:55:03.836067+0000","last_active":"2024-04-17T09:55:04.843242+0000","last_peered":"2024-04-17T09:55:04.843242+0000","last_clean":"2024-04-17T09:55:04.843242+0000","last_became_active":"2024-04-17T09:55:03.835767+0000","last_became_peered":"2024-04-17T09:55:03.835767+0000","last_unstale":"2024-04-17T09:55:04.843242+0000","last_undegraded":"2024-04-17T09:55:04.843242+0000","last_fullsized":"2024-04-17T09:55:04.843242+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:59:44.714355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.842777+0000","last_change":"2024-04-17T09:55:03.846631+0000","last_active":"2024-04-17T09:55:04.842777+0000","last_peered":"2024-04-17T09:55:04.842777+0000","last_clean":"2024-04-17T09:55:04.842777+0000","last_became_active":"2024-04-17T09:55:03.846395+0000","last_became_peered":"2024-04-17T09:55:03.846395+0000","last_unstale":"2024-04-17T09:55:04.842777+0000","last_undegraded":"2024-04-17T09:55:04.842777+0000","last_fullsized":"2024-04-17T09:55:04.842777+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:24:11.783113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.843925+0000","last_change":"2024-04-17T09:55:03.848777+0000","last_active":"2024-04-17T09:55:04.843925+0000","last_peered":"2024-04-17T09:55:04.843925+0000","last_clean":"2024-04-17T09:55:04.843925+0000","last_became_active":"2024-04-17T09:55:03.848529+0000","last_became_peered":"2024-04-17T09:55:03.848529+0000","last_unstale":"2024-04-17T09:55:04.843925+0000","last_undegraded":"2024-04-17T09:55:04.843925+0000","last_fullsized":"2024-04-17T09:55:04.843925+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:57:06.558212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.842903+0000","last_change":"2024-04-17T09:55:03.845536+0000","last_active":"2024-04-17T09:55:04.842903+0000","last_peered":"2024-04-17T09:55:04.842903+0000","last_clean":"2024-04-17T09:55:04.842903+0000","last_became_active":"2024-04-17T09:55:03.845265+0000","last_became_peered":"2024-04-17T09:55:03.845265+0000","last_unstale":"2024-04-17T09:55:04.842903+0000","last_undegraded":"2024-04-17T09:55:04.842903+0000","last_fullsized":"2024-04-17T09:55:04.842903+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:47:33.674100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.871661+0000","last_change":"2024-04-17T09:55:03.868089+0000","last_active":"2024-04-17T09:55:03.871661+0000","last_peered":"2024-04-17T09:55:03.871661+0000","last_clean":"2024-04-17T09:55:03.871661+0000","last_became_active":"2024-04-17T09:55:03.866411+0000","last_became_peered":"2024-04-17T09:55:03.866411+0000","last_unstale":"2024-04-17T09:55:03.871661+0000","last_undegraded":"2024-04-17T09:55:03.871661+0000","last_fullsized":"2024-04-17T09:55:03.871661+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:35:24.615538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.855955+0000","last_change":"2024-04-17T09:55:03.849541+0000","last_active":"2024-04-17T09:55:03.855955+0000","last_peered":"2024-04-17T09:55:03.855955+0000","last_clean":"2024-04-17T09:55:03.855955+0000","last_became_active":"2024-04-17T09:55:03.849049+0000","last_became_peered":"2024-04-17T09:55:03.849049+0000","last_unstale":"2024-04-17T09:55:03.855955+0000","last_undegraded":"2024-04-17T09:55:03.855955+0000","last_fullsized":"2024-04-17T09:55:03.855955+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:26:02.753909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.843084+0000","last_change":"2024-04-17T09:55:03.849935+0000","last_active":"2024-04-17T09:55:04.843084+0000","last_peered":"2024-04-17T09:55:04.843084+0000","last_clean":"2024-04-17T09:55:04.843084+0000","last_became_active":"2024-04-17T09:55:03.849205+0000","last_became_peered":"2024-04-17T09:55:03.849205+0000","last_unstale":"2024-04-17T09:55:04.843084+0000","last_undegraded":"2024-04-17T09:55:04.843084+0000","last_fullsized":"2024-04-17T09:55:04.843084+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:23.883842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.865766+0000","last_change":"2024-04-17T09:55:03.859935+0000","last_active":"2024-04-17T09:55:03.865766+0000","last_peered":"2024-04-17T09:55:03.865766+0000","last_clean":"2024-04-17T09:55:03.865766+0000","last_became_active":"2024-04-17T09:55:03.859643+0000","last_became_peered":"2024-04-17T09:55:03.859643+0000","last_unstale":"2024-04-17T09:55:03.865766+0000","last_undegraded":"2024-04-17T09:55:03.865766+0000","last_fullsized":"2024-04-17T09:55:03.865766+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:46:05.201092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.855684+0000","last_change":"2024-04-17T09:55:03.856787+0000","last_active":"2024-04-17T09:55:04.855684+0000","last_peered":"2024-04-17T09:55:04.855684+0000","last_clean":"2024-04-17T09:55:04.855684+0000","last_became_active":"2024-04-17T09:55:03.856343+0000","last_became_peered":"2024-04-17T09:55:03.856343+0000","last_unstale":"2024-04-17T09:55:04.855684+0000","last_undegraded":"2024-04-17T09:55:04.855684+0000","last_fullsized":"2024-04-17T09:55:04.855684+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:26:06.903831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.842686+0000","last_change":"2024-04-17T09:55:03.845403+0000","last_active":"2024-04-17T09:55:04.842686+0000","last_peered":"2024-04-17T09:55:04.842686+0000","last_clean":"2024-04-17T09:55:04.842686+0000","last_became_active":"2024-04-17T09:55:03.845135+0000","last_became_peered":"2024-04-17T09:55:03.845135+0000","last_unstale":"2024-04-17T09:55:04.842686+0000","last_undegraded":"2024-04-17T09:55:04.842686+0000","last_fullsized":"2024-04-17T09:55:04.842686+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:15:49.139439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.859065+0000","last_change":"2024-04-17T09:55:03.849511+0000","last_active":"2024-04-17T09:55:03.859065+0000","last_peered":"2024-04-17T09:55:03.859065+0000","last_clean":"2024-04-17T09:55:03.859065+0000","last_became_active":"2024-04-17T09:55:03.849205+0000","last_became_peered":"2024-04-17T09:55:03.849205+0000","last_unstale":"2024-04-17T09:55:03.859065+0000","last_undegraded":"2024-04-17T09:55:03.859065+0000","last_fullsized":"2024-04-17T09:55:03.859065+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:56:23.155554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.856230+0000","last_change":"2024-04-17T09:55:02.849881+0000","last_active":"2024-04-17T09:55:04.856230+0000","last_peered":"2024-04-17T09:55:04.856230+0000","last_clean":"2024-04-17T09:55:04.856230+0000","last_became_active":"2024-04-17T09:55:02.849598+0000","last_became_peered":"2024-04-17T09:55:02.849598+0000","last_unstale":"2024-04-17T09:55:04.856230+0000","last_undegraded":"2024-04-17T09:55:04.856230+0000","last_fullsized":"2024-04-17T09:55:04.856230+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:08.775226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.844068+0000","last_change":"2024-04-17T09:55:03.851114+0000","last_active":"2024-04-17T09:55:04.844068+0000","last_peered":"2024-04-17T09:55:04.844068+0000","last_clean":"2024-04-17T09:55:04.844068+0000","last_became_active":"2024-04-17T09:55:03.850826+0000","last_became_peered":"2024-04-17T09:55:03.850826+0000","last_unstale":"2024-04-17T09:55:04.844068+0000","last_undegraded":"2024-04-17T09:55:04.844068+0000","last_fullsized":"2024-04-17T09:55:04.844068+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:26:02.753909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.872369+0000","last_change":"2024-04-17T09:55:02.838094+0000","last_active":"2024-04-17T09:55:03.872369+0000","last_peered":"2024-04-17T09:55:03.872369+0000","last_clean":"2024-04-17T09:55:03.872369+0000","last_became_active":"2024-04-17T09:55:02.837812+0000","last_became_peered":"2024-04-17T09:55:02.837812+0000","last_unstale":"2024-04-17T09:55:03.872369+0000","last_undegraded":"2024-04-17T09:55:03.872369+0000","last_fullsized":"2024-04-17T09:55:03.872369+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:02.061750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.855933+0000","last_change":"2024-04-17T09:55:03.849327+0000","last_active":"2024-04-17T09:55:03.855933+0000","last_peered":"2024-04-17T09:55:03.855933+0000","last_clean":"2024-04-17T09:55:03.855933+0000","last_became_active":"2024-04-17T09:55:03.848666+0000","last_became_peered":"2024-04-17T09:55:03.848666+0000","last_unstale":"2024-04-17T09:55:03.855933+0000","last_undegraded":"2024-04-17T09:55:03.855933+0000","last_fullsized":"2024-04-17T09:55:03.855933+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:24:35.433580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.869372+0000","last_change":"2024-04-17T09:55:02.848104+0000","last_active":"2024-04-17T09:55:03.869372+0000","last_peered":"2024-04-17T09:55:03.869372+0000","last_clean":"2024-04-17T09:55:03.869372+0000","last_became_active":"2024-04-17T09:55:02.847846+0000","last_became_peered":"2024-04-17T09:55:02.847846+0000","last_unstale":"2024-04-17T09:55:03.869372+0000","last_undegraded":"2024-04-17T09:55:03.869372+0000","last_fullsized":"2024-04-17T09:55:03.869372+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:28:57.762658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.858920+0000","last_change":"2024-04-17T09:55:03.848552+0000","last_active":"2024-04-17T09:55:03.858920+0000","last_peered":"2024-04-17T09:55:03.858920+0000","last_clean":"2024-04-17T09:55:03.858920+0000","last_became_active":"2024-04-17T09:55:03.848247+0000","last_became_peered":"2024-04-17T09:55:03.848247+0000","last_unstale":"2024-04-17T09:55:03.858920+0000","last_undegraded":"2024-04-17T09:55:03.858920+0000","last_fullsized":"2024-04-17T09:55:03.858920+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:24:11.783113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.869339+0000","last_change":"2024-04-17T09:55:02.849355+0000","last_active":"2024-04-17T09:55:03.869339+0000","last_peered":"2024-04-17T09:55:03.869339+0000","last_clean":"2024-04-17T09:55:03.869339+0000","last_became_active":"2024-04-17T09:55:02.849056+0000","last_became_peered":"2024-04-17T09:55:02.849056+0000","last_unstale":"2024-04-17T09:55:03.869339+0000","last_undegraded":"2024-04-17T09:55:03.869339+0000","last_fullsized":"2024-04-17T09:55:03.869339+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:17:02.019906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.860149+0000","last_change":"2024-04-17T09:55:03.852360+0000","last_active":"2024-04-17T09:55:03.860149+0000","last_peered":"2024-04-17T09:55:03.860149+0000","last_clean":"2024-04-17T09:55:03.860149+0000","last_became_active":"2024-04-17T09:55:03.852055+0000","last_became_peered":"2024-04-17T09:55:03.852055+0000","last_unstale":"2024-04-17T09:55:03.860149+0000","last_undegraded":"2024-04-17T09:55:03.860149+0000","last_fullsized":"2024-04-17T09:55:03.860149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:03:57.004315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.855919+0000","last_change":"2024-04-17T09:55:02.851447+0000","last_active":"2024-04-17T09:55:04.855919+0000","last_peered":"2024-04-17T09:55:04.855919+0000","last_clean":"2024-04-17T09:55:04.855919+0000","last_became_active":"2024-04-17T09:55:02.851014+0000","last_became_peered":"2024-04-17T09:55:02.851014+0000","last_unstale":"2024-04-17T09:55:04.855919+0000","last_undegraded":"2024-04-17T09:55:04.855919+0000","last_fullsized":"2024-04-17T09:55:04.855919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:07:25.155315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.857602+0000","last_change":"2024-04-17T09:55:03.850790+0000","last_active":"2024-04-17T09:55:03.857602+0000","last_peered":"2024-04-17T09:55:03.857602+0000","last_clean":"2024-04-17T09:55:03.857602+0000","last_became_active":"2024-04-17T09:55:03.849904+0000","last_became_peered":"2024-04-17T09:55:03.849904+0000","last_unstale":"2024-04-17T09:55:03.857602+0000","last_undegraded":"2024-04-17T09:55:03.857602+0000","last_fullsized":"2024-04-17T09:55:03.857602+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:54:04.483854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.872208+0000","last_change":"2024-04-17T09:55:02.842980+0000","last_active":"2024-04-17T09:55:03.872208+0000","last_peered":"2024-04-17T09:55:03.872208+0000","last_clean":"2024-04-17T09:55:03.872208+0000","last_became_active":"2024-04-17T09:55:02.842754+0000","last_became_peered":"2024-04-17T09:55:02.842754+0000","last_unstale":"2024-04-17T09:55:03.872208+0000","last_undegraded":"2024-04-17T09:55:03.872208+0000","last_fullsized":"2024-04-17T09:55:03.872208+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:21:41.297029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.860714+0000","last_change":"2024-04-17T09:55:03.854463+0000","last_active":"2024-04-17T09:55:03.860714+0000","last_peered":"2024-04-17T09:55:03.860714+0000","last_clean":"2024-04-17T09:55:03.860714+0000","last_became_active":"2024-04-17T09:55:03.854272+0000","last_became_peered":"2024-04-17T09:55:03.854272+0000","last_unstale":"2024-04-17T09:55:03.860714+0000","last_undegraded":"2024-04-17T09:55:03.860714+0000","last_fullsized":"2024-04-17T09:55:03.860714+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:35:24.615538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.834582+0000","last_change":"2024-04-17T09:55:02.846242+0000","last_active":"2024-04-17T09:55:03.834582+0000","last_peered":"2024-04-17T09:55:03.834582+0000","last_clean":"2024-04-17T09:55:03.834582+0000","last_became_active":"2024-04-17T09:55:02.845569+0000","last_became_peered":"2024-04-17T09:55:02.845569+0000","last_unstale":"2024-04-17T09:55:03.834582+0000","last_undegraded":"2024-04-17T09:55:03.834582+0000","last_fullsized":"2024-04-17T09:55:03.834582+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:14.024482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.843024+0000","last_change":"2024-04-17T09:55:03.845639+0000","last_active":"2024-04-17T09:55:04.843024+0000","last_peered":"2024-04-17T09:55:04.843024+0000","last_clean":"2024-04-17T09:55:04.843024+0000","last_became_active":"2024-04-17T09:55:03.845365+0000","last_became_peered":"2024-04-17T09:55:03.845365+0000","last_unstale":"2024-04-17T09:55:04.843024+0000","last_undegraded":"2024-04-17T09:55:04.843024+0000","last_fullsized":"2024-04-17T09:55:04.843024+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:56:23.155554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.842321+0000","last_change":"2024-04-17T09:55:02.846320+0000","last_active":"2024-04-17T09:55:03.842321+0000","last_peered":"2024-04-17T09:55:03.842321+0000","last_clean":"2024-04-17T09:55:03.842321+0000","last_became_active":"2024-04-17T09:55:02.846048+0000","last_became_peered":"2024-04-17T09:55:02.846048+0000","last_unstale":"2024-04-17T09:55:03.842321+0000","last_undegraded":"2024-04-17T09:55:03.842321+0000","last_fullsized":"2024-04-17T09:55:03.842321+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:21:13.601828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.868710+0000","last_change":"2024-04-17T09:55:03.865357+0000","last_active":"2024-04-17T09:55:03.868710+0000","last_peered":"2024-04-17T09:55:03.868710+0000","last_clean":"2024-04-17T09:55:03.868710+0000","last_became_active":"2024-04-17T09:55:03.864736+0000","last_became_peered":"2024-04-17T09:55:03.864736+0000","last_unstale":"2024-04-17T09:55:03.868710+0000","last_undegraded":"2024-04-17T09:55:03.868710+0000","last_fullsized":"2024-04-17T09:55:03.868710+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:20:50.772361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.846627+0000","last_change":"2024-04-17T09:55:02.840739+0000","last_active":"2024-04-17T09:55:03.846627+0000","last_peered":"2024-04-17T09:55:03.846627+0000","last_clean":"2024-04-17T09:55:03.846627+0000","last_became_active":"2024-04-17T09:55:02.840461+0000","last_became_peered":"2024-04-17T09:55:02.840461+0000","last_unstale":"2024-04-17T09:55:03.846627+0000","last_undegraded":"2024-04-17T09:55:03.846627+0000","last_fullsized":"2024-04-17T09:55:03.846627+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:24:21.820465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.852776+0000","last_change":"2024-04-17T09:55:03.847707+0000","last_active":"2024-04-17T09:55:03.852776+0000","last_peered":"2024-04-17T09:55:03.852776+0000","last_clean":"2024-04-17T09:55:03.852776+0000","last_became_active":"2024-04-17T09:55:03.847396+0000","last_became_peered":"2024-04-17T09:55:03.847396+0000","last_unstale":"2024-04-17T09:55:03.852776+0000","last_undegraded":"2024-04-17T09:55:03.852776+0000","last_fullsized":"2024-04-17T09:55:03.852776+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:19:43.980847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.851587+0000","last_change":"2024-04-17T09:55:02.839995+0000","last_active":"2024-04-17T09:55:03.851587+0000","last_peered":"2024-04-17T09:55:03.851587+0000","last_clean":"2024-04-17T09:55:03.851587+0000","last_became_active":"2024-04-17T09:55:02.839611+0000","last_became_peered":"2024-04-17T09:55:02.839611+0000","last_unstale":"2024-04-17T09:55:03.851587+0000","last_undegraded":"2024-04-17T09:55:03.851587+0000","last_fullsized":"2024-04-17T09:55:03.851587+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:46:04.195085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.856011+0000","last_change":"2024-04-17T09:55:03.850140+0000","last_active":"2024-04-17T09:55:04.856011+0000","last_peered":"2024-04-17T09:55:04.856011+0000","last_clean":"2024-04-17T09:55:04.856011+0000","last_became_active":"2024-04-17T09:55:03.849625+0000","last_became_peered":"2024-04-17T09:55:03.849625+0000","last_unstale":"2024-04-17T09:55:04.856011+0000","last_undegraded":"2024-04-17T09:55:04.856011+0000","last_fullsized":"2024-04-17T09:55:04.856011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:43:34.034038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.871985+0000","last_change":"2024-04-17T09:55:02.846604+0000","last_active":"2024-04-17T09:55:03.871985+0000","last_peered":"2024-04-17T09:55:03.871985+0000","last_clean":"2024-04-17T09:55:03.871985+0000","last_became_active":"2024-04-17T09:55:02.846351+0000","last_became_peered":"2024-04-17T09:55:02.846351+0000","last_unstale":"2024-04-17T09:55:03.871985+0000","last_undegraded":"2024-04-17T09:55:03.871985+0000","last_fullsized":"2024-04-17T09:55:03.871985+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:20.671582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.849102+0000","last_change":"2024-04-17T09:55:03.846805+0000","last_active":"2024-04-17T09:55:03.849102+0000","last_peered":"2024-04-17T09:55:03.849102+0000","last_clean":"2024-04-17T09:55:03.849102+0000","last_became_active":"2024-04-17T09:55:03.846502+0000","last_became_peered":"2024-04-17T09:55:03.846502+0000","last_unstale":"2024-04-17T09:55:03.849102+0000","last_undegraded":"2024-04-17T09:55:03.849102+0000","last_fullsized":"2024-04-17T09:55:03.849102+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:02.206573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.842855+0000","last_change":"2024-04-17T09:55:02.838983+0000","last_active":"2024-04-17T09:55:04.842855+0000","last_peered":"2024-04-17T09:55:04.842855+0000","last_clean":"2024-04-17T09:55:04.842855+0000","last_became_active":"2024-04-17T09:55:02.838432+0000","last_became_peered":"2024-04-17T09:55:02.838432+0000","last_unstale":"2024-04-17T09:55:04.842855+0000","last_undegraded":"2024-04-17T09:55:04.842855+0000","last_fullsized":"2024-04-17T09:55:04.842855+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:12:07.395491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.843635+0000","last_change":"2024-04-17T09:55:03.843497+0000","last_active":"2024-04-17T09:55:04.843635+0000","last_peered":"2024-04-17T09:55:04.843635+0000","last_clean":"2024-04-17T09:55:04.843635+0000","last_became_active":"2024-04-17T09:55:03.843185+0000","last_became_peered":"2024-04-17T09:55:03.843185+0000","last_unstale":"2024-04-17T09:55:04.843635+0000","last_undegraded":"2024-04-17T09:55:04.843635+0000","last_fullsized":"2024-04-17T09:55:04.843635+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:46:05.199714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.856029+0000","last_change":"2024-04-17T09:55:02.851638+0000","last_active":"2024-04-17T09:55:04.856029+0000","last_peered":"2024-04-17T09:55:04.856029+0000","last_clean":"2024-04-17T09:55:04.856029+0000","last_became_active":"2024-04-17T09:55:02.851391+0000","last_became_peered":"2024-04-17T09:55:02.851391+0000","last_unstale":"2024-04-17T09:55:04.856029+0000","last_undegraded":"2024-04-17T09:55:04.856029+0000","last_fullsized":"2024-04-17T09:55:04.856029+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:17:02.019906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.861559+0000","last_change":"2024-04-17T09:55:03.854380+0000","last_active":"2024-04-17T09:55:03.861559+0000","last_peered":"2024-04-17T09:55:03.861559+0000","last_clean":"2024-04-17T09:55:03.861559+0000","last_became_active":"2024-04-17T09:55:03.854044+0000","last_became_peered":"2024-04-17T09:55:03.854044+0000","last_unstale":"2024-04-17T09:55:03.861559+0000","last_undegraded":"2024-04-17T09:55:03.861559+0000","last_fullsized":"2024-04-17T09:55:03.861559+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:15:14.718813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.831177+0000","last_change":"2024-04-17T09:55:02.853317+0000","last_active":"2024-04-17T09:55:03.831177+0000","last_peered":"2024-04-17T09:55:03.831177+0000","last_clean":"2024-04-17T09:55:03.831177+0000","last_became_active":"2024-04-17T09:55:02.853057+0000","last_became_peered":"2024-04-17T09:55:02.853057+0000","last_unstale":"2024-04-17T09:55:03.831177+0000","last_undegraded":"2024-04-17T09:55:03.831177+0000","last_fullsized":"2024-04-17T09:55:03.831177+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:26:01.747901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.855670+0000","last_change":"2024-04-17T09:55:03.858259+0000","last_active":"2024-04-17T09:55:04.855670+0000","last_peered":"2024-04-17T09:55:04.855670+0000","last_clean":"2024-04-17T09:55:04.855670+0000","last_became_active":"2024-04-17T09:55:03.857952+0000","last_became_peered":"2024-04-17T09:55:03.857952+0000","last_unstale":"2024-04-17T09:55:04.855670+0000","last_undegraded":"2024-04-17T09:55:04.855670+0000","last_fullsized":"2024-04-17T09:55:04.855670+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:37.829815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.855792+0000","last_change":"2024-04-17T09:55:02.845545+0000","last_active":"2024-04-17T09:55:04.855792+0000","last_peered":"2024-04-17T09:55:04.855792+0000","last_clean":"2024-04-17T09:55:04.855792+0000","last_became_active":"2024-04-17T09:55:02.845226+0000","last_became_peered":"2024-04-17T09:55:02.845226+0000","last_unstale":"2024-04-17T09:55:04.855792+0000","last_undegraded":"2024-04-17T09:55:04.855792+0000","last_fullsized":"2024-04-17T09:55:04.855792+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:01.200566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.850476+0000","last_change":"2024-04-17T09:55:03.847514+0000","last_active":"2024-04-17T09:55:03.850476+0000","last_peered":"2024-04-17T09:55:03.850476+0000","last_clean":"2024-04-17T09:55:03.850476+0000","last_became_active":"2024-04-17T09:55:03.846942+0000","last_became_peered":"2024-04-17T09:55:03.846942+0000","last_unstale":"2024-04-17T09:55:03.850476+0000","last_undegraded":"2024-04-17T09:55:03.850476+0000","last_fullsized":"2024-04-17T09:55:03.850476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:09.781233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.875558+0000","last_change":"2024-04-17T09:55:02.845741+0000","last_active":"2024-04-17T09:55:03.875558+0000","last_peered":"2024-04-17T09:55:03.875558+0000","last_clean":"2024-04-17T09:55:03.875558+0000","last_became_active":"2024-04-17T09:55:02.845453+0000","last_became_peered":"2024-04-17T09:55:02.845453+0000","last_unstale":"2024-04-17T09:55:03.875558+0000","last_undegraded":"2024-04-17T09:55:03.875558+0000","last_fullsized":"2024-04-17T09:55:03.875558+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:21:41.297029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.843049+0000","last_change":"2024-04-17T09:55:03.843826+0000","last_active":"2024-04-17T09:55:04.843049+0000","last_peered":"2024-04-17T09:55:04.843049+0000","last_clean":"2024-04-17T09:55:04.843049+0000","last_became_active":"2024-04-17T09:55:03.843556+0000","last_became_peered":"2024-04-17T09:55:03.843556+0000","last_unstale":"2024-04-17T09:55:04.843049+0000","last_undegraded":"2024-04-17T09:55:04.843049+0000","last_fullsized":"2024-04-17T09:55:04.843049+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:07.056555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.842964+0000","last_change":"2024-04-17T09:55:02.851610+0000","last_active":"2024-04-17T09:55:04.842964+0000","last_peered":"2024-04-17T09:55:04.842964+0000","last_clean":"2024-04-17T09:55:04.842964+0000","last_became_active":"2024-04-17T09:55:02.851166+0000","last_became_peered":"2024-04-17T09:55:02.851166+0000","last_unstale":"2024-04-17T09:55:04.842964+0000","last_undegraded":"2024-04-17T09:55:04.842964+0000","last_fullsized":"2024-04-17T09:55:04.842964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:33:36.303801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.861291+0000","last_change":"2024-04-17T09:55:03.853543+0000","last_active":"2024-04-17T09:55:03.861291+0000","last_peered":"2024-04-17T09:55:03.861291+0000","last_clean":"2024-04-17T09:55:03.861291+0000","last_became_active":"2024-04-17T09:55:03.853269+0000","last_became_peered":"2024-04-17T09:55:03.853269+0000","last_unstale":"2024-04-17T09:55:03.861291+0000","last_undegraded":"2024-04-17T09:55:03.861291+0000","last_fullsized":"2024-04-17T09:55:03.861291+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:20:50.772361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.872455+0000","last_change":"2024-04-17T09:55:02.838898+0000","last_active":"2024-04-17T09:55:03.872455+0000","last_peered":"2024-04-17T09:55:03.872455+0000","last_clean":"2024-04-17T09:55:03.872455+0000","last_became_active":"2024-04-17T09:55:02.838259+0000","last_became_peered":"2024-04-17T09:55:02.838259+0000","last_unstale":"2024-04-17T09:55:03.872455+0000","last_undegraded":"2024-04-17T09:55:03.872455+0000","last_fullsized":"2024-04-17T09:55:03.872455+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:09:25.853640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.843393+0000","last_change":"2024-04-17T09:55:03.849813+0000","last_active":"2024-04-17T09:55:04.843393+0000","last_peered":"2024-04-17T09:55:04.843393+0000","last_clean":"2024-04-17T09:55:04.843393+0000","last_became_active":"2024-04-17T09:55:03.848297+0000","last_became_peered":"2024-04-17T09:55:03.848297+0000","last_unstale":"2024-04-17T09:55:04.843393+0000","last_undegraded":"2024-04-17T09:55:04.843393+0000","last_fullsized":"2024-04-17T09:55:04.843393+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:15.030489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.867862+0000","last_change":"2024-04-17T09:55:02.845754+0000","last_active":"2024-04-17T09:55:03.867862+0000","last_peered":"2024-04-17T09:55:03.867862+0000","last_clean":"2024-04-17T09:55:03.867862+0000","last_became_active":"2024-04-17T09:55:02.845238+0000","last_became_peered":"2024-04-17T09:55:02.845238+0000","last_unstale":"2024-04-17T09:55:03.867862+0000","last_undegraded":"2024-04-17T09:55:03.867862+0000","last_fullsized":"2024-04-17T09:55:03.867862+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:55:52.467932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.843739+0000","last_change":"2024-04-17T09:55:03.850786+0000","last_active":"2024-04-17T09:55:04.843739+0000","last_peered":"2024-04-17T09:55:04.843739+0000","last_clean":"2024-04-17T09:55:04.843739+0000","last_became_active":"2024-04-17T09:55:03.849305+0000","last_became_peered":"2024-04-17T09:55:03.849305+0000","last_unstale":"2024-04-17T09:55:04.843739+0000","last_undegraded":"2024-04-17T09:55:04.843739+0000","last_fullsized":"2024-04-17T09:55:04.843739+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:52:50.376225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.855975+0000","last_change":"2024-04-17T09:55:02.850448+0000","last_active":"2024-04-17T09:55:04.855975+0000","last_peered":"2024-04-17T09:55:04.855975+0000","last_clean":"2024-04-17T09:55:04.855975+0000","last_became_active":"2024-04-17T09:55:02.849631+0000","last_became_peered":"2024-04-17T09:55:02.849631+0000","last_unstale":"2024-04-17T09:55:04.855975+0000","last_undegraded":"2024-04-17T09:55:04.855975+0000","last_fullsized":"2024-04-17T09:55:04.855975+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:46:53.545268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.855766+0000","last_change":"2024-04-17T09:55:03.855613+0000","last_active":"2024-04-17T09:55:04.855766+0000","last_peered":"2024-04-17T09:55:04.855766+0000","last_clean":"2024-04-17T09:55:04.855766+0000","last_became_active":"2024-04-17T09:55:03.852617+0000","last_became_peered":"2024-04-17T09:55:03.852617+0000","last_unstale":"2024-04-17T09:55:04.855766+0000","last_undegraded":"2024-04-17T09:55:04.855766+0000","last_fullsized":"2024-04-17T09:55:04.855766+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:27:27.242594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.856133+0000","last_change":"2024-04-17T09:55:02.849110+0000","last_active":"2024-04-17T09:55:04.856133+0000","last_peered":"2024-04-17T09:55:04.856133+0000","last_clean":"2024-04-17T09:55:04.856133+0000","last_became_active":"2024-04-17T09:55:02.847524+0000","last_became_peered":"2024-04-17T09:55:02.847524+0000","last_unstale":"2024-04-17T09:55:04.856133+0000","last_undegraded":"2024-04-17T09:55:04.856133+0000","last_fullsized":"2024-04-17T09:55:04.856133+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:20.671582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.848967+0000","last_change":"2024-04-17T09:55:03.843367+0000","last_active":"2024-04-17T09:55:03.848967+0000","last_peered":"2024-04-17T09:55:03.848967+0000","last_clean":"2024-04-17T09:55:03.848967+0000","last_became_active":"2024-04-17T09:55:03.843054+0000","last_became_peered":"2024-04-17T09:55:03.843054+0000","last_unstale":"2024-04-17T09:55:03.848967+0000","last_undegraded":"2024-04-17T09:55:03.848967+0000","last_fullsized":"2024-04-17T09:55:03.848967+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:07.056555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.835648+0000","last_change":"2024-04-17T09:55:02.845764+0000","last_active":"2024-04-17T09:55:03.835648+0000","last_peered":"2024-04-17T09:55:03.835648+0000","last_clean":"2024-04-17T09:55:03.835648+0000","last_became_active":"2024-04-17T09:55:02.845436+0000","last_became_peered":"2024-04-17T09:55:02.845436+0000","last_unstale":"2024-04-17T09:55:03.835648+0000","last_undegraded":"2024-04-17T09:55:03.835648+0000","last_fullsized":"2024-04-17T09:55:03.835648+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:52:49.370217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.842642+0000","last_change":"2024-04-17T09:55:03.835558+0000","last_active":"2024-04-17T09:55:04.842642+0000","last_peered":"2024-04-17T09:55:04.842642+0000","last_clean":"2024-04-17T09:55:04.842642+0000","last_became_active":"2024-04-17T09:55:03.835292+0000","last_became_peered":"2024-04-17T09:55:03.835292+0000","last_unstale":"2024-04-17T09:55:04.842642+0000","last_undegraded":"2024-04-17T09:55:04.842642+0000","last_fullsized":"2024-04-17T09:55:04.842642+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:44:28.122478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.842509+0000","last_change":"2024-04-17T09:55:02.836905+0000","last_active":"2024-04-17T09:55:03.842509+0000","last_peered":"2024-04-17T09:55:03.842509+0000","last_clean":"2024-04-17T09:55:03.842509+0000","last_became_active":"2024-04-17T09:55:02.836560+0000","last_became_peered":"2024-04-17T09:55:02.836560+0000","last_unstale":"2024-04-17T09:55:03.842509+0000","last_undegraded":"2024-04-17T09:55:03.842509+0000","last_fullsized":"2024-04-17T09:55:03.842509+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:02.061750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.849751+0000","last_change":"2024-04-17T09:55:03.843921+0000","last_active":"2024-04-17T09:55:03.849751+0000","last_peered":"2024-04-17T09:55:03.849751+0000","last_clean":"2024-04-17T09:55:03.849751+0000","last_became_active":"2024-04-17T09:55:03.843719+0000","last_became_peered":"2024-04-17T09:55:03.843719+0000","last_unstale":"2024-04-17T09:55:03.849751+0000","last_undegraded":"2024-04-17T09:55:03.849751+0000","last_fullsized":"2024-04-17T09:55:03.849751+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:09:26.859647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.830893+0000","last_change":"2024-04-17T09:55:02.852972+0000","last_active":"2024-04-17T09:55:03.830893+0000","last_peered":"2024-04-17T09:55:03.830893+0000","last_clean":"2024-04-17T09:55:03.830893+0000","last_became_active":"2024-04-17T09:55:02.852753+0000","last_became_peered":"2024-04-17T09:55:02.852753+0000","last_unstale":"2024-04-17T09:55:03.830893+0000","last_undegraded":"2024-04-17T09:55:03.830893+0000","last_fullsized":"2024-04-17T09:55:03.830893+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:24:34.427573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.844057+0000","last_change":"2024-04-17T09:55:03.856033+0000","last_active":"2024-04-17T09:55:04.844057+0000","last_peered":"2024-04-17T09:55:04.844057+0000","last_clean":"2024-04-17T09:55:04.844057+0000","last_became_active":"2024-04-17T09:55:03.852577+0000","last_became_peered":"2024-04-17T09:55:03.852577+0000","last_unstale":"2024-04-17T09:55:04.844057+0000","last_undegraded":"2024-04-17T09:55:04.844057+0000","last_fullsized":"2024-04-17T09:55:04.844057+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:24:35.433580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.857625+0000","last_change":"2024-04-17T09:55:02.847850+0000","last_active":"2024-04-17T09:55:04.857625+0000","last_peered":"2024-04-17T09:55:04.857625+0000","last_clean":"2024-04-17T09:55:04.857625+0000","last_became_active":"2024-04-17T09:55:02.847505+0000","last_became_peered":"2024-04-17T09:55:02.847505+0000","last_unstale":"2024-04-17T09:55:04.857625+0000","last_undegraded":"2024-04-17T09:55:04.857625+0000","last_fullsized":"2024-04-17T09:55:04.857625+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:03.190518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.867201+0000","last_change":"2024-04-17T09:55:03.863759+0000","last_active":"2024-04-17T09:55:03.867201+0000","last_peered":"2024-04-17T09:55:03.867201+0000","last_clean":"2024-04-17T09:55:03.867201+0000","last_became_active":"2024-04-17T09:55:03.863488+0000","last_became_peered":"2024-04-17T09:55:03.863488+0000","last_unstale":"2024-04-17T09:55:03.867201+0000","last_undegraded":"2024-04-17T09:55:03.867201+0000","last_fullsized":"2024-04-17T09:55:03.867201+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:44:47.869450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.831888+0000","last_change":"2024-04-17T09:55:02.847782+0000","last_active":"2024-04-17T09:55:03.831888+0000","last_peered":"2024-04-17T09:55:03.831888+0000","last_clean":"2024-04-17T09:55:03.831888+0000","last_became_active":"2024-04-17T09:55:02.847491+0000","last_became_peered":"2024-04-17T09:55:02.847491+0000","last_unstale":"2024-04-17T09:55:03.831888+0000","last_undegraded":"2024-04-17T09:55:03.831888+0000","last_fullsized":"2024-04-17T09:55:03.831888+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:03.190518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.850986+0000","last_change":"2024-04-17T09:55:03.847527+0000","last_active":"2024-04-17T09:55:03.850986+0000","last_peered":"2024-04-17T09:55:03.850986+0000","last_clean":"2024-04-17T09:55:03.850986+0000","last_became_active":"2024-04-17T09:55:03.847162+0000","last_became_peered":"2024-04-17T09:55:03.847162+0000","last_unstale":"2024-04-17T09:55:03.850986+0000","last_undegraded":"2024-04-17T09:55:03.850986+0000","last_fullsized":"2024-04-17T09:55:03.850986+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:25:29.044313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.832605+0000","last_change":"2024-04-17T09:55:02.846877+0000","last_active":"2024-04-17T09:55:03.832605+0000","last_peered":"2024-04-17T09:55:03.832605+0000","last_clean":"2024-04-17T09:55:03.832605+0000","last_became_active":"2024-04-17T09:55:02.846522+0000","last_became_peered":"2024-04-17T09:55:02.846522+0000","last_unstale":"2024-04-17T09:55:03.832605+0000","last_undegraded":"2024-04-17T09:55:03.832605+0000","last_fullsized":"2024-04-17T09:55:03.832605+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:23:59.885800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.855854+0000","last_change":"2024-04-17T09:55:03.843549+0000","last_active":"2024-04-17T09:55:04.855854+0000","last_peered":"2024-04-17T09:55:04.855854+0000","last_clean":"2024-04-17T09:55:04.855854+0000","last_became_active":"2024-04-17T09:55:03.843248+0000","last_became_peered":"2024-04-17T09:55:03.843248+0000","last_unstale":"2024-04-17T09:55:04.855854+0000","last_undegraded":"2024-04-17T09:55:04.855854+0000","last_fullsized":"2024-04-17T09:55:04.855854+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:27:28.693741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.831459+0000","last_change":"2024-04-17T09:55:02.847288+0000","last_active":"2024-04-17T09:55:03.831459+0000","last_peered":"2024-04-17T09:55:03.831459+0000","last_clean":"2024-04-17T09:55:03.831459+0000","last_became_active":"2024-04-17T09:55:02.847030+0000","last_became_peered":"2024-04-17T09:55:02.847030+0000","last_unstale":"2024-04-17T09:55:03.831459+0000","last_undegraded":"2024-04-17T09:55:03.831459+0000","last_fullsized":"2024-04-17T09:55:03.831459+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:07:25.155315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-17T09:55:04.855997+0000","last_change":"2024-04-17T09:55:03.851746+0000","last_active":"2024-04-17T09:55:04.855997+0000","last_peered":"2024-04-17T09:55:04.855997+0000","last_clean":"2024-04-17T09:55:04.855997+0000","last_became_active":"2024-04-17T09:55:03.849989+0000","last_became_peered":"2024-04-17T09:55:03.849989+0000","last_unstale":"2024-04-17T09:55:04.855997+0000","last_undegraded":"2024-04-17T09:55:04.855997+0000","last_fullsized":"2024-04-17T09:55:04.855997+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:39.456313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.842042+0000","last_change":"2024-04-17T09:55:02.846614+0000","last_active":"2024-04-17T09:55:03.842042+0000","last_peered":"2024-04-17T09:55:03.842042+0000","last_clean":"2024-04-17T09:55:03.842042+0000","last_became_active":"2024-04-17T09:55:02.846303+0000","last_became_peered":"2024-04-17T09:55:02.846303+0000","last_unstale":"2024-04-17T09:55:03.842042+0000","last_undegraded":"2024-04-17T09:55:03.842042+0000","last_fullsized":"2024-04-17T09:55:03.842042+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:09:21.458482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.848013+0000","last_change":"2024-04-17T09:55:03.844424+0000","last_active":"2024-04-17T09:55:03.848013+0000","last_peered":"2024-04-17T09:55:03.848013+0000","last_clean":"2024-04-17T09:55:03.848013+0000","last_became_active":"2024-04-17T09:55:03.844169+0000","last_became_peered":"2024-04-17T09:55:03.844169+0000","last_unstale":"2024-04-17T09:55:03.848013+0000","last_undegraded":"2024-04-17T09:55:03.848013+0000","last_fullsized":"2024-04-17T09:55:03.848013+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:57:06.558212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.833276+0000","last_change":"2024-04-17T09:55:02.844542+0000","last_active":"2024-04-17T09:55:03.833276+0000","last_peered":"2024-04-17T09:55:03.833276+0000","last_clean":"2024-04-17T09:55:03.833276+0000","last_became_active":"2024-04-17T09:55:02.841873+0000","last_became_peered":"2024-04-17T09:55:02.841873+0000","last_unstale":"2024-04-17T09:55:03.833276+0000","last_undegraded":"2024-04-17T09:55:03.833276+0000","last_fullsized":"2024-04-17T09:55:03.833276+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:47:32.668093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.852858+0000","last_change":"2024-04-17T09:55:03.850047+0000","last_active":"2024-04-17T09:55:03.852858+0000","last_peered":"2024-04-17T09:55:03.852858+0000","last_clean":"2024-04-17T09:55:03.852858+0000","last_became_active":"2024-04-17T09:55:03.849786+0000","last_became_peered":"2024-04-17T09:55:03.849786+0000","last_unstale":"2024-04-17T09:55:03.852858+0000","last_undegraded":"2024-04-17T09:55:03.852858+0000","last_fullsized":"2024-04-17T09:55:03.852858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:54:04.483854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.842820+0000","last_change":"2024-04-17T09:55:02.850181+0000","last_active":"2024-04-17T09:55:03.842820+0000","last_peered":"2024-04-17T09:55:03.842820+0000","last_clean":"2024-04-17T09:55:03.842820+0000","last_became_active":"2024-04-17T09:55:02.849909+0000","last_became_peered":"2024-04-17T09:55:02.849909+0000","last_unstale":"2024-04-17T09:55:03.842820+0000","last_undegraded":"2024-04-17T09:55:03.842820+0000","last_fullsized":"2024-04-17T09:55:03.842820+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:09:21.458482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.861332+0000","last_change":"2024-04-17T09:55:03.853216+0000","last_active":"2024-04-17T09:55:03.861332+0000","last_peered":"2024-04-17T09:55:03.861332+0000","last_clean":"2024-04-17T09:55:03.861332+0000","last_became_active":"2024-04-17T09:55:03.852844+0000","last_became_peered":"2024-04-17T09:55:03.852844+0000","last_unstale":"2024-04-17T09:55:03.861332+0000","last_undegraded":"2024-04-17T09:55:03.861332+0000","last_fullsized":"2024-04-17T09:55:03.861332+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:02.816771+0000","last_clean_scrub_stamp":"2024-04-17T09:55:02.816771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:09:55.354856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.869028+0000","last_change":"2024-04-17T09:55:02.850319+0000","last_active":"2024-04-17T09:55:03.869028+0000","last_peered":"2024-04-17T09:55:03.869028+0000","last_clean":"2024-04-17T09:55:03.869028+0000","last_became_active":"2024-04-17T09:55:02.849973+0000","last_became_peered":"2024-04-17T09:55:02.849973+0000","last_unstale":"2024-04-17T09:55:03.869028+0000","last_undegraded":"2024-04-17T09:55:03.869028+0000","last_fullsized":"2024-04-17T09:55:03.869028+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:55:01.810763+0000","last_clean_scrub_stamp":"2024-04-17T09:55:01.810763+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:46:04.193707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":215,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-17T09:55:03.830955+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T09:55:03.830955+0000","last_peered":"2024-04-17T09:55:03.830955+0000","last_clean":"2024-04-17T09:55:03.830955+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T09:55:03.830955+0000","last_undegraded":"2024-04-17T09:55:03.830955+0000","last_fullsized":"2024-04-17T09:55:03.830955+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509468,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1118208,"data_stored":800526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":210740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509468,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":524288,"data_stored":210138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":544768,"data_stored":217006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":548864,"data_stored":218002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509468,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31700,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":524288,"data_stored":210138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542172,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31700,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":532480,"data_stored":211680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542172,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1101824,"data_stored":793661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T09:55:08.361 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T09:55:08.361 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-17T09:55:08.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.545+0000 7f5b18a00640 1 Processor -- start 2024-04-17T09:55:08.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.545+0000 7f5b18a00640 1 -- start start 2024-04-17T09:55:08.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.545+0000 7f5b18a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b141656d0 0x7f5b14167ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:08.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.545+0000 7f5b18a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b140d0130 con 0x7f5b1406ba30 2024-04-17T09:55:08.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.545+0000 7f5b18a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b1415d090 con 0x7f5b140cf370 2024-04-17T09:55:08.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.545+0000 7f5b18a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5b140cfef0 con 0x7f5b141656d0 2024-04-17T09:55:08.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.545+0000 7f5b12800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b141656d0 0x7f5b14167ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:08.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.545+0000 7f5b11e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5b1406ba30 0x7f5b1406be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33166/0 (socket says 172.21.15.32:33166) 2024-04-17T09:55:08.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.545+0000 7f5b11e00640 1 -- 172.21.15.32:0/996026414 learned_addr learned my addr 172.21.15.32:0/996026414 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:08.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.549+0000 7f5b12800640 1 -- 172.21.15.32:0/996026414 >> v1:172.21.15.144:6789/0 conn(0x7f5b140cf370 legacy=0x7f5b140cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:08.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.549+0000 7f5b12800640 1 -- 172.21.15.32:0/996026414 >> v1:172.21.15.32:6789/0 conn(0x7f5b1406ba30 legacy=0x7f5b1406be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:08.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.549+0000 7f5b12800640 1 -- 172.21.15.32:0/996026414 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1415d370 con 0x7f5b141656d0 2024-04-17T09:55:08.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.549+0000 7f5b12800640 1 --2- 172.21.15.32:0/996026414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b141656d0 0x7f5b14167ac0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009d90 tx=0x7f5b0c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514843bcf98ea59b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:08.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.549+0000 7f5b11400640 1 -- 172.21.15.32:0/996026414 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0b2070 con 0x7f5b141656d0 2024-04-17T09:55:08.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.549+0000 7f5b11400640 1 -- 172.21.15.32:0/996026414 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b0c0a58b0 con 0x7f5b141656d0 2024-04-17T09:55:08.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.549+0000 7f5b11400640 1 -- 172.21.15.32:0/996026414 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0a5c80 con 0x7f5b141656d0 2024-04-17T09:55:08.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.549+0000 7f5b18a00640 1 -- 172.21.15.32:0/996026414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b141656d0 msgr2=0x7f5b14167ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:08.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.549+0000 7f5b18a00640 1 --2- 172.21.15.32:0/996026414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b141656d0 0x7f5b14167ac0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009d90 tx=0x7f5b0c01c8e0 comp rx=0 tx=0).stop 2024-04-17T09:55:08.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.549+0000 7f5b18a00640 1 -- 172.21.15.32:0/996026414 shutdown_connections 2024-04-17T09:55:08.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.549+0000 7f5b18a00640 1 --2- 172.21.15.32:0/996026414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b141656d0 0x7f5b14167ac0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:08.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.549+0000 7f5b18a00640 1 -- 172.21.15.32:0/996026414 >> 172.21.15.32:0/996026414 conn(0x7f5b140b2120 msgr2=0x7f5b140b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:08.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.553+0000 7f5b18a00640 1 -- 172.21.15.32:0/996026414 shutdown_connections 2024-04-17T09:55:08.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.553+0000 7f5b18a00640 1 -- 172.21.15.32:0/996026414 wait complete. 2024-04-17T09:55:08.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.553+0000 7f5b18a00640 1 Processor -- start 2024-04-17T09:55:08.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.553+0000 7f5b18a00640 1 -- start start 2024-04-17T09:55:08.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.553+0000 7f5b18a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b1406ba30 0x7f5b1413a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:08.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.553+0000 7f5b18a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b140cf370 0x7f5b1413a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:08.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.553+0000 7f5b18a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b141656d0 0x7f5b1413d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:08.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.553+0000 7f5b18a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5b141463b0 con 0x7f5b141656d0 2024-04-17T09:55:08.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.553+0000 7f5b18a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5b14146230 con 0x7f5b1406ba30 2024-04-17T09:55:08.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.553+0000 7f5b18a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5b1416a820 con 0x7f5b140cf370 2024-04-17T09:55:08.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.553+0000 7f5b12800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b141656d0 0x7f5b1413d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:08.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.553+0000 7f5b12800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b141656d0 0x7f5b1413d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54038/0 (socket says 172.21.15.32:54038) 2024-04-17T09:55:08.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.553+0000 7f5b12800640 1 -- 172.21.15.32:0/1427616331 learned_addr learned my addr 172.21.15.32:0/1427616331 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:08.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.553+0000 7f5b11e00640 1 --2- 172.21.15.32:0/1427616331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b1406ba30 0x7f5b1413a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:08.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.553+0000 7f5b0be00640 1 --2- 172.21.15.32:0/1427616331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b140cf370 0x7f5b1413a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:08.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.553+0000 7f5b12800640 1 -- 172.21.15.32:0/1427616331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b140cf370 msgr2=0x7f5b1413a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:08.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.553+0000 7f5b12800640 1 --2- 172.21.15.32:0/1427616331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b140cf370 0x7f5b1413a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:08.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.553+0000 7f5b12800640 1 -- 172.21.15.32:0/1427616331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b1406ba30 msgr2=0x7f5b1413a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:08.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.553+0000 7f5b12800640 1 --2- 172.21.15.32:0/1427616331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b1406ba30 0x7f5b1413a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:08.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.553+0000 7f5b12800640 1 -- 172.21.15.32:0/1427616331 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1413a5b0 con 0x7f5b141656d0 2024-04-17T09:55:08.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.553+0000 7f5b12800640 1 --2- 172.21.15.32:0/1427616331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b141656d0 0x7f5b1413d1a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c01a510 tx=0x7f5b0c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:08.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.553+0000 7f5b0a000640 1 -- 172.21.15.32:0/1427616331 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0b2050 con 0x7f5b141656d0 2024-04-17T09:55:08.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.557+0000 7f5b0a000640 1 -- 172.21.15.32:0/1427616331 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b0c0be070 con 0x7f5b141656d0 2024-04-17T09:55:08.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.557+0000 7f5b0a000640 1 -- 172.21.15.32:0/1427616331 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0ada50 con 0x7f5b141656d0 2024-04-17T09:55:08.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.557+0000 7f5b18a00640 1 -- 172.21.15.32:0/1427616331 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b1413b2d0 con 0x7f5b141656d0 2024-04-17T09:55:08.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.557+0000 7f5b18a00640 1 -- 172.21.15.32:0/1427616331 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b1413b890 con 0x7f5b141656d0 2024-04-17T09:55:08.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.557+0000 7f5b18a00640 1 -- 172.21.15.32:0/1427616331 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b14068a90 con 0x7f5b141656d0 2024-04-17T09:55:08.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.557+0000 7f5b0a000640 1 -- 172.21.15.32:0/1427616331 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5b0c0a5730 con 0x7f5b141656d0 2024-04-17T09:55:08.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.557+0000 7f5b0a000640 1 --2- 172.21.15.32:0/1427616331 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5af0068380 0x7f5af006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:08.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.557+0000 7f5b0a000640 1 -- 172.21.15.32:0/1427616331 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f5b0c121b60 con 0x7f5b141656d0 2024-04-17T09:55:08.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.557+0000 7f5b11e00640 1 --2- 172.21.15.32:0/1427616331 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5af0068380 0x7f5af006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:08.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.557+0000 7f5b11e00640 1 --2- 172.21.15.32:0/1427616331 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5af0068380 0x7f5af006a840 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5b04005890 tx=0x7f5b04009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:08.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.561+0000 7f5b0a000640 1 -- 172.21.15.32:0/1427616331 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5b0c016560 con 0x7f5b141656d0 2024-04-17T09:55:08.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.785+0000 7f5b18a00640 1 -- 172.21.15.32:0/1427616331 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f5b1406c850 con 0x7f5b141656d0 2024-04-17T09:55:08.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.785+0000 7f5b0a000640 1 -- 172.21.15.32:0/1427616331 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7f5b0c0f1be0 con 0x7f5b141656d0 2024-04-17T09:55:08.788 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-17T09:55:08.788 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDPmx9mUXoLKxAANV7zhLvL0DqHngk4TtsdBQ== 2024-04-17T09:55:08.788 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2024-04-17T09:55:08.788 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2024-04-17T09:55:08.788 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-17T09:55:08.788 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2024-04-17T09:55:08.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.789+0000 7f5b18a00640 1 -- 172.21.15.32:0/1427616331 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5af0068380 msgr2=0x7f5af006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:08.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.789+0000 7f5b18a00640 1 --2- 172.21.15.32:0/1427616331 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5af0068380 0x7f5af006a840 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5b04005890 tx=0x7f5b04009040 comp rx=0 tx=0).stop 2024-04-17T09:55:08.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.789+0000 7f5b18a00640 1 -- 172.21.15.32:0/1427616331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b141656d0 msgr2=0x7f5b1413d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:08.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.789+0000 7f5b18a00640 1 --2- 172.21.15.32:0/1427616331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b141656d0 0x7f5b1413d1a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c01a510 tx=0x7f5b0c019680 comp rx=0 tx=0).stop 2024-04-17T09:55:08.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.789+0000 7f5b18a00640 1 -- 172.21.15.32:0/1427616331 shutdown_connections 2024-04-17T09:55:08.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.789+0000 7f5b18a00640 1 --2- 172.21.15.32:0/1427616331 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5af0068380 0x7f5af006a840 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:08.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.789+0000 7f5b18a00640 1 --2- 172.21.15.32:0/1427616331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b140cf370 0x7f5b1413a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:08.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.789+0000 7f5b18a00640 1 --2- 172.21.15.32:0/1427616331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b141656d0 0x7f5b1413d1a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:08.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.789+0000 7f5b18a00640 1 --2- 172.21.15.32:0/1427616331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b1406ba30 0x7f5b1413a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:08.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.789+0000 7f5b18a00640 1 -- 172.21.15.32:0/1427616331 >> 172.21.15.32:0/1427616331 conn(0x7f5b140b2120 msgr2=0x7f5b140b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:08.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.789+0000 7f5b18a00640 1 -- 172.21.15.32:0/1427616331 shutdown_connections 2024-04-17T09:55:08.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.789+0000 7f5b18a00640 1 -- 172.21.15.32:0/1427616331 wait complete. 2024-04-17T09:55:08.809 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:55:08.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.949+0000 7f0c22e00640 1 Processor -- start 2024-04-17T09:55:08.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.949+0000 7f0c22e00640 1 -- start start 2024-04-17T09:55:08.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.949+0000 7f0c22e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0c1c0ce8e0 0x7f0c1c0b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:08.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.949+0000 7f0c22e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0c1c09b100 0x7f0c1c0d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:08.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.949+0000 7f0c22e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c1c070d10 con 0x7f0c1c0ce500 2024-04-17T09:55:08.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.949+0000 7f0c22e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0c1c070ff0 con 0x7f0c1c09b100 2024-04-17T09:55:08.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.949+0000 7f0c22e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0c1c071340 con 0x7f0c1c0ce8e0 2024-04-17T09:55:08.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.949+0000 7f0c21a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0c1c0ce500 0x7f0c1c09a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33178/0 (socket says 172.21.15.32:33178) 2024-04-17T09:55:08.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.949+0000 7f0c21a00640 1 -- 172.21.15.32:0/1537412861 learned_addr learned my addr 172.21.15.32:0/1537412861 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:08.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.949+0000 7f0c22400640 1 --2- 172.21.15.32:0/1537412861 >> v2:172.21.15.32:3300/0 conn(0x7f0c1c09b100 0x7f0c1c0d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:08.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.949+0000 7f0c21000640 1 --2- 172.21.15.32:0/1537412861 >> v2:172.21.15.144:3300/0 conn(0x7f0c1c0ce8e0 0x7f0c1c0b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:08.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.949+0000 7f0c22400640 1 -- 172.21.15.32:0/1537412861 >> v2:172.21.15.144:3300/0 conn(0x7f0c1c0ce8e0 msgr2=0x7f0c1c0b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:08.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.949+0000 7f0c22400640 1 --2- 172.21.15.32:0/1537412861 >> v2:172.21.15.144:3300/0 conn(0x7f0c1c0ce8e0 0x7f0c1c0b4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:08.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.949+0000 7f0c22400640 1 -- 172.21.15.32:0/1537412861 >> v1:172.21.15.32:6789/0 conn(0x7f0c1c0ce500 legacy=0x7f0c1c09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:08.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.949+0000 7f0c22400640 1 -- 172.21.15.32:0/1537412861 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1c071620 con 0x7f0c1c09b100 2024-04-17T09:55:08.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.949+0000 7f0c1be00640 1 -- 172.21.15.32:0/1537412861 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2863611811 0 0) 0x7f0c1c070d10 con 0x7f0c1c0ce500 2024-04-17T09:55:08.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.949+0000 7f0c21000640 1 --2- 172.21.15.32:0/1537412861 >> v2:172.21.15.144:3300/0 conn(0x7f0c1c0ce8e0 0x7f0c1c0b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:08.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.953+0000 7f0c22400640 1 --2- 172.21.15.32:0/1537412861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c1c09b100 0x7f0c1c0d0cb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c06c4f0 tx=0x7f0c1401c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12003f6c54872efb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:08.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.953+0000 7f0c1be00640 1 -- 172.21.15.32:0/1537412861 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c140b2070 con 0x7f0c1c09b100 2024-04-17T09:55:08.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.953+0000 7f0c1be00640 1 -- 172.21.15.32:0/1537412861 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c140a57e0 con 0x7f0c1c09b100 2024-04-17T09:55:08.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.953+0000 7f0c1be00640 1 -- 172.21.15.32:0/1537412861 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c140a5bb0 con 0x7f0c1c09b100 2024-04-17T09:55:08.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.953+0000 7f0c22e00640 1 -- 172.21.15.32:0/1537412861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c1c09b100 msgr2=0x7f0c1c0d0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:08.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.953+0000 7f0c22e00640 1 --2- 172.21.15.32:0/1537412861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c1c09b100 0x7f0c1c0d0cb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c06c4f0 tx=0x7f0c1401c810 comp rx=0 tx=0).stop 2024-04-17T09:55:08.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.953+0000 7f0c22e00640 1 -- 172.21.15.32:0/1537412861 shutdown_connections 2024-04-17T09:55:08.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.953+0000 7f0c22e00640 1 --2- 172.21.15.32:0/1537412861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c1c09b100 0x7f0c1c0d0cb0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:08.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.953+0000 7f0c22e00640 1 --2- 172.21.15.32:0/1537412861 >> v2:172.21.15.144:3300/0 conn(0x7f0c1c0ce8e0 0x7f0c1c0b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:08.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.953+0000 7f0c22e00640 1 -- 172.21.15.32:0/1537412861 >> 172.21.15.32:0/1537412861 conn(0x7f0c1c0b3e50 msgr2=0x7f0c1c0b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:08.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.953+0000 7f0c22e00640 1 -- 172.21.15.32:0/1537412861 shutdown_connections 2024-04-17T09:55:08.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.953+0000 7f0c22e00640 1 -- 172.21.15.32:0/1537412861 wait complete. 2024-04-17T09:55:08.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c22e00640 1 Processor -- start 2024-04-17T09:55:08.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c22e00640 1 -- start start 2024-04-17T09:55:08.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c22e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c1c09b100 0x7f0c1c1486c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:08.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c22e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c1c0ce500 0x7f0c1c06d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:08.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c22e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c1c0ce8e0 0x7f0c1c06da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:08.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c22e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1c0cd550 con 0x7f0c1c09b100 2024-04-17T09:55:08.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c22e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1c070a60 con 0x7f0c1c0ce500 2024-04-17T09:55:08.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c22e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0c1c0cd6d0 con 0x7f0c1c0ce8e0 2024-04-17T09:55:08.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c21a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c1c09b100 0x7f0c1c1486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:08.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c21a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c1c09b100 0x7f0c1c1486c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54070/0 (socket says 172.21.15.32:54070) 2024-04-17T09:55:08.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c21a00640 1 -- 172.21.15.32:0/1297548983 learned_addr learned my addr 172.21.15.32:0/1297548983 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:08.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c21000640 1 --2- 172.21.15.32:0/1297548983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c1c0ce500 0x7f0c1c06d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:08.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c22400640 1 --2- 172.21.15.32:0/1297548983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c1c0ce8e0 0x7f0c1c06da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:08.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c21a00640 1 -- 172.21.15.32:0/1297548983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c1c0ce8e0 msgr2=0x7f0c1c06da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:08.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c21a00640 1 --2- 172.21.15.32:0/1297548983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c1c0ce8e0 0x7f0c1c06da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:08.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c21a00640 1 -- 172.21.15.32:0/1297548983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c1c0ce500 msgr2=0x7f0c1c06d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:08.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c21a00640 1 --2- 172.21.15.32:0/1297548983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c1c0ce500 0x7f0c1c06d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:08.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c21a00640 1 -- 172.21.15.32:0/1297548983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1c165470 con 0x7f0c1c09b100 2024-04-17T09:55:08.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c21a00640 1 --2- 172.21.15.32:0/1297548983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c1c09b100 0x7f0c1c1486c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0c100011f0 tx=0x7f0c1000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:08.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c19600640 1 -- 172.21.15.32:0/1297548983 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c10018070 con 0x7f0c1c09b100 2024-04-17T09:55:08.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c19600640 1 -- 172.21.15.32:0/1297548983 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c100074e0 con 0x7f0c1c09b100 2024-04-17T09:55:08.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c22e00640 1 -- 172.21.15.32:0/1297548983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c1c165750 con 0x7f0c1c09b100 2024-04-17T09:55:08.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c19600640 1 -- 172.21.15.32:0/1297548983 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c10004a10 con 0x7f0c1c09b100 2024-04-17T09:55:08.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.957+0000 7f0c22e00640 1 -- 172.21.15.32:0/1297548983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c1c165d10 con 0x7f0c1c09b100 2024-04-17T09:55:08.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.961+0000 7f0c22e00640 1 -- 172.21.15.32:0/1297548983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c1c04fd00 con 0x7f0c1c09b100 2024-04-17T09:55:08.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.961+0000 7f0c19600640 1 -- 172.21.15.32:0/1297548983 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0c1001b070 con 0x7f0c1c09b100 2024-04-17T09:55:08.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.961+0000 7f0c19600640 1 --2- 172.21.15.32:0/1297548983 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0bfc0683b0 0x7f0bfc06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:08.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.961+0000 7f0c19600640 1 -- 172.21.15.32:0/1297548983 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f0c10085cc0 con 0x7f0c1c09b100 2024-04-17T09:55:08.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.961+0000 7f0c21000640 1 --2- 172.21.15.32:0/1297548983 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0bfc0683b0 0x7f0bfc06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:08.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.961+0000 7f0c21000640 1 --2- 172.21.15.32:0/1297548983 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0bfc0683b0 0x7f0bfc06a870 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c06eb50 tx=0x7f0c08008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:08.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:08.965+0000 7f0c19600640 1 -- 172.21.15.32:0/1297548983 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0c10055b90 con 0x7f0c1c09b100 2024-04-17T09:55:09.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.165+0000 7f0c22e00640 1 -- 172.21.15.32:0/1297548983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c1c148c00 con 0x7f0c1c09b100 2024-04-17T09:55:09.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.165+0000 7f0c19600640 1 -- 172.21.15.32:0/1297548983 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5202 (secure 0 0 0) 0x7f0c10059e90 con 0x7f0c1c09b100 2024-04-17T09:55:09.167 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:09.167 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2542578926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2542578926},{"type":"v1","addr":"172.21.15.144:6834","nonce":2542578926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1424948520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1424948520},{"type":"v1","addr":"172.21.15.32:6836","nonce":1424948520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/393813154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":393813154},{"type":"v1","addr":"172.21.15.32:6837","nonce":393813154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:55:03.233552+0000","modified":"2024-04-17T09:55:06.243321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5742":{"gid":5742,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/2177384675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2177384675},{"type":"v1","addr":"172.21.15.144:6835","nonce":2177384675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742]},"id":3}]} 2024-04-17T09:55:09.167 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2024-04-17T09:55:09.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.169+0000 7f0c22e00640 1 -- 172.21.15.32:0/1297548983 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0bfc0683b0 msgr2=0x7f0bfc06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:09.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.169+0000 7f0c22e00640 1 --2- 172.21.15.32:0/1297548983 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0bfc0683b0 0x7f0bfc06a870 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c06eb50 tx=0x7f0c08008040 comp rx=0 tx=0).stop 2024-04-17T09:55:09.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.169+0000 7f0c22e00640 1 -- 172.21.15.32:0/1297548983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c1c09b100 msgr2=0x7f0c1c1486c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:09.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.169+0000 7f0c22e00640 1 --2- 172.21.15.32:0/1297548983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c1c09b100 0x7f0c1c1486c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0c100011f0 tx=0x7f0c1000c9b0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.169+0000 7f0c22e00640 1 -- 172.21.15.32:0/1297548983 shutdown_connections 2024-04-17T09:55:09.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.169+0000 7f0c22e00640 1 --2- 172.21.15.32:0/1297548983 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0bfc0683b0 0x7f0bfc06a870 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.169+0000 7f0c22e00640 1 --2- 172.21.15.32:0/1297548983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c1c0ce8e0 0x7f0c1c06da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.169+0000 7f0c22e00640 1 --2- 172.21.15.32:0/1297548983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c1c0ce500 0x7f0c1c06d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.169+0000 7f0c22e00640 1 --2- 172.21.15.32:0/1297548983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c1c09b100 0x7f0c1c1486c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.169+0000 7f0c22e00640 1 -- 172.21.15.32:0/1297548983 >> 172.21.15.32:0/1297548983 conn(0x7f0c1c0b3e50 msgr2=0x7f0c1c0ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:09.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.169+0000 7f0c22e00640 1 -- 172.21.15.32:0/1297548983 shutdown_connections 2024-04-17T09:55:09.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.169+0000 7f0c22e00640 1 -- 172.21.15.32:0/1297548983 wait complete. 2024-04-17T09:55:09.590 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T09:55:03.233552+0000', 'modified': '2024-04-17T09:55:06.243321+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5742}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5742': {'gid': 5742, 'name': 'd', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.144:6835/2177384675', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6833', 'nonce': 2177384675}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 2177384675}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5742, 'qdb_cluster': [5742]} 2024-04-17T09:55:09.591 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T09:55:09.591 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-17T09:55:09.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.733+0000 7faa7e800640 1 Processor -- start 2024-04-17T09:55:09.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.733+0000 7faa7e800640 1 -- start start 2024-04-17T09:55:09.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.737+0000 7faa7e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa78155a70 0x7faa7815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.737+0000 7faa7e800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa7815f870 con 0x7faa78155e50 2024-04-17T09:55:09.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.737+0000 7faa7e800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa7815fb50 con 0x7faa780b3340 2024-04-17T09:55:09.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.737+0000 7faa7e800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7faa780b42a0 con 0x7faa78155a70 2024-04-17T09:55:09.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.737+0000 7faa6fe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7faa78155e50 0x7faa7815d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33190/0 (socket says 172.21.15.32:33190) 2024-04-17T09:55:09.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.737+0000 7faa6fe00640 1 -- 172.21.15.32:0/210614279 learned_addr learned my addr 172.21.15.32:0/210614279 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:09.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.737+0000 7faa77400640 1 --2- 172.21.15.32:0/210614279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa78155a70 0x7faa7815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.737+0000 7faa76a00640 1 -- 172.21.15.32:0/210614279 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 955637914 0 0) 0x7faa7815f870 con 0x7faa78155e50 2024-04-17T09:55:09.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.737+0000 7faa76a00640 1 -- 172.21.15.32:0/210614279 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa5c005740 con 0x7faa78155e50 2024-04-17T09:55:09.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.737+0000 7faa76a00640 1 -- 172.21.15.32:0/210614279 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1733242893 0 0) 0x7faa7815fb50 con 0x7faa780b3340 2024-04-17T09:55:09.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.737+0000 7faa76a00640 1 -- 172.21.15.32:0/210614279 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa7815f870 con 0x7faa780b3340 2024-04-17T09:55:09.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.737+0000 7faa76a00640 1 -- 172.21.15.32:0/210614279 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 249275461 0 0) 0x7faa5c005740 con 0x7faa78155e50 2024-04-17T09:55:09.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.737+0000 7faa77400640 1 -- 172.21.15.32:0/210614279 >> v1:172.21.15.144:6789/0 conn(0x7faa780b3340 legacy=0x7faa7815c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:09.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.737+0000 7faa77400640 1 -- 172.21.15.32:0/210614279 >> v1:172.21.15.32:6789/0 conn(0x7faa78155e50 legacy=0x7faa7815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:09.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.737+0000 7faa77400640 1 -- 172.21.15.32:0/210614279 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa7815fe30 con 0x7faa78155a70 2024-04-17T09:55:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.741+0000 7faa77400640 1 --2- 172.21.15.32:0/210614279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa78155a70 0x7faa7815f190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faa780b4420 tx=0x7faa6801c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=636947c09c27ed5c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.741+0000 7faa76a00640 1 -- 172.21.15.32:0/210614279 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa680b0070 con 0x7faa78155a70 2024-04-17T09:55:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.741+0000 7faa76a00640 1 -- 172.21.15.32:0/210614279 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa680ab400 con 0x7faa78155a70 2024-04-17T09:55:09.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.741+0000 7faa76a00640 1 -- 172.21.15.32:0/210614279 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60001680 con 0x7faa78155a70 2024-04-17T09:55:09.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.741+0000 7faa7e800640 1 -- 172.21.15.32:0/210614279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa78155a70 msgr2=0x7faa7815f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:09.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.741+0000 7faa7e800640 1 --2- 172.21.15.32:0/210614279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa78155a70 0x7faa7815f190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faa780b4420 tx=0x7faa6801c3e0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.741+0000 7faa7e800640 1 -- 172.21.15.32:0/210614279 shutdown_connections 2024-04-17T09:55:09.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.741+0000 7faa7e800640 1 --2- 172.21.15.32:0/210614279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa78155a70 0x7faa7815f190 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.741+0000 7faa7e800640 1 -- 172.21.15.32:0/210614279 >> 172.21.15.32:0/210614279 conn(0x7faa7806ba30 msgr2=0x7faa7806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:09.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.741+0000 7faa7e800640 1 -- 172.21.15.32:0/210614279 shutdown_connections 2024-04-17T09:55:09.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.741+0000 7faa7e800640 1 -- 172.21.15.32:0/210614279 wait complete. 2024-04-17T09:55:09.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.741+0000 7faa7e800640 1 Processor -- start 2024-04-17T09:55:09.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.741+0000 7faa7e800640 1 -- start start 2024-04-17T09:55:09.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.741+0000 7faa7e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa780b3340 0x7faa78147c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.741+0000 7faa7e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa78155a70 0x7faa781f8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.745+0000 7faa7e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa78155e50 0x7faa78149e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.745+0000 7faa7e800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faa780d02f0 con 0x7faa78155a70 2024-04-17T09:55:09.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.745+0000 7faa7e800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faa780d0170 con 0x7faa78155e50 2024-04-17T09:55:09.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.745+0000 7faa7e800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7faa780d0470 con 0x7faa780b3340 2024-04-17T09:55:09.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.745+0000 7faa6fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa78155a70 0x7faa781f8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.745+0000 7faa6fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa78155a70 0x7faa781f8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54084/0 (socket says 172.21.15.32:54084) 2024-04-17T09:55:09.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.745+0000 7faa6fe00640 1 -- 172.21.15.32:0/1784582708 learned_addr learned my addr 172.21.15.32:0/1784582708 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:09.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.745+0000 7faa77400640 1 --2- 172.21.15.32:0/1784582708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa780b3340 0x7faa78147c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.745+0000 7faa77e00640 1 --2- 172.21.15.32:0/1784582708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa78155e50 0x7faa78149e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.745+0000 7faa6fe00640 1 -- 172.21.15.32:0/1784582708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa780b3340 msgr2=0x7faa78147c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:09.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.745+0000 7faa6fe00640 1 --2- 172.21.15.32:0/1784582708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa780b3340 0x7faa78147c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.745+0000 7faa6fe00640 1 -- 172.21.15.32:0/1784582708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa78155e50 msgr2=0x7faa78149e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:09.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.745+0000 7faa6fe00640 1 --2- 172.21.15.32:0/1784582708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa78155e50 0x7faa78149e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.745+0000 7faa6fe00640 1 -- 172.21.15.32:0/1784582708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa781f75a0 con 0x7faa78155a70 2024-04-17T09:55:09.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.745+0000 7faa6fe00640 1 --2- 172.21.15.32:0/1784582708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa78155a70 0x7faa781f8300 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7faa60000fe0 tx=0x7faa6000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.745+0000 7faa6f400640 1 -- 172.21.15.32:0/1784582708 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60017070 con 0x7faa78155a70 2024-04-17T09:55:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.745+0000 7faa6f400640 1 -- 172.21.15.32:0/1784582708 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa60005ae0 con 0x7faa78155a70 2024-04-17T09:55:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.745+0000 7faa6f400640 1 -- 172.21.15.32:0/1784582708 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60007070 con 0x7faa78155a70 2024-04-17T09:55:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.745+0000 7faa7e800640 1 -- 172.21.15.32:0/1784582708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa78155480 con 0x7faa78155a70 2024-04-17T09:55:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.745+0000 7faa7e800640 1 -- 172.21.15.32:0/1784582708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa781f7b00 con 0x7faa78155a70 2024-04-17T09:55:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.745+0000 7faa6f400640 1 -- 172.21.15.32:0/1784582708 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7faa6001a070 con 0x7faa78155a70 2024-04-17T09:55:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa6f400640 1 --2- 172.21.15.32:0/1784582708 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faa54068380 0x7faa5406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa6f400640 1 -- 172.21.15.32:0/1784582708 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7faa60085db0 con 0x7faa78155a70 2024-04-17T09:55:09.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa77400640 1 --2- 172.21.15.32:0/1784582708 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faa54068380 0x7faa5406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa81cf0000 1 Processor -- start 2024-04-17T09:55:09.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa81cf0000 1 -- start start 2024-04-17T09:55:09.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa77400640 1 --2- 172.21.15.32:0/1784582708 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faa54068380 0x7faa5406a840 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7faa680ad000 tx=0x7faa680a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:09.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa81cf0000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac4b50 0x55bd13ac4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa81cf0000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd13ac5470 0x55bd13a7a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa81cf0000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bd13a7acf0 0x55bd13a7b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa81cf0000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55bd138c84d0 con 0x55bd13ac4b50 2024-04-17T09:55:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa81cf0000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bd13862bf0 con 0x55bd13ac5470 2024-04-17T09:55:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa81cf0000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55bd138c4490 con 0x55bd13a7acf0 2024-04-17T09:55:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa77400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac4b50 0x55bd13ac4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa77400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac4b50 0x55bd13ac4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54086/0 (socket says 172.21.15.32:54086) 2024-04-17T09:55:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa77400640 1 -- 172.21.15.32:0/3058192603 learned_addr learned my addr 172.21.15.32:0/3058192603 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa77e00640 1 --2- 172.21.15.32:0/3058192603 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bd13a7acf0 0x55bd13a7b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa6fe00640 1 --2- 172.21.15.32:0/3058192603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd13ac5470 0x55bd13a7a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa77400640 1 -- 172.21.15.32:0/3058192603 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bd13a7acf0 msgr2=0x55bd13a7b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa77400640 1 --2- 172.21.15.32:0/3058192603 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bd13a7acf0 0x55bd13a7b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa77400640 1 -- 172.21.15.32:0/3058192603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd13ac5470 msgr2=0x55bd13a7a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa77400640 1 --2- 172.21.15.32:0/3058192603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd13ac5470 0x55bd13a7a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa77400640 1 -- 172.21.15.32:0/3058192603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd1384ee80 con 0x55bd13ac4b50 2024-04-17T09:55:09.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa6fe00640 1 --2- 172.21.15.32:0/3058192603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd13ac5470 0x55bd13a7a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:55:09.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa77e00640 1 --2- 172.21.15.32:0/3058192603 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bd13a7acf0 0x55bd13a7b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:55:09.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa77400640 1 --2- 172.21.15.32:0/3058192603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac4b50 0x55bd13ac4f30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7faa6801c420 tx=0x7faa68004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d003ddb21b9a3c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:09.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa67e00640 1 -- 172.21.15.32:0/3058192603 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa680b0040 con 0x55bd13ac4b50 2024-04-17T09:55:09.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa67e00640 1 -- 172.21.15.32:0/3058192603 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa680b1040 con 0x55bd13ac4b50 2024-04-17T09:55:09.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa67e00640 1 -- 172.21.15.32:0/3058192603 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa680c3410 con 0x55bd13ac4b50 2024-04-17T09:55:09.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa81cf0000 1 -- 172.21.15.32:0/3058192603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac4b50 msgr2=0x55bd13ac4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:09.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa81cf0000 1 --2- 172.21.15.32:0/3058192603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac4b50 0x55bd13ac4f30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7faa6801c420 tx=0x7faa68004740 comp rx=0 tx=0).stop 2024-04-17T09:55:09.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa81cf0000 1 -- 172.21.15.32:0/3058192603 shutdown_connections 2024-04-17T09:55:09.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa81cf0000 1 --2- 172.21.15.32:0/3058192603 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bd13a7acf0 0x55bd13a7b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa81cf0000 1 --2- 172.21.15.32:0/3058192603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd13ac5470 0x55bd13a7a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa81cf0000 1 --2- 172.21.15.32:0/3058192603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac4b50 0x55bd13ac4f30 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa81cf0000 1 -- 172.21.15.32:0/3058192603 >> 172.21.15.32:0/3058192603 conn(0x55bd13a80470 msgr2=0x55bd13a79090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:09.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa81cf0000 1 -- 172.21.15.32:0/3058192603 shutdown_connections 2024-04-17T09:55:09.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.749+0000 7faa81cf0000 1 -- 172.21.15.32:0/3058192603 wait complete. 2024-04-17T09:55:09.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa81cf0000 1 -- start start 2024-04-17T09:55:09.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa81cf0000 1 Processor -- start 2024-04-17T09:55:09.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa81cf0000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd13a7acf0 0x55bd13a7f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa81cf0000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac4b50 0x55bd13ab2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa81cf0000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bd13ac5470 0x55bd13ab2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa81cf0000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55bd13885a60 con 0x55bd13ac4b50 2024-04-17T09:55:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa81cf0000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bd1386ce80 con 0x55bd13a7acf0 2024-04-17T09:55:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa81cf0000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55bd13883cf0 con 0x55bd13ac5470 2024-04-17T09:55:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa81cf0000 10 client.0 _ll_register_callbacks cb 0x55bd13a65a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:55:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa6fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac4b50 0x55bd13ab2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa6fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac4b50 0x55bd13ab2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54088/0 (socket says 172.21.15.32:54088) 2024-04-17T09:55:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa6fe00640 1 -- 172.21.15.32:0/122446243 learned_addr learned my addr 172.21.15.32:0/122446243 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa77400640 1 --2- 172.21.15.32:0/122446243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd13a7acf0 0x55bd13a7f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa77e00640 1 --2- 172.21.15.32:0/122446243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bd13ac5470 0x55bd13ab2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa6fe00640 1 -- 172.21.15.32:0/122446243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bd13ac5470 msgr2=0x55bd13ab2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa6fe00640 1 --2- 172.21.15.32:0/122446243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bd13ac5470 0x55bd13ab2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa6fe00640 1 -- 172.21.15.32:0/122446243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd13a7acf0 msgr2=0x55bd13a7f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa6fe00640 1 --2- 172.21.15.32:0/122446243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd13a7acf0 0x55bd13a7f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa6fe00640 1 -- 172.21.15.32:0/122446243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd13862e40 con 0x55bd13ac4b50 2024-04-17T09:55:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa77e00640 1 --2- 172.21.15.32:0/122446243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bd13ac5470 0x55bd13ab2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:55:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa6fe00640 1 --2- 172.21.15.32:0/122446243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac4b50 0x55bd13ab2440 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7faa6000cfd0 tx=0x7faa60055dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a3eca3cf7e1f75 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa64c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:55:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa64c00640 1 -- 172.21.15.32:0/122446243 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60082060 con 0x55bd13ac4b50 2024-04-17T09:55:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa81cf0000 1 -- 172.21.15.32:0/122446243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa680b1040 con 0x55bd13ac4b50 2024-04-17T09:55:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa64c00640 1 -- 172.21.15.32:0/122446243 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7faa6005e5d0 con 0x55bd13ac4b50 2024-04-17T09:55:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa64c00640 1 -- 172.21.15.32:0/122446243 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faa60073050 con 0x55bd13ac4b50 2024-04-17T09:55:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa81cf0000 10 client.5883 fetch_fsmap learned FSMap version 24 2024-04-17T09:55:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa81cf0000 1 -- 172.21.15.32:0/122446243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55bd13893e00 con 0x55bd13ac4b50 2024-04-17T09:55:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa64c00640 1 -- 172.21.15.32:0/122446243 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa680b1040 con 0x55bd13ac4b50 2024-04-17T09:55:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa64c00640 1 -- 172.21.15.32:0/122446243 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) ==== 3495+0+0 (secure 0 0 0) 0x7faa60055160 con 0x55bd13ac4b50 2024-04-17T09:55:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa81cf0000 10 client.5883 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T09:55:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa81cf0000 10 client.5883 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-17T09:55:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa81cf0000 20 client.5883 populate_metadata read hostname 'smithi032' 2024-04-17T09:55:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa81cf0000 1 --2- 172.21.15.32:0/122446243 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55bd13ad40b0 0x55bd13ad64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa81cf0000 4 client.5883 mds_command: new command op to 5742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:55:09.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa81cf0000 1 -- 172.21.15.32:0/122446243 --> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd13861bc0 con 0x55bd13ad40b0 2024-04-17T09:55:09.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.753+0000 7faa77400640 1 --2- 172.21.15.32:0/122446243 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55bd13ad40b0 0x55bd13ad64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.757+0000 7faa77400640 1 --2- 172.21.15.32:0/122446243 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55bd13ad40b0 0x55bd13ad64a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e90900f6f81c6d92 server_cookie=23419a3d42e47418 in_seq=0 out_seq=0 2024-04-17T09:55:09.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.757+0000 7faa64c00640 10 client.5883 ms_handle_connect on v2:172.21.15.144:6833/2177384675 2024-04-17T09:55:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.757+0000 7faa64c00640 1 -- 172.21.15.32:0/122446243 <== mds.0 v2:172.21.15.144:6833/2177384675 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bd13861bc0 con 0x55bd13ad40b0 2024-04-17T09:55:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.757+0000 7faa64c00640 10 client.5883 handle_command_reply: tid=0 2024-04-17T09:55:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.757+0000 7faa81cf0000 1 client.5883 shutdown 2024-04-17T09:55:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.757+0000 7faa81cf0000 1 -- 172.21.15.32:0/122446243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac4b50 msgr2=0x55bd13ab2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.757+0000 7faa81cf0000 1 --2- 172.21.15.32:0/122446243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac4b50 0x55bd13ab2440 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7faa6000cfd0 tx=0x7faa60055dd0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.757+0000 7faa81cf0000 1 -- 172.21.15.32:0/122446243 shutdown_connections 2024-04-17T09:55:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.757+0000 7faa81cf0000 1 --2- 172.21.15.32:0/122446243 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55bd13ad40b0 0x55bd13ad64a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.757+0000 7faa81cf0000 1 --2- 172.21.15.32:0/122446243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bd13ac5470 0x55bd13ab2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.757+0000 7faa81cf0000 1 --2- 172.21.15.32:0/122446243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac4b50 0x55bd13ab2440 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.757+0000 7faa81cf0000 1 --2- 172.21.15.32:0/122446243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd13a7acf0 0x55bd13a7f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.757+0000 7faa64c00640 0 client.5883 ms_handle_reset on v2:172.21.15.144:6833/2177384675 2024-04-17T09:55:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.757+0000 7faa81cf0000 1 -- 172.21.15.32:0/122446243 >> 172.21.15.32:0/122446243 conn(0x55bd13a80470 msgr2=0x55bd13a797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.757+0000 7faa81cf0000 1 -- 172.21.15.32:0/122446243 shutdown_connections 2024-04-17T09:55:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.757+0000 7faa81cf0000 1 -- 172.21.15.32:0/122446243 wait complete. 2024-04-17T09:55:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.757+0000 7faa81cf0000 20 client.5883 trim_cache size 0 max 16384 2024-04-17T09:55:09.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa81cf0000 1 Processor -- start 2024-04-17T09:55:09.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa81cf0000 1 -- start start 2024-04-17T09:55:09.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa81cf0000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bd13a7acf0 0x55bd13ab2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa81cf0000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd13ac4b50 0x55bd13ab2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa81cf0000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac5470 0x55bd13ad54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa81cf0000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faa78202ed0 con 0x55bd13ac5470 2024-04-17T09:55:09.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa81cf0000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faa7814da00 con 0x55bd13ac4b50 2024-04-17T09:55:09.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa81cf0000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7faa780ceaa0 con 0x55bd13a7acf0 2024-04-17T09:55:09.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa77e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac5470 0x55bd13ad54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa77e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac5470 0x55bd13ad54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54092/0 (socket says 172.21.15.32:54092) 2024-04-17T09:55:09.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa77e00640 1 -- 172.21.15.32:0/772996194 learned_addr learned my addr 172.21.15.32:0/772996194 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:09.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa77400640 1 --2- 172.21.15.32:0/772996194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bd13a7acf0 0x55bd13ab2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa6fe00640 1 --2- 172.21.15.32:0/772996194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd13ac4b50 0x55bd13ab2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa77e00640 1 -- 172.21.15.32:0/772996194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bd13a7acf0 msgr2=0x55bd13ab2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:09.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa77e00640 1 --2- 172.21.15.32:0/772996194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bd13a7acf0 0x55bd13ab2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa77e00640 1 -- 172.21.15.32:0/772996194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd13ac4b50 msgr2=0x55bd13ab2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:09.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa77e00640 1 --2- 172.21.15.32:0/772996194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd13ac4b50 0x55bd13ab2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa77e00640 1 -- 172.21.15.32:0/772996194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd13949d30 con 0x55bd13ac5470 2024-04-17T09:55:09.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa77400640 1 --2- 172.21.15.32:0/772996194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bd13a7acf0 0x55bd13ab2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:09.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa77e00640 1 --2- 172.21.15.32:0/772996194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac5470 0x55bd13ad54a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7faa70001d20 tx=0x7faa700021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27101899873e543d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:09.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa67400640 1 -- 172.21.15.32:0/772996194 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70010e10 con 0x55bd13ac5470 2024-04-17T09:55:09.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa67400640 1 -- 172.21.15.32:0/772996194 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa70019070 con 0x55bd13ac5470 2024-04-17T09:55:09.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa67400640 1 -- 172.21.15.32:0/772996194 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70003b80 con 0x55bd13ac5470 2024-04-17T09:55:09.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa81cf0000 1 -- 172.21.15.32:0/772996194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac5470 msgr2=0x55bd13ad54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:09.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa81cf0000 1 --2- 172.21.15.32:0/772996194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac5470 0x55bd13ad54a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7faa70001d20 tx=0x7faa700021e0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa81cf0000 1 -- 172.21.15.32:0/772996194 shutdown_connections 2024-04-17T09:55:09.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa81cf0000 1 --2- 172.21.15.32:0/772996194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac5470 0x55bd13ad54a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa81cf0000 1 --2- 172.21.15.32:0/772996194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd13ac4b50 0x55bd13ab2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa81cf0000 1 --2- 172.21.15.32:0/772996194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bd13a7acf0 0x55bd13ab2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa81cf0000 1 -- 172.21.15.32:0/772996194 >> 172.21.15.32:0/772996194 conn(0x55bd13a80470 msgr2=0x55bd13ab1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:09.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa81cf0000 1 -- 172.21.15.32:0/772996194 shutdown_connections 2024-04-17T09:55:09.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa81cf0000 1 -- 172.21.15.32:0/772996194 wait complete. 2024-04-17T09:55:09.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa81cf0000 1 -- start start 2024-04-17T09:55:09.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.777+0000 7faa81cf0000 1 Processor -- start 2024-04-17T09:55:09.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa81cf0000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bd13a7acf0 0x55bd13ad8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa81cf0000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac4b50 0x55bd13af0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa81cf0000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd13ac5470 0x55bd13af0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa81cf0000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55bd138ccb00 con 0x55bd13ac4b50 2024-04-17T09:55:09.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa81cf0000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bd138c77e0 con 0x55bd13ac5470 2024-04-17T09:55:09.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa81cf0000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55bd138d56e0 con 0x55bd13a7acf0 2024-04-17T09:55:09.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa81cf0000 10 client.0 _ll_register_callbacks cb 0x7faa680ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:55:09.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa6fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac4b50 0x55bd13af0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa6fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac4b50 0x55bd13af0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54104/0 (socket says 172.21.15.32:54104) 2024-04-17T09:55:09.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa6fe00640 1 -- 172.21.15.32:0/907218458 learned_addr learned my addr 172.21.15.32:0/907218458 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:09.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa77e00640 1 --2- 172.21.15.32:0/907218458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd13ac5470 0x55bd13af0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa77400640 1 --2- 172.21.15.32:0/907218458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bd13a7acf0 0x55bd13ad8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa6fe00640 1 -- 172.21.15.32:0/907218458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bd13a7acf0 msgr2=0x55bd13ad8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:09.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa6fe00640 1 --2- 172.21.15.32:0/907218458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bd13a7acf0 0x55bd13ad8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa6fe00640 1 -- 172.21.15.32:0/907218458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd13ac5470 msgr2=0x55bd13af0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:09.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa6fe00640 1 --2- 172.21.15.32:0/907218458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd13ac5470 0x55bd13af0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa6fe00640 1 -- 172.21.15.32:0/907218458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd1385a7c0 con 0x55bd13ac4b50 2024-04-17T09:55:09.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa6fe00640 1 --2- 172.21.15.32:0/907218458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac4b50 0x55bd13af0030 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7faa6005e310 tx=0x7faa600841a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89fc5ef6078dae31 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:09.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa64c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:55:09.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa64c00640 1 -- 172.21.15.32:0/907218458 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60078020 con 0x55bd13ac4b50 2024-04-17T09:55:09.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa64c00640 1 -- 172.21.15.32:0/907218458 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7faa60074af0 con 0x55bd13ac4b50 2024-04-17T09:55:09.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa81cf0000 1 -- 172.21.15.32:0/907218458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa70019070 con 0x55bd13ac4b50 2024-04-17T09:55:09.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa64c00640 1 -- 172.21.15.32:0/907218458 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faa6006e070 con 0x55bd13ac4b50 2024-04-17T09:55:09.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa81cf0000 10 client.5889 fetch_fsmap learned FSMap version 24 2024-04-17T09:55:09.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa81cf0000 1 -- 172.21.15.32:0/907218458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55bd1386d650 con 0x55bd13ac4b50 2024-04-17T09:55:09.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa64c00640 1 -- 172.21.15.32:0/907218458 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa70019070 con 0x55bd13ac4b50 2024-04-17T09:55:09.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa64c00640 1 -- 172.21.15.32:0/907218458 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) ==== 3495+0+0 (secure 0 0 0) 0x7faa60051a60 con 0x55bd13ac4b50 2024-04-17T09:55:09.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa81cf0000 10 client.5889 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T09:55:09.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa81cf0000 10 client.5889 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-17T09:55:09.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa81cf0000 20 client.5889 populate_metadata read hostname 'smithi032' 2024-04-17T09:55:09.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa81cf0000 1 --2- 172.21.15.32:0/907218458 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55bd13b33670 0x55bd13b33a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa81cf0000 4 client.5889 mds_command: new command op to 5742 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T09:55:09.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa81cf0000 1 -- 172.21.15.32:0/907218458 --> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] -- command(tid 0: {"prefix": "status"}) -- 0x55bd138d5ef0 con 0x55bd13b33670 2024-04-17T09:55:09.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa77400640 1 --2- 172.21.15.32:0/907218458 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55bd13b33670 0x55bd13b33a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa77400640 1 --2- 172.21.15.32:0/907218458 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55bd13b33670 0x55bd13b33a50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9e02250cc5bd835 server_cookie=9333319e3f32d5da in_seq=0 out_seq=0 2024-04-17T09:55:09.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa64c00640 10 client.5889 ms_handle_connect on v2:172.21.15.144:6833/2177384675 2024-04-17T09:55:09.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa64c00640 1 -- 172.21.15.32:0/907218458 <== mds.0 v2:172.21.15.144:6833/2177384675 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55bd138d5ef0 con 0x55bd13b33670 2024-04-17T09:55:09.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa64c00640 10 client.5889 handle_command_reply: tid=0 2024-04-17T09:55:09.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa81cf0000 1 client.5889 shutdown 2024-04-17T09:55:09.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa81cf0000 1 -- 172.21.15.32:0/907218458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac4b50 msgr2=0x55bd13af0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:09.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.781+0000 7faa81cf0000 1 --2- 172.21.15.32:0/907218458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac4b50 0x55bd13af0030 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7faa6005e310 tx=0x7faa600841a0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.785+0000 7faa81cf0000 1 -- 172.21.15.32:0/907218458 shutdown_connections 2024-04-17T09:55:09.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.785+0000 7faa81cf0000 1 --2- 172.21.15.32:0/907218458 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55bd13b33670 0x55bd13b33a50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.785+0000 7faa81cf0000 1 --2- 172.21.15.32:0/907218458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd13ac5470 0x55bd13af0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.785+0000 7faa81cf0000 1 --2- 172.21.15.32:0/907218458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55bd13ac4b50 0x55bd13af0030 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.785+0000 7faa81cf0000 1 --2- 172.21.15.32:0/907218458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55bd13a7acf0 0x55bd13ad8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.785+0000 7faa64c00640 0 client.5889 ms_handle_reset on v2:172.21.15.144:6833/2177384675 2024-04-17T09:55:09.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.785+0000 7faa81cf0000 1 -- 172.21.15.32:0/907218458 >> 172.21.15.32:0/907218458 conn(0x55bd13a80470 msgr2=0x55bd13a7c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:09.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.785+0000 7faa81cf0000 1 -- 172.21.15.32:0/907218458 shutdown_connections 2024-04-17T09:55:09.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.785+0000 7faa81cf0000 1 -- 172.21.15.32:0/907218458 wait complete. 2024-04-17T09:55:09.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.785+0000 7faa81cf0000 20 client.5889 trim_cache size 0 max 16384 2024-04-17T09:55:09.788 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:55:09.788 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "ff178df8-4c06-4367-a1e9-2e3a66d870d0", 2024-04-17T09:55:09.788 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-17T09:55:09.789 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 5742, 2024-04-17T09:55:09.789 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-17T09:55:09.789 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-17T09:55:09.789 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-17T09:55:09.789 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.9484715709999998, 2024-04-17T09:55:09.789 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 24, 2024-04-17T09:55:09.789 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 46, 2024-04-17T09:55:09.789 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 46, 2024-04-17T09:55:09.789 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 10.751710906 2024-04-17T09:55:09.789 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:55:09.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.785+0000 7faa7e800640 1 -- 172.21.15.32:0/1784582708 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faa54068380 msgr2=0x7faa5406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:09.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.785+0000 7faa7e800640 1 --2- 172.21.15.32:0/1784582708 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faa54068380 0x7faa5406a840 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7faa680ad000 tx=0x7faa680a6000 comp rx=0 tx=0).stop 2024-04-17T09:55:09.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.785+0000 7faa7e800640 1 -- 172.21.15.32:0/1784582708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa78155a70 msgr2=0x7faa781f8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:09.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.785+0000 7faa7e800640 1 --2- 172.21.15.32:0/1784582708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa78155a70 0x7faa781f8300 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7faa60000fe0 tx=0x7faa6000cbe0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.785+0000 7faa7e800640 1 -- 172.21.15.32:0/1784582708 shutdown_connections 2024-04-17T09:55:09.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.785+0000 7faa7e800640 1 --2- 172.21.15.32:0/1784582708 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faa54068380 0x7faa5406a840 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.785+0000 7faa7e800640 1 --2- 172.21.15.32:0/1784582708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa78155e50 0x7faa78149e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.785+0000 7faa7e800640 1 --2- 172.21.15.32:0/1784582708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa78155a70 0x7faa781f8300 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.785+0000 7faa7e800640 1 --2- 172.21.15.32:0/1784582708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa780b3340 0x7faa78147c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.785+0000 7faa7e800640 1 -- 172.21.15.32:0/1784582708 >> 172.21.15.32:0/1784582708 conn(0x7faa7806ba30 msgr2=0x7faa7815ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:09.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.785+0000 7faa7e800640 1 -- 172.21.15.32:0/1784582708 shutdown_connections 2024-04-17T09:55:09.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.785+0000 7faa7e800640 1 -- 172.21.15.32:0/1784582708 wait complete. 2024-04-17T09:55:09.804 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T09:55:09.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.945+0000 7f855aa00640 1 Processor -- start 2024-04-17T09:55:09.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.945+0000 7f855aa00640 1 -- start start 2024-04-17T09:55:09.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f855aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85540b3980 0x7f85540cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f855aa00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85540b0d00 con 0x7f855415c760 2024-04-17T09:55:09.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f855aa00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85540b0fe0 con 0x7f85540c7ce0 2024-04-17T09:55:09.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f855aa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f85540b1bf0 con 0x7f85540b3980 2024-04-17T09:55:09.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f8559600640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f855415c760 0x7f85540c75d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33194/0 (socket says 172.21.15.32:33194) 2024-04-17T09:55:09.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f8559600640 1 -- 172.21.15.32:0/2684167388 learned_addr learned my addr 172.21.15.32:0/2684167388 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:09.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f8558c00640 1 --2- 172.21.15.32:0/2684167388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85540b3980 0x7f85540cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f8553e00640 1 -- 172.21.15.32:0/2684167388 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1574905813 0 0) 0x7f85540b0d00 con 0x7f855415c760 2024-04-17T09:55:09.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f8553e00640 1 -- 172.21.15.32:0/2684167388 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f853c003530 con 0x7f855415c760 2024-04-17T09:55:09.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f8553e00640 1 -- 172.21.15.32:0/2684167388 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 330310098 0 0) 0x7f85540b0fe0 con 0x7f85540c7ce0 2024-04-17T09:55:09.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f8553e00640 1 -- 172.21.15.32:0/2684167388 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85540b0d00 con 0x7f85540c7ce0 2024-04-17T09:55:09.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f8553e00640 1 -- 172.21.15.32:0/2684167388 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3607788675 0 0) 0x7f853c003530 con 0x7f855415c760 2024-04-17T09:55:09.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f8553e00640 1 -- 172.21.15.32:0/2684167388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85540b3980 msgr2=0x7f85540cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:09.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f8553e00640 1 --2- 172.21.15.32:0/2684167388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85540b3980 0x7f85540cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f8553e00640 1 -- 172.21.15.32:0/2684167388 >> v1:172.21.15.144:6789/0 conn(0x7f85540c7ce0 legacy=0x7f85540cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:09.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f8553e00640 1 -- 172.21.15.32:0/2684167388 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85540b12c0 con 0x7f855415c760 2024-04-17T09:55:09.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f8553e00640 1 -- 172.21.15.32:0/2684167388 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f8548001680 con 0x7f855415c760 2024-04-17T09:55:09.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f8553e00640 1 -- 172.21.15.32:0/2684167388 >> v1:172.21.15.32:6789/0 conn(0x7f855415c760 legacy=0x7f85540c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:09.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f8553e00640 1 --2- 172.21.15.32:0/2684167388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f853c003b80 0x7f853c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f8553e00640 1 --2- 172.21.15.32:0/2684167388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85540b3980 0x7f853c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f8553e00640 1 --2- 172.21.15.32:0/2684167388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f853c004d50 0x7f853c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f8553e00640 1 -- 172.21.15.32:0/2684167388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f85540b1bf0 con 0x7f853c004d50 2024-04-17T09:55:09.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f8553e00640 1 -- 172.21.15.32:0/2684167388 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f853c00bb90 con 0x7f853c003b80 2024-04-17T09:55:09.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f8553e00640 1 -- 172.21.15.32:0/2684167388 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f853c00bd90 con 0x7f85540b3980 2024-04-17T09:55:09.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f855a000640 1 --2- 172.21.15.32:0/2684167388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f853c004d50 0x7f853c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f8559600640 1 --2- 172.21.15.32:0/2684167388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f853c003b80 0x7f853c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f8558c00640 1 --2- 172.21.15.32:0/2684167388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85540b3980 0x7f853c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f855a000640 1 -- 172.21.15.32:0/2684167388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85540b3980 msgr2=0x7f853c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:09.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f855a000640 1 --2- 172.21.15.32:0/2684167388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85540b3980 0x7f853c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f855a000640 1 -- 172.21.15.32:0/2684167388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f853c003b80 msgr2=0x7f853c003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:09.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f855a000640 1 --2- 172.21.15.32:0/2684167388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f853c003b80 0x7f853c003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f855a000640 1 -- 172.21.15.32:0/2684167388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853c003530 con 0x7f853c004d50 2024-04-17T09:55:09.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.949+0000 7f8558c00640 1 --2- 172.21.15.32:0/2684167388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85540b3980 0x7f853c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.953+0000 7f855a000640 1 --2- 172.21.15.32:0/2684167388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f853c004d50 0x7f853c0051d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f854c001fd0 tx=0x7f854c01c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b35d386290ac0c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.953+0000 7f8553e00640 1 -- 172.21.15.32:0/2684167388 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854c001c20 con 0x7f853c004d50 2024-04-17T09:55:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.953+0000 7f8553e00640 1 -- 172.21.15.32:0/2684167388 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f854c003ba0 con 0x7f853c004d50 2024-04-17T09:55:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.953+0000 7f8553e00640 1 -- 172.21.15.32:0/2684167388 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854c0050d0 con 0x7f853c004d50 2024-04-17T09:55:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.953+0000 7f855aa00640 1 -- 172.21.15.32:0/2684167388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f853c004d50 msgr2=0x7f853c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.953+0000 7f855aa00640 1 --2- 172.21.15.32:0/2684167388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f853c004d50 0x7f853c0051d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f854c001fd0 tx=0x7f854c01c8d0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.953+0000 7f855a000640 1 -- 172.21.15.32:0/2684167388 reap_dead start 2024-04-17T09:55:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.953+0000 7f855aa00640 1 -- 172.21.15.32:0/2684167388 shutdown_connections 2024-04-17T09:55:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.953+0000 7f855aa00640 1 -- 172.21.15.32:0/2684167388 >> 172.21.15.32:0/2684167388 conn(0x7f85540b2e90 msgr2=0x7f85540b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.953+0000 7f855aa00640 1 -- 172.21.15.32:0/2684167388 shutdown_connections 2024-04-17T09:55:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.953+0000 7f855aa00640 1 -- 172.21.15.32:0/2684167388 wait complete. 2024-04-17T09:55:09.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f855aa00640 1 Processor -- start 2024-04-17T09:55:09.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f855aa00640 1 -- start start 2024-04-17T09:55:09.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f855aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f853c004d50 0x7f85540b0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f855aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8554159680 0x7f8554145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f855aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8554145c90 0x7f85540d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f855aa00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f85540cb4d0 con 0x7f853c004d50 2024-04-17T09:55:09.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f855aa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f85540b2bf0 con 0x7f8554145c90 2024-04-17T09:55:09.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f855aa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f85540cb650 con 0x7f8554159680 2024-04-17T09:55:09.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f8559600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f853c004d50 0x7f85540b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f8559600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f853c004d50 0x7f85540b0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54128/0 (socket says 172.21.15.32:54128) 2024-04-17T09:55:09.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f8559600640 1 -- 172.21.15.32:0/1125131365 learned_addr learned my addr 172.21.15.32:0/1125131365 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:09.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f855a000640 1 --2- 172.21.15.32:0/1125131365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8554145c90 0x7f85540d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f8558c00640 1 --2- 172.21.15.32:0/1125131365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8554159680 0x7f8554145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f8559600640 1 -- 172.21.15.32:0/1125131365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8554159680 msgr2=0x7f8554145750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:09.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f8559600640 1 --2- 172.21.15.32:0/1125131365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8554159680 0x7f8554145750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f8559600640 1 -- 172.21.15.32:0/1125131365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8554145c90 msgr2=0x7f85540d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f8559600640 1 --2- 172.21.15.32:0/1125131365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8554145c90 0x7f85540d15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f8559600640 1 -- 172.21.15.32:0/1125131365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85540b0a90 con 0x7f853c004d50 2024-04-17T09:55:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f855a000640 1 --2- 172.21.15.32:0/1125131365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8554145c90 0x7f85540d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f8559600640 1 --2- 172.21.15.32:0/1125131365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f853c004d50 0x7f85540b0550 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8548004e80 tx=0x7f8548007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f8551600640 1 -- 172.21.15.32:0/1125131365 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854800f750 con 0x7f853c004d50 2024-04-17T09:55:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f8551600640 1 -- 172.21.15.32:0/1125131365 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8548018070 con 0x7f853c004d50 2024-04-17T09:55:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f8551600640 1 -- 172.21.15.32:0/1125131365 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854800fb50 con 0x7f853c004d50 2024-04-17T09:55:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f855aa00640 1 -- 172.21.15.32:0/1125131365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8554146210 con 0x7f853c004d50 2024-04-17T09:55:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f855aa00640 1 -- 172.21.15.32:0/1125131365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85541574b0 con 0x7f853c004d50 2024-04-17T09:55:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.957+0000 7f855aa00640 1 -- 172.21.15.32:0/1125131365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85541572a0 con 0x7f853c004d50 2024-04-17T09:55:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.961+0000 7f8551600640 1 -- 172.21.15.32:0/1125131365 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f854800fcf0 con 0x7f853c004d50 2024-04-17T09:55:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.961+0000 7f8551600640 1 --2- 172.21.15.32:0/1125131365 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8534068380 0x7f853406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:09.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.961+0000 7f8551600640 1 -- 172.21.15.32:0/1125131365 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f8548086930 con 0x7f853c004d50 2024-04-17T09:55:09.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.961+0000 7f8558c00640 1 --2- 172.21.15.32:0/1125131365 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8534068380 0x7f853406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:09.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.961+0000 7f8558c00640 1 --2- 172.21.15.32:0/1125131365 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8534068380 0x7f853406a840 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f855415bc30 tx=0x7f8540008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:09.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:09.965+0000 7f8551600640 1 -- 172.21.15.32:0/1125131365 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f8548056800 con 0x7f853c004d50 2024-04-17T09:55:10.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:10.169+0000 7f855aa00640 1 -- 172.21.15.32:0/1125131365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f855404fd00 con 0x7f853c004d50 2024-04-17T09:55:10.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:10.169+0000 7f8551600640 1 -- 172.21.15.32:0/1125131365 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f854805ab00 con 0x7f853c004d50 2024-04-17T09:55:10.172 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T09:55:10.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:10.173+0000 7f855aa00640 1 -- 172.21.15.32:0/1125131365 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8534068380 msgr2=0x7f853406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:10.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:10.173+0000 7f855aa00640 1 --2- 172.21.15.32:0/1125131365 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8534068380 0x7f853406a840 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f855415bc30 tx=0x7f8540008040 comp rx=0 tx=0).stop 2024-04-17T09:55:10.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:10.173+0000 7f855aa00640 1 -- 172.21.15.32:0/1125131365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f853c004d50 msgr2=0x7f85540b0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:10.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:10.173+0000 7f855aa00640 1 --2- 172.21.15.32:0/1125131365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f853c004d50 0x7f85540b0550 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8548004e80 tx=0x7f8548007430 comp rx=0 tx=0).stop 2024-04-17T09:55:10.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:10.173+0000 7f855aa00640 1 -- 172.21.15.32:0/1125131365 shutdown_connections 2024-04-17T09:55:10.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:10.173+0000 7f855aa00640 1 --2- 172.21.15.32:0/1125131365 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8534068380 0x7f853406a840 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:10.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:10.173+0000 7f855aa00640 1 --2- 172.21.15.32:0/1125131365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8554159680 0x7f8554145750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:10.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:10.173+0000 7f855aa00640 1 --2- 172.21.15.32:0/1125131365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8554145c90 0x7f85540d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:10.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:10.173+0000 7f855aa00640 1 --2- 172.21.15.32:0/1125131365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f853c004d50 0x7f85540b0550 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:10.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:10.173+0000 7f855aa00640 1 -- 172.21.15.32:0/1125131365 >> 172.21.15.32:0/1125131365 conn(0x7f85540b2e90 msgr2=0x7f8554158e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:10.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:10.173+0000 7f855aa00640 1 -- 172.21.15.32:0/1125131365 shutdown_connections 2024-04-17T09:55:10.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:10.173+0000 7f855aa00640 1 -- 172.21.15.32:0/1125131365 wait complete. 2024-04-17T09:55:10.193 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T09:55:10.193 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T09:55:10.194 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-17T09:55:10.194 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T09:55:10.194 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T09:55:10.194 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T09:55:10.194 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T09:55:10.194 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T09:55:10.194 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:55:10.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:55:10.194 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-17T09:55:10.203 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T09:55:10.203 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T09:55:10.203 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T09:55:10.203 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:55:10.203 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-17T09:55:10.203 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:55:10.203 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:55:10.203 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-17T09:55:10.203 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-17T09:55:10.204 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T09:55:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:55:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-17T09:55:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-17T09:55:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42344sec preferred_lft 42344sec 2024-04-17T09:55:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2024-04-17T09:55:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:55:10.204 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:55:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-17T09:55:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-17T09:55:10.204 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:55:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:55:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-17T09:55:10.204 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T09:55:10.204 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:cd:f4:aa:5b brd ff:ff:ff:ff:ff:ff 2024-04-17T09:55:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T09:55:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:55:10.205 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T09:55:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T09:55:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T09:55:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:55:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T09:55:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:55:10.205 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T09:55:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:55:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T09:55:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:55:10.205 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T09:55:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:55:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T09:55:10.205 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:55:10.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:55:10.206 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T09:55:10.270 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-17T09:55:10.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:55:10.272 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-17T09:55:10.329 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-17T09:55:10.329 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T09:55:10.329 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-17T09:55:10.330 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T09:55:10.330 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-17T09:55:10.330 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T09:55:10.330 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:55:10.330 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T09:55:10.330 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-17T09:55:10.330 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-17T09:55:10.330 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-17T09:55:10.330 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T09:55:10.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:55:10.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:55:10.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:55:10.468 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-17T09:55:10.477 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T09:55:10.477 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T09:55:10.478 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:55:10.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:55:10.478 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-17T09:55:10.539 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T09:55:10.540 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:55:10.540 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T09:55:10.587 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:55:10.588 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:55:10.588 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T09:55:10.639 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T09:55:10.700 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:55:10.757 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T09:55:10.757 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T09:55:10.757 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T09:55:10.757 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T09:55:10.757 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T09:55:10.757 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T09:55:10.757 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T09:55:10.758 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T09:55:10.758 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T09:55:10.758 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T09:55:10.758 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T09:55:10.758 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T09:55:10.758 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:55:10.758 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T09:55:10.758 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T09:55:10.758 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T09:55:10.758 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T09:55:10.758 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T09:55:10.758 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T09:55:10.759 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T09:55:10.759 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:55:10.759 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T09:55:10.759 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T09:55:10.759 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T09:55:10.759 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T09:55:10.759 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T09:55:10.759 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T09:55:10.759 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T09:55:10.759 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T09:55:10.759 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:55:10.759 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:55:10.760 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:55:10.760 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:55:10.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:55:10.761 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T09:55:10.815 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T09:55:10.815 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T09:55:10.863 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T09:55:10.884 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:55:10.937 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T09:55:10.937 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T09:55:10.937 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T09:55:10.937 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T09:55:10.937 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T09:55:10.937 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T09:55:10.938 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T09:55:10.938 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T09:55:10.938 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T09:55:10.938 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T09:55:10.938 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T09:55:10.938 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T09:55:10.938 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:55:10.938 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T09:55:10.938 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T09:55:10.938 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T09:55:10.938 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T09:55:10.938 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T09:55:10.938 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T09:55:10.939 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T09:55:10.939 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:55:10.939 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T09:55:10.939 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T09:55:10.939 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T09:55:10.939 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T09:55:10.939 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T09:55:10.939 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T09:55:10.939 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T09:55:10.939 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T09:55:10.939 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:55:10.939 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:55:10.939 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:55:10.940 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:55:10.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:55:10.941 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T09:55:10.958 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2024-04-17T09:55:10.957+0000 7f0598255580 -1 init, newargv = 0x561fc67d0cd0 newargc=15 2024-04-17T09:55:10.958 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[38152]: starting ceph client 2024-04-17T09:55:10.967 INFO:teuthology.orchestra.run.smithi032.stdout:44 2024-04-17T09:55:10.967 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[38152]: starting fuse 2024-04-17T09:55:10.968 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-17T09:55:10.968 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2024-04-17T09:55:10.968 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2024-04-17T09:55:10.968 DEBUG:teuthology.orchestra.run.smithi032:> import re 2024-04-17T09:55:10.968 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T09:55:10.969 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2024-04-17T09:55:10.969 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:55:10.969 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2024-04-17T09:55:10.969 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T09:55:10.969 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2024-04-17T09:55:10.969 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T09:55:10.969 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:55:10.969 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2024-04-17T09:55:10.969 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2024-04-17T09:55:10.969 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2024-04-17T09:55:10.969 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2024-04-17T09:55:10.969 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:55:10.969 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2024-04-17T09:55:10.969 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T09:55:10.969 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T09:55:10.970 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T09:55:10.970 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2024-04-17T09:55:10.970 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2024-04-17T09:55:10.970 DEBUG:teuthology.orchestra.run.smithi032:> return f 2024-04-17T09:55:10.970 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T09:55:10.970 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:55:10.970 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2024-04-17T09:55:10.970 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T09:55:11.094 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.38152.asok 2024-04-17T09:55:11.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:55:11.106 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38152.asok 2024-04-17T09:55:11.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:55:11.106 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38152.asok status 2024-04-17T09:55:11.286 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:55:11.286 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2024-04-17T09:55:11.286 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-17T09:55:11.286 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T09:55:11.286 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2024-04-17T09:55:11.286 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2024-04-17T09:55:11.286 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T09:55:11.286 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "38152", 2024-04-17T09:55:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2024-04-17T09:55:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2024-04-17T09:55:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2024-04-17T09:55:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 5904, 2024-04-17T09:55:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2024-04-17T09:55:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2024-04-17T09:55:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2024-04-17T09:55:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 5904 2024-04-17T09:55:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T09:55:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T09:55:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2024-04-17T09:55:11.287 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3557711196 2024-04-17T09:55:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T09:55:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T09:55:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2024-04-17T09:55:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3557711196 2024-04-17T09:55:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.5904 172.21.15.32:0/3557711196", 2024-04-17T09:55:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "172.21.15.32:0/3557711196", 2024-04-17T09:55:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2024-04-17T09:55:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 24, 2024-04-17T09:55:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 46, 2024-04-17T09:55:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2024-04-17T09:55:11.288 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2024-04-17T09:55:11.289 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs" 2024-04-17T09:55:11.289 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:55:11.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:55:11.299 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2024-04-17T09:55:11.299 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T09:55:11.308 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2024-04-17T09:55:11.309 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T09:55:11.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:55:11.309 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T09:55:11.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:55:11.372 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-04-17T09:55:11.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:55:11.438 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-04-17T09:55:11.496 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T09:55:11.496 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T09:55:11.496 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T09:55:11.496 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:55:11.496 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-04-17T09:55:11.497 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T09:55:11.606 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627777 2024-04-17T09:55:11.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:55:11.615 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:55:11.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.797+0000 7f1406c00640 1 Processor -- start 2024-04-17T09:55:11.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.797+0000 7f1406c00640 1 -- start start 2024-04-17T09:55:11.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.797+0000 7f1406c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f14000b2b80 0x7f14000d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:11.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.797+0000 7f1406c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14000b3120 0x7f140006d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:11.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.797+0000 7f1406c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f140009b650 con 0x7f14000b3670 2024-04-17T09:55:11.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.797+0000 7f1406c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f14000b2f60 con 0x7f14000b2b80 2024-04-17T09:55:11.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.797+0000 7f1406c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f140009b400 con 0x7f14000b3120 2024-04-17T09:55:11.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.797+0000 7f13ff400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14000b3120 0x7f140006d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.797+0000 7f13ff400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14000b3120 0x7f140006d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:58862/0 (socket says 172.21.15.32:58862) 2024-04-17T09:55:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.797+0000 7f13ff400640 1 -- 172.21.15.32:0/3294180905 learned_addr learned my addr 172.21.15.32:0/3294180905 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.797+0000 7f13ffe00640 1 --2- 172.21.15.32:0/3294180905 >> v2:172.21.15.144:3300/0 conn(0x7f14000b2b80 0x7f14000d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.797+0000 7f13fea00640 1 -- 172.21.15.32:0/3294180905 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3053230878 0 0) 0x7f140009b650 con 0x7f14000b3670 2024-04-17T09:55:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.797+0000 7f13ff400640 1 -- 172.21.15.32:0/3294180905 >> v2:172.21.15.144:3300/0 conn(0x7f14000b2b80 msgr2=0x7f14000d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.797+0000 7f13ff400640 1 --2- 172.21.15.32:0/3294180905 >> v2:172.21.15.144:3300/0 conn(0x7f14000b2b80 0x7f14000d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.797+0000 7f13ff400640 1 -- 172.21.15.32:0/3294180905 >> v1:172.21.15.144:6789/0 conn(0x7f14000b3670 legacy=0x7f140015f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.797+0000 7f13ff400640 1 -- 172.21.15.32:0/3294180905 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f140006cd50 con 0x7f14000b3120 2024-04-17T09:55:11.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.801+0000 7f13ff400640 1 --2- 172.21.15.32:0/3294180905 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14000b3120 0x7f140006d550 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009870 tx=0x7f13ec01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca9b58518f1c9c5f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:11.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.801+0000 7f13fea00640 1 -- 172.21.15.32:0/3294180905 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec0b0070 con 0x7f14000b3120 2024-04-17T09:55:11.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.801+0000 7f13fea00640 1 -- 172.21.15.32:0/3294180905 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13ec0ab600 con 0x7f14000b3120 2024-04-17T09:55:11.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.801+0000 7f13fea00640 1 -- 172.21.15.32:0/3294180905 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec0ab9d0 con 0x7f14000b3120 2024-04-17T09:55:11.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.801+0000 7f1406c00640 1 -- 172.21.15.32:0/3294180905 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14000b3120 msgr2=0x7f140006d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:11.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.801+0000 7f1406c00640 1 --2- 172.21.15.32:0/3294180905 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14000b3120 0x7f140006d550 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009870 tx=0x7f13ec01c4d0 comp rx=0 tx=0).stop 2024-04-17T09:55:11.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.801+0000 7f1406c00640 1 -- 172.21.15.32:0/3294180905 shutdown_connections 2024-04-17T09:55:11.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.801+0000 7f1406c00640 1 --2- 172.21.15.32:0/3294180905 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14000b3120 0x7f140006d550 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:11.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.801+0000 7f1406c00640 1 --2- 172.21.15.32:0/3294180905 >> v2:172.21.15.144:3300/0 conn(0x7f14000b2b80 0x7f14000d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:11.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.801+0000 7f1406c00640 1 -- 172.21.15.32:0/3294180905 >> 172.21.15.32:0/3294180905 conn(0x7f14000b4010 msgr2=0x7f14000b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:11.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.801+0000 7f1406c00640 1 -- 172.21.15.32:0/3294180905 shutdown_connections 2024-04-17T09:55:11.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.801+0000 7f1406c00640 1 -- 172.21.15.32:0/3294180905 wait complete. 2024-04-17T09:55:11.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f1406c00640 1 Processor -- start 2024-04-17T09:55:11.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f1406c00640 1 -- start start 2024-04-17T09:55:11.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f1406c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14000b2b80 0x7f1400159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:11.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f1406c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14000b3120 0x7f1400162340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:11.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f1406c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14000b3670 0x7f1400151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:11.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f1406c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f14000b3db0 con 0x7f14000b2b80 2024-04-17T09:55:11.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f1406c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f14000b3c30 con 0x7f14000b3670 2024-04-17T09:55:11.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f1406c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f140016c2d0 con 0x7f14000b3120 2024-04-17T09:55:11.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f13ffe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14000b2b80 0x7f1400159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:11.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f13ffe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14000b2b80 0x7f1400159240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38854/0 (socket says 172.21.15.32:38854) 2024-04-17T09:55:11.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f13ffe00640 1 -- 172.21.15.32:0/99661648 learned_addr learned my addr 172.21.15.32:0/99661648 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:11.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f13ff400640 1 --2- 172.21.15.32:0/99661648 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14000b3120 0x7f1400162340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:11.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f1404c00640 1 --2- 172.21.15.32:0/99661648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14000b3670 0x7f1400151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:11.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f13ffe00640 1 -- 172.21.15.32:0/99661648 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14000b3120 msgr2=0x7f1400162340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:11.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f13ffe00640 1 --2- 172.21.15.32:0/99661648 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14000b3120 0x7f1400162340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:11.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f13ffe00640 1 -- 172.21.15.32:0/99661648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14000b3670 msgr2=0x7f1400151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:11.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f13ffe00640 1 --2- 172.21.15.32:0/99661648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14000b3670 0x7f1400151a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:11.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f13ffe00640 1 -- 172.21.15.32:0/99661648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14000b03f0 con 0x7f14000b2b80 2024-04-17T09:55:11.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f13ffe00640 1 --2- 172.21.15.32:0/99661648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14000b2b80 0x7f1400159240 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f13f000e870 tx=0x7f13f000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:11.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f13f7e00640 1 -- 172.21.15.32:0/99661648 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f0017070 con 0x7f14000b2b80 2024-04-17T09:55:11.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f13f7e00640 1 -- 172.21.15.32:0/99661648 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f000c940 con 0x7f14000b2b80 2024-04-17T09:55:11.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f13f7e00640 1 -- 172.21.15.32:0/99661648 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f000ccf0 con 0x7f14000b2b80 2024-04-17T09:55:11.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f1406c00640 1 -- 172.21.15.32:0/99661648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1400162b90 con 0x7f14000b2b80 2024-04-17T09:55:11.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f1406c00640 1 -- 172.21.15.32:0/99661648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14000b0990 con 0x7f14000b2b80 2024-04-17T09:55:11.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.805+0000 7f1406c00640 1 -- 172.21.15.32:0/99661648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1400068a90 con 0x7f14000b2b80 2024-04-17T09:55:11.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.809+0000 7f13f7e00640 1 -- 172.21.15.32:0/99661648 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f13f001a070 con 0x7f14000b2b80 2024-04-17T09:55:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.809+0000 7f13f7e00640 1 --2- 172.21.15.32:0/99661648 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f13e00683b0 0x7f13e006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.809+0000 7f13f7e00640 1 -- 172.21.15.32:0/99661648 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f13f00867a0 con 0x7f14000b2b80 2024-04-17T09:55:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.809+0000 7f13ff400640 1 --2- 172.21.15.32:0/99661648 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f13e00683b0 0x7f13e006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:11.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.809+0000 7f13ff400640 1 --2- 172.21.15.32:0/99661648 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f13e00683b0 0x7f13e006a870 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0ad000 tx=0x7f13ec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:11.813+0000 7f13f7e00640 1 -- 172.21.15.32:0/99661648 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f13f0056700 con 0x7f14000b2b80 2024-04-17T09:55:12.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.017+0000 7f1406c00640 1 -- 172.21.15.32:0/99661648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14000b84a0 con 0x7f14000b2b80 2024-04-17T09:55:12.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.017+0000 7f13f7e00640 1 -- 172.21.15.32:0/99661648 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5202 (secure 0 0 0) 0x7f13f005aa00 con 0x7f14000b2b80 2024-04-17T09:55:12.018 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:12.018 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2542578926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2542578926},{"type":"v1","addr":"172.21.15.144:6834","nonce":2542578926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1424948520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1424948520},{"type":"v1","addr":"172.21.15.32:6836","nonce":1424948520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/393813154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":393813154},{"type":"v1","addr":"172.21.15.32:6837","nonce":393813154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:55:03.233552+0000","modified":"2024-04-17T09:55:06.243321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5742":{"gid":5742,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/2177384675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2177384675},{"type":"v1","addr":"172.21.15.144:6835","nonce":2177384675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742]},"id":3}]} 2024-04-17T09:55:12.019 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2024-04-17T09:55:12.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.017+0000 7f1406c00640 1 -- 172.21.15.32:0/99661648 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f13e00683b0 msgr2=0x7f13e006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.017+0000 7f1406c00640 1 --2- 172.21.15.32:0/99661648 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f13e00683b0 0x7f13e006a870 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0ad000 tx=0x7f13ec008040 comp rx=0 tx=0).stop 2024-04-17T09:55:12.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.017+0000 7f1406c00640 1 -- 172.21.15.32:0/99661648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14000b2b80 msgr2=0x7f1400159240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.017+0000 7f1406c00640 1 --2- 172.21.15.32:0/99661648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14000b2b80 0x7f1400159240 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f13f000e870 tx=0x7f13f000ec70 comp rx=0 tx=0).stop 2024-04-17T09:55:12.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.021+0000 7f1406c00640 1 -- 172.21.15.32:0/99661648 shutdown_connections 2024-04-17T09:55:12.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.021+0000 7f1406c00640 1 --2- 172.21.15.32:0/99661648 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f13e00683b0 0x7f13e006a870 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.021+0000 7f1406c00640 1 --2- 172.21.15.32:0/99661648 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14000b3120 0x7f1400162340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.021+0000 7f1406c00640 1 --2- 172.21.15.32:0/99661648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14000b3670 0x7f1400151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.021+0000 7f1406c00640 1 --2- 172.21.15.32:0/99661648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14000b2b80 0x7f1400159240 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.021+0000 7f1406c00640 1 -- 172.21.15.32:0/99661648 >> 172.21.15.32:0/99661648 conn(0x7f14000b4010 msgr2=0x7f14000d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:12.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.021+0000 7f1406c00640 1 -- 172.21.15.32:0/99661648 shutdown_connections 2024-04-17T09:55:12.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.021+0000 7f1406c00640 1 -- 172.21.15.32:0/99661648 wait complete. 2024-04-17T09:55:12.039 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-17T09:55:12.221 INFO:teuthology.orchestra.run.smithi144.stdout:{"message":"","return_code":0} 2024-04-17T09:55:12.221 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-17T09:55:12.222 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-17T09:55:12.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2857a48080 1 Processor -- start 2024-04-17T09:55:12.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2857a48080 1 -- start start 2024-04-17T09:55:12.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2857a48080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f0b02bc700 0x55f0b02baf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2857a48080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0b02bcae0 0x55f0b0230ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2857a48080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0b02bf6b0 0x55f0b0246d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2857a48080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f0b02d7ae0 con 0x55f0b02bf6b0 2024-04-17T09:55:12.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2857a48080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f0b01a7f90 con 0x55f0b02bcae0 2024-04-17T09:55:12.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2857a48080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f0b02d7c60 con 0x55f0b02bc700 2024-04-17T09:55:12.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2854a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0b02bf6b0 0x55f0b0246d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2853600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0b02bcae0 0x55f0b0230ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2854a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0b02bf6b0 0x55f0b0246d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38864/0 (socket says 172.21.15.32:38864) 2024-04-17T09:55:12.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2854a00640 1 -- 172.21.15.32:0/2883747983 learned_addr learned my addr 172.21.15.32:0/2883747983 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:12.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2854000640 1 --2- 172.21.15.32:0/2883747983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f0b02bc700 0x55f0b02baf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2854a00640 1 -- 172.21.15.32:0/2883747983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f0b02bc700 msgr2=0x55f0b02baf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:12.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2854a00640 1 --2- 172.21.15.32:0/2883747983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f0b02bc700 0x55f0b02baf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2854a00640 1 -- 172.21.15.32:0/2883747983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0b02bcae0 msgr2=0x55f0b0230ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:12.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2854a00640 1 --2- 172.21.15.32:0/2883747983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0b02bcae0 0x55f0b0230ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2854a00640 1 -- 172.21.15.32:0/2883747983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0b012c950 con 0x55f0b02bf6b0 2024-04-17T09:55:12.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2854a00640 1 --2- 172.21.15.32:0/2883747983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0b02bf6b0 0x55f0b0246d20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f284c0015a0 tx=0x7f284c001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb74e34d8020c18 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2852200640 1 -- 172.21.15.32:0/2883747983 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c016070 con 0x55f0b02bf6b0 2024-04-17T09:55:12.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2852200640 1 -- 172.21.15.32:0/2883747983 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f284c00f710 con 0x55f0b02bf6b0 2024-04-17T09:55:12.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2852200640 1 -- 172.21.15.32:0/2883747983 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c00fb00 con 0x55f0b02bf6b0 2024-04-17T09:55:12.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2857a48080 1 -- 172.21.15.32:0/2883747983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0b02bf6b0 msgr2=0x55f0b0246d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2857a48080 1 --2- 172.21.15.32:0/2883747983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0b02bf6b0 0x55f0b0246d20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f284c0015a0 tx=0x7f284c001a60 comp rx=0 tx=0).stop 2024-04-17T09:55:12.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2857a48080 1 -- 172.21.15.32:0/2883747983 shutdown_connections 2024-04-17T09:55:12.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2857a48080 1 --2- 172.21.15.32:0/2883747983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0b02bf6b0 0x55f0b0246d20 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2857a48080 1 --2- 172.21.15.32:0/2883747983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0b02bcae0 0x55f0b0230ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2857a48080 1 --2- 172.21.15.32:0/2883747983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f0b02bc700 0x55f0b02baf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2857a48080 1 -- 172.21.15.32:0/2883747983 >> 172.21.15.32:0/2883747983 conn(0x55f0b022d760 msgr2=0x55f0b0230130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:12.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2857a48080 1 -- 172.21.15.32:0/2883747983 shutdown_connections 2024-04-17T09:55:12.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.269+0000 7f2857a48080 1 -- 172.21.15.32:0/2883747983 wait complete. 2024-04-17T09:55:12.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2857a48080 1 Processor -- start 2024-04-17T09:55:12.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2857a48080 1 -- start start 2024-04-17T09:55:12.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2857a48080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f0b02bc700 0x55f0b0370ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2857a48080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0b02bcae0 0x55f0b02d9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2857a48080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0b02bf6b0 0x55f0b02ba010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2857a48080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f0b03692d0 con 0x55f0b02bf6b0 2024-04-17T09:55:12.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2857a48080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f0b02bb470 con 0x55f0b02bcae0 2024-04-17T09:55:12.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2857a48080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f0b0369570 con 0x55f0b02bc700 2024-04-17T09:55:12.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2854a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0b02bf6b0 0x55f0b02ba010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2854a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0b02bf6b0 0x55f0b02ba010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38876/0 (socket says 172.21.15.32:38876) 2024-04-17T09:55:12.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2853600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0b02bcae0 0x55f0b02d9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2854a00640 1 -- 172.21.15.32:0/1052078814 learned_addr learned my addr 172.21.15.32:0/1052078814 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:12.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2854000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f0b02bc700 0x55f0b0370ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2854a00640 1 -- 172.21.15.32:0/1052078814 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f0b02bc700 msgr2=0x55f0b0370ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2854a00640 1 --2- 172.21.15.32:0/1052078814 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f0b02bc700 0x55f0b0370ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2854a00640 1 -- 172.21.15.32:0/1052078814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0b02bcae0 msgr2=0x55f0b02d9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2854a00640 1 --2- 172.21.15.32:0/1052078814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0b02bcae0 0x55f0b02d9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2854a00640 1 -- 172.21.15.32:0/1052078814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0b01ffcb0 con 0x55f0b02bf6b0 2024-04-17T09:55:12.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2853600640 1 --2- 172.21.15.32:0/1052078814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f0b02bcae0 0x55f0b02d9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:55:12.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2854a00640 1 --2- 172.21.15.32:0/1052078814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0b02bf6b0 0x55f0b02ba010 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f284c001f60 tx=0x7f284c000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2850e00640 1 -- 172.21.15.32:0/1052078814 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c021020 con 0x55f0b02bf6b0 2024-04-17T09:55:12.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2850e00640 1 -- 172.21.15.32:0/1052078814 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f284c00f510 con 0x55f0b02bf6b0 2024-04-17T09:55:12.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2850e00640 1 -- 172.21.15.32:0/1052078814 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c016030 con 0x55f0b02bf6b0 2024-04-17T09:55:12.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2857a48080 1 -- 172.21.15.32:0/1052078814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f0b01fff50 con 0x55f0b02bf6b0 2024-04-17T09:55:12.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.273+0000 7f2857a48080 1 -- 172.21.15.32:0/1052078814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0b0200490 con 0x55f0b02bf6b0 2024-04-17T09:55:12.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.277+0000 7f2850e00640 1 -- 172.21.15.32:0/1052078814 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f284c024020 con 0x55f0b02bf6b0 2024-04-17T09:55:12.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.277+0000 7f2850e00640 1 --2- 172.21.15.32:0/1052078814 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2834066d90 0x7f2834069250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.277+0000 7f2850e00640 1 -- 172.21.15.32:0/1052078814 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f284c086a80 con 0x55f0b02bf6b0 2024-04-17T09:55:12.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.277+0000 7f2854000640 1 --2- 172.21.15.32:0/1052078814 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2834066d90 0x7f2834069250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.277+0000 7f2857a48080 1 --2- 172.21.15.32:0/1052078814 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55f0b02d9580 0x55f0b02010e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.277+0000 7f2857a48080 1 -- 172.21.15.32:0/1052078814 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46) -- 0x55f0b0201620 con 0x55f0b02d9580 2024-04-17T09:55:12.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.277+0000 7f2853600640 1 --2- 172.21.15.32:0/1052078814 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55f0b02d9580 0x55f0b02010e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.277+0000 7f2854000640 1 --2- 172.21.15.32:0/1052078814 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2834066d90 0x7f2834069250 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2844005410 tx=0x7f284400c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.277+0000 7f2853600640 1 --2- 172.21.15.32:0/1052078814 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55f0b02d9580 0x55f0b02010e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.277+0000 7f2853600640 1 -- 172.21.15.32:0/1052078814 <== osd.6 v2:172.21.15.144:6808/3198738696 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f28400af8f0 con 0x55f0b02d9580 2024-04-17T09:55:12.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.277+0000 7f2857a48080 1 -- 172.21.15.32:0/1052078814 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55f0b02d9580 msgr2=0x55f0b02010e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.277+0000 7f2857a48080 1 --2- 172.21.15.32:0/1052078814 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55f0b02d9580 0x55f0b02010e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.277+0000 7f2857a48080 1 -- 172.21.15.32:0/1052078814 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2834066d90 msgr2=0x7f2834069250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.277+0000 7f2857a48080 1 --2- 172.21.15.32:0/1052078814 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2834066d90 0x7f2834069250 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2844005410 tx=0x7f284400c040 comp rx=0 tx=0).stop 2024-04-17T09:55:12.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.277+0000 7f2857a48080 1 -- 172.21.15.32:0/1052078814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0b02bf6b0 msgr2=0x55f0b02ba010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.277+0000 7f2857a48080 1 --2- 172.21.15.32:0/1052078814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f0b02bf6b0 0x55f0b02ba010 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f284c001f60 tx=0x7f284c000f50 comp rx=0 tx=0).stop 2024-04-17T09:55:12.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.277+0000 7f2854a00640 1 -- 172.21.15.32:0/1052078814 reap_dead start 2024-04-17T09:55:12.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.277+0000 7f2857a48080 1 -- 172.21.15.32:0/1052078814 shutdown_connections 2024-04-17T09:55:12.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.277+0000 7f2857a48080 1 -- 172.21.15.32:0/1052078814 >> 172.21.15.32:0/1052078814 conn(0x55f0b022d760 msgr2=0x55f0b02bfb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:12.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.277+0000 7f2857a48080 1 -- 172.21.15.32:0/1052078814 shutdown_connections 2024-04-17T09:55:12.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.277+0000 7f2857a48080 1 -- 172.21.15.32:0/1052078814 wait complete. 2024-04-17T09:55:12.284 INFO:teuthology.orchestra.run.smithi032.stdout:Yalphaparent_a 2024-04-17T09:55:12.284 DEBUG:teuthology.orchestra.run.smithi032:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-17T09:55:12.351 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:55:12.351 INFO:teuthology.orchestra.run.smithi032.stdout: "ino": 1099511627777, 2024-04-17T09:55:12.351 INFO:teuthology.orchestra.run.smithi032.stdout: "ancestors": [ 2024-04-17T09:55:12.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:12.352 INFO:teuthology.orchestra.run.smithi032.stdout: "dirino": 1099511627776, 2024-04-17T09:55:12.352 INFO:teuthology.orchestra.run.smithi032.stdout: "dname": "alpha", 2024-04-17T09:55:12.352 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 2 2024-04-17T09:55:12.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:12.352 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:12.352 INFO:teuthology.orchestra.run.smithi032.stdout: "dirino": 1, 2024-04-17T09:55:12.352 INFO:teuthology.orchestra.run.smithi032.stdout: "dname": "parent_a", 2024-04-17T09:55:12.352 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 4 2024-04-17T09:55:12.352 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:12.352 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-17T09:55:12.352 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 7, 2024-04-17T09:55:12.352 INFO:teuthology.orchestra.run.smithi032.stdout: "old_pools": [] 2024-04-17T09:55:12.352 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:55:12.352 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:12.356 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-04-17T09:55:12.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.445+0000 7f800ff66080 1 Processor -- start 2024-04-17T09:55:12.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.445+0000 7f800ff66080 1 -- start start 2024-04-17T09:55:12.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.445+0000 7f800ff66080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c17d5c860 0x557c17d44a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.445+0000 7f800ff66080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c17deb680 0x557c17de4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.445+0000 7f800ff66080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c17deba60 0x557c17de8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.445+0000 7f800ff66080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557c17d15c60 con 0x557c17deba60 2024-04-17T09:55:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.445+0000 7f800ff66080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557c17cb9f90 con 0x557c17deb680 2024-04-17T09:55:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.445+0000 7f800ff66080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x557c17d15de0 con 0x557c17d5c860 2024-04-17T09:55:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.445+0000 7f800dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c17deba60 0x557c17de8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.445+0000 7f800dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c17deba60 0x557c17de8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38888/0 (socket says 172.21.15.32:38888) 2024-04-17T09:55:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.445+0000 7f800dc00640 1 -- 172.21.15.32:0/3089781927 learned_addr learned my addr 172.21.15.32:0/3089781927 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.445+0000 7f800d200640 1 --2- 172.21.15.32:0/3089781927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c17d5c860 0x557c17d44a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.445+0000 7f8007e00640 1 --2- 172.21.15.32:0/3089781927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c17deb680 0x557c17de4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.445+0000 7f800dc00640 1 -- 172.21.15.32:0/3089781927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c17d5c860 msgr2=0x557c17d44a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.445+0000 7f800dc00640 1 --2- 172.21.15.32:0/3089781927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c17d5c860 0x557c17d44a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.445+0000 7f800dc00640 1 -- 172.21.15.32:0/3089781927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c17deb680 msgr2=0x557c17de4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.445+0000 7f800dc00640 1 --2- 172.21.15.32:0/3089781927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c17deb680 0x557c17de4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.445+0000 7f800dc00640 1 -- 172.21.15.32:0/3089781927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c17c3e950 con 0x557c17deba60 2024-04-17T09:55:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f800d200640 1 --2- 172.21.15.32:0/3089781927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c17d5c860 0x557c17d44a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:55:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f8007e00640 1 --2- 172.21.15.32:0/3089781927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c17deb680 0x557c17de4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:55:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f800dc00640 1 --2- 172.21.15.32:0/3089781927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c17deba60 0x557c17de8120 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f800800c4d0 tx=0x7f800800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdb6acad51631b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f8006a00640 1 -- 172.21.15.32:0/3089781927 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80080116b0 con 0x557c17deba60 2024-04-17T09:55:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f8006a00640 1 -- 172.21.15.32:0/3089781927 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8008017070 con 0x557c17deba60 2024-04-17T09:55:12.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f8006a00640 1 -- 172.21.15.32:0/3089781927 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8008011ab0 con 0x557c17deba60 2024-04-17T09:55:12.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f800ff66080 1 -- 172.21.15.32:0/3089781927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c17deba60 msgr2=0x557c17de8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f800ff66080 1 --2- 172.21.15.32:0/3089781927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c17deba60 0x557c17de8120 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f800800c4d0 tx=0x7f800800c990 comp rx=0 tx=0).stop 2024-04-17T09:55:12.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f800ff66080 1 -- 172.21.15.32:0/3089781927 shutdown_connections 2024-04-17T09:55:12.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f800ff66080 1 --2- 172.21.15.32:0/3089781927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c17deba60 0x557c17de8120 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f800ff66080 1 --2- 172.21.15.32:0/3089781927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c17deb680 0x557c17de4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f800ff66080 1 --2- 172.21.15.32:0/3089781927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c17d5c860 0x557c17d44a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f800ff66080 1 -- 172.21.15.32:0/3089781927 >> 172.21.15.32:0/3089781927 conn(0x557c17d3fa00 msgr2=0x557c17d43520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:12.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f800ff66080 1 -- 172.21.15.32:0/3089781927 shutdown_connections 2024-04-17T09:55:12.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f800ff66080 1 -- 172.21.15.32:0/3089781927 wait complete. 2024-04-17T09:55:12.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f800ff66080 1 Processor -- start 2024-04-17T09:55:12.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f800ff66080 1 -- start start 2024-04-17T09:55:12.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f800ff66080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c17d5c860 0x557c17dd0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f800ff66080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c17deb680 0x557c17debe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f800ff66080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c17deba60 0x557c17dcdd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f800ff66080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557c17de44d0 con 0x557c17deb680 2024-04-17T09:55:12.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f800ff66080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557c17d450b0 con 0x557c17d5c860 2024-04-17T09:55:12.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f800ff66080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x557c17de4770 con 0x557c17deba60 2024-04-17T09:55:12.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f8007e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c17deb680 0x557c17debe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f8007e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c17deb680 0x557c17debe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38898/0 (socket says 172.21.15.32:38898) 2024-04-17T09:55:12.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f8007e00640 1 -- 172.21.15.32:0/4191920273 learned_addr learned my addr 172.21.15.32:0/4191920273 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:12.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f800d200640 1 --2- 172.21.15.32:0/4191920273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c17d5c860 0x557c17dd0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f800dc00640 1 --2- 172.21.15.32:0/4191920273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c17deba60 0x557c17dcdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f8007e00640 1 -- 172.21.15.32:0/4191920273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c17deba60 msgr2=0x557c17dcdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f8007e00640 1 --2- 172.21.15.32:0/4191920273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557c17deba60 0x557c17dcdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f8007e00640 1 -- 172.21.15.32:0/4191920273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c17d5c860 msgr2=0x557c17dd0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f8007e00640 1 --2- 172.21.15.32:0/4191920273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c17d5c860 0x557c17dd0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f8007e00640 1 -- 172.21.15.32:0/4191920273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c17dcf520 con 0x557c17deb680 2024-04-17T09:55:12.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.449+0000 7f8007e00640 1 --2- 172.21.15.32:0/4191920273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c17deb680 0x557c17debe40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8017f20 tx=0x7f7ff80180b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.453+0000 7f8005600640 1 -- 172.21.15.32:0/4191920273 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff80ba020 con 0x557c17deb680 2024-04-17T09:55:12.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.453+0000 7f8005600640 1 -- 172.21.15.32:0/4191920273 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff80b5670 con 0x557c17deb680 2024-04-17T09:55:12.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.453+0000 7f800ff66080 1 -- 172.21.15.32:0/4191920273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557c17dcf7c0 con 0x557c17deb680 2024-04-17T09:55:12.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.453+0000 7f8005600640 1 -- 172.21.15.32:0/4191920273 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff80af040 con 0x557c17deb680 2024-04-17T09:55:12.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.453+0000 7f800ff66080 1 -- 172.21.15.32:0/4191920273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c17dcfd00 con 0x557c17deb680 2024-04-17T09:55:12.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.453+0000 7f8005600640 1 -- 172.21.15.32:0/4191920273 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7ff80bd020 con 0x557c17deb680 2024-04-17T09:55:12.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.453+0000 7f8005600640 1 --2- 172.21.15.32:0/4191920273 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7fec066d90 0x7f7fec069250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.453+0000 7f8005600640 1 -- 172.21.15.32:0/4191920273 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f7ff8120680 con 0x557c17deb680 2024-04-17T09:55:12.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.453+0000 7f800d200640 1 --2- 172.21.15.32:0/4191920273 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7fec066d90 0x7f7fec069250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.453+0000 7f800ff66080 1 --2- 172.21.15.32:0/4191920273 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x557c17dec380 0x557c17dd0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.453+0000 7f800ff66080 1 -- 172.21.15.32:0/4191920273 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46) -- 0x557c17d12af0 con 0x557c17dec380 2024-04-17T09:55:12.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.453+0000 7f800dc00640 1 --2- 172.21.15.32:0/4191920273 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x557c17dec380 0x557c17dd0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.453+0000 7f800d200640 1 --2- 172.21.15.32:0/4191920273 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7fec066d90 0x7f7fec069250 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8000000f70 tx=0x7f8000001830 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.457+0000 7f800dc00640 1 --2- 172.21.15.32:0/4191920273 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x557c17dec380 0x557c17dd0f00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.457+0000 7f800dc00640 1 -- 172.21.15.32:0/4191920273 <== osd.6 v2:172.21.15.144:6808/3198738696 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v46'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f80080116b0 con 0x557c17dec380 2024-04-17T09:55:12.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.457+0000 7f800ff66080 1 -- 172.21.15.32:0/4191920273 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x557c17dec380 msgr2=0x557c17dd0f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.457+0000 7f800ff66080 1 --2- 172.21.15.32:0/4191920273 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x557c17dec380 0x557c17dd0f00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.457+0000 7f800ff66080 1 -- 172.21.15.32:0/4191920273 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7fec066d90 msgr2=0x7f7fec069250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.457+0000 7f800ff66080 1 --2- 172.21.15.32:0/4191920273 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7fec066d90 0x7f7fec069250 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8000000f70 tx=0x7f8000001830 comp rx=0 tx=0).stop 2024-04-17T09:55:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.457+0000 7f800ff66080 1 -- 172.21.15.32:0/4191920273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c17deb680 msgr2=0x557c17debe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.457+0000 7f800ff66080 1 --2- 172.21.15.32:0/4191920273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c17deb680 0x557c17debe40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8017f20 tx=0x7f7ff80180b0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.457+0000 7f800dc00640 1 -- 172.21.15.32:0/4191920273 reap_dead start 2024-04-17T09:55:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.457+0000 7f800ff66080 1 -- 172.21.15.32:0/4191920273 shutdown_connections 2024-04-17T09:55:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.457+0000 7f800ff66080 1 -- 172.21.15.32:0/4191920273 >> 172.21.15.32:0/4191920273 conn(0x557c17d3fa00 msgr2=0x557c17de8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.457+0000 7f800ff66080 1 -- 172.21.15.32:0/4191920273 shutdown_connections 2024-04-17T09:55:12.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.457+0000 7f800ff66080 1 -- 172.21.15.32:0/4191920273 wait complete. 2024-04-17T09:55:12.462 DEBUG:teuthology.orchestra.run.smithi032:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-17T09:55:12.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:55:12.507 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-04-17T09:55:12.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.649+0000 7fc534800640 1 Processor -- start 2024-04-17T09:55:12.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.649+0000 7fc534800640 1 -- start start 2024-04-17T09:55:12.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.649+0000 7fc534800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc52c162060 0x7fc52c09b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.649+0000 7fc534800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc52c162440 0x7fc52c0d17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.649+0000 7fc534800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc52c1637b0 con 0x7fc52c163300 2024-04-17T09:55:12.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.649+0000 7fc534800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc52c0d0a00 con 0x7fc52c162060 2024-04-17T09:55:12.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.649+0000 7fc534800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc52c0d0ce0 con 0x7fc52c162440 2024-04-17T09:55:12.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.653+0000 7fc531e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc52c162060 0x7fc52c09b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.653+0000 7fc531e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc52c162060 0x7fc52c09b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38904/0 (socket says 172.21.15.32:38904) 2024-04-17T09:55:12.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.653+0000 7fc531e00640 1 -- 172.21.15.32:0/3986019866 learned_addr learned my addr 172.21.15.32:0/3986019866 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:12.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.653+0000 7fc531400640 1 --2- 172.21.15.32:0/3986019866 >> v2:172.21.15.144:3300/0 conn(0x7fc52c162440 0x7fc52c0d17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.653+0000 7fc531e00640 1 -- 172.21.15.32:0/3986019866 >> v2:172.21.15.144:3300/0 conn(0x7fc52c162440 msgr2=0x7fc52c0d17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:12.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.653+0000 7fc531e00640 1 --2- 172.21.15.32:0/3986019866 >> v2:172.21.15.144:3300/0 conn(0x7fc52c162440 0x7fc52c0d17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.653+0000 7fc531e00640 1 -- 172.21.15.32:0/3986019866 >> v1:172.21.15.32:6789/0 conn(0x7fc52c163300 legacy=0x7fc52c09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.653+0000 7fc531e00640 1 -- 172.21.15.32:0/3986019866 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc52c0d0fc0 con 0x7fc52c162060 2024-04-17T09:55:12.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.653+0000 7fc531e00640 1 --2- 172.21.15.32:0/3986019866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52c162060 0x7fc52c09b730 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc52c0b3f60 tx=0x7fc52001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d9bcfc7ca9a4ce server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.653+0000 7fc530a00640 1 -- 172.21.15.32:0/3986019866 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5200ab430 con 0x7fc52c162060 2024-04-17T09:55:12.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.653+0000 7fc530a00640 1 -- 172.21.15.32:0/3986019866 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5200b0070 con 0x7fc52c162060 2024-04-17T09:55:12.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.653+0000 7fc530a00640 1 -- 172.21.15.32:0/3986019866 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5200ab7f0 con 0x7fc52c162060 2024-04-17T09:55:12.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.657+0000 7fc534800640 1 -- 172.21.15.32:0/3986019866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52c162060 msgr2=0x7fc52c09b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.657+0000 7fc534800640 1 --2- 172.21.15.32:0/3986019866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52c162060 0x7fc52c09b730 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc52c0b3f60 tx=0x7fc52001c520 comp rx=0 tx=0).stop 2024-04-17T09:55:12.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.657+0000 7fc534800640 1 -- 172.21.15.32:0/3986019866 shutdown_connections 2024-04-17T09:55:12.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.657+0000 7fc534800640 1 --2- 172.21.15.32:0/3986019866 >> v2:172.21.15.144:3300/0 conn(0x7fc52c162440 0x7fc52c0d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.657+0000 7fc534800640 1 --2- 172.21.15.32:0/3986019866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52c162060 0x7fc52c09b730 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.657+0000 7fc534800640 1 -- 172.21.15.32:0/3986019866 >> 172.21.15.32:0/3986019866 conn(0x7fc52c0b4080 msgr2=0x7fc52c0b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:12.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.657+0000 7fc534800640 1 -- 172.21.15.32:0/3986019866 shutdown_connections 2024-04-17T09:55:12.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.657+0000 7fc534800640 1 -- 172.21.15.32:0/3986019866 wait complete. 2024-04-17T09:55:12.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.657+0000 7fc534800640 1 Processor -- start 2024-04-17T09:55:12.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.657+0000 7fc534800640 1 -- start start 2024-04-17T09:55:12.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.657+0000 7fc534800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc52c162060 0x7fc52c1554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.657+0000 7fc534800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52c162440 0x7fc52c09fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.657+0000 7fc534800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc52c163300 0x7fc52c0c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.657+0000 7fc534800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c0cd3f0 con 0x7fc52c162440 2024-04-17T09:55:12.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.657+0000 7fc534800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c09b310 con 0x7fc52c162060 2024-04-17T09:55:12.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.657+0000 7fc534800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc52c0cd570 con 0x7fc52c163300 2024-04-17T09:55:12.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.657+0000 7fc531400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52c162440 0x7fc52c09fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.657+0000 7fc531400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52c162440 0x7fc52c09fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38918/0 (socket says 172.21.15.32:38918) 2024-04-17T09:55:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.657+0000 7fc531e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc52c162060 0x7fc52c1554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.657+0000 7fc531400640 1 -- 172.21.15.32:0/2319047394 learned_addr learned my addr 172.21.15.32:0/2319047394 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.657+0000 7fc532800640 1 --2- 172.21.15.32:0/2319047394 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc52c163300 0x7fc52c0c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.661+0000 7fc531400640 1 -- 172.21.15.32:0/2319047394 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc52c163300 msgr2=0x7fc52c0c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.661+0000 7fc531400640 1 --2- 172.21.15.32:0/2319047394 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc52c163300 0x7fc52c0c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.661+0000 7fc531400640 1 -- 172.21.15.32:0/2319047394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc52c162060 msgr2=0x7fc52c1554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.661+0000 7fc531400640 1 --2- 172.21.15.32:0/2319047394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc52c162060 0x7fc52c1554e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.661+0000 7fc531400640 1 -- 172.21.15.32:0/2319047394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc52c155e40 con 0x7fc52c162440 2024-04-17T09:55:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.661+0000 7fc531e00640 1 --2- 172.21.15.32:0/2319047394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc52c162060 0x7fc52c1554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:55:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.661+0000 7fc531400640 1 --2- 172.21.15.32:0/2319047394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52c162440 0x7fc52c09fb80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc51c00e930 tx=0x7fc51c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.661+0000 7fc526000640 1 -- 172.21.15.32:0/2319047394 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51c017070 con 0x7fc52c162440 2024-04-17T09:55:12.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.661+0000 7fc526000640 1 -- 172.21.15.32:0/2319047394 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc51c00ca00 con 0x7fc52c162440 2024-04-17T09:55:12.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.661+0000 7fc526000640 1 -- 172.21.15.32:0/2319047394 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51c00cdb0 con 0x7fc52c162440 2024-04-17T09:55:12.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.661+0000 7fc534800640 1 -- 172.21.15.32:0/2319047394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc52c1560b0 con 0x7fc52c162440 2024-04-17T09:55:12.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.661+0000 7fc534800640 1 -- 172.21.15.32:0/2319047394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc52c156670 con 0x7fc52c162440 2024-04-17T09:55:12.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.661+0000 7fc526000640 1 -- 172.21.15.32:0/2319047394 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc51c01a070 con 0x7fc52c162440 2024-04-17T09:55:12.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.661+0000 7fc526000640 1 --2- 172.21.15.32:0/2319047394 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc510068380 0x7fc51006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.661+0000 7fc526000640 1 -- 172.21.15.32:0/2319047394 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fc51c086a10 con 0x7fc52c162440 2024-04-17T09:55:12.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.661+0000 7fc531e00640 1 --2- 172.21.15.32:0/2319047394 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc510068380 0x7fc51006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc537ce6000 1 Processor -- start 2024-04-17T09:55:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc537ce6000 1 -- start start 2024-04-17T09:55:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc531e00640 1 --2- 172.21.15.32:0/2319047394 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc510068380 0x7fc51006a840 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc5200af000 tx=0x7fc5200a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc537ce6000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55915bca1700 0x55915bca1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc537ce6000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55915bca2020 0x55915bc544d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc537ce6000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bc54a10 0x55915bc54df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc537ce6000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55915baa65f0 con 0x55915bc54a10 2024-04-17T09:55:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc537ce6000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55915ba5f7a0 con 0x55915bca1700 2024-04-17T09:55:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc537ce6000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55915baa25b0 con 0x55915bca2020 2024-04-17T09:55:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc532800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bc54a10 0x55915bc54df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc531e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55915bca1700 0x55915bca1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc531400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55915bca2020 0x55915bc544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc532800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bc54a10 0x55915bc54df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38930/0 (socket says 172.21.15.32:38930) 2024-04-17T09:55:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc532800640 1 -- 172.21.15.32:0/3053013917 learned_addr learned my addr 172.21.15.32:0/3053013917 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc531400640 1 -- 172.21.15.32:0/3053013917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55915bca1700 msgr2=0x55915bca1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc531400640 1 --2- 172.21.15.32:0/3053013917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55915bca1700 0x55915bca1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc531400640 1 -- 172.21.15.32:0/3053013917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bc54a10 msgr2=0x55915bc54df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc531400640 1 --2- 172.21.15.32:0/3053013917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bc54a10 0x55915bc54df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc531400640 1 -- 172.21.15.32:0/3053013917 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55915bb4d6e0 con 0x55915bca2020 2024-04-17T09:55:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc531400640 1 --2- 172.21.15.32:0/3053013917 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55915bca2020 0x55915bc544d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc51c084ea0 tx=0x7fc51c05aca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad0e15b83b343b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc51aa00640 1 -- 172.21.15.32:0/3053013917 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51c083070 con 0x55915bca2020 2024-04-17T09:55:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc51aa00640 1 -- 172.21.15.32:0/3053013917 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc51c054ca0 con 0x55915bca2020 2024-04-17T09:55:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc51aa00640 1 -- 172.21.15.32:0/3053013917 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51c021070 con 0x55915bca2020 2024-04-17T09:55:12.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc537ce6000 1 -- 172.21.15.32:0/3053013917 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55915bca2020 msgr2=0x55915bc544d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc537ce6000 1 --2- 172.21.15.32:0/3053013917 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55915bca2020 0x55915bc544d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc51c084ea0 tx=0x7fc51c05aca0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc537ce6000 1 -- 172.21.15.32:0/3053013917 shutdown_connections 2024-04-17T09:55:12.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc537ce6000 1 --2- 172.21.15.32:0/3053013917 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55915bca2020 0x55915bc544d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc537ce6000 1 --2- 172.21.15.32:0/3053013917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bc54a10 0x55915bc54df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc537ce6000 1 --2- 172.21.15.32:0/3053013917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55915bca1700 0x55915bca1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc537ce6000 1 -- 172.21.15.32:0/3053013917 >> 172.21.15.32:0/3053013917 conn(0x55915bc82290 msgr2=0x55915bc84680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:12.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc537ce6000 1 -- 172.21.15.32:0/3053013917 shutdown_connections 2024-04-17T09:55:12.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.665+0000 7fc537ce6000 1 -- 172.21.15.32:0/3053013917 wait complete. 2024-04-17T09:55:12.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc537ce6000 1 -- start start 2024-04-17T09:55:12.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc537ce6000 1 Processor -- start 2024-04-17T09:55:12.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc537ce6000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55915bc54a10 0x55915bc6b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc537ce6000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55915bca1700 0x55915bc6be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc537ce6000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bca2020 0x55915bc6c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc537ce6000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55915bab1520 con 0x55915bca2020 2024-04-17T09:55:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc537ce6000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55915ba3eea0 con 0x55915bca1700 2024-04-17T09:55:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc537ce6000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55915ba65500 con 0x55915bc54a10 2024-04-17T09:55:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc537ce6000 10 client.0 _ll_register_callbacks cb 0x55915bc42170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:55:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc532800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bca2020 0x55915bc6c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc532800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bca2020 0x55915bc6c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38942/0 (socket says 172.21.15.32:38942) 2024-04-17T09:55:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc532800640 1 -- 172.21.15.32:0/2477958761 learned_addr learned my addr 172.21.15.32:0/2477958761 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc531400640 1 --2- 172.21.15.32:0/2477958761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55915bca1700 0x55915bc6be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc531e00640 1 --2- 172.21.15.32:0/2477958761 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55915bc54a10 0x55915bc6b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc532800640 1 -- 172.21.15.32:0/2477958761 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55915bc54a10 msgr2=0x55915bc6b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc532800640 1 --2- 172.21.15.32:0/2477958761 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55915bc54a10 0x55915bc6b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc532800640 1 -- 172.21.15.32:0/2477958761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55915bca1700 msgr2=0x55915bc6be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc532800640 1 --2- 172.21.15.32:0/2477958761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55915bca1700 0x55915bc6be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc532800640 1 -- 172.21.15.32:0/2477958761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55915ba7f540 con 0x55915bca2020 2024-04-17T09:55:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc532800640 1 --2- 172.21.15.32:0/2477958761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bca2020 0x55915bc6c370 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc528007c80 tx=0x7fc5280129f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de579d781aeba2e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc50f400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:55:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc50f400640 1 -- 172.21.15.32:0/2477958761 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc528019020 con 0x55915bca2020 2024-04-17T09:55:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc50f400640 1 -- 172.21.15.32:0/2477958761 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fc528003b90 con 0x55915bca2020 2024-04-17T09:55:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc537ce6000 1 -- 172.21.15.32:0/2477958761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc51c054ca0 con 0x55915bca2020 2024-04-17T09:55:12.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc50f400640 1 -- 172.21.15.32:0/2477958761 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc528023050 con 0x55915bca2020 2024-04-17T09:55:12.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc537ce6000 10 client.5943 fetch_fsmap learned FSMap version 24 2024-04-17T09:55:12.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc537ce6000 1 -- 172.21.15.32:0/2477958761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55915baeaba0 con 0x55915bca2020 2024-04-17T09:55:12.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc50f400640 1 -- 172.21.15.32:0/2477958761 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc51c054ca0 con 0x55915bca2020 2024-04-17T09:55:12.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc50f400640 1 -- 172.21.15.32:0/2477958761 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) ==== 3495+0+0 (secure 0 0 0) 0x7fc528026020 con 0x55915bca2020 2024-04-17T09:55:12.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc537ce6000 10 client.5943 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T09:55:12.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc537ce6000 10 client.5943 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-17T09:55:12.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc537ce6000 20 client.5943 populate_metadata read hostname 'smithi032' 2024-04-17T09:55:12.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc537ce6000 1 --2- 172.21.15.32:0/2477958761 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55915bc6cd80 0x55915bcb29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc537ce6000 4 client.5943 mds_command: new command op to 5742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:55:12.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.669+0000 7fc537ce6000 1 -- 172.21.15.32:0/2477958761 --> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55915bb5f450 con 0x55915bc6cd80 2024-04-17T09:55:12.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.673+0000 7fc531e00640 1 --2- 172.21.15.32:0/2477958761 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55915bc6cd80 0x55915bcb29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.673+0000 7fc531e00640 1 --2- 172.21.15.32:0/2477958761 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55915bc6cd80 0x55915bcb29b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec4f90c00615dc08 server_cookie=d7404aab1ece83b6 in_seq=0 out_seq=0 2024-04-17T09:55:12.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.673+0000 7fc50f400640 10 client.5943 ms_handle_connect on v2:172.21.15.144:6833/2177384675 2024-04-17T09:55:12.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.673+0000 7fc50f400640 1 -- 172.21.15.32:0/2477958761 <== mds.0 v2:172.21.15.144:6833/2177384675 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55915bb5f450 con 0x55915bc6cd80 2024-04-17T09:55:12.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.673+0000 7fc50f400640 10 client.5943 handle_command_reply: tid=0 2024-04-17T09:55:12.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.673+0000 7fc537ce6000 1 client.5943 shutdown 2024-04-17T09:55:12.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.673+0000 7fc537ce6000 1 -- 172.21.15.32:0/2477958761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bca2020 msgr2=0x55915bc6c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.673+0000 7fc537ce6000 1 --2- 172.21.15.32:0/2477958761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bca2020 0x55915bc6c370 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc528007c80 tx=0x7fc5280129f0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.673+0000 7fc537ce6000 1 -- 172.21.15.32:0/2477958761 shutdown_connections 2024-04-17T09:55:12.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.673+0000 7fc537ce6000 1 --2- 172.21.15.32:0/2477958761 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55915bc6cd80 0x55915bcb29b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.673+0000 7fc537ce6000 1 --2- 172.21.15.32:0/2477958761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bca2020 0x55915bc6c370 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.673+0000 7fc537ce6000 1 --2- 172.21.15.32:0/2477958761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55915bca1700 0x55915bc6be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.673+0000 7fc537ce6000 1 --2- 172.21.15.32:0/2477958761 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55915bc54a10 0x55915bc6b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.673+0000 7fc537ce6000 1 -- 172.21.15.32:0/2477958761 >> 172.21.15.32:0/2477958761 conn(0x55915bc82290 msgr2=0x55915bc84680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:12.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.673+0000 7fc50f400640 0 client.5943 ms_handle_reset on v2:172.21.15.144:6833/2177384675 2024-04-17T09:55:12.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.673+0000 7fc537ce6000 1 -- 172.21.15.32:0/2477958761 shutdown_connections 2024-04-17T09:55:12.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.673+0000 7fc537ce6000 1 -- 172.21.15.32:0/2477958761 wait complete. 2024-04-17T09:55:12.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.673+0000 7fc537ce6000 20 client.5943 trim_cache size 0 max 16384 2024-04-17T09:55:12.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.677+0000 7eff5fe00640 1 Processor -- start 2024-04-17T09:55:12.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.677+0000 7eff5fe00640 1 -- start start 2024-04-17T09:55:12.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.677+0000 7eff5fe00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7eff58145ea0 0x7eff5806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.677+0000 7eff5fe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7eff58071db0 0x7eff58072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.677+0000 7eff5fe00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff58160e80 con 0x7eff581601c0 2024-04-17T09:55:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.677+0000 7eff5fe00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7eff58161160 con 0x7eff58071db0 2024-04-17T09:55:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.677+0000 7eff5fe00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7eff5806e460 con 0x7eff58145ea0 2024-04-17T09:55:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.677+0000 7eff5ca00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7eff58071db0 0x7eff58072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.677+0000 7eff5de00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7eff581601c0 0x7eff581605a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52678/0 (socket says 172.21.15.32:52678) 2024-04-17T09:55:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.677+0000 7eff5de00640 1 -- 172.21.15.32:0/3172519309 learned_addr learned my addr 172.21.15.32:0/3172519309 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.677+0000 7eff5d400640 1 --2- 172.21.15.32:0/3172519309 >> v2:172.21.15.144:3300/0 conn(0x7eff58145ea0 0x7eff5806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.677+0000 7eff57e00640 1 -- 172.21.15.32:0/3172519309 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1531472644 0 0) 0x7eff58160e80 con 0x7eff581601c0 2024-04-17T09:55:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.677+0000 7eff5ca00640 1 -- 172.21.15.32:0/3172519309 >> v2:172.21.15.144:3300/0 conn(0x7eff58145ea0 msgr2=0x7eff5806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.677+0000 7eff5ca00640 1 --2- 172.21.15.32:0/3172519309 >> v2:172.21.15.144:3300/0 conn(0x7eff58145ea0 0x7eff5806e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.677+0000 7eff5ca00640 1 -- 172.21.15.32:0/3172519309 >> v1:172.21.15.32:6789/0 conn(0x7eff581601c0 legacy=0x7eff581605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.677+0000 7eff5ca00640 1 -- 172.21.15.32:0/3172519309 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5806de10 con 0x7eff58071db0 2024-04-17T09:55:12.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.677+0000 7eff5d400640 1 --2- 172.21.15.32:0/3172519309 >> v2:172.21.15.144:3300/0 conn(0x7eff58145ea0 0x7eff5806e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:12.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.681+0000 7eff5ca00640 1 --2- 172.21.15.32:0/3172519309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff58071db0 0x7eff58072240 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7eff58073250 tx=0x7eff4401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a3497137cc2183 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.681+0000 7eff57e00640 1 -- 172.21.15.32:0/3172519309 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff440b0070 con 0x7eff58071db0 2024-04-17T09:55:12.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.681+0000 7eff57e00640 1 -- 172.21.15.32:0/3172519309 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff440ab600 con 0x7eff58071db0 2024-04-17T09:55:12.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.681+0000 7eff57e00640 1 -- 172.21.15.32:0/3172519309 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff440ab9d0 con 0x7eff58071db0 2024-04-17T09:55:12.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.681+0000 7eff5fe00640 1 -- 172.21.15.32:0/3172519309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff58071db0 msgr2=0x7eff58072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.681+0000 7eff5fe00640 1 --2- 172.21.15.32:0/3172519309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff58071db0 0x7eff58072240 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7eff58073250 tx=0x7eff4401c3f0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.681+0000 7eff5fe00640 1 -- 172.21.15.32:0/3172519309 shutdown_connections 2024-04-17T09:55:12.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.681+0000 7eff5fe00640 1 --2- 172.21.15.32:0/3172519309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff58071db0 0x7eff58072240 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.681+0000 7eff5fe00640 1 --2- 172.21.15.32:0/3172519309 >> v2:172.21.15.144:3300/0 conn(0x7eff58145ea0 0x7eff5806e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.681+0000 7eff5fe00640 1 -- 172.21.15.32:0/3172519309 >> 172.21.15.32:0/3172519309 conn(0x7eff580b2190 msgr2=0x7eff580b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:12.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.681+0000 7eff5fe00640 1 -- 172.21.15.32:0/3172519309 shutdown_connections 2024-04-17T09:55:12.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.681+0000 7eff5fe00640 1 -- 172.21.15.32:0/3172519309 wait complete. 2024-04-17T09:55:12.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.681+0000 7eff5fe00640 1 Processor -- start 2024-04-17T09:55:12.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff5fe00640 1 -- start start 2024-04-17T09:55:12.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff5fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff58071db0 0x7eff5813ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff5fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff58145ea0 0x7eff5813c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff5fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff581601c0 0x7eff58143090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff5fe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7eff5815e720 con 0x7eff581601c0 2024-04-17T09:55:12.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff5fe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7eff5815e5a0 con 0x7eff58145ea0 2024-04-17T09:55:12.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff5fe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7eff5815e8a0 con 0x7eff58071db0 2024-04-17T09:55:12.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff5de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff581601c0 0x7eff58143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff5de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff581601c0 0x7eff58143090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38960/0 (socket says 172.21.15.32:38960) 2024-04-17T09:55:12.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff5de00640 1 -- 172.21.15.32:0/2867428902 learned_addr learned my addr 172.21.15.32:0/2867428902 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:12.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff5d400640 1 --2- 172.21.15.32:0/2867428902 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff58071db0 0x7eff5813ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff5ca00640 1 --2- 172.21.15.32:0/2867428902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff58145ea0 0x7eff5813c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff5de00640 1 -- 172.21.15.32:0/2867428902 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff58071db0 msgr2=0x7eff5813ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff5de00640 1 --2- 172.21.15.32:0/2867428902 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff58071db0 0x7eff5813ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff5de00640 1 -- 172.21.15.32:0/2867428902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff58145ea0 msgr2=0x7eff5813c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff5de00640 1 --2- 172.21.15.32:0/2867428902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff58145ea0 0x7eff5813c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff5de00640 1 -- 172.21.15.32:0/2867428902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5813c670 con 0x7eff581601c0 2024-04-17T09:55:12.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff5de00640 1 --2- 172.21.15.32:0/2867428902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff581601c0 0x7eff58143090 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7eff5000c730 tx=0x7eff5000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff55600640 1 -- 172.21.15.32:0/2867428902 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50018070 con 0x7eff581601c0 2024-04-17T09:55:12.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff55600640 1 -- 172.21.15.32:0/2867428902 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff5000ce40 con 0x7eff581601c0 2024-04-17T09:55:12.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff55600640 1 -- 172.21.15.32:0/2867428902 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5000a860 con 0x7eff581601c0 2024-04-17T09:55:12.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff5fe00640 1 -- 172.21.15.32:0/2867428902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5813f450 con 0x7eff581601c0 2024-04-17T09:55:12.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff5fe00640 1 -- 172.21.15.32:0/2867428902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff5813fa10 con 0x7eff581601c0 2024-04-17T09:55:12.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.685+0000 7eff5fe00640 1 -- 172.21.15.32:0/2867428902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7eff58068a90 con 0x7eff581601c0 2024-04-17T09:55:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.689+0000 7eff55600640 1 -- 172.21.15.32:0/2867428902 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7eff5000aa00 con 0x7eff581601c0 2024-04-17T09:55:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.689+0000 7eff55600640 1 --2- 172.21.15.32:0/2867428902 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7eff38068380 0x7eff3806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.689+0000 7eff55600640 1 -- 172.21.15.32:0/2867428902 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7eff500865b0 con 0x7eff581601c0 2024-04-17T09:55:12.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.689+0000 7eff5d400640 1 --2- 172.21.15.32:0/2867428902 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7eff38068380 0x7eff3806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.689+0000 7eff55600640 1 -- 172.21.15.32:0/2867428902 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+501 (secure 0 0 0) 0x7eff50021070 con 0x7eff581601c0 2024-04-17T09:55:12.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.689+0000 7eff5fe00640 1 -- 172.21.15.32:0/2867428902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7eff580ad780 con 0x7eff581601c0 2024-04-17T09:55:12.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.689+0000 7eff5d400640 1 --2- 172.21.15.32:0/2867428902 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7eff38068380 0x7eff3806a840 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7eff4c006e90 tx=0x7eff4c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.693+0000 7fc537ce6000 1 Processor -- start 2024-04-17T09:55:12.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.693+0000 7fc537ce6000 1 -- start start 2024-04-17T09:55:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.693+0000 7fc537ce6000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55915bc54a10 0x55915bc6cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.693+0000 7fc537ce6000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bca1700 0x55915bc6d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.693+0000 7fc537ce6000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55915bca2020 0x55915bcb29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.693+0000 7fc537ce6000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c0ca2f0 con 0x55915bca1700 2024-04-17T09:55:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.693+0000 7fc537ce6000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c0c9650 con 0x55915bc54a10 2024-04-17T09:55:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.693+0000 7fc537ce6000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc52c0c9ca0 con 0x55915bca2020 2024-04-17T09:55:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.693+0000 7fc531400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bca1700 0x55915bc6d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.693+0000 7fc531400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bca1700 0x55915bc6d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38968/0 (socket says 172.21.15.32:38968) 2024-04-17T09:55:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.693+0000 7fc531400640 1 -- 172.21.15.32:0/3130555285 learned_addr learned my addr 172.21.15.32:0/3130555285 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.693+0000 7fc531e00640 1 --2- 172.21.15.32:0/3130555285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55915bc54a10 0x55915bc6cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.693+0000 7fc532800640 1 --2- 172.21.15.32:0/3130555285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55915bca2020 0x55915bcb29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.693+0000 7fc531400640 1 -- 172.21.15.32:0/3130555285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55915bca2020 msgr2=0x55915bcb29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.693+0000 7fc531400640 1 --2- 172.21.15.32:0/3130555285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55915bca2020 0x55915bcb29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.693+0000 7fc531400640 1 -- 172.21.15.32:0/3130555285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55915bc54a10 msgr2=0x55915bc6cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.693+0000 7fc531400640 1 --2- 172.21.15.32:0/3130555285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55915bc54a10 0x55915bc6cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.693+0000 7fc531400640 1 -- 172.21.15.32:0/3130555285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55915bc5c500 con 0x55915bca1700 2024-04-17T09:55:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.693+0000 7fc531e00640 1 --2- 172.21.15.32:0/3130555285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55915bc54a10 0x55915bc6cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:55:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.693+0000 7fc531400640 1 --2- 172.21.15.32:0/3130555285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bca1700 0x55915bc6d2c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc51c056930 tx=0x7fc51c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b7c303c3eb8044 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.693+0000 7fc51a000640 1 -- 172.21.15.32:0/3130555285 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51c083040 con 0x55915bca1700 2024-04-17T09:55:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.693+0000 7fc51a000640 1 -- 172.21.15.32:0/3130555285 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc51c05d0c0 con 0x55915bca1700 2024-04-17T09:55:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.693+0000 7fc51a000640 1 -- 172.21.15.32:0/3130555285 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51c055df0 con 0x55915bca1700 2024-04-17T09:55:12.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc537ce6000 1 -- 172.21.15.32:0/3130555285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bca1700 msgr2=0x55915bc6d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc537ce6000 1 --2- 172.21.15.32:0/3130555285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bca1700 0x55915bc6d2c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc51c056930 tx=0x7fc51c021000 comp rx=0 tx=0).stop 2024-04-17T09:55:12.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc537ce6000 1 -- 172.21.15.32:0/3130555285 shutdown_connections 2024-04-17T09:55:12.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc537ce6000 1 --2- 172.21.15.32:0/3130555285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55915bca2020 0x55915bcb29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc537ce6000 1 --2- 172.21.15.32:0/3130555285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bca1700 0x55915bc6d2c0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc537ce6000 1 --2- 172.21.15.32:0/3130555285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55915bc54a10 0x55915bc6cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc537ce6000 1 -- 172.21.15.32:0/3130555285 >> 172.21.15.32:0/3130555285 conn(0x55915bc82290 msgr2=0x55915bcb5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:12.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc537ce6000 1 -- 172.21.15.32:0/3130555285 shutdown_connections 2024-04-17T09:55:12.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc537ce6000 1 -- 172.21.15.32:0/3130555285 wait complete. 2024-04-17T09:55:12.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc537ce6000 1 -- start start 2024-04-17T09:55:12.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc537ce6000 1 Processor -- start 2024-04-17T09:55:12.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc537ce6000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55915bc54a10 0x55915bd10cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc537ce6000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55915bca1700 0x55915bd111f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc537ce6000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bca2020 0x55915bccc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc537ce6000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55915baaac20 con 0x55915bca2020 2024-04-17T09:55:12.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc537ce6000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55915baa5900 con 0x55915bc54a10 2024-04-17T09:55:12.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc537ce6000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55915bab3800 con 0x55915bca1700 2024-04-17T09:55:12.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc537ce6000 10 client.0 _ll_register_callbacks cb 0x7fc5280014b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:55:12.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc532800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bca2020 0x55915bccc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc531400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55915bca1700 0x55915bd111f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc531400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55915bca1700 0x55915bd111f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:58968/0 (socket says 172.21.15.32:58968) 2024-04-17T09:55:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc531400640 1 -- 172.21.15.32:0/3756738714 learned_addr learned my addr 172.21.15.32:0/3756738714 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc531e00640 1 --2- 172.21.15.32:0/3756738714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55915bc54a10 0x55915bd10cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc531400640 1 -- 172.21.15.32:0/3756738714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55915bc54a10 msgr2=0x55915bd10cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc531400640 1 --2- 172.21.15.32:0/3756738714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55915bc54a10 0x55915bd10cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc531400640 1 -- 172.21.15.32:0/3756738714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bca2020 msgr2=0x55915bccc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc531400640 1 --2- 172.21.15.32:0/3756738714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bca2020 0x55915bccc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc531400640 1 -- 172.21.15.32:0/3756738714 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55915bab2f70 con 0x55915bca1700 2024-04-17T09:55:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc531e00640 1 --2- 172.21.15.32:0/3756738714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55915bc54a10 0x55915bd10cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:55:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc531400640 1 --2- 172.21.15.32:0/3756738714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55915bca1700 0x55915bd111f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009310 tx=0x7fc51c0092a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3785cab68368b881 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc50f400640 10 client.0 ms_handle_connect on v2:172.21.15.144:3301/0 2024-04-17T09:55:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc50f400640 1 -- 172.21.15.32:0/3756738714 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51c021070 con 0x55915bca1700 2024-04-17T09:55:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc50f400640 1 -- 172.21.15.32:0/3756738714 <== mon.2 v2:172.21.15.144:3301/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fc51c055ff0 con 0x55915bca1700 2024-04-17T09:55:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.697+0000 7fc537ce6000 1 -- 172.21.15.32:0/3756738714 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc51c05d0c0 con 0x55915bca1700 2024-04-17T09:55:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc50f400640 1 -- 172.21.15.32:0/3756738714 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc51c083070 con 0x55915bca1700 2024-04-17T09:55:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc537ce6000 10 client.5336 fetch_fsmap learned FSMap version 24 2024-04-17T09:55:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc537ce6000 1 -- 172.21.15.32:0/3756738714 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55915ba5f9f0 con 0x55915bca1700 2024-04-17T09:55:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc50f400640 1 -- 172.21.15.32:0/3756738714 <== mon.2 v2:172.21.15.144:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc51c05d0c0 con 0x55915bca1700 2024-04-17T09:55:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc50f400640 1 -- 172.21.15.32:0/3756738714 <== mon.2 v2:172.21.15.144:3301/0 5 ==== fsmap(e 24) ==== 3495+0+0 (secure 0 0 0) 0x7fc51c024060 con 0x55915bca1700 2024-04-17T09:55:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc537ce6000 10 client.5336 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T09:55:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc537ce6000 10 client.5336 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-17T09:55:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc537ce6000 20 client.5336 populate_metadata read hostname 'smithi032' 2024-04-17T09:55:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc537ce6000 1 --2- 172.21.15.32:0/3756738714 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55915bcccf60 0x55915bccd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc537ce6000 4 client.5336 mds_command: new command op to 5742 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-17T09:55:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc537ce6000 1 -- 172.21.15.32:0/3756738714 --> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x55915ba3fed0 con 0x55915bcccf60 2024-04-17T09:55:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc531e00640 1 --2- 172.21.15.32:0/3756738714 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55915bcccf60 0x55915bccd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc531e00640 1 --2- 172.21.15.32:0/3756738714 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55915bcccf60 0x55915bccd370 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6ce46ee65ef5525 server_cookie=8267803f288da084 in_seq=0 out_seq=0 2024-04-17T09:55:12.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc50f400640 10 client.5336 ms_handle_connect on v2:172.21.15.144:6833/2177384675 2024-04-17T09:55:12.706 INFO:tasks.ceph.mds.d.smithi144.stderr:2024-04-17T09:55:12.697+0000 7fb029a00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5904=0-4194304@1} caps={5904=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5904 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x557ad47f2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-17T09:55:12.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc50f400640 1 -- 172.21.15.32:0/3756738714 <== mds.0 v2:172.21.15.144:6833/2177384675 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55915ba3fed0 con 0x55915bcccf60 2024-04-17T09:55:12.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc50f400640 10 client.5336 handle_command_reply: tid=0 2024-04-17T09:55:12.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc537ce6000 1 client.5336 shutdown 2024-04-17T09:55:12.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc537ce6000 1 -- 172.21.15.32:0/3756738714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55915bca1700 msgr2=0x55915bd111f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc537ce6000 1 --2- 172.21.15.32:0/3756738714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55915bca1700 0x55915bd111f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009310 tx=0x7fc51c0092a0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc537ce6000 1 -- 172.21.15.32:0/3756738714 shutdown_connections 2024-04-17T09:55:12.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc537ce6000 1 --2- 172.21.15.32:0/3756738714 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55915bcccf60 0x55915bccd370 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc537ce6000 1 --2- 172.21.15.32:0/3756738714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55915bca1700 0x55915bd111f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc537ce6000 1 --2- 172.21.15.32:0/3756738714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55915bca2020 0x55915bccc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc537ce6000 1 --2- 172.21.15.32:0/3756738714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55915bc54a10 0x55915bd10cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc50f400640 0 client.5336 ms_handle_reset on v2:172.21.15.144:6833/2177384675 2024-04-17T09:55:12.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc537ce6000 1 -- 172.21.15.32:0/3756738714 >> 172.21.15.32:0/3756738714 conn(0x55915bc82290 msgr2=0x55915bc6b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:12.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc537ce6000 1 -- 172.21.15.32:0/3756738714 shutdown_connections 2024-04-17T09:55:12.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc537ce6000 1 -- 172.21.15.32:0/3756738714 wait complete. 2024-04-17T09:55:12.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc537ce6000 20 client.5336 trim_cache size 0 max 16384 2024-04-17T09:55:12.709 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:55:12.709 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-17T09:55:12.709 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "2d4f18ab-c5f5-43c3-8599-0fded1734ab5", 2024-04-17T09:55:12.709 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-17T09:55:12.709 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:55:12.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc534800640 1 -- 172.21.15.32:0/2319047394 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc510068380 msgr2=0x7fc51006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc534800640 1 --2- 172.21.15.32:0/2319047394 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc510068380 0x7fc51006a840 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc5200af000 tx=0x7fc5200a7000 comp rx=0 tx=0).stop 2024-04-17T09:55:12.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc534800640 1 -- 172.21.15.32:0/2319047394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52c162440 msgr2=0x7fc52c09fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc534800640 1 --2- 172.21.15.32:0/2319047394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52c162440 0x7fc52c09fb80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc51c00e930 tx=0x7fc51c00edf0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc534800640 1 -- 172.21.15.32:0/2319047394 shutdown_connections 2024-04-17T09:55:12.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc534800640 1 --2- 172.21.15.32:0/2319047394 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc510068380 0x7fc51006a840 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc534800640 1 --2- 172.21.15.32:0/2319047394 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc52c163300 0x7fc52c0c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc534800640 1 --2- 172.21.15.32:0/2319047394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52c162440 0x7fc52c09fb80 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc534800640 1 --2- 172.21.15.32:0/2319047394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc52c162060 0x7fc52c1554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc534800640 1 -- 172.21.15.32:0/2319047394 >> 172.21.15.32:0/2319047394 conn(0x7fc52c0b4080 msgr2=0x7fc52c0cee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:12.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc534800640 1 -- 172.21.15.32:0/2319047394 shutdown_connections 2024-04-17T09:55:12.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.701+0000 7fc534800640 1 -- 172.21.15.32:0/2319047394 wait complete. 2024-04-17T09:55:12.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:55:12.724 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-17T09:55:12.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.865+0000 7f2678a00640 1 Processor -- start 2024-04-17T09:55:12.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.865+0000 7f2678a00640 1 -- start start 2024-04-17T09:55:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.865+0000 7f2678a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f267413f630 0x7f2674071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.865+0000 7f2678a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2674072330 0x7f2674160320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.865+0000 7f2678a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f267415db00 con 0x7f2674160860 2024-04-17T09:55:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.865+0000 7f2678a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f267413fa10 con 0x7f2674072330 2024-04-17T09:55:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.865+0000 7f2678a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f26740727a0 con 0x7f267413f630 2024-04-17T09:55:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.865+0000 7f266aa00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2674072330 0x7f2674160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.865+0000 7f2673400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f267413f630 0x7f2674071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.865+0000 7f2673e00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f2674160860 0x7f2674141d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:39504/0 (socket says 172.21.15.32:39504) 2024-04-17T09:55:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.865+0000 7f266aa00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2674072330 0x7f2674160320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38998/0 (socket says 172.21.15.32:38998) 2024-04-17T09:55:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.865+0000 7f2673e00640 1 -- 172.21.15.32:0/386095829 learned_addr learned my addr 172.21.15.32:0/386095829 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:12.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.869+0000 7f2673400640 1 -- 172.21.15.32:0/386095829 >> v2:172.21.15.32:3300/0 conn(0x7f2674072330 msgr2=0x7f2674160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:12.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.869+0000 7f2673400640 1 --2- 172.21.15.32:0/386095829 >> v2:172.21.15.32:3300/0 conn(0x7f2674072330 0x7f2674160320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.869+0000 7f2673400640 1 -- 172.21.15.32:0/386095829 >> v1:172.21.15.144:6789/0 conn(0x7f2674160860 legacy=0x7f2674141d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.869+0000 7f2673400640 1 -- 172.21.15.32:0/386095829 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26741610c0 con 0x7f267413f630 2024-04-17T09:55:12.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.869+0000 7f266aa00640 1 --2- 172.21.15.32:0/386095829 >> v2:172.21.15.32:3300/0 conn(0x7f2674072330 0x7f2674160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:55:12.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.869+0000 7f2673400640 1 --2- 172.21.15.32:0/386095829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f267413f630 0x7f2674071df0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f26640097c0 tx=0x7f266401c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39f06c99c712f13f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.869+0000 7f2672a00640 1 -- 172.21.15.32:0/386095829 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26640b0070 con 0x7f267413f630 2024-04-17T09:55:12.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.869+0000 7f2672a00640 1 -- 172.21.15.32:0/386095829 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2664004230 con 0x7f267413f630 2024-04-17T09:55:12.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.869+0000 7f2672a00640 1 -- 172.21.15.32:0/386095829 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664003620 con 0x7f267413f630 2024-04-17T09:55:12.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2678a00640 1 -- 172.21.15.32:0/386095829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f267413f630 msgr2=0x7f2674071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2678a00640 1 --2- 172.21.15.32:0/386095829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f267413f630 0x7f2674071df0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f26640097c0 tx=0x7f266401c430 comp rx=0 tx=0).stop 2024-04-17T09:55:12.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2678a00640 1 -- 172.21.15.32:0/386095829 shutdown_connections 2024-04-17T09:55:12.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2678a00640 1 --2- 172.21.15.32:0/386095829 >> v2:172.21.15.32:3300/0 conn(0x7f2674072330 0x7f2674160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2678a00640 1 --2- 172.21.15.32:0/386095829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f267413f630 0x7f2674071df0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2678a00640 1 -- 172.21.15.32:0/386095829 >> 172.21.15.32:0/386095829 conn(0x7f2674067d70 msgr2=0x7f267406ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:12.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2678a00640 1 -- 172.21.15.32:0/386095829 shutdown_connections 2024-04-17T09:55:12.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2678a00640 1 -- 172.21.15.32:0/386095829 wait complete. 2024-04-17T09:55:12.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2678a00640 1 Processor -- start 2024-04-17T09:55:12.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2678a00640 1 -- start start 2024-04-17T09:55:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2678a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2674072330 0x7f26740aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2678a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f267413f630 0x7f2674137870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2678a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2674160860 0x7f267414f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2678a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2674160c70 con 0x7f2674160860 2024-04-17T09:55:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2678a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f267415d5e0 con 0x7f267413f630 2024-04-17T09:55:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2678a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2674160df0 con 0x7f2674072330 2024-04-17T09:55:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2673e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2674160860 0x7f267414f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2673e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2674160860 0x7f267414f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39000/0 (socket says 172.21.15.32:39000) 2024-04-17T09:55:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2673e00640 1 -- 172.21.15.32:0/2057385779 learned_addr learned my addr 172.21.15.32:0/2057385779 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f266aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f267413f630 0x7f2674137870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2673400640 1 --2- 172.21.15.32:0/2057385779 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2674072330 0x7f26740aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2673e00640 1 -- 172.21.15.32:0/2057385779 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2674072330 msgr2=0x7f26740aec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2673e00640 1 --2- 172.21.15.32:0/2057385779 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2674072330 0x7f26740aec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2673e00640 1 -- 172.21.15.32:0/2057385779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f267413f630 msgr2=0x7f2674137870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2673e00640 1 --2- 172.21.15.32:0/2057385779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f267413f630 0x7f2674137870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.873+0000 7f2673e00640 1 -- 172.21.15.32:0/2057385779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267413f0a0 con 0x7f2674160860 2024-04-17T09:55:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.877+0000 7f2673e00640 1 --2- 172.21.15.32:0/2057385779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2674160860 0x7f267414f9a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f266c00cbf0 tx=0x7f266c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.877+0000 7f266be00640 1 -- 172.21.15.32:0/2057385779 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c017070 con 0x7f2674160860 2024-04-17T09:55:12.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.877+0000 7f266be00640 1 -- 172.21.15.32:0/2057385779 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f266c00ad40 con 0x7f2674160860 2024-04-17T09:55:12.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.877+0000 7f266be00640 1 -- 172.21.15.32:0/2057385779 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c012600 con 0x7f2674160860 2024-04-17T09:55:12.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.877+0000 7f2678a00640 1 -- 172.21.15.32:0/2057385779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26741532c0 con 0x7f2674160860 2024-04-17T09:55:12.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.877+0000 7f2678a00640 1 -- 172.21.15.32:0/2057385779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2674153880 con 0x7f2674160860 2024-04-17T09:55:12.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.877+0000 7f266be00640 1 -- 172.21.15.32:0/2057385779 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f266c01a070 con 0x7f2674160860 2024-04-17T09:55:12.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.877+0000 7f266be00640 1 --2- 172.21.15.32:0/2057385779 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2650068380 0x7f265006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.877+0000 7f266be00640 1 -- 172.21.15.32:0/2057385779 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f266c0866c0 con 0x7f2674160860 2024-04-17T09:55:12.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.877+0000 7f2673400640 1 --2- 172.21.15.32:0/2057385779 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2650068380 0x7f265006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.877+0000 7f267d574000 1 Processor -- start 2024-04-17T09:55:12.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f267d574000 1 -- start start 2024-04-17T09:55:12.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f2673400640 1 --2- 172.21.15.32:0/2057385779 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2650068380 0x7f265006a840 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f26640ad000 tx=0x7f26640a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f267d574000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afe0565260 0x55afe0565640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f267d574000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afe0565b80 0x55afe0518460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f267d574000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe05189a0 0x55afe0518d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f267d574000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55afe036a4f0 con 0x55afe05189a0 2024-04-17T09:55:12.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f267d574000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55afe03236c0 con 0x55afe0565260 2024-04-17T09:55:12.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f267d574000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55afe03664b0 con 0x55afe0565b80 2024-04-17T09:55:12.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f2673e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe05189a0 0x55afe0518d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f266aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afe0565b80 0x55afe0518460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f2673400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afe0565260 0x55afe0565640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f266aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afe0565b80 0x55afe0518460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:58988/0 (socket says 172.21.15.32:58988) 2024-04-17T09:55:12.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f266aa00640 1 -- 172.21.15.32:0/4005222803 learned_addr learned my addr 172.21.15.32:0/4005222803 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:12.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f2673e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe05189a0 0x55afe0518d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39010/0 (socket says 172.21.15.32:39010) 2024-04-17T09:55:12.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f266aa00640 1 -- 172.21.15.32:0/4005222803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afe0565260 msgr2=0x55afe0565640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:12.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f266aa00640 1 --2- 172.21.15.32:0/4005222803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afe0565260 0x55afe0565640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f266aa00640 1 -- 172.21.15.32:0/4005222803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe05189a0 msgr2=0x55afe0518d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:12.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f266aa00640 1 --2- 172.21.15.32:0/4005222803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe05189a0 0x55afe0518d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f266aa00640 1 -- 172.21.15.32:0/4005222803 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afe03af8e0 con 0x55afe0565b80 2024-04-17T09:55:12.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f266aa00640 1 --2- 172.21.15.32:0/4005222803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afe0565b80 0x55afe0518460 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f265c009b10 tx=0x7f265c0112c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=898ac9bcc75c1e9d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f2663e00640 1 -- 172.21.15.32:0/4005222803 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c019070 con 0x55afe0565b80 2024-04-17T09:55:12.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f2663e00640 1 -- 172.21.15.32:0/4005222803 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f265c011cd0 con 0x55afe0565b80 2024-04-17T09:55:12.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f2663e00640 1 -- 172.21.15.32:0/4005222803 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c004230 con 0x55afe0565b80 2024-04-17T09:55:12.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f267d574000 1 -- 172.21.15.32:0/4005222803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afe0565b80 msgr2=0x55afe0518460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f267d574000 1 --2- 172.21.15.32:0/4005222803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afe0565b80 0x55afe0518460 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f265c009b10 tx=0x7f265c0112c0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f267d574000 1 -- 172.21.15.32:0/4005222803 shutdown_connections 2024-04-17T09:55:12.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f267d574000 1 --2- 172.21.15.32:0/4005222803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afe0565b80 0x55afe0518460 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f267d574000 1 --2- 172.21.15.32:0/4005222803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe05189a0 0x55afe0518d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f267d574000 1 --2- 172.21.15.32:0/4005222803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afe0565260 0x55afe0565640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f267d574000 1 -- 172.21.15.32:0/4005222803 >> 172.21.15.32:0/4005222803 conn(0x55afe0546000 msgr2=0x55afe05483f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:12.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f267d574000 1 -- 172.21.15.32:0/4005222803 shutdown_connections 2024-04-17T09:55:12.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f267d574000 1 -- 172.21.15.32:0/4005222803 wait complete. 2024-04-17T09:55:12.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.881+0000 7f267d574000 1 -- start start 2024-04-17T09:55:12.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f267d574000 1 Processor -- start 2024-04-17T09:55:12.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f267d574000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe05189a0 0x55afe051dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f267d574000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afe0565260 0x55afe051e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f267d574000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afe0565b80 0x55afe051ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f267d574000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55afe03eb7b0 con 0x55afe05189a0 2024-04-17T09:55:12.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f267d574000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55afe0425d00 con 0x55afe0565260 2024-04-17T09:55:12.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f267d574000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55afe032f2c0 con 0x55afe0565b80 2024-04-17T09:55:12.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f267d574000 10 client.0 _ll_register_callbacks cb 0x55afe033f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:55:12.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f2673400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe05189a0 0x55afe051dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f2673400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe05189a0 0x55afe051dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39018/0 (socket says 172.21.15.32:39018) 2024-04-17T09:55:12.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f2673400640 1 -- 172.21.15.32:0/2554709394 learned_addr learned my addr 172.21.15.32:0/2554709394 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:12.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f2673e00640 1 --2- 172.21.15.32:0/2554709394 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afe0565b80 0x55afe051ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f266aa00640 1 --2- 172.21.15.32:0/2554709394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afe0565260 0x55afe051e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f2673400640 1 -- 172.21.15.32:0/2554709394 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afe0565b80 msgr2=0x55afe051ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:12.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f2673400640 1 --2- 172.21.15.32:0/2554709394 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afe0565b80 0x55afe051ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f2673400640 1 -- 172.21.15.32:0/2554709394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afe0565260 msgr2=0x55afe051e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:12.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f2673400640 1 --2- 172.21.15.32:0/2554709394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afe0565260 0x55afe051e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f2673400640 1 -- 172.21.15.32:0/2554709394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afe0342490 con 0x55afe05189a0 2024-04-17T09:55:12.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f2673400640 1 --2- 172.21.15.32:0/2554709394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe05189a0 0x55afe051dff0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f266401c910 tx=0x7f266401ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28d4397a66f78c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f2660c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:55:12.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f2660c00640 1 -- 172.21.15.32:0/2554709394 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26640c6070 con 0x55afe05189a0 2024-04-17T09:55:12.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f2660c00640 1 -- 172.21.15.32:0/2554709394 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f26640b6650 con 0x55afe05189a0 2024-04-17T09:55:12.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f267d574000 1 -- 172.21.15.32:0/2554709394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f265c011cd0 con 0x55afe05189a0 2024-04-17T09:55:12.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f2660c00640 1 -- 172.21.15.32:0/2554709394 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f26640d1050 con 0x55afe05189a0 2024-04-17T09:55:12.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f267d574000 10 client.5970 fetch_fsmap learned FSMap version 24 2024-04-17T09:55:12.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f267d574000 1 -- 172.21.15.32:0/2554709394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55afe0335eb0 con 0x55afe05189a0 2024-04-17T09:55:12.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f2660c00640 1 -- 172.21.15.32:0/2554709394 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f265c011cd0 con 0x55afe05189a0 2024-04-17T09:55:12.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f2660c00640 1 -- 172.21.15.32:0/2554709394 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) ==== 3495+0+0 (secure 0 0 0) 0x7f26640d4020 con 0x55afe05189a0 2024-04-17T09:55:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f267d574000 10 client.5970 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T09:55:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f267d574000 10 client.5970 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-17T09:55:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f267d574000 20 client.5970 populate_metadata read hostname 'smithi032' 2024-04-17T09:55:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f267d574000 1 --2- 172.21.15.32:0/2554709394 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55afe0574500 0x55afe05768f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f267d574000 4 client.5970 mds_command: new command op to 5742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:55:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f267d574000 1 -- 172.21.15.32:0/2554709394 --> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afe0303dd0 con 0x55afe0574500 2024-04-17T09:55:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f266aa00640 1 --2- 172.21.15.32:0/2554709394 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55afe0574500 0x55afe05768f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.885+0000 7f266aa00640 1 --2- 172.21.15.32:0/2554709394 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55afe0574500 0x55afe05768f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eeb4915f66a15c9 server_cookie=d5faa69c1db96912 in_seq=0 out_seq=0 2024-04-17T09:55:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.889+0000 7f2660c00640 10 client.5970 ms_handle_connect on v2:172.21.15.144:6833/2177384675 2024-04-17T09:55:12.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.889+0000 7f2660c00640 1 -- 172.21.15.32:0/2554709394 <== mds.0 v2:172.21.15.144:6833/2177384675 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55afe0303dd0 con 0x55afe0574500 2024-04-17T09:55:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.889+0000 7f2660c00640 10 client.5970 handle_command_reply: tid=0 2024-04-17T09:55:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.889+0000 7f267d574000 1 client.5970 shutdown 2024-04-17T09:55:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.889+0000 7f267d574000 1 -- 172.21.15.32:0/2554709394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe05189a0 msgr2=0x55afe051dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.889+0000 7f267d574000 1 --2- 172.21.15.32:0/2554709394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe05189a0 0x55afe051dff0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f266401c910 tx=0x7f266401ac70 comp rx=0 tx=0).stop 2024-04-17T09:55:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.889+0000 7f267d574000 1 -- 172.21.15.32:0/2554709394 shutdown_connections 2024-04-17T09:55:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.889+0000 7f267d574000 1 --2- 172.21.15.32:0/2554709394 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55afe0574500 0x55afe05768f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.889+0000 7f267d574000 1 --2- 172.21.15.32:0/2554709394 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afe0565b80 0x55afe051ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.889+0000 7f267d574000 1 --2- 172.21.15.32:0/2554709394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afe0565260 0x55afe051e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.889+0000 7f2660c00640 0 client.5970 ms_handle_reset on v2:172.21.15.144:6833/2177384675 2024-04-17T09:55:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.889+0000 7f267d574000 1 --2- 172.21.15.32:0/2554709394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe05189a0 0x55afe051dff0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.889+0000 7f267d574000 1 -- 172.21.15.32:0/2554709394 >> 172.21.15.32:0/2554709394 conn(0x55afe0546000 msgr2=0x55afe05483f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.889+0000 7f267d574000 1 -- 172.21.15.32:0/2554709394 shutdown_connections 2024-04-17T09:55:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.889+0000 7f267d574000 1 -- 172.21.15.32:0/2554709394 wait complete. 2024-04-17T09:55:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.889+0000 7f267d574000 20 client.5970 trim_cache size 0 max 16384 2024-04-17T09:55:12.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.905+0000 7f267d574000 1 Processor -- start 2024-04-17T09:55:12.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.905+0000 7f267d574000 1 -- start start 2024-04-17T09:55:12.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.905+0000 7f267d574000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe05189a0 0x55afe05cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.905+0000 7f267d574000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afe0565260 0x55afe05d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.905+0000 7f267d574000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afe0565b80 0x55afe05d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.905+0000 7f267d574000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f26742029a0 con 0x55afe05189a0 2024-04-17T09:55:12.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.905+0000 7f267d574000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2674201d30 con 0x55afe0565260 2024-04-17T09:55:12.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.905+0000 7f267d574000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2674202350 con 0x55afe0565b80 2024-04-17T09:55:12.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.905+0000 7f2673400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe05189a0 0x55afe05cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.905+0000 7f2673400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe05189a0 0x55afe05cfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39034/0 (socket says 172.21.15.32:39034) 2024-04-17T09:55:12.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.905+0000 7f2673400640 1 -- 172.21.15.32:0/3700555657 learned_addr learned my addr 172.21.15.32:0/3700555657 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:12.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f2673400640 1 -- 172.21.15.32:0/3700555657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afe0565b80 msgr2=0x55afe05d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:12.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f266aa00640 1 --2- 172.21.15.32:0/3700555657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afe0565260 0x55afe05d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f2673400640 1 --2- 172.21.15.32:0/3700555657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afe0565b80 0x55afe05d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f2673400640 1 -- 172.21.15.32:0/3700555657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afe0565260 msgr2=0x55afe05d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:12.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f2673400640 1 --2- 172.21.15.32:0/3700555657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afe0565260 0x55afe05d01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f2673400640 1 -- 172.21.15.32:0/3700555657 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afe0324a20 con 0x55afe05189a0 2024-04-17T09:55:12.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f266aa00640 1 --2- 172.21.15.32:0/3700555657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afe0565260 0x55afe05d01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:55:12.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f2673400640 1 --2- 172.21.15.32:0/3700555657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe05189a0 0x55afe05cfc90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f26640097c0 tx=0x7f26640c1000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb54489e47b2b36 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f2663400640 1 -- 172.21.15.32:0/3700555657 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26640c6070 con 0x55afe05189a0 2024-04-17T09:55:12.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f2663400640 1 -- 172.21.15.32:0/3700555657 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26640b6650 con 0x55afe05189a0 2024-04-17T09:55:12.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f2663400640 1 -- 172.21.15.32:0/3700555657 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26640b0040 con 0x55afe05189a0 2024-04-17T09:55:12.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f267d574000 1 -- 172.21.15.32:0/3700555657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe05189a0 msgr2=0x55afe05cfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f267d574000 1 --2- 172.21.15.32:0/3700555657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe05189a0 0x55afe05cfc90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f26640097c0 tx=0x7f26640c1000 comp rx=0 tx=0).stop 2024-04-17T09:55:12.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f267d574000 1 -- 172.21.15.32:0/3700555657 shutdown_connections 2024-04-17T09:55:12.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f267d574000 1 --2- 172.21.15.32:0/3700555657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afe0565b80 0x55afe05d0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f267d574000 1 --2- 172.21.15.32:0/3700555657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afe0565260 0x55afe05d01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f267d574000 1 --2- 172.21.15.32:0/3700555657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe05189a0 0x55afe05cfc90 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f267d574000 1 -- 172.21.15.32:0/3700555657 >> 172.21.15.32:0/3700555657 conn(0x55afe0546000 msgr2=0x55afe0516bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f267d574000 1 -- 172.21.15.32:0/3700555657 shutdown_connections 2024-04-17T09:55:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f267d574000 1 -- 172.21.15.32:0/3700555657 wait complete. 2024-04-17T09:55:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f267d574000 1 -- start start 2024-04-17T09:55:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f267d574000 1 Processor -- start 2024-04-17T09:55:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f267d574000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afe05189a0 0x55afe0578e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f267d574000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afe0565260 0x55afe0590480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f267d574000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe0565b80 0x55afe05909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f267d574000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55afe030e050 con 0x55afe0565b80 2024-04-17T09:55:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f267d574000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55afe032c570 con 0x55afe0565260 2024-04-17T09:55:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f267d574000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55afe0369800 con 0x55afe05189a0 2024-04-17T09:55:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f267d574000 10 client.0 _ll_register_callbacks cb 0x7f2664009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:55:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f2673e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe0565b80 0x55afe05909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f2673e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe0565b80 0x55afe05909c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39042/0 (socket says 172.21.15.32:39042) 2024-04-17T09:55:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f2673e00640 1 -- 172.21.15.32:0/3142402705 learned_addr learned my addr 172.21.15.32:0/3142402705 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f2673400640 1 --2- 172.21.15.32:0/3142402705 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afe05189a0 0x55afe0578e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f266aa00640 1 --2- 172.21.15.32:0/3142402705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afe0565260 0x55afe0590480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f2673e00640 1 -- 172.21.15.32:0/3142402705 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afe05189a0 msgr2=0x55afe0578e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f2673e00640 1 --2- 172.21.15.32:0/3142402705 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afe05189a0 0x55afe0578e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f2673e00640 1 -- 172.21.15.32:0/3142402705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afe0565260 msgr2=0x55afe0590480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f2673e00640 1 --2- 172.21.15.32:0/3142402705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afe0565260 0x55afe0590480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.909+0000 7f2673e00640 1 -- 172.21.15.32:0/3142402705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afe03ebc40 con 0x55afe0565b80 2024-04-17T09:55:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2673e00640 1 --2- 172.21.15.32:0/3142402705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe0565b80 0x55afe05909c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f266c056520 tx=0x7f266c056650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318616fd6e8713e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2660c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:55:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2660c00640 1 -- 172.21.15.32:0/3142402705 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c073060 con 0x55afe0565b80 2024-04-17T09:55:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2660c00640 1 -- 172.21.15.32:0/3142402705 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f266c05ee30 con 0x55afe0565b80 2024-04-17T09:55:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f267d574000 1 -- 172.21.15.32:0/3142402705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26640b6650 con 0x55afe0565b80 2024-04-17T09:55:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2660c00640 1 -- 172.21.15.32:0/3142402705 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f266c083050 con 0x55afe0565b80 2024-04-17T09:55:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f267d574000 10 client.5976 fetch_fsmap learned FSMap version 24 2024-04-17T09:55:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f267d574000 1 -- 172.21.15.32:0/3142402705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55afe030f600 con 0x55afe0565b80 2024-04-17T09:55:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2660c00640 1 -- 172.21.15.32:0/3142402705 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26640b6650 con 0x55afe0565b80 2024-04-17T09:55:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2660c00640 1 -- 172.21.15.32:0/3142402705 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) ==== 3495+0+0 (secure 0 0 0) 0x7f266c004b70 con 0x55afe0565b80 2024-04-17T09:55:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f267d574000 10 client.5976 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T09:55:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f267d574000 10 client.5976 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-17T09:55:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f267d574000 20 client.5976 populate_metadata read hostname 'smithi032' 2024-04-17T09:55:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f267d574000 1 --2- 172.21.15.32:0/3142402705 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55afe0590f00 0x55afe05d38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f267d574000 4 client.5976 mds_command: new command op to 5742 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T09:55:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f267d574000 1 -- 172.21.15.32:0/3142402705 --> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55afe0545c40 con 0x55afe0590f00 2024-04-17T09:55:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2673400640 1 --2- 172.21.15.32:0/3142402705 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55afe0590f00 0x55afe05d38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:12.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2673400640 1 --2- 172.21.15.32:0/3142402705 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55afe0590f00 0x55afe05d38b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8da0ca30381daeb server_cookie=e83174b96d0a6a6e in_seq=0 out_seq=0 2024-04-17T09:55:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2660c00640 10 client.5976 ms_handle_connect on v2:172.21.15.144:6833/2177384675 2024-04-17T09:55:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2660c00640 1 -- 172.21.15.32:0/3142402705 <== mds.0 v2:172.21.15.144:6833/2177384675 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55afe0545c40 con 0x55afe0590f00 2024-04-17T09:55:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2660c00640 10 client.5976 handle_command_reply: tid=0 2024-04-17T09:55:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f267d574000 1 client.5976 shutdown 2024-04-17T09:55:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f267d574000 1 -- 172.21.15.32:0/3142402705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe0565b80 msgr2=0x55afe05909c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f267d574000 1 --2- 172.21.15.32:0/3142402705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe0565b80 0x55afe05909c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f266c056520 tx=0x7f266c056650 comp rx=0 tx=0).stop 2024-04-17T09:55:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f267d574000 1 -- 172.21.15.32:0/3142402705 shutdown_connections 2024-04-17T09:55:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f267d574000 1 --2- 172.21.15.32:0/3142402705 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55afe0590f00 0x55afe05d38b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f267d574000 1 --2- 172.21.15.32:0/3142402705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55afe0565b80 0x55afe05909c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f267d574000 1 --2- 172.21.15.32:0/3142402705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afe0565260 0x55afe0590480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f267d574000 1 --2- 172.21.15.32:0/3142402705 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afe05189a0 0x55afe0578e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f267d574000 1 -- 172.21.15.32:0/3142402705 >> 172.21.15.32:0/3142402705 conn(0x55afe0546000 msgr2=0x55afe0517450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:12.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2660c00640 0 client.5976 ms_handle_reset on v2:172.21.15.144:6833/2177384675 2024-04-17T09:55:12.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f267d574000 1 -- 172.21.15.32:0/3142402705 shutdown_connections 2024-04-17T09:55:12.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f267d574000 1 -- 172.21.15.32:0/3142402705 wait complete. 2024-04-17T09:55:12.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f267d574000 20 client.5976 trim_cache size 0 max 16384 2024-04-17T09:55:12.922 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:55:12.922 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T09:55:12.922 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-17T09:55:12.922 INFO:teuthology.orchestra.run.smithi032.stdout: "2d4f18ab-c5f5-43c3-8599-0fded1734ab5": { 2024-04-17T09:55:12.922 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-04-17T09:55:12.922 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "2d4f18ab-c5f5-43c3-8599-0fded1734ab5", 2024-04-17T09:55:12.922 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,repair" 2024-04-17T09:55:12.922 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:12.922 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:12.922 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:55:12.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2678a00640 1 -- 172.21.15.32:0/2057385779 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2650068380 msgr2=0x7f265006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2678a00640 1 --2- 172.21.15.32:0/2057385779 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2650068380 0x7f265006a840 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f26640ad000 tx=0x7f26640a6000 comp rx=0 tx=0).stop 2024-04-17T09:55:12.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2678a00640 1 -- 172.21.15.32:0/2057385779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2674160860 msgr2=0x7f267414f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:12.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2678a00640 1 --2- 172.21.15.32:0/2057385779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2674160860 0x7f267414f9a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f266c00cbf0 tx=0x7f266c00a430 comp rx=0 tx=0).stop 2024-04-17T09:55:12.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2678a00640 1 -- 172.21.15.32:0/2057385779 shutdown_connections 2024-04-17T09:55:12.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2678a00640 1 --2- 172.21.15.32:0/2057385779 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2650068380 0x7f265006a840 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2678a00640 1 --2- 172.21.15.32:0/2057385779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2674160860 0x7f267414f9a0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2678a00640 1 --2- 172.21.15.32:0/2057385779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f267413f630 0x7f2674137870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2678a00640 1 --2- 172.21.15.32:0/2057385779 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2674072330 0x7f26740aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:12.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2678a00640 1 -- 172.21.15.32:0/2057385779 >> 172.21.15.32:0/2057385779 conn(0x7f2674067d70 msgr2=0x7f26740cdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:12.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2678a00640 1 -- 172.21.15.32:0/2057385779 shutdown_connections 2024-04-17T09:55:12.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:12.913+0000 7f2678a00640 1 -- 172.21.15.32:0/2057385779 wait complete. 2024-04-17T09:55:12.934 INFO:tasks.cephfs.filesystem:scrub status for tag:2d4f18ab-c5f5-43c3-8599-0fded1734ab5 - {'path': '/', 'tag': '2d4f18ab-c5f5-43c3-8599-0fded1734ab5', 'options': 'recursive,repair'} 2024-04-17T09:55:13.227 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:55:13.230 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:55:13.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:13.873+0000 7eff55600640 1 -- 172.21.15.32:0/2867428902 <== mon.0 v2:172.21.15.32:3300/0 7 ==== log(6 entries from seq 1 at 2024-04-17T09:55:12.703725+0000) ==== 1078+0+0 (secure 0 0 0) 0x7eff500545b0 con 0x7eff581601c0 2024-04-17T09:55:13.876 INFO:teuthology.orchestra.run.smithi032.stdout: cluster: 2024-04-17T09:55:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: id: ff178df8-4c06-4367-a1e9-2e3a66d870d0 2024-04-17T09:55:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: health: HEALTH_OK 2024-04-17T09:55:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: services: 2024-04-17T09:55:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-04-17T09:55:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: mgr: y(active, since 2m), standbys: x 2024-04-17T09:55:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: mds: 1/1 daemons up, 3 standby 2024-04-17T09:55:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-04-17T09:55:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: data: 2024-04-17T09:55:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: volumes: 1/1 healthy 2024-04-17T09:55:13.877 INFO:teuthology.orchestra.run.smithi032.stdout: pools: 3 pools, 97 pgs 2024-04-17T09:55:13.878 INFO:teuthology.orchestra.run.smithi032.stdout: objects: 24 objects, 579 KiB 2024-04-17T09:55:13.878 INFO:teuthology.orchestra.run.smithi032.stdout: usage: 237 MiB used, 720 GiB / 720 GiB avail 2024-04-17T09:55:13.878 INFO:teuthology.orchestra.run.smithi032.stdout: pgs: 97 active+clean 2024-04-17T09:55:13.878 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:13.878 INFO:teuthology.orchestra.run.smithi032.stdout: io: 2024-04-17T09:55:13.878 INFO:teuthology.orchestra.run.smithi032.stdout: client: 1.7 KiB/s wr, 0 op/s rd, 5 op/s wr 2024-04-17T09:55:13.878 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:13.878 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:13.878 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T09:55:12.703725+0000 mds.d [INF] scrub queued for path: / 2024-04-17T09:55:13.879 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T09:55:12.703729+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-04-17T09:55:13.879 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T09:55:12.703735+0000 mds.d [INF] scrub summary: active paths [/] 2024-04-17T09:55:13.879 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T09:55:12.704558+0000 mds.d [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-04-17T09:55:13.879 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T09:55:12.704570+0000 mds.d [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-04-17T09:55:13.879 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T09:55:12.705211+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-04-17T09:55:15.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:15.877+0000 7eff55600640 1 -- 172.21.15.32:0/2867428902 <== mon.0 v2:172.21.15.32:3300/0 8 ==== log(1 entries from seq 7 at 2024-04-17T09:55:14.037234+0000) ==== 189+0+0 (secure 0 0 0) 0x7eff50054810 con 0x7eff581601c0 2024-04-17T09:55:15.880 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T09:55:14.037234+0000 mds.d [INF] scrub summary: idle 2024-04-17T09:55:42.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:55:42.935 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-17T09:55:43.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.092+0000 7ff220400640 1 Processor -- start 2024-04-17T09:55:43.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.092+0000 7ff220400640 1 -- start start 2024-04-17T09:55:43.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.092+0000 7ff220400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff21806ba30 0x7ff21806be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.092+0000 7ff220400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff2180cf370 0x7ff2180cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.092+0000 7ff220400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff21806c350 con 0x7ff218160350 2024-04-17T09:55:43.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.092+0000 7ff220400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff2180d0130 con 0x7ff2180cf370 2024-04-17T09:55:43.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.092+0000 7ff220400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff21815d090 con 0x7ff21806ba30 2024-04-17T09:55:43.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.092+0000 7ff21d000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff2180cf370 0x7ff2180cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.092+0000 7ff21d000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff2180cf370 0x7ff2180cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55860/0 (socket says 172.21.15.32:55860) 2024-04-17T09:55:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.092+0000 7ff21d000640 1 -- 172.21.15.32:0/926563182 learned_addr learned my addr 172.21.15.32:0/926563182 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.092+0000 7ff21da00640 1 --2- 172.21.15.32:0/926563182 >> v2:172.21.15.144:3300/0 conn(0x7ff21806ba30 0x7ff21806be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.092+0000 7ff21d000640 1 -- 172.21.15.32:0/926563182 >> v2:172.21.15.144:3300/0 conn(0x7ff21806ba30 msgr2=0x7ff21806be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.092+0000 7ff21d000640 1 --2- 172.21.15.32:0/926563182 >> v2:172.21.15.144:3300/0 conn(0x7ff21806ba30 0x7ff21806be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.092+0000 7ff21d000640 1 -- 172.21.15.32:0/926563182 >> v1:172.21.15.144:6789/0 conn(0x7ff218160350 legacy=0x7ff218160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.092+0000 7ff21d000640 1 -- 172.21.15.32:0/926563182 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21815d370 con 0x7ff2180cf370 2024-04-17T09:55:43.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.096+0000 7ff21d000640 1 --2- 172.21.15.32:0/926563182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2180cf370 0x7ff2180cf7e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff2180b5730 tx=0x7ff20401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e1ec0c1b1b252c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:43.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.096+0000 7ff217e00640 1 -- 172.21.15.32:0/926563182 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2040b0070 con 0x7ff2180cf370 2024-04-17T09:55:43.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.096+0000 7ff217e00640 1 -- 172.21.15.32:0/926563182 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2040ab600 con 0x7ff2180cf370 2024-04-17T09:55:43.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.096+0000 7ff217e00640 1 -- 172.21.15.32:0/926563182 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2040ab9d0 con 0x7ff2180cf370 2024-04-17T09:55:43.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.096+0000 7ff220400640 1 -- 172.21.15.32:0/926563182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2180cf370 msgr2=0x7ff2180cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:43.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.096+0000 7ff220400640 1 --2- 172.21.15.32:0/926563182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2180cf370 0x7ff2180cf7e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff2180b5730 tx=0x7ff20401c490 comp rx=0 tx=0).stop 2024-04-17T09:55:43.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.096+0000 7ff220400640 1 -- 172.21.15.32:0/926563182 shutdown_connections 2024-04-17T09:55:43.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.096+0000 7ff220400640 1 --2- 172.21.15.32:0/926563182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2180cf370 0x7ff2180cf7e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.096+0000 7ff220400640 1 --2- 172.21.15.32:0/926563182 >> v2:172.21.15.144:3300/0 conn(0x7ff21806ba30 0x7ff21806be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.096+0000 7ff220400640 1 -- 172.21.15.32:0/926563182 >> 172.21.15.32:0/926563182 conn(0x7ff2180b2120 msgr2=0x7ff2180b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:43.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.096+0000 7ff220400640 1 -- 172.21.15.32:0/926563182 shutdown_connections 2024-04-17T09:55:43.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.096+0000 7ff220400640 1 -- 172.21.15.32:0/926563182 wait complete. 2024-04-17T09:55:43.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff220400640 1 Processor -- start 2024-04-17T09:55:43.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff220400640 1 -- start start 2024-04-17T09:55:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff220400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff21806ba30 0x7ff218141650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff220400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2180cf370 0x7ff2181597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff220400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff218160350 0x7ff21813cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff220400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff21815dcf0 con 0x7ff2180cf370 2024-04-17T09:55:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff220400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff21815db70 con 0x7ff218160350 2024-04-17T09:55:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff220400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff218146230 con 0x7ff21806ba30 2024-04-17T09:55:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff21d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2180cf370 0x7ff2181597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff21d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2180cf370 0x7ff2181597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55872/0 (socket says 172.21.15.32:55872) 2024-04-17T09:55:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff21d000640 1 -- 172.21.15.32:0/2963372736 learned_addr learned my addr 172.21.15.32:0/2963372736 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff21da00640 1 --2- 172.21.15.32:0/2963372736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff21806ba30 0x7ff218141650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff21e400640 1 --2- 172.21.15.32:0/2963372736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff218160350 0x7ff21813cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff21d000640 1 -- 172.21.15.32:0/2963372736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff21806ba30 msgr2=0x7ff218141650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff21d000640 1 --2- 172.21.15.32:0/2963372736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff21806ba30 0x7ff218141650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff21d000640 1 -- 172.21.15.32:0/2963372736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff218160350 msgr2=0x7ff21813cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff21d000640 1 --2- 172.21.15.32:0/2963372736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff218160350 0x7ff21813cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff21d000640 1 -- 172.21.15.32:0/2963372736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2180b1040 con 0x7ff2180cf370 2024-04-17T09:55:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff21e400640 1 --2- 172.21.15.32:0/2963372736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff218160350 0x7ff21813cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff21d000640 1 --2- 172.21.15.32:0/2963372736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2180cf370 0x7ff2181597f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff2040190a0 tx=0x7ff204019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff215600640 1 -- 172.21.15.32:0/2963372736 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2040b9070 con 0x7ff2180cf370 2024-04-17T09:55:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff215600640 1 -- 172.21.15.32:0/2963372736 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2040b4400 con 0x7ff2180cf370 2024-04-17T09:55:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff215600640 1 -- 172.21.15.32:0/2963372736 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2040b0040 con 0x7ff2180cf370 2024-04-17T09:55:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff220400640 1 -- 172.21.15.32:0/2963372736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2180b12b0 con 0x7ff2180cf370 2024-04-17T09:55:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.100+0000 7ff220400640 1 -- 172.21.15.32:0/2963372736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2180b1870 con 0x7ff2180cf370 2024-04-17T09:55:43.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.104+0000 7ff215600640 1 -- 172.21.15.32:0/2963372736 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff2040b33a0 con 0x7ff2180cf370 2024-04-17T09:55:43.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.104+0000 7ff215600640 1 --2- 172.21.15.32:0/2963372736 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff1f8068380 0x7ff1f806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.104+0000 7ff215600640 1 -- 172.21.15.32:0/2963372736 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7ff20411fba0 con 0x7ff2180cf370 2024-04-17T09:55:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.104+0000 7ff21da00640 1 --2- 172.21.15.32:0/2963372736 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff1f8068380 0x7ff1f806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.104+0000 7ff223990000 1 Processor -- start 2024-04-17T09:55:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.104+0000 7ff223990000 1 -- start start 2024-04-17T09:55:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.104+0000 7ff21da00640 1 --2- 172.21.15.32:0/2963372736 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff1f8068380 0x7ff1f806a840 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff2181436e0 tx=0x7ff20c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:43.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff223990000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea13663260 0x55ea13663640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff223990000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663b80 0x55ea13616460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff223990000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea136169a0 0x55ea13616d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff223990000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ea134684f0 con 0x55ea13663b80 2024-04-17T09:55:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff223990000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ea134216c0 con 0x55ea13663260 2024-04-17T09:55:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff223990000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ea134644b0 con 0x55ea136169a0 2024-04-17T09:55:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff21d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663b80 0x55ea13616460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff21d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663b80 0x55ea13616460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55888/0 (socket says 172.21.15.32:55888) 2024-04-17T09:55:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff21d000640 1 -- 172.21.15.32:0/1314502506 learned_addr learned my addr 172.21.15.32:0/1314502506 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff21da00640 1 --2- 172.21.15.32:0/1314502506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea13663260 0x55ea13663640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff21e400640 1 --2- 172.21.15.32:0/1314502506 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea136169a0 0x55ea13616d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff21d000640 1 -- 172.21.15.32:0/1314502506 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea136169a0 msgr2=0x55ea13616d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff21d000640 1 --2- 172.21.15.32:0/1314502506 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea136169a0 0x55ea13616d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff21d000640 1 -- 172.21.15.32:0/1314502506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea13663260 msgr2=0x55ea13663640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:43.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff21d000640 1 --2- 172.21.15.32:0/1314502506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea13663260 0x55ea13663640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff21d000640 1 -- 172.21.15.32:0/1314502506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea134ad8e0 con 0x55ea13663b80 2024-04-17T09:55:43.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff21d000640 1 --2- 172.21.15.32:0/1314502506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663b80 0x55ea13616460 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff2040f8070 tx=0x7ff2040f3d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c983369e64e44482 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:43.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff20a000640 1 -- 172.21.15.32:0/1314502506 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2040edd70 con 0x55ea13663b80 2024-04-17T09:55:43.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff20a000640 1 -- 172.21.15.32:0/1314502506 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2040ec0e0 con 0x55ea13663b80 2024-04-17T09:55:43.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff20a000640 1 -- 172.21.15.32:0/1314502506 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2040f62c0 con 0x55ea13663b80 2024-04-17T09:55:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff223990000 1 -- 172.21.15.32:0/1314502506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663b80 msgr2=0x55ea13616460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff223990000 1 --2- 172.21.15.32:0/1314502506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663b80 0x55ea13616460 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff2040f8070 tx=0x7ff2040f3d70 comp rx=0 tx=0).stop 2024-04-17T09:55:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff223990000 1 -- 172.21.15.32:0/1314502506 shutdown_connections 2024-04-17T09:55:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff223990000 1 --2- 172.21.15.32:0/1314502506 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea136169a0 0x55ea13616d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff223990000 1 --2- 172.21.15.32:0/1314502506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663b80 0x55ea13616460 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff223990000 1 --2- 172.21.15.32:0/1314502506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea13663260 0x55ea13663640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff223990000 1 -- 172.21.15.32:0/1314502506 >> 172.21.15.32:0/1314502506 conn(0x55ea13644000 msgr2=0x55ea136463f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff223990000 1 -- 172.21.15.32:0/1314502506 shutdown_connections 2024-04-17T09:55:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff223990000 1 -- 172.21.15.32:0/1314502506 wait complete. 2024-04-17T09:55:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff223990000 1 -- start start 2024-04-17T09:55:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.108+0000 7ff223990000 1 Processor -- start 2024-04-17T09:55:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff223990000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea136169a0 0x55ea1361bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff223990000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea13663260 0x55ea1361c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff223990000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663b80 0x55ea1361ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff223990000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ea134e97b0 con 0x55ea13663b80 2024-04-17T09:55:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff223990000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ea13523d00 con 0x55ea13663260 2024-04-17T09:55:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff223990000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ea1342d2c0 con 0x55ea136169a0 2024-04-17T09:55:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff223990000 10 client.0 _ll_register_callbacks cb 0x55ea1343d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:55:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff21e400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663b80 0x55ea1361ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff21e400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663b80 0x55ea1361ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55904/0 (socket says 172.21.15.32:55904) 2024-04-17T09:55:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff21e400640 1 -- 172.21.15.32:0/2659076027 learned_addr learned my addr 172.21.15.32:0/2659076027 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff21d000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea13663260 0x55ea1361c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff21da00640 1 --2- 172.21.15.32:0/2659076027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea136169a0 0x55ea1361bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff21e400640 1 -- 172.21.15.32:0/2659076027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea136169a0 msgr2=0x55ea1361bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:43.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff21e400640 1 --2- 172.21.15.32:0/2659076027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea136169a0 0x55ea1361bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff21e400640 1 -- 172.21.15.32:0/2659076027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea13663260 msgr2=0x55ea1361c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:43.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff21e400640 1 --2- 172.21.15.32:0/2659076027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea13663260 0x55ea1361c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff21e400640 1 -- 172.21.15.32:0/2659076027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea13440490 con 0x55ea13663b80 2024-04-17T09:55:43.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff21d000640 1 --2- 172.21.15.32:0/2659076027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea13663260 0x55ea1361c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:55:43.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff21e400640 1 --2- 172.21.15.32:0/2659076027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663b80 0x55ea1361ca60 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff2100059b0 tx=0x7ff210001960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63326381f57df793 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:43.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff1fea00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:55:43.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff1fea00640 1 -- 172.21.15.32:0/2659076027 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2100020d0 con 0x55ea13663b80 2024-04-17T09:55:43.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff1fea00640 1 -- 172.21.15.32:0/2659076027 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7ff210013570 con 0x55ea13663b80 2024-04-17T09:55:43.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff223990000 1 -- 172.21.15.32:0/2659076027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2040ec0e0 con 0x55ea13663b80 2024-04-17T09:55:43.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff1fea00640 1 -- 172.21.15.32:0/2659076027 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff210021050 con 0x55ea13663b80 2024-04-17T09:55:43.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff223990000 10 client.5988 fetch_fsmap learned FSMap version 24 2024-04-17T09:55:43.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff223990000 1 -- 172.21.15.32:0/2659076027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55ea13433eb0 con 0x55ea13663b80 2024-04-17T09:55:43.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff1fea00640 1 -- 172.21.15.32:0/2659076027 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2040ec0e0 con 0x55ea13663b80 2024-04-17T09:55:43.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff1fea00640 1 -- 172.21.15.32:0/2659076027 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) ==== 3495+0+0 (secure 0 0 0) 0x7ff210013770 con 0x55ea13663b80 2024-04-17T09:55:43.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff223990000 10 client.5988 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T09:55:43.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff223990000 10 client.5988 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-17T09:55:43.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff223990000 20 client.5988 populate_metadata read hostname 'smithi032' 2024-04-17T09:55:43.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff223990000 1 --2- 172.21.15.32:0/2659076027 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55ea13672500 0x55ea136748f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff223990000 4 client.5988 mds_command: new command op to 5742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:55:43.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff223990000 1 -- 172.21.15.32:0/2659076027 --> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea13401dd0 con 0x55ea13672500 2024-04-17T09:55:43.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff21da00640 1 --2- 172.21.15.32:0/2659076027 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55ea13672500 0x55ea136748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff21da00640 1 --2- 172.21.15.32:0/2659076027 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55ea13672500 0x55ea136748f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18ee155c89c0dd76 server_cookie=d05791b208e199a6 in_seq=0 out_seq=0 2024-04-17T09:55:43.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.112+0000 7ff1fea00640 10 client.5988 ms_handle_connect on v2:172.21.15.144:6833/2177384675 2024-04-17T09:55:43.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.116+0000 7ff1fea00640 1 -- 172.21.15.32:0/2659076027 <== mds.0 v2:172.21.15.144:6833/2177384675 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ea13401dd0 con 0x55ea13672500 2024-04-17T09:55:43.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.116+0000 7ff1fea00640 10 client.5988 handle_command_reply: tid=0 2024-04-17T09:55:43.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.116+0000 7ff223990000 1 client.5988 shutdown 2024-04-17T09:55:43.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.116+0000 7ff223990000 1 -- 172.21.15.32:0/2659076027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663b80 msgr2=0x55ea1361ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:43.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.116+0000 7ff223990000 1 --2- 172.21.15.32:0/2659076027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663b80 0x55ea1361ca60 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff2100059b0 tx=0x7ff210001960 comp rx=0 tx=0).stop 2024-04-17T09:55:43.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.116+0000 7ff223990000 1 -- 172.21.15.32:0/2659076027 shutdown_connections 2024-04-17T09:55:43.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.116+0000 7ff223990000 1 --2- 172.21.15.32:0/2659076027 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55ea13672500 0x55ea136748f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.116+0000 7ff223990000 1 --2- 172.21.15.32:0/2659076027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663b80 0x55ea1361ca60 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.116+0000 7ff223990000 1 --2- 172.21.15.32:0/2659076027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea13663260 0x55ea1361c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.116+0000 7ff223990000 1 --2- 172.21.15.32:0/2659076027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea136169a0 0x55ea1361bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.116+0000 7ff1fea00640 0 client.5988 ms_handle_reset on v2:172.21.15.144:6833/2177384675 2024-04-17T09:55:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.116+0000 7ff223990000 1 -- 172.21.15.32:0/2659076027 >> 172.21.15.32:0/2659076027 conn(0x55ea13644000 msgr2=0x55ea136463f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.116+0000 7ff223990000 1 -- 172.21.15.32:0/2659076027 shutdown_connections 2024-04-17T09:55:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.116+0000 7ff223990000 1 -- 172.21.15.32:0/2659076027 wait complete. 2024-04-17T09:55:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.116+0000 7ff223990000 20 client.5988 trim_cache size 0 max 16384 2024-04-17T09:55:43.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.132+0000 7ff223990000 1 Processor -- start 2024-04-17T09:55:43.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.132+0000 7ff223990000 1 -- start start 2024-04-17T09:55:43.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.132+0000 7ff223990000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea136169a0 0x55ea13616140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.132+0000 7ff223990000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea13663260 0x55ea13614bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.132+0000 7ff223990000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663b80 0x55ea13615100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.132+0000 7ff223990000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff2180b4510 con 0x55ea13663b80 2024-04-17T09:55:43.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.132+0000 7ff223990000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff2180b0b40 con 0x55ea136169a0 2024-04-17T09:55:43.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.132+0000 7ff223990000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff2180b3ea0 con 0x55ea13663260 2024-04-17T09:55:43.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.132+0000 7ff21e400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663b80 0x55ea13615100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.132+0000 7ff21e400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663b80 0x55ea13615100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55908/0 (socket says 172.21.15.32:55908) 2024-04-17T09:55:43.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.132+0000 7ff21e400640 1 -- 172.21.15.32:0/1837404002 learned_addr learned my addr 172.21.15.32:0/1837404002 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:43.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff21da00640 1 --2- 172.21.15.32:0/1837404002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea136169a0 0x55ea13616140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff21d000640 1 --2- 172.21.15.32:0/1837404002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea13663260 0x55ea13614bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff21e400640 1 -- 172.21.15.32:0/1837404002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea13663260 msgr2=0x55ea13614bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:43.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff21e400640 1 --2- 172.21.15.32:0/1837404002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea13663260 0x55ea13614bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff21e400640 1 -- 172.21.15.32:0/1837404002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea136169a0 msgr2=0x55ea13616140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:43.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff21e400640 1 --2- 172.21.15.32:0/1837404002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea136169a0 0x55ea13616140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff21e400640 1 -- 172.21.15.32:0/1837404002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea13422a20 con 0x55ea13663b80 2024-04-17T09:55:43.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff21e400640 1 --2- 172.21.15.32:0/1837404002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663b80 0x55ea13615100 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff210003a00 tx=0x7ff21000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50efea302a319b7e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:43.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff209600640 1 -- 172.21.15.32:0/1837404002 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21002a070 con 0x55ea13663b80 2024-04-17T09:55:43.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff209600640 1 -- 172.21.15.32:0/1837404002 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff210019070 con 0x55ea13663b80 2024-04-17T09:55:43.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff209600640 1 -- 172.21.15.32:0/1837404002 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210013cc0 con 0x55ea13663b80 2024-04-17T09:55:43.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff223990000 1 -- 172.21.15.32:0/1837404002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663b80 msgr2=0x55ea13615100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:43.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff223990000 1 --2- 172.21.15.32:0/1837404002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663b80 0x55ea13615100 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff210003a00 tx=0x7ff21000e000 comp rx=0 tx=0).stop 2024-04-17T09:55:43.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff223990000 1 -- 172.21.15.32:0/1837404002 shutdown_connections 2024-04-17T09:55:43.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff223990000 1 --2- 172.21.15.32:0/1837404002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea13663260 0x55ea13614bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff223990000 1 --2- 172.21.15.32:0/1837404002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663b80 0x55ea13615100 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff223990000 1 --2- 172.21.15.32:0/1837404002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea136169a0 0x55ea13616140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff223990000 1 -- 172.21.15.32:0/1837404002 >> 172.21.15.32:0/1837404002 conn(0x55ea13644000 msgr2=0x55ea13677b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff223990000 1 -- 172.21.15.32:0/1837404002 shutdown_connections 2024-04-17T09:55:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff223990000 1 -- 172.21.15.32:0/1837404002 wait complete. 2024-04-17T09:55:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff223990000 1 -- start start 2024-04-17T09:55:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff223990000 1 Processor -- start 2024-04-17T09:55:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff223990000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea136169a0 0x55ea13677000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff223990000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663260 0x55ea1368e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff223990000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea13663b80 0x55ea1368e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff223990000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ea1340c050 con 0x55ea13663260 2024-04-17T09:55:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff223990000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ea1342a570 con 0x55ea13663b80 2024-04-17T09:55:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff223990000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ea13467800 con 0x55ea136169a0 2024-04-17T09:55:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff223990000 10 client.0 _ll_register_callbacks cb 0x7ff210003ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:55:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff21d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663260 0x55ea1368e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff21d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663260 0x55ea1368e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55916/0 (socket says 172.21.15.32:55916) 2024-04-17T09:55:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff21d000640 1 -- 172.21.15.32:0/2499875609 learned_addr learned my addr 172.21.15.32:0/2499875609 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff21da00640 1 --2- 172.21.15.32:0/2499875609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea136169a0 0x55ea13677000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.136+0000 7ff21e400640 1 --2- 172.21.15.32:0/2499875609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea13663b80 0x55ea1368e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff21d000640 1 -- 172.21.15.32:0/2499875609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea136169a0 msgr2=0x55ea13677000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff21d000640 1 --2- 172.21.15.32:0/2499875609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea136169a0 0x55ea13677000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff21d000640 1 -- 172.21.15.32:0/2499875609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea13663b80 msgr2=0x55ea1368e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff21d000640 1 --2- 172.21.15.32:0/2499875609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea13663b80 0x55ea1368e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff21d000640 1 -- 172.21.15.32:0/2499875609 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea134e9c40 con 0x55ea13663260 2024-04-17T09:55:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff21e400640 1 --2- 172.21.15.32:0/2499875609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea13663b80 0x55ea1368e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff21d000640 1 --2- 172.21.15.32:0/2499875609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663260 0x55ea1368e480 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff204097d60 tx=0x7ff2040b6070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a774d37db00113bc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff1fea00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:55:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff1fea00640 1 -- 172.21.15.32:0/2499875609 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20411d070 con 0x55ea13663260 2024-04-17T09:55:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff1fea00640 1 -- 172.21.15.32:0/2499875609 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7ff2040efcd0 con 0x55ea13663260 2024-04-17T09:55:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff223990000 1 -- 172.21.15.32:0/2499875609 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff210019070 con 0x55ea13663260 2024-04-17T09:55:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff1fea00640 1 -- 172.21.15.32:0/2499875609 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff2040f62c0 con 0x55ea13663260 2024-04-17T09:55:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff223990000 10 client.5994 fetch_fsmap learned FSMap version 24 2024-04-17T09:55:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff223990000 1 -- 172.21.15.32:0/2499875609 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55ea1340d600 con 0x55ea13663260 2024-04-17T09:55:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff1fea00640 1 -- 172.21.15.32:0/2499875609 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff210019070 con 0x55ea13663260 2024-04-17T09:55:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff1fea00640 1 -- 172.21.15.32:0/2499875609 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) ==== 3495+0+0 (secure 0 0 0) 0x7ff2040f8110 con 0x55ea13663260 2024-04-17T09:55:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff223990000 10 client.5994 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T09:55:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff223990000 10 client.5994 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-17T09:55:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff223990000 20 client.5994 populate_metadata read hostname 'smithi032' 2024-04-17T09:55:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff223990000 1 --2- 172.21.15.32:0/2499875609 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55ea136d1ee0 0x55ea136d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff223990000 4 client.5994 mds_command: new command op to 5742 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T09:55:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff223990000 1 -- 172.21.15.32:0/2499875609 --> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ea13643c40 con 0x55ea136d1ee0 2024-04-17T09:55:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff21da00640 1 --2- 172.21.15.32:0/2499875609 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55ea136d1ee0 0x55ea136d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff21da00640 1 --2- 172.21.15.32:0/2499875609 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55ea136d1ee0 0x55ea136d22f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9d54d12be1c80d6 server_cookie=575f1befeccb4398 in_seq=0 out_seq=0 2024-04-17T09:55:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff1fea00640 10 client.5994 ms_handle_connect on v2:172.21.15.144:6833/2177384675 2024-04-17T09:55:43.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff1fea00640 1 -- 172.21.15.32:0/2499875609 <== mds.0 v2:172.21.15.144:6833/2177384675 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ea13643c40 con 0x55ea136d1ee0 2024-04-17T09:55:43.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff1fea00640 10 client.5994 handle_command_reply: tid=0 2024-04-17T09:55:43.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff223990000 1 client.5994 shutdown 2024-04-17T09:55:43.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff223990000 1 -- 172.21.15.32:0/2499875609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663260 msgr2=0x55ea1368e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:43.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff223990000 1 --2- 172.21.15.32:0/2499875609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663260 0x55ea1368e480 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff204097d60 tx=0x7ff2040b6070 comp rx=0 tx=0).stop 2024-04-17T09:55:43.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff223990000 1 -- 172.21.15.32:0/2499875609 shutdown_connections 2024-04-17T09:55:43.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff223990000 1 --2- 172.21.15.32:0/2499875609 >> [v2:172.21.15.144:6833/2177384675,v1:172.21.15.144:6835/2177384675] conn(0x55ea136d1ee0 0x55ea136d22f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff223990000 1 --2- 172.21.15.32:0/2499875609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea13663b80 0x55ea1368e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff223990000 1 --2- 172.21.15.32:0/2499875609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ea13663260 0x55ea1368e480 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff223990000 1 --2- 172.21.15.32:0/2499875609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea136169a0 0x55ea13677000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff1fea00640 0 client.5994 ms_handle_reset on v2:172.21.15.144:6833/2177384675 2024-04-17T09:55:43.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff223990000 1 -- 172.21.15.32:0/2499875609 >> 172.21.15.32:0/2499875609 conn(0x55ea13644000 msgr2=0x55ea13661ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:43.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff223990000 1 -- 172.21.15.32:0/2499875609 shutdown_connections 2024-04-17T09:55:43.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff223990000 1 -- 172.21.15.32:0/2499875609 wait complete. 2024-04-17T09:55:43.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff223990000 20 client.5994 trim_cache size 0 max 16384 2024-04-17T09:55:43.148 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:55:43.148 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-17T09:55:43.148 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-17T09:55:43.148 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:55:43.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff220400640 1 -- 172.21.15.32:0/2963372736 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff1f8068380 msgr2=0x7ff1f806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:43.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff220400640 1 --2- 172.21.15.32:0/2963372736 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff1f8068380 0x7ff1f806a840 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff2181436e0 tx=0x7ff20c008040 comp rx=0 tx=0).stop 2024-04-17T09:55:43.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff220400640 1 -- 172.21.15.32:0/2963372736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2180cf370 msgr2=0x7ff2181597f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:43.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff220400640 1 --2- 172.21.15.32:0/2963372736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2180cf370 0x7ff2181597f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff2040190a0 tx=0x7ff204019230 comp rx=0 tx=0).stop 2024-04-17T09:55:43.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff220400640 1 -- 172.21.15.32:0/2963372736 shutdown_connections 2024-04-17T09:55:43.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff220400640 1 --2- 172.21.15.32:0/2963372736 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff1f8068380 0x7ff1f806a840 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff220400640 1 --2- 172.21.15.32:0/2963372736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff218160350 0x7ff21813cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff220400640 1 --2- 172.21.15.32:0/2963372736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2180cf370 0x7ff2181597f0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff220400640 1 --2- 172.21.15.32:0/2963372736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff21806ba30 0x7ff218141650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.140+0000 7ff220400640 1 -- 172.21.15.32:0/2963372736 >> 172.21.15.32:0/2963372736 conn(0x7ff2180b2120 msgr2=0x7ff21813de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:43.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.144+0000 7ff220400640 1 -- 172.21.15.32:0/2963372736 shutdown_connections 2024-04-17T09:55:43.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.144+0000 7ff220400640 1 -- 172.21.15.32:0/2963372736 wait complete. 2024-04-17T09:55:43.169 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T09:55:43.270 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:55:43.282 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:55:43.329 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:55:43.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.412+0000 7f400a200640 1 Processor -- start 2024-04-17T09:55:43.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.412+0000 7f400a200640 1 -- start start 2024-04-17T09:55:43.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.412+0000 7f400a200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40040ce8e0 0x7f40040b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.412+0000 7f400a200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f400409b100 0x7f40040d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.412+0000 7f400a200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4004070d10 con 0x7f40040ce500 2024-04-17T09:55:43.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.412+0000 7f400a200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f4004070ff0 con 0x7f40040ce8e0 2024-04-17T09:55:43.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.412+0000 7f400a200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f4004071340 con 0x7f400409b100 2024-04-17T09:55:43.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.412+0000 7f4003e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40040ce8e0 0x7f40040b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.412+0000 7f4003e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40040ce8e0 0x7f40040b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55932/0 (socket says 172.21.15.32:55932) 2024-04-17T09:55:43.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.412+0000 7f4003e00640 1 -- 172.21.15.32:0/189260134 learned_addr learned my addr 172.21.15.32:0/189260134 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:43.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.412+0000 7f4008e00640 1 --1- 172.21.15.32:0/189260134 >> v1:172.21.15.32:6789/0 conn(0x7f40040ce500 0x7f400409a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53630/0 (socket says 172.21.15.32:53630) 2024-04-17T09:55:43.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.412+0000 7f4009800640 1 --2- 172.21.15.32:0/189260134 >> v2:172.21.15.144:3300/0 conn(0x7f400409b100 0x7f40040d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.412+0000 7f4003e00640 1 -- 172.21.15.32:0/189260134 >> v2:172.21.15.144:3300/0 conn(0x7f400409b100 msgr2=0x7f40040d0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:43.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.412+0000 7f4003e00640 1 --2- 172.21.15.32:0/189260134 >> v2:172.21.15.144:3300/0 conn(0x7f400409b100 0x7f40040d0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.412+0000 7f4003e00640 1 -- 172.21.15.32:0/189260134 >> v1:172.21.15.32:6789/0 conn(0x7f40040ce500 legacy=0x7f400409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:43.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.412+0000 7f4003e00640 1 -- 172.21.15.32:0/189260134 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4004071620 con 0x7f40040ce8e0 2024-04-17T09:55:43.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.412+0000 7f4009800640 1 --2- 172.21.15.32:0/189260134 >> v2:172.21.15.144:3300/0 conn(0x7f400409b100 0x7f40040d0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:55:43.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.416+0000 7f4003e00640 1 --2- 172.21.15.32:0/189260134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40040ce8e0 0x7f40040b4970 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f40040d0840 tx=0x7f3ff001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c2552687c1a353 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:43.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.416+0000 7f4003400640 1 -- 172.21.15.32:0/189260134 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff00b1070 con 0x7f40040ce8e0 2024-04-17T09:55:43.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.416+0000 7f4003400640 1 -- 172.21.15.32:0/189260134 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ff00ac640 con 0x7f40040ce8e0 2024-04-17T09:55:43.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.416+0000 7f4003400640 1 -- 172.21.15.32:0/189260134 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff00aca10 con 0x7f40040ce8e0 2024-04-17T09:55:43.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.416+0000 7f400a200640 1 -- 172.21.15.32:0/189260134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40040ce8e0 msgr2=0x7f40040b4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:43.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.416+0000 7f400a200640 1 --2- 172.21.15.32:0/189260134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40040ce8e0 0x7f40040b4970 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f40040d0840 tx=0x7f3ff001c400 comp rx=0 tx=0).stop 2024-04-17T09:55:43.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.416+0000 7f400a200640 1 -- 172.21.15.32:0/189260134 shutdown_connections 2024-04-17T09:55:43.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.416+0000 7f400a200640 1 --2- 172.21.15.32:0/189260134 >> v2:172.21.15.144:3300/0 conn(0x7f400409b100 0x7f40040d0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.416+0000 7f400a200640 1 --2- 172.21.15.32:0/189260134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40040ce8e0 0x7f40040b4970 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.416+0000 7f400a200640 1 -- 172.21.15.32:0/189260134 >> 172.21.15.32:0/189260134 conn(0x7f40040b3e50 msgr2=0x7f40040b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:43.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.416+0000 7f400a200640 1 -- 172.21.15.32:0/189260134 shutdown_connections 2024-04-17T09:55:43.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.416+0000 7f400a200640 1 -- 172.21.15.32:0/189260134 wait complete. 2024-04-17T09:55:43.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.416+0000 7f400a200640 1 Processor -- start 2024-04-17T09:55:43.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.420+0000 7f400a200640 1 -- start start 2024-04-17T09:55:43.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.420+0000 7f400a200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f400409b100 0x7f4004147190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.420+0000 7f400a200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40040ce500 0x7f400415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.420+0000 7f400a200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40040ce8e0 0x7f400415a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.420+0000 7f400a200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f40040cd550 con 0x7f400409b100 2024-04-17T09:55:43.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.420+0000 7f400a200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4004070a60 con 0x7f40040ce500 2024-04-17T09:55:43.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.420+0000 7f400a200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f40040cd6d0 con 0x7f40040ce8e0 2024-04-17T09:55:43.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.420+0000 7f4008e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f400409b100 0x7f4004147190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.420+0000 7f4008e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f400409b100 0x7f4004147190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55948/0 (socket says 172.21.15.32:55948) 2024-04-17T09:55:43.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.420+0000 7f4008e00640 1 -- 172.21.15.32:0/3809936306 learned_addr learned my addr 172.21.15.32:0/3809936306 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:43.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.420+0000 7f4008e00640 1 -- 172.21.15.32:0/3809936306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40040ce8e0 msgr2=0x7f400415a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:43.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.420+0000 7f4008e00640 1 --2- 172.21.15.32:0/3809936306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40040ce8e0 0x7f400415a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.420+0000 7f4003e00640 1 --2- 172.21.15.32:0/3809936306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40040ce500 0x7f400415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.420+0000 7f4008e00640 1 -- 172.21.15.32:0/3809936306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40040ce500 msgr2=0x7f400415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:43.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.420+0000 7f4008e00640 1 --2- 172.21.15.32:0/3809936306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40040ce500 0x7f400415bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.420+0000 7f4008e00640 1 -- 172.21.15.32:0/3809936306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4004167540 con 0x7f400409b100 2024-04-17T09:55:43.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.420+0000 7f4008e00640 1 --2- 172.21.15.32:0/3809936306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f400409b100 0x7f4004147190 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800c420 tx=0x7f3ff800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:43.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.420+0000 7f4000c00640 1 -- 172.21.15.32:0/3809936306 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff80074e0 con 0x7f400409b100 2024-04-17T09:55:43.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.420+0000 7f4000c00640 1 -- 172.21.15.32:0/3809936306 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ff8007680 con 0x7f400409b100 2024-04-17T09:55:43.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.420+0000 7f4000c00640 1 -- 172.21.15.32:0/3809936306 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8004a30 con 0x7f400409b100 2024-04-17T09:55:43.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.420+0000 7f400a200640 1 -- 172.21.15.32:0/3809936306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f400415c3d0 con 0x7f400409b100 2024-04-17T09:55:43.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.420+0000 7f400a200640 1 -- 172.21.15.32:0/3809936306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40040c98b0 con 0x7f400409b100 2024-04-17T09:55:43.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.420+0000 7f400a200640 1 -- 172.21.15.32:0/3809936306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f400404fd00 con 0x7f400409b100 2024-04-17T09:55:43.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.424+0000 7f4000c00640 1 -- 172.21.15.32:0/3809936306 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3ff801a020 con 0x7f400409b100 2024-04-17T09:55:43.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.424+0000 7f4000c00640 1 --2- 172.21.15.32:0/3809936306 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3fe4068380 0x7f3fe406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.424+0000 7f4000c00640 1 -- 172.21.15.32:0/3809936306 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f3ff80862a0 con 0x7f400409b100 2024-04-17T09:55:43.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.424+0000 7f4003e00640 1 --2- 172.21.15.32:0/3809936306 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3fe4068380 0x7f3fe406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.424+0000 7f4003e00640 1 --2- 172.21.15.32:0/3809936306 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3fe4068380 0x7f3fe406a840 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00ae000 tx=0x7f3ff0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:43.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.428+0000 7f4000c00640 1 -- 172.21.15.32:0/3809936306 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3ff80567c0 con 0x7f400409b100 2024-04-17T09:55:43.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.628+0000 7f400a200640 1 -- 172.21.15.32:0/3809936306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40040b82e0 con 0x7f400409b100 2024-04-17T09:55:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.628+0000 7f4000c00640 1 -- 172.21.15.32:0/3809936306 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5202 (secure 0 0 0) 0x7f3ff80560c0 con 0x7f400409b100 2024-04-17T09:55:43.631 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:43.631 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2542578926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2542578926},{"type":"v1","addr":"172.21.15.144:6834","nonce":2542578926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1424948520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1424948520},{"type":"v1","addr":"172.21.15.32:6836","nonce":1424948520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/393813154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":393813154},{"type":"v1","addr":"172.21.15.32:6837","nonce":393813154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:55:03.233552+0000","modified":"2024-04-17T09:55:06.243321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5742":{"gid":5742,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/2177384675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2177384675},{"type":"v1","addr":"172.21.15.144:6835","nonce":2177384675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742]},"id":3}]} 2024-04-17T09:55:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2024-04-17T09:55:43.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.632+0000 7f400a200640 1 -- 172.21.15.32:0/3809936306 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3fe4068380 msgr2=0x7f3fe406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:43.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.632+0000 7f400a200640 1 --2- 172.21.15.32:0/3809936306 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3fe4068380 0x7f3fe406a840 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00ae000 tx=0x7f3ff0008040 comp rx=0 tx=0).stop 2024-04-17T09:55:43.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.632+0000 7f400a200640 1 -- 172.21.15.32:0/3809936306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f400409b100 msgr2=0x7f4004147190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:43.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.632+0000 7f400a200640 1 --2- 172.21.15.32:0/3809936306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f400409b100 0x7f4004147190 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800c420 tx=0x7f3ff800c8e0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.632+0000 7f400a200640 1 -- 172.21.15.32:0/3809936306 shutdown_connections 2024-04-17T09:55:43.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.632+0000 7f400a200640 1 --2- 172.21.15.32:0/3809936306 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3fe4068380 0x7f3fe406a840 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.632+0000 7f400a200640 1 --2- 172.21.15.32:0/3809936306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40040ce8e0 0x7f400415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.632+0000 7f400a200640 1 --2- 172.21.15.32:0/3809936306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40040ce500 0x7f400415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.632+0000 7f400a200640 1 --2- 172.21.15.32:0/3809936306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f400409b100 0x7f4004147190 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.632+0000 7f400a200640 1 -- 172.21.15.32:0/3809936306 >> 172.21.15.32:0/3809936306 conn(0x7f40040b3e50 msgr2=0x7f40040b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:43.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.632+0000 7f400a200640 1 -- 172.21.15.32:0/3809936306 shutdown_connections 2024-04-17T09:55:43.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.632+0000 7f400a200640 1 -- 172.21.15.32:0/3809936306 wait complete. 2024-04-17T09:55:43.652 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-17T09:55:43.831 INFO:teuthology.orchestra.run.smithi144.stdout:{"message":"","return_code":0} 2024-04-17T09:55:43.831 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-17T09:55:43.831 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-17T09:55:43.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417dc53080 1 Processor -- start 2024-04-17T09:55:43.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417dc53080 1 -- start start 2024-04-17T09:55:43.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417dc53080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577d11b4230 0x5577d123e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417dc53080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577d11b4610 0x5577d123b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417dc53080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5577d1241270 0x5577d1198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417dc53080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5577d11b75f0 con 0x5577d11b4230 2024-04-17T09:55:43.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417dc53080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5577d1110f90 con 0x5577d11b4610 2024-04-17T09:55:43.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417dc53080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5577d116a340 con 0x5577d1241270 2024-04-17T09:55:43.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417ae00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577d11b4230 0x5577d123e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417ae00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577d11b4230 0x5577d123e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55954/0 (socket says 172.21.15.32:55954) 2024-04-17T09:55:43.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417ae00640 1 -- 172.21.15.32:0/3189035673 learned_addr learned my addr 172.21.15.32:0/3189035673 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:43.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417b800640 1 --2- 172.21.15.32:0/3189035673 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5577d1241270 0x5577d1198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417a400640 1 --2- 172.21.15.32:0/3189035673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577d11b4610 0x5577d123b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417ae00640 1 -- 172.21.15.32:0/3189035673 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5577d1241270 msgr2=0x5577d1198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:43.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417ae00640 1 --2- 172.21.15.32:0/3189035673 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5577d1241270 0x5577d1198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417ae00640 1 -- 172.21.15.32:0/3189035673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577d11b4610 msgr2=0x5577d123b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:43.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417ae00640 1 --2- 172.21.15.32:0/3189035673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577d11b4610 0x5577d123b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417ae00640 1 -- 172.21.15.32:0/3189035673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577d1095950 con 0x5577d11b4230 2024-04-17T09:55:43.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417ae00640 1 --2- 172.21.15.32:0/3189035673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577d11b4230 0x5577d123e270 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f416c0ad040 tx=0x7f416c003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5372212085f29116 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f4179000640 1 -- 172.21.15.32:0/3189035673 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c0af040 con 0x5577d11b4230 2024-04-17T09:55:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f4179000640 1 -- 172.21.15.32:0/3189035673 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f416c0b9070 con 0x5577d11b4230 2024-04-17T09:55:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f4179000640 1 -- 172.21.15.32:0/3189035673 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c0aacd0 con 0x5577d11b4230 2024-04-17T09:55:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417dc53080 1 -- 172.21.15.32:0/3189035673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577d11b4230 msgr2=0x5577d123e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417dc53080 1 --2- 172.21.15.32:0/3189035673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577d11b4230 0x5577d123e270 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f416c0ad040 tx=0x7f416c003030 comp rx=0 tx=0).stop 2024-04-17T09:55:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417dc53080 1 -- 172.21.15.32:0/3189035673 shutdown_connections 2024-04-17T09:55:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417dc53080 1 --2- 172.21.15.32:0/3189035673 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5577d1241270 0x5577d1198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417dc53080 1 --2- 172.21.15.32:0/3189035673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577d11b4610 0x5577d123b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417dc53080 1 --2- 172.21.15.32:0/3189035673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577d11b4230 0x5577d123e270 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417dc53080 1 -- 172.21.15.32:0/3189035673 >> 172.21.15.32:0/3189035673 conn(0x5577d116a620 msgr2=0x5577d123d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417dc53080 1 -- 172.21.15.32:0/3189035673 shutdown_connections 2024-04-17T09:55:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.880+0000 7f417dc53080 1 -- 172.21.15.32:0/3189035673 wait complete. 2024-04-17T09:55:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f417dc53080 1 Processor -- start 2024-04-17T09:55:43.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f417dc53080 1 -- start start 2024-04-17T09:55:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f417dc53080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577d11b4230 0x5577d1224960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f417dc53080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5577d11b4610 0x5577d123eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f417dc53080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577d1241270 0x5577d123f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f417dc53080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5577d123a9d0 con 0x5577d11b4230 2024-04-17T09:55:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f417dc53080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5577d11b3710 con 0x5577d1241270 2024-04-17T09:55:43.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f417dc53080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5577d123ac70 con 0x5577d11b4610 2024-04-17T09:55:43.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f417ae00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577d11b4230 0x5577d1224960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f417ae00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577d11b4230 0x5577d1224960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55968/0 (socket says 172.21.15.32:55968) 2024-04-17T09:55:43.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f417ae00640 1 -- 172.21.15.32:0/2456242516 learned_addr learned my addr 172.21.15.32:0/2456242516 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:43.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f417a400640 1 --2- 172.21.15.32:0/2456242516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5577d11b4610 0x5577d123eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f417b800640 1 --2- 172.21.15.32:0/2456242516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577d1241270 0x5577d123f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f417ae00640 1 -- 172.21.15.32:0/2456242516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5577d11b4610 msgr2=0x5577d123eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:43.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f417ae00640 1 --2- 172.21.15.32:0/2456242516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5577d11b4610 0x5577d123eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f417ae00640 1 -- 172.21.15.32:0/2456242516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577d1241270 msgr2=0x5577d123f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:43.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f417ae00640 1 --2- 172.21.15.32:0/2456242516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577d1241270 0x5577d123f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f417ae00640 1 -- 172.21.15.32:0/2456242516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577d11ad920 con 0x5577d11b4230 2024-04-17T09:55:43.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f417ae00640 1 --2- 172.21.15.32:0/2456242516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577d11b4230 0x5577d1224960 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f416c018ec0 tx=0x7f416c018030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:43.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f4173400640 1 -- 172.21.15.32:0/2456242516 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c0af040 con 0x5577d11b4230 2024-04-17T09:55:43.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f4173400640 1 -- 172.21.15.32:0/2456242516 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f416c0c6680 con 0x5577d11b4230 2024-04-17T09:55:43.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f417dc53080 1 -- 172.21.15.32:0/2456242516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5577d123fa20 con 0x5577d11b4230 2024-04-17T09:55:43.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f4173400640 1 -- 172.21.15.32:0/2456242516 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c0c6a50 con 0x5577d11b4230 2024-04-17T09:55:43.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.884+0000 7f417dc53080 1 -- 172.21.15.32:0/2456242516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577d123ff60 con 0x5577d11b4230 2024-04-17T09:55:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.888+0000 7f4173400640 1 -- 172.21.15.32:0/2456242516 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f416c0b4410 con 0x5577d11b4230 2024-04-17T09:55:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.888+0000 7f4173400640 1 --2- 172.21.15.32:0/2456242516 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f415c066d90 0x7f415c069250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.888+0000 7f4173400640 1 -- 172.21.15.32:0/2456242516 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f416c121ba0 con 0x5577d11b4230 2024-04-17T09:55:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.888+0000 7f417a400640 1 --2- 172.21.15.32:0/2456242516 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f415c066d90 0x7f415c069250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.888+0000 7f417dc53080 1 --2- 172.21.15.32:0/2456242516 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x5577d1224ea0 0x5577d12220b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.888+0000 7f417dc53080 1 -- 172.21.15.32:0/2456242516 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46) -- 0x5577d12225f0 con 0x5577d1224ea0 2024-04-17T09:55:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.888+0000 7f417b800640 1 --2- 172.21.15.32:0/2456242516 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x5577d1224ea0 0x5577d12220b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:43.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.888+0000 7f417a400640 1 --2- 172.21.15.32:0/2456242516 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f415c066d90 0x7f415c069250 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4168009a80 tx=0x7f416800c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:43.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.888+0000 7f417b800640 1 --2- 172.21.15.32:0/2456242516 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x5577d1224ea0 0x5577d12220b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:43.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.888+0000 7f417b800640 1 -- 172.21.15.32:0/2456242516 <== osd.6 v2:172.21.15.144:6808/3198738696 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f4174001930 con 0x5577d1224ea0 2024-04-17T09:55:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.888+0000 7f417dc53080 1 -- 172.21.15.32:0/2456242516 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x5577d1224ea0 msgr2=0x5577d12220b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.888+0000 7f417dc53080 1 --2- 172.21.15.32:0/2456242516 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x5577d1224ea0 0x5577d12220b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.888+0000 7f417dc53080 1 -- 172.21.15.32:0/2456242516 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f415c066d90 msgr2=0x7f415c069250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.888+0000 7f417dc53080 1 --2- 172.21.15.32:0/2456242516 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f415c066d90 0x7f415c069250 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4168009a80 tx=0x7f416800c040 comp rx=0 tx=0).stop 2024-04-17T09:55:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.888+0000 7f417dc53080 1 -- 172.21.15.32:0/2456242516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577d11b4230 msgr2=0x5577d1224960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.888+0000 7f417dc53080 1 --2- 172.21.15.32:0/2456242516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5577d11b4230 0x5577d1224960 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f416c018ec0 tx=0x7f416c018030 comp rx=0 tx=0).stop 2024-04-17T09:55:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.888+0000 7f417b800640 1 -- 172.21.15.32:0/2456242516 reap_dead start 2024-04-17T09:55:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.888+0000 7f417dc53080 1 -- 172.21.15.32:0/2456242516 shutdown_connections 2024-04-17T09:55:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.888+0000 7f417dc53080 1 -- 172.21.15.32:0/2456242516 >> 172.21.15.32:0/2456242516 conn(0x5577d116a620 msgr2=0x5577d123b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.888+0000 7f417dc53080 1 -- 172.21.15.32:0/2456242516 shutdown_connections 2024-04-17T09:55:43.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:43.888+0000 7f417dc53080 1 -- 172.21.15.32:0/2456242516 wait complete. 2024-04-17T09:55:43.895 INFO:teuthology.orchestra.run.smithi032.stdout:Yalphaparent_a 2024-04-17T09:55:43.896 DEBUG:teuthology.orchestra.run.smithi032:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-17T09:55:43.940 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:55:43.940 INFO:teuthology.orchestra.run.smithi032.stdout: "ino": 1099511627777, 2024-04-17T09:55:43.940 INFO:teuthology.orchestra.run.smithi032.stdout: "ancestors": [ 2024-04-17T09:55:43.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:43.940 INFO:teuthology.orchestra.run.smithi032.stdout: "dirino": 1099511627776, 2024-04-17T09:55:43.940 INFO:teuthology.orchestra.run.smithi032.stdout: "dname": "alpha", 2024-04-17T09:55:43.940 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 4 2024-04-17T09:55:43.940 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:43.940 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:43.940 INFO:teuthology.orchestra.run.smithi032.stdout: "dirino": 1, 2024-04-17T09:55:43.941 INFO:teuthology.orchestra.run.smithi032.stdout: "dname": "parent_a", 2024-04-17T09:55:43.941 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 7 2024-04-17T09:55:43.941 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:43.941 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-17T09:55:43.941 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 7, 2024-04-17T09:55:43.941 INFO:teuthology.orchestra.run.smithi032.stdout: "old_pools": [] 2024-04-17T09:55:43.941 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:55:43.941 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:43.947 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T09:55:43.947 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T09:55:44.035 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T09:55:44.109 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T09:55:44.109 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T09:55:44.109 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:55:44.109 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:55:44.119 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:55:44.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.296+0000 7fbf6b400640 1 Processor -- start 2024-04-17T09:55:44.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.296+0000 7fbf6b400640 1 -- start start 2024-04-17T09:55:44.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.300+0000 7fbf6b400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbf640cd580 0x7fbf64167560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:44.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.300+0000 7fbf6b400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf640cdac0 con 0x7fbf64145f30 2024-04-17T09:55:44.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.300+0000 7fbf6b400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf6415cdf0 con 0x7fbf64072500 2024-04-17T09:55:44.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.300+0000 7fbf6b400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fbf6406f400 con 0x7fbf640cd580 2024-04-17T09:55:44.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.300+0000 7fbf68a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbf64145f30 0x7fbf64071df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53646/0 (socket says 172.21.15.32:53646) 2024-04-17T09:55:44.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.300+0000 7fbf68a00640 1 -- 172.21.15.32:0/3025039399 learned_addr learned my addr 172.21.15.32:0/3025039399 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:44.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.300+0000 7fbf69400640 1 --2- 172.21.15.32:0/3025039399 >> v2:172.21.15.144:3300/0 conn(0x7fbf640cd580 0x7fbf64167560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:44.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.300+0000 7fbf63400640 1 -- 172.21.15.32:0/3025039399 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 705106404 0 0) 0x7fbf640cdac0 con 0x7fbf64145f30 2024-04-17T09:55:44.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.300+0000 7fbf63400640 1 -- 172.21.15.32:0/3025039399 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf4c003530 con 0x7fbf64145f30 2024-04-17T09:55:44.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.300+0000 7fbf63400640 1 -- 172.21.15.32:0/3025039399 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1286969266 0 0) 0x7fbf6415cdf0 con 0x7fbf64072500 2024-04-17T09:55:44.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.300+0000 7fbf63400640 1 -- 172.21.15.32:0/3025039399 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf640cdac0 con 0x7fbf64072500 2024-04-17T09:55:44.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.300+0000 7fbf63400640 1 -- 172.21.15.32:0/3025039399 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 976946379 0 0) 0x7fbf4c003530 con 0x7fbf64145f30 2024-04-17T09:55:44.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.300+0000 7fbf69400640 1 -- 172.21.15.32:0/3025039399 >> v1:172.21.15.144:6789/0 conn(0x7fbf64072500 legacy=0x7fbf640cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:44.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.300+0000 7fbf69400640 1 -- 172.21.15.32:0/3025039399 >> v1:172.21.15.32:6789/0 conn(0x7fbf64145f30 legacy=0x7fbf64071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:44.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.300+0000 7fbf69400640 1 -- 172.21.15.32:0/3025039399 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6415d0d0 con 0x7fbf640cd580 2024-04-17T09:55:44.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.304+0000 7fbf69400640 1 --2- 172.21.15.32:0/3025039399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf640cd580 0x7fbf64167560 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009d90 tx=0x7fbf5c01f080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad49b4e14ee06ca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:44.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.304+0000 7fbf63400640 1 -- 172.21.15.32:0/3025039399 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c0b5070 con 0x7fbf640cd580 2024-04-17T09:55:44.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.304+0000 7fbf63400640 1 -- 172.21.15.32:0/3025039399 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf5c0a8c10 con 0x7fbf640cd580 2024-04-17T09:55:44.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.304+0000 7fbf63400640 1 -- 172.21.15.32:0/3025039399 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58001680 con 0x7fbf640cd580 2024-04-17T09:55:44.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.304+0000 7fbf6b400640 1 -- 172.21.15.32:0/3025039399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf640cd580 msgr2=0x7fbf64167560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:44.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.304+0000 7fbf6b400640 1 --2- 172.21.15.32:0/3025039399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf640cd580 0x7fbf64167560 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009d90 tx=0x7fbf5c01f080 comp rx=0 tx=0).stop 2024-04-17T09:55:44.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.304+0000 7fbf6b400640 1 -- 172.21.15.32:0/3025039399 shutdown_connections 2024-04-17T09:55:44.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.304+0000 7fbf6b400640 1 --2- 172.21.15.32:0/3025039399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf640cd580 0x7fbf64167560 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:44.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.304+0000 7fbf6b400640 1 -- 172.21.15.32:0/3025039399 >> 172.21.15.32:0/3025039399 conn(0x7fbf6406ba30 msgr2=0x7fbf6406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:44.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.304+0000 7fbf6b400640 1 -- 172.21.15.32:0/3025039399 shutdown_connections 2024-04-17T09:55:44.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.304+0000 7fbf6b400640 1 -- 172.21.15.32:0/3025039399 wait complete. 2024-04-17T09:55:44.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.304+0000 7fbf6b400640 1 Processor -- start 2024-04-17T09:55:44.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.304+0000 7fbf6b400640 1 -- start start 2024-04-17T09:55:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.304+0000 7fbf6b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf64072500 0x7fbf641e7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.304+0000 7fbf6b400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf640cd580 0x7fbf6415bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.304+0000 7fbf6b400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf64145f30 0x7fbf6415c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.304+0000 7fbf6b400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbf64146310 con 0x7fbf64072500 2024-04-17T09:55:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.304+0000 7fbf6b400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6415dba0 con 0x7fbf640cd580 2024-04-17T09:55:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.304+0000 7fbf6b400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6415d440 con 0x7fbf64145f30 2024-04-17T09:55:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.304+0000 7fbf68a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf64072500 0x7fbf641e7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.308+0000 7fbf68a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf64072500 0x7fbf641e7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55970/0 (socket says 172.21.15.32:55970) 2024-04-17T09:55:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.308+0000 7fbf68a00640 1 -- 172.21.15.32:0/382276697 learned_addr learned my addr 172.21.15.32:0/382276697 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.308+0000 7fbf69400640 1 --2- 172.21.15.32:0/382276697 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf64145f30 0x7fbf6415c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.308+0000 7fbf63e00640 1 --2- 172.21.15.32:0/382276697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf640cd580 0x7fbf6415bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.308+0000 7fbf68a00640 1 -- 172.21.15.32:0/382276697 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf64145f30 msgr2=0x7fbf6415c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.308+0000 7fbf68a00640 1 --2- 172.21.15.32:0/382276697 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf64145f30 0x7fbf6415c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.308+0000 7fbf68a00640 1 -- 172.21.15.32:0/382276697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf640cd580 msgr2=0x7fbf6415bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.308+0000 7fbf68a00640 1 --2- 172.21.15.32:0/382276697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf640cd580 0x7fbf6415bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.308+0000 7fbf68a00640 1 -- 172.21.15.32:0/382276697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf641e6e80 con 0x7fbf64072500 2024-04-17T09:55:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.308+0000 7fbf63e00640 1 --2- 172.21.15.32:0/382276697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf640cd580 0x7fbf6415bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:55:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.308+0000 7fbf68a00640 1 --2- 172.21.15.32:0/382276697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf64072500 0x7fbf641e7dd0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800c760 tx=0x7fbf5800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.308+0000 7fbf60c00640 1 -- 172.21.15.32:0/382276697 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58017070 con 0x7fbf64072500 2024-04-17T09:55:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.308+0000 7fbf60c00640 1 -- 172.21.15.32:0/382276697 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf58005ae0 con 0x7fbf64072500 2024-04-17T09:55:44.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.308+0000 7fbf60c00640 1 -- 172.21.15.32:0/382276697 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58007070 con 0x7fbf64072500 2024-04-17T09:55:44.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.308+0000 7fbf6b400640 1 -- 172.21.15.32:0/382276697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf641e7160 con 0x7fbf64072500 2024-04-17T09:55:44.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.308+0000 7fbf6b400640 1 -- 172.21.15.32:0/382276697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf64141a90 con 0x7fbf64072500 2024-04-17T09:55:44.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.308+0000 7fbf6b400640 1 -- 172.21.15.32:0/382276697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf64068a90 con 0x7fbf64072500 2024-04-17T09:55:44.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.308+0000 7fbf60c00640 1 -- 172.21.15.32:0/382276697 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbf5801a070 con 0x7fbf64072500 2024-04-17T09:55:44.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.308+0000 7fbf60c00640 1 --2- 172.21.15.32:0/382276697 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbf44068380 0x7fbf4406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:44.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.308+0000 7fbf60c00640 1 -- 172.21.15.32:0/382276697 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fbf58085ee0 con 0x7fbf64072500 2024-04-17T09:55:44.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.312+0000 7fbf63e00640 1 --2- 172.21.15.32:0/382276697 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbf44068380 0x7fbf4406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:44.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.312+0000 7fbf63e00640 1 --2- 172.21.15.32:0/382276697 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbf44068380 0x7fbf4406a840 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbf640b3540 tx=0x7fbf50009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:44.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.312+0000 7fbf60c00640 1 -- 172.21.15.32:0/382276697 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fbf58056400 con 0x7fbf64072500 2024-04-17T09:55:44.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.476+0000 7fbf6b400640 1 -- 172.21.15.32:0/382276697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf64072f90 con 0x7fbf64072500 2024-04-17T09:55:44.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.480+0000 7fbf60c00640 1 -- 172.21.15.32:0/382276697 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16625 (secure 0 0 0) 0x7fbf58055d00 con 0x7fbf64072500 2024-04-17T09:55:44.481 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:44.481 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":46,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:55:04.834732+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:55:01.624737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-17T09:55:02.185212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:55:44.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.480+0000 7fbf6b400640 1 -- 172.21.15.32:0/382276697 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbf44068380 msgr2=0x7fbf4406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:44.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.480+0000 7fbf6b400640 1 --2- 172.21.15.32:0/382276697 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbf44068380 0x7fbf4406a840 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbf640b3540 tx=0x7fbf50009040 comp rx=0 tx=0).stop 2024-04-17T09:55:44.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.480+0000 7fbf6b400640 1 -- 172.21.15.32:0/382276697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf64072500 msgr2=0x7fbf641e7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:44.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.480+0000 7fbf6b400640 1 --2- 172.21.15.32:0/382276697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf64072500 0x7fbf641e7dd0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800c760 tx=0x7fbf5800cc20 comp rx=0 tx=0).stop 2024-04-17T09:55:44.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.480+0000 7fbf6b400640 1 -- 172.21.15.32:0/382276697 shutdown_connections 2024-04-17T09:55:44.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.480+0000 7fbf6b400640 1 --2- 172.21.15.32:0/382276697 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbf44068380 0x7fbf4406a840 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:44.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.480+0000 7fbf6b400640 1 --2- 172.21.15.32:0/382276697 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf64145f30 0x7fbf6415c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:44.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.480+0000 7fbf6b400640 1 --2- 172.21.15.32:0/382276697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf640cd580 0x7fbf6415bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:44.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.480+0000 7fbf6b400640 1 --2- 172.21.15.32:0/382276697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf64072500 0x7fbf641e7dd0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:44.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.480+0000 7fbf6b400640 1 -- 172.21.15.32:0/382276697 >> 172.21.15.32:0/382276697 conn(0x7fbf6406ba30 msgr2=0x7fbf6415e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:44.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.480+0000 7fbf6b400640 1 -- 172.21.15.32:0/382276697 shutdown_connections 2024-04-17T09:55:44.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.480+0000 7fbf6b400640 1 -- 172.21.15.32:0/382276697 wait complete. 2024-04-17T09:55:44.503 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:55:01.624737+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:55:02.185212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:55:44.504 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:55:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.644+0000 7f8309200640 1 Processor -- start 2024-04-17T09:55:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.644+0000 7f8309200640 1 -- start start 2024-04-17T09:55:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.648+0000 7f8309200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f830406c140 0x7f83040ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.648+0000 7f8309200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83040cc940 con 0x7f830406f7c0 2024-04-17T09:55:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.648+0000 7f8309200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83040ccc20 con 0x7f830415b7d0 2024-04-17T09:55:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.648+0000 7f8309200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f830406e0b0 con 0x7f830406c140 2024-04-17T09:55:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.648+0000 7f82f9400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f830406f7c0 0x7f830406ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53656/0 (socket says 172.21.15.32:53656) 2024-04-17T09:55:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.648+0000 7f82f9400640 1 -- 172.21.15.32:0/3049264451 learned_addr learned my addr 172.21.15.32:0/3049264451 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.648+0000 7f8302800640 1 --2- 172.21.15.32:0/3049264451 >> v2:172.21.15.144:3300/0 conn(0x7f830406c140 0x7f83040ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.648+0000 7f8301400640 1 -- 172.21.15.32:0/3049264451 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3072401150 0 0) 0x7f83040cc940 con 0x7f830406f7c0 2024-04-17T09:55:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.648+0000 7f8301400640 1 -- 172.21.15.32:0/3049264451 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82e8003530 con 0x7f830406f7c0 2024-04-17T09:55:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.648+0000 7f8301400640 1 -- 172.21.15.32:0/3049264451 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 599201157 0 0) 0x7f83040ccc20 con 0x7f830415b7d0 2024-04-17T09:55:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.648+0000 7f8301400640 1 -- 172.21.15.32:0/3049264451 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83040cc940 con 0x7f830415b7d0 2024-04-17T09:55:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.648+0000 7f8301400640 1 -- 172.21.15.32:0/3049264451 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2661968788 0 0) 0x7f82e8003530 con 0x7f830406f7c0 2024-04-17T09:55:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.648+0000 7f8302800640 1 -- 172.21.15.32:0/3049264451 >> v1:172.21.15.144:6789/0 conn(0x7f830415b7d0 legacy=0x7f830406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.648+0000 7f8302800640 1 -- 172.21.15.32:0/3049264451 >> v1:172.21.15.32:6789/0 conn(0x7f830406f7c0 legacy=0x7f830406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.648+0000 7f8302800640 1 -- 172.21.15.32:0/3049264451 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830406e390 con 0x7f830406c140 2024-04-17T09:55:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.652+0000 7f8302800640 1 --2- 172.21.15.32:0/3049264451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f830406c140 0x7f83040ce290 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009d90 tx=0x7f82fc01c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0c2c36ae24f527f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.652+0000 7f8301400640 1 -- 172.21.15.32:0/3049264451 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc0b2070 con 0x7f830406c140 2024-04-17T09:55:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.652+0000 7f8301400640 1 -- 172.21.15.32:0/3049264451 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82fc0a5660 con 0x7f830406c140 2024-04-17T09:55:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.652+0000 7f8301400640 1 -- 172.21.15.32:0/3049264451 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec001680 con 0x7f830406c140 2024-04-17T09:55:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.652+0000 7f8309200640 1 -- 172.21.15.32:0/3049264451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f830406c140 msgr2=0x7f83040ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.652+0000 7f8309200640 1 --2- 172.21.15.32:0/3049264451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f830406c140 0x7f83040ce290 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009d90 tx=0x7f82fc01c800 comp rx=0 tx=0).stop 2024-04-17T09:55:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.652+0000 7f8309200640 1 -- 172.21.15.32:0/3049264451 shutdown_connections 2024-04-17T09:55:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.652+0000 7f8309200640 1 --2- 172.21.15.32:0/3049264451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f830406c140 0x7f83040ce290 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.652+0000 7f8309200640 1 -- 172.21.15.32:0/3049264451 >> 172.21.15.32:0/3049264451 conn(0x7f83040b2120 msgr2=0x7f83040b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.652+0000 7f8309200640 1 -- 172.21.15.32:0/3049264451 shutdown_connections 2024-04-17T09:55:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.652+0000 7f8309200640 1 -- 172.21.15.32:0/3049264451 wait complete. 2024-04-17T09:55:44.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.652+0000 7f8309200640 1 Processor -- start 2024-04-17T09:55:44.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.652+0000 7f8309200640 1 -- start start 2024-04-17T09:55:44.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.652+0000 7f8309200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f830406c140 0x7f830415d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.652+0000 7f8309200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f830406f7c0 0x7f8304151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.652+0000 7f8309200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f830415b7d0 0x7f830414cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.652+0000 7f8309200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8304072270 con 0x7f830406c140 2024-04-17T09:55:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.652+0000 7f8309200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f83040720f0 con 0x7f830415b7d0 2024-04-17T09:55:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.652+0000 7f8309200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f83040723f0 con 0x7f830406f7c0 2024-04-17T09:55:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.652+0000 7f8301e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f830406c140 0x7f830415d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.652+0000 7f8301e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f830406c140 0x7f830415d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55974/0 (socket says 172.21.15.32:55974) 2024-04-17T09:55:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.652+0000 7f8301e00640 1 -- 172.21.15.32:0/1334953743 learned_addr learned my addr 172.21.15.32:0/1334953743 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.656+0000 7f82f9400640 1 --2- 172.21.15.32:0/1334953743 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f830406f7c0 0x7f8304151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.656+0000 7f8302800640 1 --2- 172.21.15.32:0/1334953743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f830415b7d0 0x7f830414cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:44.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.656+0000 7f8301e00640 1 -- 172.21.15.32:0/1334953743 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f830406f7c0 msgr2=0x7f8304151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:44.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.656+0000 7f8301e00640 1 --2- 172.21.15.32:0/1334953743 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f830406f7c0 0x7f8304151720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:44.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.656+0000 7f8301e00640 1 -- 172.21.15.32:0/1334953743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f830415b7d0 msgr2=0x7f830414cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:44.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.656+0000 7f8301e00640 1 --2- 172.21.15.32:0/1334953743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f830415b7d0 0x7f830414cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:44.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.656+0000 7f8301e00640 1 -- 172.21.15.32:0/1334953743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83040ce290 con 0x7f830406c140 2024-04-17T09:55:44.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.656+0000 7f8301e00640 1 --2- 172.21.15.32:0/1334953743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f830406c140 0x7f830415d900 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f82f400c430 tx=0x7f82f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:44.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.656+0000 7f82faa00640 1 -- 172.21.15.32:0/1334953743 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4017070 con 0x7f830406c140 2024-04-17T09:55:44.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.656+0000 7f82faa00640 1 -- 172.21.15.32:0/1334953743 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82f40052f0 con 0x7f830406c140 2024-04-17T09:55:44.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.656+0000 7f82faa00640 1 -- 172.21.15.32:0/1334953743 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4003840 con 0x7f830406c140 2024-04-17T09:55:44.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.656+0000 7f8309200640 1 -- 172.21.15.32:0/1334953743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830415c580 con 0x7f830406c140 2024-04-17T09:55:44.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.656+0000 7f8309200640 1 -- 172.21.15.32:0/1334953743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f830415cb40 con 0x7f830406c140 2024-04-17T09:55:44.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.656+0000 7f8309200640 1 -- 172.21.15.32:0/1334953743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8304068a90 con 0x7f830406c140 2024-04-17T09:55:44.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.656+0000 7f82faa00640 1 -- 172.21.15.32:0/1334953743 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f82f401a070 con 0x7f830406c140 2024-04-17T09:55:44.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.660+0000 7f82faa00640 1 --2- 172.21.15.32:0/1334953743 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f82e0068380 0x7f82e006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:44.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.660+0000 7f82faa00640 1 -- 172.21.15.32:0/1334953743 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f82f4085dc0 con 0x7f830406c140 2024-04-17T09:55:44.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.660+0000 7f82f9400640 1 --2- 172.21.15.32:0/1334953743 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f82e0068380 0x7f82e006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:44.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.660+0000 7f82f9400640 1 --2- 172.21.15.32:0/1334953743 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f82e0068380 0x7f82e006a840 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f83041fae00 tx=0x7f82ec001700 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.664+0000 7f82faa00640 1 -- 172.21.15.32:0/1334953743 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f82f4055c90 con 0x7f830406c140 2024-04-17T09:55:44.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.828+0000 7f8309200640 1 -- 172.21.15.32:0/1334953743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f83040b3590 con 0x7f830406c140 2024-04-17T09:55:44.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.828+0000 7f82faa00640 1 -- 172.21.15.32:0/1334953743 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) ==== 93+0+10 (secure 0 0 0) 0x7f82f4059f90 con 0x7f830406c140 2024-04-17T09:55:44.830 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:55:44.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.828+0000 7f8309200640 1 -- 172.21.15.32:0/1334953743 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f82e0068380 msgr2=0x7f82e006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:44.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.828+0000 7f8309200640 1 --2- 172.21.15.32:0/1334953743 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f82e0068380 0x7f82e006a840 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f83041fae00 tx=0x7f82ec001700 comp rx=0 tx=0).stop 2024-04-17T09:55:44.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.828+0000 7f8309200640 1 -- 172.21.15.32:0/1334953743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f830406c140 msgr2=0x7f830415d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:44.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.828+0000 7f8309200640 1 --2- 172.21.15.32:0/1334953743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f830406c140 0x7f830415d900 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f82f400c430 tx=0x7f82f400c8f0 comp rx=0 tx=0).stop 2024-04-17T09:55:44.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.828+0000 7f8309200640 1 -- 172.21.15.32:0/1334953743 shutdown_connections 2024-04-17T09:55:44.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.828+0000 7f8309200640 1 --2- 172.21.15.32:0/1334953743 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f82e0068380 0x7f82e006a840 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:44.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.828+0000 7f8309200640 1 --2- 172.21.15.32:0/1334953743 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f830406f7c0 0x7f8304151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:44.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.828+0000 7f8309200640 1 --2- 172.21.15.32:0/1334953743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f830415b7d0 0x7f830414cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:44.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.828+0000 7f8309200640 1 --2- 172.21.15.32:0/1334953743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f830406c140 0x7f830415d900 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:44.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.828+0000 7f8309200640 1 -- 172.21.15.32:0/1334953743 >> 172.21.15.32:0/1334953743 conn(0x7f83040b2120 msgr2=0x7f830415a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:44.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.828+0000 7f8309200640 1 -- 172.21.15.32:0/1334953743 shutdown_connections 2024-04-17T09:55:44.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.828+0000 7f8309200640 1 -- 172.21.15.32:0/1334953743 wait complete. 2024-04-17T09:55:44.851 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:55:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.992+0000 7f93f2a00640 1 Processor -- start 2024-04-17T09:55:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.992+0000 7f93f2a00640 1 -- start start 2024-04-17T09:55:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.996+0000 7f93f2a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f93ec06ba30 0x7f93ec06be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.996+0000 7f93f2a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93ec071e60 0x7f93ec072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.996+0000 7f93f2a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93ec06c350 con 0x7f93ec0cf370 2024-04-17T09:55:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.996+0000 7f93f2a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f93ec0cfef0 con 0x7f93ec06ba30 2024-04-17T09:55:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.996+0000 7f93f2a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f93ec0d0130 con 0x7f93ec071e60 2024-04-17T09:55:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.996+0000 7f93e3e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f93ec0cf370 0x7f93ec0cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53662/0 (socket says 172.21.15.32:53662) 2024-04-17T09:55:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.996+0000 7f93e3e00640 1 -- 172.21.15.32:0/3417382729 learned_addr learned my addr 172.21.15.32:0/3417382729 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.996+0000 7f93f0a00640 1 --2- 172.21.15.32:0/3417382729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93ec071e60 0x7f93ec072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.996+0000 7f93ebe00640 1 --2- 172.21.15.32:0/3417382729 >> v2:172.21.15.144:3300/0 conn(0x7f93ec06ba30 0x7f93ec06be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:44.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.996+0000 7f93eb400640 1 -- 172.21.15.32:0/3417382729 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3231303041 0 0) 0x7f93ec06c350 con 0x7f93ec0cf370 2024-04-17T09:55:44.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.996+0000 7f93eb400640 1 -- 172.21.15.32:0/3417382729 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93d0003530 con 0x7f93ec0cf370 2024-04-17T09:55:44.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.996+0000 7f93ebe00640 1 -- 172.21.15.32:0/3417382729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93ec071e60 msgr2=0x7f93ec072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:44.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.996+0000 7f93ebe00640 1 --2- 172.21.15.32:0/3417382729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93ec071e60 0x7f93ec072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:44.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.996+0000 7f93ebe00640 1 -- 172.21.15.32:0/3417382729 >> v1:172.21.15.32:6789/0 conn(0x7f93ec0cf370 legacy=0x7f93ec0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:44.996+0000 7f93ebe00640 1 -- 172.21.15.32:0/3417382729 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ec15d090 con 0x7f93ec06ba30 2024-04-17T09:55:45.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.000+0000 7f93ebe00640 1 --2- 172.21.15.32:0/3417382729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93ec06ba30 0x7f93ec06be10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0097c0 tx=0x7f93dc01c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6309bfad1de89cd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.000+0000 7f93eb400640 1 -- 172.21.15.32:0/3417382729 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc0ab5f0 con 0x7f93ec06ba30 2024-04-17T09:55:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.000+0000 7f93eb400640 1 -- 172.21.15.32:0/3417382729 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93dc0b0070 con 0x7f93ec06ba30 2024-04-17T09:55:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.000+0000 7f93eb400640 1 -- 172.21.15.32:0/3417382729 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc0ab9b0 con 0x7f93ec06ba30 2024-04-17T09:55:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.000+0000 7f93f2a00640 1 -- 172.21.15.32:0/3417382729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93ec06ba30 msgr2=0x7f93ec06be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.000+0000 7f93f2a00640 1 --2- 172.21.15.32:0/3417382729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93ec06ba30 0x7f93ec06be10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0097c0 tx=0x7f93dc01c440 comp rx=0 tx=0).stop 2024-04-17T09:55:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.000+0000 7f93f2a00640 1 -- 172.21.15.32:0/3417382729 shutdown_connections 2024-04-17T09:55:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.000+0000 7f93f2a00640 1 --2- 172.21.15.32:0/3417382729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93ec071e60 0x7f93ec072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.000+0000 7f93f2a00640 1 --2- 172.21.15.32:0/3417382729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93ec06ba30 0x7f93ec06be10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.000+0000 7f93f2a00640 1 -- 172.21.15.32:0/3417382729 >> 172.21.15.32:0/3417382729 conn(0x7f93ec0b2120 msgr2=0x7f93ec0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.000+0000 7f93f2a00640 1 -- 172.21.15.32:0/3417382729 shutdown_connections 2024-04-17T09:55:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.000+0000 7f93f2a00640 1 -- 172.21.15.32:0/3417382729 wait complete. 2024-04-17T09:55:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.000+0000 7f93f2a00640 1 Processor -- start 2024-04-17T09:55:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.000+0000 7f93f2a00640 1 -- start start 2024-04-17T09:55:45.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93f2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec06ba30 0x7f93ec144be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:45.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93f2a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93ec071e60 0x7f93ec142b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:45.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93f2a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93ec0cf370 0x7f93ec159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:45.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93f2a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec15dcf0 con 0x7f93ec06ba30 2024-04-17T09:55:45.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93f2a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec15db70 con 0x7f93ec0cf370 2024-04-17T09:55:45.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93f2a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f93ec146230 con 0x7f93ec071e60 2024-04-17T09:55:45.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93ebe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec06ba30 0x7f93ec144be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:45.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93ebe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec06ba30 0x7f93ec144be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55978/0 (socket says 172.21.15.32:55978) 2024-04-17T09:55:45.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93ebe00640 1 -- 172.21.15.32:0/4267835206 learned_addr learned my addr 172.21.15.32:0/4267835206 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93e3e00640 1 --2- 172.21.15.32:0/4267835206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93ec071e60 0x7f93ec142b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93f0a00640 1 --2- 172.21.15.32:0/4267835206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93ec0cf370 0x7f93ec159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93ebe00640 1 -- 172.21.15.32:0/4267835206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93ec071e60 msgr2=0x7f93ec142b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93ebe00640 1 --2- 172.21.15.32:0/4267835206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93ec071e60 0x7f93ec142b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93ebe00640 1 -- 172.21.15.32:0/4267835206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93ec0cf370 msgr2=0x7f93ec159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93ebe00640 1 --2- 172.21.15.32:0/4267835206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93ec0cf370 0x7f93ec159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93ebe00640 1 -- 172.21.15.32:0/4267835206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ec143380 con 0x7f93ec06ba30 2024-04-17T09:55:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93e3e00640 1 --2- 172.21.15.32:0/4267835206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93ec071e60 0x7f93ec142b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:55:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93f0a00640 1 --2- 172.21.15.32:0/4267835206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93ec0cf370 0x7f93ec159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:55:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93ebe00640 1 --2- 172.21.15.32:0/4267835206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec06ba30 0x7f93ec144be0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0ad040 tx=0x7f93dc01ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93e8c00640 1 -- 172.21.15.32:0/4267835206 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc0bf410 con 0x7f93ec06ba30 2024-04-17T09:55:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93e8c00640 1 -- 172.21.15.32:0/4267835206 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93dc0bf5d0 con 0x7f93ec06ba30 2024-04-17T09:55:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93e8c00640 1 -- 172.21.15.32:0/4267835206 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc0bf9c0 con 0x7f93ec06ba30 2024-04-17T09:55:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93f2a00640 1 -- 172.21.15.32:0/4267835206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93ec143660 con 0x7f93ec06ba30 2024-04-17T09:55:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93f2a00640 1 -- 172.21.15.32:0/4267835206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93ec143c20 con 0x7f93ec06ba30 2024-04-17T09:55:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93f2a00640 1 -- 172.21.15.32:0/4267835206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93ec1430f0 con 0x7f93ec06ba30 2024-04-17T09:55:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.004+0000 7f93e8c00640 1 -- 172.21.15.32:0/4267835206 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f93dc0bd020 con 0x7f93ec06ba30 2024-04-17T09:55:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.008+0000 7f93e8c00640 1 --2- 172.21.15.32:0/4267835206 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f93c8068380 0x7f93c806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.008+0000 7f93e8c00640 1 -- 172.21.15.32:0/4267835206 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f93dc1208f0 con 0x7f93ec06ba30 2024-04-17T09:55:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.008+0000 7f93e3e00640 1 --2- 172.21.15.32:0/4267835206 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f93c8068380 0x7f93c806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.008+0000 7f93e3e00640 1 --2- 172.21.15.32:0/4267835206 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f93c8068380 0x7f93c806a840 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f93ec141af0 tx=0x7f93d4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:45.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.012+0000 7f93e8c00640 1 -- 172.21.15.32:0/4267835206 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f93dc0c6030 con 0x7f93ec06ba30 2024-04-17T09:55:45.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.176+0000 7f93f2a00640 1 -- 172.21.15.32:0/4267835206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f93ec068a90 con 0x7f93ec06ba30 2024-04-17T09:55:45.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.176+0000 7f93e8c00640 1 -- 172.21.15.32:0/4267835206 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) ==== 104+0+11 (secure 0 0 0) 0x7f93dc016090 con 0x7f93ec06ba30 2024-04-17T09:55:45.178 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:55:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.176+0000 7f93f2a00640 1 -- 172.21.15.32:0/4267835206 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f93c8068380 msgr2=0x7f93c806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.176+0000 7f93f2a00640 1 --2- 172.21.15.32:0/4267835206 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f93c8068380 0x7f93c806a840 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f93ec141af0 tx=0x7f93d4009040 comp rx=0 tx=0).stop 2024-04-17T09:55:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.176+0000 7f93f2a00640 1 -- 172.21.15.32:0/4267835206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec06ba30 msgr2=0x7f93ec144be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.176+0000 7f93f2a00640 1 --2- 172.21.15.32:0/4267835206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec06ba30 0x7f93ec144be0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0ad040 tx=0x7f93dc01ac80 comp rx=0 tx=0).stop 2024-04-17T09:55:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.176+0000 7f93f2a00640 1 -- 172.21.15.32:0/4267835206 shutdown_connections 2024-04-17T09:55:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.176+0000 7f93f2a00640 1 --2- 172.21.15.32:0/4267835206 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f93c8068380 0x7f93c806a840 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.176+0000 7f93f2a00640 1 --2- 172.21.15.32:0/4267835206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93ec071e60 0x7f93ec142b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.176+0000 7f93f2a00640 1 --2- 172.21.15.32:0/4267835206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93ec0cf370 0x7f93ec159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.176+0000 7f93f2a00640 1 --2- 172.21.15.32:0/4267835206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ec06ba30 0x7f93ec144be0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.176+0000 7f93f2a00640 1 -- 172.21.15.32:0/4267835206 >> 172.21.15.32:0/4267835206 conn(0x7f93ec0b2120 msgr2=0x7f93ec0b0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.176+0000 7f93f2a00640 1 -- 172.21.15.32:0/4267835206 shutdown_connections 2024-04-17T09:55:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.176+0000 7f93f2a00640 1 -- 172.21.15.32:0/4267835206 wait complete. 2024-04-17T09:55:45.199 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:55:45.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.340+0000 7ff945c00640 1 Processor -- start 2024-04-17T09:55:45.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.340+0000 7ff945c00640 1 -- start start 2024-04-17T09:55:45.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.340+0000 7ff945c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff94006f7c0 0x7ff94006ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:45.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.340+0000 7ff945c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff94006f270 0x7ff94006bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:45.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.340+0000 7ff945c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9400cc940 con 0x7ff94015b7d0 2024-04-17T09:55:45.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.340+0000 7ff945c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff9400ccc20 con 0x7ff94006f7c0 2024-04-17T09:55:45.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.340+0000 7ff945c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff94006e0b0 con 0x7ff94006f270 2024-04-17T09:55:45.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.344+0000 7ff937e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff94006f7c0 0x7ff94006ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:45.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.344+0000 7ff937e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff94006f7c0 0x7ff94006ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55992/0 (socket says 172.21.15.32:55992) 2024-04-17T09:55:45.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.344+0000 7ff937e00640 1 -- 172.21.15.32:0/2756975672 learned_addr learned my addr 172.21.15.32:0/2756975672 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:45.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.344+0000 7ff93f400640 1 --2- 172.21.15.32:0/2756975672 >> v2:172.21.15.144:3300/0 conn(0x7ff94006f270 0x7ff94006bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:45.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.344+0000 7ff937e00640 1 -- 172.21.15.32:0/2756975672 >> v2:172.21.15.144:3300/0 conn(0x7ff94006f270 msgr2=0x7ff94006bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:45.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.344+0000 7ff937e00640 1 --2- 172.21.15.32:0/2756975672 >> v2:172.21.15.144:3300/0 conn(0x7ff94006f270 0x7ff94006bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.344+0000 7ff937e00640 1 -- 172.21.15.32:0/2756975672 >> v1:172.21.15.144:6789/0 conn(0x7ff94015b7d0 legacy=0x7ff94006e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:45.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.344+0000 7ff937e00640 1 -- 172.21.15.32:0/2756975672 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94006e390 con 0x7ff94006f7c0 2024-04-17T09:55:45.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.344+0000 7ff937e00640 1 --2- 172.21.15.32:0/2756975672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94006f7c0 0x7ff94006ba30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff9400b20b0 tx=0x7ff92801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e901db594df26cd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:45.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.344+0000 7ff93e000640 1 -- 172.21.15.32:0/2756975672 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9280aa610 con 0x7ff94006f7c0 2024-04-17T09:55:45.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.344+0000 7ff93e000640 1 -- 172.21.15.32:0/2756975672 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9280af070 con 0x7ff94006f7c0 2024-04-17T09:55:45.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.344+0000 7ff93e000640 1 -- 172.21.15.32:0/2756975672 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9280aa9d0 con 0x7ff94006f7c0 2024-04-17T09:55:45.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff945c00640 1 -- 172.21.15.32:0/2756975672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94006f7c0 msgr2=0x7ff94006ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:45.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff945c00640 1 --2- 172.21.15.32:0/2756975672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94006f7c0 0x7ff94006ba30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff9400b20b0 tx=0x7ff92801c490 comp rx=0 tx=0).stop 2024-04-17T09:55:45.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff945c00640 1 -- 172.21.15.32:0/2756975672 shutdown_connections 2024-04-17T09:55:45.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff945c00640 1 --2- 172.21.15.32:0/2756975672 >> v2:172.21.15.144:3300/0 conn(0x7ff94006f270 0x7ff94006bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff945c00640 1 --2- 172.21.15.32:0/2756975672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94006f7c0 0x7ff94006ba30 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff945c00640 1 -- 172.21.15.32:0/2756975672 >> 172.21.15.32:0/2756975672 conn(0x7ff9400b2120 msgr2=0x7ff9400b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:45.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff945c00640 1 -- 172.21.15.32:0/2756975672 shutdown_connections 2024-04-17T09:55:45.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff945c00640 1 -- 172.21.15.32:0/2756975672 wait complete. 2024-04-17T09:55:45.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff945c00640 1 Processor -- start 2024-04-17T09:55:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff945c00640 1 -- start start 2024-04-17T09:55:45.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff945c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff94006f270 0x7ff940151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:45.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff945c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94006f7c0 0x7ff94013ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:45.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff945c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff94015b7d0 0x7ff9400d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:45.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff945c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff9400cee30 con 0x7ff94006f7c0 2024-04-17T09:55:45.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff945c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff9400cecb0 con 0x7ff94015b7d0 2024-04-17T09:55:45.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff945c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff940071e60 con 0x7ff94006f270 2024-04-17T09:55:45.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff937e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94006f7c0 0x7ff94013ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:45.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff937e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94006f7c0 0x7ff94013ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56006/0 (socket says 172.21.15.32:56006) 2024-04-17T09:55:45.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff937e00640 1 -- 172.21.15.32:0/2988859833 learned_addr learned my addr 172.21.15.32:0/2988859833 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:45.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff93f400640 1 --2- 172.21.15.32:0/2988859833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff94015b7d0 0x7ff9400d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:45.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff93ea00640 1 --2- 172.21.15.32:0/2988859833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff94006f270 0x7ff940151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:45.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff937e00640 1 -- 172.21.15.32:0/2988859833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff94006f270 msgr2=0x7ff940151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:45.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff937e00640 1 --2- 172.21.15.32:0/2988859833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff94006f270 0x7ff940151880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff937e00640 1 -- 172.21.15.32:0/2988859833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff94015b7d0 msgr2=0x7ff9400d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:45.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff937e00640 1 --2- 172.21.15.32:0/2988859833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff94015b7d0 0x7ff9400d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff937e00640 1 -- 172.21.15.32:0/2988859833 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff940168680 con 0x7ff94006f7c0 2024-04-17T09:55:45.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff93ea00640 1 --2- 172.21.15.32:0/2988859833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff94006f270 0x7ff940151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:45.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.348+0000 7ff937e00640 1 --2- 172.21.15.32:0/2988859833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94006f7c0 0x7ff94013ad60 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff9280190a0 tx=0x7ff92801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:45.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.352+0000 7ff936a00640 1 -- 172.21.15.32:0/2988859833 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9280b3800 con 0x7ff94006f7c0 2024-04-17T09:55:45.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.352+0000 7ff936a00640 1 -- 172.21.15.32:0/2988859833 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9280b39a0 con 0x7ff94006f7c0 2024-04-17T09:55:45.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.352+0000 7ff936a00640 1 -- 172.21.15.32:0/2988859833 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9280b3d90 con 0x7ff94006f7c0 2024-04-17T09:55:45.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.352+0000 7ff945c00640 1 -- 172.21.15.32:0/2988859833 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff940168960 con 0x7ff94006f7c0 2024-04-17T09:55:45.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.352+0000 7ff945c00640 1 -- 172.21.15.32:0/2988859833 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff940168f20 con 0x7ff94006f7c0 2024-04-17T09:55:45.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.352+0000 7ff945c00640 1 -- 172.21.15.32:0/2988859833 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff940068a90 con 0x7ff94006f7c0 2024-04-17T09:55:45.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.352+0000 7ff936a00640 1 -- 172.21.15.32:0/2988859833 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff9280b2380 con 0x7ff94006f7c0 2024-04-17T09:55:45.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.352+0000 7ff936a00640 1 --2- 172.21.15.32:0/2988859833 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff91c068380 0x7ff91c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:45.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.352+0000 7ff936a00640 1 -- 172.21.15.32:0/2988859833 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7ff928120e70 con 0x7ff94006f7c0 2024-04-17T09:55:45.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.352+0000 7ff93ea00640 1 --2- 172.21.15.32:0/2988859833 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff91c068380 0x7ff91c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:45.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.356+0000 7ff93ea00640 1 --2- 172.21.15.32:0/2988859833 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff91c068380 0x7ff91c06a840 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff940166260 tx=0x7ff930009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:45.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.356+0000 7ff936a00640 1 -- 172.21.15.32:0/2988859833 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff9280a7070 con 0x7ff94006f7c0 2024-04-17T09:55:45.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.524+0000 7ff945c00640 1 -- 172.21.15.32:0/2988859833 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff9400b26a0 con 0x7ff94006f7c0 2024-04-17T09:55:45.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.524+0000 7ff936a00640 1 -- 172.21.15.32:0/2988859833 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) ==== 100+0+11 (secure 0 0 0) 0x7ff928016110 con 0x7ff94006f7c0 2024-04-17T09:55:45.528 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:55:45.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.528+0000 7ff945c00640 1 -- 172.21.15.32:0/2988859833 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff91c068380 msgr2=0x7ff91c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:45.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.528+0000 7ff945c00640 1 --2- 172.21.15.32:0/2988859833 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff91c068380 0x7ff91c06a840 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff940166260 tx=0x7ff930009040 comp rx=0 tx=0).stop 2024-04-17T09:55:45.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.528+0000 7ff945c00640 1 -- 172.21.15.32:0/2988859833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94006f7c0 msgr2=0x7ff94013ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:45.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.528+0000 7ff945c00640 1 --2- 172.21.15.32:0/2988859833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94006f7c0 0x7ff94013ad60 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff9280190a0 tx=0x7ff92801acd0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.528+0000 7ff945c00640 1 -- 172.21.15.32:0/2988859833 shutdown_connections 2024-04-17T09:55:45.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.528+0000 7ff945c00640 1 --2- 172.21.15.32:0/2988859833 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff91c068380 0x7ff91c06a840 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.528+0000 7ff945c00640 1 --2- 172.21.15.32:0/2988859833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff94015b7d0 0x7ff9400d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.528+0000 7ff945c00640 1 --2- 172.21.15.32:0/2988859833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94006f7c0 0x7ff94013ad60 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.528+0000 7ff945c00640 1 --2- 172.21.15.32:0/2988859833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff94006f270 0x7ff940151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.528+0000 7ff945c00640 1 -- 172.21.15.32:0/2988859833 >> 172.21.15.32:0/2988859833 conn(0x7ff9400b2120 msgr2=0x7ff94015a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:45.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.528+0000 7ff945c00640 1 -- 172.21.15.32:0/2988859833 shutdown_connections 2024-04-17T09:55:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.528+0000 7ff945c00640 1 -- 172.21.15.32:0/2988859833 wait complete. 2024-04-17T09:55:45.549 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T09:55:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.688+0000 7ff0f0400640 1 Processor -- start 2024-04-17T09:55:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.688+0000 7ff0f0400640 1 -- start start 2024-04-17T09:55:45.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.688+0000 7ff0f0400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0e8155a70 0x7ff0e815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:45.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.688+0000 7ff0f0400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0e8155e50 0x7ff0e815d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:45.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.688+0000 7ff0f0400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0e815f870 con 0x7ff0e815d5d0 2024-04-17T09:55:45.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.688+0000 7ff0f0400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff0e80b42a0 con 0x7ff0e8155a70 2024-04-17T09:55:45.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.688+0000 7ff0f0400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff0e80d1250 con 0x7ff0e8155e50 2024-04-17T09:55:45.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.688+0000 7ff0eda00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0e8155a70 0x7ff0e815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:45.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.688+0000 7ff0ee400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff0e815d5d0 0x7ff0e815c260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53666/0 (socket says 172.21.15.32:53666) 2024-04-17T09:55:45.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.688+0000 7ff0ee400640 1 -- 172.21.15.32:0/3126920080 learned_addr learned my addr 172.21.15.32:0/3126920080 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:45.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.688+0000 7ff0eda00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0e8155a70 0x7ff0e815f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56010/0 (socket says 172.21.15.32:56010) 2024-04-17T09:55:45.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.688+0000 7ff0ed000640 1 --2- 172.21.15.32:0/3126920080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0e8155e50 0x7ff0e815d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:45.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.692+0000 7ff0eda00640 1 -- 172.21.15.32:0/3126920080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0e8155e50 msgr2=0x7ff0e815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:45.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.692+0000 7ff0eda00640 1 --2- 172.21.15.32:0/3126920080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0e8155e50 0x7ff0e815d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.692+0000 7ff0eda00640 1 -- 172.21.15.32:0/3126920080 >> v1:172.21.15.32:6789/0 conn(0x7ff0e815d5d0 legacy=0x7ff0e815c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:45.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.692+0000 7ff0eda00640 1 -- 172.21.15.32:0/3126920080 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e815fb50 con 0x7ff0e8155a70 2024-04-17T09:55:45.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.692+0000 7ff0e7e00640 1 -- 172.21.15.32:0/3126920080 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1078079142 0 0) 0x7ff0e815f870 con 0x7ff0e815d5d0 2024-04-17T09:55:45.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.692+0000 7ff0eda00640 1 --2- 172.21.15.32:0/3126920080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0e8155a70 0x7ff0e815f190 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0097c0 tx=0x7ff0dc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4995699ebde21e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:45.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.692+0000 7ff0e7e00640 1 -- 172.21.15.32:0/3126920080 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc0b2070 con 0x7ff0e8155a70 2024-04-17T09:55:45.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.692+0000 7ff0e7e00640 1 -- 172.21.15.32:0/3126920080 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0dc0ad640 con 0x7ff0e8155a70 2024-04-17T09:55:45.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.692+0000 7ff0e7e00640 1 -- 172.21.15.32:0/3126920080 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc0ada10 con 0x7ff0e8155a70 2024-04-17T09:55:45.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.692+0000 7ff0f0400640 1 -- 172.21.15.32:0/3126920080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0e8155a70 msgr2=0x7ff0e815f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:45.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.692+0000 7ff0f0400640 1 --2- 172.21.15.32:0/3126920080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0e8155a70 0x7ff0e815f190 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0097c0 tx=0x7ff0dc01c4a0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.692+0000 7ff0f0400640 1 -- 172.21.15.32:0/3126920080 shutdown_connections 2024-04-17T09:55:45.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.692+0000 7ff0f0400640 1 --2- 172.21.15.32:0/3126920080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0e8155e50 0x7ff0e815d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.692+0000 7ff0f0400640 1 --2- 172.21.15.32:0/3126920080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0e8155a70 0x7ff0e815f190 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.692+0000 7ff0f0400640 1 -- 172.21.15.32:0/3126920080 >> 172.21.15.32:0/3126920080 conn(0x7ff0e806ba30 msgr2=0x7ff0e806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:45.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0f0400640 1 -- 172.21.15.32:0/3126920080 shutdown_connections 2024-04-17T09:55:45.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0f0400640 1 -- 172.21.15.32:0/3126920080 wait complete. 2024-04-17T09:55:45.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0f0400640 1 Processor -- start 2024-04-17T09:55:45.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0f0400640 1 -- start start 2024-04-17T09:55:45.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0f0400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0e8155a70 0x7ff0e80b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:45.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0f0400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0e8155e50 0x7ff0e815a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:45.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0f0400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0e815d5d0 0x7ff0e8158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:45.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0f0400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e80d0170 con 0x7ff0e8155e50 2024-04-17T09:55:45.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0f0400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e80d02f0 con 0x7ff0e815d5d0 2024-04-17T09:55:45.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0f0400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff0e80d0470 con 0x7ff0e8155a70 2024-04-17T09:55:45.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0ed000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0e8155e50 0x7ff0e815a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:45.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0ed000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0e8155e50 0x7ff0e815a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56026/0 (socket says 172.21.15.32:56026) 2024-04-17T09:55:45.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0ed000640 1 -- 172.21.15.32:0/4183393847 learned_addr learned my addr 172.21.15.32:0/4183393847 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:45.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0eda00640 1 --2- 172.21.15.32:0/4183393847 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0e8155a70 0x7ff0e80b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:45.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0ee400640 1 --2- 172.21.15.32:0/4183393847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0e815d5d0 0x7ff0e8158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:45.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0ed000640 1 -- 172.21.15.32:0/4183393847 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0e8155a70 msgr2=0x7ff0e80b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:45.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0ed000640 1 --2- 172.21.15.32:0/4183393847 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0e8155a70 0x7ff0e80b10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0ed000640 1 -- 172.21.15.32:0/4183393847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0e815d5d0 msgr2=0x7ff0e8158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:45.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0ed000640 1 --2- 172.21.15.32:0/4183393847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0e815d5d0 0x7ff0e8158470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0ed000640 1 -- 172.21.15.32:0/4183393847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e80b39f0 con 0x7ff0e8155e50 2024-04-17T09:55:45.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0ee400640 1 --2- 172.21.15.32:0/4183393847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0e815d5d0 0x7ff0e8158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:45.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0ed000640 1 --2- 172.21.15.32:0/4183393847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0e8155e50 0x7ff0e815a660 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400e900 tx=0x7ff0d400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:45.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0e5600640 1 -- 172.21.15.32:0/4183393847 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4017070 con 0x7ff0e8155e50 2024-04-17T09:55:45.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0e5600640 1 -- 172.21.15.32:0/4183393847 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0d400ca00 con 0x7ff0e8155e50 2024-04-17T09:55:45.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.696+0000 7ff0e5600640 1 -- 172.21.15.32:0/4183393847 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400cdf0 con 0x7ff0e8155e50 2024-04-17T09:55:45.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.700+0000 7ff0f0400640 1 -- 172.21.15.32:0/4183393847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0e80b1820 con 0x7ff0e8155e50 2024-04-17T09:55:45.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.700+0000 7ff0f0400640 1 -- 172.21.15.32:0/4183393847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0e81529a0 con 0x7ff0e8155e50 2024-04-17T09:55:45.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.700+0000 7ff0f0400640 1 -- 172.21.15.32:0/4183393847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0e8068a90 con 0x7ff0e8155e50 2024-04-17T09:55:45.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.700+0000 7ff0e5600640 1 -- 172.21.15.32:0/4183393847 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff0d401a070 con 0x7ff0e8155e50 2024-04-17T09:55:45.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.700+0000 7ff0e5600640 1 --2- 172.21.15.32:0/4183393847 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff0c8068380 0x7ff0c806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.700+0000 7ff0e5600640 1 -- 172.21.15.32:0/4183393847 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7ff0d4086960 con 0x7ff0e8155e50 2024-04-17T09:55:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.700+0000 7ff0eda00640 1 --2- 172.21.15.32:0/4183393847 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff0c8068380 0x7ff0c806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.700+0000 7ff0eda00640 1 --2- 172.21.15.32:0/4183393847 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff0c8068380 0x7ff0c806a840 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0af000 tx=0x7ff0dc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.704+0000 7ff0e5600640 1 -- 172.21.15.32:0/4183393847 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff0d4056830 con 0x7ff0e8155e50 2024-04-17T09:55:45.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.880+0000 7ff0f0400640 1 -- 172.21.15.32:0/4183393847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff0e815c5e0 con 0x7ff0e8155e50 2024-04-17T09:55:45.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.880+0000 7ff0e5600640 1 -- 172.21.15.32:0/4183393847 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) ==== 80+0+0 (secure 0 0 0) 0x7ff0d405ab30 con 0x7ff0e8155e50 2024-04-17T09:55:45.883 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T09:55:45.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.884+0000 7ff0f0400640 1 -- 172.21.15.32:0/4183393847 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff0c8068380 msgr2=0x7ff0c806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:45.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.884+0000 7ff0f0400640 1 --2- 172.21.15.32:0/4183393847 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff0c8068380 0x7ff0c806a840 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0af000 tx=0x7ff0dc008040 comp rx=0 tx=0).stop 2024-04-17T09:55:45.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.884+0000 7ff0f0400640 1 -- 172.21.15.32:0/4183393847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0e8155e50 msgr2=0x7ff0e815a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:45.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.884+0000 7ff0f0400640 1 --2- 172.21.15.32:0/4183393847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0e8155e50 0x7ff0e815a660 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400e900 tx=0x7ff0d400edc0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.884+0000 7ff0f0400640 1 -- 172.21.15.32:0/4183393847 shutdown_connections 2024-04-17T09:55:45.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.884+0000 7ff0f0400640 1 --2- 172.21.15.32:0/4183393847 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff0c8068380 0x7ff0c806a840 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.884+0000 7ff0f0400640 1 --2- 172.21.15.32:0/4183393847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0e815d5d0 0x7ff0e8158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.884+0000 7ff0f0400640 1 --2- 172.21.15.32:0/4183393847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0e8155e50 0x7ff0e815a660 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.884+0000 7ff0f0400640 1 --2- 172.21.15.32:0/4183393847 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0e8155a70 0x7ff0e80b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:45.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.884+0000 7ff0f0400640 1 -- 172.21.15.32:0/4183393847 >> 172.21.15.32:0/4183393847 conn(0x7ff0e806ba30 msgr2=0x7ff0e80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:45.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.884+0000 7ff0f0400640 1 -- 172.21.15.32:0/4183393847 shutdown_connections 2024-04-17T09:55:45.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:45.884+0000 7ff0f0400640 1 -- 172.21.15.32:0/4183393847 wait complete. 2024-04-17T09:55:45.905 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2024-04-17T09:55:45.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:55:45.906 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T09:55:45.925 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[38152]: fuse finished with error 0 and tester_r 0 2024-04-17T09:55:45.926 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T09:55:45.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:55:45.926 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T09:55:45.991 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T09:55:45.991 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:55:45.991 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:55:46.000 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T09:55:46.000 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T09:55:46.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:55:46.001 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-17T09:55:46.050 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-17T09:55:46.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:55:46.051 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T09:55:46.052 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:55:46.052 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:55:46.060 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T09:55:46.060 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T09:55:46.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:55:46.060 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-17T09:55:46.110 INFO:teuthology.orchestra.run.smithi032.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-17T09:55:46.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:55:46.112 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T09:55:46.112 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:55:46.112 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:55:46.120 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T09:55:46.120 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T09:55:46.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:55:46.121 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-17T09:55:46.170 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-17T09:55:46.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:55:46.171 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:55:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.308+0000 7fdb9c600640 1 Processor -- start 2024-04-17T09:55:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.308+0000 7fdb9c600640 1 -- start start 2024-04-17T09:55:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.312+0000 7fdb9c600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb94145f30 0x7fdb94071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.312+0000 7fdb9c600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdb940cd580 0x7fdb94072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.312+0000 7fdb9c600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb940cdac0 con 0x7fdb9415d8d0 2024-04-17T09:55:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.312+0000 7fdb9c600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fdb9406f400 con 0x7fdb94145f30 2024-04-17T09:55:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.312+0000 7fdb9c600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fdb94146310 con 0x7fdb940cd580 2024-04-17T09:55:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.312+0000 7fdb99c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb94145f30 0x7fdb94071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.312+0000 7fdb99c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb94145f30 0x7fdb94071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56040/0 (socket says 172.21.15.32:56040) 2024-04-17T09:55:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.312+0000 7fdb99c00640 1 -- 172.21.15.32:0/3060843932 learned_addr learned my addr 172.21.15.32:0/3060843932 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.312+0000 7fdb9a600640 1 --2- 172.21.15.32:0/3060843932 >> v2:172.21.15.144:3300/0 conn(0x7fdb940cd580 0x7fdb94072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.312+0000 7fdb99c00640 1 -- 172.21.15.32:0/3060843932 >> v2:172.21.15.144:3300/0 conn(0x7fdb940cd580 msgr2=0x7fdb94072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.312+0000 7fdb99c00640 1 --2- 172.21.15.32:0/3060843932 >> v2:172.21.15.144:3300/0 conn(0x7fdb940cd580 0x7fdb94072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:46.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.312+0000 7fdb99c00640 1 -- 172.21.15.32:0/3060843932 >> v1:172.21.15.144:6789/0 conn(0x7fdb9415d8d0 legacy=0x7fdb940cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:46.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.312+0000 7fdb99c00640 1 -- 172.21.15.32:0/3060843932 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9415cdf0 con 0x7fdb94145f30 2024-04-17T09:55:46.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.312+0000 7fdb9a600640 1 --2- 172.21.15.32:0/3060843932 >> v2:172.21.15.144:3300/0 conn(0x7fdb940cd580 0x7fdb94072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:55:46.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.316+0000 7fdb99c00640 1 --2- 172.21.15.32:0/3060843932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb94145f30 0x7fdb94071df0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fdb9415dd30 tx=0x7fdb8801ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c946318519be634 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:46.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.316+0000 7fdb93e00640 1 -- 172.21.15.32:0/3060843932 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb880b5070 con 0x7fdb94145f30 2024-04-17T09:55:46.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.316+0000 7fdb93e00640 1 -- 172.21.15.32:0/3060843932 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb880a89d0 con 0x7fdb94145f30 2024-04-17T09:55:46.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.316+0000 7fdb93e00640 1 -- 172.21.15.32:0/3060843932 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb880a8da0 con 0x7fdb94145f30 2024-04-17T09:55:46.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.316+0000 7fdb9c600640 1 -- 172.21.15.32:0/3060843932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb94145f30 msgr2=0x7fdb94071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:46.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.316+0000 7fdb9c600640 1 --2- 172.21.15.32:0/3060843932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb94145f30 0x7fdb94071df0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fdb9415dd30 tx=0x7fdb8801ecd0 comp rx=0 tx=0).stop 2024-04-17T09:55:46.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.316+0000 7fdb9c600640 1 -- 172.21.15.32:0/3060843932 shutdown_connections 2024-04-17T09:55:46.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.316+0000 7fdb9c600640 1 --2- 172.21.15.32:0/3060843932 >> v2:172.21.15.144:3300/0 conn(0x7fdb940cd580 0x7fdb94072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:46.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.316+0000 7fdb9c600640 1 --2- 172.21.15.32:0/3060843932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb94145f30 0x7fdb94071df0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:46.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.316+0000 7fdb9c600640 1 -- 172.21.15.32:0/3060843932 >> 172.21.15.32:0/3060843932 conn(0x7fdb9406ba30 msgr2=0x7fdb9406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:46.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.316+0000 7fdb9c600640 1 -- 172.21.15.32:0/3060843932 shutdown_connections 2024-04-17T09:55:46.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.316+0000 7fdb9c600640 1 -- 172.21.15.32:0/3060843932 wait complete. 2024-04-17T09:55:46.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.316+0000 7fdb9c600640 1 Processor -- start 2024-04-17T09:55:46.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.316+0000 7fdb9c600640 1 -- start start 2024-04-17T09:55:46.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.316+0000 7fdb9c600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb940cd580 0x7fdb9415c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:46.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.316+0000 7fdb9c600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb94145f30 0x7fdb941e7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:46.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.316+0000 7fdb9c600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9415d8d0 0x7fdb940cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:46.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.316+0000 7fdb9c600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9415cb90 con 0x7fdb9415d8d0 2024-04-17T09:55:46.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.316+0000 7fdb9c600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9415d440 con 0x7fdb94145f30 2024-04-17T09:55:46.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.316+0000 7fdb9c600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdb941601d0 con 0x7fdb940cd580 2024-04-17T09:55:46.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.316+0000 7fdb9a600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9415d8d0 0x7fdb940cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:46.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.316+0000 7fdb9a600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9415d8d0 0x7fdb940cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56046/0 (socket says 172.21.15.32:56046) 2024-04-17T09:55:46.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.316+0000 7fdb9a600640 1 -- 172.21.15.32:0/200773716 learned_addr learned my addr 172.21.15.32:0/200773716 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:46.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.320+0000 7fdb99c00640 1 --2- 172.21.15.32:0/200773716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb940cd580 0x7fdb9415c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:46.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.320+0000 7fdb99200640 1 --2- 172.21.15.32:0/200773716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb94145f30 0x7fdb941e7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:46.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.320+0000 7fdb9a600640 1 -- 172.21.15.32:0/200773716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb940cd580 msgr2=0x7fdb9415c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:46.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.320+0000 7fdb9a600640 1 --2- 172.21.15.32:0/200773716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb940cd580 0x7fdb9415c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:46.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.320+0000 7fdb9a600640 1 -- 172.21.15.32:0/200773716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb94145f30 msgr2=0x7fdb941e7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:46.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.320+0000 7fdb9a600640 1 --2- 172.21.15.32:0/200773716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb94145f30 0x7fdb941e7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:46.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.320+0000 7fdb9a600640 1 -- 172.21.15.32:0/200773716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9415c6b0 con 0x7fdb9415d8d0 2024-04-17T09:55:46.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.320+0000 7fdb9a600640 1 --2- 172.21.15.32:0/200773716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9415d8d0 0x7fdb940cb820 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00cef0 tx=0x7fdb8c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:46.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.320+0000 7fdb91600640 1 -- 172.21.15.32:0/200773716 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c01a070 con 0x7fdb9415d8d0 2024-04-17T09:55:46.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.320+0000 7fdb91600640 1 -- 172.21.15.32:0/200773716 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb8c00d860 con 0x7fdb9415d8d0 2024-04-17T09:55:46.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.320+0000 7fdb91600640 1 -- 172.21.15.32:0/200773716 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c00dc50 con 0x7fdb9415d8d0 2024-04-17T09:55:46.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.320+0000 7fdb9c600640 1 -- 172.21.15.32:0/200773716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb941e61d0 con 0x7fdb9415d8d0 2024-04-17T09:55:46.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.320+0000 7fdb9c600640 1 -- 172.21.15.32:0/200773716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb941e6790 con 0x7fdb9415d8d0 2024-04-17T09:55:46.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.320+0000 7fdb9c600640 1 -- 172.21.15.32:0/200773716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb94068a90 con 0x7fdb9415d8d0 2024-04-17T09:55:46.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.320+0000 7fdb91600640 1 -- 172.21.15.32:0/200773716 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdb8c01d070 con 0x7fdb9415d8d0 2024-04-17T09:55:46.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.320+0000 7fdb91600640 1 --2- 172.21.15.32:0/200773716 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdb740683b0 0x7fdb7406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:46.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.320+0000 7fdb91600640 1 -- 172.21.15.32:0/200773716 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fdb8c087290 con 0x7fdb9415d8d0 2024-04-17T09:55:46.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.324+0000 7fdb99c00640 1 --2- 172.21.15.32:0/200773716 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdb740683b0 0x7fdb7406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:46.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.324+0000 7fdb99c00640 1 --2- 172.21.15.32:0/200773716 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdb740683b0 0x7fdb7406a870 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdb880b2000 tx=0x7fdb88009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:46.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.328+0000 7fdb91600640 1 -- 172.21.15.32:0/200773716 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fdb8c057190 con 0x7fdb9415d8d0 2024-04-17T09:55:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.524+0000 7fdb9c600640 1 -- 172.21.15.32:0/200773716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb94072f90 con 0x7fdb9415d8d0 2024-04-17T09:55:46.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.524+0000 7fdb91600640 1 -- 172.21.15.32:0/200773716 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5202 (secure 0 0 0) 0x7fdb8c05b490 con 0x7fdb9415d8d0 2024-04-17T09:55:46.526 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:46.526 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2542578926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2542578926},{"type":"v1","addr":"172.21.15.144:6834","nonce":2542578926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1424948520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1424948520},{"type":"v1","addr":"172.21.15.32:6836","nonce":1424948520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/393813154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":393813154},{"type":"v1","addr":"172.21.15.32:6837","nonce":393813154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:55:03.233552+0000","modified":"2024-04-17T09:55:06.243321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5742":{"gid":5742,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/2177384675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2177384675},{"type":"v1","addr":"172.21.15.144:6835","nonce":2177384675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742]},"id":3}]} 2024-04-17T09:55:46.526 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2024-04-17T09:55:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.524+0000 7fdb9c600640 1 -- 172.21.15.32:0/200773716 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdb740683b0 msgr2=0x7fdb7406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.524+0000 7fdb9c600640 1 --2- 172.21.15.32:0/200773716 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdb740683b0 0x7fdb7406a870 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdb880b2000 tx=0x7fdb88009040 comp rx=0 tx=0).stop 2024-04-17T09:55:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.524+0000 7fdb9c600640 1 -- 172.21.15.32:0/200773716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9415d8d0 msgr2=0x7fdb940cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.524+0000 7fdb9c600640 1 --2- 172.21.15.32:0/200773716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9415d8d0 0x7fdb940cb820 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00cef0 tx=0x7fdb8c00a710 comp rx=0 tx=0).stop 2024-04-17T09:55:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.528+0000 7fdb9c600640 1 -- 172.21.15.32:0/200773716 shutdown_connections 2024-04-17T09:55:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.528+0000 7fdb9c600640 1 --2- 172.21.15.32:0/200773716 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdb740683b0 0x7fdb7406a870 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.528+0000 7fdb9c600640 1 --2- 172.21.15.32:0/200773716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9415d8d0 0x7fdb940cb820 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.528+0000 7fdb9c600640 1 --2- 172.21.15.32:0/200773716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb94145f30 0x7fdb941e7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.528+0000 7fdb9c600640 1 --2- 172.21.15.32:0/200773716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb940cd580 0x7fdb9415c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.528+0000 7fdb9c600640 1 -- 172.21.15.32:0/200773716 >> 172.21.15.32:0/200773716 conn(0x7fdb9406ba30 msgr2=0x7fdb940b3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.528+0000 7fdb9c600640 1 -- 172.21.15.32:0/200773716 shutdown_connections 2024-04-17T09:55:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.528+0000 7fdb9c600640 1 -- 172.21.15.32:0/200773716 wait complete. 2024-04-17T09:55:46.548 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:55:46.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.688+0000 7fda13e00640 1 Processor -- start 2024-04-17T09:55:46.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.688+0000 7fda13e00640 1 -- start start 2024-04-17T09:55:46.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.688+0000 7fda13e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fda0c145f30 0x7fda0c071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:46.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.688+0000 7fda13e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fda0c0cd580 0x7fda0c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:46.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.688+0000 7fda13e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda0c0cdac0 con 0x7fda0c15d8d0 2024-04-17T09:55:46.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.688+0000 7fda13e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fda0c06f400 con 0x7fda0c145f30 2024-04-17T09:55:46.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.688+0000 7fda13e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fda0c146310 con 0x7fda0c0cd580 2024-04-17T09:55:46.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.688+0000 7fda11400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fda0c145f30 0x7fda0c071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:46.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.688+0000 7fda11400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fda0c145f30 0x7fda0c071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56052/0 (socket says 172.21.15.32:56052) 2024-04-17T09:55:46.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.688+0000 7fda11400640 1 -- 172.21.15.32:0/3627930919 learned_addr learned my addr 172.21.15.32:0/3627930919 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:46.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.688+0000 7fda11e00640 1 --2- 172.21.15.32:0/3627930919 >> v2:172.21.15.144:3300/0 conn(0x7fda0c0cd580 0x7fda0c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:46.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.688+0000 7fda11400640 1 -- 172.21.15.32:0/3627930919 >> v2:172.21.15.144:3300/0 conn(0x7fda0c0cd580 msgr2=0x7fda0c072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:46.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.688+0000 7fda11400640 1 --2- 172.21.15.32:0/3627930919 >> v2:172.21.15.144:3300/0 conn(0x7fda0c0cd580 0x7fda0c072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:46.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.688+0000 7fda11400640 1 -- 172.21.15.32:0/3627930919 >> v1:172.21.15.144:6789/0 conn(0x7fda0c15d8d0 legacy=0x7fda0c0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:46.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.688+0000 7fda11400640 1 -- 172.21.15.32:0/3627930919 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0c15cdf0 con 0x7fda0c145f30 2024-04-17T09:55:46.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.692+0000 7fda11400640 1 --2- 172.21.15.32:0/3627930919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda0c145f30 0x7fda0c071df0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fda0c15dd30 tx=0x7fda0001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29806e11b773736d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:46.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.692+0000 7fda0be00640 1 -- 172.21.15.32:0/3627930919 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda000b5070 con 0x7fda0c145f30 2024-04-17T09:55:46.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.692+0000 7fda0be00640 1 -- 172.21.15.32:0/3627930919 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda000a8a10 con 0x7fda0c145f30 2024-04-17T09:55:46.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.692+0000 7fda0be00640 1 -- 172.21.15.32:0/3627930919 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda000a8de0 con 0x7fda0c145f30 2024-04-17T09:55:46.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.692+0000 7fda13e00640 1 -- 172.21.15.32:0/3627930919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda0c145f30 msgr2=0x7fda0c071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:46.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.692+0000 7fda13e00640 1 --2- 172.21.15.32:0/3627930919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda0c145f30 0x7fda0c071df0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fda0c15dd30 tx=0x7fda0001ed10 comp rx=0 tx=0).stop 2024-04-17T09:55:46.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.692+0000 7fda13e00640 1 -- 172.21.15.32:0/3627930919 shutdown_connections 2024-04-17T09:55:46.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.692+0000 7fda13e00640 1 --2- 172.21.15.32:0/3627930919 >> v2:172.21.15.144:3300/0 conn(0x7fda0c0cd580 0x7fda0c072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:46.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.692+0000 7fda13e00640 1 --2- 172.21.15.32:0/3627930919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda0c145f30 0x7fda0c071df0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:46.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.692+0000 7fda13e00640 1 -- 172.21.15.32:0/3627930919 >> 172.21.15.32:0/3627930919 conn(0x7fda0c06ba30 msgr2=0x7fda0c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:46.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.692+0000 7fda13e00640 1 -- 172.21.15.32:0/3627930919 shutdown_connections 2024-04-17T09:55:46.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.692+0000 7fda13e00640 1 -- 172.21.15.32:0/3627930919 wait complete. 2024-04-17T09:55:46.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.692+0000 7fda13e00640 1 Processor -- start 2024-04-17T09:55:46.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda13e00640 1 -- start start 2024-04-17T09:55:46.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda13e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda0c0cd580 0x7fda0c14be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda13e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda0c145f30 0x7fda0c142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda13e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda0c15d8d0 0x7fda0c14e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda13e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c15cb90 con 0x7fda0c0cd580 2024-04-17T09:55:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda13e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c15d440 con 0x7fda0c145f30 2024-04-17T09:55:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda13e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fda0c1601d0 con 0x7fda0c15d8d0 2024-04-17T09:55:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda11400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda0c0cd580 0x7fda0c14be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda11400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda0c0cd580 0x7fda0c14be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56062/0 (socket says 172.21.15.32:56062) 2024-04-17T09:55:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda11400640 1 -- 172.21.15.32:0/1532329372 learned_addr learned my addr 172.21.15.32:0/1532329372 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda11e00640 1 --2- 172.21.15.32:0/1532329372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda0c15d8d0 0x7fda0c14e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda10a00640 1 --2- 172.21.15.32:0/1532329372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda0c145f30 0x7fda0c142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda11400640 1 -- 172.21.15.32:0/1532329372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda0c15d8d0 msgr2=0x7fda0c14e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda11400640 1 --2- 172.21.15.32:0/1532329372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda0c15d8d0 0x7fda0c14e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda11400640 1 -- 172.21.15.32:0/1532329372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda0c145f30 msgr2=0x7fda0c142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda11400640 1 --2- 172.21.15.32:0/1532329372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda0c145f30 0x7fda0c142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda11400640 1 -- 172.21.15.32:0/1532329372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0c0cae80 con 0x7fda0c0cd580 2024-04-17T09:55:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda10a00640 1 --2- 172.21.15.32:0/1532329372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda0c145f30 0x7fda0c142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:55:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda11400640 1 --2- 172.21.15.32:0/1532329372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda0c0cd580 0x7fda0c14be80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fda0001b920 tx=0x7fda0001bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda09600640 1 -- 172.21.15.32:0/1532329372 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda000b5070 con 0x7fda0c0cd580 2024-04-17T09:55:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda09600640 1 -- 172.21.15.32:0/1532329372 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda000b0b40 con 0x7fda0c0cd580 2024-04-17T09:55:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda09600640 1 -- 172.21.15.32:0/1532329372 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda000bb410 con 0x7fda0c0cd580 2024-04-17T09:55:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda13e00640 1 -- 172.21.15.32:0/1532329372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda0c0cb120 con 0x7fda0c0cd580 2024-04-17T09:55:46.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda13e00640 1 -- 172.21.15.32:0/1532329372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda0c0cb6e0 con 0x7fda0c0cd580 2024-04-17T09:55:46.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.696+0000 7fda13e00640 1 -- 172.21.15.32:0/1532329372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda0c068a90 con 0x7fda0c0cd580 2024-04-17T09:55:46.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.700+0000 7fda09600640 1 -- 172.21.15.32:0/1532329372 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fda000c2060 con 0x7fda0c0cd580 2024-04-17T09:55:46.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.700+0000 7fda09600640 1 --2- 172.21.15.32:0/1532329372 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd9ec068380 0x7fd9ec06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:46.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.700+0000 7fda09600640 1 -- 172.21.15.32:0/1532329372 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fda00123c00 con 0x7fda0c0cd580 2024-04-17T09:55:46.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.700+0000 7fda10a00640 1 --2- 172.21.15.32:0/1532329372 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd9ec068380 0x7fd9ec06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.700+0000 7fda10a00640 1 --2- 172.21.15.32:0/1532329372 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd9ec068380 0x7fd9ec06a840 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fda0c1f5b90 tx=0x7fd9f8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:46.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.704+0000 7fda09600640 1 -- 172.21.15.32:0/1532329372 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fda00018870 con 0x7fda0c0cd580 2024-04-17T09:55:46.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.868+0000 7fda13e00640 1 -- 172.21.15.32:0/1532329372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda0c0cd080 con 0x7fda0c0cd580 2024-04-17T09:55:46.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.868+0000 7fda09600640 1 -- 172.21.15.32:0/1532329372 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16625 (secure 0 0 0) 0x7fda000f4120 con 0x7fda0c0cd580 2024-04-17T09:55:46.871 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:46.871 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":46,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:55:04.834732+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:55:01.624737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-17T09:55:02.185212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:55:46.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.872+0000 7fda13e00640 1 -- 172.21.15.32:0/1532329372 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd9ec068380 msgr2=0x7fd9ec06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:46.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.872+0000 7fda13e00640 1 --2- 172.21.15.32:0/1532329372 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd9ec068380 0x7fd9ec06a840 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fda0c1f5b90 tx=0x7fd9f8009040 comp rx=0 tx=0).stop 2024-04-17T09:55:46.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.872+0000 7fda13e00640 1 -- 172.21.15.32:0/1532329372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda0c0cd580 msgr2=0x7fda0c14be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:46.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.872+0000 7fda13e00640 1 --2- 172.21.15.32:0/1532329372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda0c0cd580 0x7fda0c14be80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fda0001b920 tx=0x7fda0001bab0 comp rx=0 tx=0).stop 2024-04-17T09:55:46.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.872+0000 7fda13e00640 1 -- 172.21.15.32:0/1532329372 shutdown_connections 2024-04-17T09:55:46.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.872+0000 7fda13e00640 1 --2- 172.21.15.32:0/1532329372 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd9ec068380 0x7fd9ec06a840 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:46.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.872+0000 7fda13e00640 1 --2- 172.21.15.32:0/1532329372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda0c15d8d0 0x7fda0c14e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:46.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.872+0000 7fda13e00640 1 --2- 172.21.15.32:0/1532329372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda0c145f30 0x7fda0c142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:46.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.872+0000 7fda13e00640 1 --2- 172.21.15.32:0/1532329372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda0c0cd580 0x7fda0c14be80 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:46.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.872+0000 7fda13e00640 1 -- 172.21.15.32:0/1532329372 >> 172.21.15.32:0/1532329372 conn(0x7fda0c06ba30 msgr2=0x7fda0c1605a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:46.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.872+0000 7fda13e00640 1 -- 172.21.15.32:0/1532329372 shutdown_connections 2024-04-17T09:55:46.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:46.872+0000 7fda13e00640 1 -- 172.21.15.32:0/1532329372 wait complete. 2024-04-17T09:55:46.892 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:55:01.624737+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:55:02.185212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:55:46.893 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:55:47.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.036+0000 7fa08a600640 1 Processor -- start 2024-04-17T09:55:47.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.036+0000 7fa08a600640 1 -- start start 2024-04-17T09:55:47.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.036+0000 7fa08a600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0841662c0 0x7fa0840b4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.036+0000 7fa08a600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa08406eb60 0x7fa08406f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.036+0000 7fa08a600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa08406ba30 0x7fa08406be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.036+0000 7fa08a600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa08406c3a0 con 0x7fa08406ba30 2024-04-17T09:55:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.036+0000 7fa08a600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa08406f290 con 0x7fa08406eb60 2024-04-17T09:55:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.036+0000 7fa08a600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa08406efc0 con 0x7fa0841662c0 2024-04-17T09:55:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.036+0000 7fa083e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa08406ba30 0x7fa08406be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.036+0000 7fa083e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa08406ba30 0x7fa08406be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56066/0 (socket says 172.21.15.32:56066) 2024-04-17T09:55:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.036+0000 7fa083e00640 1 -- 172.21.15.32:0/621876849 learned_addr learned my addr 172.21.15.32:0/621876849 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.036+0000 7fa07aa00640 1 --2- 172.21.15.32:0/621876849 >> v2:172.21.15.144:3300/0 conn(0x7fa08406eb60 0x7fa08406f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.036+0000 7fa083400640 1 --2- 172.21.15.32:0/621876849 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0841662c0 0x7fa0840b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.036+0000 7fa083e00640 1 -- 172.21.15.32:0/621876849 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0841662c0 msgr2=0x7fa0840b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.036+0000 7fa083e00640 1 --2- 172.21.15.32:0/621876849 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0841662c0 0x7fa0840b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.036+0000 7fa083e00640 1 -- 172.21.15.32:0/621876849 >> v2:172.21.15.144:3300/0 conn(0x7fa08406eb60 msgr2=0x7fa08406f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.036+0000 7fa083e00640 1 --2- 172.21.15.32:0/621876849 >> v2:172.21.15.144:3300/0 conn(0x7fa08406eb60 0x7fa08406f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.036+0000 7fa083e00640 1 -- 172.21.15.32:0/621876849 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0841666a0 con 0x7fa08406ba30 2024-04-17T09:55:47.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.040+0000 7fa083e00640 1 --2- 172.21.15.32:0/621876849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08406ba30 0x7fa08406be60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa08406ffd0 tx=0x7fa07c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c30b2fef1b74e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:47.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.040+0000 7fa082a00640 1 -- 172.21.15.32:0/621876849 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c0a5760 con 0x7fa08406ba30 2024-04-17T09:55:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.040+0000 7fa082a00640 1 -- 172.21.15.32:0/621876849 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa07c0a5900 con 0x7fa08406ba30 2024-04-17T09:55:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.040+0000 7fa082a00640 1 -- 172.21.15.32:0/621876849 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c0a5cd0 con 0x7fa08406ba30 2024-04-17T09:55:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.040+0000 7fa08a600640 1 -- 172.21.15.32:0/621876849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08406ba30 msgr2=0x7fa08406be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.040+0000 7fa08a600640 1 --2- 172.21.15.32:0/621876849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08406ba30 0x7fa08406be60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa08406ffd0 tx=0x7fa07c01c9f0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.040+0000 7fa08a600640 1 -- 172.21.15.32:0/621876849 shutdown_connections 2024-04-17T09:55:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.040+0000 7fa08a600640 1 --2- 172.21.15.32:0/621876849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa08406ba30 0x7fa08406be60 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.040+0000 7fa08a600640 1 --2- 172.21.15.32:0/621876849 >> v2:172.21.15.144:3300/0 conn(0x7fa08406eb60 0x7fa08406f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.040+0000 7fa08a600640 1 --2- 172.21.15.32:0/621876849 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0841662c0 0x7fa0840b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.040+0000 7fa08a600640 1 -- 172.21.15.32:0/621876849 >> 172.21.15.32:0/621876849 conn(0x7fa0840b2120 msgr2=0x7fa0840b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.040+0000 7fa08a600640 1 -- 172.21.15.32:0/621876849 shutdown_connections 2024-04-17T09:55:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.040+0000 7fa08a600640 1 -- 172.21.15.32:0/621876849 wait complete. 2024-04-17T09:55:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa08a600640 1 Processor -- start 2024-04-17T09:55:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa08a600640 1 -- start start 2024-04-17T09:55:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa08a600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa08406ba30 0x7fa08415b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa08a600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa08406eb60 0x7fa08414e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa08a600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0841662c0 0x7fa084157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa08a600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa084072670 con 0x7fa0841662c0 2024-04-17T09:55:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa08a600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa0840b3d60 con 0x7fa08406eb60 2024-04-17T09:55:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa08a600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa0840727f0 con 0x7fa08406ba30 2024-04-17T09:55:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa083e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0841662c0 0x7fa084157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa083e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0841662c0 0x7fa084157fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56082/0 (socket says 172.21.15.32:56082) 2024-04-17T09:55:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa083e00640 1 -- 172.21.15.32:0/2836063745 learned_addr learned my addr 172.21.15.32:0/2836063745 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa07aa00640 1 --2- 172.21.15.32:0/2836063745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa08406eb60 0x7fa08414e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa083400640 1 --2- 172.21.15.32:0/2836063745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa08406ba30 0x7fa08415b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa083e00640 1 -- 172.21.15.32:0/2836063745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa08406ba30 msgr2=0x7fa08415b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:47.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa083e00640 1 --2- 172.21.15.32:0/2836063745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa08406ba30 0x7fa08415b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa083e00640 1 -- 172.21.15.32:0/2836063745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa08406eb60 msgr2=0x7fa08414e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:47.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa083e00640 1 --2- 172.21.15.32:0/2836063745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa08406eb60 0x7fa08414e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa083e00640 1 -- 172.21.15.32:0/2836063745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08415eac0 con 0x7fa0841662c0 2024-04-17T09:55:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa083e00640 1 --2- 172.21.15.32:0/2836063745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0841662c0 0x7fa084157fc0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa07c0af040 tx=0x7fa07c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa07be00640 1 -- 172.21.15.32:0/2836063745 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c0bc020 con 0x7fa0841662c0 2024-04-17T09:55:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa07be00640 1 -- 172.21.15.32:0/2836063745 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa07c0b8670 con 0x7fa0841662c0 2024-04-17T09:55:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa08a600640 1 -- 172.21.15.32:0/2836063745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08415eda0 con 0x7fa0841662c0 2024-04-17T09:55:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa07be00640 1 -- 172.21.15.32:0/2836063745 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c0b8a60 con 0x7fa0841662c0 2024-04-17T09:55:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa08a600640 1 -- 172.21.15.32:0/2836063745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa08415f360 con 0x7fa0841662c0 2024-04-17T09:55:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.044+0000 7fa08a600640 1 -- 172.21.15.32:0/2836063745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa084068a90 con 0x7fa0841662c0 2024-04-17T09:55:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.048+0000 7fa07be00640 1 -- 172.21.15.32:0/2836063745 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa07c0bf020 con 0x7fa0841662c0 2024-04-17T09:55:47.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.048+0000 7fa07be00640 1 --2- 172.21.15.32:0/2836063745 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa0600683b0 0x7fa06006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:47.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.048+0000 7fa07be00640 1 -- 172.21.15.32:0/2836063745 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fa07c121850 con 0x7fa0841662c0 2024-04-17T09:55:47.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.048+0000 7fa083400640 1 --2- 172.21.15.32:0/2836063745 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa0600683b0 0x7fa06006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:47.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.048+0000 7fa083400640 1 --2- 172.21.15.32:0/2836063745 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa0600683b0 0x7fa06006a870 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa0740079c0 tx=0x7fa074008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.052+0000 7fa07be00640 1 -- 172.21.15.32:0/2836063745 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa07c0165e0 con 0x7fa0841662c0 2024-04-17T09:55:47.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.216+0000 7fa08a600640 1 -- 172.21.15.32:0/2836063745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa0840cca80 con 0x7fa0841662c0 2024-04-17T09:55:47.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.216+0000 7fa07be00640 1 -- 172.21.15.32:0/2836063745 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) ==== 93+0+10 (secure 0 0 0) 0x7fa07c0f17f0 con 0x7fa0841662c0 2024-04-17T09:55:47.220 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:55:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.220+0000 7fa08a600640 1 -- 172.21.15.32:0/2836063745 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa0600683b0 msgr2=0x7fa06006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.220+0000 7fa08a600640 1 --2- 172.21.15.32:0/2836063745 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa0600683b0 0x7fa06006a870 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa0740079c0 tx=0x7fa074008040 comp rx=0 tx=0).stop 2024-04-17T09:55:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.220+0000 7fa08a600640 1 -- 172.21.15.32:0/2836063745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0841662c0 msgr2=0x7fa084157fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.220+0000 7fa08a600640 1 --2- 172.21.15.32:0/2836063745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0841662c0 0x7fa084157fc0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa07c0af040 tx=0x7fa07c019790 comp rx=0 tx=0).stop 2024-04-17T09:55:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.220+0000 7fa08a600640 1 -- 172.21.15.32:0/2836063745 shutdown_connections 2024-04-17T09:55:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.220+0000 7fa08a600640 1 --2- 172.21.15.32:0/2836063745 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa0600683b0 0x7fa06006a870 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.220+0000 7fa08a600640 1 --2- 172.21.15.32:0/2836063745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0841662c0 0x7fa084157fc0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.220+0000 7fa08a600640 1 --2- 172.21.15.32:0/2836063745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa08406eb60 0x7fa08414e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.220+0000 7fa08a600640 1 --2- 172.21.15.32:0/2836063745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa08406ba30 0x7fa08415b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.220+0000 7fa08a600640 1 -- 172.21.15.32:0/2836063745 >> 172.21.15.32:0/2836063745 conn(0x7fa0840b2120 msgr2=0x7fa08406df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.220+0000 7fa08a600640 1 -- 172.21.15.32:0/2836063745 shutdown_connections 2024-04-17T09:55:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.220+0000 7fa08a600640 1 -- 172.21.15.32:0/2836063745 wait complete. 2024-04-17T09:55:47.241 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:55:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.384+0000 7f952d000640 1 Processor -- start 2024-04-17T09:55:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.384+0000 7f952d000640 1 -- start start 2024-04-17T09:55:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.384+0000 7f952d000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f952815c4c0 0x7f95280adb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.384+0000 7f952d000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f952806c140 0x7f95280b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.384+0000 7f952d000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95281629a0 con 0x7f952815f9c0 2024-04-17T09:55:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.384+0000 7f952d000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f9528162c80 con 0x7f952806c140 2024-04-17T09:55:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.384+0000 7f952d000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9528162f60 con 0x7f952815c4c0 2024-04-17T09:55:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.384+0000 7f9525e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f952815c4c0 0x7f95280adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.384+0000 7f9526800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f952806c140 0x7f95280b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.384+0000 7f951d400640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f952815f9c0 0x7f952806ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:42190/0 (socket says 172.21.15.32:42190) 2024-04-17T09:55:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.384+0000 7f951d400640 1 -- 172.21.15.32:0/1003862292 learned_addr learned my addr 172.21.15.32:0/1003862292 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.384+0000 7f9525e00640 1 --2- 172.21.15.32:0/1003862292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f952815c4c0 0x7f95280adb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:35600/0 (socket says 172.21.15.32:35600) 2024-04-17T09:55:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.384+0000 7f9525e00640 1 -- 172.21.15.32:0/1003862292 >> v2:172.21.15.144:3300/0 conn(0x7f952806c140 msgr2=0x7f95280b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.384+0000 7f9525e00640 1 --2- 172.21.15.32:0/1003862292 >> v2:172.21.15.144:3300/0 conn(0x7f952806c140 0x7f95280b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.384+0000 7f9525e00640 1 -- 172.21.15.32:0/1003862292 >> v1:172.21.15.144:6789/0 conn(0x7f952815f9c0 legacy=0x7f952806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.384+0000 7f9525e00640 1 -- 172.21.15.32:0/1003862292 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9528163240 con 0x7f952815c4c0 2024-04-17T09:55:47.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.388+0000 7f9525e00640 1 --2- 172.21.15.32:0/1003862292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f952815c4c0 0x7f95280adb40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9518009d90 tx=0x7f951801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f94d96c2c4b7c0a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:47.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.388+0000 7f9525400640 1 -- 172.21.15.32:0/1003862292 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95180b2070 con 0x7f952815c4c0 2024-04-17T09:55:47.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.388+0000 7f9525400640 1 -- 172.21.15.32:0/1003862292 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95180a5940 con 0x7f952815c4c0 2024-04-17T09:55:47.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.388+0000 7f9525400640 1 -- 172.21.15.32:0/1003862292 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95180a5d10 con 0x7f952815c4c0 2024-04-17T09:55:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.388+0000 7f952d000640 1 -- 172.21.15.32:0/1003862292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f952815c4c0 msgr2=0x7f95280adb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.388+0000 7f952d000640 1 --2- 172.21.15.32:0/1003862292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f952815c4c0 0x7f95280adb40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9518009d90 tx=0x7f951801c970 comp rx=0 tx=0).stop 2024-04-17T09:55:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.388+0000 7f952d000640 1 -- 172.21.15.32:0/1003862292 shutdown_connections 2024-04-17T09:55:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.388+0000 7f952d000640 1 --2- 172.21.15.32:0/1003862292 >> v2:172.21.15.144:3300/0 conn(0x7f952806c140 0x7f95280b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.388+0000 7f952d000640 1 --2- 172.21.15.32:0/1003862292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f952815c4c0 0x7f95280adb40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.388+0000 7f952d000640 1 -- 172.21.15.32:0/1003862292 >> 172.21.15.32:0/1003862292 conn(0x7f95280b2120 msgr2=0x7f95280b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.388+0000 7f952d000640 1 -- 172.21.15.32:0/1003862292 shutdown_connections 2024-04-17T09:55:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.388+0000 7f952d000640 1 -- 172.21.15.32:0/1003862292 wait complete. 2024-04-17T09:55:47.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f952d000640 1 Processor -- start 2024-04-17T09:55:47.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f952d000640 1 -- start start 2024-04-17T09:55:47.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f952d000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f952806c140 0x7f9528145fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:47.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f952d000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f952815c4c0 0x7f9528161630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:47.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f952d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952815f9c0 0x7f95280ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:47.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f952d000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f95281626b0 con 0x7f952815f9c0 2024-04-17T09:55:47.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f952d000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9528162530 con 0x7f952806c140 2024-04-17T09:55:47.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f952d000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f952816b6a0 con 0x7f952815c4c0 2024-04-17T09:55:47.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f9526800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952815f9c0 0x7f95280ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:47.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f9526800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952815f9c0 0x7f95280ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56094/0 (socket says 172.21.15.32:56094) 2024-04-17T09:55:47.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f9526800640 1 -- 172.21.15.32:0/3930967118 learned_addr learned my addr 172.21.15.32:0/3930967118 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:47.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f9525e00640 1 --2- 172.21.15.32:0/3930967118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f952806c140 0x7f9528145fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:47.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f951d400640 1 --2- 172.21.15.32:0/3930967118 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f952815c4c0 0x7f9528161630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:47.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f9526800640 1 -- 172.21.15.32:0/3930967118 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f952815c4c0 msgr2=0x7f9528161630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:47.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f9526800640 1 --2- 172.21.15.32:0/3930967118 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f952815c4c0 0x7f9528161630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f9526800640 1 -- 172.21.15.32:0/3930967118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f952806c140 msgr2=0x7f9528145fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:47.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f9526800640 1 --2- 172.21.15.32:0/3930967118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f952806c140 0x7f9528145fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f9526800640 1 -- 172.21.15.32:0/3930967118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95280cd800 con 0x7f952815f9c0 2024-04-17T09:55:47.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f9526800640 1 --2- 172.21.15.32:0/3930967118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952815f9c0 0x7f95280ce8a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f952000e880 tx=0x7f952000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:47.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f951ea00640 1 -- 172.21.15.32:0/3930967118 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520017070 con 0x7f952815f9c0 2024-04-17T09:55:47.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f951ea00640 1 -- 172.21.15.32:0/3930967118 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f952000c940 con 0x7f952815f9c0 2024-04-17T09:55:47.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f951ea00640 1 -- 172.21.15.32:0/3930967118 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952000cd10 con 0x7f952815f9c0 2024-04-17T09:55:47.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f952d000640 1 -- 172.21.15.32:0/3930967118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95280cdae0 con 0x7f952815f9c0 2024-04-17T09:55:47.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.392+0000 7f952d000640 1 -- 172.21.15.32:0/3930967118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95280ce0a0 con 0x7f952815f9c0 2024-04-17T09:55:47.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.396+0000 7f952d000640 1 -- 172.21.15.32:0/3930967118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9528068a90 con 0x7f952815f9c0 2024-04-17T09:55:47.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.396+0000 7f951ea00640 1 -- 172.21.15.32:0/3930967118 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f952001a070 con 0x7f952815f9c0 2024-04-17T09:55:47.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.396+0000 7f951ea00640 1 --2- 172.21.15.32:0/3930967118 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9504068380 0x7f950406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:47.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.396+0000 7f951ea00640 1 -- 172.21.15.32:0/3930967118 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f9520086d30 con 0x7f952815f9c0 2024-04-17T09:55:47.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.396+0000 7f9525e00640 1 --2- 172.21.15.32:0/3930967118 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9504068380 0x7f950406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:47.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.396+0000 7f9525e00640 1 --2- 172.21.15.32:0/3930967118 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9504068380 0x7f950406a840 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f95180af000 tx=0x7f95180a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:47.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.400+0000 7f951ea00640 1 -- 172.21.15.32:0/3930967118 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f9520056c00 con 0x7f952815f9c0 2024-04-17T09:55:47.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.564+0000 7f952d000640 1 -- 172.21.15.32:0/3930967118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9528146500 con 0x7f952815f9c0 2024-04-17T09:55:47.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.564+0000 7f951ea00640 1 -- 172.21.15.32:0/3930967118 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) ==== 104+0+11 (secure 0 0 0) 0x7f952005af00 con 0x7f952815f9c0 2024-04-17T09:55:47.566 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:55:47.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.564+0000 7f952d000640 1 -- 172.21.15.32:0/3930967118 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9504068380 msgr2=0x7f950406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:47.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.564+0000 7f952d000640 1 --2- 172.21.15.32:0/3930967118 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9504068380 0x7f950406a840 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f95180af000 tx=0x7f95180a7070 comp rx=0 tx=0).stop 2024-04-17T09:55:47.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.564+0000 7f952d000640 1 -- 172.21.15.32:0/3930967118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952815f9c0 msgr2=0x7f95280ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:47.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.564+0000 7f952d000640 1 --2- 172.21.15.32:0/3930967118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952815f9c0 0x7f95280ce8a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f952000e880 tx=0x7f952000ed40 comp rx=0 tx=0).stop 2024-04-17T09:55:47.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.564+0000 7f952d000640 1 -- 172.21.15.32:0/3930967118 shutdown_connections 2024-04-17T09:55:47.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.568+0000 7f952d000640 1 --2- 172.21.15.32:0/3930967118 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9504068380 0x7f950406a840 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.568+0000 7f952d000640 1 --2- 172.21.15.32:0/3930967118 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f952815c4c0 0x7f9528161630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.568+0000 7f952d000640 1 --2- 172.21.15.32:0/3930967118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952815f9c0 0x7f95280ce8a0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.568+0000 7f952d000640 1 --2- 172.21.15.32:0/3930967118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f952806c140 0x7f9528145fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.568+0000 7f952d000640 1 -- 172.21.15.32:0/3930967118 >> 172.21.15.32:0/3930967118 conn(0x7f95280b2120 msgr2=0x7f952806ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:47.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.568+0000 7f952d000640 1 -- 172.21.15.32:0/3930967118 shutdown_connections 2024-04-17T09:55:47.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.568+0000 7f952d000640 1 -- 172.21.15.32:0/3930967118 wait complete. 2024-04-17T09:55:47.588 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:55:47.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.728+0000 7f2972a00640 1 Processor -- start 2024-04-17T09:55:47.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.728+0000 7f2972a00640 1 -- start start 2024-04-17T09:55:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.732+0000 7f2972a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f296c06d150 0x7f296c0af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.732+0000 7f2972a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f296c0afce0 0x7f296c0b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.732+0000 7f2972a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f296c0b0540 con 0x7f296c141700 2024-04-17T09:55:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.732+0000 7f2972a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f296c0b0820 con 0x7f296c0afce0 2024-04-17T09:55:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.732+0000 7f2972a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f296c0b3ef0 con 0x7f296c06d150 2024-04-17T09:55:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.732+0000 7f2970a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f296c0afce0 0x7f296c0b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.732+0000 7f296be00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f296c141700 0x7f296c06ca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53678/0 (socket says 172.21.15.32:53678) 2024-04-17T09:55:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.732+0000 7f296be00640 1 -- 172.21.15.32:0/2357958554 learned_addr learned my addr 172.21.15.32:0/2357958554 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.732+0000 7f2963400640 1 --2- 172.21.15.32:0/2357958554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f296c06d150 0x7f296c0af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.732+0000 7f2970a00640 1 -- 172.21.15.32:0/2357958554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f296c06d150 msgr2=0x7f296c0af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.732+0000 7f2970a00640 1 --2- 172.21.15.32:0/2357958554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f296c06d150 0x7f296c0af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.732+0000 7f2970a00640 1 -- 172.21.15.32:0/2357958554 >> v1:172.21.15.32:6789/0 conn(0x7f296c141700 legacy=0x7f296c06ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.732+0000 7f2970a00640 1 -- 172.21.15.32:0/2357958554 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f296c0b41d0 con 0x7f296c0afce0 2024-04-17T09:55:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.732+0000 7f296b400640 1 -- 172.21.15.32:0/2357958554 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2035525900 0 0) 0x7f296c0b0540 con 0x7f296c141700 2024-04-17T09:55:47.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.736+0000 7f2970a00640 1 --2- 172.21.15.32:0/2357958554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296c0afce0 0x7f296c0b49d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2964009d90 tx=0x7f296401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913d6435e608fa22 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:47.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.736+0000 7f296b400640 1 -- 172.21.15.32:0/2357958554 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29640b2070 con 0x7f296c0afce0 2024-04-17T09:55:47.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.736+0000 7f296b400640 1 -- 172.21.15.32:0/2357958554 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29640a5a00 con 0x7f296c0afce0 2024-04-17T09:55:47.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.736+0000 7f296b400640 1 -- 172.21.15.32:0/2357958554 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29640a5dd0 con 0x7f296c0afce0 2024-04-17T09:55:47.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.736+0000 7f2972a00640 1 -- 172.21.15.32:0/2357958554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296c0afce0 msgr2=0x7f296c0b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:47.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.736+0000 7f2972a00640 1 --2- 172.21.15.32:0/2357958554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296c0afce0 0x7f296c0b49d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2964009d90 tx=0x7f296401c920 comp rx=0 tx=0).stop 2024-04-17T09:55:47.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.736+0000 7f2972a00640 1 -- 172.21.15.32:0/2357958554 shutdown_connections 2024-04-17T09:55:47.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.736+0000 7f2972a00640 1 --2- 172.21.15.32:0/2357958554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296c0afce0 0x7f296c0b49d0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.736+0000 7f2972a00640 1 --2- 172.21.15.32:0/2357958554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f296c06d150 0x7f296c0af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.736+0000 7f2972a00640 1 -- 172.21.15.32:0/2357958554 >> 172.21.15.32:0/2357958554 conn(0x7f296c072670 msgr2=0x7f296c072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:47.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.736+0000 7f2972a00640 1 -- 172.21.15.32:0/2357958554 shutdown_connections 2024-04-17T09:55:47.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.736+0000 7f2972a00640 1 -- 172.21.15.32:0/2357958554 wait complete. 2024-04-17T09:55:47.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.736+0000 7f2972a00640 1 Processor -- start 2024-04-17T09:55:47.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.736+0000 7f2972a00640 1 -- start start 2024-04-17T09:55:47.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f2972a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296c06d150 0x7f296c13e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:47.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f2972a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f296c0afce0 0x7f296c140540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:47.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f2972a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f296c141700 0x7f296c161340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:47.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f2972a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f296c0b5720 con 0x7f296c06d150 2024-04-17T09:55:47.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f2972a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f296c141cf0 con 0x7f296c141700 2024-04-17T09:55:47.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f2972a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f296c0b58a0 con 0x7f296c0afce0 2024-04-17T09:55:47.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f296be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296c06d150 0x7f296c13e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f296be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296c06d150 0x7f296c13e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56124/0 (socket says 172.21.15.32:56124) 2024-04-17T09:55:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f296be00640 1 -- 172.21.15.32:0/1397772827 learned_addr learned my addr 172.21.15.32:0/1397772827 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f2963400640 1 --2- 172.21.15.32:0/1397772827 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f296c0afce0 0x7f296c140540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f2970a00640 1 --2- 172.21.15.32:0/1397772827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f296c141700 0x7f296c161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f296be00640 1 -- 172.21.15.32:0/1397772827 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f296c0afce0 msgr2=0x7f296c140540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f296be00640 1 --2- 172.21.15.32:0/1397772827 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f296c0afce0 0x7f296c140540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f296be00640 1 -- 172.21.15.32:0/1397772827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f296c141700 msgr2=0x7f296c161340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f296be00640 1 --2- 172.21.15.32:0/1397772827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f296c141700 0x7f296c161340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f296be00640 1 -- 172.21.15.32:0/1397772827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f296c146fe0 con 0x7f296c06d150 2024-04-17T09:55:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f296be00640 1 --2- 172.21.15.32:0/1397772827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296c06d150 0x7f296c13e170 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f295c00c4b0 tx=0x7f295c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f2968c00640 1 -- 172.21.15.32:0/1397772827 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295c018070 con 0x7f296c06d150 2024-04-17T09:55:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f2968c00640 1 -- 172.21.15.32:0/1397772827 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f295c007500 con 0x7f296c06d150 2024-04-17T09:55:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f2972a00640 1 -- 172.21.15.32:0/1397772827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f296c160860 con 0x7f296c06d150 2024-04-17T09:55:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f2968c00640 1 -- 172.21.15.32:0/1397772827 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295c004a30 con 0x7f296c06d150 2024-04-17T09:55:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f2972a00640 1 -- 172.21.15.32:0/1397772827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f296c160e20 con 0x7f296c06d150 2024-04-17T09:55:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f2972a00640 1 -- 172.21.15.32:0/1397772827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f296c161020 con 0x7f296c06d150 2024-04-17T09:55:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.740+0000 7f2968c00640 1 -- 172.21.15.32:0/1397772827 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f295c01b070 con 0x7f296c06d150 2024-04-17T09:55:47.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.744+0000 7f2968c00640 1 --2- 172.21.15.32:0/1397772827 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2948068380 0x7f294806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:47.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.744+0000 7f2968c00640 1 -- 172.21.15.32:0/1397772827 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f295c085bf0 con 0x7f296c06d150 2024-04-17T09:55:47.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.744+0000 7f2963400640 1 --2- 172.21.15.32:0/1397772827 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2948068380 0x7f294806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:47.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.744+0000 7f2963400640 1 --2- 172.21.15.32:0/1397772827 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2948068380 0x7f294806a840 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f296c140f00 tx=0x7f2954009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:47.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.748+0000 7f2968c00640 1 -- 172.21.15.32:0/1397772827 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f295c055b60 con 0x7f296c06d150 2024-04-17T09:55:47.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.908+0000 7f2972a00640 1 -- 172.21.15.32:0/1397772827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f296c068a90 con 0x7f296c06d150 2024-04-17T09:55:47.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.908+0000 7f2968c00640 1 -- 172.21.15.32:0/1397772827 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) ==== 100+0+11 (secure 0 0 0) 0x7f295c059e60 con 0x7f296c06d150 2024-04-17T09:55:47.910 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:55:47.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.912+0000 7f2972a00640 1 -- 172.21.15.32:0/1397772827 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2948068380 msgr2=0x7f294806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:47.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.912+0000 7f2972a00640 1 --2- 172.21.15.32:0/1397772827 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2948068380 0x7f294806a840 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f296c140f00 tx=0x7f2954009190 comp rx=0 tx=0).stop 2024-04-17T09:55:47.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.912+0000 7f2972a00640 1 -- 172.21.15.32:0/1397772827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296c06d150 msgr2=0x7f296c13e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:47.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.912+0000 7f2972a00640 1 --2- 172.21.15.32:0/1397772827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296c06d150 0x7f296c13e170 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f295c00c4b0 tx=0x7f295c00c970 comp rx=0 tx=0).stop 2024-04-17T09:55:47.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.912+0000 7f2972a00640 1 -- 172.21.15.32:0/1397772827 shutdown_connections 2024-04-17T09:55:47.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.912+0000 7f2972a00640 1 --2- 172.21.15.32:0/1397772827 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2948068380 0x7f294806a840 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.912+0000 7f2972a00640 1 --2- 172.21.15.32:0/1397772827 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f296c0afce0 0x7f296c140540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.912+0000 7f2972a00640 1 --2- 172.21.15.32:0/1397772827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f296c141700 0x7f296c161340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.912+0000 7f2972a00640 1 --2- 172.21.15.32:0/1397772827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296c06d150 0x7f296c13e170 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:47.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.912+0000 7f2972a00640 1 -- 172.21.15.32:0/1397772827 >> 172.21.15.32:0/1397772827 conn(0x7f296c072670 msgr2=0x7f296c06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:47.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.912+0000 7f2972a00640 1 -- 172.21.15.32:0/1397772827 shutdown_connections 2024-04-17T09:55:47.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:47.912+0000 7f2972a00640 1 -- 172.21.15.32:0/1397772827 wait complete. 2024-04-17T09:55:47.931 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:55:48.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc328a00640 1 Processor -- start 2024-04-17T09:55:48.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc328a00640 1 -- start start 2024-04-17T09:55:48.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc328a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3241608f0 0x7fc324160cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:48.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc328a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc324161210 con 0x7fc3240b5be0 2024-04-17T09:55:48.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc328a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3240d05d0 con 0x7fc3240725d0 2024-04-17T09:55:48.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc328a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc3240b5750 con 0x7fc3241608f0 2024-04-17T09:55:48.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc31be00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc3240b5be0 0x7fc3240cfdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53694/0 (socket says 172.21.15.32:53694) 2024-04-17T09:55:48.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc31be00640 1 -- 172.21.15.32:0/1561262309 learned_addr learned my addr 172.21.15.32:0/1561262309 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:48.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc321e00640 1 --2- 172.21.15.32:0/1561262309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3241608f0 0x7fc324160cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:48.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc321400640 1 -- 172.21.15.32:0/1561262309 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1951275315 0 0) 0x7fc324161210 con 0x7fc3240b5be0 2024-04-17T09:55:48.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc321400640 1 -- 172.21.15.32:0/1561262309 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc308003530 con 0x7fc3240b5be0 2024-04-17T09:55:48.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc321400640 1 -- 172.21.15.32:0/1561262309 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3084464072 0 0) 0x7fc3240d05d0 con 0x7fc3240725d0 2024-04-17T09:55:48.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc321400640 1 -- 172.21.15.32:0/1561262309 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc324161210 con 0x7fc3240725d0 2024-04-17T09:55:48.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc321400640 1 -- 172.21.15.32:0/1561262309 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1433977281 0 0) 0x7fc308003530 con 0x7fc3240b5be0 2024-04-17T09:55:48.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc321400640 1 -- 172.21.15.32:0/1561262309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3241608f0 msgr2=0x7fc324160cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:48.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc321400640 1 --2- 172.21.15.32:0/1561262309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3241608f0 0x7fc324160cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:48.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc321400640 1 -- 172.21.15.32:0/1561262309 >> v1:172.21.15.144:6789/0 conn(0x7fc3240725d0 legacy=0x7fc3240ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:48.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc321400640 1 -- 172.21.15.32:0/1561262309 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3240d08b0 con 0x7fc3240b5be0 2024-04-17T09:55:48.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc321400640 1 -- 172.21.15.32:0/1561262309 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7fc30c0016a0 con 0x7fc3240b5be0 2024-04-17T09:55:48.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc321400640 1 -- 172.21.15.32:0/1561262309 >> v1:172.21.15.32:6789/0 conn(0x7fc3240b5be0 legacy=0x7fc3240cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:48.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc321400640 1 --2- 172.21.15.32:0/1561262309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc308003b80 0x7fc308003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:48.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc321400640 1 --2- 172.21.15.32:0/1561262309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc308004810 0x7fc308004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:48.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc321400640 1 --2- 172.21.15.32:0/1561262309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3241608f0 0x7fc3080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:48.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc321400640 1 -- 172.21.15.32:0/1561262309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc3240b5750 con 0x7fc308004810 2024-04-17T09:55:48.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc321400640 1 -- 172.21.15.32:0/1561262309 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc30800ba80 con 0x7fc308003b80 2024-04-17T09:55:48.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc321400640 1 -- 172.21.15.32:0/1561262309 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc30800bc50 con 0x7fc3241608f0 2024-04-17T09:55:48.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc321e00640 1 --2- 172.21.15.32:0/1561262309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc308003b80 0x7fc308003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:48.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc31be00640 1 --2- 172.21.15.32:0/1561262309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc308004810 0x7fc308004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:48.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.072+0000 7fc322800640 1 --2- 172.21.15.32:0/1561262309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3241608f0 0x7fc3080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:48.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.076+0000 7fc31be00640 1 -- 172.21.15.32:0/1561262309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3241608f0 msgr2=0x7fc3080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:48.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.076+0000 7fc31be00640 1 --2- 172.21.15.32:0/1561262309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3241608f0 0x7fc3080051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:48.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.076+0000 7fc31be00640 1 -- 172.21.15.32:0/1561262309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc308003b80 msgr2=0x7fc308003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:48.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.076+0000 7fc31be00640 1 --2- 172.21.15.32:0/1561262309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc308003b80 0x7fc308003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:48.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.076+0000 7fc31be00640 1 -- 172.21.15.32:0/1561262309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc308003530 con 0x7fc308004810 2024-04-17T09:55:48.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.076+0000 7fc321e00640 1 --2- 172.21.15.32:0/1561262309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc308003b80 0x7fc308003f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:55:48.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.076+0000 7fc31be00640 1 --2- 172.21.15.32:0/1561262309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc308004810 0x7fc308004c90 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00c160 tx=0x7fc30c01cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b347109230084549 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:48.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.076+0000 7fc321400640 1 -- 172.21.15.32:0/1561262309 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c007cd0 con 0x7fc308004810 2024-04-17T09:55:48.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.076+0000 7fc321400640 1 -- 172.21.15.32:0/1561262309 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc30c0a9020 con 0x7fc308004810 2024-04-17T09:55:48.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.076+0000 7fc321400640 1 -- 172.21.15.32:0/1561262309 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c005b00 con 0x7fc308004810 2024-04-17T09:55:48.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.076+0000 7fc328a00640 1 -- 172.21.15.32:0/1561262309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc308004810 msgr2=0x7fc308004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:48.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.076+0000 7fc328a00640 1 --2- 172.21.15.32:0/1561262309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc308004810 0x7fc308004c90 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00c160 tx=0x7fc30c01cb80 comp rx=0 tx=0).stop 2024-04-17T09:55:48.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.076+0000 7fc322800640 1 -- 172.21.15.32:0/1561262309 reap_dead start 2024-04-17T09:55:48.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.076+0000 7fc328a00640 1 -- 172.21.15.32:0/1561262309 shutdown_connections 2024-04-17T09:55:48.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.076+0000 7fc328a00640 1 -- 172.21.15.32:0/1561262309 >> 172.21.15.32:0/1561262309 conn(0x7fc3240d0c10 msgr2=0x7fc3240d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:48.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.076+0000 7fc328a00640 1 -- 172.21.15.32:0/1561262309 shutdown_connections 2024-04-17T09:55:48.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.076+0000 7fc328a00640 1 -- 172.21.15.32:0/1561262309 wait complete. 2024-04-17T09:55:48.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc328a00640 1 Processor -- start 2024-04-17T09:55:48.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc328a00640 1 -- start start 2024-04-17T09:55:48.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc328a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc308004810 0x7fc3241f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:48.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc328a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc324157650 0x7fc32415ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:48.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc328a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc32415f450 0x7fc324159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc328a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc3240b6040 con 0x7fc324157650 2024-04-17T09:55:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc328a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc3240b53e0 con 0x7fc308004810 2024-04-17T09:55:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc328a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc324160cd0 con 0x7fc32415f450 2024-04-17T09:55:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc31be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc324157650 0x7fc32415ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc31be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc324157650 0x7fc32415ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56134/0 (socket says 172.21.15.32:56134) 2024-04-17T09:55:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc31be00640 1 -- 172.21.15.32:0/954410488 learned_addr learned my addr 172.21.15.32:0/954410488 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc321e00640 1 --2- 172.21.15.32:0/954410488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc308004810 0x7fc3241f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc322800640 1 --2- 172.21.15.32:0/954410488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc32415f450 0x7fc324159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc31be00640 1 -- 172.21.15.32:0/954410488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc32415f450 msgr2=0x7fc324159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc31be00640 1 --2- 172.21.15.32:0/954410488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc32415f450 0x7fc324159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc31be00640 1 -- 172.21.15.32:0/954410488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc308004810 msgr2=0x7fc3241f5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc31be00640 1 --2- 172.21.15.32:0/954410488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc308004810 0x7fc3241f5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc31be00640 1 -- 172.21.15.32:0/954410488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3241f5d70 con 0x7fc324157650 2024-04-17T09:55:48.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc322800640 1 --2- 172.21.15.32:0/954410488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc32415f450 0x7fc324159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:48.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc31be00640 1 --2- 172.21.15.32:0/954410488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc324157650 0x7fc32415ef10 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc30c007040 tx=0x7fc308005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:48.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc31a000640 1 -- 172.21.15.32:0/954410488 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c007cd0 con 0x7fc324157650 2024-04-17T09:55:48.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc31a000640 1 -- 172.21.15.32:0/954410488 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc30c004a10 con 0x7fc324157650 2024-04-17T09:55:48.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc31a000640 1 -- 172.21.15.32:0/954410488 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c001c30 con 0x7fc324157650 2024-04-17T09:55:48.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc328a00640 1 -- 172.21.15.32:0/954410488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32415c970 con 0x7fc324157650 2024-04-17T09:55:48.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.080+0000 7fc328a00640 1 -- 172.21.15.32:0/954410488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc32415cf30 con 0x7fc324157650 2024-04-17T09:55:48.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.084+0000 7fc328a00640 1 -- 172.21.15.32:0/954410488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc32415f860 con 0x7fc324157650 2024-04-17T09:55:48.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.084+0000 7fc31a000640 1 -- 172.21.15.32:0/954410488 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc30c007070 con 0x7fc324157650 2024-04-17T09:55:48.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.084+0000 7fc31a000640 1 --2- 172.21.15.32:0/954410488 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc3000683b0 0x7fc30006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:48.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.084+0000 7fc31a000640 1 -- 172.21.15.32:0/954410488 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fc30c120b80 con 0x7fc324157650 2024-04-17T09:55:48.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.084+0000 7fc321e00640 1 --2- 172.21.15.32:0/954410488 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc3000683b0 0x7fc30006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:48.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.084+0000 7fc321e00640 1 --2- 172.21.15.32:0/954410488 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc3000683b0 0x7fc30006a870 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc314009c20 tx=0x7fc314009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:48.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.088+0000 7fc31a000640 1 -- 172.21.15.32:0/954410488 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc32415f860 con 0x7fc324157650 2024-04-17T09:55:48.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.288+0000 7fc328a00640 1 -- 172.21.15.32:0/954410488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc324068a90 con 0x7fc324157650 2024-04-17T09:55:48.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.288+0000 7fc31a000640 1 -- 172.21.15.32:0/954410488 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5202 (secure 0 0 0) 0x7fc30c016890 con 0x7fc324157650 2024-04-17T09:55:48.289 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:48.289 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2542578926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2542578926},{"type":"v1","addr":"172.21.15.144:6834","nonce":2542578926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1424948520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1424948520},{"type":"v1","addr":"172.21.15.32:6836","nonce":1424948520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/393813154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":393813154},{"type":"v1","addr":"172.21.15.32:6837","nonce":393813154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:55:03.233552+0000","modified":"2024-04-17T09:55:06.243321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5742":{"gid":5742,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/2177384675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2177384675},{"type":"v1","addr":"172.21.15.144:6835","nonce":2177384675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742]},"id":3}]} 2024-04-17T09:55:48.290 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2024-04-17T09:55:48.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.288+0000 7fc328a00640 1 -- 172.21.15.32:0/954410488 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc3000683b0 msgr2=0x7fc30006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:48.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.288+0000 7fc328a00640 1 --2- 172.21.15.32:0/954410488 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc3000683b0 0x7fc30006a870 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc314009c20 tx=0x7fc314009190 comp rx=0 tx=0).stop 2024-04-17T09:55:48.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.288+0000 7fc328a00640 1 -- 172.21.15.32:0/954410488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc324157650 msgr2=0x7fc32415ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:48.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.288+0000 7fc328a00640 1 --2- 172.21.15.32:0/954410488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc324157650 0x7fc32415ef10 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc30c007040 tx=0x7fc308005840 comp rx=0 tx=0).stop 2024-04-17T09:55:48.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.288+0000 7fc328a00640 1 -- 172.21.15.32:0/954410488 shutdown_connections 2024-04-17T09:55:48.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.288+0000 7fc328a00640 1 --2- 172.21.15.32:0/954410488 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc3000683b0 0x7fc30006a870 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:48.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.288+0000 7fc328a00640 1 --2- 172.21.15.32:0/954410488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc32415f450 0x7fc324159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:48.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.288+0000 7fc328a00640 1 --2- 172.21.15.32:0/954410488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc324157650 0x7fc32415ef10 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:48.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.288+0000 7fc328a00640 1 --2- 172.21.15.32:0/954410488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc308004810 0x7fc3241f5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:48.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.288+0000 7fc328a00640 1 -- 172.21.15.32:0/954410488 >> 172.21.15.32:0/954410488 conn(0x7fc3240d0c10 msgr2=0x7fc3240b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:48.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.288+0000 7fc328a00640 1 -- 172.21.15.32:0/954410488 shutdown_connections 2024-04-17T09:55:48.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.288+0000 7fc328a00640 1 -- 172.21.15.32:0/954410488 wait complete. 2024-04-17T09:55:48.310 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:55:48.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.448+0000 7f246be00640 1 Processor -- start 2024-04-17T09:55:48.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.448+0000 7f246be00640 1 -- start start 2024-04-17T09:55:48.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.452+0000 7f246be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f246c06ddf0 0x7f246c070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:48.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.452+0000 7f246be00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f246c06eb40 con 0x7f246c070dc0 2024-04-17T09:55:48.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.452+0000 7f246be00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f246c0b4970 con 0x7f246c09a9f0 2024-04-17T09:55:48.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.452+0000 7f246be00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f246c0b4c50 con 0x7f246c06ddf0 2024-04-17T09:55:48.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.452+0000 7f246a000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f246c06ddf0 0x7f246c070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:48.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.452+0000 7f246b400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f246c070dc0 0x7f246c09add0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53698/0 (socket says 172.21.15.32:53698) 2024-04-17T09:55:48.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.452+0000 7f246b400640 1 -- 172.21.15.32:0/461141001 learned_addr learned my addr 172.21.15.32:0/461141001 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:48.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.452+0000 7f2469600640 1 -- 172.21.15.32:0/461141001 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3421671865 0 0) 0x7f246c06eb40 con 0x7f246c070dc0 2024-04-17T09:55:48.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.452+0000 7f246a000640 1 -- 172.21.15.32:0/461141001 >> v1:172.21.15.144:6789/0 conn(0x7f246c09a9f0 legacy=0x7f246c0d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:48.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.452+0000 7f246a000640 1 -- 172.21.15.32:0/461141001 >> v1:172.21.15.32:6789/0 conn(0x7f246c070dc0 legacy=0x7f246c09add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:48.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.452+0000 7f246a000640 1 -- 172.21.15.32:0/461141001 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246c06d550 con 0x7f246c06ddf0 2024-04-17T09:55:48.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.456+0000 7f246a000640 1 --2- 172.21.15.32:0/461141001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f246c06ddf0 0x7f246c070880 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f24580097c0 tx=0x7f245801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88d2087b0741bdf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:48.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.456+0000 7f2469600640 1 -- 172.21.15.32:0/461141001 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24580b0070 con 0x7f246c06ddf0 2024-04-17T09:55:48.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.456+0000 7f2469600640 1 -- 172.21.15.32:0/461141001 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24580ab600 con 0x7f246c06ddf0 2024-04-17T09:55:48.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.456+0000 7f2469600640 1 -- 172.21.15.32:0/461141001 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24580ab9d0 con 0x7f246c06ddf0 2024-04-17T09:55:48.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.456+0000 7f246be00640 1 -- 172.21.15.32:0/461141001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f246c06ddf0 msgr2=0x7f246c070880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:48.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.456+0000 7f246be00640 1 --2- 172.21.15.32:0/461141001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f246c06ddf0 0x7f246c070880 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f24580097c0 tx=0x7f245801c4a0 comp rx=0 tx=0).stop 2024-04-17T09:55:48.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.456+0000 7f246be00640 1 -- 172.21.15.32:0/461141001 shutdown_connections 2024-04-17T09:55:48.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.456+0000 7f246be00640 1 --2- 172.21.15.32:0/461141001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f246c06ddf0 0x7f246c070880 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:48.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.456+0000 7f246be00640 1 -- 172.21.15.32:0/461141001 >> 172.21.15.32:0/461141001 conn(0x7f246c0b3e50 msgr2=0x7f246c0b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:48.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.456+0000 7f246be00640 1 -- 172.21.15.32:0/461141001 shutdown_connections 2024-04-17T09:55:48.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.456+0000 7f246be00640 1 -- 172.21.15.32:0/461141001 wait complete. 2024-04-17T09:55:48.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.456+0000 7f246be00640 1 Processor -- start 2024-04-17T09:55:48.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.456+0000 7f246be00640 1 -- start start 2024-04-17T09:55:48.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.460+0000 7f246be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f246c06ddf0 0x7f246c15c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:48.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.460+0000 7f246be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f246c070dc0 0x7f246c15ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:48.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.460+0000 7f246be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f246c09a9f0 0x7f246c162190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:48.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.460+0000 7f246be00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f246c16d0e0 con 0x7f246c06ddf0 2024-04-17T09:55:48.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.460+0000 7f246be00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f246c16cf60 con 0x7f246c09a9f0 2024-04-17T09:55:48.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.460+0000 7f246be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f246c16d260 con 0x7f246c070dc0 2024-04-17T09:55:48.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.460+0000 7f246aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f246c06ddf0 0x7f246c15c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:48.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.460+0000 7f246aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f246c06ddf0 0x7f246c15c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56144/0 (socket says 172.21.15.32:56144) 2024-04-17T09:55:48.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.460+0000 7f246aa00640 1 -- 172.21.15.32:0/3494997699 learned_addr learned my addr 172.21.15.32:0/3494997699 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.460+0000 7f246aa00640 1 -- 172.21.15.32:0/3494997699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f246c070dc0 msgr2=0x7f246c15ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.460+0000 7f246aa00640 1 --2- 172.21.15.32:0/3494997699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f246c070dc0 0x7f246c15ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.460+0000 7f246aa00640 1 -- 172.21.15.32:0/3494997699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f246c09a9f0 msgr2=0x7f246c162190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.460+0000 7f246aa00640 1 --2- 172.21.15.32:0/3494997699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f246c09a9f0 0x7f246c162190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.460+0000 7f246aa00640 1 -- 172.21.15.32:0/3494997699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246c162ba0 con 0x7f246c06ddf0 2024-04-17T09:55:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.460+0000 7f246aa00640 1 --2- 172.21.15.32:0/3494997699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f246c06ddf0 0x7f246c15c0c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f245c00c420 tx=0x7f245c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.460+0000 7f2462a00640 1 -- 172.21.15.32:0/3494997699 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c0052d0 con 0x7f246c06ddf0 2024-04-17T09:55:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.460+0000 7f2462a00640 1 -- 172.21.15.32:0/3494997699 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f245c005470 con 0x7f246c06ddf0 2024-04-17T09:55:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.460+0000 7f2462a00640 1 -- 172.21.15.32:0/3494997699 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c007240 con 0x7f246c06ddf0 2024-04-17T09:55:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.460+0000 7f246be00640 1 -- 172.21.15.32:0/3494997699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f246c0b6450 con 0x7f246c06ddf0 2024-04-17T09:55:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.460+0000 7f246be00640 1 -- 172.21.15.32:0/3494997699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f246c0b6a10 con 0x7f246c06ddf0 2024-04-17T09:55:48.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.460+0000 7f246be00640 1 -- 172.21.15.32:0/3494997699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f246c04fd00 con 0x7f246c06ddf0 2024-04-17T09:55:48.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.460+0000 7f2462a00640 1 -- 172.21.15.32:0/3494997699 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f245c0073e0 con 0x7f246c06ddf0 2024-04-17T09:55:48.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.464+0000 7f2462a00640 1 --2- 172.21.15.32:0/3494997699 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f244c068380 0x7f244c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:48.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.464+0000 7f2462a00640 1 -- 172.21.15.32:0/3494997699 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f245c085f80 con 0x7f246c06ddf0 2024-04-17T09:55:48.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.464+0000 7f246a000640 1 --2- 172.21.15.32:0/3494997699 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f244c068380 0x7f244c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.464+0000 7f246a000640 1 --2- 172.21.15.32:0/3494997699 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f244c068380 0x7f244c06a840 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f24580ad000 tx=0x7f2458008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:48.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.468+0000 7f2462a00640 1 -- 172.21.15.32:0/3494997699 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f245c0564a0 con 0x7f246c06ddf0 2024-04-17T09:55:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.632+0000 7f246be00640 1 -- 172.21.15.32:0/3494997699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f246c0d20d0 con 0x7f246c06ddf0 2024-04-17T09:55:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.632+0000 7f2462a00640 1 -- 172.21.15.32:0/3494997699 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16625 (secure 0 0 0) 0x7f245c055da0 con 0x7f246c06ddf0 2024-04-17T09:55:48.634 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:48.634 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":46,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:55:04.834732+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:55:01.624737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-17T09:55:02.185212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:55:48.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.636+0000 7f246be00640 1 -- 172.21.15.32:0/3494997699 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f244c068380 msgr2=0x7f244c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:48.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.636+0000 7f246be00640 1 --2- 172.21.15.32:0/3494997699 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f244c068380 0x7f244c06a840 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f24580ad000 tx=0x7f2458008040 comp rx=0 tx=0).stop 2024-04-17T09:55:48.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.636+0000 7f246be00640 1 -- 172.21.15.32:0/3494997699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f246c06ddf0 msgr2=0x7f246c15c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:48.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.636+0000 7f246be00640 1 --2- 172.21.15.32:0/3494997699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f246c06ddf0 0x7f246c15c0c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f245c00c420 tx=0x7f245c00c8e0 comp rx=0 tx=0).stop 2024-04-17T09:55:48.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.636+0000 7f246be00640 1 -- 172.21.15.32:0/3494997699 shutdown_connections 2024-04-17T09:55:48.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.636+0000 7f246be00640 1 --2- 172.21.15.32:0/3494997699 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f244c068380 0x7f244c06a840 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:48.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.636+0000 7f246be00640 1 --2- 172.21.15.32:0/3494997699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f246c070dc0 0x7f246c15ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:48.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.636+0000 7f246be00640 1 --2- 172.21.15.32:0/3494997699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f246c09a9f0 0x7f246c162190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:48.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.636+0000 7f246be00640 1 --2- 172.21.15.32:0/3494997699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f246c06ddf0 0x7f246c15c0c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:48.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.636+0000 7f246be00640 1 -- 172.21.15.32:0/3494997699 >> 172.21.15.32:0/3494997699 conn(0x7f246c0b3e50 msgr2=0x7f246c0af2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:48.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.636+0000 7f246be00640 1 -- 172.21.15.32:0/3494997699 shutdown_connections 2024-04-17T09:55:48.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.636+0000 7f246be00640 1 -- 172.21.15.32:0/3494997699 wait complete. 2024-04-17T09:55:48.655 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T09:55:48.655 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:55:48.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.792+0000 7f6355a00640 1 Processor -- start 2024-04-17T09:55:48.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.792+0000 7f6355a00640 1 -- start start 2024-04-17T09:55:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.796+0000 7f6355a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6350144e90 0x7f63500b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.796+0000 7f6355a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63500b4de0 0x7f635015baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.796+0000 7f6355a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6350072bb0 con 0x7f635015bfe0 2024-04-17T09:55:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.796+0000 7f6355a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f635015c430 con 0x7f6350144e90 2024-04-17T09:55:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.796+0000 7f6355a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6350145270 con 0x7f63500b4de0 2024-04-17T09:55:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.796+0000 7f634ea00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6350144e90 0x7f63500b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.796+0000 7f634ea00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6350144e90 0x7f63500b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56156/0 (socket says 172.21.15.32:56156) 2024-04-17T09:55:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.796+0000 7f634ea00640 1 -- 172.21.15.32:0/311895598 learned_addr learned my addr 172.21.15.32:0/311895598 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:48.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.796+0000 7f634e000640 1 --2- 172.21.15.32:0/311895598 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63500b4de0 0x7f635015baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:48.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.796+0000 7f634ea00640 1 -- 172.21.15.32:0/311895598 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63500b4de0 msgr2=0x7f635015baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:48.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.796+0000 7f634ea00640 1 --2- 172.21.15.32:0/311895598 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63500b4de0 0x7f635015baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:48.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.796+0000 7f634ea00640 1 -- 172.21.15.32:0/311895598 >> v1:172.21.15.144:6789/0 conn(0x7f635015bfe0 legacy=0x7f635006ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:48.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.796+0000 7f634ea00640 1 -- 172.21.15.32:0/311895598 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63500b3dc0 con 0x7f6350144e90 2024-04-17T09:55:48.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.800+0000 7f634ea00640 1 --2- 172.21.15.32:0/311895598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6350144e90 0x7f63500b48a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f63400098a0 tx=0x7f634001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65602e2d857051e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:48.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.800+0000 7f634d600640 1 -- 172.21.15.32:0/311895598 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63400a95f0 con 0x7f6350144e90 2024-04-17T09:55:48.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.800+0000 7f634d600640 1 -- 172.21.15.32:0/311895598 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63400a9790 con 0x7f6350144e90 2024-04-17T09:55:48.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.800+0000 7f634d600640 1 -- 172.21.15.32:0/311895598 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63400a9b60 con 0x7f6350144e90 2024-04-17T09:55:48.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.800+0000 7f6355a00640 1 -- 172.21.15.32:0/311895598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6350144e90 msgr2=0x7f63500b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:48.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.800+0000 7f6355a00640 1 --2- 172.21.15.32:0/311895598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6350144e90 0x7f63500b48a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f63400098a0 tx=0x7f634001c510 comp rx=0 tx=0).stop 2024-04-17T09:55:48.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.800+0000 7f6355a00640 1 -- 172.21.15.32:0/311895598 shutdown_connections 2024-04-17T09:55:48.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.800+0000 7f6355a00640 1 --2- 172.21.15.32:0/311895598 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63500b4de0 0x7f635015baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:48.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.800+0000 7f6355a00640 1 --2- 172.21.15.32:0/311895598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6350144e90 0x7f63500b48a0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:48.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.800+0000 7f6355a00640 1 -- 172.21.15.32:0/311895598 >> 172.21.15.32:0/311895598 conn(0x7f635006ba30 msgr2=0x7f635006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:48.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.800+0000 7f6355a00640 1 -- 172.21.15.32:0/311895598 shutdown_connections 2024-04-17T09:55:48.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.800+0000 7f6355a00640 1 -- 172.21.15.32:0/311895598 wait complete. 2024-04-17T09:55:48.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.800+0000 7f6355a00640 1 Processor -- start 2024-04-17T09:55:48.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.800+0000 7f6355a00640 1 -- start start 2024-04-17T09:55:48.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f6355a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63500b4de0 0x7f63500ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:48.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f6355a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6350144e90 0x7f6350143460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:48.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f6355a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f635015bfe0 0x7f63501439a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:48.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f6355a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6350072670 con 0x7f635015bfe0 2024-04-17T09:55:48.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f6355a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f635006f8c0 con 0x7f63500b4de0 2024-04-17T09:55:48.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f6355a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f63500727f0 con 0x7f6350144e90 2024-04-17T09:55:48.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f634f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f635015bfe0 0x7f63501439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:48.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f634f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f635015bfe0 0x7f63501439a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56160/0 (socket says 172.21.15.32:56160) 2024-04-17T09:55:48.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f634f400640 1 -- 172.21.15.32:0/3282998497 learned_addr learned my addr 172.21.15.32:0/3282998497 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:48.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f634ea00640 1 --2- 172.21.15.32:0/3282998497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63500b4de0 0x7f63500ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:48.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f634e000640 1 --2- 172.21.15.32:0/3282998497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6350144e90 0x7f6350143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:48.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f634f400640 1 -- 172.21.15.32:0/3282998497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6350144e90 msgr2=0x7f6350143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:48.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f634f400640 1 --2- 172.21.15.32:0/3282998497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6350144e90 0x7f6350143460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:48.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f634f400640 1 -- 172.21.15.32:0/3282998497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63500b4de0 msgr2=0x7f63500ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:48.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f634f400640 1 --2- 172.21.15.32:0/3282998497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63500b4de0 0x7f63500ae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:48.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f634f400640 1 -- 172.21.15.32:0/3282998497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f635013ff10 con 0x7f635015bfe0 2024-04-17T09:55:48.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f634f400640 1 --2- 172.21.15.32:0/3282998497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f635015bfe0 0x7f63501439a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f634800c730 tx=0x7f634800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:48.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f6346a00640 1 -- 172.21.15.32:0/3282998497 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6348003b80 con 0x7f635015bfe0 2024-04-17T09:55:48.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f6346a00640 1 -- 172.21.15.32:0/3282998497 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6348003d20 con 0x7f635015bfe0 2024-04-17T09:55:48.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f6346a00640 1 -- 172.21.15.32:0/3282998497 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634800aa30 con 0x7f635015bfe0 2024-04-17T09:55:48.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f6355a00640 1 -- 172.21.15.32:0/3282998497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63501401f0 con 0x7f635015bfe0 2024-04-17T09:55:48.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f6355a00640 1 -- 172.21.15.32:0/3282998497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63500ae7b0 con 0x7f635015bfe0 2024-04-17T09:55:48.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f6355a00640 1 -- 172.21.15.32:0/3282998497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6350068a90 con 0x7f635015bfe0 2024-04-17T09:55:48.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.804+0000 7f6346a00640 1 -- 172.21.15.32:0/3282998497 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f634800abd0 con 0x7f635015bfe0 2024-04-17T09:55:48.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.808+0000 7f6346a00640 1 --2- 172.21.15.32:0/3282998497 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6330068380 0x7f633006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:48.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.808+0000 7f6346a00640 1 -- 172.21.15.32:0/3282998497 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f6348086710 con 0x7f635015bfe0 2024-04-17T09:55:48.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.808+0000 7f634ea00640 1 --2- 172.21.15.32:0/3282998497 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6330068380 0x7f633006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:48.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.808+0000 7f634ea00640 1 --2- 172.21.15.32:0/3282998497 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6330068380 0x7f633006a840 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f63400ae000 tx=0x7f63400b3040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:48.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:48.812+0000 7f6346a00640 1 -- 172.21.15.32:0/3282998497 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f6348056670 con 0x7f635015bfe0 2024-04-17T09:55:49.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.016+0000 7f6355a00640 1 -- 172.21.15.32:0/3282998497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63500cd0c0 con 0x7f635015bfe0 2024-04-17T09:55:49.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.016+0000 7f6346a00640 1 -- 172.21.15.32:0/3282998497 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5202 (secure 0 0 0) 0x7f634805a970 con 0x7f635015bfe0 2024-04-17T09:55:49.019 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:49.020 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2542578926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2542578926},{"type":"v1","addr":"172.21.15.144:6834","nonce":2542578926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1424948520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1424948520},{"type":"v1","addr":"172.21.15.32:6836","nonce":1424948520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/393813154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":393813154},{"type":"v1","addr":"172.21.15.32:6837","nonce":393813154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:55:03.233552+0000","modified":"2024-04-17T09:55:06.243321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5742":{"gid":5742,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/2177384675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2177384675},{"type":"v1","addr":"172.21.15.144:6835","nonce":2177384675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742]},"id":3}]} 2024-04-17T09:55:49.020 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2024-04-17T09:55:49.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.020+0000 7f6355a00640 1 -- 172.21.15.32:0/3282998497 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6330068380 msgr2=0x7f633006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:49.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.020+0000 7f6355a00640 1 --2- 172.21.15.32:0/3282998497 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6330068380 0x7f633006a840 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f63400ae000 tx=0x7f63400b3040 comp rx=0 tx=0).stop 2024-04-17T09:55:49.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.020+0000 7f6355a00640 1 -- 172.21.15.32:0/3282998497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f635015bfe0 msgr2=0x7f63501439a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:49.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.020+0000 7f6355a00640 1 --2- 172.21.15.32:0/3282998497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f635015bfe0 0x7f63501439a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f634800c730 tx=0x7f634800cbf0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.020+0000 7f6355a00640 1 -- 172.21.15.32:0/3282998497 shutdown_connections 2024-04-17T09:55:49.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.020+0000 7f6355a00640 1 --2- 172.21.15.32:0/3282998497 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6330068380 0x7f633006a840 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.020+0000 7f6355a00640 1 --2- 172.21.15.32:0/3282998497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6350144e90 0x7f6350143460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.020+0000 7f6355a00640 1 --2- 172.21.15.32:0/3282998497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f635015bfe0 0x7f63501439a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.020+0000 7f6355a00640 1 --2- 172.21.15.32:0/3282998497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63500b4de0 0x7f63500ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.020+0000 7f6355a00640 1 -- 172.21.15.32:0/3282998497 >> 172.21.15.32:0/3282998497 conn(0x7f635006ba30 msgr2=0x7f63500cd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:49.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.020+0000 7f6355a00640 1 -- 172.21.15.32:0/3282998497 shutdown_connections 2024-04-17T09:55:49.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.020+0000 7f6355a00640 1 -- 172.21.15.32:0/3282998497 wait complete. 2024-04-17T09:55:49.042 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:55:49.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.180+0000 7f82f3200640 1 Processor -- start 2024-04-17T09:55:49.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.180+0000 7f82f3200640 1 -- start start 2024-04-17T09:55:49.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.180+0000 7f82f3200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f82ec145f30 0x7f82ec071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:49.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.180+0000 7f82f3200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f82ec15d8d0 0x7f82ec072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:49.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.180+0000 7f82f3200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82ec0cce70 0x7f82ec0cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:49.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.180+0000 7f82f3200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f82ec06f400 con 0x7f82ec145f30 2024-04-17T09:55:49.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.180+0000 7f82f3200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f82ec146310 con 0x7f82ec15d8d0 2024-04-17T09:55:49.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.180+0000 7f82f3200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f82ec15d440 con 0x7f82ec0cce70 2024-04-17T09:55:49.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.180+0000 7f82ebe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f82ec145f30 0x7f82ec071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:49.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.180+0000 7f82ebe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f82ec145f30 0x7f82ec071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56174/0 (socket says 172.21.15.32:56174) 2024-04-17T09:55:49.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.180+0000 7f82ebe00640 1 -- 172.21.15.32:0/2564141815 learned_addr learned my addr 172.21.15.32:0/2564141815 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:49.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.180+0000 7f82e3e00640 1 --2- 172.21.15.32:0/2564141815 >> v2:172.21.15.144:3300/0 conn(0x7f82ec15d8d0 0x7f82ec072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:49.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.180+0000 7f82f1200640 1 --2- 172.21.15.32:0/2564141815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82ec0cce70 0x7f82ec0cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:49.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.180+0000 7f82ebe00640 1 -- 172.21.15.32:0/2564141815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82ec0cce70 msgr2=0x7f82ec0cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:49.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.180+0000 7f82ebe00640 1 --2- 172.21.15.32:0/2564141815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82ec0cce70 0x7f82ec0cd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.180+0000 7f82ebe00640 1 -- 172.21.15.32:0/2564141815 >> v2:172.21.15.144:3300/0 conn(0x7f82ec15d8d0 msgr2=0x7f82ec072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:49.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.180+0000 7f82ebe00640 1 --2- 172.21.15.32:0/2564141815 >> v2:172.21.15.144:3300/0 conn(0x7f82ec15d8d0 0x7f82ec072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.184+0000 7f82ebe00640 1 -- 172.21.15.32:0/2564141815 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ec0cd820 con 0x7f82ec145f30 2024-04-17T09:55:49.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.184+0000 7f82f1200640 1 --2- 172.21.15.32:0/2564141815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82ec0cce70 0x7f82ec0cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:49.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.184+0000 7f82e3e00640 1 --2- 172.21.15.32:0/2564141815 >> v2:172.21.15.144:3300/0 conn(0x7f82ec15d8d0 0x7f82ec072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:49.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.184+0000 7f82ebe00640 1 --2- 172.21.15.32:0/2564141815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82ec145f30 0x7f82ec071df0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0702a0 tx=0x7f82dc01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1f56abb6697e94 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:49.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.184+0000 7f82eb400640 1 -- 172.21.15.32:0/2564141815 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc0b5070 con 0x7f82ec145f30 2024-04-17T09:55:49.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.184+0000 7f82eb400640 1 -- 172.21.15.32:0/2564141815 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82dc0a89d0 con 0x7f82ec145f30 2024-04-17T09:55:49.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.184+0000 7f82eb400640 1 -- 172.21.15.32:0/2564141815 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc0a8da0 con 0x7f82ec145f30 2024-04-17T09:55:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.184+0000 7f82f3200640 1 -- 172.21.15.32:0/2564141815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82ec145f30 msgr2=0x7f82ec071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.184+0000 7f82f3200640 1 --2- 172.21.15.32:0/2564141815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82ec145f30 0x7f82ec071df0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0702a0 tx=0x7f82dc01ed10 comp rx=0 tx=0).stop 2024-04-17T09:55:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82f3200640 1 -- 172.21.15.32:0/2564141815 shutdown_connections 2024-04-17T09:55:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82f3200640 1 --2- 172.21.15.32:0/2564141815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82ec0cce70 0x7f82ec0cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82f3200640 1 --2- 172.21.15.32:0/2564141815 >> v2:172.21.15.144:3300/0 conn(0x7f82ec15d8d0 0x7f82ec072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82f3200640 1 --2- 172.21.15.32:0/2564141815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82ec145f30 0x7f82ec071df0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82f3200640 1 -- 172.21.15.32:0/2564141815 >> 172.21.15.32:0/2564141815 conn(0x7f82ec06ba30 msgr2=0x7f82ec06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82f3200640 1 -- 172.21.15.32:0/2564141815 shutdown_connections 2024-04-17T09:55:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82f3200640 1 -- 172.21.15.32:0/2564141815 wait complete. 2024-04-17T09:55:49.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82f3200640 1 Processor -- start 2024-04-17T09:55:49.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82f3200640 1 -- start start 2024-04-17T09:55:49.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82f3200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82ec0cce70 0x7f82ec1407e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:49.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82f3200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82ec145f30 0x7f82ec0cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:49.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82f3200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82ec15d8d0 0x7f82ec15baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:49.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82f3200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec15cdf0 con 0x7f82ec0cce70 2024-04-17T09:55:49.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82f3200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec15cb90 con 0x7f82ec15d8d0 2024-04-17T09:55:49.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82f3200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f82ec15d0d0 con 0x7f82ec145f30 2024-04-17T09:55:49.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82ebe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82ec0cce70 0x7f82ec1407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:49.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82ebe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82ec0cce70 0x7f82ec1407e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56186/0 (socket says 172.21.15.32:56186) 2024-04-17T09:55:49.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82ebe00640 1 -- 172.21.15.32:0/255694280 learned_addr learned my addr 172.21.15.32:0/255694280 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:49.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82e3e00640 1 --2- 172.21.15.32:0/255694280 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82ec145f30 0x7f82ec0cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:49.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82f1200640 1 --2- 172.21.15.32:0/255694280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82ec15d8d0 0x7f82ec15baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:49.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82ebe00640 1 -- 172.21.15.32:0/255694280 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82ec145f30 msgr2=0x7f82ec0cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:49.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82ebe00640 1 --2- 172.21.15.32:0/255694280 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82ec145f30 0x7f82ec0cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82ebe00640 1 -- 172.21.15.32:0/255694280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82ec15d8d0 msgr2=0x7f82ec15baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:49.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82ebe00640 1 --2- 172.21.15.32:0/255694280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82ec15d8d0 0x7f82ec15baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82ebe00640 1 -- 172.21.15.32:0/255694280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ec143210 con 0x7f82ec0cce70 2024-04-17T09:55:49.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82ebe00640 1 --2- 172.21.15.32:0/255694280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82ec0cce70 0x7f82ec1407e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0b3040 tx=0x7f82dc01bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:49.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82e8c00640 1 -- 172.21.15.32:0/255694280 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc0b5070 con 0x7f82ec0cce70 2024-04-17T09:55:49.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82e8c00640 1 -- 172.21.15.32:0/255694280 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82dc0b0a70 con 0x7f82ec0cce70 2024-04-17T09:55:49.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82e8c00640 1 -- 172.21.15.32:0/255694280 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc0b0e60 con 0x7f82ec0cce70 2024-04-17T09:55:49.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.188+0000 7f82f3200640 1 -- 172.21.15.32:0/255694280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ec0cad10 con 0x7f82ec0cce70 2024-04-17T09:55:49.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.192+0000 7f82f3200640 1 -- 172.21.15.32:0/255694280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ec0cb2d0 con 0x7f82ec0cce70 2024-04-17T09:55:49.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.192+0000 7f82f3200640 1 -- 172.21.15.32:0/255694280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82ec068a90 con 0x7f82ec0cce70 2024-04-17T09:55:49.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.192+0000 7f82e8c00640 1 -- 172.21.15.32:0/255694280 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f82dc0c2060 con 0x7f82ec0cce70 2024-04-17T09:55:49.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.192+0000 7f82e8c00640 1 --2- 172.21.15.32:0/255694280 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f82c8068380 0x7f82c806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:49.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.192+0000 7f82e8c00640 1 -- 172.21.15.32:0/255694280 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f82dc123b90 con 0x7f82ec0cce70 2024-04-17T09:55:49.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.192+0000 7f82e3e00640 1 --2- 172.21.15.32:0/255694280 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f82c8068380 0x7f82c806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:49.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.192+0000 7f82e3e00640 1 --2- 172.21.15.32:0/255694280 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f82c8068380 0x7f82c806a840 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f82ec160350 tx=0x7f82d4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:49.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.196+0000 7f82e8c00640 1 -- 172.21.15.32:0/255694280 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f82dc018870 con 0x7f82ec0cce70 2024-04-17T09:55:49.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.396+0000 7f82f3200640 1 -- 172.21.15.32:0/255694280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82ec06fa40 con 0x7f82ec0cce70 2024-04-17T09:55:49.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.400+0000 7f82e8c00640 1 -- 172.21.15.32:0/255694280 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5202 (secure 0 0 0) 0x7f82dc0c2350 con 0x7f82ec0cce70 2024-04-17T09:55:49.400 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:49.400 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2542578926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2542578926},{"type":"v1","addr":"172.21.15.144:6834","nonce":2542578926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1424948520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1424948520},{"type":"v1","addr":"172.21.15.32:6836","nonce":1424948520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/393813154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":393813154},{"type":"v1","addr":"172.21.15.32:6837","nonce":393813154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:55:03.233552+0000","modified":"2024-04-17T09:55:06.243321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5742":{"gid":5742,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/2177384675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2177384675},{"type":"v1","addr":"172.21.15.144:6835","nonce":2177384675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742]},"id":3}]} 2024-04-17T09:55:49.401 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2024-04-17T09:55:49.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.400+0000 7f82f3200640 1 -- 172.21.15.32:0/255694280 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f82c8068380 msgr2=0x7f82c806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:49.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.400+0000 7f82f3200640 1 --2- 172.21.15.32:0/255694280 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f82c8068380 0x7f82c806a840 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f82ec160350 tx=0x7f82d4008040 comp rx=0 tx=0).stop 2024-04-17T09:55:49.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.400+0000 7f82f3200640 1 -- 172.21.15.32:0/255694280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82ec0cce70 msgr2=0x7f82ec1407e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:49.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.400+0000 7f82f3200640 1 --2- 172.21.15.32:0/255694280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82ec0cce70 0x7f82ec1407e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0b3040 tx=0x7f82dc01bab0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.400+0000 7f82f3200640 1 -- 172.21.15.32:0/255694280 shutdown_connections 2024-04-17T09:55:49.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.400+0000 7f82f3200640 1 --2- 172.21.15.32:0/255694280 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f82c8068380 0x7f82c806a840 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.400+0000 7f82f3200640 1 --2- 172.21.15.32:0/255694280 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82ec145f30 0x7f82ec0cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.400+0000 7f82f3200640 1 --2- 172.21.15.32:0/255694280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82ec15d8d0 0x7f82ec15baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.400+0000 7f82f3200640 1 --2- 172.21.15.32:0/255694280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82ec0cce70 0x7f82ec1407e0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.400+0000 7f82f3200640 1 -- 172.21.15.32:0/255694280 >> 172.21.15.32:0/255694280 conn(0x7f82ec06ba30 msgr2=0x7f82ec0b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.400+0000 7f82f3200640 1 -- 172.21.15.32:0/255694280 shutdown_connections 2024-04-17T09:55:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.400+0000 7f82f3200640 1 -- 172.21.15.32:0/255694280 wait complete. 2024-04-17T09:55:49.421 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:55:49.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.560+0000 7fb567e00640 1 Processor -- start 2024-04-17T09:55:49.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.560+0000 7fb567e00640 1 -- start start 2024-04-17T09:55:49.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.560+0000 7fb567e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb560144e90 0x7fb5600b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:49.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.560+0000 7fb567e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb56015c1b0 0x7fb56006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:49.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.560+0000 7fb567e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb560072bb0 con 0x7fb5600b4de0 2024-04-17T09:55:49.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.560+0000 7fb567e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb560145270 con 0x7fb560144e90 2024-04-17T09:55:49.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.560+0000 7fb567e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb5600b3dc0 con 0x7fb56015c1b0 2024-04-17T09:55:49.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.560+0000 7fb565400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb560144e90 0x7fb5600b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:49.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.560+0000 7fb565400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb560144e90 0x7fb5600b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56188/0 (socket says 172.21.15.32:56188) 2024-04-17T09:55:49.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.560+0000 7fb565400640 1 -- 172.21.15.32:0/1725020094 learned_addr learned my addr 172.21.15.32:0/1725020094 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:49.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.560+0000 7fb565e00640 1 --2- 172.21.15.32:0/1725020094 >> v2:172.21.15.144:3300/0 conn(0x7fb56015c1b0 0x7fb56006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:49.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.560+0000 7fb565400640 1 -- 172.21.15.32:0/1725020094 >> v2:172.21.15.144:3300/0 conn(0x7fb56015c1b0 msgr2=0x7fb56006ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:49.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.560+0000 7fb565400640 1 --2- 172.21.15.32:0/1725020094 >> v2:172.21.15.144:3300/0 conn(0x7fb56015c1b0 0x7fb56006ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.560+0000 7fb565400640 1 -- 172.21.15.32:0/1725020094 >> v1:172.21.15.144:6789/0 conn(0x7fb5600b4de0 legacy=0x7fb56015baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:49.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.560+0000 7fb565400640 1 -- 172.21.15.32:0/1725020094 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5600b40a0 con 0x7fb560144e90 2024-04-17T09:55:49.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.564+0000 7fb565400640 1 --2- 172.21.15.32:0/1725020094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb560144e90 0x7fb5600b48a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb5600b3a00 tx=0x7fb55401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1dcd126091311f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:49.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.564+0000 7fb55fe00640 1 -- 172.21.15.32:0/1725020094 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5540ab600 con 0x7fb560144e90 2024-04-17T09:55:49.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.564+0000 7fb55fe00640 1 -- 172.21.15.32:0/1725020094 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5540b0070 con 0x7fb560144e90 2024-04-17T09:55:49.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.564+0000 7fb55fe00640 1 -- 172.21.15.32:0/1725020094 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5540ab9c0 con 0x7fb560144e90 2024-04-17T09:55:49.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.564+0000 7fb567e00640 1 -- 172.21.15.32:0/1725020094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb560144e90 msgr2=0x7fb5600b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:49.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.564+0000 7fb567e00640 1 --2- 172.21.15.32:0/1725020094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb560144e90 0x7fb5600b48a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb5600b3a00 tx=0x7fb55401c4f0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.564+0000 7fb567e00640 1 -- 172.21.15.32:0/1725020094 shutdown_connections 2024-04-17T09:55:49.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.564+0000 7fb567e00640 1 --2- 172.21.15.32:0/1725020094 >> v2:172.21.15.144:3300/0 conn(0x7fb56015c1b0 0x7fb56006ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.564+0000 7fb567e00640 1 --2- 172.21.15.32:0/1725020094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb560144e90 0x7fb5600b48a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.564+0000 7fb567e00640 1 -- 172.21.15.32:0/1725020094 >> 172.21.15.32:0/1725020094 conn(0x7fb56006ba30 msgr2=0x7fb56006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:49.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.564+0000 7fb567e00640 1 -- 172.21.15.32:0/1725020094 shutdown_connections 2024-04-17T09:55:49.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.564+0000 7fb567e00640 1 -- 172.21.15.32:0/1725020094 wait complete. 2024-04-17T09:55:49.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb567e00640 1 Processor -- start 2024-04-17T09:55:49.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb567e00640 1 -- start start 2024-04-17T09:55:49.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb567e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5600b4de0 0x7fb56015de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:49.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb567e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb560144e90 0x7fb5601402a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:49.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb567e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb56015c1b0 0x7fb56013e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:49.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb567e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb560168110 con 0x7fb56015c1b0 2024-04-17T09:55:49.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb567e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb560072880 con 0x7fb5600b4de0 2024-04-17T09:55:49.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb567e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb560168290 con 0x7fb560144e90 2024-04-17T09:55:49.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb565e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb56015c1b0 0x7fb56013e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:49.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb565e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb56015c1b0 0x7fb56013e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56198/0 (socket says 172.21.15.32:56198) 2024-04-17T09:55:49.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb565e00640 1 -- 172.21.15.32:0/3453407027 learned_addr learned my addr 172.21.15.32:0/3453407027 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:49.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb565400640 1 --2- 172.21.15.32:0/3453407027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5600b4de0 0x7fb56015de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:49.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb564a00640 1 --2- 172.21.15.32:0/3453407027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb560144e90 0x7fb5601402a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:49.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb565e00640 1 -- 172.21.15.32:0/3453407027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb560144e90 msgr2=0x7fb5601402a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:49.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb565e00640 1 --2- 172.21.15.32:0/3453407027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb560144e90 0x7fb5601402a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb565e00640 1 -- 172.21.15.32:0/3453407027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5600b4de0 msgr2=0x7fb56015de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:49.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb565e00640 1 --2- 172.21.15.32:0/3453407027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5600b4de0 0x7fb56015de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb565e00640 1 -- 172.21.15.32:0/3453407027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb56015f280 con 0x7fb56015c1b0 2024-04-17T09:55:49.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb565e00640 1 --2- 172.21.15.32:0/3453407027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb56015c1b0 0x7fb56013e830 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb558007e90 tx=0x7fb55800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:49.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb55d600640 1 -- 172.21.15.32:0/3453407027 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558014420 con 0x7fb56015c1b0 2024-04-17T09:55:49.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb55d600640 1 -- 172.21.15.32:0/3453407027 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb558019070 con 0x7fb56015c1b0 2024-04-17T09:55:49.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb55d600640 1 -- 172.21.15.32:0/3453407027 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558014800 con 0x7fb56015c1b0 2024-04-17T09:55:49.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb567e00640 1 -- 172.21.15.32:0/3453407027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb56015f560 con 0x7fb56015c1b0 2024-04-17T09:55:49.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb567e00640 1 -- 172.21.15.32:0/3453407027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb56015e3b0 con 0x7fb56015c1b0 2024-04-17T09:55:49.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.568+0000 7fb567e00640 1 -- 172.21.15.32:0/3453407027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb56015eff0 con 0x7fb56015c1b0 2024-04-17T09:55:49.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.572+0000 7fb55d600640 1 -- 172.21.15.32:0/3453407027 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb5580149a0 con 0x7fb56015c1b0 2024-04-17T09:55:49.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.572+0000 7fb55d600640 1 --2- 172.21.15.32:0/3453407027 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb5400683b0 0x7fb54006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:49.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.572+0000 7fb55d600640 1 -- 172.21.15.32:0/3453407027 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fb558087060 con 0x7fb56015c1b0 2024-04-17T09:55:49.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.572+0000 7fb565400640 1 --2- 172.21.15.32:0/3453407027 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb5400683b0 0x7fb54006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:49.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.572+0000 7fb565400640 1 --2- 172.21.15.32:0/3453407027 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb5400683b0 0x7fb54006a870 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb5540af000 tx=0x7fb5540a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:49.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.576+0000 7fb55d600640 1 -- 172.21.15.32:0/3453407027 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fb558056fc0 con 0x7fb56015c1b0 2024-04-17T09:55:49.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.740+0000 7fb567e00640 1 -- 172.21.15.32:0/3453407027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb560068a90 con 0x7fb56015c1b0 2024-04-17T09:55:49.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.744+0000 7fb55d600640 1 -- 172.21.15.32:0/3453407027 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16625 (secure 0 0 0) 0x7fb55805b2c0 con 0x7fb56015c1b0 2024-04-17T09:55:49.744 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:49.744 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":46,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:55:04.834732+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:55:01.624737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-17T09:55:02.185212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:55:49.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.744+0000 7fb567e00640 1 -- 172.21.15.32:0/3453407027 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb5400683b0 msgr2=0x7fb54006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:49.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.744+0000 7fb567e00640 1 --2- 172.21.15.32:0/3453407027 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb5400683b0 0x7fb54006a870 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb5540af000 tx=0x7fb5540a7000 comp rx=0 tx=0).stop 2024-04-17T09:55:49.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.744+0000 7fb567e00640 1 -- 172.21.15.32:0/3453407027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb56015c1b0 msgr2=0x7fb56013e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:49.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.744+0000 7fb567e00640 1 --2- 172.21.15.32:0/3453407027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb56015c1b0 0x7fb56013e830 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb558007e90 tx=0x7fb55800e4b0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.744+0000 7fb567e00640 1 -- 172.21.15.32:0/3453407027 shutdown_connections 2024-04-17T09:55:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.744+0000 7fb567e00640 1 --2- 172.21.15.32:0/3453407027 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb5400683b0 0x7fb54006a870 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.744+0000 7fb567e00640 1 --2- 172.21.15.32:0/3453407027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb560144e90 0x7fb5601402a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.744+0000 7fb567e00640 1 --2- 172.21.15.32:0/3453407027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb56015c1b0 0x7fb56013e830 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.744+0000 7fb567e00640 1 --2- 172.21.15.32:0/3453407027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5600b4de0 0x7fb56015de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.744+0000 7fb567e00640 1 -- 172.21.15.32:0/3453407027 >> 172.21.15.32:0/3453407027 conn(0x7fb56006ba30 msgr2=0x7fb560144160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.744+0000 7fb567e00640 1 -- 172.21.15.32:0/3453407027 shutdown_connections 2024-04-17T09:55:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.744+0000 7fb567e00640 1 -- 172.21.15.32:0/3453407027 wait complete. 2024-04-17T09:55:49.765 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T09:55:49.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.904+0000 7f343c200640 1 Processor -- start 2024-04-17T09:55:49.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.904+0000 7f343c200640 1 -- start start 2024-04-17T09:55:49.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.908+0000 7f343c200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f343415c290 0x7f34340d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:49.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.908+0000 7f343c200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f34340d00f0 0x7f34340af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:49.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.908+0000 7f343c200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f343415c670 0x7f34340af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:49.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.908+0000 7f343c200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3434071eb0 con 0x7f343415c290 2024-04-17T09:55:49.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.908+0000 7f343c200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f3434072190 con 0x7f34340d00f0 2024-04-17T09:55:49.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.908+0000 7f343c200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3434072470 con 0x7f343415c670 2024-04-17T09:55:49.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.908+0000 7f3439800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f343415c290 0x7f34340d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:49.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.908+0000 7f3439800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f343415c290 0x7f34340d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56204/0 (socket says 172.21.15.32:56204) 2024-04-17T09:55:49.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.908+0000 7f3439800640 1 -- 172.21.15.32:0/3867178678 learned_addr learned my addr 172.21.15.32:0/3867178678 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:49.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.908+0000 7f3438e00640 1 --2- 172.21.15.32:0/3867178678 >> v2:172.21.15.144:3300/0 conn(0x7f34340d00f0 0x7f34340af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:49.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.908+0000 7f343a200640 1 --2- 172.21.15.32:0/3867178678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f343415c670 0x7f34340af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:49.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.908+0000 7f3439800640 1 -- 172.21.15.32:0/3867178678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f343415c670 msgr2=0x7f34340af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:49.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.908+0000 7f3439800640 1 --2- 172.21.15.32:0/3867178678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f343415c670 0x7f34340af8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.908+0000 7f3439800640 1 -- 172.21.15.32:0/3867178678 >> v2:172.21.15.144:3300/0 conn(0x7f34340d00f0 msgr2=0x7f34340af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:49.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.908+0000 7f3439800640 1 --2- 172.21.15.32:0/3867178678 >> v2:172.21.15.144:3300/0 conn(0x7f34340d00f0 0x7f34340af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.908+0000 7f3439800640 1 -- 172.21.15.32:0/3867178678 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3434072750 con 0x7f343415c290 2024-04-17T09:55:49.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.912+0000 7f3439800640 1 --2- 172.21.15.32:0/3867178678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f343415c290 0x7f34340d0720 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f343406c8b0 tx=0x7f342801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879597cc3ded9b96 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:49.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.912+0000 7f3433e00640 1 -- 172.21.15.32:0/3867178678 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34280b0070 con 0x7f343415c290 2024-04-17T09:55:49.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.912+0000 7f3433e00640 1 -- 172.21.15.32:0/3867178678 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34280a4e50 con 0x7f343415c290 2024-04-17T09:55:49.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.912+0000 7f3433e00640 1 -- 172.21.15.32:0/3867178678 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34280ab820 con 0x7f343415c290 2024-04-17T09:55:49.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.912+0000 7f343c200640 1 -- 172.21.15.32:0/3867178678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f343415c290 msgr2=0x7f34340d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:49.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.912+0000 7f343c200640 1 --2- 172.21.15.32:0/3867178678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f343415c290 0x7f34340d0720 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f343406c8b0 tx=0x7f342801c400 comp rx=0 tx=0).stop 2024-04-17T09:55:49.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.912+0000 7f343c200640 1 -- 172.21.15.32:0/3867178678 shutdown_connections 2024-04-17T09:55:49.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.912+0000 7f343c200640 1 --2- 172.21.15.32:0/3867178678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f343415c670 0x7f34340af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.912+0000 7f343c200640 1 --2- 172.21.15.32:0/3867178678 >> v2:172.21.15.144:3300/0 conn(0x7f34340d00f0 0x7f34340af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.912+0000 7f343c200640 1 --2- 172.21.15.32:0/3867178678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f343415c290 0x7f34340d0720 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.912+0000 7f343c200640 1 -- 172.21.15.32:0/3867178678 >> 172.21.15.32:0/3867178678 conn(0x7f343406ba30 msgr2=0x7f343406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:49.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.912+0000 7f343c200640 1 -- 172.21.15.32:0/3867178678 shutdown_connections 2024-04-17T09:55:49.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.912+0000 7f343c200640 1 -- 172.21.15.32:0/3867178678 wait complete. 2024-04-17T09:55:49.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.912+0000 7f343c200640 1 Processor -- start 2024-04-17T09:55:49.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.912+0000 7f343c200640 1 -- start start 2024-04-17T09:55:49.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.912+0000 7f343c200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34340d00f0 0x7f343414e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:49.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.912+0000 7f343c200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f343415c290 0x7f3434161b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:49.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.912+0000 7f343c200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f343415c670 0x7f343414ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:49.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.916+0000 7f343c200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3434162ff0 con 0x7f34340d00f0 2024-04-17T09:55:49.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.916+0000 7f343c200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3434162e70 con 0x7f343415c670 2024-04-17T09:55:49.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.916+0000 7f343c200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3434163170 con 0x7f343415c290 2024-04-17T09:55:49.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.916+0000 7f3439800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34340d00f0 0x7f343414e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:49.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.916+0000 7f3439800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34340d00f0 0x7f343414e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56208/0 (socket says 172.21.15.32:56208) 2024-04-17T09:55:49.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.916+0000 7f3439800640 1 -- 172.21.15.32:0/3622193737 learned_addr learned my addr 172.21.15.32:0/3622193737 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:49.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.916+0000 7f343a200640 1 --2- 172.21.15.32:0/3622193737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f343415c670 0x7f343414ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:49.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.916+0000 7f3438e00640 1 --2- 172.21.15.32:0/3622193737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f343415c290 0x7f3434161b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:49.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.916+0000 7f3439800640 1 -- 172.21.15.32:0/3622193737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f343415c290 msgr2=0x7f3434161b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:49.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.916+0000 7f3439800640 1 --2- 172.21.15.32:0/3622193737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f343415c290 0x7f3434161b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.916+0000 7f3439800640 1 -- 172.21.15.32:0/3622193737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f343415c670 msgr2=0x7f343414ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:49.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.916+0000 7f3439800640 1 --2- 172.21.15.32:0/3622193737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f343415c670 0x7f343414ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:49.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.916+0000 7f3439800640 1 -- 172.21.15.32:0/3622193737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34340cef60 con 0x7f34340d00f0 2024-04-17T09:55:49.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.916+0000 7f343a200640 1 --2- 172.21.15.32:0/3622193737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f343415c670 0x7f343414ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:49.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.916+0000 7f3439800640 1 --2- 172.21.15.32:0/3622193737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34340d00f0 0x7f343414e490 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f34280ae040 tx=0x7f34280191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:49.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.916+0000 7f3431600640 1 -- 172.21.15.32:0/3622193737 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34280b9070 con 0x7f34340d00f0 2024-04-17T09:55:49.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.916+0000 7f3431600640 1 -- 172.21.15.32:0/3622193737 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34280abe20 con 0x7f34340d00f0 2024-04-17T09:55:49.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.916+0000 7f343c200640 1 -- 172.21.15.32:0/3622193737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34340cf240 con 0x7f34340d00f0 2024-04-17T09:55:49.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.916+0000 7f3431600640 1 -- 172.21.15.32:0/3622193737 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34280b0040 con 0x7f34340d00f0 2024-04-17T09:55:49.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.916+0000 7f343c200640 1 -- 172.21.15.32:0/3622193737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f343415f330 con 0x7f34340d00f0 2024-04-17T09:55:49.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.916+0000 7f343c200640 1 -- 172.21.15.32:0/3622193737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34340cf760 con 0x7f34340d00f0 2024-04-17T09:55:49.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.916+0000 7f3431600640 1 -- 172.21.15.32:0/3622193737 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f34280bc070 con 0x7f34340d00f0 2024-04-17T09:55:49.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.916+0000 7f3431600640 1 --2- 172.21.15.32:0/3622193737 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3414068380 0x7f341406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:49.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.920+0000 7f3431600640 1 -- 172.21.15.32:0/3622193737 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f342811f9d0 con 0x7f34340d00f0 2024-04-17T09:55:49.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.920+0000 7f3438e00640 1 --2- 172.21.15.32:0/3622193737 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3414068380 0x7f341406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:49.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.920+0000 7f3438e00640 1 --2- 172.21.15.32:0/3622193737 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3414068380 0x7f341406a840 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f34340b1790 tx=0x7f3420008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:49.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:49.924+0000 7f3431600640 1 -- 172.21.15.32:0/3622193737 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f34340cf760 con 0x7f34340d00f0 2024-04-17T09:55:50.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.124+0000 7f343c200640 1 -- 172.21.15.32:0/3622193737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3434068a90 con 0x7f34340d00f0 2024-04-17T09:55:50.125 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-17T09:55:50.124+0000 7f93c9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T09:55:50.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.124+0000 7f3431600640 1 -- 172.21.15.32:0/3622193737 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) ==== 161+0+0 (secure 0 0 0) 0x7f3428016110 con 0x7f34340d00f0 2024-04-17T09:55:50.127 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T09:55:50.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.128+0000 7f343c200640 1 -- 172.21.15.32:0/3622193737 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3414068380 msgr2=0x7f341406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:50.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.128+0000 7f343c200640 1 --2- 172.21.15.32:0/3622193737 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3414068380 0x7f341406a840 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f34340b1790 tx=0x7f3420008040 comp rx=0 tx=0).stop 2024-04-17T09:55:50.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.128+0000 7f343c200640 1 -- 172.21.15.32:0/3622193737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34340d00f0 msgr2=0x7f343414e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:50.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.128+0000 7f343c200640 1 --2- 172.21.15.32:0/3622193737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34340d00f0 0x7f343414e490 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f34280ae040 tx=0x7f34280191a0 comp rx=0 tx=0).stop 2024-04-17T09:55:50.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.128+0000 7f343c200640 1 -- 172.21.15.32:0/3622193737 shutdown_connections 2024-04-17T09:55:50.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.128+0000 7f343c200640 1 --2- 172.21.15.32:0/3622193737 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3414068380 0x7f341406a840 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:50.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.128+0000 7f343c200640 1 --2- 172.21.15.32:0/3622193737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f343415c290 0x7f3434161b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:50.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.128+0000 7f343c200640 1 --2- 172.21.15.32:0/3622193737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f343415c670 0x7f343414ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:50.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.128+0000 7f343c200640 1 --2- 172.21.15.32:0/3622193737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34340d00f0 0x7f343414e490 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:50.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.128+0000 7f343c200640 1 -- 172.21.15.32:0/3622193737 >> 172.21.15.32:0/3622193737 conn(0x7f343406ba30 msgr2=0x7f34340ae300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:50.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.128+0000 7f343c200640 1 -- 172.21.15.32:0/3622193737 shutdown_connections 2024-04-17T09:55:50.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.128+0000 7f343c200640 1 -- 172.21.15.32:0/3622193737 wait complete. 2024-04-17T09:55:50.141 INFO:tasks.ceph.mds.d.smithi144.stderr: -5002> 2024-04-17T09:55:05.813+0000 7fb02ea00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T09:55:50.146 INFO:tasks.ceph.mds.d.smithi144.stderr: -3225> 2024-04-17T09:55:12.697+0000 7fb029a00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5904=0-4194304@1} caps={5904=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5904 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x557ad47f2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-17T09:55:50.149 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-17T09:55:50.194 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2024-04-17T09:55:50.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.292+0000 7fe56b600640 1 Processor -- start 2024-04-17T09:55:50.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.292+0000 7fe56b600640 1 -- start start 2024-04-17T09:55:50.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.292+0000 7fe56b600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe56415c4c0 0x7fe5640adb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:50.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.292+0000 7fe56b600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe56415f9c0 0x7fe56406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:50.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.292+0000 7fe56b600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe56406bf70 0x7fe564071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:50.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.292+0000 7fe56b600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe56406c3a0 con 0x7fe56406bf70 2024-04-17T09:55:50.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.292+0000 7fe56b600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe5641629a0 con 0x7fe56415f9c0 2024-04-17T09:55:50.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.292+0000 7fe56b600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe564162c80 con 0x7fe56415c4c0 2024-04-17T09:55:50.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.292+0000 7fe569600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe56406bf70 0x7fe564071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:50.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.292+0000 7fe569600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe56406bf70 0x7fe564071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56212/0 (socket says 172.21.15.32:56212) 2024-04-17T09:55:50.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.292+0000 7fe569600640 1 -- 172.21.15.32:0/2892298103 learned_addr learned my addr 172.21.15.32:0/2892298103 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:50.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.292+0000 7fe55be00640 1 --2- 172.21.15.32:0/2892298103 >> v2:172.21.15.144:3300/0 conn(0x7fe56415f9c0 0x7fe56406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:50.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.292+0000 7fe568c00640 1 --2- 172.21.15.32:0/2892298103 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe56415c4c0 0x7fe5640adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:50.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.292+0000 7fe569600640 1 -- 172.21.15.32:0/2892298103 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe56415c4c0 msgr2=0x7fe5640adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:50.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.292+0000 7fe569600640 1 --2- 172.21.15.32:0/2892298103 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe56415c4c0 0x7fe5640adb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:50.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.292+0000 7fe569600640 1 -- 172.21.15.32:0/2892298103 >> v2:172.21.15.144:3300/0 conn(0x7fe56415f9c0 msgr2=0x7fe56406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:50.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.292+0000 7fe569600640 1 --2- 172.21.15.32:0/2892298103 >> v2:172.21.15.144:3300/0 conn(0x7fe56415f9c0 0x7fe56406ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:50.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.292+0000 7fe569600640 1 -- 172.21.15.32:0/2892298103 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe564162f60 con 0x7fe56406bf70 2024-04-17T09:55:50.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.296+0000 7fe569600640 1 --2- 172.21.15.32:0/2892298103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe56406bf70 0x7fe564071e60 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe56406ffd0 tx=0x7fe55c01c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520ee5c8d7e5a1df server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:50.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.296+0000 7fe563e00640 1 -- 172.21.15.32:0/2892298103 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c0a64d0 con 0x7fe56406bf70 2024-04-17T09:55:50.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.296+0000 7fe563e00640 1 -- 172.21.15.32:0/2892298103 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe55c0a6670 con 0x7fe56406bf70 2024-04-17T09:55:50.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.296+0000 7fe563e00640 1 -- 172.21.15.32:0/2892298103 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c0a6a40 con 0x7fe56406bf70 2024-04-17T09:55:50.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.296+0000 7fe56b600640 1 -- 172.21.15.32:0/2892298103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe56406bf70 msgr2=0x7fe564071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:50.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.296+0000 7fe56b600640 1 --2- 172.21.15.32:0/2892298103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe56406bf70 0x7fe564071e60 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe56406ffd0 tx=0x7fe55c01c730 comp rx=0 tx=0).stop 2024-04-17T09:55:50.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.296+0000 7fe56b600640 1 -- 172.21.15.32:0/2892298103 shutdown_connections 2024-04-17T09:55:50.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.296+0000 7fe56b600640 1 --2- 172.21.15.32:0/2892298103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe56406bf70 0x7fe564071e60 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:50.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.296+0000 7fe56b600640 1 --2- 172.21.15.32:0/2892298103 >> v2:172.21.15.144:3300/0 conn(0x7fe56415f9c0 0x7fe56406ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:50.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.296+0000 7fe56b600640 1 --2- 172.21.15.32:0/2892298103 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe56415c4c0 0x7fe5640adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:50.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.296+0000 7fe56b600640 1 -- 172.21.15.32:0/2892298103 >> 172.21.15.32:0/2892298103 conn(0x7fe5640b2120 msgr2=0x7fe5640b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:50.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.296+0000 7fe56b600640 1 -- 172.21.15.32:0/2892298103 shutdown_connections 2024-04-17T09:55:50.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.296+0000 7fe56b600640 1 -- 172.21.15.32:0/2892298103 wait complete. 2024-04-17T09:55:50.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.296+0000 7fe56b600640 1 Processor -- start 2024-04-17T09:55:50.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.296+0000 7fe56b600640 1 -- start start 2024-04-17T09:55:50.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe56b600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe56406bf70 0x7fe56415aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:50.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe56b600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe56415c4c0 0x7fe564159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:50.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe56b600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe56415f9c0 0x7fe56414e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:50.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe56b600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe564162530 con 0x7fe56415f9c0 2024-04-17T09:55:50.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe56b600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe5640af670 con 0x7fe56406bf70 2024-04-17T09:55:50.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe56b600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe5641626b0 con 0x7fe56415c4c0 2024-04-17T09:55:50.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe568c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe56406bf70 0x7fe56415aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:50.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe569600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe56415f9c0 0x7fe56414e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:50.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe568c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe56406bf70 0x7fe56415aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:46174/0 (socket says 172.21.15.32:46174) 2024-04-17T09:55:50.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe568c00640 1 -- 172.21.15.32:0/4104193967 learned_addr learned my addr 172.21.15.32:0/4104193967 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:50.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe55be00640 1 --2- 172.21.15.32:0/4104193967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe56415c4c0 0x7fe564159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:50.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe568c00640 1 -- 172.21.15.32:0/4104193967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe56415c4c0 msgr2=0x7fe564159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:50.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe568c00640 1 --2- 172.21.15.32:0/4104193967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe56415c4c0 0x7fe564159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:50.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe568c00640 1 -- 172.21.15.32:0/4104193967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe56415f9c0 msgr2=0x7fe56414e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:50.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe568c00640 1 --2- 172.21.15.32:0/4104193967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe56415f9c0 0x7fe56414e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:50.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe568c00640 1 -- 172.21.15.32:0/4104193967 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56414edf0 con 0x7fe56406bf70 2024-04-17T09:55:50.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe569600640 1 --2- 172.21.15.32:0/4104193967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe56415f9c0 0x7fe56414e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:55:50.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe568c00640 1 --2- 172.21.15.32:0/4104193967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe56406bf70 0x7fe56415aea0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe55400d800 tx=0x7fe55400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:50.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe561600640 1 -- 172.21.15.32:0/4104193967 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554018030 con 0x7fe56406bf70 2024-04-17T09:55:50.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe561600640 1 -- 172.21.15.32:0/4104193967 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5540096a0 con 0x7fe56406bf70 2024-04-17T09:55:50.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe56b600640 1 -- 172.21.15.32:0/4104193967 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5641537a0 con 0x7fe56406bf70 2024-04-17T09:55:50.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe561600640 1 -- 172.21.15.32:0/4104193967 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554009870 con 0x7fe56406bf70 2024-04-17T09:55:50.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe56b600640 1 -- 172.21.15.32:0/4104193967 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe564153a40 con 0x7fe56406bf70 2024-04-17T09:55:50.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.300+0000 7fe56b600640 1 -- 172.21.15.32:0/4104193967 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe564068a90 con 0x7fe56406bf70 2024-04-17T09:55:50.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.304+0000 7fe561600640 1 -- 172.21.15.32:0/4104193967 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe55401b070 con 0x7fe56406bf70 2024-04-17T09:55:50.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.304+0000 7fe561600640 1 --2- 172.21.15.32:0/4104193967 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe540068380 0x7fe54006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:50.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.304+0000 7fe561600640 1 -- 172.21.15.32:0/4104193967 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5795+0+0 (secure 0 0 0) 0x7fe554086fe0 con 0x7fe56406bf70 2024-04-17T09:55:50.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.304+0000 7fe55be00640 1 --2- 172.21.15.32:0/4104193967 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe540068380 0x7fe54006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:50.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.304+0000 7fe55be00640 1 --2- 172.21.15.32:0/4104193967 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe540068380 0x7fe54006a840 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe564155200 tx=0x7fe54c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:50.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.308+0000 7fe561600640 1 -- 172.21.15.32:0/4104193967 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fe56414edf0 con 0x7fe56406bf70 2024-04-17T09:55:50.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:50.508+0000 7fe56b600640 1 -- 172.21.15.32:0/4104193967 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe5640b38a0 con 0x7fe56406bf70 2024-04-17T09:55:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.132+0000 7fe561600640 1 -- 172.21.15.32:0/4104193967 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) ==== 104+0+0 (secure 0 0 0) 0x7fe554056fd0 con 0x7fe56406bf70 2024-04-17T09:55:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.132+0000 7fe56b600640 1 -- 172.21.15.32:0/4104193967 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe540068380 msgr2=0x7fe54006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.132+0000 7fe56b600640 1 --2- 172.21.15.32:0/4104193967 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe540068380 0x7fe54006a840 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe564155200 tx=0x7fe54c008040 comp rx=0 tx=0).stop 2024-04-17T09:55:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.132+0000 7fe56b600640 1 -- 172.21.15.32:0/4104193967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe56406bf70 msgr2=0x7fe56415aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.132+0000 7fe56b600640 1 --2- 172.21.15.32:0/4104193967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe56406bf70 0x7fe56415aea0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe55400d800 tx=0x7fe55400dcc0 comp rx=0 tx=0).stop 2024-04-17T09:55:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.132+0000 7fe56b600640 1 -- 172.21.15.32:0/4104193967 shutdown_connections 2024-04-17T09:55:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.132+0000 7fe56b600640 1 --2- 172.21.15.32:0/4104193967 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe540068380 0x7fe54006a840 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.132+0000 7fe56b600640 1 --2- 172.21.15.32:0/4104193967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe56415c4c0 0x7fe564159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.132+0000 7fe56b600640 1 --2- 172.21.15.32:0/4104193967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe56415f9c0 0x7fe56414e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.132+0000 7fe56b600640 1 --2- 172.21.15.32:0/4104193967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe56406bf70 0x7fe56415aea0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.132+0000 7fe56b600640 1 -- 172.21.15.32:0/4104193967 >> 172.21.15.32:0/4104193967 conn(0x7fe5640b2120 msgr2=0x7fe5640b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.132+0000 7fe56b600640 1 -- 172.21.15.32:0/4104193967 shutdown_connections 2024-04-17T09:55:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.132+0000 7fe56b600640 1 -- 172.21.15.32:0/4104193967 wait complete. 2024-04-17T09:55:51.155 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-17T09:55:51.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.300+0000 7fa732a00640 1 Processor -- start 2024-04-17T09:55:51.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.300+0000 7fa732a00640 1 -- start start 2024-04-17T09:55:51.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.300+0000 7fa732a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa72c09ba30 0x7fa72c0b5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:51.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.300+0000 7fa732a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa72c15ff70 0x7fa72c0b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.300+0000 7fa732a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa72c09b5a0 con 0x7fa72c15fb90 2024-04-17T09:55:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.300+0000 7fa732a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa72c0b3b30 con 0x7fa72c09ba30 2024-04-17T09:55:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.300+0000 7fa732a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa72c0b3570 con 0x7fa72c15ff70 2024-04-17T09:55:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.300+0000 7fa731600640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa72c15fb90 0x7fa72c0d0960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43320/0 (socket says 172.21.15.32:43320) 2024-04-17T09:55:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.300+0000 7fa730c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa72c09ba30 0x7fa72c0b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.300+0000 7fa731600640 1 -- 172.21.15.32:0/3125543132 learned_addr learned my addr 172.21.15.32:0/3125543132 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.300+0000 7fa732000640 1 --2- 172.21.15.32:0/3125543132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa72c15ff70 0x7fa72c0b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.300+0000 7fa730c00640 1 -- 172.21.15.32:0/3125543132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa72c15ff70 msgr2=0x7fa72c0b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.300+0000 7fa730c00640 1 --2- 172.21.15.32:0/3125543132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa72c15ff70 0x7fa72c0b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.300+0000 7fa730c00640 1 -- 172.21.15.32:0/3125543132 >> v1:172.21.15.32:6789/0 conn(0x7fa72c15fb90 legacy=0x7fa72c0d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.300+0000 7fa730c00640 1 -- 172.21.15.32:0/3125543132 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72c0b3850 con 0x7fa72c09ba30 2024-04-17T09:55:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.304+0000 7fa730c00640 1 --2- 172.21.15.32:0/3125543132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa72c09ba30 0x7fa72c0b5540 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa7180097c0 tx=0x7fa71801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ef00b1e274a80d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.304+0000 7fa72be00640 1 -- 172.21.15.32:0/3125543132 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7180b1070 con 0x7fa72c09ba30 2024-04-17T09:55:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.304+0000 7fa72be00640 1 -- 172.21.15.32:0/3125543132 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7180ac410 con 0x7fa72c09ba30 2024-04-17T09:55:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.304+0000 7fa72be00640 1 -- 172.21.15.32:0/3125543132 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7180ac7e0 con 0x7fa72c09ba30 2024-04-17T09:55:51.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.304+0000 7fa732a00640 1 -- 172.21.15.32:0/3125543132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa72c09ba30 msgr2=0x7fa72c0b5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:51.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.304+0000 7fa732a00640 1 --2- 172.21.15.32:0/3125543132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa72c09ba30 0x7fa72c0b5540 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa7180097c0 tx=0x7fa71801c4a0 comp rx=0 tx=0).stop 2024-04-17T09:55:51.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.304+0000 7fa732a00640 1 -- 172.21.15.32:0/3125543132 shutdown_connections 2024-04-17T09:55:51.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.304+0000 7fa732a00640 1 --2- 172.21.15.32:0/3125543132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa72c15ff70 0x7fa72c0b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:51.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.304+0000 7fa732a00640 1 --2- 172.21.15.32:0/3125543132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa72c09ba30 0x7fa72c0b5540 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:51.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.304+0000 7fa732a00640 1 -- 172.21.15.32:0/3125543132 >> 172.21.15.32:0/3125543132 conn(0x7fa72c0b3e70 msgr2=0x7fa72c0b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:51.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.304+0000 7fa732a00640 1 -- 172.21.15.32:0/3125543132 shutdown_connections 2024-04-17T09:55:51.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.304+0000 7fa732a00640 1 -- 172.21.15.32:0/3125543132 wait complete. 2024-04-17T09:55:51.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.304+0000 7fa732a00640 1 Processor -- start 2024-04-17T09:55:51.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.304+0000 7fa732a00640 1 -- start start 2024-04-17T09:55:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa732a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa72c09ba30 0x7fa72c1416d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa732a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa72c15fb90 0x7fa72c070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa732a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa72c15ff70 0x7fa72c150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa732a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa72c169850 con 0x7fa72c15ff70 2024-04-17T09:55:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa732a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa72c1696d0 con 0x7fa72c09ba30 2024-04-17T09:55:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa732a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa72c1699d0 con 0x7fa72c15fb90 2024-04-17T09:55:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa732000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa72c15ff70 0x7fa72c150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa732000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa72c15ff70 0x7fa72c150ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37210/0 (socket says 172.21.15.32:37210) 2024-04-17T09:55:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa732000640 1 -- 172.21.15.32:0/2099540484 learned_addr learned my addr 172.21.15.32:0/2099540484 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:51.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa730c00640 1 --2- 172.21.15.32:0/2099540484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa72c15fb90 0x7fa72c070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:51.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa731600640 1 --2- 172.21.15.32:0/2099540484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa72c09ba30 0x7fa72c1416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:51.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa732000640 1 -- 172.21.15.32:0/2099540484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa72c15fb90 msgr2=0x7fa72c070550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:51.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa732000640 1 --2- 172.21.15.32:0/2099540484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa72c15fb90 0x7fa72c070550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:51.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa732000640 1 -- 172.21.15.32:0/2099540484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa72c09ba30 msgr2=0x7fa72c1416d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:51.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa732000640 1 --2- 172.21.15.32:0/2099540484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa72c09ba30 0x7fa72c1416d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:51.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa732000640 1 -- 172.21.15.32:0/2099540484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72c1508b0 con 0x7fa72c15ff70 2024-04-17T09:55:51.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa730c00640 1 --2- 172.21.15.32:0/2099540484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa72c15fb90 0x7fa72c070550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:51.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa732000640 1 --2- 172.21.15.32:0/2099540484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa72c15ff70 0x7fa72c150ac0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa724009fd0 tx=0x7fa72400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:51.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa729600640 1 -- 172.21.15.32:0/2099540484 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa724014420 con 0x7fa72c15ff70 2024-04-17T09:55:51.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa729600640 1 -- 172.21.15.32:0/2099540484 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa724019070 con 0x7fa72c15ff70 2024-04-17T09:55:51.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa729600640 1 -- 172.21.15.32:0/2099540484 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa724014800 con 0x7fa72c15ff70 2024-04-17T09:55:51.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa732a00640 1 -- 172.21.15.32:0/2099540484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa72c150010 con 0x7fa72c15ff70 2024-04-17T09:55:51.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa732a00640 1 -- 172.21.15.32:0/2099540484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa72c1505d0 con 0x7fa72c15ff70 2024-04-17T09:55:51.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.308+0000 7fa732a00640 1 -- 172.21.15.32:0/2099540484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa72c04fd00 con 0x7fa72c15ff70 2024-04-17T09:55:51.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.312+0000 7fa729600640 1 -- 172.21.15.32:0/2099540484 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa7240149a0 con 0x7fa72c15ff70 2024-04-17T09:55:51.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.312+0000 7fa729600640 1 --2- 172.21.15.32:0/2099540484 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa70c0683b0 0x7fa70c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:51.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.312+0000 7fa729600640 1 -- 172.21.15.32:0/2099540484 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5795+0+0 (secure 0 0 0) 0x7fa7240871d0 con 0x7fa72c15ff70 2024-04-17T09:55:51.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.312+0000 7fa731600640 1 --2- 172.21.15.32:0/2099540484 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa70c0683b0 0x7fa70c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:51.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.312+0000 7fa731600640 1 --2- 172.21.15.32:0/2099540484 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa70c0683b0 0x7fa70c06a870 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa72c1f9200 tx=0x7fa720009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:51.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.316+0000 7fa729600640 1 -- 172.21.15.32:0/2099540484 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa7240577c0 con 0x7fa72c15ff70 2024-04-17T09:55:51.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:51.480+0000 7fa732a00640 1 -- 172.21.15.32:0/2099540484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa72c0b57c0 con 0x7fa72c15ff70 2024-04-17T09:55:52.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.132+0000 7fa729600640 1 -- 172.21.15.32:0/2099540484 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v48) ==== 181+0+0 (secure 0 0 0) 0x7fa7240570c0 con 0x7fa72c15ff70 2024-04-17T09:55:52.135 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-04-17T09:55:52.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.136+0000 7fa732a00640 1 -- 172.21.15.32:0/2099540484 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa70c0683b0 msgr2=0x7fa70c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:52.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.136+0000 7fa732a00640 1 --2- 172.21.15.32:0/2099540484 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa70c0683b0 0x7fa70c06a870 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa72c1f9200 tx=0x7fa720009040 comp rx=0 tx=0).stop 2024-04-17T09:55:52.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.136+0000 7fa732a00640 1 -- 172.21.15.32:0/2099540484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa72c15ff70 msgr2=0x7fa72c150ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:52.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.136+0000 7fa732a00640 1 --2- 172.21.15.32:0/2099540484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa72c15ff70 0x7fa72c150ac0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa724009fd0 tx=0x7fa72400e4b0 comp rx=0 tx=0).stop 2024-04-17T09:55:52.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.136+0000 7fa732a00640 1 -- 172.21.15.32:0/2099540484 shutdown_connections 2024-04-17T09:55:52.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.136+0000 7fa732a00640 1 --2- 172.21.15.32:0/2099540484 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa70c0683b0 0x7fa70c06a870 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:52.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.136+0000 7fa732a00640 1 --2- 172.21.15.32:0/2099540484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa72c15fb90 0x7fa72c070550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:52.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.136+0000 7fa732a00640 1 --2- 172.21.15.32:0/2099540484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa72c15ff70 0x7fa72c150ac0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:52.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.136+0000 7fa732a00640 1 --2- 172.21.15.32:0/2099540484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa72c09ba30 0x7fa72c1416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:52.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.136+0000 7fa732a00640 1 -- 172.21.15.32:0/2099540484 >> 172.21.15.32:0/2099540484 conn(0x7fa72c0b3e70 msgr2=0x7fa72c15e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:52.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.136+0000 7fa732a00640 1 -- 172.21.15.32:0/2099540484 shutdown_connections 2024-04-17T09:55:52.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.136+0000 7fa732a00640 1 -- 172.21.15.32:0/2099540484 wait complete. 2024-04-17T09:55:52.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-17T09:55:52.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.312+0000 7f4399800640 1 Processor -- start 2024-04-17T09:55:52.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.312+0000 7f4399800640 1 -- start start 2024-04-17T09:55:52.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.312+0000 7f4399800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f43941460a0 0x7f43940ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:52.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.312+0000 7f4399800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f439406e6f0 0x7f4394071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:52.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.312+0000 7f4399800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f439406eeb0 con 0x7f43940722f0 2024-04-17T09:55:52.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.312+0000 7f4399800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f439415de00 con 0x7f439406e6f0 2024-04-17T09:55:52.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.312+0000 7f4399800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f439406f190 con 0x7f43941460a0 2024-04-17T09:55:52.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.312+0000 7f438be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f439406e6f0 0x7f4394071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:52.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.312+0000 7f438be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f439406e6f0 0x7f4394071db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37216/0 (socket says 172.21.15.32:37216) 2024-04-17T09:55:52.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.312+0000 7f438be00640 1 -- 172.21.15.32:0/2937647210 learned_addr learned my addr 172.21.15.32:0/2937647210 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:52.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.312+0000 7f4392a00640 1 --2- 172.21.15.32:0/2937647210 >> v2:172.21.15.144:3300/0 conn(0x7f43941460a0 0x7f43940ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:52.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.316+0000 7f438be00640 1 -- 172.21.15.32:0/2937647210 >> v2:172.21.15.144:3300/0 conn(0x7f43941460a0 msgr2=0x7f43940ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:52.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.316+0000 7f438be00640 1 --2- 172.21.15.32:0/2937647210 >> v2:172.21.15.144:3300/0 conn(0x7f43941460a0 0x7f43940ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:52.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.316+0000 7f438be00640 1 -- 172.21.15.32:0/2937647210 >> v1:172.21.15.144:6789/0 conn(0x7f43940722f0 legacy=0x7f439415cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:52.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.316+0000 7f438be00640 1 -- 172.21.15.32:0/2937647210 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439406f470 con 0x7f439406e6f0 2024-04-17T09:55:52.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.316+0000 7f438be00640 1 --2- 172.21.15.32:0/2937647210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f439406e6f0 0x7f4394071db0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f439406cab0 tx=0x7f437c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9368240ce68f7581 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:52.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.316+0000 7f4392000640 1 -- 172.21.15.32:0/2937647210 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c0b0070 con 0x7f439406e6f0 2024-04-17T09:55:52.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.316+0000 7f4392000640 1 -- 172.21.15.32:0/2937647210 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f439415db50 con 0x7f439406e6f0 2024-04-17T09:55:52.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.316+0000 7f4392000640 1 -- 172.21.15.32:0/2937647210 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c0ab820 con 0x7f439406e6f0 2024-04-17T09:55:52.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.316+0000 7f4399800640 1 -- 172.21.15.32:0/2937647210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f439406e6f0 msgr2=0x7f4394071db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:52.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.316+0000 7f4399800640 1 --2- 172.21.15.32:0/2937647210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f439406e6f0 0x7f4394071db0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f439406cab0 tx=0x7f437c01c520 comp rx=0 tx=0).stop 2024-04-17T09:55:52.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4399800640 1 -- 172.21.15.32:0/2937647210 shutdown_connections 2024-04-17T09:55:52.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4399800640 1 --2- 172.21.15.32:0/2937647210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f439406e6f0 0x7f4394071db0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:52.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4399800640 1 --2- 172.21.15.32:0/2937647210 >> v2:172.21.15.144:3300/0 conn(0x7f43941460a0 0x7f43940ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:52.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4399800640 1 -- 172.21.15.32:0/2937647210 >> 172.21.15.32:0/2937647210 conn(0x7f43940b2190 msgr2=0x7f43940b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:52.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4399800640 1 -- 172.21.15.32:0/2937647210 shutdown_connections 2024-04-17T09:55:52.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4399800640 1 -- 172.21.15.32:0/2937647210 wait complete. 2024-04-17T09:55:52.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4399800640 1 Processor -- start 2024-04-17T09:55:52.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4399800640 1 -- start start 2024-04-17T09:55:52.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4399800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f439406e6f0 0x7f43940b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:52.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4399800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43940722f0 0x7f4394145550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:52.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4399800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43941460a0 0x7f4394141e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:52.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4399800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4394169610 con 0x7f439406e6f0 2024-04-17T09:55:52.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4399800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4394169490 con 0x7f43941460a0 2024-04-17T09:55:52.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4399800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4394169790 con 0x7f43940722f0 2024-04-17T09:55:52.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4392a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f439406e6f0 0x7f43940b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:52.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4392a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f439406e6f0 0x7f43940b3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37222/0 (socket says 172.21.15.32:37222) 2024-04-17T09:55:52.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4392a00640 1 -- 172.21.15.32:0/586930057 learned_addr learned my addr 172.21.15.32:0/586930057 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:52.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f438be00640 1 --2- 172.21.15.32:0/586930057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43940722f0 0x7f4394145550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:52.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4393400640 1 --2- 172.21.15.32:0/586930057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43941460a0 0x7f4394141e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:52.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4392a00640 1 -- 172.21.15.32:0/586930057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43940722f0 msgr2=0x7f4394145550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:52.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4392a00640 1 --2- 172.21.15.32:0/586930057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43940722f0 0x7f4394145550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:52.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4392a00640 1 -- 172.21.15.32:0/586930057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43941460a0 msgr2=0x7f4394141e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:52.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4392a00640 1 --2- 172.21.15.32:0/586930057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43941460a0 0x7f4394141e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:52.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4392a00640 1 -- 172.21.15.32:0/586930057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43941423a0 con 0x7f439406e6f0 2024-04-17T09:55:52.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4392a00640 1 --2- 172.21.15.32:0/586930057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f439406e6f0 0x7f43940b3fa0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f438400e8a0 tx=0x7f438400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:52.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f438aa00640 1 -- 172.21.15.32:0/586930057 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4384017070 con 0x7f439406e6f0 2024-04-17T09:55:52.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f438aa00640 1 -- 172.21.15.32:0/586930057 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f438400c940 con 0x7f439406e6f0 2024-04-17T09:55:52.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f438aa00640 1 -- 172.21.15.32:0/586930057 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438400ccf0 con 0x7f439406e6f0 2024-04-17T09:55:52.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.320+0000 7f4399800640 1 -- 172.21.15.32:0/586930057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4394141380 con 0x7f439406e6f0 2024-04-17T09:55:52.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.324+0000 7f4399800640 1 -- 172.21.15.32:0/586930057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4394141940 con 0x7f439406e6f0 2024-04-17T09:55:52.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.324+0000 7f4399800640 1 -- 172.21.15.32:0/586930057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4394068a90 con 0x7f439406e6f0 2024-04-17T09:55:52.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.324+0000 7f438aa00640 1 -- 172.21.15.32:0/586930057 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f438401a070 con 0x7f439406e6f0 2024-04-17T09:55:52.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.324+0000 7f438aa00640 1 --2- 172.21.15.32:0/586930057 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4370068380 0x7f437006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:52.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.324+0000 7f438aa00640 1 -- 172.21.15.32:0/586930057 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5357+0+0 (secure 0 0 0) 0x7f43840867e0 con 0x7f439406e6f0 2024-04-17T09:55:52.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.324+0000 7f438be00640 1 --2- 172.21.15.32:0/586930057 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4370068380 0x7f437006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:52.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.328+0000 7f438be00640 1 --2- 172.21.15.32:0/586930057 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4370068380 0x7f437006a840 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f437c0ad000 tx=0x7f437c0a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:52.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.328+0000 7f438aa00640 1 -- 172.21.15.32:0/586930057 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f4384056e60 con 0x7f439406e6f0 2024-04-17T09:55:52.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:52.496+0000 7f4399800640 1 -- 172.21.15.32:0/586930057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f439413cd00 con 0x7f439406e6f0 2024-04-17T09:55:53.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.136+0000 7f438aa00640 1 -- 172.21.15.32:0/586930057 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v49) ==== 169+0+0 (secure 0 0 0) 0x7f4384056760 con 0x7f439406e6f0 2024-04-17T09:55:53.137 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-04-17T09:55:53.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.140+0000 7f4399800640 1 -- 172.21.15.32:0/586930057 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4370068380 msgr2=0x7f437006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:53.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.140+0000 7f4399800640 1 --2- 172.21.15.32:0/586930057 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4370068380 0x7f437006a840 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f437c0ad000 tx=0x7f437c0a6000 comp rx=0 tx=0).stop 2024-04-17T09:55:53.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.140+0000 7f4399800640 1 -- 172.21.15.32:0/586930057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f439406e6f0 msgr2=0x7f43940b3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:53.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.140+0000 7f4399800640 1 --2- 172.21.15.32:0/586930057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f439406e6f0 0x7f43940b3fa0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f438400e8a0 tx=0x7f438400eca0 comp rx=0 tx=0).stop 2024-04-17T09:55:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.140+0000 7f4399800640 1 -- 172.21.15.32:0/586930057 shutdown_connections 2024-04-17T09:55:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.140+0000 7f4399800640 1 --2- 172.21.15.32:0/586930057 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4370068380 0x7f437006a840 secure :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f437c0ad000 tx=0x7f437c0a6000 comp rx=0 tx=0).stop 2024-04-17T09:55:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.140+0000 7f4399800640 1 --2- 172.21.15.32:0/586930057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43940722f0 0x7f4394145550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.140+0000 7f4399800640 1 --2- 172.21.15.32:0/586930057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43941460a0 0x7f4394141e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.140+0000 7f4399800640 1 --2- 172.21.15.32:0/586930057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f439406e6f0 0x7f43940b3fa0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.140+0000 7f4399800640 1 -- 172.21.15.32:0/586930057 >> 172.21.15.32:0/586930057 conn(0x7f43940b2190 msgr2=0x7f439413c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.140+0000 7f4399800640 1 -- 172.21.15.32:0/586930057 shutdown_connections 2024-04-17T09:55:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.140+0000 7f4399800640 1 -- 172.21.15.32:0/586930057 wait complete. 2024-04-17T09:55:53.168 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-17T09:55:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.312+0000 7f080fc00640 1 Processor -- start 2024-04-17T09:55:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.312+0000 7f080fc00640 1 -- start start 2024-04-17T09:55:53.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.312+0000 7f080fc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f080815c890 0x7f08080b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:53.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.312+0000 7f080fc00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08080cd9c0 0x7f08080cde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:53.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.312+0000 7f080fc00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08080b1a70 con 0x7f0808160300 2024-04-17T09:55:53.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.312+0000 7f080fc00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f080815cc70 con 0x7f08080cd9c0 2024-04-17T09:55:53.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.312+0000 7f080fc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f08080cd200 con 0x7f080815c890 2024-04-17T09:55:53.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.312+0000 7f0807e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08080cd9c0 0x7f08080cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:53.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.312+0000 7f0807e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08080cd9c0 0x7f08080cde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37230/0 (socket says 172.21.15.32:37230) 2024-04-17T09:55:53.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.312+0000 7f0807e00640 1 -- 172.21.15.32:0/686031497 learned_addr learned my addr 172.21.15.32:0/686031497 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:53.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.312+0000 7f0807e00640 1 -- 172.21.15.32:0/686031497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f080815c890 msgr2=0x7f08080b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:53.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.312+0000 7f0807e00640 1 --2- 172.21.15.32:0/686031497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f080815c890 0x7f08080b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:53.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.312+0000 7f0807e00640 1 -- 172.21.15.32:0/686031497 >> v1:172.21.15.144:6789/0 conn(0x7f0808160300 legacy=0x7f0808160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:53.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.312+0000 7f0807e00640 1 -- 172.21.15.32:0/686031497 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080809b5a0 con 0x7f08080cd9c0 2024-04-17T09:55:53.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.316+0000 7f0807e00640 1 --2- 172.21.15.32:0/686031497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08080cd9c0 0x7f08080cde20 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f07f4009870 tx=0x7f07f401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4aaa3c02b4a75 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:53.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.316+0000 7f0807400640 1 -- 172.21.15.32:0/686031497 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f40aa610 con 0x7f08080cd9c0 2024-04-17T09:55:53.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.316+0000 7f0807400640 1 -- 172.21.15.32:0/686031497 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07f40af070 con 0x7f08080cd9c0 2024-04-17T09:55:53.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.316+0000 7f0807400640 1 -- 172.21.15.32:0/686031497 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f40aa9d0 con 0x7f08080cd9c0 2024-04-17T09:55:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.316+0000 7f080fc00640 1 -- 172.21.15.32:0/686031497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08080cd9c0 msgr2=0x7f08080cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.316+0000 7f080fc00640 1 --2- 172.21.15.32:0/686031497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08080cd9c0 0x7f08080cde20 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f07f4009870 tx=0x7f07f401c4f0 comp rx=0 tx=0).stop 2024-04-17T09:55:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.316+0000 7f080fc00640 1 -- 172.21.15.32:0/686031497 shutdown_connections 2024-04-17T09:55:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.316+0000 7f080fc00640 1 --2- 172.21.15.32:0/686031497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08080cd9c0 0x7f08080cde20 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.316+0000 7f080fc00640 1 --2- 172.21.15.32:0/686031497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f080815c890 0x7f08080b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.316+0000 7f080fc00640 1 -- 172.21.15.32:0/686031497 >> 172.21.15.32:0/686031497 conn(0x7f08080b4080 msgr2=0x7f08080b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.316+0000 7f080fc00640 1 -- 172.21.15.32:0/686031497 shutdown_connections 2024-04-17T09:55:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.316+0000 7f080fc00640 1 -- 172.21.15.32:0/686031497 wait complete. 2024-04-17T09:55:53.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f080fc00640 1 Processor -- start 2024-04-17T09:55:53.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f080fc00640 1 -- start start 2024-04-17T09:55:53.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f080fc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08080cd9c0 0x7f08081560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:53.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f080fc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f080815c890 0x7f08080c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:53.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f080fc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0808160300 0x7f08080c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:53.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f080fc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f08080b2270 con 0x7f0808160300 2024-04-17T09:55:53.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f080fc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f08080ce360 con 0x7f08080cd9c0 2024-04-17T09:55:53.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f080fc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f08080b23f0 con 0x7f080815c890 2024-04-17T09:55:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f080dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0808160300 0x7f08080c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f080dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0808160300 0x7f08080c81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37240/0 (socket says 172.21.15.32:37240) 2024-04-17T09:55:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f080dc00640 1 -- 172.21.15.32:0/1751091094 learned_addr learned my addr 172.21.15.32:0/1751091094 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f0807e00640 1 --2- 172.21.15.32:0/1751091094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f080815c890 0x7f08080c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f080d200640 1 --2- 172.21.15.32:0/1751091094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08080cd9c0 0x7f08081560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f080dc00640 1 -- 172.21.15.32:0/1751091094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f080815c890 msgr2=0x7f08080c7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f080dc00640 1 --2- 172.21.15.32:0/1751091094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f080815c890 0x7f08080c7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f080dc00640 1 -- 172.21.15.32:0/1751091094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08080cd9c0 msgr2=0x7f08081560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f080dc00640 1 --2- 172.21.15.32:0/1751091094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08080cd9c0 0x7f08081560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f080dc00640 1 -- 172.21.15.32:0/1751091094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08080caa80 con 0x7f0808160300 2024-04-17T09:55:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f080dc00640 1 --2- 172.21.15.32:0/1751091094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0808160300 0x7f08080c81c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f080000b9c0 tx=0x7f080000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f0804c00640 1 -- 172.21.15.32:0/1751091094 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800019070 con 0x7f0808160300 2024-04-17T09:55:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f0804c00640 1 -- 172.21.15.32:0/1751091094 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f080000cba0 con 0x7f0808160300 2024-04-17T09:55:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f0804c00640 1 -- 172.21.15.32:0/1751091094 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800014440 con 0x7f0808160300 2024-04-17T09:55:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f080fc00640 1 -- 172.21.15.32:0/1751091094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08080c8820 con 0x7f0808160300 2024-04-17T09:55:53.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.320+0000 7f080fc00640 1 -- 172.21.15.32:0/1751091094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08080c8de0 con 0x7f0808160300 2024-04-17T09:55:53.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.324+0000 7f080fc00640 1 -- 172.21.15.32:0/1751091094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0808068a90 con 0x7f0808160300 2024-04-17T09:55:53.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.324+0000 7f0804c00640 1 -- 172.21.15.32:0/1751091094 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f080001c070 con 0x7f0808160300 2024-04-17T09:55:53.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.324+0000 7f0804c00640 1 --2- 172.21.15.32:0/1751091094 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07e8068380 0x7f07e806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:53.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.324+0000 7f0804c00640 1 -- 172.21.15.32:0/1751091094 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f08000863e0 con 0x7f0808160300 2024-04-17T09:55:53.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.324+0000 7f080d200640 1 --2- 172.21.15.32:0/1751091094 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07e8068380 0x7f07e806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:53.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.324+0000 7f080d200640 1 --2- 172.21.15.32:0/1751091094 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07e8068380 0x7f07e806a840 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f08080b8440 tx=0x7f07fc006f90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:53.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.328+0000 7f0804c00640 1 -- 172.21.15.32:0/1751091094 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f08000565b0 con 0x7f0808160300 2024-04-17T09:55:53.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.508+0000 7f080fc00640 1 -- 172.21.15.32:0/1751091094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f08080b4cb0 con 0x7f0808160300 2024-04-17T09:55:53.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.512+0000 7f0804c00640 1 -- 172.21.15.32:0/1751091094 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f080005a8b0 con 0x7f0808160300 2024-04-17T09:55:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.512+0000 7f080fc00640 1 -- 172.21.15.32:0/1751091094 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07e8068380 msgr2=0x7f07e806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.512+0000 7f080fc00640 1 --2- 172.21.15.32:0/1751091094 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07e8068380 0x7f07e806a840 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f08080b8440 tx=0x7f07fc006f90 comp rx=0 tx=0).stop 2024-04-17T09:55:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.512+0000 7f080fc00640 1 -- 172.21.15.32:0/1751091094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0808160300 msgr2=0x7f08080c81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.512+0000 7f080fc00640 1 --2- 172.21.15.32:0/1751091094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0808160300 0x7f08080c81c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f080000b9c0 tx=0x7f080000be80 comp rx=0 tx=0).stop 2024-04-17T09:55:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.512+0000 7f080fc00640 1 -- 172.21.15.32:0/1751091094 shutdown_connections 2024-04-17T09:55:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.512+0000 7f080fc00640 1 --2- 172.21.15.32:0/1751091094 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07e8068380 0x7f07e806a840 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.512+0000 7f080fc00640 1 --2- 172.21.15.32:0/1751091094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f080815c890 0x7f08080c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.512+0000 7f080fc00640 1 --2- 172.21.15.32:0/1751091094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0808160300 0x7f08080c81c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.512+0000 7f080fc00640 1 --2- 172.21.15.32:0/1751091094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08080cd9c0 0x7f08081560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.512+0000 7f080fc00640 1 -- 172.21.15.32:0/1751091094 >> 172.21.15.32:0/1751091094 conn(0x7f08080b4080 msgr2=0x7f080815b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.512+0000 7f080fc00640 1 -- 172.21.15.32:0/1751091094 shutdown_connections 2024-04-17T09:55:53.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.512+0000 7f080fc00640 1 -- 172.21.15.32:0/1751091094 wait complete. 2024-04-17T09:55:53.535 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-17T09:55:53.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.672+0000 7f45e1200640 1 Processor -- start 2024-04-17T09:55:53.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45e1200640 1 -- start start 2024-04-17T09:55:53.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45e1200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f45dc15baa0 0x7f45dc15bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:53.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45e1200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45dc072bb0 con 0x7f45dc144e90 2024-04-17T09:55:53.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45e1200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45dc0b3dc0 con 0x7f45dc06ef60 2024-04-17T09:55:53.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45e1200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f45dc145270 con 0x7f45dc15baa0 2024-04-17T09:55:53.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f45dc144e90 0x7f45dc0b48a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43334/0 (socket says 172.21.15.32:43334) 2024-04-17T09:55:53.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9e00640 1 -- 172.21.15.32:0/442905435 learned_addr learned my addr 172.21.15.32:0/442905435 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:53.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d1400640 1 --2- 172.21.15.32:0/442905435 >> v2:172.21.15.144:3300/0 conn(0x7f45dc15baa0 0x7f45dc15bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:53.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9400640 1 -- 172.21.15.32:0/442905435 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1211946034 0 0) 0x7f45dc072bb0 con 0x7f45dc144e90 2024-04-17T09:55:53.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9400640 1 -- 172.21.15.32:0/442905435 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45c0005740 con 0x7f45dc144e90 2024-04-17T09:55:53.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9400640 1 -- 172.21.15.32:0/442905435 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 24443607 0 0) 0x7f45dc0b3dc0 con 0x7f45dc06ef60 2024-04-17T09:55:53.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9400640 1 -- 172.21.15.32:0/442905435 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45dc072bb0 con 0x7f45dc06ef60 2024-04-17T09:55:53.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9400640 1 -- 172.21.15.32:0/442905435 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 589758234 0 0) 0x7f45c0005740 con 0x7f45dc144e90 2024-04-17T09:55:53.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9400640 1 -- 172.21.15.32:0/442905435 >> v2:172.21.15.144:3300/0 conn(0x7f45dc15baa0 msgr2=0x7f45dc15bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:53.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9400640 1 --2- 172.21.15.32:0/442905435 >> v2:172.21.15.144:3300/0 conn(0x7f45dc15baa0 0x7f45dc15bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:53.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9400640 1 -- 172.21.15.32:0/442905435 >> v1:172.21.15.144:6789/0 conn(0x7f45dc06ef60 legacy=0x7f45dc071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:53.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9400640 1 -- 172.21.15.32:0/442905435 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45dc0b40a0 con 0x7f45dc144e90 2024-04-17T09:55:53.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9400640 1 -- 172.21.15.32:0/442905435 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f45cc001a00 con 0x7f45dc144e90 2024-04-17T09:55:53.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d1400640 1 --2- 172.21.15.32:0/442905435 >> v2:172.21.15.144:3300/0 conn(0x7f45dc15baa0 0x7f45dc15bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:55:53.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9400640 1 -- 172.21.15.32:0/442905435 >> v1:172.21.15.32:6789/0 conn(0x7f45dc144e90 legacy=0x7f45dc0b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:53.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9400640 1 --2- 172.21.15.32:0/442905435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c0005d60 0x7f45c0006160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:53.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9400640 1 --2- 172.21.15.32:0/442905435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45c0006a20 0x7f45c0006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:53.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9400640 1 --2- 172.21.15.32:0/442905435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45c00073e0 0x7f45c000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:53.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9400640 1 -- 172.21.15.32:0/442905435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc145270 con 0x7f45c0005d60 2024-04-17T09:55:53.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9400640 1 -- 172.21.15.32:0/442905435 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f45c000e260 con 0x7f45c00073e0 2024-04-17T09:55:53.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9400640 1 -- 172.21.15.32:0/442905435 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f45c000e460 con 0x7f45c0006a20 2024-04-17T09:55:53.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9e00640 1 --2- 172.21.15.32:0/442905435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c0005d60 0x7f45c0006160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:53.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d1400640 1 --2- 172.21.15.32:0/442905435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45c0006a20 0x7f45c0006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:53.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45da800640 1 --2- 172.21.15.32:0/442905435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45c00073e0 0x7f45c000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:53.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9e00640 1 -- 172.21.15.32:0/442905435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45c0006a20 msgr2=0x7f45c0006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:53.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9e00640 1 --2- 172.21.15.32:0/442905435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45c0006a20 0x7f45c0006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:53.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9e00640 1 -- 172.21.15.32:0/442905435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45c00073e0 msgr2=0x7f45c000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:53.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9e00640 1 --2- 172.21.15.32:0/442905435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45c00073e0 0x7f45c000dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:53.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d9e00640 1 -- 172.21.15.32:0/442905435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c0005740 con 0x7f45c0005d60 2024-04-17T09:55:53.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45da800640 1 -- 172.21.15.32:0/442905435 reap_dead start 2024-04-17T09:55:53.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.676+0000 7f45d1400640 1 --2- 172.21.15.32:0/442905435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45c0006a20 0x7f45c0006ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:53.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.680+0000 7f45d9e00640 1 --2- 172.21.15.32:0/442905435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c0005d60 0x7f45c0006160 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00a820 tx=0x7f45cc01d2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f353f8974e7f2e06 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:53.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.680+0000 7f45d9400640 1 -- 172.21.15.32:0/442905435 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc00a450 con 0x7f45c0005d60 2024-04-17T09:55:53.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.680+0000 7f45d9400640 1 -- 172.21.15.32:0/442905435 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45cc0072d0 con 0x7f45c0005d60 2024-04-17T09:55:53.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.680+0000 7f45d9400640 1 -- 172.21.15.32:0/442905435 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc007740 con 0x7f45c0005d60 2024-04-17T09:55:53.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.680+0000 7f45e1200640 1 -- 172.21.15.32:0/442905435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c0005d60 msgr2=0x7f45c0006160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:53.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.680+0000 7f45e1200640 1 --2- 172.21.15.32:0/442905435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c0005d60 0x7f45c0006160 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00a820 tx=0x7f45cc01d2f0 comp rx=0 tx=0).stop 2024-04-17T09:55:53.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.680+0000 7f45e1200640 1 -- 172.21.15.32:0/442905435 shutdown_connections 2024-04-17T09:55:53.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.680+0000 7f45e1200640 1 --2- 172.21.15.32:0/442905435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45c0005d60 0x7f45c0006160 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:53.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.680+0000 7f45e1200640 1 -- 172.21.15.32:0/442905435 >> 172.21.15.32:0/442905435 conn(0x7f45dc06ba30 msgr2=0x7f45dc06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:53.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.680+0000 7f45e1200640 1 -- 172.21.15.32:0/442905435 shutdown_connections 2024-04-17T09:55:53.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.680+0000 7f45e1200640 1 -- 172.21.15.32:0/442905435 wait complete. 2024-04-17T09:55:53.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45e1200640 1 Processor -- start 2024-04-17T09:55:53.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45e1200640 1 -- start start 2024-04-17T09:55:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45e1200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45c0005d60 0x7f45dc1e6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45e1200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45dc140420 0x7f45dc15f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45e1200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45dc1eb4e0 0x7f45dc1eb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45e1200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc06f4c0 con 0x7f45dc1eb4e0 2024-04-17T09:55:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45e1200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc06f340 con 0x7f45dc140420 2024-04-17T09:55:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45e1200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f45dc06f640 con 0x7f45c0005d60 2024-04-17T09:55:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45da800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45dc1eb4e0 0x7f45dc1eb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:53.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45da800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45dc1eb4e0 0x7f45dc1eb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37260/0 (socket says 172.21.15.32:37260) 2024-04-17T09:55:53.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45da800640 1 -- 172.21.15.32:0/2490009491 learned_addr learned my addr 172.21.15.32:0/2490009491 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:53.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45d9e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45c0005d60 0x7f45dc1e6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:53.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45d1400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45dc140420 0x7f45dc15f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:53.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45da800640 1 -- 172.21.15.32:0/2490009491 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45c0005d60 msgr2=0x7f45dc1e6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:53.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45da800640 1 --2- 172.21.15.32:0/2490009491 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45c0005d60 0x7f45dc1e6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:53.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45da800640 1 -- 172.21.15.32:0/2490009491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45dc140420 msgr2=0x7f45dc15f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:53.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45da800640 1 --2- 172.21.15.32:0/2490009491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45dc140420 0x7f45dc15f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:53.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45da800640 1 -- 172.21.15.32:0/2490009491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45dc1ebeb0 con 0x7f45dc1eb4e0 2024-04-17T09:55:53.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45da800640 1 --2- 172.21.15.32:0/2490009491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45dc1eb4e0 0x7f45dc1eb970 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0cceb0 tx=0x7f45d400e760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:53.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45d2a00640 1 -- 172.21.15.32:0/2490009491 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d4005030 con 0x7f45dc1eb4e0 2024-04-17T09:55:53.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45d2a00640 1 -- 172.21.15.32:0/2490009491 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45d40051d0 con 0x7f45dc1eb4e0 2024-04-17T09:55:53.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45e1200640 1 -- 172.21.15.32:0/2490009491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45dc1e73d0 con 0x7f45dc1eb4e0 2024-04-17T09:55:53.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45d2a00640 1 -- 172.21.15.32:0/2490009491 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d40069f0 con 0x7f45dc1eb4e0 2024-04-17T09:55:53.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45e1200640 1 -- 172.21.15.32:0/2490009491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45dc1e7990 con 0x7f45dc1eb4e0 2024-04-17T09:55:53.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.684+0000 7f45e1200640 1 -- 172.21.15.32:0/2490009491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45dc068a90 con 0x7f45dc1eb4e0 2024-04-17T09:55:53.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.688+0000 7f45d2a00640 1 -- 172.21.15.32:0/2490009491 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f45d4006b90 con 0x7f45dc1eb4e0 2024-04-17T09:55:53.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.688+0000 7f45d2a00640 1 --2- 172.21.15.32:0/2490009491 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f45b80683b0 0x7f45b806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:53.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.688+0000 7f45d2a00640 1 -- 172.21.15.32:0/2490009491 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f45d4085ca0 con 0x7f45dc1eb4e0 2024-04-17T09:55:53.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.688+0000 7f45d9e00640 1 --2- 172.21.15.32:0/2490009491 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f45b80683b0 0x7f45b806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:53.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.688+0000 7f45d9e00640 1 --2- 172.21.15.32:0/2490009491 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f45b80683b0 0x7f45b806a870 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0ad000 tx=0x7f45cc001dc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:53.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.692+0000 7f45d2a00640 1 -- 172.21.15.32:0/2490009491 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f45d4056560 con 0x7f45dc1eb4e0 2024-04-17T09:55:53.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:53.928+0000 7f45e1200640 1 -- 172.21.15.32:0/2490009491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7f45dc06cdd0 con 0x7f45dc1eb4e0 2024-04-17T09:55:54.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.148+0000 7f45d2a00640 1 -- 172.21.15.32:0/2490009491 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v156) ==== 149+0+0 (secure 0 0 0) 0x7f45d4055e60 con 0x7f45dc1eb4e0 2024-04-17T09:55:54.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.152+0000 7f45e1200640 1 -- 172.21.15.32:0/2490009491 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f45b80683b0 msgr2=0x7f45b806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:54.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.152+0000 7f45e1200640 1 --2- 172.21.15.32:0/2490009491 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f45b80683b0 0x7f45b806a870 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0ad000 tx=0x7f45cc001dc0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.152+0000 7f45e1200640 1 -- 172.21.15.32:0/2490009491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45dc1eb4e0 msgr2=0x7f45dc1eb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:54.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.152+0000 7f45e1200640 1 --2- 172.21.15.32:0/2490009491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45dc1eb4e0 0x7f45dc1eb970 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0cceb0 tx=0x7f45d400e760 comp rx=0 tx=0).stop 2024-04-17T09:55:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.152+0000 7f45e1200640 1 -- 172.21.15.32:0/2490009491 shutdown_connections 2024-04-17T09:55:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.152+0000 7f45e1200640 1 --2- 172.21.15.32:0/2490009491 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f45b80683b0 0x7f45b806a870 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.152+0000 7f45e1200640 1 --2- 172.21.15.32:0/2490009491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45dc1eb4e0 0x7f45dc1eb970 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.152+0000 7f45e1200640 1 --2- 172.21.15.32:0/2490009491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45dc140420 0x7f45dc15f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.152+0000 7f45e1200640 1 --2- 172.21.15.32:0/2490009491 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45c0005d60 0x7f45dc1e6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.152+0000 7f45e1200640 1 -- 172.21.15.32:0/2490009491 >> 172.21.15.32:0/2490009491 conn(0x7f45dc06ba30 msgr2=0x7f45dc06e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.152+0000 7f45e1200640 1 -- 172.21.15.32:0/2490009491 shutdown_connections 2024-04-17T09:55:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.152+0000 7f45e1200640 1 -- 172.21.15.32:0/2490009491 wait complete. 2024-04-17T09:55:54.172 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-04-17T09:55:54.173 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-17T09:55:54.173 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-17T09:55:54.173 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:55:54.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.316+0000 7f042be00640 1 Processor -- start 2024-04-17T09:55:54.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.316+0000 7f042be00640 1 -- start start 2024-04-17T09:55:54.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.316+0000 7f042be00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f042c06f250 0x7f042c06dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:54.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.316+0000 7f042be00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f042c06e1b0 con 0x7f042c0cf5b0 2024-04-17T09:55:54.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.316+0000 7f042be00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f042c06e490 con 0x7f042c0cf990 2024-04-17T09:55:54.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.316+0000 7f042be00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f042c06edc0 con 0x7f042c06f250 2024-04-17T09:55:54.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.316+0000 7f042aa00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f042c0cf5b0 0x7f042c15d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43344/0 (socket says 172.21.15.32:43344) 2024-04-17T09:55:54.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.316+0000 7f042aa00640 1 -- 172.21.15.32:0/1750159454 learned_addr learned my addr 172.21.15.32:0/1750159454 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:54.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.316+0000 7f042b400640 1 --2- 172.21.15.32:0/1750159454 >> v2:172.21.15.144:3300/0 conn(0x7f042c06f250 0x7f042c06dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:54.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f0429600640 1 -- 172.21.15.32:0/1750159454 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4053525683 0 0) 0x7f042c06e1b0 con 0x7f042c0cf5b0 2024-04-17T09:55:54.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f0429600640 1 -- 172.21.15.32:0/1750159454 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0410003530 con 0x7f042c0cf5b0 2024-04-17T09:55:54.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f0429600640 1 -- 172.21.15.32:0/1750159454 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 8730449 0 0) 0x7f0410003530 con 0x7f042c0cf5b0 2024-04-17T09:55:54.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f0429600640 1 -- 172.21.15.32:0/1750159454 >> v2:172.21.15.144:3300/0 conn(0x7f042c06f250 msgr2=0x7f042c06dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:54.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f0429600640 1 --2- 172.21.15.32:0/1750159454 >> v2:172.21.15.144:3300/0 conn(0x7f042c06f250 0x7f042c06dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f0429600640 1 -- 172.21.15.32:0/1750159454 >> v1:172.21.15.144:6789/0 conn(0x7f042c0cf990 legacy=0x7f042c0aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:54.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f0429600640 1 -- 172.21.15.32:0/1750159454 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042c06e770 con 0x7f042c0cf5b0 2024-04-17T09:55:54.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f0429600640 1 -- 172.21.15.32:0/1750159454 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f041c001680 con 0x7f042c0cf5b0 2024-04-17T09:55:54.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f0429600640 1 -- 172.21.15.32:0/1750159454 >> v1:172.21.15.32:6789/0 conn(0x7f042c0cf5b0 legacy=0x7f042c15d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:54.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f0429600640 1 --2- 172.21.15.32:0/1750159454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04100039c0 0x7f0410003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:54.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f0429600640 1 --2- 172.21.15.32:0/1750159454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0410004810 0x7f0410004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:54.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f0429600640 1 --2- 172.21.15.32:0/1750159454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0410005220 0x7f041000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:54.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f0429600640 1 -- 172.21.15.32:0/1750159454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f042c06edc0 con 0x7f0410005220 2024-04-17T09:55:54.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f0429600640 1 -- 172.21.15.32:0/1750159454 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f041000c150 con 0x7f0410004810 2024-04-17T09:55:54.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f0429600640 1 -- 172.21.15.32:0/1750159454 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f041000c350 con 0x7f04100039c0 2024-04-17T09:55:54.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f042b400640 1 --2- 172.21.15.32:0/1750159454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0410005220 0x7f041000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:54.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f042a000640 1 --2- 172.21.15.32:0/1750159454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0410004810 0x7f0410004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:54.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f042aa00640 1 --2- 172.21.15.32:0/1750159454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04100039c0 0x7f0410003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:54.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f042b400640 1 -- 172.21.15.32:0/1750159454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04100039c0 msgr2=0x7f0410003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:54.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f042b400640 1 --2- 172.21.15.32:0/1750159454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04100039c0 0x7f0410003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f042b400640 1 -- 172.21.15.32:0/1750159454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0410004810 msgr2=0x7f0410004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:54.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f042b400640 1 --2- 172.21.15.32:0/1750159454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0410004810 0x7f0410004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f042b400640 1 -- 172.21.15.32:0/1750159454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0410003530 con 0x7f0410005220 2024-04-17T09:55:54.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.320+0000 7f042b400640 1 -- 172.21.15.32:0/1750159454 reap_dead start 2024-04-17T09:55:54.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.324+0000 7f042b400640 1 --2- 172.21.15.32:0/1750159454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0410005220 0x7f041000b9d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0424009fc0 tx=0x7f042401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3090d78e2869b68 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:54.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.324+0000 7f0429600640 1 -- 172.21.15.32:0/1750159454 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424009c10 con 0x7f0410005220 2024-04-17T09:55:54.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.324+0000 7f0429600640 1 -- 172.21.15.32:0/1750159454 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04240a5c30 con 0x7f0410005220 2024-04-17T09:55:54.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.324+0000 7f0429600640 1 -- 172.21.15.32:0/1750159454 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04240039a0 con 0x7f0410005220 2024-04-17T09:55:54.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.324+0000 7f042be00640 1 -- 172.21.15.32:0/1750159454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0410005220 msgr2=0x7f041000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:54.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.324+0000 7f042be00640 1 --2- 172.21.15.32:0/1750159454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0410005220 0x7f041000b9d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0424009fc0 tx=0x7f042401ccc0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.324+0000 7f042be00640 1 -- 172.21.15.32:0/1750159454 shutdown_connections 2024-04-17T09:55:54.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.324+0000 7f042be00640 1 --2- 172.21.15.32:0/1750159454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0410005220 0x7f041000b9d0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.324+0000 7f042be00640 1 -- 172.21.15.32:0/1750159454 >> 172.21.15.32:0/1750159454 conn(0x7f042c0d0180 msgr2=0x7f042c0d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:54.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.324+0000 7f042be00640 1 -- 172.21.15.32:0/1750159454 shutdown_connections 2024-04-17T09:55:54.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.324+0000 7f042be00640 1 -- 172.21.15.32:0/1750159454 wait complete. 2024-04-17T09:55:54.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.324+0000 7f042be00640 1 Processor -- start 2024-04-17T09:55:54.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.324+0000 7f042be00640 1 -- start start 2024-04-17T09:55:54.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.324+0000 7f042be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0410005220 0x7f042c143860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:54.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.324+0000 7f042be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f042c1443c0 0x7f042c13d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:54.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.324+0000 7f042be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f042c1f58e0 0x7f042c1f5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:54.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.324+0000 7f042be00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f042c06dd50 con 0x7f042c1f58e0 2024-04-17T09:55:54.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.324+0000 7f042be00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f042c06dbd0 con 0x7f0410005220 2024-04-17T09:55:54.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.324+0000 7f042be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f042c06ded0 con 0x7f042c1443c0 2024-04-17T09:55:54.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.324+0000 7f042b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f042c1f58e0 0x7f042c1f5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:54.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.324+0000 7f042b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f042c1f58e0 0x7f042c1f5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37284/0 (socket says 172.21.15.32:37284) 2024-04-17T09:55:54.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.324+0000 7f042b400640 1 -- 172.21.15.32:0/760911065 learned_addr learned my addr 172.21.15.32:0/760911065 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:54.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.328+0000 7f042aa00640 1 --2- 172.21.15.32:0/760911065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0410005220 0x7f042c143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.328+0000 7f042a000640 1 --2- 172.21.15.32:0/760911065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f042c1443c0 0x7f042c13d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.328+0000 7f042b400640 1 -- 172.21.15.32:0/760911065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f042c1443c0 msgr2=0x7f042c13d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.328+0000 7f042b400640 1 --2- 172.21.15.32:0/760911065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f042c1443c0 0x7f042c13d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.328+0000 7f042b400640 1 -- 172.21.15.32:0/760911065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0410005220 msgr2=0x7f042c143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.328+0000 7f042b400640 1 --2- 172.21.15.32:0/760911065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0410005220 0x7f042c143860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.328+0000 7f042b400640 1 -- 172.21.15.32:0/760911065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042c13b1c0 con 0x7f042c1f58e0 2024-04-17T09:55:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.328+0000 7f042aa00640 1 --2- 172.21.15.32:0/760911065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0410005220 0x7f042c143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:55:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.328+0000 7f042a000640 1 --2- 172.21.15.32:0/760911065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f042c1443c0 0x7f042c13d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:55:54.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.328+0000 7f042b400640 1 --2- 172.21.15.32:0/760911065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f042c1f58e0 0x7f042c1f5d60 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f042401d1a0 tx=0x7f0424009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:54.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.328+0000 7f0422a00640 1 -- 172.21.15.32:0/760911065 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424009c10 con 0x7f042c1f58e0 2024-04-17T09:55:54.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.328+0000 7f0422a00640 1 -- 172.21.15.32:0/760911065 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04240bd070 con 0x7f042c1f58e0 2024-04-17T09:55:54.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.328+0000 7f0422a00640 1 -- 172.21.15.32:0/760911065 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424003ba0 con 0x7f042c1f58e0 2024-04-17T09:55:54.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.328+0000 7f042be00640 1 -- 172.21.15.32:0/760911065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042c13b4a0 con 0x7f042c1f58e0 2024-04-17T09:55:54.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.328+0000 7f042be00640 1 -- 172.21.15.32:0/760911065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f042c13d720 con 0x7f042c1f58e0 2024-04-17T09:55:54.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.328+0000 7f042be00640 1 -- 172.21.15.32:0/760911065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f042c04fd00 con 0x7f042c1f58e0 2024-04-17T09:55:54.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.328+0000 7f0422a00640 1 -- 172.21.15.32:0/760911065 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f04240ae950 con 0x7f042c1f58e0 2024-04-17T09:55:54.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.328+0000 7f0422a00640 1 --2- 172.21.15.32:0/760911065 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f040c068380 0x7f040c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:54.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.328+0000 7f0422a00640 1 -- 172.21.15.32:0/760911065 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f0424121e10 con 0x7f042c1f58e0 2024-04-17T09:55:54.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.332+0000 7f042aa00640 1 --2- 172.21.15.32:0/760911065 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f040c068380 0x7f040c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:54.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.332+0000 7f042aa00640 1 --2- 172.21.15.32:0/760911065 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f040c068380 0x7f040c06a840 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f041c000c10 tx=0x7f041c007900 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:54.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.336+0000 7f0422a00640 1 -- 172.21.15.32:0/760911065 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0424125030 con 0x7f042c1f58e0 2024-04-17T09:55:54.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.500+0000 7f042be00640 1 -- 172.21.15.32:0/760911065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f042c06fc10 con 0x7f042c1f58e0 2024-04-17T09:55:54.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.500+0000 7f0422a00640 1 -- 172.21.15.32:0/760911065 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+13034 (secure 0 0 0) 0x7f04240169d0 con 0x7f042c1f58e0 2024-04-17T09:55:54.501 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:54.501 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":49,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:55:53.137856+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.144:6835/2177384675":"2024-04-18T09:55:50.127928+0000","172.21.15.144:6833/2177384675":"2024-04-18T09:55:50.127928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:55:54.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.500+0000 7f042be00640 1 -- 172.21.15.32:0/760911065 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f040c068380 msgr2=0x7f040c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:54.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.500+0000 7f042be00640 1 --2- 172.21.15.32:0/760911065 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f040c068380 0x7f040c06a840 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f041c000c10 tx=0x7f041c007900 comp rx=0 tx=0).stop 2024-04-17T09:55:54.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.500+0000 7f042be00640 1 -- 172.21.15.32:0/760911065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f042c1f58e0 msgr2=0x7f042c1f5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:54.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.500+0000 7f042be00640 1 --2- 172.21.15.32:0/760911065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f042c1f58e0 0x7f042c1f5d60 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f042401d1a0 tx=0x7f0424009750 comp rx=0 tx=0).stop 2024-04-17T09:55:54.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.500+0000 7f042be00640 1 -- 172.21.15.32:0/760911065 shutdown_connections 2024-04-17T09:55:54.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.500+0000 7f042be00640 1 --2- 172.21.15.32:0/760911065 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f040c068380 0x7f040c06a840 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.500+0000 7f042be00640 1 --2- 172.21.15.32:0/760911065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f042c1443c0 0x7f042c13d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.500+0000 7f042be00640 1 --2- 172.21.15.32:0/760911065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f042c1f58e0 0x7f042c1f5d60 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.500+0000 7f042be00640 1 --2- 172.21.15.32:0/760911065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0410005220 0x7f042c143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.500+0000 7f042be00640 1 -- 172.21.15.32:0/760911065 >> 172.21.15.32:0/760911065 conn(0x7f042c0d0180 msgr2=0x7f042c15dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:54.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.500+0000 7f042be00640 1 -- 172.21.15.32:0/760911065 shutdown_connections 2024-04-17T09:55:54.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.500+0000 7f042be00640 1 -- 172.21.15.32:0/760911065 wait complete. 2024-04-17T09:55:54.522 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:55:54.523 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:55:54.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.664+0000 7f22a8200640 1 Processor -- start 2024-04-17T09:55:54.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.664+0000 7f22a8200640 1 -- start start 2024-04-17T09:55:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.664+0000 7f22a8200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f22a006ba30 0x7f22a006be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.664+0000 7f22a8200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22a00cf370 0x7f22a00cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.664+0000 7f22a8200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f22a0160350 0x7f22a01607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.664+0000 7f22a8200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f22a006c350 con 0x7f22a006ba30 2024-04-17T09:55:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.664+0000 7f22a8200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f22a00d0130 con 0x7f22a0160350 2024-04-17T09:55:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.664+0000 7f22a8200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f22a015d090 con 0x7f22a00cf370 2024-04-17T09:55:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.664+0000 7f22a5800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f22a006ba30 0x7f22a006be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.664+0000 7f22a5800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f22a006ba30 0x7f22a006be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37296/0 (socket says 172.21.15.32:37296) 2024-04-17T09:55:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.664+0000 7f22a5800640 1 -- 172.21.15.32:0/3762437051 learned_addr learned my addr 172.21.15.32:0/3762437051 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.664+0000 7f22a4e00640 1 --2- 172.21.15.32:0/3762437051 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22a00cf370 0x7f22a00cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.664+0000 7f22a6200640 1 --2- 172.21.15.32:0/3762437051 >> v2:172.21.15.144:3300/0 conn(0x7f22a0160350 0x7f22a01607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.668+0000 7f22a5800640 1 -- 172.21.15.32:0/3762437051 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22a00cf370 msgr2=0x7f22a00cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.668+0000 7f22a5800640 1 --2- 172.21.15.32:0/3762437051 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22a00cf370 0x7f22a00cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.668+0000 7f22a5800640 1 -- 172.21.15.32:0/3762437051 >> v2:172.21.15.144:3300/0 conn(0x7f22a0160350 msgr2=0x7f22a01607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.668+0000 7f22a5800640 1 --2- 172.21.15.32:0/3762437051 >> v2:172.21.15.144:3300/0 conn(0x7f22a0160350 0x7f22a01607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.668+0000 7f22a5800640 1 -- 172.21.15.32:0/3762437051 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a015d370 con 0x7f22a006ba30 2024-04-17T09:55:54.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.668+0000 7f22a5800640 1 --2- 172.21.15.32:0/3762437051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a006ba30 0x7f22a006be10 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f22a00b58d0 tx=0x7f229401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525badca3870371d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:54.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.668+0000 7f229fe00640 1 -- 172.21.15.32:0/3762437051 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22940a7070 con 0x7f22a006ba30 2024-04-17T09:55:54.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.668+0000 7f229fe00640 1 -- 172.21.15.32:0/3762437051 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22940b0070 con 0x7f22a006ba30 2024-04-17T09:55:54.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.668+0000 7f229fe00640 1 -- 172.21.15.32:0/3762437051 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22940ab890 con 0x7f22a006ba30 2024-04-17T09:55:54.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a8200640 1 -- 172.21.15.32:0/3762437051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a006ba30 msgr2=0x7f22a006be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:54.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a8200640 1 --2- 172.21.15.32:0/3762437051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a006ba30 0x7f22a006be10 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f22a00b58d0 tx=0x7f229401c4f0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a8200640 1 -- 172.21.15.32:0/3762437051 shutdown_connections 2024-04-17T09:55:54.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a8200640 1 --2- 172.21.15.32:0/3762437051 >> v2:172.21.15.144:3300/0 conn(0x7f22a0160350 0x7f22a01607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a8200640 1 --2- 172.21.15.32:0/3762437051 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22a00cf370 0x7f22a00cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a8200640 1 --2- 172.21.15.32:0/3762437051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a006ba30 0x7f22a006be10 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a8200640 1 -- 172.21.15.32:0/3762437051 >> 172.21.15.32:0/3762437051 conn(0x7f22a00b2120 msgr2=0x7f22a00b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:54.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a8200640 1 -- 172.21.15.32:0/3762437051 shutdown_connections 2024-04-17T09:55:54.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a8200640 1 -- 172.21.15.32:0/3762437051 wait complete. 2024-04-17T09:55:54.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a8200640 1 Processor -- start 2024-04-17T09:55:54.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a8200640 1 -- start start 2024-04-17T09:55:54.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a8200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22a006ba30 0x7f22a015f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:54.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a8200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a00cf370 0x7f22a01fb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:54.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a8200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22a0160350 0x7f22a01fbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:54.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a8200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f22a00cfea0 con 0x7f22a00cf370 2024-04-17T09:55:54.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a8200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f22a00cfd20 con 0x7f22a006ba30 2024-04-17T09:55:54.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a8200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f22a015db70 con 0x7f22a0160350 2024-04-17T09:55:54.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a4e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a00cf370 0x7f22a01fb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:54.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a4e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a00cf370 0x7f22a01fb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37306/0 (socket says 172.21.15.32:37306) 2024-04-17T09:55:54.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a4e00640 1 -- 172.21.15.32:0/1746312800 learned_addr learned my addr 172.21.15.32:0/1746312800 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:54.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a5800640 1 --2- 172.21.15.32:0/1746312800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22a006ba30 0x7f22a015f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:54.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a6200640 1 --2- 172.21.15.32:0/1746312800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22a0160350 0x7f22a01fbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:54.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a4e00640 1 -- 172.21.15.32:0/1746312800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22a0160350 msgr2=0x7f22a01fbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:54.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a4e00640 1 --2- 172.21.15.32:0/1746312800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22a0160350 0x7f22a01fbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a4e00640 1 -- 172.21.15.32:0/1746312800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22a006ba30 msgr2=0x7f22a015f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:54.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a4e00640 1 --2- 172.21.15.32:0/1746312800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22a006ba30 0x7f22a015f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a4e00640 1 -- 172.21.15.32:0/1746312800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a006c0a0 con 0x7f22a00cf370 2024-04-17T09:55:54.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.672+0000 7f22a4e00640 1 --2- 172.21.15.32:0/1746312800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a00cf370 0x7f22a01fb640 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f228c00e8b0 tx=0x7f228c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:54.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.676+0000 7f229d600640 1 -- 172.21.15.32:0/1746312800 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c017070 con 0x7f22a00cf370 2024-04-17T09:55:54.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.676+0000 7f229d600640 1 -- 172.21.15.32:0/1746312800 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f228c00c940 con 0x7f22a00cf370 2024-04-17T09:55:54.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.676+0000 7f22a8200640 1 -- 172.21.15.32:0/1746312800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a01430f0 con 0x7f22a00cf370 2024-04-17T09:55:54.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.676+0000 7f229d600640 1 -- 172.21.15.32:0/1746312800 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c00cd10 con 0x7f22a00cf370 2024-04-17T09:55:54.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.676+0000 7f22a8200640 1 -- 172.21.15.32:0/1746312800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a01436b0 con 0x7f22a00cf370 2024-04-17T09:55:54.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.676+0000 7f22a8200640 1 -- 172.21.15.32:0/1746312800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a0068a90 con 0x7f22a00cf370 2024-04-17T09:55:54.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.676+0000 7f229d600640 1 -- 172.21.15.32:0/1746312800 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f228c01a070 con 0x7f22a00cf370 2024-04-17T09:55:54.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.676+0000 7f229d600640 1 --2- 172.21.15.32:0/1746312800 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2280068380 0x7f228006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:54.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.676+0000 7f229d600640 1 -- 172.21.15.32:0/1746312800 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f228c0869e0 con 0x7f22a00cf370 2024-04-17T09:55:54.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.676+0000 7f22a5800640 1 --2- 172.21.15.32:0/1746312800 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2280068380 0x7f228006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:54.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.676+0000 7f22a5800640 1 --2- 172.21.15.32:0/1746312800 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2280068380 0x7f228006a840 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f22940af000 tx=0x7f22940a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:54.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.680+0000 7f229d600640 1 -- 172.21.15.32:0/1746312800 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f228c056bd0 con 0x7f22a00cf370 2024-04-17T09:55:54.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.844+0000 7f22a8200640 1 -- 172.21.15.32:0/1746312800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f22a01459f0 con 0x7f22a00cf370 2024-04-17T09:55:54.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.844+0000 7f229d600640 1 -- 172.21.15.32:0/1746312800 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) ==== 93+0+10 (secure 0 0 0) 0x7f228c05aed0 con 0x7f22a00cf370 2024-04-17T09:55:54.847 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:55:54.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.844+0000 7f22a8200640 1 -- 172.21.15.32:0/1746312800 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2280068380 msgr2=0x7f228006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:54.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.844+0000 7f22a8200640 1 --2- 172.21.15.32:0/1746312800 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2280068380 0x7f228006a840 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f22940af000 tx=0x7f22940a7000 comp rx=0 tx=0).stop 2024-04-17T09:55:54.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.848+0000 7f22a8200640 1 -- 172.21.15.32:0/1746312800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a00cf370 msgr2=0x7f22a01fb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:54.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.848+0000 7f22a8200640 1 --2- 172.21.15.32:0/1746312800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a00cf370 0x7f22a01fb640 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f228c00e8b0 tx=0x7f228c00ed70 comp rx=0 tx=0).stop 2024-04-17T09:55:54.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.848+0000 7f22a8200640 1 -- 172.21.15.32:0/1746312800 shutdown_connections 2024-04-17T09:55:54.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.848+0000 7f22a8200640 1 --2- 172.21.15.32:0/1746312800 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2280068380 0x7f228006a840 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.848+0000 7f22a8200640 1 --2- 172.21.15.32:0/1746312800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22a0160350 0x7f22a01fbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.848+0000 7f22a8200640 1 --2- 172.21.15.32:0/1746312800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a00cf370 0x7f22a01fb640 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.848+0000 7f22a8200640 1 --2- 172.21.15.32:0/1746312800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22a006ba30 0x7f22a015f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:54.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.848+0000 7f22a8200640 1 -- 172.21.15.32:0/1746312800 >> 172.21.15.32:0/1746312800 conn(0x7f22a00b2120 msgr2=0x7f22a00ccfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:54.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.848+0000 7f22a8200640 1 -- 172.21.15.32:0/1746312800 shutdown_connections 2024-04-17T09:55:54.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:54.848+0000 7f22a8200640 1 -- 172.21.15.32:0/1746312800 wait complete. 2024-04-17T09:55:54.868 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-17T09:55:55.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.284+0000 7f779e800640 1 Processor -- start 2024-04-17T09:55:55.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.284+0000 7f779e800640 1 -- start start 2024-04-17T09:55:55.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.284+0000 7f779e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7798145f30 0x7f7798071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:55.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.288+0000 7f779e800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f77980cd580 0x7f7798072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:55.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.288+0000 7f779e800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77980cdac0 con 0x7f779815d8d0 2024-04-17T09:55:55.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.288+0000 7f779e800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f779806f400 con 0x7f77980cd580 2024-04-17T09:55:55.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.288+0000 7f779e800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7798146310 con 0x7f7798145f30 2024-04-17T09:55:55.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.288+0000 7f778ea00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f779815d8d0 0x7f77980cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43348/0 (socket says 172.21.15.32:43348) 2024-04-17T09:55:55.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.288+0000 7f778ea00640 1 -- 172.21.15.32:0/4175748605 learned_addr learned my addr 172.21.15.32:0/4175748605 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:55.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.288+0000 7f7797e00640 1 --2- 172.21.15.32:0/4175748605 >> v2:172.21.15.32:3300/0 conn(0x7f77980cd580 0x7f7798072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:55.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.288+0000 7f7797400640 1 --2- 172.21.15.32:0/4175748605 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7798145f30 0x7f7798071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:55.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.288+0000 7f7797e00640 1 -- 172.21.15.32:0/4175748605 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7798145f30 msgr2=0x7f7798071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:55.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.288+0000 7f7797e00640 1 --2- 172.21.15.32:0/4175748605 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7798145f30 0x7f7798071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:55.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.288+0000 7f7797e00640 1 -- 172.21.15.32:0/4175748605 >> v1:172.21.15.32:6789/0 conn(0x7f779815d8d0 legacy=0x7f77980cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:55.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.288+0000 7f7797e00640 1 -- 172.21.15.32:0/4175748605 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f779815cdf0 con 0x7f77980cd580 2024-04-17T09:55:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.288+0000 7f7797e00640 1 --2- 172.21.15.32:0/4175748605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77980cd580 0x7f7798072330 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7790009d90 tx=0x7f779001f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0e1e2bf38600b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.288+0000 7f7796a00640 1 -- 172.21.15.32:0/4175748605 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77900b4070 con 0x7f77980cd580 2024-04-17T09:55:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.288+0000 7f7796a00640 1 -- 172.21.15.32:0/4175748605 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77900af600 con 0x7f77980cd580 2024-04-17T09:55:55.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.288+0000 7f7796a00640 1 -- 172.21.15.32:0/4175748605 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77900af9d0 con 0x7f77980cd580 2024-04-17T09:55:55.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f779e800640 1 -- 172.21.15.32:0/4175748605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77980cd580 msgr2=0x7f7798072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:55.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f779e800640 1 --2- 172.21.15.32:0/4175748605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77980cd580 0x7f7798072330 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7790009d90 tx=0x7f779001f150 comp rx=0 tx=0).stop 2024-04-17T09:55:55.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f779e800640 1 -- 172.21.15.32:0/4175748605 shutdown_connections 2024-04-17T09:55:55.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f779e800640 1 --2- 172.21.15.32:0/4175748605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77980cd580 0x7f7798072330 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:55.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f779e800640 1 --2- 172.21.15.32:0/4175748605 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7798145f30 0x7f7798071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:55.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f779e800640 1 -- 172.21.15.32:0/4175748605 >> 172.21.15.32:0/4175748605 conn(0x7f779806ba30 msgr2=0x7f779806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:55.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f779e800640 1 -- 172.21.15.32:0/4175748605 shutdown_connections 2024-04-17T09:55:55.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f779e800640 1 -- 172.21.15.32:0/4175748605 wait complete. 2024-04-17T09:55:55.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f779e800640 1 Processor -- start 2024-04-17T09:55:55.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f779e800640 1 -- start start 2024-04-17T09:55:55.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f779e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77980cd580 0x7f7798140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f779e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7798145f30 0x7f77981e8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f779e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f779815d8d0 0x7f77981e8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f779e800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f779815cb90 con 0x7f7798145f30 2024-04-17T09:55:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f779e800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f779815d440 con 0x7f77980cd580 2024-04-17T09:55:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f779e800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f77981601d0 con 0x7f779815d8d0 2024-04-17T09:55:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f778ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7798145f30 0x7f77981e8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f778ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7798145f30 0x7f77981e8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37314/0 (socket says 172.21.15.32:37314) 2024-04-17T09:55:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f778ea00640 1 -- 172.21.15.32:0/330806752 learned_addr learned my addr 172.21.15.32:0/330806752 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f7797400640 1 --2- 172.21.15.32:0/330806752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77980cd580 0x7f7798140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f7797e00640 1 --2- 172.21.15.32:0/330806752 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f779815d8d0 0x7f77981e8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f778ea00640 1 -- 172.21.15.32:0/330806752 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f779815d8d0 msgr2=0x7f77981e8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f778ea00640 1 --2- 172.21.15.32:0/330806752 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f779815d8d0 0x7f77981e8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f778ea00640 1 -- 172.21.15.32:0/330806752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77980cd580 msgr2=0x7f7798140dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f778ea00640 1 --2- 172.21.15.32:0/330806752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77980cd580 0x7f7798140dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.292+0000 7f778ea00640 1 -- 172.21.15.32:0/330806752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7798142a20 con 0x7f7798145f30 2024-04-17T09:55:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.296+0000 7f778ea00640 1 --2- 172.21.15.32:0/330806752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7798145f30 0x7f77981e8710 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f778000c370 tx=0x7f778000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.296+0000 7f778fe00640 1 -- 172.21.15.32:0/330806752 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780017070 con 0x7f7798145f30 2024-04-17T09:55:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.296+0000 7f778fe00640 1 -- 172.21.15.32:0/330806752 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7780005cf0 con 0x7f7798145f30 2024-04-17T09:55:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.296+0000 7f778fe00640 1 -- 172.21.15.32:0/330806752 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77800072f0 con 0x7f7798145f30 2024-04-17T09:55:55.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.296+0000 7f779e800640 1 -- 172.21.15.32:0/330806752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7798142d00 con 0x7f7798145f30 2024-04-17T09:55:55.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.296+0000 7f779e800640 1 -- 172.21.15.32:0/330806752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77981432c0 con 0x7f7798145f30 2024-04-17T09:55:55.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.296+0000 7f779e800640 1 -- 172.21.15.32:0/330806752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7798068a90 con 0x7f7798145f30 2024-04-17T09:55:55.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.296+0000 7f778fe00640 1 -- 172.21.15.32:0/330806752 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f778001a070 con 0x7f7798145f30 2024-04-17T09:55:55.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.296+0000 7f778fe00640 1 --2- 172.21.15.32:0/330806752 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7774068380 0x7f777406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:55.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.296+0000 7f778fe00640 1 -- 172.21.15.32:0/330806752 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f77800859b0 con 0x7f7798145f30 2024-04-17T09:55:55.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.296+0000 7f7797400640 1 --2- 172.21.15.32:0/330806752 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7774068380 0x7f777406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:55.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.300+0000 7f7797400640 1 --2- 172.21.15.32:0/330806752 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7774068380 0x7f777406a840 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7788007a60 tx=0x7f7788008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:55.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.300+0000 7f778fe00640 1 -- 172.21.15.32:0/330806752 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f7780055c10 con 0x7f7798145f30 2024-04-17T09:55:55.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:55.524+0000 7f779e800640 1 -- 172.21.15.32:0/330806752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f77980cd1a0 con 0x7f7798145f30 2024-04-17T09:55:56.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.152+0000 7f778fe00640 1 -- 172.21.15.32:0/330806752 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v158) ==== 172+0+0 (secure 0 0 0) 0x7f7780059f10 con 0x7f7798145f30 2024-04-17T09:55:56.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.156+0000 7f779e800640 1 -- 172.21.15.32:0/330806752 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7774068380 msgr2=0x7f777406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:56.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.156+0000 7f779e800640 1 --2- 172.21.15.32:0/330806752 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7774068380 0x7f777406a840 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7788007a60 tx=0x7f7788008040 comp rx=0 tx=0).stop 2024-04-17T09:55:56.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.156+0000 7f779e800640 1 -- 172.21.15.32:0/330806752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7798145f30 msgr2=0x7f77981e8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:56.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.156+0000 7f779e800640 1 --2- 172.21.15.32:0/330806752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7798145f30 0x7f77981e8710 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f778000c370 tx=0x7f778000c830 comp rx=0 tx=0).stop 2024-04-17T09:55:56.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.156+0000 7f779e800640 1 -- 172.21.15.32:0/330806752 shutdown_connections 2024-04-17T09:55:56.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.156+0000 7f779e800640 1 --2- 172.21.15.32:0/330806752 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7774068380 0x7f777406a840 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:56.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.156+0000 7f779e800640 1 --2- 172.21.15.32:0/330806752 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f779815d8d0 0x7f77981e8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.156+0000 7f779e800640 1 --2- 172.21.15.32:0/330806752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7798145f30 0x7f77981e8710 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.156+0000 7f779e800640 1 --2- 172.21.15.32:0/330806752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77980cd580 0x7f7798140dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.156+0000 7f779e800640 1 -- 172.21.15.32:0/330806752 >> 172.21.15.32:0/330806752 conn(0x7f779806ba30 msgr2=0x7f7798160600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.156+0000 7f779e800640 1 -- 172.21.15.32:0/330806752 shutdown_connections 2024-04-17T09:55:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.156+0000 7f779e800640 1 -- 172.21.15.32:0/330806752 wait complete. 2024-04-17T09:55:56.179 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-17T09:55:56.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.320+0000 7fab4a600640 1 Processor -- start 2024-04-17T09:55:56.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.320+0000 7fab4a600640 1 -- start start 2024-04-17T09:55:56.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.320+0000 7fab4a600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab4415b7d0 0x7fab4406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:56.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.324+0000 7fab4a600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab4406f7c0 0x7fab4406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:56.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.324+0000 7fab4a600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fab4406f270 0x7fab4406bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:56.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.324+0000 7fab4a600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fab440cc940 con 0x7fab4415b7d0 2024-04-17T09:55:56.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.324+0000 7fab4a600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fab440ccc20 con 0x7fab4406f270 2024-04-17T09:55:56.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.324+0000 7fab4a600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fab4406e0b0 con 0x7fab4406f7c0 2024-04-17T09:55:56.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.324+0000 7fab43400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab4415b7d0 0x7fab4406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:56.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.324+0000 7fab43400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab4415b7d0 0x7fab4406e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37316/0 (socket says 172.21.15.32:37316) 2024-04-17T09:55:56.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.324+0000 7fab43400640 1 -- 172.21.15.32:0/1382188910 learned_addr learned my addr 172.21.15.32:0/1382188910 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:56.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.324+0000 7fab43400640 1 -- 172.21.15.32:0/1382188910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab4406f7c0 msgr2=0x7fab4406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:56.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.324+0000 7fab43400640 1 --2- 172.21.15.32:0/1382188910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab4406f7c0 0x7fab4406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:56.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.324+0000 7fab43400640 1 -- 172.21.15.32:0/1382188910 >> v2:172.21.15.144:3300/0 conn(0x7fab4406f270 msgr2=0x7fab4406bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:56.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.324+0000 7fab43e00640 1 --2- 172.21.15.32:0/1382188910 >> v2:172.21.15.144:3300/0 conn(0x7fab4406f270 0x7fab4406bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:56.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.324+0000 7fab43400640 1 --2- 172.21.15.32:0/1382188910 >> v2:172.21.15.144:3300/0 conn(0x7fab4406f270 0x7fab4406bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:56.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.324+0000 7fab43400640 1 -- 172.21.15.32:0/1382188910 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4406e390 con 0x7fab4415b7d0 2024-04-17T09:55:56.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.324+0000 7fab43e00640 1 --2- 172.21.15.32:0/1382188910 >> v2:172.21.15.144:3300/0 conn(0x7fab4406f270 0x7fab4406bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:55:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.324+0000 7fab43400640 1 --2- 172.21.15.32:0/1382188910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4415b7d0 0x7fab4406e8b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fab440b44e0 tx=0x7fab3401eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc09ed77af667ee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.324+0000 7fab42a00640 1 -- 172.21.15.32:0/1382188910 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab340b4070 con 0x7fab4415b7d0 2024-04-17T09:55:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.324+0000 7fab42a00640 1 -- 172.21.15.32:0/1382188910 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab340a7b00 con 0x7fab4415b7d0 2024-04-17T09:55:56.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.324+0000 7fab42a00640 1 -- 172.21.15.32:0/1382188910 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab340af440 con 0x7fab4415b7d0 2024-04-17T09:55:56.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab4a600640 1 -- 172.21.15.32:0/1382188910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4415b7d0 msgr2=0x7fab4406e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:56.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab4a600640 1 --2- 172.21.15.32:0/1382188910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4415b7d0 0x7fab4406e8b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fab440b44e0 tx=0x7fab3401eb60 comp rx=0 tx=0).stop 2024-04-17T09:55:56.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab4a600640 1 -- 172.21.15.32:0/1382188910 shutdown_connections 2024-04-17T09:55:56.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab4a600640 1 --2- 172.21.15.32:0/1382188910 >> v2:172.21.15.144:3300/0 conn(0x7fab4406f270 0x7fab4406bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:56.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab4a600640 1 --2- 172.21.15.32:0/1382188910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab4406f7c0 0x7fab4406ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:56.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab4a600640 1 --2- 172.21.15.32:0/1382188910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4415b7d0 0x7fab4406e8b0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:56.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab4a600640 1 -- 172.21.15.32:0/1382188910 >> 172.21.15.32:0/1382188910 conn(0x7fab440b2120 msgr2=0x7fab440b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:56.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab4a600640 1 -- 172.21.15.32:0/1382188910 shutdown_connections 2024-04-17T09:55:56.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab4a600640 1 -- 172.21.15.32:0/1382188910 wait complete. 2024-04-17T09:55:56.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab4a600640 1 Processor -- start 2024-04-17T09:55:56.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab4a600640 1 -- start start 2024-04-17T09:55:56.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab4a600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab4406f270 0x7fab4415e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab4a600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4406f7c0 0x7fab44071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab4a600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab4415b7d0 0x7fab440723a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab4a600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fab440cee70 con 0x7fab4406f7c0 2024-04-17T09:55:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab4a600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fab440cecf0 con 0x7fab4415b7d0 2024-04-17T09:55:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab4a600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fab440b3420 con 0x7fab4406f270 2024-04-17T09:55:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab3aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4406f7c0 0x7fab44071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab3aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4406f7c0 0x7fab44071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37324/0 (socket says 172.21.15.32:37324) 2024-04-17T09:55:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab3aa00640 1 -- 172.21.15.32:0/780166297 learned_addr learned my addr 172.21.15.32:0/780166297 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab43e00640 1 --2- 172.21.15.32:0/780166297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab4415b7d0 0x7fab440723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab43400640 1 --2- 172.21.15.32:0/780166297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab4406f270 0x7fab4415e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab3aa00640 1 -- 172.21.15.32:0/780166297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab4406f270 msgr2=0x7fab4415e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab3aa00640 1 --2- 172.21.15.32:0/780166297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab4406f270 0x7fab4415e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab3aa00640 1 -- 172.21.15.32:0/780166297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab4415b7d0 msgr2=0x7fab440723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab3aa00640 1 --2- 172.21.15.32:0/780166297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab4415b7d0 0x7fab440723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab3aa00640 1 -- 172.21.15.32:0/780166297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4414ffa0 con 0x7fab4406f7c0 2024-04-17T09:55:56.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.328+0000 7fab43400640 1 --2- 172.21.15.32:0/780166297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab4406f270 0x7fab4415e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:55:56.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.332+0000 7fab3aa00640 1 --2- 172.21.15.32:0/780166297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4406f7c0 0x7fab44071e60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00b550 tx=0x7fab2c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:56.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.332+0000 7fab3be00640 1 -- 172.21.15.32:0/780166297 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c014070 con 0x7fab4406f7c0 2024-04-17T09:55:56.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.332+0000 7fab3be00640 1 -- 172.21.15.32:0/780166297 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab2c010890 con 0x7fab4406f7c0 2024-04-17T09:55:56.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.332+0000 7fab3be00640 1 -- 172.21.15.32:0/780166297 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c010c80 con 0x7fab4406f7c0 2024-04-17T09:55:56.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.332+0000 7fab4a600640 1 -- 172.21.15.32:0/780166297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab44150280 con 0x7fab4406f7c0 2024-04-17T09:55:56.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.332+0000 7fab4a600640 1 -- 172.21.15.32:0/780166297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab4415e990 con 0x7fab4406f7c0 2024-04-17T09:55:56.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.332+0000 7fab4a600640 1 -- 172.21.15.32:0/780166297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab44068a90 con 0x7fab4406f7c0 2024-04-17T09:55:56.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.332+0000 7fab3be00640 1 -- 172.21.15.32:0/780166297 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fab2c017070 con 0x7fab4406f7c0 2024-04-17T09:55:56.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.332+0000 7fab3be00640 1 --2- 172.21.15.32:0/780166297 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab20068380 0x7fab2006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:56.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.332+0000 7fab3be00640 1 -- 172.21.15.32:0/780166297 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7fab2c086580 con 0x7fab4406f7c0 2024-04-17T09:55:56.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.332+0000 7fab43400640 1 --2- 172.21.15.32:0/780166297 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab20068380 0x7fab2006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:56.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.336+0000 7fab43400640 1 --2- 172.21.15.32:0/780166297 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab20068380 0x7fab2006a840 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fab340b1000 tx=0x7fab34008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:56.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.336+0000 7fab3be00640 1 -- 172.21.15.32:0/780166297 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fab2c056da0 con 0x7fab4406f7c0 2024-04-17T09:55:56.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.504+0000 7fab4a600640 1 -- 172.21.15.32:0/780166297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fab44145990 con 0x7fab4406f7c0 2024-04-17T09:55:56.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.508+0000 7fab3be00640 1 -- 172.21.15.32:0/780166297 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7fab2c010e20 con 0x7fab4406f7c0 2024-04-17T09:55:56.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.508+0000 7fab4a600640 1 -- 172.21.15.32:0/780166297 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab20068380 msgr2=0x7fab2006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:56.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.508+0000 7fab4a600640 1 --2- 172.21.15.32:0/780166297 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab20068380 0x7fab2006a840 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fab340b1000 tx=0x7fab34008040 comp rx=0 tx=0).stop 2024-04-17T09:55:56.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.508+0000 7fab4a600640 1 -- 172.21.15.32:0/780166297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4406f7c0 msgr2=0x7fab44071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:56.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.508+0000 7fab4a600640 1 --2- 172.21.15.32:0/780166297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4406f7c0 0x7fab44071e60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00b550 tx=0x7fab2c00ba10 comp rx=0 tx=0).stop 2024-04-17T09:55:56.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.508+0000 7fab4a600640 1 -- 172.21.15.32:0/780166297 shutdown_connections 2024-04-17T09:55:56.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.508+0000 7fab4a600640 1 --2- 172.21.15.32:0/780166297 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab20068380 0x7fab2006a840 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:56.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.508+0000 7fab4a600640 1 --2- 172.21.15.32:0/780166297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab4415b7d0 0x7fab440723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:56.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.508+0000 7fab4a600640 1 --2- 172.21.15.32:0/780166297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4406f7c0 0x7fab44071e60 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:56.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.508+0000 7fab4a600640 1 --2- 172.21.15.32:0/780166297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab4406f270 0x7fab4415e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:56.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.508+0000 7fab4a600640 1 -- 172.21.15.32:0/780166297 >> 172.21.15.32:0/780166297 conn(0x7fab440b2120 msgr2=0x7fab441549f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:56.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.508+0000 7fab4a600640 1 -- 172.21.15.32:0/780166297 shutdown_connections 2024-04-17T09:55:56.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.508+0000 7fab4a600640 1 -- 172.21.15.32:0/780166297 wait complete. 2024-04-17T09:55:56.529 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T09:55:56.530 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T09:55:56.599 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T09:55:56.693 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T09:55:56.694 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:55:56.694 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:55:56.702 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T09:55:56.703 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:55:56.703 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:55:56.712 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T09:55:56.712 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:55:56.712 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:55:56.751 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T09:55:56.751 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:55:56.751 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:55:56.762 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T09:55:56.763 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-17T09:55:56.763 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:55:56.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.936+0000 7f2971a00640 1 Processor -- start 2024-04-17T09:55:56.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.936+0000 7f2971a00640 1 -- start start 2024-04-17T09:55:56.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.936+0000 7f2971a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f296c144e90 0x7f296c0b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:56.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.936+0000 7f2971a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f296c0b4de0 0x7f296c15baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:56.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.936+0000 7f2971a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f296c072bb0 con 0x7f296c15bfe0 2024-04-17T09:55:56.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.936+0000 7f2971a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f296c15c430 con 0x7f296c0b4de0 2024-04-17T09:55:56.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.936+0000 7f2971a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f296c145270 con 0x7f296c144e90 2024-04-17T09:55:56.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.936+0000 7f296a000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f296c0b4de0 0x7f296c15baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:56.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.936+0000 7f296b400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f296c15bfe0 0x7f296c06ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43352/0 (socket says 172.21.15.32:43352) 2024-04-17T09:55:56.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.936+0000 7f296b400640 1 -- 172.21.15.32:0/2849585965 learned_addr learned my addr 172.21.15.32:0/2849585965 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:56.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.936+0000 7f296aa00640 1 --2- 172.21.15.32:0/2849585965 >> v2:172.21.15.144:3300/0 conn(0x7f296c144e90 0x7f296c0b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:56.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.936+0000 7f296a000640 1 -- 172.21.15.32:0/2849585965 >> v2:172.21.15.144:3300/0 conn(0x7f296c144e90 msgr2=0x7f296c0b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:56.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.936+0000 7f296a000640 1 --2- 172.21.15.32:0/2849585965 >> v2:172.21.15.144:3300/0 conn(0x7f296c144e90 0x7f296c0b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:56.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.936+0000 7f296a000640 1 -- 172.21.15.32:0/2849585965 >> v1:172.21.15.32:6789/0 conn(0x7f296c15bfe0 legacy=0x7f296c06ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:56.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.936+0000 7f296a000640 1 -- 172.21.15.32:0/2849585965 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f296c0b3dc0 con 0x7f296c0b4de0 2024-04-17T09:55:56.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.936+0000 7f2969600640 1 -- 172.21.15.32:0/2849585965 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3890211239 0 0) 0x7f296c072bb0 con 0x7f296c15bfe0 2024-04-17T09:55:56.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.940+0000 7f296a000640 1 --2- 172.21.15.32:0/2849585965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296c0b4de0 0x7f296c15baa0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f296c06ff30 tx=0x7f295801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f78a6c6f638be84 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:56.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.940+0000 7f2969600640 1 -- 172.21.15.32:0/2849585965 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29580ab430 con 0x7f296c0b4de0 2024-04-17T09:55:56.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.940+0000 7f2969600640 1 -- 172.21.15.32:0/2849585965 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29580b0070 con 0x7f296c0b4de0 2024-04-17T09:55:56.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.940+0000 7f2969600640 1 -- 172.21.15.32:0/2849585965 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29580ab7f0 con 0x7f296c0b4de0 2024-04-17T09:55:56.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.940+0000 7f2971a00640 1 -- 172.21.15.32:0/2849585965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296c0b4de0 msgr2=0x7f296c15baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:56.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.940+0000 7f2971a00640 1 --2- 172.21.15.32:0/2849585965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296c0b4de0 0x7f296c15baa0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f296c06ff30 tx=0x7f295801c430 comp rx=0 tx=0).stop 2024-04-17T09:55:56.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.940+0000 7f2971a00640 1 -- 172.21.15.32:0/2849585965 shutdown_connections 2024-04-17T09:55:56.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.940+0000 7f2971a00640 1 --2- 172.21.15.32:0/2849585965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296c0b4de0 0x7f296c15baa0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:56.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.940+0000 7f2971a00640 1 --2- 172.21.15.32:0/2849585965 >> v2:172.21.15.144:3300/0 conn(0x7f296c144e90 0x7f296c0b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:56.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.940+0000 7f2971a00640 1 -- 172.21.15.32:0/2849585965 >> 172.21.15.32:0/2849585965 conn(0x7f296c06ba30 msgr2=0x7f296c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:56.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.940+0000 7f2971a00640 1 -- 172.21.15.32:0/2849585965 shutdown_connections 2024-04-17T09:55:56.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.940+0000 7f2971a00640 1 -- 172.21.15.32:0/2849585965 wait complete. 2024-04-17T09:55:56.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.940+0000 7f2971a00640 1 Processor -- start 2024-04-17T09:55:56.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.940+0000 7f2971a00640 1 -- start start 2024-04-17T09:55:56.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.944+0000 7f2971a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f296c0b4de0 0x7f296c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:56.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.944+0000 7f2971a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296c144e90 0x7f296c140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:56.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.944+0000 7f2971a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f296c15bfe0 0x7f296c140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:56.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.944+0000 7f2971a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f296c072670 con 0x7f296c144e90 2024-04-17T09:55:56.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.944+0000 7f2971a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f296c06f880 con 0x7f296c0b4de0 2024-04-17T09:55:56.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.944+0000 7f2971a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f296c0727f0 con 0x7f296c15bfe0 2024-04-17T09:55:56.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.944+0000 7f296a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296c144e90 0x7f296c140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:56.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.944+0000 7f296a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296c144e90 0x7f296c140ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37344/0 (socket says 172.21.15.32:37344) 2024-04-17T09:55:56.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.944+0000 7f296a000640 1 -- 172.21.15.32:0/1921162784 learned_addr learned my addr 172.21.15.32:0/1921162784 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:56.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.944+0000 7f296aa00640 1 --2- 172.21.15.32:0/1921162784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f296c0b4de0 0x7f296c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:56.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.944+0000 7f296b400640 1 --2- 172.21.15.32:0/1921162784 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f296c15bfe0 0x7f296c140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:56.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.944+0000 7f296a000640 1 -- 172.21.15.32:0/1921162784 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f296c15bfe0 msgr2=0x7f296c140ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:56.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.944+0000 7f296a000640 1 --2- 172.21.15.32:0/1921162784 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f296c15bfe0 0x7f296c140ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:56.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.944+0000 7f296a000640 1 -- 172.21.15.32:0/1921162784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f296c0b4de0 msgr2=0x7f296c15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:56.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.944+0000 7f296a000640 1 --2- 172.21.15.32:0/1921162784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f296c0b4de0 0x7f296c15ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:56.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.944+0000 7f296a000640 1 -- 172.21.15.32:0/1921162784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f296c0b0e50 con 0x7f296c144e90 2024-04-17T09:55:56.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.944+0000 7f296a000640 1 --2- 172.21.15.32:0/1921162784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296c144e90 0x7f296c140ab0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f29580ad040 tx=0x7f295801ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:56.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.944+0000 7f2962a00640 1 -- 172.21.15.32:0/1921162784 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29580bf610 con 0x7f296c144e90 2024-04-17T09:55:56.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.944+0000 7f2962a00640 1 -- 172.21.15.32:0/1921162784 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29580bf7f0 con 0x7f296c144e90 2024-04-17T09:55:56.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.944+0000 7f2962a00640 1 -- 172.21.15.32:0/1921162784 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29580bfbe0 con 0x7f296c144e90 2024-04-17T09:55:56.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.944+0000 7f2971a00640 1 -- 172.21.15.32:0/1921162784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f296c0b1130 con 0x7f296c144e90 2024-04-17T09:55:56.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.944+0000 7f2971a00640 1 -- 172.21.15.32:0/1921162784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f296c0b16f0 con 0x7f296c144e90 2024-04-17T09:55:56.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.944+0000 7f2971a00640 1 -- 172.21.15.32:0/1921162784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f296c068a90 con 0x7f296c144e90 2024-04-17T09:55:56.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.948+0000 7f2962a00640 1 -- 172.21.15.32:0/1921162784 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f29580bd020 con 0x7f296c144e90 2024-04-17T09:55:56.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.948+0000 7f2962a00640 1 --2- 172.21.15.32:0/1921162784 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f294c068380 0x7f294c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:56.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.948+0000 7f2962a00640 1 -- 172.21.15.32:0/1921162784 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f29580a8030 con 0x7f296c144e90 2024-04-17T09:55:56.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.948+0000 7f296aa00640 1 --2- 172.21.15.32:0/1921162784 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f294c068380 0x7f294c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:56.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.948+0000 7f296aa00640 1 --2- 172.21.15.32:0/1921162784 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f294c068380 0x7f294c06a840 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f295c007a60 tx=0x7f295c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:56.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:56.952+0000 7f2962a00640 1 -- 172.21.15.32:0/1921162784 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2958016050 con 0x7f296c144e90 2024-04-17T09:55:57.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.148+0000 7f2971a00640 1 -- 172.21.15.32:0/1921162784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f296c0cda90 con 0x7f296c144e90 2024-04-17T09:55:57.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.148+0000 7f2962a00640 1 -- 172.21.15.32:0/1921162784 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+3869 (secure 0 0 0) 0x7f29580f0840 con 0x7f296c144e90 2024-04-17T09:55:57.152 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:57.152 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2542578926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2542578926},{"type":"v1","addr":"172.21.15.144:6834","nonce":2542578926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1424948520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1424948520},{"type":"v1","addr":"172.21.15.32:6836","nonce":1424948520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/393813154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":393813154},{"type":"v1","addr":"172.21.15.32:6837","nonce":393813154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5830,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/305283392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":305283392},{"type":"v1","addr":"172.21.15.144:6835","nonce":305283392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[]} 2024-04-17T09:55:57.152 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 26 2024-04-17T09:55:57.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.152+0000 7f2971a00640 1 -- 172.21.15.32:0/1921162784 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f294c068380 msgr2=0x7f294c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:57.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.152+0000 7f2971a00640 1 --2- 172.21.15.32:0/1921162784 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f294c068380 0x7f294c06a840 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f295c007a60 tx=0x7f295c008040 comp rx=0 tx=0).stop 2024-04-17T09:55:57.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.152+0000 7f2971a00640 1 -- 172.21.15.32:0/1921162784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296c144e90 msgr2=0x7f296c140ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:57.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.152+0000 7f2971a00640 1 --2- 172.21.15.32:0/1921162784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296c144e90 0x7f296c140ab0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f29580ad040 tx=0x7f295801ac70 comp rx=0 tx=0).stop 2024-04-17T09:55:57.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.152+0000 7f2971a00640 1 -- 172.21.15.32:0/1921162784 shutdown_connections 2024-04-17T09:55:57.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.152+0000 7f2971a00640 1 --2- 172.21.15.32:0/1921162784 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f294c068380 0x7f294c06a840 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:57.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.152+0000 7f2971a00640 1 --2- 172.21.15.32:0/1921162784 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f296c15bfe0 0x7f296c140ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:57.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.152+0000 7f2971a00640 1 --2- 172.21.15.32:0/1921162784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296c144e90 0x7f296c140ab0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:57.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.152+0000 7f2971a00640 1 --2- 172.21.15.32:0/1921162784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f296c0b4de0 0x7f296c15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:57.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.152+0000 7f2971a00640 1 -- 172.21.15.32:0/1921162784 >> 172.21.15.32:0/1921162784 conn(0x7f296c06ba30 msgr2=0x7f296c144020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:57.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.152+0000 7f2971a00640 1 -- 172.21.15.32:0/1921162784 shutdown_connections 2024-04-17T09:55:57.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.152+0000 7f2971a00640 1 -- 172.21.15.32:0/1921162784 wait complete. 2024-04-17T09:55:57.174 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-17T09:55:57.174 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-17T09:55:57.174 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-17T09:55:57.175 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:55:57.175 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-17T09:55:57.175 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-17T09:55:57.175 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-17T09:55:57.175 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:55:57.175 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-17T09:55:57.175 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-17T09:55:57.176 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-17T09:55:57.176 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:55:57.176 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-17T09:55:57.176 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-17T09:55:57.176 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-17T09:55:57.176 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:55:57.187 INFO:tasks.ceph.mds.c:Stopped 2024-04-17T09:55:57.187 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-17T09:55:57.188 INFO:tasks.ceph.mds.a:Stopped 2024-04-17T09:55:57.188 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-17T09:55:57.193 INFO:tasks.ceph.mds.b:Stopped 2024-04-17T09:55:57.193 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-17T09:55:57.194 INFO:tasks.ceph.mds.d:Stopped 2024-04-17T09:55:57.195 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-17T09:55:57.234 INFO:tasks.ceph.mds.c:Started 2024-04-17T09:55:57.235 DEBUG:teuthology.parallel:result is None 2024-04-17T09:55:57.235 INFO:tasks.ceph.mds.a:Started 2024-04-17T09:55:57.236 DEBUG:teuthology.parallel:result is None 2024-04-17T09:55:57.244 INFO:tasks.ceph.mds.b:Started 2024-04-17T09:55:57.245 DEBUG:teuthology.parallel:result is None 2024-04-17T09:55:57.247 INFO:tasks.ceph.mds.d:Started 2024-04-17T09:55:57.247 DEBUG:teuthology.parallel:result is None 2024-04-17T09:55:57.247 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-17T09:55:57.322 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-17T09:55:57.322 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-17T09:55:57.335 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2024-04-17T09:55:57.336 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2024-04-17T09:55:57.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.392+0000 7f3c32e00640 1 Processor -- start 2024-04-17T09:55:57.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.392+0000 7f3c32e00640 1 -- start start 2024-04-17T09:55:57.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.392+0000 7f3c32e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c2c155e50 0x7f3c2c15f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:57.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.392+0000 7f3c32e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c2c15f870 con 0x7f3c2c155a70 2024-04-17T09:55:57.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.392+0000 7f3c32e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c2c15fb50 con 0x7f3c2c0b3340 2024-04-17T09:55:57.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.392+0000 7f3c32e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3c2c0b42a0 con 0x7f3c2c155e50 2024-04-17T09:55:57.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.396+0000 7f3c2be00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3c2c155a70 0x7f3c2c15d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43372/0 (socket says 172.21.15.32:43372) 2024-04-17T09:55:57.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.396+0000 7f3c2be00640 1 -- 172.21.15.32:0/3746524351 learned_addr learned my addr 172.21.15.32:0/3746524351 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:57.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.396+0000 7f3c23e00640 1 --2- 172.21.15.32:0/3746524351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c2c155e50 0x7f3c2c15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.396+0000 7f3c2b400640 1 -- 172.21.15.32:0/3746524351 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 103238087 0 0) 0x7f3c2c15f870 con 0x7f3c2c155a70 2024-04-17T09:55:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.396+0000 7f3c2b400640 1 -- 172.21.15.32:0/3746524351 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c10005740 con 0x7f3c2c155a70 2024-04-17T09:55:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.396+0000 7f3c2b400640 1 -- 172.21.15.32:0/3746524351 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4246844202 0 0) 0x7f3c2c15fb50 con 0x7f3c2c0b3340 2024-04-17T09:55:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.396+0000 7f3c2b400640 1 -- 172.21.15.32:0/3746524351 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c2c15f870 con 0x7f3c2c0b3340 2024-04-17T09:55:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.396+0000 7f3c23e00640 1 -- 172.21.15.32:0/3746524351 >> v1:172.21.15.144:6789/0 conn(0x7f3c2c0b3340 legacy=0x7f3c2c15c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.396+0000 7f3c23e00640 1 -- 172.21.15.32:0/3746524351 >> v1:172.21.15.32:6789/0 conn(0x7f3c2c155a70 legacy=0x7f3c2c15d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.396+0000 7f3c23e00640 1 -- 172.21.15.32:0/3746524351 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2c15fe30 con 0x7f3c2c155e50 2024-04-17T09:55:57.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.396+0000 7f3c23e00640 1 --2- 172.21.15.32:0/3746524351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c2c155e50 0x7f3c2c15f190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c0b4420 tx=0x7f3c1401c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e14acd22669b6b0e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:57.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.400+0000 7f3c2b400640 1 -- 172.21.15.32:0/3746524351 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c140b0070 con 0x7f3c2c155e50 2024-04-17T09:55:57.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.400+0000 7f3c2b400640 1 -- 172.21.15.32:0/3746524351 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c140ab600 con 0x7f3c2c155e50 2024-04-17T09:55:57.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.400+0000 7f3c2b400640 1 -- 172.21.15.32:0/3746524351 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c140ab9d0 con 0x7f3c2c155e50 2024-04-17T09:55:57.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.400+0000 7f3c32e00640 1 -- 172.21.15.32:0/3746524351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c2c155e50 msgr2=0x7f3c2c15f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:57.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.400+0000 7f3c32e00640 1 --2- 172.21.15.32:0/3746524351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c2c155e50 0x7f3c2c15f190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c0b4420 tx=0x7f3c1401c4c0 comp rx=0 tx=0).stop 2024-04-17T09:55:57.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.400+0000 7f3c32e00640 1 -- 172.21.15.32:0/3746524351 shutdown_connections 2024-04-17T09:55:57.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.400+0000 7f3c32e00640 1 --2- 172.21.15.32:0/3746524351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c2c155e50 0x7f3c2c15f190 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:57.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.400+0000 7f3c32e00640 1 -- 172.21.15.32:0/3746524351 >> 172.21.15.32:0/3746524351 conn(0x7f3c2c06ba30 msgr2=0x7f3c2c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:57.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.400+0000 7f3c32e00640 1 -- 172.21.15.32:0/3746524351 shutdown_connections 2024-04-17T09:55:57.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.400+0000 7f3c32e00640 1 -- 172.21.15.32:0/3746524351 wait complete. 2024-04-17T09:55:57.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.400+0000 7f3c32e00640 1 Processor -- start 2024-04-17T09:55:57.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.400+0000 7f3c32e00640 1 -- start start 2024-04-17T09:55:57.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.400+0000 7f3c32e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c2c0b3340 0x7f3c2c148ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.400+0000 7f3c32e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c2c155a70 0x7f3c2c14aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.400+0000 7f3c32e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c2c155e50 0x7f3c2c14bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.400+0000 7f3c32e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c0d02f0 con 0x7f3c2c0b3340 2024-04-17T09:55:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.400+0000 7f3c32e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c0d0170 con 0x7f3c2c155a70 2024-04-17T09:55:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.400+0000 7f3c32e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3c2c0d0470 con 0x7f3c2c155e50 2024-04-17T09:55:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.400+0000 7f3c2be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c2c0b3340 0x7f3c2c148ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.400+0000 7f3c2be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c2c0b3340 0x7f3c2c148ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37380/0 (socket says 172.21.15.32:37380) 2024-04-17T09:55:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.400+0000 7f3c2be00640 1 -- 172.21.15.32:0/79959877 learned_addr learned my addr 172.21.15.32:0/79959877 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.400+0000 7f3c23e00640 1 --2- 172.21.15.32:0/79959877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c2c155a70 0x7f3c2c14aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.400+0000 7f3c30e00640 1 --2- 172.21.15.32:0/79959877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c2c155e50 0x7f3c2c14bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.404+0000 7f3c2be00640 1 -- 172.21.15.32:0/79959877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c2c155e50 msgr2=0x7f3c2c14bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.404+0000 7f3c2be00640 1 --2- 172.21.15.32:0/79959877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c2c155e50 0x7f3c2c14bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.404+0000 7f3c2be00640 1 -- 172.21.15.32:0/79959877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c2c155a70 msgr2=0x7f3c2c14aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:57.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.404+0000 7f3c2be00640 1 --2- 172.21.15.32:0/79959877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c2c155a70 0x7f3c2c14aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:57.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.404+0000 7f3c2be00640 1 -- 172.21.15.32:0/79959877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2c14b260 con 0x7f3c2c0b3340 2024-04-17T09:55:57.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.404+0000 7f3c30e00640 1 --2- 172.21.15.32:0/79959877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c2c155e50 0x7f3c2c14bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:55:57.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.404+0000 7f3c2be00640 1 --2- 172.21.15.32:0/79959877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c2c0b3340 0x7f3c2c148ee0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00c520 tx=0x7f3c1c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:57.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.404+0000 7f3c28c00640 1 -- 172.21.15.32:0/79959877 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c017070 con 0x7f3c2c0b3340 2024-04-17T09:55:57.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.404+0000 7f3c28c00640 1 -- 172.21.15.32:0/79959877 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c1c005cf0 con 0x7f3c2c0b3340 2024-04-17T09:55:57.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.404+0000 7f3c32e00640 1 -- 172.21.15.32:0/79959877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c2c0ce2a0 con 0x7f3c2c0b3340 2024-04-17T09:55:57.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.404+0000 7f3c28c00640 1 -- 172.21.15.32:0/79959877 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c007270 con 0x7f3c2c0b3340 2024-04-17T09:55:57.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.404+0000 7f3c32e00640 1 -- 172.21.15.32:0/79959877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c2c0ce860 con 0x7f3c2c0b3340 2024-04-17T09:55:57.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.404+0000 7f3c32e00640 1 -- 172.21.15.32:0/79959877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c2c068a90 con 0x7f3c2c0b3340 2024-04-17T09:55:57.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.404+0000 7f3c28c00640 1 -- 172.21.15.32:0/79959877 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3c1c01a070 con 0x7f3c2c0b3340 2024-04-17T09:55:57.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.404+0000 7f3c28c00640 1 --2- 172.21.15.32:0/79959877 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c08068380 0x7f3c0806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:57.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.404+0000 7f3c28c00640 1 -- 172.21.15.32:0/79959877 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f3c1c085a00 con 0x7f3c2c0b3340 2024-04-17T09:55:57.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.404+0000 7f3c23e00640 1 --2- 172.21.15.32:0/79959877 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c08068380 0x7f3c0806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:57.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.408+0000 7f3c23e00640 1 --2- 172.21.15.32:0/79959877 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c08068380 0x7f3c0806a840 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3c140ad000 tx=0x7f3c140a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:57.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.408+0000 7f3c28c00640 1 -- 172.21.15.32:0/79959877 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3c1c055bf0 con 0x7f3c2c0b3340 2024-04-17T09:55:57.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:57.584+0000 7f3c32e00640 1 -- 172.21.15.32:0/79959877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3c2c06f720 con 0x7f3c2c0b3340 2024-04-17T09:55:58.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.168+0000 7f3c28c00640 1 -- 172.21.15.32:0/79959877 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v51) ==== 97+0+0 (secure 0 0 0) 0x7f3c1c059ef0 con 0x7f3c2c0b3340 2024-04-17T09:55:58.169 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2024-04-17T09:55:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.172+0000 7f3c32e00640 1 -- 172.21.15.32:0/79959877 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c08068380 msgr2=0x7f3c0806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.172+0000 7f3c32e00640 1 --2- 172.21.15.32:0/79959877 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c08068380 0x7f3c0806a840 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3c140ad000 tx=0x7f3c140a6000 comp rx=0 tx=0).stop 2024-04-17T09:55:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.172+0000 7f3c32e00640 1 -- 172.21.15.32:0/79959877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c2c0b3340 msgr2=0x7f3c2c148ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.172+0000 7f3c32e00640 1 --2- 172.21.15.32:0/79959877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c2c0b3340 0x7f3c2c148ee0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00c520 tx=0x7f3c1c00c9e0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.172+0000 7f3c32e00640 1 -- 172.21.15.32:0/79959877 shutdown_connections 2024-04-17T09:55:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.172+0000 7f3c32e00640 1 --2- 172.21.15.32:0/79959877 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c08068380 0x7f3c0806a840 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.172+0000 7f3c32e00640 1 --2- 172.21.15.32:0/79959877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c2c155e50 0x7f3c2c14bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.172+0000 7f3c32e00640 1 --2- 172.21.15.32:0/79959877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c2c155a70 0x7f3c2c14aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.172+0000 7f3c32e00640 1 --2- 172.21.15.32:0/79959877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c2c0b3340 0x7f3c2c148ee0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.172+0000 7f3c32e00640 1 -- 172.21.15.32:0/79959877 >> 172.21.15.32:0/79959877 conn(0x7f3c2c06ba30 msgr2=0x7f3c2c15e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.172+0000 7f3c32e00640 1 -- 172.21.15.32:0/79959877 shutdown_connections 2024-04-17T09:55:58.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.172+0000 7f3c32e00640 1 -- 172.21.15.32:0/79959877 wait complete. 2024-04-17T09:55:58.196 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-17T09:55:58.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.336+0000 7f1e71a00640 1 Processor -- start 2024-04-17T09:55:58.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.336+0000 7f1e71a00640 1 -- start start 2024-04-17T09:55:58.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.336+0000 7f1e71a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e6c09ba90 0x7f1e6c0cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:58.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.336+0000 7f1e71a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1e6c0ce0c0 0x7f1e6c15e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:58.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.336+0000 7f1e71a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e6c09a9f0 con 0x7f1e6c1604f0 2024-04-17T09:55:58.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.336+0000 7f1e71a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1e6c09b600 con 0x7f1e6c09ba90 2024-04-17T09:55:58.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.336+0000 7f1e71a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1e6c09acd0 con 0x7f1e6c0ce0c0 2024-04-17T09:55:58.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.340+0000 7f1e63e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e6c09ba90 0x7f1e6c0cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.340+0000 7f1e6aa00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1e6c1604f0 0x7f1e6c1608d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43382/0 (socket says 172.21.15.32:43382) 2024-04-17T09:55:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.340+0000 7f1e6aa00640 1 -- 172.21.15.32:0/719571961 learned_addr learned my addr 172.21.15.32:0/719571961 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.340+0000 7f1e6b400640 1 --2- 172.21.15.32:0/719571961 >> v2:172.21.15.144:3300/0 conn(0x7f1e6c0ce0c0 0x7f1e6c15e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.340+0000 7f1e63e00640 1 -- 172.21.15.32:0/719571961 >> v2:172.21.15.144:3300/0 conn(0x7f1e6c0ce0c0 msgr2=0x7f1e6c15e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.340+0000 7f1e63e00640 1 --2- 172.21.15.32:0/719571961 >> v2:172.21.15.144:3300/0 conn(0x7f1e6c0ce0c0 0x7f1e6c15e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.340+0000 7f1e63e00640 1 -- 172.21.15.32:0/719571961 >> v1:172.21.15.32:6789/0 conn(0x7f1e6c1604f0 legacy=0x7f1e6c1608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.340+0000 7f1e63e00640 1 -- 172.21.15.32:0/719571961 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6c09afb0 con 0x7f1e6c09ba90 2024-04-17T09:55:58.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.340+0000 7f1e6a000640 1 -- 172.21.15.32:0/719571961 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 373684701 0 0) 0x7f1e6c09a9f0 con 0x7f1e6c1604f0 2024-04-17T09:55:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.340+0000 7f1e63e00640 1 --2- 172.21.15.32:0/719571961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6c09ba90 0x7f1e6c0cdb80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f1e540097c0 tx=0x7f1e5401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3992d1dcc73e91 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.340+0000 7f1e6a000640 1 -- 172.21.15.32:0/719571961 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e540b0070 con 0x7f1e6c09ba90 2024-04-17T09:55:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.340+0000 7f1e6a000640 1 -- 172.21.15.32:0/719571961 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e540ab600 con 0x7f1e6c09ba90 2024-04-17T09:55:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.340+0000 7f1e6a000640 1 -- 172.21.15.32:0/719571961 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e540ab9d0 con 0x7f1e6c09ba90 2024-04-17T09:55:58.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e71a00640 1 -- 172.21.15.32:0/719571961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6c09ba90 msgr2=0x7f1e6c0cdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:58.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e71a00640 1 --2- 172.21.15.32:0/719571961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6c09ba90 0x7f1e6c0cdb80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f1e540097c0 tx=0x7f1e5401c460 comp rx=0 tx=0).stop 2024-04-17T09:55:58.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e71a00640 1 -- 172.21.15.32:0/719571961 shutdown_connections 2024-04-17T09:55:58.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e71a00640 1 --2- 172.21.15.32:0/719571961 >> v2:172.21.15.144:3300/0 conn(0x7f1e6c0ce0c0 0x7f1e6c15e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e71a00640 1 --2- 172.21.15.32:0/719571961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6c09ba90 0x7f1e6c0cdb80 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e71a00640 1 -- 172.21.15.32:0/719571961 >> 172.21.15.32:0/719571961 conn(0x7f1e6c0b4010 msgr2=0x7f1e6c0b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:58.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e71a00640 1 -- 172.21.15.32:0/719571961 shutdown_connections 2024-04-17T09:55:58.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e71a00640 1 -- 172.21.15.32:0/719571961 wait complete. 2024-04-17T09:55:58.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e71a00640 1 Processor -- start 2024-04-17T09:55:58.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e71a00640 1 -- start start 2024-04-17T09:55:58.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e71a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e6c09ba90 0x7f1e6c0ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:58.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e71a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6c0ce0c0 0x7f1e6c0cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:58.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e71a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e6c1604f0 0x7f1e6c152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:58.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e71a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c1699f0 con 0x7f1e6c0ce0c0 2024-04-17T09:55:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e71a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c169870 con 0x7f1e6c1604f0 2024-04-17T09:55:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e71a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1e6c169b70 con 0x7f1e6c09ba90 2024-04-17T09:55:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e63e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6c0ce0c0 0x7f1e6c0cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e63e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6c0ce0c0 0x7f1e6c0cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37394/0 (socket says 172.21.15.32:37394) 2024-04-17T09:55:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e63e00640 1 -- 172.21.15.32:0/1837048517 learned_addr learned my addr 172.21.15.32:0/1837048517 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e6aa00640 1 --2- 172.21.15.32:0/1837048517 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e6c09ba90 0x7f1e6c0ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e6b400640 1 --2- 172.21.15.32:0/1837048517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e6c1604f0 0x7f1e6c152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e63e00640 1 -- 172.21.15.32:0/1837048517 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e6c09ba90 msgr2=0x7f1e6c0ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e63e00640 1 --2- 172.21.15.32:0/1837048517 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e6c09ba90 0x7f1e6c0ca500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e63e00640 1 -- 172.21.15.32:0/1837048517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e6c1604f0 msgr2=0x7f1e6c152920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e63e00640 1 --2- 172.21.15.32:0/1837048517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e6c1604f0 0x7f1e6c152920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.344+0000 7f1e63e00640 1 -- 172.21.15.32:0/1837048517 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6c0c7810 con 0x7f1e6c0ce0c0 2024-04-17T09:55:58.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.348+0000 7f1e63e00640 1 --2- 172.21.15.32:0/1837048517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6c0ce0c0 0x7f1e6c0cbf40 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1e540ae040 tx=0x7f1e54019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:58.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.348+0000 7f1e62a00640 1 -- 172.21.15.32:0/1837048517 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e540b0040 con 0x7f1e6c0ce0c0 2024-04-17T09:55:58.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.348+0000 7f1e62a00640 1 -- 172.21.15.32:0/1837048517 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e540ab400 con 0x7f1e6c0ce0c0 2024-04-17T09:55:58.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.348+0000 7f1e62a00640 1 -- 172.21.15.32:0/1837048517 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e540b4650 con 0x7f1e6c0ce0c0 2024-04-17T09:55:58.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.348+0000 7f1e71a00640 1 -- 172.21.15.32:0/1837048517 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e6c0c7ab0 con 0x7f1e6c0ce0c0 2024-04-17T09:55:58.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.348+0000 7f1e71a00640 1 -- 172.21.15.32:0/1837048517 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e6c0c8070 con 0x7f1e6c0ce0c0 2024-04-17T09:55:58.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.348+0000 7f1e71a00640 1 -- 172.21.15.32:0/1837048517 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e6c068a90 con 0x7f1e6c0ce0c0 2024-04-17T09:55:58.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.348+0000 7f1e62a00640 1 -- 172.21.15.32:0/1837048517 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1e540bc020 con 0x7f1e6c0ce0c0 2024-04-17T09:55:58.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.348+0000 7f1e62a00640 1 --2- 172.21.15.32:0/1837048517 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1e48068380 0x7f1e4806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:58.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.348+0000 7f1e62a00640 1 -- 172.21.15.32:0/1837048517 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4857+0+0 (secure 0 0 0) 0x7f1e5411f840 con 0x7f1e6c0ce0c0 2024-04-17T09:55:58.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.348+0000 7f1e6aa00640 1 --2- 172.21.15.32:0/1837048517 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1e48068380 0x7f1e4806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:58.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.352+0000 7f1e6aa00640 1 --2- 172.21.15.32:0/1837048517 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1e48068380 0x7f1e4806a840 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c000f70 tx=0x7f1e5c007df0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:58.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.352+0000 7f1e62a00640 1 -- 172.21.15.32:0/1837048517 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f1e540c3020 con 0x7f1e6c0ce0c0 2024-04-17T09:55:58.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.576+0000 7f1e71a00640 1 -- 172.21.15.32:0/1837048517 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1e6c0ced10 con 0x7f1e6c0ce0c0 2024-04-17T09:55:58.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.576+0000 7f1e62a00640 1 -- 172.21.15.32:0/1837048517 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7f1e54016050 con 0x7f1e6c0ce0c0 2024-04-17T09:55:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:58.578 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:55:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2024-04-17T09:55:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2024-04-17T09:55:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mzM1KExAAn0WH+Gf6gGV/dJonXzAgPQ==", 2024-04-17T09:55:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:55:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:55:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:55:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:58.578 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2024-04-17T09:55:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mQnJ4HxAAT0rFHhdAj7PMxZGiqEyWbg==", 2024-04-17T09:55:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:55:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:55:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:55:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2024-04-17T09:55:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mWWi5GhAA3RAdRxK6GRwuVCH4DHoMeA==", 2024-04-17T09:55:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:58.579 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:55:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:55:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:55:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2024-04-17T09:55:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9m9TMAJxAAbmUp05GHRE1AqCmq83scVQ==", 2024-04-17T09:55:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:55:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:55:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:55:58.580 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2024-04-17T09:55:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDRmx9mnPwyDBAAxD10jhxptVH5zhupengElg==", 2024-04-17T09:55:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:55:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:55:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2024-04-17T09:55:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUmx9mh3iuGBAAqvWSRyNX3u/TjEH6TJpqdA==", 2024-04-17T09:55:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:58.581 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:55:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:55:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2024-04-17T09:55:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXmx9mbe1CIRAA1pmzAl04tGkTu6QLpmCkag==", 2024-04-17T09:55:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:55:58.582 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:55:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2024-04-17T09:55:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDamx9mIwXoKxAAWPTFaYdp4lIZhuOU7F0x8g==", 2024-04-17T09:55:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:55:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:55:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2024-04-17T09:55:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDgmx9mp0s/JxAAv+usQqwrxgKINRlkDhhydQ==", 2024-04-17T09:55:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:55:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:55:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2024-04-17T09:55:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDjmx9mMVStMhAA/n8jk+LVDosWhziXk/lLRQ==", 2024-04-17T09:55:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:55:58.584 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:55:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2024-04-17T09:55:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDnmx9m/YQlABAAOqfWzVtgonD7b+WlR95mnA==", 2024-04-17T09:55:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:55:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:55:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:58.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2024-04-17T09:55:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDqmx9m572mCBAAqBib/yeQOBno+0B1K1dLxA==", 2024-04-17T09:55:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:55:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:55:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2024-04-17T09:55:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mUXoLKxAANV7zhLvL0DqHngk4TtsdBQ==", 2024-04-17T09:55:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:58.586 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:55:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2024-04-17T09:55:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-17T09:55:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2024-04-17T09:55:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mZBeRNBAAQg5K950mp6+XND7NCABAKQ==", 2024-04-17T09:55:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:55:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-17T09:55:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-17T09:55:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-17T09:55:58.587 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2024-04-17T09:55:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mWKstMRAAp18Hbnk02w+hT0sYZ3tSLA==", 2024-04-17T09:55:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:55:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-17T09:55:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-17T09:55:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-17T09:55:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:58.588 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2024-04-17T09:55:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mRQ2wOhAAzlWXFtP0WaXWmneqxnZ5/w==", 2024-04-17T09:55:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:55:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-17T09:55:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-17T09:55:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-17T09:55:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2024-04-17T09:55:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDOmx9maDStMxAAMCa36HfycwV3WbLg6Voo9Q==", 2024-04-17T09:55:58.589 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-17T09:55:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:55:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:55:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2024-04-17T09:55:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9myy0oDxAAPD2BTjIP09hiU5L7i6gUMg==", 2024-04-17T09:55:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-17T09:55:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-17T09:55:58.590 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:55:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2024-04-17T09:55:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9m3FmACxAApri5AI5PeR0+/HLev8DKcw==", 2024-04-17T09:55:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:55:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-17T09:55:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-17T09:55:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:55:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:58.591 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-17T09:55:58.591 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:55:58.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.576+0000 7f1e71a00640 1 -- 172.21.15.32:0/1837048517 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1e48068380 msgr2=0x7f1e4806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:58.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.576+0000 7f1e71a00640 1 --2- 172.21.15.32:0/1837048517 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1e48068380 0x7f1e4806a840 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c000f70 tx=0x7f1e5c007df0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.576+0000 7f1e71a00640 1 -- 172.21.15.32:0/1837048517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6c0ce0c0 msgr2=0x7f1e6c0cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:58.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.576+0000 7f1e71a00640 1 --2- 172.21.15.32:0/1837048517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6c0ce0c0 0x7f1e6c0cbf40 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1e540ae040 tx=0x7f1e54019200 comp rx=0 tx=0).stop 2024-04-17T09:55:58.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.576+0000 7f1e71a00640 1 -- 172.21.15.32:0/1837048517 shutdown_connections 2024-04-17T09:55:58.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.576+0000 7f1e71a00640 1 --2- 172.21.15.32:0/1837048517 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1e48068380 0x7f1e4806a840 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.576+0000 7f1e71a00640 1 --2- 172.21.15.32:0/1837048517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e6c1604f0 0x7f1e6c152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.576+0000 7f1e71a00640 1 --2- 172.21.15.32:0/1837048517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e6c0ce0c0 0x7f1e6c0cbf40 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.576+0000 7f1e71a00640 1 --2- 172.21.15.32:0/1837048517 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e6c09ba90 0x7f1e6c0ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.576+0000 7f1e71a00640 1 -- 172.21.15.32:0/1837048517 >> 172.21.15.32:0/1837048517 conn(0x7f1e6c0b4010 msgr2=0x7f1e6c0cd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:58.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.580+0000 7f1e71a00640 1 -- 172.21.15.32:0/1837048517 shutdown_connections 2024-04-17T09:55:58.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.580+0000 7f1e71a00640 1 -- 172.21.15.32:0/1837048517 wait complete. 2024-04-17T09:55:58.599 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:55:58.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.736+0000 7f8b17200640 1 Processor -- start 2024-04-17T09:55:58.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.736+0000 7f8b17200640 1 -- start start 2024-04-17T09:55:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b17200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8b100cea30 0x7f8b100cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b17200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b10073600 0x7f8b1015f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b17200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b1006cfa0 con 0x7f8b100cd190 2024-04-17T09:55:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b17200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f8b100b4dd0 con 0x7f8b100cea30 2024-04-17T09:55:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b17200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8b100b5970 con 0x7f8b10073600 2024-04-17T09:55:58.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b15200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8b100cd190 0x7f8b1015fd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43398/0 (socket says 172.21.15.32:43398) 2024-04-17T09:55:58.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b15200640 1 -- 172.21.15.32:0/3920361585 learned_addr learned my addr 172.21.15.32:0/3920361585 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:58.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b0fe00640 1 --2- 172.21.15.32:0/3920361585 >> v2:172.21.15.144:3300/0 conn(0x7f8b100cea30 0x7f8b100cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:58.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b07e00640 1 --2- 172.21.15.32:0/3920361585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b10073600 0x7f8b1015f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:58.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b0f400640 1 -- 172.21.15.32:0/3920361585 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3204203977 0 0) 0x7f8b1006cfa0 con 0x7f8b100cd190 2024-04-17T09:55:58.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b0f400640 1 -- 172.21.15.32:0/3920361585 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8af4003530 con 0x7f8b100cd190 2024-04-17T09:55:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b0f400640 1 -- 172.21.15.32:0/3920361585 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2415308237 0 0) 0x7f8af4003530 con 0x7f8b100cd190 2024-04-17T09:55:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b0f400640 1 -- 172.21.15.32:0/3920361585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b10073600 msgr2=0x7f8b1015f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b0f400640 1 --2- 172.21.15.32:0/3920361585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b10073600 0x7f8b1015f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b0f400640 1 -- 172.21.15.32:0/3920361585 >> v2:172.21.15.144:3300/0 conn(0x7f8b100cea30 msgr2=0x7f8b100cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b0f400640 1 --2- 172.21.15.32:0/3920361585 >> v2:172.21.15.144:3300/0 conn(0x7f8b100cea30 0x7f8b100cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b0f400640 1 -- 172.21.15.32:0/3920361585 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1006d210 con 0x7f8b100cd190 2024-04-17T09:55:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b0f400640 1 -- 172.21.15.32:0/3920361585 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f8b08001a00 con 0x7f8b100cd190 2024-04-17T09:55:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b0fe00640 1 --2- 172.21.15.32:0/3920361585 >> v2:172.21.15.144:3300/0 conn(0x7f8b100cea30 0x7f8b100cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:55:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b0f400640 1 -- 172.21.15.32:0/3920361585 >> v1:172.21.15.32:6789/0 conn(0x7f8b100cd190 legacy=0x7f8b1015fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b0f400640 1 --2- 172.21.15.32:0/3920361585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b10073600 0x7f8af40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b0f400640 1 --2- 172.21.15.32:0/3920361585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af4003f40 0x7f8af4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b0f400640 1 --2- 172.21.15.32:0/3920361585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8af4004d50 0x7f8af40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b0f400640 1 -- 172.21.15.32:0/3920361585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8b100b5970 con 0x7f8af4003f40 2024-04-17T09:55:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b0f400640 1 -- 172.21.15.32:0/3920361585 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8af400b9d0 con 0x7f8af4004d50 2024-04-17T09:55:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b0f400640 1 -- 172.21.15.32:0/3920361585 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8af400bb80 con 0x7f8b10073600 2024-04-17T09:55:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b07e00640 1 --2- 172.21.15.32:0/3920361585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af4003f40 0x7f8af4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b0fe00640 1 --2- 172.21.15.32:0/3920361585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b10073600 0x7f8af40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b15200640 1 --2- 172.21.15.32:0/3920361585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8af4004d50 0x7f8af40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b07e00640 1 -- 172.21.15.32:0/3920361585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b10073600 msgr2=0x7f8af40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b07e00640 1 --2- 172.21.15.32:0/3920361585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b10073600 0x7f8af40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b07e00640 1 -- 172.21.15.32:0/3920361585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8af4004d50 msgr2=0x7f8af40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b07e00640 1 --2- 172.21.15.32:0/3920361585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8af4004d50 0x7f8af40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b07e00640 1 -- 172.21.15.32:0/3920361585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af4003530 con 0x7f8af4003f40 2024-04-17T09:55:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.740+0000 7f8b15200640 1 --2- 172.21.15.32:0/3920361585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8af4004d50 0x7f8af40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.744+0000 7f8b07e00640 1 --2- 172.21.15.32:0/3920361585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af4003f40 0x7f8af4004810 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8af8009ae0 tx=0x7f8af801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9eed0d374246be server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.744+0000 7f8b0f400640 1 -- 172.21.15.32:0/3920361585 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af80b2070 con 0x7f8af4003f40 2024-04-17T09:55:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.744+0000 7f8b0f400640 1 -- 172.21.15.32:0/3920361585 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af80a5660 con 0x7f8af4003f40 2024-04-17T09:55:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.744+0000 7f8b0f400640 1 -- 172.21.15.32:0/3920361585 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af80096b0 con 0x7f8af4003f40 2024-04-17T09:55:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.744+0000 7f8b17200640 1 -- 172.21.15.32:0/3920361585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af4003f40 msgr2=0x7f8af4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.744+0000 7f8b17200640 1 --2- 172.21.15.32:0/3920361585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af4003f40 0x7f8af4004810 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8af8009ae0 tx=0x7f8af801ca70 comp rx=0 tx=0).stop 2024-04-17T09:55:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.744+0000 7f8b15200640 1 -- 172.21.15.32:0/3920361585 reap_dead start 2024-04-17T09:55:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.744+0000 7f8b17200640 1 -- 172.21.15.32:0/3920361585 shutdown_connections 2024-04-17T09:55:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.744+0000 7f8b17200640 1 -- 172.21.15.32:0/3920361585 >> 172.21.15.32:0/3920361585 conn(0x7f8b100d0c00 msgr2=0x7f8b100d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.744+0000 7f8b17200640 1 -- 172.21.15.32:0/3920361585 shutdown_connections 2024-04-17T09:55:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.744+0000 7f8b17200640 1 -- 172.21.15.32:0/3920361585 wait complete. 2024-04-17T09:55:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b17200640 1 Processor -- start 2024-04-17T09:55:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b17200640 1 -- start start 2024-04-17T09:55:58.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b17200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af4003f40 0x7f8b101511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b17200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b10152650 0x7f8b10148930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b17200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b1014a900 0x7f8b1006bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b17200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8b100cf6c0 con 0x7f8af4003f40 2024-04-17T09:55:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b17200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8b100cf350 con 0x7f8b1014a900 2024-04-17T09:55:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b17200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8b100cf840 con 0x7f8b10152650 2024-04-17T09:55:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b0fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af4003f40 0x7f8b101511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b0fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af4003f40 0x7f8b101511b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37412/0 (socket says 172.21.15.32:37412) 2024-04-17T09:55:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b0fe00640 1 -- 172.21.15.32:0/2621811910 learned_addr learned my addr 172.21.15.32:0/2621811910 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b15200640 1 --2- 172.21.15.32:0/2621811910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b1014a900 0x7f8b1006bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b07e00640 1 --2- 172.21.15.32:0/2621811910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b10152650 0x7f8b10148930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b0fe00640 1 -- 172.21.15.32:0/2621811910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b10152650 msgr2=0x7f8b10148930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b0fe00640 1 --2- 172.21.15.32:0/2621811910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b10152650 0x7f8b10148930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b0fe00640 1 -- 172.21.15.32:0/2621811910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b1014a900 msgr2=0x7f8b1006bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b0fe00640 1 --2- 172.21.15.32:0/2621811910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b1014a900 0x7f8b1006bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b0fe00640 1 -- 172.21.15.32:0/2621811910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b10151e60 con 0x7f8af4003f40 2024-04-17T09:55:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b15200640 1 --2- 172.21.15.32:0/2621811910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b1014a900 0x7f8b1006bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b0fe00640 1 --2- 172.21.15.32:0/2621811910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af4003f40 0x7f8b101511b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8b00011d00 tx=0x7f8b0000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b0cc00640 1 -- 172.21.15.32:0/2621811910 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b00004020 con 0x7f8af4003f40 2024-04-17T09:55:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b0cc00640 1 -- 172.21.15.32:0/2621811910 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b000041c0 con 0x7f8af4003f40 2024-04-17T09:55:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b0cc00640 1 -- 172.21.15.32:0/2621811910 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b00003620 con 0x7f8af4003f40 2024-04-17T09:55:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b17200640 1 -- 172.21.15.32:0/2621811910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b10152140 con 0x7f8af4003f40 2024-04-17T09:55:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b17200640 1 -- 172.21.15.32:0/2621811910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b100cb6e0 con 0x7f8af4003f40 2024-04-17T09:55:58.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.748+0000 7f8b17200640 1 -- 172.21.15.32:0/2621811910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b100cb1f0 con 0x7f8af4003f40 2024-04-17T09:55:58.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.752+0000 7f8b0cc00640 1 -- 172.21.15.32:0/2621811910 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8b00012040 con 0x7f8af4003f40 2024-04-17T09:55:58.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.752+0000 7f8b0cc00640 1 --2- 172.21.15.32:0/2621811910 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8aec0683b0 0x7f8aec06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:58.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.752+0000 7f8b0cc00640 1 -- 172.21.15.32:0/2621811910 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4857+0+0 (secure 0 0 0) 0x7f8b00086df0 con 0x7f8af4003f40 2024-04-17T09:55:58.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.752+0000 7f8b07e00640 1 --2- 172.21.15.32:0/2621811910 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8aec0683b0 0x7f8aec06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:58.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.752+0000 7f8b07e00640 1 --2- 172.21.15.32:0/2621811910 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8aec0683b0 0x7f8aec06a870 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8af80af000 tx=0x7f8af80046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:58.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.756+0000 7f8b0cc00640 1 -- 172.21.15.32:0/2621811910 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f8b00019030 con 0x7f8af4003f40 2024-04-17T09:55:58.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.920+0000 7f8b17200640 1 -- 172.21.15.32:0/2621811910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b10068a90 con 0x7f8af4003f40 2024-04-17T09:55:58.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.920+0000 7f8b0cc00640 1 -- 172.21.15.32:0/2621811910 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+12903 (secure 0 0 0) 0x7f8b00057660 con 0x7f8af4003f40 2024-04-17T09:55:58.923 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:58.923 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":53,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:55:58.178713+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:55:58.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.924+0000 7f8b17200640 1 -- 172.21.15.32:0/2621811910 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8aec0683b0 msgr2=0x7f8aec06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:58.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.924+0000 7f8b17200640 1 --2- 172.21.15.32:0/2621811910 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8aec0683b0 0x7f8aec06a870 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8af80af000 tx=0x7f8af80046d0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.924+0000 7f8b17200640 1 -- 172.21.15.32:0/2621811910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af4003f40 msgr2=0x7f8b101511b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:58.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.924+0000 7f8b17200640 1 --2- 172.21.15.32:0/2621811910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af4003f40 0x7f8b101511b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8b00011d00 tx=0x7f8b0000f610 comp rx=0 tx=0).stop 2024-04-17T09:55:58.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.924+0000 7f8b17200640 1 -- 172.21.15.32:0/2621811910 shutdown_connections 2024-04-17T09:55:58.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.924+0000 7f8b17200640 1 --2- 172.21.15.32:0/2621811910 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8aec0683b0 0x7f8aec06a870 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.924+0000 7f8b17200640 1 --2- 172.21.15.32:0/2621811910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b10152650 0x7f8b10148930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.924+0000 7f8b17200640 1 --2- 172.21.15.32:0/2621811910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b1014a900 0x7f8b1006bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.924+0000 7f8b17200640 1 --2- 172.21.15.32:0/2621811910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af4003f40 0x7f8b101511b0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:58.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.924+0000 7f8b17200640 1 -- 172.21.15.32:0/2621811910 >> 172.21.15.32:0/2621811910 conn(0x7f8b100d0c00 msgr2=0x7f8b100b3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:58.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.924+0000 7f8b17200640 1 -- 172.21.15.32:0/2621811910 shutdown_connections 2024-04-17T09:55:58.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:58.924+0000 7f8b17200640 1 -- 172.21.15.32:0/2621811910 wait complete. 2024-04-17T09:55:58.945 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:55:58.945 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:55:59.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.088+0000 7f0f00e00640 1 Processor -- start 2024-04-17T09:55:59.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.088+0000 7f0f00e00640 1 -- start start 2024-04-17T09:55:59.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.088+0000 7f0f00e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0efc15ac30 0x7f0efc06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:59.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.088+0000 7f0f00e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0efc15b010 0x7f0efc06dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:59.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.088+0000 7f0f00e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0efc06fcd0 con 0x7f0efc0b3d80 2024-04-17T09:55:59.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.088+0000 7f0f00e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0efc06f300 con 0x7f0efc15b010 2024-04-17T09:55:59.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.088+0000 7f0f00e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0efc06e6f0 con 0x7f0efc15ac30 2024-04-17T09:55:59.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.088+0000 7f0ef9400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0efc15b010 0x7f0efc06dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:59.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.088+0000 7f0efa800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0efc0b3d80 0x7f0efc071e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43408/0 (socket says 172.21.15.32:43408) 2024-04-17T09:55:59.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.088+0000 7f0efa800640 1 -- 172.21.15.32:0/2789442436 learned_addr learned my addr 172.21.15.32:0/2789442436 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:59.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.088+0000 7f0ef9e00640 1 --2- 172.21.15.32:0/2789442436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0efc15ac30 0x7f0efc06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:59.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.088+0000 7f0ef9400640 1 -- 172.21.15.32:0/2789442436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0efc15ac30 msgr2=0x7f0efc06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:59.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.088+0000 7f0ef9400640 1 --2- 172.21.15.32:0/2789442436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0efc15ac30 0x7f0efc06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.088+0000 7f0ef9400640 1 -- 172.21.15.32:0/2789442436 >> v1:172.21.15.32:6789/0 conn(0x7f0efc0b3d80 legacy=0x7f0efc071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:59.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.088+0000 7f0ef9400640 1 -- 172.21.15.32:0/2789442436 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0efc06e9d0 con 0x7f0efc15b010 2024-04-17T09:55:59.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.092+0000 7f0ef9400640 1 --2- 172.21.15.32:0/2789442436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0efc15b010 0x7f0efc06dea0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80097c0 tx=0x7f0ee801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533bba821fb5cb8d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:59.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.092+0000 7f0ef8a00640 1 -- 172.21.15.32:0/2789442436 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee80aa600 con 0x7f0efc15b010 2024-04-17T09:55:59.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.092+0000 7f0ef8a00640 1 -- 172.21.15.32:0/2789442436 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ee80af070 con 0x7f0efc15b010 2024-04-17T09:55:59.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.092+0000 7f0ef8a00640 1 -- 172.21.15.32:0/2789442436 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee80aa9c0 con 0x7f0efc15b010 2024-04-17T09:55:59.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.092+0000 7f0f00e00640 1 -- 172.21.15.32:0/2789442436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0efc15b010 msgr2=0x7f0efc06dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:59.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.092+0000 7f0f00e00640 1 --2- 172.21.15.32:0/2789442436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0efc15b010 0x7f0efc06dea0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80097c0 tx=0x7f0ee801c4a0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.092+0000 7f0f00e00640 1 -- 172.21.15.32:0/2789442436 shutdown_connections 2024-04-17T09:55:59.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.092+0000 7f0f00e00640 1 --2- 172.21.15.32:0/2789442436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0efc15b010 0x7f0efc06dea0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.092+0000 7f0f00e00640 1 --2- 172.21.15.32:0/2789442436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0efc15ac30 0x7f0efc06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.092+0000 7f0f00e00640 1 -- 172.21.15.32:0/2789442436 >> 172.21.15.32:0/2789442436 conn(0x7f0efc0b2120 msgr2=0x7f0efc0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:59.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.092+0000 7f0f00e00640 1 -- 172.21.15.32:0/2789442436 shutdown_connections 2024-04-17T09:55:59.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.092+0000 7f0f00e00640 1 -- 172.21.15.32:0/2789442436 wait complete. 2024-04-17T09:55:59.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.092+0000 7f0f00e00640 1 Processor -- start 2024-04-17T09:55:59.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.092+0000 7f0f00e00640 1 -- start start 2024-04-17T09:55:59.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.096+0000 7f0f00e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0efc0b3d80 0x7f0efc0b1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:59.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.096+0000 7f0f00e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0efc15ac30 0x7f0efc0cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:59.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.096+0000 7f0f00e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0efc15b010 0x7f0efc0cf740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:59.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.096+0000 7f0f00e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc0726f0 con 0x7f0efc15ac30 2024-04-17T09:55:59.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.096+0000 7f0f00e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc072570 con 0x7f0efc15b010 2024-04-17T09:55:59.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.096+0000 7f0f00e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0efc0cd9c0 con 0x7f0efc0b3d80 2024-04-17T09:55:59.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.096+0000 7f0ef9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0efc15ac30 0x7f0efc0cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:59.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.096+0000 7f0ef9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0efc15ac30 0x7f0efc0cf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37434/0 (socket says 172.21.15.32:37434) 2024-04-17T09:55:59.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.096+0000 7f0ef9400640 1 -- 172.21.15.32:0/186976492 learned_addr learned my addr 172.21.15.32:0/186976492 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:59.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.096+0000 7f0ef9400640 1 -- 172.21.15.32:0/186976492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0efc0b3d80 msgr2=0x7f0efc0b1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:59.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.096+0000 7f0ef9400640 1 --2- 172.21.15.32:0/186976492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0efc0b3d80 0x7f0efc0b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.096+0000 7f0ef9400640 1 -- 172.21.15.32:0/186976492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0efc15b010 msgr2=0x7f0efc0cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:59.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.096+0000 7f0ef9400640 1 --2- 172.21.15.32:0/186976492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0efc15b010 0x7f0efc0cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.096+0000 7f0ef9400640 1 -- 172.21.15.32:0/186976492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0efc0cede0 con 0x7f0efc15ac30 2024-04-17T09:55:59.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.096+0000 7f0ef9400640 1 --2- 172.21.15.32:0/186976492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0efc15ac30 0x7f0efc0cf200 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80ac040 tx=0x7f0ee801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:59.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.096+0000 7f0ef2000640 1 -- 172.21.15.32:0/186976492 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee80be610 con 0x7f0efc15ac30 2024-04-17T09:55:59.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.096+0000 7f0ef2000640 1 -- 172.21.15.32:0/186976492 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ee80be7d0 con 0x7f0efc15ac30 2024-04-17T09:55:59.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.096+0000 7f0f00e00640 1 -- 172.21.15.32:0/186976492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0efc0cfc80 con 0x7f0efc15ac30 2024-04-17T09:55:59.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.096+0000 7f0ef2000640 1 -- 172.21.15.32:0/186976492 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee80bebc0 con 0x7f0efc15ac30 2024-04-17T09:55:59.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.096+0000 7f0f00e00640 1 -- 172.21.15.32:0/186976492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0efc0ce020 con 0x7f0efc15ac30 2024-04-17T09:55:59.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.096+0000 7f0f00e00640 1 -- 172.21.15.32:0/186976492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0efc0cead0 con 0x7f0efc15ac30 2024-04-17T09:55:59.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.096+0000 7f0ef2000640 1 -- 172.21.15.32:0/186976492 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0ee80bc020 con 0x7f0efc15ac30 2024-04-17T09:55:59.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.100+0000 7f0ef2000640 1 --2- 172.21.15.32:0/186976492 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0edc068380 0x7f0edc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:59.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.100+0000 7f0ef2000640 1 -- 172.21.15.32:0/186976492 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4857+0+0 (secure 0 0 0) 0x7f0ee8120de0 con 0x7f0efc15ac30 2024-04-17T09:55:59.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.100+0000 7f0ef9e00640 1 --2- 172.21.15.32:0/186976492 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0edc068380 0x7f0edc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:59.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.100+0000 7f0ef9e00640 1 --2- 172.21.15.32:0/186976492 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0edc068380 0x7f0edc06a840 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0eec0098f0 tx=0x7f0eec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:59.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.104+0000 7f0ef2000640 1 -- 172.21.15.32:0/186976492 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0efc0cede0 con 0x7f0efc15ac30 2024-04-17T09:55:59.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.268+0000 7f0f00e00640 1 -- 172.21.15.32:0/186976492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0efc068a90 con 0x7f0efc15ac30 2024-04-17T09:55:59.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.268+0000 7f0ef2000640 1 -- 172.21.15.32:0/186976492 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) ==== 93+0+10 (secure 0 0 0) 0x7f0ee8016090 con 0x7f0efc15ac30 2024-04-17T09:55:59.272 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:55:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.272+0000 7f0f00e00640 1 -- 172.21.15.32:0/186976492 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0edc068380 msgr2=0x7f0edc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.272+0000 7f0f00e00640 1 --2- 172.21.15.32:0/186976492 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0edc068380 0x7f0edc06a840 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0eec0098f0 tx=0x7f0eec008040 comp rx=0 tx=0).stop 2024-04-17T09:55:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.272+0000 7f0f00e00640 1 -- 172.21.15.32:0/186976492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0efc15ac30 msgr2=0x7f0efc0cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.272+0000 7f0f00e00640 1 --2- 172.21.15.32:0/186976492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0efc15ac30 0x7f0efc0cf200 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80ac040 tx=0x7f0ee801ace0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.272+0000 7f0f00e00640 1 -- 172.21.15.32:0/186976492 shutdown_connections 2024-04-17T09:55:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.272+0000 7f0f00e00640 1 --2- 172.21.15.32:0/186976492 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0edc068380 0x7f0edc06a840 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.272+0000 7f0f00e00640 1 --2- 172.21.15.32:0/186976492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0efc15b010 0x7f0efc0cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.272+0000 7f0f00e00640 1 --2- 172.21.15.32:0/186976492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0efc15ac30 0x7f0efc0cf200 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.272+0000 7f0f00e00640 1 --2- 172.21.15.32:0/186976492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0efc0b3d80 0x7f0efc0b1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.272+0000 7f0f00e00640 1 -- 172.21.15.32:0/186976492 >> 172.21.15.32:0/186976492 conn(0x7f0efc0b2120 msgr2=0x7f0efc15b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:59.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.272+0000 7f0f00e00640 1 -- 172.21.15.32:0/186976492 shutdown_connections 2024-04-17T09:55:59.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.272+0000 7f0f00e00640 1 -- 172.21.15.32:0/186976492 wait complete. 2024-04-17T09:55:59.292 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T09:55:59.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7af4e00640 1 Processor -- start 2024-04-17T09:55:59.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7af4e00640 1 -- start start 2024-04-17T09:55:59.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7af4e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7af015cb30 0x7f7af0160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:59.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7af4e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7af00cd100 con 0x7f7af00b2880 2024-04-17T09:55:59.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7af4e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7af00b3c40 con 0x7f7af015c750 2024-04-17T09:55:59.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7af4e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7af0161220 con 0x7f7af015cb30 2024-04-17T09:55:59.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7ae7e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7af00b2880 0x7f7af00cd5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43420/0 (socket says 172.21.15.32:43420) 2024-04-17T09:55:59.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7ae7e00640 1 -- 172.21.15.32:0/3131033594 learned_addr learned my addr 172.21.15.32:0/3131033594 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:55:59.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7aee800640 1 --2- 172.21.15.32:0/3131033594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7af015cb30 0x7f7af0160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:59.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7aed400640 1 -- 172.21.15.32:0/3131033594 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3837514774 0 0) 0x7f7af00cd100 con 0x7f7af00b2880 2024-04-17T09:55:59.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7aed400640 1 -- 172.21.15.32:0/3131033594 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ad4003530 con 0x7f7af00b2880 2024-04-17T09:55:59.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7aed400640 1 -- 172.21.15.32:0/3131033594 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2552967034 0 0) 0x7f7ad4003530 con 0x7f7af00b2880 2024-04-17T09:55:59.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7aed400640 1 -- 172.21.15.32:0/3131033594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7af015cb30 msgr2=0x7f7af0160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:59.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7aed400640 1 --2- 172.21.15.32:0/3131033594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7af015cb30 0x7f7af0160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7aed400640 1 -- 172.21.15.32:0/3131033594 >> v1:172.21.15.144:6789/0 conn(0x7f7af015c750 legacy=0x7f7af00b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:59.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7aed400640 1 -- 172.21.15.32:0/3131033594 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af015d1a0 con 0x7f7af00b2880 2024-04-17T09:55:59.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7aed400640 1 -- 172.21.15.32:0/3131033594 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f7ad8001680 con 0x7f7af00b2880 2024-04-17T09:55:59.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7aed400640 1 -- 172.21.15.32:0/3131033594 >> v1:172.21.15.32:6789/0 conn(0x7f7af00b2880 legacy=0x7f7af00cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:59.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7aed400640 1 --2- 172.21.15.32:0/3131033594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ad40039f0 0x7f7ad4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:59.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7aed400640 1 --2- 172.21.15.32:0/3131033594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7af015cb30 0x7f7ad4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:59.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7aed400640 1 --2- 172.21.15.32:0/3131033594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ad4004d50 0x7f7ad40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:59.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7aed400640 1 -- 172.21.15.32:0/3131033594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7af0161220 con 0x7f7ad40039f0 2024-04-17T09:55:59.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7aed400640 1 -- 172.21.15.32:0/3131033594 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad400ba80 con 0x7f7ad4004d50 2024-04-17T09:55:59.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7aed400640 1 -- 172.21.15.32:0/3131033594 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7ad400bc80 con 0x7f7af015cb30 2024-04-17T09:55:59.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.432+0000 7f7aede00640 1 --2- 172.21.15.32:0/3131033594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ad40039f0 0x7f7ad4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:59.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.436+0000 7f7ae7e00640 1 --2- 172.21.15.32:0/3131033594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7af015cb30 0x7f7ad4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:59.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.436+0000 7f7aee800640 1 --2- 172.21.15.32:0/3131033594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ad4004d50 0x7f7ad40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:59.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.436+0000 7f7aede00640 1 -- 172.21.15.32:0/3131033594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7af015cb30 msgr2=0x7f7ad4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:59.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.436+0000 7f7aede00640 1 --2- 172.21.15.32:0/3131033594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7af015cb30 0x7f7ad4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.436+0000 7f7aede00640 1 -- 172.21.15.32:0/3131033594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ad4004d50 msgr2=0x7f7ad40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:59.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.436+0000 7f7aede00640 1 --2- 172.21.15.32:0/3131033594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ad4004d50 0x7f7ad40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.436+0000 7f7aede00640 1 -- 172.21.15.32:0/3131033594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad4003530 con 0x7f7ad40039f0 2024-04-17T09:55:59.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.436+0000 7f7aede00640 1 --2- 172.21.15.32:0/3131033594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ad40039f0 0x7f7ad4003e10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0001750 tx=0x7f7ae001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128b91ba47eae416 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:59.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.436+0000 7f7aed400640 1 -- 172.21.15.32:0/3131033594 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae00013a0 con 0x7f7ad40039f0 2024-04-17T09:55:59.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.436+0000 7f7aed400640 1 -- 172.21.15.32:0/3131033594 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae00052f0 con 0x7f7ad40039f0 2024-04-17T09:55:59.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.436+0000 7f7aed400640 1 -- 172.21.15.32:0/3131033594 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0003840 con 0x7f7ad40039f0 2024-04-17T09:55:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.436+0000 7f7af4e00640 1 -- 172.21.15.32:0/3131033594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ad40039f0 msgr2=0x7f7ad4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.436+0000 7f7af4e00640 1 --2- 172.21.15.32:0/3131033594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ad40039f0 0x7f7ad4003e10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0001750 tx=0x7f7ae001c3e0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.436+0000 7f7aee800640 1 -- 172.21.15.32:0/3131033594 reap_dead start 2024-04-17T09:55:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.436+0000 7f7af4e00640 1 -- 172.21.15.32:0/3131033594 shutdown_connections 2024-04-17T09:55:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.436+0000 7f7af4e00640 1 -- 172.21.15.32:0/3131033594 >> 172.21.15.32:0/3131033594 conn(0x7f7af009a9f0 msgr2=0x7f7af00b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.440+0000 7f7af4e00640 1 -- 172.21.15.32:0/3131033594 shutdown_connections 2024-04-17T09:55:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.440+0000 7f7af4e00640 1 -- 172.21.15.32:0/3131033594 wait complete. 2024-04-17T09:55:59.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.440+0000 7f7af4e00640 1 Processor -- start 2024-04-17T09:55:59.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.440+0000 7f7af4e00640 1 -- start start 2024-04-17T09:55:59.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.440+0000 7f7af4e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ad40039f0 0x7f7af0154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:59.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.440+0000 7f7af4e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7af01549e0 0x7f7af015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:59.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.440+0000 7f7af4e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af00c85a0 0x7f7af015ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:59.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.440+0000 7f7af4e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7af00ce640 con 0x7f7af00c85a0 2024-04-17T09:55:59.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.440+0000 7f7af4e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7af00ce4c0 con 0x7f7af01549e0 2024-04-17T09:55:59.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.440+0000 7f7af4e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7af00ce7c0 con 0x7f7ad40039f0 2024-04-17T09:55:59.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.440+0000 7f7aee800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af00c85a0 0x7f7af015ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:59.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.440+0000 7f7aee800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af00c85a0 0x7f7af015ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37462/0 (socket says 172.21.15.32:37462) 2024-04-17T09:55:59.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.440+0000 7f7aee800640 1 -- 172.21.15.32:0/2858358477 learned_addr learned my addr 172.21.15.32:0/2858358477 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:59.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.440+0000 7f7aede00640 1 --2- 172.21.15.32:0/2858358477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ad40039f0 0x7f7af0154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:59.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.440+0000 7f7ae7e00640 1 --2- 172.21.15.32:0/2858358477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7af01549e0 0x7f7af015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:59.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.440+0000 7f7aee800640 1 -- 172.21.15.32:0/2858358477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ad40039f0 msgr2=0x7f7af0154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:59.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.440+0000 7f7aee800640 1 --2- 172.21.15.32:0/2858358477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ad40039f0 0x7f7af0154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.440+0000 7f7aee800640 1 -- 172.21.15.32:0/2858358477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7af01549e0 msgr2=0x7f7af015a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:59.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.440+0000 7f7aee800640 1 --2- 172.21.15.32:0/2858358477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7af01549e0 0x7f7af015a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.440+0000 7f7aee800640 1 -- 172.21.15.32:0/2858358477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af015a9f0 con 0x7f7af00c85a0 2024-04-17T09:55:59.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.440+0000 7f7ae7e00640 1 --2- 172.21.15.32:0/2858358477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7af01549e0 0x7f7af015a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:55:59.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.440+0000 7f7aee800640 1 --2- 172.21.15.32:0/2858358477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af00c85a0 0x7f7af015ede0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7af00cc890 tx=0x7f7ae800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:59.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.444+0000 7f7ae6000640 1 -- 172.21.15.32:0/2858358477 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae800fe10 con 0x7f7af00c85a0 2024-04-17T09:55:59.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.444+0000 7f7ae6000640 1 -- 172.21.15.32:0/2858358477 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae8015430 con 0x7f7af00c85a0 2024-04-17T09:55:59.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.444+0000 7f7ae6000640 1 -- 172.21.15.32:0/2858358477 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8015820 con 0x7f7af00c85a0 2024-04-17T09:55:59.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.444+0000 7f7af4e00640 1 -- 172.21.15.32:0/2858358477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af00c8b60 con 0x7f7af00c85a0 2024-04-17T09:55:59.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.444+0000 7f7af4e00640 1 -- 172.21.15.32:0/2858358477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af00c9120 con 0x7f7af00c85a0 2024-04-17T09:55:59.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.444+0000 7f7af4e00640 1 -- 172.21.15.32:0/2858358477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7af0068a90 con 0x7f7af00c85a0 2024-04-17T09:55:59.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.444+0000 7f7ae6000640 1 -- 172.21.15.32:0/2858358477 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7ae80159c0 con 0x7f7af00c85a0 2024-04-17T09:55:59.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.444+0000 7f7ae6000640 1 --2- 172.21.15.32:0/2858358477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7acc068380 0x7f7acc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:59.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.444+0000 7f7ae6000640 1 -- 172.21.15.32:0/2858358477 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4857+0+0 (secure 0 0 0) 0x7f7ae8086ff0 con 0x7f7af00c85a0 2024-04-17T09:55:59.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.444+0000 7f7aede00640 1 --2- 172.21.15.32:0/2858358477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7acc068380 0x7f7acc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:59.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.448+0000 7f7aede00640 1 --2- 172.21.15.32:0/2858358477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7acc068380 0x7f7acc06a840 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40057f0 tx=0x7f7ae00011a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:59.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.448+0000 7f7ae6000640 1 -- 172.21.15.32:0/2858358477 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f7ae8024030 con 0x7f7af00c85a0 2024-04-17T09:55:59.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.616+0000 7f7af4e00640 1 -- 172.21.15.32:0/2858358477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7af0073b10 con 0x7f7af00c85a0 2024-04-17T09:55:59.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.616+0000 7f7ae6000640 1 -- 172.21.15.32:0/2858358477 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v53) ==== 84+0+65 (secure 0 0 0) 0x7f7ae8019030 con 0x7f7af00c85a0 2024-04-17T09:55:59.617 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:55:59.618 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-17T09:55:59.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:55:59.618 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-04-17T09:55:59.618 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-04-17T09:55:59.618 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:55:59.618 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-17T09:55:59.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.616+0000 7f7af4e00640 1 -- 172.21.15.32:0/2858358477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7acc068380 msgr2=0x7f7acc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:59.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.616+0000 7f7af4e00640 1 --2- 172.21.15.32:0/2858358477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7acc068380 0x7f7acc06a840 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40057f0 tx=0x7f7ae00011a0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.616+0000 7f7af4e00640 1 -- 172.21.15.32:0/2858358477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af00c85a0 msgr2=0x7f7af015ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:59.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.616+0000 7f7af4e00640 1 --2- 172.21.15.32:0/2858358477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af00c85a0 0x7f7af015ede0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7af00cc890 tx=0x7f7ae800f580 comp rx=0 tx=0).stop 2024-04-17T09:55:59.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.616+0000 7f7af4e00640 1 -- 172.21.15.32:0/2858358477 shutdown_connections 2024-04-17T09:55:59.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.616+0000 7f7af4e00640 1 --2- 172.21.15.32:0/2858358477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7acc068380 0x7f7acc06a840 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.616+0000 7f7af4e00640 1 --2- 172.21.15.32:0/2858358477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7af00c85a0 0x7f7af015ede0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.616+0000 7f7af4e00640 1 --2- 172.21.15.32:0/2858358477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7af01549e0 0x7f7af015a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.616+0000 7f7af4e00640 1 --2- 172.21.15.32:0/2858358477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ad40039f0 0x7f7af0154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.616+0000 7f7af4e00640 1 -- 172.21.15.32:0/2858358477 >> 172.21.15.32:0/2858358477 conn(0x7f7af009a9f0 msgr2=0x7f7af00b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:59.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.616+0000 7f7af4e00640 1 -- 172.21.15.32:0/2858358477 shutdown_connections 2024-04-17T09:55:59.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.616+0000 7f7af4e00640 1 -- 172.21.15.32:0/2858358477 wait complete. 2024-04-17T09:55:59.638 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T09:55:59.639 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T09:55:59.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.780+0000 7ff9e3c00640 1 Processor -- start 2024-04-17T09:55:59.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.780+0000 7ff9e3c00640 1 -- start start 2024-04-17T09:55:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.780+0000 7ff9e3c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9dc1460a0 0x7ff9dc0ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.784+0000 7ff9e3c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9dc0724c0 0x7ff9dc15e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.784+0000 7ff9e3c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9dc06eeb0 con 0x7ff9dc06e6f0 2024-04-17T09:55:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.784+0000 7ff9e3c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff9dc15de00 con 0x7ff9dc1460a0 2024-04-17T09:55:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.784+0000 7ff9e3c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff9dc06f190 con 0x7ff9dc0724c0 2024-04-17T09:55:59.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.784+0000 7ff9e1200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9dc1460a0 0x7ff9dc0ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.784+0000 7ff9e1200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9dc1460a0 0x7ff9dc0ceb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37466/0 (socket says 172.21.15.32:37466) 2024-04-17T09:55:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.784+0000 7ff9e1200640 1 -- 172.21.15.32:0/3393616064 learned_addr learned my addr 172.21.15.32:0/3393616064 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.784+0000 7ff9e1c00640 1 --2- 172.21.15.32:0/3393616064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9dc0724c0 0x7ff9dc15e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.784+0000 7ff9e1200640 1 -- 172.21.15.32:0/3393616064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9dc0724c0 msgr2=0x7ff9dc15e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:55:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.784+0000 7ff9e1200640 1 --2- 172.21.15.32:0/3393616064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9dc0724c0 0x7ff9dc15e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.784+0000 7ff9e1200640 1 -- 172.21.15.32:0/3393616064 >> v1:172.21.15.144:6789/0 conn(0x7ff9dc06e6f0 legacy=0x7ff9dc071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.784+0000 7ff9e1200640 1 -- 172.21.15.32:0/3393616064 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc06f470 con 0x7ff9dc1460a0 2024-04-17T09:55:59.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.784+0000 7ff9e1c00640 1 --2- 172.21.15.32:0/3393616064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9dc0724c0 0x7ff9dc15e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:55:59.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.784+0000 7ff9e1200640 1 --2- 172.21.15.32:0/3393616064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9dc1460a0 0x7ff9dc0ceb40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009870 tx=0x7ff9cc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80954f3612b3d70c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:59.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.784+0000 7ff9dbe00640 1 -- 172.21.15.32:0/3393616064 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0b2070 con 0x7ff9dc1460a0 2024-04-17T09:55:59.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.784+0000 7ff9dbe00640 1 -- 172.21.15.32:0/3393616064 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9cc0a9070 con 0x7ff9dc1460a0 2024-04-17T09:55:59.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.784+0000 7ff9dbe00640 1 -- 172.21.15.32:0/3393616064 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0ad880 con 0x7ff9dc1460a0 2024-04-17T09:55:59.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.788+0000 7ff9e3c00640 1 -- 172.21.15.32:0/3393616064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9dc1460a0 msgr2=0x7ff9dc0ceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:59.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.788+0000 7ff9e3c00640 1 --2- 172.21.15.32:0/3393616064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9dc1460a0 0x7ff9dc0ceb40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009870 tx=0x7ff9cc01c560 comp rx=0 tx=0).stop 2024-04-17T09:55:59.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.788+0000 7ff9e3c00640 1 -- 172.21.15.32:0/3393616064 shutdown_connections 2024-04-17T09:55:59.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.788+0000 7ff9e3c00640 1 --2- 172.21.15.32:0/3393616064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9dc0724c0 0x7ff9dc15e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.788+0000 7ff9e3c00640 1 --2- 172.21.15.32:0/3393616064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9dc1460a0 0x7ff9dc0ceb40 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.788+0000 7ff9e3c00640 1 -- 172.21.15.32:0/3393616064 >> 172.21.15.32:0/3393616064 conn(0x7ff9dc0b2190 msgr2=0x7ff9dc0b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:55:59.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.788+0000 7ff9e3c00640 1 -- 172.21.15.32:0/3393616064 shutdown_connections 2024-04-17T09:55:59.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.788+0000 7ff9e3c00640 1 -- 172.21.15.32:0/3393616064 wait complete. 2024-04-17T09:55:59.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.788+0000 7ff9e3c00640 1 Processor -- start 2024-04-17T09:55:59.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.788+0000 7ff9e3c00640 1 -- start start 2024-04-17T09:55:59.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.788+0000 7ff9e3c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9dc06e6f0 0x7ff9dc139bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:59.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.788+0000 7ff9e3c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9dc0724c0 0x7ff9dc158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:59.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.788+0000 7ff9e3c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9dc1460a0 0x7ff9dc15a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:59.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.788+0000 7ff9e3c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc15f630 con 0x7ff9dc1460a0 2024-04-17T09:55:59.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.788+0000 7ff9e3c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc15f4b0 con 0x7ff9dc06e6f0 2024-04-17T09:55:59.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.788+0000 7ff9e3c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff9dc15f7b0 con 0x7ff9dc0724c0 2024-04-17T09:55:59.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.788+0000 7ff9e1c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9dc1460a0 0x7ff9dc15a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:59.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.788+0000 7ff9e1c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9dc1460a0 0x7ff9dc15a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37476/0 (socket says 172.21.15.32:37476) 2024-04-17T09:55:59.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.788+0000 7ff9e1c00640 1 -- 172.21.15.32:0/2910431656 learned_addr learned my addr 172.21.15.32:0/2910431656 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:55:59.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.788+0000 7ff9d3e00640 1 --2- 172.21.15.32:0/2910431656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9dc0724c0 0x7ff9dc158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:59.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.788+0000 7ff9e1200640 1 --2- 172.21.15.32:0/2910431656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9dc06e6f0 0x7ff9dc139bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:59.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.792+0000 7ff9e1c00640 1 -- 172.21.15.32:0/2910431656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9dc0724c0 msgr2=0x7ff9dc158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:59.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.792+0000 7ff9e1c00640 1 --2- 172.21.15.32:0/2910431656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9dc0724c0 0x7ff9dc158f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.792+0000 7ff9e1c00640 1 -- 172.21.15.32:0/2910431656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9dc06e6f0 msgr2=0x7ff9dc139bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:55:59.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.792+0000 7ff9e1c00640 1 --2- 172.21.15.32:0/2910431656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9dc06e6f0 0x7ff9dc139bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:55:59.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.792+0000 7ff9e1c00640 1 -- 172.21.15.32:0/2910431656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc159870 con 0x7ff9dc1460a0 2024-04-17T09:55:59.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.792+0000 7ff9e1c00640 1 --2- 172.21.15.32:0/2910431656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9dc1460a0 0x7ff9dc15a9f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009fa0 tx=0x7ff9d400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:59.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.792+0000 7ff9d9600640 1 -- 172.21.15.32:0/2910431656 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d401a070 con 0x7ff9dc1460a0 2024-04-17T09:55:59.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.792+0000 7ff9d9600640 1 -- 172.21.15.32:0/2910431656 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d4003bc0 con 0x7ff9dc1460a0 2024-04-17T09:55:59.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.792+0000 7ff9d9600640 1 -- 172.21.15.32:0/2910431656 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4004030 con 0x7ff9dc1460a0 2024-04-17T09:55:59.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.792+0000 7ff9e3c00640 1 -- 172.21.15.32:0/2910431656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9dc159b50 con 0x7ff9dc1460a0 2024-04-17T09:55:59.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.792+0000 7ff9e3c00640 1 -- 172.21.15.32:0/2910431656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9dc15a110 con 0x7ff9dc1460a0 2024-04-17T09:55:59.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.792+0000 7ff9e3c00640 1 -- 172.21.15.32:0/2910431656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9dc068a90 con 0x7ff9dc1460a0 2024-04-17T09:55:59.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.792+0000 7ff9d9600640 1 -- 172.21.15.32:0/2910431656 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff9d401d070 con 0x7ff9dc1460a0 2024-04-17T09:55:59.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.792+0000 7ff9d9600640 1 --2- 172.21.15.32:0/2910431656 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff9b80683b0 0x7ff9b806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:55:59.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.792+0000 7ff9d9600640 1 -- 172.21.15.32:0/2910431656 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4857+0+0 (secure 0 0 0) 0x7ff9d40868b0 con 0x7ff9dc1460a0 2024-04-17T09:55:59.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.792+0000 7ff9e1200640 1 --2- 172.21.15.32:0/2910431656 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff9b80683b0 0x7ff9b806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:55:59.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.796+0000 7ff9e1200640 1 --2- 172.21.15.32:0/2910431656 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff9b80683b0 0x7ff9b806a870 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc0af000 tx=0x7ff9cc0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:55:59.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.796+0000 7ff9d9600640 1 -- 172.21.15.32:0/2910431656 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff9d4010070 con 0x7ff9dc1460a0 2024-04-17T09:55:59.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:55:59.964+0000 7ff9e3c00640 1 -- 172.21.15.32:0/2910431656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff9dc06cd90 con 0x7ff9dc1460a0 2024-04-17T09:56:00.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.172+0000 7ff9d9600640 1 -- 172.21.15.32:0/2910431656 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v54) ==== 138+0+0 (secure 0 0 0) 0x7ff9d4056b60 con 0x7ff9dc1460a0 2024-04-17T09:56:00.173 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-04-17T09:56:00.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.172+0000 7ff9e3c00640 1 -- 172.21.15.32:0/2910431656 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff9b80683b0 msgr2=0x7ff9b806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:00.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.172+0000 7ff9e3c00640 1 --2- 172.21.15.32:0/2910431656 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff9b80683b0 0x7ff9b806a870 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc0af000 tx=0x7ff9cc0a7000 comp rx=0 tx=0).stop 2024-04-17T09:56:00.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.172+0000 7ff9e3c00640 1 -- 172.21.15.32:0/2910431656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9dc1460a0 msgr2=0x7ff9dc15a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:00.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.172+0000 7ff9e3c00640 1 --2- 172.21.15.32:0/2910431656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9dc1460a0 0x7ff9dc15a9f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009fa0 tx=0x7ff9d400d560 comp rx=0 tx=0).stop 2024-04-17T09:56:00.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.172+0000 7ff9e3c00640 1 -- 172.21.15.32:0/2910431656 shutdown_connections 2024-04-17T09:56:00.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.172+0000 7ff9e3c00640 1 --2- 172.21.15.32:0/2910431656 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff9b80683b0 0x7ff9b806a870 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:00.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.172+0000 7ff9e3c00640 1 --2- 172.21.15.32:0/2910431656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9dc0724c0 0x7ff9dc158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:00.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.172+0000 7ff9e3c00640 1 --2- 172.21.15.32:0/2910431656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9dc1460a0 0x7ff9dc15a9f0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:00.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.172+0000 7ff9e3c00640 1 --2- 172.21.15.32:0/2910431656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9dc06e6f0 0x7ff9dc139bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:00.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.172+0000 7ff9e3c00640 1 -- 172.21.15.32:0/2910431656 >> 172.21.15.32:0/2910431656 conn(0x7ff9dc0b2190 msgr2=0x7ff9dc06dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:00.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.176+0000 7ff9e3c00640 1 -- 172.21.15.32:0/2910431656 shutdown_connections 2024-04-17T09:56:00.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.176+0000 7ff9e3c00640 1 -- 172.21.15.32:0/2910431656 wait complete. 2024-04-17T09:56:00.200 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-17T09:56:00.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.348+0000 7f28e5400640 1 Processor -- start 2024-04-17T09:56:00.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.348+0000 7f28e5400640 1 -- start start 2024-04-17T09:56:00.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.348+0000 7f28e5400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28e00b59e0 0x7f28e00afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:00.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.348+0000 7f28e5400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28e00b0540 con 0x7f28e0141700 2024-04-17T09:56:00.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.348+0000 7f28e5400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28e00b3ef0 con 0x7f28e006d150 2024-04-17T09:56:00.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.348+0000 7f28e5400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f28e00b0820 con 0x7f28e00b59e0 2024-04-17T09:56:00.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.348+0000 7f28df200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28e00b59e0 0x7f28e00afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:00.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.348+0000 7f28de800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f28e0141700 0x7f28e006ca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58400/0 (socket says 172.21.15.32:58400) 2024-04-17T09:56:00.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.348+0000 7f28de800640 1 -- 172.21.15.32:0/1391350179 learned_addr learned my addr 172.21.15.32:0/1391350179 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:00.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.348+0000 7f28df200640 1 -- 172.21.15.32:0/1391350179 >> v1:172.21.15.144:6789/0 conn(0x7f28e006d150 legacy=0x7f28e00af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:00.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.348+0000 7f28df200640 1 -- 172.21.15.32:0/1391350179 >> v1:172.21.15.32:6789/0 conn(0x7f28e0141700 legacy=0x7f28e006ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:00.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.348+0000 7f28df200640 1 -- 172.21.15.32:0/1391350179 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e00b41d0 con 0x7f28e00b59e0 2024-04-17T09:56:00.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.352+0000 7f28df200640 1 --2- 172.21.15.32:0/1391350179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28e00b59e0 0x7f28e00afeb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f28d8009d90 tx=0x7f28d801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596079d4343c5734 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:00.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.352+0000 7f28dde00640 1 -- 172.21.15.32:0/1391350179 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d80b2070 con 0x7f28e00b59e0 2024-04-17T09:56:00.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.352+0000 7f28dde00640 1 -- 172.21.15.32:0/1391350179 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28d80a5940 con 0x7f28e00b59e0 2024-04-17T09:56:00.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.352+0000 7f28dde00640 1 -- 172.21.15.32:0/1391350179 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d80a5d10 con 0x7f28e00b59e0 2024-04-17T09:56:00.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.352+0000 7f28e5400640 1 -- 172.21.15.32:0/1391350179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28e00b59e0 msgr2=0x7f28e00afeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:00.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.352+0000 7f28e5400640 1 --2- 172.21.15.32:0/1391350179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28e00b59e0 0x7f28e00afeb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f28d8009d90 tx=0x7f28d801c970 comp rx=0 tx=0).stop 2024-04-17T09:56:00.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.352+0000 7f28e5400640 1 -- 172.21.15.32:0/1391350179 shutdown_connections 2024-04-17T09:56:00.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.352+0000 7f28e5400640 1 --2- 172.21.15.32:0/1391350179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28e00b59e0 0x7f28e00afeb0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:00.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.352+0000 7f28e5400640 1 -- 172.21.15.32:0/1391350179 >> 172.21.15.32:0/1391350179 conn(0x7f28e0072670 msgr2=0x7f28e0072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:00.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.352+0000 7f28e5400640 1 -- 172.21.15.32:0/1391350179 shutdown_connections 2024-04-17T09:56:00.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.352+0000 7f28e5400640 1 -- 172.21.15.32:0/1391350179 wait complete. 2024-04-17T09:56:00.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28e5400640 1 Processor -- start 2024-04-17T09:56:00.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28e5400640 1 -- start start 2024-04-17T09:56:00.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28e5400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28e006d150 0x7f28e01573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:00.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28e5400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28e00b59e0 0x7f28e0158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:00.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28e5400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28e0141700 0x7f28e0158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:00.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28e5400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f28e016c260 con 0x7f28e0141700 2024-04-17T09:56:00.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28e5400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f28e016c0e0 con 0x7f28e006d150 2024-04-17T09:56:00.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28e5400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f28e016c3e0 con 0x7f28e00b59e0 2024-04-17T09:56:00.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28df200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28e0141700 0x7f28e0158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:00.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28de800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28e006d150 0x7f28e01573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:00.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28df200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28e0141700 0x7f28e0158ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49148/0 (socket says 172.21.15.32:49148) 2024-04-17T09:56:00.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28df200640 1 -- 172.21.15.32:0/2051560546 learned_addr learned my addr 172.21.15.32:0/2051560546 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:00.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28d5e00640 1 --2- 172.21.15.32:0/2051560546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28e00b59e0 0x7f28e0158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:00.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28df200640 1 -- 172.21.15.32:0/2051560546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28e00b59e0 msgr2=0x7f28e0158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:00.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28df200640 1 --2- 172.21.15.32:0/2051560546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28e00b59e0 0x7f28e0158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:00.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28df200640 1 -- 172.21.15.32:0/2051560546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28e006d150 msgr2=0x7f28e01573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:00.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28df200640 1 --2- 172.21.15.32:0/2051560546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28e006d150 0x7f28e01573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:00.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28df200640 1 -- 172.21.15.32:0/2051560546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e0159310 con 0x7f28e0141700 2024-04-17T09:56:00.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28df200640 1 --2- 172.21.15.32:0/2051560546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28e0141700 0x7f28e0158ba0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f28d80b0040 tx=0x7f28d8019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:00.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28d7400640 1 -- 172.21.15.32:0/2051560546 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d80b2050 con 0x7f28e0141700 2024-04-17T09:56:00.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28d7400640 1 -- 172.21.15.32:0/2051560546 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28d80be070 con 0x7f28e0141700 2024-04-17T09:56:00.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28d7400640 1 -- 172.21.15.32:0/2051560546 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d80adb50 con 0x7f28e0141700 2024-04-17T09:56:00.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28e5400640 1 -- 172.21.15.32:0/2051560546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e01595f0 con 0x7f28e0141700 2024-04-17T09:56:00.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28e5400640 1 -- 172.21.15.32:0/2051560546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e0139a20 con 0x7f28e0141700 2024-04-17T09:56:00.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.356+0000 7f28e5400640 1 -- 172.21.15.32:0/2051560546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28e0068a90 con 0x7f28e0141700 2024-04-17T09:56:00.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.360+0000 7f28d7400640 1 -- 172.21.15.32:0/2051560546 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f28d80adcf0 con 0x7f28e0141700 2024-04-17T09:56:00.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.360+0000 7f28d7400640 1 --2- 172.21.15.32:0/2051560546 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f28bc068380 0x7f28bc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:00.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.360+0000 7f28d7400640 1 -- 172.21.15.32:0/2051560546 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5227+0+0 (secure 0 0 0) 0x7f28d8122550 con 0x7f28e0141700 2024-04-17T09:56:00.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.360+0000 7f28de800640 1 --2- 172.21.15.32:0/2051560546 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f28bc068380 0x7f28bc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:00.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.360+0000 7f28de800640 1 --2- 172.21.15.32:0/2051560546 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f28bc068380 0x7f28bc06a840 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f28d0005890 tx=0x7f28d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:00.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.364+0000 7f28d7400640 1 -- 172.21.15.32:0/2051560546 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f28d8016560 con 0x7f28e0141700 2024-04-17T09:56:00.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:00.528+0000 7f28e5400640 1 -- 172.21.15.32:0/2051560546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f28e0073930 con 0x7f28e0141700 2024-04-17T09:56:01.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.176+0000 7f28d7400640 1 -- 172.21.15.32:0/2051560546 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v55) ==== 170+0+0 (secure 0 0 0) 0x7f28d80f2680 con 0x7f28e0141700 2024-04-17T09:56:01.176 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-04-17T09:56:01.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.176+0000 7f28e5400640 1 -- 172.21.15.32:0/2051560546 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f28bc068380 msgr2=0x7f28bc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:01.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.176+0000 7f28e5400640 1 --2- 172.21.15.32:0/2051560546 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f28bc068380 0x7f28bc06a840 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f28d0005890 tx=0x7f28d0009040 comp rx=0 tx=0).stop 2024-04-17T09:56:01.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.176+0000 7f28e5400640 1 -- 172.21.15.32:0/2051560546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28e0141700 msgr2=0x7f28e0158ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:01.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.176+0000 7f28e5400640 1 --2- 172.21.15.32:0/2051560546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28e0141700 0x7f28e0158ba0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f28d80b0040 tx=0x7f28d8019710 comp rx=0 tx=0).stop 2024-04-17T09:56:01.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.176+0000 7f28e5400640 1 -- 172.21.15.32:0/2051560546 shutdown_connections 2024-04-17T09:56:01.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.176+0000 7f28e5400640 1 --2- 172.21.15.32:0/2051560546 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f28bc068380 0x7f28bc06a840 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:01.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.176+0000 7f28e5400640 1 --2- 172.21.15.32:0/2051560546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28e00b59e0 0x7f28e0158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:01.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.176+0000 7f28e5400640 1 --2- 172.21.15.32:0/2051560546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28e0141700 0x7f28e0158ba0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:01.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.176+0000 7f28e5400640 1 --2- 172.21.15.32:0/2051560546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28e006d150 0x7f28e01573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:01.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.176+0000 7f28e5400640 1 -- 172.21.15.32:0/2051560546 >> 172.21.15.32:0/2051560546 conn(0x7f28e0072670 msgr2=0x7f28e00b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:01.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.176+0000 7f28e5400640 1 -- 172.21.15.32:0/2051560546 shutdown_connections 2024-04-17T09:56:01.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.176+0000 7f28e5400640 1 -- 172.21.15.32:0/2051560546 wait complete. 2024-04-17T09:56:01.214 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T09:56:01.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.356+0000 7fd197e00640 1 Processor -- start 2024-04-17T09:56:01.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.356+0000 7fd197e00640 1 -- start start 2024-04-17T09:56:01.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.356+0000 7fd197e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd19815e7f0 0x7fd1980d1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:01.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.356+0000 7fd197e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd19806d080 0x7fd19806d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:01.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.356+0000 7fd197e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1980ce920 con 0x7fd19806da90 2024-04-17T09:56:01.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.356+0000 7fd197e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd1980b52b0 con 0x7fd19806d080 2024-04-17T09:56:01.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.356+0000 7fd197e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd1980d1aa0 con 0x7fd19815e7f0 2024-04-17T09:56:01.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.356+0000 7fd18fe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd19806d080 0x7fd19806d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:01.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.356+0000 7fd18fe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd19806d080 0x7fd19806d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49162/0 (socket says 172.21.15.32:49162) 2024-04-17T09:56:01.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.356+0000 7fd18fe00640 1 -- 172.21.15.32:0/3080013169 learned_addr learned my addr 172.21.15.32:0/3080013169 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:01.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.360+0000 7fd196a00640 1 --2- 172.21.15.32:0/3080013169 >> v2:172.21.15.144:3300/0 conn(0x7fd19815e7f0 0x7fd1980d1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:01.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.360+0000 7fd18fe00640 1 -- 172.21.15.32:0/3080013169 >> v2:172.21.15.144:3300/0 conn(0x7fd19815e7f0 msgr2=0x7fd1980d1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:01.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.360+0000 7fd18fe00640 1 --2- 172.21.15.32:0/3080013169 >> v2:172.21.15.144:3300/0 conn(0x7fd19815e7f0 0x7fd1980d1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:01.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.360+0000 7fd18fe00640 1 -- 172.21.15.32:0/3080013169 >> v1:172.21.15.144:6789/0 conn(0x7fd19806da90 legacy=0x7fd19809a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:01.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.360+0000 7fd18fe00640 1 -- 172.21.15.32:0/3080013169 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1980ccb50 con 0x7fd19806d080 2024-04-17T09:56:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.360+0000 7fd18fe00640 1 --2- 172.21.15.32:0/3080013169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19806d080 0x7fd19806d550 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd198073220 tx=0x7fd18001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca771c7f7a06db86 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.360+0000 7fd196000640 1 -- 172.21.15.32:0/3080013169 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1800b1070 con 0x7fd19806d080 2024-04-17T09:56:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.360+0000 7fd196000640 1 -- 172.21.15.32:0/3080013169 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1800ac630 con 0x7fd19806d080 2024-04-17T09:56:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.360+0000 7fd196000640 1 -- 172.21.15.32:0/3080013169 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1800a8050 con 0x7fd19806d080 2024-04-17T09:56:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.360+0000 7fd197e00640 1 -- 172.21.15.32:0/3080013169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19806d080 msgr2=0x7fd19806d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:01.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.360+0000 7fd197e00640 1 --2- 172.21.15.32:0/3080013169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19806d080 0x7fd19806d550 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd198073220 tx=0x7fd18001c4b0 comp rx=0 tx=0).stop 2024-04-17T09:56:01.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.360+0000 7fd197e00640 1 -- 172.21.15.32:0/3080013169 shutdown_connections 2024-04-17T09:56:01.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.360+0000 7fd197e00640 1 --2- 172.21.15.32:0/3080013169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19806d080 0x7fd19806d550 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:01.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.360+0000 7fd197e00640 1 --2- 172.21.15.32:0/3080013169 >> v2:172.21.15.144:3300/0 conn(0x7fd19815e7f0 0x7fd1980d1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:01.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.360+0000 7fd197e00640 1 -- 172.21.15.32:0/3080013169 >> 172.21.15.32:0/3080013169 conn(0x7fd1980b3f50 msgr2=0x7fd1980b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:01.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.360+0000 7fd197e00640 1 -- 172.21.15.32:0/3080013169 shutdown_connections 2024-04-17T09:56:01.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.360+0000 7fd197e00640 1 -- 172.21.15.32:0/3080013169 wait complete. 2024-04-17T09:56:01.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.364+0000 7fd197e00640 1 Processor -- start 2024-04-17T09:56:01.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.364+0000 7fd197e00640 1 -- start start 2024-04-17T09:56:01.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.364+0000 7fd197e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19806d080 0x7fd198159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:01.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.364+0000 7fd197e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd19806da90 0x7fd1981534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:01.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.364+0000 7fd197e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19815e7f0 0x7fd1980cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:01.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.364+0000 7fd197e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd1980d16b0 con 0x7fd19815e7f0 2024-04-17T09:56:01.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.364+0000 7fd197e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd19806dea0 con 0x7fd19806da90 2024-04-17T09:56:01.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.364+0000 7fd197e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd1980d1830 con 0x7fd19806d080 2024-04-17T09:56:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.364+0000 7fd196a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19806d080 0x7fd198159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.364+0000 7fd18fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd19806da90 0x7fd1981534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.364+0000 7fd197400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19815e7f0 0x7fd1980cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.364+0000 7fd18fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd19806da90 0x7fd1981534b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:45866/0 (socket says 172.21.15.32:45866) 2024-04-17T09:56:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.364+0000 7fd18fe00640 1 -- 172.21.15.32:0/2944494519 learned_addr learned my addr 172.21.15.32:0/2944494519 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.364+0000 7fd196a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19806d080 0x7fd198159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:58260/0 (socket says 172.21.15.32:58260) 2024-04-17T09:56:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.364+0000 7fd18fe00640 1 -- 172.21.15.32:0/2944494519 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19806d080 msgr2=0x7fd198159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.364+0000 7fd18fe00640 1 --2- 172.21.15.32:0/2944494519 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19806d080 0x7fd198159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.364+0000 7fd18fe00640 1 -- 172.21.15.32:0/2944494519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19815e7f0 msgr2=0x7fd1980cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.364+0000 7fd18fe00640 1 --2- 172.21.15.32:0/2944494519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19815e7f0 0x7fd1980cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.364+0000 7fd18fe00640 1 -- 172.21.15.32:0/2944494519 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19815f610 con 0x7fd19806da90 2024-04-17T09:56:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.364+0000 7fd196a00640 1 --2- 172.21.15.32:0/2944494519 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19806d080 0x7fd198159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:56:01.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.364+0000 7fd18fe00640 1 --2- 172.21.15.32:0/2944494519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd19806da90 0x7fd1981534b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd1800af040 tx=0x7fd180019250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:01.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.368+0000 7fd18ea00640 1 -- 172.21.15.32:0/2944494519 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1800b1070 con 0x7fd19806da90 2024-04-17T09:56:01.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.368+0000 7fd18ea00640 1 -- 172.21.15.32:0/2944494519 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1800ab5e0 con 0x7fd19806da90 2024-04-17T09:56:01.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.368+0000 7fd197e00640 1 -- 172.21.15.32:0/2944494519 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19815f8f0 con 0x7fd19806da90 2024-04-17T09:56:01.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.368+0000 7fd18ea00640 1 -- 172.21.15.32:0/2944494519 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1800a8040 con 0x7fd19806da90 2024-04-17T09:56:01.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.368+0000 7fd197e00640 1 -- 172.21.15.32:0/2944494519 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1981539f0 con 0x7fd19806da90 2024-04-17T09:56:01.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.368+0000 7fd197e00640 1 -- 172.21.15.32:0/2944494519 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd19815fe10 con 0x7fd19806da90 2024-04-17T09:56:01.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.368+0000 7fd18ea00640 1 -- 172.21.15.32:0/2944494519 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd1800be060 con 0x7fd19806da90 2024-04-17T09:56:01.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.368+0000 7fd18ea00640 1 --2- 172.21.15.32:0/2944494519 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd174068380 0x7fd17406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:01.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.368+0000 7fd18ea00640 1 -- 172.21.15.32:0/2944494519 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5609+0+0 (secure 0 0 0) 0x7fd18011ffd0 con 0x7fd19806da90 2024-04-17T09:56:01.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.368+0000 7fd196a00640 1 --2- 172.21.15.32:0/2944494519 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd174068380 0x7fd17406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:01.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.372+0000 7fd196a00640 1 --2- 172.21.15.32:0/2944494519 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd174068380 0x7fd17406a840 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd188007db0 tx=0x7fd188008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:01.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.372+0000 7fd18ea00640 1 -- 172.21.15.32:0/2944494519 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd19815fe10 con 0x7fd19806da90 2024-04-17T09:56:01.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:01.572+0000 7fd197e00640 1 -- 172.21.15.32:0/2944494519 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd19804fda0 con 0x7fd19806da90 2024-04-17T09:56:02.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.196+0000 7fd18ea00640 1 -- 172.21.15.32:0/2944494519 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v29) ==== 172+0+0 (secure 0 0 0) 0x7fd1800160d0 con 0x7fd19806da90 2024-04-17T09:56:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 8 and data pool 9 2024-04-17T09:56:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.204+0000 7fd197e00640 1 -- 172.21.15.32:0/2944494519 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd174068380 msgr2=0x7fd17406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.204+0000 7fd197e00640 1 --2- 172.21.15.32:0/2944494519 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd174068380 0x7fd17406a840 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd188007db0 tx=0x7fd188008040 comp rx=0 tx=0).stop 2024-04-17T09:56:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.204+0000 7fd197e00640 1 -- 172.21.15.32:0/2944494519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd19806da90 msgr2=0x7fd1981534b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:02.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.204+0000 7fd197e00640 1 --2- 172.21.15.32:0/2944494519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd19806da90 0x7fd1981534b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd1800af040 tx=0x7fd180019250 comp rx=0 tx=0).stop 2024-04-17T09:56:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.208+0000 7fd197e00640 1 -- 172.21.15.32:0/2944494519 shutdown_connections 2024-04-17T09:56:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.208+0000 7fd197e00640 1 --2- 172.21.15.32:0/2944494519 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd174068380 0x7fd17406a840 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.208+0000 7fd197e00640 1 --2- 172.21.15.32:0/2944494519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19815e7f0 0x7fd1980cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.208+0000 7fd197e00640 1 --2- 172.21.15.32:0/2944494519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd19806da90 0x7fd1981534b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.208+0000 7fd197e00640 1 --2- 172.21.15.32:0/2944494519 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19806d080 0x7fd198159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.208+0000 7fd197e00640 1 -- 172.21.15.32:0/2944494519 >> 172.21.15.32:0/2944494519 conn(0x7fd1980b3f50 msgr2=0x7fd1980ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:02.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.208+0000 7fd197e00640 1 -- 172.21.15.32:0/2944494519 shutdown_connections 2024-04-17T09:56:02.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.208+0000 7fd197e00640 1 -- 172.21.15.32:0/2944494519 wait complete. 2024-04-17T09:56:02.248 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:56:02.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.388+0000 7fd5f7c00640 1 Processor -- start 2024-04-17T09:56:02.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.388+0000 7fd5f7c00640 1 -- start start 2024-04-17T09:56:02.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.388+0000 7fd5f7c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5f00b2430 0x7fd5f00b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:02.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.388+0000 7fd5f7c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd5f00d0890 0x7fd5f00d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:02.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.388+0000 7fd5f7c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5f006ba30 0x7fd5f006be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:02.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.388+0000 7fd5f7c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd5f00b2d50 con 0x7fd5f00b2430 2024-04-17T09:56:02.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.388+0000 7fd5f7c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd5f0162620 con 0x7fd5f00d0890 2024-04-17T09:56:02.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.388+0000 7fd5f7c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd5f0162900 con 0x7fd5f006ba30 2024-04-17T09:56:02.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.388+0000 7fd5f5200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5f00b2430 0x7fd5f00b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:02.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.388+0000 7fd5f5200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5f00b2430 0x7fd5f00b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49172/0 (socket says 172.21.15.32:49172) 2024-04-17T09:56:02.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.388+0000 7fd5f5200640 1 -- 172.21.15.32:0/3910849787 learned_addr learned my addr 172.21.15.32:0/3910849787 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:02.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.388+0000 7fd5efe00640 1 --2- 172.21.15.32:0/3910849787 >> v2:172.21.15.144:3300/0 conn(0x7fd5f00d0890 0x7fd5f00d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:02.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.388+0000 7fd5f5c00640 1 --2- 172.21.15.32:0/3910849787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5f006ba30 0x7fd5f006be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:02.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.388+0000 7fd5f5200640 1 -- 172.21.15.32:0/3910849787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5f006ba30 msgr2=0x7fd5f006be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:02.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.388+0000 7fd5f5200640 1 --2- 172.21.15.32:0/3910849787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5f006ba30 0x7fd5f006be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.388+0000 7fd5f5200640 1 -- 172.21.15.32:0/3910849787 >> v2:172.21.15.144:3300/0 conn(0x7fd5f00d0890 msgr2=0x7fd5f00d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:02.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.388+0000 7fd5f5200640 1 --2- 172.21.15.32:0/3910849787 >> v2:172.21.15.144:3300/0 conn(0x7fd5f00d0890 0x7fd5f00d0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.388+0000 7fd5f5200640 1 -- 172.21.15.32:0/3910849787 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f0162be0 con 0x7fd5f00b2430 2024-04-17T09:56:02.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.392+0000 7fd5f5200640 1 --2- 172.21.15.32:0/3910849787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f00b2430 0x7fd5f00b2810 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00b1f10 tx=0x7fd5e401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78658a7643366d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:02.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.392+0000 7fd5ef400640 1 -- 172.21.15.32:0/3910849787 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40b0070 con 0x7fd5f00b2430 2024-04-17T09:56:02.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.392+0000 7fd5ef400640 1 -- 172.21.15.32:0/3910849787 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5e40ab600 con 0x7fd5f00b2430 2024-04-17T09:56:02.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.392+0000 7fd5ef400640 1 -- 172.21.15.32:0/3910849787 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40ab9d0 con 0x7fd5f00b2430 2024-04-17T09:56:02.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.392+0000 7fd5f7c00640 1 -- 172.21.15.32:0/3910849787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f00b2430 msgr2=0x7fd5f00b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:02.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.392+0000 7fd5f7c00640 1 --2- 172.21.15.32:0/3910849787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f00b2430 0x7fd5f00b2810 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00b1f10 tx=0x7fd5e401c400 comp rx=0 tx=0).stop 2024-04-17T09:56:02.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.392+0000 7fd5f7c00640 1 -- 172.21.15.32:0/3910849787 shutdown_connections 2024-04-17T09:56:02.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.392+0000 7fd5f7c00640 1 --2- 172.21.15.32:0/3910849787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5f006ba30 0x7fd5f006be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.392+0000 7fd5f7c00640 1 --2- 172.21.15.32:0/3910849787 >> v2:172.21.15.144:3300/0 conn(0x7fd5f00d0890 0x7fd5f00d0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.392+0000 7fd5f7c00640 1 --2- 172.21.15.32:0/3910849787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f00b2430 0x7fd5f00b2810 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.392+0000 7fd5f7c00640 1 -- 172.21.15.32:0/3910849787 >> 172.21.15.32:0/3910849787 conn(0x7fd5f009a9f0 msgr2=0x7fd5f00b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:02.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.392+0000 7fd5f7c00640 1 -- 172.21.15.32:0/3910849787 shutdown_connections 2024-04-17T09:56:02.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.392+0000 7fd5f7c00640 1 -- 172.21.15.32:0/3910849787 wait complete. 2024-04-17T09:56:02.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5f7c00640 1 Processor -- start 2024-04-17T09:56:02.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5f7c00640 1 -- start start 2024-04-17T09:56:02.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5f7c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5f006ba30 0x7fd5f00c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:02.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5f7c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5f00b2430 0x7fd5f00b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:02.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5f7c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f00d0890 0x7fd5f00b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:02.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5f7c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f00ce320 con 0x7fd5f00d0890 2024-04-17T09:56:02.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5f7c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f015fdf0 con 0x7fd5f006ba30 2024-04-17T09:56:02.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5f7c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd5f00ce4a0 con 0x7fd5f00b2430 2024-04-17T09:56:02.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5f5c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f00d0890 0x7fd5f00b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:02.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5f5c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f00d0890 0x7fd5f00b08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49174/0 (socket says 172.21.15.32:49174) 2024-04-17T09:56:02.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5f5c00640 1 -- 172.21.15.32:0/157356477 learned_addr learned my addr 172.21.15.32:0/157356477 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:02.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5f5200640 1 --2- 172.21.15.32:0/157356477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5f006ba30 0x7fd5f00c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:02.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5efe00640 1 --2- 172.21.15.32:0/157356477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5f00b2430 0x7fd5f00b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:02.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5f5c00640 1 -- 172.21.15.32:0/157356477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5f00b2430 msgr2=0x7fd5f00b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:02.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5f5c00640 1 --2- 172.21.15.32:0/157356477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5f00b2430 0x7fd5f00b03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5f5c00640 1 -- 172.21.15.32:0/157356477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5f006ba30 msgr2=0x7fd5f00c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:02.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5f5c00640 1 --2- 172.21.15.32:0/157356477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5f006ba30 0x7fd5f00c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5f5c00640 1 -- 172.21.15.32:0/157356477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f01fbf10 con 0x7fd5f00d0890 2024-04-17T09:56:02.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5efe00640 1 --2- 172.21.15.32:0/157356477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5f00b2430 0x7fd5f00b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:02.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5f5c00640 1 --2- 172.21.15.32:0/157356477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f00d0890 0x7fd5f00b08f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8007e60 tx=0x7fd5e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:02.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5ecc00640 1 -- 172.21.15.32:0/157356477 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8014620 con 0x7fd5f00d0890 2024-04-17T09:56:02.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5ecc00640 1 -- 172.21.15.32:0/157356477 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5e8019070 con 0x7fd5f00d0890 2024-04-17T09:56:02.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5f7c00640 1 -- 172.21.15.32:0/157356477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f01f8960 con 0x7fd5f00d0890 2024-04-17T09:56:02.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5ecc00640 1 -- 172.21.15.32:0/157356477 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8014a20 con 0x7fd5f00d0890 2024-04-17T09:56:02.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.396+0000 7fd5f7c00640 1 -- 172.21.15.32:0/157356477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f01fb430 con 0x7fd5f00d0890 2024-04-17T09:56:02.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.400+0000 7fd5f7c00640 1 -- 172.21.15.32:0/157356477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5f0068a90 con 0x7fd5f00d0890 2024-04-17T09:56:02.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.400+0000 7fd5ecc00640 1 -- 172.21.15.32:0/157356477 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd5e8014bc0 con 0x7fd5f00d0890 2024-04-17T09:56:02.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.400+0000 7fd5ecc00640 1 --2- 172.21.15.32:0/157356477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5d0068380 0x7fd5d006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:02.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.400+0000 7fd5ecc00640 1 -- 172.21.15.32:0/157356477 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fd5e8087ff0 con 0x7fd5f00d0890 2024-04-17T09:56:02.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.400+0000 7fd5f5200640 1 --2- 172.21.15.32:0/157356477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5d0068380 0x7fd5d006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:02.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.404+0000 7fd5f5200640 1 --2- 172.21.15.32:0/157356477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5d0068380 0x7fd5d006a840 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40ad000 tx=0x7fd5e4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:02.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.404+0000 7fd5ecc00640 1 -- 172.21.15.32:0/157356477 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd5e8058620 con 0x7fd5f00d0890 2024-04-17T09:56:02.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.572+0000 7fd5f7c00640 1 -- 172.21.15.32:0/157356477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5f009af00 con 0x7fd5f00d0890 2024-04-17T09:56:02.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.572+0000 7fd5ecc00640 1 -- 172.21.15.32:0/157356477 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16690 (secure 0 0 0) 0x7fd5e8057f20 con 0x7fd5f00d0890 2024-04-17T09:56:02.576 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:02.576 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":56,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:56:01.579225+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:55:59.976232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-17T09:56:00.543788+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:56:02.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.576+0000 7fd5f7c00640 1 -- 172.21.15.32:0/157356477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5d0068380 msgr2=0x7fd5d006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:02.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.576+0000 7fd5f7c00640 1 --2- 172.21.15.32:0/157356477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5d0068380 0x7fd5d006a840 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40ad000 tx=0x7fd5e4008040 comp rx=0 tx=0).stop 2024-04-17T09:56:02.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.576+0000 7fd5f7c00640 1 -- 172.21.15.32:0/157356477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f00d0890 msgr2=0x7fd5f00b08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:02.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.576+0000 7fd5f7c00640 1 --2- 172.21.15.32:0/157356477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f00d0890 0x7fd5f00b08f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8007e60 tx=0x7fd5e800e4b0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.576+0000 7fd5f7c00640 1 -- 172.21.15.32:0/157356477 shutdown_connections 2024-04-17T09:56:02.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.576+0000 7fd5f7c00640 1 --2- 172.21.15.32:0/157356477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd5d0068380 0x7fd5d006a840 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.576+0000 7fd5f7c00640 1 --2- 172.21.15.32:0/157356477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5f00b2430 0x7fd5f00b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.576+0000 7fd5f7c00640 1 --2- 172.21.15.32:0/157356477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f00d0890 0x7fd5f00b08f0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.576+0000 7fd5f7c00640 1 --2- 172.21.15.32:0/157356477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5f006ba30 0x7fd5f00c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.576+0000 7fd5f7c00640 1 -- 172.21.15.32:0/157356477 >> 172.21.15.32:0/157356477 conn(0x7fd5f009a9f0 msgr2=0x7fd5f0161bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:02.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.576+0000 7fd5f7c00640 1 -- 172.21.15.32:0/157356477 shutdown_connections 2024-04-17T09:56:02.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.576+0000 7fd5f7c00640 1 -- 172.21.15.32:0/157356477 wait complete. 2024-04-17T09:56:02.597 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:56:02.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.736+0000 7ff49b200640 1 Processor -- start 2024-04-17T09:56:02.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.736+0000 7ff49b200640 1 -- start start 2024-04-17T09:56:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.736+0000 7ff49b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff49c06fc80 0x7ff49c0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.736+0000 7ff49b200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff49c15b800 0x7ff49c15bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.736+0000 7ff49b200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff49c15c6f0 con 0x7ff49c0af660 2024-04-17T09:56:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.736+0000 7ff49b200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff49c15c1d0 con 0x7ff49c15b800 2024-04-17T09:56:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.736+0000 7ff49b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff49c073a40 con 0x7ff49c06fc80 2024-04-17T09:56:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.736+0000 7ff49a800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff49c15b800 0x7ff49c15bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.736+0000 7ff49a800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff49c15b800 0x7ff49c15bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49176/0 (socket says 172.21.15.32:49176) 2024-04-17T09:56:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.736+0000 7ff49a800640 1 -- 172.21.15.32:0/1740734650 learned_addr learned my addr 172.21.15.32:0/1740734650 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.736+0000 7ff499e00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7ff49c0af660 0x7ff49c06f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:46698/0 (socket says 172.21.15.32:46698) 2024-04-17T09:56:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.736+0000 7ff493e00640 1 --2- 172.21.15.32:0/1740734650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff49c06fc80 0x7ff49c0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.736+0000 7ff49a800640 1 -- 172.21.15.32:0/1740734650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff49c06fc80 msgr2=0x7ff49c0b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.736+0000 7ff49a800640 1 --2- 172.21.15.32:0/1740734650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff49c06fc80 0x7ff49c0b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.736+0000 7ff49a800640 1 -- 172.21.15.32:0/1740734650 >> v1:172.21.15.144:6789/0 conn(0x7ff49c0af660 legacy=0x7ff49c06f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.736+0000 7ff49a800640 1 -- 172.21.15.32:0/1740734650 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49c15c410 con 0x7ff49c15b800 2024-04-17T09:56:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.736+0000 7ff499400640 1 -- 172.21.15.32:0/1740734650 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3594195306 0 0) 0x7ff49c15c6f0 con 0x7ff49c0af660 2024-04-17T09:56:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.740+0000 7ff49a800640 1 --2- 172.21.15.32:0/1740734650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff49c15b800 0x7ff49c15bc90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff49400d150 tx=0x7ff49401ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b8896bbfacc5d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.740+0000 7ff499400640 1 -- 172.21.15.32:0/1740734650 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4940a7cd0 con 0x7ff49c15b800 2024-04-17T09:56:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.740+0000 7ff499400640 1 -- 172.21.15.32:0/1740734650 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff49400b0d0 con 0x7ff49c15b800 2024-04-17T09:56:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.740+0000 7ff499400640 1 -- 172.21.15.32:0/1740734650 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff494009600 con 0x7ff49c15b800 2024-04-17T09:56:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.740+0000 7ff49b200640 1 -- 172.21.15.32:0/1740734650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff49c15b800 msgr2=0x7ff49c15bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.740+0000 7ff49b200640 1 --2- 172.21.15.32:0/1740734650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff49c15b800 0x7ff49c15bc90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff49400d150 tx=0x7ff49401ec00 comp rx=0 tx=0).stop 2024-04-17T09:56:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.740+0000 7ff49b200640 1 -- 172.21.15.32:0/1740734650 shutdown_connections 2024-04-17T09:56:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.740+0000 7ff49b200640 1 --2- 172.21.15.32:0/1740734650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff49c15b800 0x7ff49c15bc90 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.740+0000 7ff49b200640 1 --2- 172.21.15.32:0/1740734650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff49c06fc80 0x7ff49c0b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.740+0000 7ff49b200640 1 -- 172.21.15.32:0/1740734650 >> 172.21.15.32:0/1740734650 conn(0x7ff49c0d0180 msgr2=0x7ff49c0d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.740+0000 7ff49b200640 1 -- 172.21.15.32:0/1740734650 shutdown_connections 2024-04-17T09:56:02.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.740+0000 7ff49b200640 1 -- 172.21.15.32:0/1740734650 wait complete. 2024-04-17T09:56:02.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff49b200640 1 Processor -- start 2024-04-17T09:56:02.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff49b200640 1 -- start start 2024-04-17T09:56:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff49b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff49c06fc80 0x7ff49c14f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff49b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff49c0af660 0x7ff49c099050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff49b200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff49c15b800 0x7ff49c150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff49b200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c0afbc0 con 0x7ff49c0af660 2024-04-17T09:56:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff49b200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c0afa40 con 0x7ff49c15b800 2024-04-17T09:56:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff49b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff49c072430 con 0x7ff49c06fc80 2024-04-17T09:56:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff493e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff49c0af660 0x7ff49c099050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff493e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff49c0af660 0x7ff49c099050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49184/0 (socket says 172.21.15.32:49184) 2024-04-17T09:56:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff493e00640 1 -- 172.21.15.32:0/3249690749 learned_addr learned my addr 172.21.15.32:0/3249690749 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff49a800640 1 --2- 172.21.15.32:0/3249690749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff49c15b800 0x7ff49c150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff493e00640 1 -- 172.21.15.32:0/3249690749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff49c06fc80 msgr2=0x7ff49c14f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff499e00640 1 --2- 172.21.15.32:0/3249690749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff49c06fc80 0x7ff49c14f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff493e00640 1 --2- 172.21.15.32:0/3249690749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff49c06fc80 0x7ff49c14f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff493e00640 1 -- 172.21.15.32:0/3249690749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff49c15b800 msgr2=0x7ff49c150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff493e00640 1 --2- 172.21.15.32:0/3249690749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff49c15b800 0x7ff49c150df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff493e00640 1 -- 172.21.15.32:0/3249690749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49c15fd10 con 0x7ff49c0af660 2024-04-17T09:56:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff49a800640 1 --2- 172.21.15.32:0/3249690749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff49c15b800 0x7ff49c150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff499e00640 1 --2- 172.21.15.32:0/3249690749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff49c06fc80 0x7ff49c14f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff493e00640 1 --2- 172.21.15.32:0/3249690749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff49c0af660 0x7ff49c099050 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff48400e930 tx=0x7ff48400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff492000640 1 -- 172.21.15.32:0/3249690749 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff484017070 con 0x7ff49c0af660 2024-04-17T09:56:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff492000640 1 -- 172.21.15.32:0/3249690749 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48400ca00 con 0x7ff49c0af660 2024-04-17T09:56:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff492000640 1 -- 172.21.15.32:0/3249690749 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48400cdb0 con 0x7ff49c0af660 2024-04-17T09:56:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff49b200640 1 -- 172.21.15.32:0/3249690749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff49c152720 con 0x7ff49c0af660 2024-04-17T09:56:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.744+0000 7ff49b200640 1 -- 172.21.15.32:0/3249690749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff49c152160 con 0x7ff49c0af660 2024-04-17T09:56:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.748+0000 7ff49b200640 1 -- 172.21.15.32:0/3249690749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff49c04fd00 con 0x7ff49c0af660 2024-04-17T09:56:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.748+0000 7ff492000640 1 -- 172.21.15.32:0/3249690749 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff48401a070 con 0x7ff49c0af660 2024-04-17T09:56:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.748+0000 7ff492000640 1 --2- 172.21.15.32:0/3249690749 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff4780683b0 0x7ff47806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.748+0000 7ff492000640 1 -- 172.21.15.32:0/3249690749 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7ff4840869e0 con 0x7ff49c0af660 2024-04-17T09:56:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.748+0000 7ff499e00640 1 --2- 172.21.15.32:0/3249690749 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff4780683b0 0x7ff47806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:02.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.748+0000 7ff499e00640 1 --2- 172.21.15.32:0/3249690749 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff4780683b0 0x7ff47806a870 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff48c000f70 tx=0x7ff48c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.752+0000 7ff492000640 1 -- 172.21.15.32:0/3249690749 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff484057050 con 0x7ff49c0af660 2024-04-17T09:56:02.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.920+0000 7ff49b200640 1 -- 172.21.15.32:0/3249690749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff49c0b4be0 con 0x7ff49c0af660 2024-04-17T09:56:02.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.920+0000 7ff492000640 1 -- 172.21.15.32:0/3249690749 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16690 (secure 0 0 0) 0x7ff484056950 con 0x7ff49c0af660 2024-04-17T09:56:02.922 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:02.923 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":56,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:56:01.579225+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:55:59.976232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-17T09:56:00.543788+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:56:02.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.924+0000 7ff49b200640 1 -- 172.21.15.32:0/3249690749 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff4780683b0 msgr2=0x7ff47806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:02.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.924+0000 7ff49b200640 1 --2- 172.21.15.32:0/3249690749 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff4780683b0 0x7ff47806a870 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff48c000f70 tx=0x7ff48c009040 comp rx=0 tx=0).stop 2024-04-17T09:56:02.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.924+0000 7ff49b200640 1 -- 172.21.15.32:0/3249690749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff49c0af660 msgr2=0x7ff49c099050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:02.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.924+0000 7ff49b200640 1 --2- 172.21.15.32:0/3249690749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff49c0af660 0x7ff49c099050 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff48400e930 tx=0x7ff48400edf0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.924+0000 7ff49b200640 1 -- 172.21.15.32:0/3249690749 shutdown_connections 2024-04-17T09:56:02.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.924+0000 7ff49b200640 1 --2- 172.21.15.32:0/3249690749 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff4780683b0 0x7ff47806a870 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.924+0000 7ff49b200640 1 --2- 172.21.15.32:0/3249690749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff49c15b800 0x7ff49c150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.924+0000 7ff49b200640 1 --2- 172.21.15.32:0/3249690749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff49c0af660 0x7ff49c099050 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.924+0000 7ff49b200640 1 --2- 172.21.15.32:0/3249690749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff49c06fc80 0x7ff49c14f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:02.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.924+0000 7ff49b200640 1 -- 172.21.15.32:0/3249690749 >> 172.21.15.32:0/3249690749 conn(0x7ff49c0d0180 msgr2=0x7ff49c0adce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:02.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.924+0000 7ff49b200640 1 -- 172.21.15.32:0/3249690749 shutdown_connections 2024-04-17T09:56:02.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:02.924+0000 7ff49b200640 1 -- 172.21.15.32:0/3249690749 wait complete. 2024-04-17T09:56:02.946 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T09:56:03.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.088+0000 7fb088800640 1 Processor -- start 2024-04-17T09:56:03.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.088+0000 7fb088800640 1 -- start start 2024-04-17T09:56:03.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.088+0000 7fb088800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb08006ba30 0x7fb08006be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:03.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.088+0000 7fb088800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0800cf370 0x7fb0800cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:03.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.088+0000 7fb088800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb08006c350 con 0x7fb080160350 2024-04-17T09:56:03.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.088+0000 7fb088800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb0800d0130 con 0x7fb0800cf370 2024-04-17T09:56:03.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.088+0000 7fb088800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb08015d090 con 0x7fb08006ba30 2024-04-17T09:56:03.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.088+0000 7fb085400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0800cf370 0x7fb0800cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:03.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.088+0000 7fb085400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0800cf370 0x7fb0800cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49198/0 (socket says 172.21.15.32:49198) 2024-04-17T09:56:03.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.088+0000 7fb085400640 1 -- 172.21.15.32:0/249200296 learned_addr learned my addr 172.21.15.32:0/249200296 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:03.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.088+0000 7fb085e00640 1 --2- 172.21.15.32:0/249200296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb08006ba30 0x7fb08006be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:03.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.088+0000 7fb085400640 1 -- 172.21.15.32:0/249200296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb08006ba30 msgr2=0x7fb08006be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:03.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.088+0000 7fb085400640 1 --2- 172.21.15.32:0/249200296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb08006ba30 0x7fb08006be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:03.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.088+0000 7fb085400640 1 -- 172.21.15.32:0/249200296 >> v1:172.21.15.144:6789/0 conn(0x7fb080160350 legacy=0x7fb0801607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:03.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.088+0000 7fb085400640 1 -- 172.21.15.32:0/249200296 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08015d370 con 0x7fb0800cf370 2024-04-17T09:56:03.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.088+0000 7fb085e00640 1 --2- 172.21.15.32:0/249200296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb08006ba30 0x7fb08006be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:03.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.092+0000 7fb085400640 1 --2- 172.21.15.32:0/249200296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0800cf370 0x7fb0800cf7e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb070009870 tx=0x7fb07001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406d2505797f8b20 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:03.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.092+0000 7fb084a00640 1 -- 172.21.15.32:0/249200296 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0700a7070 con 0x7fb0800cf370 2024-04-17T09:56:03.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.092+0000 7fb084a00640 1 -- 172.21.15.32:0/249200296 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0700b0070 con 0x7fb0800cf370 2024-04-17T09:56:03.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.092+0000 7fb084a00640 1 -- 172.21.15.32:0/249200296 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0700ab890 con 0x7fb0800cf370 2024-04-17T09:56:03.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.092+0000 7fb088800640 1 -- 172.21.15.32:0/249200296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0800cf370 msgr2=0x7fb0800cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:03.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.092+0000 7fb088800640 1 --2- 172.21.15.32:0/249200296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0800cf370 0x7fb0800cf7e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb070009870 tx=0x7fb07001c560 comp rx=0 tx=0).stop 2024-04-17T09:56:03.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.092+0000 7fb088800640 1 -- 172.21.15.32:0/249200296 shutdown_connections 2024-04-17T09:56:03.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.092+0000 7fb088800640 1 --2- 172.21.15.32:0/249200296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0800cf370 0x7fb0800cf7e0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:03.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.092+0000 7fb088800640 1 --2- 172.21.15.32:0/249200296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb08006ba30 0x7fb08006be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:03.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.092+0000 7fb088800640 1 -- 172.21.15.32:0/249200296 >> 172.21.15.32:0/249200296 conn(0x7fb0800b2120 msgr2=0x7fb0800b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:03.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.092+0000 7fb088800640 1 -- 172.21.15.32:0/249200296 shutdown_connections 2024-04-17T09:56:03.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.092+0000 7fb088800640 1 -- 172.21.15.32:0/249200296 wait complete. 2024-04-17T09:56:03.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb088800640 1 Processor -- start 2024-04-17T09:56:03.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb088800640 1 -- start start 2024-04-17T09:56:03.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb088800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb08006ba30 0x7fb08015fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:03.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb088800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0800cf370 0x7fb08015df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:03.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb088800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb080160350 0x7fb0801412d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:03.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb088800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb08015dcf0 con 0x7fb080160350 2024-04-17T09:56:03.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb088800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb08015db70 con 0x7fb08006ba30 2024-04-17T09:56:03.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb088800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb080146230 con 0x7fb0800cf370 2024-04-17T09:56:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb086800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb080160350 0x7fb0801412d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb086800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb080160350 0x7fb0801412d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49204/0 (socket says 172.21.15.32:49204) 2024-04-17T09:56:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb086800640 1 -- 172.21.15.32:0/2420876496 learned_addr learned my addr 172.21.15.32:0/2420876496 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb085400640 1 --2- 172.21.15.32:0/2420876496 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0800cf370 0x7fb08015df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb085e00640 1 --2- 172.21.15.32:0/2420876496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb08006ba30 0x7fb08015fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb086800640 1 -- 172.21.15.32:0/2420876496 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0800cf370 msgr2=0x7fb08015df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb086800640 1 --2- 172.21.15.32:0/2420876496 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0800cf370 0x7fb08015df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb086800640 1 -- 172.21.15.32:0/2420876496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb08006ba30 msgr2=0x7fb08015fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb086800640 1 --2- 172.21.15.32:0/2420876496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb08006ba30 0x7fb08015fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb086800640 1 -- 172.21.15.32:0/2420876496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb080141810 con 0x7fb080160350 2024-04-17T09:56:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb085e00640 1 --2- 172.21.15.32:0/2420876496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb08006ba30 0x7fb08015fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:03.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb086800640 1 --2- 172.21.15.32:0/2420876496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb080160350 0x7fb0801412d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00c730 tx=0x7fb07c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:03.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb07a000640 1 -- 172.21.15.32:0/2420876496 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c003b80 con 0x7fb080160350 2024-04-17T09:56:03.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb07a000640 1 -- 172.21.15.32:0/2420876496 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb07c003d20 con 0x7fb080160350 2024-04-17T09:56:03.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb088800640 1 -- 172.21.15.32:0/2420876496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08015e670 con 0x7fb080160350 2024-04-17T09:56:03.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb07a000640 1 -- 172.21.15.32:0/2420876496 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c00aa30 con 0x7fb080160350 2024-04-17T09:56:03.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb088800640 1 -- 172.21.15.32:0/2420876496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb08015ec30 con 0x7fb080160350 2024-04-17T09:56:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.096+0000 7fb088800640 1 -- 172.21.15.32:0/2420876496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb080068a90 con 0x7fb080160350 2024-04-17T09:56:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.100+0000 7fb07a000640 1 -- 172.21.15.32:0/2420876496 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb07c00abd0 con 0x7fb080160350 2024-04-17T09:56:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.100+0000 7fb07a000640 1 --2- 172.21.15.32:0/2420876496 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb0640683b0 0x7fb06406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.100+0000 7fb07a000640 1 -- 172.21.15.32:0/2420876496 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fb07c086830 con 0x7fb080160350 2024-04-17T09:56:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.100+0000 7fb085e00640 1 --2- 172.21.15.32:0/2420876496 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb0640683b0 0x7fb06406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.100+0000 7fb085e00640 1 --2- 172.21.15.32:0/2420876496 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb0640683b0 0x7fb06406a870 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb080070980 tx=0x7fb074008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:03.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.104+0000 7fb07a000640 1 -- 172.21.15.32:0/2420876496 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fb07c056d50 con 0x7fb080160350 2024-04-17T09:56:03.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:03.308+0000 7fb088800640 1 -- 172.21.15.32:0/2420876496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb08006c850 con 0x7fb080160350 2024-04-17T09:56:04.172 INFO:tasks.ceph.mds.a.smithi032.stderr:2024-04-17T09:56:04.172+0000 7f7c95000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T09:56:04.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.212+0000 7fb07a000640 1 -- 172.21.15.32:0/2420876496 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) ==== 118+0+0 (secure 0 0 0) 0x7fb07c056650 con 0x7fb080160350 2024-04-17T09:56:04.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.216+0000 7fb088800640 1 -- 172.21.15.32:0/2420876496 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb0640683b0 msgr2=0x7fb06406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:04.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.216+0000 7fb088800640 1 --2- 172.21.15.32:0/2420876496 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb0640683b0 0x7fb06406a870 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb080070980 tx=0x7fb074008040 comp rx=0 tx=0).stop 2024-04-17T09:56:04.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.216+0000 7fb088800640 1 -- 172.21.15.32:0/2420876496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb080160350 msgr2=0x7fb0801412d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:04.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.216+0000 7fb088800640 1 --2- 172.21.15.32:0/2420876496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb080160350 0x7fb0801412d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00c730 tx=0x7fb07c00cbf0 comp rx=0 tx=0).stop 2024-04-17T09:56:04.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.216+0000 7fb088800640 1 -- 172.21.15.32:0/2420876496 shutdown_connections 2024-04-17T09:56:04.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.216+0000 7fb088800640 1 --2- 172.21.15.32:0/2420876496 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb0640683b0 0x7fb06406a870 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:04.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.216+0000 7fb088800640 1 --2- 172.21.15.32:0/2420876496 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0800cf370 0x7fb08015df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:04.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.216+0000 7fb088800640 1 --2- 172.21.15.32:0/2420876496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb080160350 0x7fb0801412d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:04.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.216+0000 7fb088800640 1 --2- 172.21.15.32:0/2420876496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb08006ba30 0x7fb08015fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:04.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.216+0000 7fb088800640 1 -- 172.21.15.32:0/2420876496 >> 172.21.15.32:0/2420876496 conn(0x7fb0800b2120 msgr2=0x7fb08006f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:04.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.216+0000 7fb088800640 1 -- 172.21.15.32:0/2420876496 shutdown_connections 2024-04-17T09:56:04.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.216+0000 7fb088800640 1 -- 172.21.15.32:0/2420876496 wait complete. 2024-04-17T09:56:04.242 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-17T09:56:04.242 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-17T09:56:04.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.388+0000 7fa6a5a00640 1 Processor -- start 2024-04-17T09:56:04.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.388+0000 7fa6a5a00640 1 -- start start 2024-04-17T09:56:04.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.392+0000 7fa6a5a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6a006f7c0 0x7fa6a006ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:04.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.392+0000 7fa6a5a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa6a006f270 0x7fa6a006bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:04.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.392+0000 7fa6a5a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6a00cc940 con 0x7fa6a015b7d0 2024-04-17T09:56:04.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.392+0000 7fa6a5a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa6a00ccc20 con 0x7fa6a006f270 2024-04-17T09:56:04.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.392+0000 7fa6a5a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a006e0b0 con 0x7fa6a006f7c0 2024-04-17T09:56:04.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.392+0000 7fa697e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6a006f7c0 0x7fa6a006ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:04.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.392+0000 7fa69ea00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fa6a015b7d0 0x7fa6a006e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:46720/0 (socket says 172.21.15.32:46720) 2024-04-17T09:56:04.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.392+0000 7fa69ea00640 1 -- 172.21.15.32:0/3732666324 learned_addr learned my addr 172.21.15.32:0/3732666324 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:04.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.392+0000 7fa69f400640 1 --2- 172.21.15.32:0/3732666324 >> v2:172.21.15.144:3300/0 conn(0x7fa6a006f270 0x7fa6a006bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:04.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.392+0000 7fa69e000640 1 -- 172.21.15.32:0/3732666324 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1370412485 0 0) 0x7fa6a00cc940 con 0x7fa6a015b7d0 2024-04-17T09:56:04.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.392+0000 7fa697e00640 1 -- 172.21.15.32:0/3732666324 >> v2:172.21.15.144:3300/0 conn(0x7fa6a006f270 msgr2=0x7fa6a006bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:04.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.392+0000 7fa697e00640 1 --2- 172.21.15.32:0/3732666324 >> v2:172.21.15.144:3300/0 conn(0x7fa6a006f270 0x7fa6a006bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:04.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.392+0000 7fa697e00640 1 -- 172.21.15.32:0/3732666324 >> v1:172.21.15.144:6789/0 conn(0x7fa6a015b7d0 legacy=0x7fa6a006e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:04.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.392+0000 7fa697e00640 1 -- 172.21.15.32:0/3732666324 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a006e390 con 0x7fa6a006f7c0 2024-04-17T09:56:04.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.392+0000 7fa69f400640 1 --2- 172.21.15.32:0/3732666324 >> v2:172.21.15.144:3300/0 conn(0x7fa6a006f270 0x7fa6a006bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:56:04.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.396+0000 7fa697e00640 1 --2- 172.21.15.32:0/3732666324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6a006f7c0 0x7fa6a006ba30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa6880097c0 tx=0x7fa68801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=466d3cd7c1b72ba1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:04.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.396+0000 7fa69e000640 1 -- 172.21.15.32:0/3732666324 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6880ab430 con 0x7fa6a006f7c0 2024-04-17T09:56:04.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.396+0000 7fa69e000640 1 -- 172.21.15.32:0/3732666324 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6880b0070 con 0x7fa6a006f7c0 2024-04-17T09:56:04.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.396+0000 7fa69e000640 1 -- 172.21.15.32:0/3732666324 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6880ab7f0 con 0x7fa6a006f7c0 2024-04-17T09:56:04.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.396+0000 7fa6a5a00640 1 -- 172.21.15.32:0/3732666324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6a006f7c0 msgr2=0x7fa6a006ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:04.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.396+0000 7fa6a5a00640 1 --2- 172.21.15.32:0/3732666324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6a006f7c0 0x7fa6a006ba30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa6880097c0 tx=0x7fa68801c410 comp rx=0 tx=0).stop 2024-04-17T09:56:04.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.396+0000 7fa6a5a00640 1 -- 172.21.15.32:0/3732666324 shutdown_connections 2024-04-17T09:56:04.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.396+0000 7fa6a5a00640 1 --2- 172.21.15.32:0/3732666324 >> v2:172.21.15.144:3300/0 conn(0x7fa6a006f270 0x7fa6a006bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:04.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.396+0000 7fa6a5a00640 1 --2- 172.21.15.32:0/3732666324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6a006f7c0 0x7fa6a006ba30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:04.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.396+0000 7fa6a5a00640 1 -- 172.21.15.32:0/3732666324 >> 172.21.15.32:0/3732666324 conn(0x7fa6a00b2120 msgr2=0x7fa6a00b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:04.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.396+0000 7fa6a5a00640 1 -- 172.21.15.32:0/3732666324 shutdown_connections 2024-04-17T09:56:04.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.396+0000 7fa6a5a00640 1 -- 172.21.15.32:0/3732666324 wait complete. 2024-04-17T09:56:04.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.396+0000 7fa6a5a00640 1 Processor -- start 2024-04-17T09:56:04.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.396+0000 7fa6a5a00640 1 -- start start 2024-04-17T09:56:04.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.400+0000 7fa6a5a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6a006f270 0x7fa6a014f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:04.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.400+0000 7fa6a5a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6a006f7c0 0x7fa6a0159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:04.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.400+0000 7fa6a5a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6a015b7d0 0x7fa6a015a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:04.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.400+0000 7fa6a5a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a00ceeb0 con 0x7fa6a006f7c0 2024-04-17T09:56:04.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.400+0000 7fa6a5a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a00ced30 con 0x7fa6a006f270 2024-04-17T09:56:04.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.400+0000 7fa6a5a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a01682a0 con 0x7fa6a015b7d0 2024-04-17T09:56:04.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.400+0000 7fa697e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6a006f7c0 0x7fa6a0159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:04.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.400+0000 7fa697e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6a006f7c0 0x7fa6a0159de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49216/0 (socket says 172.21.15.32:49216) 2024-04-17T09:56:04.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.400+0000 7fa697e00640 1 -- 172.21.15.32:0/2321240298 learned_addr learned my addr 172.21.15.32:0/2321240298 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:04.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.400+0000 7fa69ea00640 1 --2- 172.21.15.32:0/2321240298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6a006f270 0x7fa6a014f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:04.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.400+0000 7fa69f400640 1 --2- 172.21.15.32:0/2321240298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6a015b7d0 0x7fa6a015a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:04.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.400+0000 7fa697e00640 1 -- 172.21.15.32:0/2321240298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6a015b7d0 msgr2=0x7fa6a015a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:04.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.400+0000 7fa697e00640 1 --2- 172.21.15.32:0/2321240298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6a015b7d0 0x7fa6a015a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:04.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.400+0000 7fa697e00640 1 -- 172.21.15.32:0/2321240298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6a006f270 msgr2=0x7fa6a014f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:04.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.400+0000 7fa697e00640 1 --2- 172.21.15.32:0/2321240298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6a006f270 0x7fa6a014f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:04.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.400+0000 7fa697e00640 1 -- 172.21.15.32:0/2321240298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a015e7f0 con 0x7fa6a006f7c0 2024-04-17T09:56:04.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.400+0000 7fa697e00640 1 --2- 172.21.15.32:0/2321240298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6a006f7c0 0x7fa6a0159de0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa6880ad040 tx=0x7fa6880191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:04.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.400+0000 7fa696a00640 1 -- 172.21.15.32:0/2321240298 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6880abe20 con 0x7fa6a006f7c0 2024-04-17T09:56:04.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.400+0000 7fa696a00640 1 -- 172.21.15.32:0/2321240298 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6880b4600 con 0x7fa6a006f7c0 2024-04-17T09:56:04.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.400+0000 7fa696a00640 1 -- 172.21.15.32:0/2321240298 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6880b49d0 con 0x7fa6a006f7c0 2024-04-17T09:56:04.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.400+0000 7fa6a5a00640 1 -- 172.21.15.32:0/2321240298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a0154660 con 0x7fa6a006f7c0 2024-04-17T09:56:04.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.400+0000 7fa6a5a00640 1 -- 172.21.15.32:0/2321240298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a0154c20 con 0x7fa6a006f7c0 2024-04-17T09:56:04.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.400+0000 7fa6a5a00640 1 -- 172.21.15.32:0/2321240298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a0068a90 con 0x7fa6a006f7c0 2024-04-17T09:56:04.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.404+0000 7fa696a00640 1 -- 172.21.15.32:0/2321240298 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa6880c3060 con 0x7fa6a006f7c0 2024-04-17T09:56:04.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.404+0000 7fa696a00640 1 --2- 172.21.15.32:0/2321240298 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa67c068380 0x7fa67c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:04.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.404+0000 7fa696a00640 1 -- 172.21.15.32:0/2321240298 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fa688120ab0 con 0x7fa6a006f7c0 2024-04-17T09:56:04.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.404+0000 7fa69ea00640 1 --2- 172.21.15.32:0/2321240298 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa67c068380 0x7fa67c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:04.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.404+0000 7fa69ea00640 1 --2- 172.21.15.32:0/2321240298 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa67c068380 0x7fa67c06a840 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa690000f70 tx=0x7fa690007240 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:04.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.408+0000 7fa696a00640 1 -- 172.21.15.32:0/2321240298 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa688016090 con 0x7fa6a006f7c0 2024-04-17T09:56:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:04.616+0000 7fa6a5a00640 1 -- 172.21.15.32:0/2321240298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa6a006eb90 con 0x7fa6a006f7c0 2024-04-17T09:56:05.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.216+0000 7fa696a00640 1 -- 172.21.15.32:0/2321240298 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7fa6880f10f0 con 0x7fa6a006f7c0 2024-04-17T09:56:05.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.216+0000 7fa6a5a00640 1 -- 172.21.15.32:0/2321240298 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa67c068380 msgr2=0x7fa67c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:05.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.216+0000 7fa6a5a00640 1 --2- 172.21.15.32:0/2321240298 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa67c068380 0x7fa67c06a840 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa690000f70 tx=0x7fa690007240 comp rx=0 tx=0).stop 2024-04-17T09:56:05.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.216+0000 7fa6a5a00640 1 -- 172.21.15.32:0/2321240298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6a006f7c0 msgr2=0x7fa6a0159de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:05.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.216+0000 7fa6a5a00640 1 --2- 172.21.15.32:0/2321240298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6a006f7c0 0x7fa6a0159de0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa6880ad040 tx=0x7fa6880191b0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.216+0000 7fa6a5a00640 1 -- 172.21.15.32:0/2321240298 shutdown_connections 2024-04-17T09:56:05.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.216+0000 7fa6a5a00640 1 --2- 172.21.15.32:0/2321240298 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa67c068380 0x7fa67c06a840 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.216+0000 7fa6a5a00640 1 --2- 172.21.15.32:0/2321240298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6a015b7d0 0x7fa6a015a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.216+0000 7fa6a5a00640 1 --2- 172.21.15.32:0/2321240298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6a006f7c0 0x7fa6a0159de0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.216+0000 7fa6a5a00640 1 --2- 172.21.15.32:0/2321240298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6a006f270 0x7fa6a014f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.216+0000 7fa6a5a00640 1 -- 172.21.15.32:0/2321240298 >> 172.21.15.32:0/2321240298 conn(0x7fa6a00b2120 msgr2=0x7fa6a0155ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:05.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.216+0000 7fa6a5a00640 1 -- 172.21.15.32:0/2321240298 shutdown_connections 2024-04-17T09:56:05.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.216+0000 7fa6a5a00640 1 -- 172.21.15.32:0/2321240298 wait complete. 2024-04-17T09:56:05.242 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-17T09:56:05.243 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:56:05.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.388+0000 7effada00640 1 Processor -- start 2024-04-17T09:56:05.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.388+0000 7effada00640 1 -- start start 2024-04-17T09:56:05.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.388+0000 7effada00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effa8145f30 0x7effa8071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:05.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.388+0000 7effada00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7effa815d8d0 0x7effa8072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:05.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.388+0000 7effada00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa80cdac0 con 0x7effa80cce70 2024-04-17T09:56:05.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.388+0000 7effada00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7effa806f400 con 0x7effa815d8d0 2024-04-17T09:56:05.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.388+0000 7effada00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7effa8146310 con 0x7effa8145f30 2024-04-17T09:56:05.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.392+0000 7effa7400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7effa80cce70 0x7effa80cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58406/0 (socket says 172.21.15.32:58406) 2024-04-17T09:56:05.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.392+0000 7effa7400640 1 -- 172.21.15.32:0/3024852042 learned_addr learned my addr 172.21.15.32:0/3024852042 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:05.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.392+0000 7eff9fe00640 1 --2- 172.21.15.32:0/3024852042 >> v2:172.21.15.32:3300/0 conn(0x7effa815d8d0 0x7effa8072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:05.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.392+0000 7effa6a00640 1 --2- 172.21.15.32:0/3024852042 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effa8145f30 0x7effa8071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:05.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.392+0000 7eff9fe00640 1 -- 172.21.15.32:0/3024852042 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effa8145f30 msgr2=0x7effa8071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:05.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.392+0000 7eff9fe00640 1 --2- 172.21.15.32:0/3024852042 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effa8145f30 0x7effa8071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.392+0000 7eff9fe00640 1 -- 172.21.15.32:0/3024852042 >> v1:172.21.15.32:6789/0 conn(0x7effa80cce70 legacy=0x7effa80cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:05.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.392+0000 7eff9fe00640 1 -- 172.21.15.32:0/3024852042 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa815cdf0 con 0x7effa815d8d0 2024-04-17T09:56:05.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.392+0000 7eff9fe00640 1 --2- 172.21.15.32:0/3024852042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effa815d8d0 0x7effa8072330 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7eff900097c0 tx=0x7eff9001ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533e9a2198f90e2b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:05.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.392+0000 7effa6000640 1 -- 172.21.15.32:0/3024852042 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff900b5070 con 0x7effa815d8d0 2024-04-17T09:56:05.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.392+0000 7effa6000640 1 -- 172.21.15.32:0/3024852042 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff900a8990 con 0x7effa815d8d0 2024-04-17T09:56:05.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.392+0000 7effa6000640 1 -- 172.21.15.32:0/3024852042 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff900a8d60 con 0x7effa815d8d0 2024-04-17T09:56:05.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effada00640 1 -- 172.21.15.32:0/3024852042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effa815d8d0 msgr2=0x7effa8072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:05.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effada00640 1 --2- 172.21.15.32:0/3024852042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effa815d8d0 0x7effa8072330 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7eff900097c0 tx=0x7eff9001ec90 comp rx=0 tx=0).stop 2024-04-17T09:56:05.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effada00640 1 -- 172.21.15.32:0/3024852042 shutdown_connections 2024-04-17T09:56:05.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effada00640 1 --2- 172.21.15.32:0/3024852042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effa815d8d0 0x7effa8072330 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effada00640 1 --2- 172.21.15.32:0/3024852042 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effa8145f30 0x7effa8071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effada00640 1 -- 172.21.15.32:0/3024852042 >> 172.21.15.32:0/3024852042 conn(0x7effa806ba30 msgr2=0x7effa806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:05.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effada00640 1 -- 172.21.15.32:0/3024852042 shutdown_connections 2024-04-17T09:56:05.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effada00640 1 -- 172.21.15.32:0/3024852042 wait complete. 2024-04-17T09:56:05.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effada00640 1 Processor -- start 2024-04-17T09:56:05.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effada00640 1 -- start start 2024-04-17T09:56:05.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effada00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effa80cce70 0x7effa8160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:05.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effada00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effa8145f30 0x7effa8160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:05.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effada00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effa815d8d0 0x7effa80b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:05.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effada00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7effa815cb90 con 0x7effa80cce70 2024-04-17T09:56:05.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effada00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7effa815d440 con 0x7effa815d8d0 2024-04-17T09:56:05.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effada00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7effa81601d0 con 0x7effa8145f30 2024-04-17T09:56:05.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effa6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effa80cce70 0x7effa8160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:05.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effa6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effa80cce70 0x7effa8160600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49222/0 (socket says 172.21.15.32:49222) 2024-04-17T09:56:05.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effa6a00640 1 -- 172.21.15.32:0/2721996090 learned_addr learned my addr 172.21.15.32:0/2721996090 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:05.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7eff9fe00640 1 --2- 172.21.15.32:0/2721996090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effa8145f30 0x7effa8160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:05.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effa7400640 1 --2- 172.21.15.32:0/2721996090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effa815d8d0 0x7effa80b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:05.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effa6a00640 1 -- 172.21.15.32:0/2721996090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effa8145f30 msgr2=0x7effa8160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:05.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effa6a00640 1 --2- 172.21.15.32:0/2721996090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effa8145f30 0x7effa8160b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effa6a00640 1 -- 172.21.15.32:0/2721996090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effa815d8d0 msgr2=0x7effa80b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:05.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effa6a00640 1 --2- 172.21.15.32:0/2721996090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effa815d8d0 0x7effa80b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effa6a00640 1 -- 172.21.15.32:0/2721996090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa80ca5d0 con 0x7effa80cce70 2024-04-17T09:56:05.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effa7400640 1 --2- 172.21.15.32:0/2721996090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effa815d8d0 0x7effa80b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:05.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.396+0000 7effa6a00640 1 --2- 172.21.15.32:0/2721996090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effa80cce70 0x7effa8160600 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7eff9800e930 tx=0x7eff9800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:05.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.400+0000 7eff9ea00640 1 -- 172.21.15.32:0/2721996090 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98017070 con 0x7effa80cce70 2024-04-17T09:56:05.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.400+0000 7eff9ea00640 1 -- 172.21.15.32:0/2721996090 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff9800c9a0 con 0x7effa80cce70 2024-04-17T09:56:05.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.400+0000 7eff9ea00640 1 -- 172.21.15.32:0/2721996090 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9800cd50 con 0x7effa80cce70 2024-04-17T09:56:05.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.400+0000 7effada00640 1 -- 172.21.15.32:0/2721996090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa80b4c00 con 0x7effa80cce70 2024-04-17T09:56:05.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.400+0000 7effada00640 1 -- 172.21.15.32:0/2721996090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa80cab50 con 0x7effa80cce70 2024-04-17T09:56:05.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.400+0000 7effada00640 1 -- 172.21.15.32:0/2721996090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa8068a90 con 0x7effa80cce70 2024-04-17T09:56:05.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.400+0000 7eff9ea00640 1 -- 172.21.15.32:0/2721996090 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7eff9801a070 con 0x7effa80cce70 2024-04-17T09:56:05.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.400+0000 7eff9ea00640 1 --2- 172.21.15.32:0/2721996090 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7eff84068380 0x7eff8406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:05.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.400+0000 7eff9ea00640 1 -- 172.21.15.32:0/2721996090 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7eff980869b0 con 0x7effa80cce70 2024-04-17T09:56:05.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.400+0000 7eff9fe00640 1 --2- 172.21.15.32:0/2721996090 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7eff84068380 0x7eff8406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:05.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.404+0000 7eff9fe00640 1 --2- 172.21.15.32:0/2721996090 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7eff84068380 0x7eff8406a840 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7eff900b2000 tx=0x7eff90008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:05.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.404+0000 7eff9ea00640 1 -- 172.21.15.32:0/2721996090 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7eff98057020 con 0x7effa80cce70 2024-04-17T09:56:05.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.604+0000 7effada00640 1 -- 172.21.15.32:0/2721996090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effa80b4850 con 0x7effa80cce70 2024-04-17T09:56:05.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.604+0000 7eff9ea00640 1 -- 172.21.15.32:0/2721996090 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5202 (secure 0 0 0) 0x7eff98056920 con 0x7effa80cce70 2024-04-17T09:56:05.607 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:05.607 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5495,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/1236891874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1236891874},{"type":"v1","addr":"172.21.15.144:6834","nonce":1236891874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5498,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3538881342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3538881342},{"type":"v1","addr":"172.21.15.144:6835","nonce":3538881342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6201,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3549801633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3549801633},{"type":"v1","addr":"172.21.15.32:6836","nonce":3549801633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:56:01.579180+0000","modified":"2024-04-17T09:56:04.622581+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/866826055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":866826055},{"type":"v1","addr":"172.21.15.32:6837","nonce":866826055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-04-17T09:56:05.607 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 32 2024-04-17T09:56:05.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.608+0000 7effada00640 1 -- 172.21.15.32:0/2721996090 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7eff84068380 msgr2=0x7eff8406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:05.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.608+0000 7effada00640 1 --2- 172.21.15.32:0/2721996090 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7eff84068380 0x7eff8406a840 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7eff900b2000 tx=0x7eff90008040 comp rx=0 tx=0).stop 2024-04-17T09:56:05.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.608+0000 7effada00640 1 -- 172.21.15.32:0/2721996090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effa80cce70 msgr2=0x7effa8160600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:05.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.608+0000 7effada00640 1 --2- 172.21.15.32:0/2721996090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effa80cce70 0x7effa8160600 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7eff9800e930 tx=0x7eff9800edf0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.608+0000 7effada00640 1 -- 172.21.15.32:0/2721996090 shutdown_connections 2024-04-17T09:56:05.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.608+0000 7effada00640 1 --2- 172.21.15.32:0/2721996090 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7eff84068380 0x7eff8406a840 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.608+0000 7effada00640 1 --2- 172.21.15.32:0/2721996090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effa8145f30 0x7effa8160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.608+0000 7effada00640 1 --2- 172.21.15.32:0/2721996090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effa815d8d0 0x7effa80b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.608+0000 7effada00640 1 --2- 172.21.15.32:0/2721996090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effa80cce70 0x7effa8160600 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.608+0000 7effada00640 1 -- 172.21.15.32:0/2721996090 >> 172.21.15.32:0/2721996090 conn(0x7effa806ba30 msgr2=0x7effa815e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:05.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.608+0000 7effada00640 1 -- 172.21.15.32:0/2721996090 shutdown_connections 2024-04-17T09:56:05.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.608+0000 7effada00640 1 -- 172.21.15.32:0/2721996090 wait complete. 2024-04-17T09:56:05.628 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:56:05.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff788400640 1 Processor -- start 2024-04-17T09:56:05.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff788400640 1 -- start start 2024-04-17T09:56:05.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff788400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff780144e90 0x7ff7800b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:05.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff788400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7800b4de0 0x7ff78015baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:05.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff788400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff780072bb0 con 0x7ff78015bfe0 2024-04-17T09:56:05.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff788400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff78015c430 con 0x7ff780144e90 2024-04-17T09:56:05.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff788400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff780145270 con 0x7ff7800b4de0 2024-04-17T09:56:05.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff786400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff78015bfe0 0x7ff78006ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58414/0 (socket says 172.21.15.32:58414) 2024-04-17T09:56:05.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff786400640 1 -- 172.21.15.32:0/1466755485 learned_addr learned my addr 172.21.15.32:0/1466755485 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:05.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff785a00640 1 --2- 172.21.15.32:0/1466755485 >> v2:172.21.15.144:3300/0 conn(0x7ff780144e90 0x7ff7800b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:05.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff785000640 1 --2- 172.21.15.32:0/1466755485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7800b4de0 0x7ff78015baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:05.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff77fe00640 1 -- 172.21.15.32:0/1466755485 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 711689163 0 0) 0x7ff780072bb0 con 0x7ff78015bfe0 2024-04-17T09:56:05.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff77fe00640 1 -- 172.21.15.32:0/1466755485 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff768003530 con 0x7ff78015bfe0 2024-04-17T09:56:05.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff77fe00640 1 -- 172.21.15.32:0/1466755485 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4017370587 0 0) 0x7ff768003530 con 0x7ff78015bfe0 2024-04-17T09:56:05.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff77fe00640 1 -- 172.21.15.32:0/1466755485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7800b4de0 msgr2=0x7ff78015baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:05.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff77fe00640 1 --2- 172.21.15.32:0/1466755485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7800b4de0 0x7ff78015baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff77fe00640 1 -- 172.21.15.32:0/1466755485 >> v2:172.21.15.144:3300/0 conn(0x7ff780144e90 msgr2=0x7ff7800b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:05.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff77fe00640 1 --2- 172.21.15.32:0/1466755485 >> v2:172.21.15.144:3300/0 conn(0x7ff780144e90 0x7ff7800b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff77fe00640 1 -- 172.21.15.32:0/1466755485 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7800b3dc0 con 0x7ff78015bfe0 2024-04-17T09:56:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff77fe00640 1 -- 172.21.15.32:0/1466755485 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7ff778001a00 con 0x7ff78015bfe0 2024-04-17T09:56:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff77fe00640 1 -- 172.21.15.32:0/1466755485 >> v1:172.21.15.32:6789/0 conn(0x7ff78015bfe0 legacy=0x7ff78006ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff77fe00640 1 --2- 172.21.15.32:0/1466755485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7800b4de0 0x7ff7680039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff77fe00640 1 --2- 172.21.15.32:0/1466755485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff768003f70 0x7ff768004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff77fe00640 1 --2- 172.21.15.32:0/1466755485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff768004d50 0x7ff7680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff77fe00640 1 -- 172.21.15.32:0/1466755485 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff78015c430 con 0x7ff768003f70 2024-04-17T09:56:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff77fe00640 1 -- 172.21.15.32:0/1466755485 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff780145270 con 0x7ff768004d50 2024-04-17T09:56:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff77fe00640 1 -- 172.21.15.32:0/1466755485 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff76800b9d0 con 0x7ff7800b4de0 2024-04-17T09:56:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff785000640 1 --2- 172.21.15.32:0/1466755485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff768003f70 0x7ff768004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff786400640 1 --2- 172.21.15.32:0/1466755485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff768004d50 0x7ff7680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff785a00640 1 --2- 172.21.15.32:0/1466755485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7800b4de0 0x7ff7680039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff785000640 1 -- 172.21.15.32:0/1466755485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7800b4de0 msgr2=0x7ff7680039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff785000640 1 --2- 172.21.15.32:0/1466755485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7800b4de0 0x7ff7680039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff785000640 1 -- 172.21.15.32:0/1466755485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff768004d50 msgr2=0x7ff7680051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff785000640 1 --2- 172.21.15.32:0/1466755485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff768004d50 0x7ff7680051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.768+0000 7ff785000640 1 -- 172.21.15.32:0/1466755485 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff768003530 con 0x7ff768003f70 2024-04-17T09:56:05.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.772+0000 7ff785000640 1 --2- 172.21.15.32:0/1466755485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff768003f70 0x7ff768004810 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009670 tx=0x7ff76c01eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eeeeda8d4aee14b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:05.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.772+0000 7ff77fe00640 1 -- 172.21.15.32:0/1466755485 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0b4070 con 0x7ff768003f70 2024-04-17T09:56:05.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.772+0000 7ff77fe00640 1 -- 172.21.15.32:0/1466755485 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff76c0a7790 con 0x7ff768003f70 2024-04-17T09:56:05.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.772+0000 7ff77fe00640 1 -- 172.21.15.32:0/1466755485 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0096b0 con 0x7ff768003f70 2024-04-17T09:56:05.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.772+0000 7ff788400640 1 -- 172.21.15.32:0/1466755485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff768003f70 msgr2=0x7ff768004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:05.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.772+0000 7ff788400640 1 --2- 172.21.15.32:0/1466755485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff768003f70 0x7ff768004810 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009670 tx=0x7ff76c01eca0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.772+0000 7ff786400640 1 -- 172.21.15.32:0/1466755485 reap_dead start 2024-04-17T09:56:05.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.772+0000 7ff788400640 1 -- 172.21.15.32:0/1466755485 shutdown_connections 2024-04-17T09:56:05.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.772+0000 7ff788400640 1 -- 172.21.15.32:0/1466755485 >> 172.21.15.32:0/1466755485 conn(0x7ff78006ba30 msgr2=0x7ff78006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:05.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.772+0000 7ff788400640 1 -- 172.21.15.32:0/1466755485 shutdown_connections 2024-04-17T09:56:05.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.772+0000 7ff788400640 1 -- 172.21.15.32:0/1466755485 wait complete. 2024-04-17T09:56:05.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff788400640 1 Processor -- start 2024-04-17T09:56:05.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff788400640 1 -- start start 2024-04-17T09:56:05.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff788400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff768003f70 0x7ff7801f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:05.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff788400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7800ad860 0x7ff78013f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:05.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff788400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7801f4420 0x7ff780142f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:05.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff788400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff78006f670 con 0x7ff768003f70 2024-04-17T09:56:05.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff788400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff7800b4380 con 0x7ff7801f4420 2024-04-17T09:56:05.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff788400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff78006f7f0 con 0x7ff7800ad860 2024-04-17T09:56:05.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff785a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff768003f70 0x7ff7801f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:05.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff785a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff768003f70 0x7ff7801f85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49236/0 (socket says 172.21.15.32:49236) 2024-04-17T09:56:05.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff785a00640 1 -- 172.21.15.32:0/3249354689 learned_addr learned my addr 172.21.15.32:0/3249354689 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:05.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff786400640 1 --2- 172.21.15.32:0/3249354689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7801f4420 0x7ff780142f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:05.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff785000640 1 --2- 172.21.15.32:0/3249354689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7800ad860 0x7ff78013f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:05.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff785a00640 1 -- 172.21.15.32:0/3249354689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7800ad860 msgr2=0x7ff78013f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:05.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff785a00640 1 --2- 172.21.15.32:0/3249354689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7800ad860 0x7ff78013f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff785a00640 1 -- 172.21.15.32:0/3249354689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7801f4420 msgr2=0x7ff780142f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:05.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff785a00640 1 --2- 172.21.15.32:0/3249354689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7801f4420 0x7ff780142f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff785a00640 1 -- 172.21.15.32:0/3249354689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78013fb70 con 0x7ff768003f70 2024-04-17T09:56:05.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff785a00640 1 --2- 172.21.15.32:0/3249354689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff768003f70 0x7ff7801f85a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff774011690 tx=0x7ff774011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:05.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff77d600640 1 -- 172.21.15.32:0/3249354689 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774015870 con 0x7ff768003f70 2024-04-17T09:56:05.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff77d600640 1 -- 172.21.15.32:0/3249354689 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff774015a10 con 0x7ff768003f70 2024-04-17T09:56:05.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff77d600640 1 -- 172.21.15.32:0/3249354689 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774015dc0 con 0x7ff768003f70 2024-04-17T09:56:05.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff788400640 1 -- 172.21.15.32:0/3249354689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7801f4e50 con 0x7ff768003f70 2024-04-17T09:56:05.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.776+0000 7ff788400640 1 -- 172.21.15.32:0/3249354689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7800c55a0 con 0x7ff768003f70 2024-04-17T09:56:05.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.780+0000 7ff788400640 1 -- 172.21.15.32:0/3249354689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7801f5350 con 0x7ff768003f70 2024-04-17T09:56:05.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.780+0000 7ff77d600640 1 -- 172.21.15.32:0/3249354689 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff77400f460 con 0x7ff768003f70 2024-04-17T09:56:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.780+0000 7ff77d600640 1 --2- 172.21.15.32:0/3249354689 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff760068380 0x7ff76006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.780+0000 7ff77d600640 1 -- 172.21.15.32:0/3249354689 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7ff774086dd0 con 0x7ff768003f70 2024-04-17T09:56:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.780+0000 7ff785000640 1 --2- 172.21.15.32:0/3249354689 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff760068380 0x7ff76006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.780+0000 7ff785000640 1 --2- 172.21.15.32:0/3249354689 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff760068380 0x7ff76006a840 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0b1000 tx=0x7ff76c010040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:05.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.784+0000 7ff77d600640 1 -- 172.21.15.32:0/3249354689 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff78013fb70 con 0x7ff768003f70 2024-04-17T09:56:05.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.952+0000 7ff788400640 1 -- 172.21.15.32:0/3249354689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff780068a90 con 0x7ff768003f70 2024-04-17T09:56:05.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.952+0000 7ff77d600640 1 -- 172.21.15.32:0/3249354689 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16677 (secure 0 0 0) 0x7ff774019030 con 0x7ff768003f70 2024-04-17T09:56:05.954 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:05.955 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":57,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:56:03.192930+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:55:59.976232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-17T09:56:00.543788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:56:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.956+0000 7ff788400640 1 -- 172.21.15.32:0/3249354689 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff760068380 msgr2=0x7ff76006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.956+0000 7ff788400640 1 --2- 172.21.15.32:0/3249354689 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff760068380 0x7ff76006a840 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0b1000 tx=0x7ff76c010040 comp rx=0 tx=0).stop 2024-04-17T09:56:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.956+0000 7ff788400640 1 -- 172.21.15.32:0/3249354689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff768003f70 msgr2=0x7ff7801f85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.956+0000 7ff788400640 1 --2- 172.21.15.32:0/3249354689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff768003f70 0x7ff7801f85a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff774011690 tx=0x7ff774011a90 comp rx=0 tx=0).stop 2024-04-17T09:56:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.956+0000 7ff788400640 1 -- 172.21.15.32:0/3249354689 shutdown_connections 2024-04-17T09:56:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.956+0000 7ff788400640 1 --2- 172.21.15.32:0/3249354689 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff760068380 0x7ff76006a840 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.956+0000 7ff788400640 1 --2- 172.21.15.32:0/3249354689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7800ad860 0x7ff78013f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.956+0000 7ff788400640 1 --2- 172.21.15.32:0/3249354689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7801f4420 0x7ff780142f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.956+0000 7ff788400640 1 --2- 172.21.15.32:0/3249354689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff768003f70 0x7ff7801f85a0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.956+0000 7ff788400640 1 -- 172.21.15.32:0/3249354689 >> 172.21.15.32:0/3249354689 conn(0x7ff78006ba30 msgr2=0x7ff7800b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.956+0000 7ff788400640 1 -- 172.21.15.32:0/3249354689 shutdown_connections 2024-04-17T09:56:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:05.956+0000 7ff788400640 1 -- 172.21.15.32:0/3249354689 wait complete. 2024-04-17T09:56:05.975 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T09:56:05.976 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T09:56:06.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.112+0000 7f788ee00640 1 Processor -- start 2024-04-17T09:56:06.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.112+0000 7f788ee00640 1 -- start start 2024-04-17T09:56:06.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.116+0000 7f788ee00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7888145f30 0x7f7888071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:06.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.116+0000 7f788ee00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f78880cd580 0x7f7888072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:06.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.116+0000 7f788ee00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78880cdac0 con 0x7f788815d8d0 2024-04-17T09:56:06.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.116+0000 7f788ee00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f788806f400 con 0x7f78880cd580 2024-04-17T09:56:06.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.116+0000 7f788ee00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f7888146310 con 0x7f7888145f30 2024-04-17T09:56:06.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.116+0000 7f788ce00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f78880cd580 0x7f7888072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:06.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.116+0000 7f788ce00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f78880cd580 0x7f7888072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49240/0 (socket says 172.21.15.32:49240) 2024-04-17T09:56:06.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.116+0000 7f788ce00640 1 -- 172.21.15.32:0/4105413093 learned_addr learned my addr 172.21.15.32:0/4105413093 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:06.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.116+0000 7f7887e00640 1 --2- 172.21.15.32:0/4105413093 >> v2:172.21.15.144:3300/0 conn(0x7f7888145f30 0x7f7888071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:06.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.116+0000 7f788ce00640 1 -- 172.21.15.32:0/4105413093 >> v2:172.21.15.144:3300/0 conn(0x7f7888145f30 msgr2=0x7f7888071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:06.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.116+0000 7f788ce00640 1 --2- 172.21.15.32:0/4105413093 >> v2:172.21.15.144:3300/0 conn(0x7f7888145f30 0x7f7888071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.116+0000 7f788ce00640 1 -- 172.21.15.32:0/4105413093 >> v1:172.21.15.144:6789/0 conn(0x7f788815d8d0 legacy=0x7f78880cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:06.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.116+0000 7f788ce00640 1 -- 172.21.15.32:0/4105413093 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788815cdf0 con 0x7f78880cd580 2024-04-17T09:56:06.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.120+0000 7f788ce00640 1 --2- 172.21.15.32:0/4105413093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78880cd580 0x7f7888072330 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f788806ff00 tx=0x7f788001f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df33228b2f78d443 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:06.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.120+0000 7f7887400640 1 -- 172.21.15.32:0/4105413093 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78800b4070 con 0x7f78880cd580 2024-04-17T09:56:06.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.120+0000 7f7887400640 1 -- 172.21.15.32:0/4105413093 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78800af400 con 0x7f78880cd580 2024-04-17T09:56:06.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.120+0000 7f7887400640 1 -- 172.21.15.32:0/4105413093 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78800af7d0 con 0x7f78880cd580 2024-04-17T09:56:06.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.120+0000 7f788ee00640 1 -- 172.21.15.32:0/4105413093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78880cd580 msgr2=0x7f7888072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:06.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.120+0000 7f788ee00640 1 --2- 172.21.15.32:0/4105413093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78880cd580 0x7f7888072330 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f788806ff00 tx=0x7f788001f1e0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.120+0000 7f788ee00640 1 -- 172.21.15.32:0/4105413093 shutdown_connections 2024-04-17T09:56:06.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.120+0000 7f788ee00640 1 --2- 172.21.15.32:0/4105413093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78880cd580 0x7f7888072330 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.120+0000 7f788ee00640 1 --2- 172.21.15.32:0/4105413093 >> v2:172.21.15.144:3300/0 conn(0x7f7888145f30 0x7f7888071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.120+0000 7f788ee00640 1 -- 172.21.15.32:0/4105413093 >> 172.21.15.32:0/4105413093 conn(0x7f788806ba30 msgr2=0x7f788806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:06.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.120+0000 7f788ee00640 1 -- 172.21.15.32:0/4105413093 shutdown_connections 2024-04-17T09:56:06.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.120+0000 7f788ee00640 1 -- 172.21.15.32:0/4105413093 wait complete. 2024-04-17T09:56:06.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.120+0000 7f788ee00640 1 Processor -- start 2024-04-17T09:56:06.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.120+0000 7f788ee00640 1 -- start start 2024-04-17T09:56:06.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f788ee00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78880cd580 0x7f788815bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:06.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f788ee00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7888145f30 0x7f78881e8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:06.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f788ee00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f788815d8d0 0x7f78881e8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:06.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f788ee00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f788815cb90 con 0x7f78880cd580 2024-04-17T09:56:06.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f788ee00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f788815d440 con 0x7f788815d8d0 2024-04-17T09:56:06.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f788ee00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f78881601d0 con 0x7f7888145f30 2024-04-17T09:56:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f7887e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78880cd580 0x7f788815bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f7887e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78880cd580 0x7f788815bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49248/0 (socket says 172.21.15.32:49248) 2024-04-17T09:56:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f7887e00640 1 -- 172.21.15.32:0/3892763032 learned_addr learned my addr 172.21.15.32:0/3892763032 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f788ce00640 1 --2- 172.21.15.32:0/3892763032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f788815d8d0 0x7f78881e8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f787fe00640 1 --2- 172.21.15.32:0/3892763032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7888145f30 0x7f78881e8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f7887e00640 1 -- 172.21.15.32:0/3892763032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7888145f30 msgr2=0x7f78881e8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f7887e00640 1 --2- 172.21.15.32:0/3892763032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7888145f30 0x7f78881e8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f7887e00640 1 -- 172.21.15.32:0/3892763032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f788815d8d0 msgr2=0x7f78881e8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f7887e00640 1 --2- 172.21.15.32:0/3892763032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f788815d8d0 0x7f78881e8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f7887e00640 1 -- 172.21.15.32:0/3892763032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78880c6b30 con 0x7f78880cd580 2024-04-17T09:56:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f7887e00640 1 --2- 172.21.15.32:0/3892763032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78880cd580 0x7f788815bfc0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f787800e930 tx=0x7f787800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f7884c00640 1 -- 172.21.15.32:0/3892763032 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7878017070 con 0x7f78880cd580 2024-04-17T09:56:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f7884c00640 1 -- 172.21.15.32:0/3892763032 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f787800ca00 con 0x7f78880cd580 2024-04-17T09:56:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f7884c00640 1 -- 172.21.15.32:0/3892763032 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787800cdb0 con 0x7f78880cd580 2024-04-17T09:56:06.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f788ee00640 1 -- 172.21.15.32:0/3892763032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78880cc0b0 con 0x7f78880cd580 2024-04-17T09:56:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f788ee00640 1 -- 172.21.15.32:0/3892763032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78880c70d0 con 0x7f78880cd580 2024-04-17T09:56:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f788ee00640 1 -- 172.21.15.32:0/3892763032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7888068a90 con 0x7f78880cd580 2024-04-17T09:56:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.124+0000 7f7884c00640 1 -- 172.21.15.32:0/3892763032 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f787801a070 con 0x7f78880cd580 2024-04-17T09:56:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.128+0000 7f7884c00640 1 --2- 172.21.15.32:0/3892763032 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7864068380 0x7f786406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.128+0000 7f7884c00640 1 -- 172.21.15.32:0/3892763032 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f7878086980 con 0x7f78880cd580 2024-04-17T09:56:06.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.128+0000 7f787fe00640 1 --2- 172.21.15.32:0/3892763032 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7864068380 0x7f786406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:06.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.128+0000 7f787fe00640 1 --2- 172.21.15.32:0/3892763032 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7864068380 0x7f786406a840 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f78880c68b0 tx=0x7f7870009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:06.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.132+0000 7f7884c00640 1 -- 172.21.15.32:0/3892763032 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f78780569a0 con 0x7f78880cd580 2024-04-17T09:56:06.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.296+0000 7f788ee00640 1 -- 172.21.15.32:0/3892763032 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7888072f90 con 0x7f7864068380 2024-04-17T09:56:06.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.304+0000 7f7884c00640 1 -- 172.21.15.32:0/3892763032 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+243016 (secure 0 0 0) 0x7f7888072f90 con 0x7f7864068380 2024-04-17T09:56:06.307 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:06.308 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T09:56:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.304+0000 7f788ee00640 1 -- 172.21.15.32:0/3892763032 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7864068380 msgr2=0x7f786406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.308+0000 7f788ee00640 1 --2- 172.21.15.32:0/3892763032 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7864068380 0x7f786406a840 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f78880c68b0 tx=0x7f7870009040 comp rx=0 tx=0).stop 2024-04-17T09:56:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.308+0000 7f788ee00640 1 -- 172.21.15.32:0/3892763032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78880cd580 msgr2=0x7f788815bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.308+0000 7f788ee00640 1 --2- 172.21.15.32:0/3892763032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78880cd580 0x7f788815bfc0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f787800e930 tx=0x7f787800edf0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.308+0000 7f788ee00640 1 -- 172.21.15.32:0/3892763032 shutdown_connections 2024-04-17T09:56:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.308+0000 7f788ee00640 1 --2- 172.21.15.32:0/3892763032 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7864068380 0x7f786406a840 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.308+0000 7f788ee00640 1 --2- 172.21.15.32:0/3892763032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7888145f30 0x7f78881e8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.308+0000 7f788ee00640 1 --2- 172.21.15.32:0/3892763032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f788815d8d0 0x7f78881e8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.308+0000 7f788ee00640 1 --2- 172.21.15.32:0/3892763032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78880cd580 0x7f788815bfc0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.308+0000 7f788ee00640 1 -- 172.21.15.32:0/3892763032 >> 172.21.15.32:0/3892763032 conn(0x7f788806ba30 msgr2=0x7f7888144b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.308+0000 7f788ee00640 1 -- 172.21.15.32:0/3892763032 shutdown_connections 2024-04-17T09:56:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.308+0000 7f788ee00640 1 -- 172.21.15.32:0/3892763032 wait complete. 2024-04-17T09:56:06.327 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-04-17T09:56:05.321817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592562,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":176,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":98,"acting":98,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":243180,"kb_used_data":6192,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731540,"statfs":{"total":773094113280,"available":772845096960,"internally_reserved":0,"allocated":6340608,"data_stored":3275885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2194,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.145090"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197311+0000","last_change":"2024-04-17T09:56:02.221498+0000","last_active":"2024-04-17T09:56:03.197311+0000","last_peered":"2024-04-17T09:56:03.197311+0000","last_clean":"2024-04-17T09:56:03.197311+0000","last_became_active":"2024-04-17T09:56:02.217279+0000","last_became_peered":"2024-04-17T09:56:02.217279+0000","last_unstale":"2024-04-17T09:56:03.197311+0000","last_undegraded":"2024-04-17T09:56:03.197311+0000","last_fullsized":"2024-04-17T09:56:03.197311+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:53.818908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203103+0000","last_change":"2024-04-17T09:56:02.220843+0000","last_active":"2024-04-17T09:56:03.203103+0000","last_peered":"2024-04-17T09:56:03.203103+0000","last_clean":"2024-04-17T09:56:03.203103+0000","last_became_active":"2024-04-17T09:56:02.220566+0000","last_became_peered":"2024-04-17T09:56:02.220566+0000","last_unstale":"2024-04-17T09:56:03.203103+0000","last_undegraded":"2024-04-17T09:56:03.203103+0000","last_fullsized":"2024-04-17T09:56:03.203103+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:24:38.324835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203578+0000","last_change":"2024-04-17T09:56:02.230219+0000","last_active":"2024-04-17T09:56:03.203578+0000","last_peered":"2024-04-17T09:56:03.203578+0000","last_clean":"2024-04-17T09:56:03.203578+0000","last_became_active":"2024-04-17T09:56:02.229840+0000","last_became_peered":"2024-04-17T09:56:02.229840+0000","last_unstale":"2024-04-17T09:56:03.203578+0000","last_undegraded":"2024-04-17T09:56:03.203578+0000","last_fullsized":"2024-04-17T09:56:03.203578+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:14:09.222091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203987+0000","last_change":"2024-04-17T09:56:02.213294+0000","last_active":"2024-04-17T09:56:03.203987+0000","last_peered":"2024-04-17T09:56:03.203987+0000","last_clean":"2024-04-17T09:56:03.203987+0000","last_became_active":"2024-04-17T09:56:02.212904+0000","last_became_peered":"2024-04-17T09:56:02.212904+0000","last_unstale":"2024-04-17T09:56:03.203987+0000","last_undegraded":"2024-04-17T09:56:03.203987+0000","last_fullsized":"2024-04-17T09:56:03.203987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:41:23.655282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203242+0000","last_change":"2024-04-17T09:56:02.222178+0000","last_active":"2024-04-17T09:56:03.203242+0000","last_peered":"2024-04-17T09:56:03.203242+0000","last_clean":"2024-04-17T09:56:03.203242+0000","last_became_active":"2024-04-17T09:56:02.221835+0000","last_became_peered":"2024-04-17T09:56:02.221835+0000","last_unstale":"2024-04-17T09:56:03.203242+0000","last_undegraded":"2024-04-17T09:56:03.203242+0000","last_fullsized":"2024-04-17T09:56:03.203242+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:40:42.141566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197006+0000","last_change":"2024-04-17T09:56:02.215969+0000","last_active":"2024-04-17T09:56:03.197006+0000","last_peered":"2024-04-17T09:56:03.197006+0000","last_clean":"2024-04-17T09:56:03.197006+0000","last_became_active":"2024-04-17T09:56:02.215670+0000","last_became_peered":"2024-04-17T09:56:02.215670+0000","last_unstale":"2024-04-17T09:56:03.197006+0000","last_undegraded":"2024-04-17T09:56:03.197006+0000","last_fullsized":"2024-04-17T09:56:03.197006+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:01:34.123890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197587+0000","last_change":"2024-04-17T09:56:02.215021+0000","last_active":"2024-04-17T09:56:03.197587+0000","last_peered":"2024-04-17T09:56:03.197587+0000","last_clean":"2024-04-17T09:56:03.197587+0000","last_became_active":"2024-04-17T09:56:02.214653+0000","last_became_peered":"2024-04-17T09:56:02.214653+0000","last_unstale":"2024-04-17T09:56:03.197587+0000","last_undegraded":"2024-04-17T09:56:03.197587+0000","last_fullsized":"2024-04-17T09:56:03.197587+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:00:41.794977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197005+0000","last_change":"2024-04-17T09:56:02.225982+0000","last_active":"2024-04-17T09:56:03.197005+0000","last_peered":"2024-04-17T09:56:03.197005+0000","last_clean":"2024-04-17T09:56:03.197005+0000","last_became_active":"2024-04-17T09:56:02.223371+0000","last_became_peered":"2024-04-17T09:56:02.223371+0000","last_unstale":"2024-04-17T09:56:03.197005+0000","last_undegraded":"2024-04-17T09:56:03.197005+0000","last_fullsized":"2024-04-17T09:56:03.197005+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:48:17.886969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197834+0000","last_change":"2024-04-17T09:56:02.224842+0000","last_active":"2024-04-17T09:56:03.197834+0000","last_peered":"2024-04-17T09:56:03.197834+0000","last_clean":"2024-04-17T09:56:03.197834+0000","last_became_active":"2024-04-17T09:56:02.224545+0000","last_became_peered":"2024-04-17T09:56:02.224545+0000","last_unstale":"2024-04-17T09:56:03.197834+0000","last_undegraded":"2024-04-17T09:56:03.197834+0000","last_fullsized":"2024-04-17T09:56:03.197834+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:57.566831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"56'4","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197875+0000","last_change":"2024-04-17T09:56:01.206168+0000","last_active":"2024-04-17T09:56:03.197875+0000","last_peered":"2024-04-17T09:56:03.197875+0000","last_clean":"2024-04-17T09:56:03.197875+0000","last_became_active":"2024-04-17T09:56:01.204386+0000","last_became_peered":"2024-04-17T09:56:01.204386+0000","last_unstale":"2024-04-17T09:56:03.197875+0000","last_undegraded":"2024-04-17T09:56:03.197875+0000","last_fullsized":"2024-04-17T09:56:03.197875+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:34.492070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.185133+0000","last_change":"2024-04-17T09:56:00.185133+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.185133+0000","last_undegraded":"2024-04-17T09:56:00.185133+0000","last_fullsized":"2024-04-17T09:56:00.185133+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197603+0000","last_change":"2024-04-17T09:56:02.234199+0000","last_active":"2024-04-17T09:56:03.197603+0000","last_peered":"2024-04-17T09:56:03.197603+0000","last_clean":"2024-04-17T09:56:03.197603+0000","last_became_active":"2024-04-17T09:56:02.233686+0000","last_became_peered":"2024-04-17T09:56:02.233686+0000","last_unstale":"2024-04-17T09:56:03.197603+0000","last_undegraded":"2024-04-17T09:56:03.197603+0000","last_fullsized":"2024-04-17T09:56:03.197603+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:08.333973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197902+0000","last_change":"2024-04-17T09:56:01.201154+0000","last_active":"2024-04-17T09:56:03.197902+0000","last_peered":"2024-04-17T09:56:03.197902+0000","last_clean":"2024-04-17T09:56:03.197902+0000","last_became_active":"2024-04-17T09:56:01.200926+0000","last_became_peered":"2024-04-17T09:56:01.200926+0000","last_unstale":"2024-04-17T09:56:03.197902+0000","last_undegraded":"2024-04-17T09:56:03.197902+0000","last_fullsized":"2024-04-17T09:56:03.197902+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:41:22.649311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197793+0000","last_change":"2024-04-17T09:56:02.230469+0000","last_active":"2024-04-17T09:56:03.197793+0000","last_peered":"2024-04-17T09:56:03.197793+0000","last_clean":"2024-04-17T09:56:03.197793+0000","last_became_active":"2024-04-17T09:56:02.229210+0000","last_became_peered":"2024-04-17T09:56:02.229210+0000","last_unstale":"2024-04-17T09:56:03.197793+0000","last_undegraded":"2024-04-17T09:56:03.197793+0000","last_fullsized":"2024-04-17T09:56:03.197793+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:58:25.393385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179530+0000","last_change":"2024-04-17T09:56:00.179530+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179530+0000","last_undegraded":"2024-04-17T09:56:00.179530+0000","last_fullsized":"2024-04-17T09:56:00.179530+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203408+0000","last_change":"2024-04-17T09:56:01.206115+0000","last_active":"2024-04-17T09:56:03.203408+0000","last_peered":"2024-04-17T09:56:03.203408+0000","last_clean":"2024-04-17T09:56:03.203408+0000","last_became_active":"2024-04-17T09:56:01.205818+0000","last_became_peered":"2024-04-17T09:56:01.205818+0000","last_unstale":"2024-04-17T09:56:03.203408+0000","last_undegraded":"2024-04-17T09:56:03.203408+0000","last_fullsized":"2024-04-17T09:56:03.203408+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:53.672442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197353+0000","last_change":"2024-04-17T09:56:01.206766+0000","last_active":"2024-04-17T09:56:03.197353+0000","last_peered":"2024-04-17T09:56:03.197353+0000","last_clean":"2024-04-17T09:56:03.197353+0000","last_became_active":"2024-04-17T09:56:01.206438+0000","last_became_peered":"2024-04-17T09:56:01.206438+0000","last_unstale":"2024-04-17T09:56:03.197353+0000","last_undegraded":"2024-04-17T09:56:03.197353+0000","last_fullsized":"2024-04-17T09:56:03.197353+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:40:41.135595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179664+0000","last_change":"2024-04-17T09:56:00.179664+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179664+0000","last_undegraded":"2024-04-17T09:56:00.179664+0000","last_fullsized":"2024-04-17T09:56:00.179664+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203390+0000","last_change":"2024-04-17T09:56:01.197166+0000","last_active":"2024-04-17T09:56:03.203390+0000","last_peered":"2024-04-17T09:56:03.203390+0000","last_clean":"2024-04-17T09:56:03.203390+0000","last_became_active":"2024-04-17T09:56:01.196416+0000","last_became_peered":"2024-04-17T09:56:01.196416+0000","last_unstale":"2024-04-17T09:56:03.203390+0000","last_undegraded":"2024-04-17T09:56:03.203390+0000","last_fullsized":"2024-04-17T09:56:03.203390+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:58:47.664590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203664+0000","last_change":"2024-04-17T09:56:02.222098+0000","last_active":"2024-04-17T09:56:03.203664+0000","last_peered":"2024-04-17T09:56:03.203664+0000","last_clean":"2024-04-17T09:56:03.203664+0000","last_became_active":"2024-04-17T09:56:02.221642+0000","last_became_peered":"2024-04-17T09:56:02.221642+0000","last_unstale":"2024-04-17T09:56:03.203664+0000","last_undegraded":"2024-04-17T09:56:03.203664+0000","last_fullsized":"2024-04-17T09:56:03.203664+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:02:18.696690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.184943+0000","last_change":"2024-04-17T09:56:00.184943+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.184943+0000","last_undegraded":"2024-04-17T09:56:00.184943+0000","last_fullsized":"2024-04-17T09:56:00.184943+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.184736+0000","last_change":"2024-04-17T09:56:00.184736+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.184736+0000","last_undegraded":"2024-04-17T09:56:00.184736+0000","last_fullsized":"2024-04-17T09:56:00.184736+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.184535+0000","last_change":"2024-04-17T09:56:00.184535+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.184535+0000","last_undegraded":"2024-04-17T09:56:00.184535+0000","last_fullsized":"2024-04-17T09:56:00.184535+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"56'6","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203761+0000","last_change":"2024-04-17T09:56:01.206040+0000","last_active":"2024-04-17T09:56:03.203761+0000","last_peered":"2024-04-17T09:56:03.203761+0000","last_clean":"2024-04-17T09:56:03.203761+0000","last_became_active":"2024-04-17T09:56:01.205744+0000","last_became_peered":"2024-04-17T09:56:01.205744+0000","last_unstale":"2024-04-17T09:56:03.203761+0000","last_undegraded":"2024-04-17T09:56:03.203761+0000","last_fullsized":"2024-04-17T09:56:03.203761+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:55:07.841245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.184703+0000","last_change":"2024-04-17T09:56:00.184703+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.184703+0000","last_undegraded":"2024-04-17T09:56:00.184703+0000","last_fullsized":"2024-04-17T09:56:00.184703+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179797+0000","last_change":"2024-04-17T09:56:00.179797+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179797+0000","last_undegraded":"2024-04-17T09:56:00.179797+0000","last_fullsized":"2024-04-17T09:56:00.179797+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203271+0000","last_change":"2024-04-17T09:56:02.210009+0000","last_active":"2024-04-17T09:56:03.203271+0000","last_peered":"2024-04-17T09:56:03.203271+0000","last_clean":"2024-04-17T09:56:03.203271+0000","last_became_active":"2024-04-17T09:56:02.209655+0000","last_became_peered":"2024-04-17T09:56:02.209655+0000","last_unstale":"2024-04-17T09:56:03.203271+0000","last_undegraded":"2024-04-17T09:56:03.203271+0000","last_fullsized":"2024-04-17T09:56:03.203271+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:23.603548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179043+0000","last_change":"2024-04-17T09:56:00.179043+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179043+0000","last_undegraded":"2024-04-17T09:56:00.179043+0000","last_fullsized":"2024-04-17T09:56:00.179043+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.183970+0000","last_change":"2024-04-17T09:56:00.183970+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.183970+0000","last_undegraded":"2024-04-17T09:56:00.183970+0000","last_fullsized":"2024-04-17T09:56:00.183970+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.185476+0000","last_change":"2024-04-17T09:56:00.185476+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.185476+0000","last_undegraded":"2024-04-17T09:56:00.185476+0000","last_fullsized":"2024-04-17T09:56:00.185476+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197845+0000","last_change":"2024-04-17T09:56:01.208863+0000","last_active":"2024-04-17T09:56:03.197845+0000","last_peered":"2024-04-17T09:56:03.197845+0000","last_clean":"2024-04-17T09:56:03.197845+0000","last_became_active":"2024-04-17T09:56:01.208605+0000","last_became_peered":"2024-04-17T09:56:01.208605+0000","last_unstale":"2024-04-17T09:56:03.197845+0000","last_undegraded":"2024-04-17T09:56:03.197845+0000","last_fullsized":"2024-04-17T09:56:03.197845+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:36.056444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179120+0000","last_change":"2024-04-17T09:56:00.179120+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179120+0000","last_undegraded":"2024-04-17T09:56:00.179120+0000","last_fullsized":"2024-04-17T09:56:00.179120+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197523+0000","last_change":"2024-04-17T09:56:01.203071+0000","last_active":"2024-04-17T09:56:03.197523+0000","last_peered":"2024-04-17T09:56:03.197523+0000","last_clean":"2024-04-17T09:56:03.197523+0000","last_became_active":"2024-04-17T09:56:01.202764+0000","last_became_peered":"2024-04-17T09:56:01.202764+0000","last_unstale":"2024-04-17T09:56:03.197523+0000","last_undegraded":"2024-04-17T09:56:03.197523+0000","last_fullsized":"2024-04-17T09:56:03.197523+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:22.597577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.184285+0000","last_change":"2024-04-17T09:56:00.184285+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.184285+0000","last_undegraded":"2024-04-17T09:56:00.184285+0000","last_fullsized":"2024-04-17T09:56:00.184285+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197598+0000","last_change":"2024-04-17T09:56:02.211557+0000","last_active":"2024-04-17T09:56:03.197598+0000","last_peered":"2024-04-17T09:56:03.197598+0000","last_clean":"2024-04-17T09:56:03.197598+0000","last_became_active":"2024-04-17T09:56:02.211186+0000","last_became_peered":"2024-04-17T09:56:02.211186+0000","last_unstale":"2024-04-17T09:56:03.197598+0000","last_undegraded":"2024-04-17T09:56:03.197598+0000","last_fullsized":"2024-04-17T09:56:03.197598+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:42:46.838421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179612+0000","last_change":"2024-04-17T09:56:00.179612+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179612+0000","last_undegraded":"2024-04-17T09:56:00.179612+0000","last_fullsized":"2024-04-17T09:56:00.179612+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197057+0000","last_change":"2024-04-17T09:56:02.211632+0000","last_active":"2024-04-17T09:56:03.197057+0000","last_peered":"2024-04-17T09:56:03.197057+0000","last_clean":"2024-04-17T09:56:03.197057+0000","last_became_active":"2024-04-17T09:56:02.211357+0000","last_became_peered":"2024-04-17T09:56:02.211357+0000","last_unstale":"2024-04-17T09:56:03.197057+0000","last_undegraded":"2024-04-17T09:56:03.197057+0000","last_fullsized":"2024-04-17T09:56:03.197057+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:16:53.336176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179313+0000","last_change":"2024-04-17T09:56:00.179313+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179313+0000","last_undegraded":"2024-04-17T09:56:00.179313+0000","last_fullsized":"2024-04-17T09:56:00.179313+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179682+0000","last_change":"2024-04-17T09:56:00.179682+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179682+0000","last_undegraded":"2024-04-17T09:56:00.179682+0000","last_fullsized":"2024-04-17T09:56:00.179682+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"57'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203049+0000","last_change":"2024-04-17T09:56:01.202823+0000","last_active":"2024-04-17T09:56:03.203049+0000","last_peered":"2024-04-17T09:56:03.203049+0000","last_clean":"2024-04-17T09:56:03.203049+0000","last_became_active":"2024-04-17T09:56:01.202563+0000","last_became_peered":"2024-04-17T09:56:01.202563+0000","last_unstale":"2024-04-17T09:56:03.203049+0000","last_undegraded":"2024-04-17T09:56:03.203049+0000","last_fullsized":"2024-04-17T09:56:03.203049+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:34:57.389627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.180425+0000","last_change":"2024-04-17T09:56:00.180425+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.180425+0000","last_undegraded":"2024-04-17T09:56:00.180425+0000","last_fullsized":"2024-04-17T09:56:00.180425+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197782+0000","last_change":"2024-04-17T09:56:02.230708+0000","last_active":"2024-04-17T09:56:03.197782+0000","last_peered":"2024-04-17T09:56:03.197782+0000","last_clean":"2024-04-17T09:56:03.197782+0000","last_became_active":"2024-04-17T09:56:02.230207+0000","last_became_peered":"2024-04-17T09:56:02.230207+0000","last_unstale":"2024-04-17T09:56:03.197782+0000","last_undegraded":"2024-04-17T09:56:03.197782+0000","last_fullsized":"2024-04-17T09:56:03.197782+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:43:27.595946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179054+0000","last_change":"2024-04-17T09:56:00.179054+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179054+0000","last_undegraded":"2024-04-17T09:56:00.179054+0000","last_fullsized":"2024-04-17T09:56:00.179054+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179594+0000","last_change":"2024-04-17T09:56:00.179594+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179594+0000","last_undegraded":"2024-04-17T09:56:00.179594+0000","last_fullsized":"2024-04-17T09:56:00.179594+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203268+0000","last_change":"2024-04-17T09:56:01.207103+0000","last_active":"2024-04-17T09:56:03.203268+0000","last_peered":"2024-04-17T09:56:03.203268+0000","last_clean":"2024-04-17T09:56:03.203268+0000","last_became_active":"2024-04-17T09:56:01.206837+0000","last_became_peered":"2024-04-17T09:56:01.206837+0000","last_unstale":"2024-04-17T09:56:03.203268+0000","last_undegraded":"2024-04-17T09:56:03.203268+0000","last_fullsized":"2024-04-17T09:56:03.203268+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:28:36.102136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179601+0000","last_change":"2024-04-17T09:56:00.179601+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179601+0000","last_undegraded":"2024-04-17T09:56:00.179601+0000","last_fullsized":"2024-04-17T09:56:00.179601+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.184252+0000","last_change":"2024-04-17T09:56:00.184252+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.184252+0000","last_undegraded":"2024-04-17T09:56:00.184252+0000","last_fullsized":"2024-04-17T09:56:00.184252+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.180271+0000","last_change":"2024-04-17T09:56:00.180271+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.180271+0000","last_undegraded":"2024-04-17T09:56:00.180271+0000","last_fullsized":"2024-04-17T09:56:00.180271+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":231,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-17T09:56:00.184271+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T09:56:00.184271+0000","last_peered":"2024-04-17T09:56:00.184271+0000","last_clean":"2024-04-17T09:56:00.184271+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T09:56:00.184271+0000","last_undegraded":"2024-04-17T09:56:00.184271+0000","last_fullsized":"2024-04-17T09:56:00.184271+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":2},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2194,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1220608,"data_stored":844769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":626688,"data_stored":254401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":254401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31796,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340044,"statfs":{"total":96636764160,"available":96604205056,"internally_reserved":0,"allocated":626688,"data_stored":254401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31804,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340036,"statfs":{"total":96636764160,"available":96604196864,"internally_reserved":0,"allocated":638976,"data_stored":260218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31792,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340048,"statfs":{"total":96636764160,"available":96604209152,"internally_reserved":0,"allocated":626688,"data_stored":254401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":688128,"data_stored":281172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542184,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":872122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T09:56:06.332 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T09:56:06.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.512+0000 7f65ba400640 1 Processor -- start 2024-04-17T09:56:06.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.512+0000 7f65ba400640 1 -- start start 2024-04-17T09:56:06.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.512+0000 7f65ba400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65b4145f30 0x7f65b4071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:06.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.512+0000 7f65ba400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f65b415d8d0 0x7f65b4072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:06.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.512+0000 7f65ba400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65b40cdac0 con 0x7f65b40cce70 2024-04-17T09:56:06.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.512+0000 7f65ba400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f65b406f400 con 0x7f65b415d8d0 2024-04-17T09:56:06.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.512+0000 7f65ba400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f65b4146310 con 0x7f65b4145f30 2024-04-17T09:56:06.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.512+0000 7f65abe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f65b415d8d0 0x7f65b4072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:06.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.512+0000 7f65b3e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f65b40cce70 0x7f65b40cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58420/0 (socket says 172.21.15.32:58420) 2024-04-17T09:56:06.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.512+0000 7f65b3e00640 1 -- 172.21.15.32:0/2979119342 learned_addr learned my addr 172.21.15.32:0/2979119342 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:06.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.512+0000 7f65b3400640 1 --2- 172.21.15.32:0/2979119342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65b4145f30 0x7f65b4071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:06.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.512+0000 7f65abe00640 1 -- 172.21.15.32:0/2979119342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65b4145f30 msgr2=0x7f65b4071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:06.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.512+0000 7f65abe00640 1 --2- 172.21.15.32:0/2979119342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65b4145f30 0x7f65b4071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.512+0000 7f65abe00640 1 -- 172.21.15.32:0/2979119342 >> v1:172.21.15.32:6789/0 conn(0x7f65b40cce70 legacy=0x7f65b40cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:06.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.512+0000 7f65abe00640 1 -- 172.21.15.32:0/2979119342 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b415cdf0 con 0x7f65b415d8d0 2024-04-17T09:56:06.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.512+0000 7f65b2a00640 1 -- 172.21.15.32:0/2979119342 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1865488020 0 0) 0x7f65b40cdac0 con 0x7f65b40cce70 2024-04-17T09:56:06.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.516+0000 7f65abe00640 1 --2- 172.21.15.32:0/2979119342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b415d8d0 0x7f65b4072330 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f659c0097c0 tx=0x7f659c01ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4902bc2f811805b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:06.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.516+0000 7f65b2a00640 1 -- 172.21.15.32:0/2979119342 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c0b5070 con 0x7f65b415d8d0 2024-04-17T09:56:06.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.516+0000 7f65b2a00640 1 -- 172.21.15.32:0/2979119342 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f659c0a8990 con 0x7f65b415d8d0 2024-04-17T09:56:06.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.516+0000 7f65b2a00640 1 -- 172.21.15.32:0/2979119342 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c0a8d60 con 0x7f65b415d8d0 2024-04-17T09:56:06.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.516+0000 7f65ba400640 1 -- 172.21.15.32:0/2979119342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b415d8d0 msgr2=0x7f65b4072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:06.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.516+0000 7f65ba400640 1 --2- 172.21.15.32:0/2979119342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b415d8d0 0x7f65b4072330 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f659c0097c0 tx=0x7f659c01ec90 comp rx=0 tx=0).stop 2024-04-17T09:56:06.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.516+0000 7f65ba400640 1 -- 172.21.15.32:0/2979119342 shutdown_connections 2024-04-17T09:56:06.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.516+0000 7f65ba400640 1 --2- 172.21.15.32:0/2979119342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b415d8d0 0x7f65b4072330 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.516+0000 7f65ba400640 1 --2- 172.21.15.32:0/2979119342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65b4145f30 0x7f65b4071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.516+0000 7f65ba400640 1 -- 172.21.15.32:0/2979119342 >> 172.21.15.32:0/2979119342 conn(0x7f65b406ba30 msgr2=0x7f65b406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:06.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.516+0000 7f65ba400640 1 -- 172.21.15.32:0/2979119342 shutdown_connections 2024-04-17T09:56:06.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.516+0000 7f65ba400640 1 -- 172.21.15.32:0/2979119342 wait complete. 2024-04-17T09:56:06.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65ba400640 1 Processor -- start 2024-04-17T09:56:06.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65ba400640 1 -- start start 2024-04-17T09:56:06.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65ba400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65b40cce70 0x7f65b4142df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:06.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65ba400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65b4145f30 0x7f65b415c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:06.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65ba400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b415d8d0 0x7f65b41e7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:06.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65ba400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f65b415cb90 con 0x7f65b415d8d0 2024-04-17T09:56:06.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65ba400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65b415d440 con 0x7f65b40cce70 2024-04-17T09:56:06.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65ba400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f65b41601d0 con 0x7f65b4145f30 2024-04-17T09:56:06.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65b3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b415d8d0 0x7f65b41e7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:06.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65b3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b415d8d0 0x7f65b41e7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49260/0 (socket says 172.21.15.32:49260) 2024-04-17T09:56:06.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65b3e00640 1 -- 172.21.15.32:0/2491505459 learned_addr learned my addr 172.21.15.32:0/2491505459 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:06.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65abe00640 1 --2- 172.21.15.32:0/2491505459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65b4145f30 0x7f65b415c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:06.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65b3400640 1 --2- 172.21.15.32:0/2491505459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65b40cce70 0x7f65b4142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:06.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65b3e00640 1 -- 172.21.15.32:0/2491505459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65b4145f30 msgr2=0x7f65b415c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:06.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65b3e00640 1 --2- 172.21.15.32:0/2491505459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65b4145f30 0x7f65b415c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65b3e00640 1 -- 172.21.15.32:0/2491505459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65b40cce70 msgr2=0x7f65b4142df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:06.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65b3e00640 1 --2- 172.21.15.32:0/2491505459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65b40cce70 0x7f65b4142df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65b3e00640 1 -- 172.21.15.32:0/2491505459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b4144850 con 0x7f65b415d8d0 2024-04-17T09:56:06.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65b3e00640 1 --2- 172.21.15.32:0/2491505459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b415d8d0 0x7f65b41e7f60 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00c7a0 tx=0x7f65ac00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:06.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65ab400640 1 -- 172.21.15.32:0/2491505459 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac018070 con 0x7f65b415d8d0 2024-04-17T09:56:06.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65ab400640 1 -- 172.21.15.32:0/2491505459 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65ac00a640 con 0x7f65b415d8d0 2024-04-17T09:56:06.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65ab400640 1 -- 172.21.15.32:0/2491505459 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac00aab0 con 0x7f65b415d8d0 2024-04-17T09:56:06.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65ba400640 1 -- 172.21.15.32:0/2491505459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b4142310 con 0x7f65b415d8d0 2024-04-17T09:56:06.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65ba400640 1 -- 172.21.15.32:0/2491505459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b41428d0 con 0x7f65b415d8d0 2024-04-17T09:56:06.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.520+0000 7f65ba400640 1 -- 172.21.15.32:0/2491505459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b4068a90 con 0x7f65b415d8d0 2024-04-17T09:56:06.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.524+0000 7f65ab400640 1 -- 172.21.15.32:0/2491505459 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f65ac00ac50 con 0x7f65b415d8d0 2024-04-17T09:56:06.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.524+0000 7f65ab400640 1 --2- 172.21.15.32:0/2491505459 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6590068380 0x7f659006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:06.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.524+0000 7f65ab400640 1 -- 172.21.15.32:0/2491505459 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f65ac085ee0 con 0x7f65b415d8d0 2024-04-17T09:56:06.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.524+0000 7f65b3400640 1 --2- 172.21.15.32:0/2491505459 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6590068380 0x7f659006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:06.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.524+0000 7f65b3400640 1 --2- 172.21.15.32:0/2491505459 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6590068380 0x7f659006a840 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009b60 tx=0x7f65a4009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:06.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.528+0000 7f65ab400640 1 -- 172.21.15.32:0/2491505459 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f65ac055ee0 con 0x7f65b415d8d0 2024-04-17T09:56:06.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.692+0000 7f65ba400640 1 -- 172.21.15.32:0/2491505459 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f65b4144b30 con 0x7f6590068380 2024-04-17T09:56:06.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.696+0000 7f65ab400640 1 -- 172.21.15.32:0/2491505459 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+243016 (secure 0 0 0) 0x7f65b4144b30 con 0x7f6590068380 2024-04-17T09:56:06.701 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:06.703 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T09:56:06.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.700+0000 7f65ba400640 1 -- 172.21.15.32:0/2491505459 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6590068380 msgr2=0x7f659006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:06.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.700+0000 7f65ba400640 1 --2- 172.21.15.32:0/2491505459 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6590068380 0x7f659006a840 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009b60 tx=0x7f65a4009190 comp rx=0 tx=0).stop 2024-04-17T09:56:06.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.700+0000 7f65ba400640 1 -- 172.21.15.32:0/2491505459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b415d8d0 msgr2=0x7f65b41e7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:06.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.700+0000 7f65ba400640 1 --2- 172.21.15.32:0/2491505459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b415d8d0 0x7f65b41e7f60 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00c7a0 tx=0x7f65ac00cc60 comp rx=0 tx=0).stop 2024-04-17T09:56:06.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.700+0000 7f65ba400640 1 -- 172.21.15.32:0/2491505459 shutdown_connections 2024-04-17T09:56:06.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.700+0000 7f65ba400640 1 --2- 172.21.15.32:0/2491505459 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6590068380 0x7f659006a840 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.700+0000 7f65ba400640 1 --2- 172.21.15.32:0/2491505459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65b4145f30 0x7f65b415c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.700+0000 7f65ba400640 1 --2- 172.21.15.32:0/2491505459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65b415d8d0 0x7f65b41e7f60 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.700+0000 7f65ba400640 1 --2- 172.21.15.32:0/2491505459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65b40cce70 0x7f65b4142df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.700+0000 7f65ba400640 1 -- 172.21.15.32:0/2491505459 >> 172.21.15.32:0/2491505459 conn(0x7f65b406ba30 msgr2=0x7f65b415dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:06.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.700+0000 7f65ba400640 1 -- 172.21.15.32:0/2491505459 shutdown_connections 2024-04-17T09:56:06.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.700+0000 7f65ba400640 1 -- 172.21.15.32:0/2491505459 wait complete. 2024-04-17T09:56:06.723 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-04-17T09:56:05.321817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592562,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":176,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":98,"acting":98,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":243180,"kb_used_data":6192,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731540,"statfs":{"total":773094113280,"available":772845096960,"internally_reserved":0,"allocated":6340608,"data_stored":3275885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2194,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.145090"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197311+0000","last_change":"2024-04-17T09:56:02.221498+0000","last_active":"2024-04-17T09:56:03.197311+0000","last_peered":"2024-04-17T09:56:03.197311+0000","last_clean":"2024-04-17T09:56:03.197311+0000","last_became_active":"2024-04-17T09:56:02.217279+0000","last_became_peered":"2024-04-17T09:56:02.217279+0000","last_unstale":"2024-04-17T09:56:03.197311+0000","last_undegraded":"2024-04-17T09:56:03.197311+0000","last_fullsized":"2024-04-17T09:56:03.197311+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:53.818908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203103+0000","last_change":"2024-04-17T09:56:02.220843+0000","last_active":"2024-04-17T09:56:03.203103+0000","last_peered":"2024-04-17T09:56:03.203103+0000","last_clean":"2024-04-17T09:56:03.203103+0000","last_became_active":"2024-04-17T09:56:02.220566+0000","last_became_peered":"2024-04-17T09:56:02.220566+0000","last_unstale":"2024-04-17T09:56:03.203103+0000","last_undegraded":"2024-04-17T09:56:03.203103+0000","last_fullsized":"2024-04-17T09:56:03.203103+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:24:38.324835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203578+0000","last_change":"2024-04-17T09:56:02.230219+0000","last_active":"2024-04-17T09:56:03.203578+0000","last_peered":"2024-04-17T09:56:03.203578+0000","last_clean":"2024-04-17T09:56:03.203578+0000","last_became_active":"2024-04-17T09:56:02.229840+0000","last_became_peered":"2024-04-17T09:56:02.229840+0000","last_unstale":"2024-04-17T09:56:03.203578+0000","last_undegraded":"2024-04-17T09:56:03.203578+0000","last_fullsized":"2024-04-17T09:56:03.203578+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:14:09.222091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203987+0000","last_change":"2024-04-17T09:56:02.213294+0000","last_active":"2024-04-17T09:56:03.203987+0000","last_peered":"2024-04-17T09:56:03.203987+0000","last_clean":"2024-04-17T09:56:03.203987+0000","last_became_active":"2024-04-17T09:56:02.212904+0000","last_became_peered":"2024-04-17T09:56:02.212904+0000","last_unstale":"2024-04-17T09:56:03.203987+0000","last_undegraded":"2024-04-17T09:56:03.203987+0000","last_fullsized":"2024-04-17T09:56:03.203987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:41:23.655282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203242+0000","last_change":"2024-04-17T09:56:02.222178+0000","last_active":"2024-04-17T09:56:03.203242+0000","last_peered":"2024-04-17T09:56:03.203242+0000","last_clean":"2024-04-17T09:56:03.203242+0000","last_became_active":"2024-04-17T09:56:02.221835+0000","last_became_peered":"2024-04-17T09:56:02.221835+0000","last_unstale":"2024-04-17T09:56:03.203242+0000","last_undegraded":"2024-04-17T09:56:03.203242+0000","last_fullsized":"2024-04-17T09:56:03.203242+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:40:42.141566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197006+0000","last_change":"2024-04-17T09:56:02.215969+0000","last_active":"2024-04-17T09:56:03.197006+0000","last_peered":"2024-04-17T09:56:03.197006+0000","last_clean":"2024-04-17T09:56:03.197006+0000","last_became_active":"2024-04-17T09:56:02.215670+0000","last_became_peered":"2024-04-17T09:56:02.215670+0000","last_unstale":"2024-04-17T09:56:03.197006+0000","last_undegraded":"2024-04-17T09:56:03.197006+0000","last_fullsized":"2024-04-17T09:56:03.197006+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:01:34.123890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197587+0000","last_change":"2024-04-17T09:56:02.215021+0000","last_active":"2024-04-17T09:56:03.197587+0000","last_peered":"2024-04-17T09:56:03.197587+0000","last_clean":"2024-04-17T09:56:03.197587+0000","last_became_active":"2024-04-17T09:56:02.214653+0000","last_became_peered":"2024-04-17T09:56:02.214653+0000","last_unstale":"2024-04-17T09:56:03.197587+0000","last_undegraded":"2024-04-17T09:56:03.197587+0000","last_fullsized":"2024-04-17T09:56:03.197587+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:00:41.794977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197005+0000","last_change":"2024-04-17T09:56:02.225982+0000","last_active":"2024-04-17T09:56:03.197005+0000","last_peered":"2024-04-17T09:56:03.197005+0000","last_clean":"2024-04-17T09:56:03.197005+0000","last_became_active":"2024-04-17T09:56:02.223371+0000","last_became_peered":"2024-04-17T09:56:02.223371+0000","last_unstale":"2024-04-17T09:56:03.197005+0000","last_undegraded":"2024-04-17T09:56:03.197005+0000","last_fullsized":"2024-04-17T09:56:03.197005+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:48:17.886969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197834+0000","last_change":"2024-04-17T09:56:02.224842+0000","last_active":"2024-04-17T09:56:03.197834+0000","last_peered":"2024-04-17T09:56:03.197834+0000","last_clean":"2024-04-17T09:56:03.197834+0000","last_became_active":"2024-04-17T09:56:02.224545+0000","last_became_peered":"2024-04-17T09:56:02.224545+0000","last_unstale":"2024-04-17T09:56:03.197834+0000","last_undegraded":"2024-04-17T09:56:03.197834+0000","last_fullsized":"2024-04-17T09:56:03.197834+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:57.566831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"56'4","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197875+0000","last_change":"2024-04-17T09:56:01.206168+0000","last_active":"2024-04-17T09:56:03.197875+0000","last_peered":"2024-04-17T09:56:03.197875+0000","last_clean":"2024-04-17T09:56:03.197875+0000","last_became_active":"2024-04-17T09:56:01.204386+0000","last_became_peered":"2024-04-17T09:56:01.204386+0000","last_unstale":"2024-04-17T09:56:03.197875+0000","last_undegraded":"2024-04-17T09:56:03.197875+0000","last_fullsized":"2024-04-17T09:56:03.197875+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:34.492070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.185133+0000","last_change":"2024-04-17T09:56:00.185133+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.185133+0000","last_undegraded":"2024-04-17T09:56:00.185133+0000","last_fullsized":"2024-04-17T09:56:00.185133+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197603+0000","last_change":"2024-04-17T09:56:02.234199+0000","last_active":"2024-04-17T09:56:03.197603+0000","last_peered":"2024-04-17T09:56:03.197603+0000","last_clean":"2024-04-17T09:56:03.197603+0000","last_became_active":"2024-04-17T09:56:02.233686+0000","last_became_peered":"2024-04-17T09:56:02.233686+0000","last_unstale":"2024-04-17T09:56:03.197603+0000","last_undegraded":"2024-04-17T09:56:03.197603+0000","last_fullsized":"2024-04-17T09:56:03.197603+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:08.333973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197902+0000","last_change":"2024-04-17T09:56:01.201154+0000","last_active":"2024-04-17T09:56:03.197902+0000","last_peered":"2024-04-17T09:56:03.197902+0000","last_clean":"2024-04-17T09:56:03.197902+0000","last_became_active":"2024-04-17T09:56:01.200926+0000","last_became_peered":"2024-04-17T09:56:01.200926+0000","last_unstale":"2024-04-17T09:56:03.197902+0000","last_undegraded":"2024-04-17T09:56:03.197902+0000","last_fullsized":"2024-04-17T09:56:03.197902+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:41:22.649311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197793+0000","last_change":"2024-04-17T09:56:02.230469+0000","last_active":"2024-04-17T09:56:03.197793+0000","last_peered":"2024-04-17T09:56:03.197793+0000","last_clean":"2024-04-17T09:56:03.197793+0000","last_became_active":"2024-04-17T09:56:02.229210+0000","last_became_peered":"2024-04-17T09:56:02.229210+0000","last_unstale":"2024-04-17T09:56:03.197793+0000","last_undegraded":"2024-04-17T09:56:03.197793+0000","last_fullsized":"2024-04-17T09:56:03.197793+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:58:25.393385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179530+0000","last_change":"2024-04-17T09:56:00.179530+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179530+0000","last_undegraded":"2024-04-17T09:56:00.179530+0000","last_fullsized":"2024-04-17T09:56:00.179530+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203408+0000","last_change":"2024-04-17T09:56:01.206115+0000","last_active":"2024-04-17T09:56:03.203408+0000","last_peered":"2024-04-17T09:56:03.203408+0000","last_clean":"2024-04-17T09:56:03.203408+0000","last_became_active":"2024-04-17T09:56:01.205818+0000","last_became_peered":"2024-04-17T09:56:01.205818+0000","last_unstale":"2024-04-17T09:56:03.203408+0000","last_undegraded":"2024-04-17T09:56:03.203408+0000","last_fullsized":"2024-04-17T09:56:03.203408+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:53.672442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197353+0000","last_change":"2024-04-17T09:56:01.206766+0000","last_active":"2024-04-17T09:56:03.197353+0000","last_peered":"2024-04-17T09:56:03.197353+0000","last_clean":"2024-04-17T09:56:03.197353+0000","last_became_active":"2024-04-17T09:56:01.206438+0000","last_became_peered":"2024-04-17T09:56:01.206438+0000","last_unstale":"2024-04-17T09:56:03.197353+0000","last_undegraded":"2024-04-17T09:56:03.197353+0000","last_fullsized":"2024-04-17T09:56:03.197353+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:40:41.135595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179664+0000","last_change":"2024-04-17T09:56:00.179664+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179664+0000","last_undegraded":"2024-04-17T09:56:00.179664+0000","last_fullsized":"2024-04-17T09:56:00.179664+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203390+0000","last_change":"2024-04-17T09:56:01.197166+0000","last_active":"2024-04-17T09:56:03.203390+0000","last_peered":"2024-04-17T09:56:03.203390+0000","last_clean":"2024-04-17T09:56:03.203390+0000","last_became_active":"2024-04-17T09:56:01.196416+0000","last_became_peered":"2024-04-17T09:56:01.196416+0000","last_unstale":"2024-04-17T09:56:03.203390+0000","last_undegraded":"2024-04-17T09:56:03.203390+0000","last_fullsized":"2024-04-17T09:56:03.203390+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:58:47.664590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203664+0000","last_change":"2024-04-17T09:56:02.222098+0000","last_active":"2024-04-17T09:56:03.203664+0000","last_peered":"2024-04-17T09:56:03.203664+0000","last_clean":"2024-04-17T09:56:03.203664+0000","last_became_active":"2024-04-17T09:56:02.221642+0000","last_became_peered":"2024-04-17T09:56:02.221642+0000","last_unstale":"2024-04-17T09:56:03.203664+0000","last_undegraded":"2024-04-17T09:56:03.203664+0000","last_fullsized":"2024-04-17T09:56:03.203664+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:02:18.696690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.184943+0000","last_change":"2024-04-17T09:56:00.184943+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.184943+0000","last_undegraded":"2024-04-17T09:56:00.184943+0000","last_fullsized":"2024-04-17T09:56:00.184943+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.184736+0000","last_change":"2024-04-17T09:56:00.184736+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.184736+0000","last_undegraded":"2024-04-17T09:56:00.184736+0000","last_fullsized":"2024-04-17T09:56:00.184736+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.184535+0000","last_change":"2024-04-17T09:56:00.184535+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.184535+0000","last_undegraded":"2024-04-17T09:56:00.184535+0000","last_fullsized":"2024-04-17T09:56:00.184535+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"56'6","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203761+0000","last_change":"2024-04-17T09:56:01.206040+0000","last_active":"2024-04-17T09:56:03.203761+0000","last_peered":"2024-04-17T09:56:03.203761+0000","last_clean":"2024-04-17T09:56:03.203761+0000","last_became_active":"2024-04-17T09:56:01.205744+0000","last_became_peered":"2024-04-17T09:56:01.205744+0000","last_unstale":"2024-04-17T09:56:03.203761+0000","last_undegraded":"2024-04-17T09:56:03.203761+0000","last_fullsized":"2024-04-17T09:56:03.203761+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:55:07.841245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.184703+0000","last_change":"2024-04-17T09:56:00.184703+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.184703+0000","last_undegraded":"2024-04-17T09:56:00.184703+0000","last_fullsized":"2024-04-17T09:56:00.184703+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179797+0000","last_change":"2024-04-17T09:56:00.179797+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179797+0000","last_undegraded":"2024-04-17T09:56:00.179797+0000","last_fullsized":"2024-04-17T09:56:00.179797+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203271+0000","last_change":"2024-04-17T09:56:02.210009+0000","last_active":"2024-04-17T09:56:03.203271+0000","last_peered":"2024-04-17T09:56:03.203271+0000","last_clean":"2024-04-17T09:56:03.203271+0000","last_became_active":"2024-04-17T09:56:02.209655+0000","last_became_peered":"2024-04-17T09:56:02.209655+0000","last_unstale":"2024-04-17T09:56:03.203271+0000","last_undegraded":"2024-04-17T09:56:03.203271+0000","last_fullsized":"2024-04-17T09:56:03.203271+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:23.603548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179043+0000","last_change":"2024-04-17T09:56:00.179043+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179043+0000","last_undegraded":"2024-04-17T09:56:00.179043+0000","last_fullsized":"2024-04-17T09:56:00.179043+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.183970+0000","last_change":"2024-04-17T09:56:00.183970+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.183970+0000","last_undegraded":"2024-04-17T09:56:00.183970+0000","last_fullsized":"2024-04-17T09:56:00.183970+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.185476+0000","last_change":"2024-04-17T09:56:00.185476+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.185476+0000","last_undegraded":"2024-04-17T09:56:00.185476+0000","last_fullsized":"2024-04-17T09:56:00.185476+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197845+0000","last_change":"2024-04-17T09:56:01.208863+0000","last_active":"2024-04-17T09:56:03.197845+0000","last_peered":"2024-04-17T09:56:03.197845+0000","last_clean":"2024-04-17T09:56:03.197845+0000","last_became_active":"2024-04-17T09:56:01.208605+0000","last_became_peered":"2024-04-17T09:56:01.208605+0000","last_unstale":"2024-04-17T09:56:03.197845+0000","last_undegraded":"2024-04-17T09:56:03.197845+0000","last_fullsized":"2024-04-17T09:56:03.197845+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:36.056444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179120+0000","last_change":"2024-04-17T09:56:00.179120+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179120+0000","last_undegraded":"2024-04-17T09:56:00.179120+0000","last_fullsized":"2024-04-17T09:56:00.179120+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197523+0000","last_change":"2024-04-17T09:56:01.203071+0000","last_active":"2024-04-17T09:56:03.197523+0000","last_peered":"2024-04-17T09:56:03.197523+0000","last_clean":"2024-04-17T09:56:03.197523+0000","last_became_active":"2024-04-17T09:56:01.202764+0000","last_became_peered":"2024-04-17T09:56:01.202764+0000","last_unstale":"2024-04-17T09:56:03.197523+0000","last_undegraded":"2024-04-17T09:56:03.197523+0000","last_fullsized":"2024-04-17T09:56:03.197523+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:22.597577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.184285+0000","last_change":"2024-04-17T09:56:00.184285+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.184285+0000","last_undegraded":"2024-04-17T09:56:00.184285+0000","last_fullsized":"2024-04-17T09:56:00.184285+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197598+0000","last_change":"2024-04-17T09:56:02.211557+0000","last_active":"2024-04-17T09:56:03.197598+0000","last_peered":"2024-04-17T09:56:03.197598+0000","last_clean":"2024-04-17T09:56:03.197598+0000","last_became_active":"2024-04-17T09:56:02.211186+0000","last_became_peered":"2024-04-17T09:56:02.211186+0000","last_unstale":"2024-04-17T09:56:03.197598+0000","last_undegraded":"2024-04-17T09:56:03.197598+0000","last_fullsized":"2024-04-17T09:56:03.197598+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:42:46.838421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179612+0000","last_change":"2024-04-17T09:56:00.179612+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179612+0000","last_undegraded":"2024-04-17T09:56:00.179612+0000","last_fullsized":"2024-04-17T09:56:00.179612+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197057+0000","last_change":"2024-04-17T09:56:02.211632+0000","last_active":"2024-04-17T09:56:03.197057+0000","last_peered":"2024-04-17T09:56:03.197057+0000","last_clean":"2024-04-17T09:56:03.197057+0000","last_became_active":"2024-04-17T09:56:02.211357+0000","last_became_peered":"2024-04-17T09:56:02.211357+0000","last_unstale":"2024-04-17T09:56:03.197057+0000","last_undegraded":"2024-04-17T09:56:03.197057+0000","last_fullsized":"2024-04-17T09:56:03.197057+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:16:53.336176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179313+0000","last_change":"2024-04-17T09:56:00.179313+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179313+0000","last_undegraded":"2024-04-17T09:56:00.179313+0000","last_fullsized":"2024-04-17T09:56:00.179313+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179682+0000","last_change":"2024-04-17T09:56:00.179682+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179682+0000","last_undegraded":"2024-04-17T09:56:00.179682+0000","last_fullsized":"2024-04-17T09:56:00.179682+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"57'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203049+0000","last_change":"2024-04-17T09:56:01.202823+0000","last_active":"2024-04-17T09:56:03.203049+0000","last_peered":"2024-04-17T09:56:03.203049+0000","last_clean":"2024-04-17T09:56:03.203049+0000","last_became_active":"2024-04-17T09:56:01.202563+0000","last_became_peered":"2024-04-17T09:56:01.202563+0000","last_unstale":"2024-04-17T09:56:03.203049+0000","last_undegraded":"2024-04-17T09:56:03.203049+0000","last_fullsized":"2024-04-17T09:56:03.203049+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:34:57.389627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.180425+0000","last_change":"2024-04-17T09:56:00.180425+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.180425+0000","last_undegraded":"2024-04-17T09:56:00.180425+0000","last_fullsized":"2024-04-17T09:56:00.180425+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197782+0000","last_change":"2024-04-17T09:56:02.230708+0000","last_active":"2024-04-17T09:56:03.197782+0000","last_peered":"2024-04-17T09:56:03.197782+0000","last_clean":"2024-04-17T09:56:03.197782+0000","last_became_active":"2024-04-17T09:56:02.230207+0000","last_became_peered":"2024-04-17T09:56:02.230207+0000","last_unstale":"2024-04-17T09:56:03.197782+0000","last_undegraded":"2024-04-17T09:56:03.197782+0000","last_fullsized":"2024-04-17T09:56:03.197782+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:43:27.595946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179054+0000","last_change":"2024-04-17T09:56:00.179054+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179054+0000","last_undegraded":"2024-04-17T09:56:00.179054+0000","last_fullsized":"2024-04-17T09:56:00.179054+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179594+0000","last_change":"2024-04-17T09:56:00.179594+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179594+0000","last_undegraded":"2024-04-17T09:56:00.179594+0000","last_fullsized":"2024-04-17T09:56:00.179594+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203268+0000","last_change":"2024-04-17T09:56:01.207103+0000","last_active":"2024-04-17T09:56:03.203268+0000","last_peered":"2024-04-17T09:56:03.203268+0000","last_clean":"2024-04-17T09:56:03.203268+0000","last_became_active":"2024-04-17T09:56:01.206837+0000","last_became_peered":"2024-04-17T09:56:01.206837+0000","last_unstale":"2024-04-17T09:56:03.203268+0000","last_undegraded":"2024-04-17T09:56:03.203268+0000","last_fullsized":"2024-04-17T09:56:03.203268+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:28:36.102136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179601+0000","last_change":"2024-04-17T09:56:00.179601+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179601+0000","last_undegraded":"2024-04-17T09:56:00.179601+0000","last_fullsized":"2024-04-17T09:56:00.179601+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.184252+0000","last_change":"2024-04-17T09:56:00.184252+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.184252+0000","last_undegraded":"2024-04-17T09:56:00.184252+0000","last_fullsized":"2024-04-17T09:56:00.184252+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.180271+0000","last_change":"2024-04-17T09:56:00.180271+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.180271+0000","last_undegraded":"2024-04-17T09:56:00.180271+0000","last_fullsized":"2024-04-17T09:56:00.180271+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":231,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-17T09:56:00.184271+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T09:56:00.184271+0000","last_peered":"2024-04-17T09:56:00.184271+0000","last_clean":"2024-04-17T09:56:00.184271+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T09:56:00.184271+0000","last_undegraded":"2024-04-17T09:56:00.184271+0000","last_fullsized":"2024-04-17T09:56:00.184271+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":2},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2194,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1220608,"data_stored":844769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":626688,"data_stored":254401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":254401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31796,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340044,"statfs":{"total":96636764160,"available":96604205056,"internally_reserved":0,"allocated":626688,"data_stored":254401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31804,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340036,"statfs":{"total":96636764160,"available":96604196864,"internally_reserved":0,"allocated":638976,"data_stored":260218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31792,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340048,"statfs":{"total":96636764160,"available":96604209152,"internally_reserved":0,"allocated":626688,"data_stored":254401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":688128,"data_stored":281172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542184,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":872122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T09:56:06.727 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T09:56:06.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.900+0000 7f055be00640 1 Processor -- start 2024-04-17T09:56:06.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.900+0000 7f055be00640 1 -- start start 2024-04-17T09:56:06.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.904+0000 7f055be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f055415e870 0x7f0554162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:06.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.904+0000 7f055be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05540b0320 0x7f05540d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:06.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.904+0000 7f055be00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0554163340 con 0x7f05540d0780 2024-04-17T09:56:06.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.904+0000 7f055be00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f05540cf810 con 0x7f055415e870 2024-04-17T09:56:06.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.904+0000 7f055be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0554067dd0 con 0x7f05540b0320 2024-04-17T09:56:06.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.904+0000 7f0559400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f055415e870 0x7f0554162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:06.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.904+0000 7f0559400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f055415e870 0x7f0554162e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49262/0 (socket says 172.21.15.32:49262) 2024-04-17T09:56:06.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.904+0000 7f0559400640 1 -- 172.21.15.32:0/2656394807 learned_addr learned my addr 172.21.15.32:0/2656394807 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:06.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.904+0000 7f0558a00640 1 --2- 172.21.15.32:0/2656394807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05540b0320 0x7f05540d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:06.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.904+0000 7f0559400640 1 -- 172.21.15.32:0/2656394807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05540b0320 msgr2=0x7f05540d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:06.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.904+0000 7f0559400640 1 --2- 172.21.15.32:0/2656394807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05540b0320 0x7f05540d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.904+0000 7f0559400640 1 -- 172.21.15.32:0/2656394807 >> v1:172.21.15.144:6789/0 conn(0x7f05540d0780 legacy=0x7f055406f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:06.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.904+0000 7f0559400640 1 -- 172.21.15.32:0/2656394807 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055415ec50 con 0x7f055415e870 2024-04-17T09:56:06.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.908+0000 7f0559400640 1 --2- 172.21.15.32:0/2656394807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f055415e870 0x7f0554162e00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f0548009870 tx=0x7f054801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77c2130cee118b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:06.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.908+0000 7f0553e00640 1 -- 172.21.15.32:0/2656394807 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05480ab650 con 0x7f055415e870 2024-04-17T09:56:06.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.908+0000 7f0553e00640 1 -- 172.21.15.32:0/2656394807 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05480b0070 con 0x7f055415e870 2024-04-17T09:56:06.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.908+0000 7f0553e00640 1 -- 172.21.15.32:0/2656394807 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05480aba10 con 0x7f055415e870 2024-04-17T09:56:06.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.908+0000 7f055be00640 1 -- 172.21.15.32:0/2656394807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f055415e870 msgr2=0x7f0554162e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:06.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.908+0000 7f055be00640 1 --2- 172.21.15.32:0/2656394807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f055415e870 0x7f0554162e00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f0548009870 tx=0x7f054801c560 comp rx=0 tx=0).stop 2024-04-17T09:56:06.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.908+0000 7f055be00640 1 -- 172.21.15.32:0/2656394807 shutdown_connections 2024-04-17T09:56:06.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.908+0000 7f055be00640 1 --2- 172.21.15.32:0/2656394807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05540b0320 0x7f05540d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.908+0000 7f055be00640 1 --2- 172.21.15.32:0/2656394807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f055415e870 0x7f0554162e00 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.908+0000 7f055be00640 1 -- 172.21.15.32:0/2656394807 >> 172.21.15.32:0/2656394807 conn(0x7f05540d0c00 msgr2=0x7f05540d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:06.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.908+0000 7f055be00640 1 -- 172.21.15.32:0/2656394807 shutdown_connections 2024-04-17T09:56:06.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.908+0000 7f055be00640 1 -- 172.21.15.32:0/2656394807 wait complete. 2024-04-17T09:56:06.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.908+0000 7f055be00640 1 Processor -- start 2024-04-17T09:56:06.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.908+0000 7f055be00640 1 -- start start 2024-04-17T09:56:06.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.908+0000 7f055be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05540b0320 0x7f0554148e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:06.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.908+0000 7f055be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f055415e870 0x7f055415d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.908+0000 7f055be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f055415de70 0x7f0554149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.908+0000 7f055be00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f055406c3c0 con 0x7f05540b0320 2024-04-17T09:56:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.908+0000 7f055be00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f055406c240 con 0x7f055415e870 2024-04-17T09:56:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.908+0000 7f055be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f055406fed0 con 0x7f055415de70 2024-04-17T09:56:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.908+0000 7f0559400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05540b0320 0x7f0554148e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.912+0000 7f0559400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05540b0320 0x7f0554148e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49266/0 (socket says 172.21.15.32:49266) 2024-04-17T09:56:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.912+0000 7f0559400640 1 -- 172.21.15.32:0/3717746995 learned_addr learned my addr 172.21.15.32:0/3717746995 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.912+0000 7f0558a00640 1 --2- 172.21.15.32:0/3717746995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f055415e870 0x7f055415d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.912+0000 7f0559e00640 1 --2- 172.21.15.32:0/3717746995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f055415de70 0x7f0554149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.912+0000 7f0559400640 1 -- 172.21.15.32:0/3717746995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f055415de70 msgr2=0x7f0554149870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.912+0000 7f0559400640 1 --2- 172.21.15.32:0/3717746995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f055415de70 0x7f0554149870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.912+0000 7f0559400640 1 -- 172.21.15.32:0/3717746995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f055415e870 msgr2=0x7f055415d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.912+0000 7f0559400640 1 --2- 172.21.15.32:0/3717746995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f055415e870 0x7f055415d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.912+0000 7f0559400640 1 -- 172.21.15.32:0/3717746995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055409ac40 con 0x7f05540b0320 2024-04-17T09:56:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.912+0000 7f0559400640 1 --2- 172.21.15.32:0/3717746995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05540b0320 0x7f0554148e80 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f05480ad040 tx=0x7f054801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.912+0000 7f0551600640 1 -- 172.21.15.32:0/3717746995 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05480be5f0 con 0x7f05540b0320 2024-04-17T09:56:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.912+0000 7f0551600640 1 -- 172.21.15.32:0/3717746995 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05480be7b0 con 0x7f05540b0320 2024-04-17T09:56:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.912+0000 7f0551600640 1 -- 172.21.15.32:0/3717746995 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05480beba0 con 0x7f05540b0320 2024-04-17T09:56:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.912+0000 7f055be00640 1 -- 172.21.15.32:0/3717746995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055409af20 con 0x7f05540b0320 2024-04-17T09:56:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.912+0000 7f055be00640 1 -- 172.21.15.32:0/3717746995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f055409b4e0 con 0x7f05540b0320 2024-04-17T09:56:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.912+0000 7f055be00640 1 -- 172.21.15.32:0/3717746995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0554068a90 con 0x7f05540b0320 2024-04-17T09:56:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.912+0000 7f0551600640 1 -- 172.21.15.32:0/3717746995 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f05480b43f0 con 0x7f05540b0320 2024-04-17T09:56:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.912+0000 7f0551600640 1 --2- 172.21.15.32:0/3717746995 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0534068380 0x7f053406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.912+0000 7f0551600640 1 -- 172.21.15.32:0/3717746995 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f0548120d60 con 0x7f05540b0320 2024-04-17T09:56:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.916+0000 7f0558a00640 1 --2- 172.21.15.32:0/3717746995 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0534068380 0x7f053406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.916+0000 7f0558a00640 1 --2- 172.21.15.32:0/3717746995 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0534068380 0x7f053406a840 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f05540b1a70 tx=0x7f0540009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:06.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:06.920+0000 7f0551600640 1 -- 172.21.15.32:0/3717746995 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0548016150 con 0x7f05540b0320 2024-04-17T09:56:07.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.080+0000 7f055be00640 1 -- 172.21.15.32:0/3717746995 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f055409a9f0 con 0x7f0534068380 2024-04-17T09:56:07.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.084+0000 7f0551600640 1 -- 172.21.15.32:0/3717746995 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+243016 (secure 0 0 0) 0x7f055409a9f0 con 0x7f0534068380 2024-04-17T09:56:07.090 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:07.091 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T09:56:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.088+0000 7f055be00640 1 -- 172.21.15.32:0/3717746995 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0534068380 msgr2=0x7f053406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.088+0000 7f055be00640 1 --2- 172.21.15.32:0/3717746995 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0534068380 0x7f053406a840 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f05540b1a70 tx=0x7f0540009190 comp rx=0 tx=0).stop 2024-04-17T09:56:07.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.088+0000 7f055be00640 1 -- 172.21.15.32:0/3717746995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05540b0320 msgr2=0x7f0554148e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.088+0000 7f055be00640 1 --2- 172.21.15.32:0/3717746995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05540b0320 0x7f0554148e80 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f05480ad040 tx=0x7f054801ada0 comp rx=0 tx=0).stop 2024-04-17T09:56:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.088+0000 7f055be00640 1 -- 172.21.15.32:0/3717746995 shutdown_connections 2024-04-17T09:56:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.088+0000 7f055be00640 1 --2- 172.21.15.32:0/3717746995 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0534068380 0x7f053406a840 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.088+0000 7f055be00640 1 --2- 172.21.15.32:0/3717746995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f055415de70 0x7f0554149870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.088+0000 7f055be00640 1 --2- 172.21.15.32:0/3717746995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f055415e870 0x7f055415d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.088+0000 7f055be00640 1 --2- 172.21.15.32:0/3717746995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05540b0320 0x7f0554148e80 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.088+0000 7f055be00640 1 -- 172.21.15.32:0/3717746995 >> 172.21.15.32:0/3717746995 conn(0x7f05540d0c00 msgr2=0x7f05540b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.088+0000 7f055be00640 1 -- 172.21.15.32:0/3717746995 shutdown_connections 2024-04-17T09:56:07.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.088+0000 7f055be00640 1 -- 172.21.15.32:0/3717746995 wait complete. 2024-04-17T09:56:07.111 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-04-17T09:56:05.321817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592562,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":176,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":98,"acting":98,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":243180,"kb_used_data":6192,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731540,"statfs":{"total":773094113280,"available":772845096960,"internally_reserved":0,"allocated":6340608,"data_stored":3275885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2194,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.145090"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197311+0000","last_change":"2024-04-17T09:56:02.221498+0000","last_active":"2024-04-17T09:56:03.197311+0000","last_peered":"2024-04-17T09:56:03.197311+0000","last_clean":"2024-04-17T09:56:03.197311+0000","last_became_active":"2024-04-17T09:56:02.217279+0000","last_became_peered":"2024-04-17T09:56:02.217279+0000","last_unstale":"2024-04-17T09:56:03.197311+0000","last_undegraded":"2024-04-17T09:56:03.197311+0000","last_fullsized":"2024-04-17T09:56:03.197311+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:53.818908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203103+0000","last_change":"2024-04-17T09:56:02.220843+0000","last_active":"2024-04-17T09:56:03.203103+0000","last_peered":"2024-04-17T09:56:03.203103+0000","last_clean":"2024-04-17T09:56:03.203103+0000","last_became_active":"2024-04-17T09:56:02.220566+0000","last_became_peered":"2024-04-17T09:56:02.220566+0000","last_unstale":"2024-04-17T09:56:03.203103+0000","last_undegraded":"2024-04-17T09:56:03.203103+0000","last_fullsized":"2024-04-17T09:56:03.203103+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:24:38.324835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203578+0000","last_change":"2024-04-17T09:56:02.230219+0000","last_active":"2024-04-17T09:56:03.203578+0000","last_peered":"2024-04-17T09:56:03.203578+0000","last_clean":"2024-04-17T09:56:03.203578+0000","last_became_active":"2024-04-17T09:56:02.229840+0000","last_became_peered":"2024-04-17T09:56:02.229840+0000","last_unstale":"2024-04-17T09:56:03.203578+0000","last_undegraded":"2024-04-17T09:56:03.203578+0000","last_fullsized":"2024-04-17T09:56:03.203578+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:14:09.222091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203987+0000","last_change":"2024-04-17T09:56:02.213294+0000","last_active":"2024-04-17T09:56:03.203987+0000","last_peered":"2024-04-17T09:56:03.203987+0000","last_clean":"2024-04-17T09:56:03.203987+0000","last_became_active":"2024-04-17T09:56:02.212904+0000","last_became_peered":"2024-04-17T09:56:02.212904+0000","last_unstale":"2024-04-17T09:56:03.203987+0000","last_undegraded":"2024-04-17T09:56:03.203987+0000","last_fullsized":"2024-04-17T09:56:03.203987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:41:23.655282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203242+0000","last_change":"2024-04-17T09:56:02.222178+0000","last_active":"2024-04-17T09:56:03.203242+0000","last_peered":"2024-04-17T09:56:03.203242+0000","last_clean":"2024-04-17T09:56:03.203242+0000","last_became_active":"2024-04-17T09:56:02.221835+0000","last_became_peered":"2024-04-17T09:56:02.221835+0000","last_unstale":"2024-04-17T09:56:03.203242+0000","last_undegraded":"2024-04-17T09:56:03.203242+0000","last_fullsized":"2024-04-17T09:56:03.203242+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:40:42.141566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197006+0000","last_change":"2024-04-17T09:56:02.215969+0000","last_active":"2024-04-17T09:56:03.197006+0000","last_peered":"2024-04-17T09:56:03.197006+0000","last_clean":"2024-04-17T09:56:03.197006+0000","last_became_active":"2024-04-17T09:56:02.215670+0000","last_became_peered":"2024-04-17T09:56:02.215670+0000","last_unstale":"2024-04-17T09:56:03.197006+0000","last_undegraded":"2024-04-17T09:56:03.197006+0000","last_fullsized":"2024-04-17T09:56:03.197006+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:01:34.123890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197587+0000","last_change":"2024-04-17T09:56:02.215021+0000","last_active":"2024-04-17T09:56:03.197587+0000","last_peered":"2024-04-17T09:56:03.197587+0000","last_clean":"2024-04-17T09:56:03.197587+0000","last_became_active":"2024-04-17T09:56:02.214653+0000","last_became_peered":"2024-04-17T09:56:02.214653+0000","last_unstale":"2024-04-17T09:56:03.197587+0000","last_undegraded":"2024-04-17T09:56:03.197587+0000","last_fullsized":"2024-04-17T09:56:03.197587+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:00:41.794977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197005+0000","last_change":"2024-04-17T09:56:02.225982+0000","last_active":"2024-04-17T09:56:03.197005+0000","last_peered":"2024-04-17T09:56:03.197005+0000","last_clean":"2024-04-17T09:56:03.197005+0000","last_became_active":"2024-04-17T09:56:02.223371+0000","last_became_peered":"2024-04-17T09:56:02.223371+0000","last_unstale":"2024-04-17T09:56:03.197005+0000","last_undegraded":"2024-04-17T09:56:03.197005+0000","last_fullsized":"2024-04-17T09:56:03.197005+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:48:17.886969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197834+0000","last_change":"2024-04-17T09:56:02.224842+0000","last_active":"2024-04-17T09:56:03.197834+0000","last_peered":"2024-04-17T09:56:03.197834+0000","last_clean":"2024-04-17T09:56:03.197834+0000","last_became_active":"2024-04-17T09:56:02.224545+0000","last_became_peered":"2024-04-17T09:56:02.224545+0000","last_unstale":"2024-04-17T09:56:03.197834+0000","last_undegraded":"2024-04-17T09:56:03.197834+0000","last_fullsized":"2024-04-17T09:56:03.197834+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:57.566831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"56'4","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197875+0000","last_change":"2024-04-17T09:56:01.206168+0000","last_active":"2024-04-17T09:56:03.197875+0000","last_peered":"2024-04-17T09:56:03.197875+0000","last_clean":"2024-04-17T09:56:03.197875+0000","last_became_active":"2024-04-17T09:56:01.204386+0000","last_became_peered":"2024-04-17T09:56:01.204386+0000","last_unstale":"2024-04-17T09:56:03.197875+0000","last_undegraded":"2024-04-17T09:56:03.197875+0000","last_fullsized":"2024-04-17T09:56:03.197875+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:34.492070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.185133+0000","last_change":"2024-04-17T09:56:00.185133+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.185133+0000","last_undegraded":"2024-04-17T09:56:00.185133+0000","last_fullsized":"2024-04-17T09:56:00.185133+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197603+0000","last_change":"2024-04-17T09:56:02.234199+0000","last_active":"2024-04-17T09:56:03.197603+0000","last_peered":"2024-04-17T09:56:03.197603+0000","last_clean":"2024-04-17T09:56:03.197603+0000","last_became_active":"2024-04-17T09:56:02.233686+0000","last_became_peered":"2024-04-17T09:56:02.233686+0000","last_unstale":"2024-04-17T09:56:03.197603+0000","last_undegraded":"2024-04-17T09:56:03.197603+0000","last_fullsized":"2024-04-17T09:56:03.197603+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:08.333973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197902+0000","last_change":"2024-04-17T09:56:01.201154+0000","last_active":"2024-04-17T09:56:03.197902+0000","last_peered":"2024-04-17T09:56:03.197902+0000","last_clean":"2024-04-17T09:56:03.197902+0000","last_became_active":"2024-04-17T09:56:01.200926+0000","last_became_peered":"2024-04-17T09:56:01.200926+0000","last_unstale":"2024-04-17T09:56:03.197902+0000","last_undegraded":"2024-04-17T09:56:03.197902+0000","last_fullsized":"2024-04-17T09:56:03.197902+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:41:22.649311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197793+0000","last_change":"2024-04-17T09:56:02.230469+0000","last_active":"2024-04-17T09:56:03.197793+0000","last_peered":"2024-04-17T09:56:03.197793+0000","last_clean":"2024-04-17T09:56:03.197793+0000","last_became_active":"2024-04-17T09:56:02.229210+0000","last_became_peered":"2024-04-17T09:56:02.229210+0000","last_unstale":"2024-04-17T09:56:03.197793+0000","last_undegraded":"2024-04-17T09:56:03.197793+0000","last_fullsized":"2024-04-17T09:56:03.197793+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:58:25.393385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179530+0000","last_change":"2024-04-17T09:56:00.179530+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179530+0000","last_undegraded":"2024-04-17T09:56:00.179530+0000","last_fullsized":"2024-04-17T09:56:00.179530+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203408+0000","last_change":"2024-04-17T09:56:01.206115+0000","last_active":"2024-04-17T09:56:03.203408+0000","last_peered":"2024-04-17T09:56:03.203408+0000","last_clean":"2024-04-17T09:56:03.203408+0000","last_became_active":"2024-04-17T09:56:01.205818+0000","last_became_peered":"2024-04-17T09:56:01.205818+0000","last_unstale":"2024-04-17T09:56:03.203408+0000","last_undegraded":"2024-04-17T09:56:03.203408+0000","last_fullsized":"2024-04-17T09:56:03.203408+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:53.672442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197353+0000","last_change":"2024-04-17T09:56:01.206766+0000","last_active":"2024-04-17T09:56:03.197353+0000","last_peered":"2024-04-17T09:56:03.197353+0000","last_clean":"2024-04-17T09:56:03.197353+0000","last_became_active":"2024-04-17T09:56:01.206438+0000","last_became_peered":"2024-04-17T09:56:01.206438+0000","last_unstale":"2024-04-17T09:56:03.197353+0000","last_undegraded":"2024-04-17T09:56:03.197353+0000","last_fullsized":"2024-04-17T09:56:03.197353+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:40:41.135595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179664+0000","last_change":"2024-04-17T09:56:00.179664+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179664+0000","last_undegraded":"2024-04-17T09:56:00.179664+0000","last_fullsized":"2024-04-17T09:56:00.179664+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203390+0000","last_change":"2024-04-17T09:56:01.197166+0000","last_active":"2024-04-17T09:56:03.203390+0000","last_peered":"2024-04-17T09:56:03.203390+0000","last_clean":"2024-04-17T09:56:03.203390+0000","last_became_active":"2024-04-17T09:56:01.196416+0000","last_became_peered":"2024-04-17T09:56:01.196416+0000","last_unstale":"2024-04-17T09:56:03.203390+0000","last_undegraded":"2024-04-17T09:56:03.203390+0000","last_fullsized":"2024-04-17T09:56:03.203390+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:58:47.664590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203664+0000","last_change":"2024-04-17T09:56:02.222098+0000","last_active":"2024-04-17T09:56:03.203664+0000","last_peered":"2024-04-17T09:56:03.203664+0000","last_clean":"2024-04-17T09:56:03.203664+0000","last_became_active":"2024-04-17T09:56:02.221642+0000","last_became_peered":"2024-04-17T09:56:02.221642+0000","last_unstale":"2024-04-17T09:56:03.203664+0000","last_undegraded":"2024-04-17T09:56:03.203664+0000","last_fullsized":"2024-04-17T09:56:03.203664+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:02:18.696690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.184943+0000","last_change":"2024-04-17T09:56:00.184943+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.184943+0000","last_undegraded":"2024-04-17T09:56:00.184943+0000","last_fullsized":"2024-04-17T09:56:00.184943+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.184736+0000","last_change":"2024-04-17T09:56:00.184736+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.184736+0000","last_undegraded":"2024-04-17T09:56:00.184736+0000","last_fullsized":"2024-04-17T09:56:00.184736+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.184535+0000","last_change":"2024-04-17T09:56:00.184535+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.184535+0000","last_undegraded":"2024-04-17T09:56:00.184535+0000","last_fullsized":"2024-04-17T09:56:00.184535+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"56'6","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203761+0000","last_change":"2024-04-17T09:56:01.206040+0000","last_active":"2024-04-17T09:56:03.203761+0000","last_peered":"2024-04-17T09:56:03.203761+0000","last_clean":"2024-04-17T09:56:03.203761+0000","last_became_active":"2024-04-17T09:56:01.205744+0000","last_became_peered":"2024-04-17T09:56:01.205744+0000","last_unstale":"2024-04-17T09:56:03.203761+0000","last_undegraded":"2024-04-17T09:56:03.203761+0000","last_fullsized":"2024-04-17T09:56:03.203761+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:55:07.841245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.184703+0000","last_change":"2024-04-17T09:56:00.184703+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.184703+0000","last_undegraded":"2024-04-17T09:56:00.184703+0000","last_fullsized":"2024-04-17T09:56:00.184703+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179797+0000","last_change":"2024-04-17T09:56:00.179797+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179797+0000","last_undegraded":"2024-04-17T09:56:00.179797+0000","last_fullsized":"2024-04-17T09:56:00.179797+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203271+0000","last_change":"2024-04-17T09:56:02.210009+0000","last_active":"2024-04-17T09:56:03.203271+0000","last_peered":"2024-04-17T09:56:03.203271+0000","last_clean":"2024-04-17T09:56:03.203271+0000","last_became_active":"2024-04-17T09:56:02.209655+0000","last_became_peered":"2024-04-17T09:56:02.209655+0000","last_unstale":"2024-04-17T09:56:03.203271+0000","last_undegraded":"2024-04-17T09:56:03.203271+0000","last_fullsized":"2024-04-17T09:56:03.203271+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:23.603548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179043+0000","last_change":"2024-04-17T09:56:00.179043+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179043+0000","last_undegraded":"2024-04-17T09:56:00.179043+0000","last_fullsized":"2024-04-17T09:56:00.179043+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.183970+0000","last_change":"2024-04-17T09:56:00.183970+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.183970+0000","last_undegraded":"2024-04-17T09:56:00.183970+0000","last_fullsized":"2024-04-17T09:56:00.183970+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.185476+0000","last_change":"2024-04-17T09:56:00.185476+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.185476+0000","last_undegraded":"2024-04-17T09:56:00.185476+0000","last_fullsized":"2024-04-17T09:56:00.185476+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197845+0000","last_change":"2024-04-17T09:56:01.208863+0000","last_active":"2024-04-17T09:56:03.197845+0000","last_peered":"2024-04-17T09:56:03.197845+0000","last_clean":"2024-04-17T09:56:03.197845+0000","last_became_active":"2024-04-17T09:56:01.208605+0000","last_became_peered":"2024-04-17T09:56:01.208605+0000","last_unstale":"2024-04-17T09:56:03.197845+0000","last_undegraded":"2024-04-17T09:56:03.197845+0000","last_fullsized":"2024-04-17T09:56:03.197845+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:36.056444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179120+0000","last_change":"2024-04-17T09:56:00.179120+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179120+0000","last_undegraded":"2024-04-17T09:56:00.179120+0000","last_fullsized":"2024-04-17T09:56:00.179120+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197523+0000","last_change":"2024-04-17T09:56:01.203071+0000","last_active":"2024-04-17T09:56:03.197523+0000","last_peered":"2024-04-17T09:56:03.197523+0000","last_clean":"2024-04-17T09:56:03.197523+0000","last_became_active":"2024-04-17T09:56:01.202764+0000","last_became_peered":"2024-04-17T09:56:01.202764+0000","last_unstale":"2024-04-17T09:56:03.197523+0000","last_undegraded":"2024-04-17T09:56:03.197523+0000","last_fullsized":"2024-04-17T09:56:03.197523+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:22.597577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.184285+0000","last_change":"2024-04-17T09:56:00.184285+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.184285+0000","last_undegraded":"2024-04-17T09:56:00.184285+0000","last_fullsized":"2024-04-17T09:56:00.184285+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197598+0000","last_change":"2024-04-17T09:56:02.211557+0000","last_active":"2024-04-17T09:56:03.197598+0000","last_peered":"2024-04-17T09:56:03.197598+0000","last_clean":"2024-04-17T09:56:03.197598+0000","last_became_active":"2024-04-17T09:56:02.211186+0000","last_became_peered":"2024-04-17T09:56:02.211186+0000","last_unstale":"2024-04-17T09:56:03.197598+0000","last_undegraded":"2024-04-17T09:56:03.197598+0000","last_fullsized":"2024-04-17T09:56:03.197598+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:42:46.838421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179612+0000","last_change":"2024-04-17T09:56:00.179612+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179612+0000","last_undegraded":"2024-04-17T09:56:00.179612+0000","last_fullsized":"2024-04-17T09:56:00.179612+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197057+0000","last_change":"2024-04-17T09:56:02.211632+0000","last_active":"2024-04-17T09:56:03.197057+0000","last_peered":"2024-04-17T09:56:03.197057+0000","last_clean":"2024-04-17T09:56:03.197057+0000","last_became_active":"2024-04-17T09:56:02.211357+0000","last_became_peered":"2024-04-17T09:56:02.211357+0000","last_unstale":"2024-04-17T09:56:03.197057+0000","last_undegraded":"2024-04-17T09:56:03.197057+0000","last_fullsized":"2024-04-17T09:56:03.197057+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:16:53.336176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179313+0000","last_change":"2024-04-17T09:56:00.179313+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179313+0000","last_undegraded":"2024-04-17T09:56:00.179313+0000","last_fullsized":"2024-04-17T09:56:00.179313+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179682+0000","last_change":"2024-04-17T09:56:00.179682+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179682+0000","last_undegraded":"2024-04-17T09:56:00.179682+0000","last_fullsized":"2024-04-17T09:56:00.179682+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"57'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203049+0000","last_change":"2024-04-17T09:56:01.202823+0000","last_active":"2024-04-17T09:56:03.203049+0000","last_peered":"2024-04-17T09:56:03.203049+0000","last_clean":"2024-04-17T09:56:03.203049+0000","last_became_active":"2024-04-17T09:56:01.202563+0000","last_became_peered":"2024-04-17T09:56:01.202563+0000","last_unstale":"2024-04-17T09:56:03.203049+0000","last_undegraded":"2024-04-17T09:56:03.203049+0000","last_fullsized":"2024-04-17T09:56:03.203049+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:34:57.389627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.180425+0000","last_change":"2024-04-17T09:56:00.180425+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.180425+0000","last_undegraded":"2024-04-17T09:56:00.180425+0000","last_fullsized":"2024-04-17T09:56:00.180425+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197782+0000","last_change":"2024-04-17T09:56:02.230708+0000","last_active":"2024-04-17T09:56:03.197782+0000","last_peered":"2024-04-17T09:56:03.197782+0000","last_clean":"2024-04-17T09:56:03.197782+0000","last_became_active":"2024-04-17T09:56:02.230207+0000","last_became_peered":"2024-04-17T09:56:02.230207+0000","last_unstale":"2024-04-17T09:56:03.197782+0000","last_undegraded":"2024-04-17T09:56:03.197782+0000","last_fullsized":"2024-04-17T09:56:03.197782+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:43:27.595946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179054+0000","last_change":"2024-04-17T09:56:00.179054+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179054+0000","last_undegraded":"2024-04-17T09:56:00.179054+0000","last_fullsized":"2024-04-17T09:56:00.179054+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179594+0000","last_change":"2024-04-17T09:56:00.179594+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179594+0000","last_undegraded":"2024-04-17T09:56:00.179594+0000","last_fullsized":"2024-04-17T09:56:00.179594+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203268+0000","last_change":"2024-04-17T09:56:01.207103+0000","last_active":"2024-04-17T09:56:03.203268+0000","last_peered":"2024-04-17T09:56:03.203268+0000","last_clean":"2024-04-17T09:56:03.203268+0000","last_became_active":"2024-04-17T09:56:01.206837+0000","last_became_peered":"2024-04-17T09:56:01.206837+0000","last_unstale":"2024-04-17T09:56:03.203268+0000","last_undegraded":"2024-04-17T09:56:03.203268+0000","last_fullsized":"2024-04-17T09:56:03.203268+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:28:36.102136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.179601+0000","last_change":"2024-04-17T09:56:00.179601+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.179601+0000","last_undegraded":"2024-04-17T09:56:00.179601+0000","last_fullsized":"2024-04-17T09:56:00.179601+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.184252+0000","last_change":"2024-04-17T09:56:00.184252+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.184252+0000","last_undegraded":"2024-04-17T09:56:00.184252+0000","last_fullsized":"2024-04-17T09:56:00.184252+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-17T09:56:01.176148+0000","last_change":"2024-04-17T09:56:01.176148+0000","last_active":"2024-04-17T09:56:01.176148+0000","last_peered":"2024-04-17T09:56:01.176148+0000","last_clean":"2024-04-17T09:56:01.176148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:01.176148+0000","last_undegraded":"2024-04-17T09:56:01.176148+0000","last_fullsized":"2024-04-17T09:56:01.176148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":54,"state":"creating+peering","last_fresh":"2024-04-17T09:56:00.180271+0000","last_change":"2024-04-17T09:56:00.180271+0000","last_active":"2024-04-17T09:56:00.170177+0000","last_peered":"2024-04-17T09:56:00.170177+0000","last_clean":"2024-04-17T09:56:00.170177+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:56:00.180271+0000","last_undegraded":"2024-04-17T09:56:00.180271+0000","last_fullsized":"2024-04-17T09:56:00.180271+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":231,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-17T09:56:00.184271+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T09:56:00.184271+0000","last_peered":"2024-04-17T09:56:00.184271+0000","last_clean":"2024-04-17T09:56:00.184271+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T09:56:00.184271+0000","last_undegraded":"2024-04-17T09:56:00.184271+0000","last_fullsized":"2024-04-17T09:56:00.184271+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":2},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2194,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1220608,"data_stored":844769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":626688,"data_stored":254401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":254401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31796,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340044,"statfs":{"total":96636764160,"available":96604205056,"internally_reserved":0,"allocated":626688,"data_stored":254401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31804,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340036,"statfs":{"total":96636764160,"available":96604196864,"internally_reserved":0,"allocated":638976,"data_stored":260218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31792,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340048,"statfs":{"total":96636764160,"available":96604209152,"internally_reserved":0,"allocated":626688,"data_stored":254401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":688128,"data_stored":281172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542184,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":872122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T09:56:07.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3f is not active+clean 2024-04-17T09:56:07.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3e is not active+clean 2024-04-17T09:56:07.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3d is not active+clean 2024-04-17T09:56:07.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3c is not active+clean 2024-04-17T09:56:07.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3b is not active+clean 2024-04-17T09:56:07.116 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.38 is not active+clean 2024-04-17T09:56:07.116 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.36 is not active+clean 2024-04-17T09:56:07.116 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.35 is not active+clean 2024-04-17T09:56:07.116 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.34 is not active+clean 2024-04-17T09:56:07.116 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.117 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.33 is not active+clean 2024-04-17T09:56:07.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.117 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2f is not active+clean 2024-04-17T09:56:07.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.117 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2e is not active+clean 2024-04-17T09:56:07.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.117 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2d is not active+clean 2024-04-17T09:56:07.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.117 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2c is not active+clean 2024-04-17T09:56:07.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2b is not active+clean 2024-04-17T09:56:07.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2a is not active+clean 2024-04-17T09:56:07.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.29 is not active+clean 2024-04-17T09:56:07.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.28 is not active+clean 2024-04-17T09:56:07.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.27 is not active+clean 2024-04-17T09:56:07.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.26 is not active+clean 2024-04-17T09:56:07.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.24 is not active+clean 2024-04-17T09:56:07.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.23 is not active+clean 2024-04-17T09:56:07.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.22 is not active+clean 2024-04-17T09:56:07.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.21 is not active+clean 2024-04-17T09:56:07.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2024-04-17T09:56:07.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2024-04-17T09:56:07.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:56:00.185133+0000', 'last_change': '2024-04-17T09:56:00.185133+0000', 'last_active': '2024-04-17T09:56:00.170177+0000', 'last_peered': '2024-04-17T09:56:00.170177+0000', 'last_clean': '2024-04-17T09:56:00.170177+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:00.185133+0000', 'last_undegraded': '2024-04-17T09:56:00.185133+0000', 'last_fullsized': '2024-04-17T09:56:00.185133+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T09:56:07.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2024-04-17T09:56:07.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:56:00.179530+0000', 'last_change': '2024-04-17T09:56:00.179530+0000', 'last_active': '2024-04-17T09:56:00.170177+0000', 'last_peered': '2024-04-17T09:56:00.170177+0000', 'last_clean': '2024-04-17T09:56:00.170177+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:00.179530+0000', 'last_undegraded': '2024-04-17T09:56:00.179530+0000', 'last_fullsized': '2024-04-17T09:56:00.179530+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T09:56:07.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1a is not active+clean 2024-04-17T09:56:07.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1b is not active+clean 2024-04-17T09:56:07.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2024-04-17T09:56:07.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2024-04-17T09:56:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:56:00.179664+0000', 'last_change': '2024-04-17T09:56:00.179664+0000', 'last_active': '2024-04-17T09:56:00.170177+0000', 'last_peered': '2024-04-17T09:56:00.170177+0000', 'last_clean': '2024-04-17T09:56:00.170177+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:00.179664+0000', 'last_undegraded': '2024-04-17T09:56:00.179664+0000', 'last_fullsized': '2024-04-17T09:56:00.179664+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T09:56:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.19 is not active+clean 2024-04-17T09:56:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2024-04-17T09:56:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:56:00.184943+0000', 'last_change': '2024-04-17T09:56:00.184943+0000', 'last_active': '2024-04-17T09:56:00.170177+0000', 'last_peered': '2024-04-17T09:56:00.170177+0000', 'last_clean': '2024-04-17T09:56:00.170177+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:00.184943+0000', 'last_undegraded': '2024-04-17T09:56:00.184943+0000', 'last_fullsized': '2024-04-17T09:56:00.184943+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T09:56:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2024-04-17T09:56:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2024-04-17T09:56:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:56:00.184736+0000', 'last_change': '2024-04-17T09:56:00.184736+0000', 'last_active': '2024-04-17T09:56:00.170177+0000', 'last_peered': '2024-04-17T09:56:00.170177+0000', 'last_clean': '2024-04-17T09:56:00.170177+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:00.184736+0000', 'last_undegraded': '2024-04-17T09:56:00.184736+0000', 'last_fullsized': '2024-04-17T09:56:00.184736+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T09:56:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2024-04-17T09:56:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2024-04-17T09:56:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:56:00.184535+0000', 'last_change': '2024-04-17T09:56:00.184535+0000', 'last_active': '2024-04-17T09:56:00.170177+0000', 'last_peered': '2024-04-17T09:56:00.170177+0000', 'last_clean': '2024-04-17T09:56:00.170177+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:00.184535+0000', 'last_undegraded': '2024-04-17T09:56:00.184535+0000', 'last_fullsized': '2024-04-17T09:56:00.184535+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T09:56:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2024-04-17T09:56:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2024-04-17T09:56:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2024-04-17T09:56:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:56:00.184703+0000', 'last_change': '2024-04-17T09:56:00.184703+0000', 'last_active': '2024-04-17T09:56:00.170177+0000', 'last_peered': '2024-04-17T09:56:00.170177+0000', 'last_clean': '2024-04-17T09:56:00.170177+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:00.184703+0000', 'last_undegraded': '2024-04-17T09:56:00.184703+0000', 'last_fullsized': '2024-04-17T09:56:00.184703+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T09:56:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2024-04-17T09:56:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2024-04-17T09:56:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:56:00.179797+0000', 'last_change': '2024-04-17T09:56:00.179797+0000', 'last_active': '2024-04-17T09:56:00.170177+0000', 'last_peered': '2024-04-17T09:56:00.170177+0000', 'last_clean': '2024-04-17T09:56:00.170177+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:00.179797+0000', 'last_undegraded': '2024-04-17T09:56:00.179797+0000', 'last_fullsized': '2024-04-17T09:56:00.179797+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T09:56:07.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2024-04-17T09:56:07.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:56:00.179043+0000', 'last_change': '2024-04-17T09:56:00.179043+0000', 'last_active': '2024-04-17T09:56:00.170177+0000', 'last_peered': '2024-04-17T09:56:00.170177+0000', 'last_clean': '2024-04-17T09:56:00.170177+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:00.179043+0000', 'last_undegraded': '2024-04-17T09:56:00.179043+0000', 'last_fullsized': '2024-04-17T09:56:00.179043+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T09:56:07.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2024-04-17T09:56:07.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2024-04-17T09:56:07.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:56:00.183970+0000', 'last_change': '2024-04-17T09:56:00.183970+0000', 'last_active': '2024-04-17T09:56:00.170177+0000', 'last_peered': '2024-04-17T09:56:00.170177+0000', 'last_clean': '2024-04-17T09:56:00.170177+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:00.183970+0000', 'last_undegraded': '2024-04-17T09:56:00.183970+0000', 'last_fullsized': '2024-04-17T09:56:00.183970+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T09:56:07.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2024-04-17T09:56:07.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2024-04-17T09:56:07.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:56:00.185476+0000', 'last_change': '2024-04-17T09:56:00.185476+0000', 'last_active': '2024-04-17T09:56:00.170177+0000', 'last_peered': '2024-04-17T09:56:00.170177+0000', 'last_clean': '2024-04-17T09:56:00.170177+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:00.185476+0000', 'last_undegraded': '2024-04-17T09:56:00.185476+0000', 'last_fullsized': '2024-04-17T09:56:00.185476+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T09:56:07.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2024-04-17T09:56:07.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.c is not active+clean 2024-04-17T09:56:07.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2024-04-17T09:56:07.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:56:00.179120+0000', 'last_change': '2024-04-17T09:56:00.179120+0000', 'last_active': '2024-04-17T09:56:00.170177+0000', 'last_peered': '2024-04-17T09:56:00.170177+0000', 'last_clean': '2024-04-17T09:56:00.170177+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:00.179120+0000', 'last_undegraded': '2024-04-17T09:56:00.179120+0000', 'last_fullsized': '2024-04-17T09:56:00.179120+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T09:56:07.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2024-04-17T09:56:07.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2024-04-17T09:56:07.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2024-04-17T09:56:07.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:56:00.184285+0000', 'last_change': '2024-04-17T09:56:00.184285+0000', 'last_active': '2024-04-17T09:56:00.170177+0000', 'last_peered': '2024-04-17T09:56:00.170177+0000', 'last_clean': '2024-04-17T09:56:00.170177+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:00.184285+0000', 'last_undegraded': '2024-04-17T09:56:00.184285+0000', 'last_fullsized': '2024-04-17T09:56:00.184285+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T09:56:07.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2024-04-17T09:56:07.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:56:00.179612+0000', 'last_change': '2024-04-17T09:56:00.179612+0000', 'last_active': '2024-04-17T09:56:00.170177+0000', 'last_peered': '2024-04-17T09:56:00.170177+0000', 'last_clean': '2024-04-17T09:56:00.170177+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:00.179612+0000', 'last_undegraded': '2024-04-17T09:56:00.179612+0000', 'last_fullsized': '2024-04-17T09:56:00.179612+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T09:56:07.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2024-04-17T09:56:07.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:56:00.179313+0000', 'last_change': '2024-04-17T09:56:00.179313+0000', 'last_active': '2024-04-17T09:56:00.170177+0000', 'last_peered': '2024-04-17T09:56:00.170177+0000', 'last_clean': '2024-04-17T09:56:00.170177+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:00.179313+0000', 'last_undegraded': '2024-04-17T09:56:00.179313+0000', 'last_fullsized': '2024-04-17T09:56:00.179313+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T09:56:07.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2024-04-17T09:56:07.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2024-04-17T09:56:07.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:56:00.179682+0000', 'last_change': '2024-04-17T09:56:00.179682+0000', 'last_active': '2024-04-17T09:56:00.170177+0000', 'last_peered': '2024-04-17T09:56:00.170177+0000', 'last_clean': '2024-04-17T09:56:00.170177+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:00.179682+0000', 'last_undegraded': '2024-04-17T09:56:00.179682+0000', 'last_fullsized': '2024-04-17T09:56:00.179682+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T09:56:07.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2024-04-17T09:56:07.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.126 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.4 is not active+clean 2024-04-17T09:56:07.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.126 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2024-04-17T09:56:07.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:56:00.180425+0000', 'last_change': '2024-04-17T09:56:00.180425+0000', 'last_active': '2024-04-17T09:56:00.170177+0000', 'last_peered': '2024-04-17T09:56:00.170177+0000', 'last_clean': '2024-04-17T09:56:00.170177+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:00.180425+0000', 'last_undegraded': '2024-04-17T09:56:00.180425+0000', 'last_fullsized': '2024-04-17T09:56:00.180425+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T09:56:07.126 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2024-04-17T09:56:07.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:56:00.179054+0000', 'last_change': '2024-04-17T09:56:00.179054+0000', 'last_active': '2024-04-17T09:56:00.170177+0000', 'last_peered': '2024-04-17T09:56:00.170177+0000', 'last_clean': '2024-04-17T09:56:00.170177+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:00.179054+0000', 'last_undegraded': '2024-04-17T09:56:00.179054+0000', 'last_fullsized': '2024-04-17T09:56:00.179054+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T09:56:07.126 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2 is not active+clean 2024-04-17T09:56:07.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.126 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2024-04-17T09:56:07.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:56:00.179594+0000', 'last_change': '2024-04-17T09:56:00.179594+0000', 'last_active': '2024-04-17T09:56:00.170177+0000', 'last_peered': '2024-04-17T09:56:00.170177+0000', 'last_clean': '2024-04-17T09:56:00.170177+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:00.179594+0000', 'last_undegraded': '2024-04-17T09:56:00.179594+0000', 'last_fullsized': '2024-04-17T09:56:00.179594+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T09:56:07.126 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2024-04-17T09:56:07.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.127 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2024-04-17T09:56:07.127 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.127 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2024-04-17T09:56:07.127 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:56:00.179601+0000', 'last_change': '2024-04-17T09:56:00.179601+0000', 'last_active': '2024-04-17T09:56:00.170177+0000', 'last_peered': '2024-04-17T09:56:00.170177+0000', 'last_clean': '2024-04-17T09:56:00.170177+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:00.179601+0000', 'last_undegraded': '2024-04-17T09:56:00.179601+0000', 'last_fullsized': '2024-04-17T09:56:00.179601+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T09:56:07.127 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1 is not active+clean 2024-04-17T09:56:07.127 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.127 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2024-04-17T09:56:07.127 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:56:00.184252+0000', 'last_change': '2024-04-17T09:56:00.184252+0000', 'last_active': '2024-04-17T09:56:00.170177+0000', 'last_peered': '2024-04-17T09:56:00.170177+0000', 'last_clean': '2024-04-17T09:56:00.170177+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:00.184252+0000', 'last_undegraded': '2024-04-17T09:56:00.184252+0000', 'last_fullsized': '2024-04-17T09:56:00.184252+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T09:56:07.127 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.8 is not active+clean 2024-04-17T09:56:07.127 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-17T09:56:01.176148+0000', 'last_change': '2024-04-17T09:56:01.176148+0000', 'last_active': '2024-04-17T09:56:01.176148+0000', 'last_peered': '2024-04-17T09:56:01.176148+0000', 'last_clean': '2024-04-17T09:56:01.176148+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:01.176148+0000', 'last_undegraded': '2024-04-17T09:56:01.176148+0000', 'last_fullsized': '2024-04-17T09:56:01.176148+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:01.176148+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-17T09:56:07.128 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2024-04-17T09:56:07.128 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 54, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:56:00.180271+0000', 'last_change': '2024-04-17T09:56:00.180271+0000', 'last_active': '2024-04-17T09:56:00.170177+0000', 'last_peered': '2024-04-17T09:56:00.170177+0000', 'last_clean': '2024-04-17T09:56:00.170177+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:56:00.180271+0000', 'last_undegraded': '2024-04-17T09:56:00.180271+0000', 'last_fullsized': '2024-04-17T09:56:00.180271+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 54, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'last_clean_scrub_stamp': '2024-04-17T09:56:00.170177+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T09:56:07.128 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T09:56:07.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.288+0000 7f3bfee00640 1 Processor -- start 2024-04-17T09:56:07.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.288+0000 7f3bfee00640 1 -- start start 2024-04-17T09:56:07.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.292+0000 7f3bfee00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3bf809a9f0 0x7f3bf806ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:07.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.292+0000 7f3bfee00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3bf80d03f0 0x7f3bf8070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:07.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.292+0000 7f3bfee00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bf806eb40 con 0x7f3bf80d0870 2024-04-17T09:56:07.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.292+0000 7f3bfee00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f3bf80b4970 con 0x7f3bf80d03f0 2024-04-17T09:56:07.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.292+0000 7f3bfee00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3bf80b4c50 con 0x7f3bf809a9f0 2024-04-17T09:56:07.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.292+0000 7f3bfe400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3bf80d0870 0x7f3bf809add0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58428/0 (socket says 172.21.15.32:58428) 2024-04-17T09:56:07.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.292+0000 7f3bfe400640 1 -- 172.21.15.32:0/2174947801 learned_addr learned my addr 172.21.15.32:0/2174947801 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:07.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.292+0000 7f3bfd000640 1 --2- 172.21.15.32:0/2174947801 >> v2:172.21.15.144:3300/0 conn(0x7f3bf80d03f0 0x7f3bf8070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:07.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.292+0000 7f3bfda00640 1 --2- 172.21.15.32:0/2174947801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3bf809a9f0 0x7f3bf806ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:07.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.292+0000 7f3bf7e00640 1 -- 172.21.15.32:0/2174947801 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3107240365 0 0) 0x7f3bf806eb40 con 0x7f3bf80d0870 2024-04-17T09:56:07.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.292+0000 7f3bf7e00640 1 -- 172.21.15.32:0/2174947801 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3be0003530 con 0x7f3bf80d0870 2024-04-17T09:56:07.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.292+0000 7f3bfda00640 1 -- 172.21.15.32:0/2174947801 >> v2:172.21.15.144:3300/0 conn(0x7f3bf80d03f0 msgr2=0x7f3bf8070880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:07.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.292+0000 7f3bfda00640 1 --2- 172.21.15.32:0/2174947801 >> v2:172.21.15.144:3300/0 conn(0x7f3bf80d03f0 0x7f3bf8070880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:07.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.292+0000 7f3bfda00640 1 -- 172.21.15.32:0/2174947801 >> v1:172.21.15.32:6789/0 conn(0x7f3bf80d0870 legacy=0x7f3bf809add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:07.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.292+0000 7f3bfda00640 1 -- 172.21.15.32:0/2174947801 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf806d550 con 0x7f3bf809a9f0 2024-04-17T09:56:07.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.292+0000 7f3bfd000640 1 --2- 172.21.15.32:0/2174947801 >> v2:172.21.15.144:3300/0 conn(0x7f3bf80d03f0 0x7f3bf8070880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:56:07.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.296+0000 7f3bfda00640 1 --2- 172.21.15.32:0/2174947801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3bf809a9f0 0x7f3bf806ddf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0097c0 tx=0x7f3bec01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7579d4becd16fcf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:07.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.296+0000 7f3bf7e00640 1 -- 172.21.15.32:0/2174947801 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec0b2070 con 0x7f3bf809a9f0 2024-04-17T09:56:07.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.296+0000 7f3bf7e00640 1 -- 172.21.15.32:0/2174947801 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bec0ad640 con 0x7f3bf809a9f0 2024-04-17T09:56:07.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.296+0000 7f3bf7e00640 1 -- 172.21.15.32:0/2174947801 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec0ad9f0 con 0x7f3bf809a9f0 2024-04-17T09:56:07.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.296+0000 7f3bfee00640 1 -- 172.21.15.32:0/2174947801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3bf809a9f0 msgr2=0x7f3bf806ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:07.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.296+0000 7f3bfee00640 1 --2- 172.21.15.32:0/2174947801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3bf809a9f0 0x7f3bf806ddf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0097c0 tx=0x7f3bec01c4a0 comp rx=0 tx=0).stop 2024-04-17T09:56:07.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.296+0000 7f3bfee00640 1 -- 172.21.15.32:0/2174947801 shutdown_connections 2024-04-17T09:56:07.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.296+0000 7f3bfee00640 1 --2- 172.21.15.32:0/2174947801 >> v2:172.21.15.144:3300/0 conn(0x7f3bf80d03f0 0x7f3bf8070880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:07.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.296+0000 7f3bfee00640 1 --2- 172.21.15.32:0/2174947801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3bf809a9f0 0x7f3bf806ddf0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:07.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.296+0000 7f3bfee00640 1 -- 172.21.15.32:0/2174947801 >> 172.21.15.32:0/2174947801 conn(0x7f3bf80b3e50 msgr2=0x7f3bf80b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:07.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.296+0000 7f3bfee00640 1 -- 172.21.15.32:0/2174947801 shutdown_connections 2024-04-17T09:56:07.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.296+0000 7f3bfee00640 1 -- 172.21.15.32:0/2174947801 wait complete. 2024-04-17T09:56:07.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.296+0000 7f3bfee00640 1 Processor -- start 2024-04-17T09:56:07.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.296+0000 7f3bfee00640 1 -- start start 2024-04-17T09:56:07.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.296+0000 7f3bfee00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bf809a9f0 0x7f3bf814ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:07.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bfee00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bf80d03f0 0x7f3bf8157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:07.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bfee00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3bf80d0870 0x7f3bf815e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:07.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bfee00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf816bf50 con 0x7f3bf809a9f0 2024-04-17T09:56:07.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bfee00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf80705a0 con 0x7f3bf80d03f0 2024-04-17T09:56:07.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bfee00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3bf816c0d0 con 0x7f3bf80d0870 2024-04-17T09:56:07.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bfda00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bf809a9f0 0x7f3bf814ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:07.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bfda00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bf809a9f0 0x7f3bf814ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49280/0 (socket says 172.21.15.32:49280) 2024-04-17T09:56:07.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bfda00640 1 -- 172.21.15.32:0/3704563309 learned_addr learned my addr 172.21.15.32:0/3704563309 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:07.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bfd000640 1 --2- 172.21.15.32:0/3704563309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bf80d03f0 0x7f3bf8157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:07.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bfe400640 1 --2- 172.21.15.32:0/3704563309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3bf80d0870 0x7f3bf815e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:07.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bfda00640 1 -- 172.21.15.32:0/3704563309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3bf80d0870 msgr2=0x7f3bf815e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:07.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bfda00640 1 --2- 172.21.15.32:0/3704563309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3bf80d0870 0x7f3bf815e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:07.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bfda00640 1 -- 172.21.15.32:0/3704563309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bf80d03f0 msgr2=0x7f3bf8157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:07.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bfda00640 1 --2- 172.21.15.32:0/3704563309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bf80d03f0 0x7f3bf8157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:07.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bfda00640 1 -- 172.21.15.32:0/3704563309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf806d290 con 0x7f3bf809a9f0 2024-04-17T09:56:07.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bfd000640 1 --2- 172.21.15.32:0/3704563309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bf80d03f0 0x7f3bf8157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:07.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bfda00640 1 --2- 172.21.15.32:0/3704563309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bf809a9f0 0x7f3bf814ee70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0b0040 tx=0x7f3bec019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:07.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bf5600640 1 -- 172.21.15.32:0/3704563309 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec0b2040 con 0x7f3bf809a9f0 2024-04-17T09:56:07.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bf5600640 1 -- 172.21.15.32:0/3704563309 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bec0ad440 con 0x7f3bf809a9f0 2024-04-17T09:56:07.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bf5600640 1 -- 172.21.15.32:0/3704563309 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec006e20 con 0x7f3bf809a9f0 2024-04-17T09:56:07.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bfee00640 1 -- 172.21.15.32:0/3704563309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf80b03a0 con 0x7f3bf809a9f0 2024-04-17T09:56:07.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bfee00640 1 -- 172.21.15.32:0/3704563309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf80b0960 con 0x7f3bf809a9f0 2024-04-17T09:56:07.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bfee00640 1 -- 172.21.15.32:0/3704563309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bf804fd00 con 0x7f3bf809a9f0 2024-04-17T09:56:07.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.300+0000 7f3bf5600640 1 -- 172.21.15.32:0/3704563309 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3bec0be020 con 0x7f3bf809a9f0 2024-04-17T09:56:07.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.304+0000 7f3bf5600640 1 --2- 172.21.15.32:0/3704563309 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3bd80683b0 0x7f3bd806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:07.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.304+0000 7f3bf5600640 1 -- 172.21.15.32:0/3704563309 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f3bec1201b0 con 0x7f3bf809a9f0 2024-04-17T09:56:07.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.304+0000 7f3bfd000640 1 --2- 172.21.15.32:0/3704563309 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3bd80683b0 0x7f3bd806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:07.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.304+0000 7f3bfd000640 1 --2- 172.21.15.32:0/3704563309 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3bd80683b0 0x7f3bd806a870 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3bf81fb5a0 tx=0x7f3be4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:07.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.308+0000 7f3bf5600640 1 -- 172.21.15.32:0/3704563309 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3bec016090 con 0x7f3bf809a9f0 2024-04-17T09:56:07.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.528+0000 7f3bfee00640 1 -- 172.21.15.32:0/3704563309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f3bf80d20d0 con 0x7f3bf809a9f0 2024-04-17T09:56:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.528+0000 7f3bf5600640 1 -- 172.21.15.32:0/3704563309 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1722 (secure 0 0 0) 0x7f3bec0f07f0 con 0x7f3bf809a9f0 2024-04-17T09:56:07.532 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:07.532 INFO:teuthology.orchestra.run.smithi032.stdout:{"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":199,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":8,"num_up_osds":8,"osd_up_since":1713347579,"num_in_osds":8,"osd_in_since":1713347570,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":48},{"state_name":"active+clean","count":27},{"state_name":"creating+peering","count":22}],"num_pgs":97,"num_pools":3,"num_objects":11,"data_bytes":592562,"bytes_used":249016320,"bytes_avail":772845096960,"bytes_total":773094113280,"unknown_pgs_ratio":0.49484536051750183,"inactive_pgs_ratio":0.22680412232875824,"write_bytes_sec":796,"read_op_per_sec":0,"write_op_per_sec":3},"fsmap":{"epoch":32,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"a","status":"up:active","gid":6204}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-04-17T09:55:21.311602+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T09:56:07.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.532+0000 7f3bfee00640 1 -- 172.21.15.32:0/3704563309 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3bd80683b0 msgr2=0x7f3bd806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:07.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.532+0000 7f3bfee00640 1 --2- 172.21.15.32:0/3704563309 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3bd80683b0 0x7f3bd806a870 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3bf81fb5a0 tx=0x7f3be4008040 comp rx=0 tx=0).stop 2024-04-17T09:56:07.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.532+0000 7f3bfee00640 1 -- 172.21.15.32:0/3704563309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bf809a9f0 msgr2=0x7f3bf814ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:07.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.532+0000 7f3bfee00640 1 --2- 172.21.15.32:0/3704563309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bf809a9f0 0x7f3bf814ee70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0b0040 tx=0x7f3bec019240 comp rx=0 tx=0).stop 2024-04-17T09:56:07.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.532+0000 7f3bfee00640 1 -- 172.21.15.32:0/3704563309 shutdown_connections 2024-04-17T09:56:07.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.532+0000 7f3bfee00640 1 --2- 172.21.15.32:0/3704563309 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3bd80683b0 0x7f3bd806a870 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:07.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.532+0000 7f3bfee00640 1 --2- 172.21.15.32:0/3704563309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3bf80d0870 0x7f3bf815e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:07.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.532+0000 7f3bfee00640 1 --2- 172.21.15.32:0/3704563309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bf80d03f0 0x7f3bf8157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:07.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.532+0000 7f3bfee00640 1 --2- 172.21.15.32:0/3704563309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bf809a9f0 0x7f3bf814ee70 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:07.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.532+0000 7f3bfee00640 1 -- 172.21.15.32:0/3704563309 >> 172.21.15.32:0/3704563309 conn(0x7f3bf80b3e50 msgr2=0x7f3bf8158f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:07.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.532+0000 7f3bfee00640 1 -- 172.21.15.32:0/3704563309 shutdown_connections 2024-04-17T09:56:07.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.532+0000 7f3bfee00640 1 -- 172.21.15.32:0/3704563309 wait complete. 2024-04-17T09:56:07.554 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-17T09:56:07.555 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T09:56:07.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.692+0000 7f263e000640 1 Processor -- start 2024-04-17T09:56:07.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.692+0000 7f263e000640 1 -- start start 2024-04-17T09:56:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.692+0000 7f263e000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2638144e90 0x7f26380b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.696+0000 7f263e000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2638072bb0 con 0x7f263815c1b0 2024-04-17T09:56:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.696+0000 7f263e000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26380b3dc0 con 0x7f26380b4de0 2024-04-17T09:56:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.696+0000 7f263e000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2638145270 con 0x7f2638144e90 2024-04-17T09:56:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.696+0000 7f2637e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f263815c1b0 0x7f263806ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58438/0 (socket says 172.21.15.32:58438) 2024-04-17T09:56:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.696+0000 7f2637e00640 1 -- 172.21.15.32:0/1283846068 learned_addr learned my addr 172.21.15.32:0/1283846068 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.696+0000 7f2637400640 1 --2- 172.21.15.32:0/1283846068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2638144e90 0x7f26380b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.696+0000 7f2636a00640 1 -- 172.21.15.32:0/1283846068 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4233438840 0 0) 0x7f2638072bb0 con 0x7f263815c1b0 2024-04-17T09:56:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.696+0000 7f2636a00640 1 -- 172.21.15.32:0/1283846068 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f261c003530 con 0x7f263815c1b0 2024-04-17T09:56:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.696+0000 7f2636a00640 1 -- 172.21.15.32:0/1283846068 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3769718283 0 0) 0x7f26380b3dc0 con 0x7f26380b4de0 2024-04-17T09:56:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.696+0000 7f2636a00640 1 -- 172.21.15.32:0/1283846068 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2638072bb0 con 0x7f26380b4de0 2024-04-17T09:56:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.696+0000 7f2637400640 1 -- 172.21.15.32:0/1283846068 >> v1:172.21.15.144:6789/0 conn(0x7f26380b4de0 legacy=0x7f263815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.696+0000 7f2637400640 1 -- 172.21.15.32:0/1283846068 >> v1:172.21.15.32:6789/0 conn(0x7f263815c1b0 legacy=0x7f263806ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.696+0000 7f2636a00640 1 -- 172.21.15.32:0/1283846068 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4082843484 0 0) 0x7f261c003530 con 0x7f263815c1b0 2024-04-17T09:56:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.696+0000 7f2637400640 1 -- 172.21.15.32:0/1283846068 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26380b40a0 con 0x7f2638144e90 2024-04-17T09:56:07.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.696+0000 7f2637400640 1 --2- 172.21.15.32:0/1283846068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2638144e90 0x7f26380b48a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f26280097f0 tx=0x7f262801c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84b0ab86c2d9bc07 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:07.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.700+0000 7f2636a00640 1 -- 172.21.15.32:0/1283846068 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26280b0070 con 0x7f2638144e90 2024-04-17T09:56:07.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.700+0000 7f2636a00640 1 -- 172.21.15.32:0/1283846068 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26280ab400 con 0x7f2638144e90 2024-04-17T09:56:07.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.700+0000 7f2636a00640 1 -- 172.21.15.32:0/1283846068 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2630001a20 con 0x7f2638144e90 2024-04-17T09:56:07.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.700+0000 7f263e000640 1 -- 172.21.15.32:0/1283846068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2638144e90 msgr2=0x7f26380b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:07.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.700+0000 7f263e000640 1 --2- 172.21.15.32:0/1283846068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2638144e90 0x7f26380b48a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f26280097f0 tx=0x7f262801c3d0 comp rx=0 tx=0).stop 2024-04-17T09:56:07.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.700+0000 7f263e000640 1 -- 172.21.15.32:0/1283846068 shutdown_connections 2024-04-17T09:56:07.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.700+0000 7f263e000640 1 --2- 172.21.15.32:0/1283846068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2638144e90 0x7f26380b48a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:07.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.700+0000 7f263e000640 1 -- 172.21.15.32:0/1283846068 >> 172.21.15.32:0/1283846068 conn(0x7f263806ba30 msgr2=0x7f263806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:07.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.700+0000 7f263e000640 1 -- 172.21.15.32:0/1283846068 shutdown_connections 2024-04-17T09:56:07.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.700+0000 7f263e000640 1 -- 172.21.15.32:0/1283846068 wait complete. 2024-04-17T09:56:07.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.700+0000 7f263e000640 1 Processor -- start 2024-04-17T09:56:07.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.700+0000 7f263e000640 1 -- start start 2024-04-17T09:56:07.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.700+0000 7f263e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26380b4de0 0x7f26381440b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:07.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.700+0000 7f263e000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2638144e90 0x7f2638140430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:07.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.700+0000 7f263e000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f263815c1b0 0x7f263816a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:07.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.700+0000 7f263e000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2638072670 con 0x7f26380b4de0 2024-04-17T09:56:07.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.700+0000 7f263e000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f263806f880 con 0x7f2638144e90 2024-04-17T09:56:07.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.700+0000 7f263e000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f26380727f0 con 0x7f263815c1b0 2024-04-17T09:56:07.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.704+0000 7f2637400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26380b4de0 0x7f26381440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:07.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.704+0000 7f2637400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26380b4de0 0x7f26381440b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49286/0 (socket says 172.21.15.32:49286) 2024-04-17T09:56:07.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.704+0000 7f2637400640 1 -- 172.21.15.32:0/652648881 learned_addr learned my addr 172.21.15.32:0/652648881 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:07.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.704+0000 7f262ea00640 1 --2- 172.21.15.32:0/652648881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2638144e90 0x7f2638140430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:07.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.704+0000 7f2637e00640 1 --2- 172.21.15.32:0/652648881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f263815c1b0 0x7f263816a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:07.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.704+0000 7f2637400640 1 -- 172.21.15.32:0/652648881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f263815c1b0 msgr2=0x7f263816a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:07.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.704+0000 7f2637400640 1 --2- 172.21.15.32:0/652648881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f263815c1b0 0x7f263816a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:07.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.704+0000 7f2637400640 1 -- 172.21.15.32:0/652648881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2638144e90 msgr2=0x7f2638140430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:07.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.704+0000 7f2637400640 1 --2- 172.21.15.32:0/652648881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2638144e90 0x7f2638140430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:07.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.704+0000 7f2637400640 1 -- 172.21.15.32:0/652648881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26381445f0 con 0x7f26380b4de0 2024-04-17T09:56:07.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.704+0000 7f262ea00640 1 --2- 172.21.15.32:0/652648881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2638144e90 0x7f2638140430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:07.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.704+0000 7f2637e00640 1 --2- 172.21.15.32:0/652648881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f263815c1b0 0x7f263816a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:07.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.704+0000 7f2637400640 1 --2- 172.21.15.32:0/652648881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26380b4de0 0x7f26381440b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f26280ae040 tx=0x7f262801ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:07.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.704+0000 7f262fe00640 1 -- 172.21.15.32:0/652648881 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26280bb020 con 0x7f26380b4de0 2024-04-17T09:56:07.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.704+0000 7f262fe00640 1 -- 172.21.15.32:0/652648881 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26280b1040 con 0x7f26380b4de0 2024-04-17T09:56:07.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.704+0000 7f262fe00640 1 -- 172.21.15.32:0/652648881 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26280b0050 con 0x7f26380b4de0 2024-04-17T09:56:07.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.704+0000 7f263e000640 1 -- 172.21.15.32:0/652648881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2638142ef0 con 0x7f26380b4de0 2024-04-17T09:56:07.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.704+0000 7f263e000640 1 -- 172.21.15.32:0/652648881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26381438b0 con 0x7f26380b4de0 2024-04-17T09:56:07.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.704+0000 7f263e000640 1 -- 172.21.15.32:0/652648881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2638068a90 con 0x7f26380b4de0 2024-04-17T09:56:07.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.704+0000 7f262fe00640 1 -- 172.21.15.32:0/652648881 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f26280be070 con 0x7f26380b4de0 2024-04-17T09:56:07.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.708+0000 7f262fe00640 1 --2- 172.21.15.32:0/652648881 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2614068380 0x7f261406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:07.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.708+0000 7f262fe00640 1 -- 172.21.15.32:0/652648881 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f2628120100 con 0x7f26380b4de0 2024-04-17T09:56:07.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.708+0000 7f262ea00640 1 --2- 172.21.15.32:0/652648881 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2614068380 0x7f261406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:07.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.708+0000 7f262ea00640 1 --2- 172.21.15.32:0/652648881 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2614068380 0x7f261406a840 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f26381fab60 tx=0x7f2620009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:07.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.712+0000 7f262fe00640 1 -- 172.21.15.32:0/652648881 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2628015ff0 con 0x7f26380b4de0 2024-04-17T09:56:07.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.872+0000 7f263e000640 1 -- 172.21.15.32:0/652648881 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f263806eed0 con 0x7f2614068380 2024-04-17T09:56:07.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.876+0000 7f262fe00640 1 -- 172.21.15.32:0/652648881 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251954 (secure 0 0 0) 0x7f263806eed0 con 0x7f2614068380 2024-04-17T09:56:07.882 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:07.883 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T09:56:07.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.880+0000 7f263e000640 1 -- 172.21.15.32:0/652648881 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2614068380 msgr2=0x7f261406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:07.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.880+0000 7f263e000640 1 --2- 172.21.15.32:0/652648881 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2614068380 0x7f261406a840 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f26381fab60 tx=0x7f2620009040 comp rx=0 tx=0).stop 2024-04-17T09:56:07.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.880+0000 7f263e000640 1 -- 172.21.15.32:0/652648881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26380b4de0 msgr2=0x7f26381440b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:07.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.880+0000 7f263e000640 1 --2- 172.21.15.32:0/652648881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26380b4de0 0x7f26381440b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f26280ae040 tx=0x7f262801ac10 comp rx=0 tx=0).stop 2024-04-17T09:56:07.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.880+0000 7f263e000640 1 -- 172.21.15.32:0/652648881 shutdown_connections 2024-04-17T09:56:07.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.880+0000 7f263e000640 1 --2- 172.21.15.32:0/652648881 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2614068380 0x7f261406a840 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:07.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.880+0000 7f263e000640 1 --2- 172.21.15.32:0/652648881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f263815c1b0 0x7f263816a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:07.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.880+0000 7f263e000640 1 --2- 172.21.15.32:0/652648881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2638144e90 0x7f2638140430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:07.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.880+0000 7f263e000640 1 --2- 172.21.15.32:0/652648881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26380b4de0 0x7f26381440b0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:07.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.880+0000 7f263e000640 1 -- 172.21.15.32:0/652648881 >> 172.21.15.32:0/652648881 conn(0x7f263806ba30 msgr2=0x7f26380cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:07.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.880+0000 7f263e000640 1 -- 172.21.15.32:0/652648881 shutdown_connections 2024-04-17T09:56:07.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:07.880+0000 7f263e000640 1 -- 172.21.15.32:0/652648881 wait complete. 2024-04-17T09:56:07.902 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-04-17T09:56:07.322249+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243436,"kb_used_data":6456,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731284,"statfs":{"total":773094113280,"available":772844834816,"internally_reserved":0,"allocated":6610944,"data_stored":3396836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.142574"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.360759+0000","last_change":"2024-04-17T09:56:02.229036+0000","last_active":"2024-04-17T09:56:03.360759+0000","last_peered":"2024-04-17T09:56:03.360759+0000","last_clean":"2024-04-17T09:56:03.360759+0000","last_became_active":"2024-04-17T09:56:02.228759+0000","last_became_peered":"2024-04-17T09:56:02.228759+0000","last_unstale":"2024-04-17T09:56:03.360759+0000","last_undegraded":"2024-04-17T09:56:03.360759+0000","last_fullsized":"2024-04-17T09:56:03.360759+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:34:34.405899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261433+0000","last_change":"2024-04-17T09:56:02.224588+0000","last_active":"2024-04-17T09:56:05.261433+0000","last_peered":"2024-04-17T09:56:05.261433+0000","last_clean":"2024-04-17T09:56:05.261433+0000","last_became_active":"2024-04-17T09:56:02.224240+0000","last_became_peered":"2024-04-17T09:56:02.224240+0000","last_unstale":"2024-04-17T09:56:05.261433+0000","last_undegraded":"2024-04-17T09:56:05.261433+0000","last_fullsized":"2024-04-17T09:56:05.261433+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:15:50.953634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260735+0000","last_change":"2024-04-17T09:56:02.211108+0000","last_active":"2024-04-17T09:56:05.260735+0000","last_peered":"2024-04-17T09:56:05.260735+0000","last_clean":"2024-04-17T09:56:05.260735+0000","last_became_active":"2024-04-17T09:56:02.210847+0000","last_became_peered":"2024-04-17T09:56:02.210847+0000","last_unstale":"2024-04-17T09:56:05.260735+0000","last_undegraded":"2024-04-17T09:56:05.260735+0000","last_fullsized":"2024-04-17T09:56:05.260735+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:35:05.000285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260436+0000","last_change":"2024-04-17T09:56:02.224673+0000","last_active":"2024-04-17T09:56:05.260436+0000","last_peered":"2024-04-17T09:56:05.260436+0000","last_clean":"2024-04-17T09:56:05.260436+0000","last_became_active":"2024-04-17T09:56:02.224412+0000","last_became_peered":"2024-04-17T09:56:02.224412+0000","last_unstale":"2024-04-17T09:56:05.260436+0000","last_undegraded":"2024-04-17T09:56:05.260436+0000","last_fullsized":"2024-04-17T09:56:05.260436+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:04:55.654330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.360752+0000","last_change":"2024-04-17T09:56:02.220342+0000","last_active":"2024-04-17T09:56:03.360752+0000","last_peered":"2024-04-17T09:56:03.360752+0000","last_clean":"2024-04-17T09:56:03.360752+0000","last_became_active":"2024-04-17T09:56:02.220041+0000","last_became_peered":"2024-04-17T09:56:02.220041+0000","last_unstale":"2024-04-17T09:56:03.360752+0000","last_undegraded":"2024-04-17T09:56:03.360752+0000","last_fullsized":"2024-04-17T09:56:03.360752+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:40.237318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197311+0000","last_change":"2024-04-17T09:56:02.221498+0000","last_active":"2024-04-17T09:56:03.197311+0000","last_peered":"2024-04-17T09:56:03.197311+0000","last_clean":"2024-04-17T09:56:03.197311+0000","last_became_active":"2024-04-17T09:56:02.217279+0000","last_became_peered":"2024-04-17T09:56:02.217279+0000","last_unstale":"2024-04-17T09:56:03.197311+0000","last_undegraded":"2024-04-17T09:56:03.197311+0000","last_fullsized":"2024-04-17T09:56:03.197311+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:53.818908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203103+0000","last_change":"2024-04-17T09:56:02.220843+0000","last_active":"2024-04-17T09:56:03.203103+0000","last_peered":"2024-04-17T09:56:03.203103+0000","last_clean":"2024-04-17T09:56:03.203103+0000","last_became_active":"2024-04-17T09:56:02.220566+0000","last_became_peered":"2024-04-17T09:56:02.220566+0000","last_unstale":"2024-04-17T09:56:03.203103+0000","last_undegraded":"2024-04-17T09:56:03.203103+0000","last_fullsized":"2024-04-17T09:56:03.203103+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:24:38.324835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261328+0000","last_change":"2024-04-17T09:56:02.211847+0000","last_active":"2024-04-17T09:56:05.261328+0000","last_peered":"2024-04-17T09:56:05.261328+0000","last_clean":"2024-04-17T09:56:05.261328+0000","last_became_active":"2024-04-17T09:56:02.211588+0000","last_became_peered":"2024-04-17T09:56:02.211588+0000","last_unstale":"2024-04-17T09:56:05.261328+0000","last_undegraded":"2024-04-17T09:56:05.261328+0000","last_fullsized":"2024-04-17T09:56:05.261328+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:10:29.259502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203578+0000","last_change":"2024-04-17T09:56:02.230219+0000","last_active":"2024-04-17T09:56:03.203578+0000","last_peered":"2024-04-17T09:56:03.203578+0000","last_clean":"2024-04-17T09:56:03.203578+0000","last_became_active":"2024-04-17T09:56:02.229840+0000","last_became_peered":"2024-04-17T09:56:02.229840+0000","last_unstale":"2024-04-17T09:56:03.203578+0000","last_undegraded":"2024-04-17T09:56:03.203578+0000","last_fullsized":"2024-04-17T09:56:03.203578+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:14:09.222091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262523+0000","last_change":"2024-04-17T09:56:02.221953+0000","last_active":"2024-04-17T09:56:05.262523+0000","last_peered":"2024-04-17T09:56:05.262523+0000","last_clean":"2024-04-17T09:56:05.262523+0000","last_became_active":"2024-04-17T09:56:02.221594+0000","last_became_peered":"2024-04-17T09:56:02.221594+0000","last_unstale":"2024-04-17T09:56:05.262523+0000","last_undegraded":"2024-04-17T09:56:05.262523+0000","last_fullsized":"2024-04-17T09:56:05.262523+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:58:59.479429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262281+0000","last_change":"2024-04-17T09:56:02.227065+0000","last_active":"2024-04-17T09:56:05.262281+0000","last_peered":"2024-04-17T09:56:05.262281+0000","last_clean":"2024-04-17T09:56:05.262281+0000","last_became_active":"2024-04-17T09:56:02.226787+0000","last_became_peered":"2024-04-17T09:56:02.226787+0000","last_unstale":"2024-04-17T09:56:05.262281+0000","last_undegraded":"2024-04-17T09:56:05.262281+0000","last_fullsized":"2024-04-17T09:56:05.262281+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:44:28.409976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261050+0000","last_change":"2024-04-17T09:56:02.216456+0000","last_active":"2024-04-17T09:56:05.261050+0000","last_peered":"2024-04-17T09:56:05.261050+0000","last_clean":"2024-04-17T09:56:05.261050+0000","last_became_active":"2024-04-17T09:56:02.216072+0000","last_became_peered":"2024-04-17T09:56:02.216072+0000","last_unstale":"2024-04-17T09:56:05.261050+0000","last_undegraded":"2024-04-17T09:56:05.261050+0000","last_fullsized":"2024-04-17T09:56:05.261050+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:55:11.309539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203124+0000","last_change":"2024-04-17T09:56:02.215465+0000","last_active":"2024-04-17T09:56:03.203124+0000","last_peered":"2024-04-17T09:56:03.203124+0000","last_clean":"2024-04-17T09:56:03.203124+0000","last_became_active":"2024-04-17T09:56:02.215176+0000","last_became_peered":"2024-04-17T09:56:02.215176+0000","last_unstale":"2024-04-17T09:56:03.203124+0000","last_undegraded":"2024-04-17T09:56:03.203124+0000","last_fullsized":"2024-04-17T09:56:03.203124+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:13:28.472859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203987+0000","last_change":"2024-04-17T09:56:02.213294+0000","last_active":"2024-04-17T09:56:03.203987+0000","last_peered":"2024-04-17T09:56:03.203987+0000","last_clean":"2024-04-17T09:56:03.203987+0000","last_became_active":"2024-04-17T09:56:02.212904+0000","last_became_peered":"2024-04-17T09:56:02.212904+0000","last_unstale":"2024-04-17T09:56:03.203987+0000","last_undegraded":"2024-04-17T09:56:03.203987+0000","last_fullsized":"2024-04-17T09:56:03.203987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:41:23.655282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203242+0000","last_change":"2024-04-17T09:56:02.222178+0000","last_active":"2024-04-17T09:56:03.203242+0000","last_peered":"2024-04-17T09:56:03.203242+0000","last_clean":"2024-04-17T09:56:03.203242+0000","last_became_active":"2024-04-17T09:56:02.221835+0000","last_became_peered":"2024-04-17T09:56:02.221835+0000","last_unstale":"2024-04-17T09:56:03.203242+0000","last_undegraded":"2024-04-17T09:56:03.203242+0000","last_fullsized":"2024-04-17T09:56:03.203242+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:40:42.141566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197006+0000","last_change":"2024-04-17T09:56:02.215969+0000","last_active":"2024-04-17T09:56:03.197006+0000","last_peered":"2024-04-17T09:56:03.197006+0000","last_clean":"2024-04-17T09:56:03.197006+0000","last_became_active":"2024-04-17T09:56:02.215670+0000","last_became_peered":"2024-04-17T09:56:02.215670+0000","last_unstale":"2024-04-17T09:56:03.197006+0000","last_undegraded":"2024-04-17T09:56:03.197006+0000","last_fullsized":"2024-04-17T09:56:03.197006+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:01:34.123890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262211+0000","last_change":"2024-04-17T09:56:02.210928+0000","last_active":"2024-04-17T09:56:05.262211+0000","last_peered":"2024-04-17T09:56:05.262211+0000","last_clean":"2024-04-17T09:56:05.262211+0000","last_became_active":"2024-04-17T09:56:02.210640+0000","last_became_peered":"2024-04-17T09:56:02.210640+0000","last_unstale":"2024-04-17T09:56:05.262211+0000","last_undegraded":"2024-04-17T09:56:05.262211+0000","last_fullsized":"2024-04-17T09:56:05.262211+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:42.518838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260690+0000","last_change":"2024-04-17T09:56:02.229718+0000","last_active":"2024-04-17T09:56:05.260690+0000","last_peered":"2024-04-17T09:56:05.260690+0000","last_clean":"2024-04-17T09:56:05.260690+0000","last_became_active":"2024-04-17T09:56:02.229408+0000","last_became_peered":"2024-04-17T09:56:02.229408+0000","last_unstale":"2024-04-17T09:56:05.260690+0000","last_undegraded":"2024-04-17T09:56:05.260690+0000","last_fullsized":"2024-04-17T09:56:05.260690+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:41:11.361171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.360652+0000","last_change":"2024-04-17T09:56:02.230129+0000","last_active":"2024-04-17T09:56:03.360652+0000","last_peered":"2024-04-17T09:56:03.360652+0000","last_clean":"2024-04-17T09:56:03.360652+0000","last_became_active":"2024-04-17T09:56:02.229876+0000","last_became_peered":"2024-04-17T09:56:02.229876+0000","last_unstale":"2024-04-17T09:56:03.360652+0000","last_undegraded":"2024-04-17T09:56:03.360652+0000","last_fullsized":"2024-04-17T09:56:03.360652+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:13:48.083231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261631+0000","last_change":"2024-04-17T09:56:02.229569+0000","last_active":"2024-04-17T09:56:05.261631+0000","last_peered":"2024-04-17T09:56:05.261631+0000","last_clean":"2024-04-17T09:56:05.261631+0000","last_became_active":"2024-04-17T09:56:02.229290+0000","last_became_peered":"2024-04-17T09:56:02.229290+0000","last_unstale":"2024-04-17T09:56:05.261631+0000","last_undegraded":"2024-04-17T09:56:05.261631+0000","last_fullsized":"2024-04-17T09:56:05.261631+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:54.063917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260149+0000","last_change":"2024-04-17T09:56:02.227809+0000","last_active":"2024-04-17T09:56:05.260149+0000","last_peered":"2024-04-17T09:56:05.260149+0000","last_clean":"2024-04-17T09:56:05.260149+0000","last_became_active":"2024-04-17T09:56:02.227554+0000","last_became_peered":"2024-04-17T09:56:02.227554+0000","last_unstale":"2024-04-17T09:56:05.260149+0000","last_undegraded":"2024-04-17T09:56:05.260149+0000","last_fullsized":"2024-04-17T09:56:05.260149+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:53:24.666275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261775+0000","last_change":"2024-04-17T09:56:02.220664+0000","last_active":"2024-04-17T09:56:05.261775+0000","last_peered":"2024-04-17T09:56:05.261775+0000","last_clean":"2024-04-17T09:56:05.261775+0000","last_became_active":"2024-04-17T09:56:02.216070+0000","last_became_peered":"2024-04-17T09:56:02.216070+0000","last_unstale":"2024-04-17T09:56:05.261775+0000","last_undegraded":"2024-04-17T09:56:05.261775+0000","last_fullsized":"2024-04-17T09:56:05.261775+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:46:26.870103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260688+0000","last_change":"2024-04-17T09:56:02.214345+0000","last_active":"2024-04-17T09:56:05.260688+0000","last_peered":"2024-04-17T09:56:05.260688+0000","last_clean":"2024-04-17T09:56:05.260688+0000","last_became_active":"2024-04-17T09:56:02.213946+0000","last_became_peered":"2024-04-17T09:56:02.213946+0000","last_unstale":"2024-04-17T09:56:05.260688+0000","last_undegraded":"2024-04-17T09:56:05.260688+0000","last_fullsized":"2024-04-17T09:56:05.260688+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:35:18.947071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203578+0000","last_change":"2024-04-17T09:56:02.215807+0000","last_active":"2024-04-17T09:56:03.203578+0000","last_peered":"2024-04-17T09:56:03.203578+0000","last_clean":"2024-04-17T09:56:03.203578+0000","last_became_active":"2024-04-17T09:56:02.215224+0000","last_became_peered":"2024-04-17T09:56:02.215224+0000","last_unstale":"2024-04-17T09:56:03.203578+0000","last_undegraded":"2024-04-17T09:56:03.203578+0000","last_fullsized":"2024-04-17T09:56:03.203578+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:55:47.229362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.360883+0000","last_change":"2024-04-17T09:56:02.219425+0000","last_active":"2024-04-17T09:56:03.360883+0000","last_peered":"2024-04-17T09:56:03.360883+0000","last_clean":"2024-04-17T09:56:03.360883+0000","last_became_active":"2024-04-17T09:56:02.219113+0000","last_became_peered":"2024-04-17T09:56:02.219113+0000","last_unstale":"2024-04-17T09:56:03.360883+0000","last_undegraded":"2024-04-17T09:56:03.360883+0000","last_fullsized":"2024-04-17T09:56:03.360883+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:54:36.189192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261169+0000","last_change":"2024-04-17T09:56:02.220120+0000","last_active":"2024-04-17T09:56:05.261169+0000","last_peered":"2024-04-17T09:56:05.261169+0000","last_clean":"2024-04-17T09:56:05.261169+0000","last_became_active":"2024-04-17T09:56:02.219824+0000","last_became_peered":"2024-04-17T09:56:02.219824+0000","last_unstale":"2024-04-17T09:56:05.261169+0000","last_undegraded":"2024-04-17T09:56:05.261169+0000","last_fullsized":"2024-04-17T09:56:05.261169+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:32:39.633696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197587+0000","last_change":"2024-04-17T09:56:02.215021+0000","last_active":"2024-04-17T09:56:03.197587+0000","last_peered":"2024-04-17T09:56:03.197587+0000","last_clean":"2024-04-17T09:56:03.197587+0000","last_became_active":"2024-04-17T09:56:02.214653+0000","last_became_peered":"2024-04-17T09:56:02.214653+0000","last_unstale":"2024-04-17T09:56:03.197587+0000","last_undegraded":"2024-04-17T09:56:03.197587+0000","last_fullsized":"2024-04-17T09:56:03.197587+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:00:41.794977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.360669+0000","last_change":"2024-04-17T09:56:02.226322+0000","last_active":"2024-04-17T09:56:03.360669+0000","last_peered":"2024-04-17T09:56:03.360669+0000","last_clean":"2024-04-17T09:56:03.360669+0000","last_became_active":"2024-04-17T09:56:02.226061+0000","last_became_peered":"2024-04-17T09:56:02.226061+0000","last_unstale":"2024-04-17T09:56:03.360669+0000","last_undegraded":"2024-04-17T09:56:03.360669+0000","last_fullsized":"2024-04-17T09:56:03.360669+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:36:06.628580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262401+0000","last_change":"2024-04-17T09:56:02.227824+0000","last_active":"2024-04-17T09:56:05.262401+0000","last_peered":"2024-04-17T09:56:05.262401+0000","last_clean":"2024-04-17T09:56:05.262401+0000","last_became_active":"2024-04-17T09:56:02.226305+0000","last_became_peered":"2024-04-17T09:56:02.226305+0000","last_unstale":"2024-04-17T09:56:05.262401+0000","last_undegraded":"2024-04-17T09:56:05.262401+0000","last_fullsized":"2024-04-17T09:56:05.262401+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:59:28.074910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260065+0000","last_change":"2024-04-17T09:56:02.209581+0000","last_active":"2024-04-17T09:56:05.260065+0000","last_peered":"2024-04-17T09:56:05.260065+0000","last_clean":"2024-04-17T09:56:05.260065+0000","last_became_active":"2024-04-17T09:56:02.209325+0000","last_became_peered":"2024-04-17T09:56:02.209325+0000","last_unstale":"2024-04-17T09:56:05.260065+0000","last_undegraded":"2024-04-17T09:56:05.260065+0000","last_fullsized":"2024-04-17T09:56:05.260065+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:32:39.633696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.360804+0000","last_change":"2024-04-17T09:56:02.219449+0000","last_active":"2024-04-17T09:56:03.360804+0000","last_peered":"2024-04-17T09:56:03.360804+0000","last_clean":"2024-04-17T09:56:03.360804+0000","last_became_active":"2024-04-17T09:56:02.219178+0000","last_became_peered":"2024-04-17T09:56:02.219178+0000","last_unstale":"2024-04-17T09:56:03.360804+0000","last_undegraded":"2024-04-17T09:56:03.360804+0000","last_fullsized":"2024-04-17T09:56:03.360804+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:05:04.703617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197005+0000","last_change":"2024-04-17T09:56:02.225982+0000","last_active":"2024-04-17T09:56:03.197005+0000","last_peered":"2024-04-17T09:56:03.197005+0000","last_clean":"2024-04-17T09:56:03.197005+0000","last_became_active":"2024-04-17T09:56:02.223371+0000","last_became_peered":"2024-04-17T09:56:02.223371+0000","last_unstale":"2024-04-17T09:56:03.197005+0000","last_undegraded":"2024-04-17T09:56:03.197005+0000","last_fullsized":"2024-04-17T09:56:03.197005+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:48:17.886969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197834+0000","last_change":"2024-04-17T09:56:02.224842+0000","last_active":"2024-04-17T09:56:03.197834+0000","last_peered":"2024-04-17T09:56:03.197834+0000","last_clean":"2024-04-17T09:56:03.197834+0000","last_became_active":"2024-04-17T09:56:02.224545+0000","last_became_peered":"2024-04-17T09:56:02.224545+0000","last_unstale":"2024-04-17T09:56:03.197834+0000","last_undegraded":"2024-04-17T09:56:03.197834+0000","last_fullsized":"2024-04-17T09:56:03.197834+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:57.566831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"56'4","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197875+0000","last_change":"2024-04-17T09:56:01.206168+0000","last_active":"2024-04-17T09:56:03.197875+0000","last_peered":"2024-04-17T09:56:03.197875+0000","last_clean":"2024-04-17T09:56:03.197875+0000","last_became_active":"2024-04-17T09:56:01.204386+0000","last_became_peered":"2024-04-17T09:56:01.204386+0000","last_unstale":"2024-04-17T09:56:03.197875+0000","last_undegraded":"2024-04-17T09:56:03.197875+0000","last_fullsized":"2024-04-17T09:56:03.197875+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:34.492070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262619+0000","last_change":"2024-04-17T09:56:02.222971+0000","last_active":"2024-04-17T09:56:05.262619+0000","last_peered":"2024-04-17T09:56:05.262619+0000","last_clean":"2024-04-17T09:56:05.262619+0000","last_became_active":"2024-04-17T09:56:02.222712+0000","last_became_peered":"2024-04-17T09:56:02.222712+0000","last_unstale":"2024-04-17T09:56:05.262619+0000","last_undegraded":"2024-04-17T09:56:05.262619+0000","last_fullsized":"2024-04-17T09:56:05.262619+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:47.438505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261926+0000","last_change":"2024-04-17T09:56:01.205411+0000","last_active":"2024-04-17T09:56:05.261926+0000","last_peered":"2024-04-17T09:56:05.261926+0000","last_clean":"2024-04-17T09:56:05.261926+0000","last_became_active":"2024-04-17T09:56:01.204027+0000","last_became_peered":"2024-04-17T09:56:01.204027+0000","last_unstale":"2024-04-17T09:56:05.261926+0000","last_undegraded":"2024-04-17T09:56:05.261926+0000","last_fullsized":"2024-04-17T09:56:05.261926+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:20:59.234480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197603+0000","last_change":"2024-04-17T09:56:02.234199+0000","last_active":"2024-04-17T09:56:03.197603+0000","last_peered":"2024-04-17T09:56:03.197603+0000","last_clean":"2024-04-17T09:56:03.197603+0000","last_became_active":"2024-04-17T09:56:02.233686+0000","last_became_peered":"2024-04-17T09:56:02.233686+0000","last_unstale":"2024-04-17T09:56:03.197603+0000","last_undegraded":"2024-04-17T09:56:03.197603+0000","last_fullsized":"2024-04-17T09:56:03.197603+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:08.333973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197902+0000","last_change":"2024-04-17T09:56:01.201154+0000","last_active":"2024-04-17T09:56:03.197902+0000","last_peered":"2024-04-17T09:56:03.197902+0000","last_clean":"2024-04-17T09:56:03.197902+0000","last_became_active":"2024-04-17T09:56:01.200926+0000","last_became_peered":"2024-04-17T09:56:01.200926+0000","last_unstale":"2024-04-17T09:56:03.197902+0000","last_undegraded":"2024-04-17T09:56:03.197902+0000","last_fullsized":"2024-04-17T09:56:03.197902+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:41:22.649311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197793+0000","last_change":"2024-04-17T09:56:02.230469+0000","last_active":"2024-04-17T09:56:03.197793+0000","last_peered":"2024-04-17T09:56:03.197793+0000","last_clean":"2024-04-17T09:56:03.197793+0000","last_became_active":"2024-04-17T09:56:02.229210+0000","last_became_peered":"2024-04-17T09:56:02.229210+0000","last_unstale":"2024-04-17T09:56:03.197793+0000","last_undegraded":"2024-04-17T09:56:03.197793+0000","last_fullsized":"2024-04-17T09:56:03.197793+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:58:25.393385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260914+0000","last_change":"2024-04-17T09:56:01.191420+0000","last_active":"2024-04-17T09:56:05.260914+0000","last_peered":"2024-04-17T09:56:05.260914+0000","last_clean":"2024-04-17T09:56:05.260914+0000","last_became_active":"2024-04-17T09:56:01.191177+0000","last_became_peered":"2024-04-17T09:56:01.191177+0000","last_unstale":"2024-04-17T09:56:05.260914+0000","last_undegraded":"2024-04-17T09:56:05.260914+0000","last_fullsized":"2024-04-17T09:56:05.260914+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:46.264824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261922+0000","last_change":"2024-04-17T09:56:02.220667+0000","last_active":"2024-04-17T09:56:05.261922+0000","last_peered":"2024-04-17T09:56:05.261922+0000","last_clean":"2024-04-17T09:56:05.261922+0000","last_became_active":"2024-04-17T09:56:02.216557+0000","last_became_peered":"2024-04-17T09:56:02.216557+0000","last_unstale":"2024-04-17T09:56:05.261922+0000","last_undegraded":"2024-04-17T09:56:05.261922+0000","last_fullsized":"2024-04-17T09:56:05.261922+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:18:25.708772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203408+0000","last_change":"2024-04-17T09:56:01.206115+0000","last_active":"2024-04-17T09:56:03.203408+0000","last_peered":"2024-04-17T09:56:03.203408+0000","last_clean":"2024-04-17T09:56:03.203408+0000","last_became_active":"2024-04-17T09:56:01.205818+0000","last_became_peered":"2024-04-17T09:56:01.205818+0000","last_unstale":"2024-04-17T09:56:03.203408+0000","last_undegraded":"2024-04-17T09:56:03.203408+0000","last_fullsized":"2024-04-17T09:56:03.203408+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:53.672442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261574+0000","last_change":"2024-04-17T09:56:02.216255+0000","last_active":"2024-04-17T09:56:05.261574+0000","last_peered":"2024-04-17T09:56:05.261574+0000","last_clean":"2024-04-17T09:56:05.261574+0000","last_became_active":"2024-04-17T09:56:02.215855+0000","last_became_peered":"2024-04-17T09:56:02.215855+0000","last_unstale":"2024-04-17T09:56:05.261574+0000","last_undegraded":"2024-04-17T09:56:05.261574+0000","last_fullsized":"2024-04-17T09:56:05.261574+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:54:33.483886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197353+0000","last_change":"2024-04-17T09:56:01.206766+0000","last_active":"2024-04-17T09:56:03.197353+0000","last_peered":"2024-04-17T09:56:03.197353+0000","last_clean":"2024-04-17T09:56:03.197353+0000","last_became_active":"2024-04-17T09:56:01.206438+0000","last_became_peered":"2024-04-17T09:56:01.206438+0000","last_unstale":"2024-04-17T09:56:03.197353+0000","last_undegraded":"2024-04-17T09:56:03.197353+0000","last_fullsized":"2024-04-17T09:56:03.197353+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:40:41.135595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203266+0000","last_change":"2024-04-17T09:56:02.208267+0000","last_active":"2024-04-17T09:56:03.203266+0000","last_peered":"2024-04-17T09:56:03.203266+0000","last_clean":"2024-04-17T09:56:03.203266+0000","last_became_active":"2024-04-17T09:56:02.207427+0000","last_became_peered":"2024-04-17T09:56:02.207427+0000","last_unstale":"2024-04-17T09:56:03.203266+0000","last_undegraded":"2024-04-17T09:56:03.203266+0000","last_fullsized":"2024-04-17T09:56:03.203266+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:46:36.134877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.259829+0000","last_change":"2024-04-17T09:56:01.194530+0000","last_active":"2024-04-17T09:56:05.259829+0000","last_peered":"2024-04-17T09:56:05.259829+0000","last_clean":"2024-04-17T09:56:05.259829+0000","last_became_active":"2024-04-17T09:56:01.194139+0000","last_became_peered":"2024-04-17T09:56:01.194139+0000","last_unstale":"2024-04-17T09:56:05.259829+0000","last_undegraded":"2024-04-17T09:56:05.259829+0000","last_fullsized":"2024-04-17T09:56:05.259829+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:59.853192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261634+0000","last_change":"2024-04-17T09:56:02.204678+0000","last_active":"2024-04-17T09:56:05.261634+0000","last_peered":"2024-04-17T09:56:05.261634+0000","last_clean":"2024-04-17T09:56:05.261634+0000","last_became_active":"2024-04-17T09:56:02.204378+0000","last_became_peered":"2024-04-17T09:56:02.204378+0000","last_unstale":"2024-04-17T09:56:05.261634+0000","last_undegraded":"2024-04-17T09:56:05.261634+0000","last_fullsized":"2024-04-17T09:56:05.261634+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:13:25.794083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203390+0000","last_change":"2024-04-17T09:56:01.197166+0000","last_active":"2024-04-17T09:56:03.203390+0000","last_peered":"2024-04-17T09:56:03.203390+0000","last_clean":"2024-04-17T09:56:03.203390+0000","last_became_active":"2024-04-17T09:56:01.196416+0000","last_became_peered":"2024-04-17T09:56:01.196416+0000","last_unstale":"2024-04-17T09:56:03.203390+0000","last_undegraded":"2024-04-17T09:56:03.203390+0000","last_fullsized":"2024-04-17T09:56:03.203390+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:58:47.664590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203664+0000","last_change":"2024-04-17T09:56:02.222098+0000","last_active":"2024-04-17T09:56:03.203664+0000","last_peered":"2024-04-17T09:56:03.203664+0000","last_clean":"2024-04-17T09:56:03.203664+0000","last_became_active":"2024-04-17T09:56:02.221642+0000","last_became_peered":"2024-04-17T09:56:02.221642+0000","last_unstale":"2024-04-17T09:56:03.203664+0000","last_undegraded":"2024-04-17T09:56:03.203664+0000","last_fullsized":"2024-04-17T09:56:03.203664+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:02:18.696690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260413+0000","last_change":"2024-04-17T09:56:01.203745+0000","last_active":"2024-04-17T09:56:05.260413+0000","last_peered":"2024-04-17T09:56:05.260413+0000","last_clean":"2024-04-17T09:56:05.260413+0000","last_became_active":"2024-04-17T09:56:01.203487+0000","last_became_peered":"2024-04-17T09:56:01.203487+0000","last_unstale":"2024-04-17T09:56:05.260413+0000","last_undegraded":"2024-04-17T09:56:05.260413+0000","last_fullsized":"2024-04-17T09:56:05.260413+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:48:06.817655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260492+0000","last_change":"2024-04-17T09:56:02.219782+0000","last_active":"2024-04-17T09:56:05.260492+0000","last_peered":"2024-04-17T09:56:05.260492+0000","last_clean":"2024-04-17T09:56:05.260492+0000","last_became_active":"2024-04-17T09:56:02.219409+0000","last_became_peered":"2024-04-17T09:56:02.219409+0000","last_unstale":"2024-04-17T09:56:05.260492+0000","last_undegraded":"2024-04-17T09:56:05.260492+0000","last_fullsized":"2024-04-17T09:56:05.260492+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:10:01.305797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262086+0000","last_change":"2024-04-17T09:56:01.208993+0000","last_active":"2024-04-17T09:56:05.262086+0000","last_peered":"2024-04-17T09:56:05.262086+0000","last_clean":"2024-04-17T09:56:05.262086+0000","last_became_active":"2024-04-17T09:56:01.208722+0000","last_became_peered":"2024-04-17T09:56:01.208722+0000","last_unstale":"2024-04-17T09:56:05.262086+0000","last_undegraded":"2024-04-17T09:56:05.262086+0000","last_fullsized":"2024-04-17T09:56:05.262086+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:55:10.303568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262017+0000","last_change":"2024-04-17T09:56:02.217472+0000","last_active":"2024-04-17T09:56:05.262017+0000","last_peered":"2024-04-17T09:56:05.262017+0000","last_clean":"2024-04-17T09:56:05.262017+0000","last_became_active":"2024-04-17T09:56:02.215926+0000","last_became_peered":"2024-04-17T09:56:02.215926+0000","last_unstale":"2024-04-17T09:56:05.262017+0000","last_undegraded":"2024-04-17T09:56:05.262017+0000","last_fullsized":"2024-04-17T09:56:05.262017+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:46:36.134877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"56'3","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260945+0000","last_change":"2024-04-17T09:56:01.197552+0000","last_active":"2024-04-17T09:56:05.260945+0000","last_peered":"2024-04-17T09:56:05.260945+0000","last_clean":"2024-04-17T09:56:05.260945+0000","last_became_active":"2024-04-17T09:56:01.197216+0000","last_became_peered":"2024-04-17T09:56:01.197216+0000","last_unstale":"2024-04-17T09:56:05.260945+0000","last_undegraded":"2024-04-17T09:56:05.260945+0000","last_fullsized":"2024-04-17T09:56:05.260945+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:40:04.394560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260822+0000","last_change":"2024-04-17T09:56:02.222301+0000","last_active":"2024-04-17T09:56:05.260822+0000","last_peered":"2024-04-17T09:56:05.260822+0000","last_clean":"2024-04-17T09:56:05.260822+0000","last_became_active":"2024-04-17T09:56:02.221854+0000","last_became_peered":"2024-04-17T09:56:02.221854+0000","last_unstale":"2024-04-17T09:56:05.260822+0000","last_undegraded":"2024-04-17T09:56:05.260822+0000","last_fullsized":"2024-04-17T09:56:05.260822+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:15:55.836315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"56'6","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203761+0000","last_change":"2024-04-17T09:56:01.206040+0000","last_active":"2024-04-17T09:56:03.203761+0000","last_peered":"2024-04-17T09:56:03.203761+0000","last_clean":"2024-04-17T09:56:03.203761+0000","last_became_active":"2024-04-17T09:56:01.205744+0000","last_became_peered":"2024-04-17T09:56:01.205744+0000","last_unstale":"2024-04-17T09:56:03.203761+0000","last_undegraded":"2024-04-17T09:56:03.203761+0000","last_fullsized":"2024-04-17T09:56:03.203761+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:55:07.841245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260001+0000","last_change":"2024-04-17T09:56:02.222193+0000","last_active":"2024-04-17T09:56:05.260001+0000","last_peered":"2024-04-17T09:56:05.260001+0000","last_clean":"2024-04-17T09:56:05.260001+0000","last_became_active":"2024-04-17T09:56:02.221951+0000","last_became_peered":"2024-04-17T09:56:02.221951+0000","last_unstale":"2024-04-17T09:56:05.260001+0000","last_undegraded":"2024-04-17T09:56:05.260001+0000","last_fullsized":"2024-04-17T09:56:05.260001+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:37:03.542306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262197+0000","last_change":"2024-04-17T09:56:01.208069+0000","last_active":"2024-04-17T09:56:05.262197+0000","last_peered":"2024-04-17T09:56:05.262197+0000","last_clean":"2024-04-17T09:56:05.262197+0000","last_became_active":"2024-04-17T09:56:01.207756+0000","last_became_peered":"2024-04-17T09:56:01.207756+0000","last_unstale":"2024-04-17T09:56:05.262197+0000","last_undegraded":"2024-04-17T09:56:05.262197+0000","last_fullsized":"2024-04-17T09:56:05.262197+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:04:54.648358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260228+0000","last_change":"2024-04-17T09:56:02.204783+0000","last_active":"2024-04-17T09:56:05.260228+0000","last_peered":"2024-04-17T09:56:05.260228+0000","last_clean":"2024-04-17T09:56:05.260228+0000","last_became_active":"2024-04-17T09:56:02.204482+0000","last_became_peered":"2024-04-17T09:56:02.204482+0000","last_unstale":"2024-04-17T09:56:05.260228+0000","last_undegraded":"2024-04-17T09:56:05.260228+0000","last_fullsized":"2024-04-17T09:56:05.260228+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:52:40.143883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.202981+0000","last_change":"2024-04-17T09:56:01.191609+0000","last_active":"2024-04-17T09:56:03.202981+0000","last_peered":"2024-04-17T09:56:03.202981+0000","last_clean":"2024-04-17T09:56:03.202981+0000","last_became_active":"2024-04-17T09:56:01.191314+0000","last_became_peered":"2024-04-17T09:56:01.191314+0000","last_unstale":"2024-04-17T09:56:03.202981+0000","last_undegraded":"2024-04-17T09:56:03.202981+0000","last_fullsized":"2024-04-17T09:56:03.202981+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:29.004521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203271+0000","last_change":"2024-04-17T09:56:02.210009+0000","last_active":"2024-04-17T09:56:03.203271+0000","last_peered":"2024-04-17T09:56:03.203271+0000","last_clean":"2024-04-17T09:56:03.203271+0000","last_became_active":"2024-04-17T09:56:02.209655+0000","last_became_peered":"2024-04-17T09:56:02.209655+0000","last_unstale":"2024-04-17T09:56:03.203271+0000","last_undegraded":"2024-04-17T09:56:03.203271+0000","last_fullsized":"2024-04-17T09:56:03.203271+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:23.603548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.259952+0000","last_change":"2024-04-17T09:56:01.189043+0000","last_active":"2024-04-17T09:56:05.259952+0000","last_peered":"2024-04-17T09:56:05.259952+0000","last_clean":"2024-04-17T09:56:05.259952+0000","last_became_active":"2024-04-17T09:56:01.188778+0000","last_became_peered":"2024-04-17T09:56:01.188778+0000","last_unstale":"2024-04-17T09:56:05.259952+0000","last_undegraded":"2024-04-17T09:56:05.259952+0000","last_fullsized":"2024-04-17T09:56:05.259952+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:41:45.654626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260707+0000","last_change":"2024-04-17T09:56:02.214563+0000","last_active":"2024-04-17T09:56:05.260707+0000","last_peered":"2024-04-17T09:56:05.260707+0000","last_clean":"2024-04-17T09:56:05.260707+0000","last_became_active":"2024-04-17T09:56:02.213985+0000","last_became_peered":"2024-04-17T09:56:02.213985+0000","last_unstale":"2024-04-17T09:56:05.260707+0000","last_undegraded":"2024-04-17T09:56:05.260707+0000","last_fullsized":"2024-04-17T09:56:05.260707+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:27:28.602507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"56'8","reported_seq":22,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.259829+0000","last_change":"2024-04-17T09:56:01.196179+0000","last_active":"2024-04-17T09:56:05.259829+0000","last_peered":"2024-04-17T09:56:05.259829+0000","last_clean":"2024-04-17T09:56:05.259829+0000","last_became_active":"2024-04-17T09:56:01.195954+0000","last_became_peered":"2024-04-17T09:56:01.195954+0000","last_unstale":"2024-04-17T09:56:05.259829+0000","last_undegraded":"2024-04-17T09:56:05.259829+0000","last_fullsized":"2024-04-17T09:56:05.259829+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:41:45.654626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260479+0000","last_change":"2024-04-17T09:56:02.210099+0000","last_active":"2024-04-17T09:56:05.260479+0000","last_peered":"2024-04-17T09:56:05.260479+0000","last_clean":"2024-04-17T09:56:05.260479+0000","last_became_active":"2024-04-17T09:56:02.209740+0000","last_became_peered":"2024-04-17T09:56:02.209740+0000","last_unstale":"2024-04-17T09:56:05.260479+0000","last_undegraded":"2024-04-17T09:56:05.260479+0000","last_fullsized":"2024-04-17T09:56:05.260479+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:05:05.510667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262278+0000","last_change":"2024-04-17T09:56:01.207663+0000","last_active":"2024-04-17T09:56:05.262278+0000","last_peered":"2024-04-17T09:56:05.262278+0000","last_clean":"2024-04-17T09:56:05.262278+0000","last_became_active":"2024-04-17T09:56:01.206630+0000","last_became_peered":"2024-04-17T09:56:01.206630+0000","last_unstale":"2024-04-17T09:56:05.262278+0000","last_undegraded":"2024-04-17T09:56:05.262278+0000","last_fullsized":"2024-04-17T09:56:05.262278+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:53:23.660304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261244+0000","last_change":"2024-04-17T09:56:02.223670+0000","last_active":"2024-04-17T09:56:05.261244+0000","last_peered":"2024-04-17T09:56:05.261244+0000","last_clean":"2024-04-17T09:56:05.261244+0000","last_became_active":"2024-04-17T09:56:02.223378+0000","last_became_peered":"2024-04-17T09:56:02.223378+0000","last_unstale":"2024-04-17T09:56:05.261244+0000","last_undegraded":"2024-04-17T09:56:05.261244+0000","last_fullsized":"2024-04-17T09:56:05.261244+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:53:24.666275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197845+0000","last_change":"2024-04-17T09:56:01.208863+0000","last_active":"2024-04-17T09:56:03.197845+0000","last_peered":"2024-04-17T09:56:03.197845+0000","last_clean":"2024-04-17T09:56:03.197845+0000","last_became_active":"2024-04-17T09:56:01.208605+0000","last_became_peered":"2024-04-17T09:56:01.208605+0000","last_unstale":"2024-04-17T09:56:03.197845+0000","last_undegraded":"2024-04-17T09:56:03.197845+0000","last_fullsized":"2024-04-17T09:56:03.197845+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:36.056444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261792+0000","last_change":"2024-04-17T09:56:02.210781+0000","last_active":"2024-04-17T09:56:05.261792+0000","last_peered":"2024-04-17T09:56:05.261792+0000","last_clean":"2024-04-17T09:56:05.261792+0000","last_became_active":"2024-04-17T09:56:02.210423+0000","last_became_peered":"2024-04-17T09:56:02.210423+0000","last_unstale":"2024-04-17T09:56:05.261792+0000","last_undegraded":"2024-04-17T09:56:05.261792+0000","last_fullsized":"2024-04-17T09:56:05.261792+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:07:34.467885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261230+0000","last_change":"2024-04-17T09:56:01.189844+0000","last_active":"2024-04-17T09:56:05.261230+0000","last_peered":"2024-04-17T09:56:05.261230+0000","last_clean":"2024-04-17T09:56:05.261230+0000","last_became_active":"2024-04-17T09:56:01.189601+0000","last_became_peered":"2024-04-17T09:56:01.189601+0000","last_unstale":"2024-04-17T09:56:05.261230+0000","last_undegraded":"2024-04-17T09:56:05.261230+0000","last_fullsized":"2024-04-17T09:56:05.261230+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:09:38.167038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260885+0000","last_change":"2024-04-17T09:56:02.222099+0000","last_active":"2024-04-17T09:56:05.260885+0000","last_peered":"2024-04-17T09:56:05.260885+0000","last_clean":"2024-04-17T09:56:05.260885+0000","last_became_active":"2024-04-17T09:56:02.221484+0000","last_became_peered":"2024-04-17T09:56:02.221484+0000","last_unstale":"2024-04-17T09:56:05.260885+0000","last_undegraded":"2024-04-17T09:56:05.260885+0000","last_fullsized":"2024-04-17T09:56:05.260885+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:34:44.623141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197523+0000","last_change":"2024-04-17T09:56:01.203071+0000","last_active":"2024-04-17T09:56:03.197523+0000","last_peered":"2024-04-17T09:56:03.197523+0000","last_clean":"2024-04-17T09:56:03.197523+0000","last_became_active":"2024-04-17T09:56:01.202764+0000","last_became_peered":"2024-04-17T09:56:01.202764+0000","last_unstale":"2024-04-17T09:56:03.197523+0000","last_undegraded":"2024-04-17T09:56:03.197523+0000","last_fullsized":"2024-04-17T09:56:03.197523+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:22.597577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260935+0000","last_change":"2024-04-17T09:56:02.222320+0000","last_active":"2024-04-17T09:56:05.260935+0000","last_peered":"2024-04-17T09:56:05.260935+0000","last_clean":"2024-04-17T09:56:05.260935+0000","last_became_active":"2024-04-17T09:56:02.222071+0000","last_became_peered":"2024-04-17T09:56:02.222071+0000","last_unstale":"2024-04-17T09:56:05.260935+0000","last_undegraded":"2024-04-17T09:56:05.260935+0000","last_fullsized":"2024-04-17T09:56:05.260935+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:00.240451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262449+0000","last_change":"2024-04-17T09:56:01.200432+0000","last_active":"2024-04-17T09:56:05.262449+0000","last_peered":"2024-04-17T09:56:05.262449+0000","last_clean":"2024-04-17T09:56:05.262449+0000","last_became_active":"2024-04-17T09:56:01.200159+0000","last_became_peered":"2024-04-17T09:56:01.200159+0000","last_unstale":"2024-04-17T09:56:05.262449+0000","last_undegraded":"2024-04-17T09:56:05.262449+0000","last_fullsized":"2024-04-17T09:56:05.262449+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:32:38.627725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197598+0000","last_change":"2024-04-17T09:56:02.211557+0000","last_active":"2024-04-17T09:56:03.197598+0000","last_peered":"2024-04-17T09:56:03.197598+0000","last_clean":"2024-04-17T09:56:03.197598+0000","last_became_active":"2024-04-17T09:56:02.211186+0000","last_became_peered":"2024-04-17T09:56:02.211186+0000","last_unstale":"2024-04-17T09:56:03.197598+0000","last_undegraded":"2024-04-17T09:56:03.197598+0000","last_fullsized":"2024-04-17T09:56:03.197598+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:42:46.838421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262265+0000","last_change":"2024-04-17T09:56:01.200296+0000","last_active":"2024-04-17T09:56:05.262265+0000","last_peered":"2024-04-17T09:56:05.262265+0000","last_clean":"2024-04-17T09:56:05.262265+0000","last_became_active":"2024-04-17T09:56:01.200020+0000","last_became_peered":"2024-04-17T09:56:01.200020+0000","last_unstale":"2024-04-17T09:56:05.262265+0000","last_undegraded":"2024-04-17T09:56:05.262265+0000","last_fullsized":"2024-04-17T09:56:05.262265+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:54:35.183221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197057+0000","last_change":"2024-04-17T09:56:02.211632+0000","last_active":"2024-04-17T09:56:03.197057+0000","last_peered":"2024-04-17T09:56:03.197057+0000","last_clean":"2024-04-17T09:56:03.197057+0000","last_became_active":"2024-04-17T09:56:02.211357+0000","last_became_peered":"2024-04-17T09:56:02.211357+0000","last_unstale":"2024-04-17T09:56:03.197057+0000","last_undegraded":"2024-04-17T09:56:03.197057+0000","last_fullsized":"2024-04-17T09:56:03.197057+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:16:53.336176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.259831+0000","last_change":"2024-04-17T09:56:01.195917+0000","last_active":"2024-04-17T09:56:05.259831+0000","last_peered":"2024-04-17T09:56:05.259831+0000","last_clean":"2024-04-17T09:56:05.259831+0000","last_became_active":"2024-04-17T09:56:01.195682+0000","last_became_peered":"2024-04-17T09:56:01.195682+0000","last_unstale":"2024-04-17T09:56:05.259831+0000","last_undegraded":"2024-04-17T09:56:05.259831+0000","last_fullsized":"2024-04-17T09:56:05.259831+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:46.432533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261254+0000","last_change":"2024-04-17T09:56:02.218398+0000","last_active":"2024-04-17T09:56:05.261254+0000","last_peered":"2024-04-17T09:56:05.261254+0000","last_clean":"2024-04-17T09:56:05.261254+0000","last_became_active":"2024-04-17T09:56:02.218097+0000","last_became_peered":"2024-04-17T09:56:02.218097+0000","last_unstale":"2024-04-17T09:56:05.261254+0000","last_undegraded":"2024-04-17T09:56:05.261254+0000","last_fullsized":"2024-04-17T09:56:05.261254+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:52:40.143883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262592+0000","last_change":"2024-04-17T09:56:01.204766+0000","last_active":"2024-04-17T09:56:05.262592+0000","last_peered":"2024-04-17T09:56:05.262592+0000","last_clean":"2024-04-17T09:56:05.262592+0000","last_became_active":"2024-04-17T09:56:01.204490+0000","last_became_peered":"2024-04-17T09:56:01.204490+0000","last_unstale":"2024-04-17T09:56:05.262592+0000","last_undegraded":"2024-04-17T09:56:05.262592+0000","last_fullsized":"2024-04-17T09:56:05.262592+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:14:07.937430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261265+0000","last_change":"2024-04-17T09:56:02.216520+0000","last_active":"2024-04-17T09:56:05.261265+0000","last_peered":"2024-04-17T09:56:05.261265+0000","last_clean":"2024-04-17T09:56:05.261265+0000","last_became_active":"2024-04-17T09:56:02.216195+0000","last_became_peered":"2024-04-17T09:56:02.216195+0000","last_unstale":"2024-04-17T09:56:05.261265+0000","last_undegraded":"2024-04-17T09:56:05.261265+0000","last_fullsized":"2024-04-17T09:56:05.261265+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:05:05.510667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"57'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203049+0000","last_change":"2024-04-17T09:56:01.202823+0000","last_active":"2024-04-17T09:56:03.203049+0000","last_peered":"2024-04-17T09:56:03.203049+0000","last_clean":"2024-04-17T09:56:03.203049+0000","last_became_active":"2024-04-17T09:56:01.202563+0000","last_became_peered":"2024-04-17T09:56:01.202563+0000","last_unstale":"2024-04-17T09:56:03.203049+0000","last_undegraded":"2024-04-17T09:56:03.203049+0000","last_fullsized":"2024-04-17T09:56:03.203049+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:34:57.389627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262171+0000","last_change":"2024-04-17T09:56:02.215682+0000","last_active":"2024-04-17T09:56:05.262171+0000","last_peered":"2024-04-17T09:56:05.262171+0000","last_clean":"2024-04-17T09:56:05.262171+0000","last_became_active":"2024-04-17T09:56:02.215368+0000","last_became_peered":"2024-04-17T09:56:02.215368+0000","last_unstale":"2024-04-17T09:56:05.262171+0000","last_undegraded":"2024-04-17T09:56:05.262171+0000","last_fullsized":"2024-04-17T09:56:05.262171+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:55:47.229362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.202866+0000","last_change":"2024-04-17T09:56:01.198070+0000","last_active":"2024-04-17T09:56:03.202866+0000","last_peered":"2024-04-17T09:56:03.202866+0000","last_clean":"2024-04-17T09:56:03.202866+0000","last_became_active":"2024-04-17T09:56:01.197737+0000","last_became_peered":"2024-04-17T09:56:01.197737+0000","last_unstale":"2024-04-17T09:56:03.202866+0000","last_undegraded":"2024-04-17T09:56:03.202866+0000","last_fullsized":"2024-04-17T09:56:03.202866+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:28:26.047148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197782+0000","last_change":"2024-04-17T09:56:02.230708+0000","last_active":"2024-04-17T09:56:03.197782+0000","last_peered":"2024-04-17T09:56:03.197782+0000","last_clean":"2024-04-17T09:56:03.197782+0000","last_became_active":"2024-04-17T09:56:02.230207+0000","last_became_peered":"2024-04-17T09:56:02.230207+0000","last_unstale":"2024-04-17T09:56:03.197782+0000","last_undegraded":"2024-04-17T09:56:03.197782+0000","last_fullsized":"2024-04-17T09:56:03.197782+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:43:27.595946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261138+0000","last_change":"2024-04-17T09:56:01.197907+0000","last_active":"2024-04-17T09:56:05.261138+0000","last_peered":"2024-04-17T09:56:05.261138+0000","last_clean":"2024-04-17T09:56:05.261138+0000","last_became_active":"2024-04-17T09:56:01.197669+0000","last_became_peered":"2024-04-17T09:56:01.197669+0000","last_unstale":"2024-04-17T09:56:05.261138+0000","last_undegraded":"2024-04-17T09:56:05.261138+0000","last_fullsized":"2024-04-17T09:56:05.261138+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:48:06.817655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262071+0000","last_change":"2024-04-17T09:56:02.224786+0000","last_active":"2024-04-17T09:56:05.262071+0000","last_peered":"2024-04-17T09:56:05.262071+0000","last_clean":"2024-04-17T09:56:05.262071+0000","last_became_active":"2024-04-17T09:56:02.224468+0000","last_became_peered":"2024-04-17T09:56:02.224468+0000","last_unstale":"2024-04-17T09:56:05.262071+0000","last_undegraded":"2024-04-17T09:56:05.262071+0000","last_fullsized":"2024-04-17T09:56:05.262071+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:17:53.933023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261567+0000","last_change":"2024-04-17T09:56:01.186515+0000","last_active":"2024-04-17T09:56:05.261567+0000","last_peered":"2024-04-17T09:56:05.261567+0000","last_clean":"2024-04-17T09:56:05.261567+0000","last_became_active":"2024-04-17T09:56:01.186266+0000","last_became_peered":"2024-04-17T09:56:01.186266+0000","last_unstale":"2024-04-17T09:56:05.261567+0000","last_undegraded":"2024-04-17T09:56:05.261567+0000","last_fullsized":"2024-04-17T09:56:05.261567+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:27:04.257237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260715+0000","last_change":"2024-04-17T09:56:02.219403+0000","last_active":"2024-04-17T09:56:05.260715+0000","last_peered":"2024-04-17T09:56:05.260715+0000","last_clean":"2024-04-17T09:56:05.260715+0000","last_became_active":"2024-04-17T09:56:02.217760+0000","last_became_peered":"2024-04-17T09:56:02.217760+0000","last_unstale":"2024-04-17T09:56:05.260715+0000","last_undegraded":"2024-04-17T09:56:05.260715+0000","last_fullsized":"2024-04-17T09:56:05.260715+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:46.319880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203268+0000","last_change":"2024-04-17T09:56:01.207103+0000","last_active":"2024-04-17T09:56:03.203268+0000","last_peered":"2024-04-17T09:56:03.203268+0000","last_clean":"2024-04-17T09:56:03.203268+0000","last_became_active":"2024-04-17T09:56:01.206837+0000","last_became_peered":"2024-04-17T09:56:01.206837+0000","last_unstale":"2024-04-17T09:56:03.203268+0000","last_undegraded":"2024-04-17T09:56:03.203268+0000","last_fullsized":"2024-04-17T09:56:03.203268+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:28:36.102136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261326+0000","last_change":"2024-04-17T09:56:02.220640+0000","last_active":"2024-04-17T09:56:05.261326+0000","last_peered":"2024-04-17T09:56:05.261326+0000","last_clean":"2024-04-17T09:56:05.261326+0000","last_became_active":"2024-04-17T09:56:02.219659+0000","last_became_peered":"2024-04-17T09:56:02.219659+0000","last_unstale":"2024-04-17T09:56:05.261326+0000","last_undegraded":"2024-04-17T09:56:05.261326+0000","last_fullsized":"2024-04-17T09:56:05.261326+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:13:28.472859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261631+0000","last_change":"2024-04-17T09:56:01.203488+0000","last_active":"2024-04-17T09:56:05.261631+0000","last_peered":"2024-04-17T09:56:05.261631+0000","last_clean":"2024-04-17T09:56:05.261631+0000","last_became_active":"2024-04-17T09:56:01.199625+0000","last_became_peered":"2024-04-17T09:56:01.199625+0000","last_unstale":"2024-04-17T09:56:05.261631+0000","last_undegraded":"2024-04-17T09:56:05.261631+0000","last_fullsized":"2024-04-17T09:56:05.261631+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:23:44.149563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261403+0000","last_change":"2024-04-17T09:56:02.207578+0000","last_active":"2024-04-17T09:56:05.261403+0000","last_peered":"2024-04-17T09:56:05.261403+0000","last_clean":"2024-04-17T09:56:05.261403+0000","last_became_active":"2024-04-17T09:56:02.207285+0000","last_became_peered":"2024-04-17T09:56:02.207285+0000","last_unstale":"2024-04-17T09:56:05.261403+0000","last_undegraded":"2024-04-17T09:56:05.261403+0000","last_fullsized":"2024-04-17T09:56:05.261403+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:11:27.415411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262018+0000","last_change":"2024-04-17T09:56:01.200290+0000","last_active":"2024-04-17T09:56:05.262018+0000","last_peered":"2024-04-17T09:56:05.262018+0000","last_clean":"2024-04-17T09:56:05.262018+0000","last_became_active":"2024-04-17T09:56:01.199410+0000","last_became_peered":"2024-04-17T09:56:01.199410+0000","last_unstale":"2024-04-17T09:56:05.262018+0000","last_undegraded":"2024-04-17T09:56:05.262018+0000","last_fullsized":"2024-04-17T09:56:05.262018+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:37:02.536335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261911+0000","last_change":"2024-04-17T09:56:02.220646+0000","last_active":"2024-04-17T09:56:05.261911+0000","last_peered":"2024-04-17T09:56:05.261911+0000","last_clean":"2024-04-17T09:56:05.261911+0000","last_became_active":"2024-04-17T09:56:02.219995+0000","last_became_peered":"2024-04-17T09:56:02.219995+0000","last_unstale":"2024-04-17T09:56:05.261911+0000","last_undegraded":"2024-04-17T09:56:05.261911+0000","last_fullsized":"2024-04-17T09:56:05.261911+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:28:47.541595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.202812+0000","last_change":"2024-04-17T09:56:01.192727+0000","last_active":"2024-04-17T09:56:03.202812+0000","last_peered":"2024-04-17T09:56:03.202812+0000","last_clean":"2024-04-17T09:56:03.202812+0000","last_became_active":"2024-04-17T09:56:01.192476+0000","last_became_peered":"2024-04-17T09:56:01.192476+0000","last_unstale":"2024-04-17T09:56:03.202812+0000","last_undegraded":"2024-04-17T09:56:03.202812+0000","last_fullsized":"2024-04-17T09:56:03.202812+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:28:24.596000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":237,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.259942+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T09:56:05.259942+0000","last_peered":"2024-04-17T09:56:05.259942+0000","last_clean":"2024-04-17T09:56:05.259942+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T09:56:05.259942+0000","last_undegraded":"2024-04-17T09:56:05.259942+0000","last_fullsized":"2024-04-17T09:56:05.259942+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28260,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343580,"statfs":{"total":96636764160,"available":96607825920,"internally_reserved":0,"allocated":1265664,"data_stored":863788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":675840,"data_stored":279560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509480,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":671744,"data_stored":273912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31836,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":671744,"data_stored":273316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31852,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339988,"statfs":{"total":96636764160,"available":96604147712,"internally_reserved":0,"allocated":684032,"data_stored":279616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":667648,"data_stored":273350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":688128,"data_stored":281172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542184,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":872122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T09:56:10.908 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T09:56:11.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.056+0000 7fab79e00640 1 Processor -- start 2024-04-17T09:56:11.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.056+0000 7fab79e00640 1 -- start start 2024-04-17T09:56:11.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.056+0000 7fab79e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab7406f7c0 0x7fab7406fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:11.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.056+0000 7fab79e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab7415ec50 con 0x7fab7415e870 2024-04-17T09:56:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.056+0000 7fab79e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab740d0950 con 0x7fab740b0320 2024-04-17T09:56:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.056+0000 7fab79e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fab740cf810 con 0x7fab7406f7c0 2024-04-17T09:56:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.056+0000 7fab73400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab7406f7c0 0x7fab7406fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.056+0000 7fab72a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fab7415e870 0x7fab74162e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42808/0 (socket says 172.21.15.32:42808) 2024-04-17T09:56:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.056+0000 7fab72a00640 1 -- 172.21.15.32:0/2347441516 learned_addr learned my addr 172.21.15.32:0/2347441516 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.060+0000 7fab73400640 1 -- 172.21.15.32:0/2347441516 >> v1:172.21.15.144:6789/0 conn(0x7fab740b0320 legacy=0x7fab740d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.060+0000 7fab73400640 1 -- 172.21.15.32:0/2347441516 >> v1:172.21.15.32:6789/0 conn(0x7fab7415e870 legacy=0x7fab74162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:11.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.060+0000 7fab73400640 1 -- 172.21.15.32:0/2347441516 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab74067810 con 0x7fab7406f7c0 2024-04-17T09:56:11.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.060+0000 7fab72000640 1 -- 172.21.15.32:0/2347441516 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1611900268 0 0) 0x7fab7415ec50 con 0x7fab7415e870 2024-04-17T09:56:11.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.060+0000 7fab73400640 1 --2- 172.21.15.32:0/2347441516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab7406f7c0 0x7fab7406fc50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009d90 tx=0x7fab6c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed1d6bd498d79bb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:11.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.060+0000 7fab72000640 1 -- 172.21.15.32:0/2347441516 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c0b2070 con 0x7fab7406f7c0 2024-04-17T09:56:11.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.060+0000 7fab72000640 1 -- 172.21.15.32:0/2347441516 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab6c0a5830 con 0x7fab7406f7c0 2024-04-17T09:56:11.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.060+0000 7fab72000640 1 -- 172.21.15.32:0/2347441516 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c0a5c00 con 0x7fab7406f7c0 2024-04-17T09:56:11.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.060+0000 7fab79e00640 1 -- 172.21.15.32:0/2347441516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab7406f7c0 msgr2=0x7fab7406fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:11.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.060+0000 7fab79e00640 1 --2- 172.21.15.32:0/2347441516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab7406f7c0 0x7fab7406fc50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009d90 tx=0x7fab6c01c970 comp rx=0 tx=0).stop 2024-04-17T09:56:11.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.060+0000 7fab79e00640 1 -- 172.21.15.32:0/2347441516 shutdown_connections 2024-04-17T09:56:11.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.060+0000 7fab79e00640 1 --2- 172.21.15.32:0/2347441516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab7406f7c0 0x7fab7406fc50 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.060+0000 7fab79e00640 1 -- 172.21.15.32:0/2347441516 >> 172.21.15.32:0/2347441516 conn(0x7fab740d0c00 msgr2=0x7fab740d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:11.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.064+0000 7fab79e00640 1 -- 172.21.15.32:0/2347441516 shutdown_connections 2024-04-17T09:56:11.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.064+0000 7fab79e00640 1 -- 172.21.15.32:0/2347441516 wait complete. 2024-04-17T09:56:11.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.064+0000 7fab79e00640 1 Processor -- start 2024-04-17T09:56:11.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.064+0000 7fab79e00640 1 -- start start 2024-04-17T09:56:11.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.064+0000 7fab79e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab7406f7c0 0x7fab74152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:11.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.064+0000 7fab79e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab740b0320 0x7fab740b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:11.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.064+0000 7fab79e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab7415e870 0x7fab740b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:11.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.064+0000 7fab79e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fab7416c9b0 con 0x7fab7415e870 2024-04-17T09:56:11.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.064+0000 7fab79e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fab740cfdb0 con 0x7fab7406f7c0 2024-04-17T09:56:11.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.064+0000 7fab79e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fab7416cb30 con 0x7fab740b0320 2024-04-17T09:56:11.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.064+0000 7fab73400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab7415e870 0x7fab740b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:11.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.064+0000 7fab73400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab7415e870 0x7fab740b5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46260/0 (socket says 172.21.15.32:46260) 2024-04-17T09:56:11.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.064+0000 7fab73400640 1 -- 172.21.15.32:0/3740787655 learned_addr learned my addr 172.21.15.32:0/3740787655 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:11.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.064+0000 7fab72a00640 1 --2- 172.21.15.32:0/3740787655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab7406f7c0 0x7fab74152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:11.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.064+0000 7fab6a000640 1 --2- 172.21.15.32:0/3740787655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab740b0320 0x7fab740b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:11.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.064+0000 7fab73400640 1 -- 172.21.15.32:0/3740787655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab740b0320 msgr2=0x7fab740b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:11.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.064+0000 7fab73400640 1 --2- 172.21.15.32:0/3740787655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab740b0320 0x7fab740b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.064+0000 7fab73400640 1 -- 172.21.15.32:0/3740787655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab7406f7c0 msgr2=0x7fab74152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:11.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.064+0000 7fab73400640 1 --2- 172.21.15.32:0/3740787655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab7406f7c0 0x7fab74152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.064+0000 7fab73400640 1 -- 172.21.15.32:0/3740787655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7415c2f0 con 0x7fab7415e870 2024-04-17T09:56:11.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.064+0000 7fab6a000640 1 --2- 172.21.15.32:0/3740787655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab740b0320 0x7fab740b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:11.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.064+0000 7fab72a00640 1 --2- 172.21.15.32:0/3740787655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab7406f7c0 0x7fab74152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:11.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.064+0000 7fab73400640 1 --2- 172.21.15.32:0/3740787655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab7415e870 0x7fab740b5440 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fab6c0b0040 tx=0x7fab6c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:11.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.068+0000 7fab6be00640 1 -- 172.21.15.32:0/3740787655 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c0b2040 con 0x7fab7415e870 2024-04-17T09:56:11.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.068+0000 7fab6be00640 1 -- 172.21.15.32:0/3740787655 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab6c0bf070 con 0x7fab7415e870 2024-04-17T09:56:11.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.068+0000 7fab79e00640 1 -- 172.21.15.32:0/3740787655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab7415c5d0 con 0x7fab7415e870 2024-04-17T09:56:11.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.068+0000 7fab6be00640 1 -- 172.21.15.32:0/3740787655 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c0ad900 con 0x7fab7415e870 2024-04-17T09:56:11.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.068+0000 7fab79e00640 1 -- 172.21.15.32:0/3740787655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab7415cb90 con 0x7fab7415e870 2024-04-17T09:56:11.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.068+0000 7fab79e00640 1 -- 172.21.15.32:0/3740787655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab74068a90 con 0x7fab7415e870 2024-04-17T09:56:11.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.068+0000 7fab6be00640 1 -- 172.21.15.32:0/3740787655 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fab6c0adaa0 con 0x7fab7415e870 2024-04-17T09:56:11.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.068+0000 7fab6be00640 1 --2- 172.21.15.32:0/3740787655 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab50068380 0x7fab5006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:11.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.068+0000 7fab6be00640 1 -- 172.21.15.32:0/3740787655 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fab6c121ce0 con 0x7fab7415e870 2024-04-17T09:56:11.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.068+0000 7fab72a00640 1 --2- 172.21.15.32:0/3740787655 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab50068380 0x7fab5006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:11.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.068+0000 7fab72a00640 1 --2- 172.21.15.32:0/3740787655 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab50068380 0x7fab5006a840 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fab64000f70 tx=0x7fab64007df0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:11.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.072+0000 7fab6be00640 1 -- 172.21.15.32:0/3740787655 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fab6c016560 con 0x7fab7415e870 2024-04-17T09:56:11.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.236+0000 7fab79e00640 1 -- 172.21.15.32:0/3740787655 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fab74073690 con 0x7fab50068380 2024-04-17T09:56:11.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.244+0000 7fab6be00640 1 -- 172.21.15.32:0/3740787655 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251942 (secure 0 0 0) 0x7fab74073690 con 0x7fab50068380 2024-04-17T09:56:11.248 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:11.249 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T09:56:11.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.248+0000 7fab79e00640 1 -- 172.21.15.32:0/3740787655 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab50068380 msgr2=0x7fab5006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:11.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.248+0000 7fab79e00640 1 --2- 172.21.15.32:0/3740787655 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab50068380 0x7fab5006a840 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fab64000f70 tx=0x7fab64007df0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.248+0000 7fab79e00640 1 -- 172.21.15.32:0/3740787655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab7415e870 msgr2=0x7fab740b5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:11.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.248+0000 7fab79e00640 1 --2- 172.21.15.32:0/3740787655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab7415e870 0x7fab740b5440 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fab6c0b0040 tx=0x7fab6c019710 comp rx=0 tx=0).stop 2024-04-17T09:56:11.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.248+0000 7fab79e00640 1 -- 172.21.15.32:0/3740787655 shutdown_connections 2024-04-17T09:56:11.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.248+0000 7fab79e00640 1 --2- 172.21.15.32:0/3740787655 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fab50068380 0x7fab5006a840 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.248+0000 7fab79e00640 1 --2- 172.21.15.32:0/3740787655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab740b0320 0x7fab740b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.248+0000 7fab79e00640 1 --2- 172.21.15.32:0/3740787655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab7415e870 0x7fab740b5440 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.248+0000 7fab79e00640 1 --2- 172.21.15.32:0/3740787655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab7406f7c0 0x7fab74152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.248+0000 7fab79e00640 1 -- 172.21.15.32:0/3740787655 >> 172.21.15.32:0/3740787655 conn(0x7fab740d0c00 msgr2=0x7fab740d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:11.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.248+0000 7fab79e00640 1 -- 172.21.15.32:0/3740787655 shutdown_connections 2024-04-17T09:56:11.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.248+0000 7fab79e00640 1 -- 172.21.15.32:0/3740787655 wait complete. 2024-04-17T09:56:11.269 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-04-17T09:56:09.322668+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243436,"kb_used_data":6456,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731284,"statfs":{"total":773094113280,"available":772844834816,"internally_reserved":0,"allocated":6610944,"data_stored":3396836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001687"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.360759+0000","last_change":"2024-04-17T09:56:02.229036+0000","last_active":"2024-04-17T09:56:03.360759+0000","last_peered":"2024-04-17T09:56:03.360759+0000","last_clean":"2024-04-17T09:56:03.360759+0000","last_became_active":"2024-04-17T09:56:02.228759+0000","last_became_peered":"2024-04-17T09:56:02.228759+0000","last_unstale":"2024-04-17T09:56:03.360759+0000","last_undegraded":"2024-04-17T09:56:03.360759+0000","last_fullsized":"2024-04-17T09:56:03.360759+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:34:34.405899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261433+0000","last_change":"2024-04-17T09:56:02.224588+0000","last_active":"2024-04-17T09:56:05.261433+0000","last_peered":"2024-04-17T09:56:05.261433+0000","last_clean":"2024-04-17T09:56:05.261433+0000","last_became_active":"2024-04-17T09:56:02.224240+0000","last_became_peered":"2024-04-17T09:56:02.224240+0000","last_unstale":"2024-04-17T09:56:05.261433+0000","last_undegraded":"2024-04-17T09:56:05.261433+0000","last_fullsized":"2024-04-17T09:56:05.261433+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:15:50.953634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260735+0000","last_change":"2024-04-17T09:56:02.211108+0000","last_active":"2024-04-17T09:56:05.260735+0000","last_peered":"2024-04-17T09:56:05.260735+0000","last_clean":"2024-04-17T09:56:05.260735+0000","last_became_active":"2024-04-17T09:56:02.210847+0000","last_became_peered":"2024-04-17T09:56:02.210847+0000","last_unstale":"2024-04-17T09:56:05.260735+0000","last_undegraded":"2024-04-17T09:56:05.260735+0000","last_fullsized":"2024-04-17T09:56:05.260735+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:35:05.000285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260436+0000","last_change":"2024-04-17T09:56:02.224673+0000","last_active":"2024-04-17T09:56:05.260436+0000","last_peered":"2024-04-17T09:56:05.260436+0000","last_clean":"2024-04-17T09:56:05.260436+0000","last_became_active":"2024-04-17T09:56:02.224412+0000","last_became_peered":"2024-04-17T09:56:02.224412+0000","last_unstale":"2024-04-17T09:56:05.260436+0000","last_undegraded":"2024-04-17T09:56:05.260436+0000","last_fullsized":"2024-04-17T09:56:05.260436+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:04:55.654330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.360752+0000","last_change":"2024-04-17T09:56:02.220342+0000","last_active":"2024-04-17T09:56:03.360752+0000","last_peered":"2024-04-17T09:56:03.360752+0000","last_clean":"2024-04-17T09:56:03.360752+0000","last_became_active":"2024-04-17T09:56:02.220041+0000","last_became_peered":"2024-04-17T09:56:02.220041+0000","last_unstale":"2024-04-17T09:56:03.360752+0000","last_undegraded":"2024-04-17T09:56:03.360752+0000","last_fullsized":"2024-04-17T09:56:03.360752+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:40.237318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197311+0000","last_change":"2024-04-17T09:56:02.221498+0000","last_active":"2024-04-17T09:56:03.197311+0000","last_peered":"2024-04-17T09:56:03.197311+0000","last_clean":"2024-04-17T09:56:03.197311+0000","last_became_active":"2024-04-17T09:56:02.217279+0000","last_became_peered":"2024-04-17T09:56:02.217279+0000","last_unstale":"2024-04-17T09:56:03.197311+0000","last_undegraded":"2024-04-17T09:56:03.197311+0000","last_fullsized":"2024-04-17T09:56:03.197311+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:53.818908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203103+0000","last_change":"2024-04-17T09:56:02.220843+0000","last_active":"2024-04-17T09:56:03.203103+0000","last_peered":"2024-04-17T09:56:03.203103+0000","last_clean":"2024-04-17T09:56:03.203103+0000","last_became_active":"2024-04-17T09:56:02.220566+0000","last_became_peered":"2024-04-17T09:56:02.220566+0000","last_unstale":"2024-04-17T09:56:03.203103+0000","last_undegraded":"2024-04-17T09:56:03.203103+0000","last_fullsized":"2024-04-17T09:56:03.203103+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:24:38.324835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261328+0000","last_change":"2024-04-17T09:56:02.211847+0000","last_active":"2024-04-17T09:56:05.261328+0000","last_peered":"2024-04-17T09:56:05.261328+0000","last_clean":"2024-04-17T09:56:05.261328+0000","last_became_active":"2024-04-17T09:56:02.211588+0000","last_became_peered":"2024-04-17T09:56:02.211588+0000","last_unstale":"2024-04-17T09:56:05.261328+0000","last_undegraded":"2024-04-17T09:56:05.261328+0000","last_fullsized":"2024-04-17T09:56:05.261328+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:10:29.259502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203578+0000","last_change":"2024-04-17T09:56:02.230219+0000","last_active":"2024-04-17T09:56:03.203578+0000","last_peered":"2024-04-17T09:56:03.203578+0000","last_clean":"2024-04-17T09:56:03.203578+0000","last_became_active":"2024-04-17T09:56:02.229840+0000","last_became_peered":"2024-04-17T09:56:02.229840+0000","last_unstale":"2024-04-17T09:56:03.203578+0000","last_undegraded":"2024-04-17T09:56:03.203578+0000","last_fullsized":"2024-04-17T09:56:03.203578+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:14:09.222091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262523+0000","last_change":"2024-04-17T09:56:02.221953+0000","last_active":"2024-04-17T09:56:05.262523+0000","last_peered":"2024-04-17T09:56:05.262523+0000","last_clean":"2024-04-17T09:56:05.262523+0000","last_became_active":"2024-04-17T09:56:02.221594+0000","last_became_peered":"2024-04-17T09:56:02.221594+0000","last_unstale":"2024-04-17T09:56:05.262523+0000","last_undegraded":"2024-04-17T09:56:05.262523+0000","last_fullsized":"2024-04-17T09:56:05.262523+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:58:59.479429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262281+0000","last_change":"2024-04-17T09:56:02.227065+0000","last_active":"2024-04-17T09:56:05.262281+0000","last_peered":"2024-04-17T09:56:05.262281+0000","last_clean":"2024-04-17T09:56:05.262281+0000","last_became_active":"2024-04-17T09:56:02.226787+0000","last_became_peered":"2024-04-17T09:56:02.226787+0000","last_unstale":"2024-04-17T09:56:05.262281+0000","last_undegraded":"2024-04-17T09:56:05.262281+0000","last_fullsized":"2024-04-17T09:56:05.262281+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:44:28.409976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261050+0000","last_change":"2024-04-17T09:56:02.216456+0000","last_active":"2024-04-17T09:56:05.261050+0000","last_peered":"2024-04-17T09:56:05.261050+0000","last_clean":"2024-04-17T09:56:05.261050+0000","last_became_active":"2024-04-17T09:56:02.216072+0000","last_became_peered":"2024-04-17T09:56:02.216072+0000","last_unstale":"2024-04-17T09:56:05.261050+0000","last_undegraded":"2024-04-17T09:56:05.261050+0000","last_fullsized":"2024-04-17T09:56:05.261050+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:55:11.309539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203124+0000","last_change":"2024-04-17T09:56:02.215465+0000","last_active":"2024-04-17T09:56:03.203124+0000","last_peered":"2024-04-17T09:56:03.203124+0000","last_clean":"2024-04-17T09:56:03.203124+0000","last_became_active":"2024-04-17T09:56:02.215176+0000","last_became_peered":"2024-04-17T09:56:02.215176+0000","last_unstale":"2024-04-17T09:56:03.203124+0000","last_undegraded":"2024-04-17T09:56:03.203124+0000","last_fullsized":"2024-04-17T09:56:03.203124+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:13:28.472859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203987+0000","last_change":"2024-04-17T09:56:02.213294+0000","last_active":"2024-04-17T09:56:03.203987+0000","last_peered":"2024-04-17T09:56:03.203987+0000","last_clean":"2024-04-17T09:56:03.203987+0000","last_became_active":"2024-04-17T09:56:02.212904+0000","last_became_peered":"2024-04-17T09:56:02.212904+0000","last_unstale":"2024-04-17T09:56:03.203987+0000","last_undegraded":"2024-04-17T09:56:03.203987+0000","last_fullsized":"2024-04-17T09:56:03.203987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:41:23.655282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203242+0000","last_change":"2024-04-17T09:56:02.222178+0000","last_active":"2024-04-17T09:56:03.203242+0000","last_peered":"2024-04-17T09:56:03.203242+0000","last_clean":"2024-04-17T09:56:03.203242+0000","last_became_active":"2024-04-17T09:56:02.221835+0000","last_became_peered":"2024-04-17T09:56:02.221835+0000","last_unstale":"2024-04-17T09:56:03.203242+0000","last_undegraded":"2024-04-17T09:56:03.203242+0000","last_fullsized":"2024-04-17T09:56:03.203242+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:40:42.141566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197006+0000","last_change":"2024-04-17T09:56:02.215969+0000","last_active":"2024-04-17T09:56:03.197006+0000","last_peered":"2024-04-17T09:56:03.197006+0000","last_clean":"2024-04-17T09:56:03.197006+0000","last_became_active":"2024-04-17T09:56:02.215670+0000","last_became_peered":"2024-04-17T09:56:02.215670+0000","last_unstale":"2024-04-17T09:56:03.197006+0000","last_undegraded":"2024-04-17T09:56:03.197006+0000","last_fullsized":"2024-04-17T09:56:03.197006+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:01:34.123890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262211+0000","last_change":"2024-04-17T09:56:02.210928+0000","last_active":"2024-04-17T09:56:05.262211+0000","last_peered":"2024-04-17T09:56:05.262211+0000","last_clean":"2024-04-17T09:56:05.262211+0000","last_became_active":"2024-04-17T09:56:02.210640+0000","last_became_peered":"2024-04-17T09:56:02.210640+0000","last_unstale":"2024-04-17T09:56:05.262211+0000","last_undegraded":"2024-04-17T09:56:05.262211+0000","last_fullsized":"2024-04-17T09:56:05.262211+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:42.518838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260690+0000","last_change":"2024-04-17T09:56:02.229718+0000","last_active":"2024-04-17T09:56:05.260690+0000","last_peered":"2024-04-17T09:56:05.260690+0000","last_clean":"2024-04-17T09:56:05.260690+0000","last_became_active":"2024-04-17T09:56:02.229408+0000","last_became_peered":"2024-04-17T09:56:02.229408+0000","last_unstale":"2024-04-17T09:56:05.260690+0000","last_undegraded":"2024-04-17T09:56:05.260690+0000","last_fullsized":"2024-04-17T09:56:05.260690+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:41:11.361171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.360652+0000","last_change":"2024-04-17T09:56:02.230129+0000","last_active":"2024-04-17T09:56:03.360652+0000","last_peered":"2024-04-17T09:56:03.360652+0000","last_clean":"2024-04-17T09:56:03.360652+0000","last_became_active":"2024-04-17T09:56:02.229876+0000","last_became_peered":"2024-04-17T09:56:02.229876+0000","last_unstale":"2024-04-17T09:56:03.360652+0000","last_undegraded":"2024-04-17T09:56:03.360652+0000","last_fullsized":"2024-04-17T09:56:03.360652+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:13:48.083231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261631+0000","last_change":"2024-04-17T09:56:02.229569+0000","last_active":"2024-04-17T09:56:05.261631+0000","last_peered":"2024-04-17T09:56:05.261631+0000","last_clean":"2024-04-17T09:56:05.261631+0000","last_became_active":"2024-04-17T09:56:02.229290+0000","last_became_peered":"2024-04-17T09:56:02.229290+0000","last_unstale":"2024-04-17T09:56:05.261631+0000","last_undegraded":"2024-04-17T09:56:05.261631+0000","last_fullsized":"2024-04-17T09:56:05.261631+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:54.063917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260149+0000","last_change":"2024-04-17T09:56:02.227809+0000","last_active":"2024-04-17T09:56:05.260149+0000","last_peered":"2024-04-17T09:56:05.260149+0000","last_clean":"2024-04-17T09:56:05.260149+0000","last_became_active":"2024-04-17T09:56:02.227554+0000","last_became_peered":"2024-04-17T09:56:02.227554+0000","last_unstale":"2024-04-17T09:56:05.260149+0000","last_undegraded":"2024-04-17T09:56:05.260149+0000","last_fullsized":"2024-04-17T09:56:05.260149+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:53:24.666275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261775+0000","last_change":"2024-04-17T09:56:02.220664+0000","last_active":"2024-04-17T09:56:05.261775+0000","last_peered":"2024-04-17T09:56:05.261775+0000","last_clean":"2024-04-17T09:56:05.261775+0000","last_became_active":"2024-04-17T09:56:02.216070+0000","last_became_peered":"2024-04-17T09:56:02.216070+0000","last_unstale":"2024-04-17T09:56:05.261775+0000","last_undegraded":"2024-04-17T09:56:05.261775+0000","last_fullsized":"2024-04-17T09:56:05.261775+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:46:26.870103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260688+0000","last_change":"2024-04-17T09:56:02.214345+0000","last_active":"2024-04-17T09:56:05.260688+0000","last_peered":"2024-04-17T09:56:05.260688+0000","last_clean":"2024-04-17T09:56:05.260688+0000","last_became_active":"2024-04-17T09:56:02.213946+0000","last_became_peered":"2024-04-17T09:56:02.213946+0000","last_unstale":"2024-04-17T09:56:05.260688+0000","last_undegraded":"2024-04-17T09:56:05.260688+0000","last_fullsized":"2024-04-17T09:56:05.260688+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:35:18.947071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203578+0000","last_change":"2024-04-17T09:56:02.215807+0000","last_active":"2024-04-17T09:56:03.203578+0000","last_peered":"2024-04-17T09:56:03.203578+0000","last_clean":"2024-04-17T09:56:03.203578+0000","last_became_active":"2024-04-17T09:56:02.215224+0000","last_became_peered":"2024-04-17T09:56:02.215224+0000","last_unstale":"2024-04-17T09:56:03.203578+0000","last_undegraded":"2024-04-17T09:56:03.203578+0000","last_fullsized":"2024-04-17T09:56:03.203578+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:55:47.229362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.360883+0000","last_change":"2024-04-17T09:56:02.219425+0000","last_active":"2024-04-17T09:56:03.360883+0000","last_peered":"2024-04-17T09:56:03.360883+0000","last_clean":"2024-04-17T09:56:03.360883+0000","last_became_active":"2024-04-17T09:56:02.219113+0000","last_became_peered":"2024-04-17T09:56:02.219113+0000","last_unstale":"2024-04-17T09:56:03.360883+0000","last_undegraded":"2024-04-17T09:56:03.360883+0000","last_fullsized":"2024-04-17T09:56:03.360883+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:54:36.189192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261169+0000","last_change":"2024-04-17T09:56:02.220120+0000","last_active":"2024-04-17T09:56:05.261169+0000","last_peered":"2024-04-17T09:56:05.261169+0000","last_clean":"2024-04-17T09:56:05.261169+0000","last_became_active":"2024-04-17T09:56:02.219824+0000","last_became_peered":"2024-04-17T09:56:02.219824+0000","last_unstale":"2024-04-17T09:56:05.261169+0000","last_undegraded":"2024-04-17T09:56:05.261169+0000","last_fullsized":"2024-04-17T09:56:05.261169+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:32:39.633696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197587+0000","last_change":"2024-04-17T09:56:02.215021+0000","last_active":"2024-04-17T09:56:03.197587+0000","last_peered":"2024-04-17T09:56:03.197587+0000","last_clean":"2024-04-17T09:56:03.197587+0000","last_became_active":"2024-04-17T09:56:02.214653+0000","last_became_peered":"2024-04-17T09:56:02.214653+0000","last_unstale":"2024-04-17T09:56:03.197587+0000","last_undegraded":"2024-04-17T09:56:03.197587+0000","last_fullsized":"2024-04-17T09:56:03.197587+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:00:41.794977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.360669+0000","last_change":"2024-04-17T09:56:02.226322+0000","last_active":"2024-04-17T09:56:03.360669+0000","last_peered":"2024-04-17T09:56:03.360669+0000","last_clean":"2024-04-17T09:56:03.360669+0000","last_became_active":"2024-04-17T09:56:02.226061+0000","last_became_peered":"2024-04-17T09:56:02.226061+0000","last_unstale":"2024-04-17T09:56:03.360669+0000","last_undegraded":"2024-04-17T09:56:03.360669+0000","last_fullsized":"2024-04-17T09:56:03.360669+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:36:06.628580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262401+0000","last_change":"2024-04-17T09:56:02.227824+0000","last_active":"2024-04-17T09:56:05.262401+0000","last_peered":"2024-04-17T09:56:05.262401+0000","last_clean":"2024-04-17T09:56:05.262401+0000","last_became_active":"2024-04-17T09:56:02.226305+0000","last_became_peered":"2024-04-17T09:56:02.226305+0000","last_unstale":"2024-04-17T09:56:05.262401+0000","last_undegraded":"2024-04-17T09:56:05.262401+0000","last_fullsized":"2024-04-17T09:56:05.262401+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:59:28.074910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260065+0000","last_change":"2024-04-17T09:56:02.209581+0000","last_active":"2024-04-17T09:56:05.260065+0000","last_peered":"2024-04-17T09:56:05.260065+0000","last_clean":"2024-04-17T09:56:05.260065+0000","last_became_active":"2024-04-17T09:56:02.209325+0000","last_became_peered":"2024-04-17T09:56:02.209325+0000","last_unstale":"2024-04-17T09:56:05.260065+0000","last_undegraded":"2024-04-17T09:56:05.260065+0000","last_fullsized":"2024-04-17T09:56:05.260065+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:32:39.633696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.360804+0000","last_change":"2024-04-17T09:56:02.219449+0000","last_active":"2024-04-17T09:56:03.360804+0000","last_peered":"2024-04-17T09:56:03.360804+0000","last_clean":"2024-04-17T09:56:03.360804+0000","last_became_active":"2024-04-17T09:56:02.219178+0000","last_became_peered":"2024-04-17T09:56:02.219178+0000","last_unstale":"2024-04-17T09:56:03.360804+0000","last_undegraded":"2024-04-17T09:56:03.360804+0000","last_fullsized":"2024-04-17T09:56:03.360804+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:05:04.703617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197005+0000","last_change":"2024-04-17T09:56:02.225982+0000","last_active":"2024-04-17T09:56:03.197005+0000","last_peered":"2024-04-17T09:56:03.197005+0000","last_clean":"2024-04-17T09:56:03.197005+0000","last_became_active":"2024-04-17T09:56:02.223371+0000","last_became_peered":"2024-04-17T09:56:02.223371+0000","last_unstale":"2024-04-17T09:56:03.197005+0000","last_undegraded":"2024-04-17T09:56:03.197005+0000","last_fullsized":"2024-04-17T09:56:03.197005+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:48:17.886969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197834+0000","last_change":"2024-04-17T09:56:02.224842+0000","last_active":"2024-04-17T09:56:03.197834+0000","last_peered":"2024-04-17T09:56:03.197834+0000","last_clean":"2024-04-17T09:56:03.197834+0000","last_became_active":"2024-04-17T09:56:02.224545+0000","last_became_peered":"2024-04-17T09:56:02.224545+0000","last_unstale":"2024-04-17T09:56:03.197834+0000","last_undegraded":"2024-04-17T09:56:03.197834+0000","last_fullsized":"2024-04-17T09:56:03.197834+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:57.566831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"56'4","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197875+0000","last_change":"2024-04-17T09:56:01.206168+0000","last_active":"2024-04-17T09:56:03.197875+0000","last_peered":"2024-04-17T09:56:03.197875+0000","last_clean":"2024-04-17T09:56:03.197875+0000","last_became_active":"2024-04-17T09:56:01.204386+0000","last_became_peered":"2024-04-17T09:56:01.204386+0000","last_unstale":"2024-04-17T09:56:03.197875+0000","last_undegraded":"2024-04-17T09:56:03.197875+0000","last_fullsized":"2024-04-17T09:56:03.197875+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:34.492070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262619+0000","last_change":"2024-04-17T09:56:02.222971+0000","last_active":"2024-04-17T09:56:05.262619+0000","last_peered":"2024-04-17T09:56:05.262619+0000","last_clean":"2024-04-17T09:56:05.262619+0000","last_became_active":"2024-04-17T09:56:02.222712+0000","last_became_peered":"2024-04-17T09:56:02.222712+0000","last_unstale":"2024-04-17T09:56:05.262619+0000","last_undegraded":"2024-04-17T09:56:05.262619+0000","last_fullsized":"2024-04-17T09:56:05.262619+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:47.438505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261926+0000","last_change":"2024-04-17T09:56:01.205411+0000","last_active":"2024-04-17T09:56:05.261926+0000","last_peered":"2024-04-17T09:56:05.261926+0000","last_clean":"2024-04-17T09:56:05.261926+0000","last_became_active":"2024-04-17T09:56:01.204027+0000","last_became_peered":"2024-04-17T09:56:01.204027+0000","last_unstale":"2024-04-17T09:56:05.261926+0000","last_undegraded":"2024-04-17T09:56:05.261926+0000","last_fullsized":"2024-04-17T09:56:05.261926+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:20:59.234480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197603+0000","last_change":"2024-04-17T09:56:02.234199+0000","last_active":"2024-04-17T09:56:03.197603+0000","last_peered":"2024-04-17T09:56:03.197603+0000","last_clean":"2024-04-17T09:56:03.197603+0000","last_became_active":"2024-04-17T09:56:02.233686+0000","last_became_peered":"2024-04-17T09:56:02.233686+0000","last_unstale":"2024-04-17T09:56:03.197603+0000","last_undegraded":"2024-04-17T09:56:03.197603+0000","last_fullsized":"2024-04-17T09:56:03.197603+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:08.333973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197902+0000","last_change":"2024-04-17T09:56:01.201154+0000","last_active":"2024-04-17T09:56:03.197902+0000","last_peered":"2024-04-17T09:56:03.197902+0000","last_clean":"2024-04-17T09:56:03.197902+0000","last_became_active":"2024-04-17T09:56:01.200926+0000","last_became_peered":"2024-04-17T09:56:01.200926+0000","last_unstale":"2024-04-17T09:56:03.197902+0000","last_undegraded":"2024-04-17T09:56:03.197902+0000","last_fullsized":"2024-04-17T09:56:03.197902+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:41:22.649311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197793+0000","last_change":"2024-04-17T09:56:02.230469+0000","last_active":"2024-04-17T09:56:03.197793+0000","last_peered":"2024-04-17T09:56:03.197793+0000","last_clean":"2024-04-17T09:56:03.197793+0000","last_became_active":"2024-04-17T09:56:02.229210+0000","last_became_peered":"2024-04-17T09:56:02.229210+0000","last_unstale":"2024-04-17T09:56:03.197793+0000","last_undegraded":"2024-04-17T09:56:03.197793+0000","last_fullsized":"2024-04-17T09:56:03.197793+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:58:25.393385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260914+0000","last_change":"2024-04-17T09:56:01.191420+0000","last_active":"2024-04-17T09:56:05.260914+0000","last_peered":"2024-04-17T09:56:05.260914+0000","last_clean":"2024-04-17T09:56:05.260914+0000","last_became_active":"2024-04-17T09:56:01.191177+0000","last_became_peered":"2024-04-17T09:56:01.191177+0000","last_unstale":"2024-04-17T09:56:05.260914+0000","last_undegraded":"2024-04-17T09:56:05.260914+0000","last_fullsized":"2024-04-17T09:56:05.260914+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:46.264824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261922+0000","last_change":"2024-04-17T09:56:02.220667+0000","last_active":"2024-04-17T09:56:05.261922+0000","last_peered":"2024-04-17T09:56:05.261922+0000","last_clean":"2024-04-17T09:56:05.261922+0000","last_became_active":"2024-04-17T09:56:02.216557+0000","last_became_peered":"2024-04-17T09:56:02.216557+0000","last_unstale":"2024-04-17T09:56:05.261922+0000","last_undegraded":"2024-04-17T09:56:05.261922+0000","last_fullsized":"2024-04-17T09:56:05.261922+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:18:25.708772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203408+0000","last_change":"2024-04-17T09:56:01.206115+0000","last_active":"2024-04-17T09:56:03.203408+0000","last_peered":"2024-04-17T09:56:03.203408+0000","last_clean":"2024-04-17T09:56:03.203408+0000","last_became_active":"2024-04-17T09:56:01.205818+0000","last_became_peered":"2024-04-17T09:56:01.205818+0000","last_unstale":"2024-04-17T09:56:03.203408+0000","last_undegraded":"2024-04-17T09:56:03.203408+0000","last_fullsized":"2024-04-17T09:56:03.203408+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:53.672442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261574+0000","last_change":"2024-04-17T09:56:02.216255+0000","last_active":"2024-04-17T09:56:05.261574+0000","last_peered":"2024-04-17T09:56:05.261574+0000","last_clean":"2024-04-17T09:56:05.261574+0000","last_became_active":"2024-04-17T09:56:02.215855+0000","last_became_peered":"2024-04-17T09:56:02.215855+0000","last_unstale":"2024-04-17T09:56:05.261574+0000","last_undegraded":"2024-04-17T09:56:05.261574+0000","last_fullsized":"2024-04-17T09:56:05.261574+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:54:33.483886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197353+0000","last_change":"2024-04-17T09:56:01.206766+0000","last_active":"2024-04-17T09:56:03.197353+0000","last_peered":"2024-04-17T09:56:03.197353+0000","last_clean":"2024-04-17T09:56:03.197353+0000","last_became_active":"2024-04-17T09:56:01.206438+0000","last_became_peered":"2024-04-17T09:56:01.206438+0000","last_unstale":"2024-04-17T09:56:03.197353+0000","last_undegraded":"2024-04-17T09:56:03.197353+0000","last_fullsized":"2024-04-17T09:56:03.197353+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:40:41.135595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203266+0000","last_change":"2024-04-17T09:56:02.208267+0000","last_active":"2024-04-17T09:56:03.203266+0000","last_peered":"2024-04-17T09:56:03.203266+0000","last_clean":"2024-04-17T09:56:03.203266+0000","last_became_active":"2024-04-17T09:56:02.207427+0000","last_became_peered":"2024-04-17T09:56:02.207427+0000","last_unstale":"2024-04-17T09:56:03.203266+0000","last_undegraded":"2024-04-17T09:56:03.203266+0000","last_fullsized":"2024-04-17T09:56:03.203266+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:46:36.134877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.259829+0000","last_change":"2024-04-17T09:56:01.194530+0000","last_active":"2024-04-17T09:56:05.259829+0000","last_peered":"2024-04-17T09:56:05.259829+0000","last_clean":"2024-04-17T09:56:05.259829+0000","last_became_active":"2024-04-17T09:56:01.194139+0000","last_became_peered":"2024-04-17T09:56:01.194139+0000","last_unstale":"2024-04-17T09:56:05.259829+0000","last_undegraded":"2024-04-17T09:56:05.259829+0000","last_fullsized":"2024-04-17T09:56:05.259829+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:59.853192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261634+0000","last_change":"2024-04-17T09:56:02.204678+0000","last_active":"2024-04-17T09:56:05.261634+0000","last_peered":"2024-04-17T09:56:05.261634+0000","last_clean":"2024-04-17T09:56:05.261634+0000","last_became_active":"2024-04-17T09:56:02.204378+0000","last_became_peered":"2024-04-17T09:56:02.204378+0000","last_unstale":"2024-04-17T09:56:05.261634+0000","last_undegraded":"2024-04-17T09:56:05.261634+0000","last_fullsized":"2024-04-17T09:56:05.261634+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:13:25.794083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203390+0000","last_change":"2024-04-17T09:56:01.197166+0000","last_active":"2024-04-17T09:56:03.203390+0000","last_peered":"2024-04-17T09:56:03.203390+0000","last_clean":"2024-04-17T09:56:03.203390+0000","last_became_active":"2024-04-17T09:56:01.196416+0000","last_became_peered":"2024-04-17T09:56:01.196416+0000","last_unstale":"2024-04-17T09:56:03.203390+0000","last_undegraded":"2024-04-17T09:56:03.203390+0000","last_fullsized":"2024-04-17T09:56:03.203390+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:58:47.664590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203664+0000","last_change":"2024-04-17T09:56:02.222098+0000","last_active":"2024-04-17T09:56:03.203664+0000","last_peered":"2024-04-17T09:56:03.203664+0000","last_clean":"2024-04-17T09:56:03.203664+0000","last_became_active":"2024-04-17T09:56:02.221642+0000","last_became_peered":"2024-04-17T09:56:02.221642+0000","last_unstale":"2024-04-17T09:56:03.203664+0000","last_undegraded":"2024-04-17T09:56:03.203664+0000","last_fullsized":"2024-04-17T09:56:03.203664+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:02:18.696690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260413+0000","last_change":"2024-04-17T09:56:01.203745+0000","last_active":"2024-04-17T09:56:05.260413+0000","last_peered":"2024-04-17T09:56:05.260413+0000","last_clean":"2024-04-17T09:56:05.260413+0000","last_became_active":"2024-04-17T09:56:01.203487+0000","last_became_peered":"2024-04-17T09:56:01.203487+0000","last_unstale":"2024-04-17T09:56:05.260413+0000","last_undegraded":"2024-04-17T09:56:05.260413+0000","last_fullsized":"2024-04-17T09:56:05.260413+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:48:06.817655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260492+0000","last_change":"2024-04-17T09:56:02.219782+0000","last_active":"2024-04-17T09:56:05.260492+0000","last_peered":"2024-04-17T09:56:05.260492+0000","last_clean":"2024-04-17T09:56:05.260492+0000","last_became_active":"2024-04-17T09:56:02.219409+0000","last_became_peered":"2024-04-17T09:56:02.219409+0000","last_unstale":"2024-04-17T09:56:05.260492+0000","last_undegraded":"2024-04-17T09:56:05.260492+0000","last_fullsized":"2024-04-17T09:56:05.260492+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:10:01.305797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262086+0000","last_change":"2024-04-17T09:56:01.208993+0000","last_active":"2024-04-17T09:56:05.262086+0000","last_peered":"2024-04-17T09:56:05.262086+0000","last_clean":"2024-04-17T09:56:05.262086+0000","last_became_active":"2024-04-17T09:56:01.208722+0000","last_became_peered":"2024-04-17T09:56:01.208722+0000","last_unstale":"2024-04-17T09:56:05.262086+0000","last_undegraded":"2024-04-17T09:56:05.262086+0000","last_fullsized":"2024-04-17T09:56:05.262086+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:55:10.303568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262017+0000","last_change":"2024-04-17T09:56:02.217472+0000","last_active":"2024-04-17T09:56:05.262017+0000","last_peered":"2024-04-17T09:56:05.262017+0000","last_clean":"2024-04-17T09:56:05.262017+0000","last_became_active":"2024-04-17T09:56:02.215926+0000","last_became_peered":"2024-04-17T09:56:02.215926+0000","last_unstale":"2024-04-17T09:56:05.262017+0000","last_undegraded":"2024-04-17T09:56:05.262017+0000","last_fullsized":"2024-04-17T09:56:05.262017+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:46:36.134877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"56'3","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260945+0000","last_change":"2024-04-17T09:56:01.197552+0000","last_active":"2024-04-17T09:56:05.260945+0000","last_peered":"2024-04-17T09:56:05.260945+0000","last_clean":"2024-04-17T09:56:05.260945+0000","last_became_active":"2024-04-17T09:56:01.197216+0000","last_became_peered":"2024-04-17T09:56:01.197216+0000","last_unstale":"2024-04-17T09:56:05.260945+0000","last_undegraded":"2024-04-17T09:56:05.260945+0000","last_fullsized":"2024-04-17T09:56:05.260945+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:40:04.394560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260822+0000","last_change":"2024-04-17T09:56:02.222301+0000","last_active":"2024-04-17T09:56:05.260822+0000","last_peered":"2024-04-17T09:56:05.260822+0000","last_clean":"2024-04-17T09:56:05.260822+0000","last_became_active":"2024-04-17T09:56:02.221854+0000","last_became_peered":"2024-04-17T09:56:02.221854+0000","last_unstale":"2024-04-17T09:56:05.260822+0000","last_undegraded":"2024-04-17T09:56:05.260822+0000","last_fullsized":"2024-04-17T09:56:05.260822+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:15:55.836315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"56'6","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203761+0000","last_change":"2024-04-17T09:56:01.206040+0000","last_active":"2024-04-17T09:56:03.203761+0000","last_peered":"2024-04-17T09:56:03.203761+0000","last_clean":"2024-04-17T09:56:03.203761+0000","last_became_active":"2024-04-17T09:56:01.205744+0000","last_became_peered":"2024-04-17T09:56:01.205744+0000","last_unstale":"2024-04-17T09:56:03.203761+0000","last_undegraded":"2024-04-17T09:56:03.203761+0000","last_fullsized":"2024-04-17T09:56:03.203761+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:55:07.841245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260001+0000","last_change":"2024-04-17T09:56:02.222193+0000","last_active":"2024-04-17T09:56:05.260001+0000","last_peered":"2024-04-17T09:56:05.260001+0000","last_clean":"2024-04-17T09:56:05.260001+0000","last_became_active":"2024-04-17T09:56:02.221951+0000","last_became_peered":"2024-04-17T09:56:02.221951+0000","last_unstale":"2024-04-17T09:56:05.260001+0000","last_undegraded":"2024-04-17T09:56:05.260001+0000","last_fullsized":"2024-04-17T09:56:05.260001+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:37:03.542306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262197+0000","last_change":"2024-04-17T09:56:01.208069+0000","last_active":"2024-04-17T09:56:05.262197+0000","last_peered":"2024-04-17T09:56:05.262197+0000","last_clean":"2024-04-17T09:56:05.262197+0000","last_became_active":"2024-04-17T09:56:01.207756+0000","last_became_peered":"2024-04-17T09:56:01.207756+0000","last_unstale":"2024-04-17T09:56:05.262197+0000","last_undegraded":"2024-04-17T09:56:05.262197+0000","last_fullsized":"2024-04-17T09:56:05.262197+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:04:54.648358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260228+0000","last_change":"2024-04-17T09:56:02.204783+0000","last_active":"2024-04-17T09:56:05.260228+0000","last_peered":"2024-04-17T09:56:05.260228+0000","last_clean":"2024-04-17T09:56:05.260228+0000","last_became_active":"2024-04-17T09:56:02.204482+0000","last_became_peered":"2024-04-17T09:56:02.204482+0000","last_unstale":"2024-04-17T09:56:05.260228+0000","last_undegraded":"2024-04-17T09:56:05.260228+0000","last_fullsized":"2024-04-17T09:56:05.260228+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:52:40.143883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.202981+0000","last_change":"2024-04-17T09:56:01.191609+0000","last_active":"2024-04-17T09:56:03.202981+0000","last_peered":"2024-04-17T09:56:03.202981+0000","last_clean":"2024-04-17T09:56:03.202981+0000","last_became_active":"2024-04-17T09:56:01.191314+0000","last_became_peered":"2024-04-17T09:56:01.191314+0000","last_unstale":"2024-04-17T09:56:03.202981+0000","last_undegraded":"2024-04-17T09:56:03.202981+0000","last_fullsized":"2024-04-17T09:56:03.202981+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:29.004521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203271+0000","last_change":"2024-04-17T09:56:02.210009+0000","last_active":"2024-04-17T09:56:03.203271+0000","last_peered":"2024-04-17T09:56:03.203271+0000","last_clean":"2024-04-17T09:56:03.203271+0000","last_became_active":"2024-04-17T09:56:02.209655+0000","last_became_peered":"2024-04-17T09:56:02.209655+0000","last_unstale":"2024-04-17T09:56:03.203271+0000","last_undegraded":"2024-04-17T09:56:03.203271+0000","last_fullsized":"2024-04-17T09:56:03.203271+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:23.603548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.259952+0000","last_change":"2024-04-17T09:56:01.189043+0000","last_active":"2024-04-17T09:56:05.259952+0000","last_peered":"2024-04-17T09:56:05.259952+0000","last_clean":"2024-04-17T09:56:05.259952+0000","last_became_active":"2024-04-17T09:56:01.188778+0000","last_became_peered":"2024-04-17T09:56:01.188778+0000","last_unstale":"2024-04-17T09:56:05.259952+0000","last_undegraded":"2024-04-17T09:56:05.259952+0000","last_fullsized":"2024-04-17T09:56:05.259952+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:41:45.654626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260707+0000","last_change":"2024-04-17T09:56:02.214563+0000","last_active":"2024-04-17T09:56:05.260707+0000","last_peered":"2024-04-17T09:56:05.260707+0000","last_clean":"2024-04-17T09:56:05.260707+0000","last_became_active":"2024-04-17T09:56:02.213985+0000","last_became_peered":"2024-04-17T09:56:02.213985+0000","last_unstale":"2024-04-17T09:56:05.260707+0000","last_undegraded":"2024-04-17T09:56:05.260707+0000","last_fullsized":"2024-04-17T09:56:05.260707+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:27:28.602507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"56'8","reported_seq":22,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.259829+0000","last_change":"2024-04-17T09:56:01.196179+0000","last_active":"2024-04-17T09:56:05.259829+0000","last_peered":"2024-04-17T09:56:05.259829+0000","last_clean":"2024-04-17T09:56:05.259829+0000","last_became_active":"2024-04-17T09:56:01.195954+0000","last_became_peered":"2024-04-17T09:56:01.195954+0000","last_unstale":"2024-04-17T09:56:05.259829+0000","last_undegraded":"2024-04-17T09:56:05.259829+0000","last_fullsized":"2024-04-17T09:56:05.259829+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:41:45.654626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260479+0000","last_change":"2024-04-17T09:56:02.210099+0000","last_active":"2024-04-17T09:56:05.260479+0000","last_peered":"2024-04-17T09:56:05.260479+0000","last_clean":"2024-04-17T09:56:05.260479+0000","last_became_active":"2024-04-17T09:56:02.209740+0000","last_became_peered":"2024-04-17T09:56:02.209740+0000","last_unstale":"2024-04-17T09:56:05.260479+0000","last_undegraded":"2024-04-17T09:56:05.260479+0000","last_fullsized":"2024-04-17T09:56:05.260479+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:05:05.510667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262278+0000","last_change":"2024-04-17T09:56:01.207663+0000","last_active":"2024-04-17T09:56:05.262278+0000","last_peered":"2024-04-17T09:56:05.262278+0000","last_clean":"2024-04-17T09:56:05.262278+0000","last_became_active":"2024-04-17T09:56:01.206630+0000","last_became_peered":"2024-04-17T09:56:01.206630+0000","last_unstale":"2024-04-17T09:56:05.262278+0000","last_undegraded":"2024-04-17T09:56:05.262278+0000","last_fullsized":"2024-04-17T09:56:05.262278+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:53:23.660304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261244+0000","last_change":"2024-04-17T09:56:02.223670+0000","last_active":"2024-04-17T09:56:05.261244+0000","last_peered":"2024-04-17T09:56:05.261244+0000","last_clean":"2024-04-17T09:56:05.261244+0000","last_became_active":"2024-04-17T09:56:02.223378+0000","last_became_peered":"2024-04-17T09:56:02.223378+0000","last_unstale":"2024-04-17T09:56:05.261244+0000","last_undegraded":"2024-04-17T09:56:05.261244+0000","last_fullsized":"2024-04-17T09:56:05.261244+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:53:24.666275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197845+0000","last_change":"2024-04-17T09:56:01.208863+0000","last_active":"2024-04-17T09:56:03.197845+0000","last_peered":"2024-04-17T09:56:03.197845+0000","last_clean":"2024-04-17T09:56:03.197845+0000","last_became_active":"2024-04-17T09:56:01.208605+0000","last_became_peered":"2024-04-17T09:56:01.208605+0000","last_unstale":"2024-04-17T09:56:03.197845+0000","last_undegraded":"2024-04-17T09:56:03.197845+0000","last_fullsized":"2024-04-17T09:56:03.197845+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:36.056444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261792+0000","last_change":"2024-04-17T09:56:02.210781+0000","last_active":"2024-04-17T09:56:05.261792+0000","last_peered":"2024-04-17T09:56:05.261792+0000","last_clean":"2024-04-17T09:56:05.261792+0000","last_became_active":"2024-04-17T09:56:02.210423+0000","last_became_peered":"2024-04-17T09:56:02.210423+0000","last_unstale":"2024-04-17T09:56:05.261792+0000","last_undegraded":"2024-04-17T09:56:05.261792+0000","last_fullsized":"2024-04-17T09:56:05.261792+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:07:34.467885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261230+0000","last_change":"2024-04-17T09:56:01.189844+0000","last_active":"2024-04-17T09:56:05.261230+0000","last_peered":"2024-04-17T09:56:05.261230+0000","last_clean":"2024-04-17T09:56:05.261230+0000","last_became_active":"2024-04-17T09:56:01.189601+0000","last_became_peered":"2024-04-17T09:56:01.189601+0000","last_unstale":"2024-04-17T09:56:05.261230+0000","last_undegraded":"2024-04-17T09:56:05.261230+0000","last_fullsized":"2024-04-17T09:56:05.261230+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:09:38.167038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260885+0000","last_change":"2024-04-17T09:56:02.222099+0000","last_active":"2024-04-17T09:56:05.260885+0000","last_peered":"2024-04-17T09:56:05.260885+0000","last_clean":"2024-04-17T09:56:05.260885+0000","last_became_active":"2024-04-17T09:56:02.221484+0000","last_became_peered":"2024-04-17T09:56:02.221484+0000","last_unstale":"2024-04-17T09:56:05.260885+0000","last_undegraded":"2024-04-17T09:56:05.260885+0000","last_fullsized":"2024-04-17T09:56:05.260885+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:34:44.623141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197523+0000","last_change":"2024-04-17T09:56:01.203071+0000","last_active":"2024-04-17T09:56:03.197523+0000","last_peered":"2024-04-17T09:56:03.197523+0000","last_clean":"2024-04-17T09:56:03.197523+0000","last_became_active":"2024-04-17T09:56:01.202764+0000","last_became_peered":"2024-04-17T09:56:01.202764+0000","last_unstale":"2024-04-17T09:56:03.197523+0000","last_undegraded":"2024-04-17T09:56:03.197523+0000","last_fullsized":"2024-04-17T09:56:03.197523+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:22.597577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260935+0000","last_change":"2024-04-17T09:56:02.222320+0000","last_active":"2024-04-17T09:56:05.260935+0000","last_peered":"2024-04-17T09:56:05.260935+0000","last_clean":"2024-04-17T09:56:05.260935+0000","last_became_active":"2024-04-17T09:56:02.222071+0000","last_became_peered":"2024-04-17T09:56:02.222071+0000","last_unstale":"2024-04-17T09:56:05.260935+0000","last_undegraded":"2024-04-17T09:56:05.260935+0000","last_fullsized":"2024-04-17T09:56:05.260935+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:00.240451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262449+0000","last_change":"2024-04-17T09:56:01.200432+0000","last_active":"2024-04-17T09:56:05.262449+0000","last_peered":"2024-04-17T09:56:05.262449+0000","last_clean":"2024-04-17T09:56:05.262449+0000","last_became_active":"2024-04-17T09:56:01.200159+0000","last_became_peered":"2024-04-17T09:56:01.200159+0000","last_unstale":"2024-04-17T09:56:05.262449+0000","last_undegraded":"2024-04-17T09:56:05.262449+0000","last_fullsized":"2024-04-17T09:56:05.262449+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:32:38.627725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197598+0000","last_change":"2024-04-17T09:56:02.211557+0000","last_active":"2024-04-17T09:56:03.197598+0000","last_peered":"2024-04-17T09:56:03.197598+0000","last_clean":"2024-04-17T09:56:03.197598+0000","last_became_active":"2024-04-17T09:56:02.211186+0000","last_became_peered":"2024-04-17T09:56:02.211186+0000","last_unstale":"2024-04-17T09:56:03.197598+0000","last_undegraded":"2024-04-17T09:56:03.197598+0000","last_fullsized":"2024-04-17T09:56:03.197598+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:42:46.838421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262265+0000","last_change":"2024-04-17T09:56:01.200296+0000","last_active":"2024-04-17T09:56:05.262265+0000","last_peered":"2024-04-17T09:56:05.262265+0000","last_clean":"2024-04-17T09:56:05.262265+0000","last_became_active":"2024-04-17T09:56:01.200020+0000","last_became_peered":"2024-04-17T09:56:01.200020+0000","last_unstale":"2024-04-17T09:56:05.262265+0000","last_undegraded":"2024-04-17T09:56:05.262265+0000","last_fullsized":"2024-04-17T09:56:05.262265+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:54:35.183221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197057+0000","last_change":"2024-04-17T09:56:02.211632+0000","last_active":"2024-04-17T09:56:03.197057+0000","last_peered":"2024-04-17T09:56:03.197057+0000","last_clean":"2024-04-17T09:56:03.197057+0000","last_became_active":"2024-04-17T09:56:02.211357+0000","last_became_peered":"2024-04-17T09:56:02.211357+0000","last_unstale":"2024-04-17T09:56:03.197057+0000","last_undegraded":"2024-04-17T09:56:03.197057+0000","last_fullsized":"2024-04-17T09:56:03.197057+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:16:53.336176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.259831+0000","last_change":"2024-04-17T09:56:01.195917+0000","last_active":"2024-04-17T09:56:05.259831+0000","last_peered":"2024-04-17T09:56:05.259831+0000","last_clean":"2024-04-17T09:56:05.259831+0000","last_became_active":"2024-04-17T09:56:01.195682+0000","last_became_peered":"2024-04-17T09:56:01.195682+0000","last_unstale":"2024-04-17T09:56:05.259831+0000","last_undegraded":"2024-04-17T09:56:05.259831+0000","last_fullsized":"2024-04-17T09:56:05.259831+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:46.432533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261254+0000","last_change":"2024-04-17T09:56:02.218398+0000","last_active":"2024-04-17T09:56:05.261254+0000","last_peered":"2024-04-17T09:56:05.261254+0000","last_clean":"2024-04-17T09:56:05.261254+0000","last_became_active":"2024-04-17T09:56:02.218097+0000","last_became_peered":"2024-04-17T09:56:02.218097+0000","last_unstale":"2024-04-17T09:56:05.261254+0000","last_undegraded":"2024-04-17T09:56:05.261254+0000","last_fullsized":"2024-04-17T09:56:05.261254+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:52:40.143883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262592+0000","last_change":"2024-04-17T09:56:01.204766+0000","last_active":"2024-04-17T09:56:05.262592+0000","last_peered":"2024-04-17T09:56:05.262592+0000","last_clean":"2024-04-17T09:56:05.262592+0000","last_became_active":"2024-04-17T09:56:01.204490+0000","last_became_peered":"2024-04-17T09:56:01.204490+0000","last_unstale":"2024-04-17T09:56:05.262592+0000","last_undegraded":"2024-04-17T09:56:05.262592+0000","last_fullsized":"2024-04-17T09:56:05.262592+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:14:07.937430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261265+0000","last_change":"2024-04-17T09:56:02.216520+0000","last_active":"2024-04-17T09:56:05.261265+0000","last_peered":"2024-04-17T09:56:05.261265+0000","last_clean":"2024-04-17T09:56:05.261265+0000","last_became_active":"2024-04-17T09:56:02.216195+0000","last_became_peered":"2024-04-17T09:56:02.216195+0000","last_unstale":"2024-04-17T09:56:05.261265+0000","last_undegraded":"2024-04-17T09:56:05.261265+0000","last_fullsized":"2024-04-17T09:56:05.261265+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:05:05.510667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"57'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203049+0000","last_change":"2024-04-17T09:56:01.202823+0000","last_active":"2024-04-17T09:56:03.203049+0000","last_peered":"2024-04-17T09:56:03.203049+0000","last_clean":"2024-04-17T09:56:03.203049+0000","last_became_active":"2024-04-17T09:56:01.202563+0000","last_became_peered":"2024-04-17T09:56:01.202563+0000","last_unstale":"2024-04-17T09:56:03.203049+0000","last_undegraded":"2024-04-17T09:56:03.203049+0000","last_fullsized":"2024-04-17T09:56:03.203049+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:34:57.389627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262171+0000","last_change":"2024-04-17T09:56:02.215682+0000","last_active":"2024-04-17T09:56:05.262171+0000","last_peered":"2024-04-17T09:56:05.262171+0000","last_clean":"2024-04-17T09:56:05.262171+0000","last_became_active":"2024-04-17T09:56:02.215368+0000","last_became_peered":"2024-04-17T09:56:02.215368+0000","last_unstale":"2024-04-17T09:56:05.262171+0000","last_undegraded":"2024-04-17T09:56:05.262171+0000","last_fullsized":"2024-04-17T09:56:05.262171+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:55:47.229362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.202866+0000","last_change":"2024-04-17T09:56:01.198070+0000","last_active":"2024-04-17T09:56:03.202866+0000","last_peered":"2024-04-17T09:56:03.202866+0000","last_clean":"2024-04-17T09:56:03.202866+0000","last_became_active":"2024-04-17T09:56:01.197737+0000","last_became_peered":"2024-04-17T09:56:01.197737+0000","last_unstale":"2024-04-17T09:56:03.202866+0000","last_undegraded":"2024-04-17T09:56:03.202866+0000","last_fullsized":"2024-04-17T09:56:03.202866+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:28:26.047148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.197782+0000","last_change":"2024-04-17T09:56:02.230708+0000","last_active":"2024-04-17T09:56:03.197782+0000","last_peered":"2024-04-17T09:56:03.197782+0000","last_clean":"2024-04-17T09:56:03.197782+0000","last_became_active":"2024-04-17T09:56:02.230207+0000","last_became_peered":"2024-04-17T09:56:02.230207+0000","last_unstale":"2024-04-17T09:56:03.197782+0000","last_undegraded":"2024-04-17T09:56:03.197782+0000","last_fullsized":"2024-04-17T09:56:03.197782+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:43:27.595946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261138+0000","last_change":"2024-04-17T09:56:01.197907+0000","last_active":"2024-04-17T09:56:05.261138+0000","last_peered":"2024-04-17T09:56:05.261138+0000","last_clean":"2024-04-17T09:56:05.261138+0000","last_became_active":"2024-04-17T09:56:01.197669+0000","last_became_peered":"2024-04-17T09:56:01.197669+0000","last_unstale":"2024-04-17T09:56:05.261138+0000","last_undegraded":"2024-04-17T09:56:05.261138+0000","last_fullsized":"2024-04-17T09:56:05.261138+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:48:06.817655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262071+0000","last_change":"2024-04-17T09:56:02.224786+0000","last_active":"2024-04-17T09:56:05.262071+0000","last_peered":"2024-04-17T09:56:05.262071+0000","last_clean":"2024-04-17T09:56:05.262071+0000","last_became_active":"2024-04-17T09:56:02.224468+0000","last_became_peered":"2024-04-17T09:56:02.224468+0000","last_unstale":"2024-04-17T09:56:05.262071+0000","last_undegraded":"2024-04-17T09:56:05.262071+0000","last_fullsized":"2024-04-17T09:56:05.262071+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:17:53.933023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261567+0000","last_change":"2024-04-17T09:56:01.186515+0000","last_active":"2024-04-17T09:56:05.261567+0000","last_peered":"2024-04-17T09:56:05.261567+0000","last_clean":"2024-04-17T09:56:05.261567+0000","last_became_active":"2024-04-17T09:56:01.186266+0000","last_became_peered":"2024-04-17T09:56:01.186266+0000","last_unstale":"2024-04-17T09:56:05.261567+0000","last_undegraded":"2024-04-17T09:56:05.261567+0000","last_fullsized":"2024-04-17T09:56:05.261567+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:27:04.257237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.260715+0000","last_change":"2024-04-17T09:56:02.219403+0000","last_active":"2024-04-17T09:56:05.260715+0000","last_peered":"2024-04-17T09:56:05.260715+0000","last_clean":"2024-04-17T09:56:05.260715+0000","last_became_active":"2024-04-17T09:56:02.217760+0000","last_became_peered":"2024-04-17T09:56:02.217760+0000","last_unstale":"2024-04-17T09:56:05.260715+0000","last_undegraded":"2024-04-17T09:56:05.260715+0000","last_fullsized":"2024-04-17T09:56:05.260715+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:46.319880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.203268+0000","last_change":"2024-04-17T09:56:01.207103+0000","last_active":"2024-04-17T09:56:03.203268+0000","last_peered":"2024-04-17T09:56:03.203268+0000","last_clean":"2024-04-17T09:56:03.203268+0000","last_became_active":"2024-04-17T09:56:01.206837+0000","last_became_peered":"2024-04-17T09:56:01.206837+0000","last_unstale":"2024-04-17T09:56:03.203268+0000","last_undegraded":"2024-04-17T09:56:03.203268+0000","last_fullsized":"2024-04-17T09:56:03.203268+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:28:36.102136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261326+0000","last_change":"2024-04-17T09:56:02.220640+0000","last_active":"2024-04-17T09:56:05.261326+0000","last_peered":"2024-04-17T09:56:05.261326+0000","last_clean":"2024-04-17T09:56:05.261326+0000","last_became_active":"2024-04-17T09:56:02.219659+0000","last_became_peered":"2024-04-17T09:56:02.219659+0000","last_unstale":"2024-04-17T09:56:05.261326+0000","last_undegraded":"2024-04-17T09:56:05.261326+0000","last_fullsized":"2024-04-17T09:56:05.261326+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:13:28.472859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261631+0000","last_change":"2024-04-17T09:56:01.203488+0000","last_active":"2024-04-17T09:56:05.261631+0000","last_peered":"2024-04-17T09:56:05.261631+0000","last_clean":"2024-04-17T09:56:05.261631+0000","last_became_active":"2024-04-17T09:56:01.199625+0000","last_became_peered":"2024-04-17T09:56:01.199625+0000","last_unstale":"2024-04-17T09:56:05.261631+0000","last_undegraded":"2024-04-17T09:56:05.261631+0000","last_fullsized":"2024-04-17T09:56:05.261631+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:23:44.149563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261403+0000","last_change":"2024-04-17T09:56:02.207578+0000","last_active":"2024-04-17T09:56:05.261403+0000","last_peered":"2024-04-17T09:56:05.261403+0000","last_clean":"2024-04-17T09:56:05.261403+0000","last_became_active":"2024-04-17T09:56:02.207285+0000","last_became_peered":"2024-04-17T09:56:02.207285+0000","last_unstale":"2024-04-17T09:56:05.261403+0000","last_undegraded":"2024-04-17T09:56:05.261403+0000","last_fullsized":"2024-04-17T09:56:05.261403+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:11:27.415411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.262018+0000","last_change":"2024-04-17T09:56:01.200290+0000","last_active":"2024-04-17T09:56:05.262018+0000","last_peered":"2024-04-17T09:56:05.262018+0000","last_clean":"2024-04-17T09:56:05.262018+0000","last_became_active":"2024-04-17T09:56:01.199410+0000","last_became_peered":"2024-04-17T09:56:01.199410+0000","last_unstale":"2024-04-17T09:56:05.262018+0000","last_undegraded":"2024-04-17T09:56:05.262018+0000","last_fullsized":"2024-04-17T09:56:05.262018+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:37:02.536335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.261911+0000","last_change":"2024-04-17T09:56:02.220646+0000","last_active":"2024-04-17T09:56:05.261911+0000","last_peered":"2024-04-17T09:56:05.261911+0000","last_clean":"2024-04-17T09:56:05.261911+0000","last_became_active":"2024-04-17T09:56:02.219995+0000","last_became_peered":"2024-04-17T09:56:02.219995+0000","last_unstale":"2024-04-17T09:56:05.261911+0000","last_undegraded":"2024-04-17T09:56:05.261911+0000","last_fullsized":"2024-04-17T09:56:05.261911+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:01.176148+0000","last_clean_scrub_stamp":"2024-04-17T09:56:01.176148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:28:47.541595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:03.202812+0000","last_change":"2024-04-17T09:56:01.192727+0000","last_active":"2024-04-17T09:56:03.202812+0000","last_peered":"2024-04-17T09:56:03.202812+0000","last_clean":"2024-04-17T09:56:03.202812+0000","last_became_active":"2024-04-17T09:56:01.192476+0000","last_became_peered":"2024-04-17T09:56:01.192476+0000","last_unstale":"2024-04-17T09:56:03.202812+0000","last_undegraded":"2024-04-17T09:56:03.202812+0000","last_fullsized":"2024-04-17T09:56:03.202812+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:56:00.170177+0000","last_clean_scrub_stamp":"2024-04-17T09:56:00.170177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:28:24.596000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":237,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-17T09:56:05.259942+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T09:56:05.259942+0000","last_peered":"2024-04-17T09:56:05.259942+0000","last_clean":"2024-04-17T09:56:05.259942+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T09:56:05.259942+0000","last_undegraded":"2024-04-17T09:56:05.259942+0000","last_fullsized":"2024-04-17T09:56:05.259942+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28260,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343580,"statfs":{"total":96636764160,"available":96607825920,"internally_reserved":0,"allocated":1265664,"data_stored":863788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":675840,"data_stored":279560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509480,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":671744,"data_stored":273912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31836,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":671744,"data_stored":273316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31852,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339988,"statfs":{"total":96636764160,"available":96604147712,"internally_reserved":0,"allocated":684032,"data_stored":279616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":667648,"data_stored":273350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542185,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":688128,"data_stored":281172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542184,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":872122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T09:56:11.276 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T09:56:11.276 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-17T09:56:11.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.456+0000 7f2eff200640 1 Processor -- start 2024-04-17T09:56:11.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.456+0000 7f2eff200640 1 -- start start 2024-04-17T09:56:11.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2eff200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f0013f630 0x7f2f00071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:11.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2eff200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f0015db00 con 0x7f2f00160a30 2024-04-17T09:56:11.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2eff200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f001610c0 con 0x7f2f00072330 2024-04-17T09:56:11.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2eff200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2f0013fa10 con 0x7f2f0013f630 2024-04-17T09:56:11.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efe800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2f00160a30 0x7f2f000ce100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42810/0 (socket says 172.21.15.32:42810) 2024-04-17T09:56:11.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efe800640 1 -- 172.21.15.32:0/85031500 learned_addr learned my addr 172.21.15.32:0/85031500 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:11.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efde00640 1 --2- 172.21.15.32:0/85031500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f0013f630 0x7f2f00071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:11.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efd400640 1 -- 172.21.15.32:0/85031500 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 750298206 0 0) 0x7f2f0015db00 con 0x7f2f00160a30 2024-04-17T09:56:11.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efd400640 1 -- 172.21.15.32:0/85031500 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ee4003530 con 0x7f2f00160a30 2024-04-17T09:56:11.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efd400640 1 -- 172.21.15.32:0/85031500 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2141549935 0 0) 0x7f2f001610c0 con 0x7f2f00072330 2024-04-17T09:56:11.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efd400640 1 -- 172.21.15.32:0/85031500 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f0015db00 con 0x7f2f00072330 2024-04-17T09:56:11.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efd400640 1 -- 172.21.15.32:0/85031500 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1746992075 0 0) 0x7f2ee4003530 con 0x7f2f00160a30 2024-04-17T09:56:11.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efd400640 1 -- 172.21.15.32:0/85031500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f0013f630 msgr2=0x7f2f00071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:11.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efd400640 1 --2- 172.21.15.32:0/85031500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f0013f630 0x7f2f00071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efd400640 1 -- 172.21.15.32:0/85031500 >> v1:172.21.15.144:6789/0 conn(0x7f2f00072330 legacy=0x7f2f00160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:11.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efd400640 1 -- 172.21.15.32:0/85031500 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0015cd40 con 0x7f2f00160a30 2024-04-17T09:56:11.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efd400640 1 -- 172.21.15.32:0/85031500 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f2ef8001a00 con 0x7f2f00160a30 2024-04-17T09:56:11.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efd400640 1 -- 172.21.15.32:0/85031500 >> v1:172.21.15.32:6789/0 conn(0x7f2f00160a30 legacy=0x7f2f000ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:11.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efd400640 1 --2- 172.21.15.32:0/85031500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f0013f630 0x7f2ee4003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:11.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efd400640 1 --2- 172.21.15.32:0/85031500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ee4004150 0x7f2ee4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:11.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efd400640 1 --2- 172.21.15.32:0/85031500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee4004d50 0x7f2ee40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:11.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efd400640 1 -- 172.21.15.32:0/85031500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0013fa10 con 0x7f2ee4004d50 2024-04-17T09:56:11.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efd400640 1 -- 172.21.15.32:0/85031500 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee400bb10 con 0x7f2ee4004150 2024-04-17T09:56:11.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efd400640 1 -- 172.21.15.32:0/85031500 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2ee400bd10 con 0x7f2f0013f630 2024-04-17T09:56:11.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efe800640 1 --2- 172.21.15.32:0/85031500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee4004d50 0x7f2ee40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:11.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efde00640 1 --2- 172.21.15.32:0/85031500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f0013f630 0x7f2ee4003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:11.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2ef5400640 1 --2- 172.21.15.32:0/85031500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ee4004150 0x7f2ee4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:11.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efe800640 1 -- 172.21.15.32:0/85031500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f0013f630 msgr2=0x7f2ee4003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:11.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efe800640 1 --2- 172.21.15.32:0/85031500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f0013f630 0x7f2ee4003bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efe800640 1 -- 172.21.15.32:0/85031500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ee4004150 msgr2=0x7f2ee4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:11.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efe800640 1 --2- 172.21.15.32:0/85031500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ee4004150 0x7f2ee4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2efe800640 1 -- 172.21.15.32:0/85031500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee4003530 con 0x7f2ee4004d50 2024-04-17T09:56:11.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.460+0000 7f2ef5400640 1 --2- 172.21.15.32:0/85031500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ee4004150 0x7f2ee4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:11.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.464+0000 7f2efe800640 1 --2- 172.21.15.32:0/85031500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee4004d50 0x7f2ee40051d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800a820 tx=0x7f2ef801fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d18f94e50ccf458 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:11.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.464+0000 7f2efd400640 1 -- 172.21.15.32:0/85031500 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef800a450 con 0x7f2ee4004d50 2024-04-17T09:56:11.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.464+0000 7f2efd400640 1 -- 172.21.15.32:0/85031500 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef80b0070 con 0x7f2ee4004d50 2024-04-17T09:56:11.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.464+0000 7f2efd400640 1 -- 172.21.15.32:0/85031500 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80aa950 con 0x7f2ee4004d50 2024-04-17T09:56:11.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.464+0000 7f2eff200640 1 -- 172.21.15.32:0/85031500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee4004d50 msgr2=0x7f2ee40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:11.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.464+0000 7f2eff200640 1 --2- 172.21.15.32:0/85031500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee4004d50 0x7f2ee40051d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800a820 tx=0x7f2ef801fa60 comp rx=0 tx=0).stop 2024-04-17T09:56:11.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.464+0000 7f2efe800640 1 -- 172.21.15.32:0/85031500 reap_dead start 2024-04-17T09:56:11.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.464+0000 7f2eff200640 1 -- 172.21.15.32:0/85031500 shutdown_connections 2024-04-17T09:56:11.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.464+0000 7f2eff200640 1 -- 172.21.15.32:0/85031500 >> 172.21.15.32:0/85031500 conn(0x7f2f00067d70 msgr2=0x7f2f0006ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:11.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.464+0000 7f2eff200640 1 -- 172.21.15.32:0/85031500 shutdown_connections 2024-04-17T09:56:11.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.464+0000 7f2eff200640 1 -- 172.21.15.32:0/85031500 wait complete. 2024-04-17T09:56:11.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.464+0000 7f2eff200640 1 Processor -- start 2024-04-17T09:56:11.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.464+0000 7f2eff200640 1 -- start start 2024-04-17T09:56:11.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.468+0000 7f2eff200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ee4004d50 0x7f2f0013e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:11.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.468+0000 7f2eff200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f00155dd0 0x7f2f00158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:11.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.468+0000 7f2eff200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f00159190 0x7f2f000b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:11.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.468+0000 7f2eff200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0015d300 con 0x7f2f00155dd0 2024-04-17T09:56:11.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.468+0000 7f2eff200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2f00072770 con 0x7f2f00159190 2024-04-17T09:56:11.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.468+0000 7f2eff200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2f0015d5e0 con 0x7f2ee4004d50 2024-04-17T09:56:11.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.468+0000 7f2ef5400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f00155dd0 0x7f2f00158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:11.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.468+0000 7f2ef5400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f00155dd0 0x7f2f00158c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46264/0 (socket says 172.21.15.32:46264) 2024-04-17T09:56:11.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.468+0000 7f2ef5400640 1 -- 172.21.15.32:0/3050704610 learned_addr learned my addr 172.21.15.32:0/3050704610 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:11.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.468+0000 7f2ef5400640 1 -- 172.21.15.32:0/3050704610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ee4004d50 msgr2=0x7f2f0013e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:11.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.468+0000 7f2ef5400640 1 --2- 172.21.15.32:0/3050704610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ee4004d50 0x7f2f0013e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.468+0000 7f2ef5400640 1 -- 172.21.15.32:0/3050704610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f00159190 msgr2=0x7f2f000b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:11.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.468+0000 7f2ef5400640 1 --2- 172.21.15.32:0/3050704610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f00159190 0x7f2f000b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.468+0000 7f2ef5400640 1 -- 172.21.15.32:0/3050704610 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f001581b0 con 0x7f2f00155dd0 2024-04-17T09:56:11.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.468+0000 7f2ef5400640 1 --2- 172.21.15.32:0/3050704610 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f00155dd0 0x7f2f00158c50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800e7b0 tx=0x7f2ee800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:11.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.468+0000 7f2ef6a00640 1 -- 172.21.15.32:0/3050704610 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee80050d0 con 0x7f2f00155dd0 2024-04-17T09:56:11.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.468+0000 7f2ef6a00640 1 -- 172.21.15.32:0/3050704610 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ee8005270 con 0x7f2f00155dd0 2024-04-17T09:56:11.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.468+0000 7f2ef6a00640 1 -- 172.21.15.32:0/3050704610 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8005420 con 0x7f2f00155dd0 2024-04-17T09:56:11.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.468+0000 7f2eff200640 1 -- 172.21.15.32:0/3050704610 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f001567b0 con 0x7f2f00155dd0 2024-04-17T09:56:11.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.468+0000 7f2eff200640 1 -- 172.21.15.32:0/3050704610 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f00158730 con 0x7f2f00155dd0 2024-04-17T09:56:11.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.468+0000 7f2eff200640 1 -- 172.21.15.32:0/3050704610 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f0004fda0 con 0x7f2f00155dd0 2024-04-17T09:56:11.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.468+0000 7f2ef6a00640 1 -- 172.21.15.32:0/3050704610 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2ee801a020 con 0x7f2f00155dd0 2024-04-17T09:56:11.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.472+0000 7f2ef6a00640 1 --2- 172.21.15.32:0/3050704610 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2edc068380 0x7f2edc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:11.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.472+0000 7f2ef6a00640 1 -- 172.21.15.32:0/3050704610 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f2ee8086160 con 0x7f2f00155dd0 2024-04-17T09:56:11.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.472+0000 7f2efde00640 1 --2- 172.21.15.32:0/3050704610 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2edc068380 0x7f2edc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:11.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.472+0000 7f2efde00640 1 --2- 172.21.15.32:0/3050704610 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2edc068380 0x7f2edc06a840 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2f00072300 tx=0x7f2ef00095f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:11.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.476+0000 7f2ef6a00640 1 -- 172.21.15.32:0/3050704610 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2ee8056710 con 0x7f2f00155dd0 2024-04-17T09:56:11.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.696+0000 7f2eff200640 1 -- 172.21.15.32:0/3050704610 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f2f0006f970 con 0x7f2f00155dd0 2024-04-17T09:56:11.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.696+0000 7f2ef6a00640 1 -- 172.21.15.32:0/3050704610 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f2ee8056010 con 0x7f2f00155dd0 2024-04-17T09:56:11.699 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-17T09:56:11.700 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDPmx9mUXoLKxAANV7zhLvL0DqHngk4TtsdBQ== 2024-04-17T09:56:11.700 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2024-04-17T09:56:11.700 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2024-04-17T09:56:11.700 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-17T09:56:11.700 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2024-04-17T09:56:11.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.700+0000 7f2eff200640 1 -- 172.21.15.32:0/3050704610 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2edc068380 msgr2=0x7f2edc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:11.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.700+0000 7f2eff200640 1 --2- 172.21.15.32:0/3050704610 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2edc068380 0x7f2edc06a840 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2f00072300 tx=0x7f2ef00095f0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.700+0000 7f2eff200640 1 -- 172.21.15.32:0/3050704610 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f00155dd0 msgr2=0x7f2f00158c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:11.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.700+0000 7f2eff200640 1 --2- 172.21.15.32:0/3050704610 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f00155dd0 0x7f2f00158c50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800e7b0 tx=0x7f2ee800ec70 comp rx=0 tx=0).stop 2024-04-17T09:56:11.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.700+0000 7f2eff200640 1 -- 172.21.15.32:0/3050704610 shutdown_connections 2024-04-17T09:56:11.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.700+0000 7f2eff200640 1 --2- 172.21.15.32:0/3050704610 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2edc068380 0x7f2edc06a840 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.700+0000 7f2eff200640 1 --2- 172.21.15.32:0/3050704610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f00159190 0x7f2f000b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.700+0000 7f2eff200640 1 --2- 172.21.15.32:0/3050704610 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f00155dd0 0x7f2f00158c50 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.700+0000 7f2eff200640 1 --2- 172.21.15.32:0/3050704610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ee4004d50 0x7f2f0013e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.700+0000 7f2eff200640 1 -- 172.21.15.32:0/3050704610 >> 172.21.15.32:0/3050704610 conn(0x7f2f00067d70 msgr2=0x7f2f0015c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:11.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.700+0000 7f2eff200640 1 -- 172.21.15.32:0/3050704610 shutdown_connections 2024-04-17T09:56:11.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.700+0000 7f2eff200640 1 -- 172.21.15.32:0/3050704610 wait complete. 2024-04-17T09:56:11.723 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:56:11.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.860+0000 7fc999800640 1 Processor -- start 2024-04-17T09:56:11.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.860+0000 7fc999800640 1 -- start start 2024-04-17T09:56:11.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc999800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc994145f30 0x7fc994071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:11.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc999800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc99415d8d0 0x7fc994072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:11.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc999800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9940cdac0 con 0x7fc9940cce70 2024-04-17T09:56:11.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc999800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc99406f400 con 0x7fc99415d8d0 2024-04-17T09:56:11.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc999800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc994146310 con 0x7fc994145f30 2024-04-17T09:56:11.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc993400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc9940cce70 0x7fc9940cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42812/0 (socket says 172.21.15.32:42812) 2024-04-17T09:56:11.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc993400640 1 -- 172.21.15.32:0/1424501394 learned_addr learned my addr 172.21.15.32:0/1424501394 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:11.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc992a00640 1 --2- 172.21.15.32:0/1424501394 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc994145f30 0x7fc994071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:11.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc992000640 1 -- 172.21.15.32:0/1424501394 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3364229004 0 0) 0x7fc9940cdac0 con 0x7fc9940cce70 2024-04-17T09:56:11.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc98a000640 1 --2- 172.21.15.32:0/1424501394 >> v2:172.21.15.144:3300/0 conn(0x7fc99415d8d0 0x7fc994072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:11.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc992000640 1 -- 172.21.15.32:0/1424501394 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc978003530 con 0x7fc9940cce70 2024-04-17T09:56:11.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc992000640 1 -- 172.21.15.32:0/1424501394 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3191002094 0 0) 0x7fc978003530 con 0x7fc9940cce70 2024-04-17T09:56:11.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc992000640 1 -- 172.21.15.32:0/1424501394 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc994145f30 msgr2=0x7fc994071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:11.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc992000640 1 --2- 172.21.15.32:0/1424501394 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc994145f30 0x7fc994071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc992000640 1 -- 172.21.15.32:0/1424501394 >> v2:172.21.15.144:3300/0 conn(0x7fc99415d8d0 msgr2=0x7fc994072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:11.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc992000640 1 --2- 172.21.15.32:0/1424501394 >> v2:172.21.15.144:3300/0 conn(0x7fc99415d8d0 0x7fc994072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc992000640 1 -- 172.21.15.32:0/1424501394 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99415cdf0 con 0x7fc9940cce70 2024-04-17T09:56:11.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc992000640 1 -- 172.21.15.32:0/1424501394 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7fc98c001a00 con 0x7fc9940cce70 2024-04-17T09:56:11.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc992000640 1 -- 172.21.15.32:0/1424501394 >> v1:172.21.15.32:6789/0 conn(0x7fc9940cce70 legacy=0x7fc9940cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:11.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc992000640 1 --2- 172.21.15.32:0/1424501394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9780039c0 0x7fc978003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:11.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc992000640 1 --2- 172.21.15.32:0/1424501394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc978004810 0x7fc978004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:11.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc992000640 1 --2- 172.21.15.32:0/1424501394 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc994145f30 0x7fc9780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:11.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc992000640 1 -- 172.21.15.32:0/1424501394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc99406f400 con 0x7fc9780039c0 2024-04-17T09:56:11.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc992000640 1 -- 172.21.15.32:0/1424501394 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc994146310 con 0x7fc978004810 2024-04-17T09:56:11.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc992000640 1 -- 172.21.15.32:0/1424501394 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc97800b9d0 con 0x7fc994145f30 2024-04-17T09:56:11.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc992a00640 1 --2- 172.21.15.32:0/1424501394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9780039c0 0x7fc978003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:11.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc98a000640 1 --2- 172.21.15.32:0/1424501394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc978004810 0x7fc978004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:11.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc993400640 1 --2- 172.21.15.32:0/1424501394 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc994145f30 0x7fc9780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:11.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc992a00640 1 -- 172.21.15.32:0/1424501394 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc994145f30 msgr2=0x7fc9780051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:11.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc992a00640 1 --2- 172.21.15.32:0/1424501394 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc994145f30 0x7fc9780051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc992a00640 1 -- 172.21.15.32:0/1424501394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc978004810 msgr2=0x7fc978004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:11.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc992a00640 1 --2- 172.21.15.32:0/1424501394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc978004810 0x7fc978004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.864+0000 7fc992a00640 1 -- 172.21.15.32:0/1424501394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc978003530 con 0x7fc9780039c0 2024-04-17T09:56:11.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.868+0000 7fc992a00640 1 --2- 172.21.15.32:0/1424501394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9780039c0 0x7fc978003de0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc984009ae0 tx=0x7fc98401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af355e72f4e63d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:11.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.868+0000 7fc992000640 1 -- 172.21.15.32:0/1424501394 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9840096b0 con 0x7fc9780039c0 2024-04-17T09:56:11.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.868+0000 7fc992000640 1 -- 172.21.15.32:0/1424501394 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9840a6480 con 0x7fc9780039c0 2024-04-17T09:56:11.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.868+0000 7fc992000640 1 -- 172.21.15.32:0/1424501394 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9840a6870 con 0x7fc9780039c0 2024-04-17T09:56:11.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.868+0000 7fc999800640 1 -- 172.21.15.32:0/1424501394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9780039c0 msgr2=0x7fc978003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:11.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.868+0000 7fc999800640 1 --2- 172.21.15.32:0/1424501394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9780039c0 0x7fc978003de0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc984009ae0 tx=0x7fc98401ca70 comp rx=0 tx=0).stop 2024-04-17T09:56:11.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.868+0000 7fc993400640 1 -- 172.21.15.32:0/1424501394 reap_dead start 2024-04-17T09:56:11.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.868+0000 7fc999800640 1 -- 172.21.15.32:0/1424501394 shutdown_connections 2024-04-17T09:56:11.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.868+0000 7fc999800640 1 -- 172.21.15.32:0/1424501394 >> 172.21.15.32:0/1424501394 conn(0x7fc99406ba30 msgr2=0x7fc99406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:11.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.868+0000 7fc999800640 1 -- 172.21.15.32:0/1424501394 shutdown_connections 2024-04-17T09:56:11.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.868+0000 7fc999800640 1 -- 172.21.15.32:0/1424501394 wait complete. 2024-04-17T09:56:11.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.868+0000 7fc999800640 1 Processor -- start 2024-04-17T09:56:11.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.868+0000 7fc999800640 1 -- start start 2024-04-17T09:56:11.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.872+0000 7fc999800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9780039c0 0x7fc9940b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:11.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.872+0000 7fc999800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc994132370 0x7fc994136090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:11.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.872+0000 7fc999800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9941e76e0 0x7fc9941e7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:11.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.872+0000 7fc999800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc99415cb90 con 0x7fc9941e76e0 2024-04-17T09:56:11.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.872+0000 7fc999800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc99415d440 con 0x7fc994132370 2024-04-17T09:56:11.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.872+0000 7fc999800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc994071df0 con 0x7fc9780039c0 2024-04-17T09:56:11.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.872+0000 7fc993400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9941e76e0 0x7fc9941e7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:11.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.872+0000 7fc993400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9941e76e0 0x7fc9941e7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46290/0 (socket says 172.21.15.32:46290) 2024-04-17T09:56:11.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.872+0000 7fc993400640 1 -- 172.21.15.32:0/958989196 learned_addr learned my addr 172.21.15.32:0/958989196 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:11.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.872+0000 7fc992a00640 1 --2- 172.21.15.32:0/958989196 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9780039c0 0x7fc9940b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:11.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.872+0000 7fc98a000640 1 --2- 172.21.15.32:0/958989196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc994132370 0x7fc994136090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:11.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.872+0000 7fc993400640 1 -- 172.21.15.32:0/958989196 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9780039c0 msgr2=0x7fc9940b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:11.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.872+0000 7fc993400640 1 --2- 172.21.15.32:0/958989196 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9780039c0 0x7fc9940b5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.872+0000 7fc993400640 1 -- 172.21.15.32:0/958989196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc994132370 msgr2=0x7fc994136090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:11.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.872+0000 7fc993400640 1 --2- 172.21.15.32:0/958989196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc994132370 0x7fc994136090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:11.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.872+0000 7fc993400640 1 -- 172.21.15.32:0/958989196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc994132ba0 con 0x7fc9941e76e0 2024-04-17T09:56:11.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.872+0000 7fc993400640 1 --2- 172.21.15.32:0/958989196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9941e76e0 0x7fc9941e7b40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc9940b5880 tx=0x7fc98c011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:11.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.872+0000 7fc98b400640 1 -- 172.21.15.32:0/958989196 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c010040 con 0x7fc9941e76e0 2024-04-17T09:56:11.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.872+0000 7fc98b400640 1 -- 172.21.15.32:0/958989196 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98c01a070 con 0x7fc9941e76e0 2024-04-17T09:56:11.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.872+0000 7fc98b400640 1 -- 172.21.15.32:0/958989196 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c00fba0 con 0x7fc9941e76e0 2024-04-17T09:56:11.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.872+0000 7fc999800640 1 -- 172.21.15.32:0/958989196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc994132e80 con 0x7fc9941e76e0 2024-04-17T09:56:11.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.872+0000 7fc999800640 1 -- 172.21.15.32:0/958989196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9940c9810 con 0x7fc9941e76e0 2024-04-17T09:56:11.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.872+0000 7fc999800640 1 -- 172.21.15.32:0/958989196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc994068a90 con 0x7fc9941e76e0 2024-04-17T09:56:11.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.876+0000 7fc98b400640 1 -- 172.21.15.32:0/958989196 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc98c00fd40 con 0x7fc9941e76e0 2024-04-17T09:56:11.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.876+0000 7fc98b400640 1 --2- 172.21.15.32:0/958989196 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc970068380 0x7fc97006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:11.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.876+0000 7fc98b400640 1 -- 172.21.15.32:0/958989196 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fc98c0896c0 con 0x7fc9941e76e0 2024-04-17T09:56:11.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.876+0000 7fc992a00640 1 --2- 172.21.15.32:0/958989196 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc970068380 0x7fc97006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:11.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.876+0000 7fc992a00640 1 --2- 172.21.15.32:0/958989196 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc970068380 0x7fc97006a840 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc9840ad000 tx=0x7fc9840b3040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:11.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:11.880+0000 7fc98b400640 1 -- 172.21.15.32:0/958989196 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc98c059620 con 0x7fc9941e76e0 2024-04-17T09:56:12.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.080+0000 7fc999800640 1 -- 172.21.15.32:0/958989196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc994072f90 con 0x7fc9941e76e0 2024-04-17T09:56:12.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.080+0000 7fc98b400640 1 -- 172.21.15.32:0/958989196 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5202 (secure 0 0 0) 0x7fc98c05d920 con 0x7fc9941e76e0 2024-04-17T09:56:12.080 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:12.080 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5495,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/1236891874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1236891874},{"type":"v1","addr":"172.21.15.144:6834","nonce":1236891874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5498,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3538881342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3538881342},{"type":"v1","addr":"172.21.15.144:6835","nonce":3538881342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6201,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3549801633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3549801633},{"type":"v1","addr":"172.21.15.32:6836","nonce":3549801633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:56:01.579180+0000","modified":"2024-04-17T09:56:04.622581+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/866826055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":866826055},{"type":"v1","addr":"172.21.15.32:6837","nonce":866826055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-04-17T09:56:12.081 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 32 2024-04-17T09:56:12.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.080+0000 7fc999800640 1 -- 172.21.15.32:0/958989196 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc970068380 msgr2=0x7fc97006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:12.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.080+0000 7fc999800640 1 --2- 172.21.15.32:0/958989196 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc970068380 0x7fc97006a840 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc9840ad000 tx=0x7fc9840b3040 comp rx=0 tx=0).stop 2024-04-17T09:56:12.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.080+0000 7fc999800640 1 -- 172.21.15.32:0/958989196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9941e76e0 msgr2=0x7fc9941e7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:12.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.080+0000 7fc999800640 1 --2- 172.21.15.32:0/958989196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9941e76e0 0x7fc9941e7b40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc9940b5880 tx=0x7fc98c011c20 comp rx=0 tx=0).stop 2024-04-17T09:56:12.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.080+0000 7fc999800640 1 -- 172.21.15.32:0/958989196 shutdown_connections 2024-04-17T09:56:12.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.080+0000 7fc999800640 1 --2- 172.21.15.32:0/958989196 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc970068380 0x7fc97006a840 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.080+0000 7fc999800640 1 --2- 172.21.15.32:0/958989196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9941e76e0 0x7fc9941e7b40 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.080+0000 7fc999800640 1 --2- 172.21.15.32:0/958989196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc994132370 0x7fc994136090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.080+0000 7fc999800640 1 --2- 172.21.15.32:0/958989196 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9780039c0 0x7fc9940b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.080+0000 7fc999800640 1 -- 172.21.15.32:0/958989196 >> 172.21.15.32:0/958989196 conn(0x7fc99406ba30 msgr2=0x7fc99406be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:12.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.080+0000 7fc999800640 1 -- 172.21.15.32:0/958989196 shutdown_connections 2024-04-17T09:56:12.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.080+0000 7fc999800640 1 -- 172.21.15.32:0/958989196 wait complete. 2024-04-17T09:56:12.504 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T09:56:01.579180+0000', 'modified': '2024-04-17T09:56:04.622581+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6204}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6204': {'gid': 6204, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6837/866826055', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6835', 'nonce': 866826055}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 866826055}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6204, 'qdb_cluster': [6204]} 2024-04-17T09:56:12.504 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T09:56:12.504 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-17T09:56:12.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.648+0000 7f5313800640 1 Processor -- start 2024-04-17T09:56:12.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.648+0000 7f5313800640 1 -- start start 2024-04-17T09:56:12.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.648+0000 7f5313800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f530c0cf370 0x7f530c0d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.648+0000 7f5313800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f530c0d0820 0x7f530c071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.648+0000 7f5313800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f530c0cff60 con 0x7f530c13f6e0 2024-04-17T09:56:12.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.648+0000 7f5313800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f530c13fac0 con 0x7f530c0d0820 2024-04-17T09:56:12.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.648+0000 7f5313800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f530c1601e0 con 0x7f530c0cf370 2024-04-17T09:56:12.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.648+0000 7f5310e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f530c13f6e0 0x7f530c0cf850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42824/0 (socket says 172.21.15.32:42824) 2024-04-17T09:56:12.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.648+0000 7f5310e00640 1 -- 172.21.15.32:0/3273501782 learned_addr learned my addr 172.21.15.32:0/3273501782 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:12.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.648+0000 7f5311800640 1 --2- 172.21.15.32:0/3273501782 >> v2:172.21.15.32:3300/0 conn(0x7f530c0d0820 0x7f530c071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.648+0000 7f530be00640 1 --2- 172.21.15.32:0/3273501782 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f530c0cf370 0x7f530c0d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.648+0000 7f530b400640 1 -- 172.21.15.32:0/3273501782 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 408351575 0 0) 0x7f530c0cff60 con 0x7f530c13f6e0 2024-04-17T09:56:12.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.648+0000 7f530b400640 1 -- 172.21.15.32:0/3273501782 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52f4003530 con 0x7f530c13f6e0 2024-04-17T09:56:12.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.648+0000 7f5311800640 1 -- 172.21.15.32:0/3273501782 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f530c0cf370 msgr2=0x7f530c0d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:12.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.648+0000 7f5311800640 1 --2- 172.21.15.32:0/3273501782 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f530c0cf370 0x7f530c0d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.648+0000 7f5311800640 1 -- 172.21.15.32:0/3273501782 >> v1:172.21.15.32:6789/0 conn(0x7f530c13f6e0 legacy=0x7f530c0cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:12.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.648+0000 7f5311800640 1 -- 172.21.15.32:0/3273501782 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530c160480 con 0x7f530c0d0820 2024-04-17T09:56:12.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.652+0000 7f5311800640 1 --2- 172.21.15.32:0/3273501782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530c0d0820 0x7f530c071d80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5304009d90 tx=0x7f530401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c7ab2dda50bcaf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:12.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.652+0000 7f530b400640 1 -- 172.21.15.32:0/3273501782 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53040b2070 con 0x7f530c0d0820 2024-04-17T09:56:12.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.652+0000 7f530b400640 1 -- 172.21.15.32:0/3273501782 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53040a5920 con 0x7f530c0d0820 2024-04-17T09:56:12.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.652+0000 7f530b400640 1 -- 172.21.15.32:0/3273501782 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53040a5cf0 con 0x7f530c0d0820 2024-04-17T09:56:12.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.652+0000 7f5313800640 1 -- 172.21.15.32:0/3273501782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530c0d0820 msgr2=0x7f530c071d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:12.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.652+0000 7f5313800640 1 --2- 172.21.15.32:0/3273501782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530c0d0820 0x7f530c071d80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5304009d90 tx=0x7f530401c970 comp rx=0 tx=0).stop 2024-04-17T09:56:12.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.652+0000 7f5313800640 1 -- 172.21.15.32:0/3273501782 shutdown_connections 2024-04-17T09:56:12.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.652+0000 7f5313800640 1 --2- 172.21.15.32:0/3273501782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530c0d0820 0x7f530c071d80 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.652+0000 7f5313800640 1 --2- 172.21.15.32:0/3273501782 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f530c0cf370 0x7f530c0d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.652+0000 7f5313800640 1 -- 172.21.15.32:0/3273501782 >> 172.21.15.32:0/3273501782 conn(0x7f530c06ba30 msgr2=0x7f530c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:12.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.652+0000 7f5313800640 1 -- 172.21.15.32:0/3273501782 shutdown_connections 2024-04-17T09:56:12.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.652+0000 7f5313800640 1 -- 172.21.15.32:0/3273501782 wait complete. 2024-04-17T09:56:12.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f5313800640 1 Processor -- start 2024-04-17T09:56:12.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f5313800640 1 -- start start 2024-04-17T09:56:12.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f5313800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530c0cf370 0x7f530c0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f5313800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f530c0d0820 0x7f530c158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f5313800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f530c13f6e0 0x7f530c139d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f5313800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f530c072ad0 con 0x7f530c0cf370 2024-04-17T09:56:12.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f5313800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f530c13fd40 con 0x7f530c13f6e0 2024-04-17T09:56:12.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f5313800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f530c072c50 con 0x7f530c0d0820 2024-04-17T09:56:12.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f5310e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530c0cf370 0x7f530c0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f5310e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530c0cf370 0x7f530c0affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46320/0 (socket says 172.21.15.32:46320) 2024-04-17T09:56:12.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f5310e00640 1 -- 172.21.15.32:0/428451766 learned_addr learned my addr 172.21.15.32:0/428451766 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f530be00640 1 --2- 172.21.15.32:0/428451766 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f530c0d0820 0x7f530c158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f5311800640 1 --2- 172.21.15.32:0/428451766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f530c13f6e0 0x7f530c139d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f5310e00640 1 -- 172.21.15.32:0/428451766 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f530c0d0820 msgr2=0x7f530c158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f5310e00640 1 --2- 172.21.15.32:0/428451766 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f530c0d0820 0x7f530c158370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f5310e00640 1 -- 172.21.15.32:0/428451766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f530c13f6e0 msgr2=0x7f530c139d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f5310e00640 1 --2- 172.21.15.32:0/428451766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f530c13f6e0 0x7f530c139d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f5310e00640 1 -- 172.21.15.32:0/428451766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530c144080 con 0x7f530c0cf370 2024-04-17T09:56:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f5310e00640 1 --2- 172.21.15.32:0/428451766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530c0cf370 0x7f530c0affb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f530000c480 tx=0x7f530000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f5308c00640 1 -- 172.21.15.32:0/428451766 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300017070 con 0x7f530c0cf370 2024-04-17T09:56:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f5308c00640 1 -- 172.21.15.32:0/428451766 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5300005cf0 con 0x7f530c0cf370 2024-04-17T09:56:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f5313800640 1 -- 172.21.15.32:0/428451766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f530c144360 con 0x7f530c0cf370 2024-04-17T09:56:12.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f5308c00640 1 -- 172.21.15.32:0/428451766 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300007290 con 0x7f530c0cf370 2024-04-17T09:56:12.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.656+0000 7f5313800640 1 -- 172.21.15.32:0/428451766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f530c155a70 con 0x7f530c0cf370 2024-04-17T09:56:12.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.660+0000 7f5308c00640 1 -- 172.21.15.32:0/428451766 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f530001a070 con 0x7f530c0cf370 2024-04-17T09:56:12.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.660+0000 7f5308c00640 1 --2- 172.21.15.32:0/428451766 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f52ec0683b0 0x7f52ec06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.660+0000 7f5308c00640 1 -- 172.21.15.32:0/428451766 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f5300085db0 con 0x7f530c0cf370 2024-04-17T09:56:12.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.660+0000 7f530be00640 1 --2- 172.21.15.32:0/428451766 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f52ec0683b0 0x7f52ec06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.660+0000 7f5316c8d000 1 Processor -- start 2024-04-17T09:56:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.660+0000 7f5316c8d000 1 -- start start 2024-04-17T09:56:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.660+0000 7f530be00640 1 --2- 172.21.15.32:0/428451766 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f52ec0683b0 0x7f52ec06a870 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f530c0cfba0 tx=0x7f52f8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0ebbc0 0x55d4ea0ebfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ea0f1f20 0x55d4ea0f2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ea0f2840 0x55d4ea135fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d4e9f354d0 con 0x55d4ea0ebbc0 2024-04-17T09:56:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d4e9ecfbf0 con 0x55d4ea0f1f20 2024-04-17T09:56:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55d4e9f31490 con 0x55d4ea0f2840 2024-04-17T09:56:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5310e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0ebbc0 0x55d4ea0ebfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5310e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0ebbc0 0x55d4ea0ebfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46332/0 (socket says 172.21.15.32:46332) 2024-04-17T09:56:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5310e00640 1 -- 172.21.15.32:0/453669766 learned_addr learned my addr 172.21.15.32:0/453669766 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5311800640 1 --2- 172.21.15.32:0/453669766 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ea0f2840 0x55d4ea135fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f530be00640 1 --2- 172.21.15.32:0/453669766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ea0f1f20 0x55d4ea0f2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5310e00640 1 -- 172.21.15.32:0/453669766 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ea0f2840 msgr2=0x55d4ea135fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5310e00640 1 --2- 172.21.15.32:0/453669766 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ea0f2840 0x55d4ea135fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5310e00640 1 -- 172.21.15.32:0/453669766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ea0f1f20 msgr2=0x55d4ea0f2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5310e00640 1 --2- 172.21.15.32:0/453669766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ea0f1f20 0x55d4ea0f2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5310e00640 1 -- 172.21.15.32:0/453669766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4e9ebbe80 con 0x55d4ea0ebbc0 2024-04-17T09:56:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5310e00640 1 --2- 172.21.15.32:0/453669766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0ebbc0 0x55d4ea0ebfa0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5300084240 tx=0x7f5300055d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a01d87f965281be server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f52fd600640 1 -- 172.21.15.32:0/453669766 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530006e020 con 0x55d4ea0ebbc0 2024-04-17T09:56:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f52fd600640 1 -- 172.21.15.32:0/453669766 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5300053d00 con 0x55d4ea0ebbc0 2024-04-17T09:56:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f52fd600640 1 -- 172.21.15.32:0/453669766 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53000522d0 con 0x55d4ea0ebbc0 2024-04-17T09:56:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 1 -- 172.21.15.32:0/453669766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0ebbc0 msgr2=0x55d4ea0ebfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 1 --2- 172.21.15.32:0/453669766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0ebbc0 0x55d4ea0ebfa0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5300084240 tx=0x7f5300055d20 comp rx=0 tx=0).stop 2024-04-17T09:56:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 1 -- 172.21.15.32:0/453669766 shutdown_connections 2024-04-17T09:56:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 1 --2- 172.21.15.32:0/453669766 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ea0f2840 0x55d4ea135fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 1 --2- 172.21.15.32:0/453669766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ea0f1f20 0x55d4ea0f2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 1 --2- 172.21.15.32:0/453669766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0ebbc0 0x55d4ea0ebfa0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 1 -- 172.21.15.32:0/453669766 >> 172.21.15.32:0/453669766 conn(0x55d4ea0ed470 msgr2=0x55d4ea0e6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 1 -- 172.21.15.32:0/453669766 shutdown_connections 2024-04-17T09:56:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 1 -- 172.21.15.32:0/453669766 wait complete. 2024-04-17T09:56:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 1 -- start start 2024-04-17T09:56:12.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 1 Processor -- start 2024-04-17T09:56:12.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0ebbc0 0x55d4ea13ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ea0f1f20 0x55d4ea13bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ea0f2840 0x55d4ea0f6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d4e9ef2a60 con 0x55d4ea0ebbc0 2024-04-17T09:56:12.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d4e9ed9e80 con 0x55d4ea0f2840 2024-04-17T09:56:12.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55d4e9ef0cf0 con 0x55d4ea0f1f20 2024-04-17T09:56:12.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.664+0000 7f5316c8d000 10 client.0 _ll_register_callbacks cb 0x55d4ea0d2a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:56:12.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5310e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0ebbc0 0x55d4ea13ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5310e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0ebbc0 0x55d4ea13ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46348/0 (socket says 172.21.15.32:46348) 2024-04-17T09:56:12.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5310e00640 1 -- 172.21.15.32:0/1998338160 learned_addr learned my addr 172.21.15.32:0/1998338160 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:12.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f530be00640 1 --2- 172.21.15.32:0/1998338160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ea0f1f20 0x55d4ea13bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5311800640 1 --2- 172.21.15.32:0/1998338160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ea0f2840 0x55d4ea0f6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5310e00640 1 -- 172.21.15.32:0/1998338160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ea0f1f20 msgr2=0x55d4ea13bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:12.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5310e00640 1 --2- 172.21.15.32:0/1998338160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ea0f1f20 0x55d4ea13bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5310e00640 1 -- 172.21.15.32:0/1998338160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ea0f2840 msgr2=0x55d4ea0f6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:12.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5310e00640 1 --2- 172.21.15.32:0/1998338160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ea0f2840 0x55d4ea0f6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5310e00640 1 -- 172.21.15.32:0/1998338160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4e9ecfe40 con 0x55d4ea0ebbc0 2024-04-17T09:56:12.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5310e00640 1 --2- 172.21.15.32:0/1998338160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0ebbc0 0x55d4ea13ba50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5300055960 tx=0x7f5300000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf05abaf1e81405d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:12.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f52f2000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:56:12.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f52f2000640 1 -- 172.21.15.32:0/1998338160 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300072070 con 0x55d4ea0ebbc0 2024-04-17T09:56:12.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f52f2000640 1 -- 172.21.15.32:0/1998338160 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f530005e5b0 con 0x55d4ea0ebbc0 2024-04-17T09:56:12.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5316c8d000 1 -- 172.21.15.32:0/1998338160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5300053d00 con 0x55d4ea0ebbc0 2024-04-17T09:56:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f52f2000640 1 -- 172.21.15.32:0/1998338160 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f53000550b0 con 0x55d4ea0ebbc0 2024-04-17T09:56:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5316c8d000 10 client.6378 fetch_fsmap learned FSMap version 32 2024-04-17T09:56:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5316c8d000 1 -- 172.21.15.32:0/1998338160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x55d4e9f00e00 con 0x55d4ea0ebbc0 2024-04-17T09:56:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f52f2000640 1 -- 172.21.15.32:0/1998338160 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5300053d00 con 0x55d4ea0ebbc0 2024-04-17T09:56:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f52f2000640 1 -- 172.21.15.32:0/1998338160 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 32) ==== 3495+0+0 (secure 0 0 0) 0x7f5300055250 con 0x55d4ea0ebbc0 2024-04-17T09:56:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5316c8d000 10 client.6378 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T09:56:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5316c8d000 10 client.6378 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T09:56:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5316c8d000 20 client.6378 populate_metadata read hostname 'smithi032' 2024-04-17T09:56:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5316c8d000 1 --2- 172.21.15.32:0/1998338160 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x55d4ea0f7210 0x55d4ea0f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5316c8d000 4 client.6378 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:56:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5316c8d000 1 -- 172.21.15.32:0/1998338160 --> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4e9ecebc0 con 0x55d4ea0f7210 2024-04-17T09:56:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f530be00640 1 --2- 172.21.15.32:0/1998338160 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x55d4ea0f7210 0x55d4ea0f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f52f2000640 10 client.6378 ms_handle_connect on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f530be00640 1 --2- 172.21.15.32:0/1998338160 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x55d4ea0f7210 0x55d4ea0f75f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=904cbd3dfb07cd7b server_cookie=83c6e2dc5d967a4e in_seq=0 out_seq=0 2024-04-17T09:56:12.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f52f2000640 1 -- 172.21.15.32:0/1998338160 <== mds.0 v2:172.21.15.32:6835/866826055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d4e9ecebc0 con 0x55d4ea0f7210 2024-04-17T09:56:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f52f2000640 10 client.6378 handle_command_reply: tid=0 2024-04-17T09:56:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5316c8d000 1 client.6378 shutdown 2024-04-17T09:56:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5316c8d000 1 -- 172.21.15.32:0/1998338160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0ebbc0 msgr2=0x55d4ea13ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5316c8d000 1 --2- 172.21.15.32:0/1998338160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0ebbc0 0x55d4ea13ba50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5300055960 tx=0x7f5300000bd0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5316c8d000 1 -- 172.21.15.32:0/1998338160 shutdown_connections 2024-04-17T09:56:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5316c8d000 1 --2- 172.21.15.32:0/1998338160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ea0f1f20 0x55d4ea13bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5316c8d000 1 --2- 172.21.15.32:0/1998338160 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x55d4ea0f7210 0x55d4ea0f75f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5316c8d000 1 --2- 172.21.15.32:0/1998338160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ea0f2840 0x55d4ea0f6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5316c8d000 1 --2- 172.21.15.32:0/1998338160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0ebbc0 0x55d4ea13ba50 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f52f2000640 0 client.6378 ms_handle_reset on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5316c8d000 1 -- 172.21.15.32:0/1998338160 >> 172.21.15.32:0/1998338160 conn(0x55d4ea0ed470 msgr2=0x55d4ea0e5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5316c8d000 1 -- 172.21.15.32:0/1998338160 shutdown_connections 2024-04-17T09:56:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5316c8d000 1 -- 172.21.15.32:0/1998338160 wait complete. 2024-04-17T09:56:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.668+0000 7f5316c8d000 20 client.6378 trim_cache size 0 max 16384 2024-04-17T09:56:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.688+0000 7f5316c8d000 1 Processor -- start 2024-04-17T09:56:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.688+0000 7f5316c8d000 1 -- start start 2024-04-17T09:56:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.688+0000 7f5316c8d000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ea0ebbc0 0x55d4ea0e5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.688+0000 7f5316c8d000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0f1f20 0x55d4ea0e5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.688+0000 7f5316c8d000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ea0f2840 0x55d4ea0e6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.688+0000 7f5316c8d000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f530c0b52f0 con 0x55d4ea0f1f20 2024-04-17T09:56:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.688+0000 7f5316c8d000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f530c0b4650 con 0x55d4ea0ebbc0 2024-04-17T09:56:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.688+0000 7f5316c8d000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f530c0b4ca0 con 0x55d4ea0f2840 2024-04-17T09:56:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.688+0000 7f530be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0f1f20 0x55d4ea0e5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.688+0000 7f530be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0f1f20 0x55d4ea0e5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46354/0 (socket says 172.21.15.32:46354) 2024-04-17T09:56:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.688+0000 7f530be00640 1 -- 172.21.15.32:0/1831766890 learned_addr learned my addr 172.21.15.32:0/1831766890 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.688+0000 7f5311800640 1 --2- 172.21.15.32:0/1831766890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ea0f2840 0x55d4ea0e6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.688+0000 7f5310e00640 1 --2- 172.21.15.32:0/1831766890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ea0ebbc0 0x55d4ea0e5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.688+0000 7f530be00640 1 -- 172.21.15.32:0/1831766890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ea0f2840 msgr2=0x55d4ea0e6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:12.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.688+0000 7f530be00640 1 --2- 172.21.15.32:0/1831766890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ea0f2840 0x55d4ea0e6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.688+0000 7f530be00640 1 -- 172.21.15.32:0/1831766890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ea0ebbc0 msgr2=0x55d4ea0e5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:12.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.688+0000 7f530be00640 1 --2- 172.21.15.32:0/1831766890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ea0ebbc0 0x55d4ea0e5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.688+0000 7f530be00640 1 -- 172.21.15.32:0/1831766890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4ea0f2670 con 0x55d4ea0f1f20 2024-04-17T09:56:12.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.688+0000 7f530be00640 1 --2- 172.21.15.32:0/1831766890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0f1f20 0x55d4ea0e5d40 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x55d4e9bf62c0 tx=0x7f52f8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3f25ccee9a2b1e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:12.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.688+0000 7f52fcc00640 1 -- 172.21.15.32:0/1831766890 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f8030070 con 0x55d4ea0f1f20 2024-04-17T09:56:12.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.688+0000 7f52fcc00640 1 -- 172.21.15.32:0/1831766890 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52f801a040 con 0x55d4ea0f1f20 2024-04-17T09:56:12.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.688+0000 7f52fcc00640 1 -- 172.21.15.32:0/1831766890 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f8015890 con 0x55d4ea0f1f20 2024-04-17T09:56:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 1 -- 172.21.15.32:0/1831766890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0f1f20 msgr2=0x55d4ea0e5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 1 --2- 172.21.15.32:0/1831766890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0f1f20 0x55d4ea0e5d40 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x55d4e9bf62c0 tx=0x7f52f8014000 comp rx=0 tx=0).stop 2024-04-17T09:56:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 1 -- 172.21.15.32:0/1831766890 shutdown_connections 2024-04-17T09:56:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 1 --2- 172.21.15.32:0/1831766890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ea0f2840 0x55d4ea0e6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 1 --2- 172.21.15.32:0/1831766890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0f1f20 0x55d4ea0e5d40 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 1 --2- 172.21.15.32:0/1831766890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ea0ebbc0 0x55d4ea0e5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 1 -- 172.21.15.32:0/1831766890 >> 172.21.15.32:0/1831766890 conn(0x55d4ea0ed470 msgr2=0x55d4ea0f3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 1 -- 172.21.15.32:0/1831766890 shutdown_connections 2024-04-17T09:56:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 1 -- 172.21.15.32:0/1831766890 wait complete. 2024-04-17T09:56:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 1 -- start start 2024-04-17T09:56:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 1 Processor -- start 2024-04-17T09:56:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ea0ebbc0 0x55d4ea13ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0f1f20 0x55d4ea13e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ea0f2840 0x55d4ea13ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d4e9f39b00 con 0x55d4ea0f1f20 2024-04-17T09:56:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d4e9f347e0 con 0x55d4ea0ebbc0 2024-04-17T09:56:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55d4e9f426e0 con 0x55d4ea0f2840 2024-04-17T09:56:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 10 client.0 _ll_register_callbacks cb 0x7f5300056360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:56:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f530be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0f1f20 0x55d4ea13e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f530be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0f1f20 0x55d4ea13e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46366/0 (socket says 172.21.15.32:46366) 2024-04-17T09:56:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f530be00640 1 -- 172.21.15.32:0/1799221978 learned_addr learned my addr 172.21.15.32:0/1799221978 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5310e00640 1 --2- 172.21.15.32:0/1799221978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ea0ebbc0 0x55d4ea13ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5311800640 1 --2- 172.21.15.32:0/1799221978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ea0f2840 0x55d4ea13ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f530be00640 1 -- 172.21.15.32:0/1799221978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ea0f2840 msgr2=0x55d4ea13ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:12.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f530be00640 1 --2- 172.21.15.32:0/1799221978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ea0f2840 0x55d4ea13ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f530be00640 1 -- 172.21.15.32:0/1799221978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ea0ebbc0 msgr2=0x55d4ea13ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:12.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f530be00640 1 --2- 172.21.15.32:0/1799221978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ea0ebbc0 0x55d4ea13ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f530be00640 1 -- 172.21.15.32:0/1799221978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4e9fb6d30 con 0x55d4ea0f1f20 2024-04-17T09:56:12.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f530be00640 1 --2- 172.21.15.32:0/1799221978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0f1f20 0x55d4ea13e8d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f52f8008010 tx=0x7f52f80046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f83d0af4fe4e5ae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:12.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f52f2000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:56:12.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f52f2000640 1 -- 172.21.15.32:0/1799221978 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f803b060 con 0x55d4ea0f1f20 2024-04-17T09:56:12.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f52f2000640 1 -- 172.21.15.32:0/1799221978 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f52f800f040 con 0x55d4ea0f1f20 2024-04-17T09:56:12.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 1 -- 172.21.15.32:0/1799221978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52f801a040 con 0x55d4ea0f1f20 2024-04-17T09:56:12.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f52f2000640 1 -- 172.21.15.32:0/1799221978 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f52f8041050 con 0x55d4ea0f1f20 2024-04-17T09:56:12.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 10 client.6384 fetch_fsmap learned FSMap version 32 2024-04-17T09:56:12.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 1 -- 172.21.15.32:0/1799221978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x55d4e9ec77c0 con 0x55d4ea0f1f20 2024-04-17T09:56:12.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f52f2000640 1 -- 172.21.15.32:0/1799221978 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52f801a040 con 0x55d4ea0f1f20 2024-04-17T09:56:12.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f52f2000640 1 -- 172.21.15.32:0/1799221978 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 32) ==== 3495+0+0 (secure 0 0 0) 0x7f52f8044020 con 0x55d4ea0f1f20 2024-04-17T09:56:12.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 10 client.6384 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T09:56:12.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 10 client.6384 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T09:56:12.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 20 client.6384 populate_metadata read hostname 'smithi032' 2024-04-17T09:56:12.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 1 --2- 172.21.15.32:0/1799221978 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x55d4ea1a4a50 0x55d4ea1a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 4 client.6384 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T09:56:12.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.692+0000 7f5316c8d000 1 -- 172.21.15.32:0/1799221978 --> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] -- command(tid 0: {"prefix": "status"}) -- 0x55d4e9f42ef0 con 0x55d4ea1a4a50 2024-04-17T09:56:12.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5310e00640 1 --2- 172.21.15.32:0/1799221978 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x55d4ea1a4a50 0x55d4ea1a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5310e00640 1 --2- 172.21.15.32:0/1799221978 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x55d4ea1a4a50 0x55d4ea1a4e30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0e8c9fe22ab1642 server_cookie=755b3be39c067ae2 in_seq=0 out_seq=0 2024-04-17T09:56:12.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f52f2000640 10 client.6384 ms_handle_connect on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:12.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f52f2000640 1 -- 172.21.15.32:0/1799221978 <== mds.0 v2:172.21.15.32:6835/866826055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55d4e9f42ef0 con 0x55d4ea1a4a50 2024-04-17T09:56:12.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f52f2000640 10 client.6384 handle_command_reply: tid=0 2024-04-17T09:56:12.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5316c8d000 1 client.6384 shutdown 2024-04-17T09:56:12.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5316c8d000 1 -- 172.21.15.32:0/1799221978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0f1f20 msgr2=0x55d4ea13e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:12.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5316c8d000 1 --2- 172.21.15.32:0/1799221978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0f1f20 0x55d4ea13e8d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f52f8008010 tx=0x7f52f80046d0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5316c8d000 1 -- 172.21.15.32:0/1799221978 shutdown_connections 2024-04-17T09:56:12.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5316c8d000 1 --2- 172.21.15.32:0/1799221978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4ea0f2840 0x55d4ea13ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5316c8d000 1 --2- 172.21.15.32:0/1799221978 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x55d4ea1a4a50 0x55d4ea1a4e30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5316c8d000 1 --2- 172.21.15.32:0/1799221978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d4ea0f1f20 0x55d4ea13e8d0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5316c8d000 1 --2- 172.21.15.32:0/1799221978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4ea0ebbc0 0x55d4ea13ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f52f2000640 0 client.6384 ms_handle_reset on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:12.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5316c8d000 1 -- 172.21.15.32:0/1799221978 >> 172.21.15.32:0/1799221978 conn(0x55d4ea0ed470 msgr2=0x55d4ea14abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:12.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5316c8d000 1 -- 172.21.15.32:0/1799221978 shutdown_connections 2024-04-17T09:56:12.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5316c8d000 1 -- 172.21.15.32:0/1799221978 wait complete. 2024-04-17T09:56:12.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5316c8d000 20 client.6384 trim_cache size 0 max 16384 2024-04-17T09:56:12.702 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:56:12.702 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "ff178df8-4c06-4367-a1e9-2e3a66d870d0", 2024-04-17T09:56:12.702 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-17T09:56:12.702 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 6204, 2024-04-17T09:56:12.702 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-17T09:56:12.702 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-17T09:56:12.702 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-17T09:56:12.702 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 10.50565886, 2024-04-17T09:56:12.703 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 32, 2024-04-17T09:56:12.703 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 57, 2024-04-17T09:56:12.703 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 57, 2024-04-17T09:56:12.703 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 15.370391572999999 2024-04-17T09:56:12.703 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:56:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5313800640 1 -- 172.21.15.32:0/428451766 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f52ec0683b0 msgr2=0x7f52ec06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5313800640 1 --2- 172.21.15.32:0/428451766 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f52ec0683b0 0x7f52ec06a870 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f530c0cfba0 tx=0x7f52f8008040 comp rx=0 tx=0).stop 2024-04-17T09:56:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5313800640 1 -- 172.21.15.32:0/428451766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530c0cf370 msgr2=0x7f530c0affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5313800640 1 --2- 172.21.15.32:0/428451766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530c0cf370 0x7f530c0affb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f530000c480 tx=0x7f530000c940 comp rx=0 tx=0).stop 2024-04-17T09:56:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5313800640 1 -- 172.21.15.32:0/428451766 shutdown_connections 2024-04-17T09:56:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5313800640 1 --2- 172.21.15.32:0/428451766 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f52ec0683b0 0x7f52ec06a870 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5313800640 1 --2- 172.21.15.32:0/428451766 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f530c0d0820 0x7f530c158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5313800640 1 --2- 172.21.15.32:0/428451766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f530c13f6e0 0x7f530c139d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5313800640 1 --2- 172.21.15.32:0/428451766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530c0cf370 0x7f530c0affb0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5313800640 1 -- 172.21.15.32:0/428451766 >> 172.21.15.32:0/428451766 conn(0x7f530c06ba30 msgr2=0x7f530c06ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5313800640 1 -- 172.21.15.32:0/428451766 shutdown_connections 2024-04-17T09:56:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.696+0000 7f5313800640 1 -- 172.21.15.32:0/428451766 wait complete. 2024-04-17T09:56:12.717 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T09:56:12.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.856+0000 7fe909e00640 1 Processor -- start 2024-04-17T09:56:12.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.856+0000 7fe909e00640 1 -- start start 2024-04-17T09:56:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.856+0000 7fe909e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9041608f0 0x7fe904160cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.856+0000 7fe909e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe9040b5be0 0x7fe9040cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.856+0000 7fe909e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe904161210 con 0x7fe9040725d0 2024-04-17T09:56:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.856+0000 7fe909e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe9040b5750 con 0x7fe9040b5be0 2024-04-17T09:56:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.856+0000 7fe909e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe9040d0300 con 0x7fe9041608f0 2024-04-17T09:56:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.856+0000 7fe8fa000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe9040b5be0 0x7fe9040cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.856+0000 7fe8fa000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe9040b5be0 0x7fe9040cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46372/0 (socket says 172.21.15.32:46372) 2024-04-17T09:56:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.856+0000 7fe8fa000640 1 -- 172.21.15.32:0/896606408 learned_addr learned my addr 172.21.15.32:0/896606408 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.856+0000 7fe902a00640 1 --2- 172.21.15.32:0/896606408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9041608f0 0x7fe904160cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.856+0000 7fe8fa000640 1 -- 172.21.15.32:0/896606408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9041608f0 msgr2=0x7fe904160cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:12.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.856+0000 7fe8fa000640 1 --2- 172.21.15.32:0/896606408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9041608f0 0x7fe904160cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.856+0000 7fe8fa000640 1 -- 172.21.15.32:0/896606408 >> v1:172.21.15.144:6789/0 conn(0x7fe9040725d0 legacy=0x7fe9040ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:12.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.856+0000 7fe8fa000640 1 -- 172.21.15.32:0/896606408 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9040d05d0 con 0x7fe9040b5be0 2024-04-17T09:56:12.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.856+0000 7fe902a00640 1 --2- 172.21.15.32:0/896606408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9041608f0 0x7fe904160cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:12.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.860+0000 7fe8fa000640 1 --2- 172.21.15.32:0/896606408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9040b5be0 0x7fe9040cfdc0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009870 tx=0x7fe8ec01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba87d7116e77076 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:12.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.860+0000 7fe902000640 1 -- 172.21.15.32:0/896606408 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec0a95f0 con 0x7fe9040b5be0 2024-04-17T09:56:12.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.860+0000 7fe902000640 1 -- 172.21.15.32:0/896606408 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8ec0a9790 con 0x7fe9040b5be0 2024-04-17T09:56:12.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.860+0000 7fe902000640 1 -- 172.21.15.32:0/896606408 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec0a9b60 con 0x7fe9040b5be0 2024-04-17T09:56:12.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.860+0000 7fe909e00640 1 -- 172.21.15.32:0/896606408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9040b5be0 msgr2=0x7fe9040cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:12.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.860+0000 7fe909e00640 1 --2- 172.21.15.32:0/896606408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9040b5be0 0x7fe9040cfdc0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009870 tx=0x7fe8ec01c530 comp rx=0 tx=0).stop 2024-04-17T09:56:12.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.860+0000 7fe909e00640 1 -- 172.21.15.32:0/896606408 shutdown_connections 2024-04-17T09:56:12.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.860+0000 7fe909e00640 1 --2- 172.21.15.32:0/896606408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9040b5be0 0x7fe9040cfdc0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.860+0000 7fe909e00640 1 --2- 172.21.15.32:0/896606408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9041608f0 0x7fe904160cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.860+0000 7fe909e00640 1 -- 172.21.15.32:0/896606408 >> 172.21.15.32:0/896606408 conn(0x7fe9040d0c10 msgr2=0x7fe9040d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:12.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.860+0000 7fe909e00640 1 -- 172.21.15.32:0/896606408 shutdown_connections 2024-04-17T09:56:12.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.860+0000 7fe909e00640 1 -- 172.21.15.32:0/896606408 wait complete. 2024-04-17T09:56:12.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe909e00640 1 Processor -- start 2024-04-17T09:56:12.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe909e00640 1 -- start start 2024-04-17T09:56:12.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe909e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9040725d0 0x7fe9041f7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe909e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9040b5be0 0x7fe9041657f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe909e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9041608f0 0x7fe904165d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe909e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe9040cf630 con 0x7fe9040725d0 2024-04-17T09:56:12.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe909e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe9040b6040 con 0x7fe9041608f0 2024-04-17T09:56:12.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe909e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe904072df0 con 0x7fe9040b5be0 2024-04-17T09:56:12.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe902a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9040725d0 0x7fe9041f7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe902a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9040725d0 0x7fe9041f7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46386/0 (socket says 172.21.15.32:46386) 2024-04-17T09:56:12.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe902a00640 1 -- 172.21.15.32:0/1046496433 learned_addr learned my addr 172.21.15.32:0/1046496433 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:12.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe8fa000640 1 --2- 172.21.15.32:0/1046496433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9040b5be0 0x7fe9041657f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe903400640 1 --2- 172.21.15.32:0/1046496433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9041608f0 0x7fe904165d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe902a00640 1 -- 172.21.15.32:0/1046496433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9040b5be0 msgr2=0x7fe9041657f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:12.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe902a00640 1 --2- 172.21.15.32:0/1046496433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9040b5be0 0x7fe9041657f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe902a00640 1 -- 172.21.15.32:0/1046496433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9041608f0 msgr2=0x7fe904165d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:12.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe902a00640 1 --2- 172.21.15.32:0/1046496433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9041608f0 0x7fe904165d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe902a00640 1 -- 172.21.15.32:0/1046496433 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe904166f80 con 0x7fe9040725d0 2024-04-17T09:56:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe903400640 1 --2- 172.21.15.32:0/1046496433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9041608f0 0x7fe904165d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe902a00640 1 --2- 172.21.15.32:0/1046496433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9040725d0 0x7fe9041f7ec0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400d830 tx=0x7fe8f400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe8fbe00640 1 -- 172.21.15.32:0/1046496433 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4018020 con 0x7fe9040725d0 2024-04-17T09:56:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe8fbe00640 1 -- 172.21.15.32:0/1046496433 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f40096c0 con 0x7fe9040725d0 2024-04-17T09:56:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe909e00640 1 -- 172.21.15.32:0/1046496433 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe90409ab20 con 0x7fe9040725d0 2024-04-17T09:56:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe8fbe00640 1 -- 172.21.15.32:0/1046496433 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4009ab0 con 0x7fe9040725d0 2024-04-17T09:56:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe909e00640 1 -- 172.21.15.32:0/1046496433 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe90409b0e0 con 0x7fe9040725d0 2024-04-17T09:56:12.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.864+0000 7fe909e00640 1 -- 172.21.15.32:0/1046496433 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe904068a90 con 0x7fe9040725d0 2024-04-17T09:56:12.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.868+0000 7fe8fbe00640 1 -- 172.21.15.32:0/1046496433 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe8f401b020 con 0x7fe9040725d0 2024-04-17T09:56:12.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.868+0000 7fe8fbe00640 1 --2- 172.21.15.32:0/1046496433 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe8e0068380 0x7fe8e006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:12.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.868+0000 7fe8fbe00640 1 -- 172.21.15.32:0/1046496433 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fe8f4086ff0 con 0x7fe9040725d0 2024-04-17T09:56:12.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.868+0000 7fe8fa000640 1 --2- 172.21.15.32:0/1046496433 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe8e0068380 0x7fe8e006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:12.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.868+0000 7fe8fa000640 1 --2- 172.21.15.32:0/1046496433 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe8e0068380 0x7fe8e006a840 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0ae000 tx=0x7fe8ec0b3040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:12.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:12.872+0000 7fe8fbe00640 1 -- 172.21.15.32:0/1046496433 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fe8f4022020 con 0x7fe9040725d0 2024-04-17T09:56:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:13.072+0000 7fe909e00640 1 -- 172.21.15.32:0/1046496433 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe90406f610 con 0x7fe9040725d0 2024-04-17T09:56:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:13.072+0000 7fe8fbe00640 1 -- 172.21.15.32:0/1046496433 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) ==== 53+0+73 (secure 0 0 0) 0x7fe8f4057600 con 0x7fe9040725d0 2024-04-17T09:56:13.076 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T09:56:13.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:13.076+0000 7fe909e00640 1 -- 172.21.15.32:0/1046496433 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe8e0068380 msgr2=0x7fe8e006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:13.076+0000 7fe909e00640 1 --2- 172.21.15.32:0/1046496433 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe8e0068380 0x7fe8e006a840 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0ae000 tx=0x7fe8ec0b3040 comp rx=0 tx=0).stop 2024-04-17T09:56:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:13.076+0000 7fe909e00640 1 -- 172.21.15.32:0/1046496433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9040725d0 msgr2=0x7fe9041f7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:13.076+0000 7fe909e00640 1 --2- 172.21.15.32:0/1046496433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9040725d0 0x7fe9041f7ec0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400d830 tx=0x7fe8f400dcf0 comp rx=0 tx=0).stop 2024-04-17T09:56:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:13.076+0000 7fe909e00640 1 -- 172.21.15.32:0/1046496433 shutdown_connections 2024-04-17T09:56:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:13.076+0000 7fe909e00640 1 --2- 172.21.15.32:0/1046496433 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe8e0068380 0x7fe8e006a840 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:13.076+0000 7fe909e00640 1 --2- 172.21.15.32:0/1046496433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9040b5be0 0x7fe9041657f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:13.076+0000 7fe909e00640 1 --2- 172.21.15.32:0/1046496433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9041608f0 0x7fe904165d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:13.076+0000 7fe909e00640 1 --2- 172.21.15.32:0/1046496433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9040725d0 0x7fe9041f7ec0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:13.076+0000 7fe909e00640 1 -- 172.21.15.32:0/1046496433 >> 172.21.15.32:0/1046496433 conn(0x7fe9040d0c10 msgr2=0x7fe9040ae240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:13.076+0000 7fe909e00640 1 -- 172.21.15.32:0/1046496433 shutdown_connections 2024-04-17T09:56:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:13.076+0000 7fe909e00640 1 -- 172.21.15.32:0/1046496433 wait complete. 2024-04-17T09:56:13.097 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T09:56:13.097 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T09:56:13.097 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-17T09:56:13.097 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T09:56:13.097 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T09:56:13.097 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T09:56:13.097 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T09:56:13.097 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T09:56:13.097 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:56:13.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:56:13.098 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-17T09:56:13.106 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T09:56:13.107 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T09:56:13.107 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T09:56:13.107 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:56:13.107 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-17T09:56:13.107 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:56:13.107 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:56:13.107 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-17T09:56:13.107 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-17T09:56:13.107 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T09:56:13.107 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:56:13.107 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-17T09:56:13.107 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-17T09:56:13.107 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42281sec preferred_lft 42281sec 2024-04-17T09:56:13.107 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2024-04-17T09:56:13.108 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:56:13.108 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:56:13.108 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-17T09:56:13.108 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-17T09:56:13.108 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:56:13.108 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:56:13.108 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-17T09:56:13.108 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T09:56:13.108 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:cd:f4:aa:5b brd ff:ff:ff:ff:ff:ff 2024-04-17T09:56:13.108 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T09:56:13.108 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:56:13.108 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T09:56:13.109 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T09:56:13.109 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T09:56:13.109 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:56:13.109 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T09:56:13.109 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:56:13.109 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T09:56:13.109 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:56:13.109 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T09:56:13.109 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:56:13.109 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T09:56:13.109 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:56:13.109 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T09:56:13.109 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:56:13.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:56:13.110 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T09:56:13.173 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-17T09:56:13.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:56:13.175 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-17T09:56:13.228 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-17T09:56:13.229 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T09:56:13.229 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-17T09:56:13.229 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T09:56:13.229 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-17T09:56:13.229 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T09:56:13.229 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:56:13.229 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T09:56:13.229 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-17T09:56:13.229 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-17T09:56:13.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-17T09:56:13.230 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T09:56:13.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:56:13.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:56:13.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:56:13.375 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-17T09:56:13.385 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:56:13.386 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T09:56:13.386 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T09:56:13.386 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:56:13.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:56:13.386 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-17T09:56:13.429 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:56:13.449 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T09:56:13.450 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:56:13.450 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T09:56:13.499 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:56:13.499 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:56:13.499 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T09:56:13.555 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T09:56:13.616 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:56:13.673 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T09:56:13.673 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T09:56:13.673 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T09:56:13.673 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T09:56:13.673 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T09:56:13.673 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T09:56:13.673 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T09:56:13.673 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T09:56:13.673 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T09:56:13.673 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T09:56:13.673 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T09:56:13.674 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T09:56:13.674 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:56:13.674 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T09:56:13.674 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T09:56:13.674 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T09:56:13.674 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T09:56:13.674 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T09:56:13.675 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T09:56:13.675 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T09:56:13.675 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:56:13.675 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T09:56:13.675 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T09:56:13.675 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T09:56:13.675 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T09:56:13.676 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T09:56:13.676 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T09:56:13.676 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T09:56:13.676 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T09:56:13.676 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:56:13.676 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:56:13.676 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:56:13.676 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:56:13.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:56:13.678 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T09:56:13.731 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T09:56:13.731 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T09:56:13.778 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T09:56:13.799 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:56:13.849 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T09:56:13.849 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T09:56:13.849 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T09:56:13.849 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T09:56:13.849 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T09:56:13.849 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T09:56:13.849 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T09:56:13.849 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T09:56:13.849 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T09:56:13.849 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T09:56:13.850 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T09:56:13.850 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T09:56:13.850 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:56:13.850 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T09:56:13.850 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T09:56:13.850 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T09:56:13.850 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T09:56:13.850 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T09:56:13.850 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T09:56:13.850 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T09:56:13.850 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:56:13.850 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T09:56:13.851 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T09:56:13.851 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T09:56:13.851 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T09:56:13.851 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T09:56:13.851 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T09:56:13.851 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T09:56:13.851 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T09:56:13.851 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:56:13.851 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:56:13.851 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:56:13.851 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:56:13.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:56:13.853 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T09:56:13.874 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2024-04-17T09:56:13.872+0000 7f107bc72580 -1 init, newargv = 0x55a75e331de0 newargc=15 2024-04-17T09:56:13.874 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[39948]: starting ceph client 2024-04-17T09:56:13.881 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[39948]: starting fuse 2024-04-17T09:56:13.883 INFO:teuthology.orchestra.run.smithi032.stdout:44 2024-04-17T09:56:13.883 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-17T09:56:13.883 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2024-04-17T09:56:13.883 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2024-04-17T09:56:13.884 DEBUG:teuthology.orchestra.run.smithi032:> import re 2024-04-17T09:56:13.884 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T09:56:13.884 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2024-04-17T09:56:13.884 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:56:13.884 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2024-04-17T09:56:13.884 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T09:56:13.884 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2024-04-17T09:56:13.884 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T09:56:13.884 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:56:13.884 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2024-04-17T09:56:13.884 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2024-04-17T09:56:13.884 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2024-04-17T09:56:13.884 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2024-04-17T09:56:13.884 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:56:13.884 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2024-04-17T09:56:13.884 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T09:56:13.885 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T09:56:13.885 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T09:56:13.885 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2024-04-17T09:56:13.885 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2024-04-17T09:56:13.885 DEBUG:teuthology.orchestra.run.smithi032:> return f 2024-04-17T09:56:13.885 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T09:56:13.885 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:56:13.885 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2024-04-17T09:56:13.885 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T09:56:14.013 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.39948.asok 2024-04-17T09:56:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:56:14.025 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.39948.asok 2024-04-17T09:56:14.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:56:14.026 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.39948.asok status 2024-04-17T09:56:14.204 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:56:14.204 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2024-04-17T09:56:14.204 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-17T09:56:14.204 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T09:56:14.204 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2024-04-17T09:56:14.204 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2024-04-17T09:56:14.204 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T09:56:14.204 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "39948", 2024-04-17T09:56:14.204 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2024-04-17T09:56:14.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:56:14.205 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2024-04-17T09:56:14.205 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2024-04-17T09:56:14.205 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 6396, 2024-04-17T09:56:14.205 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2024-04-17T09:56:14.205 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2024-04-17T09:56:14.205 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2024-04-17T09:56:14.205 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 6396 2024-04-17T09:56:14.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:56:14.205 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T09:56:14.205 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T09:56:14.205 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T09:56:14.205 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1723225914 2024-04-17T09:56:14.205 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:56:14.205 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:56:14.205 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T09:56:14.205 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T09:56:14.206 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T09:56:14.206 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1723225914 2024-04-17T09:56:14.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:56:14.206 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.6396 192.168.0.1:0/1723225914", 2024-04-17T09:56:14.206 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/1723225914", 2024-04-17T09:56:14.206 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2024-04-17T09:56:14.206 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 32, 2024-04-17T09:56:14.206 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 57, 2024-04-17T09:56:14.206 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2024-04-17T09:56:14.206 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2024-04-17T09:56:14.206 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs" 2024-04-17T09:56:14.206 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:56:14.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:56:14.220 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2024-04-17T09:56:14.221 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T09:56:14.230 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2024-04-17T09:56:14.230 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T09:56:14.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:56:14.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T09:56:14.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:56:14.291 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-04-17T09:56:14.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:56:14.351 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-04-17T09:56:14.407 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T09:56:14.407 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T09:56:14.407 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T09:56:14.407 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:56:14.407 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-04-17T09:56:14.407 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T09:56:14.517 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627777 2024-04-17T09:56:14.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:56:14.526 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:56:14.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.704+0000 7fa3d5800640 1 Processor -- start 2024-04-17T09:56:14.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.704+0000 7fa3d5800640 1 -- start start 2024-04-17T09:56:14.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.708+0000 7fa3d5800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3d00b2b80 0x7fa3d00d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:14.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.708+0000 7fa3d5800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3d00b3120 0x7fa3d006d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:14.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.708+0000 7fa3d5800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3d009b650 con 0x7fa3d00b3670 2024-04-17T09:56:14.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.708+0000 7fa3d5800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa3d00b2f60 con 0x7fa3d00b3120 2024-04-17T09:56:14.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.708+0000 7fa3d5800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa3d009b400 con 0x7fa3d00b2b80 2024-04-17T09:56:14.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.708+0000 7fa3c7e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3d00b3120 0x7fa3d006d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:14.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.708+0000 7fa3c7e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3d00b3120 0x7fa3d006d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46398/0 (socket says 172.21.15.32:46398) 2024-04-17T09:56:14.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.708+0000 7fa3c7e00640 1 -- 172.21.15.32:0/895765253 learned_addr learned my addr 172.21.15.32:0/895765253 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:14.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.708+0000 7fa3cf400640 1 --1- 172.21.15.32:0/895765253 >> v1:172.21.15.32:6789/0 conn(0x7fa3d00b3670 0x7fa3d015f5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42840/0 (socket says 172.21.15.32:42840) 2024-04-17T09:56:14.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.708+0000 7fa3cea00640 1 --2- 172.21.15.32:0/895765253 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3d00b2b80 0x7fa3d00d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:14.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.708+0000 7fa3c7e00640 1 -- 172.21.15.32:0/895765253 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3d00b2b80 msgr2=0x7fa3d00d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:14.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.708+0000 7fa3c7e00640 1 --2- 172.21.15.32:0/895765253 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3d00b2b80 0x7fa3d00d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:14.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.708+0000 7fa3c7e00640 1 -- 172.21.15.32:0/895765253 >> v1:172.21.15.32:6789/0 conn(0x7fa3d00b3670 legacy=0x7fa3d015f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:14.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.708+0000 7fa3c7e00640 1 -- 172.21.15.32:0/895765253 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d006cd50 con 0x7fa3d00b3120 2024-04-17T09:56:14.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.712+0000 7fa3c7e00640 1 --2- 172.21.15.32:0/895765253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d00b3120 0x7fa3d006d550 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009870 tx=0x7fa3b801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81828e933eb4bfe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:14.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.712+0000 7fa3ce000640 1 -- 172.21.15.32:0/895765253 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80b0070 con 0x7fa3d00b3120 2024-04-17T09:56:14.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.712+0000 7fa3ce000640 1 -- 172.21.15.32:0/895765253 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b80ab600 con 0x7fa3d00b3120 2024-04-17T09:56:14.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.712+0000 7fa3ce000640 1 -- 172.21.15.32:0/895765253 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80ab9d0 con 0x7fa3d00b3120 2024-04-17T09:56:14.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.712+0000 7fa3d5800640 1 -- 172.21.15.32:0/895765253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d00b3120 msgr2=0x7fa3d006d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:14.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.712+0000 7fa3d5800640 1 --2- 172.21.15.32:0/895765253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d00b3120 0x7fa3d006d550 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009870 tx=0x7fa3b801c4d0 comp rx=0 tx=0).stop 2024-04-17T09:56:14.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.712+0000 7fa3d5800640 1 -- 172.21.15.32:0/895765253 shutdown_connections 2024-04-17T09:56:14.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.712+0000 7fa3d5800640 1 --2- 172.21.15.32:0/895765253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d00b3120 0x7fa3d006d550 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:14.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.712+0000 7fa3d5800640 1 --2- 172.21.15.32:0/895765253 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3d00b2b80 0x7fa3d00d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:14.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.712+0000 7fa3d5800640 1 -- 172.21.15.32:0/895765253 >> 172.21.15.32:0/895765253 conn(0x7fa3d00b4010 msgr2=0x7fa3d00b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:14.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.712+0000 7fa3d5800640 1 -- 172.21.15.32:0/895765253 shutdown_connections 2024-04-17T09:56:14.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.712+0000 7fa3d5800640 1 -- 172.21.15.32:0/895765253 wait complete. 2024-04-17T09:56:14.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.712+0000 7fa3d5800640 1 Processor -- start 2024-04-17T09:56:14.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.712+0000 7fa3d5800640 1 -- start start 2024-04-17T09:56:14.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3d5800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3d00b2b80 0x7fa3d014e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:14.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3d5800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3d00b3120 0x7fa3d0156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:14.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3d5800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d00b3670 0x7fa3d0157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:14.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3d5800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d00b3db0 con 0x7fa3d00b3670 2024-04-17T09:56:14.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3d5800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d00b3c30 con 0x7fa3d00b2b80 2024-04-17T09:56:14.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3d5800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa3d016c2d0 con 0x7fa3d00b3120 2024-04-17T09:56:14.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3cf400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d00b3670 0x7fa3d0157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:14.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3cf400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d00b3670 0x7fa3d0157410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46410/0 (socket says 172.21.15.32:46410) 2024-04-17T09:56:14.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3cf400640 1 -- 172.21.15.32:0/3825336824 learned_addr learned my addr 172.21.15.32:0/3825336824 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:14.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3cea00640 1 --2- 172.21.15.32:0/3825336824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3d00b2b80 0x7fa3d014e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:14.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3c7e00640 1 --2- 172.21.15.32:0/3825336824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3d00b3120 0x7fa3d0156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:14.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3cf400640 1 -- 172.21.15.32:0/3825336824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3d00b3120 msgr2=0x7fa3d0156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:14.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3cf400640 1 --2- 172.21.15.32:0/3825336824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3d00b3120 0x7fa3d0156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:14.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3cf400640 1 -- 172.21.15.32:0/3825336824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3d00b2b80 msgr2=0x7fa3d014e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:14.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3cf400640 1 --2- 172.21.15.32:0/3825336824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3d00b2b80 0x7fa3d014e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:14.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3cf400640 1 -- 172.21.15.32:0/3825336824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d00afce0 con 0x7fa3d00b3670 2024-04-17T09:56:14.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3c7e00640 1 --2- 172.21.15.32:0/3825336824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3d00b3120 0x7fa3d0156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:14.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3cf400640 1 --2- 172.21.15.32:0/3825336824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d00b3670 0x7fa3d0157410 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800c7a0 tx=0x7fa3c800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:14.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3c6a00640 1 -- 172.21.15.32:0/3825336824 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8017070 con 0x7fa3d00b3670 2024-04-17T09:56:14.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3c6a00640 1 -- 172.21.15.32:0/3825336824 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c800a640 con 0x7fa3d00b3670 2024-04-17T09:56:14.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3d5800640 1 -- 172.21.15.32:0/3825336824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d00affc0 con 0x7fa3d00b3670 2024-04-17T09:56:14.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3c6a00640 1 -- 172.21.15.32:0/3825336824 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c800a9f0 con 0x7fa3d00b3670 2024-04-17T09:56:14.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3d5800640 1 -- 172.21.15.32:0/3825336824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d00b0580 con 0x7fa3d00b3670 2024-04-17T09:56:14.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3d5800640 1 -- 172.21.15.32:0/3825336824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3d0068a90 con 0x7fa3d00b3670 2024-04-17T09:56:14.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.716+0000 7fa3c6a00640 1 -- 172.21.15.32:0/3825336824 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa3c801a070 con 0x7fa3d00b3670 2024-04-17T09:56:14.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.720+0000 7fa3c6a00640 1 --2- 172.21.15.32:0/3825336824 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa3ac068380 0x7fa3ac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:14.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.720+0000 7fa3c6a00640 1 -- 172.21.15.32:0/3825336824 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fa3c8086030 con 0x7fa3d00b3670 2024-04-17T09:56:14.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.720+0000 7fa3cea00640 1 --2- 172.21.15.32:0/3825336824 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa3ac068380 0x7fa3ac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:14.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.720+0000 7fa3cea00640 1 --2- 172.21.15.32:0/3825336824 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa3ac068380 0x7fa3ac06a840 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0158100 tx=0x7fa3c0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:14.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.724+0000 7fa3c6a00640 1 -- 172.21.15.32:0/3825336824 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa3c8055f00 con 0x7fa3d00b3670 2024-04-17T09:56:14.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.928+0000 7fa3d5800640 1 -- 172.21.15.32:0/3825336824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3d00b4e40 con 0x7fa3d00b3670 2024-04-17T09:56:14.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.928+0000 7fa3c6a00640 1 -- 172.21.15.32:0/3825336824 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5202 (secure 0 0 0) 0x7fa3c805a200 con 0x7fa3d00b3670 2024-04-17T09:56:14.930 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:14.930 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5495,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/1236891874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1236891874},{"type":"v1","addr":"172.21.15.144:6834","nonce":1236891874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5498,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3538881342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3538881342},{"type":"v1","addr":"172.21.15.144:6835","nonce":3538881342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6201,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3549801633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3549801633},{"type":"v1","addr":"172.21.15.32:6836","nonce":3549801633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:56:01.579180+0000","modified":"2024-04-17T09:56:04.622581+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/866826055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":866826055},{"type":"v1","addr":"172.21.15.32:6837","nonce":866826055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-04-17T09:56:14.931 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 32 2024-04-17T09:56:14.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.932+0000 7fa3d5800640 1 -- 172.21.15.32:0/3825336824 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa3ac068380 msgr2=0x7fa3ac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:14.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.932+0000 7fa3d5800640 1 --2- 172.21.15.32:0/3825336824 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa3ac068380 0x7fa3ac06a840 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0158100 tx=0x7fa3c0008040 comp rx=0 tx=0).stop 2024-04-17T09:56:14.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.932+0000 7fa3d5800640 1 -- 172.21.15.32:0/3825336824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d00b3670 msgr2=0x7fa3d0157410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:14.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.932+0000 7fa3d5800640 1 --2- 172.21.15.32:0/3825336824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d00b3670 0x7fa3d0157410 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800c7a0 tx=0x7fa3c800cc60 comp rx=0 tx=0).stop 2024-04-17T09:56:14.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.932+0000 7fa3d5800640 1 -- 172.21.15.32:0/3825336824 shutdown_connections 2024-04-17T09:56:14.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.932+0000 7fa3d5800640 1 --2- 172.21.15.32:0/3825336824 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa3ac068380 0x7fa3ac06a840 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:14.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.932+0000 7fa3d5800640 1 --2- 172.21.15.32:0/3825336824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3d00b3120 0x7fa3d0156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:14.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.932+0000 7fa3d5800640 1 --2- 172.21.15.32:0/3825336824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d00b3670 0x7fa3d0157410 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:14.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.932+0000 7fa3d5800640 1 --2- 172.21.15.32:0/3825336824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3d00b2b80 0x7fa3d014e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:14.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.932+0000 7fa3d5800640 1 -- 172.21.15.32:0/3825336824 >> 172.21.15.32:0/3825336824 conn(0x7fa3d00b4010 msgr2=0x7fa3d00b5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:14.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.932+0000 7fa3d5800640 1 -- 172.21.15.32:0/3825336824 shutdown_connections 2024-04-17T09:56:14.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:14.932+0000 7fa3d5800640 1 -- 172.21.15.32:0/3825336824 wait complete. 2024-04-17T09:56:14.952 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-17T09:56:15.126 INFO:teuthology.orchestra.run.smithi032.stdout:{"message":"","return_code":0} 2024-04-17T09:56:15.126 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-17T09:56:15.127 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-17T09:56:15.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.172+0000 7fe6fb469080 1 Processor -- start 2024-04-17T09:56:15.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.172+0000 7fe6fb469080 1 -- start start 2024-04-17T09:56:15.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.172+0000 7fe6fb469080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563e6dbe3100 0x563e6dd4f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.172+0000 7fe6fb469080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e6dc2d230 0x563e6dd4fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.172+0000 7fe6fb469080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6dcba270 0x563e6dc280c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.172+0000 7fe6fb469080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563e6db89f90 con 0x563e6dc2d230 2024-04-17T09:56:15.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.172+0000 7fe6fb469080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563e6db8dfd0 con 0x563e6dcba270 2024-04-17T09:56:15.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.172+0000 7fe6fb469080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x563e6dc305f0 con 0x563e6dbe3100 2024-04-17T09:56:15.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.172+0000 7fe6f7c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e6dc2d230 0x563e6dd4fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.172+0000 7fe6f8600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563e6dbe3100 0x563e6dd4f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.172+0000 7fe6f8600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563e6dbe3100 0x563e6dd4f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:55384/0 (socket says 172.21.15.32:55384) 2024-04-17T09:56:15.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.172+0000 7fe6f8600640 1 -- 172.21.15.32:0/4270751721 learned_addr learned my addr 172.21.15.32:0/4270751721 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:15.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.172+0000 7fe6f9000640 1 --2- 172.21.15.32:0/4270751721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6dcba270 0x563e6dc280c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.172+0000 7fe6f7c00640 1 -- 172.21.15.32:0/4270751721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563e6dbe3100 msgr2=0x563e6dd4f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:15.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.172+0000 7fe6f7c00640 1 --2- 172.21.15.32:0/4270751721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563e6dbe3100 0x563e6dd4f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.172+0000 7fe6f7c00640 1 -- 172.21.15.32:0/4270751721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6dcba270 msgr2=0x563e6dc280c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:15.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.172+0000 7fe6f7c00640 1 --2- 172.21.15.32:0/4270751721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6dcba270 0x563e6dc280c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.172+0000 7fe6f7c00640 1 -- 172.21.15.32:0/4270751721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e6dc26a90 con 0x563e6dc2d230 2024-04-17T09:56:15.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.172+0000 7fe6f7c00640 1 --2- 172.21.15.32:0/4270751721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e6dc2d230 0x563e6dd4fca0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4000c10 tx=0x7fe6e400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd588025852ea1e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:15.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.172+0000 7fe6f6800640 1 -- 172.21.15.32:0/4270751721 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4012020 con 0x563e6dc2d230 2024-04-17T09:56:15.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.172+0000 7fe6f6800640 1 -- 172.21.15.32:0/4270751721 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6e400d6b0 con 0x563e6dc2d230 2024-04-17T09:56:15.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.172+0000 7fe6f6800640 1 -- 172.21.15.32:0/4270751721 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400da80 con 0x563e6dc2d230 2024-04-17T09:56:15.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6fb469080 1 -- 172.21.15.32:0/4270751721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e6dc2d230 msgr2=0x563e6dd4fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6fb469080 1 --2- 172.21.15.32:0/4270751721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e6dc2d230 0x563e6dd4fca0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4000c10 tx=0x7fe6e400b860 comp rx=0 tx=0).stop 2024-04-17T09:56:15.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6fb469080 1 -- 172.21.15.32:0/4270751721 shutdown_connections 2024-04-17T09:56:15.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6fb469080 1 --2- 172.21.15.32:0/4270751721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6dcba270 0x563e6dc280c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6fb469080 1 --2- 172.21.15.32:0/4270751721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e6dc2d230 0x563e6dd4fca0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6fb469080 1 --2- 172.21.15.32:0/4270751721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563e6dbe3100 0x563e6dd4f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6fb469080 1 -- 172.21.15.32:0/4270751721 >> 172.21.15.32:0/4270751721 conn(0x563e6dbe3620 msgr2=0x563e6dbe1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:15.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6fb469080 1 -- 172.21.15.32:0/4270751721 shutdown_connections 2024-04-17T09:56:15.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6fb469080 1 -- 172.21.15.32:0/4270751721 wait complete. 2024-04-17T09:56:15.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6fb469080 1 Processor -- start 2024-04-17T09:56:15.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6fb469080 1 -- start start 2024-04-17T09:56:15.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6fb469080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6dbe3100 0x563e6dcb2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6fb469080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563e6dcba270 0x563e6dcb77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6fb469080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e6dcb2dd0 0x563e6dcb7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6fb469080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563e6dc26260 con 0x563e6dcb2dd0 2024-04-17T09:56:15.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6fb469080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563e6dc25fc0 con 0x563e6dbe3100 2024-04-17T09:56:15.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6fb469080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x563e6dcb8cf0 con 0x563e6dcba270 2024-04-17T09:56:15.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6f9000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e6dcb2dd0 0x563e6dcb7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6f9000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e6dcb2dd0 0x563e6dcb7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46432/0 (socket says 172.21.15.32:46432) 2024-04-17T09:56:15.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6f9000640 1 -- 172.21.15.32:0/1872520696 learned_addr learned my addr 172.21.15.32:0/1872520696 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:15.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6f7c00640 1 --2- 172.21.15.32:0/1872520696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563e6dcba270 0x563e6dcb77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6f8600640 1 --2- 172.21.15.32:0/1872520696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6dbe3100 0x563e6dcb2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6f9000640 1 -- 172.21.15.32:0/1872520696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563e6dcba270 msgr2=0x563e6dcb77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6f9000640 1 --2- 172.21.15.32:0/1872520696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563e6dcba270 0x563e6dcb77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6f9000640 1 -- 172.21.15.32:0/1872520696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6dbe3100 msgr2=0x563e6dcb2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6f9000640 1 --2- 172.21.15.32:0/1872520696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e6dbe3100 0x563e6dcb2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6f9000640 1 -- 172.21.15.32:0/1872520696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e6dc2bdd0 con 0x563e6dcb2dd0 2024-04-17T09:56:15.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6f9000640 1 --2- 172.21.15.32:0/1872520696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e6dcb2dd0 0x563e6dcb7cf0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000d3c0 tx=0x7fe6f000d880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:15.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6f5400640 1 -- 172.21.15.32:0/1872520696 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f00125f0 con 0x563e6dcb2dd0 2024-04-17T09:56:15.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6f5400640 1 -- 172.21.15.32:0/1872520696 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6f0018070 con 0x563e6dcb2dd0 2024-04-17T09:56:15.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6fb469080 1 -- 172.21.15.32:0/1872520696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563e6dc2c070 con 0x563e6dcb2dd0 2024-04-17T09:56:15.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6f5400640 1 -- 172.21.15.32:0/1872520696 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f00129d0 con 0x563e6dcb2dd0 2024-04-17T09:56:15.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.176+0000 7fe6fb469080 1 -- 172.21.15.32:0/1872520696 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e6dc2c5b0 con 0x563e6dcb2dd0 2024-04-17T09:56:15.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.180+0000 7fe6f5400640 1 -- 172.21.15.32:0/1872520696 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe6f0012b70 con 0x563e6dcb2dd0 2024-04-17T09:56:15.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.180+0000 7fe6f5400640 1 --2- 172.21.15.32:0/1872520696 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe6d8066dc0 0x7fe6d8069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.180+0000 7fe6f5400640 1 -- 172.21.15.32:0/1872520696 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fe6f0085760 con 0x563e6dcb2dd0 2024-04-17T09:56:15.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.180+0000 7fe6f8600640 1 --2- 172.21.15.32:0/1872520696 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe6d8066dc0 0x7fe6d8069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.180+0000 7fe6fb469080 1 --2- 172.21.15.32:0/1872520696 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x563e6dcb3510 0x563e6dcb38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.180+0000 7fe6fb469080 1 -- 172.21.15.32:0/1872520696 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57) -- 0x563e6dcb3e30 con 0x563e6dcb3510 2024-04-17T09:56:15.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.180+0000 7fe6f7c00640 1 --2- 172.21.15.32:0/1872520696 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x563e6dcb3510 0x563e6dcb38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.180+0000 7fe6f8600640 1 --2- 172.21.15.32:0/1872520696 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe6d8066dc0 0x7fe6d8069280 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8018e20 tx=0x7fe6e80a5000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:15.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.180+0000 7fe6f7c00640 1 --2- 172.21.15.32:0/1872520696 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x563e6dcb3510 0x563e6dcb38f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:15.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.184+0000 7fe6f7c00640 1 -- 172.21.15.32:0/1872520696 <== osd.5 v2:172.21.15.144:6800/2379578648 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) ==== 164+0+94 (crc 0 0 0) 0x7fe6e4008030 con 0x563e6dcb3510 2024-04-17T09:56:15.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.184+0000 7fe6fb469080 1 -- 172.21.15.32:0/1872520696 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x563e6dcb3510 msgr2=0x563e6dcb38f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.184+0000 7fe6fb469080 1 --2- 172.21.15.32:0/1872520696 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x563e6dcb3510 0x563e6dcb38f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.184+0000 7fe6fb469080 1 -- 172.21.15.32:0/1872520696 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe6d8066dc0 msgr2=0x7fe6d8069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.184+0000 7fe6fb469080 1 --2- 172.21.15.32:0/1872520696 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe6d8066dc0 0x7fe6d8069280 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8018e20 tx=0x7fe6e80a5000 comp rx=0 tx=0).stop 2024-04-17T09:56:15.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.184+0000 7fe6fb469080 1 -- 172.21.15.32:0/1872520696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e6dcb2dd0 msgr2=0x563e6dcb7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.184+0000 7fe6fb469080 1 --2- 172.21.15.32:0/1872520696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e6dcb2dd0 0x563e6dcb7cf0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000d3c0 tx=0x7fe6f000d880 comp rx=0 tx=0).stop 2024-04-17T09:56:15.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.184+0000 7fe6f9000640 1 -- 172.21.15.32:0/1872520696 reap_dead start 2024-04-17T09:56:15.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.184+0000 7fe6fb469080 1 -- 172.21.15.32:0/1872520696 shutdown_connections 2024-04-17T09:56:15.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.184+0000 7fe6fb469080 1 -- 172.21.15.32:0/1872520696 >> 172.21.15.32:0/1872520696 conn(0x563e6dbe3620 msgr2=0x563e6dbe1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:15.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.184+0000 7fe6fb469080 1 -- 172.21.15.32:0/1872520696 shutdown_connections 2024-04-17T09:56:15.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.184+0000 7fe6fb469080 1 -- 172.21.15.32:0/1872520696 wait complete. 2024-04-17T09:56:15.188 INFO:teuthology.orchestra.run.smithi032.stdout:Xfiledir_test 2024-04-17T09:56:15.189 DEBUG:teuthology.orchestra.run.smithi032:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-17T09:56:15.236 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:56:15.236 INFO:teuthology.orchestra.run.smithi032.stdout: "ino": 1099511627777, 2024-04-17T09:56:15.236 INFO:teuthology.orchestra.run.smithi032.stdout: "ancestors": [ 2024-04-17T09:56:15.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:56:15.236 INFO:teuthology.orchestra.run.smithi032.stdout: "dirino": 1099511627776, 2024-04-17T09:56:15.236 INFO:teuthology.orchestra.run.smithi032.stdout: "dname": "file", 2024-04-17T09:56:15.236 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 2 2024-04-17T09:56:15.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:56:15.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:56:15.236 INFO:teuthology.orchestra.run.smithi032.stdout: "dirino": 1, 2024-04-17T09:56:15.236 INFO:teuthology.orchestra.run.smithi032.stdout: "dname": "dir_test", 2024-04-17T09:56:15.237 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 4 2024-04-17T09:56:15.237 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:56:15.237 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-17T09:56:15.237 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 9, 2024-04-17T09:56:15.237 INFO:teuthology.orchestra.run.smithi032.stdout: "old_pools": [] 2024-04-17T09:56:15.237 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:56:15.237 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:15.240 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-04-17T09:56:15.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.328+0000 7faaddd0e080 1 Processor -- start 2024-04-17T09:56:15.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.328+0000 7faaddd0e080 1 -- start start 2024-04-17T09:56:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faaddd0e080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9c6647a30 0x55d9c66cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faaddd0e080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d9c66d6680 0x55d9c662e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faaddd0e080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9c66d6a60 0x55d9c662ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faaddd0e080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d9c6600c60 con 0x55d9c66d6a60 2024-04-17T09:56:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faaddd0e080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d9c65a4f90 con 0x55d9c6647a30 2024-04-17T09:56:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faaddd0e080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55d9c6600de0 con 0x55d9c66d6680 2024-04-17T09:56:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faadba00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9c66d6a60 0x55d9c662ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faadba00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9c66d6a60 0x55d9c662ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46446/0 (socket says 172.21.15.32:46446) 2024-04-17T09:56:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faadba00640 1 -- 172.21.15.32:0/3512150533 learned_addr learned my addr 172.21.15.32:0/3512150533 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faadb000640 1 --2- 172.21.15.32:0/3512150533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9c6647a30 0x55d9c66cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faada600640 1 --2- 172.21.15.32:0/3512150533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d9c66d6680 0x55d9c662e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faadba00640 1 -- 172.21.15.32:0/3512150533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d9c66d6680 msgr2=0x55d9c662e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faadba00640 1 --2- 172.21.15.32:0/3512150533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d9c66d6680 0x55d9c662e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faadba00640 1 -- 172.21.15.32:0/3512150533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9c6647a30 msgr2=0x55d9c66cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faadba00640 1 --2- 172.21.15.32:0/3512150533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9c6647a30 0x55d9c66cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faadba00640 1 -- 172.21.15.32:0/3512150533 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9c6529950 con 0x55d9c66d6a60 2024-04-17T09:56:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faadba00640 1 --2- 172.21.15.32:0/3512150533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9c66d6a60 0x55d9c662ea60 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7faad4009710 tx=0x7faad4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b84f7d45f7a9c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faad9200640 1 -- 172.21.15.32:0/3512150533 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4014070 con 0x55d9c66d6a60 2024-04-17T09:56:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faad9200640 1 -- 172.21.15.32:0/3512150533 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faad400bb00 con 0x55d9c66d6a60 2024-04-17T09:56:15.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faad9200640 1 -- 172.21.15.32:0/3512150533 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4009e20 con 0x55d9c66d6a60 2024-04-17T09:56:15.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faaddd0e080 1 -- 172.21.15.32:0/3512150533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9c66d6a60 msgr2=0x55d9c662ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faaddd0e080 1 --2- 172.21.15.32:0/3512150533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9c66d6a60 0x55d9c662ea60 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7faad4009710 tx=0x7faad4009bd0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faaddd0e080 1 -- 172.21.15.32:0/3512150533 shutdown_connections 2024-04-17T09:56:15.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faaddd0e080 1 --2- 172.21.15.32:0/3512150533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d9c66d6680 0x55d9c662e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faaddd0e080 1 --2- 172.21.15.32:0/3512150533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9c66d6a60 0x55d9c662ea60 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faaddd0e080 1 --2- 172.21.15.32:0/3512150533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9c6647a30 0x55d9c66cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faaddd0e080 1 -- 172.21.15.32:0/3512150533 >> 172.21.15.32:0/3512150533 conn(0x55d9c662aa00 msgr2=0x55d9c662ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:15.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faaddd0e080 1 -- 172.21.15.32:0/3512150533 shutdown_connections 2024-04-17T09:56:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faaddd0e080 1 -- 172.21.15.32:0/3512150533 wait complete. 2024-04-17T09:56:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faaddd0e080 1 Processor -- start 2024-04-17T09:56:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.332+0000 7faaddd0e080 1 -- start start 2024-04-17T09:56:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faaddd0e080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9c6647a30 0x55d9c66ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faaddd0e080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9c66d6680 0x55d9c66d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faaddd0e080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d9c66d6a60 0x55d9c66cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faaddd0e080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d9c676a980 con 0x55d9c6647a30 2024-04-17T09:56:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faaddd0e080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d9c662fca0 con 0x55d9c66d6680 2024-04-17T09:56:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faaddd0e080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55d9c676ac20 con 0x55d9c66d6a60 2024-04-17T09:56:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faadb000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9c6647a30 0x55d9c66ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faadb000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9c6647a30 0x55d9c66ca560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46456/0 (socket says 172.21.15.32:46456) 2024-04-17T09:56:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faadb000640 1 -- 172.21.15.32:0/1660572098 learned_addr learned my addr 172.21.15.32:0/1660572098 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faada600640 1 --2- 172.21.15.32:0/1660572098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9c66d6680 0x55d9c66d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faadba00640 1 --2- 172.21.15.32:0/1660572098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d9c66d6a60 0x55d9c66cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faadb000640 1 -- 172.21.15.32:0/1660572098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d9c66d6a60 msgr2=0x55d9c66cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faadb000640 1 --2- 172.21.15.32:0/1660572098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d9c66d6a60 0x55d9c66cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faadb000640 1 -- 172.21.15.32:0/1660572098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9c66d6680 msgr2=0x55d9c66d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faadb000640 1 --2- 172.21.15.32:0/1660572098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9c66d6680 0x55d9c66d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faadb000640 1 -- 172.21.15.32:0/1660572098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9c66cb160 con 0x55d9c6647a30 2024-04-17T09:56:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faadba00640 1 --2- 172.21.15.32:0/1660572098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d9c66d6a60 0x55d9c66cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faadb000640 1 --2- 172.21.15.32:0/1660572098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9c6647a30 0x55d9c66ca560 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7faacc096cb0 tx=0x7faacc0180b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faad3400640 1 -- 172.21.15.32:0/1660572098 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc0bb070 con 0x55d9c6647a30 2024-04-17T09:56:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faad3400640 1 -- 172.21.15.32:0/1660572098 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faacc0b53f0 con 0x55d9c6647a30 2024-04-17T09:56:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faad3400640 1 -- 172.21.15.32:0/1660572098 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc0af040 con 0x55d9c6647a30 2024-04-17T09:56:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faaddd0e080 1 -- 172.21.15.32:0/1660572098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d9c66cb400 con 0x55d9c6647a30 2024-04-17T09:56:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faaddd0e080 1 -- 172.21.15.32:0/1660572098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9c66c9c20 con 0x55d9c6647a30 2024-04-17T09:56:15.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.336+0000 7faad3400640 1 -- 172.21.15.32:0/1660572098 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7faacc0be070 con 0x55d9c6647a30 2024-04-17T09:56:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.340+0000 7faad3400640 1 --2- 172.21.15.32:0/1660572098 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faabc066dc0 0x7faabc069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.340+0000 7faad3400640 1 -- 172.21.15.32:0/1660572098 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7faacc1203f0 con 0x55d9c6647a30 2024-04-17T09:56:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.340+0000 7faada600640 1 --2- 172.21.15.32:0/1660572098 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faabc066dc0 0x7faabc069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.340+0000 7faaddd0e080 1 --2- 172.21.15.32:0/1660572098 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55d9c66d3de0 0x55d9c66cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.340+0000 7faaddd0e080 1 -- 172.21.15.32:0/1660572098 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57) -- 0x55d9c66d03e0 con 0x55d9c66d3de0 2024-04-17T09:56:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.340+0000 7faadba00640 1 --2- 172.21.15.32:0/1660572098 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55d9c66d3de0 0x55d9c66cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.340+0000 7faada600640 1 --2- 172.21.15.32:0/1660572098 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faabc066dc0 0x7faabc069280 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7faac800c600 tx=0x7faac80057f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.340+0000 7faadba00640 1 --2- 172.21.15.32:0/1660572098 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55d9c66d3de0 0x55d9c66cd2f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.340+0000 7faadba00640 1 -- 172.21.15.32:0/1660572098 <== osd.5 v2:172.21.15.144:6800/2379578648 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v57'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7faad4016030 con 0x55d9c66d3de0 2024-04-17T09:56:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.340+0000 7faaddd0e080 1 -- 172.21.15.32:0/1660572098 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55d9c66d3de0 msgr2=0x55d9c66cd2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.340+0000 7faaddd0e080 1 --2- 172.21.15.32:0/1660572098 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55d9c66d3de0 0x55d9c66cd2f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.340+0000 7faaddd0e080 1 -- 172.21.15.32:0/1660572098 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faabc066dc0 msgr2=0x7faabc069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.340+0000 7faaddd0e080 1 --2- 172.21.15.32:0/1660572098 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faabc066dc0 0x7faabc069280 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7faac800c600 tx=0x7faac80057f0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.340+0000 7faaddd0e080 1 -- 172.21.15.32:0/1660572098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9c6647a30 msgr2=0x55d9c66ca560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.340+0000 7faaddd0e080 1 --2- 172.21.15.32:0/1660572098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d9c6647a30 0x55d9c66ca560 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7faacc096cb0 tx=0x7faacc0180b0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.340+0000 7faadba00640 1 -- 172.21.15.32:0/1660572098 reap_dead start 2024-04-17T09:56:15.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.340+0000 7faaddd0e080 1 -- 172.21.15.32:0/1660572098 shutdown_connections 2024-04-17T09:56:15.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.340+0000 7faaddd0e080 1 -- 172.21.15.32:0/1660572098 >> 172.21.15.32:0/1660572098 conn(0x55d9c662aa00 msgr2=0x55d9c65ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:15.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.340+0000 7faaddd0e080 1 -- 172.21.15.32:0/1660572098 shutdown_connections 2024-04-17T09:56:15.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.344+0000 7faaddd0e080 1 -- 172.21.15.32:0/1660572098 wait complete. 2024-04-17T09:56:15.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:56:15.348 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-04-17T09:56:15.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.532+0000 7f8555800640 1 Processor -- start 2024-04-17T09:56:15.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.532+0000 7f8555800640 1 -- start start 2024-04-17T09:56:15.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.536+0000 7f8555800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85501460a0 0x7f85500ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.536+0000 7f8555800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f85500724c0 0x7f855015e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.536+0000 7f8555800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f855006eeb0 con 0x7f855006e6f0 2024-04-17T09:56:15.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.536+0000 7f8555800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f855015de00 con 0x7f85500724c0 2024-04-17T09:56:15.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.536+0000 7f8555800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f855006f190 con 0x7f85501460a0 2024-04-17T09:56:15.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.536+0000 7f854f400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f85500724c0 0x7f855015e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.536+0000 7f854ea00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85501460a0 0x7f85500ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.536+0000 7f8546000640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f855006e6f0 0x7f8550071db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:50136/0 (socket says 172.21.15.32:50136) 2024-04-17T09:56:15.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.536+0000 7f8546000640 1 -- 172.21.15.32:0/1235830972 learned_addr learned my addr 172.21.15.32:0/1235830972 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:15.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.536+0000 7f854f400640 1 -- 172.21.15.32:0/1235830972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85501460a0 msgr2=0x7f85500ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:15.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.536+0000 7f854f400640 1 --2- 172.21.15.32:0/1235830972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85501460a0 0x7f85500ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.536+0000 7f854f400640 1 -- 172.21.15.32:0/1235830972 >> v1:172.21.15.144:6789/0 conn(0x7f855006e6f0 legacy=0x7f8550071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.536+0000 7f854f400640 1 -- 172.21.15.32:0/1235830972 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855006f470 con 0x7f85500724c0 2024-04-17T09:56:15.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.536+0000 7f854ea00640 1 --2- 172.21.15.32:0/1235830972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85501460a0 0x7f85500ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:56:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.540+0000 7f854f400640 1 --2- 172.21.15.32:0/1235830972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85500724c0 0x7f855015e550 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8548009d90 tx=0x7f854801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7546fa6f38ad3346 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.540+0000 7f854e000640 1 -- 172.21.15.32:0/1235830972 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85480b2070 con 0x7f85500724c0 2024-04-17T09:56:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.540+0000 7f854e000640 1 -- 172.21.15.32:0/1235830972 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85480a5830 con 0x7f85500724c0 2024-04-17T09:56:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.540+0000 7f854e000640 1 -- 172.21.15.32:0/1235830972 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85480a5c00 con 0x7f85500724c0 2024-04-17T09:56:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.540+0000 7f8555800640 1 -- 172.21.15.32:0/1235830972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85500724c0 msgr2=0x7f855015e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.540+0000 7f8555800640 1 --2- 172.21.15.32:0/1235830972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85500724c0 0x7f855015e550 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8548009d90 tx=0x7f854801c970 comp rx=0 tx=0).stop 2024-04-17T09:56:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.540+0000 7f8555800640 1 -- 172.21.15.32:0/1235830972 shutdown_connections 2024-04-17T09:56:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.540+0000 7f8555800640 1 --2- 172.21.15.32:0/1235830972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85500724c0 0x7f855015e550 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.540+0000 7f8555800640 1 --2- 172.21.15.32:0/1235830972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85501460a0 0x7f85500ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.540+0000 7f8555800640 1 -- 172.21.15.32:0/1235830972 >> 172.21.15.32:0/1235830972 conn(0x7f85500b2190 msgr2=0x7f85500b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.540+0000 7f8555800640 1 -- 172.21.15.32:0/1235830972 shutdown_connections 2024-04-17T09:56:15.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.540+0000 7f8555800640 1 -- 172.21.15.32:0/1235830972 wait complete. 2024-04-17T09:56:15.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.540+0000 7f8555800640 1 Processor -- start 2024-04-17T09:56:15.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.540+0000 7f8555800640 1 -- start start 2024-04-17T09:56:15.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.544+0000 7f8555800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f855006e6f0 0x7f85501587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.544+0000 7f8555800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85500724c0 0x7f8550143220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.544+0000 7f8555800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85501460a0 0x7f8550156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.544+0000 7f8555800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f855015f6b0 con 0x7f855006e6f0 2024-04-17T09:56:15.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.544+0000 7f8555800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f855015f530 con 0x7f85500724c0 2024-04-17T09:56:15.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.544+0000 7f8555800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f855015f830 con 0x7f85501460a0 2024-04-17T09:56:15.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.544+0000 7f854ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f855006e6f0 0x7f85501587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.544+0000 7f854ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f855006e6f0 0x7f85501587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46460/0 (socket says 172.21.15.32:46460) 2024-04-17T09:56:15.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.544+0000 7f854ea00640 1 -- 172.21.15.32:0/2144551220 learned_addr learned my addr 172.21.15.32:0/2144551220 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:15.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.544+0000 7f8546000640 1 --2- 172.21.15.32:0/2144551220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85500724c0 0x7f8550143220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.544+0000 7f854f400640 1 --2- 172.21.15.32:0/2144551220 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85501460a0 0x7f8550156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.544+0000 7f854ea00640 1 -- 172.21.15.32:0/2144551220 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85501460a0 msgr2=0x7f8550156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.544+0000 7f854ea00640 1 --2- 172.21.15.32:0/2144551220 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85501460a0 0x7f8550156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.544+0000 7f854ea00640 1 -- 172.21.15.32:0/2144551220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85500724c0 msgr2=0x7f8550143220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.544+0000 7f854ea00640 1 --2- 172.21.15.32:0/2144551220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85500724c0 0x7f8550143220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.544+0000 7f854ea00640 1 -- 172.21.15.32:0/2144551220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8550157730 con 0x7f855006e6f0 2024-04-17T09:56:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.544+0000 7f854ea00640 1 --2- 172.21.15.32:0/2144551220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f855006e6f0 0x7f85501587d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f854000e9b0 tx=0x7f854000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.544+0000 7f8547400640 1 -- 172.21.15.32:0/2144551220 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540017070 con 0x7f855006e6f0 2024-04-17T09:56:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.544+0000 7f8547400640 1 -- 172.21.15.32:0/2144551220 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f854000ca00 con 0x7f855006e6f0 2024-04-17T09:56:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.544+0000 7f8555800640 1 -- 172.21.15.32:0/2144551220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8550157a10 con 0x7f855006e6f0 2024-04-17T09:56:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.544+0000 7f8547400640 1 -- 172.21.15.32:0/2144551220 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854000cdd0 con 0x7f855006e6f0 2024-04-17T09:56:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.544+0000 7f8555800640 1 -- 172.21.15.32:0/2144551220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8550157fd0 con 0x7f855006e6f0 2024-04-17T09:56:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.544+0000 7f8547400640 1 -- 172.21.15.32:0/2144551220 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f854001a070 con 0x7f855006e6f0 2024-04-17T09:56:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f8547400640 1 --2- 172.21.15.32:0/2144551220 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f852c068380 0x7f852c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f8547400640 1 -- 172.21.15.32:0/2144551220 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f8540086b60 con 0x7f855006e6f0 2024-04-17T09:56:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f8546000640 1 --2- 172.21.15.32:0/2144551220 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f852c068380 0x7f852c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f8558d08000 1 Processor -- start 2024-04-17T09:56:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f8558d08000 1 -- start start 2024-04-17T09:56:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f8546000640 1 --2- 172.21.15.32:0/2144551220 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f852c068380 0x7f852c06a840 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f85501ee650 tx=0x7f8538007e90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:15.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f8558d08000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56521a8cd4d0 0x56521a8cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f8558d08000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8cddf0 0x56521a880670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f8558d08000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56521a880bb0 0x56521a880f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f8558d08000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56521a6d25b0 con 0x56521a8cddf0 2024-04-17T09:56:15.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f8558d08000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56521a68b780 con 0x56521a880bb0 2024-04-17T09:56:15.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f8558d08000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x56521a6ce570 con 0x56521a8cd4d0 2024-04-17T09:56:15.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f8546000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8cddf0 0x56521a880670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f8546000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8cddf0 0x56521a880670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46472/0 (socket says 172.21.15.32:46472) 2024-04-17T09:56:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f8546000640 1 -- 172.21.15.32:0/3556350984 learned_addr learned my addr 172.21.15.32:0/3556350984 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f854f400640 1 --2- 172.21.15.32:0/3556350984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56521a880bb0 0x56521a880f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f854ea00640 1 --2- 172.21.15.32:0/3556350984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56521a8cd4d0 0x56521a8cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f8546000640 1 -- 172.21.15.32:0/3556350984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56521a8cd4d0 msgr2=0x56521a8cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f8546000640 1 --2- 172.21.15.32:0/3556350984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56521a8cd4d0 0x56521a8cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f8546000640 1 -- 172.21.15.32:0/3556350984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56521a880bb0 msgr2=0x56521a880f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f8546000640 1 --2- 172.21.15.32:0/3556350984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56521a880bb0 0x56521a880f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f8546000640 1 -- 172.21.15.32:0/3556350984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56521a6aaf80 con 0x56521a8cddf0 2024-04-17T09:56:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f854ea00640 1 --2- 172.21.15.32:0/3556350984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56521a8cd4d0 0x56521a8cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f854f400640 1 --2- 172.21.15.32:0/3556350984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56521a880bb0 0x56521a880f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f8546000640 1 --2- 172.21.15.32:0/3556350984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8cddf0 0x56521a880670 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8538000fc0 tx=0x7f8538012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1b0a00bff1d3fc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f853f400640 1 -- 172.21.15.32:0/3556350984 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538026070 con 0x56521a8cddf0 2024-04-17T09:56:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f853f400640 1 -- 172.21.15.32:0/3556350984 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85380014c0 con 0x56521a8cddf0 2024-04-17T09:56:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.548+0000 7f853f400640 1 -- 172.21.15.32:0/3556350984 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538001870 con 0x56521a8cddf0 2024-04-17T09:56:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8558d08000 1 -- 172.21.15.32:0/3556350984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8cddf0 msgr2=0x56521a880670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8558d08000 1 --2- 172.21.15.32:0/3556350984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8cddf0 0x56521a880670 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8538000fc0 tx=0x7f8538012000 comp rx=0 tx=0).stop 2024-04-17T09:56:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8558d08000 1 -- 172.21.15.32:0/3556350984 shutdown_connections 2024-04-17T09:56:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8558d08000 1 --2- 172.21.15.32:0/3556350984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56521a880bb0 0x56521a880f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8558d08000 1 --2- 172.21.15.32:0/3556350984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8cddf0 0x56521a880670 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8558d08000 1 --2- 172.21.15.32:0/3556350984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56521a8cd4d0 0x56521a8cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8558d08000 1 -- 172.21.15.32:0/3556350984 >> 172.21.15.32:0/3556350984 conn(0x56521a8ae270 msgr2=0x56521a8b0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8558d08000 1 -- 172.21.15.32:0/3556350984 shutdown_connections 2024-04-17T09:56:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8558d08000 1 -- 172.21.15.32:0/3556350984 wait complete. 2024-04-17T09:56:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8558d08000 1 -- start start 2024-04-17T09:56:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8558d08000 1 Processor -- start 2024-04-17T09:56:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8558d08000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56521a880bb0 0x56521a8d4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8558d08000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8cd4d0 0x56521a8d4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8558d08000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56521a8cddf0 0x56521a8864b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8558d08000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56521a753400 con 0x56521a8cd4d0 2024-04-17T09:56:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8558d08000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56521a78e2d0 con 0x56521a8cddf0 2024-04-17T09:56:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8558d08000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x56521a697380 con 0x56521a880bb0 2024-04-17T09:56:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8558d08000 10 client.0 _ll_register_callbacks cb 0x56521a86e570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:56:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8546000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8cd4d0 0x56521a8d4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8546000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8cd4d0 0x56521a8d4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46480/0 (socket says 172.21.15.32:46480) 2024-04-17T09:56:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8546000640 1 -- 172.21.15.32:0/1797717608 learned_addr learned my addr 172.21.15.32:0/1797717608 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f854f400640 1 --2- 172.21.15.32:0/1797717608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56521a8cddf0 0x56521a8864b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f854ea00640 1 --2- 172.21.15.32:0/1797717608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56521a880bb0 0x56521a8d4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8546000640 1 -- 172.21.15.32:0/1797717608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56521a880bb0 msgr2=0x56521a8d4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8546000640 1 --2- 172.21.15.32:0/1797717608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56521a880bb0 0x56521a8d4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8546000640 1 -- 172.21.15.32:0/1797717608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56521a8cddf0 msgr2=0x56521a8864b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8546000640 1 --2- 172.21.15.32:0/1797717608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56521a8cddf0 0x56521a8864b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8546000640 1 -- 172.21.15.32:0/1797717608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56521a6a6fd0 con 0x56521a8cd4d0 2024-04-17T09:56:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f854f400640 1 --2- 172.21.15.32:0/1797717608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56521a8cddf0 0x56521a8864b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8546000640 1 --2- 172.21.15.32:0/1797717608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8cd4d0 0x56521a8d4d70 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f85380067d0 tx=0x7f8538000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2028fc80450f2149 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8533e00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:56:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8533e00640 1 -- 172.21.15.32:0/1797717608 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538033060 con 0x56521a8cd4d0 2024-04-17T09:56:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8533e00640 1 -- 172.21.15.32:0/1797717608 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f853802eae0 con 0x56521a8cd4d0 2024-04-17T09:56:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8558d08000 1 -- 172.21.15.32:0/1797717608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85380014c0 con 0x56521a8cd4d0 2024-04-17T09:56:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8533e00640 1 -- 172.21.15.32:0/1797717608 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8538026070 con 0x56521a8cd4d0 2024-04-17T09:56:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8558d08000 10 client.6432 fetch_fsmap learned FSMap version 32 2024-04-17T09:56:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8558d08000 1 -- 172.21.15.32:0/1797717608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x56521a69df70 con 0x56521a8cd4d0 2024-04-17T09:56:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8533e00640 1 -- 172.21.15.32:0/1797717608 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85380014c0 con 0x56521a8cd4d0 2024-04-17T09:56:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.552+0000 7f8533e00640 1 -- 172.21.15.32:0/1797717608 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 32) ==== 3495+0+0 (secure 0 0 0) 0x7f853802c070 con 0x56521a8cd4d0 2024-04-17T09:56:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f8558d08000 10 client.6432 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T09:56:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f8558d08000 10 client.6432 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-17T09:56:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f8558d08000 20 client.6432 populate_metadata read hostname 'smithi032' 2024-04-17T09:56:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f8558d08000 1 --2- 172.21.15.32:0/1797717608 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x56521a887220 0x56521a887630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f8558d08000 4 client.6432 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:56:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f8558d08000 1 -- 172.21.15.32:0/1797717608 --> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56521a66bf70 con 0x56521a887220 2024-04-17T09:56:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f854ea00640 1 --2- 172.21.15.32:0/1797717608 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x56521a887220 0x56521a887630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f854ea00640 1 --2- 172.21.15.32:0/1797717608 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x56521a887220 0x56521a887630 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d5560bf2b315c31 server_cookie=734522a232b6a74f in_seq=0 out_seq=0 2024-04-17T09:56:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f8533e00640 10 client.6432 ms_handle_connect on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:15.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f8533e00640 1 -- 172.21.15.32:0/1797717608 <== mds.0 v2:172.21.15.32:6835/866826055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56521a66bf70 con 0x56521a887220 2024-04-17T09:56:15.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f8533e00640 10 client.6432 handle_command_reply: tid=0 2024-04-17T09:56:15.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f8558d08000 1 client.6432 shutdown 2024-04-17T09:56:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f8558d08000 1 -- 172.21.15.32:0/1797717608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8cd4d0 msgr2=0x56521a8d4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f8558d08000 1 --2- 172.21.15.32:0/1797717608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8cd4d0 0x56521a8d4d70 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f85380067d0 tx=0x7f8538000bd0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f8558d08000 1 -- 172.21.15.32:0/1797717608 shutdown_connections 2024-04-17T09:56:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f8558d08000 1 --2- 172.21.15.32:0/1797717608 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x56521a887220 0x56521a887630 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f8558d08000 1 --2- 172.21.15.32:0/1797717608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56521a8cddf0 0x56521a8864b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f8558d08000 1 --2- 172.21.15.32:0/1797717608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8cd4d0 0x56521a8d4d70 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f8533e00640 0 client.6432 ms_handle_reset on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f8558d08000 1 --2- 172.21.15.32:0/1797717608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56521a880bb0 0x56521a8d4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f8558d08000 1 -- 172.21.15.32:0/1797717608 >> 172.21.15.32:0/1797717608 conn(0x56521a8ae270 msgr2=0x56521a87fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f8558d08000 1 -- 172.21.15.32:0/1797717608 shutdown_connections 2024-04-17T09:56:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f8558d08000 1 -- 172.21.15.32:0/1797717608 wait complete. 2024-04-17T09:56:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.556+0000 7f8558d08000 20 client.6432 trim_cache size 0 max 16384 2024-04-17T09:56:15.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.572+0000 7f8558d08000 1 Processor -- start 2024-04-17T09:56:15.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.572+0000 7f8558d08000 1 -- start start 2024-04-17T09:56:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f8558d08000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8ae270 0x56521a8e4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f8558d08000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56521a880bb0 0x56521a887600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f8558d08000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56521a8cd4d0 0x56521a887b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f8558d08000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f85501fb910 con 0x56521a8ae270 2024-04-17T09:56:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f8558d08000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f85501fac70 con 0x56521a8cd4d0 2024-04-17T09:56:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f8558d08000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f85501fb2c0 con 0x56521a880bb0 2024-04-17T09:56:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f854ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8ae270 0x56521a8e4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f854ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8ae270 0x56521a8e4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46484/0 (socket says 172.21.15.32:46484) 2024-04-17T09:56:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f854ea00640 1 -- 172.21.15.32:0/2355501583 learned_addr learned my addr 172.21.15.32:0/2355501583 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f854ea00640 1 -- 172.21.15.32:0/2355501583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56521a880bb0 msgr2=0x56521a887600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f854ea00640 1 --2- 172.21.15.32:0/2355501583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56521a880bb0 0x56521a887600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f854ea00640 1 -- 172.21.15.32:0/2355501583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56521a8cd4d0 msgr2=0x56521a887b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f854ea00640 1 --2- 172.21.15.32:0/2355501583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56521a8cd4d0 0x56521a887b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f854ea00640 1 -- 172.21.15.32:0/2355501583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56521a68cae0 con 0x56521a8ae270 2024-04-17T09:56:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f854ea00640 1 --2- 172.21.15.32:0/2355501583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8ae270 0x56521a8e4df0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f854000c520 tx=0x7f8540084f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede418a77c909d12 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f853ea00640 1 -- 172.21.15.32:0/2355501583 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540084070 con 0x56521a8ae270 2024-04-17T09:56:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f853ea00640 1 -- 172.21.15.32:0/2355501583 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8540054b70 con 0x56521a8ae270 2024-04-17T09:56:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f853ea00640 1 -- 172.21.15.32:0/2355501583 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540021040 con 0x56521a8ae270 2024-04-17T09:56:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f8558d08000 1 -- 172.21.15.32:0/2355501583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8ae270 msgr2=0x56521a8e4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f8558d08000 1 --2- 172.21.15.32:0/2355501583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8ae270 0x56521a8e4df0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f854000c520 tx=0x7f8540084f60 comp rx=0 tx=0).stop 2024-04-17T09:56:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f8558d08000 1 -- 172.21.15.32:0/2355501583 shutdown_connections 2024-04-17T09:56:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f8558d08000 1 --2- 172.21.15.32:0/2355501583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56521a880bb0 0x56521a887600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f8558d08000 1 --2- 172.21.15.32:0/2355501583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56521a8cd4d0 0x56521a887b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f8558d08000 1 --2- 172.21.15.32:0/2355501583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8ae270 0x56521a8e4df0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f8558d08000 1 -- 172.21.15.32:0/2355501583 >> 172.21.15.32:0/2355501583 conn(0x56521a8b0c00 msgr2=0x56521a8daf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f8558d08000 1 -- 172.21.15.32:0/2355501583 shutdown_connections 2024-04-17T09:56:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f8558d08000 1 -- 172.21.15.32:0/2355501583 wait complete. 2024-04-17T09:56:15.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f8558d08000 1 -- start start 2024-04-17T09:56:15.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.576+0000 7f8558d08000 1 Processor -- start 2024-04-17T09:56:15.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56521a880bb0 0x56521a8864b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8ae270 0x56521a8d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56521a8cd4d0 0x56521a8d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56521a6d18c0 con 0x56521a8ae270 2024-04-17T09:56:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56521a694630 con 0x56521a880bb0 2024-04-17T09:56:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x56521a6d6be0 con 0x56521a8cd4d0 2024-04-17T09:56:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 10 client.0 _ll_register_callbacks cb 0x7f8538024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:56:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8546000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8ae270 0x56521a8d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8546000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8ae270 0x56521a8d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46486/0 (socket says 172.21.15.32:46486) 2024-04-17T09:56:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8546000640 1 -- 172.21.15.32:0/615919436 learned_addr learned my addr 172.21.15.32:0/615919436 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f854f400640 1 --2- 172.21.15.32:0/615919436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56521a8cd4d0 0x56521a8d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f854ea00640 1 --2- 172.21.15.32:0/615919436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56521a880bb0 0x56521a8864b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8546000640 1 -- 172.21.15.32:0/615919436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56521a8cd4d0 msgr2=0x56521a8d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8546000640 1 --2- 172.21.15.32:0/615919436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56521a8cd4d0 0x56521a8d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8546000640 1 -- 172.21.15.32:0/615919436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56521a880bb0 msgr2=0x56521a8864b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8546000640 1 --2- 172.21.15.32:0/615919436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56521a880bb0 0x56521a8864b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8546000640 1 -- 172.21.15.32:0/615919436 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56521a753890 con 0x56521a8ae270 2024-04-17T09:56:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f854ea00640 1 --2- 172.21.15.32:0/615919436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56521a880bb0 0x56521a8864b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8546000640 1 --2- 172.21.15.32:0/615919436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8ae270 0x56521a8d01e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8538031040 tx=0x7f8538000f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c5267803ae364c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8533e00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:56:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8533e00640 1 -- 172.21.15.32:0/615919436 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538006800 con 0x56521a8ae270 2024-04-17T09:56:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8533e00640 1 -- 172.21.15.32:0/615919436 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f853803e070 con 0x56521a8ae270 2024-04-17T09:56:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 1 -- 172.21.15.32:0/615919436 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8540054b70 con 0x56521a8ae270 2024-04-17T09:56:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8533e00640 1 -- 172.21.15.32:0/615919436 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f85380014c0 con 0x56521a8ae270 2024-04-17T09:56:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 10 client.6438 fetch_fsmap learned FSMap version 32 2024-04-17T09:56:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 1 -- 172.21.15.32:0/615919436 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x56521a6776c0 con 0x56521a8ae270 2024-04-17T09:56:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8533e00640 1 -- 172.21.15.32:0/615919436 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8540054b70 con 0x56521a8ae270 2024-04-17T09:56:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8533e00640 1 -- 172.21.15.32:0/615919436 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 32) ==== 3495+0+0 (secure 0 0 0) 0x7f8538039430 con 0x56521a8ae270 2024-04-17T09:56:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 10 client.6438 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T09:56:15.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 10 client.6438 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-17T09:56:15.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 20 client.6438 populate_metadata read hostname 'smithi032' 2024-04-17T09:56:15.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 1 --2- 172.21.15.32:0/615919436 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x56521a8cddf0 0x56521a93c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 4 client.6438 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-04-17T09:56:15.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 1 -- 172.21.15.32:0/615919436 --> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x56521a6dffd0 con 0x56521a8cddf0 2024-04-17T09:56:15.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f854ea00640 1 --2- 172.21.15.32:0/615919436 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x56521a8cddf0 0x56521a93c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f854ea00640 1 --2- 172.21.15.32:0/615919436 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x56521a8cddf0 0x56521a93c610 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de58967a1a9fa379 server_cookie=85c52f2592222619 in_seq=0 out_seq=0 2024-04-17T09:56:15.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8533e00640 10 client.6438 ms_handle_connect on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:15.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8533e00640 1 -- 172.21.15.32:0/615919436 <== mds.0 v2:172.21.15.32:6835/866826055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56521a6dffd0 con 0x56521a8cddf0 2024-04-17T09:56:15.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8533e00640 10 client.6438 handle_command_reply: tid=0 2024-04-17T09:56:15.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 1 client.6438 shutdown 2024-04-17T09:56:15.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 1 -- 172.21.15.32:0/615919436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8ae270 msgr2=0x56521a8d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 1 --2- 172.21.15.32:0/615919436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8ae270 0x56521a8d01e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8538031040 tx=0x7f8538000f30 comp rx=0 tx=0).stop 2024-04-17T09:56:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 1 -- 172.21.15.32:0/615919436 shutdown_connections 2024-04-17T09:56:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 1 --2- 172.21.15.32:0/615919436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56521a8cd4d0 0x56521a8d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 1 --2- 172.21.15.32:0/615919436 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x56521a8cddf0 0x56521a93c610 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 1 --2- 172.21.15.32:0/615919436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56521a8ae270 0x56521a8d01e0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 1 --2- 172.21.15.32:0/615919436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56521a880bb0 0x56521a8864b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8558d08000 1 -- 172.21.15.32:0/615919436 >> 172.21.15.32:0/615919436 conn(0x56521a8b0c00 msgr2=0x56521a8cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.580+0000 7f8533e00640 0 client.6438 ms_handle_reset on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.584+0000 7f8558d08000 1 -- 172.21.15.32:0/615919436 shutdown_connections 2024-04-17T09:56:15.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.584+0000 7f8558d08000 1 -- 172.21.15.32:0/615919436 wait complete. 2024-04-17T09:56:15.587 INFO:tasks.ceph.mds.a.smithi032.stderr:2024-04-17T09:56:15.584+0000 7f7c90000640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6396=0-4194304@1} caps={6396=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6396 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x5570aa0c0b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-17T09:56:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.584+0000 7f8558d08000 20 client.6438 trim_cache size 0 max 16384 2024-04-17T09:56:15.588 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:56:15.588 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-17T09:56:15.588 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "c80f6de2-1e20-4a02-b68e-5541f6c7ca57", 2024-04-17T09:56:15.588 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-17T09:56:15.588 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:56:15.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.584+0000 7f8555800640 1 -- 172.21.15.32:0/2144551220 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f852c068380 msgr2=0x7f852c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.584+0000 7f8555800640 1 --2- 172.21.15.32:0/2144551220 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f852c068380 0x7f852c06a840 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f85501ee650 tx=0x7f8538007e90 comp rx=0 tx=0).stop 2024-04-17T09:56:15.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.584+0000 7f8555800640 1 -- 172.21.15.32:0/2144551220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f855006e6f0 msgr2=0x7f85501587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.584+0000 7f8555800640 1 --2- 172.21.15.32:0/2144551220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f855006e6f0 0x7f85501587d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f854000e9b0 tx=0x7f854000ee70 comp rx=0 tx=0).stop 2024-04-17T09:56:15.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.584+0000 7f8555800640 1 -- 172.21.15.32:0/2144551220 shutdown_connections 2024-04-17T09:56:15.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.584+0000 7f8555800640 1 --2- 172.21.15.32:0/2144551220 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f852c068380 0x7f852c06a840 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.584+0000 7f8555800640 1 --2- 172.21.15.32:0/2144551220 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85501460a0 0x7f8550156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.584+0000 7f8555800640 1 --2- 172.21.15.32:0/2144551220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85500724c0 0x7f8550143220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.584+0000 7f8555800640 1 --2- 172.21.15.32:0/2144551220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f855006e6f0 0x7f85501587d0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.584+0000 7f8555800640 1 -- 172.21.15.32:0/2144551220 >> 172.21.15.32:0/2144551220 conn(0x7f85500b2190 msgr2=0x7f8550144a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:15.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.584+0000 7f8555800640 1 -- 172.21.15.32:0/2144551220 shutdown_connections 2024-04-17T09:56:15.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.584+0000 7f8555800640 1 -- 172.21.15.32:0/2144551220 wait complete. 2024-04-17T09:56:15.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:56:15.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-17T09:56:15.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.744+0000 7f7a00600640 1 Processor -- start 2024-04-17T09:56:15.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.744+0000 7f7a00600640 1 -- start start 2024-04-17T09:56:15.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.744+0000 7f7a00600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f79f815b7d0 0x7f79f806e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.744+0000 7f7a00600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f806f7c0 0x7f79f806ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.744+0000 7f7a00600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79f80cc940 con 0x7f79f806f270 2024-04-17T09:56:15.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.744+0000 7f7a00600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f79f80ccc20 con 0x7f79f815b7d0 2024-04-17T09:56:15.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.744+0000 7f7a00600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f79f806e0b0 con 0x7f79f806f7c0 2024-04-17T09:56:15.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.744+0000 7f79fdc00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f79f815b7d0 0x7f79f806e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.748+0000 7f79fe600640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f79f806f270 0x7f79f80ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42858/0 (socket says 172.21.15.32:42858) 2024-04-17T09:56:15.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.748+0000 7f79fe600640 1 -- 172.21.15.32:0/4160072676 learned_addr learned my addr 172.21.15.32:0/4160072676 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:15.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.748+0000 7f79fd200640 1 --2- 172.21.15.32:0/4160072676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f806f7c0 0x7f79f806ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.748+0000 7f79fdc00640 1 -- 172.21.15.32:0/4160072676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f806f7c0 msgr2=0x7f79f806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:15.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.748+0000 7f79fdc00640 1 --2- 172.21.15.32:0/4160072676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f806f7c0 0x7f79f806ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.748+0000 7f79fdc00640 1 -- 172.21.15.32:0/4160072676 >> v1:172.21.15.32:6789/0 conn(0x7f79f806f270 legacy=0x7f79f80ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.748+0000 7f79fdc00640 1 -- 172.21.15.32:0/4160072676 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f806e390 con 0x7f79f815b7d0 2024-04-17T09:56:15.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.748+0000 7f79fdc00640 1 --2- 172.21.15.32:0/4160072676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f815b7d0 0x7f79f806e8b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0097c0 tx=0x7f79ec01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669eb5b0e0de189 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:15.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.748+0000 7f79f7e00640 1 -- 172.21.15.32:0/4160072676 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0b0070 con 0x7f79f815b7d0 2024-04-17T09:56:15.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.748+0000 7f79f7e00640 1 -- 172.21.15.32:0/4160072676 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79ec0ab600 con 0x7f79f815b7d0 2024-04-17T09:56:15.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.748+0000 7f79f7e00640 1 -- 172.21.15.32:0/4160072676 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0ab9d0 con 0x7f79f815b7d0 2024-04-17T09:56:15.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.748+0000 7f7a00600640 1 -- 172.21.15.32:0/4160072676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f815b7d0 msgr2=0x7f79f806e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f7a00600640 1 --2- 172.21.15.32:0/4160072676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f815b7d0 0x7f79f806e8b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0097c0 tx=0x7f79ec01c410 comp rx=0 tx=0).stop 2024-04-17T09:56:15.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f7a00600640 1 -- 172.21.15.32:0/4160072676 shutdown_connections 2024-04-17T09:56:15.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f7a00600640 1 --2- 172.21.15.32:0/4160072676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f806f7c0 0x7f79f806ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f7a00600640 1 --2- 172.21.15.32:0/4160072676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f815b7d0 0x7f79f806e8b0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f7a00600640 1 -- 172.21.15.32:0/4160072676 >> 172.21.15.32:0/4160072676 conn(0x7f79f80b2120 msgr2=0x7f79f80b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:15.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f7a00600640 1 -- 172.21.15.32:0/4160072676 shutdown_connections 2024-04-17T09:56:15.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f7a00600640 1 -- 172.21.15.32:0/4160072676 wait complete. 2024-04-17T09:56:15.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f7a00600640 1 Processor -- start 2024-04-17T09:56:15.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f7a00600640 1 -- start start 2024-04-17T09:56:15.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f7a00600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f806f270 0x7f79f813fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f7a00600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79f806f7c0 0x7f79f8144a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f7a00600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f815b7d0 0x7f79f80b3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f7a00600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f79f8072670 con 0x7f79f815b7d0 2024-04-17T09:56:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f7a00600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f79f806c380 con 0x7f79f806f7c0 2024-04-17T09:56:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f7a00600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f79f80727f0 con 0x7f79f806f270 2024-04-17T09:56:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f79fe600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f815b7d0 0x7f79f80b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f79fe600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f815b7d0 0x7f79f80b3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46500/0 (socket says 172.21.15.32:46500) 2024-04-17T09:56:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f79fe600640 1 -- 172.21.15.32:0/3225723016 learned_addr learned my addr 172.21.15.32:0/3225723016 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f79fdc00640 1 --2- 172.21.15.32:0/3225723016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f806f270 0x7f79f813fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f79fd200640 1 --2- 172.21.15.32:0/3225723016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79f806f7c0 0x7f79f8144a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f79fe600640 1 -- 172.21.15.32:0/3225723016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f806f270 msgr2=0x7f79f813fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f79fe600640 1 --2- 172.21.15.32:0/3225723016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f806f270 0x7f79f813fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f79fe600640 1 -- 172.21.15.32:0/3225723016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79f806f7c0 msgr2=0x7f79f8144a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f79fe600640 1 --2- 172.21.15.32:0/3225723016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79f806f7c0 0x7f79f8144a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f79fe600640 1 -- 172.21.15.32:0/3225723016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f8140be0 con 0x7f79f815b7d0 2024-04-17T09:56:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f79fd200640 1 --2- 172.21.15.32:0/3225723016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79f806f7c0 0x7f79f8144a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.752+0000 7f79fe600640 1 --2- 172.21.15.32:0/3225723016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f815b7d0 0x7f79f80b3210 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f79f000cbc0 tx=0x7f79f000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.756+0000 7f79f5600640 1 -- 172.21.15.32:0/3225723016 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f0017070 con 0x7f79f815b7d0 2024-04-17T09:56:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.756+0000 7f79f5600640 1 -- 172.21.15.32:0/3225723016 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79f000ad40 con 0x7f79f815b7d0 2024-04-17T09:56:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.756+0000 7f7a00600640 1 -- 172.21.15.32:0/3225723016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f8140ec0 con 0x7f79f815b7d0 2024-04-17T09:56:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.756+0000 7f79f5600640 1 -- 172.21.15.32:0/3225723016 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f0012600 con 0x7f79f815b7d0 2024-04-17T09:56:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.756+0000 7f7a00600640 1 -- 172.21.15.32:0/3225723016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f8141480 con 0x7f79f815b7d0 2024-04-17T09:56:15.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.756+0000 7f79f5600640 1 -- 172.21.15.32:0/3225723016 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f79f001a070 con 0x7f79f815b7d0 2024-04-17T09:56:15.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.756+0000 7f79f5600640 1 --2- 172.21.15.32:0/3225723016 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f79d8068380 0x7f79d806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.756+0000 7f79f5600640 1 -- 172.21.15.32:0/3225723016 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f79f00866a0 con 0x7f79f815b7d0 2024-04-17T09:56:15.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.756+0000 7f79fdc00640 1 --2- 172.21.15.32:0/3225723016 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f79d8068380 0x7f79d806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.756+0000 7f7a03a6c000 1 Processor -- start 2024-04-17T09:56:15.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.756+0000 7f7a03a6c000 1 -- start start 2024-04-17T09:56:15.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.756+0000 7f79fdc00640 1 --2- 172.21.15.32:0/3225723016 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f79d8068380 0x7f79d806a840 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0ad000 tx=0x7f79ec0a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:15.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f7a03a6c000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604b07dd260 0x5604b07dd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f7a03a6c000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b07ddb80 0x5604b0790460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f7a03a6c000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b0790d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f7a03a6c000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5604b05e24f0 con 0x5604b07909a0 2024-04-17T09:56:15.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f7a03a6c000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5604b059b6c0 con 0x5604b07ddb80 2024-04-17T09:56:15.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f7a03a6c000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5604b05de4b0 con 0x5604b07dd260 2024-04-17T09:56:15.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f79fe600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b0790d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f79fe600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b0790d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46510/0 (socket says 172.21.15.32:46510) 2024-04-17T09:56:15.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f79fe600640 1 -- 172.21.15.32:0/2206534057 learned_addr learned my addr 172.21.15.32:0/2206534057 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f79fd200640 1 --2- 172.21.15.32:0/2206534057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b07ddb80 0x5604b0790460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f79fdc00640 1 --2- 172.21.15.32:0/2206534057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604b07dd260 0x5604b07dd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f79fe600640 1 -- 172.21.15.32:0/2206534057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604b07dd260 msgr2=0x5604b07dd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f79fe600640 1 --2- 172.21.15.32:0/2206534057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604b07dd260 0x5604b07dd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f79fe600640 1 -- 172.21.15.32:0/2206534057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b07ddb80 msgr2=0x5604b0790460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f79fe600640 1 --2- 172.21.15.32:0/2206534057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b07ddb80 0x5604b0790460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f79fe600640 1 -- 172.21.15.32:0/2206534057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604b06278e0 con 0x5604b07909a0 2024-04-17T09:56:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f79fd200640 1 --2- 172.21.15.32:0/2206534057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b07ddb80 0x5604b0790460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f79fdc00640 1 --2- 172.21.15.32:0/2206534057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604b07dd260 0x5604b07dd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f79fe600640 1 --2- 172.21.15.32:0/2206534057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b0790d80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f79f005ec00 tx=0x7f79f0056500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6292ae518d9bc56a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f79ea000640 1 -- 172.21.15.32:0/2206534057 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f0054900 con 0x5604b07909a0 2024-04-17T09:56:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f79ea000640 1 -- 172.21.15.32:0/2206534057 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79f0052c70 con 0x5604b07909a0 2024-04-17T09:56:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f79ea000640 1 -- 172.21.15.32:0/2206534057 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f005eec0 con 0x5604b07909a0 2024-04-17T09:56:15.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f7a03a6c000 1 -- 172.21.15.32:0/2206534057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 msgr2=0x5604b0790d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f7a03a6c000 1 --2- 172.21.15.32:0/2206534057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b0790d80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f79f005ec00 tx=0x7f79f0056500 comp rx=0 tx=0).stop 2024-04-17T09:56:15.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f7a03a6c000 1 -- 172.21.15.32:0/2206534057 shutdown_connections 2024-04-17T09:56:15.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f7a03a6c000 1 --2- 172.21.15.32:0/2206534057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b0790d80 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f7a03a6c000 1 --2- 172.21.15.32:0/2206534057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b07ddb80 0x5604b0790460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f7a03a6c000 1 --2- 172.21.15.32:0/2206534057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604b07dd260 0x5604b07dd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f7a03a6c000 1 -- 172.21.15.32:0/2206534057 >> 172.21.15.32:0/2206534057 conn(0x5604b07be000 msgr2=0x5604b07c03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:15.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f7a03a6c000 1 -- 172.21.15.32:0/2206534057 shutdown_connections 2024-04-17T09:56:15.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f7a03a6c000 1 -- 172.21.15.32:0/2206534057 wait complete. 2024-04-17T09:56:15.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f7a03a6c000 1 -- start start 2024-04-17T09:56:15.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.760+0000 7f7a03a6c000 1 Processor -- start 2024-04-17T09:56:15.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b0796060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604b07dd260 0x5604b07965a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b07ddb80 0x5604b0796ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5604b06637b0 con 0x5604b07909a0 2024-04-17T09:56:15.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5604b069dd00 con 0x5604b07ddb80 2024-04-17T09:56:15.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5604b05a72c0 con 0x5604b07dd260 2024-04-17T09:56:15.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 10 client.0 _ll_register_callbacks cb 0x5604b05b7030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:56:15.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f79fdc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b0796060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f79fdc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b0796060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46520/0 (socket says 172.21.15.32:46520) 2024-04-17T09:56:15.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f79fdc00640 1 -- 172.21.15.32:0/1636683765 learned_addr learned my addr 172.21.15.32:0/1636683765 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:15.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f79fd200640 1 --2- 172.21.15.32:0/1636683765 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604b07dd260 0x5604b07965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f79fe600640 1 --2- 172.21.15.32:0/1636683765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b07ddb80 0x5604b0796ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f79fdc00640 1 -- 172.21.15.32:0/1636683765 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604b07dd260 msgr2=0x5604b07965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f79fdc00640 1 --2- 172.21.15.32:0/1636683765 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604b07dd260 0x5604b07965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f79fdc00640 1 -- 172.21.15.32:0/1636683765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b07ddb80 msgr2=0x5604b0796ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f79fdc00640 1 --2- 172.21.15.32:0/1636683765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b07ddb80 0x5604b0796ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f79fdc00640 1 -- 172.21.15.32:0/1636683765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604b05ba490 con 0x5604b07909a0 2024-04-17T09:56:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f79fdc00640 1 --2- 172.21.15.32:0/1636683765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b0796060 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f79ec01a040 tx=0x7f79ec01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb84de1dd22f181 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f79dea00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:56:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f79dea00640 1 -- 172.21.15.32:0/1636683765 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0b0040 con 0x5604b07909a0 2024-04-17T09:56:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 1 -- 172.21.15.32:0/1636683765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79f0052c70 con 0x5604b07909a0 2024-04-17T09:56:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f79dea00640 1 -- 172.21.15.32:0/1636683765 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f79ec0ab6c0 con 0x5604b07909a0 2024-04-17T09:56:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f79dea00640 1 -- 172.21.15.32:0/1636683765 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f79ec0ab8c0 con 0x5604b07909a0 2024-04-17T09:56:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 10 client.6453 fetch_fsmap learned FSMap version 32 2024-04-17T09:56:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 1 -- 172.21.15.32:0/1636683765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x5604b05adeb0 con 0x5604b07909a0 2024-04-17T09:56:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f79dea00640 1 -- 172.21.15.32:0/1636683765 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79f0052c70 con 0x5604b07909a0 2024-04-17T09:56:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f79dea00640 1 -- 172.21.15.32:0/1636683765 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 32) ==== 3495+0+0 (secure 0 0 0) 0x7f79ec0aba60 con 0x5604b07909a0 2024-04-17T09:56:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 10 client.6453 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T09:56:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 10 client.6453 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-17T09:56:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 20 client.6453 populate_metadata read hostname 'smithi032' 2024-04-17T09:56:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 1 --2- 172.21.15.32:0/1636683765 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x5604b07ec500 0x5604b07ee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 4 client.6453 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:56:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 1 -- 172.21.15.32:0/1636683765 --> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604b057bdd0 con 0x5604b07ec500 2024-04-17T09:56:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f79fd200640 1 --2- 172.21.15.32:0/1636683765 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x5604b07ec500 0x5604b07ee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f79fd200640 1 --2- 172.21.15.32:0/1636683765 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x5604b07ec500 0x5604b07ee8f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=176536878e297351 server_cookie=db375da6b2f0f8a0 in_seq=0 out_seq=0 2024-04-17T09:56:15.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f79dea00640 10 client.6453 ms_handle_connect on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:15.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f79dea00640 1 -- 172.21.15.32:0/1636683765 <== mds.0 v2:172.21.15.32:6835/866826055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5604b057bdd0 con 0x5604b07ec500 2024-04-17T09:56:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f79dea00640 10 client.6453 handle_command_reply: tid=0 2024-04-17T09:56:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 1 client.6453 shutdown 2024-04-17T09:56:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 1 -- 172.21.15.32:0/1636683765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 msgr2=0x5604b0796060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 1 --2- 172.21.15.32:0/1636683765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b0796060 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f79ec01a040 tx=0x7f79ec01ac50 comp rx=0 tx=0).stop 2024-04-17T09:56:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 1 -- 172.21.15.32:0/1636683765 shutdown_connections 2024-04-17T09:56:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 1 --2- 172.21.15.32:0/1636683765 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604b07dd260 0x5604b07965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 1 --2- 172.21.15.32:0/1636683765 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x5604b07ec500 0x5604b07ee8f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 1 --2- 172.21.15.32:0/1636683765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b07ddb80 0x5604b0796ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 1 --2- 172.21.15.32:0/1636683765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b0796060 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f79dea00640 0 client.6453 ms_handle_reset on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.764+0000 7f7a03a6c000 1 -- 172.21.15.32:0/1636683765 >> 172.21.15.32:0/1636683765 conn(0x5604b07be000 msgr2=0x5604b07c03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.768+0000 7f7a03a6c000 1 -- 172.21.15.32:0/1636683765 shutdown_connections 2024-04-17T09:56:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.768+0000 7f7a03a6c000 1 -- 172.21.15.32:0/1636683765 wait complete. 2024-04-17T09:56:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.768+0000 7f7a03a6c000 20 client.6453 trim_cache size 0 max 16384 2024-04-17T09:56:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.784+0000 7f7a03a6c000 1 Processor -- start 2024-04-17T09:56:15.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.784+0000 7f7a03a6c000 1 -- start start 2024-04-17T09:56:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.784+0000 7f7a03a6c000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b078f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.784+0000 7f7a03a6c000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604b07dd260 0x5604b078f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.784+0000 7f7a03a6c000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b07ddb80 0x5604b07ee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.784+0000 7f7a03a6c000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f79f8150750 con 0x5604b07909a0 2024-04-17T09:56:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.784+0000 7f7a03a6c000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f79f81432c0 con 0x5604b07ddb80 2024-04-17T09:56:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.784+0000 7f7a03a6c000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f79f81500e0 con 0x5604b07dd260 2024-04-17T09:56:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.784+0000 7f79fdc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b078f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.784+0000 7f79fdc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b078f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46528/0 (socket says 172.21.15.32:46528) 2024-04-17T09:56:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.784+0000 7f79fdc00640 1 -- 172.21.15.32:0/2829071518 learned_addr learned my addr 172.21.15.32:0/2829071518 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.784+0000 7f79fd200640 1 --2- 172.21.15.32:0/2829071518 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604b07dd260 0x5604b078f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.784+0000 7f79fe600640 1 --2- 172.21.15.32:0/2829071518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b07ddb80 0x5604b07ee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.784+0000 7f79fdc00640 1 -- 172.21.15.32:0/2829071518 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604b07dd260 msgr2=0x5604b078f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.784+0000 7f79fdc00640 1 --2- 172.21.15.32:0/2829071518 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604b07dd260 0x5604b078f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.784+0000 7f79fdc00640 1 -- 172.21.15.32:0/2829071518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b07ddb80 msgr2=0x5604b07ee8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.784+0000 7f79fdc00640 1 --2- 172.21.15.32:0/2829071518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b07ddb80 0x5604b07ee8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.784+0000 7f79fdc00640 1 -- 172.21.15.32:0/2829071518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604b059ca20 con 0x5604b07909a0 2024-04-17T09:56:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.784+0000 7f79fe600640 1 --2- 172.21.15.32:0/2829071518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b07ddb80 0x5604b07ee8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.784+0000 7f79fdc00640 1 --2- 172.21.15.32:0/2829071518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b078f450 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f79ec01aac0 tx=0x7f79ec0bf000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cefb6e57fe28523d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.784+0000 7f79e9600640 1 -- 172.21.15.32:0/2829071518 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0b0040 con 0x5604b07909a0 2024-04-17T09:56:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.784+0000 7f79e9600640 1 -- 172.21.15.32:0/2829071518 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79ec0c7070 con 0x5604b07909a0 2024-04-17T09:56:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.784+0000 7f79e9600640 1 -- 172.21.15.32:0/2829071518 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0c25e0 con 0x5604b07909a0 2024-04-17T09:56:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 1 -- 172.21.15.32:0/2829071518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 msgr2=0x5604b078f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 1 --2- 172.21.15.32:0/2829071518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b078f450 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f79ec01aac0 tx=0x7f79ec0bf000 comp rx=0 tx=0).stop 2024-04-17T09:56:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 1 -- 172.21.15.32:0/2829071518 shutdown_connections 2024-04-17T09:56:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 1 --2- 172.21.15.32:0/2829071518 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604b07dd260 0x5604b078f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 1 --2- 172.21.15.32:0/2829071518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b07ddb80 0x5604b07ee8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 1 --2- 172.21.15.32:0/2829071518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b078f450 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 1 -- 172.21.15.32:0/2829071518 >> 172.21.15.32:0/2829071518 conn(0x5604b07be000 msgr2=0x5604b07dc530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 1 -- 172.21.15.32:0/2829071518 shutdown_connections 2024-04-17T09:56:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 1 -- 172.21.15.32:0/2829071518 wait complete. 2024-04-17T09:56:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 1 -- start start 2024-04-17T09:56:15.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 1 Processor -- start 2024-04-17T09:56:15.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b07f1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b07dd260 0x5604b0808480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604b07ddb80 0x5604b08089c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5604b0586050 con 0x5604b07909a0 2024-04-17T09:56:15.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5604b05a4570 con 0x5604b07dd260 2024-04-17T09:56:15.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5604b05e1800 con 0x5604b07ddb80 2024-04-17T09:56:15.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 10 client.0 _ll_register_callbacks cb 0x7f79ec0ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:56:15.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f79fdc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b07f1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f79fdc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b07f1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46544/0 (socket says 172.21.15.32:46544) 2024-04-17T09:56:15.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f79fdc00640 1 -- 172.21.15.32:0/1726400511 learned_addr learned my addr 172.21.15.32:0/1726400511 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:15.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f79fd200640 1 --2- 172.21.15.32:0/1726400511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b07dd260 0x5604b0808480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f79fe600640 1 --2- 172.21.15.32:0/1726400511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604b07ddb80 0x5604b08089c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f79fdc00640 1 -- 172.21.15.32:0/1726400511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604b07ddb80 msgr2=0x5604b08089c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:15.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f79fdc00640 1 --2- 172.21.15.32:0/1726400511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604b07ddb80 0x5604b08089c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f79fdc00640 1 -- 172.21.15.32:0/1726400511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b07dd260 msgr2=0x5604b0808480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:15.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f79fdc00640 1 --2- 172.21.15.32:0/1726400511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b07dd260 0x5604b0808480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f79fdc00640 1 -- 172.21.15.32:0/1726400511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604b0663c40 con 0x5604b07909a0 2024-04-17T09:56:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f79fd200640 1 --2- 172.21.15.32:0/1726400511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b07dd260 0x5604b0808480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f79fdc00640 1 --2- 172.21.15.32:0/1726400511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b07f1460 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f79ec097f00 tx=0x7f79ec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc4e03cc5aa73f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f79dea00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:56:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f79dea00640 1 -- 172.21.15.32:0/1726400511 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0b0040 con 0x5604b07909a0 2024-04-17T09:56:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f79dea00640 1 -- 172.21.15.32:0/1726400511 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f79ec0dcab0 con 0x5604b07909a0 2024-04-17T09:56:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 1 -- 172.21.15.32:0/1726400511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79ec0c7070 con 0x5604b07909a0 2024-04-17T09:56:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f79dea00640 1 -- 172.21.15.32:0/1726400511 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f79ec0dccb0 con 0x5604b07909a0 2024-04-17T09:56:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 10 client.6459 fetch_fsmap learned FSMap version 32 2024-04-17T09:56:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 1 -- 172.21.15.32:0/1726400511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x5604b0587600 con 0x5604b07909a0 2024-04-17T09:56:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f79dea00640 1 -- 172.21.15.32:0/1726400511 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79ec0c7070 con 0x5604b07909a0 2024-04-17T09:56:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f79dea00640 1 -- 172.21.15.32:0/1726400511 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 32) ==== 3495+0+0 (secure 0 0 0) 0x7f79ec0d3890 con 0x5604b07909a0 2024-04-17T09:56:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 10 client.6459 fetch_fsmap finished waiting for FSMap version 32 2024-04-17T09:56:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 10 client.6459 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-17T09:56:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 20 client.6459 populate_metadata read hostname 'smithi032' 2024-04-17T09:56:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 1 --2- 172.21.15.32:0/1726400511 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x5604b084c720 0x5604b084cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 4 client.6459 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T09:56:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.788+0000 7f7a03a6c000 1 -- 172.21.15.32:0/1726400511 --> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5604b05eff10 con 0x5604b084c720 2024-04-17T09:56:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f79fd200640 1 --2- 172.21.15.32:0/1726400511 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x5604b084c720 0x5604b084cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f79fd200640 1 --2- 172.21.15.32:0/1726400511 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x5604b084c720 0x5604b084cb30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62541476f2c83adb server_cookie=60760e394b68f2aa in_seq=0 out_seq=0 2024-04-17T09:56:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f79dea00640 10 client.6459 ms_handle_connect on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f79dea00640 1 -- 172.21.15.32:0/1726400511 <== mds.0 v2:172.21.15.32:6835/866826055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+253 (crc 0 0 0) 0x5604b05eff10 con 0x5604b084c720 2024-04-17T09:56:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f79dea00640 10 client.6459 handle_command_reply: tid=0 2024-04-17T09:56:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a03a6c000 1 client.6459 shutdown 2024-04-17T09:56:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a03a6c000 1 -- 172.21.15.32:0/1726400511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 msgr2=0x5604b07f1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a03a6c000 1 --2- 172.21.15.32:0/1726400511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b07f1460 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f79ec097f00 tx=0x7f79ec008040 comp rx=0 tx=0).stop 2024-04-17T09:56:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a03a6c000 1 -- 172.21.15.32:0/1726400511 shutdown_connections 2024-04-17T09:56:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a03a6c000 1 --2- 172.21.15.32:0/1726400511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604b07ddb80 0x5604b08089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a03a6c000 1 --2- 172.21.15.32:0/1726400511 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x5604b084c720 0x5604b084cb30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a03a6c000 1 --2- 172.21.15.32:0/1726400511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b07dd260 0x5604b0808480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a03a6c000 1 --2- 172.21.15.32:0/1726400511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604b07909a0 0x5604b07f1460 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f79dea00640 0 client.6459 ms_handle_reset on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:15.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a03a6c000 1 -- 172.21.15.32:0/1726400511 >> 172.21.15.32:0/1726400511 conn(0x5604b07be000 msgr2=0x5604b07dbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:15.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a03a6c000 1 -- 172.21.15.32:0/1726400511 shutdown_connections 2024-04-17T09:56:15.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a03a6c000 1 -- 172.21.15.32:0/1726400511 wait complete. 2024-04-17T09:56:15.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a03a6c000 20 client.6459 trim_cache size 0 max 16384 2024-04-17T09:56:15.797 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:56:15.797 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T09:56:15.797 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-17T09:56:15.797 INFO:teuthology.orchestra.run.smithi032.stdout: "c80f6de2-1e20-4a02-b68e-5541f6c7ca57": { 2024-04-17T09:56:15.798 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-04-17T09:56:15.798 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "c80f6de2-1e20-4a02-b68e-5541f6c7ca57", 2024-04-17T09:56:15.798 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive" 2024-04-17T09:56:15.798 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:56:15.798 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:56:15.798 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:56:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a00600640 1 -- 172.21.15.32:0/3225723016 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f79d8068380 msgr2=0x7f79d806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a00600640 1 --2- 172.21.15.32:0/3225723016 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f79d8068380 0x7f79d806a840 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0ad000 tx=0x7f79ec0a6000 comp rx=0 tx=0).stop 2024-04-17T09:56:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a00600640 1 -- 172.21.15.32:0/3225723016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f815b7d0 msgr2=0x7f79f80b3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a00600640 1 --2- 172.21.15.32:0/3225723016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f815b7d0 0x7f79f80b3210 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f79f000cbc0 tx=0x7f79f000a430 comp rx=0 tx=0).stop 2024-04-17T09:56:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a00600640 1 -- 172.21.15.32:0/3225723016 shutdown_connections 2024-04-17T09:56:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a00600640 1 --2- 172.21.15.32:0/3225723016 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f79d8068380 0x7f79d806a840 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a00600640 1 --2- 172.21.15.32:0/3225723016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f815b7d0 0x7f79f80b3210 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a00600640 1 --2- 172.21.15.32:0/3225723016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79f806f7c0 0x7f79f8144a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a00600640 1 --2- 172.21.15.32:0/3225723016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79f806f270 0x7f79f813fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a00600640 1 -- 172.21.15.32:0/3225723016 >> 172.21.15.32:0/3225723016 conn(0x7f79f80b2120 msgr2=0x7f79f80b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a00600640 1 -- 172.21.15.32:0/3225723016 shutdown_connections 2024-04-17T09:56:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:15.792+0000 7f7a00600640 1 -- 172.21.15.32:0/3225723016 wait complete. 2024-04-17T09:56:15.813 INFO:tasks.cephfs.filesystem:scrub status for tag:c80f6de2-1e20-4a02-b68e-5541f6c7ca57 - {'path': '/', 'tag': 'c80f6de2-1e20-4a02-b68e-5541f6c7ca57', 'options': 'recursive'} 2024-04-17T09:56:18.398 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-17T09:56:18.400+0000 7f93cce00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-17T09:56:43.496 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:56:43.500 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:56:45.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:56:45.814 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-17T09:56:45.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.968+0000 7fcd95200640 1 Processor -- start 2024-04-17T09:56:45.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.968+0000 7fcd95200640 1 -- start start 2024-04-17T09:56:45.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.968+0000 7fcd95200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd9006ba30 0x7fcd9006be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:45.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.968+0000 7fcd95200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcd90071e60 0x7fcd900722b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:45.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.968+0000 7fcd95200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd9006c350 con 0x7fcd900cf370 2024-04-17T09:56:45.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.968+0000 7fcd95200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fcd900cfef0 con 0x7fcd90071e60 2024-04-17T09:56:45.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.968+0000 7fcd95200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcd900d0130 con 0x7fcd9006ba30 2024-04-17T09:56:45.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.972+0000 7fcd85400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcd900cf370 0x7fcd900cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37032/0 (socket says 172.21.15.32:37032) 2024-04-17T09:56:45.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.972+0000 7fcd85400640 1 -- 172.21.15.32:0/2834394783 learned_addr learned my addr 172.21.15.32:0/2834394783 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:45.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.972+0000 7fcd8e800640 1 --2- 172.21.15.32:0/2834394783 >> v2:172.21.15.144:3300/0 conn(0x7fcd90071e60 0x7fcd900722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:45.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.972+0000 7fcd8de00640 1 --2- 172.21.15.32:0/2834394783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd9006ba30 0x7fcd9006be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:45.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.972+0000 7fcd8d400640 1 -- 172.21.15.32:0/2834394783 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4147768393 0 0) 0x7fcd9006c350 con 0x7fcd900cf370 2024-04-17T09:56:45.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.972+0000 7fcd8d400640 1 -- 172.21.15.32:0/2834394783 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd74003530 con 0x7fcd900cf370 2024-04-17T09:56:45.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.972+0000 7fcd8e800640 1 -- 172.21.15.32:0/2834394783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd9006ba30 msgr2=0x7fcd9006be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:45.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.972+0000 7fcd8e800640 1 --2- 172.21.15.32:0/2834394783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd9006ba30 0x7fcd9006be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:45.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.972+0000 7fcd8e800640 1 -- 172.21.15.32:0/2834394783 >> v1:172.21.15.32:6789/0 conn(0x7fcd900cf370 legacy=0x7fcd900cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:45.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.972+0000 7fcd8e800640 1 -- 172.21.15.32:0/2834394783 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9015d090 con 0x7fcd90071e60 2024-04-17T09:56:45.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.972+0000 7fcd8e800640 1 --2- 172.21.15.32:0/2834394783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd90071e60 0x7fcd900722b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcd880097c0 tx=0x7fcd8801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a430a61edb1455ec server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:45.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd8d400640 1 -- 172.21.15.32:0/2834394783 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd880a7070 con 0x7fcd90071e60 2024-04-17T09:56:45.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd8d400640 1 -- 172.21.15.32:0/2834394783 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd880b0070 con 0x7fcd90071e60 2024-04-17T09:56:45.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd8d400640 1 -- 172.21.15.32:0/2834394783 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd880ab890 con 0x7fcd90071e60 2024-04-17T09:56:45.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd95200640 1 -- 172.21.15.32:0/2834394783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd90071e60 msgr2=0x7fcd900722b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:45.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd95200640 1 --2- 172.21.15.32:0/2834394783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd90071e60 0x7fcd900722b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcd880097c0 tx=0x7fcd8801c470 comp rx=0 tx=0).stop 2024-04-17T09:56:45.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd95200640 1 -- 172.21.15.32:0/2834394783 shutdown_connections 2024-04-17T09:56:45.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd95200640 1 --2- 172.21.15.32:0/2834394783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd90071e60 0x7fcd900722b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:45.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd95200640 1 --2- 172.21.15.32:0/2834394783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd9006ba30 0x7fcd9006be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:45.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd95200640 1 -- 172.21.15.32:0/2834394783 >> 172.21.15.32:0/2834394783 conn(0x7fcd900b2120 msgr2=0x7fcd900b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:45.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd95200640 1 -- 172.21.15.32:0/2834394783 shutdown_connections 2024-04-17T09:56:45.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd95200640 1 -- 172.21.15.32:0/2834394783 wait complete. 2024-04-17T09:56:45.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd95200640 1 Processor -- start 2024-04-17T09:56:45.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd95200640 1 -- start start 2024-04-17T09:56:45.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd95200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd9006ba30 0x7fcd900b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:45.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd95200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd90071e60 0x7fcd901599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:45.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd95200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd900cf370 0x7fcd900b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:45.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd95200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9015dcf0 con 0x7fcd900cf370 2024-04-17T09:56:45.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd95200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9015db70 con 0x7fcd90071e60 2024-04-17T09:56:45.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd95200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcd90146230 con 0x7fcd9006ba30 2024-04-17T09:56:45.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd8e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd900cf370 0x7fcd900b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:45.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd8e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd900cf370 0x7fcd900b3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34934/0 (socket says 172.21.15.32:34934) 2024-04-17T09:56:45.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd8e800640 1 -- 172.21.15.32:0/2033028120 learned_addr learned my addr 172.21.15.32:0/2033028120 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:45.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd85400640 1 --2- 172.21.15.32:0/2033028120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd90071e60 0x7fcd901599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:45.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.976+0000 7fcd8de00640 1 --2- 172.21.15.32:0/2033028120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd9006ba30 0x7fcd900b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:45.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.980+0000 7fcd8e800640 1 -- 172.21.15.32:0/2033028120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd9006ba30 msgr2=0x7fcd900b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:45.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.980+0000 7fcd8e800640 1 --2- 172.21.15.32:0/2033028120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd9006ba30 0x7fcd900b4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:45.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.980+0000 7fcd8e800640 1 -- 172.21.15.32:0/2033028120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd90071e60 msgr2=0x7fcd901599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:45.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.980+0000 7fcd8e800640 1 --2- 172.21.15.32:0/2033028120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd90071e60 0x7fcd901599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:45.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.980+0000 7fcd8e800640 1 -- 172.21.15.32:0/2033028120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd900b3ee0 con 0x7fcd900cf370 2024-04-17T09:56:45.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.980+0000 7fcd8de00640 1 --2- 172.21.15.32:0/2033028120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd9006ba30 0x7fcd900b4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:45.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.980+0000 7fcd8e800640 1 --2- 172.21.15.32:0/2033028120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd900cf370 0x7fcd900b3490 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcd8801a0a0 tx=0x7fcd8801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:45.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.980+0000 7fcd86a00640 1 -- 172.21.15.32:0/2033028120 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88006e20 con 0x7fcd900cf370 2024-04-17T09:56:45.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.980+0000 7fcd86a00640 1 -- 172.21.15.32:0/2033028120 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd880bf610 con 0x7fcd900cf370 2024-04-17T09:56:45.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.980+0000 7fcd86a00640 1 -- 172.21.15.32:0/2033028120 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd880bfa00 con 0x7fcd900cf370 2024-04-17T09:56:45.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.980+0000 7fcd95200640 1 -- 172.21.15.32:0/2033028120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd900b41c0 con 0x7fcd900cf370 2024-04-17T09:56:45.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.980+0000 7fcd95200640 1 -- 172.21.15.32:0/2033028120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd900b4780 con 0x7fcd900cf370 2024-04-17T09:56:45.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.980+0000 7fcd86a00640 1 -- 172.21.15.32:0/2033028120 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcd880bd020 con 0x7fcd900cf370 2024-04-17T09:56:45.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.980+0000 7fcd86a00640 1 --2- 172.21.15.32:0/2033028120 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcd6c068380 0x7fcd6c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:45.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.980+0000 7fcd86a00640 1 -- 172.21.15.32:0/2033028120 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fcd88120d10 con 0x7fcd900cf370 2024-04-17T09:56:45.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.980+0000 7fcd8de00640 1 --2- 172.21.15.32:0/2033028120 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcd6c068380 0x7fcd6c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:45.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd986d3000 1 Processor -- start 2024-04-17T09:56:45.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd986d3000 1 -- start start 2024-04-17T09:56:45.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd8de00640 1 --2- 172.21.15.32:0/2033028120 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcd6c068380 0x7fcd6c06a840 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcd90141eb0 tx=0x7fcd8000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:45.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd986d3000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d67dd5260 0x564d67dd5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:45.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd986d3000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67dd5b80 0x564d67d88460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:45.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd986d3000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d67d889a0 0x564d67d88d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:45.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd986d3000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564d67bda4f0 con 0x564d67dd5b80 2024-04-17T09:56:45.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd986d3000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564d67b936c0 con 0x564d67dd5260 2024-04-17T09:56:45.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd986d3000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x564d67bd64b0 con 0x564d67d889a0 2024-04-17T09:56:45.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd85400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67dd5b80 0x564d67d88460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:45.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd85400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67dd5b80 0x564d67d88460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34950/0 (socket says 172.21.15.32:34950) 2024-04-17T09:56:45.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd85400640 1 -- 172.21.15.32:0/3105611379 learned_addr learned my addr 172.21.15.32:0/3105611379 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:45.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd85400640 1 -- 172.21.15.32:0/3105611379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d67d889a0 msgr2=0x564d67d88d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:45.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd85400640 1 --2- 172.21.15.32:0/3105611379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d67d889a0 0x564d67d88d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:45.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd85400640 1 -- 172.21.15.32:0/3105611379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d67dd5260 msgr2=0x564d67dd5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:45.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd85400640 1 --2- 172.21.15.32:0/3105611379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d67dd5260 0x564d67dd5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:45.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd85400640 1 -- 172.21.15.32:0/3105611379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d67c1f8e0 con 0x564d67dd5b80 2024-04-17T09:56:45.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd85400640 1 --2- 172.21.15.32:0/3105611379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67dd5b80 0x564d67d88460 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcd78001860 tx=0x7fcd78000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c956a02d871e57e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:45.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd7ea00640 1 -- 172.21.15.32:0/3105611379 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd78005ad0 con 0x564d67dd5b80 2024-04-17T09:56:45.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd7ea00640 1 -- 172.21.15.32:0/3105611379 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd78018070 con 0x564d67dd5b80 2024-04-17T09:56:45.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd7ea00640 1 -- 172.21.15.32:0/3105611379 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7800f870 con 0x564d67dd5b80 2024-04-17T09:56:45.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd986d3000 1 -- 172.21.15.32:0/3105611379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67dd5b80 msgr2=0x564d67d88460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:45.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd986d3000 1 --2- 172.21.15.32:0/3105611379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67dd5b80 0x564d67d88460 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcd78001860 tx=0x7fcd78000bd0 comp rx=0 tx=0).stop 2024-04-17T09:56:45.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd986d3000 1 -- 172.21.15.32:0/3105611379 shutdown_connections 2024-04-17T09:56:45.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd986d3000 1 --2- 172.21.15.32:0/3105611379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d67d889a0 0x564d67d88d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:45.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd986d3000 1 --2- 172.21.15.32:0/3105611379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67dd5b80 0x564d67d88460 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:45.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd986d3000 1 --2- 172.21.15.32:0/3105611379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d67dd5260 0x564d67dd5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:45.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd986d3000 1 -- 172.21.15.32:0/3105611379 >> 172.21.15.32:0/3105611379 conn(0x564d67db6000 msgr2=0x564d67db83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:45.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd986d3000 1 -- 172.21.15.32:0/3105611379 shutdown_connections 2024-04-17T09:56:45.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd986d3000 1 -- 172.21.15.32:0/3105611379 wait complete. 2024-04-17T09:56:45.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd986d3000 1 -- start start 2024-04-17T09:56:45.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.984+0000 7fcd986d3000 1 Processor -- start 2024-04-17T09:56:45.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd986d3000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67d889a0 0x564d67d8ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:45.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd986d3000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d67dd5260 0x564d67d8ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:45.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd986d3000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d67dd5b80 0x564d67d8f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:45.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd986d3000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564d67c5b7b0 con 0x564d67d889a0 2024-04-17T09:56:45.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd986d3000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564d67c95d00 con 0x564d67dd5b80 2024-04-17T09:56:45.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd986d3000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x564d67b9f2c0 con 0x564d67dd5260 2024-04-17T09:56:45.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd986d3000 10 client.0 _ll_register_callbacks cb 0x564d67baf030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:56:45.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd8de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67d889a0 0x564d67d8ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:45.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd8de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67d889a0 0x564d67d8ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34962/0 (socket says 172.21.15.32:34962) 2024-04-17T09:56:45.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd8de00640 1 -- 172.21.15.32:0/2283595334 learned_addr learned my addr 172.21.15.32:0/2283595334 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:45.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd85400640 1 --2- 172.21.15.32:0/2283595334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d67dd5260 0x564d67d8ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:45.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd8e800640 1 --2- 172.21.15.32:0/2283595334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d67dd5b80 0x564d67d8f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:45.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd8de00640 1 -- 172.21.15.32:0/2283595334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d67dd5260 msgr2=0x564d67d8ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd8de00640 1 --2- 172.21.15.32:0/2283595334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d67dd5260 0x564d67d8ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd8de00640 1 -- 172.21.15.32:0/2283595334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d67dd5b80 msgr2=0x564d67d8f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd8de00640 1 --2- 172.21.15.32:0/2283595334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d67dd5b80 0x564d67d8f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd8de00640 1 -- 172.21.15.32:0/2283595334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d67bb2490 con 0x564d67d889a0 2024-04-17T09:56:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd8de00640 1 --2- 172.21.15.32:0/2283595334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67d889a0 0x564d67d8ea50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fcd80008010 tx=0x7fcd80009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2915d04389e16b74 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd73400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:56:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd73400640 1 -- 172.21.15.32:0/2283595334 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd80027060 con 0x564d67d889a0 2024-04-17T09:56:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd73400640 1 -- 172.21.15.32:0/2283595334 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fcd80022400 con 0x564d67d889a0 2024-04-17T09:56:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd986d3000 1 -- 172.21.15.32:0/2283595334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd78018070 con 0x564d67d889a0 2024-04-17T09:56:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd73400640 1 -- 172.21.15.32:0/2283595334 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcd80022600 con 0x564d67d889a0 2024-04-17T09:56:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd986d3000 10 client.6471 fetch_fsmap learned FSMap version 33 2024-04-17T09:56:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd986d3000 1 -- 172.21.15.32:0/2283595334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x564d67ba5eb0 con 0x564d67d889a0 2024-04-17T09:56:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd73400640 1 -- 172.21.15.32:0/2283595334 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd78018070 con 0x564d67d889a0 2024-04-17T09:56:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd73400640 1 -- 172.21.15.32:0/2283595334 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 33) ==== 3495+0+0 (secure 0 0 0) 0x7fcd800227a0 con 0x564d67d889a0 2024-04-17T09:56:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd986d3000 10 client.6471 fetch_fsmap finished waiting for FSMap version 33 2024-04-17T09:56:45.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd986d3000 10 client.6471 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-17T09:56:45.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd986d3000 20 client.6471 populate_metadata read hostname 'smithi032' 2024-04-17T09:56:45.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd986d3000 1 --2- 172.21.15.32:0/2283595334 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x564d67d8fee0 0x564d67de7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:45.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd986d3000 4 client.6471 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:56:45.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd986d3000 1 -- 172.21.15.32:0/2283595334 --> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d67b73dd0 con 0x564d67d8fee0 2024-04-17T09:56:45.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd85400640 1 --2- 172.21.15.32:0/2283595334 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x564d67d8fee0 0x564d67de7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:45.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd85400640 1 --2- 172.21.15.32:0/2283595334 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x564d67d8fee0 0x564d67de7520 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feeb030ef28366f4 server_cookie=5ae771c0361dc6fd in_seq=0 out_seq=0 2024-04-17T09:56:45.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.988+0000 7fcd73400640 10 client.6471 ms_handle_connect on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:45.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.992+0000 7fcd73400640 1 -- 172.21.15.32:0/2283595334 <== mds.0 v2:172.21.15.32:6835/866826055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d67b73dd0 con 0x564d67d8fee0 2024-04-17T09:56:45.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.992+0000 7fcd73400640 10 client.6471 handle_command_reply: tid=0 2024-04-17T09:56:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.992+0000 7fcd986d3000 1 client.6471 shutdown 2024-04-17T09:56:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.992+0000 7fcd986d3000 1 -- 172.21.15.32:0/2283595334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67d889a0 msgr2=0x564d67d8ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.992+0000 7fcd986d3000 1 --2- 172.21.15.32:0/2283595334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67d889a0 0x564d67d8ea50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fcd80008010 tx=0x7fcd80009750 comp rx=0 tx=0).stop 2024-04-17T09:56:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.992+0000 7fcd986d3000 1 -- 172.21.15.32:0/2283595334 shutdown_connections 2024-04-17T09:56:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.992+0000 7fcd986d3000 1 --2- 172.21.15.32:0/2283595334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d67dd5260 0x564d67d8ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.992+0000 7fcd986d3000 1 --2- 172.21.15.32:0/2283595334 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x564d67d8fee0 0x564d67de7520 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.992+0000 7fcd986d3000 1 --2- 172.21.15.32:0/2283595334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d67dd5b80 0x564d67d8f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.992+0000 7fcd986d3000 1 --2- 172.21.15.32:0/2283595334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67d889a0 0x564d67d8ea50 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.992+0000 7fcd986d3000 1 -- 172.21.15.32:0/2283595334 >> 172.21.15.32:0/2283595334 conn(0x564d67db6000 msgr2=0x564d67d87b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.992+0000 7fcd73400640 0 client.6471 ms_handle_reset on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.992+0000 7fcd986d3000 1 -- 172.21.15.32:0/2283595334 shutdown_connections 2024-04-17T09:56:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.992+0000 7fcd986d3000 1 -- 172.21.15.32:0/2283595334 wait complete. 2024-04-17T09:56:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:45.992+0000 7fcd986d3000 20 client.6471 trim_cache size 0 max 16384 2024-04-17T09:56:46.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.008+0000 7fcd986d3000 1 Processor -- start 2024-04-17T09:56:46.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.008+0000 7fcd986d3000 1 -- start start 2024-04-17T09:56:46.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.008+0000 7fcd986d3000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d67d889a0 0x564d67d87bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.008+0000 7fcd986d3000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d67dd5260 0x564d67d88110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.008+0000 7fcd986d3000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67dd5b80 0x564d67de7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.008+0000 7fcd986d3000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcd90203cd0 con 0x564d67dd5b80 2024-04-17T09:56:46.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.008+0000 7fcd986d3000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcd90144d50 con 0x564d67d889a0 2024-04-17T09:56:46.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.008+0000 7fcd986d3000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcd900b1b10 con 0x564d67dd5260 2024-04-17T09:56:46.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.008+0000 7fcd8e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67dd5b80 0x564d67de7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.008+0000 7fcd8e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67dd5b80 0x564d67de7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34978/0 (socket says 172.21.15.32:34978) 2024-04-17T09:56:46.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.008+0000 7fcd8e800640 1 -- 172.21.15.32:0/561155195 learned_addr learned my addr 172.21.15.32:0/561155195 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:46.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.008+0000 7fcd8e800640 1 -- 172.21.15.32:0/561155195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d67dd5260 msgr2=0x564d67d88110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:46.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.008+0000 7fcd8e800640 1 --2- 172.21.15.32:0/561155195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d67dd5260 0x564d67d88110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.008+0000 7fcd8de00640 1 --2- 172.21.15.32:0/561155195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d67d889a0 0x564d67d87bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.008+0000 7fcd8e800640 1 -- 172.21.15.32:0/561155195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d67d889a0 msgr2=0x564d67d87bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:46.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.008+0000 7fcd8e800640 1 --2- 172.21.15.32:0/561155195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d67d889a0 0x564d67d87bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.008+0000 7fcd8e800640 1 -- 172.21.15.32:0/561155195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d67b94a20 con 0x564d67dd5b80 2024-04-17T09:56:46.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd8e800640 1 --2- 172.21.15.32:0/561155195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67dd5b80 0x564d67de7520 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fcd8811f230 tx=0x7fcd880f06f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca35e07c0575348 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:46.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd7e000640 1 -- 172.21.15.32:0/561155195 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd880f97a0 con 0x564d67dd5b80 2024-04-17T09:56:46.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd7e000640 1 -- 172.21.15.32:0/561155195 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd880b0040 con 0x564d67dd5b80 2024-04-17T09:56:46.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd7e000640 1 -- 172.21.15.32:0/561155195 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd880abe10 con 0x564d67dd5b80 2024-04-17T09:56:46.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd986d3000 1 -- 172.21.15.32:0/561155195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67dd5b80 msgr2=0x564d67de7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd986d3000 1 --2- 172.21.15.32:0/561155195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67dd5b80 0x564d67de7520 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fcd8811f230 tx=0x7fcd880f06f0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd986d3000 1 -- 172.21.15.32:0/561155195 shutdown_connections 2024-04-17T09:56:46.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd986d3000 1 --2- 172.21.15.32:0/561155195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d67dd5260 0x564d67d88110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd986d3000 1 --2- 172.21.15.32:0/561155195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67dd5b80 0x564d67de7520 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd986d3000 1 --2- 172.21.15.32:0/561155195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d67d889a0 0x564d67d87bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd986d3000 1 -- 172.21.15.32:0/561155195 >> 172.21.15.32:0/561155195 conn(0x564d67db6000 msgr2=0x564d67dd3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:46.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd986d3000 1 -- 172.21.15.32:0/561155195 shutdown_connections 2024-04-17T09:56:46.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd986d3000 1 -- 172.21.15.32:0/561155195 wait complete. 2024-04-17T09:56:46.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd986d3000 1 -- start start 2024-04-17T09:56:46.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd986d3000 1 Processor -- start 2024-04-17T09:56:46.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd986d3000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d67d889a0 0x564d67dea090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd986d3000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67dd5260 0x564d67e010b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd986d3000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d67dd5b80 0x564d67e015f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd986d3000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564d67b7e050 con 0x564d67dd5260 2024-04-17T09:56:46.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd986d3000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564d67b9c570 con 0x564d67dd5b80 2024-04-17T09:56:46.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd986d3000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x564d67bd9800 con 0x564d67d889a0 2024-04-17T09:56:46.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd986d3000 10 client.0 _ll_register_callbacks cb 0x7fcd80009bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:56:46.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd85400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67dd5260 0x564d67e010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd85400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67dd5260 0x564d67e010b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34988/0 (socket says 172.21.15.32:34988) 2024-04-17T09:56:46.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd85400640 1 -- 172.21.15.32:0/2975896445 learned_addr learned my addr 172.21.15.32:0/2975896445 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:46.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd8e800640 1 --2- 172.21.15.32:0/2975896445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d67dd5b80 0x564d67e015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd8de00640 1 --2- 172.21.15.32:0/2975896445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d67d889a0 0x564d67dea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd85400640 1 -- 172.21.15.32:0/2975896445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d67d889a0 msgr2=0x564d67dea090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:46.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd85400640 1 --2- 172.21.15.32:0/2975896445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d67d889a0 0x564d67dea090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd85400640 1 -- 172.21.15.32:0/2975896445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d67dd5b80 msgr2=0x564d67e015f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:46.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd85400640 1 --2- 172.21.15.32:0/2975896445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d67dd5b80 0x564d67e015f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd85400640 1 -- 172.21.15.32:0/2975896445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d67c5bc40 con 0x564d67dd5260 2024-04-17T09:56:46.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd8de00640 1 --2- 172.21.15.32:0/2975896445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d67d889a0 0x564d67dea090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:46.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd85400640 1 --2- 172.21.15.32:0/2975896445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67dd5260 0x564d67e010b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fcd78005890 tx=0x7fcd78007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad5143acaf4b62 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:46.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.012+0000 7fcd73400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:56:46.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd73400640 1 -- 172.21.15.32:0/2975896445 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd780047f0 con 0x564d67dd5260 2024-04-17T09:56:46.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd73400640 1 -- 172.21.15.32:0/2975896445 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fcd78004990 con 0x564d67dd5260 2024-04-17T09:56:46.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd986d3000 1 -- 172.21.15.32:0/2975896445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd880b0040 con 0x564d67dd5260 2024-04-17T09:56:46.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd73400640 1 -- 172.21.15.32:0/2975896445 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcd78004b90 con 0x564d67dd5260 2024-04-17T09:56:46.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd986d3000 10 client.6477 fetch_fsmap learned FSMap version 33 2024-04-17T09:56:46.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd986d3000 1 -- 172.21.15.32:0/2975896445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x564d67b7f600 con 0x564d67dd5260 2024-04-17T09:56:46.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd73400640 1 -- 172.21.15.32:0/2975896445 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd880b0040 con 0x564d67dd5260 2024-04-17T09:56:46.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd73400640 1 -- 172.21.15.32:0/2975896445 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 33) ==== 3495+0+0 (secure 0 0 0) 0x7fcd780070d0 con 0x564d67dd5260 2024-04-17T09:56:46.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd986d3000 10 client.6477 fetch_fsmap finished waiting for FSMap version 33 2024-04-17T09:56:46.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd986d3000 10 client.6477 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-17T09:56:46.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd986d3000 20 client.6477 populate_metadata read hostname 'smithi032' 2024-04-17T09:56:46.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd986d3000 1 --2- 172.21.15.32:0/2975896445 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x564d67e44900 0x564d67e44ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd986d3000 4 client.6477 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T09:56:46.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd986d3000 1 -- 172.21.15.32:0/2975896445 --> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564d67be7f10 con 0x564d67e44900 2024-04-17T09:56:46.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd8de00640 1 --2- 172.21.15.32:0/2975896445 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x564d67e44900 0x564d67e44ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd8de00640 1 --2- 172.21.15.32:0/2975896445 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x564d67e44900 0x564d67e44ce0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b6e60ed89823d00 server_cookie=5557a2d403a8eb6b in_seq=0 out_seq=0 2024-04-17T09:56:46.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd73400640 10 client.6477 ms_handle_connect on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:46.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd73400640 1 -- 172.21.15.32:0/2975896445 <== mds.0 v2:172.21.15.32:6835/866826055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564d67be7f10 con 0x564d67e44900 2024-04-17T09:56:46.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd73400640 10 client.6477 handle_command_reply: tid=0 2024-04-17T09:56:46.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd986d3000 1 client.6477 shutdown 2024-04-17T09:56:46.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd986d3000 1 -- 172.21.15.32:0/2975896445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67dd5260 msgr2=0x564d67e010b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd986d3000 1 --2- 172.21.15.32:0/2975896445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67dd5260 0x564d67e010b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fcd78005890 tx=0x7fcd78007d20 comp rx=0 tx=0).stop 2024-04-17T09:56:46.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd986d3000 1 -- 172.21.15.32:0/2975896445 shutdown_connections 2024-04-17T09:56:46.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd986d3000 1 --2- 172.21.15.32:0/2975896445 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x564d67e44900 0x564d67e44ce0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd986d3000 1 --2- 172.21.15.32:0/2975896445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d67dd5b80 0x564d67e015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd986d3000 1 --2- 172.21.15.32:0/2975896445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d67dd5260 0x564d67e010b0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd986d3000 1 --2- 172.21.15.32:0/2975896445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d67d889a0 0x564d67dea090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd73400640 0 client.6477 ms_handle_reset on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:46.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd986d3000 1 -- 172.21.15.32:0/2975896445 >> 172.21.15.32:0/2975896445 conn(0x564d67db6000 msgr2=0x564d67dd3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:46.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd986d3000 1 -- 172.21.15.32:0/2975896445 shutdown_connections 2024-04-17T09:56:46.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd986d3000 1 -- 172.21.15.32:0/2975896445 wait complete. 2024-04-17T09:56:46.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd986d3000 20 client.6477 trim_cache size 0 max 16384 2024-04-17T09:56:46.023 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:56:46.023 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-17T09:56:46.023 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-17T09:56:46.023 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:56:46.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd95200640 1 -- 172.21.15.32:0/2033028120 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcd6c068380 msgr2=0x7fcd6c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd95200640 1 --2- 172.21.15.32:0/2033028120 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcd6c068380 0x7fcd6c06a840 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcd90141eb0 tx=0x7fcd8000b040 comp rx=0 tx=0).stop 2024-04-17T09:56:46.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd95200640 1 -- 172.21.15.32:0/2033028120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd900cf370 msgr2=0x7fcd900b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd95200640 1 --2- 172.21.15.32:0/2033028120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd900cf370 0x7fcd900b3490 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcd8801a0a0 tx=0x7fcd8801acb0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd95200640 1 -- 172.21.15.32:0/2033028120 shutdown_connections 2024-04-17T09:56:46.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd95200640 1 --2- 172.21.15.32:0/2033028120 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcd6c068380 0x7fcd6c06a840 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd95200640 1 --2- 172.21.15.32:0/2033028120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd900cf370 0x7fcd900b3490 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd95200640 1 --2- 172.21.15.32:0/2033028120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd90071e60 0x7fcd901599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd95200640 1 --2- 172.21.15.32:0/2033028120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd9006ba30 0x7fcd900b4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd95200640 1 -- 172.21.15.32:0/2033028120 >> 172.21.15.32:0/2033028120 conn(0x7fcd900b2120 msgr2=0x7fcd9013c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:46.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd95200640 1 -- 172.21.15.32:0/2033028120 shutdown_connections 2024-04-17T09:56:46.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.016+0000 7fcd95200640 1 -- 172.21.15.32:0/2033028120 wait complete. 2024-04-17T09:56:46.038 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T09:56:46.038 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:56:46.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.180+0000 7f3c2ee00640 1 Processor -- start 2024-04-17T09:56:46.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.180+0000 7f3c2ee00640 1 -- start start 2024-04-17T09:56:46.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.180+0000 7f3c2ee00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3c28072500 0x7f3c280cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.180+0000 7f3c2ee00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3c2815d8d0 0x7f3c280cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.180+0000 7f3c2ee00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c280cdac0 con 0x7f3c28145f30 2024-04-17T09:56:46.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.180+0000 7f3c2ee00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3c2806f400 con 0x7f3c2815d8d0 2024-04-17T09:56:46.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.180+0000 7f3c2ee00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f3c28146310 con 0x7f3c28072500 2024-04-17T09:56:46.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.180+0000 7f3c27e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3c28145f30 0x7f3c28071df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37038/0 (socket says 172.21.15.32:37038) 2024-04-17T09:56:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.180+0000 7f3c27e00640 1 -- 172.21.15.32:0/3542311881 learned_addr learned my addr 172.21.15.32:0/3542311881 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.180+0000 7f3c2ce00640 1 --2- 172.21.15.32:0/3542311881 >> v2:172.21.15.32:3300/0 conn(0x7f3c2815d8d0 0x7f3c280cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.180+0000 7f3c27400640 1 -- 172.21.15.32:0/3542311881 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3098076668 0 0) 0x7f3c280cdac0 con 0x7f3c28145f30 2024-04-17T09:56:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.180+0000 7f3c1f400640 1 --2- 172.21.15.32:0/3542311881 >> v2:172.21.15.144:3300/0 conn(0x7f3c28072500 0x7f3c280cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.180+0000 7f3c27400640 1 -- 172.21.15.32:0/3542311881 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c0c003530 con 0x7f3c28145f30 2024-04-17T09:56:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.180+0000 7f3c2ce00640 1 -- 172.21.15.32:0/3542311881 >> v2:172.21.15.144:3300/0 conn(0x7f3c28072500 msgr2=0x7f3c280cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.180+0000 7f3c2ce00640 1 --2- 172.21.15.32:0/3542311881 >> v2:172.21.15.144:3300/0 conn(0x7f3c28072500 0x7f3c280cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.180+0000 7f3c2ce00640 1 -- 172.21.15.32:0/3542311881 >> v1:172.21.15.32:6789/0 conn(0x7f3c28145f30 legacy=0x7f3c28071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.180+0000 7f3c2ce00640 1 -- 172.21.15.32:0/3542311881 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2815cdf0 con 0x7f3c2815d8d0 2024-04-17T09:56:46.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.184+0000 7f3c2ce00640 1 --2- 172.21.15.32:0/3542311881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c2815d8d0 0x7f3c280cd3b0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3c280702a0 tx=0x7f3c2001f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a958776bb8b14278 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:46.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.184+0000 7f3c27400640 1 -- 172.21.15.32:0/3542311881 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c200b4070 con 0x7f3c2815d8d0 2024-04-17T09:56:46.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.184+0000 7f3c27400640 1 -- 172.21.15.32:0/3542311881 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c200af600 con 0x7f3c2815d8d0 2024-04-17T09:56:46.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.184+0000 7f3c27400640 1 -- 172.21.15.32:0/3542311881 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c200af9b0 con 0x7f3c2815d8d0 2024-04-17T09:56:46.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.184+0000 7f3c2ee00640 1 -- 172.21.15.32:0/3542311881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c2815d8d0 msgr2=0x7f3c280cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.184+0000 7f3c2ee00640 1 --2- 172.21.15.32:0/3542311881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c2815d8d0 0x7f3c280cd3b0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3c280702a0 tx=0x7f3c2001f110 comp rx=0 tx=0).stop 2024-04-17T09:56:46.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.184+0000 7f3c2ee00640 1 -- 172.21.15.32:0/3542311881 shutdown_connections 2024-04-17T09:56:46.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.184+0000 7f3c2ee00640 1 --2- 172.21.15.32:0/3542311881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c2815d8d0 0x7f3c280cd3b0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.184+0000 7f3c2ee00640 1 --2- 172.21.15.32:0/3542311881 >> v2:172.21.15.144:3300/0 conn(0x7f3c28072500 0x7f3c280cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.184+0000 7f3c2ee00640 1 -- 172.21.15.32:0/3542311881 >> 172.21.15.32:0/3542311881 conn(0x7f3c2806ba30 msgr2=0x7f3c2806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:46.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.184+0000 7f3c2ee00640 1 -- 172.21.15.32:0/3542311881 shutdown_connections 2024-04-17T09:56:46.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.184+0000 7f3c2ee00640 1 -- 172.21.15.32:0/3542311881 wait complete. 2024-04-17T09:56:46.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c2ee00640 1 Processor -- start 2024-04-17T09:56:46.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c2ee00640 1 -- start start 2024-04-17T09:56:46.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c2ee00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c28072500 0x7f3c281f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c2ee00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c28145f30 0x7f3c281f1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c2ee00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c2815d8d0 0x7f3c281f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c2ee00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2815cb90 con 0x7f3c28145f30 2024-04-17T09:56:46.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c2ee00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2815d440 con 0x7f3c2815d8d0 2024-04-17T09:56:46.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c2ee00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3c281601d0 con 0x7f3c28072500 2024-04-17T09:56:46.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c1f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c28145f30 0x7f3c281f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c1f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c28145f30 0x7f3c281f1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34998/0 (socket says 172.21.15.32:34998) 2024-04-17T09:56:46.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c1f400640 1 -- 172.21.15.32:0/1769997618 learned_addr learned my addr 172.21.15.32:0/1769997618 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:46.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c2ce00640 1 --2- 172.21.15.32:0/1769997618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c2815d8d0 0x7f3c281f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c27e00640 1 --2- 172.21.15.32:0/1769997618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c28072500 0x7f3c281f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c1f400640 1 -- 172.21.15.32:0/1769997618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c28072500 msgr2=0x7f3c281f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c1f400640 1 --2- 172.21.15.32:0/1769997618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c28072500 0x7f3c281f4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c1f400640 1 -- 172.21.15.32:0/1769997618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c2815d8d0 msgr2=0x7f3c281f53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c1f400640 1 --2- 172.21.15.32:0/1769997618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c2815d8d0 0x7f3c281f53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c1f400640 1 -- 172.21.15.32:0/1769997618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c281f51c0 con 0x7f3c28145f30 2024-04-17T09:56:46.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c2ce00640 1 --2- 172.21.15.32:0/1769997618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c2815d8d0 0x7f3c281f53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:46.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c27e00640 1 --2- 172.21.15.32:0/1769997618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c28072500 0x7f3c281f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:46.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c1f400640 1 --2- 172.21.15.32:0/1769997618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c28145f30 0x7f3c281f1600 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f3c1000d800 tx=0x7f3c1000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:46.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c24c00640 1 -- 172.21.15.32:0/1769997618 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1000b6c0 con 0x7f3c28145f30 2024-04-17T09:56:46.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c24c00640 1 -- 172.21.15.32:0/1769997618 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c10019070 con 0x7f3c28145f30 2024-04-17T09:56:46.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c24c00640 1 -- 172.21.15.32:0/1769997618 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1000ba60 con 0x7f3c28145f30 2024-04-17T09:56:46.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c2ee00640 1 -- 172.21.15.32:0/1769997618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c280b45a0 con 0x7f3c28145f30 2024-04-17T09:56:46.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c2ee00640 1 -- 172.21.15.32:0/1769997618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c280ca220 con 0x7f3c28145f30 2024-04-17T09:56:46.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.188+0000 7f3c2ee00640 1 -- 172.21.15.32:0/1769997618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c28068a90 con 0x7f3c28145f30 2024-04-17T09:56:46.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.192+0000 7f3c24c00640 1 -- 172.21.15.32:0/1769997618 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3c1000bc00 con 0x7f3c28145f30 2024-04-17T09:56:46.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.192+0000 7f3c24c00640 1 --2- 172.21.15.32:0/1769997618 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c04068380 0x7f3c0406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.192+0000 7f3c24c00640 1 -- 172.21.15.32:0/1769997618 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f3c10086f50 con 0x7f3c28145f30 2024-04-17T09:56:46.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.192+0000 7f3c27e00640 1 --2- 172.21.15.32:0/1769997618 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c04068380 0x7f3c0406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.192+0000 7f3c27e00640 1 --2- 172.21.15.32:0/1769997618 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c04068380 0x7f3c0406a840 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3c18005890 tx=0x7f3c18009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:46.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.196+0000 7f3c24c00640 1 -- 172.21.15.32:0/1769997618 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3c281f51c0 con 0x7f3c28145f30 2024-04-17T09:56:46.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.396+0000 7f3c2ee00640 1 -- 172.21.15.32:0/1769997618 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c280c9cc0 con 0x7f3c28145f30 2024-04-17T09:56:46.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.396+0000 7f3c24c00640 1 -- 172.21.15.32:0/1769997618 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5202 (secure 0 0 0) 0x7f3c280c9cc0 con 0x7f3c28145f30 2024-04-17T09:56:46.399 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:46.399 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5495,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/1236891874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1236891874},{"type":"v1","addr":"172.21.15.144:6834","nonce":1236891874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5498,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3538881342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3538881342},{"type":"v1","addr":"172.21.15.144:6835","nonce":3538881342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6201,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3549801633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3549801633},{"type":"v1","addr":"172.21.15.32:6836","nonce":3549801633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:56:01.579180+0000","modified":"2024-04-17T09:56:18.240809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":7,"addr":"172.21.15.32:6837/866826055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":866826055},{"type":"v1","addr":"172.21.15.32:6837","nonce":866826055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-04-17T09:56:46.399 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 33 2024-04-17T09:56:46.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.400+0000 7f3c2ee00640 1 -- 172.21.15.32:0/1769997618 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c04068380 msgr2=0x7f3c0406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.400+0000 7f3c2ee00640 1 --2- 172.21.15.32:0/1769997618 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c04068380 0x7f3c0406a840 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3c18005890 tx=0x7f3c18009040 comp rx=0 tx=0).stop 2024-04-17T09:56:46.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.400+0000 7f3c2ee00640 1 -- 172.21.15.32:0/1769997618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c28145f30 msgr2=0x7f3c281f1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.400+0000 7f3c2ee00640 1 --2- 172.21.15.32:0/1769997618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c28145f30 0x7f3c281f1600 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f3c1000d800 tx=0x7f3c1000dcc0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.400+0000 7f3c2ee00640 1 -- 172.21.15.32:0/1769997618 shutdown_connections 2024-04-17T09:56:46.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.400+0000 7f3c2ee00640 1 --2- 172.21.15.32:0/1769997618 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c04068380 0x7f3c0406a840 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.400+0000 7f3c2ee00640 1 --2- 172.21.15.32:0/1769997618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c2815d8d0 0x7f3c281f53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.400+0000 7f3c2ee00640 1 --2- 172.21.15.32:0/1769997618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c28145f30 0x7f3c281f1600 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.400+0000 7f3c2ee00640 1 --2- 172.21.15.32:0/1769997618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c28072500 0x7f3c281f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.400+0000 7f3c2ee00640 1 -- 172.21.15.32:0/1769997618 >> 172.21.15.32:0/1769997618 conn(0x7f3c2806ba30 msgr2=0x7f3c2815dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:46.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.400+0000 7f3c2ee00640 1 -- 172.21.15.32:0/1769997618 shutdown_connections 2024-04-17T09:56:46.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.400+0000 7f3c2ee00640 1 -- 172.21.15.32:0/1769997618 wait complete. 2024-04-17T09:56:46.421 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-17T09:56:46.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.564+0000 7ff29f200640 1 Processor -- start 2024-04-17T09:56:46.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.564+0000 7ff29f200640 1 -- start start 2024-04-17T09:56:46.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.564+0000 7ff29f200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff29806c140 0x7ff2980ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.564+0000 7ff29f200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2980cc940 con 0x7ff29806f7c0 2024-04-17T09:56:46.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.564+0000 7ff29f200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2980ccc20 con 0x7ff29815b7d0 2024-04-17T09:56:46.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.564+0000 7ff29f200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff29806e0b0 con 0x7ff29806c140 2024-04-17T09:56:46.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.564+0000 7ff297400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff29806f7c0 0x7ff29806ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37052/0 (socket says 172.21.15.32:37052) 2024-04-17T09:56:46.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.564+0000 7ff297400640 1 -- 172.21.15.32:0/592863415 learned_addr learned my addr 172.21.15.32:0/592863415 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:46.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.564+0000 7ff29d200640 1 --2- 172.21.15.32:0/592863415 >> v2:172.21.15.144:3300/0 conn(0x7ff29806c140 0x7ff2980ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.564+0000 7ff296a00640 1 -- 172.21.15.32:0/592863415 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 839026183 0 0) 0x7ff2980cc940 con 0x7ff29806f7c0 2024-04-17T09:56:46.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.564+0000 7ff296a00640 1 -- 172.21.15.32:0/592863415 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff27c003530 con 0x7ff29806f7c0 2024-04-17T09:56:46.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.564+0000 7ff29d200640 1 -- 172.21.15.32:0/592863415 >> v1:172.21.15.144:6789/0 conn(0x7ff29815b7d0 legacy=0x7ff29806e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.564+0000 7ff29d200640 1 -- 172.21.15.32:0/592863415 >> v1:172.21.15.32:6789/0 conn(0x7ff29806f7c0 legacy=0x7ff29806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.564+0000 7ff29d200640 1 -- 172.21.15.32:0/592863415 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29806e390 con 0x7ff29806c140 2024-04-17T09:56:46.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.568+0000 7ff29d200640 1 --2- 172.21.15.32:0/592863415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff29806c140 0x7ff2980ce290 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff290009d90 tx=0x7ff29001c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71091aa8554a3af server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:46.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.568+0000 7ff296a00640 1 -- 172.21.15.32:0/592863415 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2900b2070 con 0x7ff29806c140 2024-04-17T09:56:46.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.568+0000 7ff296a00640 1 -- 172.21.15.32:0/592863415 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2900a59c0 con 0x7ff29806c140 2024-04-17T09:56:46.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.568+0000 7ff296a00640 1 -- 172.21.15.32:0/592863415 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2900a5d70 con 0x7ff29806c140 2024-04-17T09:56:46.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.568+0000 7ff29f200640 1 -- 172.21.15.32:0/592863415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff29806c140 msgr2=0x7ff2980ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.568+0000 7ff29f200640 1 --2- 172.21.15.32:0/592863415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff29806c140 0x7ff2980ce290 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff290009d90 tx=0x7ff29001c8e0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.568+0000 7ff29f200640 1 -- 172.21.15.32:0/592863415 shutdown_connections 2024-04-17T09:56:46.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.568+0000 7ff29f200640 1 --2- 172.21.15.32:0/592863415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff29806c140 0x7ff2980ce290 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.568+0000 7ff29f200640 1 -- 172.21.15.32:0/592863415 >> 172.21.15.32:0/592863415 conn(0x7ff2980b2120 msgr2=0x7ff2980b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:46.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.568+0000 7ff29f200640 1 -- 172.21.15.32:0/592863415 shutdown_connections 2024-04-17T09:56:46.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.568+0000 7ff29f200640 1 -- 172.21.15.32:0/592863415 wait complete. 2024-04-17T09:56:46.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.568+0000 7ff29f200640 1 Processor -- start 2024-04-17T09:56:46.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.572+0000 7ff29f200640 1 -- start start 2024-04-17T09:56:46.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.572+0000 7ff29f200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff29806c140 0x7ff298147370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.572+0000 7ff29f200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff29806f7c0 0x7ff298157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.572+0000 7ff29f200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff29815b7d0 0x7ff2981579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.572+0000 7ff29f200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff298169640 con 0x7ff29806c140 2024-04-17T09:56:46.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.572+0000 7ff29f200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff2981694c0 con 0x7ff29806f7c0 2024-04-17T09:56:46.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.572+0000 7ff29f200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff2981697c0 con 0x7ff29815b7d0 2024-04-17T09:56:46.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.572+0000 7ff297e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff29806c140 0x7ff298147370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.572+0000 7ff297e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff29806c140 0x7ff298147370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35014/0 (socket says 172.21.15.32:35014) 2024-04-17T09:56:46.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.572+0000 7ff297e00640 1 -- 172.21.15.32:0/376004550 learned_addr learned my addr 172.21.15.32:0/376004550 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:46.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.572+0000 7ff297400640 1 --2- 172.21.15.32:0/376004550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff29806f7c0 0x7ff298157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.572+0000 7ff29d200640 1 --2- 172.21.15.32:0/376004550 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff29815b7d0 0x7ff2981579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.572+0000 7ff297e00640 1 -- 172.21.15.32:0/376004550 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff29815b7d0 msgr2=0x7ff2981579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.572+0000 7ff297e00640 1 --2- 172.21.15.32:0/376004550 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff29815b7d0 0x7ff2981579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.572+0000 7ff297e00640 1 -- 172.21.15.32:0/376004550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff29806f7c0 msgr2=0x7ff298157480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.572+0000 7ff297e00640 1 --2- 172.21.15.32:0/376004550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff29806f7c0 0x7ff298157480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.572+0000 7ff297e00640 1 -- 172.21.15.32:0/376004550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2980692a0 con 0x7ff29806c140 2024-04-17T09:56:46.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.572+0000 7ff297e00640 1 --2- 172.21.15.32:0/376004550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff29806c140 0x7ff298147370 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff28800c370 tx=0x7ff28800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:46.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.572+0000 7ff28fe00640 1 -- 172.21.15.32:0/376004550 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2880052d0 con 0x7ff29806c140 2024-04-17T09:56:46.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.572+0000 7ff28fe00640 1 -- 172.21.15.32:0/376004550 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff288005470 con 0x7ff29806c140 2024-04-17T09:56:46.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.572+0000 7ff28fe00640 1 -- 172.21.15.32:0/376004550 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff288007260 con 0x7ff29806c140 2024-04-17T09:56:46.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.572+0000 7ff29f200640 1 -- 172.21.15.32:0/376004550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2980b1070 con 0x7ff29806c140 2024-04-17T09:56:46.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.572+0000 7ff29f200640 1 -- 172.21.15.32:0/376004550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2981514f0 con 0x7ff29806c140 2024-04-17T09:56:46.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff28fe00640 1 -- 172.21.15.32:0/376004550 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff288007400 con 0x7ff29806c140 2024-04-17T09:56:46.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff28fe00640 1 --2- 172.21.15.32:0/376004550 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff278068380 0x7ff27806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff28fe00640 1 -- 172.21.15.32:0/376004550 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7ff288085e30 con 0x7ff29806c140 2024-04-17T09:56:46.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff297400640 1 --2- 172.21.15.32:0/376004550 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff278068380 0x7ff27806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff2a278c000 1 Processor -- start 2024-04-17T09:56:46.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff2a278c000 1 -- start start 2024-04-17T09:56:46.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff297400640 1 --2- 172.21.15.32:0/376004550 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff278068380 0x7ff27806a840 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff298167270 tx=0x7ff284007d40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:46.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff2a278c000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74435aa50 0x55b74435ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff2a278c000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74435b370 0x55b74430da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff2a278c000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74430df70 0x55b74430e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff2a278c000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b74415f590 con 0x55b74430df70 2024-04-17T09:56:46.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff2a278c000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b744118580 con 0x55b74435b370 2024-04-17T09:56:46.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff2a278c000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b74415b550 con 0x55b74435aa50 2024-04-17T09:56:46.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff29d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74430df70 0x55b74430e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff29d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74430df70 0x55b74430e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35024/0 (socket says 172.21.15.32:35024) 2024-04-17T09:56:46.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff29d200640 1 -- 172.21.15.32:0/2570119196 learned_addr learned my addr 172.21.15.32:0/2570119196 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:46.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff297e00640 1 --2- 172.21.15.32:0/2570119196 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74435aa50 0x55b74435ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff297400640 1 --2- 172.21.15.32:0/2570119196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74435b370 0x55b74430da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff297e00640 1 -- 172.21.15.32:0/2570119196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74435b370 msgr2=0x55b74430da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:46.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff297e00640 1 --2- 172.21.15.32:0/2570119196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74435b370 0x55b74430da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff297e00640 1 -- 172.21.15.32:0/2570119196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74430df70 msgr2=0x55b74430e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:46.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff297e00640 1 --2- 172.21.15.32:0/2570119196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74430df70 0x55b74430e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff297e00640 1 -- 172.21.15.32:0/2570119196 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b744206b90 con 0x55b74435aa50 2024-04-17T09:56:46.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff29d200640 1 --2- 172.21.15.32:0/2570119196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74430df70 0x55b74430e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:56:46.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.576+0000 7ff297e00640 1 --2- 172.21.15.32:0/2570119196 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74435aa50 0x55b74435ae30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff2880842c0 tx=0x7ff288055c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95a13a4d4172e7ee server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:46.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff28cc00640 1 -- 172.21.15.32:0/2570119196 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28806e170 con 0x55b74435aa50 2024-04-17T09:56:46.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff28cc00640 1 -- 172.21.15.32:0/2570119196 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff288054000 con 0x55b74435aa50 2024-04-17T09:56:46.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff28cc00640 1 -- 172.21.15.32:0/2570119196 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff288055150 con 0x55b74435aa50 2024-04-17T09:56:46.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff2a278c000 1 -- 172.21.15.32:0/2570119196 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74435aa50 msgr2=0x55b74435ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff2a278c000 1 --2- 172.21.15.32:0/2570119196 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74435aa50 0x55b74435ae30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff2880842c0 tx=0x7ff288055c00 comp rx=0 tx=0).stop 2024-04-17T09:56:46.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff2a278c000 1 -- 172.21.15.32:0/2570119196 shutdown_connections 2024-04-17T09:56:46.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff2a278c000 1 --2- 172.21.15.32:0/2570119196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74430df70 0x55b74430e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff2a278c000 1 --2- 172.21.15.32:0/2570119196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74435b370 0x55b74430da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff2a278c000 1 --2- 172.21.15.32:0/2570119196 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74435aa50 0x55b74435ae30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff2a278c000 1 -- 172.21.15.32:0/2570119196 >> 172.21.15.32:0/2570119196 conn(0x55b74433b7f0 msgr2=0x55b74433dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:46.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff2a278c000 1 -- 172.21.15.32:0/2570119196 shutdown_connections 2024-04-17T09:56:46.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff2a278c000 1 -- 172.21.15.32:0/2570119196 wait complete. 2024-04-17T09:56:46.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff2a278c000 1 -- start start 2024-04-17T09:56:46.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff2a278c000 1 Processor -- start 2024-04-17T09:56:46.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff2a278c000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74430df70 0x55b7443126b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff2a278c000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74435aa50 0x55b744312bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff2a278c000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74435b370 0x55b744313130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff2a278c000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b74416a4c0 con 0x55b74430df70 2024-04-17T09:56:46.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff2a278c000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b7440ffc80 con 0x55b74435aa50 2024-04-17T09:56:46.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff2a278c000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b74412a930 con 0x55b74435b370 2024-04-17T09:56:46.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff2a278c000 10 client.0 _ll_register_callbacks cb 0x55b7442fb6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:56:46.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff297e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74430df70 0x55b7443126b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff297e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74430df70 0x55b7443126b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35026/0 (socket says 172.21.15.32:35026) 2024-04-17T09:56:46.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff297e00640 1 -- 172.21.15.32:0/3615960603 learned_addr learned my addr 172.21.15.32:0/3615960603 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:46.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff297400640 1 --2- 172.21.15.32:0/3615960603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74435aa50 0x55b744312bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff29d200640 1 --2- 172.21.15.32:0/3615960603 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74435b370 0x55b744313130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff297e00640 1 -- 172.21.15.32:0/3615960603 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74435b370 msgr2=0x55b744313130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:46.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff297e00640 1 --2- 172.21.15.32:0/3615960603 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74435b370 0x55b744313130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff297e00640 1 -- 172.21.15.32:0/3615960603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74435aa50 msgr2=0x55b744312bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:46.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff297e00640 1 --2- 172.21.15.32:0/3615960603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74435aa50 0x55b744312bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff297e00640 1 -- 172.21.15.32:0/3615960603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b744138b20 con 0x55b74430df70 2024-04-17T09:56:46.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff297e00640 1 --2- 172.21.15.32:0/3615960603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74430df70 0x55b7443126b0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff288052160 tx=0x7ff28801d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6727b5aba372f821 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:46.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff281600640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:56:46.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff281600640 1 -- 172.21.15.32:0/3615960603 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff288055350 con 0x55b74430df70 2024-04-17T09:56:46.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff281600640 1 -- 172.21.15.32:0/3615960603 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7ff28805c200 con 0x55b74430df70 2024-04-17T09:56:46.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff2a278c000 1 -- 172.21.15.32:0/3615960603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff288054000 con 0x55b74430df70 2024-04-17T09:56:46.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff281600640 1 -- 172.21.15.32:0/3615960603 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff28805c400 con 0x55b74430df70 2024-04-17T09:56:46.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.580+0000 7ff2a278c000 10 client.6498 fetch_fsmap learned FSMap version 33 2024-04-17T09:56:46.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff2a278c000 1 -- 172.21.15.32:0/3615960603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55b7441a3610 con 0x55b74430df70 2024-04-17T09:56:46.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff281600640 1 -- 172.21.15.32:0/3615960603 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff288054000 con 0x55b74430df70 2024-04-17T09:56:46.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff281600640 1 -- 172.21.15.32:0/3615960603 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 33) ==== 3495+0+0 (secure 0 0 0) 0x7ff288074ae0 con 0x55b74430df70 2024-04-17T09:56:46.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff2a278c000 10 client.6498 fetch_fsmap finished waiting for FSMap version 33 2024-04-17T09:56:46.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff2a278c000 10 client.6498 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T09:56:46.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff2a278c000 20 client.6498 populate_metadata read hostname 'smithi032' 2024-04-17T09:56:46.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff2a278c000 1 --2- 172.21.15.32:0/3615960603 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x55b744369ae0 0x55b74436bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff2a278c000 4 client.6498 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:56:46.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff2a278c000 1 -- 172.21.15.32:0/3615960603 --> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7440f8e40 con 0x55b744369ae0 2024-04-17T09:56:46.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff297400640 1 --2- 172.21.15.32:0/3615960603 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x55b744369ae0 0x55b74436bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff297400640 1 --2- 172.21.15.32:0/3615960603 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x55b744369ae0 0x55b74436bed0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df5964fd4e885f5e server_cookie=ac555f0f0054e54c in_seq=0 out_seq=0 2024-04-17T09:56:46.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff281600640 10 client.6498 ms_handle_connect on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:46.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff281600640 1 -- 172.21.15.32:0/3615960603 <== mds.0 v2:172.21.15.32:6835/866826055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b7440f8e40 con 0x55b744369ae0 2024-04-17T09:56:46.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff281600640 10 client.6498 handle_command_reply: tid=0 2024-04-17T09:56:46.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff2a278c000 1 client.6498 shutdown 2024-04-17T09:56:46.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff2a278c000 1 -- 172.21.15.32:0/3615960603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74430df70 msgr2=0x55b7443126b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff2a278c000 1 --2- 172.21.15.32:0/3615960603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74430df70 0x55b7443126b0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff288052160 tx=0x7ff28801d040 comp rx=0 tx=0).stop 2024-04-17T09:56:46.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff2a278c000 1 -- 172.21.15.32:0/3615960603 shutdown_connections 2024-04-17T09:56:46.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff2a278c000 1 --2- 172.21.15.32:0/3615960603 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74435b370 0x55b744313130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff2a278c000 1 --2- 172.21.15.32:0/3615960603 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x55b744369ae0 0x55b74436bed0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff2a278c000 1 --2- 172.21.15.32:0/3615960603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74435aa50 0x55b744312bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff2a278c000 1 --2- 172.21.15.32:0/3615960603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74430df70 0x55b7443126b0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff281600640 0 client.6498 ms_handle_reset on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:46.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff2a278c000 1 -- 172.21.15.32:0/3615960603 >> 172.21.15.32:0/3615960603 conn(0x55b74433b7f0 msgr2=0x55b74433dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:46.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff2a278c000 1 -- 172.21.15.32:0/3615960603 shutdown_connections 2024-04-17T09:56:46.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff2a278c000 1 -- 172.21.15.32:0/3615960603 wait complete. 2024-04-17T09:56:46.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.584+0000 7ff2a278c000 20 client.6498 trim_cache size 0 max 16384 2024-04-17T09:56:46.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff2a278c000 1 Processor -- start 2024-04-17T09:56:46.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff2a278c000 1 -- start start 2024-04-17T09:56:46.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff2a278c000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74430df70 0x55b744359f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff2a278c000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74435aa50 0x55b74435a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff2a278c000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74435b370 0x55b74430e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff2a278c000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b744163bc0 con 0x55b74435b370 2024-04-17T09:56:46.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff2a278c000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b74416a4c0 con 0x55b74435aa50 2024-04-17T09:56:46.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff2a278c000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b74412a930 con 0x55b74430df70 2024-04-17T09:56:46.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff29d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74435b370 0x55b74430e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff29d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74435b370 0x55b74430e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35036/0 (socket says 172.21.15.32:35036) 2024-04-17T09:56:46.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff29d200640 1 -- 172.21.15.32:0/4167685299 learned_addr learned my addr 172.21.15.32:0/4167685299 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:46.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff297e00640 1 --2- 172.21.15.32:0/4167685299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74430df70 0x55b744359f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff297400640 1 --2- 172.21.15.32:0/4167685299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74435aa50 0x55b74435a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff29d200640 1 -- 172.21.15.32:0/4167685299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74430df70 msgr2=0x55b744359f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:46.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff29d200640 1 --2- 172.21.15.32:0/4167685299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74430df70 0x55b744359f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff29d200640 1 -- 172.21.15.32:0/4167685299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74435aa50 msgr2=0x55b74435a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:46.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff29d200640 1 --2- 172.21.15.32:0/4167685299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74435aa50 0x55b74435a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff29d200640 1 -- 172.21.15.32:0/4167685299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b744135190 con 0x55b74435b370 2024-04-17T09:56:46.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff29d200640 1 --2- 172.21.15.32:0/4167685299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74435b370 0x55b74430e350 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff2900984e0 tx=0x7ff290004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2323d8438007b55b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff283e00640 1 -- 172.21.15.32:0/4167685299 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2900b2040 con 0x55b74435b370 2024-04-17T09:56:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff283e00640 1 -- 172.21.15.32:0/4167685299 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2900bf070 con 0x55b74435b370 2024-04-17T09:56:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff283e00640 1 -- 172.21.15.32:0/4167685299 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2900a57c0 con 0x55b74435b370 2024-04-17T09:56:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff2a278c000 1 -- 172.21.15.32:0/4167685299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74435b370 msgr2=0x55b74430e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff2a278c000 1 --2- 172.21.15.32:0/4167685299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74435b370 0x55b74430e350 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff2900984e0 tx=0x7ff290004ac0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff2a278c000 1 -- 172.21.15.32:0/4167685299 shutdown_connections 2024-04-17T09:56:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff2a278c000 1 --2- 172.21.15.32:0/4167685299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74435b370 0x55b74430e350 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff2a278c000 1 --2- 172.21.15.32:0/4167685299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74435aa50 0x55b74435a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff2a278c000 1 --2- 172.21.15.32:0/4167685299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74430df70 0x55b744359f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff2a278c000 1 -- 172.21.15.32:0/4167685299 >> 172.21.15.32:0/4167685299 conn(0x55b74433b7f0 msgr2=0x55b744338c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff2a278c000 1 -- 172.21.15.32:0/4167685299 shutdown_connections 2024-04-17T09:56:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff2a278c000 1 -- 172.21.15.32:0/4167685299 wait complete. 2024-04-17T09:56:46.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff2a278c000 1 -- start start 2024-04-17T09:56:46.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.600+0000 7ff2a278c000 1 Processor -- start 2024-04-17T09:56:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74430df70 0x55b74430f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74435aa50 0x55b74436cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74435b370 0x55b74436d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b744319360 con 0x55b74430df70 2024-04-17T09:56:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b744104230 con 0x55b74435b370 2024-04-17T09:56:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b74411da90 con 0x55b74435aa50 2024-04-17T09:56:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 10 client.0 _ll_register_callbacks cb 0x7ff28806d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:56:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff297e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74430df70 0x55b74430f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff297e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74430df70 0x55b74430f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35048/0 (socket says 172.21.15.32:35048) 2024-04-17T09:56:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff297e00640 1 -- 172.21.15.32:0/2391342907 learned_addr learned my addr 172.21.15.32:0/2391342907 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff297400640 1 --2- 172.21.15.32:0/2391342907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74435aa50 0x55b74436cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff29d200640 1 --2- 172.21.15.32:0/2391342907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74435b370 0x55b74436d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff297e00640 1 -- 172.21.15.32:0/2391342907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74435aa50 msgr2=0x55b74436cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff297e00640 1 --2- 172.21.15.32:0/2391342907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74435aa50 0x55b74436cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff297e00640 1 -- 172.21.15.32:0/2391342907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74435b370 msgr2=0x55b74436d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff297e00640 1 --2- 172.21.15.32:0/2391342907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74435b370 0x55b74436d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff297e00640 1 -- 172.21.15.32:0/2391342907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7441187d0 con 0x55b74430df70 2024-04-17T09:56:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff297400640 1 --2- 172.21.15.32:0/2391342907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74435aa50 0x55b74436cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff297e00640 1 --2- 172.21.15.32:0/2391342907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74430df70 0x55b74430f750 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff288055c40 tx=0x7ff288007770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a0b3862a5428b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff281600640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:56:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff281600640 1 -- 172.21.15.32:0/2391342907 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28805a000 con 0x55b74430df70 2024-04-17T09:56:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff281600640 1 -- 172.21.15.32:0/2391342907 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7ff28806bd10 con 0x55b74430df70 2024-04-17T09:56:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 1 -- 172.21.15.32:0/2391342907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2900bf070 con 0x55b74430df70 2024-04-17T09:56:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff281600640 1 -- 172.21.15.32:0/2391342907 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff288055150 con 0x55b74430df70 2024-04-17T09:56:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 10 client.6504 fetch_fsmap learned FSMap version 33 2024-04-17T09:56:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 1 -- 172.21.15.32:0/2391342907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55b744104440 con 0x55b74430df70 2024-04-17T09:56:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff281600640 1 -- 172.21.15.32:0/2391342907 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2900bf070 con 0x55b74430df70 2024-04-17T09:56:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff281600640 1 -- 172.21.15.32:0/2391342907 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 33) ==== 3495+0+0 (secure 0 0 0) 0x7ff2880552f0 con 0x55b74430df70 2024-04-17T09:56:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 10 client.6504 fetch_fsmap finished waiting for FSMap version 33 2024-04-17T09:56:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 10 client.6504 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T09:56:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 20 client.6504 populate_metadata read hostname 'smithi032' 2024-04-17T09:56:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 1 --2- 172.21.15.32:0/2391342907 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x55b7443c9090 0x55b7443c9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 4 client.6504 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-17T09:56:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 1 -- 172.21.15.32:0/2391342907 --> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55b74416cfb0 con 0x55b7443c9090 2024-04-17T09:56:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff297400640 1 --2- 172.21.15.32:0/2391342907 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x55b7443c9090 0x55b7443c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff297400640 1 --2- 172.21.15.32:0/2391342907 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x55b7443c9090 0x55b7443c9470 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc03008f222b0a26 server_cookie=490aad2c93b525e2 in_seq=0 out_seq=0 2024-04-17T09:56:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff281600640 10 client.6504 ms_handle_connect on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff281600640 1 -- 172.21.15.32:0/2391342907 <== mds.0 v2:172.21.15.32:6835/866826055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+140 (crc 0 0 0) 0x55b74416cfb0 con 0x55b7443c9090 2024-04-17T09:56:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff281600640 10 client.6504 handle_command_reply: tid=0 2024-04-17T09:56:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 1 client.6504 shutdown 2024-04-17T09:56:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 1 -- 172.21.15.32:0/2391342907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74430df70 msgr2=0x55b74430f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 1 --2- 172.21.15.32:0/2391342907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74430df70 0x55b74430f750 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff288055c40 tx=0x7ff288007770 comp rx=0 tx=0).stop 2024-04-17T09:56:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 1 -- 172.21.15.32:0/2391342907 shutdown_connections 2024-04-17T09:56:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 1 --2- 172.21.15.32:0/2391342907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74435aa50 0x55b74436cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 1 --2- 172.21.15.32:0/2391342907 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x55b7443c9090 0x55b7443c9470 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 1 --2- 172.21.15.32:0/2391342907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74435b370 0x55b74436d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 1 --2- 172.21.15.32:0/2391342907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b74430df70 0x55b74430f750 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff281600640 0 client.6504 ms_handle_reset on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 1 -- 172.21.15.32:0/2391342907 >> 172.21.15.32:0/2391342907 conn(0x55b74433b7f0 msgr2=0x55b744339520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:46.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 1 -- 172.21.15.32:0/2391342907 shutdown_connections 2024-04-17T09:56:46.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 1 -- 172.21.15.32:0/2391342907 wait complete. 2024-04-17T09:56:46.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.604+0000 7ff2a278c000 20 client.6504 trim_cache size 0 max 16384 2024-04-17T09:56:46.614 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:46.614 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-17T09:56:46.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:56:46.614 INFO:teuthology.orchestra.run.smithi032.stdout: "damage_type": "backtrace", 2024-04-17T09:56:46.614 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 795710424, 2024-04-17T09:56:46.614 INFO:teuthology.orchestra.run.smithi032.stdout: "ino": 1099511627777, 2024-04-17T09:56:46.615 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/dir_test/file" 2024-04-17T09:56:46.615 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:56:46.615 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-17T09:56:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.608+0000 7ff29f200640 1 -- 172.21.15.32:0/376004550 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff278068380 msgr2=0x7ff27806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.608+0000 7ff29f200640 1 --2- 172.21.15.32:0/376004550 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff278068380 0x7ff27806a840 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff298167270 tx=0x7ff284007d40 comp rx=0 tx=0).stop 2024-04-17T09:56:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.608+0000 7ff29f200640 1 -- 172.21.15.32:0/376004550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff29806c140 msgr2=0x7ff298147370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.608+0000 7ff29f200640 1 --2- 172.21.15.32:0/376004550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff29806c140 0x7ff298147370 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff28800c370 tx=0x7ff28800c830 comp rx=0 tx=0).stop 2024-04-17T09:56:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.608+0000 7ff29f200640 1 -- 172.21.15.32:0/376004550 shutdown_connections 2024-04-17T09:56:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.608+0000 7ff29f200640 1 --2- 172.21.15.32:0/376004550 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff278068380 0x7ff27806a840 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.608+0000 7ff29f200640 1 --2- 172.21.15.32:0/376004550 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff29815b7d0 0x7ff2981579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.608+0000 7ff29f200640 1 --2- 172.21.15.32:0/376004550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff29806f7c0 0x7ff298157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.608+0000 7ff29f200640 1 --2- 172.21.15.32:0/376004550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff29806c140 0x7ff298147370 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.608+0000 7ff29f200640 1 -- 172.21.15.32:0/376004550 >> 172.21.15.32:0/376004550 conn(0x7ff2980b2120 msgr2=0x7ff298158c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.608+0000 7ff29f200640 1 -- 172.21.15.32:0/376004550 shutdown_connections 2024-04-17T09:56:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.608+0000 7ff29f200640 1 -- 172.21.15.32:0/376004550 wait complete. 2024-04-17T09:56:46.627 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-17T09:56:46.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.764+0000 7f8f1d200640 1 Processor -- start 2024-04-17T09:56:46.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.764+0000 7f8f1d200640 1 -- start start 2024-04-17T09:56:46.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.764+0000 7f8f1d200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8f18151db0 0x7f8f180b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.764+0000 7f8f1d200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f180b3490 con 0x7f8f1806f630 2024-04-17T09:56:46.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.764+0000 7f8f1d200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f180cea40 con 0x7f8f18072d90 2024-04-17T09:56:46.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.764+0000 7f8f1d200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f8f180b44f0 con 0x7f8f18151db0 2024-04-17T09:56:46.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.764+0000 7f8f0fe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8f1806f630 0x7f8f18072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37064/0 (socket says 172.21.15.32:37064) 2024-04-17T09:56:46.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f0fe00640 1 -- 172.21.15.32:0/347107940 learned_addr learned my addr 172.21.15.32:0/347107940 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:46.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15e00640 1 --2- 172.21.15.32:0/347107940 >> v2:172.21.15.144:3300/0 conn(0x7f8f18151db0 0x7f8f180b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15400640 1 -- 172.21.15.32:0/347107940 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1106717396 0 0) 0x7f8f180b3490 con 0x7f8f1806f630 2024-04-17T09:56:46.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15400640 1 -- 172.21.15.32:0/347107940 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8efc003530 con 0x7f8f1806f630 2024-04-17T09:56:46.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15400640 1 -- 172.21.15.32:0/347107940 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3881150071 0 0) 0x7f8f180cea40 con 0x7f8f18072d90 2024-04-17T09:56:46.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15400640 1 -- 172.21.15.32:0/347107940 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f180b3490 con 0x7f8f18072d90 2024-04-17T09:56:46.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15400640 1 -- 172.21.15.32:0/347107940 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 123682827 0 0) 0x7f8efc003530 con 0x7f8f1806f630 2024-04-17T09:56:46.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15400640 1 -- 172.21.15.32:0/347107940 >> v2:172.21.15.144:3300/0 conn(0x7f8f18151db0 msgr2=0x7f8f180b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15400640 1 --2- 172.21.15.32:0/347107940 >> v2:172.21.15.144:3300/0 conn(0x7f8f18151db0 0x7f8f180b51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15400640 1 -- 172.21.15.32:0/347107940 >> v1:172.21.15.144:6789/0 conn(0x7f8f18072d90 legacy=0x7f8f1806de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15400640 1 -- 172.21.15.32:0/347107940 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f180b3730 con 0x7f8f1806f630 2024-04-17T09:56:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15400640 1 -- 172.21.15.32:0/347107940 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f8f00001620 con 0x7f8f1806f630 2024-04-17T09:56:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15e00640 1 --2- 172.21.15.32:0/347107940 >> v2:172.21.15.144:3300/0 conn(0x7f8f18151db0 0x7f8f180b51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:56:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15400640 1 -- 172.21.15.32:0/347107940 >> v1:172.21.15.32:6789/0 conn(0x7f8f1806f630 legacy=0x7f8f18072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15400640 1 --2- 172.21.15.32:0/347107940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8efc003ba0 0x7f8efc003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15400640 1 --2- 172.21.15.32:0/347107940 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8efc004810 0x7f8efc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15400640 1 --2- 172.21.15.32:0/347107940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8efc0051d0 0x7f8efc00cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15400640 1 -- 172.21.15.32:0/347107940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8f180b44f0 con 0x7f8efc0051d0 2024-04-17T09:56:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15400640 1 -- 172.21.15.32:0/347107940 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc00d1b0 con 0x7f8efc003ba0 2024-04-17T09:56:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15400640 1 -- 172.21.15.32:0/347107940 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8efc00d3b0 con 0x7f8efc004810 2024-04-17T09:56:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15e00640 1 --2- 172.21.15.32:0/347107940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8efc003ba0 0x7f8efc003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f16800640 1 --2- 172.21.15.32:0/347107940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8efc0051d0 0x7f8efc00cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f0fe00640 1 --2- 172.21.15.32:0/347107940 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8efc004810 0x7f8efc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15e00640 1 -- 172.21.15.32:0/347107940 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8efc004810 msgr2=0x7f8efc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:46.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15e00640 1 --2- 172.21.15.32:0/347107940 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8efc004810 0x7f8efc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15e00640 1 -- 172.21.15.32:0/347107940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8efc0051d0 msgr2=0x7f8efc00cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:46.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15e00640 1 --2- 172.21.15.32:0/347107940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8efc0051d0 0x7f8efc00cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f15e00640 1 -- 172.21.15.32:0/347107940 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8efc003530 con 0x7f8efc003ba0 2024-04-17T09:56:46.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.768+0000 7f8f16800640 1 -- 172.21.15.32:0/347107940 reap_dead start 2024-04-17T09:56:46.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.772+0000 7f8f15e00640 1 --2- 172.21.15.32:0/347107940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8efc003ba0 0x7f8efc003fa0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009cb0 tx=0x7f8f0801cc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37239a0fa821747c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:46.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.772+0000 7f8f15400640 1 -- 172.21.15.32:0/347107940 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f080b2070 con 0x7f8efc003ba0 2024-04-17T09:56:46.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.772+0000 7f8f15400640 1 -- 172.21.15.32:0/347107940 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f080a57c0 con 0x7f8efc003ba0 2024-04-17T09:56:46.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.772+0000 7f8f15400640 1 -- 172.21.15.32:0/347107940 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08009880 con 0x7f8efc003ba0 2024-04-17T09:56:46.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.772+0000 7f8f1d200640 1 -- 172.21.15.32:0/347107940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8efc003ba0 msgr2=0x7f8efc003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.772+0000 7f8f1d200640 1 --2- 172.21.15.32:0/347107940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8efc003ba0 0x7f8efc003fa0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009cb0 tx=0x7f8f0801cc10 comp rx=0 tx=0).stop 2024-04-17T09:56:46.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.772+0000 7f8f1d200640 1 -- 172.21.15.32:0/347107940 shutdown_connections 2024-04-17T09:56:46.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.772+0000 7f8f1d200640 1 --2- 172.21.15.32:0/347107940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8efc003ba0 0x7f8efc003fa0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.772+0000 7f8f1d200640 1 -- 172.21.15.32:0/347107940 >> 172.21.15.32:0/347107940 conn(0x7f8f1806ba30 msgr2=0x7f8f1806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:46.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.772+0000 7f8f1d200640 1 -- 172.21.15.32:0/347107940 shutdown_connections 2024-04-17T09:56:46.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.772+0000 7f8f1d200640 1 -- 172.21.15.32:0/347107940 wait complete. 2024-04-17T09:56:46.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.772+0000 7f8f1d200640 1 Processor -- start 2024-04-17T09:56:46.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.772+0000 7f8f1d200640 1 -- start start 2024-04-17T09:56:46.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f1d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8efc003ba0 0x7f8f18153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f1d200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f18152190 0x7f8f18152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f1d200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f181f06f0 0x7f8f181f0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f1d200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8f180b3fd0 con 0x7f8efc003ba0 2024-04-17T09:56:46.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f1d200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f180b3cf0 con 0x7f8f18152190 2024-04-17T09:56:46.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f1d200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8f180b51c0 con 0x7f8f181f06f0 2024-04-17T09:56:46.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f15e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8efc003ba0 0x7f8f18153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f15e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8efc003ba0 0x7f8f18153e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35060/0 (socket says 172.21.15.32:35060) 2024-04-17T09:56:46.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f15e00640 1 -- 172.21.15.32:0/610102351 learned_addr learned my addr 172.21.15.32:0/610102351 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:46.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f0fe00640 1 --2- 172.21.15.32:0/610102351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f18152190 0x7f8f18152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f16800640 1 --2- 172.21.15.32:0/610102351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f181f06f0 0x7f8f181f0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f15e00640 1 -- 172.21.15.32:0/610102351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f181f06f0 msgr2=0x7f8f181f0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f15e00640 1 --2- 172.21.15.32:0/610102351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f181f06f0 0x7f8f181f0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f15e00640 1 -- 172.21.15.32:0/610102351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f18152190 msgr2=0x7f8f18152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f15e00640 1 --2- 172.21.15.32:0/610102351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f18152190 0x7f8f18152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f15e00640 1 -- 172.21.15.32:0/610102351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f18147810 con 0x7f8efc003ba0 2024-04-17T09:56:46.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f16800640 1 --2- 172.21.15.32:0/610102351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f181f06f0 0x7f8f181f0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:46.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f15e00640 1 --2- 172.21.15.32:0/610102351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8efc003ba0 0x7f8f18153e10 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f8f0801ca40 tx=0x7f8f080199b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:46.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f0e000640 1 -- 172.21.15.32:0/610102351 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f080bb070 con 0x7f8efc003ba0 2024-04-17T09:56:46.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f0e000640 1 -- 172.21.15.32:0/610102351 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f080ad440 con 0x7f8efc003ba0 2024-04-17T09:56:46.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f0e000640 1 -- 172.21.15.32:0/610102351 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f080b2040 con 0x7f8efc003ba0 2024-04-17T09:56:46.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f1d200640 1 -- 172.21.15.32:0/610102351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f18154350 con 0x7f8efc003ba0 2024-04-17T09:56:46.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f1d200640 1 -- 172.21.15.32:0/610102351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f18153050 con 0x7f8efc003ba0 2024-04-17T09:56:46.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f1d200640 1 -- 172.21.15.32:0/610102351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f18068a90 con 0x7f8efc003ba0 2024-04-17T09:56:46.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.776+0000 7f8f0e000640 1 -- 172.21.15.32:0/610102351 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8f080ad810 con 0x7f8efc003ba0 2024-04-17T09:56:46.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.780+0000 7f8f0e000640 1 --2- 172.21.15.32:0/610102351 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8ef4068380 0x7f8ef406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:46.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.780+0000 7f8f0e000640 1 -- 172.21.15.32:0/610102351 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f8f0811fe90 con 0x7f8efc003ba0 2024-04-17T09:56:46.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.780+0000 7f8f0fe00640 1 --2- 172.21.15.32:0/610102351 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8ef4068380 0x7f8ef406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:46.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.780+0000 7f8f0fe00640 1 --2- 172.21.15.32:0/610102351 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8ef4068380 0x7f8ef406a840 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8f00001b70 tx=0x7f8f00000ee0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.784+0000 7f8f0e000640 1 -- 172.21.15.32:0/610102351 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f8f18147810 con 0x7f8efc003ba0 2024-04-17T09:56:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.984+0000 7f8f1d200640 1 -- 172.21.15.32:0/610102351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8f180cd480 con 0x7f8efc003ba0 2024-04-17T09:56:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.984+0000 7f8f0e000640 1 -- 172.21.15.32:0/610102351 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f8f08016920 con 0x7f8efc003ba0 2024-04-17T09:56:46.984 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:46.985 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-17T09:56:46.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.984+0000 7f8f1d200640 1 -- 172.21.15.32:0/610102351 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8ef4068380 msgr2=0x7f8ef406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.984+0000 7f8f1d200640 1 --2- 172.21.15.32:0/610102351 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8ef4068380 0x7f8ef406a840 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8f00001b70 tx=0x7f8f00000ee0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.984+0000 7f8f1d200640 1 -- 172.21.15.32:0/610102351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8efc003ba0 msgr2=0x7f8f18153e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.984+0000 7f8f1d200640 1 --2- 172.21.15.32:0/610102351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8efc003ba0 0x7f8f18153e10 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f8f0801ca40 tx=0x7f8f080199b0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.984+0000 7f8f1d200640 1 -- 172.21.15.32:0/610102351 shutdown_connections 2024-04-17T09:56:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.984+0000 7f8f1d200640 1 --2- 172.21.15.32:0/610102351 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8ef4068380 0x7f8ef406a840 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.984+0000 7f8f1d200640 1 --2- 172.21.15.32:0/610102351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f181f06f0 0x7f8f181f0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.984+0000 7f8f1d200640 1 --2- 172.21.15.32:0/610102351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f18152190 0x7f8f18152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.984+0000 7f8f1d200640 1 --2- 172.21.15.32:0/610102351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8efc003ba0 0x7f8f18153e10 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.984+0000 7f8f1d200640 1 -- 172.21.15.32:0/610102351 >> 172.21.15.32:0/610102351 conn(0x7f8f1806ba30 msgr2=0x7f8f180b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.984+0000 7f8f1d200640 1 -- 172.21.15.32:0/610102351 shutdown_connections 2024-04-17T09:56:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:46.984+0000 7f8f1d200640 1 -- 172.21.15.32:0/610102351 wait complete. 2024-04-17T09:56:47.006 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-17T09:56:47.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:56:47.007 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-04-17T09:56:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.148+0000 7fcc61000640 1 Processor -- start 2024-04-17T09:56:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.148+0000 7fcc61000640 1 -- start start 2024-04-17T09:56:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.152+0000 7fcc61000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc5c1460a0 0x7fcc5c0ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.152+0000 7fcc61000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc5c06eeb0 con 0x7fcc5c0724c0 2024-04-17T09:56:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.152+0000 7fcc61000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc5c06f190 con 0x7fcc5c06e6f0 2024-04-17T09:56:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.152+0000 7fcc61000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fcc5c15de00 con 0x7fcc5c1460a0 2024-04-17T09:56:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.152+0000 7fcc59e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc5c1460a0 0x7fcc5c0ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.152+0000 7fcc5a800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcc5c0724c0 0x7fcc5c15e550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37068/0 (socket says 172.21.15.32:37068) 2024-04-17T09:56:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.152+0000 7fcc59e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc5c1460a0 0x7fcc5c0ceb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35062/0 (socket says 172.21.15.32:35062) 2024-04-17T09:56:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.152+0000 7fcc5a800640 1 -- 172.21.15.32:0/3696116620 learned_addr learned my addr 172.21.15.32:0/3696116620 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.152+0000 7fcc59e00640 1 -- 172.21.15.32:0/3696116620 >> v1:172.21.15.144:6789/0 conn(0x7fcc5c06e6f0 legacy=0x7fcc5c071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.152+0000 7fcc59e00640 1 -- 172.21.15.32:0/3696116620 >> v1:172.21.15.32:6789/0 conn(0x7fcc5c0724c0 legacy=0x7fcc5c15e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.152+0000 7fcc59e00640 1 -- 172.21.15.32:0/3696116620 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5c06f470 con 0x7fcc5c1460a0 2024-04-17T09:56:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.156+0000 7fcc59e00640 1 --2- 172.21.15.32:0/3696116620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc5c1460a0 0x7fcc5c0ceb40 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c0097c0 tx=0x7fcc4c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51866d255ab71790 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.156+0000 7fcc59400640 1 -- 172.21.15.32:0/3696116620 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c0b0070 con 0x7fcc5c1460a0 2024-04-17T09:56:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.156+0000 7fcc59400640 1 -- 172.21.15.32:0/3696116620 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc4c0ab600 con 0x7fcc5c1460a0 2024-04-17T09:56:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.156+0000 7fcc59400640 1 -- 172.21.15.32:0/3696116620 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c0ab9d0 con 0x7fcc5c1460a0 2024-04-17T09:56:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.156+0000 7fcc61000640 1 -- 172.21.15.32:0/3696116620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc5c1460a0 msgr2=0x7fcc5c0ceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.156+0000 7fcc61000640 1 --2- 172.21.15.32:0/3696116620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc5c1460a0 0x7fcc5c0ceb40 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c0097c0 tx=0x7fcc4c01c4a0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.156+0000 7fcc61000640 1 -- 172.21.15.32:0/3696116620 shutdown_connections 2024-04-17T09:56:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.156+0000 7fcc61000640 1 --2- 172.21.15.32:0/3696116620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc5c1460a0 0x7fcc5c0ceb40 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.156+0000 7fcc61000640 1 -- 172.21.15.32:0/3696116620 >> 172.21.15.32:0/3696116620 conn(0x7fcc5c0b2190 msgr2=0x7fcc5c0b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.156+0000 7fcc61000640 1 -- 172.21.15.32:0/3696116620 shutdown_connections 2024-04-17T09:56:47.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.156+0000 7fcc61000640 1 -- 172.21.15.32:0/3696116620 wait complete. 2024-04-17T09:56:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.156+0000 7fcc61000640 1 Processor -- start 2024-04-17T09:56:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.156+0000 7fcc61000640 1 -- start start 2024-04-17T09:56:47.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc61000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc5c06e6f0 0x7fcc5c13cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc61000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc5c0724c0 0x7fcc5c15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc61000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc5c1460a0 0x7fcc5c15c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc61000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c15f800 con 0x7fcc5c0724c0 2024-04-17T09:56:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc61000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c15f680 con 0x7fcc5c06e6f0 2024-04-17T09:56:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc61000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcc5c16a290 con 0x7fcc5c1460a0 2024-04-17T09:56:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc51400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc5c0724c0 0x7fcc5c15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc51400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc5c0724c0 0x7fcc5c15bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35066/0 (socket says 172.21.15.32:35066) 2024-04-17T09:56:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc51400640 1 -- 172.21.15.32:0/131498560 learned_addr learned my addr 172.21.15.32:0/131498560 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc59e00640 1 --2- 172.21.15.32:0/131498560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc5c06e6f0 0x7fcc5c13cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc51400640 1 -- 172.21.15.32:0/131498560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc5c1460a0 msgr2=0x7fcc5c15c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc5a800640 1 --2- 172.21.15.32:0/131498560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc5c1460a0 0x7fcc5c15c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc51400640 1 --2- 172.21.15.32:0/131498560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc5c1460a0 0x7fcc5c15c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc51400640 1 -- 172.21.15.32:0/131498560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc5c06e6f0 msgr2=0x7fcc5c13cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc51400640 1 --2- 172.21.15.32:0/131498560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc5c06e6f0 0x7fcc5c13cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc51400640 1 -- 172.21.15.32:0/131498560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5c1fba60 con 0x7fcc5c0724c0 2024-04-17T09:56:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc59e00640 1 --2- 172.21.15.32:0/131498560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc5c06e6f0 0x7fcc5c13cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc5a800640 1 --2- 172.21.15.32:0/131498560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc5c1460a0 0x7fcc5c15c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc51400640 1 --2- 172.21.15.32:0/131498560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc5c0724c0 0x7fcc5c15bb30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400c400 tx=0x7fcc4400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc52a00640 1 -- 172.21.15.32:0/131498560 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc440052d0 con 0x7fcc5c0724c0 2024-04-17T09:56:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc52a00640 1 -- 172.21.15.32:0/131498560 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc440054b0 con 0x7fcc5c0724c0 2024-04-17T09:56:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc52a00640 1 -- 172.21.15.32:0/131498560 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44006a70 con 0x7fcc5c0724c0 2024-04-17T09:56:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc61000640 1 -- 172.21.15.32:0/131498560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5c145280 con 0x7fcc5c0724c0 2024-04-17T09:56:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.160+0000 7fcc61000640 1 -- 172.21.15.32:0/131498560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc5c142310 con 0x7fcc5c0724c0 2024-04-17T09:56:47.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc52a00640 1 -- 172.21.15.32:0/131498560 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcc44006cd0 con 0x7fcc5c0724c0 2024-04-17T09:56:47.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc52a00640 1 --2- 172.21.15.32:0/131498560 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc380683d0 0x7fcc3806a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc52a00640 1 -- 172.21.15.32:0/131498560 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fcc44085f10 con 0x7fcc5c0724c0 2024-04-17T09:56:47.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc64506000 1 Processor -- start 2024-04-17T09:56:47.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc59e00640 1 --2- 172.21.15.32:0/131498560 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc380683d0 0x7fcc3806a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc64506000 1 -- start start 2024-04-17T09:56:47.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc59e00640 1 --2- 172.21.15.32:0/131498560 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc380683d0 0x7fcc3806a890 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c0ad000 tx=0x7fcc4c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc64506000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555fb21ea280 0x555fb21ea660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc64506000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555fb21eaba0 0x555fb21a0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc64506000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21a0ac0 0x555fb21a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc64506000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555fb2087f10 con 0x555fb21a0ac0 2024-04-17T09:56:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc64506000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555fb1fed350 con 0x555fb21eaba0 2024-04-17T09:56:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc64506000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x555fb1fa6a40 con 0x555fb21ea280 2024-04-17T09:56:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc5a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21a0ac0 0x555fb21a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc5a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21a0ac0 0x555fb21a0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35068/0 (socket says 172.21.15.32:35068) 2024-04-17T09:56:47.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc5a800640 1 -- 172.21.15.32:0/2972403767 learned_addr learned my addr 172.21.15.32:0/2972403767 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:47.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc59e00640 1 --2- 172.21.15.32:0/2972403767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555fb21ea280 0x555fb21ea660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc51400640 1 --2- 172.21.15.32:0/2972403767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555fb21eaba0 0x555fb21a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc5a800640 1 -- 172.21.15.32:0/2972403767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555fb21ea280 msgr2=0x555fb21ea660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc5a800640 1 --2- 172.21.15.32:0/2972403767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555fb21ea280 0x555fb21ea660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc5a800640 1 -- 172.21.15.32:0/2972403767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555fb21eaba0 msgr2=0x555fb21a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc5a800640 1 --2- 172.21.15.32:0/2972403767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555fb21eaba0 0x555fb21a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc5a800640 1 -- 172.21.15.32:0/2972403767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fb1f70660 con 0x555fb21a0ac0 2024-04-17T09:56:47.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc5a800640 1 --2- 172.21.15.32:0/2972403767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21a0ac0 0x555fb21a0ea0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fcc540021d0 tx=0x7fcc54002200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117b16d58a442dcc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:47.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc4aa00640 1 -- 172.21.15.32:0/2972403767 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54018070 con 0x555fb21a0ac0 2024-04-17T09:56:47.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc4aa00640 1 -- 172.21.15.32:0/2972403767 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc54013670 con 0x555fb21a0ac0 2024-04-17T09:56:47.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.164+0000 7fcc4aa00640 1 -- 172.21.15.32:0/2972403767 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54013a20 con 0x555fb21a0ac0 2024-04-17T09:56:47.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc64506000 1 -- 172.21.15.32:0/2972403767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21a0ac0 msgr2=0x555fb21a0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc64506000 1 --2- 172.21.15.32:0/2972403767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21a0ac0 0x555fb21a0ea0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fcc540021d0 tx=0x7fcc54002200 comp rx=0 tx=0).stop 2024-04-17T09:56:47.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc64506000 1 -- 172.21.15.32:0/2972403767 shutdown_connections 2024-04-17T09:56:47.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc64506000 1 --2- 172.21.15.32:0/2972403767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21a0ac0 0x555fb21a0ea0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc64506000 1 --2- 172.21.15.32:0/2972403767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555fb21eaba0 0x555fb21a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc64506000 1 --2- 172.21.15.32:0/2972403767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555fb21ea280 0x555fb21ea660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc64506000 1 -- 172.21.15.32:0/2972403767 >> 172.21.15.32:0/2972403767 conn(0x555fb219ca70 msgr2=0x555fb219ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:47.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc64506000 1 -- 172.21.15.32:0/2972403767 shutdown_connections 2024-04-17T09:56:47.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc64506000 1 -- 172.21.15.32:0/2972403767 wait complete. 2024-04-17T09:56:47.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc64506000 1 -- start start 2024-04-17T09:56:47.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc64506000 1 Processor -- start 2024-04-17T09:56:47.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc64506000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555fb21a0ac0 0x555fb21f15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc64506000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21ea280 0x555fb21f1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc64506000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555fb21eaba0 0x555fb21ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc64506000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555fb1fa87c0 con 0x555fb21ea280 2024-04-17T09:56:47.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc64506000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555fb1f83730 con 0x555fb21eaba0 2024-04-17T09:56:47.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc64506000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x555fb1faf2e0 con 0x555fb21a0ac0 2024-04-17T09:56:47.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc64506000 10 client.0 _ll_register_callbacks cb 0x555fb218b740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:56:47.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc51400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21ea280 0x555fb21f1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc51400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21ea280 0x555fb21f1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35076/0 (socket says 172.21.15.32:35076) 2024-04-17T09:56:47.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc51400640 1 -- 172.21.15.32:0/3846034782 learned_addr learned my addr 172.21.15.32:0/3846034782 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:47.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc59e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555fb21a0ac0 0x555fb21f15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc5a800640 1 --2- 172.21.15.32:0/3846034782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555fb21eaba0 0x555fb21ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc51400640 1 -- 172.21.15.32:0/3846034782 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555fb21a0ac0 msgr2=0x555fb21f15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc51400640 1 --2- 172.21.15.32:0/3846034782 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555fb21a0ac0 0x555fb21f15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc51400640 1 -- 172.21.15.32:0/3846034782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555fb21eaba0 msgr2=0x555fb21ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc51400640 1 --2- 172.21.15.32:0/3846034782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555fb21eaba0 0x555fb21ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc51400640 1 -- 172.21.15.32:0/3846034782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fb1fb08f0 con 0x555fb21ea280 2024-04-17T09:56:47.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc51400640 1 --2- 172.21.15.32:0/3846034782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21ea280 0x555fb21f1b20 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fcc44001070 tx=0x7fcc4405a220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108309d6d3ea2fbd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:47.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc3f400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:56:47.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc3f400640 1 -- 172.21.15.32:0/3846034782 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44082060 con 0x555fb21ea280 2024-04-17T09:56:47.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc3f400640 1 -- 172.21.15.32:0/3846034782 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fcc44055200 con 0x555fb21ea280 2024-04-17T09:56:47.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.168+0000 7fcc64506000 1 -- 172.21.15.32:0/3846034782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc54013670 con 0x555fb21ea280 2024-04-17T09:56:47.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc3f400640 1 -- 172.21.15.32:0/3846034782 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcc44073050 con 0x555fb21ea280 2024-04-17T09:56:47.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc64506000 10 client.6525 fetch_fsmap learned FSMap version 33 2024-04-17T09:56:47.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc64506000 1 -- 172.21.15.32:0/3846034782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x555fb1ff9cd0 con 0x555fb21ea280 2024-04-17T09:56:47.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc3f400640 1 -- 172.21.15.32:0/3846034782 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc54013670 con 0x555fb21ea280 2024-04-17T09:56:47.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc3f400640 1 -- 172.21.15.32:0/3846034782 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 33) ==== 3495+0+0 (secure 0 0 0) 0x7fcc4405c040 con 0x555fb21ea280 2024-04-17T09:56:47.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc64506000 10 client.6525 fetch_fsmap finished waiting for FSMap version 33 2024-04-17T09:56:47.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc64506000 10 client.6525 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-17T09:56:47.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc64506000 20 client.6525 populate_metadata read hostname 'smithi032' 2024-04-17T09:56:47.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc64506000 1 --2- 172.21.15.32:0/3846034782 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x555fb21cedc0 0x555fb21cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc64506000 4 client.6525 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:56:47.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc64506000 1 -- 172.21.15.32:0/3846034782 --> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fb1f86b40 con 0x555fb21cedc0 2024-04-17T09:56:47.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc59e00640 1 --2- 172.21.15.32:0/3846034782 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x555fb21cedc0 0x555fb21cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc59e00640 1 --2- 172.21.15.32:0/3846034782 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x555fb21cedc0 0x555fb21cf1a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f48927a0def1698 server_cookie=89d65c2f09250b84 in_seq=0 out_seq=0 2024-04-17T09:56:47.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc3f400640 10 client.6525 ms_handle_connect on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:47.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc3f400640 1 -- 172.21.15.32:0/3846034782 <== mds.0 v2:172.21.15.32:6835/866826055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555fb1f86b40 con 0x555fb21cedc0 2024-04-17T09:56:47.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc3f400640 10 client.6525 handle_command_reply: tid=0 2024-04-17T09:56:47.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc64506000 1 client.6525 shutdown 2024-04-17T09:56:47.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc64506000 1 -- 172.21.15.32:0/3846034782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21ea280 msgr2=0x555fb21f1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc64506000 1 --2- 172.21.15.32:0/3846034782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21ea280 0x555fb21f1b20 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fcc44001070 tx=0x7fcc4405a220 comp rx=0 tx=0).stop 2024-04-17T09:56:47.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc64506000 1 -- 172.21.15.32:0/3846034782 shutdown_connections 2024-04-17T09:56:47.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc64506000 1 --2- 172.21.15.32:0/3846034782 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x555fb21cedc0 0x555fb21cf1a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc64506000 1 --2- 172.21.15.32:0/3846034782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555fb21eaba0 0x555fb21ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc64506000 1 --2- 172.21.15.32:0/3846034782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21ea280 0x555fb21f1b20 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc64506000 1 --2- 172.21.15.32:0/3846034782 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555fb21a0ac0 0x555fb21f15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc3f400640 0 client.6525 ms_handle_reset on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:47.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc64506000 1 -- 172.21.15.32:0/3846034782 >> 172.21.15.32:0/3846034782 conn(0x555fb219ca70 msgr2=0x555fb219f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:47.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc64506000 1 -- 172.21.15.32:0/3846034782 shutdown_connections 2024-04-17T09:56:47.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc64506000 1 -- 172.21.15.32:0/3846034782 wait complete. 2024-04-17T09:56:47.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.172+0000 7fcc64506000 20 client.6525 trim_cache size 0 max 16384 2024-04-17T09:56:47.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc64506000 1 Processor -- start 2024-04-17T09:56:47.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc64506000 1 -- start start 2024-04-17T09:56:47.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc64506000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555fb21a0ac0 0x555fb21a0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc64506000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21ea280 0x555fb21f7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc64506000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555fb21eaba0 0x555fb21f7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc64506000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c2073b0 con 0x555fb21ea280 2024-04-17T09:56:47.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc64506000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c1ffe10 con 0x555fb21a0ac0 2024-04-17T09:56:47.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc64506000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcc5c143880 con 0x555fb21eaba0 2024-04-17T09:56:47.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc51400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21ea280 0x555fb21f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc51400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21ea280 0x555fb21f7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35080/0 (socket says 172.21.15.32:35080) 2024-04-17T09:56:47.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc51400640 1 -- 172.21.15.32:0/3444548944 learned_addr learned my addr 172.21.15.32:0/3444548944 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:47.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc59e00640 1 --2- 172.21.15.32:0/3444548944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555fb21a0ac0 0x555fb21a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc5a800640 1 --2- 172.21.15.32:0/3444548944 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555fb21eaba0 0x555fb21f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc51400640 1 -- 172.21.15.32:0/3444548944 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555fb21eaba0 msgr2=0x555fb21f7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc51400640 1 --2- 172.21.15.32:0/3444548944 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555fb21eaba0 0x555fb21f7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc51400640 1 -- 172.21.15.32:0/3444548944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555fb21a0ac0 msgr2=0x555fb21a0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc51400640 1 --2- 172.21.15.32:0/3444548944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555fb21a0ac0 0x555fb21a0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc51400640 1 -- 172.21.15.32:0/3444548944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fb21a3800 con 0x555fb21ea280 2024-04-17T09:56:47.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc5a800640 1 --2- 172.21.15.32:0/3444548944 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555fb21eaba0 0x555fb21f7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:47.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc51400640 1 --2- 172.21.15.32:0/3444548944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21ea280 0x555fb21f7430 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400c3d0 tx=0x7fcc4406c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb576da1e462642f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:47.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc4a000640 1 -- 172.21.15.32:0/3444548944 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc440558f0 con 0x555fb21ea280 2024-04-17T09:56:47.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc4a000640 1 -- 172.21.15.32:0/3444548944 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc4406e030 con 0x555fb21ea280 2024-04-17T09:56:47.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc4a000640 1 -- 172.21.15.32:0/3444548944 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4405c360 con 0x555fb21ea280 2024-04-17T09:56:47.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc64506000 1 -- 172.21.15.32:0/3444548944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21ea280 msgr2=0x555fb21f7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc64506000 1 --2- 172.21.15.32:0/3444548944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21ea280 0x555fb21f7430 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400c3d0 tx=0x7fcc4406c000 comp rx=0 tx=0).stop 2024-04-17T09:56:47.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc64506000 1 -- 172.21.15.32:0/3444548944 shutdown_connections 2024-04-17T09:56:47.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc64506000 1 --2- 172.21.15.32:0/3444548944 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555fb21eaba0 0x555fb21f7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc64506000 1 --2- 172.21.15.32:0/3444548944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21ea280 0x555fb21f7430 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc64506000 1 --2- 172.21.15.32:0/3444548944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555fb21a0ac0 0x555fb21a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc64506000 1 -- 172.21.15.32:0/3444548944 >> 172.21.15.32:0/3444548944 conn(0x555fb219ca70 msgr2=0x555fb21e9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:47.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc64506000 1 -- 172.21.15.32:0/3444548944 shutdown_connections 2024-04-17T09:56:47.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc64506000 1 -- 172.21.15.32:0/3444548944 wait complete. 2024-04-17T09:56:47.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.192+0000 7fcc64506000 1 -- start start 2024-04-17T09:56:47.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc64506000 1 Processor -- start 2024-04-17T09:56:47.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc64506000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555fb21a0ac0 0x555fb22597c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc64506000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21ea280 0x555fb2259d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc64506000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555fb21eaba0 0x555fb21edfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc64506000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555fb20a79b0 con 0x555fb21ea280 2024-04-17T09:56:47.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc64506000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555fb20a8340 con 0x555fb21a0ac0 2024-04-17T09:56:47.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc64506000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x555fb1ff8280 con 0x555fb21eaba0 2024-04-17T09:56:47.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc64506000 10 client.0 _ll_register_callbacks cb 0x7fcc44055db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:56:47.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc51400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21ea280 0x555fb2259d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc51400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21ea280 0x555fb2259d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35086/0 (socket says 172.21.15.32:35086) 2024-04-17T09:56:47.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc51400640 1 -- 172.21.15.32:0/1246437311 learned_addr learned my addr 172.21.15.32:0/1246437311 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:47.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc59e00640 1 --2- 172.21.15.32:0/1246437311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555fb21a0ac0 0x555fb22597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc5a800640 1 --2- 172.21.15.32:0/1246437311 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555fb21eaba0 0x555fb21edfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc51400640 1 -- 172.21.15.32:0/1246437311 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555fb21eaba0 msgr2=0x555fb21edfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc51400640 1 --2- 172.21.15.32:0/1246437311 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555fb21eaba0 0x555fb21edfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc51400640 1 -- 172.21.15.32:0/1246437311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555fb21a0ac0 msgr2=0x555fb22597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc51400640 1 --2- 172.21.15.32:0/1246437311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555fb21a0ac0 0x555fb22597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc51400640 1 -- 172.21.15.32:0/1246437311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fb206baa0 con 0x555fb21ea280 2024-04-17T09:56:47.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc51400640 1 --2- 172.21.15.32:0/1246437311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21ea280 0x555fb2259d00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fcc44084420 tx=0x7fcc440842f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2962df5f8f0ef7c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:47.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc3f400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:56:47.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc3f400640 1 -- 172.21.15.32:0/1246437311 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44079020 con 0x555fb21ea280 2024-04-17T09:56:47.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc3f400640 1 -- 172.21.15.32:0/1246437311 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fcc44004d00 con 0x555fb21ea280 2024-04-17T09:56:47.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc64506000 1 -- 172.21.15.32:0/1246437311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc4406e030 con 0x555fb21ea280 2024-04-17T09:56:47.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc3f400640 1 -- 172.21.15.32:0/1246437311 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcc440558f0 con 0x555fb21ea280 2024-04-17T09:56:47.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc64506000 10 client.6531 fetch_fsmap learned FSMap version 33 2024-04-17T09:56:47.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc64506000 1 -- 172.21.15.32:0/1246437311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x555fb1fb2590 con 0x555fb21ea280 2024-04-17T09:56:47.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc3f400640 1 -- 172.21.15.32:0/1246437311 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc4406e030 con 0x555fb21ea280 2024-04-17T09:56:47.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc3f400640 1 -- 172.21.15.32:0/1246437311 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 33) ==== 3495+0+0 (secure 0 0 0) 0x7fcc4405c040 con 0x555fb21ea280 2024-04-17T09:56:47.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc64506000 10 client.6531 fetch_fsmap finished waiting for FSMap version 33 2024-04-17T09:56:47.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc64506000 10 client.6531 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-17T09:56:47.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc64506000 20 client.6531 populate_metadata read hostname 'smithi032' 2024-04-17T09:56:47.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc64506000 1 --2- 172.21.15.32:0/1246437311 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x555fb21eea00 0x555fb21eee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc64506000 4 client.6531 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-04-17T09:56:47.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc64506000 1 -- 172.21.15.32:0/1246437311 --> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) -- 0x555fb1ffad70 con 0x555fb21eea00 2024-04-17T09:56:47.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc59e00640 1 --2- 172.21.15.32:0/1246437311 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x555fb21eea00 0x555fb21eee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc3f400640 10 client.6531 ms_handle_connect on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:47.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.196+0000 7fcc59e00640 1 --2- 172.21.15.32:0/1246437311 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x555fb21eea00 0x555fb21eee10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b01c3f775ca740e1 server_cookie=9ce624899f3b9a60 in_seq=0 out_seq=0 2024-04-17T09:56:47.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc3f400640 1 -- 172.21.15.32:0/1246437311 <== mds.0 v2:172.21.15.32:6835/866826055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555fb1ffad70 con 0x555fb21eea00 2024-04-17T09:56:47.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc3f400640 10 client.6531 handle_command_reply: tid=0 2024-04-17T09:56:47.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc64506000 1 client.6531 shutdown 2024-04-17T09:56:47.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc64506000 1 -- 172.21.15.32:0/1246437311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21ea280 msgr2=0x555fb2259d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc64506000 1 --2- 172.21.15.32:0/1246437311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21ea280 0x555fb2259d00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fcc44084420 tx=0x7fcc440842f0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc64506000 1 -- 172.21.15.32:0/1246437311 shutdown_connections 2024-04-17T09:56:47.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc64506000 1 --2- 172.21.15.32:0/1246437311 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555fb21eaba0 0x555fb21edfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc64506000 1 --2- 172.21.15.32:0/1246437311 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x555fb21eea00 0x555fb21eee10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc64506000 1 --2- 172.21.15.32:0/1246437311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555fb21ea280 0x555fb2259d00 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc64506000 1 --2- 172.21.15.32:0/1246437311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555fb21a0ac0 0x555fb22597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc3f400640 0 client.6531 ms_handle_reset on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:47.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc64506000 1 -- 172.21.15.32:0/1246437311 >> 172.21.15.32:0/1246437311 conn(0x555fb219ca70 msgr2=0x555fb21e9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:47.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc64506000 1 -- 172.21.15.32:0/1246437311 shutdown_connections 2024-04-17T09:56:47.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc64506000 1 -- 172.21.15.32:0/1246437311 wait complete. 2024-04-17T09:56:47.206 INFO:tasks.ceph.mds.a.smithi032.stderr:2024-04-17T09:56:47.200+0000 7f7c90000640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-04-17T09:56:14.409449+0000 1=1+0) caps={6396=pAsLsXsFs/-@2},l=6396(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x5570aa0c0b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-17T09:56:47.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc64506000 20 client.6531 trim_cache size 0 max 16384 2024-04-17T09:56:47.207 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:56:47.207 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-17T09:56:47.207 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "750f9c62-da7b-4bca-aa11-a73c312793b4", 2024-04-17T09:56:47.207 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-17T09:56:47.207 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:56:47.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc61000640 1 -- 172.21.15.32:0/131498560 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc380683d0 msgr2=0x7fcc3806a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc61000640 1 --2- 172.21.15.32:0/131498560 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc380683d0 0x7fcc3806a890 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c0ad000 tx=0x7fcc4c008040 comp rx=0 tx=0).stop 2024-04-17T09:56:47.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc61000640 1 -- 172.21.15.32:0/131498560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc5c0724c0 msgr2=0x7fcc5c15bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc61000640 1 --2- 172.21.15.32:0/131498560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc5c0724c0 0x7fcc5c15bb30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400c400 tx=0x7fcc4400c8c0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc61000640 1 -- 172.21.15.32:0/131498560 shutdown_connections 2024-04-17T09:56:47.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc61000640 1 --2- 172.21.15.32:0/131498560 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc380683d0 0x7fcc3806a890 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc61000640 1 --2- 172.21.15.32:0/131498560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc5c1460a0 0x7fcc5c15c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc61000640 1 --2- 172.21.15.32:0/131498560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc5c0724c0 0x7fcc5c15bb30 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc61000640 1 --2- 172.21.15.32:0/131498560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc5c06e6f0 0x7fcc5c13cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc61000640 1 -- 172.21.15.32:0/131498560 >> 172.21.15.32:0/131498560 conn(0x7fcc5c0b2190 msgr2=0x7fcc5c15e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:47.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc61000640 1 -- 172.21.15.32:0/131498560 shutdown_connections 2024-04-17T09:56:47.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.200+0000 7fcc61000640 1 -- 172.21.15.32:0/131498560 wait complete. 2024-04-17T09:56:47.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:56:47.221 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-17T09:56:47.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.364+0000 7f312b000640 1 Processor -- start 2024-04-17T09:56:47.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.364+0000 7f312b000640 1 -- start start 2024-04-17T09:56:47.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.364+0000 7f312b000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f312406cdc0 0x7f312409a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.364+0000 7f312b000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31241604b0 0x7f312409af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.364+0000 7f312b000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31240d0b30 con 0x7f3124161ec0 2024-04-17T09:56:47.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.364+0000 7f312b000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f3124162fb0 con 0x7f312406cdc0 2024-04-17T09:56:47.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.364+0000 7f312b000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3124163290 con 0x7f31241604b0 2024-04-17T09:56:47.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.364+0000 7f3129000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31241604b0 0x7f312409af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.364+0000 7f311b400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f312406cdc0 0x7f312409a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.364+0000 7f3123e00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f3124161ec0 0x7f31241626d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:40892/0 (socket says 172.21.15.32:40892) 2024-04-17T09:56:47.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.364+0000 7f3123e00640 1 -- 172.21.15.32:0/2849757626 learned_addr learned my addr 172.21.15.32:0/2849757626 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:47.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.364+0000 7f3129000640 1 -- 172.21.15.32:0/2849757626 >> v2:172.21.15.144:3300/0 conn(0x7f312406cdc0 msgr2=0x7f312409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.364+0000 7f3129000640 1 --2- 172.21.15.32:0/2849757626 >> v2:172.21.15.144:3300/0 conn(0x7f312406cdc0 0x7f312409a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.364+0000 7f3129000640 1 -- 172.21.15.32:0/2849757626 >> v1:172.21.15.144:6789/0 conn(0x7f3124161ec0 legacy=0x7f31241626d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.364+0000 7f3129000640 1 -- 172.21.15.32:0/2849757626 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31240d0da0 con 0x7f31241604b0 2024-04-17T09:56:47.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.364+0000 7f311b400640 1 --2- 172.21.15.32:0/2849757626 >> v2:172.21.15.144:3300/0 conn(0x7f312406cdc0 0x7f312409a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:56:47.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.368+0000 7f3129000640 1 --2- 172.21.15.32:0/2849757626 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31241604b0 0x7f312409af30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f311c009d90 tx=0x7f311c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d250fce2bcd4aad7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.368+0000 7f3123400640 1 -- 172.21.15.32:0/2849757626 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c0b2070 con 0x7f31241604b0 2024-04-17T09:56:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.368+0000 7f3123400640 1 -- 172.21.15.32:0/2849757626 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31241624b0 con 0x7f31241604b0 2024-04-17T09:56:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.368+0000 7f3123400640 1 -- 172.21.15.32:0/2849757626 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c0a5b60 con 0x7f31241604b0 2024-04-17T09:56:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.368+0000 7f312b000640 1 -- 172.21.15.32:0/2849757626 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31241604b0 msgr2=0x7f312409af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.368+0000 7f312b000640 1 --2- 172.21.15.32:0/2849757626 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31241604b0 0x7f312409af30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f311c009d90 tx=0x7f311c01c970 comp rx=0 tx=0).stop 2024-04-17T09:56:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.368+0000 7f312b000640 1 -- 172.21.15.32:0/2849757626 shutdown_connections 2024-04-17T09:56:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.368+0000 7f312b000640 1 --2- 172.21.15.32:0/2849757626 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31241604b0 0x7f312409af30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.368+0000 7f312b000640 1 --2- 172.21.15.32:0/2849757626 >> v2:172.21.15.144:3300/0 conn(0x7f312406cdc0 0x7f312409a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.368+0000 7f312b000640 1 -- 172.21.15.32:0/2849757626 >> 172.21.15.32:0/2849757626 conn(0x7f31240b4110 msgr2=0x7f31240b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.368+0000 7f312b000640 1 -- 172.21.15.32:0/2849757626 shutdown_connections 2024-04-17T09:56:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.368+0000 7f312b000640 1 -- 172.21.15.32:0/2849757626 wait complete. 2024-04-17T09:56:47.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f312b000640 1 Processor -- start 2024-04-17T09:56:47.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f312b000640 1 -- start start 2024-04-17T09:56:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f312b000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f312406cdc0 0x7f3124157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f312b000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31241604b0 0x7f31240b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f312b000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3124161ec0 0x7f3124159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f312b000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f312415fe40 con 0x7f3124161ec0 2024-04-17T09:56:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f312b000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f312415fcc0 con 0x7f312406cdc0 2024-04-17T09:56:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f312b000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f312415ffc0 con 0x7f31241604b0 2024-04-17T09:56:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f3129000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3124161ec0 0x7f3124159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f3129000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3124161ec0 0x7f3124159e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35090/0 (socket says 172.21.15.32:35090) 2024-04-17T09:56:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f3129000640 1 -- 172.21.15.32:0/1846458960 learned_addr learned my addr 172.21.15.32:0/1846458960 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f3123e00640 1 --2- 172.21.15.32:0/1846458960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f312406cdc0 0x7f3124157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f311b400640 1 --2- 172.21.15.32:0/1846458960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31241604b0 0x7f31240b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f3129000640 1 -- 172.21.15.32:0/1846458960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31241604b0 msgr2=0x7f31240b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f3129000640 1 --2- 172.21.15.32:0/1846458960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31241604b0 0x7f31240b80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f3129000640 1 -- 172.21.15.32:0/1846458960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f312406cdc0 msgr2=0x7f3124157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f3129000640 1 --2- 172.21.15.32:0/1846458960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f312406cdc0 0x7f3124157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f3129000640 1 -- 172.21.15.32:0/1846458960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312415a650 con 0x7f3124161ec0 2024-04-17T09:56:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f311b400640 1 --2- 172.21.15.32:0/1846458960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31241604b0 0x7f31240b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f3129000640 1 --2- 172.21.15.32:0/1846458960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3124161ec0 0x7f3124159e00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f311c0b0040 tx=0x7f311c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f3120c00640 1 -- 172.21.15.32:0/1846458960 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c0b2050 con 0x7f3124161ec0 2024-04-17T09:56:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f3120c00640 1 -- 172.21.15.32:0/1846458960 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f311c0be070 con 0x7f3124161ec0 2024-04-17T09:56:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f3120c00640 1 -- 172.21.15.32:0/1846458960 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c009450 con 0x7f3124161ec0 2024-04-17T09:56:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f312b000640 1 -- 172.21.15.32:0/1846458960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f312415a930 con 0x7f3124161ec0 2024-04-17T09:56:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.372+0000 7f312b000640 1 -- 172.21.15.32:0/1846458960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f312415aef0 con 0x7f3124161ec0 2024-04-17T09:56:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.376+0000 7f3120c00640 1 -- 172.21.15.32:0/1846458960 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f311c0ada90 con 0x7f3124161ec0 2024-04-17T09:56:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.376+0000 7f3120c00640 1 --2- 172.21.15.32:0/1846458960 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3100068380 0x7f310006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.376+0000 7f3120c00640 1 -- 172.21.15.32:0/1846458960 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f311c121b70 con 0x7f3124161ec0 2024-04-17T09:56:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.376+0000 7f3123e00640 1 --2- 172.21.15.32:0/1846458960 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3100068380 0x7f310006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.376+0000 7f312e3e8000 1 Processor -- start 2024-04-17T09:56:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.376+0000 7f312e3e8000 1 -- start start 2024-04-17T09:56:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.376+0000 7f3123e00640 1 --2- 172.21.15.32:0/1846458960 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3100068380 0x7f310006a840 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3114000f70 tx=0x7f3114007e90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f312e3e8000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55817375c260 0x55817375c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f312e3e8000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817375cb80 0x55817370f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f312e3e8000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817370f9a0 0x55817370fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f312e3e8000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5581735614f0 con 0x55817375cb80 2024-04-17T09:56:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f312e3e8000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55817351a6c0 con 0x55817370f9a0 2024-04-17T09:56:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f312e3e8000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55817355d4b0 con 0x55817375c260 2024-04-17T09:56:47.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f311b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817375cb80 0x55817370f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f311b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817375cb80 0x55817370f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35092/0 (socket says 172.21.15.32:35092) 2024-04-17T09:56:47.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f311b400640 1 -- 172.21.15.32:0/4119952950 learned_addr learned my addr 172.21.15.32:0/4119952950 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:47.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f3123e00640 1 --2- 172.21.15.32:0/4119952950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55817375c260 0x55817375c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f3129000640 1 --2- 172.21.15.32:0/4119952950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817370f9a0 0x55817370fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f311b400640 1 -- 172.21.15.32:0/4119952950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55817375c260 msgr2=0x55817375c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f311b400640 1 --2- 172.21.15.32:0/4119952950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55817375c260 0x55817375c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f311b400640 1 -- 172.21.15.32:0/4119952950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817370f9a0 msgr2=0x55817370fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f311b400640 1 --2- 172.21.15.32:0/4119952950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817370f9a0 0x55817370fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f311b400640 1 -- 172.21.15.32:0/4119952950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581735a68e0 con 0x55817375cb80 2024-04-17T09:56:47.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f3123e00640 1 --2- 172.21.15.32:0/4119952950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55817375c260 0x55817375c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:47.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f311b400640 1 --2- 172.21.15.32:0/4119952950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817375cb80 0x55817370f460 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f3124159d90 tx=0x7f310c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4c20e3ecd9a164 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:47.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f3118c00640 1 -- 172.21.15.32:0/4119952950 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c019070 con 0x55817375cb80 2024-04-17T09:56:47.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f3118c00640 1 -- 172.21.15.32:0/4119952950 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f310c011ba0 con 0x55817375cb80 2024-04-17T09:56:47.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f3118c00640 1 -- 172.21.15.32:0/4119952950 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c015460 con 0x55817375cb80 2024-04-17T09:56:47.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f312e3e8000 1 -- 172.21.15.32:0/4119952950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817375cb80 msgr2=0x55817370f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f312e3e8000 1 --2- 172.21.15.32:0/4119952950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817375cb80 0x55817370f460 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f3124159d90 tx=0x7f310c009190 comp rx=0 tx=0).stop 2024-04-17T09:56:47.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f312e3e8000 1 -- 172.21.15.32:0/4119952950 shutdown_connections 2024-04-17T09:56:47.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f312e3e8000 1 --2- 172.21.15.32:0/4119952950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817370f9a0 0x55817370fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f312e3e8000 1 --2- 172.21.15.32:0/4119952950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817375cb80 0x55817370f460 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f312e3e8000 1 --2- 172.21.15.32:0/4119952950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55817375c260 0x55817375c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f312e3e8000 1 -- 172.21.15.32:0/4119952950 >> 172.21.15.32:0/4119952950 conn(0x55817373d000 msgr2=0x55817373f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:47.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f312e3e8000 1 -- 172.21.15.32:0/4119952950 shutdown_connections 2024-04-17T09:56:47.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f312e3e8000 1 -- 172.21.15.32:0/4119952950 wait complete. 2024-04-17T09:56:47.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f312e3e8000 1 -- start start 2024-04-17T09:56:47.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.380+0000 7f312e3e8000 1 Processor -- start 2024-04-17T09:56:47.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f312e3e8000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817370f9a0 0x558173715a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f312e3e8000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817375c260 0x558173715f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f312e3e8000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55817375cb80 0x5581737164d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f312e3e8000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5581735e27b0 con 0x55817370f9a0 2024-04-17T09:56:47.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f312e3e8000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55817361cd00 con 0x55817375c260 2024-04-17T09:56:47.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f312e3e8000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5581735262c0 con 0x55817375cb80 2024-04-17T09:56:47.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f312e3e8000 10 client.0 _ll_register_callbacks cb 0x558173536030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:56:47.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f3123e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817370f9a0 0x558173715a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f3123e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817370f9a0 0x558173715a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35108/0 (socket says 172.21.15.32:35108) 2024-04-17T09:56:47.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f3123e00640 1 -- 172.21.15.32:0/265090676 learned_addr learned my addr 172.21.15.32:0/265090676 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:47.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f311b400640 1 --2- 172.21.15.32:0/265090676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817375c260 0x558173715f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f3129000640 1 --2- 172.21.15.32:0/265090676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55817375cb80 0x5581737164d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f3123e00640 1 -- 172.21.15.32:0/265090676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55817375cb80 msgr2=0x5581737164d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f3123e00640 1 --2- 172.21.15.32:0/265090676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55817375cb80 0x5581737164d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f3123e00640 1 -- 172.21.15.32:0/265090676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817375c260 msgr2=0x558173715f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f3123e00640 1 --2- 172.21.15.32:0/265090676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817375c260 0x558173715f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f3123e00640 1 -- 172.21.15.32:0/265090676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558173539490 con 0x55817370f9a0 2024-04-17T09:56:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f3123e00640 1 --2- 172.21.15.32:0/265090676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817370f9a0 0x558173715a50 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f3114014450 tx=0x7f3114000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146df1e7aa91ba06 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f3111600640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:56:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f3111600640 1 -- 172.21.15.32:0/265090676 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31140135c0 con 0x55817370f9a0 2024-04-17T09:56:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f3111600640 1 -- 172.21.15.32:0/265090676 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f3114013760 con 0x55817370f9a0 2024-04-17T09:56:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f312e3e8000 1 -- 172.21.15.32:0/265090676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f310c011ba0 con 0x55817370f9a0 2024-04-17T09:56:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f3111600640 1 -- 172.21.15.32:0/265090676 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f311402e050 con 0x55817370f9a0 2024-04-17T09:56:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f312e3e8000 10 client.6540 fetch_fsmap learned FSMap version 33 2024-04-17T09:56:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f312e3e8000 1 -- 172.21.15.32:0/265090676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55817352ceb0 con 0x55817370f9a0 2024-04-17T09:56:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f3111600640 1 -- 172.21.15.32:0/265090676 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f310c011ba0 con 0x55817370f9a0 2024-04-17T09:56:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f3111600640 1 -- 172.21.15.32:0/265090676 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 33) ==== 3495+0+0 (secure 0 0 0) 0x7f3114031020 con 0x55817370f9a0 2024-04-17T09:56:47.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f312e3e8000 10 client.6540 fetch_fsmap finished waiting for FSMap version 33 2024-04-17T09:56:47.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f312e3e8000 10 client.6540 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-17T09:56:47.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f312e3e8000 20 client.6540 populate_metadata read hostname 'smithi032' 2024-04-17T09:56:47.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f312e3e8000 1 --2- 172.21.15.32:0/265090676 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x558173716ee0 0x55817376e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f312e3e8000 4 client.6540 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:56:47.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f312e3e8000 1 -- 172.21.15.32:0/265090676 --> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581734fadd0 con 0x558173716ee0 2024-04-17T09:56:47.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f311b400640 1 --2- 172.21.15.32:0/265090676 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x558173716ee0 0x55817376e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f311b400640 1 --2- 172.21.15.32:0/265090676 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x558173716ee0 0x55817376e520 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e96aa105093bdc02 server_cookie=47584e41cf937336 in_seq=0 out_seq=0 2024-04-17T09:56:47.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.384+0000 7f3111600640 10 client.6540 ms_handle_connect on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:47.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.388+0000 7f3111600640 1 -- 172.21.15.32:0/265090676 <== mds.0 v2:172.21.15.32:6835/866826055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581734fadd0 con 0x558173716ee0 2024-04-17T09:56:47.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.388+0000 7f3111600640 10 client.6540 handle_command_reply: tid=0 2024-04-17T09:56:47.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.388+0000 7f312e3e8000 1 client.6540 shutdown 2024-04-17T09:56:47.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.388+0000 7f312e3e8000 1 -- 172.21.15.32:0/265090676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817370f9a0 msgr2=0x558173715a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.388+0000 7f312e3e8000 1 --2- 172.21.15.32:0/265090676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817370f9a0 0x558173715a50 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f3114014450 tx=0x7f3114000ee0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.388+0000 7f312e3e8000 1 -- 172.21.15.32:0/265090676 shutdown_connections 2024-04-17T09:56:47.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.388+0000 7f312e3e8000 1 --2- 172.21.15.32:0/265090676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55817375cb80 0x5581737164d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.388+0000 7f312e3e8000 1 --2- 172.21.15.32:0/265090676 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x558173716ee0 0x55817376e520 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.388+0000 7f312e3e8000 1 --2- 172.21.15.32:0/265090676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817375c260 0x558173715f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.388+0000 7f312e3e8000 1 --2- 172.21.15.32:0/265090676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817370f9a0 0x558173715a50 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.388+0000 7f312e3e8000 1 -- 172.21.15.32:0/265090676 >> 172.21.15.32:0/265090676 conn(0x55817373d000 msgr2=0x55817370eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:47.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.388+0000 7f3111600640 0 client.6540 ms_handle_reset on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:47.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.388+0000 7f312e3e8000 1 -- 172.21.15.32:0/265090676 shutdown_connections 2024-04-17T09:56:47.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.388+0000 7f312e3e8000 1 -- 172.21.15.32:0/265090676 wait complete. 2024-04-17T09:56:47.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.388+0000 7f312e3e8000 20 client.6540 trim_cache size 0 max 16384 2024-04-17T09:56:47.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.404+0000 7f312e3e8000 1 Processor -- start 2024-04-17T09:56:47.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.404+0000 7f312e3e8000 1 -- start start 2024-04-17T09:56:47.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.404+0000 7f312e3e8000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817370f9a0 0x55817370ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.404+0000 7f312e3e8000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817375c260 0x55817370f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.404+0000 7f312e3e8000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55817375cb80 0x55817376e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.404+0000 7f312e3e8000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3124154600 con 0x55817370f9a0 2024-04-17T09:56:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.404+0000 7f312e3e8000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3124153960 con 0x55817375c260 2024-04-17T09:56:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.404+0000 7f312e3e8000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3124153fb0 con 0x55817375cb80 2024-04-17T09:56:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.404+0000 7f3123e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817370f9a0 0x55817370ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.404+0000 7f3123e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817370f9a0 0x55817370ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35120/0 (socket says 172.21.15.32:35120) 2024-04-17T09:56:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.404+0000 7f3123e00640 1 -- 172.21.15.32:0/297788375 learned_addr learned my addr 172.21.15.32:0/297788375 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.404+0000 7f311b400640 1 --2- 172.21.15.32:0/297788375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817375c260 0x55817370f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.404+0000 7f3129000640 1 --2- 172.21.15.32:0/297788375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55817375cb80 0x55817376e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.404+0000 7f3123e00640 1 -- 172.21.15.32:0/297788375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55817375cb80 msgr2=0x55817376e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.404+0000 7f3123e00640 1 --2- 172.21.15.32:0/297788375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55817375cb80 0x55817376e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.404+0000 7f3123e00640 1 -- 172.21.15.32:0/297788375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817375c260 msgr2=0x55817370f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.404+0000 7f3123e00640 1 --2- 172.21.15.32:0/297788375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817375c260 0x55817370f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.404+0000 7f3123e00640 1 -- 172.21.15.32:0/297788375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55817351ba20 con 0x55817370f9a0 2024-04-17T09:56:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.404+0000 7f3129000640 1 --2- 172.21.15.32:0/297788375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55817375cb80 0x55817376e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f3123e00640 1 --2- 172.21.15.32:0/297788375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817370f9a0 0x55817370ebd0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f31140018a0 tx=0x7f3114019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cdb2362faae377 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f3113e00640 1 -- 172.21.15.32:0/297788375 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114036070 con 0x55817370f9a0 2024-04-17T09:56:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f3113e00640 1 -- 172.21.15.32:0/297788375 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3114026070 con 0x55817370f9a0 2024-04-17T09:56:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f3113e00640 1 -- 172.21.15.32:0/297788375 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114014ce0 con 0x55817370f9a0 2024-04-17T09:56:47.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f312e3e8000 1 -- 172.21.15.32:0/297788375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817370f9a0 msgr2=0x55817370ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f312e3e8000 1 --2- 172.21.15.32:0/297788375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817370f9a0 0x55817370ebd0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f31140018a0 tx=0x7f3114019000 comp rx=0 tx=0).stop 2024-04-17T09:56:47.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f312e3e8000 1 -- 172.21.15.32:0/297788375 shutdown_connections 2024-04-17T09:56:47.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f312e3e8000 1 --2- 172.21.15.32:0/297788375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55817375cb80 0x55817376e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f312e3e8000 1 --2- 172.21.15.32:0/297788375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817375c260 0x55817370f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f312e3e8000 1 --2- 172.21.15.32:0/297788375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817370f9a0 0x55817370ebd0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f312e3e8000 1 -- 172.21.15.32:0/297788375 >> 172.21.15.32:0/297788375 conn(0x55817373d000 msgr2=0x55817375a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:47.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f312e3e8000 1 -- 172.21.15.32:0/297788375 shutdown_connections 2024-04-17T09:56:47.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f312e3e8000 1 -- 172.21.15.32:0/297788375 wait complete. 2024-04-17T09:56:47.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f312e3e8000 1 -- start start 2024-04-17T09:56:47.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f312e3e8000 1 Processor -- start 2024-04-17T09:56:47.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f312e3e8000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817370f9a0 0x5581737715a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f312e3e8000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55817375c260 0x5581737880b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f312e3e8000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817375cb80 0x5581737885f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f312e3e8000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558173505050 con 0x55817370f9a0 2024-04-17T09:56:47.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f312e3e8000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558173523570 con 0x55817375cb80 2024-04-17T09:56:47.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f312e3e8000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x558173560800 con 0x55817375c260 2024-04-17T09:56:47.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f312e3e8000 10 client.0 _ll_register_callbacks cb 0x7f3114017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:56:47.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f3123e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817370f9a0 0x5581737715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f3123e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817370f9a0 0x5581737715a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35136/0 (socket says 172.21.15.32:35136) 2024-04-17T09:56:47.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f3123e00640 1 -- 172.21.15.32:0/606030744 learned_addr learned my addr 172.21.15.32:0/606030744 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:47.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f311b400640 1 --2- 172.21.15.32:0/606030744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55817375c260 0x5581737880b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f3129000640 1 --2- 172.21.15.32:0/606030744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817375cb80 0x5581737885f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f3123e00640 1 -- 172.21.15.32:0/606030744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55817375c260 msgr2=0x5581737880b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f3123e00640 1 --2- 172.21.15.32:0/606030744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55817375c260 0x5581737880b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f3123e00640 1 -- 172.21.15.32:0/606030744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817375cb80 msgr2=0x5581737885f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f3123e00640 1 --2- 172.21.15.32:0/606030744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817375cb80 0x5581737885f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f3123e00640 1 -- 172.21.15.32:0/606030744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581735e2c40 con 0x55817370f9a0 2024-04-17T09:56:47.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f3129000640 1 --2- 172.21.15.32:0/606030744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817375cb80 0x5581737885f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:47.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f3123e00640 1 --2- 172.21.15.32:0/606030744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817370f9a0 0x5581737715a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f3114006e70 tx=0x7f3114015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943036e282082885 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:47.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.408+0000 7f3111600640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:56:47.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f3111600640 1 -- 172.21.15.32:0/606030744 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114036070 con 0x55817370f9a0 2024-04-17T09:56:47.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f3111600640 1 -- 172.21.15.32:0/606030744 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f3114005860 con 0x55817370f9a0 2024-04-17T09:56:47.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312e3e8000 1 -- 172.21.15.32:0/606030744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3114026070 con 0x55817370f9a0 2024-04-17T09:56:47.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f3111600640 1 -- 172.21.15.32:0/606030744 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3114049050 con 0x55817370f9a0 2024-04-17T09:56:47.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312e3e8000 10 client.6546 fetch_fsmap learned FSMap version 33 2024-04-17T09:56:47.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312e3e8000 1 -- 172.21.15.32:0/606030744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x558173506600 con 0x55817370f9a0 2024-04-17T09:56:47.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f3111600640 1 -- 172.21.15.32:0/606030744 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3114026070 con 0x55817370f9a0 2024-04-17T09:56:47.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f3111600640 1 -- 172.21.15.32:0/606030744 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 33) ==== 3495+0+0 (secure 0 0 0) 0x7f311404c020 con 0x55817370f9a0 2024-04-17T09:56:47.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312e3e8000 10 client.6546 fetch_fsmap finished waiting for FSMap version 33 2024-04-17T09:56:47.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312e3e8000 10 client.6546 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-17T09:56:47.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312e3e8000 20 client.6546 populate_metadata read hostname 'smithi032' 2024-04-17T09:56:47.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312e3e8000 1 --2- 172.21.15.32:0/606030744 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x5581737cbd20 0x5581737cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312e3e8000 4 client.6546 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T09:56:47.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312e3e8000 1 -- 172.21.15.32:0/606030744 --> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55817356ef10 con 0x5581737cbd20 2024-04-17T09:56:47.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f311b400640 1 --2- 172.21.15.32:0/606030744 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x5581737cbd20 0x5581737cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f311b400640 1 --2- 172.21.15.32:0/606030744 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x5581737cbd20 0x5581737cc130 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87cb945fedc85a96 server_cookie=8698cc7d98d1aa62 in_seq=0 out_seq=0 2024-04-17T09:56:47.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f3111600640 10 client.6546 ms_handle_connect on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:47.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f3111600640 1 -- 172.21.15.32:0/606030744 <== mds.0 v2:172.21.15.32:6835/866826055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55817356ef10 con 0x5581737cbd20 2024-04-17T09:56:47.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f3111600640 10 client.6546 handle_command_reply: tid=0 2024-04-17T09:56:47.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312e3e8000 1 client.6546 shutdown 2024-04-17T09:56:47.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312e3e8000 1 -- 172.21.15.32:0/606030744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817370f9a0 msgr2=0x5581737715a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312e3e8000 1 --2- 172.21.15.32:0/606030744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817370f9a0 0x5581737715a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f3114006e70 tx=0x7f3114015000 comp rx=0 tx=0).stop 2024-04-17T09:56:47.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312e3e8000 1 -- 172.21.15.32:0/606030744 shutdown_connections 2024-04-17T09:56:47.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312e3e8000 1 --2- 172.21.15.32:0/606030744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55817375c260 0x5581737880b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312e3e8000 1 --2- 172.21.15.32:0/606030744 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x5581737cbd20 0x5581737cc130 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312e3e8000 1 --2- 172.21.15.32:0/606030744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817375cb80 0x5581737885f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312e3e8000 1 --2- 172.21.15.32:0/606030744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55817370f9a0 0x5581737715a0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f3111600640 0 client.6546 ms_handle_reset on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:47.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312e3e8000 1 -- 172.21.15.32:0/606030744 >> 172.21.15.32:0/606030744 conn(0x55817373d000 msgr2=0x55817370ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:47.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312e3e8000 1 -- 172.21.15.32:0/606030744 shutdown_connections 2024-04-17T09:56:47.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312e3e8000 1 -- 172.21.15.32:0/606030744 wait complete. 2024-04-17T09:56:47.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312e3e8000 20 client.6546 trim_cache size 0 max 16384 2024-04-17T09:56:47.420 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:56:47.420 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-17T09:56:47.420 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-17T09:56:47.420 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:56:47.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312b000640 1 -- 172.21.15.32:0/1846458960 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3100068380 msgr2=0x7f310006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312b000640 1 --2- 172.21.15.32:0/1846458960 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3100068380 0x7f310006a840 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3114000f70 tx=0x7f3114007e90 comp rx=0 tx=0).stop 2024-04-17T09:56:47.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312b000640 1 -- 172.21.15.32:0/1846458960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3124161ec0 msgr2=0x7f3124159e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312b000640 1 --2- 172.21.15.32:0/1846458960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3124161ec0 0x7f3124159e00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f311c0b0040 tx=0x7f311c019710 comp rx=0 tx=0).stop 2024-04-17T09:56:47.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312b000640 1 -- 172.21.15.32:0/1846458960 shutdown_connections 2024-04-17T09:56:47.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312b000640 1 --2- 172.21.15.32:0/1846458960 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3100068380 0x7f310006a840 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312b000640 1 --2- 172.21.15.32:0/1846458960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31241604b0 0x7f31240b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312b000640 1 --2- 172.21.15.32:0/1846458960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3124161ec0 0x7f3124159e00 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312b000640 1 --2- 172.21.15.32:0/1846458960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f312406cdc0 0x7f3124157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312b000640 1 -- 172.21.15.32:0/1846458960 >> 172.21.15.32:0/1846458960 conn(0x7f31240b4110 msgr2=0x7f31240716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312b000640 1 -- 172.21.15.32:0/1846458960 shutdown_connections 2024-04-17T09:56:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.412+0000 7f312b000640 1 -- 172.21.15.32:0/1846458960 wait complete. 2024-04-17T09:56:47.433 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T09:56:47.433 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:56:47.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.572+0000 7f2c0a800640 1 Processor -- start 2024-04-17T09:56:47.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.572+0000 7f2c0a800640 1 -- start start 2024-04-17T09:56:47.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.576+0000 7f2c0a800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2c04166ef0 0x7f2c04072260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.576+0000 7f2c0a800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c041673f0 con 0x7f2c04162bb0 2024-04-17T09:56:47.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.576+0000 7f2c0a800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c041676d0 con 0x7f2c04071df0 2024-04-17T09:56:47.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.576+0000 7f2c0a800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2c040727a0 con 0x7f2c04166ef0 2024-04-17T09:56:47.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.576+0000 7f2c03400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2c04162bb0 0x7f2c0406e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37080/0 (socket says 172.21.15.32:37080) 2024-04-17T09:56:47.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.576+0000 7f2c03400640 1 -- 172.21.15.32:0/1437354647 learned_addr learned my addr 172.21.15.32:0/1437354647 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:47.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.576+0000 7f2c03e00640 1 --2- 172.21.15.32:0/1437354647 >> v2:172.21.15.32:3300/0 conn(0x7f2c04166ef0 0x7f2c04072260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.576+0000 7f2c02a00640 1 -- 172.21.15.32:0/1437354647 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2404596351 0 0) 0x7f2c041673f0 con 0x7f2c04162bb0 2024-04-17T09:56:47.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.576+0000 7f2c02a00640 1 -- 172.21.15.32:0/1437354647 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2be8003530 con 0x7f2c04162bb0 2024-04-17T09:56:47.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.576+0000 7f2c03e00640 1 -- 172.21.15.32:0/1437354647 >> v1:172.21.15.144:6789/0 conn(0x7f2c04071df0 legacy=0x7f2c041667e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.576+0000 7f2c03e00640 1 -- 172.21.15.32:0/1437354647 >> v1:172.21.15.32:6789/0 conn(0x7f2c04162bb0 legacy=0x7f2c0406e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.576+0000 7f2c03e00640 1 -- 172.21.15.32:0/1437354647 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0406def0 con 0x7f2c04166ef0 2024-04-17T09:56:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.576+0000 7f2c03e00640 1 --2- 172.21.15.32:0/1437354647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c04166ef0 0x7f2c04072260 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009d90 tx=0x7f2bfc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c2cc0cc4baad37 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.576+0000 7f2c02a00640 1 -- 172.21.15.32:0/1437354647 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0b2070 con 0x7f2c04166ef0 2024-04-17T09:56:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.576+0000 7f2c02a00640 1 -- 172.21.15.32:0/1437354647 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bfc0a5830 con 0x7f2c04166ef0 2024-04-17T09:56:47.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.580+0000 7f2c02a00640 1 -- 172.21.15.32:0/1437354647 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0a5c00 con 0x7f2c04166ef0 2024-04-17T09:56:47.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.580+0000 7f2c0a800640 1 -- 172.21.15.32:0/1437354647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c04166ef0 msgr2=0x7f2c04072260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.580+0000 7f2c0a800640 1 --2- 172.21.15.32:0/1437354647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c04166ef0 0x7f2c04072260 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009d90 tx=0x7f2bfc01c970 comp rx=0 tx=0).stop 2024-04-17T09:56:47.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.580+0000 7f2c0a800640 1 -- 172.21.15.32:0/1437354647 shutdown_connections 2024-04-17T09:56:47.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.580+0000 7f2c0a800640 1 --2- 172.21.15.32:0/1437354647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c04166ef0 0x7f2c04072260 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.580+0000 7f2c0a800640 1 -- 172.21.15.32:0/1437354647 >> 172.21.15.32:0/1437354647 conn(0x7f2c0406ba30 msgr2=0x7f2c0406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:47.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.580+0000 7f2c0a800640 1 -- 172.21.15.32:0/1437354647 shutdown_connections 2024-04-17T09:56:47.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.580+0000 7f2c0a800640 1 -- 172.21.15.32:0/1437354647 wait complete. 2024-04-17T09:56:47.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.580+0000 7f2c0a800640 1 Processor -- start 2024-04-17T09:56:47.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.580+0000 7f2c0a800640 1 -- start start 2024-04-17T09:56:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.580+0000 7f2c0a800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c04071df0 0x7f2c04153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.580+0000 7f2c0a800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c04162bb0 0x7f2c040abee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.580+0000 7f2c0a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c04166ef0 0x7f2c040ac420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.580+0000 7f2c0a800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2c040b53d0 con 0x7f2c04166ef0 2024-04-17T09:56:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.580+0000 7f2c0a800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2c040b5250 con 0x7f2c04071df0 2024-04-17T09:56:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.580+0000 7f2c0a800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2c04170980 con 0x7f2c04162bb0 2024-04-17T09:56:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.580+0000 7f2c03e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c04166ef0 0x7f2c040ac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.580+0000 7f2c03e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c04166ef0 0x7f2c040ac420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35152/0 (socket says 172.21.15.32:35152) 2024-04-17T09:56:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.580+0000 7f2c03e00640 1 -- 172.21.15.32:0/1238000141 learned_addr learned my addr 172.21.15.32:0/1238000141 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:47.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.580+0000 7f2c03400640 1 --2- 172.21.15.32:0/1238000141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c04071df0 0x7f2c04153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.580+0000 7f2bfbe00640 1 --2- 172.21.15.32:0/1238000141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c04162bb0 0x7f2c040abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.584+0000 7f2c03e00640 1 -- 172.21.15.32:0/1238000141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c04162bb0 msgr2=0x7f2c040abee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.584+0000 7f2c03e00640 1 --2- 172.21.15.32:0/1238000141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c04162bb0 0x7f2c040abee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.584+0000 7f2c03e00640 1 -- 172.21.15.32:0/1238000141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c04071df0 msgr2=0x7f2c04153b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.584+0000 7f2c03e00640 1 --2- 172.21.15.32:0/1238000141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c04071df0 0x7f2c04153b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.584+0000 7f2c03e00640 1 -- 172.21.15.32:0/1238000141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c040b2660 con 0x7f2c04166ef0 2024-04-17T09:56:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.584+0000 7f2bfbe00640 1 --2- 172.21.15.32:0/1238000141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c04162bb0 0x7f2c040abee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.584+0000 7f2c03400640 1 --2- 172.21.15.32:0/1238000141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c04071df0 0x7f2c04153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.584+0000 7f2c03e00640 1 --2- 172.21.15.32:0/1238000141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c04166ef0 0x7f2c040ac420 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc01a5a0 tx=0x7f2bfc019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.584+0000 7f2bfb400640 1 -- 172.21.15.32:0/1238000141 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0b2050 con 0x7f2c04166ef0 2024-04-17T09:56:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.584+0000 7f2bfb400640 1 -- 172.21.15.32:0/1238000141 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bfc0be070 con 0x7f2c04166ef0 2024-04-17T09:56:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.584+0000 7f2bfb400640 1 -- 172.21.15.32:0/1238000141 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0ad850 con 0x7f2c04166ef0 2024-04-17T09:56:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.584+0000 7f2c0a800640 1 -- 172.21.15.32:0/1238000141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c040b2940 con 0x7f2c04166ef0 2024-04-17T09:56:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.584+0000 7f2c0a800640 1 -- 172.21.15.32:0/1238000141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c04154090 con 0x7f2c04166ef0 2024-04-17T09:56:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.584+0000 7f2c0a800640 1 -- 172.21.15.32:0/1238000141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c04068a90 con 0x7f2c04166ef0 2024-04-17T09:56:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.584+0000 7f2bfb400640 1 -- 172.21.15.32:0/1238000141 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2bfc0ad9f0 con 0x7f2c04166ef0 2024-04-17T09:56:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.584+0000 7f2bfb400640 1 --2- 172.21.15.32:0/1238000141 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2be00683b0 0x7f2be006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.584+0000 7f2bfb400640 1 -- 172.21.15.32:0/1238000141 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f2bfc1214a0 con 0x7f2c04166ef0 2024-04-17T09:56:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.584+0000 7f2c03400640 1 --2- 172.21.15.32:0/1238000141 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2be00683b0 0x7f2be006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.588+0000 7f2c03400640 1 --2- 172.21.15.32:0/1238000141 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2be00683b0 0x7f2be006a870 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40011f0 tx=0x7f2bf4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:47.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.588+0000 7f2bfb400640 1 -- 172.21.15.32:0/1238000141 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2bfc0c7020 con 0x7f2c04166ef0 2024-04-17T09:56:47.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.792+0000 7f2c0a800640 1 -- 172.21.15.32:0/1238000141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c04163d50 con 0x7f2c04166ef0 2024-04-17T09:56:47.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.792+0000 7f2bfb400640 1 -- 172.21.15.32:0/1238000141 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5202 (secure 0 0 0) 0x7f2bfc016560 con 0x7f2c04166ef0 2024-04-17T09:56:47.793 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:47.793 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5495,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/1236891874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1236891874},{"type":"v1","addr":"172.21.15.144:6834","nonce":1236891874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5498,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3538881342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3538881342},{"type":"v1","addr":"172.21.15.144:6835","nonce":3538881342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6201,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3549801633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3549801633},{"type":"v1","addr":"172.21.15.32:6836","nonce":3549801633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:56:01.579180+0000","modified":"2024-04-17T09:56:18.240809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":7,"addr":"172.21.15.32:6837/866826055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":866826055},{"type":"v1","addr":"172.21.15.32:6837","nonce":866826055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-04-17T09:56:47.793 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 33 2024-04-17T09:56:47.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.792+0000 7f2c0a800640 1 -- 172.21.15.32:0/1238000141 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2be00683b0 msgr2=0x7f2be006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.792+0000 7f2c0a800640 1 --2- 172.21.15.32:0/1238000141 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2be00683b0 0x7f2be006a870 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40011f0 tx=0x7f2bf4009040 comp rx=0 tx=0).stop 2024-04-17T09:56:47.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.792+0000 7f2c0a800640 1 -- 172.21.15.32:0/1238000141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c04166ef0 msgr2=0x7f2c040ac420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.792+0000 7f2c0a800640 1 --2- 172.21.15.32:0/1238000141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c04166ef0 0x7f2c040ac420 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc01a5a0 tx=0x7f2bfc019710 comp rx=0 tx=0).stop 2024-04-17T09:56:47.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.792+0000 7f2c0a800640 1 -- 172.21.15.32:0/1238000141 shutdown_connections 2024-04-17T09:56:47.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.792+0000 7f2c0a800640 1 --2- 172.21.15.32:0/1238000141 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2be00683b0 0x7f2be006a870 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.792+0000 7f2c0a800640 1 --2- 172.21.15.32:0/1238000141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c04162bb0 0x7f2c040abee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.792+0000 7f2c0a800640 1 --2- 172.21.15.32:0/1238000141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c04166ef0 0x7f2c040ac420 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.792+0000 7f2c0a800640 1 --2- 172.21.15.32:0/1238000141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c04071df0 0x7f2c04153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.792+0000 7f2c0a800640 1 -- 172.21.15.32:0/1238000141 >> 172.21.15.32:0/1238000141 conn(0x7f2c0406ba30 msgr2=0x7f2c0406e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:47.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.792+0000 7f2c0a800640 1 -- 172.21.15.32:0/1238000141 shutdown_connections 2024-04-17T09:56:47.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.792+0000 7f2c0a800640 1 -- 172.21.15.32:0/1238000141 wait complete. 2024-04-17T09:56:47.815 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-17T09:56:47.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.956+0000 7f6de1e00640 1 Processor -- start 2024-04-17T09:56:47.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.956+0000 7f6de1e00640 1 -- start start 2024-04-17T09:56:47.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.956+0000 7f6de1e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ddc06ba30 0x7f6ddc06be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.956+0000 7f6de1e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ddc06c350 con 0x7f6ddc0cf370 2024-04-17T09:56:47.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.956+0000 7f6de1e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ddc0d0130 con 0x7f6ddc071e60 2024-04-17T09:56:47.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.956+0000 7f6de1e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6ddc0cfef0 con 0x7f6ddc06ba30 2024-04-17T09:56:47.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.956+0000 7f6dd2000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6ddc0cf370 0x7f6ddc0cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37090/0 (socket says 172.21.15.32:37090) 2024-04-17T09:56:47.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.956+0000 7f6dd2000640 1 -- 172.21.15.32:0/3484925691 learned_addr learned my addr 172.21.15.32:0/3484925691 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.956+0000 7f6ddaa00640 1 --2- 172.21.15.32:0/3484925691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ddc06ba30 0x7f6ddc06be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.956+0000 7f6dda000640 1 -- 172.21.15.32:0/3484925691 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1709513180 0 0) 0x7f6ddc06c350 con 0x7f6ddc0cf370 2024-04-17T09:56:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6dda000640 1 -- 172.21.15.32:0/3484925691 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6dc0003530 con 0x7f6ddc0cf370 2024-04-17T09:56:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6dda000640 1 -- 172.21.15.32:0/3484925691 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3535669345 0 0) 0x7f6ddc0d0130 con 0x7f6ddc071e60 2024-04-17T09:56:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6dda000640 1 -- 172.21.15.32:0/3484925691 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ddc06c350 con 0x7f6ddc071e60 2024-04-17T09:56:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6dda000640 1 -- 172.21.15.32:0/3484925691 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3159385374 0 0) 0x7f6dc0003530 con 0x7f6ddc0cf370 2024-04-17T09:56:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6dda000640 1 -- 172.21.15.32:0/3484925691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ddc06ba30 msgr2=0x7f6ddc06be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6dda000640 1 --2- 172.21.15.32:0/3484925691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ddc06ba30 0x7f6ddc06be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6dda000640 1 -- 172.21.15.32:0/3484925691 >> v1:172.21.15.144:6789/0 conn(0x7f6ddc071e60 legacy=0x7f6ddc0722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6dda000640 1 -- 172.21.15.32:0/3484925691 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc15d090 con 0x7f6ddc0cf370 2024-04-17T09:56:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6dda000640 1 -- 172.21.15.32:0/3484925691 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f6dc4001680 con 0x7f6ddc0cf370 2024-04-17T09:56:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6ddaa00640 1 --2- 172.21.15.32:0/3484925691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ddc06ba30 0x7f6ddc06be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:56:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6dda000640 1 -- 172.21.15.32:0/3484925691 >> v1:172.21.15.32:6789/0 conn(0x7f6ddc0cf370 legacy=0x7f6ddc0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6dda000640 1 --2- 172.21.15.32:0/3484925691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc0003b80 0x7f6dc0003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6dda000640 1 --2- 172.21.15.32:0/3484925691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dc0004810 0x7f6dc0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6dda000640 1 --2- 172.21.15.32:0/3484925691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ddc06ba30 0x7f6dc00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6dda000640 1 -- 172.21.15.32:0/3484925691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc0cfef0 con 0x7f6dc0003b80 2024-04-17T09:56:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6dda000640 1 -- 172.21.15.32:0/3484925691 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc000ba80 con 0x7f6dc0004810 2024-04-17T09:56:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6dda000640 1 -- 172.21.15.32:0/3484925691 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6dc000bc50 con 0x7f6ddc06ba30 2024-04-17T09:56:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6ddaa00640 1 --2- 172.21.15.32:0/3484925691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc0003b80 0x7f6dc0003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6dd2000640 1 --2- 172.21.15.32:0/3484925691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dc0004810 0x7f6dc0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6ddb400640 1 --2- 172.21.15.32:0/3484925691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ddc06ba30 0x7f6dc00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6ddaa00640 1 -- 172.21.15.32:0/3484925691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ddc06ba30 msgr2=0x7f6dc00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6ddaa00640 1 --2- 172.21.15.32:0/3484925691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ddc06ba30 0x7f6dc00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6ddaa00640 1 -- 172.21.15.32:0/3484925691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dc0004810 msgr2=0x7f6dc0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6ddaa00640 1 --2- 172.21.15.32:0/3484925691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dc0004810 0x7f6dc0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6ddaa00640 1 -- 172.21.15.32:0/3484925691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc0003530 con 0x7f6dc0003b80 2024-04-17T09:56:47.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6ddb400640 1 --2- 172.21.15.32:0/3484925691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ddc06ba30 0x7f6dc00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.960+0000 7f6ddaa00640 1 --2- 172.21.15.32:0/3484925691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc0003b80 0x7f6dc0003f80 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009c80 tx=0x7f6dcc01cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12cc104cdb547231 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6dda000640 1 -- 172.21.15.32:0/3484925691 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc0b2070 con 0x7f6dc0003b80 2024-04-17T09:56:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6dda000640 1 -- 172.21.15.32:0/3484925691 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dcc0a5730 con 0x7f6dc0003b80 2024-04-17T09:56:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6dda000640 1 -- 172.21.15.32:0/3484925691 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc009880 con 0x7f6dc0003b80 2024-04-17T09:56:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6de1e00640 1 -- 172.21.15.32:0/3484925691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc0003b80 msgr2=0x7f6dc0003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6de1e00640 1 --2- 172.21.15.32:0/3484925691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc0003b80 0x7f6dc0003f80 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009c80 tx=0x7f6dcc01cb80 comp rx=0 tx=0).stop 2024-04-17T09:56:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6ddb400640 1 -- 172.21.15.32:0/3484925691 reap_dead start 2024-04-17T09:56:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6de1e00640 1 -- 172.21.15.32:0/3484925691 shutdown_connections 2024-04-17T09:56:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6de1e00640 1 -- 172.21.15.32:0/3484925691 >> 172.21.15.32:0/3484925691 conn(0x7f6ddc0b2120 msgr2=0x7f6ddc0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6de1e00640 1 -- 172.21.15.32:0/3484925691 shutdown_connections 2024-04-17T09:56:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6de1e00640 1 -- 172.21.15.32:0/3484925691 wait complete. 2024-04-17T09:56:47.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6de1e00640 1 Processor -- start 2024-04-17T09:56:47.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6de1e00640 1 -- start start 2024-04-17T09:56:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6de1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc13dc20 0x7f6ddc15f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6de1e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ddc13e000 0x7f6ddc15f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6de1e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ddc159620 0x7f6ddc159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6de1e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc15db70 con 0x7f6ddc13dc20 2024-04-17T09:56:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6de1e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc072dd0 con 0x7f6ddc159620 2024-04-17T09:56:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6de1e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6ddc15dcf0 con 0x7f6ddc13e000 2024-04-17T09:56:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6ddaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc13dc20 0x7f6ddc15f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6ddaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc13dc20 0x7f6ddc15f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35174/0 (socket says 172.21.15.32:35174) 2024-04-17T09:56:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6ddaa00640 1 -- 172.21.15.32:0/2281901453 learned_addr learned my addr 172.21.15.32:0/2281901453 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6ddb400640 1 --2- 172.21.15.32:0/2281901453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ddc159620 0x7f6ddc159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6dd2000640 1 --2- 172.21.15.32:0/2281901453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ddc13e000 0x7f6ddc15f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6ddaa00640 1 -- 172.21.15.32:0/2281901453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ddc13e000 msgr2=0x7f6ddc15f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6ddaa00640 1 --2- 172.21.15.32:0/2281901453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ddc13e000 0x7f6ddc15f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6ddaa00640 1 -- 172.21.15.32:0/2281901453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ddc159620 msgr2=0x7f6ddc159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6ddaa00640 1 --2- 172.21.15.32:0/2281901453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ddc159620 0x7f6ddc159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6ddaa00640 1 -- 172.21.15.32:0/2281901453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc15ff00 con 0x7f6ddc13dc20 2024-04-17T09:56:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.964+0000 7f6dd2000640 1 --2- 172.21.15.32:0/2281901453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ddc13e000 0x7f6ddc15f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.968+0000 7f6ddaa00640 1 --2- 172.21.15.32:0/2281901453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc13dc20 0x7f6ddc15f220 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc01d420 tx=0x7f6dcc019920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.968+0000 7f6dd3e00640 1 -- 172.21.15.32:0/2281901453 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc0bd070 con 0x7f6ddc13dc20 2024-04-17T09:56:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.968+0000 7f6dd3e00640 1 -- 172.21.15.32:0/2281901453 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dcc009400 con 0x7f6ddc13dc20 2024-04-17T09:56:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.968+0000 7f6de1e00640 1 -- 172.21.15.32:0/2281901453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ddc1601e0 con 0x7f6ddc13dc20 2024-04-17T09:56:47.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.968+0000 7f6dd3e00640 1 -- 172.21.15.32:0/2281901453 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc0b2040 con 0x7f6ddc13dc20 2024-04-17T09:56:47.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.968+0000 7f6de1e00640 1 -- 172.21.15.32:0/2281901453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ddc143830 con 0x7f6ddc13dc20 2024-04-17T09:56:47.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.968+0000 7f6dd3e00640 1 -- 172.21.15.32:0/2281901453 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6dcc0c0070 con 0x7f6ddc13dc20 2024-04-17T09:56:47.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.968+0000 7f6dd3e00640 1 --2- 172.21.15.32:0/2281901453 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6db8068380 0x7f6db806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.968+0000 7f6dd3e00640 1 -- 172.21.15.32:0/2281901453 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f6dcc1201e0 con 0x7f6ddc13dc20 2024-04-17T09:56:47.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.968+0000 7f6dd2000640 1 --2- 172.21.15.32:0/2281901453 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6db8068380 0x7f6db806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6de52d2000 1 Processor -- start 2024-04-17T09:56:47.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6de52d2000 1 -- start start 2024-04-17T09:56:47.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6dd2000640 1 --2- 172.21.15.32:0/2281901453 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6db8068380 0x7f6db806a840 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4001ec0 tx=0x7f6dc4001e30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:47.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6de52d2000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabde8a50 0x562eabde8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6de52d2000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabde9370 0x562eabd9ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6de52d2000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eabd9bf70 0x562eabd9c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6de52d2000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562eabbed590 con 0x562eabde8a50 2024-04-17T09:56:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6de52d2000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562eabba6580 con 0x562eabde9370 2024-04-17T09:56:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6de52d2000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x562eabbe9550 con 0x562eabd9bf70 2024-04-17T09:56:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6ddaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabde8a50 0x562eabde8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6ddaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabde8a50 0x562eabde8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35186/0 (socket says 172.21.15.32:35186) 2024-04-17T09:56:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6ddaa00640 1 -- 172.21.15.32:0/2732518883 learned_addr learned my addr 172.21.15.32:0/2732518883 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6dd2000640 1 --2- 172.21.15.32:0/2732518883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabde9370 0x562eabd9ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6ddb400640 1 --2- 172.21.15.32:0/2732518883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eabd9bf70 0x562eabd9c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6ddaa00640 1 -- 172.21.15.32:0/2732518883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eabd9bf70 msgr2=0x562eabd9c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6ddaa00640 1 --2- 172.21.15.32:0/2732518883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eabd9bf70 0x562eabd9c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6ddaa00640 1 -- 172.21.15.32:0/2732518883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabde9370 msgr2=0x562eabd9ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6ddaa00640 1 --2- 172.21.15.32:0/2732518883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabde9370 0x562eabd9ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6ddaa00640 1 -- 172.21.15.32:0/2732518883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eabc94b90 con 0x562eabde8a50 2024-04-17T09:56:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6ddb400640 1 --2- 172.21.15.32:0/2732518883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eabd9bf70 0x562eabd9c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6dd2000640 1 --2- 172.21.15.32:0/2732518883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabde9370 0x562eabd9ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:47.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6ddaa00640 1 --2- 172.21.15.32:0/2732518883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabde8a50 0x562eabde8e30 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc11e730 tx=0x7f6dcc0efd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963621b89ee98e74 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:47.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6dcbe00640 1 -- 172.21.15.32:0/2732518883 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc0f8c80 con 0x562eabde8a50 2024-04-17T09:56:47.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6dcbe00640 1 -- 172.21.15.32:0/2732518883 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dcc0ee1f0 con 0x562eabde8a50 2024-04-17T09:56:47.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6dcbe00640 1 -- 172.21.15.32:0/2732518883 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc009880 con 0x562eabde8a50 2024-04-17T09:56:47.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6de52d2000 1 -- 172.21.15.32:0/2732518883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabde8a50 msgr2=0x562eabde8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6de52d2000 1 --2- 172.21.15.32:0/2732518883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabde8a50 0x562eabde8e30 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc11e730 tx=0x7f6dcc0efd50 comp rx=0 tx=0).stop 2024-04-17T09:56:47.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6de52d2000 1 -- 172.21.15.32:0/2732518883 shutdown_connections 2024-04-17T09:56:47.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6de52d2000 1 --2- 172.21.15.32:0/2732518883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eabd9bf70 0x562eabd9c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6de52d2000 1 --2- 172.21.15.32:0/2732518883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabde9370 0x562eabd9ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6de52d2000 1 --2- 172.21.15.32:0/2732518883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabde8a50 0x562eabde8e30 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6de52d2000 1 -- 172.21.15.32:0/2732518883 >> 172.21.15.32:0/2732518883 conn(0x562eabdc97f0 msgr2=0x562eabdcbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:47.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6de52d2000 1 -- 172.21.15.32:0/2732518883 shutdown_connections 2024-04-17T09:56:47.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6de52d2000 1 -- 172.21.15.32:0/2732518883 wait complete. 2024-04-17T09:56:47.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6de52d2000 1 -- start start 2024-04-17T09:56:47.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.972+0000 7f6de52d2000 1 Processor -- start 2024-04-17T09:56:47.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6de52d2000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabd9bf70 0x562eabdeeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6de52d2000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabde8a50 0x562eabdef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6de52d2000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eabde9370 0x562eabda04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6de52d2000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562eabbf84c0 con 0x562eabd9bf70 2024-04-17T09:56:47.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6de52d2000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562eabb8dc80 con 0x562eabde8a50 2024-04-17T09:56:47.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6de52d2000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x562eabbb8930 con 0x562eabde9370 2024-04-17T09:56:47.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6de52d2000 10 client.0 _ll_register_callbacks cb 0x562eabd896d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:56:47.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6ddaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabd9bf70 0x562eabdeeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6ddaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabd9bf70 0x562eabdeeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35200/0 (socket says 172.21.15.32:35200) 2024-04-17T09:56:47.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6ddaa00640 1 -- 172.21.15.32:0/986973548 learned_addr learned my addr 172.21.15.32:0/986973548 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:47.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6ddb400640 1 --2- 172.21.15.32:0/986973548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eabde9370 0x562eabda04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6dd2000640 1 --2- 172.21.15.32:0/986973548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabde8a50 0x562eabdef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6ddaa00640 1 -- 172.21.15.32:0/986973548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eabde9370 msgr2=0x562eabda04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6ddaa00640 1 --2- 172.21.15.32:0/986973548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eabde9370 0x562eabda04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6ddaa00640 1 -- 172.21.15.32:0/986973548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabde8a50 msgr2=0x562eabdef2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6ddaa00640 1 --2- 172.21.15.32:0/986973548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabde8a50 0x562eabdef2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6ddaa00640 1 -- 172.21.15.32:0/986973548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eabbc6b20 con 0x562eabd9bf70 2024-04-17T09:56:47.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6dd2000640 1 --2- 172.21.15.32:0/986973548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabde8a50 0x562eabdef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:47.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6ddaa00640 1 --2- 172.21.15.32:0/986973548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabd9bf70 0x562eabdeeda0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc01c5f0 tx=0x7f6dcc000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0a3bbff78b4329 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:47.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6dc8c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:56:47.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6dc8c00640 1 -- 172.21.15.32:0/986973548 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc0f8c80 con 0x562eabd9bf70 2024-04-17T09:56:47.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6dc8c00640 1 -- 172.21.15.32:0/986973548 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f6dcc0f4070 con 0x562eabd9bf70 2024-04-17T09:56:47.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6de52d2000 1 -- 172.21.15.32:0/986973548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dcc0ee1f0 con 0x562eabd9bf70 2024-04-17T09:56:47.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6dc8c00640 1 -- 172.21.15.32:0/986973548 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6dcc0f87f0 con 0x562eabd9bf70 2024-04-17T09:56:47.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6de52d2000 10 client.6567 fetch_fsmap learned FSMap version 33 2024-04-17T09:56:47.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6de52d2000 1 -- 172.21.15.32:0/986973548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x562eabc31610 con 0x562eabd9bf70 2024-04-17T09:56:47.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6dc8c00640 1 -- 172.21.15.32:0/986973548 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dcc0ee1f0 con 0x562eabd9bf70 2024-04-17T09:56:47.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6dc8c00640 1 -- 172.21.15.32:0/986973548 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 33) ==== 3495+0+0 (secure 0 0 0) 0x7f6dcc11e120 con 0x562eabd9bf70 2024-04-17T09:56:47.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6de52d2000 10 client.6567 fetch_fsmap finished waiting for FSMap version 33 2024-04-17T09:56:47.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6de52d2000 10 client.6567 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T09:56:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6de52d2000 20 client.6567 populate_metadata read hostname 'smithi032' 2024-04-17T09:56:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6de52d2000 1 --2- 172.21.15.32:0/986973548 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x562eabda1240 0x562eabda1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6de52d2000 4 client.6567 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:56:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6de52d2000 1 -- 172.21.15.32:0/986973548 --> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562eabb86e40 con 0x562eabda1240 2024-04-17T09:56:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6dd2000640 1 --2- 172.21.15.32:0/986973548 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x562eabda1240 0x562eabda1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6dd2000640 1 --2- 172.21.15.32:0/986973548 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x562eabda1240 0x562eabda1620 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fc112f85a870455 server_cookie=f6d6c2e8bfd561af in_seq=0 out_seq=0 2024-04-17T09:56:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6dc8c00640 10 client.6567 ms_handle_connect on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6dc8c00640 1 -- 172.21.15.32:0/986973548 <== mds.0 v2:172.21.15.32:6835/866826055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562eabb86e40 con 0x562eabda1240 2024-04-17T09:56:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.976+0000 7f6dc8c00640 10 client.6567 handle_command_reply: tid=0 2024-04-17T09:56:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.980+0000 7f6de52d2000 1 client.6567 shutdown 2024-04-17T09:56:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.980+0000 7f6de52d2000 1 -- 172.21.15.32:0/986973548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabd9bf70 msgr2=0x562eabdeeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.980+0000 7f6de52d2000 1 --2- 172.21.15.32:0/986973548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabd9bf70 0x562eabdeeda0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc01c5f0 tx=0x7f6dcc000bd0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.980+0000 7f6de52d2000 1 -- 172.21.15.32:0/986973548 shutdown_connections 2024-04-17T09:56:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.980+0000 7f6de52d2000 1 --2- 172.21.15.32:0/986973548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eabde9370 0x562eabda04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.980+0000 7f6de52d2000 1 --2- 172.21.15.32:0/986973548 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x562eabda1240 0x562eabda1620 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.980+0000 7f6de52d2000 1 --2- 172.21.15.32:0/986973548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabde8a50 0x562eabdef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.980+0000 7f6de52d2000 1 --2- 172.21.15.32:0/986973548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabd9bf70 0x562eabdeeda0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.980+0000 7f6dc8c00640 0 client.6567 ms_handle_reset on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.980+0000 7f6de52d2000 1 -- 172.21.15.32:0/986973548 >> 172.21.15.32:0/986973548 conn(0x562eabdc97f0 msgr2=0x562eabdcbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.980+0000 7f6de52d2000 1 -- 172.21.15.32:0/986973548 shutdown_connections 2024-04-17T09:56:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.980+0000 7f6de52d2000 1 -- 172.21.15.32:0/986973548 wait complete. 2024-04-17T09:56:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.980+0000 7f6de52d2000 20 client.6567 trim_cache size 0 max 16384 2024-04-17T09:56:47.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.992+0000 7f6de52d2000 1 Processor -- start 2024-04-17T09:56:47.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.992+0000 7f6de52d2000 1 -- start start 2024-04-17T09:56:47.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.992+0000 7f6de52d2000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabd9bf70 0x562eabd9b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.992+0000 7f6de52d2000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eabde8a50 0x562eabd9b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.992+0000 7f6de52d2000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabde9370 0x562eabd9d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:47.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.992+0000 7f6de52d2000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562eabbec8a0 con 0x562eabde9370 2024-04-17T09:56:47.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.992+0000 7f6de52d2000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562eabbaba90 con 0x562eabd9bf70 2024-04-17T09:56:47.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.992+0000 7f6de52d2000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x562eabbf84c0 con 0x562eabde8a50 2024-04-17T09:56:47.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.992+0000 7f6ddb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabde9370 0x562eabd9d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.992+0000 7f6ddb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabde9370 0x562eabd9d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35202/0 (socket says 172.21.15.32:35202) 2024-04-17T09:56:47.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.992+0000 7f6ddb400640 1 -- 172.21.15.32:0/1657732521 learned_addr learned my addr 172.21.15.32:0/1657732521 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:47.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.992+0000 7f6ddaa00640 1 --2- 172.21.15.32:0/1657732521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabd9bf70 0x562eabd9b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.992+0000 7f6dd2000640 1 --2- 172.21.15.32:0/1657732521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eabde8a50 0x562eabd9b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:47.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.992+0000 7f6ddb400640 1 -- 172.21.15.32:0/1657732521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eabde8a50 msgr2=0x562eabd9b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.992+0000 7f6ddb400640 1 --2- 172.21.15.32:0/1657732521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eabde8a50 0x562eabd9b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.992+0000 7f6ddb400640 1 -- 172.21.15.32:0/1657732521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabd9bf70 msgr2=0x562eabd9b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:47.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.992+0000 7f6ddb400640 1 --2- 172.21.15.32:0/1657732521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabd9bf70 0x562eabd9b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.992+0000 7f6ddb400640 1 -- 172.21.15.32:0/1657732521 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eabbc3190 con 0x562eabde9370 2024-04-17T09:56:47.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.992+0000 7f6dd2000640 1 --2- 172.21.15.32:0/1657732521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eabde8a50 0x562eabd9b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:47.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.992+0000 7f6ddaa00640 1 --2- 172.21.15.32:0/1657732521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabd9bf70 0x562eabd9b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:47.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6ddb400640 1 --2- 172.21.15.32:0/1657732521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabde9370 0x562eabd9d360 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0cf750 tx=0x7f6dd4005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8654a349a4082514 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:47.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6dcb400640 1 -- 172.21.15.32:0/1657732521 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd401a070 con 0x562eabde9370 2024-04-17T09:56:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6dcb400640 1 -- 172.21.15.32:0/1657732521 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dd40057e0 con 0x562eabde9370 2024-04-17T09:56:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6dcb400640 1 -- 172.21.15.32:0/1657732521 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4016420 con 0x562eabde9370 2024-04-17T09:56:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6de52d2000 1 -- 172.21.15.32:0/1657732521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabde9370 msgr2=0x562eabd9d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6de52d2000 1 --2- 172.21.15.32:0/1657732521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabde9370 0x562eabd9d360 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0cf750 tx=0x7f6dd4005590 comp rx=0 tx=0).stop 2024-04-17T09:56:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6de52d2000 1 -- 172.21.15.32:0/1657732521 shutdown_connections 2024-04-17T09:56:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6de52d2000 1 --2- 172.21.15.32:0/1657732521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eabde8a50 0x562eabd9b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6de52d2000 1 --2- 172.21.15.32:0/1657732521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabde9370 0x562eabd9d360 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6de52d2000 1 --2- 172.21.15.32:0/1657732521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabd9bf70 0x562eabd9b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6de52d2000 1 -- 172.21.15.32:0/1657732521 >> 172.21.15.32:0/1657732521 conn(0x562eabdc97f0 msgr2=0x562eabdf2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6de52d2000 1 -- 172.21.15.32:0/1657732521 shutdown_connections 2024-04-17T09:56:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6de52d2000 1 -- 172.21.15.32:0/1657732521 wait complete. 2024-04-17T09:56:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6de52d2000 1 -- start start 2024-04-17T09:56:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6de52d2000 1 Processor -- start 2024-04-17T09:56:48.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6de52d2000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabd9bf70 0x562eabde8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:48.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6de52d2000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eabde8a50 0x562eabd9c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:48.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6de52d2000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabde9370 0x562eabd9c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:48.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6de52d2000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562eabda7360 con 0x562eabde9370 2024-04-17T09:56:48.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6de52d2000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562eabbb8930 con 0x562eabd9bf70 2024-04-17T09:56:48.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6de52d2000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x562eabb92230 con 0x562eabde8a50 2024-04-17T09:56:48.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6de52d2000 10 client.0 _ll_register_callbacks cb 0x7f6dcc0eff20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:56:48.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6ddb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabde9370 0x562eabd9c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6ddb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabde9370 0x562eabd9c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35210/0 (socket says 172.21.15.32:35210) 2024-04-17T09:56:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6ddb400640 1 -- 172.21.15.32:0/826517871 learned_addr learned my addr 172.21.15.32:0/826517871 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6ddaa00640 1 --2- 172.21.15.32:0/826517871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabd9bf70 0x562eabde8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6dd2000640 1 --2- 172.21.15.32:0/826517871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eabde8a50 0x562eabd9c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6ddb400640 1 -- 172.21.15.32:0/826517871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eabde8a50 msgr2=0x562eabd9c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6ddb400640 1 --2- 172.21.15.32:0/826517871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eabde8a50 0x562eabd9c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6ddb400640 1 -- 172.21.15.32:0/826517871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabd9bf70 msgr2=0x562eabde8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6ddb400640 1 --2- 172.21.15.32:0/826517871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabd9bf70 0x562eabde8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6ddb400640 1 -- 172.21.15.32:0/826517871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eabba67d0 con 0x562eabde9370 2024-04-17T09:56:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6ddaa00640 1 --2- 172.21.15.32:0/826517871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabd9bf70 0x562eabde8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:56:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6ddb400640 1 --2- 172.21.15.32:0/826517871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabde9370 0x562eabd9c890 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009010 tx=0x7f6dd4001660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec9f7049fca0c37 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:47.996+0000 7f6dc8c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:56:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6dc8c00640 1 -- 172.21.15.32:0/826517871 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd401a040 con 0x562eabde9370 2024-04-17T09:56:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6dc8c00640 1 -- 172.21.15.32:0/826517871 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f6dd4006b60 con 0x562eabde9370 2024-04-17T09:56:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de52d2000 1 -- 172.21.15.32:0/826517871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dd40057e0 con 0x562eabde9370 2024-04-17T09:56:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6dc8c00640 1 -- 172.21.15.32:0/826517871 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6dd4031050 con 0x562eabde9370 2024-04-17T09:56:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de52d2000 10 client.6573 fetch_fsmap learned FSMap version 33 2024-04-17T09:56:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de52d2000 1 -- 172.21.15.32:0/826517871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x562eabb92440 con 0x562eabde9370 2024-04-17T09:56:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6dc8c00640 1 -- 172.21.15.32:0/826517871 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dd40057e0 con 0x562eabde9370 2024-04-17T09:56:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6dc8c00640 1 -- 172.21.15.32:0/826517871 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 33) ==== 3495+0+0 (secure 0 0 0) 0x7f6dd4034020 con 0x562eabde9370 2024-04-17T09:56:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de52d2000 10 client.6573 fetch_fsmap finished waiting for FSMap version 33 2024-04-17T09:56:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de52d2000 10 client.6573 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T09:56:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de52d2000 20 client.6573 populate_metadata read hostname 'smithi032' 2024-04-17T09:56:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de52d2000 1 --2- 172.21.15.32:0/826517871 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x562eabe619d0 0x562eabe63dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de52d2000 4 client.6573 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-17T09:56:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de52d2000 1 -- 172.21.15.32:0/826517871 --> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x562eabbfafb0 con 0x562eabe619d0 2024-04-17T09:56:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6ddaa00640 1 --2- 172.21.15.32:0/826517871 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x562eabe619d0 0x562eabe63dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6ddaa00640 1 --2- 172.21.15.32:0/826517871 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x562eabe619d0 0x562eabe63dc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b270cbb54cdfe34 server_cookie=a0440bc125b3645 in_seq=0 out_seq=0 2024-04-17T09:56:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6dc8c00640 10 client.6573 ms_handle_connect on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6dc8c00640 1 -- 172.21.15.32:0/826517871 <== mds.0 v2:172.21.15.32:6835/866826055 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562eabbfafb0 con 0x562eabe619d0 2024-04-17T09:56:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6dc8c00640 10 client.6573 handle_command_reply: tid=0 2024-04-17T09:56:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de52d2000 1 client.6573 shutdown 2024-04-17T09:56:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de52d2000 1 -- 172.21.15.32:0/826517871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabde9370 msgr2=0x562eabd9c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de52d2000 1 --2- 172.21.15.32:0/826517871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabde9370 0x562eabd9c890 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009010 tx=0x7f6dd4001660 comp rx=0 tx=0).stop 2024-04-17T09:56:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de52d2000 1 -- 172.21.15.32:0/826517871 shutdown_connections 2024-04-17T09:56:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de52d2000 1 --2- 172.21.15.32:0/826517871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eabde8a50 0x562eabd9c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de52d2000 1 --2- 172.21.15.32:0/826517871 >> [v2:172.21.15.32:6835/866826055,v1:172.21.15.32:6837/866826055] conn(0x562eabe619d0 0x562eabe63dc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de52d2000 1 --2- 172.21.15.32:0/826517871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eabde9370 0x562eabd9c890 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de52d2000 1 --2- 172.21.15.32:0/826517871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eabd9bf70 0x562eabde8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6dc8c00640 0 client.6573 ms_handle_reset on v2:172.21.15.32:6835/866826055 2024-04-17T09:56:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de52d2000 1 -- 172.21.15.32:0/826517871 >> 172.21.15.32:0/826517871 conn(0x562eabdc97f0 msgr2=0x562eabda3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de52d2000 1 -- 172.21.15.32:0/826517871 shutdown_connections 2024-04-17T09:56:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de52d2000 1 -- 172.21.15.32:0/826517871 wait complete. 2024-04-17T09:56:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de52d2000 20 client.6573 trim_cache size 0 max 16384 2024-04-17T09:56:48.006 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:48.006 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-17T09:56:48.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de1e00640 1 -- 172.21.15.32:0/2281901453 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6db8068380 msgr2=0x7f6db806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:48.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de1e00640 1 --2- 172.21.15.32:0/2281901453 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6db8068380 0x7f6db806a840 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4001ec0 tx=0x7f6dc4001e30 comp rx=0 tx=0).stop 2024-04-17T09:56:48.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de1e00640 1 -- 172.21.15.32:0/2281901453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc13dc20 msgr2=0x7f6ddc15f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:48.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de1e00640 1 --2- 172.21.15.32:0/2281901453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc13dc20 0x7f6ddc15f220 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc01d420 tx=0x7f6dcc019920 comp rx=0 tx=0).stop 2024-04-17T09:56:48.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de1e00640 1 -- 172.21.15.32:0/2281901453 shutdown_connections 2024-04-17T09:56:48.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de1e00640 1 --2- 172.21.15.32:0/2281901453 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6db8068380 0x7f6db806a840 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:48.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de1e00640 1 --2- 172.21.15.32:0/2281901453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ddc13e000 0x7f6ddc15f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:48.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de1e00640 1 --2- 172.21.15.32:0/2281901453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ddc159620 0x7f6ddc159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:48.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de1e00640 1 --2- 172.21.15.32:0/2281901453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ddc13dc20 0x7f6ddc15f220 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:48.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de1e00640 1 -- 172.21.15.32:0/2281901453 >> 172.21.15.32:0/2281901453 conn(0x7f6ddc0b2120 msgr2=0x7f6ddc0cc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:48.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de1e00640 1 -- 172.21.15.32:0/2281901453 shutdown_connections 2024-04-17T09:56:48.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.000+0000 7f6de1e00640 1 -- 172.21.15.32:0/2281901453 wait complete. 2024-04-17T09:56:48.021 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-17T09:56:48.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.160+0000 7f04e3200640 1 Processor -- start 2024-04-17T09:56:48.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.160+0000 7f04e3200640 1 -- start start 2024-04-17T09:56:48.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.160+0000 7f04e3200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04e40af660 0x7f04e40b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:48.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.160+0000 7f04e3200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04e406f570 0x7f04e406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:48.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.160+0000 7f04e3200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04e415c6f0 con 0x7f04e415b800 2024-04-17T09:56:48.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.160+0000 7f04e3200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f04e406ff10 con 0x7f04e40af660 2024-04-17T09:56:48.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.160+0000 7f04e3200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f04e4073a40 con 0x7f04e406f570 2024-04-17T09:56:48.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.160+0000 7f04e1e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04e40af660 0x7f04e40b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:48.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.160+0000 7f04e2800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f04e415b800 0x7f04e4071c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37094/0 (socket says 172.21.15.32:37094) 2024-04-17T09:56:48.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.160+0000 7f04e2800640 1 -- 172.21.15.32:0/3998906440 learned_addr learned my addr 172.21.15.32:0/3998906440 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:48.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.160+0000 7f04e1e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04e40af660 0x7f04e40b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35224/0 (socket says 172.21.15.32:35224) 2024-04-17T09:56:48.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.160+0000 7f04d9400640 1 --2- 172.21.15.32:0/3998906440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04e406f570 0x7f04e406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:48.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.160+0000 7f04e1e00640 1 -- 172.21.15.32:0/3998906440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04e406f570 msgr2=0x7f04e406f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:48.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.160+0000 7f04e1e00640 1 --2- 172.21.15.32:0/3998906440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04e406f570 0x7f04e406f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:48.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.160+0000 7f04e1e00640 1 -- 172.21.15.32:0/3998906440 >> v1:172.21.15.32:6789/0 conn(0x7f04e415b800 legacy=0x7f04e4071c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:48.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.160+0000 7f04e1e00640 1 -- 172.21.15.32:0/3998906440 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e415c410 con 0x7f04e40af660 2024-04-17T09:56:48.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.164+0000 7f04e1e00640 1 --2- 172.21.15.32:0/3998906440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e40af660 0x7f04e40b3c90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f04d40097c0 tx=0x7f04d401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34090d91ac90193b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:48.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.164+0000 7f04e1400640 1 -- 172.21.15.32:0/3998906440 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d40a7070 con 0x7f04e40af660 2024-04-17T09:56:48.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.164+0000 7f04e1400640 1 -- 172.21.15.32:0/3998906440 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04d40b0070 con 0x7f04e40af660 2024-04-17T09:56:48.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.164+0000 7f04e1400640 1 -- 172.21.15.32:0/3998906440 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d40ab890 con 0x7f04e40af660 2024-04-17T09:56:48.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.164+0000 7f04e3200640 1 -- 172.21.15.32:0/3998906440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e40af660 msgr2=0x7f04e40b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:48.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.164+0000 7f04e3200640 1 --2- 172.21.15.32:0/3998906440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e40af660 0x7f04e40b3c90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f04d40097c0 tx=0x7f04d401c470 comp rx=0 tx=0).stop 2024-04-17T09:56:48.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.164+0000 7f04e3200640 1 -- 172.21.15.32:0/3998906440 shutdown_connections 2024-04-17T09:56:48.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.164+0000 7f04e3200640 1 --2- 172.21.15.32:0/3998906440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04e406f570 0x7f04e406f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:48.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.164+0000 7f04e3200640 1 --2- 172.21.15.32:0/3998906440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e40af660 0x7f04e40b3c90 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:48.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.164+0000 7f04e3200640 1 -- 172.21.15.32:0/3998906440 >> 172.21.15.32:0/3998906440 conn(0x7f04e40d0180 msgr2=0x7f04e40d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:48.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.164+0000 7f04e3200640 1 -- 172.21.15.32:0/3998906440 shutdown_connections 2024-04-17T09:56:48.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.164+0000 7f04e3200640 1 -- 172.21.15.32:0/3998906440 wait complete. 2024-04-17T09:56:48.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04e3200640 1 Processor -- start 2024-04-17T09:56:48.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04e3200640 1 -- start start 2024-04-17T09:56:48.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04e3200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e406f570 0x7f04e4167e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:48.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04e3200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04e40af660 0x7f04e4151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:48.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04e3200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04e415b800 0x7f04e4152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:48.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04e3200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f04e40afbc0 con 0x7f04e406f570 2024-04-17T09:56:48.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04e3200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f04e40afa40 con 0x7f04e40af660 2024-04-17T09:56:48.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04e3200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f04e415c020 con 0x7f04e415b800 2024-04-17T09:56:48.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04e1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e406f570 0x7f04e4167e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:48.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04e1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e406f570 0x7f04e4167e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35234/0 (socket says 172.21.15.32:35234) 2024-04-17T09:56:48.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04e1e00640 1 -- 172.21.15.32:0/3677210919 learned_addr learned my addr 172.21.15.32:0/3677210919 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:48.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04e2800640 1 --2- 172.21.15.32:0/3677210919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04e415b800 0x7f04e4152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:48.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04d9400640 1 --2- 172.21.15.32:0/3677210919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04e40af660 0x7f04e4151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:48.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04e1e00640 1 -- 172.21.15.32:0/3677210919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04e415b800 msgr2=0x7f04e4152830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:48.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04e1e00640 1 --2- 172.21.15.32:0/3677210919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04e415b800 0x7f04e4152830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:48.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04e1e00640 1 -- 172.21.15.32:0/3677210919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04e40af660 msgr2=0x7f04e4151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:48.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04e1e00640 1 --2- 172.21.15.32:0/3677210919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04e40af660 0x7f04e4151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:48.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04e1e00640 1 -- 172.21.15.32:0/3677210919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e4150780 con 0x7f04e406f570 2024-04-17T09:56:48.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04d9400640 1 --2- 172.21.15.32:0/3677210919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04e40af660 0x7f04e4151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:48.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04e1e00640 1 --2- 172.21.15.32:0/3677210919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e406f570 0x7f04e4167e40 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f04d40ad040 tx=0x7f04d401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:48.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04daa00640 1 -- 172.21.15.32:0/3677210919 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d40bf610 con 0x7f04e406f570 2024-04-17T09:56:48.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04daa00640 1 -- 172.21.15.32:0/3677210919 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04d40bf7d0 con 0x7f04e406f570 2024-04-17T09:56:48.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04daa00640 1 -- 172.21.15.32:0/3677210919 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d40bfbc0 con 0x7f04e406f570 2024-04-17T09:56:48.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04e3200640 1 -- 172.21.15.32:0/3677210919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e4150a00 con 0x7f04e406f570 2024-04-17T09:56:48.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.168+0000 7f04e3200640 1 -- 172.21.15.32:0/3677210919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e4150fc0 con 0x7f04e406f570 2024-04-17T09:56:48.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.172+0000 7f04e3200640 1 -- 172.21.15.32:0/3677210919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e404fd00 con 0x7f04e406f570 2024-04-17T09:56:48.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.172+0000 7f04daa00640 1 -- 172.21.15.32:0/3677210919 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f04d40bd020 con 0x7f04e406f570 2024-04-17T09:56:48.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.172+0000 7f04daa00640 1 --2- 172.21.15.32:0/3677210919 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f04c0068380 0x7f04c006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:48.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.172+0000 7f04daa00640 1 -- 172.21.15.32:0/3677210919 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f04d4120ed0 con 0x7f04e406f570 2024-04-17T09:56:48.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.172+0000 7f04d9400640 1 --2- 172.21.15.32:0/3677210919 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f04c0068380 0x7f04c006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:48.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.172+0000 7f04d9400640 1 --2- 172.21.15.32:0/3677210919 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f04c0068380 0x7f04c006a840 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f04e41690a0 tx=0x7f04cc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:48.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.176+0000 7f04daa00640 1 -- 172.21.15.32:0/3677210919 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f04d40c6030 con 0x7f04e406f570 2024-04-17T09:56:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.376+0000 7f04e3200640 1 -- 172.21.15.32:0/3677210919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f04e40aeb80 con 0x7f04e406f570 2024-04-17T09:56:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.376+0000 7f04daa00640 1 -- 172.21.15.32:0/3677210919 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f04e4150780 con 0x7f04e406f570 2024-04-17T09:56:48.378 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:48.378 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-17T09:56:48.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.380+0000 7f04e3200640 1 -- 172.21.15.32:0/3677210919 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f04c0068380 msgr2=0x7f04c006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:48.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.380+0000 7f04e3200640 1 --2- 172.21.15.32:0/3677210919 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f04c0068380 0x7f04c006a840 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f04e41690a0 tx=0x7f04cc008040 comp rx=0 tx=0).stop 2024-04-17T09:56:48.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.380+0000 7f04e3200640 1 -- 172.21.15.32:0/3677210919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e406f570 msgr2=0x7f04e4167e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:48.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.380+0000 7f04e3200640 1 --2- 172.21.15.32:0/3677210919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e406f570 0x7f04e4167e40 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f04d40ad040 tx=0x7f04d401acb0 comp rx=0 tx=0).stop 2024-04-17T09:56:48.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.380+0000 7f04e3200640 1 -- 172.21.15.32:0/3677210919 shutdown_connections 2024-04-17T09:56:48.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.380+0000 7f04e3200640 1 --2- 172.21.15.32:0/3677210919 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f04c0068380 0x7f04c006a840 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:48.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.380+0000 7f04e3200640 1 --2- 172.21.15.32:0/3677210919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04e415b800 0x7f04e4152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:48.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.380+0000 7f04e3200640 1 --2- 172.21.15.32:0/3677210919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04e40af660 0x7f04e4151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:48.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.380+0000 7f04e3200640 1 --2- 172.21.15.32:0/3677210919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e406f570 0x7f04e4167e40 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:48.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.380+0000 7f04e3200640 1 -- 172.21.15.32:0/3677210919 >> 172.21.15.32:0/3677210919 conn(0x7f04e40d0180 msgr2=0x7f04e415f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:48.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.380+0000 7f04e3200640 1 -- 172.21.15.32:0/3677210919 shutdown_connections 2024-04-17T09:56:48.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:48.380+0000 7f04e3200640 1 -- 172.21.15.32:0/3677210919 wait complete. 2024-04-17T09:56:48.399 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2024-04-17T09:56:53.401 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-17T09:56:53.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.552+0000 7f0823e00640 1 Processor -- start 2024-04-17T09:56:53.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.552+0000 7f0823e00640 1 -- start start 2024-04-17T09:56:53.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.552+0000 7f0823e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f081c072500 0x7f081c0cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:53.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.552+0000 7f0823e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f081c0cdac0 con 0x7f081c15d8d0 2024-04-17T09:56:53.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.552+0000 7f0823e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f081c15cdf0 con 0x7f081c145f30 2024-04-17T09:56:53.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.552+0000 7f0823e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f081c06f400 con 0x7f081c072500 2024-04-17T09:56:53.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.552+0000 7f0820a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f081c072500 0x7f081c0cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:53.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.552+0000 7f0821e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f081c15d8d0 0x7f081c1601d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44188/0 (socket says 172.21.15.32:44188) 2024-04-17T09:56:53.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.552+0000 7f0821e00640 1 -- 172.21.15.32:0/232297332 learned_addr learned my addr 172.21.15.32:0/232297332 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:53.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.556+0000 7f081be00640 1 -- 172.21.15.32:0/232297332 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 861901611 0 0) 0x7f081c0cdac0 con 0x7f081c15d8d0 2024-04-17T09:56:53.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.556+0000 7f081be00640 1 -- 172.21.15.32:0/232297332 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0804003530 con 0x7f081c15d8d0 2024-04-17T09:56:53.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.556+0000 7f0820a00640 1 -- 172.21.15.32:0/232297332 >> v1:172.21.15.144:6789/0 conn(0x7f081c145f30 legacy=0x7f081c071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:53.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.556+0000 7f0820a00640 1 -- 172.21.15.32:0/232297332 >> v1:172.21.15.32:6789/0 conn(0x7f081c15d8d0 legacy=0x7f081c1601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:53.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.556+0000 7f0820a00640 1 -- 172.21.15.32:0/232297332 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081c15d0d0 con 0x7f081c072500 2024-04-17T09:56:53.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.556+0000 7f0820a00640 1 --2- 172.21.15.32:0/232297332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f081c072500 0x7f081c0cce70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f08080097c0 tx=0x7f080801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf45b2aa94b23d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:53.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.556+0000 7f081be00640 1 -- 172.21.15.32:0/232297332 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08080b5070 con 0x7f081c072500 2024-04-17T09:56:53.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.556+0000 7f081be00640 1 -- 172.21.15.32:0/232297332 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08080a8990 con 0x7f081c072500 2024-04-17T09:56:53.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.556+0000 7f081be00640 1 -- 172.21.15.32:0/232297332 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08080a8d60 con 0x7f081c072500 2024-04-17T09:56:53.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.556+0000 7f0823e00640 1 -- 172.21.15.32:0/232297332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f081c072500 msgr2=0x7f081c0cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:53.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.556+0000 7f0823e00640 1 --2- 172.21.15.32:0/232297332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f081c072500 0x7f081c0cce70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f08080097c0 tx=0x7f080801ec90 comp rx=0 tx=0).stop 2024-04-17T09:56:53.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.560+0000 7f0823e00640 1 -- 172.21.15.32:0/232297332 shutdown_connections 2024-04-17T09:56:53.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.560+0000 7f0823e00640 1 --2- 172.21.15.32:0/232297332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f081c072500 0x7f081c0cce70 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:53.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.560+0000 7f0823e00640 1 -- 172.21.15.32:0/232297332 >> 172.21.15.32:0/232297332 conn(0x7f081c06ba30 msgr2=0x7f081c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:53.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.560+0000 7f0823e00640 1 -- 172.21.15.32:0/232297332 shutdown_connections 2024-04-17T09:56:53.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.560+0000 7f0823e00640 1 -- 172.21.15.32:0/232297332 wait complete. 2024-04-17T09:56:53.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.560+0000 7f0823e00640 1 Processor -- start 2024-04-17T09:56:53.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.560+0000 7f0823e00640 1 -- start start 2024-04-17T09:56:53.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.560+0000 7f0823e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f081c072500 0x7f081c1392f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:53.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.560+0000 7f0823e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f081c145f30 0x7f081c0a9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.560+0000 7f0823e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f081c15d8d0 0x7f081c0a9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.560+0000 7f0823e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f081c0cd800 con 0x7f081c15d8d0 2024-04-17T09:56:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.560+0000 7f0823e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f081c0cd680 con 0x7f081c145f30 2024-04-17T09:56:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.560+0000 7f0823e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f081c146310 con 0x7f081c072500 2024-04-17T09:56:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.560+0000 7f0821e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f081c15d8d0 0x7f081c0a9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.560+0000 7f0821e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f081c15d8d0 0x7f081c0a9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34718/0 (socket says 172.21.15.32:34718) 2024-04-17T09:56:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.560+0000 7f0821e00640 1 -- 172.21.15.32:0/3221623019 learned_addr learned my addr 172.21.15.32:0/3221623019 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.560+0000 7f0820a00640 1 --2- 172.21.15.32:0/3221623019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f081c145f30 0x7f081c0a9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.560+0000 7f0821400640 1 --2- 172.21.15.32:0/3221623019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f081c072500 0x7f081c1392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.560+0000 7f0821e00640 1 -- 172.21.15.32:0/3221623019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f081c072500 msgr2=0x7f081c1392f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.560+0000 7f0821e00640 1 --2- 172.21.15.32:0/3221623019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f081c072500 0x7f081c1392f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.560+0000 7f0821e00640 1 -- 172.21.15.32:0/3221623019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f081c145f30 msgr2=0x7f081c0a9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.560+0000 7f0821e00640 1 --2- 172.21.15.32:0/3221623019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f081c145f30 0x7f081c0a9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.560+0000 7f0821e00640 1 -- 172.21.15.32:0/3221623019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081c1334b0 con 0x7f081c15d8d0 2024-04-17T09:56:53.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.564+0000 7f0821e00640 1 --2- 172.21.15.32:0/3221623019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f081c15d8d0 0x7f081c0a9ad0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f081400cad0 tx=0x7f081400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:53.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.564+0000 7f0819600640 1 -- 172.21.15.32:0/3221623019 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814017070 con 0x7f081c15d8d0 2024-04-17T09:56:53.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.564+0000 7f0819600640 1 -- 172.21.15.32:0/3221623019 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f081400ac30 con 0x7f081c15d8d0 2024-04-17T09:56:53.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.564+0000 7f0819600640 1 -- 172.21.15.32:0/3221623019 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814012400 con 0x7f081c15d8d0 2024-04-17T09:56:53.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.564+0000 7f0823e00640 1 -- 172.21.15.32:0/3221623019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f081c133720 con 0x7f081c15d8d0 2024-04-17T09:56:53.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.564+0000 7f0823e00640 1 -- 172.21.15.32:0/3221623019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f081c133ce0 con 0x7f081c15d8d0 2024-04-17T09:56:53.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.564+0000 7f0823e00640 1 -- 172.21.15.32:0/3221623019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f081c068a90 con 0x7f081c15d8d0 2024-04-17T09:56:53.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.564+0000 7f0819600640 1 -- 172.21.15.32:0/3221623019 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f081401a070 con 0x7f081c15d8d0 2024-04-17T09:56:53.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.564+0000 7f0819600640 1 --2- 172.21.15.32:0/3221623019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07fc068380 0x7f07fc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:53.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.564+0000 7f0819600640 1 -- 172.21.15.32:0/3221623019 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f0814086770 con 0x7f081c15d8d0 2024-04-17T09:56:53.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.564+0000 7f0821400640 1 --2- 172.21.15.32:0/3221623019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07fc068380 0x7f07fc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:53.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.564+0000 7f0821400640 1 --2- 172.21.15.32:0/3221623019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07fc068380 0x7f07fc06a840 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0810001810 tx=0x7f0810009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:53.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.568+0000 7f0819600640 1 -- 172.21.15.32:0/3221623019 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0814056640 con 0x7f081c15d8d0 2024-04-17T09:56:53.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.768+0000 7f0823e00640 1 -- 172.21.15.32:0/3221623019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f081c0ccc90 con 0x7f081c15d8d0 2024-04-17T09:56:53.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.768+0000 7f0819600640 1 -- 172.21.15.32:0/3221623019 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f081c0ccc90 con 0x7f081c15d8d0 2024-04-17T09:56:53.771 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:53.772 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-17T09:56:53.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.772+0000 7f0823e00640 1 -- 172.21.15.32:0/3221623019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07fc068380 msgr2=0x7f07fc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:53.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.772+0000 7f0823e00640 1 --2- 172.21.15.32:0/3221623019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07fc068380 0x7f07fc06a840 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0810001810 tx=0x7f0810009040 comp rx=0 tx=0).stop 2024-04-17T09:56:53.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.772+0000 7f0823e00640 1 -- 172.21.15.32:0/3221623019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f081c15d8d0 msgr2=0x7f081c0a9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:53.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.772+0000 7f0823e00640 1 --2- 172.21.15.32:0/3221623019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f081c15d8d0 0x7f081c0a9ad0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f081400cad0 tx=0x7f081400cf90 comp rx=0 tx=0).stop 2024-04-17T09:56:53.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.772+0000 7f0823e00640 1 -- 172.21.15.32:0/3221623019 shutdown_connections 2024-04-17T09:56:53.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.772+0000 7f0823e00640 1 --2- 172.21.15.32:0/3221623019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07fc068380 0x7f07fc06a840 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:53.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.772+0000 7f0823e00640 1 --2- 172.21.15.32:0/3221623019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f081c15d8d0 0x7f081c0a9ad0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:53.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.772+0000 7f0823e00640 1 --2- 172.21.15.32:0/3221623019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f081c145f30 0x7f081c0a9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:53.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.772+0000 7f0823e00640 1 --2- 172.21.15.32:0/3221623019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f081c072500 0x7f081c1392f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:53.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.772+0000 7f0823e00640 1 -- 172.21.15.32:0/3221623019 >> 172.21.15.32:0/3221623019 conn(0x7f081c06ba30 msgr2=0x7f081c071df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:53.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.772+0000 7f0823e00640 1 -- 172.21.15.32:0/3221623019 shutdown_connections 2024-04-17T09:56:53.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:53.772+0000 7f0823e00640 1 -- 172.21.15.32:0/3221623019 wait complete. 2024-04-17T09:56:53.793 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-17T09:56:53.793 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T09:56:53.793 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T09:56:53.874 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T09:56:53.921 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T09:56:53.922 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T09:56:53.922 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:56:53.922 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:56:53.931 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:56:54.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fc8000640 1 Processor -- start 2024-04-17T09:56:54.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fc8000640 1 -- start start 2024-04-17T09:56:54.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fc8000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fc006f630 0x7f2fc0072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:54.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fc8000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fc00b3490 con 0x7f2fc0072bc0 2024-04-17T09:56:54.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fc8000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fc00b58d0 con 0x7f2fc0151db0 2024-04-17T09:56:54.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fc8000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc00b44f0 con 0x7f2fc006f630 2024-04-17T09:56:54.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fc6000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2fc0072bc0 0x7f2fc006de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44196/0 (socket says 172.21.15.32:44196) 2024-04-17T09:56:54.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fc6000640 1 -- 172.21.15.32:0/3598176671 learned_addr learned my addr 172.21.15.32:0/3598176671 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:54.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fc4c00640 1 --2- 172.21.15.32:0/3598176671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fc006f630 0x7f2fc0072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:54.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fbfe00640 1 -- 172.21.15.32:0/3598176671 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2047858376 0 0) 0x7f2fc00b3490 con 0x7f2fc0072bc0 2024-04-17T09:56:54.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fbfe00640 1 -- 172.21.15.32:0/3598176671 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fa8003530 con 0x7f2fc0072bc0 2024-04-17T09:56:54.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fbfe00640 1 -- 172.21.15.32:0/3598176671 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 89637431 0 0) 0x7f2fa8003530 con 0x7f2fc0072bc0 2024-04-17T09:56:54.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fbfe00640 1 -- 172.21.15.32:0/3598176671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fc006f630 msgr2=0x7f2fc0072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:54.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fbfe00640 1 --2- 172.21.15.32:0/3598176671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fc006f630 0x7f2fc0072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fbfe00640 1 -- 172.21.15.32:0/3598176671 >> v1:172.21.15.144:6789/0 conn(0x7f2fc0151db0 legacy=0x7f2fc00b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:54.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fbfe00640 1 -- 172.21.15.32:0/3598176671 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc00cea40 con 0x7f2fc0072bc0 2024-04-17T09:56:54.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fbfe00640 1 -- 172.21.15.32:0/3598176671 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f2fb8001a00 con 0x7f2fc0072bc0 2024-04-17T09:56:54.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fbfe00640 1 -- 172.21.15.32:0/3598176671 >> v1:172.21.15.32:6789/0 conn(0x7f2fc0072bc0 legacy=0x7f2fc006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:54.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fbfe00640 1 --2- 172.21.15.32:0/3598176671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fa80039f0 0x7f2fa8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:54.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fbfe00640 1 --2- 172.21.15.32:0/3598176671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fa8004810 0x7f2fa8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:54.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fbfe00640 1 --2- 172.21.15.32:0/3598176671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fc006f630 0x7f2fa80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:54.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fbfe00640 1 -- 172.21.15.32:0/3598176671 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc00b44f0 con 0x7f2fa80039f0 2024-04-17T09:56:54.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fbfe00640 1 -- 172.21.15.32:0/3598176671 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa800ba80 con 0x7f2fa8004810 2024-04-17T09:56:54.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fbfe00640 1 -- 172.21.15.32:0/3598176671 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2fa800bc80 con 0x7f2fc006f630 2024-04-17T09:56:54.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fc5600640 1 --2- 172.21.15.32:0/3598176671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fa80039f0 0x7f2fa8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:54.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fc4c00640 1 --2- 172.21.15.32:0/3598176671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fa8004810 0x7f2fa8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:54.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fc6000640 1 --2- 172.21.15.32:0/3598176671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fc006f630 0x7f2fa80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:54.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fc5600640 1 -- 172.21.15.32:0/3598176671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fc006f630 msgr2=0x7f2fa80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:54.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fc5600640 1 --2- 172.21.15.32:0/3598176671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fc006f630 0x7f2fa80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fc5600640 1 -- 172.21.15.32:0/3598176671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fa8004810 msgr2=0x7f2fa8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:54.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fc5600640 1 --2- 172.21.15.32:0/3598176671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fa8004810 0x7f2fa8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fc5600640 1 -- 172.21.15.32:0/3598176671 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa8003530 con 0x7f2fa80039f0 2024-04-17T09:56:54.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.112+0000 7f2fc6000640 1 --2- 172.21.15.32:0/3598176671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fc006f630 0x7f2fa80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:54.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.116+0000 7f2fc5600640 1 --2- 172.21.15.32:0/3598176671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fa80039f0 0x7f2fa8003e10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00b5aa0 tx=0x7f2fb401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44730440a4a8c5c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:54.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.116+0000 7f2fbfe00640 1 -- 172.21.15.32:0/3598176671 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb40013a0 con 0x7f2fa80039f0 2024-04-17T09:56:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.116+0000 7f2fbfe00640 1 -- 172.21.15.32:0/3598176671 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb40052d0 con 0x7f2fa80039f0 2024-04-17T09:56:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.116+0000 7f2fbfe00640 1 -- 172.21.15.32:0/3598176671 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4003820 con 0x7f2fa80039f0 2024-04-17T09:56:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.116+0000 7f2fc8000640 1 -- 172.21.15.32:0/3598176671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fa80039f0 msgr2=0x7f2fa8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.116+0000 7f2fc8000640 1 --2- 172.21.15.32:0/3598176671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fa80039f0 0x7f2fa8003e10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00b5aa0 tx=0x7f2fb401c3a0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.116+0000 7f2fc6000640 1 -- 172.21.15.32:0/3598176671 reap_dead start 2024-04-17T09:56:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.116+0000 7f2fc8000640 1 -- 172.21.15.32:0/3598176671 shutdown_connections 2024-04-17T09:56:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.116+0000 7f2fc8000640 1 -- 172.21.15.32:0/3598176671 >> 172.21.15.32:0/3598176671 conn(0x7f2fc006ba30 msgr2=0x7f2fc006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.116+0000 7f2fc8000640 1 -- 172.21.15.32:0/3598176671 shutdown_connections 2024-04-17T09:56:54.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.116+0000 7f2fc8000640 1 -- 172.21.15.32:0/3598176671 wait complete. 2024-04-17T09:56:54.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fc8000640 1 Processor -- start 2024-04-17T09:56:54.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fc8000640 1 -- start start 2024-04-17T09:56:54.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fc8000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fa80039f0 0x7f2fc013f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:54.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fc8000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fc0159ca0 0x7f2fc015a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:54.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fc8000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fc0148640 0x7f2fc0148ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:54.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fc8000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc00b3fd0 con 0x7f2fc0159ca0 2024-04-17T09:56:54.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fc8000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc0073030 con 0x7f2fc0148640 2024-04-17T09:56:54.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fc8000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc0071df0 con 0x7f2fa80039f0 2024-04-17T09:56:54.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fc4c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fc0159ca0 0x7f2fc015a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:54.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fc4c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fc0159ca0 0x7f2fc015a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34732/0 (socket says 172.21.15.32:34732) 2024-04-17T09:56:54.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fc4c00640 1 -- 172.21.15.32:0/3160080689 learned_addr learned my addr 172.21.15.32:0/3160080689 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:54.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fc5600640 1 --2- 172.21.15.32:0/3160080689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fa80039f0 0x7f2fc013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:54.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fc6000640 1 --2- 172.21.15.32:0/3160080689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fc0148640 0x7f2fc0148ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:54.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fc4c00640 1 -- 172.21.15.32:0/3160080689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fa80039f0 msgr2=0x7f2fc013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:54.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fc4c00640 1 --2- 172.21.15.32:0/3160080689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fa80039f0 0x7f2fc013f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fc4c00640 1 -- 172.21.15.32:0/3160080689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fc0148640 msgr2=0x7f2fc0148ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:54.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fc4c00640 1 --2- 172.21.15.32:0/3160080689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fc0148640 0x7f2fc0148ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fc4c00640 1 -- 172.21.15.32:0/3160080689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc0149350 con 0x7f2fc0159ca0 2024-04-17T09:56:54.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fc6000640 1 --2- 172.21.15.32:0/3160080689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fc0148640 0x7f2fc0148ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:54.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fc4c00640 1 --2- 172.21.15.32:0/3160080689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fc0159ca0 0x7f2fc015a120 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2fac011940 tx=0x7f2fac011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:54.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fbd600640 1 -- 172.21.15.32:0/3160080689 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac019070 con 0x7f2fc0159ca0 2024-04-17T09:56:54.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fbd600640 1 -- 172.21.15.32:0/3160080689 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fac004240 con 0x7f2fc0159ca0 2024-04-17T09:56:54.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fbd600640 1 -- 172.21.15.32:0/3160080689 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac00f8e0 con 0x7f2fc0159ca0 2024-04-17T09:56:54.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fc8000640 1 -- 172.21.15.32:0/3160080689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc015d5f0 con 0x7f2fc0159ca0 2024-04-17T09:56:54.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.120+0000 7f2fc8000640 1 -- 172.21.15.32:0/3160080689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc015dbb0 con 0x7f2fc0159ca0 2024-04-17T09:56:54.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.124+0000 7f2fc8000640 1 -- 172.21.15.32:0/3160080689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fc0068a90 con 0x7f2fc0159ca0 2024-04-17T09:56:54.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.124+0000 7f2fbd600640 1 -- 172.21.15.32:0/3160080689 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2fac01c070 con 0x7f2fc0159ca0 2024-04-17T09:56:54.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.124+0000 7f2fbd600640 1 --2- 172.21.15.32:0/3160080689 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2fa0068380 0x7f2fa006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:54.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.124+0000 7f2fbd600640 1 -- 172.21.15.32:0/3160080689 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f2fac086d70 con 0x7f2fc0159ca0 2024-04-17T09:56:54.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.124+0000 7f2fc5600640 1 --2- 172.21.15.32:0/3160080689 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2fa0068380 0x7f2fa006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:54.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.124+0000 7f2fc5600640 1 --2- 172.21.15.32:0/3160080689 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2fa0068380 0x7f2fa006a840 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2fa80057f0 tx=0x7f2fb40011a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:54.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.128+0000 7f2fbd600640 1 -- 172.21.15.32:0/3160080689 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2fac056cd0 con 0x7f2fc0159ca0 2024-04-17T09:56:54.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.300+0000 7f2fc8000640 1 -- 172.21.15.32:0/3160080689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2fc0140450 con 0x7f2fc0159ca0 2024-04-17T09:56:54.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.300+0000 7f2fbd600640 1 -- 172.21.15.32:0/3160080689 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16677 (secure 0 0 0) 0x7f2fac05afd0 con 0x7f2fc0159ca0 2024-04-17T09:56:54.301 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:54.301 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":57,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:56:03.192930+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:55:59.976232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-17T09:56:00.543788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:56:54.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.300+0000 7f2fc8000640 1 -- 172.21.15.32:0/3160080689 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2fa0068380 msgr2=0x7f2fa006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:54.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.300+0000 7f2fc8000640 1 --2- 172.21.15.32:0/3160080689 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2fa0068380 0x7f2fa006a840 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2fa80057f0 tx=0x7f2fb40011a0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.300+0000 7f2fc8000640 1 -- 172.21.15.32:0/3160080689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fc0159ca0 msgr2=0x7f2fc015a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:54.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.300+0000 7f2fc8000640 1 --2- 172.21.15.32:0/3160080689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fc0159ca0 0x7f2fc015a120 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2fac011940 tx=0x7f2fac011e00 comp rx=0 tx=0).stop 2024-04-17T09:56:54.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.300+0000 7f2fc8000640 1 -- 172.21.15.32:0/3160080689 shutdown_connections 2024-04-17T09:56:54.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.300+0000 7f2fc8000640 1 --2- 172.21.15.32:0/3160080689 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2fa0068380 0x7f2fa006a840 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.300+0000 7f2fc8000640 1 --2- 172.21.15.32:0/3160080689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fc0148640 0x7f2fc0148ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.300+0000 7f2fc8000640 1 --2- 172.21.15.32:0/3160080689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fc0159ca0 0x7f2fc015a120 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.300+0000 7f2fc8000640 1 --2- 172.21.15.32:0/3160080689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fa80039f0 0x7f2fc013f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.300+0000 7f2fc8000640 1 -- 172.21.15.32:0/3160080689 >> 172.21.15.32:0/3160080689 conn(0x7f2fc006ba30 msgr2=0x7f2fc00cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:54.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.300+0000 7f2fc8000640 1 -- 172.21.15.32:0/3160080689 shutdown_connections 2024-04-17T09:56:54.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.300+0000 7f2fc8000640 1 -- 172.21.15.32:0/3160080689 wait complete. 2024-04-17T09:56:54.322 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:55:59.976232+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:56:00.543788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:56:54.323 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:56:54.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.464+0000 7fcc3f000640 1 Processor -- start 2024-04-17T09:56:54.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.464+0000 7fcc3f000640 1 -- start start 2024-04-17T09:56:54.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.464+0000 7fcc3f000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc3806c140 0x7fcc380ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:54.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.464+0000 7fcc3f000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc380cc940 con 0x7fcc3806f7c0 2024-04-17T09:56:54.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.464+0000 7fcc3f000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc380ccc20 con 0x7fcc3815b7d0 2024-04-17T09:56:54.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.464+0000 7fcc3f000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fcc3806e0b0 con 0x7fcc3806c140 2024-04-17T09:56:54.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.464+0000 7fcc2fe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcc3806f7c0 0x7fcc3806ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44208/0 (socket says 172.21.15.32:44208) 2024-04-17T09:56:54.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.464+0000 7fcc2fe00640 1 -- 172.21.15.32:0/2941398156 learned_addr learned my addr 172.21.15.32:0/2941398156 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:54.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.464+0000 7fcc3d000640 1 --2- 172.21.15.32:0/2941398156 >> v2:172.21.15.32:3300/0 conn(0x7fcc3806c140 0x7fcc380ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.468+0000 7fcc37400640 1 -- 172.21.15.32:0/2941398156 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2837232556 0 0) 0x7fcc380cc940 con 0x7fcc3806f7c0 2024-04-17T09:56:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.468+0000 7fcc37400640 1 -- 172.21.15.32:0/2941398156 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc1c005740 con 0x7fcc3806f7c0 2024-04-17T09:56:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.468+0000 7fcc3d000640 1 -- 172.21.15.32:0/2941398156 >> v1:172.21.15.144:6789/0 conn(0x7fcc3815b7d0 legacy=0x7fcc3806e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.468+0000 7fcc3d000640 1 -- 172.21.15.32:0/2941398156 >> v1:172.21.15.32:6789/0 conn(0x7fcc3806f7c0 legacy=0x7fcc3806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:54.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.468+0000 7fcc3d000640 1 -- 172.21.15.32:0/2941398156 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3806e390 con 0x7fcc3806c140 2024-04-17T09:56:54.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.468+0000 7fcc3d000640 1 --2- 172.21.15.32:0/2941398156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc3806c140 0x7fcc380ce290 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009d90 tx=0x7fcc3001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f79e70f1a95dd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:54.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.468+0000 7fcc37400640 1 -- 172.21.15.32:0/2941398156 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc300b2070 con 0x7fcc3806c140 2024-04-17T09:56:54.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.468+0000 7fcc37400640 1 -- 172.21.15.32:0/2941398156 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc300a59c0 con 0x7fcc3806c140 2024-04-17T09:56:54.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.468+0000 7fcc37400640 1 -- 172.21.15.32:0/2941398156 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc300a5d70 con 0x7fcc3806c140 2024-04-17T09:56:54.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc3f000640 1 -- 172.21.15.32:0/2941398156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc3806c140 msgr2=0x7fcc380ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc3f000640 1 --2- 172.21.15.32:0/2941398156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc3806c140 0x7fcc380ce290 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009d90 tx=0x7fcc3001c8e0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc3f000640 1 -- 172.21.15.32:0/2941398156 shutdown_connections 2024-04-17T09:56:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc3f000640 1 --2- 172.21.15.32:0/2941398156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc3806c140 0x7fcc380ce290 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc3f000640 1 -- 172.21.15.32:0/2941398156 >> 172.21.15.32:0/2941398156 conn(0x7fcc380b2120 msgr2=0x7fcc380b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc3f000640 1 -- 172.21.15.32:0/2941398156 shutdown_connections 2024-04-17T09:56:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc3f000640 1 -- 172.21.15.32:0/2941398156 wait complete. 2024-04-17T09:56:54.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc3f000640 1 Processor -- start 2024-04-17T09:56:54.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc3f000640 1 -- start start 2024-04-17T09:56:54.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc3f000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc3806c140 0x7fcc3815d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:54.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc3f000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc3806f7c0 0x7fcc3815d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:54.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc3f000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc3815b7d0 0x7fcc38148030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc3f000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38072270 con 0x7fcc3806f7c0 2024-04-17T09:56:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc3f000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcc380720f0 con 0x7fcc3815b7d0 2024-04-17T09:56:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc3f000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcc380723f0 con 0x7fcc3806c140 2024-04-17T09:56:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc2fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc3806f7c0 0x7fcc3815d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc2fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc3806f7c0 0x7fcc3815d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34746/0 (socket says 172.21.15.32:34746) 2024-04-17T09:56:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc2fe00640 1 -- 172.21.15.32:0/2317176418 learned_addr learned my addr 172.21.15.32:0/2317176418 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc37e00640 1 --2- 172.21.15.32:0/2317176418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc3806c140 0x7fcc3815d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc3d000640 1 --2- 172.21.15.32:0/2317176418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc3815b7d0 0x7fcc38148030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc2fe00640 1 -- 172.21.15.32:0/2317176418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc3806c140 msgr2=0x7fcc3815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc2fe00640 1 --2- 172.21.15.32:0/2317176418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc3806c140 0x7fcc3815d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc2fe00640 1 -- 172.21.15.32:0/2317176418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc3815b7d0 msgr2=0x7fcc38148030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc2fe00640 1 --2- 172.21.15.32:0/2317176418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc3815b7d0 0x7fcc38148030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc2fe00640 1 -- 172.21.15.32:0/2317176418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3815c160 con 0x7fcc3806f7c0 2024-04-17T09:56:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.472+0000 7fcc2fe00640 1 --2- 172.21.15.32:0/2317176418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc3806f7c0 0x7fcc3815d740 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000c500 tx=0x7fcc2000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:54.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.476+0000 7fcc34c00640 1 -- 172.21.15.32:0/2317176418 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20017070 con 0x7fcc3806f7c0 2024-04-17T09:56:54.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.476+0000 7fcc34c00640 1 -- 172.21.15.32:0/2317176418 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc20005cf0 con 0x7fcc3806f7c0 2024-04-17T09:56:54.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.476+0000 7fcc34c00640 1 -- 172.21.15.32:0/2317176418 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc200072f0 con 0x7fcc3806f7c0 2024-04-17T09:56:54.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.476+0000 7fcc3f000640 1 -- 172.21.15.32:0/2317176418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3815c440 con 0x7fcc3806f7c0 2024-04-17T09:56:54.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.476+0000 7fcc3f000640 1 -- 172.21.15.32:0/2317176418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc3815ca00 con 0x7fcc3806f7c0 2024-04-17T09:56:54.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.476+0000 7fcc3f000640 1 -- 172.21.15.32:0/2317176418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc38068a90 con 0x7fcc3806f7c0 2024-04-17T09:56:54.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.476+0000 7fcc34c00640 1 -- 172.21.15.32:0/2317176418 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcc2001a070 con 0x7fcc3806f7c0 2024-04-17T09:56:54.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.476+0000 7fcc34c00640 1 --2- 172.21.15.32:0/2317176418 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc14068380 0x7fcc1406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:54.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.476+0000 7fcc34c00640 1 -- 172.21.15.32:0/2317176418 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fcc20085d50 con 0x7fcc3806f7c0 2024-04-17T09:56:54.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.476+0000 7fcc37e00640 1 --2- 172.21.15.32:0/2317176418 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc14068380 0x7fcc1406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:54.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.476+0000 7fcc37e00640 1 --2- 172.21.15.32:0/2317176418 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc14068380 0x7fcc1406a840 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcc28001770 tx=0x7fcc28009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:54.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.480+0000 7fcc34c00640 1 -- 172.21.15.32:0/2317176418 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fcc20055cb0 con 0x7fcc3806f7c0 2024-04-17T09:56:54.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.644+0000 7fcc3f000640 1 -- 172.21.15.32:0/2317176418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcc3806bb40 con 0x7fcc3806f7c0 2024-04-17T09:56:54.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.644+0000 7fcc34c00640 1 -- 172.21.15.32:0/2317176418 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) ==== 93+0+10 (secure 0 0 0) 0x7fcc20059fb0 con 0x7fcc3806f7c0 2024-04-17T09:56:54.647 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:56:54.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.648+0000 7fcc3f000640 1 -- 172.21.15.32:0/2317176418 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc14068380 msgr2=0x7fcc1406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:54.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.648+0000 7fcc3f000640 1 --2- 172.21.15.32:0/2317176418 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc14068380 0x7fcc1406a840 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcc28001770 tx=0x7fcc28009040 comp rx=0 tx=0).stop 2024-04-17T09:56:54.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.648+0000 7fcc3f000640 1 -- 172.21.15.32:0/2317176418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc3806f7c0 msgr2=0x7fcc3815d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:54.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.648+0000 7fcc3f000640 1 --2- 172.21.15.32:0/2317176418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc3806f7c0 0x7fcc3815d740 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000c500 tx=0x7fcc2000c9c0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.648+0000 7fcc3f000640 1 -- 172.21.15.32:0/2317176418 shutdown_connections 2024-04-17T09:56:54.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.648+0000 7fcc3f000640 1 --2- 172.21.15.32:0/2317176418 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc14068380 0x7fcc1406a840 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.648+0000 7fcc3f000640 1 --2- 172.21.15.32:0/2317176418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc3815b7d0 0x7fcc38148030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.648+0000 7fcc3f000640 1 --2- 172.21.15.32:0/2317176418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc3806f7c0 0x7fcc3815d740 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.648+0000 7fcc3f000640 1 --2- 172.21.15.32:0/2317176418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc3806c140 0x7fcc3815d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.648+0000 7fcc3f000640 1 -- 172.21.15.32:0/2317176418 >> 172.21.15.32:0/2317176418 conn(0x7fcc380b2120 msgr2=0x7fcc38159fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:54.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.648+0000 7fcc3f000640 1 -- 172.21.15.32:0/2317176418 shutdown_connections 2024-04-17T09:56:54.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.648+0000 7fcc3f000640 1 -- 172.21.15.32:0/2317176418 wait complete. 2024-04-17T09:56:54.669 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:56:54.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.812+0000 7f92b5400640 1 Processor -- start 2024-04-17T09:56:54.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.812+0000 7f92b5400640 1 -- start start 2024-04-17T09:56:54.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.812+0000 7f92b5400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92b006c140 0x7f92b00ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:54.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.812+0000 7f92b5400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b00cc940 con 0x7f92b015b7d0 2024-04-17T09:56:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.812+0000 7f92b5400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b00ccc20 con 0x7f92b006f7c0 2024-04-17T09:56:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.812+0000 7f92b5400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f92b006e0b0 con 0x7f92b006c140 2024-04-17T09:56:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.812+0000 7f92ae800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f92b015b7d0 0x7f92b006e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44220/0 (socket says 172.21.15.32:44220) 2024-04-17T09:56:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.812+0000 7f92ae800640 1 -- 172.21.15.32:0/556573881 learned_addr learned my addr 172.21.15.32:0/556573881 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.812+0000 7f92af200640 1 --2- 172.21.15.32:0/556573881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92b006c140 0x7f92b00ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.812+0000 7f92ade00640 1 -- 172.21.15.32:0/556573881 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1483673629 0 0) 0x7f92b00cc940 con 0x7f92b015b7d0 2024-04-17T09:56:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.812+0000 7f92ade00640 1 -- 172.21.15.32:0/556573881 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9294003530 con 0x7f92b015b7d0 2024-04-17T09:56:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.812+0000 7f92ade00640 1 -- 172.21.15.32:0/556573881 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2025156620 0 0) 0x7f92b00ccc20 con 0x7f92b006f7c0 2024-04-17T09:56:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.812+0000 7f92ade00640 1 -- 172.21.15.32:0/556573881 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92b00cc940 con 0x7f92b006f7c0 2024-04-17T09:56:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.812+0000 7f92af200640 1 -- 172.21.15.32:0/556573881 >> v1:172.21.15.144:6789/0 conn(0x7f92b006f7c0 legacy=0x7f92b006ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.812+0000 7f92af200640 1 -- 172.21.15.32:0/556573881 >> v1:172.21.15.32:6789/0 conn(0x7f92b015b7d0 legacy=0x7f92b006e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.812+0000 7f92ade00640 1 -- 172.21.15.32:0/556573881 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 856221141 0 0) 0x7f9294003530 con 0x7f92b015b7d0 2024-04-17T09:56:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.812+0000 7f92af200640 1 -- 172.21.15.32:0/556573881 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b006e390 con 0x7f92b006c140 2024-04-17T09:56:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.816+0000 7f92af200640 1 --2- 172.21.15.32:0/556573881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92b006c140 0x7f92b00ce290 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f92a80097f0 tx=0x7f92a801c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcfa3fdfaf08e5c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.816+0000 7f92ade00640 1 -- 172.21.15.32:0/556573881 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a80b1070 con 0x7f92b006c140 2024-04-17T09:56:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.816+0000 7f92ade00640 1 -- 172.21.15.32:0/556573881 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92a80ac440 con 0x7f92b006c140 2024-04-17T09:56:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.816+0000 7f92ade00640 1 -- 172.21.15.32:0/556573881 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a0001a00 con 0x7f92b006c140 2024-04-17T09:56:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.816+0000 7f92b5400640 1 -- 172.21.15.32:0/556573881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92b006c140 msgr2=0x7f92b00ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.816+0000 7f92b5400640 1 --2- 172.21.15.32:0/556573881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92b006c140 0x7f92b00ce290 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f92a80097f0 tx=0x7f92a801c370 comp rx=0 tx=0).stop 2024-04-17T09:56:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.816+0000 7f92b5400640 1 -- 172.21.15.32:0/556573881 shutdown_connections 2024-04-17T09:56:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.816+0000 7f92b5400640 1 --2- 172.21.15.32:0/556573881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92b006c140 0x7f92b00ce290 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.816+0000 7f92b5400640 1 -- 172.21.15.32:0/556573881 >> 172.21.15.32:0/556573881 conn(0x7f92b00b2120 msgr2=0x7f92b00b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.816+0000 7f92b5400640 1 -- 172.21.15.32:0/556573881 shutdown_connections 2024-04-17T09:56:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.816+0000 7f92b5400640 1 -- 172.21.15.32:0/556573881 wait complete. 2024-04-17T09:56:54.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92b5400640 1 Processor -- start 2024-04-17T09:56:54.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92b5400640 1 -- start start 2024-04-17T09:56:54.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92b5400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92b006c140 0x7f92b0150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:54.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92b5400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92b006f7c0 0x7f92b013dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:54.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92b5400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b015b7d0 0x7f92b00ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:54.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92b5400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f92b0072270 con 0x7f92b015b7d0 2024-04-17T09:56:54.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92b5400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f92b00720f0 con 0x7f92b006c140 2024-04-17T09:56:54.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92b5400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f92b00723f0 con 0x7f92b006f7c0 2024-04-17T09:56:54.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92af200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b015b7d0 0x7f92b00ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:54.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92af200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b015b7d0 0x7f92b00ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34748/0 (socket says 172.21.15.32:34748) 2024-04-17T09:56:54.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92af200640 1 -- 172.21.15.32:0/835602704 learned_addr learned my addr 172.21.15.32:0/835602704 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:54.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92ae800640 1 --2- 172.21.15.32:0/835602704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92b006c140 0x7f92b0150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:54.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92a5e00640 1 --2- 172.21.15.32:0/835602704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92b006f7c0 0x7f92b013dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:54.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92af200640 1 -- 172.21.15.32:0/835602704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92b006f7c0 msgr2=0x7f92b013dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:54.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92af200640 1 --2- 172.21.15.32:0/835602704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92b006f7c0 0x7f92b013dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92af200640 1 -- 172.21.15.32:0/835602704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92b006c140 msgr2=0x7f92b0150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:54.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92af200640 1 --2- 172.21.15.32:0/835602704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92b006c140 0x7f92b0150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92af200640 1 -- 172.21.15.32:0/835602704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b014e5e0 con 0x7f92b015b7d0 2024-04-17T09:56:54.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92af200640 1 --2- 172.21.15.32:0/835602704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b015b7d0 0x7f92b00ae8b0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009740 tx=0x7f92a801abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:54.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92a7400640 1 -- 172.21.15.32:0/835602704 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a80b1050 con 0x7f92b015b7d0 2024-04-17T09:56:54.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92a7400640 1 -- 172.21.15.32:0/835602704 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92a80b2040 con 0x7f92b015b7d0 2024-04-17T09:56:54.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92a7400640 1 -- 172.21.15.32:0/835602704 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a8006e20 con 0x7f92b015b7d0 2024-04-17T09:56:54.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92b5400640 1 -- 172.21.15.32:0/835602704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b015eb00 con 0x7f92b015b7d0 2024-04-17T09:56:54.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.820+0000 7f92b5400640 1 -- 172.21.15.32:0/835602704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b015dd40 con 0x7f92b015b7d0 2024-04-17T09:56:54.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.824+0000 7f92b5400640 1 -- 172.21.15.32:0/835602704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b0068a90 con 0x7f92b015b7d0 2024-04-17T09:56:54.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.824+0000 7f92a7400640 1 -- 172.21.15.32:0/835602704 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f92a80bf020 con 0x7f92b015b7d0 2024-04-17T09:56:54.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.824+0000 7f92a7400640 1 --2- 172.21.15.32:0/835602704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f928c068380 0x7f928c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:54.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.824+0000 7f92a7400640 1 -- 172.21.15.32:0/835602704 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f92a8120790 con 0x7f92b015b7d0 2024-04-17T09:56:54.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.824+0000 7f92ae800640 1 --2- 172.21.15.32:0/835602704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f928c068380 0x7f928c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:54.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.824+0000 7f92ae800640 1 --2- 172.21.15.32:0/835602704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f928c068380 0x7f928c06a840 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f92a0001360 tx=0x7f92a0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:54.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.828+0000 7f92a7400640 1 -- 172.21.15.32:0/835602704 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f92a8015ff0 con 0x7f92b015b7d0 2024-04-17T09:56:54.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.992+0000 7f92b5400640 1 -- 172.21.15.32:0/835602704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f92b015a4d0 con 0x7f92b015b7d0 2024-04-17T09:56:54.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.992+0000 7f92a7400640 1 -- 172.21.15.32:0/835602704 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v57) ==== 104+0+11 (secure 0 0 0) 0x7f92a80f0810 con 0x7f92b015b7d0 2024-04-17T09:56:54.995 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:56:54.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.996+0000 7f92b5400640 1 -- 172.21.15.32:0/835602704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f928c068380 msgr2=0x7f928c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:54.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.996+0000 7f92b5400640 1 --2- 172.21.15.32:0/835602704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f928c068380 0x7f928c06a840 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f92a0001360 tx=0x7f92a0009040 comp rx=0 tx=0).stop 2024-04-17T09:56:54.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.996+0000 7f92b5400640 1 -- 172.21.15.32:0/835602704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b015b7d0 msgr2=0x7f92b00ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:54.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.996+0000 7f92b5400640 1 --2- 172.21.15.32:0/835602704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b015b7d0 0x7f92b00ae8b0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009740 tx=0x7f92a801abb0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.996+0000 7f92b5400640 1 -- 172.21.15.32:0/835602704 shutdown_connections 2024-04-17T09:56:54.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.996+0000 7f92b5400640 1 --2- 172.21.15.32:0/835602704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f928c068380 0x7f928c06a840 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.996+0000 7f92b5400640 1 --2- 172.21.15.32:0/835602704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92b006f7c0 0x7f92b013dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.996+0000 7f92b5400640 1 --2- 172.21.15.32:0/835602704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b015b7d0 0x7f92b00ae8b0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.996+0000 7f92b5400640 1 --2- 172.21.15.32:0/835602704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92b006c140 0x7f92b0150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:54.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.996+0000 7f92b5400640 1 -- 172.21.15.32:0/835602704 >> 172.21.15.32:0/835602704 conn(0x7f92b00b2120 msgr2=0x7f92b006e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:54.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.996+0000 7f92b5400640 1 -- 172.21.15.32:0/835602704 shutdown_connections 2024-04-17T09:56:54.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:54.996+0000 7f92b5400640 1 -- 172.21.15.32:0/835602704 wait complete. 2024-04-17T09:56:55.017 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:56:55.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.160+0000 7fcb7d000640 1 Processor -- start 2024-04-17T09:56:55.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.160+0000 7fcb7d000640 1 -- start start 2024-04-17T09:56:55.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.160+0000 7fcb7d000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb7815fac0 0x7fcb78162580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:55.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.160+0000 7fcb7d000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb780b4410 con 0x7fcb780b01e0 2024-04-17T09:56:55.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.160+0000 7fcb7d000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb780b46f0 con 0x7fcb78162ac0 2024-04-17T09:56:55.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.160+0000 7fcb7d000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fcb78162f30 con 0x7fcb7815fac0 2024-04-17T09:56:55.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.160+0000 7fcb75400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb7815fac0 0x7fcb78162580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:55.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.160+0000 7fcb75400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb7815fac0 0x7fcb78162580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34764/0 (socket says 172.21.15.32:34764) 2024-04-17T09:56:55.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.160+0000 7fcb75e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcb780b01e0 0x7fcb780b3c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44230/0 (socket says 172.21.15.32:44230) 2024-04-17T09:56:55.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.160+0000 7fcb75400640 1 -- 172.21.15.32:0/2187753389 learned_addr learned my addr 172.21.15.32:0/2187753389 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:55.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.160+0000 7fcb75400640 1 -- 172.21.15.32:0/2187753389 >> v1:172.21.15.144:6789/0 conn(0x7fcb78162ac0 legacy=0x7fcb7806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:55.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.160+0000 7fcb75400640 1 -- 172.21.15.32:0/2187753389 >> v1:172.21.15.32:6789/0 conn(0x7fcb780b01e0 legacy=0x7fcb780b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:55.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.160+0000 7fcb75400640 1 -- 172.21.15.32:0/2187753389 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb780b49d0 con 0x7fcb7815fac0 2024-04-17T09:56:55.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.164+0000 7fcb75400640 1 --2- 172.21.15.32:0/2187753389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7815fac0 0x7fcb78162580 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009870 tx=0x7fcb6401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d7eb8ab7f5b475 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:55.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.164+0000 7fcb74a00640 1 -- 172.21.15.32:0/2187753389 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb640b0070 con 0x7fcb7815fac0 2024-04-17T09:56:55.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.164+0000 7fcb74a00640 1 -- 172.21.15.32:0/2187753389 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb640ab600 con 0x7fcb7815fac0 2024-04-17T09:56:55.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.164+0000 7fcb74a00640 1 -- 172.21.15.32:0/2187753389 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb640ab9d0 con 0x7fcb7815fac0 2024-04-17T09:56:55.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.164+0000 7fcb7d000640 1 -- 172.21.15.32:0/2187753389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7815fac0 msgr2=0x7fcb78162580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:55.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.164+0000 7fcb7d000640 1 --2- 172.21.15.32:0/2187753389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7815fac0 0x7fcb78162580 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009870 tx=0x7fcb6401c4d0 comp rx=0 tx=0).stop 2024-04-17T09:56:55.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.164+0000 7fcb7d000640 1 -- 172.21.15.32:0/2187753389 shutdown_connections 2024-04-17T09:56:55.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.164+0000 7fcb7d000640 1 --2- 172.21.15.32:0/2187753389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7815fac0 0x7fcb78162580 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:55.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.164+0000 7fcb7d000640 1 -- 172.21.15.32:0/2187753389 >> 172.21.15.32:0/2187753389 conn(0x7fcb780b3480 msgr2=0x7fcb780d0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:55.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.164+0000 7fcb7d000640 1 -- 172.21.15.32:0/2187753389 shutdown_connections 2024-04-17T09:56:55.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.164+0000 7fcb7d000640 1 -- 172.21.15.32:0/2187753389 wait complete. 2024-04-17T09:56:55.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.164+0000 7fcb7d000640 1 Processor -- start 2024-04-17T09:56:55.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.164+0000 7fcb7d000640 1 -- start start 2024-04-17T09:56:55.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb7d000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb780b01e0 0x7fcb7814d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:55.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb7d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7815fac0 0x7fcb7815d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:55.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb7d000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb78162ac0 0x7fcb78150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:55.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb7d000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7806d3b0 con 0x7fcb7815fac0 2024-04-17T09:56:55.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb7d000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7806d230 con 0x7fcb78162ac0 2024-04-17T09:56:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb7d000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcb7806bbf0 con 0x7fcb780b01e0 2024-04-17T09:56:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb75400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7815fac0 0x7fcb7815d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb75400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7815fac0 0x7fcb7815d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34766/0 (socket says 172.21.15.32:34766) 2024-04-17T09:56:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb75400640 1 -- 172.21.15.32:0/3962083568 learned_addr learned my addr 172.21.15.32:0/3962083568 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb75e00640 1 --2- 172.21.15.32:0/3962083568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb780b01e0 0x7fcb7814d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb75400640 1 -- 172.21.15.32:0/3962083568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb780b01e0 msgr2=0x7fcb7814d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb76800640 1 --2- 172.21.15.32:0/3962083568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb78162ac0 0x7fcb78150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb75400640 1 --2- 172.21.15.32:0/3962083568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb780b01e0 0x7fcb7814d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb75400640 1 -- 172.21.15.32:0/3962083568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb78162ac0 msgr2=0x7fcb78150720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb75400640 1 --2- 172.21.15.32:0/3962083568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb78162ac0 0x7fcb78150720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb75400640 1 -- 172.21.15.32:0/3962083568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7814dda0 con 0x7fcb7815fac0 2024-04-17T09:56:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb75e00640 1 --2- 172.21.15.32:0/3962083568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb780b01e0 0x7fcb7814d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb76800640 1 --2- 172.21.15.32:0/3962083568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb78162ac0 0x7fcb78150720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:55.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb75400640 1 --2- 172.21.15.32:0/3962083568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7815fac0 0x7fcb7815d330 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fcb640ae040 tx=0x7fcb64019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:55.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb6e000640 1 -- 172.21.15.32:0/3962083568 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb640bb070 con 0x7fcb7815fac0 2024-04-17T09:56:55.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb6e000640 1 -- 172.21.15.32:0/3962083568 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb640aa750 con 0x7fcb7815fac0 2024-04-17T09:56:55.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb6e000640 1 -- 172.21.15.32:0/3962083568 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb640b0040 con 0x7fcb7815fac0 2024-04-17T09:56:55.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb7d000640 1 -- 172.21.15.32:0/3962083568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb78159df0 con 0x7fcb7815fac0 2024-04-17T09:56:55.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb7d000640 1 -- 172.21.15.32:0/3962083568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb7815a3b0 con 0x7fcb7815fac0 2024-04-17T09:56:55.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb7d000640 1 -- 172.21.15.32:0/3962083568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb780691a0 con 0x7fcb7815fac0 2024-04-17T09:56:55.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.168+0000 7fcb6e000640 1 -- 172.21.15.32:0/3962083568 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcb640aab40 con 0x7fcb7815fac0 2024-04-17T09:56:55.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.172+0000 7fcb6e000640 1 --2- 172.21.15.32:0/3962083568 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcb580683b0 0x7fcb5806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:55.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.172+0000 7fcb6e000640 1 -- 172.21.15.32:0/3962083568 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fcb64120470 con 0x7fcb7815fac0 2024-04-17T09:56:55.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.172+0000 7fcb75e00640 1 --2- 172.21.15.32:0/3962083568 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcb580683b0 0x7fcb5806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:55.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.172+0000 7fcb75e00640 1 --2- 172.21.15.32:0/3962083568 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcb580683b0 0x7fcb5806a870 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcb68000f70 tx=0x7fcb68009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:55.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.176+0000 7fcb6e000640 1 -- 172.21.15.32:0/3962083568 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fcb64016150 con 0x7fcb7815fac0 2024-04-17T09:56:55.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.340+0000 7fcb7d000640 1 -- 172.21.15.32:0/3962083568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcb780d1bd0 con 0x7fcb7815fac0 2024-04-17T09:56:55.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.340+0000 7fcb6e000640 1 -- 172.21.15.32:0/3962083568 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v57) ==== 100+0+11 (secure 0 0 0) 0x7fcb640f0a20 con 0x7fcb7815fac0 2024-04-17T09:56:55.343 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:56:55.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.344+0000 7fcb7d000640 1 -- 172.21.15.32:0/3962083568 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcb580683b0 msgr2=0x7fcb5806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:55.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.344+0000 7fcb7d000640 1 --2- 172.21.15.32:0/3962083568 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcb580683b0 0x7fcb5806a870 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcb68000f70 tx=0x7fcb68009040 comp rx=0 tx=0).stop 2024-04-17T09:56:55.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.344+0000 7fcb7d000640 1 -- 172.21.15.32:0/3962083568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7815fac0 msgr2=0x7fcb7815d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:55.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.344+0000 7fcb7d000640 1 --2- 172.21.15.32:0/3962083568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7815fac0 0x7fcb7815d330 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fcb640ae040 tx=0x7fcb64019270 comp rx=0 tx=0).stop 2024-04-17T09:56:55.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.344+0000 7fcb7d000640 1 -- 172.21.15.32:0/3962083568 shutdown_connections 2024-04-17T09:56:55.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.344+0000 7fcb7d000640 1 --2- 172.21.15.32:0/3962083568 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcb580683b0 0x7fcb5806a870 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:55.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.344+0000 7fcb7d000640 1 --2- 172.21.15.32:0/3962083568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb78162ac0 0x7fcb78150720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:55.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.344+0000 7fcb7d000640 1 --2- 172.21.15.32:0/3962083568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7815fac0 0x7fcb7815d330 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:55.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.344+0000 7fcb7d000640 1 --2- 172.21.15.32:0/3962083568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb780b01e0 0x7fcb7814d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:55.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.344+0000 7fcb7d000640 1 -- 172.21.15.32:0/3962083568 >> 172.21.15.32:0/3962083568 conn(0x7fcb780b3480 msgr2=0x7fcb78151460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:55.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.344+0000 7fcb7d000640 1 -- 172.21.15.32:0/3962083568 shutdown_connections 2024-04-17T09:56:55.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.344+0000 7fcb7d000640 1 -- 172.21.15.32:0/3962083568 wait complete. 2024-04-17T09:56:55.364 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T09:56:55.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.504+0000 7f7682400640 1 Processor -- start 2024-04-17T09:56:55.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.504+0000 7f7682400640 1 -- start start 2024-04-17T09:56:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.504+0000 7f7682400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f767c15c820 0x7f767c0d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.504+0000 7f7682400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767c0cdbd0 0x7f767c09b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.504+0000 7f7682400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f767c15cc00 con 0x7f767c09bc60 2024-04-17T09:56:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.504+0000 7f7682400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f767c09a9f0 con 0x7f767c15c820 2024-04-17T09:56:55.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.504+0000 7f7682400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f767c0ce340 con 0x7f767c0cdbd0 2024-04-17T09:56:55.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.508+0000 7f767b400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f767c15c820 0x7f767c0d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:55.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.508+0000 7f767be00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f767c09bc60 0x7f767c0d0b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44232/0 (socket says 172.21.15.32:44232) 2024-04-17T09:56:55.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.508+0000 7f767be00640 1 -- 172.21.15.32:0/1368499258 learned_addr learned my addr 172.21.15.32:0/1368499258 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:55.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.508+0000 7f767aa00640 1 --2- 172.21.15.32:0/1368499258 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767c0cdbd0 0x7f767c09b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:55.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.508+0000 7f767b400640 1 -- 172.21.15.32:0/1368499258 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767c0cdbd0 msgr2=0x7f767c09b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:55.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.508+0000 7f767b400640 1 --2- 172.21.15.32:0/1368499258 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767c0cdbd0 0x7f767c09b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:55.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.508+0000 7f767b400640 1 -- 172.21.15.32:0/1368499258 >> v1:172.21.15.32:6789/0 conn(0x7f767c09bc60 legacy=0x7f767c0d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:55.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.508+0000 7f767b400640 1 -- 172.21.15.32:0/1368499258 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767c09ac40 con 0x7f767c15c820 2024-04-17T09:56:55.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.508+0000 7f767a000640 1 -- 172.21.15.32:0/1368499258 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4076783879 0 0) 0x7f767c15cc00 con 0x7f767c09bc60 2024-04-17T09:56:55.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.508+0000 7f767aa00640 1 --2- 172.21.15.32:0/1368499258 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767c0cdbd0 0x7f767c09b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:55.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.508+0000 7f767b400640 1 --2- 172.21.15.32:0/1368499258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f767c15c820 0x7f767c0d0540 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f766c0097c0 tx=0x7f766c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27623996a614fa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:55.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.508+0000 7f767a000640 1 -- 172.21.15.32:0/1368499258 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c0a7070 con 0x7f767c15c820 2024-04-17T09:56:55.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.508+0000 7f767a000640 1 -- 172.21.15.32:0/1368499258 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f766c0b0070 con 0x7f767c15c820 2024-04-17T09:56:55.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.508+0000 7f767a000640 1 -- 172.21.15.32:0/1368499258 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c0ab890 con 0x7f767c15c820 2024-04-17T09:56:55.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f7682400640 1 -- 172.21.15.32:0/1368499258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f767c15c820 msgr2=0x7f767c0d0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:55.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f7682400640 1 --2- 172.21.15.32:0/1368499258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f767c15c820 0x7f767c0d0540 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f766c0097c0 tx=0x7f766c01c470 comp rx=0 tx=0).stop 2024-04-17T09:56:55.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f7682400640 1 -- 172.21.15.32:0/1368499258 shutdown_connections 2024-04-17T09:56:55.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f7682400640 1 --2- 172.21.15.32:0/1368499258 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767c0cdbd0 0x7f767c09b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:55.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f7682400640 1 --2- 172.21.15.32:0/1368499258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f767c15c820 0x7f767c0d0540 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:55.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f7682400640 1 -- 172.21.15.32:0/1368499258 >> 172.21.15.32:0/1368499258 conn(0x7f767c0b40a0 msgr2=0x7f767c0b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:55.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f7682400640 1 -- 172.21.15.32:0/1368499258 shutdown_connections 2024-04-17T09:56:55.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f7682400640 1 -- 172.21.15.32:0/1368499258 wait complete. 2024-04-17T09:56:55.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f7682400640 1 Processor -- start 2024-04-17T09:56:55.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f7682400640 1 -- start start 2024-04-17T09:56:55.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f7682400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f767c09bc60 0x7f767c14e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:55.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f7682400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f767c0cdbd0 0x7f767c150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:55.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f7682400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767c15c820 0x7f767c14d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:55.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f7682400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f767c0c6a90 con 0x7f767c09bc60 2024-04-17T09:56:55.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f7682400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f767c0c6910 con 0x7f767c0cdbd0 2024-04-17T09:56:55.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f7682400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f767c0d12a0 con 0x7f767c15c820 2024-04-17T09:56:55.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f767b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f767c09bc60 0x7f767c14e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:55.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f767b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f767c09bc60 0x7f767c14e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34786/0 (socket says 172.21.15.32:34786) 2024-04-17T09:56:55.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f767b400640 1 -- 172.21.15.32:0/3072349180 learned_addr learned my addr 172.21.15.32:0/3072349180 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:55.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f767be00640 1 --2- 172.21.15.32:0/3072349180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767c15c820 0x7f767c14d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:55.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f767aa00640 1 --2- 172.21.15.32:0/3072349180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f767c0cdbd0 0x7f767c150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:55.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f767b400640 1 -- 172.21.15.32:0/3072349180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767c15c820 msgr2=0x7f767c14d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:55.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f767b400640 1 --2- 172.21.15.32:0/3072349180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767c15c820 0x7f767c14d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:55.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f767b400640 1 -- 172.21.15.32:0/3072349180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f767c0cdbd0 msgr2=0x7f767c150540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:55.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f767b400640 1 --2- 172.21.15.32:0/3072349180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f767c0cdbd0 0x7f767c150540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:55.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.512+0000 7f767b400640 1 -- 172.21.15.32:0/3072349180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767c14ea00 con 0x7f767c09bc60 2024-04-17T09:56:55.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.516+0000 7f767b400640 1 --2- 172.21.15.32:0/3072349180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f767c09bc60 0x7f767c14e320 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f766c0ad040 tx=0x7f766c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:55.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.516+0000 7f7673400640 1 -- 172.21.15.32:0/3072349180 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c0bf610 con 0x7f767c09bc60 2024-04-17T09:56:55.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.516+0000 7f7673400640 1 -- 172.21.15.32:0/3072349180 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f766c0bf7f0 con 0x7f767c09bc60 2024-04-17T09:56:55.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.516+0000 7f7673400640 1 -- 172.21.15.32:0/3072349180 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c0bfbe0 con 0x7f767c09bc60 2024-04-17T09:56:55.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.516+0000 7f7682400640 1 -- 172.21.15.32:0/3072349180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f767c14ece0 con 0x7f767c09bc60 2024-04-17T09:56:55.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.516+0000 7f7682400640 1 -- 172.21.15.32:0/3072349180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f767c14f2a0 con 0x7f767c09bc60 2024-04-17T09:56:55.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.516+0000 7f7682400640 1 -- 172.21.15.32:0/3072349180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f767c068a90 con 0x7f767c09bc60 2024-04-17T09:56:55.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.516+0000 7f7673400640 1 -- 172.21.15.32:0/3072349180 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f766c0bd020 con 0x7f767c09bc60 2024-04-17T09:56:55.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.516+0000 7f7673400640 1 --2- 172.21.15.32:0/3072349180 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f765c068380 0x7f765c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:55.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.516+0000 7f7673400640 1 -- 172.21.15.32:0/3072349180 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f766c120e30 con 0x7f767c09bc60 2024-04-17T09:56:55.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.516+0000 7f767aa00640 1 --2- 172.21.15.32:0/3072349180 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f765c068380 0x7f765c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:55.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.520+0000 7f767aa00640 1 --2- 172.21.15.32:0/3072349180 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f765c068380 0x7f765c06a840 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f767c1670e0 tx=0x7f7668008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:55.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.520+0000 7f7673400640 1 -- 172.21.15.32:0/3072349180 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f766c0c6030 con 0x7f767c09bc60 2024-04-17T09:56:55.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.696+0000 7f7682400640 1 -- 172.21.15.32:0/3072349180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f767c0b4c90 con 0x7f767c09bc60 2024-04-17T09:56:55.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.696+0000 7f7673400640 1 -- 172.21.15.32:0/3072349180 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v57) ==== 80+0+0 (secure 0 0 0) 0x7f766c016090 con 0x7f767c09bc60 2024-04-17T09:56:55.699 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T09:56:55.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.700+0000 7f7682400640 1 -- 172.21.15.32:0/3072349180 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f765c068380 msgr2=0x7f765c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:55.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.700+0000 7f7682400640 1 --2- 172.21.15.32:0/3072349180 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f765c068380 0x7f765c06a840 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f767c1670e0 tx=0x7f7668008040 comp rx=0 tx=0).stop 2024-04-17T09:56:55.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.700+0000 7f7682400640 1 -- 172.21.15.32:0/3072349180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f767c09bc60 msgr2=0x7f767c14e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:55.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.700+0000 7f7682400640 1 --2- 172.21.15.32:0/3072349180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f767c09bc60 0x7f767c14e320 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f766c0ad040 tx=0x7f766c01acb0 comp rx=0 tx=0).stop 2024-04-17T09:56:55.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.700+0000 7f7682400640 1 -- 172.21.15.32:0/3072349180 shutdown_connections 2024-04-17T09:56:55.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.700+0000 7f7682400640 1 --2- 172.21.15.32:0/3072349180 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f765c068380 0x7f765c06a840 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:55.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.700+0000 7f7682400640 1 --2- 172.21.15.32:0/3072349180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767c15c820 0x7f767c14d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:55.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.700+0000 7f7682400640 1 --2- 172.21.15.32:0/3072349180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f767c0cdbd0 0x7f767c150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:55.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.700+0000 7f7682400640 1 --2- 172.21.15.32:0/3072349180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f767c09bc60 0x7f767c14e320 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:55.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.700+0000 7f7682400640 1 -- 172.21.15.32:0/3072349180 >> 172.21.15.32:0/3072349180 conn(0x7f767c0b40a0 msgr2=0x7f767c0cd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:55.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.700+0000 7f7682400640 1 -- 172.21.15.32:0/3072349180 shutdown_connections 2024-04-17T09:56:55.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:55.700+0000 7f7682400640 1 -- 172.21.15.32:0/3072349180 wait complete. 2024-04-17T09:56:55.719 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2024-04-17T09:56:55.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:56:55.720 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T09:56:55.737 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[39948]: fuse finished with error 0 and tester_r 0 2024-04-17T09:56:55.740 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T09:56:55.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:56:55.740 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T09:56:55.908 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T09:56:55.908 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:56:55.908 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:56:55.918 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T09:56:55.918 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T09:56:55.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:56:55.918 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-17T09:56:55.970 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-17T09:56:55.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:56:55.971 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T09:56:55.971 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:56:55.971 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:56:55.980 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T09:56:55.980 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T09:56:55.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:56:55.980 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-17T09:56:56.030 INFO:teuthology.orchestra.run.smithi032.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-17T09:56:56.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:56:56.031 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T09:56:56.032 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:56:56.032 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:56:56.041 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T09:56:56.041 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T09:56:56.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:56:56.041 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-17T09:56:56.090 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-17T09:56:56.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:56:56.091 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:56:56.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65b2c00640 1 Processor -- start 2024-04-17T09:56:56.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65b2c00640 1 -- start start 2024-04-17T09:56:56.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65b2c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65ac15eb30 0x7f65ac09b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65b2c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f65ac09b100 0x7f65ac0ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65b2c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65ac09bcb0 con 0x7f65ac0d0890 2024-04-17T09:56:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65b2c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f65ac06dd60 con 0x7f65ac09b100 2024-04-17T09:56:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65b2c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f65ac15ef10 con 0x7f65ac15eb30 2024-04-17T09:56:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65a3e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f65ac0d0890 0x7f65ac09a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44248/0 (socket says 172.21.15.32:44248) 2024-04-17T09:56:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65a3e00640 1 -- 172.21.15.32:0/1307480087 learned_addr learned my addr 172.21.15.32:0/1307480087 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:56.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65ab400640 1 -- 172.21.15.32:0/1307480087 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1661035303 0 0) 0x7f65ac09bcb0 con 0x7f65ac0d0890 2024-04-17T09:56:56.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65b0c00640 1 --2- 172.21.15.32:0/1307480087 >> v2:172.21.15.144:3300/0 conn(0x7f65ac09b100 0x7f65ac0ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65abe00640 1 --2- 172.21.15.32:0/1307480087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65ac15eb30 0x7f65ac09b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65ab400640 1 -- 172.21.15.32:0/1307480087 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6590003530 con 0x7f65ac0d0890 2024-04-17T09:56:56.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65ab400640 1 -- 172.21.15.32:0/1307480087 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3151436937 0 0) 0x7f6590003530 con 0x7f65ac0d0890 2024-04-17T09:56:56.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65ab400640 1 -- 172.21.15.32:0/1307480087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65ac15eb30 msgr2=0x7f65ac09b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:56.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65ab400640 1 --2- 172.21.15.32:0/1307480087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65ac15eb30 0x7f65ac09b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65ab400640 1 -- 172.21.15.32:0/1307480087 >> v2:172.21.15.144:3300/0 conn(0x7f65ac09b100 msgr2=0x7f65ac0ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:56.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65ab400640 1 --2- 172.21.15.32:0/1307480087 >> v2:172.21.15.144:3300/0 conn(0x7f65ac09b100 0x7f65ac0ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65ab400640 1 -- 172.21.15.32:0/1307480087 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ac0d0db0 con 0x7f65ac0d0890 2024-04-17T09:56:56.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65ab400640 1 -- 172.21.15.32:0/1307480087 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f6594001680 con 0x7f65ac0d0890 2024-04-17T09:56:56.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65ab400640 1 -- 172.21.15.32:0/1307480087 >> v1:172.21.15.32:6789/0 conn(0x7f65ac0d0890 legacy=0x7f65ac09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:56.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65ab400640 1 --2- 172.21.15.32:0/1307480087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65900039c0 0x7f6590003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65ab400640 1 --2- 172.21.15.32:0/1307480087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6590004810 0x7f6590004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65ab400640 1 --2- 172.21.15.32:0/1307480087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65900051d0 0x7f659000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65ab400640 1 -- 172.21.15.32:0/1307480087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac06dd60 con 0x7f65900051d0 2024-04-17T09:56:56.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65ab400640 1 -- 172.21.15.32:0/1307480087 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac15ef10 con 0x7f65900039c0 2024-04-17T09:56:56.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65ab400640 1 -- 172.21.15.32:0/1307480087 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f659000bf90 con 0x7f6590004810 2024-04-17T09:56:56.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65abe00640 1 --2- 172.21.15.32:0/1307480087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65ac15eb30 0x7f65ac09b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:56.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.232+0000 7f65b0c00640 1 --2- 172.21.15.32:0/1307480087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65900051d0 0x7f659000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.236+0000 7f65a3e00640 1 --2- 172.21.15.32:0/1307480087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6590004810 0x7f6590004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.236+0000 7f65abe00640 1 --2- 172.21.15.32:0/1307480087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65900039c0 0x7f6590003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.236+0000 7f65b0c00640 1 -- 172.21.15.32:0/1307480087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6590004810 msgr2=0x7f6590004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:56.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.236+0000 7f65b0c00640 1 --2- 172.21.15.32:0/1307480087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6590004810 0x7f6590004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.236+0000 7f65b0c00640 1 -- 172.21.15.32:0/1307480087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65900039c0 msgr2=0x7f6590003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:56.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.236+0000 7f65b0c00640 1 --2- 172.21.15.32:0/1307480087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65900039c0 0x7f6590003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.236+0000 7f65b0c00640 1 -- 172.21.15.32:0/1307480087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6590003530 con 0x7f65900051d0 2024-04-17T09:56:56.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.236+0000 7f65abe00640 1 --2- 172.21.15.32:0/1307480087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65900039c0 0x7f6590003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:56.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.236+0000 7f65b0c00640 1 --2- 172.21.15.32:0/1307480087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65900051d0 0x7f659000b9d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009f90 tx=0x7f65a401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b119c6c2e22c0038 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:56.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.236+0000 7f65ab400640 1 -- 172.21.15.32:0/1307480087 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a40b2070 con 0x7f65900051d0 2024-04-17T09:56:56.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.236+0000 7f65ab400640 1 -- 172.21.15.32:0/1307480087 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65a40a5c90 con 0x7f65900051d0 2024-04-17T09:56:56.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.236+0000 7f65ab400640 1 -- 172.21.15.32:0/1307480087 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a4009c10 con 0x7f65900051d0 2024-04-17T09:56:56.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.236+0000 7f65b2c00640 1 -- 172.21.15.32:0/1307480087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65900051d0 msgr2=0x7f659000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:56.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.236+0000 7f65b2c00640 1 --2- 172.21.15.32:0/1307480087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65900051d0 0x7f659000b9d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009f90 tx=0x7f65a401cc80 comp rx=0 tx=0).stop 2024-04-17T09:56:56.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.236+0000 7f65b0c00640 1 -- 172.21.15.32:0/1307480087 reap_dead start 2024-04-17T09:56:56.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65b2c00640 1 -- 172.21.15.32:0/1307480087 shutdown_connections 2024-04-17T09:56:56.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65b2c00640 1 -- 172.21.15.32:0/1307480087 >> 172.21.15.32:0/1307480087 conn(0x7f65ac0b3fd0 msgr2=0x7f65ac0b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:56.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65b2c00640 1 -- 172.21.15.32:0/1307480087 shutdown_connections 2024-04-17T09:56:56.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65b2c00640 1 -- 172.21.15.32:0/1307480087 wait complete. 2024-04-17T09:56:56.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65b2c00640 1 Processor -- start 2024-04-17T09:56:56.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65b2c00640 1 -- start start 2024-04-17T09:56:56.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65b2c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65ac155bc0 0x7f65ac1575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65b2c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65ac06cfd0 0x7f65ac160840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65b2c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65ac160d80 0x7f65ac0b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65b2c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac0d1650 con 0x7f65ac155bc0 2024-04-17T09:56:56.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65b2c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac0d1370 con 0x7f65ac160d80 2024-04-17T09:56:56.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65b2c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f65ac0d1930 con 0x7f65ac06cfd0 2024-04-17T09:56:56.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65abe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65ac155bc0 0x7f65ac1575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65abe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65ac155bc0 0x7f65ac1575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34804/0 (socket says 172.21.15.32:34804) 2024-04-17T09:56:56.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65abe00640 1 -- 172.21.15.32:0/3485792172 learned_addr learned my addr 172.21.15.32:0/3485792172 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:56.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65b0c00640 1 --2- 172.21.15.32:0/3485792172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65ac160d80 0x7f65ac0b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65a3e00640 1 --2- 172.21.15.32:0/3485792172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65ac06cfd0 0x7f65ac160840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65abe00640 1 -- 172.21.15.32:0/3485792172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65ac06cfd0 msgr2=0x7f65ac160840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:56.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65abe00640 1 --2- 172.21.15.32:0/3485792172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65ac06cfd0 0x7f65ac160840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65abe00640 1 -- 172.21.15.32:0/3485792172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65ac160d80 msgr2=0x7f65ac0b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:56.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65abe00640 1 --2- 172.21.15.32:0/3485792172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65ac160d80 0x7f65ac0b6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65abe00640 1 -- 172.21.15.32:0/3485792172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ac157bb0 con 0x7f65ac155bc0 2024-04-17T09:56:56.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65a3e00640 1 --2- 172.21.15.32:0/3485792172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65ac06cfd0 0x7f65ac160840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:56.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65abe00640 1 --2- 172.21.15.32:0/3485792172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65ac155bc0 0x7f65ac1575b0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f659c00f490 tx=0x7f659c00f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:56.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.240+0000 7f65a8c00640 1 -- 172.21.15.32:0/3485792172 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c018020 con 0x7f65ac155bc0 2024-04-17T09:56:56.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.244+0000 7f65a8c00640 1 -- 172.21.15.32:0/3485792172 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f659c014890 con 0x7f65ac155bc0 2024-04-17T09:56:56.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.244+0000 7f65b2c00640 1 -- 172.21.15.32:0/3485792172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ac157e90 con 0x7f65ac155bc0 2024-04-17T09:56:56.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.244+0000 7f65a8c00640 1 -- 172.21.15.32:0/3485792172 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c014c40 con 0x7f65ac155bc0 2024-04-17T09:56:56.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.244+0000 7f65b2c00640 1 -- 172.21.15.32:0/3485792172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ac158450 con 0x7f65ac155bc0 2024-04-17T09:56:56.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.244+0000 7f65b2c00640 1 -- 172.21.15.32:0/3485792172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65ac068a90 con 0x7f65ac155bc0 2024-04-17T09:56:56.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.244+0000 7f65a8c00640 1 -- 172.21.15.32:0/3485792172 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f659c01b020 con 0x7f65ac155bc0 2024-04-17T09:56:56.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.244+0000 7f65a8c00640 1 --2- 172.21.15.32:0/3485792172 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f65880683b0 0x7f658806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.244+0000 7f65a8c00640 1 -- 172.21.15.32:0/3485792172 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f659c086c50 con 0x7f65ac155bc0 2024-04-17T09:56:56.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.244+0000 7f65a3e00640 1 --2- 172.21.15.32:0/3485792172 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f65880683b0 0x7f658806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.244+0000 7f65a3e00640 1 --2- 172.21.15.32:0/3485792172 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f65880683b0 0x7f658806a870 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6594001060 tx=0x7f6594009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:56.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.248+0000 7f65a8c00640 1 -- 172.21.15.32:0/3485792172 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f659c022020 con 0x7f65ac155bc0 2024-04-17T09:56:56.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.452+0000 7f65b2c00640 1 -- 172.21.15.32:0/3485792172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65ac0ce9a0 con 0x7f65ac155bc0 2024-04-17T09:56:56.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.452+0000 7f65a8c00640 1 -- 172.21.15.32:0/3485792172 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5203 (secure 0 0 0) 0x7f659c057170 con 0x7f65ac155bc0 2024-04-17T09:56:56.455 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:56.455 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5495,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/1236891874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1236891874},{"type":"v1","addr":"172.21.15.144:6834","nonce":1236891874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5498,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3538881342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3538881342},{"type":"v1","addr":"172.21.15.144:6835","nonce":3538881342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6201,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3549801633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3549801633},{"type":"v1","addr":"172.21.15.32:6836","nonce":3549801633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:56:01.579180+0000","modified":"2024-04-17T09:56:50.242920+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":15,"addr":"172.21.15.32:6837/866826055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":866826055},{"type":"v1","addr":"172.21.15.32:6837","nonce":866826055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-04-17T09:56:56.456 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 34 2024-04-17T09:56:56.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.456+0000 7f65b2c00640 1 -- 172.21.15.32:0/3485792172 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f65880683b0 msgr2=0x7f658806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:56.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.456+0000 7f65b2c00640 1 --2- 172.21.15.32:0/3485792172 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f65880683b0 0x7f658806a870 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6594001060 tx=0x7f6594009040 comp rx=0 tx=0).stop 2024-04-17T09:56:56.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.456+0000 7f65b2c00640 1 -- 172.21.15.32:0/3485792172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65ac155bc0 msgr2=0x7f65ac1575b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:56.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.456+0000 7f65b2c00640 1 --2- 172.21.15.32:0/3485792172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65ac155bc0 0x7f65ac1575b0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f659c00f490 tx=0x7f659c00f950 comp rx=0 tx=0).stop 2024-04-17T09:56:56.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.456+0000 7f65b2c00640 1 -- 172.21.15.32:0/3485792172 shutdown_connections 2024-04-17T09:56:56.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.456+0000 7f65b2c00640 1 --2- 172.21.15.32:0/3485792172 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f65880683b0 0x7f658806a870 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.456+0000 7f65b2c00640 1 --2- 172.21.15.32:0/3485792172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65ac06cfd0 0x7f65ac160840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.456+0000 7f65b2c00640 1 --2- 172.21.15.32:0/3485792172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65ac160d80 0x7f65ac0b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.456+0000 7f65b2c00640 1 --2- 172.21.15.32:0/3485792172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65ac155bc0 0x7f65ac1575b0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.456+0000 7f65b2c00640 1 -- 172.21.15.32:0/3485792172 >> 172.21.15.32:0/3485792172 conn(0x7f65ac0b3fd0 msgr2=0x7f65ac161a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:56.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.456+0000 7f65b2c00640 1 -- 172.21.15.32:0/3485792172 shutdown_connections 2024-04-17T09:56:56.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.456+0000 7f65b2c00640 1 -- 172.21.15.32:0/3485792172 wait complete. 2024-04-17T09:56:56.475 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:56:56.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.616+0000 7f044ee00640 1 Processor -- start 2024-04-17T09:56:56.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.616+0000 7f044ee00640 1 -- start start 2024-04-17T09:56:56.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.616+0000 7f044ee00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f044815baa0 0x7f044815bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.616+0000 7f044ee00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f044806ef60 0x7f044806f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.616+0000 7f044ee00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0448072bb0 con 0x7f0448144e90 2024-04-17T09:56:56.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.616+0000 7f044ee00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f044815c440 con 0x7f044806ef60 2024-04-17T09:56:56.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.616+0000 7f044ee00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f044806f8f0 con 0x7f044815baa0 2024-04-17T09:56:56.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.616+0000 7f043f400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f044815baa0 0x7f044815bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.616+0000 7f044ce00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f044806ef60 0x7f044806f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.616+0000 7f0447e00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f0448144e90 0x7f04480b48a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:42762/0 (socket says 172.21.15.32:42762) 2024-04-17T09:56:56.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.616+0000 7f0447e00640 1 -- 172.21.15.32:0/481165630 learned_addr learned my addr 172.21.15.32:0/481165630 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:56.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.620+0000 7f044ce00640 1 -- 172.21.15.32:0/481165630 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f044815baa0 msgr2=0x7f044815bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:56.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.620+0000 7f044ce00640 1 --2- 172.21.15.32:0/481165630 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f044815baa0 0x7f044815bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.620+0000 7f044ce00640 1 -- 172.21.15.32:0/481165630 >> v1:172.21.15.144:6789/0 conn(0x7f0448144e90 legacy=0x7f04480b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:56.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.620+0000 7f044ce00640 1 -- 172.21.15.32:0/481165630 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04480b3dc0 con 0x7f044806ef60 2024-04-17T09:56:56.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.620+0000 7f044ce00640 1 --2- 172.21.15.32:0/481165630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f044806ef60 0x7f044806f3b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0440009d90 tx=0x7f044001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a6e1b767be2f71c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:56.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.620+0000 7f0447400640 1 -- 172.21.15.32:0/481165630 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04400b2070 con 0x7f044806ef60 2024-04-17T09:56:56.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.620+0000 7f0447400640 1 -- 172.21.15.32:0/481165630 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04400a5920 con 0x7f044806ef60 2024-04-17T09:56:56.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.620+0000 7f0447400640 1 -- 172.21.15.32:0/481165630 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04400a5cf0 con 0x7f044806ef60 2024-04-17T09:56:56.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f044ee00640 1 -- 172.21.15.32:0/481165630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f044806ef60 msgr2=0x7f044806f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:56.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f044ee00640 1 --2- 172.21.15.32:0/481165630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f044806ef60 0x7f044806f3b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0440009d90 tx=0x7f044001c970 comp rx=0 tx=0).stop 2024-04-17T09:56:56.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f044ee00640 1 -- 172.21.15.32:0/481165630 shutdown_connections 2024-04-17T09:56:56.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f044ee00640 1 --2- 172.21.15.32:0/481165630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f044806ef60 0x7f044806f3b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f044ee00640 1 --2- 172.21.15.32:0/481165630 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f044815baa0 0x7f044815bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f044ee00640 1 -- 172.21.15.32:0/481165630 >> 172.21.15.32:0/481165630 conn(0x7f044806ba30 msgr2=0x7f044806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:56.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f044ee00640 1 -- 172.21.15.32:0/481165630 shutdown_connections 2024-04-17T09:56:56.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f044ee00640 1 -- 172.21.15.32:0/481165630 wait complete. 2024-04-17T09:56:56.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f044ee00640 1 Processor -- start 2024-04-17T09:56:56.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f044ee00640 1 -- start start 2024-04-17T09:56:56.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f044ee00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f044806ef60 0x7f04481f9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f044ee00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0448144e90 0x7f044813c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f044ee00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f044815baa0 0x7f0448157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f044ee00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f04480727f0 con 0x7f0448144e90 2024-04-17T09:56:56.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f044ee00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0448072670 con 0x7f044806ef60 2024-04-17T09:56:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f044ee00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0448072970 con 0x7f044815baa0 2024-04-17T09:56:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f043f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0448144e90 0x7f044813c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f043f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0448144e90 0x7f044813c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34820/0 (socket says 172.21.15.32:34820) 2024-04-17T09:56:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f043f400640 1 -- 172.21.15.32:0/278573357 learned_addr learned my addr 172.21.15.32:0/278573357 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f0447e00640 1 --2- 172.21.15.32:0/278573357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f044806ef60 0x7f04481f9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f044ce00640 1 --2- 172.21.15.32:0/278573357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f044815baa0 0x7f0448157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f043f400640 1 -- 172.21.15.32:0/278573357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f044815baa0 msgr2=0x7f0448157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f043f400640 1 --2- 172.21.15.32:0/278573357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f044815baa0 0x7f0448157860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f043f400640 1 -- 172.21.15.32:0/278573357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f044806ef60 msgr2=0x7f04481f9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f043f400640 1 --2- 172.21.15.32:0/278573357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f044806ef60 0x7f04481f9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f043f400640 1 -- 172.21.15.32:0/278573357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0448142e80 con 0x7f0448144e90 2024-04-17T09:56:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.624+0000 7f0447e00640 1 --2- 172.21.15.32:0/278573357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f044806ef60 0x7f04481f9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.628+0000 7f043f400640 1 --2- 172.21.15.32:0/278573357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0448144e90 0x7f044813c600 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f043000e870 tx=0x7f043000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.628+0000 7f0444c00640 1 -- 172.21.15.32:0/278573357 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430017070 con 0x7f0448144e90 2024-04-17T09:56:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.628+0000 7f0444c00640 1 -- 172.21.15.32:0/278573357 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f043000c940 con 0x7f0448144e90 2024-04-17T09:56:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.628+0000 7f0444c00640 1 -- 172.21.15.32:0/278573357 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043000ccf0 con 0x7f0448144e90 2024-04-17T09:56:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.628+0000 7f044ee00640 1 -- 172.21.15.32:0/278573357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f044813f490 con 0x7f0448144e90 2024-04-17T09:56:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.628+0000 7f044ee00640 1 -- 172.21.15.32:0/278573357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f044813fa50 con 0x7f0448144e90 2024-04-17T09:56:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.628+0000 7f044ee00640 1 -- 172.21.15.32:0/278573357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0448068a90 con 0x7f0448144e90 2024-04-17T09:56:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.628+0000 7f0444c00640 1 -- 172.21.15.32:0/278573357 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f043001a070 con 0x7f0448144e90 2024-04-17T09:56:56.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.628+0000 7f0444c00640 1 --2- 172.21.15.32:0/278573357 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0424068380 0x7f042406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.628+0000 7f0444c00640 1 -- 172.21.15.32:0/278573357 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f04300867f0 con 0x7f0448144e90 2024-04-17T09:56:56.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.628+0000 7f0447e00640 1 --2- 172.21.15.32:0/278573357 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0424068380 0x7f042406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.632+0000 7f0447e00640 1 --2- 172.21.15.32:0/278573357 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0424068380 0x7f042406a840 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0438000f70 tx=0x7f0438009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:56.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.632+0000 7f0444c00640 1 -- 172.21.15.32:0/278573357 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0430056750 con 0x7f0448144e90 2024-04-17T09:56:56.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.800+0000 7f044ee00640 1 -- 172.21.15.32:0/278573357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f044806f6b0 con 0x7f0448144e90 2024-04-17T09:56:56.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.800+0000 7f0444c00640 1 -- 172.21.15.32:0/278573357 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16677 (secure 0 0 0) 0x7f043005aa50 con 0x7f0448144e90 2024-04-17T09:56:56.801 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:56.801 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":57,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:56:03.192930+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:55:59.976232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-17T09:56:00.543788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:56:56.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.800+0000 7f044ee00640 1 -- 172.21.15.32:0/278573357 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0424068380 msgr2=0x7f042406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:56.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.800+0000 7f044ee00640 1 --2- 172.21.15.32:0/278573357 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0424068380 0x7f042406a840 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0438000f70 tx=0x7f0438009040 comp rx=0 tx=0).stop 2024-04-17T09:56:56.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.800+0000 7f044ee00640 1 -- 172.21.15.32:0/278573357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0448144e90 msgr2=0x7f044813c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:56.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.800+0000 7f044ee00640 1 --2- 172.21.15.32:0/278573357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0448144e90 0x7f044813c600 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f043000e870 tx=0x7f043000ec70 comp rx=0 tx=0).stop 2024-04-17T09:56:56.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.800+0000 7f044ee00640 1 -- 172.21.15.32:0/278573357 shutdown_connections 2024-04-17T09:56:56.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.800+0000 7f044ee00640 1 --2- 172.21.15.32:0/278573357 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0424068380 0x7f042406a840 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.800+0000 7f044ee00640 1 --2- 172.21.15.32:0/278573357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f044815baa0 0x7f0448157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.800+0000 7f044ee00640 1 --2- 172.21.15.32:0/278573357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0448144e90 0x7f044813c600 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.800+0000 7f044ee00640 1 --2- 172.21.15.32:0/278573357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f044806ef60 0x7f04481f9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.800+0000 7f044ee00640 1 -- 172.21.15.32:0/278573357 >> 172.21.15.32:0/278573357 conn(0x7f044806ba30 msgr2=0x7f044815e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:56.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.800+0000 7f044ee00640 1 -- 172.21.15.32:0/278573357 shutdown_connections 2024-04-17T09:56:56.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.800+0000 7f044ee00640 1 -- 172.21.15.32:0/278573357 wait complete. 2024-04-17T09:56:56.823 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:55:59.976232+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:56:00.543788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:56:56.823 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:56:56.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f68a2600640 1 Processor -- start 2024-04-17T09:56:56.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f68a2600640 1 -- start start 2024-04-17T09:56:56.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f68a2600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f689c15b7d0 0x7f689c06e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f68a2600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f689c06c140 0x7f689c0ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f68a2600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f689c0cc940 con 0x7f689c06f7c0 2024-04-17T09:56:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f68a2600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f689c0ccc20 con 0x7f689c15b7d0 2024-04-17T09:56:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f68a2600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f689c06e0b0 con 0x7f689c06c140 2024-04-17T09:56:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f6892a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f689c06f7c0 0x7f689c06ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44258/0 (socket says 172.21.15.32:44258) 2024-04-17T09:56:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f6892a00640 1 -- 172.21.15.32:0/2048421703 learned_addr learned my addr 172.21.15.32:0/2048421703 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:56.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f689be00640 1 --2- 172.21.15.32:0/2048421703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f689c06c140 0x7f689c0ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f689b400640 1 --2- 172.21.15.32:0/2048421703 >> v2:172.21.15.144:3300/0 conn(0x7f689c15b7d0 0x7f689c06e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f689aa00640 1 -- 172.21.15.32:0/2048421703 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 286369007 0 0) 0x7f689c0cc940 con 0x7f689c06f7c0 2024-04-17T09:56:56.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f689aa00640 1 -- 172.21.15.32:0/2048421703 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6880005740 con 0x7f689c06f7c0 2024-04-17T09:56:56.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f689aa00640 1 -- 172.21.15.32:0/2048421703 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2916041459 0 0) 0x7f6880005740 con 0x7f689c06f7c0 2024-04-17T09:56:56.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f689aa00640 1 -- 172.21.15.32:0/2048421703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f689c06c140 msgr2=0x7f689c0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f689aa00640 1 --2- 172.21.15.32:0/2048421703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f689c06c140 0x7f689c0ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f689aa00640 1 -- 172.21.15.32:0/2048421703 >> v2:172.21.15.144:3300/0 conn(0x7f689c15b7d0 msgr2=0x7f689c06e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f689aa00640 1 --2- 172.21.15.32:0/2048421703 >> v2:172.21.15.144:3300/0 conn(0x7f689c15b7d0 0x7f689c06e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f689aa00640 1 -- 172.21.15.32:0/2048421703 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689c06e390 con 0x7f689c06f7c0 2024-04-17T09:56:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f689aa00640 1 -- 172.21.15.32:0/2048421703 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f6884001680 con 0x7f689c06f7c0 2024-04-17T09:56:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f689aa00640 1 -- 172.21.15.32:0/2048421703 >> v1:172.21.15.32:6789/0 conn(0x7f689c06f7c0 legacy=0x7f689c06ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f689aa00640 1 --2- 172.21.15.32:0/2048421703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f689c06c140 0x7f6880005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f689aa00640 1 --2- 172.21.15.32:0/2048421703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6880006150 0x7f6880006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f689aa00640 1 --2- 172.21.15.32:0/2048421703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6880006f60 0x7f68800073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f689aa00640 1 -- 172.21.15.32:0/2048421703 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f689c0ccc20 con 0x7f6880006150 2024-04-17T09:56:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f689aa00640 1 -- 172.21.15.32:0/2048421703 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f689c06e0b0 con 0x7f6880006f60 2024-04-17T09:56:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f689aa00640 1 -- 172.21.15.32:0/2048421703 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f688000dc30 con 0x7f689c06c140 2024-04-17T09:56:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f6892a00640 1 --2- 172.21.15.32:0/2048421703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6880006150 0x7f6880006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f689b400640 1 --2- 172.21.15.32:0/2048421703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f689c06c140 0x7f6880005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f689be00640 1 --2- 172.21.15.32:0/2048421703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6880006f60 0x7f68800073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f6892a00640 1 -- 172.21.15.32:0/2048421703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f689c06c140 msgr2=0x7f6880005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f6892a00640 1 --2- 172.21.15.32:0/2048421703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f689c06c140 0x7f6880005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f6892a00640 1 -- 172.21.15.32:0/2048421703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6880006f60 msgr2=0x7f68800073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f6892a00640 1 --2- 172.21.15.32:0/2048421703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6880006f60 0x7f68800073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.968+0000 7f6892a00640 1 -- 172.21.15.32:0/2048421703 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6880005740 con 0x7f6880006150 2024-04-17T09:56:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.972+0000 7f6892a00640 1 --2- 172.21.15.32:0/2048421703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6880006150 0x7f6880006a20 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f688400c270 tx=0x7f688401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4916a426b68c7832 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.972+0000 7f689aa00640 1 -- 172.21.15.32:0/2048421703 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884007d60 con 0x7f6880006150 2024-04-17T09:56:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.972+0000 7f689aa00640 1 -- 172.21.15.32:0/2048421703 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68840a9020 con 0x7f6880006150 2024-04-17T09:56:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.972+0000 7f689aa00640 1 -- 172.21.15.32:0/2048421703 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68840a5bc0 con 0x7f6880006150 2024-04-17T09:56:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.972+0000 7f68a2600640 1 -- 172.21.15.32:0/2048421703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6880006150 msgr2=0x7f6880006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.972+0000 7f68a2600640 1 --2- 172.21.15.32:0/2048421703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6880006150 0x7f6880006a20 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f688400c270 tx=0x7f688401ccc0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.972+0000 7f689be00640 1 -- 172.21.15.32:0/2048421703 reap_dead start 2024-04-17T09:56:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.972+0000 7f68a2600640 1 -- 172.21.15.32:0/2048421703 shutdown_connections 2024-04-17T09:56:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.972+0000 7f68a2600640 1 -- 172.21.15.32:0/2048421703 >> 172.21.15.32:0/2048421703 conn(0x7f689c0b2120 msgr2=0x7f689c0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.972+0000 7f68a2600640 1 -- 172.21.15.32:0/2048421703 shutdown_connections 2024-04-17T09:56:56.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.972+0000 7f68a2600640 1 -- 172.21.15.32:0/2048421703 wait complete. 2024-04-17T09:56:56.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f68a2600640 1 Processor -- start 2024-04-17T09:56:56.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f68a2600640 1 -- start start 2024-04-17T09:56:56.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f68a2600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6880006150 0x7f689c13f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f68a2600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f689c15d860 0x7f689c14fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f68a2600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f689c163340 0x7f689c1637c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f68a2600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f689c0cee30 con 0x7f689c15d860 2024-04-17T09:56:56.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f68a2600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f689c0cecb0 con 0x7f6880006150 2024-04-17T09:56:56.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f68a2600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f689c071e60 con 0x7f689c163340 2024-04-17T09:56:56.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f6892a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f689c15d860 0x7f689c14fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f6892a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f689c15d860 0x7f689c14fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34842/0 (socket says 172.21.15.32:34842) 2024-04-17T09:56:56.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f6892a00640 1 -- 172.21.15.32:0/2218501641 learned_addr learned my addr 172.21.15.32:0/2218501641 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:56.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f689b400640 1 --2- 172.21.15.32:0/2218501641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6880006150 0x7f689c13f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f689be00640 1 --2- 172.21.15.32:0/2218501641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f689c163340 0x7f689c1637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f6892a00640 1 -- 172.21.15.32:0/2218501641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f689c163340 msgr2=0x7f689c1637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:56.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f6892a00640 1 --2- 172.21.15.32:0/2218501641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f689c163340 0x7f689c1637c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f6892a00640 1 -- 172.21.15.32:0/2218501641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6880006150 msgr2=0x7f689c13f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:56.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f6892a00640 1 --2- 172.21.15.32:0/2218501641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6880006150 0x7f689c13f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:56.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f6892a00640 1 -- 172.21.15.32:0/2218501641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689c13cb70 con 0x7f689c15d860 2024-04-17T09:56:56.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f689be00640 1 --2- 172.21.15.32:0/2218501641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f689c163340 0x7f689c1637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:56.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f6892a00640 1 --2- 172.21.15.32:0/2218501641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f689c15d860 0x7f689c14fb80 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f688400c270 tx=0x7f6880007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:56.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f6893e00640 1 -- 172.21.15.32:0/2218501641 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68840a5bc0 con 0x7f689c15d860 2024-04-17T09:56:56.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f6893e00640 1 -- 172.21.15.32:0/2218501641 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68840a5d60 con 0x7f689c15d860 2024-04-17T09:56:56.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f68a2600640 1 -- 172.21.15.32:0/2218501641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689c15bfd0 con 0x7f689c15d860 2024-04-17T09:56:56.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f6893e00640 1 -- 172.21.15.32:0/2218501641 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68840049f0 con 0x7f689c15d860 2024-04-17T09:56:56.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f68a2600640 1 -- 172.21.15.32:0/2218501641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f689c15de00 con 0x7f689c15d860 2024-04-17T09:56:56.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.976+0000 7f68a2600640 1 -- 172.21.15.32:0/2218501641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f689c068a90 con 0x7f689c15d860 2024-04-17T09:56:56.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.980+0000 7f6893e00640 1 -- 172.21.15.32:0/2218501641 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6884004b90 con 0x7f689c15d860 2024-04-17T09:56:56.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.980+0000 7f6893e00640 1 --2- 172.21.15.32:0/2218501641 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f68780683b0 0x7f687806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:56.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.980+0000 7f6893e00640 1 -- 172.21.15.32:0/2218501641 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f6884120710 con 0x7f689c15d860 2024-04-17T09:56:56.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.980+0000 7f689b400640 1 --2- 172.21.15.32:0/2218501641 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f68780683b0 0x7f687806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:56.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.980+0000 7f689b400640 1 --2- 172.21.15.32:0/2218501641 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f68780683b0 0x7f687806a870 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f688c009be0 tx=0x7f688c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:56.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:56.984+0000 7f6893e00640 1 -- 172.21.15.32:0/2218501641 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f689c15bfd0 con 0x7f689c15d860 2024-04-17T09:56:57.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.152+0000 7f68a2600640 1 -- 172.21.15.32:0/2218501641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f689c0cda80 con 0x7f689c15d860 2024-04-17T09:56:57.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.152+0000 7f6893e00640 1 -- 172.21.15.32:0/2218501641 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) ==== 93+0+10 (secure 0 0 0) 0x7f68840169d0 con 0x7f689c15d860 2024-04-17T09:56:57.153 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:56:57.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.152+0000 7f68a2600640 1 -- 172.21.15.32:0/2218501641 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f68780683b0 msgr2=0x7f687806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:57.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.152+0000 7f68a2600640 1 --2- 172.21.15.32:0/2218501641 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f68780683b0 0x7f687806a870 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f688c009be0 tx=0x7f688c009190 comp rx=0 tx=0).stop 2024-04-17T09:56:57.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.152+0000 7f68a2600640 1 -- 172.21.15.32:0/2218501641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f689c15d860 msgr2=0x7f689c14fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:57.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.152+0000 7f68a2600640 1 --2- 172.21.15.32:0/2218501641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f689c15d860 0x7f689c14fb80 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f688400c270 tx=0x7f6880007a50 comp rx=0 tx=0).stop 2024-04-17T09:56:57.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.152+0000 7f68a2600640 1 -- 172.21.15.32:0/2218501641 shutdown_connections 2024-04-17T09:56:57.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.152+0000 7f68a2600640 1 --2- 172.21.15.32:0/2218501641 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f68780683b0 0x7f687806a870 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.152+0000 7f68a2600640 1 --2- 172.21.15.32:0/2218501641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f689c163340 0x7f689c1637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.152+0000 7f68a2600640 1 --2- 172.21.15.32:0/2218501641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f689c15d860 0x7f689c14fb80 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.152+0000 7f68a2600640 1 --2- 172.21.15.32:0/2218501641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6880006150 0x7f689c13f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.152+0000 7f68a2600640 1 -- 172.21.15.32:0/2218501641 >> 172.21.15.32:0/2218501641 conn(0x7f689c0b2120 msgr2=0x7f689c06ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:57.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.156+0000 7f68a2600640 1 -- 172.21.15.32:0/2218501641 shutdown_connections 2024-04-17T09:56:57.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.156+0000 7f68a2600640 1 -- 172.21.15.32:0/2218501641 wait complete. 2024-04-17T09:56:57.175 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:56:57.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.316+0000 7ffad2200640 1 Processor -- start 2024-04-17T09:56:57.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.320+0000 7ffad2200640 1 -- start start 2024-04-17T09:56:57.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.320+0000 7ffad2200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffacc06ba30 0x7ffacc06be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:57.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.320+0000 7ffad2200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffacc0cf370 0x7ffacc0cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:57.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.320+0000 7ffad2200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffacc06c350 con 0x7ffacc160350 2024-04-17T09:56:57.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.320+0000 7ffad2200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ffacc0d0130 con 0x7ffacc0cf370 2024-04-17T09:56:57.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.320+0000 7ffad2200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffacc15d090 con 0x7ffacc06ba30 2024-04-17T09:56:57.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.320+0000 7ffacaa00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffacc0cf370 0x7ffacc0cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:57.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.320+0000 7ffacaa00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffacc0cf370 0x7ffacc0cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34858/0 (socket says 172.21.15.32:34858) 2024-04-17T09:56:57.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.320+0000 7ffacaa00640 1 -- 172.21.15.32:0/3614305413 learned_addr learned my addr 172.21.15.32:0/3614305413 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:57.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.320+0000 7ffacbe00640 1 --1- 172.21.15.32:0/3614305413 >> v1:172.21.15.144:6789/0 conn(0x7ffacc160350 0x7ffacc1607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:42776/0 (socket says 172.21.15.32:42776) 2024-04-17T09:56:57.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.320+0000 7ffacb400640 1 --2- 172.21.15.32:0/3614305413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffacc06ba30 0x7ffacc06be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:57.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.320+0000 7ffacaa00640 1 -- 172.21.15.32:0/3614305413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffacc06ba30 msgr2=0x7ffacc06be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:57.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.320+0000 7ffacaa00640 1 --2- 172.21.15.32:0/3614305413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffacc06ba30 0x7ffacc06be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.320+0000 7ffacaa00640 1 -- 172.21.15.32:0/3614305413 >> v1:172.21.15.144:6789/0 conn(0x7ffacc160350 legacy=0x7ffacc1607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:57.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.320+0000 7ffacaa00640 1 -- 172.21.15.32:0/3614305413 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffacc15d370 con 0x7ffacc0cf370 2024-04-17T09:56:57.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.324+0000 7ffacaa00640 1 --2- 172.21.15.32:0/3614305413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffacc0cf370 0x7ffacc0cf7e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009870 tx=0x7ffab801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf91482fd7f2d347 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:57.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.324+0000 7ffaca000640 1 -- 172.21.15.32:0/3614305413 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab80a7070 con 0x7ffacc0cf370 2024-04-17T09:56:57.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.324+0000 7ffaca000640 1 -- 172.21.15.32:0/3614305413 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffab80b0070 con 0x7ffacc0cf370 2024-04-17T09:56:57.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.324+0000 7ffaca000640 1 -- 172.21.15.32:0/3614305413 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab80ab890 con 0x7ffacc0cf370 2024-04-17T09:56:57.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.324+0000 7ffad2200640 1 -- 172.21.15.32:0/3614305413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffacc0cf370 msgr2=0x7ffacc0cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:57.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.324+0000 7ffad2200640 1 --2- 172.21.15.32:0/3614305413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffacc0cf370 0x7ffacc0cf7e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009870 tx=0x7ffab801c560 comp rx=0 tx=0).stop 2024-04-17T09:56:57.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.324+0000 7ffad2200640 1 -- 172.21.15.32:0/3614305413 shutdown_connections 2024-04-17T09:56:57.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.324+0000 7ffad2200640 1 --2- 172.21.15.32:0/3614305413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffacc0cf370 0x7ffacc0cf7e0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.324+0000 7ffad2200640 1 --2- 172.21.15.32:0/3614305413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffacc06ba30 0x7ffacc06be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.324+0000 7ffad2200640 1 -- 172.21.15.32:0/3614305413 >> 172.21.15.32:0/3614305413 conn(0x7ffacc0b2120 msgr2=0x7ffacc0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:57.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.324+0000 7ffad2200640 1 -- 172.21.15.32:0/3614305413 shutdown_connections 2024-04-17T09:56:57.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.324+0000 7ffad2200640 1 -- 172.21.15.32:0/3614305413 wait complete. 2024-04-17T09:56:57.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.324+0000 7ffad2200640 1 Processor -- start 2024-04-17T09:56:57.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.324+0000 7ffad2200640 1 -- start start 2024-04-17T09:56:57.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.328+0000 7ffad2200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffacc06ba30 0x7ffacc142ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:57.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.328+0000 7ffad2200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffacc0cf370 0x7ffacc0aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:57.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.328+0000 7ffad2200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffacc160350 0x7ffacc0b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:57.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.328+0000 7ffad2200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffacc15dcf0 con 0x7ffacc160350 2024-04-17T09:56:57.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.328+0000 7ffad2200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffacc15db70 con 0x7ffacc06ba30 2024-04-17T09:56:57.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.328+0000 7ffad2200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffacc146230 con 0x7ffacc0cf370 2024-04-17T09:56:57.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.328+0000 7ffacbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffacc160350 0x7ffacc0b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:57.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.328+0000 7ffacbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffacc160350 0x7ffacc0b42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34870/0 (socket says 172.21.15.32:34870) 2024-04-17T09:56:57.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.328+0000 7ffacbe00640 1 -- 172.21.15.32:0/1695350489 learned_addr learned my addr 172.21.15.32:0/1695350489 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:57.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.328+0000 7ffacaa00640 1 --2- 172.21.15.32:0/1695350489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffacc0cf370 0x7ffacc0aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:57.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.328+0000 7ffacb400640 1 --2- 172.21.15.32:0/1695350489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffacc06ba30 0x7ffacc142ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:57.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.328+0000 7ffacbe00640 1 -- 172.21.15.32:0/1695350489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffacc0cf370 msgr2=0x7ffacc0aff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:57.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.328+0000 7ffacbe00640 1 --2- 172.21.15.32:0/1695350489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffacc0cf370 0x7ffacc0aff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.328+0000 7ffacbe00640 1 -- 172.21.15.32:0/1695350489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffacc06ba30 msgr2=0x7ffacc142ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:57.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.328+0000 7ffacbe00640 1 --2- 172.21.15.32:0/1695350489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffacc06ba30 0x7ffacc142ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.328+0000 7ffacbe00640 1 -- 172.21.15.32:0/1695350489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffacc13b0a0 con 0x7ffacc160350 2024-04-17T09:56:57.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.328+0000 7ffacbe00640 1 --2- 172.21.15.32:0/1695350489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffacc160350 0x7ffacc0b42c0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ffac400c730 tx=0x7ffac400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:57.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.328+0000 7ffac3400640 1 -- 172.21.15.32:0/1695350489 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac4017070 con 0x7ffacc160350 2024-04-17T09:56:57.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.328+0000 7ffac3400640 1 -- 172.21.15.32:0/1695350489 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffac400ce40 con 0x7ffacc160350 2024-04-17T09:56:57.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.328+0000 7ffad2200640 1 -- 172.21.15.32:0/1695350489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffacc15e6e0 con 0x7ffacc160350 2024-04-17T09:56:57.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.328+0000 7ffac3400640 1 -- 172.21.15.32:0/1695350489 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac400a840 con 0x7ffacc160350 2024-04-17T09:56:57.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.328+0000 7ffad2200640 1 -- 172.21.15.32:0/1695350489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffacc15ec60 con 0x7ffacc160350 2024-04-17T09:56:57.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.328+0000 7ffad2200640 1 -- 172.21.15.32:0/1695350489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffacc0b4fb0 con 0x7ffacc160350 2024-04-17T09:56:57.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.332+0000 7ffac3400640 1 -- 172.21.15.32:0/1695350489 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ffac401a070 con 0x7ffacc160350 2024-04-17T09:56:57.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.332+0000 7ffac3400640 1 --2- 172.21.15.32:0/1695350489 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffaac068380 0x7ffaac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:57.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.332+0000 7ffac3400640 1 -- 172.21.15.32:0/1695350489 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7ffac4085ef0 con 0x7ffacc160350 2024-04-17T09:56:57.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.332+0000 7ffacb400640 1 --2- 172.21.15.32:0/1695350489 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffaac068380 0x7ffaac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:57.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.332+0000 7ffacb400640 1 --2- 172.21.15.32:0/1695350489 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffaac068380 0x7ffaac06a840 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ffacc0af5d0 tx=0x7ffabc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:57.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.336+0000 7ffac3400640 1 -- 172.21.15.32:0/1695350489 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ffacc13b0a0 con 0x7ffacc160350 2024-04-17T09:56:57.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.500+0000 7ffad2200640 1 -- 172.21.15.32:0/1695350489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ffacc068a90 con 0x7ffacc160350 2024-04-17T09:56:57.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.500+0000 7ffac3400640 1 -- 172.21.15.32:0/1695350489 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v57) ==== 104+0+11 (secure 0 0 0) 0x7ffac4055e50 con 0x7ffacc160350 2024-04-17T09:56:57.503 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:56:57.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.504+0000 7ffad2200640 1 -- 172.21.15.32:0/1695350489 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffaac068380 msgr2=0x7ffaac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:57.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.504+0000 7ffad2200640 1 --2- 172.21.15.32:0/1695350489 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffaac068380 0x7ffaac06a840 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ffacc0af5d0 tx=0x7ffabc008040 comp rx=0 tx=0).stop 2024-04-17T09:56:57.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.504+0000 7ffad2200640 1 -- 172.21.15.32:0/1695350489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffacc160350 msgr2=0x7ffacc0b42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:57.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.504+0000 7ffad2200640 1 --2- 172.21.15.32:0/1695350489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffacc160350 0x7ffacc0b42c0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ffac400c730 tx=0x7ffac400cbf0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.504+0000 7ffad2200640 1 -- 172.21.15.32:0/1695350489 shutdown_connections 2024-04-17T09:56:57.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.504+0000 7ffad2200640 1 --2- 172.21.15.32:0/1695350489 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffaac068380 0x7ffaac06a840 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.504+0000 7ffad2200640 1 --2- 172.21.15.32:0/1695350489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffacc0cf370 0x7ffacc0aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.504+0000 7ffad2200640 1 --2- 172.21.15.32:0/1695350489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffacc160350 0x7ffacc0b42c0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.504+0000 7ffad2200640 1 --2- 172.21.15.32:0/1695350489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffacc06ba30 0x7ffacc142ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.504+0000 7ffad2200640 1 -- 172.21.15.32:0/1695350489 >> 172.21.15.32:0/1695350489 conn(0x7ffacc0b2120 msgr2=0x7ffacc06f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:57.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.504+0000 7ffad2200640 1 -- 172.21.15.32:0/1695350489 shutdown_connections 2024-04-17T09:56:57.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.504+0000 7ffad2200640 1 -- 172.21.15.32:0/1695350489 wait complete. 2024-04-17T09:56:57.525 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:56:57.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.668+0000 7f38a8a00640 1 Processor -- start 2024-04-17T09:56:57.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.668+0000 7f38a8a00640 1 -- start start 2024-04-17T09:56:57.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.668+0000 7f38a8a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38a4141700 0x7f38a406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:57.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.668+0000 7f38a8a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38a406cf80 0x7f38a40af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:57.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.668+0000 7f38a8a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f38a40afce0 0x7f38a4162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:57.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.668+0000 7f38a8a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f38a406d3e0 con 0x7f38a4141700 2024-04-17T09:56:57.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.668+0000 7f38a8a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f38a40b0820 con 0x7f38a40afce0 2024-04-17T09:56:57.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.668+0000 7f38a8a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f38a40b0540 con 0x7f38a406cf80 2024-04-17T09:56:57.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.668+0000 7f38a1e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38a4141700 0x7f38a406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:57.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.668+0000 7f38a1e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38a4141700 0x7f38a406ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34884/0 (socket says 172.21.15.32:34884) 2024-04-17T09:56:57.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.668+0000 7f38a1e00640 1 -- 172.21.15.32:0/2095545208 learned_addr learned my addr 172.21.15.32:0/2095545208 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:57.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.668+0000 7f3899400640 1 --2- 172.21.15.32:0/2095545208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38a406cf80 0x7f38a40af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:57.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.668+0000 7f38a2800640 1 --2- 172.21.15.32:0/2095545208 >> v2:172.21.15.144:3300/0 conn(0x7f38a40afce0 0x7f38a4162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:57.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.668+0000 7f38a1e00640 1 -- 172.21.15.32:0/2095545208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38a406cf80 msgr2=0x7f38a40af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:57.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.668+0000 7f38a1e00640 1 --2- 172.21.15.32:0/2095545208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38a406cf80 0x7f38a40af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.668+0000 7f38a1e00640 1 -- 172.21.15.32:0/2095545208 >> v2:172.21.15.144:3300/0 conn(0x7f38a40afce0 msgr2=0x7f38a4162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:57.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.668+0000 7f38a1e00640 1 --2- 172.21.15.32:0/2095545208 >> v2:172.21.15.144:3300/0 conn(0x7f38a40afce0 0x7f38a4162320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.668+0000 7f38a1e00640 1 -- 172.21.15.32:0/2095545208 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a40b3ef0 con 0x7f38a4141700 2024-04-17T09:56:57.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.668+0000 7f3899400640 1 --2- 172.21.15.32:0/2095545208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38a406cf80 0x7f38a40af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:57.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.668+0000 7f38a2800640 1 --2- 172.21.15.32:0/2095545208 >> v2:172.21.15.144:3300/0 conn(0x7f38a40afce0 0x7f38a4162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:57.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.672+0000 7f38a1e00640 1 --2- 172.21.15.32:0/2095545208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38a4141700 0x7f38a406ca40 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f38a40706e0 tx=0x7f389401c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34493fe798e0a859 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:57.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.672+0000 7f38a1400640 1 -- 172.21.15.32:0/2095545208 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38940b0070 con 0x7f38a4141700 2024-04-17T09:56:57.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.672+0000 7f38a1400640 1 -- 172.21.15.32:0/2095545208 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38940ab600 con 0x7f38a4141700 2024-04-17T09:56:57.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.672+0000 7f38a1400640 1 -- 172.21.15.32:0/2095545208 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38940ab9d0 con 0x7f38a4141700 2024-04-17T09:56:57.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.672+0000 7f38a8a00640 1 -- 172.21.15.32:0/2095545208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38a4141700 msgr2=0x7f38a406ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:57.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.672+0000 7f38a8a00640 1 --2- 172.21.15.32:0/2095545208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38a4141700 0x7f38a406ca40 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f38a40706e0 tx=0x7f389401c230 comp rx=0 tx=0).stop 2024-04-17T09:56:57.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.672+0000 7f38a8a00640 1 -- 172.21.15.32:0/2095545208 shutdown_connections 2024-04-17T09:56:57.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.672+0000 7f38a8a00640 1 --2- 172.21.15.32:0/2095545208 >> v2:172.21.15.144:3300/0 conn(0x7f38a40afce0 0x7f38a4162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.672+0000 7f38a8a00640 1 --2- 172.21.15.32:0/2095545208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38a406cf80 0x7f38a40af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.672+0000 7f38a8a00640 1 --2- 172.21.15.32:0/2095545208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38a4141700 0x7f38a406ca40 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.672+0000 7f38a8a00640 1 -- 172.21.15.32:0/2095545208 >> 172.21.15.32:0/2095545208 conn(0x7f38a4072670 msgr2=0x7f38a4072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:57.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.672+0000 7f38a8a00640 1 -- 172.21.15.32:0/2095545208 shutdown_connections 2024-04-17T09:56:57.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.672+0000 7f38a8a00640 1 -- 172.21.15.32:0/2095545208 wait complete. 2024-04-17T09:56:57.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f38a8a00640 1 Processor -- start 2024-04-17T09:56:57.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f38a8a00640 1 -- start start 2024-04-17T09:56:57.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f38a8a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38a406cf80 0x7f38a415b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:57.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f38a8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38a40afce0 0x7f38a413eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:57.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f38a8a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38a4141700 0x7f38a413c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:57.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f38a8a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f38a4141ae0 con 0x7f38a40afce0 2024-04-17T09:56:57.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f38a8a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f38a40b0300 con 0x7f38a406cf80 2024-04-17T09:56:57.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f38a8a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f38a4141c60 con 0x7f38a4141700 2024-04-17T09:56:57.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f3899400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38a40afce0 0x7f38a413eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:57.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f3899400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38a40afce0 0x7f38a413eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34888/0 (socket says 172.21.15.32:34888) 2024-04-17T09:56:57.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f3899400640 1 -- 172.21.15.32:0/3736981941 learned_addr learned my addr 172.21.15.32:0/3736981941 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:57.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f38a1e00640 1 --2- 172.21.15.32:0/3736981941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38a406cf80 0x7f38a415b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:57.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f38a2800640 1 --2- 172.21.15.32:0/3736981941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38a4141700 0x7f38a413c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:57.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f3899400640 1 -- 172.21.15.32:0/3736981941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38a4141700 msgr2=0x7f38a413c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:57.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f3899400640 1 --2- 172.21.15.32:0/3736981941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38a4141700 0x7f38a413c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f3899400640 1 -- 172.21.15.32:0/3736981941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38a406cf80 msgr2=0x7f38a415b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:57.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f3899400640 1 --2- 172.21.15.32:0/3736981941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38a406cf80 0x7f38a415b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f3899400640 1 -- 172.21.15.32:0/3736981941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a413f590 con 0x7f38a40afce0 2024-04-17T09:56:57.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f38a2800640 1 --2- 172.21.15.32:0/3736981941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38a4141700 0x7f38a413c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:57.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f3899400640 1 --2- 172.21.15.32:0/3736981941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38a40afce0 0x7f38a413eb40 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f388c00c7f0 tx=0x7f388c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:57.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f389aa00640 1 -- 172.21.15.32:0/3736981941 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c015070 con 0x7f38a40afce0 2024-04-17T09:56:57.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f389aa00640 1 -- 172.21.15.32:0/3736981941 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f388c007820 con 0x7f38a40afce0 2024-04-17T09:56:57.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f389aa00640 1 -- 172.21.15.32:0/3736981941 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c007c10 con 0x7f38a40afce0 2024-04-17T09:56:57.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f38a8a00640 1 -- 172.21.15.32:0/3736981941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a413f870 con 0x7f38a40afce0 2024-04-17T09:56:57.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f38a8a00640 1 -- 172.21.15.32:0/3736981941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a413fe30 con 0x7f38a40afce0 2024-04-17T09:56:57.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.676+0000 7f38a8a00640 1 -- 172.21.15.32:0/3736981941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a4068a90 con 0x7f38a40afce0 2024-04-17T09:56:57.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.680+0000 7f389aa00640 1 -- 172.21.15.32:0/3736981941 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f388c018070 con 0x7f38a40afce0 2024-04-17T09:56:57.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.680+0000 7f389aa00640 1 --2- 172.21.15.32:0/3736981941 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3880068380 0x7f388006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:57.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.680+0000 7f389aa00640 1 -- 172.21.15.32:0/3736981941 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f388c086ae0 con 0x7f38a40afce0 2024-04-17T09:56:57.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.680+0000 7f38a1e00640 1 --2- 172.21.15.32:0/3736981941 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3880068380 0x7f388006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:57.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.680+0000 7f38a1e00640 1 --2- 172.21.15.32:0/3736981941 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3880068380 0x7f388006a840 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f38940ad000 tx=0x7f3894008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:57.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.684+0000 7f389aa00640 1 -- 172.21.15.32:0/3736981941 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f388c056a50 con 0x7f38a40afce0 2024-04-17T09:56:57.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.844+0000 7f38a8a00640 1 -- 172.21.15.32:0/3736981941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f38a413e3e0 con 0x7f38a40afce0 2024-04-17T09:56:57.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.848+0000 7f389aa00640 1 -- 172.21.15.32:0/3736981941 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v57) ==== 100+0+11 (secure 0 0 0) 0x7f388c05ad50 con 0x7f38a40afce0 2024-04-17T09:56:57.848 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:56:57.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.848+0000 7f38a8a00640 1 -- 172.21.15.32:0/3736981941 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3880068380 msgr2=0x7f388006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:57.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.848+0000 7f38a8a00640 1 --2- 172.21.15.32:0/3736981941 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3880068380 0x7f388006a840 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f38940ad000 tx=0x7f3894008040 comp rx=0 tx=0).stop 2024-04-17T09:56:57.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.848+0000 7f38a8a00640 1 -- 172.21.15.32:0/3736981941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38a40afce0 msgr2=0x7f38a413eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:57.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.848+0000 7f38a8a00640 1 --2- 172.21.15.32:0/3736981941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38a40afce0 0x7f38a413eb40 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f388c00c7f0 tx=0x7f388c00ccb0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.848+0000 7f38a8a00640 1 -- 172.21.15.32:0/3736981941 shutdown_connections 2024-04-17T09:56:57.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.848+0000 7f38a8a00640 1 --2- 172.21.15.32:0/3736981941 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3880068380 0x7f388006a840 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.848+0000 7f38a8a00640 1 --2- 172.21.15.32:0/3736981941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38a4141700 0x7f38a413c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.848+0000 7f38a8a00640 1 --2- 172.21.15.32:0/3736981941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38a40afce0 0x7f38a413eb40 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.848+0000 7f38a8a00640 1 --2- 172.21.15.32:0/3736981941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38a406cf80 0x7f38a415b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:57.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.848+0000 7f38a8a00640 1 -- 172.21.15.32:0/3736981941 >> 172.21.15.32:0/3736981941 conn(0x7f38a4072670 msgr2=0x7f38a40b5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:57.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.848+0000 7f38a8a00640 1 -- 172.21.15.32:0/3736981941 shutdown_connections 2024-04-17T09:56:57.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:57.848+0000 7f38a8a00640 1 -- 172.21.15.32:0/3736981941 wait complete. 2024-04-17T09:56:57.870 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:56:58.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.008+0000 7f7615400640 1 Processor -- start 2024-04-17T09:56:58.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.008+0000 7f7615400640 1 -- start start 2024-04-17T09:56:58.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.008+0000 7f7615400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7610145f30 0x7f7610071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:58.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.008+0000 7f7615400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f761015d8d0 0x7f7610072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:58.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.008+0000 7f7615400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76100cdac0 con 0x7f76100cce70 2024-04-17T09:56:58.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.008+0000 7f7615400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f761006f400 con 0x7f7610145f30 2024-04-17T09:56:58.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.008+0000 7f7615400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f7610146310 con 0x7f761015d8d0 2024-04-17T09:56:58.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.012+0000 7f760e800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7610145f30 0x7f7610071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:58.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.012+0000 7f760f200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f76100cce70 0x7f76100cd2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44272/0 (socket says 172.21.15.32:44272) 2024-04-17T09:56:58.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.012+0000 7f760f200640 1 -- 172.21.15.32:0/2444700542 learned_addr learned my addr 172.21.15.32:0/2444700542 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:58.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.012+0000 7f7607e00640 1 --2- 172.21.15.32:0/2444700542 >> v2:172.21.15.144:3300/0 conn(0x7f761015d8d0 0x7f7610072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:58.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.012+0000 7f760e800640 1 -- 172.21.15.32:0/2444700542 >> v2:172.21.15.144:3300/0 conn(0x7f761015d8d0 msgr2=0x7f7610072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:58.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.012+0000 7f760e800640 1 --2- 172.21.15.32:0/2444700542 >> v2:172.21.15.144:3300/0 conn(0x7f761015d8d0 0x7f7610072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.012+0000 7f760e800640 1 -- 172.21.15.32:0/2444700542 >> v1:172.21.15.32:6789/0 conn(0x7f76100cce70 legacy=0x7f76100cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:58.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.012+0000 7f760e800640 1 -- 172.21.15.32:0/2444700542 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761015cdf0 con 0x7f7610145f30 2024-04-17T09:56:58.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.012+0000 7f760de00640 1 -- 172.21.15.32:0/2444700542 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1164130108 0 0) 0x7f76100cdac0 con 0x7f76100cce70 2024-04-17T09:56:58.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.012+0000 7f760e800640 1 --2- 172.21.15.32:0/2444700542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7610145f30 0x7f7610071df0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f76100702a0 tx=0x7f760001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae7dd2d0d35abe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:58.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.012+0000 7f760de00640 1 -- 172.21.15.32:0/2444700542 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76000b5070 con 0x7f7610145f30 2024-04-17T09:56:58.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.012+0000 7f760de00640 1 -- 172.21.15.32:0/2444700542 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76000a8970 con 0x7f7610145f30 2024-04-17T09:56:58.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.012+0000 7f760de00640 1 -- 172.21.15.32:0/2444700542 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76000a8d40 con 0x7f7610145f30 2024-04-17T09:56:58.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f7615400640 1 -- 172.21.15.32:0/2444700542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7610145f30 msgr2=0x7f7610071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:58.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f7615400640 1 --2- 172.21.15.32:0/2444700542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7610145f30 0x7f7610071df0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f76100702a0 tx=0x7f760001ec70 comp rx=0 tx=0).stop 2024-04-17T09:56:58.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f7615400640 1 -- 172.21.15.32:0/2444700542 shutdown_connections 2024-04-17T09:56:58.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f7615400640 1 --2- 172.21.15.32:0/2444700542 >> v2:172.21.15.144:3300/0 conn(0x7f761015d8d0 0x7f7610072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f7615400640 1 --2- 172.21.15.32:0/2444700542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7610145f30 0x7f7610071df0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f7615400640 1 -- 172.21.15.32:0/2444700542 >> 172.21.15.32:0/2444700542 conn(0x7f761006ba30 msgr2=0x7f761006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:58.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f7615400640 1 -- 172.21.15.32:0/2444700542 shutdown_connections 2024-04-17T09:56:58.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f7615400640 1 -- 172.21.15.32:0/2444700542 wait complete. 2024-04-17T09:56:58.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f7615400640 1 Processor -- start 2024-04-17T09:56:58.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f7615400640 1 -- start start 2024-04-17T09:56:58.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f7615400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76100cce70 0x7f76100c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f7615400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7610145f30 0x7f76100b1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f7615400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f761015d8d0 0x7f76100c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f7615400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f761015cb90 con 0x7f76100cce70 2024-04-17T09:56:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f7615400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f761015d440 con 0x7f7610145f30 2024-04-17T09:56:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f7615400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f76101601d0 con 0x7f761015d8d0 2024-04-17T09:56:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f760e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76100cce70 0x7f76100c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f760e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76100cce70 0x7f76100c9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34908/0 (socket says 172.21.15.32:34908) 2024-04-17T09:56:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f760e800640 1 -- 172.21.15.32:0/2474393246 learned_addr learned my addr 172.21.15.32:0/2474393246 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f7607e00640 1 --2- 172.21.15.32:0/2474393246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7610145f30 0x7f76100b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f760e800640 1 -- 172.21.15.32:0/2474393246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f761015d8d0 msgr2=0x7f76100c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f760e800640 1 --2- 172.21.15.32:0/2474393246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f761015d8d0 0x7f76100c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f760e800640 1 -- 172.21.15.32:0/2474393246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7610145f30 msgr2=0x7f76100b1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f760e800640 1 --2- 172.21.15.32:0/2474393246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7610145f30 0x7f76100b1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.016+0000 7f760e800640 1 -- 172.21.15.32:0/2474393246 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7610130b60 con 0x7f76100cce70 2024-04-17T09:56:58.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.020+0000 7f760e800640 1 --2- 172.21.15.32:0/2474393246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76100cce70 0x7f76100c9000 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f760001c8a0 tx=0x7f760001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:58.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.020+0000 7f7606a00640 1 -- 172.21.15.32:0/2474393246 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76000c0070 con 0x7f76100cce70 2024-04-17T09:56:58.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.020+0000 7f7606a00640 1 -- 172.21.15.32:0/2474393246 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7600004230 con 0x7f76100cce70 2024-04-17T09:56:58.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.020+0000 7f7606a00640 1 -- 172.21.15.32:0/2474393246 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76000b5040 con 0x7f76100cce70 2024-04-17T09:56:58.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.020+0000 7f7615400640 1 -- 172.21.15.32:0/2474393246 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7610130e40 con 0x7f76100cce70 2024-04-17T09:56:58.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.020+0000 7f7615400640 1 -- 172.21.15.32:0/2474393246 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7610131400 con 0x7f76100cce70 2024-04-17T09:56:58.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.020+0000 7f7615400640 1 -- 172.21.15.32:0/2474393246 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7610068a90 con 0x7f76100cce70 2024-04-17T09:56:58.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.020+0000 7f7606a00640 1 -- 172.21.15.32:0/2474393246 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7600003620 con 0x7f76100cce70 2024-04-17T09:56:58.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.024+0000 7f7606a00640 1 --2- 172.21.15.32:0/2474393246 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f75ec068380 0x7f75ec06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:58.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.024+0000 7f7606a00640 1 -- 172.21.15.32:0/2474393246 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f7600123500 con 0x7f76100cce70 2024-04-17T09:56:58.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.024+0000 7f7607e00640 1 --2- 172.21.15.32:0/2474393246 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f75ec068380 0x7f75ec06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:58.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.024+0000 7f7607e00640 1 --2- 172.21.15.32:0/2474393246 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f75ec068380 0x7f75ec06a840 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f76100c8f90 tx=0x7f75f8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:58.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.028+0000 7f7606a00640 1 -- 172.21.15.32:0/2474393246 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f76000ca080 con 0x7f76100cce70 2024-04-17T09:56:58.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.232+0000 7f7615400640 1 -- 172.21.15.32:0/2474393246 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7610072f90 con 0x7f76100cce70 2024-04-17T09:56:58.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.232+0000 7f7606a00640 1 -- 172.21.15.32:0/2474393246 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5203 (secure 0 0 0) 0x7f76000187d0 con 0x7f76100cce70 2024-04-17T09:56:58.233 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:58.233 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5495,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/1236891874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1236891874},{"type":"v1","addr":"172.21.15.144:6834","nonce":1236891874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5498,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3538881342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3538881342},{"type":"v1","addr":"172.21.15.144:6835","nonce":3538881342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6201,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3549801633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3549801633},{"type":"v1","addr":"172.21.15.32:6836","nonce":3549801633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:56:01.579180+0000","modified":"2024-04-17T09:56:50.242920+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":15,"addr":"172.21.15.32:6837/866826055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":866826055},{"type":"v1","addr":"172.21.15.32:6837","nonce":866826055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-04-17T09:56:58.234 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 34 2024-04-17T09:56:58.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.232+0000 7f7615400640 1 -- 172.21.15.32:0/2474393246 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f75ec068380 msgr2=0x7f75ec06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:58.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.232+0000 7f7615400640 1 --2- 172.21.15.32:0/2474393246 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f75ec068380 0x7f75ec06a840 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f76100c8f90 tx=0x7f75f8008040 comp rx=0 tx=0).stop 2024-04-17T09:56:58.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.232+0000 7f7615400640 1 -- 172.21.15.32:0/2474393246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76100cce70 msgr2=0x7f76100c9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:58.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.232+0000 7f7615400640 1 --2- 172.21.15.32:0/2474393246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76100cce70 0x7f76100c9000 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f760001c8a0 tx=0x7f760001ba10 comp rx=0 tx=0).stop 2024-04-17T09:56:58.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.236+0000 7f7615400640 1 -- 172.21.15.32:0/2474393246 shutdown_connections 2024-04-17T09:56:58.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.236+0000 7f7615400640 1 --2- 172.21.15.32:0/2474393246 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f75ec068380 0x7f75ec06a840 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.236+0000 7f7615400640 1 --2- 172.21.15.32:0/2474393246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f761015d8d0 0x7f76100c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.236+0000 7f7615400640 1 --2- 172.21.15.32:0/2474393246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7610145f30 0x7f76100b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.236+0000 7f7615400640 1 --2- 172.21.15.32:0/2474393246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76100cce70 0x7f76100c9000 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.236+0000 7f7615400640 1 -- 172.21.15.32:0/2474393246 >> 172.21.15.32:0/2474393246 conn(0x7f761006ba30 msgr2=0x7f76100b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:58.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.236+0000 7f7615400640 1 -- 172.21.15.32:0/2474393246 shutdown_connections 2024-04-17T09:56:58.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.236+0000 7f7615400640 1 -- 172.21.15.32:0/2474393246 wait complete. 2024-04-17T09:56:58.255 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:56:58.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.396+0000 7fedf4a00640 1 Processor -- start 2024-04-17T09:56:58.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.396+0000 7fedf4a00640 1 -- start start 2024-04-17T09:56:58.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.396+0000 7fedf4a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fedf015eb10 0x7fedf015f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:58.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.396+0000 7fedf4a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fedf00d0f20 0x7fedf01603c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:58.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.396+0000 7fedf4a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedf00b2ac0 con 0x7fedf00b2550 2024-04-17T09:56:58.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.396+0000 7fedf4a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fedf015ff10 con 0x7fedf015eb10 2024-04-17T09:56:58.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.396+0000 7fedf4a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fedf00b2da0 con 0x7fedf00d0f20 2024-04-17T09:56:58.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.396+0000 7fede7e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fedf00b2550 0x7fedf00d0810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44280/0 (socket says 172.21.15.32:44280) 2024-04-17T09:56:58.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.396+0000 7fedede00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fedf015eb10 0x7fedf015f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:58.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.396+0000 7fede7e00640 1 -- 172.21.15.32:0/2278277167 learned_addr learned my addr 172.21.15.32:0/2278277167 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:58.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.400+0000 7fedede00640 1 -- 172.21.15.32:0/2278277167 >> v2:172.21.15.144:3300/0 conn(0x7fedf00d0f20 msgr2=0x7fedf01603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:58.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.400+0000 7fedee800640 1 --2- 172.21.15.32:0/2278277167 >> v2:172.21.15.144:3300/0 conn(0x7fedf00d0f20 0x7fedf01603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:58.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.400+0000 7fedede00640 1 --2- 172.21.15.32:0/2278277167 >> v2:172.21.15.144:3300/0 conn(0x7fedf00d0f20 0x7fedf01603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.400+0000 7feded400640 1 -- 172.21.15.32:0/2278277167 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 796014417 0 0) 0x7fedf00b2ac0 con 0x7fedf00b2550 2024-04-17T09:56:58.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.400+0000 7fedede00640 1 -- 172.21.15.32:0/2278277167 >> v1:172.21.15.32:6789/0 conn(0x7fedf00b2550 legacy=0x7fedf00d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:58.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.400+0000 7fedede00640 1 -- 172.21.15.32:0/2278277167 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf006bb60 con 0x7fedf015eb10 2024-04-17T09:56:58.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.400+0000 7fedee800640 1 --2- 172.21.15.32:0/2278277167 >> v2:172.21.15.144:3300/0 conn(0x7fedf00d0f20 0x7fedf01603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.400+0000 7fedede00640 1 --2- 172.21.15.32:0/2278277167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedf015eb10 0x7fedf015f9d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fedf00b29c0 tx=0x7fede001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f688a568b0285a5f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.400+0000 7feded400640 1 -- 172.21.15.32:0/2278277167 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede00b0070 con 0x7fedf015eb10 2024-04-17T09:56:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.400+0000 7feded400640 1 -- 172.21.15.32:0/2278277167 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fede00ab600 con 0x7fedf015eb10 2024-04-17T09:56:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.400+0000 7feded400640 1 -- 172.21.15.32:0/2278277167 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede00ab9d0 con 0x7fedf015eb10 2024-04-17T09:56:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.400+0000 7fedf4a00640 1 -- 172.21.15.32:0/2278277167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedf015eb10 msgr2=0x7fedf015f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.400+0000 7fedf4a00640 1 --2- 172.21.15.32:0/2278277167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedf015eb10 0x7fedf015f9d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fedf00b29c0 tx=0x7fede001c3f0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.400+0000 7fedf4a00640 1 -- 172.21.15.32:0/2278277167 shutdown_connections 2024-04-17T09:56:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.400+0000 7fedf4a00640 1 --2- 172.21.15.32:0/2278277167 >> v2:172.21.15.144:3300/0 conn(0x7fedf00d0f20 0x7fedf01603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.400+0000 7fedf4a00640 1 --2- 172.21.15.32:0/2278277167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedf015eb10 0x7fedf015f9d0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.400+0000 7fedf4a00640 1 -- 172.21.15.32:0/2278277167 >> 172.21.15.32:0/2278277167 conn(0x7fedf009a9f0 msgr2=0x7fedf00b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.400+0000 7fedf4a00640 1 -- 172.21.15.32:0/2278277167 shutdown_connections 2024-04-17T09:56:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.400+0000 7fedf4a00640 1 -- 172.21.15.32:0/2278277167 wait complete. 2024-04-17T09:56:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.404+0000 7fedf4a00640 1 Processor -- start 2024-04-17T09:56:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.404+0000 7fedf4a00640 1 -- start start 2024-04-17T09:56:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.404+0000 7fedf4a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedf00b2550 0x7fedf00b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.404+0000 7fedf4a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedf00d0f20 0x7fedf0152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.404+0000 7fedf4a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedf015eb10 0x7fedf0151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.404+0000 7fedf4a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fedf016bbf0 con 0x7fedf015eb10 2024-04-17T09:56:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.404+0000 7fedf4a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fedf016ba70 con 0x7fedf00d0f20 2024-04-17T09:56:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.404+0000 7fedf4a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fedf016bd70 con 0x7fedf00b2550 2024-04-17T09:56:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.404+0000 7fedee800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedf015eb10 0x7fedf0151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.404+0000 7fedee800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedf015eb10 0x7fedf0151b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34912/0 (socket says 172.21.15.32:34912) 2024-04-17T09:56:58.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.404+0000 7fedee800640 1 -- 172.21.15.32:0/520008900 learned_addr learned my addr 172.21.15.32:0/520008900 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:58.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.404+0000 7fedee800640 1 -- 172.21.15.32:0/520008900 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedf00b2550 msgr2=0x7fedf00b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:58.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.404+0000 7fedee800640 1 --2- 172.21.15.32:0/520008900 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedf00b2550 0x7fedf00b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.404+0000 7fedee800640 1 -- 172.21.15.32:0/520008900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedf00d0f20 msgr2=0x7fedf0152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:58.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.404+0000 7fedee800640 1 --2- 172.21.15.32:0/520008900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedf00d0f20 0x7fedf0152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.404+0000 7fedee800640 1 -- 172.21.15.32:0/520008900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf0153270 con 0x7fedf015eb10 2024-04-17T09:56:58.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.404+0000 7fedee800640 1 --2- 172.21.15.32:0/520008900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedf015eb10 0x7fedf0151b80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fede800cf20 tx=0x7fede800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.404+0000 7fede6000640 1 -- 172.21.15.32:0/520008900 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede801a070 con 0x7fedf015eb10 2024-04-17T09:56:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.404+0000 7fede6000640 1 -- 172.21.15.32:0/520008900 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fede800d840 con 0x7fedf015eb10 2024-04-17T09:56:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.404+0000 7fede6000640 1 -- 172.21.15.32:0/520008900 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede800dc30 con 0x7fedf015eb10 2024-04-17T09:56:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.404+0000 7fedf4a00640 1 -- 172.21.15.32:0/520008900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf0153510 con 0x7fedf015eb10 2024-04-17T09:56:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.404+0000 7fedf4a00640 1 -- 172.21.15.32:0/520008900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf0159db0 con 0x7fedf015eb10 2024-04-17T09:56:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.408+0000 7fedf4a00640 1 -- 172.21.15.32:0/520008900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedf0068a90 con 0x7fedf015eb10 2024-04-17T09:56:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.408+0000 7fede6000640 1 -- 172.21.15.32:0/520008900 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fede801d070 con 0x7fedf015eb10 2024-04-17T09:56:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.408+0000 7fede6000640 1 --2- 172.21.15.32:0/520008900 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fedcc068380 0x7fedcc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.408+0000 7fede6000640 1 -- 172.21.15.32:0/520008900 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fede8087270 con 0x7fedf015eb10 2024-04-17T09:56:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.408+0000 7fedede00640 1 --2- 172.21.15.32:0/520008900 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fedcc068380 0x7fedcc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.408+0000 7fedede00640 1 --2- 172.21.15.32:0/520008900 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fedcc068380 0x7fedcc06a840 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fede00ad000 tx=0x7fede0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:58.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.412+0000 7fede6000640 1 -- 172.21.15.32:0/520008900 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fede8057820 con 0x7fedf015eb10 2024-04-17T09:56:58.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.580+0000 7fedf4a00640 1 -- 172.21.15.32:0/520008900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fedf00b7450 con 0x7fedf015eb10 2024-04-17T09:56:58.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.580+0000 7fede6000640 1 -- 172.21.15.32:0/520008900 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16677 (secure 0 0 0) 0x7fede8057120 con 0x7fedf015eb10 2024-04-17T09:56:58.583 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:58.583 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":57,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:56:03.192930+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:55:59.976232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-17T09:56:00.543788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:56:58.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.584+0000 7fedf4a00640 1 -- 172.21.15.32:0/520008900 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fedcc068380 msgr2=0x7fedcc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:58.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.584+0000 7fedf4a00640 1 --2- 172.21.15.32:0/520008900 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fedcc068380 0x7fedcc06a840 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fede00ad000 tx=0x7fede0008040 comp rx=0 tx=0).stop 2024-04-17T09:56:58.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.584+0000 7fedf4a00640 1 -- 172.21.15.32:0/520008900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedf015eb10 msgr2=0x7fedf0151b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:58.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.584+0000 7fedf4a00640 1 --2- 172.21.15.32:0/520008900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedf015eb10 0x7fedf0151b80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fede800cf20 tx=0x7fede800a7d0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.584+0000 7fedf4a00640 1 -- 172.21.15.32:0/520008900 shutdown_connections 2024-04-17T09:56:58.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.584+0000 7fedf4a00640 1 --2- 172.21.15.32:0/520008900 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fedcc068380 0x7fedcc06a840 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.584+0000 7fedf4a00640 1 --2- 172.21.15.32:0/520008900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedf015eb10 0x7fedf0151b80 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.584+0000 7fedf4a00640 1 --2- 172.21.15.32:0/520008900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedf00d0f20 0x7fedf0152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.584+0000 7fedf4a00640 1 --2- 172.21.15.32:0/520008900 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedf00b2550 0x7fedf00b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.584+0000 7fedf4a00640 1 -- 172.21.15.32:0/520008900 >> 172.21.15.32:0/520008900 conn(0x7fedf009a9f0 msgr2=0x7fedf00d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:58.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.584+0000 7fedf4a00640 1 -- 172.21.15.32:0/520008900 shutdown_connections 2024-04-17T09:56:58.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.584+0000 7fedf4a00640 1 -- 172.21.15.32:0/520008900 wait complete. 2024-04-17T09:56:58.606 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T09:56:58.606 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:56:58.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.744+0000 7ffbd6c00640 1 Processor -- start 2024-04-17T09:56:58.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.744+0000 7ffbd6c00640 1 -- start start 2024-04-17T09:56:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.748+0000 7ffbd6c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffbd0145f30 0x7ffbd0071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.748+0000 7ffbd6c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ffbd015d8d0 0x7ffbd0072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.748+0000 7ffbd6c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbd00cdac0 con 0x7ffbd00cce70 2024-04-17T09:56:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.748+0000 7ffbd6c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ffbd006f400 con 0x7ffbd0145f30 2024-04-17T09:56:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.748+0000 7ffbd6c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ffbd0146310 con 0x7ffbd015d8d0 2024-04-17T09:56:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.748+0000 7ffbcfe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffbd0145f30 0x7ffbd0071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.748+0000 7ffbcfe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffbd0145f30 0x7ffbd0071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34924/0 (socket says 172.21.15.32:34924) 2024-04-17T09:56:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.748+0000 7ffbcfe00640 1 -- 172.21.15.32:0/130610967 learned_addr learned my addr 172.21.15.32:0/130610967 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.748+0000 7ffbcfe00640 1 -- 172.21.15.32:0/130610967 >> v2:172.21.15.144:3300/0 conn(0x7ffbd015d8d0 msgr2=0x7ffbd0072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:58.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.748+0000 7ffbcfe00640 1 --2- 172.21.15.32:0/130610967 >> v2:172.21.15.144:3300/0 conn(0x7ffbd015d8d0 0x7ffbd0072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.748+0000 7ffbcfe00640 1 -- 172.21.15.32:0/130610967 >> v1:172.21.15.144:6789/0 conn(0x7ffbd00cce70 legacy=0x7ffbd00cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:58.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.748+0000 7ffbcfe00640 1 -- 172.21.15.32:0/130610967 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd015cdf0 con 0x7ffbd0145f30 2024-04-17T09:56:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.752+0000 7ffbcfe00640 1 --2- 172.21.15.32:0/130610967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbd0145f30 0x7ffbd0071df0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00702a0 tx=0x7ffbc001ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a697e9b7e8125 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.752+0000 7ffbcf400640 1 -- 172.21.15.32:0/130610967 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc00b5070 con 0x7ffbd0145f30 2024-04-17T09:56:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.752+0000 7ffbcf400640 1 -- 172.21.15.32:0/130610967 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbc00a89d0 con 0x7ffbd0145f30 2024-04-17T09:56:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.752+0000 7ffbcf400640 1 -- 172.21.15.32:0/130610967 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc00a8da0 con 0x7ffbd0145f30 2024-04-17T09:56:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.752+0000 7ffbd6c00640 1 -- 172.21.15.32:0/130610967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbd0145f30 msgr2=0x7ffbd0071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.752+0000 7ffbd6c00640 1 --2- 172.21.15.32:0/130610967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbd0145f30 0x7ffbd0071df0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00702a0 tx=0x7ffbc001ecd0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.752+0000 7ffbd6c00640 1 -- 172.21.15.32:0/130610967 shutdown_connections 2024-04-17T09:56:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.752+0000 7ffbd6c00640 1 --2- 172.21.15.32:0/130610967 >> v2:172.21.15.144:3300/0 conn(0x7ffbd015d8d0 0x7ffbd0072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.752+0000 7ffbd6c00640 1 --2- 172.21.15.32:0/130610967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbd0145f30 0x7ffbd0071df0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.752+0000 7ffbd6c00640 1 -- 172.21.15.32:0/130610967 >> 172.21.15.32:0/130610967 conn(0x7ffbd006ba30 msgr2=0x7ffbd006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:58.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.752+0000 7ffbd6c00640 1 -- 172.21.15.32:0/130610967 shutdown_connections 2024-04-17T09:56:58.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.752+0000 7ffbd6c00640 1 -- 172.21.15.32:0/130610967 wait complete. 2024-04-17T09:56:58.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.752+0000 7ffbd6c00640 1 Processor -- start 2024-04-17T09:56:58.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.752+0000 7ffbd6c00640 1 -- start start 2024-04-17T09:56:58.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.752+0000 7ffbd6c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbd00cce70 0x7ffbd013dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:58.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.752+0000 7ffbd6c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbd0145f30 0x7ffbd0141a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:58.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.752+0000 7ffbd6c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbd015d8d0 0x7ffbd00cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:58.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.752+0000 7ffbd6c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd015cb90 con 0x7ffbd0145f30 2024-04-17T09:56:58.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.752+0000 7ffbd6c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd015d440 con 0x7ffbd015d8d0 2024-04-17T09:56:58.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.752+0000 7ffbd6c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffbd01601d0 con 0x7ffbd00cce70 2024-04-17T09:56:58.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.752+0000 7ffbc7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbd0145f30 0x7ffbd0141a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:58.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.752+0000 7ffbc7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbd0145f30 0x7ffbd0141a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34936/0 (socket says 172.21.15.32:34936) 2024-04-17T09:56:58.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.752+0000 7ffbc7400640 1 -- 172.21.15.32:0/4250666143 learned_addr learned my addr 172.21.15.32:0/4250666143 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:58.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.756+0000 7ffbcfe00640 1 --2- 172.21.15.32:0/4250666143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbd00cce70 0x7ffbd013dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:58.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.756+0000 7ffbd4c00640 1 --2- 172.21.15.32:0/4250666143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbd015d8d0 0x7ffbd00cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:58.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.756+0000 7ffbc7400640 1 -- 172.21.15.32:0/4250666143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbd00cce70 msgr2=0x7ffbd013dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:58.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.756+0000 7ffbc7400640 1 --2- 172.21.15.32:0/4250666143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbd00cce70 0x7ffbd013dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.756+0000 7ffbc7400640 1 -- 172.21.15.32:0/4250666143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbd015d8d0 msgr2=0x7ffbd00cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:58.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.756+0000 7ffbc7400640 1 --2- 172.21.15.32:0/4250666143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbd015d8d0 0x7ffbd00cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.756+0000 7ffbc7400640 1 -- 172.21.15.32:0/4250666143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd013e2d0 con 0x7ffbd0145f30 2024-04-17T09:56:58.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.756+0000 7ffbc7400640 1 --2- 172.21.15.32:0/4250666143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbd0145f30 0x7ffbd0141a40 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800b440 tx=0x7ffbb800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:58.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.756+0000 7ffbccc00640 1 -- 172.21.15.32:0/4250666143 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8015020 con 0x7ffbd0145f30 2024-04-17T09:56:58.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.756+0000 7ffbccc00640 1 -- 172.21.15.32:0/4250666143 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbb80118b0 con 0x7ffbd0145f30 2024-04-17T09:56:58.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.756+0000 7ffbccc00640 1 -- 172.21.15.32:0/4250666143 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8011ca0 con 0x7ffbd0145f30 2024-04-17T09:56:58.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.756+0000 7ffbd6c00640 1 -- 172.21.15.32:0/4250666143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd0142390 con 0x7ffbd0145f30 2024-04-17T09:56:58.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.756+0000 7ffbd6c00640 1 -- 172.21.15.32:0/4250666143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd0142950 con 0x7ffbd0145f30 2024-04-17T09:56:58.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.756+0000 7ffbd6c00640 1 -- 172.21.15.32:0/4250666143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbd0068a90 con 0x7ffbd0145f30 2024-04-17T09:56:58.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.756+0000 7ffbccc00640 1 -- 172.21.15.32:0/4250666143 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ffbb8018020 con 0x7ffbd0145f30 2024-04-17T09:56:58.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.756+0000 7ffbccc00640 1 --2- 172.21.15.32:0/4250666143 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffbac068380 0x7ffbac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:58.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.756+0000 7ffbccc00640 1 -- 172.21.15.32:0/4250666143 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7ffbb8086a00 con 0x7ffbd0145f30 2024-04-17T09:56:58.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.760+0000 7ffbcfe00640 1 --2- 172.21.15.32:0/4250666143 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffbac068380 0x7ffbac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:58.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.760+0000 7ffbcfe00640 1 --2- 172.21.15.32:0/4250666143 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffbac068380 0x7ffbac06a840 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00b2000 tx=0x7ffbc0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:58.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.764+0000 7ffbccc00640 1 -- 172.21.15.32:0/4250666143 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ffbb801f020 con 0x7ffbd0145f30 2024-04-17T09:56:58.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.960+0000 7ffbd6c00640 1 -- 172.21.15.32:0/4250666143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbd0072f90 con 0x7ffbd0145f30 2024-04-17T09:56:58.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.960+0000 7ffbccc00640 1 -- 172.21.15.32:0/4250666143 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5203 (secure 0 0 0) 0x7ffbb800d040 con 0x7ffbd0145f30 2024-04-17T09:56:58.962 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:58.962 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5495,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/1236891874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1236891874},{"type":"v1","addr":"172.21.15.144:6834","nonce":1236891874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5498,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3538881342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3538881342},{"type":"v1","addr":"172.21.15.144:6835","nonce":3538881342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6201,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3549801633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3549801633},{"type":"v1","addr":"172.21.15.32:6836","nonce":3549801633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:56:01.579180+0000","modified":"2024-04-17T09:56:50.242920+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":15,"addr":"172.21.15.32:6837/866826055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":866826055},{"type":"v1","addr":"172.21.15.32:6837","nonce":866826055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-04-17T09:56:58.962 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 34 2024-04-17T09:56:58.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.964+0000 7ffbd6c00640 1 -- 172.21.15.32:0/4250666143 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffbac068380 msgr2=0x7ffbac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:58.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.964+0000 7ffbd6c00640 1 --2- 172.21.15.32:0/4250666143 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffbac068380 0x7ffbac06a840 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00b2000 tx=0x7ffbc0008040 comp rx=0 tx=0).stop 2024-04-17T09:56:58.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.964+0000 7ffbd6c00640 1 -- 172.21.15.32:0/4250666143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbd0145f30 msgr2=0x7ffbd0141a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:58.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.964+0000 7ffbd6c00640 1 --2- 172.21.15.32:0/4250666143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbd0145f30 0x7ffbd0141a40 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800b440 tx=0x7ffbb800b900 comp rx=0 tx=0).stop 2024-04-17T09:56:58.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.964+0000 7ffbd6c00640 1 -- 172.21.15.32:0/4250666143 shutdown_connections 2024-04-17T09:56:58.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.964+0000 7ffbd6c00640 1 --2- 172.21.15.32:0/4250666143 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffbac068380 0x7ffbac06a840 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.964+0000 7ffbd6c00640 1 --2- 172.21.15.32:0/4250666143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbd015d8d0 0x7ffbd00cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.964+0000 7ffbd6c00640 1 --2- 172.21.15.32:0/4250666143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbd0145f30 0x7ffbd0141a40 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.964+0000 7ffbd6c00640 1 --2- 172.21.15.32:0/4250666143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbd00cce70 0x7ffbd013dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:58.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.964+0000 7ffbd6c00640 1 -- 172.21.15.32:0/4250666143 >> 172.21.15.32:0/4250666143 conn(0x7ffbd006ba30 msgr2=0x7ffbd00ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:58.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.964+0000 7ffbd6c00640 1 -- 172.21.15.32:0/4250666143 shutdown_connections 2024-04-17T09:56:58.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:58.964+0000 7ffbd6c00640 1 -- 172.21.15.32:0/4250666143 wait complete. 2024-04-17T09:56:58.984 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:56:59.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.124+0000 7fe031800640 1 Processor -- start 2024-04-17T09:56:59.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.124+0000 7fe031800640 1 -- start start 2024-04-17T09:56:59.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.124+0000 7fe031800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe02c15eb10 0x7fe02c15f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:59.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.124+0000 7fe031800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe02c0b2ac0 con 0x7fe02c0b2550 2024-04-17T09:56:59.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.124+0000 7fe031800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe02c06bb60 con 0x7fe02c0d0f20 2024-04-17T09:56:59.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.124+0000 7fe031800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe02c15ff10 con 0x7fe02c15eb10 2024-04-17T09:56:59.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.124+0000 7fe02a000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe02c0b2550 0x7fe02c0d0810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44294/0 (socket says 172.21.15.32:44294) 2024-04-17T09:56:59.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.124+0000 7fe02a000640 1 -- 172.21.15.32:0/1810750455 learned_addr learned my addr 172.21.15.32:0/1810750455 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:56:59.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.124+0000 7fe02aa00640 1 --2- 172.21.15.32:0/1810750455 >> v2:172.21.15.32:3300/0 conn(0x7fe02c15eb10 0x7fe02c15f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:59.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.124+0000 7fe029600640 1 -- 172.21.15.32:0/1810750455 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1350429346 0 0) 0x7fe02c0b2ac0 con 0x7fe02c0b2550 2024-04-17T09:56:59.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.124+0000 7fe029600640 1 -- 172.21.15.32:0/1810750455 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe010003530 con 0x7fe02c0b2550 2024-04-17T09:56:59.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.124+0000 7fe02aa00640 1 -- 172.21.15.32:0/1810750455 >> v1:172.21.15.144:6789/0 conn(0x7fe02c0d0f20 legacy=0x7fe02c1603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:59.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.124+0000 7fe02aa00640 1 -- 172.21.15.32:0/1810750455 >> v1:172.21.15.32:6789/0 conn(0x7fe02c0b2550 legacy=0x7fe02c0d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:59.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.124+0000 7fe02aa00640 1 -- 172.21.15.32:0/1810750455 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe02c06be40 con 0x7fe02c15eb10 2024-04-17T09:56:59.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.128+0000 7fe02aa00640 1 --2- 172.21.15.32:0/1810750455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe02c15eb10 0x7fe02c15f9d0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0097c0 tx=0x7fe02c1612e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c66f89426f899af server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:59.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.128+0000 7fe029600640 1 -- 172.21.15.32:0/1810750455 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c0b0070 con 0x7fe02c15eb10 2024-04-17T09:56:59.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.128+0000 7fe029600640 1 -- 172.21.15.32:0/1810750455 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe01c0ab600 con 0x7fe02c15eb10 2024-04-17T09:56:59.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.128+0000 7fe029600640 1 -- 172.21.15.32:0/1810750455 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c0ab9d0 con 0x7fe02c15eb10 2024-04-17T09:56:59.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.128+0000 7fe031800640 1 -- 172.21.15.32:0/1810750455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe02c15eb10 msgr2=0x7fe02c15f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:59.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.128+0000 7fe031800640 1 --2- 172.21.15.32:0/1810750455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe02c15eb10 0x7fe02c15f9d0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0097c0 tx=0x7fe02c1612e0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.128+0000 7fe031800640 1 -- 172.21.15.32:0/1810750455 shutdown_connections 2024-04-17T09:56:59.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.128+0000 7fe031800640 1 --2- 172.21.15.32:0/1810750455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe02c15eb10 0x7fe02c15f9d0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.128+0000 7fe031800640 1 -- 172.21.15.32:0/1810750455 >> 172.21.15.32:0/1810750455 conn(0x7fe02c09a9f0 msgr2=0x7fe02c0b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:59.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.128+0000 7fe031800640 1 -- 172.21.15.32:0/1810750455 shutdown_connections 2024-04-17T09:56:59.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.128+0000 7fe031800640 1 -- 172.21.15.32:0/1810750455 wait complete. 2024-04-17T09:56:59.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.128+0000 7fe031800640 1 Processor -- start 2024-04-17T09:56:59.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.128+0000 7fe031800640 1 -- start start 2024-04-17T09:56:59.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe031800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe02c0b2550 0x7fe02c1553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:59.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe031800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe02c0d0f20 0x7fe02c0b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:59.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe031800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe02c15eb10 0x7fe02c0b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:59.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe031800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe02c16ca80 con 0x7fe02c0d0f20 2024-04-17T09:56:59.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe031800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe02c0b2da0 con 0x7fe02c0b2550 2024-04-17T09:56:59.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe031800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe02c16cc00 con 0x7fe02c15eb10 2024-04-17T09:56:59.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe02a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe02c0d0f20 0x7fe02c0b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:59.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe02a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe02c0d0f20 0x7fe02c0b08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34958/0 (socket says 172.21.15.32:34958) 2024-04-17T09:56:59.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe02a000640 1 -- 172.21.15.32:0/3646836025 learned_addr learned my addr 172.21.15.32:0/3646836025 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:59.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe02b400640 1 --2- 172.21.15.32:0/3646836025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe02c15eb10 0x7fe02c0b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:59.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe02aa00640 1 --2- 172.21.15.32:0/3646836025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe02c0b2550 0x7fe02c1553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:59.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe02a000640 1 -- 172.21.15.32:0/3646836025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe02c15eb10 msgr2=0x7fe02c0b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:59.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe02a000640 1 --2- 172.21.15.32:0/3646836025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe02c15eb10 0x7fe02c0b0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe02a000640 1 -- 172.21.15.32:0/3646836025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe02c0b2550 msgr2=0x7fe02c1553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:59.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe02a000640 1 --2- 172.21.15.32:0/3646836025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe02c0b2550 0x7fe02c1553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe02a000640 1 -- 172.21.15.32:0/3646836025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe02c0b57f0 con 0x7fe02c0d0f20 2024-04-17T09:56:59.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe02b400640 1 --2- 172.21.15.32:0/3646836025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe02c15eb10 0x7fe02c0b0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:59.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe02aa00640 1 --2- 172.21.15.32:0/3646836025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe02c0b2550 0x7fe02c1553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:56:59.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe02a000640 1 --2- 172.21.15.32:0/3646836025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe02c0d0f20 0x7fe02c0b08c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe01800c4d0 tx=0x7fe01800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:59.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe022a00640 1 -- 172.21.15.32:0/3646836025 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe018017070 con 0x7fe02c0d0f20 2024-04-17T09:56:59.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe022a00640 1 -- 172.21.15.32:0/3646836025 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe018005cf0 con 0x7fe02c0d0f20 2024-04-17T09:56:59.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe022a00640 1 -- 172.21.15.32:0/3646836025 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe018007270 con 0x7fe02c0d0f20 2024-04-17T09:56:59.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe031800640 1 -- 172.21.15.32:0/3646836025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe02c0b5ad0 con 0x7fe02c0d0f20 2024-04-17T09:56:59.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe031800640 1 -- 172.21.15.32:0/3646836025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe02c0b6090 con 0x7fe02c0d0f20 2024-04-17T09:56:59.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe031800640 1 -- 172.21.15.32:0/3646836025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe02c068a90 con 0x7fe02c0d0f20 2024-04-17T09:56:59.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.132+0000 7fe022a00640 1 -- 172.21.15.32:0/3646836025 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe01801a070 con 0x7fe02c0d0f20 2024-04-17T09:56:59.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.136+0000 7fe022a00640 1 --2- 172.21.15.32:0/3646836025 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe00c068380 0x7fe00c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:59.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.136+0000 7fe022a00640 1 -- 172.21.15.32:0/3646836025 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fe018085d90 con 0x7fe02c0d0f20 2024-04-17T09:56:59.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.136+0000 7fe02aa00640 1 --2- 172.21.15.32:0/3646836025 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe00c068380 0x7fe00c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:59.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.136+0000 7fe02aa00640 1 --2- 172.21.15.32:0/3646836025 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe00c068380 0x7fe00c06a840 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0ad000 tx=0x7fe01c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.140+0000 7fe022a00640 1 -- 172.21.15.32:0/3646836025 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fe018055c60 con 0x7fe02c0d0f20 2024-04-17T09:56:59.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.340+0000 7fe031800640 1 -- 172.21.15.32:0/3646836025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe02c0731e0 con 0x7fe02c0d0f20 2024-04-17T09:56:59.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.340+0000 7fe022a00640 1 -- 172.21.15.32:0/3646836025 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5203 (secure 0 0 0) 0x7fe018059f60 con 0x7fe02c0d0f20 2024-04-17T09:56:59.343 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:59.343 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5495,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/1236891874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1236891874},{"type":"v1","addr":"172.21.15.144:6834","nonce":1236891874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5498,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3538881342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3538881342},{"type":"v1","addr":"172.21.15.144:6835","nonce":3538881342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6201,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3549801633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3549801633},{"type":"v1","addr":"172.21.15.32:6836","nonce":3549801633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:56:01.579180+0000","modified":"2024-04-17T09:56:50.242920+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":15,"addr":"172.21.15.32:6837/866826055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":866826055},{"type":"v1","addr":"172.21.15.32:6837","nonce":866826055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-04-17T09:56:59.343 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 34 2024-04-17T09:56:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.344+0000 7fe031800640 1 -- 172.21.15.32:0/3646836025 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe00c068380 msgr2=0x7fe00c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.344+0000 7fe031800640 1 --2- 172.21.15.32:0/3646836025 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe00c068380 0x7fe00c06a840 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0ad000 tx=0x7fe01c008040 comp rx=0 tx=0).stop 2024-04-17T09:56:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.344+0000 7fe031800640 1 -- 172.21.15.32:0/3646836025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe02c0d0f20 msgr2=0x7fe02c0b08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.344+0000 7fe031800640 1 --2- 172.21.15.32:0/3646836025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe02c0d0f20 0x7fe02c0b08c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe01800c4d0 tx=0x7fe01800c990 comp rx=0 tx=0).stop 2024-04-17T09:56:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.344+0000 7fe031800640 1 -- 172.21.15.32:0/3646836025 shutdown_connections 2024-04-17T09:56:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.344+0000 7fe031800640 1 --2- 172.21.15.32:0/3646836025 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe00c068380 0x7fe00c06a840 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.344+0000 7fe031800640 1 --2- 172.21.15.32:0/3646836025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe02c15eb10 0x7fe02c0b0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.344+0000 7fe031800640 1 --2- 172.21.15.32:0/3646836025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe02c0d0f20 0x7fe02c0b08c0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.344+0000 7fe031800640 1 --2- 172.21.15.32:0/3646836025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe02c0b2550 0x7fe02c1553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.344+0000 7fe031800640 1 -- 172.21.15.32:0/3646836025 >> 172.21.15.32:0/3646836025 conn(0x7fe02c09a9f0 msgr2=0x7fe02c0ae330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.344+0000 7fe031800640 1 -- 172.21.15.32:0/3646836025 shutdown_connections 2024-04-17T09:56:59.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.344+0000 7fe031800640 1 -- 172.21.15.32:0/3646836025 wait complete. 2024-04-17T09:56:59.365 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:56:59.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.504+0000 7fbdbc000640 1 Processor -- start 2024-04-17T09:56:59.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.504+0000 7fbdbc000640 1 -- start start 2024-04-17T09:56:59.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.504+0000 7fbdbc000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdb4162bb0 0x7fbdb406e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:59.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.504+0000 7fbdbc000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbdb4071df0 0x7fbdb4072250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:59.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.504+0000 7fbdbc000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbdb41667e0 0x7fbdb4166c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:59.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.504+0000 7fbdbc000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fbdb4072790 con 0x7fbdb4071df0 2024-04-17T09:56:59.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.504+0000 7fbdbc000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fbdb40cfe60 con 0x7fbdb41667e0 2024-04-17T09:56:59.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.504+0000 7fbdbc000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbdb406e540 con 0x7fbdb4162bb0 2024-04-17T09:56:59.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.504+0000 7fbdb8c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbdb4071df0 0x7fbdb4072250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:59.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.504+0000 7fbdb8c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbdb4071df0 0x7fbdb4072250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34960/0 (socket says 172.21.15.32:34960) 2024-04-17T09:56:59.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.504+0000 7fbdb8c00640 1 -- 172.21.15.32:0/2369169820 learned_addr learned my addr 172.21.15.32:0/2369169820 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:59.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.504+0000 7fbdba000640 1 --2- 172.21.15.32:0/2369169820 >> v2:172.21.15.144:3300/0 conn(0x7fbdb41667e0 0x7fbdb4166c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:59.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.504+0000 7fbdb9600640 1 --2- 172.21.15.32:0/2369169820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdb4162bb0 0x7fbdb406e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:59.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.508+0000 7fbdb8c00640 1 -- 172.21.15.32:0/2369169820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdb4162bb0 msgr2=0x7fbdb406e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:59.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.508+0000 7fbdb8c00640 1 --2- 172.21.15.32:0/2369169820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdb4162bb0 0x7fbdb406e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.508+0000 7fbdb8c00640 1 -- 172.21.15.32:0/2369169820 >> v2:172.21.15.144:3300/0 conn(0x7fbdb41667e0 msgr2=0x7fbdb4166c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:59.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.508+0000 7fbdb8c00640 1 --2- 172.21.15.32:0/2369169820 >> v2:172.21.15.144:3300/0 conn(0x7fbdb41667e0 0x7fbdb4166c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.508+0000 7fbdb8c00640 1 -- 172.21.15.32:0/2369169820 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb4167170 con 0x7fbdb4071df0 2024-04-17T09:56:59.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.512+0000 7fbdb8c00640 1 --2- 172.21.15.32:0/2369169820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdb4071df0 0x7fbdb4072250 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4073270 tx=0x7fbda001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bbee1fd32d3707c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:59.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.512+0000 7fbdb3e00640 1 -- 172.21.15.32:0/2369169820 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda00b1070 con 0x7fbdb4071df0 2024-04-17T09:56:59.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.512+0000 7fbdb3e00640 1 -- 172.21.15.32:0/2369169820 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbda00ac630 con 0x7fbdb4071df0 2024-04-17T09:56:59.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.512+0000 7fbdb3e00640 1 -- 172.21.15.32:0/2369169820 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda00a8050 con 0x7fbdb4071df0 2024-04-17T09:56:59.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.512+0000 7fbdbc000640 1 -- 172.21.15.32:0/2369169820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdb4071df0 msgr2=0x7fbdb4072250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:59.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.512+0000 7fbdbc000640 1 --2- 172.21.15.32:0/2369169820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdb4071df0 0x7fbdb4072250 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4073270 tx=0x7fbda001c520 comp rx=0 tx=0).stop 2024-04-17T09:56:59.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.512+0000 7fbdbc000640 1 -- 172.21.15.32:0/2369169820 shutdown_connections 2024-04-17T09:56:59.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.512+0000 7fbdbc000640 1 --2- 172.21.15.32:0/2369169820 >> v2:172.21.15.144:3300/0 conn(0x7fbdb41667e0 0x7fbdb4166c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.512+0000 7fbdbc000640 1 --2- 172.21.15.32:0/2369169820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdb4071df0 0x7fbdb4072250 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.512+0000 7fbdbc000640 1 --2- 172.21.15.32:0/2369169820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdb4162bb0 0x7fbdb406e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.512+0000 7fbdbc000640 1 -- 172.21.15.32:0/2369169820 >> 172.21.15.32:0/2369169820 conn(0x7fbdb406ba30 msgr2=0x7fbdb406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:59.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.512+0000 7fbdbc000640 1 -- 172.21.15.32:0/2369169820 shutdown_connections 2024-04-17T09:56:59.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.512+0000 7fbdbc000640 1 -- 172.21.15.32:0/2369169820 wait complete. 2024-04-17T09:56:59.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdbc000640 1 Processor -- start 2024-04-17T09:56:59.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdbc000640 1 -- start start 2024-04-17T09:56:59.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdbc000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdb4071df0 0x7fbdb4147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:59.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdbc000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdb4162bb0 0x7fbdb4163620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:59.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdbc000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdb41667e0 0x7fbdb4163b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:59.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdbc000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb4162f90 con 0x7fbdb41667e0 2024-04-17T09:56:59.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdbc000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb406e1d0 con 0x7fbdb4162bb0 2024-04-17T09:56:59.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdbc000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbdb40b4fc0 con 0x7fbdb4071df0 2024-04-17T09:56:59.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdba000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdb41667e0 0x7fbdb4163b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:59.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdba000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdb41667e0 0x7fbdb4163b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34964/0 (socket says 172.21.15.32:34964) 2024-04-17T09:56:59.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdba000640 1 -- 172.21.15.32:0/1282851682 learned_addr learned my addr 172.21.15.32:0/1282851682 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:59.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdb8c00640 1 --2- 172.21.15.32:0/1282851682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdb4162bb0 0x7fbdb4163620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:59.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdb9600640 1 --2- 172.21.15.32:0/1282851682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdb4071df0 0x7fbdb4147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:59.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdba000640 1 -- 172.21.15.32:0/1282851682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdb4071df0 msgr2=0x7fbdb4147380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:59.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdba000640 1 --2- 172.21.15.32:0/1282851682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdb4071df0 0x7fbdb4147380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdba000640 1 -- 172.21.15.32:0/1282851682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdb4162bb0 msgr2=0x7fbdb4163620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:59.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdba000640 1 --2- 172.21.15.32:0/1282851682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdb4162bb0 0x7fbdb4163620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdba000640 1 -- 172.21.15.32:0/1282851682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb40b0ee0 con 0x7fbdb41667e0 2024-04-17T09:56:59.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdba000640 1 --2- 172.21.15.32:0/1282851682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdb41667e0 0x7fbdb4163b60 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fbdac009d30 tx=0x7fbdac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:59.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdb1600640 1 -- 172.21.15.32:0/1282851682 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac018020 con 0x7fbdb41667e0 2024-04-17T09:56:59.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdb1600640 1 -- 172.21.15.32:0/1282851682 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdac014480 con 0x7fbdb41667e0 2024-04-17T09:56:59.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdb1600640 1 -- 172.21.15.32:0/1282851682 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac014870 con 0x7fbdb41667e0 2024-04-17T09:56:59.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdbc000640 1 -- 172.21.15.32:0/1282851682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb40b0120 con 0x7fbdb41667e0 2024-04-17T09:56:59.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.516+0000 7fbdbc000640 1 -- 172.21.15.32:0/1282851682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb40b06e0 con 0x7fbdb41667e0 2024-04-17T09:56:59.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.520+0000 7fbdbc000640 1 -- 172.21.15.32:0/1282851682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdb4068a90 con 0x7fbdb41667e0 2024-04-17T09:56:59.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.520+0000 7fbdb1600640 1 -- 172.21.15.32:0/1282851682 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbdac01b020 con 0x7fbdb41667e0 2024-04-17T09:56:59.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.520+0000 7fbdb1600640 1 --2- 172.21.15.32:0/1282851682 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbd94068380 0x7fbd9406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:59.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.520+0000 7fbdb1600640 1 -- 172.21.15.32:0/1282851682 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fbdac087350 con 0x7fbdb41667e0 2024-04-17T09:56:59.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.520+0000 7fbdb9600640 1 --2- 172.21.15.32:0/1282851682 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbd94068380 0x7fbd9406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:59.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.520+0000 7fbdb9600640 1 --2- 172.21.15.32:0/1282851682 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbd94068380 0x7fbd9406a840 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbda8009ae0 tx=0x7fbda8009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:59.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.524+0000 7fbdb1600640 1 -- 172.21.15.32:0/1282851682 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fbdac022070 con 0x7fbdb41667e0 2024-04-17T09:56:59.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.692+0000 7fbdbc000640 1 -- 172.21.15.32:0/1282851682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbdb416d400 con 0x7fbdb41667e0 2024-04-17T09:56:59.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.692+0000 7fbdb1600640 1 -- 172.21.15.32:0/1282851682 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16677 (secure 0 0 0) 0x7fbdac010040 con 0x7fbdb41667e0 2024-04-17T09:56:59.695 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:56:59.695 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":57,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:56:03.192930+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:55:59.976232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-17T09:56:00.543788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:56:59.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.696+0000 7fbdbc000640 1 -- 172.21.15.32:0/1282851682 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbd94068380 msgr2=0x7fbd9406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:59.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.696+0000 7fbdbc000640 1 --2- 172.21.15.32:0/1282851682 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbd94068380 0x7fbd9406a840 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbda8009ae0 tx=0x7fbda8009190 comp rx=0 tx=0).stop 2024-04-17T09:56:59.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.696+0000 7fbdbc000640 1 -- 172.21.15.32:0/1282851682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdb41667e0 msgr2=0x7fbdb4163b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:59.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.696+0000 7fbdbc000640 1 --2- 172.21.15.32:0/1282851682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdb41667e0 0x7fbdb4163b60 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fbdac009d30 tx=0x7fbdac00c490 comp rx=0 tx=0).stop 2024-04-17T09:56:59.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.696+0000 7fbdbc000640 1 -- 172.21.15.32:0/1282851682 shutdown_connections 2024-04-17T09:56:59.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.696+0000 7fbdbc000640 1 --2- 172.21.15.32:0/1282851682 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbd94068380 0x7fbd9406a840 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.696+0000 7fbdbc000640 1 --2- 172.21.15.32:0/1282851682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdb41667e0 0x7fbdb4163b60 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.696+0000 7fbdbc000640 1 --2- 172.21.15.32:0/1282851682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdb4162bb0 0x7fbdb4163620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.696+0000 7fbdbc000640 1 --2- 172.21.15.32:0/1282851682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdb4071df0 0x7fbdb4147380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.696+0000 7fbdbc000640 1 -- 172.21.15.32:0/1282851682 >> 172.21.15.32:0/1282851682 conn(0x7fbdb406ba30 msgr2=0x7fbdb406f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:59.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.696+0000 7fbdbc000640 1 -- 172.21.15.32:0/1282851682 shutdown_connections 2024-04-17T09:56:59.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.696+0000 7fbdbc000640 1 -- 172.21.15.32:0/1282851682 wait complete. 2024-04-17T09:56:59.717 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T09:56:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.860+0000 7f04ac000640 1 Processor -- start 2024-04-17T09:56:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.860+0000 7f04ac000640 1 -- start start 2024-04-17T09:56:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.860+0000 7f04ac000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04a4155a70 0x7f04a415f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.860+0000 7f04ac000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f04a4155e50 0x7f04a415d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.860+0000 7f04ac000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04a415d5d0 0x7f04a415c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:59.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.860+0000 7f04ac000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f04a40b42a0 con 0x7f04a4155a70 2024-04-17T09:56:59.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.860+0000 7f04ac000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f04a40d1250 con 0x7f04a4155e50 2024-04-17T09:56:59.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.860+0000 7f04ac000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f04a4160110 con 0x7f04a415d5d0 2024-04-17T09:56:59.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.860+0000 7f04a9600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04a4155a70 0x7f04a415f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:59.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.860+0000 7f04a9600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04a4155a70 0x7f04a415f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34980/0 (socket says 172.21.15.32:34980) 2024-04-17T09:56:59.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.860+0000 7f04a9600640 1 -- 172.21.15.32:0/1684990405 learned_addr learned my addr 172.21.15.32:0/1684990405 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:59.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.860+0000 7f04aa000640 1 --2- 172.21.15.32:0/1684990405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04a415d5d0 0x7f04a415c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:59.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.860+0000 7f04a8c00640 1 --2- 172.21.15.32:0/1684990405 >> v2:172.21.15.144:3300/0 conn(0x7f04a4155e50 0x7f04a415d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:59.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.860+0000 7f04a9600640 1 -- 172.21.15.32:0/1684990405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04a415d5d0 msgr2=0x7f04a415c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:59.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.860+0000 7f04a9600640 1 --2- 172.21.15.32:0/1684990405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04a415d5d0 0x7f04a415c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.860+0000 7f04a9600640 1 -- 172.21.15.32:0/1684990405 >> v2:172.21.15.144:3300/0 conn(0x7f04a4155e50 msgr2=0x7f04a415d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:56:59.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.860+0000 7f04a9600640 1 --2- 172.21.15.32:0/1684990405 >> v2:172.21.15.144:3300/0 conn(0x7f04a4155e50 0x7f04a415d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.860+0000 7f04a9600640 1 -- 172.21.15.32:0/1684990405 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a415f870 con 0x7f04a4155a70 2024-04-17T09:56:59.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.860+0000 7f04aa000640 1 --2- 172.21.15.32:0/1684990405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04a415d5d0 0x7f04a415c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:59.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.860+0000 7f04a8c00640 1 --2- 172.21.15.32:0/1684990405 >> v2:172.21.15.144:3300/0 conn(0x7f04a4155e50 0x7f04a415d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:59.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.864+0000 7f04a9600640 1 --2- 172.21.15.32:0/1684990405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a4155a70 0x7f04a415f190 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f04a4160290 tx=0x7f049801eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f37e3cb3ff2deb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:59.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.864+0000 7f04a3e00640 1 -- 172.21.15.32:0/1684990405 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04980b4070 con 0x7f04a4155a70 2024-04-17T09:56:59.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.864+0000 7f04a3e00640 1 -- 172.21.15.32:0/1684990405 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04980a7c30 con 0x7f04a4155a70 2024-04-17T09:56:59.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.864+0000 7f04a3e00640 1 -- 172.21.15.32:0/1684990405 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04980af440 con 0x7f04a4155a70 2024-04-17T09:56:59.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.864+0000 7f04ac000640 1 -- 172.21.15.32:0/1684990405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a4155a70 msgr2=0x7f04a415f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:59.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.864+0000 7f04ac000640 1 --2- 172.21.15.32:0/1684990405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a4155a70 0x7f04a415f190 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f04a4160290 tx=0x7f049801eba0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.864+0000 7f04ac000640 1 -- 172.21.15.32:0/1684990405 shutdown_connections 2024-04-17T09:56:59.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.864+0000 7f04ac000640 1 --2- 172.21.15.32:0/1684990405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04a415d5d0 0x7f04a415c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.864+0000 7f04ac000640 1 --2- 172.21.15.32:0/1684990405 >> v2:172.21.15.144:3300/0 conn(0x7f04a4155e50 0x7f04a415d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.864+0000 7f04ac000640 1 --2- 172.21.15.32:0/1684990405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a4155a70 0x7f04a415f190 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.864+0000 7f04ac000640 1 -- 172.21.15.32:0/1684990405 >> 172.21.15.32:0/1684990405 conn(0x7f04a406ba30 msgr2=0x7f04a406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:56:59.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.864+0000 7f04ac000640 1 -- 172.21.15.32:0/1684990405 shutdown_connections 2024-04-17T09:56:59.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.864+0000 7f04ac000640 1 -- 172.21.15.32:0/1684990405 wait complete. 2024-04-17T09:56:59.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.864+0000 7f04ac000640 1 Processor -- start 2024-04-17T09:56:59.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04ac000640 1 -- start start 2024-04-17T09:56:59.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04ac000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04a4155a70 0x7f04a414b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:59.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04ac000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a4155e50 0x7f04a414c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:59.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04ac000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04a415d5d0 0x7f04a414cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:59.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04ac000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f04a40b5320 con 0x7f04a4155e50 2024-04-17T09:56:59.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04ac000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f04a40b51a0 con 0x7f04a415d5d0 2024-04-17T09:56:59.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04ac000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f04a40d0170 con 0x7f04a4155a70 2024-04-17T09:56:59.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04a8c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a4155e50 0x7f04a414c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:59.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04a8c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a4155e50 0x7f04a414c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34984/0 (socket says 172.21.15.32:34984) 2024-04-17T09:56:59.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04a8c00640 1 -- 172.21.15.32:0/2908452830 learned_addr learned my addr 172.21.15.32:0/2908452830 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:56:59.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04a9600640 1 --2- 172.21.15.32:0/2908452830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04a4155a70 0x7f04a414b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:59.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04aa000640 1 --2- 172.21.15.32:0/2908452830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04a415d5d0 0x7f04a414cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:59.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04a8c00640 1 -- 172.21.15.32:0/2908452830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04a4155a70 msgr2=0x7f04a414b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:59.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04a8c00640 1 --2- 172.21.15.32:0/2908452830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04a4155a70 0x7f04a414b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04a8c00640 1 -- 172.21.15.32:0/2908452830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04a415d5d0 msgr2=0x7f04a414cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:56:59.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04a8c00640 1 --2- 172.21.15.32:0/2908452830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04a415d5d0 0x7f04a414cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:56:59.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04a8c00640 1 -- 172.21.15.32:0/2908452830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a414d4b0 con 0x7f04a4155e50 2024-04-17T09:56:59.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04a9600640 1 --2- 172.21.15.32:0/2908452830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04a4155a70 0x7f04a414b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:56:59.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04a8c00640 1 --2- 172.21.15.32:0/2908452830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a4155e50 0x7f04a414c960 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f049000e7e0 tx=0x7f049000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:59.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04a1600640 1 -- 172.21.15.32:0/2908452830 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049000f040 con 0x7f04a4155e50 2024-04-17T09:56:59.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04a1600640 1 -- 172.21.15.32:0/2908452830 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0490004030 con 0x7f04a4155e50 2024-04-17T09:56:59.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04a1600640 1 -- 172.21.15.32:0/2908452830 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04900043e0 con 0x7f04a4155e50 2024-04-17T09:56:59.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04ac000640 1 -- 172.21.15.32:0/2908452830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a414d790 con 0x7f04a4155e50 2024-04-17T09:56:59.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04ac000640 1 -- 172.21.15.32:0/2908452830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a40afba0 con 0x7f04a4155e50 2024-04-17T09:56:59.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.868+0000 7f04ac000640 1 -- 172.21.15.32:0/2908452830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04a4068a90 con 0x7f04a4155e50 2024-04-17T09:56:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.872+0000 7f04a1600640 1 -- 172.21.15.32:0/2908452830 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f049000c6d0 con 0x7f04a4155e50 2024-04-17T09:56:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.872+0000 7f04a1600640 1 --2- 172.21.15.32:0/2908452830 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0484068380 0x7f048406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:56:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.872+0000 7f04a1600640 1 -- 172.21.15.32:0/2908452830 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f0490086690 con 0x7f04a4155e50 2024-04-17T09:56:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.872+0000 7f04a9600640 1 --2- 172.21.15.32:0/2908452830 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0484068380 0x7f048406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:56:59.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.872+0000 7f04a9600640 1 --2- 172.21.15.32:0/2908452830 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0484068380 0x7f048406a840 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f04980b1000 tx=0x7f0498008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:56:59.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:56:59.876+0000 7f04a1600640 1 -- 172.21.15.32:0/2908452830 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0490017020 con 0x7f04a4155e50 2024-04-17T09:57:00.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.076+0000 7f04ac000640 1 -- 172.21.15.32:0/2908452830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f04a406e100 con 0x7f04a4155e50 2024-04-17T09:57:00.079 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-17T09:57:00.076+0000 7f93c9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T09:57:00.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.080+0000 7f04a1600640 1 -- 172.21.15.32:0/2908452830 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) ==== 161+0+0 (secure 0 0 0) 0x7f0490056c40 con 0x7f04a4155e50 2024-04-17T09:57:00.081 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T09:57:00.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.084+0000 7f04ac000640 1 -- 172.21.15.32:0/2908452830 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0484068380 msgr2=0x7f048406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:00.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.084+0000 7f04ac000640 1 --2- 172.21.15.32:0/2908452830 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0484068380 0x7f048406a840 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f04980b1000 tx=0x7f0498008040 comp rx=0 tx=0).stop 2024-04-17T09:57:00.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.084+0000 7f04ac000640 1 -- 172.21.15.32:0/2908452830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a4155e50 msgr2=0x7f04a414c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:00.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.084+0000 7f04ac000640 1 --2- 172.21.15.32:0/2908452830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a4155e50 0x7f04a414c960 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f049000e7e0 tx=0x7f049000eca0 comp rx=0 tx=0).stop 2024-04-17T09:57:00.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.084+0000 7f04ac000640 1 -- 172.21.15.32:0/2908452830 shutdown_connections 2024-04-17T09:57:00.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.084+0000 7f04ac000640 1 --2- 172.21.15.32:0/2908452830 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0484068380 0x7f048406a840 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:00.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.084+0000 7f04ac000640 1 --2- 172.21.15.32:0/2908452830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04a415d5d0 0x7f04a414cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:00.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.084+0000 7f04ac000640 1 --2- 172.21.15.32:0/2908452830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04a4155e50 0x7f04a414c960 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:00.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.084+0000 7f04ac000640 1 --2- 172.21.15.32:0/2908452830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04a4155a70 0x7f04a414b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:00.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.084+0000 7f04ac000640 1 -- 172.21.15.32:0/2908452830 >> 172.21.15.32:0/2908452830 conn(0x7f04a406ba30 msgr2=0x7f04a40d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:00.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.084+0000 7f04ac000640 1 -- 172.21.15.32:0/2908452830 shutdown_connections 2024-04-17T09:57:00.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.084+0000 7f04ac000640 1 -- 172.21.15.32:0/2908452830 wait complete. 2024-04-17T09:57:00.102 INFO:tasks.ceph.mds.a.smithi032.stderr: -5648> 2024-04-17T09:56:04.172+0000 7f7c95000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T09:57:00.105 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-17T09:57:00.106 INFO:tasks.ceph.mds.a.smithi032.stderr: -3711> 2024-04-17T09:56:15.584+0000 7f7c90000640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6396=0-4194304@1} caps={6396=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6396 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x5570aa0c0b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-17T09:57:00.110 INFO:tasks.ceph.mds.a.smithi032.stderr: -1776> 2024-04-17T09:56:47.200+0000 7f7c90000640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-04-17T09:56:14.409449+0000 1=1+0) caps={6396=pAsLsXsFs/-@2},l=6396(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x5570aa0c0b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-17T09:57:00.156 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-17T09:57:00.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.248+0000 7f6a33800640 1 Processor -- start 2024-04-17T09:57:00.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.248+0000 7f6a33800640 1 -- start start 2024-04-17T09:57:00.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.252+0000 7f6a33800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2c06f7c0 0x7f6a2c06ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:00.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.252+0000 7f6a33800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6a2c06f270 0x7f6a2c06bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:00.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.252+0000 7f6a33800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a2c0cc940 con 0x7f6a2c15b7d0 2024-04-17T09:57:00.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.252+0000 7f6a33800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f6a2c0ccc20 con 0x7f6a2c06f270 2024-04-17T09:57:00.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.252+0000 7f6a33800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6a2c06e0b0 con 0x7f6a2c06f7c0 2024-04-17T09:57:00.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.252+0000 7f6a30e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6a2c15b7d0 0x7f6a2c06e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44304/0 (socket says 172.21.15.32:44304) 2024-04-17T09:57:00.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.252+0000 7f6a30e00640 1 -- 172.21.15.32:0/3890576027 learned_addr learned my addr 172.21.15.32:0/3890576027 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:00.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.252+0000 7f6a31800640 1 --2- 172.21.15.32:0/3890576027 >> v2:172.21.15.144:3300/0 conn(0x7f6a2c06f270 0x7f6a2c06bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:00.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.252+0000 7f6a2be00640 1 --2- 172.21.15.32:0/3890576027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2c06f7c0 0x7f6a2c06ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:00.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.252+0000 7f6a2b400640 1 -- 172.21.15.32:0/3890576027 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3990331445 0 0) 0x7f6a2c0cc940 con 0x7f6a2c15b7d0 2024-04-17T09:57:00.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.252+0000 7f6a2b400640 1 -- 172.21.15.32:0/3890576027 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a14005740 con 0x7f6a2c15b7d0 2024-04-17T09:57:00.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.252+0000 7f6a31800640 1 -- 172.21.15.32:0/3890576027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2c06f7c0 msgr2=0x7f6a2c06ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:00.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.252+0000 7f6a31800640 1 --2- 172.21.15.32:0/3890576027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2c06f7c0 0x7f6a2c06ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:00.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.252+0000 7f6a31800640 1 -- 172.21.15.32:0/3890576027 >> v1:172.21.15.32:6789/0 conn(0x7f6a2c15b7d0 legacy=0x7f6a2c06e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:00.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.252+0000 7f6a31800640 1 -- 172.21.15.32:0/3890576027 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2c06e390 con 0x7f6a2c06f270 2024-04-17T09:57:00.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.252+0000 7f6a2be00640 1 --2- 172.21.15.32:0/3890576027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2c06f7c0 0x7f6a2c06ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:57:00.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.256+0000 7f6a31800640 1 --2- 172.21.15.32:0/3890576027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a2c06f270 0x7f6a2c06bf70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6a24009d90 tx=0x7f6a2401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47a5f94a248c3d7b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:00.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.256+0000 7f6a2b400640 1 -- 172.21.15.32:0/3890576027 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a240b2070 con 0x7f6a2c06f270 2024-04-17T09:57:00.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.256+0000 7f6a2b400640 1 -- 172.21.15.32:0/3890576027 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a240a58b0 con 0x7f6a2c06f270 2024-04-17T09:57:00.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.256+0000 7f6a2b400640 1 -- 172.21.15.32:0/3890576027 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a240a5c80 con 0x7f6a2c06f270 2024-04-17T09:57:00.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.256+0000 7f6a33800640 1 -- 172.21.15.32:0/3890576027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a2c06f270 msgr2=0x7f6a2c06bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:00.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.256+0000 7f6a33800640 1 --2- 172.21.15.32:0/3890576027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a2c06f270 0x7f6a2c06bf70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6a24009d90 tx=0x7f6a2401c8e0 comp rx=0 tx=0).stop 2024-04-17T09:57:00.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.256+0000 7f6a33800640 1 -- 172.21.15.32:0/3890576027 shutdown_connections 2024-04-17T09:57:00.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.256+0000 7f6a33800640 1 --2- 172.21.15.32:0/3890576027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a2c06f270 0x7f6a2c06bf70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:00.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.256+0000 7f6a33800640 1 --2- 172.21.15.32:0/3890576027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2c06f7c0 0x7f6a2c06ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:00.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.256+0000 7f6a33800640 1 -- 172.21.15.32:0/3890576027 >> 172.21.15.32:0/3890576027 conn(0x7f6a2c0b2120 msgr2=0x7f6a2c0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:00.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.256+0000 7f6a33800640 1 -- 172.21.15.32:0/3890576027 shutdown_connections 2024-04-17T09:57:00.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.256+0000 7f6a33800640 1 -- 172.21.15.32:0/3890576027 wait complete. 2024-04-17T09:57:00.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.256+0000 7f6a33800640 1 Processor -- start 2024-04-17T09:57:00.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.256+0000 7f6a33800640 1 -- start start 2024-04-17T09:57:00.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.256+0000 7f6a33800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a2c06f270 0x7f6a2c14db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:00.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.256+0000 7f6a33800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a2c06f7c0 0x7f6a2c13a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:00.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.256+0000 7f6a33800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2c15b7d0 0x7f6a2c0b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:00.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.256+0000 7f6a33800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c0cee70 con 0x7f6a2c06f7c0 2024-04-17T09:57:00.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.256+0000 7f6a33800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c0cecf0 con 0x7f6a2c06f270 2024-04-17T09:57:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.256+0000 7f6a33800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6a2c071e60 con 0x7f6a2c15b7d0 2024-04-17T09:57:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.260+0000 7f6a2be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a2c06f7c0 0x7f6a2c13a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.260+0000 7f6a2be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a2c06f7c0 0x7f6a2c13a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34996/0 (socket says 172.21.15.32:34996) 2024-04-17T09:57:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.260+0000 7f6a31800640 1 --2- 172.21.15.32:0/2301087699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2c15b7d0 0x7f6a2c0b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.260+0000 7f6a30e00640 1 --2- 172.21.15.32:0/2301087699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a2c06f270 0x7f6a2c14db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.260+0000 7f6a2be00640 1 -- 172.21.15.32:0/2301087699 learned_addr learned my addr 172.21.15.32:0/2301087699 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.260+0000 7f6a31800640 1 -- 172.21.15.32:0/2301087699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a2c06f270 msgr2=0x7f6a2c14db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.260+0000 7f6a31800640 1 --2- 172.21.15.32:0/2301087699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a2c06f270 0x7f6a2c14db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.260+0000 7f6a31800640 1 -- 172.21.15.32:0/2301087699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a2c06f7c0 msgr2=0x7f6a2c13a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.260+0000 7f6a31800640 1 --2- 172.21.15.32:0/2301087699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a2c06f7c0 0x7f6a2c13a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:00.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.260+0000 7f6a31800640 1 -- 172.21.15.32:0/2301087699 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2c13cf00 con 0x7f6a2c15b7d0 2024-04-17T09:57:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.260+0000 7f6a31800640 1 --2- 172.21.15.32:0/2301087699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2c15b7d0 0x7f6a2c0b0110 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6a240ac000 tx=0x7f6a24019680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.260+0000 7f6a28c00640 1 -- 172.21.15.32:0/2301087699 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a240bd020 con 0x7f6a2c15b7d0 2024-04-17T09:57:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.260+0000 7f6a28c00640 1 -- 172.21.15.32:0/2301087699 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a240b3040 con 0x7f6a2c15b7d0 2024-04-17T09:57:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.260+0000 7f6a28c00640 1 -- 172.21.15.32:0/2301087699 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a240b2050 con 0x7f6a2c15b7d0 2024-04-17T09:57:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.260+0000 7f6a33800640 1 -- 172.21.15.32:0/2301087699 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2c13d190 con 0x7f6a2c15b7d0 2024-04-17T09:57:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.260+0000 7f6a33800640 1 -- 172.21.15.32:0/2301087699 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a2c13d750 con 0x7f6a2c15b7d0 2024-04-17T09:57:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.260+0000 7f6a33800640 1 -- 172.21.15.32:0/2301087699 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a2c068a90 con 0x7f6a2c15b7d0 2024-04-17T09:57:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.260+0000 7f6a28c00640 1 -- 172.21.15.32:0/2301087699 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6a240c0070 con 0x7f6a2c15b7d0 2024-04-17T09:57:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.264+0000 7f6a28c00640 1 --2- 172.21.15.32:0/2301087699 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6a0c068380 0x7f6a0c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.264+0000 7f6a28c00640 1 -- 172.21.15.32:0/2301087699 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(58..58 src has 1..58) ==== 5795+0+0 (secure 0 0 0) 0x7f6a24120ca0 con 0x7f6a2c15b7d0 2024-04-17T09:57:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.264+0000 7f6a30e00640 1 --2- 172.21.15.32:0/2301087699 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6a0c068380 0x7f6a0c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:00.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.264+0000 7f6a30e00640 1 --2- 172.21.15.32:0/2301087699 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6a0c068380 0x7f6a0c06a840 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6a20007fd0 tx=0x7f6a20009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:00.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.268+0000 7f6a28c00640 1 -- 172.21.15.32:0/2301087699 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f6a24016560 con 0x7f6a2c15b7d0 2024-04-17T09:57:00.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:00.468+0000 7f6a33800640 1 -- 172.21.15.32:0/2301087699 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6a2c158880 con 0x7f6a2c15b7d0 2024-04-17T09:57:01.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.084+0000 7f6a28c00640 1 -- 172.21.15.32:0/2301087699 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) ==== 104+0+0 (secure 0 0 0) 0x7f6a240f0ba0 con 0x7f6a2c15b7d0 2024-04-17T09:57:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.088+0000 7f6a33800640 1 -- 172.21.15.32:0/2301087699 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6a0c068380 msgr2=0x7f6a0c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.088+0000 7f6a33800640 1 --2- 172.21.15.32:0/2301087699 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6a0c068380 0x7f6a0c06a840 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6a20007fd0 tx=0x7f6a20009040 comp rx=0 tx=0).stop 2024-04-17T09:57:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.088+0000 7f6a33800640 1 -- 172.21.15.32:0/2301087699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2c15b7d0 msgr2=0x7f6a2c0b0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.088+0000 7f6a33800640 1 --2- 172.21.15.32:0/2301087699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2c15b7d0 0x7f6a2c0b0110 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6a240ac000 tx=0x7f6a24019680 comp rx=0 tx=0).stop 2024-04-17T09:57:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.088+0000 7f6a33800640 1 -- 172.21.15.32:0/2301087699 shutdown_connections 2024-04-17T09:57:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.088+0000 7f6a33800640 1 --2- 172.21.15.32:0/2301087699 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6a0c068380 0x7f6a0c06a840 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.088+0000 7f6a33800640 1 --2- 172.21.15.32:0/2301087699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a2c15b7d0 0x7f6a2c0b0110 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.088+0000 7f6a33800640 1 --2- 172.21.15.32:0/2301087699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a2c06f7c0 0x7f6a2c13a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.088+0000 7f6a33800640 1 --2- 172.21.15.32:0/2301087699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a2c06f270 0x7f6a2c14db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.088+0000 7f6a33800640 1 -- 172.21.15.32:0/2301087699 >> 172.21.15.32:0/2301087699 conn(0x7f6a2c0b2120 msgr2=0x7f6a2c1594f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.088+0000 7f6a33800640 1 -- 172.21.15.32:0/2301087699 shutdown_connections 2024-04-17T09:57:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.088+0000 7f6a33800640 1 -- 172.21.15.32:0/2301087699 wait complete. 2024-04-17T09:57:01.111 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-17T09:57:01.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.304+0000 7fba23200640 1 Processor -- start 2024-04-17T09:57:01.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.304+0000 7fba23200640 1 -- start start 2024-04-17T09:57:01.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.304+0000 7fba23200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba2409a9f0 0x7fba2409b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:01.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.304+0000 7fba23200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fba240b4970 0x7fba240cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:01.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.304+0000 7fba23200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba240d1340 con 0x7fba2415ea00 2024-04-17T09:57:01.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.304+0000 7fba23200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fba240cc9e0 con 0x7fba240b4970 2024-04-17T09:57:01.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.304+0000 7fba23200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fba240cccc0 con 0x7fba2409a9f0 2024-04-17T09:57:01.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.304+0000 7fba1be00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fba2415ea00 0x7fba2409add0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55160/0 (socket says 172.21.15.32:55160) 2024-04-17T09:57:01.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.304+0000 7fba1be00640 1 -- 172.21.15.32:0/3324852471 learned_addr learned my addr 172.21.15.32:0/3324852471 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:01.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.304+0000 7fba22800640 1 --2- 172.21.15.32:0/3324852471 >> v2:172.21.15.144:3300/0 conn(0x7fba240b4970 0x7fba240cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:01.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.304+0000 7fba21e00640 1 --2- 172.21.15.32:0/3324852471 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba2409a9f0 0x7fba2409b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:01.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.304+0000 7fba21400640 1 -- 172.21.15.32:0/3324852471 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3710305759 0 0) 0x7fba240d1340 con 0x7fba2415ea00 2024-04-17T09:57:01.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.304+0000 7fba21400640 1 -- 172.21.15.32:0/3324852471 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba08003530 con 0x7fba2415ea00 2024-04-17T09:57:01.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.304+0000 7fba22800640 1 -- 172.21.15.32:0/3324852471 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba2409a9f0 msgr2=0x7fba2409b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:01.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.304+0000 7fba22800640 1 --2- 172.21.15.32:0/3324852471 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba2409a9f0 0x7fba2409b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:01.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.304+0000 7fba22800640 1 -- 172.21.15.32:0/3324852471 >> v1:172.21.15.32:6789/0 conn(0x7fba2415ea00 legacy=0x7fba2409add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:01.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.304+0000 7fba22800640 1 -- 172.21.15.32:0/3324852471 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba240d1620 con 0x7fba240b4970 2024-04-17T09:57:01.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.304+0000 7fba21e00640 1 --2- 172.21.15.32:0/3324852471 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba2409a9f0 0x7fba2409b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:57:01.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.308+0000 7fba22800640 1 --2- 172.21.15.32:0/3324852471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba240b4970 0x7fba240cd150 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009d90 tx=0x7fba1c01c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=185cd9558e0deddb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:01.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.308+0000 7fba21400640 1 -- 172.21.15.32:0/3324852471 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0b2070 con 0x7fba240b4970 2024-04-17T09:57:01.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.308+0000 7fba21400640 1 -- 172.21.15.32:0/3324852471 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba1c0a58b0 con 0x7fba240b4970 2024-04-17T09:57:01.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.308+0000 7fba21400640 1 -- 172.21.15.32:0/3324852471 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0a5c80 con 0x7fba240b4970 2024-04-17T09:57:01.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.308+0000 7fba23200640 1 -- 172.21.15.32:0/3324852471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba240b4970 msgr2=0x7fba240cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:01.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.308+0000 7fba23200640 1 --2- 172.21.15.32:0/3324852471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba240b4970 0x7fba240cd150 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009d90 tx=0x7fba1c01c8e0 comp rx=0 tx=0).stop 2024-04-17T09:57:01.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.308+0000 7fba23200640 1 -- 172.21.15.32:0/3324852471 shutdown_connections 2024-04-17T09:57:01.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.308+0000 7fba23200640 1 --2- 172.21.15.32:0/3324852471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba240b4970 0x7fba240cd150 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:01.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.308+0000 7fba23200640 1 --2- 172.21.15.32:0/3324852471 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba2409a9f0 0x7fba2409b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:01.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.308+0000 7fba23200640 1 -- 172.21.15.32:0/3324852471 >> 172.21.15.32:0/3324852471 conn(0x7fba240b3e80 msgr2=0x7fba240b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:01.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.308+0000 7fba23200640 1 -- 172.21.15.32:0/3324852471 shutdown_connections 2024-04-17T09:57:01.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.308+0000 7fba23200640 1 -- 172.21.15.32:0/3324852471 wait complete. 2024-04-17T09:57:01.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.308+0000 7fba23200640 1 Processor -- start 2024-04-17T09:57:01.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba23200640 1 -- start start 2024-04-17T09:57:01.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba23200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba2409a9f0 0x7fba241613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:01.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba23200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba240b4970 0x7fba2415eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:01.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba23200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba2415ea00 0x7fba240afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:01.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba23200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fba240ce230 con 0x7fba2409a9f0 2024-04-17T09:57:01.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba23200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fba240ce0b0 con 0x7fba240b4970 2024-04-17T09:57:01.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba23200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fba240ce3b0 con 0x7fba2415ea00 2024-04-17T09:57:01.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba21e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba2409a9f0 0x7fba241613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:01.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba21e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba2409a9f0 0x7fba241613c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39140/0 (socket says 172.21.15.32:39140) 2024-04-17T09:57:01.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba21e00640 1 -- 172.21.15.32:0/2816790127 learned_addr learned my addr 172.21.15.32:0/2816790127 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:01.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba1be00640 1 --2- 172.21.15.32:0/2816790127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba240b4970 0x7fba2415eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:01.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba22800640 1 --2- 172.21.15.32:0/2816790127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba2415ea00 0x7fba240afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:01.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba21e00640 1 -- 172.21.15.32:0/2816790127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba2415ea00 msgr2=0x7fba240afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:01.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba21e00640 1 --2- 172.21.15.32:0/2816790127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba2415ea00 0x7fba240afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:01.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba21e00640 1 -- 172.21.15.32:0/2816790127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba240b4970 msgr2=0x7fba2415eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:01.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba21e00640 1 --2- 172.21.15.32:0/2816790127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba240b4970 0x7fba2415eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:01.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba21e00640 1 -- 172.21.15.32:0/2816790127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2415f420 con 0x7fba2409a9f0 2024-04-17T09:57:01.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba1be00640 1 --2- 172.21.15.32:0/2816790127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba240b4970 0x7fba2415eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:57:01.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba21e00640 1 --2- 172.21.15.32:0/2816790127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba2409a9f0 0x7fba241613c0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fba1400e9b0 tx=0x7fba1400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:01.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba1a000640 1 -- 172.21.15.32:0/2816790127 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba14017070 con 0x7fba2409a9f0 2024-04-17T09:57:01.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba1a000640 1 -- 172.21.15.32:0/2816790127 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba1400ca00 con 0x7fba2409a9f0 2024-04-17T09:57:01.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba1a000640 1 -- 172.21.15.32:0/2816790127 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1400cdd0 con 0x7fba2409a9f0 2024-04-17T09:57:01.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba23200640 1 -- 172.21.15.32:0/2816790127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba24161a60 con 0x7fba2409a9f0 2024-04-17T09:57:01.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba23200640 1 -- 172.21.15.32:0/2816790127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba240af4b0 con 0x7fba2409a9f0 2024-04-17T09:57:01.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.312+0000 7fba23200640 1 -- 172.21.15.32:0/2816790127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba2404fd00 con 0x7fba2409a9f0 2024-04-17T09:57:01.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.316+0000 7fba1a000640 1 -- 172.21.15.32:0/2816790127 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fba1401a070 con 0x7fba2409a9f0 2024-04-17T09:57:01.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.316+0000 7fba1a000640 1 --2- 172.21.15.32:0/2816790127 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fba00068380 0x7fba0006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:01.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.316+0000 7fba1a000640 1 -- 172.21.15.32:0/2816790127 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5795+0+0 (secure 0 0 0) 0x7fba14086d20 con 0x7fba2409a9f0 2024-04-17T09:57:01.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.316+0000 7fba1be00640 1 --2- 172.21.15.32:0/2816790127 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fba00068380 0x7fba0006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:01.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.316+0000 7fba1be00640 1 --2- 172.21.15.32:0/2816790127 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fba00068380 0x7fba0006a840 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fba240b8050 tx=0x7fba0c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:01.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.320+0000 7fba1a000640 1 -- 172.21.15.32:0/2816790127 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fba140571e0 con 0x7fba2409a9f0 2024-04-17T09:57:01.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:01.484+0000 7fba23200640 1 -- 172.21.15.32:0/2816790127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fba2415b2c0 con 0x7fba2409a9f0 2024-04-17T09:57:02.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.088+0000 7fba1a000640 1 -- 172.21.15.32:0/2816790127 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v59) ==== 181+0+0 (secure 0 0 0) 0x7fba14056ae0 con 0x7fba2409a9f0 2024-04-17T09:57:02.089 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-04-17T09:57:02.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.088+0000 7fba23200640 1 -- 172.21.15.32:0/2816790127 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fba00068380 msgr2=0x7fba0006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:02.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.088+0000 7fba23200640 1 --2- 172.21.15.32:0/2816790127 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fba00068380 0x7fba0006a840 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fba240b8050 tx=0x7fba0c009040 comp rx=0 tx=0).stop 2024-04-17T09:57:02.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.088+0000 7fba23200640 1 -- 172.21.15.32:0/2816790127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba2409a9f0 msgr2=0x7fba241613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:02.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.088+0000 7fba23200640 1 --2- 172.21.15.32:0/2816790127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba2409a9f0 0x7fba241613c0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fba1400e9b0 tx=0x7fba1400ee70 comp rx=0 tx=0).stop 2024-04-17T09:57:02.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.092+0000 7fba23200640 1 -- 172.21.15.32:0/2816790127 shutdown_connections 2024-04-17T09:57:02.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.092+0000 7fba23200640 1 --2- 172.21.15.32:0/2816790127 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fba00068380 0x7fba0006a840 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:02.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.092+0000 7fba23200640 1 --2- 172.21.15.32:0/2816790127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba2415ea00 0x7fba240afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:02.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.092+0000 7fba23200640 1 --2- 172.21.15.32:0/2816790127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba240b4970 0x7fba2415eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:02.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.092+0000 7fba23200640 1 --2- 172.21.15.32:0/2816790127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba2409a9f0 0x7fba241613c0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:02.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.092+0000 7fba23200640 1 -- 172.21.15.32:0/2816790127 >> 172.21.15.32:0/2816790127 conn(0x7fba240b3e80 msgr2=0x7fba24070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:02.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.092+0000 7fba23200640 1 -- 172.21.15.32:0/2816790127 shutdown_connections 2024-04-17T09:57:02.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.092+0000 7fba23200640 1 -- 172.21.15.32:0/2816790127 wait complete. 2024-04-17T09:57:02.115 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-17T09:57:02.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.268+0000 7fcc51e00640 1 Processor -- start 2024-04-17T09:57:02.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.268+0000 7fcc51e00640 1 -- start start 2024-04-17T09:57:02.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.268+0000 7fcc51e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcc4c0b4e60 0x7fcc4c06e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:02.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.268+0000 7fcc51e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c0d0600 0x7fcc4c0d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:02.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.268+0000 7fcc51e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc4c163ea0 con 0x7fcc4c163190 2024-04-17T09:57:02.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.268+0000 7fcc51e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fcc4c164180 con 0x7fcc4c0b4e60 2024-04-17T09:57:02.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.268+0000 7fcc51e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcc4c06dd10 con 0x7fcc4c0d0600 2024-04-17T09:57:02.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.268+0000 7fcc4be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c0d0600 0x7fcc4c0d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:02.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.268+0000 7fcc50a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcc4c0b4e60 0x7fcc4c06e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:02.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.268+0000 7fcc51400640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fcc4c163190 0x7fcc4c163620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:49958/0 (socket says 172.21.15.32:49958) 2024-04-17T09:57:02.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.268+0000 7fcc51400640 1 -- 172.21.15.32:0/572947399 learned_addr learned my addr 172.21.15.32:0/572947399 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:02.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.272+0000 7fcc4b400640 1 -- 172.21.15.32:0/572947399 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2464760620 0 0) 0x7fcc4c163ea0 con 0x7fcc4c163190 2024-04-17T09:57:02.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.272+0000 7fcc4b400640 1 -- 172.21.15.32:0/572947399 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc34003530 con 0x7fcc4c163190 2024-04-17T09:57:02.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.272+0000 7fcc50a00640 1 -- 172.21.15.32:0/572947399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c0d0600 msgr2=0x7fcc4c0d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:02.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.272+0000 7fcc50a00640 1 --2- 172.21.15.32:0/572947399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c0d0600 0x7fcc4c0d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:02.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.272+0000 7fcc50a00640 1 -- 172.21.15.32:0/572947399 >> v1:172.21.15.144:6789/0 conn(0x7fcc4c163190 legacy=0x7fcc4c163620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:02.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.272+0000 7fcc50a00640 1 -- 172.21.15.32:0/572947399 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4c06dff0 con 0x7fcc4c0b4e60 2024-04-17T09:57:02.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.272+0000 7fcc50a00640 1 --2- 172.21.15.32:0/572947399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc4c0b4e60 0x7fcc4c06e7f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcc400097c0 tx=0x7fcc4001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12dc314ea0e1c130 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:02.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.272+0000 7fcc4b400640 1 -- 172.21.15.32:0/572947399 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc400b0070 con 0x7fcc4c0b4e60 2024-04-17T09:57:02.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.272+0000 7fcc4b400640 1 -- 172.21.15.32:0/572947399 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc400ab600 con 0x7fcc4c0b4e60 2024-04-17T09:57:02.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.272+0000 7fcc4b400640 1 -- 172.21.15.32:0/572947399 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc400ab9d0 con 0x7fcc4c0b4e60 2024-04-17T09:57:02.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.276+0000 7fcc51e00640 1 -- 172.21.15.32:0/572947399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc4c0b4e60 msgr2=0x7fcc4c06e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:02.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.276+0000 7fcc51e00640 1 --2- 172.21.15.32:0/572947399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc4c0b4e60 0x7fcc4c06e7f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcc400097c0 tx=0x7fcc4001c410 comp rx=0 tx=0).stop 2024-04-17T09:57:02.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.276+0000 7fcc51e00640 1 -- 172.21.15.32:0/572947399 shutdown_connections 2024-04-17T09:57:02.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.276+0000 7fcc51e00640 1 --2- 172.21.15.32:0/572947399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c0d0600 0x7fcc4c0d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:02.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.276+0000 7fcc51e00640 1 --2- 172.21.15.32:0/572947399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc4c0b4e60 0x7fcc4c06e7f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:02.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.276+0000 7fcc51e00640 1 -- 172.21.15.32:0/572947399 >> 172.21.15.32:0/572947399 conn(0x7fcc4c0b2170 msgr2=0x7fcc4c0b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:02.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.276+0000 7fcc51e00640 1 -- 172.21.15.32:0/572947399 shutdown_connections 2024-04-17T09:57:02.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.276+0000 7fcc51e00640 1 -- 172.21.15.32:0/572947399 wait complete. 2024-04-17T09:57:02.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.276+0000 7fcc51e00640 1 Processor -- start 2024-04-17T09:57:02.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.276+0000 7fcc51e00640 1 -- start start 2024-04-17T09:57:02.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.276+0000 7fcc51e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c0b4e60 0x7fcc4c154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:02.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.276+0000 7fcc51e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc4c0d0600 0x7fcc4c0cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:02.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.276+0000 7fcc51e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc4c163190 0x7fcc4c0cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:02.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.276+0000 7fcc51e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c160a50 con 0x7fcc4c0d0600 2024-04-17T09:57:02.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.276+0000 7fcc51e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c1608d0 con 0x7fcc4c163190 2024-04-17T09:57:02.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.276+0000 7fcc51e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcc4c160bd0 con 0x7fcc4c0b4e60 2024-04-17T09:57:02.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.276+0000 7fcc4be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc4c0d0600 0x7fcc4c0cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:02.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.276+0000 7fcc4be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc4c0d0600 0x7fcc4c0cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39146/0 (socket says 172.21.15.32:39146) 2024-04-17T09:57:02.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.276+0000 7fcc4be00640 1 -- 172.21.15.32:0/2671844723 learned_addr learned my addr 172.21.15.32:0/2671844723 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:02.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.276+0000 7fcc51400640 1 --2- 172.21.15.32:0/2671844723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc4c163190 0x7fcc4c0cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:02.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.276+0000 7fcc50a00640 1 --2- 172.21.15.32:0/2671844723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c0b4e60 0x7fcc4c154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:02.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.280+0000 7fcc4be00640 1 -- 172.21.15.32:0/2671844723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c0b4e60 msgr2=0x7fcc4c154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:02.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.280+0000 7fcc4be00640 1 --2- 172.21.15.32:0/2671844723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c0b4e60 0x7fcc4c154e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:02.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.280+0000 7fcc4be00640 1 -- 172.21.15.32:0/2671844723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc4c163190 msgr2=0x7fcc4c0cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:02.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.280+0000 7fcc4be00640 1 --2- 172.21.15.32:0/2671844723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc4c163190 0x7fcc4c0cd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:02.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.280+0000 7fcc4be00640 1 -- 172.21.15.32:0/2671844723 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4c1566f0 con 0x7fcc4c0d0600 2024-04-17T09:57:02.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.280+0000 7fcc4be00640 1 --2- 172.21.15.32:0/2671844723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc4c0d0600 0x7fcc4c0cd290 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fcc3800e870 tx=0x7fcc3800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:02.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.280+0000 7fcc48c00640 1 -- 172.21.15.32:0/2671844723 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc38017070 con 0x7fcc4c0d0600 2024-04-17T09:57:02.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.280+0000 7fcc48c00640 1 -- 172.21.15.32:0/2671844723 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc3800c940 con 0x7fcc4c0d0600 2024-04-17T09:57:02.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.280+0000 7fcc48c00640 1 -- 172.21.15.32:0/2671844723 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3800ccf0 con 0x7fcc4c0d0600 2024-04-17T09:57:02.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.280+0000 7fcc51e00640 1 -- 172.21.15.32:0/2671844723 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc4c156990 con 0x7fcc4c0d0600 2024-04-17T09:57:02.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.280+0000 7fcc51e00640 1 -- 172.21.15.32:0/2671844723 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc4c156f50 con 0x7fcc4c0d0600 2024-04-17T09:57:02.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.280+0000 7fcc51e00640 1 -- 172.21.15.32:0/2671844723 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc4c04fd00 con 0x7fcc4c0d0600 2024-04-17T09:57:02.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.280+0000 7fcc48c00640 1 -- 172.21.15.32:0/2671844723 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcc3801a070 con 0x7fcc4c0d0600 2024-04-17T09:57:02.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.280+0000 7fcc48c00640 1 --2- 172.21.15.32:0/2671844723 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc2c068380 0x7fcc2c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:02.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.284+0000 7fcc48c00640 1 -- 172.21.15.32:0/2671844723 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5357+0+0 (secure 0 0 0) 0x7fcc380865f0 con 0x7fcc4c0d0600 2024-04-17T09:57:02.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.284+0000 7fcc50a00640 1 --2- 172.21.15.32:0/2671844723 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc2c068380 0x7fcc2c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:02.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.284+0000 7fcc50a00640 1 --2- 172.21.15.32:0/2671844723 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc2c068380 0x7fcc2c06a840 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcc400ad000 tx=0x7fcc400a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:02.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.288+0000 7fcc48c00640 1 -- 172.21.15.32:0/2671844723 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fcc38056750 con 0x7fcc4c0d0600 2024-04-17T09:57:02.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:02.452+0000 7fcc51e00640 1 -- 172.21.15.32:0/2671844723 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcc4c163af0 con 0x7fcc4c0d0600 2024-04-17T09:57:03.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.092+0000 7fcc48c00640 1 -- 172.21.15.32:0/2671844723 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v60) ==== 169+0+0 (secure 0 0 0) 0x7fcc3805aa50 con 0x7fcc4c0d0600 2024-04-17T09:57:03.091 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-04-17T09:57:03.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.092+0000 7fcc51e00640 1 -- 172.21.15.32:0/2671844723 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc2c068380 msgr2=0x7fcc2c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:03.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.092+0000 7fcc51e00640 1 --2- 172.21.15.32:0/2671844723 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc2c068380 0x7fcc2c06a840 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcc400ad000 tx=0x7fcc400a6000 comp rx=0 tx=0).stop 2024-04-17T09:57:03.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.092+0000 7fcc51e00640 1 -- 172.21.15.32:0/2671844723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc4c0d0600 msgr2=0x7fcc4c0cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:03.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.092+0000 7fcc51e00640 1 --2- 172.21.15.32:0/2671844723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc4c0d0600 0x7fcc4c0cd290 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fcc3800e870 tx=0x7fcc3800ec70 comp rx=0 tx=0).stop 2024-04-17T09:57:03.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.092+0000 7fcc51e00640 1 -- 172.21.15.32:0/2671844723 shutdown_connections 2024-04-17T09:57:03.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.092+0000 7fcc51e00640 1 --2- 172.21.15.32:0/2671844723 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc2c068380 0x7fcc2c06a840 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:03.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.092+0000 7fcc51e00640 1 --2- 172.21.15.32:0/2671844723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc4c163190 0x7fcc4c0cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:03.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.092+0000 7fcc51e00640 1 --2- 172.21.15.32:0/2671844723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc4c0d0600 0x7fcc4c0cd290 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:03.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.092+0000 7fcc51e00640 1 --2- 172.21.15.32:0/2671844723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c0b4e60 0x7fcc4c154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:03.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.092+0000 7fcc51e00640 1 -- 172.21.15.32:0/2671844723 >> 172.21.15.32:0/2671844723 conn(0x7fcc4c0b2170 msgr2=0x7fcc4c160e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:03.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.096+0000 7fcc51e00640 1 -- 172.21.15.32:0/2671844723 shutdown_connections 2024-04-17T09:57:03.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.096+0000 7fcc51e00640 1 -- 172.21.15.32:0/2671844723 wait complete. 2024-04-17T09:57:03.121 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-17T09:57:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.268+0000 7f8979a00640 1 Processor -- start 2024-04-17T09:57:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.268+0000 7f8979a00640 1 -- start start 2024-04-17T09:57:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.268+0000 7f8979a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89740b3cd0 0x7f89740c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.268+0000 7f8979a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89740c5b30 0x7f89740d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.268+0000 7f8979a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f897409b650 con 0x7f89740b2c10 2024-04-17T09:57:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.268+0000 7f8979a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f897409b400 con 0x7f89740b3cd0 2024-04-17T09:57:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.268+0000 7f8979a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f897406d250 con 0x7f89740c5b30 2024-04-17T09:57:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.268+0000 7f896be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89740b3cd0 0x7f89740c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.268+0000 7f8972a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f89740b2c10 0x7f897406cb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55176/0 (socket says 172.21.15.32:55176) 2024-04-17T09:57:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.268+0000 7f8972a00640 1 -- 172.21.15.32:0/917633886 learned_addr learned my addr 172.21.15.32:0/917633886 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.268+0000 7f8973400640 1 --2- 172.21.15.32:0/917633886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89740c5b30 0x7f89740d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.268+0000 7f8972000640 1 -- 172.21.15.32:0/917633886 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2406030783 0 0) 0x7f897409b650 con 0x7f89740b2c10 2024-04-17T09:57:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.268+0000 7f8972000640 1 -- 172.21.15.32:0/917633886 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8958003530 con 0x7f89740b2c10 2024-04-17T09:57:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.268+0000 7f896be00640 1 -- 172.21.15.32:0/917633886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89740c5b30 msgr2=0x7f89740d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.268+0000 7f896be00640 1 --2- 172.21.15.32:0/917633886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89740c5b30 0x7f89740d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.268+0000 7f896be00640 1 -- 172.21.15.32:0/917633886 >> v1:172.21.15.32:6789/0 conn(0x7f89740b2c10 legacy=0x7f897406cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.268+0000 7f896be00640 1 -- 172.21.15.32:0/917633886 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897409b930 con 0x7f89740b3cd0 2024-04-17T09:57:03.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.272+0000 7f896be00640 1 --2- 172.21.15.32:0/917633886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89740b3cd0 0x7f89740c55f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f895c0097c0 tx=0x7f895c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafa2d7df19b5cfe server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:03.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.272+0000 7f8972000640 1 -- 172.21.15.32:0/917633886 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c0b0070 con 0x7f89740b3cd0 2024-04-17T09:57:03.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.272+0000 7f8972000640 1 -- 172.21.15.32:0/917633886 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f895c0ab600 con 0x7f89740b3cd0 2024-04-17T09:57:03.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.272+0000 7f8972000640 1 -- 172.21.15.32:0/917633886 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c0ab9b0 con 0x7f89740b3cd0 2024-04-17T09:57:03.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.272+0000 7f8979a00640 1 -- 172.21.15.32:0/917633886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89740b3cd0 msgr2=0x7f89740c55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.272+0000 7f8979a00640 1 --2- 172.21.15.32:0/917633886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89740b3cd0 0x7f89740c55f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f895c0097c0 tx=0x7f895c01c4a0 comp rx=0 tx=0).stop 2024-04-17T09:57:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.272+0000 7f8979a00640 1 -- 172.21.15.32:0/917633886 shutdown_connections 2024-04-17T09:57:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.272+0000 7f8979a00640 1 --2- 172.21.15.32:0/917633886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89740c5b30 0x7f89740d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.272+0000 7f8979a00640 1 --2- 172.21.15.32:0/917633886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89740b3cd0 0x7f89740c55f0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.272+0000 7f8979a00640 1 -- 172.21.15.32:0/917633886 >> 172.21.15.32:0/917633886 conn(0x7f89740b4110 msgr2=0x7f89740b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.272+0000 7f8979a00640 1 -- 172.21.15.32:0/917633886 shutdown_connections 2024-04-17T09:57:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.272+0000 7f8979a00640 1 -- 172.21.15.32:0/917633886 wait complete. 2024-04-17T09:57:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.272+0000 7f8979a00640 1 Processor -- start 2024-04-17T09:57:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.272+0000 7f8979a00640 1 -- start start 2024-04-17T09:57:03.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f8979a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89740b2c10 0x7f897415b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:03.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f8979a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89740b3cd0 0x7f89740b6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:03.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f8979a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89740c5b30 0x7f897415de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:03.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f8979a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f89740d1910 con 0x7f89740b2c10 2024-04-17T09:57:03.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f8979a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f89740d1790 con 0x7f89740b3cd0 2024-04-17T09:57:03.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f8979a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f89740d1a90 con 0x7f89740c5b30 2024-04-17T09:57:03.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f8972a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89740b2c10 0x7f897415b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:03.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f8972a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89740b2c10 0x7f897415b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39164/0 (socket says 172.21.15.32:39164) 2024-04-17T09:57:03.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f8972a00640 1 -- 172.21.15.32:0/3446943806 learned_addr learned my addr 172.21.15.32:0/3446943806 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:03.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f8973400640 1 --2- 172.21.15.32:0/3446943806 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89740c5b30 0x7f897415de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:03.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f8972a00640 1 -- 172.21.15.32:0/3446943806 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89740c5b30 msgr2=0x7f897415de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:03.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f896be00640 1 --2- 172.21.15.32:0/3446943806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89740b3cd0 0x7f89740b6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:03.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f8972a00640 1 --2- 172.21.15.32:0/3446943806 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89740c5b30 0x7f897415de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:03.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f8972a00640 1 -- 172.21.15.32:0/3446943806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89740b3cd0 msgr2=0x7f89740b6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:03.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f8972a00640 1 --2- 172.21.15.32:0/3446943806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89740b3cd0 0x7f89740b6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:03.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f8972a00640 1 -- 172.21.15.32:0/3446943806 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89740c8c40 con 0x7f89740b2c10 2024-04-17T09:57:03.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f8973400640 1 --2- 172.21.15.32:0/3446943806 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89740c5b30 0x7f897415de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:57:03.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f8972a00640 1 --2- 172.21.15.32:0/3446943806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89740b2c10 0x7f897415b9d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f896400c3f0 tx=0x7f896400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:03.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f896aa00640 1 -- 172.21.15.32:0/3446943806 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8964017070 con 0x7f89740b2c10 2024-04-17T09:57:03.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f896aa00640 1 -- 172.21.15.32:0/3446943806 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8964005cf0 con 0x7f89740b2c10 2024-04-17T09:57:03.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f896aa00640 1 -- 172.21.15.32:0/3446943806 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89640072f0 con 0x7f89740b2c10 2024-04-17T09:57:03.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f8979a00640 1 -- 172.21.15.32:0/3446943806 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89740c8f20 con 0x7f89740b2c10 2024-04-17T09:57:03.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f8979a00640 1 -- 172.21.15.32:0/3446943806 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f897415bf10 con 0x7f89740b2c10 2024-04-17T09:57:03.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.276+0000 7f8979a00640 1 -- 172.21.15.32:0/3446943806 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8974068a90 con 0x7f89740b2c10 2024-04-17T09:57:03.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.280+0000 7f896aa00640 1 -- 172.21.15.32:0/3446943806 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f896401a070 con 0x7f89740b2c10 2024-04-17T09:57:03.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.280+0000 7f896aa00640 1 --2- 172.21.15.32:0/3446943806 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8950068380 0x7f895006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:03.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.280+0000 7f896aa00640 1 -- 172.21.15.32:0/3446943806 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7f8964085b90 con 0x7f89740b2c10 2024-04-17T09:57:03.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.280+0000 7f896be00640 1 --2- 172.21.15.32:0/3446943806 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8950068380 0x7f895006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:03.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.280+0000 7f896be00640 1 --2- 172.21.15.32:0/3446943806 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8950068380 0x7f895006a840 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f895c0ad000 tx=0x7f895c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:03.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.284+0000 7f896aa00640 1 -- 172.21.15.32:0/3446943806 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f89640563b0 con 0x7f89740b2c10 2024-04-17T09:57:03.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.460+0000 7f8979a00640 1 -- 172.21.15.32:0/3446943806 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f89740b4d00 con 0x7f89740b2c10 2024-04-17T09:57:03.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.464+0000 7f896aa00640 1 -- 172.21.15.32:0/3446943806 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f8964055cb0 con 0x7f89740b2c10 2024-04-17T09:57:03.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.468+0000 7f8979a00640 1 -- 172.21.15.32:0/3446943806 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8950068380 msgr2=0x7f895006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:03.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.468+0000 7f8979a00640 1 --2- 172.21.15.32:0/3446943806 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8950068380 0x7f895006a840 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f895c0ad000 tx=0x7f895c008040 comp rx=0 tx=0).stop 2024-04-17T09:57:03.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.468+0000 7f8979a00640 1 -- 172.21.15.32:0/3446943806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89740b2c10 msgr2=0x7f897415b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:03.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.468+0000 7f8979a00640 1 --2- 172.21.15.32:0/3446943806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89740b2c10 0x7f897415b9d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f896400c3f0 tx=0x7f896400c8b0 comp rx=0 tx=0).stop 2024-04-17T09:57:03.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.468+0000 7f8979a00640 1 -- 172.21.15.32:0/3446943806 shutdown_connections 2024-04-17T09:57:03.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.468+0000 7f8979a00640 1 --2- 172.21.15.32:0/3446943806 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8950068380 0x7f895006a840 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:03.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.468+0000 7f8979a00640 1 --2- 172.21.15.32:0/3446943806 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f89740c5b30 0x7f897415de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:03.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.468+0000 7f8979a00640 1 --2- 172.21.15.32:0/3446943806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89740b3cd0 0x7f89740b6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:03.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.468+0000 7f8979a00640 1 --2- 172.21.15.32:0/3446943806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89740b2c10 0x7f897415b9d0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:03.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.468+0000 7f8979a00640 1 -- 172.21.15.32:0/3446943806 >> 172.21.15.32:0/3446943806 conn(0x7f89740b4110 msgr2=0x7f897415c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:03.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.468+0000 7f8979a00640 1 -- 172.21.15.32:0/3446943806 shutdown_connections 2024-04-17T09:57:03.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.468+0000 7f8979a00640 1 -- 172.21.15.32:0/3446943806 wait complete. 2024-04-17T09:57:03.487 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-17T09:57:03.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.628+0000 7f6854800640 1 Processor -- start 2024-04-17T09:57:03.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.628+0000 7f6854800640 1 -- start start 2024-04-17T09:57:03.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.628+0000 7f6854800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f684c09bc30 0x7f684c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:03.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.628+0000 7f6854800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f684c0d09a0 con 0x7f684c15e9a0 2024-04-17T09:57:03.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.628+0000 7f6854800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f684c0d0c80 con 0x7f684c15ed80 2024-04-17T09:57:03.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.628+0000 7f6854800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f684c0d0f60 con 0x7f684c09bc30 2024-04-17T09:57:03.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.628+0000 7f6851e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f684c15e9a0 0x7f684c0d1a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55182/0 (socket says 172.21.15.32:55182) 2024-04-17T09:57:03.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.628+0000 7f6851e00640 1 -- 172.21.15.32:0/3102421484 learned_addr learned my addr 172.21.15.32:0/3102421484 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:03.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.628+0000 7f6852800640 1 --2- 172.21.15.32:0/3102421484 >> v2:172.21.15.32:3300/0 conn(0x7f684c09bc30 0x7f684c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:03.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.628+0000 7f6850a00640 1 -- 172.21.15.32:0/3102421484 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2107119611 0 0) 0x7f684c0d09a0 con 0x7f684c15e9a0 2024-04-17T09:57:03.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.628+0000 7f6850a00640 1 -- 172.21.15.32:0/3102421484 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6834003530 con 0x7f684c15e9a0 2024-04-17T09:57:03.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.628+0000 7f6852800640 1 -- 172.21.15.32:0/3102421484 >> v1:172.21.15.144:6789/0 conn(0x7f684c15ed80 legacy=0x7f684c09b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:03.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.628+0000 7f6852800640 1 -- 172.21.15.32:0/3102421484 >> v1:172.21.15.32:6789/0 conn(0x7f684c15e9a0 legacy=0x7f684c0d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:03.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.628+0000 7f6852800640 1 -- 172.21.15.32:0/3102421484 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f684c0d1240 con 0x7f684c09bc30 2024-04-17T09:57:03.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.632+0000 7f6852800640 1 --2- 172.21.15.32:0/3102421484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684c09bc30 0x7f684c06d450 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f6848009d90 tx=0x7f684801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa37aa8ad0913ae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:03.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.632+0000 7f6850a00640 1 -- 172.21.15.32:0/3102421484 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68480b2070 con 0x7f684c09bc30 2024-04-17T09:57:03.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.632+0000 7f6850a00640 1 -- 172.21.15.32:0/3102421484 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68480a5910 con 0x7f684c09bc30 2024-04-17T09:57:03.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.632+0000 7f6850a00640 1 -- 172.21.15.32:0/3102421484 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68480a5ce0 con 0x7f684c09bc30 2024-04-17T09:57:03.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.632+0000 7f6854800640 1 -- 172.21.15.32:0/3102421484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684c09bc30 msgr2=0x7f684c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:03.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.632+0000 7f6854800640 1 --2- 172.21.15.32:0/3102421484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684c09bc30 0x7f684c06d450 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f6848009d90 tx=0x7f684801c940 comp rx=0 tx=0).stop 2024-04-17T09:57:03.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.632+0000 7f6854800640 1 -- 172.21.15.32:0/3102421484 shutdown_connections 2024-04-17T09:57:03.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.632+0000 7f6854800640 1 --2- 172.21.15.32:0/3102421484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684c09bc30 0x7f684c06d450 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:03.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.632+0000 7f6854800640 1 -- 172.21.15.32:0/3102421484 >> 172.21.15.32:0/3102421484 conn(0x7f684c0b3fd0 msgr2=0x7f684c0b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:03.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.632+0000 7f6854800640 1 -- 172.21.15.32:0/3102421484 shutdown_connections 2024-04-17T09:57:03.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.632+0000 7f6854800640 1 -- 172.21.15.32:0/3102421484 wait complete. 2024-04-17T09:57:03.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6854800640 1 Processor -- start 2024-04-17T09:57:03.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6854800640 1 -- start start 2024-04-17T09:57:03.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6854800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684c09bc30 0x7f684c15de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:03.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6854800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c15e9a0 0x7f684c14c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:03.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6854800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684c15ed80 0x7f684c0af170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:03.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6854800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f684c16ca20 con 0x7f684c09bc30 2024-04-17T09:57:03.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6854800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f684c0cd550 con 0x7f684c15e9a0 2024-04-17T09:57:03.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6854800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f684c16cba0 con 0x7f684c15ed80 2024-04-17T09:57:03.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6851e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684c09bc30 0x7f684c15de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:03.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6851e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684c09bc30 0x7f684c15de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39174/0 (socket says 172.21.15.32:39174) 2024-04-17T09:57:03.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6851e00640 1 -- 172.21.15.32:0/1483813667 learned_addr learned my addr 172.21.15.32:0/1483813667 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:03.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6852800640 1 --2- 172.21.15.32:0/1483813667 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684c15ed80 0x7f684c0af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:03.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6851400640 1 --2- 172.21.15.32:0/1483813667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c15e9a0 0x7f684c14c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:03.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6851e00640 1 -- 172.21.15.32:0/1483813667 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684c15ed80 msgr2=0x7f684c0af170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:03.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6851e00640 1 --2- 172.21.15.32:0/1483813667 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684c15ed80 0x7f684c0af170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:03.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6851e00640 1 -- 172.21.15.32:0/1483813667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c15e9a0 msgr2=0x7f684c14c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:03.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6851e00640 1 --2- 172.21.15.32:0/1483813667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c15e9a0 0x7f684c14c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:03.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6851e00640 1 -- 172.21.15.32:0/1483813667 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f684c14ca00 con 0x7f684c09bc30 2024-04-17T09:57:03.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6852800640 1 --2- 172.21.15.32:0/1483813667 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684c15ed80 0x7f684c0af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:57:03.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6851400640 1 --2- 172.21.15.32:0/1483813667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c15e9a0 0x7f684c14c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:57:03.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6851e00640 1 --2- 172.21.15.32:0/1483813667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684c09bc30 0x7f684c15de80 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f684000c4d0 tx=0x7f684000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:03.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6846000640 1 -- 172.21.15.32:0/1483813667 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840017070 con 0x7f684c09bc30 2024-04-17T09:57:03.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6846000640 1 -- 172.21.15.32:0/1483813667 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6840005cf0 con 0x7f684c09bc30 2024-04-17T09:57:03.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6846000640 1 -- 172.21.15.32:0/1483813667 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840007270 con 0x7f684c09bc30 2024-04-17T09:57:03.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6854800640 1 -- 172.21.15.32:0/1483813667 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f684c0ca6a0 con 0x7f684c09bc30 2024-04-17T09:57:03.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.636+0000 7f6854800640 1 -- 172.21.15.32:0/1483813667 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f684c154020 con 0x7f684c09bc30 2024-04-17T09:57:03.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.640+0000 7f6854800640 1 -- 172.21.15.32:0/1483813667 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f684c068a90 con 0x7f684c09bc30 2024-04-17T09:57:03.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.640+0000 7f6846000640 1 -- 172.21.15.32:0/1483813667 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f684001a070 con 0x7f684c09bc30 2024-04-17T09:57:03.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.640+0000 7f6846000640 1 --2- 172.21.15.32:0/1483813667 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6830068380 0x7f683006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:03.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.640+0000 7f6846000640 1 -- 172.21.15.32:0/1483813667 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7f68400859b0 con 0x7f684c09bc30 2024-04-17T09:57:03.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.640+0000 7f6851400640 1 --2- 172.21.15.32:0/1483813667 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6830068380 0x7f683006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:03.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.640+0000 7f6851400640 1 --2- 172.21.15.32:0/1483813667 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6830068380 0x7f683006a840 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f684c071bb0 tx=0x7f683c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:03.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.644+0000 7f6846000640 1 -- 172.21.15.32:0/1483813667 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f6840055c10 con 0x7f684c09bc30 2024-04-17T09:57:03.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:03.864+0000 7f6854800640 1 -- 172.21.15.32:0/1483813667 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f684c0b4af0 con 0x7f684c09bc30 2024-04-17T09:57:04.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.104+0000 7f6846000640 1 -- 172.21.15.32:0/1483813667 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v207) ==== 169+0+0 (secure 0 0 0) 0x7f6840059f10 con 0x7f684c09bc30 2024-04-17T09:57:04.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.108+0000 7f6854800640 1 -- 172.21.15.32:0/1483813667 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6830068380 msgr2=0x7f683006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:04.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.108+0000 7f6854800640 1 --2- 172.21.15.32:0/1483813667 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6830068380 0x7f683006a840 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f684c071bb0 tx=0x7f683c009040 comp rx=0 tx=0).stop 2024-04-17T09:57:04.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.108+0000 7f6854800640 1 -- 172.21.15.32:0/1483813667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684c09bc30 msgr2=0x7f684c15de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:04.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.108+0000 7f6854800640 1 --2- 172.21.15.32:0/1483813667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684c09bc30 0x7f684c15de80 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f684000c4d0 tx=0x7f684000c990 comp rx=0 tx=0).stop 2024-04-17T09:57:04.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.108+0000 7f6854800640 1 -- 172.21.15.32:0/1483813667 shutdown_connections 2024-04-17T09:57:04.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.108+0000 7f6854800640 1 --2- 172.21.15.32:0/1483813667 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6830068380 0x7f683006a840 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.108+0000 7f6854800640 1 --2- 172.21.15.32:0/1483813667 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684c15ed80 0x7f684c0af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.108+0000 7f6854800640 1 --2- 172.21.15.32:0/1483813667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c15e9a0 0x7f684c14c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.108+0000 7f6854800640 1 --2- 172.21.15.32:0/1483813667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684c09bc30 0x7f684c15de80 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.108+0000 7f6854800640 1 -- 172.21.15.32:0/1483813667 >> 172.21.15.32:0/1483813667 conn(0x7f684c0b3fd0 msgr2=0x7f684c0cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:04.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.108+0000 7f6854800640 1 -- 172.21.15.32:0/1483813667 shutdown_connections 2024-04-17T09:57:04.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.108+0000 7f6854800640 1 -- 172.21.15.32:0/1483813667 wait complete. 2024-04-17T09:57:04.129 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-17T09:57:04.129 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-17T09:57:04.129 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-17T09:57:04.130 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:57:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.268+0000 7fe6bf600640 1 Processor -- start 2024-04-17T09:57:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.268+0000 7fe6bf600640 1 -- start start 2024-04-17T09:57:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.272+0000 7fe6bf600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6b8144e90 0x7fe6b80b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.272+0000 7fe6bf600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe6b815c1b0 0x7fe6b806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.272+0000 7fe6bf600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6b8072bb0 con 0x7fe6b80b4de0 2024-04-17T09:57:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.272+0000 7fe6bf600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe6b8145270 con 0x7fe6b815c1b0 2024-04-17T09:57:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.272+0000 7fe6bf600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe6b80b3dc0 con 0x7fe6b8144e90 2024-04-17T09:57:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.272+0000 7fe6bcc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6b8144e90 0x7fe6b80b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.272+0000 7fe6afe00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fe6b80b4de0 0x7fe6b815baa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:49978/0 (socket says 172.21.15.32:49978) 2024-04-17T09:57:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.272+0000 7fe6afe00640 1 -- 172.21.15.32:0/2916764625 learned_addr learned my addr 172.21.15.32:0/2916764625 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.272+0000 7fe6bd600640 1 --2- 172.21.15.32:0/2916764625 >> v2:172.21.15.144:3300/0 conn(0x7fe6b815c1b0 0x7fe6b806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.272+0000 7fe6bd600640 1 -- 172.21.15.32:0/2916764625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6b8144e90 msgr2=0x7fe6b80b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.272+0000 7fe6bd600640 1 --2- 172.21.15.32:0/2916764625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6b8144e90 0x7fe6b80b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.272+0000 7fe6bd600640 1 -- 172.21.15.32:0/2916764625 >> v1:172.21.15.144:6789/0 conn(0x7fe6b80b4de0 legacy=0x7fe6b815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.272+0000 7fe6b7e00640 1 -- 172.21.15.32:0/2916764625 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3584437618 0 0) 0x7fe6b8072bb0 con 0x7fe6b80b4de0 2024-04-17T09:57:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.272+0000 7fe6bd600640 1 -- 172.21.15.32:0/2916764625 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b80b40a0 con 0x7fe6b815c1b0 2024-04-17T09:57:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.272+0000 7fe6bcc00640 1 --2- 172.21.15.32:0/2916764625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6b8144e90 0x7fe6b80b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:57:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.276+0000 7fe6bd600640 1 --2- 172.21.15.32:0/2916764625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6b815c1b0 0x7fe6b806ef60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0009d90 tx=0x7fe6b001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65a1603d74e6fac5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.276+0000 7fe6b7e00640 1 -- 172.21.15.32:0/2916764625 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b00b2070 con 0x7fe6b815c1b0 2024-04-17T09:57:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.276+0000 7fe6b7e00640 1 -- 172.21.15.32:0/2916764625 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6b00a5910 con 0x7fe6b815c1b0 2024-04-17T09:57:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.276+0000 7fe6b7e00640 1 -- 172.21.15.32:0/2916764625 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b00a5ce0 con 0x7fe6b815c1b0 2024-04-17T09:57:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.276+0000 7fe6bf600640 1 -- 172.21.15.32:0/2916764625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6b815c1b0 msgr2=0x7fe6b806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:04.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.276+0000 7fe6bf600640 1 --2- 172.21.15.32:0/2916764625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6b815c1b0 0x7fe6b806ef60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0009d90 tx=0x7fe6b001c940 comp rx=0 tx=0).stop 2024-04-17T09:57:04.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.276+0000 7fe6bf600640 1 -- 172.21.15.32:0/2916764625 shutdown_connections 2024-04-17T09:57:04.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.276+0000 7fe6bf600640 1 --2- 172.21.15.32:0/2916764625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6b815c1b0 0x7fe6b806ef60 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.276+0000 7fe6bf600640 1 --2- 172.21.15.32:0/2916764625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6b8144e90 0x7fe6b80b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.276+0000 7fe6bf600640 1 -- 172.21.15.32:0/2916764625 >> 172.21.15.32:0/2916764625 conn(0x7fe6b806ba30 msgr2=0x7fe6b806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:04.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.276+0000 7fe6bf600640 1 -- 172.21.15.32:0/2916764625 shutdown_connections 2024-04-17T09:57:04.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.276+0000 7fe6bf600640 1 -- 172.21.15.32:0/2916764625 wait complete. 2024-04-17T09:57:04.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.276+0000 7fe6bf600640 1 Processor -- start 2024-04-17T09:57:04.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.276+0000 7fe6bf600640 1 -- start start 2024-04-17T09:57:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.280+0000 7fe6bf600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6b80b4de0 0x7fe6b80aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.280+0000 7fe6bf600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6b8144e90 0x7fe6b813f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.280+0000 7fe6bf600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6b815c1b0 0x7fe6b815d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.280+0000 7fe6bf600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b8168110 con 0x7fe6b80b4de0 2024-04-17T09:57:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.280+0000 7fe6bf600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b8072880 con 0x7fe6b815c1b0 2024-04-17T09:57:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.280+0000 7fe6bf600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe6b8168290 con 0x7fe6b8144e90 2024-04-17T09:57:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.280+0000 7fe6bcc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6b80b4de0 0x7fe6b80aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.280+0000 7fe6bcc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6b80b4de0 0x7fe6b80aec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39182/0 (socket says 172.21.15.32:39182) 2024-04-17T09:57:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.280+0000 7fe6bcc00640 1 -- 172.21.15.32:0/1517343954 learned_addr learned my addr 172.21.15.32:0/1517343954 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.280+0000 7fe6afe00640 1 --2- 172.21.15.32:0/1517343954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6b8144e90 0x7fe6b813f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:04.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.280+0000 7fe6bd600640 1 --2- 172.21.15.32:0/1517343954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6b815c1b0 0x7fe6b815d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:04.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.280+0000 7fe6bcc00640 1 -- 172.21.15.32:0/1517343954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6b8144e90 msgr2=0x7fe6b813f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:04.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.280+0000 7fe6bcc00640 1 --2- 172.21.15.32:0/1517343954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6b8144e90 0x7fe6b813f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.280+0000 7fe6bcc00640 1 -- 172.21.15.32:0/1517343954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6b815c1b0 msgr2=0x7fe6b815d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:04.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.280+0000 7fe6bcc00640 1 --2- 172.21.15.32:0/1517343954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6b815c1b0 0x7fe6b815d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.280+0000 7fe6bcc00640 1 -- 172.21.15.32:0/1517343954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b80b2ee0 con 0x7fe6b80b4de0 2024-04-17T09:57:04.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.280+0000 7fe6bcc00640 1 --2- 172.21.15.32:0/1517343954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6b80b4de0 0x7fe6b80aec30 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800ebe0 tx=0x7fe6a800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:04.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.280+0000 7fe6b5600640 1 -- 172.21.15.32:0/1517343954 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8017070 con 0x7fe6b80b4de0 2024-04-17T09:57:04.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.280+0000 7fe6b5600640 1 -- 172.21.15.32:0/1517343954 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6a800cc30 con 0x7fe6b80b4de0 2024-04-17T09:57:04.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.280+0000 7fe6b5600640 1 -- 172.21.15.32:0/1517343954 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8013470 con 0x7fe6b80b4de0 2024-04-17T09:57:04.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.280+0000 7fe6bf600640 1 -- 172.21.15.32:0/1517343954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b813e530 con 0x7fe6b80b4de0 2024-04-17T09:57:04.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.280+0000 7fe6bf600640 1 -- 172.21.15.32:0/1517343954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b813eaf0 con 0x7fe6b80b4de0 2024-04-17T09:57:04.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.280+0000 7fe6bf600640 1 -- 172.21.15.32:0/1517343954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6b8068a90 con 0x7fe6b80b4de0 2024-04-17T09:57:04.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.284+0000 7fe6b5600640 1 -- 172.21.15.32:0/1517343954 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe6a801a070 con 0x7fe6b80b4de0 2024-04-17T09:57:04.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.284+0000 7fe6b5600640 1 --2- 172.21.15.32:0/1517343954 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe694068380 0x7fe69406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:04.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.284+0000 7fe6b5600640 1 -- 172.21.15.32:0/1517343954 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7fe6a8086610 con 0x7fe6b80b4de0 2024-04-17T09:57:04.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.284+0000 7fe6afe00640 1 --2- 172.21.15.32:0/1517343954 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe694068380 0x7fe69406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:04.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.284+0000 7fe6afe00640 1 --2- 172.21.15.32:0/1517343954 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe694068380 0x7fe69406a840 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe6b813d2b0 tx=0x7fe6a0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:04.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.288+0000 7fe6b5600640 1 -- 172.21.15.32:0/1517343954 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fe6a80567e0 con 0x7fe6b80b4de0 2024-04-17T09:57:04.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.452+0000 7fe6bf600640 1 -- 172.21.15.32:0/1517343954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6b8144230 con 0x7fe6b80b4de0 2024-04-17T09:57:04.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.452+0000 7fe6b5600640 1 -- 172.21.15.32:0/1517343954 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+13030 (secure 0 0 0) 0x7fe6a805aae0 con 0x7fe6b80b4de0 2024-04-17T09:57:04.453 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:57:04.453 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":60,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:57:03.092376+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6837/866826055":"2024-04-18T09:57:00.082564+0000","172.21.15.32:6835/866826055":"2024-04-18T09:57:00.082564+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:57:04.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.456+0000 7fe6bf600640 1 -- 172.21.15.32:0/1517343954 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe694068380 msgr2=0x7fe69406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:04.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.456+0000 7fe6bf600640 1 --2- 172.21.15.32:0/1517343954 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe694068380 0x7fe69406a840 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe6b813d2b0 tx=0x7fe6a0009040 comp rx=0 tx=0).stop 2024-04-17T09:57:04.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.456+0000 7fe6bf600640 1 -- 172.21.15.32:0/1517343954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6b80b4de0 msgr2=0x7fe6b80aec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:04.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.456+0000 7fe6bf600640 1 --2- 172.21.15.32:0/1517343954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6b80b4de0 0x7fe6b80aec30 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800ebe0 tx=0x7fe6a800c520 comp rx=0 tx=0).stop 2024-04-17T09:57:04.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.456+0000 7fe6bf600640 1 -- 172.21.15.32:0/1517343954 shutdown_connections 2024-04-17T09:57:04.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.456+0000 7fe6bf600640 1 --2- 172.21.15.32:0/1517343954 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe694068380 0x7fe69406a840 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.456+0000 7fe6bf600640 1 --2- 172.21.15.32:0/1517343954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6b8144e90 0x7fe6b813f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.456+0000 7fe6bf600640 1 --2- 172.21.15.32:0/1517343954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6b815c1b0 0x7fe6b815d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.456+0000 7fe6bf600640 1 --2- 172.21.15.32:0/1517343954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6b80b4de0 0x7fe6b80aec30 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.456+0000 7fe6bf600640 1 -- 172.21.15.32:0/1517343954 >> 172.21.15.32:0/1517343954 conn(0x7fe6b806ba30 msgr2=0x7fe6b806be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:04.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.456+0000 7fe6bf600640 1 -- 172.21.15.32:0/1517343954 shutdown_connections 2024-04-17T09:57:04.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.456+0000 7fe6bf600640 1 -- 172.21.15.32:0/1517343954 wait complete. 2024-04-17T09:57:04.474 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:57:04.475 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:57:04.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.616+0000 7fdce3e00640 1 Processor -- start 2024-04-17T09:57:04.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.616+0000 7fdce3e00640 1 -- start start 2024-04-17T09:57:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.616+0000 7fdce3e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcdc15b010 0x7fdcdc06dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.616+0000 7fdce3e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdcdc0b3d80 0x7fdcdc071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.616+0000 7fdce3e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcdc06e6f0 con 0x7fdcdc15ac30 2024-04-17T09:57:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.616+0000 7fdce3e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fdcdc06f300 con 0x7fdcdc0b3d80 2024-04-17T09:57:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.616+0000 7fdce3e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdcdc06e9d0 con 0x7fdcdc15b010 2024-04-17T09:57:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.616+0000 7fdce1400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdcdc15ac30 0x7fdcdc06f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55194/0 (socket says 172.21.15.32:55194) 2024-04-17T09:57:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.616+0000 7fdce1400640 1 -- 172.21.15.32:0/3591609358 learned_addr learned my addr 172.21.15.32:0/3591609358 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdce0a00640 1 --2- 172.21.15.32:0/3591609358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcdc15b010 0x7fdcdc06dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdce1e00640 1 --2- 172.21.15.32:0/3591609358 >> v2:172.21.15.144:3300/0 conn(0x7fdcdc0b3d80 0x7fdcdc071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdcdbe00640 1 -- 172.21.15.32:0/3591609358 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3941082677 0 0) 0x7fdcdc06e6f0 con 0x7fdcdc15ac30 2024-04-17T09:57:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdcdbe00640 1 -- 172.21.15.32:0/3591609358 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcc4005740 con 0x7fdcdc15ac30 2024-04-17T09:57:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdcdbe00640 1 -- 172.21.15.32:0/3591609358 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 66538036 0 0) 0x7fdcc4005740 con 0x7fdcdc15ac30 2024-04-17T09:57:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdcdbe00640 1 -- 172.21.15.32:0/3591609358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcdc15b010 msgr2=0x7fdcdc06dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdcdbe00640 1 --2- 172.21.15.32:0/3591609358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcdc15b010 0x7fdcdc06dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdcdbe00640 1 -- 172.21.15.32:0/3591609358 >> v2:172.21.15.144:3300/0 conn(0x7fdcdc0b3d80 msgr2=0x7fdcdc071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdcdbe00640 1 --2- 172.21.15.32:0/3591609358 >> v2:172.21.15.144:3300/0 conn(0x7fdcdc0b3d80 0x7fdcdc071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdcdbe00640 1 -- 172.21.15.32:0/3591609358 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcdc06ecb0 con 0x7fdcdc15ac30 2024-04-17T09:57:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdcdbe00640 1 -- 172.21.15.32:0/3591609358 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7fdcd0001680 con 0x7fdcdc15ac30 2024-04-17T09:57:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdcdbe00640 1 -- 172.21.15.32:0/3591609358 >> v1:172.21.15.32:6789/0 conn(0x7fdcdc15ac30 legacy=0x7fdcdc06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdcdbe00640 1 --2- 172.21.15.32:0/3591609358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc4005bd0 0x7fdcc4006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdcdbe00640 1 --2- 172.21.15.32:0/3591609358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcc4006a20 0x7fdcc4006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdcdbe00640 1 --2- 172.21.15.32:0/3591609358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcdc15b010 0x7fdcc40073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdcdbe00640 1 -- 172.21.15.32:0/3591609358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc06f300 con 0x7fdcc4006a20 2024-04-17T09:57:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdcdbe00640 1 -- 172.21.15.32:0/3591609358 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc06e9d0 con 0x7fdcc4005bd0 2024-04-17T09:57:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdcdbe00640 1 -- 172.21.15.32:0/3591609358 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc400dbe0 con 0x7fdcdc15b010 2024-04-17T09:57:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdce0a00640 1 --2- 172.21.15.32:0/3591609358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcc4006a20 0x7fdcc4006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdce1400640 1 --2- 172.21.15.32:0/3591609358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc4005bd0 0x7fdcc4006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdce1e00640 1 --2- 172.21.15.32:0/3591609358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcdc15b010 0x7fdcc40073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdce0a00640 1 -- 172.21.15.32:0/3591609358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcdc15b010 msgr2=0x7fdcc40073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdce0a00640 1 --2- 172.21.15.32:0/3591609358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcdc15b010 0x7fdcc40073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdce0a00640 1 -- 172.21.15.32:0/3591609358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc4005bd0 msgr2=0x7fdcc4006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdce0a00640 1 --2- 172.21.15.32:0/3591609358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc4005bd0 0x7fdcc4006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.620+0000 7fdce0a00640 1 -- 172.21.15.32:0/3591609358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc4005740 con 0x7fdcc4006a20 2024-04-17T09:57:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.624+0000 7fdce0a00640 1 --2- 172.21.15.32:0/3591609358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcc4006a20 0x7fdcc4006ea0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009ae0 tx=0x7fdcc801c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d682d3e8f461a18 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.624+0000 7fdcdbe00640 1 -- 172.21.15.32:0/3591609358 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80b1070 con 0x7fdcc4006a20 2024-04-17T09:57:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.624+0000 7fdcdbe00640 1 -- 172.21.15.32:0/3591609358 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcc80a4a70 con 0x7fdcc4006a20 2024-04-17T09:57:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.624+0000 7fdcdbe00640 1 -- 172.21.15.32:0/3591609358 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80096b0 con 0x7fdcc4006a20 2024-04-17T09:57:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.624+0000 7fdce3e00640 1 -- 172.21.15.32:0/3591609358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcc4006a20 msgr2=0x7fdcc4006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.624+0000 7fdce3e00640 1 --2- 172.21.15.32:0/3591609358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcc4006a20 0x7fdcc4006ea0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009ae0 tx=0x7fdcc801c710 comp rx=0 tx=0).stop 2024-04-17T09:57:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.624+0000 7fdce1e00640 1 -- 172.21.15.32:0/3591609358 reap_dead start 2024-04-17T09:57:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.624+0000 7fdce3e00640 1 -- 172.21.15.32:0/3591609358 shutdown_connections 2024-04-17T09:57:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.624+0000 7fdce3e00640 1 -- 172.21.15.32:0/3591609358 >> 172.21.15.32:0/3591609358 conn(0x7fdcdc0b2120 msgr2=0x7fdcdc0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.624+0000 7fdce3e00640 1 -- 172.21.15.32:0/3591609358 shutdown_connections 2024-04-17T09:57:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.624+0000 7fdce3e00640 1 -- 172.21.15.32:0/3591609358 wait complete. 2024-04-17T09:57:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.624+0000 7fdce3e00640 1 Processor -- start 2024-04-17T09:57:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.624+0000 7fdce3e00640 1 -- start start 2024-04-17T09:57:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.624+0000 7fdce3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcc4006a20 0x7fdcdc1e6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.624+0000 7fdce3e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcdc15a0e0 0x7fdcdc1466a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.624+0000 7fdce3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcdc1e8ed0 0x7fdcdc0cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.624+0000 7fdce3e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc072520 con 0x7fdcc4006a20 2024-04-17T09:57:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.624+0000 7fdce3e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc0723a0 con 0x7fdcdc1e8ed0 2024-04-17T09:57:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.624+0000 7fdce3e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdcdc0726a0 con 0x7fdcdc15a0e0 2024-04-17T09:57:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.624+0000 7fdce1400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcc4006a20 0x7fdcdc1e6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.628+0000 7fdce1400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcc4006a20 0x7fdcdc1e6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39206/0 (socket says 172.21.15.32:39206) 2024-04-17T09:57:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.628+0000 7fdce1400640 1 -- 172.21.15.32:0/3423589781 learned_addr learned my addr 172.21.15.32:0/3423589781 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.628+0000 7fdce1400640 1 -- 172.21.15.32:0/3423589781 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcdc15a0e0 msgr2=0x7fdcdc1466a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.628+0000 7fdce1400640 1 --2- 172.21.15.32:0/3423589781 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcdc15a0e0 0x7fdcdc1466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.628+0000 7fdce1400640 1 -- 172.21.15.32:0/3423589781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcdc1e8ed0 msgr2=0x7fdcdc0cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.628+0000 7fdce1400640 1 --2- 172.21.15.32:0/3423589781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcdc1e8ed0 0x7fdcdc0cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.628+0000 7fdce1400640 1 -- 172.21.15.32:0/3423589781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcdc147010 con 0x7fdcc4006a20 2024-04-17T09:57:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.628+0000 7fdce1400640 1 --2- 172.21.15.32:0/3423589781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcc4006a20 0x7fdcdc1e6020 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0012510 tx=0x7fdcd00129d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.628+0000 7fdcd9600640 1 -- 172.21.15.32:0/3423589781 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd001c070 con 0x7fdcc4006a20 2024-04-17T09:57:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.628+0000 7fdcd9600640 1 -- 172.21.15.32:0/3423589781 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcd0014750 con 0x7fdcc4006a20 2024-04-17T09:57:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.628+0000 7fdcd9600640 1 -- 172.21.15.32:0/3423589781 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0014b20 con 0x7fdcc4006a20 2024-04-17T09:57:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.628+0000 7fdce3e00640 1 -- 172.21.15.32:0/3423589781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcdc146be0 con 0x7fdcc4006a20 2024-04-17T09:57:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.628+0000 7fdce3e00640 1 -- 172.21.15.32:0/3423589781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcdc147590 con 0x7fdcc4006a20 2024-04-17T09:57:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.628+0000 7fdce3e00640 1 -- 172.21.15.32:0/3423589781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcdc068a90 con 0x7fdcc4006a20 2024-04-17T09:57:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.628+0000 7fdcd9600640 1 -- 172.21.15.32:0/3423589781 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdcd0014cc0 con 0x7fdcc4006a20 2024-04-17T09:57:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.628+0000 7fdcd9600640 1 --2- 172.21.15.32:0/3423589781 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdcbc068380 0x7fdcbc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.628+0000 7fdcd9600640 1 -- 172.21.15.32:0/3423589781 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7fdcd0088ff0 con 0x7fdcc4006a20 2024-04-17T09:57:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.628+0000 7fdce0a00640 1 --2- 172.21.15.32:0/3423589781 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdcbc068380 0x7fdcbc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.632+0000 7fdce0a00640 1 --2- 172.21.15.32:0/3423589781 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdcbc068380 0x7fdcbc06a840 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80ae000 tx=0x7fdcc8004740 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:04.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.632+0000 7fdcd9600640 1 -- 172.21.15.32:0/3423589781 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fdcd00598c0 con 0x7fdcc4006a20 2024-04-17T09:57:04.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.796+0000 7fdce3e00640 1 -- 172.21.15.32:0/3423589781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdcdc072a60 con 0x7fdcc4006a20 2024-04-17T09:57:04.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.796+0000 7fdcd9600640 1 -- 172.21.15.32:0/3423589781 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) ==== 93+0+10 (secure 0 0 0) 0x7fdcd00591c0 con 0x7fdcc4006a20 2024-04-17T09:57:04.799 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:57:04.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.800+0000 7fdce3e00640 1 -- 172.21.15.32:0/3423589781 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdcbc068380 msgr2=0x7fdcbc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:04.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.800+0000 7fdce3e00640 1 --2- 172.21.15.32:0/3423589781 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdcbc068380 0x7fdcbc06a840 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80ae000 tx=0x7fdcc8004740 comp rx=0 tx=0).stop 2024-04-17T09:57:04.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.800+0000 7fdce3e00640 1 -- 172.21.15.32:0/3423589781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcc4006a20 msgr2=0x7fdcdc1e6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:04.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.800+0000 7fdce3e00640 1 --2- 172.21.15.32:0/3423589781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcc4006a20 0x7fdcdc1e6020 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0012510 tx=0x7fdcd00129d0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.800+0000 7fdce3e00640 1 -- 172.21.15.32:0/3423589781 shutdown_connections 2024-04-17T09:57:04.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.800+0000 7fdce3e00640 1 --2- 172.21.15.32:0/3423589781 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdcbc068380 0x7fdcbc06a840 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.800+0000 7fdce3e00640 1 --2- 172.21.15.32:0/3423589781 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcdc15a0e0 0x7fdcdc1466a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.800+0000 7fdce3e00640 1 --2- 172.21.15.32:0/3423589781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcdc1e8ed0 0x7fdcdc0cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.800+0000 7fdce3e00640 1 --2- 172.21.15.32:0/3423589781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcc4006a20 0x7fdcdc1e6020 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.800+0000 7fdce3e00640 1 -- 172.21.15.32:0/3423589781 >> 172.21.15.32:0/3423589781 conn(0x7fdcdc0b2120 msgr2=0x7fdcdc0af0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:04.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.800+0000 7fdce3e00640 1 -- 172.21.15.32:0/3423589781 shutdown_connections 2024-04-17T09:57:04.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.800+0000 7fdce3e00640 1 -- 172.21.15.32:0/3423589781 wait complete. 2024-04-17T09:57:04.821 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-17T09:57:04.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.960+0000 7f1aebe00640 1 Processor -- start 2024-04-17T09:57:04.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.960+0000 7f1aebe00640 1 -- start start 2024-04-17T09:57:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.960+0000 7f1aebe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ae4145f30 0x7f1ae4071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.960+0000 7f1aebe00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1ae415d8d0 0x7f1ae4072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.960+0000 7f1aebe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ae40cce70 0x7f1ae40cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.960+0000 7f1aebe00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1ae406f400 con 0x7f1ae4145f30 2024-04-17T09:57:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.960+0000 7f1aebe00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1ae4146310 con 0x7f1ae415d8d0 2024-04-17T09:57:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.960+0000 7f1aebe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1ae415d440 con 0x7f1ae40cce70 2024-04-17T09:57:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.960+0000 7f1ae9400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ae4145f30 0x7f1ae4071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.960+0000 7f1ae9400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ae4145f30 0x7f1ae4071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39220/0 (socket says 172.21.15.32:39220) 2024-04-17T09:57:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.960+0000 7f1ae9400640 1 -- 172.21.15.32:0/3368908214 learned_addr learned my addr 172.21.15.32:0/3368908214 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.960+0000 7f1ae9400640 1 -- 172.21.15.32:0/3368908214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ae40cce70 msgr2=0x7f1ae40cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.960+0000 7f1ae9400640 1 --2- 172.21.15.32:0/3368908214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ae40cce70 0x7f1ae40cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.960+0000 7f1ae9400640 1 -- 172.21.15.32:0/3368908214 >> v2:172.21.15.144:3300/0 conn(0x7f1ae415d8d0 msgr2=0x7f1ae4072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.960+0000 7f1ae8a00640 1 --2- 172.21.15.32:0/3368908214 >> v2:172.21.15.144:3300/0 conn(0x7f1ae415d8d0 0x7f1ae4072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.960+0000 7f1ae9400640 1 --2- 172.21.15.32:0/3368908214 >> v2:172.21.15.144:3300/0 conn(0x7f1ae415d8d0 0x7f1ae4072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.960+0000 7f1ae9400640 1 -- 172.21.15.32:0/3368908214 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae40cd820 con 0x7f1ae4145f30 2024-04-17T09:57:04.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.960+0000 7f1ae8a00640 1 --2- 172.21.15.32:0/3368908214 >> v2:172.21.15.144:3300/0 conn(0x7f1ae415d8d0 0x7f1ae4072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:57:04.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.964+0000 7f1ae9400640 1 --2- 172.21.15.32:0/3368908214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae4145f30 0x7f1ae4071df0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40702a0 tx=0x7f1ad801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2ef23fc9898a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.964+0000 7f1ae3e00640 1 -- 172.21.15.32:0/3368908214 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad80b5070 con 0x7f1ae4145f30 2024-04-17T09:57:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.964+0000 7f1ae3e00640 1 -- 172.21.15.32:0/3368908214 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ad80a89d0 con 0x7f1ae4145f30 2024-04-17T09:57:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.964+0000 7f1ae3e00640 1 -- 172.21.15.32:0/3368908214 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad80a8da0 con 0x7f1ae4145f30 2024-04-17T09:57:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.964+0000 7f1aebe00640 1 -- 172.21.15.32:0/3368908214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae4145f30 msgr2=0x7f1ae4071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.964+0000 7f1aebe00640 1 --2- 172.21.15.32:0/3368908214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae4145f30 0x7f1ae4071df0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40702a0 tx=0x7f1ad801ed10 comp rx=0 tx=0).stop 2024-04-17T09:57:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.964+0000 7f1aebe00640 1 -- 172.21.15.32:0/3368908214 shutdown_connections 2024-04-17T09:57:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.964+0000 7f1aebe00640 1 --2- 172.21.15.32:0/3368908214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ae40cce70 0x7f1ae40cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.964+0000 7f1aebe00640 1 --2- 172.21.15.32:0/3368908214 >> v2:172.21.15.144:3300/0 conn(0x7f1ae415d8d0 0x7f1ae4072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.964+0000 7f1aebe00640 1 --2- 172.21.15.32:0/3368908214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae4145f30 0x7f1ae4071df0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.964+0000 7f1aebe00640 1 -- 172.21.15.32:0/3368908214 >> 172.21.15.32:0/3368908214 conn(0x7f1ae406ba30 msgr2=0x7f1ae406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.964+0000 7f1aebe00640 1 -- 172.21.15.32:0/3368908214 shutdown_connections 2024-04-17T09:57:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.964+0000 7f1aebe00640 1 -- 172.21.15.32:0/3368908214 wait complete. 2024-04-17T09:57:04.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1aebe00640 1 Processor -- start 2024-04-17T09:57:04.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1aebe00640 1 -- start start 2024-04-17T09:57:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1aebe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae40cce70 0x7f1ae415bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1aebe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ae4145f30 0x7f1ae40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1aebe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ae415d8d0 0x7f1ae4160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1aebe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae415cdf0 con 0x7f1ae40cce70 2024-04-17T09:57:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1aebe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae415cb90 con 0x7f1ae4145f30 2024-04-17T09:57:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1aebe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1ae415d0d0 con 0x7f1ae415d8d0 2024-04-17T09:57:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1ae9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae40cce70 0x7f1ae415bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1ae9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae40cce70 0x7f1ae415bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39230/0 (socket says 172.21.15.32:39230) 2024-04-17T09:57:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1ae9400640 1 -- 172.21.15.32:0/2615952875 learned_addr learned my addr 172.21.15.32:0/2615952875 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1ae8a00640 1 --2- 172.21.15.32:0/2615952875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ae4145f30 0x7f1ae40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1ae9e00640 1 --2- 172.21.15.32:0/2615952875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ae415d8d0 0x7f1ae4160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1ae9400640 1 -- 172.21.15.32:0/2615952875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ae415d8d0 msgr2=0x7f1ae4160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1ae9400640 1 --2- 172.21.15.32:0/2615952875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ae415d8d0 0x7f1ae4160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1ae9400640 1 -- 172.21.15.32:0/2615952875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ae4145f30 msgr2=0x7f1ae40b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1ae9400640 1 --2- 172.21.15.32:0/2615952875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ae4145f30 0x7f1ae40b4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1ae9400640 1 -- 172.21.15.32:0/2615952875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae415c4b0 con 0x7f1ae40cce70 2024-04-17T09:57:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1ae9400640 1 --2- 172.21.15.32:0/2615952875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae40cce70 0x7f1ae415bf70 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80b3040 tx=0x7f1ad801bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1ae1600640 1 -- 172.21.15.32:0/2615952875 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad80b5070 con 0x7f1ae40cce70 2024-04-17T09:57:04.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1ae1600640 1 -- 172.21.15.32:0/2615952875 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ad80b0ba0 con 0x7f1ae40cce70 2024-04-17T09:57:04.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1ae1600640 1 -- 172.21.15.32:0/2615952875 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad80bb410 con 0x7f1ae40cce70 2024-04-17T09:57:04.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1aebe00640 1 -- 172.21.15.32:0/2615952875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae40c9c80 con 0x7f1ae40cce70 2024-04-17T09:57:04.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.968+0000 7f1aebe00640 1 -- 172.21.15.32:0/2615952875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae40ca240 con 0x7f1ae40cce70 2024-04-17T09:57:04.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.972+0000 7f1aebe00640 1 -- 172.21.15.32:0/2615952875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae4068a90 con 0x7f1ae40cce70 2024-04-17T09:57:04.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.972+0000 7f1ae1600640 1 -- 172.21.15.32:0/2615952875 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1ad80c2060 con 0x7f1ae40cce70 2024-04-17T09:57:04.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.972+0000 7f1ae1600640 1 --2- 172.21.15.32:0/2615952875 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1ac4068380 0x7f1ac406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:04.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.972+0000 7f1ae1600640 1 -- 172.21.15.32:0/2615952875 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7f1ad81239f0 con 0x7f1ae40cce70 2024-04-17T09:57:04.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.972+0000 7f1ae8a00640 1 --2- 172.21.15.32:0/2615952875 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1ac4068380 0x7f1ac406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:04.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.972+0000 7f1ae8a00640 1 --2- 172.21.15.32:0/2615952875 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1ac4068380 0x7f1ac406a840 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40cb530 tx=0x7f1ad0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:04.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:04.976+0000 7f1ae1600640 1 -- 172.21.15.32:0/2615952875 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f1ad8018870 con 0x7f1ae40cce70 2024-04-17T09:57:05.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:05.200+0000 7f1aebe00640 1 -- 172.21.15.32:0/2615952875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f1ae4072f90 con 0x7f1ae40cce70 2024-04-17T09:57:06.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.112+0000 7f1ae1600640 1 -- 172.21.15.32:0/2615952875 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v209) ==== 169+0+0 (secure 0 0 0) 0x7f1ad80f42a0 con 0x7f1ae40cce70 2024-04-17T09:57:06.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.116+0000 7f1aebe00640 1 -- 172.21.15.32:0/2615952875 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1ac4068380 msgr2=0x7f1ac406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:06.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.116+0000 7f1aebe00640 1 --2- 172.21.15.32:0/2615952875 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1ac4068380 0x7f1ac406a840 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40cb530 tx=0x7f1ad0008040 comp rx=0 tx=0).stop 2024-04-17T09:57:06.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.116+0000 7f1aebe00640 1 -- 172.21.15.32:0/2615952875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae40cce70 msgr2=0x7f1ae415bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:06.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.116+0000 7f1aebe00640 1 --2- 172.21.15.32:0/2615952875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae40cce70 0x7f1ae415bf70 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80b3040 tx=0x7f1ad801bab0 comp rx=0 tx=0).stop 2024-04-17T09:57:06.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.116+0000 7f1aebe00640 1 -- 172.21.15.32:0/2615952875 shutdown_connections 2024-04-17T09:57:06.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.116+0000 7f1aebe00640 1 --2- 172.21.15.32:0/2615952875 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1ac4068380 0x7f1ac406a840 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:06.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.116+0000 7f1aebe00640 1 --2- 172.21.15.32:0/2615952875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ae415d8d0 0x7f1ae4160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:06.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.116+0000 7f1aebe00640 1 --2- 172.21.15.32:0/2615952875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ae4145f30 0x7f1ae40b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:06.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.116+0000 7f1aebe00640 1 --2- 172.21.15.32:0/2615952875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae40cce70 0x7f1ae415bf70 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:06.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.116+0000 7f1aebe00640 1 -- 172.21.15.32:0/2615952875 >> 172.21.15.32:0/2615952875 conn(0x7f1ae406ba30 msgr2=0x7f1ae415ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:06.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.116+0000 7f1aebe00640 1 -- 172.21.15.32:0/2615952875 shutdown_connections 2024-04-17T09:57:06.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.116+0000 7f1aebe00640 1 -- 172.21.15.32:0/2615952875 wait complete. 2024-04-17T09:57:06.140 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-17T09:57:06.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.288+0000 7fa148e00640 1 Processor -- start 2024-04-17T09:57:06.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.288+0000 7fa148e00640 1 -- start start 2024-04-17T09:57:06.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.292+0000 7fa148e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa14415b7d0 0x7fa14406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:06.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.292+0000 7fa148e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14406f7c0 0x7fa14406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:06.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.292+0000 7fa148e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa14406f270 0x7fa14406bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:06.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.292+0000 7fa148e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa1440cc940 con 0x7fa14406f270 2024-04-17T09:57:06.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.292+0000 7fa148e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa1440ccc20 con 0x7fa14415b7d0 2024-04-17T09:57:06.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.292+0000 7fa148e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa14406e0b0 con 0x7fa14406f7c0 2024-04-17T09:57:06.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.292+0000 7fa142800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa14406f270 0x7fa14406bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:06.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.292+0000 7fa142800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa14406f270 0x7fa14406bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39238/0 (socket says 172.21.15.32:39238) 2024-04-17T09:57:06.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.292+0000 7fa142800640 1 -- 172.21.15.32:0/2996254087 learned_addr learned my addr 172.21.15.32:0/2996254087 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:06.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.292+0000 7fa141e00640 1 --2- 172.21.15.32:0/2996254087 >> v2:172.21.15.144:3300/0 conn(0x7fa14415b7d0 0x7fa14406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:06.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.292+0000 7fa139400640 1 --2- 172.21.15.32:0/2996254087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14406f7c0 0x7fa14406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:06.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.292+0000 7fa142800640 1 -- 172.21.15.32:0/2996254087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14406f7c0 msgr2=0x7fa14406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:06.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.292+0000 7fa142800640 1 --2- 172.21.15.32:0/2996254087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14406f7c0 0x7fa14406ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:06.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.292+0000 7fa142800640 1 -- 172.21.15.32:0/2996254087 >> v2:172.21.15.144:3300/0 conn(0x7fa14415b7d0 msgr2=0x7fa14406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:06.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.292+0000 7fa142800640 1 --2- 172.21.15.32:0/2996254087 >> v2:172.21.15.144:3300/0 conn(0x7fa14415b7d0 0x7fa14406e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:06.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.292+0000 7fa142800640 1 -- 172.21.15.32:0/2996254087 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14406e390 con 0x7fa14406f270 2024-04-17T09:57:06.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.292+0000 7fa142800640 1 --2- 172.21.15.32:0/2996254087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa14406f270 0x7fa14406bf70 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fa1440d1530 tx=0x7fa13c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8034dfa69dbd28ae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:06.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa141400640 1 -- 172.21.15.32:0/2996254087 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0b2070 con 0x7fa14406f270 2024-04-17T09:57:06.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa141400640 1 -- 172.21.15.32:0/2996254087 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa13c0a9070 con 0x7fa14406f270 2024-04-17T09:57:06.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa141400640 1 -- 172.21.15.32:0/2996254087 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0ad880 con 0x7fa14406f270 2024-04-17T09:57:06.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa148e00640 1 -- 172.21.15.32:0/2996254087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa14406f270 msgr2=0x7fa14406bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:06.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa148e00640 1 --2- 172.21.15.32:0/2996254087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa14406f270 0x7fa14406bf70 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fa1440d1530 tx=0x7fa13c01c490 comp rx=0 tx=0).stop 2024-04-17T09:57:06.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa148e00640 1 -- 172.21.15.32:0/2996254087 shutdown_connections 2024-04-17T09:57:06.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa148e00640 1 --2- 172.21.15.32:0/2996254087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa14406f270 0x7fa14406bf70 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:06.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa148e00640 1 --2- 172.21.15.32:0/2996254087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14406f7c0 0x7fa14406ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:06.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa148e00640 1 --2- 172.21.15.32:0/2996254087 >> v2:172.21.15.144:3300/0 conn(0x7fa14415b7d0 0x7fa14406e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:06.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa148e00640 1 -- 172.21.15.32:0/2996254087 >> 172.21.15.32:0/2996254087 conn(0x7fa1440b2120 msgr2=0x7fa1440b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:06.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa148e00640 1 -- 172.21.15.32:0/2996254087 shutdown_connections 2024-04-17T09:57:06.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa148e00640 1 -- 172.21.15.32:0/2996254087 wait complete. 2024-04-17T09:57:06.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa148e00640 1 Processor -- start 2024-04-17T09:57:06.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa148e00640 1 -- start start 2024-04-17T09:57:06.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa148e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa14406f270 0x7fa14414ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:06.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa148e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa14406f7c0 0x7fa14414e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:06.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa148e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14415b7d0 0x7fa14415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:06.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa148e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa1440cee30 con 0x7fa14406f7c0 2024-04-17T09:57:06.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa148e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa1440cecb0 con 0x7fa14406f270 2024-04-17T09:57:06.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa148e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa1440b3420 con 0x7fa14415b7d0 2024-04-17T09:57:06.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa139400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa14406f7c0 0x7fa14414e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:06.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa139400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa14406f7c0 0x7fa14414e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39250/0 (socket says 172.21.15.32:39250) 2024-04-17T09:57:06.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa139400640 1 -- 172.21.15.32:0/970104882 learned_addr learned my addr 172.21.15.32:0/970104882 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:06.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa141e00640 1 --2- 172.21.15.32:0/970104882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa14406f270 0x7fa14414ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:06.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.296+0000 7fa142800640 1 --2- 172.21.15.32:0/970104882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14415b7d0 0x7fa14415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:06.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.300+0000 7fa139400640 1 -- 172.21.15.32:0/970104882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14415b7d0 msgr2=0x7fa14415a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:06.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.300+0000 7fa139400640 1 --2- 172.21.15.32:0/970104882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14415b7d0 0x7fa14415a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:06.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.300+0000 7fa139400640 1 -- 172.21.15.32:0/970104882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa14406f270 msgr2=0x7fa14414ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:06.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.300+0000 7fa139400640 1 --2- 172.21.15.32:0/970104882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa14406f270 0x7fa14414ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:06.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.300+0000 7fa139400640 1 -- 172.21.15.32:0/970104882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14414f370 con 0x7fa14406f7c0 2024-04-17T09:57:06.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.300+0000 7fa139400640 1 --2- 172.21.15.32:0/970104882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa14406f7c0 0x7fa14414e320 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa12c00d800 tx=0x7fa12c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:06.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.300+0000 7fa13aa00640 1 -- 172.21.15.32:0/970104882 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c018020 con 0x7fa14406f7c0 2024-04-17T09:57:06.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.300+0000 7fa13aa00640 1 -- 172.21.15.32:0/970104882 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa12c0096a0 con 0x7fa14406f7c0 2024-04-17T09:57:06.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.300+0000 7fa13aa00640 1 -- 172.21.15.32:0/970104882 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c009a50 con 0x7fa14406f7c0 2024-04-17T09:57:06.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.300+0000 7fa148e00640 1 -- 172.21.15.32:0/970104882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1440aee90 con 0x7fa14406f7c0 2024-04-17T09:57:06.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.300+0000 7fa148e00640 1 -- 172.21.15.32:0/970104882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1440af450 con 0x7fa14406f7c0 2024-04-17T09:57:06.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.300+0000 7fa148e00640 1 -- 172.21.15.32:0/970104882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa144068a90 con 0x7fa14406f7c0 2024-04-17T09:57:06.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.300+0000 7fa13aa00640 1 -- 172.21.15.32:0/970104882 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa12c01b020 con 0x7fa14406f7c0 2024-04-17T09:57:06.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.300+0000 7fa13aa00640 1 --2- 172.21.15.32:0/970104882 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa120068380 0x7fa12006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:06.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.300+0000 7fa13aa00640 1 -- 172.21.15.32:0/970104882 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7fa12c086af0 con 0x7fa14406f7c0 2024-04-17T09:57:06.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.300+0000 7fa141e00640 1 --2- 172.21.15.32:0/970104882 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa120068380 0x7fa12006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:06.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.304+0000 7fa141e00640 1 --2- 172.21.15.32:0/970104882 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa120068380 0x7fa12006a840 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa14415ac10 tx=0x7fa134009830 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.304+0000 7fa13aa00640 1 -- 172.21.15.32:0/970104882 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa12c022020 con 0x7fa14406f7c0 2024-04-17T09:57:06.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.472+0000 7fa148e00640 1 -- 172.21.15.32:0/970104882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa1440af8e0 con 0x7fa14406f7c0 2024-04-17T09:57:06.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.476+0000 7fa13aa00640 1 -- 172.21.15.32:0/970104882 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7fa12c056d50 con 0x7fa14406f7c0 2024-04-17T09:57:06.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.476+0000 7fa148e00640 1 -- 172.21.15.32:0/970104882 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa120068380 msgr2=0x7fa12006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:06.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.476+0000 7fa148e00640 1 --2- 172.21.15.32:0/970104882 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa120068380 0x7fa12006a840 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa14415ac10 tx=0x7fa134009830 comp rx=0 tx=0).stop 2024-04-17T09:57:06.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.476+0000 7fa148e00640 1 -- 172.21.15.32:0/970104882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa14406f7c0 msgr2=0x7fa14414e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:06.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.476+0000 7fa148e00640 1 --2- 172.21.15.32:0/970104882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa14406f7c0 0x7fa14414e320 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa12c00d800 tx=0x7fa12c00dcc0 comp rx=0 tx=0).stop 2024-04-17T09:57:06.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.476+0000 7fa148e00640 1 -- 172.21.15.32:0/970104882 shutdown_connections 2024-04-17T09:57:06.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.476+0000 7fa148e00640 1 --2- 172.21.15.32:0/970104882 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa120068380 0x7fa12006a840 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:06.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.476+0000 7fa148e00640 1 --2- 172.21.15.32:0/970104882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14415b7d0 0x7fa14415a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:06.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.476+0000 7fa148e00640 1 --2- 172.21.15.32:0/970104882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa14406f7c0 0x7fa14414e320 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:06.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.476+0000 7fa148e00640 1 --2- 172.21.15.32:0/970104882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa14406f270 0x7fa14414ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:06.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.476+0000 7fa148e00640 1 -- 172.21.15.32:0/970104882 >> 172.21.15.32:0/970104882 conn(0x7fa1440b2120 msgr2=0x7fa144071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:06.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.476+0000 7fa148e00640 1 -- 172.21.15.32:0/970104882 shutdown_connections 2024-04-17T09:57:06.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.476+0000 7fa148e00640 1 -- 172.21.15.32:0/970104882 wait complete. 2024-04-17T09:57:06.498 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T09:57:06.498 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T09:57:06.618 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T09:57:06.689 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T09:57:06.689 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:57:06.689 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:57:06.698 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T09:57:06.698 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:57:06.698 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:57:06.709 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T09:57:06.709 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:57:06.709 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:57:06.749 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T09:57:06.750 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:57:06.750 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:57:06.759 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T09:57:06.759 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-17T09:57:06.759 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:57:06.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.932+0000 7f83dbe00640 1 Processor -- start 2024-04-17T09:57:06.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.932+0000 7f83dbe00640 1 -- start start 2024-04-17T09:57:06.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.932+0000 7f83dbe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83dc0cf5b0 0x7f83dc06dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:06.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.932+0000 7f83dbe00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f83dc06f250 0x7f83dc0aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:06.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.932+0000 7f83dbe00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83dc06e1b0 con 0x7f83dc0cf990 2024-04-17T09:57:06.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.932+0000 7f83dbe00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f83dc06edc0 con 0x7f83dc06f250 2024-04-17T09:57:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.932+0000 7f83dbe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f83dc06e490 con 0x7f83dc0cf5b0 2024-04-17T09:57:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.932+0000 7f83daa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83dc0cf5b0 0x7f83dc06dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.932+0000 7f83da000640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f83dc0cf990 0x7f83dc15d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:49988/0 (socket says 172.21.15.32:49988) 2024-04-17T09:57:06.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.932+0000 7f83da000640 1 -- 172.21.15.32:0/3402268195 learned_addr learned my addr 172.21.15.32:0/3402268195 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:06.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.932+0000 7f83db400640 1 --2- 172.21.15.32:0/3402268195 >> v2:172.21.15.144:3300/0 conn(0x7f83dc06f250 0x7f83dc0aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:06.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.932+0000 7f83d9600640 1 -- 172.21.15.32:0/3402268195 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3723649796 0 0) 0x7f83dc06e1b0 con 0x7f83dc0cf990 2024-04-17T09:57:06.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.932+0000 7f83daa00640 1 -- 172.21.15.32:0/3402268195 >> v2:172.21.15.144:3300/0 conn(0x7f83dc06f250 msgr2=0x7f83dc0aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:06.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.932+0000 7f83daa00640 1 --2- 172.21.15.32:0/3402268195 >> v2:172.21.15.144:3300/0 conn(0x7f83dc06f250 0x7f83dc0aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:06.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.932+0000 7f83daa00640 1 -- 172.21.15.32:0/3402268195 >> v1:172.21.15.144:6789/0 conn(0x7f83dc0cf990 legacy=0x7f83dc15d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:06.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.932+0000 7f83daa00640 1 -- 172.21.15.32:0/3402268195 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83dc06e770 con 0x7f83dc0cf5b0 2024-04-17T09:57:06.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.932+0000 7f83db400640 1 --2- 172.21.15.32:0/3402268195 >> v2:172.21.15.144:3300/0 conn(0x7f83dc06f250 0x7f83dc0aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:57:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.936+0000 7f83daa00640 1 --2- 172.21.15.32:0/3402268195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83dc0cf5b0 0x7f83dc06dbd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0097c0 tx=0x7f83cc01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18deaef5a3e6d046 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.936+0000 7f83d9600640 1 -- 172.21.15.32:0/3402268195 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc0aa5f0 con 0x7f83dc0cf5b0 2024-04-17T09:57:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.936+0000 7f83d9600640 1 -- 172.21.15.32:0/3402268195 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83cc0af070 con 0x7f83dc0cf5b0 2024-04-17T09:57:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.936+0000 7f83d9600640 1 -- 172.21.15.32:0/3402268195 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc0aa9b0 con 0x7f83dc0cf5b0 2024-04-17T09:57:06.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.936+0000 7f83dbe00640 1 -- 172.21.15.32:0/3402268195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83dc0cf5b0 msgr2=0x7f83dc06dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:06.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.936+0000 7f83dbe00640 1 --2- 172.21.15.32:0/3402268195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83dc0cf5b0 0x7f83dc06dbd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0097c0 tx=0x7f83cc01c470 comp rx=0 tx=0).stop 2024-04-17T09:57:06.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.936+0000 7f83dbe00640 1 -- 172.21.15.32:0/3402268195 shutdown_connections 2024-04-17T09:57:06.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.936+0000 7f83dbe00640 1 --2- 172.21.15.32:0/3402268195 >> v2:172.21.15.144:3300/0 conn(0x7f83dc06f250 0x7f83dc0aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:06.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.936+0000 7f83dbe00640 1 --2- 172.21.15.32:0/3402268195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83dc0cf5b0 0x7f83dc06dbd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:06.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.936+0000 7f83dbe00640 1 -- 172.21.15.32:0/3402268195 >> 172.21.15.32:0/3402268195 conn(0x7f83dc0d0180 msgr2=0x7f83dc0d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:06.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.936+0000 7f83dbe00640 1 -- 172.21.15.32:0/3402268195 shutdown_connections 2024-04-17T09:57:06.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.936+0000 7f83dbe00640 1 -- 172.21.15.32:0/3402268195 wait complete. 2024-04-17T09:57:06.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.936+0000 7f83dbe00640 1 Processor -- start 2024-04-17T09:57:06.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83dbe00640 1 -- start start 2024-04-17T09:57:06.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83dbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83dc06f250 0x7f83dc1563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83dbe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83dc0cf5b0 0x7f83dc143290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83dbe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83dc0cf990 0x7f83dc0998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83dbe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc15e6a0 con 0x7f83dc06f250 2024-04-17T09:57:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83dbe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc15e520 con 0x7f83dc0cf990 2024-04-17T09:57:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83dbe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f83dc15e820 con 0x7f83dc0cf5b0 2024-04-17T09:57:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83daa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83dc06f250 0x7f83dc1563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83daa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83dc06f250 0x7f83dc1563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39252/0 (socket says 172.21.15.32:39252) 2024-04-17T09:57:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83daa00640 1 -- 172.21.15.32:0/3456643955 learned_addr learned my addr 172.21.15.32:0/3456643955 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83da000640 1 --2- 172.21.15.32:0/3456643955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83dc0cf5b0 0x7f83dc143290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83db400640 1 --2- 172.21.15.32:0/3456643955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83dc0cf990 0x7f83dc0998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83daa00640 1 -- 172.21.15.32:0/3456643955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83dc0cf5b0 msgr2=0x7f83dc143290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83daa00640 1 --2- 172.21.15.32:0/3456643955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83dc0cf5b0 0x7f83dc143290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83daa00640 1 -- 172.21.15.32:0/3456643955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83dc0cf990 msgr2=0x7f83dc0998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83daa00640 1 --2- 172.21.15.32:0/3456643955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83dc0cf990 0x7f83dc0998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83daa00640 1 -- 172.21.15.32:0/3456643955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83dc143a00 con 0x7f83dc06f250 2024-04-17T09:57:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83daa00640 1 --2- 172.21.15.32:0/3456643955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83dc06f250 0x7f83dc1563b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0ac040 tx=0x7f83cc01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83d2a00640 1 -- 172.21.15.32:0/3456643955 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc006e20 con 0x7f83dc06f250 2024-04-17T09:57:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83d2a00640 1 -- 172.21.15.32:0/3456643955 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83cc004220 con 0x7f83dc06f250 2024-04-17T09:57:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83dbe00640 1 -- 172.21.15.32:0/3456643955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83dc143ce0 con 0x7f83dc06f250 2024-04-17T09:57:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83d2a00640 1 -- 172.21.15.32:0/3456643955 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc0aae60 con 0x7f83dc06f250 2024-04-17T09:57:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83dbe00640 1 -- 172.21.15.32:0/3456643955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83dc099de0 con 0x7f83dc06f250 2024-04-17T09:57:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.940+0000 7f83dbe00640 1 -- 172.21.15.32:0/3456643955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83dc04fd00 con 0x7f83dc06f250 2024-04-17T09:57:06.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.944+0000 7f83d2a00640 1 -- 172.21.15.32:0/3456643955 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f83cc003620 con 0x7f83dc06f250 2024-04-17T09:57:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.944+0000 7f83d2a00640 1 --2- 172.21.15.32:0/3456643955 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f83bc0683b0 0x7f83bc06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.944+0000 7f83d2a00640 1 -- 172.21.15.32:0/3456643955 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7f83cc120b50 con 0x7f83dc06f250 2024-04-17T09:57:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.944+0000 7f83da000640 1 --2- 172.21.15.32:0/3456643955 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f83bc0683b0 0x7f83bc06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:06.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.944+0000 7f83da000640 1 --2- 172.21.15.32:0/3456643955 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f83bc0683b0 0x7f83bc06a870 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0cf320 tx=0x7f83c8007d60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:06.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:06.948+0000 7f83d2a00640 1 -- 172.21.15.32:0/3456643955 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f83cc016090 con 0x7f83dc06f250 2024-04-17T09:57:07.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.148+0000 7f83dbe00640 1 -- 172.21.15.32:0/3456643955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83dc06fc10 con 0x7f83dc06f250 2024-04-17T09:57:07.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.148+0000 7f83d2a00640 1 -- 172.21.15.32:0/3456643955 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+3875 (secure 0 0 0) 0x7f83cc0f0db0 con 0x7f83dc06f250 2024-04-17T09:57:07.148 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:57:07.149 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":36,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5495,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/1236891874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1236891874},{"type":"v1","addr":"172.21.15.144:6834","nonce":1236891874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5498,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3538881342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3538881342},{"type":"v1","addr":"172.21.15.144:6835","nonce":3538881342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5819,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2482667348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2482667348},{"type":"v1","addr":"172.21.15.32:6837","nonce":2482667348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6201,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3549801633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3549801633},{"type":"v1","addr":"172.21.15.32:6836","nonce":3549801633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[]} 2024-04-17T09:57:07.149 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 36 2024-04-17T09:57:07.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.148+0000 7f83dbe00640 1 -- 172.21.15.32:0/3456643955 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f83bc0683b0 msgr2=0x7f83bc06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:07.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.148+0000 7f83dbe00640 1 --2- 172.21.15.32:0/3456643955 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f83bc0683b0 0x7f83bc06a870 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0cf320 tx=0x7f83c8007d60 comp rx=0 tx=0).stop 2024-04-17T09:57:07.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.148+0000 7f83dbe00640 1 -- 172.21.15.32:0/3456643955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83dc06f250 msgr2=0x7f83dc1563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:07.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.148+0000 7f83dbe00640 1 --2- 172.21.15.32:0/3456643955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83dc06f250 0x7f83dc1563b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0ac040 tx=0x7f83cc01acb0 comp rx=0 tx=0).stop 2024-04-17T09:57:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.148+0000 7f83dbe00640 1 -- 172.21.15.32:0/3456643955 shutdown_connections 2024-04-17T09:57:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.148+0000 7f83dbe00640 1 --2- 172.21.15.32:0/3456643955 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f83bc0683b0 0x7f83bc06a870 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.148+0000 7f83dbe00640 1 --2- 172.21.15.32:0/3456643955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83dc0cf5b0 0x7f83dc143290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.148+0000 7f83dbe00640 1 --2- 172.21.15.32:0/3456643955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83dc0cf990 0x7f83dc0998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.148+0000 7f83dbe00640 1 --2- 172.21.15.32:0/3456643955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83dc06f250 0x7f83dc1563b0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.148+0000 7f83dbe00640 1 -- 172.21.15.32:0/3456643955 >> 172.21.15.32:0/3456643955 conn(0x7f83dc0d0180 msgr2=0x7f83dc06c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.148+0000 7f83dbe00640 1 -- 172.21.15.32:0/3456643955 shutdown_connections 2024-04-17T09:57:07.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.148+0000 7f83dbe00640 1 -- 172.21.15.32:0/3456643955 wait complete. 2024-04-17T09:57:07.171 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-17T09:57:07.171 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-17T09:57:07.171 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-17T09:57:07.171 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:57:07.171 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-17T09:57:07.171 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-17T09:57:07.172 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-17T09:57:07.172 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:57:07.172 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-17T09:57:07.172 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-17T09:57:07.172 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-17T09:57:07.173 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:57:07.173 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-17T09:57:07.173 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-17T09:57:07.173 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-17T09:57:07.174 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:57:07.183 INFO:tasks.ceph.mds.a:Stopped 2024-04-17T09:57:07.183 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-17T09:57:07.184 INFO:tasks.ceph.mds.c:Stopped 2024-04-17T09:57:07.184 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-17T09:57:07.185 INFO:tasks.ceph.mds.b:Stopped 2024-04-17T09:57:07.185 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-17T09:57:07.187 INFO:tasks.ceph.mds.d:Stopped 2024-04-17T09:57:07.188 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-17T09:57:07.232 INFO:tasks.ceph.mds.b:Started 2024-04-17T09:57:07.232 DEBUG:teuthology.parallel:result is None 2024-04-17T09:57:07.233 INFO:tasks.ceph.mds.a:Started 2024-04-17T09:57:07.233 DEBUG:teuthology.parallel:result is None 2024-04-17T09:57:07.234 INFO:tasks.ceph.mds.d:Started 2024-04-17T09:57:07.235 DEBUG:teuthology.parallel:result is None 2024-04-17T09:57:07.235 INFO:tasks.ceph.mds.c:Started 2024-04-17T09:57:07.235 DEBUG:teuthology.parallel:result is None 2024-04-17T09:57:07.236 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-17T09:57:07.321 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-17T09:57:07.322 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2024-04-17T09:57:07.322 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2024-04-17T09:57:07.323 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-17T09:57:07.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.380+0000 7ff986c00640 1 Processor -- start 2024-04-17T09:57:07.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.380+0000 7ff986c00640 1 -- start start 2024-04-17T09:57:07.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.384+0000 7ff986c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff98015c820 0x7ff9800d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:07.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.384+0000 7ff986c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff9800cdbd0 0x7ff98009b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:07.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.384+0000 7ff986c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff98015cc00 con 0x7ff98009bc60 2024-04-17T09:57:07.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.384+0000 7ff986c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff98009a9f0 con 0x7ff9800cdbd0 2024-04-17T09:57:07.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.384+0000 7ff986c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff9800ce340 con 0x7ff98015c820 2024-04-17T09:57:07.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.384+0000 7ff97fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff98015c820 0x7ff9800d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:07.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.384+0000 7ff97f400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff9800cdbd0 0x7ff98009b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:07.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.384+0000 7ff984c00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7ff98009bc60 0x7ff9800d0b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:50014/0 (socket says 172.21.15.32:50014) 2024-04-17T09:57:07.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.384+0000 7ff984c00640 1 -- 172.21.15.32:0/3197540262 learned_addr learned my addr 172.21.15.32:0/3197540262 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:07.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.384+0000 7ff97fe00640 1 -- 172.21.15.32:0/3197540262 >> v2:172.21.15.144:3300/0 conn(0x7ff9800cdbd0 msgr2=0x7ff98009b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:07.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.384+0000 7ff97fe00640 1 --2- 172.21.15.32:0/3197540262 >> v2:172.21.15.144:3300/0 conn(0x7ff9800cdbd0 0x7ff98009b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:07.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.384+0000 7ff97fe00640 1 -- 172.21.15.32:0/3197540262 >> v1:172.21.15.144:6789/0 conn(0x7ff98009bc60 legacy=0x7ff9800d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:07.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.384+0000 7ff97fe00640 1 -- 172.21.15.32:0/3197540262 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff98009ac40 con 0x7ff98015c820 2024-04-17T09:57:07.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.384+0000 7ff97f400640 1 --2- 172.21.15.32:0/3197540262 >> v2:172.21.15.144:3300/0 conn(0x7ff9800cdbd0 0x7ff98009b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:57:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.388+0000 7ff97fe00640 1 --2- 172.21.15.32:0/3197540262 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff98015c820 0x7ff9800d0540 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff9700097c0 tx=0x7ff97001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aede9104708df95d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.388+0000 7ff97ea00640 1 -- 172.21.15.32:0/3197540262 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9700a7070 con 0x7ff98015c820 2024-04-17T09:57:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.388+0000 7ff97ea00640 1 -- 172.21.15.32:0/3197540262 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9700b0070 con 0x7ff98015c820 2024-04-17T09:57:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.388+0000 7ff97ea00640 1 -- 172.21.15.32:0/3197540262 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9700ab890 con 0x7ff98015c820 2024-04-17T09:57:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.388+0000 7ff986c00640 1 -- 172.21.15.32:0/3197540262 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff98015c820 msgr2=0x7ff9800d0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.388+0000 7ff986c00640 1 --2- 172.21.15.32:0/3197540262 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff98015c820 0x7ff9800d0540 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff9700097c0 tx=0x7ff97001c470 comp rx=0 tx=0).stop 2024-04-17T09:57:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.388+0000 7ff986c00640 1 -- 172.21.15.32:0/3197540262 shutdown_connections 2024-04-17T09:57:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.388+0000 7ff986c00640 1 --2- 172.21.15.32:0/3197540262 >> v2:172.21.15.144:3300/0 conn(0x7ff9800cdbd0 0x7ff98009b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.388+0000 7ff986c00640 1 --2- 172.21.15.32:0/3197540262 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff98015c820 0x7ff9800d0540 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.388+0000 7ff986c00640 1 -- 172.21.15.32:0/3197540262 >> 172.21.15.32:0/3197540262 conn(0x7ff9800b40a0 msgr2=0x7ff9800b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.388+0000 7ff986c00640 1 -- 172.21.15.32:0/3197540262 shutdown_connections 2024-04-17T09:57:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.388+0000 7ff986c00640 1 -- 172.21.15.32:0/3197540262 wait complete. 2024-04-17T09:57:07.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.388+0000 7ff986c00640 1 Processor -- start 2024-04-17T09:57:07.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.388+0000 7ff986c00640 1 -- start start 2024-04-17T09:57:07.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.388+0000 7ff986c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff98009bc60 0x7ff98015aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:07.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.388+0000 7ff986c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9800cdbd0 0x7ff98014ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:07.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.388+0000 7ff986c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff98015c820 0x7ff980157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:07.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.388+0000 7ff986c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff9800c6a90 con 0x7ff9800cdbd0 2024-04-17T09:57:07.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.388+0000 7ff986c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff9800c6910 con 0x7ff98009bc60 2024-04-17T09:57:07.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.388+0000 7ff986c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff9800d12a0 con 0x7ff98015c820 2024-04-17T09:57:07.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.392+0000 7ff97f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9800cdbd0 0x7ff98014ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:07.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.392+0000 7ff97f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9800cdbd0 0x7ff98014ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39316/0 (socket says 172.21.15.32:39316) 2024-04-17T09:57:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.392+0000 7ff97fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff98009bc60 0x7ff98015aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.392+0000 7ff97f400640 1 -- 172.21.15.32:0/3363004769 learned_addr learned my addr 172.21.15.32:0/3363004769 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.392+0000 7ff984c00640 1 --2- 172.21.15.32:0/3363004769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff98015c820 0x7ff980157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.392+0000 7ff97f400640 1 -- 172.21.15.32:0/3363004769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff98015c820 msgr2=0x7ff980157c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.392+0000 7ff97f400640 1 --2- 172.21.15.32:0/3363004769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff98015c820 0x7ff980157c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.392+0000 7ff97f400640 1 -- 172.21.15.32:0/3363004769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff98009bc60 msgr2=0x7ff98015aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.392+0000 7ff97f400640 1 --2- 172.21.15.32:0/3363004769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff98009bc60 0x7ff98015aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.392+0000 7ff97f400640 1 -- 172.21.15.32:0/3363004769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff98014bce0 con 0x7ff9800cdbd0 2024-04-17T09:57:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.392+0000 7ff97fe00640 1 --2- 172.21.15.32:0/3363004769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff98009bc60 0x7ff98015aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:57:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.392+0000 7ff97f400640 1 --2- 172.21.15.32:0/3363004769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9800cdbd0 0x7ff98014ace0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00e980 tx=0x7ff96c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.392+0000 7ff977e00640 1 -- 172.21.15.32:0/3363004769 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c017070 con 0x7ff9800cdbd0 2024-04-17T09:57:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.392+0000 7ff977e00640 1 -- 172.21.15.32:0/3363004769 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff96c00ca00 con 0x7ff9800cdbd0 2024-04-17T09:57:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.392+0000 7ff977e00640 1 -- 172.21.15.32:0/3363004769 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c00cdd0 con 0x7ff9800cdbd0 2024-04-17T09:57:07.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.392+0000 7ff986c00640 1 -- 172.21.15.32:0/3363004769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff980156c30 con 0x7ff9800cdbd0 2024-04-17T09:57:07.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.392+0000 7ff986c00640 1 -- 172.21.15.32:0/3363004769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff980156ed0 con 0x7ff9800cdbd0 2024-04-17T09:57:07.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.392+0000 7ff986c00640 1 -- 172.21.15.32:0/3363004769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff980068a90 con 0x7ff9800cdbd0 2024-04-17T09:57:07.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.392+0000 7ff977e00640 1 -- 172.21.15.32:0/3363004769 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff96c01a070 con 0x7ff9800cdbd0 2024-04-17T09:57:07.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.396+0000 7ff977e00640 1 --2- 172.21.15.32:0/3363004769 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff960068380 0x7ff96006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.396+0000 7ff977e00640 1 -- 172.21.15.32:0/3363004769 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7ff96c086510 con 0x7ff9800cdbd0 2024-04-17T09:57:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.396+0000 7ff97fe00640 1 --2- 172.21.15.32:0/3363004769 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff960068380 0x7ff96006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.396+0000 7ff97fe00640 1 --2- 172.21.15.32:0/3363004769 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff960068380 0x7ff96006a840 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff9700af000 tx=0x7ff9700b6040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.400+0000 7ff977e00640 1 -- 172.21.15.32:0/3363004769 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff98014bce0 con 0x7ff9800cdbd0 2024-04-17T09:57:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:07.576+0000 7ff986c00640 1 -- 172.21.15.32:0/3363004769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff9800b4c90 con 0x7ff9800cdbd0 2024-04-17T09:57:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.128+0000 7ff977e00640 1 -- 172.21.15.32:0/3363004769 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v62) ==== 97+0+0 (secure 0 0 0) 0x7ff96c056770 con 0x7ff9800cdbd0 2024-04-17T09:57:08.128 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2024-04-17T09:57:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.128+0000 7ff986c00640 1 -- 172.21.15.32:0/3363004769 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff960068380 msgr2=0x7ff96006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.128+0000 7ff986c00640 1 --2- 172.21.15.32:0/3363004769 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff960068380 0x7ff96006a840 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff9700af000 tx=0x7ff9700b6040 comp rx=0 tx=0).stop 2024-04-17T09:57:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.128+0000 7ff986c00640 1 -- 172.21.15.32:0/3363004769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9800cdbd0 msgr2=0x7ff98014ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.128+0000 7ff986c00640 1 --2- 172.21.15.32:0/3363004769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9800cdbd0 0x7ff98014ace0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00e980 tx=0x7ff96c00ee40 comp rx=0 tx=0).stop 2024-04-17T09:57:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.128+0000 7ff986c00640 1 -- 172.21.15.32:0/3363004769 shutdown_connections 2024-04-17T09:57:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.128+0000 7ff986c00640 1 --2- 172.21.15.32:0/3363004769 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff960068380 0x7ff96006a840 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.128+0000 7ff986c00640 1 --2- 172.21.15.32:0/3363004769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff98015c820 0x7ff980157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.128+0000 7ff986c00640 1 --2- 172.21.15.32:0/3363004769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9800cdbd0 0x7ff98014ace0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.128+0000 7ff986c00640 1 --2- 172.21.15.32:0/3363004769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff98009bc60 0x7ff98015aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.128+0000 7ff986c00640 1 -- 172.21.15.32:0/3363004769 >> 172.21.15.32:0/3363004769 conn(0x7ff9800b40a0 msgr2=0x7ff980074c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.128+0000 7ff986c00640 1 -- 172.21.15.32:0/3363004769 shutdown_connections 2024-04-17T09:57:08.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.128+0000 7ff986c00640 1 -- 172.21.15.32:0/3363004769 wait complete. 2024-04-17T09:57:08.155 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-17T09:57:08.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.296+0000 7fbdc6a00640 1 Processor -- start 2024-04-17T09:57:08.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.296+0000 7fbdc6a00640 1 -- start start 2024-04-17T09:57:08.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.296+0000 7fbdc6a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdc015ed80 0x7fbdc009b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:08.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.296+0000 7fbdc6a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdc00d09a0 con 0x7fbdc009ba60 2024-04-17T09:57:08.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.296+0000 7fbdc6a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdc00d0c80 con 0x7fbdc015e9a0 2024-04-17T09:57:08.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.296+0000 7fbdc6a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbdc00d0f60 con 0x7fbdc015ed80 2024-04-17T09:57:08.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.300+0000 7fbdc4a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbdc009ba60 0x7fbdc006d450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55236/0 (socket says 172.21.15.32:55236) 2024-04-17T09:57:08.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.300+0000 7fbdc4a00640 1 -- 172.21.15.32:0/3803851895 learned_addr learned my addr 172.21.15.32:0/3803851895 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:08.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.300+0000 7fbdb7e00640 1 --2- 172.21.15.32:0/3803851895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdc015ed80 0x7fbdc009b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:08.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.300+0000 7fbdbf400640 1 -- 172.21.15.32:0/3803851895 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3837006399 0 0) 0x7fbdc00d09a0 con 0x7fbdc009ba60 2024-04-17T09:57:08.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.300+0000 7fbdbf400640 1 -- 172.21.15.32:0/3803851895 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbda4003530 con 0x7fbdc009ba60 2024-04-17T09:57:08.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.300+0000 7fbdbf400640 1 -- 172.21.15.32:0/3803851895 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1019539386 0 0) 0x7fbdc00d0c80 con 0x7fbdc015e9a0 2024-04-17T09:57:08.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.300+0000 7fbdb7e00640 1 -- 172.21.15.32:0/3803851895 >> v1:172.21.15.144:6789/0 conn(0x7fbdc015e9a0 legacy=0x7fbdc00d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:08.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.300+0000 7fbdb7e00640 1 -- 172.21.15.32:0/3803851895 >> v1:172.21.15.32:6789/0 conn(0x7fbdc009ba60 legacy=0x7fbdc006d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:08.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.300+0000 7fbdb7e00640 1 -- 172.21.15.32:0/3803851895 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc00d1240 con 0x7fbdc015ed80 2024-04-17T09:57:08.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.300+0000 7fbdb7e00640 1 --2- 172.21.15.32:0/3803851895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdc015ed80 0x7fbdc009b520 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbda80097f0 tx=0x7fbda801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1fbdb1f13f19bd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:08.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.304+0000 7fbdbf400640 1 -- 172.21.15.32:0/3803851895 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda80ab5f0 con 0x7fbdc015ed80 2024-04-17T09:57:08.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.304+0000 7fbdbf400640 1 -- 172.21.15.32:0/3803851895 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbda80b0070 con 0x7fbdc015ed80 2024-04-17T09:57:08.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.304+0000 7fbdbf400640 1 -- 172.21.15.32:0/3803851895 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda80ab9b0 con 0x7fbdc015ed80 2024-04-17T09:57:08.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.304+0000 7fbdc6a00640 1 -- 172.21.15.32:0/3803851895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdc015ed80 msgr2=0x7fbdc009b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:08.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.304+0000 7fbdc6a00640 1 --2- 172.21.15.32:0/3803851895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdc015ed80 0x7fbdc009b520 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbda80097f0 tx=0x7fbda801c4e0 comp rx=0 tx=0).stop 2024-04-17T09:57:08.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.304+0000 7fbdc6a00640 1 -- 172.21.15.32:0/3803851895 shutdown_connections 2024-04-17T09:57:08.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.304+0000 7fbdc6a00640 1 --2- 172.21.15.32:0/3803851895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdc015ed80 0x7fbdc009b520 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:08.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.304+0000 7fbdc6a00640 1 -- 172.21.15.32:0/3803851895 >> 172.21.15.32:0/3803851895 conn(0x7fbdc00b3fd0 msgr2=0x7fbdc00b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:08.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.304+0000 7fbdc6a00640 1 -- 172.21.15.32:0/3803851895 shutdown_connections 2024-04-17T09:57:08.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.304+0000 7fbdc6a00640 1 -- 172.21.15.32:0/3803851895 wait complete. 2024-04-17T09:57:08.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.304+0000 7fbdc6a00640 1 Processor -- start 2024-04-17T09:57:08.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.304+0000 7fbdc6a00640 1 -- start start 2024-04-17T09:57:08.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.304+0000 7fbdc6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc009ba60 0x7fbdc00c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:08.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.304+0000 7fbdc6a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdc015e9a0 0x7fbdc0153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:08.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.304+0000 7fbdc6a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdc015ed80 0x7fbdc01541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:08.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.304+0000 7fbdc6a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc00cd340 con 0x7fbdc009ba60 2024-04-17T09:57:08.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.304+0000 7fbdc6a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc006dd70 con 0x7fbdc015ed80 2024-04-17T09:57:08.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.304+0000 7fbdc6a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbdc00cd4c0 con 0x7fbdc015e9a0 2024-04-17T09:57:08.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.304+0000 7fbdbfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc009ba60 0x7fbdc00c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:08.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.304+0000 7fbdbfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc009ba60 0x7fbdc00c9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39332/0 (socket says 172.21.15.32:39332) 2024-04-17T09:57:08.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.304+0000 7fbdbfe00640 1 -- 172.21.15.32:0/3465990398 learned_addr learned my addr 172.21.15.32:0/3465990398 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:08.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.304+0000 7fbdc4a00640 1 --2- 172.21.15.32:0/3465990398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdc015ed80 0x7fbdc01541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:08.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.304+0000 7fbdb7e00640 1 --2- 172.21.15.32:0/3465990398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdc015e9a0 0x7fbdc0153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:08.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.308+0000 7fbdbfe00640 1 -- 172.21.15.32:0/3465990398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdc015e9a0 msgr2=0x7fbdc0153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:08.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.308+0000 7fbdbfe00640 1 --2- 172.21.15.32:0/3465990398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdc015e9a0 0x7fbdc0153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:08.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.308+0000 7fbdbfe00640 1 -- 172.21.15.32:0/3465990398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdc015ed80 msgr2=0x7fbdc01541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:08.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.308+0000 7fbdbfe00640 1 --2- 172.21.15.32:0/3465990398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdc015ed80 0x7fbdc01541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:08.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.308+0000 7fbdbfe00640 1 -- 172.21.15.32:0/3465990398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc0158560 con 0x7fbdc009ba60 2024-04-17T09:57:08.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.308+0000 7fbdc4a00640 1 --2- 172.21.15.32:0/3465990398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdc015ed80 0x7fbdc01541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:57:08.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.308+0000 7fbdbfe00640 1 --2- 172.21.15.32:0/3465990398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc009ba60 0x7fbdc00c9b10 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000c480 tx=0x7fbdb000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:08.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.308+0000 7fbdbcc00640 1 -- 172.21.15.32:0/3465990398 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0005480 con 0x7fbdc009ba60 2024-04-17T09:57:08.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.308+0000 7fbdbcc00640 1 -- 172.21.15.32:0/3465990398 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdb0003600 con 0x7fbdc009ba60 2024-04-17T09:57:08.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.308+0000 7fbdc6a00640 1 -- 172.21.15.32:0/3465990398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc0158840 con 0x7fbdc009ba60 2024-04-17T09:57:08.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.308+0000 7fbdbcc00640 1 -- 172.21.15.32:0/3465990398 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb00039f0 con 0x7fbdc009ba60 2024-04-17T09:57:08.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.308+0000 7fbdc6a00640 1 -- 172.21.15.32:0/3465990398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc0158e00 con 0x7fbdc009ba60 2024-04-17T09:57:08.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.308+0000 7fbdc6a00640 1 -- 172.21.15.32:0/3465990398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdc0068a90 con 0x7fbdc009ba60 2024-04-17T09:57:08.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.308+0000 7fbdbcc00640 1 -- 172.21.15.32:0/3465990398 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbdb0003b90 con 0x7fbdc009ba60 2024-04-17T09:57:08.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.308+0000 7fbdbcc00640 1 --2- 172.21.15.32:0/3465990398 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbd9c0683b0 0x7fbd9c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:08.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.308+0000 7fbdbcc00640 1 -- 172.21.15.32:0/3465990398 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4857+0+0 (secure 0 0 0) 0x7fbdb0086380 con 0x7fbdc009ba60 2024-04-17T09:57:08.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.308+0000 7fbdb7e00640 1 --2- 172.21.15.32:0/3465990398 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbd9c0683b0 0x7fbd9c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:08.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.312+0000 7fbdb7e00640 1 --2- 172.21.15.32:0/3465990398 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbd9c0683b0 0x7fbd9c06a870 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbda80af000 tx=0x7fbda80a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:08.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.312+0000 7fbdbcc00640 1 -- 172.21.15.32:0/3465990398 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fbdb0056c80 con 0x7fbdc009ba60 2024-04-17T09:57:08.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.536+0000 7fbdc6a00640 1 -- 172.21.15.32:0/3465990398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fbdc0071a80 con 0x7fbdc009ba60 2024-04-17T09:57:08.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.536+0000 7fbdbcc00640 1 -- 172.21.15.32:0/3465990398 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7fbdb0056580 con 0x7fbdc009ba60 2024-04-17T09:57:08.537 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:57:08.537 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:57:08.537 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2024-04-17T09:57:08.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:57:08.537 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2024-04-17T09:57:08.538 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mzM1KExAAn0WH+Gf6gGV/dJonXzAgPQ==", 2024-04-17T09:57:08.538 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:57:08.538 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:57:08.538 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:57:08.538 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:57:08.538 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:57:08.538 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:08.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:08.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:57:08.538 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2024-04-17T09:57:08.538 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mQnJ4HxAAT0rFHhdAj7PMxZGiqEyWbg==", 2024-04-17T09:57:08.538 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:57:08.538 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:57:08.539 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:57:08.539 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:57:08.539 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:57:08.539 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:08.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:08.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:57:08.539 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2024-04-17T09:57:08.539 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mWWi5GhAA3RAdRxK6GRwuVCH4DHoMeA==", 2024-04-17T09:57:08.539 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:57:08.539 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:57:08.539 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:57:08.539 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:57:08.539 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:57:08.539 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:08.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:08.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:57:08.540 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2024-04-17T09:57:08.540 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9m9TMAJxAAbmUp05GHRE1AqCmq83scVQ==", 2024-04-17T09:57:08.540 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:57:08.540 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:57:08.540 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:57:08.540 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:57:08.540 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:57:08.540 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:08.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:08.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:57:08.541 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2024-04-17T09:57:08.541 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDRmx9mnPwyDBAAxD10jhxptVH5zhupengElg==", 2024-04-17T09:57:08.541 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:57:08.541 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:57:08.541 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:57:08.541 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:57:08.541 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:08.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:08.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:57:08.541 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2024-04-17T09:57:08.541 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUmx9mh3iuGBAAqvWSRyNX3u/TjEH6TJpqdA==", 2024-04-17T09:57:08.541 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:57:08.541 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:57:08.541 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:57:08.541 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:57:08.542 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:08.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:08.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:57:08.542 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2024-04-17T09:57:08.542 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXmx9mbe1CIRAA1pmzAl04tGkTu6QLpmCkag==", 2024-04-17T09:57:08.542 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:57:08.542 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:57:08.542 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:57:08.542 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:57:08.542 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:08.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:08.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:57:08.542 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2024-04-17T09:57:08.542 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDamx9mIwXoKxAAWPTFaYdp4lIZhuOU7F0x8g==", 2024-04-17T09:57:08.543 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:57:08.543 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:57:08.543 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:57:08.543 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:57:08.543 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:08.543 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:08.543 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:57:08.543 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2024-04-17T09:57:08.543 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDgmx9mp0s/JxAAv+usQqwrxgKINRlkDhhydQ==", 2024-04-17T09:57:08.543 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:57:08.543 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:57:08.543 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:57:08.543 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:57:08.543 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:08.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:08.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:57:08.544 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2024-04-17T09:57:08.544 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDjmx9mMVStMhAA/n8jk+LVDosWhziXk/lLRQ==", 2024-04-17T09:57:08.544 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:57:08.544 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:57:08.544 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:57:08.544 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:57:08.544 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:08.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:08.544 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:57:08.544 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2024-04-17T09:57:08.544 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDnmx9m/YQlABAAOqfWzVtgonD7b+WlR95mnA==", 2024-04-17T09:57:08.544 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:57:08.544 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:57:08.545 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:57:08.545 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:57:08.545 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:08.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:08.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:57:08.545 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2024-04-17T09:57:08.545 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDqmx9m572mCBAAqBib/yeQOBno+0B1K1dLxA==", 2024-04-17T09:57:08.545 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:57:08.545 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:57:08.545 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:57:08.545 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:57:08.545 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:08.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:08.545 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:57:08.545 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2024-04-17T09:57:08.546 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mUXoLKxAANV7zhLvL0DqHngk4TtsdBQ==", 2024-04-17T09:57:08.546 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:57:08.546 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:57:08.546 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2024-04-17T09:57:08.546 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-17T09:57:08.546 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:08.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:08.546 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:57:08.546 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2024-04-17T09:57:08.546 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mZBeRNBAAQg5K950mp6+XND7NCABAKQ==", 2024-04-17T09:57:08.546 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:57:08.546 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:57:08.546 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-17T09:57:08.546 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-17T09:57:08.547 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-17T09:57:08.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:08.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:08.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:57:08.547 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2024-04-17T09:57:08.547 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mWKstMRAAp18Hbnk02w+hT0sYZ3tSLA==", 2024-04-17T09:57:08.547 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:57:08.547 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:57:08.547 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-17T09:57:08.547 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-17T09:57:08.547 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-17T09:57:08.547 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:08.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:08.547 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:57:08.547 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2024-04-17T09:57:08.548 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mRQ2wOhAAzlWXFtP0WaXWmneqxnZ5/w==", 2024-04-17T09:57:08.548 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:57:08.548 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:57:08.548 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-17T09:57:08.548 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-17T09:57:08.548 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-17T09:57:08.548 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:08.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:08.548 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:57:08.548 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2024-04-17T09:57:08.548 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDOmx9maDStMxAAMCa36HfycwV3WbLg6Voo9Q==", 2024-04-17T09:57:08.548 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:57:08.548 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-17T09:57:08.548 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:57:08.549 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:57:08.549 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:57:08.549 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:08.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:08.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:57:08.549 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2024-04-17T09:57:08.549 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9myy0oDxAAPD2BTjIP09hiU5L7i6gUMg==", 2024-04-17T09:57:08.549 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:57:08.549 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-17T09:57:08.549 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-17T09:57:08.549 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:57:08.549 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:08.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:08.549 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:57:08.550 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2024-04-17T09:57:08.550 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9m3FmACxAApri5AI5PeR0+/HLev8DKcw==", 2024-04-17T09:57:08.550 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:57:08.550 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-17T09:57:08.550 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-17T09:57:08.550 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:57:08.550 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:08.550 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:08.550 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-17T09:57:08.550 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:57:08.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.536+0000 7fbdc6a00640 1 -- 172.21.15.32:0/3465990398 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbd9c0683b0 msgr2=0x7fbd9c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:08.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.536+0000 7fbdc6a00640 1 --2- 172.21.15.32:0/3465990398 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbd9c0683b0 0x7fbd9c06a870 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbda80af000 tx=0x7fbda80a7000 comp rx=0 tx=0).stop 2024-04-17T09:57:08.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.536+0000 7fbdc6a00640 1 -- 172.21.15.32:0/3465990398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc009ba60 msgr2=0x7fbdc00c9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:08.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.536+0000 7fbdc6a00640 1 --2- 172.21.15.32:0/3465990398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc009ba60 0x7fbdc00c9b10 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000c480 tx=0x7fbdb000c940 comp rx=0 tx=0).stop 2024-04-17T09:57:08.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.536+0000 7fbdc6a00640 1 -- 172.21.15.32:0/3465990398 shutdown_connections 2024-04-17T09:57:08.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.536+0000 7fbdc6a00640 1 --2- 172.21.15.32:0/3465990398 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbd9c0683b0 0x7fbd9c06a870 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:08.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.536+0000 7fbdc6a00640 1 --2- 172.21.15.32:0/3465990398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdc015e9a0 0x7fbdc0153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:08.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.536+0000 7fbdc6a00640 1 --2- 172.21.15.32:0/3465990398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdc015ed80 0x7fbdc01541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:08.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.536+0000 7fbdc6a00640 1 --2- 172.21.15.32:0/3465990398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc009ba60 0x7fbdc00c9b10 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:08.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.536+0000 7fbdc6a00640 1 -- 172.21.15.32:0/3465990398 >> 172.21.15.32:0/3465990398 conn(0x7fbdc00b3fd0 msgr2=0x7fbdc015dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:08.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.536+0000 7fbdc6a00640 1 -- 172.21.15.32:0/3465990398 shutdown_connections 2024-04-17T09:57:08.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.536+0000 7fbdc6a00640 1 -- 172.21.15.32:0/3465990398 wait complete. 2024-04-17T09:57:08.559 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:57:08.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.699+0000 7facb2200640 1 Processor -- start 2024-04-17T09:57:08.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.699+0000 7facb2200640 1 -- start start 2024-04-17T09:57:08.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.699+0000 7facb2200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7facac0d00b0 0x7facac09a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:08.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.699+0000 7facb2200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7facac15c9a0 0x7facac09af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:08.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.699+0000 7facb2200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facac0d05e0 con 0x7facac15c5c0 2024-04-17T09:57:08.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.699+0000 7facb2200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7facac0ce560 con 0x7facac0d00b0 2024-04-17T09:57:08.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.699+0000 7facb2200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7facac0d11f0 con 0x7facac15c9a0 2024-04-17T09:57:08.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.699+0000 7faca2a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7facac0d00b0 0x7facac09a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:08.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.699+0000 7faca2a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7facac0d00b0 0x7facac09a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39346/0 (socket says 172.21.15.32:39346) 2024-04-17T09:57:08.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.699+0000 7faca2a00640 1 -- 172.21.15.32:0/981909294 learned_addr learned my addr 172.21.15.32:0/981909294 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:08.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.699+0000 7facabe00640 1 --2- 172.21.15.32:0/981909294 >> v2:172.21.15.144:3300/0 conn(0x7facac15c9a0 0x7facac09af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:08.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.699+0000 7faca2a00640 1 -- 172.21.15.32:0/981909294 >> v2:172.21.15.144:3300/0 conn(0x7facac15c9a0 msgr2=0x7facac09af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:08.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.699+0000 7faca2a00640 1 --2- 172.21.15.32:0/981909294 >> v2:172.21.15.144:3300/0 conn(0x7facac15c9a0 0x7facac09af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:08.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.699+0000 7faca2a00640 1 -- 172.21.15.32:0/981909294 >> v1:172.21.15.144:6789/0 conn(0x7facac15c5c0 legacy=0x7facac0cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:08.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.699+0000 7faca2a00640 1 -- 172.21.15.32:0/981909294 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facac0d08c0 con 0x7facac0d00b0 2024-04-17T09:57:08.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.703+0000 7faca2a00640 1 --2- 172.21.15.32:0/981909294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facac0d00b0 0x7facac09a9f0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7facac161460 tx=0x7fac9401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e8760b519c192e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:08.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.703+0000 7facaaa00640 1 -- 172.21.15.32:0/981909294 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac940b5070 con 0x7facac0d00b0 2024-04-17T09:57:08.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.703+0000 7facaaa00640 1 -- 172.21.15.32:0/981909294 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac940a8a10 con 0x7facac0d00b0 2024-04-17T09:57:08.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.703+0000 7facaaa00640 1 -- 172.21.15.32:0/981909294 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac940a8de0 con 0x7facac0d00b0 2024-04-17T09:57:08.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.703+0000 7facb2200640 1 -- 172.21.15.32:0/981909294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facac0d00b0 msgr2=0x7facac09a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:08.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.703+0000 7facb2200640 1 --2- 172.21.15.32:0/981909294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facac0d00b0 0x7facac09a9f0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7facac161460 tx=0x7fac9401ed10 comp rx=0 tx=0).stop 2024-04-17T09:57:08.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.703+0000 7facb2200640 1 -- 172.21.15.32:0/981909294 shutdown_connections 2024-04-17T09:57:08.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.703+0000 7facb2200640 1 --2- 172.21.15.32:0/981909294 >> v2:172.21.15.144:3300/0 conn(0x7facac15c9a0 0x7facac09af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:08.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.703+0000 7facb2200640 1 --2- 172.21.15.32:0/981909294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facac0d00b0 0x7facac09a9f0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:08.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.703+0000 7facb2200640 1 -- 172.21.15.32:0/981909294 >> 172.21.15.32:0/981909294 conn(0x7facac0b3fd0 msgr2=0x7facac0b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:08.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.703+0000 7facb2200640 1 -- 172.21.15.32:0/981909294 shutdown_connections 2024-04-17T09:57:08.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.703+0000 7facb2200640 1 -- 172.21.15.32:0/981909294 wait complete. 2024-04-17T09:57:08.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7facb2200640 1 Processor -- start 2024-04-17T09:57:08.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7facb2200640 1 -- start start 2024-04-17T09:57:08.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7facb2200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facac0d00b0 0x7facac1508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:08.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7facb2200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facac15c5c0 0x7facac15e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:08.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7facb2200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facac15c9a0 0x7facac0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:08.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7facb2200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7facac0b31b0 con 0x7facac15c5c0 2024-04-17T09:57:08.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7facb2200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7facac0d0e80 con 0x7facac15c9a0 2024-04-17T09:57:08.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7facb2200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7facac0b3330 con 0x7facac0d00b0 2024-04-17T09:57:08.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7faca2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facac15c5c0 0x7facac15e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:08.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7faca2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facac15c5c0 0x7facac15e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39362/0 (socket says 172.21.15.32:39362) 2024-04-17T09:57:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7faca2a00640 1 -- 172.21.15.32:0/1939423378 learned_addr learned my addr 172.21.15.32:0/1939423378 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7facab400640 1 --2- 172.21.15.32:0/1939423378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facac0d00b0 0x7facac1508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7facabe00640 1 --2- 172.21.15.32:0/1939423378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facac15c9a0 0x7facac0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7faca2a00640 1 -- 172.21.15.32:0/1939423378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facac0d00b0 msgr2=0x7facac1508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7faca2a00640 1 --2- 172.21.15.32:0/1939423378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facac0d00b0 0x7facac1508e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7faca2a00640 1 -- 172.21.15.32:0/1939423378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facac15c9a0 msgr2=0x7facac0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7faca2a00640 1 --2- 172.21.15.32:0/1939423378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facac15c9a0 0x7facac0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7faca2a00640 1 -- 172.21.15.32:0/1939423378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facac15ed10 con 0x7facac15c5c0 2024-04-17T09:57:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7faca2a00640 1 --2- 172.21.15.32:0/1939423378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facac15c5c0 0x7facac15e2c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fac9401b920 tx=0x7fac9401bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7faca3e00640 1 -- 172.21.15.32:0/1939423378 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac940b5070 con 0x7facac15c5c0 2024-04-17T09:57:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7faca3e00640 1 -- 172.21.15.32:0/1939423378 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac940b0b40 con 0x7facac15c5c0 2024-04-17T09:57:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7faca3e00640 1 -- 172.21.15.32:0/1939423378 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac940bb410 con 0x7facac15c5c0 2024-04-17T09:57:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7facb2200640 1 -- 172.21.15.32:0/1939423378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facac15eff0 con 0x7facac15c5c0 2024-04-17T09:57:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7facb2200640 1 -- 172.21.15.32:0/1939423378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facac15f5b0 con 0x7facac15c5c0 2024-04-17T09:57:08.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.707+0000 7facb2200640 1 -- 172.21.15.32:0/1939423378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facac068a90 con 0x7facac15c5c0 2024-04-17T09:57:08.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.711+0000 7faca3e00640 1 -- 172.21.15.32:0/1939423378 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fac940c2060 con 0x7facac15c5c0 2024-04-17T09:57:08.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.711+0000 7faca3e00640 1 --2- 172.21.15.32:0/1939423378 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fac88068380 0x7fac8806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:08.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.711+0000 7faca3e00640 1 -- 172.21.15.32:0/1939423378 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4857+0+0 (secure 0 0 0) 0x7fac94123750 con 0x7facac15c5c0 2024-04-17T09:57:08.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.711+0000 7facab400640 1 --2- 172.21.15.32:0/1939423378 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fac88068380 0x7fac8806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:08.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.711+0000 7facab400640 1 --2- 172.21.15.32:0/1939423378 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fac88068380 0x7fac8806a840 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7facac0cd230 tx=0x7fac9c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:08.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.715+0000 7faca3e00640 1 -- 172.21.15.32:0/1939423378 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fac94018870 con 0x7facac15c5c0 2024-04-17T09:57:08.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.879+0000 7facb2200640 1 -- 172.21.15.32:0/1939423378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7facac0cd6c0 con 0x7facac15c5c0 2024-04-17T09:57:08.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.879+0000 7faca3e00640 1 -- 172.21.15.32:0/1939423378 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+12903 (secure 0 0 0) 0x7fac940f3970 con 0x7facac15c5c0 2024-04-17T09:57:08.881 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:57:08.881 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":64,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:57:08.137746+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:57:08.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.879+0000 7facb2200640 1 -- 172.21.15.32:0/1939423378 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fac88068380 msgr2=0x7fac8806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:08.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.879+0000 7facb2200640 1 --2- 172.21.15.32:0/1939423378 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fac88068380 0x7fac8806a840 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7facac0cd230 tx=0x7fac9c009040 comp rx=0 tx=0).stop 2024-04-17T09:57:08.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.879+0000 7facb2200640 1 -- 172.21.15.32:0/1939423378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facac15c5c0 msgr2=0x7facac15e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:08.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.879+0000 7facb2200640 1 --2- 172.21.15.32:0/1939423378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facac15c5c0 0x7facac15e2c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fac9401b920 tx=0x7fac9401bab0 comp rx=0 tx=0).stop 2024-04-17T09:57:08.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.879+0000 7facb2200640 1 -- 172.21.15.32:0/1939423378 shutdown_connections 2024-04-17T09:57:08.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.879+0000 7facb2200640 1 --2- 172.21.15.32:0/1939423378 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fac88068380 0x7fac8806a840 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:08.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.883+0000 7facb2200640 1 --2- 172.21.15.32:0/1939423378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facac15c9a0 0x7facac0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:08.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.883+0000 7facb2200640 1 --2- 172.21.15.32:0/1939423378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facac15c5c0 0x7facac15e2c0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:08.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.883+0000 7facb2200640 1 --2- 172.21.15.32:0/1939423378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facac0d00b0 0x7facac1508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:08.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.883+0000 7facb2200640 1 -- 172.21.15.32:0/1939423378 >> 172.21.15.32:0/1939423378 conn(0x7facac0b3fd0 msgr2=0x7facac09b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:08.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.883+0000 7facb2200640 1 -- 172.21.15.32:0/1939423378 shutdown_connections 2024-04-17T09:57:08.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:08.883+0000 7facb2200640 1 -- 172.21.15.32:0/1939423378 wait complete. 2024-04-17T09:57:08.903 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:57:08.903 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:57:09.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.047+0000 7fd176a00640 1 Processor -- start 2024-04-17T09:57:09.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.047+0000 7fd176a00640 1 -- start start 2024-04-17T09:57:09.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.047+0000 7fd176a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd17006ba30 0x7fd17006be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:09.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.047+0000 7fd176a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1700cf370 0x7fd1700cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:09.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.047+0000 7fd176a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd17006c350 con 0x7fd170160350 2024-04-17T09:57:09.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.047+0000 7fd176a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd1700d0130 con 0x7fd1700cf370 2024-04-17T09:57:09.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.047+0000 7fd176a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd17015d090 con 0x7fd17006ba30 2024-04-17T09:57:09.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.047+0000 7fd16f400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1700cf370 0x7fd1700cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:09.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.047+0000 7fd16f400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1700cf370 0x7fd1700cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39378/0 (socket says 172.21.15.32:39378) 2024-04-17T09:57:09.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.047+0000 7fd16f400640 1 -- 172.21.15.32:0/2771025309 learned_addr learned my addr 172.21.15.32:0/2771025309 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:09.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.047+0000 7fd16fe00640 1 --2- 172.21.15.32:0/2771025309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd17006ba30 0x7fd17006be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:09.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.047+0000 7fd16f400640 1 -- 172.21.15.32:0/2771025309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd17006ba30 msgr2=0x7fd17006be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:09.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.047+0000 7fd16f400640 1 --2- 172.21.15.32:0/2771025309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd17006ba30 0x7fd17006be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.047+0000 7fd16f400640 1 -- 172.21.15.32:0/2771025309 >> v1:172.21.15.144:6789/0 conn(0x7fd170160350 legacy=0x7fd1701607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:09.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.047+0000 7fd16f400640 1 -- 172.21.15.32:0/2771025309 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17015d370 con 0x7fd1700cf370 2024-04-17T09:57:09.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.051+0000 7fd16f400640 1 --2- 172.21.15.32:0/2771025309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1700cf370 0x7fd1700cf7e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd15c009870 tx=0x7fd15c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b8653a791be1f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.051+0000 7fd16ea00640 1 -- 172.21.15.32:0/2771025309 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c0a7070 con 0x7fd1700cf370 2024-04-17T09:57:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.051+0000 7fd16ea00640 1 -- 172.21.15.32:0/2771025309 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd15c0b0070 con 0x7fd1700cf370 2024-04-17T09:57:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.051+0000 7fd16ea00640 1 -- 172.21.15.32:0/2771025309 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c0ab890 con 0x7fd1700cf370 2024-04-17T09:57:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.051+0000 7fd176a00640 1 -- 172.21.15.32:0/2771025309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1700cf370 msgr2=0x7fd1700cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.051+0000 7fd176a00640 1 --2- 172.21.15.32:0/2771025309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1700cf370 0x7fd1700cf7e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd15c009870 tx=0x7fd15c01c560 comp rx=0 tx=0).stop 2024-04-17T09:57:09.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.051+0000 7fd176a00640 1 -- 172.21.15.32:0/2771025309 shutdown_connections 2024-04-17T09:57:09.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.051+0000 7fd176a00640 1 --2- 172.21.15.32:0/2771025309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1700cf370 0x7fd1700cf7e0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.051+0000 7fd176a00640 1 --2- 172.21.15.32:0/2771025309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd17006ba30 0x7fd17006be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.051+0000 7fd176a00640 1 -- 172.21.15.32:0/2771025309 >> 172.21.15.32:0/2771025309 conn(0x7fd1700b2120 msgr2=0x7fd1700b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:09.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.051+0000 7fd176a00640 1 -- 172.21.15.32:0/2771025309 shutdown_connections 2024-04-17T09:57:09.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.051+0000 7fd176a00640 1 -- 172.21.15.32:0/2771025309 wait complete. 2024-04-17T09:57:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.051+0000 7fd176a00640 1 Processor -- start 2024-04-17T09:57:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.051+0000 7fd176a00640 1 -- start start 2024-04-17T09:57:09.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.055+0000 7fd176a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd17006ba30 0x7fd1701f6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:09.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.055+0000 7fd176a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1700cf370 0x7fd170159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.055+0000 7fd176a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd170160350 0x7fd17015f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.055+0000 7fd176a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd17015dcf0 con 0x7fd1700cf370 2024-04-17T09:57:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.055+0000 7fd176a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd17015db70 con 0x7fd17006ba30 2024-04-17T09:57:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.055+0000 7fd176a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd170146230 con 0x7fd170160350 2024-04-17T09:57:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.055+0000 7fd16f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1700cf370 0x7fd170159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.055+0000 7fd16f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1700cf370 0x7fd170159cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39388/0 (socket says 172.21.15.32:39388) 2024-04-17T09:57:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.055+0000 7fd16f400640 1 -- 172.21.15.32:0/179221851 learned_addr learned my addr 172.21.15.32:0/179221851 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.055+0000 7fd16fe00640 1 --2- 172.21.15.32:0/179221851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd17006ba30 0x7fd1701f6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.055+0000 7fd174a00640 1 --2- 172.21.15.32:0/179221851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd170160350 0x7fd17015f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.055+0000 7fd16f400640 1 -- 172.21.15.32:0/179221851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd170160350 msgr2=0x7fd17015f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.055+0000 7fd16f400640 1 --2- 172.21.15.32:0/179221851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd170160350 0x7fd17015f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.055+0000 7fd16f400640 1 -- 172.21.15.32:0/179221851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd17006ba30 msgr2=0x7fd1701f6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.055+0000 7fd16f400640 1 --2- 172.21.15.32:0/179221851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd17006ba30 0x7fd1701f6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.055+0000 7fd16f400640 1 -- 172.21.15.32:0/179221851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1701f74e0 con 0x7fd1700cf370 2024-04-17T09:57:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.055+0000 7fd16f400640 1 --2- 172.21.15.32:0/179221851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1700cf370 0x7fd170159cc0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd15c01a190 tx=0x7fd15c01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.055+0000 7fd167e00640 1 -- 172.21.15.32:0/179221851 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c0be5f0 con 0x7fd1700cf370 2024-04-17T09:57:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.055+0000 7fd167e00640 1 -- 172.21.15.32:0/179221851 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd15c0be790 con 0x7fd1700cf370 2024-04-17T09:57:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.055+0000 7fd176a00640 1 -- 172.21.15.32:0/179221851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17015fc50 con 0x7fd1700cf370 2024-04-17T09:57:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.055+0000 7fd167e00640 1 -- 172.21.15.32:0/179221851 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c0beb80 con 0x7fd1700cf370 2024-04-17T09:57:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.055+0000 7fd176a00640 1 -- 172.21.15.32:0/179221851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1700b0730 con 0x7fd1700cf370 2024-04-17T09:57:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.055+0000 7fd176a00640 1 -- 172.21.15.32:0/179221851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd170068a90 con 0x7fd1700cf370 2024-04-17T09:57:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.059+0000 7fd167e00640 1 -- 172.21.15.32:0/179221851 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd15c0b43f0 con 0x7fd1700cf370 2024-04-17T09:57:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.059+0000 7fd167e00640 1 --2- 172.21.15.32:0/179221851 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd150068380 0x7fd15006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.059+0000 7fd167e00640 1 -- 172.21.15.32:0/179221851 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4857+0+0 (secure 0 0 0) 0x7fd15c121540 con 0x7fd1700cf370 2024-04-17T09:57:09.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.059+0000 7fd16fe00640 1 --2- 172.21.15.32:0/179221851 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd150068380 0x7fd15006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:09.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.059+0000 7fd16fe00640 1 --2- 172.21.15.32:0/179221851 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd150068380 0x7fd15006a840 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd17015edf0 tx=0x7fd160008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:09.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.063+0000 7fd167e00640 1 -- 172.21.15.32:0/179221851 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd15c0c4060 con 0x7fd1700cf370 2024-04-17T09:57:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.239+0000 7fd176a00640 1 -- 172.21.15.32:0/179221851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd17006c850 con 0x7fd1700cf370 2024-04-17T09:57:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.239+0000 7fd167e00640 1 -- 172.21.15.32:0/179221851 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) ==== 93+0+10 (secure 0 0 0) 0x7fd15c016150 con 0x7fd1700cf370 2024-04-17T09:57:09.240 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:57:09.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.239+0000 7fd176a00640 1 -- 172.21.15.32:0/179221851 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd150068380 msgr2=0x7fd15006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:09.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.239+0000 7fd176a00640 1 --2- 172.21.15.32:0/179221851 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd150068380 0x7fd15006a840 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd17015edf0 tx=0x7fd160008040 comp rx=0 tx=0).stop 2024-04-17T09:57:09.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.239+0000 7fd176a00640 1 -- 172.21.15.32:0/179221851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1700cf370 msgr2=0x7fd170159cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:09.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.239+0000 7fd176a00640 1 --2- 172.21.15.32:0/179221851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1700cf370 0x7fd170159cc0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd15c01a190 tx=0x7fd15c01ada0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.239+0000 7fd176a00640 1 -- 172.21.15.32:0/179221851 shutdown_connections 2024-04-17T09:57:09.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.239+0000 7fd176a00640 1 --2- 172.21.15.32:0/179221851 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd150068380 0x7fd15006a840 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.239+0000 7fd176a00640 1 --2- 172.21.15.32:0/179221851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd170160350 0x7fd17015f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.239+0000 7fd176a00640 1 --2- 172.21.15.32:0/179221851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1700cf370 0x7fd170159cc0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.239+0000 7fd176a00640 1 --2- 172.21.15.32:0/179221851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd17006ba30 0x7fd1701f6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.239+0000 7fd176a00640 1 -- 172.21.15.32:0/179221851 >> 172.21.15.32:0/179221851 conn(0x7fd1700b2120 msgr2=0x7fd17006efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:09.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.239+0000 7fd176a00640 1 -- 172.21.15.32:0/179221851 shutdown_connections 2024-04-17T09:57:09.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.239+0000 7fd176a00640 1 -- 172.21.15.32:0/179221851 wait complete. 2024-04-17T09:57:09.261 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T09:57:09.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.399+0000 7f289be00640 1 Processor -- start 2024-04-17T09:57:09.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.399+0000 7f289be00640 1 -- start start 2024-04-17T09:57:09.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.399+0000 7f289be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2894144f20 0x7f289406e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:09.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.399+0000 7f289be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f289406eda0 0x7f2894071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:09.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.399+0000 7f289be00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f28941481e0 0x7f2894072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:09.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.399+0000 7f289be00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f28940aed20 con 0x7f2894144f20 2024-04-17T09:57:09.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.399+0000 7f289be00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f28940af000 con 0x7f28941481e0 2024-04-17T09:57:09.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.399+0000 7f289be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f28940af2e0 con 0x7f289406eda0 2024-04-17T09:57:09.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.399+0000 7f2899400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2894144f20 0x7f289406e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:09.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.399+0000 7f2899400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2894144f20 0x7f289406e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39402/0 (socket says 172.21.15.32:39402) 2024-04-17T09:57:09.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.399+0000 7f2899400640 1 -- 172.21.15.32:0/2119315323 learned_addr learned my addr 172.21.15.32:0/2119315323 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:09.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.399+0000 7f2899e00640 1 --2- 172.21.15.32:0/2119315323 >> v2:172.21.15.144:3300/0 conn(0x7f28941481e0 0x7f2894072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:09.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.399+0000 7f2898a00640 1 --2- 172.21.15.32:0/2119315323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f289406eda0 0x7f2894071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:09.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.403+0000 7f2899400640 1 -- 172.21.15.32:0/2119315323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f289406eda0 msgr2=0x7f2894071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:09.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.403+0000 7f2899400640 1 --2- 172.21.15.32:0/2119315323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f289406eda0 0x7f2894071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.403+0000 7f2899400640 1 -- 172.21.15.32:0/2119315323 >> v2:172.21.15.144:3300/0 conn(0x7f28941481e0 msgr2=0x7f2894072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:09.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.403+0000 7f2899400640 1 --2- 172.21.15.32:0/2119315323 >> v2:172.21.15.144:3300/0 conn(0x7f28941481e0 0x7f2894072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.403+0000 7f2899400640 1 -- 172.21.15.32:0/2119315323 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28940af5c0 con 0x7f2894144f20 2024-04-17T09:57:09.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.403+0000 7f2899e00640 1 --2- 172.21.15.32:0/2119315323 >> v2:172.21.15.144:3300/0 conn(0x7f28941481e0 0x7f2894072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:57:09.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.403+0000 7f2899400640 1 --2- 172.21.15.32:0/2119315323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2894144f20 0x7f289406e860 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f28940cc910 tx=0x7f288801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94747058ff28688c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:09.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.403+0000 7f2893e00640 1 -- 172.21.15.32:0/2119315323 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28880b0070 con 0x7f2894144f20 2024-04-17T09:57:09.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.403+0000 7f2893e00640 1 -- 172.21.15.32:0/2119315323 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28880ab600 con 0x7f2894144f20 2024-04-17T09:57:09.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.403+0000 7f2893e00640 1 -- 172.21.15.32:0/2119315323 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28880ab9d0 con 0x7f2894144f20 2024-04-17T09:57:09.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.403+0000 7f289be00640 1 -- 172.21.15.32:0/2119315323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2894144f20 msgr2=0x7f289406e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:09.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.403+0000 7f289be00640 1 --2- 172.21.15.32:0/2119315323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2894144f20 0x7f289406e860 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f28940cc910 tx=0x7f288801c520 comp rx=0 tx=0).stop 2024-04-17T09:57:09.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f289be00640 1 -- 172.21.15.32:0/2119315323 shutdown_connections 2024-04-17T09:57:09.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f289be00640 1 --2- 172.21.15.32:0/2119315323 >> v2:172.21.15.144:3300/0 conn(0x7f28941481e0 0x7f2894072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f289be00640 1 --2- 172.21.15.32:0/2119315323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f289406eda0 0x7f2894071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f289be00640 1 --2- 172.21.15.32:0/2119315323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2894144f20 0x7f289406e860 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f289be00640 1 -- 172.21.15.32:0/2119315323 >> 172.21.15.32:0/2119315323 conn(0x7f289406ba30 msgr2=0x7f289406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:09.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f289be00640 1 -- 172.21.15.32:0/2119315323 shutdown_connections 2024-04-17T09:57:09.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f289be00640 1 -- 172.21.15.32:0/2119315323 wait complete. 2024-04-17T09:57:09.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f289be00640 1 Processor -- start 2024-04-17T09:57:09.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f289be00640 1 -- start start 2024-04-17T09:57:09.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f289be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289406eda0 0x7f289415d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:09.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f289be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2894144f20 0x7f28940cef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:09.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f289be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28941481e0 0x7f289413cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:09.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f289be00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2894162fe0 con 0x7f289406eda0 2024-04-17T09:57:09.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f289be00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2894162e60 con 0x7f28941481e0 2024-04-17T09:57:09.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f289be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2894163160 con 0x7f2894144f20 2024-04-17T09:57:09.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f2899400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289406eda0 0x7f289415d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:09.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f2899400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289406eda0 0x7f289415d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39412/0 (socket says 172.21.15.32:39412) 2024-04-17T09:57:09.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f2899400640 1 -- 172.21.15.32:0/2435689011 learned_addr learned my addr 172.21.15.32:0/2435689011 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:09.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f2899400640 1 -- 172.21.15.32:0/2435689011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2894144f20 msgr2=0x7f28940cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:09.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f2899400640 1 --2- 172.21.15.32:0/2435689011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2894144f20 0x7f28940cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f2899400640 1 -- 172.21.15.32:0/2435689011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28941481e0 msgr2=0x7f289413cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:09.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f2899400640 1 --2- 172.21.15.32:0/2435689011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28941481e0 0x7f289413cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f2899400640 1 -- 172.21.15.32:0/2435689011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289415e370 con 0x7f289406eda0 2024-04-17T09:57:09.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f2899400640 1 --2- 172.21.15.32:0/2435689011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289406eda0 0x7f289415d000 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f288801abd0 tx=0x7f28880192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:09.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f2891600640 1 -- 172.21.15.32:0/2435689011 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28880b9070 con 0x7f289406eda0 2024-04-17T09:57:09.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f2891600640 1 -- 172.21.15.32:0/2435689011 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28880b4400 con 0x7f289406eda0 2024-04-17T09:57:09.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f2891600640 1 -- 172.21.15.32:0/2435689011 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28880b0040 con 0x7f289406eda0 2024-04-17T09:57:09.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.407+0000 7f289be00640 1 -- 172.21.15.32:0/2435689011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f289415d540 con 0x7f289406eda0 2024-04-17T09:57:09.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.411+0000 7f289be00640 1 -- 172.21.15.32:0/2435689011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f289415ddb0 con 0x7f289406eda0 2024-04-17T09:57:09.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.411+0000 7f289be00640 1 -- 172.21.15.32:0/2435689011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f289415d850 con 0x7f289406eda0 2024-04-17T09:57:09.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.411+0000 7f2891600640 1 -- 172.21.15.32:0/2435689011 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f28880bc070 con 0x7f289406eda0 2024-04-17T09:57:09.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.411+0000 7f2891600640 1 --2- 172.21.15.32:0/2435689011 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f28740683b0 0x7f287406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:09.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.411+0000 7f2891600640 1 -- 172.21.15.32:0/2435689011 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4857+0+0 (secure 0 0 0) 0x7f288811f690 con 0x7f289406eda0 2024-04-17T09:57:09.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.411+0000 7f2898a00640 1 --2- 172.21.15.32:0/2435689011 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f28740683b0 0x7f287406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:09.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.415+0000 7f2898a00640 1 --2- 172.21.15.32:0/2435689011 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f28740683b0 0x7f287406a870 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f28940ce860 tx=0x7f2880008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:09.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.415+0000 7f2891600640 1 -- 172.21.15.32:0/2435689011 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f289415d850 con 0x7f289406eda0 2024-04-17T09:57:09.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.583+0000 7f289be00640 1 -- 172.21.15.32:0/2435689011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2894068a90 con 0x7f289406eda0 2024-04-17T09:57:09.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.583+0000 7f2891600640 1 -- 172.21.15.32:0/2435689011 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v64) ==== 84+0+65 (secure 0 0 0) 0x7f2888016110 con 0x7f289406eda0 2024-04-17T09:57:09.586 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:57:09.586 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-17T09:57:09.586 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:57:09.586 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-04-17T09:57:09.586 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-04-17T09:57:09.586 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:09.586 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-17T09:57:09.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.587+0000 7f289be00640 1 -- 172.21.15.32:0/2435689011 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f28740683b0 msgr2=0x7f287406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.587+0000 7f289be00640 1 --2- 172.21.15.32:0/2435689011 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f28740683b0 0x7f287406a870 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f28940ce860 tx=0x7f2880008040 comp rx=0 tx=0).stop 2024-04-17T09:57:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.587+0000 7f289be00640 1 -- 172.21.15.32:0/2435689011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289406eda0 msgr2=0x7f289415d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.587+0000 7f289be00640 1 --2- 172.21.15.32:0/2435689011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289406eda0 0x7f289415d000 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f288801abd0 tx=0x7f28880192c0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.587+0000 7f289be00640 1 -- 172.21.15.32:0/2435689011 shutdown_connections 2024-04-17T09:57:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.587+0000 7f289be00640 1 --2- 172.21.15.32:0/2435689011 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f28740683b0 0x7f287406a870 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.587+0000 7f289be00640 1 --2- 172.21.15.32:0/2435689011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2894144f20 0x7f28940cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.587+0000 7f289be00640 1 --2- 172.21.15.32:0/2435689011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28941481e0 0x7f289413cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.587+0000 7f289be00640 1 --2- 172.21.15.32:0/2435689011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f289406eda0 0x7f289415d000 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.587+0000 7f289be00640 1 -- 172.21.15.32:0/2435689011 >> 172.21.15.32:0/2435689011 conn(0x7f289406ba30 msgr2=0x7f28940b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.587+0000 7f289be00640 1 -- 172.21.15.32:0/2435689011 shutdown_connections 2024-04-17T09:57:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.587+0000 7f289be00640 1 -- 172.21.15.32:0/2435689011 wait complete. 2024-04-17T09:57:09.607 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T09:57:09.608 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T09:57:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.751+0000 7efdcf600640 1 Processor -- start 2024-04-17T09:57:09.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.751+0000 7efdcf600640 1 -- start start 2024-04-17T09:57:09.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.751+0000 7efdcf600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efdc806f500 0x7efdc80cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:09.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.751+0000 7efdcf600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdc815f200 con 0x7efdc80cd6a0 2024-04-17T09:57:09.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.751+0000 7efdcf600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdc815f4e0 con 0x7efdc80cfc70 2024-04-17T09:57:09.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.751+0000 7efdcf600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7efdc815fb30 con 0x7efdc806f500 2024-04-17T09:57:09.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.751+0000 7efdcd600640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efdc80cd6a0 0x7efdc815ba70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55238/0 (socket says 172.21.15.32:55238) 2024-04-17T09:57:09.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.751+0000 7efdcd600640 1 -- 172.21.15.32:0/2045840650 learned_addr learned my addr 172.21.15.32:0/2045840650 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:09.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.751+0000 7efdc7e00640 1 --2- 172.21.15.32:0/2045840650 >> v2:172.21.15.144:3300/0 conn(0x7efdc806f500 0x7efdc80cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:09.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.751+0000 7efdc7400640 1 -- 172.21.15.32:0/2045840650 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 628398804 0 0) 0x7efdc815f200 con 0x7efdc80cd6a0 2024-04-17T09:57:09.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.751+0000 7efdc7400640 1 -- 172.21.15.32:0/2045840650 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdb0003530 con 0x7efdc80cd6a0 2024-04-17T09:57:09.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.751+0000 7efdc7400640 1 -- 172.21.15.32:0/2045840650 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 890225775 0 0) 0x7efdc815f4e0 con 0x7efdc80cfc70 2024-04-17T09:57:09.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.751+0000 7efdc7400640 1 -- 172.21.15.32:0/2045840650 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdc815f200 con 0x7efdc80cfc70 2024-04-17T09:57:09.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.751+0000 7efdc7e00640 1 -- 172.21.15.32:0/2045840650 >> v1:172.21.15.144:6789/0 conn(0x7efdc80cfc70 legacy=0x7efdc806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:09.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.751+0000 7efdc7e00640 1 -- 172.21.15.32:0/2045840650 >> v1:172.21.15.32:6789/0 conn(0x7efdc80cd6a0 legacy=0x7efdc815ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:09.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.751+0000 7efdc7e00640 1 -- 172.21.15.32:0/2045840650 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc815f7c0 con 0x7efdc806f500 2024-04-17T09:57:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.755+0000 7efdc7e00640 1 --2- 172.21.15.32:0/2045840650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdc806f500 0x7efdc80cd160 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efdb40097a0 tx=0x7efdb401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1996224db8f43072 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.755+0000 7efdc7400640 1 -- 172.21.15.32:0/2045840650 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb40b0070 con 0x7efdc806f500 2024-04-17T09:57:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.755+0000 7efdc7400640 1 -- 172.21.15.32:0/2045840650 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb40ab600 con 0x7efdc806f500 2024-04-17T09:57:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.755+0000 7efdc7400640 1 -- 172.21.15.32:0/2045840650 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb40ab9d0 con 0x7efdc806f500 2024-04-17T09:57:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.755+0000 7efdcf600640 1 -- 172.21.15.32:0/2045840650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdc806f500 msgr2=0x7efdc80cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.755+0000 7efdcf600640 1 --2- 172.21.15.32:0/2045840650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdc806f500 0x7efdc80cd160 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efdb40097a0 tx=0x7efdb401c480 comp rx=0 tx=0).stop 2024-04-17T09:57:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.755+0000 7efdcf600640 1 -- 172.21.15.32:0/2045840650 shutdown_connections 2024-04-17T09:57:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.755+0000 7efdcf600640 1 --2- 172.21.15.32:0/2045840650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdc806f500 0x7efdc80cd160 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.755+0000 7efdcf600640 1 -- 172.21.15.32:0/2045840650 >> 172.21.15.32:0/2045840650 conn(0x7efdc80b2190 msgr2=0x7efdc80b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:09.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.755+0000 7efdcf600640 1 -- 172.21.15.32:0/2045840650 shutdown_connections 2024-04-17T09:57:09.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.755+0000 7efdcf600640 1 -- 172.21.15.32:0/2045840650 wait complete. 2024-04-17T09:57:09.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdcf600640 1 Processor -- start 2024-04-17T09:57:09.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdcf600640 1 -- start start 2024-04-17T09:57:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdcf600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdc806f500 0x7efdc81f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdcf600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc80cd6a0 0x7efdc81525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdcf600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdc80cfc70 0x7efdc8153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdcf600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efdc806de60 con 0x7efdc80cd6a0 2024-04-17T09:57:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdcf600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efdc806dce0 con 0x7efdc806f500 2024-04-17T09:57:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdcf600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efdc806dfe0 con 0x7efdc80cfc70 2024-04-17T09:57:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdc7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc80cd6a0 0x7efdc81525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdc7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc80cd6a0 0x7efdc81525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39424/0 (socket says 172.21.15.32:39424) 2024-04-17T09:57:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdc7e00640 1 -- 172.21.15.32:0/1776560129 learned_addr learned my addr 172.21.15.32:0/1776560129 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdccc00640 1 --2- 172.21.15.32:0/1776560129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdc806f500 0x7efdc81f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdcd600640 1 --2- 172.21.15.32:0/1776560129 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdc80cfc70 0x7efdc8153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdc7e00640 1 -- 172.21.15.32:0/1776560129 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdc80cfc70 msgr2=0x7efdc8153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdc7e00640 1 --2- 172.21.15.32:0/1776560129 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdc80cfc70 0x7efdc8153660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdc7e00640 1 -- 172.21.15.32:0/1776560129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdc806f500 msgr2=0x7efdc81f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdc7e00640 1 --2- 172.21.15.32:0/1776560129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdc806f500 0x7efdc81f7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdc7e00640 1 -- 172.21.15.32:0/1776560129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc80ad860 con 0x7efdc80cd6a0 2024-04-17T09:57:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdcd600640 1 --2- 172.21.15.32:0/1776560129 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdc80cfc70 0x7efdc8153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:57:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdc7e00640 1 --2- 172.21.15.32:0/1776560129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc80cd6a0 0x7efdc81525b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7efdb40ae040 tx=0x7efdb4019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:09.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdc4c00640 1 -- 172.21.15.32:0/1776560129 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb40b0040 con 0x7efdc80cd6a0 2024-04-17T09:57:09.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdc4c00640 1 -- 172.21.15.32:0/1776560129 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb40b9070 con 0x7efdc80cd6a0 2024-04-17T09:57:09.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdc4c00640 1 -- 172.21.15.32:0/1776560129 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb40b4670 con 0x7efdc80cd6a0 2024-04-17T09:57:09.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdcf600640 1 -- 172.21.15.32:0/1776560129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc8156140 con 0x7efdc80cd6a0 2024-04-17T09:57:09.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.759+0000 7efdcf600640 1 -- 172.21.15.32:0/1776560129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc8156700 con 0x7efdc80cd6a0 2024-04-17T09:57:09.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.763+0000 7efdcf600640 1 -- 172.21.15.32:0/1776560129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdc8068a90 con 0x7efdc80cd6a0 2024-04-17T09:57:09.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.763+0000 7efdc4c00640 1 -- 172.21.15.32:0/1776560129 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7efdb40b4810 con 0x7efdc80cd6a0 2024-04-17T09:57:09.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.763+0000 7efdc4c00640 1 --2- 172.21.15.32:0/1776560129 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efda8068380 0x7efda806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:09.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.763+0000 7efdc4c00640 1 -- 172.21.15.32:0/1776560129 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4857+0+0 (secure 0 0 0) 0x7efdb41202a0 con 0x7efdc80cd6a0 2024-04-17T09:57:09.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.763+0000 7efdccc00640 1 --2- 172.21.15.32:0/1776560129 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efda8068380 0x7efda806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:09.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.763+0000 7efdccc00640 1 --2- 172.21.15.32:0/1776560129 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efda8068380 0x7efda806a840 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7efdbc001720 tx=0x7efdbc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:09.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.767+0000 7efdc4c00640 1 -- 172.21.15.32:0/1776560129 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7efdb4016070 con 0x7efdc80cd6a0 2024-04-17T09:57:09.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:09.935+0000 7efdcf600640 1 -- 172.21.15.32:0/1776560129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7efdc8072e50 con 0x7efdc80cd6a0 2024-04-17T09:57:10.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.139+0000 7efdc4c00640 1 -- 172.21.15.32:0/1776560129 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v65) ==== 138+0+0 (secure 0 0 0) 0x7efdb40f0ba0 con 0x7efdc80cd6a0 2024-04-17T09:57:10.141 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-04-17T09:57:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.139+0000 7efdcf600640 1 -- 172.21.15.32:0/1776560129 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efda8068380 msgr2=0x7efda806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.139+0000 7efdcf600640 1 --2- 172.21.15.32:0/1776560129 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efda8068380 0x7efda806a840 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7efdbc001720 tx=0x7efdbc009040 comp rx=0 tx=0).stop 2024-04-17T09:57:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.139+0000 7efdcf600640 1 -- 172.21.15.32:0/1776560129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc80cd6a0 msgr2=0x7efdc81525b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.139+0000 7efdcf600640 1 --2- 172.21.15.32:0/1776560129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc80cd6a0 0x7efdc81525b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7efdb40ae040 tx=0x7efdb4019220 comp rx=0 tx=0).stop 2024-04-17T09:57:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.143+0000 7efdcf600640 1 -- 172.21.15.32:0/1776560129 shutdown_connections 2024-04-17T09:57:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.143+0000 7efdcf600640 1 --2- 172.21.15.32:0/1776560129 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efda8068380 0x7efda806a840 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.143+0000 7efdcf600640 1 --2- 172.21.15.32:0/1776560129 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdc80cfc70 0x7efdc8153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.143+0000 7efdcf600640 1 --2- 172.21.15.32:0/1776560129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdc80cd6a0 0x7efdc81525b0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.143+0000 7efdcf600640 1 --2- 172.21.15.32:0/1776560129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdc806f500 0x7efdc81f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.143+0000 7efdcf600640 1 -- 172.21.15.32:0/1776560129 >> 172.21.15.32:0/1776560129 conn(0x7efdc80b2190 msgr2=0x7efdc81669b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.143+0000 7efdcf600640 1 -- 172.21.15.32:0/1776560129 shutdown_connections 2024-04-17T09:57:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.143+0000 7efdcf600640 1 -- 172.21.15.32:0/1776560129 wait complete. 2024-04-17T09:57:10.168 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-17T09:57:10.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.311+0000 7fa3af600640 1 Processor -- start 2024-04-17T09:57:10.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.311+0000 7fa3af600640 1 -- start start 2024-04-17T09:57:10.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.315+0000 7fa3af600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3a80b3d80 0x7fa3a8071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:10.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.315+0000 7fa3af600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3a806e6f0 con 0x7fa3a815ac30 2024-04-17T09:57:10.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.315+0000 7fa3af600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3a806e9d0 con 0x7fa3a815b010 2024-04-17T09:57:10.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.315+0000 7fa3af600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa3a806f300 con 0x7fa3a80b3d80 2024-04-17T09:57:10.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.315+0000 7fa3acc00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa3a815ac30 0x7fa3a806f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41580/0 (socket says 172.21.15.32:41580) 2024-04-17T09:57:10.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.315+0000 7fa3acc00640 1 -- 172.21.15.32:0/917488992 learned_addr learned my addr 172.21.15.32:0/917488992 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:10.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.315+0000 7fa3ad600640 1 --2- 172.21.15.32:0/917488992 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3a80b3d80 0x7fa3a8071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.315+0000 7fa3a7e00640 1 -- 172.21.15.32:0/917488992 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3012348691 0 0) 0x7fa3a806e6f0 con 0x7fa3a815ac30 2024-04-17T09:57:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.315+0000 7fa3a7e00640 1 -- 172.21.15.32:0/917488992 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa38c003530 con 0x7fa3a815ac30 2024-04-17T09:57:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.315+0000 7fa3a7e00640 1 -- 172.21.15.32:0/917488992 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 66193589 0 0) 0x7fa3a806e9d0 con 0x7fa3a815b010 2024-04-17T09:57:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.315+0000 7fa3a7e00640 1 -- 172.21.15.32:0/917488992 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3a806e6f0 con 0x7fa3a815b010 2024-04-17T09:57:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.315+0000 7fa3ad600640 1 -- 172.21.15.32:0/917488992 >> v1:172.21.15.144:6789/0 conn(0x7fa3a815b010 legacy=0x7fa3a806dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.315+0000 7fa3ad600640 1 -- 172.21.15.32:0/917488992 >> v1:172.21.15.32:6789/0 conn(0x7fa3a815ac30 legacy=0x7fa3a806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:10.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.315+0000 7fa3ad600640 1 -- 172.21.15.32:0/917488992 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a806ecb0 con 0x7fa3a80b3d80 2024-04-17T09:57:10.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.319+0000 7fa3ad600640 1 --2- 172.21.15.32:0/917488992 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3a80b3d80 0x7fa3a8071e60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009dc0 tx=0x7fa3a001c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7a1dfaa6a41b056 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:10.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.319+0000 7fa3a7e00640 1 -- 172.21.15.32:0/917488992 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a00b2070 con 0x7fa3a80b3d80 2024-04-17T09:57:10.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.319+0000 7fa3a7e00640 1 -- 172.21.15.32:0/917488992 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3a00a58e0 con 0x7fa3a80b3d80 2024-04-17T09:57:10.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.319+0000 7fa3a7e00640 1 -- 172.21.15.32:0/917488992 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a00a5cb0 con 0x7fa3a80b3d80 2024-04-17T09:57:10.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.319+0000 7fa3af600640 1 -- 172.21.15.32:0/917488992 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3a80b3d80 msgr2=0x7fa3a8071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:10.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.319+0000 7fa3af600640 1 --2- 172.21.15.32:0/917488992 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3a80b3d80 0x7fa3a8071e60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009dc0 tx=0x7fa3a001c990 comp rx=0 tx=0).stop 2024-04-17T09:57:10.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.319+0000 7fa3af600640 1 -- 172.21.15.32:0/917488992 shutdown_connections 2024-04-17T09:57:10.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.319+0000 7fa3af600640 1 --2- 172.21.15.32:0/917488992 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3a80b3d80 0x7fa3a8071e60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:10.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.319+0000 7fa3af600640 1 -- 172.21.15.32:0/917488992 >> 172.21.15.32:0/917488992 conn(0x7fa3a80b2120 msgr2=0x7fa3a80b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:10.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.319+0000 7fa3af600640 1 -- 172.21.15.32:0/917488992 shutdown_connections 2024-04-17T09:57:10.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.319+0000 7fa3af600640 1 -- 172.21.15.32:0/917488992 wait complete. 2024-04-17T09:57:10.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.319+0000 7fa3af600640 1 Processor -- start 2024-04-17T09:57:10.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa3af600640 1 -- start start 2024-04-17T09:57:10.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa3af600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3a80b3d80 0x7fa3a8141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:10.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa3af600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3a815ac30 0x7fa3a8158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:10.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa3af600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3a815b010 0x7fa3a81594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:10.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa3af600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a80cddd0 con 0x7fa3a815ac30 2024-04-17T09:57:10.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa3af600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a80cdc50 con 0x7fa3a815b010 2024-04-17T09:57:10.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa3af600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa3a80cdf50 con 0x7fa3a80b3d80 2024-04-17T09:57:10.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa39fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3a815ac30 0x7fa3a8158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:10.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa3acc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3a80b3d80 0x7fa3a8141ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:10.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa39fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3a815ac30 0x7fa3a8158f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59566/0 (socket says 172.21.15.32:59566) 2024-04-17T09:57:10.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa39fe00640 1 -- 172.21.15.32:0/260530587 learned_addr learned my addr 172.21.15.32:0/260530587 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:10.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa3ad600640 1 --2- 172.21.15.32:0/260530587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3a815b010 0x7fa3a81594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:10.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa39fe00640 1 -- 172.21.15.32:0/260530587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3a80b3d80 msgr2=0x7fa3a8141ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:10.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa39fe00640 1 --2- 172.21.15.32:0/260530587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3a80b3d80 0x7fa3a8141ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:10.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa39fe00640 1 -- 172.21.15.32:0/260530587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3a815b010 msgr2=0x7fa3a81594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:10.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa39fe00640 1 --2- 172.21.15.32:0/260530587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3a815b010 0x7fa3a81594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:10.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa39fe00640 1 -- 172.21.15.32:0/260530587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a8159c40 con 0x7fa3a815ac30 2024-04-17T09:57:10.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa39fe00640 1 --2- 172.21.15.32:0/260530587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3a815ac30 0x7fa3a8158f70 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fa39000c430 tx=0x7fa39000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:10.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa3a5600640 1 -- 172.21.15.32:0/260530587 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa390017070 con 0x7fa3a815ac30 2024-04-17T09:57:10.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa3a5600640 1 -- 172.21.15.32:0/260530587 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3900052f0 con 0x7fa3a815ac30 2024-04-17T09:57:10.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa3af600640 1 -- 172.21.15.32:0/260530587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a8159f20 con 0x7fa3a815ac30 2024-04-17T09:57:10.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa3a5600640 1 -- 172.21.15.32:0/260530587 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa390003840 con 0x7fa3a815ac30 2024-04-17T09:57:10.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa3af600640 1 -- 172.21.15.32:0/260530587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a815a4e0 con 0x7fa3a815ac30 2024-04-17T09:57:10.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.323+0000 7fa3af600640 1 -- 172.21.15.32:0/260530587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3a8068a90 con 0x7fa3a815ac30 2024-04-17T09:57:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.327+0000 7fa3a5600640 1 -- 172.21.15.32:0/260530587 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa39001a070 con 0x7fa3a815ac30 2024-04-17T09:57:10.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.327+0000 7fa3a5600640 1 --2- 172.21.15.32:0/260530587 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa3840683b0 0x7fa38406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:10.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.327+0000 7fa3a5600640 1 -- 172.21.15.32:0/260530587 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5227+0+0 (secure 0 0 0) 0x7fa390085de0 con 0x7fa3a815ac30 2024-04-17T09:57:10.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.327+0000 7fa3acc00640 1 --2- 172.21.15.32:0/260530587 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa3840683b0 0x7fa38406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:10.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.327+0000 7fa3acc00640 1 --2- 172.21.15.32:0/260530587 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa3840683b0 0x7fa38406a870 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa398007fd0 tx=0x7fa398007d60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:10.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.331+0000 7fa3a5600640 1 -- 172.21.15.32:0/260530587 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa390021080 con 0x7fa3a815ac30 2024-04-17T09:57:10.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:10.499+0000 7fa3af600640 1 -- 172.21.15.32:0/260530587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa3a8072a60 con 0x7fa3a815ac30 2024-04-17T09:57:11.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.143+0000 7fa3a5600640 1 -- 172.21.15.32:0/260530587 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v66) ==== 170+0+0 (secure 0 0 0) 0x7fa390055e90 con 0x7fa3a815ac30 2024-04-17T09:57:11.145 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-04-17T09:57:11.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.147+0000 7fa3af600640 1 -- 172.21.15.32:0/260530587 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa3840683b0 msgr2=0x7fa38406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:11.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.147+0000 7fa3af600640 1 --2- 172.21.15.32:0/260530587 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa3840683b0 0x7fa38406a870 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa398007fd0 tx=0x7fa398007d60 comp rx=0 tx=0).stop 2024-04-17T09:57:11.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.147+0000 7fa3af600640 1 -- 172.21.15.32:0/260530587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3a815ac30 msgr2=0x7fa3a8158f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:11.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.147+0000 7fa3af600640 1 --2- 172.21.15.32:0/260530587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3a815ac30 0x7fa3a8158f70 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fa39000c430 tx=0x7fa39000c8f0 comp rx=0 tx=0).stop 2024-04-17T09:57:11.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.147+0000 7fa3af600640 1 -- 172.21.15.32:0/260530587 shutdown_connections 2024-04-17T09:57:11.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.147+0000 7fa3af600640 1 --2- 172.21.15.32:0/260530587 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa3840683b0 0x7fa38406a870 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:11.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.147+0000 7fa3af600640 1 --2- 172.21.15.32:0/260530587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3a815b010 0x7fa3a81594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:11.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.147+0000 7fa3af600640 1 --2- 172.21.15.32:0/260530587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3a815ac30 0x7fa3a8158f70 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:11.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.147+0000 7fa3af600640 1 --2- 172.21.15.32:0/260530587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3a80b3d80 0x7fa3a8141ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:11.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.147+0000 7fa3af600640 1 -- 172.21.15.32:0/260530587 >> 172.21.15.32:0/260530587 conn(0x7fa3a80b2120 msgr2=0x7fa3a80b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:11.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.147+0000 7fa3af600640 1 -- 172.21.15.32:0/260530587 shutdown_connections 2024-04-17T09:57:11.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.147+0000 7fa3af600640 1 -- 172.21.15.32:0/260530587 wait complete. 2024-04-17T09:57:11.187 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T09:57:11.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.331+0000 7f47ec800640 1 Processor -- start 2024-04-17T09:57:11.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.331+0000 7f47ec800640 1 -- start start 2024-04-17T09:57:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.331+0000 7f47ec800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47e406f7c0 0x7f47e406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.331+0000 7f47ec800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f47e406f270 0x7f47e406bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.331+0000 7f47ec800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47e40cc940 con 0x7f47e415b7d0 2024-04-17T09:57:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.331+0000 7f47ec800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f47e40ccc20 con 0x7f47e406f270 2024-04-17T09:57:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.331+0000 7f47ec800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f47e406e0b0 con 0x7f47e406f7c0 2024-04-17T09:57:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.331+0000 7f47e9e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f47e415b7d0 0x7f47e406e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41588/0 (socket says 172.21.15.32:41588) 2024-04-17T09:57:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.331+0000 7f47e9e00640 1 -- 172.21.15.32:0/1940332361 learned_addr learned my addr 172.21.15.32:0/1940332361 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.331+0000 7f47e9400640 1 --2- 172.21.15.32:0/1940332361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47e406f7c0 0x7f47e406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.331+0000 7f47ea800640 1 --2- 172.21.15.32:0/1940332361 >> v2:172.21.15.144:3300/0 conn(0x7f47e406f270 0x7f47e406bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:11.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.335+0000 7f47e8a00640 1 -- 172.21.15.32:0/1940332361 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 15353718 0 0) 0x7f47e40cc940 con 0x7f47e415b7d0 2024-04-17T09:57:11.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.335+0000 7f47e8a00640 1 -- 172.21.15.32:0/1940332361 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47cc005740 con 0x7f47e415b7d0 2024-04-17T09:57:11.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.335+0000 7f47ea800640 1 -- 172.21.15.32:0/1940332361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47e406f7c0 msgr2=0x7f47e406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:11.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.335+0000 7f47ea800640 1 --2- 172.21.15.32:0/1940332361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47e406f7c0 0x7f47e406ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:11.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.335+0000 7f47ea800640 1 -- 172.21.15.32:0/1940332361 >> v1:172.21.15.32:6789/0 conn(0x7f47e415b7d0 legacy=0x7f47e406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:11.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.335+0000 7f47ea800640 1 -- 172.21.15.32:0/1940332361 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e406e390 con 0x7f47e406f270 2024-04-17T09:57:11.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.335+0000 7f47e9400640 1 --2- 172.21.15.32:0/1940332361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47e406f7c0 0x7f47e406ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:57:11.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.335+0000 7f47ea800640 1 --2- 172.21.15.32:0/1940332361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47e406f270 0x7f47e406bf70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009d90 tx=0x7f47e001c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f14b89ea676ca6ee server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:11.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.335+0000 7f47e8a00640 1 -- 172.21.15.32:0/1940332361 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e00b2070 con 0x7f47e406f270 2024-04-17T09:57:11.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.335+0000 7f47e8a00640 1 -- 172.21.15.32:0/1940332361 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47e00a58b0 con 0x7f47e406f270 2024-04-17T09:57:11.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.335+0000 7f47e8a00640 1 -- 172.21.15.32:0/1940332361 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e00a5c60 con 0x7f47e406f270 2024-04-17T09:57:11.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47ec800640 1 -- 172.21.15.32:0/1940332361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47e406f270 msgr2=0x7f47e406bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:11.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47ec800640 1 --2- 172.21.15.32:0/1940332361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47e406f270 0x7f47e406bf70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009d90 tx=0x7f47e001c8e0 comp rx=0 tx=0).stop 2024-04-17T09:57:11.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47ec800640 1 -- 172.21.15.32:0/1940332361 shutdown_connections 2024-04-17T09:57:11.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47ec800640 1 --2- 172.21.15.32:0/1940332361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47e406f270 0x7f47e406bf70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:11.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47ec800640 1 --2- 172.21.15.32:0/1940332361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47e406f7c0 0x7f47e406ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:11.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47ec800640 1 -- 172.21.15.32:0/1940332361 >> 172.21.15.32:0/1940332361 conn(0x7f47e40b2120 msgr2=0x7f47e40b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:11.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47ec800640 1 -- 172.21.15.32:0/1940332361 shutdown_connections 2024-04-17T09:57:11.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47ec800640 1 -- 172.21.15.32:0/1940332361 wait complete. 2024-04-17T09:57:11.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47ec800640 1 Processor -- start 2024-04-17T09:57:11.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47ec800640 1 -- start start 2024-04-17T09:57:11.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47ec800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47e406f270 0x7f47e4139d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:11.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47ec800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e406f7c0 0x7f47e4150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:11.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47ec800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47e415b7d0 0x7f47e413a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:11.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47ec800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f47e40cee70 con 0x7f47e406f7c0 2024-04-17T09:57:11.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47ec800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f47e40cecf0 con 0x7f47e415b7d0 2024-04-17T09:57:11.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47ec800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f47e4071e60 con 0x7f47e406f270 2024-04-17T09:57:11.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47e9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e406f7c0 0x7f47e4150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:11.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47e9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e406f7c0 0x7f47e4150c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59572/0 (socket says 172.21.15.32:59572) 2024-04-17T09:57:11.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47e9400640 1 -- 172.21.15.32:0/473262477 learned_addr learned my addr 172.21.15.32:0/473262477 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:11.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47e9e00640 1 --2- 172.21.15.32:0/473262477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47e406f270 0x7f47e4139d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:11.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47ea800640 1 --2- 172.21.15.32:0/473262477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47e415b7d0 0x7f47e413a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:11.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47e9400640 1 -- 172.21.15.32:0/473262477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47e406f270 msgr2=0x7f47e4139d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:11.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47e9400640 1 --2- 172.21.15.32:0/473262477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47e406f270 0x7f47e4139d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:11.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47e9400640 1 -- 172.21.15.32:0/473262477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47e415b7d0 msgr2=0x7f47e413a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:11.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47e9400640 1 --2- 172.21.15.32:0/473262477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47e415b7d0 0x7f47e413a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:11.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.339+0000 7f47e9400640 1 -- 172.21.15.32:0/473262477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e4152820 con 0x7f47e406f7c0 2024-04-17T09:57:11.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.343+0000 7f47e9400640 1 --2- 172.21.15.32:0/473262477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e406f7c0 0x7f47e4150c20 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f47d400ebe0 tx=0x7f47d400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:11.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.343+0000 7f47de000640 1 -- 172.21.15.32:0/473262477 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d4017070 con 0x7f47e406f7c0 2024-04-17T09:57:11.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.343+0000 7f47de000640 1 -- 172.21.15.32:0/473262477 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47d400cbd0 con 0x7f47e406f7c0 2024-04-17T09:57:11.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.343+0000 7f47ec800640 1 -- 172.21.15.32:0/473262477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e4152ae0 con 0x7f47e406f7c0 2024-04-17T09:57:11.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.343+0000 7f47de000640 1 -- 172.21.15.32:0/473262477 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d4013470 con 0x7f47e406f7c0 2024-04-17T09:57:11.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.343+0000 7f47ec800640 1 -- 172.21.15.32:0/473262477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e41530a0 con 0x7f47e406f7c0 2024-04-17T09:57:11.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.343+0000 7f47ec800640 1 -- 172.21.15.32:0/473262477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e4068a90 con 0x7f47e406f7c0 2024-04-17T09:57:11.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.343+0000 7f47de000640 1 -- 172.21.15.32:0/473262477 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f47d401a070 con 0x7f47e406f7c0 2024-04-17T09:57:11.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.343+0000 7f47de000640 1 --2- 172.21.15.32:0/473262477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f47c8068380 0x7f47c806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:11.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.343+0000 7f47de000640 1 -- 172.21.15.32:0/473262477 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5609+0+0 (secure 0 0 0) 0x7f47d4086bc0 con 0x7f47e406f7c0 2024-04-17T09:57:11.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.343+0000 7f47e9e00640 1 --2- 172.21.15.32:0/473262477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f47c8068380 0x7f47c806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:11.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.347+0000 7f47e9e00640 1 --2- 172.21.15.32:0/473262477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f47c8068380 0x7f47c806a840 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f47d8007fd0 tx=0x7f47d8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:11.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.351+0000 7f47de000640 1 -- 172.21.15.32:0/473262477 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f47d4056af0 con 0x7f47e406f7c0 2024-04-17T09:57:11.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:11.551+0000 7f47ec800640 1 -- 172.21.15.32:0/473262477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f47e4151890 con 0x7f47e406f7c0 2024-04-17T09:57:12.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.163+0000 7f47de000640 1 -- 172.21.15.32:0/473262477 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v39) ==== 174+0+0 (secure 0 0 0) 0x7f47d405adf0 con 0x7f47e406f7c0 2024-04-17T09:57:12.168 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 10 and data pool 11 2024-04-17T09:57:12.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.171+0000 7f47ec800640 1 -- 172.21.15.32:0/473262477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f47c8068380 msgr2=0x7f47c806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:12.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.171+0000 7f47ec800640 1 --2- 172.21.15.32:0/473262477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f47c8068380 0x7f47c806a840 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f47d8007fd0 tx=0x7f47d8009040 comp rx=0 tx=0).stop 2024-04-17T09:57:12.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.171+0000 7f47ec800640 1 -- 172.21.15.32:0/473262477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e406f7c0 msgr2=0x7f47e4150c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:12.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.171+0000 7f47ec800640 1 --2- 172.21.15.32:0/473262477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e406f7c0 0x7f47e4150c20 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f47d400ebe0 tx=0x7f47d400c520 comp rx=0 tx=0).stop 2024-04-17T09:57:12.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.179+0000 7f47ec800640 1 -- 172.21.15.32:0/473262477 shutdown_connections 2024-04-17T09:57:12.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.179+0000 7f47ec800640 1 --2- 172.21.15.32:0/473262477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f47c8068380 0x7f47c806a840 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.179+0000 7f47ec800640 1 --2- 172.21.15.32:0/473262477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47e415b7d0 0x7f47e413a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.179+0000 7f47ec800640 1 --2- 172.21.15.32:0/473262477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e406f7c0 0x7f47e4150c20 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.179+0000 7f47ec800640 1 --2- 172.21.15.32:0/473262477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47e406f270 0x7f47e4139d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.179+0000 7f47ec800640 1 -- 172.21.15.32:0/473262477 >> 172.21.15.32:0/473262477 conn(0x7f47e40b2120 msgr2=0x7f47e4152090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:12.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.183+0000 7f47ec800640 1 -- 172.21.15.32:0/473262477 shutdown_connections 2024-04-17T09:57:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.183+0000 7f47ec800640 1 -- 172.21.15.32:0/473262477 wait complete. 2024-04-17T09:57:12.216 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:57:12.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.355+0000 7f6be4c00640 1 Processor -- start 2024-04-17T09:57:12.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.355+0000 7f6be4c00640 1 -- start start 2024-04-17T09:57:12.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6be4c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6be0072500 0x7f6be01695c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:12.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6be4c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6be006ef70 con 0x7f6be0144f20 2024-04-17T09:57:12.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6be4c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6be00aed20 con 0x7f6be01481e0 2024-04-17T09:57:12.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6be4c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f6be00b44a0 con 0x7f6be0072500 2024-04-17T09:57:12.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bdde00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6be0144f20 0x7f6be006e860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41600/0 (socket says 172.21.15.32:41600) 2024-04-17T09:57:12.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bdde00640 1 -- 172.21.15.32:0/3574460881 learned_addr learned my addr 172.21.15.32:0/3574460881 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:12.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bde800640 1 --2- 172.21.15.32:0/3574460881 >> v2:172.21.15.144:3300/0 conn(0x7f6be0072500 0x7f6be01695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:12.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bdd400640 1 -- 172.21.15.32:0/3574460881 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3104333672 0 0) 0x7f6be006ef70 con 0x7f6be0144f20 2024-04-17T09:57:12.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bdd400640 1 -- 172.21.15.32:0/3574460881 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6bc4003530 con 0x7f6be0144f20 2024-04-17T09:57:12.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bdd400640 1 -- 172.21.15.32:0/3574460881 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 862065234 0 0) 0x7f6bc4003530 con 0x7f6be0144f20 2024-04-17T09:57:12.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bdd400640 1 -- 172.21.15.32:0/3574460881 >> v2:172.21.15.144:3300/0 conn(0x7f6be0072500 msgr2=0x7f6be01695c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:12.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bdd400640 1 --2- 172.21.15.32:0/3574460881 >> v2:172.21.15.144:3300/0 conn(0x7f6be0072500 0x7f6be01695c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bdd400640 1 -- 172.21.15.32:0/3574460881 >> v1:172.21.15.144:6789/0 conn(0x7f6be01481e0 legacy=0x7f6be0071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:12.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bdd400640 1 -- 172.21.15.32:0/3574460881 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be00af000 con 0x7f6be0144f20 2024-04-17T09:57:12.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bdd400640 1 -- 172.21.15.32:0/3574460881 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f6bd0001680 con 0x7f6be0144f20 2024-04-17T09:57:12.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bdd400640 1 -- 172.21.15.32:0/3574460881 >> v1:172.21.15.32:6789/0 conn(0x7f6be0144f20 legacy=0x7f6be006e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:12.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bdd400640 1 --2- 172.21.15.32:0/3574460881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bc40039c0 0x7f6bc4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:12.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bdd400640 1 --2- 172.21.15.32:0/3574460881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bc4004810 0x7f6bc4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:12.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bdd400640 1 --2- 172.21.15.32:0/3574460881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bc40051d0 0x7f6bc400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:12.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bdd400640 1 -- 172.21.15.32:0/3574460881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6be00b44a0 con 0x7f6bc4004810 2024-04-17T09:57:12.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bdd400640 1 -- 172.21.15.32:0/3574460881 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc400c070 con 0x7f6bc40039c0 2024-04-17T09:57:12.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bdd400640 1 -- 172.21.15.32:0/3574460881 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6bc400c270 con 0x7f6bc40051d0 2024-04-17T09:57:12.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bd5400640 1 --2- 172.21.15.32:0/3574460881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bc4004810 0x7f6bc4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:12.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bdde00640 1 --2- 172.21.15.32:0/3574460881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bc40039c0 0x7f6bc4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:12.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bde800640 1 --2- 172.21.15.32:0/3574460881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bc40051d0 0x7f6bc400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:12.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bd5400640 1 -- 172.21.15.32:0/3574460881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bc40051d0 msgr2=0x7f6bc400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:12.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bd5400640 1 --2- 172.21.15.32:0/3574460881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bc40051d0 0x7f6bc400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bd5400640 1 -- 172.21.15.32:0/3574460881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bc40039c0 msgr2=0x7f6bc4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:12.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bd5400640 1 --2- 172.21.15.32:0/3574460881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bc40039c0 0x7f6bc4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bd5400640 1 -- 172.21.15.32:0/3574460881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc4003530 con 0x7f6bc4004810 2024-04-17T09:57:12.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bde800640 1 --2- 172.21.15.32:0/3574460881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bc40051d0 0x7f6bc400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:57:12.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.359+0000 7f6bde800640 1 -- 172.21.15.32:0/3574460881 reap_dead start 2024-04-17T09:57:12.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.363+0000 7f6bd5400640 1 --2- 172.21.15.32:0/3574460881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bc4004810 0x7f6bc4004c90 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8001750 tx=0x7f6bc801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171acd0664a9f0bb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:12.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.363+0000 7f6bdd400640 1 -- 172.21.15.32:0/3574460881 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc80013a0 con 0x7f6bc4004810 2024-04-17T09:57:12.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.363+0000 7f6bdd400640 1 -- 172.21.15.32:0/3574460881 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc80052f0 con 0x7f6bc4004810 2024-04-17T09:57:12.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.363+0000 7f6bdd400640 1 -- 172.21.15.32:0/3574460881 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8003840 con 0x7f6bc4004810 2024-04-17T09:57:12.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.363+0000 7f6be4c00640 1 -- 172.21.15.32:0/3574460881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bc4004810 msgr2=0x7f6bc4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:12.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.363+0000 7f6be4c00640 1 --2- 172.21.15.32:0/3574460881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bc4004810 0x7f6bc4004c90 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8001750 tx=0x7f6bc801c3c0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.363+0000 7f6be4c00640 1 -- 172.21.15.32:0/3574460881 shutdown_connections 2024-04-17T09:57:12.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.363+0000 7f6be4c00640 1 --2- 172.21.15.32:0/3574460881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bc4004810 0x7f6bc4004c90 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.363+0000 7f6be4c00640 1 -- 172.21.15.32:0/3574460881 >> 172.21.15.32:0/3574460881 conn(0x7f6be006ba30 msgr2=0x7f6be006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:12.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.363+0000 7f6be4c00640 1 -- 172.21.15.32:0/3574460881 shutdown_connections 2024-04-17T09:57:12.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.363+0000 7f6be4c00640 1 -- 172.21.15.32:0/3574460881 wait complete. 2024-04-17T09:57:12.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.363+0000 7f6be4c00640 1 Processor -- start 2024-04-17T09:57:12.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.363+0000 7f6be4c00640 1 -- start start 2024-04-17T09:57:12.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6be4c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bc4004810 0x7f6be01437d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:12.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6be4c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6be015eec0 0x7f6be00cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:12.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6be4c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6be00cd350 0x7f6be015dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:12.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6be4c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6be00af8a0 con 0x7f6be00cd350 2024-04-17T09:57:12.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6be4c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6be00af5c0 con 0x7f6be015eec0 2024-04-17T09:57:12.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6be4c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6be00afdc0 con 0x7f6bc4004810 2024-04-17T09:57:12.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6bde800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6be00cd350 0x7f6be015dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:12.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6bde800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6be00cd350 0x7f6be015dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59594/0 (socket says 172.21.15.32:59594) 2024-04-17T09:57:12.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6bde800640 1 -- 172.21.15.32:0/274696217 learned_addr learned my addr 172.21.15.32:0/274696217 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:12.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6bd5400640 1 --2- 172.21.15.32:0/274696217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6be015eec0 0x7f6be00cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:12.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6bdde00640 1 --2- 172.21.15.32:0/274696217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bc4004810 0x7f6be01437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:12.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6bde800640 1 -- 172.21.15.32:0/274696217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bc4004810 msgr2=0x7f6be01437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:12.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6bde800640 1 --2- 172.21.15.32:0/274696217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bc4004810 0x7f6be01437d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6bde800640 1 -- 172.21.15.32:0/274696217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6be015eec0 msgr2=0x7f6be00cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:12.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6bde800640 1 --2- 172.21.15.32:0/274696217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6be015eec0 0x7f6be00cce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6bde800640 1 -- 172.21.15.32:0/274696217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be0145e50 con 0x7f6be00cd350 2024-04-17T09:57:12.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6bde800640 1 --2- 172.21.15.32:0/274696217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6be00cd350 0x7f6be015dd50 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6be0072fd0 tx=0x7f6bd80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:12.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6bd6a00640 1 -- 172.21.15.32:0/274696217 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8011db0 con 0x7f6be00cd350 2024-04-17T09:57:12.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6bd6a00640 1 -- 172.21.15.32:0/274696217 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bd801c070 con 0x7f6be00cd350 2024-04-17T09:57:12.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6bd6a00640 1 -- 172.21.15.32:0/274696217 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd80166b0 con 0x7f6be00cd350 2024-04-17T09:57:12.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6be4c00640 1 -- 172.21.15.32:0/274696217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be0163330 con 0x7f6be00cd350 2024-04-17T09:57:12.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6be4c00640 1 -- 172.21.15.32:0/274696217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be00cdcd0 con 0x7f6be00cd350 2024-04-17T09:57:12.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6be4c00640 1 -- 172.21.15.32:0/274696217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be0068a90 con 0x7f6be00cd350 2024-04-17T09:57:12.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.367+0000 7f6bd6a00640 1 -- 172.21.15.32:0/274696217 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6bd8016850 con 0x7f6be00cd350 2024-04-17T09:57:12.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.371+0000 7f6bd6a00640 1 --2- 172.21.15.32:0/274696217 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6bbc068380 0x7f6bbc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:12.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.371+0000 7f6bd6a00640 1 -- 172.21.15.32:0/274696217 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f6bd8087480 con 0x7f6be00cd350 2024-04-17T09:57:12.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.371+0000 7f6bdde00640 1 --2- 172.21.15.32:0/274696217 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6bbc068380 0x7f6bbc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:12.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.371+0000 7f6bdde00640 1 --2- 172.21.15.32:0/274696217 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6bbc068380 0x7f6bbc06a840 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0001060 tx=0x7f6bd0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:12.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.375+0000 7f6bd6a00640 1 -- 172.21.15.32:0/274696217 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f6bd80579a0 con 0x7f6be00cd350 2024-04-17T09:57:12.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.539+0000 7f6be4c00640 1 -- 172.21.15.32:0/274696217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6be00cdea0 con 0x7f6be00cd350 2024-04-17T09:57:12.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.539+0000 7f6bd6a00640 1 -- 172.21.15.32:0/274696217 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+16693 (secure 0 0 0) 0x7f6bd80572a0 con 0x7f6be00cd350 2024-04-17T09:57:12.543 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:57:12.543 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":67,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:57:11.558221+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:57:09.948964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-17T09:57:10.512460+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:57:12.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.543+0000 7f6be4c00640 1 -- 172.21.15.32:0/274696217 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6bbc068380 msgr2=0x7f6bbc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:12.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.543+0000 7f6be4c00640 1 --2- 172.21.15.32:0/274696217 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6bbc068380 0x7f6bbc06a840 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0001060 tx=0x7f6bd0009040 comp rx=0 tx=0).stop 2024-04-17T09:57:12.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.543+0000 7f6be4c00640 1 -- 172.21.15.32:0/274696217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6be00cd350 msgr2=0x7f6be015dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:12.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.543+0000 7f6be4c00640 1 --2- 172.21.15.32:0/274696217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6be00cd350 0x7f6be015dd50 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6be0072fd0 tx=0x7f6bd80115a0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.543+0000 7f6be4c00640 1 -- 172.21.15.32:0/274696217 shutdown_connections 2024-04-17T09:57:12.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.543+0000 7f6be4c00640 1 --2- 172.21.15.32:0/274696217 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6bbc068380 0x7f6bbc06a840 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.543+0000 7f6be4c00640 1 --2- 172.21.15.32:0/274696217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6be00cd350 0x7f6be015dd50 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.543+0000 7f6be4c00640 1 --2- 172.21.15.32:0/274696217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6be015eec0 0x7f6be00cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.543+0000 7f6be4c00640 1 --2- 172.21.15.32:0/274696217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bc4004810 0x7f6be01437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.543+0000 7f6be4c00640 1 -- 172.21.15.32:0/274696217 >> 172.21.15.32:0/274696217 conn(0x7f6be006ba30 msgr2=0x7f6be0140760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:12.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.543+0000 7f6be4c00640 1 -- 172.21.15.32:0/274696217 shutdown_connections 2024-04-17T09:57:12.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.543+0000 7f6be4c00640 1 -- 172.21.15.32:0/274696217 wait complete. 2024-04-17T09:57:12.565 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:57:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.703+0000 7fcf70e00640 1 Processor -- start 2024-04-17T09:57:12.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.703+0000 7fcf70e00640 1 -- start start 2024-04-17T09:57:12.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.703+0000 7fcf70e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcf6c15baa0 0x7fcf6c15bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:12.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.703+0000 7fcf70e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcf6c06ef60 0x7fcf6c06f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:12.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.703+0000 7fcf70e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf6c072bb0 con 0x7fcf6c144e90 2024-04-17T09:57:12.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.703+0000 7fcf70e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fcf6c06f8f0 con 0x7fcf6c06ef60 2024-04-17T09:57:12.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.703+0000 7fcf70e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fcf6c145270 con 0x7fcf6c15baa0 2024-04-17T09:57:12.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.703+0000 7fcf6a800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcf6c06ef60 0x7fcf6c06f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:12.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.703+0000 7fcf6a800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcf6c06ef60 0x7fcf6c06f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59602/0 (socket says 172.21.15.32:59602) 2024-04-17T09:57:12.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.703+0000 7fcf6a800640 1 -- 172.21.15.32:0/4231332296 learned_addr learned my addr 172.21.15.32:0/4231332296 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:12.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.707+0000 7fcf61400640 1 --2- 172.21.15.32:0/4231332296 >> v2:172.21.15.144:3300/0 conn(0x7fcf6c15baa0 0x7fcf6c15bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:12.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.707+0000 7fcf6a800640 1 -- 172.21.15.32:0/4231332296 >> v2:172.21.15.144:3300/0 conn(0x7fcf6c15baa0 msgr2=0x7fcf6c15bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:12.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.707+0000 7fcf6a800640 1 --2- 172.21.15.32:0/4231332296 >> v2:172.21.15.144:3300/0 conn(0x7fcf6c15baa0 0x7fcf6c15bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.707+0000 7fcf6a800640 1 -- 172.21.15.32:0/4231332296 >> v1:172.21.15.32:6789/0 conn(0x7fcf6c144e90 legacy=0x7fcf6c0b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:12.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.707+0000 7fcf6a800640 1 -- 172.21.15.32:0/4231332296 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6c0b3dc0 con 0x7fcf6c06ef60 2024-04-17T09:57:12.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.707+0000 7fcf6a800640 1 --2- 172.21.15.32:0/4231332296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf6c06ef60 0x7fcf6c06f3b0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c06fb40 tx=0x7fcf6401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4629d24c2eb4fc89 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:12.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.707+0000 7fcf69400640 1 -- 172.21.15.32:0/4231332296 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf640a57a0 con 0x7fcf6c06ef60 2024-04-17T09:57:12.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.707+0000 7fcf69400640 1 -- 172.21.15.32:0/4231332296 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf640a5940 con 0x7fcf6c06ef60 2024-04-17T09:57:12.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.707+0000 7fcf69400640 1 -- 172.21.15.32:0/4231332296 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf640a5d10 con 0x7fcf6c06ef60 2024-04-17T09:57:12.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf70e00640 1 -- 172.21.15.32:0/4231332296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf6c06ef60 msgr2=0x7fcf6c06f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:12.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf70e00640 1 --2- 172.21.15.32:0/4231332296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf6c06ef60 0x7fcf6c06f3b0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c06fb40 tx=0x7fcf6401c9f0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf70e00640 1 -- 172.21.15.32:0/4231332296 shutdown_connections 2024-04-17T09:57:12.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf70e00640 1 --2- 172.21.15.32:0/4231332296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf6c06ef60 0x7fcf6c06f3b0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf70e00640 1 --2- 172.21.15.32:0/4231332296 >> v2:172.21.15.144:3300/0 conn(0x7fcf6c15baa0 0x7fcf6c15bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf70e00640 1 -- 172.21.15.32:0/4231332296 >> 172.21.15.32:0/4231332296 conn(0x7fcf6c06ba30 msgr2=0x7fcf6c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:12.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf70e00640 1 -- 172.21.15.32:0/4231332296 shutdown_connections 2024-04-17T09:57:12.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf70e00640 1 -- 172.21.15.32:0/4231332296 wait complete. 2024-04-17T09:57:12.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf70e00640 1 Processor -- start 2024-04-17T09:57:12.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf70e00640 1 -- start start 2024-04-17T09:57:12.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf70e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf6c06ef60 0x7fcf6c0c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:12.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf70e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf6c144e90 0x7fcf6c0b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:12.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf70e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf6c15baa0 0x7fcf6c0adac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:12.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf70e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c0727f0 con 0x7fcf6c144e90 2024-04-17T09:57:12.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf70e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c072670 con 0x7fcf6c15baa0 2024-04-17T09:57:12.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf70e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcf6c072970 con 0x7fcf6c06ef60 2024-04-17T09:57:12.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf61400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf6c144e90 0x7fcf6c0b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:12.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf61400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf6c144e90 0x7fcf6c0b13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59604/0 (socket says 172.21.15.32:59604) 2024-04-17T09:57:12.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf61400640 1 -- 172.21.15.32:0/3888896214 learned_addr learned my addr 172.21.15.32:0/3888896214 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:12.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf6a800640 1 --2- 172.21.15.32:0/3888896214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf6c15baa0 0x7fcf6c0adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:12.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf69e00640 1 --2- 172.21.15.32:0/3888896214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf6c06ef60 0x7fcf6c0c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:12.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf61400640 1 -- 172.21.15.32:0/3888896214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf6c06ef60 msgr2=0x7fcf6c0c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:12.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf61400640 1 --2- 172.21.15.32:0/3888896214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf6c06ef60 0x7fcf6c0c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf61400640 1 -- 172.21.15.32:0/3888896214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf6c15baa0 msgr2=0x7fcf6c0adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:12.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf61400640 1 --2- 172.21.15.32:0/3888896214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf6c15baa0 0x7fcf6c0adac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf61400640 1 -- 172.21.15.32:0/3888896214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6c0b1bf0 con 0x7fcf6c144e90 2024-04-17T09:57:12.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.711+0000 7fcf61400640 1 --2- 172.21.15.32:0/3888896214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf6c144e90 0x7fcf6c0b13a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400e930 tx=0x7fcf5400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:12.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.715+0000 7fcf62a00640 1 -- 172.21.15.32:0/3888896214 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf54017070 con 0x7fcf6c144e90 2024-04-17T09:57:12.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.715+0000 7fcf62a00640 1 -- 172.21.15.32:0/3888896214 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf5400ca00 con 0x7fcf6c144e90 2024-04-17T09:57:12.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.715+0000 7fcf62a00640 1 -- 172.21.15.32:0/3888896214 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5400cdb0 con 0x7fcf6c144e90 2024-04-17T09:57:12.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.715+0000 7fcf70e00640 1 -- 172.21.15.32:0/3888896214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf6c0b21b0 con 0x7fcf6c144e90 2024-04-17T09:57:12.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.715+0000 7fcf70e00640 1 -- 172.21.15.32:0/3888896214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf6c0b2770 con 0x7fcf6c144e90 2024-04-17T09:57:12.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.715+0000 7fcf70e00640 1 -- 172.21.15.32:0/3888896214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf6c0b1940 con 0x7fcf6c144e90 2024-04-17T09:57:12.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.715+0000 7fcf62a00640 1 -- 172.21.15.32:0/3888896214 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcf5401a070 con 0x7fcf6c144e90 2024-04-17T09:57:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.715+0000 7fcf62a00640 1 --2- 172.21.15.32:0/3888896214 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcf48068380 0x7fcf4806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.715+0000 7fcf62a00640 1 -- 172.21.15.32:0/3888896214 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7fcf54086da0 con 0x7fcf6c144e90 2024-04-17T09:57:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.715+0000 7fcf69e00640 1 --2- 172.21.15.32:0/3888896214 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcf48068380 0x7fcf4806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.715+0000 7fcf69e00640 1 --2- 172.21.15.32:0/3888896214 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcf48068380 0x7fcf4806a840 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c001650 tx=0x7fcf5c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.719+0000 7fcf62a00640 1 -- 172.21.15.32:0/3888896214 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fcf54057410 con 0x7fcf6c144e90 2024-04-17T09:57:12.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.883+0000 7fcf70e00640 1 -- 172.21.15.32:0/3888896214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf6c068a90 con 0x7fcf6c144e90 2024-04-17T09:57:12.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.887+0000 7fcf62a00640 1 -- 172.21.15.32:0/3888896214 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+16693 (secure 0 0 0) 0x7fcf54056d10 con 0x7fcf6c144e90 2024-04-17T09:57:12.890 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:57:12.890 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":67,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:57:11.558221+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:57:09.948964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-17T09:57:10.512460+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:57:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.887+0000 7fcf70e00640 1 -- 172.21.15.32:0/3888896214 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcf48068380 msgr2=0x7fcf4806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.887+0000 7fcf70e00640 1 --2- 172.21.15.32:0/3888896214 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcf48068380 0x7fcf4806a840 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c001650 tx=0x7fcf5c009040 comp rx=0 tx=0).stop 2024-04-17T09:57:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.887+0000 7fcf70e00640 1 -- 172.21.15.32:0/3888896214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf6c144e90 msgr2=0x7fcf6c0b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.887+0000 7fcf70e00640 1 --2- 172.21.15.32:0/3888896214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf6c144e90 0x7fcf6c0b13a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400e930 tx=0x7fcf5400edf0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.887+0000 7fcf70e00640 1 -- 172.21.15.32:0/3888896214 shutdown_connections 2024-04-17T09:57:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.887+0000 7fcf70e00640 1 --2- 172.21.15.32:0/3888896214 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcf48068380 0x7fcf4806a840 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.887+0000 7fcf70e00640 1 --2- 172.21.15.32:0/3888896214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf6c15baa0 0x7fcf6c0adac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.887+0000 7fcf70e00640 1 --2- 172.21.15.32:0/3888896214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf6c144e90 0x7fcf6c0b13a0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.887+0000 7fcf70e00640 1 --2- 172.21.15.32:0/3888896214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf6c06ef60 0x7fcf6c0c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.887+0000 7fcf70e00640 1 -- 172.21.15.32:0/3888896214 >> 172.21.15.32:0/3888896214 conn(0x7fcf6c06ba30 msgr2=0x7fcf6c06be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.887+0000 7fcf70e00640 1 -- 172.21.15.32:0/3888896214 shutdown_connections 2024-04-17T09:57:12.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:12.887+0000 7fcf70e00640 1 -- 172.21.15.32:0/3888896214 wait complete. 2024-04-17T09:57:12.910 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T09:57:13.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.051+0000 7fdbd4800640 1 Processor -- start 2024-04-17T09:57:13.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.051+0000 7fdbd4800640 1 -- start start 2024-04-17T09:57:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.051+0000 7fdbd4800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdbcc06f7c0 0x7fdbcc06ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.051+0000 7fdbd4800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdbcc06f270 0x7fdbcc06bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.051+0000 7fdbd4800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbcc0cc940 con 0x7fdbcc15b7d0 2024-04-17T09:57:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.051+0000 7fdbd4800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fdbcc0ccc20 con 0x7fdbcc06f270 2024-04-17T09:57:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.051+0000 7fdbd4800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fdbcc06e0b0 con 0x7fdbcc06f7c0 2024-04-17T09:57:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.051+0000 7fdbd2800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdbcc06f270 0x7fdbcc06bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.051+0000 7fdbd2800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdbcc06f270 0x7fdbcc06bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59618/0 (socket says 172.21.15.32:59618) 2024-04-17T09:57:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.051+0000 7fdbd2800640 1 -- 172.21.15.32:0/4098471574 learned_addr learned my addr 172.21.15.32:0/4098471574 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.055+0000 7fdbd2800640 1 -- 172.21.15.32:0/4098471574 >> v2:172.21.15.144:3300/0 conn(0x7fdbcc06f7c0 msgr2=0x7fdbcc06ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.055+0000 7fdbd2800640 1 --2- 172.21.15.32:0/4098471574 >> v2:172.21.15.144:3300/0 conn(0x7fdbcc06f7c0 0x7fdbcc06ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.055+0000 7fdbd2800640 1 -- 172.21.15.32:0/4098471574 >> v1:172.21.15.144:6789/0 conn(0x7fdbcc15b7d0 legacy=0x7fdbcc06e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.055+0000 7fdbd2800640 1 -- 172.21.15.32:0/4098471574 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbcc06e390 con 0x7fdbcc06f270 2024-04-17T09:57:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.055+0000 7fdbd2800640 1 --2- 172.21.15.32:0/4098471574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbcc06f270 0x7fdbcc06bf70 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0b58f0 tx=0x7fdbc801eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4d61ce916d9fbc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.055+0000 7fdbd0a00640 1 -- 172.21.15.32:0/4098471574 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc800b2d0 con 0x7fdbcc06f270 2024-04-17T09:57:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.055+0000 7fdbd0a00640 1 -- 172.21.15.32:0/4098471574 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbc800b470 con 0x7fdbcc06f270 2024-04-17T09:57:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.055+0000 7fdbd0a00640 1 -- 172.21.15.32:0/4098471574 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc80a7b20 con 0x7fdbcc06f270 2024-04-17T09:57:13.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.055+0000 7fdbd4800640 1 -- 172.21.15.32:0/4098471574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbcc06f270 msgr2=0x7fdbcc06bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:13.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.055+0000 7fdbd4800640 1 --2- 172.21.15.32:0/4098471574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbcc06f270 0x7fdbcc06bf70 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0b58f0 tx=0x7fdbc801eb20 comp rx=0 tx=0).stop 2024-04-17T09:57:13.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.055+0000 7fdbd4800640 1 -- 172.21.15.32:0/4098471574 shutdown_connections 2024-04-17T09:57:13.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.055+0000 7fdbd4800640 1 --2- 172.21.15.32:0/4098471574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbcc06f270 0x7fdbcc06bf70 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.055+0000 7fdbd4800640 1 --2- 172.21.15.32:0/4098471574 >> v2:172.21.15.144:3300/0 conn(0x7fdbcc06f7c0 0x7fdbcc06ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.055+0000 7fdbd4800640 1 -- 172.21.15.32:0/4098471574 >> 172.21.15.32:0/4098471574 conn(0x7fdbcc0b2120 msgr2=0x7fdbcc0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.055+0000 7fdbd4800640 1 -- 172.21.15.32:0/4098471574 shutdown_connections 2024-04-17T09:57:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbd4800640 1 -- 172.21.15.32:0/4098471574 wait complete. 2024-04-17T09:57:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbd4800640 1 Processor -- start 2024-04-17T09:57:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbd4800640 1 -- start start 2024-04-17T09:57:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbd4800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdbcc06f270 0x7fdbcc157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbd4800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbcc06f7c0 0x7fdbcc152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbd4800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbcc15b7d0 0x7fdbcc15be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbd4800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc0cee30 con 0x7fdbcc06f7c0 2024-04-17T09:57:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbd4800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc0cecb0 con 0x7fdbcc15b7d0 2024-04-17T09:57:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbd4800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdbcc071e60 con 0x7fdbcc06f270 2024-04-17T09:57:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbd1400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbcc06f7c0 0x7fdbcc152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbd1400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbcc06f7c0 0x7fdbcc152230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59620/0 (socket says 172.21.15.32:59620) 2024-04-17T09:57:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbd1400640 1 -- 172.21.15.32:0/4156451524 learned_addr learned my addr 172.21.15.32:0/4156451524 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbd2800640 1 --2- 172.21.15.32:0/4156451524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbcc15b7d0 0x7fdbcc15be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbd1e00640 1 --2- 172.21.15.32:0/4156451524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdbcc06f270 0x7fdbcc157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbd1400640 1 -- 172.21.15.32:0/4156451524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdbcc06f270 msgr2=0x7fdbcc157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbd1400640 1 --2- 172.21.15.32:0/4156451524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdbcc06f270 0x7fdbcc157cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbd1400640 1 -- 172.21.15.32:0/4156451524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbcc15b7d0 msgr2=0x7fdbcc15be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbd1400640 1 --2- 172.21.15.32:0/4156451524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbcc15b7d0 0x7fdbcc15be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbd1400640 1 -- 172.21.15.32:0/4156451524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbcc156ab0 con 0x7fdbcc06f7c0 2024-04-17T09:57:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbd1400640 1 --2- 172.21.15.32:0/4156451524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbcc06f7c0 0x7fdbcc152230 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00b580 tx=0x7fdbbc00ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbc6000640 1 -- 172.21.15.32:0/4156451524 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc014070 con 0x7fdbcc06f7c0 2024-04-17T09:57:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbc6000640 1 -- 172.21.15.32:0/4156451524 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbbc010890 con 0x7fdbcc06f7c0 2024-04-17T09:57:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbc6000640 1 -- 172.21.15.32:0/4156451524 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc010c80 con 0x7fdbcc06f7c0 2024-04-17T09:57:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbd4800640 1 -- 172.21.15.32:0/4156451524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbcc156d90 con 0x7fdbcc06f7c0 2024-04-17T09:57:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.059+0000 7fdbd4800640 1 -- 172.21.15.32:0/4156451524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbcc157350 con 0x7fdbcc06f7c0 2024-04-17T09:57:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.063+0000 7fdbd4800640 1 -- 172.21.15.32:0/4156451524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbcc068a90 con 0x7fdbcc06f7c0 2024-04-17T09:57:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.063+0000 7fdbc6000640 1 -- 172.21.15.32:0/4156451524 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdbbc017070 con 0x7fdbcc06f7c0 2024-04-17T09:57:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.063+0000 7fdbc6000640 1 --2- 172.21.15.32:0/4156451524 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdbb0068380 0x7fdbb006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.063+0000 7fdbc6000640 1 -- 172.21.15.32:0/4156451524 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7fdbbc086740 con 0x7fdbcc06f7c0 2024-04-17T09:57:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.063+0000 7fdbd1e00640 1 --2- 172.21.15.32:0/4156451524 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdbb0068380 0x7fdbb006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.067+0000 7fdbd1e00640 1 --2- 172.21.15.32:0/4156451524 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdbb0068380 0x7fdbb006a840 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00098e0 tx=0x7fdbc0009060 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:13.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.067+0000 7fdbc6000640 1 -- 172.21.15.32:0/4156451524 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fdbbc0566a0 con 0x7fdbcc06f7c0 2024-04-17T09:57:13.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:13.271+0000 7fdbd4800640 1 -- 172.21.15.32:0/4156451524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdbcc0b4660 con 0x7fdbcc06f7c0 2024-04-17T09:57:13.572 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:57:13.576 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:57:14.139 INFO:tasks.ceph.mds.a.smithi032.stderr:2024-04-17T09:57:14.135+0000 7f99a4600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T09:57:14.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.175+0000 7fdbc6000640 1 -- 172.21.15.32:0/4156451524 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v41) ==== 118+0+0 (secure 0 0 0) 0x7fdbbc010e20 con 0x7fdbcc06f7c0 2024-04-17T09:57:14.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.179+0000 7fdbd4800640 1 -- 172.21.15.32:0/4156451524 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdbb0068380 msgr2=0x7fdbb006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:14.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.179+0000 7fdbd4800640 1 --2- 172.21.15.32:0/4156451524 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdbb0068380 0x7fdbb006a840 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00098e0 tx=0x7fdbc0009060 comp rx=0 tx=0).stop 2024-04-17T09:57:14.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.179+0000 7fdbd4800640 1 -- 172.21.15.32:0/4156451524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbcc06f7c0 msgr2=0x7fdbcc152230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:14.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.179+0000 7fdbd4800640 1 --2- 172.21.15.32:0/4156451524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbcc06f7c0 0x7fdbcc152230 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00b580 tx=0x7fdbbc00ba40 comp rx=0 tx=0).stop 2024-04-17T09:57:14.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.179+0000 7fdbd4800640 1 -- 172.21.15.32:0/4156451524 shutdown_connections 2024-04-17T09:57:14.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.179+0000 7fdbd4800640 1 --2- 172.21.15.32:0/4156451524 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdbb0068380 0x7fdbb006a840 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:14.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.179+0000 7fdbd4800640 1 --2- 172.21.15.32:0/4156451524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbcc15b7d0 0x7fdbcc15be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:14.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.179+0000 7fdbd4800640 1 --2- 172.21.15.32:0/4156451524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbcc06f7c0 0x7fdbcc152230 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:14.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.179+0000 7fdbd4800640 1 --2- 172.21.15.32:0/4156451524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdbcc06f270 0x7fdbcc157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:14.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.179+0000 7fdbd4800640 1 -- 172.21.15.32:0/4156451524 >> 172.21.15.32:0/4156451524 conn(0x7fdbcc0b2120 msgr2=0x7fdbcc0b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:14.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.179+0000 7fdbd4800640 1 -- 172.21.15.32:0/4156451524 shutdown_connections 2024-04-17T09:57:14.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.179+0000 7fdbd4800640 1 -- 172.21.15.32:0/4156451524 wait complete. 2024-04-17T09:57:14.205 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-17T09:57:14.205 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-17T09:57:14.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.351+0000 7f9b75000640 1 Processor -- start 2024-04-17T09:57:14.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.351+0000 7f9b75000640 1 -- start start 2024-04-17T09:57:14.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.355+0000 7f9b75000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b7006ba30 0x7f9b7006be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:14.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.355+0000 7f9b75000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b700cf370 0x7f9b700cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:14.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.355+0000 7f9b75000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9b70160350 0x7f9b701607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:14.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.355+0000 7f9b75000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9b7006c350 con 0x7f9b7006ba30 2024-04-17T09:57:14.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.355+0000 7f9b75000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f9b700d0130 con 0x7f9b70160350 2024-04-17T09:57:14.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.355+0000 7f9b75000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9b7015d090 con 0x7f9b700cf370 2024-04-17T09:57:14.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.355+0000 7f9b6de00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b7006ba30 0x7f9b7006be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:14.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.355+0000 7f9b6de00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b7006ba30 0x7f9b7006be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59626/0 (socket says 172.21.15.32:59626) 2024-04-17T09:57:14.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.355+0000 7f9b6de00640 1 -- 172.21.15.32:0/18472149 learned_addr learned my addr 172.21.15.32:0/18472149 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:14.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.355+0000 7f9b6e800640 1 --2- 172.21.15.32:0/18472149 >> v2:172.21.15.144:3300/0 conn(0x7f9b70160350 0x7f9b701607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:14.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.355+0000 7f9b65400640 1 --2- 172.21.15.32:0/18472149 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b700cf370 0x7f9b700cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:14.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.355+0000 7f9b6de00640 1 -- 172.21.15.32:0/18472149 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b700cf370 msgr2=0x7f9b700cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:14.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.355+0000 7f9b6de00640 1 --2- 172.21.15.32:0/18472149 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b700cf370 0x7f9b700cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:14.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.355+0000 7f9b6de00640 1 -- 172.21.15.32:0/18472149 >> v2:172.21.15.144:3300/0 conn(0x7f9b70160350 msgr2=0x7f9b701607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:14.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.355+0000 7f9b6de00640 1 --2- 172.21.15.32:0/18472149 >> v2:172.21.15.144:3300/0 conn(0x7f9b70160350 0x7f9b701607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:14.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.355+0000 7f9b6de00640 1 -- 172.21.15.32:0/18472149 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7015d370 con 0x7f9b7006ba30 2024-04-17T09:57:14.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.355+0000 7f9b65400640 1 --2- 172.21.15.32:0/18472149 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b700cf370 0x7f9b700cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:57:14.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.355+0000 7f9b6e800640 1 --2- 172.21.15.32:0/18472149 >> v2:172.21.15.144:3300/0 conn(0x7f9b70160350 0x7f9b701607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:57:14.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.355+0000 7f9b6de00640 1 --2- 172.21.15.32:0/18472149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b7006ba30 0x7f9b7006be10 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f9b700b58d0 tx=0x7f9b6001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca307025bb66fb39 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:14.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b6d400640 1 -- 172.21.15.32:0/18472149 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b600a7070 con 0x7f9b7006ba30 2024-04-17T09:57:14.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b6d400640 1 -- 172.21.15.32:0/18472149 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b600b0070 con 0x7f9b7006ba30 2024-04-17T09:57:14.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b6d400640 1 -- 172.21.15.32:0/18472149 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b600ab890 con 0x7f9b7006ba30 2024-04-17T09:57:14.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b75000640 1 -- 172.21.15.32:0/18472149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b7006ba30 msgr2=0x7f9b7006be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:14.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b75000640 1 --2- 172.21.15.32:0/18472149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b7006ba30 0x7f9b7006be10 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f9b700b58d0 tx=0x7f9b6001c4f0 comp rx=0 tx=0).stop 2024-04-17T09:57:14.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b75000640 1 -- 172.21.15.32:0/18472149 shutdown_connections 2024-04-17T09:57:14.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b75000640 1 --2- 172.21.15.32:0/18472149 >> v2:172.21.15.144:3300/0 conn(0x7f9b70160350 0x7f9b701607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:14.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b75000640 1 --2- 172.21.15.32:0/18472149 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b700cf370 0x7f9b700cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:14.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b75000640 1 --2- 172.21.15.32:0/18472149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b7006ba30 0x7f9b7006be10 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:14.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b75000640 1 -- 172.21.15.32:0/18472149 >> 172.21.15.32:0/18472149 conn(0x7f9b700b2120 msgr2=0x7f9b700b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:14.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b75000640 1 -- 172.21.15.32:0/18472149 shutdown_connections 2024-04-17T09:57:14.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b75000640 1 -- 172.21.15.32:0/18472149 wait complete. 2024-04-17T09:57:14.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b75000640 1 Processor -- start 2024-04-17T09:57:14.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b75000640 1 -- start start 2024-04-17T09:57:14.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b75000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b7006ba30 0x7f9b7013f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:14.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b75000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b700cf370 0x7f9b701fb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:14.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b75000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b70160350 0x7f9b70142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:14.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b75000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9b700cfea0 con 0x7f9b700cf370 2024-04-17T09:57:14.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b75000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9b700cfd20 con 0x7f9b70160350 2024-04-17T09:57:14.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b75000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9b7015db70 con 0x7f9b7006ba30 2024-04-17T09:57:14.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b65400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b700cf370 0x7f9b701fb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:14.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b65400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b700cf370 0x7f9b701fb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59642/0 (socket says 172.21.15.32:59642) 2024-04-17T09:57:14.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b65400640 1 -- 172.21.15.32:0/3260535492 learned_addr learned my addr 172.21.15.32:0/3260535492 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:14.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b6e800640 1 --2- 172.21.15.32:0/3260535492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b70160350 0x7f9b70142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:14.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.359+0000 7f9b6de00640 1 --2- 172.21.15.32:0/3260535492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b7006ba30 0x7f9b7013f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:14.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.363+0000 7f9b65400640 1 -- 172.21.15.32:0/3260535492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b7006ba30 msgr2=0x7f9b7013f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:14.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.363+0000 7f9b65400640 1 --2- 172.21.15.32:0/3260535492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b7006ba30 0x7f9b7013f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:14.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.363+0000 7f9b65400640 1 -- 172.21.15.32:0/3260535492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b70160350 msgr2=0x7f9b70142fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:14.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.363+0000 7f9b65400640 1 --2- 172.21.15.32:0/3260535492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b70160350 0x7f9b70142fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:14.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.363+0000 7f9b65400640 1 -- 172.21.15.32:0/3260535492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b700b4090 con 0x7f9b700cf370 2024-04-17T09:57:14.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.363+0000 7f9b65400640 1 --2- 172.21.15.32:0/3260535492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b700cf370 0x7f9b701fb800 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800c7c0 tx=0x7f9b5800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:14.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.363+0000 7f9b66a00640 1 -- 172.21.15.32:0/3260535492 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b58015070 con 0x7f9b700cf370 2024-04-17T09:57:14.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.363+0000 7f9b66a00640 1 -- 172.21.15.32:0/3260535492 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b58007a30 con 0x7f9b700cf370 2024-04-17T09:57:14.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.363+0000 7f9b66a00640 1 -- 172.21.15.32:0/3260535492 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b58007e00 con 0x7f9b700cf370 2024-04-17T09:57:14.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.363+0000 7f9b75000640 1 -- 172.21.15.32:0/3260535492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b700b3ad0 con 0x7f9b700cf370 2024-04-17T09:57:14.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.363+0000 7f9b75000640 1 -- 172.21.15.32:0/3260535492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b701fb2e0 con 0x7f9b700cf370 2024-04-17T09:57:14.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.363+0000 7f9b75000640 1 -- 172.21.15.32:0/3260535492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b70068a90 con 0x7f9b700cf370 2024-04-17T09:57:14.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.363+0000 7f9b66a00640 1 -- 172.21.15.32:0/3260535492 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9b58018070 con 0x7f9b700cf370 2024-04-17T09:57:14.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.363+0000 7f9b66a00640 1 --2- 172.21.15.32:0/3260535492 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9b4c068380 0x7f9b4c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:14.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.363+0000 7f9b66a00640 1 -- 172.21.15.32:0/3260535492 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f9b58086880 con 0x7f9b700cf370 2024-04-17T09:57:14.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.363+0000 7f9b6de00640 1 --2- 172.21.15.32:0/3260535492 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9b4c068380 0x7f9b4c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:14.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.367+0000 7f9b6de00640 1 --2- 172.21.15.32:0/3260535492 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9b4c068380 0x7f9b4c06a840 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9b600af000 tx=0x7f9b600a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:14.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.367+0000 7f9b66a00640 1 -- 172.21.15.32:0/3260535492 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f9b58056750 con 0x7f9b700cf370 2024-04-17T09:57:14.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:14.567+0000 7f9b75000640 1 -- 172.21.15.32:0/3260535492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9b7006c850 con 0x7f9b700cf370 2024-04-17T09:57:15.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.183+0000 7f9b66a00640 1 -- 172.21.15.32:0/3260535492 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v42) ==== 122+0+0 (secure 0 0 0) 0x7f9b5805aa50 con 0x7f9b700cf370 2024-04-17T09:57:15.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.183+0000 7f9b75000640 1 -- 172.21.15.32:0/3260535492 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9b4c068380 msgr2=0x7f9b4c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:15.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.183+0000 7f9b75000640 1 --2- 172.21.15.32:0/3260535492 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9b4c068380 0x7f9b4c06a840 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9b600af000 tx=0x7f9b600a7000 comp rx=0 tx=0).stop 2024-04-17T09:57:15.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.183+0000 7f9b75000640 1 -- 172.21.15.32:0/3260535492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b700cf370 msgr2=0x7f9b701fb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:15.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.183+0000 7f9b75000640 1 --2- 172.21.15.32:0/3260535492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b700cf370 0x7f9b701fb800 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800c7c0 tx=0x7f9b5800cc80 comp rx=0 tx=0).stop 2024-04-17T09:57:15.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.183+0000 7f9b75000640 1 -- 172.21.15.32:0/3260535492 shutdown_connections 2024-04-17T09:57:15.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.183+0000 7f9b75000640 1 --2- 172.21.15.32:0/3260535492 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9b4c068380 0x7f9b4c06a840 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:15.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.183+0000 7f9b75000640 1 --2- 172.21.15.32:0/3260535492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b70160350 0x7f9b70142fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:15.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.187+0000 7f9b75000640 1 --2- 172.21.15.32:0/3260535492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b700cf370 0x7f9b701fb800 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:15.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.187+0000 7f9b75000640 1 --2- 172.21.15.32:0/3260535492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b7006ba30 0x7f9b7013f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:15.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.187+0000 7f9b75000640 1 -- 172.21.15.32:0/3260535492 >> 172.21.15.32:0/3260535492 conn(0x7f9b700b2120 msgr2=0x7f9b7015baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:15.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.187+0000 7f9b75000640 1 -- 172.21.15.32:0/3260535492 shutdown_connections 2024-04-17T09:57:15.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.187+0000 7f9b75000640 1 -- 172.21.15.32:0/3260535492 wait complete. 2024-04-17T09:57:15.211 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-17T09:57:15.211 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:57:15.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.355+0000 7fcc75a00640 1 Processor -- start 2024-04-17T09:57:15.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.355+0000 7fcc75a00640 1 -- start start 2024-04-17T09:57:15.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.355+0000 7fcc75a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcc70072500 0x7fcc700cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:15.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.355+0000 7fcc75a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc700cdac0 con 0x7fcc7015d8d0 2024-04-17T09:57:15.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.355+0000 7fcc75a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc7015cdf0 con 0x7fcc70145f30 2024-04-17T09:57:15.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.355+0000 7fcc75a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fcc7006f400 con 0x7fcc70072500 2024-04-17T09:57:15.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.355+0000 7fcc6f400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcc7015d8d0 0x7fcc701601d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41614/0 (socket says 172.21.15.32:41614) 2024-04-17T09:57:15.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.355+0000 7fcc6f400640 1 -- 172.21.15.32:0/2424794871 learned_addr learned my addr 172.21.15.32:0/2424794871 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:15.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.355+0000 7fcc66000640 1 --2- 172.21.15.32:0/2424794871 >> v2:172.21.15.144:3300/0 conn(0x7fcc70072500 0x7fcc700cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:15.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.359+0000 7fcc6e000640 1 -- 172.21.15.32:0/2424794871 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1497898823 0 0) 0x7fcc700cdac0 con 0x7fcc7015d8d0 2024-04-17T09:57:15.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.359+0000 7fcc6e000640 1 -- 172.21.15.32:0/2424794871 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc54003530 con 0x7fcc7015d8d0 2024-04-17T09:57:15.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.359+0000 7fcc66000640 1 -- 172.21.15.32:0/2424794871 >> v1:172.21.15.144:6789/0 conn(0x7fcc70145f30 legacy=0x7fcc70071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:15.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.359+0000 7fcc66000640 1 -- 172.21.15.32:0/2424794871 >> v1:172.21.15.32:6789/0 conn(0x7fcc7015d8d0 legacy=0x7fcc701601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:15.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.359+0000 7fcc66000640 1 -- 172.21.15.32:0/2424794871 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7015d0d0 con 0x7fcc70072500 2024-04-17T09:57:15.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.359+0000 7fcc66000640 1 --2- 172.21.15.32:0/2424794871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc70072500 0x7fcc700cce70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fcc580097c0 tx=0x7fcc5801ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6ba5d5c2fa81add server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:15.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.359+0000 7fcc6e000640 1 -- 172.21.15.32:0/2424794871 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc580b5070 con 0x7fcc70072500 2024-04-17T09:57:15.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.359+0000 7fcc6e000640 1 -- 172.21.15.32:0/2424794871 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc580a8990 con 0x7fcc70072500 2024-04-17T09:57:15.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.359+0000 7fcc6e000640 1 -- 172.21.15.32:0/2424794871 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc580a8d40 con 0x7fcc70072500 2024-04-17T09:57:15.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc75a00640 1 -- 172.21.15.32:0/2424794871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc70072500 msgr2=0x7fcc700cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:15.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc75a00640 1 --2- 172.21.15.32:0/2424794871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc70072500 0x7fcc700cce70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fcc580097c0 tx=0x7fcc5801ec90 comp rx=0 tx=0).stop 2024-04-17T09:57:15.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc75a00640 1 -- 172.21.15.32:0/2424794871 shutdown_connections 2024-04-17T09:57:15.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc75a00640 1 --2- 172.21.15.32:0/2424794871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc70072500 0x7fcc700cce70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:15.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc75a00640 1 -- 172.21.15.32:0/2424794871 >> 172.21.15.32:0/2424794871 conn(0x7fcc7006ba30 msgr2=0x7fcc7006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:15.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc75a00640 1 -- 172.21.15.32:0/2424794871 shutdown_connections 2024-04-17T09:57:15.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc75a00640 1 -- 172.21.15.32:0/2424794871 wait complete. 2024-04-17T09:57:15.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc75a00640 1 Processor -- start 2024-04-17T09:57:15.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc75a00640 1 -- start start 2024-04-17T09:57:15.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc75a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc70072500 0x7fcc701e8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:15.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc75a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc70145f30 0x7fcc7013aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:15.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc75a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc7015d8d0 0x7fcc7013afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:15.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc75a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcc700cd800 con 0x7fcc70072500 2024-04-17T09:57:15.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc75a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcc700cd680 con 0x7fcc7015d8d0 2024-04-17T09:57:15.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc75a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcc70146310 con 0x7fcc70145f30 2024-04-17T09:57:15.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc6ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc70072500 0x7fcc701e8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:15.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc6ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc70072500 0x7fcc701e8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59654/0 (socket says 172.21.15.32:59654) 2024-04-17T09:57:15.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc6ea00640 1 -- 172.21.15.32:0/2679218622 learned_addr learned my addr 172.21.15.32:0/2679218622 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:15.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc66000640 1 --2- 172.21.15.32:0/2679218622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc70145f30 0x7fcc7013aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:15.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc6f400640 1 --2- 172.21.15.32:0/2679218622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc7015d8d0 0x7fcc7013afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:15.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc6ea00640 1 -- 172.21.15.32:0/2679218622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc70145f30 msgr2=0x7fcc7013aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:15.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc6ea00640 1 --2- 172.21.15.32:0/2679218622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc70145f30 0x7fcc7013aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:15.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc6ea00640 1 -- 172.21.15.32:0/2679218622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc7015d8d0 msgr2=0x7fcc7013afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:15.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc6ea00640 1 --2- 172.21.15.32:0/2679218622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc7015d8d0 0x7fcc7013afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:15.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.363+0000 7fcc6ea00640 1 -- 172.21.15.32:0/2679218622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc701e74b0 con 0x7fcc70072500 2024-04-17T09:57:15.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.367+0000 7fcc6ea00640 1 --2- 172.21.15.32:0/2679218622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc70072500 0x7fcc701e8550 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000c450 tx=0x7fcc6000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:15.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.367+0000 7fcc67400640 1 -- 172.21.15.32:0/2679218622 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc60016020 con 0x7fcc70072500 2024-04-17T09:57:15.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.367+0000 7fcc67400640 1 -- 172.21.15.32:0/2679218622 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc600052d0 con 0x7fcc70072500 2024-04-17T09:57:15.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.367+0000 7fcc75a00640 1 -- 172.21.15.32:0/2679218622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc701e7790 con 0x7fcc70072500 2024-04-17T09:57:15.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.367+0000 7fcc67400640 1 -- 172.21.15.32:0/2679218622 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc60007260 con 0x7fcc70072500 2024-04-17T09:57:15.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.367+0000 7fcc75a00640 1 -- 172.21.15.32:0/2679218622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc701e7d50 con 0x7fcc70072500 2024-04-17T09:57:15.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.367+0000 7fcc75a00640 1 -- 172.21.15.32:0/2679218622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc70068a90 con 0x7fcc70072500 2024-04-17T09:57:15.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.367+0000 7fcc67400640 1 -- 172.21.15.32:0/2679218622 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcc60007400 con 0x7fcc70072500 2024-04-17T09:57:15.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.367+0000 7fcc67400640 1 --2- 172.21.15.32:0/2679218622 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc4c068380 0x7fcc4c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:15.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.367+0000 7fcc67400640 1 -- 172.21.15.32:0/2679218622 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fcc60086350 con 0x7fcc70072500 2024-04-17T09:57:15.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.367+0000 7fcc66000640 1 --2- 172.21.15.32:0/2679218622 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc4c068380 0x7fcc4c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:15.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.371+0000 7fcc66000640 1 --2- 172.21.15.32:0/2679218622 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc4c068380 0x7fcc4c06a840 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcc580b2000 tx=0x7fcc580aa070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:15.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.371+0000 7fcc67400640 1 -- 172.21.15.32:0/2679218622 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fcc60020020 con 0x7fcc70072500 2024-04-17T09:57:15.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.571+0000 7fcc75a00640 1 -- 172.21.15.32:0/2679218622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc70072f90 con 0x7fcc70072500 2024-04-17T09:57:15.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.571+0000 7fcc67400640 1 -- 172.21.15.32:0/2679218622 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5201 (secure 0 0 0) 0x7fcc60005480 con 0x7fcc70072500 2024-04-17T09:57:15.574 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:57:15.574 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2424282223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2424282223},{"type":"v1","addr":"172.21.15.144:6834","nonce":2424282223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3537056692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3537056692},{"type":"v1","addr":"172.21.15.144:6835","nonce":3537056692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5891,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/725333724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":725333724},{"type":"v1","addr":"172.21.15.32:6837","nonce":725333724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:57:11.558183+0000","modified":"2024-04-17T09:57:14.573355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6786},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6786":{"gid":6786,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6836/663537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":663537244},{"type":"v1","addr":"172.21.15.32:6836","nonce":663537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6786,"qdb_cluster":[6786]},"id":5}]} 2024-04-17T09:57:15.574 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 42 2024-04-17T09:57:15.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.575+0000 7fcc75a00640 1 -- 172.21.15.32:0/2679218622 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc4c068380 msgr2=0x7fcc4c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:15.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.575+0000 7fcc75a00640 1 --2- 172.21.15.32:0/2679218622 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc4c068380 0x7fcc4c06a840 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcc580b2000 tx=0x7fcc580aa070 comp rx=0 tx=0).stop 2024-04-17T09:57:15.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.575+0000 7fcc75a00640 1 -- 172.21.15.32:0/2679218622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc70072500 msgr2=0x7fcc701e8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:15.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.575+0000 7fcc75a00640 1 --2- 172.21.15.32:0/2679218622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc70072500 0x7fcc701e8550 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000c450 tx=0x7fcc6000c910 comp rx=0 tx=0).stop 2024-04-17T09:57:15.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.575+0000 7fcc75a00640 1 -- 172.21.15.32:0/2679218622 shutdown_connections 2024-04-17T09:57:15.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.575+0000 7fcc75a00640 1 --2- 172.21.15.32:0/2679218622 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcc4c068380 0x7fcc4c06a840 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:15.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.575+0000 7fcc75a00640 1 --2- 172.21.15.32:0/2679218622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc70145f30 0x7fcc7013aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:15.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.575+0000 7fcc75a00640 1 --2- 172.21.15.32:0/2679218622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc7015d8d0 0x7fcc7013afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:15.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.575+0000 7fcc75a00640 1 --2- 172.21.15.32:0/2679218622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc70072500 0x7fcc701e8550 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:15.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.575+0000 7fcc75a00640 1 -- 172.21.15.32:0/2679218622 >> 172.21.15.32:0/2679218622 conn(0x7fcc7006ba30 msgr2=0x7fcc7006be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:15.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.575+0000 7fcc75a00640 1 -- 172.21.15.32:0/2679218622 shutdown_connections 2024-04-17T09:57:15.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.575+0000 7fcc75a00640 1 -- 172.21.15.32:0/2679218622 wait complete. 2024-04-17T09:57:15.596 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:57:15.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.735+0000 7fdd65c00640 1 Processor -- start 2024-04-17T09:57:15.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.735+0000 7fdd65c00640 1 -- start start 2024-04-17T09:57:15.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.735+0000 7fdd65c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdd6015baa0 0x7fdd6015bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:15.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.735+0000 7fdd65c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd60072bb0 con 0x7fdd6006ef60 2024-04-17T09:57:15.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.735+0000 7fdd65c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd600b3dc0 con 0x7fdd60144e90 2024-04-17T09:57:15.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.735+0000 7fdd65c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fdd60145270 con 0x7fdd6015baa0 2024-04-17T09:57:15.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.735+0000 7fdd5f400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdd6006ef60 0x7fdd60071e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41616/0 (socket says 172.21.15.32:41616) 2024-04-17T09:57:15.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.735+0000 7fdd5f400640 1 -- 172.21.15.32:0/2897119071 learned_addr learned my addr 172.21.15.32:0/2897119071 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:15.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.735+0000 7fdd57e00640 1 --2- 172.21.15.32:0/2897119071 >> v2:172.21.15.144:3300/0 conn(0x7fdd6015baa0 0x7fdd6015bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:15.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.735+0000 7fdd5e000640 1 -- 172.21.15.32:0/2897119071 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3509930614 0 0) 0x7fdd60072bb0 con 0x7fdd6006ef60 2024-04-17T09:57:15.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.735+0000 7fdd5e000640 1 -- 172.21.15.32:0/2897119071 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd44005740 con 0x7fdd6006ef60 2024-04-17T09:57:15.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.735+0000 7fdd5e000640 1 -- 172.21.15.32:0/2897119071 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 603795339 0 0) 0x7fdd600b3dc0 con 0x7fdd60144e90 2024-04-17T09:57:15.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.735+0000 7fdd5e000640 1 -- 172.21.15.32:0/2897119071 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd60072bb0 con 0x7fdd60144e90 2024-04-17T09:57:15.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.735+0000 7fdd57e00640 1 -- 172.21.15.32:0/2897119071 >> v1:172.21.15.144:6789/0 conn(0x7fdd60144e90 legacy=0x7fdd600b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:15.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.735+0000 7fdd57e00640 1 -- 172.21.15.32:0/2897119071 >> v1:172.21.15.32:6789/0 conn(0x7fdd6006ef60 legacy=0x7fdd60071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:15.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.735+0000 7fdd57e00640 1 -- 172.21.15.32:0/2897119071 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd600b40a0 con 0x7fdd6015baa0 2024-04-17T09:57:15.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.739+0000 7fdd57e00640 1 --2- 172.21.15.32:0/2897119071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd6015baa0 0x7fdd6015bf30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdd480097c0 tx=0x7fdd4801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f409c7948bf87ae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:15.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.739+0000 7fdd5e000640 1 -- 172.21.15.32:0/2897119071 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd480b0070 con 0x7fdd6015baa0 2024-04-17T09:57:15.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.739+0000 7fdd5e000640 1 -- 172.21.15.32:0/2897119071 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd480ab600 con 0x7fdd6015baa0 2024-04-17T09:57:15.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.739+0000 7fdd5e000640 1 -- 172.21.15.32:0/2897119071 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd480ab9d0 con 0x7fdd6015baa0 2024-04-17T09:57:15.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.739+0000 7fdd65c00640 1 -- 172.21.15.32:0/2897119071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd6015baa0 msgr2=0x7fdd6015bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:15.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.739+0000 7fdd65c00640 1 --2- 172.21.15.32:0/2897119071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd6015baa0 0x7fdd6015bf30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdd480097c0 tx=0x7fdd4801c4a0 comp rx=0 tx=0).stop 2024-04-17T09:57:15.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.739+0000 7fdd65c00640 1 -- 172.21.15.32:0/2897119071 shutdown_connections 2024-04-17T09:57:15.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.739+0000 7fdd65c00640 1 --2- 172.21.15.32:0/2897119071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd6015baa0 0x7fdd6015bf30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:15.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.739+0000 7fdd65c00640 1 -- 172.21.15.32:0/2897119071 >> 172.21.15.32:0/2897119071 conn(0x7fdd6006ba30 msgr2=0x7fdd6006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:15.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.739+0000 7fdd65c00640 1 -- 172.21.15.32:0/2897119071 shutdown_connections 2024-04-17T09:57:15.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.739+0000 7fdd65c00640 1 -- 172.21.15.32:0/2897119071 wait complete. 2024-04-17T09:57:15.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd65c00640 1 Processor -- start 2024-04-17T09:57:15.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd65c00640 1 -- start start 2024-04-17T09:57:15.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd65c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd6006ef60 0x7fdd600c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:15.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd65c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd60144e90 0x7fdd6013fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:15.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd65c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd6015baa0 0x7fdd600ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:15.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd65c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6006e7e0 con 0x7fdd6006ef60 2024-04-17T09:57:15.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd65c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6006e660 con 0x7fdd6015baa0 2024-04-17T09:57:15.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd65c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdd6006e960 con 0x7fdd60144e90 2024-04-17T09:57:15.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd5ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd6006ef60 0x7fdd600c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:15.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd5ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd6006ef60 0x7fdd600c90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59670/0 (socket says 172.21.15.32:59670) 2024-04-17T09:57:15.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd5ea00640 1 -- 172.21.15.32:0/124358750 learned_addr learned my addr 172.21.15.32:0/124358750 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:15.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd5f400640 1 --2- 172.21.15.32:0/124358750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd6015baa0 0x7fdd600ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:15.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd57e00640 1 --2- 172.21.15.32:0/124358750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd60144e90 0x7fdd6013fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:15.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd5ea00640 1 -- 172.21.15.32:0/124358750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd60144e90 msgr2=0x7fdd6013fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:15.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd5ea00640 1 --2- 172.21.15.32:0/124358750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd60144e90 0x7fdd6013fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:15.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd5ea00640 1 -- 172.21.15.32:0/124358750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd6015baa0 msgr2=0x7fdd600ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:15.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd5ea00640 1 --2- 172.21.15.32:0/124358750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd6015baa0 0x7fdd600ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:15.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd5ea00640 1 -- 172.21.15.32:0/124358750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd600c66a0 con 0x7fdd6006ef60 2024-04-17T09:57:15.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd5ea00640 1 --2- 172.21.15.32:0/124358750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd6006ef60 0x7fdd600c90f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000c490 tx=0x7fdd5000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:15.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd56a00640 1 -- 172.21.15.32:0/124358750 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd500052d0 con 0x7fdd6006ef60 2024-04-17T09:57:15.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd56a00640 1 -- 172.21.15.32:0/124358750 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd50005470 con 0x7fdd6006ef60 2024-04-17T09:57:15.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd56a00640 1 -- 172.21.15.32:0/124358750 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd50012870 con 0x7fdd6006ef60 2024-04-17T09:57:15.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd65c00640 1 -- 172.21.15.32:0/124358750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd600c6980 con 0x7fdd6006ef60 2024-04-17T09:57:15.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.743+0000 7fdd65c00640 1 -- 172.21.15.32:0/124358750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd600c6f40 con 0x7fdd6006ef60 2024-04-17T09:57:15.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.747+0000 7fdd65c00640 1 -- 172.21.15.32:0/124358750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd60068a90 con 0x7fdd6006ef60 2024-04-17T09:57:15.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.747+0000 7fdd56a00640 1 -- 172.21.15.32:0/124358750 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdd50012a10 con 0x7fdd6006ef60 2024-04-17T09:57:15.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.747+0000 7fdd56a00640 1 --2- 172.21.15.32:0/124358750 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdd3c068380 0x7fdd3c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:15.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.747+0000 7fdd56a00640 1 -- 172.21.15.32:0/124358750 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fdd50086680 con 0x7fdd6006ef60 2024-04-17T09:57:15.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.747+0000 7fdd57e00640 1 --2- 172.21.15.32:0/124358750 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdd3c068380 0x7fdd3c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:15.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.747+0000 7fdd57e00640 1 --2- 172.21.15.32:0/124358750 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdd3c068380 0x7fdd3c06a840 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdd480ad000 tx=0x7fdd48008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:15.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.751+0000 7fdd56a00640 1 -- 172.21.15.32:0/124358750 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fdd50056c30 con 0x7fdd6006ef60 2024-04-17T09:57:15.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.915+0000 7fdd65c00640 1 -- 172.21.15.32:0/124358750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd600c9630 con 0x7fdd6006ef60 2024-04-17T09:57:15.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.919+0000 7fdd56a00640 1 -- 172.21.15.32:0/124358750 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16680 (secure 0 0 0) 0x7fdd50056530 con 0x7fdd6006ef60 2024-04-17T09:57:15.920 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:57:15.920 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":68,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:57:13.170280+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:57:09.948964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-17T09:57:10.512460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:57:15.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.919+0000 7fdd65c00640 1 -- 172.21.15.32:0/124358750 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdd3c068380 msgr2=0x7fdd3c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:15.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.919+0000 7fdd65c00640 1 --2- 172.21.15.32:0/124358750 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdd3c068380 0x7fdd3c06a840 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdd480ad000 tx=0x7fdd48008040 comp rx=0 tx=0).stop 2024-04-17T09:57:15.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.919+0000 7fdd65c00640 1 -- 172.21.15.32:0/124358750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd6006ef60 msgr2=0x7fdd600c90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:15.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.919+0000 7fdd65c00640 1 --2- 172.21.15.32:0/124358750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd6006ef60 0x7fdd600c90f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000c490 tx=0x7fdd5000c950 comp rx=0 tx=0).stop 2024-04-17T09:57:15.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.919+0000 7fdd65c00640 1 -- 172.21.15.32:0/124358750 shutdown_connections 2024-04-17T09:57:15.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.919+0000 7fdd65c00640 1 --2- 172.21.15.32:0/124358750 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdd3c068380 0x7fdd3c06a840 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:15.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.919+0000 7fdd65c00640 1 --2- 172.21.15.32:0/124358750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd60144e90 0x7fdd6013fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:15.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.919+0000 7fdd65c00640 1 --2- 172.21.15.32:0/124358750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd6015baa0 0x7fdd600ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:15.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.919+0000 7fdd65c00640 1 --2- 172.21.15.32:0/124358750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd6006ef60 0x7fdd600c90f0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:15.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.919+0000 7fdd65c00640 1 -- 172.21.15.32:0/124358750 >> 172.21.15.32:0/124358750 conn(0x7fdd6006ba30 msgr2=0x7fdd600b51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:15.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.919+0000 7fdd65c00640 1 -- 172.21.15.32:0/124358750 shutdown_connections 2024-04-17T09:57:15.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:15.919+0000 7fdd65c00640 1 -- 172.21.15.32:0/124358750 wait complete. 2024-04-17T09:57:15.941 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T09:57:15.941 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T09:57:16.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.079+0000 7f5fbb200640 1 Processor -- start 2024-04-17T09:57:16.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.079+0000 7f5fbb200640 1 -- start start 2024-04-17T09:57:16.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.079+0000 7f5fbb200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5fb4145f30 0x7f5fb4071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:16.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.079+0000 7f5fbb200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fb40cd580 0x7f5fb4072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:16.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.079+0000 7f5fbb200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fb40cdac0 con 0x7f5fb415d8d0 2024-04-17T09:57:16.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.079+0000 7f5fbb200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5fb406f400 con 0x7f5fb4145f30 2024-04-17T09:57:16.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.079+0000 7f5fbb200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5fb4146310 con 0x7f5fb40cd580 2024-04-17T09:57:16.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.079+0000 7f5fb3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5fb4145f30 0x7f5fb4071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:16.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.079+0000 7f5fb3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5fb4145f30 0x7f5fb4071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59686/0 (socket says 172.21.15.32:59686) 2024-04-17T09:57:16.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.079+0000 7f5fb3e00640 1 -- 172.21.15.32:0/1966697554 learned_addr learned my addr 172.21.15.32:0/1966697554 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:16.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.079+0000 7f5fabe00640 1 --1- 172.21.15.32:0/1966697554 >> v1:172.21.15.32:6789/0 conn(0x7f5fb415d8d0 0x7f5fb40cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41632/0 (socket says 172.21.15.32:41632) 2024-04-17T09:57:16.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.079+0000 7f5fb9200640 1 --2- 172.21.15.32:0/1966697554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fb40cd580 0x7f5fb4072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:16.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.083+0000 7f5fb3e00640 1 -- 172.21.15.32:0/1966697554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fb40cd580 msgr2=0x7f5fb4072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:16.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.083+0000 7f5fb3e00640 1 --2- 172.21.15.32:0/1966697554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fb40cd580 0x7f5fb4072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.083+0000 7f5fb3e00640 1 -- 172.21.15.32:0/1966697554 >> v1:172.21.15.32:6789/0 conn(0x7f5fb415d8d0 legacy=0x7f5fb40cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:16.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.083+0000 7f5fb3e00640 1 -- 172.21.15.32:0/1966697554 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb415cdf0 con 0x7f5fb4145f30 2024-04-17T09:57:16.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.083+0000 7f5fb3e00640 1 --2- 172.21.15.32:0/1966697554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fb4145f30 0x7f5fb4071df0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009870 tx=0x7f5fa401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b632139a4d379d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:16.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.083+0000 7f5fb3400640 1 -- 172.21.15.32:0/1966697554 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40b5070 con 0x7f5fb4145f30 2024-04-17T09:57:16.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.083+0000 7f5fb3400640 1 -- 172.21.15.32:0/1966697554 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa40a8a50 con 0x7f5fb4145f30 2024-04-17T09:57:16.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.083+0000 7f5fb3400640 1 -- 172.21.15.32:0/1966697554 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40a8e20 con 0x7f5fb4145f30 2024-04-17T09:57:16.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.083+0000 7f5fbb200640 1 -- 172.21.15.32:0/1966697554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fb4145f30 msgr2=0x7f5fb4071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:16.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.083+0000 7f5fbb200640 1 --2- 172.21.15.32:0/1966697554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fb4145f30 0x7f5fb4071df0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009870 tx=0x7f5fa401ed50 comp rx=0 tx=0).stop 2024-04-17T09:57:16.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fbb200640 1 -- 172.21.15.32:0/1966697554 shutdown_connections 2024-04-17T09:57:16.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fbb200640 1 --2- 172.21.15.32:0/1966697554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fb40cd580 0x7f5fb4072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fbb200640 1 --2- 172.21.15.32:0/1966697554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fb4145f30 0x7f5fb4071df0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fbb200640 1 -- 172.21.15.32:0/1966697554 >> 172.21.15.32:0/1966697554 conn(0x7f5fb406ba30 msgr2=0x7f5fb406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:16.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fbb200640 1 -- 172.21.15.32:0/1966697554 shutdown_connections 2024-04-17T09:57:16.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fbb200640 1 -- 172.21.15.32:0/1966697554 wait complete. 2024-04-17T09:57:16.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fbb200640 1 Processor -- start 2024-04-17T09:57:16.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fbb200640 1 -- start start 2024-04-17T09:57:16.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fbb200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fb40cd580 0x7f5fb413dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:16.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fbb200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fb4145f30 0x7f5fb40b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:16.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fbb200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fb415d8d0 0x7f5fb40b4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:16.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fbb200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb415cb90 con 0x7f5fb40cd580 2024-04-17T09:57:16.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fbb200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb415d440 con 0x7f5fb415d8d0 2024-04-17T09:57:16.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fbb200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5fb41601d0 con 0x7f5fb4145f30 2024-04-17T09:57:16.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fb3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fb40cd580 0x7f5fb413dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:16.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fb3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fb40cd580 0x7f5fb413dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59700/0 (socket says 172.21.15.32:59700) 2024-04-17T09:57:16.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fb3e00640 1 -- 172.21.15.32:0/1323438192 learned_addr learned my addr 172.21.15.32:0/1323438192 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:16.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fb3e00640 1 -- 172.21.15.32:0/1323438192 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fb4145f30 msgr2=0x7f5fb40b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:16.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fb3e00640 1 --2- 172.21.15.32:0/1323438192 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fb4145f30 0x7f5fb40b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fb3e00640 1 -- 172.21.15.32:0/1323438192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fb415d8d0 msgr2=0x7f5fb40b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:16.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fb3e00640 1 --2- 172.21.15.32:0/1323438192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fb415d8d0 0x7f5fb40b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fb3e00640 1 -- 172.21.15.32:0/1323438192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb413e4f0 con 0x7f5fb40cd580 2024-04-17T09:57:16.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fb3e00640 1 --2- 172.21.15.32:0/1323438192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fb40cd580 0x7f5fb413dfb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40b3040 tx=0x7f5fa401baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:16.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.087+0000 7f5fb0c00640 1 -- 172.21.15.32:0/1323438192 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40b5070 con 0x7f5fb40cd580 2024-04-17T09:57:16.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.091+0000 7f5fb0c00640 1 -- 172.21.15.32:0/1323438192 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa40b0d60 con 0x7f5fb40cd580 2024-04-17T09:57:16.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.091+0000 7f5fb0c00640 1 -- 172.21.15.32:0/1323438192 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40bb670 con 0x7f5fb40cd580 2024-04-17T09:57:16.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.091+0000 7f5fbb200640 1 -- 172.21.15.32:0/1323438192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb4140090 con 0x7f5fb40cd580 2024-04-17T09:57:16.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.091+0000 7f5fbb200640 1 -- 172.21.15.32:0/1323438192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb4140650 con 0x7f5fb40cd580 2024-04-17T09:57:16.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.091+0000 7f5fbb200640 1 -- 172.21.15.32:0/1323438192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb4140d10 con 0x7f5fb40cd580 2024-04-17T09:57:16.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.091+0000 7f5fb0c00640 1 -- 172.21.15.32:0/1323438192 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5fa40c2060 con 0x7f5fb40cd580 2024-04-17T09:57:16.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.091+0000 7f5fb0c00640 1 --2- 172.21.15.32:0/1323438192 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5f90068380 0x7f5f9006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:16.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.091+0000 7f5fb0c00640 1 -- 172.21.15.32:0/1323438192 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f5fa4123ee0 con 0x7f5fb40cd580 2024-04-17T09:57:16.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.091+0000 7f5fabe00640 1 --2- 172.21.15.32:0/1323438192 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5f90068380 0x7f5f9006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:16.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.091+0000 7f5fabe00640 1 --2- 172.21.15.32:0/1323438192 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5f90068380 0x7f5f9006a840 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c000f70 tx=0x7f5f9c007a10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:16.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.095+0000 7f5fb0c00640 1 -- 172.21.15.32:0/1323438192 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5fb4140d10 con 0x7f5fb40cd580 2024-04-17T09:57:16.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.279+0000 7f5fbb200640 1 -- 172.21.15.32:0/1323438192 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5fb4068a90 con 0x7f5f90068380 2024-04-17T09:57:16.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.283+0000 7f5fb0c00640 1 -- 172.21.15.32:0/1323438192 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252067 (secure 0 0 0) 0x7f5fb4068a90 con 0x7f5f90068380 2024-04-17T09:57:16.288 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:57:16.290 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T09:57:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.287+0000 7f5fbb200640 1 -- 172.21.15.32:0/1323438192 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5f90068380 msgr2=0x7f5f9006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:16.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.287+0000 7f5fbb200640 1 --2- 172.21.15.32:0/1323438192 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5f90068380 0x7f5f9006a840 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c000f70 tx=0x7f5f9c007a10 comp rx=0 tx=0).stop 2024-04-17T09:57:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.287+0000 7f5fbb200640 1 -- 172.21.15.32:0/1323438192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fb40cd580 msgr2=0x7f5fb413dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.287+0000 7f5fbb200640 1 --2- 172.21.15.32:0/1323438192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fb40cd580 0x7f5fb413dfb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40b3040 tx=0x7f5fa401baf0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.287+0000 7f5fbb200640 1 -- 172.21.15.32:0/1323438192 shutdown_connections 2024-04-17T09:57:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.287+0000 7f5fbb200640 1 --2- 172.21.15.32:0/1323438192 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5f90068380 0x7f5f9006a840 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.287+0000 7f5fbb200640 1 --2- 172.21.15.32:0/1323438192 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fb4145f30 0x7f5fb40b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.287+0000 7f5fbb200640 1 --2- 172.21.15.32:0/1323438192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fb415d8d0 0x7f5fb40b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.287+0000 7f5fbb200640 1 --2- 172.21.15.32:0/1323438192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fb40cd580 0x7f5fb413dfb0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.287+0000 7f5fbb200640 1 -- 172.21.15.32:0/1323438192 >> 172.21.15.32:0/1323438192 conn(0x7f5fb406ba30 msgr2=0x7f5fb40cce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.287+0000 7f5fbb200640 1 -- 172.21.15.32:0/1323438192 shutdown_connections 2024-04-17T09:57:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.287+0000 7f5fbb200640 1 -- 172.21.15.32:0/1323438192 wait complete. 2024-04-17T09:57:16.308 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-04-17T09:57:15.337486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248784,"kb_used_data":7624,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754725936,"statfs":{"total":773094113280,"available":772839358464,"internally_reserved":0,"allocated":7806976,"data_stored":3927636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.192705"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280340+0000","last_change":"2024-04-17T09:57:12.195669+0000","last_active":"2024-04-17T09:57:13.280340+0000","last_peered":"2024-04-17T09:57:13.280340+0000","last_clean":"2024-04-17T09:57:13.280340+0000","last_became_active":"2024-04-17T09:57:12.190252+0000","last_became_peered":"2024-04-17T09:57:12.190252+0000","last_unstale":"2024-04-17T09:57:13.280340+0000","last_undegraded":"2024-04-17T09:57:13.280340+0000","last_fullsized":"2024-04-17T09:57:13.280340+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:14:16.115515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178791+0000","last_change":"2024-04-17T09:57:12.187616+0000","last_active":"2024-04-17T09:57:13.178791+0000","last_peered":"2024-04-17T09:57:13.178791+0000","last_clean":"2024-04-17T09:57:13.178791+0000","last_became_active":"2024-04-17T09:57:12.187185+0000","last_became_peered":"2024-04-17T09:57:12.187185+0000","last_unstale":"2024-04-17T09:57:13.178791+0000","last_undegraded":"2024-04-17T09:57:13.178791+0000","last_fullsized":"2024-04-17T09:57:13.178791+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:13:41.171346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280793+0000","last_change":"2024-04-17T09:57:12.181957+0000","last_active":"2024-04-17T09:57:13.280793+0000","last_peered":"2024-04-17T09:57:13.280793+0000","last_clean":"2024-04-17T09:57:13.280793+0000","last_became_active":"2024-04-17T09:57:12.181283+0000","last_became_peered":"2024-04-17T09:57:12.181283+0000","last_unstale":"2024-04-17T09:57:13.280793+0000","last_undegraded":"2024-04-17T09:57:13.280793+0000","last_fullsized":"2024-04-17T09:57:13.280793+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:26:33.940662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178084+0000","last_change":"2024-04-17T09:57:12.188878+0000","last_active":"2024-04-17T09:57:13.178084+0000","last_peered":"2024-04-17T09:57:13.178084+0000","last_clean":"2024-04-17T09:57:13.178084+0000","last_became_active":"2024-04-17T09:57:12.188550+0000","last_became_peered":"2024-04-17T09:57:12.188550+0000","last_unstale":"2024-04-17T09:57:13.178084+0000","last_undegraded":"2024-04-17T09:57:13.178084+0000","last_fullsized":"2024-04-17T09:57:13.178084+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:11:39.227578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.281001+0000","last_change":"2024-04-17T09:57:12.190639+0000","last_active":"2024-04-17T09:57:13.281001+0000","last_peered":"2024-04-17T09:57:13.281001+0000","last_clean":"2024-04-17T09:57:13.281001+0000","last_became_active":"2024-04-17T09:57:12.190274+0000","last_became_peered":"2024-04-17T09:57:12.190274+0000","last_unstale":"2024-04-17T09:57:13.281001+0000","last_undegraded":"2024-04-17T09:57:13.281001+0000","last_fullsized":"2024-04-17T09:57:13.281001+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:32:56.287955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178105+0000","last_change":"2024-04-17T09:57:12.186965+0000","last_active":"2024-04-17T09:57:13.178105+0000","last_peered":"2024-04-17T09:57:13.178105+0000","last_clean":"2024-04-17T09:57:13.178105+0000","last_became_active":"2024-04-17T09:57:12.186585+0000","last_became_peered":"2024-04-17T09:57:12.186585+0000","last_unstale":"2024-04-17T09:57:13.178105+0000","last_undegraded":"2024-04-17T09:57:13.178105+0000","last_fullsized":"2024-04-17T09:57:13.178105+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:27:58.379259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.184494+0000","last_change":"2024-04-17T09:57:12.184163+0000","last_active":"2024-04-17T09:57:13.184494+0000","last_peered":"2024-04-17T09:57:13.184494+0000","last_clean":"2024-04-17T09:57:13.184494+0000","last_became_active":"2024-04-17T09:57:12.183604+0000","last_became_peered":"2024-04-17T09:57:12.183604+0000","last_unstale":"2024-04-17T09:57:13.184494+0000","last_undegraded":"2024-04-17T09:57:13.184494+0000","last_fullsized":"2024-04-17T09:57:13.184494+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:31:42.597310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178506+0000","last_change":"2024-04-17T09:57:12.177540+0000","last_active":"2024-04-17T09:57:13.178506+0000","last_peered":"2024-04-17T09:57:13.178506+0000","last_clean":"2024-04-17T09:57:13.178506+0000","last_became_active":"2024-04-17T09:57:12.177261+0000","last_became_peered":"2024-04-17T09:57:12.177261+0000","last_unstale":"2024-04-17T09:57:13.178506+0000","last_undegraded":"2024-04-17T09:57:13.178506+0000","last_fullsized":"2024-04-17T09:57:13.178506+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:04:36.836807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280611+0000","last_change":"2024-04-17T09:57:12.186583+0000","last_active":"2024-04-17T09:57:13.280611+0000","last_peered":"2024-04-17T09:57:13.280611+0000","last_clean":"2024-04-17T09:57:13.280611+0000","last_became_active":"2024-04-17T09:57:12.186268+0000","last_became_peered":"2024-04-17T09:57:12.186268+0000","last_unstale":"2024-04-17T09:57:13.280611+0000","last_undegraded":"2024-04-17T09:57:13.280611+0000","last_fullsized":"2024-04-17T09:57:13.280611+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:15.478742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.179029+0000","last_change":"2024-04-17T09:57:12.189431+0000","last_active":"2024-04-17T09:57:13.179029+0000","last_peered":"2024-04-17T09:57:13.179029+0000","last_clean":"2024-04-17T09:57:13.179029+0000","last_became_active":"2024-04-17T09:57:12.189155+0000","last_became_peered":"2024-04-17T09:57:12.189155+0000","last_unstale":"2024-04-17T09:57:13.179029+0000","last_undegraded":"2024-04-17T09:57:13.179029+0000","last_fullsized":"2024-04-17T09:57:13.179029+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:28:41.242185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280250+0000","last_change":"2024-04-17T09:57:12.189979+0000","last_active":"2024-04-17T09:57:13.280250+0000","last_peered":"2024-04-17T09:57:13.280250+0000","last_clean":"2024-04-17T09:57:13.280250+0000","last_became_active":"2024-04-17T09:57:12.188653+0000","last_became_peered":"2024-04-17T09:57:12.188653+0000","last_unstale":"2024-04-17T09:57:13.280250+0000","last_undegraded":"2024-04-17T09:57:13.280250+0000","last_fullsized":"2024-04-17T09:57:13.280250+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:52:11.560396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178131+0000","last_change":"2024-04-17T09:57:12.193122+0000","last_active":"2024-04-17T09:57:13.178131+0000","last_peered":"2024-04-17T09:57:13.178131+0000","last_clean":"2024-04-17T09:57:13.178131+0000","last_became_active":"2024-04-17T09:57:12.192854+0000","last_became_peered":"2024-04-17T09:57:12.192854+0000","last_unstale":"2024-04-17T09:57:13.178131+0000","last_undegraded":"2024-04-17T09:57:13.178131+0000","last_fullsized":"2024-04-17T09:57:13.178131+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:17:00.921709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.190395+0000","last_change":"2024-04-17T09:57:12.181457+0000","last_active":"2024-04-17T09:57:13.190395+0000","last_peered":"2024-04-17T09:57:13.190395+0000","last_clean":"2024-04-17T09:57:13.190395+0000","last_became_active":"2024-04-17T09:57:12.181182+0000","last_became_peered":"2024-04-17T09:57:12.181182+0000","last_unstale":"2024-04-17T09:57:13.190395+0000","last_undegraded":"2024-04-17T09:57:13.190395+0000","last_fullsized":"2024-04-17T09:57:13.190395+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:02:08.135909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280809+0000","last_change":"2024-04-17T09:57:12.192162+0000","last_active":"2024-04-17T09:57:13.280809+0000","last_peered":"2024-04-17T09:57:13.280809+0000","last_clean":"2024-04-17T09:57:13.280809+0000","last_became_active":"2024-04-17T09:57:12.191616+0000","last_became_peered":"2024-04-17T09:57:12.191616+0000","last_unstale":"2024-04-17T09:57:13.280809+0000","last_undegraded":"2024-04-17T09:57:13.280809+0000","last_fullsized":"2024-04-17T09:57:13.280809+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:36:28.915146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.190166+0000","last_change":"2024-04-17T09:57:12.184628+0000","last_active":"2024-04-17T09:57:13.190166+0000","last_peered":"2024-04-17T09:57:13.190166+0000","last_clean":"2024-04-17T09:57:13.190166+0000","last_became_active":"2024-04-17T09:57:12.184103+0000","last_became_peered":"2024-04-17T09:57:12.184103+0000","last_unstale":"2024-04-17T09:57:13.190166+0000","last_undegraded":"2024-04-17T09:57:13.190166+0000","last_fullsized":"2024-04-17T09:57:13.190166+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:36:02.427710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178070+0000","last_change":"2024-04-17T09:57:12.183674+0000","last_active":"2024-04-17T09:57:13.178070+0000","last_peered":"2024-04-17T09:57:13.178070+0000","last_clean":"2024-04-17T09:57:13.178070+0000","last_became_active":"2024-04-17T09:57:12.183368+0000","last_became_peered":"2024-04-17T09:57:12.183368+0000","last_unstale":"2024-04-17T09:57:13.178070+0000","last_undegraded":"2024-04-17T09:57:13.178070+0000","last_fullsized":"2024-04-17T09:57:13.178070+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:03:48.599405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280754+0000","last_change":"2024-04-17T09:57:12.186766+0000","last_active":"2024-04-17T09:57:13.280754+0000","last_peered":"2024-04-17T09:57:13.280754+0000","last_clean":"2024-04-17T09:57:13.280754+0000","last_became_active":"2024-04-17T09:57:12.185955+0000","last_became_peered":"2024-04-17T09:57:12.185955+0000","last_unstale":"2024-04-17T09:57:13.280754+0000","last_undegraded":"2024-04-17T09:57:13.280754+0000","last_fullsized":"2024-04-17T09:57:13.280754+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:11.273872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178056+0000","last_change":"2024-04-17T09:57:12.183751+0000","last_active":"2024-04-17T09:57:13.178056+0000","last_peered":"2024-04-17T09:57:13.178056+0000","last_clean":"2024-04-17T09:57:13.178056+0000","last_became_active":"2024-04-17T09:57:12.183411+0000","last_became_peered":"2024-04-17T09:57:12.183411+0000","last_unstale":"2024-04-17T09:57:13.178056+0000","last_undegraded":"2024-04-17T09:57:13.178056+0000","last_fullsized":"2024-04-17T09:57:13.178056+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:54.274231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178440+0000","last_change":"2024-04-17T09:57:12.182399+0000","last_active":"2024-04-17T09:57:13.178440+0000","last_peered":"2024-04-17T09:57:13.178440+0000","last_clean":"2024-04-17T09:57:13.178440+0000","last_became_active":"2024-04-17T09:57:12.182114+0000","last_became_peered":"2024-04-17T09:57:12.182114+0000","last_unstale":"2024-04-17T09:57:13.178440+0000","last_undegraded":"2024-04-17T09:57:13.178440+0000","last_fullsized":"2024-04-17T09:57:13.178440+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:59:48.176397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.190359+0000","last_change":"2024-04-17T09:57:12.192235+0000","last_active":"2024-04-17T09:57:13.190359+0000","last_peered":"2024-04-17T09:57:13.190359+0000","last_clean":"2024-04-17T09:57:13.190359+0000","last_became_active":"2024-04-17T09:57:12.188188+0000","last_became_peered":"2024-04-17T09:57:12.188188+0000","last_unstale":"2024-04-17T09:57:13.190359+0000","last_undegraded":"2024-04-17T09:57:13.190359+0000","last_fullsized":"2024-04-17T09:57:13.190359+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:57.006419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.177913+0000","last_change":"2024-04-17T09:57:12.174012+0000","last_active":"2024-04-17T09:57:13.177913+0000","last_peered":"2024-04-17T09:57:13.177913+0000","last_clean":"2024-04-17T09:57:13.177913+0000","last_became_active":"2024-04-17T09:57:12.172539+0000","last_became_peered":"2024-04-17T09:57:12.172539+0000","last_unstale":"2024-04-17T09:57:13.177913+0000","last_undegraded":"2024-04-17T09:57:13.177913+0000","last_fullsized":"2024-04-17T09:57:13.177913+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:14:03.254732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.177957+0000","last_change":"2024-04-17T09:57:12.197226+0000","last_active":"2024-04-17T09:57:13.177957+0000","last_peered":"2024-04-17T09:57:13.177957+0000","last_clean":"2024-04-17T09:57:13.177957+0000","last_became_active":"2024-04-17T09:57:12.196859+0000","last_became_peered":"2024-04-17T09:57:12.196859+0000","last_unstale":"2024-04-17T09:57:13.177957+0000","last_undegraded":"2024-04-17T09:57:13.177957+0000","last_fullsized":"2024-04-17T09:57:13.177957+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:31:04.978298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.279906+0000","last_change":"2024-04-17T09:57:12.185124+0000","last_active":"2024-04-17T09:57:13.279906+0000","last_peered":"2024-04-17T09:57:13.279906+0000","last_clean":"2024-04-17T09:57:13.279906+0000","last_became_active":"2024-04-17T09:57:12.184473+0000","last_became_peered":"2024-04-17T09:57:12.184473+0000","last_unstale":"2024-04-17T09:57:13.279906+0000","last_undegraded":"2024-04-17T09:57:13.279906+0000","last_fullsized":"2024-04-17T09:57:13.279906+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:58:43.470530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.279825+0000","last_change":"2024-04-17T09:57:12.182939+0000","last_active":"2024-04-17T09:57:13.279825+0000","last_peered":"2024-04-17T09:57:13.279825+0000","last_clean":"2024-04-17T09:57:13.279825+0000","last_became_active":"2024-04-17T09:57:12.182666+0000","last_became_peered":"2024-04-17T09:57:12.182666+0000","last_unstale":"2024-04-17T09:57:13.279825+0000","last_undegraded":"2024-04-17T09:57:13.279825+0000","last_fullsized":"2024-04-17T09:57:13.279825+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:33:49.810693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.179045+0000","last_change":"2024-04-17T09:57:12.180577+0000","last_active":"2024-04-17T09:57:13.179045+0000","last_peered":"2024-04-17T09:57:13.179045+0000","last_clean":"2024-04-17T09:57:13.179045+0000","last_became_active":"2024-04-17T09:57:12.179258+0000","last_became_peered":"2024-04-17T09:57:12.179258+0000","last_unstale":"2024-04-17T09:57:13.179045+0000","last_undegraded":"2024-04-17T09:57:13.179045+0000","last_fullsized":"2024-04-17T09:57:13.179045+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:24:16.821303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.184721+0000","last_change":"2024-04-17T09:57:12.193247+0000","last_active":"2024-04-17T09:57:13.184721+0000","last_peered":"2024-04-17T09:57:13.184721+0000","last_clean":"2024-04-17T09:57:13.184721+0000","last_became_active":"2024-04-17T09:57:12.192192+0000","last_became_peered":"2024-04-17T09:57:12.192192+0000","last_unstale":"2024-04-17T09:57:13.184721+0000","last_undegraded":"2024-04-17T09:57:13.184721+0000","last_fullsized":"2024-04-17T09:57:13.184721+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:51:37.257497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280959+0000","last_change":"2024-04-17T09:57:12.182270+0000","last_active":"2024-04-17T09:57:13.280959+0000","last_peered":"2024-04-17T09:57:13.280959+0000","last_clean":"2024-04-17T09:57:13.280959+0000","last_became_active":"2024-04-17T09:57:12.181623+0000","last_became_peered":"2024-04-17T09:57:12.181623+0000","last_unstale":"2024-04-17T09:57:13.280959+0000","last_undegraded":"2024-04-17T09:57:13.280959+0000","last_fullsized":"2024-04-17T09:57:13.280959+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:18:50.937330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178423+0000","last_change":"2024-04-17T09:57:12.190674+0000","last_active":"2024-04-17T09:57:13.178423+0000","last_peered":"2024-04-17T09:57:13.178423+0000","last_clean":"2024-04-17T09:57:13.178423+0000","last_became_active":"2024-04-17T09:57:12.189886+0000","last_became_peered":"2024-04-17T09:57:12.189886+0000","last_unstale":"2024-04-17T09:57:13.178423+0000","last_undegraded":"2024-04-17T09:57:13.178423+0000","last_fullsized":"2024-04-17T09:57:13.178423+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:51:57.138305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280729+0000","last_change":"2024-04-17T09:57:12.186070+0000","last_active":"2024-04-17T09:57:13.280729+0000","last_peered":"2024-04-17T09:57:13.280729+0000","last_clean":"2024-04-17T09:57:13.280729+0000","last_became_active":"2024-04-17T09:57:12.185160+0000","last_became_peered":"2024-04-17T09:57:12.185160+0000","last_unstale":"2024-04-17T09:57:13.280729+0000","last_undegraded":"2024-04-17T09:57:13.280729+0000","last_fullsized":"2024-04-17T09:57:13.280729+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:48:29.586244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.184534+0000","last_change":"2024-04-17T09:57:12.189730+0000","last_active":"2024-04-17T09:57:13.184534+0000","last_peered":"2024-04-17T09:57:13.184534+0000","last_clean":"2024-04-17T09:57:13.184534+0000","last_became_active":"2024-04-17T09:57:12.188417+0000","last_became_peered":"2024-04-17T09:57:12.188417+0000","last_unstale":"2024-04-17T09:57:13.184534+0000","last_undegraded":"2024-04-17T09:57:13.184534+0000","last_fullsized":"2024-04-17T09:57:13.184534+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:17:22.472846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.184735+0000","last_change":"2024-04-17T09:57:12.193999+0000","last_active":"2024-04-17T09:57:13.184735+0000","last_peered":"2024-04-17T09:57:13.184735+0000","last_clean":"2024-04-17T09:57:13.184735+0000","last_became_active":"2024-04-17T09:57:12.192355+0000","last_became_peered":"2024-04-17T09:57:12.192355+0000","last_unstale":"2024-04-17T09:57:13.184735+0000","last_undegraded":"2024-04-17T09:57:13.184735+0000","last_fullsized":"2024-04-17T09:57:13.184735+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:56:05.919839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178523+0000","last_change":"2024-04-17T09:57:12.186381+0000","last_active":"2024-04-17T09:57:13.178523+0000","last_peered":"2024-04-17T09:57:13.178523+0000","last_clean":"2024-04-17T09:57:13.178523+0000","last_became_active":"2024-04-17T09:57:12.185938+0000","last_became_peered":"2024-04-17T09:57:12.185938+0000","last_unstale":"2024-04-17T09:57:13.178523+0000","last_undegraded":"2024-04-17T09:57:13.178523+0000","last_fullsized":"2024-04-17T09:57:13.178523+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:56:27.144132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280125+0000","last_change":"2024-04-17T09:57:12.185704+0000","last_active":"2024-04-17T09:57:13.280125+0000","last_peered":"2024-04-17T09:57:13.280125+0000","last_clean":"2024-04-17T09:57:13.280125+0000","last_became_active":"2024-04-17T09:57:12.185357+0000","last_became_peered":"2024-04-17T09:57:12.185357+0000","last_unstale":"2024-04-17T09:57:13.280125+0000","last_undegraded":"2024-04-17T09:57:13.280125+0000","last_fullsized":"2024-04-17T09:57:13.280125+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:02:10.369272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"67'4","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280399+0000","last_change":"2024-04-17T09:57:11.170675+0000","last_active":"2024-04-17T09:57:13.280399+0000","last_peered":"2024-04-17T09:57:13.280399+0000","last_clean":"2024-04-17T09:57:13.280399+0000","last_became_active":"2024-04-17T09:57:11.170375+0000","last_became_peered":"2024-04-17T09:57:11.170375+0000","last_unstale":"2024-04-17T09:57:13.280399+0000","last_undegraded":"2024-04-17T09:57:13.280399+0000","last_fullsized":"2024-04-17T09:57:13.280399+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:32:01.757683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280757+0000","last_change":"2024-04-17T09:57:12.186253+0000","last_active":"2024-04-17T09:57:13.280757+0000","last_peered":"2024-04-17T09:57:13.280757+0000","last_clean":"2024-04-17T09:57:13.280757+0000","last_became_active":"2024-04-17T09:57:12.185924+0000","last_became_peered":"2024-04-17T09:57:12.185924+0000","last_unstale":"2024-04-17T09:57:13.280757+0000","last_undegraded":"2024-04-17T09:57:13.280757+0000","last_fullsized":"2024-04-17T09:57:13.280757+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:42:21.329246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.190398+0000","last_change":"2024-04-17T09:57:11.170736+0000","last_active":"2024-04-17T09:57:13.190398+0000","last_peered":"2024-04-17T09:57:13.190398+0000","last_clean":"2024-04-17T09:57:13.190398+0000","last_became_active":"2024-04-17T09:57:11.169279+0000","last_became_peered":"2024-04-17T09:57:11.169279+0000","last_unstale":"2024-04-17T09:57:13.190398+0000","last_undegraded":"2024-04-17T09:57:13.190398+0000","last_fullsized":"2024-04-17T09:57:13.190398+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:38:41.791856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178321+0000","last_change":"2024-04-17T09:57:12.184681+0000","last_active":"2024-04-17T09:57:13.178321+0000","last_peered":"2024-04-17T09:57:13.178321+0000","last_clean":"2024-04-17T09:57:13.178321+0000","last_became_active":"2024-04-17T09:57:12.184234+0000","last_became_peered":"2024-04-17T09:57:12.184234+0000","last_unstale":"2024-04-17T09:57:13.178321+0000","last_undegraded":"2024-04-17T09:57:13.178321+0000","last_fullsized":"2024-04-17T09:57:13.178321+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:48:51.551216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178726+0000","last_change":"2024-04-17T09:57:11.165813+0000","last_active":"2024-04-17T09:57:13.178726+0000","last_peered":"2024-04-17T09:57:13.178726+0000","last_clean":"2024-04-17T09:57:13.178726+0000","last_became_active":"2024-04-17T09:57:11.165560+0000","last_became_peered":"2024-04-17T09:57:11.165560+0000","last_unstale":"2024-04-17T09:57:13.178726+0000","last_undegraded":"2024-04-17T09:57:13.178726+0000","last_fullsized":"2024-04-17T09:57:13.178726+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:42:12.470791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280208+0000","last_change":"2024-04-17T09:57:12.189349+0000","last_active":"2024-04-17T09:57:13.280208+0000","last_peered":"2024-04-17T09:57:13.280208+0000","last_clean":"2024-04-17T09:57:13.280208+0000","last_became_active":"2024-04-17T09:57:12.189012+0000","last_became_peered":"2024-04-17T09:57:12.189012+0000","last_unstale":"2024-04-17T09:57:13.280208+0000","last_undegraded":"2024-04-17T09:57:13.280208+0000","last_fullsized":"2024-04-17T09:57:13.280208+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:06:50.422951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178589+0000","last_change":"2024-04-17T09:57:11.170949+0000","last_active":"2024-04-17T09:57:13.178589+0000","last_peered":"2024-04-17T09:57:13.178589+0000","last_clean":"2024-04-17T09:57:13.178589+0000","last_became_active":"2024-04-17T09:57:11.170678+0000","last_became_peered":"2024-04-17T09:57:11.170678+0000","last_unstale":"2024-04-17T09:57:13.178589+0000","last_undegraded":"2024-04-17T09:57:13.178589+0000","last_fullsized":"2024-04-17T09:57:13.178589+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:14:34.753436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.184832+0000","last_change":"2024-04-17T09:57:12.186747+0000","last_active":"2024-04-17T09:57:13.184832+0000","last_peered":"2024-04-17T09:57:13.184832+0000","last_clean":"2024-04-17T09:57:13.184832+0000","last_became_active":"2024-04-17T09:57:12.185746+0000","last_became_peered":"2024-04-17T09:57:12.185746+0000","last_unstale":"2024-04-17T09:57:13.184832+0000","last_undegraded":"2024-04-17T09:57:13.184832+0000","last_fullsized":"2024-04-17T09:57:13.184832+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:21:18.012312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280901+0000","last_change":"2024-04-17T09:57:11.175339+0000","last_active":"2024-04-17T09:57:13.280901+0000","last_peered":"2024-04-17T09:57:13.280901+0000","last_clean":"2024-04-17T09:57:13.280901+0000","last_became_active":"2024-04-17T09:57:11.174554+0000","last_became_peered":"2024-04-17T09:57:11.174554+0000","last_unstale":"2024-04-17T09:57:13.280901+0000","last_undegraded":"2024-04-17T09:57:13.280901+0000","last_fullsized":"2024-04-17T09:57:13.280901+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:36:02.179193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178090+0000","last_change":"2024-04-17T09:57:12.179416+0000","last_active":"2024-04-17T09:57:13.178090+0000","last_peered":"2024-04-17T09:57:13.178090+0000","last_clean":"2024-04-17T09:57:13.178090+0000","last_became_active":"2024-04-17T09:57:12.178618+0000","last_became_peered":"2024-04-17T09:57:12.178618+0000","last_unstale":"2024-04-17T09:57:13.178090+0000","last_undegraded":"2024-04-17T09:57:13.178090+0000","last_fullsized":"2024-04-17T09:57:13.178090+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:19:36.202473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178159+0000","last_change":"2024-04-17T09:57:11.171969+0000","last_active":"2024-04-17T09:57:13.178159+0000","last_peered":"2024-04-17T09:57:13.178159+0000","last_clean":"2024-04-17T09:57:13.178159+0000","last_became_active":"2024-04-17T09:57:11.171116+0000","last_became_peered":"2024-04-17T09:57:11.171116+0000","last_unstale":"2024-04-17T09:57:13.178159+0000","last_undegraded":"2024-04-17T09:57:13.178159+0000","last_fullsized":"2024-04-17T09:57:13.178159+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:26:50.890315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280114+0000","last_change":"2024-04-17T09:57:12.189773+0000","last_active":"2024-04-17T09:57:13.280114+0000","last_peered":"2024-04-17T09:57:13.280114+0000","last_clean":"2024-04-17T09:57:13.280114+0000","last_became_active":"2024-04-17T09:57:12.189492+0000","last_became_peered":"2024-04-17T09:57:12.189492+0000","last_unstale":"2024-04-17T09:57:13.280114+0000","last_undegraded":"2024-04-17T09:57:13.280114+0000","last_fullsized":"2024-04-17T09:57:13.280114+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:54:34.634350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.177750+0000","last_change":"2024-04-17T09:57:11.171698+0000","last_active":"2024-04-17T09:57:13.177750+0000","last_peered":"2024-04-17T09:57:13.177750+0000","last_clean":"2024-04-17T09:57:13.177750+0000","last_became_active":"2024-04-17T09:57:11.169753+0000","last_became_peered":"2024-04-17T09:57:11.169753+0000","last_unstale":"2024-04-17T09:57:13.177750+0000","last_undegraded":"2024-04-17T09:57:13.177750+0000","last_fullsized":"2024-04-17T09:57:13.177750+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:36:27.906424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.190193+0000","last_change":"2024-04-17T09:57:12.176677+0000","last_active":"2024-04-17T09:57:13.190193+0000","last_peered":"2024-04-17T09:57:13.190193+0000","last_clean":"2024-04-17T09:57:13.190193+0000","last_became_active":"2024-04-17T09:57:12.176403+0000","last_became_peered":"2024-04-17T09:57:12.176403+0000","last_unstale":"2024-04-17T09:57:13.190193+0000","last_undegraded":"2024-04-17T09:57:13.190193+0000","last_fullsized":"2024-04-17T09:57:13.190193+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:54:37.533520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280098+0000","last_change":"2024-04-17T09:57:11.173923+0000","last_active":"2024-04-17T09:57:13.280098+0000","last_peered":"2024-04-17T09:57:13.280098+0000","last_clean":"2024-04-17T09:57:13.280098+0000","last_became_active":"2024-04-17T09:57:11.173611+0000","last_became_peered":"2024-04-17T09:57:11.173611+0000","last_unstale":"2024-04-17T09:57:13.280098+0000","last_undegraded":"2024-04-17T09:57:13.280098+0000","last_fullsized":"2024-04-17T09:57:13.280098+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:07:36.784027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.184874+0000","last_change":"2024-04-17T09:57:12.192530+0000","last_active":"2024-04-17T09:57:13.184874+0000","last_peered":"2024-04-17T09:57:13.184874+0000","last_clean":"2024-04-17T09:57:13.184874+0000","last_became_active":"2024-04-17T09:57:12.191774+0000","last_became_peered":"2024-04-17T09:57:12.191774+0000","last_unstale":"2024-04-17T09:57:13.184874+0000","last_undegraded":"2024-04-17T09:57:13.184874+0000","last_fullsized":"2024-04-17T09:57:13.184874+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:23:49.049846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.179200+0000","last_change":"2024-04-17T09:57:11.167108+0000","last_active":"2024-04-17T09:57:13.179200+0000","last_peered":"2024-04-17T09:57:13.179200+0000","last_clean":"2024-04-17T09:57:13.179200+0000","last_became_active":"2024-04-17T09:57:11.166856+0000","last_became_peered":"2024-04-17T09:57:11.166856+0000","last_unstale":"2024-04-17T09:57:13.179200+0000","last_undegraded":"2024-04-17T09:57:13.179200+0000","last_fullsized":"2024-04-17T09:57:13.179200+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:13.063150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280331+0000","last_change":"2024-04-17T09:57:12.195940+0000","last_active":"2024-04-17T09:57:13.280331+0000","last_peered":"2024-04-17T09:57:13.280331+0000","last_clean":"2024-04-17T09:57:13.280331+0000","last_became_active":"2024-04-17T09:57:12.190743+0000","last_became_peered":"2024-04-17T09:57:12.190743+0000","last_unstale":"2024-04-17T09:57:13.280331+0000","last_undegraded":"2024-04-17T09:57:13.280331+0000","last_fullsized":"2024-04-17T09:57:13.280331+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:04:47.401445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178953+0000","last_change":"2024-04-17T09:57:11.170518+0000","last_active":"2024-04-17T09:57:13.178953+0000","last_peered":"2024-04-17T09:57:13.178953+0000","last_clean":"2024-04-17T09:57:13.178953+0000","last_became_active":"2024-04-17T09:57:11.168881+0000","last_became_peered":"2024-04-17T09:57:11.168881+0000","last_unstale":"2024-04-17T09:57:13.178953+0000","last_undegraded":"2024-04-17T09:57:13.178953+0000","last_fullsized":"2024-04-17T09:57:13.178953+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:58:00.221995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178509+0000","last_change":"2024-04-17T09:57:12.186953+0000","last_active":"2024-04-17T09:57:13.178509+0000","last_peered":"2024-04-17T09:57:13.178509+0000","last_clean":"2024-04-17T09:57:13.178509+0000","last_became_active":"2024-04-17T09:57:12.186674+0000","last_became_peered":"2024-04-17T09:57:12.186674+0000","last_unstale":"2024-04-17T09:57:13.178509+0000","last_undegraded":"2024-04-17T09:57:13.178509+0000","last_fullsized":"2024-04-17T09:57:13.178509+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:46:15.041323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"67'3","reported_seq":17,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280924+0000","last_change":"2024-04-17T09:57:11.176105+0000","last_active":"2024-04-17T09:57:13.280924+0000","last_peered":"2024-04-17T09:57:13.280924+0000","last_clean":"2024-04-17T09:57:13.280924+0000","last_became_active":"2024-04-17T09:57:11.175555+0000","last_became_peered":"2024-04-17T09:57:11.175555+0000","last_unstale":"2024-04-17T09:57:13.280924+0000","last_undegraded":"2024-04-17T09:57:13.280924+0000","last_fullsized":"2024-04-17T09:57:13.280924+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:30.144661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.281057+0000","last_change":"2024-04-17T09:57:12.187481+0000","last_active":"2024-04-17T09:57:13.281057+0000","last_peered":"2024-04-17T09:57:13.281057+0000","last_clean":"2024-04-17T09:57:13.281057+0000","last_became_active":"2024-04-17T09:57:12.187199+0000","last_became_peered":"2024-04-17T09:57:12.187199+0000","last_unstale":"2024-04-17T09:57:13.281057+0000","last_undegraded":"2024-04-17T09:57:13.281057+0000","last_fullsized":"2024-04-17T09:57:13.281057+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:36:14.968360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"67'6","reported_seq":20,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178171+0000","last_change":"2024-04-17T09:57:11.173441+0000","last_active":"2024-04-17T09:57:13.178171+0000","last_peered":"2024-04-17T09:57:13.178171+0000","last_clean":"2024-04-17T09:57:13.178171+0000","last_became_active":"2024-04-17T09:57:11.173130+0000","last_became_peered":"2024-04-17T09:57:11.173130+0000","last_unstale":"2024-04-17T09:57:13.178171+0000","last_undegraded":"2024-04-17T09:57:13.178171+0000","last_fullsized":"2024-04-17T09:57:13.178171+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:28:37.561860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280031+0000","last_change":"2024-04-17T09:57:12.192904+0000","last_active":"2024-04-17T09:57:13.280031+0000","last_peered":"2024-04-17T09:57:13.280031+0000","last_clean":"2024-04-17T09:57:13.280031+0000","last_became_active":"2024-04-17T09:57:12.192617+0000","last_became_peered":"2024-04-17T09:57:12.192617+0000","last_unstale":"2024-04-17T09:57:13.280031+0000","last_undegraded":"2024-04-17T09:57:13.280031+0000","last_fullsized":"2024-04-17T09:57:13.280031+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:36:03.187915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.177899+0000","last_change":"2024-04-17T09:57:11.174377+0000","last_active":"2024-04-17T09:57:13.177899+0000","last_peered":"2024-04-17T09:57:13.177899+0000","last_clean":"2024-04-17T09:57:13.177899+0000","last_became_active":"2024-04-17T09:57:11.174102+0000","last_became_peered":"2024-04-17T09:57:11.174102+0000","last_unstale":"2024-04-17T09:57:13.177899+0000","last_undegraded":"2024-04-17T09:57:13.177899+0000","last_fullsized":"2024-04-17T09:57:13.177899+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:42:20.320524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.184621+0000","last_change":"2024-04-17T09:57:12.189552+0000","last_active":"2024-04-17T09:57:13.184621+0000","last_peered":"2024-04-17T09:57:13.184621+0000","last_clean":"2024-04-17T09:57:13.184621+0000","last_became_active":"2024-04-17T09:57:12.189206+0000","last_became_peered":"2024-04-17T09:57:12.189206+0000","last_unstale":"2024-04-17T09:57:13.184621+0000","last_undegraded":"2024-04-17T09:57:13.184621+0000","last_fullsized":"2024-04-17T09:57:13.184621+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:32:02.766405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178261+0000","last_change":"2024-04-17T09:57:11.172197+0000","last_active":"2024-04-17T09:57:13.178261+0000","last_peered":"2024-04-17T09:57:13.178261+0000","last_clean":"2024-04-17T09:57:13.178261+0000","last_became_active":"2024-04-17T09:57:11.171480+0000","last_became_peered":"2024-04-17T09:57:11.171480+0000","last_unstale":"2024-04-17T09:57:13.178261+0000","last_undegraded":"2024-04-17T09:57:13.178261+0000","last_fullsized":"2024-04-17T09:57:13.178261+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:53.265509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.190063+0000","last_change":"2024-04-17T09:57:12.182697+0000","last_active":"2024-04-17T09:57:13.190063+0000","last_peered":"2024-04-17T09:57:13.190063+0000","last_clean":"2024-04-17T09:57:13.190063+0000","last_became_active":"2024-04-17T09:57:12.182406+0000","last_became_peered":"2024-04-17T09:57:12.182406+0000","last_unstale":"2024-04-17T09:57:13.190063+0000","last_undegraded":"2024-04-17T09:57:13.190063+0000","last_fullsized":"2024-04-17T09:57:13.190063+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:36:54.671129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.184479+0000","last_change":"2024-04-17T09:57:11.175028+0000","last_active":"2024-04-17T09:57:13.184479+0000","last_peered":"2024-04-17T09:57:13.184479+0000","last_clean":"2024-04-17T09:57:13.184479+0000","last_became_active":"2024-04-17T09:57:11.174647+0000","last_became_peered":"2024-04-17T09:57:11.174647+0000","last_unstale":"2024-04-17T09:57:13.184479+0000","last_undegraded":"2024-04-17T09:57:13.184479+0000","last_fullsized":"2024-04-17T09:57:13.184479+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:23:51.478191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280850+0000","last_change":"2024-04-17T09:57:12.192516+0000","last_active":"2024-04-17T09:57:13.280850+0000","last_peered":"2024-04-17T09:57:13.280850+0000","last_clean":"2024-04-17T09:57:13.280850+0000","last_became_active":"2024-04-17T09:57:12.192220+0000","last_became_peered":"2024-04-17T09:57:12.192220+0000","last_unstale":"2024-04-17T09:57:13.280850+0000","last_undegraded":"2024-04-17T09:57:13.280850+0000","last_fullsized":"2024-04-17T09:57:13.280850+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:28:38.570582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"67'8","reported_seq":22,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.177678+0000","last_change":"2024-04-17T09:57:11.166857+0000","last_active":"2024-04-17T09:57:13.177678+0000","last_peered":"2024-04-17T09:57:13.177678+0000","last_clean":"2024-04-17T09:57:13.177678+0000","last_became_active":"2024-04-17T09:57:11.166543+0000","last_became_peered":"2024-04-17T09:57:11.166543+0000","last_unstale":"2024-04-17T09:57:13.177678+0000","last_undegraded":"2024-04-17T09:57:13.177678+0000","last_fullsized":"2024-04-17T09:57:13.177678+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:23:15.843700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.179206+0000","last_change":"2024-04-17T09:57:12.179053+0000","last_active":"2024-04-17T09:57:13.179206+0000","last_peered":"2024-04-17T09:57:13.179206+0000","last_clean":"2024-04-17T09:57:13.179206+0000","last_became_active":"2024-04-17T09:57:12.172803+0000","last_became_peered":"2024-04-17T09:57:12.172803+0000","last_unstale":"2024-04-17T09:57:13.179206+0000","last_undegraded":"2024-04-17T09:57:13.179206+0000","last_fullsized":"2024-04-17T09:57:13.179206+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:38:42.800578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.179029+0000","last_change":"2024-04-17T09:57:11.174027+0000","last_active":"2024-04-17T09:57:13.179029+0000","last_peered":"2024-04-17T09:57:13.179029+0000","last_clean":"2024-04-17T09:57:13.179029+0000","last_became_active":"2024-04-17T09:57:11.173713+0000","last_became_peered":"2024-04-17T09:57:11.173713+0000","last_unstale":"2024-04-17T09:57:13.179029+0000","last_undegraded":"2024-04-17T09:57:13.179029+0000","last_fullsized":"2024-04-17T09:57:13.179029+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:32:55.279233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280446+0000","last_change":"2024-04-17T09:57:12.184938+0000","last_active":"2024-04-17T09:57:13.280446+0000","last_peered":"2024-04-17T09:57:13.280446+0000","last_clean":"2024-04-17T09:57:13.280446+0000","last_became_active":"2024-04-17T09:57:12.184175+0000","last_became_peered":"2024-04-17T09:57:12.184175+0000","last_unstale":"2024-04-17T09:57:13.280446+0000","last_undegraded":"2024-04-17T09:57:13.280446+0000","last_fullsized":"2024-04-17T09:57:13.280446+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:04:03.401353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.179248+0000","last_change":"2024-04-17T09:57:11.169402+0000","last_active":"2024-04-17T09:57:13.179248+0000","last_peered":"2024-04-17T09:57:13.179248+0000","last_clean":"2024-04-17T09:57:13.179248+0000","last_became_active":"2024-04-17T09:57:11.168674+0000","last_became_peered":"2024-04-17T09:57:11.168674+0000","last_unstale":"2024-04-17T09:57:13.179248+0000","last_undegraded":"2024-04-17T09:57:13.179248+0000","last_fullsized":"2024-04-17T09:57:13.179248+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:48.357059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178222+0000","last_change":"2024-04-17T09:57:12.193004+0000","last_active":"2024-04-17T09:57:13.178222+0000","last_peered":"2024-04-17T09:57:13.178222+0000","last_clean":"2024-04-17T09:57:13.178222+0000","last_became_active":"2024-04-17T09:57:12.192651+0000","last_became_peered":"2024-04-17T09:57:12.192651+0000","last_unstale":"2024-04-17T09:57:13.178222+0000","last_undegraded":"2024-04-17T09:57:13.178222+0000","last_fullsized":"2024-04-17T09:57:13.178222+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:55:43.451961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178588+0000","last_change":"2024-04-17T09:57:11.169785+0000","last_active":"2024-04-17T09:57:13.178588+0000","last_peered":"2024-04-17T09:57:13.178588+0000","last_clean":"2024-04-17T09:57:13.178588+0000","last_became_active":"2024-04-17T09:57:11.169471+0000","last_became_peered":"2024-04-17T09:57:11.169471+0000","last_unstale":"2024-04-17T09:57:13.178588+0000","last_undegraded":"2024-04-17T09:57:13.178588+0000","last_fullsized":"2024-04-17T09:57:13.178588+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:53.582494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280560+0000","last_change":"2024-04-17T09:57:12.189990+0000","last_active":"2024-04-17T09:57:13.280560+0000","last_peered":"2024-04-17T09:57:13.280560+0000","last_clean":"2024-04-17T09:57:13.280560+0000","last_became_active":"2024-04-17T09:57:12.189615+0000","last_became_peered":"2024-04-17T09:57:12.189615+0000","last_unstale":"2024-04-17T09:57:13.280560+0000","last_undegraded":"2024-04-17T09:57:13.280560+0000","last_fullsized":"2024-04-17T09:57:13.280560+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:53:50.111959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.279976+0000","last_change":"2024-04-17T09:57:11.165394+0000","last_active":"2024-04-17T09:57:13.279976+0000","last_peered":"2024-04-17T09:57:13.279976+0000","last_clean":"2024-04-17T09:57:13.279976+0000","last_became_active":"2024-04-17T09:57:11.165120+0000","last_became_peered":"2024-04-17T09:57:11.165120+0000","last_unstale":"2024-04-17T09:57:13.279976+0000","last_undegraded":"2024-04-17T09:57:13.279976+0000","last_fullsized":"2024-04-17T09:57:13.279976+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:17:21.464124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280986+0000","last_change":"2024-04-17T09:57:12.176860+0000","last_active":"2024-04-17T09:57:13.280986+0000","last_peered":"2024-04-17T09:57:13.280986+0000","last_clean":"2024-04-17T09:57:13.280986+0000","last_became_active":"2024-04-17T09:57:12.174767+0000","last_became_peered":"2024-04-17T09:57:12.174767+0000","last_unstale":"2024-04-17T09:57:13.280986+0000","last_undegraded":"2024-04-17T09:57:13.280986+0000","last_fullsized":"2024-04-17T09:57:13.280986+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:08:40.914887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178957+0000","last_change":"2024-04-17T09:57:11.171737+0000","last_active":"2024-04-17T09:57:13.178957+0000","last_peered":"2024-04-17T09:57:13.178957+0000","last_clean":"2024-04-17T09:57:13.178957+0000","last_became_active":"2024-04-17T09:57:11.170748+0000","last_became_peered":"2024-04-17T09:57:11.170748+0000","last_unstale":"2024-04-17T09:57:13.178957+0000","last_undegraded":"2024-04-17T09:57:13.178957+0000","last_fullsized":"2024-04-17T09:57:13.178957+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:11:38.218856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280416+0000","last_change":"2024-04-17T09:57:12.182080+0000","last_active":"2024-04-17T09:57:13.280416+0000","last_peered":"2024-04-17T09:57:13.280416+0000","last_clean":"2024-04-17T09:57:13.280416+0000","last_became_active":"2024-04-17T09:57:12.181786+0000","last_became_peered":"2024-04-17T09:57:12.181786+0000","last_unstale":"2024-04-17T09:57:13.280416+0000","last_undegraded":"2024-04-17T09:57:13.280416+0000","last_fullsized":"2024-04-17T09:57:13.280416+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:46.142617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.184805+0000","last_change":"2024-04-17T09:57:11.175061+0000","last_active":"2024-04-17T09:57:13.184805+0000","last_peered":"2024-04-17T09:57:13.184805+0000","last_clean":"2024-04-17T09:57:13.184805+0000","last_became_active":"2024-04-17T09:57:11.174687+0000","last_became_peered":"2024-04-17T09:57:11.174687+0000","last_unstale":"2024-04-17T09:57:13.184805+0000","last_undegraded":"2024-04-17T09:57:13.184805+0000","last_fullsized":"2024-04-17T09:57:13.184805+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:37.034263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280622+0000","last_change":"2024-04-17T09:57:12.184298+0000","last_active":"2024-04-17T09:57:13.280622+0000","last_peered":"2024-04-17T09:57:13.280622+0000","last_clean":"2024-04-17T09:57:13.280622+0000","last_became_active":"2024-04-17T09:57:12.181830+0000","last_became_peered":"2024-04-17T09:57:12.181830+0000","last_unstale":"2024-04-17T09:57:13.280622+0000","last_undegraded":"2024-04-17T09:57:13.280622+0000","last_fullsized":"2024-04-17T09:57:13.280622+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:44:19.484223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.189969+0000","last_change":"2024-04-17T09:57:11.177802+0000","last_active":"2024-04-17T09:57:13.189969+0000","last_peered":"2024-04-17T09:57:13.189969+0000","last_clean":"2024-04-17T09:57:13.189969+0000","last_became_active":"2024-04-17T09:57:11.175507+0000","last_became_peered":"2024-04-17T09:57:11.175507+0000","last_unstale":"2024-04-17T09:57:13.189969+0000","last_undegraded":"2024-04-17T09:57:13.189969+0000","last_fullsized":"2024-04-17T09:57:13.189969+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:24:15.812581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.190410+0000","last_change":"2024-04-17T09:57:12.189135+0000","last_active":"2024-04-17T09:57:13.190410+0000","last_peered":"2024-04-17T09:57:13.190410+0000","last_clean":"2024-04-17T09:57:13.190410+0000","last_became_active":"2024-04-17T09:57:12.188881+0000","last_became_peered":"2024-04-17T09:57:12.188881+0000","last_unstale":"2024-04-17T09:57:13.190410+0000","last_undegraded":"2024-04-17T09:57:13.190410+0000","last_fullsized":"2024-04-17T09:57:13.190410+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:07.597746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178877+0000","last_change":"2024-04-17T09:57:11.173965+0000","last_active":"2024-04-17T09:57:13.178877+0000","last_peered":"2024-04-17T09:57:13.178877+0000","last_clean":"2024-04-17T09:57:13.178877+0000","last_became_active":"2024-04-17T09:57:11.173657+0000","last_became_peered":"2024-04-17T09:57:11.173657+0000","last_unstale":"2024-04-17T09:57:13.178877+0000","last_undegraded":"2024-04-17T09:57:13.178877+0000","last_fullsized":"2024-04-17T09:57:13.178877+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:36:13.959638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.184765+0000","last_change":"2024-04-17T09:57:12.191094+0000","last_active":"2024-04-17T09:57:13.184765+0000","last_peered":"2024-04-17T09:57:13.184765+0000","last_clean":"2024-04-17T09:57:13.184765+0000","last_became_active":"2024-04-17T09:57:12.190774+0000","last_became_peered":"2024-04-17T09:57:12.190774+0000","last_unstale":"2024-04-17T09:57:13.184765+0000","last_undegraded":"2024-04-17T09:57:13.184765+0000","last_fullsized":"2024-04-17T09:57:13.184765+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:32:52.852878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"68'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.184852+0000","last_change":"2024-04-17T09:57:11.173169+0000","last_active":"2024-04-17T09:57:13.184852+0000","last_peered":"2024-04-17T09:57:13.184852+0000","last_clean":"2024-04-17T09:57:13.184852+0000","last_became_active":"2024-04-17T09:57:11.172785+0000","last_became_peered":"2024-04-17T09:57:11.172785+0000","last_unstale":"2024-04-17T09:57:13.184852+0000","last_undegraded":"2024-04-17T09:57:13.184852+0000","last_fullsized":"2024-04-17T09:57:13.184852+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:27:57.370537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280118+0000","last_change":"2024-04-17T09:57:12.194481+0000","last_active":"2024-04-17T09:57:13.280118+0000","last_peered":"2024-04-17T09:57:13.280118+0000","last_clean":"2024-04-17T09:57:13.280118+0000","last_became_active":"2024-04-17T09:57:12.194128+0000","last_became_peered":"2024-04-17T09:57:12.194128+0000","last_unstale":"2024-04-17T09:57:13.280118+0000","last_undegraded":"2024-04-17T09:57:13.280118+0000","last_fullsized":"2024-04-17T09:57:13.280118+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:31.153383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280820+0000","last_change":"2024-04-17T09:57:11.175574+0000","last_active":"2024-04-17T09:57:13.280820+0000","last_peered":"2024-04-17T09:57:13.280820+0000","last_clean":"2024-04-17T09:57:13.280820+0000","last_became_active":"2024-04-17T09:57:11.175285+0000","last_became_peered":"2024-04-17T09:57:11.175285+0000","last_unstale":"2024-04-17T09:57:13.280820+0000","last_undegraded":"2024-04-17T09:57:13.280820+0000","last_fullsized":"2024-04-17T09:57:13.280820+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:49:03.521739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280390+0000","last_change":"2024-04-17T09:57:12.195374+0000","last_active":"2024-04-17T09:57:13.280390+0000","last_peered":"2024-04-17T09:57:13.280390+0000","last_clean":"2024-04-17T09:57:13.280390+0000","last_became_active":"2024-04-17T09:57:12.195122+0000","last_became_peered":"2024-04-17T09:57:12.195122+0000","last_unstale":"2024-04-17T09:57:13.280390+0000","last_undegraded":"2024-04-17T09:57:13.280390+0000","last_fullsized":"2024-04-17T09:57:13.280390+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:41:15.457475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.190371+0000","last_change":"2024-04-17T09:57:11.161353+0000","last_active":"2024-04-17T09:57:13.190371+0000","last_peered":"2024-04-17T09:57:13.190371+0000","last_clean":"2024-04-17T09:57:13.190371+0000","last_became_active":"2024-04-17T09:57:11.161100+0000","last_became_peered":"2024-04-17T09:57:11.161100+0000","last_unstale":"2024-04-17T09:57:13.190371+0000","last_undegraded":"2024-04-17T09:57:13.190371+0000","last_fullsized":"2024-04-17T09:57:13.190371+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:28:40.233463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.281146+0000","last_change":"2024-04-17T09:57:12.191880+0000","last_active":"2024-04-17T09:57:13.281146+0000","last_peered":"2024-04-17T09:57:13.281146+0000","last_clean":"2024-04-17T09:57:13.281146+0000","last_became_active":"2024-04-17T09:57:12.191130+0000","last_became_peered":"2024-04-17T09:57:12.191130+0000","last_unstale":"2024-04-17T09:57:13.281146+0000","last_undegraded":"2024-04-17T09:57:13.281146+0000","last_fullsized":"2024-04-17T09:57:13.281146+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:17:05.804391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280266+0000","last_change":"2024-04-17T09:57:11.172495+0000","last_active":"2024-04-17T09:57:13.280266+0000","last_peered":"2024-04-17T09:57:13.280266+0000","last_clean":"2024-04-17T09:57:13.280266+0000","last_became_active":"2024-04-17T09:57:11.172113+0000","last_became_peered":"2024-04-17T09:57:11.172113+0000","last_unstale":"2024-04-17T09:57:13.280266+0000","last_undegraded":"2024-04-17T09:57:13.280266+0000","last_fullsized":"2024-04-17T09:57:13.280266+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:54:44.975970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178849+0000","last_change":"2024-04-17T09:57:12.181471+0000","last_active":"2024-04-17T09:57:13.178849+0000","last_peered":"2024-04-17T09:57:13.178849+0000","last_clean":"2024-04-17T09:57:13.178849+0000","last_became_active":"2024-04-17T09:57:12.181165+0000","last_became_peered":"2024-04-17T09:57:12.181165+0000","last_unstale":"2024-04-17T09:57:13.178849+0000","last_undegraded":"2024-04-17T09:57:13.178849+0000","last_fullsized":"2024-04-17T09:57:13.178849+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:15.170971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.177750+0000","last_change":"2024-04-17T09:57:11.161883+0000","last_active":"2024-04-17T09:57:13.177750+0000","last_peered":"2024-04-17T09:57:13.177750+0000","last_clean":"2024-04-17T09:57:13.177750+0000","last_became_active":"2024-04-17T09:57:11.161601+0000","last_became_peered":"2024-04-17T09:57:11.161601+0000","last_unstale":"2024-04-17T09:57:13.177750+0000","last_undegraded":"2024-04-17T09:57:13.177750+0000","last_fullsized":"2024-04-17T09:57:13.177750+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:10.265150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178091+0000","last_change":"2024-04-17T09:57:12.178615+0000","last_active":"2024-04-17T09:57:13.178091+0000","last_peered":"2024-04-17T09:57:13.178091+0000","last_clean":"2024-04-17T09:57:13.178091+0000","last_became_active":"2024-04-17T09:57:12.177618+0000","last_became_peered":"2024-04-17T09:57:12.177618+0000","last_unstale":"2024-04-17T09:57:13.178091+0000","last_undegraded":"2024-04-17T09:57:13.178091+0000","last_fullsized":"2024-04-17T09:57:13.178091+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:12:37.383487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.177796+0000","last_change":"2024-04-17T09:57:11.167633+0000","last_active":"2024-04-17T09:57:13.177796+0000","last_peered":"2024-04-17T09:57:13.177796+0000","last_clean":"2024-04-17T09:57:13.177796+0000","last_became_active":"2024-04-17T09:57:11.167370+0000","last_became_peered":"2024-04-17T09:57:11.167370+0000","last_unstale":"2024-04-17T09:57:13.177796+0000","last_undegraded":"2024-04-17T09:57:13.177796+0000","last_fullsized":"2024-04-17T09:57:13.177796+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:49:40.199728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.177916+0000","last_change":"2024-04-17T09:57:12.185559+0000","last_active":"2024-04-17T09:57:13.177916+0000","last_peered":"2024-04-17T09:57:13.177916+0000","last_clean":"2024-04-17T09:57:13.177916+0000","last_became_active":"2024-04-17T09:57:12.185287+0000","last_became_peered":"2024-04-17T09:57:12.185287+0000","last_unstale":"2024-04-17T09:57:13.177916+0000","last_undegraded":"2024-04-17T09:57:13.177916+0000","last_fullsized":"2024-04-17T09:57:13.177916+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:38.042986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178568+0000","last_change":"2024-04-17T09:57:11.169107+0000","last_active":"2024-04-17T09:57:13.178568+0000","last_peered":"2024-04-17T09:57:13.178568+0000","last_clean":"2024-04-17T09:57:13.178568+0000","last_became_active":"2024-04-17T09:57:11.168808+0000","last_became_peered":"2024-04-17T09:57:11.168808+0000","last_unstale":"2024-04-17T09:57:13.178568+0000","last_undegraded":"2024-04-17T09:57:13.178568+0000","last_fullsized":"2024-04-17T09:57:13.178568+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:12:36.374765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.281028+0000","last_change":"2024-04-17T09:57:12.180324+0000","last_active":"2024-04-17T09:57:13.281028+0000","last_peered":"2024-04-17T09:57:13.281028+0000","last_clean":"2024-04-17T09:57:13.281028+0000","last_became_active":"2024-04-17T09:57:12.180048+0000","last_became_peered":"2024-04-17T09:57:12.180048+0000","last_unstale":"2024-04-17T09:57:13.281028+0000","last_undegraded":"2024-04-17T09:57:13.281028+0000","last_fullsized":"2024-04-17T09:57:13.281028+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:01:25.562824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178917+0000","last_change":"2024-04-17T09:57:11.170068+0000","last_active":"2024-04-17T09:57:13.178917+0000","last_peered":"2024-04-17T09:57:13.178917+0000","last_clean":"2024-04-17T09:57:13.178917+0000","last_became_active":"2024-04-17T09:57:11.168615+0000","last_became_peered":"2024-04-17T09:57:11.168615+0000","last_unstale":"2024-04-17T09:57:13.178917+0000","last_undegraded":"2024-04-17T09:57:13.178917+0000","last_fullsized":"2024-04-17T09:57:13.178917+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:17:04.795668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":257,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.279765+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T09:57:13.279765+0000","last_peered":"2024-04-17T09:57:13.279765+0000","last_clean":"2024-04-17T09:57:13.279765+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T09:57:13.279765+0000","last_undegraded":"2024-04-17T09:57:13.279765+0000","last_fullsized":"2024-04-17T09:57:13.279765+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509494,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1421312,"data_stored":933744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509495,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":827392,"data_stored":342852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509494,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":823296,"data_stored":342850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509494,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":342872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509495,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":831488,"data_stored":344372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509494,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":839680,"data_stored":344988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542198,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":342830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1413120,"data_stored":933128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T09:57:16.313 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T09:57:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.487+0000 7f0565400640 1 Processor -- start 2024-04-17T09:57:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.487+0000 7f0565400640 1 -- start start 2024-04-17T09:57:16.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.491+0000 7f0565400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f056006f630 0x7f0560072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:16.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.491+0000 7f0565400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0560072bc0 0x7f056006de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:16.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.491+0000 7f0565400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05600b3490 con 0x7f0560151db0 2024-04-17T09:57:16.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.491+0000 7f0565400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f05600b44f0 con 0x7f0560072bc0 2024-04-17T09:57:16.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.491+0000 7f0565400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0560073030 con 0x7f056006f630 2024-04-17T09:57:16.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.491+0000 7f055f200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0560072bc0 0x7f056006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:16.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.491+0000 7f055f200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0560072bc0 0x7f056006de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59714/0 (socket says 172.21.15.32:59714) 2024-04-17T09:57:16.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.491+0000 7f055f200640 1 -- 172.21.15.32:0/2258253069 learned_addr learned my addr 172.21.15.32:0/2258253069 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:16.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.491+0000 7f0555e00640 1 --2- 172.21.15.32:0/2258253069 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f056006f630 0x7f0560072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:16.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.491+0000 7f055f200640 1 -- 172.21.15.32:0/2258253069 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f056006f630 msgr2=0x7f0560072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:16.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.491+0000 7f055f200640 1 --2- 172.21.15.32:0/2258253069 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f056006f630 0x7f0560072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.491+0000 7f055f200640 1 -- 172.21.15.32:0/2258253069 >> v1:172.21.15.144:6789/0 conn(0x7f0560151db0 legacy=0x7f05600b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:16.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.491+0000 7f055f200640 1 -- 172.21.15.32:0/2258253069 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05600b58d0 con 0x7f0560072bc0 2024-04-17T09:57:16.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.495+0000 7f055f200640 1 --2- 172.21.15.32:0/2258253069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0560072bc0 0x7f056006de50 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f05600b5aa0 tx=0x7f055801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b8dad7896ee51f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:16.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.495+0000 7f055de00640 1 -- 172.21.15.32:0/2258253069 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05580a5640 con 0x7f0560072bc0 2024-04-17T09:57:16.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.495+0000 7f055de00640 1 -- 172.21.15.32:0/2258253069 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05580a57e0 con 0x7f0560072bc0 2024-04-17T09:57:16.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.495+0000 7f055de00640 1 -- 172.21.15.32:0/2258253069 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05580a5bb0 con 0x7f0560072bc0 2024-04-17T09:57:16.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.495+0000 7f0565400640 1 -- 172.21.15.32:0/2258253069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0560072bc0 msgr2=0x7f056006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:16.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.495+0000 7f0565400640 1 --2- 172.21.15.32:0/2258253069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0560072bc0 0x7f056006de50 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f05600b5aa0 tx=0x7f055801ca10 comp rx=0 tx=0).stop 2024-04-17T09:57:16.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.495+0000 7f0565400640 1 -- 172.21.15.32:0/2258253069 shutdown_connections 2024-04-17T09:57:16.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.495+0000 7f0565400640 1 --2- 172.21.15.32:0/2258253069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0560072bc0 0x7f056006de50 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.495+0000 7f0565400640 1 --2- 172.21.15.32:0/2258253069 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f056006f630 0x7f0560072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.495+0000 7f0565400640 1 -- 172.21.15.32:0/2258253069 >> 172.21.15.32:0/2258253069 conn(0x7f056006ba30 msgr2=0x7f056006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:16.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.495+0000 7f0565400640 1 -- 172.21.15.32:0/2258253069 shutdown_connections 2024-04-17T09:57:16.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.495+0000 7f0565400640 1 -- 172.21.15.32:0/2258253069 wait complete. 2024-04-17T09:57:16.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.495+0000 7f0565400640 1 Processor -- start 2024-04-17T09:57:16.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.495+0000 7f0565400640 1 -- start start 2024-04-17T09:57:16.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f0565400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f056006f630 0x7f056015e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:16.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f0565400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0560072bc0 0x7f0560142a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:16.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f0565400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0560151db0 0x7f0560142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:16.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f0565400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f05600b3fd0 con 0x7f0560151db0 2024-04-17T09:57:16.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f0565400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f05600b3cf0 con 0x7f056006f630 2024-04-17T09:57:16.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f0565400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f056006e390 con 0x7f0560072bc0 2024-04-17T09:57:16.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f055f200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0560151db0 0x7f0560142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:16.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f055f200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0560151db0 0x7f0560142fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59728/0 (socket says 172.21.15.32:59728) 2024-04-17T09:57:16.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f055f200640 1 -- 172.21.15.32:0/999842123 learned_addr learned my addr 172.21.15.32:0/999842123 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:16.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f0555e00640 1 --2- 172.21.15.32:0/999842123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0560072bc0 0x7f0560142a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:16.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f055e800640 1 --2- 172.21.15.32:0/999842123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f056006f630 0x7f056015e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:16.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f055f200640 1 -- 172.21.15.32:0/999842123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0560072bc0 msgr2=0x7f0560142a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:16.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f055f200640 1 --2- 172.21.15.32:0/999842123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0560072bc0 0x7f0560142a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f055f200640 1 -- 172.21.15.32:0/999842123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f056006f630 msgr2=0x7f056015e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:16.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f055f200640 1 --2- 172.21.15.32:0/999842123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f056006f630 0x7f056015e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f055f200640 1 -- 172.21.15.32:0/999842123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056014cf70 con 0x7f0560151db0 2024-04-17T09:57:16.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f055f200640 1 --2- 172.21.15.32:0/999842123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0560151db0 0x7f0560142fa0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f05580af040 tx=0x7f05580197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:16.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f0557400640 1 -- 172.21.15.32:0/999842123 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05580bc020 con 0x7f0560151db0 2024-04-17T09:57:16.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f0557400640 1 -- 172.21.15.32:0/999842123 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05580b8880 con 0x7f0560151db0 2024-04-17T09:57:16.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f0565400640 1 -- 172.21.15.32:0/999842123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f056014d210 con 0x7f0560151db0 2024-04-17T09:57:16.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f0557400640 1 -- 172.21.15.32:0/999842123 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0558009510 con 0x7f0560151db0 2024-04-17T09:57:16.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f0565400640 1 -- 172.21.15.32:0/999842123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f056015d160 con 0x7f0560151db0 2024-04-17T09:57:16.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f0565400640 1 -- 172.21.15.32:0/999842123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0560068a90 con 0x7f0560151db0 2024-04-17T09:57:16.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.499+0000 7f0557400640 1 -- 172.21.15.32:0/999842123 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f05580bf020 con 0x7f0560151db0 2024-04-17T09:57:16.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.503+0000 7f0557400640 1 --2- 172.21.15.32:0/999842123 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f053c068380 0x7f053c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:16.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.503+0000 7f0557400640 1 -- 172.21.15.32:0/999842123 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f0558121a90 con 0x7f0560151db0 2024-04-17T09:57:16.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.503+0000 7f055e800640 1 --2- 172.21.15.32:0/999842123 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f053c068380 0x7f053c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:16.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.503+0000 7f055e800640 1 --2- 172.21.15.32:0/999842123 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f053c068380 0x7f053c06a840 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0550001770 tx=0x7f0550009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:16.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.507+0000 7f0557400640 1 -- 172.21.15.32:0/999842123 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f05580c6030 con 0x7f0560151db0 2024-04-17T09:57:16.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.667+0000 7f0565400640 1 -- 172.21.15.32:0/999842123 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f056014f6b0 con 0x7f053c068380 2024-04-17T09:57:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.675+0000 7f0557400640 1 -- 172.21.15.32:0/999842123 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252067 (secure 0 0 0) 0x7f056014f6b0 con 0x7f053c068380 2024-04-17T09:57:16.677 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:57:16.679 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T09:57:16.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.675+0000 7f0565400640 1 -- 172.21.15.32:0/999842123 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f053c068380 msgr2=0x7f053c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:16.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.675+0000 7f0565400640 1 --2- 172.21.15.32:0/999842123 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f053c068380 0x7f053c06a840 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0550001770 tx=0x7f0550009040 comp rx=0 tx=0).stop 2024-04-17T09:57:16.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.675+0000 7f0565400640 1 -- 172.21.15.32:0/999842123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0560151db0 msgr2=0x7f0560142fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:16.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.675+0000 7f0565400640 1 --2- 172.21.15.32:0/999842123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0560151db0 0x7f0560142fa0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f05580af040 tx=0x7f05580197b0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.679+0000 7f0565400640 1 -- 172.21.15.32:0/999842123 shutdown_connections 2024-04-17T09:57:16.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.679+0000 7f0565400640 1 --2- 172.21.15.32:0/999842123 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f053c068380 0x7f053c06a840 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.679+0000 7f0565400640 1 --2- 172.21.15.32:0/999842123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0560072bc0 0x7f0560142a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.679+0000 7f0565400640 1 --2- 172.21.15.32:0/999842123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0560151db0 0x7f0560142fa0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.679+0000 7f0565400640 1 --2- 172.21.15.32:0/999842123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f056006f630 0x7f056015e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.679+0000 7f0565400640 1 -- 172.21.15.32:0/999842123 >> 172.21.15.32:0/999842123 conn(0x7f056006ba30 msgr2=0x7f0560147cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:16.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.679+0000 7f0565400640 1 -- 172.21.15.32:0/999842123 shutdown_connections 2024-04-17T09:57:16.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.679+0000 7f0565400640 1 -- 172.21.15.32:0/999842123 wait complete. 2024-04-17T09:57:16.698 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-04-17T09:57:15.337486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248784,"kb_used_data":7624,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754725936,"statfs":{"total":773094113280,"available":772839358464,"internally_reserved":0,"allocated":7806976,"data_stored":3927636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.192705"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280340+0000","last_change":"2024-04-17T09:57:12.195669+0000","last_active":"2024-04-17T09:57:13.280340+0000","last_peered":"2024-04-17T09:57:13.280340+0000","last_clean":"2024-04-17T09:57:13.280340+0000","last_became_active":"2024-04-17T09:57:12.190252+0000","last_became_peered":"2024-04-17T09:57:12.190252+0000","last_unstale":"2024-04-17T09:57:13.280340+0000","last_undegraded":"2024-04-17T09:57:13.280340+0000","last_fullsized":"2024-04-17T09:57:13.280340+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:14:16.115515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178791+0000","last_change":"2024-04-17T09:57:12.187616+0000","last_active":"2024-04-17T09:57:13.178791+0000","last_peered":"2024-04-17T09:57:13.178791+0000","last_clean":"2024-04-17T09:57:13.178791+0000","last_became_active":"2024-04-17T09:57:12.187185+0000","last_became_peered":"2024-04-17T09:57:12.187185+0000","last_unstale":"2024-04-17T09:57:13.178791+0000","last_undegraded":"2024-04-17T09:57:13.178791+0000","last_fullsized":"2024-04-17T09:57:13.178791+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:13:41.171346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280793+0000","last_change":"2024-04-17T09:57:12.181957+0000","last_active":"2024-04-17T09:57:13.280793+0000","last_peered":"2024-04-17T09:57:13.280793+0000","last_clean":"2024-04-17T09:57:13.280793+0000","last_became_active":"2024-04-17T09:57:12.181283+0000","last_became_peered":"2024-04-17T09:57:12.181283+0000","last_unstale":"2024-04-17T09:57:13.280793+0000","last_undegraded":"2024-04-17T09:57:13.280793+0000","last_fullsized":"2024-04-17T09:57:13.280793+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:26:33.940662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178084+0000","last_change":"2024-04-17T09:57:12.188878+0000","last_active":"2024-04-17T09:57:13.178084+0000","last_peered":"2024-04-17T09:57:13.178084+0000","last_clean":"2024-04-17T09:57:13.178084+0000","last_became_active":"2024-04-17T09:57:12.188550+0000","last_became_peered":"2024-04-17T09:57:12.188550+0000","last_unstale":"2024-04-17T09:57:13.178084+0000","last_undegraded":"2024-04-17T09:57:13.178084+0000","last_fullsized":"2024-04-17T09:57:13.178084+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:11:39.227578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.281001+0000","last_change":"2024-04-17T09:57:12.190639+0000","last_active":"2024-04-17T09:57:13.281001+0000","last_peered":"2024-04-17T09:57:13.281001+0000","last_clean":"2024-04-17T09:57:13.281001+0000","last_became_active":"2024-04-17T09:57:12.190274+0000","last_became_peered":"2024-04-17T09:57:12.190274+0000","last_unstale":"2024-04-17T09:57:13.281001+0000","last_undegraded":"2024-04-17T09:57:13.281001+0000","last_fullsized":"2024-04-17T09:57:13.281001+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:32:56.287955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178105+0000","last_change":"2024-04-17T09:57:12.186965+0000","last_active":"2024-04-17T09:57:13.178105+0000","last_peered":"2024-04-17T09:57:13.178105+0000","last_clean":"2024-04-17T09:57:13.178105+0000","last_became_active":"2024-04-17T09:57:12.186585+0000","last_became_peered":"2024-04-17T09:57:12.186585+0000","last_unstale":"2024-04-17T09:57:13.178105+0000","last_undegraded":"2024-04-17T09:57:13.178105+0000","last_fullsized":"2024-04-17T09:57:13.178105+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:27:58.379259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.184494+0000","last_change":"2024-04-17T09:57:12.184163+0000","last_active":"2024-04-17T09:57:13.184494+0000","last_peered":"2024-04-17T09:57:13.184494+0000","last_clean":"2024-04-17T09:57:13.184494+0000","last_became_active":"2024-04-17T09:57:12.183604+0000","last_became_peered":"2024-04-17T09:57:12.183604+0000","last_unstale":"2024-04-17T09:57:13.184494+0000","last_undegraded":"2024-04-17T09:57:13.184494+0000","last_fullsized":"2024-04-17T09:57:13.184494+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:31:42.597310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178506+0000","last_change":"2024-04-17T09:57:12.177540+0000","last_active":"2024-04-17T09:57:13.178506+0000","last_peered":"2024-04-17T09:57:13.178506+0000","last_clean":"2024-04-17T09:57:13.178506+0000","last_became_active":"2024-04-17T09:57:12.177261+0000","last_became_peered":"2024-04-17T09:57:12.177261+0000","last_unstale":"2024-04-17T09:57:13.178506+0000","last_undegraded":"2024-04-17T09:57:13.178506+0000","last_fullsized":"2024-04-17T09:57:13.178506+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:04:36.836807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280611+0000","last_change":"2024-04-17T09:57:12.186583+0000","last_active":"2024-04-17T09:57:13.280611+0000","last_peered":"2024-04-17T09:57:13.280611+0000","last_clean":"2024-04-17T09:57:13.280611+0000","last_became_active":"2024-04-17T09:57:12.186268+0000","last_became_peered":"2024-04-17T09:57:12.186268+0000","last_unstale":"2024-04-17T09:57:13.280611+0000","last_undegraded":"2024-04-17T09:57:13.280611+0000","last_fullsized":"2024-04-17T09:57:13.280611+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:15.478742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.179029+0000","last_change":"2024-04-17T09:57:12.189431+0000","last_active":"2024-04-17T09:57:13.179029+0000","last_peered":"2024-04-17T09:57:13.179029+0000","last_clean":"2024-04-17T09:57:13.179029+0000","last_became_active":"2024-04-17T09:57:12.189155+0000","last_became_peered":"2024-04-17T09:57:12.189155+0000","last_unstale":"2024-04-17T09:57:13.179029+0000","last_undegraded":"2024-04-17T09:57:13.179029+0000","last_fullsized":"2024-04-17T09:57:13.179029+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:28:41.242185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280250+0000","last_change":"2024-04-17T09:57:12.189979+0000","last_active":"2024-04-17T09:57:13.280250+0000","last_peered":"2024-04-17T09:57:13.280250+0000","last_clean":"2024-04-17T09:57:13.280250+0000","last_became_active":"2024-04-17T09:57:12.188653+0000","last_became_peered":"2024-04-17T09:57:12.188653+0000","last_unstale":"2024-04-17T09:57:13.280250+0000","last_undegraded":"2024-04-17T09:57:13.280250+0000","last_fullsized":"2024-04-17T09:57:13.280250+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:52:11.560396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178131+0000","last_change":"2024-04-17T09:57:12.193122+0000","last_active":"2024-04-17T09:57:13.178131+0000","last_peered":"2024-04-17T09:57:13.178131+0000","last_clean":"2024-04-17T09:57:13.178131+0000","last_became_active":"2024-04-17T09:57:12.192854+0000","last_became_peered":"2024-04-17T09:57:12.192854+0000","last_unstale":"2024-04-17T09:57:13.178131+0000","last_undegraded":"2024-04-17T09:57:13.178131+0000","last_fullsized":"2024-04-17T09:57:13.178131+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:17:00.921709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.190395+0000","last_change":"2024-04-17T09:57:12.181457+0000","last_active":"2024-04-17T09:57:13.190395+0000","last_peered":"2024-04-17T09:57:13.190395+0000","last_clean":"2024-04-17T09:57:13.190395+0000","last_became_active":"2024-04-17T09:57:12.181182+0000","last_became_peered":"2024-04-17T09:57:12.181182+0000","last_unstale":"2024-04-17T09:57:13.190395+0000","last_undegraded":"2024-04-17T09:57:13.190395+0000","last_fullsized":"2024-04-17T09:57:13.190395+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:02:08.135909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280809+0000","last_change":"2024-04-17T09:57:12.192162+0000","last_active":"2024-04-17T09:57:13.280809+0000","last_peered":"2024-04-17T09:57:13.280809+0000","last_clean":"2024-04-17T09:57:13.280809+0000","last_became_active":"2024-04-17T09:57:12.191616+0000","last_became_peered":"2024-04-17T09:57:12.191616+0000","last_unstale":"2024-04-17T09:57:13.280809+0000","last_undegraded":"2024-04-17T09:57:13.280809+0000","last_fullsized":"2024-04-17T09:57:13.280809+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:36:28.915146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.190166+0000","last_change":"2024-04-17T09:57:12.184628+0000","last_active":"2024-04-17T09:57:13.190166+0000","last_peered":"2024-04-17T09:57:13.190166+0000","last_clean":"2024-04-17T09:57:13.190166+0000","last_became_active":"2024-04-17T09:57:12.184103+0000","last_became_peered":"2024-04-17T09:57:12.184103+0000","last_unstale":"2024-04-17T09:57:13.190166+0000","last_undegraded":"2024-04-17T09:57:13.190166+0000","last_fullsized":"2024-04-17T09:57:13.190166+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:36:02.427710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178070+0000","last_change":"2024-04-17T09:57:12.183674+0000","last_active":"2024-04-17T09:57:13.178070+0000","last_peered":"2024-04-17T09:57:13.178070+0000","last_clean":"2024-04-17T09:57:13.178070+0000","last_became_active":"2024-04-17T09:57:12.183368+0000","last_became_peered":"2024-04-17T09:57:12.183368+0000","last_unstale":"2024-04-17T09:57:13.178070+0000","last_undegraded":"2024-04-17T09:57:13.178070+0000","last_fullsized":"2024-04-17T09:57:13.178070+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:03:48.599405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280754+0000","last_change":"2024-04-17T09:57:12.186766+0000","last_active":"2024-04-17T09:57:13.280754+0000","last_peered":"2024-04-17T09:57:13.280754+0000","last_clean":"2024-04-17T09:57:13.280754+0000","last_became_active":"2024-04-17T09:57:12.185955+0000","last_became_peered":"2024-04-17T09:57:12.185955+0000","last_unstale":"2024-04-17T09:57:13.280754+0000","last_undegraded":"2024-04-17T09:57:13.280754+0000","last_fullsized":"2024-04-17T09:57:13.280754+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:11.273872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178056+0000","last_change":"2024-04-17T09:57:12.183751+0000","last_active":"2024-04-17T09:57:13.178056+0000","last_peered":"2024-04-17T09:57:13.178056+0000","last_clean":"2024-04-17T09:57:13.178056+0000","last_became_active":"2024-04-17T09:57:12.183411+0000","last_became_peered":"2024-04-17T09:57:12.183411+0000","last_unstale":"2024-04-17T09:57:13.178056+0000","last_undegraded":"2024-04-17T09:57:13.178056+0000","last_fullsized":"2024-04-17T09:57:13.178056+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:54.274231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178440+0000","last_change":"2024-04-17T09:57:12.182399+0000","last_active":"2024-04-17T09:57:13.178440+0000","last_peered":"2024-04-17T09:57:13.178440+0000","last_clean":"2024-04-17T09:57:13.178440+0000","last_became_active":"2024-04-17T09:57:12.182114+0000","last_became_peered":"2024-04-17T09:57:12.182114+0000","last_unstale":"2024-04-17T09:57:13.178440+0000","last_undegraded":"2024-04-17T09:57:13.178440+0000","last_fullsized":"2024-04-17T09:57:13.178440+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:59:48.176397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.190359+0000","last_change":"2024-04-17T09:57:12.192235+0000","last_active":"2024-04-17T09:57:13.190359+0000","last_peered":"2024-04-17T09:57:13.190359+0000","last_clean":"2024-04-17T09:57:13.190359+0000","last_became_active":"2024-04-17T09:57:12.188188+0000","last_became_peered":"2024-04-17T09:57:12.188188+0000","last_unstale":"2024-04-17T09:57:13.190359+0000","last_undegraded":"2024-04-17T09:57:13.190359+0000","last_fullsized":"2024-04-17T09:57:13.190359+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:57.006419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.177913+0000","last_change":"2024-04-17T09:57:12.174012+0000","last_active":"2024-04-17T09:57:13.177913+0000","last_peered":"2024-04-17T09:57:13.177913+0000","last_clean":"2024-04-17T09:57:13.177913+0000","last_became_active":"2024-04-17T09:57:12.172539+0000","last_became_peered":"2024-04-17T09:57:12.172539+0000","last_unstale":"2024-04-17T09:57:13.177913+0000","last_undegraded":"2024-04-17T09:57:13.177913+0000","last_fullsized":"2024-04-17T09:57:13.177913+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:14:03.254732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.177957+0000","last_change":"2024-04-17T09:57:12.197226+0000","last_active":"2024-04-17T09:57:13.177957+0000","last_peered":"2024-04-17T09:57:13.177957+0000","last_clean":"2024-04-17T09:57:13.177957+0000","last_became_active":"2024-04-17T09:57:12.196859+0000","last_became_peered":"2024-04-17T09:57:12.196859+0000","last_unstale":"2024-04-17T09:57:13.177957+0000","last_undegraded":"2024-04-17T09:57:13.177957+0000","last_fullsized":"2024-04-17T09:57:13.177957+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:31:04.978298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.279906+0000","last_change":"2024-04-17T09:57:12.185124+0000","last_active":"2024-04-17T09:57:13.279906+0000","last_peered":"2024-04-17T09:57:13.279906+0000","last_clean":"2024-04-17T09:57:13.279906+0000","last_became_active":"2024-04-17T09:57:12.184473+0000","last_became_peered":"2024-04-17T09:57:12.184473+0000","last_unstale":"2024-04-17T09:57:13.279906+0000","last_undegraded":"2024-04-17T09:57:13.279906+0000","last_fullsized":"2024-04-17T09:57:13.279906+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:58:43.470530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.279825+0000","last_change":"2024-04-17T09:57:12.182939+0000","last_active":"2024-04-17T09:57:13.279825+0000","last_peered":"2024-04-17T09:57:13.279825+0000","last_clean":"2024-04-17T09:57:13.279825+0000","last_became_active":"2024-04-17T09:57:12.182666+0000","last_became_peered":"2024-04-17T09:57:12.182666+0000","last_unstale":"2024-04-17T09:57:13.279825+0000","last_undegraded":"2024-04-17T09:57:13.279825+0000","last_fullsized":"2024-04-17T09:57:13.279825+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:33:49.810693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.179045+0000","last_change":"2024-04-17T09:57:12.180577+0000","last_active":"2024-04-17T09:57:13.179045+0000","last_peered":"2024-04-17T09:57:13.179045+0000","last_clean":"2024-04-17T09:57:13.179045+0000","last_became_active":"2024-04-17T09:57:12.179258+0000","last_became_peered":"2024-04-17T09:57:12.179258+0000","last_unstale":"2024-04-17T09:57:13.179045+0000","last_undegraded":"2024-04-17T09:57:13.179045+0000","last_fullsized":"2024-04-17T09:57:13.179045+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:24:16.821303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.184721+0000","last_change":"2024-04-17T09:57:12.193247+0000","last_active":"2024-04-17T09:57:13.184721+0000","last_peered":"2024-04-17T09:57:13.184721+0000","last_clean":"2024-04-17T09:57:13.184721+0000","last_became_active":"2024-04-17T09:57:12.192192+0000","last_became_peered":"2024-04-17T09:57:12.192192+0000","last_unstale":"2024-04-17T09:57:13.184721+0000","last_undegraded":"2024-04-17T09:57:13.184721+0000","last_fullsized":"2024-04-17T09:57:13.184721+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:51:37.257497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280959+0000","last_change":"2024-04-17T09:57:12.182270+0000","last_active":"2024-04-17T09:57:13.280959+0000","last_peered":"2024-04-17T09:57:13.280959+0000","last_clean":"2024-04-17T09:57:13.280959+0000","last_became_active":"2024-04-17T09:57:12.181623+0000","last_became_peered":"2024-04-17T09:57:12.181623+0000","last_unstale":"2024-04-17T09:57:13.280959+0000","last_undegraded":"2024-04-17T09:57:13.280959+0000","last_fullsized":"2024-04-17T09:57:13.280959+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:18:50.937330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178423+0000","last_change":"2024-04-17T09:57:12.190674+0000","last_active":"2024-04-17T09:57:13.178423+0000","last_peered":"2024-04-17T09:57:13.178423+0000","last_clean":"2024-04-17T09:57:13.178423+0000","last_became_active":"2024-04-17T09:57:12.189886+0000","last_became_peered":"2024-04-17T09:57:12.189886+0000","last_unstale":"2024-04-17T09:57:13.178423+0000","last_undegraded":"2024-04-17T09:57:13.178423+0000","last_fullsized":"2024-04-17T09:57:13.178423+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:51:57.138305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280729+0000","last_change":"2024-04-17T09:57:12.186070+0000","last_active":"2024-04-17T09:57:13.280729+0000","last_peered":"2024-04-17T09:57:13.280729+0000","last_clean":"2024-04-17T09:57:13.280729+0000","last_became_active":"2024-04-17T09:57:12.185160+0000","last_became_peered":"2024-04-17T09:57:12.185160+0000","last_unstale":"2024-04-17T09:57:13.280729+0000","last_undegraded":"2024-04-17T09:57:13.280729+0000","last_fullsized":"2024-04-17T09:57:13.280729+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:48:29.586244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.184534+0000","last_change":"2024-04-17T09:57:12.189730+0000","last_active":"2024-04-17T09:57:13.184534+0000","last_peered":"2024-04-17T09:57:13.184534+0000","last_clean":"2024-04-17T09:57:13.184534+0000","last_became_active":"2024-04-17T09:57:12.188417+0000","last_became_peered":"2024-04-17T09:57:12.188417+0000","last_unstale":"2024-04-17T09:57:13.184534+0000","last_undegraded":"2024-04-17T09:57:13.184534+0000","last_fullsized":"2024-04-17T09:57:13.184534+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:17:22.472846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.184735+0000","last_change":"2024-04-17T09:57:12.193999+0000","last_active":"2024-04-17T09:57:13.184735+0000","last_peered":"2024-04-17T09:57:13.184735+0000","last_clean":"2024-04-17T09:57:13.184735+0000","last_became_active":"2024-04-17T09:57:12.192355+0000","last_became_peered":"2024-04-17T09:57:12.192355+0000","last_unstale":"2024-04-17T09:57:13.184735+0000","last_undegraded":"2024-04-17T09:57:13.184735+0000","last_fullsized":"2024-04-17T09:57:13.184735+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:56:05.919839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178523+0000","last_change":"2024-04-17T09:57:12.186381+0000","last_active":"2024-04-17T09:57:13.178523+0000","last_peered":"2024-04-17T09:57:13.178523+0000","last_clean":"2024-04-17T09:57:13.178523+0000","last_became_active":"2024-04-17T09:57:12.185938+0000","last_became_peered":"2024-04-17T09:57:12.185938+0000","last_unstale":"2024-04-17T09:57:13.178523+0000","last_undegraded":"2024-04-17T09:57:13.178523+0000","last_fullsized":"2024-04-17T09:57:13.178523+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:56:27.144132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280125+0000","last_change":"2024-04-17T09:57:12.185704+0000","last_active":"2024-04-17T09:57:13.280125+0000","last_peered":"2024-04-17T09:57:13.280125+0000","last_clean":"2024-04-17T09:57:13.280125+0000","last_became_active":"2024-04-17T09:57:12.185357+0000","last_became_peered":"2024-04-17T09:57:12.185357+0000","last_unstale":"2024-04-17T09:57:13.280125+0000","last_undegraded":"2024-04-17T09:57:13.280125+0000","last_fullsized":"2024-04-17T09:57:13.280125+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:02:10.369272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"67'4","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280399+0000","last_change":"2024-04-17T09:57:11.170675+0000","last_active":"2024-04-17T09:57:13.280399+0000","last_peered":"2024-04-17T09:57:13.280399+0000","last_clean":"2024-04-17T09:57:13.280399+0000","last_became_active":"2024-04-17T09:57:11.170375+0000","last_became_peered":"2024-04-17T09:57:11.170375+0000","last_unstale":"2024-04-17T09:57:13.280399+0000","last_undegraded":"2024-04-17T09:57:13.280399+0000","last_fullsized":"2024-04-17T09:57:13.280399+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:32:01.757683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280757+0000","last_change":"2024-04-17T09:57:12.186253+0000","last_active":"2024-04-17T09:57:13.280757+0000","last_peered":"2024-04-17T09:57:13.280757+0000","last_clean":"2024-04-17T09:57:13.280757+0000","last_became_active":"2024-04-17T09:57:12.185924+0000","last_became_peered":"2024-04-17T09:57:12.185924+0000","last_unstale":"2024-04-17T09:57:13.280757+0000","last_undegraded":"2024-04-17T09:57:13.280757+0000","last_fullsized":"2024-04-17T09:57:13.280757+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:42:21.329246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.190398+0000","last_change":"2024-04-17T09:57:11.170736+0000","last_active":"2024-04-17T09:57:13.190398+0000","last_peered":"2024-04-17T09:57:13.190398+0000","last_clean":"2024-04-17T09:57:13.190398+0000","last_became_active":"2024-04-17T09:57:11.169279+0000","last_became_peered":"2024-04-17T09:57:11.169279+0000","last_unstale":"2024-04-17T09:57:13.190398+0000","last_undegraded":"2024-04-17T09:57:13.190398+0000","last_fullsized":"2024-04-17T09:57:13.190398+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:38:41.791856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178321+0000","last_change":"2024-04-17T09:57:12.184681+0000","last_active":"2024-04-17T09:57:13.178321+0000","last_peered":"2024-04-17T09:57:13.178321+0000","last_clean":"2024-04-17T09:57:13.178321+0000","last_became_active":"2024-04-17T09:57:12.184234+0000","last_became_peered":"2024-04-17T09:57:12.184234+0000","last_unstale":"2024-04-17T09:57:13.178321+0000","last_undegraded":"2024-04-17T09:57:13.178321+0000","last_fullsized":"2024-04-17T09:57:13.178321+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:48:51.551216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178726+0000","last_change":"2024-04-17T09:57:11.165813+0000","last_active":"2024-04-17T09:57:13.178726+0000","last_peered":"2024-04-17T09:57:13.178726+0000","last_clean":"2024-04-17T09:57:13.178726+0000","last_became_active":"2024-04-17T09:57:11.165560+0000","last_became_peered":"2024-04-17T09:57:11.165560+0000","last_unstale":"2024-04-17T09:57:13.178726+0000","last_undegraded":"2024-04-17T09:57:13.178726+0000","last_fullsized":"2024-04-17T09:57:13.178726+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:42:12.470791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280208+0000","last_change":"2024-04-17T09:57:12.189349+0000","last_active":"2024-04-17T09:57:13.280208+0000","last_peered":"2024-04-17T09:57:13.280208+0000","last_clean":"2024-04-17T09:57:13.280208+0000","last_became_active":"2024-04-17T09:57:12.189012+0000","last_became_peered":"2024-04-17T09:57:12.189012+0000","last_unstale":"2024-04-17T09:57:13.280208+0000","last_undegraded":"2024-04-17T09:57:13.280208+0000","last_fullsized":"2024-04-17T09:57:13.280208+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:06:50.422951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178589+0000","last_change":"2024-04-17T09:57:11.170949+0000","last_active":"2024-04-17T09:57:13.178589+0000","last_peered":"2024-04-17T09:57:13.178589+0000","last_clean":"2024-04-17T09:57:13.178589+0000","last_became_active":"2024-04-17T09:57:11.170678+0000","last_became_peered":"2024-04-17T09:57:11.170678+0000","last_unstale":"2024-04-17T09:57:13.178589+0000","last_undegraded":"2024-04-17T09:57:13.178589+0000","last_fullsized":"2024-04-17T09:57:13.178589+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:14:34.753436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.184832+0000","last_change":"2024-04-17T09:57:12.186747+0000","last_active":"2024-04-17T09:57:13.184832+0000","last_peered":"2024-04-17T09:57:13.184832+0000","last_clean":"2024-04-17T09:57:13.184832+0000","last_became_active":"2024-04-17T09:57:12.185746+0000","last_became_peered":"2024-04-17T09:57:12.185746+0000","last_unstale":"2024-04-17T09:57:13.184832+0000","last_undegraded":"2024-04-17T09:57:13.184832+0000","last_fullsized":"2024-04-17T09:57:13.184832+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:21:18.012312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280901+0000","last_change":"2024-04-17T09:57:11.175339+0000","last_active":"2024-04-17T09:57:13.280901+0000","last_peered":"2024-04-17T09:57:13.280901+0000","last_clean":"2024-04-17T09:57:13.280901+0000","last_became_active":"2024-04-17T09:57:11.174554+0000","last_became_peered":"2024-04-17T09:57:11.174554+0000","last_unstale":"2024-04-17T09:57:13.280901+0000","last_undegraded":"2024-04-17T09:57:13.280901+0000","last_fullsized":"2024-04-17T09:57:13.280901+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:36:02.179193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178090+0000","last_change":"2024-04-17T09:57:12.179416+0000","last_active":"2024-04-17T09:57:13.178090+0000","last_peered":"2024-04-17T09:57:13.178090+0000","last_clean":"2024-04-17T09:57:13.178090+0000","last_became_active":"2024-04-17T09:57:12.178618+0000","last_became_peered":"2024-04-17T09:57:12.178618+0000","last_unstale":"2024-04-17T09:57:13.178090+0000","last_undegraded":"2024-04-17T09:57:13.178090+0000","last_fullsized":"2024-04-17T09:57:13.178090+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:19:36.202473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178159+0000","last_change":"2024-04-17T09:57:11.171969+0000","last_active":"2024-04-17T09:57:13.178159+0000","last_peered":"2024-04-17T09:57:13.178159+0000","last_clean":"2024-04-17T09:57:13.178159+0000","last_became_active":"2024-04-17T09:57:11.171116+0000","last_became_peered":"2024-04-17T09:57:11.171116+0000","last_unstale":"2024-04-17T09:57:13.178159+0000","last_undegraded":"2024-04-17T09:57:13.178159+0000","last_fullsized":"2024-04-17T09:57:13.178159+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:26:50.890315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280114+0000","last_change":"2024-04-17T09:57:12.189773+0000","last_active":"2024-04-17T09:57:13.280114+0000","last_peered":"2024-04-17T09:57:13.280114+0000","last_clean":"2024-04-17T09:57:13.280114+0000","last_became_active":"2024-04-17T09:57:12.189492+0000","last_became_peered":"2024-04-17T09:57:12.189492+0000","last_unstale":"2024-04-17T09:57:13.280114+0000","last_undegraded":"2024-04-17T09:57:13.280114+0000","last_fullsized":"2024-04-17T09:57:13.280114+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:54:34.634350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.177750+0000","last_change":"2024-04-17T09:57:11.171698+0000","last_active":"2024-04-17T09:57:13.177750+0000","last_peered":"2024-04-17T09:57:13.177750+0000","last_clean":"2024-04-17T09:57:13.177750+0000","last_became_active":"2024-04-17T09:57:11.169753+0000","last_became_peered":"2024-04-17T09:57:11.169753+0000","last_unstale":"2024-04-17T09:57:13.177750+0000","last_undegraded":"2024-04-17T09:57:13.177750+0000","last_fullsized":"2024-04-17T09:57:13.177750+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:36:27.906424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.190193+0000","last_change":"2024-04-17T09:57:12.176677+0000","last_active":"2024-04-17T09:57:13.190193+0000","last_peered":"2024-04-17T09:57:13.190193+0000","last_clean":"2024-04-17T09:57:13.190193+0000","last_became_active":"2024-04-17T09:57:12.176403+0000","last_became_peered":"2024-04-17T09:57:12.176403+0000","last_unstale":"2024-04-17T09:57:13.190193+0000","last_undegraded":"2024-04-17T09:57:13.190193+0000","last_fullsized":"2024-04-17T09:57:13.190193+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:54:37.533520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280098+0000","last_change":"2024-04-17T09:57:11.173923+0000","last_active":"2024-04-17T09:57:13.280098+0000","last_peered":"2024-04-17T09:57:13.280098+0000","last_clean":"2024-04-17T09:57:13.280098+0000","last_became_active":"2024-04-17T09:57:11.173611+0000","last_became_peered":"2024-04-17T09:57:11.173611+0000","last_unstale":"2024-04-17T09:57:13.280098+0000","last_undegraded":"2024-04-17T09:57:13.280098+0000","last_fullsized":"2024-04-17T09:57:13.280098+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:07:36.784027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.184874+0000","last_change":"2024-04-17T09:57:12.192530+0000","last_active":"2024-04-17T09:57:13.184874+0000","last_peered":"2024-04-17T09:57:13.184874+0000","last_clean":"2024-04-17T09:57:13.184874+0000","last_became_active":"2024-04-17T09:57:12.191774+0000","last_became_peered":"2024-04-17T09:57:12.191774+0000","last_unstale":"2024-04-17T09:57:13.184874+0000","last_undegraded":"2024-04-17T09:57:13.184874+0000","last_fullsized":"2024-04-17T09:57:13.184874+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:23:49.049846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.179200+0000","last_change":"2024-04-17T09:57:11.167108+0000","last_active":"2024-04-17T09:57:13.179200+0000","last_peered":"2024-04-17T09:57:13.179200+0000","last_clean":"2024-04-17T09:57:13.179200+0000","last_became_active":"2024-04-17T09:57:11.166856+0000","last_became_peered":"2024-04-17T09:57:11.166856+0000","last_unstale":"2024-04-17T09:57:13.179200+0000","last_undegraded":"2024-04-17T09:57:13.179200+0000","last_fullsized":"2024-04-17T09:57:13.179200+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:13.063150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280331+0000","last_change":"2024-04-17T09:57:12.195940+0000","last_active":"2024-04-17T09:57:13.280331+0000","last_peered":"2024-04-17T09:57:13.280331+0000","last_clean":"2024-04-17T09:57:13.280331+0000","last_became_active":"2024-04-17T09:57:12.190743+0000","last_became_peered":"2024-04-17T09:57:12.190743+0000","last_unstale":"2024-04-17T09:57:13.280331+0000","last_undegraded":"2024-04-17T09:57:13.280331+0000","last_fullsized":"2024-04-17T09:57:13.280331+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:04:47.401445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178953+0000","last_change":"2024-04-17T09:57:11.170518+0000","last_active":"2024-04-17T09:57:13.178953+0000","last_peered":"2024-04-17T09:57:13.178953+0000","last_clean":"2024-04-17T09:57:13.178953+0000","last_became_active":"2024-04-17T09:57:11.168881+0000","last_became_peered":"2024-04-17T09:57:11.168881+0000","last_unstale":"2024-04-17T09:57:13.178953+0000","last_undegraded":"2024-04-17T09:57:13.178953+0000","last_fullsized":"2024-04-17T09:57:13.178953+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:58:00.221995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178509+0000","last_change":"2024-04-17T09:57:12.186953+0000","last_active":"2024-04-17T09:57:13.178509+0000","last_peered":"2024-04-17T09:57:13.178509+0000","last_clean":"2024-04-17T09:57:13.178509+0000","last_became_active":"2024-04-17T09:57:12.186674+0000","last_became_peered":"2024-04-17T09:57:12.186674+0000","last_unstale":"2024-04-17T09:57:13.178509+0000","last_undegraded":"2024-04-17T09:57:13.178509+0000","last_fullsized":"2024-04-17T09:57:13.178509+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:46:15.041323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"67'3","reported_seq":17,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280924+0000","last_change":"2024-04-17T09:57:11.176105+0000","last_active":"2024-04-17T09:57:13.280924+0000","last_peered":"2024-04-17T09:57:13.280924+0000","last_clean":"2024-04-17T09:57:13.280924+0000","last_became_active":"2024-04-17T09:57:11.175555+0000","last_became_peered":"2024-04-17T09:57:11.175555+0000","last_unstale":"2024-04-17T09:57:13.280924+0000","last_undegraded":"2024-04-17T09:57:13.280924+0000","last_fullsized":"2024-04-17T09:57:13.280924+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:30.144661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.281057+0000","last_change":"2024-04-17T09:57:12.187481+0000","last_active":"2024-04-17T09:57:13.281057+0000","last_peered":"2024-04-17T09:57:13.281057+0000","last_clean":"2024-04-17T09:57:13.281057+0000","last_became_active":"2024-04-17T09:57:12.187199+0000","last_became_peered":"2024-04-17T09:57:12.187199+0000","last_unstale":"2024-04-17T09:57:13.281057+0000","last_undegraded":"2024-04-17T09:57:13.281057+0000","last_fullsized":"2024-04-17T09:57:13.281057+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:36:14.968360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"67'6","reported_seq":20,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178171+0000","last_change":"2024-04-17T09:57:11.173441+0000","last_active":"2024-04-17T09:57:13.178171+0000","last_peered":"2024-04-17T09:57:13.178171+0000","last_clean":"2024-04-17T09:57:13.178171+0000","last_became_active":"2024-04-17T09:57:11.173130+0000","last_became_peered":"2024-04-17T09:57:11.173130+0000","last_unstale":"2024-04-17T09:57:13.178171+0000","last_undegraded":"2024-04-17T09:57:13.178171+0000","last_fullsized":"2024-04-17T09:57:13.178171+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:28:37.561860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280031+0000","last_change":"2024-04-17T09:57:12.192904+0000","last_active":"2024-04-17T09:57:13.280031+0000","last_peered":"2024-04-17T09:57:13.280031+0000","last_clean":"2024-04-17T09:57:13.280031+0000","last_became_active":"2024-04-17T09:57:12.192617+0000","last_became_peered":"2024-04-17T09:57:12.192617+0000","last_unstale":"2024-04-17T09:57:13.280031+0000","last_undegraded":"2024-04-17T09:57:13.280031+0000","last_fullsized":"2024-04-17T09:57:13.280031+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:36:03.187915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.177899+0000","last_change":"2024-04-17T09:57:11.174377+0000","last_active":"2024-04-17T09:57:13.177899+0000","last_peered":"2024-04-17T09:57:13.177899+0000","last_clean":"2024-04-17T09:57:13.177899+0000","last_became_active":"2024-04-17T09:57:11.174102+0000","last_became_peered":"2024-04-17T09:57:11.174102+0000","last_unstale":"2024-04-17T09:57:13.177899+0000","last_undegraded":"2024-04-17T09:57:13.177899+0000","last_fullsized":"2024-04-17T09:57:13.177899+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:42:20.320524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.184621+0000","last_change":"2024-04-17T09:57:12.189552+0000","last_active":"2024-04-17T09:57:13.184621+0000","last_peered":"2024-04-17T09:57:13.184621+0000","last_clean":"2024-04-17T09:57:13.184621+0000","last_became_active":"2024-04-17T09:57:12.189206+0000","last_became_peered":"2024-04-17T09:57:12.189206+0000","last_unstale":"2024-04-17T09:57:13.184621+0000","last_undegraded":"2024-04-17T09:57:13.184621+0000","last_fullsized":"2024-04-17T09:57:13.184621+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:32:02.766405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178261+0000","last_change":"2024-04-17T09:57:11.172197+0000","last_active":"2024-04-17T09:57:13.178261+0000","last_peered":"2024-04-17T09:57:13.178261+0000","last_clean":"2024-04-17T09:57:13.178261+0000","last_became_active":"2024-04-17T09:57:11.171480+0000","last_became_peered":"2024-04-17T09:57:11.171480+0000","last_unstale":"2024-04-17T09:57:13.178261+0000","last_undegraded":"2024-04-17T09:57:13.178261+0000","last_fullsized":"2024-04-17T09:57:13.178261+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:53.265509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.190063+0000","last_change":"2024-04-17T09:57:12.182697+0000","last_active":"2024-04-17T09:57:13.190063+0000","last_peered":"2024-04-17T09:57:13.190063+0000","last_clean":"2024-04-17T09:57:13.190063+0000","last_became_active":"2024-04-17T09:57:12.182406+0000","last_became_peered":"2024-04-17T09:57:12.182406+0000","last_unstale":"2024-04-17T09:57:13.190063+0000","last_undegraded":"2024-04-17T09:57:13.190063+0000","last_fullsized":"2024-04-17T09:57:13.190063+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:36:54.671129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.184479+0000","last_change":"2024-04-17T09:57:11.175028+0000","last_active":"2024-04-17T09:57:13.184479+0000","last_peered":"2024-04-17T09:57:13.184479+0000","last_clean":"2024-04-17T09:57:13.184479+0000","last_became_active":"2024-04-17T09:57:11.174647+0000","last_became_peered":"2024-04-17T09:57:11.174647+0000","last_unstale":"2024-04-17T09:57:13.184479+0000","last_undegraded":"2024-04-17T09:57:13.184479+0000","last_fullsized":"2024-04-17T09:57:13.184479+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:23:51.478191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280850+0000","last_change":"2024-04-17T09:57:12.192516+0000","last_active":"2024-04-17T09:57:13.280850+0000","last_peered":"2024-04-17T09:57:13.280850+0000","last_clean":"2024-04-17T09:57:13.280850+0000","last_became_active":"2024-04-17T09:57:12.192220+0000","last_became_peered":"2024-04-17T09:57:12.192220+0000","last_unstale":"2024-04-17T09:57:13.280850+0000","last_undegraded":"2024-04-17T09:57:13.280850+0000","last_fullsized":"2024-04-17T09:57:13.280850+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:28:38.570582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"67'8","reported_seq":22,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.177678+0000","last_change":"2024-04-17T09:57:11.166857+0000","last_active":"2024-04-17T09:57:13.177678+0000","last_peered":"2024-04-17T09:57:13.177678+0000","last_clean":"2024-04-17T09:57:13.177678+0000","last_became_active":"2024-04-17T09:57:11.166543+0000","last_became_peered":"2024-04-17T09:57:11.166543+0000","last_unstale":"2024-04-17T09:57:13.177678+0000","last_undegraded":"2024-04-17T09:57:13.177678+0000","last_fullsized":"2024-04-17T09:57:13.177678+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:23:15.843700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.179206+0000","last_change":"2024-04-17T09:57:12.179053+0000","last_active":"2024-04-17T09:57:13.179206+0000","last_peered":"2024-04-17T09:57:13.179206+0000","last_clean":"2024-04-17T09:57:13.179206+0000","last_became_active":"2024-04-17T09:57:12.172803+0000","last_became_peered":"2024-04-17T09:57:12.172803+0000","last_unstale":"2024-04-17T09:57:13.179206+0000","last_undegraded":"2024-04-17T09:57:13.179206+0000","last_fullsized":"2024-04-17T09:57:13.179206+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:38:42.800578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.179029+0000","last_change":"2024-04-17T09:57:11.174027+0000","last_active":"2024-04-17T09:57:13.179029+0000","last_peered":"2024-04-17T09:57:13.179029+0000","last_clean":"2024-04-17T09:57:13.179029+0000","last_became_active":"2024-04-17T09:57:11.173713+0000","last_became_peered":"2024-04-17T09:57:11.173713+0000","last_unstale":"2024-04-17T09:57:13.179029+0000","last_undegraded":"2024-04-17T09:57:13.179029+0000","last_fullsized":"2024-04-17T09:57:13.179029+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:32:55.279233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280446+0000","last_change":"2024-04-17T09:57:12.184938+0000","last_active":"2024-04-17T09:57:13.280446+0000","last_peered":"2024-04-17T09:57:13.280446+0000","last_clean":"2024-04-17T09:57:13.280446+0000","last_became_active":"2024-04-17T09:57:12.184175+0000","last_became_peered":"2024-04-17T09:57:12.184175+0000","last_unstale":"2024-04-17T09:57:13.280446+0000","last_undegraded":"2024-04-17T09:57:13.280446+0000","last_fullsized":"2024-04-17T09:57:13.280446+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:04:03.401353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.179248+0000","last_change":"2024-04-17T09:57:11.169402+0000","last_active":"2024-04-17T09:57:13.179248+0000","last_peered":"2024-04-17T09:57:13.179248+0000","last_clean":"2024-04-17T09:57:13.179248+0000","last_became_active":"2024-04-17T09:57:11.168674+0000","last_became_peered":"2024-04-17T09:57:11.168674+0000","last_unstale":"2024-04-17T09:57:13.179248+0000","last_undegraded":"2024-04-17T09:57:13.179248+0000","last_fullsized":"2024-04-17T09:57:13.179248+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:48.357059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178222+0000","last_change":"2024-04-17T09:57:12.193004+0000","last_active":"2024-04-17T09:57:13.178222+0000","last_peered":"2024-04-17T09:57:13.178222+0000","last_clean":"2024-04-17T09:57:13.178222+0000","last_became_active":"2024-04-17T09:57:12.192651+0000","last_became_peered":"2024-04-17T09:57:12.192651+0000","last_unstale":"2024-04-17T09:57:13.178222+0000","last_undegraded":"2024-04-17T09:57:13.178222+0000","last_fullsized":"2024-04-17T09:57:13.178222+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:55:43.451961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178588+0000","last_change":"2024-04-17T09:57:11.169785+0000","last_active":"2024-04-17T09:57:13.178588+0000","last_peered":"2024-04-17T09:57:13.178588+0000","last_clean":"2024-04-17T09:57:13.178588+0000","last_became_active":"2024-04-17T09:57:11.169471+0000","last_became_peered":"2024-04-17T09:57:11.169471+0000","last_unstale":"2024-04-17T09:57:13.178588+0000","last_undegraded":"2024-04-17T09:57:13.178588+0000","last_fullsized":"2024-04-17T09:57:13.178588+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:53.582494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280560+0000","last_change":"2024-04-17T09:57:12.189990+0000","last_active":"2024-04-17T09:57:13.280560+0000","last_peered":"2024-04-17T09:57:13.280560+0000","last_clean":"2024-04-17T09:57:13.280560+0000","last_became_active":"2024-04-17T09:57:12.189615+0000","last_became_peered":"2024-04-17T09:57:12.189615+0000","last_unstale":"2024-04-17T09:57:13.280560+0000","last_undegraded":"2024-04-17T09:57:13.280560+0000","last_fullsized":"2024-04-17T09:57:13.280560+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:53:50.111959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.279976+0000","last_change":"2024-04-17T09:57:11.165394+0000","last_active":"2024-04-17T09:57:13.279976+0000","last_peered":"2024-04-17T09:57:13.279976+0000","last_clean":"2024-04-17T09:57:13.279976+0000","last_became_active":"2024-04-17T09:57:11.165120+0000","last_became_peered":"2024-04-17T09:57:11.165120+0000","last_unstale":"2024-04-17T09:57:13.279976+0000","last_undegraded":"2024-04-17T09:57:13.279976+0000","last_fullsized":"2024-04-17T09:57:13.279976+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:17:21.464124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280986+0000","last_change":"2024-04-17T09:57:12.176860+0000","last_active":"2024-04-17T09:57:13.280986+0000","last_peered":"2024-04-17T09:57:13.280986+0000","last_clean":"2024-04-17T09:57:13.280986+0000","last_became_active":"2024-04-17T09:57:12.174767+0000","last_became_peered":"2024-04-17T09:57:12.174767+0000","last_unstale":"2024-04-17T09:57:13.280986+0000","last_undegraded":"2024-04-17T09:57:13.280986+0000","last_fullsized":"2024-04-17T09:57:13.280986+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:08:40.914887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178957+0000","last_change":"2024-04-17T09:57:11.171737+0000","last_active":"2024-04-17T09:57:13.178957+0000","last_peered":"2024-04-17T09:57:13.178957+0000","last_clean":"2024-04-17T09:57:13.178957+0000","last_became_active":"2024-04-17T09:57:11.170748+0000","last_became_peered":"2024-04-17T09:57:11.170748+0000","last_unstale":"2024-04-17T09:57:13.178957+0000","last_undegraded":"2024-04-17T09:57:13.178957+0000","last_fullsized":"2024-04-17T09:57:13.178957+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:11:38.218856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280416+0000","last_change":"2024-04-17T09:57:12.182080+0000","last_active":"2024-04-17T09:57:13.280416+0000","last_peered":"2024-04-17T09:57:13.280416+0000","last_clean":"2024-04-17T09:57:13.280416+0000","last_became_active":"2024-04-17T09:57:12.181786+0000","last_became_peered":"2024-04-17T09:57:12.181786+0000","last_unstale":"2024-04-17T09:57:13.280416+0000","last_undegraded":"2024-04-17T09:57:13.280416+0000","last_fullsized":"2024-04-17T09:57:13.280416+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:46.142617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.184805+0000","last_change":"2024-04-17T09:57:11.175061+0000","last_active":"2024-04-17T09:57:13.184805+0000","last_peered":"2024-04-17T09:57:13.184805+0000","last_clean":"2024-04-17T09:57:13.184805+0000","last_became_active":"2024-04-17T09:57:11.174687+0000","last_became_peered":"2024-04-17T09:57:11.174687+0000","last_unstale":"2024-04-17T09:57:13.184805+0000","last_undegraded":"2024-04-17T09:57:13.184805+0000","last_fullsized":"2024-04-17T09:57:13.184805+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:37.034263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280622+0000","last_change":"2024-04-17T09:57:12.184298+0000","last_active":"2024-04-17T09:57:13.280622+0000","last_peered":"2024-04-17T09:57:13.280622+0000","last_clean":"2024-04-17T09:57:13.280622+0000","last_became_active":"2024-04-17T09:57:12.181830+0000","last_became_peered":"2024-04-17T09:57:12.181830+0000","last_unstale":"2024-04-17T09:57:13.280622+0000","last_undegraded":"2024-04-17T09:57:13.280622+0000","last_fullsized":"2024-04-17T09:57:13.280622+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:44:19.484223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.189969+0000","last_change":"2024-04-17T09:57:11.177802+0000","last_active":"2024-04-17T09:57:13.189969+0000","last_peered":"2024-04-17T09:57:13.189969+0000","last_clean":"2024-04-17T09:57:13.189969+0000","last_became_active":"2024-04-17T09:57:11.175507+0000","last_became_peered":"2024-04-17T09:57:11.175507+0000","last_unstale":"2024-04-17T09:57:13.189969+0000","last_undegraded":"2024-04-17T09:57:13.189969+0000","last_fullsized":"2024-04-17T09:57:13.189969+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:24:15.812581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.190410+0000","last_change":"2024-04-17T09:57:12.189135+0000","last_active":"2024-04-17T09:57:13.190410+0000","last_peered":"2024-04-17T09:57:13.190410+0000","last_clean":"2024-04-17T09:57:13.190410+0000","last_became_active":"2024-04-17T09:57:12.188881+0000","last_became_peered":"2024-04-17T09:57:12.188881+0000","last_unstale":"2024-04-17T09:57:13.190410+0000","last_undegraded":"2024-04-17T09:57:13.190410+0000","last_fullsized":"2024-04-17T09:57:13.190410+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:07.597746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178877+0000","last_change":"2024-04-17T09:57:11.173965+0000","last_active":"2024-04-17T09:57:13.178877+0000","last_peered":"2024-04-17T09:57:13.178877+0000","last_clean":"2024-04-17T09:57:13.178877+0000","last_became_active":"2024-04-17T09:57:11.173657+0000","last_became_peered":"2024-04-17T09:57:11.173657+0000","last_unstale":"2024-04-17T09:57:13.178877+0000","last_undegraded":"2024-04-17T09:57:13.178877+0000","last_fullsized":"2024-04-17T09:57:13.178877+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:36:13.959638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.184765+0000","last_change":"2024-04-17T09:57:12.191094+0000","last_active":"2024-04-17T09:57:13.184765+0000","last_peered":"2024-04-17T09:57:13.184765+0000","last_clean":"2024-04-17T09:57:13.184765+0000","last_became_active":"2024-04-17T09:57:12.190774+0000","last_became_peered":"2024-04-17T09:57:12.190774+0000","last_unstale":"2024-04-17T09:57:13.184765+0000","last_undegraded":"2024-04-17T09:57:13.184765+0000","last_fullsized":"2024-04-17T09:57:13.184765+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:32:52.852878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"68'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.184852+0000","last_change":"2024-04-17T09:57:11.173169+0000","last_active":"2024-04-17T09:57:13.184852+0000","last_peered":"2024-04-17T09:57:13.184852+0000","last_clean":"2024-04-17T09:57:13.184852+0000","last_became_active":"2024-04-17T09:57:11.172785+0000","last_became_peered":"2024-04-17T09:57:11.172785+0000","last_unstale":"2024-04-17T09:57:13.184852+0000","last_undegraded":"2024-04-17T09:57:13.184852+0000","last_fullsized":"2024-04-17T09:57:13.184852+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:27:57.370537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280118+0000","last_change":"2024-04-17T09:57:12.194481+0000","last_active":"2024-04-17T09:57:13.280118+0000","last_peered":"2024-04-17T09:57:13.280118+0000","last_clean":"2024-04-17T09:57:13.280118+0000","last_became_active":"2024-04-17T09:57:12.194128+0000","last_became_peered":"2024-04-17T09:57:12.194128+0000","last_unstale":"2024-04-17T09:57:13.280118+0000","last_undegraded":"2024-04-17T09:57:13.280118+0000","last_fullsized":"2024-04-17T09:57:13.280118+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:31.153383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280820+0000","last_change":"2024-04-17T09:57:11.175574+0000","last_active":"2024-04-17T09:57:13.280820+0000","last_peered":"2024-04-17T09:57:13.280820+0000","last_clean":"2024-04-17T09:57:13.280820+0000","last_became_active":"2024-04-17T09:57:11.175285+0000","last_became_peered":"2024-04-17T09:57:11.175285+0000","last_unstale":"2024-04-17T09:57:13.280820+0000","last_undegraded":"2024-04-17T09:57:13.280820+0000","last_fullsized":"2024-04-17T09:57:13.280820+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:49:03.521739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280390+0000","last_change":"2024-04-17T09:57:12.195374+0000","last_active":"2024-04-17T09:57:13.280390+0000","last_peered":"2024-04-17T09:57:13.280390+0000","last_clean":"2024-04-17T09:57:13.280390+0000","last_became_active":"2024-04-17T09:57:12.195122+0000","last_became_peered":"2024-04-17T09:57:12.195122+0000","last_unstale":"2024-04-17T09:57:13.280390+0000","last_undegraded":"2024-04-17T09:57:13.280390+0000","last_fullsized":"2024-04-17T09:57:13.280390+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:41:15.457475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.190371+0000","last_change":"2024-04-17T09:57:11.161353+0000","last_active":"2024-04-17T09:57:13.190371+0000","last_peered":"2024-04-17T09:57:13.190371+0000","last_clean":"2024-04-17T09:57:13.190371+0000","last_became_active":"2024-04-17T09:57:11.161100+0000","last_became_peered":"2024-04-17T09:57:11.161100+0000","last_unstale":"2024-04-17T09:57:13.190371+0000","last_undegraded":"2024-04-17T09:57:13.190371+0000","last_fullsized":"2024-04-17T09:57:13.190371+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:28:40.233463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.281146+0000","last_change":"2024-04-17T09:57:12.191880+0000","last_active":"2024-04-17T09:57:13.281146+0000","last_peered":"2024-04-17T09:57:13.281146+0000","last_clean":"2024-04-17T09:57:13.281146+0000","last_became_active":"2024-04-17T09:57:12.191130+0000","last_became_peered":"2024-04-17T09:57:12.191130+0000","last_unstale":"2024-04-17T09:57:13.281146+0000","last_undegraded":"2024-04-17T09:57:13.281146+0000","last_fullsized":"2024-04-17T09:57:13.281146+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:17:05.804391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.280266+0000","last_change":"2024-04-17T09:57:11.172495+0000","last_active":"2024-04-17T09:57:13.280266+0000","last_peered":"2024-04-17T09:57:13.280266+0000","last_clean":"2024-04-17T09:57:13.280266+0000","last_became_active":"2024-04-17T09:57:11.172113+0000","last_became_peered":"2024-04-17T09:57:11.172113+0000","last_unstale":"2024-04-17T09:57:13.280266+0000","last_undegraded":"2024-04-17T09:57:13.280266+0000","last_fullsized":"2024-04-17T09:57:13.280266+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:54:44.975970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178849+0000","last_change":"2024-04-17T09:57:12.181471+0000","last_active":"2024-04-17T09:57:13.178849+0000","last_peered":"2024-04-17T09:57:13.178849+0000","last_clean":"2024-04-17T09:57:13.178849+0000","last_became_active":"2024-04-17T09:57:12.181165+0000","last_became_peered":"2024-04-17T09:57:12.181165+0000","last_unstale":"2024-04-17T09:57:13.178849+0000","last_undegraded":"2024-04-17T09:57:13.178849+0000","last_fullsized":"2024-04-17T09:57:13.178849+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:15.170971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.177750+0000","last_change":"2024-04-17T09:57:11.161883+0000","last_active":"2024-04-17T09:57:13.177750+0000","last_peered":"2024-04-17T09:57:13.177750+0000","last_clean":"2024-04-17T09:57:13.177750+0000","last_became_active":"2024-04-17T09:57:11.161601+0000","last_became_peered":"2024-04-17T09:57:11.161601+0000","last_unstale":"2024-04-17T09:57:13.177750+0000","last_undegraded":"2024-04-17T09:57:13.177750+0000","last_fullsized":"2024-04-17T09:57:13.177750+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:11:10.265150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178091+0000","last_change":"2024-04-17T09:57:12.178615+0000","last_active":"2024-04-17T09:57:13.178091+0000","last_peered":"2024-04-17T09:57:13.178091+0000","last_clean":"2024-04-17T09:57:13.178091+0000","last_became_active":"2024-04-17T09:57:12.177618+0000","last_became_peered":"2024-04-17T09:57:12.177618+0000","last_unstale":"2024-04-17T09:57:13.178091+0000","last_undegraded":"2024-04-17T09:57:13.178091+0000","last_fullsized":"2024-04-17T09:57:13.178091+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:12:37.383487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.177796+0000","last_change":"2024-04-17T09:57:11.167633+0000","last_active":"2024-04-17T09:57:13.177796+0000","last_peered":"2024-04-17T09:57:13.177796+0000","last_clean":"2024-04-17T09:57:13.177796+0000","last_became_active":"2024-04-17T09:57:11.167370+0000","last_became_peered":"2024-04-17T09:57:11.167370+0000","last_unstale":"2024-04-17T09:57:13.177796+0000","last_undegraded":"2024-04-17T09:57:13.177796+0000","last_fullsized":"2024-04-17T09:57:13.177796+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:49:40.199728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.177916+0000","last_change":"2024-04-17T09:57:12.185559+0000","last_active":"2024-04-17T09:57:13.177916+0000","last_peered":"2024-04-17T09:57:13.177916+0000","last_clean":"2024-04-17T09:57:13.177916+0000","last_became_active":"2024-04-17T09:57:12.185287+0000","last_became_peered":"2024-04-17T09:57:12.185287+0000","last_unstale":"2024-04-17T09:57:13.177916+0000","last_undegraded":"2024-04-17T09:57:13.177916+0000","last_fullsized":"2024-04-17T09:57:13.177916+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:38.042986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178568+0000","last_change":"2024-04-17T09:57:11.169107+0000","last_active":"2024-04-17T09:57:13.178568+0000","last_peered":"2024-04-17T09:57:13.178568+0000","last_clean":"2024-04-17T09:57:13.178568+0000","last_became_active":"2024-04-17T09:57:11.168808+0000","last_became_peered":"2024-04-17T09:57:11.168808+0000","last_unstale":"2024-04-17T09:57:13.178568+0000","last_undegraded":"2024-04-17T09:57:13.178568+0000","last_fullsized":"2024-04-17T09:57:13.178568+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:12:36.374765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.281028+0000","last_change":"2024-04-17T09:57:12.180324+0000","last_active":"2024-04-17T09:57:13.281028+0000","last_peered":"2024-04-17T09:57:13.281028+0000","last_clean":"2024-04-17T09:57:13.281028+0000","last_became_active":"2024-04-17T09:57:12.180048+0000","last_became_peered":"2024-04-17T09:57:12.180048+0000","last_unstale":"2024-04-17T09:57:13.281028+0000","last_undegraded":"2024-04-17T09:57:13.281028+0000","last_fullsized":"2024-04-17T09:57:13.281028+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:11.144223+0000","last_clean_scrub_stamp":"2024-04-17T09:57:11.144223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:01:25.562824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.178917+0000","last_change":"2024-04-17T09:57:11.170068+0000","last_active":"2024-04-17T09:57:13.178917+0000","last_peered":"2024-04-17T09:57:13.178917+0000","last_clean":"2024-04-17T09:57:13.178917+0000","last_became_active":"2024-04-17T09:57:11.168615+0000","last_became_peered":"2024-04-17T09:57:11.168615+0000","last_unstale":"2024-04-17T09:57:13.178917+0000","last_undegraded":"2024-04-17T09:57:13.178917+0000","last_fullsized":"2024-04-17T09:57:13.178917+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:57:10.135501+0000","last_clean_scrub_stamp":"2024-04-17T09:57:10.135501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:17:04.795668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":257,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-17T09:57:13.279765+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T09:57:13.279765+0000","last_peered":"2024-04-17T09:57:13.279765+0000","last_clean":"2024-04-17T09:57:13.279765+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T09:57:13.279765+0000","last_undegraded":"2024-04-17T09:57:13.279765+0000","last_fullsized":"2024-04-17T09:57:13.279765+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509494,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1421312,"data_stored":933744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509495,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":827392,"data_stored":342852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509494,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":823296,"data_stored":342850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509494,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":342872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509495,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":831488,"data_stored":344372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509494,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":839680,"data_stored":344988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542198,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":342830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1413120,"data_stored":933128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T09:57:16.702 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T09:57:16.702 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-17T09:57:16.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.887+0000 7f521d400640 1 Processor -- start 2024-04-17T09:57:16.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.887+0000 7f521d400640 1 -- start start 2024-04-17T09:57:16.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.891+0000 7f521d400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5218144e90 0x7f521806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:16.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.891+0000 7f521d400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f521806f4a0 0x7f52180cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:16.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.891+0000 7f521d400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f52180cfc20 0x7f521815f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:16.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.891+0000 7f521d400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5218145270 con 0x7f5218144e90 2024-04-17T09:57:16.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.891+0000 7f521d400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f5218071e90 con 0x7f52180cfc20 2024-04-17T09:57:16.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.891+0000 7f521d400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5218072170 con 0x7f521806f4a0 2024-04-17T09:57:16.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.891+0000 7f5217e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5218144e90 0x7f521806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:16.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.891+0000 7f5217e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5218144e90 0x7f521806ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59742/0 (socket says 172.21.15.32:59742) 2024-04-17T09:57:16.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.891+0000 7f5217e00640 1 -- 172.21.15.32:0/80305226 learned_addr learned my addr 172.21.15.32:0/80305226 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:16.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.891+0000 7f520f400640 1 --2- 172.21.15.32:0/80305226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f521806f4a0 0x7f52180cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:16.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.891+0000 7f521ca00640 1 --2- 172.21.15.32:0/80305226 >> v2:172.21.15.144:3300/0 conn(0x7f52180cfc20 0x7f521815f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:16.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.891+0000 7f5217e00640 1 -- 172.21.15.32:0/80305226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f521806f4a0 msgr2=0x7f52180cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:16.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.891+0000 7f5217e00640 1 --2- 172.21.15.32:0/80305226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f521806f4a0 0x7f52180cf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.891+0000 7f5217e00640 1 -- 172.21.15.32:0/80305226 >> v2:172.21.15.144:3300/0 conn(0x7f52180cfc20 msgr2=0x7f521815f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:16.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.891+0000 7f5217e00640 1 --2- 172.21.15.32:0/80305226 >> v2:172.21.15.144:3300/0 conn(0x7f52180cfc20 0x7f521815f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.891+0000 7f5217e00640 1 -- 172.21.15.32:0/80305226 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5218072450 con 0x7f5218144e90 2024-04-17T09:57:16.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.895+0000 7f5217e00640 1 --2- 172.21.15.32:0/80305226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5218144e90 0x7f521806ef60 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f521806fbd0 tx=0x7f520801c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61c38ea154c0039 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:16.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.895+0000 7f5217400640 1 -- 172.21.15.32:0/80305226 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52080b0070 con 0x7f5218144e90 2024-04-17T09:57:16.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.895+0000 7f5217400640 1 -- 172.21.15.32:0/80305226 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52080ab600 con 0x7f5218144e90 2024-04-17T09:57:16.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.895+0000 7f5217400640 1 -- 172.21.15.32:0/80305226 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52080ab9d0 con 0x7f5218144e90 2024-04-17T09:57:16.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.895+0000 7f521d400640 1 -- 172.21.15.32:0/80305226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5218144e90 msgr2=0x7f521806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:16.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.895+0000 7f521d400640 1 --2- 172.21.15.32:0/80305226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5218144e90 0x7f521806ef60 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f521806fbd0 tx=0x7f520801c140 comp rx=0 tx=0).stop 2024-04-17T09:57:16.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.895+0000 7f521d400640 1 -- 172.21.15.32:0/80305226 shutdown_connections 2024-04-17T09:57:16.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.895+0000 7f521d400640 1 --2- 172.21.15.32:0/80305226 >> v2:172.21.15.144:3300/0 conn(0x7f52180cfc20 0x7f521815f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.895+0000 7f521d400640 1 --2- 172.21.15.32:0/80305226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f521806f4a0 0x7f52180cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.895+0000 7f521d400640 1 --2- 172.21.15.32:0/80305226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5218144e90 0x7f521806ef60 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.895+0000 7f521d400640 1 -- 172.21.15.32:0/80305226 >> 172.21.15.32:0/80305226 conn(0x7f5218067d70 msgr2=0x7f521806ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:16.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.895+0000 7f521d400640 1 -- 172.21.15.32:0/80305226 shutdown_connections 2024-04-17T09:57:16.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.895+0000 7f521d400640 1 -- 172.21.15.32:0/80305226 wait complete. 2024-04-17T09:57:16.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.895+0000 7f521d400640 1 Processor -- start 2024-04-17T09:57:16.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.895+0000 7f521d400640 1 -- start start 2024-04-17T09:57:16.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.895+0000 7f521d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f521806f4a0 0x7f521815cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:16.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.895+0000 7f521d400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52180cfc20 0x7f52181439b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:16.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.895+0000 7f521d400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5218144e90 0x7f52180b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.895+0000 7f521d400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f521806de50 con 0x7f521806f4a0 2024-04-17T09:57:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.895+0000 7f521d400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f52180d0240 con 0x7f52180cfc20 2024-04-17T09:57:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.895+0000 7f521d400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f521806dfd0 con 0x7f5218144e90 2024-04-17T09:57:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.895+0000 7f5217e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f521806f4a0 0x7f521815cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.899+0000 7f5217e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f521806f4a0 0x7f521815cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59756/0 (socket says 172.21.15.32:59756) 2024-04-17T09:57:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.899+0000 7f5217e00640 1 -- 172.21.15.32:0/2883653441 learned_addr learned my addr 172.21.15.32:0/2883653441 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.899+0000 7f520f400640 1 --2- 172.21.15.32:0/2883653441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52180cfc20 0x7f52181439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.899+0000 7f521ca00640 1 --2- 172.21.15.32:0/2883653441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5218144e90 0x7f52180b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.899+0000 7f5217e00640 1 -- 172.21.15.32:0/2883653441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5218144e90 msgr2=0x7f52180b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.899+0000 7f5217e00640 1 --2- 172.21.15.32:0/2883653441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5218144e90 0x7f52180b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.899+0000 7f5217e00640 1 -- 172.21.15.32:0/2883653441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52180cfc20 msgr2=0x7f52181439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.899+0000 7f5217e00640 1 --2- 172.21.15.32:0/2883653441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52180cfc20 0x7f52181439b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:16.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.899+0000 7f5217e00640 1 -- 172.21.15.32:0/2883653441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5218143ef0 con 0x7f521806f4a0 2024-04-17T09:57:16.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.899+0000 7f520f400640 1 --2- 172.21.15.32:0/2883653441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52180cfc20 0x7f52181439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:57:16.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.899+0000 7f5217e00640 1 --2- 172.21.15.32:0/2883653441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f521806f4a0 0x7f521815cca0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f52080ae040 tx=0x7f5208018ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:16.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.899+0000 7f5214c00640 1 -- 172.21.15.32:0/2883653441 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52080b0040 con 0x7f521806f4a0 2024-04-17T09:57:16.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.899+0000 7f5214c00640 1 -- 172.21.15.32:0/2883653441 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52080b9070 con 0x7f521806f4a0 2024-04-17T09:57:16.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.899+0000 7f5214c00640 1 -- 172.21.15.32:0/2883653441 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52080b46a0 con 0x7f521806f4a0 2024-04-17T09:57:16.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.899+0000 7f521d400640 1 -- 172.21.15.32:0/2883653441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f521815bc00 con 0x7f521806f4a0 2024-04-17T09:57:16.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.899+0000 7f521d400640 1 -- 172.21.15.32:0/2883653441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f521815c1c0 con 0x7f521806f4a0 2024-04-17T09:57:16.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.899+0000 7f521d400640 1 -- 172.21.15.32:0/2883653441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f521804fda0 con 0x7f521806f4a0 2024-04-17T09:57:16.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.899+0000 7f5214c00640 1 -- 172.21.15.32:0/2883653441 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f52080b4840 con 0x7f521806f4a0 2024-04-17T09:57:16.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.903+0000 7f5214c00640 1 --2- 172.21.15.32:0/2883653441 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f51f40683b0 0x7f51f406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:16.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.903+0000 7f5214c00640 1 -- 172.21.15.32:0/2883653441 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f5208120750 con 0x7f521806f4a0 2024-04-17T09:57:16.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.903+0000 7f520f400640 1 --2- 172.21.15.32:0/2883653441 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f51f40683b0 0x7f51f406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:16.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.903+0000 7f520f400640 1 --2- 172.21.15.32:0/2883653441 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f51f40683b0 0x7f51f406a870 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5200004660 tx=0x7f5200008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:16.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:16.907+0000 7f5214c00640 1 -- 172.21.15.32:0/2883653441 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5208016110 con 0x7f521806f4a0 2024-04-17T09:57:17.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.131+0000 7f521d400640 1 -- 172.21.15.32:0/2883653441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f52180ccf80 con 0x7f521806f4a0 2024-04-17T09:57:17.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.131+0000 7f5214c00640 1 -- 172.21.15.32:0/2883653441 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7f52080f0c70 con 0x7f521806f4a0 2024-04-17T09:57:17.135 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-17T09:57:17.135 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDPmx9mUXoLKxAANV7zhLvL0DqHngk4TtsdBQ== 2024-04-17T09:57:17.135 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2024-04-17T09:57:17.135 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2024-04-17T09:57:17.135 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-17T09:57:17.135 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2024-04-17T09:57:17.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.135+0000 7f521d400640 1 -- 172.21.15.32:0/2883653441 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f51f40683b0 msgr2=0x7f51f406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:17.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.135+0000 7f521d400640 1 --2- 172.21.15.32:0/2883653441 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f51f40683b0 0x7f51f406a870 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5200004660 tx=0x7f5200008040 comp rx=0 tx=0).stop 2024-04-17T09:57:17.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.135+0000 7f521d400640 1 -- 172.21.15.32:0/2883653441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f521806f4a0 msgr2=0x7f521815cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:17.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.135+0000 7f521d400640 1 --2- 172.21.15.32:0/2883653441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f521806f4a0 0x7f521815cca0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f52080ae040 tx=0x7f5208018ee0 comp rx=0 tx=0).stop 2024-04-17T09:57:17.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.135+0000 7f521d400640 1 -- 172.21.15.32:0/2883653441 shutdown_connections 2024-04-17T09:57:17.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.135+0000 7f521d400640 1 --2- 172.21.15.32:0/2883653441 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f51f40683b0 0x7f51f406a870 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:17.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.135+0000 7f521d400640 1 --2- 172.21.15.32:0/2883653441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5218144e90 0x7f52180b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:17.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.135+0000 7f521d400640 1 --2- 172.21.15.32:0/2883653441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52180cfc20 0x7f52181439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:17.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.135+0000 7f521d400640 1 --2- 172.21.15.32:0/2883653441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f521806f4a0 0x7f521815cca0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:17.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.135+0000 7f521d400640 1 -- 172.21.15.32:0/2883653441 >> 172.21.15.32:0/2883653441 conn(0x7f5218067d70 msgr2=0x7f52180b4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:17.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.135+0000 7f521d400640 1 -- 172.21.15.32:0/2883653441 shutdown_connections 2024-04-17T09:57:17.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.135+0000 7f521d400640 1 -- 172.21.15.32:0/2883653441 wait complete. 2024-04-17T09:57:17.157 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:57:17.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.295+0000 7f8a13800640 1 Processor -- start 2024-04-17T09:57:17.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.295+0000 7f8a13800640 1 -- start start 2024-04-17T09:57:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.299+0000 7f8a13800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8a0c06f7c0 0x7f8a0c06fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.299+0000 7f8a13800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a0c15ec50 con 0x7f8a0c0b0320 2024-04-17T09:57:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.299+0000 7f8a13800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a0c0d0950 con 0x7f8a0c15e870 2024-04-17T09:57:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.299+0000 7f8a13800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f8a0c0cf810 con 0x7f8a0c06f7c0 2024-04-17T09:57:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.299+0000 7f8a03e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8a0c0b0320 0x7f8a0c0d0240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41642/0 (socket says 172.21.15.32:41642) 2024-04-17T09:57:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.299+0000 7f8a03e00640 1 -- 172.21.15.32:0/784334037 learned_addr learned my addr 172.21.15.32:0/784334037 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.299+0000 7f8a11800640 1 --2- 172.21.15.32:0/784334037 >> v2:172.21.15.144:3300/0 conn(0x7f8a0c06f7c0 0x7f8a0c06fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.299+0000 7f8a0be00640 1 -- 172.21.15.32:0/784334037 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2769494605 0 0) 0x7f8a0c15ec50 con 0x7f8a0c0b0320 2024-04-17T09:57:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.299+0000 7f8a0be00640 1 -- 172.21.15.32:0/784334037 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89f0003530 con 0x7f8a0c0b0320 2024-04-17T09:57:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.299+0000 7f8a11800640 1 -- 172.21.15.32:0/784334037 >> v1:172.21.15.144:6789/0 conn(0x7f8a0c15e870 legacy=0x7f8a0c162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.299+0000 7f8a11800640 1 -- 172.21.15.32:0/784334037 >> v1:172.21.15.32:6789/0 conn(0x7f8a0c0b0320 legacy=0x7f8a0c0d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.299+0000 7f8a11800640 1 -- 172.21.15.32:0/784334037 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0c067810 con 0x7f8a0c06f7c0 2024-04-17T09:57:17.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.303+0000 7f8a11800640 1 --2- 172.21.15.32:0/784334037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a0c06f7c0 0x7f8a0c06fc50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009d90 tx=0x7f8a0401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b8c46d6beceea3f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:17.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.303+0000 7f8a0be00640 1 -- 172.21.15.32:0/784334037 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a040b2070 con 0x7f8a0c06f7c0 2024-04-17T09:57:17.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.303+0000 7f8a0be00640 1 -- 172.21.15.32:0/784334037 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a040a5830 con 0x7f8a0c06f7c0 2024-04-17T09:57:17.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.303+0000 7f8a0be00640 1 -- 172.21.15.32:0/784334037 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a040a5c00 con 0x7f8a0c06f7c0 2024-04-17T09:57:17.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.303+0000 7f8a13800640 1 -- 172.21.15.32:0/784334037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a0c06f7c0 msgr2=0x7f8a0c06fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:17.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.303+0000 7f8a13800640 1 --2- 172.21.15.32:0/784334037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a0c06f7c0 0x7f8a0c06fc50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009d90 tx=0x7f8a0401c970 comp rx=0 tx=0).stop 2024-04-17T09:57:17.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.303+0000 7f8a13800640 1 -- 172.21.15.32:0/784334037 shutdown_connections 2024-04-17T09:57:17.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.303+0000 7f8a13800640 1 --2- 172.21.15.32:0/784334037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a0c06f7c0 0x7f8a0c06fc50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:17.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.303+0000 7f8a13800640 1 -- 172.21.15.32:0/784334037 >> 172.21.15.32:0/784334037 conn(0x7f8a0c0d0c00 msgr2=0x7f8a0c0d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:17.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.303+0000 7f8a13800640 1 -- 172.21.15.32:0/784334037 shutdown_connections 2024-04-17T09:57:17.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.303+0000 7f8a13800640 1 -- 172.21.15.32:0/784334037 wait complete. 2024-04-17T09:57:17.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.303+0000 7f8a13800640 1 Processor -- start 2024-04-17T09:57:17.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.303+0000 7f8a13800640 1 -- start start 2024-04-17T09:57:17.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a13800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a0c06f7c0 0x7f8a0c14a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:17.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a13800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a0c0b0320 0x7f8a0c14fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:17.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a13800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a0c15e870 0x7f8a0c14d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:17.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a13800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c16c9b0 con 0x7f8a0c0b0320 2024-04-17T09:57:17.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a13800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c0cfdb0 con 0x7f8a0c15e870 2024-04-17T09:57:17.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a13800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0c16cb30 con 0x7f8a0c06f7c0 2024-04-17T09:57:17.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a03e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a0c0b0320 0x7f8a0c14fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:17.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a03e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a0c0b0320 0x7f8a0c14fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59770/0 (socket says 172.21.15.32:59770) 2024-04-17T09:57:17.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a03e00640 1 -- 172.21.15.32:0/75783933 learned_addr learned my addr 172.21.15.32:0/75783933 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:17.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a10e00640 1 --2- 172.21.15.32:0/75783933 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a0c06f7c0 0x7f8a0c14a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:17.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a11800640 1 --2- 172.21.15.32:0/75783933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a0c15e870 0x7f8a0c14d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:17.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a03e00640 1 -- 172.21.15.32:0/75783933 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a0c06f7c0 msgr2=0x7f8a0c14a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:17.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a03e00640 1 --2- 172.21.15.32:0/75783933 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a0c06f7c0 0x7f8a0c14a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:17.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a03e00640 1 -- 172.21.15.32:0/75783933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a0c15e870 msgr2=0x7f8a0c14d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:17.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a03e00640 1 --2- 172.21.15.32:0/75783933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a0c15e870 0x7f8a0c14d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:17.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a03e00640 1 -- 172.21.15.32:0/75783933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0c15aa40 con 0x7f8a0c0b0320 2024-04-17T09:57:17.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a11800640 1 --2- 172.21.15.32:0/75783933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a0c15e870 0x7f8a0c14d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:57:17.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a03e00640 1 --2- 172.21.15.32:0/75783933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a0c0b0320 0x7f8a0c14fae0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f89f400c480 tx=0x7f89f400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:17.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a09600640 1 -- 172.21.15.32:0/75783933 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4017070 con 0x7f8a0c0b0320 2024-04-17T09:57:17.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a09600640 1 -- 172.21.15.32:0/75783933 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89f4005cf0 con 0x7f8a0c0b0320 2024-04-17T09:57:17.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a13800640 1 -- 172.21.15.32:0/75783933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0c15ad20 con 0x7f8a0c0b0320 2024-04-17T09:57:17.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a09600640 1 -- 172.21.15.32:0/75783933 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4007290 con 0x7f8a0c0b0320 2024-04-17T09:57:17.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a13800640 1 -- 172.21.15.32:0/75783933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0c15b2e0 con 0x7f8a0c0b0320 2024-04-17T09:57:17.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a13800640 1 -- 172.21.15.32:0/75783933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a0c068a90 con 0x7f8a0c0b0320 2024-04-17T09:57:17.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.307+0000 7f8a09600640 1 -- 172.21.15.32:0/75783933 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f89f401a070 con 0x7f8a0c0b0320 2024-04-17T09:57:17.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.311+0000 7f8a09600640 1 --2- 172.21.15.32:0/75783933 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f89e8068380 0x7f89e806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:17.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.311+0000 7f8a09600640 1 -- 172.21.15.32:0/75783933 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f89f4085d30 con 0x7f8a0c0b0320 2024-04-17T09:57:17.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.311+0000 7f8a10e00640 1 --2- 172.21.15.32:0/75783933 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f89e8068380 0x7f89e806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:17.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.311+0000 7f8a10e00640 1 --2- 172.21.15.32:0/75783933 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f89e8068380 0x7f89e806a840 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f89fc001770 tx=0x7f89fc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:17.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.315+0000 7f8a09600640 1 -- 172.21.15.32:0/75783933 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f89f4055c90 con 0x7f8a0c0b0320 2024-04-17T09:57:17.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.515+0000 7f8a13800640 1 -- 172.21.15.32:0/75783933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a0c0693a0 con 0x7f8a0c0b0320 2024-04-17T09:57:17.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.515+0000 7f8a09600640 1 -- 172.21.15.32:0/75783933 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5201 (secure 0 0 0) 0x7f89f4059f90 con 0x7f8a0c0b0320 2024-04-17T09:57:17.517 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:57:17.517 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2424282223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2424282223},{"type":"v1","addr":"172.21.15.144:6834","nonce":2424282223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3537056692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3537056692},{"type":"v1","addr":"172.21.15.144:6835","nonce":3537056692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5891,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/725333724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":725333724},{"type":"v1","addr":"172.21.15.32:6837","nonce":725333724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:57:11.558183+0000","modified":"2024-04-17T09:57:14.573355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6786},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6786":{"gid":6786,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6836/663537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":663537244},{"type":"v1","addr":"172.21.15.32:6836","nonce":663537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6786,"qdb_cluster":[6786]},"id":5}]} 2024-04-17T09:57:17.517 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 42 2024-04-17T09:57:17.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.515+0000 7f8a13800640 1 -- 172.21.15.32:0/75783933 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f89e8068380 msgr2=0x7f89e806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:17.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.515+0000 7f8a13800640 1 --2- 172.21.15.32:0/75783933 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f89e8068380 0x7f89e806a840 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f89fc001770 tx=0x7f89fc009040 comp rx=0 tx=0).stop 2024-04-17T09:57:17.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.515+0000 7f8a13800640 1 -- 172.21.15.32:0/75783933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a0c0b0320 msgr2=0x7f8a0c14fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:17.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.515+0000 7f8a13800640 1 --2- 172.21.15.32:0/75783933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a0c0b0320 0x7f8a0c14fae0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f89f400c480 tx=0x7f89f400c940 comp rx=0 tx=0).stop 2024-04-17T09:57:17.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.519+0000 7f8a13800640 1 -- 172.21.15.32:0/75783933 shutdown_connections 2024-04-17T09:57:17.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.519+0000 7f8a13800640 1 --2- 172.21.15.32:0/75783933 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f89e8068380 0x7f89e806a840 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:17.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.519+0000 7f8a13800640 1 --2- 172.21.15.32:0/75783933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a0c15e870 0x7f8a0c14d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:17.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.519+0000 7f8a13800640 1 --2- 172.21.15.32:0/75783933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a0c0b0320 0x7f8a0c14fae0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:17.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.519+0000 7f8a13800640 1 --2- 172.21.15.32:0/75783933 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a0c06f7c0 0x7f8a0c14a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:17.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.519+0000 7f8a13800640 1 -- 172.21.15.32:0/75783933 >> 172.21.15.32:0/75783933 conn(0x7f8a0c0d0c00 msgr2=0x7f8a0c156700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:17.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.519+0000 7f8a13800640 1 -- 172.21.15.32:0/75783933 shutdown_connections 2024-04-17T09:57:17.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:17.519+0000 7f8a13800640 1 -- 172.21.15.32:0/75783933 wait complete. 2024-04-17T09:57:17.941 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T09:57:11.558183+0000', 'modified': '2024-04-17T09:57:14.573355+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6786}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6786': {'gid': 6786, 'name': 'a', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6836/663537244', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 663537244}, {'type': 'v1', 'addr': '172.21.15.32:6836', 'nonce': 663537244}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6786, 'qdb_cluster': [6786]} 2024-04-17T09:57:17.941 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T09:57:17.941 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-17T09:57:18.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.087+0000 7f68e2000640 1 Processor -- start 2024-04-17T09:57:18.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.087+0000 7f68e2000640 1 -- start start 2024-04-17T09:57:18.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.087+0000 7f68e2000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f68dc155a70 0x7f68dc15f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.087+0000 7f68e2000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68dc0b3340 0x7f68dc15c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.087+0000 7f68e2000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68dc15f870 con 0x7f68dc155e50 2024-04-17T09:57:18.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.087+0000 7f68e2000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f68dc0b42a0 con 0x7f68dc0b3340 2024-04-17T09:57:18.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.087+0000 7f68e2000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f68dc0d1250 con 0x7f68dc155a70 2024-04-17T09:57:18.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.087+0000 7f68dbe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68dc0b3340 0x7f68dc15c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.087+0000 7f68dbe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68dc0b3340 0x7f68dc15c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59786/0 (socket says 172.21.15.32:59786) 2024-04-17T09:57:18.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.087+0000 7f68dbe00640 1 -- 172.21.15.32:0/2527516729 learned_addr learned my addr 172.21.15.32:0/2527516729 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:18.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.087+0000 7f68db400640 1 --2- 172.21.15.32:0/2527516729 >> v2:172.21.15.144:3300/0 conn(0x7f68dc155a70 0x7f68dc15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.087+0000 7f68dbe00640 1 -- 172.21.15.32:0/2527516729 >> v2:172.21.15.144:3300/0 conn(0x7f68dc155a70 msgr2=0x7f68dc15f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:18.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.087+0000 7f68dbe00640 1 --2- 172.21.15.32:0/2527516729 >> v2:172.21.15.144:3300/0 conn(0x7f68dc155a70 0x7f68dc15f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.087+0000 7f68dbe00640 1 -- 172.21.15.32:0/2527516729 >> v1:172.21.15.32:6789/0 conn(0x7f68dc155e50 legacy=0x7f68dc15d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:18.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.087+0000 7f68dbe00640 1 -- 172.21.15.32:0/2527516729 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68dc15fb50 con 0x7f68dc0b3340 2024-04-17T09:57:18.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.087+0000 7f68db400640 1 --2- 172.21.15.32:0/2527516729 >> v2:172.21.15.144:3300/0 conn(0x7f68dc155a70 0x7f68dc15f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:57:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.091+0000 7f68dbe00640 1 --2- 172.21.15.32:0/2527516729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68dc0b3340 0x7f68dc15c260 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009e50 tx=0x7f68d401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179b7740e0f715a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.091+0000 7f68daa00640 1 -- 172.21.15.32:0/2527516729 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d40a56c0 con 0x7f68dc0b3340 2024-04-17T09:57:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.091+0000 7f68daa00640 1 -- 172.21.15.32:0/2527516729 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68d40a5860 con 0x7f68dc0b3340 2024-04-17T09:57:18.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.091+0000 7f68daa00640 1 -- 172.21.15.32:0/2527516729 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d40a5c30 con 0x7f68dc0b3340 2024-04-17T09:57:18.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.091+0000 7f68e2000640 1 -- 172.21.15.32:0/2527516729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68dc0b3340 msgr2=0x7f68dc15c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:18.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.091+0000 7f68e2000640 1 --2- 172.21.15.32:0/2527516729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68dc0b3340 0x7f68dc15c260 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009e50 tx=0x7f68d401c9f0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.091+0000 7f68e2000640 1 -- 172.21.15.32:0/2527516729 shutdown_connections 2024-04-17T09:57:18.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.091+0000 7f68e2000640 1 --2- 172.21.15.32:0/2527516729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68dc0b3340 0x7f68dc15c260 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.091+0000 7f68e2000640 1 --2- 172.21.15.32:0/2527516729 >> v2:172.21.15.144:3300/0 conn(0x7f68dc155a70 0x7f68dc15f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.091+0000 7f68e2000640 1 -- 172.21.15.32:0/2527516729 >> 172.21.15.32:0/2527516729 conn(0x7f68dc06ba30 msgr2=0x7f68dc06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:18.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.091+0000 7f68e2000640 1 -- 172.21.15.32:0/2527516729 shutdown_connections 2024-04-17T09:57:18.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.091+0000 7f68e2000640 1 -- 172.21.15.32:0/2527516729 wait complete. 2024-04-17T09:57:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.091+0000 7f68e2000640 1 Processor -- start 2024-04-17T09:57:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.095+0000 7f68e2000640 1 -- start start 2024-04-17T09:57:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.095+0000 7f68e2000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68dc0b3340 0x7f68dc0cef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.095+0000 7f68e2000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68dc155a70 0x7f68dc15aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.095+0000 7f68e2000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68dc155e50 0x7f68dc153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.095+0000 7f68e2000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc0d02f0 con 0x7f68dc155e50 2024-04-17T09:57:18.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.095+0000 7f68e2000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc0d0170 con 0x7f68dc155a70 2024-04-17T09:57:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.095+0000 7f68e2000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f68dc0d0470 con 0x7f68dc0b3340 2024-04-17T09:57:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.095+0000 7f68dbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68dc155e50 0x7f68dc153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.095+0000 7f68dbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68dc155e50 0x7f68dc153280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59798/0 (socket says 172.21.15.32:59798) 2024-04-17T09:57:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.095+0000 7f68dbe00640 1 -- 172.21.15.32:0/2367741421 learned_addr learned my addr 172.21.15.32:0/2367741421 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.095+0000 7f68d3e00640 1 --2- 172.21.15.32:0/2367741421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68dc155a70 0x7f68dc15aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.095+0000 7f68db400640 1 --2- 172.21.15.32:0/2367741421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68dc0b3340 0x7f68dc0cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.095+0000 7f68dbe00640 1 -- 172.21.15.32:0/2367741421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68dc0b3340 msgr2=0x7f68dc0cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.095+0000 7f68dbe00640 1 --2- 172.21.15.32:0/2367741421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68dc0b3340 0x7f68dc0cef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.095+0000 7f68dbe00640 1 -- 172.21.15.32:0/2367741421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68dc155a70 msgr2=0x7f68dc15aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.095+0000 7f68dbe00640 1 --2- 172.21.15.32:0/2367741421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68dc155a70 0x7f68dc15aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.095+0000 7f68dbe00640 1 -- 172.21.15.32:0/2367741421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68dc0cfea0 con 0x7f68dc155e50 2024-04-17T09:57:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.095+0000 7f68dbe00640 1 --2- 172.21.15.32:0/2367741421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68dc155e50 0x7f68dc153280 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f68d4019600 tx=0x7f68d4019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.095+0000 7f68d3400640 1 -- 172.21.15.32:0/2367741421 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d40b8670 con 0x7f68dc155e50 2024-04-17T09:57:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.095+0000 7f68d3400640 1 -- 172.21.15.32:0/2367741421 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68d40be070 con 0x7f68dc155e50 2024-04-17T09:57:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.095+0000 7f68e2000640 1 -- 172.21.15.32:0/2367741421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68dc0cf600 con 0x7f68dc155e50 2024-04-17T09:57:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.095+0000 7f68d3400640 1 -- 172.21.15.32:0/2367741421 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d40b8a50 con 0x7f68dc155e50 2024-04-17T09:57:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.095+0000 7f68e2000640 1 -- 172.21.15.32:0/2367741421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68dc0cfbc0 con 0x7f68dc155e50 2024-04-17T09:57:18.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.099+0000 7f68d3400640 1 -- 172.21.15.32:0/2367741421 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f68d40b8bf0 con 0x7f68dc155e50 2024-04-17T09:57:18.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.099+0000 7f68d3400640 1 --2- 172.21.15.32:0/2367741421 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f68b8068380 0x7f68b806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.099+0000 7f68d3400640 1 -- 172.21.15.32:0/2367741421 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f68d4121be0 con 0x7f68dc155e50 2024-04-17T09:57:18.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.099+0000 7f68db400640 1 --2- 172.21.15.32:0/2367741421 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f68b8068380 0x7f68b806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.099+0000 7f68e5550000 1 Processor -- start 2024-04-17T09:57:18.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.099+0000 7f68e5550000 1 -- start start 2024-04-17T09:57:18.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.099+0000 7f68db400640 1 --2- 172.21.15.32:0/2367741421 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f68b8068380 0x7f68b806a840 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00a9a0 tx=0x7f68cc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:18.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.099+0000 7f68e5550000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f945632b50 0x55f945632f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.099+0000 7f68e5550000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f945633470 0x55f9455e87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.099+0000 7f68e5550000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9455e8cf0 0x55f9455e90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.099+0000 7f68e5550000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f9454364d0 con 0x55f945632b50 2024-04-17T09:57:18.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.099+0000 7f68e5550000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f9453d0bf0 con 0x55f945633470 2024-04-17T09:57:18.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.099+0000 7f68e5550000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f945432490 con 0x55f9455e8cf0 2024-04-17T09:57:18.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.099+0000 7f68db400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f945632b50 0x55f945632f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.099+0000 7f68db400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f945632b50 0x55f945632f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59802/0 (socket says 172.21.15.32:59802) 2024-04-17T09:57:18.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.099+0000 7f68db400640 1 -- 172.21.15.32:0/1405222527 learned_addr learned my addr 172.21.15.32:0/1405222527 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:18.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.099+0000 7f68d3e00640 1 --2- 172.21.15.32:0/1405222527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f945633470 0x55f9455e87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.099+0000 7f68dbe00640 1 --2- 172.21.15.32:0/1405222527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9455e8cf0 0x55f9455e90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68db400640 1 -- 172.21.15.32:0/1405222527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9455e8cf0 msgr2=0x55f9455e90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:18.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68db400640 1 --2- 172.21.15.32:0/1405222527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9455e8cf0 0x55f9455e90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68db400640 1 -- 172.21.15.32:0/1405222527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f945633470 msgr2=0x55f9455e87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:18.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68db400640 1 --2- 172.21.15.32:0/1405222527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f945633470 0x55f9455e87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68db400640 1 -- 172.21.15.32:0/1405222527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9453bce80 con 0x55f945632b50 2024-04-17T09:57:18.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68db400640 1 --2- 172.21.15.32:0/1405222527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f945632b50 0x55f945632f30 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00fa50 tx=0x7f68cc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11f6027c14a0d08 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:18.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68cbe00640 1 -- 172.21.15.32:0/1405222527 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc01b050 con 0x55f945632b50 2024-04-17T09:57:18.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68cbe00640 1 -- 172.21.15.32:0/1405222527 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68cc025070 con 0x55f945632b50 2024-04-17T09:57:18.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68cbe00640 1 -- 172.21.15.32:0/1405222527 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc014de0 con 0x55f945632b50 2024-04-17T09:57:18.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68e5550000 1 -- 172.21.15.32:0/1405222527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f945632b50 msgr2=0x55f945632f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:18.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68e5550000 1 --2- 172.21.15.32:0/1405222527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f945632b50 0x55f945632f30 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00fa50 tx=0x7f68cc012000 comp rx=0 tx=0).stop 2024-04-17T09:57:18.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68e5550000 1 -- 172.21.15.32:0/1405222527 shutdown_connections 2024-04-17T09:57:18.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68e5550000 1 --2- 172.21.15.32:0/1405222527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9455e8cf0 0x55f9455e90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68e5550000 1 --2- 172.21.15.32:0/1405222527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f945633470 0x55f9455e87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68e5550000 1 --2- 172.21.15.32:0/1405222527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f945632b50 0x55f945632f30 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68e5550000 1 -- 172.21.15.32:0/1405222527 >> 172.21.15.32:0/1405222527 conn(0x55f9455ee470 msgr2=0x55f9455e7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:18.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68e5550000 1 -- 172.21.15.32:0/1405222527 shutdown_connections 2024-04-17T09:57:18.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68e5550000 1 -- 172.21.15.32:0/1405222527 wait complete. 2024-04-17T09:57:18.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68e5550000 1 -- start start 2024-04-17T09:57:18.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68e5550000 1 Processor -- start 2024-04-17T09:57:18.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68e5550000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9455e8cf0 0x55f9455ed820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68e5550000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f945632b50 0x55f945620440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68e5550000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f945633470 0x55f945620980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68e5550000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f9453f3a60 con 0x55f9455e8cf0 2024-04-17T09:57:18.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68e5550000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f9453dae80 con 0x55f945632b50 2024-04-17T09:57:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68e5550000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f9453f1cf0 con 0x55f945633470 2024-04-17T09:57:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68e5550000 10 client.0 _ll_register_callbacks cb 0x55f9455d3a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:57:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68db400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9455e8cf0 0x55f9455ed820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68db400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9455e8cf0 0x55f9455ed820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59818/0 (socket says 172.21.15.32:59818) 2024-04-17T09:57:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68db400640 1 -- 172.21.15.32:0/1055546569 learned_addr learned my addr 172.21.15.32:0/1055546569 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68d3e00640 1 --2- 172.21.15.32:0/1055546569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f945632b50 0x55f945620440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68dbe00640 1 --2- 172.21.15.32:0/1055546569 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f945633470 0x55f945620980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68db400640 1 -- 172.21.15.32:0/1055546569 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f945633470 msgr2=0x55f945620980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68db400640 1 --2- 172.21.15.32:0/1055546569 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f945633470 0x55f945620980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68db400640 1 -- 172.21.15.32:0/1055546569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f945632b50 msgr2=0x55f945620440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.103+0000 7f68db400640 1 --2- 172.21.15.32:0/1055546569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f945632b50 0x55f945620440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68db400640 1 -- 172.21.15.32:0/1055546569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9453d0e40 con 0x55f9455e8cf0 2024-04-17T09:57:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68db400640 1 --2- 172.21.15.32:0/1055546569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9455e8cf0 0x55f9455ed820 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f68cc008010 tx=0x7f68cc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce71036f8182a61 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:57:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68c8c00640 1 -- 172.21.15.32:0/1055546569 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc006040 con 0x55f9455e8cf0 2024-04-17T09:57:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68c8c00640 1 -- 172.21.15.32:0/1055546569 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f68cc0061e0 con 0x55f9455e8cf0 2024-04-17T09:57:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68e5550000 1 -- 172.21.15.32:0/1055546569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68cc025070 con 0x55f9455e8cf0 2024-04-17T09:57:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68c8c00640 1 -- 172.21.15.32:0/1055546569 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f68cc01b020 con 0x55f9455e8cf0 2024-04-17T09:57:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68e5550000 10 client.6915 fetch_fsmap learned FSMap version 42 2024-04-17T09:57:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68e5550000 1 -- 172.21.15.32:0/1055546569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) -- 0x55f945401e00 con 0x55f9455e8cf0 2024-04-17T09:57:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68c8c00640 1 -- 172.21.15.32:0/1055546569 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68cc025070 con 0x55f9455e8cf0 2024-04-17T09:57:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68c8c00640 1 -- 172.21.15.32:0/1055546569 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 42) ==== 3495+0+0 (secure 0 0 0) 0x7f68cc0063e0 con 0x55f9455e8cf0 2024-04-17T09:57:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68e5550000 10 client.6915 fetch_fsmap finished waiting for FSMap version 42 2024-04-17T09:57:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68e5550000 10 client.6915 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T09:57:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68e5550000 20 client.6915 populate_metadata read hostname 'smithi032' 2024-04-17T09:57:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68e5550000 1 --2- 172.21.15.32:0/1055546569 >> [v2:172.21.15.32:6834/663537244,v1:172.21.15.32:6836/663537244] conn(0x55f9456420b0 0x55f9456444a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68e5550000 4 client.6915 mds_command: new command op to 6786 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:57:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68e5550000 1 -- 172.21.15.32:0/1055546569 --> [v2:172.21.15.32:6834/663537244,v1:172.21.15.32:6836/663537244] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9453cfbc0 con 0x55f9456420b0 2024-04-17T09:57:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68d3e00640 1 --2- 172.21.15.32:0/1055546569 >> [v2:172.21.15.32:6834/663537244,v1:172.21.15.32:6836/663537244] conn(0x55f9456420b0 0x55f9456444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68d3e00640 1 --2- 172.21.15.32:0/1055546569 >> [v2:172.21.15.32:6834/663537244,v1:172.21.15.32:6836/663537244] conn(0x55f9456420b0 0x55f9456444a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de98728edab3bf27 server_cookie=749b71c6c0297861 in_seq=0 out_seq=0 2024-04-17T09:57:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68c8c00640 10 client.6915 ms_handle_connect on v2:172.21.15.32:6834/663537244 2024-04-17T09:57:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68c8c00640 1 -- 172.21.15.32:0/1055546569 <== mds.0 v2:172.21.15.32:6834/663537244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f9453cfbc0 con 0x55f9456420b0 2024-04-17T09:57:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68c8c00640 10 client.6915 handle_command_reply: tid=0 2024-04-17T09:57:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68e5550000 1 client.6915 shutdown 2024-04-17T09:57:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68e5550000 1 -- 172.21.15.32:0/1055546569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9455e8cf0 msgr2=0x55f9455ed820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68e5550000 1 --2- 172.21.15.32:0/1055546569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9455e8cf0 0x55f9455ed820 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f68cc008010 tx=0x7f68cc004740 comp rx=0 tx=0).stop 2024-04-17T09:57:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68e5550000 1 -- 172.21.15.32:0/1055546569 shutdown_connections 2024-04-17T09:57:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68e5550000 1 --2- 172.21.15.32:0/1055546569 >> [v2:172.21.15.32:6834/663537244,v1:172.21.15.32:6836/663537244] conn(0x55f9456420b0 0x55f9456444a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68e5550000 1 --2- 172.21.15.32:0/1055546569 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f945633470 0x55f945620980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68e5550000 1 --2- 172.21.15.32:0/1055546569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f945632b50 0x55f945620440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68c8c00640 0 client.6915 ms_handle_reset on v2:172.21.15.32:6834/663537244 2024-04-17T09:57:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68e5550000 1 --2- 172.21.15.32:0/1055546569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9455e8cf0 0x55f9455ed820 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68e5550000 1 -- 172.21.15.32:0/1055546569 >> 172.21.15.32:0/1055546569 conn(0x55f9455ee470 msgr2=0x55f9455e77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68e5550000 1 -- 172.21.15.32:0/1055546569 shutdown_connections 2024-04-17T09:57:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68e5550000 1 -- 172.21.15.32:0/1055546569 wait complete. 2024-04-17T09:57:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.107+0000 7f68e5550000 20 client.6915 trim_cache size 0 max 16384 2024-04-17T09:57:18.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68e5550000 1 Processor -- start 2024-04-17T09:57:18.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68e5550000 1 -- start start 2024-04-17T09:57:18.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68e5550000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9455e8cf0 0x55f945620020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68e5550000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f945632b50 0x55f945620560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68e5550000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f945633470 0x55f9456434a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68e5550000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc203440 con 0x55f945632b50 2024-04-17T09:57:18.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68e5550000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc1f8b20 con 0x55f9455e8cf0 2024-04-17T09:57:18.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68e5550000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f68dc1f9170 con 0x55f945633470 2024-04-17T09:57:18.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68d3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f945632b50 0x55f945620560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68d3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f945632b50 0x55f945620560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59832/0 (socket says 172.21.15.32:59832) 2024-04-17T09:57:18.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68d3e00640 1 -- 172.21.15.32:0/2205523781 learned_addr learned my addr 172.21.15.32:0/2205523781 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:18.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68db400640 1 --2- 172.21.15.32:0/2205523781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9455e8cf0 0x55f945620020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68dbe00640 1 --2- 172.21.15.32:0/2205523781 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f945633470 0x55f9456434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68d3e00640 1 -- 172.21.15.32:0/2205523781 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f945633470 msgr2=0x55f9456434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68d3e00640 1 --2- 172.21.15.32:0/2205523781 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f945633470 0x55f9456434a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68d3e00640 1 -- 172.21.15.32:0/2205523781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9455e8cf0 msgr2=0x55f945620020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68d3e00640 1 --2- 172.21.15.32:0/2205523781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9455e8cf0 0x55f945620020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68d3e00640 1 -- 172.21.15.32:0/2205523781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9454b7d30 con 0x55f945632b50 2024-04-17T09:57:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68db400640 1 --2- 172.21.15.32:0/2205523781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9455e8cf0 0x55f945620020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:57:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68d3e00640 1 --2- 172.21.15.32:0/2205523781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f945632b50 0x55f945620560 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0d1170 tx=0x7f68c4001c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c99ded8e1a82e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:18.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68cb400640 1 -- 172.21.15.32:0/2205523781 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c4005110 con 0x55f945632b50 2024-04-17T09:57:18.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68cb400640 1 -- 172.21.15.32:0/2205523781 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68c40052b0 con 0x55f945632b50 2024-04-17T09:57:18.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68cb400640 1 -- 172.21.15.32:0/2205523781 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c40168d0 con 0x55f945632b50 2024-04-17T09:57:18.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68e5550000 1 -- 172.21.15.32:0/2205523781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f945632b50 msgr2=0x55f945620560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:18.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.127+0000 7f68e5550000 1 --2- 172.21.15.32:0/2205523781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f945632b50 0x55f945620560 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0d1170 tx=0x7f68c4001c60 comp rx=0 tx=0).stop 2024-04-17T09:57:18.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 1 -- 172.21.15.32:0/2205523781 shutdown_connections 2024-04-17T09:57:18.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 1 --2- 172.21.15.32:0/2205523781 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f945633470 0x55f9456434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 1 --2- 172.21.15.32:0/2205523781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f945632b50 0x55f945620560 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 1 --2- 172.21.15.32:0/2205523781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9455e8cf0 0x55f945620020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 1 -- 172.21.15.32:0/2205523781 >> 172.21.15.32:0/2205523781 conn(0x55f9455ee470 msgr2=0x55f94561f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:18.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 1 -- 172.21.15.32:0/2205523781 shutdown_connections 2024-04-17T09:57:18.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 1 -- 172.21.15.32:0/2205523781 wait complete. 2024-04-17T09:57:18.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 1 -- start start 2024-04-17T09:57:18.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 1 Processor -- start 2024-04-17T09:57:18.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9455e8cf0 0x55f945646ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f945632b50 0x55f94565e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f945633470 0x55f94565e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f94543ab00 con 0x55f945632b50 2024-04-17T09:57:18.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f9454357e0 con 0x55f9455e8cf0 2024-04-17T09:57:18.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f9454436e0 con 0x55f945633470 2024-04-17T09:57:18.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 10 client.0 _ll_register_callbacks cb 0x7f68c4007e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:57:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68d3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f945632b50 0x55f94565e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68d3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f945632b50 0x55f94565e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59840/0 (socket says 172.21.15.32:59840) 2024-04-17T09:57:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68d3e00640 1 -- 172.21.15.32:0/1278310440 learned_addr learned my addr 172.21.15.32:0/1278310440 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68db400640 1 --2- 172.21.15.32:0/1278310440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9455e8cf0 0x55f945646ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68dbe00640 1 --2- 172.21.15.32:0/1278310440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f945633470 0x55f94565e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68d3e00640 1 -- 172.21.15.32:0/1278310440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f945633470 msgr2=0x55f94565e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68d3e00640 1 --2- 172.21.15.32:0/1278310440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f945633470 0x55f94565e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68d3e00640 1 -- 172.21.15.32:0/1278310440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9455e8cf0 msgr2=0x55f945646ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68d3e00640 1 --2- 172.21.15.32:0/1278310440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9455e8cf0 0x55f945646ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68d3e00640 1 -- 172.21.15.32:0/1278310440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9453c87c0 con 0x55f945632b50 2024-04-17T09:57:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68db400640 1 --2- 172.21.15.32:0/1278310440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9455e8cf0 0x55f945646ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:57:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68d3e00640 1 --2- 172.21.15.32:0/1278310440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f945632b50 0x55f94565e030 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0cde40 tx=0x7f68c4000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abbd53c586afca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:57:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68c8c00640 1 -- 172.21.15.32:0/1278310440 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c4005110 con 0x55f945632b50 2024-04-17T09:57:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68c8c00640 1 -- 172.21.15.32:0/1278310440 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f68c4010040 con 0x55f945632b50 2024-04-17T09:57:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 1 -- 172.21.15.32:0/1278310440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68c40052b0 con 0x55f945632b50 2024-04-17T09:57:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68c8c00640 1 -- 172.21.15.32:0/1278310440 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f68c4005460 con 0x55f945632b50 2024-04-17T09:57:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 10 client.6921 fetch_fsmap learned FSMap version 42 2024-04-17T09:57:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 1 -- 172.21.15.32:0/1278310440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) -- 0x55f9453db650 con 0x55f945632b50 2024-04-17T09:57:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68c8c00640 1 -- 172.21.15.32:0/1278310440 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68c40052b0 con 0x55f945632b50 2024-04-17T09:57:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68c8c00640 1 -- 172.21.15.32:0/1278310440 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 42) ==== 3495+0+0 (secure 0 0 0) 0x7f68c4038020 con 0x55f945632b50 2024-04-17T09:57:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 10 client.6921 fetch_fsmap finished waiting for FSMap version 42 2024-04-17T09:57:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 10 client.6921 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T09:57:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 20 client.6921 populate_metadata read hostname 'smithi032' 2024-04-17T09:57:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 1 --2- 172.21.15.32:0/1278310440 >> [v2:172.21.15.32:6834/663537244,v1:172.21.15.32:6836/663537244] conn(0x55f9456a1670 0x55f9456a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 4 client.6921 mds_command: new command op to 6786 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T09:57:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.131+0000 7f68e5550000 1 -- 172.21.15.32:0/1278310440 --> [v2:172.21.15.32:6834/663537244,v1:172.21.15.32:6836/663537244] -- command(tid 0: {"prefix": "status"}) -- 0x55f945443ef0 con 0x55f9456a1670 2024-04-17T09:57:18.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68db400640 1 --2- 172.21.15.32:0/1278310440 >> [v2:172.21.15.32:6834/663537244,v1:172.21.15.32:6836/663537244] conn(0x55f9456a1670 0x55f9456a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68db400640 1 --2- 172.21.15.32:0/1278310440 >> [v2:172.21.15.32:6834/663537244,v1:172.21.15.32:6836/663537244] conn(0x55f9456a1670 0x55f9456a1a50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e19db2a76a04d2d5 server_cookie=75d3ed8581cfba4 in_seq=0 out_seq=0 2024-04-17T09:57:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68c8c00640 10 client.6921 ms_handle_connect on v2:172.21.15.32:6834/663537244 2024-04-17T09:57:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68c8c00640 1 -- 172.21.15.32:0/1278310440 <== mds.0 v2:172.21.15.32:6834/663537244 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x55f945443ef0 con 0x55f9456a1670 2024-04-17T09:57:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68c8c00640 10 client.6921 handle_command_reply: tid=0 2024-04-17T09:57:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e5550000 1 client.6921 shutdown 2024-04-17T09:57:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e5550000 1 -- 172.21.15.32:0/1278310440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f945632b50 msgr2=0x55f94565e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e5550000 1 --2- 172.21.15.32:0/1278310440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f945632b50 0x55f94565e030 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0cde40 tx=0x7f68c4000bd0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e5550000 1 -- 172.21.15.32:0/1278310440 shutdown_connections 2024-04-17T09:57:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e5550000 1 --2- 172.21.15.32:0/1278310440 >> [v2:172.21.15.32:6834/663537244,v1:172.21.15.32:6836/663537244] conn(0x55f9456a1670 0x55f9456a1a50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e5550000 1 --2- 172.21.15.32:0/1278310440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f945633470 0x55f94565e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e5550000 1 --2- 172.21.15.32:0/1278310440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f945632b50 0x55f94565e030 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e5550000 1 --2- 172.21.15.32:0/1278310440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9455e8cf0 0x55f945646ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68c8c00640 0 client.6921 ms_handle_reset on v2:172.21.15.32:6834/663537244 2024-04-17T09:57:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e5550000 1 -- 172.21.15.32:0/1278310440 >> 172.21.15.32:0/1278310440 conn(0x55f9455ee470 msgr2=0x55f9455ea070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:18.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e5550000 1 -- 172.21.15.32:0/1278310440 shutdown_connections 2024-04-17T09:57:18.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e5550000 1 -- 172.21.15.32:0/1278310440 wait complete. 2024-04-17T09:57:18.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e5550000 20 client.6921 trim_cache size 0 max 16384 2024-04-17T09:57:18.141 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:57:18.141 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "ff178df8-4c06-4367-a1e9-2e3a66d870d0", 2024-04-17T09:57:18.141 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-17T09:57:18.141 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 6786, 2024-04-17T09:57:18.141 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-17T09:57:18.141 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-17T09:57:18.141 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-17T09:57:18.141 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.967140508, 2024-04-17T09:57:18.142 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 42, 2024-04-17T09:57:18.142 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 68, 2024-04-17T09:57:18.142 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 68, 2024-04-17T09:57:18.142 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 10.810171078 2024-04-17T09:57:18.142 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:57:18.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e2000640 1 -- 172.21.15.32:0/2367741421 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f68b8068380 msgr2=0x7f68b806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:18.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e2000640 1 --2- 172.21.15.32:0/2367741421 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f68b8068380 0x7f68b806a840 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00a9a0 tx=0x7f68cc008040 comp rx=0 tx=0).stop 2024-04-17T09:57:18.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e2000640 1 -- 172.21.15.32:0/2367741421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68dc155e50 msgr2=0x7f68dc153280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:18.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e2000640 1 --2- 172.21.15.32:0/2367741421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68dc155e50 0x7f68dc153280 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f68d4019600 tx=0x7f68d4019790 comp rx=0 tx=0).stop 2024-04-17T09:57:18.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e2000640 1 -- 172.21.15.32:0/2367741421 shutdown_connections 2024-04-17T09:57:18.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e2000640 1 --2- 172.21.15.32:0/2367741421 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f68b8068380 0x7f68b806a840 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e2000640 1 --2- 172.21.15.32:0/2367741421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68dc155e50 0x7f68dc153280 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e2000640 1 --2- 172.21.15.32:0/2367741421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68dc155a70 0x7f68dc15aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e2000640 1 --2- 172.21.15.32:0/2367741421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68dc0b3340 0x7f68dc0cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e2000640 1 -- 172.21.15.32:0/2367741421 >> 172.21.15.32:0/2367741421 conn(0x7f68dc06ba30 msgr2=0x7f68dc154260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:18.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e2000640 1 -- 172.21.15.32:0/2367741421 shutdown_connections 2024-04-17T09:57:18.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.135+0000 7f68e2000640 1 -- 172.21.15.32:0/2367741421 wait complete. 2024-04-17T09:57:18.157 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T09:57:18.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.295+0000 7fd21b600640 1 Processor -- start 2024-04-17T09:57:18.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.295+0000 7fd21b600640 1 -- start start 2024-04-17T09:57:18.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.299+0000 7fd21b600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd21415baa0 0x7fd21415bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.299+0000 7fd21b600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd21406ef60 0x7fd21406f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.299+0000 7fd21b600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd214072bb0 con 0x7fd214144e90 2024-04-17T09:57:18.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.299+0000 7fd21b600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd21406f8f0 con 0x7fd21415baa0 2024-04-17T09:57:18.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.299+0000 7fd21b600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd214145270 con 0x7fd21406ef60 2024-04-17T09:57:18.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.299+0000 7fd20be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd21415baa0 0x7fd21415bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.299+0000 7fd20be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd21415baa0 0x7fd21415bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59848/0 (socket says 172.21.15.32:59848) 2024-04-17T09:57:18.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.299+0000 7fd20be00640 1 -- 172.21.15.32:0/3640253713 learned_addr learned my addr 172.21.15.32:0/3640253713 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:18.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.299+0000 7fd20be00640 1 -- 172.21.15.32:0/3640253713 >> v2:172.21.15.144:3300/0 conn(0x7fd21406ef60 msgr2=0x7fd21406f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:18.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.299+0000 7fd219600640 1 --2- 172.21.15.32:0/3640253713 >> v2:172.21.15.144:3300/0 conn(0x7fd21406ef60 0x7fd21406f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.299+0000 7fd20be00640 1 --2- 172.21.15.32:0/3640253713 >> v2:172.21.15.144:3300/0 conn(0x7fd21406ef60 0x7fd21406f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.299+0000 7fd20be00640 1 -- 172.21.15.32:0/3640253713 >> v1:172.21.15.144:6789/0 conn(0x7fd214144e90 legacy=0x7fd2140b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:18.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.299+0000 7fd20be00640 1 -- 172.21.15.32:0/3640253713 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2140b3dc0 con 0x7fd21415baa0 2024-04-17T09:57:18.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.299+0000 7fd219600640 1 --2- 172.21.15.32:0/3640253713 >> v2:172.21.15.144:3300/0 conn(0x7fd21406ef60 0x7fd21406f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:57:18.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.303+0000 7fd20be00640 1 --2- 172.21.15.32:0/3640253713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21415baa0 0x7fd21415bf30 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fd2140b3a00 tx=0x7fd1fc01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a454fee8ff05a01b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:18.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.303+0000 7fd213e00640 1 -- 172.21.15.32:0/3640253713 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc0b1070 con 0x7fd21415baa0 2024-04-17T09:57:18.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.303+0000 7fd213e00640 1 -- 172.21.15.32:0/3640253713 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1fc0ac630 con 0x7fd21415baa0 2024-04-17T09:57:18.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.303+0000 7fd213e00640 1 -- 172.21.15.32:0/3640253713 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc0a8050 con 0x7fd21415baa0 2024-04-17T09:57:18.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.303+0000 7fd21b600640 1 -- 172.21.15.32:0/3640253713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21415baa0 msgr2=0x7fd21415bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:18.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.303+0000 7fd21b600640 1 --2- 172.21.15.32:0/3640253713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21415baa0 0x7fd21415bf30 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fd2140b3a00 tx=0x7fd1fc01c4e0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.303+0000 7fd21b600640 1 -- 172.21.15.32:0/3640253713 shutdown_connections 2024-04-17T09:57:18.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.303+0000 7fd21b600640 1 --2- 172.21.15.32:0/3640253713 >> v2:172.21.15.144:3300/0 conn(0x7fd21406ef60 0x7fd21406f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.303+0000 7fd21b600640 1 --2- 172.21.15.32:0/3640253713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21415baa0 0x7fd21415bf30 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.303+0000 7fd21b600640 1 -- 172.21.15.32:0/3640253713 >> 172.21.15.32:0/3640253713 conn(0x7fd21406ba30 msgr2=0x7fd21406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:18.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.303+0000 7fd21b600640 1 -- 172.21.15.32:0/3640253713 shutdown_connections 2024-04-17T09:57:18.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.303+0000 7fd21b600640 1 -- 172.21.15.32:0/3640253713 wait complete. 2024-04-17T09:57:18.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.303+0000 7fd21b600640 1 Processor -- start 2024-04-17T09:57:18.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.303+0000 7fd21b600640 1 -- start start 2024-04-17T09:57:18.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.303+0000 7fd21b600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd21406ef60 0x7fd2140b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.303+0000 7fd21b600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd214144e90 0x7fd2140b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.303+0000 7fd21b600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd21415baa0 0x7fd21413c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.303+0000 7fd21b600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd2140727f0 con 0x7fd214144e90 2024-04-17T09:57:18.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.303+0000 7fd21b600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd214072670 con 0x7fd21406ef60 2024-04-17T09:57:18.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.303+0000 7fd21b600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd214072970 con 0x7fd21415baa0 2024-04-17T09:57:18.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.307+0000 7fd20be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd214144e90 0x7fd2140b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.307+0000 7fd20be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd214144e90 0x7fd2140b1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59862/0 (socket says 172.21.15.32:59862) 2024-04-17T09:57:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.307+0000 7fd20be00640 1 -- 172.21.15.32:0/1322349893 learned_addr learned my addr 172.21.15.32:0/1322349893 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.307+0000 7fd20be00640 1 -- 172.21.15.32:0/1322349893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd21415baa0 msgr2=0x7fd21413c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.307+0000 7fd218c00640 1 --2- 172.21.15.32:0/1322349893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd21406ef60 0x7fd2140b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.307+0000 7fd219600640 1 --2- 172.21.15.32:0/1322349893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd21415baa0 0x7fd21413c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.307+0000 7fd20be00640 1 --2- 172.21.15.32:0/1322349893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd21415baa0 0x7fd21413c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.307+0000 7fd20be00640 1 -- 172.21.15.32:0/1322349893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd21406ef60 msgr2=0x7fd2140b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.307+0000 7fd20be00640 1 --2- 172.21.15.32:0/1322349893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd21406ef60 0x7fd2140b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.307+0000 7fd20be00640 1 -- 172.21.15.32:0/1322349893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21415b200 con 0x7fd214144e90 2024-04-17T09:57:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.307+0000 7fd218c00640 1 --2- 172.21.15.32:0/1322349893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd21406ef60 0x7fd2140b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:57:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.307+0000 7fd219600640 1 --2- 172.21.15.32:0/1322349893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd21415baa0 0x7fd21413c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:57:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.307+0000 7fd20be00640 1 --2- 172.21.15.32:0/1322349893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd214144e90 0x7fd2140b1b70 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc01ab90 tx=0x7fd1fc019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.307+0000 7fd211600640 1 -- 172.21.15.32:0/1322349893 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc0bc070 con 0x7fd214144e90 2024-04-17T09:57:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.307+0000 7fd211600640 1 -- 172.21.15.32:0/1322349893 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1fc0ab820 con 0x7fd214144e90 2024-04-17T09:57:18.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.307+0000 7fd211600640 1 -- 172.21.15.32:0/1322349893 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc0b1070 con 0x7fd214144e90 2024-04-17T09:57:18.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.307+0000 7fd21b600640 1 -- 172.21.15.32:0/1322349893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21415b4e0 con 0x7fd214144e90 2024-04-17T09:57:18.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.307+0000 7fd21b600640 1 -- 172.21.15.32:0/1322349893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd21413cbc0 con 0x7fd214144e90 2024-04-17T09:57:18.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.307+0000 7fd21b600640 1 -- 172.21.15.32:0/1322349893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd214068a90 con 0x7fd214144e90 2024-04-17T09:57:18.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.307+0000 7fd211600640 1 -- 172.21.15.32:0/1322349893 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd1fc0bf070 con 0x7fd214144e90 2024-04-17T09:57:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.307+0000 7fd211600640 1 --2- 172.21.15.32:0/1322349893 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd1f00683b0 0x7fd1f006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.311+0000 7fd211600640 1 -- 172.21.15.32:0/1322349893 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fd1fc120540 con 0x7fd214144e90 2024-04-17T09:57:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.311+0000 7fd218c00640 1 --2- 172.21.15.32:0/1322349893 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd1f00683b0 0x7fd1f006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.311+0000 7fd218c00640 1 --2- 172.21.15.32:0/1322349893 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd1f00683b0 0x7fd1f006a870 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd204003e40 tx=0x7fd204009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:18.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.315+0000 7fd211600640 1 -- 172.21.15.32:0/1322349893 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd1fc0160d0 con 0x7fd214144e90 2024-04-17T09:57:18.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.515+0000 7fd21b600640 1 -- 172.21.15.32:0/1322349893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd21413d0b0 con 0x7fd214144e90 2024-04-17T09:57:18.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.515+0000 7fd211600640 1 -- 172.21.15.32:0/1322349893 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v42) ==== 53+0+73 (secure 0 0 0) 0x7fd1fc0f0a60 con 0x7fd214144e90 2024-04-17T09:57:18.518 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T09:57:18.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.519+0000 7fd21b600640 1 -- 172.21.15.32:0/1322349893 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd1f00683b0 msgr2=0x7fd1f006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:18.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.519+0000 7fd21b600640 1 --2- 172.21.15.32:0/1322349893 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd1f00683b0 0x7fd1f006a870 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd204003e40 tx=0x7fd204009040 comp rx=0 tx=0).stop 2024-04-17T09:57:18.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.519+0000 7fd21b600640 1 -- 172.21.15.32:0/1322349893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd214144e90 msgr2=0x7fd2140b1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:18.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.519+0000 7fd21b600640 1 --2- 172.21.15.32:0/1322349893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd214144e90 0x7fd2140b1b70 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc01ab90 tx=0x7fd1fc019280 comp rx=0 tx=0).stop 2024-04-17T09:57:18.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.519+0000 7fd21b600640 1 -- 172.21.15.32:0/1322349893 shutdown_connections 2024-04-17T09:57:18.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.519+0000 7fd21b600640 1 --2- 172.21.15.32:0/1322349893 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd1f00683b0 0x7fd1f006a870 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.519+0000 7fd21b600640 1 --2- 172.21.15.32:0/1322349893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd21415baa0 0x7fd21413c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.519+0000 7fd21b600640 1 --2- 172.21.15.32:0/1322349893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd214144e90 0x7fd2140b1b70 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.519+0000 7fd21b600640 1 --2- 172.21.15.32:0/1322349893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd21406ef60 0x7fd2140b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:18.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.519+0000 7fd21b600640 1 -- 172.21.15.32:0/1322349893 >> 172.21.15.32:0/1322349893 conn(0x7fd21406ba30 msgr2=0x7fd2140adeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:18.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.519+0000 7fd21b600640 1 -- 172.21.15.32:0/1322349893 shutdown_connections 2024-04-17T09:57:18.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18.519+0000 7fd21b600640 1 -- 172.21.15.32:0/1322349893 wait complete. 2024-04-17T09:57:18.539 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T09:57:18.540 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T09:57:18.540 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-17T09:57:18.540 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T09:57:18.540 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T09:57:18.540 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T09:57:18.540 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T09:57:18.540 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T09:57:18.540 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:57:18.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:18.540 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-17T09:57:18.549 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T09:57:18.549 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T09:57:18.549 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T09:57:18.549 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:57:18.550 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-17T09:57:18.550 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:57:18.550 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:57:18.550 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-17T09:57:18.550 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-17T09:57:18.550 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T09:57:18.550 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:57:18.550 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-17T09:57:18.550 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-17T09:57:18.550 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42215sec preferred_lft 42215sec 2024-04-17T09:57:18.550 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2024-04-17T09:57:18.550 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:57:18.550 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:57:18.550 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-17T09:57:18.551 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-17T09:57:18.551 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:57:18.551 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:57:18.551 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-17T09:57:18.551 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T09:57:18.551 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:cd:f4:aa:5b brd ff:ff:ff:ff:ff:ff 2024-04-17T09:57:18.551 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T09:57:18.551 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:57:18.551 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T09:57:18.551 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T09:57:18.551 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T09:57:18.551 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:57:18.551 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T09:57:18.551 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:57:18.551 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T09:57:18.552 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:57:18.552 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T09:57:18.552 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:57:18.552 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T09:57:18.552 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:57:18.552 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T09:57:18.552 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:57:18.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:18.554 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T09:57:18.612 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-17T09:57:18.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:18.615 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-17T09:57:18.668 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-17T09:57:18.668 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T09:57:18.668 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-17T09:57:18.668 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T09:57:18.668 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-17T09:57:18.668 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T09:57:18.669 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:57:18.669 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T09:57:18.669 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-17T09:57:18.669 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-17T09:57:18.669 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-17T09:57:18.669 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T09:57:18.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:57:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:57:18.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:18.807 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-17T09:57:18.816 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T09:57:18.816 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T09:57:18.817 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:57:18.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:18.817 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-17T09:57:18.879 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T09:57:18.879 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:57:18.879 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T09:57:18.931 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:57:18.931 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:57:18.931 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T09:57:18.982 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T09:57:19.043 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:57:19.096 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T09:57:19.096 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T09:57:19.096 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T09:57:19.096 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T09:57:19.096 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T09:57:19.097 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T09:57:19.097 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T09:57:19.097 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T09:57:19.097 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T09:57:19.097 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T09:57:19.097 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T09:57:19.097 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T09:57:19.097 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:57:19.097 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T09:57:19.097 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T09:57:19.097 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T09:57:19.097 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T09:57:19.097 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T09:57:19.098 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T09:57:19.098 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T09:57:19.098 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:57:19.098 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T09:57:19.098 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T09:57:19.098 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T09:57:19.098 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T09:57:19.098 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T09:57:19.098 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T09:57:19.098 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T09:57:19.098 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T09:57:19.098 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:57:19.098 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:57:19.099 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:57:19.099 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:57:19.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:19.100 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T09:57:19.150 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T09:57:19.151 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T09:57:19.201 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T09:57:19.222 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:57:19.277 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T09:57:19.277 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T09:57:19.277 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T09:57:19.277 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T09:57:19.278 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T09:57:19.278 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T09:57:19.278 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T09:57:19.278 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T09:57:19.278 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T09:57:19.278 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T09:57:19.278 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T09:57:19.278 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T09:57:19.278 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:57:19.278 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T09:57:19.278 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T09:57:19.278 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T09:57:19.278 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T09:57:19.279 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T09:57:19.279 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T09:57:19.279 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T09:57:19.279 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:57:19.279 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T09:57:19.279 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T09:57:19.279 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T09:57:19.279 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T09:57:19.279 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T09:57:19.279 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T09:57:19.279 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T09:57:19.279 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T09:57:19.279 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:57:19.279 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:57:19.280 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:57:19.280 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:57:19.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:19.281 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T09:57:19.299 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2024-04-17T09:57:19.295+0000 7f485a59c580 -1 init, newargv = 0x559011c4de70 newargc=15 2024-04-17T09:57:19.299 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[41916]: starting ceph client 2024-04-17T09:57:19.306 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[41916]: starting fuse 2024-04-17T09:57:19.309 INFO:teuthology.orchestra.run.smithi032.stdout:44 2024-04-17T09:57:19.309 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-17T09:57:19.309 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2024-04-17T09:57:19.309 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2024-04-17T09:57:19.309 DEBUG:teuthology.orchestra.run.smithi032:> import re 2024-04-17T09:57:19.309 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T09:57:19.309 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2024-04-17T09:57:19.309 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:57:19.309 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2024-04-17T09:57:19.309 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T09:57:19.310 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2024-04-17T09:57:19.310 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T09:57:19.310 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:57:19.310 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2024-04-17T09:57:19.310 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2024-04-17T09:57:19.310 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2024-04-17T09:57:19.310 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2024-04-17T09:57:19.310 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:57:19.310 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2024-04-17T09:57:19.310 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T09:57:19.310 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T09:57:19.310 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T09:57:19.310 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2024-04-17T09:57:19.310 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2024-04-17T09:57:19.310 DEBUG:teuthology.orchestra.run.smithi032:> return f 2024-04-17T09:57:19.311 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T09:57:19.311 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:57:19.311 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2024-04-17T09:57:19.311 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T09:57:19.437 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.41916.asok 2024-04-17T09:57:19.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:57:19.449 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41916.asok 2024-04-17T09:57:19.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:19.450 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41916.asok status 2024-04-17T09:57:19.627 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:57:19.627 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2024-04-17T09:57:19.627 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-17T09:57:19.627 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T09:57:19.627 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2024-04-17T09:57:19.628 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2024-04-17T09:57:19.628 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T09:57:19.628 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "41916", 2024-04-17T09:57:19.628 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2024-04-17T09:57:19.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:19.628 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2024-04-17T09:57:19.628 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2024-04-17T09:57:19.628 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 6930, 2024-04-17T09:57:19.628 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2024-04-17T09:57:19.628 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2024-04-17T09:57:19.628 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2024-04-17T09:57:19.628 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 6930 2024-04-17T09:57:19.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:19.628 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T09:57:19.628 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T09:57:19.629 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T09:57:19.629 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1672926327 2024-04-17T09:57:19.629 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:19.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:19.629 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T09:57:19.629 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T09:57:19.629 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T09:57:19.629 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1672926327 2024-04-17T09:57:19.629 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:19.629 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.6930 192.168.0.1:0/1672926327", 2024-04-17T09:57:19.629 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/1672926327", 2024-04-17T09:57:19.629 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2024-04-17T09:57:19.629 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 42, 2024-04-17T09:57:19.629 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 68, 2024-04-17T09:57:19.630 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2024-04-17T09:57:19.630 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2024-04-17T09:57:19.630 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs" 2024-04-17T09:57:19.630 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:57:19.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:19.643 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2024-04-17T09:57:19.644 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T09:57:19.653 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2024-04-17T09:57:19.653 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T09:57:19.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:19.653 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T09:57:19.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:19.715 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-04-17T09:57:19.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:19.773 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-04-17T09:57:19.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:19.826 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-04-17T09:57:19.877 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T09:57:19.877 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T09:57:19.878 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T09:57:19.878 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:57:19.878 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-04-17T09:57:19.878 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T09:57:19.984 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627776 2024-04-17T09:57:19.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:57:19.994 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:57:19.995 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:57:20.043 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:57:20.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.227+0000 7f3043200640 1 Processor -- start 2024-04-17T09:57:20.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.227+0000 7f3043200640 1 -- start start 2024-04-17T09:57:20.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.227+0000 7f3043200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f303c145f30 0x7f303c071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:20.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.231+0000 7f3043200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f303c0cdac0 con 0x7f303c15d8d0 2024-04-17T09:57:20.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.231+0000 7f3043200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f303c15cdf0 con 0x7f303c0cd580 2024-04-17T09:57:20.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.231+0000 7f3043200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f303c06f400 con 0x7f303c145f30 2024-04-17T09:57:20.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.231+0000 7f3033e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f303c15d8d0 0x7f303c0cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41662/0 (socket says 172.21.15.32:41662) 2024-04-17T09:57:20.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.231+0000 7f3033e00640 1 -- 172.21.15.32:0/350930432 learned_addr learned my addr 172.21.15.32:0/350930432 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:20.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.231+0000 7f303be00640 1 --2- 172.21.15.32:0/350930432 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f303c145f30 0x7f303c071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:20.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.231+0000 7f303b400640 1 -- 172.21.15.32:0/350930432 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2558049528 0 0) 0x7f303c0cdac0 con 0x7f303c15d8d0 2024-04-17T09:57:20.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.231+0000 7f303b400640 1 -- 172.21.15.32:0/350930432 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3020003530 con 0x7f303c15d8d0 2024-04-17T09:57:20.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.231+0000 7f303b400640 1 -- 172.21.15.32:0/350930432 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1591908725 0 0) 0x7f303c15cdf0 con 0x7f303c0cd580 2024-04-17T09:57:20.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.231+0000 7f303b400640 1 -- 172.21.15.32:0/350930432 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f303c0cdac0 con 0x7f303c0cd580 2024-04-17T09:57:20.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.231+0000 7f303be00640 1 -- 172.21.15.32:0/350930432 >> v1:172.21.15.144:6789/0 conn(0x7f303c0cd580 legacy=0x7f303c1601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:20.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.231+0000 7f303be00640 1 -- 172.21.15.32:0/350930432 >> v1:172.21.15.32:6789/0 conn(0x7f303c15d8d0 legacy=0x7f303c0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:20.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.231+0000 7f303be00640 1 -- 172.21.15.32:0/350930432 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303c15d0d0 con 0x7f303c145f30 2024-04-17T09:57:20.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.231+0000 7f303be00640 1 --2- 172.21.15.32:0/350930432 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f303c145f30 0x7f303c071df0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f302c0097f0 tx=0x7f302c01ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fea76f24ca49e70b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:20.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.235+0000 7f303b400640 1 -- 172.21.15.32:0/350930432 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c0b5070 con 0x7f303c145f30 2024-04-17T09:57:20.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.235+0000 7f303b400640 1 -- 172.21.15.32:0/350930432 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f302c0a89d0 con 0x7f303c145f30 2024-04-17T09:57:20.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.235+0000 7f303b400640 1 -- 172.21.15.32:0/350930432 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c0a8da0 con 0x7f303c145f30 2024-04-17T09:57:20.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.235+0000 7f3043200640 1 -- 172.21.15.32:0/350930432 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f303c145f30 msgr2=0x7f303c071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:20.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.235+0000 7f3043200640 1 --2- 172.21.15.32:0/350930432 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f303c145f30 0x7f303c071df0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f302c0097f0 tx=0x7f302c01ecd0 comp rx=0 tx=0).stop 2024-04-17T09:57:20.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.235+0000 7f3043200640 1 -- 172.21.15.32:0/350930432 shutdown_connections 2024-04-17T09:57:20.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.235+0000 7f3043200640 1 --2- 172.21.15.32:0/350930432 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f303c145f30 0x7f303c071df0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:20.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.235+0000 7f3043200640 1 -- 172.21.15.32:0/350930432 >> 172.21.15.32:0/350930432 conn(0x7f303c06ba30 msgr2=0x7f303c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:20.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.235+0000 7f3043200640 1 -- 172.21.15.32:0/350930432 shutdown_connections 2024-04-17T09:57:20.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.235+0000 7f3043200640 1 -- 172.21.15.32:0/350930432 wait complete. 2024-04-17T09:57:20.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.235+0000 7f3043200640 1 Processor -- start 2024-04-17T09:57:20.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.235+0000 7f3043200640 1 -- start start 2024-04-17T09:57:20.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.235+0000 7f3043200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f303c0cd580 0x7f303c0cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:20.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.235+0000 7f3043200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f303c145f30 0x7f303c0b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:20.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.235+0000 7f3043200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f303c15d8d0 0x7f303c133410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:20.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.235+0000 7f3043200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f303c0726c0 con 0x7f303c15d8d0 2024-04-17T09:57:20.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.235+0000 7f3043200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f303c072540 con 0x7f303c145f30 2024-04-17T09:57:20.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.235+0000 7f3043200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f303c146310 con 0x7f303c0cd580 2024-04-17T09:57:20.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.239+0000 7f3041200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f303c15d8d0 0x7f303c133410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:20.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.239+0000 7f3041200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f303c15d8d0 0x7f303c133410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59864/0 (socket says 172.21.15.32:59864) 2024-04-17T09:57:20.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.239+0000 7f3041200640 1 -- 172.21.15.32:0/1252732391 learned_addr learned my addr 172.21.15.32:0/1252732391 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:20.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.239+0000 7f3033e00640 1 --2- 172.21.15.32:0/1252732391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f303c145f30 0x7f303c0b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:20.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.239+0000 7f303be00640 1 --2- 172.21.15.32:0/1252732391 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f303c0cd580 0x7f303c0cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:20.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.239+0000 7f3041200640 1 -- 172.21.15.32:0/1252732391 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f303c0cd580 msgr2=0x7f303c0cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:20.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.239+0000 7f3041200640 1 --2- 172.21.15.32:0/1252732391 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f303c0cd580 0x7f303c0cb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:20.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.239+0000 7f3041200640 1 -- 172.21.15.32:0/1252732391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f303c145f30 msgr2=0x7f303c0b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:20.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.239+0000 7f3041200640 1 --2- 172.21.15.32:0/1252732391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f303c145f30 0x7f303c0b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:20.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.239+0000 7f3041200640 1 -- 172.21.15.32:0/1252732391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303c0cc2c0 con 0x7f303c15d8d0 2024-04-17T09:57:20.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.239+0000 7f3041200640 1 --2- 172.21.15.32:0/1252732391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f303c15d8d0 0x7f303c133410 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f303400c7a0 tx=0x7f303400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:20.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.239+0000 7f3038c00640 1 -- 172.21.15.32:0/1252732391 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3034003b80 con 0x7f303c15d8d0 2024-04-17T09:57:20.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.239+0000 7f3038c00640 1 -- 172.21.15.32:0/1252732391 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3034003d20 con 0x7f303c15d8d0 2024-04-17T09:57:20.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.239+0000 7f3043200640 1 -- 172.21.15.32:0/1252732391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f303c1e8160 con 0x7f303c15d8d0 2024-04-17T09:57:20.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.239+0000 7f3038c00640 1 -- 172.21.15.32:0/1252732391 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303400aa90 con 0x7f303c15d8d0 2024-04-17T09:57:20.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.239+0000 7f3043200640 1 -- 172.21.15.32:0/1252732391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f303c1e8720 con 0x7f303c15d8d0 2024-04-17T09:57:20.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.239+0000 7f3043200640 1 -- 172.21.15.32:0/1252732391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f303c068a90 con 0x7f303c15d8d0 2024-04-17T09:57:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.239+0000 7f3038c00640 1 -- 172.21.15.32:0/1252732391 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f303400ac30 con 0x7f303c15d8d0 2024-04-17T09:57:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.243+0000 7f3038c00640 1 --2- 172.21.15.32:0/1252732391 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3018068380 0x7f301806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.243+0000 7f3038c00640 1 -- 172.21.15.32:0/1252732391 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f30340868b0 con 0x7f303c15d8d0 2024-04-17T09:57:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.243+0000 7f303be00640 1 --2- 172.21.15.32:0/1252732391 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3018068380 0x7f301806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.243+0000 7f303be00640 1 --2- 172.21.15.32:0/1252732391 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3018068380 0x7f301806a840 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f302c0b2000 tx=0x7f302c0aa070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:20.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.247+0000 7f3038c00640 1 -- 172.21.15.32:0/1252732391 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3034056790 con 0x7f303c15d8d0 2024-04-17T09:57:20.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.411+0000 7f3043200640 1 -- 172.21.15.32:0/1252732391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f303c072f90 con 0x7f303c15d8d0 2024-04-17T09:57:20.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.415+0000 7f3038c00640 1 -- 172.21.15.32:0/1252732391 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16680 (secure 0 0 0) 0x7f303405aa90 con 0x7f303c15d8d0 2024-04-17T09:57:20.416 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:57:20.416 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":68,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:57:13.170280+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:57:09.948964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-17T09:57:10.512460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:57:20.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.415+0000 7f3043200640 1 -- 172.21.15.32:0/1252732391 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3018068380 msgr2=0x7f301806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:20.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.415+0000 7f3043200640 1 --2- 172.21.15.32:0/1252732391 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3018068380 0x7f301806a840 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f302c0b2000 tx=0x7f302c0aa070 comp rx=0 tx=0).stop 2024-04-17T09:57:20.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.415+0000 7f3043200640 1 -- 172.21.15.32:0/1252732391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f303c15d8d0 msgr2=0x7f303c133410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:20.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.415+0000 7f3043200640 1 --2- 172.21.15.32:0/1252732391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f303c15d8d0 0x7f303c133410 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f303400c7a0 tx=0x7f303400cc60 comp rx=0 tx=0).stop 2024-04-17T09:57:20.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.415+0000 7f3043200640 1 -- 172.21.15.32:0/1252732391 shutdown_connections 2024-04-17T09:57:20.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.415+0000 7f3043200640 1 --2- 172.21.15.32:0/1252732391 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3018068380 0x7f301806a840 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:20.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.415+0000 7f3043200640 1 --2- 172.21.15.32:0/1252732391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f303c15d8d0 0x7f303c133410 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:20.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.415+0000 7f3043200640 1 --2- 172.21.15.32:0/1252732391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f303c145f30 0x7f303c0b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:20.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.415+0000 7f3043200640 1 --2- 172.21.15.32:0/1252732391 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f303c0cd580 0x7f303c0cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:20.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.415+0000 7f3043200640 1 -- 172.21.15.32:0/1252732391 >> 172.21.15.32:0/1252732391 conn(0x7f303c06ba30 msgr2=0x7f303c15f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:20.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.415+0000 7f3043200640 1 -- 172.21.15.32:0/1252732391 shutdown_connections 2024-04-17T09:57:20.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.415+0000 7f3043200640 1 -- 172.21.15.32:0/1252732391 wait complete. 2024-04-17T09:57:20.438 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:57:09.948964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:57:10.512460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:57:20.439 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:57:20.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.579+0000 7f93f7200640 1 Processor -- start 2024-04-17T09:57:20.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.579+0000 7f93f7200640 1 -- start start 2024-04-17T09:57:20.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.583+0000 7f93f7200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93f006f7c0 0x7f93f006ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:20.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.583+0000 7f93f7200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f006f270 0x7f93f006bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:20.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.583+0000 7f93f7200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93f00cc940 con 0x7f93f015b7d0 2024-04-17T09:57:20.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.583+0000 7f93f7200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f93f00ccc20 con 0x7f93f006f7c0 2024-04-17T09:57:20.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.583+0000 7f93f7200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f93f006e0b0 con 0x7f93f006f270 2024-04-17T09:57:20.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.583+0000 7f93e7400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93f006f7c0 0x7f93f006ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:20.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.583+0000 7f93efe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f93f015b7d0 0x7f93f006e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53366/0 (socket says 172.21.15.32:53366) 2024-04-17T09:57:20.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.583+0000 7f93efe00640 1 -- 172.21.15.32:0/4224403684 learned_addr learned my addr 172.21.15.32:0/4224403684 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:20.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.583+0000 7f93f5200640 1 --2- 172.21.15.32:0/4224403684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f006f270 0x7f93f006bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:20.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.583+0000 7f93e7400640 1 -- 172.21.15.32:0/4224403684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f006f270 msgr2=0x7f93f006bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:20.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.583+0000 7f93e7400640 1 --2- 172.21.15.32:0/4224403684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f006f270 0x7f93f006bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:20.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.583+0000 7f93e7400640 1 -- 172.21.15.32:0/4224403684 >> v1:172.21.15.32:6789/0 conn(0x7f93f015b7d0 legacy=0x7f93f006e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:20.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.583+0000 7f93e7400640 1 -- 172.21.15.32:0/4224403684 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f006e390 con 0x7f93f006f7c0 2024-04-17T09:57:20.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.583+0000 7f93f5200640 1 --2- 172.21.15.32:0/4224403684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f006f270 0x7f93f006bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:57:20.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.583+0000 7f93ef400640 1 -- 172.21.15.32:0/4224403684 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2000765262 0 0) 0x7f93f00cc940 con 0x7f93f015b7d0 2024-04-17T09:57:20.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.587+0000 7f93e7400640 1 --2- 172.21.15.32:0/4224403684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93f006f7c0 0x7f93f006ba30 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f93d80097c0 tx=0x7f93d801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48099bc3bd7d6180 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:20.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.587+0000 7f93ef400640 1 -- 172.21.15.32:0/4224403684 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d80ab430 con 0x7f93f006f7c0 2024-04-17T09:57:20.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.587+0000 7f93ef400640 1 -- 172.21.15.32:0/4224403684 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d80b0070 con 0x7f93f006f7c0 2024-04-17T09:57:20.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.587+0000 7f93ef400640 1 -- 172.21.15.32:0/4224403684 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d80ab7f0 con 0x7f93f006f7c0 2024-04-17T09:57:20.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.587+0000 7f93f7200640 1 -- 172.21.15.32:0/4224403684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93f006f7c0 msgr2=0x7f93f006ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:20.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.587+0000 7f93f7200640 1 --2- 172.21.15.32:0/4224403684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93f006f7c0 0x7f93f006ba30 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f93d80097c0 tx=0x7f93d801c410 comp rx=0 tx=0).stop 2024-04-17T09:57:20.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.587+0000 7f93f7200640 1 -- 172.21.15.32:0/4224403684 shutdown_connections 2024-04-17T09:57:20.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.587+0000 7f93f7200640 1 --2- 172.21.15.32:0/4224403684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f006f270 0x7f93f006bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:20.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.587+0000 7f93f7200640 1 --2- 172.21.15.32:0/4224403684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93f006f7c0 0x7f93f006ba30 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:20.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.587+0000 7f93f7200640 1 -- 172.21.15.32:0/4224403684 >> 172.21.15.32:0/4224403684 conn(0x7f93f00b2120 msgr2=0x7f93f00b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:20.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.587+0000 7f93f7200640 1 -- 172.21.15.32:0/4224403684 shutdown_connections 2024-04-17T09:57:20.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.587+0000 7f93f7200640 1 -- 172.21.15.32:0/4224403684 wait complete. 2024-04-17T09:57:20.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.587+0000 7f93f7200640 1 Processor -- start 2024-04-17T09:57:20.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.587+0000 7f93f7200640 1 -- start start 2024-04-17T09:57:20.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93f7200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93f006f270 0x7f93f0140d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:20.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93f7200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93f006f7c0 0x7f93f014e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:20.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93f7200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f015b7d0 0x7f93f014e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:20.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93f7200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f93f00ceeb0 con 0x7f93f006f270 2024-04-17T09:57:20.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93f7200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f93f00ced30 con 0x7f93f006f7c0 2024-04-17T09:57:20.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93f7200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f93f01682a0 con 0x7f93f015b7d0 2024-04-17T09:57:20.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93efe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93f006f270 0x7f93f0140d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:20.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93efe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93f006f270 0x7f93f0140d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42618/0 (socket says 172.21.15.32:42618) 2024-04-17T09:57:20.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93efe00640 1 -- 172.21.15.32:0/1472879054 learned_addr learned my addr 172.21.15.32:0/1472879054 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:20.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93e7400640 1 --2- 172.21.15.32:0/1472879054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93f006f7c0 0x7f93f014e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:20.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93f5200640 1 --2- 172.21.15.32:0/1472879054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f015b7d0 0x7f93f014e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:20.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93efe00640 1 -- 172.21.15.32:0/1472879054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f015b7d0 msgr2=0x7f93f014e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:20.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93efe00640 1 --2- 172.21.15.32:0/1472879054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f015b7d0 0x7f93f014e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:20.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93efe00640 1 -- 172.21.15.32:0/1472879054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93f006f7c0 msgr2=0x7f93f014e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:20.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93efe00640 1 --2- 172.21.15.32:0/1472879054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93f006f7c0 0x7f93f014e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:20.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93efe00640 1 -- 172.21.15.32:0/1472879054 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f0154770 con 0x7f93f006f270 2024-04-17T09:57:20.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93e7400640 1 --2- 172.21.15.32:0/1472879054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93f006f7c0 0x7f93f014e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:57:20.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93efe00640 1 --2- 172.21.15.32:0/1472879054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93f006f270 0x7f93f0140d20 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f93e0001570 tx=0x7f93e000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:20.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93ecc00640 1 -- 172.21.15.32:0/1472879054 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0018070 con 0x7f93f006f270 2024-04-17T09:57:20.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93ecc00640 1 -- 172.21.15.32:0/1472879054 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93e000a640 con 0x7f93f006f270 2024-04-17T09:57:20.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93ecc00640 1 -- 172.21.15.32:0/1472879054 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e000aa30 con 0x7f93f006f270 2024-04-17T09:57:20.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93f7200640 1 -- 172.21.15.32:0/1472879054 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f0154a50 con 0x7f93f006f270 2024-04-17T09:57:20.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93f7200640 1 -- 172.21.15.32:0/1472879054 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f0155010 con 0x7f93f006f270 2024-04-17T09:57:20.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93f7200640 1 -- 172.21.15.32:0/1472879054 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93f0068a90 con 0x7f93f006f270 2024-04-17T09:57:20.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.591+0000 7f93ecc00640 1 -- 172.21.15.32:0/1472879054 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f93e001b070 con 0x7f93f006f270 2024-04-17T09:57:20.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.595+0000 7f93ecc00640 1 --2- 172.21.15.32:0/1472879054 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f93cc068380 0x7f93cc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:20.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.595+0000 7f93ecc00640 1 -- 172.21.15.32:0/1472879054 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f93e0086150 con 0x7f93f006f270 2024-04-17T09:57:20.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.595+0000 7f93e7400640 1 --2- 172.21.15.32:0/1472879054 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f93cc068380 0x7f93cc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:20.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.595+0000 7f93e7400640 1 --2- 172.21.15.32:0/1472879054 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f93cc068380 0x7f93cc06a840 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f93d80af000 tx=0x7f93d80a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:20.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.599+0000 7f93ecc00640 1 -- 172.21.15.32:0/1472879054 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f93e0056670 con 0x7f93f006f270 2024-04-17T09:57:20.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.763+0000 7f93f7200640 1 -- 172.21.15.32:0/1472879054 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f93f00b47c0 con 0x7f93f006f270 2024-04-17T09:57:20.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.763+0000 7f93ecc00640 1 -- 172.21.15.32:0/1472879054 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) ==== 93+0+10 (secure 0 0 0) 0x7f93e0055f70 con 0x7f93f006f270 2024-04-17T09:57:20.764 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:57:20.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.763+0000 7f93f7200640 1 -- 172.21.15.32:0/1472879054 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f93cc068380 msgr2=0x7f93cc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:20.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.763+0000 7f93f7200640 1 --2- 172.21.15.32:0/1472879054 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f93cc068380 0x7f93cc06a840 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f93d80af000 tx=0x7f93d80a7000 comp rx=0 tx=0).stop 2024-04-17T09:57:20.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.763+0000 7f93f7200640 1 -- 172.21.15.32:0/1472879054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93f006f270 msgr2=0x7f93f0140d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:20.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.763+0000 7f93f7200640 1 --2- 172.21.15.32:0/1472879054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93f006f270 0x7f93f0140d20 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f93e0001570 tx=0x7f93e000cd20 comp rx=0 tx=0).stop 2024-04-17T09:57:20.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.763+0000 7f93f7200640 1 -- 172.21.15.32:0/1472879054 shutdown_connections 2024-04-17T09:57:20.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.763+0000 7f93f7200640 1 --2- 172.21.15.32:0/1472879054 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f93cc068380 0x7f93cc06a840 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:20.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.763+0000 7f93f7200640 1 --2- 172.21.15.32:0/1472879054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f015b7d0 0x7f93f014e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:20.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.763+0000 7f93f7200640 1 --2- 172.21.15.32:0/1472879054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93f006f7c0 0x7f93f014e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:20.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.763+0000 7f93f7200640 1 --2- 172.21.15.32:0/1472879054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93f006f270 0x7f93f0140d20 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:20.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.763+0000 7f93f7200640 1 -- 172.21.15.32:0/1472879054 >> 172.21.15.32:0/1472879054 conn(0x7f93f00b2120 msgr2=0x7f93f0155ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:20.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.763+0000 7f93f7200640 1 -- 172.21.15.32:0/1472879054 shutdown_connections 2024-04-17T09:57:20.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.763+0000 7f93f7200640 1 -- 172.21.15.32:0/1472879054 wait complete. 2024-04-17T09:57:20.785 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:57:20.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.927+0000 7f58ece00640 1 Processor -- start 2024-04-17T09:57:20.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.927+0000 7f58ece00640 1 -- start start 2024-04-17T09:57:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.927+0000 7f58ece00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f58e806ba30 0x7f58e806be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.927+0000 7f58ece00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58e806c350 con 0x7f58e80cf370 2024-04-17T09:57:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.927+0000 7f58ece00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58e80d0130 con 0x7f58e8071e60 2024-04-17T09:57:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.927+0000 7f58ece00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f58e80cfef0 con 0x7f58e806ba30 2024-04-17T09:57:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.927+0000 7f58dd400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f58e80cf370 0x7f58e80cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53368/0 (socket says 172.21.15.32:53368) 2024-04-17T09:57:20.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.927+0000 7f58dd400640 1 -- 172.21.15.32:0/1939714948 learned_addr learned my addr 172.21.15.32:0/1939714948 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:20.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.927+0000 7f58e5e00640 1 --2- 172.21.15.32:0/1939714948 >> v2:172.21.15.32:3300/0 conn(0x7f58e806ba30 0x7f58e806be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:20.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.927+0000 7f58e5400640 1 -- 172.21.15.32:0/1939714948 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1869884894 0 0) 0x7f58e806c350 con 0x7f58e80cf370 2024-04-17T09:57:20.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.927+0000 7f58e5400640 1 -- 172.21.15.32:0/1939714948 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58cc003530 con 0x7f58e80cf370 2024-04-17T09:57:20.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.927+0000 7f58e5e00640 1 -- 172.21.15.32:0/1939714948 >> v1:172.21.15.144:6789/0 conn(0x7f58e8071e60 legacy=0x7f58e8072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:20.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.927+0000 7f58e5e00640 1 -- 172.21.15.32:0/1939714948 >> v1:172.21.15.32:6789/0 conn(0x7f58e80cf370 legacy=0x7f58e80cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:20.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.927+0000 7f58e5e00640 1 -- 172.21.15.32:0/1939714948 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e815d090 con 0x7f58e806ba30 2024-04-17T09:57:20.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.931+0000 7f58e5e00640 1 --2- 172.21.15.32:0/1939714948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58e806ba30 0x7f58e806be10 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f58d8009d90 tx=0x7f58d801c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75d4182d97e1435 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:20.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.931+0000 7f58e5400640 1 -- 172.21.15.32:0/1939714948 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d80b2070 con 0x7f58e806ba30 2024-04-17T09:57:20.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.931+0000 7f58e5400640 1 -- 172.21.15.32:0/1939714948 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d80a5970 con 0x7f58e806ba30 2024-04-17T09:57:20.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.931+0000 7f58e5400640 1 -- 172.21.15.32:0/1939714948 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d80a5d40 con 0x7f58e806ba30 2024-04-17T09:57:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.931+0000 7f58ece00640 1 -- 172.21.15.32:0/1939714948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58e806ba30 msgr2=0x7f58e806be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.931+0000 7f58ece00640 1 --2- 172.21.15.32:0/1939714948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58e806ba30 0x7f58e806be10 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f58d8009d90 tx=0x7f58d801c8b0 comp rx=0 tx=0).stop 2024-04-17T09:57:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.931+0000 7f58ece00640 1 -- 172.21.15.32:0/1939714948 shutdown_connections 2024-04-17T09:57:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.931+0000 7f58ece00640 1 --2- 172.21.15.32:0/1939714948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58e806ba30 0x7f58e806be10 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.931+0000 7f58ece00640 1 -- 172.21.15.32:0/1939714948 >> 172.21.15.32:0/1939714948 conn(0x7f58e80b2120 msgr2=0x7f58e80b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.931+0000 7f58ece00640 1 -- 172.21.15.32:0/1939714948 shutdown_connections 2024-04-17T09:57:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.931+0000 7f58ece00640 1 -- 172.21.15.32:0/1939714948 wait complete. 2024-04-17T09:57:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58ece00640 1 Processor -- start 2024-04-17T09:57:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58ece00640 1 -- start start 2024-04-17T09:57:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58ece00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58e806ba30 0x7f58e813d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58ece00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58e8071e60 0x7f58e813dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58ece00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58e80cf370 0x7f58e8155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58ece00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f58e815db70 con 0x7f58e80cf370 2024-04-17T09:57:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58ece00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f58e815cd00 con 0x7f58e806ba30 2024-04-17T09:57:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58ece00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f58e815dcf0 con 0x7f58e8071e60 2024-04-17T09:57:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58e6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58e80cf370 0x7f58e8155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58dd400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58e8071e60 0x7f58e813dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58e6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58e80cf370 0x7f58e8155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42630/0 (socket says 172.21.15.32:42630) 2024-04-17T09:57:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58e6800640 1 -- 172.21.15.32:0/1538522122 learned_addr learned my addr 172.21.15.32:0/1538522122 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58e5e00640 1 --2- 172.21.15.32:0/1538522122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58e806ba30 0x7f58e813d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58e6800640 1 -- 172.21.15.32:0/1538522122 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58e8071e60 msgr2=0x7f58e813dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58e6800640 1 --2- 172.21.15.32:0/1538522122 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58e8071e60 0x7f58e813dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58e6800640 1 -- 172.21.15.32:0/1538522122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58e806ba30 msgr2=0x7f58e813d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58e6800640 1 --2- 172.21.15.32:0/1538522122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58e806ba30 0x7f58e813d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58e6800640 1 -- 172.21.15.32:0/1538522122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e813e860 con 0x7f58e80cf370 2024-04-17T09:57:20.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58e6800640 1 --2- 172.21.15.32:0/1538522122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58e80cf370 0x7f58e8155230 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f58e000c430 tx=0x7f58e000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58dea00640 1 -- 172.21.15.32:0/1538522122 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0005220 con 0x7f58e80cf370 2024-04-17T09:57:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58dea00640 1 -- 172.21.15.32:0/1538522122 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58e00053c0 con 0x7f58e80cf370 2024-04-17T09:57:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58ece00640 1 -- 172.21.15.32:0/1538522122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e813eb40 con 0x7f58e80cf370 2024-04-17T09:57:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58dea00640 1 -- 172.21.15.32:0/1538522122 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0007270 con 0x7f58e80cf370 2024-04-17T09:57:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.935+0000 7f58ece00640 1 -- 172.21.15.32:0/1538522122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e80ac850 con 0x7f58e80cf370 2024-04-17T09:57:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.939+0000 7f58ece00640 1 -- 172.21.15.32:0/1538522122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e8068a90 con 0x7f58e80cf370 2024-04-17T09:57:20.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.939+0000 7f58dea00640 1 -- 172.21.15.32:0/1538522122 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f58e0019020 con 0x7f58e80cf370 2024-04-17T09:57:20.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.939+0000 7f58dea00640 1 --2- 172.21.15.32:0/1538522122 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f58c4068380 0x7f58c406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:20.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.939+0000 7f58dea00640 1 -- 172.21.15.32:0/1538522122 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f58e0085c30 con 0x7f58e80cf370 2024-04-17T09:57:20.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.939+0000 7f58e5e00640 1 --2- 172.21.15.32:0/1538522122 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f58c4068380 0x7f58c406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:20.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.939+0000 7f58e5e00640 1 --2- 172.21.15.32:0/1538522122 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f58c4068380 0x7f58c406a840 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f58d80af000 tx=0x7f58d80a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:20.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:20.943+0000 7f58dea00640 1 -- 172.21.15.32:0/1538522122 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f58e0055b90 con 0x7f58e80cf370 2024-04-17T09:57:21.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.107+0000 7f58ece00640 1 -- 172.21.15.32:0/1538522122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f58e806c850 con 0x7f58e80cf370 2024-04-17T09:57:21.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.107+0000 7f58dea00640 1 -- 172.21.15.32:0/1538522122 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v68) ==== 104+0+11 (secure 0 0 0) 0x7f58e0059e90 con 0x7f58e80cf370 2024-04-17T09:57:21.109 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:57:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.107+0000 7f58ece00640 1 -- 172.21.15.32:0/1538522122 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f58c4068380 msgr2=0x7f58c406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.107+0000 7f58ece00640 1 --2- 172.21.15.32:0/1538522122 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f58c4068380 0x7f58c406a840 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f58d80af000 tx=0x7f58d80a7070 comp rx=0 tx=0).stop 2024-04-17T09:57:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.107+0000 7f58ece00640 1 -- 172.21.15.32:0/1538522122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58e80cf370 msgr2=0x7f58e8155230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.107+0000 7f58ece00640 1 --2- 172.21.15.32:0/1538522122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58e80cf370 0x7f58e8155230 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f58e000c430 tx=0x7f58e000c8f0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.111+0000 7f58ece00640 1 -- 172.21.15.32:0/1538522122 shutdown_connections 2024-04-17T09:57:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.111+0000 7f58ece00640 1 --2- 172.21.15.32:0/1538522122 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f58c4068380 0x7f58c406a840 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.111+0000 7f58ece00640 1 --2- 172.21.15.32:0/1538522122 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58e8071e60 0x7f58e813dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.111+0000 7f58ece00640 1 --2- 172.21.15.32:0/1538522122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58e80cf370 0x7f58e8155230 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.111+0000 7f58ece00640 1 --2- 172.21.15.32:0/1538522122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58e806ba30 0x7f58e813d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.111+0000 7f58ece00640 1 -- 172.21.15.32:0/1538522122 >> 172.21.15.32:0/1538522122 conn(0x7f58e80b2120 msgr2=0x7f58e815c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.111+0000 7f58ece00640 1 -- 172.21.15.32:0/1538522122 shutdown_connections 2024-04-17T09:57:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.111+0000 7f58ece00640 1 -- 172.21.15.32:0/1538522122 wait complete. 2024-04-17T09:57:21.130 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:57:21.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.271+0000 7fb1e9200640 1 Processor -- start 2024-04-17T09:57:21.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.271+0000 7fb1e9200640 1 -- start start 2024-04-17T09:57:21.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.275+0000 7fb1e9200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb1e40cf370 0x7fb1e40cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:21.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.275+0000 7fb1e9200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1e4071e60 0x7fb1e4072270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:21.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.275+0000 7fb1e9200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1e40d0130 con 0x7fb1e406ba30 2024-04-17T09:57:21.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.275+0000 7fb1e9200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb1e415d090 con 0x7fb1e4071e60 2024-04-17T09:57:21.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.275+0000 7fb1e9200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb1e415d370 con 0x7fb1e40cf370 2024-04-17T09:57:21.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.275+0000 7fb1e2800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1e4071e60 0x7fb1e4072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:21.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.275+0000 7fb1e2800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1e4071e60 0x7fb1e4072270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42638/0 (socket says 172.21.15.32:42638) 2024-04-17T09:57:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.275+0000 7fb1e2800640 1 -- 172.21.15.32:0/877282312 learned_addr learned my addr 172.21.15.32:0/877282312 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.275+0000 7fb1e1400640 1 --2- 172.21.15.32:0/877282312 >> v2:172.21.15.144:3300/0 conn(0x7fb1e40cf370 0x7fb1e40cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.275+0000 7fb1e2800640 1 -- 172.21.15.32:0/877282312 >> v2:172.21.15.144:3300/0 conn(0x7fb1e40cf370 msgr2=0x7fb1e40cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.275+0000 7fb1e2800640 1 --2- 172.21.15.32:0/877282312 >> v2:172.21.15.144:3300/0 conn(0x7fb1e40cf370 0x7fb1e40cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.275+0000 7fb1e2800640 1 -- 172.21.15.32:0/877282312 >> v1:172.21.15.144:6789/0 conn(0x7fb1e406ba30 legacy=0x7fb1e406be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:21.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.275+0000 7fb1e2800640 1 -- 172.21.15.32:0/877282312 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e415d650 con 0x7fb1e4071e60 2024-04-17T09:57:21.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.279+0000 7fb1e2800640 1 --2- 172.21.15.32:0/877282312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1e4071e60 0x7fb1e4072270 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40cc7c0 tx=0x7fb1dc01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426a2863055ba057 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:21.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.279+0000 7fb1e0a00640 1 -- 172.21.15.32:0/877282312 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc0a5890 con 0x7fb1e4071e60 2024-04-17T09:57:21.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.279+0000 7fb1e0a00640 1 -- 172.21.15.32:0/877282312 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1dc0a5a30 con 0x7fb1e4071e60 2024-04-17T09:57:21.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.279+0000 7fb1e0a00640 1 -- 172.21.15.32:0/877282312 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc0a5e00 con 0x7fb1e4071e60 2024-04-17T09:57:21.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.279+0000 7fb1e9200640 1 -- 172.21.15.32:0/877282312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1e4071e60 msgr2=0x7fb1e4072270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:21.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.279+0000 7fb1e9200640 1 --2- 172.21.15.32:0/877282312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1e4071e60 0x7fb1e4072270 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40cc7c0 tx=0x7fb1dc01c9b0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.279+0000 7fb1e9200640 1 -- 172.21.15.32:0/877282312 shutdown_connections 2024-04-17T09:57:21.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.279+0000 7fb1e9200640 1 --2- 172.21.15.32:0/877282312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1e4071e60 0x7fb1e4072270 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.279+0000 7fb1e9200640 1 --2- 172.21.15.32:0/877282312 >> v2:172.21.15.144:3300/0 conn(0x7fb1e40cf370 0x7fb1e40cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.279+0000 7fb1e9200640 1 -- 172.21.15.32:0/877282312 >> 172.21.15.32:0/877282312 conn(0x7fb1e40b2120 msgr2=0x7fb1e40b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:21.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.279+0000 7fb1e9200640 1 -- 172.21.15.32:0/877282312 shutdown_connections 2024-04-17T09:57:21.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.279+0000 7fb1e9200640 1 -- 172.21.15.32:0/877282312 wait complete. 2024-04-17T09:57:21.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.279+0000 7fb1e9200640 1 Processor -- start 2024-04-17T09:57:21.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.279+0000 7fb1e9200640 1 -- start start 2024-04-17T09:57:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1e9200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1e406ba30 0x7fb1e412ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1e9200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1e4071e60 0x7fb1e4129400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1e9200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1e40cf370 0x7fb1e4140410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1e9200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e4146230 con 0x7fb1e406ba30 2024-04-17T09:57:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1e9200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e415dd80 con 0x7fb1e40cf370 2024-04-17T09:57:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1e9200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb1e41463b0 con 0x7fb1e4071e60 2024-04-17T09:57:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1e1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1e406ba30 0x7fb1e412ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1e1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1e406ba30 0x7fb1e412ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42648/0 (socket says 172.21.15.32:42648) 2024-04-17T09:57:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1e1e00640 1 -- 172.21.15.32:0/2710196249 learned_addr learned my addr 172.21.15.32:0/2710196249 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1e2800640 1 --2- 172.21.15.32:0/2710196249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1e40cf370 0x7fb1e4140410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:21.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1e1400640 1 --2- 172.21.15.32:0/2710196249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1e4071e60 0x7fb1e4129400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:21.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1e1e00640 1 -- 172.21.15.32:0/2710196249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1e4071e60 msgr2=0x7fb1e4129400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:21.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1e1e00640 1 --2- 172.21.15.32:0/2710196249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1e4071e60 0x7fb1e4129400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1e1e00640 1 -- 172.21.15.32:0/2710196249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1e40cf370 msgr2=0x7fb1e4140410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:21.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1e1e00640 1 --2- 172.21.15.32:0/2710196249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1e40cf370 0x7fb1e4140410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1e1e00640 1 -- 172.21.15.32:0/2710196249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e4129d00 con 0x7fb1e406ba30 2024-04-17T09:57:21.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1e1e00640 1 --2- 172.21.15.32:0/2710196249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1e406ba30 0x7fb1e412ada0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400c480 tx=0x7fb1d400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:21.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1da000640 1 -- 172.21.15.32:0/2710196249 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4012830 con 0x7fb1e406ba30 2024-04-17T09:57:21.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1da000640 1 -- 172.21.15.32:0/2710196249 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1d4017070 con 0x7fb1e406ba30 2024-04-17T09:57:21.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1da000640 1 -- 172.21.15.32:0/2710196249 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4012c30 con 0x7fb1e406ba30 2024-04-17T09:57:21.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1e9200640 1 -- 172.21.15.32:0/2710196249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e4129fe0 con 0x7fb1e406ba30 2024-04-17T09:57:21.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1e9200640 1 -- 172.21.15.32:0/2710196249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e412a5a0 con 0x7fb1e406ba30 2024-04-17T09:57:21.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1e9200640 1 -- 172.21.15.32:0/2710196249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1e4068a90 con 0x7fb1e406ba30 2024-04-17T09:57:21.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.283+0000 7fb1da000640 1 -- 172.21.15.32:0/2710196249 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb1d401c460 con 0x7fb1e406ba30 2024-04-17T09:57:21.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.287+0000 7fb1da000640 1 --2- 172.21.15.32:0/2710196249 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1c4068380 0x7fb1c406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:21.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.287+0000 7fb1da000640 1 -- 172.21.15.32:0/2710196249 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fb1d4086700 con 0x7fb1e406ba30 2024-04-17T09:57:21.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.287+0000 7fb1e1400640 1 --2- 172.21.15.32:0/2710196249 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1c4068380 0x7fb1c406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:21.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.287+0000 7fb1e1400640 1 --2- 172.21.15.32:0/2710196249 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1c4068380 0x7fb1c406a840 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4140ed0 tx=0x7fb1d0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:21.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.291+0000 7fb1da000640 1 -- 172.21.15.32:0/2710196249 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fb1d4056cb0 con 0x7fb1e406ba30 2024-04-17T09:57:21.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.451+0000 7fb1e9200640 1 -- 172.21.15.32:0/2710196249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb1e406c850 con 0x7fb1e406ba30 2024-04-17T09:57:21.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.451+0000 7fb1da000640 1 -- 172.21.15.32:0/2710196249 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v68) ==== 100+0+11 (secure 0 0 0) 0x7fb1d40565b0 con 0x7fb1e406ba30 2024-04-17T09:57:21.455 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:57:21.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.455+0000 7fb1e9200640 1 -- 172.21.15.32:0/2710196249 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1c4068380 msgr2=0x7fb1c406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:21.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.455+0000 7fb1e9200640 1 --2- 172.21.15.32:0/2710196249 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1c4068380 0x7fb1c406a840 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4140ed0 tx=0x7fb1d0008040 comp rx=0 tx=0).stop 2024-04-17T09:57:21.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.455+0000 7fb1e9200640 1 -- 172.21.15.32:0/2710196249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1e406ba30 msgr2=0x7fb1e412ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:21.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.455+0000 7fb1e9200640 1 --2- 172.21.15.32:0/2710196249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1e406ba30 0x7fb1e412ada0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400c480 tx=0x7fb1d400c940 comp rx=0 tx=0).stop 2024-04-17T09:57:21.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.455+0000 7fb1e9200640 1 -- 172.21.15.32:0/2710196249 shutdown_connections 2024-04-17T09:57:21.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.455+0000 7fb1e9200640 1 --2- 172.21.15.32:0/2710196249 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb1c4068380 0x7fb1c406a840 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.455+0000 7fb1e9200640 1 --2- 172.21.15.32:0/2710196249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb1e4071e60 0x7fb1e4129400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.455+0000 7fb1e9200640 1 --2- 172.21.15.32:0/2710196249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1e40cf370 0x7fb1e4140410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.455+0000 7fb1e9200640 1 --2- 172.21.15.32:0/2710196249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1e406ba30 0x7fb1e412ada0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.455+0000 7fb1e9200640 1 -- 172.21.15.32:0/2710196249 >> 172.21.15.32:0/2710196249 conn(0x7fb1e40b2120 msgr2=0x7fb1e40b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:21.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.455+0000 7fb1e9200640 1 -- 172.21.15.32:0/2710196249 shutdown_connections 2024-04-17T09:57:21.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.455+0000 7fb1e9200640 1 -- 172.21.15.32:0/2710196249 wait complete. 2024-04-17T09:57:21.476 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T09:57:21.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.615+0000 7f5e10200640 1 Processor -- start 2024-04-17T09:57:21.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.615+0000 7f5e10200640 1 -- start start 2024-04-17T09:57:21.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.619+0000 7f5e10200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5e08155a70 0x7f5e0815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:21.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.619+0000 7f5e10200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e08155e50 0x7f5e0815d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:21.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.619+0000 7f5e10200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e0815f870 con 0x7f5e0815d5d0 2024-04-17T09:57:21.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.619+0000 7f5e10200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f5e080b42a0 con 0x7f5e08155a70 2024-04-17T09:57:21.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.619+0000 7f5e10200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5e080d1250 con 0x7f5e08155e50 2024-04-17T09:57:21.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.619+0000 7f5e0ce00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e08155e50 0x7f5e0815d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:21.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.619+0000 7f5e0d800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5e08155a70 0x7f5e0815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:21.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.619+0000 7f5e0e200640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f5e0815d5d0 0x7f5e0815c260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:51460/0 (socket says 172.21.15.32:51460) 2024-04-17T09:57:21.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.619+0000 7f5e0d800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5e08155a70 0x7f5e0815f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:45260/0 (socket says 172.21.15.32:45260) 2024-04-17T09:57:21.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.619+0000 7f5e0e200640 1 -- 172.21.15.32:0/1358165661 learned_addr learned my addr 172.21.15.32:0/1358165661 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:21.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.619+0000 7f5e0ce00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e08155e50 0x7f5e0815d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:48266/0 (socket says 172.21.15.32:48266) 2024-04-17T09:57:21.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.619+0000 7f5e0ce00640 1 -- 172.21.15.32:0/1358165661 >> v2:172.21.15.144:3300/0 conn(0x7f5e08155a70 msgr2=0x7f5e0815f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:21.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.619+0000 7f5e0ce00640 1 --2- 172.21.15.32:0/1358165661 >> v2:172.21.15.144:3300/0 conn(0x7f5e08155a70 0x7f5e0815f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.619+0000 7f5e0ce00640 1 -- 172.21.15.32:0/1358165661 >> v1:172.21.15.144:6789/0 conn(0x7f5e0815d5d0 legacy=0x7f5e0815c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:21.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.619+0000 7f5e0ce00640 1 -- 172.21.15.32:0/1358165661 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0815fb50 con 0x7f5e08155e50 2024-04-17T09:57:21.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.619+0000 7f5e0d800640 1 --2- 172.21.15.32:0/1358165661 >> v2:172.21.15.144:3300/0 conn(0x7f5e08155a70 0x7f5e0815f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:57:21.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.623+0000 7f5e0ce00640 1 --2- 172.21.15.32:0/1358165661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e08155e50 0x7f5e0815d090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5df40097c0 tx=0x7f5df401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de0df346d3891bc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:21.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.623+0000 7f5e07e00640 1 -- 172.21.15.32:0/1358165661 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df40b2070 con 0x7f5e08155e50 2024-04-17T09:57:21.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.623+0000 7f5e07e00640 1 -- 172.21.15.32:0/1358165661 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df40ad640 con 0x7f5e08155e50 2024-04-17T09:57:21.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.623+0000 7f5e07e00640 1 -- 172.21.15.32:0/1358165661 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df40ada10 con 0x7f5e08155e50 2024-04-17T09:57:21.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.623+0000 7f5e10200640 1 -- 172.21.15.32:0/1358165661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e08155e50 msgr2=0x7f5e0815d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:21.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.623+0000 7f5e10200640 1 --2- 172.21.15.32:0/1358165661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e08155e50 0x7f5e0815d090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5df40097c0 tx=0x7f5df401c4a0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.623+0000 7f5e10200640 1 -- 172.21.15.32:0/1358165661 shutdown_connections 2024-04-17T09:57:21.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.623+0000 7f5e10200640 1 --2- 172.21.15.32:0/1358165661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e08155e50 0x7f5e0815d090 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.623+0000 7f5e10200640 1 --2- 172.21.15.32:0/1358165661 >> v2:172.21.15.144:3300/0 conn(0x7f5e08155a70 0x7f5e0815f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.623+0000 7f5e10200640 1 -- 172.21.15.32:0/1358165661 >> 172.21.15.32:0/1358165661 conn(0x7f5e0806ba30 msgr2=0x7f5e0806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:21.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.623+0000 7f5e10200640 1 -- 172.21.15.32:0/1358165661 shutdown_connections 2024-04-17T09:57:21.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.623+0000 7f5e10200640 1 -- 172.21.15.32:0/1358165661 wait complete. 2024-04-17T09:57:21.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.623+0000 7f5e10200640 1 Processor -- start 2024-04-17T09:57:21.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.623+0000 7f5e10200640 1 -- start start 2024-04-17T09:57:21.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.627+0000 7f5e10200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e08155a70 0x7f5e080aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:21.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.627+0000 7f5e10200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e08155e50 0x7f5e08159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:21.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.627+0000 7f5e10200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e0815d5d0 0x7f5e0815a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:21.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.627+0000 7f5e10200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5e080d02f0 con 0x7f5e08155a70 2024-04-17T09:57:21.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.627+0000 7f5e10200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5e080d0170 con 0x7f5e0815d5d0 2024-04-17T09:57:21.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.627+0000 7f5e10200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5e080d0470 con 0x7f5e08155e50 2024-04-17T09:57:21.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.627+0000 7f5e0d800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e08155a70 0x7f5e080aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:21.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.627+0000 7f5e0d800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e08155a70 0x7f5e080aeca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42654/0 (socket says 172.21.15.32:42654) 2024-04-17T09:57:21.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.627+0000 7f5e0d800640 1 -- 172.21.15.32:0/683076010 learned_addr learned my addr 172.21.15.32:0/683076010 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:21.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.627+0000 7f5e0e200640 1 --2- 172.21.15.32:0/683076010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e0815d5d0 0x7f5e0815a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:21.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.627+0000 7f5e0ce00640 1 --2- 172.21.15.32:0/683076010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e08155e50 0x7f5e08159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:21.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.627+0000 7f5e0d800640 1 -- 172.21.15.32:0/683076010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e08155e50 msgr2=0x7f5e08159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:21.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.627+0000 7f5e0d800640 1 --2- 172.21.15.32:0/683076010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e08155e50 0x7f5e08159e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.627+0000 7f5e0d800640 1 -- 172.21.15.32:0/683076010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e0815d5d0 msgr2=0x7f5e0815a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:21.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.627+0000 7f5e0d800640 1 --2- 172.21.15.32:0/683076010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e0815d5d0 0x7f5e0815a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.627+0000 7f5e0d800640 1 -- 172.21.15.32:0/683076010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e080af1e0 con 0x7f5e08155a70 2024-04-17T09:57:21.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.627+0000 7f5e0d800640 1 --2- 172.21.15.32:0/683076010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e08155a70 0x7f5e080aeca0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc010f40 tx=0x7f5dfc00e7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:21.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.627+0000 7f5e05600640 1 -- 172.21.15.32:0/683076010 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc01e070 con 0x7f5e08155a70 2024-04-17T09:57:21.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.627+0000 7f5e05600640 1 -- 172.21.15.32:0/683076010 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dfc011860 con 0x7f5e08155a70 2024-04-17T09:57:21.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.627+0000 7f5e10200640 1 -- 172.21.15.32:0/683076010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e08154fc0 con 0x7f5e08155a70 2024-04-17T09:57:21.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.627+0000 7f5e05600640 1 -- 172.21.15.32:0/683076010 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc011c30 con 0x7f5e08155a70 2024-04-17T09:57:21.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.627+0000 7f5e10200640 1 -- 172.21.15.32:0/683076010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e08155580 con 0x7f5e08155a70 2024-04-17T09:57:21.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.627+0000 7f5e10200640 1 -- 172.21.15.32:0/683076010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e08068a90 con 0x7f5e08155a70 2024-04-17T09:57:21.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.631+0000 7f5e05600640 1 -- 172.21.15.32:0/683076010 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5dfc021070 con 0x7f5e08155a70 2024-04-17T09:57:21.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.631+0000 7f5e05600640 1 --2- 172.21.15.32:0/683076010 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5de8068380 0x7f5de806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:21.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.631+0000 7f5e05600640 1 -- 172.21.15.32:0/683076010 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f5dfc089520 con 0x7f5e08155a70 2024-04-17T09:57:21.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.631+0000 7f5e0ce00640 1 --2- 172.21.15.32:0/683076010 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5de8068380 0x7f5de806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:21.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.631+0000 7f5e0ce00640 1 --2- 172.21.15.32:0/683076010 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5de8068380 0x7f5de806a840 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5df40af000 tx=0x7f5df4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:21.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.635+0000 7f5e05600640 1 -- 172.21.15.32:0/683076010 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5dfc0593f0 con 0x7f5e08155a70 2024-04-17T09:57:21.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.815+0000 7f5e10200640 1 -- 172.21.15.32:0/683076010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5e0806e850 con 0x7f5e08155a70 2024-04-17T09:57:21.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.815+0000 7f5e05600640 1 -- 172.21.15.32:0/683076010 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v68) ==== 80+0+0 (secure 0 0 0) 0x7f5dfc05d6f0 con 0x7f5e08155a70 2024-04-17T09:57:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T09:57:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.815+0000 7f5e10200640 1 -- 172.21.15.32:0/683076010 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5de8068380 msgr2=0x7f5de806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.815+0000 7f5e10200640 1 --2- 172.21.15.32:0/683076010 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5de8068380 0x7f5de806a840 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5df40af000 tx=0x7f5df4008040 comp rx=0 tx=0).stop 2024-04-17T09:57:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.815+0000 7f5e10200640 1 -- 172.21.15.32:0/683076010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e08155a70 msgr2=0x7f5e080aeca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.815+0000 7f5e10200640 1 --2- 172.21.15.32:0/683076010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e08155a70 0x7f5e080aeca0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc010f40 tx=0x7f5dfc00e7f0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.815+0000 7f5e10200640 1 -- 172.21.15.32:0/683076010 shutdown_connections 2024-04-17T09:57:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.815+0000 7f5e10200640 1 --2- 172.21.15.32:0/683076010 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5de8068380 0x7f5de806a840 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.815+0000 7f5e10200640 1 --2- 172.21.15.32:0/683076010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e08155e50 0x7f5e08159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.815+0000 7f5e10200640 1 --2- 172.21.15.32:0/683076010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e0815d5d0 0x7f5e0815a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.815+0000 7f5e10200640 1 --2- 172.21.15.32:0/683076010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e08155a70 0x7f5e080aeca0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.815+0000 7f5e10200640 1 -- 172.21.15.32:0/683076010 >> 172.21.15.32:0/683076010 conn(0x7f5e0806ba30 msgr2=0x7f5e080d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.815+0000 7f5e10200640 1 -- 172.21.15.32:0/683076010 shutdown_connections 2024-04-17T09:57:21.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:21.815+0000 7f5e10200640 1 -- 172.21.15.32:0/683076010 wait complete. 2024-04-17T09:57:21.837 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:57:21.837 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:57:21.846 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:57:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.027+0000 7f0942600640 1 Processor -- start 2024-04-17T09:57:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.027+0000 7f0942600640 1 -- start start 2024-04-17T09:57:22.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.031+0000 7f0942600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f093c15e870 0x7f093c162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:22.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.031+0000 7f0942600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f093c0b0320 0x7f093c0d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:22.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.031+0000 7f0942600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f093c0d0780 0x7f093c06f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:22.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.031+0000 7f0942600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f093c0cf810 con 0x7f093c0b0320 2024-04-17T09:57:22.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.031+0000 7f0942600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f093c067dd0 con 0x7f093c15e870 2024-04-17T09:57:22.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.031+0000 7f0942600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f093c0cfdb0 con 0x7f093c0d0780 2024-04-17T09:57:22.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.031+0000 7f0932a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f093c0b0320 0x7f093c0d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:22.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.031+0000 7f0932a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f093c0b0320 0x7f093c0d0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42658/0 (socket says 172.21.15.32:42658) 2024-04-17T09:57:22.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.031+0000 7f0932a00640 1 -- 172.21.15.32:0/2196464633 learned_addr learned my addr 172.21.15.32:0/2196464633 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:22.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.031+0000 7f093b400640 1 --2- 172.21.15.32:0/2196464633 >> v2:172.21.15.144:3300/0 conn(0x7f093c15e870 0x7f093c162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:22.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.031+0000 7f093be00640 1 --2- 172.21.15.32:0/2196464633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f093c0d0780 0x7f093c06f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:22.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.031+0000 7f0932a00640 1 -- 172.21.15.32:0/2196464633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f093c0d0780 msgr2=0x7f093c06f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:22.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.031+0000 7f0932a00640 1 --2- 172.21.15.32:0/2196464633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f093c0d0780 0x7f093c06f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.031+0000 7f0932a00640 1 -- 172.21.15.32:0/2196464633 >> v2:172.21.15.144:3300/0 conn(0x7f093c15e870 msgr2=0x7f093c162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:22.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.031+0000 7f0932a00640 1 --2- 172.21.15.32:0/2196464633 >> v2:172.21.15.144:3300/0 conn(0x7f093c15e870 0x7f093c162e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.031+0000 7f0932a00640 1 -- 172.21.15.32:0/2196464633 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093c163340 con 0x7f093c0b0320 2024-04-17T09:57:22.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.035+0000 7f0932a00640 1 --2- 172.21.15.32:0/2196464633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f093c0b0320 0x7f093c0d0240 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f093c06e0d0 tx=0x7f092401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b93c40b1609f25c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:22.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.035+0000 7f093aa00640 1 -- 172.21.15.32:0/2196464633 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09240ab5f0 con 0x7f093c0b0320 2024-04-17T09:57:22.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.035+0000 7f093aa00640 1 -- 172.21.15.32:0/2196464633 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09240b0070 con 0x7f093c0b0320 2024-04-17T09:57:22.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.035+0000 7f093aa00640 1 -- 172.21.15.32:0/2196464633 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09240ab9b0 con 0x7f093c0b0320 2024-04-17T09:57:22.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.035+0000 7f0942600640 1 -- 172.21.15.32:0/2196464633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f093c0b0320 msgr2=0x7f093c0d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:22.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.035+0000 7f0942600640 1 --2- 172.21.15.32:0/2196464633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f093c0b0320 0x7f093c0d0240 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f093c06e0d0 tx=0x7f092401c520 comp rx=0 tx=0).stop 2024-04-17T09:57:22.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.035+0000 7f0942600640 1 -- 172.21.15.32:0/2196464633 shutdown_connections 2024-04-17T09:57:22.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.035+0000 7f0942600640 1 --2- 172.21.15.32:0/2196464633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f093c0d0780 0x7f093c06f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.035+0000 7f0942600640 1 --2- 172.21.15.32:0/2196464633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f093c0b0320 0x7f093c0d0240 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.035+0000 7f0942600640 1 --2- 172.21.15.32:0/2196464633 >> v2:172.21.15.144:3300/0 conn(0x7f093c15e870 0x7f093c162e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.035+0000 7f0942600640 1 -- 172.21.15.32:0/2196464633 >> 172.21.15.32:0/2196464633 conn(0x7f093c0d0c00 msgr2=0x7f093c0d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:22.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.035+0000 7f0942600640 1 -- 172.21.15.32:0/2196464633 shutdown_connections 2024-04-17T09:57:22.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.035+0000 7f0942600640 1 -- 172.21.15.32:0/2196464633 wait complete. 2024-04-17T09:57:22.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.035+0000 7f0942600640 1 Processor -- start 2024-04-17T09:57:22.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.035+0000 7f0942600640 1 -- start start 2024-04-17T09:57:22.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f0942600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f093c0b0320 0x7f093c155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:22.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f0942600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f093c15e870 0x7f093c1563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f0942600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f093c0af230 0x7f093c14bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f0942600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f093c06e370 con 0x7f093c15e870 2024-04-17T09:57:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f0942600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f093c0cfa40 con 0x7f093c0b0320 2024-04-17T09:57:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f0942600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f093c06e4f0 con 0x7f093c0af230 2024-04-17T09:57:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f0932a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f093c15e870 0x7f093c1563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f0932a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f093c15e870 0x7f093c1563e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42668/0 (socket says 172.21.15.32:42668) 2024-04-17T09:57:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f0932a00640 1 -- 172.21.15.32:0/2521851207 learned_addr learned my addr 172.21.15.32:0/2521851207 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f093b400640 1 --2- 172.21.15.32:0/2521851207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f093c0b0320 0x7f093c155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f093be00640 1 --2- 172.21.15.32:0/2521851207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f093c0af230 0x7f093c14bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f0932a00640 1 -- 172.21.15.32:0/2521851207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f093c0af230 msgr2=0x7f093c14bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f0932a00640 1 --2- 172.21.15.32:0/2521851207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f093c0af230 0x7f093c14bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f0932a00640 1 -- 172.21.15.32:0/2521851207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f093c0b0320 msgr2=0x7f093c155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f0932a00640 1 --2- 172.21.15.32:0/2521851207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f093c0b0320 0x7f093c155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f0932a00640 1 -- 172.21.15.32:0/2521851207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093c156a20 con 0x7f093c15e870 2024-04-17T09:57:22.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f093b400640 1 --2- 172.21.15.32:0/2521851207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f093c0b0320 0x7f093c155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:57:22.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f0932a00640 1 --2- 172.21.15.32:0/2521851207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f093c15e870 0x7f093c1563e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f092401abd0 tx=0x7f092401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:22.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f0933e00640 1 -- 172.21.15.32:0/2521851207 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09240b4800 con 0x7f093c15e870 2024-04-17T09:57:22.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f0933e00640 1 -- 172.21.15.32:0/2521851207 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09240b49a0 con 0x7f093c15e870 2024-04-17T09:57:22.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f0933e00640 1 -- 172.21.15.32:0/2521851207 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09240b4d90 con 0x7f093c15e870 2024-04-17T09:57:22.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f0942600640 1 -- 172.21.15.32:0/2521851207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f093c156d00 con 0x7f093c15e870 2024-04-17T09:57:22.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f0942600640 1 -- 172.21.15.32:0/2521851207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f093c15c880 con 0x7f093c15e870 2024-04-17T09:57:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f0942600640 1 -- 172.21.15.32:0/2521851207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f093c068a90 con 0x7f093c15e870 2024-04-17T09:57:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.039+0000 7f0933e00640 1 -- 172.21.15.32:0/2521851207 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f09240b3380 con 0x7f093c15e870 2024-04-17T09:57:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.043+0000 7f0933e00640 1 --2- 172.21.15.32:0/2521851207 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0918068380 0x7f091806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.043+0000 7f0933e00640 1 -- 172.21.15.32:0/2521851207 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f0924120be0 con 0x7f093c15e870 2024-04-17T09:57:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.043+0000 7f093b400640 1 --2- 172.21.15.32:0/2521851207 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0918068380 0x7f091806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:22.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.043+0000 7f093b400640 1 --2- 172.21.15.32:0/2521851207 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0918068380 0x7f091806a840 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f092c00b9f0 tx=0x7f092c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:22.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.047+0000 7f0933e00640 1 -- 172.21.15.32:0/2521851207 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0924016110 con 0x7f093c15e870 2024-04-17T09:57:22.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.211+0000 7f0942600640 1 -- 172.21.15.32:0/2521851207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f093c073690 con 0x7f093c15e870 2024-04-17T09:57:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.215+0000 7f0933e00640 1 -- 172.21.15.32:0/2521851207 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16680 (secure 0 0 0) 0x7f09240f0bd0 con 0x7f093c15e870 2024-04-17T09:57:22.216 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:57:22.216 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":68,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:57:13.170280+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:57:09.948964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-17T09:57:10.512460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:57:22.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.215+0000 7f0942600640 1 -- 172.21.15.32:0/2521851207 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0918068380 msgr2=0x7f091806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:22.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.215+0000 7f0942600640 1 --2- 172.21.15.32:0/2521851207 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0918068380 0x7f091806a840 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f092c00b9f0 tx=0x7f092c008040 comp rx=0 tx=0).stop 2024-04-17T09:57:22.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.215+0000 7f0942600640 1 -- 172.21.15.32:0/2521851207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f093c15e870 msgr2=0x7f093c1563e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:22.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.215+0000 7f0942600640 1 --2- 172.21.15.32:0/2521851207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f093c15e870 0x7f093c1563e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f092401abd0 tx=0x7f092401ad60 comp rx=0 tx=0).stop 2024-04-17T09:57:22.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.215+0000 7f0942600640 1 -- 172.21.15.32:0/2521851207 shutdown_connections 2024-04-17T09:57:22.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.215+0000 7f0942600640 1 --2- 172.21.15.32:0/2521851207 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0918068380 0x7f091806a840 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.215+0000 7f0942600640 1 --2- 172.21.15.32:0/2521851207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f093c0af230 0x7f093c14bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.215+0000 7f0942600640 1 --2- 172.21.15.32:0/2521851207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f093c15e870 0x7f093c1563e0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.215+0000 7f0942600640 1 --2- 172.21.15.32:0/2521851207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f093c0b0320 0x7f093c155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.215+0000 7f0942600640 1 -- 172.21.15.32:0/2521851207 >> 172.21.15.32:0/2521851207 conn(0x7f093c0d0c00 msgr2=0x7f093c0b4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:22.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.215+0000 7f0942600640 1 -- 172.21.15.32:0/2521851207 shutdown_connections 2024-04-17T09:57:22.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.215+0000 7f0942600640 1 -- 172.21.15.32:0/2521851207 wait complete. 2024-04-17T09:57:22.237 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:57:09.948964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:57:10.512460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:57:22.237 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:57:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.379+0000 7f4483a00640 1 Processor -- start 2024-04-17T09:57:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.379+0000 7f4483a00640 1 -- start start 2024-04-17T09:57:22.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.383+0000 7f4483a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f447c06d150 0x7f447c0af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:22.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.383+0000 7f4483a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f447c0b0540 con 0x7f447c0afce0 2024-04-17T09:57:22.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.383+0000 7f4483a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f447c0b3ef0 con 0x7f447c141700 2024-04-17T09:57:22.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.383+0000 7f4483a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f447c0b0820 con 0x7f447c06d150 2024-04-17T09:57:22.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.383+0000 7f4481a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f447c0afce0 0x7f447c0b49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53384/0 (socket says 172.21.15.32:53384) 2024-04-17T09:57:22.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.383+0000 7f4481a00640 1 -- 172.21.15.32:0/545153997 learned_addr learned my addr 172.21.15.32:0/545153997 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:22.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.383+0000 7f4473e00640 1 --2- 172.21.15.32:0/545153997 >> v2:172.21.15.144:3300/0 conn(0x7f447c06d150 0x7f447c0af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.383+0000 7f447be00640 1 -- 172.21.15.32:0/545153997 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3899837556 0 0) 0x7f447c0b0540 con 0x7f447c0afce0 2024-04-17T09:57:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.383+0000 7f447be00640 1 -- 172.21.15.32:0/545153997 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4460003530 con 0x7f447c0afce0 2024-04-17T09:57:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.383+0000 7f447be00640 1 -- 172.21.15.32:0/545153997 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2424869800 0 0) 0x7f447c0b3ef0 con 0x7f447c141700 2024-04-17T09:57:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.383+0000 7f447be00640 1 -- 172.21.15.32:0/545153997 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f447c0b0540 con 0x7f447c141700 2024-04-17T09:57:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.383+0000 7f447be00640 1 -- 172.21.15.32:0/545153997 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2433559123 0 0) 0x7f4460003530 con 0x7f447c0afce0 2024-04-17T09:57:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.383+0000 7f4473e00640 1 -- 172.21.15.32:0/545153997 >> v1:172.21.15.144:6789/0 conn(0x7f447c141700 legacy=0x7f447c06ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.383+0000 7f4473e00640 1 -- 172.21.15.32:0/545153997 >> v1:172.21.15.32:6789/0 conn(0x7f447c0afce0 legacy=0x7f447c0b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.383+0000 7f4473e00640 1 -- 172.21.15.32:0/545153997 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c0b41d0 con 0x7f447c06d150 2024-04-17T09:57:22.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.387+0000 7f4473e00640 1 --2- 172.21.15.32:0/545153997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f447c06d150 0x7f447c0af7a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f44640097c0 tx=0x7f446401c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d167bde79e2dc2b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:22.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.387+0000 7f447be00640 1 -- 172.21.15.32:0/545153997 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44640b0070 con 0x7f447c06d150 2024-04-17T09:57:22.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.387+0000 7f447be00640 1 -- 172.21.15.32:0/545153997 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44640ab400 con 0x7f447c06d150 2024-04-17T09:57:22.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.387+0000 7f447be00640 1 -- 172.21.15.32:0/545153997 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447c0b0300 con 0x7f447c06d150 2024-04-17T09:57:22.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.387+0000 7f4483a00640 1 -- 172.21.15.32:0/545153997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f447c06d150 msgr2=0x7f447c0af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.387+0000 7f4483a00640 1 --2- 172.21.15.32:0/545153997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f447c06d150 0x7f447c0af7a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f44640097c0 tx=0x7f446401c390 comp rx=0 tx=0).stop 2024-04-17T09:57:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.387+0000 7f4483a00640 1 -- 172.21.15.32:0/545153997 shutdown_connections 2024-04-17T09:57:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.387+0000 7f4483a00640 1 --2- 172.21.15.32:0/545153997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f447c06d150 0x7f447c0af7a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.387+0000 7f4483a00640 1 -- 172.21.15.32:0/545153997 >> 172.21.15.32:0/545153997 conn(0x7f447c072670 msgr2=0x7f447c072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.387+0000 7f4483a00640 1 -- 172.21.15.32:0/545153997 shutdown_connections 2024-04-17T09:57:22.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.387+0000 7f4483a00640 1 -- 172.21.15.32:0/545153997 wait complete. 2024-04-17T09:57:22.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.387+0000 7f4483a00640 1 Processor -- start 2024-04-17T09:57:22.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.387+0000 7f4483a00640 1 -- start start 2024-04-17T09:57:22.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4483a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f447c06d150 0x7f447c1613a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:22.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4483a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f447c0afce0 0x7f447c15ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:22.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4483a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f447c141700 0x7f447c1588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:22.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4483a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f447c0b58f0 con 0x7f447c0afce0 2024-04-17T09:57:22.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4483a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f447c141d30 con 0x7f447c06d150 2024-04-17T09:57:22.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4483a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f447c0b5a70 con 0x7f447c141700 2024-04-17T09:57:22.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4473e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f447c0afce0 0x7f447c15ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:22.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4473e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f447c0afce0 0x7f447c15ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42674/0 (socket says 172.21.15.32:42674) 2024-04-17T09:57:22.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4473e00640 1 -- 172.21.15.32:0/1655857142 learned_addr learned my addr 172.21.15.32:0/1655857142 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:22.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4481000640 1 --2- 172.21.15.32:0/1655857142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f447c06d150 0x7f447c1613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:22.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4481a00640 1 --2- 172.21.15.32:0/1655857142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f447c141700 0x7f447c1588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:22.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4473e00640 1 -- 172.21.15.32:0/1655857142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f447c141700 msgr2=0x7f447c1588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:22.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4473e00640 1 --2- 172.21.15.32:0/1655857142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f447c141700 0x7f447c1588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4473e00640 1 -- 172.21.15.32:0/1655857142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f447c06d150 msgr2=0x7f447c1613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:22.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4473e00640 1 --2- 172.21.15.32:0/1655857142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f447c06d150 0x7f447c1613a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4473e00640 1 -- 172.21.15.32:0/1655857142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c15ba50 con 0x7f447c0afce0 2024-04-17T09:57:22.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4481000640 1 --2- 172.21.15.32:0/1655857142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f447c06d150 0x7f447c1613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:57:22.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4473e00640 1 --2- 172.21.15.32:0/1655857142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f447c0afce0 0x7f447c15ecd0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f44640ae040 tx=0x7f4464019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:22.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4479600640 1 -- 172.21.15.32:0/1655857142 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44640b8070 con 0x7f447c0afce0 2024-04-17T09:57:22.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4479600640 1 -- 172.21.15.32:0/1655857142 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44640b4410 con 0x7f447c0afce0 2024-04-17T09:57:22.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4479600640 1 -- 172.21.15.32:0/1655857142 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44640b0040 con 0x7f447c0afce0 2024-04-17T09:57:22.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4483a00640 1 -- 172.21.15.32:0/1655857142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447c15f450 con 0x7f447c0afce0 2024-04-17T09:57:22.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4483a00640 1 -- 172.21.15.32:0/1655857142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447c15f9f0 con 0x7f447c0afce0 2024-04-17T09:57:22.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.391+0000 7f4483a00640 1 -- 172.21.15.32:0/1655857142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f447c068a90 con 0x7f447c0afce0 2024-04-17T09:57:22.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.395+0000 7f4479600640 1 -- 172.21.15.32:0/1655857142 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f44640bb070 con 0x7f447c0afce0 2024-04-17T09:57:22.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.395+0000 7f4479600640 1 --2- 172.21.15.32:0/1655857142 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f44580683b0 0x7f445806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:22.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.395+0000 7f4479600640 1 -- 172.21.15.32:0/1655857142 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f44641202c0 con 0x7f447c0afce0 2024-04-17T09:57:22.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.395+0000 7f4481000640 1 --2- 172.21.15.32:0/1655857142 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f44580683b0 0x7f445806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:22.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.395+0000 7f4481000640 1 --2- 172.21.15.32:0/1655857142 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f44580683b0 0x7f445806a870 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f446c001b10 tx=0x7f446c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:22.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.399+0000 7f4479600640 1 -- 172.21.15.32:0/1655857142 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f4464015fb0 con 0x7f447c0afce0 2024-04-17T09:57:22.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.563+0000 7f4483a00640 1 -- 172.21.15.32:0/1655857142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f447c073930 con 0x7f447c0afce0 2024-04-17T09:57:22.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.563+0000 7f4479600640 1 -- 172.21.15.32:0/1655857142 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) ==== 93+0+10 (secure 0 0 0) 0x7f44640f0220 con 0x7f447c0afce0 2024-04-17T09:57:22.566 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:57:22.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.567+0000 7f4483a00640 1 -- 172.21.15.32:0/1655857142 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f44580683b0 msgr2=0x7f445806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:22.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.567+0000 7f4483a00640 1 --2- 172.21.15.32:0/1655857142 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f44580683b0 0x7f445806a870 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f446c001b10 tx=0x7f446c009040 comp rx=0 tx=0).stop 2024-04-17T09:57:22.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.567+0000 7f4483a00640 1 -- 172.21.15.32:0/1655857142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f447c0afce0 msgr2=0x7f447c15ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:22.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.567+0000 7f4483a00640 1 --2- 172.21.15.32:0/1655857142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f447c0afce0 0x7f447c15ecd0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f44640ae040 tx=0x7f4464019130 comp rx=0 tx=0).stop 2024-04-17T09:57:22.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.567+0000 7f4483a00640 1 -- 172.21.15.32:0/1655857142 shutdown_connections 2024-04-17T09:57:22.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.567+0000 7f4483a00640 1 --2- 172.21.15.32:0/1655857142 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f44580683b0 0x7f445806a870 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.567+0000 7f4483a00640 1 --2- 172.21.15.32:0/1655857142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f447c141700 0x7f447c1588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.567+0000 7f4483a00640 1 --2- 172.21.15.32:0/1655857142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f447c0afce0 0x7f447c15ecd0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.567+0000 7f4483a00640 1 --2- 172.21.15.32:0/1655857142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f447c06d150 0x7f447c1613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.567+0000 7f4483a00640 1 -- 172.21.15.32:0/1655857142 >> 172.21.15.32:0/1655857142 conn(0x7f447c072670 msgr2=0x7f447c06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:22.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.567+0000 7f4483a00640 1 -- 172.21.15.32:0/1655857142 shutdown_connections 2024-04-17T09:57:22.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.567+0000 7f4483a00640 1 -- 172.21.15.32:0/1655857142 wait complete. 2024-04-17T09:57:22.587 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:57:22.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.727+0000 7f954b800640 1 Processor -- start 2024-04-17T09:57:22.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.727+0000 7f954b800640 1 -- start start 2024-04-17T09:57:22.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f954b800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f954406c140 0x7f95440ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:22.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f954b800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95440cc940 con 0x7f954415b7d0 2024-04-17T09:57:22.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f954b800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95440ccc20 con 0x7f954406f7c0 2024-04-17T09:57:22.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f954b800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f954406e0b0 con 0x7f954406c140 2024-04-17T09:57:22.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9548e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f954415b7d0 0x7f954406e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53400/0 (socket says 172.21.15.32:53400) 2024-04-17T09:57:22.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9548e00640 1 -- 172.21.15.32:0/2648216539 learned_addr learned my addr 172.21.15.32:0/2648216539 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:22.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9543e00640 1 -- 172.21.15.32:0/2648216539 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2504478262 0 0) 0x7f95440cc940 con 0x7f954415b7d0 2024-04-17T09:57:22.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9549800640 1 --2- 172.21.15.32:0/2648216539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f954406c140 0x7f95440ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:22.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9543e00640 1 -- 172.21.15.32:0/2648216539 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9528003530 con 0x7f954415b7d0 2024-04-17T09:57:22.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9543e00640 1 -- 172.21.15.32:0/2648216539 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1967103377 0 0) 0x7f9528003530 con 0x7f954415b7d0 2024-04-17T09:57:22.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9543e00640 1 -- 172.21.15.32:0/2648216539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f954406c140 msgr2=0x7f95440ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:22.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9543e00640 1 --2- 172.21.15.32:0/2648216539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f954406c140 0x7f95440ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9543e00640 1 -- 172.21.15.32:0/2648216539 >> v1:172.21.15.144:6789/0 conn(0x7f954406f7c0 legacy=0x7f954406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:22.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9543e00640 1 -- 172.21.15.32:0/2648216539 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954406e390 con 0x7f954415b7d0 2024-04-17T09:57:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9543e00640 1 -- 172.21.15.32:0/2648216539 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f9534001680 con 0x7f954415b7d0 2024-04-17T09:57:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9543e00640 1 -- 172.21.15.32:0/2648216539 >> v1:172.21.15.32:6789/0 conn(0x7f954415b7d0 legacy=0x7f954406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9543e00640 1 --2- 172.21.15.32:0/2648216539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f954406c140 0x7f95280039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9543e00640 1 --2- 172.21.15.32:0/2648216539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9528003fc0 0x7f9528004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9543e00640 1 --2- 172.21.15.32:0/2648216539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9528004d50 0x7f95280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9543e00640 1 -- 172.21.15.32:0/2648216539 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f954406e0b0 con 0x7f9528004d50 2024-04-17T09:57:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9543e00640 1 -- 172.21.15.32:0/2648216539 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f952800bbc0 con 0x7f9528003fc0 2024-04-17T09:57:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9543e00640 1 -- 172.21.15.32:0/2648216539 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f952800bdc0 con 0x7f954406c140 2024-04-17T09:57:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9549800640 1 --2- 172.21.15.32:0/2648216539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9528004d50 0x7f95280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9548e00640 1 --2- 172.21.15.32:0/2648216539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f954406c140 0x7f95280039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f953be00640 1 --2- 172.21.15.32:0/2648216539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9528003fc0 0x7f9528004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9549800640 1 -- 172.21.15.32:0/2648216539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f954406c140 msgr2=0x7f95280039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:22.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9549800640 1 --2- 172.21.15.32:0/2648216539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f954406c140 0x7f95280039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9549800640 1 -- 172.21.15.32:0/2648216539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9528003fc0 msgr2=0x7f9528004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:22.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9549800640 1 --2- 172.21.15.32:0/2648216539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9528003fc0 0x7f9528004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.731+0000 7f9549800640 1 -- 172.21.15.32:0/2648216539 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9528003530 con 0x7f9528004d50 2024-04-17T09:57:22.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.735+0000 7f9549800640 1 --2- 172.21.15.32:0/2648216539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9528004d50 0x7f95280051d0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f953c009f90 tx=0x7f953c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc234010ec23c5a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:22.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.735+0000 7f9543e00640 1 -- 172.21.15.32:0/2648216539 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c0b2070 con 0x7f9528004d50 2024-04-17T09:57:22.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.735+0000 7f9543e00640 1 -- 172.21.15.32:0/2648216539 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f953c0a5b40 con 0x7f9528004d50 2024-04-17T09:57:22.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.735+0000 7f9543e00640 1 -- 172.21.15.32:0/2648216539 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c009c10 con 0x7f9528004d50 2024-04-17T09:57:22.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.735+0000 7f954b800640 1 -- 172.21.15.32:0/2648216539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9528004d50 msgr2=0x7f95280051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:22.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.735+0000 7f954b800640 1 --2- 172.21.15.32:0/2648216539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9528004d50 0x7f95280051d0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f953c009f90 tx=0x7f953c01cc40 comp rx=0 tx=0).stop 2024-04-17T09:57:22.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.735+0000 7f9549800640 1 -- 172.21.15.32:0/2648216539 reap_dead start 2024-04-17T09:57:22.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.735+0000 7f954b800640 1 -- 172.21.15.32:0/2648216539 shutdown_connections 2024-04-17T09:57:22.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.735+0000 7f954b800640 1 -- 172.21.15.32:0/2648216539 >> 172.21.15.32:0/2648216539 conn(0x7f95440b2120 msgr2=0x7f95440b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:22.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.735+0000 7f954b800640 1 -- 172.21.15.32:0/2648216539 shutdown_connections 2024-04-17T09:57:22.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.735+0000 7f954b800640 1 -- 172.21.15.32:0/2648216539 wait complete. 2024-04-17T09:57:22.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.735+0000 7f954b800640 1 Processor -- start 2024-04-17T09:57:22.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.735+0000 7f954b800640 1 -- start start 2024-04-17T09:57:22.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.739+0000 7f954b800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95440cbcd0 0x7f95441f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:22.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.739+0000 7f954b800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95440ad650 0x7f95441f3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:22.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.739+0000 7f954b800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9544156a90 0x7f9544156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:22.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.739+0000 7f954b800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f954406f3f0 con 0x7f95440cbcd0 2024-04-17T09:57:22.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.739+0000 7f954b800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f954406f270 con 0x7f9544156a90 2024-04-17T09:57:22.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.739+0000 7f954b800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f954406f570 con 0x7f95440ad650 2024-04-17T09:57:22.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.739+0000 7f9548e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95440cbcd0 0x7f95441f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:22.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.739+0000 7f9548e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95440cbcd0 0x7f95441f5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42686/0 (socket says 172.21.15.32:42686) 2024-04-17T09:57:22.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.739+0000 7f9548e00640 1 -- 172.21.15.32:0/3564133189 learned_addr learned my addr 172.21.15.32:0/3564133189 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:22.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.739+0000 7f9548e00640 1 -- 172.21.15.32:0/3564133189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95440ad650 msgr2=0x7f95441f3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:22.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.739+0000 7f9548e00640 1 --2- 172.21.15.32:0/3564133189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95440ad650 0x7f95441f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.739+0000 7f9548e00640 1 -- 172.21.15.32:0/3564133189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9544156a90 msgr2=0x7f9544156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:22.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.739+0000 7f9548e00640 1 --2- 172.21.15.32:0/3564133189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9544156a90 0x7f9544156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.739+0000 7f9548e00640 1 -- 172.21.15.32:0/3564133189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95441f63c0 con 0x7f95440cbcd0 2024-04-17T09:57:22.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.739+0000 7f9548e00640 1 --2- 172.21.15.32:0/3564133189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95440cbcd0 0x7f95441f5e80 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f9534012800 tx=0x7f9534012cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:22.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.739+0000 7f9541600640 1 -- 172.21.15.32:0/3564133189 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534010940 con 0x7f95440cbcd0 2024-04-17T09:57:22.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.739+0000 7f9541600640 1 -- 172.21.15.32:0/3564133189 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f953401b070 con 0x7f95440cbcd0 2024-04-17T09:57:22.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.739+0000 7f9541600640 1 -- 172.21.15.32:0/3564133189 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534010d20 con 0x7f95440cbcd0 2024-04-17T09:57:22.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.739+0000 7f954b800640 1 -- 172.21.15.32:0/3564133189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95440cc0b0 con 0x7f95440cbcd0 2024-04-17T09:57:22.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.739+0000 7f954b800640 1 -- 172.21.15.32:0/3564133189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95440aded0 con 0x7f95440cbcd0 2024-04-17T09:57:22.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.739+0000 7f954b800640 1 -- 172.21.15.32:0/3564133189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9544068a90 con 0x7f95440cbcd0 2024-04-17T09:57:22.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.739+0000 7f9541600640 1 -- 172.21.15.32:0/3564133189 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9534016480 con 0x7f95440cbcd0 2024-04-17T09:57:22.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.739+0000 7f9541600640 1 --2- 172.21.15.32:0/3564133189 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9520068380 0x7f952006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:22.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.743+0000 7f9541600640 1 -- 172.21.15.32:0/3564133189 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f9534089e80 con 0x7f95440cbcd0 2024-04-17T09:57:22.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.743+0000 7f953be00640 1 --2- 172.21.15.32:0/3564133189 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9520068380 0x7f952006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:22.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.743+0000 7f953be00640 1 --2- 172.21.15.32:0/3564133189 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9520068380 0x7f952006a840 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9544157780 tx=0x7f952c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:22.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.747+0000 7f9541600640 1 -- 172.21.15.32:0/3564133189 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f95441f63c0 con 0x7f95440cbcd0 2024-04-17T09:57:22.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.911+0000 7f954b800640 1 -- 172.21.15.32:0/3564133189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f95441f3580 con 0x7f95440cbcd0 2024-04-17T09:57:22.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.911+0000 7f9541600640 1 -- 172.21.15.32:0/3564133189 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v68) ==== 104+0+11 (secure 0 0 0) 0x7f953405a3a0 con 0x7f95440cbcd0 2024-04-17T09:57:22.912 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:57:22.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.911+0000 7f954b800640 1 -- 172.21.15.32:0/3564133189 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9520068380 msgr2=0x7f952006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:22.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.911+0000 7f954b800640 1 --2- 172.21.15.32:0/3564133189 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9520068380 0x7f952006a840 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9544157780 tx=0x7f952c009040 comp rx=0 tx=0).stop 2024-04-17T09:57:22.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.911+0000 7f954b800640 1 -- 172.21.15.32:0/3564133189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95440cbcd0 msgr2=0x7f95441f5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:22.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.911+0000 7f954b800640 1 --2- 172.21.15.32:0/3564133189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95440cbcd0 0x7f95441f5e80 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f9534012800 tx=0x7f9534012cc0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.911+0000 7f954b800640 1 -- 172.21.15.32:0/3564133189 shutdown_connections 2024-04-17T09:57:22.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.911+0000 7f954b800640 1 --2- 172.21.15.32:0/3564133189 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9520068380 0x7f952006a840 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.911+0000 7f954b800640 1 --2- 172.21.15.32:0/3564133189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95440ad650 0x7f95441f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.911+0000 7f954b800640 1 --2- 172.21.15.32:0/3564133189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9544156a90 0x7f9544156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.911+0000 7f954b800640 1 --2- 172.21.15.32:0/3564133189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95440cbcd0 0x7f95441f5e80 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:22.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.911+0000 7f954b800640 1 -- 172.21.15.32:0/3564133189 >> 172.21.15.32:0/3564133189 conn(0x7f95440b2120 msgr2=0x7f9544159680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:22.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.911+0000 7f954b800640 1 -- 172.21.15.32:0/3564133189 shutdown_connections 2024-04-17T09:57:22.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:22.911+0000 7f954b800640 1 -- 172.21.15.32:0/3564133189 wait complete. 2024-04-17T09:57:22.934 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:57:23.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.075+0000 7fba91800640 1 Processor -- start 2024-04-17T09:57:23.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.075+0000 7fba91800640 1 -- start start 2024-04-17T09:57:23.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.079+0000 7fba91800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fba8c15e880 0x7fba8c0b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:23.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.079+0000 7fba91800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba8c06c140 0x7fba8c0cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:23.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.079+0000 7fba91800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba8c0d0690 con 0x7fba8c0b0a30 2024-04-17T09:57:23.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.079+0000 7fba91800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fba8c0d11d0 con 0x7fba8c15e880 2024-04-17T09:57:23.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.079+0000 7fba91800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fba8c0d0930 con 0x7fba8c06c140 2024-04-17T09:57:23.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.079+0000 7fba8b400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba8c06c140 0x7fba8c0cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:23.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.079+0000 7fba83e00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fba8c0b0a30 0x7fba8c06ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:51502/0 (socket says 172.21.15.32:51502) 2024-04-17T09:57:23.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.079+0000 7fba8aa00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fba8c15e880 0x7fba8c0b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:23.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.079+0000 7fba83e00640 1 -- 172.21.15.32:0/196176732 learned_addr learned my addr 172.21.15.32:0/196176732 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:23.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.079+0000 7fba8a000640 1 -- 172.21.15.32:0/196176732 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1720018189 0 0) 0x7fba8c0d0690 con 0x7fba8c0b0a30 2024-04-17T09:57:23.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.079+0000 7fba8b400640 1 -- 172.21.15.32:0/196176732 >> v2:172.21.15.144:3300/0 conn(0x7fba8c15e880 msgr2=0x7fba8c0b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:23.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.079+0000 7fba8b400640 1 --2- 172.21.15.32:0/196176732 >> v2:172.21.15.144:3300/0 conn(0x7fba8c15e880 0x7fba8c0b04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:23.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.079+0000 7fba8b400640 1 -- 172.21.15.32:0/196176732 >> v1:172.21.15.144:6789/0 conn(0x7fba8c0b0a30 legacy=0x7fba8c06ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:23.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.079+0000 7fba8b400640 1 -- 172.21.15.32:0/196176732 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8c0d0c10 con 0x7fba8c06c140 2024-04-17T09:57:23.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.079+0000 7fba8aa00640 1 --2- 172.21.15.32:0/196176732 >> v2:172.21.15.144:3300/0 conn(0x7fba8c15e880 0x7fba8c0b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:57:23.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.083+0000 7fba8b400640 1 --2- 172.21.15.32:0/196176732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba8c06c140 0x7fba8c0cdda0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fba84009d90 tx=0x7fba8401c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bc315e4da99e7eb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:23.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.083+0000 7fba8a000640 1 -- 172.21.15.32:0/196176732 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba840b2070 con 0x7fba8c06c140 2024-04-17T09:57:23.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.083+0000 7fba8a000640 1 -- 172.21.15.32:0/196176732 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba840a58a0 con 0x7fba8c06c140 2024-04-17T09:57:23.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.083+0000 7fba8a000640 1 -- 172.21.15.32:0/196176732 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba840a5c70 con 0x7fba8c06c140 2024-04-17T09:57:23.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.083+0000 7fba91800640 1 -- 172.21.15.32:0/196176732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba8c06c140 msgr2=0x7fba8c0cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:23.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.083+0000 7fba91800640 1 --2- 172.21.15.32:0/196176732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba8c06c140 0x7fba8c0cdda0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fba84009d90 tx=0x7fba8401c820 comp rx=0 tx=0).stop 2024-04-17T09:57:23.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.083+0000 7fba91800640 1 -- 172.21.15.32:0/196176732 shutdown_connections 2024-04-17T09:57:23.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.083+0000 7fba91800640 1 --2- 172.21.15.32:0/196176732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba8c06c140 0x7fba8c0cdda0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:23.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.083+0000 7fba91800640 1 --2- 172.21.15.32:0/196176732 >> v2:172.21.15.144:3300/0 conn(0x7fba8c15e880 0x7fba8c0b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:23.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.083+0000 7fba91800640 1 -- 172.21.15.32:0/196176732 >> 172.21.15.32:0/196176732 conn(0x7fba8c09a9f0 msgr2=0x7fba8c0b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:23.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.083+0000 7fba91800640 1 -- 172.21.15.32:0/196176732 shutdown_connections 2024-04-17T09:57:23.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.083+0000 7fba91800640 1 -- 172.21.15.32:0/196176732 wait complete. 2024-04-17T09:57:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.083+0000 7fba91800640 1 Processor -- start 2024-04-17T09:57:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.083+0000 7fba91800640 1 -- start start 2024-04-17T09:57:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.083+0000 7fba91800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba8c06c140 0x7fba8c154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.083+0000 7fba91800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba8c0b0a30 0x7fba8c1530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.083+0000 7fba91800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba8c15e880 0x7fba8c1567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.083+0000 7fba91800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c0cbc20 con 0x7fba8c0b0a30 2024-04-17T09:57:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.083+0000 7fba91800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c0cbaa0 con 0x7fba8c15e880 2024-04-17T09:57:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.083+0000 7fba91800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fba8c16b490 con 0x7fba8c06c140 2024-04-17T09:57:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.087+0000 7fba83e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba8c0b0a30 0x7fba8c1530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.087+0000 7fba83e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba8c0b0a30 0x7fba8c1530e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42698/0 (socket says 172.21.15.32:42698) 2024-04-17T09:57:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.087+0000 7fba83e00640 1 -- 172.21.15.32:0/2746240837 learned_addr learned my addr 172.21.15.32:0/2746240837 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.087+0000 7fba8b400640 1 --2- 172.21.15.32:0/2746240837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba8c15e880 0x7fba8c1567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.087+0000 7fba8aa00640 1 --2- 172.21.15.32:0/2746240837 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba8c06c140 0x7fba8c154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:23.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.087+0000 7fba83e00640 1 -- 172.21.15.32:0/2746240837 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba8c06c140 msgr2=0x7fba8c154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:23.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.087+0000 7fba83e00640 1 --2- 172.21.15.32:0/2746240837 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba8c06c140 0x7fba8c154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:23.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.087+0000 7fba83e00640 1 -- 172.21.15.32:0/2746240837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba8c15e880 msgr2=0x7fba8c1567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:23.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.087+0000 7fba83e00640 1 --2- 172.21.15.32:0/2746240837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba8c15e880 0x7fba8c1567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:23.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.087+0000 7fba83e00640 1 -- 172.21.15.32:0/2746240837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8c153e50 con 0x7fba8c0b0a30 2024-04-17T09:57:23.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.087+0000 7fba8b400640 1 --2- 172.21.15.32:0/2746240837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba8c15e880 0x7fba8c1567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:57:23.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.087+0000 7fba83e00640 1 --2- 172.21.15.32:0/2746240837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba8c0b0a30 0x7fba8c1530e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fba7400c580 tx=0x7fba7400ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:23.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.087+0000 7fba82a00640 1 -- 172.21.15.32:0/2746240837 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba74017070 con 0x7fba8c0b0a30 2024-04-17T09:57:23.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.087+0000 7fba82a00640 1 -- 172.21.15.32:0/2746240837 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba74005d10 con 0x7fba8c0b0a30 2024-04-17T09:57:23.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.087+0000 7fba82a00640 1 -- 172.21.15.32:0/2746240837 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba74007330 con 0x7fba8c0b0a30 2024-04-17T09:57:23.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.087+0000 7fba91800640 1 -- 172.21.15.32:0/2746240837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba8c154130 con 0x7fba8c0b0a30 2024-04-17T09:57:23.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.087+0000 7fba91800640 1 -- 172.21.15.32:0/2746240837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba8c1546f0 con 0x7fba8c0b0a30 2024-04-17T09:57:23.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.087+0000 7fba91800640 1 -- 172.21.15.32:0/2746240837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba8c068a90 con 0x7fba8c0b0a30 2024-04-17T09:57:23.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.087+0000 7fba82a00640 1 -- 172.21.15.32:0/2746240837 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fba7401a070 con 0x7fba8c0b0a30 2024-04-17T09:57:23.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.091+0000 7fba82a00640 1 --2- 172.21.15.32:0/2746240837 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fba68068380 0x7fba6806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:23.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.091+0000 7fba82a00640 1 -- 172.21.15.32:0/2746240837 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fba74085ea0 con 0x7fba8c0b0a30 2024-04-17T09:57:23.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.091+0000 7fba8aa00640 1 --2- 172.21.15.32:0/2746240837 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fba68068380 0x7fba6806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:23.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.091+0000 7fba8aa00640 1 --2- 172.21.15.32:0/2746240837 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fba68068380 0x7fba6806a840 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fba7c009580 tx=0x7fba7c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:23.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.095+0000 7fba82a00640 1 -- 172.21.15.32:0/2746240837 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fba74055d80 con 0x7fba8c0b0a30 2024-04-17T09:57:23.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.259+0000 7fba91800640 1 -- 172.21.15.32:0/2746240837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fba8c0cfdb0 con 0x7fba8c0b0a30 2024-04-17T09:57:23.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.259+0000 7fba82a00640 1 -- 172.21.15.32:0/2746240837 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v68) ==== 100+0+11 (secure 0 0 0) 0x7fba7405a080 con 0x7fba8c0b0a30 2024-04-17T09:57:23.262 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:57:23.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.263+0000 7fba91800640 1 -- 172.21.15.32:0/2746240837 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fba68068380 msgr2=0x7fba6806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:23.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.263+0000 7fba91800640 1 --2- 172.21.15.32:0/2746240837 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fba68068380 0x7fba6806a840 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fba7c009580 tx=0x7fba7c009190 comp rx=0 tx=0).stop 2024-04-17T09:57:23.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.263+0000 7fba91800640 1 -- 172.21.15.32:0/2746240837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba8c0b0a30 msgr2=0x7fba8c1530e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:23.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.263+0000 7fba91800640 1 --2- 172.21.15.32:0/2746240837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba8c0b0a30 0x7fba8c1530e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fba7400c580 tx=0x7fba7400ca40 comp rx=0 tx=0).stop 2024-04-17T09:57:23.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.263+0000 7fba91800640 1 -- 172.21.15.32:0/2746240837 shutdown_connections 2024-04-17T09:57:23.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.263+0000 7fba91800640 1 --2- 172.21.15.32:0/2746240837 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fba68068380 0x7fba6806a840 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:23.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.263+0000 7fba91800640 1 --2- 172.21.15.32:0/2746240837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba8c15e880 0x7fba8c1567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:23.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.263+0000 7fba91800640 1 --2- 172.21.15.32:0/2746240837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba8c0b0a30 0x7fba8c1530e0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:23.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.263+0000 7fba91800640 1 --2- 172.21.15.32:0/2746240837 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba8c06c140 0x7fba8c154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:23.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.263+0000 7fba91800640 1 -- 172.21.15.32:0/2746240837 >> 172.21.15.32:0/2746240837 conn(0x7fba8c09a9f0 msgr2=0x7fba8c06ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:23.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.263+0000 7fba91800640 1 -- 172.21.15.32:0/2746240837 shutdown_connections 2024-04-17T09:57:23.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.263+0000 7fba91800640 1 -- 172.21.15.32:0/2746240837 wait complete. 2024-04-17T09:57:23.283 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T09:57:23.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.423+0000 7f989a600640 1 Processor -- start 2024-04-17T09:57:23.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.423+0000 7f989a600640 1 -- start start 2024-04-17T09:57:23.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.423+0000 7f989a600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f98940d09f0 0x7f9894167570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:23.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.423+0000 7f989a600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98940cff60 con 0x7f989413f6e0 2024-04-17T09:57:23.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.423+0000 7f989a600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98941601e0 con 0x7f98940cf370 2024-04-17T09:57:23.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.423+0000 7f989a600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f989413fac0 con 0x7f98940d09f0 2024-04-17T09:57:23.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.423+0000 7f9893e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f98940d09f0 0x7f9894167570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:23.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.423+0000 7f9893400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f989413f6e0 0x7f98940cf850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53414/0 (socket says 172.21.15.32:53414) 2024-04-17T09:57:23.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.423+0000 7f9893400640 1 -- 172.21.15.32:0/2580475408 learned_addr learned my addr 172.21.15.32:0/2580475408 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:23.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.423+0000 7f9893e00640 1 -- 172.21.15.32:0/2580475408 >> v1:172.21.15.144:6789/0 conn(0x7f98940cf370 legacy=0x7f98940d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:23.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.423+0000 7f9893e00640 1 -- 172.21.15.32:0/2580475408 >> v1:172.21.15.32:6789/0 conn(0x7f989413f6e0 legacy=0x7f98940cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:23.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.423+0000 7f9893e00640 1 -- 172.21.15.32:0/2580475408 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9894160480 con 0x7f98940d09f0 2024-04-17T09:57:23.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.427+0000 7f9893e00640 1 --2- 172.21.15.32:0/2580475408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98940d09f0 0x7f9894167570 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f988c009d90 tx=0x7f988c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828f3e160a2f85d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:23.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.427+0000 7f9892a00640 1 -- 172.21.15.32:0/2580475408 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c0b2070 con 0x7f98940d09f0 2024-04-17T09:57:23.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.427+0000 7f9892a00640 1 -- 172.21.15.32:0/2580475408 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f988c0a58e0 con 0x7f98940d09f0 2024-04-17T09:57:23.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.427+0000 7f9892a00640 1 -- 172.21.15.32:0/2580475408 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c0a5cb0 con 0x7f98940d09f0 2024-04-17T09:57:23.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.427+0000 7f989a600640 1 -- 172.21.15.32:0/2580475408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98940d09f0 msgr2=0x7f9894167570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:23.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.427+0000 7f989a600640 1 --2- 172.21.15.32:0/2580475408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98940d09f0 0x7f9894167570 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f988c009d90 tx=0x7f988c01c930 comp rx=0 tx=0).stop 2024-04-17T09:57:23.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.427+0000 7f989a600640 1 -- 172.21.15.32:0/2580475408 shutdown_connections 2024-04-17T09:57:23.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.427+0000 7f989a600640 1 --2- 172.21.15.32:0/2580475408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98940d09f0 0x7f9894167570 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:23.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.427+0000 7f989a600640 1 -- 172.21.15.32:0/2580475408 >> 172.21.15.32:0/2580475408 conn(0x7f989406ba30 msgr2=0x7f989406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:23.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.427+0000 7f989a600640 1 -- 172.21.15.32:0/2580475408 shutdown_connections 2024-04-17T09:57:23.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.427+0000 7f989a600640 1 -- 172.21.15.32:0/2580475408 wait complete. 2024-04-17T09:57:23.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f989a600640 1 Processor -- start 2024-04-17T09:57:23.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f989a600640 1 -- start start 2024-04-17T09:57:23.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f989a600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98940cf370 0x7f989413bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:23.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f989a600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98940d09f0 0x7f9894154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:23.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f989a600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f989413f6e0 0x7f9894155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:23.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f989a600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f989416a6e0 con 0x7f98940cf370 2024-04-17T09:57:23.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f989a600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f989413fd40 con 0x7f989413f6e0 2024-04-17T09:57:23.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f989a600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f989416a860 con 0x7f98940d09f0 2024-04-17T09:57:23.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f9893400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98940cf370 0x7f989413bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f9893400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98940cf370 0x7f989413bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42716/0 (socket says 172.21.15.32:42716) 2024-04-17T09:57:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f9893400640 1 -- 172.21.15.32:0/1646134867 learned_addr learned my addr 172.21.15.32:0/1646134867 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f988aa00640 1 --2- 172.21.15.32:0/1646134867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98940d09f0 0x7f9894154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f9893e00640 1 --2- 172.21.15.32:0/1646134867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f989413f6e0 0x7f9894155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f9893400640 1 -- 172.21.15.32:0/1646134867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98940d09f0 msgr2=0x7f9894154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f9893400640 1 --2- 172.21.15.32:0/1646134867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98940d09f0 0x7f9894154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f9893400640 1 -- 172.21.15.32:0/1646134867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f989413f6e0 msgr2=0x7f9894155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f9893400640 1 --2- 172.21.15.32:0/1646134867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f989413f6e0 0x7f9894155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f9893400640 1 -- 172.21.15.32:0/1646134867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98941563d0 con 0x7f98940cf370 2024-04-17T09:57:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f9893e00640 1 --2- 172.21.15.32:0/1646134867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f989413f6e0 0x7f9894155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:57:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f9893400640 1 --2- 172.21.15.32:0/1646134867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98940cf370 0x7f989413bbc0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f988400c3a0 tx=0x7f988400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f988be00640 1 -- 172.21.15.32:0/1646134867 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884017070 con 0x7f98940cf370 2024-04-17T09:57:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f988be00640 1 -- 172.21.15.32:0/1646134867 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9884005cf0 con 0x7f98940cf370 2024-04-17T09:57:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f988be00640 1 -- 172.21.15.32:0/1646134867 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98840072f0 con 0x7f98940cf370 2024-04-17T09:57:23.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f989a600640 1 -- 172.21.15.32:0/1646134867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9894138a70 con 0x7f98940cf370 2024-04-17T09:57:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.431+0000 7f989a600640 1 -- 172.21.15.32:0/1646134867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9894138790 con 0x7f98940cf370 2024-04-17T09:57:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.435+0000 7f989a600640 1 -- 172.21.15.32:0/1646134867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9894068a90 con 0x7f98940cf370 2024-04-17T09:57:23.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.435+0000 7f988be00640 1 -- 172.21.15.32:0/1646134867 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f988401a070 con 0x7f98940cf370 2024-04-17T09:57:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.435+0000 7f988be00640 1 --2- 172.21.15.32:0/1646134867 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9870068380 0x7f987006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.435+0000 7f988be00640 1 -- 172.21.15.32:0/1646134867 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f9884085d90 con 0x7f98940cf370 2024-04-17T09:57:23.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.435+0000 7f988aa00640 1 --2- 172.21.15.32:0/1646134867 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9870068380 0x7f987006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:23.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.435+0000 7f988aa00640 1 --2- 172.21.15.32:0/1646134867 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9870068380 0x7f987006a840 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f98940d0450 tx=0x7f987c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:23.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.439+0000 7f988be00640 1 -- 172.21.15.32:0/1646134867 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f98840562b0 con 0x7f98940cf370 2024-04-17T09:57:23.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.615+0000 7f989a600640 1 -- 172.21.15.32:0/1646134867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f98940d0480 con 0x7f98940cf370 2024-04-17T09:57:23.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.615+0000 7f988be00640 1 -- 172.21.15.32:0/1646134867 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v68) ==== 80+0+0 (secure 0 0 0) 0x7f9884055bb0 con 0x7f98940cf370 2024-04-17T09:57:23.618 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T09:57:23.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.619+0000 7f989a600640 1 -- 172.21.15.32:0/1646134867 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9870068380 msgr2=0x7f987006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:23.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.619+0000 7f989a600640 1 --2- 172.21.15.32:0/1646134867 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9870068380 0x7f987006a840 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f98940d0450 tx=0x7f987c009040 comp rx=0 tx=0).stop 2024-04-17T09:57:23.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.619+0000 7f989a600640 1 -- 172.21.15.32:0/1646134867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98940cf370 msgr2=0x7f989413bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:23.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.619+0000 7f989a600640 1 --2- 172.21.15.32:0/1646134867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98940cf370 0x7f989413bbc0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f988400c3a0 tx=0x7f988400c860 comp rx=0 tx=0).stop 2024-04-17T09:57:23.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.619+0000 7f989a600640 1 -- 172.21.15.32:0/1646134867 shutdown_connections 2024-04-17T09:57:23.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.619+0000 7f989a600640 1 --2- 172.21.15.32:0/1646134867 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9870068380 0x7f987006a840 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:23.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.619+0000 7f989a600640 1 --2- 172.21.15.32:0/1646134867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98940d09f0 0x7f9894154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:23.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.619+0000 7f989a600640 1 --2- 172.21.15.32:0/1646134867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f989413f6e0 0x7f9894155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:23.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.619+0000 7f989a600640 1 --2- 172.21.15.32:0/1646134867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98940cf370 0x7f989413bbc0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:23.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.619+0000 7f989a600640 1 -- 172.21.15.32:0/1646134867 >> 172.21.15.32:0/1646134867 conn(0x7f989406ba30 msgr2=0x7f989415ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:23.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.619+0000 7f989a600640 1 -- 172.21.15.32:0/1646134867 shutdown_connections 2024-04-17T09:57:23.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:23.619+0000 7f989a600640 1 -- 172.21.15.32:0/1646134867 wait complete. 2024-04-17T09:57:23.640 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2024-04-17T09:57:23.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:23.641 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T09:57:23.659 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[41916]: fuse finished with error 0 and tester_r 0 2024-04-17T09:57:23.662 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:57:29.764 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T09:57:29.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:29.764 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T09:57:29.775 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:57:29.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.971+0000 7f0200200640 1 Processor -- start 2024-04-17T09:57:29.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.971+0000 7f0200200640 1 -- start start 2024-04-17T09:57:29.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.971+0000 7f0200200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01f8144e90 0x7f01f80b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:29.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.971+0000 7f0200200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f01f815c1b0 0x7f01f806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:29.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.971+0000 7f0200200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01f8072bb0 con 0x7f01f80b4de0 2024-04-17T09:57:29.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.971+0000 7f0200200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f01f8145270 con 0x7f01f8144e90 2024-04-17T09:57:29.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.971+0000 7f0200200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f01f80b3dc0 con 0x7f01f815c1b0 2024-04-17T09:57:29.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.971+0000 7f01fd800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01f8144e90 0x7f01f80b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:29.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.971+0000 7f01fd800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01f8144e90 0x7f01f80b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42722/0 (socket says 172.21.15.32:42722) 2024-04-17T09:57:29.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.971+0000 7f01fd800640 1 -- 172.21.15.32:0/801210236 learned_addr learned my addr 172.21.15.32:0/801210236 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:29.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.975+0000 7f01fe200640 1 --2- 172.21.15.32:0/801210236 >> v2:172.21.15.144:3300/0 conn(0x7f01f815c1b0 0x7f01f806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:29.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.975+0000 7f01fd800640 1 -- 172.21.15.32:0/801210236 >> v2:172.21.15.144:3300/0 conn(0x7f01f815c1b0 msgr2=0x7f01f806ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:29.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.975+0000 7f01fd800640 1 --2- 172.21.15.32:0/801210236 >> v2:172.21.15.144:3300/0 conn(0x7f01f815c1b0 0x7f01f806ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:29.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.975+0000 7f01fd800640 1 -- 172.21.15.32:0/801210236 >> v1:172.21.15.144:6789/0 conn(0x7f01f80b4de0 legacy=0x7f01f815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:29.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.975+0000 7f01fd800640 1 -- 172.21.15.32:0/801210236 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f80b40a0 con 0x7f01f8144e90 2024-04-17T09:57:29.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.975+0000 7f01fd800640 1 --2- 172.21.15.32:0/801210236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01f8144e90 0x7f01f80b48a0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f01f80b3a00 tx=0x7f01ec01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d1863c3a422d7c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:29.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.979+0000 7f01f7e00640 1 -- 172.21.15.32:0/801210236 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec0ab600 con 0x7f01f8144e90 2024-04-17T09:57:29.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.979+0000 7f01f7e00640 1 -- 172.21.15.32:0/801210236 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01ec0b0070 con 0x7f01f8144e90 2024-04-17T09:57:29.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.979+0000 7f01f7e00640 1 -- 172.21.15.32:0/801210236 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec0ab9c0 con 0x7f01f8144e90 2024-04-17T09:57:29.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.979+0000 7f0200200640 1 -- 172.21.15.32:0/801210236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01f8144e90 msgr2=0x7f01f80b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:29.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.979+0000 7f0200200640 1 --2- 172.21.15.32:0/801210236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01f8144e90 0x7f01f80b48a0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f01f80b3a00 tx=0x7f01ec01c4f0 comp rx=0 tx=0).stop 2024-04-17T09:57:29.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.979+0000 7f0200200640 1 -- 172.21.15.32:0/801210236 shutdown_connections 2024-04-17T09:57:29.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.979+0000 7f0200200640 1 --2- 172.21.15.32:0/801210236 >> v2:172.21.15.144:3300/0 conn(0x7f01f815c1b0 0x7f01f806ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:29.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.979+0000 7f0200200640 1 --2- 172.21.15.32:0/801210236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01f8144e90 0x7f01f80b48a0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:29.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.979+0000 7f0200200640 1 -- 172.21.15.32:0/801210236 >> 172.21.15.32:0/801210236 conn(0x7f01f806ba30 msgr2=0x7f01f806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:29.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.979+0000 7f0200200640 1 -- 172.21.15.32:0/801210236 shutdown_connections 2024-04-17T09:57:29.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.979+0000 7f0200200640 1 -- 172.21.15.32:0/801210236 wait complete. 2024-04-17T09:57:29.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.979+0000 7f0200200640 1 Processor -- start 2024-04-17T09:57:29.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.979+0000 7f0200200640 1 -- start start 2024-04-17T09:57:29.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.979+0000 7f0200200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01f80b4de0 0x7f01f815de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:29.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.979+0000 7f0200200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01f8144e90 0x7f01f81402a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:29.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.979+0000 7f0200200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01f815c1b0 0x7f01f813e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:29.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.979+0000 7f0200200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f01f8168110 con 0x7f01f815c1b0 2024-04-17T09:57:29.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.979+0000 7f0200200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f01f8072880 con 0x7f01f8144e90 2024-04-17T09:57:29.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.979+0000 7f0200200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f01f8168290 con 0x7f01f80b4de0 2024-04-17T09:57:29.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.979+0000 7f01fe200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01f815c1b0 0x7f01f813e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:29.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.979+0000 7f01fe200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01f815c1b0 0x7f01f813e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42736/0 (socket says 172.21.15.32:42736) 2024-04-17T09:57:29.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.979+0000 7f01fe200640 1 -- 172.21.15.32:0/492850408 learned_addr learned my addr 172.21.15.32:0/492850408 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:29.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.983+0000 7f01fce00640 1 --2- 172.21.15.32:0/492850408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01f8144e90 0x7f01f81402a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:29.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.983+0000 7f01fd800640 1 --2- 172.21.15.32:0/492850408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01f80b4de0 0x7f01f815de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:29.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.983+0000 7f01fe200640 1 -- 172.21.15.32:0/492850408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01f80b4de0 msgr2=0x7f01f815de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:29.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.983+0000 7f01fe200640 1 --2- 172.21.15.32:0/492850408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01f80b4de0 0x7f01f815de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:29.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.983+0000 7f01fe200640 1 -- 172.21.15.32:0/492850408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01f8144e90 msgr2=0x7f01f81402a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:29.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.983+0000 7f01fe200640 1 --2- 172.21.15.32:0/492850408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01f8144e90 0x7f01f81402a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:29.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.983+0000 7f01fe200640 1 -- 172.21.15.32:0/492850408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f815f280 con 0x7f01f815c1b0 2024-04-17T09:57:29.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.983+0000 7f01fd800640 1 --2- 172.21.15.32:0/492850408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01f80b4de0 0x7f01f815de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:57:29.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.983+0000 7f01fce00640 1 --2- 172.21.15.32:0/492850408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01f8144e90 0x7f01f81402a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:57:29.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.983+0000 7f01fe200640 1 --2- 172.21.15.32:0/492850408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01f815c1b0 0x7f01f813e830 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f01f0007e90 tx=0x7f01f000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:29.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.983+0000 7f01f5600640 1 -- 172.21.15.32:0/492850408 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f0014420 con 0x7f01f815c1b0 2024-04-17T09:57:29.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.983+0000 7f01f5600640 1 -- 172.21.15.32:0/492850408 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f0019070 con 0x7f01f815c1b0 2024-04-17T09:57:29.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.983+0000 7f0200200640 1 -- 172.21.15.32:0/492850408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f815f560 con 0x7f01f815c1b0 2024-04-17T09:57:29.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.983+0000 7f01f5600640 1 -- 172.21.15.32:0/492850408 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f0014800 con 0x7f01f815c1b0 2024-04-17T09:57:29.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.983+0000 7f0200200640 1 -- 172.21.15.32:0/492850408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f815e3b0 con 0x7f01f815c1b0 2024-04-17T09:57:29.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.983+0000 7f0200200640 1 -- 172.21.15.32:0/492850408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f815eff0 con 0x7f01f815c1b0 2024-04-17T09:57:29.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.983+0000 7f01f5600640 1 -- 172.21.15.32:0/492850408 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f01f00149a0 con 0x7f01f815c1b0 2024-04-17T09:57:29.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.983+0000 7f01f5600640 1 --2- 172.21.15.32:0/492850408 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f01d80683b0 0x7f01d806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:29.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.983+0000 7f01f5600640 1 -- 172.21.15.32:0/492850408 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f01f00871f0 con 0x7f01f815c1b0 2024-04-17T09:57:29.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.983+0000 7f01fd800640 1 --2- 172.21.15.32:0/492850408 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f01d80683b0 0x7f01d806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:29.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.987+0000 7f01fd800640 1 --2- 172.21.15.32:0/492850408 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f01d80683b0 0x7f01d806a870 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0af000 tx=0x7f01ec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:29.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:29.987+0000 7f01f5600640 1 -- 172.21.15.32:0/492850408 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f01f0011260 con 0x7f01f815c1b0 2024-04-17T09:57:30.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.187+0000 7f0200200640 1 -- 172.21.15.32:0/492850408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01f8068a90 con 0x7f01f815c1b0 2024-04-17T09:57:30.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.187+0000 7f01f5600640 1 -- 172.21.15.32:0/492850408 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5201 (secure 0 0 0) 0x7f01f00577a0 con 0x7f01f815c1b0 2024-04-17T09:57:30.188 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:57:30.189 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2424282223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2424282223},{"type":"v1","addr":"172.21.15.144:6834","nonce":2424282223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3537056692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3537056692},{"type":"v1","addr":"172.21.15.144:6835","nonce":3537056692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5891,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/725333724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":725333724},{"type":"v1","addr":"172.21.15.32:6837","nonce":725333724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:57:11.558183+0000","modified":"2024-04-17T09:57:14.573355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6786},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6786":{"gid":6786,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6836/663537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":663537244},{"type":"v1","addr":"172.21.15.32:6836","nonce":663537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6786,"qdb_cluster":[6786]},"id":5}]} 2024-04-17T09:57:30.189 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 42 2024-04-17T09:57:30.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.187+0000 7f0200200640 1 -- 172.21.15.32:0/492850408 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f01d80683b0 msgr2=0x7f01d806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.187+0000 7f0200200640 1 --2- 172.21.15.32:0/492850408 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f01d80683b0 0x7f01d806a870 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0af000 tx=0x7f01ec008040 comp rx=0 tx=0).stop 2024-04-17T09:57:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.187+0000 7f0200200640 1 -- 172.21.15.32:0/492850408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01f815c1b0 msgr2=0x7f01f813e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.187+0000 7f0200200640 1 --2- 172.21.15.32:0/492850408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01f815c1b0 0x7f01f813e830 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f01f0007e90 tx=0x7f01f000e4b0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.187+0000 7f0200200640 1 -- 172.21.15.32:0/492850408 shutdown_connections 2024-04-17T09:57:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.187+0000 7f0200200640 1 --2- 172.21.15.32:0/492850408 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f01d80683b0 0x7f01d806a870 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.187+0000 7f0200200640 1 --2- 172.21.15.32:0/492850408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01f815c1b0 0x7f01f813e830 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.187+0000 7f0200200640 1 --2- 172.21.15.32:0/492850408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01f8144e90 0x7f01f81402a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.187+0000 7f0200200640 1 --2- 172.21.15.32:0/492850408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01f80b4de0 0x7f01f815de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.187+0000 7f0200200640 1 -- 172.21.15.32:0/492850408 >> 172.21.15.32:0/492850408 conn(0x7f01f806ba30 msgr2=0x7f01f8144160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.187+0000 7f0200200640 1 -- 172.21.15.32:0/492850408 shutdown_connections 2024-04-17T09:57:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.187+0000 7f0200200640 1 -- 172.21.15.32:0/492850408 wait complete. 2024-04-17T09:57:30.211 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-17T09:57:30.384 INFO:teuthology.orchestra.run.smithi032.stdout:{"message":"","return_code":0} 2024-04-17T09:57:30.385 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-17T09:57:30.385 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T09:57:30.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.523+0000 7f2c21e00640 1 Processor -- start 2024-04-17T09:57:30.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.523+0000 7f2c21e00640 1 -- start start 2024-04-17T09:57:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.527+0000 7f2c21e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c1c155a70 0x7f2c1c15f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.527+0000 7f2c21e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2c1c155e50 0x7f2c1c15d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.527+0000 7f2c21e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2c1c15d5d0 0x7f2c1c15c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.527+0000 7f2c21e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2c1c0b42a0 con 0x7f2c1c155e50 2024-04-17T09:57:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.527+0000 7f2c21e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f2c1c0d1250 con 0x7f2c1c15d5d0 2024-04-17T09:57:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.527+0000 7f2c21e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2c1c160110 con 0x7f2c1c155a70 2024-04-17T09:57:30.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.527+0000 7f2c12000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2c1c155e50 0x7f2c1c15d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:30.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.527+0000 7f2c12000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2c1c155e50 0x7f2c1c15d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59374/0 (socket says 172.21.15.32:59374) 2024-04-17T09:57:30.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.527+0000 7f2c12000640 1 -- 172.21.15.32:0/841762597 learned_addr learned my addr 172.21.15.32:0/841762597 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:30.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.527+0000 7f2c1aa00640 1 --2- 172.21.15.32:0/841762597 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c1c155a70 0x7f2c1c15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:30.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.527+0000 7f2c1b400640 1 --2- 172.21.15.32:0/841762597 >> v2:172.21.15.144:3300/0 conn(0x7f2c1c15d5d0 0x7f2c1c15c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:30.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.527+0000 7f2c12000640 1 -- 172.21.15.32:0/841762597 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c1c155a70 msgr2=0x7f2c1c15f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:30.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.527+0000 7f2c12000640 1 --2- 172.21.15.32:0/841762597 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c1c155a70 0x7f2c1c15f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.527+0000 7f2c12000640 1 -- 172.21.15.32:0/841762597 >> v2:172.21.15.144:3300/0 conn(0x7f2c1c15d5d0 msgr2=0x7f2c1c15c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:30.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.527+0000 7f2c12000640 1 --2- 172.21.15.32:0/841762597 >> v2:172.21.15.144:3300/0 conn(0x7f2c1c15d5d0 0x7f2c1c15c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.527+0000 7f2c12000640 1 -- 172.21.15.32:0/841762597 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1c15f870 con 0x7f2c1c155e50 2024-04-17T09:57:30.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.531+0000 7f2c12000640 1 --2- 172.21.15.32:0/841762597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c1c155e50 0x7f2c1c15d090 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c160290 tx=0x7f2c0401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c7fa230acfe8ca server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:30.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.531+0000 7f2c1a000640 1 -- 172.21.15.32:0/841762597 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c040aa610 con 0x7f2c1c155e50 2024-04-17T09:57:30.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.531+0000 7f2c1a000640 1 -- 172.21.15.32:0/841762597 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c040af070 con 0x7f2c1c155e50 2024-04-17T09:57:30.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.531+0000 7f2c1a000640 1 -- 172.21.15.32:0/841762597 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c040aa9d0 con 0x7f2c1c155e50 2024-04-17T09:57:30.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.531+0000 7f2c21e00640 1 -- 172.21.15.32:0/841762597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c1c155e50 msgr2=0x7f2c1c15d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:30.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.531+0000 7f2c21e00640 1 --2- 172.21.15.32:0/841762597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c1c155e50 0x7f2c1c15d090 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c160290 tx=0x7f2c0401c4f0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.531+0000 7f2c21e00640 1 -- 172.21.15.32:0/841762597 shutdown_connections 2024-04-17T09:57:30.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.531+0000 7f2c21e00640 1 --2- 172.21.15.32:0/841762597 >> v2:172.21.15.144:3300/0 conn(0x7f2c1c15d5d0 0x7f2c1c15c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.531+0000 7f2c21e00640 1 --2- 172.21.15.32:0/841762597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c1c155e50 0x7f2c1c15d090 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.531+0000 7f2c21e00640 1 --2- 172.21.15.32:0/841762597 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c1c155a70 0x7f2c1c15f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.531+0000 7f2c21e00640 1 -- 172.21.15.32:0/841762597 >> 172.21.15.32:0/841762597 conn(0x7f2c1c06ba30 msgr2=0x7f2c1c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:30.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.531+0000 7f2c21e00640 1 -- 172.21.15.32:0/841762597 shutdown_connections 2024-04-17T09:57:30.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.531+0000 7f2c21e00640 1 -- 172.21.15.32:0/841762597 wait complete. 2024-04-17T09:57:30.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.531+0000 7f2c21e00640 1 Processor -- start 2024-04-17T09:57:30.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.531+0000 7f2c21e00640 1 -- start start 2024-04-17T09:57:30.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c21e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c1c155a70 0x7f2c1c1f6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:30.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c21e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c1c155e50 0x7f2c1c15dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:30.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c21e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c1c15d5d0 0x7f2c1c14e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:30.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c21e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c0b5320 con 0x7f2c1c155e50 2024-04-17T09:57:30.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c21e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c0b51a0 con 0x7f2c1c15d5d0 2024-04-17T09:57:30.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c21e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2c1c0d0170 con 0x7f2c1c155a70 2024-04-17T09:57:30.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c12000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c1c155e50 0x7f2c1c15dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:30.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c12000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c1c155e50 0x7f2c1c15dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59386/0 (socket says 172.21.15.32:59386) 2024-04-17T09:57:30.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c12000640 1 -- 172.21.15.32:0/1587931873 learned_addr learned my addr 172.21.15.32:0/1587931873 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:30.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c1aa00640 1 --2- 172.21.15.32:0/1587931873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c1c155a70 0x7f2c1c1f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:30.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c1b400640 1 --2- 172.21.15.32:0/1587931873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c1c15d5d0 0x7f2c1c14e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:30.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c12000640 1 -- 172.21.15.32:0/1587931873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c1c155a70 msgr2=0x7f2c1c1f6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:30.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c12000640 1 --2- 172.21.15.32:0/1587931873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c1c155a70 0x7f2c1c1f6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c12000640 1 -- 172.21.15.32:0/1587931873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c1c15d5d0 msgr2=0x7f2c1c14e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:30.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c12000640 1 --2- 172.21.15.32:0/1587931873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c1c15d5d0 0x7f2c1c14e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c12000640 1 -- 172.21.15.32:0/1587931873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1c157800 con 0x7f2c1c155e50 2024-04-17T09:57:30.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c12000640 1 --2- 172.21.15.32:0/1587931873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c1c155e50 0x7f2c1c15dd70 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2c040ac040 tx=0x7f2c0401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:30.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c13e00640 1 -- 172.21.15.32:0/1587931873 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c040bd5f0 con 0x7f2c1c155e50 2024-04-17T09:57:30.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c13e00640 1 -- 172.21.15.32:0/1587931873 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c040bd790 con 0x7f2c1c155e50 2024-04-17T09:57:30.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c21e00640 1 -- 172.21.15.32:0/1587931873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c1c157ae0 con 0x7f2c1c155e50 2024-04-17T09:57:30.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c13e00640 1 -- 172.21.15.32:0/1587931873 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c040bdb80 con 0x7f2c1c155e50 2024-04-17T09:57:30.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c21e00640 1 -- 172.21.15.32:0/1587931873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c1c1580a0 con 0x7f2c1c155e50 2024-04-17T09:57:30.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c21e00640 1 -- 172.21.15.32:0/1587931873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c1c068a90 con 0x7f2c1c155e50 2024-04-17T09:57:30.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.535+0000 7f2c13e00640 1 -- 172.21.15.32:0/1587931873 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2c040b33f0 con 0x7f2c1c155e50 2024-04-17T09:57:30.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.539+0000 7f2c13e00640 1 --2- 172.21.15.32:0/1587931873 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2bf8068380 0x7f2bf806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:30.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.539+0000 7f2c13e00640 1 -- 172.21.15.32:0/1587931873 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f2c041214e0 con 0x7f2c1c155e50 2024-04-17T09:57:30.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.539+0000 7f2c1aa00640 1 --2- 172.21.15.32:0/1587931873 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2bf8068380 0x7f2bf806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:30.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.539+0000 7f2c1aa00640 1 --2- 172.21.15.32:0/1587931873 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2bf8068380 0x7f2bf806a840 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c1f8bb0 tx=0x7f2c0c0098d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:30.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.543+0000 7f2c13e00640 1 -- 172.21.15.32:0/1587931873 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2c040c3060 con 0x7f2c1c155e50 2024-04-17T09:57:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.739+0000 7f2c21e00640 1 -- 172.21.15.32:0/1587931873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2c1c15afb0 con 0x7f2c1c155e50 2024-04-17T09:57:30.743 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-17T09:57:30.743+0000 7f93c9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T09:57:30.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.743+0000 7f2c13e00640 1 -- 172.21.15.32:0/1587931873 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) ==== 161+0+0 (secure 0 0 0) 0x7f2c040a7070 con 0x7f2c1c155e50 2024-04-17T09:57:30.745 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T09:57:30.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.747+0000 7f2c21e00640 1 -- 172.21.15.32:0/1587931873 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2bf8068380 msgr2=0x7f2bf806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:30.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.747+0000 7f2c21e00640 1 --2- 172.21.15.32:0/1587931873 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2bf8068380 0x7f2bf806a840 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c1f8bb0 tx=0x7f2c0c0098d0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.747+0000 7f2c21e00640 1 -- 172.21.15.32:0/1587931873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c1c155e50 msgr2=0x7f2c1c15dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:30.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.747+0000 7f2c21e00640 1 --2- 172.21.15.32:0/1587931873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c1c155e50 0x7f2c1c15dd70 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2c040ac040 tx=0x7f2c0401ad30 comp rx=0 tx=0).stop 2024-04-17T09:57:30.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.747+0000 7f2c21e00640 1 -- 172.21.15.32:0/1587931873 shutdown_connections 2024-04-17T09:57:30.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.747+0000 7f2c21e00640 1 --2- 172.21.15.32:0/1587931873 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2bf8068380 0x7f2bf806a840 secure :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c1f8bb0 tx=0x7f2c0c0098d0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.747+0000 7f2c21e00640 1 --2- 172.21.15.32:0/1587931873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c1c15d5d0 0x7f2c1c14e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.747+0000 7f2c21e00640 1 --2- 172.21.15.32:0/1587931873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c1c155e50 0x7f2c1c15dd70 secure :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2c040ac040 tx=0x7f2c0401ad30 comp rx=0 tx=0).stop 2024-04-17T09:57:30.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.747+0000 7f2c21e00640 1 --2- 172.21.15.32:0/1587931873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c1c155a70 0x7f2c1c1f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.747+0000 7f2c21e00640 1 -- 172.21.15.32:0/1587931873 >> 172.21.15.32:0/1587931873 conn(0x7f2c1c06ba30 msgr2=0x7f2c1c155100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:30.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.747+0000 7f2c21e00640 1 -- 172.21.15.32:0/1587931873 shutdown_connections 2024-04-17T09:57:30.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.747+0000 7f2c21e00640 1 -- 172.21.15.32:0/1587931873 wait complete. 2024-04-17T09:57:30.755 INFO:tasks.ceph.mds.a.smithi032.stderr: -3668> 2024-04-17T09:57:14.135+0000 7f99a4600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T09:57:30.769 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-17T09:57:30.806 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-17T09:57:30.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.815+0000 7fa543608080 1 Processor -- start 2024-04-17T09:57:30.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.815+0000 7fa543608080 1 -- start start 2024-04-17T09:57:30.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa543608080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56337d8d8230 0x56337d95fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:30.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa543608080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56337d8d8610 0x56337d9fa200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:30.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa543608080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56337d965270 0x56337d9fa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:30.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa543608080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56337d8bfa70 con 0x56337d8d8230 2024-04-17T09:57:30.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa543608080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56337d8db5f0 con 0x56337d8d8610 2024-04-17T09:57:30.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa543608080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x56337d8bfbf0 con 0x56337d965270 2024-04-17T09:57:30.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa540800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56337d8d8230 0x56337d95fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:30.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa540800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56337d8d8230 0x56337d95fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59402/0 (socket says 172.21.15.32:59402) 2024-04-17T09:57:30.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa540800640 1 -- 172.21.15.32:0/3932419971 learned_addr learned my addr 172.21.15.32:0/3932419971 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:30.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa540800640 1 -- 172.21.15.32:0/3932419971 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56337d965270 msgr2=0x56337d9fa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:30.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa540800640 1 --2- 172.21.15.32:0/3932419971 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56337d965270 0x56337d9fa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa53fe00640 1 --2- 172.21.15.32:0/3932419971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56337d8d8610 0x56337d9fa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:30.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa540800640 1 -- 172.21.15.32:0/3932419971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56337d8d8610 msgr2=0x56337d9fa200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:30.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa540800640 1 --2- 172.21.15.32:0/3932419971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56337d8d8610 0x56337d9fa200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa540800640 1 -- 172.21.15.32:0/3932419971 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56337d7dd940 con 0x56337d8d8230 2024-04-17T09:57:30.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa53fe00640 1 --2- 172.21.15.32:0/3932419971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56337d8d8610 0x56337d9fa200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:57:30.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa540800640 1 --2- 172.21.15.32:0/3932419971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56337d8d8230 0x56337d95fdf0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fa5340099b0 tx=0x7fa534005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9765aed2d11e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:30.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa53ea00640 1 -- 172.21.15.32:0/3932419971 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa534019070 con 0x56337d8d8230 2024-04-17T09:57:30.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa53ea00640 1 -- 172.21.15.32:0/3932419971 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa534014480 con 0x56337d8d8230 2024-04-17T09:57:30.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa53ea00640 1 -- 172.21.15.32:0/3932419971 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa534008e60 con 0x56337d8d8230 2024-04-17T09:57:30.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa543608080 1 -- 172.21.15.32:0/3932419971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56337d8d8230 msgr2=0x56337d95fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:30.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa543608080 1 --2- 172.21.15.32:0/3932419971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56337d8d8230 0x56337d95fdf0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fa5340099b0 tx=0x7fa534005a10 comp rx=0 tx=0).stop 2024-04-17T09:57:30.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa543608080 1 -- 172.21.15.32:0/3932419971 shutdown_connections 2024-04-17T09:57:30.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa543608080 1 --2- 172.21.15.32:0/3932419971 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56337d965270 0x56337d9fa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa543608080 1 --2- 172.21.15.32:0/3932419971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56337d8d8610 0x56337d9fa200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa543608080 1 --2- 172.21.15.32:0/3932419971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56337d8d8230 0x56337d95fdf0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa543608080 1 -- 172.21.15.32:0/3932419971 >> 172.21.15.32:0/3932419971 conn(0x56337d88e620 msgr2=0x56337d963e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa543608080 1 -- 172.21.15.32:0/3932419971 shutdown_connections 2024-04-17T09:57:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa543608080 1 -- 172.21.15.32:0/3932419971 wait complete. 2024-04-17T09:57:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa543608080 1 Processor -- start 2024-04-17T09:57:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa543608080 1 -- start start 2024-04-17T09:57:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa543608080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56337d8d8230 0x56337d9f8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa543608080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56337d8d8610 0x56337da00f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa543608080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56337d965270 0x56337d95a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa543608080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56337d9624c0 con 0x56337d965270 2024-04-17T09:57:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa543608080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56337d962220 con 0x56337d8d8610 2024-04-17T09:57:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.819+0000 7fa543608080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x56337d9f6de0 con 0x56337d8d8230 2024-04-17T09:57:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.823+0000 7fa541200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56337d965270 0x56337d95a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.823+0000 7fa541200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56337d965270 0x56337d95a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59412/0 (socket says 172.21.15.32:59412) 2024-04-17T09:57:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.823+0000 7fa541200640 1 -- 172.21.15.32:0/3468537663 learned_addr learned my addr 172.21.15.32:0/3468537663 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:30.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.823+0000 7fa53fe00640 1 --2- 172.21.15.32:0/3468537663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56337d8d8610 0x56337da00f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:30.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.823+0000 7fa540800640 1 --2- 172.21.15.32:0/3468537663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56337d8d8230 0x56337d9f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:30.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.823+0000 7fa541200640 1 -- 172.21.15.32:0/3468537663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56337d8d8230 msgr2=0x56337d9f8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:30.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.823+0000 7fa541200640 1 --2- 172.21.15.32:0/3468537663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56337d8d8230 0x56337d9f8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.823+0000 7fa541200640 1 -- 172.21.15.32:0/3468537663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56337d8d8610 msgr2=0x56337da00f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:30.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.823+0000 7fa541200640 1 --2- 172.21.15.32:0/3468537663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56337d8d8610 0x56337da00f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.823+0000 7fa541200640 1 -- 172.21.15.32:0/3468537663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56337d7b9950 con 0x56337d965270 2024-04-17T09:57:30.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.823+0000 7fa541200640 1 --2- 172.21.15.32:0/3468537663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56337d965270 0x56337d95a420 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fa53800d7f0 tx=0x7fa53800dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:30.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.823+0000 7fa53d600640 1 -- 172.21.15.32:0/3468537663 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa538016020 con 0x56337d965270 2024-04-17T09:57:30.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.823+0000 7fa53d600640 1 -- 172.21.15.32:0/3468537663 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa53800a6a0 con 0x56337d965270 2024-04-17T09:57:30.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.823+0000 7fa543608080 1 -- 172.21.15.32:0/3468537663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56337da00050 con 0x56337d965270 2024-04-17T09:57:30.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.823+0000 7fa53d600640 1 -- 172.21.15.32:0/3468537663 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53800aa70 con 0x56337d965270 2024-04-17T09:57:30.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.823+0000 7fa543608080 1 -- 172.21.15.32:0/3468537663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56337da00590 con 0x56337d965270 2024-04-17T09:57:30.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.823+0000 7fa53d600640 1 -- 172.21.15.32:0/3468537663 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa538019020 con 0x56337d965270 2024-04-17T09:57:30.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.823+0000 7fa53d600640 1 --2- 172.21.15.32:0/3468537663 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa520066d90 0x7fa520069250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:30.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.823+0000 7fa53d600640 1 -- 172.21.15.32:0/3468537663 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7fa538085820 con 0x56337d965270 2024-04-17T09:57:30.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.827+0000 7fa540800640 1 --2- 172.21.15.32:0/3468537663 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa520066d90 0x7fa520069250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:30.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.827+0000 7fa543608080 1 --2- 172.21.15.32:0/3468537663 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x56337d9f8ec0 0x56337d959d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:30.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.827+0000 7fa543608080 1 -- 172.21.15.32:0/3468537663 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69) -- 0x56337da014b0 con 0x56337d9f8ec0 2024-04-17T09:57:30.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.827+0000 7fa540800640 1 --2- 172.21.15.32:0/3468537663 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa520066d90 0x7fa520069250 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa534017040 tx=0x7fa534009f90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:30.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.827+0000 7fa53fe00640 1 --2- 172.21.15.32:0/3468537663 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x56337d9f8ec0 0x56337d959d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:30.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.827+0000 7fa53fe00640 1 --2- 172.21.15.32:0/3468537663 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x56337d9f8ec0 0x56337d959d80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:30.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.831+0000 7fa53fe00640 1 -- 172.21.15.32:0/3468537663 <== osd.4 v2:172.21.15.144:6809/4076371964 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v69'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa52c0aa620 con 0x56337d9f8ec0 2024-04-17T09:57:30.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.831+0000 7fa543608080 1 -- 172.21.15.32:0/3468537663 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x56337d9f8ec0 msgr2=0x56337d959d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:30.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.831+0000 7fa543608080 1 --2- 172.21.15.32:0/3468537663 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x56337d9f8ec0 0x56337d959d80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.831+0000 7fa543608080 1 -- 172.21.15.32:0/3468537663 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa520066d90 msgr2=0x7fa520069250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:30.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.831+0000 7fa543608080 1 --2- 172.21.15.32:0/3468537663 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa520066d90 0x7fa520069250 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa534017040 tx=0x7fa534009f90 comp rx=0 tx=0).stop 2024-04-17T09:57:30.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.831+0000 7fa543608080 1 -- 172.21.15.32:0/3468537663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56337d965270 msgr2=0x56337d95a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:30.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.831+0000 7fa543608080 1 --2- 172.21.15.32:0/3468537663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56337d965270 0x56337d95a420 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fa53800d7f0 tx=0x7fa53800dcb0 comp rx=0 tx=0).stop 2024-04-17T09:57:30.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.831+0000 7fa541200640 1 -- 172.21.15.32:0/3468537663 reap_dead start 2024-04-17T09:57:30.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.831+0000 7fa543608080 1 -- 172.21.15.32:0/3468537663 shutdown_connections 2024-04-17T09:57:30.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.831+0000 7fa543608080 1 -- 172.21.15.32:0/3468537663 >> 172.21.15.32:0/3468537663 conn(0x56337d88e620 msgr2=0x56337d95fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:30.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.831+0000 7fa543608080 1 -- 172.21.15.32:0/3468537663 shutdown_connections 2024-04-17T09:57:30.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:30.831+0000 7fa543608080 1 -- 172.21.15.32:0/3468537663 wait complete. 2024-04-17T09:57:30.837 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-17T09:57:31.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.019+0000 7f7457c00640 1 Processor -- start 2024-04-17T09:57:31.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.019+0000 7f7457c00640 1 -- start start 2024-04-17T09:57:31.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.023+0000 7f7457c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f745006ba30 0x7f745006be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:31.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.023+0000 7f7457c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74500cf370 0x7f74500cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:31.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.023+0000 7f7457c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f745006c350 con 0x7f7450160350 2024-04-17T09:57:31.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.023+0000 7f7457c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f74500d0130 con 0x7f745006ba30 2024-04-17T09:57:31.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.023+0000 7f7457c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f745015d090 con 0x7f74500cf370 2024-04-17T09:57:31.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.023+0000 7f7455200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f745006ba30 0x7f745006be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:31.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.023+0000 7f7455c00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7450160350 0x7f74501607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53054/0 (socket says 172.21.15.32:53054) 2024-04-17T09:57:31.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.023+0000 7f7455200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f745006ba30 0x7f745006be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59418/0 (socket says 172.21.15.32:59418) 2024-04-17T09:57:31.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.023+0000 7f7455c00640 1 -- 172.21.15.32:0/4189812373 learned_addr learned my addr 172.21.15.32:0/4189812373 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:31.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.023+0000 7f744fe00640 1 --2- 172.21.15.32:0/4189812373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74500cf370 0x7f74500cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:31.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.023+0000 7f7455200640 1 -- 172.21.15.32:0/4189812373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74500cf370 msgr2=0x7f74500cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:31.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.023+0000 7f7455200640 1 --2- 172.21.15.32:0/4189812373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74500cf370 0x7f74500cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:31.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.023+0000 7f744f400640 1 -- 172.21.15.32:0/4189812373 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1528078000 0 0) 0x7f745006c350 con 0x7f7450160350 2024-04-17T09:57:31.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.023+0000 7f7455200640 1 -- 172.21.15.32:0/4189812373 >> v1:172.21.15.32:6789/0 conn(0x7f7450160350 legacy=0x7f74501607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:31.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.023+0000 7f7455200640 1 -- 172.21.15.32:0/4189812373 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745015d370 con 0x7f745006ba30 2024-04-17T09:57:31.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.027+0000 7f7455200640 1 --2- 172.21.15.32:0/4189812373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f745006ba30 0x7f745006be10 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f74440097c0 tx=0x7f744401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a77bd288c691611 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:31.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.027+0000 7f744f400640 1 -- 172.21.15.32:0/4189812373 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74440b1070 con 0x7f745006ba30 2024-04-17T09:57:31.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.027+0000 7f744f400640 1 -- 172.21.15.32:0/4189812373 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74440ac410 con 0x7f745006ba30 2024-04-17T09:57:31.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.027+0000 7f744f400640 1 -- 172.21.15.32:0/4189812373 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74440ac7e0 con 0x7f745006ba30 2024-04-17T09:57:31.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.027+0000 7f7457c00640 1 -- 172.21.15.32:0/4189812373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f745006ba30 msgr2=0x7f745006be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:31.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.027+0000 7f7457c00640 1 --2- 172.21.15.32:0/4189812373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f745006ba30 0x7f745006be10 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f74440097c0 tx=0x7f744401c4a0 comp rx=0 tx=0).stop 2024-04-17T09:57:31.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.027+0000 7f7457c00640 1 -- 172.21.15.32:0/4189812373 shutdown_connections 2024-04-17T09:57:31.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.027+0000 7f7457c00640 1 --2- 172.21.15.32:0/4189812373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74500cf370 0x7f74500cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:31.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.027+0000 7f7457c00640 1 --2- 172.21.15.32:0/4189812373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f745006ba30 0x7f745006be10 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:31.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.027+0000 7f7457c00640 1 -- 172.21.15.32:0/4189812373 >> 172.21.15.32:0/4189812373 conn(0x7f74500b2120 msgr2=0x7f74500b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:31.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.027+0000 7f7457c00640 1 -- 172.21.15.32:0/4189812373 shutdown_connections 2024-04-17T09:57:31.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.027+0000 7f7457c00640 1 -- 172.21.15.32:0/4189812373 wait complete. 2024-04-17T09:57:31.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.027+0000 7f7457c00640 1 Processor -- start 2024-04-17T09:57:31.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.027+0000 7f7457c00640 1 -- start start 2024-04-17T09:57:31.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.031+0000 7f7457c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f745006ba30 0x7f7450144160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:31.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.031+0000 7f7457c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74500cf370 0x7f745015e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:31.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.031+0000 7f7457c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7450160350 0x7f74501431a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:31.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.031+0000 7f7457c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f745015dcf0 con 0x7f745006ba30 2024-04-17T09:57:31.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.031+0000 7f7457c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f745015db70 con 0x7f7450160350 2024-04-17T09:57:31.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.031+0000 7f7457c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7450146230 con 0x7f74500cf370 2024-04-17T09:57:31.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.031+0000 7f7455200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f745006ba30 0x7f7450144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:31.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.031+0000 7f7455200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f745006ba30 0x7f7450144160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59428/0 (socket says 172.21.15.32:59428) 2024-04-17T09:57:31.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.031+0000 7f7455200640 1 -- 172.21.15.32:0/32170264 learned_addr learned my addr 172.21.15.32:0/32170264 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:31.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.031+0000 7f7455200640 1 -- 172.21.15.32:0/32170264 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74500cf370 msgr2=0x7f745015e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:31.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.031+0000 7f7455200640 1 --2- 172.21.15.32:0/32170264 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74500cf370 0x7f745015e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:31.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.031+0000 7f7455200640 1 -- 172.21.15.32:0/32170264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7450160350 msgr2=0x7f74501431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:31.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.031+0000 7f7455200640 1 --2- 172.21.15.32:0/32170264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7450160350 0x7f74501431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:31.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.031+0000 7f7455200640 1 -- 172.21.15.32:0/32170264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7450144a60 con 0x7f745006ba30 2024-04-17T09:57:31.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.031+0000 7f7455200640 1 --2- 172.21.15.32:0/32170264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f745006ba30 0x7f7450144160 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f74440097c0 tx=0x7f744401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:31.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.031+0000 7f744cc00640 1 -- 172.21.15.32:0/32170264 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74440b1050 con 0x7f745006ba30 2024-04-17T09:57:31.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.031+0000 7f744cc00640 1 -- 172.21.15.32:0/32170264 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74440b2040 con 0x7f745006ba30 2024-04-17T09:57:31.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.031+0000 7f7457c00640 1 -- 172.21.15.32:0/32170264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7450144d40 con 0x7f745006ba30 2024-04-17T09:57:31.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.031+0000 7f744cc00640 1 -- 172.21.15.32:0/32170264 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7444006e20 con 0x7f745006ba30 2024-04-17T09:57:31.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.031+0000 7f7457c00640 1 -- 172.21.15.32:0/32170264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74501429a0 con 0x7f745006ba30 2024-04-17T09:57:31.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.031+0000 7f7457c00640 1 -- 172.21.15.32:0/32170264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7450068a90 con 0x7f745006ba30 2024-04-17T09:57:31.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.031+0000 7f744cc00640 1 -- 172.21.15.32:0/32170264 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f74440bf060 con 0x7f745006ba30 2024-04-17T09:57:31.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.035+0000 7f744cc00640 1 --2- 172.21.15.32:0/32170264 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7430068380 0x7f743006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:31.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.035+0000 7f744cc00640 1 -- 172.21.15.32:0/32170264 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7f7444120210 con 0x7f745006ba30 2024-04-17T09:57:31.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.035+0000 7f744fe00640 1 --2- 172.21.15.32:0/32170264 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7430068380 0x7f743006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:31.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.035+0000 7f744fe00640 1 --2- 172.21.15.32:0/32170264 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7430068380 0x7f743006a840 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f743c006aa0 tx=0x7f743c006a30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:31.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.039+0000 7f744cc00640 1 -- 172.21.15.32:0/32170264 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f7444016090 con 0x7f745006ba30 2024-04-17T09:57:31.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.235+0000 7f7457c00640 1 -- 172.21.15.32:0/32170264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f745006e150 con 0x7f745006ba30 2024-04-17T09:57:31.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.747+0000 7f744cc00640 1 -- 172.21.15.32:0/32170264 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v44) ==== 162+0+0 (secure 0 0 0) 0x7f74440f0760 con 0x7f745006ba30 2024-04-17T09:57:31.750 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-17T09:57:31.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.751+0000 7f7457c00640 1 -- 172.21.15.32:0/32170264 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7430068380 msgr2=0x7f743006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:31.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.751+0000 7f7457c00640 1 --2- 172.21.15.32:0/32170264 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7430068380 0x7f743006a840 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f743c006aa0 tx=0x7f743c006a30 comp rx=0 tx=0).stop 2024-04-17T09:57:31.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.751+0000 7f7457c00640 1 -- 172.21.15.32:0/32170264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f745006ba30 msgr2=0x7f7450144160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:31.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.751+0000 7f7457c00640 1 --2- 172.21.15.32:0/32170264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f745006ba30 0x7f7450144160 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f74440097c0 tx=0x7f744401ace0 comp rx=0 tx=0).stop 2024-04-17T09:57:31.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.751+0000 7f7457c00640 1 -- 172.21.15.32:0/32170264 shutdown_connections 2024-04-17T09:57:31.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.751+0000 7f7457c00640 1 --2- 172.21.15.32:0/32170264 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7430068380 0x7f743006a840 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:31.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.751+0000 7f7457c00640 1 --2- 172.21.15.32:0/32170264 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74500cf370 0x7f745015e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:31.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.751+0000 7f7457c00640 1 --2- 172.21.15.32:0/32170264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7450160350 0x7f74501431a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:31.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.751+0000 7f7457c00640 1 --2- 172.21.15.32:0/32170264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f745006ba30 0x7f7450144160 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:31.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.751+0000 7f7457c00640 1 -- 172.21.15.32:0/32170264 >> 172.21.15.32:0/32170264 conn(0x7f74500b2120 msgr2=0x7f74500ccfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:31.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.751+0000 7f7457c00640 1 -- 172.21.15.32:0/32170264 shutdown_connections 2024-04-17T09:57:31.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.751+0000 7f7457c00640 1 -- 172.21.15.32:0/32170264 wait complete. 2024-04-17T09:57:31.778 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:57:31.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.919+0000 7f40d9e00640 1 Processor -- start 2024-04-17T09:57:31.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.919+0000 7f40d9e00640 1 -- start start 2024-04-17T09:57:31.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.919+0000 7f40d9e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40d4144f20 0x7f40d406e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:31.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.919+0000 7f40d9e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40d41481e0 0x7f40d40b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:31.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.919+0000 7f40d9e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40d40aed20 con 0x7f40d406eda0 2024-04-17T09:57:31.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.919+0000 7f40d9e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f40d40af000 con 0x7f40d4144f20 2024-04-17T09:57:31.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.919+0000 7f40d9e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f40d40af2e0 con 0x7f40d41481e0 2024-04-17T09:57:31.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.919+0000 7f40d2a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40d4144f20 0x7f40d406e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:31.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.919+0000 7f40cbe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f40d406eda0 0x7f40d4071df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53056/0 (socket says 172.21.15.32:53056) 2024-04-17T09:57:31.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.919+0000 7f40cbe00640 1 -- 172.21.15.32:0/1310116462 learned_addr learned my addr 172.21.15.32:0/1310116462 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:31.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.919+0000 7f40d3400640 1 --2- 172.21.15.32:0/1310116462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40d41481e0 0x7f40d40b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:31.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.919+0000 7f40d2a00640 1 -- 172.21.15.32:0/1310116462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40d41481e0 msgr2=0x7f40d40b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:31.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.919+0000 7f40d2a00640 1 --2- 172.21.15.32:0/1310116462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40d41481e0 0x7f40d40b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:31.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.919+0000 7f40d2a00640 1 -- 172.21.15.32:0/1310116462 >> v1:172.21.15.32:6789/0 conn(0x7f40d406eda0 legacy=0x7f40d4071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:31.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.919+0000 7f40d2000640 1 -- 172.21.15.32:0/1310116462 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2492482175 0 0) 0x7f40d40aed20 con 0x7f40d406eda0 2024-04-17T09:57:31.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.919+0000 7f40d2a00640 1 -- 172.21.15.32:0/1310116462 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d40af5c0 con 0x7f40d4144f20 2024-04-17T09:57:31.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.923+0000 7f40d2a00640 1 --2- 172.21.15.32:0/1310116462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40d4144f20 0x7f40d406e860 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f40c40097c0 tx=0x7f40c401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6182d6b8249b76 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:31.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.923+0000 7f40d2000640 1 -- 172.21.15.32:0/1310116462 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c40b0070 con 0x7f40d4144f20 2024-04-17T09:57:31.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.923+0000 7f40d2000640 1 -- 172.21.15.32:0/1310116462 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40c40ab600 con 0x7f40d4144f20 2024-04-17T09:57:31.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.923+0000 7f40d2000640 1 -- 172.21.15.32:0/1310116462 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c40ab9d0 con 0x7f40d4144f20 2024-04-17T09:57:31.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.923+0000 7f40d9e00640 1 -- 172.21.15.32:0/1310116462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40d4144f20 msgr2=0x7f40d406e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:31.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.923+0000 7f40d9e00640 1 --2- 172.21.15.32:0/1310116462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40d4144f20 0x7f40d406e860 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f40c40097c0 tx=0x7f40c401c4a0 comp rx=0 tx=0).stop 2024-04-17T09:57:31.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.923+0000 7f40d9e00640 1 -- 172.21.15.32:0/1310116462 shutdown_connections 2024-04-17T09:57:31.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.923+0000 7f40d9e00640 1 --2- 172.21.15.32:0/1310116462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40d41481e0 0x7f40d40b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:31.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.923+0000 7f40d9e00640 1 --2- 172.21.15.32:0/1310116462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40d4144f20 0x7f40d406e860 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:31.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.923+0000 7f40d9e00640 1 -- 172.21.15.32:0/1310116462 >> 172.21.15.32:0/1310116462 conn(0x7f40d406ba30 msgr2=0x7f40d406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:31.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.923+0000 7f40d9e00640 1 -- 172.21.15.32:0/1310116462 shutdown_connections 2024-04-17T09:57:31.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.923+0000 7f40d9e00640 1 -- 172.21.15.32:0/1310116462 wait complete. 2024-04-17T09:57:31.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40d9e00640 1 Processor -- start 2024-04-17T09:57:31.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40d9e00640 1 -- start start 2024-04-17T09:57:31.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40d9e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40d406eda0 0x7f40d413cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:31.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40d9e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40d4144f20 0x7f40d413f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:31.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40d9e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40d41481e0 0x7f40d4145430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:31.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40d9e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f40d40b41e0 con 0x7f40d4144f20 2024-04-17T09:57:31.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40d9e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f40d40b4360 con 0x7f40d406eda0 2024-04-17T09:57:31.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40d9e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f40d416b5d0 con 0x7f40d41481e0 2024-04-17T09:57:31.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40cbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40d4144f20 0x7f40d413f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:31.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40cbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40d4144f20 0x7f40d413f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59446/0 (socket says 172.21.15.32:59446) 2024-04-17T09:57:31.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40cbe00640 1 -- 172.21.15.32:0/2324459285 learned_addr learned my addr 172.21.15.32:0/2324459285 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:31.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40d2a00640 1 --2- 172.21.15.32:0/2324459285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40d406eda0 0x7f40d413cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:31.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40d3400640 1 --2- 172.21.15.32:0/2324459285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40d41481e0 0x7f40d4145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:31.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40cbe00640 1 -- 172.21.15.32:0/2324459285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40d41481e0 msgr2=0x7f40d4145430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:31.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40cbe00640 1 --2- 172.21.15.32:0/2324459285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40d41481e0 0x7f40d4145430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:31.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40cbe00640 1 -- 172.21.15.32:0/2324459285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40d406eda0 msgr2=0x7f40d413cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:31.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40cbe00640 1 --2- 172.21.15.32:0/2324459285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40d406eda0 0x7f40d413cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:31.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40cbe00640 1 -- 172.21.15.32:0/2324459285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d415dab0 con 0x7f40d4144f20 2024-04-17T09:57:31.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40cbe00640 1 --2- 172.21.15.32:0/2324459285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40d4144f20 0x7f40d413f5e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0011f0 tx=0x7f40bc00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:31.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40cb400640 1 -- 172.21.15.32:0/2324459285 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc018070 con 0x7f40d4144f20 2024-04-17T09:57:31.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40cb400640 1 -- 172.21.15.32:0/2324459285 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40bc0074e0 con 0x7f40d4144f20 2024-04-17T09:57:31.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40cb400640 1 -- 172.21.15.32:0/2324459285 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc004a10 con 0x7f40d4144f20 2024-04-17T09:57:31.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40d9e00640 1 -- 172.21.15.32:0/2324459285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d415dd90 con 0x7f40d4144f20 2024-04-17T09:57:31.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40d9e00640 1 -- 172.21.15.32:0/2324459285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d40cd080 con 0x7f40d4144f20 2024-04-17T09:57:31.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.927+0000 7f40d9e00640 1 -- 172.21.15.32:0/2324459285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40d4068a90 con 0x7f40d4144f20 2024-04-17T09:57:31.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.931+0000 7f40cb400640 1 -- 172.21.15.32:0/2324459285 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f40bc004bb0 con 0x7f40d4144f20 2024-04-17T09:57:31.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.931+0000 7f40cb400640 1 --2- 172.21.15.32:0/2324459285 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40b0068380 0x7f40b006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:31.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.931+0000 7f40cb400640 1 -- 172.21.15.32:0/2324459285 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7f40bc085d90 con 0x7f40d4144f20 2024-04-17T09:57:31.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.931+0000 7f40d2a00640 1 --2- 172.21.15.32:0/2324459285 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40b0068380 0x7f40b006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:31.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.931+0000 7f40d2a00640 1 --2- 172.21.15.32:0/2324459285 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40b0068380 0x7f40b006a840 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f40c40ad000 tx=0x7f40c40a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:31.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:31.935+0000 7f40cb400640 1 -- 172.21.15.32:0/2324459285 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f40bc056250 con 0x7f40d4144f20 2024-04-17T09:57:32.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:32.131+0000 7f40d9e00640 1 -- 172.21.15.32:0/2324459285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40d40b4860 con 0x7f40d4144f20 2024-04-17T09:57:32.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:32.131+0000 7f40cb400640 1 -- 172.21.15.32:0/2324459285 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5198 (secure 0 0 0) 0x7f40bc055b50 con 0x7f40d4144f20 2024-04-17T09:57:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:57:32.135 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2424282223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2424282223},{"type":"v1","addr":"172.21.15.144:6834","nonce":2424282223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3537056692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3537056692},{"type":"v1","addr":"172.21.15.144:6835","nonce":3537056692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5891,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/725333724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":725333724},{"type":"v1","addr":"172.21.15.32:6837","nonce":725333724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:57:11.558183+0000","modified":"2024-04-17T09:57:31.756285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"a","rank":0,"incarnation":45,"state":"up:replay","state_seq":1,"addr":"172.21.15.32:6836/2416784965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2416784965},{"type":"v1","addr":"172.21.15.32:6836","nonce":2416784965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-04-17T09:57:32.135 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 45 2024-04-17T09:57:32.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:32.135+0000 7f40d9e00640 1 -- 172.21.15.32:0/2324459285 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40b0068380 msgr2=0x7f40b006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:32.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:32.135+0000 7f40d9e00640 1 --2- 172.21.15.32:0/2324459285 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40b0068380 0x7f40b006a840 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f40c40ad000 tx=0x7f40c40a6000 comp rx=0 tx=0).stop 2024-04-17T09:57:32.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:32.135+0000 7f40d9e00640 1 -- 172.21.15.32:0/2324459285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40d4144f20 msgr2=0x7f40d413f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:32.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:32.135+0000 7f40d9e00640 1 --2- 172.21.15.32:0/2324459285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40d4144f20 0x7f40d413f5e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0011f0 tx=0x7f40bc00c9b0 comp rx=0 tx=0).stop 2024-04-17T09:57:32.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:32.135+0000 7f40d9e00640 1 -- 172.21.15.32:0/2324459285 shutdown_connections 2024-04-17T09:57:32.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:32.135+0000 7f40d9e00640 1 --2- 172.21.15.32:0/2324459285 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40b0068380 0x7f40b006a840 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:32.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:32.135+0000 7f40d9e00640 1 --2- 172.21.15.32:0/2324459285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40d41481e0 0x7f40d4145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:32.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:32.135+0000 7f40d9e00640 1 --2- 172.21.15.32:0/2324459285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40d4144f20 0x7f40d413f5e0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:32.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:32.135+0000 7f40d9e00640 1 --2- 172.21.15.32:0/2324459285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40d406eda0 0x7f40d413cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:32.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:32.135+0000 7f40d9e00640 1 -- 172.21.15.32:0/2324459285 >> 172.21.15.32:0/2324459285 conn(0x7f40d406ba30 msgr2=0x7f40d4071df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:32.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:32.135+0000 7f40d9e00640 1 -- 172.21.15.32:0/2324459285 shutdown_connections 2024-04-17T09:57:32.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:32.135+0000 7f40d9e00640 1 -- 172.21.15.32:0/2324459285 wait complete. 2024-04-17T09:57:32.559 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T09:57:11.558183+0000', 'modified': '2024-04-17T09:57:31.756285+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 69, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7011}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7011': {'gid': 7011, 'name': 'a', 'rank': 0, 'incarnation': 45, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.32:6836/2416784965', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 2416784965}, {'type': 'v1', 'addr': '172.21.15.32:6836', 'nonce': 2416784965}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-17T09:57:32.559 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7011:up:replay 2024-04-17T09:57:33.560 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:57:33.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.711+0000 7facd2400640 1 Processor -- start 2024-04-17T09:57:33.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.711+0000 7facd2400640 1 -- start start 2024-04-17T09:57:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.711+0000 7facd2400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faccc15c750 0x7faccc15cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.711+0000 7facd2400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faccc160ce0 0x7faccc0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.711+0000 7facd2400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faccc0b26b0 0x7faccc06ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.711+0000 7facd2400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7faccc0cd100 con 0x7faccc160ce0 2024-04-17T09:57:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.711+0000 7facd2400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7faccc0b3c40 con 0x7faccc15c750 2024-04-17T09:57:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.711+0000 7facd2400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7faccc15d1a0 con 0x7faccc0b26b0 2024-04-17T09:57:33.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.711+0000 7facc3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faccc160ce0 0x7faccc0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:33.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.711+0000 7facc3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faccc160ce0 0x7faccc0b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59456/0 (socket says 172.21.15.32:59456) 2024-04-17T09:57:33.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.711+0000 7facc3e00640 1 -- 172.21.15.32:0/4206618860 learned_addr learned my addr 172.21.15.32:0/4206618860 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:33.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.711+0000 7faccb400640 1 --2- 172.21.15.32:0/4206618860 >> v2:172.21.15.144:3300/0 conn(0x7faccc15c750 0x7faccc15cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:33.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.711+0000 7faccbe00640 1 --2- 172.21.15.32:0/4206618860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faccc0b26b0 0x7faccc06ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:33.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.711+0000 7facc3e00640 1 -- 172.21.15.32:0/4206618860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faccc0b26b0 msgr2=0x7faccc06ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:33.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.711+0000 7facc3e00640 1 --2- 172.21.15.32:0/4206618860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faccc0b26b0 0x7faccc06ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:33.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.711+0000 7facc3e00640 1 -- 172.21.15.32:0/4206618860 >> v2:172.21.15.144:3300/0 conn(0x7faccc15c750 msgr2=0x7faccc15cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:33.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.711+0000 7facc3e00640 1 --2- 172.21.15.32:0/4206618860 >> v2:172.21.15.144:3300/0 conn(0x7faccc15c750 0x7faccc15cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:33.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.711+0000 7facc3e00640 1 -- 172.21.15.32:0/4206618860 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faccc15d480 con 0x7faccc160ce0 2024-04-17T09:57:33.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.715+0000 7facc3e00640 1 --2- 172.21.15.32:0/4206618860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faccc160ce0 0x7faccc0b2170 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7faccc0d1410 tx=0x7facb401c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5e86bf884ac5d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:33.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.715+0000 7faccaa00640 1 -- 172.21.15.32:0/4206618860 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb40b0070 con 0x7faccc160ce0 2024-04-17T09:57:33.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.715+0000 7faccaa00640 1 -- 172.21.15.32:0/4206618860 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facb40ab600 con 0x7faccc160ce0 2024-04-17T09:57:33.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.715+0000 7faccaa00640 1 -- 172.21.15.32:0/4206618860 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb40ab9d0 con 0x7faccc160ce0 2024-04-17T09:57:33.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.715+0000 7facd2400640 1 -- 172.21.15.32:0/4206618860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faccc160ce0 msgr2=0x7faccc0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:33.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.715+0000 7facd2400640 1 --2- 172.21.15.32:0/4206618860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faccc160ce0 0x7faccc0b2170 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7faccc0d1410 tx=0x7facb401c1d0 comp rx=0 tx=0).stop 2024-04-17T09:57:33.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.715+0000 7facd2400640 1 -- 172.21.15.32:0/4206618860 shutdown_connections 2024-04-17T09:57:33.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.715+0000 7facd2400640 1 --2- 172.21.15.32:0/4206618860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faccc0b26b0 0x7faccc06ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:33.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.715+0000 7facd2400640 1 --2- 172.21.15.32:0/4206618860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faccc160ce0 0x7faccc0b2170 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:33.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.715+0000 7facd2400640 1 --2- 172.21.15.32:0/4206618860 >> v2:172.21.15.144:3300/0 conn(0x7faccc15c750 0x7faccc15cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:33.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.715+0000 7facd2400640 1 -- 172.21.15.32:0/4206618860 >> 172.21.15.32:0/4206618860 conn(0x7faccc09a9f0 msgr2=0x7faccc0b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:33.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.715+0000 7facd2400640 1 -- 172.21.15.32:0/4206618860 shutdown_connections 2024-04-17T09:57:33.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.715+0000 7facd2400640 1 -- 172.21.15.32:0/4206618860 wait complete. 2024-04-17T09:57:33.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7facd2400640 1 Processor -- start 2024-04-17T09:57:33.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7facd2400640 1 -- start start 2024-04-17T09:57:33.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7facd2400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faccc0b26b0 0x7faccc0af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:33.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7facd2400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faccc15c750 0x7faccc0c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:33.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7facd2400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faccc160ce0 0x7faccc0c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:33.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7facd2400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faccc0cd700 con 0x7faccc0b26b0 2024-04-17T09:57:33.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7facd2400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faccc0cd580 con 0x7faccc15c750 2024-04-17T09:57:33.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7facd2400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7faccc0cd880 con 0x7faccc160ce0 2024-04-17T09:57:33.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7faccb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faccc0b26b0 0x7faccc0af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:33.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7faccb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faccc0b26b0 0x7faccc0af0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59460/0 (socket says 172.21.15.32:59460) 2024-04-17T09:57:33.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7faccb400640 1 -- 172.21.15.32:0/3806986256 learned_addr learned my addr 172.21.15.32:0/3806986256 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:33.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7facc3e00640 1 --2- 172.21.15.32:0/3806986256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faccc15c750 0x7faccc0c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:33.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7faccbe00640 1 --2- 172.21.15.32:0/3806986256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faccc160ce0 0x7faccc0c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:33.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7faccb400640 1 -- 172.21.15.32:0/3806986256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faccc160ce0 msgr2=0x7faccc0c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:33.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7faccb400640 1 --2- 172.21.15.32:0/3806986256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faccc160ce0 0x7faccc0c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:33.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7faccb400640 1 -- 172.21.15.32:0/3806986256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faccc15c750 msgr2=0x7faccc0c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:33.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7faccb400640 1 --2- 172.21.15.32:0/3806986256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faccc15c750 0x7faccc0c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:33.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7faccb400640 1 -- 172.21.15.32:0/3806986256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faccc154310 con 0x7faccc0b26b0 2024-04-17T09:57:33.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7faccb400640 1 --2- 172.21.15.32:0/3806986256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faccc0b26b0 0x7faccc0af0b0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7facbc00d800 tx=0x7facbc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:33.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7facc3400640 1 -- 172.21.15.32:0/3806986256 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc00b660 con 0x7faccc0b26b0 2024-04-17T09:57:33.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7facc3400640 1 -- 172.21.15.32:0/3806986256 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facbc019070 con 0x7faccc0b26b0 2024-04-17T09:57:33.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7facc3400640 1 -- 172.21.15.32:0/3806986256 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc00ba00 con 0x7faccc0b26b0 2024-04-17T09:57:33.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7facd2400640 1 -- 172.21.15.32:0/3806986256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faccc0b05c0 con 0x7faccc0b26b0 2024-04-17T09:57:33.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.719+0000 7facd2400640 1 -- 172.21.15.32:0/3806986256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faccc15f030 con 0x7faccc0b26b0 2024-04-17T09:57:33.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.723+0000 7facd2400640 1 -- 172.21.15.32:0/3806986256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faccc068a90 con 0x7faccc0b26b0 2024-04-17T09:57:33.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.723+0000 7facc3400640 1 -- 172.21.15.32:0/3806986256 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7facbc00bba0 con 0x7faccc0b26b0 2024-04-17T09:57:33.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.723+0000 7facc3400640 1 --2- 172.21.15.32:0/3806986256 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faca8068380 0x7faca806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:33.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.723+0000 7facc3400640 1 -- 172.21.15.32:0/3806986256 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7facbc086d40 con 0x7faccc0b26b0 2024-04-17T09:57:33.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.723+0000 7facc3e00640 1 --2- 172.21.15.32:0/3806986256 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faca8068380 0x7faca806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:33.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.723+0000 7facc3e00640 1 --2- 172.21.15.32:0/3806986256 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faca8068380 0x7faca806a840 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7facb40ad000 tx=0x7facb40a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:33.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.727+0000 7facc3400640 1 -- 172.21.15.32:0/3806986256 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7facbc056c40 con 0x7faccc0b26b0 2024-04-17T09:57:33.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.927+0000 7facd2400640 1 -- 172.21.15.32:0/3806986256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faccc073b10 con 0x7faccc0b26b0 2024-04-17T09:57:33.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.931+0000 7facc3400640 1 -- 172.21.15.32:0/3806986256 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5198 (secure 0 0 0) 0x7facbc05af40 con 0x7faccc0b26b0 2024-04-17T09:57:33.932 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:57:33.932 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2424282223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2424282223},{"type":"v1","addr":"172.21.15.144:6834","nonce":2424282223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3537056692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3537056692},{"type":"v1","addr":"172.21.15.144:6835","nonce":3537056692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5891,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/725333724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":725333724},{"type":"v1","addr":"172.21.15.32:6837","nonce":725333724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:57:11.558183+0000","modified":"2024-04-17T09:57:32.762131+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"a","rank":0,"incarnation":45,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.32:6836/2416784965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2416784965},{"type":"v1","addr":"172.21.15.32:6836","nonce":2416784965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-04-17T09:57:33.932 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 47 2024-04-17T09:57:33.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.931+0000 7facd2400640 1 -- 172.21.15.32:0/3806986256 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faca8068380 msgr2=0x7faca806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:33.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.931+0000 7facd2400640 1 --2- 172.21.15.32:0/3806986256 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faca8068380 0x7faca806a840 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7facb40ad000 tx=0x7facb40a6000 comp rx=0 tx=0).stop 2024-04-17T09:57:33.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.931+0000 7facd2400640 1 -- 172.21.15.32:0/3806986256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faccc0b26b0 msgr2=0x7faccc0af0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:33.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.931+0000 7facd2400640 1 --2- 172.21.15.32:0/3806986256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faccc0b26b0 0x7faccc0af0b0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7facbc00d800 tx=0x7facbc00dcc0 comp rx=0 tx=0).stop 2024-04-17T09:57:33.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.931+0000 7facd2400640 1 -- 172.21.15.32:0/3806986256 shutdown_connections 2024-04-17T09:57:33.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.931+0000 7facd2400640 1 --2- 172.21.15.32:0/3806986256 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faca8068380 0x7faca806a840 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:33.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.931+0000 7facd2400640 1 --2- 172.21.15.32:0/3806986256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faccc160ce0 0x7faccc0c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:33.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.931+0000 7facd2400640 1 --2- 172.21.15.32:0/3806986256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faccc15c750 0x7faccc0c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:33.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.931+0000 7facd2400640 1 --2- 172.21.15.32:0/3806986256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faccc0b26b0 0x7faccc0af0b0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:33.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.931+0000 7facd2400640 1 -- 172.21.15.32:0/3806986256 >> 172.21.15.32:0/3806986256 conn(0x7faccc09a9f0 msgr2=0x7faccc0d0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:33.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.931+0000 7facd2400640 1 -- 172.21.15.32:0/3806986256 shutdown_connections 2024-04-17T09:57:33.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:33.931+0000 7facd2400640 1 -- 172.21.15.32:0/3806986256 wait complete. 2024-04-17T09:57:34.357 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T09:57:11.558183+0000', 'modified': '2024-04-17T09:57:32.762131+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 69, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7011}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7011': {'gid': 7011, 'name': 'a', 'rank': 0, 'incarnation': 45, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.32:6836/2416784965', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 2416784965}, {'type': 'v1', 'addr': '172.21.15.32:6836', 'nonce': 2416784965}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-17T09:57:34.357 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7011:up:rejoin 2024-04-17T09:57:35.358 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:57:35.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.511+0000 7f520fc00640 1 Processor -- start 2024-04-17T09:57:35.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.511+0000 7f520fc00640 1 -- start start 2024-04-17T09:57:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.511+0000 7f520fc00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5208144e90 0x7f52080b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.511+0000 7f520fc00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f520815c1b0 0x7f520806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.511+0000 7f520fc00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5208072bb0 con 0x7f52080b4de0 2024-04-17T09:57:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.511+0000 7f520fc00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5208145270 con 0x7f5208144e90 2024-04-17T09:57:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.511+0000 7f520fc00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f52080b3dc0 con 0x7f520815c1b0 2024-04-17T09:57:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.511+0000 7f520d200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5208144e90 0x7f52080b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.511+0000 7f520d200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5208144e90 0x7f52080b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59466/0 (socket says 172.21.15.32:59466) 2024-04-17T09:57:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.511+0000 7f520d200640 1 -- 172.21.15.32:0/3569645716 learned_addr learned my addr 172.21.15.32:0/3569645716 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.511+0000 7f5207e00640 1 --1- 172.21.15.32:0/3569645716 >> v1:172.21.15.32:6789/0 conn(0x7f52080b4de0 0x7f520815baa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53066/0 (socket says 172.21.15.32:53066) 2024-04-17T09:57:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.511+0000 7f520dc00640 1 --2- 172.21.15.32:0/3569645716 >> v2:172.21.15.144:3300/0 conn(0x7f520815c1b0 0x7f520806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.511+0000 7f520d200640 1 -- 172.21.15.32:0/3569645716 >> v2:172.21.15.144:3300/0 conn(0x7f520815c1b0 msgr2=0x7f520806ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.511+0000 7f520d200640 1 --2- 172.21.15.32:0/3569645716 >> v2:172.21.15.144:3300/0 conn(0x7f520815c1b0 0x7f520806ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.511+0000 7f520d200640 1 -- 172.21.15.32:0/3569645716 >> v1:172.21.15.32:6789/0 conn(0x7f52080b4de0 legacy=0x7f520815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.511+0000 7f520d200640 1 -- 172.21.15.32:0/3569645716 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52080b40a0 con 0x7f5208144e90 2024-04-17T09:57:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.511+0000 7f5207400640 1 -- 172.21.15.32:0/3569645716 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1920160025 0 0) 0x7f5208072bb0 con 0x7f52080b4de0 2024-04-17T09:57:35.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.515+0000 7f520d200640 1 --2- 172.21.15.32:0/3569645716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5208144e90 0x7f52080b48a0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f52080b3a00 tx=0x7f51fc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290e9a58573452d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:35.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.515+0000 7f5207400640 1 -- 172.21.15.32:0/3569645716 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc0ab600 con 0x7f5208144e90 2024-04-17T09:57:35.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.515+0000 7f5207400640 1 -- 172.21.15.32:0/3569645716 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51fc0b0070 con 0x7f5208144e90 2024-04-17T09:57:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.515+0000 7f5207400640 1 -- 172.21.15.32:0/3569645716 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc0ab9c0 con 0x7f5208144e90 2024-04-17T09:57:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.515+0000 7f520fc00640 1 -- 172.21.15.32:0/3569645716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5208144e90 msgr2=0x7f52080b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.515+0000 7f520fc00640 1 --2- 172.21.15.32:0/3569645716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5208144e90 0x7f52080b48a0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f52080b3a00 tx=0x7f51fc01c4f0 comp rx=0 tx=0).stop 2024-04-17T09:57:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.515+0000 7f520fc00640 1 -- 172.21.15.32:0/3569645716 shutdown_connections 2024-04-17T09:57:35.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.515+0000 7f520fc00640 1 --2- 172.21.15.32:0/3569645716 >> v2:172.21.15.144:3300/0 conn(0x7f520815c1b0 0x7f520806ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:35.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.515+0000 7f520fc00640 1 --2- 172.21.15.32:0/3569645716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5208144e90 0x7f52080b48a0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:35.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.515+0000 7f520fc00640 1 -- 172.21.15.32:0/3569645716 >> 172.21.15.32:0/3569645716 conn(0x7f520806ba30 msgr2=0x7f520806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:35.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.515+0000 7f520fc00640 1 -- 172.21.15.32:0/3569645716 shutdown_connections 2024-04-17T09:57:35.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.515+0000 7f520fc00640 1 -- 172.21.15.32:0/3569645716 wait complete. 2024-04-17T09:57:35.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f520fc00640 1 Processor -- start 2024-04-17T09:57:35.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f520fc00640 1 -- start start 2024-04-17T09:57:35.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f520fc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52080b4de0 0x7f520815a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:35.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f520fc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5208144e90 0x7f52080c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:35.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f520fc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f520815c1b0 0x7f520813c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:35.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f520fc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f520806de50 con 0x7f5208144e90 2024-04-17T09:57:35.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f520fc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5208072880 con 0x7f520815c1b0 2024-04-17T09:57:35.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f520fc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f520806dfd0 con 0x7f52080b4de0 2024-04-17T09:57:35.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f5207e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5208144e90 0x7f52080c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:35.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f5207e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5208144e90 0x7f52080c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59482/0 (socket says 172.21.15.32:59482) 2024-04-17T09:57:35.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f5207e00640 1 -- 172.21.15.32:0/2824923477 learned_addr learned my addr 172.21.15.32:0/2824923477 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:35.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f520d200640 1 --2- 172.21.15.32:0/2824923477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52080b4de0 0x7f520815a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:35.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f520dc00640 1 --2- 172.21.15.32:0/2824923477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f520815c1b0 0x7f520813c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:35.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f5207e00640 1 -- 172.21.15.32:0/2824923477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52080b4de0 msgr2=0x7f520815a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:35.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f5207e00640 1 --2- 172.21.15.32:0/2824923477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52080b4de0 0x7f520815a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:35.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f5207e00640 1 -- 172.21.15.32:0/2824923477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f520815c1b0 msgr2=0x7f520813c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:35.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f5207e00640 1 --2- 172.21.15.32:0/2824923477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f520815c1b0 0x7f520813c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:35.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f5207e00640 1 -- 172.21.15.32:0/2824923477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520815a910 con 0x7f5208144e90 2024-04-17T09:57:35.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f520dc00640 1 --2- 172.21.15.32:0/2824923477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f520815c1b0 0x7f520813c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:57:35.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f5207e00640 1 --2- 172.21.15.32:0/2824923477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5208144e90 0x7f52080c5400 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f51f400c4b0 tx=0x7f51f400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:35.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f5204c00640 1 -- 172.21.15.32:0/2824923477 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4018070 con 0x7f5208144e90 2024-04-17T09:57:35.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f5204c00640 1 -- 172.21.15.32:0/2824923477 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f4007480 con 0x7f5208144e90 2024-04-17T09:57:35.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f5204c00640 1 -- 172.21.15.32:0/2824923477 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4004a30 con 0x7f5208144e90 2024-04-17T09:57:35.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f520fc00640 1 -- 172.21.15.32:0/2824923477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52081422c0 con 0x7f5208144e90 2024-04-17T09:57:35.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f520fc00640 1 -- 172.21.15.32:0/2824923477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5208142880 con 0x7f5208144e90 2024-04-17T09:57:35.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.519+0000 7f520fc00640 1 -- 172.21.15.32:0/2824923477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5208068a90 con 0x7f5208144e90 2024-04-17T09:57:35.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.523+0000 7f5204c00640 1 -- 172.21.15.32:0/2824923477 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f51f4004bd0 con 0x7f5208144e90 2024-04-17T09:57:35.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.523+0000 7f5204c00640 1 --2- 172.21.15.32:0/2824923477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f51e8068380 0x7f51e806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:35.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.523+0000 7f5204c00640 1 -- 172.21.15.32:0/2824923477 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7f51f4085cd0 con 0x7f5208144e90 2024-04-17T09:57:35.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.523+0000 7f520d200640 1 --2- 172.21.15.32:0/2824923477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f51e8068380 0x7f51e806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:35.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.523+0000 7f520d200640 1 --2- 172.21.15.32:0/2824923477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f51e8068380 0x7f51e806a840 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0af000 tx=0x7f51fc0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:35.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.527+0000 7f5204c00640 1 -- 172.21.15.32:0/2824923477 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f51f4055c00 con 0x7f5208144e90 2024-04-17T09:57:35.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.727+0000 7f520fc00640 1 -- 172.21.15.32:0/2824923477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52080b1cc0 con 0x7f5208144e90 2024-04-17T09:57:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.727+0000 7f5204c00640 1 -- 172.21.15.32:0/2824923477 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5205 (secure 0 0 0) 0x7f51f4059f00 con 0x7f5208144e90 2024-04-17T09:57:35.729 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:57:35.729 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2424282223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2424282223},{"type":"v1","addr":"172.21.15.144:6834","nonce":2424282223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3537056692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3537056692},{"type":"v1","addr":"172.21.15.144:6835","nonce":3537056692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5891,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/725333724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":725333724},{"type":"v1","addr":"172.21.15.32:6837","nonce":725333724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:57:11.558183+0000","modified":"2024-04-17T09:57:34.766815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6836/2416784965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2416784965},{"type":"v1","addr":"172.21.15.32:6836","nonce":2416784965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7011,"qdb_cluster":[7011]},"id":5}]} 2024-04-17T09:57:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 48 2024-04-17T09:57:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.727+0000 7f520fc00640 1 -- 172.21.15.32:0/2824923477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f51e8068380 msgr2=0x7f51e806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.727+0000 7f520fc00640 1 --2- 172.21.15.32:0/2824923477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f51e8068380 0x7f51e806a840 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0af000 tx=0x7f51fc0a7000 comp rx=0 tx=0).stop 2024-04-17T09:57:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.727+0000 7f520fc00640 1 -- 172.21.15.32:0/2824923477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5208144e90 msgr2=0x7f52080c5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.727+0000 7f520fc00640 1 --2- 172.21.15.32:0/2824923477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5208144e90 0x7f52080c5400 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f51f400c4b0 tx=0x7f51f400c970 comp rx=0 tx=0).stop 2024-04-17T09:57:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.727+0000 7f520fc00640 1 -- 172.21.15.32:0/2824923477 shutdown_connections 2024-04-17T09:57:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.727+0000 7f520fc00640 1 --2- 172.21.15.32:0/2824923477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f51e8068380 0x7f51e806a840 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.727+0000 7f520fc00640 1 --2- 172.21.15.32:0/2824923477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f520815c1b0 0x7f520813c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.727+0000 7f520fc00640 1 --2- 172.21.15.32:0/2824923477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5208144e90 0x7f52080c5400 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.727+0000 7f520fc00640 1 --2- 172.21.15.32:0/2824923477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52080b4de0 0x7f520815a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.727+0000 7f520fc00640 1 -- 172.21.15.32:0/2824923477 >> 172.21.15.32:0/2824923477 conn(0x7f520806ba30 msgr2=0x7f52080b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:35.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.727+0000 7f520fc00640 1 -- 172.21.15.32:0/2824923477 shutdown_connections 2024-04-17T09:57:35.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:35.727+0000 7f520fc00640 1 -- 172.21.15.32:0/2824923477 wait complete. 2024-04-17T09:57:36.153 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T09:57:11.558183+0000', 'modified': '2024-04-17T09:57:34.766815+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 69, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7011}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7011': {'gid': 7011, 'name': 'a', 'rank': 0, 'incarnation': 45, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.32:6836/2416784965', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 2416784965}, {'type': 'v1', 'addr': '172.21.15.32:6836', 'nonce': 2416784965}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7011, 'qdb_cluster': [7011]} 2024-04-17T09:57:36.153 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T09:57:36.153 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-17T09:57:36.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.295+0000 7f25e6800640 1 Processor -- start 2024-04-17T09:57:36.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.295+0000 7f25e6800640 1 -- start start 2024-04-17T09:57:36.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25e6800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25e015ff10 0x7f25e00cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25e6800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f25e00d0360 0x7f25e0160500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25e6800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25e0161280 con 0x7f25e00b3d10 2024-04-17T09:57:36.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25e6800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f25e009bf30 con 0x7f25e00d0360 2024-04-17T09:57:36.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25e6800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f25e01602f0 con 0x7f25e015ff10 2024-04-17T09:57:36.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dfe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f25e00b3d10 0x7f25e0160a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53078/0 (socket says 172.21.15.32:53078) 2024-04-17T09:57:36.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dfe00640 1 -- 172.21.15.32:0/3716971677 learned_addr learned my addr 172.21.15.32:0/3716971677 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:36.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25d7e00640 1 --2- 172.21.15.32:0/3716971677 >> v2:172.21.15.144:3300/0 conn(0x7f25e00d0360 0x7f25e0160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25df400640 1 --2- 172.21.15.32:0/3716971677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25e015ff10 0x7f25e00cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dea00640 1 -- 172.21.15.32:0/3716971677 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 756653598 0 0) 0x7f25e0161280 con 0x7f25e00b3d10 2024-04-17T09:57:36.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dea00640 1 -- 172.21.15.32:0/3716971677 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25c4003530 con 0x7f25e00b3d10 2024-04-17T09:57:36.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dea00640 1 -- 172.21.15.32:0/3716971677 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3212469298 0 0) 0x7f25c4003530 con 0x7f25e00b3d10 2024-04-17T09:57:36.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dea00640 1 -- 172.21.15.32:0/3716971677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25e015ff10 msgr2=0x7f25e00cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:36.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dea00640 1 --2- 172.21.15.32:0/3716971677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25e015ff10 0x7f25e00cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dea00640 1 -- 172.21.15.32:0/3716971677 >> v2:172.21.15.144:3300/0 conn(0x7f25e00d0360 msgr2=0x7f25e0160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:36.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dea00640 1 --2- 172.21.15.32:0/3716971677 >> v2:172.21.15.144:3300/0 conn(0x7f25e00d0360 0x7f25e0160500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dea00640 1 -- 172.21.15.32:0/3716971677 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e009bc50 con 0x7f25e00b3d10 2024-04-17T09:57:36.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dea00640 1 -- 172.21.15.32:0/3716971677 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f25d8001a00 con 0x7f25e00b3d10 2024-04-17T09:57:36.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25d7e00640 1 --2- 172.21.15.32:0/3716971677 >> v2:172.21.15.144:3300/0 conn(0x7f25e00d0360 0x7f25e0160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:57:36.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dea00640 1 -- 172.21.15.32:0/3716971677 >> v1:172.21.15.32:6789/0 conn(0x7f25e00b3d10 legacy=0x7f25e0160a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:36.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dea00640 1 --2- 172.21.15.32:0/3716971677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25e015ff10 0x7f25c40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dea00640 1 --2- 172.21.15.32:0/3716971677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c4003f40 0x7f25c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dea00640 1 --2- 172.21.15.32:0/3716971677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25c4004d50 0x7f25c40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dea00640 1 -- 172.21.15.32:0/3716971677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f25e01602f0 con 0x7f25c4004d50 2024-04-17T09:57:36.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dea00640 1 -- 172.21.15.32:0/3716971677 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f25c400ba30 con 0x7f25c4003f40 2024-04-17T09:57:36.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dea00640 1 -- 172.21.15.32:0/3716971677 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f25c400bc30 con 0x7f25e015ff10 2024-04-17T09:57:36.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dfe00640 1 --2- 172.21.15.32:0/3716971677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25c4004d50 0x7f25c40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25df400640 1 --2- 172.21.15.32:0/3716971677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25e015ff10 0x7f25c40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25d7e00640 1 --2- 172.21.15.32:0/3716971677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c4003f40 0x7f25c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dfe00640 1 -- 172.21.15.32:0/3716971677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25e015ff10 msgr2=0x7f25c40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:36.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dfe00640 1 --2- 172.21.15.32:0/3716971677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25e015ff10 0x7f25c40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dfe00640 1 -- 172.21.15.32:0/3716971677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c4003f40 msgr2=0x7f25c4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:36.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dfe00640 1 --2- 172.21.15.32:0/3716971677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c4003f40 0x7f25c4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.299+0000 7f25dfe00640 1 -- 172.21.15.32:0/3716971677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c4003530 con 0x7f25c4004d50 2024-04-17T09:57:36.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.303+0000 7f25dfe00640 1 --2- 172.21.15.32:0/3716971677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25c4004d50 0x7f25c40051d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f25d800a8a0 tx=0x7f25d801d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c85f7ddcca66b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:36.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.303+0000 7f25dea00640 1 -- 172.21.15.32:0/3716971677 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d800a4c0 con 0x7f25c4004d50 2024-04-17T09:57:36.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.303+0000 7f25dea00640 1 -- 172.21.15.32:0/3716971677 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d80ad650 con 0x7f25c4004d50 2024-04-17T09:57:36.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.303+0000 7f25dea00640 1 -- 172.21.15.32:0/3716971677 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d80ada80 con 0x7f25c4004d50 2024-04-17T09:57:36.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.303+0000 7f25e6800640 1 -- 172.21.15.32:0/3716971677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25c4004d50 msgr2=0x7f25c40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:36.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.303+0000 7f25e6800640 1 --2- 172.21.15.32:0/3716971677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25c4004d50 0x7f25c40051d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f25d800a8a0 tx=0x7f25d801d3c0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.303+0000 7f25dfe00640 1 -- 172.21.15.32:0/3716971677 reap_dead start 2024-04-17T09:57:36.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.303+0000 7f25e6800640 1 -- 172.21.15.32:0/3716971677 shutdown_connections 2024-04-17T09:57:36.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.303+0000 7f25e6800640 1 -- 172.21.15.32:0/3716971677 >> 172.21.15.32:0/3716971677 conn(0x7f25e00b4280 msgr2=0x7f25e00b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:36.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.303+0000 7f25e6800640 1 -- 172.21.15.32:0/3716971677 shutdown_connections 2024-04-17T09:57:36.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.303+0000 7f25e6800640 1 -- 172.21.15.32:0/3716971677 wait complete. 2024-04-17T09:57:36.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25e6800640 1 Processor -- start 2024-04-17T09:57:36.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25e6800640 1 -- start start 2024-04-17T09:57:36.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25e6800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25c4004d50 0x7f25e015d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25e6800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25e01f7780 0x7f25e01f7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25e6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e00b05f0 0x7f25e00b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25e6800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f25e00b3040 con 0x7f25e00b05f0 2024-04-17T09:57:36.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25e6800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f25e00b2ec0 con 0x7f25e01f7780 2024-04-17T09:57:36.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25e6800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f25e009ba00 con 0x7f25c4004d50 2024-04-17T09:57:36.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25dfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e00b05f0 0x7f25e00b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25dfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e00b05f0 0x7f25e00b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59496/0 (socket says 172.21.15.32:59496) 2024-04-17T09:57:36.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25dfe00640 1 -- 172.21.15.32:0/2792073900 learned_addr learned my addr 172.21.15.32:0/2792073900 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:36.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25d7e00640 1 --2- 172.21.15.32:0/2792073900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25e01f7780 0x7f25e01f7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25df400640 1 --2- 172.21.15.32:0/2792073900 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25c4004d50 0x7f25e015d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25dfe00640 1 -- 172.21.15.32:0/2792073900 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25c4004d50 msgr2=0x7f25e015d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:36.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25dfe00640 1 --2- 172.21.15.32:0/2792073900 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25c4004d50 0x7f25e015d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25dfe00640 1 -- 172.21.15.32:0/2792073900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25e01f7780 msgr2=0x7f25e01f7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:36.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25dfe00640 1 --2- 172.21.15.32:0/2792073900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25e01f7780 0x7f25e01f7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25dfe00640 1 -- 172.21.15.32:0/2792073900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e01f8270 con 0x7f25e00b05f0 2024-04-17T09:57:36.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25dfe00640 1 --2- 172.21.15.32:0/2792073900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e00b05f0 0x7f25e00b0a70 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f25d801d1f0 tx=0x7f25c4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:36.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25d7400640 1 -- 172.21.15.32:0/2792073900 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d80b2040 con 0x7f25e00b05f0 2024-04-17T09:57:36.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25d7400640 1 -- 172.21.15.32:0/2792073900 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d80bc070 con 0x7f25e00b05f0 2024-04-17T09:57:36.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25d7400640 1 -- 172.21.15.32:0/2792073900 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8002020 con 0x7f25e00b05f0 2024-04-17T09:57:36.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25e6800640 1 -- 172.21.15.32:0/2792073900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e01f8550 con 0x7f25e00b05f0 2024-04-17T09:57:36.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.307+0000 7f25e6800640 1 -- 172.21.15.32:0/2792073900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e01f8e70 con 0x7f25e00b05f0 2024-04-17T09:57:36.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.311+0000 7f25d7400640 1 -- 172.21.15.32:0/2792073900 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f25d80017c0 con 0x7f25e00b05f0 2024-04-17T09:57:36.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.311+0000 7f25d7400640 1 --2- 172.21.15.32:0/2792073900 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f25bc068380 0x7f25bc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.311+0000 7f25d7400640 1 -- 172.21.15.32:0/2792073900 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7f25d8121300 con 0x7f25e00b05f0 2024-04-17T09:57:36.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.311+0000 7f25df400640 1 --2- 172.21.15.32:0/2792073900 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f25bc068380 0x7f25bc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.311+0000 7f25e9d82000 1 Processor -- start 2024-04-17T09:57:36.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.311+0000 7f25e9d82000 1 -- start start 2024-04-17T09:57:36.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.311+0000 7f25df400640 1 --2- 172.21.15.32:0/2792073900 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f25bc068380 0x7f25bc06a840 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009be0 tx=0x7f25d0009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:36.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25e9d82000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c7bb90b50 0x555c7bb90f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25e9d82000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb91470 0x555c7bb467b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25e9d82000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c7bb46cf0 0x555c7bb470d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25e9d82000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555c7b9944d0 con 0x555c7bb91470 2024-04-17T09:57:36.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25e9d82000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555c7b92ebf0 con 0x555c7bb90b50 2024-04-17T09:57:36.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25e9d82000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x555c7b990490 con 0x555c7bb46cf0 2024-04-17T09:57:36.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25d7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb91470 0x555c7bb467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25d7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb91470 0x555c7bb467b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59512/0 (socket says 172.21.15.32:59512) 2024-04-17T09:57:36.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25d7e00640 1 -- 172.21.15.32:0/1505869085 learned_addr learned my addr 172.21.15.32:0/1505869085 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:36.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25df400640 1 --2- 172.21.15.32:0/1505869085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c7bb90b50 0x555c7bb90f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25dfe00640 1 --2- 172.21.15.32:0/1505869085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c7bb46cf0 0x555c7bb470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25d7e00640 1 -- 172.21.15.32:0/1505869085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c7bb46cf0 msgr2=0x555c7bb470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:36.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25d7e00640 1 --2- 172.21.15.32:0/1505869085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c7bb46cf0 0x555c7bb470d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25d7e00640 1 -- 172.21.15.32:0/1505869085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c7bb90b50 msgr2=0x555c7bb90f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:36.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25d7e00640 1 --2- 172.21.15.32:0/1505869085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c7bb90b50 0x555c7bb90f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25d7e00640 1 -- 172.21.15.32:0/1505869085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c7b91ae80 con 0x555c7bb91470 2024-04-17T09:57:36.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25d7e00640 1 --2- 172.21.15.32:0/1505869085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb91470 0x555c7bb467b0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009d80 tx=0x7f25c8014780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b30521315f811ae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:36.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25cfe00640 1 -- 172.21.15.32:0/1505869085 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c8004020 con 0x555c7bb91470 2024-04-17T09:57:36.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25cfe00640 1 -- 172.21.15.32:0/1505869085 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25c80041c0 con 0x555c7bb91470 2024-04-17T09:57:36.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25cfe00640 1 -- 172.21.15.32:0/1505869085 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c8015630 con 0x555c7bb91470 2024-04-17T09:57:36.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25e9d82000 1 -- 172.21.15.32:0/1505869085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb91470 msgr2=0x555c7bb467b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:36.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25e9d82000 1 --2- 172.21.15.32:0/1505869085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb91470 0x555c7bb467b0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009d80 tx=0x7f25c8014780 comp rx=0 tx=0).stop 2024-04-17T09:57:36.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25e9d82000 1 -- 172.21.15.32:0/1505869085 shutdown_connections 2024-04-17T09:57:36.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25e9d82000 1 --2- 172.21.15.32:0/1505869085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c7bb46cf0 0x555c7bb470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25e9d82000 1 --2- 172.21.15.32:0/1505869085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb91470 0x555c7bb467b0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25e9d82000 1 --2- 172.21.15.32:0/1505869085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c7bb90b50 0x555c7bb90f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25e9d82000 1 -- 172.21.15.32:0/1505869085 >> 172.21.15.32:0/1505869085 conn(0x555c7bb4c470 msgr2=0x555c7bb45090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:36.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25e9d82000 1 -- 172.21.15.32:0/1505869085 shutdown_connections 2024-04-17T09:57:36.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25e9d82000 1 -- 172.21.15.32:0/1505869085 wait complete. 2024-04-17T09:57:36.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25e9d82000 1 -- start start 2024-04-17T09:57:36.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.315+0000 7f25e9d82000 1 Processor -- start 2024-04-17T09:57:36.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb46cf0 0x555c7bb4b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c7bb90b50 0x555c7bb7e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c7bb91470 0x555c7bb7e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555c7b951a60 con 0x555c7bb46cf0 2024-04-17T09:57:36.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555c7b938e80 con 0x555c7bb90b50 2024-04-17T09:57:36.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x555c7b94fcf0 con 0x555c7bb91470 2024-04-17T09:57:36.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 10 client.0 _ll_register_callbacks cb 0x555c7bb31a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:57:36.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25df400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb46cf0 0x555c7bb4b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25df400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb46cf0 0x555c7bb4b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59518/0 (socket says 172.21.15.32:59518) 2024-04-17T09:57:36.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25df400640 1 -- 172.21.15.32:0/4006619574 learned_addr learned my addr 172.21.15.32:0/4006619574 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:36.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25d7e00640 1 --2- 172.21.15.32:0/4006619574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c7bb90b50 0x555c7bb7e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25dfe00640 1 --2- 172.21.15.32:0/4006619574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c7bb91470 0x555c7bb7e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25df400640 1 -- 172.21.15.32:0/4006619574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c7bb91470 msgr2=0x555c7bb7e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:36.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25df400640 1 --2- 172.21.15.32:0/4006619574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c7bb91470 0x555c7bb7e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25df400640 1 -- 172.21.15.32:0/4006619574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c7bb90b50 msgr2=0x555c7bb7e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:36.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25df400640 1 --2- 172.21.15.32:0/4006619574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c7bb90b50 0x555c7bb7e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25df400640 1 -- 172.21.15.32:0/4006619574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c7b92ee40 con 0x555c7bb46cf0 2024-04-17T09:57:36.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25d7e00640 1 --2- 172.21.15.32:0/4006619574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c7bb90b50 0x555c7bb7e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:57:36.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25dfe00640 1 --2- 172.21.15.32:0/4006619574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c7bb91470 0x555c7bb7e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:57:36.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25df400640 1 --2- 172.21.15.32:0/4006619574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb46cf0 0x555c7bb4b7a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f25d00038a0 tx=0x7f25d00046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb6166c641ec03d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:36.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:57:36.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25ccc00640 1 -- 172.21.15.32:0/4006619574 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d001b070 con 0x555c7bb46cf0 2024-04-17T09:57:36.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25ccc00640 1 -- 172.21.15.32:0/4006619574 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7f25d00159a0 con 0x555c7bb46cf0 2024-04-17T09:57:36.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 1 -- 172.21.15.32:0/4006619574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25c80041c0 con 0x555c7bb46cf0 2024-04-17T09:57:36.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25ccc00640 1 -- 172.21.15.32:0/4006619574 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f25d0034050 con 0x555c7bb46cf0 2024-04-17T09:57:36.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 10 client.7065 fetch_fsmap learned FSMap version 48 2024-04-17T09:57:36.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 1 -- 172.21.15.32:0/4006619574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) -- 0x555c7b95fe00 con 0x555c7bb46cf0 2024-04-17T09:57:36.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25ccc00640 1 -- 172.21.15.32:0/4006619574 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25c80041c0 con 0x555c7bb46cf0 2024-04-17T09:57:36.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25ccc00640 1 -- 172.21.15.32:0/4006619574 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 48) ==== 3495+0+0 (secure 0 0 0) 0x7f25d0015be0 con 0x555c7bb46cf0 2024-04-17T09:57:36.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 10 client.7065 fetch_fsmap finished waiting for FSMap version 48 2024-04-17T09:57:36.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 10 client.7065 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T09:57:36.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 20 client.7065 populate_metadata read hostname 'smithi032' 2024-04-17T09:57:36.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 1 --2- 172.21.15.32:0/4006619574 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x555c7bba00b0 0x555c7bba24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 4 client.7065 mds_command: new command op to 7011 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:57:36.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 1 -- 172.21.15.32:0/4006619574 --> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c7b92dbc0 con 0x555c7bba00b0 2024-04-17T09:57:36.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25d7e00640 1 --2- 172.21.15.32:0/4006619574 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x555c7bba00b0 0x555c7bba24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25ccc00640 10 client.7065 ms_handle_connect on v2:172.21.15.32:6834/2416784965 2024-04-17T09:57:36.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25d7e00640 1 --2- 172.21.15.32:0/4006619574 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x555c7bba00b0 0x555c7bba24a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3f244491cfc7195 server_cookie=90190de8795e5e0f in_seq=0 out_seq=0 2024-04-17T09:57:36.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25ccc00640 1 -- 172.21.15.32:0/4006619574 <== mds.0 v2:172.21.15.32:6834/2416784965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c7b92dbc0 con 0x555c7bba00b0 2024-04-17T09:57:36.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25ccc00640 10 client.7065 handle_command_reply: tid=0 2024-04-17T09:57:36.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 1 client.7065 shutdown 2024-04-17T09:57:36.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 1 -- 172.21.15.32:0/4006619574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb46cf0 msgr2=0x555c7bb4b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:36.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 1 --2- 172.21.15.32:0/4006619574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb46cf0 0x555c7bb4b7a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f25d00038a0 tx=0x7f25d00046d0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 1 -- 172.21.15.32:0/4006619574 shutdown_connections 2024-04-17T09:57:36.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 1 --2- 172.21.15.32:0/4006619574 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x555c7bba00b0 0x555c7bba24a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 1 --2- 172.21.15.32:0/4006619574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c7bb91470 0x555c7bb7e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25ccc00640 0 client.7065 ms_handle_reset on v2:172.21.15.32:6834/2416784965 2024-04-17T09:57:36.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 1 --2- 172.21.15.32:0/4006619574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c7bb90b50 0x555c7bb7e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 1 --2- 172.21.15.32:0/4006619574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb46cf0 0x555c7bb4b7a0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.319+0000 7f25e9d82000 1 -- 172.21.15.32:0/4006619574 >> 172.21.15.32:0/4006619574 conn(0x555c7bb4c470 msgr2=0x555c7bb457a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:36.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.323+0000 7f25e9d82000 1 -- 172.21.15.32:0/4006619574 shutdown_connections 2024-04-17T09:57:36.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.323+0000 7f25e9d82000 1 -- 172.21.15.32:0/4006619574 wait complete. 2024-04-17T09:57:36.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.323+0000 7f25e9d82000 20 client.7065 trim_cache size 0 max 16384 2024-04-17T09:57:36.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.339+0000 7f25e9d82000 1 Processor -- start 2024-04-17T09:57:36.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.339+0000 7f25e9d82000 1 -- start start 2024-04-17T09:57:36.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.339+0000 7f25e9d82000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb4c470 0x555c7bba5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.339+0000 7f25e9d82000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c7bb46cf0 0x555c7bba14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.339+0000 7f25e9d82000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c7bb90b50 0x555c7bba19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.339+0000 7f25e9d82000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f25e0203440 con 0x555c7bb4c470 2024-04-17T09:57:36.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.339+0000 7f25e9d82000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f25e01fa0b0 con 0x555c7bb90b50 2024-04-17T09:57:36.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.339+0000 7f25e9d82000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f25e0202fc0 con 0x555c7bb46cf0 2024-04-17T09:57:36.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.339+0000 7f25df400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb4c470 0x555c7bba5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.339+0000 7f25df400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb4c470 0x555c7bba5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59534/0 (socket says 172.21.15.32:59534) 2024-04-17T09:57:36.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.339+0000 7f25df400640 1 -- 172.21.15.32:0/2742823050 learned_addr learned my addr 172.21.15.32:0/2742823050 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:36.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.339+0000 7f25dfe00640 1 --2- 172.21.15.32:0/2742823050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c7bb90b50 0x555c7bba19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.339+0000 7f25d7e00640 1 --2- 172.21.15.32:0/2742823050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c7bb46cf0 0x555c7bba14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.339+0000 7f25df400640 1 -- 172.21.15.32:0/2742823050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c7bb46cf0 msgr2=0x555c7bba14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:36.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.339+0000 7f25df400640 1 --2- 172.21.15.32:0/2742823050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c7bb46cf0 0x555c7bba14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.339+0000 7f25df400640 1 -- 172.21.15.32:0/2742823050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c7bb90b50 msgr2=0x555c7bba19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:36.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.339+0000 7f25df400640 1 --2- 172.21.15.32:0/2742823050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c7bb90b50 0x555c7bba19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.339+0000 7f25df400640 1 -- 172.21.15.32:0/2742823050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c7ba15d30 con 0x555c7bb4c470 2024-04-17T09:57:36.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.339+0000 7f25dfe00640 1 --2- 172.21.15.32:0/2742823050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c7bb90b50 0x555c7bba19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:57:36.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.339+0000 7f25d7e00640 1 --2- 172.21.15.32:0/2742823050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c7bb46cf0 0x555c7bba14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:57:36.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.339+0000 7f25df400640 1 --2- 172.21.15.32:0/2742823050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb4c470 0x555c7bba5a50 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f25d000a010 tx=0x7f25d0025070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00977395ac1f6f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:36.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25cf400640 1 -- 172.21.15.32:0/2742823050 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d00159f0 con 0x555c7bb4c470 2024-04-17T09:57:36.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25cf400640 1 -- 172.21.15.32:0/2742823050 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d0027070 con 0x555c7bb4c470 2024-04-17T09:57:36.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25cf400640 1 -- 172.21.15.32:0/2742823050 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d002c770 con 0x555c7bb4c470 2024-04-17T09:57:36.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25e9d82000 1 -- 172.21.15.32:0/2742823050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb4c470 msgr2=0x555c7bba5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:36.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25e9d82000 1 --2- 172.21.15.32:0/2742823050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb4c470 0x555c7bba5a50 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f25d000a010 tx=0x7f25d0025070 comp rx=0 tx=0).stop 2024-04-17T09:57:36.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25e9d82000 1 -- 172.21.15.32:0/2742823050 shutdown_connections 2024-04-17T09:57:36.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25e9d82000 1 --2- 172.21.15.32:0/2742823050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c7bb46cf0 0x555c7bba14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25e9d82000 1 --2- 172.21.15.32:0/2742823050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c7bb90b50 0x555c7bba19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25e9d82000 1 --2- 172.21.15.32:0/2742823050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb4c470 0x555c7bba5a50 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25e9d82000 1 -- 172.21.15.32:0/2742823050 >> 172.21.15.32:0/2742823050 conn(0x555c7bb903d0 msgr2=0x555c7bb7df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25e9d82000 1 -- 172.21.15.32:0/2742823050 shutdown_connections 2024-04-17T09:57:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25e9d82000 1 -- 172.21.15.32:0/2742823050 wait complete. 2024-04-17T09:57:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25e9d82000 1 -- start start 2024-04-17T09:57:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25e9d82000 1 Processor -- start 2024-04-17T09:57:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25e9d82000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c7bb46cf0 0x555c7bba4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25e9d82000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb4c470 0x555c7bbbc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25e9d82000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c7bb90b50 0x555c7bbbc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25e9d82000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555c7b998b00 con 0x555c7bb4c470 2024-04-17T09:57:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25e9d82000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555c7b9937e0 con 0x555c7bb90b50 2024-04-17T09:57:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25e9d82000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x555c7b9a16e0 con 0x555c7bb46cf0 2024-04-17T09:57:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25e9d82000 10 client.0 _ll_register_callbacks cb 0x7f25d0014c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:57:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25d7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb4c470 0x555c7bbbc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25d7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb4c470 0x555c7bbbc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59538/0 (socket says 172.21.15.32:59538) 2024-04-17T09:57:36.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25d7e00640 1 -- 172.21.15.32:0/1805473730 learned_addr learned my addr 172.21.15.32:0/1805473730 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:36.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25dfe00640 1 --2- 172.21.15.32:0/1805473730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c7bb90b50 0x555c7bbbc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25df400640 1 --2- 172.21.15.32:0/1805473730 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c7bb46cf0 0x555c7bba4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25d7e00640 1 -- 172.21.15.32:0/1805473730 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c7bb46cf0 msgr2=0x555c7bba4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:36.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25d7e00640 1 --2- 172.21.15.32:0/1805473730 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c7bb46cf0 0x555c7bba4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25d7e00640 1 -- 172.21.15.32:0/1805473730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c7bb90b50 msgr2=0x555c7bbbc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:36.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25d7e00640 1 --2- 172.21.15.32:0/1805473730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c7bb90b50 0x555c7bbbc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25d7e00640 1 -- 172.21.15.32:0/1805473730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c7b9267c0 con 0x555c7bb4c470 2024-04-17T09:57:36.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25d7e00640 1 --2- 172.21.15.32:0/1805473730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb4c470 0x555c7bbbc030 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f25c801c000 tx=0x7f25c8017070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02b462211686db9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:36.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.343+0000 7f25ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:57:36.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25ccc00640 1 -- 172.21.15.32:0/1805473730 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c8033060 con 0x555c7bb4c470 2024-04-17T09:57:36.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e9d82000 1 -- 172.21.15.32:0/1805473730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25d0027070 con 0x555c7bb4c470 2024-04-17T09:57:36.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25ccc00640 1 -- 172.21.15.32:0/1805473730 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7f25c80059a0 con 0x555c7bb4c470 2024-04-17T09:57:36.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25ccc00640 1 -- 172.21.15.32:0/1805473730 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f25c8039050 con 0x555c7bb4c470 2024-04-17T09:57:36.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e9d82000 10 client.7071 fetch_fsmap learned FSMap version 48 2024-04-17T09:57:36.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e9d82000 1 -- 172.21.15.32:0/1805473730 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) -- 0x555c7b939650 con 0x555c7bb4c470 2024-04-17T09:57:36.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25ccc00640 1 -- 172.21.15.32:0/1805473730 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d0027070 con 0x555c7bb4c470 2024-04-17T09:57:36.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25ccc00640 1 -- 172.21.15.32:0/1805473730 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 48) ==== 3495+0+0 (secure 0 0 0) 0x7f25c803c020 con 0x555c7bb4c470 2024-04-17T09:57:36.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e9d82000 10 client.7071 fetch_fsmap finished waiting for FSMap version 48 2024-04-17T09:57:36.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e9d82000 10 client.7071 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T09:57:36.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e9d82000 20 client.7071 populate_metadata read hostname 'smithi032' 2024-04-17T09:57:36.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e9d82000 1 --2- 172.21.15.32:0/1805473730 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x555c7bb91470 0x555c7bbbcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e9d82000 4 client.7071 mds_command: new command op to 7011 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T09:57:36.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e9d82000 1 -- 172.21.15.32:0/1805473730 --> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] -- command(tid 0: {"prefix": "status"}) -- 0x555c7b9a1ef0 con 0x555c7bb91470 2024-04-17T09:57:36.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25df400640 1 --2- 172.21.15.32:0/1805473730 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x555c7bb91470 0x555c7bbbcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25df400640 1 --2- 172.21.15.32:0/1805473730 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x555c7bb91470 0x555c7bbbcab0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f69798c8db1b23a server_cookie=1b2cede4d8b950b9 in_seq=0 out_seq=0 2024-04-17T09:57:36.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25ccc00640 10 client.7071 ms_handle_connect on v2:172.21.15.32:6834/2416784965 2024-04-17T09:57:36.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25ccc00640 1 -- 172.21.15.32:0/1805473730 <== mds.0 v2:172.21.15.32:6834/2416784965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x555c7b9a1ef0 con 0x555c7bb91470 2024-04-17T09:57:36.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25ccc00640 10 client.7071 handle_command_reply: tid=0 2024-04-17T09:57:36.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e9d82000 1 client.7071 shutdown 2024-04-17T09:57:36.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e9d82000 1 -- 172.21.15.32:0/1805473730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb4c470 msgr2=0x555c7bbbc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:36.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e9d82000 1 --2- 172.21.15.32:0/1805473730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb4c470 0x555c7bbbc030 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f25c801c000 tx=0x7f25c8017070 comp rx=0 tx=0).stop 2024-04-17T09:57:36.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e9d82000 1 -- 172.21.15.32:0/1805473730 shutdown_connections 2024-04-17T09:57:36.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e9d82000 1 --2- 172.21.15.32:0/1805473730 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x555c7bb91470 0x555c7bbbcab0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e9d82000 1 --2- 172.21.15.32:0/1805473730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c7bb90b50 0x555c7bbbc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e9d82000 1 --2- 172.21.15.32:0/1805473730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555c7bb4c470 0x555c7bbbc030 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25ccc00640 0 client.7071 ms_handle_reset on v2:172.21.15.32:6834/2416784965 2024-04-17T09:57:36.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e9d82000 1 --2- 172.21.15.32:0/1805473730 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555c7bb46cf0 0x555c7bba4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e9d82000 1 -- 172.21.15.32:0/1805473730 >> 172.21.15.32:0/1805473730 conn(0x555c7bb903d0 msgr2=0x555c7bb7e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:36.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e9d82000 1 -- 172.21.15.32:0/1805473730 shutdown_connections 2024-04-17T09:57:36.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e9d82000 1 -- 172.21.15.32:0/1805473730 wait complete. 2024-04-17T09:57:36.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e9d82000 20 client.7071 trim_cache size 0 max 16384 2024-04-17T09:57:36.354 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:57:36.355 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "ff178df8-4c06-4367-a1e9-2e3a66d870d0", 2024-04-17T09:57:36.355 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-17T09:57:36.355 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 7011, 2024-04-17T09:57:36.355 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-17T09:57:36.355 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-17T09:57:36.355 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-17T09:57:36.355 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 4.5908579380000001, 2024-04-17T09:57:36.355 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 48, 2024-04-17T09:57:36.355 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 69, 2024-04-17T09:57:36.355 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 69, 2024-04-17T09:57:36.355 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 5.5391824950000004 2024-04-17T09:57:36.356 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:57:36.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e6800640 1 -- 172.21.15.32:0/2792073900 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f25bc068380 msgr2=0x7f25bc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:36.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e6800640 1 --2- 172.21.15.32:0/2792073900 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f25bc068380 0x7f25bc06a840 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009be0 tx=0x7f25d0009190 comp rx=0 tx=0).stop 2024-04-17T09:57:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e6800640 1 -- 172.21.15.32:0/2792073900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e00b05f0 msgr2=0x7f25e00b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e6800640 1 --2- 172.21.15.32:0/2792073900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e00b05f0 0x7f25e00b0a70 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f25d801d1f0 tx=0x7f25c4005840 comp rx=0 tx=0).stop 2024-04-17T09:57:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e6800640 1 -- 172.21.15.32:0/2792073900 shutdown_connections 2024-04-17T09:57:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e6800640 1 --2- 172.21.15.32:0/2792073900 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f25bc068380 0x7f25bc06a840 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e6800640 1 --2- 172.21.15.32:0/2792073900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e00b05f0 0x7f25e00b0a70 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e6800640 1 --2- 172.21.15.32:0/2792073900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25e01f7780 0x7f25e01f7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e6800640 1 --2- 172.21.15.32:0/2792073900 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25c4004d50 0x7f25e015d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e6800640 1 -- 172.21.15.32:0/2792073900 >> 172.21.15.32:0/2792073900 conn(0x7f25e00b4280 msgr2=0x7f25e00cf210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e6800640 1 -- 172.21.15.32:0/2792073900 shutdown_connections 2024-04-17T09:57:36.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.347+0000 7f25e6800640 1 -- 172.21.15.32:0/2792073900 wait complete. 2024-04-17T09:57:36.369 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T09:57:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.507+0000 7f53f7e00640 1 Processor -- start 2024-04-17T09:57:36.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.507+0000 7f53f7e00640 1 -- start start 2024-04-17T09:57:36.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.507+0000 7f53f7e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f53f8158910 0x7f53f8158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.507+0000 7f53f7e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53f815eb80 0x7f53f815f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.507+0000 7f53f7e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53f815f6b0 con 0x7f53f80b3110 2024-04-17T09:57:36.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.507+0000 7f53f7e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f53f80721a0 con 0x7f53f815eb80 2024-04-17T09:57:36.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.507+0000 7f53f7e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f53f815f990 con 0x7f53f8158910 2024-04-17T09:57:36.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.511+0000 7f53ee000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53f815eb80 0x7f53f815f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.511+0000 7f53ee000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53f815eb80 0x7f53f815f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59552/0 (socket says 172.21.15.32:59552) 2024-04-17T09:57:36.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.511+0000 7f53ee000640 1 -- 172.21.15.32:0/1960187974 learned_addr learned my addr 172.21.15.32:0/1960187974 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:36.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.511+0000 7f53f6a00640 1 --2- 172.21.15.32:0/1960187974 >> v2:172.21.15.144:3300/0 conn(0x7f53f8158910 0x7f53f8158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.511+0000 7f53ee000640 1 -- 172.21.15.32:0/1960187974 >> v2:172.21.15.144:3300/0 conn(0x7f53f8158910 msgr2=0x7f53f8158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:36.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.511+0000 7f53ee000640 1 --2- 172.21.15.32:0/1960187974 >> v2:172.21.15.144:3300/0 conn(0x7f53f8158910 0x7f53f8158cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.511+0000 7f53ee000640 1 -- 172.21.15.32:0/1960187974 >> v1:172.21.15.144:6789/0 conn(0x7f53f80b3110 legacy=0x7f53f8072630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:36.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.511+0000 7f53ee000640 1 -- 172.21.15.32:0/1960187974 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f8071e30 con 0x7f53f815eb80 2024-04-17T09:57:36.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.511+0000 7f53ee000640 1 --2- 172.21.15.32:0/1960187974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f815eb80 0x7f53f815f000 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f53f80d1260 tx=0x7f53e001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1796ead68bcbff5a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:36.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.511+0000 7f53f6000640 1 -- 172.21.15.32:0/1960187974 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e00b0070 con 0x7f53f815eb80 2024-04-17T09:57:36.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.511+0000 7f53f6000640 1 -- 172.21.15.32:0/1960187974 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e00ab600 con 0x7f53f815eb80 2024-04-17T09:57:36.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.511+0000 7f53f6000640 1 -- 172.21.15.32:0/1960187974 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e00ab9d0 con 0x7f53f815eb80 2024-04-17T09:57:36.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53f7e00640 1 -- 172.21.15.32:0/1960187974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f815eb80 msgr2=0x7f53f815f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:36.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53f7e00640 1 --2- 172.21.15.32:0/1960187974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f815eb80 0x7f53f815f000 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f53f80d1260 tx=0x7f53e001c520 comp rx=0 tx=0).stop 2024-04-17T09:57:36.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53f7e00640 1 -- 172.21.15.32:0/1960187974 shutdown_connections 2024-04-17T09:57:36.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53f7e00640 1 --2- 172.21.15.32:0/1960187974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f815eb80 0x7f53f815f000 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53f7e00640 1 --2- 172.21.15.32:0/1960187974 >> v2:172.21.15.144:3300/0 conn(0x7f53f8158910 0x7f53f8158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53f7e00640 1 -- 172.21.15.32:0/1960187974 >> 172.21.15.32:0/1960187974 conn(0x7f53f80d0180 msgr2=0x7f53f80d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:36.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53f7e00640 1 -- 172.21.15.32:0/1960187974 shutdown_connections 2024-04-17T09:57:36.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53f7e00640 1 -- 172.21.15.32:0/1960187974 wait complete. 2024-04-17T09:57:36.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53f7e00640 1 Processor -- start 2024-04-17T09:57:36.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53f7e00640 1 -- start start 2024-04-17T09:57:36.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53f7e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53f80b3110 0x7f53f80af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53f7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f8158910 0x7f53f80976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53f7e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53f815eb80 0x7f53f8097f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53f7e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f53f8144220 con 0x7f53f8158910 2024-04-17T09:57:36.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53f7e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f53f81440a0 con 0x7f53f80b3110 2024-04-17T09:57:36.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53f7e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f53f81443a0 con 0x7f53f815eb80 2024-04-17T09:57:36.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53ee000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f8158910 0x7f53f80976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53ee000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f8158910 0x7f53f80976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59560/0 (socket says 172.21.15.32:59560) 2024-04-17T09:57:36.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53f6a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53f80b3110 0x7f53f80af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53ee000640 1 -- 172.21.15.32:0/1791495771 learned_addr learned my addr 172.21.15.32:0/1791495771 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:36.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53f7400640 1 --2- 172.21.15.32:0/1791495771 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53f815eb80 0x7f53f8097f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53ee000640 1 -- 172.21.15.32:0/1791495771 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53f815eb80 msgr2=0x7f53f8097f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:36.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53ee000640 1 --2- 172.21.15.32:0/1791495771 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53f815eb80 0x7f53f8097f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53ee000640 1 -- 172.21.15.32:0/1791495771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53f80b3110 msgr2=0x7f53f80af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:36.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53ee000640 1 --2- 172.21.15.32:0/1791495771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53f80b3110 0x7f53f80af060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.515+0000 7f53ee000640 1 -- 172.21.15.32:0/1791495771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f8161810 con 0x7f53f8158910 2024-04-17T09:57:36.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.519+0000 7f53ee000640 1 --2- 172.21.15.32:0/1791495771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f8158910 0x7f53f80976e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f53e001abd0 tx=0x7f53e00192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:36.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.519+0000 7f53efe00640 1 -- 172.21.15.32:0/1791495771 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e00ba020 con 0x7f53f8158910 2024-04-17T09:57:36.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.519+0000 7f53efe00640 1 -- 172.21.15.32:0/1791495771 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e00b6420 con 0x7f53f8158910 2024-04-17T09:57:36.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.519+0000 7f53efe00640 1 -- 172.21.15.32:0/1791495771 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e00b0040 con 0x7f53f8158910 2024-04-17T09:57:36.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.519+0000 7f53f7e00640 1 -- 172.21.15.32:0/1791495771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f8161af0 con 0x7f53f8158910 2024-04-17T09:57:36.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.519+0000 7f53f7e00640 1 -- 172.21.15.32:0/1791495771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f80ae560 con 0x7f53f8158910 2024-04-17T09:57:36.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.519+0000 7f53f7e00640 1 -- 172.21.15.32:0/1791495771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f804fd00 con 0x7f53f8158910 2024-04-17T09:57:36.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.519+0000 7f53efe00640 1 -- 172.21.15.32:0/1791495771 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f53e00bd020 con 0x7f53f8158910 2024-04-17T09:57:36.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.519+0000 7f53efe00640 1 --2- 172.21.15.32:0/1791495771 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f53d4068380 0x7f53d406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:36.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.519+0000 7f53efe00640 1 -- 172.21.15.32:0/1791495771 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7f53e0120ae0 con 0x7f53f8158910 2024-04-17T09:57:36.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.519+0000 7f53f6a00640 1 --2- 172.21.15.32:0/1791495771 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f53d4068380 0x7f53d406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:36.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.523+0000 7f53f6a00640 1 --2- 172.21.15.32:0/1791495771 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f53d4068380 0x7f53d406a840 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f53f8072530 tx=0x7f53e80098a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:36.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.523+0000 7f53efe00640 1 -- 172.21.15.32:0/1791495771 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f53e00c4030 con 0x7f53f8158910 2024-04-17T09:57:36.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.723+0000 7f53f7e00640 1 -- 172.21.15.32:0/1791495771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f53f80cfef0 con 0x7f53f8158910 2024-04-17T09:57:36.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.723+0000 7f53efe00640 1 -- 172.21.15.32:0/1791495771 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) ==== 53+0+73 (secure 0 0 0) 0x7f53e0016110 con 0x7f53f8158910 2024-04-17T09:57:36.727 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T09:57:36.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.727+0000 7f53f7e00640 1 -- 172.21.15.32:0/1791495771 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f53d4068380 msgr2=0x7f53d406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:36.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.727+0000 7f53f7e00640 1 --2- 172.21.15.32:0/1791495771 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f53d4068380 0x7f53d406a840 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f53f8072530 tx=0x7f53e80098a0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.727+0000 7f53f7e00640 1 -- 172.21.15.32:0/1791495771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f8158910 msgr2=0x7f53f80976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:36.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.727+0000 7f53f7e00640 1 --2- 172.21.15.32:0/1791495771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f8158910 0x7f53f80976e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f53e001abd0 tx=0x7f53e00192c0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.727+0000 7f53f7e00640 1 -- 172.21.15.32:0/1791495771 shutdown_connections 2024-04-17T09:57:36.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.727+0000 7f53f7e00640 1 --2- 172.21.15.32:0/1791495771 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f53d4068380 0x7f53d406a840 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.727+0000 7f53f7e00640 1 --2- 172.21.15.32:0/1791495771 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53f815eb80 0x7f53f8097f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.727+0000 7f53f7e00640 1 --2- 172.21.15.32:0/1791495771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f8158910 0x7f53f80976e0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.727+0000 7f53f7e00640 1 --2- 172.21.15.32:0/1791495771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53f80b3110 0x7f53f80af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:36.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.727+0000 7f53f7e00640 1 -- 172.21.15.32:0/1791495771 >> 172.21.15.32:0/1791495771 conn(0x7f53f80d0180 msgr2=0x7f53f81434b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:36.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.727+0000 7f53f7e00640 1 -- 172.21.15.32:0/1791495771 shutdown_connections 2024-04-17T09:57:36.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36.727+0000 7f53f7e00640 1 -- 172.21.15.32:0/1791495771 wait complete. 2024-04-17T09:57:36.748 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T09:57:36.749 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T09:57:36.749 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-17T09:57:36.749 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T09:57:36.749 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T09:57:36.749 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T09:57:36.749 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T09:57:36.749 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T09:57:36.749 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:57:36.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:36.749 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-17T09:57:36.759 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T09:57:36.759 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T09:57:36.759 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T09:57:36.759 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:57:36.759 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-17T09:57:36.759 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:57:36.759 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:57:36.759 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-17T09:57:36.759 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-17T09:57:36.760 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T09:57:36.760 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:57:36.760 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-17T09:57:36.760 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-17T09:57:36.760 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42197sec preferred_lft 42197sec 2024-04-17T09:57:36.760 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2024-04-17T09:57:36.760 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:57:36.760 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:57:36.760 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-17T09:57:36.760 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-17T09:57:36.760 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:57:36.760 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:57:36.760 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-17T09:57:36.760 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T09:57:36.761 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:cd:f4:aa:5b brd ff:ff:ff:ff:ff:ff 2024-04-17T09:57:36.761 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T09:57:36.761 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:57:36.761 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T09:57:36.761 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T09:57:36.761 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T09:57:36.761 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:57:36.761 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T09:57:36.761 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:57:36.761 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T09:57:36.761 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:57:36.761 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T09:57:36.761 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:57:36.761 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T09:57:36.762 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:57:36.762 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T09:57:36.762 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:57:36.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:36.762 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T09:57:36.763 INFO:tasks.ceph.mds.a.smithi032.stderr:2024-04-17T09:57:36.759+0000 7f9ece800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T09:57:36.781 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-17T09:57:36.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:36.783 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-17T09:57:36.840 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-17T09:57:36.840 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T09:57:36.840 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-17T09:57:36.840 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T09:57:36.841 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-17T09:57:36.841 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T09:57:36.841 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:57:36.841 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T09:57:36.841 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-17T09:57:36.841 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-17T09:57:36.841 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-17T09:57:36.841 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T09:57:36.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:57:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:57:36.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:36.983 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-17T09:57:36.994 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T09:57:36.994 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T09:57:36.994 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:57:36.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:36.994 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-17T09:57:37.059 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T09:57:37.059 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:57:37.059 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T09:57:37.114 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:57:37.115 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:57:37.115 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T09:57:37.166 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T09:57:37.227 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:57:37.284 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T09:57:37.284 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T09:57:37.284 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T09:57:37.284 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T09:57:37.284 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T09:57:37.284 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T09:57:37.284 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T09:57:37.284 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T09:57:37.284 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T09:57:37.285 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T09:57:37.285 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T09:57:37.285 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T09:57:37.285 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:57:37.285 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T09:57:37.285 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T09:57:37.285 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T09:57:37.285 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T09:57:37.285 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T09:57:37.285 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T09:57:37.285 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T09:57:37.285 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:57:37.285 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T09:57:37.285 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T09:57:37.286 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T09:57:37.286 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T09:57:37.286 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T09:57:37.286 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T09:57:37.286 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T09:57:37.286 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T09:57:37.286 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:57:37.286 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:57:37.286 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:57:37.286 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:57:37.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:37.287 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T09:57:37.338 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T09:57:37.338 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T09:57:37.381 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T09:57:37.402 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:57:37.456 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T09:57:37.456 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T09:57:37.456 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T09:57:37.456 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T09:57:37.456 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T09:57:37.456 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T09:57:37.456 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T09:57:37.456 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T09:57:37.456 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T09:57:37.457 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T09:57:37.457 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T09:57:37.457 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T09:57:37.457 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:57:37.457 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T09:57:37.457 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T09:57:37.457 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T09:57:37.457 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T09:57:37.457 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T09:57:37.457 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T09:57:37.457 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T09:57:37.457 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:57:37.457 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T09:57:37.457 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T09:57:37.458 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T09:57:37.458 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T09:57:37.458 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T09:57:37.458 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T09:57:37.458 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T09:57:37.458 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T09:57:37.458 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:57:37.458 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:57:37.458 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:57:37.458 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:57:37.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:37.459 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T09:57:37.476 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2024-04-17T09:57:37.475+0000 7ff1f6cb8580 -1 init, newargv = 0x555658e25ea0 newargc=15 2024-04-17T09:57:37.476 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[42561]: starting ceph client 2024-04-17T09:57:37.484 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[42561]: starting fuse 2024-04-17T09:57:37.486 INFO:teuthology.orchestra.run.smithi032.stdout:44 2024-04-17T09:57:37.487 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-17T09:57:37.487 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2024-04-17T09:57:37.487 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2024-04-17T09:57:37.487 DEBUG:teuthology.orchestra.run.smithi032:> import re 2024-04-17T09:57:37.487 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T09:57:37.487 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2024-04-17T09:57:37.487 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:57:37.487 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2024-04-17T09:57:37.487 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T09:57:37.487 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2024-04-17T09:57:37.488 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T09:57:37.488 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:57:37.488 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2024-04-17T09:57:37.488 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2024-04-17T09:57:37.488 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2024-04-17T09:57:37.488 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2024-04-17T09:57:37.488 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:57:37.488 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2024-04-17T09:57:37.488 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T09:57:37.488 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T09:57:37.488 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T09:57:37.488 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2024-04-17T09:57:37.488 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2024-04-17T09:57:37.488 DEBUG:teuthology.orchestra.run.smithi032:> return f 2024-04-17T09:57:37.488 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T09:57:37.488 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:57:37.489 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2024-04-17T09:57:37.489 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T09:57:37.613 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.42561.asok 2024-04-17T09:57:37.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:57:37.625 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42561.asok 2024-04-17T09:57:37.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:37.625 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42561.asok status 2024-04-17T09:57:37.799 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:57:37.799 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2024-04-17T09:57:37.799 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-17T09:57:37.799 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T09:57:37.799 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2024-04-17T09:57:37.799 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2024-04-17T09:57:37.799 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T09:57:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "42561", 2024-04-17T09:57:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2024-04-17T09:57:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2024-04-17T09:57:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2024-04-17T09:57:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 7083, 2024-04-17T09:57:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2024-04-17T09:57:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2024-04-17T09:57:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2024-04-17T09:57:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 7083 2024-04-17T09:57:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T09:57:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T09:57:37.800 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T09:57:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3044125111 2024-04-17T09:57:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T09:57:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T09:57:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T09:57:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3044125111 2024-04-17T09:57:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:57:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.7083 192.168.0.1:0/3044125111", 2024-04-17T09:57:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/3044125111", 2024-04-17T09:57:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2024-04-17T09:57:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 48, 2024-04-17T09:57:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 69, 2024-04-17T09:57:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2024-04-17T09:57:37.801 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2024-04-17T09:57:37.802 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs" 2024-04-17T09:57:37.802 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:57:37.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:37.815 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2024-04-17T09:57:37.815 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T09:57:37.824 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2024-04-17T09:57:37.824 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T09:57:37.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:37.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T09:57:37.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:37.887 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-04-17T09:57:37.944 INFO:teuthology.orchestra.run.smithi032.stdout:file_undamaged 2024-04-17T09:57:37.946 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T09:57:37.946 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T09:57:37.946 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T09:57:37.946 DEBUG:teuthology.orchestra.run.smithi032:> import json 2024-04-17T09:57:37.946 DEBUG:teuthology.orchestra.run.smithi032:> import sys 2024-04-17T09:57:37.946 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:57:37.946 DEBUG:teuthology.orchestra.run.smithi032:> try: 2024-04-17T09:57:37.946 DEBUG:teuthology.orchestra.run.smithi032:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-04-17T09:57:37.946 DEBUG:teuthology.orchestra.run.smithi032:> except OSError as e: 2024-04-17T09:57:37.946 DEBUG:teuthology.orchestra.run.smithi032:> sys.exit(e.errno) 2024-04-17T09:57:37.947 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:57:37.947 DEBUG:teuthology.orchestra.run.smithi032:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-17T09:57:37.947 DEBUG:teuthology.orchestra.run.smithi032:> print(json.dumps( 2024-04-17T09:57:37.947 DEBUG:teuthology.orchestra.run.smithi032:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-17T09:57:37.947 DEBUG:teuthology.orchestra.run.smithi032:> indent=2)) 2024-04-17T09:57:37.947 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T09:57:38.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-17T09:57:38.071 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-17T09:57:38.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:38.071 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-17T09:57:38.142 INFO:teuthology.orchestra.run.smithi032.stdout:2 2024-04-17T09:57:38.142 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:57:38.142 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:57:38.195 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:57:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.379+0000 7f2bd9600640 1 Processor -- start 2024-04-17T09:57:38.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.379+0000 7f2bd9600640 1 -- start start 2024-04-17T09:57:38.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.379+0000 7f2bd9600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2bd40ce500 0x7f2bd40b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:38.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.379+0000 7f2bd9600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bd409b100 0x7f2bd40d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:38.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.379+0000 7f2bd9600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bd4070d10 con 0x7f2bd40ce8e0 2024-04-17T09:57:38.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.379+0000 7f2bd9600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2bd4070ff0 con 0x7f2bd40ce500 2024-04-17T09:57:38.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.379+0000 7f2bd9600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd4071340 con 0x7f2bd409b100 2024-04-17T09:57:38.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.379+0000 7f2bd3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2bd40ce500 0x7f2bd40b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:38.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.379+0000 7f2bcb400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2bd40ce8e0 0x7f2bd409a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53092/0 (socket says 172.21.15.32:53092) 2024-04-17T09:57:38.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.379+0000 7f2bcb400640 1 -- 172.21.15.32:0/1992874398 learned_addr learned my addr 172.21.15.32:0/1992874398 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:38.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.379+0000 7f2bd8c00640 1 --2- 172.21.15.32:0/1992874398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bd409b100 0x7f2bd40d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:38.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.379+0000 7f2bd3e00640 1 -- 172.21.15.32:0/1992874398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bd409b100 msgr2=0x7f2bd40d0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:38.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.379+0000 7f2bd3e00640 1 --2- 172.21.15.32:0/1992874398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bd409b100 0x7f2bd40d0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:38.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.379+0000 7f2bd3e00640 1 -- 172.21.15.32:0/1992874398 >> v1:172.21.15.32:6789/0 conn(0x7f2bd40ce8e0 legacy=0x7f2bd409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:38.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.379+0000 7f2bd3e00640 1 -- 172.21.15.32:0/1992874398 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd4071620 con 0x7f2bd40ce500 2024-04-17T09:57:38.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.379+0000 7f2bd3400640 1 -- 172.21.15.32:0/1992874398 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1234458195 0 0) 0x7f2bd4070d10 con 0x7f2bd40ce8e0 2024-04-17T09:57:38.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.383+0000 7f2bd3e00640 1 --2- 172.21.15.32:0/1992874398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd40ce500 0x7f2bd40b4970 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009d90 tx=0x7f2bc401c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676ecd58191d15c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:38.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.383+0000 7f2bd3400640 1 -- 172.21.15.32:0/1992874398 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc40b2070 con 0x7f2bd40ce500 2024-04-17T09:57:38.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.383+0000 7f2bd3400640 1 -- 172.21.15.32:0/1992874398 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bc40a5820 con 0x7f2bd40ce500 2024-04-17T09:57:38.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.383+0000 7f2bd3400640 1 -- 172.21.15.32:0/1992874398 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc40a5bf0 con 0x7f2bd40ce500 2024-04-17T09:57:38.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.383+0000 7f2bd9600640 1 -- 172.21.15.32:0/1992874398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd40ce500 msgr2=0x7f2bd40b4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:38.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.383+0000 7f2bd9600640 1 --2- 172.21.15.32:0/1992874398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd40ce500 0x7f2bd40b4970 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009d90 tx=0x7f2bc401c850 comp rx=0 tx=0).stop 2024-04-17T09:57:38.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.383+0000 7f2bd9600640 1 -- 172.21.15.32:0/1992874398 shutdown_connections 2024-04-17T09:57:38.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.383+0000 7f2bd9600640 1 --2- 172.21.15.32:0/1992874398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bd409b100 0x7f2bd40d0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:38.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.383+0000 7f2bd9600640 1 --2- 172.21.15.32:0/1992874398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd40ce500 0x7f2bd40b4970 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:38.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.383+0000 7f2bd9600640 1 -- 172.21.15.32:0/1992874398 >> 172.21.15.32:0/1992874398 conn(0x7f2bd40b3e50 msgr2=0x7f2bd40b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:38.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.383+0000 7f2bd9600640 1 -- 172.21.15.32:0/1992874398 shutdown_connections 2024-04-17T09:57:38.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.383+0000 7f2bd9600640 1 -- 172.21.15.32:0/1992874398 wait complete. 2024-04-17T09:57:38.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.383+0000 7f2bd9600640 1 Processor -- start 2024-04-17T09:57:38.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.383+0000 7f2bd9600640 1 -- start start 2024-04-17T09:57:38.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd9600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd409b100 0x7f2bd415ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:38.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd9600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bd40ce500 0x7f2bd40cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:38.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd9600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bd40ce8e0 0x7f2bd40b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:38.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd9600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd40cd550 con 0x7f2bd409b100 2024-04-17T09:57:38.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd9600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd4070a60 con 0x7f2bd40ce500 2024-04-17T09:57:38.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd9600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd40cd6d0 con 0x7f2bd40ce8e0 2024-04-17T09:57:38.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd409b100 0x7f2bd415ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:38.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd409b100 0x7f2bd415ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59574/0 (socket says 172.21.15.32:59574) 2024-04-17T09:57:38.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd3e00640 1 -- 172.21.15.32:0/2327469057 learned_addr learned my addr 172.21.15.32:0/2327469057 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:38.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bcb400640 1 --2- 172.21.15.32:0/2327469057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bd40ce500 0x7f2bd40cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:38.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd8c00640 1 --2- 172.21.15.32:0/2327469057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bd40ce8e0 0x7f2bd40b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:38.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd3e00640 1 -- 172.21.15.32:0/2327469057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bd40ce8e0 msgr2=0x7f2bd40b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:38.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd3e00640 1 --2- 172.21.15.32:0/2327469057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bd40ce8e0 0x7f2bd40b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:38.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd3e00640 1 -- 172.21.15.32:0/2327469057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bd40ce500 msgr2=0x7f2bd40cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd3e00640 1 --2- 172.21.15.32:0/2327469057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bd40ce500 0x7f2bd40cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd3e00640 1 -- 172.21.15.32:0/2327469057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd415c2a0 con 0x7f2bd409b100 2024-04-17T09:57:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd8c00640 1 --2- 172.21.15.32:0/2327469057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bd40ce8e0 0x7f2bd40b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:57:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bcb400640 1 --2- 172.21.15.32:0/2327469057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bd40ce500 0x7f2bd40cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:57:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd3e00640 1 --2- 172.21.15.32:0/2327469057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd409b100 0x7f2bd415ac00 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f2bc401a480 tx=0x7f2bc40195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd0c00640 1 -- 172.21.15.32:0/2327469057 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc40b2050 con 0x7f2bd409b100 2024-04-17T09:57:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd0c00640 1 -- 172.21.15.32:0/2327469057 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bc40be070 con 0x7f2bd409b100 2024-04-17T09:57:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd9600640 1 -- 172.21.15.32:0/2327469057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd415c580 con 0x7f2bd409b100 2024-04-17T09:57:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd0c00640 1 -- 172.21.15.32:0/2327469057 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4009450 con 0x7f2bd409b100 2024-04-17T09:57:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd9600640 1 -- 172.21.15.32:0/2327469057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd415b140 con 0x7f2bd409b100 2024-04-17T09:57:38.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd9600640 1 -- 172.21.15.32:0/2327469057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bd404fd00 con 0x7f2bd409b100 2024-04-17T09:57:38.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.387+0000 7f2bd0c00640 1 -- 172.21.15.32:0/2327469057 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2bc40adcc0 con 0x7f2bd409b100 2024-04-17T09:57:38.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.391+0000 7f2bd0c00640 1 --2- 172.21.15.32:0/2327469057 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2bb0068380 0x7f2bb006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:38.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.391+0000 7f2bd0c00640 1 -- 172.21.15.32:0/2327469057 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7f2bc4121c10 con 0x7f2bd409b100 2024-04-17T09:57:38.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.391+0000 7f2bcb400640 1 --2- 172.21.15.32:0/2327469057 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2bb0068380 0x7f2bb006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:38.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.391+0000 7f2bcb400640 1 --2- 172.21.15.32:0/2327469057 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2bb0068380 0x7f2bb006a840 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc007fd0 tx=0x7f2bbc007c20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:38.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.395+0000 7f2bd0c00640 1 -- 172.21.15.32:0/2327469057 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2bc4016560 con 0x7f2bd409b100 2024-04-17T09:57:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.559+0000 7f2bd9600640 1 -- 172.21.15.32:0/2327469057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2bd40cdad0 con 0x7f2bd409b100 2024-04-17T09:57:38.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.563+0000 7f2bd0c00640 1 -- 172.21.15.32:0/2327469057 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+16807 (secure 0 0 0) 0x7f2bc40ab340 con 0x7f2bd409b100 2024-04-17T09:57:38.564 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:57:38.564 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":69,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:57:30.746906+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:57:09.948964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-17T09:57:10.512460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6836/663537244":"2024-04-18T09:57:30.746884+0000","172.21.15.32:6834/663537244":"2024-04-18T09:57:30.746884+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:57:38.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.563+0000 7f2bd9600640 1 -- 172.21.15.32:0/2327469057 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2bb0068380 msgr2=0x7f2bb006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:38.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.563+0000 7f2bd9600640 1 --2- 172.21.15.32:0/2327469057 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2bb0068380 0x7f2bb006a840 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc007fd0 tx=0x7f2bbc007c20 comp rx=0 tx=0).stop 2024-04-17T09:57:38.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.563+0000 7f2bd9600640 1 -- 172.21.15.32:0/2327469057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd409b100 msgr2=0x7f2bd415ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:38.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.563+0000 7f2bd9600640 1 --2- 172.21.15.32:0/2327469057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd409b100 0x7f2bd415ac00 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f2bc401a480 tx=0x7f2bc40195f0 comp rx=0 tx=0).stop 2024-04-17T09:57:38.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.563+0000 7f2bd9600640 1 -- 172.21.15.32:0/2327469057 shutdown_connections 2024-04-17T09:57:38.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.563+0000 7f2bd9600640 1 --2- 172.21.15.32:0/2327469057 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2bb0068380 0x7f2bb006a840 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:38.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.563+0000 7f2bd9600640 1 --2- 172.21.15.32:0/2327469057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bd40ce8e0 0x7f2bd40b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:38.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.563+0000 7f2bd9600640 1 --2- 172.21.15.32:0/2327469057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bd40ce500 0x7f2bd40cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:38.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.563+0000 7f2bd9600640 1 --2- 172.21.15.32:0/2327469057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd409b100 0x7f2bd415ac00 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:38.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.563+0000 7f2bd9600640 1 -- 172.21.15.32:0/2327469057 >> 172.21.15.32:0/2327469057 conn(0x7f2bd40b3e50 msgr2=0x7f2bd4070020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:38.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.563+0000 7f2bd9600640 1 -- 172.21.15.32:0/2327469057 shutdown_connections 2024-04-17T09:57:38.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.563+0000 7f2bd9600640 1 -- 172.21.15.32:0/2327469057 wait complete. 2024-04-17T09:57:38.586 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:57:09.948964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:57:10.512460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:57:38.586 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:57:38.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.727+0000 7fed0e600640 1 Processor -- start 2024-04-17T09:57:38.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.727+0000 7fed0e600640 1 -- start start 2024-04-17T09:57:38.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.731+0000 7fed0e600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed0815b7d0 0x7fed0806e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:38.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.731+0000 7fed0e600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fed0806c140 0x7fed080ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:38.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.731+0000 7fed0e600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed080cc940 con 0x7fed0806f7c0 2024-04-17T09:57:38.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.731+0000 7fed0e600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fed080ccc20 con 0x7fed0806c140 2024-04-17T09:57:38.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.731+0000 7fed0e600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fed0806e0b0 con 0x7fed0815b7d0 2024-04-17T09:57:38.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.731+0000 7fed07e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fed0806c140 0x7fed080ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:38.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.731+0000 7fed07e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fed0806c140 0x7fed080ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59586/0 (socket says 172.21.15.32:59586) 2024-04-17T09:57:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.731+0000 7fed07e00640 1 -- 172.21.15.32:0/3971236229 learned_addr learned my addr 172.21.15.32:0/3971236229 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.731+0000 7fed07400640 1 --2- 172.21.15.32:0/3971236229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed0815b7d0 0x7fed0806e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.731+0000 7fed07e00640 1 -- 172.21.15.32:0/3971236229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed0815b7d0 msgr2=0x7fed0806e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.731+0000 7fed07e00640 1 --2- 172.21.15.32:0/3971236229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed0815b7d0 0x7fed0806e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:38.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.731+0000 7fed07e00640 1 -- 172.21.15.32:0/3971236229 >> v1:172.21.15.144:6789/0 conn(0x7fed0806f7c0 legacy=0x7fed0806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:38.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.731+0000 7fed07e00640 1 -- 172.21.15.32:0/3971236229 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0806e390 con 0x7fed0806c140 2024-04-17T09:57:38.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.731+0000 7fed07e00640 1 --2- 172.21.15.32:0/3971236229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed0806c140 0x7fed080ce290 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fed0000bb10 tx=0x7fed0001eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843410fddbc0f4ed server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:38.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fed06a00640 1 -- 172.21.15.32:0/3971236229 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed000b4070 con 0x7fed0806c140 2024-04-17T09:57:38.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fed06a00640 1 -- 172.21.15.32:0/3971236229 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed000a7c90 con 0x7fed0806c140 2024-04-17T09:57:38.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fed06a00640 1 -- 172.21.15.32:0/3971236229 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed000af660 con 0x7fed0806c140 2024-04-17T09:57:38.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fed0e600640 1 -- 172.21.15.32:0/3971236229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed0806c140 msgr2=0x7fed080ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:38.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fed0e600640 1 --2- 172.21.15.32:0/3971236229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed0806c140 0x7fed080ce290 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fed0000bb10 tx=0x7fed0001eba0 comp rx=0 tx=0).stop 2024-04-17T09:57:38.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fed0e600640 1 -- 172.21.15.32:0/3971236229 shutdown_connections 2024-04-17T09:57:38.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fed0e600640 1 --2- 172.21.15.32:0/3971236229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed0806c140 0x7fed080ce290 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:38.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fed0e600640 1 --2- 172.21.15.32:0/3971236229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed0815b7d0 0x7fed0806e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:38.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fed0e600640 1 -- 172.21.15.32:0/3971236229 >> 172.21.15.32:0/3971236229 conn(0x7fed080b2120 msgr2=0x7fed080b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:38.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fed0e600640 1 -- 172.21.15.32:0/3971236229 shutdown_connections 2024-04-17T09:57:38.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fed0e600640 1 -- 172.21.15.32:0/3971236229 wait complete. 2024-04-17T09:57:38.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fed0e600640 1 Processor -- start 2024-04-17T09:57:38.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fed0e600640 1 -- start start 2024-04-17T09:57:38.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fed0e600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed0806c140 0x7fed08157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:38.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fed0e600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed0806f7c0 0x7fed08158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:38.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fed0e600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed0815b7d0 0x7fed08158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:38.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fed0e600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fed08072270 con 0x7fed0806c140 2024-04-17T09:57:38.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fed0e600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fed080720f0 con 0x7fed0815b7d0 2024-04-17T09:57:38.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fed0e600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fed080723f0 con 0x7fed0806f7c0 2024-04-17T09:57:38.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fed07400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed0806c140 0x7fed08157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:38.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fed07400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed0806c140 0x7fed08157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59596/0 (socket says 172.21.15.32:59596) 2024-04-17T09:57:38.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fed07400640 1 -- 172.21.15.32:0/2751551855 learned_addr learned my addr 172.21.15.32:0/2751551855 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:38.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fed07e00640 1 --2- 172.21.15.32:0/2751551855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed0815b7d0 0x7fed08158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:38.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.735+0000 7fecfea00640 1 --2- 172.21.15.32:0/2751551855 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed0806f7c0 0x7fed08158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:38.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.739+0000 7fed07400640 1 -- 172.21.15.32:0/2751551855 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed0806f7c0 msgr2=0x7fed08158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:38.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.739+0000 7fed07400640 1 --2- 172.21.15.32:0/2751551855 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed0806f7c0 0x7fed08158310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:38.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.739+0000 7fed07400640 1 -- 172.21.15.32:0/2751551855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed0815b7d0 msgr2=0x7fed08158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:38.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.739+0000 7fed07400640 1 --2- 172.21.15.32:0/2751551855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed0815b7d0 0x7fed08158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:38.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.739+0000 7fed07400640 1 -- 172.21.15.32:0/2751551855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0814d7f0 con 0x7fed0806c140 2024-04-17T09:57:38.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.739+0000 7fed07400640 1 --2- 172.21.15.32:0/2751551855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed0806c140 0x7fed08157d30 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fecf8007e60 tx=0x7fecf800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:38.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.739+0000 7fecffe00640 1 -- 172.21.15.32:0/2751551855 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf8014620 con 0x7fed0806c140 2024-04-17T09:57:38.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.739+0000 7fecffe00640 1 -- 172.21.15.32:0/2751551855 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecf8019070 con 0x7fed0806c140 2024-04-17T09:57:38.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.739+0000 7fecffe00640 1 -- 172.21.15.32:0/2751551855 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf8014a20 con 0x7fed0806c140 2024-04-17T09:57:38.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.739+0000 7fed0e600640 1 -- 172.21.15.32:0/2751551855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed0814dad0 con 0x7fed0806c140 2024-04-17T09:57:38.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.739+0000 7fed0e600640 1 -- 172.21.15.32:0/2751551855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed0814e090 con 0x7fed0806c140 2024-04-17T09:57:38.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.739+0000 7fed0e600640 1 -- 172.21.15.32:0/2751551855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed08068a90 con 0x7fed0806c140 2024-04-17T09:57:38.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.739+0000 7fecffe00640 1 -- 172.21.15.32:0/2751551855 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fecf8014bc0 con 0x7fed0806c140 2024-04-17T09:57:38.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.743+0000 7fecffe00640 1 --2- 172.21.15.32:0/2751551855 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fece4068380 0x7fece406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:38.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.743+0000 7fecffe00640 1 -- 172.21.15.32:0/2751551855 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7fecf80871a0 con 0x7fed0806c140 2024-04-17T09:57:38.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.743+0000 7fecfea00640 1 --2- 172.21.15.32:0/2751551855 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fece4068380 0x7fece406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:38.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.743+0000 7fecfea00640 1 --2- 172.21.15.32:0/2751551855 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fece4068380 0x7fece406a840 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fed0814f8d0 tx=0x7fecf0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:38.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.747+0000 7fecffe00640 1 -- 172.21.15.32:0/2751551855 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fecf80570a0 con 0x7fed0806c140 2024-04-17T09:57:38.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.911+0000 7fed0e600640 1 -- 172.21.15.32:0/2751551855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fed0814ec90 con 0x7fed0806c140 2024-04-17T09:57:38.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.911+0000 7fecffe00640 1 -- 172.21.15.32:0/2751551855 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) ==== 93+0+10 (secure 0 0 0) 0x7fecf805b3a0 con 0x7fed0806c140 2024-04-17T09:57:38.913 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:57:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.915+0000 7fed0e600640 1 -- 172.21.15.32:0/2751551855 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fece4068380 msgr2=0x7fece406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.915+0000 7fed0e600640 1 --2- 172.21.15.32:0/2751551855 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fece4068380 0x7fece406a840 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fed0814f8d0 tx=0x7fecf0009040 comp rx=0 tx=0).stop 2024-04-17T09:57:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.915+0000 7fed0e600640 1 -- 172.21.15.32:0/2751551855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed0806c140 msgr2=0x7fed08157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.915+0000 7fed0e600640 1 --2- 172.21.15.32:0/2751551855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed0806c140 0x7fed08157d30 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fecf8007e60 tx=0x7fecf800e4b0 comp rx=0 tx=0).stop 2024-04-17T09:57:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.915+0000 7fed0e600640 1 -- 172.21.15.32:0/2751551855 shutdown_connections 2024-04-17T09:57:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.915+0000 7fed0e600640 1 --2- 172.21.15.32:0/2751551855 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fece4068380 0x7fece406a840 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.915+0000 7fed0e600640 1 --2- 172.21.15.32:0/2751551855 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed0806f7c0 0x7fed08158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.915+0000 7fed0e600640 1 --2- 172.21.15.32:0/2751551855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed0815b7d0 0x7fed08158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.915+0000 7fed0e600640 1 --2- 172.21.15.32:0/2751551855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed0806c140 0x7fed08157d30 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.915+0000 7fed0e600640 1 -- 172.21.15.32:0/2751551855 >> 172.21.15.32:0/2751551855 conn(0x7fed080b2120 msgr2=0x7fed08152af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.915+0000 7fed0e600640 1 -- 172.21.15.32:0/2751551855 shutdown_connections 2024-04-17T09:57:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:38.915+0000 7fed0e600640 1 -- 172.21.15.32:0/2751551855 wait complete. 2024-04-17T09:57:38.935 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:57:39.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.075+0000 7fdc97200640 1 Processor -- start 2024-04-17T09:57:39.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.075+0000 7fdc97200640 1 -- start start 2024-04-17T09:57:39.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.079+0000 7fdc97200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc9806f670 0x7fdc980cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:39.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.079+0000 7fdc97200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc98071e90 con 0x7fdc980cfc20 2024-04-17T09:57:39.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.079+0000 7fdc97200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc98072170 con 0x7fdc98144e90 2024-04-17T09:57:39.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.079+0000 7fdc97200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fdc98145270 con 0x7fdc9806f670 2024-04-17T09:57:39.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.079+0000 7fdc96800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdc980cfc20 0x7fdc980b3d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53106/0 (socket says 172.21.15.32:53106) 2024-04-17T09:57:39.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.079+0000 7fdc96800640 1 -- 172.21.15.32:0/584225981 learned_addr learned my addr 172.21.15.32:0/584225981 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:39.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.079+0000 7fdc8fe00640 1 --2- 172.21.15.32:0/584225981 >> v2:172.21.15.32:3300/0 conn(0x7fdc9806f670 0x7fdc980cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:39.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.079+0000 7fdc8fe00640 1 -- 172.21.15.32:0/584225981 >> v1:172.21.15.144:6789/0 conn(0x7fdc98144e90 legacy=0x7fdc9806ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:39.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.079+0000 7fdc8fe00640 1 -- 172.21.15.32:0/584225981 >> v1:172.21.15.32:6789/0 conn(0x7fdc980cfc20 legacy=0x7fdc980b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.079+0000 7fdc8fe00640 1 -- 172.21.15.32:0/584225981 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc98072450 con 0x7fdc9806f670 2024-04-17T09:57:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.079+0000 7fdc95400640 1 -- 172.21.15.32:0/584225981 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 274085804 0 0) 0x7fdc98071e90 con 0x7fdc980cfc20 2024-04-17T09:57:39.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc8fe00640 1 --2- 172.21.15.32:0/584225981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc9806f670 0x7fdc980cf6e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fdc800097c0 tx=0x7fdc8001c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d55355034eb1184 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:39.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc95400640 1 -- 172.21.15.32:0/584225981 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc800b0070 con 0x7fdc9806f670 2024-04-17T09:57:39.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc95400640 1 -- 172.21.15.32:0/584225981 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc80004230 con 0x7fdc9806f670 2024-04-17T09:57:39.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc95400640 1 -- 172.21.15.32:0/584225981 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80003620 con 0x7fdc9806f670 2024-04-17T09:57:39.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc97200640 1 -- 172.21.15.32:0/584225981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc9806f670 msgr2=0x7fdc980cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:39.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc97200640 1 --2- 172.21.15.32:0/584225981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc9806f670 0x7fdc980cf6e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fdc800097c0 tx=0x7fdc8001c350 comp rx=0 tx=0).stop 2024-04-17T09:57:39.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc97200640 1 -- 172.21.15.32:0/584225981 shutdown_connections 2024-04-17T09:57:39.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc97200640 1 --2- 172.21.15.32:0/584225981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc9806f670 0x7fdc980cf6e0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc97200640 1 -- 172.21.15.32:0/584225981 >> 172.21.15.32:0/584225981 conn(0x7fdc98067d70 msgr2=0x7fdc9806ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:39.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc97200640 1 -- 172.21.15.32:0/584225981 shutdown_connections 2024-04-17T09:57:39.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc97200640 1 -- 172.21.15.32:0/584225981 wait complete. 2024-04-17T09:57:39.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc97200640 1 Processor -- start 2024-04-17T09:57:39.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc97200640 1 -- start start 2024-04-17T09:57:39.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc97200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc9806f670 0x7fdc9813fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:39.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc97200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc980cfc20 0x7fdc9815e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:39.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc97200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc98144e90 0x7fdc9813eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:39.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc97200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdc980b4c60 con 0x7fdc980cfc20 2024-04-17T09:57:39.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc97200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdc980d0260 con 0x7fdc9806f670 2024-04-17T09:57:39.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc97200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdc980b4de0 con 0x7fdc98144e90 2024-04-17T09:57:39.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc8fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc980cfc20 0x7fdc9815e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:39.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc8fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc980cfc20 0x7fdc9815e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59606/0 (socket says 172.21.15.32:59606) 2024-04-17T09:57:39.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc8fe00640 1 -- 172.21.15.32:0/2211349499 learned_addr learned my addr 172.21.15.32:0/2211349499 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:39.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc96800640 1 --2- 172.21.15.32:0/2211349499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc98144e90 0x7fdc9813eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:39.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.083+0000 7fdc95e00640 1 --2- 172.21.15.32:0/2211349499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc9806f670 0x7fdc9813fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:39.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.087+0000 7fdc8fe00640 1 -- 172.21.15.32:0/2211349499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc98144e90 msgr2=0x7fdc9813eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:39.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.087+0000 7fdc8fe00640 1 --2- 172.21.15.32:0/2211349499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc98144e90 0x7fdc9813eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.087+0000 7fdc8fe00640 1 -- 172.21.15.32:0/2211349499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc9806f670 msgr2=0x7fdc9813fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:39.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.087+0000 7fdc8fe00640 1 --2- 172.21.15.32:0/2211349499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc9806f670 0x7fdc9813fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.087+0000 7fdc8fe00640 1 -- 172.21.15.32:0/2211349499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9815ccb0 con 0x7fdc980cfc20 2024-04-17T09:57:39.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.087+0000 7fdc8fe00640 1 --2- 172.21.15.32:0/2211349499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc980cfc20 0x7fdc9815e610 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fdc800ae040 tx=0x7fdc800190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:39.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.087+0000 7fdc8e000640 1 -- 172.21.15.32:0/2211349499 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc800b9070 con 0x7fdc980cfc20 2024-04-17T09:57:39.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.087+0000 7fdc8e000640 1 -- 172.21.15.32:0/2211349499 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc800043e0 con 0x7fdc980cfc20 2024-04-17T09:57:39.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.087+0000 7fdc97200640 1 -- 172.21.15.32:0/2211349499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc9815cf90 con 0x7fdc980cfc20 2024-04-17T09:57:39.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.087+0000 7fdc8e000640 1 -- 172.21.15.32:0/2211349499 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc800b0040 con 0x7fdc980cfc20 2024-04-17T09:57:39.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.087+0000 7fdc97200640 1 -- 172.21.15.32:0/2211349499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc9813f820 con 0x7fdc980cfc20 2024-04-17T09:57:39.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.087+0000 7fdc97200640 1 -- 172.21.15.32:0/2211349499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc9804fda0 con 0x7fdc980cfc20 2024-04-17T09:57:39.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.087+0000 7fdc8e000640 1 -- 172.21.15.32:0/2211349499 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdc800bc070 con 0x7fdc980cfc20 2024-04-17T09:57:39.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.091+0000 7fdc8e000640 1 --2- 172.21.15.32:0/2211349499 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdc74068380 0x7fdc7406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:39.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.091+0000 7fdc8e000640 1 -- 172.21.15.32:0/2211349499 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7fdc80120c70 con 0x7fdc980cfc20 2024-04-17T09:57:39.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.091+0000 7fdc95e00640 1 --2- 172.21.15.32:0/2211349499 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdc74068380 0x7fdc7406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:39.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.091+0000 7fdc95e00640 1 --2- 172.21.15.32:0/2211349499 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdc74068380 0x7fdc7406a840 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdc9815b770 tx=0x7fdc88009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:39.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.095+0000 7fdc8e000640 1 -- 172.21.15.32:0/2211349499 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fdc80016090 con 0x7fdc980cfc20 2024-04-17T09:57:39.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.259+0000 7fdc97200640 1 -- 172.21.15.32:0/2211349499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdc98166fd0 con 0x7fdc980cfc20 2024-04-17T09:57:39.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.259+0000 7fdc8e000640 1 -- 172.21.15.32:0/2211349499 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) ==== 104+0+11 (secure 0 0 0) 0x7fdc800f11c0 con 0x7fdc980cfc20 2024-04-17T09:57:39.261 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:57:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.259+0000 7fdc97200640 1 -- 172.21.15.32:0/2211349499 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdc74068380 msgr2=0x7fdc7406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.259+0000 7fdc97200640 1 --2- 172.21.15.32:0/2211349499 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdc74068380 0x7fdc7406a840 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdc9815b770 tx=0x7fdc88009040 comp rx=0 tx=0).stop 2024-04-17T09:57:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.259+0000 7fdc97200640 1 -- 172.21.15.32:0/2211349499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc980cfc20 msgr2=0x7fdc9815e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.259+0000 7fdc97200640 1 --2- 172.21.15.32:0/2211349499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc980cfc20 0x7fdc9815e610 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fdc800ae040 tx=0x7fdc800190f0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.259+0000 7fdc97200640 1 -- 172.21.15.32:0/2211349499 shutdown_connections 2024-04-17T09:57:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.259+0000 7fdc97200640 1 --2- 172.21.15.32:0/2211349499 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdc74068380 0x7fdc7406a840 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.259+0000 7fdc97200640 1 --2- 172.21.15.32:0/2211349499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc98144e90 0x7fdc9813eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.259+0000 7fdc97200640 1 --2- 172.21.15.32:0/2211349499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc980cfc20 0x7fdc9815e610 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.259+0000 7fdc97200640 1 --2- 172.21.15.32:0/2211349499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc9806f670 0x7fdc9813fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.259+0000 7fdc97200640 1 -- 172.21.15.32:0/2211349499 >> 172.21.15.32:0/2211349499 conn(0x7fdc98067d70 msgr2=0x7fdc98074b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.259+0000 7fdc97200640 1 -- 172.21.15.32:0/2211349499 shutdown_connections 2024-04-17T09:57:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.259+0000 7fdc97200640 1 -- 172.21.15.32:0/2211349499 wait complete. 2024-04-17T09:57:39.282 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:57:39.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.423+0000 7fd678200640 1 Processor -- start 2024-04-17T09:57:39.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.427+0000 7fd678200640 1 -- start start 2024-04-17T09:57:39.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.427+0000 7fd678200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd67006ba30 0x7fd67006be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:39.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.427+0000 7fd678200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd6700cf370 0x7fd6700cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:39.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.427+0000 7fd678200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd670160350 0x7fd6701607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:39.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.427+0000 7fd678200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd67006c350 con 0x7fd6700cf370 2024-04-17T09:57:39.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.427+0000 7fd678200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd6700d0130 con 0x7fd670160350 2024-04-17T09:57:39.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.427+0000 7fd678200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd67015d090 con 0x7fd67006ba30 2024-04-17T09:57:39.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.427+0000 7fd674e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd6700cf370 0x7fd6700cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:39.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.427+0000 7fd674e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd6700cf370 0x7fd6700cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59614/0 (socket says 172.21.15.32:59614) 2024-04-17T09:57:39.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.427+0000 7fd674e00640 1 -- 172.21.15.32:0/3361002164 learned_addr learned my addr 172.21.15.32:0/3361002164 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:39.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.427+0000 7fd675800640 1 --2- 172.21.15.32:0/3361002164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd67006ba30 0x7fd67006be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:39.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.427+0000 7fd676200640 1 --2- 172.21.15.32:0/3361002164 >> v2:172.21.15.144:3300/0 conn(0x7fd670160350 0x7fd6701607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:39.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.427+0000 7fd674e00640 1 -- 172.21.15.32:0/3361002164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd67006ba30 msgr2=0x7fd67006be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:39.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.427+0000 7fd674e00640 1 --2- 172.21.15.32:0/3361002164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd67006ba30 0x7fd67006be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.427+0000 7fd674e00640 1 -- 172.21.15.32:0/3361002164 >> v2:172.21.15.144:3300/0 conn(0x7fd670160350 msgr2=0x7fd6701607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:39.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.427+0000 7fd674e00640 1 --2- 172.21.15.32:0/3361002164 >> v2:172.21.15.144:3300/0 conn(0x7fd670160350 0x7fd6701607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.427+0000 7fd674e00640 1 -- 172.21.15.32:0/3361002164 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67015d370 con 0x7fd6700cf370 2024-04-17T09:57:39.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.431+0000 7fd674e00640 1 --2- 172.21.15.32:0/3361002164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6700cf370 0x7fd6700cf7e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd6700b58d0 tx=0x7fd65c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5262ae244fcc1eb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:39.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.431+0000 7fd66fe00640 1 -- 172.21.15.32:0/3361002164 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c0a7070 con 0x7fd6700cf370 2024-04-17T09:57:39.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.431+0000 7fd66fe00640 1 -- 172.21.15.32:0/3361002164 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd65c0b0070 con 0x7fd6700cf370 2024-04-17T09:57:39.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.431+0000 7fd66fe00640 1 -- 172.21.15.32:0/3361002164 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c0ab890 con 0x7fd6700cf370 2024-04-17T09:57:39.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.431+0000 7fd678200640 1 -- 172.21.15.32:0/3361002164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6700cf370 msgr2=0x7fd6700cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:39.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.431+0000 7fd678200640 1 --2- 172.21.15.32:0/3361002164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6700cf370 0x7fd6700cf7e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd6700b58d0 tx=0x7fd65c01c4f0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.431+0000 7fd678200640 1 -- 172.21.15.32:0/3361002164 shutdown_connections 2024-04-17T09:57:39.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.431+0000 7fd678200640 1 --2- 172.21.15.32:0/3361002164 >> v2:172.21.15.144:3300/0 conn(0x7fd670160350 0x7fd6701607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.431+0000 7fd678200640 1 --2- 172.21.15.32:0/3361002164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6700cf370 0x7fd6700cf7e0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.431+0000 7fd678200640 1 --2- 172.21.15.32:0/3361002164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd67006ba30 0x7fd67006be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.431+0000 7fd678200640 1 -- 172.21.15.32:0/3361002164 >> 172.21.15.32:0/3361002164 conn(0x7fd6700b2120 msgr2=0x7fd6700b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:39.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.431+0000 7fd678200640 1 -- 172.21.15.32:0/3361002164 shutdown_connections 2024-04-17T09:57:39.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.431+0000 7fd678200640 1 -- 172.21.15.32:0/3361002164 wait complete. 2024-04-17T09:57:39.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.431+0000 7fd678200640 1 Processor -- start 2024-04-17T09:57:39.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.431+0000 7fd678200640 1 -- start start 2024-04-17T09:57:39.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.435+0000 7fd678200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd67006ba30 0x7fd6700b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:39.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.435+0000 7fd678200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6700cf370 0x7fd6700b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:39.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.435+0000 7fd678200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd670160350 0x7fd67013ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:39.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.435+0000 7fd678200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd6700cfea0 con 0x7fd670160350 2024-04-17T09:57:39.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.435+0000 7fd678200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd6700cfd20 con 0x7fd67006ba30 2024-04-17T09:57:39.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.435+0000 7fd678200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd67015db70 con 0x7fd6700cf370 2024-04-17T09:57:39.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.435+0000 7fd676200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd670160350 0x7fd67013ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:39.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.435+0000 7fd676200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd670160350 0x7fd67013ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59622/0 (socket says 172.21.15.32:59622) 2024-04-17T09:57:39.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.435+0000 7fd676200640 1 -- 172.21.15.32:0/4161262288 learned_addr learned my addr 172.21.15.32:0/4161262288 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:39.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.435+0000 7fd674e00640 1 --2- 172.21.15.32:0/4161262288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6700cf370 0x7fd6700b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:39.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.435+0000 7fd675800640 1 --2- 172.21.15.32:0/4161262288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd67006ba30 0x7fd6700b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:39.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.435+0000 7fd676200640 1 -- 172.21.15.32:0/4161262288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6700cf370 msgr2=0x7fd6700b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:39.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.435+0000 7fd676200640 1 --2- 172.21.15.32:0/4161262288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6700cf370 0x7fd6700b0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.435+0000 7fd676200640 1 -- 172.21.15.32:0/4161262288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd67006ba30 msgr2=0x7fd6700b0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:39.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.435+0000 7fd676200640 1 --2- 172.21.15.32:0/4161262288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd67006ba30 0x7fd6700b0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.435+0000 7fd676200640 1 -- 172.21.15.32:0/4161262288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6700b1760 con 0x7fd670160350 2024-04-17T09:57:39.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.435+0000 7fd676200640 1 --2- 172.21.15.32:0/4161262288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd670160350 0x7fd67013ea70 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fd668007e90 tx=0x7fd66800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:39.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.435+0000 7fd66d600640 1 -- 172.21.15.32:0/4161262288 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668014420 con 0x7fd670160350 2024-04-17T09:57:39.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.435+0000 7fd66d600640 1 -- 172.21.15.32:0/4161262288 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd668019070 con 0x7fd670160350 2024-04-17T09:57:39.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.435+0000 7fd678200640 1 -- 172.21.15.32:0/4161262288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6700b19d0 con 0x7fd670160350 2024-04-17T09:57:39.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.435+0000 7fd66d600640 1 -- 172.21.15.32:0/4161262288 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668014800 con 0x7fd670160350 2024-04-17T09:57:39.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.435+0000 7fd678200640 1 -- 172.21.15.32:0/4161262288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd67013e550 con 0x7fd670160350 2024-04-17T09:57:39.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.435+0000 7fd678200640 1 -- 172.21.15.32:0/4161262288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd670068a90 con 0x7fd670160350 2024-04-17T09:57:39.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.439+0000 7fd66d600640 1 -- 172.21.15.32:0/4161262288 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd6680149a0 con 0x7fd670160350 2024-04-17T09:57:39.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.439+0000 7fd66d600640 1 --2- 172.21.15.32:0/4161262288 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd650068380 0x7fd65006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:39.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.439+0000 7fd66d600640 1 -- 172.21.15.32:0/4161262288 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7fd6680870e0 con 0x7fd670160350 2024-04-17T09:57:39.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.439+0000 7fd675800640 1 --2- 172.21.15.32:0/4161262288 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd650068380 0x7fd65006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:39.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.439+0000 7fd675800640 1 --2- 172.21.15.32:0/4161262288 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd650068380 0x7fd65006a840 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd6640045f0 tx=0x7fd664008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:39.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.443+0000 7fd66d600640 1 -- 172.21.15.32:0/4161262288 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd668056fe0 con 0x7fd670160350 2024-04-17T09:57:39.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.611+0000 7fd678200640 1 -- 172.21.15.32:0/4161262288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd67006c850 con 0x7fd670160350 2024-04-17T09:57:39.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.611+0000 7fd66d600640 1 -- 172.21.15.32:0/4161262288 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) ==== 100+0+11 (secure 0 0 0) 0x7fd66805b2e0 con 0x7fd670160350 2024-04-17T09:57:39.612 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:57:39.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.611+0000 7fd678200640 1 -- 172.21.15.32:0/4161262288 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd650068380 msgr2=0x7fd65006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:39.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.611+0000 7fd678200640 1 --2- 172.21.15.32:0/4161262288 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd650068380 0x7fd65006a840 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd6640045f0 tx=0x7fd664008040 comp rx=0 tx=0).stop 2024-04-17T09:57:39.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.611+0000 7fd678200640 1 -- 172.21.15.32:0/4161262288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd670160350 msgr2=0x7fd67013ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:39.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.611+0000 7fd678200640 1 --2- 172.21.15.32:0/4161262288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd670160350 0x7fd67013ea70 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fd668007e90 tx=0x7fd66800e4b0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.611+0000 7fd678200640 1 -- 172.21.15.32:0/4161262288 shutdown_connections 2024-04-17T09:57:39.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.611+0000 7fd678200640 1 --2- 172.21.15.32:0/4161262288 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd650068380 0x7fd65006a840 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.611+0000 7fd678200640 1 --2- 172.21.15.32:0/4161262288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6700cf370 0x7fd6700b0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.611+0000 7fd678200640 1 --2- 172.21.15.32:0/4161262288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd670160350 0x7fd67013ea70 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.611+0000 7fd678200640 1 --2- 172.21.15.32:0/4161262288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd67006ba30 0x7fd6700b0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.611+0000 7fd678200640 1 -- 172.21.15.32:0/4161262288 >> 172.21.15.32:0/4161262288 conn(0x7fd6700b2120 msgr2=0x7fd6700ce130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:39.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.611+0000 7fd678200640 1 -- 172.21.15.32:0/4161262288 shutdown_connections 2024-04-17T09:57:39.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.611+0000 7fd678200640 1 -- 172.21.15.32:0/4161262288 wait complete. 2024-04-17T09:57:39.635 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T09:57:39.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.775+0000 7f84ce200640 1 Processor -- start 2024-04-17T09:57:39.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.775+0000 7f84ce200640 1 -- start start 2024-04-17T09:57:39.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.775+0000 7f84ce200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84c8155a70 0x7f84c815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:39.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.775+0000 7f84ce200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f84c80b3340 0x7f84c815c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:39.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.775+0000 7f84ce200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84c815f870 con 0x7f84c8155e50 2024-04-17T09:57:39.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.775+0000 7f84ce200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f84c80b42a0 con 0x7f84c8155a70 2024-04-17T09:57:39.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.775+0000 7f84ce200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f84c80d1250 con 0x7f84c80b3340 2024-04-17T09:57:39.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.775+0000 7f84bfe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f84c8155e50 0x7f84c815d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53116/0 (socket says 172.21.15.32:53116) 2024-04-17T09:57:39.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.775+0000 7f84bfe00640 1 -- 172.21.15.32:0/474784468 learned_addr learned my addr 172.21.15.32:0/474784468 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:39.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.775+0000 7f84c7400640 1 --2- 172.21.15.32:0/474784468 >> v2:172.21.15.32:3300/0 conn(0x7f84c8155a70 0x7f84c815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:39.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.775+0000 7f84c7e00640 1 --2- 172.21.15.32:0/474784468 >> v2:172.21.15.144:3300/0 conn(0x7f84c80b3340 0x7f84c815c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:39.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.775+0000 7f84c6a00640 1 -- 172.21.15.32:0/474784468 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1104643701 0 0) 0x7f84c815f870 con 0x7f84c8155e50 2024-04-17T09:57:39.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.779+0000 7f84c6a00640 1 -- 172.21.15.32:0/474784468 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84ac003530 con 0x7f84c8155e50 2024-04-17T09:57:39.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.779+0000 7f84c7400640 1 -- 172.21.15.32:0/474784468 >> v2:172.21.15.144:3300/0 conn(0x7f84c80b3340 msgr2=0x7f84c815c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:39.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.779+0000 7f84c7400640 1 --2- 172.21.15.32:0/474784468 >> v2:172.21.15.144:3300/0 conn(0x7f84c80b3340 0x7f84c815c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.779+0000 7f84c7400640 1 -- 172.21.15.32:0/474784468 >> v1:172.21.15.32:6789/0 conn(0x7f84c8155e50 legacy=0x7f84c815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:39.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.779+0000 7f84c7400640 1 -- 172.21.15.32:0/474784468 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c815fb50 con 0x7f84c8155a70 2024-04-17T09:57:39.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.779+0000 7f84c7400640 1 --2- 172.21.15.32:0/474784468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84c8155a70 0x7f84c815f190 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f84b80097c0 tx=0x7f84b801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a73e343d2cc8efa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:39.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.779+0000 7f84c6a00640 1 -- 172.21.15.32:0/474784468 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b80b0070 con 0x7f84c8155a70 2024-04-17T09:57:39.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.779+0000 7f84c6a00640 1 -- 172.21.15.32:0/474784468 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84b80ab600 con 0x7f84c8155a70 2024-04-17T09:57:39.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.779+0000 7f84c6a00640 1 -- 172.21.15.32:0/474784468 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b80ab9d0 con 0x7f84c8155a70 2024-04-17T09:57:39.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.783+0000 7f84ce200640 1 -- 172.21.15.32:0/474784468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84c8155a70 msgr2=0x7f84c815f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:39.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.783+0000 7f84ce200640 1 --2- 172.21.15.32:0/474784468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84c8155a70 0x7f84c815f190 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f84b80097c0 tx=0x7f84b801c460 comp rx=0 tx=0).stop 2024-04-17T09:57:39.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.783+0000 7f84ce200640 1 -- 172.21.15.32:0/474784468 shutdown_connections 2024-04-17T09:57:39.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.783+0000 7f84ce200640 1 --2- 172.21.15.32:0/474784468 >> v2:172.21.15.144:3300/0 conn(0x7f84c80b3340 0x7f84c815c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.783+0000 7f84ce200640 1 --2- 172.21.15.32:0/474784468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84c8155a70 0x7f84c815f190 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.783+0000 7f84ce200640 1 -- 172.21.15.32:0/474784468 >> 172.21.15.32:0/474784468 conn(0x7f84c806ba30 msgr2=0x7f84c806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:39.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.783+0000 7f84ce200640 1 -- 172.21.15.32:0/474784468 shutdown_connections 2024-04-17T09:57:39.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.783+0000 7f84ce200640 1 -- 172.21.15.32:0/474784468 wait complete. 2024-04-17T09:57:39.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.783+0000 7f84ce200640 1 Processor -- start 2024-04-17T09:57:39.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.783+0000 7f84ce200640 1 -- start start 2024-04-17T09:57:39.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.783+0000 7f84ce200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84c80b3340 0x7f84c815a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:39.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.783+0000 7f84ce200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84c8155a70 0x7f84c814f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:39.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.783+0000 7f84ce200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84c8155e50 0x7f84c8158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:39.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.783+0000 7f84ce200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f84c80d02f0 con 0x7f84c8155e50 2024-04-17T09:57:39.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.783+0000 7f84ce200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84c80d0170 con 0x7f84c80b3340 2024-04-17T09:57:39.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.783+0000 7f84ce200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f84c80d0470 con 0x7f84c8155a70 2024-04-17T09:57:39.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.783+0000 7f84c7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84c8155e50 0x7f84c8158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:39.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.783+0000 7f84c7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84c8155e50 0x7f84c8158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59644/0 (socket says 172.21.15.32:59644) 2024-04-17T09:57:39.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.783+0000 7f84c7e00640 1 -- 172.21.15.32:0/3017486836 learned_addr learned my addr 172.21.15.32:0/3017486836 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:39.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.783+0000 7f84bfe00640 1 --2- 172.21.15.32:0/3017486836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84c8155a70 0x7f84c814f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:39.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.783+0000 7f84c7400640 1 --2- 172.21.15.32:0/3017486836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84c80b3340 0x7f84c815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:39.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.787+0000 7f84c7e00640 1 -- 172.21.15.32:0/3017486836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84c8155a70 msgr2=0x7f84c814f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:39.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.787+0000 7f84c7e00640 1 --2- 172.21.15.32:0/3017486836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84c8155a70 0x7f84c814f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.787+0000 7f84c7e00640 1 -- 172.21.15.32:0/3017486836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84c80b3340 msgr2=0x7f84c815a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:39.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.787+0000 7f84c7e00640 1 --2- 172.21.15.32:0/3017486836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84c80b3340 0x7f84c815a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.787+0000 7f84c7e00640 1 -- 172.21.15.32:0/3017486836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c81fad90 con 0x7f84c8155e50 2024-04-17T09:57:39.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.787+0000 7f84c7e00640 1 --2- 172.21.15.32:0/3017486836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84c8155e50 0x7f84c8158bc0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f84c000efc0 tx=0x7f84c000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:39.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.787+0000 7f84bf400640 1 -- 172.21.15.32:0/3017486836 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c0018020 con 0x7f84c8155e50 2024-04-17T09:57:39.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.787+0000 7f84bf400640 1 -- 172.21.15.32:0/3017486836 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84c0014480 con 0x7f84c8155e50 2024-04-17T09:57:39.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.787+0000 7f84ce200640 1 -- 172.21.15.32:0/3017486836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c815b260 con 0x7f84c8155e50 2024-04-17T09:57:39.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.787+0000 7f84bf400640 1 -- 172.21.15.32:0/3017486836 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c0014870 con 0x7f84c8155e50 2024-04-17T09:57:39.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.787+0000 7f84ce200640 1 -- 172.21.15.32:0/3017486836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c81583c0 con 0x7f84c8155e50 2024-04-17T09:57:39.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.787+0000 7f84ce200640 1 -- 172.21.15.32:0/3017486836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84c8068a90 con 0x7f84c8155e50 2024-04-17T09:57:39.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.787+0000 7f84bf400640 1 -- 172.21.15.32:0/3017486836 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f84c001b020 con 0x7f84c8155e50 2024-04-17T09:57:39.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.787+0000 7f84bf400640 1 --2- 172.21.15.32:0/3017486836 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f84a4068380 0x7f84a406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:39.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.787+0000 7f84bf400640 1 -- 172.21.15.32:0/3017486836 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7f84c0087430 con 0x7f84c8155e50 2024-04-17T09:57:39.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.787+0000 7f84c7400640 1 --2- 172.21.15.32:0/3017486836 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f84a4068380 0x7f84a406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:39.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.791+0000 7f84c7400640 1 --2- 172.21.15.32:0/3017486836 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f84a4068380 0x7f84a406a840 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f84b80ad000 tx=0x7f84b8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:39.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.791+0000 7f84bf400640 1 -- 172.21.15.32:0/3017486836 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f84c0022070 con 0x7f84c8155e50 2024-04-17T09:57:39.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.967+0000 7f84ce200640 1 -- 172.21.15.32:0/3017486836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f84c806edb0 con 0x7f84c8155e50 2024-04-17T09:57:39.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.971+0000 7f84bf400640 1 -- 172.21.15.32:0/3017486836 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v69) ==== 80+0+120 (secure 0 0 0) 0x7f84c0010040 con 0x7f84c8155e50 2024-04-17T09:57:39.971 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6836/663537244 2024-04-18T09:57:30.746884+0000 2024-04-17T09:57:39.971 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6834/663537244 2024-04-18T09:57:30.746884+0000 2024-04-17T09:57:39.972 INFO:teuthology.orchestra.run.smithi032.stderr:listed 2 entries 2024-04-17T09:57:39.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.971+0000 7f84ce200640 1 -- 172.21.15.32:0/3017486836 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f84a4068380 msgr2=0x7f84a406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:39.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.971+0000 7f84ce200640 1 --2- 172.21.15.32:0/3017486836 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f84a4068380 0x7f84a406a840 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f84b80ad000 tx=0x7f84b8008040 comp rx=0 tx=0).stop 2024-04-17T09:57:39.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.971+0000 7f84ce200640 1 -- 172.21.15.32:0/3017486836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84c8155e50 msgr2=0x7f84c8158bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:39.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.971+0000 7f84ce200640 1 --2- 172.21.15.32:0/3017486836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84c8155e50 0x7f84c8158bc0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f84c000efc0 tx=0x7f84c000c490 comp rx=0 tx=0).stop 2024-04-17T09:57:39.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.971+0000 7f84ce200640 1 -- 172.21.15.32:0/3017486836 shutdown_connections 2024-04-17T09:57:39.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.971+0000 7f84ce200640 1 --2- 172.21.15.32:0/3017486836 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f84a4068380 0x7f84a406a840 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.971+0000 7f84ce200640 1 --2- 172.21.15.32:0/3017486836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84c8155a70 0x7f84c814f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.971+0000 7f84ce200640 1 --2- 172.21.15.32:0/3017486836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84c8155e50 0x7f84c8158bc0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.971+0000 7f84ce200640 1 --2- 172.21.15.32:0/3017486836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84c80b3340 0x7f84c815a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:39.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.971+0000 7f84ce200640 1 -- 172.21.15.32:0/3017486836 >> 172.21.15.32:0/3017486836 conn(0x7f84c806ba30 msgr2=0x7f84c8154040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:39.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.971+0000 7f84ce200640 1 -- 172.21.15.32:0/3017486836 shutdown_connections 2024-04-17T09:57:39.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:39.971+0000 7f84ce200640 1 -- 172.21.15.32:0/3017486836 wait complete. 2024-04-17T09:57:39.992 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:57:39.992 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:57:40.001 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:57:40.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.183+0000 7f850ac00640 1 Processor -- start 2024-04-17T09:57:40.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.183+0000 7f850ac00640 1 -- start start 2024-04-17T09:57:40.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f850ac00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8504145f30 0x7f8504071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:40.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f850ac00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85040cdac0 con 0x7f85040cd580 2024-04-17T09:57:40.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f850ac00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f850415cdf0 con 0x7f850415d8d0 2024-04-17T09:57:40.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f850ac00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f850406f400 con 0x7f8504145f30 2024-04-17T09:57:40.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8508c00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f85040cd580 0x7f85041601d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53126/0 (socket says 172.21.15.32:53126) 2024-04-17T09:57:40.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8508c00640 1 -- 172.21.15.32:0/3345380355 learned_addr learned my addr 172.21.15.32:0/3345380355 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:40.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8503400640 1 -- 172.21.15.32:0/3345380355 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 189479984 0 0) 0x7f85040cdac0 con 0x7f85040cd580 2024-04-17T09:57:40.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8503400640 1 -- 172.21.15.32:0/3345380355 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84e8003530 con 0x7f85040cd580 2024-04-17T09:57:40.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8503e00640 1 --2- 172.21.15.32:0/3345380355 >> v2:172.21.15.144:3300/0 conn(0x7f8504145f30 0x7f8504071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:40.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8503400640 1 -- 172.21.15.32:0/3345380355 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2118530790 0 0) 0x7f84e8003530 con 0x7f85040cd580 2024-04-17T09:57:40.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8503400640 1 -- 172.21.15.32:0/3345380355 >> v2:172.21.15.144:3300/0 conn(0x7f8504145f30 msgr2=0x7f8504071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:40.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8503400640 1 --2- 172.21.15.32:0/3345380355 >> v2:172.21.15.144:3300/0 conn(0x7f8504145f30 0x7f8504071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8503400640 1 -- 172.21.15.32:0/3345380355 >> v1:172.21.15.144:6789/0 conn(0x7f850415d8d0 legacy=0x7f85040cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:40.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8503400640 1 -- 172.21.15.32:0/3345380355 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850415d0d0 con 0x7f85040cd580 2024-04-17T09:57:40.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8503400640 1 -- 172.21.15.32:0/3345380355 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f84fc001a00 con 0x7f85040cd580 2024-04-17T09:57:40.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8503400640 1 -- 172.21.15.32:0/3345380355 >> v1:172.21.15.32:6789/0 conn(0x7f85040cd580 legacy=0x7f85041601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:40.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8503400640 1 --2- 172.21.15.32:0/3345380355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84e8003a10 0x7f84e8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:40.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8503400640 1 --2- 172.21.15.32:0/3345380355 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84e8004810 0x7f84e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:40.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8503400640 1 --2- 172.21.15.32:0/3345380355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e80051d0 0x7f84e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:40.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8503400640 1 -- 172.21.15.32:0/3345380355 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f850406f400 con 0x7f84e80051d0 2024-04-17T09:57:40.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8503400640 1 -- 172.21.15.32:0/3345380355 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84e800c0b0 con 0x7f84e8003a10 2024-04-17T09:57:40.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8503400640 1 -- 172.21.15.32:0/3345380355 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f84e800c2b0 con 0x7f84e8004810 2024-04-17T09:57:40.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8508c00640 1 --2- 172.21.15.32:0/3345380355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e80051d0 0x7f84e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:40.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8503e00640 1 --2- 172.21.15.32:0/3345380355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84e8003a10 0x7f84e8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:40.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f84fbe00640 1 --2- 172.21.15.32:0/3345380355 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84e8004810 0x7f84e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:40.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8508c00640 1 -- 172.21.15.32:0/3345380355 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84e8004810 msgr2=0x7f84e8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:40.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8508c00640 1 --2- 172.21.15.32:0/3345380355 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84e8004810 0x7f84e8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8508c00640 1 -- 172.21.15.32:0/3345380355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84e8003a10 msgr2=0x7f84e8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:40.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8508c00640 1 --2- 172.21.15.32:0/3345380355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84e8003a10 0x7f84e8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8508c00640 1 -- 172.21.15.32:0/3345380355 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e8003530 con 0x7f84e80051d0 2024-04-17T09:57:40.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.187+0000 7f8503e00640 1 --2- 172.21.15.32:0/3345380355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84e8003a10 0x7f84e8003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:57:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.191+0000 7f8508c00640 1 -- 172.21.15.32:0/3345380355 reap_dead start 2024-04-17T09:57:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.191+0000 7f8508c00640 1 --2- 172.21.15.32:0/3345380355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e80051d0 0x7f84e800b9d0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00a790 tx=0x7f84fc01d2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3a2049bd56b52a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.191+0000 7f8503400640 1 -- 172.21.15.32:0/3345380355 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f40096b0 con 0x7f84e80051d0 2024-04-17T09:57:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.191+0000 7f8503400640 1 -- 172.21.15.32:0/3345380355 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84fc0ad670 con 0x7f84e80051d0 2024-04-17T09:57:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.191+0000 7f8503400640 1 -- 172.21.15.32:0/3345380355 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc00a3b0 con 0x7f84e80051d0 2024-04-17T09:57:40.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.191+0000 7f850ac00640 1 -- 172.21.15.32:0/3345380355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e80051d0 msgr2=0x7f84e800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:40.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.191+0000 7f850ac00640 1 --2- 172.21.15.32:0/3345380355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e80051d0 0x7f84e800b9d0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00a790 tx=0x7f84fc01d2c0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.191+0000 7f850ac00640 1 -- 172.21.15.32:0/3345380355 shutdown_connections 2024-04-17T09:57:40.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.191+0000 7f850ac00640 1 --2- 172.21.15.32:0/3345380355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e80051d0 0x7f84e800b9d0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.191+0000 7f850ac00640 1 -- 172.21.15.32:0/3345380355 >> 172.21.15.32:0/3345380355 conn(0x7f850406ba30 msgr2=0x7f850406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:40.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.191+0000 7f850ac00640 1 -- 172.21.15.32:0/3345380355 shutdown_connections 2024-04-17T09:57:40.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.191+0000 7f850ac00640 1 -- 172.21.15.32:0/3345380355 wait complete. 2024-04-17T09:57:40.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.191+0000 7f850ac00640 1 Processor -- start 2024-04-17T09:57:40.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.191+0000 7f850ac00640 1 -- start start 2024-04-17T09:57:40.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f850ac00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e80051d0 0x7f850413ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:40.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f850ac00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85040c5e80 0x7f850413c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:40.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f850ac00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f850415b340 0x7f850415b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:40.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f850ac00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f85040cd200 con 0x7f84e80051d0 2024-04-17T09:57:40.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f850ac00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f85040cd080 con 0x7f85040c5e80 2024-04-17T09:57:40.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f850ac00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f85040cd380 con 0x7f850415b340 2024-04-17T09:57:40.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f8503e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e80051d0 0x7f850413ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:40.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f8503e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e80051d0 0x7f850413ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59660/0 (socket says 172.21.15.32:59660) 2024-04-17T09:57:40.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f84fbe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85040c5e80 0x7f850413c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:40.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f8503e00640 1 -- 172.21.15.32:0/1536465249 learned_addr learned my addr 172.21.15.32:0/1536465249 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:40.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f8508c00640 1 --2- 172.21.15.32:0/1536465249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f850415b340 0x7f850415b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:40.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f8503e00640 1 -- 172.21.15.32:0/1536465249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f850415b340 msgr2=0x7f850415b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:40.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f8503e00640 1 --2- 172.21.15.32:0/1536465249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f850415b340 0x7f850415b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f8503e00640 1 -- 172.21.15.32:0/1536465249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85040c5e80 msgr2=0x7f850413c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:40.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f8503e00640 1 --2- 172.21.15.32:0/1536465249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85040c5e80 0x7f850413c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f8503e00640 1 -- 172.21.15.32:0/1536465249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850413f3d0 con 0x7f84e80051d0 2024-04-17T09:57:40.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f84fbe00640 1 --2- 172.21.15.32:0/1536465249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85040c5e80 0x7f850413c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:57:40.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f8503e00640 1 --2- 172.21.15.32:0/1536465249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e80051d0 0x7f850413ea80 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f84f4011910 tx=0x7f84f4011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:40.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f8500c00640 1 -- 172.21.15.32:0/1536465249 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f400faf0 con 0x7f84e80051d0 2024-04-17T09:57:40.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f8500c00640 1 -- 172.21.15.32:0/1536465249 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f400fc90 con 0x7f84e80051d0 2024-04-17T09:57:40.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f8500c00640 1 -- 172.21.15.32:0/1536465249 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4004020 con 0x7f84e80051d0 2024-04-17T09:57:40.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f850ac00640 1 -- 172.21.15.32:0/1536465249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850413f690 con 0x7f84e80051d0 2024-04-17T09:57:40.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f850ac00640 1 -- 172.21.15.32:0/1536465249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850413fc50 con 0x7f84e80051d0 2024-04-17T09:57:40.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.195+0000 7f850ac00640 1 -- 172.21.15.32:0/1536465249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8504068a90 con 0x7f84e80051d0 2024-04-17T09:57:40.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.199+0000 7f8500c00640 1 -- 172.21.15.32:0/1536465249 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f84f4012040 con 0x7f84e80051d0 2024-04-17T09:57:40.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.199+0000 7f8500c00640 1 --2- 172.21.15.32:0/1536465249 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f84e00683b0 0x7f84e006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:40.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.199+0000 7f8500c00640 1 -- 172.21.15.32:0/1536465249 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7f84f4086cb0 con 0x7f84e80051d0 2024-04-17T09:57:40.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.199+0000 7f84fbe00640 1 --2- 172.21.15.32:0/1536465249 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f84e00683b0 0x7f84e006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:40.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.199+0000 7f84fbe00640 1 --2- 172.21.15.32:0/1536465249 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f84e00683b0 0x7f84e006a870 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f84ec010d80 tx=0x7f84ec012040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:40.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.203+0000 7f8500c00640 1 -- 172.21.15.32:0/1536465249 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f84f4019030 con 0x7f84e80051d0 2024-04-17T09:57:40.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.367+0000 7f850ac00640 1 -- 172.21.15.32:0/1536465249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85040c5bf0 con 0x7f84e80051d0 2024-04-17T09:57:40.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.367+0000 7f8500c00640 1 -- 172.21.15.32:0/1536465249 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+16807 (secure 0 0 0) 0x7f84f4056bb0 con 0x7f84e80051d0 2024-04-17T09:57:40.371 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:57:40.371 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":69,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:57:30.746906+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:57:09.948964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-17T09:57:10.512460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6836/663537244":"2024-04-18T09:57:30.746884+0000","172.21.15.32:6834/663537244":"2024-04-18T09:57:30.746884+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:57:40.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.371+0000 7f850ac00640 1 -- 172.21.15.32:0/1536465249 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f84e00683b0 msgr2=0x7f84e006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:40.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.371+0000 7f850ac00640 1 --2- 172.21.15.32:0/1536465249 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f84e00683b0 0x7f84e006a870 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f84ec010d80 tx=0x7f84ec012040 comp rx=0 tx=0).stop 2024-04-17T09:57:40.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.371+0000 7f850ac00640 1 -- 172.21.15.32:0/1536465249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e80051d0 msgr2=0x7f850413ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:40.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.371+0000 7f850ac00640 1 --2- 172.21.15.32:0/1536465249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e80051d0 0x7f850413ea80 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f84f4011910 tx=0x7f84f4011dd0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.371+0000 7f850ac00640 1 -- 172.21.15.32:0/1536465249 shutdown_connections 2024-04-17T09:57:40.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.371+0000 7f850ac00640 1 --2- 172.21.15.32:0/1536465249 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f84e00683b0 0x7f84e006a870 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.371+0000 7f850ac00640 1 --2- 172.21.15.32:0/1536465249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f850415b340 0x7f850415b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.371+0000 7f850ac00640 1 --2- 172.21.15.32:0/1536465249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85040c5e80 0x7f850413c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.371+0000 7f850ac00640 1 --2- 172.21.15.32:0/1536465249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84e80051d0 0x7f850413ea80 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.371+0000 7f850ac00640 1 -- 172.21.15.32:0/1536465249 >> 172.21.15.32:0/1536465249 conn(0x7f850406ba30 msgr2=0x7f850415f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:40.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.371+0000 7f850ac00640 1 -- 172.21.15.32:0/1536465249 shutdown_connections 2024-04-17T09:57:40.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.371+0000 7f850ac00640 1 -- 172.21.15.32:0/1536465249 wait complete. 2024-04-17T09:57:40.393 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:57:09.948964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:57:10.512460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:57:40.394 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:57:40.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.535+0000 7f20e3a00640 1 Processor -- start 2024-04-17T09:57:40.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.535+0000 7f20e3a00640 1 -- start start 2024-04-17T09:57:40.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.539+0000 7f20e3a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20dc15b7d0 0x7f20dc06e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:40.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.539+0000 7f20e3a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20dc06f7c0 0x7f20dc06ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:40.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.539+0000 7f20e3a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20dc0cc940 con 0x7f20dc06f270 2024-04-17T09:57:40.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.539+0000 7f20e3a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f20dc0ccc20 con 0x7f20dc06f7c0 2024-04-17T09:57:40.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.539+0000 7f20e3a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f20dc06e0b0 con 0x7f20dc15b7d0 2024-04-17T09:57:40.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.539+0000 7f20d3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20dc06f7c0 0x7f20dc06ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:40.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.539+0000 7f20d3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20dc06f7c0 0x7f20dc06ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34600/0 (socket says 172.21.15.32:34600) 2024-04-17T09:57:40.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.539+0000 7f20d3e00640 1 -- 172.21.15.32:0/2165733599 learned_addr learned my addr 172.21.15.32:0/2165733599 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:40.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.539+0000 7f20e1000640 1 --2- 172.21.15.32:0/2165733599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20dc15b7d0 0x7f20dc06e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:40.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.539+0000 7f20d3e00640 1 -- 172.21.15.32:0/2165733599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20dc15b7d0 msgr2=0x7f20dc06e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:40.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.539+0000 7f20d3e00640 1 --2- 172.21.15.32:0/2165733599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20dc15b7d0 0x7f20dc06e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.539+0000 7f20d3e00640 1 -- 172.21.15.32:0/2165733599 >> v1:172.21.15.144:6789/0 conn(0x7f20dc06f270 legacy=0x7f20dc0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:40.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.539+0000 7f20d3e00640 1 -- 172.21.15.32:0/2165733599 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20dc06e390 con 0x7f20dc06f7c0 2024-04-17T09:57:40.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.543+0000 7f20d3e00640 1 --2- 172.21.15.32:0/2165733599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20dc06f7c0 0x7f20dc06ba30 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009870 tx=0x7f20c401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e711326297da41a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:40.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.543+0000 7f20dbe00640 1 -- 172.21.15.32:0/2165733599 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c40aa650 con 0x7f20dc06f7c0 2024-04-17T09:57:40.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.543+0000 7f20dbe00640 1 -- 172.21.15.32:0/2165733599 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c40af070 con 0x7f20dc06f7c0 2024-04-17T09:57:40.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.543+0000 7f20dbe00640 1 -- 172.21.15.32:0/2165733599 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c40aaa10 con 0x7f20dc06f7c0 2024-04-17T09:57:40.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.543+0000 7f20e3a00640 1 -- 172.21.15.32:0/2165733599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20dc06f7c0 msgr2=0x7f20dc06ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:40.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.543+0000 7f20e3a00640 1 --2- 172.21.15.32:0/2165733599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20dc06f7c0 0x7f20dc06ba30 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009870 tx=0x7f20c401c4d0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.543+0000 7f20e3a00640 1 -- 172.21.15.32:0/2165733599 shutdown_connections 2024-04-17T09:57:40.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.543+0000 7f20e3a00640 1 --2- 172.21.15.32:0/2165733599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20dc06f7c0 0x7f20dc06ba30 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.543+0000 7f20e3a00640 1 --2- 172.21.15.32:0/2165733599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20dc15b7d0 0x7f20dc06e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.543+0000 7f20e3a00640 1 -- 172.21.15.32:0/2165733599 >> 172.21.15.32:0/2165733599 conn(0x7f20dc0b2120 msgr2=0x7f20dc0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:40.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.543+0000 7f20e3a00640 1 -- 172.21.15.32:0/2165733599 shutdown_connections 2024-04-17T09:57:40.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.543+0000 7f20e3a00640 1 -- 172.21.15.32:0/2165733599 wait complete. 2024-04-17T09:57:40.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.543+0000 7f20e3a00640 1 Processor -- start 2024-04-17T09:57:40.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.543+0000 7f20e3a00640 1 -- start start 2024-04-17T09:57:40.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20e3a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20dc06f270 0x7f20dc149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:40.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20e3a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20dc06f7c0 0x7f20dc071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:40.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20e3a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20dc15b7d0 0x7f20dc0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:40.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20e3a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc168ab0 con 0x7f20dc15b7d0 2024-04-17T09:57:40.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20e3a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc06c380 con 0x7f20dc06f7c0 2024-04-17T09:57:40.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20e3a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f20dc168c30 con 0x7f20dc06f270 2024-04-17T09:57:40.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20e1a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20dc15b7d0 0x7f20dc0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:40.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20e1000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20dc06f270 0x7f20dc149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:40.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20d3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20dc06f7c0 0x7f20dc071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:40.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20e1a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20dc15b7d0 0x7f20dc0723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34614/0 (socket says 172.21.15.32:34614) 2024-04-17T09:57:40.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20e1a00640 1 -- 172.21.15.32:0/2802360236 learned_addr learned my addr 172.21.15.32:0/2802360236 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:40.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20e1a00640 1 -- 172.21.15.32:0/2802360236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20dc06f270 msgr2=0x7f20dc149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:40.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20e1a00640 1 --2- 172.21.15.32:0/2802360236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20dc06f270 0x7f20dc149c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20e1a00640 1 -- 172.21.15.32:0/2802360236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20dc06f7c0 msgr2=0x7f20dc071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:40.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20e1a00640 1 --2- 172.21.15.32:0/2802360236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20dc06f7c0 0x7f20dc071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20e1a00640 1 -- 172.21.15.32:0/2802360236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20dc154d50 con 0x7f20dc15b7d0 2024-04-17T09:57:40.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20e1a00640 1 --2- 172.21.15.32:0/2802360236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20dc15b7d0 0x7f20dc0723a0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f20d400c730 tx=0x7f20d400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:40.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20d9600640 1 -- 172.21.15.32:0/2802360236 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4003b80 con 0x7f20dc15b7d0 2024-04-17T09:57:40.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20d9600640 1 -- 172.21.15.32:0/2802360236 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d4003d20 con 0x7f20dc15b7d0 2024-04-17T09:57:40.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20d9600640 1 -- 172.21.15.32:0/2802360236 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d400aa30 con 0x7f20dc15b7d0 2024-04-17T09:57:40.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20e3a00640 1 -- 172.21.15.32:0/2802360236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20dc155030 con 0x7f20dc15b7d0 2024-04-17T09:57:40.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20e3a00640 1 -- 172.21.15.32:0/2802360236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20dc1555f0 con 0x7f20dc15b7d0 2024-04-17T09:57:40.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20e3a00640 1 -- 172.21.15.32:0/2802360236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20dc068a90 con 0x7f20dc15b7d0 2024-04-17T09:57:40.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.547+0000 7f20d9600640 1 -- 172.21.15.32:0/2802360236 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f20d400abd0 con 0x7f20dc15b7d0 2024-04-17T09:57:40.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.551+0000 7f20d9600640 1 --2- 172.21.15.32:0/2802360236 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f20b8068380 0x7f20b806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:40.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.551+0000 7f20d9600640 1 -- 172.21.15.32:0/2802360236 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7f20d4086780 con 0x7f20dc15b7d0 2024-04-17T09:57:40.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.551+0000 7f20e1000640 1 --2- 172.21.15.32:0/2802360236 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f20b8068380 0x7f20b806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:40.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.551+0000 7f20e1000640 1 --2- 172.21.15.32:0/2802360236 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f20b8068380 0x7f20b806a840 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f20cc006aa0 tx=0x7f20cc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:40.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.555+0000 7f20d9600640 1 -- 172.21.15.32:0/2802360236 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f20d4056690 con 0x7f20dc15b7d0 2024-04-17T09:57:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.715+0000 7f20e3a00640 1 -- 172.21.15.32:0/2802360236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f20dc075110 con 0x7f20dc15b7d0 2024-04-17T09:57:40.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.719+0000 7f20d9600640 1 -- 172.21.15.32:0/2802360236 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) ==== 93+0+10 (secure 0 0 0) 0x7f20d405a990 con 0x7f20dc15b7d0 2024-04-17T09:57:40.719 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:57:40.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.719+0000 7f20e3a00640 1 -- 172.21.15.32:0/2802360236 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f20b8068380 msgr2=0x7f20b806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:40.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.719+0000 7f20e3a00640 1 --2- 172.21.15.32:0/2802360236 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f20b8068380 0x7f20b806a840 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f20cc006aa0 tx=0x7f20cc008040 comp rx=0 tx=0).stop 2024-04-17T09:57:40.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.719+0000 7f20e3a00640 1 -- 172.21.15.32:0/2802360236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20dc15b7d0 msgr2=0x7f20dc0723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:40.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.719+0000 7f20e3a00640 1 --2- 172.21.15.32:0/2802360236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20dc15b7d0 0x7f20dc0723a0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f20d400c730 tx=0x7f20d400cbf0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.719+0000 7f20e3a00640 1 -- 172.21.15.32:0/2802360236 shutdown_connections 2024-04-17T09:57:40.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.719+0000 7f20e3a00640 1 --2- 172.21.15.32:0/2802360236 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f20b8068380 0x7f20b806a840 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.719+0000 7f20e3a00640 1 --2- 172.21.15.32:0/2802360236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20dc15b7d0 0x7f20dc0723a0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.719+0000 7f20e3a00640 1 --2- 172.21.15.32:0/2802360236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20dc06f7c0 0x7f20dc071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.719+0000 7f20e3a00640 1 --2- 172.21.15.32:0/2802360236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20dc06f270 0x7f20dc149c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.719+0000 7f20e3a00640 1 -- 172.21.15.32:0/2802360236 >> 172.21.15.32:0/2802360236 conn(0x7f20dc0b2120 msgr2=0x7f20dc0cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:40.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.719+0000 7f20e3a00640 1 -- 172.21.15.32:0/2802360236 shutdown_connections 2024-04-17T09:57:40.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.719+0000 7f20e3a00640 1 -- 172.21.15.32:0/2802360236 wait complete. 2024-04-17T09:57:40.741 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:57:40.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.883+0000 7fa5a8400640 1 Processor -- start 2024-04-17T09:57:40.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.883+0000 7fa5a8400640 1 -- start start 2024-04-17T09:57:40.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.883+0000 7fa5a8400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa5a006c140 0x7fa5a00ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:40.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.883+0000 7fa5a8400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a00cc940 con 0x7fa5a006f7c0 2024-04-17T09:57:40.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.883+0000 7fa5a8400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a00ccc20 con 0x7fa5a015b7d0 2024-04-17T09:57:40.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.883+0000 7fa5a8400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa5a006e0b0 con 0x7fa5a006c140 2024-04-17T09:57:40.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.883+0000 7fa5a5000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa5a006f7c0 0x7fa5a006ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37184/0 (socket says 172.21.15.32:37184) 2024-04-17T09:57:40.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.883+0000 7fa5a5000640 1 -- 172.21.15.32:0/836907405 learned_addr learned my addr 172.21.15.32:0/836907405 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:40.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.883+0000 7fa5a6400640 1 --2- 172.21.15.32:0/836907405 >> v2:172.21.15.144:3300/0 conn(0x7fa5a006c140 0x7fa5a00ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:40.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.883+0000 7fa59fe00640 1 -- 172.21.15.32:0/836907405 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 315639292 0 0) 0x7fa5a00cc940 con 0x7fa5a006f7c0 2024-04-17T09:57:40.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.887+0000 7fa59fe00640 1 -- 172.21.15.32:0/836907405 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa588003530 con 0x7fa5a006f7c0 2024-04-17T09:57:40.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.887+0000 7fa59fe00640 1 -- 172.21.15.32:0/836907405 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 6928337 0 0) 0x7fa5a00ccc20 con 0x7fa5a015b7d0 2024-04-17T09:57:40.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.887+0000 7fa59fe00640 1 -- 172.21.15.32:0/836907405 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5a00cc940 con 0x7fa5a015b7d0 2024-04-17T09:57:40.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.887+0000 7fa59fe00640 1 -- 172.21.15.32:0/836907405 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 612577368 0 0) 0x7fa588003530 con 0x7fa5a006f7c0 2024-04-17T09:57:40.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.887+0000 7fa5a6400640 1 -- 172.21.15.32:0/836907405 >> v1:172.21.15.144:6789/0 conn(0x7fa5a015b7d0 legacy=0x7fa5a006e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:40.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.887+0000 7fa5a6400640 1 -- 172.21.15.32:0/836907405 >> v1:172.21.15.32:6789/0 conn(0x7fa5a006f7c0 legacy=0x7fa5a006ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:40.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.887+0000 7fa5a6400640 1 -- 172.21.15.32:0/836907405 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a006e390 con 0x7fa5a006c140 2024-04-17T09:57:40.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.887+0000 7fa5a6400640 1 --2- 172.21.15.32:0/836907405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a006c140 0x7fa5a00ce290 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa598009d90 tx=0x7fa59801c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c464af9ccafb32ad server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:40.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.887+0000 7fa59fe00640 1 -- 172.21.15.32:0/836907405 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5980b2070 con 0x7fa5a006c140 2024-04-17T09:57:40.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.887+0000 7fa59fe00640 1 -- 172.21.15.32:0/836907405 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5980a5660 con 0x7fa5a006c140 2024-04-17T09:57:40.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.887+0000 7fa59fe00640 1 -- 172.21.15.32:0/836907405 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c001680 con 0x7fa5a006c140 2024-04-17T09:57:40.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a8400640 1 -- 172.21.15.32:0/836907405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a006c140 msgr2=0x7fa5a00ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:40.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a8400640 1 --2- 172.21.15.32:0/836907405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a006c140 0x7fa5a00ce290 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa598009d90 tx=0x7fa59801c800 comp rx=0 tx=0).stop 2024-04-17T09:57:40.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a8400640 1 -- 172.21.15.32:0/836907405 shutdown_connections 2024-04-17T09:57:40.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a8400640 1 --2- 172.21.15.32:0/836907405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a006c140 0x7fa5a00ce290 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a8400640 1 -- 172.21.15.32:0/836907405 >> 172.21.15.32:0/836907405 conn(0x7fa5a00b2120 msgr2=0x7fa5a00b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:40.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a8400640 1 -- 172.21.15.32:0/836907405 shutdown_connections 2024-04-17T09:57:40.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a8400640 1 -- 172.21.15.32:0/836907405 wait complete. 2024-04-17T09:57:40.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a8400640 1 Processor -- start 2024-04-17T09:57:40.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a8400640 1 -- start start 2024-04-17T09:57:40.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a8400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a006c140 0x7fa5a013e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:40.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a8400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a006f7c0 0x7fa5a015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:40.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a8400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a015b7d0 0x7fa5a00aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:40.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a8400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a0169640 con 0x7fa5a015b7d0 2024-04-17T09:57:40.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a8400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a01694c0 con 0x7fa5a006f7c0 2024-04-17T09:57:40.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a8400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a01697c0 con 0x7fa5a006c140 2024-04-17T09:57:40.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a6400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a015b7d0 0x7fa5a00aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:40.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a6400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a015b7d0 0x7fa5a00aee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34622/0 (socket says 172.21.15.32:34622) 2024-04-17T09:57:40.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a6400640 1 -- 172.21.15.32:0/1769439886 learned_addr learned my addr 172.21.15.32:0/1769439886 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:40.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a5000640 1 --2- 172.21.15.32:0/1769439886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a006f7c0 0x7fa5a015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:40.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a5a00640 1 --2- 172.21.15.32:0/1769439886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a006c140 0x7fa5a013e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:40.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a6400640 1 -- 172.21.15.32:0/1769439886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a006c140 msgr2=0x7fa5a013e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:40.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a6400640 1 --2- 172.21.15.32:0/1769439886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a006c140 0x7fa5a013e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a6400640 1 -- 172.21.15.32:0/1769439886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a006f7c0 msgr2=0x7fa5a015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:40.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a6400640 1 --2- 172.21.15.32:0/1769439886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a006f7c0 0x7fa5a015bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:40.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.891+0000 7fa5a6400640 1 -- 172.21.15.32:0/1769439886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a013e6e0 con 0x7fa5a015b7d0 2024-04-17T09:57:40.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.895+0000 7fa5a6400640 1 --2- 172.21.15.32:0/1769439886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a015b7d0 0x7fa5a00aee70 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa59801cce0 tx=0x7fa5980195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:40.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.895+0000 7fa59d600640 1 -- 172.21.15.32:0/1769439886 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5980b2050 con 0x7fa5a015b7d0 2024-04-17T09:57:40.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.895+0000 7fa59d600640 1 -- 172.21.15.32:0/1769439886 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5980b3040 con 0x7fa5a015b7d0 2024-04-17T09:57:40.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.895+0000 7fa59d600640 1 -- 172.21.15.32:0/1769439886 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598003da0 con 0x7fa5a015b7d0 2024-04-17T09:57:40.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.895+0000 7fa5a8400640 1 -- 172.21.15.32:0/1769439886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a015e4a0 con 0x7fa5a015b7d0 2024-04-17T09:57:40.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.895+0000 7fa5a8400640 1 -- 172.21.15.32:0/1769439886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a00ae670 con 0x7fa5a015b7d0 2024-04-17T09:57:40.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.895+0000 7fa5a8400640 1 -- 172.21.15.32:0/1769439886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a0068a90 con 0x7fa5a015b7d0 2024-04-17T09:57:40.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.895+0000 7fa59d600640 1 -- 172.21.15.32:0/1769439886 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa5980ad650 con 0x7fa5a015b7d0 2024-04-17T09:57:40.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.899+0000 7fa59d600640 1 --2- 172.21.15.32:0/1769439886 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa580068380 0x7fa58006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:40.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.899+0000 7fa59d600640 1 -- 172.21.15.32:0/1769439886 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7fa598121780 con 0x7fa5a015b7d0 2024-04-17T09:57:40.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.899+0000 7fa5a5a00640 1 --2- 172.21.15.32:0/1769439886 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa580068380 0x7fa58006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:40.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.899+0000 7fa5a5a00640 1 --2- 172.21.15.32:0/1769439886 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa580068380 0x7fa58006a840 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa5940015e0 tx=0x7fa594009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:40.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:40.903+0000 7fa59d600640 1 -- 172.21.15.32:0/1769439886 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa598016480 con 0x7fa5a015b7d0 2024-04-17T09:57:41.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.063+0000 7fa5a8400640 1 -- 172.21.15.32:0/1769439886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa5a0072ac0 con 0x7fa5a015b7d0 2024-04-17T09:57:41.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.067+0000 7fa59d600640 1 -- 172.21.15.32:0/1769439886 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) ==== 104+0+11 (secure 0 0 0) 0x7fa5980f1d60 con 0x7fa5a015b7d0 2024-04-17T09:57:41.067 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:57:41.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.067+0000 7fa5a8400640 1 -- 172.21.15.32:0/1769439886 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa580068380 msgr2=0x7fa58006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:41.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.067+0000 7fa5a8400640 1 --2- 172.21.15.32:0/1769439886 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa580068380 0x7fa58006a840 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa5940015e0 tx=0x7fa594009040 comp rx=0 tx=0).stop 2024-04-17T09:57:41.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.067+0000 7fa5a8400640 1 -- 172.21.15.32:0/1769439886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a015b7d0 msgr2=0x7fa5a00aee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:41.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.067+0000 7fa5a8400640 1 --2- 172.21.15.32:0/1769439886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a015b7d0 0x7fa5a00aee70 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa59801cce0 tx=0x7fa5980195a0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.067+0000 7fa5a8400640 1 -- 172.21.15.32:0/1769439886 shutdown_connections 2024-04-17T09:57:41.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.067+0000 7fa5a8400640 1 --2- 172.21.15.32:0/1769439886 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa580068380 0x7fa58006a840 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.067+0000 7fa5a8400640 1 --2- 172.21.15.32:0/1769439886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a015b7d0 0x7fa5a00aee70 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.067+0000 7fa5a8400640 1 --2- 172.21.15.32:0/1769439886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a006f7c0 0x7fa5a015bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.067+0000 7fa5a8400640 1 --2- 172.21.15.32:0/1769439886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a006c140 0x7fa5a013e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.067+0000 7fa5a8400640 1 -- 172.21.15.32:0/1769439886 >> 172.21.15.32:0/1769439886 conn(0x7fa5a00b2120 msgr2=0x7fa5a00b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:41.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.067+0000 7fa5a8400640 1 -- 172.21.15.32:0/1769439886 shutdown_connections 2024-04-17T09:57:41.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.067+0000 7fa5a8400640 1 -- 172.21.15.32:0/1769439886 wait complete. 2024-04-17T09:57:41.088 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:57:41.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.231+0000 7fa033a00640 1 Processor -- start 2024-04-17T09:57:41.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.231+0000 7fa033a00640 1 -- start start 2024-04-17T09:57:41.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.231+0000 7fa033a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa02c0cf370 0x7fa02c0cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:41.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.231+0000 7fa033a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa02c0d0130 con 0x7fa02c071e60 2024-04-17T09:57:41.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.231+0000 7fa033a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa02c15d090 con 0x7fa02c06ba30 2024-04-17T09:57:41.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.231+0000 7fa033a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa02c15d370 con 0x7fa02c0cf370 2024-04-17T09:57:41.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.231+0000 7fa02be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa02c0cf370 0x7fa02c0cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:41.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.231+0000 7fa031a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa02c071e60 0x7fa02c072270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37194/0 (socket says 172.21.15.32:37194) 2024-04-17T09:57:41.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.231+0000 7fa031a00640 1 -- 172.21.15.32:0/1648419395 learned_addr learned my addr 172.21.15.32:0/1648419395 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:41.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.235+0000 7fa02be00640 1 -- 172.21.15.32:0/1648419395 >> v1:172.21.15.144:6789/0 conn(0x7fa02c06ba30 legacy=0x7fa02c06be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:41.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.235+0000 7fa02be00640 1 -- 172.21.15.32:0/1648419395 >> v1:172.21.15.32:6789/0 conn(0x7fa02c071e60 legacy=0x7fa02c072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:41.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.235+0000 7fa02be00640 1 -- 172.21.15.32:0/1648419395 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa02c15d650 con 0x7fa02c0cf370 2024-04-17T09:57:41.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.235+0000 7fa02b400640 1 -- 172.21.15.32:0/1648419395 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1745561290 0 0) 0x7fa02c0d0130 con 0x7fa02c071e60 2024-04-17T09:57:41.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.235+0000 7fa02be00640 1 --2- 172.21.15.32:0/1648419395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02c0cf370 0x7fa02c0cf7e0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa0180097c0 tx=0x7fa01801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72af87bb7933ec95 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:41.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.235+0000 7fa02b400640 1 -- 172.21.15.32:0/1648419395 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0180aa600 con 0x7fa02c0cf370 2024-04-17T09:57:41.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.235+0000 7fa02b400640 1 -- 172.21.15.32:0/1648419395 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0180af070 con 0x7fa02c0cf370 2024-04-17T09:57:41.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.235+0000 7fa02b400640 1 -- 172.21.15.32:0/1648419395 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0180aa9c0 con 0x7fa02c0cf370 2024-04-17T09:57:41.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.235+0000 7fa033a00640 1 -- 172.21.15.32:0/1648419395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02c0cf370 msgr2=0x7fa02c0cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:41.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.235+0000 7fa033a00640 1 --2- 172.21.15.32:0/1648419395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02c0cf370 0x7fa02c0cf7e0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa0180097c0 tx=0x7fa01801c4a0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa033a00640 1 -- 172.21.15.32:0/1648419395 shutdown_connections 2024-04-17T09:57:41.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa033a00640 1 --2- 172.21.15.32:0/1648419395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02c0cf370 0x7fa02c0cf7e0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa033a00640 1 -- 172.21.15.32:0/1648419395 >> 172.21.15.32:0/1648419395 conn(0x7fa02c0b2120 msgr2=0x7fa02c0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:41.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa033a00640 1 -- 172.21.15.32:0/1648419395 shutdown_connections 2024-04-17T09:57:41.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa033a00640 1 -- 172.21.15.32:0/1648419395 wait complete. 2024-04-17T09:57:41.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa033a00640 1 Processor -- start 2024-04-17T09:57:41.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa033a00640 1 -- start start 2024-04-17T09:57:41.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa033a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa02c06ba30 0x7fa02c13c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:41.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa033a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa02c071e60 0x7fa02c13caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:41.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa033a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02c0cf370 0x7fa02c158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:41.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa033a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa02c15dcf0 con 0x7fa02c0cf370 2024-04-17T09:57:41.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa033a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa02c15db70 con 0x7fa02c06ba30 2024-04-17T09:57:41.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa033a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa02c146230 con 0x7fa02c071e60 2024-04-17T09:57:41.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa031a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02c0cf370 0x7fa02c158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:41.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa031a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02c0cf370 0x7fa02c158490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34646/0 (socket says 172.21.15.32:34646) 2024-04-17T09:57:41.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa031a00640 1 -- 172.21.15.32:0/1203248193 learned_addr learned my addr 172.21.15.32:0/1203248193 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:41.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa02be00640 1 --2- 172.21.15.32:0/1203248193 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa02c071e60 0x7fa02c13caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:41.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa031000640 1 --2- 172.21.15.32:0/1203248193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa02c06ba30 0x7fa02c13c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:41.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa031a00640 1 -- 172.21.15.32:0/1203248193 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa02c071e60 msgr2=0x7fa02c13caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:41.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa031a00640 1 --2- 172.21.15.32:0/1203248193 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa02c071e60 0x7fa02c13caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa031a00640 1 -- 172.21.15.32:0/1203248193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa02c06ba30 msgr2=0x7fa02c13c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:41.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa031a00640 1 --2- 172.21.15.32:0/1203248193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa02c06ba30 0x7fa02c13c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa031a00640 1 -- 172.21.15.32:0/1203248193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa02c154e50 con 0x7fa02c0cf370 2024-04-17T09:57:41.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.239+0000 7fa031a00640 1 --2- 172.21.15.32:0/1203248193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02c0cf370 0x7fa02c158490 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa02400c7a0 tx=0x7fa02400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:41.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.243+0000 7fa028c00640 1 -- 172.21.15.32:0/1203248193 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa024018070 con 0x7fa02c0cf370 2024-04-17T09:57:41.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.243+0000 7fa028c00640 1 -- 172.21.15.32:0/1203248193 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa02400a640 con 0x7fa02c0cf370 2024-04-17T09:57:41.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.243+0000 7fa028c00640 1 -- 172.21.15.32:0/1203248193 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02400aab0 con 0x7fa02c0cf370 2024-04-17T09:57:41.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.243+0000 7fa033a00640 1 -- 172.21.15.32:0/1203248193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa02c155130 con 0x7fa02c0cf370 2024-04-17T09:57:41.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.243+0000 7fa033a00640 1 -- 172.21.15.32:0/1203248193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa02c1556f0 con 0x7fa02c0cf370 2024-04-17T09:57:41.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.243+0000 7fa033a00640 1 -- 172.21.15.32:0/1203248193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa02c068a90 con 0x7fa02c0cf370 2024-04-17T09:57:41.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.243+0000 7fa028c00640 1 -- 172.21.15.32:0/1203248193 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa02400ac50 con 0x7fa02c0cf370 2024-04-17T09:57:41.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.243+0000 7fa028c00640 1 --2- 172.21.15.32:0/1203248193 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa00c0683b0 0x7fa00c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:41.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.243+0000 7fa028c00640 1 -- 172.21.15.32:0/1203248193 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7fa024085ff0 con 0x7fa02c0cf370 2024-04-17T09:57:41.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.243+0000 7fa031000640 1 --2- 172.21.15.32:0/1203248193 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa00c0683b0 0x7fa00c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:41.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.243+0000 7fa031000640 1 --2- 172.21.15.32:0/1203248193 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa00c0683b0 0x7fa00c06a870 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa020000f70 tx=0x7fa020009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:41.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.247+0000 7fa028c00640 1 -- 172.21.15.32:0/1203248193 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa0240564f0 con 0x7fa02c0cf370 2024-04-17T09:57:41.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.411+0000 7fa033a00640 1 -- 172.21.15.32:0/1203248193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa02c06c850 con 0x7fa02c0cf370 2024-04-17T09:57:41.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.411+0000 7fa028c00640 1 -- 172.21.15.32:0/1203248193 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) ==== 100+0+11 (secure 0 0 0) 0x7fa024055df0 con 0x7fa02c0cf370 2024-04-17T09:57:41.415 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:57:41.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.415+0000 7fa033a00640 1 -- 172.21.15.32:0/1203248193 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa00c0683b0 msgr2=0x7fa00c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:41.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.415+0000 7fa033a00640 1 --2- 172.21.15.32:0/1203248193 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa00c0683b0 0x7fa00c06a870 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa020000f70 tx=0x7fa020009040 comp rx=0 tx=0).stop 2024-04-17T09:57:41.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.415+0000 7fa033a00640 1 -- 172.21.15.32:0/1203248193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02c0cf370 msgr2=0x7fa02c158490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:41.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.415+0000 7fa033a00640 1 --2- 172.21.15.32:0/1203248193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02c0cf370 0x7fa02c158490 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa02400c7a0 tx=0x7fa02400cc60 comp rx=0 tx=0).stop 2024-04-17T09:57:41.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.415+0000 7fa033a00640 1 -- 172.21.15.32:0/1203248193 shutdown_connections 2024-04-17T09:57:41.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.415+0000 7fa033a00640 1 --2- 172.21.15.32:0/1203248193 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa00c0683b0 0x7fa00c06a870 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.415+0000 7fa033a00640 1 --2- 172.21.15.32:0/1203248193 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa02c071e60 0x7fa02c13caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.415+0000 7fa033a00640 1 --2- 172.21.15.32:0/1203248193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02c0cf370 0x7fa02c158490 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.415+0000 7fa033a00640 1 --2- 172.21.15.32:0/1203248193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa02c06ba30 0x7fa02c13c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.415+0000 7fa033a00640 1 -- 172.21.15.32:0/1203248193 >> 172.21.15.32:0/1203248193 conn(0x7fa02c0b2120 msgr2=0x7fa02c0b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:41.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.415+0000 7fa033a00640 1 -- 172.21.15.32:0/1203248193 shutdown_connections 2024-04-17T09:57:41.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.415+0000 7fa033a00640 1 -- 172.21.15.32:0/1203248193 wait complete. 2024-04-17T09:57:41.436 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T09:57:41.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.575+0000 7efc84800640 1 Processor -- start 2024-04-17T09:57:41.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.575+0000 7efc84800640 1 -- start start 2024-04-17T09:57:41.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.575+0000 7efc84800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc7c15eba0 0x7efc7c0700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:41.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.575+0000 7efc84800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efc7c0ce950 0x7efc7c0cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:41.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.575+0000 7efc84800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efc7c0add30 0x7efc7c0ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:41.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.575+0000 7efc84800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7efc7c06f0d0 con 0x7efc7c0add30 2024-04-17T09:57:41.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.575+0000 7efc84800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7efc7c067d60 con 0x7efc7c0ce950 2024-04-17T09:57:41.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.575+0000 7efc84800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efc7c06f320 con 0x7efc7c15eba0 2024-04-17T09:57:41.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.575+0000 7efc82800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efc7c0add30 0x7efc7c0ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:41.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.575+0000 7efc82800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efc7c0add30 0x7efc7c0ae1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34654/0 (socket says 172.21.15.32:34654) 2024-04-17T09:57:41.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.575+0000 7efc82800640 1 -- 172.21.15.32:0/2445543242 learned_addr learned my addr 172.21.15.32:0/2445543242 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:41.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.575+0000 7efc81400640 1 --2- 172.21.15.32:0/2445543242 >> v2:172.21.15.144:3300/0 conn(0x7efc7c0ce950 0x7efc7c0cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:41.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.575+0000 7efc81e00640 1 --2- 172.21.15.32:0/2445543242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc7c15eba0 0x7efc7c0700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:41.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.575+0000 7efc82800640 1 -- 172.21.15.32:0/2445543242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc7c15eba0 msgr2=0x7efc7c0700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:41.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.575+0000 7efc82800640 1 --2- 172.21.15.32:0/2445543242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc7c15eba0 0x7efc7c0700e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.575+0000 7efc82800640 1 -- 172.21.15.32:0/2445543242 >> v2:172.21.15.144:3300/0 conn(0x7efc7c0ce950 msgr2=0x7efc7c0cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:41.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.575+0000 7efc82800640 1 --2- 172.21.15.32:0/2445543242 >> v2:172.21.15.144:3300/0 conn(0x7efc7c0ce950 0x7efc7c0cedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.575+0000 7efc82800640 1 -- 172.21.15.32:0/2445543242 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc7c06f600 con 0x7efc7c0add30 2024-04-17T09:57:41.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.575+0000 7efc81400640 1 --2- 172.21.15.32:0/2445543242 >> v2:172.21.15.144:3300/0 conn(0x7efc7c0ce950 0x7efc7c0cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:57:41.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.579+0000 7efc82800640 1 --2- 172.21.15.32:0/2445543242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc7c0add30 0x7efc7c0ae1a0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7efc7c06e4f0 tx=0x7efc7801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be8c6630ed05cac server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:41.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.579+0000 7efc80a00640 1 -- 172.21.15.32:0/2445543242 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc780a56d0 con 0x7efc7c0add30 2024-04-17T09:57:41.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.579+0000 7efc80a00640 1 -- 172.21.15.32:0/2445543242 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc780a5870 con 0x7efc7c0add30 2024-04-17T09:57:41.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.579+0000 7efc80a00640 1 -- 172.21.15.32:0/2445543242 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc780a5c40 con 0x7efc7c0add30 2024-04-17T09:57:41.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.579+0000 7efc84800640 1 -- 172.21.15.32:0/2445543242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc7c0add30 msgr2=0x7efc7c0ae1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:41.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.579+0000 7efc84800640 1 --2- 172.21.15.32:0/2445543242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc7c0add30 0x7efc7c0ae1a0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7efc7c06e4f0 tx=0x7efc7801c960 comp rx=0 tx=0).stop 2024-04-17T09:57:41.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.579+0000 7efc84800640 1 -- 172.21.15.32:0/2445543242 shutdown_connections 2024-04-17T09:57:41.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.579+0000 7efc84800640 1 --2- 172.21.15.32:0/2445543242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc7c0add30 0x7efc7c0ae1a0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.579+0000 7efc84800640 1 --2- 172.21.15.32:0/2445543242 >> v2:172.21.15.144:3300/0 conn(0x7efc7c0ce950 0x7efc7c0cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.579+0000 7efc84800640 1 --2- 172.21.15.32:0/2445543242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc7c15eba0 0x7efc7c0700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.579+0000 7efc84800640 1 -- 172.21.15.32:0/2445543242 >> 172.21.15.32:0/2445543242 conn(0x7efc7c0cd800 msgr2=0x7efc7c0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:41.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.579+0000 7efc84800640 1 -- 172.21.15.32:0/2445543242 shutdown_connections 2024-04-17T09:57:41.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.579+0000 7efc84800640 1 -- 172.21.15.32:0/2445543242 wait complete. 2024-04-17T09:57:41.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc84800640 1 Processor -- start 2024-04-17T09:57:41.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc84800640 1 -- start start 2024-04-17T09:57:41.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc84800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc7c0add30 0x7efc7c154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:41.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc84800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc7c0ce950 0x7efc7c0aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:41.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc84800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc7c15eba0 0x7efc7c0af030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:41.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc84800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efc7c06c5d0 con 0x7efc7c15eba0 2024-04-17T09:57:41.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc84800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc7c06c450 con 0x7efc7c0add30 2024-04-17T09:57:41.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc84800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efc7c06c750 con 0x7efc7c0ce950 2024-04-17T09:57:41.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc82800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc7c15eba0 0x7efc7c0af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:41.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc81e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc7c0add30 0x7efc7c154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:41.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc81400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc7c0ce950 0x7efc7c0aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:41.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc81e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc7c0add30 0x7efc7c154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:54596/0 (socket says 172.21.15.32:54596) 2024-04-17T09:57:41.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc82800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc7c15eba0 0x7efc7c0af030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34662/0 (socket says 172.21.15.32:34662) 2024-04-17T09:57:41.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc81e00640 1 -- 172.21.15.32:0/4250334611 learned_addr learned my addr 172.21.15.32:0/4250334611 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:41.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc82800640 1 -- 172.21.15.32:0/4250334611 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc7c0ce950 msgr2=0x7efc7c0aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:41.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc82800640 1 --2- 172.21.15.32:0/4250334611 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc7c0ce950 0x7efc7c0aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc82800640 1 -- 172.21.15.32:0/4250334611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc7c0add30 msgr2=0x7efc7c154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:41.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc82800640 1 --2- 172.21.15.32:0/4250334611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc7c0add30 0x7efc7c154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc82800640 1 -- 172.21.15.32:0/4250334611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc7c1563f0 con 0x7efc7c15eba0 2024-04-17T09:57:41.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc82800640 1 --2- 172.21.15.32:0/4250334611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc7c15eba0 0x7efc7c0af030 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7efc780ab000 tx=0x7efc78019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:41.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc76000640 1 -- 172.21.15.32:0/4250334611 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc780bc030 con 0x7efc7c15eba0 2024-04-17T09:57:41.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc76000640 1 -- 172.21.15.32:0/4250334611 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc780b8470 con 0x7efc7c15eba0 2024-04-17T09:57:41.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc84800640 1 -- 172.21.15.32:0/4250334611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc7c155e30 con 0x7efc7c15eba0 2024-04-17T09:57:41.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.583+0000 7efc76000640 1 -- 172.21.15.32:0/4250334611 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc78009510 con 0x7efc7c15eba0 2024-04-17T09:57:41.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.587+0000 7efc84800640 1 -- 172.21.15.32:0/4250334611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc7c0c5b40 con 0x7efc7c15eba0 2024-04-17T09:57:41.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.587+0000 7efc84800640 1 -- 172.21.15.32:0/4250334611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc7c068a90 con 0x7efc7c15eba0 2024-04-17T09:57:41.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.587+0000 7efc76000640 1 -- 172.21.15.32:0/4250334611 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7efc780bf070 con 0x7efc7c15eba0 2024-04-17T09:57:41.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.587+0000 7efc76000640 1 --2- 172.21.15.32:0/4250334611 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efc60068380 0x7efc6006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:41.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.587+0000 7efc76000640 1 -- 172.21.15.32:0/4250334611 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7efc78121540 con 0x7efc7c15eba0 2024-04-17T09:57:41.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.587+0000 7efc81e00640 1 --2- 172.21.15.32:0/4250334611 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efc60068380 0x7efc6006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:41.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.587+0000 7efc81e00640 1 --2- 172.21.15.32:0/4250334611 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efc60068380 0x7efc6006a840 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7efc7c15df30 tx=0x7efc70009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:41.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.591+0000 7efc76000640 1 -- 172.21.15.32:0/4250334611 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7efc780c6070 con 0x7efc7c15eba0 2024-04-17T09:57:41.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.767+0000 7efc84800640 1 -- 172.21.15.32:0/4250334611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efc7c0cfec0 con 0x7efc7c15eba0 2024-04-17T09:57:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.767+0000 7efc76000640 1 -- 172.21.15.32:0/4250334611 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v69) ==== 80+0+120 (secure 0 0 0) 0x7efc7c1563f0 con 0x7efc7c15eba0 2024-04-17T09:57:41.774 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6836/663537244 2024-04-18T09:57:30.746884+0000 2024-04-17T09:57:41.774 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6834/663537244 2024-04-18T09:57:30.746884+0000 2024-04-17T09:57:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:listed 2 entries 2024-04-17T09:57:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.771+0000 7efc84800640 1 -- 172.21.15.32:0/4250334611 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efc60068380 msgr2=0x7efc6006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.771+0000 7efc84800640 1 --2- 172.21.15.32:0/4250334611 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efc60068380 0x7efc6006a840 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7efc7c15df30 tx=0x7efc70009190 comp rx=0 tx=0).stop 2024-04-17T09:57:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.771+0000 7efc84800640 1 -- 172.21.15.32:0/4250334611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc7c15eba0 msgr2=0x7efc7c0af030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.771+0000 7efc84800640 1 --2- 172.21.15.32:0/4250334611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc7c15eba0 0x7efc7c0af030 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7efc780ab000 tx=0x7efc78019700 comp rx=0 tx=0).stop 2024-04-17T09:57:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.771+0000 7efc84800640 1 -- 172.21.15.32:0/4250334611 shutdown_connections 2024-04-17T09:57:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.771+0000 7efc84800640 1 --2- 172.21.15.32:0/4250334611 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efc60068380 0x7efc6006a840 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.771+0000 7efc84800640 1 --2- 172.21.15.32:0/4250334611 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc7c0ce950 0x7efc7c0aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.771+0000 7efc84800640 1 --2- 172.21.15.32:0/4250334611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc7c15eba0 0x7efc7c0af030 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.771+0000 7efc84800640 1 --2- 172.21.15.32:0/4250334611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc7c0add30 0x7efc7c154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.771+0000 7efc84800640 1 -- 172.21.15.32:0/4250334611 >> 172.21.15.32:0/4250334611 conn(0x7efc7c0cd800 msgr2=0x7efc7c072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.771+0000 7efc84800640 1 -- 172.21.15.32:0/4250334611 shutdown_connections 2024-04-17T09:57:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:41.771+0000 7efc84800640 1 -- 172.21.15.32:0/4250334611 wait complete. 2024-04-17T09:57:41.786 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-17T09:57:41.783+0000 7f93cce00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-17T09:57:41.792 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2024-04-17T09:57:41.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:41.793 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T09:57:41.810 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[42561]: fuse finished with error 0 and tester_r 0 2024-04-17T09:57:41.813 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:57:41.863 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T09:57:41.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:41.864 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T09:57:41.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:41.873 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-04-17T09:57:42.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.059+0000 7f4025c00640 1 Processor -- start 2024-04-17T09:57:42.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.059+0000 7f4025c00640 1 -- start start 2024-04-17T09:57:42.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.059+0000 7f4025c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40200ceb40 0x7f4020071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.059+0000 7f4025c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40200cefa0 con 0x7f40200722f0 2024-04-17T09:57:42.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.059+0000 7f4025c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f402006eeb0 con 0x7f40201460a0 2024-04-17T09:57:42.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.059+0000 7f4025c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f402015de00 con 0x7f40200ceb40 2024-04-17T09:57:42.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.059+0000 7f401f400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f40200722f0 0x7f402015e550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37198/0 (socket says 172.21.15.32:37198) 2024-04-17T09:57:42.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.059+0000 7f401f400640 1 -- 172.21.15.32:0/1689570938 learned_addr learned my addr 172.21.15.32:0/1689570938 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:57:42.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f4017e00640 1 --2- 172.21.15.32:0/1689570938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40200ceb40 0x7f4020071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f401e000640 1 -- 172.21.15.32:0/1689570938 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3333577255 0 0) 0x7f40200cefa0 con 0x7f40200722f0 2024-04-17T09:57:42.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f401e000640 1 -- 172.21.15.32:0/1689570938 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4004003530 con 0x7f40200722f0 2024-04-17T09:57:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f401e000640 1 -- 172.21.15.32:0/1689570938 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1969496362 0 0) 0x7f4004003530 con 0x7f40200722f0 2024-04-17T09:57:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f401e000640 1 -- 172.21.15.32:0/1689570938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40200ceb40 msgr2=0x7f4020071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f401e000640 1 --2- 172.21.15.32:0/1689570938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40200ceb40 0x7f4020071db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f401e000640 1 -- 172.21.15.32:0/1689570938 >> v1:172.21.15.144:6789/0 conn(0x7f40201460a0 legacy=0x7f402006e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f401e000640 1 -- 172.21.15.32:0/1689570938 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402006f190 con 0x7f40200722f0 2024-04-17T09:57:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f401e000640 1 -- 172.21.15.32:0/1689570938 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f4018001a00 con 0x7f40200722f0 2024-04-17T09:57:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f401e000640 1 -- 172.21.15.32:0/1689570938 >> v1:172.21.15.32:6789/0 conn(0x7f40200722f0 legacy=0x7f402015e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f401e000640 1 --2- 172.21.15.32:0/1689570938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40040039f0 0x7f4004003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f401e000640 1 --2- 172.21.15.32:0/1689570938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40040043a0 0x7f4004004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f401e000640 1 --2- 172.21.15.32:0/1689570938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40200ceb40 0x7f40040050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f401e000640 1 -- 172.21.15.32:0/1689570938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f400400b9d0 con 0x7f40040043a0 2024-04-17T09:57:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f401e000640 1 -- 172.21.15.32:0/1689570938 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f400400bb50 con 0x7f40040039f0 2024-04-17T09:57:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f401e000640 1 -- 172.21.15.32:0/1689570938 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f400400bd50 con 0x7f40200ceb40 2024-04-17T09:57:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f4017e00640 1 --2- 172.21.15.32:0/1689570938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40040043a0 0x7f4004004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f401f400640 1 --2- 172.21.15.32:0/1689570938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40200ceb40 0x7f40040050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f401ea00640 1 --2- 172.21.15.32:0/1689570938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40040039f0 0x7f4004003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f4017e00640 1 -- 172.21.15.32:0/1689570938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40200ceb40 msgr2=0x7f40040050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:42.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f4017e00640 1 --2- 172.21.15.32:0/1689570938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40200ceb40 0x7f40040050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f4017e00640 1 -- 172.21.15.32:0/1689570938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40040039f0 msgr2=0x7f4004003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:42.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f4017e00640 1 --2- 172.21.15.32:0/1689570938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40040039f0 0x7f4004003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.063+0000 7f4017e00640 1 -- 172.21.15.32:0/1689570938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4004003530 con 0x7f40040043a0 2024-04-17T09:57:42.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.067+0000 7f4017e00640 1 --2- 172.21.15.32:0/1689570938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40040043a0 0x7f4004004870 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f4008009ac0 tx=0x7f400801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8c1a18a7dab423 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:42.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.067+0000 7f401e000640 1 -- 172.21.15.32:0/1689570938 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008009690 con 0x7f40040043a0 2024-04-17T09:57:42.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.067+0000 7f401e000640 1 -- 172.21.15.32:0/1689570938 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40080059a0 con 0x7f40040043a0 2024-04-17T09:57:42.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.067+0000 7f401e000640 1 -- 172.21.15.32:0/1689570938 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008005d90 con 0x7f40040043a0 2024-04-17T09:57:42.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.067+0000 7f4025c00640 1 -- 172.21.15.32:0/1689570938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40040043a0 msgr2=0x7f4004004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:42.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.067+0000 7f4025c00640 1 --2- 172.21.15.32:0/1689570938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40040043a0 0x7f4004004870 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f4008009ac0 tx=0x7f400801ca50 comp rx=0 tx=0).stop 2024-04-17T09:57:42.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.067+0000 7f401f400640 1 -- 172.21.15.32:0/1689570938 reap_dead start 2024-04-17T09:57:42.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.067+0000 7f4025c00640 1 -- 172.21.15.32:0/1689570938 shutdown_connections 2024-04-17T09:57:42.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.067+0000 7f4025c00640 1 -- 172.21.15.32:0/1689570938 >> 172.21.15.32:0/1689570938 conn(0x7f40200b2190 msgr2=0x7f40200b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:42.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.067+0000 7f4025c00640 1 -- 172.21.15.32:0/1689570938 shutdown_connections 2024-04-17T09:57:42.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.067+0000 7f4025c00640 1 -- 172.21.15.32:0/1689570938 wait complete. 2024-04-17T09:57:42.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.067+0000 7f4025c00640 1 Processor -- start 2024-04-17T09:57:42.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.067+0000 7f4025c00640 1 -- start start 2024-04-17T09:57:42.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.067+0000 7f4025c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40040043a0 0x7f4020159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.067+0000 7f4025c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f402013d9e0 0x7f402013c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.067+0000 7f4025c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4020139730 0x7f4020139be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.067+0000 7f4025c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4020071db0 con 0x7f40040043a0 2024-04-17T09:57:42.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.067+0000 7f4025c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f402015e030 con 0x7f402013d9e0 2024-04-17T09:57:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.067+0000 7f4025c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4020071f30 con 0x7f4020139730 2024-04-17T09:57:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.071+0000 7f401ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40040043a0 0x7f4020159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.071+0000 7f401ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40040043a0 0x7f4020159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34684/0 (socket says 172.21.15.32:34684) 2024-04-17T09:57:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.071+0000 7f401ea00640 1 -- 172.21.15.32:0/3676574786 learned_addr learned my addr 172.21.15.32:0/3676574786 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.071+0000 7f4017e00640 1 --2- 172.21.15.32:0/3676574786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f402013d9e0 0x7f402013c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.071+0000 7f401f400640 1 --2- 172.21.15.32:0/3676574786 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4020139730 0x7f4020139be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.071+0000 7f401ea00640 1 -- 172.21.15.32:0/3676574786 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4020139730 msgr2=0x7f4020139be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.071+0000 7f401ea00640 1 --2- 172.21.15.32:0/3676574786 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4020139730 0x7f4020139be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.071+0000 7f401ea00640 1 -- 172.21.15.32:0/3676574786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f402013d9e0 msgr2=0x7f402013c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.071+0000 7f401ea00640 1 --2- 172.21.15.32:0/3676574786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f402013d9e0 0x7f402013c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.071+0000 7f401ea00640 1 -- 172.21.15.32:0/3676574786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402015a6a0 con 0x7f40040043a0 2024-04-17T09:57:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.071+0000 7f401ea00640 1 --2- 172.21.15.32:0/3676574786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40040043a0 0x7f4020159c60 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f4010010f40 tx=0x7f401000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:42.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.071+0000 7f4016a00640 1 -- 172.21.15.32:0/3676574786 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010018020 con 0x7f40040043a0 2024-04-17T09:57:42.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.071+0000 7f4016a00640 1 -- 172.21.15.32:0/3676574786 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4010005290 con 0x7f40040043a0 2024-04-17T09:57:42.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.071+0000 7f4016a00640 1 -- 172.21.15.32:0/3676574786 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010003840 con 0x7f40040043a0 2024-04-17T09:57:42.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.071+0000 7f4025c00640 1 -- 172.21.15.32:0/3676574786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40200af1a0 con 0x7f40040043a0 2024-04-17T09:57:42.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.071+0000 7f4025c00640 1 -- 172.21.15.32:0/3676574786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4020143a50 con 0x7f40040043a0 2024-04-17T09:57:42.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.071+0000 7f4016a00640 1 -- 172.21.15.32:0/3676574786 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f401001b020 con 0x7f40040043a0 2024-04-17T09:57:42.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f4016a00640 1 --2- 172.21.15.32:0/3676574786 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3ffc0683b0 0x7f3ffc06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f4016a00640 1 -- 172.21.15.32:0/3676574786 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7f401008a330 con 0x7f40040043a0 2024-04-17T09:57:42.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f4017e00640 1 --2- 172.21.15.32:0/3676574786 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3ffc0683b0 0x7f3ffc06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f40290dd000 1 Processor -- start 2024-04-17T09:57:42.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f40290dd000 1 -- start start 2024-04-17T09:57:42.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f4017e00640 1 --2- 172.21.15.32:0/3676574786 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3ffc0683b0 0x7f3ffc06a870 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4008009ac0 tx=0x7f40080046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:42.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f40290dd000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b06375d0 0x55f9b06379b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f40290dd000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9b0637ef0 0x55f9b05eaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f40290dd000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9b05eaf70 0x55f9b05eb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f40290dd000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f9b043c380 con 0x55f9b06375d0 2024-04-17T09:57:42.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f40290dd000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f9b03f5520 con 0x55f9b0637ef0 2024-04-17T09:57:42.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f40290dd000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f9b0438340 con 0x55f9b05eaf70 2024-04-17T09:57:42.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f401ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b06375d0 0x55f9b06379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f401ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b06375d0 0x55f9b06379b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34692/0 (socket says 172.21.15.32:34692) 2024-04-17T09:57:42.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f401ea00640 1 -- 172.21.15.32:0/3884743184 learned_addr learned my addr 172.21.15.32:0/3884743184 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:42.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f401f400640 1 --2- 172.21.15.32:0/3884743184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9b05eaf70 0x55f9b05eb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f4017e00640 1 --2- 172.21.15.32:0/3884743184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9b0637ef0 0x55f9b05eaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f401ea00640 1 -- 172.21.15.32:0/3884743184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9b05eaf70 msgr2=0x55f9b05eb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:42.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f401ea00640 1 --2- 172.21.15.32:0/3884743184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9b05eaf70 0x55f9b05eb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f401ea00640 1 -- 172.21.15.32:0/3884743184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9b0637ef0 msgr2=0x55f9b05eaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:42.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f401ea00640 1 --2- 172.21.15.32:0/3884743184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9b0637ef0 0x55f9b05eaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f401ea00640 1 -- 172.21.15.32:0/3884743184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9b04e32e0 con 0x55f9b06375d0 2024-04-17T09:57:42.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f4017e00640 1 --2- 172.21.15.32:0/3884743184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9b0637ef0 0x55f9b05eaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:57:42.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f401ea00640 1 --2- 172.21.15.32:0/3884743184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b06375d0 0x55f9b06379b0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f4010088820 tx=0x7f4010059ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a3a32911cb7ec0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:42.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f400f400640 1 -- 172.21.15.32:0/3884743184 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010072020 con 0x55f9b06375d0 2024-04-17T09:57:42.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f400f400640 1 -- 172.21.15.32:0/3884743184 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f401005e540 con 0x55f9b06375d0 2024-04-17T09:57:42.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f400f400640 1 -- 172.21.15.32:0/3884743184 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010062d70 con 0x55f9b06375d0 2024-04-17T09:57:42.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f40290dd000 1 -- 172.21.15.32:0/3884743184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b06375d0 msgr2=0x55f9b06379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:42.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f40290dd000 1 --2- 172.21.15.32:0/3884743184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b06375d0 0x55f9b06379b0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f4010088820 tx=0x7f4010059ba0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.075+0000 7f40290dd000 1 -- 172.21.15.32:0/3884743184 shutdown_connections 2024-04-17T09:57:42.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 1 --2- 172.21.15.32:0/3884743184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9b05eaf70 0x55f9b05eb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 1 --2- 172.21.15.32:0/3884743184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9b0637ef0 0x55f9b05eaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 1 --2- 172.21.15.32:0/3884743184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b06375d0 0x55f9b06379b0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 1 -- 172.21.15.32:0/3884743184 >> 172.21.15.32:0/3884743184 conn(0x55f9b04d4b40 msgr2=0x55f9b061a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:42.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 1 -- 172.21.15.32:0/3884743184 shutdown_connections 2024-04-17T09:57:42.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 1 -- 172.21.15.32:0/3884743184 wait complete. 2024-04-17T09:57:42.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 1 -- start start 2024-04-17T09:57:42.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 1 Processor -- start 2024-04-17T09:57:42.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9b05eaf70 0x55f9b061fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9b06375d0 0x55f9b0620020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b0637ef0 0x55f9b0620560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f9b03dc3d0 con 0x55f9b0637ef0 2024-04-17T09:57:42.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f9b0404c30 con 0x55f9b06375d0 2024-04-17T09:57:42.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f9b04472b0 con 0x55f9b05eaf70 2024-04-17T09:57:42.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 10 client.0 _ll_register_callbacks cb 0x55f9b05d8ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:57:42.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f401f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b0637ef0 0x55f9b0620560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f401f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b0637ef0 0x55f9b0620560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34706/0 (socket says 172.21.15.32:34706) 2024-04-17T09:57:42.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f401f400640 1 -- 172.21.15.32:0/2510103363 learned_addr learned my addr 172.21.15.32:0/2510103363 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:42.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f4017e00640 1 --2- 172.21.15.32:0/2510103363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9b06375d0 0x55f9b0620020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f401ea00640 1 --2- 172.21.15.32:0/2510103363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9b05eaf70 0x55f9b061fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f401f400640 1 -- 172.21.15.32:0/2510103363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9b05eaf70 msgr2=0x55f9b061fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:42.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f401f400640 1 --2- 172.21.15.32:0/2510103363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9b05eaf70 0x55f9b061fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f401f400640 1 -- 172.21.15.32:0/2510103363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9b06375d0 msgr2=0x55f9b0620020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:42.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f401f400640 1 --2- 172.21.15.32:0/2510103363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9b06375d0 0x55f9b0620020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f401f400640 1 -- 172.21.15.32:0/2510103363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9b0481910 con 0x55f9b0637ef0 2024-04-17T09:57:42.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f401f400640 1 --2- 172.21.15.32:0/2510103363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b0637ef0 0x55f9b0620560 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f4020071a10 tx=0x7f4018007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b69ae3f00b7ff93 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:42.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f4003e00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:57:42.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f4003e00640 1 -- 172.21.15.32:0/2510103363 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4018004d80 con 0x55f9b0637ef0 2024-04-17T09:57:42.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f4003e00640 1 -- 172.21.15.32:0/2510103363 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7f4018010050 con 0x55f9b0637ef0 2024-04-17T09:57:42.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 1 -- 172.21.15.32:0/2510103363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f401005e540 con 0x55f9b0637ef0 2024-04-17T09:57:42.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f4003e00640 1 -- 172.21.15.32:0/2510103363 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f4018006110 con 0x55f9b0637ef0 2024-04-17T09:57:42.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 10 client.7167 fetch_fsmap learned FSMap version 49 2024-04-17T09:57:42.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 1 -- 172.21.15.32:0/2510103363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x55f9b04146a0 con 0x55f9b0637ef0 2024-04-17T09:57:42.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f4003e00640 1 -- 172.21.15.32:0/2510103363 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f401005e540 con 0x55f9b0637ef0 2024-04-17T09:57:42.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f4003e00640 1 -- 172.21.15.32:0/2510103363 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7f40180062b0 con 0x55f9b0637ef0 2024-04-17T09:57:42.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 10 client.7167 fetch_fsmap finished waiting for FSMap version 49 2024-04-17T09:57:42.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 10 client.7167 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-17T09:57:42.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 20 client.7167 populate_metadata read hostname 'smithi032' 2024-04-17T09:57:42.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 1 --2- 172.21.15.32:0/2510103363 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55f9b0621210 0x55f9b064d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 4 client.7167 mds_command: new command op to 7011 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:57:42.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.079+0000 7f40290dd000 1 -- 172.21.15.32:0/2510103363 --> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9b03d5df0 con 0x55f9b0621210 2024-04-17T09:57:42.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.083+0000 7f401ea00640 1 --2- 172.21.15.32:0/2510103363 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55f9b0621210 0x55f9b064d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.083+0000 7f401ea00640 1 --2- 172.21.15.32:0/2510103363 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55f9b0621210 0x55f9b064d6c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32eedf2ef5424821 server_cookie=84af6b879ebdbf50 in_seq=0 out_seq=0 2024-04-17T09:57:42.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.083+0000 7f4003e00640 10 client.7167 ms_handle_connect on v2:172.21.15.32:6834/2416784965 2024-04-17T09:57:42.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.083+0000 7f4003e00640 1 -- 172.21.15.32:0/2510103363 <== mds.0 v2:172.21.15.32:6834/2416784965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f9b03d5df0 con 0x55f9b0621210 2024-04-17T09:57:42.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.083+0000 7f4003e00640 10 client.7167 handle_command_reply: tid=0 2024-04-17T09:57:42.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.083+0000 7f40290dd000 1 client.7167 shutdown 2024-04-17T09:57:42.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.083+0000 7f40290dd000 1 -- 172.21.15.32:0/2510103363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b0637ef0 msgr2=0x55f9b0620560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:42.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.083+0000 7f40290dd000 1 --2- 172.21.15.32:0/2510103363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b0637ef0 0x55f9b0620560 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f4020071a10 tx=0x7f4018007700 comp rx=0 tx=0).stop 2024-04-17T09:57:42.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.083+0000 7f40290dd000 1 -- 172.21.15.32:0/2510103363 shutdown_connections 2024-04-17T09:57:42.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.083+0000 7f40290dd000 1 --2- 172.21.15.32:0/2510103363 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55f9b0621210 0x55f9b064d6c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.083+0000 7f40290dd000 1 --2- 172.21.15.32:0/2510103363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b0637ef0 0x55f9b0620560 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.083+0000 7f40290dd000 1 --2- 172.21.15.32:0/2510103363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9b06375d0 0x55f9b0620020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.083+0000 7f40290dd000 1 --2- 172.21.15.32:0/2510103363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9b05eaf70 0x55f9b061fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.083+0000 7f40290dd000 1 -- 172.21.15.32:0/2510103363 >> 172.21.15.32:0/2510103363 conn(0x55f9b04d4b40 msgr2=0x55f9b05ea130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:42.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.083+0000 7f4003e00640 0 client.7167 ms_handle_reset on v2:172.21.15.32:6834/2416784965 2024-04-17T09:57:42.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.083+0000 7f40290dd000 1 -- 172.21.15.32:0/2510103363 shutdown_connections 2024-04-17T09:57:42.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.083+0000 7f40290dd000 1 -- 172.21.15.32:0/2510103363 wait complete. 2024-04-17T09:57:42.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.083+0000 7f40290dd000 20 client.7167 trim_cache size 0 max 16384 2024-04-17T09:57:42.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.099+0000 7f40290dd000 1 Processor -- start 2024-04-17T09:57:42.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.099+0000 7f40290dd000 1 -- start start 2024-04-17T09:57:42.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f40290dd000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9b05eaf70 0x55f9b05f1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f40290dd000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9b06375d0 0x55f9b05f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f40290dd000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b0637ef0 0x55f9b05e9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f40290dd000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f40200b4940 con 0x55f9b0637ef0 2024-04-17T09:57:42.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f40290dd000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f40201452d0 con 0x55f9b06375d0 2024-04-17T09:57:42.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f40290dd000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f40201f67d0 con 0x55f9b05eaf70 2024-04-17T09:57:42.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f401f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b0637ef0 0x55f9b05e9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f401f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b0637ef0 0x55f9b05e9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34724/0 (socket says 172.21.15.32:34724) 2024-04-17T09:57:42.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f401f400640 1 -- 172.21.15.32:0/3836260987 learned_addr learned my addr 172.21.15.32:0/3836260987 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:42.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f4017e00640 1 --2- 172.21.15.32:0/3836260987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9b06375d0 0x55f9b05f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f401ea00640 1 --2- 172.21.15.32:0/3836260987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9b05eaf70 0x55f9b05f1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f401f400640 1 -- 172.21.15.32:0/3836260987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9b05eaf70 msgr2=0x55f9b05f1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:42.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f401f400640 1 --2- 172.21.15.32:0/3836260987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9b05eaf70 0x55f9b05f1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f401f400640 1 -- 172.21.15.32:0/3836260987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9b06375d0 msgr2=0x55f9b05f1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:42.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f401f400640 1 --2- 172.21.15.32:0/3836260987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9b06375d0 0x55f9b05f1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f401f400640 1 -- 172.21.15.32:0/3836260987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9b03dbc20 con 0x55f9b0637ef0 2024-04-17T09:57:42.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f401f400640 1 --2- 172.21.15.32:0/3836260987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b0637ef0 0x55f9b05e9190 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f4018001d00 tx=0x7f401800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7d801828d401ec server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:42.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f400ea00640 1 -- 172.21.15.32:0/3836260987 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4018001a60 con 0x55f9b0637ef0 2024-04-17T09:57:42.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f400ea00640 1 -- 172.21.15.32:0/3836260987 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4018004d80 con 0x55f9b0637ef0 2024-04-17T09:57:42.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f400ea00640 1 -- 172.21.15.32:0/3836260987 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40180062b0 con 0x55f9b0637ef0 2024-04-17T09:57:42.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f40290dd000 1 -- 172.21.15.32:0/3836260987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b0637ef0 msgr2=0x55f9b05e9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:42.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f40290dd000 1 --2- 172.21.15.32:0/3836260987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b0637ef0 0x55f9b05e9190 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f4018001d00 tx=0x7f401800e000 comp rx=0 tx=0).stop 2024-04-17T09:57:42.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f40290dd000 1 -- 172.21.15.32:0/3836260987 shutdown_connections 2024-04-17T09:57:42.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f40290dd000 1 --2- 172.21.15.32:0/3836260987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b0637ef0 0x55f9b05e9190 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f40290dd000 1 --2- 172.21.15.32:0/3836260987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9b06375d0 0x55f9b05f1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f40290dd000 1 --2- 172.21.15.32:0/3836260987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9b05eaf70 0x55f9b05f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f40290dd000 1 -- 172.21.15.32:0/3836260987 >> 172.21.15.32:0/3836260987 conn(0x55f9b04d4b40 msgr2=0x55f9b0648be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:42.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f40290dd000 1 -- 172.21.15.32:0/3836260987 shutdown_connections 2024-04-17T09:57:42.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f40290dd000 1 -- 172.21.15.32:0/3836260987 wait complete. 2024-04-17T09:57:42.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f40290dd000 1 -- start start 2024-04-17T09:57:42.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.103+0000 7f40290dd000 1 Processor -- start 2024-04-17T09:57:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b05eaf70 0x55f9b064e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9b06375d0 0x55f9b064ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9b0637ef0 0x55f9b064f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f9b043b690 con 0x55f9b05eaf70 2024-04-17T09:57:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f9b03fb2a0 con 0x55f9b0637ef0 2024-04-17T09:57:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f9b04409b0 con 0x55f9b06375d0 2024-04-17T09:57:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 10 client.0 _ll_register_callbacks cb 0x7f4018001880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:57:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f401ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b05eaf70 0x55f9b064e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f401ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b05eaf70 0x55f9b064e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34730/0 (socket says 172.21.15.32:34730) 2024-04-17T09:57:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f401ea00640 1 -- 172.21.15.32:0/2243182094 learned_addr learned my addr 172.21.15.32:0/2243182094 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:42.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f401f400640 1 --2- 172.21.15.32:0/2243182094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9b0637ef0 0x55f9b064f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f4017e00640 1 --2- 172.21.15.32:0/2243182094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9b06375d0 0x55f9b064ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f401ea00640 1 -- 172.21.15.32:0/2243182094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9b06375d0 msgr2=0x55f9b064ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f401ea00640 1 --2- 172.21.15.32:0/2243182094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9b06375d0 0x55f9b064ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f401ea00640 1 -- 172.21.15.32:0/2243182094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9b0637ef0 msgr2=0x55f9b064f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f401ea00640 1 --2- 172.21.15.32:0/2243182094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9b0637ef0 0x55f9b064f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f401ea00640 1 -- 172.21.15.32:0/2243182094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9b04bd3e0 con 0x55f9b05eaf70 2024-04-17T09:57:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f401ea00640 1 --2- 172.21.15.32:0/2243182094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b05eaf70 0x55f9b064e710 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f401000efd0 tx=0x7f4010001240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca77bcbdcc48232 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f4003e00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:57:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f4003e00640 1 -- 172.21.15.32:0/2243182094 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010072040 con 0x55f9b05eaf70 2024-04-17T09:57:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f4003e00640 1 -- 172.21.15.32:0/2243182094 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7f401008a070 con 0x55f9b05eaf70 2024-04-17T09:57:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 1 -- 172.21.15.32:0/2243182094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4018004d80 con 0x55f9b05eaf70 2024-04-17T09:57:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f4003e00640 1 -- 172.21.15.32:0/2243182094 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f40100564e0 con 0x55f9b05eaf70 2024-04-17T09:57:42.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 10 client.7173 fetch_fsmap learned FSMap version 49 2024-04-17T09:57:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 1 -- 172.21.15.32:0/2243182094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x55f9b03e1890 con 0x55f9b05eaf70 2024-04-17T09:57:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f4003e00640 1 -- 172.21.15.32:0/2243182094 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4018004d80 con 0x55f9b05eaf70 2024-04-17T09:57:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f4003e00640 1 -- 172.21.15.32:0/2243182094 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7f40100568b0 con 0x55f9b05eaf70 2024-04-17T09:57:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 10 client.7173 fetch_fsmap finished waiting for FSMap version 49 2024-04-17T09:57:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 10 client.7173 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-17T09:57:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 20 client.7173 populate_metadata read hostname 'smithi032' 2024-04-17T09:57:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 1 --2- 172.21.15.32:0/2243182094 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55f9b064fb20 0x55f9b06b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 4 client.7173 mds_command: new command op to 7011 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-04-17T09:57:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 1 -- 172.21.15.32:0/2243182094 --> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) -- 0x55f9b0449da0 con 0x55f9b064fb20 2024-04-17T09:57:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f4017e00640 1 --2- 172.21.15.32:0/2243182094 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55f9b064fb20 0x55f9b06b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f4017e00640 1 --2- 172.21.15.32:0/2243182094 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55f9b064fb20 0x55f9b06b36c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4f02207aed6b51a server_cookie=799283709d9e969d in_seq=0 out_seq=0 2024-04-17T09:57:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f4003e00640 10 client.7173 ms_handle_connect on v2:172.21.15.32:6834/2416784965 2024-04-17T09:57:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f4003e00640 1 -- 172.21.15.32:0/2243182094 <== mds.0 v2:172.21.15.32:6834/2416784965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f9b0449da0 con 0x55f9b064fb20 2024-04-17T09:57:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f4003e00640 10 client.7173 handle_command_reply: tid=0 2024-04-17T09:57:42.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 1 client.7173 shutdown 2024-04-17T09:57:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 1 -- 172.21.15.32:0/2243182094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b05eaf70 msgr2=0x55f9b064e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 1 --2- 172.21.15.32:0/2243182094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b05eaf70 0x55f9b064e710 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f401000efd0 tx=0x7f4010001240 comp rx=0 tx=0).stop 2024-04-17T09:57:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 1 -- 172.21.15.32:0/2243182094 shutdown_connections 2024-04-17T09:57:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 1 --2- 172.21.15.32:0/2243182094 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55f9b064fb20 0x55f9b06b36c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 1 --2- 172.21.15.32:0/2243182094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f9b06375d0 0x55f9b064ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 1 --2- 172.21.15.32:0/2243182094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9b0637ef0 0x55f9b064f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 1 --2- 172.21.15.32:0/2243182094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9b05eaf70 0x55f9b064e710 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 1 -- 172.21.15.32:0/2243182094 >> 172.21.15.32:0/2243182094 conn(0x55f9b04d4b40 msgr2=0x55f9b0635690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f4003e00640 0 client.7173 ms_handle_reset on v2:172.21.15.32:6834/2416784965 2024-04-17T09:57:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 1 -- 172.21.15.32:0/2243182094 shutdown_connections 2024-04-17T09:57:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 1 -- 172.21.15.32:0/2243182094 wait complete. 2024-04-17T09:57:42.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.107+0000 7f40290dd000 20 client.7173 trim_cache size 0 max 16384 2024-04-17T09:57:42.115 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:57:42.115 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-17T09:57:42.115 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "a41cda0d-1cd4-4ace-8417-3fd9f4007ff2", 2024-04-17T09:57:42.115 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-17T09:57:42.115 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:57:42.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.111+0000 7f4025c00640 1 -- 172.21.15.32:0/3676574786 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3ffc0683b0 msgr2=0x7f3ffc06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:42.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.111+0000 7f4025c00640 1 --2- 172.21.15.32:0/3676574786 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3ffc0683b0 0x7f3ffc06a870 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4008009ac0 tx=0x7f40080046d0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.111+0000 7f4025c00640 1 -- 172.21.15.32:0/3676574786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40040043a0 msgr2=0x7f4020159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:42.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.111+0000 7f4025c00640 1 --2- 172.21.15.32:0/3676574786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40040043a0 0x7f4020159c60 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f4010010f40 tx=0x7f401000e530 comp rx=0 tx=0).stop 2024-04-17T09:57:42.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.111+0000 7f4025c00640 1 -- 172.21.15.32:0/3676574786 shutdown_connections 2024-04-17T09:57:42.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.111+0000 7f4025c00640 1 --2- 172.21.15.32:0/3676574786 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3ffc0683b0 0x7f3ffc06a870 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.111+0000 7f4025c00640 1 --2- 172.21.15.32:0/3676574786 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4020139730 0x7f4020139be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.111+0000 7f4025c00640 1 --2- 172.21.15.32:0/3676574786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f402013d9e0 0x7f402013c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.111+0000 7f4025c00640 1 --2- 172.21.15.32:0/3676574786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40040043a0 0x7f4020159c60 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.111+0000 7f4025c00640 1 -- 172.21.15.32:0/3676574786 >> 172.21.15.32:0/3676574786 conn(0x7f40200b2190 msgr2=0x7f40200b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:42.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.111+0000 7f4025c00640 1 -- 172.21.15.32:0/3676574786 shutdown_connections 2024-04-17T09:57:42.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.111+0000 7f4025c00640 1 -- 172.21.15.32:0/3676574786 wait complete. 2024-04-17T09:57:42.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:57:42.131 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-17T09:57:42.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.271+0000 7fc16b200640 1 Processor -- start 2024-04-17T09:57:42.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.271+0000 7fc16b200640 1 -- start start 2024-04-17T09:57:42.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.271+0000 7fc16b200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc16406c140 0x7fc1640b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.271+0000 7fc16b200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1640b1f40 con 0x7fc16415ea80 2024-04-17T09:57:42.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.271+0000 7fc16b200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1640b29a0 con 0x7fc1640b0430 2024-04-17T09:57:42.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.271+0000 7fc16b200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc1640b2c80 con 0x7fc16406c140 2024-04-17T09:57:42.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.271+0000 7fc169400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc16406c140 0x7fc1640b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.271+0000 7fc169400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc16406c140 0x7fc1640b0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34742/0 (socket says 172.21.15.32:34742) 2024-04-17T09:57:42.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.271+0000 7fc169400640 1 -- 172.21.15.32:0/3593568681 learned_addr learned my addr 172.21.15.32:0/3593568681 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:42.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.271+0000 7fc169e00640 1 --1- 172.21.15.32:0/3593568681 >> v1:172.21.15.32:6789/0 conn(0x7fc16415ea80 0x7fc16406ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37208/0 (socket says 172.21.15.32:37208) 2024-04-17T09:57:42.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.271+0000 7fc169400640 1 -- 172.21.15.32:0/3593568681 >> v1:172.21.15.144:6789/0 conn(0x7fc1640b0430 legacy=0x7fc1640b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:42.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.271+0000 7fc169400640 1 -- 172.21.15.32:0/3593568681 >> v1:172.21.15.32:6789/0 conn(0x7fc16415ea80 legacy=0x7fc16406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:42.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.271+0000 7fc169400640 1 -- 172.21.15.32:0/3593568681 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1640b5dd0 con 0x7fc16406c140 2024-04-17T09:57:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.275+0000 7fc169400640 1 --2- 172.21.15.32:0/3593568681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc16406c140 0x7fc1640b0aa0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fc154009870 tx=0x7fc15401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80775c883706f6a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:42.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.275+0000 7fc168a00640 1 -- 172.21.15.32:0/3593568681 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1540b1070 con 0x7fc16406c140 2024-04-17T09:57:42.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.275+0000 7fc168a00640 1 -- 172.21.15.32:0/3593568681 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1540ac600 con 0x7fc16406c140 2024-04-17T09:57:42.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.275+0000 7fc168a00640 1 -- 172.21.15.32:0/3593568681 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1540ac9d0 con 0x7fc16406c140 2024-04-17T09:57:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.275+0000 7fc16b200640 1 -- 172.21.15.32:0/3593568681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc16406c140 msgr2=0x7fc1640b0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.275+0000 7fc16b200640 1 --2- 172.21.15.32:0/3593568681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc16406c140 0x7fc1640b0aa0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fc154009870 tx=0x7fc15401c490 comp rx=0 tx=0).stop 2024-04-17T09:57:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.275+0000 7fc16b200640 1 -- 172.21.15.32:0/3593568681 shutdown_connections 2024-04-17T09:57:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.275+0000 7fc16b200640 1 --2- 172.21.15.32:0/3593568681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc16406c140 0x7fc1640b0aa0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.275+0000 7fc16b200640 1 -- 172.21.15.32:0/3593568681 >> 172.21.15.32:0/3593568681 conn(0x7fc1640b6820 msgr2=0x7fc1640b6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.275+0000 7fc16b200640 1 -- 172.21.15.32:0/3593568681 shutdown_connections 2024-04-17T09:57:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.275+0000 7fc16b200640 1 -- 172.21.15.32:0/3593568681 wait complete. 2024-04-17T09:57:42.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.279+0000 7fc16b200640 1 Processor -- start 2024-04-17T09:57:42.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.279+0000 7fc16b200640 1 -- start start 2024-04-17T09:57:42.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.279+0000 7fc16b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc16406c140 0x7fc164148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.279+0000 7fc16b200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1640b0430 0x7fc164159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.279+0000 7fc16b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc16415ea80 0x7fc1641576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.279+0000 7fc16b200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc1640b1b40 con 0x7fc16415ea80 2024-04-17T09:57:42.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.279+0000 7fc16b200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc1640b19c0 con 0x7fc1640b0430 2024-04-17T09:57:42.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.279+0000 7fc16b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc1640b1cc0 con 0x7fc16406c140 2024-04-17T09:57:42.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.279+0000 7fc16a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc16415ea80 0x7fc1641576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.279+0000 7fc16a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc16415ea80 0x7fc1641576c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34752/0 (socket says 172.21.15.32:34752) 2024-04-17T09:57:42.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.279+0000 7fc16a800640 1 -- 172.21.15.32:0/1605777308 learned_addr learned my addr 172.21.15.32:0/1605777308 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:42.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.279+0000 7fc169400640 1 --2- 172.21.15.32:0/1605777308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1640b0430 0x7fc164159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.279+0000 7fc169e00640 1 --2- 172.21.15.32:0/1605777308 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc16406c140 0x7fc164148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.279+0000 7fc16a800640 1 -- 172.21.15.32:0/1605777308 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc16406c140 msgr2=0x7fc164148f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:42.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.279+0000 7fc16a800640 1 --2- 172.21.15.32:0/1605777308 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc16406c140 0x7fc164148f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.279+0000 7fc16a800640 1 -- 172.21.15.32:0/1605777308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1640b0430 msgr2=0x7fc164159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:42.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.279+0000 7fc16a800640 1 --2- 172.21.15.32:0/1605777308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1640b0430 0x7fc164159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.279+0000 7fc16a800640 1 -- 172.21.15.32:0/1605777308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc164160bd0 con 0x7fc16415ea80 2024-04-17T09:57:42.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.279+0000 7fc16a800640 1 --2- 172.21.15.32:0/1605777308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc16415ea80 0x7fc1641576c0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fc16000c780 tx=0x7fc16000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:42.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.279+0000 7fc15e000640 1 -- 172.21.15.32:0/1605777308 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160017070 con 0x7fc16415ea80 2024-04-17T09:57:42.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.279+0000 7fc15e000640 1 -- 172.21.15.32:0/1605777308 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc160003db0 con 0x7fc16415ea80 2024-04-17T09:57:42.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.279+0000 7fc16b200640 1 -- 172.21.15.32:0/1605777308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc164160eb0 con 0x7fc16415ea80 2024-04-17T09:57:42.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.279+0000 7fc15e000640 1 -- 172.21.15.32:0/1605777308 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1600052b0 con 0x7fc16415ea80 2024-04-17T09:57:42.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.283+0000 7fc16b200640 1 -- 172.21.15.32:0/1605777308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc164156ec0 con 0x7fc16415ea80 2024-04-17T09:57:42.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.283+0000 7fc15e000640 1 -- 172.21.15.32:0/1605777308 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc16001a070 con 0x7fc16415ea80 2024-04-17T09:57:42.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.283+0000 7fc15e000640 1 --2- 172.21.15.32:0/1605777308 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc148068380 0x7fc14806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.283+0000 7fc15e000640 1 -- 172.21.15.32:0/1605777308 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7fc160086090 con 0x7fc16415ea80 2024-04-17T09:57:42.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.283+0000 7fc16fc6d000 1 Processor -- start 2024-04-17T09:57:42.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.283+0000 7fc169e00640 1 --2- 172.21.15.32:0/1605777308 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc148068380 0x7fc14806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.283+0000 7fc16fc6d000 1 -- start start 2024-04-17T09:57:42.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.283+0000 7fc169e00640 1 --2- 172.21.15.32:0/1605777308 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc148068380 0x7fc14806a840 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc158000f70 tx=0x7fc158007e90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:42.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc16fc6d000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2dd260 0x556e2d2dd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc16fc6d000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e2d2ddb80 0x556e2d290460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc16fc6d000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556e2d2909a0 0x556e2d290d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc16fc6d000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556e2d0e24f0 con 0x556e2d2dd260 2024-04-17T09:57:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc16fc6d000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556e2d09b6c0 con 0x556e2d2ddb80 2024-04-17T09:57:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc16fc6d000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x556e2d0de4b0 con 0x556e2d2909a0 2024-04-17T09:57:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc169e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2dd260 0x556e2d2dd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc169e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2dd260 0x556e2d2dd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34754/0 (socket says 172.21.15.32:34754) 2024-04-17T09:57:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc169e00640 1 -- 172.21.15.32:0/3274977209 learned_addr learned my addr 172.21.15.32:0/3274977209 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc16a800640 1 --2- 172.21.15.32:0/3274977209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556e2d2909a0 0x556e2d290d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc169400640 1 --2- 172.21.15.32:0/3274977209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e2d2ddb80 0x556e2d290460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc169e00640 1 -- 172.21.15.32:0/3274977209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556e2d2909a0 msgr2=0x556e2d290d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc169e00640 1 --2- 172.21.15.32:0/3274977209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556e2d2909a0 0x556e2d290d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc169e00640 1 -- 172.21.15.32:0/3274977209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e2d2ddb80 msgr2=0x556e2d290460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc169e00640 1 --2- 172.21.15.32:0/3274977209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e2d2ddb80 0x556e2d290460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc169e00640 1 -- 172.21.15.32:0/3274977209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e2d1278e0 con 0x556e2d2dd260 2024-04-17T09:57:42.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc169400640 1 --2- 172.21.15.32:0/3274977209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e2d2ddb80 0x556e2d290460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:57:42.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc169e00640 1 --2- 172.21.15.32:0/3274977209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2dd260 0x556e2d2dd640 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fc1580077c0 tx=0x7fc158013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625c30419b592492 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:42.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc152a00640 1 -- 172.21.15.32:0/3274977209 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc158026070 con 0x556e2d2dd260 2024-04-17T09:57:42.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc152a00640 1 -- 172.21.15.32:0/3274977209 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc158013970 con 0x556e2d2dd260 2024-04-17T09:57:42.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc152a00640 1 -- 172.21.15.32:0/3274977209 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc158013d60 con 0x556e2d2dd260 2024-04-17T09:57:42.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc16fc6d000 1 -- 172.21.15.32:0/3274977209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2dd260 msgr2=0x556e2d2dd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:42.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc16fc6d000 1 --2- 172.21.15.32:0/3274977209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2dd260 0x556e2d2dd640 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fc1580077c0 tx=0x7fc158013000 comp rx=0 tx=0).stop 2024-04-17T09:57:42.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc16fc6d000 1 -- 172.21.15.32:0/3274977209 shutdown_connections 2024-04-17T09:57:42.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc16fc6d000 1 --2- 172.21.15.32:0/3274977209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556e2d2909a0 0x556e2d290d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc16fc6d000 1 --2- 172.21.15.32:0/3274977209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e2d2ddb80 0x556e2d290460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc16fc6d000 1 --2- 172.21.15.32:0/3274977209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2dd260 0x556e2d2dd640 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc16fc6d000 1 -- 172.21.15.32:0/3274977209 >> 172.21.15.32:0/3274977209 conn(0x556e2d2be000 msgr2=0x556e2d2c03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:42.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc16fc6d000 1 -- 172.21.15.32:0/3274977209 shutdown_connections 2024-04-17T09:57:42.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc16fc6d000 1 -- 172.21.15.32:0/3274977209 wait complete. 2024-04-17T09:57:42.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc16fc6d000 1 -- start start 2024-04-17T09:57:42.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.287+0000 7fc16fc6d000 1 Processor -- start 2024-04-17T09:57:42.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc16fc6d000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e2d2909a0 0x556e2d296020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc16fc6d000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2dd260 0x556e2d296560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc16fc6d000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556e2d2ddb80 0x556e2d296aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc16fc6d000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556e2d1637b0 con 0x556e2d2dd260 2024-04-17T09:57:42.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc16fc6d000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556e2d19dd00 con 0x556e2d2909a0 2024-04-17T09:57:42.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc16fc6d000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x556e2d0a72c0 con 0x556e2d2ddb80 2024-04-17T09:57:42.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc16fc6d000 10 client.0 _ll_register_callbacks cb 0x556e2d0b7030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:57:42.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc169400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2dd260 0x556e2d296560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc169400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2dd260 0x556e2d296560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34760/0 (socket says 172.21.15.32:34760) 2024-04-17T09:57:42.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc169400640 1 -- 172.21.15.32:0/2352949281 learned_addr learned my addr 172.21.15.32:0/2352949281 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:42.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc169e00640 1 --2- 172.21.15.32:0/2352949281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e2d2909a0 0x556e2d296020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc16a800640 1 --2- 172.21.15.32:0/2352949281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556e2d2ddb80 0x556e2d296aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc169400640 1 -- 172.21.15.32:0/2352949281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556e2d2ddb80 msgr2=0x556e2d296aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:42.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc169400640 1 --2- 172.21.15.32:0/2352949281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556e2d2ddb80 0x556e2d296aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc169400640 1 -- 172.21.15.32:0/2352949281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e2d2909a0 msgr2=0x556e2d296020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:42.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc169400640 1 --2- 172.21.15.32:0/2352949281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e2d2909a0 0x556e2d296020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc169400640 1 -- 172.21.15.32:0/2352949281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e2d0ba490 con 0x556e2d2dd260 2024-04-17T09:57:42.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc169400640 1 --2- 172.21.15.32:0/2352949281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2dd260 0x556e2d296560 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fc154009870 tx=0x7fc15401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b485422e62aa77 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:42.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc147400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:57:42.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc147400640 1 -- 172.21.15.32:0/2352949281 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1540b1040 con 0x556e2d2dd260 2024-04-17T09:57:42.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc16fc6d000 1 -- 172.21.15.32:0/2352949281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc158013970 con 0x556e2d2dd260 2024-04-17T09:57:42.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc147400640 1 -- 172.21.15.32:0/2352949281 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7fc1540c1430 con 0x556e2d2dd260 2024-04-17T09:57:42.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc147400640 1 -- 172.21.15.32:0/2352949281 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fc1540c8050 con 0x556e2d2dd260 2024-04-17T09:57:42.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc16fc6d000 10 client.7188 fetch_fsmap learned FSMap version 49 2024-04-17T09:57:42.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc16fc6d000 1 -- 172.21.15.32:0/2352949281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x556e2d0adeb0 con 0x556e2d2dd260 2024-04-17T09:57:42.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc147400640 1 -- 172.21.15.32:0/2352949281 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc158013970 con 0x556e2d2dd260 2024-04-17T09:57:42.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc147400640 1 -- 172.21.15.32:0/2352949281 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7fc1540cb020 con 0x556e2d2dd260 2024-04-17T09:57:42.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc16fc6d000 10 client.7188 fetch_fsmap finished waiting for FSMap version 49 2024-04-17T09:57:42.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc16fc6d000 10 client.7188 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-17T09:57:42.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc16fc6d000 20 client.7188 populate_metadata read hostname 'smithi032' 2024-04-17T09:57:42.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc16fc6d000 1 --2- 172.21.15.32:0/2352949281 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x556e2d2ec500 0x556e2d2ee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc16fc6d000 4 client.7188 mds_command: new command op to 7011 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:57:42.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc16fc6d000 1 -- 172.21.15.32:0/2352949281 --> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e2d07bdd0 con 0x556e2d2ec500 2024-04-17T09:57:42.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc169e00640 1 --2- 172.21.15.32:0/2352949281 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x556e2d2ec500 0x556e2d2ee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc169e00640 1 --2- 172.21.15.32:0/2352949281 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x556e2d2ec500 0x556e2d2ee8f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80410c1bdc01e18c server_cookie=60efdd1ec37e31fe in_seq=0 out_seq=0 2024-04-17T09:57:42.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc147400640 10 client.7188 ms_handle_connect on v2:172.21.15.32:6834/2416784965 2024-04-17T09:57:42.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc147400640 1 -- 172.21.15.32:0/2352949281 <== mds.0 v2:172.21.15.32:6834/2416784965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e2d07bdd0 con 0x556e2d2ec500 2024-04-17T09:57:42.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc147400640 10 client.7188 handle_command_reply: tid=0 2024-04-17T09:57:42.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.291+0000 7fc16fc6d000 1 client.7188 shutdown 2024-04-17T09:57:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.295+0000 7fc16fc6d000 1 -- 172.21.15.32:0/2352949281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2dd260 msgr2=0x556e2d296560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.295+0000 7fc16fc6d000 1 --2- 172.21.15.32:0/2352949281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2dd260 0x556e2d296560 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fc154009870 tx=0x7fc15401acd0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.295+0000 7fc16fc6d000 1 -- 172.21.15.32:0/2352949281 shutdown_connections 2024-04-17T09:57:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.295+0000 7fc16fc6d000 1 --2- 172.21.15.32:0/2352949281 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x556e2d2ec500 0x556e2d2ee8f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.295+0000 7fc16fc6d000 1 --2- 172.21.15.32:0/2352949281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556e2d2ddb80 0x556e2d296aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.295+0000 7fc16fc6d000 1 --2- 172.21.15.32:0/2352949281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2dd260 0x556e2d296560 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.295+0000 7fc16fc6d000 1 --2- 172.21.15.32:0/2352949281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e2d2909a0 0x556e2d296020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.295+0000 7fc16fc6d000 1 -- 172.21.15.32:0/2352949281 >> 172.21.15.32:0/2352949281 conn(0x556e2d2be000 msgr2=0x556e2d2c03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.295+0000 7fc147400640 0 client.7188 ms_handle_reset on v2:172.21.15.32:6834/2416784965 2024-04-17T09:57:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.295+0000 7fc16fc6d000 1 -- 172.21.15.32:0/2352949281 shutdown_connections 2024-04-17T09:57:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.295+0000 7fc16fc6d000 1 -- 172.21.15.32:0/2352949281 wait complete. 2024-04-17T09:57:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.295+0000 7fc16fc6d000 20 client.7188 trim_cache size 0 max 16384 2024-04-17T09:57:42.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.311+0000 7fc16fc6d000 1 Processor -- start 2024-04-17T09:57:42.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.311+0000 7fc16fc6d000 1 -- start start 2024-04-17T09:57:42.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.311+0000 7fc16fc6d000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2be000 0x556e2d2dc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.311+0000 7fc16fc6d000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e2d2909a0 0x556e2d2dc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.311+0000 7fc16fc6d000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556e2d2dd260 0x556e2d2ee510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.311+0000 7fc16fc6d000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc16415fca0 con 0x556e2d2be000 2024-04-17T09:57:42.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.311+0000 7fc16fc6d000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc16415f000 con 0x556e2d2909a0 2024-04-17T09:57:42.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.311+0000 7fc16fc6d000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc16415f650 con 0x556e2d2dd260 2024-04-17T09:57:42.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.311+0000 7fc169e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2be000 0x556e2d2dc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.311+0000 7fc169e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2be000 0x556e2d2dc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34770/0 (socket says 172.21.15.32:34770) 2024-04-17T09:57:42.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.311+0000 7fc169e00640 1 -- 172.21.15.32:0/48368969 learned_addr learned my addr 172.21.15.32:0/48368969 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:42.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.311+0000 7fc169e00640 1 -- 172.21.15.32:0/48368969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556e2d2dd260 msgr2=0x556e2d2ee510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:42.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.311+0000 7fc169e00640 1 --2- 172.21.15.32:0/48368969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556e2d2dd260 0x556e2d2ee510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.311+0000 7fc169e00640 1 -- 172.21.15.32:0/48368969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e2d2909a0 msgr2=0x556e2d2dc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:42.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.311+0000 7fc169e00640 1 --2- 172.21.15.32:0/48368969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e2d2909a0 0x556e2d2dc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.311+0000 7fc169e00640 1 -- 172.21.15.32:0/48368969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e2d2dd9b0 con 0x556e2d2be000 2024-04-17T09:57:42.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.311+0000 7fc169e00640 1 --2- 172.21.15.32:0/48368969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2be000 0x556e2d2dc320 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x556e2d0a14b0 tx=0x7fc158014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff2f71b90b2384d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:42.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.311+0000 7fc152000640 1 -- 172.21.15.32:0/48368969 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc158026040 con 0x556e2d2be000 2024-04-17T09:57:42.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.311+0000 7fc152000640 1 -- 172.21.15.32:0/48368969 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc158027040 con 0x556e2d2be000 2024-04-17T09:57:42.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.311+0000 7fc152000640 1 -- 172.21.15.32:0/48368969 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1580331c0 con 0x556e2d2be000 2024-04-17T09:57:42.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 1 -- 172.21.15.32:0/48368969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2be000 msgr2=0x556e2d2dc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 1 --2- 172.21.15.32:0/48368969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2be000 0x556e2d2dc320 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x556e2d0a14b0 tx=0x7fc158014000 comp rx=0 tx=0).stop 2024-04-17T09:57:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 1 -- 172.21.15.32:0/48368969 shutdown_connections 2024-04-17T09:57:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 1 --2- 172.21.15.32:0/48368969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556e2d2dd260 0x556e2d2ee510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 1 --2- 172.21.15.32:0/48368969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e2d2909a0 0x556e2d2dc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 1 --2- 172.21.15.32:0/48368969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2be000 0x556e2d2dc320 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 1 -- 172.21.15.32:0/48368969 >> 172.21.15.32:0/48368969 conn(0x556e2d298a20 msgr2=0x556e2d28fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 1 -- 172.21.15.32:0/48368969 shutdown_connections 2024-04-17T09:57:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 1 -- 172.21.15.32:0/48368969 wait complete. 2024-04-17T09:57:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 1 -- start start 2024-04-17T09:57:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 1 Processor -- start 2024-04-17T09:57:42.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e2d2909a0 0x556e2d34cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556e2d2be000 0x556e2d34d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2dd260 0x556e2d308480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x556e2d086050 con 0x556e2d2dd260 2024-04-17T09:57:42.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556e2d0a4570 con 0x556e2d2909a0 2024-04-17T09:57:42.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x556e2d0e1800 con 0x556e2d2be000 2024-04-17T09:57:42.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 10 client.0 _ll_register_callbacks cb 0x7fc1540096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:57:42.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2dd260 0x556e2d308480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2dd260 0x556e2d308480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34780/0 (socket says 172.21.15.32:34780) 2024-04-17T09:57:42.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16a800640 1 -- 172.21.15.32:0/3971627887 learned_addr learned my addr 172.21.15.32:0/3971627887 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:57:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc169e00640 1 --2- 172.21.15.32:0/3971627887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e2d2909a0 0x556e2d34cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc169400640 1 --2- 172.21.15.32:0/3971627887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556e2d2be000 0x556e2d34d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16a800640 1 -- 172.21.15.32:0/3971627887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556e2d2be000 msgr2=0x556e2d34d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16a800640 1 --2- 172.21.15.32:0/3971627887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556e2d2be000 0x556e2d34d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16a800640 1 -- 172.21.15.32:0/3971627887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e2d2909a0 msgr2=0x556e2d34cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:57:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16a800640 1 --2- 172.21.15.32:0/3971627887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e2d2909a0 0x556e2d34cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16a800640 1 -- 172.21.15.32:0/3971627887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e2d09ca20 con 0x556e2d2dd260 2024-04-17T09:57:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16a800640 1 --2- 172.21.15.32:0/3971627887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2dd260 0x556e2d308480 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fc16000cf40 tx=0x7fc160056080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1454d3254892968 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:57:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc147400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:57:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc147400640 1 -- 172.21.15.32:0/3971627887 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160072030 con 0x556e2d2dd260 2024-04-17T09:57:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc147400640 1 -- 172.21.15.32:0/3971627887 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7fc16005e5a0 con 0x556e2d2dd260 2024-04-17T09:57:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 1 -- 172.21.15.32:0/3971627887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc158027040 con 0x556e2d2dd260 2024-04-17T09:57:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc147400640 1 -- 172.21.15.32:0/3971627887 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fc1600542c0 con 0x556e2d2dd260 2024-04-17T09:57:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 10 client.7194 fetch_fsmap learned FSMap version 49 2024-04-17T09:57:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 1 -- 172.21.15.32:0/3971627887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x556e2d163c40 con 0x556e2d2dd260 2024-04-17T09:57:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc147400640 1 -- 172.21.15.32:0/3971627887 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc158027040 con 0x556e2d2dd260 2024-04-17T09:57:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc147400640 1 -- 172.21.15.32:0/3971627887 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7fc160074ce0 con 0x556e2d2dd260 2024-04-17T09:57:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 10 client.7194 fetch_fsmap finished waiting for FSMap version 49 2024-04-17T09:57:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 10 client.7194 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-17T09:57:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 20 client.7194 populate_metadata read hostname 'smithi032' 2024-04-17T09:57:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 1 --2- 172.21.15.32:0/3971627887 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x556e2d2ddb80 0x556e2d308e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:57:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 4 client.7194 mds_command: new command op to 7011 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T09:57:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.315+0000 7fc16fc6d000 1 -- 172.21.15.32:0/3971627887 --> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556e2d0eff10 con 0x556e2d2ddb80 2024-04-17T09:57:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc169e00640 1 --2- 172.21.15.32:0/3971627887 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x556e2d2ddb80 0x556e2d308e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:57:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc169e00640 1 --2- 172.21.15.32:0/3971627887 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x556e2d2ddb80 0x556e2d308e40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4124ffb6c3c9db1 server_cookie=4fba9ca48a1acd6e in_seq=0 out_seq=0 2024-04-17T09:57:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc147400640 10 client.7194 ms_handle_connect on v2:172.21.15.32:6834/2416784965 2024-04-17T09:57:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc147400640 1 -- 172.21.15.32:0/3971627887 <== mds.0 v2:172.21.15.32:6834/2416784965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556e2d0eff10 con 0x556e2d2ddb80 2024-04-17T09:57:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc147400640 10 client.7194 handle_command_reply: tid=0 2024-04-17T09:57:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16fc6d000 1 client.7194 shutdown 2024-04-17T09:57:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16fc6d000 1 -- 172.21.15.32:0/3971627887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2dd260 msgr2=0x556e2d308480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16fc6d000 1 --2- 172.21.15.32:0/3971627887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2dd260 0x556e2d308480 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fc16000cf40 tx=0x7fc160056080 comp rx=0 tx=0).stop 2024-04-17T09:57:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16fc6d000 1 -- 172.21.15.32:0/3971627887 shutdown_connections 2024-04-17T09:57:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16fc6d000 1 --2- 172.21.15.32:0/3971627887 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x556e2d2ddb80 0x556e2d308e40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16fc6d000 1 --2- 172.21.15.32:0/3971627887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556e2d2be000 0x556e2d34d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16fc6d000 1 --2- 172.21.15.32:0/3971627887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556e2d2dd260 0x556e2d308480 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16fc6d000 1 --2- 172.21.15.32:0/3971627887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e2d2909a0 0x556e2d34cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc147400640 0 client.7194 ms_handle_reset on v2:172.21.15.32:6834/2416784965 2024-04-17T09:57:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16fc6d000 1 -- 172.21.15.32:0/3971627887 >> 172.21.15.32:0/3971627887 conn(0x556e2d298a20 msgr2=0x556e2d2db310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16fc6d000 1 -- 172.21.15.32:0/3971627887 shutdown_connections 2024-04-17T09:57:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16fc6d000 1 -- 172.21.15.32:0/3971627887 wait complete. 2024-04-17T09:57:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16fc6d000 20 client.7194 trim_cache size 0 max 16384 2024-04-17T09:57:42.410 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:57:42.410 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T09:57:42.410 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-17T09:57:42.410 INFO:teuthology.orchestra.run.smithi032.stdout: "a41cda0d-1cd4-4ace-8417-3fd9f4007ff2": { 2024-04-17T09:57:42.411 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/subdir", 2024-04-17T09:57:42.411 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "a41cda0d-1cd4-4ace-8417-3fd9f4007ff2", 2024-04-17T09:57:42.411 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive" 2024-04-17T09:57:42.411 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:42.411 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:57:42.411 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:57:42.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16b200640 1 -- 172.21.15.32:0/1605777308 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc148068380 msgr2=0x7fc14806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:42.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16b200640 1 --2- 172.21.15.32:0/1605777308 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc148068380 0x7fc14806a840 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc158000f70 tx=0x7fc158007e90 comp rx=0 tx=0).stop 2024-04-17T09:57:42.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16b200640 1 -- 172.21.15.32:0/1605777308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc16415ea80 msgr2=0x7fc1641576c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:57:42.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16b200640 1 --2- 172.21.15.32:0/1605777308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc16415ea80 0x7fc1641576c0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fc16000c780 tx=0x7fc16000cc40 comp rx=0 tx=0).stop 2024-04-17T09:57:42.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16b200640 1 -- 172.21.15.32:0/1605777308 shutdown_connections 2024-04-17T09:57:42.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16b200640 1 --2- 172.21.15.32:0/1605777308 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc148068380 0x7fc14806a840 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16b200640 1 --2- 172.21.15.32:0/1605777308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc16415ea80 0x7fc1641576c0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16b200640 1 --2- 172.21.15.32:0/1605777308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1640b0430 0x7fc164159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16b200640 1 --2- 172.21.15.32:0/1605777308 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc16406c140 0x7fc164148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:57:42.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16b200640 1 -- 172.21.15.32:0/1605777308 >> 172.21.15.32:0/1605777308 conn(0x7fc1640b6820 msgr2=0x7fc1640707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:57:42.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16b200640 1 -- 172.21.15.32:0/1605777308 shutdown_connections 2024-04-17T09:57:42.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:57:42.319+0000 7fc16b200640 1 -- 172.21.15.32:0/1605777308 wait complete. 2024-04-17T09:57:42.414 INFO:tasks.cephfs.filesystem:scrub status for tag:a41cda0d-1cd4-4ace-8417-3fd9f4007ff2 - {'path': '/subdir', 'tag': 'a41cda0d-1cd4-4ace-8417-3fd9f4007ff2', 'options': 'recursive'} 2024-04-17T09:57:43.634 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:57:43.638 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:58:12.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:58:12.416 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-17T09:58:12.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6e1c00640 1 Processor -- start 2024-04-17T09:58:12.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6e1c00640 1 -- start start 2024-04-17T09:58:12.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6e1c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6dc06f7c0 0x7fd6dc06ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6e1c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6dc0cc940 con 0x7fd6dc15b7d0 2024-04-17T09:58:12.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6e1c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6dc0ccc20 con 0x7fd6dc06f270 2024-04-17T09:58:12.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6e1c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd6dc06e0b0 con 0x7fd6dc06f7c0 2024-04-17T09:58:12.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6daa00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd6dc15b7d0 0x7fd6dc06e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48954/0 (socket says 172.21.15.32:48954) 2024-04-17T09:58:12.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6daa00640 1 -- 172.21.15.32:0/3703051576 learned_addr learned my addr 172.21.15.32:0/3703051576 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:58:12.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6d3e00640 1 --2- 172.21.15.32:0/3703051576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6dc06f7c0 0x7fd6dc06ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6da000640 1 -- 172.21.15.32:0/3703051576 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2930430712 0 0) 0x7fd6dc0cc940 con 0x7fd6dc15b7d0 2024-04-17T09:58:12.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6da000640 1 -- 172.21.15.32:0/3703051576 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6c0003530 con 0x7fd6dc15b7d0 2024-04-17T09:58:12.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6da000640 1 -- 172.21.15.32:0/3703051576 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 690509083 0 0) 0x7fd6dc0ccc20 con 0x7fd6dc06f270 2024-04-17T09:58:12.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6da000640 1 -- 172.21.15.32:0/3703051576 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6dc0cc940 con 0x7fd6dc06f270 2024-04-17T09:58:12.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6da000640 1 -- 172.21.15.32:0/3703051576 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1186290600 0 0) 0x7fd6c0003530 con 0x7fd6dc15b7d0 2024-04-17T09:58:12.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6da000640 1 -- 172.21.15.32:0/3703051576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6dc06f7c0 msgr2=0x7fd6dc06ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:12.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6da000640 1 --2- 172.21.15.32:0/3703051576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6dc06f7c0 0x7fd6dc06ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6da000640 1 -- 172.21.15.32:0/3703051576 >> v1:172.21.15.144:6789/0 conn(0x7fd6dc06f270 legacy=0x7fd6dc0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:12.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6da000640 1 -- 172.21.15.32:0/3703051576 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6dc06e390 con 0x7fd6dc15b7d0 2024-04-17T09:58:12.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6da000640 1 -- 172.21.15.32:0/3703051576 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7fd6cc001680 con 0x7fd6dc15b7d0 2024-04-17T09:58:12.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6da000640 1 -- 172.21.15.32:0/3703051576 >> v1:172.21.15.32:6789/0 conn(0x7fd6dc15b7d0 legacy=0x7fd6dc06e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:12.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6da000640 1 --2- 172.21.15.32:0/3703051576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6dc06f7c0 0x7fd6c0003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6da000640 1 --2- 172.21.15.32:0/3703051576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c0004130 0x7fd6c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6da000640 1 --2- 172.21.15.32:0/3703051576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6c0004d50 0x7fd6c00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6da000640 1 -- 172.21.15.32:0/3703051576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc06e0b0 con 0x7fd6c0004d50 2024-04-17T09:58:12.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6da000640 1 -- 172.21.15.32:0/3703051576 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c000bb90 con 0x7fd6c0004130 2024-04-17T09:58:12.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6da000640 1 -- 172.21.15.32:0/3703051576 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c000bd90 con 0x7fd6dc06f7c0 2024-04-17T09:58:12.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6db400640 1 --2- 172.21.15.32:0/3703051576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6c0004d50 0x7fd6c00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6daa00640 1 --2- 172.21.15.32:0/3703051576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6dc06f7c0 0x7fd6c0003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6d3e00640 1 --2- 172.21.15.32:0/3703051576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c0004130 0x7fd6c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6db400640 1 -- 172.21.15.32:0/3703051576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6dc06f7c0 msgr2=0x7fd6c0003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:12.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6db400640 1 --2- 172.21.15.32:0/3703051576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6dc06f7c0 0x7fd6c0003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6db400640 1 -- 172.21.15.32:0/3703051576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c0004130 msgr2=0x7fd6c0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:12.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6db400640 1 --2- 172.21.15.32:0/3703051576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c0004130 0x7fd6c0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.571+0000 7fd6db400640 1 -- 172.21.15.32:0/3703051576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c0003530 con 0x7fd6c0004d50 2024-04-17T09:58:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.575+0000 7fd6db400640 1 --2- 172.21.15.32:0/3703051576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6c0004d50 0x7fd6c00051d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4001fd0 tx=0x7fd6d401f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cac4eebb63255c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.575+0000 7fd6da000640 1 -- 172.21.15.32:0/3703051576 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4001c20 con 0x7fd6c0004d50 2024-04-17T09:58:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.575+0000 7fd6da000640 1 -- 172.21.15.32:0/3703051576 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d40a8cb0 con 0x7fd6c0004d50 2024-04-17T09:58:12.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.575+0000 7fd6da000640 1 -- 172.21.15.32:0/3703051576 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40a8e60 con 0x7fd6c0004d50 2024-04-17T09:58:12.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.575+0000 7fd6e1c00640 1 -- 172.21.15.32:0/3703051576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6c0004d50 msgr2=0x7fd6c00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:12.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.575+0000 7fd6e1c00640 1 --2- 172.21.15.32:0/3703051576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6c0004d50 0x7fd6c00051d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4001fd0 tx=0x7fd6d401f070 comp rx=0 tx=0).stop 2024-04-17T09:58:12.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.575+0000 7fd6db400640 1 -- 172.21.15.32:0/3703051576 reap_dead start 2024-04-17T09:58:12.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.575+0000 7fd6e1c00640 1 -- 172.21.15.32:0/3703051576 shutdown_connections 2024-04-17T09:58:12.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.575+0000 7fd6e1c00640 1 -- 172.21.15.32:0/3703051576 >> 172.21.15.32:0/3703051576 conn(0x7fd6dc0b2120 msgr2=0x7fd6dc0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:12.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.575+0000 7fd6e1c00640 1 -- 172.21.15.32:0/3703051576 shutdown_connections 2024-04-17T09:58:12.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.575+0000 7fd6e1c00640 1 -- 172.21.15.32:0/3703051576 wait complete. 2024-04-17T09:58:12.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6e1c00640 1 Processor -- start 2024-04-17T09:58:12.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6e1c00640 1 -- start start 2024-04-17T09:58:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6e1c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6c0004d50 0x7fd6dc15cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6e1c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6dc1e4db0 0x7fd6dc1f4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6e1c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6dc14d300 0x7fd6dc14d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6e1c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc0ceb20 con 0x7fd6c0004d50 2024-04-17T09:58:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6e1c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc0ce9a0 con 0x7fd6dc1e4db0 2024-04-17T09:58:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6e1c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd6dc0ceca0 con 0x7fd6dc14d300 2024-04-17T09:58:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6daa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6c0004d50 0x7fd6dc15cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6daa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6c0004d50 0x7fd6dc15cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55824/0 (socket says 172.21.15.32:55824) 2024-04-17T09:58:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6daa00640 1 -- 172.21.15.32:0/1463634813 learned_addr learned my addr 172.21.15.32:0/1463634813 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6d3e00640 1 --2- 172.21.15.32:0/1463634813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6dc1e4db0 0x7fd6dc1f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6db400640 1 --2- 172.21.15.32:0/1463634813 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6dc14d300 0x7fd6dc14d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6daa00640 1 -- 172.21.15.32:0/1463634813 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6dc14d300 msgr2=0x7fd6dc14d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6daa00640 1 --2- 172.21.15.32:0/1463634813 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6dc14d300 0x7fd6dc14d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6daa00640 1 -- 172.21.15.32:0/1463634813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6dc1e4db0 msgr2=0x7fd6dc1f4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6daa00640 1 --2- 172.21.15.32:0/1463634813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6dc1e4db0 0x7fd6dc1f4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6daa00640 1 -- 172.21.15.32:0/1463634813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6dc14dcc0 con 0x7fd6c0004d50 2024-04-17T09:58:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6d3e00640 1 --2- 172.21.15.32:0/1463634813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6dc1e4db0 0x7fd6dc1f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:58:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6daa00640 1 --2- 172.21.15.32:0/1463634813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6c0004d50 0x7fd6dc15cbf0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc004f00 tx=0x7fd6cc007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6d2a00640 1 -- 172.21.15.32:0/1463634813 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc00f750 con 0x7fd6c0004d50 2024-04-17T09:58:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6d2a00640 1 -- 172.21.15.32:0/1463634813 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6cc018070 con 0x7fd6c0004d50 2024-04-17T09:58:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6e1c00640 1 -- 172.21.15.32:0/1463634813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6dc15d340 con 0x7fd6c0004d50 2024-04-17T09:58:12.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6d2a00640 1 -- 172.21.15.32:0/1463634813 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc00fb50 con 0x7fd6c0004d50 2024-04-17T09:58:12.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.579+0000 7fd6e1c00640 1 -- 172.21.15.32:0/1463634813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6dc15d900 con 0x7fd6c0004d50 2024-04-17T09:58:12.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.583+0000 7fd6d2a00640 1 -- 172.21.15.32:0/1463634813 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd6cc00fcf0 con 0x7fd6c0004d50 2024-04-17T09:58:12.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.583+0000 7fd6d2a00640 1 --2- 172.21.15.32:0/1463634813 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6b80683b0 0x7fd6b806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.583+0000 7fd6d2a00640 1 -- 172.21.15.32:0/1463634813 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fd6cc086a40 con 0x7fd6c0004d50 2024-04-17T09:58:12.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.583+0000 7fd6d3e00640 1 --2- 172.21.15.32:0/1463634813 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6b80683b0 0x7fd6b806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.583+0000 7fd6e5160000 1 Processor -- start 2024-04-17T09:58:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.583+0000 7fd6e5160000 1 -- start start 2024-04-17T09:58:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.583+0000 7fd6d3e00640 1 --2- 172.21.15.32:0/1463634813 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6b80683b0 0x7fd6b806a870 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc0afa80 tx=0x7fd6c4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6e5160000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5a0260 0x558d3d5a0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6e5160000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558d3d5a0b80 0x558d3d553460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6e5160000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558d3d5539a0 0x558d3d553d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6e5160000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558d3d3a54f0 con 0x558d3d5a0260 2024-04-17T09:58:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6e5160000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558d3d35e6c0 con 0x558d3d5539a0 2024-04-17T09:58:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6e5160000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x558d3d3a14b0 con 0x558d3d5a0b80 2024-04-17T09:58:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6daa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5a0260 0x558d3d5a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6daa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5a0260 0x558d3d5a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55830/0 (socket says 172.21.15.32:55830) 2024-04-17T09:58:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6daa00640 1 -- 172.21.15.32:0/3133536118 learned_addr learned my addr 172.21.15.32:0/3133536118 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6daa00640 1 -- 172.21.15.32:0/3133536118 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558d3d5a0b80 msgr2=0x558d3d553460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6db400640 1 --2- 172.21.15.32:0/3133536118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558d3d5539a0 0x558d3d553d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6d3e00640 1 --2- 172.21.15.32:0/3133536118 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558d3d5a0b80 0x558d3d553460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6daa00640 1 --2- 172.21.15.32:0/3133536118 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558d3d5a0b80 0x558d3d553460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6daa00640 1 -- 172.21.15.32:0/3133536118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558d3d5539a0 msgr2=0x558d3d553d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6daa00640 1 --2- 172.21.15.32:0/3133536118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558d3d5539a0 0x558d3d553d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6daa00640 1 -- 172.21.15.32:0/3133536118 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d3d3ea8e0 con 0x558d3d5a0260 2024-04-17T09:58:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6d3e00640 1 --2- 172.21.15.32:0/3133536118 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558d3d5a0b80 0x558d3d553460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:58:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6db400640 1 --2- 172.21.15.32:0/3133536118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558d3d5539a0 0x558d3d553d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:58:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6daa00640 1 --2- 172.21.15.32:0/3133536118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5a0260 0x558d3d5a0640 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0564d0 tx=0x7fd6cc001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b74ab3e636707b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6cb400640 1 -- 172.21.15.32:0/3133536118 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc05f3b0 con 0x558d3d5a0260 2024-04-17T09:58:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6cb400640 1 -- 172.21.15.32:0/3133536118 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6cc054ad0 con 0x558d3d5a0260 2024-04-17T09:58:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6cb400640 1 -- 172.21.15.32:0/3133536118 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc055c20 con 0x558d3d5a0260 2024-04-17T09:58:12.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6e5160000 1 -- 172.21.15.32:0/3133536118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5a0260 msgr2=0x558d3d5a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6e5160000 1 --2- 172.21.15.32:0/3133536118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5a0260 0x558d3d5a0640 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0564d0 tx=0x7fd6cc001980 comp rx=0 tx=0).stop 2024-04-17T09:58:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6e5160000 1 -- 172.21.15.32:0/3133536118 shutdown_connections 2024-04-17T09:58:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6e5160000 1 --2- 172.21.15.32:0/3133536118 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558d3d5a0b80 0x558d3d553460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6e5160000 1 --2- 172.21.15.32:0/3133536118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558d3d5539a0 0x558d3d553d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6e5160000 1 --2- 172.21.15.32:0/3133536118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5a0260 0x558d3d5a0640 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6e5160000 1 -- 172.21.15.32:0/3133536118 >> 172.21.15.32:0/3133536118 conn(0x558d3d581000 msgr2=0x558d3d5833f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6e5160000 1 -- 172.21.15.32:0/3133536118 shutdown_connections 2024-04-17T09:58:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6e5160000 1 -- 172.21.15.32:0/3133536118 wait complete. 2024-04-17T09:58:12.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6e5160000 1 -- start start 2024-04-17T09:58:12.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.587+0000 7fd6e5160000 1 Processor -- start 2024-04-17T09:58:12.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5539a0 0x558d3d558ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558d3d5a0260 0x558d3d559530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558d3d5a0b80 0x558d3d559a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558d3d4267b0 con 0x558d3d5539a0 2024-04-17T09:58:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558d3d460d00 con 0x558d3d5a0260 2024-04-17T09:58:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x558d3d36a2c0 con 0x558d3d5a0b80 2024-04-17T09:58:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 10 client.0 _ll_register_callbacks cb 0x558d3d37a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:58:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6daa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5539a0 0x558d3d558ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6daa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5539a0 0x558d3d558ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55838/0 (socket says 172.21.15.32:55838) 2024-04-17T09:58:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6daa00640 1 -- 172.21.15.32:0/3837648109 learned_addr learned my addr 172.21.15.32:0/3837648109 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6d3e00640 1 --2- 172.21.15.32:0/3837648109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558d3d5a0260 0x558d3d559530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6db400640 1 --2- 172.21.15.32:0/3837648109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558d3d5a0b80 0x558d3d559a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6daa00640 1 -- 172.21.15.32:0/3837648109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558d3d5a0b80 msgr2=0x558d3d559a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6daa00640 1 --2- 172.21.15.32:0/3837648109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558d3d5a0b80 0x558d3d559a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6daa00640 1 -- 172.21.15.32:0/3837648109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558d3d5a0260 msgr2=0x558d3d559530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6daa00640 1 --2- 172.21.15.32:0/3837648109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558d3d5a0260 0x558d3d559530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6daa00640 1 -- 172.21.15.32:0/3837648109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d3d37d490 con 0x558d3d5539a0 2024-04-17T09:58:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6daa00640 1 --2- 172.21.15.32:0/3837648109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5539a0 0x558d3d558ff0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc056480 tx=0x7fd6cc007890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7541938ef37894db server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:58:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6bfe00640 1 -- 172.21.15.32:0/3837648109 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc073060 con 0x558d3d5539a0 2024-04-17T09:58:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6bfe00640 1 -- 172.21.15.32:0/3837648109 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fd6cc054870 con 0x558d3d5539a0 2024-04-17T09:58:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 1 -- 172.21.15.32:0/3837648109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6cc054ad0 con 0x558d3d5539a0 2024-04-17T09:58:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6bfe00640 1 -- 172.21.15.32:0/3837648109 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fd6cc07a050 con 0x558d3d5539a0 2024-04-17T09:58:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 10 client.7206 fetch_fsmap learned FSMap version 49 2024-04-17T09:58:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 1 -- 172.21.15.32:0/3837648109 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x558d3d370eb0 con 0x558d3d5539a0 2024-04-17T09:58:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6bfe00640 1 -- 172.21.15.32:0/3837648109 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6cc054ad0 con 0x558d3d5539a0 2024-04-17T09:58:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6bfe00640 1 -- 172.21.15.32:0/3837648109 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7fd6cc052e40 con 0x558d3d5539a0 2024-04-17T09:58:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 10 client.7206 fetch_fsmap finished waiting for FSMap version 49 2024-04-17T09:58:12.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 10 client.7206 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-17T09:58:12.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 20 client.7206 populate_metadata read hostname 'smithi032' 2024-04-17T09:58:12.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 1 --2- 172.21.15.32:0/3837648109 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x558d3d5af500 0x558d3d5b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 4 client.7206 mds_command: new command op to 7011 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:58:12.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 1 -- 172.21.15.32:0/3837648109 --> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d3d33edd0 con 0x558d3d5af500 2024-04-17T09:58:12.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6d3e00640 1 --2- 172.21.15.32:0/3837648109 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x558d3d5af500 0x558d3d5b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6d3e00640 1 --2- 172.21.15.32:0/3837648109 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x558d3d5af500 0x558d3d5b18f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aa17a81945f73f7 server_cookie=c2593d11580eb1c1 in_seq=0 out_seq=0 2024-04-17T09:58:12.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6bfe00640 10 client.7206 ms_handle_connect on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:12.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6bfe00640 1 -- 172.21.15.32:0/3837648109 <== mds.0 v2:172.21.15.32:6834/2416784965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558d3d33edd0 con 0x558d3d5af500 2024-04-17T09:58:12.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6bfe00640 10 client.7206 handle_command_reply: tid=0 2024-04-17T09:58:12.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 1 client.7206 shutdown 2024-04-17T09:58:12.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 1 -- 172.21.15.32:0/3837648109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5539a0 msgr2=0x558d3d558ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:12.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 1 --2- 172.21.15.32:0/3837648109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5539a0 0x558d3d558ff0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc056480 tx=0x7fd6cc007890 comp rx=0 tx=0).stop 2024-04-17T09:58:12.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 1 -- 172.21.15.32:0/3837648109 shutdown_connections 2024-04-17T09:58:12.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 1 --2- 172.21.15.32:0/3837648109 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x558d3d5af500 0x558d3d5b18f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 1 --2- 172.21.15.32:0/3837648109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558d3d5a0b80 0x558d3d559a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 1 --2- 172.21.15.32:0/3837648109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558d3d5a0260 0x558d3d559530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 1 --2- 172.21.15.32:0/3837648109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5539a0 0x558d3d558ff0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6bfe00640 0 client.7206 ms_handle_reset on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:12.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 1 -- 172.21.15.32:0/3837648109 >> 172.21.15.32:0/3837648109 conn(0x558d3d581000 msgr2=0x558d3d5833f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:12.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 1 -- 172.21.15.32:0/3837648109 shutdown_connections 2024-04-17T09:58:12.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 1 -- 172.21.15.32:0/3837648109 wait complete. 2024-04-17T09:58:12.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.591+0000 7fd6e5160000 20 client.7206 trim_cache size 0 max 16384 2024-04-17T09:58:12.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.611+0000 7fd6e5160000 1 Processor -- start 2024-04-17T09:58:12.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.611+0000 7fd6e5160000 1 -- start start 2024-04-17T09:58:12.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.611+0000 7fd6e5160000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5539a0 0x558d3d60aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.611+0000 7fd6e5160000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558d3d5a0260 0x558d3d60afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.611+0000 7fd6e5160000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558d3d5a0b80 0x558d3d60b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.611+0000 7fd6e5160000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc2003b0 con 0x558d3d5539a0 2024-04-17T09:58:12.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.611+0000 7fd6e5160000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc0ae910 con 0x558d3d5a0260 2024-04-17T09:58:12.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.611+0000 7fd6e5160000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd6dc1ffd60 con 0x558d3d5a0b80 2024-04-17T09:58:12.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.611+0000 7fd6daa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5539a0 0x558d3d60aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.611+0000 7fd6daa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5539a0 0x558d3d60aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55842/0 (socket says 172.21.15.32:55842) 2024-04-17T09:58:12.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.611+0000 7fd6daa00640 1 -- 172.21.15.32:0/3924016169 learned_addr learned my addr 172.21.15.32:0/3924016169 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:12.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.611+0000 7fd6db400640 1 --2- 172.21.15.32:0/3924016169 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558d3d5a0b80 0x558d3d60b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.611+0000 7fd6d3e00640 1 --2- 172.21.15.32:0/3924016169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558d3d5a0260 0x558d3d60afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.611+0000 7fd6daa00640 1 -- 172.21.15.32:0/3924016169 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558d3d5a0b80 msgr2=0x558d3d60b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:12.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.611+0000 7fd6daa00640 1 --2- 172.21.15.32:0/3924016169 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558d3d5a0b80 0x558d3d60b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.611+0000 7fd6daa00640 1 -- 172.21.15.32:0/3924016169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558d3d5a0260 msgr2=0x558d3d60afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:12.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.611+0000 7fd6daa00640 1 --2- 172.21.15.32:0/3924016169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558d3d5a0260 0x558d3d60afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.611+0000 7fd6daa00640 1 -- 172.21.15.32:0/3924016169 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d3d35fa20 con 0x558d3d5539a0 2024-04-17T09:58:12.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.611+0000 7fd6d3e00640 1 --2- 172.21.15.32:0/3924016169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558d3d5a0260 0x558d3d60afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:58:12.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.611+0000 7fd6daa00640 1 --2- 172.21.15.32:0/3924016169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5539a0 0x558d3d60aa80 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0564d0 tx=0x7fd6cc083000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bfad06e8f2a8272 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:12.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.611+0000 7fd6caa00640 1 -- 172.21.15.32:0/3924016169 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc054ad0 con 0x558d3d5539a0 2024-04-17T09:58:12.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.611+0000 7fd6caa00640 1 -- 172.21.15.32:0/3924016169 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6cc07e400 con 0x558d3d5539a0 2024-04-17T09:58:12.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.611+0000 7fd6caa00640 1 -- 172.21.15.32:0/3924016169 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc055cc0 con 0x558d3d5539a0 2024-04-17T09:58:12.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6e5160000 1 -- 172.21.15.32:0/3924016169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5539a0 msgr2=0x558d3d60aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:12.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6e5160000 1 --2- 172.21.15.32:0/3924016169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5539a0 0x558d3d60aa80 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0564d0 tx=0x7fd6cc083000 comp rx=0 tx=0).stop 2024-04-17T09:58:12.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6e5160000 1 -- 172.21.15.32:0/3924016169 shutdown_connections 2024-04-17T09:58:12.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6e5160000 1 --2- 172.21.15.32:0/3924016169 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558d3d5a0b80 0x558d3d60b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6e5160000 1 --2- 172.21.15.32:0/3924016169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558d3d5a0260 0x558d3d60afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6e5160000 1 --2- 172.21.15.32:0/3924016169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5539a0 0x558d3d60aa80 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6e5160000 1 -- 172.21.15.32:0/3924016169 >> 172.21.15.32:0/3924016169 conn(0x558d3d581000 msgr2=0x558d3d551bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:12.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6e5160000 1 -- 172.21.15.32:0/3924016169 shutdown_connections 2024-04-17T09:58:12.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6e5160000 1 -- 172.21.15.32:0/3924016169 wait complete. 2024-04-17T09:58:12.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6e5160000 1 -- start start 2024-04-17T09:58:12.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6e5160000 1 Processor -- start 2024-04-17T09:58:12.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6e5160000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558d3d5539a0 0x558d3d5b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6e5160000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558d3d5a0260 0x558d3d5cb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6e5160000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5a0b80 0x558d3d5cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6e5160000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x558d3d349050 con 0x558d3d5a0b80 2024-04-17T09:58:12.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6e5160000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558d3d367570 con 0x558d3d5539a0 2024-04-17T09:58:12.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6e5160000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x558d3d3a4800 con 0x558d3d5a0260 2024-04-17T09:58:12.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6e5160000 10 client.0 _ll_register_callbacks cb 0x7fd6cc056200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:58:12.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6db400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5a0b80 0x558d3d5cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6db400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5a0b80 0x558d3d5cb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55854/0 (socket says 172.21.15.32:55854) 2024-04-17T09:58:12.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6db400640 1 -- 172.21.15.32:0/1306746151 learned_addr learned my addr 172.21.15.32:0/1306746151 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:12.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6daa00640 1 --2- 172.21.15.32:0/1306746151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558d3d5539a0 0x558d3d5b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6d3e00640 1 --2- 172.21.15.32:0/1306746151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558d3d5a0260 0x558d3d5cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6db400640 1 -- 172.21.15.32:0/1306746151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558d3d5a0260 msgr2=0x558d3d5cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:12.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6db400640 1 --2- 172.21.15.32:0/1306746151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558d3d5a0260 0x558d3d5cb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6db400640 1 -- 172.21.15.32:0/1306746151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558d3d5539a0 msgr2=0x558d3d5b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:12.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6db400640 1 --2- 172.21.15.32:0/1306746151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558d3d5539a0 0x558d3d5b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6db400640 1 -- 172.21.15.32:0/1306746151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d3d426c40 con 0x558d3d5a0b80 2024-04-17T09:58:12.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6db400640 1 --2- 172.21.15.32:0/1306746151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5a0b80 0x558d3d5cb9c0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4005040 tx=0x7fd6d401be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2edfa2a5fd08274 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:12.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:58:12.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6bfe00640 1 -- 172.21.15.32:0/1306746151 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40b5040 con 0x558d3d5a0b80 2024-04-17T09:58:12.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6bfe00640 1 -- 172.21.15.32:0/1306746151 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fd6d4006b60 con 0x558d3d5a0b80 2024-04-17T09:58:12.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6e5160000 1 -- 172.21.15.32:0/1306746151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6cc07e400 con 0x558d3d5a0b80 2024-04-17T09:58:12.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6bfe00640 1 -- 172.21.15.32:0/1306746151 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fd6d40a8e60 con 0x558d3d5a0b80 2024-04-17T09:58:12.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6e5160000 10 client.7212 fetch_fsmap learned FSMap version 49 2024-04-17T09:58:12.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.615+0000 7fd6e5160000 1 -- 172.21.15.32:0/1306746151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x558d3d34a600 con 0x558d3d5a0b80 2024-04-17T09:58:12.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6bfe00640 1 -- 172.21.15.32:0/1306746151 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6cc07e400 con 0x558d3d5a0b80 2024-04-17T09:58:12.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6bfe00640 1 -- 172.21.15.32:0/1306746151 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7fd6d4003980 con 0x558d3d5a0b80 2024-04-17T09:58:12.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e5160000 10 client.7212 fetch_fsmap finished waiting for FSMap version 49 2024-04-17T09:58:12.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e5160000 10 client.7212 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-17T09:58:12.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e5160000 20 client.7212 populate_metadata read hostname 'smithi032' 2024-04-17T09:58:12.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e5160000 1 --2- 172.21.15.32:0/1306746151 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x558d3d6148e0 0x558d3d616cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e5160000 4 client.7212 mds_command: new command op to 7011 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T09:58:12.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e5160000 1 -- 172.21.15.32:0/1306746151 --> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558d3d580c40 con 0x558d3d6148e0 2024-04-17T09:58:12.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6daa00640 1 --2- 172.21.15.32:0/1306746151 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x558d3d6148e0 0x558d3d616cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6daa00640 1 --2- 172.21.15.32:0/1306746151 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x558d3d6148e0 0x558d3d616cd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bdb83252191e0b server_cookie=6cbca46776aa3433 in_seq=0 out_seq=0 2024-04-17T09:58:12.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6bfe00640 10 client.7212 ms_handle_connect on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:12.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6bfe00640 1 -- 172.21.15.32:0/1306746151 <== mds.0 v2:172.21.15.32:6834/2416784965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558d3d580c40 con 0x558d3d6148e0 2024-04-17T09:58:12.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6bfe00640 10 client.7212 handle_command_reply: tid=0 2024-04-17T09:58:12.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e5160000 1 client.7212 shutdown 2024-04-17T09:58:12.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e5160000 1 -- 172.21.15.32:0/1306746151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5a0b80 msgr2=0x558d3d5cb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:12.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e5160000 1 --2- 172.21.15.32:0/1306746151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5a0b80 0x558d3d5cb9c0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4005040 tx=0x7fd6d401be10 comp rx=0 tx=0).stop 2024-04-17T09:58:12.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e5160000 1 -- 172.21.15.32:0/1306746151 shutdown_connections 2024-04-17T09:58:12.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e5160000 1 --2- 172.21.15.32:0/1306746151 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x558d3d6148e0 0x558d3d616cd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e5160000 1 --2- 172.21.15.32:0/1306746151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558d3d5a0260 0x558d3d5cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e5160000 1 --2- 172.21.15.32:0/1306746151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558d3d5a0b80 0x558d3d5cb9c0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e5160000 1 --2- 172.21.15.32:0/1306746151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558d3d5539a0 0x558d3d5b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6bfe00640 0 client.7212 ms_handle_reset on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:12.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e5160000 1 -- 172.21.15.32:0/1306746151 >> 172.21.15.32:0/1306746151 conn(0x558d3d581000 msgr2=0x558d3d552450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:12.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e5160000 1 -- 172.21.15.32:0/1306746151 shutdown_connections 2024-04-17T09:58:12.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e5160000 1 -- 172.21.15.32:0/1306746151 wait complete. 2024-04-17T09:58:12.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e5160000 20 client.7212 trim_cache size 0 max 16384 2024-04-17T09:58:12.625 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:58:12.625 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-17T09:58:12.625 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-17T09:58:12.625 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:58:12.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e1c00640 1 -- 172.21.15.32:0/1463634813 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6b80683b0 msgr2=0x7fd6b806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:12.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e1c00640 1 --2- 172.21.15.32:0/1463634813 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6b80683b0 0x7fd6b806a870 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc0afa80 tx=0x7fd6c4008040 comp rx=0 tx=0).stop 2024-04-17T09:58:12.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e1c00640 1 -- 172.21.15.32:0/1463634813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6c0004d50 msgr2=0x7fd6dc15cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:12.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e1c00640 1 --2- 172.21.15.32:0/1463634813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6c0004d50 0x7fd6dc15cbf0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc004f00 tx=0x7fd6cc007430 comp rx=0 tx=0).stop 2024-04-17T09:58:12.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e1c00640 1 -- 172.21.15.32:0/1463634813 shutdown_connections 2024-04-17T09:58:12.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e1c00640 1 --2- 172.21.15.32:0/1463634813 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6b80683b0 0x7fd6b806a870 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e1c00640 1 --2- 172.21.15.32:0/1463634813 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6dc14d300 0x7fd6dc14d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e1c00640 1 --2- 172.21.15.32:0/1463634813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6dc1e4db0 0x7fd6dc1f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e1c00640 1 --2- 172.21.15.32:0/1463634813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6c0004d50 0x7fd6dc15cbf0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e1c00640 1 -- 172.21.15.32:0/1463634813 >> 172.21.15.32:0/1463634813 conn(0x7fd6dc0b2120 msgr2=0x7fd6dc0b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:12.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e1c00640 1 -- 172.21.15.32:0/1463634813 shutdown_connections 2024-04-17T09:58:12.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.619+0000 7fd6e1c00640 1 -- 172.21.15.32:0/1463634813 wait complete. 2024-04-17T09:58:12.640 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T09:58:12.640 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:58:12.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.779+0000 7f348ae00640 1 Processor -- start 2024-04-17T09:58:12.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.779+0000 7f348ae00640 1 -- start start 2024-04-17T09:58:12.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.779+0000 7f348ae00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3484144e90 0x7f34840b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.779+0000 7f348ae00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f34840b4de0 0x7f348415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.779+0000 7f348ae00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f348415bfe0 0x7f348406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.779+0000 7f348ae00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f348415c430 con 0x7f3484144e90 2024-04-17T09:58:12.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.779+0000 7f348ae00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f3484145270 con 0x7f34840b4de0 2024-04-17T09:58:12.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.779+0000 7f348ae00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3484072bb0 con 0x7f348415bfe0 2024-04-17T09:58:12.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.779+0000 7f3483e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3484144e90 0x7f34840b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.779+0000 7f3483e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3484144e90 0x7f34840b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55862/0 (socket says 172.21.15.32:55862) 2024-04-17T09:58:12.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.779+0000 7f3483e00640 1 -- 172.21.15.32:0/2810483708 learned_addr learned my addr 172.21.15.32:0/2810483708 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:12.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.779+0000 7f347be00640 1 --2- 172.21.15.32:0/2810483708 >> v2:172.21.15.144:3300/0 conn(0x7f34840b4de0 0x7f348415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.779+0000 7f3488e00640 1 --2- 172.21.15.32:0/2810483708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f348415bfe0 0x7f348406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.779+0000 7f3483e00640 1 -- 172.21.15.32:0/2810483708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f348415bfe0 msgr2=0x7f348406ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:12.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.779+0000 7f3483e00640 1 --2- 172.21.15.32:0/2810483708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f348415bfe0 0x7f348406ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.779+0000 7f3483e00640 1 -- 172.21.15.32:0/2810483708 >> v2:172.21.15.144:3300/0 conn(0x7f34840b4de0 msgr2=0x7f348415baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:12.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.779+0000 7f3483e00640 1 --2- 172.21.15.32:0/2810483708 >> v2:172.21.15.144:3300/0 conn(0x7f34840b4de0 0x7f348415baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.779+0000 7f3483e00640 1 -- 172.21.15.32:0/2810483708 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34840b3dc0 con 0x7f3484144e90 2024-04-17T09:58:12.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.783+0000 7f3483e00640 1 --2- 172.21.15.32:0/2810483708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3484144e90 0x7f34840b48a0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f348406fb40 tx=0x7f347401ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6047992e79a333 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.783+0000 7f3483400640 1 -- 172.21.15.32:0/2810483708 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34740b4070 con 0x7f3484144e90 2024-04-17T09:58:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.783+0000 7f3483400640 1 -- 172.21.15.32:0/2810483708 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34740a7b60 con 0x7f3484144e90 2024-04-17T09:58:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.783+0000 7f3483400640 1 -- 172.21.15.32:0/2810483708 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34740af440 con 0x7f3484144e90 2024-04-17T09:58:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.783+0000 7f348ae00640 1 -- 172.21.15.32:0/2810483708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3484144e90 msgr2=0x7f34840b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.783+0000 7f348ae00640 1 --2- 172.21.15.32:0/2810483708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3484144e90 0x7f34840b48a0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f348406fb40 tx=0x7f347401ebc0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.783+0000 7f348ae00640 1 -- 172.21.15.32:0/2810483708 shutdown_connections 2024-04-17T09:58:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.783+0000 7f348ae00640 1 --2- 172.21.15.32:0/2810483708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f348415bfe0 0x7f348406ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.783+0000 7f348ae00640 1 --2- 172.21.15.32:0/2810483708 >> v2:172.21.15.144:3300/0 conn(0x7f34840b4de0 0x7f348415baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.783+0000 7f348ae00640 1 --2- 172.21.15.32:0/2810483708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3484144e90 0x7f34840b48a0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.783+0000 7f348ae00640 1 -- 172.21.15.32:0/2810483708 >> 172.21.15.32:0/2810483708 conn(0x7f348406ba30 msgr2=0x7f348406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:12.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.783+0000 7f348ae00640 1 -- 172.21.15.32:0/2810483708 shutdown_connections 2024-04-17T09:58:12.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.783+0000 7f348ae00640 1 -- 172.21.15.32:0/2810483708 wait complete. 2024-04-17T09:58:12.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f348ae00640 1 Processor -- start 2024-04-17T09:58:12.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f348ae00640 1 -- start start 2024-04-17T09:58:12.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f348ae00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34840b4de0 0x7f3484141380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f348ae00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3484144e90 0x7f34841438b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f348ae00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f348415bfe0 0x7f3484143df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f348ae00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f348406de50 con 0x7f3484144e90 2024-04-17T09:58:12.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f348ae00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3484072880 con 0x7f34840b4de0 2024-04-17T09:58:12.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f348ae00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f348406dfd0 con 0x7f348415bfe0 2024-04-17T09:58:12.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f347be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3484144e90 0x7f34841438b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f347be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3484144e90 0x7f34841438b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55876/0 (socket says 172.21.15.32:55876) 2024-04-17T09:58:12.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f347be00640 1 -- 172.21.15.32:0/3569953426 learned_addr learned my addr 172.21.15.32:0/3569953426 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:12.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f3483e00640 1 --2- 172.21.15.32:0/3569953426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34840b4de0 0x7f3484141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f3488e00640 1 --2- 172.21.15.32:0/3569953426 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f348415bfe0 0x7f3484143df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f347be00640 1 -- 172.21.15.32:0/3569953426 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f348415bfe0 msgr2=0x7f3484143df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:12.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f347be00640 1 --2- 172.21.15.32:0/3569953426 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f348415bfe0 0x7f3484143df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f347be00640 1 -- 172.21.15.32:0/3569953426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34840b4de0 msgr2=0x7f3484141380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:12.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f347be00640 1 --2- 172.21.15.32:0/3569953426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34840b4de0 0x7f3484141380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:12.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f347be00640 1 -- 172.21.15.32:0/3569953426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34841fb850 con 0x7f3484144e90 2024-04-17T09:58:12.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f347be00640 1 --2- 172.21.15.32:0/3569953426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3484144e90 0x7f34841438b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f346c00d800 tx=0x7f346c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:12.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f3480c00640 1 -- 172.21.15.32:0/3569953426 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c00b6c0 con 0x7f3484144e90 2024-04-17T09:58:12.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f3480c00640 1 -- 172.21.15.32:0/3569953426 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f346c019070 con 0x7f3484144e90 2024-04-17T09:58:12.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f348ae00640 1 -- 172.21.15.32:0/3569953426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34840c9390 con 0x7f3484144e90 2024-04-17T09:58:12.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f3480c00640 1 -- 172.21.15.32:0/3569953426 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c00ba60 con 0x7f3484144e90 2024-04-17T09:58:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.787+0000 7f348ae00640 1 -- 172.21.15.32:0/3569953426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34840c9950 con 0x7f3484144e90 2024-04-17T09:58:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.791+0000 7f348ae00640 1 -- 172.21.15.32:0/3569953426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3484068a90 con 0x7f3484144e90 2024-04-17T09:58:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.791+0000 7f3480c00640 1 -- 172.21.15.32:0/3569953426 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f346c00bc00 con 0x7f3484144e90 2024-04-17T09:58:12.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.791+0000 7f3480c00640 1 --2- 172.21.15.32:0/3569953426 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f34600683d0 0x7f346006a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:12.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.791+0000 7f3480c00640 1 -- 172.21.15.32:0/3569953426 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f346c086df0 con 0x7f3484144e90 2024-04-17T09:58:12.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.791+0000 7f3483e00640 1 --2- 172.21.15.32:0/3569953426 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f34600683d0 0x7f346006a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:12.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.791+0000 7f3483e00640 1 --2- 172.21.15.32:0/3569953426 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f34600683d0 0x7f346006a890 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f34740b1000 tx=0x7f3474008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:12.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.795+0000 7f3480c00640 1 -- 172.21.15.32:0/3569953426 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f346c056cb0 con 0x7f3484144e90 2024-04-17T09:58:13.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.999+0000 7f348ae00640 1 -- 172.21.15.32:0/3569953426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3484072df0 con 0x7f3484144e90 2024-04-17T09:58:13.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.999+0000 7f3480c00640 1 -- 172.21.15.32:0/3569953426 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5205 (secure 0 0 0) 0x7f346c05afb0 con 0x7f3484144e90 2024-04-17T09:58:13.000 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:58:13.000 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2424282223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2424282223},{"type":"v1","addr":"172.21.15.144:6834","nonce":2424282223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3537056692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3537056692},{"type":"v1","addr":"172.21.15.144:6835","nonce":3537056692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5891,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/725333724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":725333724},{"type":"v1","addr":"172.21.15.32:6837","nonce":725333724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:57:11.558183+0000","modified":"2024-04-17T09:57:41.764099+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":6,"addr":"172.21.15.32:6836/2416784965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2416784965},{"type":"v1","addr":"172.21.15.32:6836","nonce":2416784965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7011,"qdb_cluster":[7011]},"id":5}]} 2024-04-17T09:58:13.000 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 49 2024-04-17T09:58:13.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.999+0000 7f348ae00640 1 -- 172.21.15.32:0/3569953426 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f34600683d0 msgr2=0x7f346006a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.999+0000 7f348ae00640 1 --2- 172.21.15.32:0/3569953426 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f34600683d0 0x7f346006a890 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f34740b1000 tx=0x7f3474008040 comp rx=0 tx=0).stop 2024-04-17T09:58:13.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.999+0000 7f348ae00640 1 -- 172.21.15.32:0/3569953426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3484144e90 msgr2=0x7f34841438b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.999+0000 7f348ae00640 1 --2- 172.21.15.32:0/3569953426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3484144e90 0x7f34841438b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f346c00d800 tx=0x7f346c00dcc0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.999+0000 7f348ae00640 1 -- 172.21.15.32:0/3569953426 shutdown_connections 2024-04-17T09:58:13.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.999+0000 7f348ae00640 1 --2- 172.21.15.32:0/3569953426 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f34600683d0 0x7f346006a890 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.999+0000 7f348ae00640 1 --2- 172.21.15.32:0/3569953426 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f348415bfe0 0x7f3484143df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.999+0000 7f348ae00640 1 --2- 172.21.15.32:0/3569953426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3484144e90 0x7f34841438b0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.999+0000 7f348ae00640 1 --2- 172.21.15.32:0/3569953426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34840b4de0 0x7f3484141380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.999+0000 7f348ae00640 1 -- 172.21.15.32:0/3569953426 >> 172.21.15.32:0/3569953426 conn(0x7f348406ba30 msgr2=0x7f34840ae5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:13.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.999+0000 7f348ae00640 1 -- 172.21.15.32:0/3569953426 shutdown_connections 2024-04-17T09:58:13.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:12.999+0000 7f348ae00640 1 -- 172.21.15.32:0/3569953426 wait complete. 2024-04-17T09:58:13.021 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-17T09:58:13.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.163+0000 7f3c81200640 1 Processor -- start 2024-04-17T09:58:13.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.163+0000 7f3c81200640 1 -- start start 2024-04-17T09:58:13.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.163+0000 7f3c81200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3c7c15e7f0 0x7f3c7c0d1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.163+0000 7f3c81200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c7c06d080 0x7f3c7c06d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.163+0000 7f3c81200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c7c06df20 con 0x7f3c7c06da90 2024-04-17T09:58:13.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.163+0000 7f3c81200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3c7c0ce920 con 0x7f3c7c15e7f0 2024-04-17T09:58:13.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.163+0000 7f3c81200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3c7c0b52b0 con 0x7f3c7c06d080 2024-04-17T09:58:13.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.163+0000 7f3c7b400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3c7c15e7f0 0x7f3c7c0d1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.163+0000 7f3c7b400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3c7c15e7f0 0x7f3c7c0d1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55884/0 (socket says 172.21.15.32:55884) 2024-04-17T09:58:13.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.163+0000 7f3c7b400640 1 -- 172.21.15.32:0/817012356 learned_addr learned my addr 172.21.15.32:0/817012356 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:13.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.163+0000 7f3c7b400640 1 -- 172.21.15.32:0/817012356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c7c06d080 msgr2=0x7f3c7c06d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:13.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.163+0000 7f3c72a00640 1 --2- 172.21.15.32:0/817012356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c7c06d080 0x7f3c7c06d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.163+0000 7f3c7b400640 1 --2- 172.21.15.32:0/817012356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c7c06d080 0x7f3c7c06d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.163+0000 7f3c7b400640 1 -- 172.21.15.32:0/817012356 >> v1:172.21.15.144:6789/0 conn(0x7f3c7c06da90 legacy=0x7f3c7c09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:13.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.163+0000 7f3c7b400640 1 -- 172.21.15.32:0/817012356 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7c0d1aa0 con 0x7f3c7c15e7f0 2024-04-17T09:58:13.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.167+0000 7f3c7b400640 1 --2- 172.21.15.32:0/817012356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c7c15e7f0 0x7f3c7c0d1170 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009870 tx=0x7f3c6c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ace51873372009 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:13.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.167+0000 7f3c7aa00640 1 -- 172.21.15.32:0/817012356 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0b1070 con 0x7f3c7c15e7f0 2024-04-17T09:58:13.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.167+0000 7f3c7aa00640 1 -- 172.21.15.32:0/817012356 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c6c0ac630 con 0x7f3c7c15e7f0 2024-04-17T09:58:13.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.167+0000 7f3c7aa00640 1 -- 172.21.15.32:0/817012356 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0a8050 con 0x7f3c7c15e7f0 2024-04-17T09:58:13.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.167+0000 7f3c81200640 1 -- 172.21.15.32:0/817012356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c7c15e7f0 msgr2=0x7f3c7c0d1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.167+0000 7f3c81200640 1 --2- 172.21.15.32:0/817012356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c7c15e7f0 0x7f3c7c0d1170 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009870 tx=0x7f3c6c01c4f0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.167+0000 7f3c81200640 1 -- 172.21.15.32:0/817012356 shutdown_connections 2024-04-17T09:58:13.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.167+0000 7f3c81200640 1 --2- 172.21.15.32:0/817012356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c7c06d080 0x7f3c7c06d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.167+0000 7f3c81200640 1 --2- 172.21.15.32:0/817012356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c7c15e7f0 0x7f3c7c0d1170 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.167+0000 7f3c81200640 1 -- 172.21.15.32:0/817012356 >> 172.21.15.32:0/817012356 conn(0x7f3c7c0b3f50 msgr2=0x7f3c7c0b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:13.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.167+0000 7f3c81200640 1 -- 172.21.15.32:0/817012356 shutdown_connections 2024-04-17T09:58:13.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.167+0000 7f3c81200640 1 -- 172.21.15.32:0/817012356 wait complete. 2024-04-17T09:58:13.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c81200640 1 Processor -- start 2024-04-17T09:58:13.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c81200640 1 -- start start 2024-04-17T09:58:13.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c81200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c7c06d080 0x7f3c7c152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c81200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c7c06da90 0x7f3c7c1608c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c81200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c7c15e7f0 0x7f3c7c070030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c81200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c0d16b0 con 0x7f3c7c15e7f0 2024-04-17T09:58:13.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c81200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c0cd3f0 con 0x7f3c7c06d080 2024-04-17T09:58:13.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c81200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3c7c0d1830 con 0x7f3c7c06da90 2024-04-17T09:58:13.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c7be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c7c15e7f0 0x7f3c7c070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c7be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c7c15e7f0 0x7f3c7c070030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55900/0 (socket says 172.21.15.32:55900) 2024-04-17T09:58:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c7be00640 1 -- 172.21.15.32:0/2488124719 learned_addr learned my addr 172.21.15.32:0/2488124719 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c7b400640 1 --2- 172.21.15.32:0/2488124719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c7c06d080 0x7f3c7c152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c72a00640 1 --2- 172.21.15.32:0/2488124719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c7c06da90 0x7f3c7c1608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c7be00640 1 -- 172.21.15.32:0/2488124719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c7c06da90 msgr2=0x7f3c7c1608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c7be00640 1 --2- 172.21.15.32:0/2488124719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c7c06da90 0x7f3c7c1608c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c7be00640 1 -- 172.21.15.32:0/2488124719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c7c06d080 msgr2=0x7f3c7c152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c7be00640 1 --2- 172.21.15.32:0/2488124719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c7c06d080 0x7f3c7c152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c7be00640 1 -- 172.21.15.32:0/2488124719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7c0c9f00 con 0x7f3c7c15e7f0 2024-04-17T09:58:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c7be00640 1 --2- 172.21.15.32:0/2488124719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c7c15e7f0 0x7f3c7c070030 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400c780 tx=0x7f3c7400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c73e00640 1 -- 172.21.15.32:0/2488124719 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74004b70 con 0x7f3c7c15e7f0 2024-04-17T09:58:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c73e00640 1 -- 172.21.15.32:0/2488124719 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c74004d10 con 0x7f3c7c15e7f0 2024-04-17T09:58:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c73e00640 1 -- 172.21.15.32:0/2488124719 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7400aa90 con 0x7f3c7c15e7f0 2024-04-17T09:58:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c81200640 1 -- 172.21.15.32:0/2488124719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7c156af0 con 0x7f3c7c15e7f0 2024-04-17T09:58:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.171+0000 7f3c81200640 1 -- 172.21.15.32:0/2488124719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c7c152da0 con 0x7f3c7c15e7f0 2024-04-17T09:58:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.175+0000 7f3c73e00640 1 -- 172.21.15.32:0/2488124719 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3c7400ac30 con 0x7f3c7c15e7f0 2024-04-17T09:58:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.175+0000 7f3c73e00640 1 --2- 172.21.15.32:0/2488124719 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c580683b0 0x7f3c5806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.175+0000 7f3c73e00640 1 -- 172.21.15.32:0/2488124719 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f3c74086bc0 con 0x7f3c7c15e7f0 2024-04-17T09:58:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.175+0000 7f3c7b400640 1 --2- 172.21.15.32:0/2488124719 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c580683b0 0x7f3c5806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.175+0000 7f3c85d07000 1 Processor -- start 2024-04-17T09:58:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.175+0000 7f3c85d07000 1 -- start start 2024-04-17T09:58:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.175+0000 7f3c7b400640 1 --2- 172.21.15.32:0/2488124719 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c580683b0 0x7f3c5806a870 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c0ae000 tx=0x7f3c6c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:13.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.175+0000 7f3c85d07000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e5a50 0x55916a5e5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.175+0000 7f3c85d07000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55916a5e6370 0x55916a598a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.175+0000 7f3c85d07000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916a598f70 0x55916a599350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.175+0000 7f3c85d07000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55916a3ea590 con 0x55916a5e5a50 2024-04-17T09:58:13.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.175+0000 7f3c85d07000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55916a3a3580 con 0x55916a598f70 2024-04-17T09:58:13.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.175+0000 7f3c85d07000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55916a3e6550 con 0x55916a5e6370 2024-04-17T09:58:13.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.175+0000 7f3c7b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e5a50 0x55916a5e5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.175+0000 7f3c7b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e5a50 0x55916a5e5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55906/0 (socket says 172.21.15.32:55906) 2024-04-17T09:58:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.175+0000 7f3c7b400640 1 -- 172.21.15.32:0/3609613162 learned_addr learned my addr 172.21.15.32:0/3609613162 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.175+0000 7f3c72a00640 1 --2- 172.21.15.32:0/3609613162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55916a5e6370 0x55916a598a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.175+0000 7f3c7be00640 1 --2- 172.21.15.32:0/3609613162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916a598f70 0x55916a599350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c7b400640 1 -- 172.21.15.32:0/3609613162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55916a5e6370 msgr2=0x55916a598a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c7b400640 1 --2- 172.21.15.32:0/3609613162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55916a5e6370 0x55916a598a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c7b400640 1 -- 172.21.15.32:0/3609613162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916a598f70 msgr2=0x55916a599350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c7b400640 1 --2- 172.21.15.32:0/3609613162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916a598f70 0x55916a599350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c7b400640 1 -- 172.21.15.32:0/3609613162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55916a491b90 con 0x55916a5e5a50 2024-04-17T09:58:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c7b400640 1 --2- 172.21.15.32:0/3609613162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e5a50 0x55916a5e5e30 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c01c9d0 tx=0x7f3c6c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fffc5cd4a277ad49 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c6be00640 1 -- 172.21.15.32:0/3609613162 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0b1040 con 0x55916a5e5a50 2024-04-17T09:58:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c6be00640 1 -- 172.21.15.32:0/3609613162 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c6c0b6930 con 0x55916a5e5a50 2024-04-17T09:58:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c6be00640 1 -- 172.21.15.32:0/3609613162 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0a8040 con 0x55916a5e5a50 2024-04-17T09:58:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c85d07000 1 -- 172.21.15.32:0/3609613162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e5a50 msgr2=0x55916a5e5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c85d07000 1 --2- 172.21.15.32:0/3609613162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e5a50 0x55916a5e5e30 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c01c9d0 tx=0x7f3c6c004740 comp rx=0 tx=0).stop 2024-04-17T09:58:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c85d07000 1 -- 172.21.15.32:0/3609613162 shutdown_connections 2024-04-17T09:58:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c85d07000 1 --2- 172.21.15.32:0/3609613162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55916a5e6370 0x55916a598a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c85d07000 1 --2- 172.21.15.32:0/3609613162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916a598f70 0x55916a599350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c85d07000 1 --2- 172.21.15.32:0/3609613162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e5a50 0x55916a5e5e30 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c85d07000 1 -- 172.21.15.32:0/3609613162 >> 172.21.15.32:0/3609613162 conn(0x55916a5c67f0 msgr2=0x55916a5c8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c85d07000 1 -- 172.21.15.32:0/3609613162 shutdown_connections 2024-04-17T09:58:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c85d07000 1 -- 172.21.15.32:0/3609613162 wait complete. 2024-04-17T09:58:13.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c85d07000 1 -- start start 2024-04-17T09:58:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c85d07000 1 Processor -- start 2024-04-17T09:58:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c85d07000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916a598f70 0x55916a5e4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c85d07000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55916a5e5a50 0x55916a5e5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c85d07000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e6370 0x55916a59d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c85d07000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55916a3f54c0 con 0x55916a5e6370 2024-04-17T09:58:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c85d07000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55916a38ac80 con 0x55916a598f70 2024-04-17T09:58:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c85d07000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55916a3b5930 con 0x55916a5e5a50 2024-04-17T09:58:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c85d07000 10 client.0 _ll_register_callbacks cb 0x55916a5866d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:58:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c7be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e6370 0x55916a59d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c7be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e6370 0x55916a59d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55916/0 (socket says 172.21.15.32:55916) 2024-04-17T09:58:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c7be00640 1 -- 172.21.15.32:0/3734067622 learned_addr learned my addr 172.21.15.32:0/3734067622 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c72a00640 1 --2- 172.21.15.32:0/3734067622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55916a5e5a50 0x55916a5e5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c7b400640 1 --2- 172.21.15.32:0/3734067622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916a598f70 0x55916a5e4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.179+0000 7f3c7be00640 1 -- 172.21.15.32:0/3734067622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55916a5e5a50 msgr2=0x55916a5e5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c7be00640 1 --2- 172.21.15.32:0/3734067622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55916a5e5a50 0x55916a5e5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c7be00640 1 -- 172.21.15.32:0/3734067622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916a598f70 msgr2=0x55916a5e4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c7be00640 1 --2- 172.21.15.32:0/3734067622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916a598f70 0x55916a5e4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c7be00640 1 -- 172.21.15.32:0/3734067622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55916a3c3b20 con 0x55916a5e6370 2024-04-17T09:58:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c7b400640 1 --2- 172.21.15.32:0/3734067622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916a598f70 0x55916a5e4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:58:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c7be00640 1 --2- 172.21.15.32:0/3734067622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e6370 0x55916a59d370 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f3c74052e60 tx=0x7f3c74056b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93282296138ae3f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c68c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:58:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c68c00640 1 -- 172.21.15.32:0/3734067622 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74083070 con 0x55916a5e6370 2024-04-17T09:58:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c68c00640 1 -- 172.21.15.32:0/3734067622 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f3c7405f0a0 con 0x55916a5e6370 2024-04-17T09:58:13.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c85d07000 1 -- 172.21.15.32:0/3734067622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c6c0b6930 con 0x55916a5e6370 2024-04-17T09:58:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c68c00640 1 -- 172.21.15.32:0/3734067622 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f3c7407a050 con 0x55916a5e6370 2024-04-17T09:58:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c85d07000 10 client.7230 fetch_fsmap learned FSMap version 49 2024-04-17T09:58:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c85d07000 1 -- 172.21.15.32:0/3734067622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x55916a42e610 con 0x55916a5e6370 2024-04-17T09:58:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c68c00640 1 -- 172.21.15.32:0/3734067622 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c6c0b6930 con 0x55916a5e6370 2024-04-17T09:58:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c68c00640 1 -- 172.21.15.32:0/3734067622 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7f3c74055e50 con 0x55916a5e6370 2024-04-17T09:58:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c85d07000 10 client.7230 fetch_fsmap finished waiting for FSMap version 49 2024-04-17T09:58:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c85d07000 10 client.7230 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T09:58:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c85d07000 20 client.7230 populate_metadata read hostname 'smithi032' 2024-04-17T09:58:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c85d07000 1 --2- 172.21.15.32:0/3734067622 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55916a59dd80 0x55916a5f6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c85d07000 4 client.7230 mds_command: new command op to 7011 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:58:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c85d07000 1 -- 172.21.15.32:0/3734067622 --> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55916a383e40 con 0x55916a59dd80 2024-04-17T09:58:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c7b400640 1 --2- 172.21.15.32:0/3734067622 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55916a59dd80 0x55916a5f6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c7b400640 1 --2- 172.21.15.32:0/3734067622 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55916a59dd80 0x55916a5f6d00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ceb9d52ed9e58e7 server_cookie=ab8bb742952cbde0 in_seq=0 out_seq=0 2024-04-17T09:58:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c68c00640 10 client.7230 ms_handle_connect on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:13.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c68c00640 1 -- 172.21.15.32:0/3734067622 <== mds.0 v2:172.21.15.32:6834/2416784965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55916a383e40 con 0x55916a59dd80 2024-04-17T09:58:13.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c68c00640 10 client.7230 handle_command_reply: tid=0 2024-04-17T09:58:13.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c85d07000 1 client.7230 shutdown 2024-04-17T09:58:13.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c85d07000 1 -- 172.21.15.32:0/3734067622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e6370 msgr2=0x55916a59d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c85d07000 1 --2- 172.21.15.32:0/3734067622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e6370 0x55916a59d370 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f3c74052e60 tx=0x7f3c74056b40 comp rx=0 tx=0).stop 2024-04-17T09:58:13.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c85d07000 1 -- 172.21.15.32:0/3734067622 shutdown_connections 2024-04-17T09:58:13.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c85d07000 1 --2- 172.21.15.32:0/3734067622 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55916a59dd80 0x55916a5f6d00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c85d07000 1 --2- 172.21.15.32:0/3734067622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55916a5e5a50 0x55916a5e5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c85d07000 1 --2- 172.21.15.32:0/3734067622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e6370 0x55916a59d370 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c68c00640 0 client.7230 ms_handle_reset on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:13.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c85d07000 1 --2- 172.21.15.32:0/3734067622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916a598f70 0x55916a5e4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c85d07000 1 -- 172.21.15.32:0/3734067622 >> 172.21.15.32:0/3734067622 conn(0x55916a5c67f0 msgr2=0x55916a5c8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:13.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c85d07000 1 -- 172.21.15.32:0/3734067622 shutdown_connections 2024-04-17T09:58:13.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c85d07000 1 -- 172.21.15.32:0/3734067622 wait complete. 2024-04-17T09:58:13.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.183+0000 7f3c85d07000 20 client.7230 trim_cache size 0 max 16384 2024-04-17T09:58:13.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c85d07000 1 Processor -- start 2024-04-17T09:58:13.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c85d07000 1 -- start start 2024-04-17T09:58:13.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c85d07000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916a598f70 0x55916a5987d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c85d07000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55916a5e5a50 0x55916a5e3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c85d07000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e6370 0x55916a5e4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c85d07000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c203c30 con 0x55916a5e6370 2024-04-17T09:58:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c85d07000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c202ca0 con 0x55916a598f70 2024-04-17T09:58:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c85d07000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3c7c15fa90 con 0x55916a5e5a50 2024-04-17T09:58:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c7be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e6370 0x55916a5e4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c7be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e6370 0x55916a5e4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55930/0 (socket says 172.21.15.32:55930) 2024-04-17T09:58:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c7be00640 1 -- 172.21.15.32:0/125154755 learned_addr learned my addr 172.21.15.32:0/125154755 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c7be00640 1 -- 172.21.15.32:0/125154755 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55916a5e5a50 msgr2=0x55916a5e3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c7be00640 1 --2- 172.21.15.32:0/125154755 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55916a5e5a50 0x55916a5e3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c7be00640 1 -- 172.21.15.32:0/125154755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916a598f70 msgr2=0x55916a5987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c7be00640 1 --2- 172.21.15.32:0/125154755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916a598f70 0x55916a5987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c7be00640 1 -- 172.21.15.32:0/125154755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55916a3c0190 con 0x55916a5e6370 2024-04-17T09:58:13.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c7be00640 1 --2- 172.21.15.32:0/125154755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e6370 0x55916a5e4040 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f3c7405f000 tx=0x7f3c74080000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59fc1f7d86ffb5ad server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:13.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c6b400640 1 -- 172.21.15.32:0/125154755 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74083070 con 0x55916a5e6370 2024-04-17T09:58:13.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c6b400640 1 -- 172.21.15.32:0/125154755 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c74054dc0 con 0x55916a5e6370 2024-04-17T09:58:13.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c6b400640 1 -- 172.21.15.32:0/125154755 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74056360 con 0x55916a5e6370 2024-04-17T09:58:13.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c85d07000 1 -- 172.21.15.32:0/125154755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e6370 msgr2=0x55916a5e4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c85d07000 1 --2- 172.21.15.32:0/125154755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e6370 0x55916a5e4040 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f3c7405f000 tx=0x7f3c74080000 comp rx=0 tx=0).stop 2024-04-17T09:58:13.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c85d07000 1 -- 172.21.15.32:0/125154755 shutdown_connections 2024-04-17T09:58:13.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c85d07000 1 --2- 172.21.15.32:0/125154755 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55916a5e5a50 0x55916a5e3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c85d07000 1 --2- 172.21.15.32:0/125154755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e6370 0x55916a5e4040 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c85d07000 1 --2- 172.21.15.32:0/125154755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916a598f70 0x55916a5987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c85d07000 1 -- 172.21.15.32:0/125154755 >> 172.21.15.32:0/125154755 conn(0x55916a5c67f0 msgr2=0x55916a59bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:13.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c85d07000 1 -- 172.21.15.32:0/125154755 shutdown_connections 2024-04-17T09:58:13.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c85d07000 1 -- 172.21.15.32:0/125154755 wait complete. 2024-04-17T09:58:13.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c85d07000 1 -- start start 2024-04-17T09:58:13.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.199+0000 7f3c85d07000 1 Processor -- start 2024-04-17T09:58:13.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55916a598f70 0x55916a6546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916a5e5a50 0x55916a654c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e6370 0x55916a5f6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55916a3f77a0 con 0x55916a5e6370 2024-04-17T09:58:13.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c202650 con 0x55916a5e5a50 2024-04-17T09:58:13.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55916a3e98a0 con 0x55916a598f70 2024-04-17T09:58:13.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 10 client.0 _ll_register_callbacks cb 0x7f3c74056860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:58:13.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c7be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e6370 0x55916a5f6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c7be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e6370 0x55916a5f6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55934/0 (socket says 172.21.15.32:55934) 2024-04-17T09:58:13.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c7be00640 1 -- 172.21.15.32:0/425177799 learned_addr learned my addr 172.21.15.32:0/425177799 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:13.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c7b400640 1 --2- 172.21.15.32:0/425177799 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55916a598f70 0x55916a6546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c72a00640 1 --2- 172.21.15.32:0/425177799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916a5e5a50 0x55916a654c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c7be00640 1 -- 172.21.15.32:0/425177799 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55916a598f70 msgr2=0x55916a6546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:13.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c7be00640 1 --2- 172.21.15.32:0/425177799 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55916a598f70 0x55916a6546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c7be00640 1 -- 172.21.15.32:0/425177799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916a5e5a50 msgr2=0x55916a654c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:13.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c7be00640 1 --2- 172.21.15.32:0/425177799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916a5e5a50 0x55916a654c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c7be00640 1 -- 172.21.15.32:0/425177799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55916a3a37d0 con 0x55916a5e6370 2024-04-17T09:58:13.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c72a00640 1 --2- 172.21.15.32:0/425177799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916a5e5a50 0x55916a654c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:58:13.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c7be00640 1 --2- 172.21.15.32:0/425177799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e6370 0x55916a5f6d00 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f3c74052e60 tx=0x7f3c74084e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c271f3fe87029 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:13.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c68c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:58:13.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c68c00640 1 -- 172.21.15.32:0/425177799 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74056360 con 0x55916a5e6370 2024-04-17T09:58:13.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c68c00640 1 -- 172.21.15.32:0/425177799 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f3c7405f0a0 con 0x55916a5e6370 2024-04-17T09:58:13.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 1 -- 172.21.15.32:0/425177799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55916a3f7fb0 con 0x55916a5e6370 2024-04-17T09:58:13.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c68c00640 1 -- 172.21.15.32:0/425177799 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f3c7405d240 con 0x55916a5e6370 2024-04-17T09:58:13.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 10 client.7236 fetch_fsmap learned FSMap version 49 2024-04-17T09:58:13.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 1 -- 172.21.15.32:0/425177799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x55916a38f440 con 0x55916a5e6370 2024-04-17T09:58:13.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c68c00640 1 -- 172.21.15.32:0/425177799 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55916a3f7fb0 con 0x55916a5e6370 2024-04-17T09:58:13.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c68c00640 1 -- 172.21.15.32:0/425177799 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7f3c74006b60 con 0x55916a5e6370 2024-04-17T09:58:13.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 10 client.7236 fetch_fsmap finished waiting for FSMap version 49 2024-04-17T09:58:13.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 10 client.7236 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T09:58:13.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 20 client.7236 populate_metadata read hostname 'smithi032' 2024-04-17T09:58:13.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 1 --2- 172.21.15.32:0/425177799 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55916a5f7630 0x55916a5f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 4 client.7236 mds_command: new command op to 7011 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-17T09:58:13.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 1 -- 172.21.15.32:0/425177799 --> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55916a3f1e20 con 0x55916a5f7630 2024-04-17T09:58:13.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c7b400640 1 --2- 172.21.15.32:0/425177799 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55916a5f7630 0x55916a5f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c7b400640 1 --2- 172.21.15.32:0/425177799 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55916a5f7630 0x55916a5f7a10 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaaff603e1c4272 server_cookie=7d962b9b25baf684 in_seq=0 out_seq=0 2024-04-17T09:58:13.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c68c00640 10 client.7236 ms_handle_connect on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:13.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c68c00640 1 -- 172.21.15.32:0/425177799 <== mds.0 v2:172.21.15.32:6834/2416784965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+237 (crc 0 0 0) 0x55916a3f1e20 con 0x55916a5f7630 2024-04-17T09:58:13.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c68c00640 10 client.7236 handle_command_reply: tid=0 2024-04-17T09:58:13.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 1 client.7236 shutdown 2024-04-17T09:58:13.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 1 -- 172.21.15.32:0/425177799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e6370 msgr2=0x55916a5f6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 1 --2- 172.21.15.32:0/425177799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e6370 0x55916a5f6d00 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f3c74052e60 tx=0x7f3c74084e90 comp rx=0 tx=0).stop 2024-04-17T09:58:13.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 1 -- 172.21.15.32:0/425177799 shutdown_connections 2024-04-17T09:58:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 1 --2- 172.21.15.32:0/425177799 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55916a5f7630 0x55916a5f7a10 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 1 --2- 172.21.15.32:0/425177799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55916a5e6370 0x55916a5f6d00 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 1 --2- 172.21.15.32:0/425177799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55916a5e5a50 0x55916a654c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c85d07000 1 --2- 172.21.15.32:0/425177799 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55916a598f70 0x55916a6546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.203+0000 7f3c68c00640 0 client.7236 ms_handle_reset on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.207+0000 7f3c85d07000 1 -- 172.21.15.32:0/425177799 >> 172.21.15.32:0/425177799 conn(0x55916a5c67f0 msgr2=0x55916a5f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.207+0000 7f3c85d07000 1 -- 172.21.15.32:0/425177799 shutdown_connections 2024-04-17T09:58:13.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.207+0000 7f3c85d07000 1 -- 172.21.15.32:0/425177799 wait complete. 2024-04-17T09:58:13.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.207+0000 7f3c85d07000 20 client.7236 trim_cache size 0 max 16384 2024-04-17T09:58:13.218 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:58:13.218 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-17T09:58:13.219 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:58:13.219 INFO:teuthology.orchestra.run.smithi032.stdout: "damage_type": "dentry", 2024-04-17T09:58:13.219 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 2256461683, 2024-04-17T09:58:13.219 INFO:teuthology.orchestra.run.smithi032.stdout: "ino": 1099511627776, 2024-04-17T09:58:13.219 INFO:teuthology.orchestra.run.smithi032.stdout: "frag": "*", 2024-04-17T09:58:13.219 INFO:teuthology.orchestra.run.smithi032.stdout: "dname": "file_to_be_damaged", 2024-04-17T09:58:13.220 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_id": "head", 2024-04-17T09:58:13.220 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/subdir/file_to_be_damaged" 2024-04-17T09:58:13.220 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:58:13.220 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-17T09:58:13.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.207+0000 7f3c81200640 1 -- 172.21.15.32:0/2488124719 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c580683b0 msgr2=0x7f3c5806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.207+0000 7f3c81200640 1 --2- 172.21.15.32:0/2488124719 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c580683b0 0x7f3c5806a870 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c0ae000 tx=0x7f3c6c009040 comp rx=0 tx=0).stop 2024-04-17T09:58:13.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.207+0000 7f3c81200640 1 -- 172.21.15.32:0/2488124719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c7c15e7f0 msgr2=0x7f3c7c070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.207+0000 7f3c81200640 1 --2- 172.21.15.32:0/2488124719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c7c15e7f0 0x7f3c7c070030 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400c780 tx=0x7f3c7400cc40 comp rx=0 tx=0).stop 2024-04-17T09:58:13.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.207+0000 7f3c81200640 1 -- 172.21.15.32:0/2488124719 shutdown_connections 2024-04-17T09:58:13.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.207+0000 7f3c81200640 1 --2- 172.21.15.32:0/2488124719 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3c580683b0 0x7f3c5806a870 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.207+0000 7f3c81200640 1 --2- 172.21.15.32:0/2488124719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c7c06da90 0x7f3c7c1608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.207+0000 7f3c81200640 1 --2- 172.21.15.32:0/2488124719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c7c15e7f0 0x7f3c7c070030 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.207+0000 7f3c81200640 1 --2- 172.21.15.32:0/2488124719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c7c06d080 0x7f3c7c152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.207+0000 7f3c81200640 1 -- 172.21.15.32:0/2488124719 >> 172.21.15.32:0/2488124719 conn(0x7f3c7c0b3f50 msgr2=0x7f3c7c09a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:13.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.207+0000 7f3c81200640 1 -- 172.21.15.32:0/2488124719 shutdown_connections 2024-04-17T09:58:13.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.207+0000 7f3c81200640 1 -- 172.21.15.32:0/2488124719 wait complete. 2024-04-17T09:58:13.227 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-17T09:58:13.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.367+0000 7fad3b800640 1 Processor -- start 2024-04-17T09:58:13.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.367+0000 7fad3b800640 1 -- start start 2024-04-17T09:58:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.367+0000 7fad3b800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fad340b2b80 0x7fad340d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.367+0000 7fad3b800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad340b3120 0x7fad3406d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.367+0000 7fad3b800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad3409b650 con 0x7fad340b3670 2024-04-17T09:58:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.367+0000 7fad3b800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fad340b2f60 con 0x7fad340b2b80 2024-04-17T09:58:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.367+0000 7fad3b800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fad3409b400 con 0x7fad340b3120 2024-04-17T09:58:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.367+0000 7fad39800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fad340b3670 0x7fad3415f5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48970/0 (socket says 172.21.15.32:48970) 2024-04-17T09:58:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.367+0000 7fad39800640 1 -- 172.21.15.32:0/3036255003 learned_addr learned my addr 172.21.15.32:0/3036255003 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:58:13.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.367+0000 7fad38e00640 1 --2- 172.21.15.32:0/3036255003 >> v2:172.21.15.144:3300/0 conn(0x7fad340b2b80 0x7fad340d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.367+0000 7fad2be00640 1 --2- 172.21.15.32:0/3036255003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad340b3120 0x7fad3406d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.367+0000 7fad33e00640 1 -- 172.21.15.32:0/3036255003 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 483551573 0 0) 0x7fad3409b650 con 0x7fad340b3670 2024-04-17T09:58:13.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.367+0000 7fad33e00640 1 -- 172.21.15.32:0/3036255003 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad18003530 con 0x7fad340b3670 2024-04-17T09:58:13.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.367+0000 7fad38e00640 1 -- 172.21.15.32:0/3036255003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad340b3120 msgr2=0x7fad3406d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:13.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.367+0000 7fad38e00640 1 --2- 172.21.15.32:0/3036255003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad340b3120 0x7fad3406d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.367+0000 7fad38e00640 1 -- 172.21.15.32:0/3036255003 >> v1:172.21.15.32:6789/0 conn(0x7fad340b3670 legacy=0x7fad3415f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.367+0000 7fad38e00640 1 -- 172.21.15.32:0/3036255003 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3406cd50 con 0x7fad340b2b80 2024-04-17T09:58:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.371+0000 7fad38e00640 1 --2- 172.21.15.32:0/3036255003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad340b2b80 0x7fad340d0900 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fad240097c0 tx=0x7fad2401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35cbaca0c6370405 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.371+0000 7fad33e00640 1 -- 172.21.15.32:0/3036255003 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad240b0070 con 0x7fad340b2b80 2024-04-17T09:58:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.371+0000 7fad33e00640 1 -- 172.21.15.32:0/3036255003 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad240ab600 con 0x7fad340b2b80 2024-04-17T09:58:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.371+0000 7fad33e00640 1 -- 172.21.15.32:0/3036255003 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad240ab9d0 con 0x7fad340b2b80 2024-04-17T09:58:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.371+0000 7fad3b800640 1 -- 172.21.15.32:0/3036255003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad340b2b80 msgr2=0x7fad340d0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.371+0000 7fad3b800640 1 --2- 172.21.15.32:0/3036255003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad340b2b80 0x7fad340d0900 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fad240097c0 tx=0x7fad2401c410 comp rx=0 tx=0).stop 2024-04-17T09:58:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.371+0000 7fad3b800640 1 -- 172.21.15.32:0/3036255003 shutdown_connections 2024-04-17T09:58:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.371+0000 7fad3b800640 1 --2- 172.21.15.32:0/3036255003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad340b3120 0x7fad3406d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.371+0000 7fad3b800640 1 --2- 172.21.15.32:0/3036255003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad340b2b80 0x7fad340d0900 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.371+0000 7fad3b800640 1 -- 172.21.15.32:0/3036255003 >> 172.21.15.32:0/3036255003 conn(0x7fad340b4010 msgr2=0x7fad340b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:13.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.371+0000 7fad3b800640 1 -- 172.21.15.32:0/3036255003 shutdown_connections 2024-04-17T09:58:13.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.371+0000 7fad3b800640 1 -- 172.21.15.32:0/3036255003 wait complete. 2024-04-17T09:58:13.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad3b800640 1 Processor -- start 2024-04-17T09:58:13.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad3b800640 1 -- start start 2024-04-17T09:58:13.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad3b800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad340b2b80 0x7fad3409fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad3b800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad340b3120 0x7fad3414c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad3b800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad340b3670 0x7fad3415e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad3b800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fad340b3db0 con 0x7fad340b3670 2024-04-17T09:58:13.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad3b800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fad340b3c30 con 0x7fad340b2b80 2024-04-17T09:58:13.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad3b800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fad3416c2d0 con 0x7fad340b3120 2024-04-17T09:58:13.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad39800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad340b3670 0x7fad3415e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad39800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad340b3670 0x7fad3415e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55946/0 (socket says 172.21.15.32:55946) 2024-04-17T09:58:13.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad39800640 1 -- 172.21.15.32:0/1017943218 learned_addr learned my addr 172.21.15.32:0/1017943218 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:13.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad38e00640 1 --2- 172.21.15.32:0/1017943218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad340b2b80 0x7fad3409fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad2be00640 1 --2- 172.21.15.32:0/1017943218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad340b3120 0x7fad3414c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad39800640 1 -- 172.21.15.32:0/1017943218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad340b3120 msgr2=0x7fad3414c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad39800640 1 --2- 172.21.15.32:0/1017943218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad340b3120 0x7fad3414c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad39800640 1 -- 172.21.15.32:0/1017943218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad340b2b80 msgr2=0x7fad3409fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad39800640 1 --2- 172.21.15.32:0/1017943218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad340b2b80 0x7fad3409fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad39800640 1 -- 172.21.15.32:0/1017943218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad340a0110 con 0x7fad340b3670 2024-04-17T09:58:13.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad2be00640 1 --2- 172.21.15.32:0/1017943218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad340b3120 0x7fad3414c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:58:13.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad39800640 1 --2- 172.21.15.32:0/1017943218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad340b3670 0x7fad3415e4a0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00cb40 tx=0x7fad2c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:13.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad31600640 1 -- 172.21.15.32:0/1017943218 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c017070 con 0x7fad340b3670 2024-04-17T09:58:13.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad31600640 1 -- 172.21.15.32:0/1017943218 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad2c00ac40 con 0x7fad340b3670 2024-04-17T09:58:13.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad31600640 1 -- 172.21.15.32:0/1017943218 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c012620 con 0x7fad340b3670 2024-04-17T09:58:13.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad3b800640 1 -- 172.21.15.32:0/1017943218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad340b1650 con 0x7fad340b3670 2024-04-17T09:58:13.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.375+0000 7fad3b800640 1 -- 172.21.15.32:0/1017943218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad3409df10 con 0x7fad340b3670 2024-04-17T09:58:13.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.379+0000 7fad3b800640 1 -- 172.21.15.32:0/1017943218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad34068a90 con 0x7fad340b3670 2024-04-17T09:58:13.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.379+0000 7fad31600640 1 -- 172.21.15.32:0/1017943218 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fad2c01a070 con 0x7fad340b3670 2024-04-17T09:58:13.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.379+0000 7fad31600640 1 --2- 172.21.15.32:0/1017943218 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fad10068380 0x7fad1006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.379+0000 7fad31600640 1 -- 172.21.15.32:0/1017943218 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fad2c0867e0 con 0x7fad340b3670 2024-04-17T09:58:13.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.379+0000 7fad38e00640 1 --2- 172.21.15.32:0/1017943218 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fad10068380 0x7fad1006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.379+0000 7fad38e00640 1 --2- 172.21.15.32:0/1017943218 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fad10068380 0x7fad1006a840 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fad240ad000 tx=0x7fad24008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:13.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.383+0000 7fad31600640 1 -- 172.21.15.32:0/1017943218 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fad2c056620 con 0x7fad340b3670 2024-04-17T09:58:13.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.587+0000 7fad3b800640 1 -- 172.21.15.32:0/1017943218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fad340c9360 con 0x7fad340b3670 2024-04-17T09:58:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.587+0000 7fad31600640 1 -- 172.21.15.32:0/1017943218 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7fad2c05a920 con 0x7fad340b3670 2024-04-17T09:58:13.588 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:58:13.588 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-17T09:58:13.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.587+0000 7fad3b800640 1 -- 172.21.15.32:0/1017943218 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fad10068380 msgr2=0x7fad1006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.587+0000 7fad3b800640 1 --2- 172.21.15.32:0/1017943218 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fad10068380 0x7fad1006a840 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fad240ad000 tx=0x7fad24008040 comp rx=0 tx=0).stop 2024-04-17T09:58:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.587+0000 7fad3b800640 1 -- 172.21.15.32:0/1017943218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad340b3670 msgr2=0x7fad3415e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.587+0000 7fad3b800640 1 --2- 172.21.15.32:0/1017943218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad340b3670 0x7fad3415e4a0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00cb40 tx=0x7fad2c00a060 comp rx=0 tx=0).stop 2024-04-17T09:58:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.587+0000 7fad3b800640 1 -- 172.21.15.32:0/1017943218 shutdown_connections 2024-04-17T09:58:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.587+0000 7fad3b800640 1 --2- 172.21.15.32:0/1017943218 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fad10068380 0x7fad1006a840 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.587+0000 7fad3b800640 1 --2- 172.21.15.32:0/1017943218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad340b3120 0x7fad3414c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.587+0000 7fad3b800640 1 --2- 172.21.15.32:0/1017943218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad340b3670 0x7fad3415e4a0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.587+0000 7fad3b800640 1 --2- 172.21.15.32:0/1017943218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad340b2b80 0x7fad3409fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.587+0000 7fad3b800640 1 -- 172.21.15.32:0/1017943218 >> 172.21.15.32:0/1017943218 conn(0x7fad340b4010 msgr2=0x7fad340b5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.587+0000 7fad3b800640 1 -- 172.21.15.32:0/1017943218 shutdown_connections 2024-04-17T09:58:13.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.587+0000 7fad3b800640 1 -- 172.21.15.32:0/1017943218 wait complete. 2024-04-17T09:58:13.610 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-17T09:58:13.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:58:13.610 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-04-17T09:58:13.693 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:58:13.696 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:58:13.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.755+0000 7f81a7400640 1 Processor -- start 2024-04-17T09:58:13.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.755+0000 7f81a7400640 1 -- start start 2024-04-17T09:58:13.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.755+0000 7f81a7400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81a00cfc70 0x7f81a006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.755+0000 7f81a7400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81a00cd870 0x7f81a015ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.755+0000 7f81a7400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81a015f200 con 0x7f81a006f330 2024-04-17T09:58:13.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.755+0000 7f81a7400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f81a015fb30 con 0x7f81a00cd870 2024-04-17T09:58:13.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.755+0000 7f81a7400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f81a006f770 con 0x7f81a00cfc70 2024-04-17T09:58:13.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.755+0000 7f81a5400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81a00cd870 0x7f81a015ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.755+0000 7f81a5400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81a00cd870 0x7f81a015ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55962/0 (socket says 172.21.15.32:55962) 2024-04-17T09:58:13.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.755+0000 7f81a5400640 1 -- 172.21.15.32:0/1292038276 learned_addr learned my addr 172.21.15.32:0/1292038276 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:13.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.755+0000 7f8197e00640 1 --1- 172.21.15.32:0/1292038276 >> v1:172.21.15.32:6789/0 conn(0x7f81a006f330 0x7f81a00cd160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48974/0 (socket says 172.21.15.32:48974) 2024-04-17T09:58:13.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.755+0000 7f81a4a00640 1 --2- 172.21.15.32:0/1292038276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81a00cfc70 0x7f81a006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.755+0000 7f81a5400640 1 -- 172.21.15.32:0/1292038276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81a00cfc70 msgr2=0x7f81a006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:13.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.755+0000 7f81a5400640 1 --2- 172.21.15.32:0/1292038276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81a00cfc70 0x7f81a006edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.755+0000 7f81a5400640 1 -- 172.21.15.32:0/1292038276 >> v1:172.21.15.32:6789/0 conn(0x7f81a006f330 legacy=0x7f81a00cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.755+0000 7f81a5400640 1 -- 172.21.15.32:0/1292038276 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a015f4e0 con 0x7f81a00cd870 2024-04-17T09:58:13.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.759+0000 7f81a5400640 1 --2- 172.21.15.32:0/1292038276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a00cd870 0x7f81a015ba70 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f819800d150 tx=0x7f819801ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734af6e8b008a7c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:13.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.759+0000 7f819fe00640 1 -- 172.21.15.32:0/1292038276 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819800b2d0 con 0x7f81a00cd870 2024-04-17T09:58:13.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.759+0000 7f819fe00640 1 -- 172.21.15.32:0/1292038276 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f819800b470 con 0x7f81a00cd870 2024-04-17T09:58:13.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.759+0000 7f819fe00640 1 -- 172.21.15.32:0/1292038276 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81980a7b20 con 0x7f81a00cd870 2024-04-17T09:58:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.759+0000 7f81a7400640 1 -- 172.21.15.32:0/1292038276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a00cd870 msgr2=0x7f81a015ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.759+0000 7f81a7400640 1 --2- 172.21.15.32:0/1292038276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a00cd870 0x7f81a015ba70 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f819800d150 tx=0x7f819801ec30 comp rx=0 tx=0).stop 2024-04-17T09:58:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.759+0000 7f81a7400640 1 -- 172.21.15.32:0/1292038276 shutdown_connections 2024-04-17T09:58:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.759+0000 7f81a7400640 1 --2- 172.21.15.32:0/1292038276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a00cd870 0x7f81a015ba70 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.759+0000 7f81a7400640 1 --2- 172.21.15.32:0/1292038276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81a00cfc70 0x7f81a006edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.759+0000 7f81a7400640 1 -- 172.21.15.32:0/1292038276 >> 172.21.15.32:0/1292038276 conn(0x7f81a00b2190 msgr2=0x7f81a00b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.759+0000 7f81a7400640 1 -- 172.21.15.32:0/1292038276 shutdown_connections 2024-04-17T09:58:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.759+0000 7f81a7400640 1 -- 172.21.15.32:0/1292038276 wait complete. 2024-04-17T09:58:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.763+0000 7f81a7400640 1 Processor -- start 2024-04-17T09:58:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.763+0000 7f81a7400640 1 -- start start 2024-04-17T09:58:13.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.763+0000 7f81a7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a006f330 0x7f81a014dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.763+0000 7f81a7400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81a00cd870 0x7f81a014e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.763+0000 7f81a7400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81a00cfc70 0x7f81a015a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.763+0000 7f81a7400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f81a006de60 con 0x7f81a006f330 2024-04-17T09:58:13.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.763+0000 7f81a7400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f81a006dce0 con 0x7f81a00cfc70 2024-04-17T09:58:13.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.763+0000 7f81a7400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f81a006dfe0 con 0x7f81a00cd870 2024-04-17T09:58:13.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.763+0000 7f81a4a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a006f330 0x7f81a014dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.763+0000 7f81a4a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a006f330 0x7f81a014dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55968/0 (socket says 172.21.15.32:55968) 2024-04-17T09:58:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.763+0000 7f81a4a00640 1 -- 172.21.15.32:0/3250510113 learned_addr learned my addr 172.21.15.32:0/3250510113 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.763+0000 7f81a4a00640 1 -- 172.21.15.32:0/3250510113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81a00cd870 msgr2=0x7f81a014e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.763+0000 7f81a4a00640 1 --2- 172.21.15.32:0/3250510113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81a00cd870 0x7f81a014e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.763+0000 7f81a4a00640 1 -- 172.21.15.32:0/3250510113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81a00cfc70 msgr2=0x7f81a015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.763+0000 7f81a4a00640 1 --2- 172.21.15.32:0/3250510113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81a00cfc70 0x7f81a015a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.763+0000 7f81a4a00640 1 -- 172.21.15.32:0/3250510113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a0151fc0 con 0x7f81a006f330 2024-04-17T09:58:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.763+0000 7f81a4a00640 1 --2- 172.21.15.32:0/3250510113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a006f330 0x7f81a014dd00 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f819000d5f0 tx=0x7f819000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.763+0000 7f819d600640 1 -- 172.21.15.32:0/3250510113 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81900125f0 con 0x7f81a006f330 2024-04-17T09:58:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.763+0000 7f819d600640 1 -- 172.21.15.32:0/3250510113 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8190019070 con 0x7f81a006f330 2024-04-17T09:58:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.763+0000 7f819d600640 1 -- 172.21.15.32:0/3250510113 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81900129b0 con 0x7f81a006f330 2024-04-17T09:58:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.763+0000 7f81a7400640 1 -- 172.21.15.32:0/3250510113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a01522a0 con 0x7f81a006f330 2024-04-17T09:58:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.763+0000 7f81a7400640 1 -- 172.21.15.32:0/3250510113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a0152860 con 0x7f81a006f330 2024-04-17T09:58:13.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.767+0000 7f819d600640 1 -- 172.21.15.32:0/3250510113 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8190012b50 con 0x7f81a006f330 2024-04-17T09:58:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.767+0000 7f819d600640 1 --2- 172.21.15.32:0/3250510113 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f817c0683b0 0x7f817c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.767+0000 7f819d600640 1 -- 172.21.15.32:0/3250510113 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f8190086ee0 con 0x7f81a006f330 2024-04-17T09:58:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.767+0000 7f8197e00640 1 --2- 172.21.15.32:0/3250510113 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f817c0683b0 0x7f817c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.767+0000 7f81aa949000 1 Processor -- start 2024-04-17T09:58:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.767+0000 7f81aa949000 1 -- start start 2024-04-17T09:58:13.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.767+0000 7f8197e00640 1 --2- 172.21.15.32:0/3250510113 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f817c0683b0 0x7f817c06a870 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8188007fd0 tx=0x7f8188007940 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.767+0000 7f81aa949000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55647770a600 0x55647770a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.767+0000 7f81aa949000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647770af20 0x5564776bd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.767+0000 7f81aa949000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564776bdcd0 0x5564776be0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.767+0000 7f81aa949000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5564774ea270 con 0x5564776bdcd0 2024-04-17T09:58:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.767+0000 7f81aa949000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55647750d290 con 0x55647770af20 2024-04-17T09:58:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.767+0000 7f81aa949000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5564774c6b30 con 0x55647770a600 2024-04-17T09:58:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.767+0000 7f81a5400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564776bdcd0 0x5564776be0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81a5400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564776bdcd0 0x5564776be0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55982/0 (socket says 172.21.15.32:55982) 2024-04-17T09:58:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81a5400640 1 -- 172.21.15.32:0/3309873926 learned_addr learned my addr 172.21.15.32:0/3309873926 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81a4a00640 1 --2- 172.21.15.32:0/3309873926 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55647770a600 0x55647770a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f8197e00640 1 --2- 172.21.15.32:0/3309873926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647770af20 0x5564776bd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81a5400640 1 -- 172.21.15.32:0/3309873926 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55647770a600 msgr2=0x55647770a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81a5400640 1 --2- 172.21.15.32:0/3309873926 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55647770a600 0x55647770a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81a5400640 1 -- 172.21.15.32:0/3309873926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647770af20 msgr2=0x5564776bd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81a5400640 1 --2- 172.21.15.32:0/3309873926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647770af20 0x5564776bd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81a5400640 1 -- 172.21.15.32:0/3309873926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564774ac960 con 0x5564776bdcd0 2024-04-17T09:58:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81a5400640 1 --2- 172.21.15.32:0/3309873926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564776bdcd0 0x5564776be0b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f81980b1040 tx=0x7f8198004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bb7ddd279cc879 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f8195600640 1 -- 172.21.15.32:0/3309873926 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819800b2d0 con 0x5564776bdcd0 2024-04-17T09:58:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f8195600640 1 -- 172.21.15.32:0/3309873926 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81980be070 con 0x5564776bdcd0 2024-04-17T09:58:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f8195600640 1 -- 172.21.15.32:0/3309873926 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81980a7b40 con 0x5564776bdcd0 2024-04-17T09:58:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81aa949000 1 -- 172.21.15.32:0/3309873926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564776bdcd0 msgr2=0x5564776be0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81aa949000 1 --2- 172.21.15.32:0/3309873926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564776bdcd0 0x5564776be0b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f81980b1040 tx=0x7f8198004a50 comp rx=0 tx=0).stop 2024-04-17T09:58:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81aa949000 1 -- 172.21.15.32:0/3309873926 shutdown_connections 2024-04-17T09:58:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81aa949000 1 --2- 172.21.15.32:0/3309873926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5564776bdcd0 0x5564776be0b0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81aa949000 1 --2- 172.21.15.32:0/3309873926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647770af20 0x5564776bd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81aa949000 1 --2- 172.21.15.32:0/3309873926 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55647770a600 0x55647770a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81aa949000 1 -- 172.21.15.32:0/3309873926 >> 172.21.15.32:0/3309873926 conn(0x5564776eb190 msgr2=0x5564776ed580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:13.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81aa949000 1 -- 172.21.15.32:0/3309873926 shutdown_connections 2024-04-17T09:58:13.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81aa949000 1 -- 172.21.15.32:0/3309873926 wait complete. 2024-04-17T09:58:13.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81aa949000 1 -- start start 2024-04-17T09:58:13.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81aa949000 1 Processor -- start 2024-04-17T09:58:13.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81aa949000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5564776bdcd0 0x5564776c33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81aa949000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647770a600 0x5564776c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81aa949000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55647770af20 0x5564776c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81aa949000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5564775181c0 con 0x55647770af20 2024-04-17T09:58:13.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81aa949000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5564774d4e80 con 0x55647770a600 2024-04-17T09:58:13.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81aa949000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55647750c5a0 con 0x5564776bdcd0 2024-04-17T09:58:13.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.771+0000 7f81aa949000 10 client.0 _ll_register_callbacks cb 0x5564776ab6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:58:13.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81a5400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55647770af20 0x5564776c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81a5400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55647770af20 0x5564776c3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55992/0 (socket says 172.21.15.32:55992) 2024-04-17T09:58:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81a5400640 1 -- 172.21.15.32:0/2700993870 learned_addr learned my addr 172.21.15.32:0/2700993870 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81a4a00640 1 --2- 172.21.15.32:0/2700993870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5564776bdcd0 0x5564776c33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f8197e00640 1 --2- 172.21.15.32:0/2700993870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647770a600 0x5564776c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81a5400640 1 -- 172.21.15.32:0/2700993870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5564776bdcd0 msgr2=0x5564776c33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81a5400640 1 --2- 172.21.15.32:0/2700993870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5564776bdcd0 0x5564776c33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81a5400640 1 -- 172.21.15.32:0/2700993870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647770a600 msgr2=0x5564776c38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81a5400640 1 --2- 172.21.15.32:0/2700993870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647770a600 0x5564776c38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81a5400640 1 -- 172.21.15.32:0/2700993870 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564774c7b00 con 0x55647770af20 2024-04-17T09:58:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81a5400640 1 --2- 172.21.15.32:0/2700993870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55647770af20 0x5564776c3e20 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f819801f4d0 tx=0x7f819801d470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adaa9b4900c9fd6e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f818e000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:58:13.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f818e000640 1 -- 172.21.15.32:0/2700993870 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819800b150 con 0x55647770af20 2024-04-17T09:58:13.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f818e000640 1 -- 172.21.15.32:0/2700993870 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f81980a7900 con 0x55647770af20 2024-04-17T09:58:13.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81aa949000 1 -- 172.21.15.32:0/2700993870 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81980be070 con 0x55647770af20 2024-04-17T09:58:13.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f818e000640 1 -- 172.21.15.32:0/2700993870 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f81980d1050 con 0x55647770af20 2024-04-17T09:58:13.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81aa949000 10 client.7257 fetch_fsmap learned FSMap version 49 2024-04-17T09:58:13.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81aa949000 1 -- 172.21.15.32:0/2700993870 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x5564774a7dc0 con 0x55647770af20 2024-04-17T09:58:13.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f818e000640 1 -- 172.21.15.32:0/2700993870 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81980be070 con 0x55647770af20 2024-04-17T09:58:13.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f818e000640 1 -- 172.21.15.32:0/2700993870 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7f81980d4020 con 0x55647770af20 2024-04-17T09:58:13.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81aa949000 10 client.7257 fetch_fsmap finished waiting for FSMap version 49 2024-04-17T09:58:13.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81aa949000 10 client.7257 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-17T09:58:13.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81aa949000 20 client.7257 populate_metadata read hostname 'smithi032' 2024-04-17T09:58:13.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81aa949000 1 --2- 172.21.15.32:0/2700993870 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x5564777198a0 0x55647771bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81aa949000 4 client.7257 mds_command: new command op to 7011 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:58:13.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81aa949000 1 -- 172.21.15.32:0/2700993870 --> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564774a6cc0 con 0x5564777198a0 2024-04-17T09:58:13.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81a4a00640 1 --2- 172.21.15.32:0/2700993870 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x5564777198a0 0x55647771bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81a4a00640 1 --2- 172.21.15.32:0/2700993870 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x5564777198a0 0x55647771bc90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd35088fd4b9ff7d server_cookie=e66ff7e6e83c9c48 in_seq=0 out_seq=0 2024-04-17T09:58:13.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f818e000640 10 client.7257 ms_handle_connect on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f818e000640 1 -- 172.21.15.32:0/2700993870 <== mds.0 v2:172.21.15.32:6834/2416784965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5564774a6cc0 con 0x5564777198a0 2024-04-17T09:58:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f818e000640 10 client.7257 handle_command_reply: tid=0 2024-04-17T09:58:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81aa949000 1 client.7257 shutdown 2024-04-17T09:58:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81aa949000 1 -- 172.21.15.32:0/2700993870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55647770af20 msgr2=0x5564776c3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81aa949000 1 --2- 172.21.15.32:0/2700993870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55647770af20 0x5564776c3e20 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f819801f4d0 tx=0x7f819801d470 comp rx=0 tx=0).stop 2024-04-17T09:58:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81aa949000 1 -- 172.21.15.32:0/2700993870 shutdown_connections 2024-04-17T09:58:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81aa949000 1 --2- 172.21.15.32:0/2700993870 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x5564777198a0 0x55647771bc90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81aa949000 1 --2- 172.21.15.32:0/2700993870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55647770af20 0x5564776c3e20 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81aa949000 1 --2- 172.21.15.32:0/2700993870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647770a600 0x5564776c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81aa949000 1 --2- 172.21.15.32:0/2700993870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5564776bdcd0 0x5564776c33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f818e000640 0 client.7257 ms_handle_reset on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81aa949000 1 -- 172.21.15.32:0/2700993870 >> 172.21.15.32:0/2700993870 conn(0x5564776eb190 msgr2=0x5564776ed580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81aa949000 1 -- 172.21.15.32:0/2700993870 shutdown_connections 2024-04-17T09:58:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81aa949000 1 -- 172.21.15.32:0/2700993870 wait complete. 2024-04-17T09:58:13.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.775+0000 7f81aa949000 20 client.7257 trim_cache size 0 max 16384 2024-04-17T09:58:13.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f81aa949000 1 Processor -- start 2024-04-17T09:58:13.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f81aa949000 1 -- start start 2024-04-17T09:58:13.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f81aa949000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5564776bdcd0 0x5564777094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f81aa949000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55647770a600 0x5564777099f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f81aa949000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647770af20 0x55647771bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f81aa949000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f81a015a490 con 0x55647770a600 2024-04-17T09:58:13.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f81aa949000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f81a01597f0 con 0x55647770af20 2024-04-17T09:58:13.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f81aa949000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f81a0159e40 con 0x5564776bdcd0 2024-04-17T09:58:13.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f8197e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55647770a600 0x5564777099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f8197e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55647770a600 0x5564777099f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55998/0 (socket says 172.21.15.32:55998) 2024-04-17T09:58:13.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f8197e00640 1 -- 172.21.15.32:0/1738785690 learned_addr learned my addr 172.21.15.32:0/1738785690 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:13.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f81a4a00640 1 --2- 172.21.15.32:0/1738785690 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5564776bdcd0 0x5564777094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f81a5400640 1 --2- 172.21.15.32:0/1738785690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647770af20 0x55647771bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f8197e00640 1 -- 172.21.15.32:0/1738785690 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5564776bdcd0 msgr2=0x5564777094b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:13.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f8197e00640 1 --2- 172.21.15.32:0/1738785690 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5564776bdcd0 0x5564777094b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f8197e00640 1 -- 172.21.15.32:0/1738785690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647770af20 msgr2=0x55647771bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:13.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f8197e00640 1 --2- 172.21.15.32:0/1738785690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647770af20 0x55647771bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f8197e00640 1 -- 172.21.15.32:0/1738785690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564774d9080 con 0x55647770a600 2024-04-17T09:58:13.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f81a5400640 1 --2- 172.21.15.32:0/1738785690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647770af20 0x55647771bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:58:13.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f81a4a00640 1 --2- 172.21.15.32:0/1738785690 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5564776bdcd0 0x5564777094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:58:13.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f8197e00640 1 --2- 172.21.15.32:0/1738785690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55647770a600 0x5564777099f0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f81880075e0 tx=0x7f8188010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92273c1ed31d9f6c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:13.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f8194c00640 1 -- 172.21.15.32:0/1738785690 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188022630 con 0x55647770a600 2024-04-17T09:58:13.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f8194c00640 1 -- 172.21.15.32:0/1738785690 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81880227d0 con 0x55647770a600 2024-04-17T09:58:13.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.795+0000 7f8194c00640 1 -- 172.21.15.32:0/1738785690 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188022b80 con 0x55647770a600 2024-04-17T09:58:13.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 1 -- 172.21.15.32:0/1738785690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55647770a600 msgr2=0x5564777099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 1 --2- 172.21.15.32:0/1738785690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55647770a600 0x5564777099f0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f81880075e0 tx=0x7f8188010000 comp rx=0 tx=0).stop 2024-04-17T09:58:13.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 1 -- 172.21.15.32:0/1738785690 shutdown_connections 2024-04-17T09:58:13.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 1 --2- 172.21.15.32:0/1738785690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55647770af20 0x55647771bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 1 --2- 172.21.15.32:0/1738785690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55647770a600 0x5564777099f0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 1 --2- 172.21.15.32:0/1738785690 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5564776bdcd0 0x5564777094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 1 -- 172.21.15.32:0/1738785690 >> 172.21.15.32:0/1738785690 conn(0x5564776eb190 msgr2=0x5564776bcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:13.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 1 -- 172.21.15.32:0/1738785690 shutdown_connections 2024-04-17T09:58:13.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 1 -- 172.21.15.32:0/1738785690 wait complete. 2024-04-17T09:58:13.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 1 -- start start 2024-04-17T09:58:13.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 1 Processor -- start 2024-04-17T09:58:13.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564776bdcd0 0x55647771ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55647770a600 0x556477735820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55647770af20 0x556477735d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55647751a4a0 con 0x55647770af20 2024-04-17T09:58:13.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5564775118c0 con 0x5564776bdcd0 2024-04-17T09:58:13.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5564774a38b0 con 0x55647770a600 2024-04-17T09:58:13.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 10 client.0 _ll_register_callbacks cb 0x7f819800bdd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:58:13.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81a5400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55647770af20 0x556477735d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81a5400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55647770af20 0x556477735d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56012/0 (socket says 172.21.15.32:56012) 2024-04-17T09:58:13.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81a5400640 1 -- 172.21.15.32:0/2119418691 learned_addr learned my addr 172.21.15.32:0/2119418691 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:13.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f8197e00640 1 --2- 172.21.15.32:0/2119418691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55647770a600 0x556477735820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81a4a00640 1 --2- 172.21.15.32:0/2119418691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564776bdcd0 0x55647771ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81a5400640 1 -- 172.21.15.32:0/2119418691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55647770a600 msgr2=0x556477735820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:13.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81a5400640 1 --2- 172.21.15.32:0/2119418691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55647770a600 0x556477735820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81a5400640 1 -- 172.21.15.32:0/2119418691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564776bdcd0 msgr2=0x55647771ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:13.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81a5400640 1 --2- 172.21.15.32:0/2119418691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564776bdcd0 0x55647771ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81a5400640 1 -- 172.21.15.32:0/2119418691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55647758f8f0 con 0x55647770af20 2024-04-17T09:58:13.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81a5400640 1 --2- 172.21.15.32:0/2119418691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55647770af20 0x556477735d60 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f819809a620 tx=0x7f81980b8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5d84eee4127da6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:13.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f818e000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:58:13.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f818e000640 1 -- 172.21.15.32:0/2119418691 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819800b450 con 0x55647770af20 2024-04-17T09:58:13.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f818e000640 1 -- 172.21.15.32:0/2119418691 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f81980039a0 con 0x55647770af20 2024-04-17T09:58:13.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 1 -- 172.21.15.32:0/2119418691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81880227d0 con 0x55647770af20 2024-04-17T09:58:13.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f818e000640 1 -- 172.21.15.32:0/2119418691 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f81980db050 con 0x55647770af20 2024-04-17T09:58:13.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 10 client.7263 fetch_fsmap learned FSMap version 49 2024-04-17T09:58:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 1 -- 172.21.15.32:0/2119418691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x5564771c74c0 con 0x55647770af20 2024-04-17T09:58:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f818e000640 1 -- 172.21.15.32:0/2119418691 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81880227d0 con 0x55647770af20 2024-04-17T09:58:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f818e000640 1 -- 172.21.15.32:0/2119418691 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7f81980de080 con 0x55647770af20 2024-04-17T09:58:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 10 client.7263 fetch_fsmap finished waiting for FSMap version 49 2024-04-17T09:58:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 10 client.7263 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-17T09:58:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 20 client.7263 populate_metadata read hostname 'smithi032' 2024-04-17T09:58:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 1 --2- 172.21.15.32:0/2119418691 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x5564777362a0 0x5564777798b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 4 client.7263 mds_command: new command op to 7011 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-04-17T09:58:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.799+0000 7f81aa949000 1 -- 172.21.15.32:0/2119418691 --> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) -- 0x5564774aaec0 con 0x5564777362a0 2024-04-17T09:58:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81a4a00640 1 --2- 172.21.15.32:0/2119418691 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x5564777362a0 0x5564777798b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81a4a00640 1 --2- 172.21.15.32:0/2119418691 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x5564777362a0 0x5564777798b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfb02ced8fed989d server_cookie=501f7287c0b40c19 in_seq=0 out_seq=0 2024-04-17T09:58:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f818e000640 10 client.7263 ms_handle_connect on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f818e000640 1 -- 172.21.15.32:0/2119418691 <== mds.0 v2:172.21.15.32:6834/2416784965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5564774aaec0 con 0x5564777362a0 2024-04-17T09:58:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f818e000640 10 client.7263 handle_command_reply: tid=0 2024-04-17T09:58:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81aa949000 1 client.7263 shutdown 2024-04-17T09:58:13.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81aa949000 1 -- 172.21.15.32:0/2119418691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55647770af20 msgr2=0x556477735d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81aa949000 1 --2- 172.21.15.32:0/2119418691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55647770af20 0x556477735d60 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f819809a620 tx=0x7f81980b8000 comp rx=0 tx=0).stop 2024-04-17T09:58:13.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81aa949000 1 -- 172.21.15.32:0/2119418691 shutdown_connections 2024-04-17T09:58:13.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81aa949000 1 --2- 172.21.15.32:0/2119418691 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x5564777362a0 0x5564777798b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81aa949000 1 --2- 172.21.15.32:0/2119418691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55647770a600 0x556477735820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81aa949000 1 --2- 172.21.15.32:0/2119418691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55647770af20 0x556477735d60 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81aa949000 1 --2- 172.21.15.32:0/2119418691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564776bdcd0 0x55647771ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81aa949000 1 -- 172.21.15.32:0/2119418691 >> 172.21.15.32:0/2119418691 conn(0x5564776eb190 msgr2=0x5564777084a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:13.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f818e000640 0 client.7263 ms_handle_reset on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:13.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81aa949000 1 -- 172.21.15.32:0/2119418691 shutdown_connections 2024-04-17T09:58:13.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81aa949000 1 -- 172.21.15.32:0/2119418691 wait complete. 2024-04-17T09:58:13.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81aa949000 20 client.7263 trim_cache size 0 max 16384 2024-04-17T09:58:13.809 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:58:13.809 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-17T09:58:13.809 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "5777cf99-ff26-4b66-b163-5f617b417ec1", 2024-04-17T09:58:13.809 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-17T09:58:13.809 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:58:13.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81a7400640 1 -- 172.21.15.32:0/3250510113 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f817c0683b0 msgr2=0x7f817c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81a7400640 1 --2- 172.21.15.32:0/3250510113 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f817c0683b0 0x7f817c06a870 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8188007fd0 tx=0x7f8188007940 comp rx=0 tx=0).stop 2024-04-17T09:58:13.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81a7400640 1 -- 172.21.15.32:0/3250510113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a006f330 msgr2=0x7f81a014dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:13.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81a7400640 1 --2- 172.21.15.32:0/3250510113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a006f330 0x7f81a014dd00 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f819000d5f0 tx=0x7f819000dab0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81a7400640 1 -- 172.21.15.32:0/3250510113 shutdown_connections 2024-04-17T09:58:13.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81a7400640 1 --2- 172.21.15.32:0/3250510113 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f817c0683b0 0x7f817c06a870 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81a7400640 1 --2- 172.21.15.32:0/3250510113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81a00cd870 0x7f81a014e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81a7400640 1 --2- 172.21.15.32:0/3250510113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81a00cfc70 0x7f81a015a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81a7400640 1 --2- 172.21.15.32:0/3250510113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81a006f330 0x7f81a014dd00 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:13.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81a7400640 1 -- 172.21.15.32:0/3250510113 >> 172.21.15.32:0/3250510113 conn(0x7f81a00b2190 msgr2=0x7f81a00b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:13.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81a7400640 1 -- 172.21.15.32:0/3250510113 shutdown_connections 2024-04-17T09:58:13.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:13.803+0000 7f81a7400640 1 -- 172.21.15.32:0/3250510113 wait complete. 2024-04-17T09:58:13.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:58:13.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-17T09:58:14.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.011+0000 7f96b8a00640 1 Processor -- start 2024-04-17T09:58:14.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.011+0000 7f96b8a00640 1 -- start start 2024-04-17T09:58:14.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.011+0000 7f96b8a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b415ac30 0x7f96b406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:14.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.011+0000 7f96b8a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96b415b010 0x7f96b406dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:14.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.011+0000 7f96b8a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f96b40b3d80 0x7f96b4071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:14.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.011+0000 7f96b8a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f96b406f300 con 0x7f96b415b010 2024-04-17T09:58:14.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.011+0000 7f96b8a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f96b406fcd0 con 0x7f96b40b3d80 2024-04-17T09:58:14.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.011+0000 7f96b8a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f96b406e6f0 con 0x7f96b415ac30 2024-04-17T09:58:14.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.011+0000 7f96a9400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96b415b010 0x7f96b406dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:14.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.011+0000 7f96a9400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96b415b010 0x7f96b406dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56016/0 (socket says 172.21.15.32:56016) 2024-04-17T09:58:14.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.011+0000 7f96a9400640 1 -- 172.21.15.32:0/1563135342 learned_addr learned my addr 172.21.15.32:0/1563135342 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:14.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.011+0000 7f96b2800640 1 --2- 172.21.15.32:0/1563135342 >> v2:172.21.15.144:3300/0 conn(0x7f96b40b3d80 0x7f96b4071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.011+0000 7f96b1e00640 1 --2- 172.21.15.32:0/1563135342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b415ac30 0x7f96b406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.011+0000 7f96a9400640 1 -- 172.21.15.32:0/1563135342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b415ac30 msgr2=0x7f96b406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.011+0000 7f96a9400640 1 --2- 172.21.15.32:0/1563135342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b415ac30 0x7f96b406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.011+0000 7f96a9400640 1 -- 172.21.15.32:0/1563135342 >> v2:172.21.15.144:3300/0 conn(0x7f96b40b3d80 msgr2=0x7f96b4071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.011+0000 7f96a9400640 1 --2- 172.21.15.32:0/1563135342 >> v2:172.21.15.144:3300/0 conn(0x7f96b40b3d80 0x7f96b4071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.011+0000 7f96a9400640 1 -- 172.21.15.32:0/1563135342 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b406e9d0 con 0x7f96b415b010 2024-04-17T09:58:14.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.015+0000 7f96a9400640 1 --2- 172.21.15.32:0/1563135342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96b415b010 0x7f96b406dea0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f96b40cff50 tx=0x7f969c01e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8455c56324cf0ae7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:14.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.015+0000 7f96b1400640 1 -- 172.21.15.32:0/1563135342 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c0ab650 con 0x7f96b415b010 2024-04-17T09:58:14.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.015+0000 7f96b1400640 1 -- 172.21.15.32:0/1563135342 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f969c0ab7f0 con 0x7f96b415b010 2024-04-17T09:58:14.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.015+0000 7f96b1400640 1 -- 172.21.15.32:0/1563135342 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c0abbc0 con 0x7f96b415b010 2024-04-17T09:58:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.015+0000 7f96b8a00640 1 -- 172.21.15.32:0/1563135342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96b415b010 msgr2=0x7f96b406dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.015+0000 7f96b8a00640 1 --2- 172.21.15.32:0/1563135342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96b415b010 0x7f96b406dea0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f96b40cff50 tx=0x7f969c01e730 comp rx=0 tx=0).stop 2024-04-17T09:58:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.015+0000 7f96b8a00640 1 -- 172.21.15.32:0/1563135342 shutdown_connections 2024-04-17T09:58:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.015+0000 7f96b8a00640 1 --2- 172.21.15.32:0/1563135342 >> v2:172.21.15.144:3300/0 conn(0x7f96b40b3d80 0x7f96b4071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.015+0000 7f96b8a00640 1 --2- 172.21.15.32:0/1563135342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96b415b010 0x7f96b406dea0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.015+0000 7f96b8a00640 1 --2- 172.21.15.32:0/1563135342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b415ac30 0x7f96b406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.015+0000 7f96b8a00640 1 -- 172.21.15.32:0/1563135342 >> 172.21.15.32:0/1563135342 conn(0x7f96b40b2120 msgr2=0x7f96b40b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.015+0000 7f96b8a00640 1 -- 172.21.15.32:0/1563135342 shutdown_connections 2024-04-17T09:58:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.015+0000 7f96b8a00640 1 -- 172.21.15.32:0/1563135342 wait complete. 2024-04-17T09:58:14.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.015+0000 7f96b8a00640 1 Processor -- start 2024-04-17T09:58:14.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.015+0000 7f96b8a00640 1 -- start start 2024-04-17T09:58:14.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96b8a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b40b3d80 0x7f96b4153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:14.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96b8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96b415ac30 0x7f96b41f82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:14.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96b8a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b415b010 0x7f96b40d0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:14.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96b8a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f96b40cdb40 con 0x7f96b415ac30 2024-04-17T09:58:14.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96b8a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96b40cd9c0 con 0x7f96b415b010 2024-04-17T09:58:14.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96b8a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f96b40cdcc0 con 0x7f96b40b3d80 2024-04-17T09:58:14.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96a9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96b415ac30 0x7f96b41f82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:14.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96a9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96b415ac30 0x7f96b41f82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56018/0 (socket says 172.21.15.32:56018) 2024-04-17T09:58:14.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96a9400640 1 -- 172.21.15.32:0/2143675559 learned_addr learned my addr 172.21.15.32:0/2143675559 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:14.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96b1e00640 1 --2- 172.21.15.32:0/2143675559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b40b3d80 0x7f96b4153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:14.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96a9400640 1 -- 172.21.15.32:0/2143675559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b40b3d80 msgr2=0x7f96b4153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:14.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96a9400640 1 --2- 172.21.15.32:0/2143675559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b40b3d80 0x7f96b4153f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96b2800640 1 --2- 172.21.15.32:0/2143675559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b415b010 0x7f96b40d0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:14.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96a9400640 1 -- 172.21.15.32:0/2143675559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b415b010 msgr2=0x7f96b40d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:14.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96a9400640 1 --2- 172.21.15.32:0/2143675559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b415b010 0x7f96b40d0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96a9400640 1 -- 172.21.15.32:0/2143675559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b41f8800 con 0x7f96b415ac30 2024-04-17T09:58:14.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96b1e00640 1 --2- 172.21.15.32:0/2143675559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b40b3d80 0x7f96b4153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:58:14.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96b2800640 1 --2- 172.21.15.32:0/2143675559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b415b010 0x7f96b40d0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:58:14.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96a9400640 1 --2- 172.21.15.32:0/2143675559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96b415ac30 0x7f96b41f82c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f969c024d00 tx=0x7f969c01b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:14.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96aaa00640 1 -- 172.21.15.32:0/2143675559 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c0b3800 con 0x7f96b415ac30 2024-04-17T09:58:14.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96aaa00640 1 -- 172.21.15.32:0/2143675559 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f969c0b39a0 con 0x7f96b415ac30 2024-04-17T09:58:14.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96aaa00640 1 -- 172.21.15.32:0/2143675559 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c0b3d50 con 0x7f96b415ac30 2024-04-17T09:58:14.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96b8a00640 1 -- 172.21.15.32:0/2143675559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b40d0a70 con 0x7f96b415ac30 2024-04-17T09:58:14.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.019+0000 7f96b8a00640 1 -- 172.21.15.32:0/2143675559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b40cf770 con 0x7f96b415ac30 2024-04-17T09:58:14.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96aaa00640 1 -- 172.21.15.32:0/2143675559 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f969c0aa3b0 con 0x7f96b415ac30 2024-04-17T09:58:14.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96aaa00640 1 --2- 172.21.15.32:0/2143675559 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9690068380 0x7f969006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:14.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96aaa00640 1 -- 172.21.15.32:0/2143675559 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f969c120f20 con 0x7f96b415ac30 2024-04-17T09:58:14.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96b1e00640 1 --2- 172.21.15.32:0/2143675559 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9690068380 0x7f969006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:14.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96bbe64000 1 Processor -- start 2024-04-17T09:58:14.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96bbe64000 1 -- start start 2024-04-17T09:58:14.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96b1e00640 1 --2- 172.21.15.32:0/2143675559 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9690068380 0x7f969006a840 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009840 tx=0x7f96a4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:14.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96bbe64000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2d9c3260 0x55da2d9c3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:14.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96bbe64000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9c3b80 0x55da2d976460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:14.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96bbe64000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da2d9769a0 0x55da2d976d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:14.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96bbe64000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55da2d7c84f0 con 0x55da2d9c3b80 2024-04-17T09:58:14.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96bbe64000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55da2d7816c0 con 0x55da2d9c3260 2024-04-17T09:58:14.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96bbe64000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55da2d7c44b0 con 0x55da2d9769a0 2024-04-17T09:58:14.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96a9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9c3b80 0x55da2d976460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:14.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96a9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9c3b80 0x55da2d976460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56028/0 (socket says 172.21.15.32:56028) 2024-04-17T09:58:14.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96a9400640 1 -- 172.21.15.32:0/1249268612 learned_addr learned my addr 172.21.15.32:0/1249268612 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:14.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96b2800640 1 --2- 172.21.15.32:0/1249268612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da2d9769a0 0x55da2d976d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:14.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96b1e00640 1 --2- 172.21.15.32:0/1249268612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2d9c3260 0x55da2d9c3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:14.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96a9400640 1 -- 172.21.15.32:0/1249268612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da2d9769a0 msgr2=0x55da2d976d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:14.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96a9400640 1 --2- 172.21.15.32:0/1249268612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da2d9769a0 0x55da2d976d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96a9400640 1 -- 172.21.15.32:0/1249268612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2d9c3260 msgr2=0x55da2d9c3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:14.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96a9400640 1 --2- 172.21.15.32:0/1249268612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2d9c3260 0x55da2d9c3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96a9400640 1 -- 172.21.15.32:0/1249268612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da2d80d8e0 con 0x55da2d9c3b80 2024-04-17T09:58:14.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96a9400640 1 --2- 172.21.15.32:0/1249268612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9c3b80 0x55da2d976460 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f969c0f0a60 tx=0x7f969c0f07f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae107a04bf674d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:14.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96a2a00640 1 -- 172.21.15.32:0/1249268612 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c0c2040 con 0x55da2d9c3b80 2024-04-17T09:58:14.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96a2a00640 1 -- 172.21.15.32:0/1249268612 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f969c0ab740 con 0x55da2d9c3b80 2024-04-17T09:58:14.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.023+0000 7f96a2a00640 1 -- 172.21.15.32:0/1249268612 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c0ed3d0 con 0x55da2d9c3b80 2024-04-17T09:58:14.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96bbe64000 1 -- 172.21.15.32:0/1249268612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9c3b80 msgr2=0x55da2d976460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:14.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96bbe64000 1 --2- 172.21.15.32:0/1249268612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9c3b80 0x55da2d976460 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f969c0f0a60 tx=0x7f969c0f07f0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96bbe64000 1 -- 172.21.15.32:0/1249268612 shutdown_connections 2024-04-17T09:58:14.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96bbe64000 1 --2- 172.21.15.32:0/1249268612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da2d9769a0 0x55da2d976d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96bbe64000 1 --2- 172.21.15.32:0/1249268612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9c3b80 0x55da2d976460 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96bbe64000 1 --2- 172.21.15.32:0/1249268612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2d9c3260 0x55da2d9c3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96bbe64000 1 -- 172.21.15.32:0/1249268612 >> 172.21.15.32:0/1249268612 conn(0x55da2d9a4000 msgr2=0x55da2d9a63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:14.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96bbe64000 1 -- 172.21.15.32:0/1249268612 shutdown_connections 2024-04-17T09:58:14.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96bbe64000 1 -- 172.21.15.32:0/1249268612 wait complete. 2024-04-17T09:58:14.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96bbe64000 1 -- start start 2024-04-17T09:58:14.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96bbe64000 1 Processor -- start 2024-04-17T09:58:14.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96bbe64000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9769a0 0x55da2d97c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:14.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96bbe64000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da2d9c3260 0x55da2d97c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:14.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96bbe64000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2d9c3b80 0x55da2d97caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:14.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96bbe64000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55da2d8497b0 con 0x55da2d9769a0 2024-04-17T09:58:14.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96bbe64000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55da2d883d00 con 0x55da2d9c3b80 2024-04-17T09:58:14.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96bbe64000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55da2d78d2c0 con 0x55da2d9c3260 2024-04-17T09:58:14.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96bbe64000 10 client.0 _ll_register_callbacks cb 0x55da2d79d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:58:14.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96b1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9769a0 0x55da2d97c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:14.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96b1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9769a0 0x55da2d97c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56038/0 (socket says 172.21.15.32:56038) 2024-04-17T09:58:14.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96b1e00640 1 -- 172.21.15.32:0/872028646 learned_addr learned my addr 172.21.15.32:0/872028646 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:14.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96a9400640 1 --2- 172.21.15.32:0/872028646 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da2d9c3260 0x55da2d97c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:14.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96b2800640 1 --2- 172.21.15.32:0/872028646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2d9c3b80 0x55da2d97caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:14.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96b1e00640 1 -- 172.21.15.32:0/872028646 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da2d9c3260 msgr2=0x55da2d97c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:14.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96b1e00640 1 --2- 172.21.15.32:0/872028646 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da2d9c3260 0x55da2d97c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96b1e00640 1 -- 172.21.15.32:0/872028646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2d9c3b80 msgr2=0x55da2d97caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:14.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96b1e00640 1 --2- 172.21.15.32:0/872028646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2d9c3b80 0x55da2d97caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96b1e00640 1 -- 172.21.15.32:0/872028646 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da2d7a0490 con 0x55da2d9769a0 2024-04-17T09:58:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96b1e00640 1 --2- 172.21.15.32:0/872028646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9769a0 0x55da2d97c070 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009370 tx=0x7f96a4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944d97fa93358193 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f9697400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:58:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f9697400640 1 -- 172.21.15.32:0/872028646 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4028060 con 0x55da2d9769a0 2024-04-17T09:58:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f9697400640 1 -- 172.21.15.32:0/872028646 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f96a4023460 con 0x55da2d9769a0 2024-04-17T09:58:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96bbe64000 1 -- 172.21.15.32:0/872028646 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f969c0ab740 con 0x55da2d9769a0 2024-04-17T09:58:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f9697400640 1 -- 172.21.15.32:0/872028646 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f96a402e050 con 0x55da2d9769a0 2024-04-17T09:58:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96bbe64000 10 client.7275 fetch_fsmap learned FSMap version 49 2024-04-17T09:58:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f96bbe64000 1 -- 172.21.15.32:0/872028646 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x55da2d793eb0 con 0x55da2d9769a0 2024-04-17T09:58:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f9697400640 1 -- 172.21.15.32:0/872028646 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f969c0ab740 con 0x55da2d9769a0 2024-04-17T09:58:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.027+0000 7f9697400640 1 -- 172.21.15.32:0/872028646 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7f96a4031020 con 0x55da2d9769a0 2024-04-17T09:58:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f96bbe64000 10 client.7275 fetch_fsmap finished waiting for FSMap version 49 2024-04-17T09:58:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f96bbe64000 10 client.7275 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-17T09:58:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f96bbe64000 20 client.7275 populate_metadata read hostname 'smithi032' 2024-04-17T09:58:14.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f96bbe64000 1 --2- 172.21.15.32:0/872028646 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55da2d9d2500 0x55da2d9d48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:14.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f96bbe64000 4 client.7275 mds_command: new command op to 7011 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:58:14.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f96bbe64000 1 -- 172.21.15.32:0/872028646 --> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da2d761dd0 con 0x55da2d9d2500 2024-04-17T09:58:14.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f96a9400640 1 --2- 172.21.15.32:0/872028646 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55da2d9d2500 0x55da2d9d48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:14.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f96a9400640 1 --2- 172.21.15.32:0/872028646 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55da2d9d2500 0x55da2d9d48f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=469a91e2815661fe server_cookie=29f2a3d4c9f9c740 in_seq=0 out_seq=0 2024-04-17T09:58:14.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f9697400640 10 client.7275 ms_handle_connect on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:14.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f9697400640 1 -- 172.21.15.32:0/872028646 <== mds.0 v2:172.21.15.32:6834/2416784965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da2d761dd0 con 0x55da2d9d2500 2024-04-17T09:58:14.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f9697400640 10 client.7275 handle_command_reply: tid=0 2024-04-17T09:58:14.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f96bbe64000 1 client.7275 shutdown 2024-04-17T09:58:14.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f96bbe64000 1 -- 172.21.15.32:0/872028646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9769a0 msgr2=0x55da2d97c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:14.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f96bbe64000 1 --2- 172.21.15.32:0/872028646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9769a0 0x55da2d97c070 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009370 tx=0x7f96a4004740 comp rx=0 tx=0).stop 2024-04-17T09:58:14.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f96bbe64000 1 -- 172.21.15.32:0/872028646 shutdown_connections 2024-04-17T09:58:14.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f96bbe64000 1 --2- 172.21.15.32:0/872028646 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55da2d9d2500 0x55da2d9d48f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f96bbe64000 1 --2- 172.21.15.32:0/872028646 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da2d9c3260 0x55da2d97c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f96bbe64000 1 --2- 172.21.15.32:0/872028646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2d9c3b80 0x55da2d97caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f96bbe64000 1 --2- 172.21.15.32:0/872028646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9769a0 0x55da2d97c070 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f9697400640 0 client.7275 ms_handle_reset on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:14.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f96bbe64000 1 -- 172.21.15.32:0/872028646 >> 172.21.15.32:0/872028646 conn(0x55da2d9a4000 msgr2=0x55da2d9a63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:14.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f96bbe64000 1 -- 172.21.15.32:0/872028646 shutdown_connections 2024-04-17T09:58:14.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f96bbe64000 1 -- 172.21.15.32:0/872028646 wait complete. 2024-04-17T09:58:14.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.031+0000 7f96bbe64000 20 client.7275 trim_cache size 0 max 16384 2024-04-17T09:58:14.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.047+0000 7f96bbe64000 1 Processor -- start 2024-04-17T09:58:14.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.047+0000 7f96bbe64000 1 -- start start 2024-04-17T09:58:14.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96bbe64000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9769a0 0x55da2d975b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:14.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96bbe64000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2d9c3260 0x55da2d9760a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:14.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96bbe64000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da2d9c3b80 0x55da2d9d48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:14.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96bbe64000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f96b4203180 con 0x55da2d9769a0 2024-04-17T09:58:14.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96bbe64000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41f9360 con 0x55da2d9c3260 2024-04-17T09:58:14.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96bbe64000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f96b4202b40 con 0x55da2d9c3b80 2024-04-17T09:58:14.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96b1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9769a0 0x55da2d975b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:14.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96b1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9769a0 0x55da2d975b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56048/0 (socket says 172.21.15.32:56048) 2024-04-17T09:58:14.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96b1e00640 1 -- 172.21.15.32:0/1005184171 learned_addr learned my addr 172.21.15.32:0/1005184171 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:14.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96b2800640 1 --2- 172.21.15.32:0/1005184171 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da2d9c3b80 0x55da2d9d48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:14.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96a9400640 1 --2- 172.21.15.32:0/1005184171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2d9c3260 0x55da2d9760a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:14.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96b1e00640 1 -- 172.21.15.32:0/1005184171 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da2d9c3b80 msgr2=0x55da2d9d48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:14.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96b1e00640 1 --2- 172.21.15.32:0/1005184171 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da2d9c3b80 0x55da2d9d48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96b1e00640 1 -- 172.21.15.32:0/1005184171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2d9c3260 msgr2=0x55da2d9760a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:14.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96b1e00640 1 --2- 172.21.15.32:0/1005184171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2d9c3260 0x55da2d9760a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96b1e00640 1 -- 172.21.15.32:0/1005184171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da2d782a20 con 0x55da2d9769a0 2024-04-17T09:58:14.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96a9400640 1 --2- 172.21.15.32:0/1005184171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2d9c3260 0x55da2d9760a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:58:14.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96b1e00640 1 --2- 172.21.15.32:0/1005184171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9769a0 0x55da2d975b60 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f96a4003fd0 tx=0x7f96a4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0cdb18d9d675dc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:14.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96a2000640 1 -- 172.21.15.32:0/1005184171 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a400f040 con 0x55da2d9769a0 2024-04-17T09:58:14.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96a2000640 1 -- 172.21.15.32:0/1005184171 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a4023460 con 0x55da2d9769a0 2024-04-17T09:58:14.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96a2000640 1 -- 172.21.15.32:0/1005184171 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a40238f0 con 0x55da2d9769a0 2024-04-17T09:58:14.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96bbe64000 1 -- 172.21.15.32:0/1005184171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9769a0 msgr2=0x55da2d975b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:14.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96bbe64000 1 --2- 172.21.15.32:0/1005184171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9769a0 0x55da2d975b60 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f96a4003fd0 tx=0x7f96a4020000 comp rx=0 tx=0).stop 2024-04-17T09:58:14.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96bbe64000 1 -- 172.21.15.32:0/1005184171 shutdown_connections 2024-04-17T09:58:14.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96bbe64000 1 --2- 172.21.15.32:0/1005184171 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da2d9c3b80 0x55da2d9d48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96bbe64000 1 --2- 172.21.15.32:0/1005184171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2d9c3260 0x55da2d9760a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96bbe64000 1 --2- 172.21.15.32:0/1005184171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9769a0 0x55da2d975b60 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96bbe64000 1 -- 172.21.15.32:0/1005184171 >> 172.21.15.32:0/1005184171 conn(0x55da2d9a4000 msgr2=0x55da2d975450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:14.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96bbe64000 1 -- 172.21.15.32:0/1005184171 shutdown_connections 2024-04-17T09:58:14.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96bbe64000 1 -- 172.21.15.32:0/1005184171 wait complete. 2024-04-17T09:58:14.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96bbe64000 1 -- start start 2024-04-17T09:58:14.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.051+0000 7f96bbe64000 1 Processor -- start 2024-04-17T09:58:14.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9769a0 0x55da2d9d7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:14.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2d9c3260 0x55da2d9ee480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:14.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da2d9c3b80 0x55da2d9ee9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:14.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55da2d76c050 con 0x55da2d9769a0 2024-04-17T09:58:14.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55da2d78a570 con 0x55da2d9c3260 2024-04-17T09:58:14.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55da2d7c7800 con 0x55da2d9c3b80 2024-04-17T09:58:14.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 10 client.0 _ll_register_callbacks cb 0x7f96a4009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:58:14.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96b1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9769a0 0x55da2d9d7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:14.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96b1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9769a0 0x55da2d9d7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56058/0 (socket says 172.21.15.32:56058) 2024-04-17T09:58:14.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96b1e00640 1 -- 172.21.15.32:0/3421702682 learned_addr learned my addr 172.21.15.32:0/3421702682 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:14.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96a9400640 1 --2- 172.21.15.32:0/3421702682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2d9c3260 0x55da2d9ee480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96b2800640 1 --2- 172.21.15.32:0/3421702682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da2d9c3b80 0x55da2d9ee9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96b1e00640 1 -- 172.21.15.32:0/3421702682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da2d9c3b80 msgr2=0x55da2d9ee9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96b1e00640 1 --2- 172.21.15.32:0/3421702682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da2d9c3b80 0x55da2d9ee9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96b1e00640 1 -- 172.21.15.32:0/3421702682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2d9c3260 msgr2=0x55da2d9ee480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96b1e00640 1 --2- 172.21.15.32:0/3421702682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2d9c3260 0x55da2d9ee480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96b1e00640 1 -- 172.21.15.32:0/3421702682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da2d849c40 con 0x55da2d9769a0 2024-04-17T09:58:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96b1e00640 1 --2- 172.21.15.32:0/3421702682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9769a0 0x55da2d9d7790 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009320 tx=0x7f96a4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c211b4ff28d29d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f9697400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:58:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f9697400640 1 -- 172.21.15.32:0/3421702682 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4028040 con 0x55da2d9769a0 2024-04-17T09:58:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f9697400640 1 -- 172.21.15.32:0/3421702682 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f96a4018780 con 0x55da2d9769a0 2024-04-17T09:58:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 1 -- 172.21.15.32:0/3421702682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96a4023460 con 0x55da2d9769a0 2024-04-17T09:58:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f9697400640 1 -- 172.21.15.32:0/3421702682 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f96a40189c0 con 0x55da2d9769a0 2024-04-17T09:58:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 10 client.7281 fetch_fsmap learned FSMap version 49 2024-04-17T09:58:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 1 -- 172.21.15.32:0/3421702682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x55da2d76d600 con 0x55da2d9769a0 2024-04-17T09:58:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f9697400640 1 -- 172.21.15.32:0/3421702682 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a4023460 con 0x55da2d9769a0 2024-04-17T09:58:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f9697400640 1 -- 172.21.15.32:0/3421702682 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7f96a4031070 con 0x55da2d9769a0 2024-04-17T09:58:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 10 client.7281 fetch_fsmap finished waiting for FSMap version 49 2024-04-17T09:58:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 10 client.7281 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-17T09:58:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 20 client.7281 populate_metadata read hostname 'smithi032' 2024-04-17T09:58:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 1 --2- 172.21.15.32:0/3421702682 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55da2d9eef00 0x55da2da31ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 4 client.7281 mds_command: new command op to 7011 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T09:58:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 1 -- 172.21.15.32:0/3421702682 --> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55da2d7d5f10 con 0x55da2d9eef00 2024-04-17T09:58:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96a9400640 1 --2- 172.21.15.32:0/3421702682 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55da2d9eef00 0x55da2da31ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96a9400640 1 --2- 172.21.15.32:0/3421702682 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55da2d9eef00 0x55da2da31ee0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22bbee6f35566003 server_cookie=2210fe640b175b26 in_seq=0 out_seq=0 2024-04-17T09:58:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f9697400640 10 client.7281 ms_handle_connect on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f9697400640 1 -- 172.21.15.32:0/3421702682 <== mds.0 v2:172.21.15.32:6834/2416784965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x55da2d7d5f10 con 0x55da2d9eef00 2024-04-17T09:58:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f9697400640 10 client.7281 handle_command_reply: tid=0 2024-04-17T09:58:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 1 client.7281 shutdown 2024-04-17T09:58:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 1 -- 172.21.15.32:0/3421702682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9769a0 msgr2=0x55da2d9d7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 1 --2- 172.21.15.32:0/3421702682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9769a0 0x55da2d9d7790 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009320 tx=0x7f96a4012000 comp rx=0 tx=0).stop 2024-04-17T09:58:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 1 -- 172.21.15.32:0/3421702682 shutdown_connections 2024-04-17T09:58:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 1 --2- 172.21.15.32:0/3421702682 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55da2d9eef00 0x55da2da31ee0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 1 --2- 172.21.15.32:0/3421702682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55da2d9c3b80 0x55da2d9ee9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 1 --2- 172.21.15.32:0/3421702682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55da2d9c3260 0x55da2d9ee480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 1 --2- 172.21.15.32:0/3421702682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da2d9769a0 0x55da2d9d7790 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f9697400640 0 client.7281 ms_handle_reset on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 1 -- 172.21.15.32:0/3421702682 >> 172.21.15.32:0/3421702682 conn(0x55da2d9a4000 msgr2=0x55da2d975b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 1 -- 172.21.15.32:0/3421702682 shutdown_connections 2024-04-17T09:58:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 1 -- 172.21.15.32:0/3421702682 wait complete. 2024-04-17T09:58:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.055+0000 7f96bbe64000 20 client.7281 trim_cache size 0 max 16384 2024-04-17T09:58:14.063 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:58:14.063 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T09:58:14.063 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-17T09:58:14.063 INFO:teuthology.orchestra.run.smithi032.stdout: "5777cf99-ff26-4b66-b163-5f617b417ec1": { 2024-04-17T09:58:14.063 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/subdir", 2024-04-17T09:58:14.063 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "5777cf99-ff26-4b66-b163-5f617b417ec1", 2024-04-17T09:58:14.063 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,repair" 2024-04-17T09:58:14.063 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:58:14.063 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:58:14.063 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:58:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.059+0000 7f96b8a00640 1 -- 172.21.15.32:0/2143675559 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9690068380 msgr2=0x7f969006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.059+0000 7f96b8a00640 1 --2- 172.21.15.32:0/2143675559 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9690068380 0x7f969006a840 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009840 tx=0x7f96a4008040 comp rx=0 tx=0).stop 2024-04-17T09:58:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.059+0000 7f96b8a00640 1 -- 172.21.15.32:0/2143675559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96b415ac30 msgr2=0x7f96b41f82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.059+0000 7f96b8a00640 1 --2- 172.21.15.32:0/2143675559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96b415ac30 0x7f96b41f82c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f969c024d00 tx=0x7f969c01b4d0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.059+0000 7f96b8a00640 1 -- 172.21.15.32:0/2143675559 shutdown_connections 2024-04-17T09:58:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.059+0000 7f96b8a00640 1 --2- 172.21.15.32:0/2143675559 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9690068380 0x7f969006a840 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.059+0000 7f96b8a00640 1 --2- 172.21.15.32:0/2143675559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b415b010 0x7f96b40d0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.059+0000 7f96b8a00640 1 --2- 172.21.15.32:0/2143675559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96b415ac30 0x7f96b41f82c0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.059+0000 7f96b8a00640 1 --2- 172.21.15.32:0/2143675559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b40b3d80 0x7f96b4153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.059+0000 7f96b8a00640 1 -- 172.21.15.32:0/2143675559 >> 172.21.15.32:0/2143675559 conn(0x7f96b40b2120 msgr2=0x7f96b415da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.059+0000 7f96b8a00640 1 -- 172.21.15.32:0/2143675559 shutdown_connections 2024-04-17T09:58:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:14.059+0000 7f96b8a00640 1 -- 172.21.15.32:0/2143675559 wait complete. 2024-04-17T09:58:14.078 INFO:tasks.cephfs.filesystem:scrub status for tag:5777cf99-ff26-4b66-b163-5f617b417ec1 - {'path': '/subdir', 'tag': '5777cf99-ff26-4b66-b163-5f617b417ec1', 'options': 'recursive,repair'} 2024-04-17T09:58:43.755 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:58:43.759 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:58:44.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:58:44.079 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-17T09:58:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.234+0000 7ffbdbe00640 1 Processor -- start 2024-04-17T09:58:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.234+0000 7ffbdbe00640 1 -- start start 2024-04-17T09:58:44.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.234+0000 7ffbdbe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffbdc144e90 0x7ffbdc06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.234+0000 7ffbdbe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbdc06f4a0 0x7ffbdc0cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.234+0000 7ffbdbe00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbdc071e90 con 0x7ffbdc0cfc20 2024-04-17T09:58:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.234+0000 7ffbdbe00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ffbdc145270 con 0x7ffbdc144e90 2024-04-17T09:58:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.234+0000 7ffbdbe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffbdc072170 con 0x7ffbdc06f4a0 2024-04-17T09:58:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.234+0000 7ffbdaa00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffbdc144e90 0x7ffbdc06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.234+0000 7ffbdaa00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffbdc144e90 0x7ffbdc06ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38720/0 (socket says 172.21.15.32:38720) 2024-04-17T09:58:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.234+0000 7ffbdaa00640 1 -- 172.21.15.32:0/447774765 learned_addr learned my addr 172.21.15.32:0/447774765 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.234+0000 7ffbdaa00640 1 -- 172.21.15.32:0/447774765 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbdc06f4a0 msgr2=0x7ffbdc0cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.234+0000 7ffbd2000640 1 --2- 172.21.15.32:0/447774765 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbdc06f4a0 0x7ffbdc0cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.234+0000 7ffbdaa00640 1 --2- 172.21.15.32:0/447774765 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbdc06f4a0 0x7ffbdc0cf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.234+0000 7ffbdaa00640 1 -- 172.21.15.32:0/447774765 >> v1:172.21.15.144:6789/0 conn(0x7ffbdc0cfc20 legacy=0x7ffbdc15f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.234+0000 7ffbdaa00640 1 -- 172.21.15.32:0/447774765 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbdc072450 con 0x7ffbdc144e90 2024-04-17T09:58:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.234+0000 7ffbd2000640 1 --2- 172.21.15.32:0/447774765 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbdc06f4a0 0x7ffbdc0cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:58:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.238+0000 7ffbdaa00640 1 --2- 172.21.15.32:0/447774765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbdc144e90 0x7ffbdc06ef60 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009870 tx=0x7ffbcc01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ccaa28483daa49 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.238+0000 7ffbda000640 1 -- 172.21.15.32:0/447774765 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0b0070 con 0x7ffbdc144e90 2024-04-17T09:58:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.238+0000 7ffbda000640 1 -- 172.21.15.32:0/447774765 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbcc0ab600 con 0x7ffbdc144e90 2024-04-17T09:58:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.238+0000 7ffbda000640 1 -- 172.21.15.32:0/447774765 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0ab9d0 con 0x7ffbdc144e90 2024-04-17T09:58:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.238+0000 7ffbdbe00640 1 -- 172.21.15.32:0/447774765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbdc144e90 msgr2=0x7ffbdc06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.238+0000 7ffbdbe00640 1 --2- 172.21.15.32:0/447774765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbdc144e90 0x7ffbdc06ef60 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009870 tx=0x7ffbcc01c440 comp rx=0 tx=0).stop 2024-04-17T09:58:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.238+0000 7ffbdbe00640 1 -- 172.21.15.32:0/447774765 shutdown_connections 2024-04-17T09:58:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.238+0000 7ffbdbe00640 1 --2- 172.21.15.32:0/447774765 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbdc06f4a0 0x7ffbdc0cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.238+0000 7ffbdbe00640 1 --2- 172.21.15.32:0/447774765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbdc144e90 0x7ffbdc06ef60 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.238+0000 7ffbdbe00640 1 -- 172.21.15.32:0/447774765 >> 172.21.15.32:0/447774765 conn(0x7ffbdc067d70 msgr2=0x7ffbdc06ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.238+0000 7ffbdbe00640 1 -- 172.21.15.32:0/447774765 shutdown_connections 2024-04-17T09:58:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.238+0000 7ffbdbe00640 1 -- 172.21.15.32:0/447774765 wait complete. 2024-04-17T09:58:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.238+0000 7ffbdbe00640 1 Processor -- start 2024-04-17T09:58:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.238+0000 7ffbdbe00640 1 -- start start 2024-04-17T09:58:44.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbdbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbdc06f4a0 0x7ffbdc0af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbdbe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbdc0cfc20 0x7ffbdc0abc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbdbe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbdc144e90 0x7ffbdc0ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbdbe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc168370 con 0x7ffbdc06f4a0 2024-04-17T09:58:44.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbdbe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc1681f0 con 0x7ffbdc0cfc20 2024-04-17T09:58:44.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbdbe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffbdc1684f0 con 0x7ffbdc144e90 2024-04-17T09:58:44.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbdaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbdc06f4a0 0x7ffbdc0af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbdaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbdc06f4a0 0x7ffbdc0af930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38724/0 (socket says 172.21.15.32:38724) 2024-04-17T09:58:44.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbdaa00640 1 -- 172.21.15.32:0/2147818185 learned_addr learned my addr 172.21.15.32:0/2147818185 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:44.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbd2000640 1 --2- 172.21.15.32:0/2147818185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbdc0cfc20 0x7ffbdc0abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbdb400640 1 --2- 172.21.15.32:0/2147818185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbdc144e90 0x7ffbdc0ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbdaa00640 1 -- 172.21.15.32:0/2147818185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbdc144e90 msgr2=0x7ffbdc0ac1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:44.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbdaa00640 1 --2- 172.21.15.32:0/2147818185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbdc144e90 0x7ffbdc0ac1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbdaa00640 1 -- 172.21.15.32:0/2147818185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbdc0cfc20 msgr2=0x7ffbdc0abc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:44.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbdaa00640 1 --2- 172.21.15.32:0/2147818185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbdc0cfc20 0x7ffbdc0abc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbdaa00640 1 -- 172.21.15.32:0/2147818185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbdc074db0 con 0x7ffbdc06f4a0 2024-04-17T09:58:44.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbd2000640 1 --2- 172.21.15.32:0/2147818185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbdc0cfc20 0x7ffbdc0abc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:58:44.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbdb400640 1 --2- 172.21.15.32:0/2147818185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbdc144e90 0x7ffbdc0ac1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:58:44.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbdaa00640 1 --2- 172.21.15.32:0/2147818185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbdc06f4a0 0x7ffbdc0af930 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0ae040 tx=0x7ffbcc0191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:44.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbd3e00640 1 -- 172.21.15.32:0/2147818185 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0b0040 con 0x7ffbdc06f4a0 2024-04-17T09:58:44.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbd3e00640 1 -- 172.21.15.32:0/2147818185 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbcc0b9070 con 0x7ffbdc06f4a0 2024-04-17T09:58:44.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbd3e00640 1 -- 172.21.15.32:0/2147818185 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0b4880 con 0x7ffbdc06f4a0 2024-04-17T09:58:44.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbdbe00640 1 -- 172.21.15.32:0/2147818185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbdc12e8b0 con 0x7ffbdc06f4a0 2024-04-17T09:58:44.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.242+0000 7ffbdbe00640 1 -- 172.21.15.32:0/2147818185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbdc12ee70 con 0x7ffbdc06f4a0 2024-04-17T09:58:44.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbd3e00640 1 -- 172.21.15.32:0/2147818185 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ffbcc0ab4c0 con 0x7ffbdc06f4a0 2024-04-17T09:58:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbd3e00640 1 --2- 172.21.15.32:0/2147818185 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffbb8068380 0x7ffbb806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbd3e00640 1 -- 172.21.15.32:0/2147818185 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7ffbcc120800 con 0x7ffbdc06f4a0 2024-04-17T09:58:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbd2000640 1 --2- 172.21.15.32:0/2147818185 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffbb8068380 0x7ffbb806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbe5335000 1 Processor -- start 2024-04-17T09:58:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbe5335000 1 -- start start 2024-04-17T09:58:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbd2000640 1 --2- 172.21.15.32:0/2147818185 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffbb8068380 0x7ffbb806a840 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc06f430 tx=0x7ffbc4007530 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbe5335000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725260 0x55fa51725640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbe5335000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa51725b80 0x55fa516d8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbe5335000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa516d89a0 0x55fa516d8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbe5335000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55fa5152a4f0 con 0x55fa51725260 2024-04-17T09:58:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbe5335000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fa514e36c0 con 0x55fa516d89a0 2024-04-17T09:58:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbe5335000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55fa515264b0 con 0x55fa51725b80 2024-04-17T09:58:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbdaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725260 0x55fa51725640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbdaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725260 0x55fa51725640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38726/0 (socket says 172.21.15.32:38726) 2024-04-17T09:58:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbdaa00640 1 -- 172.21.15.32:0/200674540 learned_addr learned my addr 172.21.15.32:0/200674540 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbd2000640 1 --2- 172.21.15.32:0/200674540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa51725b80 0x55fa516d8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbdb400640 1 --2- 172.21.15.32:0/200674540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa516d89a0 0x55fa516d8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbdaa00640 1 -- 172.21.15.32:0/200674540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa51725b80 msgr2=0x55fa516d8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbdaa00640 1 --2- 172.21.15.32:0/200674540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa51725b80 0x55fa516d8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbdaa00640 1 -- 172.21.15.32:0/200674540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa516d89a0 msgr2=0x55fa516d8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbdaa00640 1 --2- 172.21.15.32:0/200674540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa516d89a0 0x55fa516d8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbdaa00640 1 -- 172.21.15.32:0/200674540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa5156f8e0 con 0x55fa51725260 2024-04-17T09:58:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbdaa00640 1 --2- 172.21.15.32:0/200674540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725260 0x55fa51725640 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0f0290 tx=0x7ffbcc0f03e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a4be9cdd2b2c2c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbcbe00640 1 -- 172.21.15.32:0/200674540 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc108070 con 0x55fa51725260 2024-04-17T09:58:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbcbe00640 1 -- 172.21.15.32:0/200674540 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbcc0f0c90 con 0x55fa51725260 2024-04-17T09:58:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.246+0000 7ffbcbe00640 1 -- 172.21.15.32:0/200674540 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0f0590 con 0x55fa51725260 2024-04-17T09:58:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbe5335000 1 -- 172.21.15.32:0/200674540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725260 msgr2=0x55fa51725640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbe5335000 1 --2- 172.21.15.32:0/200674540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725260 0x55fa51725640 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0f0290 tx=0x7ffbcc0f03e0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbe5335000 1 -- 172.21.15.32:0/200674540 shutdown_connections 2024-04-17T09:58:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbe5335000 1 --2- 172.21.15.32:0/200674540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa51725b80 0x55fa516d8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbe5335000 1 --2- 172.21.15.32:0/200674540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa516d89a0 0x55fa516d8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbe5335000 1 --2- 172.21.15.32:0/200674540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725260 0x55fa51725640 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbe5335000 1 -- 172.21.15.32:0/200674540 >> 172.21.15.32:0/200674540 conn(0x55fa51706000 msgr2=0x55fa517083f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbe5335000 1 -- 172.21.15.32:0/200674540 shutdown_connections 2024-04-17T09:58:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbe5335000 1 -- 172.21.15.32:0/200674540 wait complete. 2024-04-17T09:58:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbe5335000 1 -- start start 2024-04-17T09:58:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbe5335000 1 Processor -- start 2024-04-17T09:58:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbe5335000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa516d89a0 0x55fa51730600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbe5335000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa51725260 0x55fa51730b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbe5335000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725b80 0x55fa516f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbe5335000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55fa515ab7b0 con 0x55fa51725b80 2024-04-17T09:58:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbe5335000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fa515e5d00 con 0x55fa51725260 2024-04-17T09:58:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbe5335000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55fa514ef2c0 con 0x55fa516d89a0 2024-04-17T09:58:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbe5335000 10 client.0 _ll_register_callbacks cb 0x55fa514ff030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:58:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbdb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725b80 0x55fa516f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbdb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725b80 0x55fa516f4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38736/0 (socket says 172.21.15.32:38736) 2024-04-17T09:58:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbdb400640 1 -- 172.21.15.32:0/2076800252 learned_addr learned my addr 172.21.15.32:0/2076800252 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbdaa00640 1 --2- 172.21.15.32:0/2076800252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa516d89a0 0x55fa51730600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbd2000640 1 --2- 172.21.15.32:0/2076800252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa51725260 0x55fa51730b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbdb400640 1 -- 172.21.15.32:0/2076800252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa516d89a0 msgr2=0x55fa51730600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbdb400640 1 --2- 172.21.15.32:0/2076800252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa516d89a0 0x55fa51730600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbdb400640 1 -- 172.21.15.32:0/2076800252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa51725260 msgr2=0x55fa51730b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbdb400640 1 --2- 172.21.15.32:0/2076800252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa51725260 0x55fa51730b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbdb400640 1 -- 172.21.15.32:0/2076800252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa51502490 con 0x55fa51725b80 2024-04-17T09:58:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbdb400640 1 --2- 172.21.15.32:0/2076800252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725b80 0x55fa516f4ad0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400f1d0 tx=0x7ffbd400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7464f339cda6bc67 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbc8c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:58:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbc8c00640 1 -- 172.21.15.32:0/2076800252 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd40052e0 con 0x55fa51725b80 2024-04-17T09:58:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbc8c00640 1 -- 172.21.15.32:0/2076800252 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7ffbd4003600 con 0x55fa51725b80 2024-04-17T09:58:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbe5335000 1 -- 172.21.15.32:0/2076800252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbcc0f0c90 con 0x55fa51725b80 2024-04-17T09:58:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbc8c00640 1 -- 172.21.15.32:0/2076800252 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7ffbd4020050 con 0x55fa51725b80 2024-04-17T09:58:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbe5335000 10 client.7293 fetch_fsmap learned FSMap version 50 2024-04-17T09:58:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbe5335000 1 -- 172.21.15.32:0/2076800252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x55fa514f5eb0 con 0x55fa51725b80 2024-04-17T09:58:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.250+0000 7ffbc8c00640 1 -- 172.21.15.32:0/2076800252 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbcc0f0c90 con 0x55fa51725b80 2024-04-17T09:58:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbc8c00640 1 -- 172.21.15.32:0/2076800252 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 50) ==== 3495+0+0 (secure 0 0 0) 0x7ffbd4023020 con 0x55fa51725b80 2024-04-17T09:58:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbe5335000 10 client.7293 fetch_fsmap finished waiting for FSMap version 50 2024-04-17T09:58:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbe5335000 10 client.7293 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-17T09:58:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbe5335000 20 client.7293 populate_metadata read hostname 'smithi032' 2024-04-17T09:58:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbe5335000 1 --2- 172.21.15.32:0/2076800252 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55fa516f5870 0x55fa516f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbe5335000 4 client.7293 mds_command: new command op to 7011 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:58:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbe5335000 1 -- 172.21.15.32:0/2076800252 --> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa514c3dd0 con 0x55fa516f5870 2024-04-17T09:58:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbdaa00640 1 --2- 172.21.15.32:0/2076800252 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55fa516f5870 0x55fa516f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbdaa00640 1 --2- 172.21.15.32:0/2076800252 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55fa516f5870 0x55fa516f5c50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f44e7d5548d6c7f3 server_cookie=df4420ae666e3372 in_seq=0 out_seq=0 2024-04-17T09:58:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbc8c00640 10 client.7293 ms_handle_connect on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbc8c00640 1 -- 172.21.15.32:0/2076800252 <== mds.0 v2:172.21.15.32:6834/2416784965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa514c3dd0 con 0x55fa516f5870 2024-04-17T09:58:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbc8c00640 10 client.7293 handle_command_reply: tid=0 2024-04-17T09:58:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbe5335000 1 client.7293 shutdown 2024-04-17T09:58:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbe5335000 1 -- 172.21.15.32:0/2076800252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725b80 msgr2=0x55fa516f4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbe5335000 1 --2- 172.21.15.32:0/2076800252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725b80 0x55fa516f4ad0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400f1d0 tx=0x7ffbd400f690 comp rx=0 tx=0).stop 2024-04-17T09:58:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbe5335000 1 -- 172.21.15.32:0/2076800252 shutdown_connections 2024-04-17T09:58:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbe5335000 1 --2- 172.21.15.32:0/2076800252 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55fa516f5870 0x55fa516f5c50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbe5335000 1 --2- 172.21.15.32:0/2076800252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725b80 0x55fa516f4ad0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbe5335000 1 --2- 172.21.15.32:0/2076800252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa51725260 0x55fa51730b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbc8c00640 0 client.7293 ms_handle_reset on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbe5335000 1 --2- 172.21.15.32:0/2076800252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa516d89a0 0x55fa51730600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbe5335000 1 -- 172.21.15.32:0/2076800252 >> 172.21.15.32:0/2076800252 conn(0x55fa51706000 msgr2=0x55fa517083f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbe5335000 1 -- 172.21.15.32:0/2076800252 shutdown_connections 2024-04-17T09:58:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbe5335000 1 -- 172.21.15.32:0/2076800252 wait complete. 2024-04-17T09:58:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.254+0000 7ffbe5335000 20 client.7293 trim_cache size 0 max 16384 2024-04-17T09:58:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.274+0000 7ffbe5335000 1 Processor -- start 2024-04-17T09:58:44.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.274+0000 7ffbe5335000 1 -- start start 2024-04-17T09:58:44.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbe5335000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa516d89a0 0x55fa51724530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbe5335000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa51725260 0x55fa51724a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbe5335000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725b80 0x55fa51723310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbe5335000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc15cd40 con 0x55fa51725b80 2024-04-17T09:58:44.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbe5335000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc1303c0 con 0x55fa51725260 2024-04-17T09:58:44.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbe5335000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffbdc1311c0 con 0x55fa516d89a0 2024-04-17T09:58:44.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbdb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725b80 0x55fa51723310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbdb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725b80 0x55fa51723310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38738/0 (socket says 172.21.15.32:38738) 2024-04-17T09:58:44.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbdb400640 1 -- 172.21.15.32:0/2128441876 learned_addr learned my addr 172.21.15.32:0/2128441876 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:44.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbdaa00640 1 --2- 172.21.15.32:0/2128441876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa516d89a0 0x55fa51724530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbd2000640 1 --2- 172.21.15.32:0/2128441876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa51725260 0x55fa51724a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbdb400640 1 -- 172.21.15.32:0/2128441876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa516d89a0 msgr2=0x55fa51724530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:44.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbdb400640 1 --2- 172.21.15.32:0/2128441876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa516d89a0 0x55fa51724530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbdb400640 1 -- 172.21.15.32:0/2128441876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa51725260 msgr2=0x55fa51724a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:44.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbdb400640 1 --2- 172.21.15.32:0/2128441876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa51725260 0x55fa51724a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbdb400640 1 -- 172.21.15.32:0/2128441876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa514e4a20 con 0x55fa51725b80 2024-04-17T09:58:44.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbdb400640 1 --2- 172.21.15.32:0/2128441876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725b80 0x55fa51723310 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4007bd0 tx=0x7ffbd400d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e6dd2205e6b201 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:44.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbcb400640 1 -- 172.21.15.32:0/2128441876 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4005360 con 0x55fa51725b80 2024-04-17T09:58:44.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbcb400640 1 -- 172.21.15.32:0/2128441876 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbd4028020 con 0x55fa51725b80 2024-04-17T09:58:44.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbcb400640 1 -- 172.21.15.32:0/2128441876 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4003ad0 con 0x55fa51725b80 2024-04-17T09:58:44.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbe5335000 1 -- 172.21.15.32:0/2128441876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725b80 msgr2=0x55fa51723310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:44.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbe5335000 1 --2- 172.21.15.32:0/2128441876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725b80 0x55fa51723310 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4007bd0 tx=0x7ffbd400d000 comp rx=0 tx=0).stop 2024-04-17T09:58:44.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbe5335000 1 -- 172.21.15.32:0/2128441876 shutdown_connections 2024-04-17T09:58:44.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbe5335000 1 --2- 172.21.15.32:0/2128441876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725b80 0x55fa51723310 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbe5335000 1 --2- 172.21.15.32:0/2128441876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa51725260 0x55fa51724a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbe5335000 1 --2- 172.21.15.32:0/2128441876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa516d89a0 0x55fa51724530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbe5335000 1 -- 172.21.15.32:0/2128441876 >> 172.21.15.32:0/2128441876 conn(0x55fa51706000 msgr2=0x55fa516f4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:44.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbe5335000 1 -- 172.21.15.32:0/2128441876 shutdown_connections 2024-04-17T09:58:44.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbe5335000 1 -- 172.21.15.32:0/2128441876 wait complete. 2024-04-17T09:58:44.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbe5335000 1 -- start start 2024-04-17T09:58:44.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.278+0000 7ffbe5335000 1 Processor -- start 2024-04-17T09:58:44.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa516d89a0 0x55fa51724610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa51725260 0x55fa51733480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725b80 0x55fa517339c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55fa514ce050 con 0x55fa51725b80 2024-04-17T09:58:44.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fa514ec570 con 0x55fa51725260 2024-04-17T09:58:44.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55fa51529800 con 0x55fa516d89a0 2024-04-17T09:58:44.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 10 client.0 _ll_register_callbacks cb 0x7ffbd4007d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:58:44.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725b80 0x55fa517339c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725b80 0x55fa517339c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38740/0 (socket says 172.21.15.32:38740) 2024-04-17T09:58:44.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdb400640 1 -- 172.21.15.32:0/2478842800 learned_addr learned my addr 172.21.15.32:0/2478842800 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:44.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbd2000640 1 --2- 172.21.15.32:0/2478842800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa51725260 0x55fa51733480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdaa00640 1 --2- 172.21.15.32:0/2478842800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa516d89a0 0x55fa51724610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdb400640 1 -- 172.21.15.32:0/2478842800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa516d89a0 msgr2=0x55fa51724610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:44.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdb400640 1 --2- 172.21.15.32:0/2478842800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa516d89a0 0x55fa51724610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdb400640 1 -- 172.21.15.32:0/2478842800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa51725260 msgr2=0x55fa51733480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:44.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdb400640 1 --2- 172.21.15.32:0/2478842800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa51725260 0x55fa51733480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdb400640 1 -- 172.21.15.32:0/2478842800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa515abc40 con 0x55fa51725b80 2024-04-17T09:58:44.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdb400640 1 --2- 172.21.15.32:0/2478842800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725b80 0x55fa517339c0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4000c10 tx=0x7ffbd4001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9e56d6c03a90d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:44.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbc8c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:58:44.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbc8c00640 1 -- 172.21.15.32:0/2478842800 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4033060 con 0x55fa51725b80 2024-04-17T09:58:44.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 1 -- 172.21.15.32:0/2478842800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbd4028020 con 0x55fa51725b80 2024-04-17T09:58:44.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbc8c00640 1 -- 172.21.15.32:0/2478842800 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7ffbd4003c10 con 0x55fa51725b80 2024-04-17T09:58:44.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbc8c00640 1 -- 172.21.15.32:0/2478842800 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7ffbd4039050 con 0x55fa51725b80 2024-04-17T09:58:44.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 10 client.7299 fetch_fsmap learned FSMap version 50 2024-04-17T09:58:44.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 1 -- 172.21.15.32:0/2478842800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x55fa514cf600 con 0x55fa51725b80 2024-04-17T09:58:44.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbc8c00640 1 -- 172.21.15.32:0/2478842800 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbd4028020 con 0x55fa51725b80 2024-04-17T09:58:44.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbc8c00640 1 -- 172.21.15.32:0/2478842800 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 50) ==== 3495+0+0 (secure 0 0 0) 0x7ffbd403c020 con 0x55fa51725b80 2024-04-17T09:58:44.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 10 client.7299 fetch_fsmap finished waiting for FSMap version 50 2024-04-17T09:58:44.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 10 client.7299 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-17T09:58:44.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 20 client.7299 populate_metadata read hostname 'smithi032' 2024-04-17T09:58:44.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 1 --2- 172.21.15.32:0/2478842800 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55fa5179e210 0x55fa517a0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 4 client.7299 mds_command: new command op to 7011 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T09:58:44.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 1 -- 172.21.15.32:0/2478842800 --> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fa51537f10 con 0x55fa5179e210 2024-04-17T09:58:44.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdaa00640 1 --2- 172.21.15.32:0/2478842800 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55fa5179e210 0x55fa517a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdaa00640 1 --2- 172.21.15.32:0/2478842800 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55fa5179e210 0x55fa517a0600 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee46741aedcae5c server_cookie=1fbc4b93071ab558 in_seq=0 out_seq=0 2024-04-17T09:58:44.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbc8c00640 10 client.7299 ms_handle_connect on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbc8c00640 1 -- 172.21.15.32:0/2478842800 <== mds.0 v2:172.21.15.32:6834/2416784965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fa51537f10 con 0x55fa5179e210 2024-04-17T09:58:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbc8c00640 10 client.7299 handle_command_reply: tid=0 2024-04-17T09:58:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 1 client.7299 shutdown 2024-04-17T09:58:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 1 -- 172.21.15.32:0/2478842800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725b80 msgr2=0x55fa517339c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 1 --2- 172.21.15.32:0/2478842800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725b80 0x55fa517339c0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4000c10 tx=0x7ffbd4001130 comp rx=0 tx=0).stop 2024-04-17T09:58:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 1 -- 172.21.15.32:0/2478842800 shutdown_connections 2024-04-17T09:58:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 1 --2- 172.21.15.32:0/2478842800 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x55fa5179e210 0x55fa517a0600 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 1 --2- 172.21.15.32:0/2478842800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fa51725b80 0x55fa517339c0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 1 --2- 172.21.15.32:0/2478842800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa51725260 0x55fa51733480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 1 --2- 172.21.15.32:0/2478842800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa516d89a0 0x55fa51724610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbc8c00640 0 client.7299 ms_handle_reset on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:44.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 1 -- 172.21.15.32:0/2478842800 >> 172.21.15.32:0/2478842800 conn(0x55fa51706000 msgr2=0x55fa5173f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:44.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 1 -- 172.21.15.32:0/2478842800 shutdown_connections 2024-04-17T09:58:44.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 1 -- 172.21.15.32:0/2478842800 wait complete. 2024-04-17T09:58:44.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbe5335000 20 client.7299 trim_cache size 0 max 16384 2024-04-17T09:58:44.436 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:58:44.436 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-17T09:58:44.436 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-17T09:58:44.436 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:58:44.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdbe00640 1 -- 172.21.15.32:0/2147818185 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffbb8068380 msgr2=0x7ffbb806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:44.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdbe00640 1 --2- 172.21.15.32:0/2147818185 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffbb8068380 0x7ffbb806a840 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc06f430 tx=0x7ffbc4007530 comp rx=0 tx=0).stop 2024-04-17T09:58:44.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdbe00640 1 -- 172.21.15.32:0/2147818185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbdc06f4a0 msgr2=0x7ffbdc0af930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:44.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdbe00640 1 --2- 172.21.15.32:0/2147818185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbdc06f4a0 0x7ffbdc0af930 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0ae040 tx=0x7ffbcc0191e0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdbe00640 1 -- 172.21.15.32:0/2147818185 shutdown_connections 2024-04-17T09:58:44.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdbe00640 1 --2- 172.21.15.32:0/2147818185 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffbb8068380 0x7ffbb806a840 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdbe00640 1 --2- 172.21.15.32:0/2147818185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbdc144e90 0x7ffbdc0ac1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdbe00640 1 --2- 172.21.15.32:0/2147818185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbdc0cfc20 0x7ffbdc0abc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdbe00640 1 --2- 172.21.15.32:0/2147818185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbdc06f4a0 0x7ffbdc0af930 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdbe00640 1 -- 172.21.15.32:0/2147818185 >> 172.21.15.32:0/2147818185 conn(0x7ffbdc067d70 msgr2=0x7ffbdc06de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:44.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdbe00640 1 -- 172.21.15.32:0/2147818185 shutdown_connections 2024-04-17T09:58:44.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.282+0000 7ffbdbe00640 1 -- 172.21.15.32:0/2147818185 wait complete. 2024-04-17T09:58:44.439 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T09:58:44.440 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:58:44.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.582+0000 7ff7dd200640 1 Processor -- start 2024-04-17T09:58:44.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.582+0000 7ff7dd200640 1 -- start start 2024-04-17T09:58:44.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7dd200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7d8072500 0x7ff7d80cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7dd200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7d80cdac0 con 0x7ff7d8145f30 2024-04-17T09:58:44.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7dd200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7d815cdf0 con 0x7ff7d815d8d0 2024-04-17T09:58:44.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7dd200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff7d806f400 con 0x7ff7d8072500 2024-04-17T09:58:44.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7d5e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff7d8145f30 0x7ff7d8071df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38948/0 (socket says 172.21.15.32:38948) 2024-04-17T09:58:44.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7d5e00640 1 -- 172.21.15.32:0/3432449133 learned_addr learned my addr 172.21.15.32:0/3432449133 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:58:44.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7cd400640 1 --2- 172.21.15.32:0/3432449133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7d8072500 0x7ff7d80cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7d5400640 1 -- 172.21.15.32:0/3432449133 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2230201669 0 0) 0x7ff7d80cdac0 con 0x7ff7d8145f30 2024-04-17T09:58:44.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7d5400640 1 -- 172.21.15.32:0/3432449133 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7bc003530 con 0x7ff7d8145f30 2024-04-17T09:58:44.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7d5400640 1 -- 172.21.15.32:0/3432449133 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1140693443 0 0) 0x7ff7d815cdf0 con 0x7ff7d815d8d0 2024-04-17T09:58:44.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7d5400640 1 -- 172.21.15.32:0/3432449133 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7d80cdac0 con 0x7ff7d815d8d0 2024-04-17T09:58:44.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7cd400640 1 -- 172.21.15.32:0/3432449133 >> v1:172.21.15.144:6789/0 conn(0x7ff7d815d8d0 legacy=0x7ff7d81601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:44.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7cd400640 1 -- 172.21.15.32:0/3432449133 >> v1:172.21.15.32:6789/0 conn(0x7ff7d8145f30 legacy=0x7ff7d8071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:44.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7d5400640 1 -- 172.21.15.32:0/3432449133 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3891647478 0 0) 0x7ff7bc003530 con 0x7ff7d8145f30 2024-04-17T09:58:44.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7cd400640 1 -- 172.21.15.32:0/3432449133 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d815d0d0 con 0x7ff7d8072500 2024-04-17T09:58:44.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7cd400640 1 --2- 172.21.15.32:0/3432449133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7d8072500 0x7ff7d80cce70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00097f0 tx=0x7ff7c001ebf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6d3d2d51e21af7a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:44.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7d5400640 1 -- 172.21.15.32:0/3432449133 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c00b5070 con 0x7ff7d8072500 2024-04-17T09:58:44.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7d5400640 1 -- 172.21.15.32:0/3432449133 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7c00a8670 con 0x7ff7d8072500 2024-04-17T09:58:44.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7d5400640 1 -- 172.21.15.32:0/3432449133 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8001680 con 0x7ff7d8072500 2024-04-17T09:58:44.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7dd200640 1 -- 172.21.15.32:0/3432449133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7d8072500 msgr2=0x7ff7d80cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:44.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7dd200640 1 --2- 172.21.15.32:0/3432449133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7d8072500 0x7ff7d80cce70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00097f0 tx=0x7ff7c001ebf0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7dd200640 1 -- 172.21.15.32:0/3432449133 shutdown_connections 2024-04-17T09:58:44.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7dd200640 1 --2- 172.21.15.32:0/3432449133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7d8072500 0x7ff7d80cce70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7dd200640 1 -- 172.21.15.32:0/3432449133 >> 172.21.15.32:0/3432449133 conn(0x7ff7d806ba30 msgr2=0x7ff7d806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:44.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7dd200640 1 -- 172.21.15.32:0/3432449133 shutdown_connections 2024-04-17T09:58:44.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7dd200640 1 -- 172.21.15.32:0/3432449133 wait complete. 2024-04-17T09:58:44.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7dd200640 1 Processor -- start 2024-04-17T09:58:44.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7dd200640 1 -- start start 2024-04-17T09:58:44.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7dd200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7d8072500 0x7ff7d8138fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7dd200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7d8145f30 0x7ff7d80b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7dd200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7d815d8d0 0x7ff7d80b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7dd200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d80cd7c0 con 0x7ff7d815d8d0 2024-04-17T09:58:44.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7dd200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d80cd640 con 0x7ff7d8145f30 2024-04-17T09:58:44.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.586+0000 7ff7dd200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff7d8146310 con 0x7ff7d8072500 2024-04-17T09:58:44.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.590+0000 7ff7d6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7d815d8d0 0x7ff7d80b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.590+0000 7ff7d6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7d815d8d0 0x7ff7d80b4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38748/0 (socket says 172.21.15.32:38748) 2024-04-17T09:58:44.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.590+0000 7ff7d6800640 1 -- 172.21.15.32:0/1652038138 learned_addr learned my addr 172.21.15.32:0/1652038138 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:44.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.590+0000 7ff7cd400640 1 --2- 172.21.15.32:0/1652038138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7d8145f30 0x7ff7d80b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.590+0000 7ff7d5e00640 1 --2- 172.21.15.32:0/1652038138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7d8072500 0x7ff7d8138fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.590+0000 7ff7d6800640 1 -- 172.21.15.32:0/1652038138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7d8072500 msgr2=0x7ff7d8138fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:44.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.590+0000 7ff7d6800640 1 --2- 172.21.15.32:0/1652038138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7d8072500 0x7ff7d8138fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.590+0000 7ff7d6800640 1 -- 172.21.15.32:0/1652038138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7d8145f30 msgr2=0x7ff7d80b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:44.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.590+0000 7ff7d6800640 1 --2- 172.21.15.32:0/1652038138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7d8145f30 0x7ff7d80b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.590+0000 7ff7d6800640 1 -- 172.21.15.32:0/1652038138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d8136650 con 0x7ff7d815d8d0 2024-04-17T09:58:44.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.590+0000 7ff7cd400640 1 --2- 172.21.15.32:0/1652038138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7d8145f30 0x7ff7d80b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:58:44.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.590+0000 7ff7d6800640 1 --2- 172.21.15.32:0/1652038138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7d815d8d0 0x7ff7d80b4a80 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000c760 tx=0x7ff7d000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:44.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.590+0000 7ff7cea00640 1 -- 172.21.15.32:0/1652038138 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0003b80 con 0x7ff7d815d8d0 2024-04-17T09:58:44.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.590+0000 7ff7cea00640 1 -- 172.21.15.32:0/1652038138 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7d0003d20 con 0x7ff7d815d8d0 2024-04-17T09:58:44.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.590+0000 7ff7cea00640 1 -- 172.21.15.32:0/1652038138 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d000aa90 con 0x7ff7d815d8d0 2024-04-17T09:58:44.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.590+0000 7ff7dd200640 1 -- 172.21.15.32:0/1652038138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d80a96c0 con 0x7ff7d815d8d0 2024-04-17T09:58:44.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.590+0000 7ff7dd200640 1 -- 172.21.15.32:0/1652038138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d80a9c40 con 0x7ff7d815d8d0 2024-04-17T09:58:44.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.590+0000 7ff7dd200640 1 -- 172.21.15.32:0/1652038138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7d8068a90 con 0x7ff7d815d8d0 2024-04-17T09:58:44.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.590+0000 7ff7cea00640 1 -- 172.21.15.32:0/1652038138 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff7d000ac30 con 0x7ff7d815d8d0 2024-04-17T09:58:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.594+0000 7ff7cea00640 1 --2- 172.21.15.32:0/1652038138 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff7b40683b0 0x7ff7b406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.594+0000 7ff7cea00640 1 -- 172.21.15.32:0/1652038138 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7ff7d0086900 con 0x7ff7d815d8d0 2024-04-17T09:58:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.594+0000 7ff7d5e00640 1 --2- 172.21.15.32:0/1652038138 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff7b40683b0 0x7ff7b406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.594+0000 7ff7d5e00640 1 --2- 172.21.15.32:0/1652038138 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff7b40683b0 0x7ff7b406a870 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8005890 tx=0x7ff7c8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.598+0000 7ff7cea00640 1 -- 172.21.15.32:0/1652038138 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff7d8136650 con 0x7ff7d815d8d0 2024-04-17T09:58:44.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.794+0000 7ff7dd200640 1 -- 172.21.15.32:0/1652038138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7d8072f90 con 0x7ff7d815d8d0 2024-04-17T09:58:44.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.794+0000 7ff7cea00640 1 -- 172.21.15.32:0/1652038138 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5206 (secure 0 0 0) 0x7ff7d0056e20 con 0x7ff7d815d8d0 2024-04-17T09:58:44.797 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:58:44.797 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2424282223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2424282223},{"type":"v1","addr":"172.21.15.144:6834","nonce":2424282223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3537056692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3537056692},{"type":"v1","addr":"172.21.15.144:6835","nonce":3537056692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5891,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/725333724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":725333724},{"type":"v1","addr":"172.21.15.32:6837","nonce":725333724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:57:11.558183+0000","modified":"2024-04-17T09:58:17.770818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":15,"addr":"172.21.15.32:6836/2416784965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2416784965},{"type":"v1","addr":"172.21.15.32:6836","nonce":2416784965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7011,"qdb_cluster":[7011]},"id":5}]} 2024-04-17T09:58:44.798 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 50 2024-04-17T09:58:44.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.798+0000 7ff7dd200640 1 -- 172.21.15.32:0/1652038138 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff7b40683b0 msgr2=0x7ff7b406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:44.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.798+0000 7ff7dd200640 1 --2- 172.21.15.32:0/1652038138 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff7b40683b0 0x7ff7b406a870 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8005890 tx=0x7ff7c8009040 comp rx=0 tx=0).stop 2024-04-17T09:58:44.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.798+0000 7ff7dd200640 1 -- 172.21.15.32:0/1652038138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7d815d8d0 msgr2=0x7ff7d80b4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:44.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.798+0000 7ff7dd200640 1 --2- 172.21.15.32:0/1652038138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7d815d8d0 0x7ff7d80b4a80 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000c760 tx=0x7ff7d000cc20 comp rx=0 tx=0).stop 2024-04-17T09:58:44.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.798+0000 7ff7dd200640 1 -- 172.21.15.32:0/1652038138 shutdown_connections 2024-04-17T09:58:44.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.798+0000 7ff7dd200640 1 --2- 172.21.15.32:0/1652038138 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff7b40683b0 0x7ff7b406a870 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.798+0000 7ff7dd200640 1 --2- 172.21.15.32:0/1652038138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7d815d8d0 0x7ff7d80b4a80 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.798+0000 7ff7dd200640 1 --2- 172.21.15.32:0/1652038138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7d8145f30 0x7ff7d80b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.798+0000 7ff7dd200640 1 --2- 172.21.15.32:0/1652038138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7d8072500 0x7ff7d8138fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.798+0000 7ff7dd200640 1 -- 172.21.15.32:0/1652038138 >> 172.21.15.32:0/1652038138 conn(0x7ff7d806ba30 msgr2=0x7ff7d806be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:44.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.798+0000 7ff7dd200640 1 -- 172.21.15.32:0/1652038138 shutdown_connections 2024-04-17T09:58:44.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.798+0000 7ff7dd200640 1 -- 172.21.15.32:0/1652038138 wait complete. 2024-04-17T09:58:44.819 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-17T09:58:44.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.962+0000 7fd152800640 1 Processor -- start 2024-04-17T09:58:44.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.962+0000 7fd152800640 1 -- start start 2024-04-17T09:58:44.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd152800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd14c06ba30 0x7fd14c06be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd152800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd14c071e60 0x7fd14c072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd152800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd14c06c350 con 0x7fd14c0cf370 2024-04-17T09:58:44.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd152800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd14c0cfef0 con 0x7fd14c06ba30 2024-04-17T09:58:44.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd152800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd14c0d0130 con 0x7fd14c071e60 2024-04-17T09:58:44.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd14b400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd14c06ba30 0x7fd14c06be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd14b400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd14c06ba30 0x7fd14c06be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38760/0 (socket says 172.21.15.32:38760) 2024-04-17T09:58:44.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd14b400640 1 -- 172.21.15.32:0/2413307521 learned_addr learned my addr 172.21.15.32:0/2413307521 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:44.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd14be00640 1 --2- 172.21.15.32:0/2413307521 >> v2:172.21.15.144:3300/0 conn(0x7fd14c071e60 0x7fd14c072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd14b400640 1 -- 172.21.15.32:0/2413307521 >> v2:172.21.15.144:3300/0 conn(0x7fd14c071e60 msgr2=0x7fd14c072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:44.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd14b400640 1 --2- 172.21.15.32:0/2413307521 >> v2:172.21.15.144:3300/0 conn(0x7fd14c071e60 0x7fd14c072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd14b400640 1 -- 172.21.15.32:0/2413307521 >> v1:172.21.15.144:6789/0 conn(0x7fd14c0cf370 legacy=0x7fd14c0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:44.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd14b400640 1 -- 172.21.15.32:0/2413307521 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd14c15d090 con 0x7fd14c06ba30 2024-04-17T09:58:44.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd14b400640 1 --2- 172.21.15.32:0/2413307521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd14c06ba30 0x7fd14c06be10 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0cd7e0 tx=0x7fd13c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348f24d6713b22b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd14aa00640 1 -- 172.21.15.32:0/2413307521 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c0b2070 con 0x7fd14c06ba30 2024-04-17T09:58:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd14aa00640 1 -- 172.21.15.32:0/2413307521 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd13c0a9070 con 0x7fd14c06ba30 2024-04-17T09:58:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd14aa00640 1 -- 172.21.15.32:0/2413307521 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c0ad880 con 0x7fd14c06ba30 2024-04-17T09:58:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd152800640 1 -- 172.21.15.32:0/2413307521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd14c06ba30 msgr2=0x7fd14c06be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd152800640 1 --2- 172.21.15.32:0/2413307521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd14c06ba30 0x7fd14c06be10 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0cd7e0 tx=0x7fd13c01c430 comp rx=0 tx=0).stop 2024-04-17T09:58:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd152800640 1 -- 172.21.15.32:0/2413307521 shutdown_connections 2024-04-17T09:58:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd152800640 1 --2- 172.21.15.32:0/2413307521 >> v2:172.21.15.144:3300/0 conn(0x7fd14c071e60 0x7fd14c072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd152800640 1 --2- 172.21.15.32:0/2413307521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd14c06ba30 0x7fd14c06be10 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd152800640 1 -- 172.21.15.32:0/2413307521 >> 172.21.15.32:0/2413307521 conn(0x7fd14c0b2120 msgr2=0x7fd14c0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd152800640 1 -- 172.21.15.32:0/2413307521 shutdown_connections 2024-04-17T09:58:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd152800640 1 -- 172.21.15.32:0/2413307521 wait complete. 2024-04-17T09:58:44.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd152800640 1 Processor -- start 2024-04-17T09:58:44.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd152800640 1 -- start start 2024-04-17T09:58:44.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd152800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd14c06ba30 0x7fd14c15c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd152800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd14c071e60 0x7fd14c0adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd152800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd14c0cf370 0x7fd14c13ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd152800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c15dcf0 con 0x7fd14c0cf370 2024-04-17T09:58:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd152800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c15db70 con 0x7fd14c071e60 2024-04-17T09:58:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd152800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd14c146230 con 0x7fd14c06ba30 2024-04-17T09:58:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd14be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd14c0cf370 0x7fd14c13ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd14be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd14c0cf370 0x7fd14c13ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38772/0 (socket says 172.21.15.32:38772) 2024-04-17T09:58:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd14be00640 1 -- 172.21.15.32:0/965704661 learned_addr learned my addr 172.21.15.32:0/965704661 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd14b400640 1 --2- 172.21.15.32:0/965704661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd14c06ba30 0x7fd14c15c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.966+0000 7fd142a00640 1 --2- 172.21.15.32:0/965704661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd14c071e60 0x7fd14c0adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.970+0000 7fd14be00640 1 -- 172.21.15.32:0/965704661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd14c06ba30 msgr2=0x7fd14c15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.970+0000 7fd14be00640 1 --2- 172.21.15.32:0/965704661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd14c06ba30 0x7fd14c15c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.970+0000 7fd14be00640 1 -- 172.21.15.32:0/965704661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd14c071e60 msgr2=0x7fd14c0adf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:44.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.970+0000 7fd14be00640 1 --2- 172.21.15.32:0/965704661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd14c071e60 0x7fd14c0adf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.970+0000 7fd14be00640 1 -- 172.21.15.32:0/965704661 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd14c13e520 con 0x7fd14c0cf370 2024-04-17T09:58:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.970+0000 7fd142a00640 1 --2- 172.21.15.32:0/965704661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd14c071e60 0x7fd14c0adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:58:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.970+0000 7fd14be00640 1 --2- 172.21.15.32:0/965704661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd14c0cf370 0x7fd14c13ea00 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fd144007e40 tx=0x7fd14400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.970+0000 7fd143e00640 1 -- 172.21.15.32:0/965704661 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd144014620 con 0x7fd14c0cf370 2024-04-17T09:58:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.970+0000 7fd143e00640 1 -- 172.21.15.32:0/965704661 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd144019070 con 0x7fd14c0cf370 2024-04-17T09:58:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.970+0000 7fd143e00640 1 -- 172.21.15.32:0/965704661 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd144014a20 con 0x7fd14c0cf370 2024-04-17T09:58:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.970+0000 7fd152800640 1 -- 172.21.15.32:0/965704661 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd14c0ae840 con 0x7fd14c0cf370 2024-04-17T09:58:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.970+0000 7fd152800640 1 -- 172.21.15.32:0/965704661 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd14c0aee00 con 0x7fd14c0cf370 2024-04-17T09:58:44.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.970+0000 7fd143e00640 1 -- 172.21.15.32:0/965704661 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd144014bc0 con 0x7fd14c0cf370 2024-04-17T09:58:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd143e00640 1 --2- 172.21.15.32:0/965704661 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd1280683b0 0x7fd12806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd143e00640 1 -- 172.21.15.32:0/965704661 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fd144087270 con 0x7fd14c0cf370 2024-04-17T09:58:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd14b400640 1 --2- 172.21.15.32:0/965704661 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd1280683b0 0x7fd12806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd155ce0000 1 Processor -- start 2024-04-17T09:58:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd155ce0000 1 -- start start 2024-04-17T09:58:44.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd14b400640 1 --2- 172.21.15.32:0/965704661 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd1280683b0 0x7fd12806a870 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd13c0af000 tx=0x7fd13c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:44.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd155ce0000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604c123ea50 0x5604c123ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd155ce0000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c123f370 0x5604c11f1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd155ce0000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604c11f1f70 0x5604c11f2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd155ce0000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5604c1043590 con 0x5604c123f370 2024-04-17T09:58:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd155ce0000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5604c0ffc580 con 0x5604c123ea50 2024-04-17T09:58:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd155ce0000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5604c103f550 con 0x5604c11f1f70 2024-04-17T09:58:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd142a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c123f370 0x5604c11f1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd142a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c123f370 0x5604c11f1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38788/0 (socket says 172.21.15.32:38788) 2024-04-17T09:58:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd142a00640 1 -- 172.21.15.32:0/3482581505 learned_addr learned my addr 172.21.15.32:0/3482581505 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd14be00640 1 --2- 172.21.15.32:0/3482581505 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604c11f1f70 0x5604c11f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd14b400640 1 --2- 172.21.15.32:0/3482581505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604c123ea50 0x5604c123ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd142a00640 1 -- 172.21.15.32:0/3482581505 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604c11f1f70 msgr2=0x5604c11f2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd142a00640 1 --2- 172.21.15.32:0/3482581505 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604c11f1f70 0x5604c11f2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd142a00640 1 -- 172.21.15.32:0/3482581505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604c123ea50 msgr2=0x5604c123ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd142a00640 1 --2- 172.21.15.32:0/3482581505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604c123ea50 0x5604c123ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd142a00640 1 -- 172.21.15.32:0/3482581505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604c10eab90 con 0x5604c123f370 2024-04-17T09:58:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd14b400640 1 --2- 172.21.15.32:0/3482581505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604c123ea50 0x5604c123ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:58:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd142a00640 1 --2- 172.21.15.32:0/3482581505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c123f370 0x5604c11f1a30 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fd134001570 tx=0x7fd134001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712ed4ffae8a0cb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd13be00640 1 -- 172.21.15.32:0/3482581505 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd134019070 con 0x5604c123f370 2024-04-17T09:58:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd13be00640 1 -- 172.21.15.32:0/3482581505 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1340050d0 con 0x5604c123f370 2024-04-17T09:58:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd13be00640 1 -- 172.21.15.32:0/3482581505 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1340054a0 con 0x5604c123f370 2024-04-17T09:58:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd155ce0000 1 -- 172.21.15.32:0/3482581505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c123f370 msgr2=0x5604c11f1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.974+0000 7fd155ce0000 1 --2- 172.21.15.32:0/3482581505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c123f370 0x5604c11f1a30 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fd134001570 tx=0x7fd134001a30 comp rx=0 tx=0).stop 2024-04-17T09:58:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 1 -- 172.21.15.32:0/3482581505 shutdown_connections 2024-04-17T09:58:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 1 --2- 172.21.15.32:0/3482581505 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604c11f1f70 0x5604c11f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 1 --2- 172.21.15.32:0/3482581505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c123f370 0x5604c11f1a30 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 1 --2- 172.21.15.32:0/3482581505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604c123ea50 0x5604c123ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 1 -- 172.21.15.32:0/3482581505 >> 172.21.15.32:0/3482581505 conn(0x5604c121f7f0 msgr2=0x5604c1221be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 1 -- 172.21.15.32:0/3482581505 shutdown_connections 2024-04-17T09:58:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 1 -- 172.21.15.32:0/3482581505 wait complete. 2024-04-17T09:58:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 1 -- start start 2024-04-17T09:58:44.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 1 Processor -- start 2024-04-17T09:58:44.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604c11f1f70 0x5604c11f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c123ea50 0x5604c11f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604c123f370 0x5604c11f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5604c104e4c0 con 0x5604c123ea50 2024-04-17T09:58:44.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5604c0fe3c80 con 0x5604c123f370 2024-04-17T09:58:44.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5604c100e930 con 0x5604c11f1f70 2024-04-17T09:58:44.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 10 client.0 _ll_register_callbacks cb 0x5604c11df6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:58:44.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd142a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c123ea50 0x5604c11f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd14b400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604c11f1f70 0x5604c11f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd142a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c123ea50 0x5604c11f68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38804/0 (socket says 172.21.15.32:38804) 2024-04-17T09:58:44.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd142a00640 1 -- 172.21.15.32:0/1861334606 learned_addr learned my addr 172.21.15.32:0/1861334606 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:44.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd14be00640 1 --2- 172.21.15.32:0/1861334606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604c123f370 0x5604c11f6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd142a00640 1 -- 172.21.15.32:0/1861334606 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604c11f1f70 msgr2=0x5604c11f63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:44.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd142a00640 1 --2- 172.21.15.32:0/1861334606 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604c11f1f70 0x5604c11f63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd142a00640 1 -- 172.21.15.32:0/1861334606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604c123f370 msgr2=0x5604c11f6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:44.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd142a00640 1 --2- 172.21.15.32:0/1861334606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604c123f370 0x5604c11f6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd142a00640 1 -- 172.21.15.32:0/1861334606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604c101cb20 con 0x5604c123ea50 2024-04-17T09:58:44.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd142a00640 1 --2- 172.21.15.32:0/1861334606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c123ea50 0x5604c11f68f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fd134009010 tx=0x7fd134000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937cab6c6dc475b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:44.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd138c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:58:44.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd138c00640 1 -- 172.21.15.32:0/1861334606 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd134023020 con 0x5604c123ea50 2024-04-17T09:58:44.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd138c00640 1 -- 172.21.15.32:0/1861334606 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fd13400e950 con 0x5604c123ea50 2024-04-17T09:58:44.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 1 -- 172.21.15.32:0/1861334606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1340050d0 con 0x5604c123ea50 2024-04-17T09:58:44.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd138c00640 1 -- 172.21.15.32:0/1861334606 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fd134019070 con 0x5604c123ea50 2024-04-17T09:58:44.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 10 client.7317 fetch_fsmap learned FSMap version 50 2024-04-17T09:58:44.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 1 -- 172.21.15.32:0/1861334606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x5604c1087610 con 0x5604c123ea50 2024-04-17T09:58:44.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd138c00640 1 -- 172.21.15.32:0/1861334606 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1340050d0 con 0x5604c123ea50 2024-04-17T09:58:44.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd138c00640 1 -- 172.21.15.32:0/1861334606 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 50) ==== 3495+0+0 (secure 0 0 0) 0x7fd13400eb70 con 0x5604c123ea50 2024-04-17T09:58:44.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 10 client.7317 fetch_fsmap finished waiting for FSMap version 50 2024-04-17T09:58:44.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 10 client.7317 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T09:58:44.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 20 client.7317 populate_metadata read hostname 'smithi032' 2024-04-17T09:58:44.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 1 --2- 172.21.15.32:0/1861334606 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x5604c11f7840 0x5604c124faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 4 client.7317 mds_command: new command op to 7011 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:58:44.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd155ce0000 1 -- 172.21.15.32:0/1861334606 --> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604c0fdce40 con 0x5604c11f7840 2024-04-17T09:58:44.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.978+0000 7fd14b400640 1 --2- 172.21.15.32:0/1861334606 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x5604c11f7840 0x5604c124faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:44.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.982+0000 7fd14b400640 1 --2- 172.21.15.32:0/1861334606 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x5604c11f7840 0x5604c124faf0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=310ae74fc18582bb server_cookie=4fe188e0803dcaee in_seq=0 out_seq=0 2024-04-17T09:58:44.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.982+0000 7fd138c00640 10 client.7317 ms_handle_connect on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.982+0000 7fd138c00640 1 -- 172.21.15.32:0/1861334606 <== mds.0 v2:172.21.15.32:6834/2416784965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5604c0fdce40 con 0x5604c11f7840 2024-04-17T09:58:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.982+0000 7fd138c00640 10 client.7317 handle_command_reply: tid=0 2024-04-17T09:58:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.982+0000 7fd155ce0000 1 client.7317 shutdown 2024-04-17T09:58:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.982+0000 7fd155ce0000 1 -- 172.21.15.32:0/1861334606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c123ea50 msgr2=0x5604c11f68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.982+0000 7fd155ce0000 1 --2- 172.21.15.32:0/1861334606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c123ea50 0x5604c11f68f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fd134009010 tx=0x7fd134000bd0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.982+0000 7fd155ce0000 1 -- 172.21.15.32:0/1861334606 shutdown_connections 2024-04-17T09:58:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.982+0000 7fd155ce0000 1 --2- 172.21.15.32:0/1861334606 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x5604c11f7840 0x5604c124faf0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.982+0000 7fd155ce0000 1 --2- 172.21.15.32:0/1861334606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604c123f370 0x5604c11f6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.982+0000 7fd155ce0000 1 --2- 172.21.15.32:0/1861334606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c123ea50 0x5604c11f68f0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.982+0000 7fd155ce0000 1 --2- 172.21.15.32:0/1861334606 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604c11f1f70 0x5604c11f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.982+0000 7fd155ce0000 1 -- 172.21.15.32:0/1861334606 >> 172.21.15.32:0/1861334606 conn(0x5604c121f7f0 msgr2=0x5604c1221be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.982+0000 7fd138c00640 0 client.7317 ms_handle_reset on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.982+0000 7fd155ce0000 1 -- 172.21.15.32:0/1861334606 shutdown_connections 2024-04-17T09:58:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.982+0000 7fd155ce0000 1 -- 172.21.15.32:0/1861334606 wait complete. 2024-04-17T09:58:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.982+0000 7fd155ce0000 20 client.7317 trim_cache size 0 max 16384 2024-04-17T09:58:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.998+0000 7fd155ce0000 1 Processor -- start 2024-04-17T09:58:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:44.998+0000 7fd155ce0000 1 -- start start 2024-04-17T09:58:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604c11f1f70 0x5604c12a96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c123ea50 0x5604c12a9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604c123f370 0x5604c12aa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c202850 con 0x5604c123ea50 2024-04-17T09:58:45.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5604c10507a0 con 0x5604c11f1f70 2024-04-17T09:58:45.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5604c10428a0 con 0x5604c123f370 2024-04-17T09:58:45.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd142a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c123ea50 0x5604c12a9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:45.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd142a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c123ea50 0x5604c12a9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38810/0 (socket says 172.21.15.32:38810) 2024-04-17T09:58:45.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd142a00640 1 -- 172.21.15.32:0/2412358602 learned_addr learned my addr 172.21.15.32:0/2412358602 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:45.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd14b400640 1 --2- 172.21.15.32:0/2412358602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604c11f1f70 0x5604c12a96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:45.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd14be00640 1 --2- 172.21.15.32:0/2412358602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604c123f370 0x5604c12aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd142a00640 1 -- 172.21.15.32:0/2412358602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604c123f370 msgr2=0x5604c12aa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd142a00640 1 --2- 172.21.15.32:0/2412358602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604c123f370 0x5604c12aa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd142a00640 1 -- 172.21.15.32:0/2412358602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604c11f1f70 msgr2=0x5604c12a96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd142a00640 1 --2- 172.21.15.32:0/2412358602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604c11f1f70 0x5604c12a96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd142a00640 1 -- 172.21.15.32:0/2412358602 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604c1019190 con 0x5604c123ea50 2024-04-17T09:58:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd142a00640 1 --2- 172.21.15.32:0/2412358602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c123ea50 0x5604c12a9c10 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fd13400f010 tx=0x7fd13401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5708b4db9d11f9d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd13b400640 1 -- 172.21.15.32:0/2412358602 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd134023070 con 0x5604c123ea50 2024-04-17T09:58:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd13b400640 1 -- 172.21.15.32:0/2412358602 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd13401a040 con 0x5604c123ea50 2024-04-17T09:58:45.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd13b400640 1 -- 172.21.15.32:0/2412358602 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13400ebb0 con 0x5604c123ea50 2024-04-17T09:58:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 -- 172.21.15.32:0/2412358602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c123ea50 msgr2=0x5604c12a9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 --2- 172.21.15.32:0/2412358602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c123ea50 0x5604c12a9c10 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fd13400f010 tx=0x7fd13401d000 comp rx=0 tx=0).stop 2024-04-17T09:58:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 -- 172.21.15.32:0/2412358602 shutdown_connections 2024-04-17T09:58:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 --2- 172.21.15.32:0/2412358602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604c123f370 0x5604c12aa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 --2- 172.21.15.32:0/2412358602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c123ea50 0x5604c12a9c10 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 --2- 172.21.15.32:0/2412358602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604c11f1f70 0x5604c12a96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 -- 172.21.15.32:0/2412358602 >> 172.21.15.32:0/2412358602 conn(0x5604c121f7f0 msgr2=0x5604c1252d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 -- 172.21.15.32:0/2412358602 shutdown_connections 2024-04-17T09:58:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 -- 172.21.15.32:0/2412358602 wait complete. 2024-04-17T09:58:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 -- start start 2024-04-17T09:58:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 Processor -- start 2024-04-17T09:58:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c11f1f70 0x5604c1252200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604c123ea50 0x5604c1269680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604c123f370 0x5604c1269bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5604c0fe8230 con 0x5604c11f1f70 2024-04-17T09:58:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5604c11fd360 con 0x5604c123f370 2024-04-17T09:58:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5604c1001a90 con 0x5604c123ea50 2024-04-17T09:58:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 10 client.0 _ll_register_callbacks cb 0x7fd134017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:58:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd14b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c11f1f70 0x5604c1252200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd14b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c11f1f70 0x5604c1252200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38820/0 (socket says 172.21.15.32:38820) 2024-04-17T09:58:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd14b400640 1 -- 172.21.15.32:0/3488554487 learned_addr learned my addr 172.21.15.32:0/3488554487 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd142a00640 1 --2- 172.21.15.32:0/3488554487 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604c123ea50 0x5604c1269680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd14be00640 1 --2- 172.21.15.32:0/3488554487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604c123f370 0x5604c1269bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd14b400640 1 -- 172.21.15.32:0/3488554487 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604c123ea50 msgr2=0x5604c1269680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd14b400640 1 --2- 172.21.15.32:0/3488554487 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604c123ea50 0x5604c1269680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd14b400640 1 -- 172.21.15.32:0/3488554487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604c123f370 msgr2=0x5604c1269bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd14b400640 1 --2- 172.21.15.32:0/3488554487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604c123f370 0x5604c1269bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd14b400640 1 -- 172.21.15.32:0/3488554487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604c0ffc7d0 con 0x5604c11f1f70 2024-04-17T09:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd14b400640 1 --2- 172.21.15.32:0/3488554487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c11f1f70 0x5604c1252200 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fd13c008010 tx=0x7fd13c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c22b4348440abd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd138c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd138c00640 1 -- 172.21.15.32:0/3488554487 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c0b2040 con 0x5604c11f1f70 2024-04-17T09:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd138c00640 1 -- 172.21.15.32:0/3488554487 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fd13c0b8ac0 con 0x5604c11f1f70 2024-04-17T09:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 -- 172.21.15.32:0/3488554487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd13401a040 con 0x5604c11f1f70 2024-04-17T09:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd138c00640 1 -- 172.21.15.32:0/3488554487 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fd13c0dc050 con 0x5604c11f1f70 2024-04-17T09:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 10 client.7323 fetch_fsmap learned FSMap version 50 2024-04-17T09:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 -- 172.21.15.32:0/3488554487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x5604c0fe8440 con 0x5604c11f1f70 2024-04-17T09:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd138c00640 1 -- 172.21.15.32:0/3488554487 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd13401a040 con 0x5604c11f1f70 2024-04-17T09:58:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd138c00640 1 -- 172.21.15.32:0/3488554487 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 50) ==== 3495+0+0 (secure 0 0 0) 0x7fd13c0df020 con 0x5604c11f1f70 2024-04-17T09:58:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 10 client.7323 fetch_fsmap finished waiting for FSMap version 50 2024-04-17T09:58:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 10 client.7323 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T09:58:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 20 client.7323 populate_metadata read hostname 'smithi032' 2024-04-17T09:58:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 --2- 172.21.15.32:0/3488554487 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x5604c12ad0e0 0x5604c12ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 4 client.7323 mds_command: new command op to 7011 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-17T09:58:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 -- 172.21.15.32:0/3488554487 --> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5604c1050fb0 con 0x5604c12ad0e0 2024-04-17T09:58:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd142a00640 1 --2- 172.21.15.32:0/3488554487 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x5604c12ad0e0 0x5604c12ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd142a00640 1 --2- 172.21.15.32:0/3488554487 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x5604c12ad0e0 0x5604c12ad4c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1738532888d61cad server_cookie=c46a7cf2c4bb8f8f in_seq=0 out_seq=0 2024-04-17T09:58:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd138c00640 10 client.7323 ms_handle_connect on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd138c00640 1 -- 172.21.15.32:0/3488554487 <== mds.0 v2:172.21.15.32:6834/2416784965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5604c1050fb0 con 0x5604c12ad0e0 2024-04-17T09:58:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd138c00640 10 client.7323 handle_command_reply: tid=0 2024-04-17T09:58:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 client.7323 shutdown 2024-04-17T09:58:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 -- 172.21.15.32:0/3488554487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c11f1f70 msgr2=0x5604c1252200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 --2- 172.21.15.32:0/3488554487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c11f1f70 0x5604c1252200 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fd13c008010 tx=0x7fd13c0046d0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 -- 172.21.15.32:0/3488554487 shutdown_connections 2024-04-17T09:58:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 --2- 172.21.15.32:0/3488554487 >> [v2:172.21.15.32:6834/2416784965,v1:172.21.15.32:6836/2416784965] conn(0x5604c12ad0e0 0x5604c12ad4c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 --2- 172.21.15.32:0/3488554487 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604c123ea50 0x5604c1269680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 --2- 172.21.15.32:0/3488554487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604c123f370 0x5604c1269bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 --2- 172.21.15.32:0/3488554487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604c11f1f70 0x5604c1252200 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd138c00640 0 client.7323 ms_handle_reset on v2:172.21.15.32:6834/2416784965 2024-04-17T09:58:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 -- 172.21.15.32:0/3488554487 >> 172.21.15.32:0/3488554487 conn(0x5604c121f7f0 msgr2=0x5604c121d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 -- 172.21.15.32:0/3488554487 shutdown_connections 2024-04-17T09:58:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 1 -- 172.21.15.32:0/3488554487 wait complete. 2024-04-17T09:58:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd155ce0000 20 client.7323 trim_cache size 0 max 16384 2024-04-17T09:58:45.014 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:58:45.014 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-17T09:58:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd152800640 1 -- 172.21.15.32:0/965704661 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd1280683b0 msgr2=0x7fd12806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd152800640 1 --2- 172.21.15.32:0/965704661 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd1280683b0 0x7fd12806a870 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd13c0af000 tx=0x7fd13c008040 comp rx=0 tx=0).stop 2024-04-17T09:58:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd152800640 1 -- 172.21.15.32:0/965704661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd14c0cf370 msgr2=0x7fd14c13ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd152800640 1 --2- 172.21.15.32:0/965704661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd14c0cf370 0x7fd14c13ea00 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fd144007e40 tx=0x7fd14400e4b0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd152800640 1 -- 172.21.15.32:0/965704661 shutdown_connections 2024-04-17T09:58:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd152800640 1 --2- 172.21.15.32:0/965704661 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd1280683b0 0x7fd12806a870 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd152800640 1 --2- 172.21.15.32:0/965704661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd14c0cf370 0x7fd14c13ea00 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd152800640 1 --2- 172.21.15.32:0/965704661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd14c071e60 0x7fd14c0adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd152800640 1 --2- 172.21.15.32:0/965704661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd14c06ba30 0x7fd14c15c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd152800640 1 -- 172.21.15.32:0/965704661 >> 172.21.15.32:0/965704661 conn(0x7fd14c0b2120 msgr2=0x7fd14c145440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd152800640 1 -- 172.21.15.32:0/965704661 shutdown_connections 2024-04-17T09:58:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.002+0000 7fd152800640 1 -- 172.21.15.32:0/965704661 wait complete. 2024-04-17T09:58:45.024 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-17T09:58:45.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.162+0000 7fef25a00640 1 Processor -- start 2024-04-17T09:58:45.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.162+0000 7fef25a00640 1 -- start start 2024-04-17T09:58:45.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.162+0000 7fef25a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fef2015e810 0x7fef2015f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:45.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.162+0000 7fef25a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef200b47f0 0x7fef2006bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:45.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.162+0000 7fef25a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef20162640 con 0x7fef200b3c50 2024-04-17T09:58:45.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.162+0000 7fef25a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fef2015feb0 con 0x7fef2015e810 2024-04-17T09:58:45.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.162+0000 7fef25a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fef201628e0 con 0x7fef200b47f0 2024-04-17T09:58:45.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.162+0000 7fef1ea00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fef2015e810 0x7fef2015f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:45.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.162+0000 7fef16000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fef200b3c50 0x7fef200b40e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38954/0 (socket says 172.21.15.32:38954) 2024-04-17T09:58:45.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.162+0000 7fef16000640 1 -- 172.21.15.32:0/1639883600 learned_addr learned my addr 172.21.15.32:0/1639883600 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:58:45.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.162+0000 7fef1f400640 1 --2- 172.21.15.32:0/1639883600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef200b47f0 0x7fef2006bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:45.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.162+0000 7fef1ea00640 1 -- 172.21.15.32:0/1639883600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef200b47f0 msgr2=0x7fef2006bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:45.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.162+0000 7fef1ea00640 1 --2- 172.21.15.32:0/1639883600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef200b47f0 0x7fef2006bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.162+0000 7fef1ea00640 1 -- 172.21.15.32:0/1639883600 >> v1:172.21.15.32:6789/0 conn(0x7fef200b3c50 legacy=0x7fef200b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:45.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.162+0000 7fef1ea00640 1 -- 172.21.15.32:0/1639883600 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef20162bc0 con 0x7fef2015e810 2024-04-17T09:58:45.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.166+0000 7fef1ea00640 1 --2- 172.21.15.32:0/1639883600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef2015e810 0x7fef2015f970 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fef100097c0 tx=0x7fef1001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73523061e485e0dd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:45.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.166+0000 7fef1e000640 1 -- 172.21.15.32:0/1639883600 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef100b0070 con 0x7fef2015e810 2024-04-17T09:58:45.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.166+0000 7fef1e000640 1 -- 172.21.15.32:0/1639883600 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef100ab600 con 0x7fef2015e810 2024-04-17T09:58:45.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.166+0000 7fef1e000640 1 -- 172.21.15.32:0/1639883600 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef100ab9d0 con 0x7fef2015e810 2024-04-17T09:58:45.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.166+0000 7fef25a00640 1 -- 172.21.15.32:0/1639883600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef2015e810 msgr2=0x7fef2015f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:45.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.166+0000 7fef25a00640 1 --2- 172.21.15.32:0/1639883600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef2015e810 0x7fef2015f970 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fef100097c0 tx=0x7fef1001c380 comp rx=0 tx=0).stop 2024-04-17T09:58:45.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.166+0000 7fef25a00640 1 -- 172.21.15.32:0/1639883600 shutdown_connections 2024-04-17T09:58:45.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.166+0000 7fef25a00640 1 --2- 172.21.15.32:0/1639883600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef200b47f0 0x7fef2006bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.166+0000 7fef25a00640 1 --2- 172.21.15.32:0/1639883600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef2015e810 0x7fef2015f970 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.166+0000 7fef25a00640 1 -- 172.21.15.32:0/1639883600 >> 172.21.15.32:0/1639883600 conn(0x7fef200d0d60 msgr2=0x7fef200d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:45.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.166+0000 7fef25a00640 1 -- 172.21.15.32:0/1639883600 shutdown_connections 2024-04-17T09:58:45.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.166+0000 7fef25a00640 1 -- 172.21.15.32:0/1639883600 wait complete. 2024-04-17T09:58:45.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef25a00640 1 Processor -- start 2024-04-17T09:58:45.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef25a00640 1 -- start start 2024-04-17T09:58:45.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef25a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef200b3c50 0x7fef200ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:45.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef25a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef200b47f0 0x7fef200ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:45.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef25a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef2015e810 0x7fef200cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:45.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef25a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fef2006cbe0 con 0x7fef200b47f0 2024-04-17T09:58:45.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef25a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fef2006ca60 con 0x7fef2015e810 2024-04-17T09:58:45.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef25a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fef2006cd60 con 0x7fef200b3c50 2024-04-17T09:58:45.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef16000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef200b47f0 0x7fef200ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:45.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef16000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef200b47f0 0x7fef200ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38826/0 (socket says 172.21.15.32:38826) 2024-04-17T09:58:45.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef16000640 1 -- 172.21.15.32:0/779980675 learned_addr learned my addr 172.21.15.32:0/779980675 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:45.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef1ea00640 1 --2- 172.21.15.32:0/779980675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef200b3c50 0x7fef200ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:45.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef1f400640 1 --2- 172.21.15.32:0/779980675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef2015e810 0x7fef200cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:45.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef16000640 1 -- 172.21.15.32:0/779980675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef200b3c50 msgr2=0x7fef200ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:45.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef16000640 1 --2- 172.21.15.32:0/779980675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef200b3c50 0x7fef200ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef16000640 1 -- 172.21.15.32:0/779980675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef2015e810 msgr2=0x7fef200cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:45.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef16000640 1 --2- 172.21.15.32:0/779980675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef2015e810 0x7fef200cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef16000640 1 -- 172.21.15.32:0/779980675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef200af750 con 0x7fef200b47f0 2024-04-17T09:58:45.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef16000640 1 --2- 172.21.15.32:0/779980675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef200b47f0 0x7fef200ae740 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fef0800c370 tx=0x7fef0800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:45.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef17400640 1 -- 172.21.15.32:0/779980675 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08017070 con 0x7fef200b47f0 2024-04-17T09:58:45.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef17400640 1 -- 172.21.15.32:0/779980675 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef08005cf0 con 0x7fef200b47f0 2024-04-17T09:58:45.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef17400640 1 -- 172.21.15.32:0/779980675 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef080072f0 con 0x7fef200b47f0 2024-04-17T09:58:45.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef25a00640 1 -- 172.21.15.32:0/779980675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef200aeeb0 con 0x7fef200b47f0 2024-04-17T09:58:45.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.170+0000 7fef25a00640 1 -- 172.21.15.32:0/779980675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef200af470 con 0x7fef200b47f0 2024-04-17T09:58:45.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.174+0000 7fef25a00640 1 -- 172.21.15.32:0/779980675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef20156050 con 0x7fef200b47f0 2024-04-17T09:58:45.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.174+0000 7fef17400640 1 -- 172.21.15.32:0/779980675 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fef0801a070 con 0x7fef200b47f0 2024-04-17T09:58:45.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.174+0000 7fef17400640 1 --2- 172.21.15.32:0/779980675 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7feefc068380 0x7feefc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:45.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.174+0000 7fef17400640 1 -- 172.21.15.32:0/779980675 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fef08085d30 con 0x7fef200b47f0 2024-04-17T09:58:45.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.174+0000 7fef1ea00640 1 --2- 172.21.15.32:0/779980675 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7feefc068380 0x7feefc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:45.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.174+0000 7fef1ea00640 1 --2- 172.21.15.32:0/779980675 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7feefc068380 0x7feefc06a840 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fef100ad000 tx=0x7fef100a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.178+0000 7fef17400640 1 -- 172.21.15.32:0/779980675 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fef08055c00 con 0x7fef200b47f0 2024-04-17T09:58:45.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.382+0000 7fef25a00640 1 -- 172.21.15.32:0/779980675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fef20068a90 con 0x7fef200b47f0 2024-04-17T09:58:45.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.382+0000 7fef17400640 1 -- 172.21.15.32:0/779980675 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fef08059f00 con 0x7fef200b47f0 2024-04-17T09:58:45.384 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:58:45.384 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-17T09:58:45.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.382+0000 7fef25a00640 1 -- 172.21.15.32:0/779980675 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7feefc068380 msgr2=0x7feefc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:45.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.382+0000 7fef25a00640 1 --2- 172.21.15.32:0/779980675 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7feefc068380 0x7feefc06a840 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fef100ad000 tx=0x7fef100a6000 comp rx=0 tx=0).stop 2024-04-17T09:58:45.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.382+0000 7fef25a00640 1 -- 172.21.15.32:0/779980675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef200b47f0 msgr2=0x7fef200ae740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:45.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.382+0000 7fef25a00640 1 --2- 172.21.15.32:0/779980675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef200b47f0 0x7fef200ae740 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fef0800c370 tx=0x7fef0800c830 comp rx=0 tx=0).stop 2024-04-17T09:58:45.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.382+0000 7fef25a00640 1 -- 172.21.15.32:0/779980675 shutdown_connections 2024-04-17T09:58:45.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.382+0000 7fef25a00640 1 --2- 172.21.15.32:0/779980675 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7feefc068380 0x7feefc06a840 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.382+0000 7fef25a00640 1 --2- 172.21.15.32:0/779980675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef2015e810 0x7fef200cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.382+0000 7fef25a00640 1 --2- 172.21.15.32:0/779980675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef200b47f0 0x7fef200ae740 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.382+0000 7fef25a00640 1 --2- 172.21.15.32:0/779980675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef200b3c50 0x7fef200ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.382+0000 7fef25a00640 1 -- 172.21.15.32:0/779980675 >> 172.21.15.32:0/779980675 conn(0x7fef200d0d60 msgr2=0x7fef200d1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:45.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.382+0000 7fef25a00640 1 -- 172.21.15.32:0/779980675 shutdown_connections 2024-04-17T09:58:45.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.382+0000 7fef25a00640 1 -- 172.21.15.32:0/779980675 wait complete. 2024-04-17T09:58:45.406 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-17T09:58:45.406 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T09:58:45.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.546+0000 7f87cee00640 1 Processor -- start 2024-04-17T09:58:45.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.546+0000 7f87cee00640 1 -- start start 2024-04-17T09:58:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.550+0000 7f87cee00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87c80cd580 0x7f87c8167560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.550+0000 7f87cee00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87c80cdac0 con 0x7f87c8145f30 2024-04-17T09:58:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.550+0000 7f87cee00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87c815cdf0 con 0x7f87c8072500 2024-04-17T09:58:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.550+0000 7f87cee00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f87c806f400 con 0x7f87c80cd580 2024-04-17T09:58:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.550+0000 7f87c7e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f87c8145f30 0x7f87c8071df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38966/0 (socket says 172.21.15.32:38966) 2024-04-17T09:58:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.550+0000 7f87c7e00640 1 -- 172.21.15.32:0/722791261 learned_addr learned my addr 172.21.15.32:0/722791261 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:58:45.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.550+0000 7f87cce00640 1 --2- 172.21.15.32:0/722791261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87c80cd580 0x7f87c8167560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.550+0000 7f87c7400640 1 -- 172.21.15.32:0/722791261 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3640768009 0 0) 0x7f87c80cdac0 con 0x7f87c8145f30 2024-04-17T09:58:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.550+0000 7f87c7400640 1 -- 172.21.15.32:0/722791261 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87ac003530 con 0x7f87c8145f30 2024-04-17T09:58:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.550+0000 7f87c7400640 1 -- 172.21.15.32:0/722791261 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2767040050 0 0) 0x7f87c815cdf0 con 0x7f87c8072500 2024-04-17T09:58:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.550+0000 7f87cce00640 1 -- 172.21.15.32:0/722791261 >> v1:172.21.15.144:6789/0 conn(0x7f87c8072500 legacy=0x7f87c80cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.550+0000 7f87cce00640 1 -- 172.21.15.32:0/722791261 >> v1:172.21.15.32:6789/0 conn(0x7f87c8145f30 legacy=0x7f87c8071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.550+0000 7f87cce00640 1 -- 172.21.15.32:0/722791261 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c815d0d0 con 0x7f87c80cd580 2024-04-17T09:58:45.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.554+0000 7f87cce00640 1 --2- 172.21.15.32:0/722791261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87c80cd580 0x7f87c8167560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009dc0 tx=0x7f87c001f170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2773950e0cafebd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:45.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.554+0000 7f87c7400640 1 -- 172.21.15.32:0/722791261 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c00b4070 con 0x7f87c80cd580 2024-04-17T09:58:45.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.554+0000 7f87c7400640 1 -- 172.21.15.32:0/722791261 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c00af600 con 0x7f87c80cd580 2024-04-17T09:58:45.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.554+0000 7f87c7400640 1 -- 172.21.15.32:0/722791261 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c00af9d0 con 0x7f87c80cd580 2024-04-17T09:58:45.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.554+0000 7f87cee00640 1 -- 172.21.15.32:0/722791261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87c80cd580 msgr2=0x7f87c8167560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:45.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.554+0000 7f87cee00640 1 --2- 172.21.15.32:0/722791261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87c80cd580 0x7f87c8167560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009dc0 tx=0x7f87c001f170 comp rx=0 tx=0).stop 2024-04-17T09:58:45.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.554+0000 7f87cee00640 1 -- 172.21.15.32:0/722791261 shutdown_connections 2024-04-17T09:58:45.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.554+0000 7f87cee00640 1 --2- 172.21.15.32:0/722791261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87c80cd580 0x7f87c8167560 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.554+0000 7f87cee00640 1 -- 172.21.15.32:0/722791261 >> 172.21.15.32:0/722791261 conn(0x7f87c806ba30 msgr2=0x7f87c806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:45.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.554+0000 7f87cee00640 1 -- 172.21.15.32:0/722791261 shutdown_connections 2024-04-17T09:58:45.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.554+0000 7f87cee00640 1 -- 172.21.15.32:0/722791261 wait complete. 2024-04-17T09:58:45.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.554+0000 7f87cee00640 1 Processor -- start 2024-04-17T09:58:45.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.554+0000 7f87cee00640 1 -- start start 2024-04-17T09:58:45.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87cee00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87c8072500 0x7f87c8140590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:45.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87cee00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87c80cd580 0x7f87c80b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:45.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87cee00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c8145f30 0x7f87c80b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:45.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87cee00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f87c8146310 con 0x7f87c8145f30 2024-04-17T09:58:45.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87cee00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f87c815db60 con 0x7f87c80cd580 2024-04-17T09:58:45.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87cee00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f87c815d440 con 0x7f87c8072500 2024-04-17T09:58:45.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87cce00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c8145f30 0x7f87c80b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:45.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87cce00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c8145f30 0x7f87c80b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38836/0 (socket says 172.21.15.32:38836) 2024-04-17T09:58:45.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87cce00640 1 -- 172.21.15.32:0/1867522105 learned_addr learned my addr 172.21.15.32:0/1867522105 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:45.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87c7e00640 1 --2- 172.21.15.32:0/1867522105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87c8072500 0x7f87c8140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:45.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87bf400640 1 --2- 172.21.15.32:0/1867522105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87c80cd580 0x7f87c80b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87cce00640 1 -- 172.21.15.32:0/1867522105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87c8072500 msgr2=0x7f87c8140590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87cce00640 1 --2- 172.21.15.32:0/1867522105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87c8072500 0x7f87c8140590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87cce00640 1 -- 172.21.15.32:0/1867522105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87c80cd580 msgr2=0x7f87c80b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87cce00640 1 --2- 172.21.15.32:0/1867522105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87c80cd580 0x7f87c80b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87cce00640 1 -- 172.21.15.32:0/1867522105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c80ca540 con 0x7f87c8145f30 2024-04-17T09:58:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87bf400640 1 --2- 172.21.15.32:0/1867522105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87c80cd580 0x7f87c80b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:58:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87c7e00640 1 --2- 172.21.15.32:0/1867522105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87c8072500 0x7f87c8140590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:58:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87cce00640 1 --2- 172.21.15.32:0/1867522105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c8145f30 0x7f87c80b4450 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f87c00b2040 tx=0x7f87c001bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87c4c00640 1 -- 172.21.15.32:0/1867522105 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c00b4040 con 0x7f87c8145f30 2024-04-17T09:58:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87c4c00640 1 -- 172.21.15.32:0/1867522105 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c00af400 con 0x7f87c8145f30 2024-04-17T09:58:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87cee00640 1 -- 172.21.15.32:0/1867522105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c80ca820 con 0x7f87c8145f30 2024-04-17T09:58:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87c4c00640 1 -- 172.21.15.32:0/1867522105 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c0009450 con 0x7f87c8145f30 2024-04-17T09:58:45.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87cee00640 1 -- 172.21.15.32:0/1867522105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c80cade0 con 0x7f87c8145f30 2024-04-17T09:58:45.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87cee00640 1 -- 172.21.15.32:0/1867522105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87c8068a90 con 0x7f87c8145f30 2024-04-17T09:58:45.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.558+0000 7f87c4c00640 1 -- 172.21.15.32:0/1867522105 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f87c00c3020 con 0x7f87c8145f30 2024-04-17T09:58:45.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.562+0000 7f87c4c00640 1 --2- 172.21.15.32:0/1867522105 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f87a4068380 0x7f87a406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:45.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.562+0000 7f87c4c00640 1 -- 172.21.15.32:0/1867522105 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f87c0124f60 con 0x7f87c8145f30 2024-04-17T09:58:45.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.562+0000 7f87c7e00640 1 --2- 172.21.15.32:0/1867522105 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f87a4068380 0x7f87a406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:45.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.562+0000 7f87c7e00640 1 --2- 172.21.15.32:0/1867522105 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f87a4068380 0x7f87a406a840 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f87b8001210 tx=0x7f87b8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:45.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.566+0000 7f87c4c00640 1 -- 172.21.15.32:0/1867522105 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f87c0018cd0 con 0x7f87c8145f30 2024-04-17T09:58:45.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.770+0000 7f87cee00640 1 -- 172.21.15.32:0/1867522105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f87c8072f90 con 0x7f87c8145f30 2024-04-17T09:58:45.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.770+0000 7f87c4c00640 1 -- 172.21.15.32:0/1867522105 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) ==== 53+0+73 (secure 0 0 0) 0x7f87c00af5d0 con 0x7f87c8145f30 2024-04-17T09:58:45.768 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T09:58:45.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.770+0000 7f87cee00640 1 -- 172.21.15.32:0/1867522105 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f87a4068380 msgr2=0x7f87a406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:45.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.770+0000 7f87cee00640 1 --2- 172.21.15.32:0/1867522105 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f87a4068380 0x7f87a406a840 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f87b8001210 tx=0x7f87b8009040 comp rx=0 tx=0).stop 2024-04-17T09:58:45.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.770+0000 7f87cee00640 1 -- 172.21.15.32:0/1867522105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c8145f30 msgr2=0x7f87c80b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:45.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.770+0000 7f87cee00640 1 --2- 172.21.15.32:0/1867522105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c8145f30 0x7f87c80b4450 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f87c00b2040 tx=0x7f87c001bf10 comp rx=0 tx=0).stop 2024-04-17T09:58:45.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.770+0000 7f87cee00640 1 -- 172.21.15.32:0/1867522105 shutdown_connections 2024-04-17T09:58:45.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.770+0000 7f87cee00640 1 --2- 172.21.15.32:0/1867522105 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f87a4068380 0x7f87a406a840 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.770+0000 7f87cee00640 1 --2- 172.21.15.32:0/1867522105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87c8145f30 0x7f87c80b4450 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.770+0000 7f87cee00640 1 --2- 172.21.15.32:0/1867522105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87c80cd580 0x7f87c80b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.770+0000 7f87cee00640 1 --2- 172.21.15.32:0/1867522105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87c8072500 0x7f87c8140590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:45.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.770+0000 7f87cee00640 1 -- 172.21.15.32:0/1867522105 >> 172.21.15.32:0/1867522105 conn(0x7f87c806ba30 msgr2=0x7f87c806be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:45.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.770+0000 7f87cee00640 1 -- 172.21.15.32:0/1867522105 shutdown_connections 2024-04-17T09:58:45.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:45.770+0000 7f87cee00640 1 -- 172.21.15.32:0/1867522105 wait complete. 2024-04-17T09:58:45.788 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T09:58:45.788 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T09:58:45.789 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-17T09:58:45.789 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T09:58:45.789 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T09:58:45.789 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T09:58:45.789 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T09:58:45.789 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T09:58:45.789 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:58:45.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:58:45.789 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-17T09:58:45.799 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T09:58:45.799 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T09:58:45.799 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T09:58:45.799 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:58:45.799 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-17T09:58:45.799 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:58:45.799 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:58:45.799 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-17T09:58:45.800 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-17T09:58:45.800 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T09:58:45.800 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:58:45.800 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-17T09:58:45.800 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-17T09:58:45.800 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42128sec preferred_lft 42128sec 2024-04-17T09:58:45.800 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2024-04-17T09:58:45.800 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:58:45.800 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:58:45.800 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-17T09:58:45.800 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-17T09:58:45.800 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:58:45.800 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:58:45.800 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-17T09:58:45.801 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T09:58:45.801 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:cd:f4:aa:5b brd ff:ff:ff:ff:ff:ff 2024-04-17T09:58:45.801 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T09:58:45.801 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:58:45.801 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T09:58:45.801 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T09:58:45.801 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T09:58:45.801 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:58:45.801 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T09:58:45.801 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:58:45.801 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T09:58:45.801 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:58:45.802 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T09:58:45.802 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:58:45.802 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T09:58:45.802 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:58:45.802 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T09:58:45.802 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:58:45.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:58:45.802 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T09:58:45.860 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-17T09:58:45.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:58:45.862 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-17T09:58:45.915 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-17T09:58:45.915 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T09:58:45.915 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-17T09:58:45.915 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T09:58:45.915 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-17T09:58:45.915 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T09:58:45.916 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:58:45.916 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T09:58:45.916 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-17T09:58:45.916 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-17T09:58:45.916 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-17T09:58:45.916 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T09:58:46.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:58:46.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:58:46.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:58:46.054 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-17T09:58:46.063 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T09:58:46.064 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T09:58:46.064 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:58:46.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:58:46.064 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-17T09:58:46.127 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T09:58:46.127 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:58:46.127 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T09:58:46.183 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:58:46.183 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:58:46.183 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T09:58:46.240 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T09:58:46.308 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:58:46.363 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T09:58:46.363 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T09:58:46.364 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T09:58:46.364 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T09:58:46.364 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T09:58:46.364 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T09:58:46.364 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T09:58:46.364 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T09:58:46.364 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T09:58:46.364 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T09:58:46.364 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T09:58:46.364 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T09:58:46.364 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:58:46.364 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T09:58:46.364 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T09:58:46.364 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T09:58:46.365 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T09:58:46.365 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T09:58:46.365 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T09:58:46.365 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T09:58:46.365 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:58:46.365 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T09:58:46.365 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T09:58:46.365 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T09:58:46.365 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T09:58:46.365 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T09:58:46.365 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T09:58:46.365 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T09:58:46.365 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T09:58:46.365 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:58:46.366 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:58:46.366 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:58:46.366 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:58:46.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:58:46.367 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T09:58:46.422 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T09:58:46.422 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T09:58:46.469 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T09:58:46.490 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:58:46.543 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T09:58:46.543 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T09:58:46.543 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T09:58:46.543 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T09:58:46.543 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T09:58:46.544 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T09:58:46.544 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T09:58:46.544 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T09:58:46.544 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T09:58:46.544 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T09:58:46.544 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T09:58:46.544 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T09:58:46.544 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:58:46.544 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T09:58:46.544 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T09:58:46.544 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T09:58:46.544 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T09:58:46.544 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T09:58:46.544 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T09:58:46.544 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T09:58:46.545 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:58:46.545 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T09:58:46.545 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T09:58:46.545 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T09:58:46.545 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T09:58:46.545 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T09:58:46.545 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T09:58:46.545 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T09:58:46.545 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T09:58:46.545 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:58:46.545 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:58:46.545 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:58:46.545 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:58:46.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:58:46.546 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T09:58:46.564 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2024-04-17T09:58:46.566+0000 7f9983818580 -1 init, newargv = 0x55e55deb2c40 newargc=15 2024-04-17T09:58:46.564 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[43451]: starting ceph client 2024-04-17T09:58:46.571 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[43451]: starting fuse 2024-04-17T09:58:46.574 INFO:teuthology.orchestra.run.smithi032.stdout:44 2024-04-17T09:58:46.574 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-17T09:58:46.575 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2024-04-17T09:58:46.575 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2024-04-17T09:58:46.575 DEBUG:teuthology.orchestra.run.smithi032:> import re 2024-04-17T09:58:46.575 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T09:58:46.575 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2024-04-17T09:58:46.575 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:58:46.575 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2024-04-17T09:58:46.575 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T09:58:46.575 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2024-04-17T09:58:46.575 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T09:58:46.575 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:58:46.575 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2024-04-17T09:58:46.575 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2024-04-17T09:58:46.575 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2024-04-17T09:58:46.576 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2024-04-17T09:58:46.576 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:58:46.576 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2024-04-17T09:58:46.576 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T09:58:46.576 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T09:58:46.576 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T09:58:46.576 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2024-04-17T09:58:46.576 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2024-04-17T09:58:46.576 DEBUG:teuthology.orchestra.run.smithi032:> return f 2024-04-17T09:58:46.576 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T09:58:46.576 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:58:46.576 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2024-04-17T09:58:46.576 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T09:58:46.701 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.43451.asok 2024-04-17T09:58:46.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:58:46.712 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43451.asok 2024-04-17T09:58:46.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:58:46.713 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43451.asok status 2024-04-17T09:58:46.885 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:58:46.885 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2024-04-17T09:58:46.885 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-17T09:58:46.885 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T09:58:46.885 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2024-04-17T09:58:46.885 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2024-04-17T09:58:46.885 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T09:58:46.886 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "43451", 2024-04-17T09:58:46.886 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2024-04-17T09:58:46.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:58:46.886 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2024-04-17T09:58:46.886 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2024-04-17T09:58:46.886 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 7347, 2024-04-17T09:58:46.886 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2024-04-17T09:58:46.886 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2024-04-17T09:58:46.886 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2024-04-17T09:58:46.886 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 7347 2024-04-17T09:58:46.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:58:46.886 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T09:58:46.886 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T09:58:46.886 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T09:58:46.887 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1401618253 2024-04-17T09:58:46.887 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:58:46.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:58:46.887 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T09:58:46.887 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T09:58:46.887 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T09:58:46.887 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1401618253 2024-04-17T09:58:46.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:58:46.887 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.7347 192.168.0.1:0/1401618253", 2024-04-17T09:58:46.887 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/1401618253", 2024-04-17T09:58:46.887 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2024-04-17T09:58:46.887 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 50, 2024-04-17T09:58:46.887 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 71, 2024-04-17T09:58:46.887 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2024-04-17T09:58:46.888 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2024-04-17T09:58:46.888 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs" 2024-04-17T09:58:46.888 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:58:46.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:58:46.900 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2024-04-17T09:58:46.900 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T09:58:46.909 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2024-04-17T09:58:46.909 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T09:58:46.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:58:46.910 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T09:58:46.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:58:46.974 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-17T09:58:47.035 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-17T09:58:47.035 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-17T09:58:47.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.130+0000 7fb8be2d1080 1 Processor -- start 2024-04-17T09:58:47.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.130+0000 7fb8be2d1080 1 -- start start 2024-04-17T09:58:47.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8be2d1080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8a0002830 0x5567f9f64560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:47.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8be2d1080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5567f9ebd500 0x5567f9ebc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:47.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8be2d1080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f9f6b740 0x5567f9f6bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:47.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8be2d1080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5567f9edb5f0 con 0x7fb8a0002830 2024-04-17T09:58:47.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8be2d1080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5567f9e34f90 con 0x5567f9f6b740 2024-04-17T09:58:47.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8be2d1080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5567f9ec1760 con 0x5567f9ebd500 2024-04-17T09:58:47.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8bb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8a0002830 0x5567f9f64560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:47.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8bb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8a0002830 0x5567f9f64560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38854/0 (socket says 172.21.15.32:38854) 2024-04-17T09:58:47.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8bb400640 1 -- 172.21.15.32:0/2328951167 learned_addr learned my addr 172.21.15.32:0/2328951167 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:47.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8baa00640 1 --2- 172.21.15.32:0/2328951167 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5567f9ebd500 0x5567f9ebc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:47.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8bbe00640 1 --2- 172.21.15.32:0/2328951167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f9f6b740 0x5567f9f6bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8bb400640 1 -- 172.21.15.32:0/2328951167 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5567f9ebd500 msgr2=0x5567f9ebc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8bb400640 1 --2- 172.21.15.32:0/2328951167 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5567f9ebd500 0x5567f9ebc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8bb400640 1 -- 172.21.15.32:0/2328951167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f9f6b740 msgr2=0x5567f9f6bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8bb400640 1 --2- 172.21.15.32:0/2328951167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f9f6b740 0x5567f9f6bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8bb400640 1 -- 172.21.15.32:0/2328951167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567f9db9950 con 0x7fb8a0002830 2024-04-17T09:58:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8bbe00640 1 --2- 172.21.15.32:0/2328951167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f9f6b740 0x5567f9f6bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:58:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8bb400640 1 --2- 172.21.15.32:0/2328951167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8a0002830 0x5567f9f64560 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00d570 tx=0x7fb8ac00da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4273f2f08ad695c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8b9600640 1 -- 172.21.15.32:0/2328951167 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac00b990 con 0x7fb8a0002830 2024-04-17T09:58:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8b9600640 1 -- 172.21.15.32:0/2328951167 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8ac012030 con 0x7fb8a0002830 2024-04-17T09:58:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8b9600640 1 -- 172.21.15.32:0/2328951167 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac00bd50 con 0x7fb8a0002830 2024-04-17T09:58:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8be2d1080 1 -- 172.21.15.32:0/2328951167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8a0002830 msgr2=0x5567f9f64560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8be2d1080 1 --2- 172.21.15.32:0/2328951167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8a0002830 0x5567f9f64560 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00d570 tx=0x7fb8ac00da30 comp rx=0 tx=0).stop 2024-04-17T09:58:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8be2d1080 1 -- 172.21.15.32:0/2328951167 shutdown_connections 2024-04-17T09:58:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8be2d1080 1 --2- 172.21.15.32:0/2328951167 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5567f9ebd500 0x5567f9ebc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8be2d1080 1 --2- 172.21.15.32:0/2328951167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f9f6b740 0x5567f9f6bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8be2d1080 1 --2- 172.21.15.32:0/2328951167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8a0002830 0x5567f9f64560 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8be2d1080 1 -- 172.21.15.32:0/2328951167 >> 172.21.15.32:0/2328951167 conn(0x5567f9e8e620 msgr2=0x5567f9e8ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8be2d1080 1 -- 172.21.15.32:0/2328951167 shutdown_connections 2024-04-17T09:58:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8be2d1080 1 -- 172.21.15.32:0/2328951167 wait complete. 2024-04-17T09:58:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8be2d1080 1 Processor -- start 2024-04-17T09:58:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8be2d1080 1 -- start start 2024-04-17T09:58:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8be2d1080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f9ebd500 0x5567f9ebc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8be2d1080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5567f9f6b740 0x5567f9ed1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8be2d1080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8a0002830 0x5567f9ed2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8be2d1080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5567f9f66760 con 0x7fb8a0002830 2024-04-17T09:58:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8be2d1080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5567f9f66a00 con 0x5567f9ebd500 2024-04-17T09:58:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8be2d1080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5567f9f64aa0 con 0x5567f9f6b740 2024-04-17T09:58:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8bbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8a0002830 0x5567f9ed2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8bbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8a0002830 0x5567f9ed2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38868/0 (socket says 172.21.15.32:38868) 2024-04-17T09:58:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8bbe00640 1 -- 172.21.15.32:0/824422643 learned_addr learned my addr 172.21.15.32:0/824422643 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8baa00640 1 --2- 172.21.15.32:0/824422643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5567f9f6b740 0x5567f9ed1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.134+0000 7fb8bb400640 1 --2- 172.21.15.32:0/824422643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f9ebd500 0x5567f9ebc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.138+0000 7fb8bbe00640 1 -- 172.21.15.32:0/824422643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5567f9f6b740 msgr2=0x5567f9ed1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.138+0000 7fb8bbe00640 1 --2- 172.21.15.32:0/824422643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5567f9f6b740 0x5567f9ed1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.138+0000 7fb8bbe00640 1 -- 172.21.15.32:0/824422643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f9ebd500 msgr2=0x5567f9ebc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.138+0000 7fb8bbe00640 1 --2- 172.21.15.32:0/824422643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f9ebd500 0x5567f9ebc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.138+0000 7fb8bbe00640 1 -- 172.21.15.32:0/824422643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567f9ed28a0 con 0x7fb8a0002830 2024-04-17T09:58:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.138+0000 7fb8bbe00640 1 --2- 172.21.15.32:0/824422643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8a0002830 0x5567f9ed2080 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x5567f9ed7a90 tx=0x7fb8b4011990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.138+0000 7fb8b3e00640 1 -- 172.21.15.32:0/824422643 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4018030 con 0x7fb8a0002830 2024-04-17T09:58:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.138+0000 7fb8b3e00640 1 -- 172.21.15.32:0/824422643 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b4013740 con 0x7fb8a0002830 2024-04-17T09:58:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.138+0000 7fb8be2d1080 1 -- 172.21.15.32:0/824422643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5567f9ed2b40 con 0x7fb8a0002830 2024-04-17T09:58:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.138+0000 7fb8b3e00640 1 -- 172.21.15.32:0/824422643 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4013910 con 0x7fb8a0002830 2024-04-17T09:58:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.138+0000 7fb8be2d1080 1 -- 172.21.15.32:0/824422643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567f9ed3080 con 0x7fb8a0002830 2024-04-17T09:58:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.138+0000 7fb8b3e00640 1 -- 172.21.15.32:0/824422643 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb8b401b030 con 0x7fb8a0002830 2024-04-17T09:58:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.138+0000 7fb8b3e00640 1 --2- 172.21.15.32:0/824422643 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb89c066d90 0x7fb89c069250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.138+0000 7fb8b3e00640 1 -- 172.21.15.32:0/824422643 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fb8b4085a40 con 0x7fb8a0002830 2024-04-17T09:58:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.138+0000 7fb8bb400640 1 --2- 172.21.15.32:0/824422643 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb89c066d90 0x7fb89c069250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.138+0000 7fb8be2d1080 1 --2- 172.21.15.32:0/824422643 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x5567f9ebce90 0x5567fa002af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.138+0000 7fb8be2d1080 1 -- 172.21.15.32:0/824422643 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) -- 0x5567f9ed4a20 con 0x5567f9ebce90 2024-04-17T09:58:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.142+0000 7fb8baa00640 1 --2- 172.21.15.32:0/824422643 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x5567f9ebce90 0x5567fa002af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.142+0000 7fb8bb400640 1 --2- 172.21.15.32:0/824422643 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb89c066d90 0x7fb89c069250 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00da70 tx=0x7fb8ac004000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.142+0000 7fb8baa00640 1 --2- 172.21.15.32:0/824422643 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x5567f9ebce90 0x5567fa002af0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.142+0000 7fb8baa00640 1 -- 172.21.15.32:0/824422643 <== osd.4 v2:172.21.15.144:6809/4076371964 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v71'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb8a80a94c0 con 0x5567f9ebce90 2024-04-17T09:58:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.142+0000 7fb8be2d1080 1 -- 172.21.15.32:0/824422643 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x5567f9ebce90 msgr2=0x5567fa002af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.142+0000 7fb8be2d1080 1 --2- 172.21.15.32:0/824422643 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x5567f9ebce90 0x5567fa002af0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.142+0000 7fb8be2d1080 1 -- 172.21.15.32:0/824422643 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb89c066d90 msgr2=0x7fb89c069250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.142+0000 7fb8be2d1080 1 --2- 172.21.15.32:0/824422643 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb89c066d90 0x7fb89c069250 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00da70 tx=0x7fb8ac004000 comp rx=0 tx=0).stop 2024-04-17T09:58:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.142+0000 7fb8be2d1080 1 -- 172.21.15.32:0/824422643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8a0002830 msgr2=0x5567f9ed2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.142+0000 7fb8be2d1080 1 --2- 172.21.15.32:0/824422643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8a0002830 0x5567f9ed2080 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x5567f9ed7a90 tx=0x7fb8b4011990 comp rx=0 tx=0).stop 2024-04-17T09:58:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.142+0000 7fb8bbe00640 1 -- 172.21.15.32:0/824422643 reap_dead start 2024-04-17T09:58:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.142+0000 7fb8be2d1080 1 -- 172.21.15.32:0/824422643 shutdown_connections 2024-04-17T09:58:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.142+0000 7fb8be2d1080 1 -- 172.21.15.32:0/824422643 >> 172.21.15.32:0/824422643 conn(0x5567f9e8e620 msgr2=0x5567f9e8ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.142+0000 7fb8be2d1080 1 -- 172.21.15.32:0/824422643 shutdown_connections 2024-04-17T09:58:47.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.142+0000 7fb8be2d1080 1 -- 172.21.15.32:0/824422643 wait complete. 2024-04-17T09:58:47.147 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T09:58:47.147 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T09:58:47.233 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T09:58:47.300 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T09:58:47.300 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T09:58:47.300 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:58:47.300 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:58:47.310 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:58:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.498+0000 7f3b99c00640 1 Processor -- start 2024-04-17T09:58:47.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.498+0000 7f3b99c00640 1 -- start start 2024-04-17T09:58:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.498+0000 7f3b99c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3b941481e0 0x7f3b94071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.498+0000 7f3b99c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b94072330 0x7f3b940b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.498+0000 7f3b99c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b9406ef70 con 0x7f3b94144f20 2024-04-17T09:58:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.498+0000 7f3b99c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3b940aed20 con 0x7f3b941481e0 2024-04-17T09:58:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.498+0000 7f3b99c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3b940af000 con 0x7f3b94072330 2024-04-17T09:58:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.498+0000 7f3b8a000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3b941481e0 0x7f3b94071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.498+0000 7f3b92a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3b94144f20 0x7f3b9406e860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38990/0 (socket says 172.21.15.32:38990) 2024-04-17T09:58:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.498+0000 7f3b92a00640 1 -- 172.21.15.32:0/1847352731 learned_addr learned my addr 172.21.15.32:0/1847352731 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:58:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.498+0000 7f3b93400640 1 --2- 172.21.15.32:0/1847352731 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b94072330 0x7f3b940b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.498+0000 7f3b8a000640 1 -- 172.21.15.32:0/1847352731 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b94072330 msgr2=0x7f3b940b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.498+0000 7f3b8a000640 1 --2- 172.21.15.32:0/1847352731 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b94072330 0x7f3b940b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.498+0000 7f3b8a000640 1 -- 172.21.15.32:0/1847352731 >> v1:172.21.15.32:6789/0 conn(0x7f3b94144f20 legacy=0x7f3b9406e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.498+0000 7f3b8a000640 1 -- 172.21.15.32:0/1847352731 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b940af2e0 con 0x7f3b941481e0 2024-04-17T09:58:47.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.502+0000 7f3b8a000640 1 --2- 172.21.15.32:0/1847352731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b941481e0 0x7f3b94071df0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0097c0 tx=0x7f3b7c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ea2d13f2c3f8ab server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:47.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.502+0000 7f3b92000640 1 -- 172.21.15.32:0/1847352731 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c0b0070 con 0x7f3b941481e0 2024-04-17T09:58:47.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.502+0000 7f3b92000640 1 -- 172.21.15.32:0/1847352731 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b7c0ab600 con 0x7f3b941481e0 2024-04-17T09:58:47.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.502+0000 7f3b92000640 1 -- 172.21.15.32:0/1847352731 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c0ab9d0 con 0x7f3b941481e0 2024-04-17T09:58:47.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.502+0000 7f3b99c00640 1 -- 172.21.15.32:0/1847352731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b941481e0 msgr2=0x7f3b94071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:47.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.502+0000 7f3b99c00640 1 --2- 172.21.15.32:0/1847352731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b941481e0 0x7f3b94071df0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0097c0 tx=0x7f3b7c01c4a0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.502+0000 7f3b99c00640 1 -- 172.21.15.32:0/1847352731 shutdown_connections 2024-04-17T09:58:47.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.502+0000 7f3b99c00640 1 --2- 172.21.15.32:0/1847352731 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b94072330 0x7f3b940b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.502+0000 7f3b99c00640 1 --2- 172.21.15.32:0/1847352731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b941481e0 0x7f3b94071df0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.502+0000 7f3b99c00640 1 -- 172.21.15.32:0/1847352731 >> 172.21.15.32:0/1847352731 conn(0x7f3b9406ba30 msgr2=0x7f3b9406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:47.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.502+0000 7f3b99c00640 1 -- 172.21.15.32:0/1847352731 shutdown_connections 2024-04-17T09:58:47.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.502+0000 7f3b99c00640 1 -- 172.21.15.32:0/1847352731 wait complete. 2024-04-17T09:58:47.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b99c00640 1 Processor -- start 2024-04-17T09:58:47.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b99c00640 1 -- start start 2024-04-17T09:58:47.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b99c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b94072330 0x7f3b9415b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:47.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b99c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b94144f20 0x7f3b940ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:47.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b99c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b941481e0 0x7f3b9415ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:47.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b99c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3b940b4360 con 0x7f3b94144f20 2024-04-17T09:58:47.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b99c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3b940b41e0 con 0x7f3b941481e0 2024-04-17T09:58:47.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b99c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3b9416b5d0 con 0x7f3b94072330 2024-04-17T09:58:47.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b8a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b94144f20 0x7f3b940ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:47.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b8a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b94144f20 0x7f3b940ceba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38886/0 (socket says 172.21.15.32:38886) 2024-04-17T09:58:47.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b8a000640 1 -- 172.21.15.32:0/8415780 learned_addr learned my addr 172.21.15.32:0/8415780 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:47.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b93400640 1 --2- 172.21.15.32:0/8415780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b941481e0 0x7f3b9415ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:47.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b92a00640 1 --2- 172.21.15.32:0/8415780 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b94072330 0x7f3b9415b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:47.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b8a000640 1 -- 172.21.15.32:0/8415780 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b94072330 msgr2=0x7f3b9415b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:47.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b8a000640 1 --2- 172.21.15.32:0/8415780 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b94072330 0x7f3b9415b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b8a000640 1 -- 172.21.15.32:0/8415780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b941481e0 msgr2=0x7f3b9415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:47.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b8a000640 1 --2- 172.21.15.32:0/8415780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b941481e0 0x7f3b9415ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b8a000640 1 -- 172.21.15.32:0/8415780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b940cf280 con 0x7f3b94144f20 2024-04-17T09:58:47.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b93400640 1 --2- 172.21.15.32:0/8415780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b941481e0 0x7f3b9415ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:58:47.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b8a000640 1 --2- 172.21.15.32:0/8415780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b94144f20 0x7f3b940ceba0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c01a0d0 tx=0x7f3b7c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:47.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b8b400640 1 -- 172.21.15.32:0/8415780 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c0ba070 con 0x7f3b94144f20 2024-04-17T09:58:47.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b8b400640 1 -- 172.21.15.32:0/8415780 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b7c0b5460 con 0x7f3b94144f20 2024-04-17T09:58:47.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b99c00640 1 -- 172.21.15.32:0/8415780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b9415c560 con 0x7f3b94144f20 2024-04-17T09:58:47.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b8b400640 1 -- 172.21.15.32:0/8415780 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c0b0040 con 0x7f3b94144f20 2024-04-17T09:58:47.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b99c00640 1 -- 172.21.15.32:0/8415780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b9415cb20 con 0x7f3b94144f20 2024-04-17T09:58:47.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.506+0000 7f3b99c00640 1 -- 172.21.15.32:0/8415780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b94068a90 con 0x7f3b94144f20 2024-04-17T09:58:47.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.510+0000 7f3b8b400640 1 -- 172.21.15.32:0/8415780 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3b7c0bd070 con 0x7f3b94144f20 2024-04-17T09:58:47.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.510+0000 7f3b8b400640 1 --2- 172.21.15.32:0/8415780 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3b70068380 0x7f3b7006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:47.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.510+0000 7f3b8b400640 1 -- 172.21.15.32:0/8415780 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f3b7c120f80 con 0x7f3b94144f20 2024-04-17T09:58:47.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.510+0000 7f3b92a00640 1 --2- 172.21.15.32:0/8415780 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3b70068380 0x7f3b7006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:47.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.510+0000 7f3b92a00640 1 --2- 172.21.15.32:0/8415780 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3b70068380 0x7f3b7006a840 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3b84000f70 tx=0x7f3b84009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:47.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.514+0000 7f3b8b400640 1 -- 172.21.15.32:0/8415780 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3b7c016090 con 0x7f3b94144f20 2024-04-17T09:58:47.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.678+0000 7f3b99c00640 1 -- 172.21.15.32:0/8415780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b940ccc90 con 0x7f3b94144f20 2024-04-17T09:58:47.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.678+0000 7f3b8b400640 1 -- 172.21.15.32:0/8415780 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16904 (secure 0 0 0) 0x7f3b7c0f14a0 con 0x7f3b94144f20 2024-04-17T09:58:47.679 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:58:47.679 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":71,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:57:55.831704+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:57:09.948964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-17T09:57:10.512460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.a","mappings":[{"from":6,"to":0}]},{"pgid":"11.3a","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6836/663537244":"2024-04-18T09:57:30.746884+0000","172.21.15.32:6834/663537244":"2024-04-18T09:57:30.746884+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:58:47.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.682+0000 7f3b99c00640 1 -- 172.21.15.32:0/8415780 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3b70068380 msgr2=0x7f3b7006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:47.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.682+0000 7f3b99c00640 1 --2- 172.21.15.32:0/8415780 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3b70068380 0x7f3b7006a840 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3b84000f70 tx=0x7f3b84009040 comp rx=0 tx=0).stop 2024-04-17T09:58:47.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.682+0000 7f3b99c00640 1 -- 172.21.15.32:0/8415780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b94144f20 msgr2=0x7f3b940ceba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:47.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.682+0000 7f3b99c00640 1 --2- 172.21.15.32:0/8415780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b94144f20 0x7f3b940ceba0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c01a0d0 tx=0x7f3b7c019240 comp rx=0 tx=0).stop 2024-04-17T09:58:47.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.682+0000 7f3b99c00640 1 -- 172.21.15.32:0/8415780 shutdown_connections 2024-04-17T09:58:47.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.682+0000 7f3b99c00640 1 --2- 172.21.15.32:0/8415780 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3b70068380 0x7f3b7006a840 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.682+0000 7f3b99c00640 1 --2- 172.21.15.32:0/8415780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b941481e0 0x7f3b9415ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.682+0000 7f3b99c00640 1 --2- 172.21.15.32:0/8415780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b94144f20 0x7f3b940ceba0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.682+0000 7f3b99c00640 1 --2- 172.21.15.32:0/8415780 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b94072330 0x7f3b9415b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.682+0000 7f3b99c00640 1 -- 172.21.15.32:0/8415780 >> 172.21.15.32:0/8415780 conn(0x7f3b9406ba30 msgr2=0x7f3b94141b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.682+0000 7f3b99c00640 1 -- 172.21.15.32:0/8415780 shutdown_connections 2024-04-17T09:58:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.682+0000 7f3b99c00640 1 -- 172.21.15.32:0/8415780 wait complete. 2024-04-17T09:58:47.700 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:57:09.948964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:57:10.512460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:58:47.700 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:58:47.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.846+0000 7fedc5800640 1 Processor -- start 2024-04-17T09:58:47.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.846+0000 7fedc5800640 1 -- start start 2024-04-17T09:58:47.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.846+0000 7fedc5800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedc006ba30 0x7fedc006be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:47.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.846+0000 7fedc5800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fedc00cf370 0x7fedc00cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:47.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.846+0000 7fedc5800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fedc0160350 0x7fedc01607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:47.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.846+0000 7fedc5800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fedc006c350 con 0x7fedc0160350 2024-04-17T09:58:47.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.846+0000 7fedc5800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fedc00d0130 con 0x7fedc00cf370 2024-04-17T09:58:47.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.846+0000 7fedc5800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fedc015d090 con 0x7fedc006ba30 2024-04-17T09:58:47.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.846+0000 7fedbf400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fedc0160350 0x7fedc01607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:47.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.846+0000 7fedbf400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fedc0160350 0x7fedc01607a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38892/0 (socket says 172.21.15.32:38892) 2024-04-17T09:58:47.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.846+0000 7fedbf400640 1 -- 172.21.15.32:0/124851531 learned_addr learned my addr 172.21.15.32:0/124851531 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:47.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.850+0000 7fedbea00640 1 --2- 172.21.15.32:0/124851531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedc006ba30 0x7fedc006be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:47.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.850+0000 7fedb7e00640 1 --2- 172.21.15.32:0/124851531 >> v2:172.21.15.144:3300/0 conn(0x7fedc00cf370 0x7fedc00cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:47.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.850+0000 7fedbf400640 1 -- 172.21.15.32:0/124851531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedc006ba30 msgr2=0x7fedc006be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:47.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.850+0000 7fedbf400640 1 --2- 172.21.15.32:0/124851531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedc006ba30 0x7fedc006be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.850+0000 7fedbf400640 1 -- 172.21.15.32:0/124851531 >> v2:172.21.15.144:3300/0 conn(0x7fedc00cf370 msgr2=0x7fedc00cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:47.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.850+0000 7fedbf400640 1 --2- 172.21.15.32:0/124851531 >> v2:172.21.15.144:3300/0 conn(0x7fedc00cf370 0x7fedc00cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.850+0000 7fedbf400640 1 -- 172.21.15.32:0/124851531 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc015d370 con 0x7fedc0160350 2024-04-17T09:58:47.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.850+0000 7fedbea00640 1 --2- 172.21.15.32:0/124851531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedc006ba30 0x7fedc006be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:58:47.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.850+0000 7fedb7e00640 1 --2- 172.21.15.32:0/124851531 >> v2:172.21.15.144:3300/0 conn(0x7fedc00cf370 0x7fedc00cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:58:47.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.850+0000 7fedbf400640 1 --2- 172.21.15.32:0/124851531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedc0160350 0x7fedc01607a0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fedc00b57b0 tx=0x7fedb801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1c596015f05310 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:47.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.850+0000 7fedbe000640 1 -- 172.21.15.32:0/124851531 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb80a5800 con 0x7fedc0160350 2024-04-17T09:58:47.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.850+0000 7fedbe000640 1 -- 172.21.15.32:0/124851531 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedb80a59a0 con 0x7fedc0160350 2024-04-17T09:58:47.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.850+0000 7fedbe000640 1 -- 172.21.15.32:0/124851531 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb80a5d70 con 0x7fedc0160350 2024-04-17T09:58:47.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.850+0000 7fedc5800640 1 -- 172.21.15.32:0/124851531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedc0160350 msgr2=0x7fedc01607a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:47.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.850+0000 7fedc5800640 1 --2- 172.21.15.32:0/124851531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedc0160350 0x7fedc01607a0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fedc00b57b0 tx=0x7fedb801c9c0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.850+0000 7fedc5800640 1 -- 172.21.15.32:0/124851531 shutdown_connections 2024-04-17T09:58:47.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.850+0000 7fedc5800640 1 --2- 172.21.15.32:0/124851531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedc0160350 0x7fedc01607a0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.850+0000 7fedc5800640 1 --2- 172.21.15.32:0/124851531 >> v2:172.21.15.144:3300/0 conn(0x7fedc00cf370 0x7fedc00cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.850+0000 7fedc5800640 1 --2- 172.21.15.32:0/124851531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedc006ba30 0x7fedc006be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedc5800640 1 -- 172.21.15.32:0/124851531 >> 172.21.15.32:0/124851531 conn(0x7fedc00b2120 msgr2=0x7fedc00b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:47.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedc5800640 1 -- 172.21.15.32:0/124851531 shutdown_connections 2024-04-17T09:58:47.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedc5800640 1 -- 172.21.15.32:0/124851531 wait complete. 2024-04-17T09:58:47.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedc5800640 1 Processor -- start 2024-04-17T09:58:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedc5800640 1 -- start start 2024-04-17T09:58:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedc5800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedc006ba30 0x7fedc015ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedc5800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedc00cf370 0x7fedc0166930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedc5800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedc0160350 0x7fedc0166e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedc5800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fedc00cfea0 con 0x7fedc006ba30 2024-04-17T09:58:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedc5800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fedc00cfd20 con 0x7fedc00cf370 2024-04-17T09:58:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedc5800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fedc015db70 con 0x7fedc0160350 2024-04-17T09:58:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedbea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedc006ba30 0x7fedc015ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedbea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedc006ba30 0x7fedc015ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38908/0 (socket says 172.21.15.32:38908) 2024-04-17T09:58:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedbea00640 1 -- 172.21.15.32:0/1137675785 learned_addr learned my addr 172.21.15.32:0/1137675785 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedb7e00640 1 --2- 172.21.15.32:0/1137675785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedc00cf370 0x7fedc0166930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedbf400640 1 --2- 172.21.15.32:0/1137675785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedc0160350 0x7fedc0166e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedbea00640 1 -- 172.21.15.32:0/1137675785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedc0160350 msgr2=0x7fedc0166e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedbea00640 1 --2- 172.21.15.32:0/1137675785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedc0160350 0x7fedc0166e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedbea00640 1 -- 172.21.15.32:0/1137675785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedc00cf370 msgr2=0x7fedc0166930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedbea00640 1 --2- 172.21.15.32:0/1137675785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedc00cf370 0x7fedc0166930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedbea00640 1 -- 172.21.15.32:0/1137675785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc015f290 con 0x7fedc006ba30 2024-04-17T09:58:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedb7e00640 1 --2- 172.21.15.32:0/1137675785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedc00cf370 0x7fedc0166930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:58:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedbea00640 1 --2- 172.21.15.32:0/1137675785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedc006ba30 0x7fedc015ea40 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fedb000e810 tx=0x7fedb000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedb6a00640 1 -- 172.21.15.32:0/1137675785 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb000f040 con 0x7fedc006ba30 2024-04-17T09:58:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedb6a00640 1 -- 172.21.15.32:0/1137675785 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedb0004030 con 0x7fedc006ba30 2024-04-17T09:58:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedb6a00640 1 -- 172.21.15.32:0/1137675785 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb0004400 con 0x7fedc006ba30 2024-04-17T09:58:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedc5800640 1 -- 172.21.15.32:0/1137675785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc015f570 con 0x7fedc006ba30 2024-04-17T09:58:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.854+0000 7fedc5800640 1 -- 172.21.15.32:0/1137675785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc015fb30 con 0x7fedc006ba30 2024-04-17T09:58:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.858+0000 7fedc5800640 1 -- 172.21.15.32:0/1137675785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc0068a90 con 0x7fedc006ba30 2024-04-17T09:58:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.858+0000 7fedb6a00640 1 -- 172.21.15.32:0/1137675785 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fedb000c6d0 con 0x7fedc006ba30 2024-04-17T09:58:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.858+0000 7fedb6a00640 1 --2- 172.21.15.32:0/1137675785 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fed9c068380 0x7fed9c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.858+0000 7fedb6a00640 1 -- 172.21.15.32:0/1137675785 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fedb0086780 con 0x7fedc006ba30 2024-04-17T09:58:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.858+0000 7fedb7e00640 1 --2- 172.21.15.32:0/1137675785 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fed9c068380 0x7fed9c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.858+0000 7fedb7e00640 1 --2- 172.21.15.32:0/1137675785 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fed9c068380 0x7fed9c06a840 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fedc00a9f40 tx=0x7feda8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:47.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:47.862+0000 7fedb6a00640 1 -- 172.21.15.32:0/1137675785 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fedb0017020 con 0x7fedc006ba30 2024-04-17T09:58:48.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.030+0000 7fedc5800640 1 -- 172.21.15.32:0/1137675785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fedc006c850 con 0x7fedc006ba30 2024-04-17T09:58:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.030+0000 7fedb6a00640 1 -- 172.21.15.32:0/1137675785 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7fedb0056c10 con 0x7fedc006ba30 2024-04-17T09:58:48.027 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:58:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.030+0000 7fedc5800640 1 -- 172.21.15.32:0/1137675785 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fed9c068380 msgr2=0x7fed9c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:48.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.030+0000 7fedc5800640 1 --2- 172.21.15.32:0/1137675785 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fed9c068380 0x7fed9c06a840 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fedc00a9f40 tx=0x7feda8008040 comp rx=0 tx=0).stop 2024-04-17T09:58:48.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.030+0000 7fedc5800640 1 -- 172.21.15.32:0/1137675785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedc006ba30 msgr2=0x7fedc015ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:48.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.030+0000 7fedc5800640 1 --2- 172.21.15.32:0/1137675785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedc006ba30 0x7fedc015ea40 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fedb000e810 tx=0x7fedb000ecd0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.030+0000 7fedc5800640 1 -- 172.21.15.32:0/1137675785 shutdown_connections 2024-04-17T09:58:48.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.030+0000 7fedc5800640 1 --2- 172.21.15.32:0/1137675785 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fed9c068380 0x7fed9c06a840 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.030+0000 7fedc5800640 1 --2- 172.21.15.32:0/1137675785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedc0160350 0x7fedc0166e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.030+0000 7fedc5800640 1 --2- 172.21.15.32:0/1137675785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedc00cf370 0x7fedc0166930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.030+0000 7fedc5800640 1 --2- 172.21.15.32:0/1137675785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedc006ba30 0x7fedc015ea40 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.030+0000 7fedc5800640 1 -- 172.21.15.32:0/1137675785 >> 172.21.15.32:0/1137675785 conn(0x7fedc00b2120 msgr2=0x7fedc006f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:48.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.030+0000 7fedc5800640 1 -- 172.21.15.32:0/1137675785 shutdown_connections 2024-04-17T09:58:48.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.030+0000 7fedc5800640 1 -- 172.21.15.32:0/1137675785 wait complete. 2024-04-17T09:58:48.048 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:58:48.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.194+0000 7f5adc600640 1 Processor -- start 2024-04-17T09:58:48.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.194+0000 7f5adc600640 1 -- start start 2024-04-17T09:58:48.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.198+0000 7f5adc600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ad40cf370 0x7f5ad40cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:48.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.198+0000 7f5adc600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5ad4071e60 0x7f5ad4072270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:48.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.198+0000 7f5adc600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ad40d0130 con 0x7f5ad406ba30 2024-04-17T09:58:48.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.198+0000 7f5adc600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5ad415d090 con 0x7f5ad40cf370 2024-04-17T09:58:48.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.198+0000 7f5adc600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f5ad415d370 con 0x7f5ad4071e60 2024-04-17T09:58:48.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.198+0000 7f5ad9c00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5ad406ba30 0x7f5ad406be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38998/0 (socket says 172.21.15.32:38998) 2024-04-17T09:58:48.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.198+0000 7f5ad9c00640 1 -- 172.21.15.32:0/350725157 learned_addr learned my addr 172.21.15.32:0/350725157 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:58:48.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.198+0000 7f5ad9200640 1 --2- 172.21.15.32:0/350725157 >> v2:172.21.15.32:3300/0 conn(0x7f5ad40cf370 0x7f5ad40cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:48.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.198+0000 7f5ada600640 1 --2- 172.21.15.32:0/350725157 >> v2:172.21.15.144:3300/0 conn(0x7f5ad4071e60 0x7f5ad4072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:48.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.198+0000 7f5ad3e00640 1 -- 172.21.15.32:0/350725157 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3743440706 0 0) 0x7f5ad40d0130 con 0x7f5ad406ba30 2024-04-17T09:58:48.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.198+0000 7f5ad3e00640 1 -- 172.21.15.32:0/350725157 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5abc003530 con 0x7f5ad406ba30 2024-04-17T09:58:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.198+0000 7f5ad9200640 1 -- 172.21.15.32:0/350725157 >> v2:172.21.15.144:3300/0 conn(0x7f5ad4071e60 msgr2=0x7f5ad4072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.198+0000 7f5ad9200640 1 --2- 172.21.15.32:0/350725157 >> v2:172.21.15.144:3300/0 conn(0x7f5ad4071e60 0x7f5ad4072270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.198+0000 7f5ad9200640 1 -- 172.21.15.32:0/350725157 >> v1:172.21.15.32:6789/0 conn(0x7f5ad406ba30 legacy=0x7f5ad406be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.198+0000 7f5ad9200640 1 -- 172.21.15.32:0/350725157 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad415d650 con 0x7f5ad40cf370 2024-04-17T09:58:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.202+0000 7f5ad9200640 1 --2- 172.21.15.32:0/350725157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ad40cf370 0x7f5ad40cf7e0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f5ad415a950 tx=0x7f5ac001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa3728248da5084 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.202+0000 7f5ad3e00640 1 -- 172.21.15.32:0/350725157 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac00aa600 con 0x7f5ad40cf370 2024-04-17T09:58:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.202+0000 7f5ad3e00640 1 -- 172.21.15.32:0/350725157 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac00af070 con 0x7f5ad40cf370 2024-04-17T09:58:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.202+0000 7f5ad3e00640 1 -- 172.21.15.32:0/350725157 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac00aa9c0 con 0x7f5ad40cf370 2024-04-17T09:58:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.202+0000 7f5adc600640 1 -- 172.21.15.32:0/350725157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ad40cf370 msgr2=0x7f5ad40cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.202+0000 7f5adc600640 1 --2- 172.21.15.32:0/350725157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ad40cf370 0x7f5ad40cf7e0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f5ad415a950 tx=0x7f5ac001c480 comp rx=0 tx=0).stop 2024-04-17T09:58:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.202+0000 7f5adc600640 1 -- 172.21.15.32:0/350725157 shutdown_connections 2024-04-17T09:58:48.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.202+0000 7f5adc600640 1 --2- 172.21.15.32:0/350725157 >> v2:172.21.15.144:3300/0 conn(0x7f5ad4071e60 0x7f5ad4072270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.202+0000 7f5adc600640 1 --2- 172.21.15.32:0/350725157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ad40cf370 0x7f5ad40cf7e0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.202+0000 7f5adc600640 1 -- 172.21.15.32:0/350725157 >> 172.21.15.32:0/350725157 conn(0x7f5ad40b2120 msgr2=0x7f5ad40b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:48.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.202+0000 7f5adc600640 1 -- 172.21.15.32:0/350725157 shutdown_connections 2024-04-17T09:58:48.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.202+0000 7f5adc600640 1 -- 172.21.15.32:0/350725157 wait complete. 2024-04-17T09:58:48.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.202+0000 7f5adc600640 1 Processor -- start 2024-04-17T09:58:48.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.202+0000 7f5adc600640 1 -- start start 2024-04-17T09:58:48.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.202+0000 7f5adc600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ad406ba30 0x7f5ad415be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.202+0000 7f5adc600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ad4071e60 0x7f5ad413bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.202+0000 7f5adc600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad40cf370 0x7f5ad40b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.202+0000 7f5adc600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad4146230 con 0x7f5ad4071e60 2024-04-17T09:58:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.202+0000 7f5adc600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad415dd80 con 0x7f5ad40cf370 2024-04-17T09:58:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.202+0000 7f5adc600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad41463b0 con 0x7f5ad406ba30 2024-04-17T09:58:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.206+0000 7f5ad9200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ad4071e60 0x7f5ad413bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.206+0000 7f5ad9200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ad4071e60 0x7f5ad413bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38924/0 (socket says 172.21.15.32:38924) 2024-04-17T09:58:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.206+0000 7f5ad9200640 1 -- 172.21.15.32:0/3079533270 learned_addr learned my addr 172.21.15.32:0/3079533270 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.206+0000 7f5ada600640 1 --2- 172.21.15.32:0/3079533270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad40cf370 0x7f5ad40b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.206+0000 7f5ad9c00640 1 --2- 172.21.15.32:0/3079533270 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ad406ba30 0x7f5ad415be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.206+0000 7f5ad9200640 1 -- 172.21.15.32:0/3079533270 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ad406ba30 msgr2=0x7f5ad415be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.206+0000 7f5ad9200640 1 --2- 172.21.15.32:0/3079533270 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ad406ba30 0x7f5ad415be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.206+0000 7f5ad9200640 1 -- 172.21.15.32:0/3079533270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad40cf370 msgr2=0x7f5ad40b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.206+0000 7f5ad9200640 1 --2- 172.21.15.32:0/3079533270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad40cf370 0x7f5ad40b4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.206+0000 7f5ad9200640 1 -- 172.21.15.32:0/3079533270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad40b3dd0 con 0x7f5ad4071e60 2024-04-17T09:58:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.206+0000 7f5ad9200640 1 --2- 172.21.15.32:0/3079533270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ad4071e60 0x7f5ad413bf00 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f5ac001a0b0 tx=0x7f5ac001acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.206+0000 7f5ad1600640 1 -- 172.21.15.32:0/3079533270 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac00be610 con 0x7f5ad4071e60 2024-04-17T09:58:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.206+0000 7f5ad1600640 1 -- 172.21.15.32:0/3079533270 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac00aae50 con 0x7f5ad4071e60 2024-04-17T09:58:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.206+0000 7f5ad1600640 1 -- 172.21.15.32:0/3079533270 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac00bea10 con 0x7f5ad4071e60 2024-04-17T09:58:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.206+0000 7f5adc600640 1 -- 172.21.15.32:0/3079533270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad40b40b0 con 0x7f5ad4071e60 2024-04-17T09:58:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.206+0000 7f5adc600640 1 -- 172.21.15.32:0/3079533270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad40b4670 con 0x7f5ad4071e60 2024-04-17T09:58:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.206+0000 7f5adc600640 1 -- 172.21.15.32:0/3079533270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ad4068a90 con 0x7f5ad4071e60 2024-04-17T09:58:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.206+0000 7f5ad1600640 1 -- 172.21.15.32:0/3079533270 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5ac00bc020 con 0x7f5ad4071e60 2024-04-17T09:58:48.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.210+0000 7f5ad1600640 1 --2- 172.21.15.32:0/3079533270 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ab4068380 0x7f5ab406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:48.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.210+0000 7f5ad1600640 1 -- 172.21.15.32:0/3079533270 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f5ac0120970 con 0x7f5ad4071e60 2024-04-17T09:58:48.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.210+0000 7f5ad9c00640 1 --2- 172.21.15.32:0/3079533270 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ab4068380 0x7f5ab406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:48.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.210+0000 7f5ad9c00640 1 --2- 172.21.15.32:0/3079533270 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ab4068380 0x7f5ab406a840 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8007fd0 tx=0x7f5ac8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:48.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.214+0000 7f5ad1600640 1 -- 172.21.15.32:0/3079533270 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5ac00c5070 con 0x7f5ad4071e60 2024-04-17T09:58:48.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.378+0000 7f5adc600640 1 -- 172.21.15.32:0/3079533270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5ad406c850 con 0x7f5ad4071e60 2024-04-17T09:58:48.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.378+0000 7f5ad1600640 1 -- 172.21.15.32:0/3079533270 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) ==== 104+0+11 (secure 0 0 0) 0x7f5ac0016070 con 0x7f5ad4071e60 2024-04-17T09:58:48.375 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:58:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.378+0000 7f5adc600640 1 -- 172.21.15.32:0/3079533270 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ab4068380 msgr2=0x7f5ab406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.378+0000 7f5adc600640 1 --2- 172.21.15.32:0/3079533270 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ab4068380 0x7f5ab406a840 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8007fd0 tx=0x7f5ac8009040 comp rx=0 tx=0).stop 2024-04-17T09:58:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.378+0000 7f5adc600640 1 -- 172.21.15.32:0/3079533270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ad4071e60 msgr2=0x7f5ad413bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.378+0000 7f5adc600640 1 --2- 172.21.15.32:0/3079533270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ad4071e60 0x7f5ad413bf00 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f5ac001a0b0 tx=0x7f5ac001acc0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.378+0000 7f5adc600640 1 -- 172.21.15.32:0/3079533270 shutdown_connections 2024-04-17T09:58:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.378+0000 7f5adc600640 1 --2- 172.21.15.32:0/3079533270 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ab4068380 0x7f5ab406a840 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.378+0000 7f5adc600640 1 --2- 172.21.15.32:0/3079533270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad40cf370 0x7f5ad40b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.378+0000 7f5adc600640 1 --2- 172.21.15.32:0/3079533270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ad4071e60 0x7f5ad413bf00 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.378+0000 7f5adc600640 1 --2- 172.21.15.32:0/3079533270 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ad406ba30 0x7f5ad415be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.378+0000 7f5adc600640 1 -- 172.21.15.32:0/3079533270 >> 172.21.15.32:0/3079533270 conn(0x7f5ad40b2120 msgr2=0x7f5ad4142600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:48.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.378+0000 7f5adc600640 1 -- 172.21.15.32:0/3079533270 shutdown_connections 2024-04-17T09:58:48.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.378+0000 7f5adc600640 1 -- 172.21.15.32:0/3079533270 wait complete. 2024-04-17T09:58:48.397 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:58:48.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.542+0000 7fbb1ba00640 1 Processor -- start 2024-04-17T09:58:48.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.542+0000 7fbb1ba00640 1 -- start start 2024-04-17T09:58:48.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.542+0000 7fbb1ba00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb1415b7d0 0x7fbb1406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:48.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.542+0000 7fbb1ba00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbb1406f7c0 0x7fbb1406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:48.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.542+0000 7fbb1ba00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbb1406f270 0x7fbb1406bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:48.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.542+0000 7fbb1ba00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fbb140cc940 con 0x7fbb1406f7c0 2024-04-17T09:58:48.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.542+0000 7fbb1ba00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fbb140ccc20 con 0x7fbb1406f270 2024-04-17T09:58:48.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.542+0000 7fbb1ba00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbb1406e0b0 con 0x7fbb1415b7d0 2024-04-17T09:58:48.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.542+0000 7fbb0be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbb1406f7c0 0x7fbb1406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:48.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.542+0000 7fbb0be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbb1406f7c0 0x7fbb1406ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38934/0 (socket says 172.21.15.32:38934) 2024-04-17T09:58:48.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.542+0000 7fbb0be00640 1 -- 172.21.15.32:0/1501687850 learned_addr learned my addr 172.21.15.32:0/1501687850 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:48.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.542+0000 7fbb19000640 1 --2- 172.21.15.32:0/1501687850 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb1415b7d0 0x7fbb1406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:48.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.542+0000 7fbb19a00640 1 --2- 172.21.15.32:0/1501687850 >> v2:172.21.15.144:3300/0 conn(0x7fbb1406f270 0x7fbb1406bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:48.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.542+0000 7fbb0be00640 1 -- 172.21.15.32:0/1501687850 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb1415b7d0 msgr2=0x7fbb1406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:48.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.542+0000 7fbb0be00640 1 --2- 172.21.15.32:0/1501687850 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb1415b7d0 0x7fbb1406e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.542+0000 7fbb0be00640 1 -- 172.21.15.32:0/1501687850 >> v2:172.21.15.144:3300/0 conn(0x7fbb1406f270 msgr2=0x7fbb1406bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:48.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.542+0000 7fbb0be00640 1 --2- 172.21.15.32:0/1501687850 >> v2:172.21.15.144:3300/0 conn(0x7fbb1406f270 0x7fbb1406bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.542+0000 7fbb0be00640 1 -- 172.21.15.32:0/1501687850 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1406e390 con 0x7fbb1406f7c0 2024-04-17T09:58:48.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.546+0000 7fbb0be00640 1 --2- 172.21.15.32:0/1501687850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb1406f7c0 0x7fbb1406ba30 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fbb140b44e0 tx=0x7fbafc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f798923ea4f45352 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:48.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.546+0000 7fbb13e00640 1 -- 172.21.15.32:0/1501687850 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc0a7070 con 0x7fbb1406f7c0 2024-04-17T09:58:48.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.546+0000 7fbb13e00640 1 -- 172.21.15.32:0/1501687850 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbafc0b0070 con 0x7fbb1406f7c0 2024-04-17T09:58:48.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.546+0000 7fbb13e00640 1 -- 172.21.15.32:0/1501687850 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc0ab890 con 0x7fbb1406f7c0 2024-04-17T09:58:48.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.546+0000 7fbb1ba00640 1 -- 172.21.15.32:0/1501687850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb1406f7c0 msgr2=0x7fbb1406ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:48.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.546+0000 7fbb1ba00640 1 --2- 172.21.15.32:0/1501687850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb1406f7c0 0x7fbb1406ba30 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fbb140b44e0 tx=0x7fbafc01c490 comp rx=0 tx=0).stop 2024-04-17T09:58:48.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.546+0000 7fbb1ba00640 1 -- 172.21.15.32:0/1501687850 shutdown_connections 2024-04-17T09:58:48.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.546+0000 7fbb1ba00640 1 --2- 172.21.15.32:0/1501687850 >> v2:172.21.15.144:3300/0 conn(0x7fbb1406f270 0x7fbb1406bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.546+0000 7fbb1ba00640 1 --2- 172.21.15.32:0/1501687850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb1406f7c0 0x7fbb1406ba30 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.546+0000 7fbb1ba00640 1 --2- 172.21.15.32:0/1501687850 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb1415b7d0 0x7fbb1406e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.546+0000 7fbb1ba00640 1 -- 172.21.15.32:0/1501687850 >> 172.21.15.32:0/1501687850 conn(0x7fbb140b2120 msgr2=0x7fbb140b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:48.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.546+0000 7fbb1ba00640 1 -- 172.21.15.32:0/1501687850 shutdown_connections 2024-04-17T09:58:48.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.546+0000 7fbb1ba00640 1 -- 172.21.15.32:0/1501687850 wait complete. 2024-04-17T09:58:48.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb1ba00640 1 Processor -- start 2024-04-17T09:58:48.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb1ba00640 1 -- start start 2024-04-17T09:58:48.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb1ba00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb1406f270 0x7fbb1414e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb1ba00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb1406f7c0 0x7fbb1414fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb1ba00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb1415b7d0 0x7fbb1415a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb1ba00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbb140cee70 con 0x7fbb1406f270 2024-04-17T09:58:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb1ba00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbb140cecf0 con 0x7fbb1406f7c0 2024-04-17T09:58:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb1ba00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbb140b3420 con 0x7fbb1415b7d0 2024-04-17T09:58:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb19000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb1406f270 0x7fbb1414e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb19000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb1406f270 0x7fbb1414e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38936/0 (socket says 172.21.15.32:38936) 2024-04-17T09:58:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb19000640 1 -- 172.21.15.32:0/503339011 learned_addr learned my addr 172.21.15.32:0/503339011 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb19a00640 1 --2- 172.21.15.32:0/503339011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb1415b7d0 0x7fbb1415a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb0be00640 1 --2- 172.21.15.32:0/503339011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb1406f7c0 0x7fbb1414fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb19000640 1 -- 172.21.15.32:0/503339011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb1415b7d0 msgr2=0x7fbb1415a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:48.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb19000640 1 --2- 172.21.15.32:0/503339011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb1415b7d0 0x7fbb1415a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb19000640 1 -- 172.21.15.32:0/503339011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb1406f7c0 msgr2=0x7fbb1414fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:48.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb19000640 1 --2- 172.21.15.32:0/503339011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb1406f7c0 0x7fbb1414fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb19000640 1 -- 172.21.15.32:0/503339011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1414eb50 con 0x7fbb1406f270 2024-04-17T09:58:48.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb19a00640 1 --2- 172.21.15.32:0/503339011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb1415b7d0 0x7fbb1415a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:58:48.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb19000640 1 --2- 172.21.15.32:0/503339011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb1406f270 0x7fbb1414e4e0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400e8a0 tx=0x7fbb0400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:48.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb11600640 1 -- 172.21.15.32:0/503339011 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04017070 con 0x7fbb1406f270 2024-04-17T09:58:48.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb11600640 1 -- 172.21.15.32:0/503339011 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb0400c940 con 0x7fbb1406f270 2024-04-17T09:58:48.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb11600640 1 -- 172.21.15.32:0/503339011 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0400ccf0 con 0x7fbb1406f270 2024-04-17T09:58:48.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb1ba00640 1 -- 172.21.15.32:0/503339011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb1414ee30 con 0x7fbb1406f270 2024-04-17T09:58:48.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.550+0000 7fbb1ba00640 1 -- 172.21.15.32:0/503339011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb1414f6d0 con 0x7fbb1406f270 2024-04-17T09:58:48.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.554+0000 7fbb1ba00640 1 -- 172.21.15.32:0/503339011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb1414f000 con 0x7fbb1406f270 2024-04-17T09:58:48.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.554+0000 7fbb11600640 1 -- 172.21.15.32:0/503339011 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbb0401a070 con 0x7fbb1406f270 2024-04-17T09:58:48.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.554+0000 7fbb11600640 1 --2- 172.21.15.32:0/503339011 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbaf0068380 0x7fbaf006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:48.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.554+0000 7fbb11600640 1 -- 172.21.15.32:0/503339011 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fbb04086c20 con 0x7fbb1406f270 2024-04-17T09:58:48.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.554+0000 7fbb0be00640 1 --2- 172.21.15.32:0/503339011 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbaf0068380 0x7fbaf006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:48.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.554+0000 7fbb0be00640 1 --2- 172.21.15.32:0/503339011 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbaf0068380 0x7fbaf006a840 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0af000 tx=0x7fbafc0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:48.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.558+0000 7fbb11600640 1 -- 172.21.15.32:0/503339011 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fbb04056af0 con 0x7fbb1406f270 2024-04-17T09:58:48.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.722+0000 7fbb1ba00640 1 -- 172.21.15.32:0/503339011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbb14068a90 con 0x7fbb1406f270 2024-04-17T09:58:48.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.722+0000 7fbb11600640 1 -- 172.21.15.32:0/503339011 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) ==== 100+0+11 (secure 0 0 0) 0x7fbb0405adf0 con 0x7fbb1406f270 2024-04-17T09:58:48.721 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:58:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.726+0000 7fbb1ba00640 1 -- 172.21.15.32:0/503339011 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbaf0068380 msgr2=0x7fbaf006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.726+0000 7fbb1ba00640 1 --2- 172.21.15.32:0/503339011 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbaf0068380 0x7fbaf006a840 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0af000 tx=0x7fbafc0a7000 comp rx=0 tx=0).stop 2024-04-17T09:58:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.726+0000 7fbb1ba00640 1 -- 172.21.15.32:0/503339011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb1406f270 msgr2=0x7fbb1414e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.726+0000 7fbb1ba00640 1 --2- 172.21.15.32:0/503339011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb1406f270 0x7fbb1414e4e0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400e8a0 tx=0x7fbb0400eca0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.726+0000 7fbb1ba00640 1 -- 172.21.15.32:0/503339011 shutdown_connections 2024-04-17T09:58:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.726+0000 7fbb1ba00640 1 --2- 172.21.15.32:0/503339011 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbaf0068380 0x7fbaf006a840 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.726+0000 7fbb1ba00640 1 --2- 172.21.15.32:0/503339011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb1415b7d0 0x7fbb1415a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.726+0000 7fbb1ba00640 1 --2- 172.21.15.32:0/503339011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb1406f7c0 0x7fbb1414fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.726+0000 7fbb1ba00640 1 --2- 172.21.15.32:0/503339011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb1406f270 0x7fbb1414e4e0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.726+0000 7fbb1ba00640 1 -- 172.21.15.32:0/503339011 >> 172.21.15.32:0/503339011 conn(0x7fbb140b2120 msgr2=0x7fbb140b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:48.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.726+0000 7fbb1ba00640 1 -- 172.21.15.32:0/503339011 shutdown_connections 2024-04-17T09:58:48.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.726+0000 7fbb1ba00640 1 -- 172.21.15.32:0/503339011 wait complete. 2024-04-17T09:58:48.743 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T09:58:48.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.886+0000 7fdfd2800640 1 Processor -- start 2024-04-17T09:58:48.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.886+0000 7fdfd2800640 1 -- start start 2024-04-17T09:58:48.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.886+0000 7fdfd2800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdfcc0b3340 0x7fdfcc15f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:48.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.886+0000 7fdfd2800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfcc15f870 con 0x7fdfcc155e50 2024-04-17T09:58:48.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.886+0000 7fdfd2800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfcc15fb50 con 0x7fdfcc155a70 2024-04-17T09:58:48.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.886+0000 7fdfd2800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fdfcc0b42a0 con 0x7fdfcc0b3340 2024-04-17T09:58:48.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.886+0000 7fdfc2a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdfcc155e50 0x7fdfcc15c260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39014/0 (socket says 172.21.15.32:39014) 2024-04-17T09:58:48.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.886+0000 7fdfc2a00640 1 -- 172.21.15.32:0/2977902209 learned_addr learned my addr 172.21.15.32:0/2977902209 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:58:48.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.886+0000 7fdfcbe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdfcc0b3340 0x7fdfcc15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:48.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.890+0000 7fdfcbe00640 1 -- 172.21.15.32:0/2977902209 >> v1:172.21.15.144:6789/0 conn(0x7fdfcc155a70 legacy=0x7fdfcc15d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:48.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.890+0000 7fdfcbe00640 1 -- 172.21.15.32:0/2977902209 >> v1:172.21.15.32:6789/0 conn(0x7fdfcc155e50 legacy=0x7fdfcc15c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:48.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.890+0000 7fdfcbe00640 1 -- 172.21.15.32:0/2977902209 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc15fe30 con 0x7fdfcc0b3340 2024-04-17T09:58:48.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.890+0000 7fdfcaa00640 1 -- 172.21.15.32:0/2977902209 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3251776295 0 0) 0x7fdfcc15f870 con 0x7fdfcc155e50 2024-04-17T09:58:48.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.890+0000 7fdfcbe00640 1 --2- 172.21.15.32:0/2977902209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfcc0b3340 0x7fdfcc15f190 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0b4420 tx=0x7fdfc401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5ab4391a71f759 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:48.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.890+0000 7fdfcaa00640 1 -- 172.21.15.32:0/2977902209 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40b2070 con 0x7fdfcc0b3340 2024-04-17T09:58:48.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.890+0000 7fdfcaa00640 1 -- 172.21.15.32:0/2977902209 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc40a57d0 con 0x7fdfcc0b3340 2024-04-17T09:58:48.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.890+0000 7fdfcaa00640 1 -- 172.21.15.32:0/2977902209 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40a5ba0 con 0x7fdfcc0b3340 2024-04-17T09:58:48.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.890+0000 7fdfd2800640 1 -- 172.21.15.32:0/2977902209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfcc0b3340 msgr2=0x7fdfcc15f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:48.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.890+0000 7fdfd2800640 1 --2- 172.21.15.32:0/2977902209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfcc0b3340 0x7fdfcc15f190 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0b4420 tx=0x7fdfc401c910 comp rx=0 tx=0).stop 2024-04-17T09:58:48.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfd2800640 1 -- 172.21.15.32:0/2977902209 shutdown_connections 2024-04-17T09:58:48.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfd2800640 1 --2- 172.21.15.32:0/2977902209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfcc0b3340 0x7fdfcc15f190 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfd2800640 1 -- 172.21.15.32:0/2977902209 >> 172.21.15.32:0/2977902209 conn(0x7fdfcc06ba30 msgr2=0x7fdfcc06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:48.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfd2800640 1 -- 172.21.15.32:0/2977902209 shutdown_connections 2024-04-17T09:58:48.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfd2800640 1 -- 172.21.15.32:0/2977902209 wait complete. 2024-04-17T09:58:48.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfd2800640 1 Processor -- start 2024-04-17T09:58:48.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfd2800640 1 -- start start 2024-04-17T09:58:48.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfd2800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfcc0b3340 0x7fdfcc0add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:48.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfd2800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfcc155a70 0x7fdfcc144190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:48.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfd2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfcc155e50 0x7fdfcc157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:48.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfd2800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc0d0170 con 0x7fdfcc155e50 2024-04-17T09:58:48.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfd2800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc0d02f0 con 0x7fdfcc155a70 2024-04-17T09:58:48.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfd2800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdfcc0d0470 con 0x7fdfcc0b3340 2024-04-17T09:58:48.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfcbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfcc155e50 0x7fdfcc157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:48.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfcbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfcc155e50 0x7fdfcc157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38952/0 (socket says 172.21.15.32:38952) 2024-04-17T09:58:48.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfcbe00640 1 -- 172.21.15.32:0/2236512347 learned_addr learned my addr 172.21.15.32:0/2236512347 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:48.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfc2a00640 1 --2- 172.21.15.32:0/2236512347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfcc155a70 0x7fdfcc144190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:48.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfcb400640 1 --2- 172.21.15.32:0/2236512347 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfcc0b3340 0x7fdfcc0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:48.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfcbe00640 1 -- 172.21.15.32:0/2236512347 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfcc0b3340 msgr2=0x7fdfcc0add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:48.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfcbe00640 1 --2- 172.21.15.32:0/2236512347 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfcc0b3340 0x7fdfcc0add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfcbe00640 1 -- 172.21.15.32:0/2236512347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfcc155a70 msgr2=0x7fdfcc144190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:48.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfcbe00640 1 --2- 172.21.15.32:0/2236512347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfcc155a70 0x7fdfcc144190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:48.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfcbe00640 1 -- 172.21.15.32:0/2236512347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc157aa0 con 0x7fdfcc155e50 2024-04-17T09:58:48.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfc2a00640 1 --2- 172.21.15.32:0/2236512347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfcc155a70 0x7fdfcc144190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:58:48.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.894+0000 7fdfcbe00640 1 --2- 172.21.15.32:0/2236512347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfcc155e50 0x7fdfcc157190 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40b0040 tx=0x7fdfc40196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:48.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.898+0000 7fdfc3e00640 1 -- 172.21.15.32:0/2236512347 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40bd020 con 0x7fdfcc155e50 2024-04-17T09:58:48.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.898+0000 7fdfc3e00640 1 -- 172.21.15.32:0/2236512347 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc40b8670 con 0x7fdfcc155e50 2024-04-17T09:58:48.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.898+0000 7fdfc3e00640 1 -- 172.21.15.32:0/2236512347 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40b2040 con 0x7fdfcc155e50 2024-04-17T09:58:48.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.898+0000 7fdfd2800640 1 -- 172.21.15.32:0/2236512347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfcc157d80 con 0x7fdfcc155e50 2024-04-17T09:58:48.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.898+0000 7fdfd2800640 1 -- 172.21.15.32:0/2236512347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfcc154190 con 0x7fdfcc155e50 2024-04-17T09:58:48.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.898+0000 7fdfd2800640 1 -- 172.21.15.32:0/2236512347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfcc068a90 con 0x7fdfcc155e50 2024-04-17T09:58:48.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.898+0000 7fdfc3e00640 1 -- 172.21.15.32:0/2236512347 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdfc40c0020 con 0x7fdfcc155e50 2024-04-17T09:58:48.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.898+0000 7fdfc3e00640 1 --2- 172.21.15.32:0/2236512347 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdfa8068380 0x7fdfa806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:48.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.898+0000 7fdfc3e00640 1 -- 172.21.15.32:0/2236512347 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fdfc4121a00 con 0x7fdfcc155e50 2024-04-17T09:58:48.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.898+0000 7fdfcb400640 1 --2- 172.21.15.32:0/2236512347 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdfa8068380 0x7fdfa806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:48.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.902+0000 7fdfcb400640 1 --2- 172.21.15.32:0/2236512347 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdfa8068380 0x7fdfa806a840 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc1fac80 tx=0x7fdfbc007e40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:48.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:48.902+0000 7fdfc3e00640 1 -- 172.21.15.32:0/2236512347 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fdfc40c7030 con 0x7fdfcc155e50 2024-04-17T09:58:49.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.082+0000 7fdfd2800640 1 -- 172.21.15.32:0/2236512347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdfcc154360 con 0x7fdfcc155e50 2024-04-17T09:58:49.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.082+0000 7fdfc3e00640 1 -- 172.21.15.32:0/2236512347 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v71) ==== 80+0+120 (secure 0 0 0) 0x7fdfc4016500 con 0x7fdfcc155e50 2024-04-17T09:58:49.081 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6836/663537244 2024-04-18T09:57:30.746884+0000 2024-04-17T09:58:49.081 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6834/663537244 2024-04-18T09:57:30.746884+0000 2024-04-17T09:58:49.081 INFO:teuthology.orchestra.run.smithi032.stderr:listed 2 entries 2024-04-17T09:58:49.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.086+0000 7fdfd2800640 1 -- 172.21.15.32:0/2236512347 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdfa8068380 msgr2=0x7fdfa806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:49.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.086+0000 7fdfd2800640 1 --2- 172.21.15.32:0/2236512347 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdfa8068380 0x7fdfa806a840 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc1fac80 tx=0x7fdfbc007e40 comp rx=0 tx=0).stop 2024-04-17T09:58:49.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.086+0000 7fdfd2800640 1 -- 172.21.15.32:0/2236512347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfcc155e50 msgr2=0x7fdfcc157190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:49.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.086+0000 7fdfd2800640 1 --2- 172.21.15.32:0/2236512347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfcc155e50 0x7fdfcc157190 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40b0040 tx=0x7fdfc40196b0 comp rx=0 tx=0).stop 2024-04-17T09:58:49.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.086+0000 7fdfd2800640 1 -- 172.21.15.32:0/2236512347 shutdown_connections 2024-04-17T09:58:49.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.086+0000 7fdfd2800640 1 --2- 172.21.15.32:0/2236512347 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdfa8068380 0x7fdfa806a840 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:49.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.086+0000 7fdfd2800640 1 --2- 172.21.15.32:0/2236512347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfcc155e50 0x7fdfcc157190 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:49.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.086+0000 7fdfd2800640 1 --2- 172.21.15.32:0/2236512347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfcc155a70 0x7fdfcc144190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:49.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.086+0000 7fdfd2800640 1 --2- 172.21.15.32:0/2236512347 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfcc0b3340 0x7fdfcc0add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:49.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.086+0000 7fdfd2800640 1 -- 172.21.15.32:0/2236512347 >> 172.21.15.32:0/2236512347 conn(0x7fdfcc06ba30 msgr2=0x7fdfcc15e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:49.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.086+0000 7fdfd2800640 1 -- 172.21.15.32:0/2236512347 shutdown_connections 2024-04-17T09:58:49.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.086+0000 7fdfd2800640 1 -- 172.21.15.32:0/2236512347 wait complete. 2024-04-17T09:58:49.102 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2024-04-17T09:58:49.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:58:49.102 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T09:58:49.120 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[43451]: fuse finished with error 0 and tester_r 0 2024-04-17T09:58:49.123 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T09:58:49.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:58:49.123 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T09:58:49.201 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T09:58:49.202 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:58:49.202 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:58:49.212 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T09:58:49.212 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T09:58:49.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:58:49.212 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-17T09:58:49.261 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-17T09:58:49.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:58:49.263 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T09:58:49.263 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:58:49.263 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:58:49.272 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T09:58:49.272 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T09:58:49.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:58:49.272 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-17T09:58:49.322 INFO:teuthology.orchestra.run.smithi032.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-17T09:58:49.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:58:49.324 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T09:58:49.324 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:58:49.324 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:58:49.333 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T09:58:49.333 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T09:58:49.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:58:49.333 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-17T09:58:49.389 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-17T09:58:49.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:58:49.391 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:58:49.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.534+0000 7fee2e200640 1 Processor -- start 2024-04-17T09:58:49.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.534+0000 7fee2e200640 1 -- start start 2024-04-17T09:58:49.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.534+0000 7fee2e200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee28144e90 0x7fee280b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:49.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.534+0000 7fee2e200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fee280b4de0 0x7fee2815baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:49.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.534+0000 7fee2e200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee28072bb0 con 0x7fee2815bfe0 2024-04-17T09:58:49.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.534+0000 7fee2e200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fee2815c430 con 0x7fee280b4de0 2024-04-17T09:58:49.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.534+0000 7fee2e200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fee28145270 con 0x7fee28144e90 2024-04-17T09:58:49.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.534+0000 7fee27400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee28144e90 0x7fee280b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:49.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.534+0000 7fee27400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee28144e90 0x7fee280b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:51732/0 (socket says 172.21.15.32:51732) 2024-04-17T09:58:49.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.534+0000 7fee27400640 1 -- 172.21.15.32:0/802967943 learned_addr learned my addr 172.21.15.32:0/802967943 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:49.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.534+0000 7fee26a00640 1 --2- 172.21.15.32:0/802967943 >> v2:172.21.15.144:3300/0 conn(0x7fee280b4de0 0x7fee2815baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:49.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.534+0000 7fee26000640 1 -- 172.21.15.32:0/802967943 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 486534924 0 0) 0x7fee28072bb0 con 0x7fee2815bfe0 2024-04-17T09:58:49.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.534+0000 7fee26000640 1 -- 172.21.15.32:0/802967943 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee0c003530 con 0x7fee2815bfe0 2024-04-17T09:58:49.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.534+0000 7fee27400640 1 -- 172.21.15.32:0/802967943 >> v2:172.21.15.144:3300/0 conn(0x7fee280b4de0 msgr2=0x7fee2815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:49.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.534+0000 7fee27400640 1 --2- 172.21.15.32:0/802967943 >> v2:172.21.15.144:3300/0 conn(0x7fee280b4de0 0x7fee2815baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:49.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.534+0000 7fee27400640 1 -- 172.21.15.32:0/802967943 >> v1:172.21.15.144:6789/0 conn(0x7fee2815bfe0 legacy=0x7fee2806ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:49.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.534+0000 7fee27400640 1 -- 172.21.15.32:0/802967943 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee280b3dc0 con 0x7fee28144e90 2024-04-17T09:58:49.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.538+0000 7fee27400640 1 --2- 172.21.15.32:0/802967943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee28144e90 0x7fee280b48a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fee18009870 tx=0x7fee1801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b641a667c7241a34 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:49.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.538+0000 7fee26000640 1 -- 172.21.15.32:0/802967943 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee180a9610 con 0x7fee28144e90 2024-04-17T09:58:49.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.538+0000 7fee26000640 1 -- 172.21.15.32:0/802967943 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee180a97b0 con 0x7fee28144e90 2024-04-17T09:58:49.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.538+0000 7fee26000640 1 -- 172.21.15.32:0/802967943 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee180a9b80 con 0x7fee28144e90 2024-04-17T09:58:49.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.538+0000 7fee2e200640 1 -- 172.21.15.32:0/802967943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee28144e90 msgr2=0x7fee280b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:49.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.538+0000 7fee2e200640 1 --2- 172.21.15.32:0/802967943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee28144e90 0x7fee280b48a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fee18009870 tx=0x7fee1801c530 comp rx=0 tx=0).stop 2024-04-17T09:58:49.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.538+0000 7fee2e200640 1 -- 172.21.15.32:0/802967943 shutdown_connections 2024-04-17T09:58:49.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.538+0000 7fee2e200640 1 --2- 172.21.15.32:0/802967943 >> v2:172.21.15.144:3300/0 conn(0x7fee280b4de0 0x7fee2815baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:49.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.538+0000 7fee2e200640 1 --2- 172.21.15.32:0/802967943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee28144e90 0x7fee280b48a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:49.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.538+0000 7fee2e200640 1 -- 172.21.15.32:0/802967943 >> 172.21.15.32:0/802967943 conn(0x7fee2806ba30 msgr2=0x7fee2806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:49.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.538+0000 7fee2e200640 1 -- 172.21.15.32:0/802967943 shutdown_connections 2024-04-17T09:58:49.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.538+0000 7fee2e200640 1 -- 172.21.15.32:0/802967943 wait complete. 2024-04-17T09:58:49.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee2e200640 1 Processor -- start 2024-04-17T09:58:49.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee2e200640 1 -- start start 2024-04-17T09:58:49.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee2e200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee280b4de0 0x7fee2814ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:49.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee2e200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee28144e90 0x7fee280ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:49.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee2e200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee2815bfe0 0x7fee280aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:49.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee2e200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fee28072670 con 0x7fee280b4de0 2024-04-17T09:58:49.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee2e200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fee2806f8c0 con 0x7fee2815bfe0 2024-04-17T09:58:49.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee2e200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fee280727f0 con 0x7fee28144e90 2024-04-17T09:58:49.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee27400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee280b4de0 0x7fee2814ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:49.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee27400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee280b4de0 0x7fee2814ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38962/0 (socket says 172.21.15.32:38962) 2024-04-17T09:58:49.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee27400640 1 -- 172.21.15.32:0/1560672699 learned_addr learned my addr 172.21.15.32:0/1560672699 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:49.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee26a00640 1 --2- 172.21.15.32:0/1560672699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee28144e90 0x7fee280ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:49.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee27e00640 1 --2- 172.21.15.32:0/1560672699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee2815bfe0 0x7fee280aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:49.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee27400640 1 -- 172.21.15.32:0/1560672699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee28144e90 msgr2=0x7fee280ae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:49.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee27400640 1 --2- 172.21.15.32:0/1560672699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee28144e90 0x7fee280ae870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:49.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee27400640 1 -- 172.21.15.32:0/1560672699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee2815bfe0 msgr2=0x7fee280aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:49.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee27400640 1 --2- 172.21.15.32:0/1560672699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee2815bfe0 0x7fee280aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:49.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee27400640 1 -- 172.21.15.32:0/1560672699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2814f260 con 0x7fee280b4de0 2024-04-17T09:58:49.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee27400640 1 --2- 172.21.15.32:0/1560672699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee280b4de0 0x7fee2814ed20 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fee180ab040 tx=0x7fee1801ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:49.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee1f400640 1 -- 172.21.15.32:0/1560672699 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee180bc450 con 0x7fee280b4de0 2024-04-17T09:58:49.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee1f400640 1 -- 172.21.15.32:0/1560672699 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee180bc5f0 con 0x7fee280b4de0 2024-04-17T09:58:49.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee1f400640 1 -- 172.21.15.32:0/1560672699 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee180bc9c0 con 0x7fee280b4de0 2024-04-17T09:58:49.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee2e200640 1 -- 172.21.15.32:0/1560672699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee2813e8c0 con 0x7fee280b4de0 2024-04-17T09:58:49.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee2e200640 1 -- 172.21.15.32:0/1560672699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee2813db00 con 0x7fee280b4de0 2024-04-17T09:58:49.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.542+0000 7fee2e200640 1 -- 172.21.15.32:0/1560672699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee28068a90 con 0x7fee280b4de0 2024-04-17T09:58:49.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.546+0000 7fee1f400640 1 -- 172.21.15.32:0/1560672699 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fee180b13f0 con 0x7fee280b4de0 2024-04-17T09:58:49.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.546+0000 7fee1f400640 1 --2- 172.21.15.32:0/1560672699 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee08068380 0x7fee0806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:49.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.546+0000 7fee1f400640 1 -- 172.21.15.32:0/1560672699 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fee18120a50 con 0x7fee280b4de0 2024-04-17T09:58:49.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.546+0000 7fee26a00640 1 --2- 172.21.15.32:0/1560672699 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee08068380 0x7fee0806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:49.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.546+0000 7fee26a00640 1 --2- 172.21.15.32:0/1560672699 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee08068380 0x7fee0806a840 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fee2815bf30 tx=0x7fee14009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:49.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.550+0000 7fee1f400640 1 -- 172.21.15.32:0/1560672699 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fee180c3070 con 0x7fee280b4de0 2024-04-17T09:58:49.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.746+0000 7fee2e200640 1 -- 172.21.15.32:0/1560672699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee280d0100 con 0x7fee280b4de0 2024-04-17T09:58:49.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.746+0000 7fee1f400640 1 -- 172.21.15.32:0/1560672699 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5206 (secure 0 0 0) 0x7fee2814f260 con 0x7fee280b4de0 2024-04-17T09:58:49.746 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:58:49.746 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2424282223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2424282223},{"type":"v1","addr":"172.21.15.144:6834","nonce":2424282223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3537056692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3537056692},{"type":"v1","addr":"172.21.15.144:6835","nonce":3537056692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5891,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/725333724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":725333724},{"type":"v1","addr":"172.21.15.32:6837","nonce":725333724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:57:11.558183+0000","modified":"2024-04-17T09:58:17.770818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":15,"addr":"172.21.15.32:6836/2416784965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2416784965},{"type":"v1","addr":"172.21.15.32:6836","nonce":2416784965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7011,"qdb_cluster":[7011]},"id":5}]} 2024-04-17T09:58:49.746 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 50 2024-04-17T09:58:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.750+0000 7fee2e200640 1 -- 172.21.15.32:0/1560672699 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee08068380 msgr2=0x7fee0806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.750+0000 7fee2e200640 1 --2- 172.21.15.32:0/1560672699 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee08068380 0x7fee0806a840 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fee2815bf30 tx=0x7fee14009190 comp rx=0 tx=0).stop 2024-04-17T09:58:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.750+0000 7fee2e200640 1 -- 172.21.15.32:0/1560672699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee280b4de0 msgr2=0x7fee2814ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.750+0000 7fee2e200640 1 --2- 172.21.15.32:0/1560672699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee280b4de0 0x7fee2814ed20 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fee180ab040 tx=0x7fee1801ad70 comp rx=0 tx=0).stop 2024-04-17T09:58:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.750+0000 7fee2e200640 1 -- 172.21.15.32:0/1560672699 shutdown_connections 2024-04-17T09:58:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.750+0000 7fee2e200640 1 --2- 172.21.15.32:0/1560672699 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee08068380 0x7fee0806a840 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.750+0000 7fee2e200640 1 --2- 172.21.15.32:0/1560672699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee28144e90 0x7fee280ae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.750+0000 7fee2e200640 1 --2- 172.21.15.32:0/1560672699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee2815bfe0 0x7fee280aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:49.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.750+0000 7fee2e200640 1 --2- 172.21.15.32:0/1560672699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee280b4de0 0x7fee2814ed20 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:49.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.750+0000 7fee2e200640 1 -- 172.21.15.32:0/1560672699 >> 172.21.15.32:0/1560672699 conn(0x7fee2806ba30 msgr2=0x7fee280b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:49.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.750+0000 7fee2e200640 1 -- 172.21.15.32:0/1560672699 shutdown_connections 2024-04-17T09:58:49.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.750+0000 7fee2e200640 1 -- 172.21.15.32:0/1560672699 wait complete. 2024-04-17T09:58:49.767 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:58:49.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.910+0000 7f0761a00640 1 Processor -- start 2024-04-17T09:58:49.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.910+0000 7f0761a00640 1 -- start start 2024-04-17T09:58:49.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f0761a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f075c144f20 0x7f075c06e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:49.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f0761a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f075c06eda0 0x7f075c071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:49.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f0761a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f075c0aed20 con 0x7f075c1481e0 2024-04-17T09:58:49.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f0761a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f075c0b44a0 con 0x7f075c06eda0 2024-04-17T09:58:49.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f0761a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f075c0af000 con 0x7f075c144f20 2024-04-17T09:58:49.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075b400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f075c1481e0 0x7f075c162440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39018/0 (socket says 172.21.15.32:39018) 2024-04-17T09:58:49.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075b400640 1 -- 172.21.15.32:0/1783437565 learned_addr learned my addr 172.21.15.32:0/1783437565 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:58:49.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f0752000640 1 --2- 172.21.15.32:0/1783437565 >> v2:172.21.15.144:3300/0 conn(0x7f075c06eda0 0x7f075c071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:49.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075aa00640 1 --2- 172.21.15.32:0/1783437565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f075c144f20 0x7f075c06e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:49.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075a000640 1 -- 172.21.15.32:0/1783437565 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3774648894 0 0) 0x7f075c0aed20 con 0x7f075c1481e0 2024-04-17T09:58:49.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075a000640 1 -- 172.21.15.32:0/1783437565 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0740003530 con 0x7f075c1481e0 2024-04-17T09:58:49.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075a000640 1 -- 172.21.15.32:0/1783437565 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 468570516 0 0) 0x7f0740003530 con 0x7f075c1481e0 2024-04-17T09:58:49.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075a000640 1 -- 172.21.15.32:0/1783437565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f075c144f20 msgr2=0x7f075c06e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:49.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075a000640 1 --2- 172.21.15.32:0/1783437565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f075c144f20 0x7f075c06e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:49.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075a000640 1 -- 172.21.15.32:0/1783437565 >> v2:172.21.15.144:3300/0 conn(0x7f075c06eda0 msgr2=0x7f075c071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:49.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075a000640 1 --2- 172.21.15.32:0/1783437565 >> v2:172.21.15.144:3300/0 conn(0x7f075c06eda0 0x7f075c071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:49.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075a000640 1 -- 172.21.15.32:0/1783437565 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075c0af2e0 con 0x7f075c1481e0 2024-04-17T09:58:49.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075a000640 1 -- 172.21.15.32:0/1783437565 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f0754001a00 con 0x7f075c1481e0 2024-04-17T09:58:49.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075a000640 1 -- 172.21.15.32:0/1783437565 >> v1:172.21.15.32:6789/0 conn(0x7f075c1481e0 legacy=0x7f075c162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:49.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075a000640 1 --2- 172.21.15.32:0/1783437565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07400039c0 0x7f0740003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:49.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075a000640 1 --2- 172.21.15.32:0/1783437565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f075c144f20 0x7f0740004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:49.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075a000640 1 --2- 172.21.15.32:0/1783437565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0740004d50 0x7f07400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:49.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075a000640 1 -- 172.21.15.32:0/1783437565 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f075c0b44a0 con 0x7f07400039c0 2024-04-17T09:58:49.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075a000640 1 -- 172.21.15.32:0/1783437565 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f075c0af000 con 0x7f0740004d50 2024-04-17T09:58:49.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075a000640 1 -- 172.21.15.32:0/1783437565 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f074000b9d0 con 0x7f075c144f20 2024-04-17T09:58:49.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f0752000640 1 --2- 172.21.15.32:0/1783437565 >> v2:172.21.15.144:3300/0 conn(0x7f075c06eda0 0x7f075c071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:58:49.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075aa00640 1 --2- 172.21.15.32:0/1783437565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07400039c0 0x7f0740003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:49.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075b400640 1 --2- 172.21.15.32:0/1783437565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0740004d50 0x7f07400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:49.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f0752000640 1 --2- 172.21.15.32:0/1783437565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f075c144f20 0x7f0740004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:49.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075aa00640 1 -- 172.21.15.32:0/1783437565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f075c144f20 msgr2=0x7f0740004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:49.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075aa00640 1 --2- 172.21.15.32:0/1783437565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f075c144f20 0x7f0740004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:49.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075aa00640 1 -- 172.21.15.32:0/1783437565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0740004d50 msgr2=0x7f07400051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:49.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075aa00640 1 --2- 172.21.15.32:0/1783437565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0740004d50 0x7f07400051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:49.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f075aa00640 1 -- 172.21.15.32:0/1783437565 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0740003530 con 0x7f07400039c0 2024-04-17T09:58:49.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.914+0000 7f0752000640 1 --2- 172.21.15.32:0/1783437565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f075c144f20 0x7f0740004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:58:49.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.918+0000 7f075aa00640 1 --2- 172.21.15.32:0/1783437565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07400039c0 0x7f0740003de0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f074c009ae0 tx=0x7f074c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f36c9596ee15ed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:49.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.918+0000 7f075a000640 1 -- 172.21.15.32:0/1783437565 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c0096b0 con 0x7f07400039c0 2024-04-17T09:58:49.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.918+0000 7f075a000640 1 -- 172.21.15.32:0/1783437565 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f074c0a6450 con 0x7f07400039c0 2024-04-17T09:58:49.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.918+0000 7f075a000640 1 -- 172.21.15.32:0/1783437565 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c0a6840 con 0x7f07400039c0 2024-04-17T09:58:49.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.918+0000 7f0761a00640 1 -- 172.21.15.32:0/1783437565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07400039c0 msgr2=0x7f0740003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:49.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.918+0000 7f0761a00640 1 --2- 172.21.15.32:0/1783437565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07400039c0 0x7f0740003de0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f074c009ae0 tx=0x7f074c01ca70 comp rx=0 tx=0).stop 2024-04-17T09:58:49.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.918+0000 7f075b400640 1 -- 172.21.15.32:0/1783437565 reap_dead start 2024-04-17T09:58:49.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.918+0000 7f0761a00640 1 -- 172.21.15.32:0/1783437565 shutdown_connections 2024-04-17T09:58:49.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.918+0000 7f0761a00640 1 -- 172.21.15.32:0/1783437565 >> 172.21.15.32:0/1783437565 conn(0x7f075c06ba30 msgr2=0x7f075c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:49.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.918+0000 7f0761a00640 1 -- 172.21.15.32:0/1783437565 shutdown_connections 2024-04-17T09:58:49.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.918+0000 7f0761a00640 1 -- 172.21.15.32:0/1783437565 wait complete. 2024-04-17T09:58:49.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f0761a00640 1 Processor -- start 2024-04-17T09:58:49.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f0761a00640 1 -- start start 2024-04-17T09:58:49.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f0761a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07400039c0 0x7f075c0cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:49.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f0761a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f075c131870 0x7f075c0ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:49.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f0761a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f075c130c20 0x7f075c1310a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:49.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f0761a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1695c0 con 0x7f075c131870 2024-04-17T09:58:49.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f0761a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f075c072740 con 0x7f075c130c20 2024-04-17T09:58:49.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f0761a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f075c169740 con 0x7f07400039c0 2024-04-17T09:58:49.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f0752000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f075c131870 0x7f075c0ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:49.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f0752000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f075c131870 0x7f075c0ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38992/0 (socket says 172.21.15.32:38992) 2024-04-17T09:58:49.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f0752000640 1 -- 172.21.15.32:0/3446479263 learned_addr learned my addr 172.21.15.32:0/3446479263 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:49.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f075aa00640 1 --2- 172.21.15.32:0/3446479263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07400039c0 0x7f075c0cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:49.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f075b400640 1 --2- 172.21.15.32:0/3446479263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f075c130c20 0x7f075c1310a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:49.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f0752000640 1 -- 172.21.15.32:0/3446479263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07400039c0 msgr2=0x7f075c0cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:49.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f0752000640 1 --2- 172.21.15.32:0/3446479263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07400039c0 0x7f075c0cdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:49.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f0752000640 1 -- 172.21.15.32:0/3446479263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f075c130c20 msgr2=0x7f075c1310a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:49.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f0752000640 1 --2- 172.21.15.32:0/3446479263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f075c130c20 0x7f075c1310a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:49.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f0752000640 1 -- 172.21.15.32:0/3446479263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075c0b2bf0 con 0x7f075c131870 2024-04-17T09:58:49.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f0752000640 1 --2- 172.21.15.32:0/3446479263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f075c131870 0x7f075c0ceec0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f0744010c30 tx=0x7f074400e540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:49.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f0753400640 1 -- 172.21.15.32:0/3446479263 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074400f040 con 0x7f075c131870 2024-04-17T09:58:49.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f0753400640 1 -- 172.21.15.32:0/3446479263 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f074400ee50 con 0x7f075c131870 2024-04-17T09:58:49.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f0753400640 1 -- 172.21.15.32:0/3446479263 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074401c600 con 0x7f075c131870 2024-04-17T09:58:49.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f0761a00640 1 -- 172.21.15.32:0/3446479263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075c0ce700 con 0x7f075c131870 2024-04-17T09:58:49.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f0761a00640 1 -- 172.21.15.32:0/3446479263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f075c165920 con 0x7f075c131870 2024-04-17T09:58:49.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.922+0000 7f0761a00640 1 -- 172.21.15.32:0/3446479263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f075c068a90 con 0x7f075c131870 2024-04-17T09:58:49.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.926+0000 7f0753400640 1 -- 172.21.15.32:0/3446479263 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f07440143f0 con 0x7f075c131870 2024-04-17T09:58:49.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.926+0000 7f0753400640 1 --2- 172.21.15.32:0/3446479263 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0738068380 0x7f073806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:49.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.926+0000 7f0753400640 1 -- 172.21.15.32:0/3446479263 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f0744087190 con 0x7f075c131870 2024-04-17T09:58:49.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.926+0000 7f075aa00640 1 --2- 172.21.15.32:0/3446479263 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0738068380 0x7f073806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:49.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.926+0000 7f075aa00640 1 --2- 172.21.15.32:0/3446479263 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0738068380 0x7f073806a840 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f074c0ad000 tx=0x7f074c0b3040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:49.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:49.930+0000 7f0753400640 1 -- 172.21.15.32:0/3446479263 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0744018030 con 0x7f075c131870 2024-04-17T09:58:50.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.094+0000 7f0761a00640 1 -- 172.21.15.32:0/3446479263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f075c1468f0 con 0x7f075c131870 2024-04-17T09:58:50.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.094+0000 7f0753400640 1 -- 172.21.15.32:0/3446479263 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16904 (secure 0 0 0) 0x7f0744056f40 con 0x7f075c131870 2024-04-17T09:58:50.094 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:58:50.094 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":71,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:57:55.831704+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:57:09.948964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-17T09:57:10.512460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.a","mappings":[{"from":6,"to":0}]},{"pgid":"11.3a","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6836/663537244":"2024-04-18T09:57:30.746884+0000","172.21.15.32:6834/663537244":"2024-04-18T09:57:30.746884+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:58:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.098+0000 7f0761a00640 1 -- 172.21.15.32:0/3446479263 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0738068380 msgr2=0x7f073806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.098+0000 7f0761a00640 1 --2- 172.21.15.32:0/3446479263 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0738068380 0x7f073806a840 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f074c0ad000 tx=0x7f074c0b3040 comp rx=0 tx=0).stop 2024-04-17T09:58:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.098+0000 7f0761a00640 1 -- 172.21.15.32:0/3446479263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f075c131870 msgr2=0x7f075c0ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.098+0000 7f0761a00640 1 --2- 172.21.15.32:0/3446479263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f075c131870 0x7f075c0ceec0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f0744010c30 tx=0x7f074400e540 comp rx=0 tx=0).stop 2024-04-17T09:58:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.098+0000 7f0761a00640 1 -- 172.21.15.32:0/3446479263 shutdown_connections 2024-04-17T09:58:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.098+0000 7f0761a00640 1 --2- 172.21.15.32:0/3446479263 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0738068380 0x7f073806a840 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.098+0000 7f0761a00640 1 --2- 172.21.15.32:0/3446479263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f075c130c20 0x7f075c1310a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.098+0000 7f0761a00640 1 --2- 172.21.15.32:0/3446479263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f075c131870 0x7f075c0ceec0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.098+0000 7f0761a00640 1 --2- 172.21.15.32:0/3446479263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07400039c0 0x7f075c0cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.098+0000 7f0761a00640 1 -- 172.21.15.32:0/3446479263 >> 172.21.15.32:0/3446479263 conn(0x7f075c06ba30 msgr2=0x7f075c147290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:50.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.098+0000 7f0761a00640 1 -- 172.21.15.32:0/3446479263 shutdown_connections 2024-04-17T09:58:50.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.098+0000 7f0761a00640 1 -- 172.21.15.32:0/3446479263 wait complete. 2024-04-17T09:58:50.114 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:57:09.948964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:57:10.512460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:58:50.115 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:58:50.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.262+0000 7fc169000640 1 Processor -- start 2024-04-17T09:58:50.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.262+0000 7fc169000640 1 -- start start 2024-04-17T09:58:50.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.262+0000 7fc169000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc16406ba30 0x7fc16406be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.262+0000 7fc169000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1640cf370 0x7fc1640cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.262+0000 7fc169000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc16406c350 con 0x7fc164160350 2024-04-17T09:58:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.262+0000 7fc169000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc1640d0130 con 0x7fc16406ba30 2024-04-17T09:58:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.262+0000 7fc169000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc16415d090 con 0x7fc1640cf370 2024-04-17T09:58:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.262+0000 7fc161e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc16406ba30 0x7fc16406be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.262+0000 7fc161e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc16406ba30 0x7fc16406be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39002/0 (socket says 172.21.15.32:39002) 2024-04-17T09:58:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.262+0000 7fc161e00640 1 -- 172.21.15.32:0/3763099629 learned_addr learned my addr 172.21.15.32:0/3763099629 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:50.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.262+0000 7fc161400640 1 --2- 172.21.15.32:0/3763099629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1640cf370 0x7fc1640cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.262+0000 7fc161e00640 1 -- 172.21.15.32:0/3763099629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1640cf370 msgr2=0x7fc1640cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.262+0000 7fc161e00640 1 --2- 172.21.15.32:0/3763099629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1640cf370 0x7fc1640cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.262+0000 7fc161e00640 1 -- 172.21.15.32:0/3763099629 >> v1:172.21.15.32:6789/0 conn(0x7fc164160350 legacy=0x7fc1641607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.262+0000 7fc161e00640 1 -- 172.21.15.32:0/3763099629 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc16415d370 con 0x7fc16406ba30 2024-04-17T09:58:50.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.262+0000 7fc161400640 1 --2- 172.21.15.32:0/3763099629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1640cf370 0x7fc1640cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:58:50.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.266+0000 7fc161e00640 1 --2- 172.21.15.32:0/3763099629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc16406ba30 0x7fc16406be10 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fc154009870 tx=0x7fc15401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c7d81c375e7d9e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:50.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.266+0000 7fc160a00640 1 -- 172.21.15.32:0/3763099629 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1540a7070 con 0x7fc16406ba30 2024-04-17T09:58:50.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.266+0000 7fc160a00640 1 -- 172.21.15.32:0/3763099629 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1540b0070 con 0x7fc16406ba30 2024-04-17T09:58:50.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.266+0000 7fc160a00640 1 -- 172.21.15.32:0/3763099629 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1540ab890 con 0x7fc16406ba30 2024-04-17T09:58:50.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.266+0000 7fc169000640 1 -- 172.21.15.32:0/3763099629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc16406ba30 msgr2=0x7fc16406be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:50.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.266+0000 7fc169000640 1 --2- 172.21.15.32:0/3763099629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc16406ba30 0x7fc16406be10 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fc154009870 tx=0x7fc15401c560 comp rx=0 tx=0).stop 2024-04-17T09:58:50.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.266+0000 7fc169000640 1 -- 172.21.15.32:0/3763099629 shutdown_connections 2024-04-17T09:58:50.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.266+0000 7fc169000640 1 --2- 172.21.15.32:0/3763099629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1640cf370 0x7fc1640cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.266+0000 7fc169000640 1 --2- 172.21.15.32:0/3763099629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc16406ba30 0x7fc16406be10 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.266+0000 7fc169000640 1 -- 172.21.15.32:0/3763099629 >> 172.21.15.32:0/3763099629 conn(0x7fc1640b2120 msgr2=0x7fc1640b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:50.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.266+0000 7fc169000640 1 -- 172.21.15.32:0/3763099629 shutdown_connections 2024-04-17T09:58:50.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.266+0000 7fc169000640 1 -- 172.21.15.32:0/3763099629 wait complete. 2024-04-17T09:58:50.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.266+0000 7fc169000640 1 Processor -- start 2024-04-17T09:58:50.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.266+0000 7fc169000640 1 -- start start 2024-04-17T09:58:50.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc169000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc16406ba30 0x7fc1641f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:50.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc169000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1640cf370 0x7fc164141780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:50.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc169000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc164160350 0x7fc16413d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:50.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc169000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc16415dcf0 con 0x7fc1640cf370 2024-04-17T09:58:50.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc169000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc16415db70 con 0x7fc16406ba30 2024-04-17T09:58:50.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc169000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc164146230 con 0x7fc164160350 2024-04-17T09:58:50.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc161400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1640cf370 0x7fc164141780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:50.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc161400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1640cf370 0x7fc164141780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39006/0 (socket says 172.21.15.32:39006) 2024-04-17T09:58:50.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc161400640 1 -- 172.21.15.32:0/619495720 learned_addr learned my addr 172.21.15.32:0/619495720 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:50.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc161e00640 1 --2- 172.21.15.32:0/619495720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc16406ba30 0x7fc1641f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:50.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc162800640 1 --2- 172.21.15.32:0/619495720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc164160350 0x7fc16413d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:50.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc161400640 1 -- 172.21.15.32:0/619495720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc164160350 msgr2=0x7fc16413d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:50.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc161400640 1 --2- 172.21.15.32:0/619495720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc164160350 0x7fc16413d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc161400640 1 -- 172.21.15.32:0/619495720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc16406ba30 msgr2=0x7fc1641f72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:50.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc161400640 1 --2- 172.21.15.32:0/619495720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc16406ba30 0x7fc1641f72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc161400640 1 -- 172.21.15.32:0/619495720 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1641f77f0 con 0x7fc1640cf370 2024-04-17T09:58:50.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc161400640 1 --2- 172.21.15.32:0/619495720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1640cf370 0x7fc164141780 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fc15000c7c0 tx=0x7fc15000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:50.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc15a000640 1 -- 172.21.15.32:0/619495720 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc150007a30 con 0x7fc1640cf370 2024-04-17T09:58:50.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc15a000640 1 -- 172.21.15.32:0/619495720 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc150007bd0 con 0x7fc1640cf370 2024-04-17T09:58:50.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc169000640 1 -- 172.21.15.32:0/619495720 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1641406e0 con 0x7fc1640cf370 2024-04-17T09:58:50.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc15a000640 1 -- 172.21.15.32:0/619495720 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1500113f0 con 0x7fc1640cf370 2024-04-17T09:58:50.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc169000640 1 -- 172.21.15.32:0/619495720 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc164140ca0 con 0x7fc1640cf370 2024-04-17T09:58:50.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc169000640 1 -- 172.21.15.32:0/619495720 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc164068a90 con 0x7fc1640cf370 2024-04-17T09:58:50.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.270+0000 7fc15a000640 1 -- 172.21.15.32:0/619495720 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc150011590 con 0x7fc1640cf370 2024-04-17T09:58:50.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.274+0000 7fc15a000640 1 --2- 172.21.15.32:0/619495720 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc144068380 0x7fc14406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:50.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.274+0000 7fc15a000640 1 -- 172.21.15.32:0/619495720 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fc150086c10 con 0x7fc1640cf370 2024-04-17T09:58:50.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.274+0000 7fc161e00640 1 --2- 172.21.15.32:0/619495720 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc144068380 0x7fc14406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:50.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.274+0000 7fc161e00640 1 --2- 172.21.15.32:0/619495720 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc144068380 0x7fc14406a840 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc1540af000 tx=0x7fc1540a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:50.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.278+0000 7fc15a000640 1 -- 172.21.15.32:0/619495720 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc150015020 con 0x7fc1640cf370 2024-04-17T09:58:50.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.442+0000 7fc169000640 1 -- 172.21.15.32:0/619495720 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc16406c850 con 0x7fc1640cf370 2024-04-17T09:58:50.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.442+0000 7fc15a000640 1 -- 172.21.15.32:0/619495720 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7fc150056a50 con 0x7fc1640cf370 2024-04-17T09:58:50.440 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:58:50.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.442+0000 7fc169000640 1 -- 172.21.15.32:0/619495720 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc144068380 msgr2=0x7fc14406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:50.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.442+0000 7fc169000640 1 --2- 172.21.15.32:0/619495720 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc144068380 0x7fc14406a840 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc1540af000 tx=0x7fc1540a7000 comp rx=0 tx=0).stop 2024-04-17T09:58:50.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.442+0000 7fc169000640 1 -- 172.21.15.32:0/619495720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1640cf370 msgr2=0x7fc164141780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:50.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.442+0000 7fc169000640 1 --2- 172.21.15.32:0/619495720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1640cf370 0x7fc164141780 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fc15000c7c0 tx=0x7fc15000cc80 comp rx=0 tx=0).stop 2024-04-17T09:58:50.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.442+0000 7fc169000640 1 -- 172.21.15.32:0/619495720 shutdown_connections 2024-04-17T09:58:50.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.442+0000 7fc169000640 1 --2- 172.21.15.32:0/619495720 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc144068380 0x7fc14406a840 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.442+0000 7fc169000640 1 --2- 172.21.15.32:0/619495720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc164160350 0x7fc16413d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.442+0000 7fc169000640 1 --2- 172.21.15.32:0/619495720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1640cf370 0x7fc164141780 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.442+0000 7fc169000640 1 --2- 172.21.15.32:0/619495720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc16406ba30 0x7fc1641f72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.442+0000 7fc169000640 1 -- 172.21.15.32:0/619495720 >> 172.21.15.32:0/619495720 conn(0x7fc1640b2120 msgr2=0x7fc1640d0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:50.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.446+0000 7fc169000640 1 -- 172.21.15.32:0/619495720 shutdown_connections 2024-04-17T09:58:50.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.446+0000 7fc169000640 1 -- 172.21.15.32:0/619495720 wait complete. 2024-04-17T09:58:50.462 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:58:50.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.606+0000 7f226b200640 1 Processor -- start 2024-04-17T09:58:50.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.606+0000 7f226b200640 1 -- start start 2024-04-17T09:58:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.610+0000 7f226b200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f226c15ffc0 0x7f226c0b3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.610+0000 7f226b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f226c09b7c0 0x7f226c09a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.610+0000 7f226b200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f226c09af30 0x7f226c160490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.610+0000 7f226b200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f226c09bc20 con 0x7f226c09af30 2024-04-17T09:58:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.610+0000 7f226b200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f226c09b390 con 0x7f226c15ffc0 2024-04-17T09:58:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.610+0000 7f226b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f226c0d00e0 con 0x7f226c09b7c0 2024-04-17T09:58:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.610+0000 7f226a800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f226c09af30 0x7f226c160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.610+0000 7f226a800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f226c09af30 0x7f226c160490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34628/0 (socket says 172.21.15.32:34628) 2024-04-17T09:58:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.610+0000 7f226a800640 1 -- 172.21.15.32:0/2808871886 learned_addr learned my addr 172.21.15.32:0/2808871886 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.610+0000 7f2261400640 1 --2- 172.21.15.32:0/2808871886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f226c09b7c0 0x7f226c09a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.610+0000 7f2269e00640 1 --2- 172.21.15.32:0/2808871886 >> v2:172.21.15.144:3300/0 conn(0x7f226c15ffc0 0x7f226c0b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.610+0000 7f226a800640 1 -- 172.21.15.32:0/2808871886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f226c09b7c0 msgr2=0x7f226c09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.610+0000 7f226a800640 1 --2- 172.21.15.32:0/2808871886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f226c09b7c0 0x7f226c09a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.610+0000 7f226a800640 1 -- 172.21.15.32:0/2808871886 >> v2:172.21.15.144:3300/0 conn(0x7f226c15ffc0 msgr2=0x7f226c0b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:50.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.610+0000 7f226a800640 1 --2- 172.21.15.32:0/2808871886 >> v2:172.21.15.144:3300/0 conn(0x7f226c15ffc0 0x7f226c0b3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.610+0000 7f226a800640 1 -- 172.21.15.32:0/2808871886 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226c0cddb0 con 0x7f226c09af30 2024-04-17T09:58:50.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.614+0000 7f226a800640 1 --2- 172.21.15.32:0/2808871886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f226c09af30 0x7f226c160490 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f226c073360 tx=0x7f226401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139c8662c403d763 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:50.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.614+0000 7f2269400640 1 -- 172.21.15.32:0/2808871886 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22640b0070 con 0x7f226c09af30 2024-04-17T09:58:50.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.614+0000 7f2269400640 1 -- 172.21.15.32:0/2808871886 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22640ab600 con 0x7f226c09af30 2024-04-17T09:58:50.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.614+0000 7f2269400640 1 -- 172.21.15.32:0/2808871886 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22640ab9d0 con 0x7f226c09af30 2024-04-17T09:58:50.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.614+0000 7f226b200640 1 -- 172.21.15.32:0/2808871886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f226c09af30 msgr2=0x7f226c160490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:50.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.614+0000 7f226b200640 1 --2- 172.21.15.32:0/2808871886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f226c09af30 0x7f226c160490 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f226c073360 tx=0x7f226401c260 comp rx=0 tx=0).stop 2024-04-17T09:58:50.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.614+0000 7f226b200640 1 -- 172.21.15.32:0/2808871886 shutdown_connections 2024-04-17T09:58:50.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.614+0000 7f226b200640 1 --2- 172.21.15.32:0/2808871886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f226c09af30 0x7f226c160490 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.614+0000 7f226b200640 1 --2- 172.21.15.32:0/2808871886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f226c09b7c0 0x7f226c09a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.614+0000 7f226b200640 1 --2- 172.21.15.32:0/2808871886 >> v2:172.21.15.144:3300/0 conn(0x7f226c15ffc0 0x7f226c0b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.614+0000 7f226b200640 1 -- 172.21.15.32:0/2808871886 >> 172.21.15.32:0/2808871886 conn(0x7f226c0b4000 msgr2=0x7f226c0b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:50.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.614+0000 7f226b200640 1 -- 172.21.15.32:0/2808871886 shutdown_connections 2024-04-17T09:58:50.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.614+0000 7f226b200640 1 -- 172.21.15.32:0/2808871886 wait complete. 2024-04-17T09:58:50.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.614+0000 7f226b200640 1 Processor -- start 2024-04-17T09:58:50.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.614+0000 7f226b200640 1 -- start start 2024-04-17T09:58:50.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.614+0000 7f226b200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f226c09af30 0x7f226c0c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:50.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.614+0000 7f226b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f226c09b7c0 0x7f226c1faa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:50.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.614+0000 7f226b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f226c15ffc0 0x7f226c1faf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:50.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.614+0000 7f226b200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f226c0b1ea0 con 0x7f226c09b7c0 2024-04-17T09:58:50.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.614+0000 7f226b200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f226c0b1d20 con 0x7f226c09af30 2024-04-17T09:58:50.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.614+0000 7f226b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f226c0d0570 con 0x7f226c15ffc0 2024-04-17T09:58:50.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.618+0000 7f2261400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f226c09b7c0 0x7f226c1faa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:50.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.618+0000 7f2261400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f226c09b7c0 0x7f226c1faa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34638/0 (socket says 172.21.15.32:34638) 2024-04-17T09:58:50.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.618+0000 7f2261400640 1 -- 172.21.15.32:0/1963637578 learned_addr learned my addr 172.21.15.32:0/1963637578 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:50.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.618+0000 7f2269e00640 1 --2- 172.21.15.32:0/1963637578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f226c09af30 0x7f226c0c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:50.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.618+0000 7f226a800640 1 --2- 172.21.15.32:0/1963637578 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f226c15ffc0 0x7f226c1faf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:50.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.618+0000 7f2261400640 1 -- 172.21.15.32:0/1963637578 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f226c15ffc0 msgr2=0x7f226c1faf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:50.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.618+0000 7f2261400640 1 --2- 172.21.15.32:0/1963637578 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f226c15ffc0 0x7f226c1faf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.618+0000 7f2261400640 1 -- 172.21.15.32:0/1963637578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f226c09af30 msgr2=0x7f226c0c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:50.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.618+0000 7f2261400640 1 --2- 172.21.15.32:0/1963637578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f226c09af30 0x7f226c0c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.618+0000 7f2261400640 1 -- 172.21.15.32:0/1963637578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226c0b6f10 con 0x7f226c09b7c0 2024-04-17T09:58:50.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.618+0000 7f2269e00640 1 --2- 172.21.15.32:0/1963637578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f226c09af30 0x7f226c0c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:58:50.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.618+0000 7f2261400640 1 --2- 172.21.15.32:0/1963637578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f226c09b7c0 0x7f226c1faa50 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f225400e8e0 tx=0x7f225400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:50.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.618+0000 7f2262a00640 1 -- 172.21.15.32:0/1963637578 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2254017070 con 0x7f226c09b7c0 2024-04-17T09:58:50.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.618+0000 7f2262a00640 1 -- 172.21.15.32:0/1963637578 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f225400ca00 con 0x7f226c09b7c0 2024-04-17T09:58:50.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.618+0000 7f226b200640 1 -- 172.21.15.32:0/1963637578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f226c0b71f0 con 0x7f226c09b7c0 2024-04-17T09:58:50.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.618+0000 7f2262a00640 1 -- 172.21.15.32:0/1963637578 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225400cdd0 con 0x7f226c09b7c0 2024-04-17T09:58:50.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.618+0000 7f226b200640 1 -- 172.21.15.32:0/1963637578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f226c0b5260 con 0x7f226c09b7c0 2024-04-17T09:58:50.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.618+0000 7f226b200640 1 -- 172.21.15.32:0/1963637578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f226c04fda0 con 0x7f226c09b7c0 2024-04-17T09:58:50.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.618+0000 7f2262a00640 1 -- 172.21.15.32:0/1963637578 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f225401a070 con 0x7f226c09b7c0 2024-04-17T09:58:50.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.618+0000 7f2262a00640 1 --2- 172.21.15.32:0/1963637578 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2248068380 0x7f224806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:50.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.622+0000 7f2262a00640 1 -- 172.21.15.32:0/1963637578 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f2254086e60 con 0x7f226c09b7c0 2024-04-17T09:58:50.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.622+0000 7f2269e00640 1 --2- 172.21.15.32:0/1963637578 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2248068380 0x7f224806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:50.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.622+0000 7f2269e00640 1 --2- 172.21.15.32:0/1963637578 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2248068380 0x7f224806a840 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f225c009fd0 tx=0x7f225c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:50.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.626+0000 7f2262a00640 1 -- 172.21.15.32:0/1963637578 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2254057380 con 0x7f226c09b7c0 2024-04-17T09:58:50.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.790+0000 7f226b200640 1 -- 172.21.15.32:0/1963637578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f226c0b4bf0 con 0x7f226c09b7c0 2024-04-17T09:58:50.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.790+0000 7f2262a00640 1 -- 172.21.15.32:0/1963637578 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) ==== 104+0+11 (secure 0 0 0) 0x7f2254056c80 con 0x7f226c09b7c0 2024-04-17T09:58:50.789 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:58:50.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.794+0000 7f226b200640 1 -- 172.21.15.32:0/1963637578 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2248068380 msgr2=0x7f224806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:50.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.794+0000 7f226b200640 1 --2- 172.21.15.32:0/1963637578 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2248068380 0x7f224806a840 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f225c009fd0 tx=0x7f225c00b040 comp rx=0 tx=0).stop 2024-04-17T09:58:50.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.794+0000 7f226b200640 1 -- 172.21.15.32:0/1963637578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f226c09b7c0 msgr2=0x7f226c1faa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:50.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.794+0000 7f226b200640 1 --2- 172.21.15.32:0/1963637578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f226c09b7c0 0x7f226c1faa50 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f225400e8e0 tx=0x7f225400eda0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.794+0000 7f226b200640 1 -- 172.21.15.32:0/1963637578 shutdown_connections 2024-04-17T09:58:50.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.794+0000 7f226b200640 1 --2- 172.21.15.32:0/1963637578 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2248068380 0x7f224806a840 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.794+0000 7f226b200640 1 --2- 172.21.15.32:0/1963637578 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f226c15ffc0 0x7f226c1faf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.794+0000 7f226b200640 1 --2- 172.21.15.32:0/1963637578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f226c09b7c0 0x7f226c1faa50 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.794+0000 7f226b200640 1 --2- 172.21.15.32:0/1963637578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f226c09af30 0x7f226c0c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.794+0000 7f226b200640 1 -- 172.21.15.32:0/1963637578 >> 172.21.15.32:0/1963637578 conn(0x7f226c0b4000 msgr2=0x7f226c0716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:50.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.794+0000 7f226b200640 1 -- 172.21.15.32:0/1963637578 shutdown_connections 2024-04-17T09:58:50.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.794+0000 7f226b200640 1 -- 172.21.15.32:0/1963637578 wait complete. 2024-04-17T09:58:50.810 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:58:50.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc0a200640 1 Processor -- start 2024-04-17T09:58:50.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc0a200640 1 -- start start 2024-04-17T09:58:50.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc0a200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc040ce740 0x7fdc0409a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:50.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc0a200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc0415cc40 con 0x7fdc0415c860 2024-04-17T09:58:50.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc0a200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc040cd6a0 con 0x7fdc040ceba0 2024-04-17T09:58:50.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc0a200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdc040ce2b0 con 0x7fdc040ce740 2024-04-17T09:58:50.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc03400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdc0415c860 0x7fdc040d1050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49046/0 (socket says 172.21.15.32:49046) 2024-04-17T09:58:50.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc03400640 1 -- 172.21.15.32:0/232773973 learned_addr learned my addr 172.21.15.32:0/232773973 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:58:50.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdbfbe00640 1 --2- 172.21.15.32:0/232773973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc040ce740 0x7fdc0409a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:50.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc02a00640 1 -- 172.21.15.32:0/232773973 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 206934487 0 0) 0x7fdc0415cc40 con 0x7fdc0415c860 2024-04-17T09:58:50.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc02a00640 1 -- 172.21.15.32:0/232773973 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbe8003530 con 0x7fdc0415c860 2024-04-17T09:58:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc02a00640 1 -- 172.21.15.32:0/232773973 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 733921205 0 0) 0x7fdbe8003530 con 0x7fdc0415c860 2024-04-17T09:58:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc02a00640 1 -- 172.21.15.32:0/232773973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc040ce740 msgr2=0x7fdc0409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc02a00640 1 --2- 172.21.15.32:0/232773973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc040ce740 0x7fdc0409a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc02a00640 1 -- 172.21.15.32:0/232773973 >> v1:172.21.15.144:6789/0 conn(0x7fdc040ceba0 legacy=0x7fdc040b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc02a00640 1 -- 172.21.15.32:0/232773973 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc040cd980 con 0x7fdc0415c860 2024-04-17T09:58:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc02a00640 1 -- 172.21.15.32:0/232773973 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7fdbf4001680 con 0x7fdc0415c860 2024-04-17T09:58:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc02a00640 1 -- 172.21.15.32:0/232773973 >> v1:172.21.15.32:6789/0 conn(0x7fdc0415c860 legacy=0x7fdc040d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc02a00640 1 --2- 172.21.15.32:0/232773973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbe80039f0 0x7fdbe8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc02a00640 1 --2- 172.21.15.32:0/232773973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc040ce740 0x7fdbe8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc02a00640 1 --2- 172.21.15.32:0/232773973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbe8004d50 0x7fdbe80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc02a00640 1 -- 172.21.15.32:0/232773973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdc040ce2b0 con 0x7fdbe80039f0 2024-04-17T09:58:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc02a00640 1 -- 172.21.15.32:0/232773973 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe800b9d0 con 0x7fdbe8004d50 2024-04-17T09:58:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc02a00640 1 -- 172.21.15.32:0/232773973 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdbe800bbd0 con 0x7fdc040ce740 2024-04-17T09:58:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc03400640 1 --2- 172.21.15.32:0/232773973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbe80039f0 0x7fdbe8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc03e00640 1 --2- 172.21.15.32:0/232773973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbe8004d50 0x7fdbe80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdbfbe00640 1 --2- 172.21.15.32:0/232773973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc040ce740 0x7fdbe8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc03400640 1 -- 172.21.15.32:0/232773973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc040ce740 msgr2=0x7fdbe8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc03400640 1 --2- 172.21.15.32:0/232773973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc040ce740 0x7fdbe8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc03400640 1 -- 172.21.15.32:0/232773973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbe8004d50 msgr2=0x7fdbe80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc03400640 1 --2- 172.21.15.32:0/232773973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbe8004d50 0x7fdbe80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc03400640 1 -- 172.21.15.32:0/232773973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe8003530 con 0x7fdbe80039f0 2024-04-17T09:58:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.958+0000 7fdc03e00640 1 --2- 172.21.15.32:0/232773973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbe8004d50 0x7fdbe80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:58:50.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.962+0000 7fdc03400640 1 --2- 172.21.15.32:0/232773973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbe80039f0 0x7fdbe8003dd0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fdbf400c2c0 tx=0x7fdbf401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dcfde1e3cc09305 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:50.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.962+0000 7fdc02a00640 1 -- 172.21.15.32:0/232773973 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4007d60 con 0x7fdbe80039f0 2024-04-17T09:58:50.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.962+0000 7fdc02a00640 1 -- 172.21.15.32:0/232773973 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbf40a9070 con 0x7fdbe80039f0 2024-04-17T09:58:50.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.962+0000 7fdc02a00640 1 -- 172.21.15.32:0/232773973 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4005d40 con 0x7fdbe80039f0 2024-04-17T09:58:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.962+0000 7fdc0a200640 1 -- 172.21.15.32:0/232773973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbe80039f0 msgr2=0x7fdbe8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.962+0000 7fdc0a200640 1 --2- 172.21.15.32:0/232773973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbe80039f0 0x7fdbe8003dd0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fdbf400c2c0 tx=0x7fdbf401cc80 comp rx=0 tx=0).stop 2024-04-17T09:58:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.962+0000 7fdc03e00640 1 -- 172.21.15.32:0/232773973 reap_dead start 2024-04-17T09:58:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.962+0000 7fdc0a200640 1 -- 172.21.15.32:0/232773973 shutdown_connections 2024-04-17T09:58:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.962+0000 7fdc0a200640 1 -- 172.21.15.32:0/232773973 >> 172.21.15.32:0/232773973 conn(0x7fdc040b4050 msgr2=0x7fdc040b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.962+0000 7fdc0a200640 1 -- 172.21.15.32:0/232773973 shutdown_connections 2024-04-17T09:58:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.962+0000 7fdc0a200640 1 -- 172.21.15.32:0/232773973 wait complete. 2024-04-17T09:58:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdc0a200640 1 Processor -- start 2024-04-17T09:58:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdc0a200640 1 -- start start 2024-04-17T09:58:50.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdc0a200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbe80039f0 0x7fdc040c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:50.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdc0a200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc040b8060 0x7fdc040cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:50.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdc0a200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc040ca1b0 0x7fdc0415ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:50.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdc0a200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdc04168720 con 0x7fdc040b8060 2024-04-17T09:58:50.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdc0a200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdc040cd350 con 0x7fdbe80039f0 2024-04-17T09:58:50.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdc0a200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdc041688a0 con 0x7fdc040ca1b0 2024-04-17T09:58:50.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdbfbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc040b8060 0x7fdc040cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:50.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdbfbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc040b8060 0x7fdc040cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34646/0 (socket says 172.21.15.32:34646) 2024-04-17T09:58:50.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdbfbe00640 1 -- 172.21.15.32:0/95377872 learned_addr learned my addr 172.21.15.32:0/95377872 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:50.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdc03400640 1 --2- 172.21.15.32:0/95377872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbe80039f0 0x7fdc040c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:50.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdc03e00640 1 --2- 172.21.15.32:0/95377872 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc040ca1b0 0x7fdc0415ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:50.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdbfbe00640 1 -- 172.21.15.32:0/95377872 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc040ca1b0 msgr2=0x7fdc0415ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:50.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdbfbe00640 1 --2- 172.21.15.32:0/95377872 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc040ca1b0 0x7fdc0415ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdbfbe00640 1 -- 172.21.15.32:0/95377872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbe80039f0 msgr2=0x7fdc040c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:50.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdbfbe00640 1 --2- 172.21.15.32:0/95377872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbe80039f0 0x7fdc040c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:50.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdbfbe00640 1 -- 172.21.15.32:0/95377872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc040cbfe0 con 0x7fdc040b8060 2024-04-17T09:58:50.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdbfbe00640 1 --2- 172.21.15.32:0/95377872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc040b8060 0x7fdc040cbaa0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fdbec011640 tx=0x7fdbec011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:50.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdbfb400640 1 -- 172.21.15.32:0/95377872 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec015870 con 0x7fdc040b8060 2024-04-17T09:58:50.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdbfb400640 1 -- 172.21.15.32:0/95377872 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbec015a10 con 0x7fdc040b8060 2024-04-17T09:58:50.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdc0a200640 1 -- 172.21.15.32:0/95377872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc040caa00 con 0x7fdc040b8060 2024-04-17T09:58:50.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdbfb400640 1 -- 172.21.15.32:0/95377872 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec015dc0 con 0x7fdc040b8060 2024-04-17T09:58:50.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdc0a200640 1 -- 172.21.15.32:0/95377872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc040cafc0 con 0x7fdc040b8060 2024-04-17T09:58:50.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.966+0000 7fdc0a200640 1 -- 172.21.15.32:0/95377872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc04068a90 con 0x7fdc040b8060 2024-04-17T09:58:50.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.970+0000 7fdbfb400640 1 -- 172.21.15.32:0/95377872 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdbec01d400 con 0x7fdc040b8060 2024-04-17T09:58:50.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.970+0000 7fdbfb400640 1 --2- 172.21.15.32:0/95377872 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdbe0068380 0x7fdbe006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:50.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.970+0000 7fdbfb400640 1 -- 172.21.15.32:0/95377872 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fdbec087120 con 0x7fdc040b8060 2024-04-17T09:58:50.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.970+0000 7fdc03400640 1 --2- 172.21.15.32:0/95377872 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdbe0068380 0x7fdbe006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:50.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.970+0000 7fdc03400640 1 --2- 172.21.15.32:0/95377872 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdbe0068380 0x7fdbe006a840 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40ab000 tx=0x7fdbf40a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:50.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:50.974+0000 7fdbfb400640 1 -- 172.21.15.32:0/95377872 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fdbec019030 con 0x7fdc040b8060 2024-04-17T09:58:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.138+0000 7fdc0a200640 1 -- 172.21.15.32:0/95377872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdc0415cc40 con 0x7fdc040b8060 2024-04-17T09:58:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.138+0000 7fdbfb400640 1 -- 172.21.15.32:0/95377872 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) ==== 100+0+11 (secure 0 0 0) 0x7fdbec056f60 con 0x7fdc040b8060 2024-04-17T09:58:51.137 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:58:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.142+0000 7fdc0a200640 1 -- 172.21.15.32:0/95377872 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdbe0068380 msgr2=0x7fdbe006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.142+0000 7fdc0a200640 1 --2- 172.21.15.32:0/95377872 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdbe0068380 0x7fdbe006a840 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40ab000 tx=0x7fdbf40a7070 comp rx=0 tx=0).stop 2024-04-17T09:58:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.142+0000 7fdc0a200640 1 -- 172.21.15.32:0/95377872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc040b8060 msgr2=0x7fdc040cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.142+0000 7fdc0a200640 1 --2- 172.21.15.32:0/95377872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc040b8060 0x7fdc040cbaa0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fdbec011640 tx=0x7fdbec011b00 comp rx=0 tx=0).stop 2024-04-17T09:58:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.142+0000 7fdc0a200640 1 -- 172.21.15.32:0/95377872 shutdown_connections 2024-04-17T09:58:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.142+0000 7fdc0a200640 1 --2- 172.21.15.32:0/95377872 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdbe0068380 0x7fdbe006a840 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.142+0000 7fdc0a200640 1 --2- 172.21.15.32:0/95377872 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc040ca1b0 0x7fdc0415ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.142+0000 7fdc0a200640 1 --2- 172.21.15.32:0/95377872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc040b8060 0x7fdc040cbaa0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.142+0000 7fdc0a200640 1 --2- 172.21.15.32:0/95377872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbe80039f0 0x7fdc040c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.142+0000 7fdc0a200640 1 -- 172.21.15.32:0/95377872 >> 172.21.15.32:0/95377872 conn(0x7fdc040b4050 msgr2=0x7fdc040d0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.142+0000 7fdc0a200640 1 -- 172.21.15.32:0/95377872 shutdown_connections 2024-04-17T09:58:51.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.142+0000 7fdc0a200640 1 -- 172.21.15.32:0/95377872 wait complete. 2024-04-17T09:58:51.158 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:58:51.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.302+0000 7fcabf200640 1 Processor -- start 2024-04-17T09:58:51.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.302+0000 7fcabf200640 1 -- start start 2024-04-17T09:58:51.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.302+0000 7fcabf200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcac015eb60 0x7fcac00b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:51.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.302+0000 7fcabf200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcac00b4970 0x7fcac00cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:51.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.302+0000 7fcabf200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcac00d0c70 con 0x7fcac009a9f0 2024-04-17T09:58:51.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.302+0000 7fcabf200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fcac015ef40 con 0x7fcac00b4970 2024-04-17T09:58:51.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.302+0000 7fcabf200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fcac006d480 con 0x7fcac015eb60 2024-04-17T09:58:51.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.302+0000 7fcab5400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcac00b4970 0x7fcac00cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.302+0000 7fcab5400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcac00b4970 0x7fcac00cc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34654/0 (socket says 172.21.15.32:34654) 2024-04-17T09:58:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.302+0000 7fcab5400640 1 -- 172.21.15.32:0/2497852859 learned_addr learned my addr 172.21.15.32:0/2497852859 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.302+0000 7fcabde00640 1 --2- 172.21.15.32:0/2497852859 >> v2:172.21.15.144:3300/0 conn(0x7fcac015eb60 0x7fcac00b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.302+0000 7fcab5400640 1 -- 172.21.15.32:0/2497852859 >> v2:172.21.15.144:3300/0 conn(0x7fcac015eb60 msgr2=0x7fcac00b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.302+0000 7fcab5400640 1 --2- 172.21.15.32:0/2497852859 >> v2:172.21.15.144:3300/0 conn(0x7fcac015eb60 0x7fcac00b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.302+0000 7fcab5400640 1 -- 172.21.15.32:0/2497852859 >> v1:172.21.15.32:6789/0 conn(0x7fcac009a9f0 legacy=0x7fcac006d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.302+0000 7fcab5400640 1 -- 172.21.15.32:0/2497852859 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac00d0f50 con 0x7fcac00b4970 2024-04-17T09:58:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.302+0000 7fcabd400640 1 -- 172.21.15.32:0/2497852859 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3485521215 0 0) 0x7fcac00d0c70 con 0x7fcac009a9f0 2024-04-17T09:58:51.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.306+0000 7fcab5400640 1 --2- 172.21.15.32:0/2497852859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcac00b4970 0x7fcac00cc9b0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009870 tx=0x7fcaa801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16b65b05e17cbc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:51.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.306+0000 7fcabd400640 1 -- 172.21.15.32:0/2497852859 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa80ab600 con 0x7fcac00b4970 2024-04-17T09:58:51.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.306+0000 7fcabd400640 1 -- 172.21.15.32:0/2497852859 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaa80b0070 con 0x7fcac00b4970 2024-04-17T09:58:51.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.306+0000 7fcabd400640 1 -- 172.21.15.32:0/2497852859 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa80ab9c0 con 0x7fcac00b4970 2024-04-17T09:58:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.306+0000 7fcabf200640 1 -- 172.21.15.32:0/2497852859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcac00b4970 msgr2=0x7fcac00cc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.306+0000 7fcabf200640 1 --2- 172.21.15.32:0/2497852859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcac00b4970 0x7fcac00cc9b0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009870 tx=0x7fcaa801c460 comp rx=0 tx=0).stop 2024-04-17T09:58:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.306+0000 7fcabf200640 1 -- 172.21.15.32:0/2497852859 shutdown_connections 2024-04-17T09:58:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.306+0000 7fcabf200640 1 --2- 172.21.15.32:0/2497852859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcac00b4970 0x7fcac00cc9b0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.306+0000 7fcabf200640 1 --2- 172.21.15.32:0/2497852859 >> v2:172.21.15.144:3300/0 conn(0x7fcac015eb60 0x7fcac00b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.306+0000 7fcabf200640 1 -- 172.21.15.32:0/2497852859 >> 172.21.15.32:0/2497852859 conn(0x7fcac00b3e50 msgr2=0x7fcac00b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.306+0000 7fcabf200640 1 -- 172.21.15.32:0/2497852859 shutdown_connections 2024-04-17T09:58:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.306+0000 7fcabf200640 1 -- 172.21.15.32:0/2497852859 wait complete. 2024-04-17T09:58:51.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcabf200640 1 Processor -- start 2024-04-17T09:58:51.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcabf200640 1 -- start start 2024-04-17T09:58:51.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcabf200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcac009a9f0 0x7fcac00b7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:51.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcabf200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcac00b4970 0x7fcac0157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:51.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcabf200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcac015eb60 0x7fcac0158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:51.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcabf200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcac009b210 con 0x7fcac015eb60 2024-04-17T09:58:51.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcabf200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcac016b900 con 0x7fcac009a9f0 2024-04-17T09:58:51.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcabf200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcac01582d0 con 0x7fcac00b4970 2024-04-17T09:58:51.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcabe800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcac015eb60 0x7fcac0158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcabe800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcac015eb60 0x7fcac0158510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34666/0 (socket says 172.21.15.32:34666) 2024-04-17T09:58:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcabe800640 1 -- 172.21.15.32:0/2118278033 learned_addr learned my addr 172.21.15.32:0/2118278033 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcabde00640 1 --2- 172.21.15.32:0/2118278033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcac009a9f0 0x7fcac00b7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcab5400640 1 --2- 172.21.15.32:0/2118278033 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcac00b4970 0x7fcac0157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcabe800640 1 -- 172.21.15.32:0/2118278033 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcac00b4970 msgr2=0x7fcac0157d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcabe800640 1 --2- 172.21.15.32:0/2118278033 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcac00b4970 0x7fcac0157d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcabe800640 1 -- 172.21.15.32:0/2118278033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcac009a9f0 msgr2=0x7fcac00b7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcabe800640 1 --2- 172.21.15.32:0/2118278033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcac009a9f0 0x7fcac00b7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcabe800640 1 -- 172.21.15.32:0/2118278033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac0158d60 con 0x7fcac015eb60 2024-04-17T09:58:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcabe800640 1 --2- 172.21.15.32:0/2118278033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcac015eb60 0x7fcac0158510 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fcab800c7b0 tx=0x7fcab800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcab6a00640 1 -- 172.21.15.32:0/2118278033 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab800ce60 con 0x7fcac015eb60 2024-04-17T09:58:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcab6a00640 1 -- 172.21.15.32:0/2118278033 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcab8003db0 con 0x7fcac015eb60 2024-04-17T09:58:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcab6a00640 1 -- 172.21.15.32:0/2118278033 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab80052b0 con 0x7fcac015eb60 2024-04-17T09:58:51.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcabf200640 1 -- 172.21.15.32:0/2118278033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac0159040 con 0x7fcac015eb60 2024-04-17T09:58:51.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.310+0000 7fcabf200640 1 -- 172.21.15.32:0/2118278033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac0159600 con 0x7fcac015eb60 2024-04-17T09:58:51.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.314+0000 7fcabf200640 1 -- 172.21.15.32:0/2118278033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcac004fd00 con 0x7fcac015eb60 2024-04-17T09:58:51.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.314+0000 7fcab6a00640 1 -- 172.21.15.32:0/2118278033 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcab8005450 con 0x7fcac015eb60 2024-04-17T09:58:51.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.314+0000 7fcab6a00640 1 --2- 172.21.15.32:0/2118278033 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fca9c068380 0x7fca9c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:51.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.314+0000 7fcab6a00640 1 -- 172.21.15.32:0/2118278033 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fcab8086800 con 0x7fcac015eb60 2024-04-17T09:58:51.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.314+0000 7fcabde00640 1 --2- 172.21.15.32:0/2118278033 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fca9c068380 0x7fca9c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:51.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.314+0000 7fcabde00640 1 --2- 172.21.15.32:0/2118278033 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fca9c068380 0x7fca9c06a840 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fcab00079c0 tx=0x7fcab0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:51.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.318+0000 7fcab6a00640 1 -- 172.21.15.32:0/2118278033 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fcab80566d0 con 0x7fcac015eb60 2024-04-17T09:58:51.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.518+0000 7fcabf200640 1 -- 172.21.15.32:0/2118278033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcac00b1aa0 con 0x7fcac015eb60 2024-04-17T09:58:51.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.518+0000 7fcab6a00640 1 -- 172.21.15.32:0/2118278033 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5206 (secure 0 0 0) 0x7fcab805a9d0 con 0x7fcac015eb60 2024-04-17T09:58:51.516 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:58:51.516 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2424282223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2424282223},{"type":"v1","addr":"172.21.15.144:6834","nonce":2424282223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3537056692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3537056692},{"type":"v1","addr":"172.21.15.144:6835","nonce":3537056692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5891,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/725333724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":725333724},{"type":"v1","addr":"172.21.15.32:6837","nonce":725333724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:57:11.558183+0000","modified":"2024-04-17T09:58:17.770818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":15,"addr":"172.21.15.32:6836/2416784965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2416784965},{"type":"v1","addr":"172.21.15.32:6836","nonce":2416784965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7011,"qdb_cluster":[7011]},"id":5}]} 2024-04-17T09:58:51.516 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 50 2024-04-17T09:58:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.518+0000 7fcabf200640 1 -- 172.21.15.32:0/2118278033 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fca9c068380 msgr2=0x7fca9c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.518+0000 7fcabf200640 1 --2- 172.21.15.32:0/2118278033 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fca9c068380 0x7fca9c06a840 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fcab00079c0 tx=0x7fcab0008040 comp rx=0 tx=0).stop 2024-04-17T09:58:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.518+0000 7fcabf200640 1 -- 172.21.15.32:0/2118278033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcac015eb60 msgr2=0x7fcac0158510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.518+0000 7fcabf200640 1 --2- 172.21.15.32:0/2118278033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcac015eb60 0x7fcac0158510 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fcab800c7b0 tx=0x7fcab800cc70 comp rx=0 tx=0).stop 2024-04-17T09:58:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.518+0000 7fcabf200640 1 -- 172.21.15.32:0/2118278033 shutdown_connections 2024-04-17T09:58:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.518+0000 7fcabf200640 1 --2- 172.21.15.32:0/2118278033 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fca9c068380 0x7fca9c06a840 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.518+0000 7fcabf200640 1 --2- 172.21.15.32:0/2118278033 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcac00b4970 0x7fcac0157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.518+0000 7fcabf200640 1 --2- 172.21.15.32:0/2118278033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcac015eb60 0x7fcac0158510 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.518+0000 7fcabf200640 1 --2- 172.21.15.32:0/2118278033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcac009a9f0 0x7fcac00b7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:51.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.518+0000 7fcabf200640 1 -- 172.21.15.32:0/2118278033 >> 172.21.15.32:0/2118278033 conn(0x7fcac00b3e50 msgr2=0x7fcac015c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.518+0000 7fcabf200640 1 -- 172.21.15.32:0/2118278033 shutdown_connections 2024-04-17T09:58:51.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.518+0000 7fcabf200640 1 -- 172.21.15.32:0/2118278033 wait complete. 2024-04-17T09:58:51.537 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:58:51.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.678+0000 7f0be0400640 1 Processor -- start 2024-04-17T09:58:51.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.678+0000 7f0be0400640 1 -- start start 2024-04-17T09:58:51.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.678+0000 7f0be0400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0bd8072500 0x7f0bd80cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:51.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.678+0000 7f0be0400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bd80cdac0 con 0x7f0bd815d8d0 2024-04-17T09:58:51.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.678+0000 7f0be0400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bd815cdf0 con 0x7f0bd8145f30 2024-04-17T09:58:51.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.678+0000 7f0be0400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0bd806f400 con 0x7f0bd8072500 2024-04-17T09:58:51.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.678+0000 7f0bdd000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0bd8072500 0x7f0bd80cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:51.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.678+0000 7f0bdd000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0bd8072500 0x7f0bd80cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34682/0 (socket says 172.21.15.32:34682) 2024-04-17T09:58:51.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.678+0000 7f0bdd000640 1 -- 172.21.15.32:0/1236488686 learned_addr learned my addr 172.21.15.32:0/1236488686 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:51.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.682+0000 7f0bdd000640 1 -- 172.21.15.32:0/1236488686 >> v1:172.21.15.144:6789/0 conn(0x7f0bd8145f30 legacy=0x7f0bd8071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:51.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.682+0000 7f0bdd000640 1 -- 172.21.15.32:0/1236488686 >> v1:172.21.15.32:6789/0 conn(0x7f0bd815d8d0 legacy=0x7f0bd81601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:51.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.682+0000 7f0bdd000640 1 -- 172.21.15.32:0/1236488686 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd815d0d0 con 0x7f0bd8072500 2024-04-17T09:58:51.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.682+0000 7f0bd7e00640 1 -- 172.21.15.32:0/1236488686 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3552163159 0 0) 0x7f0bd80cdac0 con 0x7f0bd815d8d0 2024-04-17T09:58:51.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.682+0000 7f0bdd000640 1 --2- 172.21.15.32:0/1236488686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bd8072500 0x7f0bd80cce70 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009870 tx=0x7f0bc401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eadc2d1f8ebfeca6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:51.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.682+0000 7f0bd7e00640 1 -- 172.21.15.32:0/1236488686 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40b5070 con 0x7f0bd8072500 2024-04-17T09:58:51.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.682+0000 7f0bd7e00640 1 -- 172.21.15.32:0/1236488686 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc40a8a50 con 0x7f0bd8072500 2024-04-17T09:58:51.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.682+0000 7f0bd7e00640 1 -- 172.21.15.32:0/1236488686 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40a8e20 con 0x7f0bd8072500 2024-04-17T09:58:51.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.682+0000 7f0be0400640 1 -- 172.21.15.32:0/1236488686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bd8072500 msgr2=0x7f0bd80cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:51.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.682+0000 7f0be0400640 1 --2- 172.21.15.32:0/1236488686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bd8072500 0x7f0bd80cce70 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009870 tx=0x7f0bc401ed50 comp rx=0 tx=0).stop 2024-04-17T09:58:51.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0be0400640 1 -- 172.21.15.32:0/1236488686 shutdown_connections 2024-04-17T09:58:51.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0be0400640 1 --2- 172.21.15.32:0/1236488686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bd8072500 0x7f0bd80cce70 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:51.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0be0400640 1 -- 172.21.15.32:0/1236488686 >> 172.21.15.32:0/1236488686 conn(0x7f0bd806ba30 msgr2=0x7f0bd806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:51.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0be0400640 1 -- 172.21.15.32:0/1236488686 shutdown_connections 2024-04-17T09:58:51.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0be0400640 1 -- 172.21.15.32:0/1236488686 wait complete. 2024-04-17T09:58:51.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0be0400640 1 Processor -- start 2024-04-17T09:58:51.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0be0400640 1 -- start start 2024-04-17T09:58:51.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0be0400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bd8072500 0x7f0bd81381f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:51.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0be0400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0bd8145f30 0x7f0bd80cb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:51.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0be0400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bd815d8d0 0x7f0bd80cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:51.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0be0400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd80cd800 con 0x7f0bd8072500 2024-04-17T09:58:51.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0be0400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd80cd680 con 0x7f0bd815d8d0 2024-04-17T09:58:51.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0be0400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0bd8146310 con 0x7f0bd8145f30 2024-04-17T09:58:51.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0bdda00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bd8072500 0x7f0bd81381f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:51.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0bdda00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bd8072500 0x7f0bd81381f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34684/0 (socket says 172.21.15.32:34684) 2024-04-17T09:58:51.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0bdda00640 1 -- 172.21.15.32:0/997146724 learned_addr learned my addr 172.21.15.32:0/997146724 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:51.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0bde400640 1 --2- 172.21.15.32:0/997146724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bd815d8d0 0x7f0bd80cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:51.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0bdd000640 1 --2- 172.21.15.32:0/997146724 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0bd8145f30 0x7f0bd80cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:51.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0bdda00640 1 -- 172.21.15.32:0/997146724 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0bd8145f30 msgr2=0x7f0bd80cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:51.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0bdda00640 1 --2- 172.21.15.32:0/997146724 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0bd8145f30 0x7f0bd80cb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:51.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0bdda00640 1 -- 172.21.15.32:0/997146724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bd815d8d0 msgr2=0x7f0bd80cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:51.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0bdda00640 1 --2- 172.21.15.32:0/997146724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bd815d8d0 0x7f0bd80cb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:51.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0bdda00640 1 -- 172.21.15.32:0/997146724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd80cbe70 con 0x7f0bd8072500 2024-04-17T09:58:51.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0bde400640 1 --2- 172.21.15.32:0/997146724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bd815d8d0 0x7f0bd80cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:58:51.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0bdd000640 1 --2- 172.21.15.32:0/997146724 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0bd8145f30 0x7f0bd80cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:58:51.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0bdda00640 1 --2- 172.21.15.32:0/997146724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bd8072500 0x7f0bd81381f0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00c500 tx=0x7f0bcc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:51.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.686+0000 7f0bd5600640 1 -- 172.21.15.32:0/997146724 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc016020 con 0x7f0bd8072500 2024-04-17T09:58:51.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.690+0000 7f0bd5600640 1 -- 172.21.15.32:0/997146724 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bcc005220 con 0x7f0bd8072500 2024-04-17T09:58:51.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.690+0000 7f0be0400640 1 -- 172.21.15.32:0/997146724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd80cc150 con 0x7f0bd8072500 2024-04-17T09:58:51.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.690+0000 7f0bd5600640 1 -- 172.21.15.32:0/997146724 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc007030 con 0x7f0bd8072500 2024-04-17T09:58:51.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.690+0000 7f0be0400640 1 -- 172.21.15.32:0/997146724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd80cc710 con 0x7f0bd8072500 2024-04-17T09:58:51.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.690+0000 7f0be0400640 1 -- 172.21.15.32:0/997146724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bd8068a90 con 0x7f0bd8072500 2024-04-17T09:58:51.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.690+0000 7f0bd5600640 1 -- 172.21.15.32:0/997146724 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0bcc0071f0 con 0x7f0bd8072500 2024-04-17T09:58:51.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.690+0000 7f0bd5600640 1 --2- 172.21.15.32:0/997146724 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0bb80683b0 0x7f0bb806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:51.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.690+0000 7f0bd5600640 1 -- 172.21.15.32:0/997146724 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f0bcc0864b0 con 0x7f0bd8072500 2024-04-17T09:58:51.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.690+0000 7f0bdd000640 1 --2- 172.21.15.32:0/997146724 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0bb80683b0 0x7f0bb806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:51.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.690+0000 7f0bdd000640 1 --2- 172.21.15.32:0/997146724 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0bb80683b0 0x7f0bb806a870 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40b2000 tx=0x7f0bc40aa070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:51.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.694+0000 7f0bd5600640 1 -- 172.21.15.32:0/997146724 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0bcc020020 con 0x7f0bd8072500 2024-04-17T09:58:51.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.858+0000 7f0be0400640 1 -- 172.21.15.32:0/997146724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0bd80b4ff0 con 0x7f0bd8072500 2024-04-17T09:58:51.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.862+0000 7f0bd5600640 1 -- 172.21.15.32:0/997146724 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16904 (secure 0 0 0) 0x7f0bcc056950 con 0x7f0bd8072500 2024-04-17T09:58:51.859 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:58:51.859 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":71,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:57:55.831704+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:57:09.948964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-17T09:57:10.512460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.a","mappings":[{"from":6,"to":0}]},{"pgid":"11.3a","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6836/663537244":"2024-04-18T09:57:30.746884+0000","172.21.15.32:6834/663537244":"2024-04-18T09:57:30.746884+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:58:51.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.862+0000 7f0be0400640 1 -- 172.21.15.32:0/997146724 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0bb80683b0 msgr2=0x7f0bb806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:51.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.862+0000 7f0be0400640 1 --2- 172.21.15.32:0/997146724 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0bb80683b0 0x7f0bb806a870 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40b2000 tx=0x7f0bc40aa070 comp rx=0 tx=0).stop 2024-04-17T09:58:51.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.862+0000 7f0be0400640 1 -- 172.21.15.32:0/997146724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bd8072500 msgr2=0x7f0bd81381f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:51.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.862+0000 7f0be0400640 1 --2- 172.21.15.32:0/997146724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bd8072500 0x7f0bd81381f0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00c500 tx=0x7f0bcc00c9c0 comp rx=0 tx=0).stop 2024-04-17T09:58:51.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.862+0000 7f0be0400640 1 -- 172.21.15.32:0/997146724 shutdown_connections 2024-04-17T09:58:51.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.862+0000 7f0be0400640 1 --2- 172.21.15.32:0/997146724 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0bb80683b0 0x7f0bb806a870 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:51.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.862+0000 7f0be0400640 1 --2- 172.21.15.32:0/997146724 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0bd8145f30 0x7f0bd80cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:51.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.862+0000 7f0be0400640 1 --2- 172.21.15.32:0/997146724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bd815d8d0 0x7f0bd80cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:51.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.862+0000 7f0be0400640 1 --2- 172.21.15.32:0/997146724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bd8072500 0x7f0bd81381f0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:51.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.862+0000 7f0be0400640 1 -- 172.21.15.32:0/997146724 >> 172.21.15.32:0/997146724 conn(0x7f0bd806ba30 msgr2=0x7f0bd806be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:51.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.862+0000 7f0be0400640 1 -- 172.21.15.32:0/997146724 shutdown_connections 2024-04-17T09:58:51.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:51.862+0000 7f0be0400640 1 -- 172.21.15.32:0/997146724 wait complete. 2024-04-17T09:58:51.881 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T09:58:51.881 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:58:52.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.022+0000 7f4219000640 1 Processor -- start 2024-04-17T09:58:52.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.022+0000 7f4219000640 1 -- start start 2024-04-17T09:58:52.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4219000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f421406f670 0x7f421415bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:52.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4219000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4214072bb0 con 0x7f421415baa0 2024-04-17T09:58:52.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4219000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42140b3dc0 con 0x7f4214144e90 2024-04-17T09:58:52.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4219000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4214145270 con 0x7f421406f670 2024-04-17T09:58:52.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4209400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f421415baa0 0x7f421406ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49074/0 (socket says 172.21.15.32:49074) 2024-04-17T09:58:52.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4209400640 1 -- 172.21.15.32:0/916490287 learned_addr learned my addr 172.21.15.32:0/916490287 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:58:52.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4212800640 1 --2- 172.21.15.32:0/916490287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f421406f670 0x7f421415bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:52.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4211400640 1 -- 172.21.15.32:0/916490287 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3250007892 0 0) 0x7f4214072bb0 con 0x7f421415baa0 2024-04-17T09:58:52.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4211400640 1 -- 172.21.15.32:0/916490287 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41f8003530 con 0x7f421415baa0 2024-04-17T09:58:52.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4211400640 1 -- 172.21.15.32:0/916490287 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4131766898 0 0) 0x7f42140b3dc0 con 0x7f4214144e90 2024-04-17T09:58:52.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4211400640 1 -- 172.21.15.32:0/916490287 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4214072bb0 con 0x7f4214144e90 2024-04-17T09:58:52.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4211400640 1 -- 172.21.15.32:0/916490287 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3435325797 0 0) 0x7f41f8003530 con 0x7f421415baa0 2024-04-17T09:58:52.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4211400640 1 -- 172.21.15.32:0/916490287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f421406f670 msgr2=0x7f421415bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:52.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4211400640 1 --2- 172.21.15.32:0/916490287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f421406f670 0x7f421415bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4211400640 1 -- 172.21.15.32:0/916490287 >> v1:172.21.15.144:6789/0 conn(0x7f4214144e90 legacy=0x7f42140b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:52.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4211400640 1 -- 172.21.15.32:0/916490287 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42140b40a0 con 0x7f421415baa0 2024-04-17T09:58:52.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4211400640 1 -- 172.21.15.32:0/916490287 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f41fc001680 con 0x7f421415baa0 2024-04-17T09:58:52.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4211400640 1 -- 172.21.15.32:0/916490287 >> v1:172.21.15.32:6789/0 conn(0x7f421415baa0 legacy=0x7f421406ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:52.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4211400640 1 --2- 172.21.15.32:0/916490287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f8003b80 0x7f41f8003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:52.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4211400640 1 --2- 172.21.15.32:0/916490287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41f8004810 0x7f41f8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:52.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4211400640 1 --2- 172.21.15.32:0/916490287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f421406f670 0x7f41f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:52.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4211400640 1 -- 172.21.15.32:0/916490287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4214145270 con 0x7f41f8003b80 2024-04-17T09:58:52.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4211400640 1 -- 172.21.15.32:0/916490287 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f41f800dc90 con 0x7f41f8004810 2024-04-17T09:58:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4211400640 1 -- 172.21.15.32:0/916490287 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f41f800de90 con 0x7f421406f670 2024-04-17T09:58:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4211e00640 1 --2- 172.21.15.32:0/916490287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f8003b80 0x7f41f8003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4209400640 1 --2- 172.21.15.32:0/916490287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41f8004810 0x7f41f8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4212800640 1 --2- 172.21.15.32:0/916490287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f421406f670 0x7f41f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4211e00640 1 -- 172.21.15.32:0/916490287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f421406f670 msgr2=0x7f41f80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4211e00640 1 --2- 172.21.15.32:0/916490287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f421406f670 0x7f41f80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4211e00640 1 -- 172.21.15.32:0/916490287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41f8004810 msgr2=0x7f41f8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4211e00640 1 --2- 172.21.15.32:0/916490287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41f8004810 0x7f41f8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.026+0000 7f4211e00640 1 -- 172.21.15.32:0/916490287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f8003530 con 0x7f41f8003b80 2024-04-17T09:58:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.030+0000 7f4211e00640 1 --2- 172.21.15.32:0/916490287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f8003b80 0x7f41f8003f80 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f4204007fd0 tx=0x7f420401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241c075ac7de17ff server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.030+0000 7f4211400640 1 -- 172.21.15.32:0/916490287 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4204001c20 con 0x7f41f8003b80 2024-04-17T09:58:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.030+0000 7f4211400640 1 -- 172.21.15.32:0/916490287 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4204003ba0 con 0x7f41f8003b80 2024-04-17T09:58:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.030+0000 7f4211400640 1 -- 172.21.15.32:0/916490287 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42040050d0 con 0x7f41f8003b80 2024-04-17T09:58:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.030+0000 7f4219000640 1 -- 172.21.15.32:0/916490287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f8003b80 msgr2=0x7f41f8003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.030+0000 7f4219000640 1 --2- 172.21.15.32:0/916490287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f8003b80 0x7f41f8003f80 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f4204007fd0 tx=0x7f420401c890 comp rx=0 tx=0).stop 2024-04-17T09:58:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.030+0000 7f4212800640 1 -- 172.21.15.32:0/916490287 reap_dead start 2024-04-17T09:58:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.030+0000 7f4219000640 1 -- 172.21.15.32:0/916490287 shutdown_connections 2024-04-17T09:58:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.030+0000 7f4219000640 1 -- 172.21.15.32:0/916490287 >> 172.21.15.32:0/916490287 conn(0x7f421406ba30 msgr2=0x7f421406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.030+0000 7f4219000640 1 -- 172.21.15.32:0/916490287 shutdown_connections 2024-04-17T09:58:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.030+0000 7f4219000640 1 -- 172.21.15.32:0/916490287 wait complete. 2024-04-17T09:58:52.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f4219000640 1 Processor -- start 2024-04-17T09:58:52.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f4219000640 1 -- start start 2024-04-17T09:58:52.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f4219000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f8003b80 0x7f421415aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f4219000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f421413ada0 0x7f421413b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f4219000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f421415d430 0x7f421415d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f4219000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f42140727f0 con 0x7f41f8003b80 2024-04-17T09:58:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f4219000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4214072670 con 0x7f421413ada0 2024-04-17T09:58:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f4219000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4214072970 con 0x7f421415d430 2024-04-17T09:58:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f4211e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f8003b80 0x7f421415aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f4211e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f8003b80 0x7f421415aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34700/0 (socket says 172.21.15.32:34700) 2024-04-17T09:58:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f4211e00640 1 -- 172.21.15.32:0/40327519 learned_addr learned my addr 172.21.15.32:0/40327519 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f4211e00640 1 -- 172.21.15.32:0/40327519 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f421415d430 msgr2=0x7f421415d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f4211e00640 1 --2- 172.21.15.32:0/40327519 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f421415d430 0x7f421415d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f4209400640 1 --2- 172.21.15.32:0/40327519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f421413ada0 0x7f421413b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f4211e00640 1 -- 172.21.15.32:0/40327519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f421413ada0 msgr2=0x7f421413b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f4211e00640 1 --2- 172.21.15.32:0/40327519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f421413ada0 0x7f421413b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f4211e00640 1 -- 172.21.15.32:0/40327519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421415b3f0 con 0x7f41f8003b80 2024-04-17T09:58:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f4211e00640 1 --2- 172.21.15.32:0/40327519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f8003b80 0x7f421415aeb0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f420401c6c0 tx=0x7f41f8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:52.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f420aa00640 1 -- 172.21.15.32:0/40327519 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4204001c00 con 0x7f41f8003b80 2024-04-17T09:58:52.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f420aa00640 1 -- 172.21.15.32:0/40327519 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42040b6070 con 0x7f41f8003b80 2024-04-17T09:58:52.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f420aa00640 1 -- 172.21.15.32:0/40327519 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42040056f0 con 0x7f41f8003b80 2024-04-17T09:58:52.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f4219000640 1 -- 172.21.15.32:0/40327519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f421413ba20 con 0x7f41f8003b80 2024-04-17T09:58:52.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f4219000640 1 -- 172.21.15.32:0/40327519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f421413bfe0 con 0x7f41f8003b80 2024-04-17T09:58:52.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.034+0000 7f4219000640 1 -- 172.21.15.32:0/40327519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4214068a90 con 0x7f41f8003b80 2024-04-17T09:58:52.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.038+0000 7f420aa00640 1 -- 172.21.15.32:0/40327519 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f42040b8020 con 0x7f41f8003b80 2024-04-17T09:58:52.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.038+0000 7f420aa00640 1 --2- 172.21.15.32:0/40327519 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f41f0068380 0x7f41f006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:52.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.038+0000 7f420aa00640 1 -- 172.21.15.32:0/40327519 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f4204120c30 con 0x7f41f8003b80 2024-04-17T09:58:52.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.038+0000 7f4209400640 1 --2- 172.21.15.32:0/40327519 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f41f0068380 0x7f41f006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:52.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.038+0000 7f4209400640 1 --2- 172.21.15.32:0/40327519 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f41f0068380 0x7f41f006a840 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f421415eff0 tx=0x7f41fc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:52.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.042+0000 7f420aa00640 1 -- 172.21.15.32:0/40327519 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f42040165d0 con 0x7f41f8003b80 2024-04-17T09:58:52.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.242+0000 7f4219000640 1 -- 172.21.15.32:0/40327519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f421413ce10 con 0x7f41f8003b80 2024-04-17T09:58:52.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.246+0000 7f420aa00640 1 -- 172.21.15.32:0/40327519 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5206 (secure 0 0 0) 0x7f42040f10c0 con 0x7f41f8003b80 2024-04-17T09:58:52.243 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:58:52.243 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2424282223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2424282223},{"type":"v1","addr":"172.21.15.144:6834","nonce":2424282223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3537056692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3537056692},{"type":"v1","addr":"172.21.15.144:6835","nonce":3537056692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5891,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/725333724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":725333724},{"type":"v1","addr":"172.21.15.32:6837","nonce":725333724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:57:11.558183+0000","modified":"2024-04-17T09:58:17.770818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":15,"addr":"172.21.15.32:6836/2416784965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2416784965},{"type":"v1","addr":"172.21.15.32:6836","nonce":2416784965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7011,"qdb_cluster":[7011]},"id":5}]} 2024-04-17T09:58:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 50 2024-04-17T09:58:52.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.246+0000 7f4219000640 1 -- 172.21.15.32:0/40327519 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f41f0068380 msgr2=0x7f41f006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:52.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.246+0000 7f4219000640 1 --2- 172.21.15.32:0/40327519 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f41f0068380 0x7f41f006a840 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f421415eff0 tx=0x7f41fc009040 comp rx=0 tx=0).stop 2024-04-17T09:58:52.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.246+0000 7f4219000640 1 -- 172.21.15.32:0/40327519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f8003b80 msgr2=0x7f421415aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:52.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.246+0000 7f4219000640 1 --2- 172.21.15.32:0/40327519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f8003b80 0x7f421415aeb0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f420401c6c0 tx=0x7f41f8005840 comp rx=0 tx=0).stop 2024-04-17T09:58:52.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.246+0000 7f4219000640 1 -- 172.21.15.32:0/40327519 shutdown_connections 2024-04-17T09:58:52.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.246+0000 7f4219000640 1 --2- 172.21.15.32:0/40327519 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f41f0068380 0x7f41f006a840 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.246+0000 7f4219000640 1 --2- 172.21.15.32:0/40327519 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f421415d430 0x7f421415d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.246+0000 7f4219000640 1 --2- 172.21.15.32:0/40327519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f421413ada0 0x7f421413b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.246+0000 7f4219000640 1 --2- 172.21.15.32:0/40327519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f8003b80 0x7f421415aeb0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.246+0000 7f4219000640 1 -- 172.21.15.32:0/40327519 >> 172.21.15.32:0/40327519 conn(0x7f421406ba30 msgr2=0x7f4214140210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:52.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.246+0000 7f4219000640 1 -- 172.21.15.32:0/40327519 shutdown_connections 2024-04-17T09:58:52.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.246+0000 7f4219000640 1 -- 172.21.15.32:0/40327519 wait complete. 2024-04-17T09:58:52.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:58:52.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.406+0000 7fdb2c800640 1 Processor -- start 2024-04-17T09:58:52.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.406+0000 7fdb2c800640 1 -- start start 2024-04-17T09:58:52.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.406+0000 7fdb2c800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb24144e90 0x7fdb240b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:52.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.406+0000 7fdb2c800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb24072bb0 con 0x7fdb240b4de0 2024-04-17T09:58:52.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.406+0000 7fdb2c800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb240b3dc0 con 0x7fdb2415c1b0 2024-04-17T09:58:52.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.406+0000 7fdb2c800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdb24145270 con 0x7fdb24144e90 2024-04-17T09:58:52.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb29400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdb240b4de0 0x7fdb2415baa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49080/0 (socket says 172.21.15.32:49080) 2024-04-17T09:58:52.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb29400640 1 -- 172.21.15.32:0/1239331115 learned_addr learned my addr 172.21.15.32:0/1239331115 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:58:52.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb29e00640 1 --2- 172.21.15.32:0/1239331115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb24144e90 0x7fdb240b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:52.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb28a00640 1 -- 172.21.15.32:0/1239331115 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2526127473 0 0) 0x7fdb24072bb0 con 0x7fdb240b4de0 2024-04-17T09:58:52.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb28a00640 1 -- 172.21.15.32:0/1239331115 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb0c003530 con 0x7fdb240b4de0 2024-04-17T09:58:52.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb28a00640 1 -- 172.21.15.32:0/1239331115 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 756767867 0 0) 0x7fdb0c003530 con 0x7fdb240b4de0 2024-04-17T09:58:52.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb28a00640 1 -- 172.21.15.32:0/1239331115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb24144e90 msgr2=0x7fdb240b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:52.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb28a00640 1 --2- 172.21.15.32:0/1239331115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb24144e90 0x7fdb240b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb28a00640 1 -- 172.21.15.32:0/1239331115 >> v1:172.21.15.144:6789/0 conn(0x7fdb2415c1b0 legacy=0x7fdb2406ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:52.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb28a00640 1 -- 172.21.15.32:0/1239331115 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb240b40a0 con 0x7fdb240b4de0 2024-04-17T09:58:52.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb28a00640 1 -- 172.21.15.32:0/1239331115 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7fdb140016a0 con 0x7fdb240b4de0 2024-04-17T09:58:52.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb28a00640 1 -- 172.21.15.32:0/1239331115 >> v1:172.21.15.32:6789/0 conn(0x7fdb240b4de0 legacy=0x7fdb2415baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:52.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb28a00640 1 --2- 172.21.15.32:0/1239331115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0c0039f0 0x7fdb0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:52.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb28a00640 1 --2- 172.21.15.32:0/1239331115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb24144e90 0x7fdb0c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:52.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb28a00640 1 --2- 172.21.15.32:0/1239331115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb0c004d50 0x7fdb0c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb28a00640 1 -- 172.21.15.32:0/1239331115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdb24145270 con 0x7fdb0c004d50 2024-04-17T09:58:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb28a00640 1 -- 172.21.15.32:0/1239331115 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c00b9d0 con 0x7fdb0c0039f0 2024-04-17T09:58:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb28a00640 1 -- 172.21.15.32:0/1239331115 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdb0c00dde0 con 0x7fdb24144e90 2024-04-17T09:58:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb2a800640 1 --2- 172.21.15.32:0/1239331115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb0c004d50 0x7fdb0c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb29400640 1 --2- 172.21.15.32:0/1239331115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb24144e90 0x7fdb0c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb29e00640 1 --2- 172.21.15.32:0/1239331115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0c0039f0 0x7fdb0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb2a800640 1 -- 172.21.15.32:0/1239331115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb24144e90 msgr2=0x7fdb0c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb2a800640 1 --2- 172.21.15.32:0/1239331115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb24144e90 0x7fdb0c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb2a800640 1 -- 172.21.15.32:0/1239331115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0c0039f0 msgr2=0x7fdb0c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb2a800640 1 --2- 172.21.15.32:0/1239331115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0c0039f0 0x7fdb0c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.410+0000 7fdb2a800640 1 -- 172.21.15.32:0/1239331115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0c003530 con 0x7fdb0c004d50 2024-04-17T09:58:52.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.414+0000 7fdb2a800640 1 --2- 172.21.15.32:0/1239331115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb0c004d50 0x7fdb0c0051d0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fdb20001f60 tx=0x7fdb2001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e033c6174d711c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:52.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.414+0000 7fdb28a00640 1 -- 172.21.15.32:0/1239331115 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb20001b60 con 0x7fdb0c004d50 2024-04-17T09:58:52.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.414+0000 7fdb28a00640 1 -- 172.21.15.32:0/1239331115 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb200a6700 con 0x7fdb0c004d50 2024-04-17T09:58:52.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.414+0000 7fdb28a00640 1 -- 172.21.15.32:0/1239331115 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb200a6af0 con 0x7fdb0c004d50 2024-04-17T09:58:52.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.414+0000 7fdb2c800640 1 -- 172.21.15.32:0/1239331115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb0c004d50 msgr2=0x7fdb0c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:52.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.414+0000 7fdb2c800640 1 --2- 172.21.15.32:0/1239331115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb0c004d50 0x7fdb0c0051d0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fdb20001f60 tx=0x7fdb2001c880 comp rx=0 tx=0).stop 2024-04-17T09:58:52.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.414+0000 7fdb2a800640 1 -- 172.21.15.32:0/1239331115 reap_dead start 2024-04-17T09:58:52.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.414+0000 7fdb2c800640 1 -- 172.21.15.32:0/1239331115 shutdown_connections 2024-04-17T09:58:52.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.414+0000 7fdb2c800640 1 -- 172.21.15.32:0/1239331115 >> 172.21.15.32:0/1239331115 conn(0x7fdb2406ba30 msgr2=0x7fdb2406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:52.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.414+0000 7fdb2c800640 1 -- 172.21.15.32:0/1239331115 shutdown_connections 2024-04-17T09:58:52.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.414+0000 7fdb2c800640 1 -- 172.21.15.32:0/1239331115 wait complete. 2024-04-17T09:58:52.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.414+0000 7fdb2c800640 1 Processor -- start 2024-04-17T09:58:52.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.414+0000 7fdb2c800640 1 -- start start 2024-04-17T09:58:52.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb2c800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0c004d50 0x7fdb2413e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:52.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb2c800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb2413eaa0 0x7fdb241efa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:52.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb2c800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb241effa0 0x7fdb2413a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:52.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb2c800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2406f7f0 con 0x7fdb241effa0 2024-04-17T09:58:52.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb2c800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2406f670 con 0x7fdb0c004d50 2024-04-17T09:58:52.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb2c800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdb240b48a0 con 0x7fdb2413eaa0 2024-04-17T09:58:52.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb2a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb241effa0 0x7fdb2413a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:52.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb2a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb241effa0 0x7fdb2413a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34712/0 (socket says 172.21.15.32:34712) 2024-04-17T09:58:52.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb2a800640 1 -- 172.21.15.32:0/4079900162 learned_addr learned my addr 172.21.15.32:0/4079900162 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:52.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb29400640 1 --2- 172.21.15.32:0/4079900162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb2413eaa0 0x7fdb241efa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:52.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb29e00640 1 --2- 172.21.15.32:0/4079900162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0c004d50 0x7fdb2413e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:52.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb2a800640 1 -- 172.21.15.32:0/4079900162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb2413eaa0 msgr2=0x7fdb241efa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:52.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb2a800640 1 --2- 172.21.15.32:0/4079900162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb2413eaa0 0x7fdb241efa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb2a800640 1 -- 172.21.15.32:0/4079900162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0c004d50 msgr2=0x7fdb2413e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:52.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb2a800640 1 --2- 172.21.15.32:0/4079900162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0c004d50 0x7fdb2413e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb2a800640 1 -- 172.21.15.32:0/4079900162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb241f0530 con 0x7fdb241effa0 2024-04-17T09:58:52.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb2a800640 1 --2- 172.21.15.32:0/4079900162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb241effa0 0x7fdb2413a6b0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fdb200b0040 tx=0x7fdb0c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:52.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb1e000640 1 -- 172.21.15.32:0/4079900162 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb200011d0 con 0x7fdb241effa0 2024-04-17T09:58:52.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb1e000640 1 -- 172.21.15.32:0/4079900162 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb20001b40 con 0x7fdb241effa0 2024-04-17T09:58:52.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb1e000640 1 -- 172.21.15.32:0/4079900162 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb20003b90 con 0x7fdb241effa0 2024-04-17T09:58:52.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb2c800640 1 -- 172.21.15.32:0/4079900162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb241f0810 con 0x7fdb241effa0 2024-04-17T09:58:52.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb2c800640 1 -- 172.21.15.32:0/4079900162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb241f0dd0 con 0x7fdb241effa0 2024-04-17T09:58:52.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb2c800640 1 -- 172.21.15.32:0/4079900162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb24068a90 con 0x7fdb241effa0 2024-04-17T09:58:52.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.418+0000 7fdb1e000640 1 -- 172.21.15.32:0/4079900162 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdb200be020 con 0x7fdb241effa0 2024-04-17T09:58:52.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.422+0000 7fdb1e000640 1 --2- 172.21.15.32:0/4079900162 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdb080683b0 0x7fdb0806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:52.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.422+0000 7fdb1e000640 1 -- 172.21.15.32:0/4079900162 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fdb20120670 con 0x7fdb241effa0 2024-04-17T09:58:52.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.422+0000 7fdb29e00640 1 --2- 172.21.15.32:0/4079900162 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdb080683b0 0x7fdb0806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:52.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.422+0000 7fdb29e00640 1 --2- 172.21.15.32:0/4079900162 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdb080683b0 0x7fdb0806a870 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009be0 tx=0x7fdb18009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:52.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.426+0000 7fdb1e000640 1 -- 172.21.15.32:0/4079900162 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fdb200c5020 con 0x7fdb241effa0 2024-04-17T09:58:52.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.626+0000 7fdb2c800640 1 -- 172.21.15.32:0/4079900162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb2414f890 con 0x7fdb241effa0 2024-04-17T09:58:52.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.626+0000 7fdb1e000640 1 -- 172.21.15.32:0/4079900162 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5206 (secure 0 0 0) 0x7fdb20016590 con 0x7fdb241effa0 2024-04-17T09:58:52.623 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:58:52.623 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2424282223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2424282223},{"type":"v1","addr":"172.21.15.144:6834","nonce":2424282223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3537056692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3537056692},{"type":"v1","addr":"172.21.15.144:6835","nonce":3537056692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5891,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/725333724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":725333724},{"type":"v1","addr":"172.21.15.32:6837","nonce":725333724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:57:11.558183+0000","modified":"2024-04-17T09:58:17.770818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":15,"addr":"172.21.15.32:6836/2416784965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2416784965},{"type":"v1","addr":"172.21.15.32:6836","nonce":2416784965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7011,"qdb_cluster":[7011]},"id":5}]} 2024-04-17T09:58:52.623 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 50 2024-04-17T09:58:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.626+0000 7fdb2c800640 1 -- 172.21.15.32:0/4079900162 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdb080683b0 msgr2=0x7fdb0806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.626+0000 7fdb2c800640 1 --2- 172.21.15.32:0/4079900162 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdb080683b0 0x7fdb0806a870 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009be0 tx=0x7fdb18009190 comp rx=0 tx=0).stop 2024-04-17T09:58:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.626+0000 7fdb2c800640 1 -- 172.21.15.32:0/4079900162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb241effa0 msgr2=0x7fdb2413a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.626+0000 7fdb2c800640 1 --2- 172.21.15.32:0/4079900162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb241effa0 0x7fdb2413a6b0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fdb200b0040 tx=0x7fdb0c005840 comp rx=0 tx=0).stop 2024-04-17T09:58:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.626+0000 7fdb2c800640 1 -- 172.21.15.32:0/4079900162 shutdown_connections 2024-04-17T09:58:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.626+0000 7fdb2c800640 1 --2- 172.21.15.32:0/4079900162 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdb080683b0 0x7fdb0806a870 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.626+0000 7fdb2c800640 1 --2- 172.21.15.32:0/4079900162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb2413eaa0 0x7fdb241efa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.626+0000 7fdb2c800640 1 --2- 172.21.15.32:0/4079900162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb241effa0 0x7fdb2413a6b0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.626+0000 7fdb2c800640 1 --2- 172.21.15.32:0/4079900162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb0c004d50 0x7fdb2413e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.626+0000 7fdb2c800640 1 -- 172.21.15.32:0/4079900162 >> 172.21.15.32:0/4079900162 conn(0x7fdb2406ba30 msgr2=0x7fdb240af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:52.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.626+0000 7fdb2c800640 1 -- 172.21.15.32:0/4079900162 shutdown_connections 2024-04-17T09:58:52.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.626+0000 7fdb2c800640 1 -- 172.21.15.32:0/4079900162 wait complete. 2024-04-17T09:58:52.645 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:58:52.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.786+0000 7fc291e00640 1 Processor -- start 2024-04-17T09:58:52.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.786+0000 7fc291e00640 1 -- start start 2024-04-17T09:58:52.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.790+0000 7fc291e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc28c1481e0 0x7fc28c071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:52.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.790+0000 7fc291e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc28c072330 0x7fc28c0b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:52.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.790+0000 7fc291e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc28c06ef70 con 0x7fc28c144f20 2024-04-17T09:58:52.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.790+0000 7fc291e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc28c0727a0 con 0x7fc28c1481e0 2024-04-17T09:58:52.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.790+0000 7fc291e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc28c0aed20 con 0x7fc28c072330 2024-04-17T09:58:52.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.790+0000 7fc282000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc28c1481e0 0x7fc28c071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:52.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.790+0000 7fc282000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc28c1481e0 0x7fc28c071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34724/0 (socket says 172.21.15.32:34724) 2024-04-17T09:58:52.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.790+0000 7fc282000640 1 -- 172.21.15.32:0/1745578294 learned_addr learned my addr 172.21.15.32:0/1745578294 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:52.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.790+0000 7fc28b400640 1 --2- 172.21.15.32:0/1745578294 >> v2:172.21.15.144:3300/0 conn(0x7fc28c072330 0x7fc28c0b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:52.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.790+0000 7fc282000640 1 -- 172.21.15.32:0/1745578294 >> v2:172.21.15.144:3300/0 conn(0x7fc28c072330 msgr2=0x7fc28c0b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:52.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.790+0000 7fc282000640 1 --2- 172.21.15.32:0/1745578294 >> v2:172.21.15.144:3300/0 conn(0x7fc28c072330 0x7fc28c0b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.790+0000 7fc282000640 1 -- 172.21.15.32:0/1745578294 >> v1:172.21.15.144:6789/0 conn(0x7fc28c144f20 legacy=0x7fc28c06e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:52.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.790+0000 7fc282000640 1 -- 172.21.15.32:0/1745578294 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28c0af000 con 0x7fc28c1481e0 2024-04-17T09:58:52.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.790+0000 7fc282000640 1 --2- 172.21.15.32:0/1745578294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc28c1481e0 0x7fc28c071df0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fc28c1486a0 tx=0x7fc27401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692dcee4d81e6fae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:52.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.790+0000 7fc28a000640 1 -- 172.21.15.32:0/1745578294 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2740b1070 con 0x7fc28c1481e0 2024-04-17T09:58:52.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.790+0000 7fc28a000640 1 -- 172.21.15.32:0/1745578294 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2740ac630 con 0x7fc28c1481e0 2024-04-17T09:58:52.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.790+0000 7fc28a000640 1 -- 172.21.15.32:0/1745578294 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2740a8050 con 0x7fc28c1481e0 2024-04-17T09:58:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.794+0000 7fc291e00640 1 -- 172.21.15.32:0/1745578294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc28c1481e0 msgr2=0x7fc28c071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.794+0000 7fc291e00640 1 --2- 172.21.15.32:0/1745578294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc28c1481e0 0x7fc28c071df0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fc28c1486a0 tx=0x7fc27401c4e0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.794+0000 7fc291e00640 1 -- 172.21.15.32:0/1745578294 shutdown_connections 2024-04-17T09:58:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.794+0000 7fc291e00640 1 --2- 172.21.15.32:0/1745578294 >> v2:172.21.15.144:3300/0 conn(0x7fc28c072330 0x7fc28c0b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.794+0000 7fc291e00640 1 --2- 172.21.15.32:0/1745578294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc28c1481e0 0x7fc28c071df0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.794+0000 7fc291e00640 1 -- 172.21.15.32:0/1745578294 >> 172.21.15.32:0/1745578294 conn(0x7fc28c06ba30 msgr2=0x7fc28c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.794+0000 7fc291e00640 1 -- 172.21.15.32:0/1745578294 shutdown_connections 2024-04-17T09:58:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.794+0000 7fc291e00640 1 -- 172.21.15.32:0/1745578294 wait complete. 2024-04-17T09:58:52.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.794+0000 7fc291e00640 1 Processor -- start 2024-04-17T09:58:52.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.794+0000 7fc291e00640 1 -- start start 2024-04-17T09:58:52.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.794+0000 7fc291e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc28c072330 0x7fc28c0b2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.794+0000 7fc291e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc28c144f20 0x7fc28c140170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.794+0000 7fc291e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc28c1481e0 0x7fc28c13a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.794+0000 7fc291e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c0afdc0 con 0x7fc28c1481e0 2024-04-17T09:58:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.794+0000 7fc291e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c0b41e0 con 0x7fc28c072330 2024-04-17T09:58:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.794+0000 7fc291e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc28c156bb0 con 0x7fc28c144f20 2024-04-17T09:58:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.794+0000 7fc28b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc28c1481e0 0x7fc28c13a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.794+0000 7fc28b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc28c1481e0 0x7fc28c13a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34740/0 (socket says 172.21.15.32:34740) 2024-04-17T09:58:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.794+0000 7fc28b400640 1 -- 172.21.15.32:0/4128077756 learned_addr learned my addr 172.21.15.32:0/4128077756 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.794+0000 7fc282000640 1 --2- 172.21.15.32:0/4128077756 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc28c144f20 0x7fc28c140170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.794+0000 7fc28aa00640 1 --2- 172.21.15.32:0/4128077756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc28c072330 0x7fc28c0b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.794+0000 7fc28b400640 1 -- 172.21.15.32:0/4128077756 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc28c144f20 msgr2=0x7fc28c140170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.798+0000 7fc28b400640 1 --2- 172.21.15.32:0/4128077756 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc28c144f20 0x7fc28c140170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.798+0000 7fc28b400640 1 -- 172.21.15.32:0/4128077756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc28c072330 msgr2=0x7fc28c0b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:52.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.798+0000 7fc28b400640 1 --2- 172.21.15.32:0/4128077756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc28c072330 0x7fc28c0b2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.798+0000 7fc28b400640 1 -- 172.21.15.32:0/4128077756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28c0b5740 con 0x7fc28c1481e0 2024-04-17T09:58:52.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.798+0000 7fc28aa00640 1 --2- 172.21.15.32:0/4128077756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc28c072330 0x7fc28c0b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:58:52.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.798+0000 7fc28b400640 1 --2- 172.21.15.32:0/4128077756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc28c1481e0 0x7fc28c13a550 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fc28400d800 tx=0x7fc28400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:52.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.798+0000 7fc283e00640 1 -- 172.21.15.32:0/4128077756 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28400b6c0 con 0x7fc28c1481e0 2024-04-17T09:58:52.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.798+0000 7fc283e00640 1 -- 172.21.15.32:0/4128077756 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc284019070 con 0x7fc28c1481e0 2024-04-17T09:58:52.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.798+0000 7fc283e00640 1 -- 172.21.15.32:0/4128077756 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28400baa0 con 0x7fc28c1481e0 2024-04-17T09:58:52.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.798+0000 7fc291e00640 1 -- 172.21.15.32:0/4128077756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc28c0b2ac0 con 0x7fc28c1481e0 2024-04-17T09:58:52.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.798+0000 7fc291e00640 1 -- 172.21.15.32:0/4128077756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc28c1570e0 con 0x7fc28c1481e0 2024-04-17T09:58:52.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.798+0000 7fc291e00640 1 -- 172.21.15.32:0/4128077756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc28c068a90 con 0x7fc28c1481e0 2024-04-17T09:58:52.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.798+0000 7fc283e00640 1 -- 172.21.15.32:0/4128077756 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc28400bc40 con 0x7fc28c1481e0 2024-04-17T09:58:52.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.798+0000 7fc283e00640 1 --2- 172.21.15.32:0/4128077756 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc268068380 0x7fc26806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:52.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.798+0000 7fc283e00640 1 -- 172.21.15.32:0/4128077756 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fc284086e30 con 0x7fc28c1481e0 2024-04-17T09:58:52.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.798+0000 7fc28aa00640 1 --2- 172.21.15.32:0/4128077756 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc268068380 0x7fc26806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:52.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.802+0000 7fc28aa00640 1 --2- 172.21.15.32:0/4128077756 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc268068380 0x7fc26806a840 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc28c0b5150 tx=0x7fc27c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:52.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.802+0000 7fc283e00640 1 -- 172.21.15.32:0/4128077756 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc284057350 con 0x7fc28c1481e0 2024-04-17T09:58:52.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.970+0000 7fc291e00640 1 -- 172.21.15.32:0/4128077756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc28c0b5180 con 0x7fc28c1481e0 2024-04-17T09:58:52.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.970+0000 7fc283e00640 1 -- 172.21.15.32:0/4128077756 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16904 (secure 0 0 0) 0x7fc284056c50 con 0x7fc28c1481e0 2024-04-17T09:58:52.970 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:58:52.970 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":71,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:57:55.831704+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:57:09.948964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-17T09:57:10.512460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.a","mappings":[{"from":6,"to":0}]},{"pgid":"11.3a","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6836/663537244":"2024-04-18T09:57:30.746884+0000","172.21.15.32:6834/663537244":"2024-04-18T09:57:30.746884+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:58:52.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.974+0000 7fc291e00640 1 -- 172.21.15.32:0/4128077756 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc268068380 msgr2=0x7fc26806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:52.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.974+0000 7fc291e00640 1 --2- 172.21.15.32:0/4128077756 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc268068380 0x7fc26806a840 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc28c0b5150 tx=0x7fc27c009040 comp rx=0 tx=0).stop 2024-04-17T09:58:52.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.974+0000 7fc291e00640 1 -- 172.21.15.32:0/4128077756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc28c1481e0 msgr2=0x7fc28c13a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:52.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.974+0000 7fc291e00640 1 --2- 172.21.15.32:0/4128077756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc28c1481e0 0x7fc28c13a550 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fc28400d800 tx=0x7fc28400dcc0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.974+0000 7fc291e00640 1 -- 172.21.15.32:0/4128077756 shutdown_connections 2024-04-17T09:58:52.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.974+0000 7fc291e00640 1 --2- 172.21.15.32:0/4128077756 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc268068380 0x7fc26806a840 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.974+0000 7fc291e00640 1 --2- 172.21.15.32:0/4128077756 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc28c144f20 0x7fc28c140170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.974+0000 7fc291e00640 1 --2- 172.21.15.32:0/4128077756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc28c1481e0 0x7fc28c13a550 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.974+0000 7fc291e00640 1 --2- 172.21.15.32:0/4128077756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc28c072330 0x7fc28c0b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:52.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.974+0000 7fc291e00640 1 -- 172.21.15.32:0/4128077756 >> 172.21.15.32:0/4128077756 conn(0x7fc28c06ba30 msgr2=0x7fc28c15c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:52.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.974+0000 7fc291e00640 1 -- 172.21.15.32:0/4128077756 shutdown_connections 2024-04-17T09:58:52.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:52.974+0000 7fc291e00640 1 -- 172.21.15.32:0/4128077756 wait complete. 2024-04-17T09:58:52.992 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T09:58:53.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.134+0000 7fb7f5200640 1 Processor -- start 2024-04-17T09:58:53.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.134+0000 7fb7f5200640 1 -- start start 2024-04-17T09:58:53.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.134+0000 7fb7f5200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7f013f6e0 0x7fb7f00cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:53.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.138+0000 7fb7f5200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7f01601e0 con 0x7fb7f00d09f0 2024-04-17T09:58:53.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.138+0000 7fb7f5200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7f0160480 con 0x7fb7f00cf370 2024-04-17T09:58:53.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.138+0000 7fb7f5200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb7f013fac0 con 0x7fb7f013f6e0 2024-04-17T09:58:53.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.138+0000 7fb7ede00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7f013f6e0 0x7fb7f00cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:53.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.138+0000 7fb7ede00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7f013f6e0 0x7fb7f00cf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34752/0 (socket says 172.21.15.32:34752) 2024-04-17T09:58:53.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.138+0000 7fb7ee800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb7f00d09f0 0x7fb7f0071d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49082/0 (socket says 172.21.15.32:49082) 2024-04-17T09:58:53.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.138+0000 7fb7ede00640 1 -- 172.21.15.32:0/2539845935 learned_addr learned my addr 172.21.15.32:0/2539845935 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:53.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.138+0000 7fb7ede00640 1 -- 172.21.15.32:0/2539845935 >> v1:172.21.15.144:6789/0 conn(0x7fb7f00cf370 legacy=0x7fb7f00d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:53.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.138+0000 7fb7ede00640 1 -- 172.21.15.32:0/2539845935 >> v1:172.21.15.32:6789/0 conn(0x7fb7f00d09f0 legacy=0x7fb7f0071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:53.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.138+0000 7fb7ede00640 1 -- 172.21.15.32:0/2539845935 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f0160760 con 0x7fb7f013f6e0 2024-04-17T09:58:53.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.138+0000 7fb7ede00640 1 --2- 172.21.15.32:0/2539845935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f013f6e0 0x7fb7f00cf850 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009870 tx=0x7fb7e001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275d9530c4659d32 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:53.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.138+0000 7fb7ed400640 1 -- 172.21.15.32:0/2539845935 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e00aa650 con 0x7fb7f013f6e0 2024-04-17T09:58:53.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.138+0000 7fb7ed400640 1 -- 172.21.15.32:0/2539845935 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e00af070 con 0x7fb7f013f6e0 2024-04-17T09:58:53.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.138+0000 7fb7ed400640 1 -- 172.21.15.32:0/2539845935 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e00aaa10 con 0x7fb7f013f6e0 2024-04-17T09:58:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7f5200640 1 -- 172.21.15.32:0/2539845935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f013f6e0 msgr2=0x7fb7f00cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7f5200640 1 --2- 172.21.15.32:0/2539845935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f013f6e0 0x7fb7f00cf850 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009870 tx=0x7fb7e001c4f0 comp rx=0 tx=0).stop 2024-04-17T09:58:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7f5200640 1 -- 172.21.15.32:0/2539845935 shutdown_connections 2024-04-17T09:58:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7f5200640 1 --2- 172.21.15.32:0/2539845935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f013f6e0 0x7fb7f00cf850 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7f5200640 1 -- 172.21.15.32:0/2539845935 >> 172.21.15.32:0/2539845935 conn(0x7fb7f006ba30 msgr2=0x7fb7f006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7f5200640 1 -- 172.21.15.32:0/2539845935 shutdown_connections 2024-04-17T09:58:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7f5200640 1 -- 172.21.15.32:0/2539845935 wait complete. 2024-04-17T09:58:53.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7f5200640 1 Processor -- start 2024-04-17T09:58:53.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7f5200640 1 -- start start 2024-04-17T09:58:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7f5200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7f00cf370 0x7fb7f0136d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7f5200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f00d09f0 0x7fb7f0154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7f5200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f013f6e0 0x7fb7f01551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7f5200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f00cff10 con 0x7fb7f00d09f0 2024-04-17T09:58:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7f5200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f00cfd90 con 0x7fb7f013f6e0 2024-04-17T09:58:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7f5200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb7f016a590 con 0x7fb7f00cf370 2024-04-17T09:58:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7e7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f00d09f0 0x7fb7f0154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7e7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f00d09f0 0x7fb7f0154cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34768/0 (socket says 172.21.15.32:34768) 2024-04-17T09:58:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7e7e00640 1 -- 172.21.15.32:0/1567151707 learned_addr learned my addr 172.21.15.32:0/1567151707 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7e7e00640 1 -- 172.21.15.32:0/1567151707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7f00cf370 msgr2=0x7fb7f0136d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7e7e00640 1 --2- 172.21.15.32:0/1567151707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7f00cf370 0x7fb7f0136d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7e7e00640 1 -- 172.21.15.32:0/1567151707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f013f6e0 msgr2=0x7fb7f01551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7e7e00640 1 --2- 172.21.15.32:0/1567151707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f013f6e0 0x7fb7f01551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7e7e00640 1 -- 172.21.15.32:0/1567151707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f0155a50 con 0x7fb7f00d09f0 2024-04-17T09:58:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.142+0000 7fb7e7e00640 1 --2- 172.21.15.32:0/1567151707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f00d09f0 0x7fb7f0154cb0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fb7d800c410 tx=0x7fb7d800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.146+0000 7fb7e6000640 1 -- 172.21.15.32:0/1567151707 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8011790 con 0x7fb7f00d09f0 2024-04-17T09:58:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.146+0000 7fb7e6000640 1 -- 172.21.15.32:0/1567151707 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7d8017070 con 0x7fb7f00d09f0 2024-04-17T09:58:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.146+0000 7fb7f5200640 1 -- 172.21.15.32:0/1567151707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f00b3e50 con 0x7fb7f00d09f0 2024-04-17T09:58:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.146+0000 7fb7e6000640 1 -- 172.21.15.32:0/1567151707 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8011bb0 con 0x7fb7f00d09f0 2024-04-17T09:58:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.146+0000 7fb7f5200640 1 -- 172.21.15.32:0/1567151707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f00b4410 con 0x7fb7f00d09f0 2024-04-17T09:58:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.146+0000 7fb7f5200640 1 -- 172.21.15.32:0/1567151707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f0068a90 con 0x7fb7f00d09f0 2024-04-17T09:58:53.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.146+0000 7fb7e6000640 1 -- 172.21.15.32:0/1567151707 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb7d801c460 con 0x7fb7f00d09f0 2024-04-17T09:58:53.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.146+0000 7fb7e6000640 1 --2- 172.21.15.32:0/1567151707 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb7cc068380 0x7fb7cc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:53.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.146+0000 7fb7e6000640 1 -- 172.21.15.32:0/1567151707 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fb7d80866d0 con 0x7fb7f00d09f0 2024-04-17T09:58:53.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.146+0000 7fb7ede00640 1 --2- 172.21.15.32:0/1567151707 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb7cc068380 0x7fb7cc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:53.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.146+0000 7fb7ede00640 1 --2- 172.21.15.32:0/1567151707 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb7cc068380 0x7fb7cc06a840 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00ae000 tx=0x7fb7e0004740 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:53.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.150+0000 7fb7e6000640 1 -- 172.21.15.32:0/1567151707 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fb7d8021020 con 0x7fb7f00d09f0 2024-04-17T09:58:53.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.354+0000 7fb7f5200640 1 -- 172.21.15.32:0/1567151707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb7f00d1030 con 0x7fb7f00d09f0 2024-04-17T09:58:53.352 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-17T09:58:53.354+0000 7f93c9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T09:58:53.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.354+0000 7fb7e6000640 1 -- 172.21.15.32:0/1567151707 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v51) ==== 161+0+0 (secure 0 0 0) 0x7fb7d8056bf0 con 0x7fb7f00d09f0 2024-04-17T09:58:53.354 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T09:58:53.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.358+0000 7fb7f5200640 1 -- 172.21.15.32:0/1567151707 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb7cc068380 msgr2=0x7fb7cc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:53.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.358+0000 7fb7f5200640 1 --2- 172.21.15.32:0/1567151707 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb7cc068380 0x7fb7cc06a840 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00ae000 tx=0x7fb7e0004740 comp rx=0 tx=0).stop 2024-04-17T09:58:53.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.358+0000 7fb7f5200640 1 -- 172.21.15.32:0/1567151707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f00d09f0 msgr2=0x7fb7f0154cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:53.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.358+0000 7fb7f5200640 1 --2- 172.21.15.32:0/1567151707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f00d09f0 0x7fb7f0154cb0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fb7d800c410 tx=0x7fb7d800c8d0 comp rx=0 tx=0).stop 2024-04-17T09:58:53.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.358+0000 7fb7f5200640 1 -- 172.21.15.32:0/1567151707 shutdown_connections 2024-04-17T09:58:53.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.358+0000 7fb7f5200640 1 --2- 172.21.15.32:0/1567151707 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb7cc068380 0x7fb7cc06a840 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:53.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.358+0000 7fb7f5200640 1 --2- 172.21.15.32:0/1567151707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f013f6e0 0x7fb7f01551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:53.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.358+0000 7fb7f5200640 1 --2- 172.21.15.32:0/1567151707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7f00d09f0 0x7fb7f0154cb0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:53.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.358+0000 7fb7f5200640 1 --2- 172.21.15.32:0/1567151707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7f00cf370 0x7fb7f0136d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:53.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.358+0000 7fb7f5200640 1 -- 172.21.15.32:0/1567151707 >> 172.21.15.32:0/1567151707 conn(0x7fb7f006ba30 msgr2=0x7fb7f00d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:53.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.358+0000 7fb7f5200640 1 -- 172.21.15.32:0/1567151707 shutdown_connections 2024-04-17T09:58:53.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.358+0000 7fb7f5200640 1 -- 172.21.15.32:0/1567151707 wait complete. 2024-04-17T09:58:53.367 INFO:tasks.ceph.mds.a.smithi032.stderr: -4387> 2024-04-17T09:57:36.759+0000 7f9ece800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T09:58:53.379 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-17T09:58:53.416 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-17T09:58:53.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.526+0000 7fcd25e00640 1 Processor -- start 2024-04-17T09:58:53.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.526+0000 7fcd25e00640 1 -- start start 2024-04-17T09:58:53.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.526+0000 7fcd25e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd2006f7c0 0x7fcd2006ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:53.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.526+0000 7fcd25e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcd2006f270 0x7fcd2006bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:53.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.526+0000 7fcd25e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd200cc940 con 0x7fcd2015b7d0 2024-04-17T09:58:53.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.526+0000 7fcd25e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fcd200ccc20 con 0x7fcd2006f270 2024-04-17T09:58:53.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.526+0000 7fcd25e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcd2006e0b0 con 0x7fcd2006f7c0 2024-04-17T09:58:53.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.526+0000 7fcd1f400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcd2006f270 0x7fcd2006bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:53.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.526+0000 7fcd1f400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcd2006f270 0x7fcd2006bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34792/0 (socket says 172.21.15.32:34792) 2024-04-17T09:58:53.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.526+0000 7fcd1f400640 1 -- 172.21.15.32:0/3515761809 learned_addr learned my addr 172.21.15.32:0/3515761809 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:53.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.526+0000 7fcd16000640 1 --2- 172.21.15.32:0/3515761809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd2006f7c0 0x7fcd2006ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:53.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.526+0000 7fcd1f400640 1 -- 172.21.15.32:0/3515761809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd2006f7c0 msgr2=0x7fcd2006ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:53.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.526+0000 7fcd1f400640 1 --2- 172.21.15.32:0/3515761809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd2006f7c0 0x7fcd2006ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:53.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.526+0000 7fcd1f400640 1 -- 172.21.15.32:0/3515761809 >> v1:172.21.15.144:6789/0 conn(0x7fcd2015b7d0 legacy=0x7fcd2006e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:53.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.526+0000 7fcd1f400640 1 -- 172.21.15.32:0/3515761809 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2006e390 con 0x7fcd2006f270 2024-04-17T09:58:53.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.526+0000 7fcd1e000640 1 -- 172.21.15.32:0/3515761809 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1178123035 0 0) 0x7fcd200cc940 con 0x7fcd2015b7d0 2024-04-17T09:58:53.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.530+0000 7fcd1f400640 1 --2- 172.21.15.32:0/3515761809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd2006f270 0x7fcd2006bf70 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009e50 tx=0x7fcd1801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e142970c30ce62 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.530+0000 7fcd1e000640 1 -- 172.21.15.32:0/3515761809 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd180a5770 con 0x7fcd2006f270 2024-04-17T09:58:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.530+0000 7fcd1e000640 1 -- 172.21.15.32:0/3515761809 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd180a5910 con 0x7fcd2006f270 2024-04-17T09:58:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.530+0000 7fcd1e000640 1 -- 172.21.15.32:0/3515761809 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd180a5ce0 con 0x7fcd2006f270 2024-04-17T09:58:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.530+0000 7fcd25e00640 1 -- 172.21.15.32:0/3515761809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd2006f270 msgr2=0x7fcd2006bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:53.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.530+0000 7fcd25e00640 1 --2- 172.21.15.32:0/3515761809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd2006f270 0x7fcd2006bf70 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009e50 tx=0x7fcd1801c9a0 comp rx=0 tx=0).stop 2024-04-17T09:58:53.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.530+0000 7fcd25e00640 1 -- 172.21.15.32:0/3515761809 shutdown_connections 2024-04-17T09:58:53.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.530+0000 7fcd25e00640 1 --2- 172.21.15.32:0/3515761809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd2006f270 0x7fcd2006bf70 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:53.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.530+0000 7fcd25e00640 1 --2- 172.21.15.32:0/3515761809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd2006f7c0 0x7fcd2006ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:53.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.530+0000 7fcd25e00640 1 -- 172.21.15.32:0/3515761809 >> 172.21.15.32:0/3515761809 conn(0x7fcd200b2120 msgr2=0x7fcd200b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:53.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.530+0000 7fcd25e00640 1 -- 172.21.15.32:0/3515761809 shutdown_connections 2024-04-17T09:58:53.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.530+0000 7fcd25e00640 1 -- 172.21.15.32:0/3515761809 wait complete. 2024-04-17T09:58:53.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd25e00640 1 Processor -- start 2024-04-17T09:58:53.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd25e00640 1 -- start start 2024-04-17T09:58:53.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd25e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd2006f270 0x7fcd20144210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:53.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd25e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd2006f7c0 0x7fcd20144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:53.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd25e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd2015b7d0 0x7fcd20148070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:53.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd25e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcd200cee30 con 0x7fcd2006f7c0 2024-04-17T09:58:53.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd25e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcd200cecb0 con 0x7fcd2006f270 2024-04-17T09:58:53.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd25e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcd201682a0 con 0x7fcd2015b7d0 2024-04-17T09:58:53.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd16000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd2006f7c0 0x7fcd20144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:53.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd16000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd2006f7c0 0x7fcd20144e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34794/0 (socket says 172.21.15.32:34794) 2024-04-17T09:58:53.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd16000640 1 -- 172.21.15.32:0/2493659333 learned_addr learned my addr 172.21.15.32:0/2493659333 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:53.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd1f400640 1 --2- 172.21.15.32:0/2493659333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd2015b7d0 0x7fcd20148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:53.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd1ea00640 1 --2- 172.21.15.32:0/2493659333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd2006f270 0x7fcd20144210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:53.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd16000640 1 -- 172.21.15.32:0/2493659333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd2015b7d0 msgr2=0x7fcd20148070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:53.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd16000640 1 --2- 172.21.15.32:0/2493659333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd2015b7d0 0x7fcd20148070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:53.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd16000640 1 -- 172.21.15.32:0/2493659333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd2006f270 msgr2=0x7fcd20144210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:53.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd16000640 1 --2- 172.21.15.32:0/2493659333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd2006f270 0x7fcd20144210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:53.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd16000640 1 -- 172.21.15.32:0/2493659333 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd201556d0 con 0x7fcd2006f7c0 2024-04-17T09:58:53.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd16000640 1 --2- 172.21.15.32:0/2493659333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd2006f7c0 0x7fcd20144e70 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800e8a0 tx=0x7fcd0800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:53.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd17e00640 1 -- 172.21.15.32:0/2493659333 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08017070 con 0x7fcd2006f7c0 2024-04-17T09:58:53.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd17e00640 1 -- 172.21.15.32:0/2493659333 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd0800c940 con 0x7fcd2006f7c0 2024-04-17T09:58:53.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd17e00640 1 -- 172.21.15.32:0/2493659333 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0800ccf0 con 0x7fcd2006f7c0 2024-04-17T09:58:53.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd25e00640 1 -- 172.21.15.32:0/2493659333 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd201553f0 con 0x7fcd2006f7c0 2024-04-17T09:58:53.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.534+0000 7fcd25e00640 1 -- 172.21.15.32:0/2493659333 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd20151120 con 0x7fcd2006f7c0 2024-04-17T09:58:53.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.538+0000 7fcd25e00640 1 -- 172.21.15.32:0/2493659333 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd20068a90 con 0x7fcd2006f7c0 2024-04-17T09:58:53.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.538+0000 7fcd17e00640 1 -- 172.21.15.32:0/2493659333 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcd0801a070 con 0x7fcd2006f7c0 2024-04-17T09:58:53.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.538+0000 7fcd17e00640 1 --2- 172.21.15.32:0/2493659333 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fccfc068380 0x7fccfc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:53.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.538+0000 7fcd17e00640 1 -- 172.21.15.32:0/2493659333 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5939+0+0 (secure 0 0 0) 0x7fcd08086d40 con 0x7fcd2006f7c0 2024-04-17T09:58:53.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.538+0000 7fcd1ea00640 1 --2- 172.21.15.32:0/2493659333 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fccfc068380 0x7fccfc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:53.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.538+0000 7fcd1ea00640 1 --2- 172.21.15.32:0/2493659333 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fccfc068380 0x7fccfc06a840 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fcd10000f70 tx=0x7fcd10009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:53.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.542+0000 7fcd17e00640 1 -- 172.21.15.32:0/2493659333 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fcd08057200 con 0x7fcd2006f7c0 2024-04-17T09:58:53.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:53.746+0000 7fcd25e00640 1 -- 172.21.15.32:0/2493659333 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fcd20069320 con 0x7fcd2006f7c0 2024-04-17T09:58:54.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.358+0000 7fcd17e00640 1 -- 172.21.15.32:0/2493659333 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v52) ==== 104+0+0 (secure 0 0 0) 0x7fcd08056b00 con 0x7fcd2006f7c0 2024-04-17T09:58:54.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.362+0000 7fcd25e00640 1 -- 172.21.15.32:0/2493659333 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fccfc068380 msgr2=0x7fccfc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:54.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.362+0000 7fcd25e00640 1 --2- 172.21.15.32:0/2493659333 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fccfc068380 0x7fccfc06a840 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fcd10000f70 tx=0x7fcd10009040 comp rx=0 tx=0).stop 2024-04-17T09:58:54.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.362+0000 7fcd25e00640 1 -- 172.21.15.32:0/2493659333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd2006f7c0 msgr2=0x7fcd20144e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:54.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.362+0000 7fcd25e00640 1 --2- 172.21.15.32:0/2493659333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd2006f7c0 0x7fcd20144e70 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800e8a0 tx=0x7fcd0800eca0 comp rx=0 tx=0).stop 2024-04-17T09:58:54.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.362+0000 7fcd25e00640 1 -- 172.21.15.32:0/2493659333 shutdown_connections 2024-04-17T09:58:54.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.362+0000 7fcd25e00640 1 --2- 172.21.15.32:0/2493659333 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fccfc068380 0x7fccfc06a840 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:54.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.362+0000 7fcd25e00640 1 --2- 172.21.15.32:0/2493659333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd2015b7d0 0x7fcd20148070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:54.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.362+0000 7fcd25e00640 1 --2- 172.21.15.32:0/2493659333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd2006f7c0 0x7fcd20144e70 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:54.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.362+0000 7fcd25e00640 1 --2- 172.21.15.32:0/2493659333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd2006f270 0x7fcd20144210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:54.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.362+0000 7fcd25e00640 1 -- 172.21.15.32:0/2493659333 >> 172.21.15.32:0/2493659333 conn(0x7fcd200b2120 msgr2=0x7fcd2006ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:54.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.362+0000 7fcd25e00640 1 -- 172.21.15.32:0/2493659333 shutdown_connections 2024-04-17T09:58:54.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.362+0000 7fcd25e00640 1 -- 172.21.15.32:0/2493659333 wait complete. 2024-04-17T09:58:54.383 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-17T09:58:54.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.574+0000 7f854d800640 1 Processor -- start 2024-04-17T09:58:54.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.574+0000 7f854d800640 1 -- start start 2024-04-17T09:58:54.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.578+0000 7f854d800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f85481482d0 0x7f8548163070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:54.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.578+0000 7f854d800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f854815fb50 con 0x7f854815f5b0 2024-04-17T09:58:54.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.578+0000 7f854d800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f854815fe30 con 0x7f85480c9a90 2024-04-17T09:58:54.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.578+0000 7f854d800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f85480ce680 con 0x7f85481482d0 2024-04-17T09:58:54.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.578+0000 7f8547400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f854815f5b0 0x7f85480721b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49106/0 (socket says 172.21.15.32:49106) 2024-04-17T09:58:54.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.578+0000 7f8547400640 1 -- 172.21.15.32:0/1027137045 learned_addr learned my addr 172.21.15.32:0/1027137045 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:58:54.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.578+0000 7f8546a00640 1 --2- 172.21.15.32:0/1027137045 >> v2:172.21.15.144:3300/0 conn(0x7f85481482d0 0x7f8548163070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:54.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.578+0000 7f8546000640 1 -- 172.21.15.32:0/1027137045 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1327830883 0 0) 0x7f854815fb50 con 0x7f854815f5b0 2024-04-17T09:58:54.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.578+0000 7f8546000640 1 -- 172.21.15.32:0/1027137045 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f852c003530 con 0x7f854815f5b0 2024-04-17T09:58:54.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.578+0000 7f8546000640 1 -- 172.21.15.32:0/1027137045 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3306252328 0 0) 0x7f854815fe30 con 0x7f85480c9a90 2024-04-17T09:58:54.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.578+0000 7f8546000640 1 -- 172.21.15.32:0/1027137045 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f854815fb50 con 0x7f85480c9a90 2024-04-17T09:58:54.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.578+0000 7f8546a00640 1 -- 172.21.15.32:0/1027137045 >> v1:172.21.15.144:6789/0 conn(0x7f85480c9a90 legacy=0x7f854815eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:54.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.578+0000 7f8546a00640 1 -- 172.21.15.32:0/1027137045 >> v1:172.21.15.32:6789/0 conn(0x7f854815f5b0 legacy=0x7f85480721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:54.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.578+0000 7f8546a00640 1 -- 172.21.15.32:0/1027137045 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8548162590 con 0x7f85481482d0 2024-04-17T09:58:54.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.582+0000 7f8546a00640 1 --2- 172.21.15.32:0/1027137045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85481482d0 0x7f8548163070 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f85380097c0 tx=0x7f853801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31ca9ea413c07dc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:54.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.582+0000 7f8546000640 1 -- 172.21.15.32:0/1027137045 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85380b1070 con 0x7f85481482d0 2024-04-17T09:58:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.582+0000 7f8546000640 1 -- 172.21.15.32:0/1027137045 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85380a8050 con 0x7f85481482d0 2024-04-17T09:58:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.582+0000 7f8546000640 1 -- 172.21.15.32:0/1027137045 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85380ac860 con 0x7f85481482d0 2024-04-17T09:58:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.582+0000 7f854d800640 1 -- 172.21.15.32:0/1027137045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85481482d0 msgr2=0x7f8548163070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.582+0000 7f854d800640 1 --2- 172.21.15.32:0/1027137045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85481482d0 0x7f8548163070 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f85380097c0 tx=0x7f853801c3e0 comp rx=0 tx=0).stop 2024-04-17T09:58:54.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.582+0000 7f854d800640 1 -- 172.21.15.32:0/1027137045 shutdown_connections 2024-04-17T09:58:54.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.582+0000 7f854d800640 1 --2- 172.21.15.32:0/1027137045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85481482d0 0x7f8548163070 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:54.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.582+0000 7f854d800640 1 -- 172.21.15.32:0/1027137045 >> 172.21.15.32:0/1027137045 conn(0x7f85480d0740 msgr2=0x7f85480d0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:54.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.582+0000 7f854d800640 1 -- 172.21.15.32:0/1027137045 shutdown_connections 2024-04-17T09:58:54.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.582+0000 7f854d800640 1 -- 172.21.15.32:0/1027137045 wait complete. 2024-04-17T09:58:54.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.582+0000 7f854d800640 1 Processor -- start 2024-04-17T09:58:54.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.582+0000 7f854d800640 1 -- start start 2024-04-17T09:58:54.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.582+0000 7f854d800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85480c9a90 0x7f854806ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:54.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.582+0000 7f854d800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85481482d0 0x7f8548140a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:54.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.582+0000 7f854d800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f854815f5b0 0x7f8548140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:54.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.582+0000 7f854d800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f85480ca2b0 con 0x7f85480c9a90 2024-04-17T09:58:54.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.582+0000 7f854d800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8548072cd0 con 0x7f85481482d0 2024-04-17T09:58:54.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.582+0000 7f854d800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f854815e620 con 0x7f854815f5b0 2024-04-17T09:58:54.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.586+0000 7f8546a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85480c9a90 0x7f854806ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:54.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.586+0000 7f8546a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85480c9a90 0x7f854806ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34802/0 (socket says 172.21.15.32:34802) 2024-04-17T09:58:54.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.586+0000 7f8546a00640 1 -- 172.21.15.32:0/243618714 learned_addr learned my addr 172.21.15.32:0/243618714 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:54.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.586+0000 7f853fe00640 1 --2- 172.21.15.32:0/243618714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85481482d0 0x7f8548140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:54.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.586+0000 7f8547400640 1 --2- 172.21.15.32:0/243618714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f854815f5b0 0x7f8548140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:54.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.586+0000 7f8546a00640 1 -- 172.21.15.32:0/243618714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f854815f5b0 msgr2=0x7f8548140fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:54.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.586+0000 7f8546a00640 1 --2- 172.21.15.32:0/243618714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f854815f5b0 0x7f8548140fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:54.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.586+0000 7f8546a00640 1 -- 172.21.15.32:0/243618714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85481482d0 msgr2=0x7f8548140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:54.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.586+0000 7f8546a00640 1 --2- 172.21.15.32:0/243618714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85481482d0 0x7f8548140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:54.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.586+0000 7f8546a00640 1 -- 172.21.15.32:0/243618714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8548141660 con 0x7f85480c9a90 2024-04-17T09:58:54.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.586+0000 7f853fe00640 1 --2- 172.21.15.32:0/243618714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85481482d0 0x7f8548140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:58:54.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.586+0000 7f8546a00640 1 --2- 172.21.15.32:0/243618714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85480c9a90 0x7f854806ff50 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f8538018ff0 tx=0x7f853801ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:54.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.586+0000 7f853ea00640 1 -- 172.21.15.32:0/243618714 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85380b1050 con 0x7f85480c9a90 2024-04-17T09:58:54.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.586+0000 7f853ea00640 1 -- 172.21.15.32:0/243618714 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85380bd070 con 0x7f85480c9a90 2024-04-17T09:58:54.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.586+0000 7f853ea00640 1 -- 172.21.15.32:0/243618714 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85380b8690 con 0x7f85480c9a90 2024-04-17T09:58:54.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.586+0000 7f854d800640 1 -- 172.21.15.32:0/243618714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85480cc4f0 con 0x7f85480c9a90 2024-04-17T09:58:54.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.586+0000 7f854d800640 1 -- 172.21.15.32:0/243618714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85480ccab0 con 0x7f85480c9a90 2024-04-17T09:58:54.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.586+0000 7f854d800640 1 -- 172.21.15.32:0/243618714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8548068a90 con 0x7f85480c9a90 2024-04-17T09:58:54.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.586+0000 7f853ea00640 1 -- 172.21.15.32:0/243618714 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f85380b8830 con 0x7f85480c9a90 2024-04-17T09:58:54.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.586+0000 7f853ea00640 1 --2- 172.21.15.32:0/243618714 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8524068380 0x7f852406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:54.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.586+0000 7f853ea00640 1 -- 172.21.15.32:0/243618714 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5939+0+0 (secure 0 0 0) 0x7f8538121250 con 0x7f85480c9a90 2024-04-17T09:58:54.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.590+0000 7f853fe00640 1 --2- 172.21.15.32:0/243618714 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8524068380 0x7f852406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:54.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.590+0000 7f853fe00640 1 --2- 172.21.15.32:0/243618714 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8524068380 0x7f852406a840 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f85480d0620 tx=0x7f8530009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:54.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.594+0000 7f853ea00640 1 -- 172.21.15.32:0/243618714 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f8538016090 con 0x7f85480c9a90 2024-04-17T09:58:54.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:54.758+0000 7f854d800640 1 -- 172.21.15.32:0/243618714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8548075530 con 0x7f85480c9a90 2024-04-17T09:58:55.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.366+0000 7f853ea00640 1 -- 172.21.15.32:0/243618714 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v73) ==== 181+0+0 (secure 0 0 0) 0x7f85380f1710 con 0x7f85480c9a90 2024-04-17T09:58:55.363 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-04-17T09:58:55.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.370+0000 7f854d800640 1 -- 172.21.15.32:0/243618714 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8524068380 msgr2=0x7f852406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:55.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.370+0000 7f854d800640 1 --2- 172.21.15.32:0/243618714 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8524068380 0x7f852406a840 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f85480d0620 tx=0x7f8530009040 comp rx=0 tx=0).stop 2024-04-17T09:58:55.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.370+0000 7f854d800640 1 -- 172.21.15.32:0/243618714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85480c9a90 msgr2=0x7f854806ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:55.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.370+0000 7f854d800640 1 --2- 172.21.15.32:0/243618714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85480c9a90 0x7f854806ff50 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f8538018ff0 tx=0x7f853801ac20 comp rx=0 tx=0).stop 2024-04-17T09:58:55.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.370+0000 7f854d800640 1 -- 172.21.15.32:0/243618714 shutdown_connections 2024-04-17T09:58:55.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.370+0000 7f854d800640 1 --2- 172.21.15.32:0/243618714 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8524068380 0x7f852406a840 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:55.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.370+0000 7f854d800640 1 --2- 172.21.15.32:0/243618714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f854815f5b0 0x7f8548140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:55.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.370+0000 7f854d800640 1 --2- 172.21.15.32:0/243618714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85481482d0 0x7f8548140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:55.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.370+0000 7f854d800640 1 --2- 172.21.15.32:0/243618714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85480c9a90 0x7f854806ff50 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:55.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.370+0000 7f854d800640 1 -- 172.21.15.32:0/243618714 >> 172.21.15.32:0/243618714 conn(0x7f85480d0740 msgr2=0x7f854806e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:55.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.370+0000 7f854d800640 1 -- 172.21.15.32:0/243618714 shutdown_connections 2024-04-17T09:58:55.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.370+0000 7f854d800640 1 -- 172.21.15.32:0/243618714 wait complete. 2024-04-17T09:58:55.396 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-17T09:58:55.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.546+0000 7fbaed600640 1 Processor -- start 2024-04-17T09:58:55.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.546+0000 7fbaed600640 1 -- start start 2024-04-17T09:58:55.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.546+0000 7fbaed600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbae80ceb40 0x7fbae8071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:55.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.550+0000 7fbaed600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbae80722f0 0x7fbae815e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:55.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.550+0000 7fbaed600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbae80cefa0 con 0x7fbae81460a0 2024-04-17T09:58:55.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.550+0000 7fbaed600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fbae815de00 con 0x7fbae80ceb40 2024-04-17T09:58:55.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.550+0000 7fbaed600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbae806eeb0 con 0x7fbae80722f0 2024-04-17T09:58:55.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.550+0000 7fbade000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbae80ceb40 0x7fbae8071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:55.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.550+0000 7fbade000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbae80ceb40 0x7fbae8071db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34816/0 (socket says 172.21.15.32:34816) 2024-04-17T09:58:55.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.550+0000 7fbade000640 1 -- 172.21.15.32:0/2139060756 learned_addr learned my addr 172.21.15.32:0/2139060756 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:55.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.550+0000 7fbae7400640 1 --2- 172.21.15.32:0/2139060756 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbae80722f0 0x7fbae815e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:55.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.550+0000 7fbade000640 1 -- 172.21.15.32:0/2139060756 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbae80722f0 msgr2=0x7fbae815e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:55.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.550+0000 7fbade000640 1 --2- 172.21.15.32:0/2139060756 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbae80722f0 0x7fbae815e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:55.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.550+0000 7fbade000640 1 -- 172.21.15.32:0/2139060756 >> v1:172.21.15.144:6789/0 conn(0x7fbae81460a0 legacy=0x7fbae806e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:55.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.550+0000 7fbade000640 1 -- 172.21.15.32:0/2139060756 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae806f190 con 0x7fbae80ceb40 2024-04-17T09:58:55.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.550+0000 7fbade000640 1 --2- 172.21.15.32:0/2139060756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae80ceb40 0x7fbae8071db0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fbad0006fd0 tx=0x7fbad001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac5e1cf9fc4fc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:55.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.550+0000 7fbae6000640 1 -- 172.21.15.32:0/2139060756 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad00b0070 con 0x7fbae80ceb40 2024-04-17T09:58:55.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbae6000640 1 -- 172.21.15.32:0/2139060756 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbad00ab600 con 0x7fbae80ceb40 2024-04-17T09:58:55.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbae6000640 1 -- 172.21.15.32:0/2139060756 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad00ab9d0 con 0x7fbae80ceb40 2024-04-17T09:58:55.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbaed600640 1 -- 172.21.15.32:0/2139060756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae80ceb40 msgr2=0x7fbae8071db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:55.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbaed600640 1 --2- 172.21.15.32:0/2139060756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae80ceb40 0x7fbae8071db0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fbad0006fd0 tx=0x7fbad001c540 comp rx=0 tx=0).stop 2024-04-17T09:58:55.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbaed600640 1 -- 172.21.15.32:0/2139060756 shutdown_connections 2024-04-17T09:58:55.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbaed600640 1 --2- 172.21.15.32:0/2139060756 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbae80722f0 0x7fbae815e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:55.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbaed600640 1 --2- 172.21.15.32:0/2139060756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae80ceb40 0x7fbae8071db0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:55.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbaed600640 1 -- 172.21.15.32:0/2139060756 >> 172.21.15.32:0/2139060756 conn(0x7fbae80b2190 msgr2=0x7fbae80b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:55.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbaed600640 1 -- 172.21.15.32:0/2139060756 shutdown_connections 2024-04-17T09:58:55.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbaed600640 1 -- 172.21.15.32:0/2139060756 wait complete. 2024-04-17T09:58:55.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbaed600640 1 Processor -- start 2024-04-17T09:58:55.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbaed600640 1 -- start start 2024-04-17T09:58:55.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbaed600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbae80722f0 0x7fbae815c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:55.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbaed600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbae80ceb40 0x7fbae80b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:55.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbaed600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae81460a0 0x7fbae80b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:55.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbaed600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbae815f670 con 0x7fbae81460a0 2024-04-17T09:58:55.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbaed600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbae815f4f0 con 0x7fbae80722f0 2024-04-17T09:58:55.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbaed600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbae815f7f0 con 0x7fbae80ceb40 2024-04-17T09:58:55.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbae7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae81460a0 0x7fbae80b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:55.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbae7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae81460a0 0x7fbae80b50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34832/0 (socket says 172.21.15.32:34832) 2024-04-17T09:58:55.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbae7400640 1 -- 172.21.15.32:0/474423614 learned_addr learned my addr 172.21.15.32:0/474423614 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:55.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbade000640 1 --2- 172.21.15.32:0/474423614 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbae80ceb40 0x7fbae80b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:55.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbae6a00640 1 --2- 172.21.15.32:0/474423614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbae80722f0 0x7fbae815c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:55.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbae7400640 1 -- 172.21.15.32:0/474423614 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbae80ceb40 msgr2=0x7fbae80b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:55.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbae7400640 1 --2- 172.21.15.32:0/474423614 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbae80ceb40 0x7fbae80b4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:55.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbae7400640 1 -- 172.21.15.32:0/474423614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbae80722f0 msgr2=0x7fbae815c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:55.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbae7400640 1 --2- 172.21.15.32:0/474423614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbae80722f0 0x7fbae815c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:55.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.554+0000 7fbae7400640 1 -- 172.21.15.32:0/474423614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae8142f40 con 0x7fbae81460a0 2024-04-17T09:58:55.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.558+0000 7fbae7400640 1 --2- 172.21.15.32:0/474423614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae81460a0 0x7fbae80b50a0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fbae000efc0 tx=0x7fbae000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:55.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.558+0000 7fbadf400640 1 -- 172.21.15.32:0/474423614 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae0018020 con 0x7fbae81460a0 2024-04-17T09:58:55.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.558+0000 7fbadf400640 1 -- 172.21.15.32:0/474423614 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae0014480 con 0x7fbae81460a0 2024-04-17T09:58:55.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.558+0000 7fbadf400640 1 -- 172.21.15.32:0/474423614 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae0014870 con 0x7fbae81460a0 2024-04-17T09:58:55.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.558+0000 7fbaed600640 1 -- 172.21.15.32:0/474423614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae8143220 con 0x7fbae81460a0 2024-04-17T09:58:55.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.558+0000 7fbaed600640 1 -- 172.21.15.32:0/474423614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae80ad860 con 0x7fbae81460a0 2024-04-17T09:58:55.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.558+0000 7fbaed600640 1 -- 172.21.15.32:0/474423614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbae8068a90 con 0x7fbae81460a0 2024-04-17T09:58:55.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.558+0000 7fbadf400640 1 -- 172.21.15.32:0/474423614 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbae001b020 con 0x7fbae81460a0 2024-04-17T09:58:55.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.558+0000 7fbadf400640 1 --2- 172.21.15.32:0/474423614 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbac4068380 0x7fbac406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:55.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.558+0000 7fbadf400640 1 -- 172.21.15.32:0/474423614 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5501+0+0 (secure 0 0 0) 0x7fbae00874a0 con 0x7fbae81460a0 2024-04-17T09:58:55.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.558+0000 7fbae6a00640 1 --2- 172.21.15.32:0/474423614 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbac4068380 0x7fbac406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:55.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.562+0000 7fbae6a00640 1 --2- 172.21.15.32:0/474423614 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbac4068380 0x7fbac406a840 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fbad80015b0 tx=0x7fbad8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:55.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.562+0000 7fbadf400640 1 -- 172.21.15.32:0/474423614 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fbae0022030 con 0x7fbae81460a0 2024-04-17T09:58:55.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:55.726+0000 7fbaed600640 1 -- 172.21.15.32:0/474423614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbae8072020 con 0x7fbae81460a0 2024-04-17T09:58:56.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.366+0000 7fbadf400640 1 -- 172.21.15.32:0/474423614 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v74) ==== 169+0+0 (secure 0 0 0) 0x7fbae0085e20 con 0x7fbae81460a0 2024-04-17T09:58:56.365 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-04-17T09:58:56.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.370+0000 7fbaed600640 1 -- 172.21.15.32:0/474423614 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbac4068380 msgr2=0x7fbac406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:56.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.370+0000 7fbaed600640 1 --2- 172.21.15.32:0/474423614 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbac4068380 0x7fbac406a840 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fbad80015b0 tx=0x7fbad8009040 comp rx=0 tx=0).stop 2024-04-17T09:58:56.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.370+0000 7fbaed600640 1 -- 172.21.15.32:0/474423614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae81460a0 msgr2=0x7fbae80b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:56.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.370+0000 7fbaed600640 1 --2- 172.21.15.32:0/474423614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae81460a0 0x7fbae80b50a0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fbae000efc0 tx=0x7fbae000c490 comp rx=0 tx=0).stop 2024-04-17T09:58:56.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.370+0000 7fbaed600640 1 -- 172.21.15.32:0/474423614 shutdown_connections 2024-04-17T09:58:56.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.370+0000 7fbaed600640 1 --2- 172.21.15.32:0/474423614 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbac4068380 0x7fbac406a840 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:56.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.370+0000 7fbaed600640 1 --2- 172.21.15.32:0/474423614 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbae80ceb40 0x7fbae80b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:56.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.370+0000 7fbaed600640 1 --2- 172.21.15.32:0/474423614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbae81460a0 0x7fbae80b50a0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:56.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.370+0000 7fbaed600640 1 --2- 172.21.15.32:0/474423614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbae80722f0 0x7fbae815c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:56.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.370+0000 7fbaed600640 1 -- 172.21.15.32:0/474423614 >> 172.21.15.32:0/474423614 conn(0x7fbae80b2190 msgr2=0x7fbae80b16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:56.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.370+0000 7fbaed600640 1 -- 172.21.15.32:0/474423614 shutdown_connections 2024-04-17T09:58:56.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.370+0000 7fbaed600640 1 -- 172.21.15.32:0/474423614 wait complete. 2024-04-17T09:58:56.399 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-17T09:58:56.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.546+0000 7fa7ad800640 1 Processor -- start 2024-04-17T09:58:56.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.546+0000 7fa7ad800640 1 -- start start 2024-04-17T09:58:56.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.546+0000 7fa7ad800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa7a806ba30 0x7fa7a806be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:56.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.546+0000 7fa7ad800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7a80cf370 0x7fa7a80cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:56.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.550+0000 7fa7ad800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7a806c350 con 0x7fa7a8160350 2024-04-17T09:58:56.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.550+0000 7fa7ad800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa7a80d0130 con 0x7fa7a806ba30 2024-04-17T09:58:56.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.550+0000 7fa7ad800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa7a815d090 con 0x7fa7a80cf370 2024-04-17T09:58:56.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.550+0000 7fa7a6a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa7a806ba30 0x7fa7a806be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:56.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.550+0000 7fa7a7400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa7a8160350 0x7fa7a81607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49112/0 (socket says 172.21.15.32:49112) 2024-04-17T09:58:56.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.550+0000 7fa7a7400640 1 -- 172.21.15.32:0/1030418671 learned_addr learned my addr 172.21.15.32:0/1030418671 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:58:56.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.550+0000 7fa79e000640 1 --2- 172.21.15.32:0/1030418671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7a80cf370 0x7fa7a80cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:56.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.550+0000 7fa7a6a00640 1 -- 172.21.15.32:0/1030418671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7a80cf370 msgr2=0x7fa7a80cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:56.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.550+0000 7fa7a6a00640 1 --2- 172.21.15.32:0/1030418671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7a80cf370 0x7fa7a80cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:56.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.550+0000 7fa7a6a00640 1 -- 172.21.15.32:0/1030418671 >> v1:172.21.15.32:6789/0 conn(0x7fa7a8160350 legacy=0x7fa7a81607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:56.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.550+0000 7fa7a6a00640 1 -- 172.21.15.32:0/1030418671 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a815d370 con 0x7fa7a806ba30 2024-04-17T09:58:56.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.550+0000 7fa7a6a00640 1 --2- 172.21.15.32:0/1030418671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7a806ba30 0x7fa7a806be10 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fa798009d90 tx=0x7fa79801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d9e3b46e850112 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:56.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.550+0000 7fa7a6000640 1 -- 172.21.15.32:0/1030418671 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7980b2070 con 0x7fa7a806ba30 2024-04-17T09:58:56.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.550+0000 7fa7a6000640 1 -- 172.21.15.32:0/1030418671 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7980a5940 con 0x7fa7a806ba30 2024-04-17T09:58:56.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.550+0000 7fa7a6000640 1 -- 172.21.15.32:0/1030418671 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7980a5d10 con 0x7fa7a806ba30 2024-04-17T09:58:56.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7ad800640 1 -- 172.21.15.32:0/1030418671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7a806ba30 msgr2=0x7fa7a806be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:56.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7ad800640 1 --2- 172.21.15.32:0/1030418671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7a806ba30 0x7fa7a806be10 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fa798009d90 tx=0x7fa79801c970 comp rx=0 tx=0).stop 2024-04-17T09:58:56.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7ad800640 1 -- 172.21.15.32:0/1030418671 shutdown_connections 2024-04-17T09:58:56.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7ad800640 1 --2- 172.21.15.32:0/1030418671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7a80cf370 0x7fa7a80cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:56.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7ad800640 1 --2- 172.21.15.32:0/1030418671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7a806ba30 0x7fa7a806be10 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:56.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7ad800640 1 -- 172.21.15.32:0/1030418671 >> 172.21.15.32:0/1030418671 conn(0x7fa7a80b2120 msgr2=0x7fa7a80b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:56.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7ad800640 1 -- 172.21.15.32:0/1030418671 shutdown_connections 2024-04-17T09:58:56.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7ad800640 1 -- 172.21.15.32:0/1030418671 wait complete. 2024-04-17T09:58:56.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7ad800640 1 Processor -- start 2024-04-17T09:58:56.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7ad800640 1 -- start start 2024-04-17T09:58:56.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7ad800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7a806ba30 0x7fa7a813f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:56.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7ad800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7a80cf370 0x7fa7a80a9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:56.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7ad800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7a8160350 0x7fa7a80aa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:56.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7ad800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a815dcf0 con 0x7fa7a8160350 2024-04-17T09:58:56.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7ad800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a815db70 con 0x7fa7a806ba30 2024-04-17T09:58:56.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7ad800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa7a8146230 con 0x7fa7a80cf370 2024-04-17T09:58:56.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7a7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7a8160350 0x7fa7a80aa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:56.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7a7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7a8160350 0x7fa7a80aa0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34854/0 (socket says 172.21.15.32:34854) 2024-04-17T09:58:56.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa79e000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7a80cf370 0x7fa7a80a9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:56.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7a6a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7a806ba30 0x7fa7a813f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:56.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7a7400640 1 -- 172.21.15.32:0/683454783 learned_addr learned my addr 172.21.15.32:0/683454783 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:56.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7a7400640 1 -- 172.21.15.32:0/683454783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7a80cf370 msgr2=0x7fa7a80a9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:56.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7a7400640 1 --2- 172.21.15.32:0/683454783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7a80cf370 0x7fa7a80a9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:56.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7a7400640 1 -- 172.21.15.32:0/683454783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7a806ba30 msgr2=0x7fa7a813f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:56.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7a7400640 1 --2- 172.21.15.32:0/683454783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7a806ba30 0x7fa7a813f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:56.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.554+0000 7fa7a7400640 1 -- 172.21.15.32:0/683454783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a8141740 con 0x7fa7a8160350 2024-04-17T09:58:56.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.558+0000 7fa7a7400640 1 --2- 172.21.15.32:0/683454783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7a8160350 0x7fa7a80aa0f0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa7a000c400 tx=0x7fa7a000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:56.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.558+0000 7fa79f400640 1 -- 172.21.15.32:0/683454783 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0017070 con 0x7fa7a8160350 2024-04-17T09:58:56.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.558+0000 7fa79f400640 1 -- 172.21.15.32:0/683454783 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a0005cf0 con 0x7fa7a8160350 2024-04-17T09:58:56.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.558+0000 7fa7ad800640 1 -- 172.21.15.32:0/683454783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a8141a20 con 0x7fa7a8160350 2024-04-17T09:58:56.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.558+0000 7fa79f400640 1 -- 172.21.15.32:0/683454783 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a00072d0 con 0x7fa7a8160350 2024-04-17T09:58:56.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.558+0000 7fa7ad800640 1 -- 172.21.15.32:0/683454783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a8141fe0 con 0x7fa7a8160350 2024-04-17T09:58:56.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.558+0000 7fa7ad800640 1 -- 172.21.15.32:0/683454783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7a8068a90 con 0x7fa7a8160350 2024-04-17T09:58:56.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.558+0000 7fa79f400640 1 -- 172.21.15.32:0/683454783 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa7a001a070 con 0x7fa7a8160350 2024-04-17T09:58:56.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.558+0000 7fa79f400640 1 --2- 172.21.15.32:0/683454783 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa784068380 0x7fa78406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:56.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.558+0000 7fa79f400640 1 -- 172.21.15.32:0/683454783 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5029+0+0 (secure 0 0 0) 0x7fa7a0085970 con 0x7fa7a8160350 2024-04-17T09:58:56.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.558+0000 7fa7a6a00640 1 --2- 172.21.15.32:0/683454783 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa784068380 0x7fa78406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:56.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.562+0000 7fa7a6a00640 1 --2- 172.21.15.32:0/683454783 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa784068380 0x7fa78406a840 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa7980af000 tx=0x7fa7980a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:56.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.562+0000 7fa79f400640 1 -- 172.21.15.32:0/683454783 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa7a0055b70 con 0x7fa7a8160350 2024-04-17T09:58:56.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.734+0000 7fa7ad800640 1 -- 172.21.15.32:0/683454783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fa7a806c850 con 0x7fa7a8160350 2024-04-17T09:58:56.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.738+0000 7fa79f400640 1 -- 172.21.15.32:0/683454783 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7fa7a0059e70 con 0x7fa7a8160350 2024-04-17T09:58:56.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.742+0000 7fa7ad800640 1 -- 172.21.15.32:0/683454783 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa784068380 msgr2=0x7fa78406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:56.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.742+0000 7fa7ad800640 1 --2- 172.21.15.32:0/683454783 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa784068380 0x7fa78406a840 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa7980af000 tx=0x7fa7980a7070 comp rx=0 tx=0).stop 2024-04-17T09:58:56.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.742+0000 7fa7ad800640 1 -- 172.21.15.32:0/683454783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7a8160350 msgr2=0x7fa7a80aa0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:56.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.742+0000 7fa7ad800640 1 --2- 172.21.15.32:0/683454783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7a8160350 0x7fa7a80aa0f0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa7a000c400 tx=0x7fa7a000c8c0 comp rx=0 tx=0).stop 2024-04-17T09:58:56.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.742+0000 7fa7ad800640 1 -- 172.21.15.32:0/683454783 shutdown_connections 2024-04-17T09:58:56.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.742+0000 7fa7ad800640 1 --2- 172.21.15.32:0/683454783 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa784068380 0x7fa78406a840 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:56.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.742+0000 7fa7ad800640 1 --2- 172.21.15.32:0/683454783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7a80cf370 0x7fa7a80a9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:56.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.742+0000 7fa7ad800640 1 --2- 172.21.15.32:0/683454783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7a8160350 0x7fa7a80aa0f0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:56.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.742+0000 7fa7ad800640 1 --2- 172.21.15.32:0/683454783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7a806ba30 0x7fa7a813f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:56.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.742+0000 7fa7ad800640 1 -- 172.21.15.32:0/683454783 >> 172.21.15.32:0/683454783 conn(0x7fa7a80b2120 msgr2=0x7fa7a815ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:56.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.742+0000 7fa7ad800640 1 -- 172.21.15.32:0/683454783 shutdown_connections 2024-04-17T09:58:56.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.742+0000 7fa7ad800640 1 -- 172.21.15.32:0/683454783 wait complete. 2024-04-17T09:58:56.757 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-17T09:58:56.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.898+0000 7f1500c00640 1 Processor -- start 2024-04-17T09:58:56.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.902+0000 7f1500c00640 1 -- start start 2024-04-17T09:58:56.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.902+0000 7f1500c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14fc072500 0x7f14fc0cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:56.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.902+0000 7f1500c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14fc15d8d0 0x7f14fc0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:56.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.902+0000 7f1500c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14fc0cdac0 con 0x7f14fc145f30 2024-04-17T09:58:56.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.902+0000 7f1500c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f14fc06f400 con 0x7f14fc15d8d0 2024-04-17T09:58:56.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.902+0000 7f1500c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f14fc146310 con 0x7f14fc072500 2024-04-17T09:58:56.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.902+0000 7f14f9e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f14fc145f30 0x7f14fc071df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49126/0 (socket says 172.21.15.32:49126) 2024-04-17T09:58:56.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.902+0000 7f14f9e00640 1 -- 172.21.15.32:0/1892974405 learned_addr learned my addr 172.21.15.32:0/1892974405 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:58:56.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.902+0000 7f14fa800640 1 --2- 172.21.15.32:0/1892974405 >> v2:172.21.15.32:3300/0 conn(0x7f14fc15d8d0 0x7f14fc0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:56.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.902+0000 7f14f3e00640 1 --2- 172.21.15.32:0/1892974405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14fc072500 0x7f14fc0cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:56.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.902+0000 7f14fa800640 1 -- 172.21.15.32:0/1892974405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14fc072500 msgr2=0x7f14fc0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:56.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.902+0000 7f14f9400640 1 -- 172.21.15.32:0/1892974405 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2490400797 0 0) 0x7f14fc0cdac0 con 0x7f14fc145f30 2024-04-17T09:58:56.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.902+0000 7f14fa800640 1 --2- 172.21.15.32:0/1892974405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14fc072500 0x7f14fc0cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:56.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.902+0000 7f14fa800640 1 -- 172.21.15.32:0/1892974405 >> v1:172.21.15.32:6789/0 conn(0x7f14fc145f30 legacy=0x7f14fc071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:56.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.902+0000 7f14fa800640 1 -- 172.21.15.32:0/1892974405 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14fc15cdf0 con 0x7f14fc15d8d0 2024-04-17T09:58:56.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.902+0000 7f14f3e00640 1 --2- 172.21.15.32:0/1892974405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14fc072500 0x7f14fc0cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:58:56.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.906+0000 7f14fa800640 1 --2- 172.21.15.32:0/1892974405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14fc15d8d0 0x7f14fc0cd3b0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009d90 tx=0x7f14f401f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbcf3fa0977b3980 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:56.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.906+0000 7f14f9400640 1 -- 172.21.15.32:0/1892974405 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f40b4070 con 0x7f14fc15d8d0 2024-04-17T09:58:56.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.906+0000 7f14f9400640 1 -- 172.21.15.32:0/1892974405 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14f40af600 con 0x7f14fc15d8d0 2024-04-17T09:58:56.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.906+0000 7f14f9400640 1 -- 172.21.15.32:0/1892974405 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f40af9d0 con 0x7f14fc15d8d0 2024-04-17T09:58:56.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.906+0000 7f1500c00640 1 -- 172.21.15.32:0/1892974405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14fc15d8d0 msgr2=0x7f14fc0cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:56.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.906+0000 7f1500c00640 1 --2- 172.21.15.32:0/1892974405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14fc15d8d0 0x7f14fc0cd3b0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009d90 tx=0x7f14f401f150 comp rx=0 tx=0).stop 2024-04-17T09:58:56.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.906+0000 7f1500c00640 1 -- 172.21.15.32:0/1892974405 shutdown_connections 2024-04-17T09:58:56.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.906+0000 7f1500c00640 1 --2- 172.21.15.32:0/1892974405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14fc15d8d0 0x7f14fc0cd3b0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:56.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.906+0000 7f1500c00640 1 --2- 172.21.15.32:0/1892974405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14fc072500 0x7f14fc0cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:56.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.906+0000 7f1500c00640 1 -- 172.21.15.32:0/1892974405 >> 172.21.15.32:0/1892974405 conn(0x7f14fc06ba30 msgr2=0x7f14fc06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:56.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.906+0000 7f1500c00640 1 -- 172.21.15.32:0/1892974405 shutdown_connections 2024-04-17T09:58:56.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.906+0000 7f1500c00640 1 -- 172.21.15.32:0/1892974405 wait complete. 2024-04-17T09:58:56.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.906+0000 7f1500c00640 1 Processor -- start 2024-04-17T09:58:56.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.906+0000 7f1500c00640 1 -- start start 2024-04-17T09:58:56.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f1500c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14fc072500 0x7f14fc0a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:56.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f1500c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14fc145f30 0x7f14fc132fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:56.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f1500c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14fc15d8d0 0x7f14fc1e91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:56.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f1500c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc15cb90 con 0x7f14fc145f30 2024-04-17T09:58:56.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f1500c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc15d440 con 0x7f14fc15d8d0 2024-04-17T09:58:56.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f1500c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f14fc1601d0 con 0x7f14fc072500 2024-04-17T09:58:56.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f14f3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14fc145f30 0x7f14fc132fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:56.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f14f3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14fc145f30 0x7f14fc132fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34870/0 (socket says 172.21.15.32:34870) 2024-04-17T09:58:56.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f14f3e00640 1 -- 172.21.15.32:0/2248826296 learned_addr learned my addr 172.21.15.32:0/2248826296 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:56.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f14f9e00640 1 --2- 172.21.15.32:0/2248826296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14fc072500 0x7f14fc0a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:56.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f14fa800640 1 --2- 172.21.15.32:0/2248826296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14fc15d8d0 0x7f14fc1e91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:56.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f14f3e00640 1 -- 172.21.15.32:0/2248826296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14fc072500 msgr2=0x7f14fc0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:56.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f14f3e00640 1 --2- 172.21.15.32:0/2248826296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14fc072500 0x7f14fc0a9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:56.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f14f3e00640 1 -- 172.21.15.32:0/2248826296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14fc15d8d0 msgr2=0x7f14fc1e91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:56.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f14f3e00640 1 --2- 172.21.15.32:0/2248826296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14fc15d8d0 0x7f14fc1e91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:56.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f14f3e00640 1 -- 172.21.15.32:0/2248826296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14fc133a80 con 0x7f14fc145f30 2024-04-17T09:58:56.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f14f9e00640 1 --2- 172.21.15.32:0/2248826296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14fc072500 0x7f14fc0a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:58:56.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f14f3e00640 1 --2- 172.21.15.32:0/2248826296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14fc145f30 0x7f14fc132fe0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f14e400c7c0 tx=0x7f14e400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:56.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f14f2000640 1 -- 172.21.15.32:0/2248826296 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4007a30 con 0x7f14fc145f30 2024-04-17T09:58:56.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f14f2000640 1 -- 172.21.15.32:0/2248826296 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e4007bd0 con 0x7f14fc145f30 2024-04-17T09:58:56.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f14f2000640 1 -- 172.21.15.32:0/2248826296 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e40113f0 con 0x7f14fc145f30 2024-04-17T09:58:56.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f1500c00640 1 -- 172.21.15.32:0/2248826296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14fc133d60 con 0x7f14fc145f30 2024-04-17T09:58:56.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f1500c00640 1 -- 172.21.15.32:0/2248826296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14fc0aa1a0 con 0x7f14fc145f30 2024-04-17T09:58:56.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f1500c00640 1 -- 172.21.15.32:0/2248826296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14fc130f50 con 0x7f14fc145f30 2024-04-17T09:58:56.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.910+0000 7f14f2000640 1 -- 172.21.15.32:0/2248826296 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f14e4011590 con 0x7f14fc145f30 2024-04-17T09:58:56.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.914+0000 7f14f2000640 1 --2- 172.21.15.32:0/2248826296 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f14d80683b0 0x7f14d806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:56.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.914+0000 7f14f2000640 1 -- 172.21.15.32:0/2248826296 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5029+0+0 (secure 0 0 0) 0x7f14e4086800 con 0x7f14fc145f30 2024-04-17T09:58:56.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.914+0000 7f14f9e00640 1 --2- 172.21.15.32:0/2248826296 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f14d80683b0 0x7f14d806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:56.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.914+0000 7f14f9e00640 1 --2- 172.21.15.32:0/2248826296 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f14d80683b0 0x7f14d806a870 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f14ec007fd0 tx=0x7f14ec007c20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:56.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:56.918+0000 7f14f2000640 1 -- 172.21.15.32:0/2248826296 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f14e4015020 con 0x7f14fc145f30 2024-04-17T09:58:57.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.138+0000 7f1500c00640 1 -- 172.21.15.32:0/2248826296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f14fc068a90 con 0x7f14fc145f30 2024-04-17T09:58:57.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.378+0000 7f14f2000640 1 -- 172.21.15.32:0/2248826296 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v289) ==== 166+0+0 (secure 0 0 0) 0x7f14e4056a00 con 0x7f14fc145f30 2024-04-17T09:58:57.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.382+0000 7f1500c00640 1 -- 172.21.15.32:0/2248826296 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f14d80683b0 msgr2=0x7f14d806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:57.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.382+0000 7f1500c00640 1 --2- 172.21.15.32:0/2248826296 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f14d80683b0 0x7f14d806a870 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f14ec007fd0 tx=0x7f14ec007c20 comp rx=0 tx=0).stop 2024-04-17T09:58:57.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.382+0000 7f1500c00640 1 -- 172.21.15.32:0/2248826296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14fc145f30 msgr2=0x7f14fc132fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:57.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.382+0000 7f1500c00640 1 --2- 172.21.15.32:0/2248826296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14fc145f30 0x7f14fc132fe0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f14e400c7c0 tx=0x7f14e400cc80 comp rx=0 tx=0).stop 2024-04-17T09:58:57.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.382+0000 7f1500c00640 1 -- 172.21.15.32:0/2248826296 shutdown_connections 2024-04-17T09:58:57.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.382+0000 7f1500c00640 1 --2- 172.21.15.32:0/2248826296 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f14d80683b0 0x7f14d806a870 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:57.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.382+0000 7f1500c00640 1 --2- 172.21.15.32:0/2248826296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14fc15d8d0 0x7f14fc1e91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:57.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.382+0000 7f1500c00640 1 --2- 172.21.15.32:0/2248826296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14fc145f30 0x7f14fc132fe0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:57.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.382+0000 7f1500c00640 1 --2- 172.21.15.32:0/2248826296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14fc072500 0x7f14fc0a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:57.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.382+0000 7f1500c00640 1 -- 172.21.15.32:0/2248826296 >> 172.21.15.32:0/2248826296 conn(0x7f14fc06ba30 msgr2=0x7f14fc15eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:57.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.382+0000 7f1500c00640 1 -- 172.21.15.32:0/2248826296 shutdown_connections 2024-04-17T09:58:57.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.382+0000 7f1500c00640 1 -- 172.21.15.32:0/2248826296 wait complete. 2024-04-17T09:58:57.400 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-17T09:58:57.400 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-17T09:58:57.400 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-17T09:58:57.401 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:58:57.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.546+0000 7ff77c400640 1 Processor -- start 2024-04-17T09:58:57.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.546+0000 7ff77c400640 1 -- start start 2024-04-17T09:58:57.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.546+0000 7ff77c400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff774144e90 0x7ff7740b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:57.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.546+0000 7ff77c400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff7740b4de0 0x7ff77415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:57.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.546+0000 7ff77c400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff774072bb0 con 0x7ff77415bfe0 2024-04-17T09:58:57.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.546+0000 7ff77c400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff77415c430 con 0x7ff7740b4de0 2024-04-17T09:58:57.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.546+0000 7ff77c400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff774145270 con 0x7ff774144e90 2024-04-17T09:58:57.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.546+0000 7ff779000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff7740b4de0 0x7ff77415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:57.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.546+0000 7ff779000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff7740b4de0 0x7ff77415baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34880/0 (socket says 172.21.15.32:34880) 2024-04-17T09:58:57.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.546+0000 7ff779000640 1 -- 172.21.15.32:0/168730539 learned_addr learned my addr 172.21.15.32:0/168730539 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:57.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.546+0000 7ff779a00640 1 --2- 172.21.15.32:0/168730539 >> v2:172.21.15.144:3300/0 conn(0x7ff774144e90 0x7ff7740b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:57.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.546+0000 7ff779000640 1 -- 172.21.15.32:0/168730539 >> v2:172.21.15.144:3300/0 conn(0x7ff774144e90 msgr2=0x7ff7740b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:57.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.546+0000 7ff779000640 1 --2- 172.21.15.32:0/168730539 >> v2:172.21.15.144:3300/0 conn(0x7ff774144e90 0x7ff7740b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:57.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.546+0000 7ff779000640 1 -- 172.21.15.32:0/168730539 >> v1:172.21.15.144:6789/0 conn(0x7ff77415bfe0 legacy=0x7ff77406ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:57.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.546+0000 7ff779000640 1 -- 172.21.15.32:0/168730539 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7740b3dc0 con 0x7ff7740b4de0 2024-04-17T09:58:57.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.550+0000 7ff779000640 1 --2- 172.21.15.32:0/168730539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7740b4de0 0x7ff77415baa0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7ff77406ff30 tx=0x7ff76001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4f895de24b3b0d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:57.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.550+0000 7ff773e00640 1 -- 172.21.15.32:0/168730539 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7600aa650 con 0x7ff7740b4de0 2024-04-17T09:58:57.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.550+0000 7ff773e00640 1 -- 172.21.15.32:0/168730539 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7600af070 con 0x7ff7740b4de0 2024-04-17T09:58:57.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.550+0000 7ff773e00640 1 -- 172.21.15.32:0/168730539 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7600aaa10 con 0x7ff7740b4de0 2024-04-17T09:58:57.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.550+0000 7ff77c400640 1 -- 172.21.15.32:0/168730539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7740b4de0 msgr2=0x7ff77415baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:57.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.550+0000 7ff77c400640 1 --2- 172.21.15.32:0/168730539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7740b4de0 0x7ff77415baa0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7ff77406ff30 tx=0x7ff76001c4f0 comp rx=0 tx=0).stop 2024-04-17T09:58:57.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.550+0000 7ff77c400640 1 -- 172.21.15.32:0/168730539 shutdown_connections 2024-04-17T09:58:57.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.550+0000 7ff77c400640 1 --2- 172.21.15.32:0/168730539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7740b4de0 0x7ff77415baa0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:57.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.550+0000 7ff77c400640 1 --2- 172.21.15.32:0/168730539 >> v2:172.21.15.144:3300/0 conn(0x7ff774144e90 0x7ff7740b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:57.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.550+0000 7ff77c400640 1 -- 172.21.15.32:0/168730539 >> 172.21.15.32:0/168730539 conn(0x7ff77406ba30 msgr2=0x7ff77406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:57.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.550+0000 7ff77c400640 1 -- 172.21.15.32:0/168730539 shutdown_connections 2024-04-17T09:58:57.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.550+0000 7ff77c400640 1 -- 172.21.15.32:0/168730539 wait complete. 2024-04-17T09:58:57.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.550+0000 7ff77c400640 1 Processor -- start 2024-04-17T09:58:57.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff77c400640 1 -- start start 2024-04-17T09:58:57.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff77c400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7740b4de0 0x7ff7740cba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:57.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff77c400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff774144e90 0x7ff774136d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:57.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff77c400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff77415bfe0 0x7ff77413d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:57.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff77c400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff774072670 con 0x7ff774144e90 2024-04-17T09:58:57.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff77c400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff77406f880 con 0x7ff77415bfe0 2024-04-17T09:58:57.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff77c400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff7740727f0 con 0x7ff7740b4de0 2024-04-17T09:58:57.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff779000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff774144e90 0x7ff774136d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:57.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff779000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff774144e90 0x7ff774136d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34882/0 (socket says 172.21.15.32:34882) 2024-04-17T09:58:57.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff779000640 1 -- 172.21.15.32:0/3014795968 learned_addr learned my addr 172.21.15.32:0/3014795968 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:57.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff779a00640 1 --2- 172.21.15.32:0/3014795968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7740b4de0 0x7ff7740cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:57.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff77a400640 1 --2- 172.21.15.32:0/3014795968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff77415bfe0 0x7ff77413d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:57.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff779000640 1 -- 172.21.15.32:0/3014795968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7740b4de0 msgr2=0x7ff7740cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:57.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff779000640 1 --2- 172.21.15.32:0/3014795968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7740b4de0 0x7ff7740cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:57.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff779000640 1 -- 172.21.15.32:0/3014795968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff77415bfe0 msgr2=0x7ff77413d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:57.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff779000640 1 --2- 172.21.15.32:0/3014795968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff77415bfe0 0x7ff77413d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:57.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff779000640 1 -- 172.21.15.32:0/3014795968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff774138560 con 0x7ff774144e90 2024-04-17T09:58:57.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff77a400640 1 --2- 172.21.15.32:0/3014795968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff77415bfe0 0x7ff77413d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:58:57.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff779000640 1 --2- 172.21.15.32:0/3014795968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff774144e90 0x7ff774136d20 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7ff760019100 tx=0x7ff76001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:57.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff771600640 1 -- 172.21.15.32:0/3014795968 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7600bd5f0 con 0x7ff774144e90 2024-04-17T09:58:57.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff771600640 1 -- 172.21.15.32:0/3014795968 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7600a6040 con 0x7ff774144e90 2024-04-17T09:58:57.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff771600640 1 -- 172.21.15.32:0/3014795968 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7600bd9d0 con 0x7ff774144e90 2024-04-17T09:58:57.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff77c400640 1 -- 172.21.15.32:0/3014795968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff774138840 con 0x7ff774144e90 2024-04-17T09:58:57.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff77c400640 1 -- 172.21.15.32:0/3014795968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff774138e00 con 0x7ff774144e90 2024-04-17T09:58:57.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.554+0000 7ff77c400640 1 -- 172.21.15.32:0/3014795968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff774068a90 con 0x7ff774144e90 2024-04-17T09:58:57.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.558+0000 7ff771600640 1 -- 172.21.15.32:0/3014795968 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff7600b33f0 con 0x7ff774144e90 2024-04-17T09:58:57.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.558+0000 7ff771600640 1 --2- 172.21.15.32:0/3014795968 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff754068380 0x7ff75406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:57.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.558+0000 7ff771600640 1 -- 172.21.15.32:0/3014795968 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5029+0+0 (secure 0 0 0) 0x7ff760120950 con 0x7ff774144e90 2024-04-17T09:58:57.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.558+0000 7ff779a00640 1 --2- 172.21.15.32:0/3014795968 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff754068380 0x7ff75406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:57.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.558+0000 7ff779a00640 1 --2- 172.21.15.32:0/3014795968 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff754068380 0x7ff75406a840 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff768007a60 tx=0x7ff768008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:57.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.562+0000 7ff771600640 1 -- 172.21.15.32:0/3014795968 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff760016110 con 0x7ff774144e90 2024-04-17T09:58:57.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.730+0000 7ff77c400640 1 -- 172.21.15.32:0/3014795968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7740cff40 con 0x7ff774144e90 2024-04-17T09:58:57.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.730+0000 7ff771600640 1 -- 172.21.15.32:0/3014795968 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+13161 (secure 0 0 0) 0x7ff7600f0ac0 con 0x7ff774144e90 2024-04-17T09:58:57.729 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:58:57.729 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":74,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:58:56.367677+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6834/2416784965":"2024-04-18T09:58:53.356467+0000","172.21.15.32:6836/663537244":"2024-04-18T09:57:30.746884+0000","172.21.15.32:6836/2416784965":"2024-04-18T09:58:53.356467+0000","172.21.15.32:6834/663537244":"2024-04-18T09:57:30.746884+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:58:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.734+0000 7ff77c400640 1 -- 172.21.15.32:0/3014795968 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff754068380 msgr2=0x7ff75406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.734+0000 7ff77c400640 1 --2- 172.21.15.32:0/3014795968 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff754068380 0x7ff75406a840 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff768007a60 tx=0x7ff768008040 comp rx=0 tx=0).stop 2024-04-17T09:58:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.734+0000 7ff77c400640 1 -- 172.21.15.32:0/3014795968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff774144e90 msgr2=0x7ff774136d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.734+0000 7ff77c400640 1 --2- 172.21.15.32:0/3014795968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff774144e90 0x7ff774136d20 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7ff760019100 tx=0x7ff76001ad30 comp rx=0 tx=0).stop 2024-04-17T09:58:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.734+0000 7ff77c400640 1 -- 172.21.15.32:0/3014795968 shutdown_connections 2024-04-17T09:58:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.734+0000 7ff77c400640 1 --2- 172.21.15.32:0/3014795968 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff754068380 0x7ff75406a840 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.734+0000 7ff77c400640 1 --2- 172.21.15.32:0/3014795968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff77415bfe0 0x7ff77413d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:57.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.734+0000 7ff77c400640 1 --2- 172.21.15.32:0/3014795968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff774144e90 0x7ff774136d20 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:57.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.734+0000 7ff77c400640 1 --2- 172.21.15.32:0/3014795968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7740b4de0 0x7ff7740cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:57.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.734+0000 7ff77c400640 1 -- 172.21.15.32:0/3014795968 >> 172.21.15.32:0/3014795968 conn(0x7ff77406ba30 msgr2=0x7ff7740cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:57.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.734+0000 7ff77c400640 1 -- 172.21.15.32:0/3014795968 shutdown_connections 2024-04-17T09:58:57.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.734+0000 7ff77c400640 1 -- 172.21.15.32:0/3014795968 wait complete. 2024-04-17T09:58:57.751 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:58:57.752 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:58:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.898+0000 7fb771600640 1 Processor -- start 2024-04-17T09:58:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.898+0000 7fb771600640 1 -- start start 2024-04-17T09:58:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.898+0000 7fb771600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb76c06f7c0 0x7fb76c06ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.898+0000 7fb771600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb76c06f270 0x7fb76c06bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.898+0000 7fb771600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb76c0cc940 con 0x7fb76c15b7d0 2024-04-17T09:58:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.898+0000 7fb771600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb76c0ccc20 con 0x7fb76c06f7c0 2024-04-17T09:58:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.898+0000 7fb771600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb76c06e0b0 con 0x7fb76c06f270 2024-04-17T09:58:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.898+0000 7fb76a000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb76c06f7c0 0x7fb76c06ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.898+0000 7fb76a000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb76c06f7c0 0x7fb76c06ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34884/0 (socket says 172.21.15.32:34884) 2024-04-17T09:58:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.898+0000 7fb76a000640 1 -- 172.21.15.32:0/780768856 learned_addr learned my addr 172.21.15.32:0/780768856 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:57.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.898+0000 7fb76b400640 1 --2- 172.21.15.32:0/780768856 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb76c06f270 0x7fb76c06bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:57.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.898+0000 7fb76a000640 1 -- 172.21.15.32:0/780768856 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb76c06f270 msgr2=0x7fb76c06bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:58:57.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.898+0000 7fb76a000640 1 --2- 172.21.15.32:0/780768856 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb76c06f270 0x7fb76c06bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:57.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.898+0000 7fb76a000640 1 -- 172.21.15.32:0/780768856 >> v1:172.21.15.144:6789/0 conn(0x7fb76c15b7d0 legacy=0x7fb76c06e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:57.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.898+0000 7fb76a000640 1 -- 172.21.15.32:0/780768856 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76c06e390 con 0x7fb76c06f7c0 2024-04-17T09:58:57.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.898+0000 7fb76b400640 1 --2- 172.21.15.32:0/780768856 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb76c06f270 0x7fb76c06bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:58:57.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.902+0000 7fb76a000640 1 --2- 172.21.15.32:0/780768856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb76c06f7c0 0x7fb76c06ba30 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fb758009870 tx=0x7fb75801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e230787b2fad7eb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:57.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.902+0000 7fb769600640 1 -- 172.21.15.32:0/780768856 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7580aa650 con 0x7fb76c06f7c0 2024-04-17T09:58:57.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.902+0000 7fb769600640 1 -- 172.21.15.32:0/780768856 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7580af070 con 0x7fb76c06f7c0 2024-04-17T09:58:57.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.902+0000 7fb769600640 1 -- 172.21.15.32:0/780768856 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7580aaa10 con 0x7fb76c06f7c0 2024-04-17T09:58:57.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.902+0000 7fb771600640 1 -- 172.21.15.32:0/780768856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb76c06f7c0 msgr2=0x7fb76c06ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:57.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.902+0000 7fb771600640 1 --2- 172.21.15.32:0/780768856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb76c06f7c0 0x7fb76c06ba30 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fb758009870 tx=0x7fb75801c4d0 comp rx=0 tx=0).stop 2024-04-17T09:58:57.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.902+0000 7fb771600640 1 -- 172.21.15.32:0/780768856 shutdown_connections 2024-04-17T09:58:57.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.902+0000 7fb771600640 1 --2- 172.21.15.32:0/780768856 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb76c06f270 0x7fb76c06bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:57.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.902+0000 7fb771600640 1 --2- 172.21.15.32:0/780768856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb76c06f7c0 0x7fb76c06ba30 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:57.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.902+0000 7fb771600640 1 -- 172.21.15.32:0/780768856 >> 172.21.15.32:0/780768856 conn(0x7fb76c0b2120 msgr2=0x7fb76c0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:57.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.902+0000 7fb771600640 1 -- 172.21.15.32:0/780768856 shutdown_connections 2024-04-17T09:58:57.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.902+0000 7fb771600640 1 -- 172.21.15.32:0/780768856 wait complete. 2024-04-17T09:58:57.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb771600640 1 Processor -- start 2024-04-17T09:58:57.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb771600640 1 -- start start 2024-04-17T09:58:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb771600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb76c06f270 0x7fb76c0b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb771600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb76c06f7c0 0x7fb76c151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb771600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb76c15b7d0 0x7fb76c148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb771600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c0ceeb0 con 0x7fb76c06f7c0 2024-04-17T09:58:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb771600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c0ced30 con 0x7fb76c06f270 2024-04-17T09:58:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb771600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb76c1682a0 con 0x7fb76c15b7d0 2024-04-17T09:58:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb76a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb76c06f7c0 0x7fb76c151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb76a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb76c06f7c0 0x7fb76c151400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34890/0 (socket says 172.21.15.32:34890) 2024-04-17T09:58:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb76a000640 1 -- 172.21.15.32:0/2667437078 learned_addr learned my addr 172.21.15.32:0/2667437078 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb76aa00640 1 --2- 172.21.15.32:0/2667437078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb76c06f270 0x7fb76c0b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb76a000640 1 -- 172.21.15.32:0/2667437078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb76c15b7d0 msgr2=0x7fb76c148e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb76a000640 1 --2- 172.21.15.32:0/2667437078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb76c15b7d0 0x7fb76c148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb76a000640 1 -- 172.21.15.32:0/2667437078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb76c06f270 msgr2=0x7fb76c0b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb76a000640 1 --2- 172.21.15.32:0/2667437078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb76c06f270 0x7fb76c0b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb76a000640 1 -- 172.21.15.32:0/2667437078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76c14e190 con 0x7fb76c06f7c0 2024-04-17T09:58:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb76a000640 1 --2- 172.21.15.32:0/2667437078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb76c06f7c0 0x7fb76c151400 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fb75801a100 tx=0x7fb75801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb762a00640 1 -- 172.21.15.32:0/2667437078 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7580b3800 con 0x7fb76c06f7c0 2024-04-17T09:58:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb762a00640 1 -- 172.21.15.32:0/2667437078 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7580a6040 con 0x7fb76c06f7c0 2024-04-17T09:58:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb762a00640 1 -- 172.21.15.32:0/2667437078 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7580b3be0 con 0x7fb76c06f7c0 2024-04-17T09:58:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb771600640 1 -- 172.21.15.32:0/2667437078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb76c0af170 con 0x7fb76c06f7c0 2024-04-17T09:58:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb771600640 1 -- 172.21.15.32:0/2667437078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb76c14f820 con 0x7fb76c06f7c0 2024-04-17T09:58:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.906+0000 7fb771600640 1 -- 172.21.15.32:0/2667437078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb76c068a90 con 0x7fb76c06f7c0 2024-04-17T09:58:57.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.910+0000 7fb762a00640 1 -- 172.21.15.32:0/2667437078 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb7580b2380 con 0x7fb76c06f7c0 2024-04-17T09:58:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.910+0000 7fb762a00640 1 --2- 172.21.15.32:0/2667437078 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb74c0683b0 0x7fb74c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.910+0000 7fb762a00640 1 -- 172.21.15.32:0/2667437078 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5029+0+0 (secure 0 0 0) 0x7fb758120a10 con 0x7fb76c06f7c0 2024-04-17T09:58:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.910+0000 7fb76aa00640 1 --2- 172.21.15.32:0/2667437078 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb74c0683b0 0x7fb74c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.910+0000 7fb76aa00640 1 --2- 172.21.15.32:0/2667437078 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb74c0683b0 0x7fb74c06a870 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb75c001810 tx=0x7fb75c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:57.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:57.914+0000 7fb762a00640 1 -- 172.21.15.32:0/2667437078 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fb76c14e190 con 0x7fb76c06f7c0 2024-04-17T09:58:58.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.078+0000 7fb771600640 1 -- 172.21.15.32:0/2667437078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb76c14f9f0 con 0x7fb76c06f7c0 2024-04-17T09:58:58.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.078+0000 7fb762a00640 1 -- 172.21.15.32:0/2667437078 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7fb758016150 con 0x7fb76c06f7c0 2024-04-17T09:58:58.076 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:58:58.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.078+0000 7fb771600640 1 -- 172.21.15.32:0/2667437078 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb74c0683b0 msgr2=0x7fb74c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:58.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.078+0000 7fb771600640 1 --2- 172.21.15.32:0/2667437078 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb74c0683b0 0x7fb74c06a870 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb75c001810 tx=0x7fb75c009040 comp rx=0 tx=0).stop 2024-04-17T09:58:58.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.078+0000 7fb771600640 1 -- 172.21.15.32:0/2667437078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb76c06f7c0 msgr2=0x7fb76c151400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:58.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.078+0000 7fb771600640 1 --2- 172.21.15.32:0/2667437078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb76c06f7c0 0x7fb76c151400 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fb75801a100 tx=0x7fb75801ad10 comp rx=0 tx=0).stop 2024-04-17T09:58:58.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.078+0000 7fb771600640 1 -- 172.21.15.32:0/2667437078 shutdown_connections 2024-04-17T09:58:58.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.078+0000 7fb771600640 1 --2- 172.21.15.32:0/2667437078 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb74c0683b0 0x7fb74c06a870 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:58.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.078+0000 7fb771600640 1 --2- 172.21.15.32:0/2667437078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb76c15b7d0 0x7fb76c148e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:58.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.078+0000 7fb771600640 1 --2- 172.21.15.32:0/2667437078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb76c06f7c0 0x7fb76c151400 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:58.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.078+0000 7fb771600640 1 --2- 172.21.15.32:0/2667437078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb76c06f270 0x7fb76c0b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:58.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.082+0000 7fb771600640 1 -- 172.21.15.32:0/2667437078 >> 172.21.15.32:0/2667437078 conn(0x7fb76c0b2120 msgr2=0x7fb76c156040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:58.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.082+0000 7fb771600640 1 -- 172.21.15.32:0/2667437078 shutdown_connections 2024-04-17T09:58:58.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.082+0000 7fb771600640 1 -- 172.21.15.32:0/2667437078 wait complete. 2024-04-17T09:58:58.097 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-17T09:58:58.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.238+0000 7f704e400640 1 Processor -- start 2024-04-17T09:58:58.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.242+0000 7f704e400640 1 -- start start 2024-04-17T09:58:58.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.242+0000 7f704e400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7048145f30 0x7f7048071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:58.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.242+0000 7f704e400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70480cdac0 con 0x7f704815d8d0 2024-04-17T09:58:58.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.242+0000 7f704e400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f704815cdf0 con 0x7f70480cd580 2024-04-17T09:58:58.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.242+0000 7f704e400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f704806f400 con 0x7f7048145f30 2024-04-17T09:58:58.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.242+0000 7f703ea00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f704815d8d0 0x7f70480cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49128/0 (socket says 172.21.15.32:49128) 2024-04-17T09:58:58.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.242+0000 7f703ea00640 1 -- 172.21.15.32:0/53378302 learned_addr learned my addr 172.21.15.32:0/53378302 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:58:58.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.242+0000 7f7047400640 1 --2- 172.21.15.32:0/53378302 >> v2:172.21.15.144:3300/0 conn(0x7f7048145f30 0x7f7048071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:58.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.242+0000 7f7046a00640 1 -- 172.21.15.32:0/53378302 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1429410991 0 0) 0x7f70480cdac0 con 0x7f704815d8d0 2024-04-17T09:58:58.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.242+0000 7f7046a00640 1 -- 172.21.15.32:0/53378302 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f702c003530 con 0x7f704815d8d0 2024-04-17T09:58:58.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.242+0000 7f7047400640 1 -- 172.21.15.32:0/53378302 >> v1:172.21.15.144:6789/0 conn(0x7f70480cd580 legacy=0x7f70481601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:58.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.242+0000 7f7047400640 1 -- 172.21.15.32:0/53378302 >> v1:172.21.15.32:6789/0 conn(0x7f704815d8d0 legacy=0x7f70480cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:58.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.242+0000 7f7047400640 1 -- 172.21.15.32:0/53378302 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704815d0d0 con 0x7f7048145f30 2024-04-17T09:58:58.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.246+0000 7f7047400640 1 --2- 172.21.15.32:0/53378302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7048145f30 0x7f7048071df0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f70300097c0 tx=0x7f703001ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=705ce5fc048c2a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:58.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.246+0000 7f7046a00640 1 -- 172.21.15.32:0/53378302 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70300b5070 con 0x7f7048145f30 2024-04-17T09:58:58.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.246+0000 7f7046a00640 1 -- 172.21.15.32:0/53378302 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70300a8990 con 0x7f7048145f30 2024-04-17T09:58:58.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.246+0000 7f7046a00640 1 -- 172.21.15.32:0/53378302 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70300a8d60 con 0x7f7048145f30 2024-04-17T09:58:58.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.246+0000 7f704e400640 1 -- 172.21.15.32:0/53378302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7048145f30 msgr2=0x7f7048071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:58.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.246+0000 7f704e400640 1 --2- 172.21.15.32:0/53378302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7048145f30 0x7f7048071df0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f70300097c0 tx=0x7f703001ec90 comp rx=0 tx=0).stop 2024-04-17T09:58:58.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.246+0000 7f704e400640 1 -- 172.21.15.32:0/53378302 shutdown_connections 2024-04-17T09:58:58.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.246+0000 7f704e400640 1 --2- 172.21.15.32:0/53378302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7048145f30 0x7f7048071df0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:58.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.246+0000 7f704e400640 1 -- 172.21.15.32:0/53378302 >> 172.21.15.32:0/53378302 conn(0x7f704806ba30 msgr2=0x7f704806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:58.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.246+0000 7f704e400640 1 -- 172.21.15.32:0/53378302 shutdown_connections 2024-04-17T09:58:58.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.246+0000 7f704e400640 1 -- 172.21.15.32:0/53378302 wait complete. 2024-04-17T09:58:58.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.246+0000 7f704e400640 1 Processor -- start 2024-04-17T09:58:58.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f704e400640 1 -- start start 2024-04-17T09:58:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f704e400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70480cd580 0x7f70480c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f704e400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7048145f30 0x7f70480c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f704e400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f704815d8d0 0x7f7048140750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f704e400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7048146310 con 0x7f70480cd580 2024-04-17T09:58:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f704e400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7048072600 con 0x7f704815d8d0 2024-04-17T09:58:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f704e400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7048072780 con 0x7f7048145f30 2024-04-17T09:58:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f7047400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70480cd580 0x7f70480c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f7047400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70480cd580 0x7f70480c7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34898/0 (socket says 172.21.15.32:34898) 2024-04-17T09:58:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f7047400640 1 -- 172.21.15.32:0/2846121215 learned_addr learned my addr 172.21.15.32:0/2846121215 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f703ea00640 1 --2- 172.21.15.32:0/2846121215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7048145f30 0x7f70480c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f7047e00640 1 --2- 172.21.15.32:0/2846121215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f704815d8d0 0x7f7048140750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f7047400640 1 -- 172.21.15.32:0/2846121215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7048145f30 msgr2=0x7f70480c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f7047400640 1 --2- 172.21.15.32:0/2846121215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7048145f30 0x7f70480c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f7047400640 1 -- 172.21.15.32:0/2846121215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f704815d8d0 msgr2=0x7f7048140750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f7047400640 1 --2- 172.21.15.32:0/2846121215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f704815d8d0 0x7f7048140750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f7047400640 1 -- 172.21.15.32:0/2846121215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70480ca210 con 0x7f70480cd580 2024-04-17T09:58:58.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f7047400640 1 --2- 172.21.15.32:0/2846121215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70480cd580 0x7f70480c7cc0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f70300b3040 tx=0x7f703001ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:58.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f703fe00640 1 -- 172.21.15.32:0/2846121215 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70300bf070 con 0x7f70480cd580 2024-04-17T09:58:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f703fe00640 1 -- 172.21.15.32:0/2846121215 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70300b07a0 con 0x7f70480cd580 2024-04-17T09:58:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f703fe00640 1 -- 172.21.15.32:0/2846121215 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70300b5040 con 0x7f70480cd580 2024-04-17T09:58:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f704e400640 1 -- 172.21.15.32:0/2846121215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70480ca4e0 con 0x7f70480cd580 2024-04-17T09:58:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f704e400640 1 -- 172.21.15.32:0/2846121215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70480caaa0 con 0x7f70480cd580 2024-04-17T09:58:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.250+0000 7f704e400640 1 -- 172.21.15.32:0/2846121215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7048068a90 con 0x7f70480cd580 2024-04-17T09:58:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.254+0000 7f703fe00640 1 -- 172.21.15.32:0/2846121215 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f70300b0b90 con 0x7f70480cd580 2024-04-17T09:58:58.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.254+0000 7f703fe00640 1 --2- 172.21.15.32:0/2846121215 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7024068380 0x7f702406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:58.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.254+0000 7f703fe00640 1 -- 172.21.15.32:0/2846121215 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5029+0+0 (secure 0 0 0) 0x7f7030122f80 con 0x7f70480cd580 2024-04-17T09:58:58.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.254+0000 7f703ea00640 1 --2- 172.21.15.32:0/2846121215 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7024068380 0x7f702406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:58.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.254+0000 7f703ea00640 1 --2- 172.21.15.32:0/2846121215 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7024068380 0x7f702406a840 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f70480c8310 tx=0x7f7038009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:58.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.258+0000 7f703fe00640 1 -- 172.21.15.32:0/2846121215 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f70300187f0 con 0x7f70480cd580 2024-04-17T09:58:58.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:58.482+0000 7f704e400640 1 -- 172.21.15.32:0/2846121215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f70480b1d00 con 0x7f70480cd580 2024-04-17T09:58:59.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.386+0000 7f703fe00640 1 -- 172.21.15.32:0/2846121215 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v291) ==== 170+0+0 (secure 0 0 0) 0x7f70300f37d0 con 0x7f70480cd580 2024-04-17T09:58:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.390+0000 7f704e400640 1 -- 172.21.15.32:0/2846121215 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7024068380 msgr2=0x7f702406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.390+0000 7f704e400640 1 --2- 172.21.15.32:0/2846121215 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7024068380 0x7f702406a840 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f70480c8310 tx=0x7f7038009040 comp rx=0 tx=0).stop 2024-04-17T09:58:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.390+0000 7f704e400640 1 -- 172.21.15.32:0/2846121215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70480cd580 msgr2=0x7f70480c7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.390+0000 7f704e400640 1 --2- 172.21.15.32:0/2846121215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70480cd580 0x7f70480c7cc0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f70300b3040 tx=0x7f703001ba30 comp rx=0 tx=0).stop 2024-04-17T09:58:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.390+0000 7f704e400640 1 -- 172.21.15.32:0/2846121215 shutdown_connections 2024-04-17T09:58:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.390+0000 7f704e400640 1 --2- 172.21.15.32:0/2846121215 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7024068380 0x7f702406a840 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.390+0000 7f704e400640 1 --2- 172.21.15.32:0/2846121215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7048145f30 0x7f70480c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.390+0000 7f704e400640 1 --2- 172.21.15.32:0/2846121215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f704815d8d0 0x7f7048140750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.390+0000 7f704e400640 1 --2- 172.21.15.32:0/2846121215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70480cd580 0x7f70480c7cc0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.390+0000 7f704e400640 1 -- 172.21.15.32:0/2846121215 >> 172.21.15.32:0/2846121215 conn(0x7f704806ba30 msgr2=0x7f70480cce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.390+0000 7f704e400640 1 -- 172.21.15.32:0/2846121215 shutdown_connections 2024-04-17T09:58:59.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.390+0000 7f704e400640 1 -- 172.21.15.32:0/2846121215 wait complete. 2024-04-17T09:58:59.415 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-17T09:58:59.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.566+0000 7f8021600640 1 Processor -- start 2024-04-17T09:58:59.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.566+0000 7f8021600640 1 -- start start 2024-04-17T09:58:59.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.570+0000 7f8021600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f801c1656d0 0x7f801c167ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:59.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.570+0000 7f8021600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f801c0d0130 con 0x7f801c06ba30 2024-04-17T09:58:59.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.570+0000 7f8021600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f801c15d090 con 0x7f801c0cf370 2024-04-17T09:58:59.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.570+0000 7f8021600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f801c0cfef0 con 0x7f801c1656d0 2024-04-17T09:58:59.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.570+0000 7f801aa00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f801c06ba30 0x7f801c06be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49142/0 (socket says 172.21.15.32:49142) 2024-04-17T09:58:59.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.570+0000 7f801aa00640 1 -- 172.21.15.32:0/4135089870 learned_addr learned my addr 172.21.15.32:0/4135089870 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:58:59.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.570+0000 7f801b400640 1 --2- 172.21.15.32:0/4135089870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f801c1656d0 0x7f801c167ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:59.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.570+0000 7f801a000640 1 -- 172.21.15.32:0/4135089870 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1122947535 0 0) 0x7f801c0d0130 con 0x7f801c06ba30 2024-04-17T09:58:59.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.570+0000 7f801a000640 1 -- 172.21.15.32:0/4135089870 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8000003530 con 0x7f801c06ba30 2024-04-17T09:58:59.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.570+0000 7f801a000640 1 -- 172.21.15.32:0/4135089870 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2621317099 0 0) 0x7f801c15d090 con 0x7f801c0cf370 2024-04-17T09:58:59.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.570+0000 7f801b400640 1 -- 172.21.15.32:0/4135089870 >> v1:172.21.15.144:6789/0 conn(0x7f801c0cf370 legacy=0x7f801c0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:59.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.570+0000 7f801b400640 1 -- 172.21.15.32:0/4135089870 >> v1:172.21.15.32:6789/0 conn(0x7f801c06ba30 legacy=0x7f801c06be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:59.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.570+0000 7f801b400640 1 -- 172.21.15.32:0/4135089870 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801c15d370 con 0x7f801c1656d0 2024-04-17T09:58:59.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.574+0000 7f801b400640 1 --2- 172.21.15.32:0/4135089870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f801c1656d0 0x7f801c167ac0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8014009dc0 tx=0x7f801401c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7a8cc4217542e5e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:59.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.574+0000 7f801a000640 1 -- 172.21.15.32:0/4135089870 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80140b2070 con 0x7f801c1656d0 2024-04-17T09:58:59.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.574+0000 7f801a000640 1 -- 172.21.15.32:0/4135089870 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80140a5960 con 0x7f801c1656d0 2024-04-17T09:58:59.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.574+0000 7f801a000640 1 -- 172.21.15.32:0/4135089870 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80140a5d10 con 0x7f801c1656d0 2024-04-17T09:58:59.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.574+0000 7f8021600640 1 -- 172.21.15.32:0/4135089870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f801c1656d0 msgr2=0x7f801c167ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:59.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.574+0000 7f8021600640 1 --2- 172.21.15.32:0/4135089870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f801c1656d0 0x7f801c167ac0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8014009dc0 tx=0x7f801401c990 comp rx=0 tx=0).stop 2024-04-17T09:58:59.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.574+0000 7f8021600640 1 -- 172.21.15.32:0/4135089870 shutdown_connections 2024-04-17T09:58:59.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.574+0000 7f8021600640 1 --2- 172.21.15.32:0/4135089870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f801c1656d0 0x7f801c167ac0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:59.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.574+0000 7f8021600640 1 -- 172.21.15.32:0/4135089870 >> 172.21.15.32:0/4135089870 conn(0x7f801c0b2120 msgr2=0x7f801c0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:59.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.574+0000 7f8021600640 1 -- 172.21.15.32:0/4135089870 shutdown_connections 2024-04-17T09:58:59.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.574+0000 7f8021600640 1 -- 172.21.15.32:0/4135089870 wait complete. 2024-04-17T09:58:59.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.574+0000 7f8021600640 1 Processor -- start 2024-04-17T09:58:59.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.574+0000 7f8021600640 1 -- start start 2024-04-17T09:58:59.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f8021600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f801c06ba30 0x7f801c13dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:59.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f8021600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f801c0cf370 0x7f801c0b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:59.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f8021600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f801c1656d0 0x7f801c0b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:59.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f8021600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f801c146230 con 0x7f801c0cf370 2024-04-17T09:58:59.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f8021600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f801c15dd80 con 0x7f801c06ba30 2024-04-17T09:58:59.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f8021600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f801c1463b0 con 0x7f801c1656d0 2024-04-17T09:58:59.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f8012000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f801c0cf370 0x7f801c0b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:59.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f8012000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f801c0cf370 0x7f801c0b44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34902/0 (socket says 172.21.15.32:34902) 2024-04-17T09:58:59.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f8012000640 1 -- 172.21.15.32:0/3230560102 learned_addr learned my addr 172.21.15.32:0/3230560102 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:58:59.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f801b400640 1 --2- 172.21.15.32:0/3230560102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f801c1656d0 0x7f801c0b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:59.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f801aa00640 1 --2- 172.21.15.32:0/3230560102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f801c06ba30 0x7f801c13dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:59.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f8012000640 1 -- 172.21.15.32:0/3230560102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f801c1656d0 msgr2=0x7f801c0b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:59.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f8012000640 1 --2- 172.21.15.32:0/3230560102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f801c1656d0 0x7f801c0b4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:59.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f8012000640 1 -- 172.21.15.32:0/3230560102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f801c06ba30 msgr2=0x7f801c13dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:59.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f8012000640 1 --2- 172.21.15.32:0/3230560102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f801c06ba30 0x7f801c13dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:59.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f8012000640 1 -- 172.21.15.32:0/3230560102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801c13fe10 con 0x7f801c0cf370 2024-04-17T09:58:59.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f801aa00640 1 --2- 172.21.15.32:0/3230560102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f801c06ba30 0x7f801c13dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:58:59.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f8012000640 1 --2- 172.21.15.32:0/3230560102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f801c0cf370 0x7f801c0b44c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f800400c500 tx=0x7f800400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:59.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f8013400640 1 -- 172.21.15.32:0/3230560102 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004017070 con 0x7f801c0cf370 2024-04-17T09:58:59.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f8013400640 1 -- 172.21.15.32:0/3230560102 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8004003600 con 0x7f801c0cf370 2024-04-17T09:58:59.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f8013400640 1 -- 172.21.15.32:0/3230560102 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80040039f0 con 0x7f801c0cf370 2024-04-17T09:58:59.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f8021600640 1 -- 172.21.15.32:0/3230560102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801c1400f0 con 0x7f801c0cf370 2024-04-17T09:58:59.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f8021600640 1 -- 172.21.15.32:0/3230560102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f801c13e4f0 con 0x7f801c0cf370 2024-04-17T09:58:59.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f8021600640 1 -- 172.21.15.32:0/3230560102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f801c141630 con 0x7f801c0cf370 2024-04-17T09:58:59.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.578+0000 7f8013400640 1 -- 172.21.15.32:0/3230560102 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f800401a070 con 0x7f801c0cf370 2024-04-17T09:58:59.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.582+0000 7f8013400640 1 --2- 172.21.15.32:0/3230560102 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7ff8068380 0x7f7ff806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:58:59.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.582+0000 7f8013400640 1 -- 172.21.15.32:0/3230560102 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5029+0+0 (secure 0 0 0) 0x7f8004085b00 con 0x7f801c0cf370 2024-04-17T09:58:59.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.582+0000 7f801aa00640 1 --2- 172.21.15.32:0/3230560102 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7ff8068380 0x7f7ff806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:58:59.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.582+0000 7f801aa00640 1 --2- 172.21.15.32:0/3230560102 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7ff8068380 0x7f7ff806a840 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f800c005890 tx=0x7f800c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:58:59.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.586+0000 7f8013400640 1 -- 172.21.15.32:0/3230560102 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f8004055dc0 con 0x7f801c0cf370 2024-04-17T09:58:59.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.750+0000 7f8021600640 1 -- 172.21.15.32:0/3230560102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f801c068a90 con 0x7f801c0cf370 2024-04-17T09:58:59.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.754+0000 7f8013400640 1 -- 172.21.15.32:0/3230560102 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f800405a0c0 con 0x7f801c0cf370 2024-04-17T09:58:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.754+0000 7f8021600640 1 -- 172.21.15.32:0/3230560102 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7ff8068380 msgr2=0x7f7ff806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.754+0000 7f8021600640 1 --2- 172.21.15.32:0/3230560102 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7ff8068380 0x7f7ff806a840 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f800c005890 tx=0x7f800c009040 comp rx=0 tx=0).stop 2024-04-17T09:58:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.754+0000 7f8021600640 1 -- 172.21.15.32:0/3230560102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f801c0cf370 msgr2=0x7f801c0b44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:58:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.754+0000 7f8021600640 1 --2- 172.21.15.32:0/3230560102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f801c0cf370 0x7f801c0b44c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f800400c500 tx=0x7f800400c9c0 comp rx=0 tx=0).stop 2024-04-17T09:58:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.754+0000 7f8021600640 1 -- 172.21.15.32:0/3230560102 shutdown_connections 2024-04-17T09:58:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.754+0000 7f8021600640 1 --2- 172.21.15.32:0/3230560102 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7ff8068380 0x7f7ff806a840 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.754+0000 7f8021600640 1 --2- 172.21.15.32:0/3230560102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f801c1656d0 0x7f801c0b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.754+0000 7f8021600640 1 --2- 172.21.15.32:0/3230560102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f801c0cf370 0x7f801c0b44c0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.754+0000 7f8021600640 1 --2- 172.21.15.32:0/3230560102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f801c06ba30 0x7f801c13dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:58:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.754+0000 7f8021600640 1 -- 172.21.15.32:0/3230560102 >> 172.21.15.32:0/3230560102 conn(0x7f801c0b2120 msgr2=0x7f801c06e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:58:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.754+0000 7f8021600640 1 -- 172.21.15.32:0/3230560102 shutdown_connections 2024-04-17T09:58:59.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:58:59.754+0000 7f8021600640 1 -- 172.21.15.32:0/3230560102 wait complete. 2024-04-17T09:58:59.775 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T09:58:59.775 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T09:58:59.877 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T09:58:59.960 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T09:58:59.960 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:58:59.961 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:58:59.969 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T09:58:59.970 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:58:59.970 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:58:59.979 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T09:58:59.980 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:58:59.980 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:59:00.025 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T09:59:00.026 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T09:59:00.026 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:59:00.034 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T09:59:00.034 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-17T09:59:00.035 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:59:00.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.210+0000 7fbe1f200640 1 Processor -- start 2024-04-17T09:59:00.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.210+0000 7fbe1f200640 1 -- start start 2024-04-17T09:59:00.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.210+0000 7fbe1f200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe18072500 0x7fbe180cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:00.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.210+0000 7fbe1f200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbe1815d8d0 0x7fbe180cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:00.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.210+0000 7fbe1f200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe180cdac0 con 0x7fbe18145f30 2024-04-17T09:59:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.210+0000 7fbe1f200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fbe1806f400 con 0x7fbe1815d8d0 2024-04-17T09:59:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.210+0000 7fbe1f200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbe18146310 con 0x7fbe18072500 2024-04-17T09:59:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.210+0000 7fbe17e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbe18145f30 0x7fbe18071df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49148/0 (socket says 172.21.15.32:49148) 2024-04-17T09:59:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.210+0000 7fbe17e00640 1 -- 172.21.15.32:0/2166642367 learned_addr learned my addr 172.21.15.32:0/2166642367 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.214+0000 7fbe1d200640 1 --2- 172.21.15.32:0/2166642367 >> v2:172.21.15.144:3300/0 conn(0x7fbe1815d8d0 0x7fbe180cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.214+0000 7fbe0fe00640 1 --2- 172.21.15.32:0/2166642367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe18072500 0x7fbe180cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.214+0000 7fbe17400640 1 -- 172.21.15.32:0/2166642367 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3979976015 0 0) 0x7fbe180cdac0 con 0x7fbe18145f30 2024-04-17T09:59:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.214+0000 7fbe17400640 1 -- 172.21.15.32:0/2166642367 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbdfc003530 con 0x7fbe18145f30 2024-04-17T09:59:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.214+0000 7fbe1d200640 1 -- 172.21.15.32:0/2166642367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe18072500 msgr2=0x7fbe180cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.214+0000 7fbe1d200640 1 --2- 172.21.15.32:0/2166642367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe18072500 0x7fbe180cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.214+0000 7fbe1d200640 1 -- 172.21.15.32:0/2166642367 >> v1:172.21.15.32:6789/0 conn(0x7fbe18145f30 legacy=0x7fbe18071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.214+0000 7fbe1d200640 1 -- 172.21.15.32:0/2166642367 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1815cdf0 con 0x7fbe1815d8d0 2024-04-17T09:59:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.214+0000 7fbe1d200640 1 --2- 172.21.15.32:0/2166642367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe1815d8d0 0x7fbe180cd3b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbe10009d90 tx=0x7fbe1001f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3602f2579edd8e7e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.214+0000 7fbe17400640 1 -- 172.21.15.32:0/2166642367 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe100b4070 con 0x7fbe1815d8d0 2024-04-17T09:59:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.214+0000 7fbe17400640 1 -- 172.21.15.32:0/2166642367 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe100af600 con 0x7fbe1815d8d0 2024-04-17T09:59:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.214+0000 7fbe17400640 1 -- 172.21.15.32:0/2166642367 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe100af9d0 con 0x7fbe1815d8d0 2024-04-17T09:59:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe1f200640 1 -- 172.21.15.32:0/2166642367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe1815d8d0 msgr2=0x7fbe180cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:00.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe1f200640 1 --2- 172.21.15.32:0/2166642367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe1815d8d0 0x7fbe180cd3b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbe10009d90 tx=0x7fbe1001f150 comp rx=0 tx=0).stop 2024-04-17T09:59:00.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe1f200640 1 -- 172.21.15.32:0/2166642367 shutdown_connections 2024-04-17T09:59:00.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe1f200640 1 --2- 172.21.15.32:0/2166642367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe1815d8d0 0x7fbe180cd3b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:00.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe1f200640 1 --2- 172.21.15.32:0/2166642367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe18072500 0x7fbe180cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:00.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe1f200640 1 -- 172.21.15.32:0/2166642367 >> 172.21.15.32:0/2166642367 conn(0x7fbe1806ba30 msgr2=0x7fbe1806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:00.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe1f200640 1 -- 172.21.15.32:0/2166642367 shutdown_connections 2024-04-17T09:59:00.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe1f200640 1 -- 172.21.15.32:0/2166642367 wait complete. 2024-04-17T09:59:00.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe1f200640 1 Processor -- start 2024-04-17T09:59:00.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe1f200640 1 -- start start 2024-04-17T09:59:00.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe1f200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe18072500 0x7fbe180c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:00.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe1f200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe18145f30 0x7fbe181f2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:00.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe1f200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe1815d8d0 0x7fbe181f26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:00.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe1f200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1815cb90 con 0x7fbe18145f30 2024-04-17T09:59:00.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe1f200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1815d440 con 0x7fbe1815d8d0 2024-04-17T09:59:00.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe1f200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbe181601d0 con 0x7fbe18072500 2024-04-17T09:59:00.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe0fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe18145f30 0x7fbe181f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:00.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe0fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe18145f30 0x7fbe181f2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34912/0 (socket says 172.21.15.32:34912) 2024-04-17T09:59:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe0fe00640 1 -- 172.21.15.32:0/997215374 learned_addr learned my addr 172.21.15.32:0/997215374 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe1d200640 1 --2- 172.21.15.32:0/997215374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe1815d8d0 0x7fbe181f26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe17e00640 1 --2- 172.21.15.32:0/997215374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe18072500 0x7fbe180c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe0fe00640 1 -- 172.21.15.32:0/997215374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe18072500 msgr2=0x7fbe180c5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe0fe00640 1 --2- 172.21.15.32:0/997215374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe18072500 0x7fbe180c5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe0fe00640 1 -- 172.21.15.32:0/997215374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe1815d8d0 msgr2=0x7fbe181f26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe0fe00640 1 --2- 172.21.15.32:0/997215374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe1815d8d0 0x7fbe181f26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe0fe00640 1 -- 172.21.15.32:0/997215374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe18140290 con 0x7fbe18145f30 2024-04-17T09:59:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.218+0000 7fbe0fe00640 1 --2- 172.21.15.32:0/997215374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe18145f30 0x7fbe181f2180 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000e930 tx=0x7fbe0000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.222+0000 7fbe14c00640 1 -- 172.21.15.32:0/997215374 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00017070 con 0x7fbe18145f30 2024-04-17T09:59:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.222+0000 7fbe14c00640 1 -- 172.21.15.32:0/997215374 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe0000ca00 con 0x7fbe18145f30 2024-04-17T09:59:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.222+0000 7fbe14c00640 1 -- 172.21.15.32:0/997215374 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0000cdb0 con 0x7fbe18145f30 2024-04-17T09:59:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.222+0000 7fbe1f200640 1 -- 172.21.15.32:0/997215374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe18140570 con 0x7fbe18145f30 2024-04-17T09:59:00.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.222+0000 7fbe1f200640 1 -- 172.21.15.32:0/997215374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe18140b30 con 0x7fbe18145f30 2024-04-17T09:59:00.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.222+0000 7fbe1f200640 1 -- 172.21.15.32:0/997215374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe18068a90 con 0x7fbe18145f30 2024-04-17T09:59:00.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.222+0000 7fbe14c00640 1 -- 172.21.15.32:0/997215374 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbe0001a070 con 0x7fbe18145f30 2024-04-17T09:59:00.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.222+0000 7fbe14c00640 1 --2- 172.21.15.32:0/997215374 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbdf4068380 0x7fbdf406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:00.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.222+0000 7fbe14c00640 1 -- 172.21.15.32:0/997215374 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5029+0+0 (secure 0 0 0) 0x7fbe00086680 con 0x7fbe18145f30 2024-04-17T09:59:00.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.222+0000 7fbe17e00640 1 --2- 172.21.15.32:0/997215374 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbdf4068380 0x7fbdf406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:00.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.222+0000 7fbe17e00640 1 --2- 172.21.15.32:0/997215374 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbdf4068380 0x7fbdf406a840 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fbe08007fd0 tx=0x7fbe08009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.226+0000 7fbe14c00640 1 -- 172.21.15.32:0/997215374 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fbe000569e0 con 0x7fbe18145f30 2024-04-17T09:59:00.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.426+0000 7fbe1f200640 1 -- 172.21.15.32:0/997215374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe18072f90 con 0x7fbe18145f30 2024-04-17T09:59:00.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.426+0000 7fbe14c00640 1 -- 172.21.15.32:0/997215374 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+3872 (secure 0 0 0) 0x7fbe0005ace0 con 0x7fbe18145f30 2024-04-17T09:59:00.425 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:59:00.425 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":52,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2424282223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2424282223},{"type":"v1","addr":"172.21.15.144:6834","nonce":2424282223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3537056692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3537056692},{"type":"v1","addr":"172.21.15.144:6835","nonce":3537056692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5891,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/725333724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":725333724},{"type":"v1","addr":"172.21.15.32:6837","nonce":725333724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":7476,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1786985774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1786985774},{"type":"v1","addr":"172.21.15.32:6836","nonce":1786985774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[]} 2024-04-17T09:59:00.426 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 52 2024-04-17T09:59:00.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.430+0000 7fbe1f200640 1 -- 172.21.15.32:0/997215374 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbdf4068380 msgr2=0x7fbdf406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:00.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.430+0000 7fbe1f200640 1 --2- 172.21.15.32:0/997215374 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbdf4068380 0x7fbdf406a840 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fbe08007fd0 tx=0x7fbe08009040 comp rx=0 tx=0).stop 2024-04-17T09:59:00.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.430+0000 7fbe1f200640 1 -- 172.21.15.32:0/997215374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe18145f30 msgr2=0x7fbe181f2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:00.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.430+0000 7fbe1f200640 1 --2- 172.21.15.32:0/997215374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe18145f30 0x7fbe181f2180 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000e930 tx=0x7fbe0000edf0 comp rx=0 tx=0).stop 2024-04-17T09:59:00.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.430+0000 7fbe1f200640 1 -- 172.21.15.32:0/997215374 shutdown_connections 2024-04-17T09:59:00.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.430+0000 7fbe1f200640 1 --2- 172.21.15.32:0/997215374 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbdf4068380 0x7fbdf406a840 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:00.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.430+0000 7fbe1f200640 1 --2- 172.21.15.32:0/997215374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe1815d8d0 0x7fbe181f26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:00.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.430+0000 7fbe1f200640 1 --2- 172.21.15.32:0/997215374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe18145f30 0x7fbe181f2180 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:00.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.430+0000 7fbe1f200640 1 --2- 172.21.15.32:0/997215374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe18072500 0x7fbe180c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:00.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.430+0000 7fbe1f200640 1 -- 172.21.15.32:0/997215374 >> 172.21.15.32:0/997215374 conn(0x7fbe1806ba30 msgr2=0x7fbe180ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:00.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.430+0000 7fbe1f200640 1 -- 172.21.15.32:0/997215374 shutdown_connections 2024-04-17T09:59:00.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.430+0000 7fbe1f200640 1 -- 172.21.15.32:0/997215374 wait complete. 2024-04-17T09:59:00.446 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-17T09:59:00.447 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-17T09:59:00.447 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-17T09:59:00.447 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:59:00.447 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-17T09:59:00.447 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-17T09:59:00.447 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-17T09:59:00.447 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:59:00.448 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-17T09:59:00.448 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-17T09:59:00.448 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-17T09:59:00.448 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:59:00.448 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-17T09:59:00.448 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-17T09:59:00.448 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-17T09:59:00.448 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:59:00.460 INFO:tasks.ceph.mds.a:Stopped 2024-04-17T09:59:00.460 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-17T09:59:00.461 INFO:tasks.ceph.mds.c:Stopped 2024-04-17T09:59:00.461 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-17T09:59:00.462 INFO:tasks.ceph.mds.d:Stopped 2024-04-17T09:59:00.463 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-17T09:59:00.464 INFO:tasks.ceph.mds.b:Stopped 2024-04-17T09:59:00.464 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-17T09:59:00.505 INFO:tasks.ceph.mds.a:Started 2024-04-17T09:59:00.505 DEBUG:teuthology.parallel:result is None 2024-04-17T09:59:00.506 INFO:tasks.ceph.mds.c:Started 2024-04-17T09:59:00.507 DEBUG:teuthology.parallel:result is None 2024-04-17T09:59:00.508 INFO:tasks.ceph.mds.d:Started 2024-04-17T09:59:00.508 DEBUG:teuthology.parallel:result is None 2024-04-17T09:59:00.510 INFO:tasks.ceph.mds.b:Started 2024-04-17T09:59:00.510 DEBUG:teuthology.parallel:result is None 2024-04-17T09:59:00.510 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-17T09:59:00.592 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-17T09:59:00.592 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-17T09:59:00.597 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2024-04-17T09:59:00.598 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2024-04-17T09:59:00.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.662+0000 7f2e8be00640 1 Processor -- start 2024-04-17T09:59:00.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.662+0000 7f2e8be00640 1 -- start start 2024-04-17T09:59:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.662+0000 7f2e8be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e840cf370 0x7f2e840d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.662+0000 7f2e8be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e840d0820 0x7f2e84071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.662+0000 7f2e8be00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e840cff60 con 0x7f2e8413f6e0 2024-04-17T09:59:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.662+0000 7f2e8be00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2e8413fac0 con 0x7f2e840d0820 2024-04-17T09:59:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.662+0000 7f2e8be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2e841601e0 con 0x7f2e840cf370 2024-04-17T09:59:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.662+0000 7f2e89e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e840d0820 0x7f2e84071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.662+0000 7f2e89e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e840d0820 0x7f2e84071d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34408/0 (socket says 172.21.15.32:34408) 2024-04-17T09:59:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.662+0000 7f2e89e00640 1 -- 172.21.15.32:0/2633464534 learned_addr learned my addr 172.21.15.32:0/2633464534 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:00.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.662+0000 7f2e88a00640 1 --2- 172.21.15.32:0/2633464534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e840cf370 0x7f2e840d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:00.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.662+0000 7f2e89e00640 1 -- 172.21.15.32:0/2633464534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e840cf370 msgr2=0x7f2e840d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:00.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.662+0000 7f2e89e00640 1 --2- 172.21.15.32:0/2633464534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e840cf370 0x7f2e840d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:00.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.662+0000 7f2e89e00640 1 -- 172.21.15.32:0/2633464534 >> v1:172.21.15.144:6789/0 conn(0x7f2e8413f6e0 legacy=0x7f2e840cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:00.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.662+0000 7f2e89e00640 1 -- 172.21.15.32:0/2633464534 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e84160480 con 0x7f2e840d0820 2024-04-17T09:59:00.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.662+0000 7f2e88a00640 1 --2- 172.21.15.32:0/2633464534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e840cf370 0x7f2e840d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:59:00.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.666+0000 7f2e89e00640 1 --2- 172.21.15.32:0/2633464534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e840d0820 0x7f2e84071d80 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009e50 tx=0x7f2e7c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1adccaffbfb40b35 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:00.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.666+0000 7f2e83e00640 1 -- 172.21.15.32:0/2633464534 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c0a5760 con 0x7f2e840d0820 2024-04-17T09:59:00.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.666+0000 7f2e83e00640 1 -- 172.21.15.32:0/2633464534 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e7c0a5900 con 0x7f2e840d0820 2024-04-17T09:59:00.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.666+0000 7f2e83e00640 1 -- 172.21.15.32:0/2633464534 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c0a5cd0 con 0x7f2e840d0820 2024-04-17T09:59:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.666+0000 7f2e8be00640 1 -- 172.21.15.32:0/2633464534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e840d0820 msgr2=0x7f2e84071d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.666+0000 7f2e8be00640 1 --2- 172.21.15.32:0/2633464534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e840d0820 0x7f2e84071d80 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009e50 tx=0x7f2e7c01c9f0 comp rx=0 tx=0).stop 2024-04-17T09:59:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.666+0000 7f2e8be00640 1 -- 172.21.15.32:0/2633464534 shutdown_connections 2024-04-17T09:59:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.666+0000 7f2e8be00640 1 --2- 172.21.15.32:0/2633464534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e840d0820 0x7f2e84071d80 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.666+0000 7f2e8be00640 1 --2- 172.21.15.32:0/2633464534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e840cf370 0x7f2e840d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.666+0000 7f2e8be00640 1 -- 172.21.15.32:0/2633464534 >> 172.21.15.32:0/2633464534 conn(0x7f2e8406ba30 msgr2=0x7f2e8406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:00.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.666+0000 7f2e8be00640 1 -- 172.21.15.32:0/2633464534 shutdown_connections 2024-04-17T09:59:00.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.666+0000 7f2e8be00640 1 -- 172.21.15.32:0/2633464534 wait complete. 2024-04-17T09:59:00.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.666+0000 7f2e8be00640 1 Processor -- start 2024-04-17T09:59:00.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.666+0000 7f2e8be00640 1 -- start start 2024-04-17T09:59:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e8be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e840cf370 0x7f2e8415e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e8be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e840d0820 0x7f2e840b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e8be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e8413f6e0 0x7f2e8415b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e8be00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2e84072ad0 con 0x7f2e8413f6e0 2024-04-17T09:59:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e8be00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8413fd40 con 0x7f2e840d0820 2024-04-17T09:59:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e8be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2e84072c50 con 0x7f2e840cf370 2024-04-17T09:59:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e89e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e8413f6e0 0x7f2e8415b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e89e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e8413f6e0 0x7f2e8415b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34422/0 (socket says 172.21.15.32:34422) 2024-04-17T09:59:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e89e00640 1 -- 172.21.15.32:0/3034449756 learned_addr learned my addr 172.21.15.32:0/3034449756 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e88a00640 1 --2- 172.21.15.32:0/3034449756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e840d0820 0x7f2e840b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e89400640 1 --2- 172.21.15.32:0/3034449756 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e840cf370 0x7f2e8415e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e89e00640 1 -- 172.21.15.32:0/3034449756 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e840cf370 msgr2=0x7f2e8415e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e89e00640 1 --2- 172.21.15.32:0/3034449756 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e840cf370 0x7f2e8415e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e89e00640 1 -- 172.21.15.32:0/3034449756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e840d0820 msgr2=0x7f2e840b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e89e00640 1 --2- 172.21.15.32:0/3034449756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e840d0820 0x7f2e840b4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e89e00640 1 -- 172.21.15.32:0/3034449756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8415f150 con 0x7f2e8413f6e0 2024-04-17T09:59:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e88a00640 1 --2- 172.21.15.32:0/3034449756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e840d0820 0x7f2e840b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e89e00640 1 --2- 172.21.15.32:0/3034449756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e8413f6e0 0x7f2e8415b840 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0af040 tx=0x7f2e7c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e81600640 1 -- 172.21.15.32:0/3034449756 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c0bc020 con 0x7f2e8413f6e0 2024-04-17T09:59:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e81600640 1 -- 172.21.15.32:0/3034449756 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e7c0b8670 con 0x7f2e8413f6e0 2024-04-17T09:59:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e81600640 1 -- 172.21.15.32:0/3034449756 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c0b8a60 con 0x7f2e8413f6e0 2024-04-17T09:59:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e8be00640 1 -- 172.21.15.32:0/3034449756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e8415f430 con 0x7f2e8413f6e0 2024-04-17T09:59:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e8be00640 1 -- 172.21.15.32:0/3034449756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e8415f9f0 con 0x7f2e8413f6e0 2024-04-17T09:59:00.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e8be00640 1 -- 172.21.15.32:0/3034449756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e84068a90 con 0x7f2e8413f6e0 2024-04-17T09:59:00.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.670+0000 7f2e81600640 1 -- 172.21.15.32:0/3034449756 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2e7c0bf020 con 0x7f2e8413f6e0 2024-04-17T09:59:00.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.674+0000 7f2e81600640 1 --2- 172.21.15.32:0/3034449756 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e64068380 0x7f2e6406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:00.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.674+0000 7f2e81600640 1 -- 172.21.15.32:0/3034449756 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5029+0+0 (secure 0 0 0) 0x7f2e7c121930 con 0x7f2e8413f6e0 2024-04-17T09:59:00.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.674+0000 7f2e89400640 1 --2- 172.21.15.32:0/3034449756 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e64068380 0x7f2e6406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:00.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.674+0000 7f2e89400640 1 --2- 172.21.15.32:0/3034449756 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e64068380 0x7f2e6406a840 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2e78004e30 tx=0x7f2e78009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:00.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.678+0000 7f2e81600640 1 -- 172.21.15.32:0/3034449756 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2e7c0c6030 con 0x7f2e8413f6e0 2024-04-17T09:59:00.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:00.854+0000 7f2e8be00640 1 -- 172.21.15.32:0/3034449756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2e840d1030 con 0x7f2e8413f6e0 2024-04-17T09:59:01.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.402+0000 7f2e81600640 1 -- 172.21.15.32:0/3034449756 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v76) ==== 97+0+0 (secure 0 0 0) 0x7f2e7c0165e0 con 0x7f2e8413f6e0 2024-04-17T09:59:01.399 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2024-04-17T09:59:01.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.402+0000 7f2e8be00640 1 -- 172.21.15.32:0/3034449756 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e64068380 msgr2=0x7f2e6406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:01.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.402+0000 7f2e8be00640 1 --2- 172.21.15.32:0/3034449756 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e64068380 0x7f2e6406a840 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2e78004e30 tx=0x7f2e78009040 comp rx=0 tx=0).stop 2024-04-17T09:59:01.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.402+0000 7f2e8be00640 1 -- 172.21.15.32:0/3034449756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e8413f6e0 msgr2=0x7f2e8415b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:01.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.402+0000 7f2e8be00640 1 --2- 172.21.15.32:0/3034449756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e8413f6e0 0x7f2e8415b840 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0af040 tx=0x7f2e7c019790 comp rx=0 tx=0).stop 2024-04-17T09:59:01.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.402+0000 7f2e8be00640 1 -- 172.21.15.32:0/3034449756 shutdown_connections 2024-04-17T09:59:01.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.402+0000 7f2e8be00640 1 --2- 172.21.15.32:0/3034449756 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e64068380 0x7f2e6406a840 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:01.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.402+0000 7f2e8be00640 1 --2- 172.21.15.32:0/3034449756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e8413f6e0 0x7f2e8415b840 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:01.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.402+0000 7f2e8be00640 1 --2- 172.21.15.32:0/3034449756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e840d0820 0x7f2e840b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:01.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.402+0000 7f2e8be00640 1 --2- 172.21.15.32:0/3034449756 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e840cf370 0x7f2e8415e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:01.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.402+0000 7f2e8be00640 1 -- 172.21.15.32:0/3034449756 >> 172.21.15.32:0/3034449756 conn(0x7f2e8406ba30 msgr2=0x7f2e84141b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:01.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.406+0000 7f2e8be00640 1 -- 172.21.15.32:0/3034449756 shutdown_connections 2024-04-17T09:59:01.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.406+0000 7f2e8be00640 1 -- 172.21.15.32:0/3034449756 wait complete. 2024-04-17T09:59:01.426 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-17T09:59:01.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.570+0000 7f3a67600640 1 Processor -- start 2024-04-17T09:59:01.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.570+0000 7f3a67600640 1 -- start start 2024-04-17T09:59:01.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.574+0000 7f3a67600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3a60144e90 0x7f3a600b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:01.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.574+0000 7f3a67600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a6015c1b0 0x7f3a6006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:01.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.574+0000 7f3a67600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a60072bb0 con 0x7f3a600b4de0 2024-04-17T09:59:01.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.574+0000 7f3a67600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3a60145270 con 0x7f3a60144e90 2024-04-17T09:59:01.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.574+0000 7f3a67600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3a600b3dc0 con 0x7f3a6015c1b0 2024-04-17T09:59:01.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.574+0000 7f3a57e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3a600b4de0 0x7f3a6015baa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38002/0 (socket says 172.21.15.32:38002) 2024-04-17T09:59:01.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.574+0000 7f3a57e00640 1 -- 172.21.15.32:0/3557164021 learned_addr learned my addr 172.21.15.32:0/3557164021 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.574+0000 7f3a64c00640 1 --2- 172.21.15.32:0/3557164021 >> v2:172.21.15.32:3300/0 conn(0x7f3a60144e90 0x7f3a600b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.574+0000 7f3a65600640 1 --2- 172.21.15.32:0/3557164021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a6015c1b0 0x7f3a6006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.574+0000 7f3a64c00640 1 -- 172.21.15.32:0/3557164021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a6015c1b0 msgr2=0x7f3a6006ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.574+0000 7f3a5fe00640 1 -- 172.21.15.32:0/3557164021 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2607317061 0 0) 0x7f3a60072bb0 con 0x7f3a600b4de0 2024-04-17T09:59:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.574+0000 7f3a64c00640 1 --2- 172.21.15.32:0/3557164021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a6015c1b0 0x7f3a6006ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.574+0000 7f3a64c00640 1 -- 172.21.15.32:0/3557164021 >> v1:172.21.15.32:6789/0 conn(0x7f3a600b4de0 legacy=0x7f3a6015baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.574+0000 7f3a64c00640 1 -- 172.21.15.32:0/3557164021 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a600b40a0 con 0x7f3a60144e90 2024-04-17T09:59:01.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.574+0000 7f3a64c00640 1 --2- 172.21.15.32:0/3557164021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a60144e90 0x7f3a600b48a0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f3a500097c0 tx=0x7f3a5001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2c2cf32bc5f563 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:01.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.578+0000 7f3a5fe00640 1 -- 172.21.15.32:0/3557164021 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a500b2070 con 0x7f3a60144e90 2024-04-17T09:59:01.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.578+0000 7f3a5fe00640 1 -- 172.21.15.32:0/3557164021 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a500ad680 con 0x7f3a60144e90 2024-04-17T09:59:01.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.578+0000 7f3a5fe00640 1 -- 172.21.15.32:0/3557164021 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a500ada50 con 0x7f3a60144e90 2024-04-17T09:59:01.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.578+0000 7f3a67600640 1 -- 172.21.15.32:0/3557164021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a60144e90 msgr2=0x7f3a600b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:01.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.578+0000 7f3a67600640 1 --2- 172.21.15.32:0/3557164021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a60144e90 0x7f3a600b48a0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f3a500097c0 tx=0x7f3a5001c470 comp rx=0 tx=0).stop 2024-04-17T09:59:01.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.578+0000 7f3a67600640 1 -- 172.21.15.32:0/3557164021 shutdown_connections 2024-04-17T09:59:01.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.578+0000 7f3a67600640 1 --2- 172.21.15.32:0/3557164021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a6015c1b0 0x7f3a6006ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:01.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.578+0000 7f3a67600640 1 --2- 172.21.15.32:0/3557164021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a60144e90 0x7f3a600b48a0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:01.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.578+0000 7f3a67600640 1 -- 172.21.15.32:0/3557164021 >> 172.21.15.32:0/3557164021 conn(0x7f3a6006ba30 msgr2=0x7f3a6006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:01.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.578+0000 7f3a67600640 1 -- 172.21.15.32:0/3557164021 shutdown_connections 2024-04-17T09:59:01.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.578+0000 7f3a67600640 1 -- 172.21.15.32:0/3557164021 wait complete. 2024-04-17T09:59:01.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.578+0000 7f3a67600640 1 Processor -- start 2024-04-17T09:59:01.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.578+0000 7f3a67600640 1 -- start start 2024-04-17T09:59:01.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.578+0000 7f3a67600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a600b4de0 0x7f3a601658b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:01.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.578+0000 7f3a67600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a60144e90 0x7f3a601408d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:01.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.578+0000 7f3a67600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a6015c1b0 0x7f3a60140e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:01.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.578+0000 7f3a67600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3a60168110 con 0x7f3a600b4de0 2024-04-17T09:59:01.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.578+0000 7f3a67600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3a60072880 con 0x7f3a6015c1b0 2024-04-17T09:59:01.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.578+0000 7f3a67600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3a60168290 con 0x7f3a60144e90 2024-04-17T09:59:01.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.578+0000 7f3a64c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a600b4de0 0x7f3a601658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:01.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.578+0000 7f3a64c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a600b4de0 0x7f3a601658b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34446/0 (socket says 172.21.15.32:34446) 2024-04-17T09:59:01.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.578+0000 7f3a64c00640 1 -- 172.21.15.32:0/2407374439 learned_addr learned my addr 172.21.15.32:0/2407374439 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:01.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.582+0000 7f3a65600640 1 --2- 172.21.15.32:0/2407374439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a6015c1b0 0x7f3a60140e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:01.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.582+0000 7f3a57e00640 1 --2- 172.21.15.32:0/2407374439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a60144e90 0x7f3a601408d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:01.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.582+0000 7f3a64c00640 1 -- 172.21.15.32:0/2407374439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a60144e90 msgr2=0x7f3a601408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:01.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.582+0000 7f3a64c00640 1 --2- 172.21.15.32:0/2407374439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a60144e90 0x7f3a601408d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:01.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.582+0000 7f3a64c00640 1 -- 172.21.15.32:0/2407374439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a6015c1b0 msgr2=0x7f3a60140e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:01.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.582+0000 7f3a64c00640 1 --2- 172.21.15.32:0/2407374439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a6015c1b0 0x7f3a60140e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:01.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.582+0000 7f3a64c00640 1 -- 172.21.15.32:0/2407374439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6013fcb0 con 0x7f3a600b4de0 2024-04-17T09:59:01.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.582+0000 7f3a57e00640 1 --2- 172.21.15.32:0/2407374439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a60144e90 0x7f3a601408d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:59:01.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.582+0000 7f3a65600640 1 --2- 172.21.15.32:0/2407374439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a6015c1b0 0x7f3a60140e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:59:01.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.582+0000 7f3a64c00640 1 --2- 172.21.15.32:0/2407374439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a600b4de0 0x7f3a601658b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f3a5001a0a0 tx=0x7f3a5001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:01.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.582+0000 7f3a5d600640 1 -- 172.21.15.32:0/2407374439 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a500b2040 con 0x7f3a600b4de0 2024-04-17T09:59:01.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.582+0000 7f3a5d600640 1 -- 172.21.15.32:0/2407374439 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a500bd070 con 0x7f3a600b4de0 2024-04-17T09:59:01.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.582+0000 7f3a67600640 1 -- 172.21.15.32:0/2407374439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a60141440 con 0x7f3a600b4de0 2024-04-17T09:59:01.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.582+0000 7f3a5d600640 1 -- 172.21.15.32:0/2407374439 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a500ade60 con 0x7f3a600b4de0 2024-04-17T09:59:01.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.582+0000 7f3a67600640 1 -- 172.21.15.32:0/2407374439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a6015c7e0 con 0x7f3a600b4de0 2024-04-17T09:59:01.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.582+0000 7f3a67600640 1 -- 172.21.15.32:0/2407374439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a60141960 con 0x7f3a600b4de0 2024-04-17T09:59:01.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.582+0000 7f3a5d600640 1 -- 172.21.15.32:0/2407374439 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3a500b8690 con 0x7f3a600b4de0 2024-04-17T09:59:01.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.582+0000 7f3a5d600640 1 --2- 172.21.15.32:0/2407374439 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3a3c068380 0x7f3a3c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:01.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.582+0000 7f3a5d600640 1 -- 172.21.15.32:0/2407374439 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4857+0+0 (secure 0 0 0) 0x7f3a50120d20 con 0x7f3a600b4de0 2024-04-17T09:59:01.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.582+0000 7f3a57e00640 1 --2- 172.21.15.32:0/2407374439 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3a3c068380 0x7f3a3c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:01.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.586+0000 7f3a57e00640 1 --2- 172.21.15.32:0/2407374439 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3a3c068380 0x7f3a3c06a840 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3a600cce50 tx=0x7f3a48009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:01.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.586+0000 7f3a5d600640 1 -- 172.21.15.32:0/2407374439 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3a60141960 con 0x7f3a600b4de0 2024-04-17T09:59:01.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.810+0000 7f3a67600640 1 -- 172.21.15.32:0/2407374439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f3a60142330 con 0x7f3a600b4de0 2024-04-17T09:59:01.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.810+0000 7f3a5d600640 1 -- 172.21.15.32:0/2407374439 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7f3a50016090 con 0x7f3a600b4de0 2024-04-17T09:59:01.809 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:59:01.809 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:59:01.809 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2024-04-17T09:59:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:59:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2024-04-17T09:59:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mzM1KExAAn0WH+Gf6gGV/dJonXzAgPQ==", 2024-04-17T09:59:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:59:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:59:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:59:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:59:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:59:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:59:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2024-04-17T09:59:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mQnJ4HxAAT0rFHhdAj7PMxZGiqEyWbg==", 2024-04-17T09:59:01.810 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:59:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:59:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:59:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:59:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:59:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:59:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2024-04-17T09:59:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mWWi5GhAA3RAdRxK6GRwuVCH4DHoMeA==", 2024-04-17T09:59:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:59:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:59:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:59:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:59:01.811 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:59:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:59:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2024-04-17T09:59:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9m9TMAJxAAbmUp05GHRE1AqCmq83scVQ==", 2024-04-17T09:59:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:59:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:59:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:59:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:59:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:59:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:59:01.812 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2024-04-17T09:59:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDRmx9mnPwyDBAAxD10jhxptVH5zhupengElg==", 2024-04-17T09:59:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:59:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:59:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:59:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:59:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:59:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2024-04-17T09:59:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUmx9mh3iuGBAAqvWSRyNX3u/TjEH6TJpqdA==", 2024-04-17T09:59:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:59:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:59:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:59:01.813 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:59:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:59:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2024-04-17T09:59:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXmx9mbe1CIRAA1pmzAl04tGkTu6QLpmCkag==", 2024-04-17T09:59:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:59:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:59:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:59:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:59:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:59:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2024-04-17T09:59:01.814 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDamx9mIwXoKxAAWPTFaYdp4lIZhuOU7F0x8g==", 2024-04-17T09:59:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:59:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:59:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:59:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:59:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:59:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2024-04-17T09:59:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDgmx9mp0s/JxAAv+usQqwrxgKINRlkDhhydQ==", 2024-04-17T09:59:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:59:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:59:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:59:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:59:01.815 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:59:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2024-04-17T09:59:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDjmx9mMVStMhAA/n8jk+LVDosWhziXk/lLRQ==", 2024-04-17T09:59:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:59:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:59:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:59:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:59:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:59:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2024-04-17T09:59:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDnmx9m/YQlABAAOqfWzVtgonD7b+WlR95mnA==", 2024-04-17T09:59:01.816 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:59:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:59:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:59:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:59:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:59:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2024-04-17T09:59:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDqmx9m572mCBAAqBib/yeQOBno+0B1K1dLxA==", 2024-04-17T09:59:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:59:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T09:59:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T09:59:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:59:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:59:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2024-04-17T09:59:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mUXoLKxAANV7zhLvL0DqHngk4TtsdBQ==", 2024-04-17T09:59:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:59:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:59:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2024-04-17T09:59:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-17T09:59:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:59:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2024-04-17T09:59:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mZBeRNBAAQg5K950mp6+XND7NCABAKQ==", 2024-04-17T09:59:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:59:01.818 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:59:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-17T09:59:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-17T09:59:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-17T09:59:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:59:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2024-04-17T09:59:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mWKstMRAAp18Hbnk02w+hT0sYZ3tSLA==", 2024-04-17T09:59:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:59:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:59:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-17T09:59:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-17T09:59:01.819 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-17T09:59:01.821 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:01.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:01.821 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:59:01.821 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2024-04-17T09:59:01.821 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mRQ2wOhAAzlWXFtP0WaXWmneqxnZ5/w==", 2024-04-17T09:59:01.821 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:59:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T09:59:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-17T09:59:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-17T09:59:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-17T09:59:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:59:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2024-04-17T09:59:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDOmx9maDStMxAAMCa36HfycwV3WbLg6Voo9Q==", 2024-04-17T09:59:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:59:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-17T09:59:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T09:59:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T09:59:01.822 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:59:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:59:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2024-04-17T09:59:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9myy0oDxAAPD2BTjIP09hiU5L7i6gUMg==", 2024-04-17T09:59:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:59:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-17T09:59:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-17T09:59:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:59:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:59:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2024-04-17T09:59:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9m3FmACxAApri5AI5PeR0+/HLev8DKcw==", 2024-04-17T09:59:01.823 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T09:59:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-17T09:59:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-17T09:59:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T09:59:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:01.824 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-17T09:59:01.824 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:59:01.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.814+0000 7f3a67600640 1 -- 172.21.15.32:0/2407374439 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3a3c068380 msgr2=0x7f3a3c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:01.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.814+0000 7f3a67600640 1 --2- 172.21.15.32:0/2407374439 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3a3c068380 0x7f3a3c06a840 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3a600cce50 tx=0x7f3a48009040 comp rx=0 tx=0).stop 2024-04-17T09:59:01.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.814+0000 7f3a67600640 1 -- 172.21.15.32:0/2407374439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a600b4de0 msgr2=0x7f3a601658b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:01.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.814+0000 7f3a67600640 1 --2- 172.21.15.32:0/2407374439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a600b4de0 0x7f3a601658b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f3a5001a0a0 tx=0x7f3a5001acb0 comp rx=0 tx=0).stop 2024-04-17T09:59:01.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.814+0000 7f3a67600640 1 -- 172.21.15.32:0/2407374439 shutdown_connections 2024-04-17T09:59:01.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.814+0000 7f3a67600640 1 --2- 172.21.15.32:0/2407374439 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3a3c068380 0x7f3a3c06a840 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:01.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.814+0000 7f3a67600640 1 --2- 172.21.15.32:0/2407374439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a60144e90 0x7f3a601408d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:01.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.814+0000 7f3a67600640 1 --2- 172.21.15.32:0/2407374439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a6015c1b0 0x7f3a60140e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:01.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.814+0000 7f3a67600640 1 --2- 172.21.15.32:0/2407374439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a600b4de0 0x7f3a601658b0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:01.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.814+0000 7f3a67600640 1 -- 172.21.15.32:0/2407374439 >> 172.21.15.32:0/2407374439 conn(0x7f3a6006ba30 msgr2=0x7f3a600cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:01.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.814+0000 7f3a67600640 1 -- 172.21.15.32:0/2407374439 shutdown_connections 2024-04-17T09:59:01.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.814+0000 7f3a67600640 1 -- 172.21.15.32:0/2407374439 wait complete. 2024-04-17T09:59:01.830 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:59:01.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.974+0000 7f5ef8c00640 1 Processor -- start 2024-04-17T09:59:01.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.974+0000 7f5ef8c00640 1 -- start start 2024-04-17T09:59:01.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.974+0000 7f5ef8c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ef4145f30 0x7f5ef4071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:01.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.974+0000 7f5ef8c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ef40cdac0 con 0x7f5ef415d8d0 2024-04-17T09:59:01.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.974+0000 7f5ef8c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ef415cdf0 con 0x7f5ef40cd580 2024-04-17T09:59:01.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.974+0000 7f5ef8c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5ef406f400 con 0x7f5ef4145f30 2024-04-17T09:59:01.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.974+0000 7f5eebe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5ef415d8d0 0x7f5ef40cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38012/0 (socket says 172.21.15.32:38012) 2024-04-17T09:59:01.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.974+0000 7f5eebe00640 1 -- 172.21.15.32:0/2353098239 learned_addr learned my addr 172.21.15.32:0/2353098239 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:01.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.974+0000 7f5ef1e00640 1 --2- 172.21.15.32:0/2353098239 >> v2:172.21.15.32:3300/0 conn(0x7f5ef4145f30 0x7f5ef4071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:01.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.978+0000 7f5ef1e00640 1 -- 172.21.15.32:0/2353098239 >> v1:172.21.15.144:6789/0 conn(0x7f5ef40cd580 legacy=0x7f5ef41601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:01.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.978+0000 7f5ef1e00640 1 -- 172.21.15.32:0/2353098239 >> v1:172.21.15.32:6789/0 conn(0x7f5ef415d8d0 legacy=0x7f5ef40cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:01.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.978+0000 7f5ef1400640 1 -- 172.21.15.32:0/2353098239 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2299111687 0 0) 0x7f5ef40cdac0 con 0x7f5ef415d8d0 2024-04-17T09:59:01.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.978+0000 7f5ef1e00640 1 -- 172.21.15.32:0/2353098239 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef415d0d0 con 0x7f5ef4145f30 2024-04-17T09:59:01.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.978+0000 7f5ef1e00640 1 --2- 172.21.15.32:0/2353098239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef4145f30 0x7f5ef4071df0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40097c0 tx=0x7f5ee401ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c615ccc66a3d427 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:01.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.978+0000 7f5ef1400640 1 -- 172.21.15.32:0/2353098239 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40b5070 con 0x7f5ef4145f30 2024-04-17T09:59:01.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.978+0000 7f5ef1400640 1 -- 172.21.15.32:0/2353098239 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ee40a8950 con 0x7f5ef4145f30 2024-04-17T09:59:01.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.978+0000 7f5ef1400640 1 -- 172.21.15.32:0/2353098239 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40a8d20 con 0x7f5ef4145f30 2024-04-17T09:59:01.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.978+0000 7f5ef8c00640 1 -- 172.21.15.32:0/2353098239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef4145f30 msgr2=0x7f5ef4071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:01.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.978+0000 7f5ef8c00640 1 --2- 172.21.15.32:0/2353098239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef4145f30 0x7f5ef4071df0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40097c0 tx=0x7f5ee401ec50 comp rx=0 tx=0).stop 2024-04-17T09:59:01.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.978+0000 7f5ef8c00640 1 -- 172.21.15.32:0/2353098239 shutdown_connections 2024-04-17T09:59:01.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.978+0000 7f5ef8c00640 1 --2- 172.21.15.32:0/2353098239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef4145f30 0x7f5ef4071df0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:01.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.978+0000 7f5ef8c00640 1 -- 172.21.15.32:0/2353098239 >> 172.21.15.32:0/2353098239 conn(0x7f5ef406ba30 msgr2=0x7f5ef406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:01.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.978+0000 7f5ef8c00640 1 -- 172.21.15.32:0/2353098239 shutdown_connections 2024-04-17T09:59:01.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.978+0000 7f5ef8c00640 1 -- 172.21.15.32:0/2353098239 wait complete. 2024-04-17T09:59:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5ef8c00640 1 Processor -- start 2024-04-17T09:59:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5ef8c00640 1 -- start start 2024-04-17T09:59:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5ef8c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ef40cd580 0x7f5ef41316a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5ef8c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ef4145f30 0x7f5ef4140370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5ef8c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef415d8d0 0x7f5ef41408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5ef8c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef4146310 con 0x7f5ef415d8d0 2024-04-17T09:59:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5ef8c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef4072600 con 0x7f5ef4145f30 2024-04-17T09:59:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5ef8c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5ef4072780 con 0x7f5ef40cd580 2024-04-17T09:59:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5ef2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef415d8d0 0x7f5ef41408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5ef2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef415d8d0 0x7f5ef41408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34460/0 (socket says 172.21.15.32:34460) 2024-04-17T09:59:01.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5ef2800640 1 -- 172.21.15.32:0/3703339338 learned_addr learned my addr 172.21.15.32:0/3703339338 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:01.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5eebe00640 1 --2- 172.21.15.32:0/3703339338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ef4145f30 0x7f5ef4140370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:01.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5ef1e00640 1 --2- 172.21.15.32:0/3703339338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ef40cd580 0x7f5ef41316a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:01.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5ef2800640 1 -- 172.21.15.32:0/3703339338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ef40cd580 msgr2=0x7f5ef41316a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:01.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5ef2800640 1 --2- 172.21.15.32:0/3703339338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ef40cd580 0x7f5ef41316a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:01.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5ef2800640 1 -- 172.21.15.32:0/3703339338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ef4145f30 msgr2=0x7f5ef4140370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:01.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5ef2800640 1 --2- 172.21.15.32:0/3703339338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ef4145f30 0x7f5ef4140370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:01.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5ef2800640 1 -- 172.21.15.32:0/3703339338 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef40c6c20 con 0x7f5ef415d8d0 2024-04-17T09:59:01.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5ef2800640 1 --2- 172.21.15.32:0/3703339338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef415d8d0 0x7f5ef41408b0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00c7b0 tx=0x7f5eec00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:01.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5eea000640 1 -- 172.21.15.32:0/3703339338 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec004b70 con 0x7f5ef415d8d0 2024-04-17T09:59:01.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5eea000640 1 -- 172.21.15.32:0/3703339338 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eec004d10 con 0x7f5ef415d8d0 2024-04-17T09:59:01.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5ef8c00640 1 -- 172.21.15.32:0/3703339338 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef40c6f00 con 0x7f5ef415d8d0 2024-04-17T09:59:01.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5eea000640 1 -- 172.21.15.32:0/3703339338 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec00aa90 con 0x7f5ef415d8d0 2024-04-17T09:59:01.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.982+0000 7f5ef8c00640 1 -- 172.21.15.32:0/3703339338 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef4131be0 con 0x7f5ef415d8d0 2024-04-17T09:59:01.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.986+0000 7f5ef8c00640 1 -- 172.21.15.32:0/3703339338 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ef4068a90 con 0x7f5ef415d8d0 2024-04-17T09:59:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.986+0000 7f5eea000640 1 -- 172.21.15.32:0/3703339338 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5eec00ac30 con 0x7f5ef415d8d0 2024-04-17T09:59:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.986+0000 7f5eea000640 1 --2- 172.21.15.32:0/3703339338 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ed0068380 0x7f5ed006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:01.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.986+0000 7f5eea000640 1 -- 172.21.15.32:0/3703339338 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4857+0+0 (secure 0 0 0) 0x7f5eec086950 con 0x7f5ef415d8d0 2024-04-17T09:59:01.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.986+0000 7f5ef1e00640 1 --2- 172.21.15.32:0/3703339338 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ed0068380 0x7f5ed006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:01.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.986+0000 7f5ef1e00640 1 --2- 172.21.15.32:0/3703339338 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ed0068380 0x7f5ed006a840 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40b2000 tx=0x7f5ee4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:01.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:01.990+0000 7f5eea000640 1 -- 172.21.15.32:0/3703339338 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5eec020070 con 0x7f5ef415d8d0 2024-04-17T09:59:02.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.158+0000 7f5ef8c00640 1 -- 172.21.15.32:0/3703339338 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ef4072f90 con 0x7f5ef415d8d0 2024-04-17T09:59:02.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.158+0000 7f5eea000640 1 -- 172.21.15.32:0/3703339338 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+12904 (secure 0 0 0) 0x7f5eec057250 con 0x7f5ef415d8d0 2024-04-17T09:59:02.156 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:59:02.156 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":78,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:59:01.410906+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:59:02.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.162+0000 7f5ef8c00640 1 -- 172.21.15.32:0/3703339338 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ed0068380 msgr2=0x7f5ed006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:02.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.162+0000 7f5ef8c00640 1 --2- 172.21.15.32:0/3703339338 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ed0068380 0x7f5ed006a840 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40b2000 tx=0x7f5ee4008040 comp rx=0 tx=0).stop 2024-04-17T09:59:02.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.162+0000 7f5ef8c00640 1 -- 172.21.15.32:0/3703339338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef415d8d0 msgr2=0x7f5ef41408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:02.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.162+0000 7f5ef8c00640 1 --2- 172.21.15.32:0/3703339338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef415d8d0 0x7f5ef41408b0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00c7b0 tx=0x7f5eec00cc70 comp rx=0 tx=0).stop 2024-04-17T09:59:02.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.162+0000 7f5ef8c00640 1 -- 172.21.15.32:0/3703339338 shutdown_connections 2024-04-17T09:59:02.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.162+0000 7f5ef8c00640 1 --2- 172.21.15.32:0/3703339338 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ed0068380 0x7f5ed006a840 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:02.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.162+0000 7f5ef8c00640 1 --2- 172.21.15.32:0/3703339338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef415d8d0 0x7f5ef41408b0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:02.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.162+0000 7f5ef8c00640 1 --2- 172.21.15.32:0/3703339338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ef4145f30 0x7f5ef4140370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:02.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.162+0000 7f5ef8c00640 1 --2- 172.21.15.32:0/3703339338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ef40cd580 0x7f5ef41316a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:02.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.162+0000 7f5ef8c00640 1 -- 172.21.15.32:0/3703339338 >> 172.21.15.32:0/3703339338 conn(0x7f5ef406ba30 msgr2=0x7f5ef406be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:02.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.162+0000 7f5ef8c00640 1 -- 172.21.15.32:0/3703339338 shutdown_connections 2024-04-17T09:59:02.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.162+0000 7f5ef8c00640 1 -- 172.21.15.32:0/3703339338 wait complete. 2024-04-17T09:59:02.179 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:59:02.179 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:59:02.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.326+0000 7fc839c00640 1 Processor -- start 2024-04-17T09:59:02.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.326+0000 7fc839c00640 1 -- start start 2024-04-17T09:59:02.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.326+0000 7fc839c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc83415b7d0 0x7fc83406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:02.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.326+0000 7fc839c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8340cc940 con 0x7fc83406f7c0 2024-04-17T09:59:02.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.326+0000 7fc839c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8340ccc20 con 0x7fc83406c140 2024-04-17T09:59:02.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.326+0000 7fc839c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc83406e0b0 con 0x7fc83415b7d0 2024-04-17T09:59:02.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.326+0000 7fc82a000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc83406f7c0 0x7fc83406ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38024/0 (socket says 172.21.15.32:38024) 2024-04-17T09:59:02.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.326+0000 7fc82a000640 1 -- 172.21.15.32:0/256136080 learned_addr learned my addr 172.21.15.32:0/256136080 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:02.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.326+0000 7fc832a00640 1 --2- 172.21.15.32:0/256136080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc83415b7d0 0x7fc83406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:02.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.326+0000 7fc832000640 1 -- 172.21.15.32:0/256136080 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4149130571 0 0) 0x7fc8340cc940 con 0x7fc83406f7c0 2024-04-17T09:59:02.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.326+0000 7fc832000640 1 -- 172.21.15.32:0/256136080 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc818005740 con 0x7fc83406f7c0 2024-04-17T09:59:02.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.326+0000 7fc832000640 1 -- 172.21.15.32:0/256136080 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3344720944 0 0) 0x7fc8340ccc20 con 0x7fc83406c140 2024-04-17T09:59:02.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.326+0000 7fc832000640 1 -- 172.21.15.32:0/256136080 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8340cc940 con 0x7fc83406c140 2024-04-17T09:59:02.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.330+0000 7fc832a00640 1 -- 172.21.15.32:0/256136080 >> v1:172.21.15.144:6789/0 conn(0x7fc83406c140 legacy=0x7fc8340ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:02.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.330+0000 7fc832000640 1 -- 172.21.15.32:0/256136080 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 756122632 0 0) 0x7fc818005740 con 0x7fc83406f7c0 2024-04-17T09:59:02.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.330+0000 7fc832a00640 1 -- 172.21.15.32:0/256136080 >> v1:172.21.15.32:6789/0 conn(0x7fc83406f7c0 legacy=0x7fc83406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:02.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.330+0000 7fc832a00640 1 -- 172.21.15.32:0/256136080 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83406e390 con 0x7fc83415b7d0 2024-04-17T09:59:02.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.330+0000 7fc832a00640 1 --2- 172.21.15.32:0/256136080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc83415b7d0 0x7fc83406e8b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc8240097f0 tx=0x7fc82401c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffcc582482304238 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:02.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.330+0000 7fc832000640 1 -- 172.21.15.32:0/256136080 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8240b0070 con 0x7fc83415b7d0 2024-04-17T09:59:02.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.330+0000 7fc832000640 1 -- 172.21.15.32:0/256136080 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8240ab400 con 0x7fc83415b7d0 2024-04-17T09:59:02.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.330+0000 7fc832000640 1 -- 172.21.15.32:0/256136080 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c001680 con 0x7fc83415b7d0 2024-04-17T09:59:02.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.330+0000 7fc839c00640 1 -- 172.21.15.32:0/256136080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc83415b7d0 msgr2=0x7fc83406e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:02.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.330+0000 7fc839c00640 1 --2- 172.21.15.32:0/256136080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc83415b7d0 0x7fc83406e8b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc8240097f0 tx=0x7fc82401c370 comp rx=0 tx=0).stop 2024-04-17T09:59:02.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.330+0000 7fc839c00640 1 -- 172.21.15.32:0/256136080 shutdown_connections 2024-04-17T09:59:02.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.330+0000 7fc839c00640 1 --2- 172.21.15.32:0/256136080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc83415b7d0 0x7fc83406e8b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:02.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.330+0000 7fc839c00640 1 -- 172.21.15.32:0/256136080 >> 172.21.15.32:0/256136080 conn(0x7fc8340b2120 msgr2=0x7fc8340b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:02.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.330+0000 7fc839c00640 1 -- 172.21.15.32:0/256136080 shutdown_connections 2024-04-17T09:59:02.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.330+0000 7fc839c00640 1 -- 172.21.15.32:0/256136080 wait complete. 2024-04-17T09:59:02.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.334+0000 7fc839c00640 1 Processor -- start 2024-04-17T09:59:02.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.334+0000 7fc839c00640 1 -- start start 2024-04-17T09:59:02.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.334+0000 7fc839c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc83406c140 0x7fc83414b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:02.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.334+0000 7fc839c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc83406f7c0 0x7fc834148250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:02.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.334+0000 7fc839c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc83415b7d0 0x7fc8340ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:02.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.334+0000 7fc839c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc8340727f0 con 0x7fc83406f7c0 2024-04-17T09:59:02.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.334+0000 7fc839c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc834072670 con 0x7fc83406c140 2024-04-17T09:59:02.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.334+0000 7fc839c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc834072970 con 0x7fc83415b7d0 2024-04-17T09:59:02.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.334+0000 7fc82a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc83406f7c0 0x7fc834148250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:02.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.334+0000 7fc832a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc83406c140 0x7fc83414b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:02.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.334+0000 7fc82a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc83406f7c0 0x7fc834148250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34474/0 (socket says 172.21.15.32:34474) 2024-04-17T09:59:02.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.334+0000 7fc82a000640 1 -- 172.21.15.32:0/3742982743 learned_addr learned my addr 172.21.15.32:0/3742982743 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:02.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.334+0000 7fc833400640 1 --2- 172.21.15.32:0/3742982743 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc83415b7d0 0x7fc8340ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:02.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.334+0000 7fc832a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc83406c140 0x7fc83414b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:42514/0 (socket says 172.21.15.32:42514) 2024-04-17T09:59:02.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.334+0000 7fc82a000640 1 -- 172.21.15.32:0/3742982743 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc83415b7d0 msgr2=0x7fc8340ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:02.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.334+0000 7fc82a000640 1 --2- 172.21.15.32:0/3742982743 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc83415b7d0 0x7fc8340ae780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:02.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.334+0000 7fc82a000640 1 -- 172.21.15.32:0/3742982743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc83406c140 msgr2=0x7fc83414b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:02.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.334+0000 7fc82a000640 1 --2- 172.21.15.32:0/3742982743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc83406c140 0x7fc83414b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:02.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.334+0000 7fc82a000640 1 -- 172.21.15.32:0/3742982743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc834148790 con 0x7fc83406f7c0 2024-04-17T09:59:02.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.334+0000 7fc82a000640 1 --2- 172.21.15.32:0/3742982743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc83406f7c0 0x7fc834148250 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00c5d0 tx=0x7fc81c00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:02.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.334+0000 7fc82b400640 1 -- 172.21.15.32:0/3742982743 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c017070 con 0x7fc83406f7c0 2024-04-17T09:59:02.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.334+0000 7fc82b400640 1 -- 172.21.15.32:0/3742982743 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc81c005cf0 con 0x7fc83406f7c0 2024-04-17T09:59:02.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.334+0000 7fc839c00640 1 -- 172.21.15.32:0/3742982743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8340af010 con 0x7fc83406f7c0 2024-04-17T09:59:02.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.338+0000 7fc82b400640 1 -- 172.21.15.32:0/3742982743 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c0072b0 con 0x7fc83406f7c0 2024-04-17T09:59:02.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.338+0000 7fc839c00640 1 -- 172.21.15.32:0/3742982743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc834156230 con 0x7fc83406f7c0 2024-04-17T09:59:02.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.338+0000 7fc839c00640 1 -- 172.21.15.32:0/3742982743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc834155d40 con 0x7fc83406f7c0 2024-04-17T09:59:02.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.338+0000 7fc82b400640 1 -- 172.21.15.32:0/3742982743 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc81c01a070 con 0x7fc83406f7c0 2024-04-17T09:59:02.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.338+0000 7fc82b400640 1 --2- 172.21.15.32:0/3742982743 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc810068380 0x7fc81006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:02.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.338+0000 7fc82b400640 1 -- 172.21.15.32:0/3742982743 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4857+0+0 (secure 0 0 0) 0x7fc81c085b50 con 0x7fc83406f7c0 2024-04-17T09:59:02.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.338+0000 7fc832a00640 1 --2- 172.21.15.32:0/3742982743 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc810068380 0x7fc81006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:02.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.338+0000 7fc832a00640 1 --2- 172.21.15.32:0/3742982743 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc810068380 0x7fc81006a840 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc8240ad000 tx=0x7fc8240a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:02.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.342+0000 7fc82b400640 1 -- 172.21.15.32:0/3742982743 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc81c055e00 con 0x7fc83406f7c0 2024-04-17T09:59:02.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.514+0000 7fc839c00640 1 -- 172.21.15.32:0/3742982743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc834068a90 con 0x7fc83406f7c0 2024-04-17T09:59:02.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.514+0000 7fc82b400640 1 -- 172.21.15.32:0/3742982743 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7fc81c05a100 con 0x7fc83406f7c0 2024-04-17T09:59:02.513 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:59:02.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.518+0000 7fc839c00640 1 -- 172.21.15.32:0/3742982743 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc810068380 msgr2=0x7fc81006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:02.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.518+0000 7fc839c00640 1 --2- 172.21.15.32:0/3742982743 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc810068380 0x7fc81006a840 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc8240ad000 tx=0x7fc8240a6000 comp rx=0 tx=0).stop 2024-04-17T09:59:02.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.518+0000 7fc839c00640 1 -- 172.21.15.32:0/3742982743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc83406f7c0 msgr2=0x7fc834148250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:02.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.518+0000 7fc839c00640 1 --2- 172.21.15.32:0/3742982743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc83406f7c0 0x7fc834148250 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00c5d0 tx=0x7fc81c00ca90 comp rx=0 tx=0).stop 2024-04-17T09:59:02.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.518+0000 7fc839c00640 1 -- 172.21.15.32:0/3742982743 shutdown_connections 2024-04-17T09:59:02.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.518+0000 7fc839c00640 1 --2- 172.21.15.32:0/3742982743 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc810068380 0x7fc81006a840 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:02.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.518+0000 7fc839c00640 1 --2- 172.21.15.32:0/3742982743 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc83415b7d0 0x7fc8340ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:02.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.518+0000 7fc839c00640 1 --2- 172.21.15.32:0/3742982743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc83406f7c0 0x7fc834148250 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:02.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.518+0000 7fc839c00640 1 --2- 172.21.15.32:0/3742982743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc83406c140 0x7fc83414b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:02.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.518+0000 7fc839c00640 1 -- 172.21.15.32:0/3742982743 >> 172.21.15.32:0/3742982743 conn(0x7fc8340b2120 msgr2=0x7fc83414b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:02.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.518+0000 7fc839c00640 1 -- 172.21.15.32:0/3742982743 shutdown_connections 2024-04-17T09:59:02.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.518+0000 7fc839c00640 1 -- 172.21.15.32:0/3742982743 wait complete. 2024-04-17T09:59:02.534 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T09:59:02.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.678+0000 7f5631a00640 1 Processor -- start 2024-04-17T09:59:02.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.678+0000 7f5631a00640 1 -- start start 2024-04-17T09:59:02.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.678+0000 7f5631a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f562c15c5c0 0x7f562c0cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:02.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.678+0000 7f5631a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f562c09b100 0x7f562c0b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:02.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.678+0000 7f5631a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f562c0d05e0 con 0x7f562c0d00b0 2024-04-17T09:59:02.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.678+0000 7f5631a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f562c0d11f0 con 0x7f562c15c5c0 2024-04-17T09:59:02.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.678+0000 7f5631a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f562c0d08c0 con 0x7f562c09b100 2024-04-17T09:59:02.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.678+0000 7f562aa00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f562c15c5c0 0x7f562c0cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:02.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.678+0000 7f562aa00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f562c15c5c0 0x7f562c0cde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34486/0 (socket says 172.21.15.32:34486) 2024-04-17T09:59:02.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.678+0000 7f562aa00640 1 -- 172.21.15.32:0/1530251955 learned_addr learned my addr 172.21.15.32:0/1530251955 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:02.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.678+0000 7f562b400640 1 --2- 172.21.15.32:0/1530251955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f562c09b100 0x7f562c0b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:02.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.678+0000 7f562aa00640 1 -- 172.21.15.32:0/1530251955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f562c09b100 msgr2=0x7f562c0b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:02.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.678+0000 7f562aa00640 1 --2- 172.21.15.32:0/1530251955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f562c09b100 0x7f562c0b31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:02.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.678+0000 7f562aa00640 1 -- 172.21.15.32:0/1530251955 >> v1:172.21.15.144:6789/0 conn(0x7f562c0d00b0 legacy=0x7f562c09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:02.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.678+0000 7f562aa00640 1 -- 172.21.15.32:0/1530251955 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562c0d0ba0 con 0x7f562c15c5c0 2024-04-17T09:59:02.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.678+0000 7f562b400640 1 --2- 172.21.15.32:0/1530251955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f562c09b100 0x7f562c0b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:02.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.682+0000 7f562aa00640 1 --2- 172.21.15.32:0/1530251955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f562c15c5c0 0x7f562c0cde50 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f561c009870 tx=0x7f561c01ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973527d9e8c479a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:02.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.682+0000 7f562a000640 1 -- 172.21.15.32:0/1530251955 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c0b5070 con 0x7f562c15c5c0 2024-04-17T09:59:02.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.682+0000 7f562a000640 1 -- 172.21.15.32:0/1530251955 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f561c0a8a50 con 0x7f562c15c5c0 2024-04-17T09:59:02.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.682+0000 7f562a000640 1 -- 172.21.15.32:0/1530251955 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c0a8e20 con 0x7f562c15c5c0 2024-04-17T09:59:02.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.682+0000 7f5631a00640 1 -- 172.21.15.32:0/1530251955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f562c15c5c0 msgr2=0x7f562c0cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:02.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.682+0000 7f5631a00640 1 --2- 172.21.15.32:0/1530251955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f562c15c5c0 0x7f562c0cde50 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f561c009870 tx=0x7f561c01ed50 comp rx=0 tx=0).stop 2024-04-17T09:59:02.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.682+0000 7f5631a00640 1 -- 172.21.15.32:0/1530251955 shutdown_connections 2024-04-17T09:59:02.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.682+0000 7f5631a00640 1 --2- 172.21.15.32:0/1530251955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f562c09b100 0x7f562c0b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:02.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.682+0000 7f5631a00640 1 --2- 172.21.15.32:0/1530251955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f562c15c5c0 0x7f562c0cde50 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:02.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.682+0000 7f5631a00640 1 -- 172.21.15.32:0/1530251955 >> 172.21.15.32:0/1530251955 conn(0x7f562c0b3fd0 msgr2=0x7f562c0b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:02.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.682+0000 7f5631a00640 1 -- 172.21.15.32:0/1530251955 shutdown_connections 2024-04-17T09:59:02.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.682+0000 7f5631a00640 1 -- 172.21.15.32:0/1530251955 wait complete. 2024-04-17T09:59:02.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f5631a00640 1 Processor -- start 2024-04-17T09:59:02.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f5631a00640 1 -- start start 2024-04-17T09:59:02.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f5631a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f562c09b100 0x7f562c0cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:02.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f5631a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f562c0d00b0 0x7f562c0ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:02.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f5631a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f562c15c5c0 0x7f562c15e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:02.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f5631a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f562c0ce510 con 0x7f562c15c5c0 2024-04-17T09:59:02.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f5631a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f562c0ce390 con 0x7f562c0d00b0 2024-04-17T09:59:02.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f5631a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f562c15c9a0 con 0x7f562c09b100 2024-04-17T09:59:02.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f562b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f562c15c5c0 0x7f562c15e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:02.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f562b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f562c15c5c0 0x7f562c15e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34494/0 (socket says 172.21.15.32:34494) 2024-04-17T09:59:02.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f562b400640 1 -- 172.21.15.32:0/4179185513 learned_addr learned my addr 172.21.15.32:0/4179185513 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:02.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f5622000640 1 --2- 172.21.15.32:0/4179185513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f562c0d00b0 0x7f562c0ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:02.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f562aa00640 1 --2- 172.21.15.32:0/4179185513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f562c09b100 0x7f562c0cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:02.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f562b400640 1 -- 172.21.15.32:0/4179185513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f562c09b100 msgr2=0x7f562c0cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:02.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f562b400640 1 --2- 172.21.15.32:0/4179185513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f562c09b100 0x7f562c0cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:02.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f562b400640 1 -- 172.21.15.32:0/4179185513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f562c0d00b0 msgr2=0x7f562c0ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:02.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f562b400640 1 --2- 172.21.15.32:0/4179185513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f562c0d00b0 0x7f562c0ca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:02.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f562b400640 1 -- 172.21.15.32:0/4179185513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562c0c6580 con 0x7f562c15c5c0 2024-04-17T09:59:02.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f5622000640 1 --2- 172.21.15.32:0/4179185513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f562c0d00b0 0x7f562c0ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:02.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f562b400640 1 --2- 172.21.15.32:0/4179185513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f562c15c5c0 0x7f562c15e3e0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f5624009f70 tx=0x7f562400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:02.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f5623400640 1 -- 172.21.15.32:0/4179185513 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562401a070 con 0x7f562c15c5c0 2024-04-17T09:59:02.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f5623400640 1 -- 172.21.15.32:0/4179185513 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5624003bc0 con 0x7f562c15c5c0 2024-04-17T09:59:02.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f5623400640 1 -- 172.21.15.32:0/4179185513 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624004030 con 0x7f562c15c5c0 2024-04-17T09:59:02.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f5631a00640 1 -- 172.21.15.32:0/4179185513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f562c0c6860 con 0x7f562c15c5c0 2024-04-17T09:59:02.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.686+0000 7f5631a00640 1 -- 172.21.15.32:0/4179185513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f562c0c6e20 con 0x7f562c15c5c0 2024-04-17T09:59:02.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.690+0000 7f5631a00640 1 -- 172.21.15.32:0/4179185513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f562c068a90 con 0x7f562c15c5c0 2024-04-17T09:59:02.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.690+0000 7f5623400640 1 -- 172.21.15.32:0/4179185513 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f562401d070 con 0x7f562c15c5c0 2024-04-17T09:59:02.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.690+0000 7f5623400640 1 --2- 172.21.15.32:0/4179185513 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5608068380 0x7f560806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:02.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.690+0000 7f5623400640 1 -- 172.21.15.32:0/4179185513 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4857+0+0 (secure 0 0 0) 0x7f5624086980 con 0x7f562c15c5c0 2024-04-17T09:59:02.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.690+0000 7f562aa00640 1 --2- 172.21.15.32:0/4179185513 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5608068380 0x7f560806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:02.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.690+0000 7f562aa00640 1 --2- 172.21.15.32:0/4179185513 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5608068380 0x7f560806a840 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f561c0b2000 tx=0x7f561c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:02.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.694+0000 7f5623400640 1 -- 172.21.15.32:0/4179185513 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5624010070 con 0x7f562c15c5c0 2024-04-17T09:59:02.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.862+0000 7f5631a00640 1 -- 172.21.15.32:0/4179185513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f562c0b4c30 con 0x7f562c15c5c0 2024-04-17T09:59:02.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.862+0000 7f5623400640 1 -- 172.21.15.32:0/4179185513 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) ==== 84+0+65 (secure 0 0 0) 0x7f5624057280 con 0x7f562c15c5c0 2024-04-17T09:59:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:59:02.861 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-17T09:59:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T09:59:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-04-17T09:59:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-04-17T09:59:02.861 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:02.861 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-17T09:59:02.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.866+0000 7f5631a00640 1 -- 172.21.15.32:0/4179185513 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5608068380 msgr2=0x7f560806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:02.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.866+0000 7f5631a00640 1 --2- 172.21.15.32:0/4179185513 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5608068380 0x7f560806a840 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f561c0b2000 tx=0x7f561c008040 comp rx=0 tx=0).stop 2024-04-17T09:59:02.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.866+0000 7f5631a00640 1 -- 172.21.15.32:0/4179185513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f562c15c5c0 msgr2=0x7f562c15e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:02.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.866+0000 7f5631a00640 1 --2- 172.21.15.32:0/4179185513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f562c15c5c0 0x7f562c15e3e0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f5624009f70 tx=0x7f562400d560 comp rx=0 tx=0).stop 2024-04-17T09:59:02.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.866+0000 7f5631a00640 1 -- 172.21.15.32:0/4179185513 shutdown_connections 2024-04-17T09:59:02.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.866+0000 7f5631a00640 1 --2- 172.21.15.32:0/4179185513 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5608068380 0x7f560806a840 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:02.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.866+0000 7f5631a00640 1 --2- 172.21.15.32:0/4179185513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f562c15c5c0 0x7f562c15e3e0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:02.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.866+0000 7f5631a00640 1 --2- 172.21.15.32:0/4179185513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f562c0d00b0 0x7f562c0ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:02.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.866+0000 7f5631a00640 1 --2- 172.21.15.32:0/4179185513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f562c09b100 0x7f562c0cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:02.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.866+0000 7f5631a00640 1 -- 172.21.15.32:0/4179185513 >> 172.21.15.32:0/4179185513 conn(0x7f562c0b3fd0 msgr2=0x7f562c0b7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:02.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.866+0000 7f5631a00640 1 -- 172.21.15.32:0/4179185513 shutdown_connections 2024-04-17T09:59:02.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:02.866+0000 7f5631a00640 1 -- 172.21.15.32:0/4179185513 wait complete. 2024-04-17T09:59:02.883 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T09:59:02.883 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T09:59:03.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.030+0000 7f2f7a200640 1 Processor -- start 2024-04-17T09:59:03.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.030+0000 7f2f7a200640 1 -- start start 2024-04-17T09:59:03.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.030+0000 7f2f7a200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2f741460a0 0x7f2f740ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:03.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.030+0000 7f2f7a200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f7406e6f0 0x7f2f74071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:03.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.030+0000 7f2f7a200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2f740722f0 0x7f2f7415cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:03.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.030+0000 7f2f7a200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2f7415de00 con 0x7f2f741460a0 2024-04-17T09:59:03.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.030+0000 7f2f7a200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f2f7406eeb0 con 0x7f2f740722f0 2024-04-17T09:59:03.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.030+0000 7f2f7a200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2f7406f190 con 0x7f2f7406e6f0 2024-04-17T09:59:03.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.030+0000 7f2f73400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2f741460a0 0x7f2f740ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:03.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.030+0000 7f2f73400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2f741460a0 0x7f2f740ceb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34508/0 (socket says 172.21.15.32:34508) 2024-04-17T09:59:03.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.030+0000 7f2f73400640 1 -- 172.21.15.32:0/340970085 learned_addr learned my addr 172.21.15.32:0/340970085 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:03.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.030+0000 7f2f73e00640 1 --2- 172.21.15.32:0/340970085 >> v2:172.21.15.144:3300/0 conn(0x7f2f740722f0 0x7f2f7415cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:03.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.030+0000 7f2f6be00640 1 --2- 172.21.15.32:0/340970085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f7406e6f0 0x7f2f74071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:03.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.030+0000 7f2f73400640 1 -- 172.21.15.32:0/340970085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f7406e6f0 msgr2=0x7f2f74071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:03.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.030+0000 7f2f73400640 1 --2- 172.21.15.32:0/340970085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f7406e6f0 0x7f2f74071db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:03.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.030+0000 7f2f73400640 1 -- 172.21.15.32:0/340970085 >> v2:172.21.15.144:3300/0 conn(0x7f2f740722f0 msgr2=0x7f2f7415cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:03.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.030+0000 7f2f73400640 1 --2- 172.21.15.32:0/340970085 >> v2:172.21.15.144:3300/0 conn(0x7f2f740722f0 0x7f2f7415cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:03.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.030+0000 7f2f73400640 1 -- 172.21.15.32:0/340970085 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f7406f470 con 0x7f2f741460a0 2024-04-17T09:59:03.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.034+0000 7f2f73400640 1 --2- 172.21.15.32:0/340970085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f741460a0 0x7f2f740ceb40 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f2f740cfbc0 tx=0x7f2f6401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ca1587956fddda server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:03.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.034+0000 7f2f72a00640 1 -- 172.21.15.32:0/340970085 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f640b0070 con 0x7f2f741460a0 2024-04-17T09:59:03.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.034+0000 7f2f72a00640 1 -- 172.21.15.32:0/340970085 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f640ab600 con 0x7f2f741460a0 2024-04-17T09:59:03.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.034+0000 7f2f72a00640 1 -- 172.21.15.32:0/340970085 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f640a4e60 con 0x7f2f741460a0 2024-04-17T09:59:03.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.034+0000 7f2f7a200640 1 -- 172.21.15.32:0/340970085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f741460a0 msgr2=0x7f2f740ceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:03.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.034+0000 7f2f7a200640 1 --2- 172.21.15.32:0/340970085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f741460a0 0x7f2f740ceb40 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f2f740cfbc0 tx=0x7f2f6401c520 comp rx=0 tx=0).stop 2024-04-17T09:59:03.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.034+0000 7f2f7a200640 1 -- 172.21.15.32:0/340970085 shutdown_connections 2024-04-17T09:59:03.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.034+0000 7f2f7a200640 1 --2- 172.21.15.32:0/340970085 >> v2:172.21.15.144:3300/0 conn(0x7f2f740722f0 0x7f2f7415cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:03.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.034+0000 7f2f7a200640 1 --2- 172.21.15.32:0/340970085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f7406e6f0 0x7f2f74071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:03.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.034+0000 7f2f7a200640 1 --2- 172.21.15.32:0/340970085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f741460a0 0x7f2f740ceb40 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:03.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.034+0000 7f2f7a200640 1 -- 172.21.15.32:0/340970085 >> 172.21.15.32:0/340970085 conn(0x7f2f740b2190 msgr2=0x7f2f740b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:03.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.034+0000 7f2f7a200640 1 -- 172.21.15.32:0/340970085 shutdown_connections 2024-04-17T09:59:03.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.034+0000 7f2f7a200640 1 -- 172.21.15.32:0/340970085 wait complete. 2024-04-17T09:59:03.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f7a200640 1 Processor -- start 2024-04-17T09:59:03.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f7a200640 1 -- start start 2024-04-17T09:59:03.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f7a200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f7406e6f0 0x7f2f7413c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:03.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f7a200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f740722f0 0x7f2f7415bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:03.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f7a200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f741460a0 0x7f2f7415c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:03.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f7a200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7415e8e0 con 0x7f2f7406e6f0 2024-04-17T09:59:03.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f7a200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7415e760 con 0x7f2f740722f0 2024-04-17T09:59:03.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f7a200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2f7415ea60 con 0x7f2f741460a0 2024-04-17T09:59:03.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f73400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f7406e6f0 0x7f2f7413c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:03.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f73400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f7406e6f0 0x7f2f7413c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34514/0 (socket says 172.21.15.32:34514) 2024-04-17T09:59:03.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f73400640 1 -- 172.21.15.32:0/241428010 learned_addr learned my addr 172.21.15.32:0/241428010 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:03.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f6be00640 1 --2- 172.21.15.32:0/241428010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f740722f0 0x7f2f7415bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:03.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f73e00640 1 --2- 172.21.15.32:0/241428010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f741460a0 0x7f2f7415c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:03.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f73400640 1 -- 172.21.15.32:0/241428010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f741460a0 msgr2=0x7f2f7415c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:03.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f73400640 1 --2- 172.21.15.32:0/241428010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f741460a0 0x7f2f7415c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:03.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f73400640 1 -- 172.21.15.32:0/241428010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f740722f0 msgr2=0x7f2f7415bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:03.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f73400640 1 --2- 172.21.15.32:0/241428010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f740722f0 0x7f2f7415bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:03.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f73400640 1 -- 172.21.15.32:0/241428010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f740aef80 con 0x7f2f7406e6f0 2024-04-17T09:59:03.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f73400640 1 --2- 172.21.15.32:0/241428010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f7406e6f0 0x7f2f7413c1c0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f2f640ae040 tx=0x7f2f6401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:03.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f6b400640 1 -- 172.21.15.32:0/241428010 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f640b0040 con 0x7f2f7406e6f0 2024-04-17T09:59:03.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f6b400640 1 -- 172.21.15.32:0/241428010 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f640bc070 con 0x7f2f7406e6f0 2024-04-17T09:59:03.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f6b400640 1 -- 172.21.15.32:0/241428010 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f640b6680 con 0x7f2f7406e6f0 2024-04-17T09:59:03.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f7a200640 1 -- 172.21.15.32:0/241428010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f741424f0 con 0x7f2f7406e6f0 2024-04-17T09:59:03.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.038+0000 7f2f7a200640 1 -- 172.21.15.32:0/241428010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f74142ab0 con 0x7f2f7406e6f0 2024-04-17T09:59:03.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.042+0000 7f2f7a200640 1 -- 172.21.15.32:0/241428010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f74068a90 con 0x7f2f7406e6f0 2024-04-17T09:59:03.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.042+0000 7f2f6b400640 1 -- 172.21.15.32:0/241428010 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2f640b6820 con 0x7f2f7406e6f0 2024-04-17T09:59:03.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.042+0000 7f2f6b400640 1 --2- 172.21.15.32:0/241428010 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2f50068380 0x7f2f5006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:03.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.042+0000 7f2f6b400640 1 -- 172.21.15.32:0/241428010 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4857+0+0 (secure 0 0 0) 0x7f2f64121310 con 0x7f2f7406e6f0 2024-04-17T09:59:03.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.042+0000 7f2f6be00640 1 --2- 172.21.15.32:0/241428010 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2f50068380 0x7f2f5006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:03.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.042+0000 7f2f6be00640 1 --2- 172.21.15.32:0/241428010 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2f50068380 0x7f2f5006a840 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2f74142140 tx=0x7f2f5c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.046+0000 7f2f6b400640 1 -- 172.21.15.32:0/241428010 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2f64016110 con 0x7f2f7406e6f0 2024-04-17T09:59:03.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.210+0000 7f2f7a200640 1 -- 172.21.15.32:0/241428010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2f74145040 con 0x7f2f7406e6f0 2024-04-17T09:59:03.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.410+0000 7f2f6b400640 1 -- 172.21.15.32:0/241428010 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) ==== 138+0+0 (secure 0 0 0) 0x7f2f640f1c90 con 0x7f2f7406e6f0 2024-04-17T09:59:03.407 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-04-17T09:59:03.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.410+0000 7f2f7a200640 1 -- 172.21.15.32:0/241428010 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2f50068380 msgr2=0x7f2f5006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:03.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.410+0000 7f2f7a200640 1 --2- 172.21.15.32:0/241428010 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2f50068380 0x7f2f5006a840 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2f74142140 tx=0x7f2f5c008040 comp rx=0 tx=0).stop 2024-04-17T09:59:03.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.410+0000 7f2f7a200640 1 -- 172.21.15.32:0/241428010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f7406e6f0 msgr2=0x7f2f7413c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:03.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.410+0000 7f2f7a200640 1 --2- 172.21.15.32:0/241428010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f7406e6f0 0x7f2f7413c1c0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f2f640ae040 tx=0x7f2f6401ad60 comp rx=0 tx=0).stop 2024-04-17T09:59:03.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.410+0000 7f2f7a200640 1 -- 172.21.15.32:0/241428010 shutdown_connections 2024-04-17T09:59:03.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.410+0000 7f2f7a200640 1 --2- 172.21.15.32:0/241428010 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2f50068380 0x7f2f5006a840 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:03.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.410+0000 7f2f7a200640 1 --2- 172.21.15.32:0/241428010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f741460a0 0x7f2f7415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:03.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.410+0000 7f2f7a200640 1 --2- 172.21.15.32:0/241428010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f740722f0 0x7f2f7415bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:03.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.410+0000 7f2f7a200640 1 --2- 172.21.15.32:0/241428010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f7406e6f0 0x7f2f7413c1c0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:03.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.410+0000 7f2f7a200640 1 -- 172.21.15.32:0/241428010 >> 172.21.15.32:0/241428010 conn(0x7f2f740b2190 msgr2=0x7f2f740b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:03.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.410+0000 7f2f7a200640 1 -- 172.21.15.32:0/241428010 shutdown_connections 2024-04-17T09:59:03.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.410+0000 7f2f7a200640 1 -- 172.21.15.32:0/241428010 wait complete. 2024-04-17T09:59:03.435 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-17T09:59:03.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.586+0000 7f3936000640 1 Processor -- start 2024-04-17T09:59:03.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.586+0000 7f3936000640 1 -- start start 2024-04-17T09:59:03.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.586+0000 7f3936000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f393006ce50 0x7f393006d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:03.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.586+0000 7f3936000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f393015eec0 0x7f393006da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:03.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.586+0000 7f3936000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39300d1920 con 0x7f393015eae0 2024-04-17T09:59:03.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.586+0000 7f3936000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f393009a9f0 con 0x7f393015eec0 2024-04-17T09:59:03.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.586+0000 7f3936000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f393009b570 con 0x7f393006ce50 2024-04-17T09:59:03.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.586+0000 7f392fe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f393015eec0 0x7f393006da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:03.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.586+0000 7f392f400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f393015eae0 0x7f39300d1210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38028/0 (socket says 172.21.15.32:38028) 2024-04-17T09:59:03.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.586+0000 7f392f400640 1 -- 172.21.15.32:0/3141031027 learned_addr learned my addr 172.21.15.32:0/3141031027 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:03.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.586+0000 7f3926a00640 1 --2- 172.21.15.32:0/3141031027 >> v2:172.21.15.144:3300/0 conn(0x7f393006ce50 0x7f393006d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:03.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.586+0000 7f392fe00640 1 -- 172.21.15.32:0/3141031027 >> v2:172.21.15.144:3300/0 conn(0x7f393006ce50 msgr2=0x7f393006d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:03.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.586+0000 7f392fe00640 1 --2- 172.21.15.32:0/3141031027 >> v2:172.21.15.144:3300/0 conn(0x7f393006ce50 0x7f393006d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:03.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.586+0000 7f392fe00640 1 -- 172.21.15.32:0/3141031027 >> v1:172.21.15.32:6789/0 conn(0x7f393015eae0 legacy=0x7f39300d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:03.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.586+0000 7f392fe00640 1 -- 172.21.15.32:0/3141031027 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393009ac40 con 0x7f393015eec0 2024-04-17T09:59:03.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.586+0000 7f3926a00640 1 --2- 172.21.15.32:0/3141031027 >> v2:172.21.15.144:3300/0 conn(0x7f393006ce50 0x7f393006d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:59:03.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.586+0000 7f392ea00640 1 -- 172.21.15.32:0/3141031027 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1430404658 0 0) 0x7f39300d1920 con 0x7f393015eae0 2024-04-17T09:59:03.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.590+0000 7f392fe00640 1 --2- 172.21.15.32:0/3141031027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f393015eec0 0x7f393006da90 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f393006ee60 tx=0x7f392801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978307f4d8f512c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:03.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.590+0000 7f392ea00640 1 -- 172.21.15.32:0/3141031027 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39280b0070 con 0x7f393015eec0 2024-04-17T09:59:03.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.590+0000 7f392ea00640 1 -- 172.21.15.32:0/3141031027 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39280ab600 con 0x7f393015eec0 2024-04-17T09:59:03.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.590+0000 7f392ea00640 1 -- 172.21.15.32:0/3141031027 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39280ab9d0 con 0x7f393015eec0 2024-04-17T09:59:03.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.590+0000 7f3936000640 1 -- 172.21.15.32:0/3141031027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f393015eec0 msgr2=0x7f393006da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:03.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.590+0000 7f3936000640 1 --2- 172.21.15.32:0/3141031027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f393015eec0 0x7f393006da90 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f393006ee60 tx=0x7f392801c330 comp rx=0 tx=0).stop 2024-04-17T09:59:03.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.590+0000 7f3936000640 1 -- 172.21.15.32:0/3141031027 shutdown_connections 2024-04-17T09:59:03.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.590+0000 7f3936000640 1 --2- 172.21.15.32:0/3141031027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f393015eec0 0x7f393006da90 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:03.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.590+0000 7f3936000640 1 --2- 172.21.15.32:0/3141031027 >> v2:172.21.15.144:3300/0 conn(0x7f393006ce50 0x7f393006d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:03.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.590+0000 7f3936000640 1 -- 172.21.15.32:0/3141031027 >> 172.21.15.32:0/3141031027 conn(0x7f39300b4080 msgr2=0x7f39300b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:03.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.590+0000 7f3936000640 1 -- 172.21.15.32:0/3141031027 shutdown_connections 2024-04-17T09:59:03.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.590+0000 7f3936000640 1 -- 172.21.15.32:0/3141031027 wait complete. 2024-04-17T09:59:03.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.590+0000 7f3936000640 1 Processor -- start 2024-04-17T09:59:03.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.590+0000 7f3936000640 1 -- start start 2024-04-17T09:59:03.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.594+0000 7f3936000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f393006ce50 0x7f39300b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:03.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.594+0000 7f3936000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f393015eae0 0x7f3930156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:03.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.594+0000 7f3936000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f393015eec0 0x7f393015f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:03.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.594+0000 7f3936000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f39300cd780 con 0x7f393006ce50 2024-04-17T09:59:03.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.594+0000 7f3936000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f39300cd600 con 0x7f393015eae0 2024-04-17T09:59:03.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.594+0000 7f3936000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f39300cd900 con 0x7f393015eec0 2024-04-17T09:59:03.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.594+0000 7f392f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f393006ce50 0x7f39300b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:03.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.594+0000 7f392f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f393006ce50 0x7f39300b5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34534/0 (socket says 172.21.15.32:34534) 2024-04-17T09:59:03.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.594+0000 7f392f400640 1 -- 172.21.15.32:0/4062381629 learned_addr learned my addr 172.21.15.32:0/4062381629 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:03.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.594+0000 7f392fe00640 1 --2- 172.21.15.32:0/4062381629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f393015eec0 0x7f393015f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:03.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.594+0000 7f3926a00640 1 --2- 172.21.15.32:0/4062381629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f393015eae0 0x7f3930156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:03.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.594+0000 7f392f400640 1 -- 172.21.15.32:0/4062381629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f393015eec0 msgr2=0x7f393015f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:03.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.594+0000 7f392f400640 1 --2- 172.21.15.32:0/4062381629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f393015eec0 0x7f393015f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:03.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.594+0000 7f392f400640 1 -- 172.21.15.32:0/4062381629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f393015eae0 msgr2=0x7f3930156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:03.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.594+0000 7f392f400640 1 --2- 172.21.15.32:0/4062381629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f393015eae0 0x7f3930156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:03.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.594+0000 7f392f400640 1 -- 172.21.15.32:0/4062381629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393015d400 con 0x7f393006ce50 2024-04-17T09:59:03.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.594+0000 7f392f400640 1 --2- 172.21.15.32:0/4062381629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f393006ce50 0x7f39300b5710 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f392000c7c0 tx=0x7f392000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:03.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.594+0000 7f3927e00640 1 -- 172.21.15.32:0/4062381629 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920018070 con 0x7f393006ce50 2024-04-17T09:59:03.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.594+0000 7f3927e00640 1 -- 172.21.15.32:0/4062381629 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f392000a640 con 0x7f393006ce50 2024-04-17T09:59:03.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.594+0000 7f3927e00640 1 -- 172.21.15.32:0/4062381629 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392000aa30 con 0x7f393006ce50 2024-04-17T09:59:03.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.594+0000 7f3936000640 1 -- 172.21.15.32:0/4062381629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f393015c880 con 0x7f393006ce50 2024-04-17T09:59:03.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.594+0000 7f3936000640 1 -- 172.21.15.32:0/4062381629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f393015ce40 con 0x7f393006ce50 2024-04-17T09:59:03.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.594+0000 7f3936000640 1 -- 172.21.15.32:0/4062381629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3930068a90 con 0x7f393006ce50 2024-04-17T09:59:03.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.598+0000 7f3927e00640 1 -- 172.21.15.32:0/4062381629 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f392001b070 con 0x7f393006ce50 2024-04-17T09:59:03.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.598+0000 7f3927e00640 1 --2- 172.21.15.32:0/4062381629 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f390c068380 0x7f390c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:03.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.598+0000 7f3927e00640 1 -- 172.21.15.32:0/4062381629 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5227+0+0 (secure 0 0 0) 0x7f39200860f0 con 0x7f393006ce50 2024-04-17T09:59:03.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.598+0000 7f3926a00640 1 --2- 172.21.15.32:0/4062381629 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f390c068380 0x7f390c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:03.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.598+0000 7f3926a00640 1 --2- 172.21.15.32:0/4062381629 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f390c068380 0x7f390c06a840 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3918006fd0 tx=0x7f3918008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:03.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.602+0000 7f3927e00640 1 -- 172.21.15.32:0/4062381629 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3920056900 con 0x7f393006ce50 2024-04-17T09:59:03.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:03.766+0000 7f3936000640 1 -- 172.21.15.32:0/4062381629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f39300b8510 con 0x7f393006ce50 2024-04-17T09:59:04.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.418+0000 7f3927e00640 1 -- 172.21.15.32:0/4062381629 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) ==== 170+0+0 (secure 0 0 0) 0x7f3920056200 con 0x7f393006ce50 2024-04-17T09:59:04.415 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-04-17T09:59:04.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.422+0000 7f3936000640 1 -- 172.21.15.32:0/4062381629 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f390c068380 msgr2=0x7f390c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:04.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.422+0000 7f3936000640 1 --2- 172.21.15.32:0/4062381629 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f390c068380 0x7f390c06a840 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3918006fd0 tx=0x7f3918008040 comp rx=0 tx=0).stop 2024-04-17T09:59:04.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.422+0000 7f3936000640 1 -- 172.21.15.32:0/4062381629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f393006ce50 msgr2=0x7f39300b5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:04.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.422+0000 7f3936000640 1 --2- 172.21.15.32:0/4062381629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f393006ce50 0x7f39300b5710 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f392000c7c0 tx=0x7f392000cc80 comp rx=0 tx=0).stop 2024-04-17T09:59:04.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.422+0000 7f3936000640 1 -- 172.21.15.32:0/4062381629 shutdown_connections 2024-04-17T09:59:04.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.422+0000 7f3936000640 1 --2- 172.21.15.32:0/4062381629 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f390c068380 0x7f390c06a840 secure :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3918006fd0 tx=0x7f3918008040 comp rx=0 tx=0).stop 2024-04-17T09:59:04.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.422+0000 7f3936000640 1 --2- 172.21.15.32:0/4062381629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f393015eec0 0x7f393015f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:04.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.422+0000 7f3936000640 1 --2- 172.21.15.32:0/4062381629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f393015eae0 0x7f3930156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:04.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.422+0000 7f3936000640 1 --2- 172.21.15.32:0/4062381629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f393006ce50 0x7f39300b5710 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:04.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.422+0000 7f3936000640 1 -- 172.21.15.32:0/4062381629 >> 172.21.15.32:0/4062381629 conn(0x7f39300b4080 msgr2=0x7f3930157cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:04.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.422+0000 7f3936000640 1 -- 172.21.15.32:0/4062381629 shutdown_connections 2024-04-17T09:59:04.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.422+0000 7f3936000640 1 -- 172.21.15.32:0/4062381629 wait complete. 2024-04-17T09:59:04.458 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T09:59:04.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.610+0000 7efec0e00640 1 Processor -- start 2024-04-17T09:59:04.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.610+0000 7efec0e00640 1 -- start start 2024-04-17T09:59:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.610+0000 7efec0e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efebc06c140 0x7efebc0b4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.610+0000 7efec0e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efebc06f290 con 0x7efebc1662c0 2024-04-17T09:59:04.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.610+0000 7efec0e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efebc06efd0 con 0x7efebc06eb60 2024-04-17T09:59:04.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.610+0000 7efec0e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7efebc1666a0 con 0x7efebc06c140 2024-04-17T09:59:04.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.610+0000 7efeb1400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efebc06c140 0x7efebc0b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:04.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.610+0000 7efeb9e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efebc1662c0 0x7efebc06ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38030/0 (socket says 172.21.15.32:38030) 2024-04-17T09:59:04.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.610+0000 7efeb9e00640 1 -- 172.21.15.32:0/40888125 learned_addr learned my addr 172.21.15.32:0/40888125 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:04.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.614+0000 7efeb1400640 1 -- 172.21.15.32:0/40888125 >> v1:172.21.15.144:6789/0 conn(0x7efebc06eb60 legacy=0x7efebc0cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:04.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.614+0000 7efeb1400640 1 -- 172.21.15.32:0/40888125 >> v1:172.21.15.32:6789/0 conn(0x7efebc1662c0 legacy=0x7efebc06ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.614+0000 7efeb1400640 1 -- 172.21.15.32:0/40888125 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efebc0b34c0 con 0x7efebc06c140 2024-04-17T09:59:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.614+0000 7efeb9400640 1 -- 172.21.15.32:0/40888125 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2767149411 0 0) 0x7efebc06f290 con 0x7efebc1662c0 2024-04-17T09:59:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.614+0000 7efeb1400640 1 --2- 172.21.15.32:0/40888125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc06c140 0x7efebc0b4280 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7efea40097c0 tx=0x7efea401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7394276538fe12a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.614+0000 7efeb9400640 1 -- 172.21.15.32:0/40888125 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea40b0070 con 0x7efebc06c140 2024-04-17T09:59:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.614+0000 7efeb9400640 1 -- 172.21.15.32:0/40888125 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea40ab600 con 0x7efebc06c140 2024-04-17T09:59:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.614+0000 7efeb9400640 1 -- 172.21.15.32:0/40888125 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea40ab9d0 con 0x7efebc06c140 2024-04-17T09:59:04.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.614+0000 7efec0e00640 1 -- 172.21.15.32:0/40888125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc06c140 msgr2=0x7efebc0b4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:04.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.614+0000 7efec0e00640 1 --2- 172.21.15.32:0/40888125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc06c140 0x7efebc0b4280 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7efea40097c0 tx=0x7efea401c470 comp rx=0 tx=0).stop 2024-04-17T09:59:04.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.614+0000 7efec0e00640 1 -- 172.21.15.32:0/40888125 shutdown_connections 2024-04-17T09:59:04.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.614+0000 7efec0e00640 1 --2- 172.21.15.32:0/40888125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc06c140 0x7efebc0b4280 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:04.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.614+0000 7efec0e00640 1 -- 172.21.15.32:0/40888125 >> 172.21.15.32:0/40888125 conn(0x7efebc0b2120 msgr2=0x7efebc0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:04.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.614+0000 7efec0e00640 1 -- 172.21.15.32:0/40888125 shutdown_connections 2024-04-17T09:59:04.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efec0e00640 1 -- 172.21.15.32:0/40888125 wait complete. 2024-04-17T09:59:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efec0e00640 1 Processor -- start 2024-04-17T09:59:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efec0e00640 1 -- start start 2024-04-17T09:59:04.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efec0e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efebc06c140 0x7efebc14feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:04.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efec0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc06eb60 0x7efebc1570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:04.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efec0e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efebc1662c0 0x7efebc157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:04.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efec0e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1709b0 con 0x7efebc06eb60 2024-04-17T09:59:04.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efec0e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efebc06fc90 con 0x7efebc1662c0 2024-04-17T09:59:04.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efec0e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efebc170b30 con 0x7efebc06c140 2024-04-17T09:59:04.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efeb1400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc06eb60 0x7efebc1570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:04.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efeb1400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc06eb60 0x7efebc1570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34552/0 (socket says 172.21.15.32:34552) 2024-04-17T09:59:04.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efeb1400640 1 -- 172.21.15.32:0/662547177 learned_addr learned my addr 172.21.15.32:0/662547177 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:04.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efeb9e00640 1 --2- 172.21.15.32:0/662547177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efebc06c140 0x7efebc14feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:04.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efeba800640 1 --2- 172.21.15.32:0/662547177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efebc1662c0 0x7efebc157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:04.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efeb1400640 1 -- 172.21.15.32:0/662547177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efebc06c140 msgr2=0x7efebc14feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:04.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efeb1400640 1 --2- 172.21.15.32:0/662547177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efebc06c140 0x7efebc14feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:04.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efeb1400640 1 -- 172.21.15.32:0/662547177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efebc1662c0 msgr2=0x7efebc157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efeb1400640 1 --2- 172.21.15.32:0/662547177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efebc1662c0 0x7efebc157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efeb1400640 1 -- 172.21.15.32:0/662547177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efebc153690 con 0x7efebc06eb60 2024-04-17T09:59:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efeb9e00640 1 --2- 172.21.15.32:0/662547177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efebc06c140 0x7efebc14feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efeba800640 1 --2- 172.21.15.32:0/662547177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efebc1662c0 0x7efebc157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efeb1400640 1 --2- 172.21.15.32:0/662547177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc06eb60 0x7efebc1570d0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7efea40ae040 tx=0x7efea401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efeb2a00640 1 -- 172.21.15.32:0/662547177 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea40b0040 con 0x7efebc06eb60 2024-04-17T09:59:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efeb2a00640 1 -- 172.21.15.32:0/662547177 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea4004030 con 0x7efebc06eb60 2024-04-17T09:59:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efec0e00640 1 -- 172.21.15.32:0/662547177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efebc15ac80 con 0x7efebc06eb60 2024-04-17T09:59:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efeb2a00640 1 -- 172.21.15.32:0/662547177 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea4004420 con 0x7efebc06eb60 2024-04-17T09:59:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.618+0000 7efec0e00640 1 -- 172.21.15.32:0/662547177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efebc15d6c0 con 0x7efebc06eb60 2024-04-17T09:59:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.622+0000 7efec0e00640 1 -- 172.21.15.32:0/662547177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efebc068a90 con 0x7efebc06eb60 2024-04-17T09:59:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.622+0000 7efeb2a00640 1 -- 172.21.15.32:0/662547177 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7efea4003620 con 0x7efebc06eb60 2024-04-17T09:59:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.622+0000 7efeb2a00640 1 --2- 172.21.15.32:0/662547177 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efe98068380 0x7efe9806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.622+0000 7efeb2a00640 1 -- 172.21.15.32:0/662547177 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5609+0+0 (secure 0 0 0) 0x7efea4121130 con 0x7efebc06eb60 2024-04-17T09:59:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.622+0000 7efeb9e00640 1 --2- 172.21.15.32:0/662547177 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efe98068380 0x7efe9806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.622+0000 7efeb9e00640 1 --2- 172.21.15.32:0/662547177 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efe98068380 0x7efe9806a840 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7efeac0012f0 tx=0x7efeac005c60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.626+0000 7efeb2a00640 1 -- 172.21.15.32:0/662547177 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7efea4016090 con 0x7efebc06eb60 2024-04-17T09:59:04.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:04.830+0000 7efec0e00640 1 -- 172.21.15.32:0/662547177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7efebc0cca80 con 0x7efebc06eb60 2024-04-17T09:59:05.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.442+0000 7efeb2a00640 1 -- 172.21.15.32:0/662547177 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v55) ==== 174+0+0 (secure 0 0 0) 0x7efea40f1740 con 0x7efebc06eb60 2024-04-17T09:59:05.443 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 12 and data pool 13 2024-04-17T09:59:05.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.450+0000 7efec0e00640 1 -- 172.21.15.32:0/662547177 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efe98068380 msgr2=0x7efe9806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:05.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.450+0000 7efec0e00640 1 --2- 172.21.15.32:0/662547177 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efe98068380 0x7efe9806a840 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7efeac0012f0 tx=0x7efeac005c60 comp rx=0 tx=0).stop 2024-04-17T09:59:05.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.450+0000 7efec0e00640 1 -- 172.21.15.32:0/662547177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc06eb60 msgr2=0x7efebc1570d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:05.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.450+0000 7efec0e00640 1 --2- 172.21.15.32:0/662547177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc06eb60 0x7efebc1570d0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7efea40ae040 tx=0x7efea401acb0 comp rx=0 tx=0).stop 2024-04-17T09:59:05.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.450+0000 7efec0e00640 1 -- 172.21.15.32:0/662547177 shutdown_connections 2024-04-17T09:59:05.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.450+0000 7efec0e00640 1 --2- 172.21.15.32:0/662547177 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efe98068380 0x7efe9806a840 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:05.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.450+0000 7efec0e00640 1 --2- 172.21.15.32:0/662547177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efebc1662c0 0x7efebc157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:05.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.450+0000 7efec0e00640 1 --2- 172.21.15.32:0/662547177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc06eb60 0x7efebc1570d0 secure :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7efea40ae040 tx=0x7efea401acb0 comp rx=0 tx=0).stop 2024-04-17T09:59:05.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.450+0000 7efec0e00640 1 --2- 172.21.15.32:0/662547177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efebc06c140 0x7efebc14feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:05.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.450+0000 7efec0e00640 1 -- 172.21.15.32:0/662547177 >> 172.21.15.32:0/662547177 conn(0x7efebc0b2120 msgr2=0x7efebc06ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:05.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.454+0000 7efec0e00640 1 -- 172.21.15.32:0/662547177 shutdown_connections 2024-04-17T09:59:05.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.454+0000 7efec0e00640 1 -- 172.21.15.32:0/662547177 wait complete. 2024-04-17T09:59:05.480 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:59:05.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.622+0000 7f8a79400640 1 Processor -- start 2024-04-17T09:59:05.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.622+0000 7f8a79400640 1 -- start start 2024-04-17T09:59:05.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a79400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a74159010 0x7f8a7415ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:05.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a79400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8a740b4200 0x7f8a740b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:05.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a79400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a7406e390 con 0x7f8a740d05a0 2024-04-17T09:59:05.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a79400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f8a7406e670 con 0x7f8a740b4200 2024-04-17T09:59:05.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a79400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8a7406e950 con 0x7f8a74159010 2024-04-17T09:59:05.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a69e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8a740d05a0 0x7f8a740d0bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38044/0 (socket says 172.21.15.32:38044) 2024-04-17T09:59:05.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a69e00640 1 -- 172.21.15.32:0/1989239852 learned_addr learned my addr 172.21.15.32:0/1989239852 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:05.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a72800640 1 --2- 172.21.15.32:0/1989239852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a74159010 0x7f8a7415ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:05.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a73200640 1 --2- 172.21.15.32:0/1989239852 >> v2:172.21.15.144:3300/0 conn(0x7f8a740b4200 0x7f8a740b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:05.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a71e00640 1 -- 172.21.15.32:0/1989239852 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 345115028 0 0) 0x7f8a7406e390 con 0x7f8a740d05a0 2024-04-17T09:59:05.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a71e00640 1 -- 172.21.15.32:0/1989239852 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a58003530 con 0x7f8a740d05a0 2024-04-17T09:59:05.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a71e00640 1 -- 172.21.15.32:0/1989239852 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1711255282 0 0) 0x7f8a58003530 con 0x7f8a740d05a0 2024-04-17T09:59:05.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a71e00640 1 -- 172.21.15.32:0/1989239852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a74159010 msgr2=0x7f8a7415ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:05.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a71e00640 1 --2- 172.21.15.32:0/1989239852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a74159010 0x7f8a7415ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:05.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a71e00640 1 -- 172.21.15.32:0/1989239852 >> v2:172.21.15.144:3300/0 conn(0x7f8a740b4200 msgr2=0x7f8a740b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:05.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a71e00640 1 --2- 172.21.15.32:0/1989239852 >> v2:172.21.15.144:3300/0 conn(0x7f8a740b4200 0x7f8a740b46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:05.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a71e00640 1 -- 172.21.15.32:0/1989239852 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a7406ec30 con 0x7f8a740d05a0 2024-04-17T09:59:05.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a71e00640 1 -- 172.21.15.32:0/1989239852 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f8a5c001680 con 0x7f8a740d05a0 2024-04-17T09:59:05.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a71e00640 1 -- 172.21.15.32:0/1989239852 >> v1:172.21.15.32:6789/0 conn(0x7f8a740d05a0 legacy=0x7f8a740d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:05.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a71e00640 1 --2- 172.21.15.32:0/1989239852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a580039c0 0x7f8a58003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:05.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a71e00640 1 --2- 172.21.15.32:0/1989239852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a74159010 0x7f8a58004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:05.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a71e00640 1 --2- 172.21.15.32:0/1989239852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a58004d50 0x7f8a580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:05.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a71e00640 1 -- 172.21.15.32:0/1989239852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7406e670 con 0x7f8a58004d50 2024-04-17T09:59:05.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a71e00640 1 -- 172.21.15.32:0/1989239852 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7406e950 con 0x7f8a580039c0 2024-04-17T09:59:05.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a71e00640 1 -- 172.21.15.32:0/1989239852 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8a5800b9d0 con 0x7f8a74159010 2024-04-17T09:59:05.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a73200640 1 --2- 172.21.15.32:0/1989239852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a58004d50 0x7f8a580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:05.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a69e00640 1 --2- 172.21.15.32:0/1989239852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a74159010 0x7f8a58004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:05.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a72800640 1 --2- 172.21.15.32:0/1989239852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a580039c0 0x7f8a58003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:05.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a69e00640 1 -- 172.21.15.32:0/1989239852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a580039c0 msgr2=0x7f8a58003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:05.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a69e00640 1 --2- 172.21.15.32:0/1989239852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a580039c0 0x7f8a58003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:05.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a69e00640 1 -- 172.21.15.32:0/1989239852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a58004d50 msgr2=0x7f8a580051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:05.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a69e00640 1 --2- 172.21.15.32:0/1989239852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a58004d50 0x7f8a580051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:05.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a69e00640 1 -- 172.21.15.32:0/1989239852 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a58003530 con 0x7f8a74159010 2024-04-17T09:59:05.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.626+0000 7f8a73200640 1 --2- 172.21.15.32:0/1989239852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a58004d50 0x7f8a580051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:59:05.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.630+0000 7f8a69e00640 1 --2- 172.21.15.32:0/1989239852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a74159010 0x7f8a58004810 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00c290 tx=0x7f8a5c01f470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a47fa5255e6a619e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:05.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.630+0000 7f8a71e00640 1 -- 172.21.15.32:0/1989239852 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c007d60 con 0x7f8a74159010 2024-04-17T09:59:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.630+0000 7f8a71e00640 1 -- 172.21.15.32:0/1989239852 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a5c005b00 con 0x7f8a74159010 2024-04-17T09:59:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.630+0000 7f8a71e00640 1 -- 172.21.15.32:0/1989239852 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c007070 con 0x7f8a74159010 2024-04-17T09:59:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.630+0000 7f8a79400640 1 -- 172.21.15.32:0/1989239852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a74159010 msgr2=0x7f8a58004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.630+0000 7f8a79400640 1 --2- 172.21.15.32:0/1989239852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a74159010 0x7f8a58004810 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00c290 tx=0x7f8a5c01f470 comp rx=0 tx=0).stop 2024-04-17T09:59:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.630+0000 7f8a73200640 1 -- 172.21.15.32:0/1989239852 reap_dead start 2024-04-17T09:59:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.630+0000 7f8a79400640 1 -- 172.21.15.32:0/1989239852 shutdown_connections 2024-04-17T09:59:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.630+0000 7f8a79400640 1 -- 172.21.15.32:0/1989239852 >> 172.21.15.32:0/1989239852 conn(0x7f8a7406ba30 msgr2=0x7f8a7406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.630+0000 7f8a79400640 1 -- 172.21.15.32:0/1989239852 shutdown_connections 2024-04-17T09:59:05.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.630+0000 7f8a79400640 1 -- 172.21.15.32:0/1989239852 wait complete. 2024-04-17T09:59:05.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a79400640 1 Processor -- start 2024-04-17T09:59:05.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a79400640 1 -- start start 2024-04-17T09:59:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a79400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a74159010 0x7f8a7414cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a79400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a7414d3d0 0x7f8a74151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a79400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a741519c0 0x7f8a74154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a79400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8a741600b0 con 0x7f8a741519c0 2024-04-17T09:59:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a79400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7415ff30 con 0x7f8a7414d3d0 2024-04-17T09:59:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a79400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8a74160230 con 0x7f8a74159010 2024-04-17T09:59:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a73200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a741519c0 0x7f8a74154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a73200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a741519c0 0x7f8a74154470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34568/0 (socket says 172.21.15.32:34568) 2024-04-17T09:59:05.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a73200640 1 -- 172.21.15.32:0/2060683186 learned_addr learned my addr 172.21.15.32:0/2060683186 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:05.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a72800640 1 --2- 172.21.15.32:0/2060683186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a74159010 0x7f8a7414cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:05.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a69e00640 1 --2- 172.21.15.32:0/2060683186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a7414d3d0 0x7f8a74151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:05.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a73200640 1 -- 172.21.15.32:0/2060683186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a74159010 msgr2=0x7f8a7414cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:05.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a73200640 1 --2- 172.21.15.32:0/2060683186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a74159010 0x7f8a7414cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:05.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a73200640 1 -- 172.21.15.32:0/2060683186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a7414d3d0 msgr2=0x7f8a74151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:05.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a73200640 1 --2- 172.21.15.32:0/2060683186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a7414d3d0 0x7f8a74151480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:05.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a73200640 1 -- 172.21.15.32:0/2060683186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a741f78e0 con 0x7f8a741519c0 2024-04-17T09:59:05.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a73200640 1 --2- 172.21.15.32:0/2060683186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a741519c0 0x7f8a74154470 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f8a7406c5e0 tx=0x7f8a6c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:05.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a6b400640 1 -- 172.21.15.32:0/2060683186 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c009c10 con 0x7f8a741519c0 2024-04-17T09:59:05.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a6b400640 1 -- 172.21.15.32:0/2060683186 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a6c00fdc0 con 0x7f8a741519c0 2024-04-17T09:59:05.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a6b400640 1 -- 172.21.15.32:0/2060683186 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c003bc0 con 0x7f8a741519c0 2024-04-17T09:59:05.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a79400640 1 -- 172.21.15.32:0/2060683186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a741f7bc0 con 0x7f8a741519c0 2024-04-17T09:59:05.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a79400640 1 -- 172.21.15.32:0/2060683186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a741f8180 con 0x7f8a741519c0 2024-04-17T09:59:05.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.634+0000 7f8a79400640 1 -- 172.21.15.32:0/2060683186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a74068a90 con 0x7f8a741519c0 2024-04-17T09:59:05.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.638+0000 7f8a6b400640 1 -- 172.21.15.32:0/2060683186 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8a6c01f020 con 0x7f8a741519c0 2024-04-17T09:59:05.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.638+0000 7f8a6b400640 1 --2- 172.21.15.32:0/2060683186 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8a500683b0 0x7f8a5006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:05.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.638+0000 7f8a6b400640 1 -- 172.21.15.32:0/2060683186 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f8a6c086c60 con 0x7f8a741519c0 2024-04-17T09:59:05.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.638+0000 7f8a72800640 1 --2- 172.21.15.32:0/2060683186 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8a500683b0 0x7f8a5006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:05.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.638+0000 7f8a72800640 1 --2- 172.21.15.32:0/2060683186 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8a500683b0 0x7f8a5006a870 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8a64009a80 tx=0x7f8a64009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:05.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.642+0000 7f8a6b400640 1 -- 172.21.15.32:0/2060683186 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f8a6c010040 con 0x7f8a741519c0 2024-04-17T09:59:05.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.806+0000 7f8a79400640 1 -- 172.21.15.32:0/2060683186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a7415f3a0 con 0x7f8a741519c0 2024-04-17T09:59:05.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.806+0000 7f8a6b400640 1 -- 172.21.15.32:0/2060683186 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+16685 (secure 0 0 0) 0x7f8a6c057180 con 0x7f8a741519c0 2024-04-17T09:59:05.805 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:59:05.806 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":81,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:59:04.834143+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:59:03.223426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-17T09:59:03.780120+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:59:05.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.810+0000 7f8a79400640 1 -- 172.21.15.32:0/2060683186 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8a500683b0 msgr2=0x7f8a5006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:05.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.810+0000 7f8a79400640 1 --2- 172.21.15.32:0/2060683186 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8a500683b0 0x7f8a5006a870 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8a64009a80 tx=0x7f8a64009190 comp rx=0 tx=0).stop 2024-04-17T09:59:05.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.810+0000 7f8a79400640 1 -- 172.21.15.32:0/2060683186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a741519c0 msgr2=0x7f8a74154470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:05.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.810+0000 7f8a79400640 1 --2- 172.21.15.32:0/2060683186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a741519c0 0x7f8a74154470 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f8a7406c5e0 tx=0x7f8a6c00f580 comp rx=0 tx=0).stop 2024-04-17T09:59:05.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.810+0000 7f8a79400640 1 -- 172.21.15.32:0/2060683186 shutdown_connections 2024-04-17T09:59:05.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.810+0000 7f8a79400640 1 --2- 172.21.15.32:0/2060683186 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8a500683b0 0x7f8a5006a870 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:05.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.810+0000 7f8a79400640 1 --2- 172.21.15.32:0/2060683186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a741519c0 0x7f8a74154470 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:05.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.810+0000 7f8a79400640 1 --2- 172.21.15.32:0/2060683186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a7414d3d0 0x7f8a74151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:05.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.810+0000 7f8a79400640 1 --2- 172.21.15.32:0/2060683186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a74159010 0x7f8a7414cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:05.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.810+0000 7f8a79400640 1 -- 172.21.15.32:0/2060683186 >> 172.21.15.32:0/2060683186 conn(0x7f8a7406ba30 msgr2=0x7f8a7406f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:05.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.810+0000 7f8a79400640 1 -- 172.21.15.32:0/2060683186 shutdown_connections 2024-04-17T09:59:05.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.810+0000 7f8a79400640 1 -- 172.21.15.32:0/2060683186 wait complete. 2024-04-17T09:59:05.827 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:59:05.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.970+0000 7f95caa00640 1 Processor -- start 2024-04-17T09:59:05.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.970+0000 7f95caa00640 1 -- start start 2024-04-17T09:59:05.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.970+0000 7f95caa00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f95c4144e90 0x7f95c40b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:05.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.970+0000 7f95caa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c40b4de0 0x7f95c415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:05.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.970+0000 7f95caa00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95c4072bb0 con 0x7f95c415bfe0 2024-04-17T09:59:05.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.970+0000 7f95caa00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f95c415c430 con 0x7f95c4144e90 2024-04-17T09:59:05.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.970+0000 7f95caa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f95c4145270 con 0x7f95c40b4de0 2024-04-17T09:59:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.970+0000 7f95c3e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f95c4144e90 0x7f95c40b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.970+0000 7f95bb400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c40b4de0 0x7f95c415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.970+0000 7f95c3e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f95c4144e90 0x7f95c40b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:42614/0 (socket says 172.21.15.32:42614) 2024-04-17T09:59:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.970+0000 7f95bb400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c40b4de0 0x7f95c415baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:41872/0 (socket says 172.21.15.32:41872) 2024-04-17T09:59:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.970+0000 7f95c3e00640 1 -- 172.21.15.32:0/174677099 learned_addr learned my addr 172.21.15.32:0/174677099 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.970+0000 7f95c8a00640 1 --1- 172.21.15.32:0/174677099 >> v1:172.21.15.144:6789/0 conn(0x7f95c415bfe0 0x7f95c406ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:35170/0 (socket says 172.21.15.32:35170) 2024-04-17T09:59:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.974+0000 7f95c3e00640 1 -- 172.21.15.32:0/174677099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c40b4de0 msgr2=0x7f95c415baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:05.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.974+0000 7f95c3e00640 1 --2- 172.21.15.32:0/174677099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c40b4de0 0x7f95c415baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:05.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.974+0000 7f95c3e00640 1 -- 172.21.15.32:0/174677099 >> v1:172.21.15.144:6789/0 conn(0x7f95c415bfe0 legacy=0x7f95c406ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:05.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.974+0000 7f95c3e00640 1 -- 172.21.15.32:0/174677099 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c40b3dc0 con 0x7f95c4144e90 2024-04-17T09:59:05.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.974+0000 7f95c3e00640 1 --2- 172.21.15.32:0/174677099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95c4144e90 0x7f95c40b48a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f95b400bb40 tx=0x7f95b401ebe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b326066d3e4aa91b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:05.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.974+0000 7f95c3400640 1 -- 172.21.15.32:0/174677099 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b40b4070 con 0x7f95c4144e90 2024-04-17T09:59:05.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.974+0000 7f95c3400640 1 -- 172.21.15.32:0/174677099 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95b40a7cd0 con 0x7f95c4144e90 2024-04-17T09:59:05.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.974+0000 7f95c3400640 1 -- 172.21.15.32:0/174677099 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b40af660 con 0x7f95c4144e90 2024-04-17T09:59:05.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95caa00640 1 -- 172.21.15.32:0/174677099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95c4144e90 msgr2=0x7f95c40b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:05.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95caa00640 1 --2- 172.21.15.32:0/174677099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95c4144e90 0x7f95c40b48a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f95b400bb40 tx=0x7f95b401ebe0 comp rx=0 tx=0).stop 2024-04-17T09:59:05.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95caa00640 1 -- 172.21.15.32:0/174677099 shutdown_connections 2024-04-17T09:59:05.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95caa00640 1 --2- 172.21.15.32:0/174677099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c40b4de0 0x7f95c415baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:05.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95caa00640 1 --2- 172.21.15.32:0/174677099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95c4144e90 0x7f95c40b48a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:05.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95caa00640 1 -- 172.21.15.32:0/174677099 >> 172.21.15.32:0/174677099 conn(0x7f95c406ba30 msgr2=0x7f95c406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:05.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95caa00640 1 -- 172.21.15.32:0/174677099 shutdown_connections 2024-04-17T09:59:05.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95caa00640 1 -- 172.21.15.32:0/174677099 wait complete. 2024-04-17T09:59:05.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95caa00640 1 Processor -- start 2024-04-17T09:59:05.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95caa00640 1 -- start start 2024-04-17T09:59:05.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95caa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c40b4de0 0x7f95c41f74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:05.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95caa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95c4144e90 0x7f95c413b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:05.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95caa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95c415bfe0 0x7f95c4139870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:05.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95caa00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f95c4072670 con 0x7f95c415bfe0 2024-04-17T09:59:05.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95caa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95c406f8c0 con 0x7f95c4144e90 2024-04-17T09:59:05.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95caa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f95c40727f0 con 0x7f95c40b4de0 2024-04-17T09:59:05.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95c8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95c415bfe0 0x7f95c4139870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:05.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95c3e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c40b4de0 0x7f95c41f74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:05.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95bb400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95c4144e90 0x7f95c413b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:05.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95c8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95c415bfe0 0x7f95c4139870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34582/0 (socket says 172.21.15.32:34582) 2024-04-17T09:59:05.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95c8a00640 1 -- 172.21.15.32:0/3886012974 learned_addr learned my addr 172.21.15.32:0/3886012974 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:05.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95c8a00640 1 -- 172.21.15.32:0/3886012974 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c40b4de0 msgr2=0x7f95c41f74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:05.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95c8a00640 1 --2- 172.21.15.32:0/3886012974 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c40b4de0 0x7f95c41f74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:05.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95c8a00640 1 -- 172.21.15.32:0/3886012974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95c4144e90 msgr2=0x7f95c413b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:05.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95c8a00640 1 --2- 172.21.15.32:0/3886012974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95c4144e90 0x7f95c413b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:05.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.978+0000 7f95c8a00640 1 -- 172.21.15.32:0/3886012974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c413b6d0 con 0x7f95c415bfe0 2024-04-17T09:59:05.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.982+0000 7f95c8a00640 1 --2- 172.21.15.32:0/3886012974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95c415bfe0 0x7f95c4139870 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00c700 tx=0x7f95bc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:05.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.982+0000 7f95c0c00640 1 -- 172.21.15.32:0/3886012974 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc017070 con 0x7f95c415bfe0 2024-04-17T09:59:05.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.982+0000 7f95c0c00640 1 -- 172.21.15.32:0/3886012974 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95bc00ce10 con 0x7f95c415bfe0 2024-04-17T09:59:05.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.982+0000 7f95caa00640 1 -- 172.21.15.32:0/3886012974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c41f8460 con 0x7f95c415bfe0 2024-04-17T09:59:05.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.982+0000 7f95c0c00640 1 -- 172.21.15.32:0/3886012974 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc00a840 con 0x7f95c415bfe0 2024-04-17T09:59:05.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.982+0000 7f95caa00640 1 -- 172.21.15.32:0/3886012974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c41f7ea0 con 0x7f95c415bfe0 2024-04-17T09:59:05.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.982+0000 7f95caa00640 1 -- 172.21.15.32:0/3886012974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95c4068a90 con 0x7f95c415bfe0 2024-04-17T09:59:05.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.982+0000 7f95c0c00640 1 -- 172.21.15.32:0/3886012974 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f95bc01a070 con 0x7f95c415bfe0 2024-04-17T09:59:05.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.982+0000 7f95c0c00640 1 --2- 172.21.15.32:0/3886012974 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f95a0068380 0x7f95a006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:05.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.982+0000 7f95c0c00640 1 -- 172.21.15.32:0/3886012974 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f95bc085e50 con 0x7f95c415bfe0 2024-04-17T09:59:05.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.982+0000 7f95c3e00640 1 --2- 172.21.15.32:0/3886012974 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f95a0068380 0x7f95a006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:05.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.986+0000 7f95c3e00640 1 --2- 172.21.15.32:0/3886012974 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f95a0068380 0x7f95a006a840 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f95b40b1000 tx=0x7f95b4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:05.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:05.986+0000 7f95c0c00640 1 -- 172.21.15.32:0/3886012974 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f95bc055db0 con 0x7f95c415bfe0 2024-04-17T09:59:06.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.150+0000 7f95caa00640 1 -- 172.21.15.32:0/3886012974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95c40afb50 con 0x7f95c415bfe0 2024-04-17T09:59:06.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.154+0000 7f95c0c00640 1 -- 172.21.15.32:0/3886012974 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+16685 (secure 0 0 0) 0x7f95bc05a0b0 con 0x7f95c415bfe0 2024-04-17T09:59:06.151 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:59:06.151 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":81,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:59:04.834143+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:59:03.223426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-17T09:59:03.780120+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:59:06.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.154+0000 7f95caa00640 1 -- 172.21.15.32:0/3886012974 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f95a0068380 msgr2=0x7f95a006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:06.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.154+0000 7f95caa00640 1 --2- 172.21.15.32:0/3886012974 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f95a0068380 0x7f95a006a840 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f95b40b1000 tx=0x7f95b4008040 comp rx=0 tx=0).stop 2024-04-17T09:59:06.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.154+0000 7f95caa00640 1 -- 172.21.15.32:0/3886012974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95c415bfe0 msgr2=0x7f95c4139870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:06.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.154+0000 7f95caa00640 1 --2- 172.21.15.32:0/3886012974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95c415bfe0 0x7f95c4139870 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00c700 tx=0x7f95bc00cbc0 comp rx=0 tx=0).stop 2024-04-17T09:59:06.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.154+0000 7f95caa00640 1 -- 172.21.15.32:0/3886012974 shutdown_connections 2024-04-17T09:59:06.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.154+0000 7f95caa00640 1 --2- 172.21.15.32:0/3886012974 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f95a0068380 0x7f95a006a840 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:06.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.154+0000 7f95caa00640 1 --2- 172.21.15.32:0/3886012974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95c415bfe0 0x7f95c4139870 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:06.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.154+0000 7f95caa00640 1 --2- 172.21.15.32:0/3886012974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95c4144e90 0x7f95c413b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:06.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.154+0000 7f95caa00640 1 --2- 172.21.15.32:0/3886012974 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c40b4de0 0x7f95c41f74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:06.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.154+0000 7f95caa00640 1 -- 172.21.15.32:0/3886012974 >> 172.21.15.32:0/3886012974 conn(0x7f95c406ba30 msgr2=0x7f95c40cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:06.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.154+0000 7f95caa00640 1 -- 172.21.15.32:0/3886012974 shutdown_connections 2024-04-17T09:59:06.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.154+0000 7f95caa00640 1 -- 172.21.15.32:0/3886012974 wait complete. 2024-04-17T09:59:06.172 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T09:59:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.318+0000 7f7b73000640 1 Processor -- start 2024-04-17T09:59:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.318+0000 7f7b73000640 1 -- start start 2024-04-17T09:59:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.318+0000 7f7b73000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7b6c15c4c0 0x7f7b6c0adb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.318+0000 7f7b73000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b6c15f9c0 0x7f7b6c06ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.318+0000 7f7b73000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7b6c06bf70 0x7f7b6c071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:06.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.318+0000 7f7b73000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7b6c06c3a0 con 0x7f7b6c15c4c0 2024-04-17T09:59:06.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.318+0000 7f7b73000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f7b6c1629a0 con 0x7f7b6c06bf70 2024-04-17T09:59:06.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.318+0000 7f7b73000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7b6c162c80 con 0x7f7b6c15f9c0 2024-04-17T09:59:06.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.322+0000 7f7b6be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7b6c15c4c0 0x7f7b6c0adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:06.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.322+0000 7f7b6be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7b6c15c4c0 0x7f7b6c0adb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34584/0 (socket says 172.21.15.32:34584) 2024-04-17T09:59:06.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.322+0000 7f7b6be00640 1 -- 172.21.15.32:0/2265713413 learned_addr learned my addr 172.21.15.32:0/2265713413 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:06.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.322+0000 7f7b71000640 1 --2- 172.21.15.32:0/2265713413 >> v2:172.21.15.144:3300/0 conn(0x7f7b6c06bf70 0x7f7b6c071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:06.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.322+0000 7f7b63400640 1 --2- 172.21.15.32:0/2265713413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b6c15f9c0 0x7f7b6c06ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:06.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.322+0000 7f7b6be00640 1 -- 172.21.15.32:0/2265713413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b6c15f9c0 msgr2=0x7f7b6c06ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:06.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.322+0000 7f7b6be00640 1 --2- 172.21.15.32:0/2265713413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b6c15f9c0 0x7f7b6c06ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:06.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.322+0000 7f7b6be00640 1 -- 172.21.15.32:0/2265713413 >> v2:172.21.15.144:3300/0 conn(0x7f7b6c06bf70 msgr2=0x7f7b6c071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:06.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.322+0000 7f7b6be00640 1 --2- 172.21.15.32:0/2265713413 >> v2:172.21.15.144:3300/0 conn(0x7f7b6c06bf70 0x7f7b6c071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:06.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.322+0000 7f7b6be00640 1 -- 172.21.15.32:0/2265713413 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c162f60 con 0x7f7b6c15c4c0 2024-04-17T09:59:06.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.322+0000 7f7b6be00640 1 --2- 172.21.15.32:0/2265713413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6c15c4c0 0x7f7b6c0adb40 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0cc700 tx=0x7f7b5c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31245b08cf18a14f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.322+0000 7f7b6b400640 1 -- 172.21.15.32:0/2265713413 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c0b0070 con 0x7f7b6c15c4c0 2024-04-17T09:59:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.322+0000 7f7b6b400640 1 -- 172.21.15.32:0/2265713413 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b5c0ab600 con 0x7f7b6c15c4c0 2024-04-17T09:59:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.322+0000 7f7b6b400640 1 -- 172.21.15.32:0/2265713413 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c0ab9d0 con 0x7f7b6c15c4c0 2024-04-17T09:59:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b73000640 1 -- 172.21.15.32:0/2265713413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6c15c4c0 msgr2=0x7f7b6c0adb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b73000640 1 --2- 172.21.15.32:0/2265713413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6c15c4c0 0x7f7b6c0adb40 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0cc700 tx=0x7f7b5c01c260 comp rx=0 tx=0).stop 2024-04-17T09:59:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b73000640 1 -- 172.21.15.32:0/2265713413 shutdown_connections 2024-04-17T09:59:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b73000640 1 --2- 172.21.15.32:0/2265713413 >> v2:172.21.15.144:3300/0 conn(0x7f7b6c06bf70 0x7f7b6c071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b73000640 1 --2- 172.21.15.32:0/2265713413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b6c15f9c0 0x7f7b6c06ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b73000640 1 --2- 172.21.15.32:0/2265713413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6c15c4c0 0x7f7b6c0adb40 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b73000640 1 -- 172.21.15.32:0/2265713413 >> 172.21.15.32:0/2265713413 conn(0x7f7b6c0b2120 msgr2=0x7f7b6c0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b73000640 1 -- 172.21.15.32:0/2265713413 shutdown_connections 2024-04-17T09:59:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b73000640 1 -- 172.21.15.32:0/2265713413 wait complete. 2024-04-17T09:59:06.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b73000640 1 Processor -- start 2024-04-17T09:59:06.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b73000640 1 -- start start 2024-04-17T09:59:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b73000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6c06bf70 0x7f7b6c14f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b73000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b6c15c4c0 0x7f7b6c0ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b73000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b6c15f9c0 0x7f7b6c0aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b73000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c162530 con 0x7f7b6c06bf70 2024-04-17T09:59:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b73000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c0af670 con 0x7f7b6c15f9c0 2024-04-17T09:59:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b73000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7b6c1626b0 con 0x7f7b6c15c4c0 2024-04-17T09:59:06.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b6be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6c06bf70 0x7f7b6c14f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:06.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b6be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6c06bf70 0x7f7b6c14f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34594/0 (socket says 172.21.15.32:34594) 2024-04-17T09:59:06.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b6be00640 1 -- 172.21.15.32:0/2356634711 learned_addr learned my addr 172.21.15.32:0/2356634711 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:06.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b63400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b6c15c4c0 0x7f7b6c0ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:06.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b71000640 1 --2- 172.21.15.32:0/2356634711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b6c15f9c0 0x7f7b6c0aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b6be00640 1 -- 172.21.15.32:0/2356634711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b6c15c4c0 msgr2=0x7f7b6c0ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b6be00640 1 --2- 172.21.15.32:0/2356634711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b6c15c4c0 0x7f7b6c0ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b6be00640 1 -- 172.21.15.32:0/2356634711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b6c15f9c0 msgr2=0x7f7b6c0aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b6be00640 1 --2- 172.21.15.32:0/2356634711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b6c15f9c0 0x7f7b6c0aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.326+0000 7f7b6be00640 1 -- 172.21.15.32:0/2356634711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c151c50 con 0x7f7b6c06bf70 2024-04-17T09:59:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.330+0000 7f7b6be00640 1 --2- 172.21.15.32:0/2356634711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6c06bf70 0x7f7b6c14f940 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c000c10 tx=0x7f7b5c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.330+0000 7f7b68c00640 1 -- 172.21.15.32:0/2356634711 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c0b0040 con 0x7f7b6c06bf70 2024-04-17T09:59:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.330+0000 7f7b68c00640 1 -- 172.21.15.32:0/2356634711 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b5c0b9070 con 0x7f7b6c06bf70 2024-04-17T09:59:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.330+0000 7f7b73000640 1 -- 172.21.15.32:0/2356634711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6c151f30 con 0x7f7b6c06bf70 2024-04-17T09:59:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.330+0000 7f7b68c00640 1 -- 172.21.15.32:0/2356634711 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c0b4630 con 0x7f7b6c06bf70 2024-04-17T09:59:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.330+0000 7f7b73000640 1 -- 172.21.15.32:0/2356634711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b6c1524f0 con 0x7f7b6c06bf70 2024-04-17T09:59:06.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.330+0000 7f7b73000640 1 -- 172.21.15.32:0/2356634711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b6c068a90 con 0x7f7b6c06bf70 2024-04-17T09:59:06.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.330+0000 7f7b68c00640 1 -- 172.21.15.32:0/2356634711 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7b5c0b47d0 con 0x7f7b6c06bf70 2024-04-17T09:59:06.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.330+0000 7f7b68c00640 1 --2- 172.21.15.32:0/2356634711 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7b48068380 0x7f7b4806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:06.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.330+0000 7f7b68c00640 1 -- 172.21.15.32:0/2356634711 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f7b5c120580 con 0x7f7b6c06bf70 2024-04-17T09:59:06.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.334+0000 7f7b63400640 1 --2- 172.21.15.32:0/2356634711 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7b48068380 0x7f7b4806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:06.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.334+0000 7f7b63400640 1 --2- 172.21.15.32:0/2356634711 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7b48068380 0x7f7b4806a840 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c1fac00 tx=0x7f7b54009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:06.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.334+0000 7f7b68c00640 1 -- 172.21.15.32:0/2356634711 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f7b5c016110 con 0x7f7b6c06bf70 2024-04-17T09:59:06.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:06.542+0000 7f7b73000640 1 -- 172.21.15.32:0/2356634711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7b6c168be0 con 0x7f7b6c06bf70 2024-04-17T09:59:07.408 INFO:tasks.ceph.mds.c.smithi032.stderr:2024-04-17T09:59:07.410+0000 7fc47c000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T09:59:07.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.450+0000 7f7b68c00640 1 -- 172.21.15.32:0/2356634711 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v58) ==== 118+0+0 (secure 0 0 0) 0x7f7b5c0f0450 con 0x7f7b6c06bf70 2024-04-17T09:59:07.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.454+0000 7f7b73000640 1 -- 172.21.15.32:0/2356634711 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7b48068380 msgr2=0x7f7b4806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:07.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.454+0000 7f7b73000640 1 --2- 172.21.15.32:0/2356634711 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7b48068380 0x7f7b4806a840 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c1fac00 tx=0x7f7b54009190 comp rx=0 tx=0).stop 2024-04-17T09:59:07.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.454+0000 7f7b73000640 1 -- 172.21.15.32:0/2356634711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6c06bf70 msgr2=0x7f7b6c14f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:07.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.454+0000 7f7b73000640 1 --2- 172.21.15.32:0/2356634711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6c06bf70 0x7f7b6c14f940 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c000c10 tx=0x7f7b5c019000 comp rx=0 tx=0).stop 2024-04-17T09:59:07.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.454+0000 7f7b73000640 1 -- 172.21.15.32:0/2356634711 shutdown_connections 2024-04-17T09:59:07.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.454+0000 7f7b73000640 1 --2- 172.21.15.32:0/2356634711 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7b48068380 0x7f7b4806a840 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:07.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.454+0000 7f7b73000640 1 --2- 172.21.15.32:0/2356634711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b6c15c4c0 0x7f7b6c0ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:07.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.454+0000 7f7b73000640 1 --2- 172.21.15.32:0/2356634711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b6c15f9c0 0x7f7b6c0aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:07.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.454+0000 7f7b73000640 1 --2- 172.21.15.32:0/2356634711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b6c06bf70 0x7f7b6c14f940 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:07.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.454+0000 7f7b73000640 1 -- 172.21.15.32:0/2356634711 >> 172.21.15.32:0/2356634711 conn(0x7f7b6c0b2120 msgr2=0x7f7b6c06e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:07.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.454+0000 7f7b73000640 1 -- 172.21.15.32:0/2356634711 shutdown_connections 2024-04-17T09:59:07.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.454+0000 7f7b73000640 1 -- 172.21.15.32:0/2356634711 wait complete. 2024-04-17T09:59:07.478 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-17T09:59:07.478 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-17T09:59:07.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.630+0000 7f81d1000640 1 Processor -- start 2024-04-17T09:59:07.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.630+0000 7f81d1000640 1 -- start start 2024-04-17T09:59:07.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.630+0000 7f81d1000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f81cc15b7d0 0x7f81cc06e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:07.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.630+0000 7f81d1000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81cc06c140 0x7f81cc0ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:07.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.630+0000 7f81d1000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81cc0cc940 con 0x7f81cc06f7c0 2024-04-17T09:59:07.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.630+0000 7f81d1000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f81cc0ccc20 con 0x7f81cc06c140 2024-04-17T09:59:07.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.630+0000 7f81d1000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f81cc06e0b0 con 0x7f81cc15b7d0 2024-04-17T09:59:07.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.634+0000 7f81ca800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81cc06c140 0x7f81cc0ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:07.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.634+0000 7f81ca800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81cc06c140 0x7f81cc0ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34610/0 (socket says 172.21.15.32:34610) 2024-04-17T09:59:07.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.634+0000 7f81ca800640 1 -- 172.21.15.32:0/1263996634 learned_addr learned my addr 172.21.15.32:0/1263996634 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:07.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.634+0000 7f81c9e00640 1 --2- 172.21.15.32:0/1263996634 >> v2:172.21.15.144:3300/0 conn(0x7f81cc15b7d0 0x7f81cc06e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:07.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.634+0000 7f81ca800640 1 -- 172.21.15.32:0/1263996634 >> v2:172.21.15.144:3300/0 conn(0x7f81cc15b7d0 msgr2=0x7f81cc06e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:07.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.634+0000 7f81ca800640 1 --2- 172.21.15.32:0/1263996634 >> v2:172.21.15.144:3300/0 conn(0x7f81cc15b7d0 0x7f81cc06e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:07.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.634+0000 7f81ca800640 1 -- 172.21.15.32:0/1263996634 >> v1:172.21.15.144:6789/0 conn(0x7f81cc06f7c0 legacy=0x7f81cc06ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:07.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.634+0000 7f81ca800640 1 -- 172.21.15.32:0/1263996634 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81cc06e390 con 0x7f81cc06c140 2024-04-17T09:59:07.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.634+0000 7f81ca800640 1 --2- 172.21.15.32:0/1263996634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc06c140 0x7f81cc0ce290 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0d1530 tx=0x7f81c401eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe66ec06348b6478 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:07.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.634+0000 7f81c9400640 1 -- 172.21.15.32:0/1263996634 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c400b2d0 con 0x7f81cc06c140 2024-04-17T09:59:07.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.634+0000 7f81c9400640 1 -- 172.21.15.32:0/1263996634 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c400b470 con 0x7f81cc06c140 2024-04-17T09:59:07.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.634+0000 7f81c9400640 1 -- 172.21.15.32:0/1263996634 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c40a7c70 con 0x7f81cc06c140 2024-04-17T09:59:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81d1000640 1 -- 172.21.15.32:0/1263996634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc06c140 msgr2=0x7f81cc0ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81d1000640 1 --2- 172.21.15.32:0/1263996634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc06c140 0x7f81cc0ce290 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0d1530 tx=0x7f81c401eb60 comp rx=0 tx=0).stop 2024-04-17T09:59:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81d1000640 1 -- 172.21.15.32:0/1263996634 shutdown_connections 2024-04-17T09:59:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81d1000640 1 --2- 172.21.15.32:0/1263996634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc06c140 0x7f81cc0ce290 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81d1000640 1 --2- 172.21.15.32:0/1263996634 >> v2:172.21.15.144:3300/0 conn(0x7f81cc15b7d0 0x7f81cc06e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81d1000640 1 -- 172.21.15.32:0/1263996634 >> 172.21.15.32:0/1263996634 conn(0x7f81cc0b2120 msgr2=0x7f81cc0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81d1000640 1 -- 172.21.15.32:0/1263996634 shutdown_connections 2024-04-17T09:59:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81d1000640 1 -- 172.21.15.32:0/1263996634 wait complete. 2024-04-17T09:59:07.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81d1000640 1 Processor -- start 2024-04-17T09:59:07.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81d1000640 1 -- start start 2024-04-17T09:59:07.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81d1000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81cc06c140 0x7f81cc149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:07.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81d1000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81cc06f7c0 0x7f81cc1585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:07.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81d1000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc15b7d0 0x7f81cc15da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:07.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81d1000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc0721f0 con 0x7f81cc15b7d0 2024-04-17T09:59:07.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81d1000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc072070 con 0x7f81cc06f7c0 2024-04-17T09:59:07.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81d1000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f81cc072370 con 0x7f81cc06c140 2024-04-17T09:59:07.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81ca800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc15b7d0 0x7f81cc15da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:07.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81ca800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc15b7d0 0x7f81cc15da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34622/0 (socket says 172.21.15.32:34622) 2024-04-17T09:59:07.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81ca800640 1 -- 172.21.15.32:0/3334886754 learned_addr learned my addr 172.21.15.32:0/3334886754 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:07.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81c3e00640 1 --2- 172.21.15.32:0/3334886754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81cc06f7c0 0x7f81cc1585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:07.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81c9e00640 1 --2- 172.21.15.32:0/3334886754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81cc06c140 0x7f81cc149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:07.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81ca800640 1 -- 172.21.15.32:0/3334886754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81cc06c140 msgr2=0x7f81cc149a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:07.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81ca800640 1 --2- 172.21.15.32:0/3334886754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81cc06c140 0x7f81cc149a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:07.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81ca800640 1 -- 172.21.15.32:0/3334886754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81cc06f7c0 msgr2=0x7f81cc1585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:07.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81ca800640 1 --2- 172.21.15.32:0/3334886754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81cc06f7c0 0x7f81cc1585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:07.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81ca800640 1 -- 172.21.15.32:0/3334886754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81cc1f7b80 con 0x7f81cc15b7d0 2024-04-17T09:59:07.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.638+0000 7f81c9e00640 1 --2- 172.21.15.32:0/3334886754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81cc06c140 0x7f81cc149a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:07.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.642+0000 7f81ca800640 1 --2- 172.21.15.32:0/3334886754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc15b7d0 0x7f81cc15da70 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f81c401b770 tx=0x7f81c401b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:07.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.642+0000 7f81c2000640 1 -- 172.21.15.32:0/3334886754 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c4009800 con 0x7f81cc15b7d0 2024-04-17T09:59:07.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.642+0000 7f81c2000640 1 -- 172.21.15.32:0/3334886754 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c40be070 con 0x7f81cc15b7d0 2024-04-17T09:59:07.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.642+0000 7f81c2000640 1 -- 172.21.15.32:0/3334886754 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c4009c00 con 0x7f81cc15b7d0 2024-04-17T09:59:07.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.642+0000 7f81d1000640 1 -- 172.21.15.32:0/3334886754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81cc15e530 con 0x7f81cc15b7d0 2024-04-17T09:59:07.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.642+0000 7f81d1000640 1 -- 172.21.15.32:0/3334886754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81cc157db0 con 0x7f81cc15b7d0 2024-04-17T09:59:07.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.642+0000 7f81d1000640 1 -- 172.21.15.32:0/3334886754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81cc068a90 con 0x7f81cc15b7d0 2024-04-17T09:59:07.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.642+0000 7f81c2000640 1 -- 172.21.15.32:0/3334886754 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f81c40b6070 con 0x7f81cc15b7d0 2024-04-17T09:59:07.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.642+0000 7f81c2000640 1 --2- 172.21.15.32:0/3334886754 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f81a8068380 0x7f81a806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:07.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.642+0000 7f81c2000640 1 -- 172.21.15.32:0/3334886754 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f81c4121850 con 0x7f81cc15b7d0 2024-04-17T09:59:07.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.642+0000 7f81c9e00640 1 --2- 172.21.15.32:0/3334886754 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f81a8068380 0x7f81a806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:07.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.646+0000 7f81c9e00640 1 --2- 172.21.15.32:0/3334886754 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f81a8068380 0x7f81a806a840 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f81bc009b10 tx=0x7f81bc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:07.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.646+0000 7f81c2000640 1 -- 172.21.15.32:0/3334886754 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f81c40187e0 con 0x7f81cc15b7d0 2024-04-17T09:59:07.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:07.846+0000 7f81d1000640 1 -- 172.21.15.32:0/3334886754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f81cc0af980 con 0x7f81cc15b7d0 2024-04-17T09:59:08.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.454+0000 7f81c2000640 1 -- 172.21.15.32:0/3334886754 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v59) ==== 122+0+0 (secure 0 0 0) 0x7f81c40f1f50 con 0x7f81cc15b7d0 2024-04-17T09:59:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.454+0000 7f81d1000640 1 -- 172.21.15.32:0/3334886754 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f81a8068380 msgr2=0x7f81a806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.454+0000 7f81d1000640 1 --2- 172.21.15.32:0/3334886754 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f81a8068380 0x7f81a806a840 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f81bc009b10 tx=0x7f81bc008040 comp rx=0 tx=0).stop 2024-04-17T09:59:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.454+0000 7f81d1000640 1 -- 172.21.15.32:0/3334886754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc15b7d0 msgr2=0x7f81cc15da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.454+0000 7f81d1000640 1 --2- 172.21.15.32:0/3334886754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc15b7d0 0x7f81cc15da70 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f81c401b770 tx=0x7f81c401b900 comp rx=0 tx=0).stop 2024-04-17T09:59:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.454+0000 7f81d1000640 1 -- 172.21.15.32:0/3334886754 shutdown_connections 2024-04-17T09:59:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.454+0000 7f81d1000640 1 --2- 172.21.15.32:0/3334886754 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f81a8068380 0x7f81a806a840 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.454+0000 7f81d1000640 1 --2- 172.21.15.32:0/3334886754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc15b7d0 0x7f81cc15da70 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.454+0000 7f81d1000640 1 --2- 172.21.15.32:0/3334886754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81cc06f7c0 0x7f81cc1585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.454+0000 7f81d1000640 1 --2- 172.21.15.32:0/3334886754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81cc06c140 0x7f81cc149a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.454+0000 7f81d1000640 1 -- 172.21.15.32:0/3334886754 >> 172.21.15.32:0/3334886754 conn(0x7f81cc0b2120 msgr2=0x7f81cc0b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.454+0000 7f81d1000640 1 -- 172.21.15.32:0/3334886754 shutdown_connections 2024-04-17T09:59:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.454+0000 7f81d1000640 1 -- 172.21.15.32:0/3334886754 wait complete. 2024-04-17T09:59:08.476 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-17T09:59:08.476 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:59:08.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.622+0000 7ff907400640 1 Processor -- start 2024-04-17T09:59:08.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.622+0000 7ff907400640 1 -- start start 2024-04-17T09:59:08.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.626+0000 7ff907400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff900144e90 0x7ff9000b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:08.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.626+0000 7ff907400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff90015c1b0 0x7ff90006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:08.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.626+0000 7ff907400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff900072bb0 con 0x7ff9000b4de0 2024-04-17T09:59:08.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.626+0000 7ff907400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff900145270 con 0x7ff900144e90 2024-04-17T09:59:08.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.626+0000 7ff907400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff9000b3dc0 con 0x7ff90015c1b0 2024-04-17T09:59:08.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.626+0000 7ff8f7e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff9000b4de0 0x7ff90015baa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38060/0 (socket says 172.21.15.32:38060) 2024-04-17T09:59:08.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.626+0000 7ff8f7e00640 1 -- 172.21.15.32:0/3793828548 learned_addr learned my addr 172.21.15.32:0/3793828548 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:08.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.626+0000 7ff905400640 1 --2- 172.21.15.32:0/3793828548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff90015c1b0 0x7ff90006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:08.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.626+0000 7ff904a00640 1 --2- 172.21.15.32:0/3793828548 >> v2:172.21.15.144:3300/0 conn(0x7ff900144e90 0x7ff9000b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:08.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.626+0000 7ff8ffe00640 1 -- 172.21.15.32:0/3793828548 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1394413803 0 0) 0x7ff900072bb0 con 0x7ff9000b4de0 2024-04-17T09:59:08.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.626+0000 7ff8ffe00640 1 -- 172.21.15.32:0/3793828548 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8e4005740 con 0x7ff9000b4de0 2024-04-17T09:59:08.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.626+0000 7ff904a00640 1 -- 172.21.15.32:0/3793828548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff90015c1b0 msgr2=0x7ff90006ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:08.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.626+0000 7ff904a00640 1 --2- 172.21.15.32:0/3793828548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff90015c1b0 0x7ff90006ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:08.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.626+0000 7ff904a00640 1 -- 172.21.15.32:0/3793828548 >> v1:172.21.15.32:6789/0 conn(0x7ff9000b4de0 legacy=0x7ff90015baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:08.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.626+0000 7ff904a00640 1 -- 172.21.15.32:0/3793828548 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9000b40a0 con 0x7ff900144e90 2024-04-17T09:59:08.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.630+0000 7ff904a00640 1 --2- 172.21.15.32:0/3793828548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff900144e90 0x7ff9000b48a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00097c0 tx=0x7ff8f001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1d7354f8880e1af server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:08.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.630+0000 7ff8ffe00640 1 -- 172.21.15.32:0/3793828548 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f00b2070 con 0x7ff900144e90 2024-04-17T09:59:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.630+0000 7ff8ffe00640 1 -- 172.21.15.32:0/3793828548 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f00ad680 con 0x7ff900144e90 2024-04-17T09:59:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.630+0000 7ff8ffe00640 1 -- 172.21.15.32:0/3793828548 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f00ada30 con 0x7ff900144e90 2024-04-17T09:59:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.630+0000 7ff907400640 1 -- 172.21.15.32:0/3793828548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff900144e90 msgr2=0x7ff9000b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.630+0000 7ff907400640 1 --2- 172.21.15.32:0/3793828548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff900144e90 0x7ff9000b48a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00097c0 tx=0x7ff8f001c470 comp rx=0 tx=0).stop 2024-04-17T09:59:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.630+0000 7ff907400640 1 -- 172.21.15.32:0/3793828548 shutdown_connections 2024-04-17T09:59:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.630+0000 7ff907400640 1 --2- 172.21.15.32:0/3793828548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff90015c1b0 0x7ff90006ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.630+0000 7ff907400640 1 --2- 172.21.15.32:0/3793828548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff900144e90 0x7ff9000b48a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.630+0000 7ff907400640 1 -- 172.21.15.32:0/3793828548 >> 172.21.15.32:0/3793828548 conn(0x7ff90006ba30 msgr2=0x7ff90006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.630+0000 7ff907400640 1 -- 172.21.15.32:0/3793828548 shutdown_connections 2024-04-17T09:59:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.630+0000 7ff907400640 1 -- 172.21.15.32:0/3793828548 wait complete. 2024-04-17T09:59:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.630+0000 7ff907400640 1 Processor -- start 2024-04-17T09:59:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.630+0000 7ff907400640 1 -- start start 2024-04-17T09:59:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff907400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9000b4de0 0x7ff900165890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:08.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff907400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff900144e90 0x7ff90015da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:08.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff907400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff90015c1b0 0x7ff90015df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:08.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff907400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff90006de50 con 0x7ff9000b4de0 2024-04-17T09:59:08.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff907400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff900072880 con 0x7ff90015c1b0 2024-04-17T09:59:08.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff907400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff90006dfd0 con 0x7ff900144e90 2024-04-17T09:59:08.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff904a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9000b4de0 0x7ff900165890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:08.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff904a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9000b4de0 0x7ff900165890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34636/0 (socket says 172.21.15.32:34636) 2024-04-17T09:59:08.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff904a00640 1 -- 172.21.15.32:0/2308583203 learned_addr learned my addr 172.21.15.32:0/2308583203 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:08.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff905400640 1 --2- 172.21.15.32:0/2308583203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff90015c1b0 0x7ff90015df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:08.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff8f7e00640 1 --2- 172.21.15.32:0/2308583203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff900144e90 0x7ff90015da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:08.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff904a00640 1 -- 172.21.15.32:0/2308583203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff900144e90 msgr2=0x7ff90015da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:08.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff904a00640 1 --2- 172.21.15.32:0/2308583203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff900144e90 0x7ff90015da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:08.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff904a00640 1 -- 172.21.15.32:0/2308583203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff90015c1b0 msgr2=0x7ff90015df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:08.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff904a00640 1 --2- 172.21.15.32:0/2308583203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff90015c1b0 0x7ff90015df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:08.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff904a00640 1 -- 172.21.15.32:0/2308583203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9001f7270 con 0x7ff9000b4de0 2024-04-17T09:59:08.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff904a00640 1 --2- 172.21.15.32:0/2308583203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9000b4de0 0x7ff900165890 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0000c10 tx=0x7ff8f0019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:08.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff8fd600640 1 -- 172.21.15.32:0/2308583203 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f00bc070 con 0x7ff9000b4de0 2024-04-17T09:59:08.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff8fd600640 1 -- 172.21.15.32:0/2308583203 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f00ad830 con 0x7ff9000b4de0 2024-04-17T09:59:08.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff8fd600640 1 -- 172.21.15.32:0/2308583203 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f00b2040 con 0x7ff9000b4de0 2024-04-17T09:59:08.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff907400640 1 -- 172.21.15.32:0/2308583203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9001f7550 con 0x7ff9000b4de0 2024-04-17T09:59:08.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff907400640 1 -- 172.21.15.32:0/2308583203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9001f7b10 con 0x7ff9000b4de0 2024-04-17T09:59:08.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff907400640 1 -- 172.21.15.32:0/2308583203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff900068a90 con 0x7ff9000b4de0 2024-04-17T09:59:08.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.634+0000 7ff8fd600640 1 -- 172.21.15.32:0/2308583203 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff8f00adbe0 con 0x7ff9000b4de0 2024-04-17T09:59:08.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.638+0000 7ff8fd600640 1 --2- 172.21.15.32:0/2308583203 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff8dc068380 0x7ff8dc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:08.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.638+0000 7ff8fd600640 1 -- 172.21.15.32:0/2308583203 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7ff8f011fce0 con 0x7ff9000b4de0 2024-04-17T09:59:08.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.638+0000 7ff8f7e00640 1 --2- 172.21.15.32:0/2308583203 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff8dc068380 0x7ff8dc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:08.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.638+0000 7ff8f7e00640 1 --2- 172.21.15.32:0/2308583203 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff8dc068380 0x7ff8dc06a840 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8007fd0 tx=0x7ff8e8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:08.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.642+0000 7ff8fd600640 1 -- 172.21.15.32:0/2308583203 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff8f0016090 con 0x7ff9000b4de0 2024-04-17T09:59:08.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.842+0000 7ff907400640 1 -- 172.21.15.32:0/2308583203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9000ada40 con 0x7ff9000b4de0 2024-04-17T09:59:08.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.842+0000 7ff8fd600640 1 -- 172.21.15.32:0/2308583203 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5204 (secure 0 0 0) 0x7ff8f00efbb0 con 0x7ff9000b4de0 2024-04-17T09:59:08.839 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:59:08.839 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6419,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3534108103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3534108103},{"type":"v1","addr":"172.21.15.144:6835","nonce":3534108103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7003,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/935945019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":935945019},{"type":"v1","addr":"172.21.15.144:6834","nonce":935945019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3044362393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3044362393},{"type":"v1","addr":"172.21.15.32:6836","nonce":3044362393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:59:04.834110+0000","modified":"2024-04-17T09:59:07.850260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7557":{"gid":7557,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/2656735817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2656735817},{"type":"v1","addr":"172.21.15.32:6837","nonce":2656735817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7557,"qdb_cluster":[7557]},"id":6}]} 2024-04-17T09:59:08.839 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 59 2024-04-17T09:59:08.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.842+0000 7ff907400640 1 -- 172.21.15.32:0/2308583203 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff8dc068380 msgr2=0x7ff8dc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:08.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.842+0000 7ff907400640 1 --2- 172.21.15.32:0/2308583203 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff8dc068380 0x7ff8dc06a840 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8007fd0 tx=0x7ff8e8009040 comp rx=0 tx=0).stop 2024-04-17T09:59:08.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.842+0000 7ff907400640 1 -- 172.21.15.32:0/2308583203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9000b4de0 msgr2=0x7ff900165890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:08.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.842+0000 7ff907400640 1 --2- 172.21.15.32:0/2308583203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9000b4de0 0x7ff900165890 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0000c10 tx=0x7ff8f0019210 comp rx=0 tx=0).stop 2024-04-17T09:59:08.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.842+0000 7ff907400640 1 -- 172.21.15.32:0/2308583203 shutdown_connections 2024-04-17T09:59:08.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.842+0000 7ff907400640 1 --2- 172.21.15.32:0/2308583203 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff8dc068380 0x7ff8dc06a840 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:08.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.842+0000 7ff907400640 1 --2- 172.21.15.32:0/2308583203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff900144e90 0x7ff90015da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:08.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.842+0000 7ff907400640 1 --2- 172.21.15.32:0/2308583203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff90015c1b0 0x7ff90015df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:08.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.842+0000 7ff907400640 1 --2- 172.21.15.32:0/2308583203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9000b4de0 0x7ff900165890 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:08.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.842+0000 7ff907400640 1 -- 172.21.15.32:0/2308583203 >> 172.21.15.32:0/2308583203 conn(0x7ff90006ba30 msgr2=0x7ff90006e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:08.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.842+0000 7ff907400640 1 -- 172.21.15.32:0/2308583203 shutdown_connections 2024-04-17T09:59:08.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:08.842+0000 7ff907400640 1 -- 172.21.15.32:0/2308583203 wait complete. 2024-04-17T09:59:08.861 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:59:09.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.002+0000 7f49a2400640 1 Processor -- start 2024-04-17T09:59:09.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.002+0000 7f49a2400640 1 -- start start 2024-04-17T09:59:09.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f49a2400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f499c145f30 0x7f499c071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:09.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f49a2400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f499c0cdac0 con 0x7f499c15d8d0 2024-04-17T09:59:09.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f49a2400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f499c15cdf0 con 0x7f499c0cd580 2024-04-17T09:59:09.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f49a2400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f499c06f400 con 0x7f499c145f30 2024-04-17T09:59:09.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f4993e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f499c15d8d0 0x7f499c0cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38064/0 (socket says 172.21.15.32:38064) 2024-04-17T09:59:09.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f4993e00640 1 -- 172.21.15.32:0/675539612 learned_addr learned my addr 172.21.15.32:0/675539612 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:09.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f499b400640 1 --2- 172.21.15.32:0/675539612 >> v2:172.21.15.144:3300/0 conn(0x7f499c145f30 0x7f499c071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:09.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f499aa00640 1 -- 172.21.15.32:0/675539612 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1050669237 0 0) 0x7f499c0cdac0 con 0x7f499c15d8d0 2024-04-17T09:59:09.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f499aa00640 1 -- 172.21.15.32:0/675539612 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4980003530 con 0x7f499c15d8d0 2024-04-17T09:59:09.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f499aa00640 1 -- 172.21.15.32:0/675539612 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 271020358 0 0) 0x7f4980003530 con 0x7f499c15d8d0 2024-04-17T09:59:09.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f499aa00640 1 -- 172.21.15.32:0/675539612 >> v2:172.21.15.144:3300/0 conn(0x7f499c145f30 msgr2=0x7f499c071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:09.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f499aa00640 1 --2- 172.21.15.32:0/675539612 >> v2:172.21.15.144:3300/0 conn(0x7f499c145f30 0x7f499c071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f499aa00640 1 -- 172.21.15.32:0/675539612 >> v1:172.21.15.144:6789/0 conn(0x7f499c0cd580 legacy=0x7f499c1601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:09.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f499aa00640 1 -- 172.21.15.32:0/675539612 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499c15d0d0 con 0x7f499c15d8d0 2024-04-17T09:59:09.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f499aa00640 1 -- 172.21.15.32:0/675539612 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f4984001680 con 0x7f499c15d8d0 2024-04-17T09:59:09.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f499aa00640 1 -- 172.21.15.32:0/675539612 >> v1:172.21.15.32:6789/0 conn(0x7f499c15d8d0 legacy=0x7f499c0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:09.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f499aa00640 1 --2- 172.21.15.32:0/675539612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49800039c0 0x7f4980003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:09.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f499aa00640 1 --2- 172.21.15.32:0/675539612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4980004810 0x7f4980004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:09.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f499aa00640 1 --2- 172.21.15.32:0/675539612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49800051d0 0x7f498000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:09.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f499aa00640 1 -- 172.21.15.32:0/675539612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f499c06f400 con 0x7f4980004810 2024-04-17T09:59:09.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f499aa00640 1 -- 172.21.15.32:0/675539612 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f498000c080 con 0x7f49800051d0 2024-04-17T09:59:09.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f499aa00640 1 -- 172.21.15.32:0/675539612 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f498000c280 con 0x7f49800039c0 2024-04-17T09:59:09.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f4993e00640 1 --2- 172.21.15.32:0/675539612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4980004810 0x7f4980004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:09.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f499b400640 1 --2- 172.21.15.32:0/675539612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49800039c0 0x7f4980003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:09.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f499be00640 1 --2- 172.21.15.32:0/675539612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49800051d0 0x7f498000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:09.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f4993e00640 1 -- 172.21.15.32:0/675539612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49800039c0 msgr2=0x7f4980003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:09.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f4993e00640 1 --2- 172.21.15.32:0/675539612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49800039c0 0x7f4980003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f4993e00640 1 -- 172.21.15.32:0/675539612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49800051d0 msgr2=0x7f498000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:09.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f4993e00640 1 --2- 172.21.15.32:0/675539612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49800051d0 0x7f498000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f4993e00640 1 -- 172.21.15.32:0/675539612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4980003530 con 0x7f4980004810 2024-04-17T09:59:09.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f499b400640 1 --2- 172.21.15.32:0/675539612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49800039c0 0x7f4980003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:09.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.006+0000 7f499be00640 1 -- 172.21.15.32:0/675539612 reap_dead start 2024-04-17T09:59:09.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.010+0000 7f4993e00640 1 --2- 172.21.15.32:0/675539612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4980004810 0x7f4980004c90 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f498400c240 tx=0x7f498401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4fd31adc1ca735 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:09.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.010+0000 7f499aa00640 1 -- 172.21.15.32:0/675539612 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4984007d20 con 0x7f4980004810 2024-04-17T09:59:09.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.010+0000 7f499aa00640 1 -- 172.21.15.32:0/675539612 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49840a9070 con 0x7f4980004810 2024-04-17T09:59:09.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.010+0000 7f499aa00640 1 -- 172.21.15.32:0/675539612 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49840a5e20 con 0x7f4980004810 2024-04-17T09:59:09.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.010+0000 7f49a2400640 1 -- 172.21.15.32:0/675539612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4980004810 msgr2=0x7f4980004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:09.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.010+0000 7f49a2400640 1 --2- 172.21.15.32:0/675539612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4980004810 0x7f4980004c90 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f498400c240 tx=0x7f498401cc80 comp rx=0 tx=0).stop 2024-04-17T09:59:09.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.010+0000 7f49a2400640 1 -- 172.21.15.32:0/675539612 shutdown_connections 2024-04-17T09:59:09.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.010+0000 7f49a2400640 1 --2- 172.21.15.32:0/675539612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4980004810 0x7f4980004c90 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.010+0000 7f49a2400640 1 -- 172.21.15.32:0/675539612 >> 172.21.15.32:0/675539612 conn(0x7f499c06ba30 msgr2=0x7f499c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:09.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.010+0000 7f49a2400640 1 -- 172.21.15.32:0/675539612 shutdown_connections 2024-04-17T09:59:09.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.010+0000 7f49a2400640 1 -- 172.21.15.32:0/675539612 wait complete. 2024-04-17T09:59:09.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f49a2400640 1 Processor -- start 2024-04-17T09:59:09.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f49a2400640 1 -- start start 2024-04-17T09:59:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f49a2400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4980004810 0x7f499c15f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f49a2400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f499c1403a0 0x7f499c15f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f49a2400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f499c13ff20 0x7f499c1fc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f49a2400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f499c0cd200 con 0x7f4980004810 2024-04-17T09:59:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f49a2400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f499c0cd080 con 0x7f499c13ff20 2024-04-17T09:59:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f49a2400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f499c0cd380 con 0x7f499c1403a0 2024-04-17T09:59:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f499b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4980004810 0x7f499c15f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f499b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4980004810 0x7f499c15f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34652/0 (socket says 172.21.15.32:34652) 2024-04-17T09:59:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f499b400640 1 -- 172.21.15.32:0/2542389769 learned_addr learned my addr 172.21.15.32:0/2542389769 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f4993e00640 1 --2- 172.21.15.32:0/2542389769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f499c1403a0 0x7f499c15f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f499be00640 1 --2- 172.21.15.32:0/2542389769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f499c13ff20 0x7f499c1fc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f499b400640 1 -- 172.21.15.32:0/2542389769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f499c1403a0 msgr2=0x7f499c15f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f499b400640 1 --2- 172.21.15.32:0/2542389769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f499c1403a0 0x7f499c15f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f499b400640 1 -- 172.21.15.32:0/2542389769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f499c13ff20 msgr2=0x7f499c1fc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:09.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f499b400640 1 --2- 172.21.15.32:0/2542389769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f499c13ff20 0x7f499c1fc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f499b400640 1 -- 172.21.15.32:0/2542389769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499c1e8350 con 0x7f4980004810 2024-04-17T09:59:09.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f499be00640 1 --2- 172.21.15.32:0/2542389769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f499c13ff20 0x7f499c1fc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:09.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f499b400640 1 --2- 172.21.15.32:0/2542389769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4980004810 0x7f499c15f2e0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f498c011510 tx=0x7f498c0119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:09.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f4993400640 1 -- 172.21.15.32:0/2542389769 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c004220 con 0x7f4980004810 2024-04-17T09:59:09.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f4993400640 1 -- 172.21.15.32:0/2542389769 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f498c0043c0 con 0x7f4980004810 2024-04-17T09:59:09.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f49a2400640 1 -- 172.21.15.32:0/2542389769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f499c0b55a0 con 0x7f4980004810 2024-04-17T09:59:09.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f4993400640 1 -- 172.21.15.32:0/2542389769 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c015890 con 0x7f4980004810 2024-04-17T09:59:09.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f49a2400640 1 -- 172.21.15.32:0/2542389769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f499c0c9c40 con 0x7f4980004810 2024-04-17T09:59:09.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.014+0000 7f49a2400640 1 -- 172.21.15.32:0/2542389769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f499c068a90 con 0x7f4980004810 2024-04-17T09:59:09.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.018+0000 7f4993400640 1 -- 172.21.15.32:0/2542389769 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f498c015a30 con 0x7f4980004810 2024-04-17T09:59:09.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.018+0000 7f4993400640 1 --2- 172.21.15.32:0/2542389769 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f49780683b0 0x7f497806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:09.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.018+0000 7f4993400640 1 -- 172.21.15.32:0/2542389769 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f498c086e30 con 0x7f4980004810 2024-04-17T09:59:09.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.018+0000 7f4993e00640 1 --2- 172.21.15.32:0/2542389769 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f49780683b0 0x7f497806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:09.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.018+0000 7f4993e00640 1 --2- 172.21.15.32:0/2542389769 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f49780683b0 0x7f497806a870 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f49840ab000 tx=0x7f49840a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:09.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.022+0000 7f4993400640 1 -- 172.21.15.32:0/2542389769 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f498c019030 con 0x7f4980004810 2024-04-17T09:59:09.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.186+0000 7f49a2400640 1 -- 172.21.15.32:0/2542389769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f499c072f90 con 0x7f4980004810 2024-04-17T09:59:09.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.190+0000 7f4993400640 1 -- 172.21.15.32:0/2542389769 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16672 (secure 0 0 0) 0x7f498c056d90 con 0x7f4980004810 2024-04-17T09:59:09.187 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:59:09.188 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":82,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:59:06.422274+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:59:03.223426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-17T09:59:03.780120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:59:09.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.190+0000 7f49a2400640 1 -- 172.21.15.32:0/2542389769 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f49780683b0 msgr2=0x7f497806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:09.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.190+0000 7f49a2400640 1 --2- 172.21.15.32:0/2542389769 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f49780683b0 0x7f497806a870 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f49840ab000 tx=0x7f49840a7070 comp rx=0 tx=0).stop 2024-04-17T09:59:09.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.190+0000 7f49a2400640 1 -- 172.21.15.32:0/2542389769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4980004810 msgr2=0x7f499c15f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:09.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.190+0000 7f49a2400640 1 --2- 172.21.15.32:0/2542389769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4980004810 0x7f499c15f2e0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f498c011510 tx=0x7f498c0119d0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.190+0000 7f49a2400640 1 -- 172.21.15.32:0/2542389769 shutdown_connections 2024-04-17T09:59:09.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.190+0000 7f49a2400640 1 --2- 172.21.15.32:0/2542389769 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f49780683b0 0x7f497806a870 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.190+0000 7f49a2400640 1 --2- 172.21.15.32:0/2542389769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f499c1403a0 0x7f499c15f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.190+0000 7f49a2400640 1 --2- 172.21.15.32:0/2542389769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f499c13ff20 0x7f499c1fc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.190+0000 7f49a2400640 1 --2- 172.21.15.32:0/2542389769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4980004810 0x7f499c15f2e0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.190+0000 7f49a2400640 1 -- 172.21.15.32:0/2542389769 >> 172.21.15.32:0/2542389769 conn(0x7f499c06ba30 msgr2=0x7f499c15e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:09.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.190+0000 7f49a2400640 1 -- 172.21.15.32:0/2542389769 shutdown_connections 2024-04-17T09:59:09.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.190+0000 7f49a2400640 1 -- 172.21.15.32:0/2542389769 wait complete. 2024-04-17T09:59:09.209 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T09:59:09.209 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T09:59:09.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.354+0000 7fc20da00640 1 Processor -- start 2024-04-17T09:59:09.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.354+0000 7fc20da00640 1 -- start start 2024-04-17T09:59:09.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.354+0000 7fc20da00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc208145f30 0x7fc208071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:09.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.354+0000 7fc20da00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc20815d8d0 0x7fc208072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:09.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.354+0000 7fc20da00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2080cdac0 con 0x7fc2080cce70 2024-04-17T09:59:09.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.354+0000 7fc20da00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc20806f400 con 0x7fc208145f30 2024-04-17T09:59:09.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.354+0000 7fc20da00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc208146310 con 0x7fc20815d8d0 2024-04-17T09:59:09.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.354+0000 7fc206a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc208145f30 0x7fc208071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:09.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.354+0000 7fc206a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc208145f30 0x7fc208071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34662/0 (socket says 172.21.15.32:34662) 2024-04-17T09:59:09.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.354+0000 7fc206a00640 1 -- 172.21.15.32:0/1290996541 learned_addr learned my addr 172.21.15.32:0/1290996541 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:09.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.354+0000 7fc1ffe00640 1 --2- 172.21.15.32:0/1290996541 >> v2:172.21.15.144:3300/0 conn(0x7fc20815d8d0 0x7fc208072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:09.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.354+0000 7fc206a00640 1 -- 172.21.15.32:0/1290996541 >> v2:172.21.15.144:3300/0 conn(0x7fc20815d8d0 msgr2=0x7fc208072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:09.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.354+0000 7fc206a00640 1 --2- 172.21.15.32:0/1290996541 >> v2:172.21.15.144:3300/0 conn(0x7fc20815d8d0 0x7fc208072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.354+0000 7fc206a00640 1 -- 172.21.15.32:0/1290996541 >> v1:172.21.15.144:6789/0 conn(0x7fc2080cce70 legacy=0x7fc2080cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:09.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.354+0000 7fc206a00640 1 -- 172.21.15.32:0/1290996541 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20815cdf0 con 0x7fc208145f30 2024-04-17T09:59:09.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.354+0000 7fc1ffe00640 1 --2- 172.21.15.32:0/1290996541 >> v2:172.21.15.144:3300/0 conn(0x7fc20815d8d0 0x7fc208072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:09.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.358+0000 7fc206a00640 1 --2- 172.21.15.32:0/1290996541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc208145f30 0x7fc208071df0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fc2080702a0 tx=0x7fc1f801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4360bb2b79bea8c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:09.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.358+0000 7fc206000640 1 -- 172.21.15.32:0/1290996541 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f80b5070 con 0x7fc208145f30 2024-04-17T09:59:09.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.358+0000 7fc206000640 1 -- 172.21.15.32:0/1290996541 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1f80a8a10 con 0x7fc208145f30 2024-04-17T09:59:09.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.358+0000 7fc206000640 1 -- 172.21.15.32:0/1290996541 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f80a8de0 con 0x7fc208145f30 2024-04-17T09:59:09.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.358+0000 7fc20da00640 1 -- 172.21.15.32:0/1290996541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc208145f30 msgr2=0x7fc208071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:09.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.358+0000 7fc20da00640 1 --2- 172.21.15.32:0/1290996541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc208145f30 0x7fc208071df0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fc2080702a0 tx=0x7fc1f801ed10 comp rx=0 tx=0).stop 2024-04-17T09:59:09.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.358+0000 7fc20da00640 1 -- 172.21.15.32:0/1290996541 shutdown_connections 2024-04-17T09:59:09.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.358+0000 7fc20da00640 1 --2- 172.21.15.32:0/1290996541 >> v2:172.21.15.144:3300/0 conn(0x7fc20815d8d0 0x7fc208072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.358+0000 7fc20da00640 1 --2- 172.21.15.32:0/1290996541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc208145f30 0x7fc208071df0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.358+0000 7fc20da00640 1 -- 172.21.15.32:0/1290996541 >> 172.21.15.32:0/1290996541 conn(0x7fc20806ba30 msgr2=0x7fc20806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:09.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.358+0000 7fc20da00640 1 -- 172.21.15.32:0/1290996541 shutdown_connections 2024-04-17T09:59:09.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.358+0000 7fc20da00640 1 -- 172.21.15.32:0/1290996541 wait complete. 2024-04-17T09:59:09.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc20da00640 1 Processor -- start 2024-04-17T09:59:09.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc20da00640 1 -- start start 2024-04-17T09:59:09.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc20da00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2080cce70 0x7fc2081f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:09.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc20da00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc208145f30 0x7fc2080ab620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:09.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc20da00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc20815d8d0 0x7fc2080abb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:09.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc20da00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc20815cb90 con 0x7fc208145f30 2024-04-17T09:59:09.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc20da00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc20815d440 con 0x7fc2080cce70 2024-04-17T09:59:09.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc20da00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc2081601d0 con 0x7fc20815d8d0 2024-04-17T09:59:09.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc1ffe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc208145f30 0x7fc2080ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:09.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc1ffe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc208145f30 0x7fc2080ab620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34668/0 (socket says 172.21.15.32:34668) 2024-04-17T09:59:09.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc1ffe00640 1 -- 172.21.15.32:0/719681778 learned_addr learned my addr 172.21.15.32:0/719681778 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:09.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc206a00640 1 --2- 172.21.15.32:0/719681778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2080cce70 0x7fc2081f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:09.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc207400640 1 --2- 172.21.15.32:0/719681778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc20815d8d0 0x7fc2080abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:09.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc1ffe00640 1 -- 172.21.15.32:0/719681778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc20815d8d0 msgr2=0x7fc2080abb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:09.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc1ffe00640 1 --2- 172.21.15.32:0/719681778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc20815d8d0 0x7fc2080abb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc1ffe00640 1 -- 172.21.15.32:0/719681778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2080cce70 msgr2=0x7fc2081f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:09.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc1ffe00640 1 --2- 172.21.15.32:0/719681778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2080cce70 0x7fc2081f5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc1ffe00640 1 -- 172.21.15.32:0/719681778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2080c97d0 con 0x7fc208145f30 2024-04-17T09:59:09.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc1ffe00640 1 --2- 172.21.15.32:0/719681778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc208145f30 0x7fc2080ab620 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fc1f000d800 tx=0x7fc1f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:09.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc1fea00640 1 -- 172.21.15.32:0/719681778 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0018020 con 0x7fc208145f30 2024-04-17T09:59:09.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc1fea00640 1 -- 172.21.15.32:0/719681778 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1f00096c0 con 0x7fc208145f30 2024-04-17T09:59:09.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc1fea00640 1 -- 172.21.15.32:0/719681778 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0009ab0 con 0x7fc208145f30 2024-04-17T09:59:09.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc20da00640 1 -- 172.21.15.32:0/719681778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2080c9ab0 con 0x7fc208145f30 2024-04-17T09:59:09.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc20da00640 1 -- 172.21.15.32:0/719681778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2080ca070 con 0x7fc208145f30 2024-04-17T09:59:09.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.362+0000 7fc20da00640 1 -- 172.21.15.32:0/719681778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc208068a90 con 0x7fc208145f30 2024-04-17T09:59:09.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.366+0000 7fc1fea00640 1 -- 172.21.15.32:0/719681778 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc1f001b020 con 0x7fc208145f30 2024-04-17T09:59:09.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.366+0000 7fc1fea00640 1 --2- 172.21.15.32:0/719681778 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc1e4068380 0x7fc1e406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:09.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.366+0000 7fc1fea00640 1 -- 172.21.15.32:0/719681778 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fc1f0087410 con 0x7fc208145f30 2024-04-17T09:59:09.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.366+0000 7fc206a00640 1 --2- 172.21.15.32:0/719681778 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc1e4068380 0x7fc1e406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:09.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.366+0000 7fc206a00640 1 --2- 172.21.15.32:0/719681778 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc1e4068380 0x7fc1e406a840 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80b2000 tx=0x7fc1f8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:09.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.370+0000 7fc1fea00640 1 -- 172.21.15.32:0/719681778 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc1f0022020 con 0x7fc208145f30 2024-04-17T09:59:09.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.534+0000 7fc20da00640 1 -- 172.21.15.32:0/719681778 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc208072f90 con 0x7fc1e4068380 2024-04-17T09:59:09.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.538+0000 7fc1fea00640 1 -- 172.21.15.32:0/719681778 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248008 (secure 0 0 0) 0x7fc208072f90 con 0x7fc1e4068380 2024-04-17T09:59:09.540 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:59:09.542 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T09:59:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.542+0000 7fc20da00640 1 -- 172.21.15.32:0/719681778 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc1e4068380 msgr2=0x7fc1e406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.542+0000 7fc20da00640 1 --2- 172.21.15.32:0/719681778 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc1e4068380 0x7fc1e406a840 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80b2000 tx=0x7fc1f8009040 comp rx=0 tx=0).stop 2024-04-17T09:59:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.542+0000 7fc20da00640 1 -- 172.21.15.32:0/719681778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc208145f30 msgr2=0x7fc2080ab620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.542+0000 7fc20da00640 1 --2- 172.21.15.32:0/719681778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc208145f30 0x7fc2080ab620 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fc1f000d800 tx=0x7fc1f000dcc0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.542+0000 7fc20da00640 1 -- 172.21.15.32:0/719681778 shutdown_connections 2024-04-17T09:59:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.542+0000 7fc20da00640 1 --2- 172.21.15.32:0/719681778 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc1e4068380 0x7fc1e406a840 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.542+0000 7fc20da00640 1 --2- 172.21.15.32:0/719681778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc20815d8d0 0x7fc2080abb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.542+0000 7fc20da00640 1 --2- 172.21.15.32:0/719681778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc208145f30 0x7fc2080ab620 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.542+0000 7fc20da00640 1 --2- 172.21.15.32:0/719681778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2080cce70 0x7fc2081f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.542+0000 7fc20da00640 1 -- 172.21.15.32:0/719681778 >> 172.21.15.32:0/719681778 conn(0x7fc20806ba30 msgr2=0x7fc20815ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.542+0000 7fc20da00640 1 -- 172.21.15.32:0/719681778 shutdown_connections 2024-04-17T09:59:09.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.542+0000 7fc20da00640 1 -- 172.21.15.32:0/719681778 wait complete. 2024-04-17T09:59:09.560 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-04-17T09:59:09.365238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":180,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258280,"kb_used_data":8816,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716440,"statfs":{"total":773094113280,"available":772829634560,"internally_reserved":0,"allocated":9027584,"data_stored":4494325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.953388"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.430338+0000","last_change":"2024-04-17T09:59:04.430338+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.430338+0000","last_undegraded":"2024-04-17T09:59:04.430338+0000","last_fullsized":"2024-04-17T09:59:04.430338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429509+0000","last_change":"2024-04-17T09:59:04.429509+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429509+0000","last_undegraded":"2024-04-17T09:59:04.429509+0000","last_fullsized":"2024-04-17T09:59:04.429509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433330+0000","last_change":"2024-04-17T09:59:04.433330+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433330+0000","last_undegraded":"2024-04-17T09:59:04.433330+0000","last_fullsized":"2024-04-17T09:59:04.433330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429769+0000","last_change":"2024-04-17T09:59:04.429769+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429769+0000","last_undegraded":"2024-04-17T09:59:04.429769+0000","last_fullsized":"2024-04-17T09:59:04.429769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427070+0000","last_change":"2024-04-17T09:59:05.435295+0000","last_active":"2024-04-17T09:59:06.427070+0000","last_peered":"2024-04-17T09:59:06.427070+0000","last_clean":"2024-04-17T09:59:06.427070+0000","last_became_active":"2024-04-17T09:59:05.434948+0000","last_became_peered":"2024-04-17T09:59:05.434948+0000","last_unstale":"2024-04-17T09:59:06.427070+0000","last_undegraded":"2024-04-17T09:59:06.427070+0000","last_fullsized":"2024-04-17T09:59:06.427070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:14.347809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.438674+0000","last_change":"2024-04-17T09:59:04.438674+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.438674+0000","last_undegraded":"2024-04-17T09:59:04.438674+0000","last_fullsized":"2024-04-17T09:59:04.438674+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429372+0000","last_change":"2024-04-17T09:59:04.429372+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429372+0000","last_undegraded":"2024-04-17T09:59:04.429372+0000","last_fullsized":"2024-04-17T09:59:04.429372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.437249+0000","last_change":"2024-04-17T09:59:04.437249+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.437249+0000","last_undegraded":"2024-04-17T09:59:04.437249+0000","last_fullsized":"2024-04-17T09:59:04.437249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.440846+0000","last_change":"2024-04-17T09:59:04.440846+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.440846+0000","last_undegraded":"2024-04-17T09:59:04.440846+0000","last_fullsized":"2024-04-17T09:59:04.440846+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427452+0000","last_change":"2024-04-17T09:59:04.427452+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427452+0000","last_undegraded":"2024-04-17T09:59:04.427452+0000","last_fullsized":"2024-04-17T09:59:04.427452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427477+0000","last_change":"2024-04-17T09:59:04.427477+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427477+0000","last_undegraded":"2024-04-17T09:59:04.427477+0000","last_fullsized":"2024-04-17T09:59:04.427477+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427835+0000","last_change":"2024-04-17T09:59:04.427835+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427835+0000","last_undegraded":"2024-04-17T09:59:04.427835+0000","last_fullsized":"2024-04-17T09:59:04.427835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433600+0000","last_change":"2024-04-17T09:59:04.433600+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433600+0000","last_undegraded":"2024-04-17T09:59:04.433600+0000","last_fullsized":"2024-04-17T09:59:04.433600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.435489+0000","last_change":"2024-04-17T09:59:04.435489+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.435489+0000","last_undegraded":"2024-04-17T09:59:04.435489+0000","last_fullsized":"2024-04-17T09:59:04.435489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427347+0000","last_change":"2024-04-17T09:59:04.427347+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427347+0000","last_undegraded":"2024-04-17T09:59:04.427347+0000","last_fullsized":"2024-04-17T09:59:04.427347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426220+0000","last_change":"2024-04-17T09:59:05.445968+0000","last_active":"2024-04-17T09:59:06.426220+0000","last_peered":"2024-04-17T09:59:06.426220+0000","last_clean":"2024-04-17T09:59:06.426220+0000","last_became_active":"2024-04-17T09:59:05.445695+0000","last_became_peered":"2024-04-17T09:59:05.445695+0000","last_unstale":"2024-04-17T09:59:06.426220+0000","last_undegraded":"2024-04-17T09:59:06.426220+0000","last_fullsized":"2024-04-17T09:59:06.426220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:27:15.977393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.428029+0000","last_change":"2024-04-17T09:59:05.447691+0000","last_active":"2024-04-17T09:59:06.428029+0000","last_peered":"2024-04-17T09:59:06.428029+0000","last_clean":"2024-04-17T09:59:06.428029+0000","last_became_active":"2024-04-17T09:59:05.447403+0000","last_became_peered":"2024-04-17T09:59:05.447403+0000","last_unstale":"2024-04-17T09:59:06.428029+0000","last_undegraded":"2024-04-17T09:59:06.428029+0000","last_fullsized":"2024-04-17T09:59:06.428029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:15.267856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.432446+0000","last_change":"2024-04-17T09:59:04.432446+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.432446+0000","last_undegraded":"2024-04-17T09:59:04.432446+0000","last_fullsized":"2024-04-17T09:59:04.432446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455142+0000","last_change":"2024-04-17T09:59:05.442262+0000","last_active":"2024-04-17T09:59:06.455142+0000","last_peered":"2024-04-17T09:59:06.455142+0000","last_clean":"2024-04-17T09:59:06.455142+0000","last_became_active":"2024-04-17T09:59:05.441944+0000","last_became_peered":"2024-04-17T09:59:05.441944+0000","last_unstale":"2024-04-17T09:59:06.455142+0000","last_undegraded":"2024-04-17T09:59:06.455142+0000","last_fullsized":"2024-04-17T09:59:06.455142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:55:27.872332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429260+0000","last_change":"2024-04-17T09:59:04.429260+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429260+0000","last_undegraded":"2024-04-17T09:59:04.429260+0000","last_fullsized":"2024-04-17T09:59:04.429260+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429463+0000","last_change":"2024-04-17T09:59:04.429463+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429463+0000","last_undegraded":"2024-04-17T09:59:04.429463+0000","last_fullsized":"2024-04-17T09:59:04.429463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428078+0000","last_change":"2024-04-17T09:59:04.428078+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428078+0000","last_undegraded":"2024-04-17T09:59:04.428078+0000","last_fullsized":"2024-04-17T09:59:04.428078+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427009+0000","last_change":"2024-04-17T09:59:04.427009+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427009+0000","last_undegraded":"2024-04-17T09:59:04.427009+0000","last_fullsized":"2024-04-17T09:59:04.427009+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428239+0000","last_change":"2024-04-17T09:59:04.428239+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428239+0000","last_undegraded":"2024-04-17T09:59:04.428239+0000","last_fullsized":"2024-04-17T09:59:04.428239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.425424+0000","last_change":"2024-04-17T09:59:04.425424+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.425424+0000","last_undegraded":"2024-04-17T09:59:04.425424+0000","last_fullsized":"2024-04-17T09:59:04.425424+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.432662+0000","last_change":"2024-04-17T09:59:04.432662+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.432662+0000","last_undegraded":"2024-04-17T09:59:04.432662+0000","last_fullsized":"2024-04-17T09:59:04.432662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427077+0000","last_change":"2024-04-17T09:59:05.438772+0000","last_active":"2024-04-17T09:59:06.427077+0000","last_peered":"2024-04-17T09:59:06.427077+0000","last_clean":"2024-04-17T09:59:06.427077+0000","last_became_active":"2024-04-17T09:59:05.438444+0000","last_became_peered":"2024-04-17T09:59:05.438444+0000","last_unstale":"2024-04-17T09:59:06.427077+0000","last_undegraded":"2024-04-17T09:59:06.427077+0000","last_fullsized":"2024-04-17T09:59:06.427077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:53:56.012120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429291+0000","last_change":"2024-04-17T09:59:04.429291+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429291+0000","last_undegraded":"2024-04-17T09:59:04.429291+0000","last_fullsized":"2024-04-17T09:59:04.429291+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427246+0000","last_change":"2024-04-17T09:59:04.427246+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427246+0000","last_undegraded":"2024-04-17T09:59:04.427246+0000","last_fullsized":"2024-04-17T09:59:04.427246+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.426566+0000","last_change":"2024-04-17T09:59:04.426566+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.426566+0000","last_undegraded":"2024-04-17T09:59:04.426566+0000","last_fullsized":"2024-04-17T09:59:04.426566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426520+0000","last_change":"2024-04-17T09:59:05.438189+0000","last_active":"2024-04-17T09:59:06.426520+0000","last_peered":"2024-04-17T09:59:06.426520+0000","last_clean":"2024-04-17T09:59:06.426520+0000","last_became_active":"2024-04-17T09:59:05.437898+0000","last_became_peered":"2024-04-17T09:59:05.437898+0000","last_unstale":"2024-04-17T09:59:06.426520+0000","last_undegraded":"2024-04-17T09:59:06.426520+0000","last_fullsized":"2024-04-17T09:59:06.426520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:55.256201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428134+0000","last_change":"2024-04-17T09:59:04.428134+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428134+0000","last_undegraded":"2024-04-17T09:59:04.428134+0000","last_fullsized":"2024-04-17T09:59:04.428134+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.428003+0000","last_change":"2024-04-17T09:59:05.445880+0000","last_active":"2024-04-17T09:59:06.428003+0000","last_peered":"2024-04-17T09:59:06.428003+0000","last_clean":"2024-04-17T09:59:06.428003+0000","last_became_active":"2024-04-17T09:59:05.439231+0000","last_became_peered":"2024-04-17T09:59:05.439231+0000","last_unstale":"2024-04-17T09:59:06.428003+0000","last_undegraded":"2024-04-17T09:59:06.428003+0000","last_fullsized":"2024-04-17T09:59:06.428003+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:40.503102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.445224+0000","last_change":"2024-04-17T09:59:04.443338+0000","last_active":"2024-04-17T09:59:04.445224+0000","last_peered":"2024-04-17T09:59:04.445224+0000","last_clean":"2024-04-17T09:59:04.445224+0000","last_became_active":"2024-04-17T09:59:04.442697+0000","last_became_peered":"2024-04-17T09:59:04.442697+0000","last_unstale":"2024-04-17T09:59:04.445224+0000","last_undegraded":"2024-04-17T09:59:04.445224+0000","last_fullsized":"2024-04-17T09:59:04.445224+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:39:16.180270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429909+0000","last_change":"2024-04-17T09:59:04.429909+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429909+0000","last_undegraded":"2024-04-17T09:59:04.429909+0000","last_fullsized":"2024-04-17T09:59:04.429909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.428060+0000","last_change":"2024-04-17T09:59:04.439722+0000","last_active":"2024-04-17T09:59:06.428060+0000","last_peered":"2024-04-17T09:59:06.428060+0000","last_clean":"2024-04-17T09:59:06.428060+0000","last_became_active":"2024-04-17T09:59:04.439452+0000","last_became_peered":"2024-04-17T09:59:04.439452+0000","last_unstale":"2024-04-17T09:59:06.428060+0000","last_undegraded":"2024-04-17T09:59:06.428060+0000","last_fullsized":"2024-04-17T09:59:06.428060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:49:05.349813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428740+0000","last_change":"2024-04-17T09:59:04.428740+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428740+0000","last_undegraded":"2024-04-17T09:59:04.428740+0000","last_fullsized":"2024-04-17T09:59:04.428740+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455165+0000","last_change":"2024-04-17T09:59:04.444375+0000","last_active":"2024-04-17T09:59:06.455165+0000","last_peered":"2024-04-17T09:59:06.455165+0000","last_clean":"2024-04-17T09:59:06.455165+0000","last_became_active":"2024-04-17T09:59:04.444125+0000","last_became_peered":"2024-04-17T09:59:04.444125+0000","last_unstale":"2024-04-17T09:59:06.455165+0000","last_undegraded":"2024-04-17T09:59:06.455165+0000","last_fullsized":"2024-04-17T09:59:06.455165+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:07.141412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433823+0000","last_change":"2024-04-17T09:59:04.433823+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433823+0000","last_undegraded":"2024-04-17T09:59:04.433823+0000","last_fullsized":"2024-04-17T09:59:04.433823+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455155+0000","last_change":"2024-04-17T09:59:04.443042+0000","last_active":"2024-04-17T09:59:06.455155+0000","last_peered":"2024-04-17T09:59:06.455155+0000","last_clean":"2024-04-17T09:59:06.455155+0000","last_became_active":"2024-04-17T09:59:04.442701+0000","last_became_peered":"2024-04-17T09:59:04.442701+0000","last_unstale":"2024-04-17T09:59:06.455155+0000","last_undegraded":"2024-04-17T09:59:06.455155+0000","last_fullsized":"2024-04-17T09:59:06.455155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:02:06.697176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.425419+0000","last_change":"2024-04-17T09:59:04.425419+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.425419+0000","last_undegraded":"2024-04-17T09:59:04.425419+0000","last_fullsized":"2024-04-17T09:59:04.425419+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427370+0000","last_change":"2024-04-17T09:59:04.438652+0000","last_active":"2024-04-17T09:59:06.427370+0000","last_peered":"2024-04-17T09:59:06.427370+0000","last_clean":"2024-04-17T09:59:06.427370+0000","last_became_active":"2024-04-17T09:59:04.438385+0000","last_became_peered":"2024-04-17T09:59:04.438385+0000","last_unstale":"2024-04-17T09:59:06.427370+0000","last_undegraded":"2024-04-17T09:59:06.427370+0000","last_fullsized":"2024-04-17T09:59:06.427370+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:12.392733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427235+0000","last_change":"2024-04-17T09:59:05.436633+0000","last_active":"2024-04-17T09:59:06.427235+0000","last_peered":"2024-04-17T09:59:06.427235+0000","last_clean":"2024-04-17T09:59:06.427235+0000","last_became_active":"2024-04-17T09:59:05.436348+0000","last_became_peered":"2024-04-17T09:59:05.436348+0000","last_unstale":"2024-04-17T09:59:06.427235+0000","last_undegraded":"2024-04-17T09:59:06.427235+0000","last_fullsized":"2024-04-17T09:59:06.427235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:08.147437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426666+0000","last_change":"2024-04-17T09:59:04.440021+0000","last_active":"2024-04-17T09:59:06.426666+0000","last_peered":"2024-04-17T09:59:06.426666+0000","last_clean":"2024-04-17T09:59:06.426666+0000","last_became_active":"2024-04-17T09:59:04.439185+0000","last_became_peered":"2024-04-17T09:59:04.439185+0000","last_unstale":"2024-04-17T09:59:06.426666+0000","last_undegraded":"2024-04-17T09:59:06.426666+0000","last_fullsized":"2024-04-17T09:59:06.426666+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:51.158495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433375+0000","last_change":"2024-04-17T09:59:04.433375+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433375+0000","last_undegraded":"2024-04-17T09:59:04.433375+0000","last_fullsized":"2024-04-17T09:59:04.433375+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.454817+0000","last_change":"2024-04-17T09:59:04.449411+0000","last_active":"2024-04-17T09:59:06.454817+0000","last_peered":"2024-04-17T09:59:06.454817+0000","last_clean":"2024-04-17T09:59:06.454817+0000","last_became_active":"2024-04-17T09:59:04.449080+0000","last_became_peered":"2024-04-17T09:59:04.449080+0000","last_unstale":"2024-04-17T09:59:06.454817+0000","last_undegraded":"2024-04-17T09:59:06.454817+0000","last_fullsized":"2024-04-17T09:59:06.454817+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:13.341784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.426818+0000","last_change":"2024-04-17T09:59:04.426818+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.426818+0000","last_undegraded":"2024-04-17T09:59:04.426818+0000","last_fullsized":"2024-04-17T09:59:04.426818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.438105+0000","last_change":"2024-04-17T09:59:04.433580+0000","last_active":"2024-04-17T09:59:04.438105+0000","last_peered":"2024-04-17T09:59:04.438105+0000","last_clean":"2024-04-17T09:59:04.438105+0000","last_became_active":"2024-04-17T09:59:04.433250+0000","last_became_peered":"2024-04-17T09:59:04.433250+0000","last_unstale":"2024-04-17T09:59:04.438105+0000","last_undegraded":"2024-04-17T09:59:04.438105+0000","last_fullsized":"2024-04-17T09:59:04.438105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:08:42.684550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428201+0000","last_change":"2024-04-17T09:59:04.428201+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428201+0000","last_undegraded":"2024-04-17T09:59:04.428201+0000","last_fullsized":"2024-04-17T09:59:04.428201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.445242+0000","last_change":"2024-04-17T09:59:04.443547+0000","last_active":"2024-04-17T09:59:04.445242+0000","last_peered":"2024-04-17T09:59:04.445242+0000","last_clean":"2024-04-17T09:59:04.445242+0000","last_became_active":"2024-04-17T09:59:04.443090+0000","last_became_peered":"2024-04-17T09:59:04.443090+0000","last_unstale":"2024-04-17T09:59:04.445242+0000","last_undegraded":"2024-04-17T09:59:04.445242+0000","last_fullsized":"2024-04-17T09:59:04.445242+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:36:05.857724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.440390+0000","last_change":"2024-04-17T09:59:04.440390+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.440390+0000","last_undegraded":"2024-04-17T09:59:04.440390+0000","last_fullsized":"2024-04-17T09:59:04.440390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.444139+0000","last_change":"2024-04-17T09:59:04.438537+0000","last_active":"2024-04-17T09:59:04.444139+0000","last_peered":"2024-04-17T09:59:04.444139+0000","last_clean":"2024-04-17T09:59:04.444139+0000","last_became_active":"2024-04-17T09:59:04.438006+0000","last_became_peered":"2024-04-17T09:59:04.438006+0000","last_unstale":"2024-04-17T09:59:04.444139+0000","last_undegraded":"2024-04-17T09:59:04.444139+0000","last_fullsized":"2024-04-17T09:59:04.444139+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:15:09.312202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455087+0000","last_change":"2024-04-17T09:59:05.439118+0000","last_active":"2024-04-17T09:59:06.455087+0000","last_peered":"2024-04-17T09:59:06.455087+0000","last_clean":"2024-04-17T09:59:06.455087+0000","last_became_active":"2024-04-17T09:59:05.438852+0000","last_became_peered":"2024-04-17T09:59:05.438852+0000","last_unstale":"2024-04-17T09:59:06.455087+0000","last_undegraded":"2024-04-17T09:59:06.455087+0000","last_fullsized":"2024-04-17T09:59:06.455087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:13:00.757214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.444169+0000","last_change":"2024-04-17T09:59:04.438033+0000","last_active":"2024-04-17T09:59:04.444169+0000","last_peered":"2024-04-17T09:59:04.444169+0000","last_clean":"2024-04-17T09:59:04.444169+0000","last_became_active":"2024-04-17T09:59:04.437763+0000","last_became_peered":"2024-04-17T09:59:04.437763+0000","last_unstale":"2024-04-17T09:59:04.444169+0000","last_undegraded":"2024-04-17T09:59:04.444169+0000","last_fullsized":"2024-04-17T09:59:04.444169+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:03:17.824422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427265+0000","last_change":"2024-04-17T09:59:05.447894+0000","last_active":"2024-04-17T09:59:06.427265+0000","last_peered":"2024-04-17T09:59:06.427265+0000","last_clean":"2024-04-17T09:59:06.427265+0000","last_became_active":"2024-04-17T09:59:05.447595+0000","last_became_peered":"2024-04-17T09:59:05.447595+0000","last_unstale":"2024-04-17T09:59:06.427265+0000","last_undegraded":"2024-04-17T09:59:06.427265+0000","last_fullsized":"2024-04-17T09:59:06.427265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:11.377027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.451646+0000","last_change":"2024-04-17T09:59:04.448134+0000","last_active":"2024-04-17T09:59:04.451646+0000","last_peered":"2024-04-17T09:59:04.451646+0000","last_clean":"2024-04-17T09:59:04.451646+0000","last_became_active":"2024-04-17T09:59:04.447872+0000","last_became_peered":"2024-04-17T09:59:04.447872+0000","last_unstale":"2024-04-17T09:59:04.451646+0000","last_undegraded":"2024-04-17T09:59:04.451646+0000","last_fullsized":"2024-04-17T09:59:04.451646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:22:33.540833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455345+0000","last_change":"2024-04-17T09:59:05.439384+0000","last_active":"2024-04-17T09:59:06.455345+0000","last_peered":"2024-04-17T09:59:06.455345+0000","last_clean":"2024-04-17T09:59:06.455345+0000","last_became_active":"2024-04-17T09:59:05.439091+0000","last_became_peered":"2024-04-17T09:59:05.439091+0000","last_unstale":"2024-04-17T09:59:06.455345+0000","last_undegraded":"2024-04-17T09:59:06.455345+0000","last_fullsized":"2024-04-17T09:59:06.455345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:22:13.672459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.443120+0000","last_change":"2024-04-17T09:59:04.440489+0000","last_active":"2024-04-17T09:59:04.443120+0000","last_peered":"2024-04-17T09:59:04.443120+0000","last_clean":"2024-04-17T09:59:04.443120+0000","last_became_active":"2024-04-17T09:59:04.437665+0000","last_became_peered":"2024-04-17T09:59:04.437665+0000","last_unstale":"2024-04-17T09:59:04.443120+0000","last_undegraded":"2024-04-17T09:59:04.443120+0000","last_fullsized":"2024-04-17T09:59:04.443120+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:54:03.821994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433083+0000","last_change":"2024-04-17T09:59:04.433083+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433083+0000","last_undegraded":"2024-04-17T09:59:04.433083+0000","last_fullsized":"2024-04-17T09:59:04.433083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.446156+0000","last_change":"2024-04-17T09:59:04.441625+0000","last_active":"2024-04-17T09:59:04.446156+0000","last_peered":"2024-04-17T09:59:04.446156+0000","last_clean":"2024-04-17T09:59:04.446156+0000","last_became_active":"2024-04-17T09:59:04.441012+0000","last_became_peered":"2024-04-17T09:59:04.441012+0000","last_unstale":"2024-04-17T09:59:04.446156+0000","last_undegraded":"2024-04-17T09:59:04.446156+0000","last_fullsized":"2024-04-17T09:59:04.446156+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:31:12.043046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.426432+0000","last_change":"2024-04-17T09:59:04.426432+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.426432+0000","last_undegraded":"2024-04-17T09:59:04.426432+0000","last_fullsized":"2024-04-17T09:59:04.426432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.451592+0000","last_change":"2024-04-17T09:59:04.446837+0000","last_active":"2024-04-17T09:59:04.451592+0000","last_peered":"2024-04-17T09:59:04.451592+0000","last_clean":"2024-04-17T09:59:04.451592+0000","last_became_active":"2024-04-17T09:59:04.446469+0000","last_became_peered":"2024-04-17T09:59:04.446469+0000","last_unstale":"2024-04-17T09:59:04.451592+0000","last_undegraded":"2024-04-17T09:59:04.451592+0000","last_fullsized":"2024-04-17T09:59:04.451592+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:49.287601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.425532+0000","last_change":"2024-04-17T09:59:04.425532+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.425532+0000","last_undegraded":"2024-04-17T09:59:04.425532+0000","last_fullsized":"2024-04-17T09:59:04.425532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"81'8","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455171+0000","last_change":"2024-04-17T09:59:04.438156+0000","last_active":"2024-04-17T09:59:06.455171+0000","last_peered":"2024-04-17T09:59:06.455171+0000","last_clean":"2024-04-17T09:59:06.455171+0000","last_became_active":"2024-04-17T09:59:04.437821+0000","last_became_peered":"2024-04-17T09:59:04.437821+0000","last_unstale":"2024-04-17T09:59:06.455171+0000","last_undegraded":"2024-04-17T09:59:06.455171+0000","last_fullsized":"2024-04-17T09:59:06.455171+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:54.250176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.432091+0000","last_change":"2024-04-17T09:59:04.432091+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.432091+0000","last_undegraded":"2024-04-17T09:59:04.432091+0000","last_fullsized":"2024-04-17T09:59:04.432091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.448596+0000","last_change":"2024-04-17T09:59:04.447265+0000","last_active":"2024-04-17T09:59:04.448596+0000","last_peered":"2024-04-17T09:59:04.448596+0000","last_clean":"2024-04-17T09:59:04.448596+0000","last_became_active":"2024-04-17T09:59:04.446847+0000","last_became_peered":"2024-04-17T09:59:04.446847+0000","last_unstale":"2024-04-17T09:59:04.448596+0000","last_undegraded":"2024-04-17T09:59:04.448596+0000","last_fullsized":"2024-04-17T09:59:04.448596+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:49.287601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429393+0000","last_change":"2024-04-17T09:59:04.429393+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429393+0000","last_undegraded":"2024-04-17T09:59:04.429393+0000","last_fullsized":"2024-04-17T09:59:04.429393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455137+0000","last_change":"2024-04-17T09:59:04.443540+0000","last_active":"2024-04-17T09:59:06.455137+0000","last_peered":"2024-04-17T09:59:06.455137+0000","last_clean":"2024-04-17T09:59:06.455137+0000","last_became_active":"2024-04-17T09:59:04.443277+0000","last_became_peered":"2024-04-17T09:59:04.443277+0000","last_unstale":"2024-04-17T09:59:06.455137+0000","last_undegraded":"2024-04-17T09:59:06.455137+0000","last_fullsized":"2024-04-17T09:59:06.455137+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:10.371001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427822+0000","last_change":"2024-04-17T09:59:05.447239+0000","last_active":"2024-04-17T09:59:06.427822+0000","last_peered":"2024-04-17T09:59:06.427822+0000","last_clean":"2024-04-17T09:59:06.427822+0000","last_became_active":"2024-04-17T09:59:05.446970+0000","last_became_peered":"2024-04-17T09:59:05.446970+0000","last_unstale":"2024-04-17T09:59:06.427822+0000","last_undegraded":"2024-04-17T09:59:06.427822+0000","last_fullsized":"2024-04-17T09:59:06.427822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:02:07.703202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455269+0000","last_change":"2024-04-17T09:59:04.445958+0000","last_active":"2024-04-17T09:59:06.455269+0000","last_peered":"2024-04-17T09:59:06.455269+0000","last_clean":"2024-04-17T09:59:06.455269+0000","last_became_active":"2024-04-17T09:59:04.445632+0000","last_became_peered":"2024-04-17T09:59:04.445632+0000","last_unstale":"2024-04-17T09:59:06.455269+0000","last_undegraded":"2024-04-17T09:59:06.455269+0000","last_fullsized":"2024-04-17T09:59:06.455269+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:57.288434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.432936+0000","last_change":"2024-04-17T09:59:04.432936+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.432936+0000","last_undegraded":"2024-04-17T09:59:04.432936+0000","last_fullsized":"2024-04-17T09:59:04.432936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.448600+0000","last_change":"2024-04-17T09:59:04.445248+0000","last_active":"2024-04-17T09:59:04.448600+0000","last_peered":"2024-04-17T09:59:04.448600+0000","last_clean":"2024-04-17T09:59:04.448600+0000","last_became_active":"2024-04-17T09:59:04.444809+0000","last_became_peered":"2024-04-17T09:59:04.444809+0000","last_unstale":"2024-04-17T09:59:04.448600+0000","last_undegraded":"2024-04-17T09:59:04.448600+0000","last_fullsized":"2024-04-17T09:59:04.448600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:33:59.203901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.437546+0000","last_change":"2024-04-17T09:59:04.437546+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.437546+0000","last_undegraded":"2024-04-17T09:59:04.437546+0000","last_fullsized":"2024-04-17T09:59:04.437546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.451113+0000","last_change":"2024-04-17T09:59:04.448116+0000","last_active":"2024-04-17T09:59:04.451113+0000","last_peered":"2024-04-17T09:59:04.451113+0000","last_clean":"2024-04-17T09:59:04.451113+0000","last_became_active":"2024-04-17T09:59:04.446755+0000","last_became_peered":"2024-04-17T09:59:04.446755+0000","last_unstale":"2024-04-17T09:59:04.451113+0000","last_undegraded":"2024-04-17T09:59:04.451113+0000","last_fullsized":"2024-04-17T09:59:04.451113+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:04:29.400922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427614+0000","last_change":"2024-04-17T09:59:04.427614+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427614+0000","last_undegraded":"2024-04-17T09:59:04.427614+0000","last_fullsized":"2024-04-17T09:59:04.427614+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.451071+0000","last_change":"2024-04-17T09:59:04.448303+0000","last_active":"2024-04-17T09:59:04.451071+0000","last_peered":"2024-04-17T09:59:04.451071+0000","last_clean":"2024-04-17T09:59:04.451071+0000","last_became_active":"2024-04-17T09:59:04.446032+0000","last_became_peered":"2024-04-17T09:59:04.446032+0000","last_unstale":"2024-04-17T09:59:04.451071+0000","last_undegraded":"2024-04-17T09:59:04.451071+0000","last_fullsized":"2024-04-17T09:59:04.451071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:02:55.706845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.440407+0000","last_change":"2024-04-17T09:59:04.440407+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.440407+0000","last_undegraded":"2024-04-17T09:59:04.440407+0000","last_fullsized":"2024-04-17T09:59:04.440407+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.448496+0000","last_change":"2024-04-17T09:59:04.445983+0000","last_active":"2024-04-17T09:59:04.448496+0000","last_peered":"2024-04-17T09:59:04.448496+0000","last_clean":"2024-04-17T09:59:04.448496+0000","last_became_active":"2024-04-17T09:59:04.445671+0000","last_became_peered":"2024-04-17T09:59:04.445671+0000","last_unstale":"2024-04-17T09:59:04.448496+0000","last_undegraded":"2024-04-17T09:59:04.448496+0000","last_fullsized":"2024-04-17T09:59:04.448496+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:36.060764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.428075+0000","last_change":"2024-04-17T09:59:05.449697+0000","last_active":"2024-04-17T09:59:06.428075+0000","last_peered":"2024-04-17T09:59:06.428075+0000","last_clean":"2024-04-17T09:59:06.428075+0000","last_became_active":"2024-04-17T09:59:05.449390+0000","last_became_peered":"2024-04-17T09:59:05.449390+0000","last_unstale":"2024-04-17T09:59:06.428075+0000","last_undegraded":"2024-04-17T09:59:06.428075+0000","last_fullsized":"2024-04-17T09:59:06.428075+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:11:22.087912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.447272+0000","last_change":"2024-04-17T09:59:04.442441+0000","last_active":"2024-04-17T09:59:04.447272+0000","last_peered":"2024-04-17T09:59:04.447272+0000","last_clean":"2024-04-17T09:59:04.447272+0000","last_became_active":"2024-04-17T09:59:04.442051+0000","last_became_peered":"2024-04-17T09:59:04.442051+0000","last_unstale":"2024-04-17T09:59:04.447272+0000","last_undegraded":"2024-04-17T09:59:04.447272+0000","last_fullsized":"2024-04-17T09:59:04.447272+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:16:08.377113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433918+0000","last_change":"2024-04-17T09:59:04.433918+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433918+0000","last_undegraded":"2024-04-17T09:59:04.433918+0000","last_fullsized":"2024-04-17T09:59:04.433918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.446270+0000","last_change":"2024-04-17T09:59:04.441305+0000","last_active":"2024-04-17T09:59:04.446270+0000","last_peered":"2024-04-17T09:59:04.446270+0000","last_clean":"2024-04-17T09:59:04.446270+0000","last_became_active":"2024-04-17T09:59:04.441044+0000","last_became_peered":"2024-04-17T09:59:04.441044+0000","last_unstale":"2024-04-17T09:59:04.446270+0000","last_undegraded":"2024-04-17T09:59:04.446270+0000","last_fullsized":"2024-04-17T09:59:04.446270+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:06:39.663043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428308+0000","last_change":"2024-04-17T09:59:04.428308+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428308+0000","last_undegraded":"2024-04-17T09:59:04.428308+0000","last_fullsized":"2024-04-17T09:59:04.428308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.445184+0000","last_change":"2024-04-17T09:59:04.442494+0000","last_active":"2024-04-17T09:59:04.445184+0000","last_peered":"2024-04-17T09:59:04.445184+0000","last_clean":"2024-04-17T09:59:04.445184+0000","last_became_active":"2024-04-17T09:59:04.442254+0000","last_became_peered":"2024-04-17T09:59:04.442254+0000","last_unstale":"2024-04-17T09:59:04.445184+0000","last_undegraded":"2024-04-17T09:59:04.445184+0000","last_fullsized":"2024-04-17T09:59:04.445184+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:09:27.952101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427862+0000","last_change":"2024-04-17T09:59:05.438162+0000","last_active":"2024-04-17T09:59:06.427862+0000","last_peered":"2024-04-17T09:59:06.427862+0000","last_clean":"2024-04-17T09:59:06.427862+0000","last_became_active":"2024-04-17T09:59:05.437880+0000","last_became_peered":"2024-04-17T09:59:05.437880+0000","last_unstale":"2024-04-17T09:59:06.427862+0000","last_undegraded":"2024-04-17T09:59:06.427862+0000","last_fullsized":"2024-04-17T09:59:06.427862+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:58.294460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.445858+0000","last_change":"2024-04-17T09:59:04.444713+0000","last_active":"2024-04-17T09:59:04.445858+0000","last_peered":"2024-04-17T09:59:04.445858+0000","last_clean":"2024-04-17T09:59:04.445858+0000","last_became_active":"2024-04-17T09:59:04.444438+0000","last_became_peered":"2024-04-17T09:59:04.444438+0000","last_unstale":"2024-04-17T09:59:04.445858+0000","last_undegraded":"2024-04-17T09:59:04.445858+0000","last_fullsized":"2024-04-17T09:59:04.445858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:29:09.325067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.437197+0000","last_change":"2024-04-17T09:59:04.437197+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.437197+0000","last_undegraded":"2024-04-17T09:59:04.437197+0000","last_fullsized":"2024-04-17T09:59:04.437197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455124+0000","last_change":"2024-04-17T09:59:04.438627+0000","last_active":"2024-04-17T09:59:06.455124+0000","last_peered":"2024-04-17T09:59:06.455124+0000","last_clean":"2024-04-17T09:59:06.455124+0000","last_became_active":"2024-04-17T09:59:04.438100+0000","last_became_peered":"2024-04-17T09:59:04.438100+0000","last_unstale":"2024-04-17T09:59:06.455124+0000","last_undegraded":"2024-04-17T09:59:06.455124+0000","last_fullsized":"2024-04-17T09:59:06.455124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:53:55.006094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455088+0000","last_change":"2024-04-17T09:59:05.447417+0000","last_active":"2024-04-17T09:59:06.455088+0000","last_peered":"2024-04-17T09:59:06.455088+0000","last_clean":"2024-04-17T09:59:06.455088+0000","last_became_active":"2024-04-17T09:59:05.447176+0000","last_became_peered":"2024-04-17T09:59:05.447176+0000","last_unstale":"2024-04-17T09:59:06.455088+0000","last_undegraded":"2024-04-17T09:59:06.455088+0000","last_fullsized":"2024-04-17T09:59:06.455088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:18:04.685528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426694+0000","last_change":"2024-04-17T09:59:04.440315+0000","last_active":"2024-04-17T09:59:06.426694+0000","last_peered":"2024-04-17T09:59:06.426694+0000","last_clean":"2024-04-17T09:59:06.426694+0000","last_became_active":"2024-04-17T09:59:04.439614+0000","last_became_peered":"2024-04-17T09:59:04.439614+0000","last_unstale":"2024-04-17T09:59:06.426694+0000","last_undegraded":"2024-04-17T09:59:06.426694+0000","last_fullsized":"2024-04-17T09:59:06.426694+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:02.341478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.432875+0000","last_change":"2024-04-17T09:59:04.432875+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.432875+0000","last_undegraded":"2024-04-17T09:59:04.432875+0000","last_fullsized":"2024-04-17T09:59:04.432875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.445028+0000","last_change":"2024-04-17T09:59:04.442633+0000","last_active":"2024-04-17T09:59:04.445028+0000","last_peered":"2024-04-17T09:59:04.445028+0000","last_clean":"2024-04-17T09:59:04.445028+0000","last_became_active":"2024-04-17T09:59:04.442347+0000","last_became_peered":"2024-04-17T09:59:04.442347+0000","last_unstale":"2024-04-17T09:59:04.445028+0000","last_undegraded":"2024-04-17T09:59:04.445028+0000","last_fullsized":"2024-04-17T09:59:04.445028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:38:59.533078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427513+0000","last_change":"2024-04-17T09:59:04.427513+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427513+0000","last_undegraded":"2024-04-17T09:59:04.427513+0000","last_fullsized":"2024-04-17T09:59:04.427513+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426835+0000","last_change":"2024-04-17T09:59:04.440613+0000","last_active":"2024-04-17T09:59:06.426835+0000","last_peered":"2024-04-17T09:59:06.426835+0000","last_clean":"2024-04-17T09:59:06.426835+0000","last_became_active":"2024-04-17T09:59:04.440063+0000","last_became_peered":"2024-04-17T09:59:04.440063+0000","last_unstale":"2024-04-17T09:59:06.426835+0000","last_undegraded":"2024-04-17T09:59:06.426835+0000","last_fullsized":"2024-04-17T09:59:06.426835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:53:08.846256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.425466+0000","last_change":"2024-04-17T09:59:04.425466+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.425466+0000","last_undegraded":"2024-04-17T09:59:04.425466+0000","last_fullsized":"2024-04-17T09:59:04.425466+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.446679+0000","last_change":"2024-04-17T09:59:04.445356+0000","last_active":"2024-04-17T09:59:04.446679+0000","last_peered":"2024-04-17T09:59:04.446679+0000","last_clean":"2024-04-17T09:59:04.446679+0000","last_became_active":"2024-04-17T09:59:04.445010+0000","last_became_peered":"2024-04-17T09:59:04.445010+0000","last_unstale":"2024-04-17T09:59:04.446679+0000","last_undegraded":"2024-04-17T09:59:04.446679+0000","last_fullsized":"2024-04-17T09:59:04.446679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:32:37.586889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":280,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.434933+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T09:59:04.434933+0000","last_peered":"2024-04-17T09:59:04.434933+0000","last_clean":"2024-04-17T09:59:04.434933+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T09:59:04.434933+0000","last_undegraded":"2024-04-17T09:59:04.434933+0000","last_fullsized":"2024-04-17T09:59:04.434933+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509516,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1568768,"data_stored":1002078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":411710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509516,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":962560,"data_stored":405435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509516,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":411710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":411710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509516,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":411710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542221,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":424785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542221,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36900,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334940,"statfs":{"total":96636764160,"available":96598978560,"internally_reserved":0,"allocated":1597440,"data_stored":1015187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T09:59:09.565 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T09:59:09.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.746+0000 7f2e38000640 1 Processor -- start 2024-04-17T09:59:09.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.746+0000 7f2e38000640 1 -- start start 2024-04-17T09:59:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.750+0000 7f2e38000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e30072500 0x7f2e300cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.750+0000 7f2e38000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e300cdac0 con 0x7f2e30145f30 2024-04-17T09:59:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.750+0000 7f2e38000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e3015cdf0 con 0x7f2e3015d8d0 2024-04-17T09:59:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.750+0000 7f2e38000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2e3006f400 con 0x7f2e30072500 2024-04-17T09:59:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.750+0000 7f2e35600640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2e30145f30 0x7f2e30071df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38070/0 (socket says 172.21.15.32:38070) 2024-04-17T09:59:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.750+0000 7f2e35600640 1 -- 172.21.15.32:0/4067933611 learned_addr learned my addr 172.21.15.32:0/4067933611 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.750+0000 7f2e34c00640 1 --2- 172.21.15.32:0/4067933611 >> v2:172.21.15.32:3300/0 conn(0x7f2e30072500 0x7f2e300cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.750+0000 7f2e34c00640 1 -- 172.21.15.32:0/4067933611 >> v1:172.21.15.144:6789/0 conn(0x7f2e3015d8d0 legacy=0x7f2e301601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.750+0000 7f2e34c00640 1 -- 172.21.15.32:0/4067933611 >> v1:172.21.15.32:6789/0 conn(0x7f2e30145f30 legacy=0x7f2e30071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.750+0000 7f2e34c00640 1 -- 172.21.15.32:0/4067933611 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3015d0d0 con 0x7f2e30072500 2024-04-17T09:59:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.750+0000 7f2e2fe00640 1 -- 172.21.15.32:0/4067933611 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2439097070 0 0) 0x7f2e300cdac0 con 0x7f2e30145f30 2024-04-17T09:59:09.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.754+0000 7f2e34c00640 1 --2- 172.21.15.32:0/4067933611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e30072500 0x7f2e300cce70 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0097c0 tx=0x7f2e1c01ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156ebfdbde31613b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:09.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.754+0000 7f2e2fe00640 1 -- 172.21.15.32:0/4067933611 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0b5070 con 0x7f2e30072500 2024-04-17T09:59:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.754+0000 7f2e2fe00640 1 -- 172.21.15.32:0/4067933611 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e1c0a8950 con 0x7f2e30072500 2024-04-17T09:59:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.754+0000 7f2e2fe00640 1 -- 172.21.15.32:0/4067933611 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0a8d20 con 0x7f2e30072500 2024-04-17T09:59:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.754+0000 7f2e38000640 1 -- 172.21.15.32:0/4067933611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e30072500 msgr2=0x7f2e300cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:09.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.754+0000 7f2e38000640 1 --2- 172.21.15.32:0/4067933611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e30072500 0x7f2e300cce70 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0097c0 tx=0x7f2e1c01ec50 comp rx=0 tx=0).stop 2024-04-17T09:59:09.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.754+0000 7f2e38000640 1 -- 172.21.15.32:0/4067933611 shutdown_connections 2024-04-17T09:59:09.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.754+0000 7f2e38000640 1 --2- 172.21.15.32:0/4067933611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e30072500 0x7f2e300cce70 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.754+0000 7f2e38000640 1 -- 172.21.15.32:0/4067933611 >> 172.21.15.32:0/4067933611 conn(0x7f2e3006ba30 msgr2=0x7f2e3006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:09.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.754+0000 7f2e38000640 1 -- 172.21.15.32:0/4067933611 shutdown_connections 2024-04-17T09:59:09.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.754+0000 7f2e38000640 1 -- 172.21.15.32:0/4067933611 wait complete. 2024-04-17T09:59:09.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.754+0000 7f2e38000640 1 Processor -- start 2024-04-17T09:59:09.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.754+0000 7f2e38000640 1 -- start start 2024-04-17T09:59:09.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e38000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e30072500 0x7f2e301f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:09.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e38000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e30145f30 0x7f2e301f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:09.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e38000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3015d8d0 0x7f2e30144390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:09.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e38000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2e300cd800 con 0x7f2e3015d8d0 2024-04-17T09:59:09.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e38000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2e300cd680 con 0x7f2e30072500 2024-04-17T09:59:09.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e38000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2e30146310 con 0x7f2e30145f30 2024-04-17T09:59:09.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e36000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3015d8d0 0x7f2e30144390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:09.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e36000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3015d8d0 0x7f2e30144390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34676/0 (socket says 172.21.15.32:34676) 2024-04-17T09:59:09.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e36000640 1 -- 172.21.15.32:0/3737472162 learned_addr learned my addr 172.21.15.32:0/3737472162 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:09.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e34c00640 1 --2- 172.21.15.32:0/3737472162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e30145f30 0x7f2e301f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:09.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e35600640 1 --2- 172.21.15.32:0/3737472162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e30072500 0x7f2e301f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e36000640 1 -- 172.21.15.32:0/3737472162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e30145f30 msgr2=0x7f2e301f4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e36000640 1 --2- 172.21.15.32:0/3737472162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e30145f30 0x7f2e301f4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e36000640 1 -- 172.21.15.32:0/3737472162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e30072500 msgr2=0x7f2e301f1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e36000640 1 --2- 172.21.15.32:0/3737472162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e30072500 0x7f2e301f1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e36000640 1 -- 172.21.15.32:0/3737472162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e301415a0 con 0x7f2e3015d8d0 2024-04-17T09:59:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e35600640 1 --2- 172.21.15.32:0/3737472162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e30072500 0x7f2e301f1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:59:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e36000640 1 --2- 172.21.15.32:0/3737472162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3015d8d0 0x7f2e30144390 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800b970 tx=0x7f2e2800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e2d600640 1 -- 172.21.15.32:0/3737472162 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28019070 con 0x7f2e3015d8d0 2024-04-17T09:59:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e2d600640 1 -- 172.21.15.32:0/3737472162 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e2800cb40 con 0x7f2e3015d8d0 2024-04-17T09:59:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e38000640 1 -- 172.21.15.32:0/3737472162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e30141840 con 0x7f2e3015d8d0 2024-04-17T09:59:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e2d600640 1 -- 172.21.15.32:0/3737472162 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28014440 con 0x7f2e3015d8d0 2024-04-17T09:59:09.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e38000640 1 -- 172.21.15.32:0/3737472162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e30141e00 con 0x7f2e3015d8d0 2024-04-17T09:59:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e38000640 1 -- 172.21.15.32:0/3737472162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e30068a90 con 0x7f2e3015d8d0 2024-04-17T09:59:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.758+0000 7f2e2d600640 1 -- 172.21.15.32:0/3737472162 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2e2801c070 con 0x7f2e3015d8d0 2024-04-17T09:59:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.762+0000 7f2e2d600640 1 --2- 172.21.15.32:0/3737472162 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e10068380 0x7f2e1006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.762+0000 7f2e2d600640 1 -- 172.21.15.32:0/3737472162 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f2e28086620 con 0x7f2e3015d8d0 2024-04-17T09:59:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.762+0000 7f2e35600640 1 --2- 172.21.15.32:0/3737472162 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e10068380 0x7f2e1006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:09.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.762+0000 7f2e35600640 1 --2- 172.21.15.32:0/3737472162 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e10068380 0x7f2e1006a840 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2e24000f70 tx=0x7f2e24009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.766+0000 7f2e2d600640 1 -- 172.21.15.32:0/3737472162 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2e28056bd0 con 0x7f2e3015d8d0 2024-04-17T09:59:09.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.926+0000 7f2e38000640 1 -- 172.21.15.32:0/3737472162 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2e30144170 con 0x7f2e10068380 2024-04-17T09:59:09.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.934+0000 7f2e2d600640 1 -- 172.21.15.32:0/3737472162 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248008 (secure 0 0 0) 0x7f2e30144170 con 0x7f2e10068380 2024-04-17T09:59:09.934 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:59:09.935 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T09:59:09.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.938+0000 7f2e38000640 1 -- 172.21.15.32:0/3737472162 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e10068380 msgr2=0x7f2e1006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:09.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.938+0000 7f2e38000640 1 --2- 172.21.15.32:0/3737472162 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e10068380 0x7f2e1006a840 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2e24000f70 tx=0x7f2e24009040 comp rx=0 tx=0).stop 2024-04-17T09:59:09.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.938+0000 7f2e38000640 1 -- 172.21.15.32:0/3737472162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3015d8d0 msgr2=0x7f2e30144390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:09.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.938+0000 7f2e38000640 1 --2- 172.21.15.32:0/3737472162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3015d8d0 0x7f2e30144390 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800b970 tx=0x7f2e2800be30 comp rx=0 tx=0).stop 2024-04-17T09:59:09.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.938+0000 7f2e38000640 1 -- 172.21.15.32:0/3737472162 shutdown_connections 2024-04-17T09:59:09.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.938+0000 7f2e38000640 1 --2- 172.21.15.32:0/3737472162 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e10068380 0x7f2e1006a840 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.938+0000 7f2e38000640 1 --2- 172.21.15.32:0/3737472162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e30145f30 0x7f2e301f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.938+0000 7f2e38000640 1 --2- 172.21.15.32:0/3737472162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e3015d8d0 0x7f2e30144390 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.938+0000 7f2e38000640 1 --2- 172.21.15.32:0/3737472162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e30072500 0x7f2e301f1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:09.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.938+0000 7f2e38000640 1 -- 172.21.15.32:0/3737472162 >> 172.21.15.32:0/3737472162 conn(0x7f2e3006ba30 msgr2=0x7f2e3015dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:09.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.938+0000 7f2e38000640 1 -- 172.21.15.32:0/3737472162 shutdown_connections 2024-04-17T09:59:09.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:09.938+0000 7f2e38000640 1 -- 172.21.15.32:0/3737472162 wait complete. 2024-04-17T09:59:09.954 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-04-17T09:59:09.365238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":180,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258280,"kb_used_data":8816,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716440,"statfs":{"total":773094113280,"available":772829634560,"internally_reserved":0,"allocated":9027584,"data_stored":4494325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.953388"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.430338+0000","last_change":"2024-04-17T09:59:04.430338+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.430338+0000","last_undegraded":"2024-04-17T09:59:04.430338+0000","last_fullsized":"2024-04-17T09:59:04.430338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429509+0000","last_change":"2024-04-17T09:59:04.429509+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429509+0000","last_undegraded":"2024-04-17T09:59:04.429509+0000","last_fullsized":"2024-04-17T09:59:04.429509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433330+0000","last_change":"2024-04-17T09:59:04.433330+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433330+0000","last_undegraded":"2024-04-17T09:59:04.433330+0000","last_fullsized":"2024-04-17T09:59:04.433330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429769+0000","last_change":"2024-04-17T09:59:04.429769+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429769+0000","last_undegraded":"2024-04-17T09:59:04.429769+0000","last_fullsized":"2024-04-17T09:59:04.429769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427070+0000","last_change":"2024-04-17T09:59:05.435295+0000","last_active":"2024-04-17T09:59:06.427070+0000","last_peered":"2024-04-17T09:59:06.427070+0000","last_clean":"2024-04-17T09:59:06.427070+0000","last_became_active":"2024-04-17T09:59:05.434948+0000","last_became_peered":"2024-04-17T09:59:05.434948+0000","last_unstale":"2024-04-17T09:59:06.427070+0000","last_undegraded":"2024-04-17T09:59:06.427070+0000","last_fullsized":"2024-04-17T09:59:06.427070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:14.347809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.438674+0000","last_change":"2024-04-17T09:59:04.438674+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.438674+0000","last_undegraded":"2024-04-17T09:59:04.438674+0000","last_fullsized":"2024-04-17T09:59:04.438674+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429372+0000","last_change":"2024-04-17T09:59:04.429372+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429372+0000","last_undegraded":"2024-04-17T09:59:04.429372+0000","last_fullsized":"2024-04-17T09:59:04.429372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.437249+0000","last_change":"2024-04-17T09:59:04.437249+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.437249+0000","last_undegraded":"2024-04-17T09:59:04.437249+0000","last_fullsized":"2024-04-17T09:59:04.437249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.440846+0000","last_change":"2024-04-17T09:59:04.440846+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.440846+0000","last_undegraded":"2024-04-17T09:59:04.440846+0000","last_fullsized":"2024-04-17T09:59:04.440846+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427452+0000","last_change":"2024-04-17T09:59:04.427452+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427452+0000","last_undegraded":"2024-04-17T09:59:04.427452+0000","last_fullsized":"2024-04-17T09:59:04.427452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427477+0000","last_change":"2024-04-17T09:59:04.427477+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427477+0000","last_undegraded":"2024-04-17T09:59:04.427477+0000","last_fullsized":"2024-04-17T09:59:04.427477+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427835+0000","last_change":"2024-04-17T09:59:04.427835+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427835+0000","last_undegraded":"2024-04-17T09:59:04.427835+0000","last_fullsized":"2024-04-17T09:59:04.427835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433600+0000","last_change":"2024-04-17T09:59:04.433600+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433600+0000","last_undegraded":"2024-04-17T09:59:04.433600+0000","last_fullsized":"2024-04-17T09:59:04.433600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.435489+0000","last_change":"2024-04-17T09:59:04.435489+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.435489+0000","last_undegraded":"2024-04-17T09:59:04.435489+0000","last_fullsized":"2024-04-17T09:59:04.435489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427347+0000","last_change":"2024-04-17T09:59:04.427347+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427347+0000","last_undegraded":"2024-04-17T09:59:04.427347+0000","last_fullsized":"2024-04-17T09:59:04.427347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426220+0000","last_change":"2024-04-17T09:59:05.445968+0000","last_active":"2024-04-17T09:59:06.426220+0000","last_peered":"2024-04-17T09:59:06.426220+0000","last_clean":"2024-04-17T09:59:06.426220+0000","last_became_active":"2024-04-17T09:59:05.445695+0000","last_became_peered":"2024-04-17T09:59:05.445695+0000","last_unstale":"2024-04-17T09:59:06.426220+0000","last_undegraded":"2024-04-17T09:59:06.426220+0000","last_fullsized":"2024-04-17T09:59:06.426220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:27:15.977393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.428029+0000","last_change":"2024-04-17T09:59:05.447691+0000","last_active":"2024-04-17T09:59:06.428029+0000","last_peered":"2024-04-17T09:59:06.428029+0000","last_clean":"2024-04-17T09:59:06.428029+0000","last_became_active":"2024-04-17T09:59:05.447403+0000","last_became_peered":"2024-04-17T09:59:05.447403+0000","last_unstale":"2024-04-17T09:59:06.428029+0000","last_undegraded":"2024-04-17T09:59:06.428029+0000","last_fullsized":"2024-04-17T09:59:06.428029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:15.267856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.432446+0000","last_change":"2024-04-17T09:59:04.432446+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.432446+0000","last_undegraded":"2024-04-17T09:59:04.432446+0000","last_fullsized":"2024-04-17T09:59:04.432446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455142+0000","last_change":"2024-04-17T09:59:05.442262+0000","last_active":"2024-04-17T09:59:06.455142+0000","last_peered":"2024-04-17T09:59:06.455142+0000","last_clean":"2024-04-17T09:59:06.455142+0000","last_became_active":"2024-04-17T09:59:05.441944+0000","last_became_peered":"2024-04-17T09:59:05.441944+0000","last_unstale":"2024-04-17T09:59:06.455142+0000","last_undegraded":"2024-04-17T09:59:06.455142+0000","last_fullsized":"2024-04-17T09:59:06.455142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:55:27.872332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429260+0000","last_change":"2024-04-17T09:59:04.429260+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429260+0000","last_undegraded":"2024-04-17T09:59:04.429260+0000","last_fullsized":"2024-04-17T09:59:04.429260+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429463+0000","last_change":"2024-04-17T09:59:04.429463+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429463+0000","last_undegraded":"2024-04-17T09:59:04.429463+0000","last_fullsized":"2024-04-17T09:59:04.429463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428078+0000","last_change":"2024-04-17T09:59:04.428078+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428078+0000","last_undegraded":"2024-04-17T09:59:04.428078+0000","last_fullsized":"2024-04-17T09:59:04.428078+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427009+0000","last_change":"2024-04-17T09:59:04.427009+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427009+0000","last_undegraded":"2024-04-17T09:59:04.427009+0000","last_fullsized":"2024-04-17T09:59:04.427009+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428239+0000","last_change":"2024-04-17T09:59:04.428239+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428239+0000","last_undegraded":"2024-04-17T09:59:04.428239+0000","last_fullsized":"2024-04-17T09:59:04.428239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.425424+0000","last_change":"2024-04-17T09:59:04.425424+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.425424+0000","last_undegraded":"2024-04-17T09:59:04.425424+0000","last_fullsized":"2024-04-17T09:59:04.425424+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.432662+0000","last_change":"2024-04-17T09:59:04.432662+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.432662+0000","last_undegraded":"2024-04-17T09:59:04.432662+0000","last_fullsized":"2024-04-17T09:59:04.432662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427077+0000","last_change":"2024-04-17T09:59:05.438772+0000","last_active":"2024-04-17T09:59:06.427077+0000","last_peered":"2024-04-17T09:59:06.427077+0000","last_clean":"2024-04-17T09:59:06.427077+0000","last_became_active":"2024-04-17T09:59:05.438444+0000","last_became_peered":"2024-04-17T09:59:05.438444+0000","last_unstale":"2024-04-17T09:59:06.427077+0000","last_undegraded":"2024-04-17T09:59:06.427077+0000","last_fullsized":"2024-04-17T09:59:06.427077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:53:56.012120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429291+0000","last_change":"2024-04-17T09:59:04.429291+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429291+0000","last_undegraded":"2024-04-17T09:59:04.429291+0000","last_fullsized":"2024-04-17T09:59:04.429291+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427246+0000","last_change":"2024-04-17T09:59:04.427246+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427246+0000","last_undegraded":"2024-04-17T09:59:04.427246+0000","last_fullsized":"2024-04-17T09:59:04.427246+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.426566+0000","last_change":"2024-04-17T09:59:04.426566+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.426566+0000","last_undegraded":"2024-04-17T09:59:04.426566+0000","last_fullsized":"2024-04-17T09:59:04.426566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426520+0000","last_change":"2024-04-17T09:59:05.438189+0000","last_active":"2024-04-17T09:59:06.426520+0000","last_peered":"2024-04-17T09:59:06.426520+0000","last_clean":"2024-04-17T09:59:06.426520+0000","last_became_active":"2024-04-17T09:59:05.437898+0000","last_became_peered":"2024-04-17T09:59:05.437898+0000","last_unstale":"2024-04-17T09:59:06.426520+0000","last_undegraded":"2024-04-17T09:59:06.426520+0000","last_fullsized":"2024-04-17T09:59:06.426520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:55.256201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428134+0000","last_change":"2024-04-17T09:59:04.428134+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428134+0000","last_undegraded":"2024-04-17T09:59:04.428134+0000","last_fullsized":"2024-04-17T09:59:04.428134+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.428003+0000","last_change":"2024-04-17T09:59:05.445880+0000","last_active":"2024-04-17T09:59:06.428003+0000","last_peered":"2024-04-17T09:59:06.428003+0000","last_clean":"2024-04-17T09:59:06.428003+0000","last_became_active":"2024-04-17T09:59:05.439231+0000","last_became_peered":"2024-04-17T09:59:05.439231+0000","last_unstale":"2024-04-17T09:59:06.428003+0000","last_undegraded":"2024-04-17T09:59:06.428003+0000","last_fullsized":"2024-04-17T09:59:06.428003+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:40.503102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.445224+0000","last_change":"2024-04-17T09:59:04.443338+0000","last_active":"2024-04-17T09:59:04.445224+0000","last_peered":"2024-04-17T09:59:04.445224+0000","last_clean":"2024-04-17T09:59:04.445224+0000","last_became_active":"2024-04-17T09:59:04.442697+0000","last_became_peered":"2024-04-17T09:59:04.442697+0000","last_unstale":"2024-04-17T09:59:04.445224+0000","last_undegraded":"2024-04-17T09:59:04.445224+0000","last_fullsized":"2024-04-17T09:59:04.445224+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:39:16.180270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429909+0000","last_change":"2024-04-17T09:59:04.429909+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429909+0000","last_undegraded":"2024-04-17T09:59:04.429909+0000","last_fullsized":"2024-04-17T09:59:04.429909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.428060+0000","last_change":"2024-04-17T09:59:04.439722+0000","last_active":"2024-04-17T09:59:06.428060+0000","last_peered":"2024-04-17T09:59:06.428060+0000","last_clean":"2024-04-17T09:59:06.428060+0000","last_became_active":"2024-04-17T09:59:04.439452+0000","last_became_peered":"2024-04-17T09:59:04.439452+0000","last_unstale":"2024-04-17T09:59:06.428060+0000","last_undegraded":"2024-04-17T09:59:06.428060+0000","last_fullsized":"2024-04-17T09:59:06.428060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:49:05.349813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428740+0000","last_change":"2024-04-17T09:59:04.428740+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428740+0000","last_undegraded":"2024-04-17T09:59:04.428740+0000","last_fullsized":"2024-04-17T09:59:04.428740+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455165+0000","last_change":"2024-04-17T09:59:04.444375+0000","last_active":"2024-04-17T09:59:06.455165+0000","last_peered":"2024-04-17T09:59:06.455165+0000","last_clean":"2024-04-17T09:59:06.455165+0000","last_became_active":"2024-04-17T09:59:04.444125+0000","last_became_peered":"2024-04-17T09:59:04.444125+0000","last_unstale":"2024-04-17T09:59:06.455165+0000","last_undegraded":"2024-04-17T09:59:06.455165+0000","last_fullsized":"2024-04-17T09:59:06.455165+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:07.141412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433823+0000","last_change":"2024-04-17T09:59:04.433823+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433823+0000","last_undegraded":"2024-04-17T09:59:04.433823+0000","last_fullsized":"2024-04-17T09:59:04.433823+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455155+0000","last_change":"2024-04-17T09:59:04.443042+0000","last_active":"2024-04-17T09:59:06.455155+0000","last_peered":"2024-04-17T09:59:06.455155+0000","last_clean":"2024-04-17T09:59:06.455155+0000","last_became_active":"2024-04-17T09:59:04.442701+0000","last_became_peered":"2024-04-17T09:59:04.442701+0000","last_unstale":"2024-04-17T09:59:06.455155+0000","last_undegraded":"2024-04-17T09:59:06.455155+0000","last_fullsized":"2024-04-17T09:59:06.455155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:02:06.697176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.425419+0000","last_change":"2024-04-17T09:59:04.425419+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.425419+0000","last_undegraded":"2024-04-17T09:59:04.425419+0000","last_fullsized":"2024-04-17T09:59:04.425419+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427370+0000","last_change":"2024-04-17T09:59:04.438652+0000","last_active":"2024-04-17T09:59:06.427370+0000","last_peered":"2024-04-17T09:59:06.427370+0000","last_clean":"2024-04-17T09:59:06.427370+0000","last_became_active":"2024-04-17T09:59:04.438385+0000","last_became_peered":"2024-04-17T09:59:04.438385+0000","last_unstale":"2024-04-17T09:59:06.427370+0000","last_undegraded":"2024-04-17T09:59:06.427370+0000","last_fullsized":"2024-04-17T09:59:06.427370+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:12.392733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427235+0000","last_change":"2024-04-17T09:59:05.436633+0000","last_active":"2024-04-17T09:59:06.427235+0000","last_peered":"2024-04-17T09:59:06.427235+0000","last_clean":"2024-04-17T09:59:06.427235+0000","last_became_active":"2024-04-17T09:59:05.436348+0000","last_became_peered":"2024-04-17T09:59:05.436348+0000","last_unstale":"2024-04-17T09:59:06.427235+0000","last_undegraded":"2024-04-17T09:59:06.427235+0000","last_fullsized":"2024-04-17T09:59:06.427235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:08.147437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426666+0000","last_change":"2024-04-17T09:59:04.440021+0000","last_active":"2024-04-17T09:59:06.426666+0000","last_peered":"2024-04-17T09:59:06.426666+0000","last_clean":"2024-04-17T09:59:06.426666+0000","last_became_active":"2024-04-17T09:59:04.439185+0000","last_became_peered":"2024-04-17T09:59:04.439185+0000","last_unstale":"2024-04-17T09:59:06.426666+0000","last_undegraded":"2024-04-17T09:59:06.426666+0000","last_fullsized":"2024-04-17T09:59:06.426666+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:51.158495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433375+0000","last_change":"2024-04-17T09:59:04.433375+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433375+0000","last_undegraded":"2024-04-17T09:59:04.433375+0000","last_fullsized":"2024-04-17T09:59:04.433375+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.454817+0000","last_change":"2024-04-17T09:59:04.449411+0000","last_active":"2024-04-17T09:59:06.454817+0000","last_peered":"2024-04-17T09:59:06.454817+0000","last_clean":"2024-04-17T09:59:06.454817+0000","last_became_active":"2024-04-17T09:59:04.449080+0000","last_became_peered":"2024-04-17T09:59:04.449080+0000","last_unstale":"2024-04-17T09:59:06.454817+0000","last_undegraded":"2024-04-17T09:59:06.454817+0000","last_fullsized":"2024-04-17T09:59:06.454817+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:13.341784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.426818+0000","last_change":"2024-04-17T09:59:04.426818+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.426818+0000","last_undegraded":"2024-04-17T09:59:04.426818+0000","last_fullsized":"2024-04-17T09:59:04.426818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.438105+0000","last_change":"2024-04-17T09:59:04.433580+0000","last_active":"2024-04-17T09:59:04.438105+0000","last_peered":"2024-04-17T09:59:04.438105+0000","last_clean":"2024-04-17T09:59:04.438105+0000","last_became_active":"2024-04-17T09:59:04.433250+0000","last_became_peered":"2024-04-17T09:59:04.433250+0000","last_unstale":"2024-04-17T09:59:04.438105+0000","last_undegraded":"2024-04-17T09:59:04.438105+0000","last_fullsized":"2024-04-17T09:59:04.438105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:08:42.684550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428201+0000","last_change":"2024-04-17T09:59:04.428201+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428201+0000","last_undegraded":"2024-04-17T09:59:04.428201+0000","last_fullsized":"2024-04-17T09:59:04.428201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.445242+0000","last_change":"2024-04-17T09:59:04.443547+0000","last_active":"2024-04-17T09:59:04.445242+0000","last_peered":"2024-04-17T09:59:04.445242+0000","last_clean":"2024-04-17T09:59:04.445242+0000","last_became_active":"2024-04-17T09:59:04.443090+0000","last_became_peered":"2024-04-17T09:59:04.443090+0000","last_unstale":"2024-04-17T09:59:04.445242+0000","last_undegraded":"2024-04-17T09:59:04.445242+0000","last_fullsized":"2024-04-17T09:59:04.445242+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:36:05.857724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.440390+0000","last_change":"2024-04-17T09:59:04.440390+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.440390+0000","last_undegraded":"2024-04-17T09:59:04.440390+0000","last_fullsized":"2024-04-17T09:59:04.440390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.444139+0000","last_change":"2024-04-17T09:59:04.438537+0000","last_active":"2024-04-17T09:59:04.444139+0000","last_peered":"2024-04-17T09:59:04.444139+0000","last_clean":"2024-04-17T09:59:04.444139+0000","last_became_active":"2024-04-17T09:59:04.438006+0000","last_became_peered":"2024-04-17T09:59:04.438006+0000","last_unstale":"2024-04-17T09:59:04.444139+0000","last_undegraded":"2024-04-17T09:59:04.444139+0000","last_fullsized":"2024-04-17T09:59:04.444139+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:15:09.312202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455087+0000","last_change":"2024-04-17T09:59:05.439118+0000","last_active":"2024-04-17T09:59:06.455087+0000","last_peered":"2024-04-17T09:59:06.455087+0000","last_clean":"2024-04-17T09:59:06.455087+0000","last_became_active":"2024-04-17T09:59:05.438852+0000","last_became_peered":"2024-04-17T09:59:05.438852+0000","last_unstale":"2024-04-17T09:59:06.455087+0000","last_undegraded":"2024-04-17T09:59:06.455087+0000","last_fullsized":"2024-04-17T09:59:06.455087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:13:00.757214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.444169+0000","last_change":"2024-04-17T09:59:04.438033+0000","last_active":"2024-04-17T09:59:04.444169+0000","last_peered":"2024-04-17T09:59:04.444169+0000","last_clean":"2024-04-17T09:59:04.444169+0000","last_became_active":"2024-04-17T09:59:04.437763+0000","last_became_peered":"2024-04-17T09:59:04.437763+0000","last_unstale":"2024-04-17T09:59:04.444169+0000","last_undegraded":"2024-04-17T09:59:04.444169+0000","last_fullsized":"2024-04-17T09:59:04.444169+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:03:17.824422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427265+0000","last_change":"2024-04-17T09:59:05.447894+0000","last_active":"2024-04-17T09:59:06.427265+0000","last_peered":"2024-04-17T09:59:06.427265+0000","last_clean":"2024-04-17T09:59:06.427265+0000","last_became_active":"2024-04-17T09:59:05.447595+0000","last_became_peered":"2024-04-17T09:59:05.447595+0000","last_unstale":"2024-04-17T09:59:06.427265+0000","last_undegraded":"2024-04-17T09:59:06.427265+0000","last_fullsized":"2024-04-17T09:59:06.427265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:11.377027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.451646+0000","last_change":"2024-04-17T09:59:04.448134+0000","last_active":"2024-04-17T09:59:04.451646+0000","last_peered":"2024-04-17T09:59:04.451646+0000","last_clean":"2024-04-17T09:59:04.451646+0000","last_became_active":"2024-04-17T09:59:04.447872+0000","last_became_peered":"2024-04-17T09:59:04.447872+0000","last_unstale":"2024-04-17T09:59:04.451646+0000","last_undegraded":"2024-04-17T09:59:04.451646+0000","last_fullsized":"2024-04-17T09:59:04.451646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:22:33.540833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455345+0000","last_change":"2024-04-17T09:59:05.439384+0000","last_active":"2024-04-17T09:59:06.455345+0000","last_peered":"2024-04-17T09:59:06.455345+0000","last_clean":"2024-04-17T09:59:06.455345+0000","last_became_active":"2024-04-17T09:59:05.439091+0000","last_became_peered":"2024-04-17T09:59:05.439091+0000","last_unstale":"2024-04-17T09:59:06.455345+0000","last_undegraded":"2024-04-17T09:59:06.455345+0000","last_fullsized":"2024-04-17T09:59:06.455345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:22:13.672459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.443120+0000","last_change":"2024-04-17T09:59:04.440489+0000","last_active":"2024-04-17T09:59:04.443120+0000","last_peered":"2024-04-17T09:59:04.443120+0000","last_clean":"2024-04-17T09:59:04.443120+0000","last_became_active":"2024-04-17T09:59:04.437665+0000","last_became_peered":"2024-04-17T09:59:04.437665+0000","last_unstale":"2024-04-17T09:59:04.443120+0000","last_undegraded":"2024-04-17T09:59:04.443120+0000","last_fullsized":"2024-04-17T09:59:04.443120+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:54:03.821994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433083+0000","last_change":"2024-04-17T09:59:04.433083+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433083+0000","last_undegraded":"2024-04-17T09:59:04.433083+0000","last_fullsized":"2024-04-17T09:59:04.433083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.446156+0000","last_change":"2024-04-17T09:59:04.441625+0000","last_active":"2024-04-17T09:59:04.446156+0000","last_peered":"2024-04-17T09:59:04.446156+0000","last_clean":"2024-04-17T09:59:04.446156+0000","last_became_active":"2024-04-17T09:59:04.441012+0000","last_became_peered":"2024-04-17T09:59:04.441012+0000","last_unstale":"2024-04-17T09:59:04.446156+0000","last_undegraded":"2024-04-17T09:59:04.446156+0000","last_fullsized":"2024-04-17T09:59:04.446156+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:31:12.043046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.426432+0000","last_change":"2024-04-17T09:59:04.426432+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.426432+0000","last_undegraded":"2024-04-17T09:59:04.426432+0000","last_fullsized":"2024-04-17T09:59:04.426432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.451592+0000","last_change":"2024-04-17T09:59:04.446837+0000","last_active":"2024-04-17T09:59:04.451592+0000","last_peered":"2024-04-17T09:59:04.451592+0000","last_clean":"2024-04-17T09:59:04.451592+0000","last_became_active":"2024-04-17T09:59:04.446469+0000","last_became_peered":"2024-04-17T09:59:04.446469+0000","last_unstale":"2024-04-17T09:59:04.451592+0000","last_undegraded":"2024-04-17T09:59:04.451592+0000","last_fullsized":"2024-04-17T09:59:04.451592+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:49.287601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.425532+0000","last_change":"2024-04-17T09:59:04.425532+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.425532+0000","last_undegraded":"2024-04-17T09:59:04.425532+0000","last_fullsized":"2024-04-17T09:59:04.425532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"81'8","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455171+0000","last_change":"2024-04-17T09:59:04.438156+0000","last_active":"2024-04-17T09:59:06.455171+0000","last_peered":"2024-04-17T09:59:06.455171+0000","last_clean":"2024-04-17T09:59:06.455171+0000","last_became_active":"2024-04-17T09:59:04.437821+0000","last_became_peered":"2024-04-17T09:59:04.437821+0000","last_unstale":"2024-04-17T09:59:06.455171+0000","last_undegraded":"2024-04-17T09:59:06.455171+0000","last_fullsized":"2024-04-17T09:59:06.455171+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:54.250176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.432091+0000","last_change":"2024-04-17T09:59:04.432091+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.432091+0000","last_undegraded":"2024-04-17T09:59:04.432091+0000","last_fullsized":"2024-04-17T09:59:04.432091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.448596+0000","last_change":"2024-04-17T09:59:04.447265+0000","last_active":"2024-04-17T09:59:04.448596+0000","last_peered":"2024-04-17T09:59:04.448596+0000","last_clean":"2024-04-17T09:59:04.448596+0000","last_became_active":"2024-04-17T09:59:04.446847+0000","last_became_peered":"2024-04-17T09:59:04.446847+0000","last_unstale":"2024-04-17T09:59:04.448596+0000","last_undegraded":"2024-04-17T09:59:04.448596+0000","last_fullsized":"2024-04-17T09:59:04.448596+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:49.287601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429393+0000","last_change":"2024-04-17T09:59:04.429393+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429393+0000","last_undegraded":"2024-04-17T09:59:04.429393+0000","last_fullsized":"2024-04-17T09:59:04.429393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455137+0000","last_change":"2024-04-17T09:59:04.443540+0000","last_active":"2024-04-17T09:59:06.455137+0000","last_peered":"2024-04-17T09:59:06.455137+0000","last_clean":"2024-04-17T09:59:06.455137+0000","last_became_active":"2024-04-17T09:59:04.443277+0000","last_became_peered":"2024-04-17T09:59:04.443277+0000","last_unstale":"2024-04-17T09:59:06.455137+0000","last_undegraded":"2024-04-17T09:59:06.455137+0000","last_fullsized":"2024-04-17T09:59:06.455137+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:10.371001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427822+0000","last_change":"2024-04-17T09:59:05.447239+0000","last_active":"2024-04-17T09:59:06.427822+0000","last_peered":"2024-04-17T09:59:06.427822+0000","last_clean":"2024-04-17T09:59:06.427822+0000","last_became_active":"2024-04-17T09:59:05.446970+0000","last_became_peered":"2024-04-17T09:59:05.446970+0000","last_unstale":"2024-04-17T09:59:06.427822+0000","last_undegraded":"2024-04-17T09:59:06.427822+0000","last_fullsized":"2024-04-17T09:59:06.427822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:02:07.703202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455269+0000","last_change":"2024-04-17T09:59:04.445958+0000","last_active":"2024-04-17T09:59:06.455269+0000","last_peered":"2024-04-17T09:59:06.455269+0000","last_clean":"2024-04-17T09:59:06.455269+0000","last_became_active":"2024-04-17T09:59:04.445632+0000","last_became_peered":"2024-04-17T09:59:04.445632+0000","last_unstale":"2024-04-17T09:59:06.455269+0000","last_undegraded":"2024-04-17T09:59:06.455269+0000","last_fullsized":"2024-04-17T09:59:06.455269+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:57.288434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.432936+0000","last_change":"2024-04-17T09:59:04.432936+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.432936+0000","last_undegraded":"2024-04-17T09:59:04.432936+0000","last_fullsized":"2024-04-17T09:59:04.432936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.448600+0000","last_change":"2024-04-17T09:59:04.445248+0000","last_active":"2024-04-17T09:59:04.448600+0000","last_peered":"2024-04-17T09:59:04.448600+0000","last_clean":"2024-04-17T09:59:04.448600+0000","last_became_active":"2024-04-17T09:59:04.444809+0000","last_became_peered":"2024-04-17T09:59:04.444809+0000","last_unstale":"2024-04-17T09:59:04.448600+0000","last_undegraded":"2024-04-17T09:59:04.448600+0000","last_fullsized":"2024-04-17T09:59:04.448600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:33:59.203901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.437546+0000","last_change":"2024-04-17T09:59:04.437546+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.437546+0000","last_undegraded":"2024-04-17T09:59:04.437546+0000","last_fullsized":"2024-04-17T09:59:04.437546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.451113+0000","last_change":"2024-04-17T09:59:04.448116+0000","last_active":"2024-04-17T09:59:04.451113+0000","last_peered":"2024-04-17T09:59:04.451113+0000","last_clean":"2024-04-17T09:59:04.451113+0000","last_became_active":"2024-04-17T09:59:04.446755+0000","last_became_peered":"2024-04-17T09:59:04.446755+0000","last_unstale":"2024-04-17T09:59:04.451113+0000","last_undegraded":"2024-04-17T09:59:04.451113+0000","last_fullsized":"2024-04-17T09:59:04.451113+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:04:29.400922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427614+0000","last_change":"2024-04-17T09:59:04.427614+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427614+0000","last_undegraded":"2024-04-17T09:59:04.427614+0000","last_fullsized":"2024-04-17T09:59:04.427614+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.451071+0000","last_change":"2024-04-17T09:59:04.448303+0000","last_active":"2024-04-17T09:59:04.451071+0000","last_peered":"2024-04-17T09:59:04.451071+0000","last_clean":"2024-04-17T09:59:04.451071+0000","last_became_active":"2024-04-17T09:59:04.446032+0000","last_became_peered":"2024-04-17T09:59:04.446032+0000","last_unstale":"2024-04-17T09:59:04.451071+0000","last_undegraded":"2024-04-17T09:59:04.451071+0000","last_fullsized":"2024-04-17T09:59:04.451071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:02:55.706845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.440407+0000","last_change":"2024-04-17T09:59:04.440407+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.440407+0000","last_undegraded":"2024-04-17T09:59:04.440407+0000","last_fullsized":"2024-04-17T09:59:04.440407+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.448496+0000","last_change":"2024-04-17T09:59:04.445983+0000","last_active":"2024-04-17T09:59:04.448496+0000","last_peered":"2024-04-17T09:59:04.448496+0000","last_clean":"2024-04-17T09:59:04.448496+0000","last_became_active":"2024-04-17T09:59:04.445671+0000","last_became_peered":"2024-04-17T09:59:04.445671+0000","last_unstale":"2024-04-17T09:59:04.448496+0000","last_undegraded":"2024-04-17T09:59:04.448496+0000","last_fullsized":"2024-04-17T09:59:04.448496+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:36.060764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.428075+0000","last_change":"2024-04-17T09:59:05.449697+0000","last_active":"2024-04-17T09:59:06.428075+0000","last_peered":"2024-04-17T09:59:06.428075+0000","last_clean":"2024-04-17T09:59:06.428075+0000","last_became_active":"2024-04-17T09:59:05.449390+0000","last_became_peered":"2024-04-17T09:59:05.449390+0000","last_unstale":"2024-04-17T09:59:06.428075+0000","last_undegraded":"2024-04-17T09:59:06.428075+0000","last_fullsized":"2024-04-17T09:59:06.428075+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:11:22.087912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.447272+0000","last_change":"2024-04-17T09:59:04.442441+0000","last_active":"2024-04-17T09:59:04.447272+0000","last_peered":"2024-04-17T09:59:04.447272+0000","last_clean":"2024-04-17T09:59:04.447272+0000","last_became_active":"2024-04-17T09:59:04.442051+0000","last_became_peered":"2024-04-17T09:59:04.442051+0000","last_unstale":"2024-04-17T09:59:04.447272+0000","last_undegraded":"2024-04-17T09:59:04.447272+0000","last_fullsized":"2024-04-17T09:59:04.447272+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:16:08.377113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433918+0000","last_change":"2024-04-17T09:59:04.433918+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433918+0000","last_undegraded":"2024-04-17T09:59:04.433918+0000","last_fullsized":"2024-04-17T09:59:04.433918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.446270+0000","last_change":"2024-04-17T09:59:04.441305+0000","last_active":"2024-04-17T09:59:04.446270+0000","last_peered":"2024-04-17T09:59:04.446270+0000","last_clean":"2024-04-17T09:59:04.446270+0000","last_became_active":"2024-04-17T09:59:04.441044+0000","last_became_peered":"2024-04-17T09:59:04.441044+0000","last_unstale":"2024-04-17T09:59:04.446270+0000","last_undegraded":"2024-04-17T09:59:04.446270+0000","last_fullsized":"2024-04-17T09:59:04.446270+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:06:39.663043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428308+0000","last_change":"2024-04-17T09:59:04.428308+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428308+0000","last_undegraded":"2024-04-17T09:59:04.428308+0000","last_fullsized":"2024-04-17T09:59:04.428308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.445184+0000","last_change":"2024-04-17T09:59:04.442494+0000","last_active":"2024-04-17T09:59:04.445184+0000","last_peered":"2024-04-17T09:59:04.445184+0000","last_clean":"2024-04-17T09:59:04.445184+0000","last_became_active":"2024-04-17T09:59:04.442254+0000","last_became_peered":"2024-04-17T09:59:04.442254+0000","last_unstale":"2024-04-17T09:59:04.445184+0000","last_undegraded":"2024-04-17T09:59:04.445184+0000","last_fullsized":"2024-04-17T09:59:04.445184+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:09:27.952101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427862+0000","last_change":"2024-04-17T09:59:05.438162+0000","last_active":"2024-04-17T09:59:06.427862+0000","last_peered":"2024-04-17T09:59:06.427862+0000","last_clean":"2024-04-17T09:59:06.427862+0000","last_became_active":"2024-04-17T09:59:05.437880+0000","last_became_peered":"2024-04-17T09:59:05.437880+0000","last_unstale":"2024-04-17T09:59:06.427862+0000","last_undegraded":"2024-04-17T09:59:06.427862+0000","last_fullsized":"2024-04-17T09:59:06.427862+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:58.294460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.445858+0000","last_change":"2024-04-17T09:59:04.444713+0000","last_active":"2024-04-17T09:59:04.445858+0000","last_peered":"2024-04-17T09:59:04.445858+0000","last_clean":"2024-04-17T09:59:04.445858+0000","last_became_active":"2024-04-17T09:59:04.444438+0000","last_became_peered":"2024-04-17T09:59:04.444438+0000","last_unstale":"2024-04-17T09:59:04.445858+0000","last_undegraded":"2024-04-17T09:59:04.445858+0000","last_fullsized":"2024-04-17T09:59:04.445858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:29:09.325067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.437197+0000","last_change":"2024-04-17T09:59:04.437197+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.437197+0000","last_undegraded":"2024-04-17T09:59:04.437197+0000","last_fullsized":"2024-04-17T09:59:04.437197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455124+0000","last_change":"2024-04-17T09:59:04.438627+0000","last_active":"2024-04-17T09:59:06.455124+0000","last_peered":"2024-04-17T09:59:06.455124+0000","last_clean":"2024-04-17T09:59:06.455124+0000","last_became_active":"2024-04-17T09:59:04.438100+0000","last_became_peered":"2024-04-17T09:59:04.438100+0000","last_unstale":"2024-04-17T09:59:06.455124+0000","last_undegraded":"2024-04-17T09:59:06.455124+0000","last_fullsized":"2024-04-17T09:59:06.455124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:53:55.006094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455088+0000","last_change":"2024-04-17T09:59:05.447417+0000","last_active":"2024-04-17T09:59:06.455088+0000","last_peered":"2024-04-17T09:59:06.455088+0000","last_clean":"2024-04-17T09:59:06.455088+0000","last_became_active":"2024-04-17T09:59:05.447176+0000","last_became_peered":"2024-04-17T09:59:05.447176+0000","last_unstale":"2024-04-17T09:59:06.455088+0000","last_undegraded":"2024-04-17T09:59:06.455088+0000","last_fullsized":"2024-04-17T09:59:06.455088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:18:04.685528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426694+0000","last_change":"2024-04-17T09:59:04.440315+0000","last_active":"2024-04-17T09:59:06.426694+0000","last_peered":"2024-04-17T09:59:06.426694+0000","last_clean":"2024-04-17T09:59:06.426694+0000","last_became_active":"2024-04-17T09:59:04.439614+0000","last_became_peered":"2024-04-17T09:59:04.439614+0000","last_unstale":"2024-04-17T09:59:06.426694+0000","last_undegraded":"2024-04-17T09:59:06.426694+0000","last_fullsized":"2024-04-17T09:59:06.426694+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:02.341478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.432875+0000","last_change":"2024-04-17T09:59:04.432875+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.432875+0000","last_undegraded":"2024-04-17T09:59:04.432875+0000","last_fullsized":"2024-04-17T09:59:04.432875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.445028+0000","last_change":"2024-04-17T09:59:04.442633+0000","last_active":"2024-04-17T09:59:04.445028+0000","last_peered":"2024-04-17T09:59:04.445028+0000","last_clean":"2024-04-17T09:59:04.445028+0000","last_became_active":"2024-04-17T09:59:04.442347+0000","last_became_peered":"2024-04-17T09:59:04.442347+0000","last_unstale":"2024-04-17T09:59:04.445028+0000","last_undegraded":"2024-04-17T09:59:04.445028+0000","last_fullsized":"2024-04-17T09:59:04.445028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:38:59.533078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427513+0000","last_change":"2024-04-17T09:59:04.427513+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427513+0000","last_undegraded":"2024-04-17T09:59:04.427513+0000","last_fullsized":"2024-04-17T09:59:04.427513+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426835+0000","last_change":"2024-04-17T09:59:04.440613+0000","last_active":"2024-04-17T09:59:06.426835+0000","last_peered":"2024-04-17T09:59:06.426835+0000","last_clean":"2024-04-17T09:59:06.426835+0000","last_became_active":"2024-04-17T09:59:04.440063+0000","last_became_peered":"2024-04-17T09:59:04.440063+0000","last_unstale":"2024-04-17T09:59:06.426835+0000","last_undegraded":"2024-04-17T09:59:06.426835+0000","last_fullsized":"2024-04-17T09:59:06.426835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:53:08.846256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.425466+0000","last_change":"2024-04-17T09:59:04.425466+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.425466+0000","last_undegraded":"2024-04-17T09:59:04.425466+0000","last_fullsized":"2024-04-17T09:59:04.425466+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.446679+0000","last_change":"2024-04-17T09:59:04.445356+0000","last_active":"2024-04-17T09:59:04.446679+0000","last_peered":"2024-04-17T09:59:04.446679+0000","last_clean":"2024-04-17T09:59:04.446679+0000","last_became_active":"2024-04-17T09:59:04.445010+0000","last_became_peered":"2024-04-17T09:59:04.445010+0000","last_unstale":"2024-04-17T09:59:04.446679+0000","last_undegraded":"2024-04-17T09:59:04.446679+0000","last_fullsized":"2024-04-17T09:59:04.446679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:32:37.586889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":280,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.434933+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T09:59:04.434933+0000","last_peered":"2024-04-17T09:59:04.434933+0000","last_clean":"2024-04-17T09:59:04.434933+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T09:59:04.434933+0000","last_undegraded":"2024-04-17T09:59:04.434933+0000","last_fullsized":"2024-04-17T09:59:04.434933+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509516,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1568768,"data_stored":1002078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":411710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509516,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":962560,"data_stored":405435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509516,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":411710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":411710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509516,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":411710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542221,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":424785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542221,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36900,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334940,"statfs":{"total":96636764160,"available":96598978560,"internally_reserved":0,"allocated":1597440,"data_stored":1015187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T09:59:09.958 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T09:59:10.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.138+0000 7f6b0c000640 1 Processor -- start 2024-04-17T09:59:10.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.138+0000 7f6b0c000640 1 -- start start 2024-04-17T09:59:10.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.142+0000 7f6b0c000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b041481e0 0x7f6b04071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:10.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.142+0000 7f6b0c000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b0406ef70 con 0x7f6b04144f20 2024-04-17T09:59:10.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.142+0000 7f6b0c000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b040aed20 con 0x7f6b04072330 2024-04-17T09:59:10.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.142+0000 7f6b0c000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6b040727a0 con 0x7f6b041481e0 2024-04-17T09:59:10.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.142+0000 7f6b08c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b041481e0 0x7f6b04071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:10.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.142+0000 7f6b09600640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6b04144f20 0x7f6b0406e860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38086/0 (socket says 172.21.15.32:38086) 2024-04-17T09:59:10.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.142+0000 7f6b09600640 1 -- 172.21.15.32:0/1468685679 learned_addr learned my addr 172.21.15.32:0/1468685679 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:10.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.142+0000 7f6b08c00640 1 -- 172.21.15.32:0/1468685679 >> v1:172.21.15.144:6789/0 conn(0x7f6b04072330 legacy=0x7f6b040b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:10.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.142+0000 7f6b08c00640 1 -- 172.21.15.32:0/1468685679 >> v1:172.21.15.32:6789/0 conn(0x7f6b04144f20 legacy=0x7f6b0406e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:10.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.142+0000 7f6b08c00640 1 -- 172.21.15.32:0/1468685679 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b040af000 con 0x7f6b041481e0 2024-04-17T09:59:10.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.146+0000 7f6b08c00640 1 --2- 172.21.15.32:0/1468685679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b041481e0 0x7f6b04071df0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f6af00097c0 tx=0x7f6af001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a24f010f5a1a32 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:10.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.146+0000 7f6b03e00640 1 -- 172.21.15.32:0/1468685679 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af00b0070 con 0x7f6b041481e0 2024-04-17T09:59:10.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.146+0000 7f6b03e00640 1 -- 172.21.15.32:0/1468685679 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6af0004230 con 0x7f6b041481e0 2024-04-17T09:59:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.146+0000 7f6b03e00640 1 -- 172.21.15.32:0/1468685679 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0003620 con 0x7f6b041481e0 2024-04-17T09:59:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.146+0000 7f6b0c000640 1 -- 172.21.15.32:0/1468685679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b041481e0 msgr2=0x7f6b04071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.146+0000 7f6b0c000640 1 --2- 172.21.15.32:0/1468685679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b041481e0 0x7f6b04071df0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f6af00097c0 tx=0x7f6af001c460 comp rx=0 tx=0).stop 2024-04-17T09:59:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.146+0000 7f6b0c000640 1 -- 172.21.15.32:0/1468685679 shutdown_connections 2024-04-17T09:59:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.146+0000 7f6b0c000640 1 --2- 172.21.15.32:0/1468685679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b041481e0 0x7f6b04071df0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.146+0000 7f6b0c000640 1 -- 172.21.15.32:0/1468685679 >> 172.21.15.32:0/1468685679 conn(0x7f6b0406ba30 msgr2=0x7f6b0406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.146+0000 7f6b0c000640 1 -- 172.21.15.32:0/1468685679 shutdown_connections 2024-04-17T09:59:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.146+0000 7f6b0c000640 1 -- 172.21.15.32:0/1468685679 wait complete. 2024-04-17T09:59:10.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.146+0000 7f6b0c000640 1 Processor -- start 2024-04-17T09:59:10.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.146+0000 7f6b0c000640 1 -- start start 2024-04-17T09:59:10.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b0c000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b04072330 0x7f6b040cf430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:10.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b0c000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b04144f20 0x7f6b040b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:10.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b0c000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b041481e0 0x7f6b04160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:10.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b0c000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6b040afdc0 con 0x7f6b04072330 2024-04-17T09:59:10.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b0c000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0416c5e0 con 0x7f6b04144f20 2024-04-17T09:59:10.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b0c000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6b040b2780 con 0x7f6b041481e0 2024-04-17T09:59:10.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b09600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b04072330 0x7f6b040cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:10.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b09600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b04072330 0x7f6b040cf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34702/0 (socket says 172.21.15.32:34702) 2024-04-17T09:59:10.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b09600640 1 -- 172.21.15.32:0/371118468 learned_addr learned my addr 172.21.15.32:0/371118468 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:10.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b08c00640 1 --2- 172.21.15.32:0/371118468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b04144f20 0x7f6b040b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:10.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b0a000640 1 --2- 172.21.15.32:0/371118468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b041481e0 0x7f6b04160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:10.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b09600640 1 -- 172.21.15.32:0/371118468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b041481e0 msgr2=0x7f6b04160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:10.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b09600640 1 --2- 172.21.15.32:0/371118468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b041481e0 0x7f6b04160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:10.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b09600640 1 -- 172.21.15.32:0/371118468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b04144f20 msgr2=0x7f6b040b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:10.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b09600640 1 --2- 172.21.15.32:0/371118468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b04144f20 0x7f6b040b1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:10.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b09600640 1 -- 172.21.15.32:0/371118468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b0415b020 con 0x7f6b04072330 2024-04-17T09:59:10.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b09600640 1 --2- 172.21.15.32:0/371118468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b04072330 0x7f6b040cf430 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f6af800c420 tx=0x7f6af800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:10.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b01600640 1 -- 172.21.15.32:0/371118468 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af8017070 con 0x7f6b04072330 2024-04-17T09:59:10.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b01600640 1 -- 172.21.15.32:0/371118468 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6af8005cf0 con 0x7f6b04072330 2024-04-17T09:59:10.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b0c000640 1 -- 172.21.15.32:0/371118468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b040b1b70 con 0x7f6b04072330 2024-04-17T09:59:10.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b01600640 1 -- 172.21.15.32:0/371118468 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af80072f0 con 0x7f6b04072330 2024-04-17T09:59:10.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b0c000640 1 -- 172.21.15.32:0/371118468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b040b2130 con 0x7f6b04072330 2024-04-17T09:59:10.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b0c000640 1 -- 172.21.15.32:0/371118468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b04068a90 con 0x7f6b04072330 2024-04-17T09:59:10.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.150+0000 7f6b01600640 1 -- 172.21.15.32:0/371118468 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6af801a070 con 0x7f6b04072330 2024-04-17T09:59:10.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.154+0000 7f6b01600640 1 --2- 172.21.15.32:0/371118468 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6ae4068380 0x7f6ae406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:10.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.154+0000 7f6b01600640 1 -- 172.21.15.32:0/371118468 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f6af8085d70 con 0x7f6b04072330 2024-04-17T09:59:10.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.154+0000 7f6b08c00640 1 --2- 172.21.15.32:0/371118468 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6ae4068380 0x7f6ae406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:10.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.154+0000 7f6b08c00640 1 --2- 172.21.15.32:0/371118468 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6ae4068380 0x7f6ae406a840 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6af00ad000 tx=0x7f6af00a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:10.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.158+0000 7f6b01600640 1 -- 172.21.15.32:0/371118468 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f6af8055c40 con 0x7f6b04072330 2024-04-17T09:59:10.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.318+0000 7f6b0c000640 1 -- 172.21.15.32:0/371118468 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6b0413a620 con 0x7f6ae4068380 2024-04-17T09:59:10.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.326+0000 7f6b01600640 1 -- 172.21.15.32:0/371118468 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248008 (secure 0 0 0) 0x7f6b0413a620 con 0x7f6ae4068380 2024-04-17T09:59:10.325 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:59:10.326 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T09:59:10.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.330+0000 7f6b0c000640 1 -- 172.21.15.32:0/371118468 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6ae4068380 msgr2=0x7f6ae406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.330+0000 7f6b0c000640 1 --2- 172.21.15.32:0/371118468 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6ae4068380 0x7f6ae406a840 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6af00ad000 tx=0x7f6af00a6000 comp rx=0 tx=0).stop 2024-04-17T09:59:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.330+0000 7f6b0c000640 1 -- 172.21.15.32:0/371118468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b04072330 msgr2=0x7f6b040cf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.330+0000 7f6b0c000640 1 --2- 172.21.15.32:0/371118468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b04072330 0x7f6b040cf430 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f6af800c420 tx=0x7f6af800c8e0 comp rx=0 tx=0).stop 2024-04-17T09:59:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.330+0000 7f6b0c000640 1 -- 172.21.15.32:0/371118468 shutdown_connections 2024-04-17T09:59:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.330+0000 7f6b0c000640 1 --2- 172.21.15.32:0/371118468 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6ae4068380 0x7f6ae406a840 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.330+0000 7f6b0c000640 1 --2- 172.21.15.32:0/371118468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b041481e0 0x7f6b04160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.330+0000 7f6b0c000640 1 --2- 172.21.15.32:0/371118468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b04144f20 0x7f6b040b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.330+0000 7f6b0c000640 1 --2- 172.21.15.32:0/371118468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b04072330 0x7f6b040cf430 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.330+0000 7f6b0c000640 1 -- 172.21.15.32:0/371118468 >> 172.21.15.32:0/371118468 conn(0x7f6b0406ba30 msgr2=0x7f6b04146c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.330+0000 7f6b0c000640 1 -- 172.21.15.32:0/371118468 shutdown_connections 2024-04-17T09:59:10.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.330+0000 7f6b0c000640 1 -- 172.21.15.32:0/371118468 wait complete. 2024-04-17T09:59:10.347 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-04-17T09:59:09.365238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":180,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258280,"kb_used_data":8816,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716440,"statfs":{"total":773094113280,"available":772829634560,"internally_reserved":0,"allocated":9027584,"data_stored":4494325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.953388"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.430338+0000","last_change":"2024-04-17T09:59:04.430338+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.430338+0000","last_undegraded":"2024-04-17T09:59:04.430338+0000","last_fullsized":"2024-04-17T09:59:04.430338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429509+0000","last_change":"2024-04-17T09:59:04.429509+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429509+0000","last_undegraded":"2024-04-17T09:59:04.429509+0000","last_fullsized":"2024-04-17T09:59:04.429509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433330+0000","last_change":"2024-04-17T09:59:04.433330+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433330+0000","last_undegraded":"2024-04-17T09:59:04.433330+0000","last_fullsized":"2024-04-17T09:59:04.433330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429769+0000","last_change":"2024-04-17T09:59:04.429769+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429769+0000","last_undegraded":"2024-04-17T09:59:04.429769+0000","last_fullsized":"2024-04-17T09:59:04.429769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427070+0000","last_change":"2024-04-17T09:59:05.435295+0000","last_active":"2024-04-17T09:59:06.427070+0000","last_peered":"2024-04-17T09:59:06.427070+0000","last_clean":"2024-04-17T09:59:06.427070+0000","last_became_active":"2024-04-17T09:59:05.434948+0000","last_became_peered":"2024-04-17T09:59:05.434948+0000","last_unstale":"2024-04-17T09:59:06.427070+0000","last_undegraded":"2024-04-17T09:59:06.427070+0000","last_fullsized":"2024-04-17T09:59:06.427070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:14.347809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.438674+0000","last_change":"2024-04-17T09:59:04.438674+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.438674+0000","last_undegraded":"2024-04-17T09:59:04.438674+0000","last_fullsized":"2024-04-17T09:59:04.438674+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429372+0000","last_change":"2024-04-17T09:59:04.429372+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429372+0000","last_undegraded":"2024-04-17T09:59:04.429372+0000","last_fullsized":"2024-04-17T09:59:04.429372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.437249+0000","last_change":"2024-04-17T09:59:04.437249+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.437249+0000","last_undegraded":"2024-04-17T09:59:04.437249+0000","last_fullsized":"2024-04-17T09:59:04.437249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.440846+0000","last_change":"2024-04-17T09:59:04.440846+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.440846+0000","last_undegraded":"2024-04-17T09:59:04.440846+0000","last_fullsized":"2024-04-17T09:59:04.440846+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427452+0000","last_change":"2024-04-17T09:59:04.427452+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427452+0000","last_undegraded":"2024-04-17T09:59:04.427452+0000","last_fullsized":"2024-04-17T09:59:04.427452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427477+0000","last_change":"2024-04-17T09:59:04.427477+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427477+0000","last_undegraded":"2024-04-17T09:59:04.427477+0000","last_fullsized":"2024-04-17T09:59:04.427477+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427835+0000","last_change":"2024-04-17T09:59:04.427835+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427835+0000","last_undegraded":"2024-04-17T09:59:04.427835+0000","last_fullsized":"2024-04-17T09:59:04.427835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433600+0000","last_change":"2024-04-17T09:59:04.433600+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433600+0000","last_undegraded":"2024-04-17T09:59:04.433600+0000","last_fullsized":"2024-04-17T09:59:04.433600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.435489+0000","last_change":"2024-04-17T09:59:04.435489+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.435489+0000","last_undegraded":"2024-04-17T09:59:04.435489+0000","last_fullsized":"2024-04-17T09:59:04.435489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427347+0000","last_change":"2024-04-17T09:59:04.427347+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427347+0000","last_undegraded":"2024-04-17T09:59:04.427347+0000","last_fullsized":"2024-04-17T09:59:04.427347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426220+0000","last_change":"2024-04-17T09:59:05.445968+0000","last_active":"2024-04-17T09:59:06.426220+0000","last_peered":"2024-04-17T09:59:06.426220+0000","last_clean":"2024-04-17T09:59:06.426220+0000","last_became_active":"2024-04-17T09:59:05.445695+0000","last_became_peered":"2024-04-17T09:59:05.445695+0000","last_unstale":"2024-04-17T09:59:06.426220+0000","last_undegraded":"2024-04-17T09:59:06.426220+0000","last_fullsized":"2024-04-17T09:59:06.426220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:27:15.977393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.428029+0000","last_change":"2024-04-17T09:59:05.447691+0000","last_active":"2024-04-17T09:59:06.428029+0000","last_peered":"2024-04-17T09:59:06.428029+0000","last_clean":"2024-04-17T09:59:06.428029+0000","last_became_active":"2024-04-17T09:59:05.447403+0000","last_became_peered":"2024-04-17T09:59:05.447403+0000","last_unstale":"2024-04-17T09:59:06.428029+0000","last_undegraded":"2024-04-17T09:59:06.428029+0000","last_fullsized":"2024-04-17T09:59:06.428029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:15.267856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.432446+0000","last_change":"2024-04-17T09:59:04.432446+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.432446+0000","last_undegraded":"2024-04-17T09:59:04.432446+0000","last_fullsized":"2024-04-17T09:59:04.432446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455142+0000","last_change":"2024-04-17T09:59:05.442262+0000","last_active":"2024-04-17T09:59:06.455142+0000","last_peered":"2024-04-17T09:59:06.455142+0000","last_clean":"2024-04-17T09:59:06.455142+0000","last_became_active":"2024-04-17T09:59:05.441944+0000","last_became_peered":"2024-04-17T09:59:05.441944+0000","last_unstale":"2024-04-17T09:59:06.455142+0000","last_undegraded":"2024-04-17T09:59:06.455142+0000","last_fullsized":"2024-04-17T09:59:06.455142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:55:27.872332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429260+0000","last_change":"2024-04-17T09:59:04.429260+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429260+0000","last_undegraded":"2024-04-17T09:59:04.429260+0000","last_fullsized":"2024-04-17T09:59:04.429260+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429463+0000","last_change":"2024-04-17T09:59:04.429463+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429463+0000","last_undegraded":"2024-04-17T09:59:04.429463+0000","last_fullsized":"2024-04-17T09:59:04.429463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428078+0000","last_change":"2024-04-17T09:59:04.428078+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428078+0000","last_undegraded":"2024-04-17T09:59:04.428078+0000","last_fullsized":"2024-04-17T09:59:04.428078+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427009+0000","last_change":"2024-04-17T09:59:04.427009+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427009+0000","last_undegraded":"2024-04-17T09:59:04.427009+0000","last_fullsized":"2024-04-17T09:59:04.427009+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428239+0000","last_change":"2024-04-17T09:59:04.428239+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428239+0000","last_undegraded":"2024-04-17T09:59:04.428239+0000","last_fullsized":"2024-04-17T09:59:04.428239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.425424+0000","last_change":"2024-04-17T09:59:04.425424+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.425424+0000","last_undegraded":"2024-04-17T09:59:04.425424+0000","last_fullsized":"2024-04-17T09:59:04.425424+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.432662+0000","last_change":"2024-04-17T09:59:04.432662+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.432662+0000","last_undegraded":"2024-04-17T09:59:04.432662+0000","last_fullsized":"2024-04-17T09:59:04.432662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427077+0000","last_change":"2024-04-17T09:59:05.438772+0000","last_active":"2024-04-17T09:59:06.427077+0000","last_peered":"2024-04-17T09:59:06.427077+0000","last_clean":"2024-04-17T09:59:06.427077+0000","last_became_active":"2024-04-17T09:59:05.438444+0000","last_became_peered":"2024-04-17T09:59:05.438444+0000","last_unstale":"2024-04-17T09:59:06.427077+0000","last_undegraded":"2024-04-17T09:59:06.427077+0000","last_fullsized":"2024-04-17T09:59:06.427077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:53:56.012120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429291+0000","last_change":"2024-04-17T09:59:04.429291+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429291+0000","last_undegraded":"2024-04-17T09:59:04.429291+0000","last_fullsized":"2024-04-17T09:59:04.429291+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427246+0000","last_change":"2024-04-17T09:59:04.427246+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427246+0000","last_undegraded":"2024-04-17T09:59:04.427246+0000","last_fullsized":"2024-04-17T09:59:04.427246+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.426566+0000","last_change":"2024-04-17T09:59:04.426566+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.426566+0000","last_undegraded":"2024-04-17T09:59:04.426566+0000","last_fullsized":"2024-04-17T09:59:04.426566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426520+0000","last_change":"2024-04-17T09:59:05.438189+0000","last_active":"2024-04-17T09:59:06.426520+0000","last_peered":"2024-04-17T09:59:06.426520+0000","last_clean":"2024-04-17T09:59:06.426520+0000","last_became_active":"2024-04-17T09:59:05.437898+0000","last_became_peered":"2024-04-17T09:59:05.437898+0000","last_unstale":"2024-04-17T09:59:06.426520+0000","last_undegraded":"2024-04-17T09:59:06.426520+0000","last_fullsized":"2024-04-17T09:59:06.426520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:55.256201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428134+0000","last_change":"2024-04-17T09:59:04.428134+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428134+0000","last_undegraded":"2024-04-17T09:59:04.428134+0000","last_fullsized":"2024-04-17T09:59:04.428134+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.428003+0000","last_change":"2024-04-17T09:59:05.445880+0000","last_active":"2024-04-17T09:59:06.428003+0000","last_peered":"2024-04-17T09:59:06.428003+0000","last_clean":"2024-04-17T09:59:06.428003+0000","last_became_active":"2024-04-17T09:59:05.439231+0000","last_became_peered":"2024-04-17T09:59:05.439231+0000","last_unstale":"2024-04-17T09:59:06.428003+0000","last_undegraded":"2024-04-17T09:59:06.428003+0000","last_fullsized":"2024-04-17T09:59:06.428003+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:40.503102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.445224+0000","last_change":"2024-04-17T09:59:04.443338+0000","last_active":"2024-04-17T09:59:04.445224+0000","last_peered":"2024-04-17T09:59:04.445224+0000","last_clean":"2024-04-17T09:59:04.445224+0000","last_became_active":"2024-04-17T09:59:04.442697+0000","last_became_peered":"2024-04-17T09:59:04.442697+0000","last_unstale":"2024-04-17T09:59:04.445224+0000","last_undegraded":"2024-04-17T09:59:04.445224+0000","last_fullsized":"2024-04-17T09:59:04.445224+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:39:16.180270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429909+0000","last_change":"2024-04-17T09:59:04.429909+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429909+0000","last_undegraded":"2024-04-17T09:59:04.429909+0000","last_fullsized":"2024-04-17T09:59:04.429909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.428060+0000","last_change":"2024-04-17T09:59:04.439722+0000","last_active":"2024-04-17T09:59:06.428060+0000","last_peered":"2024-04-17T09:59:06.428060+0000","last_clean":"2024-04-17T09:59:06.428060+0000","last_became_active":"2024-04-17T09:59:04.439452+0000","last_became_peered":"2024-04-17T09:59:04.439452+0000","last_unstale":"2024-04-17T09:59:06.428060+0000","last_undegraded":"2024-04-17T09:59:06.428060+0000","last_fullsized":"2024-04-17T09:59:06.428060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:49:05.349813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428740+0000","last_change":"2024-04-17T09:59:04.428740+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428740+0000","last_undegraded":"2024-04-17T09:59:04.428740+0000","last_fullsized":"2024-04-17T09:59:04.428740+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455165+0000","last_change":"2024-04-17T09:59:04.444375+0000","last_active":"2024-04-17T09:59:06.455165+0000","last_peered":"2024-04-17T09:59:06.455165+0000","last_clean":"2024-04-17T09:59:06.455165+0000","last_became_active":"2024-04-17T09:59:04.444125+0000","last_became_peered":"2024-04-17T09:59:04.444125+0000","last_unstale":"2024-04-17T09:59:06.455165+0000","last_undegraded":"2024-04-17T09:59:06.455165+0000","last_fullsized":"2024-04-17T09:59:06.455165+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:07.141412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433823+0000","last_change":"2024-04-17T09:59:04.433823+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433823+0000","last_undegraded":"2024-04-17T09:59:04.433823+0000","last_fullsized":"2024-04-17T09:59:04.433823+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455155+0000","last_change":"2024-04-17T09:59:04.443042+0000","last_active":"2024-04-17T09:59:06.455155+0000","last_peered":"2024-04-17T09:59:06.455155+0000","last_clean":"2024-04-17T09:59:06.455155+0000","last_became_active":"2024-04-17T09:59:04.442701+0000","last_became_peered":"2024-04-17T09:59:04.442701+0000","last_unstale":"2024-04-17T09:59:06.455155+0000","last_undegraded":"2024-04-17T09:59:06.455155+0000","last_fullsized":"2024-04-17T09:59:06.455155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:02:06.697176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.425419+0000","last_change":"2024-04-17T09:59:04.425419+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.425419+0000","last_undegraded":"2024-04-17T09:59:04.425419+0000","last_fullsized":"2024-04-17T09:59:04.425419+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427370+0000","last_change":"2024-04-17T09:59:04.438652+0000","last_active":"2024-04-17T09:59:06.427370+0000","last_peered":"2024-04-17T09:59:06.427370+0000","last_clean":"2024-04-17T09:59:06.427370+0000","last_became_active":"2024-04-17T09:59:04.438385+0000","last_became_peered":"2024-04-17T09:59:04.438385+0000","last_unstale":"2024-04-17T09:59:06.427370+0000","last_undegraded":"2024-04-17T09:59:06.427370+0000","last_fullsized":"2024-04-17T09:59:06.427370+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:12.392733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427235+0000","last_change":"2024-04-17T09:59:05.436633+0000","last_active":"2024-04-17T09:59:06.427235+0000","last_peered":"2024-04-17T09:59:06.427235+0000","last_clean":"2024-04-17T09:59:06.427235+0000","last_became_active":"2024-04-17T09:59:05.436348+0000","last_became_peered":"2024-04-17T09:59:05.436348+0000","last_unstale":"2024-04-17T09:59:06.427235+0000","last_undegraded":"2024-04-17T09:59:06.427235+0000","last_fullsized":"2024-04-17T09:59:06.427235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:08.147437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426666+0000","last_change":"2024-04-17T09:59:04.440021+0000","last_active":"2024-04-17T09:59:06.426666+0000","last_peered":"2024-04-17T09:59:06.426666+0000","last_clean":"2024-04-17T09:59:06.426666+0000","last_became_active":"2024-04-17T09:59:04.439185+0000","last_became_peered":"2024-04-17T09:59:04.439185+0000","last_unstale":"2024-04-17T09:59:06.426666+0000","last_undegraded":"2024-04-17T09:59:06.426666+0000","last_fullsized":"2024-04-17T09:59:06.426666+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:51.158495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433375+0000","last_change":"2024-04-17T09:59:04.433375+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433375+0000","last_undegraded":"2024-04-17T09:59:04.433375+0000","last_fullsized":"2024-04-17T09:59:04.433375+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.454817+0000","last_change":"2024-04-17T09:59:04.449411+0000","last_active":"2024-04-17T09:59:06.454817+0000","last_peered":"2024-04-17T09:59:06.454817+0000","last_clean":"2024-04-17T09:59:06.454817+0000","last_became_active":"2024-04-17T09:59:04.449080+0000","last_became_peered":"2024-04-17T09:59:04.449080+0000","last_unstale":"2024-04-17T09:59:06.454817+0000","last_undegraded":"2024-04-17T09:59:06.454817+0000","last_fullsized":"2024-04-17T09:59:06.454817+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:13.341784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.426818+0000","last_change":"2024-04-17T09:59:04.426818+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.426818+0000","last_undegraded":"2024-04-17T09:59:04.426818+0000","last_fullsized":"2024-04-17T09:59:04.426818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.438105+0000","last_change":"2024-04-17T09:59:04.433580+0000","last_active":"2024-04-17T09:59:04.438105+0000","last_peered":"2024-04-17T09:59:04.438105+0000","last_clean":"2024-04-17T09:59:04.438105+0000","last_became_active":"2024-04-17T09:59:04.433250+0000","last_became_peered":"2024-04-17T09:59:04.433250+0000","last_unstale":"2024-04-17T09:59:04.438105+0000","last_undegraded":"2024-04-17T09:59:04.438105+0000","last_fullsized":"2024-04-17T09:59:04.438105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:08:42.684550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428201+0000","last_change":"2024-04-17T09:59:04.428201+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428201+0000","last_undegraded":"2024-04-17T09:59:04.428201+0000","last_fullsized":"2024-04-17T09:59:04.428201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.445242+0000","last_change":"2024-04-17T09:59:04.443547+0000","last_active":"2024-04-17T09:59:04.445242+0000","last_peered":"2024-04-17T09:59:04.445242+0000","last_clean":"2024-04-17T09:59:04.445242+0000","last_became_active":"2024-04-17T09:59:04.443090+0000","last_became_peered":"2024-04-17T09:59:04.443090+0000","last_unstale":"2024-04-17T09:59:04.445242+0000","last_undegraded":"2024-04-17T09:59:04.445242+0000","last_fullsized":"2024-04-17T09:59:04.445242+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:36:05.857724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.440390+0000","last_change":"2024-04-17T09:59:04.440390+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.440390+0000","last_undegraded":"2024-04-17T09:59:04.440390+0000","last_fullsized":"2024-04-17T09:59:04.440390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.444139+0000","last_change":"2024-04-17T09:59:04.438537+0000","last_active":"2024-04-17T09:59:04.444139+0000","last_peered":"2024-04-17T09:59:04.444139+0000","last_clean":"2024-04-17T09:59:04.444139+0000","last_became_active":"2024-04-17T09:59:04.438006+0000","last_became_peered":"2024-04-17T09:59:04.438006+0000","last_unstale":"2024-04-17T09:59:04.444139+0000","last_undegraded":"2024-04-17T09:59:04.444139+0000","last_fullsized":"2024-04-17T09:59:04.444139+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:15:09.312202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455087+0000","last_change":"2024-04-17T09:59:05.439118+0000","last_active":"2024-04-17T09:59:06.455087+0000","last_peered":"2024-04-17T09:59:06.455087+0000","last_clean":"2024-04-17T09:59:06.455087+0000","last_became_active":"2024-04-17T09:59:05.438852+0000","last_became_peered":"2024-04-17T09:59:05.438852+0000","last_unstale":"2024-04-17T09:59:06.455087+0000","last_undegraded":"2024-04-17T09:59:06.455087+0000","last_fullsized":"2024-04-17T09:59:06.455087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:13:00.757214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.444169+0000","last_change":"2024-04-17T09:59:04.438033+0000","last_active":"2024-04-17T09:59:04.444169+0000","last_peered":"2024-04-17T09:59:04.444169+0000","last_clean":"2024-04-17T09:59:04.444169+0000","last_became_active":"2024-04-17T09:59:04.437763+0000","last_became_peered":"2024-04-17T09:59:04.437763+0000","last_unstale":"2024-04-17T09:59:04.444169+0000","last_undegraded":"2024-04-17T09:59:04.444169+0000","last_fullsized":"2024-04-17T09:59:04.444169+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:03:17.824422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427265+0000","last_change":"2024-04-17T09:59:05.447894+0000","last_active":"2024-04-17T09:59:06.427265+0000","last_peered":"2024-04-17T09:59:06.427265+0000","last_clean":"2024-04-17T09:59:06.427265+0000","last_became_active":"2024-04-17T09:59:05.447595+0000","last_became_peered":"2024-04-17T09:59:05.447595+0000","last_unstale":"2024-04-17T09:59:06.427265+0000","last_undegraded":"2024-04-17T09:59:06.427265+0000","last_fullsized":"2024-04-17T09:59:06.427265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:11.377027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.451646+0000","last_change":"2024-04-17T09:59:04.448134+0000","last_active":"2024-04-17T09:59:04.451646+0000","last_peered":"2024-04-17T09:59:04.451646+0000","last_clean":"2024-04-17T09:59:04.451646+0000","last_became_active":"2024-04-17T09:59:04.447872+0000","last_became_peered":"2024-04-17T09:59:04.447872+0000","last_unstale":"2024-04-17T09:59:04.451646+0000","last_undegraded":"2024-04-17T09:59:04.451646+0000","last_fullsized":"2024-04-17T09:59:04.451646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:22:33.540833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455345+0000","last_change":"2024-04-17T09:59:05.439384+0000","last_active":"2024-04-17T09:59:06.455345+0000","last_peered":"2024-04-17T09:59:06.455345+0000","last_clean":"2024-04-17T09:59:06.455345+0000","last_became_active":"2024-04-17T09:59:05.439091+0000","last_became_peered":"2024-04-17T09:59:05.439091+0000","last_unstale":"2024-04-17T09:59:06.455345+0000","last_undegraded":"2024-04-17T09:59:06.455345+0000","last_fullsized":"2024-04-17T09:59:06.455345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:22:13.672459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.443120+0000","last_change":"2024-04-17T09:59:04.440489+0000","last_active":"2024-04-17T09:59:04.443120+0000","last_peered":"2024-04-17T09:59:04.443120+0000","last_clean":"2024-04-17T09:59:04.443120+0000","last_became_active":"2024-04-17T09:59:04.437665+0000","last_became_peered":"2024-04-17T09:59:04.437665+0000","last_unstale":"2024-04-17T09:59:04.443120+0000","last_undegraded":"2024-04-17T09:59:04.443120+0000","last_fullsized":"2024-04-17T09:59:04.443120+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:54:03.821994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433083+0000","last_change":"2024-04-17T09:59:04.433083+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433083+0000","last_undegraded":"2024-04-17T09:59:04.433083+0000","last_fullsized":"2024-04-17T09:59:04.433083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.446156+0000","last_change":"2024-04-17T09:59:04.441625+0000","last_active":"2024-04-17T09:59:04.446156+0000","last_peered":"2024-04-17T09:59:04.446156+0000","last_clean":"2024-04-17T09:59:04.446156+0000","last_became_active":"2024-04-17T09:59:04.441012+0000","last_became_peered":"2024-04-17T09:59:04.441012+0000","last_unstale":"2024-04-17T09:59:04.446156+0000","last_undegraded":"2024-04-17T09:59:04.446156+0000","last_fullsized":"2024-04-17T09:59:04.446156+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:31:12.043046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.426432+0000","last_change":"2024-04-17T09:59:04.426432+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.426432+0000","last_undegraded":"2024-04-17T09:59:04.426432+0000","last_fullsized":"2024-04-17T09:59:04.426432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.451592+0000","last_change":"2024-04-17T09:59:04.446837+0000","last_active":"2024-04-17T09:59:04.451592+0000","last_peered":"2024-04-17T09:59:04.451592+0000","last_clean":"2024-04-17T09:59:04.451592+0000","last_became_active":"2024-04-17T09:59:04.446469+0000","last_became_peered":"2024-04-17T09:59:04.446469+0000","last_unstale":"2024-04-17T09:59:04.451592+0000","last_undegraded":"2024-04-17T09:59:04.451592+0000","last_fullsized":"2024-04-17T09:59:04.451592+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:49.287601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.425532+0000","last_change":"2024-04-17T09:59:04.425532+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.425532+0000","last_undegraded":"2024-04-17T09:59:04.425532+0000","last_fullsized":"2024-04-17T09:59:04.425532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"81'8","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455171+0000","last_change":"2024-04-17T09:59:04.438156+0000","last_active":"2024-04-17T09:59:06.455171+0000","last_peered":"2024-04-17T09:59:06.455171+0000","last_clean":"2024-04-17T09:59:06.455171+0000","last_became_active":"2024-04-17T09:59:04.437821+0000","last_became_peered":"2024-04-17T09:59:04.437821+0000","last_unstale":"2024-04-17T09:59:06.455171+0000","last_undegraded":"2024-04-17T09:59:06.455171+0000","last_fullsized":"2024-04-17T09:59:06.455171+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:54.250176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.432091+0000","last_change":"2024-04-17T09:59:04.432091+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.432091+0000","last_undegraded":"2024-04-17T09:59:04.432091+0000","last_fullsized":"2024-04-17T09:59:04.432091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.448596+0000","last_change":"2024-04-17T09:59:04.447265+0000","last_active":"2024-04-17T09:59:04.448596+0000","last_peered":"2024-04-17T09:59:04.448596+0000","last_clean":"2024-04-17T09:59:04.448596+0000","last_became_active":"2024-04-17T09:59:04.446847+0000","last_became_peered":"2024-04-17T09:59:04.446847+0000","last_unstale":"2024-04-17T09:59:04.448596+0000","last_undegraded":"2024-04-17T09:59:04.448596+0000","last_fullsized":"2024-04-17T09:59:04.448596+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:49.287601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429393+0000","last_change":"2024-04-17T09:59:04.429393+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429393+0000","last_undegraded":"2024-04-17T09:59:04.429393+0000","last_fullsized":"2024-04-17T09:59:04.429393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455137+0000","last_change":"2024-04-17T09:59:04.443540+0000","last_active":"2024-04-17T09:59:06.455137+0000","last_peered":"2024-04-17T09:59:06.455137+0000","last_clean":"2024-04-17T09:59:06.455137+0000","last_became_active":"2024-04-17T09:59:04.443277+0000","last_became_peered":"2024-04-17T09:59:04.443277+0000","last_unstale":"2024-04-17T09:59:06.455137+0000","last_undegraded":"2024-04-17T09:59:06.455137+0000","last_fullsized":"2024-04-17T09:59:06.455137+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:10.371001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427822+0000","last_change":"2024-04-17T09:59:05.447239+0000","last_active":"2024-04-17T09:59:06.427822+0000","last_peered":"2024-04-17T09:59:06.427822+0000","last_clean":"2024-04-17T09:59:06.427822+0000","last_became_active":"2024-04-17T09:59:05.446970+0000","last_became_peered":"2024-04-17T09:59:05.446970+0000","last_unstale":"2024-04-17T09:59:06.427822+0000","last_undegraded":"2024-04-17T09:59:06.427822+0000","last_fullsized":"2024-04-17T09:59:06.427822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:02:07.703202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455269+0000","last_change":"2024-04-17T09:59:04.445958+0000","last_active":"2024-04-17T09:59:06.455269+0000","last_peered":"2024-04-17T09:59:06.455269+0000","last_clean":"2024-04-17T09:59:06.455269+0000","last_became_active":"2024-04-17T09:59:04.445632+0000","last_became_peered":"2024-04-17T09:59:04.445632+0000","last_unstale":"2024-04-17T09:59:06.455269+0000","last_undegraded":"2024-04-17T09:59:06.455269+0000","last_fullsized":"2024-04-17T09:59:06.455269+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:57.288434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.432936+0000","last_change":"2024-04-17T09:59:04.432936+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.432936+0000","last_undegraded":"2024-04-17T09:59:04.432936+0000","last_fullsized":"2024-04-17T09:59:04.432936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.448600+0000","last_change":"2024-04-17T09:59:04.445248+0000","last_active":"2024-04-17T09:59:04.448600+0000","last_peered":"2024-04-17T09:59:04.448600+0000","last_clean":"2024-04-17T09:59:04.448600+0000","last_became_active":"2024-04-17T09:59:04.444809+0000","last_became_peered":"2024-04-17T09:59:04.444809+0000","last_unstale":"2024-04-17T09:59:04.448600+0000","last_undegraded":"2024-04-17T09:59:04.448600+0000","last_fullsized":"2024-04-17T09:59:04.448600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:33:59.203901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.437546+0000","last_change":"2024-04-17T09:59:04.437546+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.437546+0000","last_undegraded":"2024-04-17T09:59:04.437546+0000","last_fullsized":"2024-04-17T09:59:04.437546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.451113+0000","last_change":"2024-04-17T09:59:04.448116+0000","last_active":"2024-04-17T09:59:04.451113+0000","last_peered":"2024-04-17T09:59:04.451113+0000","last_clean":"2024-04-17T09:59:04.451113+0000","last_became_active":"2024-04-17T09:59:04.446755+0000","last_became_peered":"2024-04-17T09:59:04.446755+0000","last_unstale":"2024-04-17T09:59:04.451113+0000","last_undegraded":"2024-04-17T09:59:04.451113+0000","last_fullsized":"2024-04-17T09:59:04.451113+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:04:29.400922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427614+0000","last_change":"2024-04-17T09:59:04.427614+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427614+0000","last_undegraded":"2024-04-17T09:59:04.427614+0000","last_fullsized":"2024-04-17T09:59:04.427614+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.451071+0000","last_change":"2024-04-17T09:59:04.448303+0000","last_active":"2024-04-17T09:59:04.451071+0000","last_peered":"2024-04-17T09:59:04.451071+0000","last_clean":"2024-04-17T09:59:04.451071+0000","last_became_active":"2024-04-17T09:59:04.446032+0000","last_became_peered":"2024-04-17T09:59:04.446032+0000","last_unstale":"2024-04-17T09:59:04.451071+0000","last_undegraded":"2024-04-17T09:59:04.451071+0000","last_fullsized":"2024-04-17T09:59:04.451071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:02:55.706845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.440407+0000","last_change":"2024-04-17T09:59:04.440407+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.440407+0000","last_undegraded":"2024-04-17T09:59:04.440407+0000","last_fullsized":"2024-04-17T09:59:04.440407+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.448496+0000","last_change":"2024-04-17T09:59:04.445983+0000","last_active":"2024-04-17T09:59:04.448496+0000","last_peered":"2024-04-17T09:59:04.448496+0000","last_clean":"2024-04-17T09:59:04.448496+0000","last_became_active":"2024-04-17T09:59:04.445671+0000","last_became_peered":"2024-04-17T09:59:04.445671+0000","last_unstale":"2024-04-17T09:59:04.448496+0000","last_undegraded":"2024-04-17T09:59:04.448496+0000","last_fullsized":"2024-04-17T09:59:04.448496+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:36.060764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.428075+0000","last_change":"2024-04-17T09:59:05.449697+0000","last_active":"2024-04-17T09:59:06.428075+0000","last_peered":"2024-04-17T09:59:06.428075+0000","last_clean":"2024-04-17T09:59:06.428075+0000","last_became_active":"2024-04-17T09:59:05.449390+0000","last_became_peered":"2024-04-17T09:59:05.449390+0000","last_unstale":"2024-04-17T09:59:06.428075+0000","last_undegraded":"2024-04-17T09:59:06.428075+0000","last_fullsized":"2024-04-17T09:59:06.428075+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:11:22.087912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.447272+0000","last_change":"2024-04-17T09:59:04.442441+0000","last_active":"2024-04-17T09:59:04.447272+0000","last_peered":"2024-04-17T09:59:04.447272+0000","last_clean":"2024-04-17T09:59:04.447272+0000","last_became_active":"2024-04-17T09:59:04.442051+0000","last_became_peered":"2024-04-17T09:59:04.442051+0000","last_unstale":"2024-04-17T09:59:04.447272+0000","last_undegraded":"2024-04-17T09:59:04.447272+0000","last_fullsized":"2024-04-17T09:59:04.447272+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:16:08.377113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433918+0000","last_change":"2024-04-17T09:59:04.433918+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433918+0000","last_undegraded":"2024-04-17T09:59:04.433918+0000","last_fullsized":"2024-04-17T09:59:04.433918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.446270+0000","last_change":"2024-04-17T09:59:04.441305+0000","last_active":"2024-04-17T09:59:04.446270+0000","last_peered":"2024-04-17T09:59:04.446270+0000","last_clean":"2024-04-17T09:59:04.446270+0000","last_became_active":"2024-04-17T09:59:04.441044+0000","last_became_peered":"2024-04-17T09:59:04.441044+0000","last_unstale":"2024-04-17T09:59:04.446270+0000","last_undegraded":"2024-04-17T09:59:04.446270+0000","last_fullsized":"2024-04-17T09:59:04.446270+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:06:39.663043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428308+0000","last_change":"2024-04-17T09:59:04.428308+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428308+0000","last_undegraded":"2024-04-17T09:59:04.428308+0000","last_fullsized":"2024-04-17T09:59:04.428308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.445184+0000","last_change":"2024-04-17T09:59:04.442494+0000","last_active":"2024-04-17T09:59:04.445184+0000","last_peered":"2024-04-17T09:59:04.445184+0000","last_clean":"2024-04-17T09:59:04.445184+0000","last_became_active":"2024-04-17T09:59:04.442254+0000","last_became_peered":"2024-04-17T09:59:04.442254+0000","last_unstale":"2024-04-17T09:59:04.445184+0000","last_undegraded":"2024-04-17T09:59:04.445184+0000","last_fullsized":"2024-04-17T09:59:04.445184+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:09:27.952101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427862+0000","last_change":"2024-04-17T09:59:05.438162+0000","last_active":"2024-04-17T09:59:06.427862+0000","last_peered":"2024-04-17T09:59:06.427862+0000","last_clean":"2024-04-17T09:59:06.427862+0000","last_became_active":"2024-04-17T09:59:05.437880+0000","last_became_peered":"2024-04-17T09:59:05.437880+0000","last_unstale":"2024-04-17T09:59:06.427862+0000","last_undegraded":"2024-04-17T09:59:06.427862+0000","last_fullsized":"2024-04-17T09:59:06.427862+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:58.294460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.445858+0000","last_change":"2024-04-17T09:59:04.444713+0000","last_active":"2024-04-17T09:59:04.445858+0000","last_peered":"2024-04-17T09:59:04.445858+0000","last_clean":"2024-04-17T09:59:04.445858+0000","last_became_active":"2024-04-17T09:59:04.444438+0000","last_became_peered":"2024-04-17T09:59:04.444438+0000","last_unstale":"2024-04-17T09:59:04.445858+0000","last_undegraded":"2024-04-17T09:59:04.445858+0000","last_fullsized":"2024-04-17T09:59:04.445858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:29:09.325067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.437197+0000","last_change":"2024-04-17T09:59:04.437197+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.437197+0000","last_undegraded":"2024-04-17T09:59:04.437197+0000","last_fullsized":"2024-04-17T09:59:04.437197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455124+0000","last_change":"2024-04-17T09:59:04.438627+0000","last_active":"2024-04-17T09:59:06.455124+0000","last_peered":"2024-04-17T09:59:06.455124+0000","last_clean":"2024-04-17T09:59:06.455124+0000","last_became_active":"2024-04-17T09:59:04.438100+0000","last_became_peered":"2024-04-17T09:59:04.438100+0000","last_unstale":"2024-04-17T09:59:06.455124+0000","last_undegraded":"2024-04-17T09:59:06.455124+0000","last_fullsized":"2024-04-17T09:59:06.455124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:53:55.006094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455088+0000","last_change":"2024-04-17T09:59:05.447417+0000","last_active":"2024-04-17T09:59:06.455088+0000","last_peered":"2024-04-17T09:59:06.455088+0000","last_clean":"2024-04-17T09:59:06.455088+0000","last_became_active":"2024-04-17T09:59:05.447176+0000","last_became_peered":"2024-04-17T09:59:05.447176+0000","last_unstale":"2024-04-17T09:59:06.455088+0000","last_undegraded":"2024-04-17T09:59:06.455088+0000","last_fullsized":"2024-04-17T09:59:06.455088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:18:04.685528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426694+0000","last_change":"2024-04-17T09:59:04.440315+0000","last_active":"2024-04-17T09:59:06.426694+0000","last_peered":"2024-04-17T09:59:06.426694+0000","last_clean":"2024-04-17T09:59:06.426694+0000","last_became_active":"2024-04-17T09:59:04.439614+0000","last_became_peered":"2024-04-17T09:59:04.439614+0000","last_unstale":"2024-04-17T09:59:06.426694+0000","last_undegraded":"2024-04-17T09:59:06.426694+0000","last_fullsized":"2024-04-17T09:59:06.426694+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:02.341478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.432875+0000","last_change":"2024-04-17T09:59:04.432875+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.432875+0000","last_undegraded":"2024-04-17T09:59:04.432875+0000","last_fullsized":"2024-04-17T09:59:04.432875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.445028+0000","last_change":"2024-04-17T09:59:04.442633+0000","last_active":"2024-04-17T09:59:04.445028+0000","last_peered":"2024-04-17T09:59:04.445028+0000","last_clean":"2024-04-17T09:59:04.445028+0000","last_became_active":"2024-04-17T09:59:04.442347+0000","last_became_peered":"2024-04-17T09:59:04.442347+0000","last_unstale":"2024-04-17T09:59:04.445028+0000","last_undegraded":"2024-04-17T09:59:04.445028+0000","last_fullsized":"2024-04-17T09:59:04.445028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:38:59.533078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427513+0000","last_change":"2024-04-17T09:59:04.427513+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427513+0000","last_undegraded":"2024-04-17T09:59:04.427513+0000","last_fullsized":"2024-04-17T09:59:04.427513+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426835+0000","last_change":"2024-04-17T09:59:04.440613+0000","last_active":"2024-04-17T09:59:06.426835+0000","last_peered":"2024-04-17T09:59:06.426835+0000","last_clean":"2024-04-17T09:59:06.426835+0000","last_became_active":"2024-04-17T09:59:04.440063+0000","last_became_peered":"2024-04-17T09:59:04.440063+0000","last_unstale":"2024-04-17T09:59:06.426835+0000","last_undegraded":"2024-04-17T09:59:06.426835+0000","last_fullsized":"2024-04-17T09:59:06.426835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:53:08.846256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.425466+0000","last_change":"2024-04-17T09:59:04.425466+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.425466+0000","last_undegraded":"2024-04-17T09:59:04.425466+0000","last_fullsized":"2024-04-17T09:59:04.425466+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.446679+0000","last_change":"2024-04-17T09:59:04.445356+0000","last_active":"2024-04-17T09:59:04.446679+0000","last_peered":"2024-04-17T09:59:04.446679+0000","last_clean":"2024-04-17T09:59:04.446679+0000","last_became_active":"2024-04-17T09:59:04.445010+0000","last_became_peered":"2024-04-17T09:59:04.445010+0000","last_unstale":"2024-04-17T09:59:04.446679+0000","last_undegraded":"2024-04-17T09:59:04.446679+0000","last_fullsized":"2024-04-17T09:59:04.446679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:32:37.586889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":280,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.434933+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T09:59:04.434933+0000","last_peered":"2024-04-17T09:59:04.434933+0000","last_clean":"2024-04-17T09:59:04.434933+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T09:59:04.434933+0000","last_undegraded":"2024-04-17T09:59:04.434933+0000","last_fullsized":"2024-04-17T09:59:04.434933+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509516,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1568768,"data_stored":1002078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":411710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509516,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":962560,"data_stored":405435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509516,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":411710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":411710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509516,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":411710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542221,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":424785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542221,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36900,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334940,"statfs":{"total":96636764160,"available":96598978560,"internally_reserved":0,"allocated":1597440,"data_stored":1015187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T09:59:10.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3f is not active+clean 2024-04-17T09:59:10.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.430338+0000', 'last_change': '2024-04-17T09:59:04.430338+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.430338+0000', 'last_undegraded': '2024-04-17T09:59:04.430338+0000', 'last_fullsized': '2024-04-17T09:59:04.430338+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T09:59:10.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3e is not active+clean 2024-04-17T09:59:10.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.429509+0000', 'last_change': '2024-04-17T09:59:04.429509+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.429509+0000', 'last_undegraded': '2024-04-17T09:59:04.429509+0000', 'last_fullsized': '2024-04-17T09:59:04.429509+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T09:59:10.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3d is not active+clean 2024-04-17T09:59:10.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.433330+0000', 'last_change': '2024-04-17T09:59:04.433330+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.433330+0000', 'last_undegraded': '2024-04-17T09:59:04.433330+0000', 'last_fullsized': '2024-04-17T09:59:04.433330+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T09:59:10.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3c is not active+clean 2024-04-17T09:59:10.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.429769+0000', 'last_change': '2024-04-17T09:59:04.429769+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.429769+0000', 'last_undegraded': '2024-04-17T09:59:04.429769+0000', 'last_fullsized': '2024-04-17T09:59:04.429769+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T09:59:10.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3a is not active+clean 2024-04-17T09:59:10.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.438674+0000', 'last_change': '2024-04-17T09:59:04.438674+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.438674+0000', 'last_undegraded': '2024-04-17T09:59:04.438674+0000', 'last_fullsized': '2024-04-17T09:59:04.438674+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T09:59:10.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.39 is not active+clean 2024-04-17T09:59:10.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.429372+0000', 'last_change': '2024-04-17T09:59:04.429372+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.429372+0000', 'last_undegraded': '2024-04-17T09:59:04.429372+0000', 'last_fullsized': '2024-04-17T09:59:04.429372+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T09:59:10.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.38 is not active+clean 2024-04-17T09:59:10.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.437249+0000', 'last_change': '2024-04-17T09:59:04.437249+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.437249+0000', 'last_undegraded': '2024-04-17T09:59:04.437249+0000', 'last_fullsized': '2024-04-17T09:59:04.437249+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T09:59:10.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.37 is not active+clean 2024-04-17T09:59:10.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.440846+0000', 'last_change': '2024-04-17T09:59:04.440846+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.440846+0000', 'last_undegraded': '2024-04-17T09:59:04.440846+0000', 'last_fullsized': '2024-04-17T09:59:04.440846+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T09:59:10.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.36 is not active+clean 2024-04-17T09:59:10.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.427452+0000', 'last_change': '2024-04-17T09:59:04.427452+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.427452+0000', 'last_undegraded': '2024-04-17T09:59:04.427452+0000', 'last_fullsized': '2024-04-17T09:59:04.427452+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T09:59:10.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.35 is not active+clean 2024-04-17T09:59:10.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.427477+0000', 'last_change': '2024-04-17T09:59:04.427477+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.427477+0000', 'last_undegraded': '2024-04-17T09:59:04.427477+0000', 'last_fullsized': '2024-04-17T09:59:04.427477+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T09:59:10.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.34 is not active+clean 2024-04-17T09:59:10.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.427835+0000', 'last_change': '2024-04-17T09:59:04.427835+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.427835+0000', 'last_undegraded': '2024-04-17T09:59:04.427835+0000', 'last_fullsized': '2024-04-17T09:59:04.427835+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T09:59:10.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.33 is not active+clean 2024-04-17T09:59:10.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.433600+0000', 'last_change': '2024-04-17T09:59:04.433600+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.433600+0000', 'last_undegraded': '2024-04-17T09:59:04.433600+0000', 'last_fullsized': '2024-04-17T09:59:04.433600+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T09:59:10.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.32 is not active+clean 2024-04-17T09:59:10.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.435489+0000', 'last_change': '2024-04-17T09:59:04.435489+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.435489+0000', 'last_undegraded': '2024-04-17T09:59:04.435489+0000', 'last_fullsized': '2024-04-17T09:59:04.435489+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T09:59:10.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.31 is not active+clean 2024-04-17T09:59:10.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.427347+0000', 'last_change': '2024-04-17T09:59:04.427347+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.427347+0000', 'last_undegraded': '2024-04-17T09:59:04.427347+0000', 'last_fullsized': '2024-04-17T09:59:04.427347+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T09:59:10.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2e is not active+clean 2024-04-17T09:59:10.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.432446+0000', 'last_change': '2024-04-17T09:59:04.432446+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.432446+0000', 'last_undegraded': '2024-04-17T09:59:04.432446+0000', 'last_fullsized': '2024-04-17T09:59:04.432446+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T09:59:10.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2c is not active+clean 2024-04-17T09:59:10.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.429260+0000', 'last_change': '2024-04-17T09:59:04.429260+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.429260+0000', 'last_undegraded': '2024-04-17T09:59:04.429260+0000', 'last_fullsized': '2024-04-17T09:59:04.429260+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T09:59:10.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2b is not active+clean 2024-04-17T09:59:10.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.429463+0000', 'last_change': '2024-04-17T09:59:04.429463+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.429463+0000', 'last_undegraded': '2024-04-17T09:59:04.429463+0000', 'last_fullsized': '2024-04-17T09:59:04.429463+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T09:59:10.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2a is not active+clean 2024-04-17T09:59:10.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.428078+0000', 'last_change': '2024-04-17T09:59:04.428078+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.428078+0000', 'last_undegraded': '2024-04-17T09:59:04.428078+0000', 'last_fullsized': '2024-04-17T09:59:04.428078+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T09:59:10.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.29 is not active+clean 2024-04-17T09:59:10.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.427009+0000', 'last_change': '2024-04-17T09:59:04.427009+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.427009+0000', 'last_undegraded': '2024-04-17T09:59:04.427009+0000', 'last_fullsized': '2024-04-17T09:59:04.427009+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T09:59:10.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.28 is not active+clean 2024-04-17T09:59:10.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.428239+0000', 'last_change': '2024-04-17T09:59:04.428239+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.428239+0000', 'last_undegraded': '2024-04-17T09:59:04.428239+0000', 'last_fullsized': '2024-04-17T09:59:04.428239+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T09:59:10.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.27 is not active+clean 2024-04-17T09:59:10.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.425424+0000', 'last_change': '2024-04-17T09:59:04.425424+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.425424+0000', 'last_undegraded': '2024-04-17T09:59:04.425424+0000', 'last_fullsized': '2024-04-17T09:59:04.425424+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T09:59:10.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.26 is not active+clean 2024-04-17T09:59:10.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.432662+0000', 'last_change': '2024-04-17T09:59:04.432662+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.432662+0000', 'last_undegraded': '2024-04-17T09:59:04.432662+0000', 'last_fullsized': '2024-04-17T09:59:04.432662+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T09:59:10.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.24 is not active+clean 2024-04-17T09:59:10.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.429291+0000', 'last_change': '2024-04-17T09:59:04.429291+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.429291+0000', 'last_undegraded': '2024-04-17T09:59:04.429291+0000', 'last_fullsized': '2024-04-17T09:59:04.429291+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T09:59:10.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.23 is not active+clean 2024-04-17T09:59:10.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.427246+0000', 'last_change': '2024-04-17T09:59:04.427246+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.427246+0000', 'last_undegraded': '2024-04-17T09:59:04.427246+0000', 'last_fullsized': '2024-04-17T09:59:04.427246+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T09:59:10.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.22 is not active+clean 2024-04-17T09:59:10.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.426566+0000', 'last_change': '2024-04-17T09:59:04.426566+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.426566+0000', 'last_undegraded': '2024-04-17T09:59:04.426566+0000', 'last_fullsized': '2024-04-17T09:59:04.426566+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T09:59:10.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.20 is not active+clean 2024-04-17T09:59:10.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.428134+0000', 'last_change': '2024-04-17T09:59:04.428134+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.428134+0000', 'last_undegraded': '2024-04-17T09:59:04.428134+0000', 'last_fullsized': '2024-04-17T09:59:04.428134+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T09:59:10.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1f is not active+clean 2024-04-17T09:59:10.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.429909+0000', 'last_change': '2024-04-17T09:59:04.429909+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.429909+0000', 'last_undegraded': '2024-04-17T09:59:04.429909+0000', 'last_fullsized': '2024-04-17T09:59:04.429909+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T09:59:10.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1c is not active+clean 2024-04-17T09:59:10.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.428740+0000', 'last_change': '2024-04-17T09:59:04.428740+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.428740+0000', 'last_undegraded': '2024-04-17T09:59:04.428740+0000', 'last_fullsized': '2024-04-17T09:59:04.428740+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T09:59:10.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1d is not active+clean 2024-04-17T09:59:10.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.433823+0000', 'last_change': '2024-04-17T09:59:04.433823+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.433823+0000', 'last_undegraded': '2024-04-17T09:59:04.433823+0000', 'last_fullsized': '2024-04-17T09:59:04.433823+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T09:59:10.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1a is not active+clean 2024-04-17T09:59:10.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.425419+0000', 'last_change': '2024-04-17T09:59:04.425419+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.425419+0000', 'last_undegraded': '2024-04-17T09:59:04.425419+0000', 'last_fullsized': '2024-04-17T09:59:04.425419+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T09:59:10.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.18 is not active+clean 2024-04-17T09:59:10.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.433375+0000', 'last_change': '2024-04-17T09:59:04.433375+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.433375+0000', 'last_undegraded': '2024-04-17T09:59:04.433375+0000', 'last_fullsized': '2024-04-17T09:59:04.433375+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T09:59:10.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.19 is not active+clean 2024-04-17T09:59:10.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.426818+0000', 'last_change': '2024-04-17T09:59:04.426818+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.426818+0000', 'last_undegraded': '2024-04-17T09:59:04.426818+0000', 'last_fullsized': '2024-04-17T09:59:04.426818+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T09:59:10.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.16 is not active+clean 2024-04-17T09:59:10.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.428201+0000', 'last_change': '2024-04-17T09:59:04.428201+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.428201+0000', 'last_undegraded': '2024-04-17T09:59:04.428201+0000', 'last_fullsized': '2024-04-17T09:59:04.428201+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-17T09:59:10.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.17 is not active+clean 2024-04-17T09:59:10.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.440390+0000', 'last_change': '2024-04-17T09:59:04.440390+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.440390+0000', 'last_undegraded': '2024-04-17T09:59:04.440390+0000', 'last_fullsized': '2024-04-17T09:59:04.440390+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T09:59:10.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.13 is not active+clean 2024-04-17T09:59:10.357 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.433083+0000', 'last_change': '2024-04-17T09:59:04.433083+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.433083+0000', 'last_undegraded': '2024-04-17T09:59:04.433083+0000', 'last_fullsized': '2024-04-17T09:59:04.433083+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T09:59:10.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.10 is not active+clean 2024-04-17T09:59:10.357 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.426432+0000', 'last_change': '2024-04-17T09:59:04.426432+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.426432+0000', 'last_undegraded': '2024-04-17T09:59:04.426432+0000', 'last_fullsized': '2024-04-17T09:59:04.426432+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T09:59:10.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.11 is not active+clean 2024-04-17T09:59:10.357 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.425532+0000', 'last_change': '2024-04-17T09:59:04.425532+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.425532+0000', 'last_undegraded': '2024-04-17T09:59:04.425532+0000', 'last_fullsized': '2024-04-17T09:59:04.425532+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T09:59:10.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.e is not active+clean 2024-04-17T09:59:10.357 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.432091+0000', 'last_change': '2024-04-17T09:59:04.432091+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.432091+0000', 'last_undegraded': '2024-04-17T09:59:04.432091+0000', 'last_fullsized': '2024-04-17T09:59:04.432091+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T09:59:10.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.f is not active+clean 2024-04-17T09:59:10.357 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.429393+0000', 'last_change': '2024-04-17T09:59:04.429393+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.429393+0000', 'last_undegraded': '2024-04-17T09:59:04.429393+0000', 'last_fullsized': '2024-04-17T09:59:04.429393+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T09:59:10.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.a is not active+clean 2024-04-17T09:59:10.357 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.432936+0000', 'last_change': '2024-04-17T09:59:04.432936+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.432936+0000', 'last_undegraded': '2024-04-17T09:59:04.432936+0000', 'last_fullsized': '2024-04-17T09:59:04.432936+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T09:59:10.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.b is not active+clean 2024-04-17T09:59:10.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.437546+0000', 'last_change': '2024-04-17T09:59:04.437546+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.437546+0000', 'last_undegraded': '2024-04-17T09:59:04.437546+0000', 'last_fullsized': '2024-04-17T09:59:04.437546+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T09:59:10.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.8 is not active+clean 2024-04-17T09:59:10.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.427614+0000', 'last_change': '2024-04-17T09:59:04.427614+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.427614+0000', 'last_undegraded': '2024-04-17T09:59:04.427614+0000', 'last_fullsized': '2024-04-17T09:59:04.427614+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-17T09:59:10.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.9 is not active+clean 2024-04-17T09:59:10.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.440407+0000', 'last_change': '2024-04-17T09:59:04.440407+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.440407+0000', 'last_undegraded': '2024-04-17T09:59:04.440407+0000', 'last_fullsized': '2024-04-17T09:59:04.440407+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T09:59:10.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.7 is not active+clean 2024-04-17T09:59:10.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.433918+0000', 'last_change': '2024-04-17T09:59:04.433918+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.433918+0000', 'last_undegraded': '2024-04-17T09:59:04.433918+0000', 'last_fullsized': '2024-04-17T09:59:04.433918+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T09:59:10.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.4 is not active+clean 2024-04-17T09:59:10.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.428308+0000', 'last_change': '2024-04-17T09:59:04.428308+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.428308+0000', 'last_undegraded': '2024-04-17T09:59:04.428308+0000', 'last_fullsized': '2024-04-17T09:59:04.428308+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T09:59:10.359 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2 is not active+clean 2024-04-17T09:59:10.359 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.437197+0000', 'last_change': '2024-04-17T09:59:04.437197+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.437197+0000', 'last_undegraded': '2024-04-17T09:59:04.437197+0000', 'last_fullsized': '2024-04-17T09:59:04.437197+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-17T09:59:10.359 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.0 is not active+clean 2024-04-17T09:59:10.359 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.432875+0000', 'last_change': '2024-04-17T09:59:04.432875+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.432875+0000', 'last_undegraded': '2024-04-17T09:59:04.432875+0000', 'last_fullsized': '2024-04-17T09:59:04.432875+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-17T09:59:10.359 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1 is not active+clean 2024-04-17T09:59:10.359 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.427513+0000', 'last_change': '2024-04-17T09:59:04.427513+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.427513+0000', 'last_undegraded': '2024-04-17T09:59:04.427513+0000', 'last_fullsized': '2024-04-17T09:59:04.427513+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-17T09:59:10.359 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.c is not active+clean 2024-04-17T09:59:10.359 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-17T09:59:04.425466+0000', 'last_change': '2024-04-17T09:59:04.425466+0000', 'last_active': '2024-04-17T09:59:04.411847+0000', 'last_peered': '2024-04-17T09:59:04.411847+0000', 'last_clean': '2024-04-17T09:59:04.411847+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-17T09:59:04.425466+0000', 'last_undegraded': '2024-04-17T09:59:04.425466+0000', 'last_fullsized': '2024-04-17T09:59:04.425466+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'last_clean_scrub_stamp': '2024-04-17T09:59:04.411847+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-17T09:59:10.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T09:59:10.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.534+0000 7fbb1c600640 1 Processor -- start 2024-04-17T09:59:10.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.534+0000 7fbb1c600640 1 -- start start 2024-04-17T09:59:10.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.534+0000 7fbb1c600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbb14072500 0x7fbb140cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:10.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb1c600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb140cdac0 con 0x7fbb1415d8d0 2024-04-17T09:59:10.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb1c600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb1415cdf0 con 0x7fbb14145f30 2024-04-17T09:59:10.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb1c600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fbb1406f400 con 0x7fbb14072500 2024-04-17T09:59:10.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb1a600640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbb1415d8d0 0x7fbb141601d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52048/0 (socket says 172.21.15.32:52048) 2024-04-17T09:59:10.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb1a600640 1 -- 172.21.15.32:0/1598198048 learned_addr learned my addr 172.21.15.32:0/1598198048 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:10.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb19200640 1 --2- 172.21.15.32:0/1598198048 >> v2:172.21.15.144:3300/0 conn(0x7fbb14072500 0x7fbb140cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:10.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb13e00640 1 -- 172.21.15.32:0/1598198048 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 733407340 0 0) 0x7fbb140cdac0 con 0x7fbb1415d8d0 2024-04-17T09:59:10.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb13e00640 1 -- 172.21.15.32:0/1598198048 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbafc003530 con 0x7fbb1415d8d0 2024-04-17T09:59:10.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb13e00640 1 -- 172.21.15.32:0/1598198048 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2690345828 0 0) 0x7fbafc003530 con 0x7fbb1415d8d0 2024-04-17T09:59:10.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb13e00640 1 -- 172.21.15.32:0/1598198048 >> v2:172.21.15.144:3300/0 conn(0x7fbb14072500 msgr2=0x7fbb140cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:10.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb13e00640 1 --2- 172.21.15.32:0/1598198048 >> v2:172.21.15.144:3300/0 conn(0x7fbb14072500 0x7fbb140cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:10.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb13e00640 1 -- 172.21.15.32:0/1598198048 >> v1:172.21.15.144:6789/0 conn(0x7fbb14145f30 legacy=0x7fbb14071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:10.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb13e00640 1 -- 172.21.15.32:0/1598198048 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1415d0d0 con 0x7fbb1415d8d0 2024-04-17T09:59:10.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb13e00640 1 -- 172.21.15.32:0/1598198048 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7fbb0c001a00 con 0x7fbb1415d8d0 2024-04-17T09:59:10.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb13e00640 1 -- 172.21.15.32:0/1598198048 >> v1:172.21.15.32:6789/0 conn(0x7fbb1415d8d0 legacy=0x7fbb141601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:10.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb13e00640 1 --2- 172.21.15.32:0/1598198048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbafc0039c0 0x7fbafc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:10.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb13e00640 1 --2- 172.21.15.32:0/1598198048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbafc004810 0x7fbafc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:10.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb13e00640 1 --2- 172.21.15.32:0/1598198048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbafc0051d0 0x7fbafc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:10.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb13e00640 1 -- 172.21.15.32:0/1598198048 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1406f400 con 0x7fbafc004810 2024-04-17T09:59:10.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb13e00640 1 -- 172.21.15.32:0/1598198048 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc00c080 con 0x7fbafc0051d0 2024-04-17T09:59:10.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb13e00640 1 -- 172.21.15.32:0/1598198048 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbafc00c280 con 0x7fbafc0039c0 2024-04-17T09:59:10.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb19200640 1 --2- 172.21.15.32:0/1598198048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbafc004810 0x7fbafc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:10.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb19c00640 1 --2- 172.21.15.32:0/1598198048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbafc0039c0 0x7fbafc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:10.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb1a600640 1 --2- 172.21.15.32:0/1598198048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbafc0051d0 0x7fbafc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:10.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb19200640 1 -- 172.21.15.32:0/1598198048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbafc0039c0 msgr2=0x7fbafc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:10.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb19200640 1 --2- 172.21.15.32:0/1598198048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbafc0039c0 0x7fbafc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:10.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb19200640 1 -- 172.21.15.32:0/1598198048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbafc0051d0 msgr2=0x7fbafc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:10.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb19200640 1 --2- 172.21.15.32:0/1598198048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbafc0051d0 0x7fbafc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:10.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb19200640 1 -- 172.21.15.32:0/1598198048 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbafc003530 con 0x7fbafc004810 2024-04-17T09:59:10.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb1a600640 1 --2- 172.21.15.32:0/1598198048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbafc0051d0 0x7fbafc00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:10.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.538+0000 7fbb1a600640 1 -- 172.21.15.32:0/1598198048 reap_dead start 2024-04-17T09:59:10.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.542+0000 7fbb19200640 1 --2- 172.21.15.32:0/1598198048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbafc004810 0x7fbafc004c90 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fbb00009ae0 tx=0x7fbb0001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617732942b58cc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:10.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.542+0000 7fbb13e00640 1 -- 172.21.15.32:0/1598198048 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb000b2070 con 0x7fbafc004810 2024-04-17T09:59:10.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.542+0000 7fbb13e00640 1 -- 172.21.15.32:0/1598198048 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb000a5660 con 0x7fbafc004810 2024-04-17T09:59:10.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.542+0000 7fbb13e00640 1 -- 172.21.15.32:0/1598198048 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb000096b0 con 0x7fbafc004810 2024-04-17T09:59:10.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.542+0000 7fbb1c600640 1 -- 172.21.15.32:0/1598198048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbafc004810 msgr2=0x7fbafc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:10.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.542+0000 7fbb1c600640 1 --2- 172.21.15.32:0/1598198048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbafc004810 0x7fbafc004c90 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fbb00009ae0 tx=0x7fbb0001ca70 comp rx=0 tx=0).stop 2024-04-17T09:59:10.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.542+0000 7fbb1c600640 1 -- 172.21.15.32:0/1598198048 shutdown_connections 2024-04-17T09:59:10.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.542+0000 7fbb1c600640 1 --2- 172.21.15.32:0/1598198048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbafc004810 0x7fbafc004c90 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:10.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.542+0000 7fbb1c600640 1 -- 172.21.15.32:0/1598198048 >> 172.21.15.32:0/1598198048 conn(0x7fbb1406ba30 msgr2=0x7fbb1406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:10.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.542+0000 7fbb1c600640 1 -- 172.21.15.32:0/1598198048 shutdown_connections 2024-04-17T09:59:10.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.542+0000 7fbb1c600640 1 -- 172.21.15.32:0/1598198048 wait complete. 2024-04-17T09:59:10.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.542+0000 7fbb1c600640 1 Processor -- start 2024-04-17T09:59:10.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.542+0000 7fbb1c600640 1 -- start start 2024-04-17T09:59:10.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb1c600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbafc004810 0x7fbb1415bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:10.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb1c600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb14130590 0x7fbb141423b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:10.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb1c600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb140b4670 0x7fbb140b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:10.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb1c600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbb14072180 con 0x7fbb140b4670 2024-04-17T09:59:10.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb1c600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbb14072000 con 0x7fbb14130590 2024-04-17T09:59:10.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb1c600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbb14072300 con 0x7fbafc004810 2024-04-17T09:59:10.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb1a600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb140b4670 0x7fbb140b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:10.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb1a600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb140b4670 0x7fbb140b4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38644/0 (socket says 172.21.15.32:38644) 2024-04-17T09:59:10.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb1a600640 1 -- 172.21.15.32:0/3653462236 learned_addr learned my addr 172.21.15.32:0/3653462236 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:10.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb19200640 1 --2- 172.21.15.32:0/3653462236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb14130590 0x7fbb141423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:10.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb19c00640 1 --2- 172.21.15.32:0/3653462236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbafc004810 0x7fbb1415bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:10.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb1a600640 1 -- 172.21.15.32:0/3653462236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbafc004810 msgr2=0x7fbb1415bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:10.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb1a600640 1 --2- 172.21.15.32:0/3653462236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbafc004810 0x7fbb1415bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:10.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb1a600640 1 -- 172.21.15.32:0/3653462236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb14130590 msgr2=0x7fbb141423b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:10.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb1a600640 1 --2- 172.21.15.32:0/3653462236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb14130590 0x7fbb141423b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:10.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb1a600640 1 -- 172.21.15.32:0/3653462236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1415c310 con 0x7fbb140b4670 2024-04-17T09:59:10.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb1a600640 1 --2- 172.21.15.32:0/3653462236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb140b4670 0x7fbb140b4af0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c000f90 tx=0x7fbb0c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:10.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb11600640 1 -- 172.21.15.32:0/3653462236 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c00eb90 con 0x7fbb140b4670 2024-04-17T09:59:10.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb11600640 1 -- 172.21.15.32:0/3653462236 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb0c018070 con 0x7fbb140b4670 2024-04-17T09:59:10.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb11600640 1 -- 172.21.15.32:0/3653462236 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c013420 con 0x7fbb140b4670 2024-04-17T09:59:10.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb1c600640 1 -- 172.21.15.32:0/3653462236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb14142c00 con 0x7fbb140b4670 2024-04-17T09:59:10.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb1c600640 1 -- 172.21.15.32:0/3653462236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb141431c0 con 0x7fbb140b4670 2024-04-17T09:59:10.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb1c600640 1 -- 172.21.15.32:0/3653462236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb14068a90 con 0x7fbb140b4670 2024-04-17T09:59:10.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.546+0000 7fbb11600640 1 -- 172.21.15.32:0/3653462236 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbb0c0135c0 con 0x7fbb140b4670 2024-04-17T09:59:10.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.550+0000 7fbb11600640 1 --2- 172.21.15.32:0/3653462236 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbaf4068380 0x7fbaf406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:10.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.550+0000 7fbb11600640 1 -- 172.21.15.32:0/3653462236 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fbb0c086f20 con 0x7fbb140b4670 2024-04-17T09:59:10.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.550+0000 7fbb19c00640 1 --2- 172.21.15.32:0/3653462236 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbaf4068380 0x7fbaf406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:10.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.550+0000 7fbb19c00640 1 --2- 172.21.15.32:0/3653462236 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbaf4068380 0x7fbaf406a840 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fbb08005510 tx=0x7fbb08009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:10.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.554+0000 7fbb11600640 1 -- 172.21.15.32:0/3653462236 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fbb0c0573b0 con 0x7fbb140b4670 2024-04-17T09:59:10.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.778+0000 7fbb1c600640 1 -- 172.21.15.32:0/3653462236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fbb14072f90 con 0x7fbb140b4670 2024-04-17T09:59:10.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.778+0000 7fbb11600640 1 -- 172.21.15.32:0/3653462236 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1096 (secure 0 0 0) 0x7fbb0c056cb0 con 0x7fbb140b4670 2024-04-17T09:59:10.777 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:59:10.777 INFO:teuthology.orchestra.run.smithi032.stdout:{"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":382,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":82,"num_osds":8,"num_up_osds":8,"osd_up_since":1713347579,"num_in_osds":8,"osd_in_since":1713347570,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":49},{"state_name":"active+clean","count":48}],"num_pgs":97,"num_pools":3,"num_objects":9,"data_bytes":590368,"bytes_used":264478720,"bytes_avail":772829634560,"bytes_total":773094113280,"inactive_pgs_ratio":0.50515460968017578,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":3},"fsmap":{"epoch":59,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7557}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-17T09:58:53.362032+0000","services":{}},"progress_events":{}} 2024-04-17T09:59:10.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.782+0000 7fbb1c600640 1 -- 172.21.15.32:0/3653462236 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbaf4068380 msgr2=0x7fbaf406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:10.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.782+0000 7fbb1c600640 1 --2- 172.21.15.32:0/3653462236 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbaf4068380 0x7fbaf406a840 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fbb08005510 tx=0x7fbb08009040 comp rx=0 tx=0).stop 2024-04-17T09:59:10.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.782+0000 7fbb1c600640 1 -- 172.21.15.32:0/3653462236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb140b4670 msgr2=0x7fbb140b4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:10.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.782+0000 7fbb1c600640 1 --2- 172.21.15.32:0/3653462236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb140b4670 0x7fbb140b4af0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c000f90 tx=0x7fbb0c00e530 comp rx=0 tx=0).stop 2024-04-17T09:59:10.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.782+0000 7fbb1c600640 1 -- 172.21.15.32:0/3653462236 shutdown_connections 2024-04-17T09:59:10.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.782+0000 7fbb1c600640 1 --2- 172.21.15.32:0/3653462236 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbaf4068380 0x7fbaf406a840 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:10.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.782+0000 7fbb1c600640 1 --2- 172.21.15.32:0/3653462236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb140b4670 0x7fbb140b4af0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:10.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.782+0000 7fbb1c600640 1 --2- 172.21.15.32:0/3653462236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb14130590 0x7fbb141423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:10.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.782+0000 7fbb1c600640 1 --2- 172.21.15.32:0/3653462236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbafc004810 0x7fbb1415bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:10.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.782+0000 7fbb1c600640 1 -- 172.21.15.32:0/3653462236 >> 172.21.15.32:0/3653462236 conn(0x7fbb1406ba30 msgr2=0x7fbb1415ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:10.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.782+0000 7fbb1c600640 1 -- 172.21.15.32:0/3653462236 shutdown_connections 2024-04-17T09:59:10.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.782+0000 7fbb1c600640 1 -- 172.21.15.32:0/3653462236 wait complete. 2024-04-17T09:59:10.799 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-17T09:59:10.799 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T09:59:10.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.942+0000 7ff1d0c00640 1 Processor -- start 2024-04-17T09:59:10.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.942+0000 7ff1d0c00640 1 -- start start 2024-04-17T09:59:10.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.942+0000 7ff1d0c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1cc0cea30 0x7ff1cc0cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:10.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.942+0000 7ff1d0c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff1cc073600 0x7ff1cc15f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:10.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.942+0000 7ff1d0c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1cc06cfa0 con 0x7ff1cc0cd190 2024-04-17T09:59:10.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.942+0000 7ff1d0c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff1cc0b4dd0 con 0x7ff1cc073600 2024-04-17T09:59:10.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.942+0000 7ff1d0c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff1cc0b5970 con 0x7ff1cc0cea30 2024-04-17T09:59:10.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.942+0000 7ff1ca800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff1cc0cd190 0x7ff1cc15fd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52064/0 (socket says 172.21.15.32:52064) 2024-04-17T09:59:10.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.942+0000 7ff1ca800640 1 -- 172.21.15.32:0/313993894 learned_addr learned my addr 172.21.15.32:0/313993894 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:10.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.942+0000 7ff1c1400640 1 --2- 172.21.15.32:0/313993894 >> v2:172.21.15.32:3300/0 conn(0x7ff1cc073600 0x7ff1cc15f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:10.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.942+0000 7ff1c9e00640 1 --2- 172.21.15.32:0/313993894 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1cc0cea30 0x7ff1cc0cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:10.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.942+0000 7ff1c1400640 1 -- 172.21.15.32:0/313993894 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1cc0cea30 msgr2=0x7ff1cc0cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:10.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.942+0000 7ff1c1400640 1 --2- 172.21.15.32:0/313993894 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1cc0cea30 0x7ff1cc0cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:10.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.942+0000 7ff1c1400640 1 -- 172.21.15.32:0/313993894 >> v1:172.21.15.32:6789/0 conn(0x7ff1cc0cd190 legacy=0x7ff1cc15fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:10.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.942+0000 7ff1c1400640 1 -- 172.21.15.32:0/313993894 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1cc06d210 con 0x7ff1cc073600 2024-04-17T09:59:10.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.946+0000 7ff1c1400640 1 --2- 172.21.15.32:0/313993894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1cc073600 0x7ff1cc15f7d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7ff1b40097c0 tx=0x7ff1b401ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870e32334dccf1d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:10.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.946+0000 7ff1c9400640 1 -- 172.21.15.32:0/313993894 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b40b5070 con 0x7ff1cc073600 2024-04-17T09:59:10.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.946+0000 7ff1c9400640 1 -- 172.21.15.32:0/313993894 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b40a8990 con 0x7ff1cc073600 2024-04-17T09:59:10.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.946+0000 7ff1c9400640 1 -- 172.21.15.32:0/313993894 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b40a8d60 con 0x7ff1cc073600 2024-04-17T09:59:10.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.946+0000 7ff1d0c00640 1 -- 172.21.15.32:0/313993894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1cc073600 msgr2=0x7ff1cc15f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:10.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.946+0000 7ff1d0c00640 1 --2- 172.21.15.32:0/313993894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1cc073600 0x7ff1cc15f7d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7ff1b40097c0 tx=0x7ff1b401ec90 comp rx=0 tx=0).stop 2024-04-17T09:59:10.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.946+0000 7ff1d0c00640 1 -- 172.21.15.32:0/313993894 shutdown_connections 2024-04-17T09:59:10.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.946+0000 7ff1d0c00640 1 --2- 172.21.15.32:0/313993894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1cc073600 0x7ff1cc15f7d0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:10.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.946+0000 7ff1d0c00640 1 --2- 172.21.15.32:0/313993894 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1cc0cea30 0x7ff1cc0cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:10.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.946+0000 7ff1d0c00640 1 -- 172.21.15.32:0/313993894 >> 172.21.15.32:0/313993894 conn(0x7ff1cc0d0c00 msgr2=0x7ff1cc0d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:10.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.946+0000 7ff1d0c00640 1 -- 172.21.15.32:0/313993894 shutdown_connections 2024-04-17T09:59:10.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.946+0000 7ff1d0c00640 1 -- 172.21.15.32:0/313993894 wait complete. 2024-04-17T09:59:10.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1d0c00640 1 Processor -- start 2024-04-17T09:59:10.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1d0c00640 1 -- start start 2024-04-17T09:59:10.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1d0c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1cc073600 0x7ff1cc1560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:10.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1d0c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1cc0cd190 0x7ff1cc0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:10.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1d0c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1cc0cea30 0x7ff1cc0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:10.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1d0c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc0cf6c0 con 0x7ff1cc0cea30 2024-04-17T09:59:10.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1d0c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc0cf350 con 0x7ff1cc073600 2024-04-17T09:59:10.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1d0c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff1cc0cf840 con 0x7ff1cc0cd190 2024-04-17T09:59:10.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1ca800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1cc0cea30 0x7ff1cc0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:10.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1ca800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1cc0cea30 0x7ff1cc0cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38668/0 (socket says 172.21.15.32:38668) 2024-04-17T09:59:10.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1ca800640 1 -- 172.21.15.32:0/1332028107 learned_addr learned my addr 172.21.15.32:0/1332028107 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:10.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1c1400640 1 --2- 172.21.15.32:0/1332028107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1cc0cd190 0x7ff1cc0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:10.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1c9e00640 1 --2- 172.21.15.32:0/1332028107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1cc073600 0x7ff1cc1560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:10.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1ca800640 1 -- 172.21.15.32:0/1332028107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1cc0cd190 msgr2=0x7ff1cc0cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:10.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1ca800640 1 --2- 172.21.15.32:0/1332028107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1cc0cd190 0x7ff1cc0cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:10.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1ca800640 1 -- 172.21.15.32:0/1332028107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1cc073600 msgr2=0x7ff1cc1560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:10.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1ca800640 1 --2- 172.21.15.32:0/1332028107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1cc073600 0x7ff1cc1560d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:10.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1ca800640 1 -- 172.21.15.32:0/1332028107 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1cc1f4bd0 con 0x7ff1cc0cea30 2024-04-17T09:59:10.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1ca800640 1 --2- 172.21.15.32:0/1332028107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1cc0cea30 0x7ff1cc0cc6e0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400caa0 tx=0x7ff1c400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:10.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1c2a00640 1 -- 172.21.15.32:0/1332028107 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4017070 con 0x7ff1cc0cea30 2024-04-17T09:59:10.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1c2a00640 1 -- 172.21.15.32:0/1332028107 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c400ac00 con 0x7ff1cc0cea30 2024-04-17T09:59:10.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1d0c00640 1 -- 172.21.15.32:0/1332028107 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1cc1f4e70 con 0x7ff1cc0cea30 2024-04-17T09:59:10.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1c2a00640 1 -- 172.21.15.32:0/1332028107 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4012400 con 0x7ff1cc0cea30 2024-04-17T09:59:10.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.950+0000 7ff1d0c00640 1 -- 172.21.15.32:0/1332028107 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1cc1f5430 con 0x7ff1cc0cea30 2024-04-17T09:59:10.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.954+0000 7ff1d0c00640 1 -- 172.21.15.32:0/1332028107 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1cc068a90 con 0x7ff1cc0cea30 2024-04-17T09:59:10.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.954+0000 7ff1c2a00640 1 -- 172.21.15.32:0/1332028107 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff1c401a070 con 0x7ff1cc0cea30 2024-04-17T09:59:10.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.954+0000 7ff1c2a00640 1 --2- 172.21.15.32:0/1332028107 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff1a8068380 0x7ff1a806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:10.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.954+0000 7ff1c2a00640 1 -- 172.21.15.32:0/1332028107 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7ff1c40866b0 con 0x7ff1cc0cea30 2024-04-17T09:59:10.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.954+0000 7ff1c9e00640 1 --2- 172.21.15.32:0/1332028107 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff1a8068380 0x7ff1a806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:10.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.954+0000 7ff1c9e00640 1 --2- 172.21.15.32:0/1332028107 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff1a8068380 0x7ff1a806a840 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0b03e0 tx=0x7ff1bc009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:10.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:10.958+0000 7ff1c2a00640 1 -- 172.21.15.32:0/1332028107 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff1c4056580 con 0x7ff1cc0cea30 2024-04-17T09:59:11.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:11.118+0000 7ff1d0c00640 1 -- 172.21.15.32:0/1332028107 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff1cc0b5750 con 0x7ff1a8068380 2024-04-17T09:59:11.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:11.126+0000 7ff1c2a00640 1 -- 172.21.15.32:0/1332028107 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248008 (secure 0 0 0) 0x7ff1cc0b5750 con 0x7ff1a8068380 2024-04-17T09:59:11.125 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:59:11.126 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T09:59:11.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:11.126+0000 7ff1d0c00640 1 -- 172.21.15.32:0/1332028107 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff1a8068380 msgr2=0x7ff1a806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:11.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:11.126+0000 7ff1d0c00640 1 --2- 172.21.15.32:0/1332028107 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff1a8068380 0x7ff1a806a840 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0b03e0 tx=0x7ff1bc009190 comp rx=0 tx=0).stop 2024-04-17T09:59:11.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:11.126+0000 7ff1d0c00640 1 -- 172.21.15.32:0/1332028107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1cc0cea30 msgr2=0x7ff1cc0cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:11.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:11.126+0000 7ff1d0c00640 1 --2- 172.21.15.32:0/1332028107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1cc0cea30 0x7ff1cc0cc6e0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400caa0 tx=0x7ff1c400cf60 comp rx=0 tx=0).stop 2024-04-17T09:59:11.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:11.130+0000 7ff1d0c00640 1 -- 172.21.15.32:0/1332028107 shutdown_connections 2024-04-17T09:59:11.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:11.130+0000 7ff1d0c00640 1 --2- 172.21.15.32:0/1332028107 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff1a8068380 0x7ff1a806a840 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:11.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:11.130+0000 7ff1d0c00640 1 --2- 172.21.15.32:0/1332028107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1cc0cd190 0x7ff1cc0cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:11.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:11.130+0000 7ff1d0c00640 1 --2- 172.21.15.32:0/1332028107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1cc0cea30 0x7ff1cc0cc6e0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:11.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:11.130+0000 7ff1d0c00640 1 --2- 172.21.15.32:0/1332028107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1cc073600 0x7ff1cc1560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:11.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:11.130+0000 7ff1d0c00640 1 -- 172.21.15.32:0/1332028107 >> 172.21.15.32:0/1332028107 conn(0x7ff1cc0d0c00 msgr2=0x7ff1cc06f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:11.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:11.130+0000 7ff1d0c00640 1 -- 172.21.15.32:0/1332028107 shutdown_connections 2024-04-17T09:59:11.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:11.130+0000 7ff1d0c00640 1 -- 172.21.15.32:0/1332028107 wait complete. 2024-04-17T09:59:11.146 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-04-17T09:59:09.365238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":180,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258280,"kb_used_data":8816,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716440,"statfs":{"total":773094113280,"available":772829634560,"internally_reserved":0,"allocated":9027584,"data_stored":4494325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.953388"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.430338+0000","last_change":"2024-04-17T09:59:04.430338+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.430338+0000","last_undegraded":"2024-04-17T09:59:04.430338+0000","last_fullsized":"2024-04-17T09:59:04.430338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429509+0000","last_change":"2024-04-17T09:59:04.429509+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429509+0000","last_undegraded":"2024-04-17T09:59:04.429509+0000","last_fullsized":"2024-04-17T09:59:04.429509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433330+0000","last_change":"2024-04-17T09:59:04.433330+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433330+0000","last_undegraded":"2024-04-17T09:59:04.433330+0000","last_fullsized":"2024-04-17T09:59:04.433330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429769+0000","last_change":"2024-04-17T09:59:04.429769+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429769+0000","last_undegraded":"2024-04-17T09:59:04.429769+0000","last_fullsized":"2024-04-17T09:59:04.429769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427070+0000","last_change":"2024-04-17T09:59:05.435295+0000","last_active":"2024-04-17T09:59:06.427070+0000","last_peered":"2024-04-17T09:59:06.427070+0000","last_clean":"2024-04-17T09:59:06.427070+0000","last_became_active":"2024-04-17T09:59:05.434948+0000","last_became_peered":"2024-04-17T09:59:05.434948+0000","last_unstale":"2024-04-17T09:59:06.427070+0000","last_undegraded":"2024-04-17T09:59:06.427070+0000","last_fullsized":"2024-04-17T09:59:06.427070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:14.347809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.438674+0000","last_change":"2024-04-17T09:59:04.438674+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.438674+0000","last_undegraded":"2024-04-17T09:59:04.438674+0000","last_fullsized":"2024-04-17T09:59:04.438674+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429372+0000","last_change":"2024-04-17T09:59:04.429372+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429372+0000","last_undegraded":"2024-04-17T09:59:04.429372+0000","last_fullsized":"2024-04-17T09:59:04.429372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.437249+0000","last_change":"2024-04-17T09:59:04.437249+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.437249+0000","last_undegraded":"2024-04-17T09:59:04.437249+0000","last_fullsized":"2024-04-17T09:59:04.437249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.440846+0000","last_change":"2024-04-17T09:59:04.440846+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.440846+0000","last_undegraded":"2024-04-17T09:59:04.440846+0000","last_fullsized":"2024-04-17T09:59:04.440846+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427452+0000","last_change":"2024-04-17T09:59:04.427452+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427452+0000","last_undegraded":"2024-04-17T09:59:04.427452+0000","last_fullsized":"2024-04-17T09:59:04.427452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427477+0000","last_change":"2024-04-17T09:59:04.427477+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427477+0000","last_undegraded":"2024-04-17T09:59:04.427477+0000","last_fullsized":"2024-04-17T09:59:04.427477+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427835+0000","last_change":"2024-04-17T09:59:04.427835+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427835+0000","last_undegraded":"2024-04-17T09:59:04.427835+0000","last_fullsized":"2024-04-17T09:59:04.427835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433600+0000","last_change":"2024-04-17T09:59:04.433600+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433600+0000","last_undegraded":"2024-04-17T09:59:04.433600+0000","last_fullsized":"2024-04-17T09:59:04.433600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.435489+0000","last_change":"2024-04-17T09:59:04.435489+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.435489+0000","last_undegraded":"2024-04-17T09:59:04.435489+0000","last_fullsized":"2024-04-17T09:59:04.435489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427347+0000","last_change":"2024-04-17T09:59:04.427347+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427347+0000","last_undegraded":"2024-04-17T09:59:04.427347+0000","last_fullsized":"2024-04-17T09:59:04.427347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426220+0000","last_change":"2024-04-17T09:59:05.445968+0000","last_active":"2024-04-17T09:59:06.426220+0000","last_peered":"2024-04-17T09:59:06.426220+0000","last_clean":"2024-04-17T09:59:06.426220+0000","last_became_active":"2024-04-17T09:59:05.445695+0000","last_became_peered":"2024-04-17T09:59:05.445695+0000","last_unstale":"2024-04-17T09:59:06.426220+0000","last_undegraded":"2024-04-17T09:59:06.426220+0000","last_fullsized":"2024-04-17T09:59:06.426220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:27:15.977393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.428029+0000","last_change":"2024-04-17T09:59:05.447691+0000","last_active":"2024-04-17T09:59:06.428029+0000","last_peered":"2024-04-17T09:59:06.428029+0000","last_clean":"2024-04-17T09:59:06.428029+0000","last_became_active":"2024-04-17T09:59:05.447403+0000","last_became_peered":"2024-04-17T09:59:05.447403+0000","last_unstale":"2024-04-17T09:59:06.428029+0000","last_undegraded":"2024-04-17T09:59:06.428029+0000","last_fullsized":"2024-04-17T09:59:06.428029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:15.267856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.432446+0000","last_change":"2024-04-17T09:59:04.432446+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.432446+0000","last_undegraded":"2024-04-17T09:59:04.432446+0000","last_fullsized":"2024-04-17T09:59:04.432446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455142+0000","last_change":"2024-04-17T09:59:05.442262+0000","last_active":"2024-04-17T09:59:06.455142+0000","last_peered":"2024-04-17T09:59:06.455142+0000","last_clean":"2024-04-17T09:59:06.455142+0000","last_became_active":"2024-04-17T09:59:05.441944+0000","last_became_peered":"2024-04-17T09:59:05.441944+0000","last_unstale":"2024-04-17T09:59:06.455142+0000","last_undegraded":"2024-04-17T09:59:06.455142+0000","last_fullsized":"2024-04-17T09:59:06.455142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:55:27.872332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429260+0000","last_change":"2024-04-17T09:59:04.429260+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429260+0000","last_undegraded":"2024-04-17T09:59:04.429260+0000","last_fullsized":"2024-04-17T09:59:04.429260+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429463+0000","last_change":"2024-04-17T09:59:04.429463+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429463+0000","last_undegraded":"2024-04-17T09:59:04.429463+0000","last_fullsized":"2024-04-17T09:59:04.429463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428078+0000","last_change":"2024-04-17T09:59:04.428078+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428078+0000","last_undegraded":"2024-04-17T09:59:04.428078+0000","last_fullsized":"2024-04-17T09:59:04.428078+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427009+0000","last_change":"2024-04-17T09:59:04.427009+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427009+0000","last_undegraded":"2024-04-17T09:59:04.427009+0000","last_fullsized":"2024-04-17T09:59:04.427009+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428239+0000","last_change":"2024-04-17T09:59:04.428239+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428239+0000","last_undegraded":"2024-04-17T09:59:04.428239+0000","last_fullsized":"2024-04-17T09:59:04.428239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.425424+0000","last_change":"2024-04-17T09:59:04.425424+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.425424+0000","last_undegraded":"2024-04-17T09:59:04.425424+0000","last_fullsized":"2024-04-17T09:59:04.425424+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.432662+0000","last_change":"2024-04-17T09:59:04.432662+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.432662+0000","last_undegraded":"2024-04-17T09:59:04.432662+0000","last_fullsized":"2024-04-17T09:59:04.432662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427077+0000","last_change":"2024-04-17T09:59:05.438772+0000","last_active":"2024-04-17T09:59:06.427077+0000","last_peered":"2024-04-17T09:59:06.427077+0000","last_clean":"2024-04-17T09:59:06.427077+0000","last_became_active":"2024-04-17T09:59:05.438444+0000","last_became_peered":"2024-04-17T09:59:05.438444+0000","last_unstale":"2024-04-17T09:59:06.427077+0000","last_undegraded":"2024-04-17T09:59:06.427077+0000","last_fullsized":"2024-04-17T09:59:06.427077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:53:56.012120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429291+0000","last_change":"2024-04-17T09:59:04.429291+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429291+0000","last_undegraded":"2024-04-17T09:59:04.429291+0000","last_fullsized":"2024-04-17T09:59:04.429291+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427246+0000","last_change":"2024-04-17T09:59:04.427246+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427246+0000","last_undegraded":"2024-04-17T09:59:04.427246+0000","last_fullsized":"2024-04-17T09:59:04.427246+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.426566+0000","last_change":"2024-04-17T09:59:04.426566+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.426566+0000","last_undegraded":"2024-04-17T09:59:04.426566+0000","last_fullsized":"2024-04-17T09:59:04.426566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426520+0000","last_change":"2024-04-17T09:59:05.438189+0000","last_active":"2024-04-17T09:59:06.426520+0000","last_peered":"2024-04-17T09:59:06.426520+0000","last_clean":"2024-04-17T09:59:06.426520+0000","last_became_active":"2024-04-17T09:59:05.437898+0000","last_became_peered":"2024-04-17T09:59:05.437898+0000","last_unstale":"2024-04-17T09:59:06.426520+0000","last_undegraded":"2024-04-17T09:59:06.426520+0000","last_fullsized":"2024-04-17T09:59:06.426520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:55.256201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428134+0000","last_change":"2024-04-17T09:59:04.428134+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428134+0000","last_undegraded":"2024-04-17T09:59:04.428134+0000","last_fullsized":"2024-04-17T09:59:04.428134+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.428003+0000","last_change":"2024-04-17T09:59:05.445880+0000","last_active":"2024-04-17T09:59:06.428003+0000","last_peered":"2024-04-17T09:59:06.428003+0000","last_clean":"2024-04-17T09:59:06.428003+0000","last_became_active":"2024-04-17T09:59:05.439231+0000","last_became_peered":"2024-04-17T09:59:05.439231+0000","last_unstale":"2024-04-17T09:59:06.428003+0000","last_undegraded":"2024-04-17T09:59:06.428003+0000","last_fullsized":"2024-04-17T09:59:06.428003+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:40.503102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.445224+0000","last_change":"2024-04-17T09:59:04.443338+0000","last_active":"2024-04-17T09:59:04.445224+0000","last_peered":"2024-04-17T09:59:04.445224+0000","last_clean":"2024-04-17T09:59:04.445224+0000","last_became_active":"2024-04-17T09:59:04.442697+0000","last_became_peered":"2024-04-17T09:59:04.442697+0000","last_unstale":"2024-04-17T09:59:04.445224+0000","last_undegraded":"2024-04-17T09:59:04.445224+0000","last_fullsized":"2024-04-17T09:59:04.445224+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:39:16.180270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429909+0000","last_change":"2024-04-17T09:59:04.429909+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429909+0000","last_undegraded":"2024-04-17T09:59:04.429909+0000","last_fullsized":"2024-04-17T09:59:04.429909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.428060+0000","last_change":"2024-04-17T09:59:04.439722+0000","last_active":"2024-04-17T09:59:06.428060+0000","last_peered":"2024-04-17T09:59:06.428060+0000","last_clean":"2024-04-17T09:59:06.428060+0000","last_became_active":"2024-04-17T09:59:04.439452+0000","last_became_peered":"2024-04-17T09:59:04.439452+0000","last_unstale":"2024-04-17T09:59:06.428060+0000","last_undegraded":"2024-04-17T09:59:06.428060+0000","last_fullsized":"2024-04-17T09:59:06.428060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:49:05.349813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428740+0000","last_change":"2024-04-17T09:59:04.428740+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428740+0000","last_undegraded":"2024-04-17T09:59:04.428740+0000","last_fullsized":"2024-04-17T09:59:04.428740+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455165+0000","last_change":"2024-04-17T09:59:04.444375+0000","last_active":"2024-04-17T09:59:06.455165+0000","last_peered":"2024-04-17T09:59:06.455165+0000","last_clean":"2024-04-17T09:59:06.455165+0000","last_became_active":"2024-04-17T09:59:04.444125+0000","last_became_peered":"2024-04-17T09:59:04.444125+0000","last_unstale":"2024-04-17T09:59:06.455165+0000","last_undegraded":"2024-04-17T09:59:06.455165+0000","last_fullsized":"2024-04-17T09:59:06.455165+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:07.141412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433823+0000","last_change":"2024-04-17T09:59:04.433823+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433823+0000","last_undegraded":"2024-04-17T09:59:04.433823+0000","last_fullsized":"2024-04-17T09:59:04.433823+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455155+0000","last_change":"2024-04-17T09:59:04.443042+0000","last_active":"2024-04-17T09:59:06.455155+0000","last_peered":"2024-04-17T09:59:06.455155+0000","last_clean":"2024-04-17T09:59:06.455155+0000","last_became_active":"2024-04-17T09:59:04.442701+0000","last_became_peered":"2024-04-17T09:59:04.442701+0000","last_unstale":"2024-04-17T09:59:06.455155+0000","last_undegraded":"2024-04-17T09:59:06.455155+0000","last_fullsized":"2024-04-17T09:59:06.455155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:02:06.697176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.425419+0000","last_change":"2024-04-17T09:59:04.425419+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.425419+0000","last_undegraded":"2024-04-17T09:59:04.425419+0000","last_fullsized":"2024-04-17T09:59:04.425419+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427370+0000","last_change":"2024-04-17T09:59:04.438652+0000","last_active":"2024-04-17T09:59:06.427370+0000","last_peered":"2024-04-17T09:59:06.427370+0000","last_clean":"2024-04-17T09:59:06.427370+0000","last_became_active":"2024-04-17T09:59:04.438385+0000","last_became_peered":"2024-04-17T09:59:04.438385+0000","last_unstale":"2024-04-17T09:59:06.427370+0000","last_undegraded":"2024-04-17T09:59:06.427370+0000","last_fullsized":"2024-04-17T09:59:06.427370+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:12.392733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427235+0000","last_change":"2024-04-17T09:59:05.436633+0000","last_active":"2024-04-17T09:59:06.427235+0000","last_peered":"2024-04-17T09:59:06.427235+0000","last_clean":"2024-04-17T09:59:06.427235+0000","last_became_active":"2024-04-17T09:59:05.436348+0000","last_became_peered":"2024-04-17T09:59:05.436348+0000","last_unstale":"2024-04-17T09:59:06.427235+0000","last_undegraded":"2024-04-17T09:59:06.427235+0000","last_fullsized":"2024-04-17T09:59:06.427235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:08.147437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426666+0000","last_change":"2024-04-17T09:59:04.440021+0000","last_active":"2024-04-17T09:59:06.426666+0000","last_peered":"2024-04-17T09:59:06.426666+0000","last_clean":"2024-04-17T09:59:06.426666+0000","last_became_active":"2024-04-17T09:59:04.439185+0000","last_became_peered":"2024-04-17T09:59:04.439185+0000","last_unstale":"2024-04-17T09:59:06.426666+0000","last_undegraded":"2024-04-17T09:59:06.426666+0000","last_fullsized":"2024-04-17T09:59:06.426666+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:51.158495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433375+0000","last_change":"2024-04-17T09:59:04.433375+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433375+0000","last_undegraded":"2024-04-17T09:59:04.433375+0000","last_fullsized":"2024-04-17T09:59:04.433375+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.454817+0000","last_change":"2024-04-17T09:59:04.449411+0000","last_active":"2024-04-17T09:59:06.454817+0000","last_peered":"2024-04-17T09:59:06.454817+0000","last_clean":"2024-04-17T09:59:06.454817+0000","last_became_active":"2024-04-17T09:59:04.449080+0000","last_became_peered":"2024-04-17T09:59:04.449080+0000","last_unstale":"2024-04-17T09:59:06.454817+0000","last_undegraded":"2024-04-17T09:59:06.454817+0000","last_fullsized":"2024-04-17T09:59:06.454817+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:13.341784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.426818+0000","last_change":"2024-04-17T09:59:04.426818+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.426818+0000","last_undegraded":"2024-04-17T09:59:04.426818+0000","last_fullsized":"2024-04-17T09:59:04.426818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.438105+0000","last_change":"2024-04-17T09:59:04.433580+0000","last_active":"2024-04-17T09:59:04.438105+0000","last_peered":"2024-04-17T09:59:04.438105+0000","last_clean":"2024-04-17T09:59:04.438105+0000","last_became_active":"2024-04-17T09:59:04.433250+0000","last_became_peered":"2024-04-17T09:59:04.433250+0000","last_unstale":"2024-04-17T09:59:04.438105+0000","last_undegraded":"2024-04-17T09:59:04.438105+0000","last_fullsized":"2024-04-17T09:59:04.438105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:08:42.684550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428201+0000","last_change":"2024-04-17T09:59:04.428201+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428201+0000","last_undegraded":"2024-04-17T09:59:04.428201+0000","last_fullsized":"2024-04-17T09:59:04.428201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.445242+0000","last_change":"2024-04-17T09:59:04.443547+0000","last_active":"2024-04-17T09:59:04.445242+0000","last_peered":"2024-04-17T09:59:04.445242+0000","last_clean":"2024-04-17T09:59:04.445242+0000","last_became_active":"2024-04-17T09:59:04.443090+0000","last_became_peered":"2024-04-17T09:59:04.443090+0000","last_unstale":"2024-04-17T09:59:04.445242+0000","last_undegraded":"2024-04-17T09:59:04.445242+0000","last_fullsized":"2024-04-17T09:59:04.445242+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:36:05.857724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.440390+0000","last_change":"2024-04-17T09:59:04.440390+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.440390+0000","last_undegraded":"2024-04-17T09:59:04.440390+0000","last_fullsized":"2024-04-17T09:59:04.440390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.444139+0000","last_change":"2024-04-17T09:59:04.438537+0000","last_active":"2024-04-17T09:59:04.444139+0000","last_peered":"2024-04-17T09:59:04.444139+0000","last_clean":"2024-04-17T09:59:04.444139+0000","last_became_active":"2024-04-17T09:59:04.438006+0000","last_became_peered":"2024-04-17T09:59:04.438006+0000","last_unstale":"2024-04-17T09:59:04.444139+0000","last_undegraded":"2024-04-17T09:59:04.444139+0000","last_fullsized":"2024-04-17T09:59:04.444139+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:15:09.312202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455087+0000","last_change":"2024-04-17T09:59:05.439118+0000","last_active":"2024-04-17T09:59:06.455087+0000","last_peered":"2024-04-17T09:59:06.455087+0000","last_clean":"2024-04-17T09:59:06.455087+0000","last_became_active":"2024-04-17T09:59:05.438852+0000","last_became_peered":"2024-04-17T09:59:05.438852+0000","last_unstale":"2024-04-17T09:59:06.455087+0000","last_undegraded":"2024-04-17T09:59:06.455087+0000","last_fullsized":"2024-04-17T09:59:06.455087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:13:00.757214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.444169+0000","last_change":"2024-04-17T09:59:04.438033+0000","last_active":"2024-04-17T09:59:04.444169+0000","last_peered":"2024-04-17T09:59:04.444169+0000","last_clean":"2024-04-17T09:59:04.444169+0000","last_became_active":"2024-04-17T09:59:04.437763+0000","last_became_peered":"2024-04-17T09:59:04.437763+0000","last_unstale":"2024-04-17T09:59:04.444169+0000","last_undegraded":"2024-04-17T09:59:04.444169+0000","last_fullsized":"2024-04-17T09:59:04.444169+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:03:17.824422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427265+0000","last_change":"2024-04-17T09:59:05.447894+0000","last_active":"2024-04-17T09:59:06.427265+0000","last_peered":"2024-04-17T09:59:06.427265+0000","last_clean":"2024-04-17T09:59:06.427265+0000","last_became_active":"2024-04-17T09:59:05.447595+0000","last_became_peered":"2024-04-17T09:59:05.447595+0000","last_unstale":"2024-04-17T09:59:06.427265+0000","last_undegraded":"2024-04-17T09:59:06.427265+0000","last_fullsized":"2024-04-17T09:59:06.427265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:11.377027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.451646+0000","last_change":"2024-04-17T09:59:04.448134+0000","last_active":"2024-04-17T09:59:04.451646+0000","last_peered":"2024-04-17T09:59:04.451646+0000","last_clean":"2024-04-17T09:59:04.451646+0000","last_became_active":"2024-04-17T09:59:04.447872+0000","last_became_peered":"2024-04-17T09:59:04.447872+0000","last_unstale":"2024-04-17T09:59:04.451646+0000","last_undegraded":"2024-04-17T09:59:04.451646+0000","last_fullsized":"2024-04-17T09:59:04.451646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:22:33.540833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455345+0000","last_change":"2024-04-17T09:59:05.439384+0000","last_active":"2024-04-17T09:59:06.455345+0000","last_peered":"2024-04-17T09:59:06.455345+0000","last_clean":"2024-04-17T09:59:06.455345+0000","last_became_active":"2024-04-17T09:59:05.439091+0000","last_became_peered":"2024-04-17T09:59:05.439091+0000","last_unstale":"2024-04-17T09:59:06.455345+0000","last_undegraded":"2024-04-17T09:59:06.455345+0000","last_fullsized":"2024-04-17T09:59:06.455345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:22:13.672459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.443120+0000","last_change":"2024-04-17T09:59:04.440489+0000","last_active":"2024-04-17T09:59:04.443120+0000","last_peered":"2024-04-17T09:59:04.443120+0000","last_clean":"2024-04-17T09:59:04.443120+0000","last_became_active":"2024-04-17T09:59:04.437665+0000","last_became_peered":"2024-04-17T09:59:04.437665+0000","last_unstale":"2024-04-17T09:59:04.443120+0000","last_undegraded":"2024-04-17T09:59:04.443120+0000","last_fullsized":"2024-04-17T09:59:04.443120+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:54:03.821994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433083+0000","last_change":"2024-04-17T09:59:04.433083+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433083+0000","last_undegraded":"2024-04-17T09:59:04.433083+0000","last_fullsized":"2024-04-17T09:59:04.433083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.446156+0000","last_change":"2024-04-17T09:59:04.441625+0000","last_active":"2024-04-17T09:59:04.446156+0000","last_peered":"2024-04-17T09:59:04.446156+0000","last_clean":"2024-04-17T09:59:04.446156+0000","last_became_active":"2024-04-17T09:59:04.441012+0000","last_became_peered":"2024-04-17T09:59:04.441012+0000","last_unstale":"2024-04-17T09:59:04.446156+0000","last_undegraded":"2024-04-17T09:59:04.446156+0000","last_fullsized":"2024-04-17T09:59:04.446156+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:31:12.043046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.426432+0000","last_change":"2024-04-17T09:59:04.426432+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.426432+0000","last_undegraded":"2024-04-17T09:59:04.426432+0000","last_fullsized":"2024-04-17T09:59:04.426432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.451592+0000","last_change":"2024-04-17T09:59:04.446837+0000","last_active":"2024-04-17T09:59:04.451592+0000","last_peered":"2024-04-17T09:59:04.451592+0000","last_clean":"2024-04-17T09:59:04.451592+0000","last_became_active":"2024-04-17T09:59:04.446469+0000","last_became_peered":"2024-04-17T09:59:04.446469+0000","last_unstale":"2024-04-17T09:59:04.451592+0000","last_undegraded":"2024-04-17T09:59:04.451592+0000","last_fullsized":"2024-04-17T09:59:04.451592+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:49.287601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.425532+0000","last_change":"2024-04-17T09:59:04.425532+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.425532+0000","last_undegraded":"2024-04-17T09:59:04.425532+0000","last_fullsized":"2024-04-17T09:59:04.425532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"81'8","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455171+0000","last_change":"2024-04-17T09:59:04.438156+0000","last_active":"2024-04-17T09:59:06.455171+0000","last_peered":"2024-04-17T09:59:06.455171+0000","last_clean":"2024-04-17T09:59:06.455171+0000","last_became_active":"2024-04-17T09:59:04.437821+0000","last_became_peered":"2024-04-17T09:59:04.437821+0000","last_unstale":"2024-04-17T09:59:06.455171+0000","last_undegraded":"2024-04-17T09:59:06.455171+0000","last_fullsized":"2024-04-17T09:59:06.455171+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:54.250176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.432091+0000","last_change":"2024-04-17T09:59:04.432091+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.432091+0000","last_undegraded":"2024-04-17T09:59:04.432091+0000","last_fullsized":"2024-04-17T09:59:04.432091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.448596+0000","last_change":"2024-04-17T09:59:04.447265+0000","last_active":"2024-04-17T09:59:04.448596+0000","last_peered":"2024-04-17T09:59:04.448596+0000","last_clean":"2024-04-17T09:59:04.448596+0000","last_became_active":"2024-04-17T09:59:04.446847+0000","last_became_peered":"2024-04-17T09:59:04.446847+0000","last_unstale":"2024-04-17T09:59:04.448596+0000","last_undegraded":"2024-04-17T09:59:04.448596+0000","last_fullsized":"2024-04-17T09:59:04.448596+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:49.287601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.429393+0000","last_change":"2024-04-17T09:59:04.429393+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.429393+0000","last_undegraded":"2024-04-17T09:59:04.429393+0000","last_fullsized":"2024-04-17T09:59:04.429393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455137+0000","last_change":"2024-04-17T09:59:04.443540+0000","last_active":"2024-04-17T09:59:06.455137+0000","last_peered":"2024-04-17T09:59:06.455137+0000","last_clean":"2024-04-17T09:59:06.455137+0000","last_became_active":"2024-04-17T09:59:04.443277+0000","last_became_peered":"2024-04-17T09:59:04.443277+0000","last_unstale":"2024-04-17T09:59:06.455137+0000","last_undegraded":"2024-04-17T09:59:06.455137+0000","last_fullsized":"2024-04-17T09:59:06.455137+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:10.371001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427822+0000","last_change":"2024-04-17T09:59:05.447239+0000","last_active":"2024-04-17T09:59:06.427822+0000","last_peered":"2024-04-17T09:59:06.427822+0000","last_clean":"2024-04-17T09:59:06.427822+0000","last_became_active":"2024-04-17T09:59:05.446970+0000","last_became_peered":"2024-04-17T09:59:05.446970+0000","last_unstale":"2024-04-17T09:59:06.427822+0000","last_undegraded":"2024-04-17T09:59:06.427822+0000","last_fullsized":"2024-04-17T09:59:06.427822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:02:07.703202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455269+0000","last_change":"2024-04-17T09:59:04.445958+0000","last_active":"2024-04-17T09:59:06.455269+0000","last_peered":"2024-04-17T09:59:06.455269+0000","last_clean":"2024-04-17T09:59:06.455269+0000","last_became_active":"2024-04-17T09:59:04.445632+0000","last_became_peered":"2024-04-17T09:59:04.445632+0000","last_unstale":"2024-04-17T09:59:06.455269+0000","last_undegraded":"2024-04-17T09:59:06.455269+0000","last_fullsized":"2024-04-17T09:59:06.455269+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:57.288434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.432936+0000","last_change":"2024-04-17T09:59:04.432936+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.432936+0000","last_undegraded":"2024-04-17T09:59:04.432936+0000","last_fullsized":"2024-04-17T09:59:04.432936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.448600+0000","last_change":"2024-04-17T09:59:04.445248+0000","last_active":"2024-04-17T09:59:04.448600+0000","last_peered":"2024-04-17T09:59:04.448600+0000","last_clean":"2024-04-17T09:59:04.448600+0000","last_became_active":"2024-04-17T09:59:04.444809+0000","last_became_peered":"2024-04-17T09:59:04.444809+0000","last_unstale":"2024-04-17T09:59:04.448600+0000","last_undegraded":"2024-04-17T09:59:04.448600+0000","last_fullsized":"2024-04-17T09:59:04.448600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:33:59.203901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.437546+0000","last_change":"2024-04-17T09:59:04.437546+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.437546+0000","last_undegraded":"2024-04-17T09:59:04.437546+0000","last_fullsized":"2024-04-17T09:59:04.437546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.451113+0000","last_change":"2024-04-17T09:59:04.448116+0000","last_active":"2024-04-17T09:59:04.451113+0000","last_peered":"2024-04-17T09:59:04.451113+0000","last_clean":"2024-04-17T09:59:04.451113+0000","last_became_active":"2024-04-17T09:59:04.446755+0000","last_became_peered":"2024-04-17T09:59:04.446755+0000","last_unstale":"2024-04-17T09:59:04.451113+0000","last_undegraded":"2024-04-17T09:59:04.451113+0000","last_fullsized":"2024-04-17T09:59:04.451113+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:04:29.400922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427614+0000","last_change":"2024-04-17T09:59:04.427614+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427614+0000","last_undegraded":"2024-04-17T09:59:04.427614+0000","last_fullsized":"2024-04-17T09:59:04.427614+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.451071+0000","last_change":"2024-04-17T09:59:04.448303+0000","last_active":"2024-04-17T09:59:04.451071+0000","last_peered":"2024-04-17T09:59:04.451071+0000","last_clean":"2024-04-17T09:59:04.451071+0000","last_became_active":"2024-04-17T09:59:04.446032+0000","last_became_peered":"2024-04-17T09:59:04.446032+0000","last_unstale":"2024-04-17T09:59:04.451071+0000","last_undegraded":"2024-04-17T09:59:04.451071+0000","last_fullsized":"2024-04-17T09:59:04.451071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:02:55.706845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.440407+0000","last_change":"2024-04-17T09:59:04.440407+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.440407+0000","last_undegraded":"2024-04-17T09:59:04.440407+0000","last_fullsized":"2024-04-17T09:59:04.440407+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.448496+0000","last_change":"2024-04-17T09:59:04.445983+0000","last_active":"2024-04-17T09:59:04.448496+0000","last_peered":"2024-04-17T09:59:04.448496+0000","last_clean":"2024-04-17T09:59:04.448496+0000","last_became_active":"2024-04-17T09:59:04.445671+0000","last_became_peered":"2024-04-17T09:59:04.445671+0000","last_unstale":"2024-04-17T09:59:04.448496+0000","last_undegraded":"2024-04-17T09:59:04.448496+0000","last_fullsized":"2024-04-17T09:59:04.448496+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:36.060764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.428075+0000","last_change":"2024-04-17T09:59:05.449697+0000","last_active":"2024-04-17T09:59:06.428075+0000","last_peered":"2024-04-17T09:59:06.428075+0000","last_clean":"2024-04-17T09:59:06.428075+0000","last_became_active":"2024-04-17T09:59:05.449390+0000","last_became_peered":"2024-04-17T09:59:05.449390+0000","last_unstale":"2024-04-17T09:59:06.428075+0000","last_undegraded":"2024-04-17T09:59:06.428075+0000","last_fullsized":"2024-04-17T09:59:06.428075+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:11:22.087912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.447272+0000","last_change":"2024-04-17T09:59:04.442441+0000","last_active":"2024-04-17T09:59:04.447272+0000","last_peered":"2024-04-17T09:59:04.447272+0000","last_clean":"2024-04-17T09:59:04.447272+0000","last_became_active":"2024-04-17T09:59:04.442051+0000","last_became_peered":"2024-04-17T09:59:04.442051+0000","last_unstale":"2024-04-17T09:59:04.447272+0000","last_undegraded":"2024-04-17T09:59:04.447272+0000","last_fullsized":"2024-04-17T09:59:04.447272+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:16:08.377113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.433918+0000","last_change":"2024-04-17T09:59:04.433918+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.433918+0000","last_undegraded":"2024-04-17T09:59:04.433918+0000","last_fullsized":"2024-04-17T09:59:04.433918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.446270+0000","last_change":"2024-04-17T09:59:04.441305+0000","last_active":"2024-04-17T09:59:04.446270+0000","last_peered":"2024-04-17T09:59:04.446270+0000","last_clean":"2024-04-17T09:59:04.446270+0000","last_became_active":"2024-04-17T09:59:04.441044+0000","last_became_peered":"2024-04-17T09:59:04.441044+0000","last_unstale":"2024-04-17T09:59:04.446270+0000","last_undegraded":"2024-04-17T09:59:04.446270+0000","last_fullsized":"2024-04-17T09:59:04.446270+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:06:39.663043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.428308+0000","last_change":"2024-04-17T09:59:04.428308+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.428308+0000","last_undegraded":"2024-04-17T09:59:04.428308+0000","last_fullsized":"2024-04-17T09:59:04.428308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.445184+0000","last_change":"2024-04-17T09:59:04.442494+0000","last_active":"2024-04-17T09:59:04.445184+0000","last_peered":"2024-04-17T09:59:04.445184+0000","last_clean":"2024-04-17T09:59:04.445184+0000","last_became_active":"2024-04-17T09:59:04.442254+0000","last_became_peered":"2024-04-17T09:59:04.442254+0000","last_unstale":"2024-04-17T09:59:04.445184+0000","last_undegraded":"2024-04-17T09:59:04.445184+0000","last_fullsized":"2024-04-17T09:59:04.445184+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:09:27.952101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427862+0000","last_change":"2024-04-17T09:59:05.438162+0000","last_active":"2024-04-17T09:59:06.427862+0000","last_peered":"2024-04-17T09:59:06.427862+0000","last_clean":"2024-04-17T09:59:06.427862+0000","last_became_active":"2024-04-17T09:59:05.437880+0000","last_became_peered":"2024-04-17T09:59:05.437880+0000","last_unstale":"2024-04-17T09:59:06.427862+0000","last_undegraded":"2024-04-17T09:59:06.427862+0000","last_fullsized":"2024-04-17T09:59:06.427862+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:58.294460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.445858+0000","last_change":"2024-04-17T09:59:04.444713+0000","last_active":"2024-04-17T09:59:04.445858+0000","last_peered":"2024-04-17T09:59:04.445858+0000","last_clean":"2024-04-17T09:59:04.445858+0000","last_became_active":"2024-04-17T09:59:04.444438+0000","last_became_peered":"2024-04-17T09:59:04.444438+0000","last_unstale":"2024-04-17T09:59:04.445858+0000","last_undegraded":"2024-04-17T09:59:04.445858+0000","last_fullsized":"2024-04-17T09:59:04.445858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:29:09.325067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.437197+0000","last_change":"2024-04-17T09:59:04.437197+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.437197+0000","last_undegraded":"2024-04-17T09:59:04.437197+0000","last_fullsized":"2024-04-17T09:59:04.437197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455124+0000","last_change":"2024-04-17T09:59:04.438627+0000","last_active":"2024-04-17T09:59:06.455124+0000","last_peered":"2024-04-17T09:59:06.455124+0000","last_clean":"2024-04-17T09:59:06.455124+0000","last_became_active":"2024-04-17T09:59:04.438100+0000","last_became_peered":"2024-04-17T09:59:04.438100+0000","last_unstale":"2024-04-17T09:59:06.455124+0000","last_undegraded":"2024-04-17T09:59:06.455124+0000","last_fullsized":"2024-04-17T09:59:06.455124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:53:55.006094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455088+0000","last_change":"2024-04-17T09:59:05.447417+0000","last_active":"2024-04-17T09:59:06.455088+0000","last_peered":"2024-04-17T09:59:06.455088+0000","last_clean":"2024-04-17T09:59:06.455088+0000","last_became_active":"2024-04-17T09:59:05.447176+0000","last_became_peered":"2024-04-17T09:59:05.447176+0000","last_unstale":"2024-04-17T09:59:06.455088+0000","last_undegraded":"2024-04-17T09:59:06.455088+0000","last_fullsized":"2024-04-17T09:59:06.455088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:18:04.685528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426694+0000","last_change":"2024-04-17T09:59:04.440315+0000","last_active":"2024-04-17T09:59:06.426694+0000","last_peered":"2024-04-17T09:59:06.426694+0000","last_clean":"2024-04-17T09:59:06.426694+0000","last_became_active":"2024-04-17T09:59:04.439614+0000","last_became_peered":"2024-04-17T09:59:04.439614+0000","last_unstale":"2024-04-17T09:59:06.426694+0000","last_undegraded":"2024-04-17T09:59:06.426694+0000","last_fullsized":"2024-04-17T09:59:06.426694+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:02.341478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.432875+0000","last_change":"2024-04-17T09:59:04.432875+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.432875+0000","last_undegraded":"2024-04-17T09:59:04.432875+0000","last_fullsized":"2024-04-17T09:59:04.432875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.445028+0000","last_change":"2024-04-17T09:59:04.442633+0000","last_active":"2024-04-17T09:59:04.445028+0000","last_peered":"2024-04-17T09:59:04.445028+0000","last_clean":"2024-04-17T09:59:04.445028+0000","last_became_active":"2024-04-17T09:59:04.442347+0000","last_became_peered":"2024-04-17T09:59:04.442347+0000","last_unstale":"2024-04-17T09:59:04.445028+0000","last_undegraded":"2024-04-17T09:59:04.445028+0000","last_fullsized":"2024-04-17T09:59:04.445028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:38:59.533078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.427513+0000","last_change":"2024-04-17T09:59:04.427513+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.427513+0000","last_undegraded":"2024-04-17T09:59:04.427513+0000","last_fullsized":"2024-04-17T09:59:04.427513+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426835+0000","last_change":"2024-04-17T09:59:04.440613+0000","last_active":"2024-04-17T09:59:06.426835+0000","last_peered":"2024-04-17T09:59:06.426835+0000","last_clean":"2024-04-17T09:59:06.426835+0000","last_became_active":"2024-04-17T09:59:04.440063+0000","last_became_peered":"2024-04-17T09:59:04.440063+0000","last_unstale":"2024-04-17T09:59:06.426835+0000","last_undegraded":"2024-04-17T09:59:06.426835+0000","last_fullsized":"2024-04-17T09:59:06.426835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:53:08.846256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-17T09:59:04.425466+0000","last_change":"2024-04-17T09:59:04.425466+0000","last_active":"2024-04-17T09:59:04.411847+0000","last_peered":"2024-04-17T09:59:04.411847+0000","last_clean":"2024-04-17T09:59:04.411847+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-17T09:59:04.425466+0000","last_undegraded":"2024-04-17T09:59:04.425466+0000","last_fullsized":"2024-04-17T09:59:04.425466+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.446679+0000","last_change":"2024-04-17T09:59:04.445356+0000","last_active":"2024-04-17T09:59:04.446679+0000","last_peered":"2024-04-17T09:59:04.446679+0000","last_clean":"2024-04-17T09:59:04.446679+0000","last_became_active":"2024-04-17T09:59:04.445010+0000","last_became_peered":"2024-04-17T09:59:04.445010+0000","last_unstale":"2024-04-17T09:59:04.446679+0000","last_undegraded":"2024-04-17T09:59:04.446679+0000","last_fullsized":"2024-04-17T09:59:04.446679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:32:37.586889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":280,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-17T09:59:04.434933+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T09:59:04.434933+0000","last_peered":"2024-04-17T09:59:04.434933+0000","last_clean":"2024-04-17T09:59:04.434933+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T09:59:04.434933+0000","last_undegraded":"2024-04-17T09:59:04.434933+0000","last_fullsized":"2024-04-17T09:59:04.434933+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509516,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1568768,"data_stored":1002078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":411710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509516,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":962560,"data_stored":405435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509516,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":411710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":411710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509516,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":411710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542221,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":424785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542221,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36900,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334940,"statfs":{"total":96636764160,"available":96598978560,"internally_reserved":0,"allocated":1597440,"data_stored":1015187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T09:59:13.817 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:59:13.821 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:59:14.150 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T09:59:14.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.306+0000 7f6dc5200640 1 Processor -- start 2024-04-17T09:59:14.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.306+0000 7f6dc5200640 1 -- start start 2024-04-17T09:59:14.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.306+0000 7f6dc5200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6dc0145f30 0x7f6dc0071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:14.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.306+0000 7f6dc5200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6dc00cd580 0x7f6dc0072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:14.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.306+0000 7f6dc5200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dc00cdac0 con 0x7f6dc015d8d0 2024-04-17T09:59:14.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.306+0000 7f6dc5200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6dc006f400 con 0x7f6dc0145f30 2024-04-17T09:59:14.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.306+0000 7f6dc5200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f6dc0146310 con 0x7f6dc00cd580 2024-04-17T09:59:14.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.306+0000 7f6dbde00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6dc0145f30 0x7f6dc0071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:14.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.306+0000 7f6db5400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6dc015d8d0 0x7f6dc00cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52074/0 (socket says 172.21.15.32:52074) 2024-04-17T09:59:14.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.306+0000 7f6db5400640 1 -- 172.21.15.32:0/1835782810 learned_addr learned my addr 172.21.15.32:0/1835782810 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:14.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.306+0000 7f6dbe800640 1 --2- 172.21.15.32:0/1835782810 >> v2:172.21.15.144:3300/0 conn(0x7f6dc00cd580 0x7f6dc0072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:14.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.306+0000 7f6dbde00640 1 -- 172.21.15.32:0/1835782810 >> v2:172.21.15.144:3300/0 conn(0x7f6dc00cd580 msgr2=0x7f6dc0072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:14.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.306+0000 7f6dbde00640 1 --2- 172.21.15.32:0/1835782810 >> v2:172.21.15.144:3300/0 conn(0x7f6dc00cd580 0x7f6dc0072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:14.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.306+0000 7f6dbde00640 1 -- 172.21.15.32:0/1835782810 >> v1:172.21.15.32:6789/0 conn(0x7f6dc015d8d0 legacy=0x7f6dc00cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:14.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.306+0000 7f6dbde00640 1 -- 172.21.15.32:0/1835782810 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc015cdf0 con 0x7f6dc0145f30 2024-04-17T09:59:14.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.310+0000 7f6dbde00640 1 --2- 172.21.15.32:0/1835782810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc0145f30 0x7f6dc0071df0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f6dc015dd30 tx=0x7f6db001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5148effbd2982f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:14.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.310+0000 7f6dbd400640 1 -- 172.21.15.32:0/1835782810 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db00b5070 con 0x7f6dc0145f30 2024-04-17T09:59:14.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.310+0000 7f6dbd400640 1 -- 172.21.15.32:0/1835782810 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6db00a8970 con 0x7f6dc0145f30 2024-04-17T09:59:14.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.310+0000 7f6dbd400640 1 -- 172.21.15.32:0/1835782810 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db00a8d40 con 0x7f6dc0145f30 2024-04-17T09:59:14.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.310+0000 7f6dc5200640 1 -- 172.21.15.32:0/1835782810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc0145f30 msgr2=0x7f6dc0071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:14.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.310+0000 7f6dc5200640 1 --2- 172.21.15.32:0/1835782810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc0145f30 0x7f6dc0071df0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f6dc015dd30 tx=0x7f6db001ec70 comp rx=0 tx=0).stop 2024-04-17T09:59:14.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.310+0000 7f6dc5200640 1 -- 172.21.15.32:0/1835782810 shutdown_connections 2024-04-17T09:59:14.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.310+0000 7f6dc5200640 1 --2- 172.21.15.32:0/1835782810 >> v2:172.21.15.144:3300/0 conn(0x7f6dc00cd580 0x7f6dc0072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:14.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.310+0000 7f6dc5200640 1 --2- 172.21.15.32:0/1835782810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc0145f30 0x7f6dc0071df0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:14.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.310+0000 7f6dc5200640 1 -- 172.21.15.32:0/1835782810 >> 172.21.15.32:0/1835782810 conn(0x7f6dc006ba30 msgr2=0x7f6dc006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:14.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.310+0000 7f6dc5200640 1 -- 172.21.15.32:0/1835782810 shutdown_connections 2024-04-17T09:59:14.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.310+0000 7f6dc5200640 1 -- 172.21.15.32:0/1835782810 wait complete. 2024-04-17T09:59:14.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6dc5200640 1 Processor -- start 2024-04-17T09:59:14.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6dc5200640 1 -- start start 2024-04-17T09:59:14.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6dc5200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc00cd580 0x7f6dc01f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:14.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6dc5200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dc0145f30 0x7f6dc013ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:14.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6dc5200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dc015d8d0 0x7f6dc015c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:14.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6dc5200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc015cb90 con 0x7f6dc00cd580 2024-04-17T09:59:14.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6dc5200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc015d440 con 0x7f6dc015d8d0 2024-04-17T09:59:14.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6dc5200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6dc01601d0 con 0x7f6dc0145f30 2024-04-17T09:59:14.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6dbde00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc00cd580 0x7f6dc01f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:14.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6dbde00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc00cd580 0x7f6dc01f46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38680/0 (socket says 172.21.15.32:38680) 2024-04-17T09:59:14.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6dbde00640 1 -- 172.21.15.32:0/635813299 learned_addr learned my addr 172.21.15.32:0/635813299 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:14.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6db5400640 1 --2- 172.21.15.32:0/635813299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dc0145f30 0x7f6dc013ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:14.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6dbe800640 1 --2- 172.21.15.32:0/635813299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dc015d8d0 0x7f6dc015c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:14.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6dbde00640 1 -- 172.21.15.32:0/635813299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dc0145f30 msgr2=0x7f6dc013ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:14.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6dbde00640 1 --2- 172.21.15.32:0/635813299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dc0145f30 0x7f6dc013ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:14.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6dbde00640 1 -- 172.21.15.32:0/635813299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dc015d8d0 msgr2=0x7f6dc015c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:14.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6dbde00640 1 --2- 172.21.15.32:0/635813299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dc015d8d0 0x7f6dc015c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:14.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6dbde00640 1 -- 172.21.15.32:0/635813299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc006c760 con 0x7f6dc00cd580 2024-04-17T09:59:14.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6dbe800640 1 --2- 172.21.15.32:0/635813299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dc015d8d0 0x7f6dc015c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:59:14.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6db5400640 1 --2- 172.21.15.32:0/635813299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dc0145f30 0x7f6dc013ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:59:14.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6dbde00640 1 --2- 172.21.15.32:0/635813299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc00cd580 0x7f6dc01f46c0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f6db00b3040 tx=0x7f6db001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:14.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6db6a00640 1 -- 172.21.15.32:0/635813299 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db00bf070 con 0x7f6dc00cd580 2024-04-17T09:59:14.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6db6a00640 1 -- 172.21.15.32:0/635813299 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6db00b0740 con 0x7f6dc00cd580 2024-04-17T09:59:14.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6db6a00640 1 -- 172.21.15.32:0/635813299 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db00b5040 con 0x7f6dc00cd580 2024-04-17T09:59:14.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6dc5200640 1 -- 172.21.15.32:0/635813299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc015c6b0 con 0x7f6dc00cd580 2024-04-17T09:59:14.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.314+0000 7f6dc5200640 1 -- 172.21.15.32:0/635813299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc013f460 con 0x7f6dc00cd580 2024-04-17T09:59:14.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.318+0000 7f6dc5200640 1 -- 172.21.15.32:0/635813299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dc0068a90 con 0x7f6dc00cd580 2024-04-17T09:59:14.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.318+0000 7f6db6a00640 1 -- 172.21.15.32:0/635813299 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6db00c2070 con 0x7f6dc00cd580 2024-04-17T09:59:14.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.318+0000 7f6db6a00640 1 --2- 172.21.15.32:0/635813299 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6d9c068380 0x7f6d9c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:14.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.318+0000 7f6db6a00640 1 -- 172.21.15.32:0/635813299 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f6db0123260 con 0x7f6dc00cd580 2024-04-17T09:59:14.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.318+0000 7f6db5400640 1 --2- 172.21.15.32:0/635813299 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6d9c068380 0x7f6d9c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:14.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.318+0000 7f6db5400640 1 --2- 172.21.15.32:0/635813299 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6d9c068380 0x7f6d9c06a840 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6da8007fd0 tx=0x7f6da8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:14.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.322+0000 7f6db6a00640 1 -- 172.21.15.32:0/635813299 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f6dc015c6b0 con 0x7f6dc00cd580 2024-04-17T09:59:14.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.482+0000 7f6dc5200640 1 -- 172.21.15.32:0/635813299 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6dc013e810 con 0x7f6d9c068380 2024-04-17T09:59:14.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.490+0000 7f6db6a00640 1 -- 172.21.15.32:0/635813299 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252078 (secure 0 0 0) 0x7f6dc013e810 con 0x7f6d9c068380 2024-04-17T09:59:14.491 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:59:14.492 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T09:59:14.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.494+0000 7f6dc5200640 1 -- 172.21.15.32:0/635813299 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6d9c068380 msgr2=0x7f6d9c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.494+0000 7f6dc5200640 1 --2- 172.21.15.32:0/635813299 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6d9c068380 0x7f6d9c06a840 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6da8007fd0 tx=0x7f6da8009040 comp rx=0 tx=0).stop 2024-04-17T09:59:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.494+0000 7f6dc5200640 1 -- 172.21.15.32:0/635813299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc00cd580 msgr2=0x7f6dc01f46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.494+0000 7f6dc5200640 1 --2- 172.21.15.32:0/635813299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc00cd580 0x7f6dc01f46c0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f6db00b3040 tx=0x7f6db001ba10 comp rx=0 tx=0).stop 2024-04-17T09:59:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.494+0000 7f6dc5200640 1 -- 172.21.15.32:0/635813299 shutdown_connections 2024-04-17T09:59:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.494+0000 7f6dc5200640 1 --2- 172.21.15.32:0/635813299 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6d9c068380 0x7f6d9c06a840 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.494+0000 7f6dc5200640 1 --2- 172.21.15.32:0/635813299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dc0145f30 0x7f6dc013ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.494+0000 7f6dc5200640 1 --2- 172.21.15.32:0/635813299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dc015d8d0 0x7f6dc015c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.494+0000 7f6dc5200640 1 --2- 172.21.15.32:0/635813299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc00cd580 0x7f6dc01f46c0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.494+0000 7f6dc5200640 1 -- 172.21.15.32:0/635813299 >> 172.21.15.32:0/635813299 conn(0x7f6dc006ba30 msgr2=0x7f6dc00b4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.494+0000 7f6dc5200640 1 -- 172.21.15.32:0/635813299 shutdown_connections 2024-04-17T09:59:14.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.494+0000 7f6dc5200640 1 -- 172.21.15.32:0/635813299 wait complete. 2024-04-17T09:59:14.512 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-04-17T09:59:13.366337+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262644,"kb_used_data":9032,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754712076,"statfs":{"total":773094113280,"available":772825165824,"internally_reserved":0,"allocated":9248768,"data_stored":4583836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001954"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427161+0000","last_change":"2024-04-17T09:59:05.440239+0000","last_active":"2024-04-17T09:59:06.427161+0000","last_peered":"2024-04-17T09:59:06.427161+0000","last_clean":"2024-04-17T09:59:06.427161+0000","last_became_active":"2024-04-17T09:59:05.439944+0000","last_became_peered":"2024-04-17T09:59:05.439944+0000","last_unstale":"2024-04-17T09:59:06.427161+0000","last_undegraded":"2024-04-17T09:59:06.427161+0000","last_fullsized":"2024-04-17T09:59:06.427161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:54:04.082671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426806+0000","last_change":"2024-04-17T09:59:05.442592+0000","last_active":"2024-04-17T09:59:06.426806+0000","last_peered":"2024-04-17T09:59:06.426806+0000","last_clean":"2024-04-17T09:59:06.426806+0000","last_became_active":"2024-04-17T09:59:05.442337+0000","last_became_peered":"2024-04-17T09:59:05.442337+0000","last_unstale":"2024-04-17T09:59:06.426806+0000","last_undegraded":"2024-04-17T09:59:06.426806+0000","last_fullsized":"2024-04-17T09:59:06.426806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:15.575545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.085631+0000","last_change":"2024-04-17T09:59:05.442112+0000","last_active":"2024-04-17T09:59:07.085631+0000","last_peered":"2024-04-17T09:59:07.085631+0000","last_clean":"2024-04-17T09:59:07.085631+0000","last_became_active":"2024-04-17T09:59:05.440050+0000","last_became_peered":"2024-04-17T09:59:05.440050+0000","last_unstale":"2024-04-17T09:59:07.085631+0000","last_undegraded":"2024-04-17T09:59:07.085631+0000","last_fullsized":"2024-04-17T09:59:07.085631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:59:08.072000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.087087+0000","last_change":"2024-04-17T09:59:05.450379+0000","last_active":"2024-04-17T09:59:07.087087+0000","last_peered":"2024-04-17T09:59:07.087087+0000","last_clean":"2024-04-17T09:59:07.087087+0000","last_became_active":"2024-04-17T09:59:05.449606+0000","last_became_peered":"2024-04-17T09:59:05.449606+0000","last_unstale":"2024-04-17T09:59:07.087087+0000","last_undegraded":"2024-04-17T09:59:07.087087+0000","last_fullsized":"2024-04-17T09:59:07.087087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:20:44.204953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427070+0000","last_change":"2024-04-17T09:59:05.435295+0000","last_active":"2024-04-17T09:59:06.427070+0000","last_peered":"2024-04-17T09:59:06.427070+0000","last_clean":"2024-04-17T09:59:06.427070+0000","last_became_active":"2024-04-17T09:59:05.434948+0000","last_became_peered":"2024-04-17T09:59:05.434948+0000","last_unstale":"2024-04-17T09:59:06.427070+0000","last_undegraded":"2024-04-17T09:59:06.427070+0000","last_fullsized":"2024-04-17T09:59:06.427070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:14.347809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.086075+0000","last_change":"2024-04-17T09:59:05.447737+0000","last_active":"2024-04-17T09:59:07.086075+0000","last_peered":"2024-04-17T09:59:07.086075+0000","last_clean":"2024-04-17T09:59:07.086075+0000","last_became_active":"2024-04-17T09:59:05.447444+0000","last_became_peered":"2024-04-17T09:59:05.447444+0000","last_unstale":"2024-04-17T09:59:07.086075+0000","last_undegraded":"2024-04-17T09:59:07.086075+0000","last_fullsized":"2024-04-17T09:59:07.086075+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:11:08.032998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.087142+0000","last_change":"2024-04-17T09:59:05.441344+0000","last_active":"2024-04-17T09:59:07.087142+0000","last_peered":"2024-04-17T09:59:07.087142+0000","last_clean":"2024-04-17T09:59:07.087142+0000","last_became_active":"2024-04-17T09:59:05.441050+0000","last_became_peered":"2024-04-17T09:59:05.441050+0000","last_unstale":"2024-04-17T09:59:07.087142+0000","last_undegraded":"2024-04-17T09:59:07.087142+0000","last_fullsized":"2024-04-17T09:59:07.087142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:03:18.830448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.086319+0000","last_change":"2024-04-17T09:59:05.447433+0000","last_active":"2024-04-17T09:59:07.086319+0000","last_peered":"2024-04-17T09:59:07.086319+0000","last_clean":"2024-04-17T09:59:07.086319+0000","last_became_active":"2024-04-17T09:59:05.447072+0000","last_became_peered":"2024-04-17T09:59:05.447072+0000","last_unstale":"2024-04-17T09:59:07.086319+0000","last_undegraded":"2024-04-17T09:59:07.086319+0000","last_fullsized":"2024-04-17T09:59:07.086319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:18:00.290128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.086265+0000","last_change":"2024-04-17T09:59:05.448496+0000","last_active":"2024-04-17T09:59:07.086265+0000","last_peered":"2024-04-17T09:59:07.086265+0000","last_clean":"2024-04-17T09:59:07.086265+0000","last_became_active":"2024-04-17T09:59:05.448035+0000","last_became_peered":"2024-04-17T09:59:05.448035+0000","last_unstale":"2024-04-17T09:59:07.086265+0000","last_undegraded":"2024-04-17T09:59:07.086265+0000","last_fullsized":"2024-04-17T09:59:07.086265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:46:51.869257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427087+0000","last_change":"2024-04-17T09:59:05.438691+0000","last_active":"2024-04-17T09:59:06.427087+0000","last_peered":"2024-04-17T09:59:06.427087+0000","last_clean":"2024-04-17T09:59:06.427087+0000","last_became_active":"2024-04-17T09:59:05.436473+0000","last_became_peered":"2024-04-17T09:59:05.436473+0000","last_unstale":"2024-04-17T09:59:06.427087+0000","last_undegraded":"2024-04-17T09:59:06.427087+0000","last_fullsized":"2024-04-17T09:59:06.427087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:48.502813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426736+0000","last_change":"2024-04-17T09:59:05.444253+0000","last_active":"2024-04-17T09:59:06.426736+0000","last_peered":"2024-04-17T09:59:06.426736+0000","last_clean":"2024-04-17T09:59:06.426736+0000","last_became_active":"2024-04-17T09:59:05.443920+0000","last_became_peered":"2024-04-17T09:59:05.443920+0000","last_unstale":"2024-04-17T09:59:06.426736+0000","last_undegraded":"2024-04-17T09:59:06.426736+0000","last_fullsized":"2024-04-17T09:59:06.426736+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:05.514331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427470+0000","last_change":"2024-04-17T09:59:05.436859+0000","last_active":"2024-04-17T09:59:06.427470+0000","last_peered":"2024-04-17T09:59:06.427470+0000","last_clean":"2024-04-17T09:59:06.427470+0000","last_became_active":"2024-04-17T09:59:05.436426+0000","last_became_peered":"2024-04-17T09:59:05.436426+0000","last_unstale":"2024-04-17T09:59:06.427470+0000","last_undegraded":"2024-04-17T09:59:06.427470+0000","last_fullsized":"2024-04-17T09:59:06.427470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:08:15.500974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.085267+0000","last_change":"2024-04-17T09:59:05.445716+0000","last_active":"2024-04-17T09:59:07.085267+0000","last_peered":"2024-04-17T09:59:07.085267+0000","last_clean":"2024-04-17T09:59:07.085267+0000","last_became_active":"2024-04-17T09:59:05.445456+0000","last_became_peered":"2024-04-17T09:59:05.445456+0000","last_unstale":"2024-04-17T09:59:07.085267+0000","last_undegraded":"2024-04-17T09:59:07.085267+0000","last_fullsized":"2024-04-17T09:59:07.085267+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:35:34.825470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.086830+0000","last_change":"2024-04-17T09:59:05.447947+0000","last_active":"2024-04-17T09:59:07.086830+0000","last_peered":"2024-04-17T09:59:07.086830+0000","last_clean":"2024-04-17T09:59:07.086830+0000","last_became_active":"2024-04-17T09:59:05.447394+0000","last_became_peered":"2024-04-17T09:59:05.447394+0000","last_unstale":"2024-04-17T09:59:07.086830+0000","last_undegraded":"2024-04-17T09:59:07.086830+0000","last_fullsized":"2024-04-17T09:59:07.086830+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:32:39.410241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.508167+0000","last_change":"2024-04-17T09:59:05.441172+0000","last_active":"2024-04-17T09:59:06.508167+0000","last_peered":"2024-04-17T09:59:06.508167+0000","last_clean":"2024-04-17T09:59:06.508167+0000","last_became_active":"2024-04-17T09:59:05.440606+0000","last_became_peered":"2024-04-17T09:59:05.440606+0000","last_unstale":"2024-04-17T09:59:06.508167+0000","last_undegraded":"2024-04-17T09:59:06.508167+0000","last_fullsized":"2024-04-17T09:59:06.508167+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:17:45.989148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426220+0000","last_change":"2024-04-17T09:59:05.445968+0000","last_active":"2024-04-17T09:59:06.426220+0000","last_peered":"2024-04-17T09:59:06.426220+0000","last_clean":"2024-04-17T09:59:06.426220+0000","last_became_active":"2024-04-17T09:59:05.445695+0000","last_became_peered":"2024-04-17T09:59:05.445695+0000","last_unstale":"2024-04-17T09:59:06.426220+0000","last_undegraded":"2024-04-17T09:59:06.426220+0000","last_fullsized":"2024-04-17T09:59:06.426220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:27:15.977393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.428029+0000","last_change":"2024-04-17T09:59:05.447691+0000","last_active":"2024-04-17T09:59:06.428029+0000","last_peered":"2024-04-17T09:59:06.428029+0000","last_clean":"2024-04-17T09:59:06.428029+0000","last_became_active":"2024-04-17T09:59:05.447403+0000","last_became_peered":"2024-04-17T09:59:05.447403+0000","last_unstale":"2024-04-17T09:59:06.428029+0000","last_undegraded":"2024-04-17T09:59:06.428029+0000","last_fullsized":"2024-04-17T09:59:06.428029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:15.267856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.086860+0000","last_change":"2024-04-17T09:59:05.443818+0000","last_active":"2024-04-17T09:59:07.086860+0000","last_peered":"2024-04-17T09:59:07.086860+0000","last_clean":"2024-04-17T09:59:07.086860+0000","last_became_active":"2024-04-17T09:59:05.443596+0000","last_became_peered":"2024-04-17T09:59:05.443596+0000","last_unstale":"2024-04-17T09:59:07.086860+0000","last_undegraded":"2024-04-17T09:59:07.086860+0000","last_fullsized":"2024-04-17T09:59:07.086860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:46:12.751847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455142+0000","last_change":"2024-04-17T09:59:05.442262+0000","last_active":"2024-04-17T09:59:06.455142+0000","last_peered":"2024-04-17T09:59:06.455142+0000","last_clean":"2024-04-17T09:59:06.455142+0000","last_became_active":"2024-04-17T09:59:05.441944+0000","last_became_peered":"2024-04-17T09:59:05.441944+0000","last_unstale":"2024-04-17T09:59:06.455142+0000","last_undegraded":"2024-04-17T09:59:06.455142+0000","last_fullsized":"2024-04-17T09:59:06.455142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:55:27.872332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.087130+0000","last_change":"2024-04-17T09:59:05.439202+0000","last_active":"2024-04-17T09:59:07.087130+0000","last_peered":"2024-04-17T09:59:07.087130+0000","last_clean":"2024-04-17T09:59:07.087130+0000","last_became_active":"2024-04-17T09:59:05.439042+0000","last_became_peered":"2024-04-17T09:59:05.439042+0000","last_unstale":"2024-04-17T09:59:07.087130+0000","last_undegraded":"2024-04-17T09:59:07.087130+0000","last_fullsized":"2024-04-17T09:59:07.087130+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:10:34.182511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.087010+0000","last_change":"2024-04-17T09:59:05.442596+0000","last_active":"2024-04-17T09:59:07.087010+0000","last_peered":"2024-04-17T09:59:07.087010+0000","last_clean":"2024-04-17T09:59:07.087010+0000","last_became_active":"2024-04-17T09:59:05.442138+0000","last_became_peered":"2024-04-17T09:59:05.442138+0000","last_unstale":"2024-04-17T09:59:07.087010+0000","last_undegraded":"2024-04-17T09:59:07.087010+0000","last_fullsized":"2024-04-17T09:59:07.087010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:50:22.853868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426672+0000","last_change":"2024-04-17T09:59:05.438131+0000","last_active":"2024-04-17T09:59:06.426672+0000","last_peered":"2024-04-17T09:59:06.426672+0000","last_clean":"2024-04-17T09:59:06.426672+0000","last_became_active":"2024-04-17T09:59:05.437861+0000","last_became_peered":"2024-04-17T09:59:05.437861+0000","last_unstale":"2024-04-17T09:59:06.426672+0000","last_undegraded":"2024-04-17T09:59:06.426672+0000","last_fullsized":"2024-04-17T09:59:06.426672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:26.847861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426692+0000","last_change":"2024-04-17T09:59:05.437346+0000","last_active":"2024-04-17T09:59:06.426692+0000","last_peered":"2024-04-17T09:59:06.426692+0000","last_clean":"2024-04-17T09:59:06.426692+0000","last_became_active":"2024-04-17T09:59:05.436797+0000","last_became_peered":"2024-04-17T09:59:05.436797+0000","last_unstale":"2024-04-17T09:59:06.426692+0000","last_undegraded":"2024-04-17T09:59:06.426692+0000","last_fullsized":"2024-04-17T09:59:06.426692+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:49:49.664070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.086652+0000","last_change":"2024-04-17T09:59:05.441379+0000","last_active":"2024-04-17T09:59:07.086652+0000","last_peered":"2024-04-17T09:59:07.086652+0000","last_clean":"2024-04-17T09:59:07.086652+0000","last_became_active":"2024-04-17T09:59:05.440960+0000","last_became_peered":"2024-04-17T09:59:05.440960+0000","last_unstale":"2024-04-17T09:59:07.086652+0000","last_undegraded":"2024-04-17T09:59:07.086652+0000","last_fullsized":"2024-04-17T09:59:07.086652+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:43:08.725099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427021+0000","last_change":"2024-04-17T09:59:05.445401+0000","last_active":"2024-04-17T09:59:06.427021+0000","last_peered":"2024-04-17T09:59:06.427021+0000","last_clean":"2024-04-17T09:59:06.427021+0000","last_became_active":"2024-04-17T09:59:05.445135+0000","last_became_peered":"2024-04-17T09:59:05.445135+0000","last_unstale":"2024-04-17T09:59:06.427021+0000","last_undegraded":"2024-04-17T09:59:06.427021+0000","last_fullsized":"2024-04-17T09:59:06.427021+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:20:07.619237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.085882+0000","last_change":"2024-04-17T09:59:05.449259+0000","last_active":"2024-04-17T09:59:07.085882+0000","last_peered":"2024-04-17T09:59:07.085882+0000","last_clean":"2024-04-17T09:59:07.085882+0000","last_became_active":"2024-04-17T09:59:05.448692+0000","last_became_peered":"2024-04-17T09:59:05.448692+0000","last_unstale":"2024-04-17T09:59:07.085882+0000","last_undegraded":"2024-04-17T09:59:07.085882+0000","last_fullsized":"2024-04-17T09:59:07.085882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:20:18.099539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427077+0000","last_change":"2024-04-17T09:59:05.438772+0000","last_active":"2024-04-17T09:59:06.427077+0000","last_peered":"2024-04-17T09:59:06.427077+0000","last_clean":"2024-04-17T09:59:06.427077+0000","last_became_active":"2024-04-17T09:59:05.438444+0000","last_became_peered":"2024-04-17T09:59:05.438444+0000","last_unstale":"2024-04-17T09:59:06.427077+0000","last_undegraded":"2024-04-17T09:59:06.427077+0000","last_fullsized":"2024-04-17T09:59:06.427077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:53:56.012120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427197+0000","last_change":"2024-04-17T09:59:05.437240+0000","last_active":"2024-04-17T09:59:06.427197+0000","last_peered":"2024-04-17T09:59:06.427197+0000","last_clean":"2024-04-17T09:59:06.427197+0000","last_became_active":"2024-04-17T09:59:05.436710+0000","last_became_peered":"2024-04-17T09:59:05.436710+0000","last_unstale":"2024-04-17T09:59:06.427197+0000","last_undegraded":"2024-04-17T09:59:06.427197+0000","last_fullsized":"2024-04-17T09:59:06.427197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:31:13.049072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426511+0000","last_change":"2024-04-17T09:59:05.438158+0000","last_active":"2024-04-17T09:59:06.426511+0000","last_peered":"2024-04-17T09:59:06.426511+0000","last_clean":"2024-04-17T09:59:06.426511+0000","last_became_active":"2024-04-17T09:59:05.437845+0000","last_became_peered":"2024-04-17T09:59:05.437845+0000","last_unstale":"2024-04-17T09:59:06.426511+0000","last_undegraded":"2024-04-17T09:59:06.426511+0000","last_fullsized":"2024-04-17T09:59:06.426511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:17:52.430779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426735+0000","last_change":"2024-04-17T09:59:05.437344+0000","last_active":"2024-04-17T09:59:06.426735+0000","last_peered":"2024-04-17T09:59:06.426735+0000","last_clean":"2024-04-17T09:59:06.426735+0000","last_became_active":"2024-04-17T09:59:05.437072+0000","last_became_peered":"2024-04-17T09:59:05.437072+0000","last_unstale":"2024-04-17T09:59:06.426735+0000","last_undegraded":"2024-04-17T09:59:06.426735+0000","last_fullsized":"2024-04-17T09:59:06.426735+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:57:00.224441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426520+0000","last_change":"2024-04-17T09:59:05.438189+0000","last_active":"2024-04-17T09:59:06.426520+0000","last_peered":"2024-04-17T09:59:06.426520+0000","last_clean":"2024-04-17T09:59:06.426520+0000","last_became_active":"2024-04-17T09:59:05.437898+0000","last_became_peered":"2024-04-17T09:59:05.437898+0000","last_unstale":"2024-04-17T09:59:06.426520+0000","last_undegraded":"2024-04-17T09:59:06.426520+0000","last_fullsized":"2024-04-17T09:59:06.426520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:55.256201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426955+0000","last_change":"2024-04-17T09:59:05.440060+0000","last_active":"2024-04-17T09:59:06.426955+0000","last_peered":"2024-04-17T09:59:06.426955+0000","last_clean":"2024-04-17T09:59:06.426955+0000","last_became_active":"2024-04-17T09:59:05.439715+0000","last_became_peered":"2024-04-17T09:59:05.439715+0000","last_unstale":"2024-04-17T09:59:06.426955+0000","last_undegraded":"2024-04-17T09:59:06.426955+0000","last_fullsized":"2024-04-17T09:59:06.426955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:18.386001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.428003+0000","last_change":"2024-04-17T09:59:05.445880+0000","last_active":"2024-04-17T09:59:06.428003+0000","last_peered":"2024-04-17T09:59:06.428003+0000","last_clean":"2024-04-17T09:59:06.428003+0000","last_became_active":"2024-04-17T09:59:05.439231+0000","last_became_peered":"2024-04-17T09:59:05.439231+0000","last_unstale":"2024-04-17T09:59:06.428003+0000","last_undegraded":"2024-04-17T09:59:06.428003+0000","last_fullsized":"2024-04-17T09:59:06.428003+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:40.503102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"81'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427618+0000","last_change":"2024-04-17T09:59:04.443338+0000","last_active":"2024-04-17T09:59:06.427618+0000","last_peered":"2024-04-17T09:59:06.427618+0000","last_clean":"2024-04-17T09:59:06.427618+0000","last_became_active":"2024-04-17T09:59:04.442697+0000","last_became_peered":"2024-04-17T09:59:04.442697+0000","last_unstale":"2024-04-17T09:59:06.427618+0000","last_undegraded":"2024-04-17T09:59:06.427618+0000","last_fullsized":"2024-04-17T09:59:06.427618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:56:59.218416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427342+0000","last_change":"2024-04-17T09:59:05.440426+0000","last_active":"2024-04-17T09:59:06.427342+0000","last_peered":"2024-04-17T09:59:06.427342+0000","last_clean":"2024-04-17T09:59:06.427342+0000","last_became_active":"2024-04-17T09:59:05.440158+0000","last_became_peered":"2024-04-17T09:59:05.440158+0000","last_unstale":"2024-04-17T09:59:06.427342+0000","last_undegraded":"2024-04-17T09:59:06.427342+0000","last_fullsized":"2024-04-17T09:59:06.427342+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:34:00.209927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.428060+0000","last_change":"2024-04-17T09:59:04.439722+0000","last_active":"2024-04-17T09:59:06.428060+0000","last_peered":"2024-04-17T09:59:06.428060+0000","last_clean":"2024-04-17T09:59:06.428060+0000","last_became_active":"2024-04-17T09:59:04.439452+0000","last_became_peered":"2024-04-17T09:59:04.439452+0000","last_unstale":"2024-04-17T09:59:06.428060+0000","last_undegraded":"2024-04-17T09:59:06.428060+0000","last_fullsized":"2024-04-17T09:59:06.428060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:49:05.349813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427225+0000","last_change":"2024-04-17T09:59:05.432894+0000","last_active":"2024-04-17T09:59:06.427225+0000","last_peered":"2024-04-17T09:59:06.427225+0000","last_clean":"2024-04-17T09:59:06.427225+0000","last_became_active":"2024-04-17T09:59:05.432635+0000","last_became_peered":"2024-04-17T09:59:05.432635+0000","last_unstale":"2024-04-17T09:59:06.427225+0000","last_undegraded":"2024-04-17T09:59:06.427225+0000","last_fullsized":"2024-04-17T09:59:06.427225+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:10:24.292361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455165+0000","last_change":"2024-04-17T09:59:04.444375+0000","last_active":"2024-04-17T09:59:06.455165+0000","last_peered":"2024-04-17T09:59:06.455165+0000","last_clean":"2024-04-17T09:59:06.455165+0000","last_became_active":"2024-04-17T09:59:04.444125+0000","last_became_peered":"2024-04-17T09:59:04.444125+0000","last_unstale":"2024-04-17T09:59:06.455165+0000","last_undegraded":"2024-04-17T09:59:06.455165+0000","last_fullsized":"2024-04-17T09:59:06.455165+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:07.141412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.085646+0000","last_change":"2024-04-17T09:59:05.446617+0000","last_active":"2024-04-17T09:59:07.085646+0000","last_peered":"2024-04-17T09:59:07.085646+0000","last_clean":"2024-04-17T09:59:07.085646+0000","last_became_active":"2024-04-17T09:59:05.446153+0000","last_became_peered":"2024-04-17T09:59:05.446153+0000","last_unstale":"2024-04-17T09:59:07.085646+0000","last_undegraded":"2024-04-17T09:59:07.085646+0000","last_fullsized":"2024-04-17T09:59:07.085646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:20:49.960659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455155+0000","last_change":"2024-04-17T09:59:04.443042+0000","last_active":"2024-04-17T09:59:06.455155+0000","last_peered":"2024-04-17T09:59:06.455155+0000","last_clean":"2024-04-17T09:59:06.455155+0000","last_became_active":"2024-04-17T09:59:04.442701+0000","last_became_peered":"2024-04-17T09:59:04.442701+0000","last_unstale":"2024-04-17T09:59:06.455155+0000","last_undegraded":"2024-04-17T09:59:06.455155+0000","last_fullsized":"2024-04-17T09:59:06.455155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:02:06.697176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426855+0000","last_change":"2024-04-17T09:59:05.443359+0000","last_active":"2024-04-17T09:59:06.426855+0000","last_peered":"2024-04-17T09:59:06.426855+0000","last_clean":"2024-04-17T09:59:06.426855+0000","last_became_active":"2024-04-17T09:59:05.443026+0000","last_became_peered":"2024-04-17T09:59:05.443026+0000","last_unstale":"2024-04-17T09:59:06.426855+0000","last_undegraded":"2024-04-17T09:59:06.426855+0000","last_fullsized":"2024-04-17T09:59:06.426855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:58:32.550728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427370+0000","last_change":"2024-04-17T09:59:04.438652+0000","last_active":"2024-04-17T09:59:06.427370+0000","last_peered":"2024-04-17T09:59:06.427370+0000","last_clean":"2024-04-17T09:59:06.427370+0000","last_became_active":"2024-04-17T09:59:04.438385+0000","last_became_peered":"2024-04-17T09:59:04.438385+0000","last_unstale":"2024-04-17T09:59:06.427370+0000","last_undegraded":"2024-04-17T09:59:06.427370+0000","last_fullsized":"2024-04-17T09:59:06.427370+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:12.392733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427235+0000","last_change":"2024-04-17T09:59:05.436633+0000","last_active":"2024-04-17T09:59:06.427235+0000","last_peered":"2024-04-17T09:59:06.427235+0000","last_clean":"2024-04-17T09:59:06.427235+0000","last_became_active":"2024-04-17T09:59:05.436348+0000","last_became_peered":"2024-04-17T09:59:05.436348+0000","last_unstale":"2024-04-17T09:59:06.427235+0000","last_undegraded":"2024-04-17T09:59:06.427235+0000","last_fullsized":"2024-04-17T09:59:06.427235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:08.147437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426666+0000","last_change":"2024-04-17T09:59:04.440021+0000","last_active":"2024-04-17T09:59:06.426666+0000","last_peered":"2024-04-17T09:59:06.426666+0000","last_clean":"2024-04-17T09:59:06.426666+0000","last_became_active":"2024-04-17T09:59:04.439185+0000","last_became_peered":"2024-04-17T09:59:04.439185+0000","last_unstale":"2024-04-17T09:59:06.426666+0000","last_undegraded":"2024-04-17T09:59:06.426666+0000","last_fullsized":"2024-04-17T09:59:06.426666+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:51.158495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.085226+0000","last_change":"2024-04-17T09:59:05.443324+0000","last_active":"2024-04-17T09:59:07.085226+0000","last_peered":"2024-04-17T09:59:07.085226+0000","last_clean":"2024-04-17T09:59:07.085226+0000","last_became_active":"2024-04-17T09:59:05.443037+0000","last_became_peered":"2024-04-17T09:59:05.443037+0000","last_unstale":"2024-04-17T09:59:07.085226+0000","last_undegraded":"2024-04-17T09:59:07.085226+0000","last_fullsized":"2024-04-17T09:59:07.085226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:10:22.948570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.454817+0000","last_change":"2024-04-17T09:59:04.449411+0000","last_active":"2024-04-17T09:59:06.454817+0000","last_peered":"2024-04-17T09:59:06.454817+0000","last_clean":"2024-04-17T09:59:06.454817+0000","last_became_active":"2024-04-17T09:59:04.449080+0000","last_became_peered":"2024-04-17T09:59:04.449080+0000","last_unstale":"2024-04-17T09:59:06.454817+0000","last_undegraded":"2024-04-17T09:59:06.454817+0000","last_fullsized":"2024-04-17T09:59:06.454817+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:13.341784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.508203+0000","last_change":"2024-04-17T09:59:05.441363+0000","last_active":"2024-04-17T09:59:06.508203+0000","last_peered":"2024-04-17T09:59:06.508203+0000","last_clean":"2024-04-17T09:59:06.508203+0000","last_became_active":"2024-04-17T09:59:05.441033+0000","last_became_peered":"2024-04-17T09:59:05.441033+0000","last_unstale":"2024-04-17T09:59:06.508203+0000","last_undegraded":"2024-04-17T09:59:06.508203+0000","last_fullsized":"2024-04-17T09:59:06.508203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:03:09.174464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426324+0000","last_change":"2024-04-17T09:59:04.433580+0000","last_active":"2024-04-17T09:59:06.426324+0000","last_peered":"2024-04-17T09:59:06.426324+0000","last_clean":"2024-04-17T09:59:06.426324+0000","last_became_active":"2024-04-17T09:59:04.433250+0000","last_became_peered":"2024-04-17T09:59:04.433250+0000","last_unstale":"2024-04-17T09:59:06.426324+0000","last_undegraded":"2024-04-17T09:59:06.426324+0000","last_fullsized":"2024-04-17T09:59:06.426324+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:05:55.662952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.508215+0000","last_change":"2024-04-17T09:59:05.439919+0000","last_active":"2024-04-17T09:59:06.508215+0000","last_peered":"2024-04-17T09:59:06.508215+0000","last_clean":"2024-04-17T09:59:06.508215+0000","last_became_active":"2024-04-17T09:59:05.439502+0000","last_became_peered":"2024-04-17T09:59:05.439502+0000","last_unstale":"2024-04-17T09:59:06.508215+0000","last_undegraded":"2024-04-17T09:59:06.508215+0000","last_fullsized":"2024-04-17T09:59:06.508215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:20:18.099539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427719+0000","last_change":"2024-04-17T09:59:04.443547+0000","last_active":"2024-04-17T09:59:06.427719+0000","last_peered":"2024-04-17T09:59:06.427719+0000","last_clean":"2024-04-17T09:59:06.427719+0000","last_became_active":"2024-04-17T09:59:04.443090+0000","last_became_peered":"2024-04-17T09:59:04.443090+0000","last_unstale":"2024-04-17T09:59:06.427719+0000","last_undegraded":"2024-04-17T09:59:06.427719+0000","last_fullsized":"2024-04-17T09:59:06.427719+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:14.569519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.086329+0000","last_change":"2024-04-17T09:59:05.448279+0000","last_active":"2024-04-17T09:59:07.086329+0000","last_peered":"2024-04-17T09:59:07.086329+0000","last_clean":"2024-04-17T09:59:07.086329+0000","last_became_active":"2024-04-17T09:59:05.447766+0000","last_became_peered":"2024-04-17T09:59:05.447766+0000","last_unstale":"2024-04-17T09:59:07.086329+0000","last_undegraded":"2024-04-17T09:59:07.086329+0000","last_fullsized":"2024-04-17T09:59:07.086329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:20:26.867651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427198+0000","last_change":"2024-04-17T09:59:04.438537+0000","last_active":"2024-04-17T09:59:06.427198+0000","last_peered":"2024-04-17T09:59:06.427198+0000","last_clean":"2024-04-17T09:59:06.427198+0000","last_became_active":"2024-04-17T09:59:04.438006+0000","last_became_peered":"2024-04-17T09:59:04.438006+0000","last_unstale":"2024-04-17T09:59:06.427198+0000","last_undegraded":"2024-04-17T09:59:06.427198+0000","last_fullsized":"2024-04-17T09:59:06.427198+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:20:06.613211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455087+0000","last_change":"2024-04-17T09:59:05.439118+0000","last_active":"2024-04-17T09:59:06.455087+0000","last_peered":"2024-04-17T09:59:06.455087+0000","last_clean":"2024-04-17T09:59:06.455087+0000","last_became_active":"2024-04-17T09:59:05.438852+0000","last_became_peered":"2024-04-17T09:59:05.438852+0000","last_unstale":"2024-04-17T09:59:06.455087+0000","last_undegraded":"2024-04-17T09:59:06.455087+0000","last_fullsized":"2024-04-17T09:59:06.455087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:13:00.757214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"81'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426863+0000","last_change":"2024-04-17T09:59:04.438033+0000","last_active":"2024-04-17T09:59:06.426863+0000","last_peered":"2024-04-17T09:59:06.426863+0000","last_clean":"2024-04-17T09:59:06.426863+0000","last_became_active":"2024-04-17T09:59:04.437763+0000","last_became_peered":"2024-04-17T09:59:04.437763+0000","last_unstale":"2024-04-17T09:59:06.426863+0000","last_undegraded":"2024-04-17T09:59:06.426863+0000","last_fullsized":"2024-04-17T09:59:06.426863+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:25.841836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427265+0000","last_change":"2024-04-17T09:59:05.447894+0000","last_active":"2024-04-17T09:59:06.427265+0000","last_peered":"2024-04-17T09:59:06.427265+0000","last_clean":"2024-04-17T09:59:06.427265+0000","last_became_active":"2024-04-17T09:59:05.447595+0000","last_became_peered":"2024-04-17T09:59:05.447595+0000","last_unstale":"2024-04-17T09:59:06.427265+0000","last_undegraded":"2024-04-17T09:59:06.427265+0000","last_fullsized":"2024-04-17T09:59:06.427265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:11.377027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"81'6","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.086267+0000","last_change":"2024-04-17T09:59:04.448134+0000","last_active":"2024-04-17T09:59:07.086267+0000","last_peered":"2024-04-17T09:59:07.086267+0000","last_clean":"2024-04-17T09:59:07.086267+0000","last_became_active":"2024-04-17T09:59:04.447872+0000","last_became_peered":"2024-04-17T09:59:04.447872+0000","last_unstale":"2024-04-17T09:59:07.086267+0000","last_undegraded":"2024-04-17T09:59:07.086267+0000","last_fullsized":"2024-04-17T09:59:07.086267+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:25.841836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455345+0000","last_change":"2024-04-17T09:59:05.439384+0000","last_active":"2024-04-17T09:59:06.455345+0000","last_peered":"2024-04-17T09:59:06.455345+0000","last_clean":"2024-04-17T09:59:06.455345+0000","last_became_active":"2024-04-17T09:59:05.439091+0000","last_became_peered":"2024-04-17T09:59:05.439091+0000","last_unstale":"2024-04-17T09:59:06.455345+0000","last_undegraded":"2024-04-17T09:59:06.455345+0000","last_fullsized":"2024-04-17T09:59:06.455345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:22:13.672459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426618+0000","last_change":"2024-04-17T09:59:04.440489+0000","last_active":"2024-04-17T09:59:06.426618+0000","last_peered":"2024-04-17T09:59:06.426618+0000","last_clean":"2024-04-17T09:59:06.426618+0000","last_became_active":"2024-04-17T09:59:04.437665+0000","last_became_peered":"2024-04-17T09:59:04.437665+0000","last_unstale":"2024-04-17T09:59:06.426618+0000","last_undegraded":"2024-04-17T09:59:06.426618+0000","last_fullsized":"2024-04-17T09:59:06.426618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:32:38.404215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.085272+0000","last_change":"2024-04-17T09:59:05.443311+0000","last_active":"2024-04-17T09:59:07.085272+0000","last_peered":"2024-04-17T09:59:07.085272+0000","last_clean":"2024-04-17T09:59:07.085272+0000","last_became_active":"2024-04-17T09:59:05.443032+0000","last_became_peered":"2024-04-17T09:59:05.443032+0000","last_unstale":"2024-04-17T09:59:07.085272+0000","last_undegraded":"2024-04-17T09:59:07.085272+0000","last_fullsized":"2024-04-17T09:59:07.085272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:54.148064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.507958+0000","last_change":"2024-04-17T09:59:04.441625+0000","last_active":"2024-04-17T09:59:06.507958+0000","last_peered":"2024-04-17T09:59:06.507958+0000","last_clean":"2024-04-17T09:59:06.507958+0000","last_became_active":"2024-04-17T09:59:04.441012+0000","last_became_peered":"2024-04-17T09:59:04.441012+0000","last_unstale":"2024-04-17T09:59:06.507958+0000","last_undegraded":"2024-04-17T09:59:06.507958+0000","last_fullsized":"2024-04-17T09:59:06.507958+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:44.629939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426350+0000","last_change":"2024-04-17T09:59:05.437447+0000","last_active":"2024-04-17T09:59:06.426350+0000","last_peered":"2024-04-17T09:59:06.426350+0000","last_clean":"2024-04-17T09:59:06.426350+0000","last_became_active":"2024-04-17T09:59:05.437101+0000","last_became_peered":"2024-04-17T09:59:05.437101+0000","last_unstale":"2024-04-17T09:59:06.426350+0000","last_undegraded":"2024-04-17T09:59:06.426350+0000","last_fullsized":"2024-04-17T09:59:06.426350+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:22:34.546858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.086149+0000","last_change":"2024-04-17T09:59:04.446837+0000","last_active":"2024-04-17T09:59:07.086149+0000","last_peered":"2024-04-17T09:59:07.086149+0000","last_clean":"2024-04-17T09:59:07.086149+0000","last_became_active":"2024-04-17T09:59:04.446469+0000","last_became_peered":"2024-04-17T09:59:04.446469+0000","last_unstale":"2024-04-17T09:59:07.086149+0000","last_undegraded":"2024-04-17T09:59:07.086149+0000","last_fullsized":"2024-04-17T09:59:07.086149+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:24:15.282702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426310+0000","last_change":"2024-04-17T09:59:05.437799+0000","last_active":"2024-04-17T09:59:06.426310+0000","last_peered":"2024-04-17T09:59:06.426310+0000","last_clean":"2024-04-17T09:59:06.426310+0000","last_became_active":"2024-04-17T09:59:05.436081+0000","last_became_peered":"2024-04-17T09:59:05.436081+0000","last_unstale":"2024-04-17T09:59:06.426310+0000","last_undegraded":"2024-04-17T09:59:06.426310+0000","last_fullsized":"2024-04-17T09:59:06.426310+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:50.293627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"81'8","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455171+0000","last_change":"2024-04-17T09:59:04.438156+0000","last_active":"2024-04-17T09:59:06.455171+0000","last_peered":"2024-04-17T09:59:06.455171+0000","last_clean":"2024-04-17T09:59:06.455171+0000","last_became_active":"2024-04-17T09:59:04.437821+0000","last_became_peered":"2024-04-17T09:59:04.437821+0000","last_unstale":"2024-04-17T09:59:06.455171+0000","last_undegraded":"2024-04-17T09:59:06.455171+0000","last_fullsized":"2024-04-17T09:59:06.455171+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:54.250176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.085836+0000","last_change":"2024-04-17T09:59:05.449902+0000","last_active":"2024-04-17T09:59:07.085836+0000","last_peered":"2024-04-17T09:59:07.085836+0000","last_clean":"2024-04-17T09:59:07.085836+0000","last_became_active":"2024-04-17T09:59:05.449666+0000","last_became_peered":"2024-04-17T09:59:05.449666+0000","last_unstale":"2024-04-17T09:59:07.085836+0000","last_undegraded":"2024-04-17T09:59:07.085836+0000","last_fullsized":"2024-04-17T09:59:07.085836+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:03:09.174464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427834+0000","last_change":"2024-04-17T09:59:04.447265+0000","last_active":"2024-04-17T09:59:06.427834+0000","last_peered":"2024-04-17T09:59:06.427834+0000","last_clean":"2024-04-17T09:59:06.427834+0000","last_became_active":"2024-04-17T09:59:04.446847+0000","last_became_peered":"2024-04-17T09:59:04.446847+0000","last_unstale":"2024-04-17T09:59:06.427834+0000","last_undegraded":"2024-04-17T09:59:06.427834+0000","last_fullsized":"2024-04-17T09:59:06.427834+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:04.508306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427516+0000","last_change":"2024-04-17T09:59:05.440839+0000","last_active":"2024-04-17T09:59:06.427516+0000","last_peered":"2024-04-17T09:59:06.427516+0000","last_clean":"2024-04-17T09:59:06.427516+0000","last_became_active":"2024-04-17T09:59:05.440559+0000","last_became_peered":"2024-04-17T09:59:05.440559+0000","last_unstale":"2024-04-17T09:59:06.427516+0000","last_undegraded":"2024-04-17T09:59:06.427516+0000","last_fullsized":"2024-04-17T09:59:06.427516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:04:06.129152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455137+0000","last_change":"2024-04-17T09:59:04.443540+0000","last_active":"2024-04-17T09:59:06.455137+0000","last_peered":"2024-04-17T09:59:06.455137+0000","last_clean":"2024-04-17T09:59:06.455137+0000","last_became_active":"2024-04-17T09:59:04.443277+0000","last_became_peered":"2024-04-17T09:59:04.443277+0000","last_unstale":"2024-04-17T09:59:06.455137+0000","last_undegraded":"2024-04-17T09:59:06.455137+0000","last_fullsized":"2024-04-17T09:59:06.455137+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:10.371001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427822+0000","last_change":"2024-04-17T09:59:05.447239+0000","last_active":"2024-04-17T09:59:06.427822+0000","last_peered":"2024-04-17T09:59:06.427822+0000","last_clean":"2024-04-17T09:59:06.427822+0000","last_became_active":"2024-04-17T09:59:05.446970+0000","last_became_peered":"2024-04-17T09:59:05.446970+0000","last_unstale":"2024-04-17T09:59:06.427822+0000","last_undegraded":"2024-04-17T09:59:06.427822+0000","last_fullsized":"2024-04-17T09:59:06.427822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:02:07.703202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455269+0000","last_change":"2024-04-17T09:59:04.445958+0000","last_active":"2024-04-17T09:59:06.455269+0000","last_peered":"2024-04-17T09:59:06.455269+0000","last_clean":"2024-04-17T09:59:06.455269+0000","last_became_active":"2024-04-17T09:59:04.445632+0000","last_became_peered":"2024-04-17T09:59:04.445632+0000","last_unstale":"2024-04-17T09:59:06.455269+0000","last_undegraded":"2024-04-17T09:59:06.455269+0000","last_fullsized":"2024-04-17T09:59:06.455269+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:57.288434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.087048+0000","last_change":"2024-04-17T09:59:05.447700+0000","last_active":"2024-04-17T09:59:07.087048+0000","last_peered":"2024-04-17T09:59:07.087048+0000","last_clean":"2024-04-17T09:59:07.087048+0000","last_became_active":"2024-04-17T09:59:05.447040+0000","last_became_peered":"2024-04-17T09:59:05.447040+0000","last_unstale":"2024-04-17T09:59:07.087048+0000","last_undegraded":"2024-04-17T09:59:07.087048+0000","last_fullsized":"2024-04-17T09:59:07.087048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:28:27.208286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.507971+0000","last_change":"2024-04-17T09:59:04.445248+0000","last_active":"2024-04-17T09:59:06.507971+0000","last_peered":"2024-04-17T09:59:06.507971+0000","last_clean":"2024-04-17T09:59:06.507971+0000","last_became_active":"2024-04-17T09:59:04.444809+0000","last_became_peered":"2024-04-17T09:59:04.444809+0000","last_unstale":"2024-04-17T09:59:06.507971+0000","last_undegraded":"2024-04-17T09:59:06.507971+0000","last_fullsized":"2024-04-17T09:59:06.507971+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:10:21.942544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.086065+0000","last_change":"2024-04-17T09:59:05.448582+0000","last_active":"2024-04-17T09:59:07.086065+0000","last_peered":"2024-04-17T09:59:07.086065+0000","last_clean":"2024-04-17T09:59:07.086065+0000","last_became_active":"2024-04-17T09:59:05.446877+0000","last_became_peered":"2024-04-17T09:59:05.446877+0000","last_unstale":"2024-04-17T09:59:07.086065+0000","last_undegraded":"2024-04-17T09:59:07.086065+0000","last_fullsized":"2024-04-17T09:59:07.086065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:01:25.939664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.087146+0000","last_change":"2024-04-17T09:59:04.448116+0000","last_active":"2024-04-17T09:59:07.087146+0000","last_peered":"2024-04-17T09:59:07.087146+0000","last_clean":"2024-04-17T09:59:07.087146+0000","last_became_active":"2024-04-17T09:59:04.446755+0000","last_became_peered":"2024-04-17T09:59:04.446755+0000","last_unstale":"2024-04-17T09:59:07.087146+0000","last_undegraded":"2024-04-17T09:59:07.087146+0000","last_fullsized":"2024-04-17T09:59:07.087146+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:35:42.072291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.086677+0000","last_change":"2024-04-17T09:59:05.454703+0000","last_active":"2024-04-17T09:59:07.086677+0000","last_peered":"2024-04-17T09:59:07.086677+0000","last_clean":"2024-04-17T09:59:07.086677+0000","last_became_active":"2024-04-17T09:59:05.454428+0000","last_became_peered":"2024-04-17T09:59:05.454428+0000","last_unstale":"2024-04-17T09:59:07.086677+0000","last_undegraded":"2024-04-17T09:59:07.086677+0000","last_fullsized":"2024-04-17T09:59:07.086677+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:05:56.668977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.086638+0000","last_change":"2024-04-17T09:59:04.448303+0000","last_active":"2024-04-17T09:59:07.086638+0000","last_peered":"2024-04-17T09:59:07.086638+0000","last_clean":"2024-04-17T09:59:07.086638+0000","last_became_active":"2024-04-17T09:59:04.446032+0000","last_became_peered":"2024-04-17T09:59:04.446032+0000","last_unstale":"2024-04-17T09:59:07.086638+0000","last_undegraded":"2024-04-17T09:59:07.086638+0000","last_fullsized":"2024-04-17T09:59:07.086638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:16:02.692659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.086043+0000","last_change":"2024-04-17T09:59:05.448865+0000","last_active":"2024-04-17T09:59:07.086043+0000","last_peered":"2024-04-17T09:59:07.086043+0000","last_clean":"2024-04-17T09:59:07.086043+0000","last_became_active":"2024-04-17T09:59:05.448286+0000","last_became_peered":"2024-04-17T09:59:05.448286+0000","last_unstale":"2024-04-17T09:59:07.086043+0000","last_undegraded":"2024-04-17T09:59:07.086043+0000","last_fullsized":"2024-04-17T09:59:07.086043+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:29:46.897945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.507815+0000","last_change":"2024-04-17T09:59:04.445983+0000","last_active":"2024-04-17T09:59:06.507815+0000","last_peered":"2024-04-17T09:59:06.507815+0000","last_clean":"2024-04-17T09:59:06.507815+0000","last_became_active":"2024-04-17T09:59:04.445671+0000","last_became_peered":"2024-04-17T09:59:04.445671+0000","last_unstale":"2024-04-17T09:59:06.507815+0000","last_undegraded":"2024-04-17T09:59:06.507815+0000","last_fullsized":"2024-04-17T09:59:06.507815+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:04:05.123127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.428075+0000","last_change":"2024-04-17T09:59:05.449697+0000","last_active":"2024-04-17T09:59:06.428075+0000","last_peered":"2024-04-17T09:59:06.428075+0000","last_clean":"2024-04-17T09:59:06.428075+0000","last_became_active":"2024-04-17T09:59:05.449390+0000","last_became_peered":"2024-04-17T09:59:05.449390+0000","last_unstale":"2024-04-17T09:59:06.428075+0000","last_undegraded":"2024-04-17T09:59:06.428075+0000","last_fullsized":"2024-04-17T09:59:06.428075+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:11:22.087912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427001+0000","last_change":"2024-04-17T09:59:04.442441+0000","last_active":"2024-04-17T09:59:06.427001+0000","last_peered":"2024-04-17T09:59:06.427001+0000","last_clean":"2024-04-17T09:59:06.427001+0000","last_became_active":"2024-04-17T09:59:04.442051+0000","last_became_peered":"2024-04-17T09:59:04.442051+0000","last_unstale":"2024-04-17T09:59:06.427001+0000","last_undegraded":"2024-04-17T09:59:06.427001+0000","last_fullsized":"2024-04-17T09:59:06.427001+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:50:21.847842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.085682+0000","last_change":"2024-04-17T09:59:05.449493+0000","last_active":"2024-04-17T09:59:07.085682+0000","last_peered":"2024-04-17T09:59:07.085682+0000","last_clean":"2024-04-17T09:59:07.085682+0000","last_became_active":"2024-04-17T09:59:05.449169+0000","last_became_peered":"2024-04-17T09:59:05.449169+0000","last_unstale":"2024-04-17T09:59:07.085682+0000","last_undegraded":"2024-04-17T09:59:07.085682+0000","last_fullsized":"2024-04-17T09:59:07.085682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:17:45.989148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426923+0000","last_change":"2024-04-17T09:59:04.441305+0000","last_active":"2024-04-17T09:59:06.426923+0000","last_peered":"2024-04-17T09:59:06.426923+0000","last_clean":"2024-04-17T09:59:06.426923+0000","last_became_active":"2024-04-17T09:59:04.441044+0000","last_became_peered":"2024-04-17T09:59:04.441044+0000","last_unstale":"2024-04-17T09:59:06.426923+0000","last_undegraded":"2024-04-17T09:59:06.426923+0000","last_fullsized":"2024-04-17T09:59:06.426923+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:46:11.745821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427404+0000","last_change":"2024-04-17T09:59:05.446566+0000","last_active":"2024-04-17T09:59:06.427404+0000","last_peered":"2024-04-17T09:59:06.427404+0000","last_clean":"2024-04-17T09:59:06.427404+0000","last_became_active":"2024-04-17T09:59:05.446293+0000","last_became_peered":"2024-04-17T09:59:05.446293+0000","last_unstale":"2024-04-17T09:59:06.427404+0000","last_undegraded":"2024-04-17T09:59:06.427404+0000","last_fullsized":"2024-04-17T09:59:06.427404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:37.066790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427205+0000","last_change":"2024-04-17T09:59:04.442494+0000","last_active":"2024-04-17T09:59:06.427205+0000","last_peered":"2024-04-17T09:59:06.427205+0000","last_clean":"2024-04-17T09:59:06.427205+0000","last_became_active":"2024-04-17T09:59:04.442254+0000","last_became_peered":"2024-04-17T09:59:04.442254+0000","last_unstale":"2024-04-17T09:59:06.427205+0000","last_undegraded":"2024-04-17T09:59:06.427205+0000","last_fullsized":"2024-04-17T09:59:06.427205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:17:51.424754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427862+0000","last_change":"2024-04-17T09:59:05.438162+0000","last_active":"2024-04-17T09:59:06.427862+0000","last_peered":"2024-04-17T09:59:06.427862+0000","last_clean":"2024-04-17T09:59:06.427862+0000","last_became_active":"2024-04-17T09:59:05.437880+0000","last_became_peered":"2024-04-17T09:59:05.437880+0000","last_unstale":"2024-04-17T09:59:06.427862+0000","last_undegraded":"2024-04-17T09:59:06.427862+0000","last_fullsized":"2024-04-17T09:59:06.427862+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:58.294460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.427470+0000","last_change":"2024-04-17T09:59:04.444713+0000","last_active":"2024-04-17T09:59:06.427470+0000","last_peered":"2024-04-17T09:59:06.427470+0000","last_clean":"2024-04-17T09:59:06.427470+0000","last_became_active":"2024-04-17T09:59:04.444438+0000","last_became_peered":"2024-04-17T09:59:04.444438+0000","last_unstale":"2024-04-17T09:59:06.427470+0000","last_undegraded":"2024-04-17T09:59:06.427470+0000","last_fullsized":"2024-04-17T09:59:06.427470+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:24:15.282702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.086301+0000","last_change":"2024-04-17T09:59:05.452454+0000","last_active":"2024-04-17T09:59:07.086301+0000","last_peered":"2024-04-17T09:59:07.086301+0000","last_clean":"2024-04-17T09:59:07.086301+0000","last_became_active":"2024-04-17T09:59:05.451560+0000","last_became_peered":"2024-04-17T09:59:05.451560+0000","last_unstale":"2024-04-17T09:59:07.086301+0000","last_undegraded":"2024-04-17T09:59:07.086301+0000","last_fullsized":"2024-04-17T09:59:07.086301+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:45:30.252915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455124+0000","last_change":"2024-04-17T09:59:04.438627+0000","last_active":"2024-04-17T09:59:06.455124+0000","last_peered":"2024-04-17T09:59:06.455124+0000","last_clean":"2024-04-17T09:59:06.455124+0000","last_became_active":"2024-04-17T09:59:04.438100+0000","last_became_peered":"2024-04-17T09:59:04.438100+0000","last_unstale":"2024-04-17T09:59:06.455124+0000","last_undegraded":"2024-04-17T09:59:06.455124+0000","last_fullsized":"2024-04-17T09:59:06.455124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:53:55.006094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.455088+0000","last_change":"2024-04-17T09:59:05.447417+0000","last_active":"2024-04-17T09:59:06.455088+0000","last_peered":"2024-04-17T09:59:06.455088+0000","last_clean":"2024-04-17T09:59:06.455088+0000","last_became_active":"2024-04-17T09:59:05.447176+0000","last_became_peered":"2024-04-17T09:59:05.447176+0000","last_unstale":"2024-04-17T09:59:06.455088+0000","last_undegraded":"2024-04-17T09:59:06.455088+0000","last_fullsized":"2024-04-17T09:59:06.455088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:18:04.685528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426694+0000","last_change":"2024-04-17T09:59:04.440315+0000","last_active":"2024-04-17T09:59:06.426694+0000","last_peered":"2024-04-17T09:59:06.426694+0000","last_clean":"2024-04-17T09:59:06.426694+0000","last_became_active":"2024-04-17T09:59:04.439614+0000","last_became_peered":"2024-04-17T09:59:04.439614+0000","last_unstale":"2024-04-17T09:59:06.426694+0000","last_undegraded":"2024-04-17T09:59:06.426694+0000","last_fullsized":"2024-04-17T09:59:06.426694+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:54:02.341478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.085260+0000","last_change":"2024-04-17T09:59:05.448748+0000","last_active":"2024-04-17T09:59:07.085260+0000","last_peered":"2024-04-17T09:59:07.085260+0000","last_clean":"2024-04-17T09:59:07.085260+0000","last_became_active":"2024-04-17T09:59:05.448478+0000","last_became_peered":"2024-04-17T09:59:05.448478+0000","last_unstale":"2024-04-17T09:59:07.085260+0000","last_undegraded":"2024-04-17T09:59:07.085260+0000","last_fullsized":"2024-04-17T09:59:07.085260+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:45.635965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426726+0000","last_change":"2024-04-17T09:59:04.442633+0000","last_active":"2024-04-17T09:59:06.426726+0000","last_peered":"2024-04-17T09:59:06.426726+0000","last_clean":"2024-04-17T09:59:06.426726+0000","last_became_active":"2024-04-17T09:59:04.442347+0000","last_became_peered":"2024-04-17T09:59:04.442347+0000","last_unstale":"2024-04-17T09:59:06.426726+0000","last_undegraded":"2024-04-17T09:59:06.426726+0000","last_fullsized":"2024-04-17T09:59:06.426726+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:22:33.540833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426598+0000","last_change":"2024-04-17T09:59:05.437441+0000","last_active":"2024-04-17T09:59:06.426598+0000","last_peered":"2024-04-17T09:59:06.426598+0000","last_clean":"2024-04-17T09:59:06.426598+0000","last_became_active":"2024-04-17T09:59:05.436624+0000","last_became_peered":"2024-04-17T09:59:05.436624+0000","last_unstale":"2024-04-17T09:59:06.426598+0000","last_undegraded":"2024-04-17T09:59:06.426598+0000","last_fullsized":"2024-04-17T09:59:06.426598+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:09:18.732149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426835+0000","last_change":"2024-04-17T09:59:04.440613+0000","last_active":"2024-04-17T09:59:06.426835+0000","last_peered":"2024-04-17T09:59:06.426835+0000","last_clean":"2024-04-17T09:59:06.426835+0000","last_became_active":"2024-04-17T09:59:04.440063+0000","last_became_peered":"2024-04-17T09:59:04.440063+0000","last_unstale":"2024-04-17T09:59:06.426835+0000","last_undegraded":"2024-04-17T09:59:06.426835+0000","last_fullsized":"2024-04-17T09:59:06.426835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:53:08.846256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:06.426529+0000","last_change":"2024-04-17T09:59:05.451753+0000","last_active":"2024-04-17T09:59:06.426529+0000","last_peered":"2024-04-17T09:59:06.426529+0000","last_clean":"2024-04-17T09:59:06.426529+0000","last_became_active":"2024-04-17T09:59:05.451567+0000","last_became_peered":"2024-04-17T09:59:05.451567+0000","last_unstale":"2024-04-17T09:59:06.426529+0000","last_undegraded":"2024-04-17T09:59:06.426529+0000","last_fullsized":"2024-04-17T09:59:06.426529+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:04.411847+0000","last_clean_scrub_stamp":"2024-04-17T09:59:04.411847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:24:16.288727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.085758+0000","last_change":"2024-04-17T09:59:04.445356+0000","last_active":"2024-04-17T09:59:07.085758+0000","last_peered":"2024-04-17T09:59:07.085758+0000","last_clean":"2024-04-17T09:59:07.085758+0000","last_became_active":"2024-04-17T09:59:04.445010+0000","last_became_peered":"2024-04-17T09:59:04.445010+0000","last_unstale":"2024-04-17T09:59:07.085758+0000","last_undegraded":"2024-04-17T09:59:07.085758+0000","last_fullsized":"2024-04-17T09:59:07.085758+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:59:03.405821+0000","last_clean_scrub_stamp":"2024-04-17T09:59:03.405821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:49:48.658044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":284,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-17T09:59:07.085402+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T09:59:07.085402+0000","last_peered":"2024-04-17T09:59:07.085402+0000","last_clean":"2024-04-17T09:59:07.085402+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T09:59:07.085402+0000","last_undegraded":"2024-04-17T09:59:07.085402+0000","last_fullsized":"2024-04-17T09:59:07.085402+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509517,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1597440,"data_stored":1015223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509518,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1011712,"data_stored":425479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509517,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1015808,"data_stored":426419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509517,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1011712,"data_stored":426397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509518,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":1003520,"data_stored":425367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509517,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1011712,"data_stored":424979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542221,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":424785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542221,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36900,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334940,"statfs":{"total":96636764160,"available":96598978560,"internally_reserved":0,"allocated":1597440,"data_stored":1015187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T09:59:14.516 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T09:59:14.516 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-17T09:59:14.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.706+0000 7f21b3e00640 1 Processor -- start 2024-04-17T09:59:14.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.706+0000 7f21b3e00640 1 -- start start 2024-04-17T09:59:14.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.706+0000 7f21b3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f21ac15ac30 0x7f21ac06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:14.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.706+0000 7f21b3e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21ac0b3d80 0x7f21ac071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:14.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.706+0000 7f21b3e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21ac06fcd0 con 0x7f21ac15b010 2024-04-17T09:59:14.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.706+0000 7f21b3e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f21ac06f300 con 0x7f21ac15ac30 2024-04-17T09:59:14.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.706+0000 7f21b3e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f21ac06e6f0 con 0x7f21ac0b3d80 2024-04-17T09:59:14.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.706+0000 7f21b1400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f21ac15ac30 0x7f21ac06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:14.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.706+0000 7f21b0a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f21ac15b010 0x7f21ac06dea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52084/0 (socket says 172.21.15.32:52084) 2024-04-17T09:59:14.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.706+0000 7f21b0a00640 1 -- 172.21.15.32:0/2029035005 learned_addr learned my addr 172.21.15.32:0/2029035005 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:14.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.706+0000 7f21b1e00640 1 --2- 172.21.15.32:0/2029035005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21ac0b3d80 0x7f21ac071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:14.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.706+0000 7f21b1400640 1 -- 172.21.15.32:0/2029035005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21ac0b3d80 msgr2=0x7f21ac071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:14.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.710+0000 7f21b1400640 1 --2- 172.21.15.32:0/2029035005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21ac0b3d80 0x7f21ac071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:14.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.710+0000 7f21b1400640 1 -- 172.21.15.32:0/2029035005 >> v1:172.21.15.32:6789/0 conn(0x7f21ac15b010 legacy=0x7f21ac06dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:14.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.710+0000 7f21b1400640 1 -- 172.21.15.32:0/2029035005 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ac06e9d0 con 0x7f21ac15ac30 2024-04-17T09:59:14.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.710+0000 7f21abe00640 1 -- 172.21.15.32:0/2029035005 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3509020164 0 0) 0x7f21ac06fcd0 con 0x7f21ac15b010 2024-04-17T09:59:14.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.710+0000 7f21b1e00640 1 --2- 172.21.15.32:0/2029035005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21ac0b3d80 0x7f21ac071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:14.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.710+0000 7f21b1400640 1 --2- 172.21.15.32:0/2029035005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21ac15ac30 0x7f21ac06f790 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f21a00097c0 tx=0x7f21a001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7406620ece331b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:14.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.710+0000 7f21abe00640 1 -- 172.21.15.32:0/2029035005 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a00b0070 con 0x7f21ac15ac30 2024-04-17T09:59:14.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.710+0000 7f21abe00640 1 -- 172.21.15.32:0/2029035005 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21a00ab600 con 0x7f21ac15ac30 2024-04-17T09:59:14.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.710+0000 7f21abe00640 1 -- 172.21.15.32:0/2029035005 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a00ab9d0 con 0x7f21ac15ac30 2024-04-17T09:59:14.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.710+0000 7f21b3e00640 1 -- 172.21.15.32:0/2029035005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21ac15ac30 msgr2=0x7f21ac06f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:14.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.710+0000 7f21b3e00640 1 --2- 172.21.15.32:0/2029035005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21ac15ac30 0x7f21ac06f790 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f21a00097c0 tx=0x7f21a001c4a0 comp rx=0 tx=0).stop 2024-04-17T09:59:14.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.710+0000 7f21b3e00640 1 -- 172.21.15.32:0/2029035005 shutdown_connections 2024-04-17T09:59:14.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.710+0000 7f21b3e00640 1 --2- 172.21.15.32:0/2029035005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21ac0b3d80 0x7f21ac071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:14.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.710+0000 7f21b3e00640 1 --2- 172.21.15.32:0/2029035005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21ac15ac30 0x7f21ac06f790 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:14.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.710+0000 7f21b3e00640 1 -- 172.21.15.32:0/2029035005 >> 172.21.15.32:0/2029035005 conn(0x7f21ac0b2120 msgr2=0x7f21ac0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:14.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.710+0000 7f21b3e00640 1 -- 172.21.15.32:0/2029035005 shutdown_connections 2024-04-17T09:59:14.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.710+0000 7f21b3e00640 1 -- 172.21.15.32:0/2029035005 wait complete. 2024-04-17T09:59:14.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.714+0000 7f21b3e00640 1 Processor -- start 2024-04-17T09:59:14.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.714+0000 7f21b3e00640 1 -- start start 2024-04-17T09:59:14.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.714+0000 7f21b3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21ac0b3d80 0x7f21ac1520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:14.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.714+0000 7f21b3e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21ac15ac30 0x7f21ac154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:14.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.714+0000 7f21b3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21ac15b010 0x7f21ac15d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:14.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.714+0000 7f21b3e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f21ac0cdd50 con 0x7f21ac0b3d80 2024-04-17T09:59:14.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.714+0000 7f21b3e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f21ac0cdbd0 con 0x7f21ac15b010 2024-04-17T09:59:14.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.714+0000 7f21b3e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f21ac0cded0 con 0x7f21ac15ac30 2024-04-17T09:59:14.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.714+0000 7f21b1400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21ac0b3d80 0x7f21ac1520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:14.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.714+0000 7f21b1400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21ac0b3d80 0x7f21ac1520f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38702/0 (socket says 172.21.15.32:38702) 2024-04-17T09:59:14.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.714+0000 7f21b1400640 1 -- 172.21.15.32:0/829717496 learned_addr learned my addr 172.21.15.32:0/829717496 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:14.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.714+0000 7f21b1400640 1 -- 172.21.15.32:0/829717496 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21ac15ac30 msgr2=0x7f21ac154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:14.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.714+0000 7f21b1400640 1 --2- 172.21.15.32:0/829717496 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21ac15ac30 0x7f21ac154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:14.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.714+0000 7f21b1400640 1 -- 172.21.15.32:0/829717496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21ac15b010 msgr2=0x7f21ac15d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:14.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.714+0000 7f21b1400640 1 --2- 172.21.15.32:0/829717496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21ac15b010 0x7f21ac15d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:14.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.714+0000 7f21b1400640 1 -- 172.21.15.32:0/829717496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ac15dbc0 con 0x7f21ac0b3d80 2024-04-17T09:59:14.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.714+0000 7f21b1400640 1 --2- 172.21.15.32:0/829717496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21ac0b3d80 0x7f21ac1520f0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f21a00097c0 tx=0x7f21a0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:14.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.714+0000 7f21a9600640 1 -- 172.21.15.32:0/829717496 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a00b0040 con 0x7f21ac0b3d80 2024-04-17T09:59:14.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.714+0000 7f21a9600640 1 -- 172.21.15.32:0/829717496 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21a00ba070 con 0x7f21ac0b3d80 2024-04-17T09:59:14.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.714+0000 7f21b3e00640 1 -- 172.21.15.32:0/829717496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21ac15dea0 con 0x7f21ac0b3d80 2024-04-17T09:59:14.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.714+0000 7f21a9600640 1 -- 172.21.15.32:0/829717496 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a00b4650 con 0x7f21ac0b3d80 2024-04-17T09:59:14.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.718+0000 7f21b3e00640 1 -- 172.21.15.32:0/829717496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21ac1f7c70 con 0x7f21ac0b3d80 2024-04-17T09:59:14.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.718+0000 7f21b3e00640 1 -- 172.21.15.32:0/829717496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21ac068a90 con 0x7f21ac0b3d80 2024-04-17T09:59:14.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.718+0000 7f21a9600640 1 -- 172.21.15.32:0/829717496 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f21a00b47f0 con 0x7f21ac0b3d80 2024-04-17T09:59:14.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.718+0000 7f21a9600640 1 --2- 172.21.15.32:0/829717496 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f218c068380 0x7f218c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:14.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.718+0000 7f21a9600640 1 -- 172.21.15.32:0/829717496 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f21a0120630 con 0x7f21ac0b3d80 2024-04-17T09:59:14.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.718+0000 7f21b0a00640 1 --2- 172.21.15.32:0/829717496 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f218c068380 0x7f218c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:14.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.718+0000 7f21b0a00640 1 --2- 172.21.15.32:0/829717496 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f218c068380 0x7f218c06a840 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0b3b80 tx=0x7f2198007970 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:14.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.722+0000 7f21a9600640 1 -- 172.21.15.32:0/829717496 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f21a0016090 con 0x7f21ac0b3d80 2024-04-17T09:59:14.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.942+0000 7f21b3e00640 1 -- 172.21.15.32:0/829717496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f21ac0cfd00 con 0x7f21ac0b3d80 2024-04-17T09:59:14.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.946+0000 7f21a9600640 1 -- 172.21.15.32:0/829717496 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7f21a00f0b50 con 0x7f21ac0b3d80 2024-04-17T09:59:14.944 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-17T09:59:14.944 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDPmx9mUXoLKxAANV7zhLvL0DqHngk4TtsdBQ== 2024-04-17T09:59:14.945 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2024-04-17T09:59:14.945 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2024-04-17T09:59:14.945 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-17T09:59:14.945 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2024-04-17T09:59:14.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.950+0000 7f21b3e00640 1 -- 172.21.15.32:0/829717496 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f218c068380 msgr2=0x7f218c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:14.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.950+0000 7f21b3e00640 1 --2- 172.21.15.32:0/829717496 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f218c068380 0x7f218c06a840 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0b3b80 tx=0x7f2198007970 comp rx=0 tx=0).stop 2024-04-17T09:59:14.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.950+0000 7f21b3e00640 1 -- 172.21.15.32:0/829717496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21ac0b3d80 msgr2=0x7f21ac1520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:14.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.950+0000 7f21b3e00640 1 --2- 172.21.15.32:0/829717496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21ac0b3d80 0x7f21ac1520f0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f21a00097c0 tx=0x7f21a0019240 comp rx=0 tx=0).stop 2024-04-17T09:59:14.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.950+0000 7f21b3e00640 1 -- 172.21.15.32:0/829717496 shutdown_connections 2024-04-17T09:59:14.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.950+0000 7f21b3e00640 1 --2- 172.21.15.32:0/829717496 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f218c068380 0x7f218c06a840 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:14.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.950+0000 7f21b3e00640 1 --2- 172.21.15.32:0/829717496 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21ac15ac30 0x7f21ac154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:14.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.950+0000 7f21b3e00640 1 --2- 172.21.15.32:0/829717496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21ac15b010 0x7f21ac15d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:14.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.950+0000 7f21b3e00640 1 --2- 172.21.15.32:0/829717496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21ac0b3d80 0x7f21ac1520f0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:14.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.950+0000 7f21b3e00640 1 -- 172.21.15.32:0/829717496 >> 172.21.15.32:0/829717496 conn(0x7f21ac0b2120 msgr2=0x7f21ac15a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:14.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.950+0000 7f21b3e00640 1 -- 172.21.15.32:0/829717496 shutdown_connections 2024-04-17T09:59:14.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:14.950+0000 7f21b3e00640 1 -- 172.21.15.32:0/829717496 wait complete. 2024-04-17T09:59:14.966 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:59:15.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.110+0000 7f2a6b600640 1 Processor -- start 2024-04-17T09:59:15.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.110+0000 7f2a6b600640 1 -- start start 2024-04-17T09:59:15.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.114+0000 7f2a6b600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2a640cd580 0x7f2a64167560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:15.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.114+0000 7f2a6b600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a640cdac0 con 0x7f2a64072500 2024-04-17T09:59:15.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.114+0000 7f2a6b600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a6415cdf0 con 0x7f2a64145f30 2024-04-17T09:59:15.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.114+0000 7f2a6b600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f2a6406f400 con 0x7f2a640cd580 2024-04-17T09:59:15.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.114+0000 7f2a5be00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2a64072500 0x7f2a640cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52086/0 (socket says 172.21.15.32:52086) 2024-04-17T09:59:15.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.114+0000 7f2a5be00640 1 -- 172.21.15.32:0/3720141154 learned_addr learned my addr 172.21.15.32:0/3720141154 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:15.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.114+0000 7f2a69600640 1 --2- 172.21.15.32:0/3720141154 >> v2:172.21.15.144:3300/0 conn(0x7f2a640cd580 0x7f2a64167560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:15.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.114+0000 7f2a63e00640 1 -- 172.21.15.32:0/3720141154 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2955806855 0 0) 0x7f2a640cdac0 con 0x7f2a64072500 2024-04-17T09:59:15.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.114+0000 7f2a63e00640 1 -- 172.21.15.32:0/3720141154 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a48003530 con 0x7f2a64072500 2024-04-17T09:59:15.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.114+0000 7f2a63e00640 1 -- 172.21.15.32:0/3720141154 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 466802393 0 0) 0x7f2a6415cdf0 con 0x7f2a64145f30 2024-04-17T09:59:15.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.114+0000 7f2a63e00640 1 -- 172.21.15.32:0/3720141154 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a640cdac0 con 0x7f2a64145f30 2024-04-17T09:59:15.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.114+0000 7f2a69600640 1 -- 172.21.15.32:0/3720141154 >> v1:172.21.15.144:6789/0 conn(0x7f2a64145f30 legacy=0x7f2a64071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:15.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.114+0000 7f2a63e00640 1 -- 172.21.15.32:0/3720141154 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3639064376 0 0) 0x7f2a48003530 con 0x7f2a64072500 2024-04-17T09:59:15.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.114+0000 7f2a69600640 1 -- 172.21.15.32:0/3720141154 >> v1:172.21.15.32:6789/0 conn(0x7f2a64072500 legacy=0x7f2a640cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:15.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.114+0000 7f2a69600640 1 -- 172.21.15.32:0/3720141154 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a6415d0d0 con 0x7f2a640cd580 2024-04-17T09:59:15.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.118+0000 7f2a69600640 1 --2- 172.21.15.32:0/3720141154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a640cd580 0x7f2a64167560 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009d90 tx=0x7f2a5c01f080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=877d20ed32072207 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.118+0000 7f2a63e00640 1 -- 172.21.15.32:0/3720141154 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c0b5070 con 0x7f2a640cd580 2024-04-17T09:59:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.118+0000 7f2a63e00640 1 -- 172.21.15.32:0/3720141154 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a5c0a8c10 con 0x7f2a640cd580 2024-04-17T09:59:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.118+0000 7f2a63e00640 1 -- 172.21.15.32:0/3720141154 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c001680 con 0x7f2a640cd580 2024-04-17T09:59:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.118+0000 7f2a6b600640 1 -- 172.21.15.32:0/3720141154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a640cd580 msgr2=0x7f2a64167560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.118+0000 7f2a6b600640 1 --2- 172.21.15.32:0/3720141154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a640cd580 0x7f2a64167560 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009d90 tx=0x7f2a5c01f080 comp rx=0 tx=0).stop 2024-04-17T09:59:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.118+0000 7f2a6b600640 1 -- 172.21.15.32:0/3720141154 shutdown_connections 2024-04-17T09:59:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.118+0000 7f2a6b600640 1 --2- 172.21.15.32:0/3720141154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a640cd580 0x7f2a64167560 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.118+0000 7f2a6b600640 1 -- 172.21.15.32:0/3720141154 >> 172.21.15.32:0/3720141154 conn(0x7f2a6406ba30 msgr2=0x7f2a6406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.118+0000 7f2a6b600640 1 -- 172.21.15.32:0/3720141154 shutdown_connections 2024-04-17T09:59:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.118+0000 7f2a6b600640 1 -- 172.21.15.32:0/3720141154 wait complete. 2024-04-17T09:59:15.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.118+0000 7f2a6b600640 1 Processor -- start 2024-04-17T09:59:15.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.118+0000 7f2a6b600640 1 -- start start 2024-04-17T09:59:15.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.118+0000 7f2a6b600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a64072500 0x7f2a64140e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:15.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.118+0000 7f2a6b600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a640cd580 0x7f2a640b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:15.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.118+0000 7f2a6b600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a64145f30 0x7f2a640ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:15.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.118+0000 7f2a6b600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2a64146310 con 0x7f2a64072500 2024-04-17T09:59:15.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.118+0000 7f2a6b600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6415dba0 con 0x7f2a640cd580 2024-04-17T09:59:15.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.118+0000 7f2a6b600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2a6415d440 con 0x7f2a64145f30 2024-04-17T09:59:15.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.122+0000 7f2a68c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a64072500 0x7f2a64140e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:15.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.122+0000 7f2a68c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a64072500 0x7f2a64140e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38714/0 (socket says 172.21.15.32:38714) 2024-04-17T09:59:15.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.122+0000 7f2a68c00640 1 -- 172.21.15.32:0/4177055184 learned_addr learned my addr 172.21.15.32:0/4177055184 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:15.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.122+0000 7f2a69600640 1 --2- 172.21.15.32:0/4177055184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a64145f30 0x7f2a640ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:15.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.122+0000 7f2a5be00640 1 --2- 172.21.15.32:0/4177055184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a640cd580 0x7f2a640b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:15.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.122+0000 7f2a68c00640 1 -- 172.21.15.32:0/4177055184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a64145f30 msgr2=0x7f2a640ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:15.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.122+0000 7f2a68c00640 1 --2- 172.21.15.32:0/4177055184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a64145f30 0x7f2a640ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:15.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.122+0000 7f2a68c00640 1 -- 172.21.15.32:0/4177055184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a640cd580 msgr2=0x7f2a640b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:15.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.122+0000 7f2a68c00640 1 --2- 172.21.15.32:0/4177055184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a640cd580 0x7f2a640b4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:15.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.122+0000 7f2a68c00640 1 -- 172.21.15.32:0/4177055184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a641f51b0 con 0x7f2a64072500 2024-04-17T09:59:15.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.122+0000 7f2a68c00640 1 --2- 172.21.15.32:0/4177055184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a64072500 0x7f2a64140e50 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400c430 tx=0x7f2a5400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:15.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.122+0000 7f2a61600640 1 -- 172.21.15.32:0/4177055184 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a54017070 con 0x7f2a64072500 2024-04-17T09:59:15.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.122+0000 7f2a61600640 1 -- 172.21.15.32:0/4177055184 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a540052f0 con 0x7f2a64072500 2024-04-17T09:59:15.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.122+0000 7f2a61600640 1 -- 172.21.15.32:0/4177055184 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a54003840 con 0x7f2a64072500 2024-04-17T09:59:15.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.122+0000 7f2a6b600640 1 -- 172.21.15.32:0/4177055184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a641f5490 con 0x7f2a64072500 2024-04-17T09:59:15.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.122+0000 7f2a6b600640 1 -- 172.21.15.32:0/4177055184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a640b5280 con 0x7f2a64072500 2024-04-17T09:59:15.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.122+0000 7f2a6b600640 1 -- 172.21.15.32:0/4177055184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a64068a90 con 0x7f2a64072500 2024-04-17T09:59:15.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.122+0000 7f2a61600640 1 -- 172.21.15.32:0/4177055184 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2a5401a070 con 0x7f2a64072500 2024-04-17T09:59:15.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.126+0000 7f2a61600640 1 --2- 172.21.15.32:0/4177055184 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2a40068380 0x7f2a4006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:15.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.126+0000 7f2a61600640 1 -- 172.21.15.32:0/4177055184 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f2a54085d30 con 0x7f2a64072500 2024-04-17T09:59:15.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.126+0000 7f2a5be00640 1 --2- 172.21.15.32:0/4177055184 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2a40068380 0x7f2a4006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:15.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.126+0000 7f2a5be00640 1 --2- 172.21.15.32:0/4177055184 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2a40068380 0x7f2a4006a840 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2a640b18a0 tx=0x7f2a4c0016a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:15.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.130+0000 7f2a61600640 1 -- 172.21.15.32:0/4177055184 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2a54055c90 con 0x7f2a64072500 2024-04-17T09:59:15.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.326+0000 7f2a6b600640 1 -- 172.21.15.32:0/4177055184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a640b4b20 con 0x7f2a64072500 2024-04-17T09:59:15.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.326+0000 7f2a61600640 1 -- 172.21.15.32:0/4177055184 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5204 (secure 0 0 0) 0x7f2a54059f90 con 0x7f2a64072500 2024-04-17T09:59:15.688 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:59:15.688 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6419,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3534108103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3534108103},{"type":"v1","addr":"172.21.15.144:6835","nonce":3534108103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7003,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/935945019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":935945019},{"type":"v1","addr":"172.21.15.144:6834","nonce":935945019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3044362393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3044362393},{"type":"v1","addr":"172.21.15.32:6836","nonce":3044362393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:59:04.834110+0000","modified":"2024-04-17T09:59:07.850260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7557":{"gid":7557,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/2656735817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2656735817},{"type":"v1","addr":"172.21.15.32:6837","nonce":2656735817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7557,"qdb_cluster":[7557]},"id":6}]} 2024-04-17T09:59:15.690 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 59 2024-04-17T09:59:15.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.330+0000 7f2a6b600640 1 -- 172.21.15.32:0/4177055184 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2a40068380 msgr2=0x7f2a4006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:15.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.330+0000 7f2a6b600640 1 --2- 172.21.15.32:0/4177055184 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2a40068380 0x7f2a4006a840 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2a640b18a0 tx=0x7f2a4c0016a0 comp rx=0 tx=0).stop 2024-04-17T09:59:15.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.330+0000 7f2a6b600640 1 -- 172.21.15.32:0/4177055184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a64072500 msgr2=0x7f2a64140e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:15.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.330+0000 7f2a6b600640 1 --2- 172.21.15.32:0/4177055184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a64072500 0x7f2a64140e50 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400c430 tx=0x7f2a5400c8f0 comp rx=0 tx=0).stop 2024-04-17T09:59:15.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.330+0000 7f2a6b600640 1 -- 172.21.15.32:0/4177055184 shutdown_connections 2024-04-17T09:59:15.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.330+0000 7f2a6b600640 1 --2- 172.21.15.32:0/4177055184 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2a40068380 0x7f2a4006a840 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:15.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.330+0000 7f2a6b600640 1 --2- 172.21.15.32:0/4177055184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a64145f30 0x7f2a640ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:15.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.330+0000 7f2a6b600640 1 --2- 172.21.15.32:0/4177055184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a640cd580 0x7f2a640b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:15.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.330+0000 7f2a6b600640 1 --2- 172.21.15.32:0/4177055184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a64072500 0x7f2a64140e50 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:15.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.330+0000 7f2a6b600640 1 -- 172.21.15.32:0/4177055184 >> 172.21.15.32:0/4177055184 conn(0x7f2a6406ba30 msgr2=0x7f2a641441e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:15.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.330+0000 7f2a6b600640 1 -- 172.21.15.32:0/4177055184 shutdown_connections 2024-04-17T09:59:15.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:15.330+0000 7f2a6b600640 1 -- 172.21.15.32:0/4177055184 wait complete. 2024-04-17T09:59:16.095 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 59, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T09:59:04.834110+0000', 'modified': '2024-04-17T09:59:07.850260+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7557}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7557': {'gid': 7557, 'name': 'c', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6837/2656735817', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6835', 'nonce': 2656735817}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 2656735817}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7557, 'qdb_cluster': [7557]} 2024-04-17T09:59:16.095 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T09:59:16.095 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-17T09:59:16.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.246+0000 7fe5cb600640 1 Processor -- start 2024-04-17T09:59:16.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.246+0000 7fe5cb600640 1 -- start start 2024-04-17T09:59:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.250+0000 7fe5cb600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe5c40cf370 0x7fe5c40d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.250+0000 7fe5cb600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5c40d0820 0x7fe5c4071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.250+0000 7fe5cb600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5c40cff60 con 0x7fe5c413f6e0 2024-04-17T09:59:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.250+0000 7fe5cb600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe5c413fac0 con 0x7fe5c40d0820 2024-04-17T09:59:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.250+0000 7fe5cb600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe5c40d0c80 con 0x7fe5c40cf370 2024-04-17T09:59:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.250+0000 7fe5c9600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5c40d0820 0x7fe5c4071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.250+0000 7fe5c9600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5c40d0820 0x7fe5c4071d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38730/0 (socket says 172.21.15.32:38730) 2024-04-17T09:59:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.250+0000 7fe5c9600640 1 -- 172.21.15.32:0/1574034052 learned_addr learned my addr 172.21.15.32:0/1574034052 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.250+0000 7fe5bbe00640 1 --2- 172.21.15.32:0/1574034052 >> v2:172.21.15.144:3300/0 conn(0x7fe5c40cf370 0x7fe5c40d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:16.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.250+0000 7fe5c9600640 1 -- 172.21.15.32:0/1574034052 >> v2:172.21.15.144:3300/0 conn(0x7fe5c40cf370 msgr2=0x7fe5c40d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:16.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.250+0000 7fe5c9600640 1 --2- 172.21.15.32:0/1574034052 >> v2:172.21.15.144:3300/0 conn(0x7fe5c40cf370 0x7fe5c40d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.250+0000 7fe5c9600640 1 -- 172.21.15.32:0/1574034052 >> v1:172.21.15.144:6789/0 conn(0x7fe5c413f6e0 legacy=0x7fe5c40cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:16.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.250+0000 7fe5c9600640 1 -- 172.21.15.32:0/1574034052 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c41601e0 con 0x7fe5c40d0820 2024-04-17T09:59:16.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.250+0000 7fe5bbe00640 1 --2- 172.21.15.32:0/1574034052 >> v2:172.21.15.144:3300/0 conn(0x7fe5c40cf370 0x7fe5c40d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:59:16.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.250+0000 7fe5c9600640 1 --2- 172.21.15.32:0/1574034052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c40d0820 0x7fe5c4071d80 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40b5790 tx=0x7fe5bc01c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5760f0605d083d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5c3e00640 1 -- 172.21.15.32:0/1574034052 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc0a5840 con 0x7fe5c40d0820 2024-04-17T09:59:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5c3e00640 1 -- 172.21.15.32:0/1574034052 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5bc0a59e0 con 0x7fe5c40d0820 2024-04-17T09:59:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5c3e00640 1 -- 172.21.15.32:0/1574034052 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc0a5db0 con 0x7fe5c40d0820 2024-04-17T09:59:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5cb600640 1 -- 172.21.15.32:0/1574034052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c40d0820 msgr2=0x7fe5c4071d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5cb600640 1 --2- 172.21.15.32:0/1574034052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c40d0820 0x7fe5c4071d80 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40b5790 tx=0x7fe5bc01c980 comp rx=0 tx=0).stop 2024-04-17T09:59:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5cb600640 1 -- 172.21.15.32:0/1574034052 shutdown_connections 2024-04-17T09:59:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5cb600640 1 --2- 172.21.15.32:0/1574034052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c40d0820 0x7fe5c4071d80 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5cb600640 1 --2- 172.21.15.32:0/1574034052 >> v2:172.21.15.144:3300/0 conn(0x7fe5c40cf370 0x7fe5c40d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5cb600640 1 -- 172.21.15.32:0/1574034052 >> 172.21.15.32:0/1574034052 conn(0x7fe5c406ba30 msgr2=0x7fe5c406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5cb600640 1 -- 172.21.15.32:0/1574034052 shutdown_connections 2024-04-17T09:59:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5cb600640 1 -- 172.21.15.32:0/1574034052 wait complete. 2024-04-17T09:59:16.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5cb600640 1 Processor -- start 2024-04-17T09:59:16.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5cb600640 1 -- start start 2024-04-17T09:59:16.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5cb600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5c40cf370 0x7fe5c4155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5cb600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c40d0820 0x7fe5c4140680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5cb600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5c413f6e0 0x7fe5c413d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5cb600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c413fd40 con 0x7fe5c40d0820 2024-04-17T09:59:16.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5cb600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c4160d20 con 0x7fe5c413f6e0 2024-04-17T09:59:16.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5cb600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c4072ad0 con 0x7fe5c40cf370 2024-04-17T09:59:16.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5bbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c40d0820 0x7fe5c4140680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:16.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5bbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c40d0820 0x7fe5c4140680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38742/0 (socket says 172.21.15.32:38742) 2024-04-17T09:59:16.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5bbe00640 1 -- 172.21.15.32:0/444455649 learned_addr learned my addr 172.21.15.32:0/444455649 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:16.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5c9600640 1 --2- 172.21.15.32:0/444455649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5c413f6e0 0x7fe5c413d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:16.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.254+0000 7fe5c8c00640 1 --2- 172.21.15.32:0/444455649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5c40cf370 0x7fe5c4155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:16.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.258+0000 7fe5bbe00640 1 -- 172.21.15.32:0/444455649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5c40cf370 msgr2=0x7fe5c4155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:16.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.258+0000 7fe5bbe00640 1 --2- 172.21.15.32:0/444455649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5c40cf370 0x7fe5c4155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.258+0000 7fe5bbe00640 1 -- 172.21.15.32:0/444455649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5c413f6e0 msgr2=0x7fe5c413d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:16.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.258+0000 7fe5bbe00640 1 --2- 172.21.15.32:0/444455649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5c413f6e0 0x7fe5c413d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.258+0000 7fe5bbe00640 1 -- 172.21.15.32:0/444455649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c4139220 con 0x7fe5c40d0820 2024-04-17T09:59:16.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.258+0000 7fe5c8c00640 1 --2- 172.21.15.32:0/444455649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5c40cf370 0x7fe5c4155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:16.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.258+0000 7fe5bbe00640 1 --2- 172.21.15.32:0/444455649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c40d0820 0x7fe5c4140680 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac00c7f0 tx=0x7fe5ac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:16.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.258+0000 7fe5c1600640 1 -- 172.21.15.32:0/444455649 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac015070 con 0x7fe5c40d0820 2024-04-17T09:59:16.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.258+0000 7fe5c1600640 1 -- 172.21.15.32:0/444455649 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5ac007a30 con 0x7fe5c40d0820 2024-04-17T09:59:16.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.258+0000 7fe5c1600640 1 -- 172.21.15.32:0/444455649 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac007e00 con 0x7fe5c40d0820 2024-04-17T09:59:16.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.258+0000 7fe5cb600640 1 -- 172.21.15.32:0/444455649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c413a540 con 0x7fe5c40d0820 2024-04-17T09:59:16.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.258+0000 7fe5cb600640 1 -- 172.21.15.32:0/444455649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c4140bc0 con 0x7fe5c40d0820 2024-04-17T09:59:16.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.258+0000 7fe5c1600640 1 -- 172.21.15.32:0/444455649 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe5ac018070 con 0x7fe5c40d0820 2024-04-17T09:59:16.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.258+0000 7fe5c1600640 1 --2- 172.21.15.32:0/444455649 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe5a0068380 0x7fe5a006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.258+0000 7fe5c1600640 1 -- 172.21.15.32:0/444455649 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fe5ac0868c0 con 0x7fe5c40d0820 2024-04-17T09:59:16.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.258+0000 7fe5c8c00640 1 --2- 172.21.15.32:0/444455649 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe5a0068380 0x7fe5a006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:16.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5cea82000 1 Processor -- start 2024-04-17T09:59:16.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5c8c00640 1 --2- 172.21.15.32:0/444455649 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe5a0068380 0x7fe5a006a840 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4069300 tx=0x7fe5b4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:16.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5cea82000 1 -- start start 2024-04-17T09:59:16.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5cea82000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d64825b50 0x563d64825f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5cea82000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d64826470 0x563d647db7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5cea82000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d647dc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5cea82000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563d646294d0 con 0x563d647dbcf0 2024-04-17T09:59:16.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5cea82000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563d645c3bf0 con 0x563d64825b50 2024-04-17T09:59:16.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5cea82000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x563d64625490 con 0x563d64826470 2024-04-17T09:59:16.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5c9600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d647dc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:16.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5c9600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d647dc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38746/0 (socket says 172.21.15.32:38746) 2024-04-17T09:59:16.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5c9600640 1 -- 172.21.15.32:0/1972712595 learned_addr learned my addr 172.21.15.32:0/1972712595 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:16.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5c8c00640 1 --2- 172.21.15.32:0/1972712595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d64825b50 0x563d64825f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:16.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5bbe00640 1 --2- 172.21.15.32:0/1972712595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d64826470 0x563d647db7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:16.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5c9600640 1 -- 172.21.15.32:0/1972712595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d64826470 msgr2=0x563d647db7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:16.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5c9600640 1 --2- 172.21.15.32:0/1972712595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d64826470 0x563d647db7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5c9600640 1 -- 172.21.15.32:0/1972712595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d64825b50 msgr2=0x563d64825f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:16.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5c9600640 1 --2- 172.21.15.32:0/1972712595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d64825b50 0x563d64825f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5c9600640 1 -- 172.21.15.32:0/1972712595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d645afe80 con 0x563d647dbcf0 2024-04-17T09:59:16.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5c9600640 1 --2- 172.21.15.32:0/1972712595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d647dc0d0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc01a5b0 tx=0x7fe5bc004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c2bf51a8346631 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:16.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5b9600640 1 -- 172.21.15.32:0/1972712595 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc0a5a40 con 0x563d647dbcf0 2024-04-17T09:59:16.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5b9600640 1 -- 172.21.15.32:0/1972712595 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5bc0a5be0 con 0x563d647dbcf0 2024-04-17T09:59:16.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5b9600640 1 -- 172.21.15.32:0/1972712595 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc009250 con 0x563d647dbcf0 2024-04-17T09:59:16.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5cea82000 1 -- 172.21.15.32:0/1972712595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 msgr2=0x563d647dc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:16.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5cea82000 1 --2- 172.21.15.32:0/1972712595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d647dc0d0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc01a5b0 tx=0x7fe5bc004ac0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5cea82000 1 -- 172.21.15.32:0/1972712595 shutdown_connections 2024-04-17T09:59:16.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5cea82000 1 --2- 172.21.15.32:0/1972712595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d64826470 0x563d647db7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5cea82000 1 --2- 172.21.15.32:0/1972712595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d647dc0d0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5cea82000 1 --2- 172.21.15.32:0/1972712595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d64825b50 0x563d64825f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5cea82000 1 -- 172.21.15.32:0/1972712595 >> 172.21.15.32:0/1972712595 conn(0x563d647e1470 msgr2=0x563d647da090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:16.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5cea82000 1 -- 172.21.15.32:0/1972712595 shutdown_connections 2024-04-17T09:59:16.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.262+0000 7fe5cea82000 1 -- 172.21.15.32:0/1972712595 wait complete. 2024-04-17T09:59:16.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5cea82000 1 -- start start 2024-04-17T09:59:16.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5cea82000 1 Processor -- start 2024-04-17T09:59:16.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5cea82000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d647e08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5cea82000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d64825b50 0x563d64813440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5cea82000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d64826470 0x563d64813980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5cea82000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563d645e6a60 con 0x563d647dbcf0 2024-04-17T09:59:16.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5cea82000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563d645cde80 con 0x563d64826470 2024-04-17T09:59:16.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5cea82000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x563d645e4cf0 con 0x563d64825b50 2024-04-17T09:59:16.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5cea82000 10 client.0 _ll_register_callbacks cb 0x563d647c6a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:59:16.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5c8c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d647e08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:16.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5c8c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d647e08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38762/0 (socket says 172.21.15.32:38762) 2024-04-17T09:59:16.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5c8c00640 1 -- 172.21.15.32:0/879487253 learned_addr learned my addr 172.21.15.32:0/879487253 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:16.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5bbe00640 1 --2- 172.21.15.32:0/879487253 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d64825b50 0x563d64813440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:16.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5c9600640 1 --2- 172.21.15.32:0/879487253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d64826470 0x563d64813980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:16.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5c8c00640 1 -- 172.21.15.32:0/879487253 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d64825b50 msgr2=0x563d64813440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:16.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5c8c00640 1 --2- 172.21.15.32:0/879487253 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d64825b50 0x563d64813440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5c8c00640 1 -- 172.21.15.32:0/879487253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d64826470 msgr2=0x563d64813980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:16.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5c8c00640 1 --2- 172.21.15.32:0/879487253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d64826470 0x563d64813980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5c8c00640 1 -- 172.21.15.32:0/879487253 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d645c3e40 con 0x563d647dbcf0 2024-04-17T09:59:16.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5c8c00640 1 --2- 172.21.15.32:0/879487253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d647e08a0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009fd0 tx=0x7fe5b4004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da8d695b89d8be8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:16.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5b2000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:59:16.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5b2000640 1 -- 172.21.15.32:0/879487253 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4004020 con 0x563d647dbcf0 2024-04-17T09:59:16.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5b2000640 1 -- 172.21.15.32:0/879487253 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fe5b401d040 con 0x563d647dbcf0 2024-04-17T09:59:16.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5cea82000 1 -- 172.21.15.32:0/879487253 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5bc0a5be0 con 0x563d647dbcf0 2024-04-17T09:59:16.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5b2000640 1 -- 172.21.15.32:0/879487253 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe5b40041c0 con 0x563d647dbcf0 2024-04-17T09:59:16.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5cea82000 10 client.7731 fetch_fsmap learned FSMap version 59 2024-04-17T09:59:16.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5cea82000 1 -- 172.21.15.32:0/879487253 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x563d645f4e00 con 0x563d647dbcf0 2024-04-17T09:59:16.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5b2000640 1 -- 172.21.15.32:0/879487253 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5bc0a5be0 con 0x563d647dbcf0 2024-04-17T09:59:16.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5b2000640 1 -- 172.21.15.32:0/879487253 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 59) ==== 3495+0+0 (secure 0 0 0) 0x7fe5b4013070 con 0x563d647dbcf0 2024-04-17T09:59:16.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5cea82000 10 client.7731 fetch_fsmap finished waiting for FSMap version 59 2024-04-17T09:59:16.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5cea82000 10 client.7731 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T09:59:16.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5cea82000 20 client.7731 populate_metadata read hostname 'smithi032' 2024-04-17T09:59:16.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5cea82000 1 --2- 172.21.15.32:0/879487253 >> [v2:172.21.15.32:6835/2656735817,v1:172.21.15.32:6837/2656735817] conn(0x563d64834be0 0x563d64836fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5cea82000 4 client.7731 mds_command: new command op to 7557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:59:16.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5cea82000 1 -- 172.21.15.32:0/879487253 --> [v2:172.21.15.32:6835/2656735817,v1:172.21.15.32:6837/2656735817] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d645c2bc0 con 0x563d64834be0 2024-04-17T09:59:16.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.266+0000 7fe5bbe00640 1 --2- 172.21.15.32:0/879487253 >> [v2:172.21.15.32:6835/2656735817,v1:172.21.15.32:6837/2656735817] conn(0x563d64834be0 0x563d64836fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:16.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.270+0000 7fe5bbe00640 1 --2- 172.21.15.32:0/879487253 >> [v2:172.21.15.32:6835/2656735817,v1:172.21.15.32:6837/2656735817] conn(0x563d64834be0 0x563d64836fd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2432583039b24d7 server_cookie=d66ec8f72ce02a7d in_seq=0 out_seq=0 2024-04-17T09:59:16.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.270+0000 7fe5b2000640 10 client.7731 ms_handle_connect on v2:172.21.15.32:6835/2656735817 2024-04-17T09:59:16.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.270+0000 7fe5b2000640 1 -- 172.21.15.32:0/879487253 <== mds.0 v2:172.21.15.32:6835/2656735817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d645c2bc0 con 0x563d64834be0 2024-04-17T09:59:16.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.270+0000 7fe5b2000640 10 client.7731 handle_command_reply: tid=0 2024-04-17T09:59:16.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.270+0000 7fe5cea82000 1 client.7731 shutdown 2024-04-17T09:59:16.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.270+0000 7fe5cea82000 1 -- 172.21.15.32:0/879487253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 msgr2=0x563d647e08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:16.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.270+0000 7fe5cea82000 1 --2- 172.21.15.32:0/879487253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d647e08a0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009fd0 tx=0x7fe5b4004690 comp rx=0 tx=0).stop 2024-04-17T09:59:16.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.270+0000 7fe5cea82000 1 -- 172.21.15.32:0/879487253 shutdown_connections 2024-04-17T09:59:16.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.270+0000 7fe5cea82000 1 --2- 172.21.15.32:0/879487253 >> [v2:172.21.15.32:6835/2656735817,v1:172.21.15.32:6837/2656735817] conn(0x563d64834be0 0x563d64836fd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.270+0000 7fe5cea82000 1 --2- 172.21.15.32:0/879487253 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d64825b50 0x563d64813440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.270+0000 7fe5cea82000 1 --2- 172.21.15.32:0/879487253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d64826470 0x563d64813980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.270+0000 7fe5cea82000 1 --2- 172.21.15.32:0/879487253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d647e08a0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.270+0000 7fe5cea82000 1 -- 172.21.15.32:0/879487253 >> 172.21.15.32:0/879487253 conn(0x563d647e1470 msgr2=0x563d647da7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:16.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.270+0000 7fe5b2000640 0 client.7731 ms_handle_reset on v2:172.21.15.32:6835/2656735817 2024-04-17T09:59:16.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.270+0000 7fe5cea82000 1 -- 172.21.15.32:0/879487253 shutdown_connections 2024-04-17T09:59:16.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.270+0000 7fe5cea82000 1 -- 172.21.15.32:0/879487253 wait complete. 2024-04-17T09:59:16.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.270+0000 7fe5cea82000 20 client.7731 trim_cache size 0 max 16384 2024-04-17T09:59:16.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.286+0000 7fe5cea82000 1 Processor -- start 2024-04-17T09:59:16.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.286+0000 7fe5cea82000 1 -- start start 2024-04-17T09:59:16.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5cea82000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d647dd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5cea82000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d64825b50 0x563d647dde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5cea82000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d64826470 0x563d64836fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5cea82000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c4202ae0 con 0x563d647dbcf0 2024-04-17T09:59:16.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5cea82000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41450c0 con 0x563d64826470 2024-04-17T09:59:16.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5cea82000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c4145710 con 0x563d64825b50 2024-04-17T09:59:16.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5c8c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d647dd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:16.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5c8c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d647dd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38764/0 (socket says 172.21.15.32:38764) 2024-04-17T09:59:16.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5c8c00640 1 -- 172.21.15.32:0/744361893 learned_addr learned my addr 172.21.15.32:0/744361893 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:16.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5c8c00640 1 -- 172.21.15.32:0/744361893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d64825b50 msgr2=0x563d647dde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:16.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5c8c00640 1 --2- 172.21.15.32:0/744361893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d64825b50 0x563d647dde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5c8c00640 1 -- 172.21.15.32:0/744361893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d64826470 msgr2=0x563d64836fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:16.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5c8c00640 1 --2- 172.21.15.32:0/744361893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d64826470 0x563d64836fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5c8c00640 1 -- 172.21.15.32:0/744361893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d646aad30 con 0x563d647dbcf0 2024-04-17T09:59:16.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5c8c00640 1 --2- 172.21.15.32:0/744361893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d647dd920 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4008010 tx=0x7fe5b401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bae7834a7f2f0e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:16.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5b8c00640 1 -- 172.21.15.32:0/744361893 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b40043b0 con 0x563d647dbcf0 2024-04-17T09:59:16.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5b8c00640 1 -- 172.21.15.32:0/744361893 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5b4005920 con 0x563d647dbcf0 2024-04-17T09:59:16.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5b8c00640 1 -- 172.21.15.32:0/744361893 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4005b10 con 0x563d647dbcf0 2024-04-17T09:59:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5cea82000 1 -- 172.21.15.32:0/744361893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 msgr2=0x563d647dd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5cea82000 1 --2- 172.21.15.32:0/744361893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d647dd920 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4008010 tx=0x7fe5b401e000 comp rx=0 tx=0).stop 2024-04-17T09:59:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5cea82000 1 -- 172.21.15.32:0/744361893 shutdown_connections 2024-04-17T09:59:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5cea82000 1 --2- 172.21.15.32:0/744361893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d64825b50 0x563d647dde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5cea82000 1 --2- 172.21.15.32:0/744361893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d64826470 0x563d64836fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5cea82000 1 --2- 172.21.15.32:0/744361893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d647dd920 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5cea82000 1 -- 172.21.15.32:0/744361893 >> 172.21.15.32:0/744361893 conn(0x563d647e1470 msgr2=0x563d647dd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5cea82000 1 -- 172.21.15.32:0/744361893 shutdown_connections 2024-04-17T09:59:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5cea82000 1 -- 172.21.15.32:0/744361893 wait complete. 2024-04-17T09:59:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5cea82000 1 -- start start 2024-04-17T09:59:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5cea82000 1 Processor -- start 2024-04-17T09:59:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5cea82000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d64839b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5cea82000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d64825b50 0x563d64850b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.290+0000 7fe5cea82000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d64826470 0x563d648510a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563d6462db00 con 0x563d647dbcf0 2024-04-17T09:59:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563d646287e0 con 0x563d64826470 2024-04-17T09:59:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x563d646366e0 con 0x563d64825b50 2024-04-17T09:59:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 10 client.0 _ll_register_callbacks cb 0x7fe5b40094c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:59:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5c8c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d64839b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5c8c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d64839b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38778/0 (socket says 172.21.15.32:38778) 2024-04-17T09:59:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5c8c00640 1 -- 172.21.15.32:0/3127461860 learned_addr learned my addr 172.21.15.32:0/3127461860 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:16.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5bbe00640 1 --2- 172.21.15.32:0/3127461860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d64825b50 0x563d64850b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5c9600640 1 --2- 172.21.15.32:0/3127461860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d64826470 0x563d648510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5c8c00640 1 -- 172.21.15.32:0/3127461860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d64825b50 msgr2=0x563d64850b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5c8c00640 1 --2- 172.21.15.32:0/3127461860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d64825b50 0x563d64850b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5c8c00640 1 -- 172.21.15.32:0/3127461860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d64826470 msgr2=0x563d648510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5c8c00640 1 --2- 172.21.15.32:0/3127461860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d64826470 0x563d648510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5c8c00640 1 -- 172.21.15.32:0/3127461860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d645bb7c0 con 0x563d647dbcf0 2024-04-17T09:59:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5c8c00640 1 --2- 172.21.15.32:0/3127461860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d64839b40 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4022000 tx=0x7fe5b4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476556357e69dd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5b2000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:59:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5b2000640 1 -- 172.21.15.32:0/3127461860 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b403f060 con 0x563d647dbcf0 2024-04-17T09:59:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5b2000640 1 -- 172.21.15.32:0/3127461860 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fe5b4025040 con 0x563d647dbcf0 2024-04-17T09:59:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 1 -- 172.21.15.32:0/3127461860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5b4005920 con 0x563d647dbcf0 2024-04-17T09:59:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5b2000640 1 -- 172.21.15.32:0/3127461860 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe5b40039d0 con 0x563d647dbcf0 2024-04-17T09:59:16.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 10 client.7737 fetch_fsmap learned FSMap version 59 2024-04-17T09:59:16.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 1 -- 172.21.15.32:0/3127461860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x563d645ce650 con 0x563d647dbcf0 2024-04-17T09:59:16.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5b2000640 1 -- 172.21.15.32:0/3127461860 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5b4005920 con 0x563d647dbcf0 2024-04-17T09:59:16.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5b2000640 1 -- 172.21.15.32:0/3127461860 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 59) ==== 3495+0+0 (secure 0 0 0) 0x7fe5b4003b70 con 0x563d647dbcf0 2024-04-17T09:59:16.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 10 client.7737 fetch_fsmap finished waiting for FSMap version 59 2024-04-17T09:59:16.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 10 client.7737 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T09:59:16.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 20 client.7737 populate_metadata read hostname 'smithi032' 2024-04-17T09:59:16.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 1 --2- 172.21.15.32:0/3127461860 >> [v2:172.21.15.32:6835/2656735817,v1:172.21.15.32:6837/2656735817] conn(0x563d648947d0 0x563d64894bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 4 client.7737 mds_command: new command op to 7557 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T09:59:16.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 1 -- 172.21.15.32:0/3127461860 --> [v2:172.21.15.32:6835/2656735817,v1:172.21.15.32:6837/2656735817] -- command(tid 0: {"prefix": "status"}) -- 0x563d64636ef0 con 0x563d648947d0 2024-04-17T09:59:16.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5bbe00640 1 --2- 172.21.15.32:0/3127461860 >> [v2:172.21.15.32:6835/2656735817,v1:172.21.15.32:6837/2656735817] conn(0x563d648947d0 0x563d64894bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:16.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5bbe00640 1 --2- 172.21.15.32:0/3127461860 >> [v2:172.21.15.32:6835/2656735817,v1:172.21.15.32:6837/2656735817] conn(0x563d648947d0 0x563d64894bb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ead541a0823ddce9 server_cookie=b1784eb55d106432 in_seq=0 out_seq=0 2024-04-17T09:59:16.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5b2000640 10 client.7737 ms_handle_connect on v2:172.21.15.32:6835/2656735817 2024-04-17T09:59:16.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5b2000640 1 -- 172.21.15.32:0/3127461860 <== mds.0 v2:172.21.15.32:6835/2656735817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x563d64636ef0 con 0x563d648947d0 2024-04-17T09:59:16.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5b2000640 10 client.7737 handle_command_reply: tid=0 2024-04-17T09:59:16.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 1 client.7737 shutdown 2024-04-17T09:59:16.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 1 -- 172.21.15.32:0/3127461860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 msgr2=0x563d64839b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:16.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 1 --2- 172.21.15.32:0/3127461860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d64839b40 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4022000 tx=0x7fe5b4014000 comp rx=0 tx=0).stop 2024-04-17T09:59:16.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 1 -- 172.21.15.32:0/3127461860 shutdown_connections 2024-04-17T09:59:16.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 1 --2- 172.21.15.32:0/3127461860 >> [v2:172.21.15.32:6835/2656735817,v1:172.21.15.32:6837/2656735817] conn(0x563d648947d0 0x563d64894bb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 1 --2- 172.21.15.32:0/3127461860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563d64825b50 0x563d64850b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 1 --2- 172.21.15.32:0/3127461860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563d64826470 0x563d648510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 1 --2- 172.21.15.32:0/3127461860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563d647dbcf0 0x563d64839b40 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5b2000640 0 client.7737 ms_handle_reset on v2:172.21.15.32:6835/2656735817 2024-04-17T09:59:16.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 1 -- 172.21.15.32:0/3127461860 >> 172.21.15.32:0/3127461860 conn(0x563d647e1470 msgr2=0x563d647dd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:16.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 1 -- 172.21.15.32:0/3127461860 shutdown_connections 2024-04-17T09:59:16.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 1 -- 172.21.15.32:0/3127461860 wait complete. 2024-04-17T09:59:16.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cea82000 20 client.7737 trim_cache size 0 max 16384 2024-04-17T09:59:16.298 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:59:16.298 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "ff178df8-4c06-4367-a1e9-2e3a66d870d0", 2024-04-17T09:59:16.298 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-17T09:59:16.298 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 7557, 2024-04-17T09:59:16.298 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-17T09:59:16.298 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-17T09:59:16.298 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-17T09:59:16.299 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 10.852303997, 2024-04-17T09:59:16.299 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 59, 2024-04-17T09:59:16.299 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 82, 2024-04-17T09:59:16.299 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 82, 2024-04-17T09:59:16.299 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 15.696945411 2024-04-17T09:59:16.299 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:59:16.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cb600640 1 -- 172.21.15.32:0/444455649 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe5a0068380 msgr2=0x7fe5a006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:16.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cb600640 1 --2- 172.21.15.32:0/444455649 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe5a0068380 0x7fe5a006a840 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4069300 tx=0x7fe5b4008040 comp rx=0 tx=0).stop 2024-04-17T09:59:16.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cb600640 1 -- 172.21.15.32:0/444455649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c40d0820 msgr2=0x7fe5c4140680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:16.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.294+0000 7fe5cb600640 1 --2- 172.21.15.32:0/444455649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c40d0820 0x7fe5c4140680 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac00c7f0 tx=0x7fe5ac00ccb0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.298+0000 7fe5cb600640 1 -- 172.21.15.32:0/444455649 shutdown_connections 2024-04-17T09:59:16.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.298+0000 7fe5cb600640 1 --2- 172.21.15.32:0/444455649 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe5a0068380 0x7fe5a006a840 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.298+0000 7fe5cb600640 1 --2- 172.21.15.32:0/444455649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5c413f6e0 0x7fe5c413d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.298+0000 7fe5cb600640 1 --2- 172.21.15.32:0/444455649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c40d0820 0x7fe5c4140680 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.298+0000 7fe5cb600640 1 --2- 172.21.15.32:0/444455649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5c40cf370 0x7fe5c4155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.298+0000 7fe5cb600640 1 -- 172.21.15.32:0/444455649 >> 172.21.15.32:0/444455649 conn(0x7fe5c406ba30 msgr2=0x7fe5c4141b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:16.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.298+0000 7fe5cb600640 1 -- 172.21.15.32:0/444455649 shutdown_connections 2024-04-17T09:59:16.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.298+0000 7fe5cb600640 1 -- 172.21.15.32:0/444455649 wait complete. 2024-04-17T09:59:16.314 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T09:59:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.458+0000 7f9a63200640 1 Processor -- start 2024-04-17T09:59:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.458+0000 7f9a63200640 1 -- start start 2024-04-17T09:59:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.458+0000 7f9a63200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a6406f250 0x7f9a6406dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.458+0000 7f9a63200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a6406e1b0 con 0x7f9a640cf5b0 2024-04-17T09:59:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.458+0000 7f9a63200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a6406e490 con 0x7f9a640cf990 2024-04-17T09:59:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.458+0000 7f9a63200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9a6406edc0 con 0x7f9a6406f250 2024-04-17T09:59:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.458+0000 7f9a62800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a6406f250 0x7f9a6406dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.458+0000 7f9a61e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9a640cf5b0 0x7f9a6415d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52100/0 (socket says 172.21.15.32:52100) 2024-04-17T09:59:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.458+0000 7f9a61e00640 1 -- 172.21.15.32:0/1378677593 learned_addr learned my addr 172.21.15.32:0/1378677593 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.458+0000 7f9a61400640 1 -- 172.21.15.32:0/1378677593 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2326073384 0 0) 0x7f9a6406e1b0 con 0x7f9a640cf5b0 2024-04-17T09:59:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.458+0000 7f9a61400640 1 -- 172.21.15.32:0/1378677593 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a48003530 con 0x7f9a640cf5b0 2024-04-17T09:59:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.458+0000 7f9a62800640 1 -- 172.21.15.32:0/1378677593 >> v1:172.21.15.144:6789/0 conn(0x7f9a640cf990 legacy=0x7f9a640aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.458+0000 7f9a61400640 1 -- 172.21.15.32:0/1378677593 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1719165946 0 0) 0x7f9a6406e490 con 0x7f9a640cf990 2024-04-17T09:59:16.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.458+0000 7f9a62800640 1 -- 172.21.15.32:0/1378677593 >> v1:172.21.15.32:6789/0 conn(0x7f9a640cf5b0 legacy=0x7f9a6415d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:16.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.458+0000 7f9a62800640 1 -- 172.21.15.32:0/1378677593 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6406e770 con 0x7f9a6406f250 2024-04-17T09:59:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.462+0000 7f9a62800640 1 --2- 172.21.15.32:0/1378677593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6406f250 0x7f9a6406dbd0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f9a6406ef40 tx=0x7f9a5c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b546557664131ee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:16.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.462+0000 7f9a61400640 1 -- 172.21.15.32:0/1378677593 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c0b2070 con 0x7f9a6406f250 2024-04-17T09:59:16.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.462+0000 7f9a61400640 1 -- 172.21.15.32:0/1378677593 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a5c0a5860 con 0x7f9a6406f250 2024-04-17T09:59:16.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.462+0000 7f9a61400640 1 -- 172.21.15.32:0/1378677593 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c0a5c30 con 0x7f9a6406f250 2024-04-17T09:59:16.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.462+0000 7f9a63200640 1 -- 172.21.15.32:0/1378677593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6406f250 msgr2=0x7f9a6406dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:16.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.462+0000 7f9a63200640 1 --2- 172.21.15.32:0/1378677593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6406f250 0x7f9a6406dbd0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f9a6406ef40 tx=0x7f9a5c01c890 comp rx=0 tx=0).stop 2024-04-17T09:59:16.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.462+0000 7f9a63200640 1 -- 172.21.15.32:0/1378677593 shutdown_connections 2024-04-17T09:59:16.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.462+0000 7f9a63200640 1 --2- 172.21.15.32:0/1378677593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6406f250 0x7f9a6406dbd0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.462+0000 7f9a63200640 1 -- 172.21.15.32:0/1378677593 >> 172.21.15.32:0/1378677593 conn(0x7f9a640d0180 msgr2=0x7f9a640d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:16.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.462+0000 7f9a63200640 1 -- 172.21.15.32:0/1378677593 shutdown_connections 2024-04-17T09:59:16.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.462+0000 7f9a63200640 1 -- 172.21.15.32:0/1378677593 wait complete. 2024-04-17T09:59:16.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a63200640 1 Processor -- start 2024-04-17T09:59:16.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a63200640 1 -- start start 2024-04-17T09:59:16.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a63200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a6406f250 0x7f9a640725e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a63200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a640cf5b0 0x7f9a640999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a63200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a640cf990 0x7f9a641564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a63200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6415e720 con 0x7f9a640cf990 2024-04-17T09:59:16.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a63200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6415e5a0 con 0x7f9a640cf5b0 2024-04-17T09:59:16.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a63200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9a6415e8a0 con 0x7f9a6406f250 2024-04-17T09:59:16.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a62800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a640cf990 0x7f9a641564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:16.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a62800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a640cf990 0x7f9a641564d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38796/0 (socket says 172.21.15.32:38796) 2024-04-17T09:59:16.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a62800640 1 -- 172.21.15.32:0/1277799843 learned_addr learned my addr 172.21.15.32:0/1277799843 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:16.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a5be00640 1 --2- 172.21.15.32:0/1277799843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a640cf5b0 0x7f9a640999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:16.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a61e00640 1 --2- 172.21.15.32:0/1277799843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a6406f250 0x7f9a640725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:16.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a62800640 1 -- 172.21.15.32:0/1277799843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a6406f250 msgr2=0x7f9a640725e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:16.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a62800640 1 --2- 172.21.15.32:0/1277799843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a6406f250 0x7f9a640725e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a62800640 1 -- 172.21.15.32:0/1277799843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a640cf5b0 msgr2=0x7f9a640999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:16.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a62800640 1 --2- 172.21.15.32:0/1277799843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a640cf5b0 0x7f9a640999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a62800640 1 -- 172.21.15.32:0/1277799843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a64072cc0 con 0x7f9a640cf990 2024-04-17T09:59:16.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a62800640 1 --2- 172.21.15.32:0/1277799843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a640cf990 0x7f9a641564d0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c098380 tx=0x7f9a5c019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:16.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a5a000640 1 -- 172.21.15.32:0/1277799843 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c0b2050 con 0x7f9a640cf990 2024-04-17T09:59:16.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a5a000640 1 -- 172.21.15.32:0/1277799843 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a5c0b3040 con 0x7f9a640cf990 2024-04-17T09:59:16.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a63200640 1 -- 172.21.15.32:0/1277799843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a64072fa0 con 0x7f9a640cf990 2024-04-17T09:59:16.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a5a000640 1 -- 172.21.15.32:0/1277799843 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c009450 con 0x7f9a640cf990 2024-04-17T09:59:16.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.466+0000 7f9a63200640 1 -- 172.21.15.32:0/1277799843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a64073560 con 0x7f9a640cf990 2024-04-17T09:59:16.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.470+0000 7f9a63200640 1 -- 172.21.15.32:0/1277799843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a6404fd00 con 0x7f9a640cf990 2024-04-17T09:59:16.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.470+0000 7f9a5a000640 1 -- 172.21.15.32:0/1277799843 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9a5c0adc00 con 0x7f9a640cf990 2024-04-17T09:59:16.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.470+0000 7f9a5a000640 1 --2- 172.21.15.32:0/1277799843 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9a40068380 0x7f9a4006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:16.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.470+0000 7f9a5a000640 1 -- 172.21.15.32:0/1277799843 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f9a5c121a20 con 0x7f9a640cf990 2024-04-17T09:59:16.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.470+0000 7f9a61e00640 1 --2- 172.21.15.32:0/1277799843 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9a40068380 0x7f9a4006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:16.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.470+0000 7f9a61e00640 1 --2- 172.21.15.32:0/1277799843 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9a40068380 0x7f9a4006a840 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9a54005890 tx=0x7f9a54009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:16.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.474+0000 7f9a5a000640 1 -- 172.21.15.32:0/1277799843 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f9a5c016540 con 0x7f9a640cf990 2024-04-17T09:59:16.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.674+0000 7f9a63200640 1 -- 172.21.15.32:0/1277799843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9a6406fc10 con 0x7f9a640cf990 2024-04-17T09:59:16.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.674+0000 7f9a5a000640 1 -- 172.21.15.32:0/1277799843 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v59) ==== 53+0+73 (secure 0 0 0) 0x7f9a5c0f1f40 con 0x7f9a640cf990 2024-04-17T09:59:16.673 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T09:59:16.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.678+0000 7f9a63200640 1 -- 172.21.15.32:0/1277799843 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9a40068380 msgr2=0x7f9a4006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.678+0000 7f9a63200640 1 --2- 172.21.15.32:0/1277799843 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9a40068380 0x7f9a4006a840 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9a54005890 tx=0x7f9a54009040 comp rx=0 tx=0).stop 2024-04-17T09:59:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.678+0000 7f9a63200640 1 -- 172.21.15.32:0/1277799843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a640cf990 msgr2=0x7f9a641564d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.678+0000 7f9a63200640 1 --2- 172.21.15.32:0/1277799843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a640cf990 0x7f9a641564d0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c098380 tx=0x7f9a5c019630 comp rx=0 tx=0).stop 2024-04-17T09:59:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.678+0000 7f9a63200640 1 -- 172.21.15.32:0/1277799843 shutdown_connections 2024-04-17T09:59:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.678+0000 7f9a63200640 1 --2- 172.21.15.32:0/1277799843 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9a40068380 0x7f9a4006a840 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.678+0000 7f9a63200640 1 --2- 172.21.15.32:0/1277799843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a640cf990 0x7f9a641564d0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.678+0000 7f9a63200640 1 --2- 172.21.15.32:0/1277799843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a640cf5b0 0x7f9a640999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.678+0000 7f9a63200640 1 --2- 172.21.15.32:0/1277799843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a6406f250 0x7f9a640725e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.678+0000 7f9a63200640 1 -- 172.21.15.32:0/1277799843 >> 172.21.15.32:0/1277799843 conn(0x7f9a640d0180 msgr2=0x7f9a640b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.678+0000 7f9a63200640 1 -- 172.21.15.32:0/1277799843 shutdown_connections 2024-04-17T09:59:16.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16.678+0000 7f9a63200640 1 -- 172.21.15.32:0/1277799843 wait complete. 2024-04-17T09:59:16.694 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T09:59:16.694 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T09:59:16.694 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-17T09:59:16.694 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T09:59:16.694 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T09:59:16.694 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T09:59:16.694 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T09:59:16.694 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T09:59:16.694 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:59:16.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:16.695 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-17T09:59:16.704 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T09:59:16.704 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T09:59:16.704 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T09:59:16.704 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:59:16.704 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-17T09:59:16.704 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:59:16.704 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:59:16.704 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-17T09:59:16.704 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-17T09:59:16.704 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T09:59:16.704 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:59:16.704 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-17T09:59:16.704 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-17T09:59:16.705 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42097sec preferred_lft 42097sec 2024-04-17T09:59:16.705 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2024-04-17T09:59:16.705 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:59:16.705 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:59:16.705 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-17T09:59:16.705 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-17T09:59:16.705 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:59:16.705 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:59:16.705 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-17T09:59:16.705 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T09:59:16.705 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:cd:f4:aa:5b brd ff:ff:ff:ff:ff:ff 2024-04-17T09:59:16.705 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T09:59:16.705 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:59:16.705 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T09:59:16.706 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T09:59:16.706 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T09:59:16.706 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:59:16.706 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T09:59:16.706 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:59:16.706 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T09:59:16.706 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:59:16.706 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T09:59:16.706 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:59:16.707 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T09:59:16.707 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:59:16.707 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T09:59:16.707 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:59:16.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:16.708 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T09:59:16.773 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-17T09:59:16.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:16.776 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-17T09:59:16.831 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-17T09:59:16.831 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T09:59:16.831 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-17T09:59:16.831 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T09:59:16.831 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-17T09:59:16.831 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T09:59:16.831 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:59:16.832 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T09:59:16.832 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-17T09:59:16.832 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-17T09:59:16.832 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-17T09:59:16.832 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T09:59:16.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:59:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:59:16.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:16.970 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-17T09:59:16.979 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T09:59:16.979 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T09:59:16.980 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:59:16.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:16.980 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-17T09:59:17.042 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T09:59:17.042 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:59:17.042 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T09:59:17.093 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:59:17.094 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:59:17.094 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T09:59:17.149 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T09:59:17.210 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:59:17.267 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T09:59:17.267 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T09:59:17.267 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T09:59:17.267 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T09:59:17.267 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T09:59:17.267 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T09:59:17.267 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T09:59:17.268 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T09:59:17.268 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T09:59:17.268 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T09:59:17.268 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T09:59:17.268 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T09:59:17.268 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:59:17.268 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T09:59:17.268 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T09:59:17.268 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T09:59:17.268 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T09:59:17.268 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T09:59:17.268 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T09:59:17.268 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T09:59:17.268 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:59:17.269 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T09:59:17.269 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T09:59:17.269 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T09:59:17.269 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T09:59:17.269 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T09:59:17.269 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T09:59:17.269 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T09:59:17.269 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T09:59:17.269 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:59:17.269 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:59:17.269 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:59:17.269 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:59:17.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:17.270 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T09:59:17.321 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T09:59:17.322 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T09:59:17.369 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T09:59:17.391 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:59:17.443 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T09:59:17.443 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T09:59:17.444 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T09:59:17.444 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T09:59:17.444 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T09:59:17.444 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T09:59:17.444 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T09:59:17.444 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T09:59:17.445 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T09:59:17.445 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T09:59:17.445 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T09:59:17.445 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T09:59:17.445 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:59:17.445 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T09:59:17.445 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T09:59:17.445 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T09:59:17.445 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T09:59:17.445 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T09:59:17.445 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T09:59:17.446 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T09:59:17.446 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:59:17.446 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T09:59:17.446 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T09:59:17.446 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T09:59:17.446 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T09:59:17.446 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T09:59:17.446 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T09:59:17.446 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T09:59:17.446 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T09:59:17.446 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:59:17.447 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:59:17.447 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:59:17.447 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:59:17.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:17.448 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T09:59:17.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2024-04-17T09:59:17.466+0000 7ff595edc580 -1 init, newargv = 0x55ad27d67ef0 newargc=15 2024-04-17T09:59:17.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[44984]: starting ceph client 2024-04-17T09:59:17.471 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[44984]: starting fuse 2024-04-17T09:59:17.474 INFO:teuthology.orchestra.run.smithi032.stdout:44 2024-04-17T09:59:17.475 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-17T09:59:17.475 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2024-04-17T09:59:17.475 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2024-04-17T09:59:17.475 DEBUG:teuthology.orchestra.run.smithi032:> import re 2024-04-17T09:59:17.475 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T09:59:17.475 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2024-04-17T09:59:17.475 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:59:17.475 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2024-04-17T09:59:17.475 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T09:59:17.475 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2024-04-17T09:59:17.476 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T09:59:17.476 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:59:17.476 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2024-04-17T09:59:17.476 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2024-04-17T09:59:17.476 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2024-04-17T09:59:17.476 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2024-04-17T09:59:17.476 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:59:17.476 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2024-04-17T09:59:17.476 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T09:59:17.476 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T09:59:17.476 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T09:59:17.476 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2024-04-17T09:59:17.476 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2024-04-17T09:59:17.476 DEBUG:teuthology.orchestra.run.smithi032:> return f 2024-04-17T09:59:17.476 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T09:59:17.476 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:59:17.476 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2024-04-17T09:59:17.477 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T09:59:17.604 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.44984.asok 2024-04-17T09:59:17.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:59:17.616 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44984.asok 2024-04-17T09:59:17.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:17.616 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44984.asok status 2024-04-17T09:59:17.794 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:59:17.794 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2024-04-17T09:59:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-17T09:59:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T09:59:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2024-04-17T09:59:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2024-04-17T09:59:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T09:59:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "44984", 2024-04-17T09:59:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2024-04-17T09:59:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2024-04-17T09:59:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2024-04-17T09:59:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 7755, 2024-04-17T09:59:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2024-04-17T09:59:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2024-04-17T09:59:17.795 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2024-04-17T09:59:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 7755 2024-04-17T09:59:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T09:59:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T09:59:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T09:59:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 988929592 2024-04-17T09:59:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T09:59:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T09:59:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T09:59:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 988929592 2024-04-17T09:59:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.7755 192.168.0.1:0/988929592", 2024-04-17T09:59:17.796 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/988929592", 2024-04-17T09:59:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2024-04-17T09:59:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 59, 2024-04-17T09:59:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 82, 2024-04-17T09:59:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2024-04-17T09:59:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2024-04-17T09:59:17.797 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs" 2024-04-17T09:59:17.797 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:59:17.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:17.809 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2024-04-17T09:59:17.809 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T09:59:17.818 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2024-04-17T09:59:17.819 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T09:59:17.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:17.819 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T09:59:17.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:17.882 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-04-17T09:59:17.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:17.941 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-04-17T09:59:17.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:17.997 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-04-17T09:59:18.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:18.052 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-04-17T09:59:18.113 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T09:59:18.113 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T09:59:18.113 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T09:59:18.113 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:59:18.113 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-04-17T09:59:18.113 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T09:59:18.223 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627776 2024-04-17T09:59:18.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:59:18.232 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:59:18.232 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:59:18.287 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:59:18.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.470+0000 7f527f800640 1 Processor -- start 2024-04-17T09:59:18.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.470+0000 7f527f800640 1 -- start start 2024-04-17T09:59:18.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.474+0000 7f527f800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f527815baa0 0x7f527815bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:18.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.474+0000 7f527f800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5278072bb0 con 0x7f527806ef60 2024-04-17T09:59:18.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.474+0000 7f527f800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52780b3dc0 con 0x7f5278144e90 2024-04-17T09:59:18.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.474+0000 7f527f800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5278145270 con 0x7f527815baa0 2024-04-17T09:59:18.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.474+0000 7f526fe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f527815baa0 0x7f527815bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:18.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.474+0000 7f527d800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f527806ef60 0x7f5278071e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52102/0 (socket says 172.21.15.32:52102) 2024-04-17T09:59:18.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.474+0000 7f527d800640 1 -- 172.21.15.32:0/3803142668 learned_addr learned my addr 172.21.15.32:0/3803142668 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:18.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.474+0000 7f5277e00640 1 -- 172.21.15.32:0/3803142668 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4022678260 0 0) 0x7f5278072bb0 con 0x7f527806ef60 2024-04-17T09:59:18.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.474+0000 7f5277e00640 1 -- 172.21.15.32:0/3803142668 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f525c003530 con 0x7f527806ef60 2024-04-17T09:59:18.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.474+0000 7f526fe00640 1 -- 172.21.15.32:0/3803142668 >> v1:172.21.15.144:6789/0 conn(0x7f5278144e90 legacy=0x7f52780b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:18.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.474+0000 7f526fe00640 1 -- 172.21.15.32:0/3803142668 >> v1:172.21.15.32:6789/0 conn(0x7f527806ef60 legacy=0x7f5278071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:18.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.474+0000 7f526fe00640 1 -- 172.21.15.32:0/3803142668 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52780b40a0 con 0x7f527815baa0 2024-04-17T09:59:18.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.478+0000 7f526fe00640 1 --2- 172.21.15.32:0/3803142668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f527815baa0 0x7f527815bf30 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f52600097c0 tx=0x7f526001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6977dc8ed2c99f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:18.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.478+0000 7f5277e00640 1 -- 172.21.15.32:0/3803142668 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52600b0070 con 0x7f527815baa0 2024-04-17T09:59:18.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.478+0000 7f5277e00640 1 -- 172.21.15.32:0/3803142668 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52600ab600 con 0x7f527815baa0 2024-04-17T09:59:18.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.478+0000 7f5277e00640 1 -- 172.21.15.32:0/3803142668 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52600ab9b0 con 0x7f527815baa0 2024-04-17T09:59:18.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.478+0000 7f527f800640 1 -- 172.21.15.32:0/3803142668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f527815baa0 msgr2=0x7f527815bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:18.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.478+0000 7f527f800640 1 --2- 172.21.15.32:0/3803142668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f527815baa0 0x7f527815bf30 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f52600097c0 tx=0x7f526001c4a0 comp rx=0 tx=0).stop 2024-04-17T09:59:18.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.478+0000 7f527f800640 1 -- 172.21.15.32:0/3803142668 shutdown_connections 2024-04-17T09:59:18.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.478+0000 7f527f800640 1 --2- 172.21.15.32:0/3803142668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f527815baa0 0x7f527815bf30 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:18.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.478+0000 7f527f800640 1 -- 172.21.15.32:0/3803142668 >> 172.21.15.32:0/3803142668 conn(0x7f527806ba30 msgr2=0x7f527806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:18.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.478+0000 7f527f800640 1 -- 172.21.15.32:0/3803142668 shutdown_connections 2024-04-17T09:59:18.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.478+0000 7f527f800640 1 -- 172.21.15.32:0/3803142668 wait complete. 2024-04-17T09:59:18.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.478+0000 7f527f800640 1 Processor -- start 2024-04-17T09:59:18.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.478+0000 7f527f800640 1 -- start start 2024-04-17T09:59:18.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f527f800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f527806ef60 0x7f527814a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:18.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f527f800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5278144e90 0x7f5278142d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:18.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f527f800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f527815baa0 0x7f52781408a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:18.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f527f800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5278169ab0 con 0x7f5278144e90 2024-04-17T09:59:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f527f800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5278169930 con 0x7f527815baa0 2024-04-17T09:59:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f527f800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5278169c30 con 0x7f527806ef60 2024-04-17T09:59:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f526fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5278144e90 0x7f5278142d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f526fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5278144e90 0x7f5278142d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38818/0 (socket says 172.21.15.32:38818) 2024-04-17T09:59:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f526fe00640 1 -- 172.21.15.32:0/180094381 learned_addr learned my addr 172.21.15.32:0/180094381 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f527d800640 1 --2- 172.21.15.32:0/180094381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f527815baa0 0x7f52781408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f527ce00640 1 --2- 172.21.15.32:0/180094381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f527806ef60 0x7f527814a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f526fe00640 1 -- 172.21.15.32:0/180094381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f527806ef60 msgr2=0x7f527814a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f526fe00640 1 --2- 172.21.15.32:0/180094381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f527806ef60 0x7f527814a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f526fe00640 1 -- 172.21.15.32:0/180094381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f527815baa0 msgr2=0x7f52781408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f526fe00640 1 --2- 172.21.15.32:0/180094381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f527815baa0 0x7f52781408a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f526fe00640 1 -- 172.21.15.32:0/180094381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527806c850 con 0x7f5278144e90 2024-04-17T09:59:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f527d800640 1 --2- 172.21.15.32:0/180094381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f527815baa0 0x7f52781408a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:59:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f527ce00640 1 --2- 172.21.15.32:0/180094381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f527806ef60 0x7f527814a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:59:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f526fe00640 1 --2- 172.21.15.32:0/180094381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5278144e90 0x7f5278142d10 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f52600097c0 tx=0x7f526001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:18.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f5275600640 1 -- 172.21.15.32:0/180094381 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52600b0050 con 0x7f5278144e90 2024-04-17T09:59:18.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f5275600640 1 -- 172.21.15.32:0/180094381 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52600b1040 con 0x7f5278144e90 2024-04-17T09:59:18.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f5275600640 1 -- 172.21.15.32:0/180094381 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5260006e20 con 0x7f5278144e90 2024-04-17T09:59:18.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f527f800640 1 -- 172.21.15.32:0/180094381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5278143250 con 0x7f5278144e90 2024-04-17T09:59:18.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f527f800640 1 -- 172.21.15.32:0/180094381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52780b05a0 con 0x7f5278144e90 2024-04-17T09:59:18.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f527f800640 1 -- 172.21.15.32:0/180094381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5278068a90 con 0x7f5278144e90 2024-04-17T09:59:18.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.482+0000 7f5275600640 1 -- 172.21.15.32:0/180094381 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f52600be060 con 0x7f5278144e90 2024-04-17T09:59:18.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.486+0000 7f5275600640 1 --2- 172.21.15.32:0/180094381 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5254068380 0x7f525406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:18.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.486+0000 7f5275600640 1 -- 172.21.15.32:0/180094381 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f5260120330 con 0x7f5278144e90 2024-04-17T09:59:18.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.486+0000 7f527ce00640 1 --2- 172.21.15.32:0/180094381 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5254068380 0x7f525406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:18.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.486+0000 7f527ce00640 1 --2- 172.21.15.32:0/180094381 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5254068380 0x7f525406a840 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5268000f70 tx=0x7f5268009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:18.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.490+0000 7f5275600640 1 -- 172.21.15.32:0/180094381 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5260016090 con 0x7f5278144e90 2024-04-17T09:59:18.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.654+0000 7f527f800640 1 -- 172.21.15.32:0/180094381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f527806cdd0 con 0x7f5278144e90 2024-04-17T09:59:18.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.654+0000 7f5275600640 1 -- 172.21.15.32:0/180094381 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16672 (secure 0 0 0) 0x7f526011d130 con 0x7f5278144e90 2024-04-17T09:59:18.652 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:59:18.653 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":82,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:59:06.422274+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:59:03.223426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-17T09:59:03.780120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:59:18.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.658+0000 7f527f800640 1 -- 172.21.15.32:0/180094381 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5254068380 msgr2=0x7f525406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:18.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.658+0000 7f527f800640 1 --2- 172.21.15.32:0/180094381 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5254068380 0x7f525406a840 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5268000f70 tx=0x7f5268009040 comp rx=0 tx=0).stop 2024-04-17T09:59:18.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.658+0000 7f527f800640 1 -- 172.21.15.32:0/180094381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5278144e90 msgr2=0x7f5278142d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:18.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.658+0000 7f527f800640 1 --2- 172.21.15.32:0/180094381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5278144e90 0x7f5278142d10 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f52600097c0 tx=0x7f526001ace0 comp rx=0 tx=0).stop 2024-04-17T09:59:18.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.658+0000 7f527f800640 1 -- 172.21.15.32:0/180094381 shutdown_connections 2024-04-17T09:59:18.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.658+0000 7f527f800640 1 --2- 172.21.15.32:0/180094381 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5254068380 0x7f525406a840 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:18.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.658+0000 7f527f800640 1 --2- 172.21.15.32:0/180094381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f527815baa0 0x7f52781408a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:18.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.658+0000 7f527f800640 1 --2- 172.21.15.32:0/180094381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5278144e90 0x7f5278142d10 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:18.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.658+0000 7f527f800640 1 --2- 172.21.15.32:0/180094381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f527806ef60 0x7f527814a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:18.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.658+0000 7f527f800640 1 -- 172.21.15.32:0/180094381 >> 172.21.15.32:0/180094381 conn(0x7f527806ba30 msgr2=0x7f52780b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:18.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.658+0000 7f527f800640 1 -- 172.21.15.32:0/180094381 shutdown_connections 2024-04-17T09:59:18.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.658+0000 7f527f800640 1 -- 172.21.15.32:0/180094381 wait complete. 2024-04-17T09:59:18.674 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:59:03.223426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:59:03.780120+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:59:18.674 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:59:18.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.818+0000 7f8a2ee00640 1 Processor -- start 2024-04-17T09:59:18.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.818+0000 7f8a2ee00640 1 -- start start 2024-04-17T09:59:18.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.822+0000 7f8a2ee00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8a2815b7d0 0x7f8a2806e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:18.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.822+0000 7f8a2ee00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a2806f7c0 0x7f8a2806ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:18.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.822+0000 7f8a2ee00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a280cc940 con 0x7f8a2806f270 2024-04-17T09:59:18.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.822+0000 7f8a2ee00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8a280ccc20 con 0x7f8a2815b7d0 2024-04-17T09:59:18.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.822+0000 7f8a2ee00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8a2806e0b0 con 0x7f8a2806f7c0 2024-04-17T09:59:18.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.822+0000 7f8a27e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8a2815b7d0 0x7f8a2806e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:18.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.822+0000 7f8a2ce00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8a2806f270 0x7f8a280ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52104/0 (socket says 172.21.15.32:52104) 2024-04-17T09:59:18.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.822+0000 7f8a27e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8a2815b7d0 0x7f8a2806e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38828/0 (socket says 172.21.15.32:38828) 2024-04-17T09:59:18.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.822+0000 7f8a2ce00640 1 -- 172.21.15.32:0/147360869 learned_addr learned my addr 172.21.15.32:0/147360869 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:18.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.822+0000 7f8a1fe00640 1 --2- 172.21.15.32:0/147360869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a2806f7c0 0x7f8a2806ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:18.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.822+0000 7f8a27e00640 1 -- 172.21.15.32:0/147360869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a2806f7c0 msgr2=0x7f8a2806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:18.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.822+0000 7f8a27e00640 1 --2- 172.21.15.32:0/147360869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a2806f7c0 0x7f8a2806ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:18.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.822+0000 7f8a27e00640 1 -- 172.21.15.32:0/147360869 >> v1:172.21.15.32:6789/0 conn(0x7f8a2806f270 legacy=0x7f8a280ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:18.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.822+0000 7f8a27e00640 1 -- 172.21.15.32:0/147360869 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a2806e390 con 0x7f8a2815b7d0 2024-04-17T09:59:18.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.826+0000 7f8a27e00640 1 --2- 172.21.15.32:0/147360869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a2815b7d0 0x7f8a2806e8b0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f8a180097c0 tx=0x7f8a1801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a6a35e2fc80490 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:18.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.826+0000 7f8a27400640 1 -- 172.21.15.32:0/147360869 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a180ab430 con 0x7f8a2815b7d0 2024-04-17T09:59:18.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.826+0000 7f8a27400640 1 -- 172.21.15.32:0/147360869 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a180b0070 con 0x7f8a2815b7d0 2024-04-17T09:59:18.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.826+0000 7f8a27400640 1 -- 172.21.15.32:0/147360869 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a180ab7f0 con 0x7f8a2815b7d0 2024-04-17T09:59:18.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.826+0000 7f8a2ee00640 1 -- 172.21.15.32:0/147360869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a2815b7d0 msgr2=0x7f8a2806e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:18.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.826+0000 7f8a2ee00640 1 --2- 172.21.15.32:0/147360869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a2815b7d0 0x7f8a2806e8b0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f8a180097c0 tx=0x7f8a1801c410 comp rx=0 tx=0).stop 2024-04-17T09:59:18.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.826+0000 7f8a2ee00640 1 -- 172.21.15.32:0/147360869 shutdown_connections 2024-04-17T09:59:18.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.826+0000 7f8a2ee00640 1 --2- 172.21.15.32:0/147360869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a2806f7c0 0x7f8a2806ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:18.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.826+0000 7f8a2ee00640 1 --2- 172.21.15.32:0/147360869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a2815b7d0 0x7f8a2806e8b0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:18.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.826+0000 7f8a2ee00640 1 -- 172.21.15.32:0/147360869 >> 172.21.15.32:0/147360869 conn(0x7f8a280b2120 msgr2=0x7f8a280b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:18.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.826+0000 7f8a2ee00640 1 -- 172.21.15.32:0/147360869 shutdown_connections 2024-04-17T09:59:18.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.826+0000 7f8a2ee00640 1 -- 172.21.15.32:0/147360869 wait complete. 2024-04-17T09:59:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.826+0000 7f8a2ee00640 1 Processor -- start 2024-04-17T09:59:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.826+0000 7f8a2ee00640 1 -- start start 2024-04-17T09:59:18.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a2ee00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a2806f270 0x7f8a28168080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:18.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a2ee00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a2806f7c0 0x7f8a28142e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:18.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a2ee00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a2815b7d0 0x7f8a2814e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:18.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a2ee00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8a28166aa0 con 0x7f8a2806f7c0 2024-04-17T09:59:18.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a2ee00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2806c380 con 0x7f8a2806f270 2024-04-17T09:59:18.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a2ee00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8a28166c20 con 0x7f8a2815b7d0 2024-04-17T09:59:18.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a1fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a2806f7c0 0x7f8a28142e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:18.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a1fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a2806f7c0 0x7f8a28142e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38834/0 (socket says 172.21.15.32:38834) 2024-04-17T09:59:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a1fe00640 1 -- 172.21.15.32:0/737943809 learned_addr learned my addr 172.21.15.32:0/737943809 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a27e00640 1 --2- 172.21.15.32:0/737943809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a2806f270 0x7f8a28168080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a2ce00640 1 --2- 172.21.15.32:0/737943809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a2815b7d0 0x7f8a2814e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a1fe00640 1 -- 172.21.15.32:0/737943809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a2815b7d0 msgr2=0x7f8a2814e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a1fe00640 1 --2- 172.21.15.32:0/737943809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a2815b7d0 0x7f8a2814e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a1fe00640 1 -- 172.21.15.32:0/737943809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a2806f270 msgr2=0x7f8a28168080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a1fe00640 1 --2- 172.21.15.32:0/737943809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a2806f270 0x7f8a28168080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a1fe00640 1 -- 172.21.15.32:0/737943809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a28144e80 con 0x7f8a2806f7c0 2024-04-17T09:59:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a1fe00640 1 --2- 172.21.15.32:0/737943809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a2806f7c0 0x7f8a28142e70 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000d830 tx=0x7f8a1000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a24c00640 1 -- 172.21.15.32:0/737943809 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1000b6c0 con 0x7f8a2806f7c0 2024-04-17T09:59:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a24c00640 1 -- 172.21.15.32:0/737943809 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a10019070 con 0x7f8a2806f7c0 2024-04-17T09:59:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a24c00640 1 -- 172.21.15.32:0/737943809 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1000ba60 con 0x7f8a2806f7c0 2024-04-17T09:59:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a2ee00640 1 -- 172.21.15.32:0/737943809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a2814f0d0 con 0x7f8a2806f7c0 2024-04-17T09:59:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a2ee00640 1 -- 172.21.15.32:0/737943809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a2814f690 con 0x7f8a2806f7c0 2024-04-17T09:59:18.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a2ee00640 1 -- 172.21.15.32:0/737943809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a28068a90 con 0x7f8a2806f7c0 2024-04-17T09:59:18.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.830+0000 7f8a24c00640 1 -- 172.21.15.32:0/737943809 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8a1001e460 con 0x7f8a2806f7c0 2024-04-17T09:59:18.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.834+0000 7f8a24c00640 1 --2- 172.21.15.32:0/737943809 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8a04068380 0x7f8a0406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:18.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.834+0000 7f8a24c00640 1 -- 172.21.15.32:0/737943809 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f8a10087090 con 0x7f8a2806f7c0 2024-04-17T09:59:18.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.834+0000 7f8a27e00640 1 --2- 172.21.15.32:0/737943809 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8a04068380 0x7f8a0406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:18.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.834+0000 7f8a27e00640 1 --2- 172.21.15.32:0/737943809 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8a04068380 0x7f8a0406a840 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8a180af000 tx=0x7f8a18008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:18.838+0000 7f8a24c00640 1 -- 172.21.15.32:0/737943809 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f8a10057640 con 0x7f8a2806f7c0 2024-04-17T09:59:19.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.002+0000 7f8a2ee00640 1 -- 172.21.15.32:0/737943809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8a2814f8d0 con 0x7f8a2806f7c0 2024-04-17T09:59:19.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.002+0000 7f8a24c00640 1 -- 172.21.15.32:0/737943809 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f8a10056f40 con 0x7f8a2806f7c0 2024-04-17T09:59:19.001 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:59:19.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.006+0000 7f8a2ee00640 1 -- 172.21.15.32:0/737943809 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8a04068380 msgr2=0x7f8a0406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:19.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.006+0000 7f8a2ee00640 1 --2- 172.21.15.32:0/737943809 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8a04068380 0x7f8a0406a840 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8a180af000 tx=0x7f8a18008040 comp rx=0 tx=0).stop 2024-04-17T09:59:19.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.006+0000 7f8a2ee00640 1 -- 172.21.15.32:0/737943809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a2806f7c0 msgr2=0x7f8a28142e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:19.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.006+0000 7f8a2ee00640 1 --2- 172.21.15.32:0/737943809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a2806f7c0 0x7f8a28142e70 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000d830 tx=0x7f8a1000dcf0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.006+0000 7f8a2ee00640 1 -- 172.21.15.32:0/737943809 shutdown_connections 2024-04-17T09:59:19.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.006+0000 7f8a2ee00640 1 --2- 172.21.15.32:0/737943809 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8a04068380 0x7f8a0406a840 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.006+0000 7f8a2ee00640 1 --2- 172.21.15.32:0/737943809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a2815b7d0 0x7f8a2814e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.006+0000 7f8a2ee00640 1 --2- 172.21.15.32:0/737943809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a2806f7c0 0x7f8a28142e70 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.006+0000 7f8a2ee00640 1 --2- 172.21.15.32:0/737943809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a2806f270 0x7f8a28168080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.006+0000 7f8a2ee00640 1 -- 172.21.15.32:0/737943809 >> 172.21.15.32:0/737943809 conn(0x7f8a280b2120 msgr2=0x7f8a280cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:19.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.006+0000 7f8a2ee00640 1 -- 172.21.15.32:0/737943809 shutdown_connections 2024-04-17T09:59:19.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.006+0000 7f8a2ee00640 1 -- 172.21.15.32:0/737943809 wait complete. 2024-04-17T09:59:19.022 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:59:19.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.166+0000 7f6682600640 1 Processor -- start 2024-04-17T09:59:19.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.166+0000 7f6682600640 1 -- start start 2024-04-17T09:59:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.170+0000 7f6682600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f667c1656d0 0x7f667c167ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.170+0000 7f6682600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f667c0d0130 con 0x7f667c0cf370 2024-04-17T09:59:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.170+0000 7f6682600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f667c15d090 con 0x7f667c06ba30 2024-04-17T09:59:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.170+0000 7f6682600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f667c0cfef0 con 0x7f667c1656d0 2024-04-17T09:59:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.170+0000 7f667be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f667c1656d0 0x7f667c167ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.170+0000 7f6673e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f667c0cf370 0x7f667c0cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52116/0 (socket says 172.21.15.32:52116) 2024-04-17T09:59:19.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.170+0000 7f6673e00640 1 -- 172.21.15.32:0/2788556198 learned_addr learned my addr 172.21.15.32:0/2788556198 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:19.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.170+0000 7f667be00640 1 -- 172.21.15.32:0/2788556198 >> v1:172.21.15.144:6789/0 conn(0x7f667c06ba30 legacy=0x7f667c06be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:19.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.170+0000 7f667be00640 1 -- 172.21.15.32:0/2788556198 >> v1:172.21.15.32:6789/0 conn(0x7f667c0cf370 legacy=0x7f667c0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:19.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.170+0000 7f667be00640 1 -- 172.21.15.32:0/2788556198 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667c15d370 con 0x7f667c1656d0 2024-04-17T09:59:19.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.170+0000 7f667aa00640 1 -- 172.21.15.32:0/2788556198 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 191651607 0 0) 0x7f667c0d0130 con 0x7f667c0cf370 2024-04-17T09:59:19.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.174+0000 7f667be00640 1 --2- 172.21.15.32:0/2788556198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f667c1656d0 0x7f667c167ac0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f6674009d90 tx=0x7f667401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3790a367136dad2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:19.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.174+0000 7f667aa00640 1 -- 172.21.15.32:0/2788556198 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66740b2070 con 0x7f667c1656d0 2024-04-17T09:59:19.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.174+0000 7f667aa00640 1 -- 172.21.15.32:0/2788556198 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66740a5870 con 0x7f667c1656d0 2024-04-17T09:59:19.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.174+0000 7f667aa00640 1 -- 172.21.15.32:0/2788556198 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66740a5c40 con 0x7f667c1656d0 2024-04-17T09:59:19.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.174+0000 7f6682600640 1 -- 172.21.15.32:0/2788556198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f667c1656d0 msgr2=0x7f667c167ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:19.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.174+0000 7f6682600640 1 --2- 172.21.15.32:0/2788556198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f667c1656d0 0x7f667c167ac0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f6674009d90 tx=0x7f667401c8a0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.174+0000 7f6682600640 1 -- 172.21.15.32:0/2788556198 shutdown_connections 2024-04-17T09:59:19.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.174+0000 7f6682600640 1 --2- 172.21.15.32:0/2788556198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f667c1656d0 0x7f667c167ac0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.174+0000 7f6682600640 1 -- 172.21.15.32:0/2788556198 >> 172.21.15.32:0/2788556198 conn(0x7f667c0b2120 msgr2=0x7f667c0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:19.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.174+0000 7f6682600640 1 -- 172.21.15.32:0/2788556198 shutdown_connections 2024-04-17T09:59:19.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.174+0000 7f6682600640 1 -- 172.21.15.32:0/2788556198 wait complete. 2024-04-17T09:59:19.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.174+0000 7f6682600640 1 Processor -- start 2024-04-17T09:59:19.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.174+0000 7f6682600640 1 -- start start 2024-04-17T09:59:19.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.174+0000 7f6682600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f667c06ba30 0x7f667c15ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:19.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.174+0000 7f6682600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f667c0cf370 0x7f667c13eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:19.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.174+0000 7f6682600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f667c1656d0 0x7f667c141230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.174+0000 7f6682600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f667c146230 con 0x7f667c0cf370 2024-04-17T09:59:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.174+0000 7f6682600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f667c15dd80 con 0x7f667c06ba30 2024-04-17T09:59:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.174+0000 7f6682600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f667c1463b0 con 0x7f667c1656d0 2024-04-17T09:59:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.178+0000 7f6673e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f667c0cf370 0x7f667c13eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.178+0000 7f6673e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f667c0cf370 0x7f667c13eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38858/0 (socket says 172.21.15.32:38858) 2024-04-17T09:59:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.178+0000 7f6673e00640 1 -- 172.21.15.32:0/1917045019 learned_addr learned my addr 172.21.15.32:0/1917045019 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.178+0000 7f667be00640 1 --2- 172.21.15.32:0/1917045019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f667c1656d0 0x7f667c141230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.178+0000 7f667b400640 1 --2- 172.21.15.32:0/1917045019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f667c06ba30 0x7f667c15ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.178+0000 7f6673e00640 1 -- 172.21.15.32:0/1917045019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f667c1656d0 msgr2=0x7f667c141230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.178+0000 7f6673e00640 1 --2- 172.21.15.32:0/1917045019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f667c1656d0 0x7f667c141230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.178+0000 7f6673e00640 1 -- 172.21.15.32:0/1917045019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f667c06ba30 msgr2=0x7f667c15ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.178+0000 7f6673e00640 1 --2- 172.21.15.32:0/1917045019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f667c06ba30 0x7f667c15ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.178+0000 7f6673e00640 1 -- 172.21.15.32:0/1917045019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667c140600 con 0x7f667c0cf370 2024-04-17T09:59:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.178+0000 7f6673e00640 1 --2- 172.21.15.32:0/1917045019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f667c0cf370 0x7f667c13eb30 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f666400c520 tx=0x7f666400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.178+0000 7f6673400640 1 -- 172.21.15.32:0/1917045019 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664018070 con 0x7f667c0cf370 2024-04-17T09:59:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.178+0000 7f6673400640 1 -- 172.21.15.32:0/1917045019 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66640074e0 con 0x7f667c0cf370 2024-04-17T09:59:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.178+0000 7f6673400640 1 -- 172.21.15.32:0/1917045019 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66640049f0 con 0x7f667c0cf370 2024-04-17T09:59:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.178+0000 7f6682600640 1 -- 172.21.15.32:0/1917045019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667c13f560 con 0x7f667c0cf370 2024-04-17T09:59:19.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.178+0000 7f6682600640 1 -- 172.21.15.32:0/1917045019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f667c13fb20 con 0x7f667c0cf370 2024-04-17T09:59:19.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.178+0000 7f6682600640 1 -- 172.21.15.32:0/1917045019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f667c068a90 con 0x7f667c0cf370 2024-04-17T09:59:19.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.178+0000 7f6673400640 1 -- 172.21.15.32:0/1917045019 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f666401b070 con 0x7f667c0cf370 2024-04-17T09:59:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.182+0000 7f6673400640 1 --2- 172.21.15.32:0/1917045019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6658068380 0x7f665806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.182+0000 7f6673400640 1 -- 172.21.15.32:0/1917045019 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f6664085cd0 con 0x7f667c0cf370 2024-04-17T09:59:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.182+0000 7f667b400640 1 --2- 172.21.15.32:0/1917045019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6658068380 0x7f665806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.182+0000 7f667b400640 1 --2- 172.21.15.32:0/1917045019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6658068380 0x7f665806a840 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f666c007fd0 tx=0x7f666c007e40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:19.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.186+0000 7f6673400640 1 -- 172.21.15.32:0/1917045019 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f6664055c30 con 0x7f667c0cf370 2024-04-17T09:59:19.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.354+0000 7f6682600640 1 -- 172.21.15.32:0/1917045019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f667c06c850 con 0x7f667c0cf370 2024-04-17T09:59:19.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.354+0000 7f6673400640 1 -- 172.21.15.32:0/1917045019 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7f6664059f30 con 0x7f667c0cf370 2024-04-17T09:59:19.352 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:59:19.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.358+0000 7f6682600640 1 -- 172.21.15.32:0/1917045019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6658068380 msgr2=0x7f665806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:19.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.358+0000 7f6682600640 1 --2- 172.21.15.32:0/1917045019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6658068380 0x7f665806a840 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f666c007fd0 tx=0x7f666c007e40 comp rx=0 tx=0).stop 2024-04-17T09:59:19.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.358+0000 7f6682600640 1 -- 172.21.15.32:0/1917045019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f667c0cf370 msgr2=0x7f667c13eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:19.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.358+0000 7f6682600640 1 --2- 172.21.15.32:0/1917045019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f667c0cf370 0x7f667c13eb30 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f666400c520 tx=0x7f666400c9e0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.358+0000 7f6682600640 1 -- 172.21.15.32:0/1917045019 shutdown_connections 2024-04-17T09:59:19.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.358+0000 7f6682600640 1 --2- 172.21.15.32:0/1917045019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6658068380 0x7f665806a840 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.358+0000 7f6682600640 1 --2- 172.21.15.32:0/1917045019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f667c1656d0 0x7f667c141230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.358+0000 7f6682600640 1 --2- 172.21.15.32:0/1917045019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f667c0cf370 0x7f667c13eb30 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.358+0000 7f6682600640 1 --2- 172.21.15.32:0/1917045019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f667c06ba30 0x7f667c15ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.358+0000 7f6682600640 1 -- 172.21.15.32:0/1917045019 >> 172.21.15.32:0/1917045019 conn(0x7f667c0b2120 msgr2=0x7f667c06ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:19.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.358+0000 7f6682600640 1 -- 172.21.15.32:0/1917045019 shutdown_connections 2024-04-17T09:59:19.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.358+0000 7f6682600640 1 -- 172.21.15.32:0/1917045019 wait complete. 2024-04-17T09:59:19.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:59:19.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.522+0000 7fdd88200640 1 Processor -- start 2024-04-17T09:59:19.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.522+0000 7fdd88200640 1 -- start start 2024-04-17T09:59:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.522+0000 7fdd88200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd8006d150 0x7fdd800af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.522+0000 7fdd88200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdd800afce0 0x7fdd800b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.522+0000 7fdd88200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd800b0540 con 0x7fdd80141700 2024-04-17T09:59:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.522+0000 7fdd88200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fdd800b0820 con 0x7fdd800afce0 2024-04-17T09:59:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.522+0000 7fdd88200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdd800b3ef0 con 0x7fdd8006d150 2024-04-17T09:59:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.522+0000 7fdd86200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdd800afce0 0x7fdd800b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.522+0000 7fdd85800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdd80141700 0x7fdd8006ca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52124/0 (socket says 172.21.15.32:52124) 2024-04-17T09:59:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.522+0000 7fdd86200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdd800afce0 0x7fdd800b49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38866/0 (socket says 172.21.15.32:38866) 2024-04-17T09:59:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.522+0000 7fdd85800640 1 -- 172.21.15.32:0/4149589678 learned_addr learned my addr 172.21.15.32:0/4149589678 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.522+0000 7fdd84e00640 1 --2- 172.21.15.32:0/4149589678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd8006d150 0x7fdd800af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.522+0000 7fdd86200640 1 -- 172.21.15.32:0/4149589678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd8006d150 msgr2=0x7fdd800af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.522+0000 7fdd86200640 1 --2- 172.21.15.32:0/4149589678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd8006d150 0x7fdd800af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.522+0000 7fdd86200640 1 -- 172.21.15.32:0/4149589678 >> v1:172.21.15.32:6789/0 conn(0x7fdd80141700 legacy=0x7fdd8006ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.522+0000 7fdd7fe00640 1 -- 172.21.15.32:0/4149589678 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3558732406 0 0) 0x7fdd800b0540 con 0x7fdd80141700 2024-04-17T09:59:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.522+0000 7fdd86200640 1 -- 172.21.15.32:0/4149589678 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd800b41d0 con 0x7fdd800afce0 2024-04-17T09:59:19.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.526+0000 7fdd86200640 1 --2- 172.21.15.32:0/4149589678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd800afce0 0x7fdd800b49d0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fdd78009d90 tx=0x7fdd7801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c766d2694167ea60 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:19.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.526+0000 7fdd7fe00640 1 -- 172.21.15.32:0/4149589678 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd780b2070 con 0x7fdd800afce0 2024-04-17T09:59:19.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.526+0000 7fdd7fe00640 1 -- 172.21.15.32:0/4149589678 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd780a5a00 con 0x7fdd800afce0 2024-04-17T09:59:19.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.526+0000 7fdd7fe00640 1 -- 172.21.15.32:0/4149589678 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd780a5dd0 con 0x7fdd800afce0 2024-04-17T09:59:19.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.526+0000 7fdd88200640 1 -- 172.21.15.32:0/4149589678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd800afce0 msgr2=0x7fdd800b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:19.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.526+0000 7fdd88200640 1 --2- 172.21.15.32:0/4149589678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd800afce0 0x7fdd800b49d0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fdd78009d90 tx=0x7fdd7801c920 comp rx=0 tx=0).stop 2024-04-17T09:59:19.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.526+0000 7fdd88200640 1 -- 172.21.15.32:0/4149589678 shutdown_connections 2024-04-17T09:59:19.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.526+0000 7fdd88200640 1 --2- 172.21.15.32:0/4149589678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd800afce0 0x7fdd800b49d0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.526+0000 7fdd88200640 1 --2- 172.21.15.32:0/4149589678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd8006d150 0x7fdd800af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.526+0000 7fdd88200640 1 -- 172.21.15.32:0/4149589678 >> 172.21.15.32:0/4149589678 conn(0x7fdd80072670 msgr2=0x7fdd80072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:19.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.526+0000 7fdd88200640 1 -- 172.21.15.32:0/4149589678 shutdown_connections 2024-04-17T09:59:19.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.526+0000 7fdd88200640 1 -- 172.21.15.32:0/4149589678 wait complete. 2024-04-17T09:59:19.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.526+0000 7fdd88200640 1 Processor -- start 2024-04-17T09:59:19.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.526+0000 7fdd88200640 1 -- start start 2024-04-17T09:59:19.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd88200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd8006d150 0x7fdd8013c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:19.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd88200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd800afce0 0x7fdd8013e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:19.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd88200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd80141700 0x7fdd80144300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:19.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd88200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdd800b5720 con 0x7fdd8006d150 2024-04-17T09:59:19.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd88200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdd80141cf0 con 0x7fdd80141700 2024-04-17T09:59:19.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd88200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdd800b58a0 con 0x7fdd800afce0 2024-04-17T09:59:19.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd85800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd8006d150 0x7fdd8013c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:19.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd85800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd8006d150 0x7fdd8013c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38878/0 (socket says 172.21.15.32:38878) 2024-04-17T09:59:19.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd85800640 1 -- 172.21.15.32:0/1419324776 learned_addr learned my addr 172.21.15.32:0/1419324776 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:19.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd84e00640 1 --2- 172.21.15.32:0/1419324776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd800afce0 0x7fdd8013e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:19.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd86200640 1 --2- 172.21.15.32:0/1419324776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd80141700 0x7fdd80144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:19.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd85800640 1 -- 172.21.15.32:0/1419324776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd800afce0 msgr2=0x7fdd8013e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:19.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd85800640 1 --2- 172.21.15.32:0/1419324776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd800afce0 0x7fdd8013e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd85800640 1 -- 172.21.15.32:0/1419324776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd80141700 msgr2=0x7fdd80144300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:19.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd85800640 1 --2- 172.21.15.32:0/1419324776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd80141700 0x7fdd80144300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd85800640 1 -- 172.21.15.32:0/1419324776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd800addc0 con 0x7fdd8006d150 2024-04-17T09:59:19.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd85800640 1 --2- 172.21.15.32:0/1419324776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd8006d150 0x7fdd8013c890 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400c480 tx=0x7fdd7400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:19.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd7d600640 1 -- 172.21.15.32:0/1419324776 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd74018070 con 0x7fdd8006d150 2024-04-17T09:59:19.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd7d600640 1 -- 172.21.15.32:0/1419324776 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd74007500 con 0x7fdd8006d150 2024-04-17T09:59:19.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd88200640 1 -- 172.21.15.32:0/1419324776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd8013e6e0 con 0x7fdd8006d150 2024-04-17T09:59:19.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd7d600640 1 -- 172.21.15.32:0/1419324776 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd74004a30 con 0x7fdd8006d150 2024-04-17T09:59:19.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd88200640 1 -- 172.21.15.32:0/1419324776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd8013d900 con 0x7fdd8006d150 2024-04-17T09:59:19.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd88200640 1 -- 172.21.15.32:0/1419324776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd80068a90 con 0x7fdd8006d150 2024-04-17T09:59:19.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.530+0000 7fdd7d600640 1 -- 172.21.15.32:0/1419324776 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdd7401b070 con 0x7fdd8006d150 2024-04-17T09:59:19.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.534+0000 7fdd7d600640 1 --2- 172.21.15.32:0/1419324776 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdd60068380 0x7fdd6006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:19.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.534+0000 7fdd7d600640 1 -- 172.21.15.32:0/1419324776 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fdd74085c30 con 0x7fdd8006d150 2024-04-17T09:59:19.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.534+0000 7fdd84e00640 1 --2- 172.21.15.32:0/1419324776 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdd60068380 0x7fdd6006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:19.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.534+0000 7fdd84e00640 1 --2- 172.21.15.32:0/1419324776 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdd60068380 0x7fdd6006a840 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fdd800af210 tx=0x7fdd6c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:19.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.538+0000 7fdd7d600640 1 -- 172.21.15.32:0/1419324776 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fdd74055b90 con 0x7fdd8006d150 2024-04-17T09:59:19.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.702+0000 7fdd88200640 1 -- 172.21.15.32:0/1419324776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdd8015b5e0 con 0x7fdd8006d150 2024-04-17T09:59:19.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.702+0000 7fdd7d600640 1 -- 172.21.15.32:0/1419324776 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7fdd74059e90 con 0x7fdd8006d150 2024-04-17T09:59:19.700 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:59:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.702+0000 7fdd88200640 1 -- 172.21.15.32:0/1419324776 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdd60068380 msgr2=0x7fdd6006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.702+0000 7fdd88200640 1 --2- 172.21.15.32:0/1419324776 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdd60068380 0x7fdd6006a840 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fdd800af210 tx=0x7fdd6c008040 comp rx=0 tx=0).stop 2024-04-17T09:59:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.702+0000 7fdd88200640 1 -- 172.21.15.32:0/1419324776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd8006d150 msgr2=0x7fdd8013c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.702+0000 7fdd88200640 1 --2- 172.21.15.32:0/1419324776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd8006d150 0x7fdd8013c890 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400c480 tx=0x7fdd7400c940 comp rx=0 tx=0).stop 2024-04-17T09:59:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.702+0000 7fdd88200640 1 -- 172.21.15.32:0/1419324776 shutdown_connections 2024-04-17T09:59:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.702+0000 7fdd88200640 1 --2- 172.21.15.32:0/1419324776 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdd60068380 0x7fdd6006a840 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.702+0000 7fdd88200640 1 --2- 172.21.15.32:0/1419324776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd800afce0 0x7fdd8013e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.702+0000 7fdd88200640 1 --2- 172.21.15.32:0/1419324776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd80141700 0x7fdd80144300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.702+0000 7fdd88200640 1 --2- 172.21.15.32:0/1419324776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd8006d150 0x7fdd8013c890 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.702+0000 7fdd88200640 1 -- 172.21.15.32:0/1419324776 >> 172.21.15.32:0/1419324776 conn(0x7fdd80072670 msgr2=0x7fdd8006c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.706+0000 7fdd88200640 1 -- 172.21.15.32:0/1419324776 shutdown_connections 2024-04-17T09:59:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.706+0000 7fdd88200640 1 -- 172.21.15.32:0/1419324776 wait complete. 2024-04-17T09:59:19.721 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T09:59:19.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.866+0000 7f2665600640 1 Processor -- start 2024-04-17T09:59:19.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.866+0000 7f2665600640 1 -- start start 2024-04-17T09:59:19.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.866+0000 7f2665600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26600cf370 0x7f26600d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:19.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.866+0000 7f2665600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26600cff60 con 0x7f26600d0820 2024-04-17T09:59:19.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.866+0000 7f2665600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26601601e0 con 0x7f266013f6e0 2024-04-17T09:59:19.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.866+0000 7f2665600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f266013fac0 con 0x7f26600cf370 2024-04-17T09:59:19.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.866+0000 7f265f400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f26600d0820 0x7f2660071d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52134/0 (socket says 172.21.15.32:52134) 2024-04-17T09:59:19.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.866+0000 7f265f400640 1 -- 172.21.15.32:0/3079410609 learned_addr learned my addr 172.21.15.32:0/3079410609 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:19.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.866+0000 7f2656000640 1 --2- 172.21.15.32:0/3079410609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26600cf370 0x7f26600d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:19.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.866+0000 7f265e000640 1 -- 172.21.15.32:0/3079410609 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2093440872 0 0) 0x7f26600cff60 con 0x7f26600d0820 2024-04-17T09:59:19.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.866+0000 7f265e000640 1 -- 172.21.15.32:0/3079410609 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2644003530 con 0x7f26600d0820 2024-04-17T09:59:19.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.866+0000 7f2656000640 1 -- 172.21.15.32:0/3079410609 >> v1:172.21.15.144:6789/0 conn(0x7f266013f6e0 legacy=0x7f26600cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:19.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.866+0000 7f265e000640 1 -- 172.21.15.32:0/3079410609 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2466991835 0 0) 0x7f26601601e0 con 0x7f266013f6e0 2024-04-17T09:59:19.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.866+0000 7f2656000640 1 -- 172.21.15.32:0/3079410609 >> v1:172.21.15.32:6789/0 conn(0x7f26600d0820 legacy=0x7f2660071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:19.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.866+0000 7f2656000640 1 -- 172.21.15.32:0/3079410609 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2660160480 con 0x7f26600cf370 2024-04-17T09:59:19.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.870+0000 7f2656000640 1 --2- 172.21.15.32:0/3079410609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26600cf370 0x7f26600d02e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f26480097f0 tx=0x7f264801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c2c156153b0c9e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:19.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.870+0000 7f265e000640 1 -- 172.21.15.32:0/3079410609 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26480b1070 con 0x7f26600cf370 2024-04-17T09:59:19.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.870+0000 7f265e000640 1 -- 172.21.15.32:0/3079410609 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26480ac630 con 0x7f26600cf370 2024-04-17T09:59:19.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.870+0000 7f265e000640 1 -- 172.21.15.32:0/3079410609 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26480a8050 con 0x7f26600cf370 2024-04-17T09:59:19.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.870+0000 7f2665600640 1 -- 172.21.15.32:0/3079410609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26600cf370 msgr2=0x7f26600d02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:19.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.870+0000 7f2665600640 1 --2- 172.21.15.32:0/3079410609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26600cf370 0x7f26600d02e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f26480097f0 tx=0x7f264801c4e0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.870+0000 7f2665600640 1 -- 172.21.15.32:0/3079410609 shutdown_connections 2024-04-17T09:59:19.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.870+0000 7f2665600640 1 --2- 172.21.15.32:0/3079410609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26600cf370 0x7f26600d02e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.870+0000 7f2665600640 1 -- 172.21.15.32:0/3079410609 >> 172.21.15.32:0/3079410609 conn(0x7f266006ba30 msgr2=0x7f266006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:19.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.870+0000 7f2665600640 1 -- 172.21.15.32:0/3079410609 shutdown_connections 2024-04-17T09:59:19.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.870+0000 7f2665600640 1 -- 172.21.15.32:0/3079410609 wait complete. 2024-04-17T09:59:19.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.870+0000 7f2665600640 1 Processor -- start 2024-04-17T09:59:19.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f2665600640 1 -- start start 2024-04-17T09:59:19.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f2665600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26600cf370 0x7f26601383c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:19.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f2665600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26600d0820 0x7f2660138c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:19.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f2665600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f266013f6e0 0x7f266013e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:19.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f2665600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f266016a590 con 0x7f26600d0820 2024-04-17T09:59:19.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f2665600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f266013fd40 con 0x7f26600cf370 2024-04-17T09:59:19.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f2665600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f266016a710 con 0x7f266013f6e0 2024-04-17T09:59:19.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f2656000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26600d0820 0x7f2660138c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:19.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f2656000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26600d0820 0x7f2660138c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38894/0 (socket says 172.21.15.32:38894) 2024-04-17T09:59:19.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f2656000640 1 -- 172.21.15.32:0/2924501645 learned_addr learned my addr 172.21.15.32:0/2924501645 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:19.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f265ea00640 1 --2- 172.21.15.32:0/2924501645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26600cf370 0x7f26601383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:19.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f265f400640 1 --2- 172.21.15.32:0/2924501645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f266013f6e0 0x7f266013e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:19.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f2656000640 1 -- 172.21.15.32:0/2924501645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f266013f6e0 msgr2=0x7f266013e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:19.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f2656000640 1 --2- 172.21.15.32:0/2924501645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f266013f6e0 0x7f266013e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f2656000640 1 -- 172.21.15.32:0/2924501645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26600cf370 msgr2=0x7f26601383c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:19.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f2656000640 1 --2- 172.21.15.32:0/2924501645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26600cf370 0x7f26601383c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:19.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f2656000640 1 -- 172.21.15.32:0/2924501645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2660142950 con 0x7f26600d0820 2024-04-17T09:59:19.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f265ea00640 1 --2- 172.21.15.32:0/2924501645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26600cf370 0x7f26601383c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:19.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f2656000640 1 --2- 172.21.15.32:0/2924501645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26600d0820 0x7f2660138c80 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f26480af040 tx=0x7f2648019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:19.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f2657400640 1 -- 172.21.15.32:0/2924501645 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26480b1040 con 0x7f26600d0820 2024-04-17T09:59:19.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f2657400640 1 -- 172.21.15.32:0/2924501645 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26480ac7e0 con 0x7f26600d0820 2024-04-17T09:59:19.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f2657400640 1 -- 172.21.15.32:0/2924501645 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26480acbb0 con 0x7f26600d0820 2024-04-17T09:59:19.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f2665600640 1 -- 172.21.15.32:0/2924501645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26600b1e90 con 0x7f26600d0820 2024-04-17T09:59:19.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f2665600640 1 -- 172.21.15.32:0/2924501645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26600b2450 con 0x7f26600d0820 2024-04-17T09:59:19.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.874+0000 7f2665600640 1 -- 172.21.15.32:0/2924501645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2660068a90 con 0x7f26600d0820 2024-04-17T09:59:19.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.878+0000 7f2657400640 1 -- 172.21.15.32:0/2924501645 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f26480ac4f0 con 0x7f26600d0820 2024-04-17T09:59:19.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.878+0000 7f2657400640 1 --2- 172.21.15.32:0/2924501645 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f263c068380 0x7f263c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:19.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.878+0000 7f2657400640 1 -- 172.21.15.32:0/2924501645 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f264811ffd0 con 0x7f26600d0820 2024-04-17T09:59:19.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.878+0000 7f265ea00640 1 --2- 172.21.15.32:0/2924501645 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f263c068380 0x7f263c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:19.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.878+0000 7f265ea00640 1 --2- 172.21.15.32:0/2924501645 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f263c068380 0x7f263c06a840 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2650007fd0 tx=0x7f2650009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:19.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:19.882+0000 7f2657400640 1 -- 172.21.15.32:0/2924501645 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f26480c3080 con 0x7f26600d0820 2024-04-17T09:59:20.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.062+0000 7f2665600640 1 -- 172.21.15.32:0/2924501645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f26600d1030 con 0x7f26600d0820 2024-04-17T09:59:20.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.062+0000 7f2657400640 1 -- 172.21.15.32:0/2924501645 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v82) ==== 80+0+0 (secure 0 0 0) 0x7f26480160d0 con 0x7f26600d0820 2024-04-17T09:59:20.059 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T09:59:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.062+0000 7f2665600640 1 -- 172.21.15.32:0/2924501645 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f263c068380 msgr2=0x7f263c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.062+0000 7f2665600640 1 --2- 172.21.15.32:0/2924501645 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f263c068380 0x7f263c06a840 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2650007fd0 tx=0x7f2650009040 comp rx=0 tx=0).stop 2024-04-17T09:59:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.062+0000 7f2665600640 1 -- 172.21.15.32:0/2924501645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26600d0820 msgr2=0x7f2660138c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.062+0000 7f2665600640 1 --2- 172.21.15.32:0/2924501645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26600d0820 0x7f2660138c80 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f26480af040 tx=0x7f2648019280 comp rx=0 tx=0).stop 2024-04-17T09:59:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.062+0000 7f2665600640 1 -- 172.21.15.32:0/2924501645 shutdown_connections 2024-04-17T09:59:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.062+0000 7f2665600640 1 --2- 172.21.15.32:0/2924501645 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f263c068380 0x7f263c06a840 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.062+0000 7f2665600640 1 --2- 172.21.15.32:0/2924501645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f266013f6e0 0x7f266013e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.062+0000 7f2665600640 1 --2- 172.21.15.32:0/2924501645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26600d0820 0x7f2660138c80 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.062+0000 7f2665600640 1 --2- 172.21.15.32:0/2924501645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26600cf370 0x7f26601383c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.062+0000 7f2665600640 1 -- 172.21.15.32:0/2924501645 >> 172.21.15.32:0/2924501645 conn(0x7f266006ba30 msgr2=0x7f2660157800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:20.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.062+0000 7f2665600640 1 -- 172.21.15.32:0/2924501645 shutdown_connections 2024-04-17T09:59:20.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.062+0000 7f2665600640 1 -- 172.21.15.32:0/2924501645 wait complete. 2024-04-17T09:59:20.080 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T09:59:20.080 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T09:59:20.089 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T09:59:20.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.270+0000 7ff63d000640 1 Processor -- start 2024-04-17T09:59:20.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.270+0000 7ff63d000640 1 -- start start 2024-04-17T09:59:20.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.274+0000 7ff63d000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff638144e90 0x7ff6380b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:20.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.274+0000 7ff63d000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff638072bb0 con 0x7ff63815c1b0 2024-04-17T09:59:20.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.274+0000 7ff63d000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6380b3dc0 con 0x7ff6380b4de0 2024-04-17T09:59:20.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.274+0000 7ff63d000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff638145270 con 0x7ff638144e90 2024-04-17T09:59:20.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.274+0000 7ff636800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff63815c1b0 0x7ff63806ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52138/0 (socket says 172.21.15.32:52138) 2024-04-17T09:59:20.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.274+0000 7ff636800640 1 -- 172.21.15.32:0/2361997513 learned_addr learned my addr 172.21.15.32:0/2361997513 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:20.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.274+0000 7ff635e00640 1 --2- 172.21.15.32:0/2361997513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff638144e90 0x7ff6380b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:20.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.274+0000 7ff635400640 1 -- 172.21.15.32:0/2361997513 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3768032373 0 0) 0x7ff638072bb0 con 0x7ff63815c1b0 2024-04-17T09:59:20.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.274+0000 7ff635400640 1 -- 172.21.15.32:0/2361997513 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff61c003530 con 0x7ff63815c1b0 2024-04-17T09:59:20.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.274+0000 7ff635400640 1 -- 172.21.15.32:0/2361997513 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2333428583 0 0) 0x7ff6380b3dc0 con 0x7ff6380b4de0 2024-04-17T09:59:20.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.274+0000 7ff635400640 1 -- 172.21.15.32:0/2361997513 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff638072bb0 con 0x7ff6380b4de0 2024-04-17T09:59:20.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.274+0000 7ff635e00640 1 -- 172.21.15.32:0/2361997513 >> v1:172.21.15.144:6789/0 conn(0x7ff6380b4de0 legacy=0x7ff63815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:20.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.274+0000 7ff635400640 1 -- 172.21.15.32:0/2361997513 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3608442246 0 0) 0x7ff61c003530 con 0x7ff63815c1b0 2024-04-17T09:59:20.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.274+0000 7ff635e00640 1 -- 172.21.15.32:0/2361997513 >> v1:172.21.15.32:6789/0 conn(0x7ff63815c1b0 legacy=0x7ff63806ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:20.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.274+0000 7ff635e00640 1 -- 172.21.15.32:0/2361997513 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6380b40a0 con 0x7ff638144e90 2024-04-17T09:59:20.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.278+0000 7ff635e00640 1 --2- 172.21.15.32:0/2361997513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff638144e90 0x7ff6380b48a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff628009dc0 tx=0x7ff62801c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22b6503053bc64c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:20.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.278+0000 7ff635400640 1 -- 172.21.15.32:0/2361997513 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6280b2070 con 0x7ff638144e90 2024-04-17T09:59:20.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.278+0000 7ff635400640 1 -- 172.21.15.32:0/2361997513 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6280a56e0 con 0x7ff638144e90 2024-04-17T09:59:20.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.278+0000 7ff635400640 1 -- 172.21.15.32:0/2361997513 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6300016a0 con 0x7ff638144e90 2024-04-17T09:59:20.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.278+0000 7ff63d000640 1 -- 172.21.15.32:0/2361997513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff638144e90 msgr2=0x7ff6380b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:20.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.278+0000 7ff63d000640 1 --2- 172.21.15.32:0/2361997513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff638144e90 0x7ff6380b48a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff628009dc0 tx=0x7ff62801c880 comp rx=0 tx=0).stop 2024-04-17T09:59:20.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.278+0000 7ff63d000640 1 -- 172.21.15.32:0/2361997513 shutdown_connections 2024-04-17T09:59:20.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.278+0000 7ff63d000640 1 --2- 172.21.15.32:0/2361997513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff638144e90 0x7ff6380b48a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.278+0000 7ff63d000640 1 -- 172.21.15.32:0/2361997513 >> 172.21.15.32:0/2361997513 conn(0x7ff63806ba30 msgr2=0x7ff63806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:20.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.278+0000 7ff63d000640 1 -- 172.21.15.32:0/2361997513 shutdown_connections 2024-04-17T09:59:20.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.278+0000 7ff63d000640 1 -- 172.21.15.32:0/2361997513 wait complete. 2024-04-17T09:59:20.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.278+0000 7ff63d000640 1 Processor -- start 2024-04-17T09:59:20.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.278+0000 7ff63d000640 1 -- start start 2024-04-17T09:59:20.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff63d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6380b4de0 0x7ff63813ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:20.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff63d000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff638144e90 0x7ff63815c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:20.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff63d000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff63815c1b0 0x7ff63815a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:20.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff63d000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff638072670 con 0x7ff6380b4de0 2024-04-17T09:59:20.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff63d000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff63806f880 con 0x7ff63815c1b0 2024-04-17T09:59:20.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff63d000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff6380727f0 con 0x7ff638144e90 2024-04-17T09:59:20.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff635e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6380b4de0 0x7ff63813ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:20.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff635e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6380b4de0 0x7ff63813ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38910/0 (socket says 172.21.15.32:38910) 2024-04-17T09:59:20.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff635e00640 1 -- 172.21.15.32:0/2827555071 learned_addr learned my addr 172.21.15.32:0/2827555071 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:20.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff636800640 1 --2- 172.21.15.32:0/2827555071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff63815c1b0 0x7ff63815a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:20.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff635e00640 1 -- 172.21.15.32:0/2827555071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff638144e90 msgr2=0x7ff63815c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:20.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff62d400640 1 --2- 172.21.15.32:0/2827555071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff638144e90 0x7ff63815c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:20.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff635e00640 1 --2- 172.21.15.32:0/2827555071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff638144e90 0x7ff63815c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff635e00640 1 -- 172.21.15.32:0/2827555071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff63815c1b0 msgr2=0x7ff63815a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:20.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff635e00640 1 --2- 172.21.15.32:0/2827555071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff63815c1b0 0x7ff63815a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff635e00640 1 -- 172.21.15.32:0/2827555071 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63815cf70 con 0x7ff6380b4de0 2024-04-17T09:59:20.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff636800640 1 --2- 172.21.15.32:0/2827555071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff63815c1b0 0x7ff63815a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:59:20.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff62d400640 1 --2- 172.21.15.32:0/2827555071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff638144e90 0x7ff63815c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:59:20.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff635e00640 1 --2- 172.21.15.32:0/2827555071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6380b4de0 0x7ff63813ab70 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7ff62801af30 tx=0x7ff628019620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:20.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff62ea00640 1 -- 172.21.15.32:0/2827555071 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6280bd020 con 0x7ff6380b4de0 2024-04-17T09:59:20.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff62ea00640 1 -- 172.21.15.32:0/2827555071 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6280b8670 con 0x7ff6380b4de0 2024-04-17T09:59:20.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff62ea00640 1 -- 172.21.15.32:0/2827555071 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6280b2040 con 0x7ff6380b4de0 2024-04-17T09:59:20.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff63d000640 1 -- 172.21.15.32:0/2827555071 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff63815d250 con 0x7ff6380b4de0 2024-04-17T09:59:20.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff63d000640 1 -- 172.21.15.32:0/2827555071 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff63815d810 con 0x7ff6380b4de0 2024-04-17T09:59:20.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff63d000640 1 -- 172.21.15.32:0/2827555071 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff638068a90 con 0x7ff6380b4de0 2024-04-17T09:59:20.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.282+0000 7ff62ea00640 1 -- 172.21.15.32:0/2827555071 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff6280c0020 con 0x7ff6380b4de0 2024-04-17T09:59:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.286+0000 7ff62ea00640 1 --2- 172.21.15.32:0/2827555071 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff6140683b0 0x7ff61406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.286+0000 7ff62ea00640 1 -- 172.21.15.32:0/2827555071 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7ff628121950 con 0x7ff6380b4de0 2024-04-17T09:59:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.286+0000 7ff62d400640 1 --2- 172.21.15.32:0/2827555071 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff6140683b0 0x7ff61406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.286+0000 7ff62d400640 1 --2- 172.21.15.32:0/2827555071 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff6140683b0 0x7ff61406a870 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff6200011f0 tx=0x7ff620009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:20.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.290+0000 7ff62ea00640 1 -- 172.21.15.32:0/2827555071 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff6280c7030 con 0x7ff6380b4de0 2024-04-17T09:59:20.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.458+0000 7ff63d000640 1 -- 172.21.15.32:0/2827555071 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff63806f2e0 con 0x7ff6380b4de0 2024-04-17T09:59:20.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.458+0000 7ff62ea00640 1 -- 172.21.15.32:0/2827555071 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16672 (secure 0 0 0) 0x7ff6280164a0 con 0x7ff6380b4de0 2024-04-17T09:59:20.456 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:59:20.456 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":82,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:59:06.422274+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:59:03.223426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-17T09:59:03.780120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T09:59:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.462+0000 7ff63d000640 1 -- 172.21.15.32:0/2827555071 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff6140683b0 msgr2=0x7ff61406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.462+0000 7ff63d000640 1 --2- 172.21.15.32:0/2827555071 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff6140683b0 0x7ff61406a870 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff6200011f0 tx=0x7ff620009040 comp rx=0 tx=0).stop 2024-04-17T09:59:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.462+0000 7ff63d000640 1 -- 172.21.15.32:0/2827555071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6380b4de0 msgr2=0x7ff63813ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.462+0000 7ff63d000640 1 --2- 172.21.15.32:0/2827555071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6380b4de0 0x7ff63813ab70 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7ff62801af30 tx=0x7ff628019620 comp rx=0 tx=0).stop 2024-04-17T09:59:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.462+0000 7ff63d000640 1 -- 172.21.15.32:0/2827555071 shutdown_connections 2024-04-17T09:59:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.462+0000 7ff63d000640 1 --2- 172.21.15.32:0/2827555071 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff6140683b0 0x7ff61406a870 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.462+0000 7ff63d000640 1 --2- 172.21.15.32:0/2827555071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff638144e90 0x7ff63815c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.462+0000 7ff63d000640 1 --2- 172.21.15.32:0/2827555071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff63815c1b0 0x7ff63815a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.462+0000 7ff63d000640 1 --2- 172.21.15.32:0/2827555071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6380b4de0 0x7ff63813ab70 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.462+0000 7ff63d000640 1 -- 172.21.15.32:0/2827555071 >> 172.21.15.32:0/2827555071 conn(0x7ff63806ba30 msgr2=0x7ff6380ae200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.462+0000 7ff63d000640 1 -- 172.21.15.32:0/2827555071 shutdown_connections 2024-04-17T09:59:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.462+0000 7ff63d000640 1 -- 172.21.15.32:0/2827555071 wait complete. 2024-04-17T09:59:20.477 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:59:03.223426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:59:03.780120+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T09:59:20.478 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T09:59:20.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.622+0000 7f8e34e00640 1 Processor -- start 2024-04-17T09:59:20.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.622+0000 7f8e34e00640 1 -- start start 2024-04-17T09:59:20.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.626+0000 7f8e34e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e3006f670 0x7f8e300cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:20.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.626+0000 7f8e34e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8e300cfc20 0x7f8e300b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:20.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.626+0000 7f8e34e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e30071e90 con 0x7f8e30144e90 2024-04-17T09:59:20.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.626+0000 7f8e34e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f8e30145270 con 0x7f8e300cfc20 2024-04-17T09:59:20.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.626+0000 7f8e34e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8e30072170 con 0x7f8e3006f670 2024-04-17T09:59:20.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.626+0000 7f8e2fe00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8e300cfc20 0x7f8e300b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:20.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.626+0000 7f8e27e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e3006f670 0x7f8e300cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:20.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.626+0000 7f8e2f400640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f8e30144e90 0x7f8e3006ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:48560/0 (socket says 172.21.15.32:48560) 2024-04-17T09:59:20.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.626+0000 7f8e2f400640 1 -- 172.21.15.32:0/527215480 learned_addr learned my addr 172.21.15.32:0/527215480 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:20.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.626+0000 7f8e2fe00640 1 -- 172.21.15.32:0/527215480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e3006f670 msgr2=0x7f8e300cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:20.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.626+0000 7f8e2fe00640 1 --2- 172.21.15.32:0/527215480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e3006f670 0x7f8e300cf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.626+0000 7f8e2fe00640 1 -- 172.21.15.32:0/527215480 >> v1:172.21.15.144:6789/0 conn(0x7f8e30144e90 legacy=0x7f8e3006ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:20.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.626+0000 7f8e2fe00640 1 -- 172.21.15.32:0/527215480 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e30072450 con 0x7f8e300cfc20 2024-04-17T09:59:20.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.626+0000 7f8e27e00640 1 --2- 172.21.15.32:0/527215480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e3006f670 0x7f8e300cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:59:20.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.626+0000 7f8e2ea00640 1 -- 172.21.15.32:0/527215480 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1953292007 0 0) 0x7f8e30071e90 con 0x7f8e30144e90 2024-04-17T09:59:20.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.630+0000 7f8e2fe00640 1 --2- 172.21.15.32:0/527215480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e300cfc20 0x7f8e300b3d40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009d90 tx=0x7f8e2801c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e7bca04882a09e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:20.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.630+0000 7f8e2ea00640 1 -- 172.21.15.32:0/527215480 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e280b2070 con 0x7f8e300cfc20 2024-04-17T09:59:20.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.630+0000 7f8e2ea00640 1 -- 172.21.15.32:0/527215480 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e280a58e0 con 0x7f8e300cfc20 2024-04-17T09:59:20.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.630+0000 7f8e2ea00640 1 -- 172.21.15.32:0/527215480 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e280a5cb0 con 0x7f8e300cfc20 2024-04-17T09:59:20.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.630+0000 7f8e34e00640 1 -- 172.21.15.32:0/527215480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e300cfc20 msgr2=0x7f8e300b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:20.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.630+0000 7f8e34e00640 1 --2- 172.21.15.32:0/527215480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e300cfc20 0x7f8e300b3d40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009d90 tx=0x7f8e2801c820 comp rx=0 tx=0).stop 2024-04-17T09:59:20.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.630+0000 7f8e34e00640 1 -- 172.21.15.32:0/527215480 shutdown_connections 2024-04-17T09:59:20.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.630+0000 7f8e34e00640 1 --2- 172.21.15.32:0/527215480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e300cfc20 0x7f8e300b3d40 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.630+0000 7f8e34e00640 1 --2- 172.21.15.32:0/527215480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e3006f670 0x7f8e300cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.630+0000 7f8e34e00640 1 -- 172.21.15.32:0/527215480 >> 172.21.15.32:0/527215480 conn(0x7f8e30067d70 msgr2=0x7f8e3006ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:20.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.630+0000 7f8e34e00640 1 -- 172.21.15.32:0/527215480 shutdown_connections 2024-04-17T09:59:20.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.630+0000 7f8e34e00640 1 -- 172.21.15.32:0/527215480 wait complete. 2024-04-17T09:59:20.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.630+0000 7f8e34e00640 1 Processor -- start 2024-04-17T09:59:20.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.630+0000 7f8e34e00640 1 -- start start 2024-04-17T09:59:20.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e34e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e3006f670 0x7f8e30160b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:20.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e34e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e300cfc20 0x7f8e3015e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:20.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e34e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e30144e90 0x7f8e3013cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:20.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e34e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8e300b4a90 con 0x7f8e30144e90 2024-04-17T09:59:20.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e34e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8e300d0240 con 0x7f8e3006f670 2024-04-17T09:59:20.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e34e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8e300b4c10 con 0x7f8e300cfc20 2024-04-17T09:59:20.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e2fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e30144e90 0x7f8e3013cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:20.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e2fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e30144e90 0x7f8e3013cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40060/0 (socket says 172.21.15.32:40060) 2024-04-17T09:59:20.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e2fe00640 1 -- 172.21.15.32:0/1881203781 learned_addr learned my addr 172.21.15.32:0/1881203781 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:20.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e27e00640 1 --2- 172.21.15.32:0/1881203781 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e300cfc20 0x7f8e3015e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:20.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e2f400640 1 --2- 172.21.15.32:0/1881203781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e3006f670 0x7f8e30160b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:20.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e2fe00640 1 -- 172.21.15.32:0/1881203781 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e300cfc20 msgr2=0x7f8e3015e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:20.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e2fe00640 1 --2- 172.21.15.32:0/1881203781 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e300cfc20 0x7f8e3015e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e2fe00640 1 -- 172.21.15.32:0/1881203781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e3006f670 msgr2=0x7f8e30160b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:20.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e2fe00640 1 --2- 172.21.15.32:0/1881203781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e3006f670 0x7f8e30160b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e2fe00640 1 -- 172.21.15.32:0/1881203781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3015ee90 con 0x7f8e30144e90 2024-04-17T09:59:20.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e2f400640 1 --2- 172.21.15.32:0/1881203781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e3006f670 0x7f8e30160b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:20.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e27e00640 1 --2- 172.21.15.32:0/1881203781 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e300cfc20 0x7f8e3015e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:20.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e2fe00640 1 --2- 172.21.15.32:0/1881203781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e30144e90 0x7f8e3013cea0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f8e2801aed0 tx=0x7f8e280195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:20.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e27400640 1 -- 172.21.15.32:0/1881203781 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e280bd020 con 0x7f8e30144e90 2024-04-17T09:59:20.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e27400640 1 -- 172.21.15.32:0/1881203781 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e280b8670 con 0x7f8e30144e90 2024-04-17T09:59:20.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e27400640 1 -- 172.21.15.32:0/1881203781 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e280b2040 con 0x7f8e30144e90 2024-04-17T09:59:20.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e34e00640 1 -- 172.21.15.32:0/1881203781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e3015f130 con 0x7f8e30144e90 2024-04-17T09:59:20.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e34e00640 1 -- 172.21.15.32:0/1881203781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e30160350 con 0x7f8e30144e90 2024-04-17T09:59:20.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.634+0000 7f8e34e00640 1 -- 172.21.15.32:0/1881203781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e3015f650 con 0x7f8e30144e90 2024-04-17T09:59:20.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.638+0000 7f8e27400640 1 -- 172.21.15.32:0/1881203781 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8e280c0020 con 0x7f8e30144e90 2024-04-17T09:59:20.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.638+0000 7f8e27400640 1 --2- 172.21.15.32:0/1881203781 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8e0c068380 0x7f8e0c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:20.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.638+0000 7f8e27400640 1 -- 172.21.15.32:0/1881203781 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f8e28121a50 con 0x7f8e30144e90 2024-04-17T09:59:20.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.638+0000 7f8e2f400640 1 --2- 172.21.15.32:0/1881203781 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8e0c068380 0x7f8e0c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:20.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.638+0000 7f8e2f400640 1 --2- 172.21.15.32:0/1881203781 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8e0c068380 0x7f8e0c06a840 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8e20000f70 tx=0x7f8e20007cd0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:20.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.642+0000 7f8e27400640 1 -- 172.21.15.32:0/1881203781 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f8e3015f650 con 0x7f8e30144e90 2024-04-17T09:59:20.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.802+0000 7f8e34e00640 1 -- 172.21.15.32:0/1881203781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8e3004fda0 con 0x7f8e30144e90 2024-04-17T09:59:20.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.806+0000 7f8e27400640 1 -- 172.21.15.32:0/1881203781 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f8e280c7070 con 0x7f8e30144e90 2024-04-17T09:59:20.802 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T09:59:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.806+0000 7f8e34e00640 1 -- 172.21.15.32:0/1881203781 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8e0c068380 msgr2=0x7f8e0c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.806+0000 7f8e34e00640 1 --2- 172.21.15.32:0/1881203781 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8e0c068380 0x7f8e0c06a840 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8e20000f70 tx=0x7f8e20007cd0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.806+0000 7f8e34e00640 1 -- 172.21.15.32:0/1881203781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e30144e90 msgr2=0x7f8e3013cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.806+0000 7f8e34e00640 1 --2- 172.21.15.32:0/1881203781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e30144e90 0x7f8e3013cea0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f8e2801aed0 tx=0x7f8e280195c0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.806+0000 7f8e34e00640 1 -- 172.21.15.32:0/1881203781 shutdown_connections 2024-04-17T09:59:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.806+0000 7f8e34e00640 1 --2- 172.21.15.32:0/1881203781 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8e0c068380 0x7f8e0c06a840 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.806+0000 7f8e34e00640 1 --2- 172.21.15.32:0/1881203781 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e300cfc20 0x7f8e3015e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.806+0000 7f8e34e00640 1 --2- 172.21.15.32:0/1881203781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e30144e90 0x7f8e3013cea0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.806+0000 7f8e34e00640 1 --2- 172.21.15.32:0/1881203781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e3006f670 0x7f8e30160b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.806+0000 7f8e34e00640 1 -- 172.21.15.32:0/1881203781 >> 172.21.15.32:0/1881203781 conn(0x7f8e30067d70 msgr2=0x7f8e300cce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.806+0000 7f8e34e00640 1 -- 172.21.15.32:0/1881203781 shutdown_connections 2024-04-17T09:59:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.806+0000 7f8e34e00640 1 -- 172.21.15.32:0/1881203781 wait complete. 2024-04-17T09:59:20.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T09:59:20.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.970+0000 7f1f1e400640 1 Processor -- start 2024-04-17T09:59:20.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.970+0000 7f1f1e400640 1 -- start start 2024-04-17T09:59:20.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.970+0000 7f1f1e400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f180ce740 0x7f1f1809a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:20.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.970+0000 7f1f1e400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f1815cc40 con 0x7f1f1815c860 2024-04-17T09:59:20.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.970+0000 7f1f1e400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f180cd6a0 con 0x7f1f180ceba0 2024-04-17T09:59:20.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.970+0000 7f1f1e400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1f180ce2b0 con 0x7f1f180ce740 2024-04-17T09:59:20.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.970+0000 7f1f0ea00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f180ce740 0x7f1f1809a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:20.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.970+0000 7f1f17400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1f1815c860 0x7f1f180d1050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52426/0 (socket says 172.21.15.32:52426) 2024-04-17T09:59:20.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.970+0000 7f1f17400640 1 -- 172.21.15.32:0/1643290404 learned_addr learned my addr 172.21.15.32:0/1643290404 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:20.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.970+0000 7f1f0ea00640 1 -- 172.21.15.32:0/1643290404 >> v1:172.21.15.144:6789/0 conn(0x7f1f180ceba0 legacy=0x7f1f180b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:20.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.970+0000 7f1f0ea00640 1 -- 172.21.15.32:0/1643290404 >> v1:172.21.15.32:6789/0 conn(0x7f1f1815c860 legacy=0x7f1f180d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:20.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.970+0000 7f1f0ea00640 1 -- 172.21.15.32:0/1643290404 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f180cd980 con 0x7f1f180ce740 2024-04-17T09:59:20.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.974+0000 7f1f0ea00640 1 --2- 172.21.15.32:0/1643290404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f180ce740 0x7f1f1809a9f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f1f000097c0 tx=0x7f1f0001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfbf1e0c79a28b1c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:20.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.974+0000 7f1f16a00640 1 -- 172.21.15.32:0/1643290404 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f000b0070 con 0x7f1f180ce740 2024-04-17T09:59:20.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.974+0000 7f1f16a00640 1 -- 172.21.15.32:0/1643290404 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f000ab600 con 0x7f1f180ce740 2024-04-17T09:59:20.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.974+0000 7f1f16a00640 1 -- 172.21.15.32:0/1643290404 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f000ab9d0 con 0x7f1f180ce740 2024-04-17T09:59:20.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.974+0000 7f1f1e400640 1 -- 172.21.15.32:0/1643290404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f180ce740 msgr2=0x7f1f1809a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:20.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.974+0000 7f1f1e400640 1 --2- 172.21.15.32:0/1643290404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f180ce740 0x7f1f1809a9f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f1f000097c0 tx=0x7f1f0001c4a0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.974+0000 7f1f1e400640 1 -- 172.21.15.32:0/1643290404 shutdown_connections 2024-04-17T09:59:20.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.974+0000 7f1f1e400640 1 --2- 172.21.15.32:0/1643290404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f180ce740 0x7f1f1809a9f0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.974+0000 7f1f1e400640 1 -- 172.21.15.32:0/1643290404 >> 172.21.15.32:0/1643290404 conn(0x7f1f180b4050 msgr2=0x7f1f180b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:20.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.974+0000 7f1f1e400640 1 -- 172.21.15.32:0/1643290404 shutdown_connections 2024-04-17T09:59:20.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.974+0000 7f1f1e400640 1 -- 172.21.15.32:0/1643290404 wait complete. 2024-04-17T09:59:20.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f1e400640 1 Processor -- start 2024-04-17T09:59:20.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f1e400640 1 -- start start 2024-04-17T09:59:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f1e400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f180ce740 0x7f1f1815f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f1e400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f180ceba0 0x7f1f18158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f1e400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f1815c860 0x7f1f181509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f1e400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1f180cd350 con 0x7f1f1815c860 2024-04-17T09:59:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f1e400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1809b340 con 0x7f1f180ce740 2024-04-17T09:59:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f1e400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1f1816af40 con 0x7f1f180ceba0 2024-04-17T09:59:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f17e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f1815c860 0x7f1f181509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:20.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f17e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f1815c860 0x7f1f181509d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40074/0 (socket says 172.21.15.32:40074) 2024-04-17T09:59:20.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f17e00640 1 -- 172.21.15.32:0/3427280412 learned_addr learned my addr 172.21.15.32:0/3427280412 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:20.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f17400640 1 --2- 172.21.15.32:0/3427280412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f180ce740 0x7f1f1815f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:20.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f0ea00640 1 --2- 172.21.15.32:0/3427280412 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f180ceba0 0x7f1f18158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:20.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f17e00640 1 -- 172.21.15.32:0/3427280412 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f180ceba0 msgr2=0x7f1f18158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:20.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f17e00640 1 --2- 172.21.15.32:0/3427280412 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f180ceba0 0x7f1f18158a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f17e00640 1 -- 172.21.15.32:0/3427280412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f180ce740 msgr2=0x7f1f1815f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:20.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f17e00640 1 --2- 172.21.15.32:0/3427280412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f180ce740 0x7f1f1815f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:20.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f17e00640 1 -- 172.21.15.32:0/3427280412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f18158f90 con 0x7f1f1815c860 2024-04-17T09:59:20.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f17400640 1 --2- 172.21.15.32:0/3427280412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f180ce740 0x7f1f1815f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:20.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f0ea00640 1 --2- 172.21.15.32:0/3427280412 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f180ceba0 0x7f1f18158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:20.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f17e00640 1 --2- 172.21.15.32:0/3427280412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f1815c860 0x7f1f181509d0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000c760 tx=0x7f1f1000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:20.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f0fe00640 1 -- 172.21.15.32:0/3427280412 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10003b80 con 0x7f1f1815c860 2024-04-17T09:59:20.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f0fe00640 1 -- 172.21.15.32:0/3427280412 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f10003d20 con 0x7f1f1815c860 2024-04-17T09:59:20.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f0fe00640 1 -- 172.21.15.32:0/3427280412 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1000aa90 con 0x7f1f1815c860 2024-04-17T09:59:20.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f1e400640 1 -- 172.21.15.32:0/3427280412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f1815fb70 con 0x7f1f1815c860 2024-04-17T09:59:20.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f1e400640 1 -- 172.21.15.32:0/3427280412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f1815b980 con 0x7f1f1815c860 2024-04-17T09:59:20.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.978+0000 7f1f1e400640 1 -- 172.21.15.32:0/3427280412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f18068a90 con 0x7f1f1815c860 2024-04-17T09:59:20.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.982+0000 7f1f0fe00640 1 -- 172.21.15.32:0/3427280412 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1f1000ac30 con 0x7f1f1815c860 2024-04-17T09:59:20.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.982+0000 7f1f0fe00640 1 --2- 172.21.15.32:0/3427280412 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1ef4068380 0x7f1ef406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:20.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.982+0000 7f1f0fe00640 1 -- 172.21.15.32:0/3427280412 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f1f10086860 con 0x7f1f1815c860 2024-04-17T09:59:20.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.982+0000 7f1f17400640 1 --2- 172.21.15.32:0/3427280412 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1ef4068380 0x7f1ef406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:20.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.982+0000 7f1f17400640 1 --2- 172.21.15.32:0/3427280412 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1ef4068380 0x7f1ef406a840 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1f08005890 tx=0x7f1f08009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:20.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:20.986+0000 7f1f0fe00640 1 -- 172.21.15.32:0/3427280412 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f1f18158f90 con 0x7f1f1815c860 2024-04-17T09:59:21.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.150+0000 7f1f1e400640 1 -- 172.21.15.32:0/3427280412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1f18154ff0 con 0x7f1f1815c860 2024-04-17T09:59:21.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.150+0000 7f1f0fe00640 1 -- 172.21.15.32:0/3427280412 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7f1f10056d80 con 0x7f1f1815c860 2024-04-17T09:59:21.147 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T09:59:21.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.150+0000 7f1f1e400640 1 -- 172.21.15.32:0/3427280412 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1ef4068380 msgr2=0x7f1ef406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:21.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.150+0000 7f1f1e400640 1 --2- 172.21.15.32:0/3427280412 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1ef4068380 0x7f1ef406a840 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1f08005890 tx=0x7f1f08009040 comp rx=0 tx=0).stop 2024-04-17T09:59:21.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.150+0000 7f1f1e400640 1 -- 172.21.15.32:0/3427280412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f1815c860 msgr2=0x7f1f181509d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:21.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.150+0000 7f1f1e400640 1 --2- 172.21.15.32:0/3427280412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f1815c860 0x7f1f181509d0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000c760 tx=0x7f1f1000cc20 comp rx=0 tx=0).stop 2024-04-17T09:59:21.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.150+0000 7f1f1e400640 1 -- 172.21.15.32:0/3427280412 shutdown_connections 2024-04-17T09:59:21.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.150+0000 7f1f1e400640 1 --2- 172.21.15.32:0/3427280412 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1ef4068380 0x7f1ef406a840 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:21.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.150+0000 7f1f1e400640 1 --2- 172.21.15.32:0/3427280412 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f180ceba0 0x7f1f18158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:21.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.150+0000 7f1f1e400640 1 --2- 172.21.15.32:0/3427280412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f1815c860 0x7f1f181509d0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:21.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.150+0000 7f1f1e400640 1 --2- 172.21.15.32:0/3427280412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f180ce740 0x7f1f1815f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:21.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.150+0000 7f1f1e400640 1 -- 172.21.15.32:0/3427280412 >> 172.21.15.32:0/3427280412 conn(0x7f1f180b4050 msgr2=0x7f1f180d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:21.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.150+0000 7f1f1e400640 1 -- 172.21.15.32:0/3427280412 shutdown_connections 2024-04-17T09:59:21.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.150+0000 7f1f1e400640 1 -- 172.21.15.32:0/3427280412 wait complete. 2024-04-17T09:59:21.170 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T09:59:21.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.314+0000 7f32b7e00640 1 Processor -- start 2024-04-17T09:59:21.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.314+0000 7f32b7e00640 1 -- start start 2024-04-17T09:59:21.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.318+0000 7f32b7e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32b806dc60 0x7f32b8169480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:21.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.318+0000 7f32b7e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32b80ce920 con 0x7f32b806d080 2024-04-17T09:59:21.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.318+0000 7f32b7e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32b80b52b0 con 0x7f32b815e7f0 2024-04-17T09:59:21.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.318+0000 7f32b7e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f32b80d1880 con 0x7f32b806dc60 2024-04-17T09:59:21.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.318+0000 7f32ae000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f32b806d080 0x7f32b806d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52442/0 (socket says 172.21.15.32:52442) 2024-04-17T09:59:21.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.318+0000 7f32ae000640 1 -- 172.21.15.32:0/2968627882 learned_addr learned my addr 172.21.15.32:0/2968627882 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:21.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.318+0000 7f32b7400640 1 --2- 172.21.15.32:0/2968627882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32b806dc60 0x7f32b8169480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:21.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.318+0000 7f32b6000640 1 -- 172.21.15.32:0/2968627882 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1771454211 0 0) 0x7f32b80ce920 con 0x7f32b806d080 2024-04-17T09:59:21.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.318+0000 7f32b6000640 1 -- 172.21.15.32:0/2968627882 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f329c003530 con 0x7f32b806d080 2024-04-17T09:59:21.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.318+0000 7f32b6000640 1 -- 172.21.15.32:0/2968627882 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2531640002 0 0) 0x7f32b80b52b0 con 0x7f32b815e7f0 2024-04-17T09:59:21.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.318+0000 7f32b6000640 1 -- 172.21.15.32:0/2968627882 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32b80ce920 con 0x7f32b815e7f0 2024-04-17T09:59:21.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.318+0000 7f32b7400640 1 -- 172.21.15.32:0/2968627882 >> v1:172.21.15.144:6789/0 conn(0x7f32b815e7f0 legacy=0x7f32b80d1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:21.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.318+0000 7f32b7400640 1 -- 172.21.15.32:0/2968627882 >> v1:172.21.15.32:6789/0 conn(0x7f32b806d080 legacy=0x7f32b806d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:21.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.318+0000 7f32b7400640 1 -- 172.21.15.32:0/2968627882 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b80d1aa0 con 0x7f32b806dc60 2024-04-17T09:59:21.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.322+0000 7f32b7400640 1 --2- 172.21.15.32:0/2968627882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32b806dc60 0x7f32b8169480 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009dc0 tx=0x7f32b001c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d001bf53856c6e7d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:21.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.322+0000 7f32b6000640 1 -- 172.21.15.32:0/2968627882 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b00b2070 con 0x7f32b806dc60 2024-04-17T09:59:21.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.322+0000 7f32b6000640 1 -- 172.21.15.32:0/2968627882 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32b00a5990 con 0x7f32b806dc60 2024-04-17T09:59:21.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.322+0000 7f32b6000640 1 -- 172.21.15.32:0/2968627882 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b00a5d60 con 0x7f32b806dc60 2024-04-17T09:59:21.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.322+0000 7f32b7e00640 1 -- 172.21.15.32:0/2968627882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32b806dc60 msgr2=0x7f32b8169480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:21.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.322+0000 7f32b7e00640 1 --2- 172.21.15.32:0/2968627882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32b806dc60 0x7f32b8169480 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009dc0 tx=0x7f32b001c8d0 comp rx=0 tx=0).stop 2024-04-17T09:59:21.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.322+0000 7f32b7e00640 1 -- 172.21.15.32:0/2968627882 shutdown_connections 2024-04-17T09:59:21.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.322+0000 7f32b7e00640 1 --2- 172.21.15.32:0/2968627882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32b806dc60 0x7f32b8169480 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:21.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.322+0000 7f32b7e00640 1 -- 172.21.15.32:0/2968627882 >> 172.21.15.32:0/2968627882 conn(0x7f32b80b3f50 msgr2=0x7f32b80b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:21.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.322+0000 7f32b7e00640 1 -- 172.21.15.32:0/2968627882 shutdown_connections 2024-04-17T09:59:21.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.322+0000 7f32b7e00640 1 -- 172.21.15.32:0/2968627882 wait complete. 2024-04-17T09:59:21.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.322+0000 7f32b7e00640 1 Processor -- start 2024-04-17T09:59:21.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.322+0000 7f32b7e00640 1 -- start start 2024-04-17T09:59:21.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.322+0000 7f32b7e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32b806d080 0x7f32b815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:21.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.322+0000 7f32b7e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32b806dc60 0x7f32b8150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:21.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.322+0000 7f32b7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32b815e7f0 0x7f32b8156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:21.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.322+0000 7f32b7e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f32b815ebd0 con 0x7f32b815e7f0 2024-04-17T09:59:21.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.322+0000 7f32b7e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f32b80cd3f0 con 0x7f32b806d080 2024-04-17T09:59:21.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.322+0000 7f32b7e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f32b815ed50 con 0x7f32b806dc60 2024-04-17T09:59:21.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.326+0000 7f32b7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32b815e7f0 0x7f32b8156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:21.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.326+0000 7f32b7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32b815e7f0 0x7f32b8156a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40076/0 (socket says 172.21.15.32:40076) 2024-04-17T09:59:21.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.326+0000 7f32b7400640 1 -- 172.21.15.32:0/2075611294 learned_addr learned my addr 172.21.15.32:0/2075611294 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:21.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.326+0000 7f32ae000640 1 --2- 172.21.15.32:0/2075611294 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32b806dc60 0x7f32b8150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:21.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.326+0000 7f32b6a00640 1 --2- 172.21.15.32:0/2075611294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32b806d080 0x7f32b815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:21.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.326+0000 7f32b7400640 1 -- 172.21.15.32:0/2075611294 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32b806dc60 msgr2=0x7f32b8150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:21.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.326+0000 7f32b7400640 1 --2- 172.21.15.32:0/2075611294 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32b806dc60 0x7f32b8150e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:21.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.326+0000 7f32b7400640 1 -- 172.21.15.32:0/2075611294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32b806d080 msgr2=0x7f32b815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:21.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.326+0000 7f32b7400640 1 --2- 172.21.15.32:0/2075611294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32b806d080 0x7f32b815aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:21.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.326+0000 7f32b7400640 1 -- 172.21.15.32:0/2075611294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b81599e0 con 0x7f32b815e7f0 2024-04-17T09:59:21.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.326+0000 7f32ae000640 1 --2- 172.21.15.32:0/2075611294 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32b806dc60 0x7f32b8150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:21.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.326+0000 7f32b6a00640 1 --2- 172.21.15.32:0/2075611294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32b806d080 0x7f32b815aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:21.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.326+0000 7f32b7400640 1 --2- 172.21.15.32:0/2075611294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32b815e7f0 0x7f32b8156a20 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f32b001a500 tx=0x7f32b0019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:21.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.326+0000 7f32af400640 1 -- 172.21.15.32:0/2075611294 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b00bd020 con 0x7f32b815e7f0 2024-04-17T09:59:21.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.326+0000 7f32af400640 1 -- 172.21.15.32:0/2075611294 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32b00b8470 con 0x7f32b815e7f0 2024-04-17T09:59:21.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.326+0000 7f32b7e00640 1 -- 172.21.15.32:0/2075611294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b8159cc0 con 0x7f32b815e7f0 2024-04-17T09:59:21.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.326+0000 7f32af400640 1 -- 172.21.15.32:0/2075611294 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b00b2040 con 0x7f32b815e7f0 2024-04-17T09:59:21.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.326+0000 7f32b7e00640 1 -- 172.21.15.32:0/2075611294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b815a280 con 0x7f32b815e7f0 2024-04-17T09:59:21.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.326+0000 7f32b7e00640 1 -- 172.21.15.32:0/2075611294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b804fda0 con 0x7f32b815e7f0 2024-04-17T09:59:21.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.326+0000 7f32af400640 1 -- 172.21.15.32:0/2075611294 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f32b00c0020 con 0x7f32b815e7f0 2024-04-17T09:59:21.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.326+0000 7f32af400640 1 --2- 172.21.15.32:0/2075611294 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3294068380 0x7f329406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:21.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.326+0000 7f32af400640 1 -- 172.21.15.32:0/2075611294 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f32b0121890 con 0x7f32b815e7f0 2024-04-17T09:59:21.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.330+0000 7f32b6a00640 1 --2- 172.21.15.32:0/2075611294 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3294068380 0x7f329406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:21.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.330+0000 7f32b6a00640 1 --2- 172.21.15.32:0/2075611294 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3294068380 0x7f329406a840 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f32a80011f0 tx=0x7f32a8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:21.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.330+0000 7f32af400640 1 -- 172.21.15.32:0/2075611294 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f32b00c7030 con 0x7f32b815e7f0 2024-04-17T09:59:21.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.502+0000 7f32b7e00640 1 -- 172.21.15.32:0/2075611294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f32b806ef50 con 0x7f32b815e7f0 2024-04-17T09:59:21.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.502+0000 7f32af400640 1 -- 172.21.15.32:0/2075611294 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7f32b0016580 con 0x7f32b815e7f0 2024-04-17T09:59:21.498 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T09:59:21.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.502+0000 7f32b7e00640 1 -- 172.21.15.32:0/2075611294 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3294068380 msgr2=0x7f329406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:21.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.502+0000 7f32b7e00640 1 --2- 172.21.15.32:0/2075611294 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3294068380 0x7f329406a840 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f32a80011f0 tx=0x7f32a8009040 comp rx=0 tx=0).stop 2024-04-17T09:59:21.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.502+0000 7f32b7e00640 1 -- 172.21.15.32:0/2075611294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32b815e7f0 msgr2=0x7f32b8156a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:21.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.502+0000 7f32b7e00640 1 --2- 172.21.15.32:0/2075611294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32b815e7f0 0x7f32b8156a20 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f32b001a500 tx=0x7f32b0019670 comp rx=0 tx=0).stop 2024-04-17T09:59:21.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.502+0000 7f32b7e00640 1 -- 172.21.15.32:0/2075611294 shutdown_connections 2024-04-17T09:59:21.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.502+0000 7f32b7e00640 1 --2- 172.21.15.32:0/2075611294 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3294068380 0x7f329406a840 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:21.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.502+0000 7f32b7e00640 1 --2- 172.21.15.32:0/2075611294 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32b806dc60 0x7f32b8150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:21.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.502+0000 7f32b7e00640 1 --2- 172.21.15.32:0/2075611294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32b815e7f0 0x7f32b8156a20 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:21.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.502+0000 7f32b7e00640 1 --2- 172.21.15.32:0/2075611294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32b806d080 0x7f32b815aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:21.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.502+0000 7f32b7e00640 1 -- 172.21.15.32:0/2075611294 >> 172.21.15.32:0/2075611294 conn(0x7f32b80b3f50 msgr2=0x7f32b8161860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:21.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.502+0000 7f32b7e00640 1 -- 172.21.15.32:0/2075611294 shutdown_connections 2024-04-17T09:59:21.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.502+0000 7f32b7e00640 1 -- 172.21.15.32:0/2075611294 wait complete. 2024-04-17T09:59:21.520 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T09:59:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.662+0000 7fee6a400640 1 Processor -- start 2024-04-17T09:59:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.662+0000 7fee6a400640 1 -- start start 2024-04-17T09:59:21.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.666+0000 7fee6a400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee6415c290 0x7fee640d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.666+0000 7fee6a400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee640d00f0 0x7fee640af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.666+0000 7fee6a400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee64071eb0 con 0x7fee6415c670 2024-04-17T09:59:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.666+0000 7fee6a400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fee64072190 con 0x7fee640d00f0 2024-04-17T09:59:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.666+0000 7fee6a400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fee64072470 con 0x7fee6415c290 2024-04-17T09:59:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.666+0000 7fee5aa00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee640d00f0 0x7fee640af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.666+0000 7fee5aa00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee640d00f0 0x7fee640af360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40090/0 (socket says 172.21.15.32:40090) 2024-04-17T09:59:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.666+0000 7fee5aa00640 1 -- 172.21.15.32:0/906045537 learned_addr learned my addr 172.21.15.32:0/906045537 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.666+0000 7fee63400640 1 --2- 172.21.15.32:0/906045537 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee6415c290 0x7fee640d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.666+0000 7fee5aa00640 1 -- 172.21.15.32:0/906045537 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee6415c290 msgr2=0x7fee640d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.666+0000 7fee5aa00640 1 --2- 172.21.15.32:0/906045537 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee6415c290 0x7fee640d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.666+0000 7fee5aa00640 1 -- 172.21.15.32:0/906045537 >> v1:172.21.15.144:6789/0 conn(0x7fee6415c670 legacy=0x7fee64162450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:21.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.666+0000 7fee5aa00640 1 -- 172.21.15.32:0/906045537 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee64072750 con 0x7fee640d00f0 2024-04-17T09:59:21.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.666+0000 7fee5aa00640 1 --2- 172.21.15.32:0/906045537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640d00f0 0x7fee640af360 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009870 tx=0x7fee4c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef7e4664d89e124 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee62a00640 1 -- 172.21.15.32:0/906045537 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c0b0070 con 0x7fee640d00f0 2024-04-17T09:59:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee62a00640 1 -- 172.21.15.32:0/906045537 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee4c0ab600 con 0x7fee640d00f0 2024-04-17T09:59:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee62a00640 1 -- 172.21.15.32:0/906045537 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c0ab9d0 con 0x7fee640d00f0 2024-04-17T09:59:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee6a400640 1 -- 172.21.15.32:0/906045537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640d00f0 msgr2=0x7fee640af360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee6a400640 1 --2- 172.21.15.32:0/906045537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640d00f0 0x7fee640af360 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009870 tx=0x7fee4c01c440 comp rx=0 tx=0).stop 2024-04-17T09:59:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee6a400640 1 -- 172.21.15.32:0/906045537 shutdown_connections 2024-04-17T09:59:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee6a400640 1 --2- 172.21.15.32:0/906045537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640d00f0 0x7fee640af360 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee6a400640 1 --2- 172.21.15.32:0/906045537 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee6415c290 0x7fee640d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee6a400640 1 -- 172.21.15.32:0/906045537 >> 172.21.15.32:0/906045537 conn(0x7fee6406ba30 msgr2=0x7fee6406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee6a400640 1 -- 172.21.15.32:0/906045537 shutdown_connections 2024-04-17T09:59:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee6a400640 1 -- 172.21.15.32:0/906045537 wait complete. 2024-04-17T09:59:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee6a400640 1 Processor -- start 2024-04-17T09:59:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee6a400640 1 -- start start 2024-04-17T09:59:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee6a400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640d00f0 0x7fee64148970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee6a400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee6415c290 0x7fee6414c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee6a400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee6415c670 0x7fee64155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee6a400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fee6416b970 con 0x7fee640d00f0 2024-04-17T09:59:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee6a400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fee6416b7f0 con 0x7fee6415c290 2024-04-17T09:59:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee6a400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fee6416baf0 con 0x7fee6415c670 2024-04-17T09:59:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee63400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640d00f0 0x7fee64148970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee63400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640d00f0 0x7fee64148970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40102/0 (socket says 172.21.15.32:40102) 2024-04-17T09:59:21.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee63400640 1 -- 172.21.15.32:0/3524967566 learned_addr learned my addr 172.21.15.32:0/3524967566 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee5aa00640 1 --2- 172.21.15.32:0/3524967566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee6415c290 0x7fee6414c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.670+0000 7fee63e00640 1 --2- 172.21.15.32:0/3524967566 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee6415c670 0x7fee64155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.674+0000 7fee63400640 1 -- 172.21.15.32:0/3524967566 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee6415c670 msgr2=0x7fee64155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.674+0000 7fee63400640 1 --2- 172.21.15.32:0/3524967566 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee6415c670 0x7fee64155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.674+0000 7fee63400640 1 -- 172.21.15.32:0/3524967566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee6415c290 msgr2=0x7fee6414c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.674+0000 7fee63400640 1 --2- 172.21.15.32:0/3524967566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee6415c290 0x7fee6414c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.674+0000 7fee63400640 1 -- 172.21.15.32:0/3524967566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6414d5a0 con 0x7fee640d00f0 2024-04-17T09:59:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.674+0000 7fee63400640 1 --2- 172.21.15.32:0/3524967566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640d00f0 0x7fee64148970 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fee5400d620 tx=0x7fee5400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.674+0000 7fee5be00640 1 -- 172.21.15.32:0/3524967566 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5400b620 con 0x7fee640d00f0 2024-04-17T09:59:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.674+0000 7fee5be00640 1 -- 172.21.15.32:0/3524967566 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee54019070 con 0x7fee640d00f0 2024-04-17T09:59:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.674+0000 7fee6a400640 1 -- 172.21.15.32:0/3524967566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6414cd00 con 0x7fee640d00f0 2024-04-17T09:59:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.674+0000 7fee5be00640 1 -- 172.21.15.32:0/3524967566 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5400ba20 con 0x7fee640d00f0 2024-04-17T09:59:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.674+0000 7fee6a400640 1 -- 172.21.15.32:0/3524967566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee641f9280 con 0x7fee640d00f0 2024-04-17T09:59:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.674+0000 7fee6a400640 1 -- 172.21.15.32:0/3524967566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee6414d0f0 con 0x7fee640d00f0 2024-04-17T09:59:21.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.674+0000 7fee5be00640 1 -- 172.21.15.32:0/3524967566 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fee5400bbc0 con 0x7fee640d00f0 2024-04-17T09:59:21.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.674+0000 7fee5be00640 1 --2- 172.21.15.32:0/3524967566 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee40068380 0x7fee4006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:21.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.674+0000 7fee5be00640 1 -- 172.21.15.32:0/3524967566 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fee54086db0 con 0x7fee640d00f0 2024-04-17T09:59:21.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.674+0000 7fee5aa00640 1 --2- 172.21.15.32:0/3524967566 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee40068380 0x7fee4006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:21.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.678+0000 7fee5aa00640 1 --2- 172.21.15.32:0/3524967566 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee40068380 0x7fee4006a840 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0ad000 tx=0x7fee4c0a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:21.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.678+0000 7fee5be00640 1 -- 172.21.15.32:0/3524967566 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fee5408b070 con 0x7fee640d00f0 2024-04-17T09:59:21.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.854+0000 7fee6a400640 1 -- 172.21.15.32:0/3524967566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fee64068a90 con 0x7fee640d00f0 2024-04-17T09:59:21.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.854+0000 7fee5be00640 1 -- 172.21.15.32:0/3524967566 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v82) ==== 80+0+0 (secure 0 0 0) 0x7fee54057360 con 0x7fee640d00f0 2024-04-17T09:59:21.854 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T09:59:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.858+0000 7fee6a400640 1 -- 172.21.15.32:0/3524967566 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee40068380 msgr2=0x7fee4006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.858+0000 7fee6a400640 1 --2- 172.21.15.32:0/3524967566 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee40068380 0x7fee4006a840 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0ad000 tx=0x7fee4c0a6000 comp rx=0 tx=0).stop 2024-04-17T09:59:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.858+0000 7fee6a400640 1 -- 172.21.15.32:0/3524967566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640d00f0 msgr2=0x7fee64148970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.858+0000 7fee6a400640 1 --2- 172.21.15.32:0/3524967566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640d00f0 0x7fee64148970 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fee5400d620 tx=0x7fee5400dae0 comp rx=0 tx=0).stop 2024-04-17T09:59:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.858+0000 7fee6a400640 1 -- 172.21.15.32:0/3524967566 shutdown_connections 2024-04-17T09:59:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.858+0000 7fee6a400640 1 --2- 172.21.15.32:0/3524967566 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee40068380 0x7fee4006a840 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.858+0000 7fee6a400640 1 --2- 172.21.15.32:0/3524967566 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee6415c670 0x7fee64155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.858+0000 7fee6a400640 1 --2- 172.21.15.32:0/3524967566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee6415c290 0x7fee6414c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.858+0000 7fee6a400640 1 --2- 172.21.15.32:0/3524967566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640d00f0 0x7fee64148970 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.858+0000 7fee6a400640 1 -- 172.21.15.32:0/3524967566 >> 172.21.15.32:0/3524967566 conn(0x7fee6406ba30 msgr2=0x7fee640adbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.858+0000 7fee6a400640 1 -- 172.21.15.32:0/3524967566 shutdown_connections 2024-04-17T09:59:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:21.858+0000 7fee6a400640 1 -- 172.21.15.32:0/3524967566 wait complete. 2024-04-17T09:59:21.876 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2024-04-17T09:59:21.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:21.876 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T09:59:21.894 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[44984]: fuse finished with error 0 and tester_r 0 2024-04-17T09:59:21.896 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T09:59:27.998 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T09:59:27.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:27.998 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T09:59:28.009 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:59:28.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.206+0000 7fd6b6800640 1 Processor -- start 2024-04-17T09:59:28.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.206+0000 7fd6b6800640 1 -- start start 2024-04-17T09:59:28.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.210+0000 7fd6b6800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd6b0144e90 0x7fd6b00b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:28.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.210+0000 7fd6b6800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd6b00b4de0 0x7fd6b015baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:28.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.210+0000 7fd6b6800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6b0072bb0 con 0x7fd6b015bfe0 2024-04-17T09:59:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.210+0000 7fd6b6800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd6b015c430 con 0x7fd6b0144e90 2024-04-17T09:59:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.210+0000 7fd6b6800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd6b0145270 con 0x7fd6b00b4de0 2024-04-17T09:59:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.210+0000 7fd6af400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd6b0144e90 0x7fd6b00b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.210+0000 7fd6af400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd6b0144e90 0x7fd6b00b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40110/0 (socket says 172.21.15.32:40110) 2024-04-17T09:59:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.210+0000 7fd6af400640 1 -- 172.21.15.32:0/1270070653 learned_addr learned my addr 172.21.15.32:0/1270070653 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.210+0000 7fd6a6a00640 1 --2- 172.21.15.32:0/1270070653 >> v2:172.21.15.144:3300/0 conn(0x7fd6b00b4de0 0x7fd6b015baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.210+0000 7fd6af400640 1 -- 172.21.15.32:0/1270070653 >> v2:172.21.15.144:3300/0 conn(0x7fd6b00b4de0 msgr2=0x7fd6b015baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.210+0000 7fd6af400640 1 --2- 172.21.15.32:0/1270070653 >> v2:172.21.15.144:3300/0 conn(0x7fd6b00b4de0 0x7fd6b015baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:28.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.210+0000 7fd6af400640 1 -- 172.21.15.32:0/1270070653 >> v1:172.21.15.144:6789/0 conn(0x7fd6b015bfe0 legacy=0x7fd6b006ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:28.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.210+0000 7fd6af400640 1 -- 172.21.15.32:0/1270070653 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b00b3dc0 con 0x7fd6b0144e90 2024-04-17T09:59:28.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.214+0000 7fd6af400640 1 --2- 172.21.15.32:0/1270070653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6b0144e90 0x7fd6b00b48a0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd6b006fb40 tx=0x7fd6a001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef365bbff6ad8d02 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:28.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.214+0000 7fd6aea00640 1 -- 172.21.15.32:0/1270070653 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a00aa650 con 0x7fd6b0144e90 2024-04-17T09:59:28.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.214+0000 7fd6aea00640 1 -- 172.21.15.32:0/1270070653 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6a00af070 con 0x7fd6b0144e90 2024-04-17T09:59:28.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.214+0000 7fd6aea00640 1 -- 172.21.15.32:0/1270070653 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a00aaa10 con 0x7fd6b0144e90 2024-04-17T09:59:28.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.214+0000 7fd6b6800640 1 -- 172.21.15.32:0/1270070653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6b0144e90 msgr2=0x7fd6b00b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:28.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.214+0000 7fd6b6800640 1 --2- 172.21.15.32:0/1270070653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6b0144e90 0x7fd6b00b48a0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd6b006fb40 tx=0x7fd6a001c4f0 comp rx=0 tx=0).stop 2024-04-17T09:59:28.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.214+0000 7fd6b6800640 1 -- 172.21.15.32:0/1270070653 shutdown_connections 2024-04-17T09:59:28.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.214+0000 7fd6b6800640 1 --2- 172.21.15.32:0/1270070653 >> v2:172.21.15.144:3300/0 conn(0x7fd6b00b4de0 0x7fd6b015baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:28.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.214+0000 7fd6b6800640 1 --2- 172.21.15.32:0/1270070653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6b0144e90 0x7fd6b00b48a0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:28.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.214+0000 7fd6b6800640 1 -- 172.21.15.32:0/1270070653 >> 172.21.15.32:0/1270070653 conn(0x7fd6b006ba30 msgr2=0x7fd6b006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:28.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.214+0000 7fd6b6800640 1 -- 172.21.15.32:0/1270070653 shutdown_connections 2024-04-17T09:59:28.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.214+0000 7fd6b6800640 1 -- 172.21.15.32:0/1270070653 wait complete. 2024-04-17T09:59:28.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.214+0000 7fd6b6800640 1 Processor -- start 2024-04-17T09:59:28.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.214+0000 7fd6b6800640 1 -- start start 2024-04-17T09:59:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.214+0000 7fd6b6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6b00b4de0 0x7fd6b014c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.214+0000 7fd6b6800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6b0144e90 0x7fd6b013b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.214+0000 7fd6b6800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6b015bfe0 0x7fd6b00cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.214+0000 7fd6b6800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b0072670 con 0x7fd6b00b4de0 2024-04-17T09:59:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.214+0000 7fd6b6800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b006f880 con 0x7fd6b015bfe0 2024-04-17T09:59:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.214+0000 7fd6b6800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b00727f0 con 0x7fd6b0144e90 2024-04-17T09:59:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.218+0000 7fd6af400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6b00b4de0 0x7fd6b014c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.218+0000 7fd6af400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6b00b4de0 0x7fd6b014c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40124/0 (socket says 172.21.15.32:40124) 2024-04-17T09:59:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.218+0000 7fd6af400640 1 -- 172.21.15.32:0/3561254750 learned_addr learned my addr 172.21.15.32:0/3561254750 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.218+0000 7fd6a6a00640 1 --2- 172.21.15.32:0/3561254750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6b0144e90 0x7fd6b013b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.218+0000 7fd6afe00640 1 --2- 172.21.15.32:0/3561254750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6b015bfe0 0x7fd6b00cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.218+0000 7fd6af400640 1 -- 172.21.15.32:0/3561254750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6b0144e90 msgr2=0x7fd6b013b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.218+0000 7fd6af400640 1 --2- 172.21.15.32:0/3561254750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6b0144e90 0x7fd6b013b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.218+0000 7fd6af400640 1 -- 172.21.15.32:0/3561254750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6b015bfe0 msgr2=0x7fd6b00cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.218+0000 7fd6af400640 1 --2- 172.21.15.32:0/3561254750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6b015bfe0 0x7fd6b00cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.218+0000 7fd6af400640 1 -- 172.21.15.32:0/3561254750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b00ce8e0 con 0x7fd6b00b4de0 2024-04-17T09:59:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.218+0000 7fd6afe00640 1 --2- 172.21.15.32:0/3561254750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6b015bfe0 0x7fd6b00cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.218+0000 7fd6af400640 1 --2- 172.21.15.32:0/3561254750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6b00b4de0 0x7fd6b014c6b0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00ac040 tx=0x7fd6a001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.218+0000 7fd6a7e00640 1 -- 172.21.15.32:0/3561254750 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a00bd5f0 con 0x7fd6b00b4de0 2024-04-17T09:59:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.218+0000 7fd6a7e00640 1 -- 172.21.15.32:0/3561254750 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6a00a6040 con 0x7fd6b00b4de0 2024-04-17T09:59:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.218+0000 7fd6a7e00640 1 -- 172.21.15.32:0/3561254750 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a00bd9d0 con 0x7fd6b00b4de0 2024-04-17T09:59:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.218+0000 7fd6b6800640 1 -- 172.21.15.32:0/3561254750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b00ca4a0 con 0x7fd6b00b4de0 2024-04-17T09:59:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.218+0000 7fd6b6800640 1 -- 172.21.15.32:0/3561254750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b00ca760 con 0x7fd6b00b4de0 2024-04-17T09:59:28.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.218+0000 7fd6b6800640 1 -- 172.21.15.32:0/3561254750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6b0068a90 con 0x7fd6b00b4de0 2024-04-17T09:59:28.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.218+0000 7fd6a7e00640 1 -- 172.21.15.32:0/3561254750 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd6a00b33f0 con 0x7fd6b00b4de0 2024-04-17T09:59:28.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.218+0000 7fd6a7e00640 1 --2- 172.21.15.32:0/3561254750 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd68c068380 0x7fd68c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:28.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.222+0000 7fd6a7e00640 1 -- 172.21.15.32:0/3561254750 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fd6a0121030 con 0x7fd6b00b4de0 2024-04-17T09:59:28.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.222+0000 7fd6a6a00640 1 --2- 172.21.15.32:0/3561254750 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd68c068380 0x7fd68c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:28.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.222+0000 7fd6a6a00640 1 --2- 172.21.15.32:0/3561254750 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd68c068380 0x7fd68c06a840 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd698003fd0 tx=0x7fd6980039a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:28.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.226+0000 7fd6a7e00640 1 -- 172.21.15.32:0/3561254750 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd6a00c3060 con 0x7fd6b00b4de0 2024-04-17T09:59:28.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.426+0000 7fd6b6800640 1 -- 172.21.15.32:0/3561254750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6b013c410 con 0x7fd6b00b4de0 2024-04-17T09:59:28.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.426+0000 7fd6a7e00640 1 -- 172.21.15.32:0/3561254750 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5204 (secure 0 0 0) 0x7fd6a0016110 con 0x7fd6b00b4de0 2024-04-17T09:59:28.423 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:59:28.424 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6419,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3534108103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3534108103},{"type":"v1","addr":"172.21.15.144:6835","nonce":3534108103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7003,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/935945019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":935945019},{"type":"v1","addr":"172.21.15.144:6834","nonce":935945019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3044362393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3044362393},{"type":"v1","addr":"172.21.15.32:6836","nonce":3044362393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:59:04.834110+0000","modified":"2024-04-17T09:59:07.850260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7557":{"gid":7557,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/2656735817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2656735817},{"type":"v1","addr":"172.21.15.32:6837","nonce":2656735817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7557,"qdb_cluster":[7557]},"id":6}]} 2024-04-17T09:59:28.424 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 59 2024-04-17T09:59:28.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.426+0000 7fd6b6800640 1 -- 172.21.15.32:0/3561254750 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd68c068380 msgr2=0x7fd68c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:28.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.426+0000 7fd6b6800640 1 --2- 172.21.15.32:0/3561254750 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd68c068380 0x7fd68c06a840 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd698003fd0 tx=0x7fd6980039a0 comp rx=0 tx=0).stop 2024-04-17T09:59:28.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.426+0000 7fd6b6800640 1 -- 172.21.15.32:0/3561254750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6b00b4de0 msgr2=0x7fd6b014c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:28.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.426+0000 7fd6b6800640 1 --2- 172.21.15.32:0/3561254750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6b00b4de0 0x7fd6b014c6b0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00ac040 tx=0x7fd6a001ad30 comp rx=0 tx=0).stop 2024-04-17T09:59:28.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.426+0000 7fd6b6800640 1 -- 172.21.15.32:0/3561254750 shutdown_connections 2024-04-17T09:59:28.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.426+0000 7fd6b6800640 1 --2- 172.21.15.32:0/3561254750 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd68c068380 0x7fd68c06a840 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:28.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.426+0000 7fd6b6800640 1 --2- 172.21.15.32:0/3561254750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6b0144e90 0x7fd6b013b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:28.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.426+0000 7fd6b6800640 1 --2- 172.21.15.32:0/3561254750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6b015bfe0 0x7fd6b00cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:28.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.426+0000 7fd6b6800640 1 --2- 172.21.15.32:0/3561254750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6b00b4de0 0x7fd6b014c6b0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:28.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.426+0000 7fd6b6800640 1 -- 172.21.15.32:0/3561254750 >> 172.21.15.32:0/3561254750 conn(0x7fd6b006ba30 msgr2=0x7fd6b00b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:28.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.426+0000 7fd6b6800640 1 -- 172.21.15.32:0/3561254750 shutdown_connections 2024-04-17T09:59:28.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.426+0000 7fd6b6800640 1 -- 172.21.15.32:0/3561254750 wait complete. 2024-04-17T09:59:28.446 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-17T09:59:28.621 INFO:teuthology.orchestra.run.smithi032.stdout:{"message":"","return_code":0} 2024-04-17T09:59:28.621 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-17T09:59:28.621 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T09:59:28.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.766+0000 7f0238c00640 1 Processor -- start 2024-04-17T09:59:28.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.766+0000 7f0238c00640 1 -- start start 2024-04-17T09:59:28.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.766+0000 7f0238c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f023413f6e0 0x7f02340cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:28.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.766+0000 7f0238c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02340cf370 0x7f02340d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:28.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.766+0000 7f0238c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02341601e0 con 0x7f02340d0820 2024-04-17T09:59:28.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.766+0000 7f0238c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f023413fac0 con 0x7f02340cf370 2024-04-17T09:59:28.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.766+0000 7f0238c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0234160480 con 0x7f023413f6e0 2024-04-17T09:59:28.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.766+0000 7f0231400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02340cf370 0x7f02340d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:28.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.766+0000 7f0232800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f02340d0820 0x7f0234141d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52454/0 (socket says 172.21.15.32:52454) 2024-04-17T09:59:28.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.766+0000 7f0232800640 1 -- 172.21.15.32:0/3884692536 learned_addr learned my addr 172.21.15.32:0/3884692536 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:28.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.766+0000 7f0231e00640 1 --2- 172.21.15.32:0/3884692536 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f023413f6e0 0x7f02340cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:28.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.766+0000 7f0231400640 1 -- 172.21.15.32:0/3884692536 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f023413f6e0 msgr2=0x7f02340cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:28.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.766+0000 7f0231400640 1 --2- 172.21.15.32:0/3884692536 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f023413f6e0 0x7f02340cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:28.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.766+0000 7f0231400640 1 -- 172.21.15.32:0/3884692536 >> v1:172.21.15.32:6789/0 conn(0x7f02340d0820 legacy=0x7f0234141d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:28.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.766+0000 7f0231400640 1 -- 172.21.15.32:0/3884692536 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0234160760 con 0x7f02340cf370 2024-04-17T09:59:28.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.766+0000 7f0230a00640 1 -- 172.21.15.32:0/3884692536 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2688702258 0 0) 0x7f02341601e0 con 0x7f02340d0820 2024-04-17T09:59:28.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.770+0000 7f0231400640 1 --2- 172.21.15.32:0/3884692536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02340cf370 0x7f02340d02e0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f02200097c0 tx=0x7f022001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4315d2ae8f1f7d5a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.770+0000 7f0230a00640 1 -- 172.21.15.32:0/3884692536 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02200b0070 con 0x7f02340cf370 2024-04-17T09:59:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.770+0000 7f0230a00640 1 -- 172.21.15.32:0/3884692536 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02200ab600 con 0x7f02340cf370 2024-04-17T09:59:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.770+0000 7f0230a00640 1 -- 172.21.15.32:0/3884692536 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02200ab9d0 con 0x7f02340cf370 2024-04-17T09:59:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.770+0000 7f0238c00640 1 -- 172.21.15.32:0/3884692536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02340cf370 msgr2=0x7f02340d02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.770+0000 7f0238c00640 1 --2- 172.21.15.32:0/3884692536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02340cf370 0x7f02340d02e0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f02200097c0 tx=0x7f022001c4a0 comp rx=0 tx=0).stop 2024-04-17T09:59:28.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.770+0000 7f0238c00640 1 -- 172.21.15.32:0/3884692536 shutdown_connections 2024-04-17T09:59:28.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.770+0000 7f0238c00640 1 --2- 172.21.15.32:0/3884692536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02340cf370 0x7f02340d02e0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:28.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.770+0000 7f0238c00640 1 --2- 172.21.15.32:0/3884692536 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f023413f6e0 0x7f02340cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:28.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.770+0000 7f0238c00640 1 -- 172.21.15.32:0/3884692536 >> 172.21.15.32:0/3884692536 conn(0x7f023406ba30 msgr2=0x7f023406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:28.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.770+0000 7f0238c00640 1 -- 172.21.15.32:0/3884692536 shutdown_connections 2024-04-17T09:59:28.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.770+0000 7f0238c00640 1 -- 172.21.15.32:0/3884692536 wait complete. 2024-04-17T09:59:28.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f0238c00640 1 Processor -- start 2024-04-17T09:59:28.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f0238c00640 1 -- start start 2024-04-17T09:59:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f0238c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02340cf370 0x7f023413e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f0238c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02340d0820 0x7f02341432b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f0238c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f023413f6e0 0x7f02341437f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f0238c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f02340cff10 con 0x7f023413f6e0 2024-04-17T09:59:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f0238c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f02340cfd90 con 0x7f02340d0820 2024-04-17T09:59:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f0238c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0234169580 con 0x7f02340cf370 2024-04-17T09:59:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f0232800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f023413f6e0 0x7f02341437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f0232800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f023413f6e0 0x7f02341437f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40136/0 (socket says 172.21.15.32:40136) 2024-04-17T09:59:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f0232800640 1 -- 172.21.15.32:0/532743746 learned_addr learned my addr 172.21.15.32:0/532743746 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f0231e00640 1 --2- 172.21.15.32:0/532743746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02340cf370 0x7f023413e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f0231400640 1 --2- 172.21.15.32:0/532743746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02340d0820 0x7f02341432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f0232800640 1 -- 172.21.15.32:0/532743746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02340cf370 msgr2=0x7f023413e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f0232800640 1 --2- 172.21.15.32:0/532743746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02340cf370 0x7f023413e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f0232800640 1 -- 172.21.15.32:0/532743746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02340d0820 msgr2=0x7f02341432b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:28.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f0232800640 1 --2- 172.21.15.32:0/532743746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02340d0820 0x7f02341432b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:28.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f0232800640 1 -- 172.21.15.32:0/532743746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023413e960 con 0x7f023413f6e0 2024-04-17T09:59:28.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f0231e00640 1 --2- 172.21.15.32:0/532743746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02340cf370 0x7f023413e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:28.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f0232800640 1 --2- 172.21.15.32:0/532743746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f023413f6e0 0x7f02341437f0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f022c00c7a0 tx=0x7f022c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:28.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f022a000640 1 -- 172.21.15.32:0/532743746 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c018070 con 0x7f023413f6e0 2024-04-17T09:59:28.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f022a000640 1 -- 172.21.15.32:0/532743746 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f022c00a640 con 0x7f023413f6e0 2024-04-17T09:59:28.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f0238c00640 1 -- 172.21.15.32:0/532743746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f023415c430 con 0x7f023413f6e0 2024-04-17T09:59:28.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f022a000640 1 -- 172.21.15.32:0/532743746 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c00aab0 con 0x7f023413f6e0 2024-04-17T09:59:28.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f0238c00640 1 -- 172.21.15.32:0/532743746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02340b2080 con 0x7f023413f6e0 2024-04-17T09:59:28.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.774+0000 7f0238c00640 1 -- 172.21.15.32:0/532743746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0234068a90 con 0x7f023413f6e0 2024-04-17T09:59:28.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.778+0000 7f022a000640 1 -- 172.21.15.32:0/532743746 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f022c01b070 con 0x7f023413f6e0 2024-04-17T09:59:28.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.778+0000 7f022a000640 1 --2- 172.21.15.32:0/532743746 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0214068380 0x7f021406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:28.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.778+0000 7f022a000640 1 -- 172.21.15.32:0/532743746 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f022c085f70 con 0x7f023413f6e0 2024-04-17T09:59:28.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.778+0000 7f0231e00640 1 --2- 172.21.15.32:0/532743746 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0214068380 0x7f021406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.778+0000 7f0231e00640 1 --2- 172.21.15.32:0/532743746 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0214068380 0x7f021406a840 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f02340cd570 tx=0x7f0224008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:28.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.782+0000 7f022a000640 1 -- 172.21.15.32:0/532743746 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f022c055ed0 con 0x7f023413f6e0 2024-04-17T09:59:28.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.982+0000 7f0238c00640 1 -- 172.21.15.32:0/532743746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0234143090 con 0x7f023413f6e0 2024-04-17T09:59:28.980 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-17T09:59:28.982+0000 7f93c9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T09:59:28.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.982+0000 7f022a000640 1 -- 172.21.15.32:0/532743746 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v60) ==== 161+0+0 (secure 0 0 0) 0x7f022c05a1d0 con 0x7f023413f6e0 2024-04-17T09:59:28.982 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T09:59:28.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.986+0000 7f0238c00640 1 -- 172.21.15.32:0/532743746 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0214068380 msgr2=0x7f021406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:28.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.986+0000 7f0238c00640 1 --2- 172.21.15.32:0/532743746 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0214068380 0x7f021406a840 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f02340cd570 tx=0x7f0224008040 comp rx=0 tx=0).stop 2024-04-17T09:59:28.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.986+0000 7f0238c00640 1 -- 172.21.15.32:0/532743746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f023413f6e0 msgr2=0x7f02341437f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:28.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.986+0000 7f0238c00640 1 --2- 172.21.15.32:0/532743746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f023413f6e0 0x7f02341437f0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f022c00c7a0 tx=0x7f022c00cc60 comp rx=0 tx=0).stop 2024-04-17T09:59:28.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.986+0000 7f0238c00640 1 -- 172.21.15.32:0/532743746 shutdown_connections 2024-04-17T09:59:28.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.986+0000 7f0238c00640 1 --2- 172.21.15.32:0/532743746 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0214068380 0x7f021406a840 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:28.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.986+0000 7f0238c00640 1 --2- 172.21.15.32:0/532743746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f023413f6e0 0x7f02341437f0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:28.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.986+0000 7f0238c00640 1 --2- 172.21.15.32:0/532743746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02340d0820 0x7f02341432b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:28.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.986+0000 7f0238c00640 1 --2- 172.21.15.32:0/532743746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02340cf370 0x7f023413e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:28.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.986+0000 7f0238c00640 1 -- 172.21.15.32:0/532743746 >> 172.21.15.32:0/532743746 conn(0x7f023406ba30 msgr2=0x7f023415ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:28.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.986+0000 7f0238c00640 1 -- 172.21.15.32:0/532743746 shutdown_connections 2024-04-17T09:59:28.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:28.986+0000 7f0238c00640 1 -- 172.21.15.32:0/532743746 wait complete. 2024-04-17T09:59:28.993 INFO:tasks.ceph.mds.c.smithi032.stderr: -4258> 2024-04-17T09:59:07.410+0000 7fc47c000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T09:59:29.006 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-04-17T09:59:29.043 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-17T09:59:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.054+0000 7f28fb5d9080 1 Processor -- start 2024-04-17T09:59:29.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.054+0000 7f28fb5d9080 1 -- start start 2024-04-17T09:59:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.054+0000 7f28fb5d9080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56230c74c860 0x56230c7327d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.054+0000 7f28fb5d9080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56230c7db680 0x56230c732d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.054+0000 7f28fb5d9080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56230c7dba60 0x56230c733250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.054+0000 7f28fb5d9080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56230c7065d0 con 0x56230c7db680 2024-04-17T09:59:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.054+0000 7f28fb5d9080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56230c74eab0 con 0x56230c7dba60 2024-04-17T09:59:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.054+0000 7f28fb5d9080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x56230c706870 con 0x56230c74c860 2024-04-17T09:59:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.054+0000 7f28f7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56230c7db680 0x56230c732d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.054+0000 7f28f7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56230c7db680 0x56230c732d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40144/0 (socket says 172.21.15.32:40144) 2024-04-17T09:59:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.054+0000 7f28f7e00640 1 -- 172.21.15.32:0/3059812333 learned_addr learned my addr 172.21.15.32:0/3059812333 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.054+0000 7f28f8800640 1 --2- 172.21.15.32:0/3059812333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56230c74c860 0x56230c7327d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.054+0000 7f28f9200640 1 --2- 172.21.15.32:0/3059812333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56230c7dba60 0x56230c733250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.054+0000 7f28f7e00640 1 -- 172.21.15.32:0/3059812333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56230c74c860 msgr2=0x56230c7327d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.054+0000 7f28f7e00640 1 --2- 172.21.15.32:0/3059812333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56230c74c860 0x56230c7327d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.054+0000 7f28f7e00640 1 -- 172.21.15.32:0/3059812333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56230c7dba60 msgr2=0x56230c733250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.054+0000 7f28f7e00640 1 --2- 172.21.15.32:0/3059812333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56230c7dba60 0x56230c733250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.054+0000 7f28f7e00640 1 -- 172.21.15.32:0/3059812333 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56230c652940 con 0x56230c7db680 2024-04-17T09:59:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28f7e00640 1 --2- 172.21.15.32:0/3059812333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56230c7db680 0x56230c732d10 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f28e40ad040 tx=0x7f28e4003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aacb34555aa28ab server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28f6a00640 1 -- 172.21.15.32:0/3059812333 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e40af040 con 0x56230c7db680 2024-04-17T09:59:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28f6a00640 1 -- 172.21.15.32:0/3059812333 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e40aa9b0 con 0x56230c7db680 2024-04-17T09:59:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28f6a00640 1 -- 172.21.15.32:0/3059812333 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e40aada0 con 0x56230c7db680 2024-04-17T09:59:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28fb5d9080 1 -- 172.21.15.32:0/3059812333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56230c7db680 msgr2=0x56230c732d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28fb5d9080 1 --2- 172.21.15.32:0/3059812333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56230c7db680 0x56230c732d10 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f28e40ad040 tx=0x7f28e4003030 comp rx=0 tx=0).stop 2024-04-17T09:59:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28fb5d9080 1 -- 172.21.15.32:0/3059812333 shutdown_connections 2024-04-17T09:59:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28fb5d9080 1 --2- 172.21.15.32:0/3059812333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56230c7dba60 0x56230c733250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28fb5d9080 1 --2- 172.21.15.32:0/3059812333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56230c7db680 0x56230c732d10 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28fb5d9080 1 --2- 172.21.15.32:0/3059812333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56230c74c860 0x56230c7327d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28fb5d9080 1 -- 172.21.15.32:0/3059812333 >> 172.21.15.32:0/3059812333 conn(0x56230c72fa00 msgr2=0x56230c7b9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28fb5d9080 1 -- 172.21.15.32:0/3059812333 shutdown_connections 2024-04-17T09:59:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28fb5d9080 1 -- 172.21.15.32:0/3059812333 wait complete. 2024-04-17T09:59:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28fb5d9080 1 Processor -- start 2024-04-17T09:59:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28fb5d9080 1 -- start start 2024-04-17T09:59:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28fb5d9080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56230c74c860 0x56230c7d2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28fb5d9080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56230c7db680 0x56230c7d6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28fb5d9080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56230c7dba60 0x56230c7d4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28fb5d9080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56230c734280 con 0x56230c7db680 2024-04-17T09:59:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28fb5d9080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56230c7442d0 con 0x56230c7dba60 2024-04-17T09:59:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28fb5d9080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x56230c744c00 con 0x56230c74c860 2024-04-17T09:59:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28f7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56230c7db680 0x56230c7d6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28f8800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56230c74c860 0x56230c7d2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28f7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56230c7db680 0x56230c7d6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40154/0 (socket says 172.21.15.32:40154) 2024-04-17T09:59:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28f7e00640 1 -- 172.21.15.32:0/2421971986 learned_addr learned my addr 172.21.15.32:0/2421971986 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28f9200640 1 --2- 172.21.15.32:0/2421971986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56230c7dba60 0x56230c7d4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28f8800640 1 -- 172.21.15.32:0/2421971986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56230c7dba60 msgr2=0x56230c7d4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28f8800640 1 --2- 172.21.15.32:0/2421971986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56230c7dba60 0x56230c7d4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28f8800640 1 -- 172.21.15.32:0/2421971986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56230c7db680 msgr2=0x56230c7d6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28f8800640 1 --2- 172.21.15.32:0/2421971986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56230c7db680 0x56230c7d6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28f8800640 1 -- 172.21.15.32:0/2421971986 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56230c62e950 con 0x56230c74c860 2024-04-17T09:59:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.058+0000 7f28f7e00640 1 --2- 172.21.15.32:0/2421971986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56230c7db680 0x56230c7d6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:59:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.062+0000 7f28f8800640 1 --2- 172.21.15.32:0/2421971986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56230c74c860 0x56230c7d2a30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0086f0 tx=0x7f28ec008bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.062+0000 7f28f5600640 1 -- 172.21.15.32:0/2421971986 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec01b070 con 0x56230c74c860 2024-04-17T09:59:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.062+0000 7f28f5600640 1 -- 172.21.15.32:0/2421971986 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28ec006b70 con 0x56230c74c860 2024-04-17T09:59:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.062+0000 7f28f5600640 1 -- 172.21.15.32:0/2421971986 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec016480 con 0x56230c74c860 2024-04-17T09:59:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.062+0000 7f28fb5d9080 1 -- 172.21.15.32:0/2421971986 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x56230c7037f0 con 0x56230c74c860 2024-04-17T09:59:29.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.062+0000 7f28fb5d9080 1 -- 172.21.15.32:0/2421971986 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x56230c703d30 con 0x56230c74c860 2024-04-17T09:59:29.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.062+0000 7f28f5600640 1 -- 172.21.15.32:0/2421971986 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f28ec01e070 con 0x56230c74c860 2024-04-17T09:59:29.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.062+0000 7f28f5600640 1 --2- 172.21.15.32:0/2421971986 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f28d8066d90 0x7f28d8069250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.062+0000 7f28f5600640 1 -- 172.21.15.32:0/2421971986 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f28ec085640 con 0x56230c74c860 2024-04-17T09:59:29.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.062+0000 7f28f7e00640 1 --2- 172.21.15.32:0/2421971986 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f28d8066d90 0x7f28d8069250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.062+0000 7f28fb5d9080 1 --2- 172.21.15.32:0/2421971986 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x56230c7d3080 0x56230c7d3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.062+0000 7f28fb5d9080 1 -- 172.21.15.32:0/2421971986 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e83) -- 0x56230c7d39d0 con 0x56230c7d3080 2024-04-17T09:59:29.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.062+0000 7f28f9200640 1 --2- 172.21.15.32:0/2421971986 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x56230c7d3080 0x56230c7d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.066+0000 7f28f7e00640 1 --2- 172.21.15.32:0/2421971986 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f28d8066d90 0x7f28d8069250 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f28e401b350 tx=0x7f28e40a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:29.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.066+0000 7f28f9200640 1 --2- 172.21.15.32:0/2421971986 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x56230c7d3080 0x56230c7d3490 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:29.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.066+0000 7f28f9200640 1 -- 172.21.15.32:0/2421971986 <== osd.0 v2:172.21.15.32:6800/2392056658 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v83'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f28f000cac0 con 0x56230c7d3080 2024-04-17T09:59:29.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.066+0000 7f28fb5d9080 1 -- 172.21.15.32:0/2421971986 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x56230c7d3080 msgr2=0x56230c7d3490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.066+0000 7f28fb5d9080 1 --2- 172.21.15.32:0/2421971986 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x56230c7d3080 0x56230c7d3490 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.066+0000 7f28fb5d9080 1 -- 172.21.15.32:0/2421971986 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f28d8066d90 msgr2=0x7f28d8069250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.066+0000 7f28fb5d9080 1 --2- 172.21.15.32:0/2421971986 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f28d8066d90 0x7f28d8069250 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f28e401b350 tx=0x7f28e40a7000 comp rx=0 tx=0).stop 2024-04-17T09:59:29.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.066+0000 7f28fb5d9080 1 -- 172.21.15.32:0/2421971986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56230c74c860 msgr2=0x56230c7d2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.066+0000 7f28fb5d9080 1 --2- 172.21.15.32:0/2421971986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56230c74c860 0x56230c7d2a30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0086f0 tx=0x7f28ec008bb0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.066+0000 7f28f9200640 1 -- 172.21.15.32:0/2421971986 reap_dead start 2024-04-17T09:59:29.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.066+0000 7f28fb5d9080 1 -- 172.21.15.32:0/2421971986 shutdown_connections 2024-04-17T09:59:29.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.066+0000 7f28fb5d9080 1 -- 172.21.15.32:0/2421971986 >> 172.21.15.32:0/2421971986 conn(0x56230c72fa00 msgr2=0x56230c7b94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:29.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.066+0000 7f28fb5d9080 1 -- 172.21.15.32:0/2421971986 shutdown_connections 2024-04-17T09:59:29.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.066+0000 7f28fb5d9080 1 -- 172.21.15.32:0/2421971986 wait complete. 2024-04-17T09:59:29.069 DEBUG:teuthology.orchestra.run.smithi032:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-04-17T09:59:29.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.158+0000 7fceeb96a400 1 -- start start 2024-04-17T09:59:29.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.158+0000 7fceeb96a400 1 Processor -- start 2024-04-17T09:59:29.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.158+0000 7fceeb96a400 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6a067000 0x562f6ae3a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.158+0000 7fceeb96a400 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f6a067400 0x562f6ae3a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.158+0000 7fceeb96a400 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562f6a067800 0x562f6ae3ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.158+0000 7fceeb96a400 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562f6ae5a600 con 0x562f6a067000 2024-04-17T09:59:29.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.158+0000 7fceeb96a400 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562f6ae5a480 con 0x562f6a067400 2024-04-17T09:59:29.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.158+0000 7fceeb96a400 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x562f6ae5a300 con 0x562f6a067800 2024-04-17T09:59:29.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.158+0000 7fcee9a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6a067000 0x562f6ae3a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.158+0000 7fcee9a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6a067000 0x562f6ae3a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40166/0 (socket says 172.21.15.32:40166) 2024-04-17T09:59:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.158+0000 7fcee9a00640 1 -- 172.21.15.32:0/3378279030 learned_addr learned my addr 172.21.15.32:0/3378279030 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fceea400640 1 --2- 172.21.15.32:0/3378279030 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562f6a067800 0x562f6ae3ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fcee9a00640 1 -- 172.21.15.32:0/3378279030 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562f6a067800 msgr2=0x562f6ae3ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fcee9a00640 1 --2- 172.21.15.32:0/3378279030 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562f6a067800 0x562f6ae3ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fcee9a00640 1 -- 172.21.15.32:0/3378279030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f6a067400 msgr2=0x562f6ae3a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fcee9a00640 1 --2- 172.21.15.32:0/3378279030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f6a067400 0x562f6ae3a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fcee9a00640 1 -- 172.21.15.32:0/3378279030 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f6a1da960 con 0x562f6a067000 2024-04-17T09:59:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fceea400640 1 --2- 172.21.15.32:0/3378279030 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562f6a067800 0x562f6ae3ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:59:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fcee9a00640 1 --2- 172.21.15.32:0/3378279030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6a067000 0x562f6ae3a000 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x562f6aee3470 tx=0x562f6a142f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28a14bfe48b55e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fcee6800640 1 -- 172.21.15.32:0/3378279030 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562f6a09c4e0 con 0x562f6a067000 2024-04-17T09:59:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fcee6800640 1 -- 172.21.15.32:0/3378279030 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x562f6a0b21c0 con 0x562f6a067000 2024-04-17T09:59:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fcee6800640 1 -- 172.21.15.32:0/3378279030 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562f6a09cb60 con 0x562f6a067000 2024-04-17T09:59:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fceeb96a400 1 -- 172.21.15.32:0/3378279030 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f6a1dad20 con 0x562f6a067000 2024-04-17T09:59:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fceeb96a400 1 -- 172.21.15.32:0/3378279030 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({fsmap=0}) -- 0x562f6a1daf00 con 0x562f6a067000 2024-04-17T09:59:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fceeb96a400 4 waiting for MDS map... 2024-04-17T09:59:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fcee6800640 1 -- 172.21.15.32:0/3378279030 <== mon.0 v2:172.21.15.32:3300/0 4 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x562f6af8c000 con 0x562f6a067000 2024-04-17T09:59:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fcee6800640 1 -- 172.21.15.32:0/3378279030 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 60) ==== 2886+0+0 (secure 0 0 0) 0x562f6a0bd180 con 0x562f6a067000 2024-04-17T09:59:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fceeb96a400 4 Got MDS map 60 2024-04-17T09:59:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fceeb96a400 10 main: JournalTool::main 2024-04-17T09:59:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fceeb96a400 4 main: JournalTool: connecting to RADOS... 2024-04-17T09:59:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fceeb96a400 1 Processor -- start 2024-04-17T09:59:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fceeb96a400 1 -- start start 2024-04-17T09:59:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fceeb96a400 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562f6af97400 0x562f6ae3b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fceeb96a400 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6af97800 0x562f6ae3b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fceeb96a400 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f6af97c00 0x562f6ae3bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fceeb96a400 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562f6ae5a180 con 0x562f6af97800 2024-04-17T09:59:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fceeb96a400 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562f6ae5a900 con 0x562f6af97c00 2024-04-17T09:59:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fceeb96a400 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x562f6a0c6900 con 0x562f6af97400 2024-04-17T09:59:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fcee9a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6af97800 0x562f6ae3b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fcee9a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6af97800 0x562f6ae3b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40172/0 (socket says 172.21.15.32:40172) 2024-04-17T09:59:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fcee9a00640 1 -- 172.21.15.32:0/2945963390 learned_addr learned my addr 172.21.15.32:0/2945963390 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fcee9000640 1 --2- 172.21.15.32:0/2945963390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f6af97c00 0x562f6ae3bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fceea400640 1 --2- 172.21.15.32:0/2945963390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562f6af97400 0x562f6ae3b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fcee9a00640 1 -- 172.21.15.32:0/2945963390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562f6af97400 msgr2=0x562f6ae3b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fcee9a00640 1 --2- 172.21.15.32:0/2945963390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562f6af97400 0x562f6ae3b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fcee9a00640 1 -- 172.21.15.32:0/2945963390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f6af97c00 msgr2=0x562f6ae3bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fcee9a00640 1 --2- 172.21.15.32:0/2945963390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f6af97c00 0x562f6ae3bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fcee9a00640 1 -- 172.21.15.32:0/2945963390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f6a1dad20 con 0x562f6af97800 2024-04-17T09:59:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fcee9000640 1 --2- 172.21.15.32:0/2945963390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f6af97c00 0x562f6ae3bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.162+0000 7fcee9a00640 1 --2- 172.21.15.32:0/2945963390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6af97800 0x562f6ae3b600 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x562f6af9e2a0 tx=0x562f6a142ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c32e07b4fb2861 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fcee4000640 1 -- 172.21.15.32:0/2945963390 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562f6a09c340 con 0x562f6af97800 2024-04-17T09:59:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fcee4000640 1 -- 172.21.15.32:0/2945963390 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x562f6a0b2e00 con 0x562f6af97800 2024-04-17T09:59:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fcee4000640 1 -- 172.21.15.32:0/2945963390 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562f6a09c4e0 con 0x562f6af97800 2024-04-17T09:59:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fceeb96a400 1 -- 172.21.15.32:0/2945963390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6af97800 msgr2=0x562f6ae3b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fceeb96a400 1 --2- 172.21.15.32:0/2945963390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6af97800 0x562f6ae3b600 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x562f6af9e2a0 tx=0x562f6a142ea0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fceeb96a400 1 -- 172.21.15.32:0/2945963390 shutdown_connections 2024-04-17T09:59:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fceeb96a400 1 --2- 172.21.15.32:0/2945963390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f6af97c00 0x562f6ae3bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fceeb96a400 1 --2- 172.21.15.32:0/2945963390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6af97800 0x562f6ae3b600 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fceeb96a400 1 --2- 172.21.15.32:0/2945963390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562f6af97400 0x562f6ae3b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fceeb96a400 1 -- 172.21.15.32:0/2945963390 >> 172.21.15.32:0/2945963390 conn(0x562f6af97000 msgr2=0x562f6a094000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fceeb96a400 1 -- 172.21.15.32:0/2945963390 shutdown_connections 2024-04-17T09:59:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fceeb96a400 1 -- 172.21.15.32:0/2945963390 wait complete. 2024-04-17T09:59:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fceeb96a400 1 Processor -- start 2024-04-17T09:59:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fceeb96a400 1 -- start start 2024-04-17T09:59:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fceeb96a400 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562f6af97400 0x562f6ae3b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fceeb96a400 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f6af97800 0x562f6ae3b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fceeb96a400 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6af97c00 0x562f6ae3bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fceeb96a400 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562f6ae5b500 con 0x562f6af97c00 2024-04-17T09:59:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fceeb96a400 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562f6ae5b680 con 0x562f6af97800 2024-04-17T09:59:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fceeb96a400 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x562f6ae5a900 con 0x562f6af97400 2024-04-17T09:59:29.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fcee9000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6af97c00 0x562f6ae3bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fcee9000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6af97c00 0x562f6ae3bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40174/0 (socket says 172.21.15.32:40174) 2024-04-17T09:59:29.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fcee9000640 1 -- 172.21.15.32:0/3024641523 learned_addr learned my addr 172.21.15.32:0/3024641523 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:29.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fcee9a00640 1 --2- 172.21.15.32:0/3024641523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f6af97800 0x562f6ae3b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fceea400640 1 --2- 172.21.15.32:0/3024641523 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562f6af97400 0x562f6ae3b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fcee9000640 1 -- 172.21.15.32:0/3024641523 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562f6af97400 msgr2=0x562f6ae3b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fcee9000640 1 --2- 172.21.15.32:0/3024641523 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562f6af97400 0x562f6ae3b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fcee9000640 1 -- 172.21.15.32:0/3024641523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f6af97800 msgr2=0x562f6ae3b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fcee9000640 1 --2- 172.21.15.32:0/3024641523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f6af97800 0x562f6ae3b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fcee9000640 1 -- 172.21.15.32:0/3024641523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f6a1dba40 con 0x562f6af97c00 2024-04-17T09:59:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fcee9000640 1 --2- 172.21.15.32:0/3024641523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6af97c00 0x562f6ae3bb80 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x562f6af9f650 tx=0x562f6afa6e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fcee2200640 1 -- 172.21.15.32:0/3024641523 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562f6a09c340 con 0x562f6af97c00 2024-04-17T09:59:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fcee2200640 1 -- 172.21.15.32:0/3024641523 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x562f6a0b2c40 con 0x562f6af97c00 2024-04-17T09:59:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fcee2200640 1 -- 172.21.15.32:0/3024641523 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562f6a09c4e0 con 0x562f6af97c00 2024-04-17T09:59:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fceeb96a400 1 -- 172.21.15.32:0/3024641523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562f6a1dbc20 con 0x562f6af97c00 2024-04-17T09:59:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fceeb96a400 1 -- 172.21.15.32:0/3024641523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f6a1dba40 con 0x562f6af97c00 2024-04-17T09:59:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.166+0000 7fceeb96a400 4 main: JournalTool: resolving pool 12 2024-04-17T09:59:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.170+0000 7fcee2200640 1 -- 172.21.15.32:0/3024641523 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x562f6afeec00 con 0x562f6af97c00 2024-04-17T09:59:29.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.170+0000 7fcee2200640 1 --2- 172.21.15.32:0/3024641523 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x562f6a36ac00 0x562f6ae3c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.170+0000 7fcee2200640 1 -- 172.21.15.32:0/3024641523 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x562f6a36c800 con 0x562f6af97c00 2024-04-17T09:59:29.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.170+0000 7fceeb96a400 4 main: JournalTool: creating IoCtx.. 2024-04-17T09:59:29.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.170+0000 7fceeb96a400 4 main: Executing for rank 0 2024-04-17T09:59:29.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.170+0000 7fceea400640 1 --2- 172.21.15.32:0/3024641523 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x562f6a36ac00 0x562f6ae3c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.170+0000 7fceea400640 1 --2- 172.21.15.32:0/3024641523 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x562f6a36ac00 0x562f6ae3c100 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x562f6a15c0f0 tx=0x562f6a142300 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:29.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.170+0000 7fceeb96a400 1 -- start start 2024-04-17T09:59:29.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.170+0000 7fceeb96a400 1 Processor -- start 2024-04-17T09:59:29.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.170+0000 7fceeb96a400 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6a36b400 0x562f6ae3c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.170+0000 7fceeb96a400 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f6a36b800 0x562f6ae3cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.170+0000 7fceeb96a400 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562f6a36bc00 0x562f6ae3d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.170+0000 7fceeb96a400 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562f6ae5b800 con 0x562f6a36b400 2024-04-17T09:59:29.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.170+0000 7fceeb96a400 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562f6ae5b980 con 0x562f6a36b800 2024-04-17T09:59:29.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.170+0000 7fceeb96a400 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x562f6ae5bb00 con 0x562f6a36bc00 2024-04-17T09:59:29.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.170+0000 7fceea400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6a36b400 0x562f6ae3c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.170+0000 7fceea400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6a36b400 0x562f6ae3c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40188/0 (socket says 172.21.15.32:40188) 2024-04-17T09:59:29.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.170+0000 7fceea400640 1 -- 172.21.15.32:0/1096089581 learned_addr learned my addr 172.21.15.32:0/1096089581 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:29.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fceea400640 1 -- 172.21.15.32:0/1096089581 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562f6a36bc00 msgr2=0x562f6ae3d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:29.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fceea400640 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562f6a36bc00 0x562f6ae3d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fceea400640 1 -- 172.21.15.32:0/1096089581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f6a36b800 msgr2=0x562f6ae3cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:29.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fcee9a00640 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f6a36b800 0x562f6ae3cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fceea400640 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f6a36b800 0x562f6ae3cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fceea400640 1 -- 172.21.15.32:0/1096089581 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f6a49e780 con 0x562f6a36b400 2024-04-17T09:59:29.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fceea400640 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6a36b400 0x562f6ae3c680 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x562f6a3ff440 tx=0x562f6a142d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3516300732fe726c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:29.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fcede600640 1 -- 172.21.15.32:0/1096089581 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562f6a09c9c0 con 0x562f6a36b400 2024-04-17T09:59:29.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fcede600640 1 -- 172.21.15.32:0/1096089581 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x562f6a0b2a80 con 0x562f6a36b400 2024-04-17T09:59:29.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fcede600640 1 -- 172.21.15.32:0/1096089581 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562f6a39e9c0 con 0x562f6a36b400 2024-04-17T09:59:29.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fceeb96a400 1 -- 172.21.15.32:0/1096089581 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f6a49e960 con 0x562f6a36b400 2024-04-17T09:59:29.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fceeb96a400 1 -- 172.21.15.32:0/1096089581 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({fsmap=0}) -- 0x562f6a49eb40 con 0x562f6a36b400 2024-04-17T09:59:29.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fceeb96a400 4 waiting for MDS map... 2024-04-17T09:59:29.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fcede600640 1 -- 172.21.15.32:0/1096089581 <== mon.0 v2:172.21.15.32:3300/0 4 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x562f6a4b2000 con 0x562f6a36b400 2024-04-17T09:59:29.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fcede600640 1 -- 172.21.15.32:0/1096089581 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 60) ==== 2886+0+0 (secure 0 0 0) 0x562f6a0bd500 con 0x562f6a36b400 2024-04-17T09:59:29.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fceeb96a400 4 Got MDS map 60 2024-04-17T09:59:29.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fceeb96a400 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x562f6a4b1800 0x562f6ae3d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fceeb96a400 1 -- 172.21.15.32:0/1096089581 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83) -- 0x562f6a4b1c00 con 0x562f6a4b1800 2024-04-17T09:59:29.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fcee9a00640 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x562f6a4b1800 0x562f6ae3d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fcee9a00640 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x562f6a4b1800 0x562f6ae3d700 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=b94a324b0731b5ee server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:29.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fcee9a00640 1 -- 172.21.15.32:0/1096089581 <== osd.6 v2:172.21.15.144:6808/3198738696 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x562f6adea000 con 0x562f6a4b1800 2024-04-17T09:59:29.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fceeb96a400 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x562f6a4cc000 0x562f6a4d2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fceeb96a400 1 -- 172.21.15.32:0/1096089581 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:2 12.14 12:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83) -- 0x562f6a4cc400 con 0x562f6a4cc000 2024-04-17T09:59:29.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.174+0000 7fcee9000640 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x562f6a4cc000 0x562f6a4d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.178+0000 7fcee9000640 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x562f6a4cc000 0x562f6a4d2000 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=912aedd4ee1f1f05 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:29.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.178+0000 7fcee9000640 1 -- 172.21.15.32:0/1096089581 <== osd.5 v2:172.21.15.144:6800/2379578648 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv20 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x562f6adea500 con 0x562f6a4cc000 2024-04-17T09:59:29.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.178+0000 7fcedc800640 1 -- 172.21.15.32:0/1096089581 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:3 12.14 12:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e83) -- 0x562f6a4cdc00 con 0x562f6a4cc000 2024-04-17T09:59:29.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.178+0000 7fcedc800640 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x562f6a4e0000 0x562f6a4d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.178+0000 7fcedc800640 1 -- 172.21.15.32:0/1096089581 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:4 12.1b 12:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e83) -- 0x562f6a4e0400 con 0x562f6a4e0000 2024-04-17T09:59:29.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.178+0000 7fcee9000640 1 -- 172.21.15.32:0/1096089581 <== osd.5 v2:172.21.15.144:6800/2379578648 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv16 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x562f6adea500 con 0x562f6a4cc000 2024-04-17T09:59:29.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.178+0000 7fceea400640 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x562f6a4e0000 0x562f6a4d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.178+0000 7fceea400640 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x562f6a4e0000 0x562f6a4d3b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=cec165b23af4e005 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:29.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.182+0000 7fceea400640 1 -- 172.21.15.32:0/1096089581 <== osd.4 v2:172.21.15.144:6809/4076371964 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x562f6adea780 con 0x562f6a4e0000 2024-04-17T09:59:29.178 INFO:teuthology.orchestra.run.smithi032.stdout:old journal was 4209834~892 2024-04-17T09:59:29.178 INFO:teuthology.orchestra.run.smithi032.stdout:new journal start will be 8388608 (4177882 bytes past old end) 2024-04-17T09:59:29.178 INFO:teuthology.orchestra.run.smithi032.stdout:writing journal head 2024-04-17T09:59:29.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.182+0000 7fceeb96a400 1 -- 172.21.15.32:0/1096089581 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:5 12.14 12:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x562f6a4cc800 con 0x562f6a4cc000 2024-04-17T09:59:29.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.182+0000 7fcee9000640 1 -- 172.21.15.32:0/1096089581 <== osd.5 v2:172.21.15.144:6800/2379578648 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v83'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x562f6adea500 con 0x562f6a4cc000 2024-04-17T09:59:29.180 INFO:teuthology.orchestra.run.smithi032.stdout:writing EResetJournal entry 2024-04-17T09:59:29.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.182+0000 7fceeb96a400 1 -- 172.21.15.32:0/1096089581 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:6 12.1b 12:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x562f6a4ccc00 con 0x562f6a4e0000 2024-04-17T09:59:29.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.182+0000 7fceeb96a400 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x562f6a4cd000 0x562f6a4d2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.182+0000 7fceeb96a400 1 -- 172.21.15.32:0/1096089581 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:7 12.b 12:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x562f6a4cd400 con 0x562f6a4cd000 2024-04-17T09:59:29.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.182+0000 7fceeb96a400 1 -- 172.21.15.32:0/1096089581 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:8 12.1f 12:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x562f6a4cd800 con 0x562f6a4b1800 2024-04-17T09:59:29.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.182+0000 7fceeb96a400 1 -- 172.21.15.32:0/1096089581 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:9 12.1a 12:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x562f6a4e0800 con 0x562f6a4e0000 2024-04-17T09:59:29.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.182+0000 7fceeb96a400 1 -- 172.21.15.32:0/1096089581 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:10 12.1e 12:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x562f6a4e0c00 con 0x562f6a4e0000 2024-04-17T09:59:29.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.182+0000 7fceeb96a400 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x562f6a4e1000 0x562f6a4d2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.182+0000 7fceeb96a400 1 -- 172.21.15.32:0/1096089581 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:11 12.1d 12:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x562f6a4e1400 con 0x562f6a4e1000 2024-04-17T09:59:29.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.182+0000 7fcee9a00640 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x562f6a4cd000 0x562f6a4d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.182+0000 7fcee9000640 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x562f6a4e1000 0x562f6a4d2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.182+0000 7fcee9a00640 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x562f6a4cd000 0x562f6a4d2580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=4c2c40faaa0c4ca4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:29.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.186+0000 7fcee9000640 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x562f6a4e1000 0x562f6a4d2b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=48a5144cf91c965f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:29.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.186+0000 7fceea400640 1 -- 172.21.15.32:0/1096089581 <== osd.4 v2:172.21.15.144:6809/4076371964 2 ==== osd_op_reply(6 200.00000002 [delete] v83'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x562f6adea780 con 0x562f6a4e0000 2024-04-17T09:59:29.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.186+0000 7fcee9a00640 1 -- 172.21.15.32:0/1096089581 <== osd.6 v2:172.21.15.144:6808/3198738696 2 ==== osd_op_reply(8 200.00000004 [delete] v83'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x562f6adea000 con 0x562f6a4b1800 2024-04-17T09:59:29.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.186+0000 7fceea400640 1 -- 172.21.15.32:0/1096089581 <== osd.4 v2:172.21.15.144:6809/4076371964 3 ==== osd_op_reply(10 200.00000006 [delete] v83'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x562f6adea780 con 0x562f6a4e0000 2024-04-17T09:59:29.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.186+0000 7fceea400640 1 -- 172.21.15.32:0/1096089581 <== osd.4 v2:172.21.15.144:6809/4076371964 4 ==== osd_op_reply(9 200.00000005 [delete] v83'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x562f6adea780 con 0x562f6a4e0000 2024-04-17T09:59:29.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.186+0000 7fcee9000640 1 -- 172.21.15.32:0/1096089581 <== osd.0 v2:172.21.15.32:6800/2392056658 1 ==== osd_op_reply(11 200.00000007 [delete] v83'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x562f6adea500 con 0x562f6a4e1000 2024-04-17T09:59:29.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.186+0000 7fcee9a00640 1 -- 172.21.15.32:0/1096089581 <== osd.3 v2:172.21.15.32:6824/2182899298 1 ==== osd_op_reply(7 200.00000003 [delete] v83'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x562f6adea000 con 0x562f6a4cd000 2024-04-17T09:59:29.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.186+0000 7fcedc800640 1 -- 172.21.15.32:0/1096089581 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:12 12.1b 12:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x562f6a4b1c00 con 0x562f6a4e0000 2024-04-17T09:59:29.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceea400640 1 -- 172.21.15.32:0/1096089581 <== osd.4 v2:172.21.15.144:6809/4076371964 5 ==== osd_op_reply(12 200.00000002 [write 0~48] v83'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x562f6adea780 con 0x562f6a4e0000 2024-04-17T09:59:29.186 INFO:teuthology.orchestra.run.smithi032.stdout:done 2024-04-17T09:59:29.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 -- 172.21.15.32:0/1096089581 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x562f6a4e1000 msgr2=0x562f6a4d2b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x562f6a4e1000 0x562f6a4d2b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 -- 172.21.15.32:0/1096089581 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x562f6a4cd000 msgr2=0x562f6a4d2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x562f6a4cd000 0x562f6a4d2580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 -- 172.21.15.32:0/1096089581 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x562f6a4e0000 msgr2=0x562f6a4d3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x562f6a4e0000 0x562f6a4d3b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 -- 172.21.15.32:0/1096089581 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x562f6a4cc000 msgr2=0x562f6a4d2000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x562f6a4cc000 0x562f6a4d2000 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 -- 172.21.15.32:0/1096089581 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x562f6a4b1800 msgr2=0x562f6ae3d700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x562f6a4b1800 0x562f6ae3d700 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 -- 172.21.15.32:0/1096089581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6a36b400 msgr2=0x562f6ae3c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 --2- 172.21.15.32:0/1096089581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6a36b400 0x562f6ae3c680 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x562f6a3ff440 tx=0x562f6a142d80 comp rx=0 tx=0).stop 2024-04-17T09:59:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fcee9a00640 1 -- 172.21.15.32:0/1096089581 reap_dead start 2024-04-17T09:59:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fcee9a00640 1 -- 172.21.15.32:0/1096089581 reap_dead start 2024-04-17T09:59:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 -- 172.21.15.32:0/1096089581 shutdown_connections 2024-04-17T09:59:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fcee9a00640 1 -- 172.21.15.32:0/1096089581 reap_dead start 2024-04-17T09:59:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 -- 172.21.15.32:0/1096089581 >> 172.21.15.32:0/1096089581 conn(0x562f6a36b000 msgr2=0x562f6a49a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 -- 172.21.15.32:0/1096089581 shutdown_connections 2024-04-17T09:59:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 -- 172.21.15.32:0/1096089581 wait complete. 2024-04-17T09:59:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 -- 172.21.15.32:0/3024641523 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x562f6a36ac00 msgr2=0x562f6ae3c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 --2- 172.21.15.32:0/3024641523 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x562f6a36ac00 0x562f6ae3c100 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x562f6a15c0f0 tx=0x562f6a142300 comp rx=0 tx=0).stop 2024-04-17T09:59:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 -- 172.21.15.32:0/3024641523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6af97c00 msgr2=0x562f6ae3bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 --2- 172.21.15.32:0/3024641523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6af97c00 0x562f6ae3bb80 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x562f6af9f650 tx=0x562f6afa6e40 comp rx=0 tx=0).stop 2024-04-17T09:59:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 -- 172.21.15.32:0/3024641523 shutdown_connections 2024-04-17T09:59:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 --2- 172.21.15.32:0/3024641523 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x562f6a36ac00 0x562f6ae3c100 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 --2- 172.21.15.32:0/3024641523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6af97c00 0x562f6ae3bb80 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 --2- 172.21.15.32:0/3024641523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f6af97800 0x562f6ae3b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 --2- 172.21.15.32:0/3024641523 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562f6af97400 0x562f6ae3b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 -- 172.21.15.32:0/3024641523 >> 172.21.15.32:0/3024641523 conn(0x562f6af97000 msgr2=0x562f6a094000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 -- 172.21.15.32:0/3024641523 shutdown_connections 2024-04-17T09:59:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 -- 172.21.15.32:0/3024641523 wait complete. 2024-04-17T09:59:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 -- 172.21.15.32:0/3378279030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6a067000 msgr2=0x562f6ae3a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 --2- 172.21.15.32:0/3378279030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6a067000 0x562f6ae3a000 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x562f6aee3470 tx=0x562f6a142f60 comp rx=0 tx=0).stop 2024-04-17T09:59:29.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 -- 172.21.15.32:0/3378279030 shutdown_connections 2024-04-17T09:59:29.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 --2- 172.21.15.32:0/3378279030 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562f6a067800 0x562f6ae3ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 --2- 172.21.15.32:0/3378279030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f6a067400 0x562f6ae3a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 --2- 172.21.15.32:0/3378279030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562f6a067000 0x562f6ae3a000 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 -- 172.21.15.32:0/3378279030 >> 172.21.15.32:0/3378279030 conn(0x562f6a066c00 msgr2=0x562f6a093800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:29.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 -- 172.21.15.32:0/3378279030 shutdown_connections 2024-04-17T09:59:29.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.190+0000 7fceeb96a400 1 -- 172.21.15.32:0/3378279030 wait complete. 2024-04-17T09:59:29.194 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.124776, result: old journal was 4209834~892 new journal start will be 8388608 (4177882 bytes past old end) writing journal head writing EResetJournal entry done 2024-04-17T09:59:29.194 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-17T09:59:29.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.386+0000 7f0560000640 1 Processor -- start 2024-04-17T09:59:29.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.386+0000 7f0560000640 1 -- start start 2024-04-17T09:59:29.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.390+0000 7f0560000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05580cf370 0x7f05580cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.390+0000 7f0560000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05580d0130 con 0x7f0558071e60 2024-04-17T09:59:29.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.390+0000 7f0560000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f055815d090 con 0x7f055806ba30 2024-04-17T09:59:29.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.390+0000 7f0560000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f055815d370 con 0x7f05580cf370 2024-04-17T09:59:29.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.390+0000 7f055cc00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05580cf370 0x7f05580cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.390+0000 7f055e000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0558071e60 0x7f0558072270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52480/0 (socket says 172.21.15.32:52480) 2024-04-17T09:59:29.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.390+0000 7f055cc00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05580cf370 0x7f05580cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40190/0 (socket says 172.21.15.32:40190) 2024-04-17T09:59:29.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.390+0000 7f055e000640 1 -- 172.21.15.32:0/3667562200 learned_addr learned my addr 172.21.15.32:0/3667562200 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:29.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.390+0000 7f055cc00640 1 -- 172.21.15.32:0/3667562200 >> v1:172.21.15.144:6789/0 conn(0x7f055806ba30 legacy=0x7f055806be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:29.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.390+0000 7f055cc00640 1 -- 172.21.15.32:0/3667562200 >> v1:172.21.15.32:6789/0 conn(0x7f0558071e60 legacy=0x7f0558072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.390+0000 7f0557e00640 1 -- 172.21.15.32:0/3667562200 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2645352940 0 0) 0x7f05580d0130 con 0x7f0558071e60 2024-04-17T09:59:29.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.390+0000 7f055cc00640 1 -- 172.21.15.32:0/3667562200 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055815d650 con 0x7f05580cf370 2024-04-17T09:59:29.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.394+0000 7f055cc00640 1 --2- 172.21.15.32:0/3667562200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05580cf370 0x7f05580cf7e0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f05440097c0 tx=0x7f054401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840f1c1067bc9fa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:29.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.394+0000 7f0557e00640 1 -- 172.21.15.32:0/3667562200 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05440ab660 con 0x7f05580cf370 2024-04-17T09:59:29.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.394+0000 7f0557e00640 1 -- 172.21.15.32:0/3667562200 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05440b0070 con 0x7f05580cf370 2024-04-17T09:59:29.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.394+0000 7f0557e00640 1 -- 172.21.15.32:0/3667562200 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05440aba20 con 0x7f05580cf370 2024-04-17T09:59:29.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.394+0000 7f0560000640 1 -- 172.21.15.32:0/3667562200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05580cf370 msgr2=0x7f05580cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.394+0000 7f0560000640 1 --2- 172.21.15.32:0/3667562200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05580cf370 0x7f05580cf7e0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f05440097c0 tx=0x7f054401c460 comp rx=0 tx=0).stop 2024-04-17T09:59:29.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.394+0000 7f0560000640 1 -- 172.21.15.32:0/3667562200 shutdown_connections 2024-04-17T09:59:29.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.394+0000 7f0560000640 1 --2- 172.21.15.32:0/3667562200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05580cf370 0x7f05580cf7e0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.394+0000 7f0560000640 1 -- 172.21.15.32:0/3667562200 >> 172.21.15.32:0/3667562200 conn(0x7f05580b2120 msgr2=0x7f05580b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:29.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.394+0000 7f0560000640 1 -- 172.21.15.32:0/3667562200 shutdown_connections 2024-04-17T09:59:29.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.394+0000 7f0560000640 1 -- 172.21.15.32:0/3667562200 wait complete. 2024-04-17T09:59:29.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.394+0000 7f0560000640 1 Processor -- start 2024-04-17T09:59:29.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.394+0000 7f0560000640 1 -- start start 2024-04-17T09:59:29.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.394+0000 7f0560000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f055806ba30 0x7f055813d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.394+0000 7f0560000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0558071e60 0x7f05581f7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.394+0000 7f0560000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05580cf370 0x7f05580a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.394+0000 7f0560000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f055815dcf0 con 0x7f05580cf370 2024-04-17T09:59:29.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.394+0000 7f0560000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f055815db70 con 0x7f0558071e60 2024-04-17T09:59:29.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.394+0000 7f0560000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0558146230 con 0x7f055806ba30 2024-04-17T09:59:29.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.398+0000 7f055e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05580cf370 0x7f05580a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.398+0000 7f055e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05580cf370 0x7f05580a9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40198/0 (socket says 172.21.15.32:40198) 2024-04-17T09:59:29.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.398+0000 7f055e000640 1 -- 172.21.15.32:0/3723867706 learned_addr learned my addr 172.21.15.32:0/3723867706 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:29.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.398+0000 7f055cc00640 1 --2- 172.21.15.32:0/3723867706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0558071e60 0x7f05581f7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.398+0000 7f055d600640 1 --2- 172.21.15.32:0/3723867706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f055806ba30 0x7f055813d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.398+0000 7f055e000640 1 -- 172.21.15.32:0/3723867706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f055806ba30 msgr2=0x7f055813d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.398+0000 7f055e000640 1 --2- 172.21.15.32:0/3723867706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f055806ba30 0x7f055813d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.398+0000 7f055e000640 1 -- 172.21.15.32:0/3723867706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0558071e60 msgr2=0x7f05581f7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.398+0000 7f055e000640 1 --2- 172.21.15.32:0/3723867706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0558071e60 0x7f05581f7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.398+0000 7f055e000640 1 -- 172.21.15.32:0/3723867706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05581574f0 con 0x7f05580cf370 2024-04-17T09:59:29.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.398+0000 7f055e000640 1 --2- 172.21.15.32:0/3723867706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05580cf370 0x7f05580a9450 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f055000c7d0 tx=0x7f055000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:29.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.398+0000 7f0555600640 1 -- 172.21.15.32:0/3723867706 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550018070 con 0x7f05580cf370 2024-04-17T09:59:29.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.398+0000 7f0555600640 1 -- 172.21.15.32:0/3723867706 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f055000a640 con 0x7f05580cf370 2024-04-17T09:59:29.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.398+0000 7f0560000640 1 -- 172.21.15.32:0/3723867706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05581577d0 con 0x7f05580cf370 2024-04-17T09:59:29.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.398+0000 7f0555600640 1 -- 172.21.15.32:0/3723867706 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055000aa70 con 0x7f05580cf370 2024-04-17T09:59:29.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.398+0000 7f0560000640 1 -- 172.21.15.32:0/3723867706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0558157d90 con 0x7f05580cf370 2024-04-17T09:59:29.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.398+0000 7f0560000640 1 -- 172.21.15.32:0/3723867706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0558068a90 con 0x7f05580cf370 2024-04-17T09:59:29.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.398+0000 7f0555600640 1 -- 172.21.15.32:0/3723867706 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f055001b070 con 0x7f05580cf370 2024-04-17T09:59:29.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.402+0000 7f0555600640 1 --2- 172.21.15.32:0/3723867706 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0538068380 0x7f053806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:29.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.402+0000 7f0555600640 1 -- 172.21.15.32:0/3723867706 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f0550085ff0 con 0x7f05580cf370 2024-04-17T09:59:29.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.402+0000 7f055d600640 1 --2- 172.21.15.32:0/3723867706 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0538068380 0x7f053806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:29.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.402+0000 7f055d600640 1 --2- 172.21.15.32:0/3723867706 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0538068380 0x7f053806a840 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f054c004e30 tx=0x7f054c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:29.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.406+0000 7f0555600640 1 -- 172.21.15.32:0/3723867706 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0550055e80 con 0x7f05580cf370 2024-04-17T09:59:29.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.610+0000 7f0560000640 1 -- 172.21.15.32:0/3723867706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f055806c850 con 0x7f05580cf370 2024-04-17T09:59:29.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.990+0000 7f0555600640 1 -- 172.21.15.32:0/3723867706 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v61) ==== 162+0+0 (secure 0 0 0) 0x7f055005a180 con 0x7f05580cf370 2024-04-17T09:59:29.989 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-17T09:59:29.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.994+0000 7f0560000640 1 -- 172.21.15.32:0/3723867706 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0538068380 msgr2=0x7f053806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.994+0000 7f0560000640 1 --2- 172.21.15.32:0/3723867706 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0538068380 0x7f053806a840 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f054c004e30 tx=0x7f054c009040 comp rx=0 tx=0).stop 2024-04-17T09:59:29.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.994+0000 7f0560000640 1 -- 172.21.15.32:0/3723867706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05580cf370 msgr2=0x7f05580a9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:29.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.994+0000 7f0560000640 1 --2- 172.21.15.32:0/3723867706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05580cf370 0x7f05580a9450 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f055000c7d0 tx=0x7f055000cc90 comp rx=0 tx=0).stop 2024-04-17T09:59:29.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.994+0000 7f0560000640 1 -- 172.21.15.32:0/3723867706 shutdown_connections 2024-04-17T09:59:29.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.994+0000 7f0560000640 1 --2- 172.21.15.32:0/3723867706 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0538068380 0x7f053806a840 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.994+0000 7f0560000640 1 --2- 172.21.15.32:0/3723867706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05580cf370 0x7f05580a9450 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.994+0000 7f0560000640 1 --2- 172.21.15.32:0/3723867706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0558071e60 0x7f05581f7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.994+0000 7f0560000640 1 --2- 172.21.15.32:0/3723867706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f055806ba30 0x7f055813d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:29.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.994+0000 7f0560000640 1 -- 172.21.15.32:0/3723867706 >> 172.21.15.32:0/3723867706 conn(0x7f05580b2120 msgr2=0x7f055806be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:29.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.994+0000 7f0560000640 1 -- 172.21.15.32:0/3723867706 shutdown_connections 2024-04-17T09:59:29.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:29.994+0000 7f0560000640 1 -- 172.21.15.32:0/3723867706 wait complete. 2024-04-17T09:59:30.017 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:59:30.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.162+0000 7f325ac00640 1 Processor -- start 2024-04-17T09:59:30.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.162+0000 7f325ac00640 1 -- start start 2024-04-17T09:59:30.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.162+0000 7f325ac00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3254145f30 0x7f3254071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:30.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.162+0000 7f325ac00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32540cd580 0x7f3254072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:30.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.162+0000 7f325ac00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32540cdac0 con 0x7f325415d8d0 2024-04-17T09:59:30.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.162+0000 7f325ac00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f325406f400 con 0x7f3254145f30 2024-04-17T09:59:30.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.162+0000 7f325ac00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3254146310 con 0x7f32540cd580 2024-04-17T09:59:30.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.162+0000 7f3253e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3254145f30 0x7f3254071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:30.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.162+0000 7f324b400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f325415d8d0 0x7f32540cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52482/0 (socket says 172.21.15.32:52482) 2024-04-17T09:59:30.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.162+0000 7f324b400640 1 -- 172.21.15.32:0/1893044495 learned_addr learned my addr 172.21.15.32:0/1893044495 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:30.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.162+0000 7f3258c00640 1 --2- 172.21.15.32:0/1893044495 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32540cd580 0x7f3254072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:30.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.162+0000 7f3253e00640 1 -- 172.21.15.32:0/1893044495 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32540cd580 msgr2=0x7f3254072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:30.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.162+0000 7f3253e00640 1 --2- 172.21.15.32:0/1893044495 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32540cd580 0x7f3254072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:30.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.162+0000 7f3253e00640 1 -- 172.21.15.32:0/1893044495 >> v1:172.21.15.32:6789/0 conn(0x7f325415d8d0 legacy=0x7f32540cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:30.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.162+0000 7f3253e00640 1 -- 172.21.15.32:0/1893044495 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f325415cdf0 con 0x7f3254145f30 2024-04-17T09:59:30.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.166+0000 7f3253e00640 1 --2- 172.21.15.32:0/1893044495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3254145f30 0x7f3254071df0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f32440097c0 tx=0x7f324401ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a3d7a8c0ba3afe server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:30.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.166+0000 7f3253400640 1 -- 172.21.15.32:0/1893044495 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32440b5070 con 0x7f3254145f30 2024-04-17T09:59:30.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.166+0000 7f3253400640 1 -- 172.21.15.32:0/1893044495 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32440a8990 con 0x7f3254145f30 2024-04-17T09:59:30.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.166+0000 7f3253400640 1 -- 172.21.15.32:0/1893044495 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32440a8d60 con 0x7f3254145f30 2024-04-17T09:59:30.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.166+0000 7f325ac00640 1 -- 172.21.15.32:0/1893044495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3254145f30 msgr2=0x7f3254071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:30.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.166+0000 7f325ac00640 1 --2- 172.21.15.32:0/1893044495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3254145f30 0x7f3254071df0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f32440097c0 tx=0x7f324401ec90 comp rx=0 tx=0).stop 2024-04-17T09:59:30.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.166+0000 7f325ac00640 1 -- 172.21.15.32:0/1893044495 shutdown_connections 2024-04-17T09:59:30.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.166+0000 7f325ac00640 1 --2- 172.21.15.32:0/1893044495 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32540cd580 0x7f3254072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:30.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.166+0000 7f325ac00640 1 --2- 172.21.15.32:0/1893044495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3254145f30 0x7f3254071df0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:30.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.166+0000 7f325ac00640 1 -- 172.21.15.32:0/1893044495 >> 172.21.15.32:0/1893044495 conn(0x7f325406ba30 msgr2=0x7f325406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:30.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.166+0000 7f325ac00640 1 -- 172.21.15.32:0/1893044495 shutdown_connections 2024-04-17T09:59:30.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.166+0000 7f325ac00640 1 -- 172.21.15.32:0/1893044495 wait complete. 2024-04-17T09:59:30.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f325ac00640 1 Processor -- start 2024-04-17T09:59:30.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f325ac00640 1 -- start start 2024-04-17T09:59:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f325ac00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32540cd580 0x7f325413dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f325ac00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3254145f30 0x7f32540b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f325ac00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f325415d8d0 0x7f32541408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f325ac00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f325415cb90 con 0x7f325415d8d0 2024-04-17T09:59:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f325ac00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f325415d440 con 0x7f3254145f30 2024-04-17T09:59:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f325ac00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f32541601d0 con 0x7f32540cd580 2024-04-17T09:59:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f3258c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f325415d8d0 0x7f32541408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f3258c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f325415d8d0 0x7f32541408e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40216/0 (socket says 172.21.15.32:40216) 2024-04-17T09:59:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f3258c00640 1 -- 172.21.15.32:0/2759026245 learned_addr learned my addr 172.21.15.32:0/2759026245 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f3253e00640 1 --2- 172.21.15.32:0/2759026245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32540cd580 0x7f325413dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f324b400640 1 --2- 172.21.15.32:0/2759026245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3254145f30 0x7f32540b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f3258c00640 1 -- 172.21.15.32:0/2759026245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32540cd580 msgr2=0x7f325413dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f3258c00640 1 --2- 172.21.15.32:0/2759026245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32540cd580 0x7f325413dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f3258c00640 1 -- 172.21.15.32:0/2759026245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3254145f30 msgr2=0x7f32540b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f3258c00640 1 --2- 172.21.15.32:0/2759026245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3254145f30 0x7f32540b5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f3258c00640 1 -- 172.21.15.32:0/2759026245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32540c9f40 con 0x7f325415d8d0 2024-04-17T09:59:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f324b400640 1 --2- 172.21.15.32:0/2759026245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3254145f30 0x7f32540b5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f3258c00640 1 --2- 172.21.15.32:0/2759026245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f325415d8d0 0x7f32541408e0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f324c007e60 tx=0x7f324c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f3250c00640 1 -- 172.21.15.32:0/2759026245 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c014620 con 0x7f325415d8d0 2024-04-17T09:59:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f3250c00640 1 -- 172.21.15.32:0/2759026245 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f324c019070 con 0x7f325415d8d0 2024-04-17T09:59:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f325ac00640 1 -- 172.21.15.32:0/2759026245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32540ca220 con 0x7f325415d8d0 2024-04-17T09:59:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f3250c00640 1 -- 172.21.15.32:0/2759026245 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c014a00 con 0x7f325415d8d0 2024-04-17T09:59:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.170+0000 7f325ac00640 1 -- 172.21.15.32:0/2759026245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32540ca7e0 con 0x7f325415d8d0 2024-04-17T09:59:30.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.174+0000 7f325ac00640 1 -- 172.21.15.32:0/2759026245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3254068a90 con 0x7f325415d8d0 2024-04-17T09:59:30.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.174+0000 7f3250c00640 1 -- 172.21.15.32:0/2759026245 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f324c014ba0 con 0x7f325415d8d0 2024-04-17T09:59:30.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.174+0000 7f3250c00640 1 --2- 172.21.15.32:0/2759026245 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3230068380 0x7f323006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:30.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.174+0000 7f3250c00640 1 -- 172.21.15.32:0/2759026245 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f324c0872a0 con 0x7f325415d8d0 2024-04-17T09:59:30.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.174+0000 7f3253e00640 1 --2- 172.21.15.32:0/2759026245 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3230068380 0x7f323006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:30.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.174+0000 7f3253e00640 1 --2- 172.21.15.32:0/2759026245 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3230068380 0x7f323006a840 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f32440b2000 tx=0x7f3244008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:30.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.178+0000 7f3250c00640 1 -- 172.21.15.32:0/2759026245 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f324c0577f0 con 0x7f325415d8d0 2024-04-17T09:59:30.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.378+0000 7f325ac00640 1 -- 172.21.15.32:0/2759026245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3254072f90 con 0x7f325415d8d0 2024-04-17T09:59:30.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.378+0000 7f3250c00640 1 -- 172.21.15.32:0/2759026245 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5198 (secure 0 0 0) 0x7f324c0570f0 con 0x7f325415d8d0 2024-04-17T09:59:30.376 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:59:30.377 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6419,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3534108103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3534108103},{"type":"v1","addr":"172.21.15.144:6835","nonce":3534108103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7003,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/935945019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":935945019},{"type":"v1","addr":"172.21.15.144:6834","nonce":935945019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3044362393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3044362393},{"type":"v1","addr":"172.21.15.32:6836","nonce":3044362393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:59:04.834110+0000","modified":"2024-04-17T09:59:29.995667+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":62,"state":"up:replay","state_seq":1,"addr":"172.21.15.32:6837/1253497636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1253497636},{"type":"v1","addr":"172.21.15.32:6837","nonce":1253497636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-17T09:59:30.377 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 62 2024-04-17T09:59:30.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.382+0000 7f325ac00640 1 -- 172.21.15.32:0/2759026245 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3230068380 msgr2=0x7f323006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:30.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.382+0000 7f325ac00640 1 --2- 172.21.15.32:0/2759026245 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3230068380 0x7f323006a840 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f32440b2000 tx=0x7f3244008040 comp rx=0 tx=0).stop 2024-04-17T09:59:30.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.382+0000 7f325ac00640 1 -- 172.21.15.32:0/2759026245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f325415d8d0 msgr2=0x7f32541408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:30.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.382+0000 7f325ac00640 1 --2- 172.21.15.32:0/2759026245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f325415d8d0 0x7f32541408e0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f324c007e60 tx=0x7f324c00e4b0 comp rx=0 tx=0).stop 2024-04-17T09:59:30.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.382+0000 7f325ac00640 1 -- 172.21.15.32:0/2759026245 shutdown_connections 2024-04-17T09:59:30.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.382+0000 7f325ac00640 1 --2- 172.21.15.32:0/2759026245 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3230068380 0x7f323006a840 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:30.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.382+0000 7f325ac00640 1 --2- 172.21.15.32:0/2759026245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f325415d8d0 0x7f32541408e0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:30.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.382+0000 7f325ac00640 1 --2- 172.21.15.32:0/2759026245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3254145f30 0x7f32540b5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:30.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.382+0000 7f325ac00640 1 --2- 172.21.15.32:0/2759026245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32540cd580 0x7f325413dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:30.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.382+0000 7f325ac00640 1 -- 172.21.15.32:0/2759026245 >> 172.21.15.32:0/2759026245 conn(0x7f325406ba30 msgr2=0x7f32540cce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:30.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.382+0000 7f325ac00640 1 -- 172.21.15.32:0/2759026245 shutdown_connections 2024-04-17T09:59:30.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:30.382+0000 7f325ac00640 1 -- 172.21.15.32:0/2759026245 wait complete. 2024-04-17T09:59:30.802 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T09:59:04.834110+0000', 'modified': '2024-04-17T09:59:29.995667+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7851}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7851': {'gid': 7851, 'name': 'c', 'rank': 0, 'incarnation': 62, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.32:6837/1253497636', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6835', 'nonce': 1253497636}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 1253497636}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-17T09:59:30.802 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7851:up:replay 2024-04-17T09:59:31.803 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:59:31.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.958+0000 7f9a11400640 1 Processor -- start 2024-04-17T09:59:31.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.958+0000 7f9a11400640 1 -- start start 2024-04-17T09:59:31.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.958+0000 7f9a11400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a0c144f20 0x7f9a0c06e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:31.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.958+0000 7f9a11400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a0c06eda0 0x7f9a0c071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:31.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.958+0000 7f9a11400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9a0c1481e0 0x7f9a0c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:31.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.958+0000 7f9a11400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9a0c0aed20 con 0x7f9a0c144f20 2024-04-17T09:59:31.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.958+0000 7f9a11400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f9a0c0af000 con 0x7f9a0c1481e0 2024-04-17T09:59:31.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.958+0000 7f9a11400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9a0c0af2e0 con 0x7f9a0c06eda0 2024-04-17T09:59:31.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.962+0000 7f9a0a800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a0c144f20 0x7f9a0c06e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:31.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.962+0000 7f9a0a800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a0c144f20 0x7f9a0c06e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42702/0 (socket says 172.21.15.32:42702) 2024-04-17T09:59:31.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.962+0000 7f9a0a800640 1 -- 172.21.15.32:0/2836020691 learned_addr learned my addr 172.21.15.32:0/2836020691 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:31.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.962+0000 7f9a01e00640 1 --2- 172.21.15.32:0/2836020691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a0c06eda0 0x7f9a0c071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:31.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.962+0000 7f9a0b200640 1 --2- 172.21.15.32:0/2836020691 >> v2:172.21.15.144:3300/0 conn(0x7f9a0c1481e0 0x7f9a0c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:31.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.962+0000 7f9a0a800640 1 -- 172.21.15.32:0/2836020691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a0c06eda0 msgr2=0x7f9a0c071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:31.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.962+0000 7f9a0a800640 1 --2- 172.21.15.32:0/2836020691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a0c06eda0 0x7f9a0c071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:31.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.962+0000 7f9a0a800640 1 -- 172.21.15.32:0/2836020691 >> v2:172.21.15.144:3300/0 conn(0x7f9a0c1481e0 msgr2=0x7f9a0c072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:31.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.962+0000 7f9a0a800640 1 --2- 172.21.15.32:0/2836020691 >> v2:172.21.15.144:3300/0 conn(0x7f9a0c1481e0 0x7f9a0c072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:31.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.962+0000 7f9a0a800640 1 -- 172.21.15.32:0/2836020691 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0c0af5c0 con 0x7f9a0c144f20 2024-04-17T09:59:31.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.962+0000 7f9a0a800640 1 --2- 172.21.15.32:0/2836020691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0c144f20 0x7f9a0c06e860 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0cc910 tx=0x7f99fc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c29ff5f21d20ea6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:31.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.962+0000 7f9a09e00640 1 -- 172.21.15.32:0/2836020691 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0b0070 con 0x7f9a0c144f20 2024-04-17T09:59:31.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.962+0000 7f9a09e00640 1 -- 172.21.15.32:0/2836020691 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99fc0ab600 con 0x7f9a0c144f20 2024-04-17T09:59:31.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.962+0000 7f9a09e00640 1 -- 172.21.15.32:0/2836020691 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0ab9d0 con 0x7f9a0c144f20 2024-04-17T09:59:31.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a11400640 1 -- 172.21.15.32:0/2836020691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0c144f20 msgr2=0x7f9a0c06e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:31.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a11400640 1 --2- 172.21.15.32:0/2836020691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0c144f20 0x7f9a0c06e860 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0cc910 tx=0x7f99fc01c520 comp rx=0 tx=0).stop 2024-04-17T09:59:31.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a11400640 1 -- 172.21.15.32:0/2836020691 shutdown_connections 2024-04-17T09:59:31.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a11400640 1 --2- 172.21.15.32:0/2836020691 >> v2:172.21.15.144:3300/0 conn(0x7f9a0c1481e0 0x7f9a0c072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:31.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a11400640 1 --2- 172.21.15.32:0/2836020691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a0c06eda0 0x7f9a0c071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:31.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a11400640 1 --2- 172.21.15.32:0/2836020691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0c144f20 0x7f9a0c06e860 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:31.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a11400640 1 -- 172.21.15.32:0/2836020691 >> 172.21.15.32:0/2836020691 conn(0x7f9a0c06ba30 msgr2=0x7f9a0c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:31.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a11400640 1 -- 172.21.15.32:0/2836020691 shutdown_connections 2024-04-17T09:59:31.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a11400640 1 -- 172.21.15.32:0/2836020691 wait complete. 2024-04-17T09:59:31.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a11400640 1 Processor -- start 2024-04-17T09:59:31.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a11400640 1 -- start start 2024-04-17T09:59:31.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a11400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0c06eda0 0x7f9a0c15d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:31.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a11400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0c144f20 0x7f9a0c0cef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:31.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a11400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a0c1481e0 0x7f9a0c13cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:31.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a11400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c162fe0 con 0x7f9a0c06eda0 2024-04-17T09:59:31.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a11400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c162e60 con 0x7f9a0c144f20 2024-04-17T09:59:31.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a11400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9a0c163160 con 0x7f9a0c1481e0 2024-04-17T09:59:31.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a0a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0c06eda0 0x7f9a0c15d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:31.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a0a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0c06eda0 0x7f9a0c15d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42706/0 (socket says 172.21.15.32:42706) 2024-04-17T09:59:31.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a0a800640 1 -- 172.21.15.32:0/1991536820 learned_addr learned my addr 172.21.15.32:0/1991536820 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:31.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a01e00640 1 --2- 172.21.15.32:0/1991536820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0c144f20 0x7f9a0c0cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:31.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a0b200640 1 --2- 172.21.15.32:0/1991536820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a0c1481e0 0x7f9a0c13cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:31.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a0a800640 1 -- 172.21.15.32:0/1991536820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a0c1481e0 msgr2=0x7f9a0c13cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:31.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a0a800640 1 --2- 172.21.15.32:0/1991536820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a0c1481e0 0x7f9a0c13cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:31.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a0a800640 1 -- 172.21.15.32:0/1991536820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0c144f20 msgr2=0x7f9a0c0cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:31.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a0a800640 1 --2- 172.21.15.32:0/1991536820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0c144f20 0x7f9a0c0cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:31.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a0a800640 1 -- 172.21.15.32:0/1991536820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0c15e370 con 0x7f9a0c06eda0 2024-04-17T09:59:31.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.966+0000 7f9a01e00640 1 --2- 172.21.15.32:0/1991536820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0c144f20 0x7f9a0c0cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:31.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.970+0000 7f9a0a800640 1 --2- 172.21.15.32:0/1991536820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0c06eda0 0x7f9a0c15d000 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f99fc01abd0 tx=0x7f99fc0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:31.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.970+0000 7f9a03400640 1 -- 172.21.15.32:0/1991536820 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0b9070 con 0x7f9a0c06eda0 2024-04-17T09:59:31.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.970+0000 7f9a03400640 1 -- 172.21.15.32:0/1991536820 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99fc0b4400 con 0x7f9a0c06eda0 2024-04-17T09:59:31.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.970+0000 7f9a03400640 1 -- 172.21.15.32:0/1991536820 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0b0040 con 0x7f9a0c06eda0 2024-04-17T09:59:31.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.970+0000 7f9a11400640 1 -- 172.21.15.32:0/1991536820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a0c15d540 con 0x7f9a0c06eda0 2024-04-17T09:59:31.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.970+0000 7f9a11400640 1 -- 172.21.15.32:0/1991536820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a0c15ddb0 con 0x7f9a0c06eda0 2024-04-17T09:59:31.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.970+0000 7f9a11400640 1 -- 172.21.15.32:0/1991536820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a0c15d850 con 0x7f9a0c06eda0 2024-04-17T09:59:31.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.970+0000 7f9a03400640 1 -- 172.21.15.32:0/1991536820 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f99fc0b33a0 con 0x7f9a0c06eda0 2024-04-17T09:59:31.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.970+0000 7f9a03400640 1 --2- 172.21.15.32:0/1991536820 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f99e80683b0 0x7f99e806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:31.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.970+0000 7f9a03400640 1 -- 172.21.15.32:0/1991536820 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f99fc11fc40 con 0x7f9a0c06eda0 2024-04-17T09:59:31.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.970+0000 7f9a01e00640 1 --2- 172.21.15.32:0/1991536820 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f99e80683b0 0x7f99e806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:31.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.974+0000 7f9a01e00640 1 --2- 172.21.15.32:0/1991536820 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f99e80683b0 0x7f99e806a870 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f99f4004660 tx=0x7f99f4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:31.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:31.974+0000 7f9a03400640 1 -- 172.21.15.32:0/1991536820 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f9a0c15d850 con 0x7f9a0c06eda0 2024-04-17T09:59:32.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:32.174+0000 7f9a11400640 1 -- 172.21.15.32:0/1991536820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a0c068a90 con 0x7f9a0c06eda0 2024-04-17T09:59:32.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:32.178+0000 7f9a03400640 1 -- 172.21.15.32:0/1991536820 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5198 (secure 0 0 0) 0x7f99fc016110 con 0x7f9a0c06eda0 2024-04-17T09:59:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:59:32.174 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6419,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3534108103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3534108103},{"type":"v1","addr":"172.21.15.144:6835","nonce":3534108103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7003,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/935945019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":935945019},{"type":"v1","addr":"172.21.15.144:6834","nonce":935945019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3044362393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3044362393},{"type":"v1","addr":"172.21.15.32:6836","nonce":3044362393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:59:04.834110+0000","modified":"2024-04-17T09:59:31.002108+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":62,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.32:6837/1253497636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1253497636},{"type":"v1","addr":"172.21.15.32:6837","nonce":1253497636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-17T09:59:32.174 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 64 2024-04-17T09:59:32.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:32.178+0000 7f9a11400640 1 -- 172.21.15.32:0/1991536820 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f99e80683b0 msgr2=0x7f99e806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:32.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:32.178+0000 7f9a11400640 1 --2- 172.21.15.32:0/1991536820 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f99e80683b0 0x7f99e806a870 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f99f4004660 tx=0x7f99f4008040 comp rx=0 tx=0).stop 2024-04-17T09:59:32.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:32.178+0000 7f9a11400640 1 -- 172.21.15.32:0/1991536820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0c06eda0 msgr2=0x7f9a0c15d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:32.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:32.178+0000 7f9a11400640 1 --2- 172.21.15.32:0/1991536820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0c06eda0 0x7f9a0c15d000 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f99fc01abd0 tx=0x7f99fc0192c0 comp rx=0 tx=0).stop 2024-04-17T09:59:32.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:32.178+0000 7f9a11400640 1 -- 172.21.15.32:0/1991536820 shutdown_connections 2024-04-17T09:59:32.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:32.178+0000 7f9a11400640 1 --2- 172.21.15.32:0/1991536820 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f99e80683b0 0x7f99e806a870 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:32.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:32.178+0000 7f9a11400640 1 --2- 172.21.15.32:0/1991536820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a0c1481e0 0x7f9a0c13cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:32.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:32.178+0000 7f9a11400640 1 --2- 172.21.15.32:0/1991536820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0c144f20 0x7f9a0c0cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:32.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:32.178+0000 7f9a11400640 1 --2- 172.21.15.32:0/1991536820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0c06eda0 0x7f9a0c15d000 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:32.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:32.178+0000 7f9a11400640 1 -- 172.21.15.32:0/1991536820 >> 172.21.15.32:0/1991536820 conn(0x7f9a0c06ba30 msgr2=0x7f9a0c0b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:32.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:32.178+0000 7f9a11400640 1 -- 172.21.15.32:0/1991536820 shutdown_connections 2024-04-17T09:59:32.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:32.178+0000 7f9a11400640 1 -- 172.21.15.32:0/1991536820 wait complete. 2024-04-17T09:59:32.600 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T09:59:04.834110+0000', 'modified': '2024-04-17T09:59:31.002108+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7851}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7851': {'gid': 7851, 'name': 'c', 'rank': 0, 'incarnation': 62, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.32:6837/1253497636', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6835', 'nonce': 1253497636}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 1253497636}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-17T09:59:32.600 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7851:up:rejoin 2024-04-17T09:59:33.602 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T09:59:33.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.754+0000 7f07e8e00640 1 Processor -- start 2024-04-17T09:59:33.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.754+0000 7f07e8e00640 1 -- start start 2024-04-17T09:59:33.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.754+0000 7f07e8e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07e4072500 0x7f07e40cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:33.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.754+0000 7f07e8e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07e40cdac0 con 0x7f07e415d8d0 2024-04-17T09:59:33.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.754+0000 7f07e8e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07e415cdf0 con 0x7f07e4145f30 2024-04-17T09:59:33.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.754+0000 7f07e8e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f07e406f400 con 0x7f07e4072500 2024-04-17T09:59:33.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.754+0000 7f07e2800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f07e415d8d0 0x7f07e41601d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45674/0 (socket says 172.21.15.32:45674) 2024-04-17T09:59:33.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.754+0000 7f07e2800640 1 -- 172.21.15.32:0/789510819 learned_addr learned my addr 172.21.15.32:0/789510819 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:33.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.758+0000 7f07dbe00640 1 --2- 172.21.15.32:0/789510819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07e4072500 0x7f07e40cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:33.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.758+0000 7f07e1400640 1 -- 172.21.15.32:0/789510819 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3288097260 0 0) 0x7f07e415cdf0 con 0x7f07e4145f30 2024-04-17T09:59:33.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.758+0000 7f07e1400640 1 -- 172.21.15.32:0/789510819 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07c8003530 con 0x7f07e4145f30 2024-04-17T09:59:33.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.758+0000 7f07e1400640 1 -- 172.21.15.32:0/789510819 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2648850348 0 0) 0x7f07e40cdac0 con 0x7f07e415d8d0 2024-04-17T09:59:33.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.758+0000 7f07e1400640 1 -- 172.21.15.32:0/789510819 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07e415cdf0 con 0x7f07e415d8d0 2024-04-17T09:59:33.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.758+0000 7f07dbe00640 1 -- 172.21.15.32:0/789510819 >> v1:172.21.15.144:6789/0 conn(0x7f07e4145f30 legacy=0x7f07e4071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:33.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.758+0000 7f07dbe00640 1 -- 172.21.15.32:0/789510819 >> v1:172.21.15.32:6789/0 conn(0x7f07e415d8d0 legacy=0x7f07e41601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:33.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.758+0000 7f07dbe00640 1 -- 172.21.15.32:0/789510819 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e415d0d0 con 0x7f07e4072500 2024-04-17T09:59:33.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.758+0000 7f07dbe00640 1 --2- 172.21.15.32:0/789510819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07e4072500 0x7f07e40cce70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0097f0 tx=0x7f07cc01ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=413340a212e4de4e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:33.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.758+0000 7f07e1400640 1 -- 172.21.15.32:0/789510819 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc0b5070 con 0x7f07e4072500 2024-04-17T09:59:33.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.758+0000 7f07e1400640 1 -- 172.21.15.32:0/789510819 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07cc0a89d0 con 0x7f07e4072500 2024-04-17T09:59:33.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.758+0000 7f07e1400640 1 -- 172.21.15.32:0/789510819 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc0a8da0 con 0x7f07e4072500 2024-04-17T09:59:33.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07e8e00640 1 -- 172.21.15.32:0/789510819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07e4072500 msgr2=0x7f07e40cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:33.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07e8e00640 1 --2- 172.21.15.32:0/789510819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07e4072500 0x7f07e40cce70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0097f0 tx=0x7f07cc01ecd0 comp rx=0 tx=0).stop 2024-04-17T09:59:33.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07e8e00640 1 -- 172.21.15.32:0/789510819 shutdown_connections 2024-04-17T09:59:33.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07e8e00640 1 --2- 172.21.15.32:0/789510819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07e4072500 0x7f07e40cce70 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:33.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07e8e00640 1 -- 172.21.15.32:0/789510819 >> 172.21.15.32:0/789510819 conn(0x7f07e406ba30 msgr2=0x7f07e406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:33.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07e8e00640 1 -- 172.21.15.32:0/789510819 shutdown_connections 2024-04-17T09:59:33.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07e8e00640 1 -- 172.21.15.32:0/789510819 wait complete. 2024-04-17T09:59:33.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07e8e00640 1 Processor -- start 2024-04-17T09:59:33.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07e8e00640 1 -- start start 2024-04-17T09:59:33.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07e8e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07e4072500 0x7f07e413d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:33.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07e8e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e4145f30 0x7f07e415ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:33.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07e8e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07e415d8d0 0x7f07e415b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:33.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07e8e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f07e40cd7c0 con 0x7f07e4145f30 2024-04-17T09:59:33.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07e8e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f07e40cd640 con 0x7f07e4072500 2024-04-17T09:59:33.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07e8e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f07e4146310 con 0x7f07e415d8d0 2024-04-17T09:59:33.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07dbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e4145f30 0x7f07e415ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:33.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07dbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e4145f30 0x7f07e415ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42722/0 (socket says 172.21.15.32:42722) 2024-04-17T09:59:33.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07dbe00640 1 -- 172.21.15.32:0/2075567990 learned_addr learned my addr 172.21.15.32:0/2075567990 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:33.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07e2800640 1 --2- 172.21.15.32:0/2075567990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07e415d8d0 0x7f07e415b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:33.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07e1e00640 1 --2- 172.21.15.32:0/2075567990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07e4072500 0x7f07e413d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:33.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07dbe00640 1 -- 172.21.15.32:0/2075567990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07e415d8d0 msgr2=0x7f07e415b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:33.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07dbe00640 1 --2- 172.21.15.32:0/2075567990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07e415d8d0 0x7f07e415b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:33.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07dbe00640 1 -- 172.21.15.32:0/2075567990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07e4072500 msgr2=0x7f07e413d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:33.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07dbe00640 1 --2- 172.21.15.32:0/2075567990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07e4072500 0x7f07e413d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:33.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07dbe00640 1 -- 172.21.15.32:0/2075567990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e413e310 con 0x7f07e4145f30 2024-04-17T09:59:33.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.762+0000 7f07e1e00640 1 --2- 172.21.15.32:0/2075567990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07e4072500 0x7f07e413d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:59:33.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.766+0000 7f07dbe00640 1 --2- 172.21.15.32:0/2075567990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e4145f30 0x7f07e415ae80 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f07cc01b8e0 tx=0x7f07cc01ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:33.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.766+0000 7f07da000640 1 -- 172.21.15.32:0/2075567990 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc0bf070 con 0x7f07e4145f30 2024-04-17T09:59:33.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.766+0000 7f07da000640 1 -- 172.21.15.32:0/2075567990 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07cc0b0720 con 0x7f07e4145f30 2024-04-17T09:59:33.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.766+0000 7f07da000640 1 -- 172.21.15.32:0/2075567990 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc0b5040 con 0x7f07e4145f30 2024-04-17T09:59:33.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.766+0000 7f07e8e00640 1 -- 172.21.15.32:0/2075567990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e41e61d0 con 0x7f07e4145f30 2024-04-17T09:59:33.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.766+0000 7f07e8e00640 1 -- 172.21.15.32:0/2075567990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e41e6790 con 0x7f07e4145f30 2024-04-17T09:59:33.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.766+0000 7f07e8e00640 1 -- 172.21.15.32:0/2075567990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e4068a90 con 0x7f07e4145f30 2024-04-17T09:59:33.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.766+0000 7f07da000640 1 -- 172.21.15.32:0/2075567990 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f07cc0b0b10 con 0x7f07e4145f30 2024-04-17T09:59:33.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.766+0000 7f07da000640 1 --2- 172.21.15.32:0/2075567990 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07c0068380 0x7f07c006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:33.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.766+0000 7f07da000640 1 -- 172.21.15.32:0/2075567990 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f07cc1232e0 con 0x7f07e4145f30 2024-04-17T09:59:33.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.766+0000 7f07e1e00640 1 --2- 172.21.15.32:0/2075567990 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07c0068380 0x7f07c006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:33.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.770+0000 7f07e1e00640 1 --2- 172.21.15.32:0/2075567990 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07c0068380 0x7f07c006a840 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f07d40011f0 tx=0x7f07d4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:33.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.774+0000 7f07da000640 1 -- 172.21.15.32:0/2075567990 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f07e413e310 con 0x7f07e4145f30 2024-04-17T09:59:33.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.974+0000 7f07e8e00640 1 -- 172.21.15.32:0/2075567990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07e4072f90 con 0x7f07e4145f30 2024-04-17T09:59:33.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.974+0000 7f07da000640 1 -- 172.21.15.32:0/2075567990 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5205 (secure 0 0 0) 0x7f07cc018830 con 0x7f07e4145f30 2024-04-17T09:59:33.971 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T09:59:33.971 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6419,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3534108103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3534108103},{"type":"v1","addr":"172.21.15.144:6835","nonce":3534108103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7003,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/935945019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":935945019},{"type":"v1","addr":"172.21.15.144:6834","nonce":935945019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3044362393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3044362393},{"type":"v1","addr":"172.21.15.32:6836","nonce":3044362393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:59:04.834110+0000","modified":"2024-04-17T09:59:33.006566+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6837/1253497636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1253497636},{"type":"v1","addr":"172.21.15.32:6837","nonce":1253497636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7851,"qdb_cluster":[7851]},"id":6}]} 2024-04-17T09:59:33.971 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 65 2024-04-17T09:59:33.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.974+0000 7f07e8e00640 1 -- 172.21.15.32:0/2075567990 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07c0068380 msgr2=0x7f07c006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:33.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.974+0000 7f07e8e00640 1 --2- 172.21.15.32:0/2075567990 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07c0068380 0x7f07c006a840 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f07d40011f0 tx=0x7f07d4009040 comp rx=0 tx=0).stop 2024-04-17T09:59:33.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.974+0000 7f07e8e00640 1 -- 172.21.15.32:0/2075567990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e4145f30 msgr2=0x7f07e415ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:33.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.974+0000 7f07e8e00640 1 --2- 172.21.15.32:0/2075567990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e4145f30 0x7f07e415ae80 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f07cc01b8e0 tx=0x7f07cc01ba70 comp rx=0 tx=0).stop 2024-04-17T09:59:33.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.974+0000 7f07e8e00640 1 -- 172.21.15.32:0/2075567990 shutdown_connections 2024-04-17T09:59:33.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.974+0000 7f07e8e00640 1 --2- 172.21.15.32:0/2075567990 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07c0068380 0x7f07c006a840 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:33.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.974+0000 7f07e8e00640 1 --2- 172.21.15.32:0/2075567990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07e415d8d0 0x7f07e415b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:33.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.974+0000 7f07e8e00640 1 --2- 172.21.15.32:0/2075567990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e4145f30 0x7f07e415ae80 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:33.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.974+0000 7f07e8e00640 1 --2- 172.21.15.32:0/2075567990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07e4072500 0x7f07e413d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:33.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.974+0000 7f07e8e00640 1 -- 172.21.15.32:0/2075567990 >> 172.21.15.32:0/2075567990 conn(0x7f07e406ba30 msgr2=0x7f07e406be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:33.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.974+0000 7f07e8e00640 1 -- 172.21.15.32:0/2075567990 shutdown_connections 2024-04-17T09:59:33.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:33.974+0000 7f07e8e00640 1 -- 172.21.15.32:0/2075567990 wait complete. 2024-04-17T09:59:34.395 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T09:59:04.834110+0000', 'modified': '2024-04-17T09:59:33.006566+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7851}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7851': {'gid': 7851, 'name': 'c', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.32:6837/1253497636', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6835', 'nonce': 1253497636}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 1253497636}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7851, 'qdb_cluster': [7851]} 2024-04-17T09:59:34.395 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T09:59:34.395 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-17T09:59:34.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.546+0000 7fb563200640 1 Processor -- start 2024-04-17T09:59:34.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.546+0000 7fb563200640 1 -- start start 2024-04-17T09:59:34.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.546+0000 7fb563200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb56415c670 0x7fb5640cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.546+0000 7fb563200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb56409b890 con 0x7fb56409b3d0 2024-04-17T09:59:34.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.546+0000 7fb563200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb56409bb70 con 0x7fb5640cd590 2024-04-17T09:59:34.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.546+0000 7fb563200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb5640b53d0 con 0x7fb56415c670 2024-04-17T09:59:34.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.546+0000 7fb561400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb56409b3d0 0x7fb5640cce80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45684/0 (socket says 172.21.15.32:45684) 2024-04-17T09:59:34.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.546+0000 7fb561400640 1 -- 172.21.15.32:0/3092907972 learned_addr learned my addr 172.21.15.32:0/3092907972 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:34.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.546+0000 7fb561e00640 1 --2- 172.21.15.32:0/3092907972 >> v2:172.21.15.144:3300/0 conn(0x7fb56415c670 0x7fb5640cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.546+0000 7fb560a00640 1 -- 172.21.15.32:0/3092907972 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3887494370 0 0) 0x7fb56409b890 con 0x7fb56409b3d0 2024-04-17T09:59:34.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.546+0000 7fb560a00640 1 -- 172.21.15.32:0/3092907972 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb548003530 con 0x7fb56409b3d0 2024-04-17T09:59:34.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.546+0000 7fb560a00640 1 -- 172.21.15.32:0/3092907972 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 32745690 0 0) 0x7fb56409bb70 con 0x7fb5640cd590 2024-04-17T09:59:34.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.546+0000 7fb560a00640 1 -- 172.21.15.32:0/3092907972 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb56409b890 con 0x7fb5640cd590 2024-04-17T09:59:34.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.546+0000 7fb561e00640 1 -- 172.21.15.32:0/3092907972 >> v1:172.21.15.144:6789/0 conn(0x7fb5640cd590 legacy=0x7fb5640b5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:34.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.546+0000 7fb561e00640 1 -- 172.21.15.32:0/3092907972 >> v1:172.21.15.32:6789/0 conn(0x7fb56409b3d0 legacy=0x7fb5640cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:34.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.546+0000 7fb561e00640 1 -- 172.21.15.32:0/3092907972 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5640b56b0 con 0x7fb56415c670 2024-04-17T09:59:34.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.550+0000 7fb561e00640 1 --2- 172.21.15.32:0/3092907972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb56415c670 0x7fb5640cdb20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb554009d90 tx=0x7fb55401c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86edec3440a738fa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:34.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.550+0000 7fb560a00640 1 -- 172.21.15.32:0/3092907972 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5540b2070 con 0x7fb56415c670 2024-04-17T09:59:34.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.550+0000 7fb560a00640 1 -- 172.21.15.32:0/3092907972 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5540a5850 con 0x7fb56415c670 2024-04-17T09:59:34.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.550+0000 7fb560a00640 1 -- 172.21.15.32:0/3092907972 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5540a5c20 con 0x7fb56415c670 2024-04-17T09:59:34.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.550+0000 7fb563200640 1 -- 172.21.15.32:0/3092907972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb56415c670 msgr2=0x7fb5640cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:34.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.550+0000 7fb563200640 1 --2- 172.21.15.32:0/3092907972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb56415c670 0x7fb5640cdb20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb554009d90 tx=0x7fb55401c880 comp rx=0 tx=0).stop 2024-04-17T09:59:34.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.550+0000 7fb563200640 1 -- 172.21.15.32:0/3092907972 shutdown_connections 2024-04-17T09:59:34.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.550+0000 7fb563200640 1 --2- 172.21.15.32:0/3092907972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb56415c670 0x7fb5640cdb20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.550+0000 7fb563200640 1 -- 172.21.15.32:0/3092907972 >> 172.21.15.32:0/3092907972 conn(0x7fb5640d23e0 msgr2=0x7fb5640d27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:34.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.550+0000 7fb563200640 1 -- 172.21.15.32:0/3092907972 shutdown_connections 2024-04-17T09:59:34.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.550+0000 7fb563200640 1 -- 172.21.15.32:0/3092907972 wait complete. 2024-04-17T09:59:34.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb563200640 1 Processor -- start 2024-04-17T09:59:34.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb563200640 1 -- start start 2024-04-17T09:59:34.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb563200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb56409b3d0 0x7fb5640c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb563200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5640cd590 0x7fb564157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb563200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb56415c670 0x7fb564152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb563200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb5640b6ec0 con 0x7fb5640cd590 2024-04-17T09:59:34.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb563200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb5640b6d40 con 0x7fb56409b3d0 2024-04-17T09:59:34.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb563200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb5641586d0 con 0x7fb56415c670 2024-04-17T09:59:34.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb561400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5640cd590 0x7fb564157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb561400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5640cd590 0x7fb564157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42732/0 (socket says 172.21.15.32:42732) 2024-04-17T09:59:34.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb561400640 1 -- 172.21.15.32:0/266657986 learned_addr learned my addr 172.21.15.32:0/266657986 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:34.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb561e00640 1 --2- 172.21.15.32:0/266657986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb56409b3d0 0x7fb5640c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb562800640 1 --2- 172.21.15.32:0/266657986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb56415c670 0x7fb564152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb561400640 1 -- 172.21.15.32:0/266657986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb56415c670 msgr2=0x7fb564152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:34.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb561400640 1 --2- 172.21.15.32:0/266657986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb56415c670 0x7fb564152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb561400640 1 -- 172.21.15.32:0/266657986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb56409b3d0 msgr2=0x7fb5640c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:34.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb561400640 1 --2- 172.21.15.32:0/266657986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb56409b3d0 0x7fb5640c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb561400640 1 -- 172.21.15.32:0/266657986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5640c6440 con 0x7fb5640cd590 2024-04-17T09:59:34.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb562800640 1 --2- 172.21.15.32:0/266657986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb56415c670 0x7fb564152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:59:34.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb561400640 1 --2- 172.21.15.32:0/266657986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5640cd590 0x7fb564157770 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fb55000c500 tx=0x7fb55000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:34.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb55a000640 1 -- 172.21.15.32:0/266657986 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb550018070 con 0x7fb5640cd590 2024-04-17T09:59:34.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb55a000640 1 -- 172.21.15.32:0/266657986 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb550005cf0 con 0x7fb5640cd590 2024-04-17T09:59:34.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb55a000640 1 -- 172.21.15.32:0/266657986 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5500072d0 con 0x7fb5640cd590 2024-04-17T09:59:34.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb563200640 1 -- 172.21.15.32:0/266657986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb56406cf40 con 0x7fb5640cd590 2024-04-17T09:59:34.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.554+0000 7fb563200640 1 -- 172.21.15.32:0/266657986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5640c8c60 con 0x7fb5640cd590 2024-04-17T09:59:34.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.558+0000 7fb55a000640 1 -- 172.21.15.32:0/266657986 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb550007470 con 0x7fb5640cd590 2024-04-17T09:59:34.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.558+0000 7fb55a000640 1 --2- 172.21.15.32:0/266657986 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb5440683b0 0x7fb54406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.558+0000 7fb55a000640 1 -- 172.21.15.32:0/266657986 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7fb550085d80 con 0x7fb5640cd590 2024-04-17T09:59:34.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.558+0000 7fb561e00640 1 --2- 172.21.15.32:0/266657986 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb5440683b0 0x7fb54406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.558+0000 7fb56be61000 1 Processor -- start 2024-04-17T09:59:34.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.558+0000 7fb56be61000 1 -- start start 2024-04-17T09:59:34.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.558+0000 7fb561e00640 1 --2- 172.21.15.32:0/266657986 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb5440683b0 0x7fb54406a870 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb5540af000 tx=0x7fb5540a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:34.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb56be61000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376fbfb50 0x565376fbff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb56be61000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565376fc0470 0x565376f757b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb56be61000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565376f75cf0 0x565376f760d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb56be61000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x565376dc34d0 con 0x565376fbfb50 2024-04-17T09:59:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb56be61000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x565376d5dbf0 con 0x565376fc0470 2024-04-17T09:59:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb56be61000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x565376dbf490 con 0x565376f75cf0 2024-04-17T09:59:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb561e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376fbfb50 0x565376fbff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb561e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376fbfb50 0x565376fbff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42748/0 (socket says 172.21.15.32:42748) 2024-04-17T09:59:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb561e00640 1 -- 172.21.15.32:0/2577525273 learned_addr learned my addr 172.21.15.32:0/2577525273 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb561400640 1 --2- 172.21.15.32:0/2577525273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565376fc0470 0x565376f757b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb562800640 1 --2- 172.21.15.32:0/2577525273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565376f75cf0 0x565376f760d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb561e00640 1 -- 172.21.15.32:0/2577525273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565376f75cf0 msgr2=0x565376f760d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb561e00640 1 --2- 172.21.15.32:0/2577525273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565376f75cf0 0x565376f760d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb561e00640 1 -- 172.21.15.32:0/2577525273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565376fc0470 msgr2=0x565376f757b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb561e00640 1 --2- 172.21.15.32:0/2577525273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565376fc0470 0x565376f757b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb561e00640 1 -- 172.21.15.32:0/2577525273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565376d49e80 con 0x565376fbfb50 2024-04-17T09:59:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb561400640 1 --2- 172.21.15.32:0/2577525273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565376fc0470 0x565376f757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb561e00640 1 --2- 172.21.15.32:0/2577525273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376fbfb50 0x565376fbff30 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fb554009bd0 tx=0x7fb554004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de013cb8ce8e655 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb54ea00640 1 -- 172.21.15.32:0/2577525273 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5540b2040 con 0x565376fbfb50 2024-04-17T09:59:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb54ea00640 1 -- 172.21.15.32:0/2577525273 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5540a5de0 con 0x565376fbfb50 2024-04-17T09:59:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb54ea00640 1 -- 172.21.15.32:0/2577525273 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5540b8650 con 0x565376fbfb50 2024-04-17T09:59:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb56be61000 1 -- 172.21.15.32:0/2577525273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376fbfb50 msgr2=0x565376fbff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb56be61000 1 --2- 172.21.15.32:0/2577525273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376fbfb50 0x565376fbff30 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fb554009bd0 tx=0x7fb554004ac0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb56be61000 1 -- 172.21.15.32:0/2577525273 shutdown_connections 2024-04-17T09:59:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb56be61000 1 --2- 172.21.15.32:0/2577525273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565376f75cf0 0x565376f760d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb56be61000 1 --2- 172.21.15.32:0/2577525273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565376fc0470 0x565376f757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb56be61000 1 --2- 172.21.15.32:0/2577525273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376fbfb50 0x565376fbff30 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb56be61000 1 -- 172.21.15.32:0/2577525273 >> 172.21.15.32:0/2577525273 conn(0x565376f7b470 msgr2=0x565376f74090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:34.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb56be61000 1 -- 172.21.15.32:0/2577525273 shutdown_connections 2024-04-17T09:59:34.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb56be61000 1 -- 172.21.15.32:0/2577525273 wait complete. 2024-04-17T09:59:34.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb56be61000 1 -- start start 2024-04-17T09:59:34.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.562+0000 7fb56be61000 1 Processor -- start 2024-04-17T09:59:34.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb56be61000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565376f75cf0 0x565376f7a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb56be61000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565376fbfb50 0x565376fad440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb56be61000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376fc0470 0x565376fad980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb56be61000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x565376d80a60 con 0x565376fc0470 2024-04-17T09:59:34.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb56be61000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x565376d67e80 con 0x565376fbfb50 2024-04-17T09:59:34.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb56be61000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x565376d7ecf0 con 0x565376f75cf0 2024-04-17T09:59:34.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb56be61000 10 client.0 _ll_register_callbacks cb 0x565376f60a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:59:34.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb562800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376fc0470 0x565376fad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb562800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376fc0470 0x565376fad980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42760/0 (socket says 172.21.15.32:42760) 2024-04-17T09:59:34.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb562800640 1 -- 172.21.15.32:0/3849432020 learned_addr learned my addr 172.21.15.32:0/3849432020 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:34.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb561400640 1 --2- 172.21.15.32:0/3849432020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565376fbfb50 0x565376fad440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb562800640 1 -- 172.21.15.32:0/3849432020 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565376f75cf0 msgr2=0x565376f7a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:34.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb561e00640 1 --2- 172.21.15.32:0/3849432020 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565376f75cf0 0x565376f7a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb562800640 1 --2- 172.21.15.32:0/3849432020 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565376f75cf0 0x565376f7a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb562800640 1 -- 172.21.15.32:0/3849432020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565376fbfb50 msgr2=0x565376fad440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:34.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb562800640 1 --2- 172.21.15.32:0/3849432020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565376fbfb50 0x565376fad440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb562800640 1 -- 172.21.15.32:0/3849432020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565376d5de40 con 0x565376fc0470 2024-04-17T09:59:34.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb561400640 1 --2- 172.21.15.32:0/3849432020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565376fbfb50 0x565376fad440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:59:34.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb561e00640 1 --2- 172.21.15.32:0/3849432020 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565376f75cf0 0x565376f7a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:59:34.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb562800640 1 --2- 172.21.15.32:0/3849432020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376fc0470 0x565376fad980 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fb55c001df0 tx=0x7fb55c00f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8e47758db27c8b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:34.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb543400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:59:34.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb543400640 1 -- 172.21.15.32:0/3849432020 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c0050d0 con 0x565376fc0470 2024-04-17T09:59:34.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb543400640 1 -- 172.21.15.32:0/3849432020 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7fb55c005270 con 0x565376fc0470 2024-04-17T09:59:34.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb56be61000 1 -- 172.21.15.32:0/3849432020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5540a5de0 con 0x565376fc0470 2024-04-17T09:59:34.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb543400640 1 -- 172.21.15.32:0/3849432020 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fb55c020050 con 0x565376fc0470 2024-04-17T09:59:34.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb56be61000 10 client.7914 fetch_fsmap learned FSMap version 65 2024-04-17T09:59:34.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb56be61000 1 -- 172.21.15.32:0/3849432020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x565376d8ee00 con 0x565376fc0470 2024-04-17T09:59:34.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb543400640 1 -- 172.21.15.32:0/3849432020 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5540a5de0 con 0x565376fc0470 2024-04-17T09:59:34.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb543400640 1 -- 172.21.15.32:0/3849432020 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 65) ==== 3495+0+0 (secure 0 0 0) 0x7fb55c023020 con 0x565376fc0470 2024-04-17T09:59:34.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb56be61000 10 client.7914 fetch_fsmap finished waiting for FSMap version 65 2024-04-17T09:59:34.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb56be61000 10 client.7914 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T09:59:34.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb56be61000 20 client.7914 populate_metadata read hostname 'smithi032' 2024-04-17T09:59:34.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb56be61000 1 --2- 172.21.15.32:0/3849432020 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x565376fcf0b0 0x565376fd14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb56be61000 4 client.7914 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:59:34.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb56be61000 1 -- 172.21.15.32:0/3849432020 --> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565376d5cbc0 con 0x565376fcf0b0 2024-04-17T09:59:34.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb561e00640 1 --2- 172.21.15.32:0/3849432020 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x565376fcf0b0 0x565376fd14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb561e00640 1 --2- 172.21.15.32:0/3849432020 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x565376fcf0b0 0x565376fd14a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37f9b73b76f3266a server_cookie=e7cc861e29c3de93 in_seq=0 out_seq=0 2024-04-17T09:59:34.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb543400640 10 client.7914 ms_handle_connect on v2:172.21.15.32:6835/1253497636 2024-04-17T09:59:34.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb543400640 1 -- 172.21.15.32:0/3849432020 <== mds.0 v2:172.21.15.32:6835/1253497636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565376d5cbc0 con 0x565376fcf0b0 2024-04-17T09:59:34.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb543400640 10 client.7914 handle_command_reply: tid=0 2024-04-17T09:59:34.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.566+0000 7fb56be61000 1 client.7914 shutdown 2024-04-17T09:59:34.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.570+0000 7fb56be61000 1 -- 172.21.15.32:0/3849432020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376fc0470 msgr2=0x565376fad980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:34.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.570+0000 7fb56be61000 1 --2- 172.21.15.32:0/3849432020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376fc0470 0x565376fad980 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fb55c001df0 tx=0x7fb55c00f5a0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.570+0000 7fb56be61000 1 -- 172.21.15.32:0/3849432020 shutdown_connections 2024-04-17T09:59:34.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.570+0000 7fb56be61000 1 --2- 172.21.15.32:0/3849432020 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x565376fcf0b0 0x565376fd14a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.570+0000 7fb56be61000 1 --2- 172.21.15.32:0/3849432020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376fc0470 0x565376fad980 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.570+0000 7fb56be61000 1 --2- 172.21.15.32:0/3849432020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565376fbfb50 0x565376fad440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.570+0000 7fb56be61000 1 --2- 172.21.15.32:0/3849432020 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565376f75cf0 0x565376f7a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.570+0000 7fb56be61000 1 -- 172.21.15.32:0/3849432020 >> 172.21.15.32:0/3849432020 conn(0x565376f7b470 msgr2=0x565376f747a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:34.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.570+0000 7fb543400640 0 client.7914 ms_handle_reset on v2:172.21.15.32:6835/1253497636 2024-04-17T09:59:34.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.570+0000 7fb56be61000 1 -- 172.21.15.32:0/3849432020 shutdown_connections 2024-04-17T09:59:34.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.570+0000 7fb56be61000 1 -- 172.21.15.32:0/3849432020 wait complete. 2024-04-17T09:59:34.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.570+0000 7fb56be61000 20 client.7914 trim_cache size 0 max 16384 2024-04-17T09:59:34.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.586+0000 7fb56be61000 1 Processor -- start 2024-04-17T09:59:34.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.586+0000 7fb56be61000 1 -- start start 2024-04-17T09:59:34.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.586+0000 7fb56be61000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376f75cf0 0x565376fad020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.586+0000 7fb56be61000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565376fbfb50 0x565376fad560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.586+0000 7fb56be61000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565376fc0470 0x565376fd04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.586+0000 7fb56be61000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb564202af0 con 0x565376f75cf0 2024-04-17T09:59:34.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.586+0000 7fb56be61000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb56406e4b0 con 0x565376fbfb50 2024-04-17T09:59:34.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.586+0000 7fb56be61000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb56406eb00 con 0x565376fc0470 2024-04-17T09:59:34.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.586+0000 7fb561e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376f75cf0 0x565376fad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.586+0000 7fb561e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376f75cf0 0x565376fad020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42762/0 (socket says 172.21.15.32:42762) 2024-04-17T09:59:34.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.586+0000 7fb561e00640 1 -- 172.21.15.32:0/3115441150 learned_addr learned my addr 172.21.15.32:0/3115441150 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:34.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.586+0000 7fb562800640 1 --2- 172.21.15.32:0/3115441150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565376fc0470 0x565376fd04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.586+0000 7fb561400640 1 --2- 172.21.15.32:0/3115441150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565376fbfb50 0x565376fad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.586+0000 7fb561e00640 1 -- 172.21.15.32:0/3115441150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565376fc0470 msgr2=0x565376fd04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:34.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.586+0000 7fb561e00640 1 --2- 172.21.15.32:0/3115441150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565376fc0470 0x565376fd04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.586+0000 7fb561e00640 1 -- 172.21.15.32:0/3115441150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565376fbfb50 msgr2=0x565376fad560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:34.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.586+0000 7fb561e00640 1 --2- 172.21.15.32:0/3115441150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565376fbfb50 0x565376fad560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.586+0000 7fb561e00640 1 -- 172.21.15.32:0/3115441150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565376e44d30 con 0x565376f75cf0 2024-04-17T09:59:34.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.586+0000 7fb562800640 1 --2- 172.21.15.32:0/3115441150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565376fc0470 0x565376fd04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:34.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb561e00640 1 --2- 172.21.15.32:0/3115441150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376f75cf0 0x565376fad020 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fb5540194f0 tx=0x7fb554009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6cd4bb3a5186adb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:34.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb54e000640 1 -- 172.21.15.32:0/3115441150 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5540b2040 con 0x565376f75cf0 2024-04-17T09:59:34.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb54e000640 1 -- 172.21.15.32:0/3115441150 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5540d2170 con 0x565376f75cf0 2024-04-17T09:59:34.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb54e000640 1 -- 172.21.15.32:0/3115441150 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5540d2520 con 0x565376f75cf0 2024-04-17T09:59:34.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb56be61000 1 -- 172.21.15.32:0/3115441150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376f75cf0 msgr2=0x565376fad020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:34.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb56be61000 1 --2- 172.21.15.32:0/3115441150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376f75cf0 0x565376fad020 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fb5540194f0 tx=0x7fb554009750 comp rx=0 tx=0).stop 2024-04-17T09:59:34.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb56be61000 1 -- 172.21.15.32:0/3115441150 shutdown_connections 2024-04-17T09:59:34.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb56be61000 1 --2- 172.21.15.32:0/3115441150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565376fc0470 0x565376fd04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb56be61000 1 --2- 172.21.15.32:0/3115441150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565376fbfb50 0x565376fad560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb56be61000 1 --2- 172.21.15.32:0/3115441150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376f75cf0 0x565376fad020 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb56be61000 1 -- 172.21.15.32:0/3115441150 >> 172.21.15.32:0/3115441150 conn(0x565376f7b470 msgr2=0x565376fac910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:34.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb56be61000 1 -- 172.21.15.32:0/3115441150 shutdown_connections 2024-04-17T09:59:34.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb56be61000 1 -- 172.21.15.32:0/3115441150 wait complete. 2024-04-17T09:59:34.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb56be61000 1 -- start start 2024-04-17T09:59:34.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb56be61000 1 Processor -- start 2024-04-17T09:59:34.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb56be61000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376f75cf0 0x565376fd38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb56be61000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565376fbfb50 0x565376feb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb56be61000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565376fc0470 0x565376feb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb56be61000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x565376dc7b00 con 0x565376f75cf0 2024-04-17T09:59:34.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb56be61000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x565376dc27e0 con 0x565376fc0470 2024-04-17T09:59:34.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb56be61000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x565376dd06e0 con 0x565376fbfb50 2024-04-17T09:59:34.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb56be61000 10 client.0 _ll_register_callbacks cb 0x7fb55c007c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:59:34.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb561e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376f75cf0 0x565376fd38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb561e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376f75cf0 0x565376fd38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42770/0 (socket says 172.21.15.32:42770) 2024-04-17T09:59:34.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb561e00640 1 -- 172.21.15.32:0/2988831043 learned_addr learned my addr 172.21.15.32:0/2988831043 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:34.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb561400640 1 --2- 172.21.15.32:0/2988831043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565376fbfb50 0x565376feb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb562800640 1 --2- 172.21.15.32:0/2988831043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565376fc0470 0x565376feb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb561e00640 1 -- 172.21.15.32:0/2988831043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565376fbfb50 msgr2=0x565376feb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:34.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb561e00640 1 --2- 172.21.15.32:0/2988831043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565376fbfb50 0x565376feb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb561e00640 1 -- 172.21.15.32:0/2988831043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565376fc0470 msgr2=0x565376feb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:34.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb561e00640 1 --2- 172.21.15.32:0/2988831043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565376fc0470 0x565376feb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb561e00640 1 -- 172.21.15.32:0/2988831043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565376d557c0 con 0x565376f75cf0 2024-04-17T09:59:34.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb561400640 1 --2- 172.21.15.32:0/2988831043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565376fbfb50 0x565376feb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:34.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb561e00640 1 --2- 172.21.15.32:0/2988831043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376f75cf0 0x565376fd38e0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fb55401cd60 tx=0x7fb55401b120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b38c54c6b5f8b30 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:34.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.590+0000 7fb543400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:59:34.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb543400640 1 -- 172.21.15.32:0/2988831043 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5540b2070 con 0x565376f75cf0 2024-04-17T09:59:34.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb543400640 1 -- 172.21.15.32:0/2988831043 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7fb5540d23e0 con 0x565376f75cf0 2024-04-17T09:59:34.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb56be61000 1 -- 172.21.15.32:0/2988831043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5540d2170 con 0x565376f75cf0 2024-04-17T09:59:34.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb543400640 1 -- 172.21.15.32:0/2988831043 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fb5540d25e0 con 0x565376f75cf0 2024-04-17T09:59:34.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb56be61000 10 client.7920 fetch_fsmap learned FSMap version 65 2024-04-17T09:59:34.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb56be61000 1 -- 172.21.15.32:0/2988831043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x565376d68650 con 0x565376f75cf0 2024-04-17T09:59:34.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb543400640 1 -- 172.21.15.32:0/2988831043 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5540d2170 con 0x565376f75cf0 2024-04-17T09:59:34.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb543400640 1 -- 172.21.15.32:0/2988831043 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 65) ==== 3495+0+0 (secure 0 0 0) 0x7fb5540d2780 con 0x565376f75cf0 2024-04-17T09:59:34.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb56be61000 10 client.7920 fetch_fsmap finished waiting for FSMap version 65 2024-04-17T09:59:34.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb56be61000 10 client.7920 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T09:59:34.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb56be61000 20 client.7920 populate_metadata read hostname 'smithi032' 2024-04-17T09:59:34.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb56be61000 1 --2- 172.21.15.32:0/2988831043 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x565376febab0 0x56537702e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb56be61000 4 client.7920 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T09:59:34.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb56be61000 1 -- 172.21.15.32:0/2988831043 --> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] -- command(tid 0: {"prefix": "status"}) -- 0x565376dd0ef0 con 0x565376febab0 2024-04-17T09:59:34.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb561400640 1 --2- 172.21.15.32:0/2988831043 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x565376febab0 0x56537702e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb561400640 1 --2- 172.21.15.32:0/2988831043 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x565376febab0 0x56537702e460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bea6952c5a7fb51d server_cookie=904121eb0732337 in_seq=0 out_seq=0 2024-04-17T09:59:34.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb543400640 10 client.7920 ms_handle_connect on v2:172.21.15.32:6835/1253497636 2024-04-17T09:59:34.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb543400640 1 -- 172.21.15.32:0/2988831043 <== mds.0 v2:172.21.15.32:6835/1253497636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x565376dd0ef0 con 0x565376febab0 2024-04-17T09:59:34.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb543400640 10 client.7920 handle_command_reply: tid=0 2024-04-17T09:59:34.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb56be61000 1 client.7920 shutdown 2024-04-17T09:59:34.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb56be61000 1 -- 172.21.15.32:0/2988831043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376f75cf0 msgr2=0x565376fd38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:34.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb56be61000 1 --2- 172.21.15.32:0/2988831043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376f75cf0 0x565376fd38e0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fb55401cd60 tx=0x7fb55401b120 comp rx=0 tx=0).stop 2024-04-17T09:59:34.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb56be61000 1 -- 172.21.15.32:0/2988831043 shutdown_connections 2024-04-17T09:59:34.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb56be61000 1 --2- 172.21.15.32:0/2988831043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565376fbfb50 0x565376feb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb56be61000 1 --2- 172.21.15.32:0/2988831043 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x565376febab0 0x56537702e460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb56be61000 1 --2- 172.21.15.32:0/2988831043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565376fc0470 0x565376feb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb56be61000 1 --2- 172.21.15.32:0/2988831043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x565376f75cf0 0x565376fd38e0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb543400640 0 client.7920 ms_handle_reset on v2:172.21.15.32:6835/1253497636 2024-04-17T09:59:34.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb56be61000 1 -- 172.21.15.32:0/2988831043 >> 172.21.15.32:0/2988831043 conn(0x565376f7b470 msgr2=0x565376f77070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:34.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb56be61000 1 -- 172.21.15.32:0/2988831043 shutdown_connections 2024-04-17T09:59:34.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb56be61000 1 -- 172.21.15.32:0/2988831043 wait complete. 2024-04-17T09:59:34.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb56be61000 20 client.7920 trim_cache size 0 max 16384 2024-04-17T09:59:34.597 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:59:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "ff178df8-4c06-4367-a1e9-2e3a66d870d0", 2024-04-17T09:59:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-17T09:59:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 7851, 2024-04-17T09:59:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-17T09:59:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-17T09:59:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-17T09:59:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 4.595209594, 2024-04-17T09:59:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 65, 2024-04-17T09:59:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 83, 2024-04-17T09:59:34.597 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 83, 2024-04-17T09:59:34.598 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 5.5440169299999997 2024-04-17T09:59:34.598 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:59:34.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb563200640 1 -- 172.21.15.32:0/266657986 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb5440683b0 msgr2=0x7fb54406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:34.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb563200640 1 --2- 172.21.15.32:0/266657986 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb5440683b0 0x7fb54406a870 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb5540af000 tx=0x7fb5540a7070 comp rx=0 tx=0).stop 2024-04-17T09:59:34.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb563200640 1 -- 172.21.15.32:0/266657986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5640cd590 msgr2=0x7fb564157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:34.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb563200640 1 --2- 172.21.15.32:0/266657986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5640cd590 0x7fb564157770 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fb55000c500 tx=0x7fb55000c9c0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb563200640 1 -- 172.21.15.32:0/266657986 shutdown_connections 2024-04-17T09:59:34.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb563200640 1 --2- 172.21.15.32:0/266657986 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb5440683b0 0x7fb54406a870 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb563200640 1 --2- 172.21.15.32:0/266657986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb56415c670 0x7fb564152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb563200640 1 --2- 172.21.15.32:0/266657986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5640cd590 0x7fb564157770 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb563200640 1 --2- 172.21.15.32:0/266657986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb56409b3d0 0x7fb5640c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb563200640 1 -- 172.21.15.32:0/266657986 >> 172.21.15.32:0/266657986 conn(0x7fb5640d23e0 msgr2=0x7fb5640cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:34.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb563200640 1 -- 172.21.15.32:0/266657986 shutdown_connections 2024-04-17T09:59:34.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.594+0000 7fb563200640 1 -- 172.21.15.32:0/266657986 wait complete. 2024-04-17T09:59:34.611 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T09:59:34.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.754+0000 7fbe5b200640 1 Processor -- start 2024-04-17T09:59:34.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.754+0000 7fbe5b200640 1 -- start start 2024-04-17T09:59:34.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.758+0000 7fbe5b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe54145f30 0x7fbe54071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.758+0000 7fbe5b200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbe540cd580 0x7fbe54072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.758+0000 7fbe5b200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe540cdac0 con 0x7fbe5415d8d0 2024-04-17T09:59:34.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.758+0000 7fbe5b200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fbe5406f400 con 0x7fbe540cd580 2024-04-17T09:59:34.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.758+0000 7fbe5b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbe54146310 con 0x7fbe54145f30 2024-04-17T09:59:34.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.758+0000 7fbe4b400640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fbe5415d8d0 0x7fbe540cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:42744/0 (socket says 172.21.15.32:42744) 2024-04-17T09:59:34.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.758+0000 7fbe4b400640 1 -- 172.21.15.32:0/2618983192 learned_addr learned my addr 172.21.15.32:0/2618983192 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:34.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.758+0000 7fbe53e00640 1 --2- 172.21.15.32:0/2618983192 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe54145f30 0x7fbe54071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.758+0000 7fbe59200640 1 --2- 172.21.15.32:0/2618983192 >> v2:172.21.15.144:3300/0 conn(0x7fbe540cd580 0x7fbe54072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.758+0000 7fbe53400640 1 -- 172.21.15.32:0/2618983192 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3670206611 0 0) 0x7fbe540cdac0 con 0x7fbe5415d8d0 2024-04-17T09:59:34.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.758+0000 7fbe53400640 1 -- 172.21.15.32:0/2618983192 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe38003530 con 0x7fbe5415d8d0 2024-04-17T09:59:34.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.758+0000 7fbe59200640 1 -- 172.21.15.32:0/2618983192 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe54145f30 msgr2=0x7fbe54071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:34.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.758+0000 7fbe59200640 1 --2- 172.21.15.32:0/2618983192 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe54145f30 0x7fbe54071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.758+0000 7fbe59200640 1 -- 172.21.15.32:0/2618983192 >> v1:172.21.15.144:6789/0 conn(0x7fbe5415d8d0 legacy=0x7fbe540cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:34.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.758+0000 7fbe59200640 1 -- 172.21.15.32:0/2618983192 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5415cdf0 con 0x7fbe540cd580 2024-04-17T09:59:34.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.762+0000 7fbe59200640 1 --2- 172.21.15.32:0/2618983192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe540cd580 0x7fbe54072330 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009d90 tx=0x7fbe4c01f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8abcd306ba93ad3b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:34.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.762+0000 7fbe53400640 1 -- 172.21.15.32:0/2618983192 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c0b4070 con 0x7fbe540cd580 2024-04-17T09:59:34.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.762+0000 7fbe53400640 1 -- 172.21.15.32:0/2618983192 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe4c0af600 con 0x7fbe540cd580 2024-04-17T09:59:34.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.762+0000 7fbe53400640 1 -- 172.21.15.32:0/2618983192 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c0af9d0 con 0x7fbe540cd580 2024-04-17T09:59:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.762+0000 7fbe5b200640 1 -- 172.21.15.32:0/2618983192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe540cd580 msgr2=0x7fbe54072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.762+0000 7fbe5b200640 1 --2- 172.21.15.32:0/2618983192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe540cd580 0x7fbe54072330 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009d90 tx=0x7fbe4c01f150 comp rx=0 tx=0).stop 2024-04-17T09:59:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.762+0000 7fbe5b200640 1 -- 172.21.15.32:0/2618983192 shutdown_connections 2024-04-17T09:59:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.762+0000 7fbe5b200640 1 --2- 172.21.15.32:0/2618983192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe540cd580 0x7fbe54072330 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.762+0000 7fbe5b200640 1 --2- 172.21.15.32:0/2618983192 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe54145f30 0x7fbe54071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.762+0000 7fbe5b200640 1 -- 172.21.15.32:0/2618983192 >> 172.21.15.32:0/2618983192 conn(0x7fbe5406ba30 msgr2=0x7fbe5406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.762+0000 7fbe5b200640 1 -- 172.21.15.32:0/2618983192 shutdown_connections 2024-04-17T09:59:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.762+0000 7fbe5b200640 1 -- 172.21.15.32:0/2618983192 wait complete. 2024-04-17T09:59:34.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.762+0000 7fbe5b200640 1 Processor -- start 2024-04-17T09:59:34.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.762+0000 7fbe5b200640 1 -- start start 2024-04-17T09:59:34.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.762+0000 7fbe5b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe540cd580 0x7fbe54140e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.762+0000 7fbe5b200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe54145f30 0x7fbe5415dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.762+0000 7fbe5b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe5415d8d0 0x7fbe5415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.762+0000 7fbe5b200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5415cb90 con 0x7fbe540cd580 2024-04-17T09:59:34.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.762+0000 7fbe5b200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5415d440 con 0x7fbe54145f30 2024-04-17T09:59:34.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.762+0000 7fbe5b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbe541601d0 con 0x7fbe5415d8d0 2024-04-17T09:59:34.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.766+0000 7fbe53e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe540cd580 0x7fbe54140e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.766+0000 7fbe53e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe540cd580 0x7fbe54140e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42774/0 (socket says 172.21.15.32:42774) 2024-04-17T09:59:34.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.766+0000 7fbe53e00640 1 -- 172.21.15.32:0/602276266 learned_addr learned my addr 172.21.15.32:0/602276266 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:34.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.766+0000 7fbe4b400640 1 --2- 172.21.15.32:0/602276266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe54145f30 0x7fbe5415dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.766+0000 7fbe59200640 1 --2- 172.21.15.32:0/602276266 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe5415d8d0 0x7fbe5415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.766+0000 7fbe53e00640 1 -- 172.21.15.32:0/602276266 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe5415d8d0 msgr2=0x7fbe5415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:34.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.766+0000 7fbe53e00640 1 --2- 172.21.15.32:0/602276266 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe5415d8d0 0x7fbe5415bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.766+0000 7fbe53e00640 1 -- 172.21.15.32:0/602276266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe54145f30 msgr2=0x7fbe5415dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:34.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.766+0000 7fbe53e00640 1 --2- 172.21.15.32:0/602276266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe54145f30 0x7fbe5415dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.766+0000 7fbe53e00640 1 -- 172.21.15.32:0/602276266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe540cad40 con 0x7fbe540cd580 2024-04-17T09:59:34.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.766+0000 7fbe53e00640 1 --2- 172.21.15.32:0/602276266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe540cd580 0x7fbe54140e70 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400e930 tx=0x7fbe4400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:34.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.766+0000 7fbe50c00640 1 -- 172.21.15.32:0/602276266 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44017070 con 0x7fbe540cd580 2024-04-17T09:59:34.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.766+0000 7fbe50c00640 1 -- 172.21.15.32:0/602276266 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe4400ca00 con 0x7fbe540cd580 2024-04-17T09:59:34.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.766+0000 7fbe50c00640 1 -- 172.21.15.32:0/602276266 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4400cdb0 con 0x7fbe540cd580 2024-04-17T09:59:34.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.766+0000 7fbe5b200640 1 -- 172.21.15.32:0/602276266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe5412fc60 con 0x7fbe540cd580 2024-04-17T09:59:34.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.766+0000 7fbe5b200640 1 -- 172.21.15.32:0/602276266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe54130220 con 0x7fbe540cd580 2024-04-17T09:59:34.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.766+0000 7fbe5b200640 1 -- 172.21.15.32:0/602276266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe54068a90 con 0x7fbe540cd580 2024-04-17T09:59:34.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.766+0000 7fbe50c00640 1 -- 172.21.15.32:0/602276266 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbe4401a070 con 0x7fbe540cd580 2024-04-17T09:59:34.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.770+0000 7fbe50c00640 1 --2- 172.21.15.32:0/602276266 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbe30068380 0x7fbe3006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:34.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.770+0000 7fbe50c00640 1 -- 172.21.15.32:0/602276266 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7fbe44086a90 con 0x7fbe540cd580 2024-04-17T09:59:34.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.770+0000 7fbe4b400640 1 --2- 172.21.15.32:0/602276266 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbe30068380 0x7fbe3006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:34.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.770+0000 7fbe4b400640 1 --2- 172.21.15.32:0/602276266 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbe30068380 0x7fbe3006a840 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fbe540b3490 tx=0x7fbe3c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:34.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.774+0000 7fbe50c00640 1 -- 172.21.15.32:0/602276266 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fbe440570a0 con 0x7fbe540cd580 2024-04-17T09:59:34.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.970+0000 7fbe5b200640 1 -- 172.21.15.32:0/602276266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbe54072f90 con 0x7fbe540cd580 2024-04-17T09:59:34.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.970+0000 7fbe50c00640 1 -- 172.21.15.32:0/602276266 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) ==== 53+0+73 (secure 0 0 0) 0x7fbe440569a0 con 0x7fbe540cd580 2024-04-17T09:59:34.970 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T09:59:34.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.974+0000 7fbe5b200640 1 -- 172.21.15.32:0/602276266 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbe30068380 msgr2=0x7fbe3006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:34.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.974+0000 7fbe5b200640 1 --2- 172.21.15.32:0/602276266 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbe30068380 0x7fbe3006a840 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fbe540b3490 tx=0x7fbe3c009040 comp rx=0 tx=0).stop 2024-04-17T09:59:34.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.974+0000 7fbe5b200640 1 -- 172.21.15.32:0/602276266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe540cd580 msgr2=0x7fbe54140e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:34.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.974+0000 7fbe5b200640 1 --2- 172.21.15.32:0/602276266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe540cd580 0x7fbe54140e70 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400e930 tx=0x7fbe4400edf0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.974+0000 7fbe5b200640 1 -- 172.21.15.32:0/602276266 shutdown_connections 2024-04-17T09:59:34.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.974+0000 7fbe5b200640 1 --2- 172.21.15.32:0/602276266 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbe30068380 0x7fbe3006a840 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.974+0000 7fbe5b200640 1 --2- 172.21.15.32:0/602276266 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe5415d8d0 0x7fbe5415bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.974+0000 7fbe5b200640 1 --2- 172.21.15.32:0/602276266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe54145f30 0x7fbe5415dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.974+0000 7fbe5b200640 1 --2- 172.21.15.32:0/602276266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe540cd580 0x7fbe54140e70 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:34.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.974+0000 7fbe5b200640 1 -- 172.21.15.32:0/602276266 >> 172.21.15.32:0/602276266 conn(0x7fbe5406ba30 msgr2=0x7fbe540b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:34.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.974+0000 7fbe5b200640 1 -- 172.21.15.32:0/602276266 shutdown_connections 2024-04-17T09:59:34.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:34.974+0000 7fbe5b200640 1 -- 172.21.15.32:0/602276266 wait complete. 2024-04-17T09:59:34.991 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T09:59:34.992 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T09:59:34.992 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-17T09:59:34.992 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T09:59:34.992 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T09:59:34.992 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T09:59:34.992 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T09:59:34.992 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T09:59:34.992 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:59:34.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:34.992 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-17T09:59:34.998 INFO:tasks.ceph.mds.c.smithi032.stderr:2024-04-17T09:59:35.002+0000 7f3487a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T09:59:35.002 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T09:59:35.002 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T09:59:35.002 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T09:59:35.002 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:59:35.002 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-17T09:59:35.002 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:59:35.002 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:59:35.002 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-17T09:59:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-17T09:59:35.003 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T09:59:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:59:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-17T09:59:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-17T09:59:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42079sec preferred_lft 42079sec 2024-04-17T09:59:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2024-04-17T09:59:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:59:35.003 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:59:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-17T09:59:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-17T09:59:35.003 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T09:59:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-17T09:59:35.003 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-17T09:59:35.004 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T09:59:35.004 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:cd:f4:aa:5b brd ff:ff:ff:ff:ff:ff 2024-04-17T09:59:35.004 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T09:59:35.004 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:59:35.004 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T09:59:35.004 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T09:59:35.004 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T09:59:35.004 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:59:35.004 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T09:59:35.004 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:59:35.005 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T09:59:35.005 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:59:35.005 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T09:59:35.005 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:59:35.005 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T09:59:35.005 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:59:35.005 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T09:59:35.005 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T09:59:35.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:35.005 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T09:59:35.067 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-17T09:59:35.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:35.069 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-17T09:59:35.123 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-17T09:59:35.123 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T09:59:35.123 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-17T09:59:35.124 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T09:59:35.124 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-17T09:59:35.124 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T09:59:35.124 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T09:59:35.124 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T09:59:35.125 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-17T09:59:35.125 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-17T09:59:35.125 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-17T09:59:35.125 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T09:59:35.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T09:59:35.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:59:35.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:35.262 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-17T09:59:35.313 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T09:59:35.314 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T09:59:35.315 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:59:35.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:35.315 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-17T09:59:35.378 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T09:59:35.378 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:59:35.378 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T09:59:35.430 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T09:59:35.430 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T09:59:35.430 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T09:59:35.481 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T09:59:35.541 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:59:35.595 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T09:59:35.595 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T09:59:35.595 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T09:59:35.595 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T09:59:35.595 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T09:59:35.595 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T09:59:35.596 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T09:59:35.596 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T09:59:35.596 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T09:59:35.596 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T09:59:35.596 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T09:59:35.596 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T09:59:35.596 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:59:35.596 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T09:59:35.596 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T09:59:35.596 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T09:59:35.597 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T09:59:35.597 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T09:59:35.597 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T09:59:35.597 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T09:59:35.597 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:59:35.597 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T09:59:35.598 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T09:59:35.598 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T09:59:35.598 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T09:59:35.598 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T09:59:35.598 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T09:59:35.598 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T09:59:35.598 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T09:59:35.598 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:59:35.598 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:59:35.599 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:59:35.599 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:59:35.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:35.600 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T09:59:35.649 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T09:59:35.649 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T09:59:35.696 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T09:59:35.717 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T09:59:35.772 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T09:59:35.772 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T09:59:35.772 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T09:59:35.772 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T09:59:35.772 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T09:59:35.772 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T09:59:35.772 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T09:59:35.772 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T09:59:35.772 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T09:59:35.772 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T09:59:35.773 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T09:59:35.773 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T09:59:35.773 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T09:59:35.773 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T09:59:35.773 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T09:59:35.773 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T09:59:35.773 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T09:59:35.773 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T09:59:35.773 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T09:59:35.773 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T09:59:35.773 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T09:59:35.773 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T09:59:35.773 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T09:59:35.773 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T09:59:35.773 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T09:59:35.774 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T09:59:35.774 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T09:59:35.774 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T09:59:35.774 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T09:59:35.774 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:59:35.774 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T09:59:35.774 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:59:35.774 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T09:59:35.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:35.775 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T09:59:35.792 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2024-04-17T09:59:35.794+0000 7f114c04b580 -1 init, newargv = 0x55dc56e41100 newargc=15 2024-04-17T09:59:35.793 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[45664]: starting ceph client 2024-04-17T09:59:35.799 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[45664]: starting fuse 2024-04-17T09:59:35.802 INFO:teuthology.orchestra.run.smithi032.stdout:44 2024-04-17T09:59:35.802 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-17T09:59:35.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2024-04-17T09:59:35.802 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2024-04-17T09:59:35.802 DEBUG:teuthology.orchestra.run.smithi032:> import re 2024-04-17T09:59:35.802 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T09:59:35.802 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2024-04-17T09:59:35.802 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:59:35.803 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2024-04-17T09:59:35.803 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T09:59:35.803 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2024-04-17T09:59:35.803 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T09:59:35.803 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:59:35.803 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2024-04-17T09:59:35.803 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2024-04-17T09:59:35.803 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2024-04-17T09:59:35.803 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2024-04-17T09:59:35.803 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:59:35.803 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2024-04-17T09:59:35.803 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T09:59:35.803 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T09:59:35.803 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T09:59:35.803 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2024-04-17T09:59:35.803 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2024-04-17T09:59:35.804 DEBUG:teuthology.orchestra.run.smithi032:> return f 2024-04-17T09:59:35.804 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T09:59:35.804 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T09:59:35.804 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2024-04-17T09:59:35.804 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T09:59:35.928 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.45664.asok 2024-04-17T09:59:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T09:59:35.940 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45664.asok 2024-04-17T09:59:35.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:35.941 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.45664.asok status 2024-04-17T09:59:36.117 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:59:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2024-04-17T09:59:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-17T09:59:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T09:59:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2024-04-17T09:59:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2024-04-17T09:59:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T09:59:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "45664", 2024-04-17T09:59:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2024-04-17T09:59:36.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2024-04-17T09:59:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2024-04-17T09:59:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 7932, 2024-04-17T09:59:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2024-04-17T09:59:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2024-04-17T09:59:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2024-04-17T09:59:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 7932 2024-04-17T09:59:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T09:59:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T09:59:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T09:59:36.118 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1497385711 2024-04-17T09:59:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T09:59:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T09:59:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T09:59:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1497385711 2024-04-17T09:59:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T09:59:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.7932 192.168.0.1:0/1497385711", 2024-04-17T09:59:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/1497385711", 2024-04-17T09:59:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2024-04-17T09:59:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 65, 2024-04-17T09:59:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 83, 2024-04-17T09:59:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2024-04-17T09:59:36.119 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2024-04-17T09:59:36.120 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs" 2024-04-17T09:59:36.120 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:59:36.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:36.133 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2024-04-17T09:59:36.133 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T09:59:36.143 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2024-04-17T09:59:36.144 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T09:59:36.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:36.144 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T09:59:36.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:36.211 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-04-17T09:59:36.266 INFO:tasks.ceph.mds.c.smithi032.stderr:2024-04-17T09:59:36.270+0000 7f3482a00640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-17T09:59:36.267 INFO:teuthology.orchestra.run.smithi032.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-04-17T09:59:36.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T09:59:36.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:36.268 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-04-17T09:59:36.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.458+0000 7f6603200640 1 Processor -- start 2024-04-17T09:59:36.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.458+0000 7f6603200640 1 -- start start 2024-04-17T09:59:36.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.458+0000 7f6603200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f65fc161ef0 0x7f65fc0b6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.458+0000 7f6603200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f65fc0d4d00 0x7f65fc0b65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.458+0000 7f6603200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65fc06df00 con 0x7f65fc0d3060 2024-04-17T09:59:36.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.458+0000 7f6603200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f65fc0ced50 con 0x7f65fc0d4d00 2024-04-17T09:59:36.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.458+0000 7f6603200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f65fc06e5e0 con 0x7f65fc161ef0 2024-04-17T09:59:36.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.458+0000 7f6601400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f65fc0d4d00 0x7f65fc0b65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.458+0000 7f6602800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f65fc0d3060 0x7f65fc06cb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45694/0 (socket says 172.21.15.32:45694) 2024-04-17T09:59:36.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.458+0000 7f6602800640 1 -- 172.21.15.32:0/3942875304 learned_addr learned my addr 172.21.15.32:0/3942875304 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T09:59:36.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.458+0000 7f6601e00640 1 --2- 172.21.15.32:0/3942875304 >> v2:172.21.15.144:3300/0 conn(0x7f65fc161ef0 0x7f65fc0b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.458+0000 7f6601400640 1 -- 172.21.15.32:0/3942875304 >> v2:172.21.15.144:3300/0 conn(0x7f65fc161ef0 msgr2=0x7f65fc0b6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:36.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.458+0000 7f6601400640 1 --2- 172.21.15.32:0/3942875304 >> v2:172.21.15.144:3300/0 conn(0x7f65fc161ef0 0x7f65fc0b6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.458+0000 7f6601400640 1 -- 172.21.15.32:0/3942875304 >> v1:172.21.15.32:6789/0 conn(0x7f65fc0d3060 legacy=0x7f65fc06cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:36.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.458+0000 7f6601400640 1 -- 172.21.15.32:0/3942875304 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65fc06ea40 con 0x7f65fc0d4d00 2024-04-17T09:59:36.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.462+0000 7f6601400640 1 --2- 172.21.15.32:0/3942875304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65fc0d4d00 0x7f65fc0b65b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f65fc073b50 tx=0x7f65ec01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e674be375b18df6c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:36.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.462+0000 7f6600a00640 1 -- 172.21.15.32:0/3942875304 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec0b0070 con 0x7f65fc0d4d00 2024-04-17T09:59:36.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.462+0000 7f6600a00640 1 -- 172.21.15.32:0/3942875304 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65ec0ab600 con 0x7f65fc0d4d00 2024-04-17T09:59:36.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.462+0000 7f6600a00640 1 -- 172.21.15.32:0/3942875304 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec0ab9d0 con 0x7f65fc0d4d00 2024-04-17T09:59:36.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.462+0000 7f6603200640 1 -- 172.21.15.32:0/3942875304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65fc0d4d00 msgr2=0x7f65fc0b65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:36.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.462+0000 7f6603200640 1 --2- 172.21.15.32:0/3942875304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65fc0d4d00 0x7f65fc0b65b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f65fc073b50 tx=0x7f65ec01c460 comp rx=0 tx=0).stop 2024-04-17T09:59:36.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.462+0000 7f6603200640 1 -- 172.21.15.32:0/3942875304 shutdown_connections 2024-04-17T09:59:36.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.462+0000 7f6603200640 1 --2- 172.21.15.32:0/3942875304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65fc0d4d00 0x7f65fc0b65b0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.462+0000 7f6603200640 1 --2- 172.21.15.32:0/3942875304 >> v2:172.21.15.144:3300/0 conn(0x7f65fc161ef0 0x7f65fc0b6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.462+0000 7f6603200640 1 -- 172.21.15.32:0/3942875304 >> 172.21.15.32:0/3942875304 conn(0x7f65fc08bf80 msgr2=0x7f65fc0d45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:36.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.462+0000 7f6603200640 1 -- 172.21.15.32:0/3942875304 shutdown_connections 2024-04-17T09:59:36.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.462+0000 7f6603200640 1 -- 172.21.15.32:0/3942875304 wait complete. 2024-04-17T09:59:36.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f6603200640 1 Processor -- start 2024-04-17T09:59:36.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f6603200640 1 -- start start 2024-04-17T09:59:36.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f6603200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65fc0d3060 0x7f65fc0b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f6603200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65fc0d4d00 0x7f65fc0c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f6603200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65fc161ef0 0x7f65fc0c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f6603200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc16b8d0 con 0x7f65fc161ef0 2024-04-17T09:59:36.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f6603200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc0d2bd0 con 0x7f65fc0d3060 2024-04-17T09:59:36.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f6603200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f65fc16ba50 con 0x7f65fc0d4d00 2024-04-17T09:59:36.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f6602800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65fc161ef0 0x7f65fc0c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f6602800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65fc161ef0 0x7f65fc0c81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42802/0 (socket says 172.21.15.32:42802) 2024-04-17T09:59:36.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f6602800640 1 -- 172.21.15.32:0/878142592 learned_addr learned my addr 172.21.15.32:0/878142592 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:36.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f6601400640 1 --2- 172.21.15.32:0/878142592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65fc0d4d00 0x7f65fc0c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f6601e00640 1 --2- 172.21.15.32:0/878142592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65fc0d3060 0x7f65fc0b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f6602800640 1 -- 172.21.15.32:0/878142592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65fc0d4d00 msgr2=0x7f65fc0c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:36.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f6602800640 1 --2- 172.21.15.32:0/878142592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65fc0d4d00 0x7f65fc0c7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f6602800640 1 -- 172.21.15.32:0/878142592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65fc0d3060 msgr2=0x7f65fc0b77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:36.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f6602800640 1 --2- 172.21.15.32:0/878142592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65fc0d3060 0x7f65fc0b77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f6602800640 1 -- 172.21.15.32:0/878142592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65fc0b2fd0 con 0x7f65fc161ef0 2024-04-17T09:59:36.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f6602800640 1 --2- 172.21.15.32:0/878142592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65fc161ef0 0x7f65fc0c81e0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f65f800ca80 tx=0x7f65f800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:36.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f65f6000640 1 -- 172.21.15.32:0/878142592 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f8017070 con 0x7f65fc161ef0 2024-04-17T09:59:36.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f65f6000640 1 -- 172.21.15.32:0/878142592 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f800ac30 con 0x7f65fc161ef0 2024-04-17T09:59:36.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f6603200640 1 -- 172.21.15.32:0/878142592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65fc0b32b0 con 0x7f65fc161ef0 2024-04-17T09:59:36.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f65f6000640 1 -- 172.21.15.32:0/878142592 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f8012600 con 0x7f65fc161ef0 2024-04-17T09:59:36.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.466+0000 7f6603200640 1 -- 172.21.15.32:0/878142592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65fc0b9bb0 con 0x7f65fc161ef0 2024-04-17T09:59:36.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.470+0000 7f65f6000640 1 -- 172.21.15.32:0/878142592 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f65f801a070 con 0x7f65fc161ef0 2024-04-17T09:59:36.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.470+0000 7f65f6000640 1 --2- 172.21.15.32:0/878142592 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f65e0068380 0x7f65e006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.470+0000 7f65f6000640 1 -- 172.21.15.32:0/878142592 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f65f8086600 con 0x7f65fc161ef0 2024-04-17T09:59:36.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.470+0000 7f6601e00640 1 --2- 172.21.15.32:0/878142592 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f65e0068380 0x7f65e006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.470+0000 7f6607cf9000 1 Processor -- start 2024-04-17T09:59:36.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.470+0000 7f6607cf9000 1 -- start start 2024-04-17T09:59:36.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.470+0000 7f6601e00640 1 --2- 172.21.15.32:0/878142592 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f65e0068380 0x7f65e006a840 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f65fc0ce560 tx=0x7f65f0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:36.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.470+0000 7f6607cf9000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564456e884d0 0x564456e888b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.470+0000 7f6607cf9000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564456e88df0 0x564456e3b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.470+0000 7f6607cf9000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e3bbb0 0x564456e3bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.470+0000 7f6607cf9000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564456c8d5b0 con 0x564456e3bbb0 2024-04-17T09:59:36.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.470+0000 7f6607cf9000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564456c46780 con 0x564456e884d0 2024-04-17T09:59:36.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.470+0000 7f6607cf9000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x564456c89570 con 0x564456e88df0 2024-04-17T09:59:36.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.470+0000 7f6602800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e3bbb0 0x564456e3bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.470+0000 7f6602800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e3bbb0 0x564456e3bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42810/0 (socket says 172.21.15.32:42810) 2024-04-17T09:59:36.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.470+0000 7f6602800640 1 -- 172.21.15.32:0/947435313 learned_addr learned my addr 172.21.15.32:0/947435313 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:36.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.470+0000 7f6601400640 1 --2- 172.21.15.32:0/947435313 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564456e88df0 0x564456e3b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.470+0000 7f6601e00640 1 --2- 172.21.15.32:0/947435313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564456e884d0 0x564456e888b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6602800640 1 -- 172.21.15.32:0/947435313 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564456e88df0 msgr2=0x564456e3b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:36.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6602800640 1 --2- 172.21.15.32:0/947435313 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564456e88df0 0x564456e3b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6602800640 1 -- 172.21.15.32:0/947435313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564456e884d0 msgr2=0x564456e888b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:36.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6602800640 1 --2- 172.21.15.32:0/947435313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564456e884d0 0x564456e888b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6602800640 1 -- 172.21.15.32:0/947435313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564456c65f80 con 0x564456e3bbb0 2024-04-17T09:59:36.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6602800640 1 --2- 172.21.15.32:0/947435313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e3bbb0 0x564456e3bf90 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f65f8052960 tx=0x7f65f8056500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6473ce2fcbfd9783 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:36.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f65eaa00640 1 -- 172.21.15.32:0/947435313 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f806ae60 con 0x564456e3bbb0 2024-04-17T09:59:36.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f65eaa00640 1 -- 172.21.15.32:0/947435313 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f8054800 con 0x564456e3bbb0 2024-04-17T09:59:36.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f65eaa00640 1 -- 172.21.15.32:0/947435313 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f805eba0 con 0x564456e3bbb0 2024-04-17T09:59:36.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6607cf9000 1 -- 172.21.15.32:0/947435313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e3bbb0 msgr2=0x564456e3bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:36.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6607cf9000 1 --2- 172.21.15.32:0/947435313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e3bbb0 0x564456e3bf90 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f65f8052960 tx=0x7f65f8056500 comp rx=0 tx=0).stop 2024-04-17T09:59:36.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6607cf9000 1 -- 172.21.15.32:0/947435313 shutdown_connections 2024-04-17T09:59:36.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6607cf9000 1 --2- 172.21.15.32:0/947435313 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564456e88df0 0x564456e3b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6607cf9000 1 --2- 172.21.15.32:0/947435313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e3bbb0 0x564456e3bf90 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6607cf9000 1 --2- 172.21.15.32:0/947435313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564456e884d0 0x564456e888b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6607cf9000 1 -- 172.21.15.32:0/947435313 >> 172.21.15.32:0/947435313 conn(0x564456e69270 msgr2=0x564456e6b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:36.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6607cf9000 1 -- 172.21.15.32:0/947435313 shutdown_connections 2024-04-17T09:59:36.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6607cf9000 1 -- 172.21.15.32:0/947435313 wait complete. 2024-04-17T09:59:36.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6607cf9000 1 -- start start 2024-04-17T09:59:36.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6607cf9000 1 Processor -- start 2024-04-17T09:59:36.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6607cf9000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564456e3bbb0 0x564456e8f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6607cf9000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564456e884d0 0x564456e8fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6607cf9000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e88df0 0x564456e414f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6607cf9000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564456d0e400 con 0x564456e88df0 2024-04-17T09:59:36.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6607cf9000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564456d492d0 con 0x564456e3bbb0 2024-04-17T09:59:36.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6607cf9000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x564456c52380 con 0x564456e884d0 2024-04-17T09:59:36.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6607cf9000 10 client.0 _ll_register_callbacks cb 0x564456e29570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:59:36.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6602800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e88df0 0x564456e414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6602800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e88df0 0x564456e414f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42820/0 (socket says 172.21.15.32:42820) 2024-04-17T09:59:36.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6602800640 1 -- 172.21.15.32:0/3541372119 learned_addr learned my addr 172.21.15.32:0/3541372119 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:36.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6601e00640 1 --2- 172.21.15.32:0/3541372119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564456e3bbb0 0x564456e8f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6601400640 1 --2- 172.21.15.32:0/3541372119 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564456e884d0 0x564456e8fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6602800640 1 -- 172.21.15.32:0/3541372119 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564456e884d0 msgr2=0x564456e8fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:36.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6602800640 1 --2- 172.21.15.32:0/3541372119 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564456e884d0 0x564456e8fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6602800640 1 -- 172.21.15.32:0/3541372119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564456e3bbb0 msgr2=0x564456e8f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:36.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6602800640 1 --2- 172.21.15.32:0/3541372119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564456e3bbb0 0x564456e8f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6602800640 1 -- 172.21.15.32:0/3541372119 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564456c61fd0 con 0x564456e88df0 2024-04-17T09:59:36.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.474+0000 7f6601400640 1 --2- 172.21.15.32:0/3541372119 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564456e884d0 0x564456e8fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:36.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6602800640 1 --2- 172.21.15.32:0/3541372119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e88df0 0x564456e414f0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f65f8052960 tx=0x7f65f800a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3a61497ea97bcc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:36.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f65df400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:59:36.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f65df400640 1 -- 172.21.15.32:0/3541372119 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f8004b70 con 0x564456e88df0 2024-04-17T09:59:36.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f65df400640 1 -- 172.21.15.32:0/3541372119 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f65f8004d10 con 0x564456e88df0 2024-04-17T09:59:36.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6607cf9000 1 -- 172.21.15.32:0/3541372119 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65f8054800 con 0x564456e88df0 2024-04-17T09:59:36.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f65df400640 1 -- 172.21.15.32:0/3541372119 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f65f8055950 con 0x564456e88df0 2024-04-17T09:59:36.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6607cf9000 10 client.7947 fetch_fsmap learned FSMap version 65 2024-04-17T09:59:36.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6607cf9000 1 -- 172.21.15.32:0/3541372119 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x564456c58f70 con 0x564456e88df0 2024-04-17T09:59:36.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f65df400640 1 -- 172.21.15.32:0/3541372119 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f8054800 con 0x564456e88df0 2024-04-17T09:59:36.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f65df400640 1 -- 172.21.15.32:0/3541372119 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 65) ==== 3495+0+0 (secure 0 0 0) 0x7f65f8055af0 con 0x564456e88df0 2024-04-17T09:59:36.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6607cf9000 10 client.7947 fetch_fsmap finished waiting for FSMap version 65 2024-04-17T09:59:36.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6607cf9000 10 client.7947 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T09:59:36.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6607cf9000 20 client.7947 populate_metadata read hostname 'smithi032' 2024-04-17T09:59:36.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6607cf9000 1 --2- 172.21.15.32:0/3541372119 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x564456e41f00 0x564456e42310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6607cf9000 4 client.7947 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:59:36.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6607cf9000 1 -- 172.21.15.32:0/3541372119 --> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564456c26f70 con 0x564456e41f00 2024-04-17T09:59:36.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6601e00640 1 --2- 172.21.15.32:0/3541372119 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x564456e41f00 0x564456e42310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6601e00640 1 --2- 172.21.15.32:0/3541372119 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x564456e41f00 0x564456e42310 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be2338267becbeda server_cookie=bc1290ab5d68622e in_seq=0 out_seq=0 2024-04-17T09:59:36.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f65df400640 10 client.7947 ms_handle_connect on v2:172.21.15.32:6835/1253497636 2024-04-17T09:59:36.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f65df400640 1 -- 172.21.15.32:0/3541372119 <== mds.0 v2:172.21.15.32:6835/1253497636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564456c26f70 con 0x564456e41f00 2024-04-17T09:59:36.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f65df400640 10 client.7947 handle_command_reply: tid=0 2024-04-17T09:59:36.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6607cf9000 1 client.7947 shutdown 2024-04-17T09:59:36.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6607cf9000 1 -- 172.21.15.32:0/3541372119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e88df0 msgr2=0x564456e414f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:36.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6607cf9000 1 --2- 172.21.15.32:0/3541372119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e88df0 0x564456e414f0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f65f8052960 tx=0x7f65f800a6a0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6607cf9000 1 -- 172.21.15.32:0/3541372119 shutdown_connections 2024-04-17T09:59:36.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6607cf9000 1 --2- 172.21.15.32:0/3541372119 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564456e884d0 0x564456e8fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6607cf9000 1 --2- 172.21.15.32:0/3541372119 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x564456e41f00 0x564456e42310 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6607cf9000 1 --2- 172.21.15.32:0/3541372119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e88df0 0x564456e414f0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6607cf9000 1 --2- 172.21.15.32:0/3541372119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564456e3bbb0 0x564456e8f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f65df400640 0 client.7947 ms_handle_reset on v2:172.21.15.32:6835/1253497636 2024-04-17T09:59:36.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6607cf9000 1 -- 172.21.15.32:0/3541372119 >> 172.21.15.32:0/3541372119 conn(0x564456e69270 msgr2=0x564456e3ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:36.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6607cf9000 1 -- 172.21.15.32:0/3541372119 shutdown_connections 2024-04-17T09:59:36.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6607cf9000 1 -- 172.21.15.32:0/3541372119 wait complete. 2024-04-17T09:59:36.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.478+0000 7f6607cf9000 20 client.7947 trim_cache size 0 max 16384 2024-04-17T09:59:36.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6607cf9000 1 Processor -- start 2024-04-17T09:59:36.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6607cf9000 1 -- start start 2024-04-17T09:59:36.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6607cf9000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e69270 0x564456e43950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6607cf9000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564456e3bbb0 0x564456e400b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6607cf9000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564456e884d0 0x564456e405f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6607cf9000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc203190 con 0x564456e69270 2024-04-17T09:59:36.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6607cf9000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc2024f0 con 0x564456e884d0 2024-04-17T09:59:36.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6607cf9000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f65fc202b40 con 0x564456e3bbb0 2024-04-17T09:59:36.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6601e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e69270 0x564456e43950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6601e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e69270 0x564456e43950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42828/0 (socket says 172.21.15.32:42828) 2024-04-17T09:59:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6601e00640 1 -- 172.21.15.32:0/1503808276 learned_addr learned my addr 172.21.15.32:0/1503808276 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6601400640 1 --2- 172.21.15.32:0/1503808276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564456e3bbb0 0x564456e400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6601e00640 1 -- 172.21.15.32:0/1503808276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564456e3bbb0 msgr2=0x564456e400b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6601e00640 1 --2- 172.21.15.32:0/1503808276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564456e3bbb0 0x564456e400b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6602800640 1 --2- 172.21.15.32:0/1503808276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564456e884d0 0x564456e405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6601e00640 1 -- 172.21.15.32:0/1503808276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564456e884d0 msgr2=0x564456e405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6601e00640 1 --2- 172.21.15.32:0/1503808276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564456e884d0 0x564456e405f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6601e00640 1 -- 172.21.15.32:0/1503808276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564456c47ae0 con 0x564456e69270 2024-04-17T09:59:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6601400640 1 --2- 172.21.15.32:0/1503808276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564456e3bbb0 0x564456e400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:59:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6602800640 1 --2- 172.21.15.32:0/1503808276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564456e884d0 0x564456e405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T09:59:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6601e00640 1 --2- 172.21.15.32:0/1503808276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e69270 0x564456e43950 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f65f000b750 tx=0x7f65f0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f8f51a0766e9a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f65ea000640 1 -- 172.21.15.32:0/1503808276 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f0030070 con 0x564456e69270 2024-04-17T09:59:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f65ea000640 1 -- 172.21.15.32:0/1503808276 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f001c040 con 0x564456e69270 2024-04-17T09:59:36.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f65ea000640 1 -- 172.21.15.32:0/1503808276 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f001b040 con 0x564456e69270 2024-04-17T09:59:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6607cf9000 1 -- 172.21.15.32:0/1503808276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e69270 msgr2=0x564456e43950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6607cf9000 1 --2- 172.21.15.32:0/1503808276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e69270 0x564456e43950 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f65f000b750 tx=0x7f65f0015000 comp rx=0 tx=0).stop 2024-04-17T09:59:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6607cf9000 1 -- 172.21.15.32:0/1503808276 shutdown_connections 2024-04-17T09:59:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6607cf9000 1 --2- 172.21.15.32:0/1503808276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564456e3bbb0 0x564456e400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6607cf9000 1 --2- 172.21.15.32:0/1503808276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564456e884d0 0x564456e405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6607cf9000 1 --2- 172.21.15.32:0/1503808276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e69270 0x564456e43950 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6607cf9000 1 -- 172.21.15.32:0/1503808276 >> 172.21.15.32:0/1503808276 conn(0x564456e6bc00 msgr2=0x564456e9fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6607cf9000 1 -- 172.21.15.32:0/1503808276 shutdown_connections 2024-04-17T09:59:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.498+0000 7f6607cf9000 1 -- 172.21.15.32:0/1503808276 wait complete. 2024-04-17T09:59:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6607cf9000 1 -- start start 2024-04-17T09:59:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6607cf9000 1 Processor -- start 2024-04-17T09:59:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6607cf9000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564456e3bbb0 0x564456ef7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6607cf9000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564456e69270 0x564456ef7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6607cf9000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e884d0 0x564456e91480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6607cf9000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564456c8c8c0 con 0x564456e884d0 2024-04-17T09:59:36.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6607cf9000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564456c4f630 con 0x564456e3bbb0 2024-04-17T09:59:36.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6607cf9000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x564456c91be0 con 0x564456e69270 2024-04-17T09:59:36.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6607cf9000 10 client.0 _ll_register_callbacks cb 0x7f65f806d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:59:36.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6602800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e884d0 0x564456e91480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6602800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e884d0 0x564456e91480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42838/0 (socket says 172.21.15.32:42838) 2024-04-17T09:59:36.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6602800640 1 -- 172.21.15.32:0/275974937 learned_addr learned my addr 172.21.15.32:0/275974937 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:36.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6601e00640 1 --2- 172.21.15.32:0/275974937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564456e3bbb0 0x564456ef7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6601400640 1 --2- 172.21.15.32:0/275974937 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564456e69270 0x564456ef7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6602800640 1 -- 172.21.15.32:0/275974937 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564456e69270 msgr2=0x564456ef7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:36.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6602800640 1 --2- 172.21.15.32:0/275974937 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564456e69270 0x564456ef7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6602800640 1 -- 172.21.15.32:0/275974937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564456e3bbb0 msgr2=0x564456ef7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:36.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6602800640 1 --2- 172.21.15.32:0/275974937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564456e3bbb0 0x564456ef7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6602800640 1 -- 172.21.15.32:0/275974937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564456d0e890 con 0x564456e884d0 2024-04-17T09:59:36.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6601e00640 1 --2- 172.21.15.32:0/275974937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564456e3bbb0 0x564456ef7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T09:59:36.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6602800640 1 --2- 172.21.15.32:0/275974937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e884d0 0x564456e91480 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f65f8084da0 tx=0x7f65f80528d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772c64712a3cdfcc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:36.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f65df400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:59:36.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f65df400640 1 -- 172.21.15.32:0/275974937 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f8086120 con 0x564456e884d0 2024-04-17T09:59:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f65df400640 1 -- 172.21.15.32:0/275974937 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f65f80862c0 con 0x564456e884d0 2024-04-17T09:59:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6607cf9000 1 -- 172.21.15.32:0/275974937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65f001c040 con 0x564456e884d0 2024-04-17T09:59:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f65df400640 1 -- 172.21.15.32:0/275974937 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f65f8052250 con 0x564456e884d0 2024-04-17T09:59:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6607cf9000 10 client.7953 fetch_fsmap learned FSMap version 65 2024-04-17T09:59:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6607cf9000 1 -- 172.21.15.32:0/275974937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x564456c326c0 con 0x564456e884d0 2024-04-17T09:59:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f65df400640 1 -- 172.21.15.32:0/275974937 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f001c040 con 0x564456e884d0 2024-04-17T09:59:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f65df400640 1 -- 172.21.15.32:0/275974937 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 65) ==== 3495+0+0 (secure 0 0 0) 0x7f65f80523f0 con 0x564456e884d0 2024-04-17T09:59:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6607cf9000 10 client.7953 fetch_fsmap finished waiting for FSMap version 65 2024-04-17T09:59:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6607cf9000 10 client.7953 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T09:59:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6607cf9000 20 client.7953 populate_metadata read hostname 'smithi032' 2024-04-17T09:59:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6607cf9000 1 --2- 172.21.15.32:0/275974937 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x564456e88df0 0x564456e91de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6607cf9000 4 client.7953 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-04-17T09:59:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6607cf9000 1 -- 172.21.15.32:0/275974937 --> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) -- 0x564456c9afd0 con 0x564456e88df0 2024-04-17T09:59:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6601e00640 1 --2- 172.21.15.32:0/275974937 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x564456e88df0 0x564456e91de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f6601e00640 1 --2- 172.21.15.32:0/275974937 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x564456e88df0 0x564456e91de0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f3450ccea79fc92 server_cookie=3082dbc97c4c59ce in_seq=0 out_seq=0 2024-04-17T09:59:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.502+0000 7f65df400640 10 client.7953 ms_handle_connect on v2:172.21.15.32:6835/1253497636 2024-04-17T09:59:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f65df400640 1 -- 172.21.15.32:0/275974937 <== mds.0 v2:172.21.15.32:6835/1253497636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564456c9afd0 con 0x564456e88df0 2024-04-17T09:59:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f65df400640 10 client.7953 handle_command_reply: tid=0 2024-04-17T09:59:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6607cf9000 1 client.7953 shutdown 2024-04-17T09:59:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6607cf9000 1 -- 172.21.15.32:0/275974937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e884d0 msgr2=0x564456e91480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6607cf9000 1 --2- 172.21.15.32:0/275974937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e884d0 0x564456e91480 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f65f8084da0 tx=0x7f65f80528d0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6607cf9000 1 -- 172.21.15.32:0/275974937 shutdown_connections 2024-04-17T09:59:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6607cf9000 1 --2- 172.21.15.32:0/275974937 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564456e69270 0x564456ef7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6607cf9000 1 --2- 172.21.15.32:0/275974937 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x564456e88df0 0x564456e91de0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6607cf9000 1 --2- 172.21.15.32:0/275974937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564456e884d0 0x564456e91480 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6607cf9000 1 --2- 172.21.15.32:0/275974937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564456e3bbb0 0x564456ef7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f65df400640 0 client.7953 ms_handle_reset on v2:172.21.15.32:6835/1253497636 2024-04-17T09:59:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6607cf9000 1 -- 172.21.15.32:0/275974937 >> 172.21.15.32:0/275974937 conn(0x564456e6bc00 msgr2=0x564456e40940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:36.507 INFO:tasks.ceph.mds.c.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f3482a00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-04-17T09:59:18.000169+0000 1=1+0) n(v0 rc2024-04-17T09:59:18.115350+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x55d3b591a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-17T09:59:18.000169+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-17T09:59:18.115350+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-17T09:59:36.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6607cf9000 1 -- 172.21.15.32:0/275974937 shutdown_connections 2024-04-17T09:59:36.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6607cf9000 1 -- 172.21.15.32:0/275974937 wait complete. 2024-04-17T09:59:36.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6607cf9000 20 client.7953 trim_cache size 0 max 16384 2024-04-17T09:59:36.508 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:59:36.508 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-17T09:59:36.508 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "df9be35b-6edd-4265-86f5-ee54633903ae", 2024-04-17T09:59:36.508 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-17T09:59:36.508 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:59:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6603200640 1 -- 172.21.15.32:0/878142592 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f65e0068380 msgr2=0x7f65e006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6603200640 1 --2- 172.21.15.32:0/878142592 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f65e0068380 0x7f65e006a840 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f65fc0ce560 tx=0x7f65f0008040 comp rx=0 tx=0).stop 2024-04-17T09:59:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6603200640 1 -- 172.21.15.32:0/878142592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65fc161ef0 msgr2=0x7f65fc0c81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6603200640 1 --2- 172.21.15.32:0/878142592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65fc161ef0 0x7f65fc0c81e0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f65f800ca80 tx=0x7f65f800cf40 comp rx=0 tx=0).stop 2024-04-17T09:59:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6603200640 1 -- 172.21.15.32:0/878142592 shutdown_connections 2024-04-17T09:59:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6603200640 1 --2- 172.21.15.32:0/878142592 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f65e0068380 0x7f65e006a840 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6603200640 1 --2- 172.21.15.32:0/878142592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65fc0d4d00 0x7f65fc0c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6603200640 1 --2- 172.21.15.32:0/878142592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65fc161ef0 0x7f65fc0c81e0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6603200640 1 --2- 172.21.15.32:0/878142592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65fc0d3060 0x7f65fc0b77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6603200640 1 -- 172.21.15.32:0/878142592 >> 172.21.15.32:0/878142592 conn(0x7f65fc08bf80 msgr2=0x7f65fc0b46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6603200640 1 -- 172.21.15.32:0/878142592 shutdown_connections 2024-04-17T09:59:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.506+0000 7f6603200640 1 -- 172.21.15.32:0/878142592 wait complete. 2024-04-17T09:59:36.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T09:59:36.522 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-17T09:59:36.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.666+0000 7f107f600640 1 Processor -- start 2024-04-17T09:59:36.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.666+0000 7f107f600640 1 -- start start 2024-04-17T09:59:36.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.670+0000 7f107f600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f107806ba30 0x7f107806be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.670+0000 7f107f600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1078071e60 0x7f1078072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.670+0000 7f107f600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f107806c350 con 0x7f10780cf370 2024-04-17T09:59:36.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.670+0000 7f107f600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f10780cfef0 con 0x7f107806ba30 2024-04-17T09:59:36.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.670+0000 7f107f600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f10780d0130 con 0x7f1078071e60 2024-04-17T09:59:36.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.670+0000 7f107cc00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f107806ba30 0x7f107806be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.670+0000 7f107cc00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f107806ba30 0x7f107806be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42852/0 (socket says 172.21.15.32:42852) 2024-04-17T09:59:36.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.670+0000 7f107cc00640 1 -- 172.21.15.32:0/634350230 learned_addr learned my addr 172.21.15.32:0/634350230 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:36.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.670+0000 7f107cc00640 1 -- 172.21.15.32:0/634350230 >> v2:172.21.15.144:3300/0 conn(0x7f1078071e60 msgr2=0x7f1078072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:36.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.670+0000 7f107cc00640 1 --2- 172.21.15.32:0/634350230 >> v2:172.21.15.144:3300/0 conn(0x7f1078071e60 0x7f1078072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.670+0000 7f107cc00640 1 -- 172.21.15.32:0/634350230 >> v1:172.21.15.32:6789/0 conn(0x7f10780cf370 legacy=0x7f10780cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:36.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.670+0000 7f107cc00640 1 -- 172.21.15.32:0/634350230 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107815d090 con 0x7f107806ba30 2024-04-17T09:59:36.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.674+0000 7f107cc00640 1 --2- 172.21.15.32:0/634350230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f107806ba30 0x7f107806be10 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f10780cd7e0 tx=0x7f106c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ec103cba8c9f24 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:36.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.674+0000 7f1077400640 1 -- 172.21.15.32:0/634350230 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c0b2070 con 0x7f107806ba30 2024-04-17T09:59:36.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.674+0000 7f1077400640 1 -- 172.21.15.32:0/634350230 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f106c0a9070 con 0x7f107806ba30 2024-04-17T09:59:36.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.674+0000 7f1077400640 1 -- 172.21.15.32:0/634350230 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c0ad880 con 0x7f107806ba30 2024-04-17T09:59:36.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.674+0000 7f107f600640 1 -- 172.21.15.32:0/634350230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f107806ba30 msgr2=0x7f107806be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:36.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.674+0000 7f107f600640 1 --2- 172.21.15.32:0/634350230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f107806ba30 0x7f107806be10 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f10780cd7e0 tx=0x7f106c01c430 comp rx=0 tx=0).stop 2024-04-17T09:59:36.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.674+0000 7f107f600640 1 -- 172.21.15.32:0/634350230 shutdown_connections 2024-04-17T09:59:36.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.674+0000 7f107f600640 1 --2- 172.21.15.32:0/634350230 >> v2:172.21.15.144:3300/0 conn(0x7f1078071e60 0x7f1078072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.674+0000 7f107f600640 1 --2- 172.21.15.32:0/634350230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f107806ba30 0x7f107806be10 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.674+0000 7f107f600640 1 -- 172.21.15.32:0/634350230 >> 172.21.15.32:0/634350230 conn(0x7f10780b2120 msgr2=0x7f10780b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:36.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.674+0000 7f107f600640 1 -- 172.21.15.32:0/634350230 shutdown_connections 2024-04-17T09:59:36.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.674+0000 7f107f600640 1 -- 172.21.15.32:0/634350230 wait complete. 2024-04-17T09:59:36.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.674+0000 7f107f600640 1 Processor -- start 2024-04-17T09:59:36.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.674+0000 7f107f600640 1 -- start start 2024-04-17T09:59:36.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f107f600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f107806ba30 0x7f10780aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f107f600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1078071e60 0x7f107813ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f107f600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10780cf370 0x7f10780b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f107f600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f107815dcf0 con 0x7f1078071e60 2024-04-17T09:59:36.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f107f600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f107815db70 con 0x7f107806ba30 2024-04-17T09:59:36.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f107f600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1078146230 con 0x7f10780cf370 2024-04-17T09:59:36.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f1077e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1078071e60 0x7f107813ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f1077e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1078071e60 0x7f107813ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42856/0 (socket says 172.21.15.32:42856) 2024-04-17T09:59:36.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f1077e00640 1 -- 172.21.15.32:0/885692678 learned_addr learned my addr 172.21.15.32:0/885692678 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:36.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f107cc00640 1 --2- 172.21.15.32:0/885692678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f107806ba30 0x7f10780aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f107d600640 1 --2- 172.21.15.32:0/885692678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10780cf370 0x7f10780b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f1077e00640 1 -- 172.21.15.32:0/885692678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10780cf370 msgr2=0x7f10780b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:36.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f1077e00640 1 --2- 172.21.15.32:0/885692678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10780cf370 0x7f10780b4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f1077e00640 1 -- 172.21.15.32:0/885692678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f107806ba30 msgr2=0x7f10780aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:36.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f1077e00640 1 --2- 172.21.15.32:0/885692678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f107806ba30 0x7f10780aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f1077e00640 1 -- 172.21.15.32:0/885692678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10780ad860 con 0x7f1078071e60 2024-04-17T09:59:36.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f107d600640 1 --2- 172.21.15.32:0/885692678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10780cf370 0x7f10780b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:36.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f1077e00640 1 --2- 172.21.15.32:0/885692678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1078071e60 0x7f107813ad60 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f106400c370 tx=0x7f106400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:36.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f1074c00640 1 -- 172.21.15.32:0/885692678 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10640052d0 con 0x7f1078071e60 2024-04-17T09:59:36.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f1074c00640 1 -- 172.21.15.32:0/885692678 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1064005470 con 0x7f1078071e60 2024-04-17T09:59:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f107f600640 1 -- 172.21.15.32:0/885692678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10780adb40 con 0x7f1078071e60 2024-04-17T09:59:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f1074c00640 1 -- 172.21.15.32:0/885692678 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1064007260 con 0x7f1078071e60 2024-04-17T09:59:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f107f600640 1 -- 172.21.15.32:0/885692678 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10780ae100 con 0x7f1078071e60 2024-04-17T09:59:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.678+0000 7f1074c00640 1 -- 172.21.15.32:0/885692678 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1064007400 con 0x7f1078071e60 2024-04-17T09:59:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f1074c00640 1 --2- 172.21.15.32:0/885692678 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1058068380 0x7f105806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f1074c00640 1 -- 172.21.15.32:0/885692678 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f1064085e20 con 0x7f1078071e60 2024-04-17T09:59:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f107cc00640 1 --2- 172.21.15.32:0/885692678 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1058068380 0x7f105806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f10829c3000 1 Processor -- start 2024-04-17T09:59:36.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f10829c3000 1 -- start start 2024-04-17T09:59:36.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f107cc00640 1 --2- 172.21.15.32:0/885692678 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1058068380 0x7f105806a840 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f106c0af000 tx=0x7f106c0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f10829c3000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209260 0x564abf209640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f10829c3000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564abf209b80 0x564abf1bc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f10829c3000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564abf1bc9a0 0x564abf1bcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f10829c3000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564abf00e4f0 con 0x564abf209260 2024-04-17T09:59:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f10829c3000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564abefc76c0 con 0x564abf1bc9a0 2024-04-17T09:59:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f10829c3000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x564abf00a4b0 con 0x564abf209b80 2024-04-17T09:59:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f107cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209260 0x564abf209640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f107cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209260 0x564abf209640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42866/0 (socket says 172.21.15.32:42866) 2024-04-17T09:59:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f107cc00640 1 -- 172.21.15.32:0/3498882285 learned_addr learned my addr 172.21.15.32:0/3498882285 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f1077e00640 1 --2- 172.21.15.32:0/3498882285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564abf209b80 0x564abf1bc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f107d600640 1 --2- 172.21.15.32:0/3498882285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564abf1bc9a0 0x564abf1bcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f107cc00640 1 -- 172.21.15.32:0/3498882285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564abf209b80 msgr2=0x564abf1bc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f107cc00640 1 --2- 172.21.15.32:0/3498882285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564abf209b80 0x564abf1bc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f107cc00640 1 -- 172.21.15.32:0/3498882285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564abf1bc9a0 msgr2=0x564abf1bcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f107cc00640 1 --2- 172.21.15.32:0/3498882285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564abf1bc9a0 0x564abf1bcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f107cc00640 1 -- 172.21.15.32:0/3498882285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564abf0538e0 con 0x564abf209260 2024-04-17T09:59:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f107d600640 1 --2- 172.21.15.32:0/3498882285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564abf1bc9a0 0x564abf1bcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f107cc00640 1 --2- 172.21.15.32:0/3498882285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209260 0x564abf209640 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f106c019040 tx=0x7f106c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a447a319252da5ed server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f1069600640 1 -- 172.21.15.32:0/3498882285 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c0ca070 con 0x564abf209260 2024-04-17T09:59:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f1069600640 1 -- 172.21.15.32:0/3498882285 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f106c0a9040 con 0x564abf209260 2024-04-17T09:59:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f1069600640 1 -- 172.21.15.32:0/3498882285 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c0b2040 con 0x564abf209260 2024-04-17T09:59:36.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f10829c3000 1 -- 172.21.15.32:0/3498882285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209260 msgr2=0x564abf209640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:36.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.682+0000 7f10829c3000 1 --2- 172.21.15.32:0/3498882285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209260 0x564abf209640 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f106c019040 tx=0x7f106c004740 comp rx=0 tx=0).stop 2024-04-17T09:59:36.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f10829c3000 1 -- 172.21.15.32:0/3498882285 shutdown_connections 2024-04-17T09:59:36.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f10829c3000 1 --2- 172.21.15.32:0/3498882285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564abf209b80 0x564abf1bc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f10829c3000 1 --2- 172.21.15.32:0/3498882285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564abf1bc9a0 0x564abf1bcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f10829c3000 1 --2- 172.21.15.32:0/3498882285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209260 0x564abf209640 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f10829c3000 1 -- 172.21.15.32:0/3498882285 >> 172.21.15.32:0/3498882285 conn(0x564abf1ea000 msgr2=0x564abf1ec3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:36.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f10829c3000 1 -- 172.21.15.32:0/3498882285 shutdown_connections 2024-04-17T09:59:36.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f10829c3000 1 -- 172.21.15.32:0/3498882285 wait complete. 2024-04-17T09:59:36.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f10829c3000 1 -- start start 2024-04-17T09:59:36.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f10829c3000 1 Processor -- start 2024-04-17T09:59:36.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f10829c3000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564abf1bc9a0 0x564abf1c2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f10829c3000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564abf209260 0x564abf1c25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f10829c3000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209b80 0x564abf1c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f10829c3000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564abf08f7b0 con 0x564abf209b80 2024-04-17T09:59:36.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f10829c3000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564abf0c9d00 con 0x564abf209260 2024-04-17T09:59:36.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f10829c3000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x564abefd32c0 con 0x564abf1bc9a0 2024-04-17T09:59:36.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f10829c3000 10 client.0 _ll_register_callbacks cb 0x564abefe3030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:59:36.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f107d600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209b80 0x564abf1c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f107d600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209b80 0x564abf1c2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42882/0 (socket says 172.21.15.32:42882) 2024-04-17T09:59:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f107d600640 1 -- 172.21.15.32:0/1506340734 learned_addr learned my addr 172.21.15.32:0/1506340734 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f107cc00640 1 --2- 172.21.15.32:0/1506340734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564abf1bc9a0 0x564abf1c2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f1077e00640 1 --2- 172.21.15.32:0/1506340734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564abf209260 0x564abf1c25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f107d600640 1 -- 172.21.15.32:0/1506340734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564abf1bc9a0 msgr2=0x564abf1c2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f107d600640 1 --2- 172.21.15.32:0/1506340734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564abf1bc9a0 0x564abf1c2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f107d600640 1 -- 172.21.15.32:0/1506340734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564abf209260 msgr2=0x564abf1c25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f107d600640 1 --2- 172.21.15.32:0/1506340734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564abf209260 0x564abf1c25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f107d600640 1 -- 172.21.15.32:0/1506340734 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564abefe6490 con 0x564abf209b80 2024-04-17T09:59:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f107d600640 1 --2- 172.21.15.32:0/1506340734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209b80 0x564abf1c2af0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f1070009fd0 tx=0x7f107000af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bec351f7177de7e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f105e000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:59:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f105e000640 1 -- 172.21.15.32:0/1506340734 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107001b060 con 0x564abf209b80 2024-04-17T09:59:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f105e000640 1 -- 172.21.15.32:0/1506340734 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f10700163f0 con 0x564abf209b80 2024-04-17T09:59:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f10829c3000 1 -- 172.21.15.32:0/1506340734 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f106c0a9040 con 0x564abf209b80 2024-04-17T09:59:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f105e000640 1 -- 172.21.15.32:0/1506340734 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f10700165f0 con 0x564abf209b80 2024-04-17T09:59:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f10829c3000 10 client.7968 fetch_fsmap learned FSMap version 65 2024-04-17T09:59:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f10829c3000 1 -- 172.21.15.32:0/1506340734 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x564abefd9eb0 con 0x564abf209b80 2024-04-17T09:59:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.686+0000 7f105e000640 1 -- 172.21.15.32:0/1506340734 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f106c0a9040 con 0x564abf209b80 2024-04-17T09:59:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f105e000640 1 -- 172.21.15.32:0/1506340734 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 65) ==== 3495+0+0 (secure 0 0 0) 0x7f1070024080 con 0x564abf209b80 2024-04-17T09:59:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f10829c3000 10 client.7968 fetch_fsmap finished waiting for FSMap version 65 2024-04-17T09:59:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f10829c3000 10 client.7968 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T09:59:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f10829c3000 20 client.7968 populate_metadata read hostname 'smithi032' 2024-04-17T09:59:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f10829c3000 1 --2- 172.21.15.32:0/1506340734 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x564abf218500 0x564abf21a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f10829c3000 4 client.7968 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T09:59:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f10829c3000 1 -- 172.21.15.32:0/1506340734 --> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564abefa7dd0 con 0x564abf218500 2024-04-17T09:59:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f107cc00640 1 --2- 172.21.15.32:0/1506340734 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x564abf218500 0x564abf21a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f107cc00640 1 --2- 172.21.15.32:0/1506340734 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x564abf218500 0x564abf21a8f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb8498778e5c0694 server_cookie=d0908362bd04c865 in_seq=0 out_seq=0 2024-04-17T09:59:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f105e000640 10 client.7968 ms_handle_connect on v2:172.21.15.32:6835/1253497636 2024-04-17T09:59:36.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f105e000640 1 -- 172.21.15.32:0/1506340734 <== mds.0 v2:172.21.15.32:6835/1253497636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564abefa7dd0 con 0x564abf218500 2024-04-17T09:59:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f105e000640 10 client.7968 handle_command_reply: tid=0 2024-04-17T09:59:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f10829c3000 1 client.7968 shutdown 2024-04-17T09:59:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f10829c3000 1 -- 172.21.15.32:0/1506340734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209b80 msgr2=0x564abf1c2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f10829c3000 1 --2- 172.21.15.32:0/1506340734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209b80 0x564abf1c2af0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f1070009fd0 tx=0x7f107000af80 comp rx=0 tx=0).stop 2024-04-17T09:59:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f10829c3000 1 -- 172.21.15.32:0/1506340734 shutdown_connections 2024-04-17T09:59:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f10829c3000 1 --2- 172.21.15.32:0/1506340734 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x564abf218500 0x564abf21a8f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f10829c3000 1 --2- 172.21.15.32:0/1506340734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209b80 0x564abf1c2af0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f10829c3000 1 --2- 172.21.15.32:0/1506340734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564abf209260 0x564abf1c25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f10829c3000 1 --2- 172.21.15.32:0/1506340734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564abf1bc9a0 0x564abf1c2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f105e000640 0 client.7968 ms_handle_reset on v2:172.21.15.32:6835/1253497636 2024-04-17T09:59:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f10829c3000 1 -- 172.21.15.32:0/1506340734 >> 172.21.15.32:0/1506340734 conn(0x564abf1ea000 msgr2=0x564abf1ec3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f10829c3000 1 -- 172.21.15.32:0/1506340734 shutdown_connections 2024-04-17T09:59:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f10829c3000 1 -- 172.21.15.32:0/1506340734 wait complete. 2024-04-17T09:59:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.690+0000 7f10829c3000 20 client.7968 trim_cache size 0 max 16384 2024-04-17T09:59:36.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.706+0000 7f10829c3000 1 Processor -- start 2024-04-17T09:59:36.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.706+0000 7f10829c3000 1 -- start start 2024-04-17T09:59:36.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f10829c3000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564abf1bc9a0 0x564abf1bbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f10829c3000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209260 0x564abf1bc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f10829c3000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564abf209b80 0x564abf21a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f10829c3000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f10780b46c0 con 0x564abf209260 2024-04-17T09:59:36.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f10829c3000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f107814d2d0 con 0x564abf209b80 2024-04-17T09:59:36.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f10829c3000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f10780b4070 con 0x564abf1bc9a0 2024-04-17T09:59:36.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f1077e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209260 0x564abf1bc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f1077e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209260 0x564abf1bc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42886/0 (socket says 172.21.15.32:42886) 2024-04-17T09:59:36.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f1077e00640 1 -- 172.21.15.32:0/554041807 learned_addr learned my addr 172.21.15.32:0/554041807 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:36.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f107cc00640 1 --2- 172.21.15.32:0/554041807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564abf1bc9a0 0x564abf1bbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f107d600640 1 --2- 172.21.15.32:0/554041807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564abf209b80 0x564abf21a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f1077e00640 1 -- 172.21.15.32:0/554041807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564abf1bc9a0 msgr2=0x564abf1bbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:36.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f1077e00640 1 --2- 172.21.15.32:0/554041807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564abf1bc9a0 0x564abf1bbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f1077e00640 1 -- 172.21.15.32:0/554041807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564abf209b80 msgr2=0x564abf21a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:36.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f1077e00640 1 --2- 172.21.15.32:0/554041807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564abf209b80 0x564abf21a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f1077e00640 1 -- 172.21.15.32:0/554041807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564abefc8a20 con 0x564abf209260 2024-04-17T09:59:36.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f107d600640 1 --2- 172.21.15.32:0/554041807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564abf209b80 0x564abf21a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:36.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f1077e00640 1 --2- 172.21.15.32:0/554041807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209260 0x564abf1bc0a0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f1064055a20 tx=0x7f106401d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235d653783cb501e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:36.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f1068c00640 1 -- 172.21.15.32:0/554041807 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1064084360 con 0x564abf209260 2024-04-17T09:59:36.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f1068c00640 1 -- 172.21.15.32:0/554041807 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1064052390 con 0x564abf209260 2024-04-17T09:59:36.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f1068c00640 1 -- 172.21.15.32:0/554041807 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10640553b0 con 0x564abf209260 2024-04-17T09:59:36.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f10829c3000 1 -- 172.21.15.32:0/554041807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209260 msgr2=0x564abf1bc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:36.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f10829c3000 1 --2- 172.21.15.32:0/554041807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209260 0x564abf1bc0a0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f1064055a20 tx=0x7f106401d040 comp rx=0 tx=0).stop 2024-04-17T09:59:36.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f10829c3000 1 -- 172.21.15.32:0/554041807 shutdown_connections 2024-04-17T09:59:36.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f10829c3000 1 --2- 172.21.15.32:0/554041807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564abf209b80 0x564abf21a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f10829c3000 1 --2- 172.21.15.32:0/554041807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209260 0x564abf1bc0a0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f10829c3000 1 --2- 172.21.15.32:0/554041807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564abf1bc9a0 0x564abf1bbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f10829c3000 1 -- 172.21.15.32:0/554041807 >> 172.21.15.32:0/554041807 conn(0x564abf1ea000 msgr2=0x564abf1bb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:36.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f10829c3000 1 -- 172.21.15.32:0/554041807 shutdown_connections 2024-04-17T09:59:36.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f10829c3000 1 -- 172.21.15.32:0/554041807 wait complete. 2024-04-17T09:59:36.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f10829c3000 1 -- start start 2024-04-17T09:59:36.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f10829c3000 1 Processor -- start 2024-04-17T09:59:36.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f10829c3000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564abf1bc9a0 0x564abf21d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f10829c3000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209260 0x564abf234480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f10829c3000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564abf209b80 0x564abf2349c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f10829c3000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564abefb2050 con 0x564abf209260 2024-04-17T09:59:36.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f10829c3000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564abefd0570 con 0x564abf209b80 2024-04-17T09:59:36.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.710+0000 7f10829c3000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x564abf00d800 con 0x564abf1bc9a0 2024-04-17T09:59:36.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f10829c3000 10 client.0 _ll_register_callbacks cb 0x7f10700097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T09:59:36.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f1077e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209260 0x564abf234480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f1077e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209260 0x564abf234480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42892/0 (socket says 172.21.15.32:42892) 2024-04-17T09:59:36.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f1077e00640 1 -- 172.21.15.32:0/1762949750 learned_addr learned my addr 172.21.15.32:0/1762949750 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T09:59:36.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f107cc00640 1 --2- 172.21.15.32:0/1762949750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564abf1bc9a0 0x564abf21d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f107d600640 1 --2- 172.21.15.32:0/1762949750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564abf209b80 0x564abf2349c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f1077e00640 1 -- 172.21.15.32:0/1762949750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564abf1bc9a0 msgr2=0x564abf21d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:36.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f1077e00640 1 --2- 172.21.15.32:0/1762949750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564abf1bc9a0 0x564abf21d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f1077e00640 1 -- 172.21.15.32:0/1762949750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564abf209b80 msgr2=0x564abf2349c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T09:59:36.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f1077e00640 1 --2- 172.21.15.32:0/1762949750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564abf209b80 0x564abf2349c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f1077e00640 1 -- 172.21.15.32:0/1762949750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564abf08fc40 con 0x564abf209260 2024-04-17T09:59:36.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f107d600640 1 --2- 172.21.15.32:0/1762949750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564abf209b80 0x564abf2349c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T09:59:36.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f1077e00640 1 --2- 172.21.15.32:0/1762949750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209260 0x564abf234480 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f1064052180 tx=0x7f1064000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3a8413ee8440b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T09:59:36.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f105e000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T09:59:36.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f105e000640 1 -- 172.21.15.32:0/1762949750 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1064055170 con 0x564abf209260 2024-04-17T09:59:36.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f105e000640 1 -- 172.21.15.32:0/1762949750 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f1064055310 con 0x564abf209260 2024-04-17T09:59:36.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f10829c3000 1 -- 172.21.15.32:0/1762949750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1064052390 con 0x564abf209260 2024-04-17T09:59:36.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f105e000640 1 -- 172.21.15.32:0/1762949750 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f1064079050 con 0x564abf209260 2024-04-17T09:59:36.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f10829c3000 10 client.7974 fetch_fsmap learned FSMap version 65 2024-04-17T09:59:36.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f10829c3000 1 -- 172.21.15.32:0/1762949750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x564abefb3600 con 0x564abf209260 2024-04-17T09:59:36.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f105e000640 1 -- 172.21.15.32:0/1762949750 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1064052390 con 0x564abf209260 2024-04-17T09:59:36.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f105e000640 1 -- 172.21.15.32:0/1762949750 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 65) ==== 3495+0+0 (secure 0 0 0) 0x7f106407c020 con 0x564abf209260 2024-04-17T09:59:36.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f10829c3000 10 client.7974 fetch_fsmap finished waiting for FSMap version 65 2024-04-17T09:59:36.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f10829c3000 10 client.7974 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T09:59:36.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f10829c3000 20 client.7974 populate_metadata read hostname 'smithi032' 2024-04-17T09:59:36.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f10829c3000 1 --2- 172.21.15.32:0/1762949750 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x564abf2780f0 0x564abf278500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T09:59:36.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f10829c3000 4 client.7974 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T09:59:36.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f10829c3000 1 -- 172.21.15.32:0/1762949750 --> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564abf01bf10 con 0x564abf2780f0 2024-04-17T09:59:36.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f107cc00640 1 --2- 172.21.15.32:0/1762949750 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x564abf2780f0 0x564abf278500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T09:59:36.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f105e000640 10 client.7974 ms_handle_connect on v2:172.21.15.32:6835/1253497636 2024-04-17T09:59:36.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f107cc00640 1 --2- 172.21.15.32:0/1762949750 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x564abf2780f0 0x564abf278500 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a54e6e376b14f94c server_cookie=3f5d178873f46b40 in_seq=0 out_seq=0 2024-04-17T09:59:36.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f105e000640 1 -- 172.21.15.32:0/1762949750 <== mds.0 v2:172.21.15.32:6835/1253497636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+256 (crc 0 0 0) 0x564abf01bf10 con 0x564abf2780f0 2024-04-17T09:59:36.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f105e000640 10 client.7974 handle_command_reply: tid=0 2024-04-17T09:59:36.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f10829c3000 1 client.7974 shutdown 2024-04-17T09:59:36.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f10829c3000 1 -- 172.21.15.32:0/1762949750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209260 msgr2=0x564abf234480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:36.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f10829c3000 1 --2- 172.21.15.32:0/1762949750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209260 0x564abf234480 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f1064052180 tx=0x7f1064000ee0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f10829c3000 1 -- 172.21.15.32:0/1762949750 shutdown_connections 2024-04-17T09:59:36.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f10829c3000 1 --2- 172.21.15.32:0/1762949750 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x564abf2780f0 0x564abf278500 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f10829c3000 1 --2- 172.21.15.32:0/1762949750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564abf209b80 0x564abf2349c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f10829c3000 1 --2- 172.21.15.32:0/1762949750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564abf209260 0x564abf234480 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f105e000640 0 client.7974 ms_handle_reset on v2:172.21.15.32:6835/1253497636 2024-04-17T09:59:36.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f10829c3000 1 --2- 172.21.15.32:0/1762949750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564abf1bc9a0 0x564abf21d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f10829c3000 1 -- 172.21.15.32:0/1762949750 >> 172.21.15.32:0/1762949750 conn(0x564abf1ea000 msgr2=0x564abf1bbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:36.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f10829c3000 1 -- 172.21.15.32:0/1762949750 shutdown_connections 2024-04-17T09:59:36.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f10829c3000 1 -- 172.21.15.32:0/1762949750 wait complete. 2024-04-17T09:59:36.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f10829c3000 20 client.7974 trim_cache size 0 max 16384 2024-04-17T09:59:36.717 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T09:59:36.717 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T09:59:36.717 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-17T09:59:36.717 INFO:teuthology.orchestra.run.smithi032.stdout: "df9be35b-6edd-4265-86f5-ee54633903ae": { 2024-04-17T09:59:36.718 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/dir", 2024-04-17T09:59:36.718 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "df9be35b-6edd-4265-86f5-ee54633903ae", 2024-04-17T09:59:36.718 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive" 2024-04-17T09:59:36.718 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:36.718 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T09:59:36.718 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T09:59:36.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f107f600640 1 -- 172.21.15.32:0/885692678 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1058068380 msgr2=0x7f105806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:36.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f107f600640 1 --2- 172.21.15.32:0/885692678 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1058068380 0x7f105806a840 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f106c0af000 tx=0x7f106c0a7000 comp rx=0 tx=0).stop 2024-04-17T09:59:36.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f107f600640 1 -- 172.21.15.32:0/885692678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1078071e60 msgr2=0x7f107813ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T09:59:36.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f107f600640 1 --2- 172.21.15.32:0/885692678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1078071e60 0x7f107813ad60 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f106400c370 tx=0x7f106400c830 comp rx=0 tx=0).stop 2024-04-17T09:59:36.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f107f600640 1 -- 172.21.15.32:0/885692678 shutdown_connections 2024-04-17T09:59:36.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f107f600640 1 --2- 172.21.15.32:0/885692678 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1058068380 0x7f105806a840 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f107f600640 1 --2- 172.21.15.32:0/885692678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10780cf370 0x7f10780b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f107f600640 1 --2- 172.21.15.32:0/885692678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1078071e60 0x7f107813ad60 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f107f600640 1 --2- 172.21.15.32:0/885692678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f107806ba30 0x7f10780aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T09:59:36.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.714+0000 7f107f600640 1 -- 172.21.15.32:0/885692678 >> 172.21.15.32:0/885692678 conn(0x7f10780b2120 msgr2=0x7f10780b14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T09:59:36.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.718+0000 7f107f600640 1 -- 172.21.15.32:0/885692678 shutdown_connections 2024-04-17T09:59:36.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T09:59:36.718+0000 7f107f600640 1 -- 172.21.15.32:0/885692678 wait complete. 2024-04-17T09:59:36.732 INFO:tasks.cephfs.filesystem:scrub status for tag:df9be35b-6edd-4265-86f5-ee54633903ae - {'path': '/dir', 'tag': 'df9be35b-6edd-4265-86f5-ee54633903ae', 'options': 'recursive'} 2024-04-17T09:59:40.022 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-17T09:59:40.026+0000 7f93cce00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-17T09:59:43.914 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:59:43.918 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T09:59:59.995 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-17T09:59:59.998+0000 7f93cce00640 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 MDSs report damaged metadata 2024-04-17T10:00:06.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:00:06.735 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-17T10:00:06.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.894+0000 7f2c4aa00640 1 Processor -- start 2024-04-17T10:00:06.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.894+0000 7f2c4aa00640 1 -- start start 2024-04-17T10:00:06.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.894+0000 7f2c4aa00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2c4415b010 0x7f2c4406dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:06.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.894+0000 7f2c4aa00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c4406e6f0 con 0x7f2c440b3d80 2024-04-17T10:00:06.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.894+0000 7f2c4aa00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c4406e9d0 con 0x7f2c4415ac30 2024-04-17T10:00:06.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.894+0000 7f2c4aa00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2c4406f300 con 0x7f2c4415b010 2024-04-17T10:00:06.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.894+0000 7f2c3be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2c4415b010 0x7f2c4406dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:06.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.894+0000 7f2c48a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2c440b3d80 0x7f2c44071e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39298/0 (socket says 172.21.15.32:39298) 2024-04-17T10:00:06.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.894+0000 7f2c48a00640 1 -- 172.21.15.32:0/3832697564 learned_addr learned my addr 172.21.15.32:0/3832697564 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:00:06.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.894+0000 7f2c3be00640 1 -- 172.21.15.32:0/3832697564 >> v1:172.21.15.144:6789/0 conn(0x7f2c4415ac30 legacy=0x7f2c4406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:06.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.894+0000 7f2c3be00640 1 -- 172.21.15.32:0/3832697564 >> v1:172.21.15.32:6789/0 conn(0x7f2c440b3d80 legacy=0x7f2c44071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:06.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.894+0000 7f2c3be00640 1 -- 172.21.15.32:0/3832697564 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4406ecb0 con 0x7f2c4415b010 2024-04-17T10:00:06.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.894+0000 7f2c43400640 1 -- 172.21.15.32:0/3832697564 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1267468148 0 0) 0x7f2c4406e6f0 con 0x7f2c440b3d80 2024-04-17T10:00:06.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.898+0000 7f2c3be00640 1 --2- 172.21.15.32:0/3832697564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c4415b010 0x7f2c4406dea0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0097c0 tx=0x7f2c2c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49759e7a6c439d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:06.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.898+0000 7f2c43400640 1 -- 172.21.15.32:0/3832697564 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c0ab660 con 0x7f2c4415b010 2024-04-17T10:00:06.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.898+0000 7f2c43400640 1 -- 172.21.15.32:0/3832697564 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c2c0b0070 con 0x7f2c4415b010 2024-04-17T10:00:06.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.898+0000 7f2c43400640 1 -- 172.21.15.32:0/3832697564 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c0aba20 con 0x7f2c4415b010 2024-04-17T10:00:06.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.898+0000 7f2c4aa00640 1 -- 172.21.15.32:0/3832697564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c4415b010 msgr2=0x7f2c4406dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:06.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.898+0000 7f2c4aa00640 1 --2- 172.21.15.32:0/3832697564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c4415b010 0x7f2c4406dea0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0097c0 tx=0x7f2c2c01c4a0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.898+0000 7f2c4aa00640 1 -- 172.21.15.32:0/3832697564 shutdown_connections 2024-04-17T10:00:06.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.898+0000 7f2c4aa00640 1 --2- 172.21.15.32:0/3832697564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c4415b010 0x7f2c4406dea0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.898+0000 7f2c4aa00640 1 -- 172.21.15.32:0/3832697564 >> 172.21.15.32:0/3832697564 conn(0x7f2c440b2120 msgr2=0x7f2c440b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:06.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.898+0000 7f2c4aa00640 1 -- 172.21.15.32:0/3832697564 shutdown_connections 2024-04-17T10:00:06.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.898+0000 7f2c4aa00640 1 -- 172.21.15.32:0/3832697564 wait complete. 2024-04-17T10:00:06.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c4aa00640 1 Processor -- start 2024-04-17T10:00:06.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c4aa00640 1 -- start start 2024-04-17T10:00:06.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c4aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c440b3d80 0x7f2c4414bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:06.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c4aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c4415ac30 0x7f2c440d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:06.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c4aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c4415b010 0x7f2c440d0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:06.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c4aa00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2c440cdb40 con 0x7f2c440b3d80 2024-04-17T10:00:06.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c4aa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2c440cd9c0 con 0x7f2c4415ac30 2024-04-17T10:00:06.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c4aa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2c440cdcc0 con 0x7f2c4415b010 2024-04-17T10:00:06.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c43e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c440b3d80 0x7f2c4414bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:06.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c43e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c440b3d80 0x7f2c4414bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59848/0 (socket says 172.21.15.32:59848) 2024-04-17T10:00:06.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c43e00640 1 -- 172.21.15.32:0/3506097411 learned_addr learned my addr 172.21.15.32:0/3506097411 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:06.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c3be00640 1 --2- 172.21.15.32:0/3506097411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c4415ac30 0x7f2c440d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:06.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c48a00640 1 --2- 172.21.15.32:0/3506097411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c4415b010 0x7f2c440d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:06.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c43e00640 1 -- 172.21.15.32:0/3506097411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c4415b010 msgr2=0x7f2c440d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:06.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c43e00640 1 --2- 172.21.15.32:0/3506097411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c4415b010 0x7f2c440d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c43e00640 1 -- 172.21.15.32:0/3506097411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c4415ac30 msgr2=0x7f2c440d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:06.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c43e00640 1 --2- 172.21.15.32:0/3506097411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c4415ac30 0x7f2c440d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c43e00640 1 -- 172.21.15.32:0/3506097411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4414c3a0 con 0x7f2c440b3d80 2024-04-17T10:00:06.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c3be00640 1 --2- 172.21.15.32:0/3506097411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c4415ac30 0x7f2c440d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:06.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c43e00640 1 --2- 172.21.15.32:0/3506097411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c440b3d80 0x7f2c4414bb20 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400c530 tx=0x7f2c3400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:06.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c40c00640 1 -- 172.21.15.32:0/3506097411 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34016020 con 0x7f2c440b3d80 2024-04-17T10:00:06.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c40c00640 1 -- 172.21.15.32:0/3506097411 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c34005220 con 0x7f2c440b3d80 2024-04-17T10:00:06.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c40c00640 1 -- 172.21.15.32:0/3506097411 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34007030 con 0x7f2c440b3d80 2024-04-17T10:00:06.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c4aa00640 1 -- 172.21.15.32:0/3506097411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4414c680 con 0x7f2c440b3d80 2024-04-17T10:00:06.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.902+0000 7f2c4aa00640 1 -- 172.21.15.32:0/3506097411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c4414cc40 con 0x7f2c440b3d80 2024-04-17T10:00:06.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.906+0000 7f2c40c00640 1 -- 172.21.15.32:0/3506097411 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2c340071d0 con 0x7f2c440b3d80 2024-04-17T10:00:06.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.906+0000 7f2c40c00640 1 --2- 172.21.15.32:0/3506097411 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2c200683b0 0x7f2c2006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:06.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.906+0000 7f2c40c00640 1 -- 172.21.15.32:0/3506097411 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7f2c340863f0 con 0x7f2c440b3d80 2024-04-17T10:00:06.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.906+0000 7f2c3be00640 1 --2- 172.21.15.32:0/3506097411 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2c200683b0 0x7f2c2006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:06.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.906+0000 7f2c4de01000 1 Processor -- start 2024-04-17T10:00:06.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.906+0000 7f2c4de01000 1 -- start start 2024-04-17T10:00:06.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.906+0000 7f2c3be00640 1 --2- 172.21.15.32:0/3506097411 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2c200683b0 0x7f2c2006a870 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0af000 tx=0x7f2c2c0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:06.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.906+0000 7f2c4de01000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e884672260 0x55e884672640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:06.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.906+0000 7f2c4de01000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672b80 0x55e884625460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:06.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.906+0000 7f2c4de01000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8846259a0 0x55e884625d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:06.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.906+0000 7f2c4de01000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e8844774f0 con 0x55e884672b80 2024-04-17T10:00:06.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.906+0000 7f2c4de01000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e8844306c0 con 0x55e8846259a0 2024-04-17T10:00:06.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.906+0000 7f2c4de01000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e8844734b0 con 0x55e884672260 2024-04-17T10:00:06.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.906+0000 7f2c3be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672b80 0x55e884625460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:06.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.906+0000 7f2c3be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672b80 0x55e884625460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59858/0 (socket says 172.21.15.32:59858) 2024-04-17T10:00:06.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.906+0000 7f2c3be00640 1 -- 172.21.15.32:0/1814310277 learned_addr learned my addr 172.21.15.32:0/1814310277 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:06.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.906+0000 7f2c48a00640 1 --2- 172.21.15.32:0/1814310277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8846259a0 0x55e884625d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:06.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.906+0000 7f2c43e00640 1 --2- 172.21.15.32:0/1814310277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e884672260 0x55e884672640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:06.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c3be00640 1 -- 172.21.15.32:0/1814310277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e884672260 msgr2=0x55e884672640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:06.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c3be00640 1 --2- 172.21.15.32:0/1814310277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e884672260 0x55e884672640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c3be00640 1 -- 172.21.15.32:0/1814310277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8846259a0 msgr2=0x55e884625d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:06.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c3be00640 1 --2- 172.21.15.32:0/1814310277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8846259a0 0x55e884625d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c3be00640 1 -- 172.21.15.32:0/1814310277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8844bc8e0 con 0x55e884672b80 2024-04-17T10:00:06.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c3be00640 1 --2- 172.21.15.32:0/1814310277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672b80 0x55e884625460 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0ad040 tx=0x7f2c2c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de0926a4dd7950e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:06.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c38c00640 1 -- 172.21.15.32:0/1814310277 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c0bb070 con 0x55e884672b80 2024-04-17T10:00:06.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c38c00640 1 -- 172.21.15.32:0/1814310277 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c2c0b0040 con 0x55e884672b80 2024-04-17T10:00:06.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c38c00640 1 -- 172.21.15.32:0/1814310277 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c0cb880 con 0x55e884672b80 2024-04-17T10:00:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c4de01000 1 -- 172.21.15.32:0/1814310277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672b80 msgr2=0x55e884625460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c4de01000 1 --2- 172.21.15.32:0/1814310277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672b80 0x55e884625460 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0ad040 tx=0x7f2c2c004740 comp rx=0 tx=0).stop 2024-04-17T10:00:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c4de01000 1 -- 172.21.15.32:0/1814310277 shutdown_connections 2024-04-17T10:00:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c4de01000 1 --2- 172.21.15.32:0/1814310277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8846259a0 0x55e884625d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c4de01000 1 --2- 172.21.15.32:0/1814310277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672b80 0x55e884625460 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c4de01000 1 --2- 172.21.15.32:0/1814310277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e884672260 0x55e884672640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c4de01000 1 -- 172.21.15.32:0/1814310277 >> 172.21.15.32:0/1814310277 conn(0x55e884653000 msgr2=0x55e8846553f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c4de01000 1 -- 172.21.15.32:0/1814310277 shutdown_connections 2024-04-17T10:00:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c4de01000 1 -- 172.21.15.32:0/1814310277 wait complete. 2024-04-17T10:00:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c4de01000 1 -- start start 2024-04-17T10:00:06.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c4de01000 1 Processor -- start 2024-04-17T10:00:06.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c4de01000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8846259a0 0x55e88462afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:06.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c4de01000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e884672260 0x55e88462b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:06.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c4de01000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672b80 0x55e88462ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:06.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c4de01000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e8844f87b0 con 0x55e884672b80 2024-04-17T10:00:06.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c4de01000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e884532d00 con 0x55e8846259a0 2024-04-17T10:00:06.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c4de01000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e88443c2c0 con 0x55e884672260 2024-04-17T10:00:06.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c4de01000 10 client.0 _ll_register_callbacks cb 0x55e88444c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:00:06.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c48a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672b80 0x55e88462ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:06.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c43e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8846259a0 0x55e88462afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:06.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c48a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672b80 0x55e88462ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59866/0 (socket says 172.21.15.32:59866) 2024-04-17T10:00:06.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c48a00640 1 -- 172.21.15.32:0/3942501818 learned_addr learned my addr 172.21.15.32:0/3942501818 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:06.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.910+0000 7f2c3be00640 1 --2- 172.21.15.32:0/3942501818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e884672260 0x55e88462b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:06.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c43e00640 1 -- 172.21.15.32:0/3942501818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e884672260 msgr2=0x55e88462b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:06.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c43e00640 1 --2- 172.21.15.32:0/3942501818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e884672260 0x55e88462b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c43e00640 1 -- 172.21.15.32:0/3942501818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672b80 msgr2=0x55e88462ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c43e00640 1 --2- 172.21.15.32:0/3942501818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672b80 0x55e88462ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c43e00640 1 -- 172.21.15.32:0/3942501818 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e88444f490 con 0x55e8846259a0 2024-04-17T10:00:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c48a00640 1 --2- 172.21.15.32:0/3942501818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672b80 0x55e88462ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:00:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c43e00640 1 --2- 172.21.15.32:0/3942501818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8846259a0 0x55e88462afe0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400c500 tx=0x7f2c34056a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ab142556c30d43 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c31600640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-04-17T10:00:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c31600640 1 -- 172.21.15.32:0/3942501818 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3405edf0 con 0x55e8846259a0 2024-04-17T10:00:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c31600640 1 -- 172.21.15.32:0/3942501818 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7f2c34052880 con 0x55e8846259a0 2024-04-17T10:00:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c4de01000 1 -- 172.21.15.32:0/3942501818 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c2c0b0040 con 0x55e8846259a0 2024-04-17T10:00:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c31600640 1 -- 172.21.15.32:0/3942501818 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f2c3405a0b0 con 0x55e8846259a0 2024-04-17T10:00:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c4de01000 10 client.7303 fetch_fsmap learned FSMap version 66 2024-04-17T10:00:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c4de01000 1 -- 172.21.15.32:0/3942501818 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55e884442eb0 con 0x55e8846259a0 2024-04-17T10:00:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c31600640 1 -- 172.21.15.32:0/3942501818 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c2c0b0040 con 0x55e8846259a0 2024-04-17T10:00:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c31600640 1 -- 172.21.15.32:0/3942501818 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7f2c34055660 con 0x55e8846259a0 2024-04-17T10:00:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c4de01000 10 client.7303 fetch_fsmap finished waiting for FSMap version 66 2024-04-17T10:00:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c4de01000 10 client.7303 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T10:00:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c4de01000 20 client.7303 populate_metadata read hostname 'smithi032' 2024-04-17T10:00:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c4de01000 1 --2- 172.21.15.32:0/3942501818 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55e884681500 0x55e8846838f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c4de01000 4 client.7303 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:00:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c4de01000 1 -- 172.21.15.32:0/3942501818 --> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e884410dd0 con 0x55e884681500 2024-04-17T10:00:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c3be00640 1 --2- 172.21.15.32:0/3942501818 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55e884681500 0x55e8846838f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c3be00640 1 --2- 172.21.15.32:0/3942501818 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55e884681500 0x55e8846838f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2335f470c9e47a0 server_cookie=14e493dc81bcea52 in_seq=0 out_seq=0 2024-04-17T10:00:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c31600640 10 client.7303 ms_handle_connect on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c31600640 1 -- 172.21.15.32:0/3942501818 <== mds.0 v2:172.21.15.32:6835/1253497636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e884410dd0 con 0x55e884681500 2024-04-17T10:00:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c31600640 10 client.7303 handle_command_reply: tid=0 2024-04-17T10:00:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c4de01000 1 client.7303 shutdown 2024-04-17T10:00:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c4de01000 1 -- 172.21.15.32:0/3942501818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8846259a0 msgr2=0x55e88462afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c4de01000 1 --2- 172.21.15.32:0/3942501818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8846259a0 0x55e88462afe0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400c500 tx=0x7f2c34056a50 comp rx=0 tx=0).stop 2024-04-17T10:00:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c4de01000 1 -- 172.21.15.32:0/3942501818 shutdown_connections 2024-04-17T10:00:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c4de01000 1 --2- 172.21.15.32:0/3942501818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e884672260 0x55e88462b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c4de01000 1 --2- 172.21.15.32:0/3942501818 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55e884681500 0x55e8846838f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c4de01000 1 --2- 172.21.15.32:0/3942501818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672b80 0x55e88462ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c4de01000 1 --2- 172.21.15.32:0/3942501818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8846259a0 0x55e88462afe0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c4de01000 1 -- 172.21.15.32:0/3942501818 >> 172.21.15.32:0/3942501818 conn(0x55e884653000 msgr2=0x55e8846553f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c31600640 0 client.7303 ms_handle_reset on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c4de01000 1 -- 172.21.15.32:0/3942501818 shutdown_connections 2024-04-17T10:00:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c4de01000 1 -- 172.21.15.32:0/3942501818 wait complete. 2024-04-17T10:00:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.914+0000 7f2c4de01000 20 client.7303 trim_cache size 0 max 16384 2024-04-17T10:00:06.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c4de01000 1 Processor -- start 2024-04-17T10:00:06.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c4de01000 1 -- start start 2024-04-17T10:00:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c4de01000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8846259a0 0x55e8846dd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c4de01000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672260 0x55e8846dd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c4de01000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e884672b80 0x55e8846ddb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c4de01000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2c44150b40 con 0x55e884672260 2024-04-17T10:00:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c4de01000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4415ca40 con 0x55e8846259a0 2024-04-17T10:00:06.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c4de01000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2c4415d090 con 0x55e884672b80 2024-04-17T10:00:06.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c3be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672260 0x55e8846dd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:06.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c3be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672260 0x55e8846dd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59882/0 (socket says 172.21.15.32:59882) 2024-04-17T10:00:06.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c3be00640 1 -- 172.21.15.32:0/4151480184 learned_addr learned my addr 172.21.15.32:0/4151480184 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:06.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c48a00640 1 --2- 172.21.15.32:0/4151480184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e884672b80 0x55e8846ddb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:06.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c43e00640 1 --2- 172.21.15.32:0/4151480184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8846259a0 0x55e8846dd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:06.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c3be00640 1 -- 172.21.15.32:0/4151480184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e884672b80 msgr2=0x55e8846ddb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:06.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c3be00640 1 --2- 172.21.15.32:0/4151480184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e884672b80 0x55e8846ddb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c3be00640 1 -- 172.21.15.32:0/4151480184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8846259a0 msgr2=0x55e8846dd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:06.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c3be00640 1 --2- 172.21.15.32:0/4151480184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8846259a0 0x55e8846dd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c3be00640 1 -- 172.21.15.32:0/4151480184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e884431a20 con 0x55e884672260 2024-04-17T10:00:06.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c43e00640 1 --2- 172.21.15.32:0/4151480184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8846259a0 0x55e8846dd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:00:06.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c3be00640 1 --2- 172.21.15.32:0/4151480184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672260 0x55e8846dd5f0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c004780 tx=0x7f2c2c0b7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7117a9e5ccc98abc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:06.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c33e00640 1 -- 172.21.15.32:0/4151480184 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c0bb040 con 0x55e884672260 2024-04-17T10:00:06.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c33e00640 1 -- 172.21.15.32:0/4151480184 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c2c0b0040 con 0x55e884672260 2024-04-17T10:00:06.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c33e00640 1 -- 172.21.15.32:0/4151480184 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c0c0aa0 con 0x55e884672260 2024-04-17T10:00:06.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c4de01000 1 -- 172.21.15.32:0/4151480184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672260 msgr2=0x55e8846dd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:06.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c4de01000 1 --2- 172.21.15.32:0/4151480184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672260 0x55e8846dd5f0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c004780 tx=0x7f2c2c0b7070 comp rx=0 tx=0).stop 2024-04-17T10:00:06.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c4de01000 1 -- 172.21.15.32:0/4151480184 shutdown_connections 2024-04-17T10:00:06.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c4de01000 1 --2- 172.21.15.32:0/4151480184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e884672b80 0x55e8846ddb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c4de01000 1 --2- 172.21.15.32:0/4151480184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672260 0x55e8846dd5f0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c4de01000 1 --2- 172.21.15.32:0/4151480184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8846259a0 0x55e8846dd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c4de01000 1 -- 172.21.15.32:0/4151480184 >> 172.21.15.32:0/4151480184 conn(0x55e884653000 msgr2=0x55e884686b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:06.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c4de01000 1 -- 172.21.15.32:0/4151480184 shutdown_connections 2024-04-17T10:00:06.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.934+0000 7f2c4de01000 1 -- 172.21.15.32:0/4151480184 wait complete. 2024-04-17T10:00:06.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c4de01000 1 -- start start 2024-04-17T10:00:06.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c4de01000 1 Processor -- start 2024-04-17T10:00:06.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c4de01000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8846259a0 0x55e884686000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:06.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c4de01000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e884672260 0x55e88469d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:06.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c4de01000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672b80 0x55e88469d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:06.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c4de01000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e88441b050 con 0x55e884672b80 2024-04-17T10:00:06.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c4de01000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e884439570 con 0x55e8846259a0 2024-04-17T10:00:06.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c4de01000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e884476800 con 0x55e884672260 2024-04-17T10:00:06.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c4de01000 10 client.0 _ll_register_callbacks cb 0x7f2c340075b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:00:06.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c48a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672b80 0x55e88469d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:06.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c48a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672b80 0x55e88469d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59888/0 (socket says 172.21.15.32:59888) 2024-04-17T10:00:06.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c48a00640 1 -- 172.21.15.32:0/3711664719 learned_addr learned my addr 172.21.15.32:0/3711664719 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:06.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c43e00640 1 --2- 172.21.15.32:0/3711664719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8846259a0 0x55e884686000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:06.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c3be00640 1 --2- 172.21.15.32:0/3711664719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e884672260 0x55e88469d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:06.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c48a00640 1 -- 172.21.15.32:0/3711664719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e884672260 msgr2=0x55e88469d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:06.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c48a00640 1 --2- 172.21.15.32:0/3711664719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e884672260 0x55e88469d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c48a00640 1 -- 172.21.15.32:0/3711664719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8846259a0 msgr2=0x55e884686000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:06.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c48a00640 1 --2- 172.21.15.32:0/3711664719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8846259a0 0x55e884686000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c48a00640 1 -- 172.21.15.32:0/3711664719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8844f8c40 con 0x55e884672b80 2024-04-17T10:00:06.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c48a00640 1 --2- 172.21.15.32:0/3711664719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672b80 0x55e88469d9c0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c002240 tx=0x7f2c3c007770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e7b850be4e2884 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:06.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c31600640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:00:06.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c31600640 1 -- 172.21.15.32:0/3711664719 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c0079c0 con 0x55e884672b80 2024-04-17T10:00:06.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c31600640 1 -- 172.21.15.32:0/3711664719 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7f2c3c016b90 con 0x55e884672b80 2024-04-17T10:00:06.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c4de01000 1 -- 172.21.15.32:0/3711664719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c2c0b0040 con 0x55e884672b80 2024-04-17T10:00:06.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c31600640 1 -- 172.21.15.32:0/3711664719 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f2c3c026050 con 0x55e884672b80 2024-04-17T10:00:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c4de01000 10 client.7995 fetch_fsmap learned FSMap version 66 2024-04-17T10:00:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c4de01000 1 -- 172.21.15.32:0/3711664719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55e88441c600 con 0x55e884672b80 2024-04-17T10:00:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c31600640 1 -- 172.21.15.32:0/3711664719 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c2c0b0040 con 0x55e884672b80 2024-04-17T10:00:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c31600640 1 -- 172.21.15.32:0/3711664719 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7f2c3c029020 con 0x55e884672b80 2024-04-17T10:00:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c4de01000 10 client.7995 fetch_fsmap finished waiting for FSMap version 66 2024-04-17T10:00:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c4de01000 10 client.7995 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T10:00:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c4de01000 20 client.7995 populate_metadata read hostname 'smithi032' 2024-04-17T10:00:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c4de01000 1 --2- 172.21.15.32:0/3711664719 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55e8846e08b0 0x55e8846e0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c4de01000 4 client.7995 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:00:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c4de01000 1 -- 172.21.15.32:0/3711664719 --> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e884652c40 con 0x55e8846e08b0 2024-04-17T10:00:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c43e00640 1 --2- 172.21.15.32:0/3711664719 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55e8846e08b0 0x55e8846e0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c31600640 10 client.7995 ms_handle_connect on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.938+0000 7f2c43e00640 1 --2- 172.21.15.32:0/3711664719 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55e8846e08b0 0x55e8846e0cc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11ce9d3ee0310521 server_cookie=abde3c3767108c07 in_seq=0 out_seq=0 2024-04-17T10:00:06.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c31600640 1 -- 172.21.15.32:0/3711664719 <== mds.0 v2:172.21.15.32:6835/1253497636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e884652c40 con 0x55e8846e08b0 2024-04-17T10:00:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c31600640 10 client.7995 handle_command_reply: tid=0 2024-04-17T10:00:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4de01000 1 client.7995 shutdown 2024-04-17T10:00:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4de01000 1 -- 172.21.15.32:0/3711664719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672b80 msgr2=0x55e88469d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4de01000 1 --2- 172.21.15.32:0/3711664719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672b80 0x55e88469d9c0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c002240 tx=0x7f2c3c007770 comp rx=0 tx=0).stop 2024-04-17T10:00:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4de01000 1 -- 172.21.15.32:0/3711664719 shutdown_connections 2024-04-17T10:00:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4de01000 1 --2- 172.21.15.32:0/3711664719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e884672260 0x55e88469d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4de01000 1 --2- 172.21.15.32:0/3711664719 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55e8846e08b0 0x55e8846e0cc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4de01000 1 --2- 172.21.15.32:0/3711664719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e884672b80 0x55e88469d9c0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4de01000 1 --2- 172.21.15.32:0/3711664719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8846259a0 0x55e884686000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4de01000 1 -- 172.21.15.32:0/3711664719 >> 172.21.15.32:0/3711664719 conn(0x55e884653000 msgr2=0x55e884624450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c31600640 0 client.7995 ms_handle_reset on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4de01000 1 -- 172.21.15.32:0/3711664719 shutdown_connections 2024-04-17T10:00:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4de01000 1 -- 172.21.15.32:0/3711664719 wait complete. 2024-04-17T10:00:06.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4de01000 20 client.7995 trim_cache size 0 max 16384 2024-04-17T10:00:06.943 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:00:06.943 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-17T10:00:06.943 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-17T10:00:06.943 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:00:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4aa00640 1 -- 172.21.15.32:0/3506097411 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2c200683b0 msgr2=0x7f2c2006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4aa00640 1 --2- 172.21.15.32:0/3506097411 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2c200683b0 0x7f2c2006a870 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0af000 tx=0x7f2c2c0a7000 comp rx=0 tx=0).stop 2024-04-17T10:00:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4aa00640 1 -- 172.21.15.32:0/3506097411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c440b3d80 msgr2=0x7f2c4414bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4aa00640 1 --2- 172.21.15.32:0/3506097411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c440b3d80 0x7f2c4414bb20 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400c530 tx=0x7f2c3400c9f0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4aa00640 1 -- 172.21.15.32:0/3506097411 shutdown_connections 2024-04-17T10:00:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4aa00640 1 --2- 172.21.15.32:0/3506097411 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2c200683b0 0x7f2c2006a870 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4aa00640 1 --2- 172.21.15.32:0/3506097411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c4415b010 0x7f2c440d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4aa00640 1 --2- 172.21.15.32:0/3506097411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c4415ac30 0x7f2c440d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4aa00640 1 --2- 172.21.15.32:0/3506097411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c440b3d80 0x7f2c4414bb20 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:06.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4aa00640 1 -- 172.21.15.32:0/3506097411 >> 172.21.15.32:0/3506097411 conn(0x7f2c440b2120 msgr2=0x7f2c44168da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:06.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4aa00640 1 -- 172.21.15.32:0/3506097411 shutdown_connections 2024-04-17T10:00:06.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:06.942+0000 7f2c4aa00640 1 -- 172.21.15.32:0/3506097411 wait complete. 2024-04-17T10:00:06.958 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T10:00:06.959 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:00:07.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.102+0000 7fbca0000640 1 Processor -- start 2024-04-17T10:00:07.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.102+0000 7fbca0000640 1 -- start start 2024-04-17T10:00:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.102+0000 7fbca0000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbc9806dc60 0x7fbc9806cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.102+0000 7fbca0000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc9809b3c0 0x7fbc980d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.102+0000 7fbca0000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc9809bd60 con 0x7fbc9815eba0 2024-04-17T10:00:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.102+0000 7fbca0000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fbc980b1e40 con 0x7fbc9806dc60 2024-04-17T10:00:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.102+0000 7fbca0000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbc980b2120 con 0x7fbc9809b3c0 2024-04-17T10:00:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.102+0000 7fbc9d600640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbc9815eba0 0x7fbc9806d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39312/0 (socket says 172.21.15.32:39312) 2024-04-17T10:00:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.102+0000 7fbc9d600640 1 -- 172.21.15.32:0/977056024 learned_addr learned my addr 172.21.15.32:0/977056024 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:00:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.106+0000 7fbc9e000640 1 --2- 172.21.15.32:0/977056024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc9809b3c0 0x7fbc980d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.106+0000 7fbc9cc00640 1 --2- 172.21.15.32:0/977056024 >> v2:172.21.15.144:3300/0 conn(0x7fbc9806dc60 0x7fbc9806cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.106+0000 7fbc97e00640 1 -- 172.21.15.32:0/977056024 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3041770959 0 0) 0x7fbc9809bd60 con 0x7fbc9815eba0 2024-04-17T10:00:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.106+0000 7fbc97e00640 1 -- 172.21.15.32:0/977056024 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc80003530 con 0x7fbc9815eba0 2024-04-17T10:00:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.106+0000 7fbc9e000640 1 -- 172.21.15.32:0/977056024 >> v2:172.21.15.144:3300/0 conn(0x7fbc9806dc60 msgr2=0x7fbc9806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.106+0000 7fbc9e000640 1 --2- 172.21.15.32:0/977056024 >> v2:172.21.15.144:3300/0 conn(0x7fbc9806dc60 0x7fbc9806cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.106+0000 7fbc9e000640 1 -- 172.21.15.32:0/977056024 >> v1:172.21.15.32:6789/0 conn(0x7fbc9815eba0 legacy=0x7fbc9806d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.106+0000 7fbc9e000640 1 -- 172.21.15.32:0/977056024 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc980b2400 con 0x7fbc9809b3c0 2024-04-17T10:00:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.106+0000 7fbc9e000640 1 --2- 172.21.15.32:0/977056024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc9809b3c0 0x7fbc980d0720 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009d90 tx=0x7fbc9001c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce3411f2c03fab99 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.106+0000 7fbc97e00640 1 -- 172.21.15.32:0/977056024 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc900b2070 con 0x7fbc9809b3c0 2024-04-17T10:00:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.106+0000 7fbc97e00640 1 -- 172.21.15.32:0/977056024 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc900a5890 con 0x7fbc9809b3c0 2024-04-17T10:00:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.106+0000 7fbc97e00640 1 -- 172.21.15.32:0/977056024 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc900a5c60 con 0x7fbc9809b3c0 2024-04-17T10:00:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbca0000640 1 -- 172.21.15.32:0/977056024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc9809b3c0 msgr2=0x7fbc980d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbca0000640 1 --2- 172.21.15.32:0/977056024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc9809b3c0 0x7fbc980d0720 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009d90 tx=0x7fbc9001c8e0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbca0000640 1 -- 172.21.15.32:0/977056024 shutdown_connections 2024-04-17T10:00:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbca0000640 1 --2- 172.21.15.32:0/977056024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc9809b3c0 0x7fbc980d0720 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbca0000640 1 --2- 172.21.15.32:0/977056024 >> v2:172.21.15.144:3300/0 conn(0x7fbc9806dc60 0x7fbc9806cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbca0000640 1 -- 172.21.15.32:0/977056024 >> 172.21.15.32:0/977056024 conn(0x7fbc980b3fd0 msgr2=0x7fbc980b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbca0000640 1 -- 172.21.15.32:0/977056024 shutdown_connections 2024-04-17T10:00:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbca0000640 1 -- 172.21.15.32:0/977056024 wait complete. 2024-04-17T10:00:07.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbca0000640 1 Processor -- start 2024-04-17T10:00:07.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbca0000640 1 -- start start 2024-04-17T10:00:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbca0000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc9806dc60 0x7fbc980cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbca0000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc9809b3c0 0x7fbc980cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbca0000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc9815eba0 0x7fbc981456a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbca0000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9816b580 con 0x7fbc9809b3c0 2024-04-17T10:00:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbca0000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9816b700 con 0x7fbc9815eba0 2024-04-17T10:00:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbca0000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbc9816b880 con 0x7fbc9806dc60 2024-04-17T10:00:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbc9cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc9809b3c0 0x7fbc980cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbc9cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc9809b3c0 0x7fbc980cb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59898/0 (socket says 172.21.15.32:59898) 2024-04-17T10:00:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbc9cc00640 1 -- 172.21.15.32:0/3184800467 learned_addr learned my addr 172.21.15.32:0/3184800467 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbc9e000640 1 --2- 172.21.15.32:0/3184800467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc9815eba0 0x7fbc981456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbc9d600640 1 --2- 172.21.15.32:0/3184800467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc9806dc60 0x7fbc980cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbc9cc00640 1 -- 172.21.15.32:0/3184800467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc9806dc60 msgr2=0x7fbc980cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbc9cc00640 1 --2- 172.21.15.32:0/3184800467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc9806dc60 0x7fbc980cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbc9cc00640 1 -- 172.21.15.32:0/3184800467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc9815eba0 msgr2=0x7fbc981456a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbc9cc00640 1 --2- 172.21.15.32:0/3184800467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc9815eba0 0x7fbc981456a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbc9cc00640 1 -- 172.21.15.32:0/3184800467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc980cbf00 con 0x7fbc9809b3c0 2024-04-17T10:00:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.110+0000 7fbc9d600640 1 --2- 172.21.15.32:0/3184800467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc9806dc60 0x7fbc980cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.114+0000 7fbc9cc00640 1 --2- 172.21.15.32:0/3184800467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc9809b3c0 0x7fbc980cb8f0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fbc8400e870 tx=0x7fbc8400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.114+0000 7fbc95600640 1 -- 172.21.15.32:0/3184800467 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc84017070 con 0x7fbc9809b3c0 2024-04-17T10:00:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.114+0000 7fbc95600640 1 -- 172.21.15.32:0/3184800467 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc8400c940 con 0x7fbc9809b3c0 2024-04-17T10:00:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.114+0000 7fbc95600640 1 -- 172.21.15.32:0/3184800467 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8400cd30 con 0x7fbc9809b3c0 2024-04-17T10:00:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.114+0000 7fbca0000640 1 -- 172.21.15.32:0/3184800467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc980cc1e0 con 0x7fbc9809b3c0 2024-04-17T10:00:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.114+0000 7fbca0000640 1 -- 172.21.15.32:0/3184800467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc980cc7a0 con 0x7fbc9809b3c0 2024-04-17T10:00:07.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.114+0000 7fbca0000640 1 -- 172.21.15.32:0/3184800467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc98068a90 con 0x7fbc9809b3c0 2024-04-17T10:00:07.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.114+0000 7fbc95600640 1 -- 172.21.15.32:0/3184800467 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbc8401a070 con 0x7fbc9809b3c0 2024-04-17T10:00:07.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.114+0000 7fbc95600640 1 --2- 172.21.15.32:0/3184800467 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbc78068380 0x7fbc7806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.114+0000 7fbc95600640 1 -- 172.21.15.32:0/3184800467 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7fbc84086cc0 con 0x7fbc9809b3c0 2024-04-17T10:00:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.114+0000 7fbc9d600640 1 --2- 172.21.15.32:0/3184800467 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbc78068380 0x7fbc7806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.118+0000 7fbc9d600640 1 --2- 172.21.15.32:0/3184800467 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbc78068380 0x7fbc7806a840 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c005890 tx=0x7fbc8c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:07.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.118+0000 7fbc95600640 1 -- 172.21.15.32:0/3184800467 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fbc8408d080 con 0x7fbc9809b3c0 2024-04-17T10:00:07.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.318+0000 7fbca0000640 1 -- 172.21.15.32:0/3184800467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc9806d330 con 0x7fbc9809b3c0 2024-04-17T10:00:07.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.318+0000 7fbc95600640 1 -- 172.21.15.32:0/3184800467 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5205 (secure 0 0 0) 0x7fbc84056b00 con 0x7fbc9809b3c0 2024-04-17T10:00:07.319 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:00:07.319 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6419,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3534108103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3534108103},{"type":"v1","addr":"172.21.15.144:6835","nonce":3534108103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7003,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/935945019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":935945019},{"type":"v1","addr":"172.21.15.144:6834","nonce":935945019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3044362393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3044362393},{"type":"v1","addr":"172.21.15.32:6836","nonce":3044362393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:59:04.834110+0000","modified":"2024-04-17T09:59:40.007746+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":6,"addr":"172.21.15.32:6837/1253497636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1253497636},{"type":"v1","addr":"172.21.15.32:6837","nonce":1253497636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7851,"qdb_cluster":[7851]},"id":6}]} 2024-04-17T10:00:07.320 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 66 2024-04-17T10:00:07.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.322+0000 7fbca0000640 1 -- 172.21.15.32:0/3184800467 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbc78068380 msgr2=0x7fbc7806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:07.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.322+0000 7fbca0000640 1 --2- 172.21.15.32:0/3184800467 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbc78068380 0x7fbc7806a840 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c005890 tx=0x7fbc8c009040 comp rx=0 tx=0).stop 2024-04-17T10:00:07.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.322+0000 7fbca0000640 1 -- 172.21.15.32:0/3184800467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc9809b3c0 msgr2=0x7fbc980cb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:07.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.322+0000 7fbca0000640 1 --2- 172.21.15.32:0/3184800467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc9809b3c0 0x7fbc980cb8f0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fbc8400e870 tx=0x7fbc8400ec70 comp rx=0 tx=0).stop 2024-04-17T10:00:07.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.322+0000 7fbca0000640 1 -- 172.21.15.32:0/3184800467 shutdown_connections 2024-04-17T10:00:07.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.322+0000 7fbca0000640 1 --2- 172.21.15.32:0/3184800467 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbc78068380 0x7fbc7806a840 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.322+0000 7fbca0000640 1 --2- 172.21.15.32:0/3184800467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc9815eba0 0x7fbc981456a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.322+0000 7fbca0000640 1 --2- 172.21.15.32:0/3184800467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc9809b3c0 0x7fbc980cb8f0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.322+0000 7fbca0000640 1 --2- 172.21.15.32:0/3184800467 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc9806dc60 0x7fbc980cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.322+0000 7fbca0000640 1 -- 172.21.15.32:0/3184800467 >> 172.21.15.32:0/3184800467 conn(0x7fbc980b3fd0 msgr2=0x7fbc980afa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.322+0000 7fbca0000640 1 -- 172.21.15.32:0/3184800467 shutdown_connections 2024-04-17T10:00:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.322+0000 7fbca0000640 1 -- 172.21.15.32:0/3184800467 wait complete. 2024-04-17T10:00:07.340 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-17T10:00:07.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.486+0000 7faa74a00640 1 Processor -- start 2024-04-17T10:00:07.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.486+0000 7faa74a00640 1 -- start start 2024-04-17T10:00:07.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.486+0000 7faa74a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faa7015b010 0x7faa7006dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.486+0000 7faa74a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faa700b3d80 0x7faa70071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.486+0000 7faa74a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa7006e6f0 con 0x7faa7015ac30 2024-04-17T10:00:07.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.486+0000 7faa74a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7faa7006f300 con 0x7faa7015b010 2024-04-17T10:00:07.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.486+0000 7faa74a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7faa7006e9d0 con 0x7faa700b3d80 2024-04-17T10:00:07.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.486+0000 7faa65400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faa7015b010 0x7faa7006dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.486+0000 7faa65400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faa7015b010 0x7faa7006dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59912/0 (socket says 172.21.15.32:59912) 2024-04-17T10:00:07.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.486+0000 7faa65400640 1 -- 172.21.15.32:0/4193408013 learned_addr learned my addr 172.21.15.32:0/4193408013 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:07.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.490+0000 7faa6e800640 1 --2- 172.21.15.32:0/4193408013 >> v2:172.21.15.144:3300/0 conn(0x7faa700b3d80 0x7faa70071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.490+0000 7faa65400640 1 -- 172.21.15.32:0/4193408013 >> v2:172.21.15.144:3300/0 conn(0x7faa700b3d80 msgr2=0x7faa70071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:07.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.490+0000 7faa65400640 1 --2- 172.21.15.32:0/4193408013 >> v2:172.21.15.144:3300/0 conn(0x7faa700b3d80 0x7faa70071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.490+0000 7faa65400640 1 -- 172.21.15.32:0/4193408013 >> v1:172.21.15.144:6789/0 conn(0x7faa7015ac30 legacy=0x7faa7006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:07.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.490+0000 7faa65400640 1 -- 172.21.15.32:0/4193408013 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa7006ecb0 con 0x7faa7015b010 2024-04-17T10:00:07.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.490+0000 7faa6e800640 1 --2- 172.21.15.32:0/4193408013 >> v2:172.21.15.144:3300/0 conn(0x7faa700b3d80 0x7faa70071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:00:07.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.490+0000 7faa65400640 1 --2- 172.21.15.32:0/4193408013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa7015b010 0x7faa7006dea0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7faa700732b0 tx=0x7faa5801e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af30676968eba1e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:07.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.490+0000 7faa6d400640 1 -- 172.21.15.32:0/4193408013 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa580ad070 con 0x7faa7015b010 2024-04-17T10:00:07.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.490+0000 7faa6d400640 1 -- 172.21.15.32:0/4193408013 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa58004230 con 0x7faa7015b010 2024-04-17T10:00:07.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.490+0000 7faa6d400640 1 -- 172.21.15.32:0/4193408013 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa580a76f0 con 0x7faa7015b010 2024-04-17T10:00:07.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.490+0000 7faa74a00640 1 -- 172.21.15.32:0/4193408013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa7015b010 msgr2=0x7faa7006dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:07.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.490+0000 7faa74a00640 1 --2- 172.21.15.32:0/4193408013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa7015b010 0x7faa7006dea0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7faa700732b0 tx=0x7faa5801e990 comp rx=0 tx=0).stop 2024-04-17T10:00:07.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.490+0000 7faa74a00640 1 -- 172.21.15.32:0/4193408013 shutdown_connections 2024-04-17T10:00:07.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.490+0000 7faa74a00640 1 --2- 172.21.15.32:0/4193408013 >> v2:172.21.15.144:3300/0 conn(0x7faa700b3d80 0x7faa70071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.490+0000 7faa74a00640 1 --2- 172.21.15.32:0/4193408013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa7015b010 0x7faa7006dea0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa74a00640 1 -- 172.21.15.32:0/4193408013 >> 172.21.15.32:0/4193408013 conn(0x7faa700b2120 msgr2=0x7faa700b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:07.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa74a00640 1 -- 172.21.15.32:0/4193408013 shutdown_connections 2024-04-17T10:00:07.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa74a00640 1 -- 172.21.15.32:0/4193408013 wait complete. 2024-04-17T10:00:07.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa74a00640 1 Processor -- start 2024-04-17T10:00:07.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa74a00640 1 -- start start 2024-04-17T10:00:07.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa74a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa700b3d80 0x7faa70152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa74a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa7015ac30 0x7faa700d0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa74a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa7015b010 0x7faa701569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa74a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faa700cddd0 con 0x7faa7015b010 2024-04-17T10:00:07.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa74a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faa700cdc50 con 0x7faa7015ac30 2024-04-17T10:00:07.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa74a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7faa700cdf50 con 0x7faa700b3d80 2024-04-17T10:00:07.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa6e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa7015b010 0x7faa701569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa6e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa7015b010 0x7faa701569e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59916/0 (socket says 172.21.15.32:59916) 2024-04-17T10:00:07.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa6e800640 1 -- 172.21.15.32:0/3582891644 learned_addr learned my addr 172.21.15.32:0/3582891644 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:07.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa6de00640 1 --2- 172.21.15.32:0/3582891644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa700b3d80 0x7faa70152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa65400640 1 --2- 172.21.15.32:0/3582891644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa7015ac30 0x7faa700d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa6e800640 1 -- 172.21.15.32:0/3582891644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa700b3d80 msgr2=0x7faa70152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:07.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa6e800640 1 --2- 172.21.15.32:0/3582891644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa700b3d80 0x7faa70152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa6e800640 1 -- 172.21.15.32:0/3582891644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa7015ac30 msgr2=0x7faa700d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:07.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa6e800640 1 --2- 172.21.15.32:0/3582891644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa7015ac30 0x7faa700d0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa6e800640 1 -- 172.21.15.32:0/3582891644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa70157430 con 0x7faa7015b010 2024-04-17T10:00:07.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa6e800640 1 --2- 172.21.15.32:0/3582891644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa7015b010 0x7faa701569e0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7faa6800cef0 tx=0x7faa6800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa66a00640 1 -- 172.21.15.32:0/3582891644 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6801a070 con 0x7faa7015b010 2024-04-17T10:00:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa66a00640 1 -- 172.21.15.32:0/3582891644 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa6800d860 con 0x7faa7015b010 2024-04-17T10:00:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa66a00640 1 -- 172.21.15.32:0/3582891644 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6800dc50 con 0x7faa7015b010 2024-04-17T10:00:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.494+0000 7faa74a00640 1 -- 172.21.15.32:0/3582891644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa70151b70 con 0x7faa7015b010 2024-04-17T10:00:07.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.498+0000 7faa74a00640 1 -- 172.21.15.32:0/3582891644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa70157710 con 0x7faa7015b010 2024-04-17T10:00:07.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.498+0000 7faa66a00640 1 -- 172.21.15.32:0/3582891644 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7faa6801d070 con 0x7faa7015b010 2024-04-17T10:00:07.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.498+0000 7faa66a00640 1 --2- 172.21.15.32:0/3582891644 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faa4c0683b0 0x7faa4c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.498+0000 7faa66a00640 1 -- 172.21.15.32:0/3582891644 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7faa68087200 con 0x7faa7015b010 2024-04-17T10:00:07.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.498+0000 7faa6de00640 1 --2- 172.21.15.32:0/3582891644 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faa4c0683b0 0x7faa4c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa77f75000 1 Processor -- start 2024-04-17T10:00:07.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa77f75000 1 -- start start 2024-04-17T10:00:07.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa6de00640 1 --2- 172.21.15.32:0/3582891644 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faa4c0683b0 0x7faa4c06a870 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7faa700ccd30 tx=0x7faa60007e40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:07.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa77f75000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7469c9050 0x55b7469c9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa77f75000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9970 0x55b74697be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa77f75000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74697c380 0x55b74697c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa77f75000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b7467c9590 con 0x55b7469c9970 2024-04-17T10:00:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa77f75000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b746782580 con 0x55b74697c380 2024-04-17T10:00:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa77f75000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b7467c5550 con 0x55b7469c9050 2024-04-17T10:00:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa65400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9970 0x55b74697be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa65400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9970 0x55b74697be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59932/0 (socket says 172.21.15.32:59932) 2024-04-17T10:00:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa65400640 1 -- 172.21.15.32:0/863675488 learned_addr learned my addr 172.21.15.32:0/863675488 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa6de00640 1 --2- 172.21.15.32:0/863675488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7469c9050 0x55b7469c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa6e800640 1 --2- 172.21.15.32:0/863675488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74697c380 0x55b74697c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa65400640 1 -- 172.21.15.32:0/863675488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7469c9050 msgr2=0x55b7469c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa65400640 1 --2- 172.21.15.32:0/863675488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7469c9050 0x55b7469c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa65400640 1 -- 172.21.15.32:0/863675488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74697c380 msgr2=0x55b74697c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:07.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa65400640 1 --2- 172.21.15.32:0/863675488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74697c380 0x55b74697c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa65400640 1 -- 172.21.15.32:0/863675488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b746870b90 con 0x55b7469c9970 2024-04-17T10:00:07.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa6e800640 1 --2- 172.21.15.32:0/863675488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74697c380 0x55b74697c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:07.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa65400640 1 --2- 172.21.15.32:0/863675488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9970 0x55b74697be40 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7faa58000c10 tx=0x7faa58004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299a0839d8bc7b17 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:07.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa5ea00640 1 -- 172.21.15.32:0/863675488 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa580ad040 con 0x55b7469c9970 2024-04-17T10:00:07.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa5ea00640 1 -- 172.21.15.32:0/863675488 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa58004170 con 0x55b7469c9970 2024-04-17T10:00:07.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa5ea00640 1 -- 172.21.15.32:0/863675488 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58004360 con 0x55b7469c9970 2024-04-17T10:00:07.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa77f75000 1 -- 172.21.15.32:0/863675488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9970 msgr2=0x55b74697be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:07.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa77f75000 1 --2- 172.21.15.32:0/863675488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9970 0x55b74697be40 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7faa58000c10 tx=0x7faa58004740 comp rx=0 tx=0).stop 2024-04-17T10:00:07.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa77f75000 1 -- 172.21.15.32:0/863675488 shutdown_connections 2024-04-17T10:00:07.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa77f75000 1 --2- 172.21.15.32:0/863675488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b74697c380 0x55b74697c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa77f75000 1 --2- 172.21.15.32:0/863675488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9970 0x55b74697be40 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa77f75000 1 --2- 172.21.15.32:0/863675488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7469c9050 0x55b7469c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa77f75000 1 -- 172.21.15.32:0/863675488 >> 172.21.15.32:0/863675488 conn(0x55b7469a57f0 msgr2=0x55b7469a7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:07.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa77f75000 1 -- 172.21.15.32:0/863675488 shutdown_connections 2024-04-17T10:00:07.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa77f75000 1 -- 172.21.15.32:0/863675488 wait complete. 2024-04-17T10:00:07.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa77f75000 1 -- start start 2024-04-17T10:00:07.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.502+0000 7faa77f75000 1 Processor -- start 2024-04-17T10:00:07.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa77f75000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74697c380 0x55b7469cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa77f75000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7469c9050 0x55b7469cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa77f75000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9970 0x55b7469a9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa77f75000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b7467d44c0 con 0x55b7469c9970 2024-04-17T10:00:07.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa77f75000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b746769c80 con 0x55b7469c9050 2024-04-17T10:00:07.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa77f75000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b746794930 con 0x55b74697c380 2024-04-17T10:00:07.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa77f75000 10 client.0 _ll_register_callbacks cb 0x55b7469656d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:00:07.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa6e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9970 0x55b7469a9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa6e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9970 0x55b7469a9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59936/0 (socket says 172.21.15.32:59936) 2024-04-17T10:00:07.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa6e800640 1 -- 172.21.15.32:0/4017903064 learned_addr learned my addr 172.21.15.32:0/4017903064 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:07.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa65400640 1 --2- 172.21.15.32:0/4017903064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7469c9050 0x55b7469cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa6de00640 1 --2- 172.21.15.32:0/4017903064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74697c380 0x55b7469cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa6e800640 1 -- 172.21.15.32:0/4017903064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74697c380 msgr2=0x55b7469cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:07.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa6e800640 1 --2- 172.21.15.32:0/4017903064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74697c380 0x55b7469cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa6e800640 1 -- 172.21.15.32:0/4017903064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7469c9050 msgr2=0x55b7469cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:07.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa6e800640 1 --2- 172.21.15.32:0/4017903064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7469c9050 0x55b7469cfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa6e800640 1 -- 172.21.15.32:0/4017903064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7467a2b20 con 0x55b7469c9970 2024-04-17T10:00:07.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa6de00640 1 --2- 172.21.15.32:0/4017903064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74697c380 0x55b7469cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:00:07.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa6e800640 1 --2- 172.21.15.32:0/4017903064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9970 0x55b7469a9a80 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7faa68056e90 tx=0x7faa6805b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b6445b79ada977 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:07.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa53400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:00:07.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa53400640 1 -- 172.21.15.32:0/4017903064 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6805b170 con 0x55b7469c9970 2024-04-17T10:00:07.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa53400640 1 -- 172.21.15.32:0/4017903064 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7faa68085290 con 0x55b7469c9970 2024-04-17T10:00:07.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa77f75000 1 -- 172.21.15.32:0/4017903064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa58004170 con 0x55b7469c9970 2024-04-17T10:00:07.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa53400640 1 -- 172.21.15.32:0/4017903064 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7faa6805f830 con 0x55b7469c9970 2024-04-17T10:00:07.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa77f75000 10 client.8013 fetch_fsmap learned FSMap version 66 2024-04-17T10:00:07.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa77f75000 1 -- 172.21.15.32:0/4017903064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55b74680d610 con 0x55b7469c9970 2024-04-17T10:00:07.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa53400640 1 -- 172.21.15.32:0/4017903064 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa58004170 con 0x55b7469c9970 2024-04-17T10:00:07.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa53400640 1 -- 172.21.15.32:0/4017903064 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7faa680565e0 con 0x55b7469c9970 2024-04-17T10:00:07.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa77f75000 10 client.8013 fetch_fsmap finished waiting for FSMap version 66 2024-04-17T10:00:07.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa77f75000 10 client.8013 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T10:00:07.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa77f75000 20 client.8013 populate_metadata read hostname 'smithi032' 2024-04-17T10:00:07.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa77f75000 1 --2- 172.21.15.32:0/4017903064 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55b7469aa7f0 0x55b7469aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa77f75000 4 client.8013 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:00:07.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa77f75000 1 -- 172.21.15.32:0/4017903064 --> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b746762e40 con 0x55b7469aa7f0 2024-04-17T10:00:07.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa6de00640 1 --2- 172.21.15.32:0/4017903064 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55b7469aa7f0 0x55b7469aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa6de00640 1 --2- 172.21.15.32:0/4017903064 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55b7469aa7f0 0x55b7469aabd0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6833b92196b15b50 server_cookie=603ed56a7960999b in_seq=0 out_seq=0 2024-04-17T10:00:07.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa53400640 10 client.8013 ms_handle_connect on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:07.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa53400640 1 -- 172.21.15.32:0/4017903064 <== mds.0 v2:172.21.15.32:6835/1253497636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b746762e40 con 0x55b7469aa7f0 2024-04-17T10:00:07.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.506+0000 7faa53400640 10 client.8013 handle_command_reply: tid=0 2024-04-17T10:00:07.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.510+0000 7faa77f75000 1 client.8013 shutdown 2024-04-17T10:00:07.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.510+0000 7faa77f75000 1 -- 172.21.15.32:0/4017903064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9970 msgr2=0x55b7469a9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:07.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.510+0000 7faa77f75000 1 --2- 172.21.15.32:0/4017903064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9970 0x55b7469a9a80 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7faa68056e90 tx=0x7faa6805b490 comp rx=0 tx=0).stop 2024-04-17T10:00:07.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.510+0000 7faa77f75000 1 -- 172.21.15.32:0/4017903064 shutdown_connections 2024-04-17T10:00:07.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.510+0000 7faa77f75000 1 --2- 172.21.15.32:0/4017903064 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55b7469aa7f0 0x55b7469aabd0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.510+0000 7faa77f75000 1 --2- 172.21.15.32:0/4017903064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9970 0x55b7469a9a80 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.510+0000 7faa77f75000 1 --2- 172.21.15.32:0/4017903064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7469c9050 0x55b7469cfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.510+0000 7faa77f75000 1 --2- 172.21.15.32:0/4017903064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74697c380 0x55b7469cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.510+0000 7faa53400640 0 client.8013 ms_handle_reset on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:07.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.510+0000 7faa77f75000 1 -- 172.21.15.32:0/4017903064 >> 172.21.15.32:0/4017903064 conn(0x55b7469a57f0 msgr2=0x55b7469a7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:07.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.510+0000 7faa77f75000 1 -- 172.21.15.32:0/4017903064 shutdown_connections 2024-04-17T10:00:07.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.510+0000 7faa77f75000 1 -- 172.21.15.32:0/4017903064 wait complete. 2024-04-17T10:00:07.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.510+0000 7faa77f75000 20 client.8013 trim_cache size 0 max 16384 2024-04-17T10:00:07.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.522+0000 7faa77f75000 1 Processor -- start 2024-04-17T10:00:07.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.522+0000 7faa77f75000 1 -- start start 2024-04-17T10:00:07.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.522+0000 7faa77f75000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74697c380 0x55b7469a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.522+0000 7faa77f75000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7469c9050 0x55b7469a8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.522+0000 7faa77f75000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9970 0x55b7469c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.522+0000 7faa77f75000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faa701ffda0 con 0x55b7469c9970 2024-04-17T10:00:07.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.522+0000 7faa77f75000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faa701f7910 con 0x55b7469c9050 2024-04-17T10:00:07.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.522+0000 7faa77f75000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7faa70155dc0 con 0x55b74697c380 2024-04-17T10:00:07.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.522+0000 7faa6e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9970 0x55b7469c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.522+0000 7faa6e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9970 0x55b7469c7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59944/0 (socket says 172.21.15.32:59944) 2024-04-17T10:00:07.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.522+0000 7faa6e800640 1 -- 172.21.15.32:0/3577875239 learned_addr learned my addr 172.21.15.32:0/3577875239 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:07.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.522+0000 7faa6de00640 1 --2- 172.21.15.32:0/3577875239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74697c380 0x55b7469a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.522+0000 7faa65400640 1 --2- 172.21.15.32:0/3577875239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7469c9050 0x55b7469a8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.522+0000 7faa6e800640 1 -- 172.21.15.32:0/3577875239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74697c380 msgr2=0x55b7469a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:07.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.522+0000 7faa6e800640 1 --2- 172.21.15.32:0/3577875239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74697c380 0x55b7469a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.522+0000 7faa6e800640 1 -- 172.21.15.32:0/3577875239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7469c9050 msgr2=0x55b7469a8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:07.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.522+0000 7faa6e800640 1 --2- 172.21.15.32:0/3577875239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7469c9050 0x55b7469a8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.522+0000 7faa6e800640 1 -- 172.21.15.32:0/3577875239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b74679f190 con 0x55b7469c9970 2024-04-17T10:00:07.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.522+0000 7faa65400640 1 --2- 172.21.15.32:0/3577875239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7469c9050 0x55b7469a8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:07.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa6e800640 1 --2- 172.21.15.32:0/3577875239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9970 0x55b7469c7cc0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7faa680570d0 tx=0x7faa68017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f2825078ca2f38 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:07.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa5e000640 1 -- 172.21.15.32:0/3577875239 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68053430 con 0x55b7469c9970 2024-04-17T10:00:07.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa5e000640 1 -- 172.21.15.32:0/3577875239 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa6805fa70 con 0x55b7469c9970 2024-04-17T10:00:07.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa5e000640 1 -- 172.21.15.32:0/3577875239 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6805b170 con 0x55b7469c9970 2024-04-17T10:00:07.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa77f75000 1 -- 172.21.15.32:0/3577875239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9970 msgr2=0x55b7469c7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:07.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa77f75000 1 --2- 172.21.15.32:0/3577875239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9970 0x55b7469c7cc0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7faa680570d0 tx=0x7faa68017000 comp rx=0 tx=0).stop 2024-04-17T10:00:07.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa77f75000 1 -- 172.21.15.32:0/3577875239 shutdown_connections 2024-04-17T10:00:07.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa77f75000 1 --2- 172.21.15.32:0/3577875239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9970 0x55b7469c7cc0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa77f75000 1 --2- 172.21.15.32:0/3577875239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7469c9050 0x55b7469a8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa77f75000 1 --2- 172.21.15.32:0/3577875239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74697c380 0x55b7469a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa77f75000 1 -- 172.21.15.32:0/3577875239 >> 172.21.15.32:0/3577875239 conn(0x55b7469a57f0 msgr2=0x55b7469ce600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:07.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa77f75000 1 -- 172.21.15.32:0/3577875239 shutdown_connections 2024-04-17T10:00:07.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa77f75000 1 -- 172.21.15.32:0/3577875239 wait complete. 2024-04-17T10:00:07.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa77f75000 1 -- start start 2024-04-17T10:00:07.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa77f75000 1 Processor -- start 2024-04-17T10:00:07.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa77f75000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74697c380 0x55b746a37990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa77f75000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9050 0x55b746a37ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa77f75000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7469c9970 0x55b7469d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa77f75000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b7467d67a0 con 0x55b7469c9050 2024-04-17T10:00:07.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa77f75000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faa701f72c0 con 0x55b7469c9970 2024-04-17T10:00:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa77f75000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b7467c88a0 con 0x55b74697c380 2024-04-17T10:00:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa77f75000 10 client.0 _ll_register_callbacks cb 0x7faa60007ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:00:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa65400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9050 0x55b746a37ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa65400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9050 0x55b746a37ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59956/0 (socket says 172.21.15.32:59956) 2024-04-17T10:00:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa65400640 1 -- 172.21.15.32:0/3111680749 learned_addr learned my addr 172.21.15.32:0/3111680749 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa6de00640 1 --2- 172.21.15.32:0/3111680749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74697c380 0x55b746a37990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa6e800640 1 --2- 172.21.15.32:0/3111680749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7469c9970 0x55b7469d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa65400640 1 -- 172.21.15.32:0/3111680749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74697c380 msgr2=0x55b746a37990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa65400640 1 --2- 172.21.15.32:0/3111680749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74697c380 0x55b746a37990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa65400640 1 -- 172.21.15.32:0/3111680749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7469c9970 msgr2=0x55b7469d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:07.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa65400640 1 --2- 172.21.15.32:0/3111680749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7469c9970 0x55b7469d13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa65400640 1 -- 172.21.15.32:0/3111680749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7467827d0 con 0x55b7469c9050 2024-04-17T10:00:07.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa6e800640 1 --2- 172.21.15.32:0/3111680749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7469c9970 0x55b7469d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:07.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa65400640 1 --2- 172.21.15.32:0/3111680749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9050 0x55b746a37ed0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7faa5809a490 tx=0x7faa580046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd32acdf9f52b1c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:07.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.526+0000 7faa53400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:00:07.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa53400640 1 -- 172.21.15.32:0/3111680749 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa580cb060 con 0x55b7469c9050 2024-04-17T10:00:07.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa53400640 1 -- 172.21.15.32:0/3111680749 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7faa580b6040 con 0x55b7469c9050 2024-04-17T10:00:07.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa77f75000 1 -- 172.21.15.32:0/3111680749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa6805fa70 con 0x55b7469c9050 2024-04-17T10:00:07.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa53400640 1 -- 172.21.15.32:0/3111680749 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7faa580d1050 con 0x55b7469c9050 2024-04-17T10:00:07.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa77f75000 10 client.8019 fetch_fsmap learned FSMap version 66 2024-04-17T10:00:07.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa77f75000 1 -- 172.21.15.32:0/3111680749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55b74676e440 con 0x55b7469c9050 2024-04-17T10:00:07.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa53400640 1 -- 172.21.15.32:0/3111680749 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa6805fa70 con 0x55b7469c9050 2024-04-17T10:00:07.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa53400640 1 -- 172.21.15.32:0/3111680749 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7faa580d4020 con 0x55b7469c9050 2024-04-17T10:00:07.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa77f75000 10 client.8019 fetch_fsmap finished waiting for FSMap version 66 2024-04-17T10:00:07.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa77f75000 10 client.8019 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T10:00:07.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa77f75000 20 client.8019 populate_metadata read hostname 'smithi032' 2024-04-17T10:00:07.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa77f75000 1 --2- 172.21.15.32:0/3111680749 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55b7469d1da0 0x55b7469d2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa77f75000 4 client.8019 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-17T10:00:07.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa77f75000 1 -- 172.21.15.32:0/3111680749 --> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55b7467d6fb0 con 0x55b7469d1da0 2024-04-17T10:00:07.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa6de00640 1 --2- 172.21.15.32:0/3111680749 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55b7469d1da0 0x55b7469d2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa6de00640 1 --2- 172.21.15.32:0/3111680749 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55b7469d1da0 0x55b7469d2180 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e07bc974bbe29193 server_cookie=55f08ad46e943dad in_seq=0 out_seq=0 2024-04-17T10:00:07.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa53400640 10 client.8019 ms_handle_connect on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:07.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa53400640 1 -- 172.21.15.32:0/3111680749 <== mds.0 v2:172.21.15.32:6835/1253497636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+419 (crc 0 0 0) 0x55b7467d6fb0 con 0x55b7469d1da0 2024-04-17T10:00:07.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa53400640 10 client.8019 handle_command_reply: tid=0 2024-04-17T10:00:07.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa77f75000 1 client.8019 shutdown 2024-04-17T10:00:07.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa77f75000 1 -- 172.21.15.32:0/3111680749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9050 msgr2=0x55b746a37ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:07.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa77f75000 1 --2- 172.21.15.32:0/3111680749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9050 0x55b746a37ed0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7faa5809a490 tx=0x7faa580046d0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa77f75000 1 -- 172.21.15.32:0/3111680749 shutdown_connections 2024-04-17T10:00:07.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa77f75000 1 --2- 172.21.15.32:0/3111680749 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55b7469d1da0 0x55b7469d2180 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa77f75000 1 --2- 172.21.15.32:0/3111680749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7469c9970 0x55b7469d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa77f75000 1 --2- 172.21.15.32:0/3111680749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b7469c9050 0x55b746a37ed0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa77f75000 1 --2- 172.21.15.32:0/3111680749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b74697c380 0x55b746a37990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa53400640 0 client.8019 ms_handle_reset on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:07.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa77f75000 1 -- 172.21.15.32:0/3111680749 >> 172.21.15.32:0/3111680749 conn(0x55b7469a57f0 msgr2=0x55b746976190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:07.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa77f75000 1 -- 172.21.15.32:0/3111680749 shutdown_connections 2024-04-17T10:00:07.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa77f75000 1 -- 172.21.15.32:0/3111680749 wait complete. 2024-04-17T10:00:07.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa77f75000 20 client.8019 trim_cache size 0 max 16384 2024-04-17T10:00:07.537 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:00:07.537 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-17T10:00:07.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:00:07.537 INFO:teuthology.orchestra.run.smithi032.stdout: "damage_type": "backtrace", 2024-04-17T10:00:07.537 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 593058767, 2024-04-17T10:00:07.537 INFO:teuthology.orchestra.run.smithi032.stdout: "ino": 1099511627777, 2024-04-17T10:00:07.537 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/testdir/hardlink" 2024-04-17T10:00:07.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:00:07.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:00:07.538 INFO:teuthology.orchestra.run.smithi032.stdout: "damage_type": "dir_frag", 2024-04-17T10:00:07.538 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 3372089521, 2024-04-17T10:00:07.538 INFO:teuthology.orchestra.run.smithi032.stdout: "ino": 1099511627776, 2024-04-17T10:00:07.538 INFO:teuthology.orchestra.run.smithi032.stdout: "frag": "*", 2024-04-17T10:00:07.538 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/dir" 2024-04-17T10:00:07.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:00:07.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:00:07.539 INFO:teuthology.orchestra.run.smithi032.stdout: "damage_type": "backtrace", 2024-04-17T10:00:07.539 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 3396527091, 2024-04-17T10:00:07.539 INFO:teuthology.orchestra.run.smithi032.stdout: "ino": 1099511627776, 2024-04-17T10:00:07.539 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/dir" 2024-04-17T10:00:07.539 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:00:07.539 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-17T10:00:07.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa74a00640 1 -- 172.21.15.32:0/3582891644 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faa4c0683b0 msgr2=0x7faa4c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:07.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa74a00640 1 --2- 172.21.15.32:0/3582891644 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faa4c0683b0 0x7faa4c06a870 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7faa700ccd30 tx=0x7faa60007e40 comp rx=0 tx=0).stop 2024-04-17T10:00:07.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa74a00640 1 -- 172.21.15.32:0/3582891644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa7015b010 msgr2=0x7faa701569e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:07.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa74a00640 1 --2- 172.21.15.32:0/3582891644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa7015b010 0x7faa701569e0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7faa6800cef0 tx=0x7faa6800a710 comp rx=0 tx=0).stop 2024-04-17T10:00:07.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa74a00640 1 -- 172.21.15.32:0/3582891644 shutdown_connections 2024-04-17T10:00:07.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa74a00640 1 --2- 172.21.15.32:0/3582891644 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faa4c0683b0 0x7faa4c06a870 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa74a00640 1 --2- 172.21.15.32:0/3582891644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa7015b010 0x7faa701569e0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa74a00640 1 --2- 172.21.15.32:0/3582891644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa7015ac30 0x7faa700d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa74a00640 1 --2- 172.21.15.32:0/3582891644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa700b3d80 0x7faa70152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa74a00640 1 -- 172.21.15.32:0/3582891644 >> 172.21.15.32:0/3582891644 conn(0x7faa700b2120 msgr2=0x7faa7006f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:07.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa74a00640 1 -- 172.21.15.32:0/3582891644 shutdown_connections 2024-04-17T10:00:07.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.530+0000 7faa74a00640 1 -- 172.21.15.32:0/3582891644 wait complete. 2024-04-17T10:00:07.549 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-17T10:00:07.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.694+0000 7ff4d9400640 1 Processor -- start 2024-04-17T10:00:07.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.694+0000 7ff4d9400640 1 -- start start 2024-04-17T10:00:07.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.694+0000 7ff4d9400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4d40d0890 0x7ff4d409a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.694+0000 7ff4d9400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4d415ef10 con 0x7ff4d409af30 2024-04-17T10:00:07.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.694+0000 7ff4d9400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4d40d0db0 con 0x7ff4d415eb30 2024-04-17T10:00:07.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.694+0000 7ff4d9400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff4d406dd60 con 0x7ff4d40d0890 2024-04-17T10:00:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.694+0000 7ff4d3200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff4d409af30 0x7ff4d40ccb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39324/0 (socket says 172.21.15.32:39324) 2024-04-17T10:00:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.694+0000 7ff4d3200640 1 -- 172.21.15.32:0/2555149348 learned_addr learned my addr 172.21.15.32:0/2555149348 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:00:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.694+0000 7ff4d1e00640 1 --2- 172.21.15.32:0/2555149348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4d40d0890 0x7ff4d409a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.694+0000 7ff4d1400640 1 -- 172.21.15.32:0/2555149348 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2507736012 0 0) 0x7ff4d415ef10 con 0x7ff4d409af30 2024-04-17T10:00:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.694+0000 7ff4d1400640 1 -- 172.21.15.32:0/2555149348 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4b8003530 con 0x7ff4d409af30 2024-04-17T10:00:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.694+0000 7ff4d1400640 1 -- 172.21.15.32:0/2555149348 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 60955659 0 0) 0x7ff4d40d0db0 con 0x7ff4d415eb30 2024-04-17T10:00:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.694+0000 7ff4d1400640 1 -- 172.21.15.32:0/2555149348 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4d415ef10 con 0x7ff4d415eb30 2024-04-17T10:00:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.694+0000 7ff4d1400640 1 -- 172.21.15.32:0/2555149348 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2036787298 0 0) 0x7ff4b8003530 con 0x7ff4d409af30 2024-04-17T10:00:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.694+0000 7ff4d1e00640 1 -- 172.21.15.32:0/2555149348 >> v1:172.21.15.144:6789/0 conn(0x7ff4d415eb30 legacy=0x7ff4d409b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:07.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.694+0000 7ff4d1e00640 1 -- 172.21.15.32:0/2555149348 >> v1:172.21.15.32:6789/0 conn(0x7ff4d409af30 legacy=0x7ff4d40ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:07.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.694+0000 7ff4d1e00640 1 -- 172.21.15.32:0/2555149348 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d40d1090 con 0x7ff4d40d0890 2024-04-17T10:00:07.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.698+0000 7ff4d1e00640 1 --2- 172.21.15.32:0/2555149348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4d40d0890 0x7ff4d409a9f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00097f0 tx=0x7ff4c001ebf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c40e3a19f3e73ef server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:07.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.698+0000 7ff4d1400640 1 -- 172.21.15.32:0/2555149348 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c00b5070 con 0x7ff4d40d0890 2024-04-17T10:00:07.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.698+0000 7ff4d1400640 1 -- 172.21.15.32:0/2555149348 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c00a8670 con 0x7ff4d40d0890 2024-04-17T10:00:07.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.698+0000 7ff4d1400640 1 -- 172.21.15.32:0/2555149348 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc001a00 con 0x7ff4d40d0890 2024-04-17T10:00:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.698+0000 7ff4d9400640 1 -- 172.21.15.32:0/2555149348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4d40d0890 msgr2=0x7ff4d409a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.698+0000 7ff4d9400640 1 --2- 172.21.15.32:0/2555149348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4d40d0890 0x7ff4d409a9f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00097f0 tx=0x7ff4c001ebf0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.698+0000 7ff4d9400640 1 -- 172.21.15.32:0/2555149348 shutdown_connections 2024-04-17T10:00:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.698+0000 7ff4d9400640 1 --2- 172.21.15.32:0/2555149348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4d40d0890 0x7ff4d409a9f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.698+0000 7ff4d9400640 1 -- 172.21.15.32:0/2555149348 >> 172.21.15.32:0/2555149348 conn(0x7ff4d40b3fd0 msgr2=0x7ff4d40b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.698+0000 7ff4d9400640 1 -- 172.21.15.32:0/2555149348 shutdown_connections 2024-04-17T10:00:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.698+0000 7ff4d9400640 1 -- 172.21.15.32:0/2555149348 wait complete. 2024-04-17T10:00:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4d9400640 1 Processor -- start 2024-04-17T10:00:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4d9400640 1 -- start start 2024-04-17T10:00:07.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4d9400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4d409af30 0x7ff4d40c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4d9400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4d40d0890 0x7ff4d4152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4d9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4d415eb30 0x7ff4d40b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4d9400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d40d1930 con 0x7ff4d415eb30 2024-04-17T10:00:07.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4d9400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d40d1650 con 0x7ff4d40d0890 2024-04-17T10:00:07.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4d9400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff4d40cd240 con 0x7ff4d409af30 2024-04-17T10:00:07.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4d3200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4d415eb30 0x7ff4d40b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4d3200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4d415eb30 0x7ff4d40b72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59970/0 (socket says 172.21.15.32:59970) 2024-04-17T10:00:07.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4d3200640 1 -- 172.21.15.32:0/1709216303 learned_addr learned my addr 172.21.15.32:0/1709216303 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:07.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4d1e00640 1 --2- 172.21.15.32:0/1709216303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4d40d0890 0x7ff4d4152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4d2800640 1 --2- 172.21.15.32:0/1709216303 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4d409af30 0x7ff4d40c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4d3200640 1 -- 172.21.15.32:0/1709216303 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4d409af30 msgr2=0x7ff4d40c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:07.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4d3200640 1 --2- 172.21.15.32:0/1709216303 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4d409af30 0x7ff4d40c77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4d3200640 1 -- 172.21.15.32:0/1709216303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4d40d0890 msgr2=0x7ff4d4152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:07.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4d3200640 1 --2- 172.21.15.32:0/1709216303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4d40d0890 0x7ff4d4152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4d3200640 1 -- 172.21.15.32:0/1709216303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d40c7ce0 con 0x7ff4d415eb30 2024-04-17T10:00:07.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4d3200640 1 --2- 172.21.15.32:0/1709216303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4d415eb30 0x7ff4d40b72d0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc001f80 tx=0x7ff4cc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:07.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4caa00640 1 -- 172.21.15.32:0/1709216303 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc012400 con 0x7ff4d415eb30 2024-04-17T10:00:07.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4caa00640 1 -- 172.21.15.32:0/1709216303 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4cc0125a0 con 0x7ff4d415eb30 2024-04-17T10:00:07.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4caa00640 1 -- 172.21.15.32:0/1709216303 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc012990 con 0x7ff4d415eb30 2024-04-17T10:00:07.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4d9400640 1 -- 172.21.15.32:0/1709216303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d40c7f60 con 0x7ff4d415eb30 2024-04-17T10:00:07.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4d9400640 1 -- 172.21.15.32:0/1709216303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d40c8520 con 0x7ff4d415eb30 2024-04-17T10:00:07.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.702+0000 7ff4d9400640 1 -- 172.21.15.32:0/1709216303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4d4068a90 con 0x7ff4d415eb30 2024-04-17T10:00:07.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.706+0000 7ff4caa00640 1 -- 172.21.15.32:0/1709216303 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff4cc012b30 con 0x7ff4d415eb30 2024-04-17T10:00:07.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.706+0000 7ff4caa00640 1 --2- 172.21.15.32:0/1709216303 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff4b4068380 0x7ff4b406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:07.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.706+0000 7ff4caa00640 1 -- 172.21.15.32:0/1709216303 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7ff4cc087190 con 0x7ff4d415eb30 2024-04-17T10:00:07.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.706+0000 7ff4d2800640 1 --2- 172.21.15.32:0/1709216303 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff4b4068380 0x7ff4b406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:07.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.706+0000 7ff4d2800640 1 --2- 172.21.15.32:0/1709216303 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff4b4068380 0x7ff4b406a840 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4001720 tx=0x7ff4c4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:07.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.710+0000 7ff4caa00640 1 -- 172.21.15.32:0/1709216303 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff4d40c7ce0 con 0x7ff4d415eb30 2024-04-17T10:00:07.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.910+0000 7ff4d9400640 1 -- 172.21.15.32:0/1709216303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff4d40ce9a0 con 0x7ff4d415eb30 2024-04-17T10:00:07.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.910+0000 7ff4caa00640 1 -- 172.21.15.32:0/1709216303 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7ff4cc012e20 con 0x7ff4d415eb30 2024-04-17T10:00:07.907 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:00:07.908 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-17T10:00:07.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.910+0000 7ff4d9400640 1 -- 172.21.15.32:0/1709216303 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff4b4068380 msgr2=0x7ff4b406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:07.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.910+0000 7ff4d9400640 1 --2- 172.21.15.32:0/1709216303 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff4b4068380 0x7ff4b406a840 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4001720 tx=0x7ff4c4009040 comp rx=0 tx=0).stop 2024-04-17T10:00:07.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.910+0000 7ff4d9400640 1 -- 172.21.15.32:0/1709216303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4d415eb30 msgr2=0x7ff4d40b72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:07.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.910+0000 7ff4d9400640 1 --2- 172.21.15.32:0/1709216303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4d415eb30 0x7ff4d40b72d0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc001f80 tx=0x7ff4cc00a430 comp rx=0 tx=0).stop 2024-04-17T10:00:07.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.910+0000 7ff4d9400640 1 -- 172.21.15.32:0/1709216303 shutdown_connections 2024-04-17T10:00:07.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.910+0000 7ff4d9400640 1 --2- 172.21.15.32:0/1709216303 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff4b4068380 0x7ff4b406a840 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.910+0000 7ff4d9400640 1 --2- 172.21.15.32:0/1709216303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4d415eb30 0x7ff4d40b72d0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.910+0000 7ff4d9400640 1 --2- 172.21.15.32:0/1709216303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4d40d0890 0x7ff4d4152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.910+0000 7ff4d9400640 1 --2- 172.21.15.32:0/1709216303 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4d409af30 0x7ff4d40c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:07.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.910+0000 7ff4d9400640 1 -- 172.21.15.32:0/1709216303 >> 172.21.15.32:0/1709216303 conn(0x7ff4d40b3fd0 msgr2=0x7ff4d4070d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:07.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.914+0000 7ff4d9400640 1 -- 172.21.15.32:0/1709216303 shutdown_connections 2024-04-17T10:00:07.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:07.914+0000 7ff4d9400640 1 -- 172.21.15.32:0/1709216303 wait complete. 2024-04-17T10:00:07.930 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-17T10:00:07.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:00:07.930 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-17T10:00:08.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.078+0000 7f2eba600640 1 Processor -- start 2024-04-17T10:00:08.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.078+0000 7f2eba600640 1 -- start start 2024-04-17T10:00:08.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.078+0000 7f2eba600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2eb40cfc70 0x7f2eb406edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.078+0000 7f2eba600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2eb40cd870 0x7f2eb415ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.078+0000 7f2eba600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2eb415f200 con 0x7f2eb406f330 2024-04-17T10:00:08.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.078+0000 7f2eba600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f2eb415fb30 con 0x7f2eb40cfc70 2024-04-17T10:00:08.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.078+0000 7f2eba600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2eb406f770 con 0x7f2eb40cd870 2024-04-17T10:00:08.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.078+0000 7f2eb3e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2eb40cd870 0x7f2eb415ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.078+0000 7f2eb3400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2eb40cfc70 0x7f2eb406edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.078+0000 7f2eb3400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2eb40cfc70 0x7f2eb406edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:43274/0 (socket says 172.21.15.32:43274) 2024-04-17T10:00:08.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.078+0000 7f2eb3400640 1 -- 172.21.15.32:0/3662845344 learned_addr learned my addr 172.21.15.32:0/3662845344 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:08.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.082+0000 7f2eb3400640 1 -- 172.21.15.32:0/3662845344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2eb40cd870 msgr2=0x7f2eb415ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:08.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.082+0000 7f2eb3400640 1 --2- 172.21.15.32:0/3662845344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2eb40cd870 0x7f2eb415ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.082+0000 7f2eb3400640 1 -- 172.21.15.32:0/3662845344 >> v1:172.21.15.144:6789/0 conn(0x7f2eb406f330 legacy=0x7f2eb40cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:08.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.082+0000 7f2eb3400640 1 -- 172.21.15.32:0/3662845344 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb415f4e0 con 0x7f2eb40cfc70 2024-04-17T10:00:08.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.082+0000 7f2eb3400640 1 --2- 172.21.15.32:0/3662845344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eb40cfc70 0x7f2eb406edf0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4009870 tx=0x7f2ea401c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c95af6c4c0954154 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.082+0000 7f2eb2a00640 1 -- 172.21.15.32:0/3662845344 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40b2070 con 0x7f2eb40cfc70 2024-04-17T10:00:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.082+0000 7f2eb2a00640 1 -- 172.21.15.32:0/3662845344 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea40a9070 con 0x7f2eb40cfc70 2024-04-17T10:00:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.082+0000 7f2eb2a00640 1 -- 172.21.15.32:0/3662845344 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40ad880 con 0x7f2eb40cfc70 2024-04-17T10:00:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.082+0000 7f2eba600640 1 -- 172.21.15.32:0/3662845344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eb40cfc70 msgr2=0x7f2eb406edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.082+0000 7f2eba600640 1 --2- 172.21.15.32:0/3662845344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eb40cfc70 0x7f2eb406edf0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4009870 tx=0x7f2ea401c560 comp rx=0 tx=0).stop 2024-04-17T10:00:08.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eba600640 1 -- 172.21.15.32:0/3662845344 shutdown_connections 2024-04-17T10:00:08.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eba600640 1 --2- 172.21.15.32:0/3662845344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2eb40cd870 0x7f2eb415ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eba600640 1 --2- 172.21.15.32:0/3662845344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eb40cfc70 0x7f2eb406edf0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eba600640 1 -- 172.21.15.32:0/3662845344 >> 172.21.15.32:0/3662845344 conn(0x7f2eb40b2190 msgr2=0x7f2eb40b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:08.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eba600640 1 -- 172.21.15.32:0/3662845344 shutdown_connections 2024-04-17T10:00:08.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eba600640 1 -- 172.21.15.32:0/3662845344 wait complete. 2024-04-17T10:00:08.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eba600640 1 Processor -- start 2024-04-17T10:00:08.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eba600640 1 -- start start 2024-04-17T10:00:08.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eba600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eb406f330 0x7f2eb40b0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eba600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2eb40cd870 0x7f2eb4164580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eba600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2eb40cfc70 0x7f2eb4164ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eba600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb4168330 con 0x7f2eb40cd870 2024-04-17T10:00:08.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eba600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb41681b0 con 0x7f2eb406f330 2024-04-17T10:00:08.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eba600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2eb41684b0 con 0x7f2eb40cfc70 2024-04-17T10:00:08.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eabe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2eb40cd870 0x7f2eb4164580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eabe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2eb40cd870 0x7f2eb4164580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59972/0 (socket says 172.21.15.32:59972) 2024-04-17T10:00:08.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eabe00640 1 -- 172.21.15.32:0/2226685749 learned_addr learned my addr 172.21.15.32:0/2226685749 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:08.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eb3400640 1 --2- 172.21.15.32:0/2226685749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eb406f330 0x7f2eb40b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eb3e00640 1 --2- 172.21.15.32:0/2226685749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2eb40cfc70 0x7f2eb4164ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eabe00640 1 -- 172.21.15.32:0/2226685749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2eb40cfc70 msgr2=0x7f2eb4164ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:08.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eabe00640 1 --2- 172.21.15.32:0/2226685749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2eb40cfc70 0x7f2eb4164ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eabe00640 1 -- 172.21.15.32:0/2226685749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eb406f330 msgr2=0x7f2eb40b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:08.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eabe00640 1 --2- 172.21.15.32:0/2226685749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eb406f330 0x7f2eb40b0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eabe00640 1 -- 172.21.15.32:0/2226685749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb40b11e0 con 0x7f2eb40cd870 2024-04-17T10:00:08.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eb3400640 1 --2- 172.21.15.32:0/2226685749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eb406f330 0x7f2eb40b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:08.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.086+0000 7f2eabe00640 1 --2- 172.21.15.32:0/2226685749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2eb40cd870 0x7f2eb4164580 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00c400 tx=0x7f2e9c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:08.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.090+0000 7f2eab400640 1 -- 172.21.15.32:0/2226685749 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c0052d0 con 0x7f2eb40cd870 2024-04-17T10:00:08.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.090+0000 7f2eab400640 1 -- 172.21.15.32:0/2226685749 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e9c0054b0 con 0x7f2eb40cd870 2024-04-17T10:00:08.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.090+0000 7f2eba600640 1 -- 172.21.15.32:0/2226685749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb40b14c0 con 0x7f2eb40cd870 2024-04-17T10:00:08.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.090+0000 7f2eab400640 1 -- 172.21.15.32:0/2226685749 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c006a70 con 0x7f2eb40cd870 2024-04-17T10:00:08.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.090+0000 7f2eba600640 1 -- 172.21.15.32:0/2226685749 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb40b1a80 con 0x7f2eb40cd870 2024-04-17T10:00:08.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.090+0000 7f2eab400640 1 -- 172.21.15.32:0/2226685749 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2e9c006cd0 con 0x7f2eb40cd870 2024-04-17T10:00:08.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.090+0000 7f2eab400640 1 --2- 172.21.15.32:0/2226685749 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e90068380 0x7f2e9006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.090+0000 7f2eab400640 1 -- 172.21.15.32:0/2226685749 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7f2e9c085ff0 con 0x7f2eb40cd870 2024-04-17T10:00:08.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.090+0000 7f2eb3400640 1 --2- 172.21.15.32:0/2226685749 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e90068380 0x7f2e9006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.090+0000 7f2ebdaf2000 1 Processor -- start 2024-04-17T10:00:08.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.090+0000 7f2ebdaf2000 1 -- start start 2024-04-17T10:00:08.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2eb3400640 1 --2- 172.21.15.32:0/2226685749 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e90068380 0x7f2e9006a840 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40af000 tx=0x7f2ea40a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2ebdaf2000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c34620d700 0x55c34620dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2ebdaf2000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c34620e020 0x55c3461c04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2ebdaf2000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3461c0a10 0x55c3461c0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2ebdaf2000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c3460125f0 con 0x55c34620d700 2024-04-17T10:00:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2ebdaf2000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c345fcb7a0 con 0x55c3461c0a10 2024-04-17T10:00:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2ebdaf2000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55c34600e5b0 con 0x55c34620e020 2024-04-17T10:00:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2eb3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c34620d700 0x55c34620dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2eb3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c34620d700 0x55c34620dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59988/0 (socket says 172.21.15.32:59988) 2024-04-17T10:00:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2eb3400640 1 -- 172.21.15.32:0/3510749870 learned_addr learned my addr 172.21.15.32:0/3510749870 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2eb3e00640 1 --2- 172.21.15.32:0/3510749870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3461c0a10 0x55c3461c0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2eabe00640 1 --2- 172.21.15.32:0/3510749870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c34620e020 0x55c3461c04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2eb3400640 1 -- 172.21.15.32:0/3510749870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c34620e020 msgr2=0x55c3461c04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2eb3400640 1 --2- 172.21.15.32:0/3510749870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c34620e020 0x55c3461c04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2eb3400640 1 -- 172.21.15.32:0/3510749870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3461c0a10 msgr2=0x55c3461c0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2eb3400640 1 --2- 172.21.15.32:0/3510749870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3461c0a10 0x55c3461c0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2eb3400640 1 -- 172.21.15.32:0/3510749870 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3460b96e0 con 0x55c34620d700 2024-04-17T10:00:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2eb3400640 1 --2- 172.21.15.32:0/3510749870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c34620d700 0x55c34620dae0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f2ea401c5a0 tx=0x7f2ea4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6dedbab66d81170 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2ea3e00640 1 -- 172.21.15.32:0/3510749870 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40b2040 con 0x55c34620d700 2024-04-17T10:00:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2ea3e00640 1 -- 172.21.15.32:0/3510749870 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea40c7070 con 0x55c34620d700 2024-04-17T10:00:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2ea3e00640 1 -- 172.21.15.32:0/3510749870 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40ade60 con 0x55c34620d700 2024-04-17T10:00:08.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/3510749870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c34620d700 msgr2=0x55c34620dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2ebdaf2000 1 --2- 172.21.15.32:0/3510749870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c34620d700 0x55c34620dae0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f2ea401c5a0 tx=0x7f2ea4004740 comp rx=0 tx=0).stop 2024-04-17T10:00:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/3510749870 shutdown_connections 2024-04-17T10:00:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2ebdaf2000 1 --2- 172.21.15.32:0/3510749870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c34620e020 0x55c3461c04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2ebdaf2000 1 --2- 172.21.15.32:0/3510749870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3461c0a10 0x55c3461c0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2ebdaf2000 1 --2- 172.21.15.32:0/3510749870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c34620d700 0x55c34620dae0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/3510749870 >> 172.21.15.32:0/3510749870 conn(0x55c3461ee290 msgr2=0x55c3461f0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/3510749870 shutdown_connections 2024-04-17T10:00:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/3510749870 wait complete. 2024-04-17T10:00:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2ebdaf2000 1 -- start start 2024-04-17T10:00:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.094+0000 7f2ebdaf2000 1 Processor -- start 2024-04-17T10:00:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ebdaf2000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3461c0a10 0x55c3461d78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ebdaf2000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c34620d700 0x55c3461d7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ebdaf2000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c34620e020 0x55c3461d8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ebdaf2000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c34601d520 con 0x55c3461c0a10 2024-04-17T10:00:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ebdaf2000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c345faaea0 con 0x55c34620d700 2024-04-17T10:00:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ebdaf2000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55c345fd1500 con 0x55c34620e020 2024-04-17T10:00:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ebdaf2000 10 client.0 _ll_register_callbacks cb 0x55c3461ae170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:00:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2eb3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3461c0a10 0x55c3461d78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2eb3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3461c0a10 0x55c3461d78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59992/0 (socket says 172.21.15.32:59992) 2024-04-17T10:00:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2eb3400640 1 -- 172.21.15.32:0/56459288 learned_addr learned my addr 172.21.15.32:0/56459288 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2eabe00640 1 --2- 172.21.15.32:0/56459288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c34620d700 0x55c3461d7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2eb3e00640 1 --2- 172.21.15.32:0/56459288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c34620e020 0x55c3461d8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2eb3400640 1 -- 172.21.15.32:0/56459288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c34620e020 msgr2=0x55c3461d8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2eb3400640 1 --2- 172.21.15.32:0/56459288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c34620e020 0x55c3461d8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2eb3400640 1 -- 172.21.15.32:0/56459288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c34620d700 msgr2=0x55c3461d7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:08.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2eb3400640 1 --2- 172.21.15.32:0/56459288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c34620d700 0x55c3461d7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2eb3400640 1 -- 172.21.15.32:0/56459288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c345feb540 con 0x55c3461c0a10 2024-04-17T10:00:08.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2eb3400640 1 --2- 172.21.15.32:0/56459288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3461c0a10 0x55c3461d78f0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f2ea401c5a0 tx=0x7f2ea401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c3ac2fb3aecf8e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:08.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ea0c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:00:08.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ea0c00640 1 -- 172.21.15.32:0/56459288 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40cf070 con 0x55c3461c0a10 2024-04-17T10:00:08.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ea0c00640 1 -- 172.21.15.32:0/56459288 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7f2ea40d4460 con 0x55c3461c0a10 2024-04-17T10:00:08.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/56459288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ea40c7070 con 0x55c3461c0a10 2024-04-17T10:00:08.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ea0c00640 1 -- 172.21.15.32:0/56459288 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f2ea40b2040 con 0x55c3461c0a10 2024-04-17T10:00:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ebdaf2000 10 client.8034 fetch_fsmap learned FSMap version 66 2024-04-17T10:00:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/56459288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55c346056ba0 con 0x55c3461c0a10 2024-04-17T10:00:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ea0c00640 1 -- 172.21.15.32:0/56459288 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea40c7070 con 0x55c3461c0a10 2024-04-17T10:00:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ea0c00640 1 -- 172.21.15.32:0/56459288 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7f2ea40cb3f0 con 0x55c3461c0a10 2024-04-17T10:00:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ebdaf2000 10 client.8034 fetch_fsmap finished waiting for FSMap version 66 2024-04-17T10:00:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ebdaf2000 10 client.8034 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T10:00:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ebdaf2000 20 client.8034 populate_metadata read hostname 'smithi032' 2024-04-17T10:00:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ebdaf2000 1 --2- 172.21.15.32:0/56459288 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55c3461d8d80 0x55c34621e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ebdaf2000 4 client.8034 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:00:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/56459288 --> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3460cb450 con 0x55c3461d8d80 2024-04-17T10:00:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2eabe00640 1 --2- 172.21.15.32:0/56459288 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55c3461d8d80 0x55c34621e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ea0c00640 10 client.8034 ms_handle_connect on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2eabe00640 1 --2- 172.21.15.32:0/56459288 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55c3461d8d80 0x55c34621e9b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2175a99ce1cec25b server_cookie=e67307dffa4ba545 in_seq=0 out_seq=0 2024-04-17T10:00:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ea0c00640 1 -- 172.21.15.32:0/56459288 <== mds.0 v2:172.21.15.32:6835/1253497636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c3460cb450 con 0x55c3461d8d80 2024-04-17T10:00:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ea0c00640 10 client.8034 handle_command_reply: tid=0 2024-04-17T10:00:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.098+0000 7f2ebdaf2000 1 client.8034 shutdown 2024-04-17T10:00:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.102+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/56459288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3461c0a10 msgr2=0x55c3461d78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.102+0000 7f2ebdaf2000 1 --2- 172.21.15.32:0/56459288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3461c0a10 0x55c3461d78f0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f2ea401c5a0 tx=0x7f2ea401ada0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.102+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/56459288 shutdown_connections 2024-04-17T10:00:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.102+0000 7f2ebdaf2000 1 --2- 172.21.15.32:0/56459288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c34620e020 0x55c3461d8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.102+0000 7f2ebdaf2000 1 --2- 172.21.15.32:0/56459288 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55c3461d8d80 0x55c34621e9b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.102+0000 7f2ebdaf2000 1 --2- 172.21.15.32:0/56459288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c34620d700 0x55c3461d7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.102+0000 7f2ebdaf2000 1 --2- 172.21.15.32:0/56459288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3461c0a10 0x55c3461d78f0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.102+0000 7f2ea0c00640 0 client.8034 ms_handle_reset on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.102+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/56459288 >> 172.21.15.32:0/56459288 conn(0x55c3461ee290 msgr2=0x55c3461f0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:08.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.102+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/56459288 shutdown_connections 2024-04-17T10:00:08.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.102+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/56459288 wait complete. 2024-04-17T10:00:08.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.102+0000 7f2ebdaf2000 20 client.8034 trim_cache size 0 max 16384 2024-04-17T10:00:08.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.118+0000 7f2ebdaf2000 1 Processor -- start 2024-04-17T10:00:08.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.118+0000 7f2ebdaf2000 1 -- start start 2024-04-17T10:00:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.118+0000 7f2ebdaf2000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3461c0a10 0x55c346221500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.118+0000 7f2ebdaf2000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c34620d700 0x55c346221a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.118+0000 7f2ebdaf2000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c34620e020 0x55c34621e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.118+0000 7f2ebdaf2000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb4158a80 con 0x55c3461c0a10 2024-04-17T10:00:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.118+0000 7f2ebdaf2000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb4154930 con 0x55c34620e020 2024-04-17T10:00:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.118+0000 7f2ebdaf2000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2eb4158430 con 0x55c34620d700 2024-04-17T10:00:08.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.118+0000 7f2eb3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3461c0a10 0x55c346221500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.118+0000 7f2eb3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3461c0a10 0x55c346221500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60000/0 (socket says 172.21.15.32:60000) 2024-04-17T10:00:08.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.118+0000 7f2eb3400640 1 -- 172.21.15.32:0/2123714292 learned_addr learned my addr 172.21.15.32:0/2123714292 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:08.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.118+0000 7f2eb3e00640 1 --2- 172.21.15.32:0/2123714292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c34620e020 0x55c34621e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.118+0000 7f2eabe00640 1 --2- 172.21.15.32:0/2123714292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c34620d700 0x55c346221a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.118+0000 7f2eb3400640 1 -- 172.21.15.32:0/2123714292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c34620d700 msgr2=0x55c346221a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:08.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.118+0000 7f2eb3400640 1 --2- 172.21.15.32:0/2123714292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c34620d700 0x55c346221a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.118+0000 7f2eb3400640 1 -- 172.21.15.32:0/2123714292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c34620e020 msgr2=0x55c34621e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:08.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.118+0000 7f2eb3400640 1 --2- 172.21.15.32:0/2123714292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c34620e020 0x55c34621e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.118+0000 7f2eb3400640 1 -- 172.21.15.32:0/2123714292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c34601ef70 con 0x55c3461c0a10 2024-04-17T10:00:08.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.118+0000 7f2eb3e00640 1 --2- 172.21.15.32:0/2123714292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c34620e020 0x55c34621e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:00:08.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.118+0000 7f2eabe00640 1 --2- 172.21.15.32:0/2123714292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c34620d700 0x55c346221a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:00:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2eb3400640 1 --2- 172.21.15.32:0/2123714292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3461c0a10 0x55c346221500 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4004780 tx=0x7f2ea400b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e3483b79aaae4b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ea3400640 1 -- 172.21.15.32:0/2123714292 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40cf070 con 0x55c3461c0a10 2024-04-17T10:00:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ea3400640 1 -- 172.21.15.32:0/2123714292 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea40c7040 con 0x55c3461c0a10 2024-04-17T10:00:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ea3400640 1 -- 172.21.15.32:0/2123714292 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40cb8f0 con 0x55c3461c0a10 2024-04-17T10:00:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/2123714292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3461c0a10 msgr2=0x55c346221500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ebdaf2000 1 --2- 172.21.15.32:0/2123714292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3461c0a10 0x55c346221500 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4004780 tx=0x7f2ea400b050 comp rx=0 tx=0).stop 2024-04-17T10:00:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/2123714292 shutdown_connections 2024-04-17T10:00:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ebdaf2000 1 --2- 172.21.15.32:0/2123714292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c34620d700 0x55c346221a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ebdaf2000 1 --2- 172.21.15.32:0/2123714292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c34620e020 0x55c34621e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ebdaf2000 1 --2- 172.21.15.32:0/2123714292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3461c0a10 0x55c346221500 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/2123714292 >> 172.21.15.32:0/2123714292 conn(0x55c3461ee290 msgr2=0x55c3461bec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/2123714292 shutdown_connections 2024-04-17T10:00:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/2123714292 wait complete. 2024-04-17T10:00:08.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ebdaf2000 1 -- start start 2024-04-17T10:00:08.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ebdaf2000 1 Processor -- start 2024-04-17T10:00:08.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ebdaf2000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c3461c0a10 0x55c346221a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ebdaf2000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c34620d700 0x55c346238540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ebdaf2000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c34620e020 0x55c346238a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ebdaf2000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c346016c20 con 0x55c34620e020 2024-04-17T10:00:08.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ebdaf2000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c346011900 con 0x55c34620d700 2024-04-17T10:00:08.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ebdaf2000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55c34601f800 con 0x55c3461c0a10 2024-04-17T10:00:08.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ebdaf2000 10 client.0 _ll_register_callbacks cb 0x7f2eac009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:00:08.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2eb3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c34620e020 0x55c346238a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2eb3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c34620e020 0x55c346238a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60012/0 (socket says 172.21.15.32:60012) 2024-04-17T10:00:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2eb3e00640 1 -- 172.21.15.32:0/2841603941 learned_addr learned my addr 172.21.15.32:0/2841603941 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2eabe00640 1 --2- 172.21.15.32:0/2841603941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c34620d700 0x55c346238540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2eb3400640 1 --2- 172.21.15.32:0/2841603941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c3461c0a10 0x55c346221a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2eb3e00640 1 -- 172.21.15.32:0/2841603941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c3461c0a10 msgr2=0x55c346221a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2eb3e00640 1 --2- 172.21.15.32:0/2841603941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c3461c0a10 0x55c346221a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2eb3e00640 1 -- 172.21.15.32:0/2841603941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c34620d700 msgr2=0x55c346238540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2eb3e00640 1 --2- 172.21.15.32:0/2841603941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c34620d700 0x55c346238540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2eb3e00640 1 -- 172.21.15.32:0/2841603941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c345fcb9f0 con 0x55c34620e020 2024-04-17T10:00:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2eb3e00640 1 --2- 172.21.15.32:0/2841603941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c34620e020 0x55c346238a80 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f2eac016d80 tx=0x7f2eac007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06264f8561f1b73 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ea0c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:00:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ea0c00640 1 -- 172.21.15.32:0/2841603941 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac018420 con 0x55c34620e020 2024-04-17T10:00:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ea0c00640 1 -- 172.21.15.32:0/2841603941 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7f2eac0185c0 con 0x55c34620e020 2024-04-17T10:00:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.122+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/2841603941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ea40c7040 con 0x55c34620e020 2024-04-17T10:00:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ea0c00640 1 -- 172.21.15.32:0/2841603941 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f2eac02a050 con 0x55c34620e020 2024-04-17T10:00:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ebdaf2000 10 client.8040 fetch_fsmap learned FSMap version 66 2024-04-17T10:00:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/2841603941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55c345fb7660 con 0x55c34620e020 2024-04-17T10:00:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ea0c00640 1 -- 172.21.15.32:0/2841603941 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea40c7040 con 0x55c34620e020 2024-04-17T10:00:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ea0c00640 1 -- 172.21.15.32:0/2841603941 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7f2eac02d020 con 0x55c34620e020 2024-04-17T10:00:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ebdaf2000 10 client.8040 fetch_fsmap finished waiting for FSMap version 66 2024-04-17T10:00:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ebdaf2000 10 client.8040 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T10:00:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ebdaf2000 20 client.8040 populate_metadata read hostname 'smithi032' 2024-04-17T10:00:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ebdaf2000 1 --2- 172.21.15.32:0/2841603941 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55c34627bfa0 0x55c34627c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ebdaf2000 4 client.8040 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-17T10:00:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/2841603941 --> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x55c345fabed0 con 0x55c34627bfa0 2024-04-17T10:00:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2eb3400640 1 --2- 172.21.15.32:0/2841603941 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55c34627bfa0 0x55c34627c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2eb3400640 1 --2- 172.21.15.32:0/2841603941 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55c34627bfa0 0x55c34627c3b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=900805d13d4e00b2 server_cookie=43ff591b5269df26 in_seq=0 out_seq=0 2024-04-17T10:00:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ea0c00640 10 client.8040 ms_handle_connect on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ea0c00640 1 -- 172.21.15.32:0/2841603941 <== mds.0 v2:172.21.15.32:6835/1253497636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c345fabed0 con 0x55c34627bfa0 2024-04-17T10:00:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ea0c00640 10 client.8040 handle_command_reply: tid=0 2024-04-17T10:00:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ebdaf2000 1 client.8040 shutdown 2024-04-17T10:00:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/2841603941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c34620e020 msgr2=0x55c346238a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ebdaf2000 1 --2- 172.21.15.32:0/2841603941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c34620e020 0x55c346238a80 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f2eac016d80 tx=0x7f2eac007700 comp rx=0 tx=0).stop 2024-04-17T10:00:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/2841603941 shutdown_connections 2024-04-17T10:00:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ebdaf2000 1 --2- 172.21.15.32:0/2841603941 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55c34627bfa0 0x55c34627c3b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ebdaf2000 1 --2- 172.21.15.32:0/2841603941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c34620e020 0x55c346238a80 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ebdaf2000 1 --2- 172.21.15.32:0/2841603941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c34620d700 0x55c346238540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ebdaf2000 1 --2- 172.21.15.32:0/2841603941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c3461c0a10 0x55c346221a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/2841603941 >> 172.21.15.32:0/2841603941 conn(0x55c3461ee290 msgr2=0x55c3461d7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ea0c00640 0 client.8040 ms_handle_reset on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/2841603941 shutdown_connections 2024-04-17T10:00:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ebdaf2000 1 -- 172.21.15.32:0/2841603941 wait complete. 2024-04-17T10:00:08.128 INFO:tasks.ceph.mds.c.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f3482a00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv17 ap=3 DIRTYPARENT f(v0 m2024-04-17T09:59:18.000169+0000 1=1+0)->f(v1 m2024-04-17T09:59:18.000169+0000) n(v0 rc2024-04-17T09:59:18.115350+0000 2=1+1)->n(v0 rc2024-04-17T09:59:18.115350+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55d3b591a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-17T09:59:18.000169+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-17T09:59:18.115350+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-17T10:00:08.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2ebdaf2000 20 client.8040 trim_cache size 0 max 16384 2024-04-17T10:00:08.128 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:00:08.128 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-17T10:00:08.129 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "6e2e9c2e-8869-407a-9121-5e5536672271", 2024-04-17T10:00:08.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-17T10:00:08.129 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:00:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2eba600640 1 -- 172.21.15.32:0/2226685749 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e90068380 msgr2=0x7f2e9006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2eba600640 1 --2- 172.21.15.32:0/2226685749 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e90068380 0x7f2e9006a840 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40af000 tx=0x7f2ea40a7000 comp rx=0 tx=0).stop 2024-04-17T10:00:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2eba600640 1 -- 172.21.15.32:0/2226685749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2eb40cd870 msgr2=0x7f2eb4164580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2eba600640 1 --2- 172.21.15.32:0/2226685749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2eb40cd870 0x7f2eb4164580 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00c400 tx=0x7f2e9c00c8c0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2eba600640 1 -- 172.21.15.32:0/2226685749 shutdown_connections 2024-04-17T10:00:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2eba600640 1 --2- 172.21.15.32:0/2226685749 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e90068380 0x7f2e9006a840 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2eba600640 1 --2- 172.21.15.32:0/2226685749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2eb40cfc70 0x7f2eb4164ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2eba600640 1 --2- 172.21.15.32:0/2226685749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2eb40cd870 0x7f2eb4164580 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2eba600640 1 --2- 172.21.15.32:0/2226685749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eb406f330 0x7f2eb40b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2eba600640 1 -- 172.21.15.32:0/2226685749 >> 172.21.15.32:0/2226685749 conn(0x7f2eb40b2190 msgr2=0x7f2eb40b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2eba600640 1 -- 172.21.15.32:0/2226685749 shutdown_connections 2024-04-17T10:00:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.126+0000 7f2eba600640 1 -- 172.21.15.32:0/2226685749 wait complete. 2024-04-17T10:00:08.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:00:08.145 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-17T10:00:08.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.290+0000 7fe3f2c00640 1 Processor -- start 2024-04-17T10:00:08.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.290+0000 7fe3f2c00640 1 -- start start 2024-04-17T10:00:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.294+0000 7fe3f2c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3ec1656d0 0x7fe3ec167ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.294+0000 7fe3f2c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3ec0d0130 con 0x7fe3ec0cf370 2024-04-17T10:00:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.294+0000 7fe3f2c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3ec15d090 con 0x7fe3ec06ba30 2024-04-17T10:00:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.294+0000 7fe3f2c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe3ec0cfef0 con 0x7fe3ec1656d0 2024-04-17T10:00:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.294+0000 7fe3f0c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3ec1656d0 0x7fe3ec167ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.294+0000 7fe3e3400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe3ec0cf370 0x7fe3ec0cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39332/0 (socket says 172.21.15.32:39332) 2024-04-17T10:00:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.294+0000 7fe3e3400640 1 -- 172.21.15.32:0/3562345395 learned_addr learned my addr 172.21.15.32:0/3562345395 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:00:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.294+0000 7fe3f0c00640 1 -- 172.21.15.32:0/3562345395 >> v1:172.21.15.144:6789/0 conn(0x7fe3ec06ba30 legacy=0x7fe3ec06be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:08.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.294+0000 7fe3f0c00640 1 -- 172.21.15.32:0/3562345395 >> v1:172.21.15.32:6789/0 conn(0x7fe3ec0cf370 legacy=0x7fe3ec0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:08.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.294+0000 7fe3f0c00640 1 -- 172.21.15.32:0/3562345395 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3ec15d370 con 0x7fe3ec1656d0 2024-04-17T10:00:08.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.294+0000 7fe3eb400640 1 -- 172.21.15.32:0/3562345395 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 567813902 0 0) 0x7fe3ec0d0130 con 0x7fe3ec0cf370 2024-04-17T10:00:08.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.298+0000 7fe3f0c00640 1 --2- 172.21.15.32:0/3562345395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3ec1656d0 0x7fe3ec167ac0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4009d90 tx=0x7fe3e401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636e2bfd1e5cdc5e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:08.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.298+0000 7fe3eb400640 1 -- 172.21.15.32:0/3562345395 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e40b2070 con 0x7fe3ec1656d0 2024-04-17T10:00:08.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.298+0000 7fe3eb400640 1 -- 172.21.15.32:0/3562345395 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3e40a58b0 con 0x7fe3ec1656d0 2024-04-17T10:00:08.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.298+0000 7fe3eb400640 1 -- 172.21.15.32:0/3562345395 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e40a5c80 con 0x7fe3ec1656d0 2024-04-17T10:00:08.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.298+0000 7fe3f2c00640 1 -- 172.21.15.32:0/3562345395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3ec1656d0 msgr2=0x7fe3ec167ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:08.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.298+0000 7fe3f2c00640 1 --2- 172.21.15.32:0/3562345395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3ec1656d0 0x7fe3ec167ac0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4009d90 tx=0x7fe3e401c8e0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.298+0000 7fe3f2c00640 1 -- 172.21.15.32:0/3562345395 shutdown_connections 2024-04-17T10:00:08.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.298+0000 7fe3f2c00640 1 --2- 172.21.15.32:0/3562345395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3ec1656d0 0x7fe3ec167ac0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.298+0000 7fe3f2c00640 1 -- 172.21.15.32:0/3562345395 >> 172.21.15.32:0/3562345395 conn(0x7fe3ec0b2120 msgr2=0x7fe3ec0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.298+0000 7fe3f2c00640 1 -- 172.21.15.32:0/3562345395 shutdown_connections 2024-04-17T10:00:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.298+0000 7fe3f2c00640 1 -- 172.21.15.32:0/3562345395 wait complete. 2024-04-17T10:00:08.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.298+0000 7fe3f2c00640 1 Processor -- start 2024-04-17T10:00:08.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.298+0000 7fe3f2c00640 1 -- start start 2024-04-17T10:00:08.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.298+0000 7fe3f2c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3ec06ba30 0x7fe3ec167430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.298+0000 7fe3f2c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3ec0cf370 0x7fe3ec141b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.302+0000 7fe3f2c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3ec1656d0 0x7fe3ec1420a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.302+0000 7fe3f2c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec146230 con 0x7fe3ec0cf370 2024-04-17T10:00:08.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.302+0000 7fe3f2c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec15dd80 con 0x7fe3ec1656d0 2024-04-17T10:00:08.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.302+0000 7fe3f2c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe3ec1463b0 con 0x7fe3ec06ba30 2024-04-17T10:00:08.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.302+0000 7fe3e3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3ec0cf370 0x7fe3ec141b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.302+0000 7fe3e3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3ec0cf370 0x7fe3ec141b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60040/0 (socket says 172.21.15.32:60040) 2024-04-17T10:00:08.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.302+0000 7fe3e3400640 1 -- 172.21.15.32:0/1341936933 learned_addr learned my addr 172.21.15.32:0/1341936933 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:08.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.302+0000 7fe3f0c00640 1 --2- 172.21.15.32:0/1341936933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3ec1656d0 0x7fe3ec1420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.302+0000 7fe3ebe00640 1 --2- 172.21.15.32:0/1341936933 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3ec06ba30 0x7fe3ec167430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.302+0000 7fe3e3400640 1 -- 172.21.15.32:0/1341936933 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3ec06ba30 msgr2=0x7fe3ec167430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:08.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.302+0000 7fe3e3400640 1 --2- 172.21.15.32:0/1341936933 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3ec06ba30 0x7fe3ec167430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.302+0000 7fe3e3400640 1 -- 172.21.15.32:0/1341936933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3ec1656d0 msgr2=0x7fe3ec1420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:08.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.302+0000 7fe3e3400640 1 --2- 172.21.15.32:0/1341936933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3ec1656d0 0x7fe3ec1420a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.302+0000 7fe3e3400640 1 -- 172.21.15.32:0/1341936933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3ec1425e0 con 0x7fe3ec0cf370 2024-04-17T10:00:08.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.302+0000 7fe3e3400640 1 --2- 172.21.15.32:0/1341936933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3ec0cf370 0x7fe3ec141b60 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400c4a0 tx=0x7fe3d400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:08.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.302+0000 7fe3e8c00640 1 -- 172.21.15.32:0/1341936933 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4018070 con 0x7fe3ec0cf370 2024-04-17T10:00:08.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.302+0000 7fe3e8c00640 1 -- 172.21.15.32:0/1341936933 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3d40074e0 con 0x7fe3ec0cf370 2024-04-17T10:00:08.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.302+0000 7fe3f2c00640 1 -- 172.21.15.32:0/1341936933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3ec142890 con 0x7fe3ec0cf370 2024-04-17T10:00:08.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.302+0000 7fe3e8c00640 1 -- 172.21.15.32:0/1341936933 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4004a30 con 0x7fe3ec0cf370 2024-04-17T10:00:08.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.302+0000 7fe3f2c00640 1 -- 172.21.15.32:0/1341936933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3ec0b39a0 con 0x7fe3ec0cf370 2024-04-17T10:00:08.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.302+0000 7fe3e8c00640 1 -- 172.21.15.32:0/1341936933 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe3d401b070 con 0x7fe3ec0cf370 2024-04-17T10:00:08.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3e8c00640 1 --2- 172.21.15.32:0/1341936933 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe3c8068380 0x7fe3c806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3e8c00640 1 -- 172.21.15.32:0/1341936933 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7fe3d4085d00 con 0x7fe3ec0cf370 2024-04-17T10:00:08.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3ebe00640 1 --2- 172.21.15.32:0/1341936933 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe3c8068380 0x7fe3c806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f6190000 1 Processor -- start 2024-04-17T10:00:08.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f6190000 1 -- start start 2024-04-17T10:00:08.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3ebe00640 1 --2- 172.21.15.32:0/1341936933 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe3c8068380 0x7fe3c806a840 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc007fd0 tx=0x7fe3dc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:08.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f6190000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff362af260 0x55ff362af640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f6190000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff362afb80 0x55ff36262460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f6190000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362629a0 0x55ff36262d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f6190000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ff360b44f0 con 0x55ff362629a0 2024-04-17T10:00:08.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f6190000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ff3606d6c0 con 0x55ff362afb80 2024-04-17T10:00:08.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f6190000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ff360b04b0 con 0x55ff362af260 2024-04-17T10:00:08.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f0c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362629a0 0x55ff36262d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3ebe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff362af260 0x55ff362af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f0c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362629a0 0x55ff36262d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60050/0 (socket says 172.21.15.32:60050) 2024-04-17T10:00:08.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f0c00640 1 -- 172.21.15.32:0/2576260114 learned_addr learned my addr 172.21.15.32:0/2576260114 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:08.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3e3400640 1 --2- 172.21.15.32:0/2576260114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff362afb80 0x55ff36262460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f0c00640 1 -- 172.21.15.32:0/2576260114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff362af260 msgr2=0x55ff362af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:08.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f0c00640 1 --2- 172.21.15.32:0/2576260114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff362af260 0x55ff362af640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f0c00640 1 -- 172.21.15.32:0/2576260114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff362afb80 msgr2=0x55ff36262460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:08.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f0c00640 1 --2- 172.21.15.32:0/2576260114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff362afb80 0x55ff36262460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f0c00640 1 -- 172.21.15.32:0/2576260114 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff360f98e0 con 0x55ff362629a0 2024-04-17T10:00:08.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f0c00640 1 --2- 172.21.15.32:0/2576260114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362629a0 0x55ff36262d80 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fe3e401c920 tx=0x7fe3e4004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9208dfe4c156437 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:08.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3e0c00640 1 -- 172.21.15.32:0/2576260114 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e40bc030 con 0x55ff362629a0 2024-04-17T10:00:08.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3e0c00640 1 -- 172.21.15.32:0/2576260114 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3e40b3040 con 0x55ff362629a0 2024-04-17T10:00:08.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3e0c00640 1 -- 172.21.15.32:0/2576260114 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e40b2070 con 0x55ff362629a0 2024-04-17T10:00:08.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f6190000 1 -- 172.21.15.32:0/2576260114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362629a0 msgr2=0x55ff36262d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:08.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f6190000 1 --2- 172.21.15.32:0/2576260114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362629a0 0x55ff36262d80 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fe3e401c920 tx=0x7fe3e4004ac0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f6190000 1 -- 172.21.15.32:0/2576260114 shutdown_connections 2024-04-17T10:00:08.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f6190000 1 --2- 172.21.15.32:0/2576260114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362629a0 0x55ff36262d80 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f6190000 1 --2- 172.21.15.32:0/2576260114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff362afb80 0x55ff36262460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f6190000 1 --2- 172.21.15.32:0/2576260114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff362af260 0x55ff362af640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f6190000 1 -- 172.21.15.32:0/2576260114 >> 172.21.15.32:0/2576260114 conn(0x55ff36290000 msgr2=0x55ff362923f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:08.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f6190000 1 -- 172.21.15.32:0/2576260114 shutdown_connections 2024-04-17T10:00:08.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.306+0000 7fe3f6190000 1 -- 172.21.15.32:0/2576260114 wait complete. 2024-04-17T10:00:08.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3f6190000 1 -- start start 2024-04-17T10:00:08.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3f6190000 1 Processor -- start 2024-04-17T10:00:08.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3f6190000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362629a0 0x55ff36268a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3f6190000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff362af260 0x55ff36268f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3f6190000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff362afb80 0x55ff362694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3f6190000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ff361357b0 con 0x55ff362629a0 2024-04-17T10:00:08.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3f6190000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ff3616fd00 con 0x55ff362af260 2024-04-17T10:00:08.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3f6190000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ff360792c0 con 0x55ff362afb80 2024-04-17T10:00:08.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3f6190000 10 client.0 _ll_register_callbacks cb 0x55ff36089030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:00:08.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3ebe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362629a0 0x55ff36268a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3ebe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362629a0 0x55ff36268a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60052/0 (socket says 172.21.15.32:60052) 2024-04-17T10:00:08.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3ebe00640 1 -- 172.21.15.32:0/494709516 learned_addr learned my addr 172.21.15.32:0/494709516 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:08.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3f0c00640 1 --2- 172.21.15.32:0/494709516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff362afb80 0x55ff362694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3e3400640 1 --2- 172.21.15.32:0/494709516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff362af260 0x55ff36268f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3ebe00640 1 -- 172.21.15.32:0/494709516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff362afb80 msgr2=0x55ff362694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:08.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3ebe00640 1 --2- 172.21.15.32:0/494709516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff362afb80 0x55ff362694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3ebe00640 1 -- 172.21.15.32:0/494709516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff362af260 msgr2=0x55ff36268f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:08.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3ebe00640 1 --2- 172.21.15.32:0/494709516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff362af260 0x55ff36268f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3ebe00640 1 -- 172.21.15.32:0/494709516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff3608c490 con 0x55ff362629a0 2024-04-17T10:00:08.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3f0c00640 1 --2- 172.21.15.32:0/494709516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff362afb80 0x55ff362694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:00:08.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3ebe00640 1 --2- 172.21.15.32:0/494709516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362629a0 0x55ff36268a20 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc014700 tx=0x7fe3dc001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5370ebc16749869f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:08.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3d9600640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:00:08.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3d9600640 1 -- 172.21.15.32:0/494709516 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc025020 con 0x55ff362629a0 2024-04-17T10:00:08.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3d9600640 1 -- 172.21.15.32:0/494709516 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7fe3dc013780 con 0x55ff362629a0 2024-04-17T10:00:08.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3f6190000 1 -- 172.21.15.32:0/494709516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3e40b3040 con 0x55ff362629a0 2024-04-17T10:00:08.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3d9600640 1 -- 172.21.15.32:0/494709516 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fe3dc02f050 con 0x55ff362629a0 2024-04-17T10:00:08.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3f6190000 10 client.8055 fetch_fsmap learned FSMap version 66 2024-04-17T10:00:08.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3f6190000 1 -- 172.21.15.32:0/494709516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55ff3607feb0 con 0x55ff362629a0 2024-04-17T10:00:08.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3d9600640 1 -- 172.21.15.32:0/494709516 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3e40b3040 con 0x55ff362629a0 2024-04-17T10:00:08.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3d9600640 1 -- 172.21.15.32:0/494709516 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7fe3dc032020 con 0x55ff362629a0 2024-04-17T10:00:08.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3f6190000 10 client.8055 fetch_fsmap finished waiting for FSMap version 66 2024-04-17T10:00:08.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3f6190000 10 client.8055 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T10:00:08.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3f6190000 20 client.8055 populate_metadata read hostname 'smithi032' 2024-04-17T10:00:08.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3f6190000 1 --2- 172.21.15.32:0/494709516 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55ff36269eb0 0x55ff362c1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3f6190000 4 client.8055 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:00:08.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3f6190000 1 -- 172.21.15.32:0/494709516 --> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff3604ddd0 con 0x55ff36269eb0 2024-04-17T10:00:08.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.310+0000 7fe3e3400640 1 --2- 172.21.15.32:0/494709516 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55ff36269eb0 0x55ff362c1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.314+0000 7fe3e3400640 1 --2- 172.21.15.32:0/494709516 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55ff36269eb0 0x55ff362c1520 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c486b423bd26763 server_cookie=77260c4a694b6f7f in_seq=0 out_seq=0 2024-04-17T10:00:08.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.314+0000 7fe3d9600640 10 client.8055 ms_handle_connect on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:08.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.314+0000 7fe3d9600640 1 -- 172.21.15.32:0/494709516 <== mds.0 v2:172.21.15.32:6835/1253497636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ff3604ddd0 con 0x55ff36269eb0 2024-04-17T10:00:08.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.314+0000 7fe3d9600640 10 client.8055 handle_command_reply: tid=0 2024-04-17T10:00:08.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.314+0000 7fe3f6190000 1 client.8055 shutdown 2024-04-17T10:00:08.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.314+0000 7fe3f6190000 1 -- 172.21.15.32:0/494709516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362629a0 msgr2=0x55ff36268a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:08.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.314+0000 7fe3f6190000 1 --2- 172.21.15.32:0/494709516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362629a0 0x55ff36268a20 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc014700 tx=0x7fe3dc001130 comp rx=0 tx=0).stop 2024-04-17T10:00:08.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.314+0000 7fe3f6190000 1 -- 172.21.15.32:0/494709516 shutdown_connections 2024-04-17T10:00:08.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.314+0000 7fe3f6190000 1 --2- 172.21.15.32:0/494709516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff362afb80 0x55ff362694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.314+0000 7fe3f6190000 1 --2- 172.21.15.32:0/494709516 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55ff36269eb0 0x55ff362c1520 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.314+0000 7fe3f6190000 1 --2- 172.21.15.32:0/494709516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff362af260 0x55ff36268f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.314+0000 7fe3f6190000 1 --2- 172.21.15.32:0/494709516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362629a0 0x55ff36268a20 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.314+0000 7fe3d9600640 0 client.8055 ms_handle_reset on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:08.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.314+0000 7fe3f6190000 1 -- 172.21.15.32:0/494709516 >> 172.21.15.32:0/494709516 conn(0x55ff36290000 msgr2=0x55ff36261b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:08.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.314+0000 7fe3f6190000 1 -- 172.21.15.32:0/494709516 shutdown_connections 2024-04-17T10:00:08.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.314+0000 7fe3f6190000 1 -- 172.21.15.32:0/494709516 wait complete. 2024-04-17T10:00:08.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.314+0000 7fe3f6190000 20 client.8055 trim_cache size 0 max 16384 2024-04-17T10:00:08.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.330+0000 7fe3f6190000 1 Processor -- start 2024-04-17T10:00:08.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.330+0000 7fe3f6190000 1 -- start start 2024-04-17T10:00:08.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.330+0000 7fe3f6190000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff362629a0 0x55ff36261bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.330+0000 7fe3f6190000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362af260 0x55ff36262110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.330+0000 7fe3f6190000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff362afb80 0x55ff362c1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.330+0000 7fe3f6190000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec202bf0 con 0x55ff362af260 2024-04-17T10:00:08.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.330+0000 7fe3f6190000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec202050 con 0x55ff362afb80 2024-04-17T10:00:08.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.330+0000 7fe3f6190000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe3ec202580 con 0x55ff362629a0 2024-04-17T10:00:08.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.330+0000 7fe3e3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362af260 0x55ff36262110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.330+0000 7fe3e3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362af260 0x55ff36262110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60058/0 (socket says 172.21.15.32:60058) 2024-04-17T10:00:08.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.330+0000 7fe3e3400640 1 -- 172.21.15.32:0/1447566595 learned_addr learned my addr 172.21.15.32:0/1447566595 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:08.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3ebe00640 1 --2- 172.21.15.32:0/1447566595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff362629a0 0x55ff36261bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3f0c00640 1 --2- 172.21.15.32:0/1447566595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff362afb80 0x55ff362c1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3e3400640 1 -- 172.21.15.32:0/1447566595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff362629a0 msgr2=0x55ff36261bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:08.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3e3400640 1 --2- 172.21.15.32:0/1447566595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff362629a0 0x55ff36261bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3e3400640 1 -- 172.21.15.32:0/1447566595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff362afb80 msgr2=0x55ff362c1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:08.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3e3400640 1 --2- 172.21.15.32:0/1447566595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff362afb80 0x55ff362c1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3e3400640 1 -- 172.21.15.32:0/1447566595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff3606ea20 con 0x55ff362af260 2024-04-17T10:00:08.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3e3400640 1 --2- 172.21.15.32:0/1447566595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362af260 0x55ff36262110 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fe3d405afd0 tx=0x7fe3d405c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ae358a61c1bdbc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:08.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3dbe00640 1 -- 172.21.15.32:0/1447566595 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4053e40 con 0x55ff362af260 2024-04-17T10:00:08.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3dbe00640 1 -- 172.21.15.32:0/1447566595 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3d40521b0 con 0x55ff362af260 2024-04-17T10:00:08.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3dbe00640 1 -- 172.21.15.32:0/1447566595 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d405e400 con 0x55ff362af260 2024-04-17T10:00:08.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3f6190000 1 -- 172.21.15.32:0/1447566595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362af260 msgr2=0x55ff36262110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:08.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3f6190000 1 --2- 172.21.15.32:0/1447566595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362af260 0x55ff36262110 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fe3d405afd0 tx=0x7fe3d405c040 comp rx=0 tx=0).stop 2024-04-17T10:00:08.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3f6190000 1 -- 172.21.15.32:0/1447566595 shutdown_connections 2024-04-17T10:00:08.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3f6190000 1 --2- 172.21.15.32:0/1447566595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff362afb80 0x55ff362c1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3f6190000 1 --2- 172.21.15.32:0/1447566595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362af260 0x55ff36262110 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3f6190000 1 --2- 172.21.15.32:0/1447566595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff362629a0 0x55ff36261bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3f6190000 1 -- 172.21.15.32:0/1447566595 >> 172.21.15.32:0/1447566595 conn(0x55ff36290000 msgr2=0x55ff362ad310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:08.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3f6190000 1 -- 172.21.15.32:0/1447566595 shutdown_connections 2024-04-17T10:00:08.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3f6190000 1 -- 172.21.15.32:0/1447566595 wait complete. 2024-04-17T10:00:08.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3f6190000 1 -- start start 2024-04-17T10:00:08.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3f6190000 1 Processor -- start 2024-04-17T10:00:08.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3f6190000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362629a0 0x55ff362c3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3f6190000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff362af260 0x55ff362db0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3f6190000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff362afb80 0x55ff362db5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3f6190000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ff36058050 con 0x55ff362629a0 2024-04-17T10:00:08.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3f6190000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ff36076570 con 0x55ff362af260 2024-04-17T10:00:08.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3f6190000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ff360b3800 con 0x55ff362afb80 2024-04-17T10:00:08.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3f6190000 10 client.0 _ll_register_callbacks cb 0x7fe3dc00fea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:00:08.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3ebe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362629a0 0x55ff362c3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3ebe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362629a0 0x55ff362c3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60062/0 (socket says 172.21.15.32:60062) 2024-04-17T10:00:08.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3ebe00640 1 -- 172.21.15.32:0/1938171716 learned_addr learned my addr 172.21.15.32:0/1938171716 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:08.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3f0c00640 1 --2- 172.21.15.32:0/1938171716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff362afb80 0x55ff362db5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.334+0000 7fe3e3400640 1 --2- 172.21.15.32:0/1938171716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff362af260 0x55ff362db0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3ebe00640 1 -- 172.21.15.32:0/1938171716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff362afb80 msgr2=0x55ff362db5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:08.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3ebe00640 1 --2- 172.21.15.32:0/1938171716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff362afb80 0x55ff362db5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3ebe00640 1 -- 172.21.15.32:0/1938171716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff362af260 msgr2=0x55ff362db0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:08.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3ebe00640 1 --2- 172.21.15.32:0/1938171716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff362af260 0x55ff362db0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3ebe00640 1 -- 172.21.15.32:0/1938171716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff36135c40 con 0x55ff362629a0 2024-04-17T10:00:08.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3ebe00640 1 --2- 172.21.15.32:0/1938171716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362629a0 0x55ff362c3c30 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc02b710 tx=0x7fe3dc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b05d4352b84909 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:08.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3d9600640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:00:08.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3d9600640 1 -- 172.21.15.32:0/1938171716 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc025040 con 0x55ff362629a0 2024-04-17T10:00:08.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3d9600640 1 -- 172.21.15.32:0/1938171716 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7fe3dc02bb00 con 0x55ff362629a0 2024-04-17T10:00:08.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f6190000 1 -- 172.21.15.32:0/1938171716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3d40521b0 con 0x55ff362629a0 2024-04-17T10:00:08.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3d9600640 1 -- 172.21.15.32:0/1938171716 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fe3dc043050 con 0x55ff362629a0 2024-04-17T10:00:08.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f6190000 10 client.8061 fetch_fsmap learned FSMap version 66 2024-04-17T10:00:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f6190000 1 -- 172.21.15.32:0/1938171716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55ff36059600 con 0x55ff362629a0 2024-04-17T10:00:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3d9600640 1 -- 172.21.15.32:0/1938171716 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3d40521b0 con 0x55ff362629a0 2024-04-17T10:00:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3d9600640 1 -- 172.21.15.32:0/1938171716 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7fe3dc046020 con 0x55ff362629a0 2024-04-17T10:00:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f6190000 10 client.8061 fetch_fsmap finished waiting for FSMap version 66 2024-04-17T10:00:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f6190000 10 client.8061 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T10:00:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f6190000 20 client.8061 populate_metadata read hostname 'smithi032' 2024-04-17T10:00:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f6190000 1 --2- 172.21.15.32:0/1938171716 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55ff3631e6f0 0x55ff3631eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f6190000 4 client.8061 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:00:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f6190000 1 -- 172.21.15.32:0/1938171716 --> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ff360c1f10 con 0x55ff3631e6f0 2024-04-17T10:00:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3e3400640 1 --2- 172.21.15.32:0/1938171716 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55ff3631e6f0 0x55ff3631eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3e3400640 1 --2- 172.21.15.32:0/1938171716 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55ff3631e6f0 0x55ff3631eb00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6541c051ea9603f server_cookie=d496abc38d761b90 in_seq=0 out_seq=0 2024-04-17T10:00:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3d9600640 10 client.8061 ms_handle_connect on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3d9600640 1 -- 172.21.15.32:0/1938171716 <== mds.0 v2:172.21.15.32:6835/1253497636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x55ff360c1f10 con 0x55ff3631e6f0 2024-04-17T10:00:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3d9600640 10 client.8061 handle_command_reply: tid=0 2024-04-17T10:00:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f6190000 1 client.8061 shutdown 2024-04-17T10:00:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f6190000 1 -- 172.21.15.32:0/1938171716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362629a0 msgr2=0x55ff362c3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f6190000 1 --2- 172.21.15.32:0/1938171716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362629a0 0x55ff362c3c30 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc02b710 tx=0x7fe3dc014000 comp rx=0 tx=0).stop 2024-04-17T10:00:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f6190000 1 -- 172.21.15.32:0/1938171716 shutdown_connections 2024-04-17T10:00:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f6190000 1 --2- 172.21.15.32:0/1938171716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ff362afb80 0x55ff362db5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f6190000 1 --2- 172.21.15.32:0/1938171716 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55ff3631e6f0 0x55ff3631eb00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f6190000 1 --2- 172.21.15.32:0/1938171716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff362af260 0x55ff362db0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f6190000 1 --2- 172.21.15.32:0/1938171716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ff362629a0 0x55ff362c3c30 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f6190000 1 -- 172.21.15.32:0/1938171716 >> 172.21.15.32:0/1938171716 conn(0x55ff36290000 msgr2=0x55ff362adba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3d9600640 0 client.8061 ms_handle_reset on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f6190000 1 -- 172.21.15.32:0/1938171716 shutdown_connections 2024-04-17T10:00:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f6190000 1 -- 172.21.15.32:0/1938171716 wait complete. 2024-04-17T10:00:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f6190000 20 client.8061 trim_cache size 0 max 16384 2024-04-17T10:00:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:00:08.349 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T10:00:08.349 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-17T10:00:08.349 INFO:teuthology.orchestra.run.smithi032.stdout: "6e2e9c2e-8869-407a-9121-5e5536672271": { 2024-04-17T10:00:08.349 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/dir", 2024-04-17T10:00:08.349 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "6e2e9c2e-8869-407a-9121-5e5536672271", 2024-04-17T10:00:08.349 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,repair" 2024-04-17T10:00:08.349 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:00:08.349 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:00:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:00:08.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f2c00640 1 -- 172.21.15.32:0/1341936933 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe3c8068380 msgr2=0x7fe3c806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:08.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f2c00640 1 --2- 172.21.15.32:0/1341936933 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe3c8068380 0x7fe3c806a840 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc007fd0 tx=0x7fe3dc009040 comp rx=0 tx=0).stop 2024-04-17T10:00:08.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f2c00640 1 -- 172.21.15.32:0/1341936933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3ec0cf370 msgr2=0x7fe3ec141b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:08.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f2c00640 1 --2- 172.21.15.32:0/1341936933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3ec0cf370 0x7fe3ec141b60 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400c4a0 tx=0x7fe3d400c960 comp rx=0 tx=0).stop 2024-04-17T10:00:08.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f2c00640 1 -- 172.21.15.32:0/1341936933 shutdown_connections 2024-04-17T10:00:08.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f2c00640 1 --2- 172.21.15.32:0/1341936933 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe3c8068380 0x7fe3c806a840 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f2c00640 1 --2- 172.21.15.32:0/1341936933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3ec1656d0 0x7fe3ec1420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f2c00640 1 --2- 172.21.15.32:0/1341936933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3ec0cf370 0x7fe3ec141b60 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f2c00640 1 --2- 172.21.15.32:0/1341936933 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3ec06ba30 0x7fe3ec167430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:08.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f2c00640 1 -- 172.21.15.32:0/1341936933 >> 172.21.15.32:0/1341936933 conn(0x7fe3ec0b2120 msgr2=0x7fe3ec0b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:08.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f2c00640 1 -- 172.21.15.32:0/1341936933 shutdown_connections 2024-04-17T10:00:08.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:08.338+0000 7fe3f2c00640 1 -- 172.21.15.32:0/1341936933 wait complete. 2024-04-17T10:00:08.356 INFO:tasks.cephfs.filesystem:scrub status for tag:6e2e9c2e-8869-407a-9121-5e5536672271 - {'path': '/dir', 'tag': '6e2e9c2e-8869-407a-9121-5e5536672271', 'options': 'recursive,repair'} 2024-04-17T10:00:13.968 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:00:13.972 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:00:38.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:00:38.358 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-17T10:00:38.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.517+0000 7f34b5a00640 1 Processor -- start 2024-04-17T10:00:38.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.517+0000 7f34b5a00640 1 -- start start 2024-04-17T10:00:38.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.517+0000 7f34b5a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f34b015b7d0 0x7f34b006e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.517+0000 7f34b5a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34b006c140 0x7f34b00ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.517+0000 7f34b5a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34b00cc940 con 0x7f34b006f7c0 2024-04-17T10:00:38.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.517+0000 7f34b5a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f34b00ccc20 con 0x7f34b006c140 2024-04-17T10:00:38.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.517+0000 7f34b5a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f34b006e0b0 con 0x7f34b015b7d0 2024-04-17T10:00:38.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.517+0000 7f34ae000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f34b006f7c0 0x7f34b006ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48706/0 (socket says 172.21.15.32:48706) 2024-04-17T10:00:38.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.517+0000 7f34ae000640 1 -- 172.21.15.32:0/820269905 learned_addr learned my addr 172.21.15.32:0/820269905 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:00:38.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.517+0000 7f34af400640 1 --2- 172.21.15.32:0/820269905 >> v2:172.21.15.32:3300/0 conn(0x7f34b006c140 0x7f34b00ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:38.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.517+0000 7f34aea00640 1 --2- 172.21.15.32:0/820269905 >> v2:172.21.15.144:3300/0 conn(0x7f34b015b7d0 0x7f34b006e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:38.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.517+0000 7f34ad600640 1 -- 172.21.15.32:0/820269905 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 967995415 0 0) 0x7f34b00cc940 con 0x7f34b006f7c0 2024-04-17T10:00:38.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.517+0000 7f34af400640 1 -- 172.21.15.32:0/820269905 >> v2:172.21.15.144:3300/0 conn(0x7f34b015b7d0 msgr2=0x7f34b006e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:38.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.517+0000 7f34af400640 1 --2- 172.21.15.32:0/820269905 >> v2:172.21.15.144:3300/0 conn(0x7f34b015b7d0 0x7f34b006e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.517+0000 7f34af400640 1 -- 172.21.15.32:0/820269905 >> v1:172.21.15.32:6789/0 conn(0x7f34b006f7c0 legacy=0x7f34b006ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:38.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.517+0000 7f34af400640 1 -- 172.21.15.32:0/820269905 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b006e390 con 0x7f34b006c140 2024-04-17T10:00:38.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.521+0000 7f34af400640 1 --2- 172.21.15.32:0/820269905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b006c140 0x7f34b00ce290 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f34b00d1530 tx=0x7f34a801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bf2781427ca32 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:38.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.521+0000 7f34ad600640 1 -- 172.21.15.32:0/820269905 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a80b2070 con 0x7f34b006c140 2024-04-17T10:00:38.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.521+0000 7f34ad600640 1 -- 172.21.15.32:0/820269905 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a80a5980 con 0x7f34b006c140 2024-04-17T10:00:38.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.521+0000 7f34ad600640 1 -- 172.21.15.32:0/820269905 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a80a5d50 con 0x7f34b006c140 2024-04-17T10:00:38.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.521+0000 7f34b5a00640 1 -- 172.21.15.32:0/820269905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b006c140 msgr2=0x7f34b00ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:38.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.521+0000 7f34b5a00640 1 --2- 172.21.15.32:0/820269905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b006c140 0x7f34b00ce290 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f34b00d1530 tx=0x7f34a801c8a0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.521+0000 7f34b5a00640 1 -- 172.21.15.32:0/820269905 shutdown_connections 2024-04-17T10:00:38.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.521+0000 7f34b5a00640 1 --2- 172.21.15.32:0/820269905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b006c140 0x7f34b00ce290 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.521+0000 7f34b5a00640 1 --2- 172.21.15.32:0/820269905 >> v2:172.21.15.144:3300/0 conn(0x7f34b015b7d0 0x7f34b006e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.521+0000 7f34b5a00640 1 -- 172.21.15.32:0/820269905 >> 172.21.15.32:0/820269905 conn(0x7f34b00b2120 msgr2=0x7f34b00b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:38.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.521+0000 7f34b5a00640 1 -- 172.21.15.32:0/820269905 shutdown_connections 2024-04-17T10:00:38.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.521+0000 7f34b5a00640 1 -- 172.21.15.32:0/820269905 wait complete. 2024-04-17T10:00:38.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34b5a00640 1 Processor -- start 2024-04-17T10:00:38.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34b5a00640 1 -- start start 2024-04-17T10:00:38.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34b5a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34b006c140 0x7f34b015c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34b5a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b006f7c0 0x7f34b014f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34b5a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34b015b7d0 0x7f34b014fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34b5a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f34b0168630 con 0x7f34b006f7c0 2024-04-17T10:00:38.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34b5a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f34b01684b0 con 0x7f34b015b7d0 2024-04-17T10:00:38.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34b5a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f34b01687b0 con 0x7f34b006c140 2024-04-17T10:00:38.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34ae000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b006f7c0 0x7f34b014f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:38.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34ae000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b006f7c0 0x7f34b014f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40022/0 (socket says 172.21.15.32:40022) 2024-04-17T10:00:38.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34ae000640 1 -- 172.21.15.32:0/2788209180 learned_addr learned my addr 172.21.15.32:0/2788209180 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:38.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34aea00640 1 --2- 172.21.15.32:0/2788209180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34b006c140 0x7f34b015c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:38.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34af400640 1 --2- 172.21.15.32:0/2788209180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34b015b7d0 0x7f34b014fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:38.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34ae000640 1 -- 172.21.15.32:0/2788209180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34b006c140 msgr2=0x7f34b015c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:38.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34ae000640 1 --2- 172.21.15.32:0/2788209180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34b006c140 0x7f34b015c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34ae000640 1 -- 172.21.15.32:0/2788209180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34b015b7d0 msgr2=0x7f34b014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:38.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34ae000640 1 --2- 172.21.15.32:0/2788209180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34b015b7d0 0x7f34b014fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34ae000640 1 -- 172.21.15.32:0/2788209180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b014e720 con 0x7f34b006f7c0 2024-04-17T10:00:38.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34ae000640 1 --2- 172.21.15.32:0/2788209180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b006f7c0 0x7f34b014f7c0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f349c00c4a0 tx=0x7f349c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:38.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34a6a00640 1 -- 172.21.15.32:0/2788209180 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c018070 con 0x7f34b006f7c0 2024-04-17T10:00:38.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34a6a00640 1 -- 172.21.15.32:0/2788209180 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f349c0074e0 con 0x7f34b006f7c0 2024-04-17T10:00:38.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34a6a00640 1 -- 172.21.15.32:0/2788209180 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c004a30 con 0x7f34b006f7c0 2024-04-17T10:00:38.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34b5a00640 1 -- 172.21.15.32:0/2788209180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b014ea00 con 0x7f34b006f7c0 2024-04-17T10:00:38.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.525+0000 7f34b5a00640 1 -- 172.21.15.32:0/2788209180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b014efc0 con 0x7f34b006f7c0 2024-04-17T10:00:38.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.529+0000 7f34a6a00640 1 -- 172.21.15.32:0/2788209180 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f349c01b070 con 0x7f34b006f7c0 2024-04-17T10:00:38.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.529+0000 7f34a6a00640 1 --2- 172.21.15.32:0/2788209180 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3490068380 0x7f349006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.529+0000 7f34a6a00640 1 -- 172.21.15.32:0/2788209180 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7f349c085d60 con 0x7f34b006f7c0 2024-04-17T10:00:38.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.529+0000 7f34aea00640 1 --2- 172.21.15.32:0/2788209180 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3490068380 0x7f349006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:38.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.529+0000 7f34b8dd2000 1 Processor -- start 2024-04-17T10:00:38.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.529+0000 7f34b8dd2000 1 -- start start 2024-04-17T10:00:38.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.529+0000 7f34aea00640 1 --2- 172.21.15.32:0/2788209180 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3490068380 0x7f349006a840 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009ac0 tx=0x7f34a0009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:38.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.529+0000 7f34b8dd2000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e48a0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.529+0000 7f34b8dd2000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604e48a0b80 0x5604e4853460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.529+0000 7f34b8dd2000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604e48539a0 0x5604e4853d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.529+0000 7f34b8dd2000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5604e46a54f0 con 0x5604e48a0260 2024-04-17T10:00:38.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.529+0000 7f34b8dd2000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5604e465e6c0 con 0x5604e48539a0 2024-04-17T10:00:38.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.529+0000 7f34b8dd2000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5604e46a14b0 con 0x5604e48a0b80 2024-04-17T10:00:38.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.529+0000 7f34aea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e48a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:38.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.529+0000 7f34aea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e48a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40026/0 (socket says 172.21.15.32:40026) 2024-04-17T10:00:38.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.529+0000 7f34aea00640 1 -- 172.21.15.32:0/223890332 learned_addr learned my addr 172.21.15.32:0/223890332 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:38.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.529+0000 7f34ae000640 1 --2- 172.21.15.32:0/223890332 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604e48a0b80 0x5604e4853460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:38.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.529+0000 7f34af400640 1 --2- 172.21.15.32:0/223890332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604e48539a0 0x5604e4853d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:38.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34aea00640 1 -- 172.21.15.32:0/223890332 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604e48a0b80 msgr2=0x5604e4853460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:38.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34aea00640 1 --2- 172.21.15.32:0/223890332 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604e48a0b80 0x5604e4853460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34aea00640 1 -- 172.21.15.32:0/223890332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604e48539a0 msgr2=0x5604e4853d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:38.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34aea00640 1 --2- 172.21.15.32:0/223890332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604e48539a0 0x5604e4853d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34aea00640 1 -- 172.21.15.32:0/223890332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604e46ea8e0 con 0x5604e48a0260 2024-04-17T10:00:38.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34af400640 1 --2- 172.21.15.32:0/223890332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604e48539a0 0x5604e4853d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:38.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34aea00640 1 --2- 172.21.15.32:0/223890332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e48a0640 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f34a0006aa0 tx=0x7f34a0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abdef630c68915a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:38.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f349b400640 1 -- 172.21.15.32:0/223890332 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a001b070 con 0x5604e48a0260 2024-04-17T10:00:38.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f349b400640 1 -- 172.21.15.32:0/223890332 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a0027070 con 0x5604e48a0260 2024-04-17T10:00:38.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f349b400640 1 -- 172.21.15.32:0/223890332 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0003a30 con 0x5604e48a0260 2024-04-17T10:00:38.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34b8dd2000 1 -- 172.21.15.32:0/223890332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 msgr2=0x5604e48a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:38.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34b8dd2000 1 --2- 172.21.15.32:0/223890332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e48a0640 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f34a0006aa0 tx=0x7f34a0014000 comp rx=0 tx=0).stop 2024-04-17T10:00:38.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34b8dd2000 1 -- 172.21.15.32:0/223890332 shutdown_connections 2024-04-17T10:00:38.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34b8dd2000 1 --2- 172.21.15.32:0/223890332 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604e48a0b80 0x5604e4853460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34b8dd2000 1 --2- 172.21.15.32:0/223890332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604e48539a0 0x5604e4853d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34b8dd2000 1 --2- 172.21.15.32:0/223890332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e48a0640 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34b8dd2000 1 -- 172.21.15.32:0/223890332 >> 172.21.15.32:0/223890332 conn(0x5604e4881000 msgr2=0x5604e48833f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:38.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34b8dd2000 1 -- 172.21.15.32:0/223890332 shutdown_connections 2024-04-17T10:00:38.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34b8dd2000 1 -- 172.21.15.32:0/223890332 wait complete. 2024-04-17T10:00:38.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34b8dd2000 1 -- start start 2024-04-17T10:00:38.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34b8dd2000 1 Processor -- start 2024-04-17T10:00:38.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34b8dd2000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604e48539a0 0x5604e4858ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34b8dd2000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e4859530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34b8dd2000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604e48a0b80 0x5604e4859a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34b8dd2000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5604e47267b0 con 0x5604e48a0260 2024-04-17T10:00:38.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34b8dd2000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5604e4760d00 con 0x5604e48a0b80 2024-04-17T10:00:38.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34b8dd2000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5604e466a2c0 con 0x5604e48539a0 2024-04-17T10:00:38.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34b8dd2000 10 client.0 _ll_register_callbacks cb 0x5604e467a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:00:38.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34ae000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e4859530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:38.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34ae000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e4859530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40034/0 (socket says 172.21.15.32:40034) 2024-04-17T10:00:38.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34ae000640 1 -- 172.21.15.32:0/279529685 learned_addr learned my addr 172.21.15.32:0/279529685 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:38.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34ae000640 1 -- 172.21.15.32:0/279529685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604e48539a0 msgr2=0x5604e4858ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:38.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34ae000640 1 --2- 172.21.15.32:0/279529685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604e48539a0 0x5604e4858ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34ae000640 1 -- 172.21.15.32:0/279529685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604e48a0b80 msgr2=0x5604e4859a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:38.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34ae000640 1 --2- 172.21.15.32:0/279529685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604e48a0b80 0x5604e4859a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.533+0000 7f34ae000640 1 -- 172.21.15.32:0/279529685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604e467d490 con 0x5604e48a0260 2024-04-17T10:00:38.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34ae000640 1 --2- 172.21.15.32:0/279529685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e4859530 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f349c055b30 tx=0x7f349c055cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0027872c46220b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:38.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f348fe00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:00:38.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f348fe00640 1 -- 172.21.15.32:0/279529685 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c082060 con 0x5604e48a0260 2024-04-17T10:00:38.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f348fe00640 1 -- 172.21.15.32:0/279529685 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7f349c05e440 con 0x5604e48a0260 2024-04-17T10:00:38.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34b8dd2000 1 -- 172.21.15.32:0/279529685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34a0027070 con 0x5604e48a0260 2024-04-17T10:00:38.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f348fe00640 1 -- 172.21.15.32:0/279529685 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f349c06e070 con 0x5604e48a0260 2024-04-17T10:00:38.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34b8dd2000 10 client.8076 fetch_fsmap learned FSMap version 66 2024-04-17T10:00:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34b8dd2000 1 -- 172.21.15.32:0/279529685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x5604e4670eb0 con 0x5604e48a0260 2024-04-17T10:00:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f348fe00640 1 -- 172.21.15.32:0/279529685 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a0027070 con 0x5604e48a0260 2024-04-17T10:00:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f348fe00640 1 -- 172.21.15.32:0/279529685 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7f349c054ff0 con 0x5604e48a0260 2024-04-17T10:00:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34b8dd2000 10 client.8076 fetch_fsmap finished waiting for FSMap version 66 2024-04-17T10:00:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34b8dd2000 10 client.8076 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T10:00:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34b8dd2000 20 client.8076 populate_metadata read hostname 'smithi032' 2024-04-17T10:00:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34b8dd2000 1 --2- 172.21.15.32:0/279529685 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x5604e48af500 0x5604e48b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34b8dd2000 4 client.8076 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:00:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34b8dd2000 1 -- 172.21.15.32:0/279529685 --> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604e463edd0 con 0x5604e48af500 2024-04-17T10:00:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34aea00640 1 --2- 172.21.15.32:0/279529685 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x5604e48af500 0x5604e48b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34aea00640 1 --2- 172.21.15.32:0/279529685 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x5604e48af500 0x5604e48b18f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a0e3e45d1fc2e9d server_cookie=c17e24837b28ede in_seq=0 out_seq=0 2024-04-17T10:00:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f348fe00640 10 client.8076 ms_handle_connect on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:38.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f348fe00640 1 -- 172.21.15.32:0/279529685 <== mds.0 v2:172.21.15.32:6835/1253497636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5604e463edd0 con 0x5604e48af500 2024-04-17T10:00:38.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f348fe00640 10 client.8076 handle_command_reply: tid=0 2024-04-17T10:00:38.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34b8dd2000 1 client.8076 shutdown 2024-04-17T10:00:38.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34b8dd2000 1 -- 172.21.15.32:0/279529685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 msgr2=0x5604e4859530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:38.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34b8dd2000 1 --2- 172.21.15.32:0/279529685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e4859530 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f349c055b30 tx=0x7f349c055cd0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34b8dd2000 1 -- 172.21.15.32:0/279529685 shutdown_connections 2024-04-17T10:00:38.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34b8dd2000 1 --2- 172.21.15.32:0/279529685 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x5604e48af500 0x5604e48b18f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34b8dd2000 1 --2- 172.21.15.32:0/279529685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604e48a0b80 0x5604e4859a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34b8dd2000 1 --2- 172.21.15.32:0/279529685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e4859530 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34b8dd2000 1 --2- 172.21.15.32:0/279529685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604e48539a0 0x5604e4858ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34b8dd2000 1 -- 172.21.15.32:0/279529685 >> 172.21.15.32:0/279529685 conn(0x5604e4881000 msgr2=0x5604e48833f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:38.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f348fe00640 0 client.8076 ms_handle_reset on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:38.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34b8dd2000 1 -- 172.21.15.32:0/279529685 shutdown_connections 2024-04-17T10:00:38.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34b8dd2000 1 -- 172.21.15.32:0/279529685 wait complete. 2024-04-17T10:00:38.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.537+0000 7f34b8dd2000 20 client.8076 trim_cache size 0 max 16384 2024-04-17T10:00:38.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34b8dd2000 1 Processor -- start 2024-04-17T10:00:38.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34b8dd2000 1 -- start start 2024-04-17T10:00:38.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34b8dd2000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604e48539a0 0x5604e489f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34b8dd2000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e489f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34b8dd2000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604e48a0b80 0x5604e48b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34b8dd2000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f34b0203190 con 0x5604e48a0260 2024-04-17T10:00:38.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34b8dd2000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f34b01fafb0 con 0x5604e48539a0 2024-04-17T10:00:38.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34b8dd2000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f34b01fb600 con 0x5604e48a0b80 2024-04-17T10:00:38.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34ae000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e489f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:38.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34ae000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e489f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40046/0 (socket says 172.21.15.32:40046) 2024-04-17T10:00:38.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34ae000640 1 -- 172.21.15.32:0/351906458 learned_addr learned my addr 172.21.15.32:0/351906458 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:38.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34ae000640 1 -- 172.21.15.32:0/351906458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604e48a0b80 msgr2=0x5604e48b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:38.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34af400640 1 --2- 172.21.15.32:0/351906458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604e48a0b80 0x5604e48b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:38.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34aea00640 1 --2- 172.21.15.32:0/351906458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604e48539a0 0x5604e489f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:38.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34ae000640 1 --2- 172.21.15.32:0/351906458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604e48a0b80 0x5604e48b18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34ae000640 1 -- 172.21.15.32:0/351906458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604e48539a0 msgr2=0x5604e489f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:38.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34ae000640 1 --2- 172.21.15.32:0/351906458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604e48539a0 0x5604e489f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34ae000640 1 -- 172.21.15.32:0/351906458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604e465fa20 con 0x5604e48a0260 2024-04-17T10:00:38.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34af400640 1 --2- 172.21.15.32:0/351906458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604e48a0b80 0x5604e48b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:00:38.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34ae000640 1 --2- 172.21.15.32:0/351906458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e489f860 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f349c05e1a0 tx=0x7f349c059070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93535eea3fe63ca server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:38.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f349aa00640 1 -- 172.21.15.32:0/351906458 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c05e440 con 0x5604e48a0260 2024-04-17T10:00:38.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f349aa00640 1 -- 172.21.15.32:0/351906458 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f349c05c090 con 0x5604e48a0260 2024-04-17T10:00:38.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f349aa00640 1 -- 172.21.15.32:0/351906458 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c0518f0 con 0x5604e48a0260 2024-04-17T10:00:38.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34b8dd2000 1 -- 172.21.15.32:0/351906458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 msgr2=0x5604e489f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:38.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34b8dd2000 1 --2- 172.21.15.32:0/351906458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e489f860 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f349c05e1a0 tx=0x7f349c059070 comp rx=0 tx=0).stop 2024-04-17T10:00:38.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34b8dd2000 1 -- 172.21.15.32:0/351906458 shutdown_connections 2024-04-17T10:00:38.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34b8dd2000 1 --2- 172.21.15.32:0/351906458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604e48a0b80 0x5604e48b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34b8dd2000 1 --2- 172.21.15.32:0/351906458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e489f860 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34b8dd2000 1 --2- 172.21.15.32:0/351906458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604e48539a0 0x5604e489f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34b8dd2000 1 -- 172.21.15.32:0/351906458 >> 172.21.15.32:0/351906458 conn(0x5604e4881000 msgr2=0x5604e4852450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:38.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34b8dd2000 1 -- 172.21.15.32:0/351906458 shutdown_connections 2024-04-17T10:00:38.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34b8dd2000 1 -- 172.21.15.32:0/351906458 wait complete. 2024-04-17T10:00:38.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34b8dd2000 1 -- start start 2024-04-17T10:00:38.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.557+0000 7f34b8dd2000 1 Processor -- start 2024-04-17T10:00:38.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604e48539a0 0x5604e48b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e48d3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604e48a0b80 0x5604e48d41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5604e4649050 con 0x5604e48a0260 2024-04-17T10:00:38.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5604e4667570 con 0x5604e48539a0 2024-04-17T10:00:38.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5604e46a4800 con 0x5604e48a0b80 2024-04-17T10:00:38.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 10 client.0 _ll_register_callbacks cb 0x7f349c055590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:00:38.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34ae000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e48d3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:38.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34ae000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e48d3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40052/0 (socket says 172.21.15.32:40052) 2024-04-17T10:00:38.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34ae000640 1 -- 172.21.15.32:0/2294354969 learned_addr learned my addr 172.21.15.32:0/2294354969 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:38.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34aea00640 1 --2- 172.21.15.32:0/2294354969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604e48539a0 0x5604e48b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34af400640 1 --2- 172.21.15.32:0/2294354969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604e48a0b80 0x5604e48d41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34ae000640 1 -- 172.21.15.32:0/2294354969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604e48a0b80 msgr2=0x5604e48d41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34ae000640 1 --2- 172.21.15.32:0/2294354969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604e48a0b80 0x5604e48d41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34ae000640 1 -- 172.21.15.32:0/2294354969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604e48539a0 msgr2=0x5604e48b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34ae000640 1 --2- 172.21.15.32:0/2294354969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604e48539a0 0x5604e48b4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34ae000640 1 -- 172.21.15.32:0/2294354969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604e4726c40 con 0x5604e48a0260 2024-04-17T10:00:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34aea00640 1 --2- 172.21.15.32:0/2294354969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604e48539a0 0x5604e48b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34af400640 1 --2- 172.21.15.32:0/2294354969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604e48a0b80 0x5604e48d41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34ae000640 1 --2- 172.21.15.32:0/2294354969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e48d3c90 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f349c055b30 tx=0x7f349c084030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684c0f8411eebb62 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f348fe00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:00:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f348fe00640 1 -- 172.21.15.32:0/2294354969 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c07dac0 con 0x5604e48a0260 2024-04-17T10:00:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f348fe00640 1 -- 172.21.15.32:0/2294354969 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7f349c07dc60 con 0x5604e48a0260 2024-04-17T10:00:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 1 -- 172.21.15.32:0/2294354969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f349c05c090 con 0x5604e48a0260 2024-04-17T10:00:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f348fe00640 1 -- 172.21.15.32:0/2294354969 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f349c078070 con 0x5604e48a0260 2024-04-17T10:00:38.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 10 client.8082 fetch_fsmap learned FSMap version 66 2024-04-17T10:00:38.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 1 -- 172.21.15.32:0/2294354969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x5604e464a600 con 0x5604e48a0260 2024-04-17T10:00:38.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f348fe00640 1 -- 172.21.15.32:0/2294354969 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f349c05c090 con 0x5604e48a0260 2024-04-17T10:00:38.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f348fe00640 1 -- 172.21.15.32:0/2294354969 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7f349c054a60 con 0x5604e48a0260 2024-04-17T10:00:38.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 10 client.8082 fetch_fsmap finished waiting for FSMap version 66 2024-04-17T10:00:38.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 10 client.8082 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T10:00:38.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 20 client.8082 populate_metadata read hostname 'smithi032' 2024-04-17T10:00:38.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 1 --2- 172.21.15.32:0/2294354969 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x5604e4917b10 0x5604e4917f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 4 client.8082 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:00:38.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 1 -- 172.21.15.32:0/2294354969 --> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5604e46b2f10 con 0x5604e4917b10 2024-04-17T10:00:38.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34aea00640 1 --2- 172.21.15.32:0/2294354969 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x5604e4917b10 0x5604e4917f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:38.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34aea00640 1 --2- 172.21.15.32:0/2294354969 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x5604e4917b10 0x5604e4917f20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c229f303177e1f85 server_cookie=f349f01d2944dd4 in_seq=0 out_seq=0 2024-04-17T10:00:38.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f348fe00640 10 client.8082 ms_handle_connect on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f348fe00640 1 -- 172.21.15.32:0/2294354969 <== mds.0 v2:172.21.15.32:6835/1253497636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5604e46b2f10 con 0x5604e4917b10 2024-04-17T10:00:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f348fe00640 10 client.8082 handle_command_reply: tid=0 2024-04-17T10:00:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 1 client.8082 shutdown 2024-04-17T10:00:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 1 -- 172.21.15.32:0/2294354969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 msgr2=0x5604e48d3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 1 --2- 172.21.15.32:0/2294354969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e48d3c90 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f349c055b30 tx=0x7f349c084030 comp rx=0 tx=0).stop 2024-04-17T10:00:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 1 -- 172.21.15.32:0/2294354969 shutdown_connections 2024-04-17T10:00:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 1 --2- 172.21.15.32:0/2294354969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5604e48a0b80 0x5604e48d41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 1 --2- 172.21.15.32:0/2294354969 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x5604e4917b10 0x5604e4917f20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 1 --2- 172.21.15.32:0/2294354969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5604e48a0260 0x5604e48d3c90 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 1 --2- 172.21.15.32:0/2294354969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604e48539a0 0x5604e48b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f348fe00640 0 client.8082 ms_handle_reset on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.561+0000 7f34b8dd2000 1 -- 172.21.15.32:0/2294354969 >> 172.21.15.32:0/2294354969 conn(0x5604e4881000 msgr2=0x5604e4852b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.565+0000 7f34b8dd2000 1 -- 172.21.15.32:0/2294354969 shutdown_connections 2024-04-17T10:00:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.565+0000 7f34b8dd2000 1 -- 172.21.15.32:0/2294354969 wait complete. 2024-04-17T10:00:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.565+0000 7f34b8dd2000 20 client.8082 trim_cache size 0 max 16384 2024-04-17T10:00:38.566 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:00:38.566 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-17T10:00:38.566 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-17T10:00:38.566 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:00:38.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.565+0000 7f34b5a00640 1 -- 172.21.15.32:0/2788209180 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3490068380 msgr2=0x7f349006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:38.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.565+0000 7f34b5a00640 1 --2- 172.21.15.32:0/2788209180 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3490068380 0x7f349006a840 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009ac0 tx=0x7f34a0009190 comp rx=0 tx=0).stop 2024-04-17T10:00:38.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.565+0000 7f34b5a00640 1 -- 172.21.15.32:0/2788209180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b006f7c0 msgr2=0x7f34b014f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:38.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.565+0000 7f34b5a00640 1 --2- 172.21.15.32:0/2788209180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b006f7c0 0x7f34b014f7c0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f349c00c4a0 tx=0x7f349c00c960 comp rx=0 tx=0).stop 2024-04-17T10:00:38.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.565+0000 7f34b5a00640 1 -- 172.21.15.32:0/2788209180 shutdown_connections 2024-04-17T10:00:38.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.565+0000 7f34b5a00640 1 --2- 172.21.15.32:0/2788209180 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3490068380 0x7f349006a840 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.565+0000 7f34b5a00640 1 --2- 172.21.15.32:0/2788209180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34b015b7d0 0x7f34b014fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.565+0000 7f34b5a00640 1 --2- 172.21.15.32:0/2788209180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b006f7c0 0x7f34b014f7c0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.565+0000 7f34b5a00640 1 --2- 172.21.15.32:0/2788209180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34b006c140 0x7f34b015c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.565+0000 7f34b5a00640 1 -- 172.21.15.32:0/2788209180 >> 172.21.15.32:0/2788209180 conn(0x7f34b00b2120 msgr2=0x7f34b0159f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:38.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.565+0000 7f34b5a00640 1 -- 172.21.15.32:0/2788209180 shutdown_connections 2024-04-17T10:00:38.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.565+0000 7f34b5a00640 1 -- 172.21.15.32:0/2788209180 wait complete. 2024-04-17T10:00:38.580 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T10:00:38.581 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:00:38.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.721+0000 7fe1e1000640 1 Processor -- start 2024-04-17T10:00:38.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.721+0000 7fe1e1000640 1 -- start start 2024-04-17T10:00:38.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.725+0000 7fe1e1000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1dc06f670 0x7fe1dc15bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.725+0000 7fe1e1000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1dc072bb0 con 0x7fe1dc15baa0 2024-04-17T10:00:38.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.725+0000 7fe1e1000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1dc0b3dc0 con 0x7fe1dc144e90 2024-04-17T10:00:38.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.725+0000 7fe1e1000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe1dc145270 con 0x7fe1dc06f670 2024-04-17T10:00:38.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.725+0000 7fe1d9400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe1dc15baa0 0x7fe1dc06ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48716/0 (socket says 172.21.15.32:48716) 2024-04-17T10:00:38.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.725+0000 7fe1d9400640 1 -- 172.21.15.32:0/2197157316 learned_addr learned my addr 172.21.15.32:0/2197157316 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:00:38.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.725+0000 7fe1da800640 1 --2- 172.21.15.32:0/2197157316 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1dc06f670 0x7fe1dc15bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:38.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.725+0000 7fe1d8a00640 1 -- 172.21.15.32:0/2197157316 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1911911180 0 0) 0x7fe1dc072bb0 con 0x7fe1dc15baa0 2024-04-17T10:00:38.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.725+0000 7fe1d8a00640 1 -- 172.21.15.32:0/2197157316 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1c0005740 con 0x7fe1dc15baa0 2024-04-17T10:00:38.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.725+0000 7fe1d8a00640 1 -- 172.21.15.32:0/2197157316 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1068506200 0 0) 0x7fe1dc0b3dc0 con 0x7fe1dc144e90 2024-04-17T10:00:38.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.725+0000 7fe1d8a00640 1 -- 172.21.15.32:0/2197157316 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1dc072bb0 con 0x7fe1dc144e90 2024-04-17T10:00:38.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.725+0000 7fe1da800640 1 -- 172.21.15.32:0/2197157316 >> v1:172.21.15.144:6789/0 conn(0x7fe1dc144e90 legacy=0x7fe1dc0b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:38.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.725+0000 7fe1da800640 1 -- 172.21.15.32:0/2197157316 >> v1:172.21.15.32:6789/0 conn(0x7fe1dc15baa0 legacy=0x7fe1dc06ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:38.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.725+0000 7fe1d8a00640 1 -- 172.21.15.32:0/2197157316 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3596732554 0 0) 0x7fe1c0005740 con 0x7fe1dc15baa0 2024-04-17T10:00:38.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.725+0000 7fe1da800640 1 -- 172.21.15.32:0/2197157316 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1dc0b40a0 con 0x7fe1dc06f670 2024-04-17T10:00:38.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.729+0000 7fe1da800640 1 --2- 172.21.15.32:0/2197157316 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1dc06f670 0x7fe1dc15bf30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4009dc0 tx=0x7fe1d401c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=136d77efa2048f4f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:38.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.729+0000 7fe1d8a00640 1 -- 172.21.15.32:0/2197157316 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d40b2070 con 0x7fe1dc06f670 2024-04-17T10:00:38.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.729+0000 7fe1d8a00640 1 -- 172.21.15.32:0/2197157316 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1d40a5960 con 0x7fe1dc06f670 2024-04-17T10:00:38.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.729+0000 7fe1d8a00640 1 -- 172.21.15.32:0/2197157316 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d40a5d30 con 0x7fe1dc06f670 2024-04-17T10:00:38.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.729+0000 7fe1e1000640 1 -- 172.21.15.32:0/2197157316 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1dc06f670 msgr2=0x7fe1dc15bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:38.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.729+0000 7fe1e1000640 1 --2- 172.21.15.32:0/2197157316 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1dc06f670 0x7fe1dc15bf30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4009dc0 tx=0x7fe1d401c990 comp rx=0 tx=0).stop 2024-04-17T10:00:38.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.729+0000 7fe1e1000640 1 -- 172.21.15.32:0/2197157316 shutdown_connections 2024-04-17T10:00:38.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.729+0000 7fe1e1000640 1 --2- 172.21.15.32:0/2197157316 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1dc06f670 0x7fe1dc15bf30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.729+0000 7fe1e1000640 1 -- 172.21.15.32:0/2197157316 >> 172.21.15.32:0/2197157316 conn(0x7fe1dc06ba30 msgr2=0x7fe1dc06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:38.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.729+0000 7fe1e1000640 1 -- 172.21.15.32:0/2197157316 shutdown_connections 2024-04-17T10:00:38.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.729+0000 7fe1e1000640 1 -- 172.21.15.32:0/2197157316 wait complete. 2024-04-17T10:00:38.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.729+0000 7fe1e1000640 1 Processor -- start 2024-04-17T10:00:38.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.729+0000 7fe1e1000640 1 -- start start 2024-04-17T10:00:38.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1e1000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1dc06f670 0x7fe1dc0c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1e1000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1dc144e90 0x7fe1dc15ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1e1000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1dc15baa0 0x7fe1dc141a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1e1000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe1dc06de50 con 0x7fe1dc06f670 2024-04-17T10:00:38.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1e1000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe1dc0728c0 con 0x7fe1dc15baa0 2024-04-17T10:00:38.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1e1000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe1dc06dfd0 con 0x7fe1dc144e90 2024-04-17T10:00:38.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1d9e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1dc06f670 0x7fe1dc0c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1d9e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1dc06f670 0x7fe1dc0c6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40068/0 (socket says 172.21.15.32:40068) 2024-04-17T10:00:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1d9e00640 1 -- 172.21.15.32:0/2867480374 learned_addr learned my addr 172.21.15.32:0/2867480374 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1d9400640 1 --2- 172.21.15.32:0/2867480374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1dc144e90 0x7fe1dc15ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1da800640 1 --2- 172.21.15.32:0/2867480374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1dc15baa0 0x7fe1dc141a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1d9e00640 1 -- 172.21.15.32:0/2867480374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1dc144e90 msgr2=0x7fe1dc15ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1d9e00640 1 --2- 172.21.15.32:0/2867480374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1dc144e90 0x7fe1dc15ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1d9e00640 1 -- 172.21.15.32:0/2867480374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1dc15baa0 msgr2=0x7fe1dc141a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1d9e00640 1 --2- 172.21.15.32:0/2867480374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1dc15baa0 0x7fe1dc141a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1d9e00640 1 -- 172.21.15.32:0/2867480374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1dc15b340 con 0x7fe1dc06f670 2024-04-17T10:00:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1da800640 1 --2- 172.21.15.32:0/2867480374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1dc15baa0 0x7fe1dc141a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1d9e00640 1 --2- 172.21.15.32:0/2867480374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1dc06f670 0x7fe1dc0c6080 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00c490 tx=0x7fe1cc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1d2000640 1 -- 172.21.15.32:0/2867480374 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc0052d0 con 0x7fe1dc06f670 2024-04-17T10:00:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1d2000640 1 -- 172.21.15.32:0/2867480374 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1cc005470 con 0x7fe1dc06f670 2024-04-17T10:00:38.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1d2000640 1 -- 172.21.15.32:0/2867480374 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc012870 con 0x7fe1dc06f670 2024-04-17T10:00:38.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1e1000640 1 -- 172.21.15.32:0/2867480374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1dc1442e0 con 0x7fe1dc06f670 2024-04-17T10:00:38.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1e1000640 1 -- 172.21.15.32:0/2867480374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1dc0c5880 con 0x7fe1dc06f670 2024-04-17T10:00:38.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.733+0000 7fe1e1000640 1 -- 172.21.15.32:0/2867480374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1dc068a90 con 0x7fe1dc06f670 2024-04-17T10:00:38.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.737+0000 7fe1d2000640 1 -- 172.21.15.32:0/2867480374 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe1cc012a10 con 0x7fe1dc06f670 2024-04-17T10:00:38.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.737+0000 7fe1d2000640 1 --2- 172.21.15.32:0/2867480374 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe1bc068380 0x7fe1bc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:38.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.737+0000 7fe1d2000640 1 -- 172.21.15.32:0/2867480374 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7fe1cc086700 con 0x7fe1dc06f670 2024-04-17T10:00:38.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.737+0000 7fe1d9400640 1 --2- 172.21.15.32:0/2867480374 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe1bc068380 0x7fe1bc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:38.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.737+0000 7fe1d9400640 1 --2- 172.21.15.32:0/2867480374 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe1bc068380 0x7fe1bc06a840 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc06c030 tx=0x7fe1c8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:38.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.741+0000 7fe1d2000640 1 -- 172.21.15.32:0/2867480374 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fe1cc056c20 con 0x7fe1dc06f670 2024-04-17T10:00:38.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.937+0000 7fe1e1000640 1 -- 172.21.15.32:0/2867480374 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1dc142980 con 0x7fe1dc06f670 2024-04-17T10:00:38.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.937+0000 7fe1d2000640 1 -- 172.21.15.32:0/2867480374 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5205 (secure 0 0 0) 0x7fe1cc056520 con 0x7fe1dc06f670 2024-04-17T10:00:38.936 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:00:38.937 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6419,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3534108103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3534108103},{"type":"v1","addr":"172.21.15.144:6835","nonce":3534108103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7003,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/935945019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":935945019},{"type":"v1","addr":"172.21.15.144:6834","nonce":935945019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3044362393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3044362393},{"type":"v1","addr":"172.21.15.32:6836","nonce":3044362393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:59:04.834110+0000","modified":"2024-04-17T09:59:40.007746+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":6,"addr":"172.21.15.32:6837/1253497636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1253497636},{"type":"v1","addr":"172.21.15.32:6837","nonce":1253497636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7851,"qdb_cluster":[7851]},"id":6}]} 2024-04-17T10:00:38.937 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 66 2024-04-17T10:00:38.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.941+0000 7fe1e1000640 1 -- 172.21.15.32:0/2867480374 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe1bc068380 msgr2=0x7fe1bc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:38.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.941+0000 7fe1e1000640 1 --2- 172.21.15.32:0/2867480374 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe1bc068380 0x7fe1bc06a840 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc06c030 tx=0x7fe1c8009040 comp rx=0 tx=0).stop 2024-04-17T10:00:38.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.941+0000 7fe1e1000640 1 -- 172.21.15.32:0/2867480374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1dc06f670 msgr2=0x7fe1dc0c6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:38.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.941+0000 7fe1e1000640 1 --2- 172.21.15.32:0/2867480374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1dc06f670 0x7fe1dc0c6080 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00c490 tx=0x7fe1cc00c950 comp rx=0 tx=0).stop 2024-04-17T10:00:38.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.941+0000 7fe1e1000640 1 -- 172.21.15.32:0/2867480374 shutdown_connections 2024-04-17T10:00:38.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.941+0000 7fe1e1000640 1 --2- 172.21.15.32:0/2867480374 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe1bc068380 0x7fe1bc06a840 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.941+0000 7fe1e1000640 1 --2- 172.21.15.32:0/2867480374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1dc144e90 0x7fe1dc15ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.941+0000 7fe1e1000640 1 --2- 172.21.15.32:0/2867480374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1dc15baa0 0x7fe1dc141a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.941+0000 7fe1e1000640 1 --2- 172.21.15.32:0/2867480374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1dc06f670 0x7fe1dc0c6080 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:38.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.941+0000 7fe1e1000640 1 -- 172.21.15.32:0/2867480374 >> 172.21.15.32:0/2867480374 conn(0x7fe1dc06ba30 msgr2=0x7fe1dc142d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:38.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.941+0000 7fe1e1000640 1 -- 172.21.15.32:0/2867480374 shutdown_connections 2024-04-17T10:00:38.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:38.941+0000 7fe1e1000640 1 -- 172.21.15.32:0/2867480374 wait complete. 2024-04-17T10:00:38.958 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-17T10:00:39.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.101+0000 7f0aa6800640 1 Processor -- start 2024-04-17T10:00:39.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.101+0000 7f0aa6800640 1 -- start start 2024-04-17T10:00:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.105+0000 7f0aa6800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0aa015b7d0 0x7f0aa006e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.105+0000 7f0aa6800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0aa006f7c0 0x7f0aa006ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.105+0000 7f0aa6800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0aa00cc940 con 0x7f0aa006f270 2024-04-17T10:00:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.105+0000 7f0aa6800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0aa00ccc20 con 0x7f0aa015b7d0 2024-04-17T10:00:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.105+0000 7f0aa6800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0aa006e0b0 con 0x7f0aa006f7c0 2024-04-17T10:00:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.105+0000 7f0a9f400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0aa015b7d0 0x7f0aa006e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.105+0000 7f0a9fe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0aa006f270 0x7f0aa00ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48726/0 (socket says 172.21.15.32:48726) 2024-04-17T10:00:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.105+0000 7f0a9fe00640 1 -- 172.21.15.32:0/1829880677 learned_addr learned my addr 172.21.15.32:0/1829880677 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:00:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.105+0000 7f0a97e00640 1 --2- 172.21.15.32:0/1829880677 >> v2:172.21.15.144:3300/0 conn(0x7f0aa006f7c0 0x7f0aa006ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.105+0000 7f0a9f400640 1 -- 172.21.15.32:0/1829880677 >> v2:172.21.15.144:3300/0 conn(0x7f0aa006f7c0 msgr2=0x7f0aa006ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:39.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.105+0000 7f0a9f400640 1 --2- 172.21.15.32:0/1829880677 >> v2:172.21.15.144:3300/0 conn(0x7f0aa006f7c0 0x7f0aa006ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.105+0000 7f0a9f400640 1 -- 172.21.15.32:0/1829880677 >> v1:172.21.15.32:6789/0 conn(0x7f0aa006f270 legacy=0x7f0aa00ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:39.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.105+0000 7f0a9f400640 1 -- 172.21.15.32:0/1829880677 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa006e390 con 0x7f0aa015b7d0 2024-04-17T10:00:39.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.105+0000 7f0a97e00640 1 --2- 172.21.15.32:0/1829880677 >> v2:172.21.15.144:3300/0 conn(0x7f0aa006f7c0 0x7f0aa006ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:39.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.105+0000 7f0a9ea00640 1 -- 172.21.15.32:0/1829880677 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4024015917 0 0) 0x7f0aa00cc940 con 0x7f0aa006f270 2024-04-17T10:00:39.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.109+0000 7f0a9f400640 1 --2- 172.21.15.32:0/1829880677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0aa015b7d0 0x7f0aa006e8b0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00b58f0 tx=0x7f0a9001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735bb41845ac5728 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:39.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.109+0000 7f0a9ea00640 1 -- 172.21.15.32:0/1829880677 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a900b0070 con 0x7f0aa015b7d0 2024-04-17T10:00:39.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.109+0000 7f0a9ea00640 1 -- 172.21.15.32:0/1829880677 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a900ab600 con 0x7f0aa015b7d0 2024-04-17T10:00:39.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.109+0000 7f0a9ea00640 1 -- 172.21.15.32:0/1829880677 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a900ab9d0 con 0x7f0aa015b7d0 2024-04-17T10:00:39.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.109+0000 7f0aa6800640 1 -- 172.21.15.32:0/1829880677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0aa015b7d0 msgr2=0x7f0aa006e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:39.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.109+0000 7f0aa6800640 1 --2- 172.21.15.32:0/1829880677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0aa015b7d0 0x7f0aa006e8b0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00b58f0 tx=0x7f0a9001c3f0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.109+0000 7f0aa6800640 1 -- 172.21.15.32:0/1829880677 shutdown_connections 2024-04-17T10:00:39.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.109+0000 7f0aa6800640 1 --2- 172.21.15.32:0/1829880677 >> v2:172.21.15.144:3300/0 conn(0x7f0aa006f7c0 0x7f0aa006ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.109+0000 7f0aa6800640 1 --2- 172.21.15.32:0/1829880677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0aa015b7d0 0x7f0aa006e8b0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.109+0000 7f0aa6800640 1 -- 172.21.15.32:0/1829880677 >> 172.21.15.32:0/1829880677 conn(0x7f0aa00b2120 msgr2=0x7f0aa00b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:39.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.109+0000 7f0aa6800640 1 -- 172.21.15.32:0/1829880677 shutdown_connections 2024-04-17T10:00:39.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.109+0000 7f0aa6800640 1 -- 172.21.15.32:0/1829880677 wait complete. 2024-04-17T10:00:39.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.109+0000 7f0aa6800640 1 Processor -- start 2024-04-17T10:00:39.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.109+0000 7f0aa6800640 1 -- start start 2024-04-17T10:00:39.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.113+0000 7f0aa6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0aa006f270 0x7f0aa0151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.113+0000 7f0aa6800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa006f7c0 0x7f0aa0149b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.113+0000 7f0aa6800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0aa015b7d0 0x7f0aa00af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.113+0000 7f0aa6800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa0168ab0 con 0x7f0aa006f270 2024-04-17T10:00:39.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.113+0000 7f0aa6800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa006c380 con 0x7f0aa006f7c0 2024-04-17T10:00:39.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.113+0000 7f0aa6800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0aa0168c30 con 0x7f0aa015b7d0 2024-04-17T10:00:39.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.113+0000 7f0a9f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0aa006f270 0x7f0aa0151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.113+0000 7f0a9f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0aa006f270 0x7f0aa0151b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40096/0 (socket says 172.21.15.32:40096) 2024-04-17T10:00:39.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.113+0000 7f0a9f400640 1 -- 172.21.15.32:0/4272243906 learned_addr learned my addr 172.21.15.32:0/4272243906 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:39.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.113+0000 7f0a9f400640 1 -- 172.21.15.32:0/4272243906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0aa015b7d0 msgr2=0x7f0aa00af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:39.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.113+0000 7f0a9f400640 1 --2- 172.21.15.32:0/4272243906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0aa015b7d0 0x7f0aa00af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.113+0000 7f0a9f400640 1 -- 172.21.15.32:0/4272243906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa006f7c0 msgr2=0x7f0aa0149b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:39.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.113+0000 7f0a9f400640 1 --2- 172.21.15.32:0/4272243906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa006f7c0 0x7f0aa0149b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.113+0000 7f0a9f400640 1 -- 172.21.15.32:0/4272243906 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa014a060 con 0x7f0aa006f270 2024-04-17T10:00:39.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.113+0000 7f0a9f400640 1 --2- 172.21.15.32:0/4272243906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0aa006f270 0x7f0aa0151b50 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f0a9001a020 tx=0x7f0a9001ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:39.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.113+0000 7f0a97400640 1 -- 172.21.15.32:0/4272243906 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a900b0050 con 0x7f0aa006f270 2024-04-17T10:00:39.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.113+0000 7f0a97400640 1 -- 172.21.15.32:0/4272243906 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a900bc070 con 0x7f0aa006f270 2024-04-17T10:00:39.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.113+0000 7f0a97400640 1 -- 172.21.15.32:0/4272243906 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a900b5830 con 0x7f0aa006f270 2024-04-17T10:00:39.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.113+0000 7f0aa6800640 1 -- 172.21.15.32:0/4272243906 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa00afa30 con 0x7f0aa006f270 2024-04-17T10:00:39.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.113+0000 7f0aa6800640 1 -- 172.21.15.32:0/4272243906 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa014fcd0 con 0x7f0aa006f270 2024-04-17T10:00:39.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.113+0000 7f0a97400640 1 -- 172.21.15.32:0/4272243906 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0a900b59d0 con 0x7f0aa006f270 2024-04-17T10:00:39.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0a97400640 1 --2- 172.21.15.32:0/4272243906 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0a7c068380 0x7f0a7c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0a97400640 1 -- 172.21.15.32:0/4272243906 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7f0a90121230 con 0x7f0aa006f270 2024-04-17T10:00:39.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0a97e00640 1 --2- 172.21.15.32:0/4272243906 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0a7c068380 0x7f0a7c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0aa9bdf000 1 Processor -- start 2024-04-17T10:00:39.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0aa9bdf000 1 -- start start 2024-04-17T10:00:39.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0a97e00640 1 --2- 172.21.15.32:0/4272243906 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0a7c068380 0x7f0a7c06a840 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0a88006f10 tx=0x7f0a88008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:39.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0aa9bdf000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f79ffa50 0x55e7f79ffe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0aa9bdf000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7f7a00370 0x55e7f79b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0aa9bdf000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e7f79b2f70 0x55e7f79b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0aa9bdf000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e7f7804590 con 0x55e7f79ffa50 2024-04-17T10:00:39.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0aa9bdf000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e7f77bd580 con 0x55e7f7a00370 2024-04-17T10:00:39.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0aa9bdf000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e7f7800550 con 0x55e7f79b2f70 2024-04-17T10:00:39.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0a9f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f79ffa50 0x55e7f79ffe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0a9f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f79ffa50 0x55e7f79ffe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40110/0 (socket says 172.21.15.32:40110) 2024-04-17T10:00:39.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0a9f400640 1 -- 172.21.15.32:0/3406784813 learned_addr learned my addr 172.21.15.32:0/3406784813 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:39.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0a9fe00640 1 --2- 172.21.15.32:0/3406784813 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e7f79b2f70 0x55e7f79b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0a97e00640 1 --2- 172.21.15.32:0/3406784813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7f7a00370 0x55e7f79b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0a9f400640 1 -- 172.21.15.32:0/3406784813 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e7f79b2f70 msgr2=0x55e7f79b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:39.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0a9f400640 1 --2- 172.21.15.32:0/3406784813 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e7f79b2f70 0x55e7f79b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0a9f400640 1 -- 172.21.15.32:0/3406784813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7f7a00370 msgr2=0x55e7f79b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:39.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0a9f400640 1 --2- 172.21.15.32:0/3406784813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7f7a00370 0x55e7f79b2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0a9f400640 1 -- 172.21.15.32:0/3406784813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7f78abb90 con 0x55e7f79ffa50 2024-04-17T10:00:39.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0a9fe00640 1 --2- 172.21.15.32:0/3406784813 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e7f79b2f70 0x55e7f79b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:00:39.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0a97e00640 1 --2- 172.21.15.32:0/3406784813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7f7a00370 0x55e7f79b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:00:39.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0a9f400640 1 --2- 172.21.15.32:0/3406784813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f79ffa50 0x55e7f79ffe30 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f0a900f9670 tx=0x7f0a900f0c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0edfbd02cdd6df server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:39.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0a8fe00640 1 -- 172.21.15.32:0/3406784813 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a900f0fc0 con 0x55e7f79ffa50 2024-04-17T10:00:39.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0a8fe00640 1 -- 172.21.15.32:0/3406784813 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a900b1040 con 0x55e7f79ffa50 2024-04-17T10:00:39.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0a8fe00640 1 -- 172.21.15.32:0/3406784813 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a900f5370 con 0x55e7f79ffa50 2024-04-17T10:00:39.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/3406784813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f79ffa50 msgr2=0x55e7f79ffe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:39.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0aa9bdf000 1 --2- 172.21.15.32:0/3406784813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f79ffa50 0x55e7f79ffe30 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f0a900f9670 tx=0x7f0a900f0c30 comp rx=0 tx=0).stop 2024-04-17T10:00:39.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/3406784813 shutdown_connections 2024-04-17T10:00:39.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0aa9bdf000 1 --2- 172.21.15.32:0/3406784813 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e7f79b2f70 0x55e7f79b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0aa9bdf000 1 --2- 172.21.15.32:0/3406784813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7f7a00370 0x55e7f79b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0aa9bdf000 1 --2- 172.21.15.32:0/3406784813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f79ffa50 0x55e7f79ffe30 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/3406784813 >> 172.21.15.32:0/3406784813 conn(0x55e7f79e07f0 msgr2=0x55e7f79e2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:39.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/3406784813 shutdown_connections 2024-04-17T10:00:39.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.117+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/3406784813 wait complete. 2024-04-17T10:00:39.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0aa9bdf000 1 -- start start 2024-04-17T10:00:39.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0aa9bdf000 1 Processor -- start 2024-04-17T10:00:39.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0aa9bdf000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e7f79b2f70 0x55e7f79b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0aa9bdf000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f79ffa50 0x55e7f79b7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0aa9bdf000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7f7a00370 0x55e7f79b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0aa9bdf000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e7f780f4c0 con 0x55e7f79ffa50 2024-04-17T10:00:39.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0aa9bdf000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e7f77a4c80 con 0x55e7f7a00370 2024-04-17T10:00:39.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0aa9bdf000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e7f77cf930 con 0x55e7f79b2f70 2024-04-17T10:00:39.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0aa9bdf000 10 client.0 _ll_register_callbacks cb 0x55e7f79a06d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:00:39.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0a97e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f79ffa50 0x55e7f79b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0a97e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f79ffa50 0x55e7f79b7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40118/0 (socket says 172.21.15.32:40118) 2024-04-17T10:00:39.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0a97e00640 1 -- 172.21.15.32:0/2070931183 learned_addr learned my addr 172.21.15.32:0/2070931183 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:39.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0a9f400640 1 --2- 172.21.15.32:0/2070931183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e7f79b2f70 0x55e7f79b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0a9fe00640 1 --2- 172.21.15.32:0/2070931183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7f7a00370 0x55e7f79b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0a97e00640 1 -- 172.21.15.32:0/2070931183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e7f79b2f70 msgr2=0x55e7f79b6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:39.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0a97e00640 1 --2- 172.21.15.32:0/2070931183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e7f79b2f70 0x55e7f79b6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0a97e00640 1 -- 172.21.15.32:0/2070931183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7f7a00370 msgr2=0x55e7f79b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:39.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0a97e00640 1 --2- 172.21.15.32:0/2070931183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7f7a00370 0x55e7f79b75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0a97e00640 1 -- 172.21.15.32:0/2070931183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7f77ddb20 con 0x55e7f79ffa50 2024-04-17T10:00:39.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0a9fe00640 1 --2- 172.21.15.32:0/2070931183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7f7a00370 0x55e7f79b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:39.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0a97e00640 1 --2- 172.21.15.32:0/2070931183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f79ffa50 0x55e7f79b7080 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f0a8800efb0 tx=0x7f0a88006e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae640682184127a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:39.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0a8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:00:39.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0a8cc00640 1 -- 172.21.15.32:0/2070931183 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8800f040 con 0x55e7f79ffa50 2024-04-17T10:00:39.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0a8cc00640 1 -- 172.21.15.32:0/2070931183 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7f0a88020410 con 0x55e7f79ffa50 2024-04-17T10:00:39.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/2070931183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a900b1040 con 0x55e7f79ffa50 2024-04-17T10:00:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0a8cc00640 1 -- 172.21.15.32:0/2070931183 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f0a8802d050 con 0x55e7f79ffa50 2024-04-17T10:00:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0aa9bdf000 10 client.8103 fetch_fsmap learned FSMap version 66 2024-04-17T10:00:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/2070931183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55e7f7848610 con 0x55e7f79ffa50 2024-04-17T10:00:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0a8cc00640 1 -- 172.21.15.32:0/2070931183 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a900b1040 con 0x55e7f79ffa50 2024-04-17T10:00:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0a8cc00640 1 -- 172.21.15.32:0/2070931183 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7f0a88030020 con 0x55e7f79ffa50 2024-04-17T10:00:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0aa9bdf000 10 client.8103 fetch_fsmap finished waiting for FSMap version 66 2024-04-17T10:00:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0aa9bdf000 10 client.8103 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T10:00:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0aa9bdf000 20 client.8103 populate_metadata read hostname 'smithi032' 2024-04-17T10:00:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0aa9bdf000 1 --2- 172.21.15.32:0/2070931183 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55e7f7a0dce0 0x55e7f7a100d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0aa9bdf000 4 client.8103 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:00:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/2070931183 --> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7f779de40 con 0x55e7f7a0dce0 2024-04-17T10:00:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.121+0000 7f0a9f400640 1 --2- 172.21.15.32:0/2070931183 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55e7f7a0dce0 0x55e7f7a100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.125+0000 7f0a9f400640 1 --2- 172.21.15.32:0/2070931183 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55e7f7a0dce0 0x55e7f7a100d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b2bfd639d622ae7 server_cookie=64b7402162cb1c17 in_seq=0 out_seq=0 2024-04-17T10:00:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.125+0000 7f0a8cc00640 10 client.8103 ms_handle_connect on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.125+0000 7f0a8cc00640 1 -- 172.21.15.32:0/2070931183 <== mds.0 v2:172.21.15.32:6835/1253497636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f0a9010fe50 con 0x55e7f7a0dce0 2024-04-17T10:00:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.125+0000 7f0a8cc00640 10 client.8103 handle_command_reply: tid=0 2024-04-17T10:00:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.125+0000 7f0aa9bdf000 1 client.8103 shutdown 2024-04-17T10:00:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.125+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/2070931183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f79ffa50 msgr2=0x55e7f79b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.125+0000 7f0aa9bdf000 1 --2- 172.21.15.32:0/2070931183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f79ffa50 0x55e7f79b7080 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f0a8800efb0 tx=0x7f0a88006e00 comp rx=0 tx=0).stop 2024-04-17T10:00:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.125+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/2070931183 shutdown_connections 2024-04-17T10:00:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.125+0000 7f0aa9bdf000 1 --2- 172.21.15.32:0/2070931183 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55e7f7a0dce0 0x55e7f7a100d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.125+0000 7f0aa9bdf000 1 --2- 172.21.15.32:0/2070931183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7f7a00370 0x55e7f79b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.125+0000 7f0aa9bdf000 1 --2- 172.21.15.32:0/2070931183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f79ffa50 0x55e7f79b7080 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.125+0000 7f0aa9bdf000 1 --2- 172.21.15.32:0/2070931183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e7f79b2f70 0x55e7f79b6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.125+0000 7f0a8cc00640 0 client.8103 ms_handle_reset on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.125+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/2070931183 >> 172.21.15.32:0/2070931183 conn(0x55e7f79e07f0 msgr2=0x55e7f79e2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:39.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.125+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/2070931183 shutdown_connections 2024-04-17T10:00:39.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.125+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/2070931183 wait complete. 2024-04-17T10:00:39.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.125+0000 7f0aa9bdf000 20 client.8103 trim_cache size 0 max 16384 2024-04-17T10:00:39.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.137+0000 7f0aa9bdf000 1 Processor -- start 2024-04-17T10:00:39.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.137+0000 7f0aa9bdf000 1 -- start start 2024-04-17T10:00:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.137+0000 7f0aa9bdf000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7f79b2f70 0x55e7f79fe840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.137+0000 7f0aa9bdf000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f79ffa50 0x55e7f79fdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.137+0000 7f0aa9bdf000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e7f7a00370 0x55e7f79fe040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.137+0000 7f0aa9bdf000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa0204430 con 0x55e7f79ffa50 2024-04-17T10:00:39.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.137+0000 7f0aa9bdf000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa0203400 con 0x55e7f79b2f70 2024-04-17T10:00:39.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.137+0000 7f0aa9bdf000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0aa0158f40 con 0x55e7f7a00370 2024-04-17T10:00:39.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0a97e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f79ffa50 0x55e7f79fdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0a97e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f79ffa50 0x55e7f79fdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40128/0 (socket says 172.21.15.32:40128) 2024-04-17T10:00:39.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0a97e00640 1 -- 172.21.15.32:0/2664650078 learned_addr learned my addr 172.21.15.32:0/2664650078 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:39.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0a9f400640 1 --2- 172.21.15.32:0/2664650078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7f79b2f70 0x55e7f79fe840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0a9fe00640 1 --2- 172.21.15.32:0/2664650078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e7f7a00370 0x55e7f79fe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0a97e00640 1 -- 172.21.15.32:0/2664650078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e7f7a00370 msgr2=0x55e7f79fe040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:39.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0a97e00640 1 --2- 172.21.15.32:0/2664650078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e7f7a00370 0x55e7f79fe040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0a97e00640 1 -- 172.21.15.32:0/2664650078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7f79b2f70 msgr2=0x55e7f79fe840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:39.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0a97e00640 1 --2- 172.21.15.32:0/2664650078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7f79b2f70 0x55e7f79fe840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0a97e00640 1 -- 172.21.15.32:0/2664650078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7f77da190 con 0x55e7f79ffa50 2024-04-17T10:00:39.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0a97e00640 1 --2- 172.21.15.32:0/2664650078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f79ffa50 0x55e7f79fdb00 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f0a88008010 tx=0x7f0a8801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21066cdfd5927c47 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0a8f400640 1 -- 172.21.15.32:0/2664650078 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a88037070 con 0x55e7f79ffa50 2024-04-17T10:00:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0a8f400640 1 -- 172.21.15.32:0/2664650078 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a88025070 con 0x55e7f79ffa50 2024-04-17T10:00:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0a8f400640 1 -- 172.21.15.32:0/2664650078 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a88020990 con 0x55e7f79ffa50 2024-04-17T10:00:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/2664650078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f79ffa50 msgr2=0x55e7f79fdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:39.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0aa9bdf000 1 --2- 172.21.15.32:0/2664650078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f79ffa50 0x55e7f79fdb00 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f0a88008010 tx=0x7f0a8801e000 comp rx=0 tx=0).stop 2024-04-17T10:00:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/2664650078 shutdown_connections 2024-04-17T10:00:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0aa9bdf000 1 --2- 172.21.15.32:0/2664650078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e7f7a00370 0x55e7f79fe040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0aa9bdf000 1 --2- 172.21.15.32:0/2664650078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f79ffa50 0x55e7f79fdb00 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0aa9bdf000 1 --2- 172.21.15.32:0/2664650078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7f79b2f70 0x55e7f79fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/2664650078 >> 172.21.15.32:0/2664650078 conn(0x55e7f79e07f0 msgr2=0x55e7f79b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/2664650078 shutdown_connections 2024-04-17T10:00:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/2664650078 wait complete. 2024-04-17T10:00:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0aa9bdf000 1 -- start start 2024-04-17T10:00:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0aa9bdf000 1 Processor -- start 2024-04-17T10:00:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0aa9bdf000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e7f79b2f70 0x55e7f79decf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0aa9bdf000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7f79ffa50 0x55e7f79df230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0aa9bdf000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f7a00370 0x55e7f7a100d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0aa9bdf000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e7f78117a0 con 0x55e7f7a00370 2024-04-17T10:00:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0aa9bdf000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa014ab80 con 0x55e7f79ffa50 2024-04-17T10:00:39.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0aa9bdf000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e7f78038a0 con 0x55e7f79b2f70 2024-04-17T10:00:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0aa9bdf000 10 client.0 _ll_register_callbacks cb 0x7f0a880094e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:00:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0a9fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f7a00370 0x55e7f7a100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0a9fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f7a00370 0x55e7f7a100d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40140/0 (socket says 172.21.15.32:40140) 2024-04-17T10:00:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0a9fe00640 1 -- 172.21.15.32:0/3129730758 learned_addr learned my addr 172.21.15.32:0/3129730758 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0a9f400640 1 --2- 172.21.15.32:0/3129730758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e7f79b2f70 0x55e7f79decf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.141+0000 7f0a97e00640 1 --2- 172.21.15.32:0/3129730758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7f79ffa50 0x55e7f79df230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0a9fe00640 1 -- 172.21.15.32:0/3129730758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e7f79b2f70 msgr2=0x55e7f79decf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0a9fe00640 1 --2- 172.21.15.32:0/3129730758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e7f79b2f70 0x55e7f79decf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0a9fe00640 1 -- 172.21.15.32:0/3129730758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7f79ffa50 msgr2=0x55e7f79df230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0a9fe00640 1 --2- 172.21.15.32:0/3129730758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7f79ffa50 0x55e7f79df230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0a9fe00640 1 -- 172.21.15.32:0/3129730758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7f77bd7d0 con 0x55e7f7a00370 2024-04-17T10:00:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0a97e00640 1 --2- 172.21.15.32:0/3129730758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7f79ffa50 0x55e7f79df230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0a9fe00640 1 --2- 172.21.15.32:0/3129730758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f7a00370 0x55e7f7a100d0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f0a98001590 tx=0x7f0a98001990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b769e47a34dd60 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:39.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0a8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:00:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0a8cc00640 1 -- 172.21.15.32:0/3129730758 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a98013540 con 0x55e7f7a00370 2024-04-17T10:00:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0a8cc00640 1 -- 172.21.15.32:0/3129730758 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7f0a98010040 con 0x55e7f7a00370 2024-04-17T10:00:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/3129730758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a88025070 con 0x55e7f7a00370 2024-04-17T10:00:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0a8cc00640 1 -- 172.21.15.32:0/3129730758 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f0a980136e0 con 0x55e7f7a00370 2024-04-17T10:00:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa9bdf000 10 client.8109 fetch_fsmap learned FSMap version 66 2024-04-17T10:00:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/3129730758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55e7f77a9440 con 0x55e7f7a00370 2024-04-17T10:00:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0a8cc00640 1 -- 172.21.15.32:0/3129730758 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a88025070 con 0x55e7f7a00370 2024-04-17T10:00:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0a8cc00640 1 -- 172.21.15.32:0/3129730758 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7f0a98013880 con 0x55e7f7a00370 2024-04-17T10:00:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa9bdf000 10 client.8109 fetch_fsmap finished waiting for FSMap version 66 2024-04-17T10:00:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa9bdf000 10 client.8109 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T10:00:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa9bdf000 20 client.8109 populate_metadata read hostname 'smithi032' 2024-04-17T10:00:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa9bdf000 1 --2- 172.21.15.32:0/3129730758 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55e7f7a10610 0x55e7f7a109f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa9bdf000 4 client.8109 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-17T10:00:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/3129730758 --> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55e7f7811fb0 con 0x55e7f7a10610 2024-04-17T10:00:39.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0a9f400640 1 --2- 172.21.15.32:0/3129730758 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55e7f7a10610 0x55e7f7a109f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0a9f400640 1 --2- 172.21.15.32:0/3129730758 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55e7f7a10610 0x55e7f7a109f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f978c381e8f7db9c server_cookie=ce5beb1eafe5f67a in_seq=0 out_seq=0 2024-04-17T10:00:39.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0a8cc00640 10 client.8109 ms_handle_connect on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:39.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0a8cc00640 1 -- 172.21.15.32:0/3129730758 <== mds.0 v2:172.21.15.32:6835/1253497636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+143 (crc 0 0 0) 0x55e7f7811fb0 con 0x55e7f7a10610 2024-04-17T10:00:39.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0a8cc00640 10 client.8109 handle_command_reply: tid=0 2024-04-17T10:00:39.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa9bdf000 1 client.8109 shutdown 2024-04-17T10:00:39.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/3129730758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f7a00370 msgr2=0x55e7f7a100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:39.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa9bdf000 1 --2- 172.21.15.32:0/3129730758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f7a00370 0x55e7f7a100d0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f0a98001590 tx=0x7f0a98001990 comp rx=0 tx=0).stop 2024-04-17T10:00:39.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/3129730758 shutdown_connections 2024-04-17T10:00:39.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa9bdf000 1 --2- 172.21.15.32:0/3129730758 >> [v2:172.21.15.32:6835/1253497636,v1:172.21.15.32:6837/1253497636] conn(0x55e7f7a10610 0x55e7f7a109f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa9bdf000 1 --2- 172.21.15.32:0/3129730758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7f7a00370 0x55e7f7a100d0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa9bdf000 1 --2- 172.21.15.32:0/3129730758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7f79ffa50 0x55e7f79df230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa9bdf000 1 --2- 172.21.15.32:0/3129730758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e7f79b2f70 0x55e7f79decf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/3129730758 >> 172.21.15.32:0/3129730758 conn(0x55e7f79e07f0 msgr2=0x55e7f79bacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:39.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0a8cc00640 0 client.8109 ms_handle_reset on v2:172.21.15.32:6835/1253497636 2024-04-17T10:00:39.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/3129730758 shutdown_connections 2024-04-17T10:00:39.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa9bdf000 1 -- 172.21.15.32:0/3129730758 wait complete. 2024-04-17T10:00:39.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa9bdf000 20 client.8109 trim_cache size 0 max 16384 2024-04-17T10:00:39.148 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:00:39.148 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-17T10:00:39.148 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:00:39.148 INFO:teuthology.orchestra.run.smithi032.stdout: "damage_type": "backtrace", 2024-04-17T10:00:39.149 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 593058767, 2024-04-17T10:00:39.149 INFO:teuthology.orchestra.run.smithi032.stdout: "ino": 1099511627777, 2024-04-17T10:00:39.149 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/testdir/hardlink" 2024-04-17T10:00:39.149 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:00:39.149 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-17T10:00:39.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa6800640 1 -- 172.21.15.32:0/4272243906 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0a7c068380 msgr2=0x7f0a7c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa6800640 1 --2- 172.21.15.32:0/4272243906 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0a7c068380 0x7f0a7c06a840 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0a88006f10 tx=0x7f0a88008040 comp rx=0 tx=0).stop 2024-04-17T10:00:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa6800640 1 -- 172.21.15.32:0/4272243906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0aa006f270 msgr2=0x7f0aa0151b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa6800640 1 --2- 172.21.15.32:0/4272243906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0aa006f270 0x7f0aa0151b50 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f0a9001a020 tx=0x7f0a9001ac30 comp rx=0 tx=0).stop 2024-04-17T10:00:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa6800640 1 -- 172.21.15.32:0/4272243906 shutdown_connections 2024-04-17T10:00:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa6800640 1 --2- 172.21.15.32:0/4272243906 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0a7c068380 0x7f0a7c06a840 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa6800640 1 --2- 172.21.15.32:0/4272243906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0aa015b7d0 0x7f0aa00af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa6800640 1 --2- 172.21.15.32:0/4272243906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa006f7c0 0x7f0aa0149b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa6800640 1 --2- 172.21.15.32:0/4272243906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0aa006f270 0x7f0aa0151b50 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa6800640 1 -- 172.21.15.32:0/4272243906 >> 172.21.15.32:0/4272243906 conn(0x7f0aa00b2120 msgr2=0x7f0aa00b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa6800640 1 -- 172.21.15.32:0/4272243906 shutdown_connections 2024-04-17T10:00:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.145+0000 7f0aa6800640 1 -- 172.21.15.32:0/4272243906 wait complete. 2024-04-17T10:00:39.164 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:00:39.164 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:00:39.173 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:00:39.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.361+0000 7f3a2bc00640 1 Processor -- start 2024-04-17T10:00:39.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.361+0000 7f3a2bc00640 1 -- start start 2024-04-17T10:00:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.365+0000 7f3a2bc00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3a24144e90 0x7f3a240b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.365+0000 7f3a2bc00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3a2415c1b0 0x7f3a2406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.365+0000 7f3a2bc00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a24072bb0 con 0x7f3a240b4de0 2024-04-17T10:00:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.365+0000 7f3a2bc00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3a24145270 con 0x7f3a2415c1b0 2024-04-17T10:00:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.365+0000 7f3a2bc00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f3a240b3dc0 con 0x7f3a24144e90 2024-04-17T10:00:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.365+0000 7f3a29c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3a2415c1b0 0x7f3a2406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.365+0000 7f3a29c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3a2415c1b0 0x7f3a2406ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40152/0 (socket says 172.21.15.32:40152) 2024-04-17T10:00:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.365+0000 7f3a29c00640 1 -- 172.21.15.32:0/3081278858 learned_addr learned my addr 172.21.15.32:0/3081278858 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:39.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.365+0000 7f3a29200640 1 --2- 172.21.15.32:0/3081278858 >> v2:172.21.15.144:3300/0 conn(0x7f3a24144e90 0x7f3a240b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.365+0000 7f3a29c00640 1 -- 172.21.15.32:0/3081278858 >> v2:172.21.15.144:3300/0 conn(0x7f3a24144e90 msgr2=0x7f3a240b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:39.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.365+0000 7f3a29c00640 1 --2- 172.21.15.32:0/3081278858 >> v2:172.21.15.144:3300/0 conn(0x7f3a24144e90 0x7f3a240b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.365+0000 7f3a29c00640 1 -- 172.21.15.32:0/3081278858 >> v1:172.21.15.144:6789/0 conn(0x7f3a240b4de0 legacy=0x7f3a2415baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:39.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.365+0000 7f3a29c00640 1 -- 172.21.15.32:0/3081278858 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a240b40a0 con 0x7f3a2415c1b0 2024-04-17T10:00:39.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.365+0000 7f3a29200640 1 --2- 172.21.15.32:0/3081278858 >> v2:172.21.15.144:3300/0 conn(0x7f3a24144e90 0x7f3a240b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:39.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a29c00640 1 --2- 172.21.15.32:0/3081278858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a2415c1b0 0x7f3a2406ef60 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f3a2406ff30 tx=0x7f3a1c01ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5c5f2055638fe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:39.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a23e00640 1 -- 172.21.15.32:0/3081278858 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c00b2d0 con 0x7f3a2415c1b0 2024-04-17T10:00:39.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a23e00640 1 -- 172.21.15.32:0/3081278858 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a1c00b470 con 0x7f3a2415c1b0 2024-04-17T10:00:39.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a23e00640 1 -- 172.21.15.32:0/3081278858 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c0a7bc0 con 0x7f3a2415c1b0 2024-04-17T10:00:39.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a2bc00640 1 -- 172.21.15.32:0/3081278858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a2415c1b0 msgr2=0x7f3a2406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:39.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a2bc00640 1 --2- 172.21.15.32:0/3081278858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a2415c1b0 0x7f3a2406ef60 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f3a2406ff30 tx=0x7f3a1c01ebc0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a2bc00640 1 -- 172.21.15.32:0/3081278858 shutdown_connections 2024-04-17T10:00:39.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a2bc00640 1 --2- 172.21.15.32:0/3081278858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a2415c1b0 0x7f3a2406ef60 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a2bc00640 1 --2- 172.21.15.32:0/3081278858 >> v2:172.21.15.144:3300/0 conn(0x7f3a24144e90 0x7f3a240b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a2bc00640 1 -- 172.21.15.32:0/3081278858 >> 172.21.15.32:0/3081278858 conn(0x7f3a2406ba30 msgr2=0x7f3a2406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a2bc00640 1 -- 172.21.15.32:0/3081278858 shutdown_connections 2024-04-17T10:00:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a2bc00640 1 -- 172.21.15.32:0/3081278858 wait complete. 2024-04-17T10:00:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a2bc00640 1 Processor -- start 2024-04-17T10:00:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a2bc00640 1 -- start start 2024-04-17T10:00:39.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a2bc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a240b4de0 0x7f3a24157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a2bc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a24144e90 0x7f3a240b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a2bc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a2415c1b0 0x7f3a241434a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a2bc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2406de50 con 0x7f3a240b4de0 2024-04-17T10:00:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a2bc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3a24072880 con 0x7f3a24144e90 2024-04-17T10:00:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a2bc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3a2406dfd0 con 0x7f3a2415c1b0 2024-04-17T10:00:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a29200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a240b4de0 0x7f3a24157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a29200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a240b4de0 0x7f3a24157ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40154/0 (socket says 172.21.15.32:40154) 2024-04-17T10:00:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a29200640 1 -- 172.21.15.32:0/2174851267 learned_addr learned my addr 172.21.15.32:0/2174851267 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a29c00640 1 --2- 172.21.15.32:0/2174851267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a2415c1b0 0x7f3a241434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.369+0000 7f3a1be00640 1 --2- 172.21.15.32:0/2174851267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a24144e90 0x7f3a240b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.373+0000 7f3a29200640 1 -- 172.21.15.32:0/2174851267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a2415c1b0 msgr2=0x7f3a241434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.373+0000 7f3a29200640 1 --2- 172.21.15.32:0/2174851267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a2415c1b0 0x7f3a241434a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.373+0000 7f3a29200640 1 -- 172.21.15.32:0/2174851267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a24144e90 msgr2=0x7f3a240b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.373+0000 7f3a29200640 1 --2- 172.21.15.32:0/2174851267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a24144e90 0x7f3a240b0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.373+0000 7f3a29200640 1 -- 172.21.15.32:0/2174851267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a24158430 con 0x7f3a240b4de0 2024-04-17T10:00:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.373+0000 7f3a1be00640 1 --2- 172.21.15.32:0/2174851267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a24144e90 0x7f3a240b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.373+0000 7f3a29200640 1 --2- 172.21.15.32:0/2174851267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a240b4de0 0x7f3a24157ef0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f3a1400e810 tx=0x7f3a1400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.373+0000 7f3a21600640 1 -- 172.21.15.32:0/2174851267 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a14017070 con 0x7f3a240b4de0 2024-04-17T10:00:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.373+0000 7f3a21600640 1 -- 172.21.15.32:0/2174851267 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a1400c870 con 0x7f3a240b4de0 2024-04-17T10:00:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.373+0000 7f3a21600640 1 -- 172.21.15.32:0/2174851267 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1400f040 con 0x7f3a240b4de0 2024-04-17T10:00:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.373+0000 7f3a2bc00640 1 -- 172.21.15.32:0/2174851267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a240b14d0 con 0x7f3a240b4de0 2024-04-17T10:00:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.373+0000 7f3a2bc00640 1 -- 172.21.15.32:0/2174851267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a240b1a90 con 0x7f3a240b4de0 2024-04-17T10:00:39.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.373+0000 7f3a2bc00640 1 -- 172.21.15.32:0/2174851267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a24068a90 con 0x7f3a240b4de0 2024-04-17T10:00:39.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.373+0000 7f3a21600640 1 -- 172.21.15.32:0/2174851267 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3a1401a070 con 0x7f3a240b4de0 2024-04-17T10:00:39.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.373+0000 7f3a21600640 1 --2- 172.21.15.32:0/2174851267 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3a00068380 0x7f3a0006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.373+0000 7f3a21600640 1 -- 172.21.15.32:0/2174851267 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7f3a140865c0 con 0x7f3a240b4de0 2024-04-17T10:00:39.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.373+0000 7f3a1be00640 1 --2- 172.21.15.32:0/2174851267 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3a00068380 0x7f3a0006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.377+0000 7f3a1be00640 1 --2- 172.21.15.32:0/2174851267 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3a00068380 0x7f3a0006a840 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3a240af450 tx=0x7f3a0c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:39.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.377+0000 7f3a21600640 1 -- 172.21.15.32:0/2174851267 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3a14056400 con 0x7f3a240b4de0 2024-04-17T10:00:39.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.545+0000 7f3a2bc00640 1 -- 172.21.15.32:0/2174851267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a24143260 con 0x7f3a240b4de0 2024-04-17T10:00:39.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.545+0000 7f3a21600640 1 -- 172.21.15.32:0/2174851267 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16899 (secure 0 0 0) 0x7f3a1405a700 con 0x7f3a240b4de0 2024-04-17T10:00:39.543 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:00:39.544 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":85,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:59:56.110989+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:59:03.223426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-17T09:59:03.780120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.11","mappings":[{"from":1,"to":3}]},{"pgid":"13.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6837/2656735817":"2024-04-18T09:59:28.985107+0000","172.21.15.32:6835/2656735817":"2024-04-18T09:59:28.985107+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:00:39.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.549+0000 7f3a2bc00640 1 -- 172.21.15.32:0/2174851267 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3a00068380 msgr2=0x7f3a0006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:39.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.549+0000 7f3a2bc00640 1 --2- 172.21.15.32:0/2174851267 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3a00068380 0x7f3a0006a840 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3a240af450 tx=0x7f3a0c009040 comp rx=0 tx=0).stop 2024-04-17T10:00:39.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.549+0000 7f3a2bc00640 1 -- 172.21.15.32:0/2174851267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a240b4de0 msgr2=0x7f3a24157ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:39.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.549+0000 7f3a2bc00640 1 --2- 172.21.15.32:0/2174851267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a240b4de0 0x7f3a24157ef0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f3a1400e810 tx=0x7f3a1400ecd0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.549+0000 7f3a2bc00640 1 -- 172.21.15.32:0/2174851267 shutdown_connections 2024-04-17T10:00:39.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.549+0000 7f3a2bc00640 1 --2- 172.21.15.32:0/2174851267 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3a00068380 0x7f3a0006a840 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.549+0000 7f3a2bc00640 1 --2- 172.21.15.32:0/2174851267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a2415c1b0 0x7f3a241434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.549+0000 7f3a2bc00640 1 --2- 172.21.15.32:0/2174851267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a24144e90 0x7f3a240b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.549+0000 7f3a2bc00640 1 --2- 172.21.15.32:0/2174851267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a240b4de0 0x7f3a24157ef0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.549+0000 7f3a2bc00640 1 -- 172.21.15.32:0/2174851267 >> 172.21.15.32:0/2174851267 conn(0x7f3a2406ba30 msgr2=0x7f3a2415ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:39.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.549+0000 7f3a2bc00640 1 -- 172.21.15.32:0/2174851267 shutdown_connections 2024-04-17T10:00:39.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.549+0000 7f3a2bc00640 1 -- 172.21.15.32:0/2174851267 wait complete. 2024-04-17T10:00:39.565 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:59:03.223426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:59:03.780120+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:00:39.565 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:00:39.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.709+0000 7fc03b200640 1 Processor -- start 2024-04-17T10:00:39.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.709+0000 7fc03b200640 1 -- start start 2024-04-17T10:00:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.713+0000 7fc03b200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc03415b7d0 0x7fc03406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.713+0000 7fc03b200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc03406c140 0x7fc0340ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.713+0000 7fc03b200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0340cc940 con 0x7fc03406f7c0 2024-04-17T10:00:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.713+0000 7fc03b200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc0340ccc20 con 0x7fc03415b7d0 2024-04-17T10:00:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.713+0000 7fc03b200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc03406e0b0 con 0x7fc03406c140 2024-04-17T10:00:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.713+0000 7fc033e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc03415b7d0 0x7fc03406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.713+0000 7fc033e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc03415b7d0 0x7fc03406e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40158/0 (socket says 172.21.15.32:40158) 2024-04-17T10:00:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.713+0000 7fc02b400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc03406f7c0 0x7fc03406ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48742/0 (socket says 172.21.15.32:48742) 2024-04-17T10:00:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.713+0000 7fc033e00640 1 -- 172.21.15.32:0/1783243502 learned_addr learned my addr 172.21.15.32:0/1783243502 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:39.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.713+0000 7fc039200640 1 --2- 172.21.15.32:0/1783243502 >> v2:172.21.15.144:3300/0 conn(0x7fc03406c140 0x7fc0340ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.713+0000 7fc033e00640 1 -- 172.21.15.32:0/1783243502 >> v2:172.21.15.144:3300/0 conn(0x7fc03406c140 msgr2=0x7fc0340ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:39.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.713+0000 7fc033e00640 1 --2- 172.21.15.32:0/1783243502 >> v2:172.21.15.144:3300/0 conn(0x7fc03406c140 0x7fc0340ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.713+0000 7fc033e00640 1 -- 172.21.15.32:0/1783243502 >> v1:172.21.15.32:6789/0 conn(0x7fc03406f7c0 legacy=0x7fc03406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:39.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.713+0000 7fc033e00640 1 -- 172.21.15.32:0/1783243502 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03406e390 con 0x7fc03415b7d0 2024-04-17T10:00:39.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.713+0000 7fc033400640 1 -- 172.21.15.32:0/1783243502 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1409540825 0 0) 0x7fc0340cc940 con 0x7fc03406f7c0 2024-04-17T10:00:39.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.717+0000 7fc033e00640 1 --2- 172.21.15.32:0/1783243502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03415b7d0 0x7fc03406e8b0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fc0340b20b0 tx=0x7fc02401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72139a2701ca01a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:39.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.717+0000 7fc033400640 1 -- 172.21.15.32:0/1783243502 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0240aa610 con 0x7fc03415b7d0 2024-04-17T10:00:39.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.717+0000 7fc033400640 1 -- 172.21.15.32:0/1783243502 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0240af070 con 0x7fc03415b7d0 2024-04-17T10:00:39.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.717+0000 7fc033400640 1 -- 172.21.15.32:0/1783243502 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0240aa9d0 con 0x7fc03415b7d0 2024-04-17T10:00:39.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.717+0000 7fc03b200640 1 -- 172.21.15.32:0/1783243502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03415b7d0 msgr2=0x7fc03406e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:39.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.717+0000 7fc03b200640 1 --2- 172.21.15.32:0/1783243502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03415b7d0 0x7fc03406e8b0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fc0340b20b0 tx=0x7fc02401c490 comp rx=0 tx=0).stop 2024-04-17T10:00:39.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.717+0000 7fc03b200640 1 -- 172.21.15.32:0/1783243502 shutdown_connections 2024-04-17T10:00:39.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.717+0000 7fc03b200640 1 --2- 172.21.15.32:0/1783243502 >> v2:172.21.15.144:3300/0 conn(0x7fc03406c140 0x7fc0340ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.717+0000 7fc03b200640 1 --2- 172.21.15.32:0/1783243502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03415b7d0 0x7fc03406e8b0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.717+0000 7fc03b200640 1 -- 172.21.15.32:0/1783243502 >> 172.21.15.32:0/1783243502 conn(0x7fc0340b2120 msgr2=0x7fc0340b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:39.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.717+0000 7fc03b200640 1 -- 172.21.15.32:0/1783243502 shutdown_connections 2024-04-17T10:00:39.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.717+0000 7fc03b200640 1 -- 172.21.15.32:0/1783243502 wait complete. 2024-04-17T10:00:39.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.717+0000 7fc03b200640 1 Processor -- start 2024-04-17T10:00:39.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.717+0000 7fc03b200640 1 -- start start 2024-04-17T10:00:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.717+0000 7fc03b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc03406c140 0x7fc03415a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.717+0000 7fc03b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03406f7c0 0x7fc0341506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.717+0000 7fc03b200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc03415b7d0 0x7fc03415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.717+0000 7fc03b200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc034168630 con 0x7fc03406f7c0 2024-04-17T10:00:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.717+0000 7fc03b200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341684b0 con 0x7fc03415b7d0 2024-04-17T10:00:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.717+0000 7fc03b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc0341687b0 con 0x7fc03406c140 2024-04-17T10:00:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.717+0000 7fc02b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03406f7c0 0x7fc0341506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.721+0000 7fc02b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03406f7c0 0x7fc0341506c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40170/0 (socket says 172.21.15.32:40170) 2024-04-17T10:00:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.721+0000 7fc02b400640 1 -- 172.21.15.32:0/2893256293 learned_addr learned my addr 172.21.15.32:0/2893256293 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.721+0000 7fc039200640 1 --2- 172.21.15.32:0/2893256293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc03415b7d0 0x7fc03415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.721+0000 7fc033e00640 1 --2- 172.21.15.32:0/2893256293 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc03406c140 0x7fc03415a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.721+0000 7fc02b400640 1 -- 172.21.15.32:0/2893256293 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc03406c140 msgr2=0x7fc03415a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.721+0000 7fc02b400640 1 --2- 172.21.15.32:0/2893256293 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc03406c140 0x7fc03415a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.721+0000 7fc02b400640 1 -- 172.21.15.32:0/2893256293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc03415b7d0 msgr2=0x7fc03415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.721+0000 7fc02b400640 1 --2- 172.21.15.32:0/2893256293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc03415b7d0 0x7fc03415bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.721+0000 7fc02b400640 1 -- 172.21.15.32:0/2893256293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc034150f10 con 0x7fc03406f7c0 2024-04-17T10:00:39.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.721+0000 7fc039200640 1 --2- 172.21.15.32:0/2893256293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc03415b7d0 0x7fc03415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:39.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.721+0000 7fc02b400640 1 --2- 172.21.15.32:0/2893256293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03406f7c0 0x7fc0341506c0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0011f0 tx=0x7fc01c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:39.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.721+0000 7fc030c00640 1 -- 172.21.15.32:0/2893256293 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c018070 con 0x7fc03406f7c0 2024-04-17T10:00:39.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.721+0000 7fc030c00640 1 -- 172.21.15.32:0/2893256293 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc01c0074e0 con 0x7fc03406f7c0 2024-04-17T10:00:39.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.721+0000 7fc030c00640 1 -- 172.21.15.32:0/2893256293 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c004a10 con 0x7fc03406f7c0 2024-04-17T10:00:39.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.721+0000 7fc03b200640 1 -- 172.21.15.32:0/2893256293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0341511f0 con 0x7fc03406f7c0 2024-04-17T10:00:39.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.721+0000 7fc03b200640 1 -- 172.21.15.32:0/2893256293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0341585a0 con 0x7fc03406f7c0 2024-04-17T10:00:39.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.721+0000 7fc03b200640 1 -- 172.21.15.32:0/2893256293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0341587a0 con 0x7fc03406f7c0 2024-04-17T10:00:39.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.721+0000 7fc030c00640 1 -- 172.21.15.32:0/2893256293 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc01c004bb0 con 0x7fc03406f7c0 2024-04-17T10:00:39.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.721+0000 7fc030c00640 1 --2- 172.21.15.32:0/2893256293 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc010068380 0x7fc01006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:39.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.725+0000 7fc030c00640 1 -- 172.21.15.32:0/2893256293 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7fc01c085e80 con 0x7fc03406f7c0 2024-04-17T10:00:39.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.725+0000 7fc033e00640 1 --2- 172.21.15.32:0/2893256293 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc010068380 0x7fc01006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:39.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.725+0000 7fc033e00640 1 --2- 172.21.15.32:0/2893256293 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc010068380 0x7fc01006a840 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc0240ae000 tx=0x7fc0240a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:39.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.725+0000 7fc030c00640 1 -- 172.21.15.32:0/2893256293 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc0341587a0 con 0x7fc03406f7c0 2024-04-17T10:00:39.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.893+0000 7fc03b200640 1 -- 172.21.15.32:0/2893256293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc034068a90 con 0x7fc03406f7c0 2024-04-17T10:00:39.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.893+0000 7fc030c00640 1 -- 172.21.15.32:0/2893256293 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7fc01c056310 con 0x7fc03406f7c0 2024-04-17T10:00:39.890 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T10:00:39.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.893+0000 7fc03b200640 1 -- 172.21.15.32:0/2893256293 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc010068380 msgr2=0x7fc01006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:39.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.893+0000 7fc03b200640 1 --2- 172.21.15.32:0/2893256293 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc010068380 0x7fc01006a840 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc0240ae000 tx=0x7fc0240a7000 comp rx=0 tx=0).stop 2024-04-17T10:00:39.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.893+0000 7fc03b200640 1 -- 172.21.15.32:0/2893256293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03406f7c0 msgr2=0x7fc0341506c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:39.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.893+0000 7fc03b200640 1 --2- 172.21.15.32:0/2893256293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03406f7c0 0x7fc0341506c0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0011f0 tx=0x7fc01c00c9b0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.893+0000 7fc03b200640 1 -- 172.21.15.32:0/2893256293 shutdown_connections 2024-04-17T10:00:39.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.893+0000 7fc03b200640 1 --2- 172.21.15.32:0/2893256293 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc010068380 0x7fc01006a840 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.893+0000 7fc03b200640 1 --2- 172.21.15.32:0/2893256293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc03415b7d0 0x7fc03415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.893+0000 7fc03b200640 1 --2- 172.21.15.32:0/2893256293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03406f7c0 0x7fc0341506c0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.893+0000 7fc03b200640 1 --2- 172.21.15.32:0/2893256293 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc03406c140 0x7fc03415a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:39.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.893+0000 7fc03b200640 1 -- 172.21.15.32:0/2893256293 >> 172.21.15.32:0/2893256293 conn(0x7fc0340b2120 msgr2=0x7fc03406ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:39.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.893+0000 7fc03b200640 1 -- 172.21.15.32:0/2893256293 shutdown_connections 2024-04-17T10:00:39.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:39.893+0000 7fc03b200640 1 -- 172.21.15.32:0/2893256293 wait complete. 2024-04-17T10:00:39.913 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:00:40.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.057+0000 7f8fd9a00640 1 Processor -- start 2024-04-17T10:00:40.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.057+0000 7f8fd9a00640 1 -- start start 2024-04-17T10:00:40.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.061+0000 7f8fd9a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8fd406ba30 0x7f8fd406be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:40.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.061+0000 7f8fd9a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8fd40cf370 0x7f8fd40cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:40.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.061+0000 7f8fd9a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fd406c350 con 0x7f8fd4160350 2024-04-17T10:00:40.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.061+0000 7f8fd9a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8fd40d0130 con 0x7f8fd406ba30 2024-04-17T10:00:40.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.061+0000 7f8fd9a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f8fd415d090 con 0x7f8fd40cf370 2024-04-17T10:00:40.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.061+0000 7f8fd2a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8fd406ba30 0x7f8fd406be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:40.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.061+0000 7f8fd2a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8fd406ba30 0x7f8fd406be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40174/0 (socket says 172.21.15.32:40174) 2024-04-17T10:00:40.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.061+0000 7f8fd2a00640 1 -- 172.21.15.32:0/3405584475 learned_addr learned my addr 172.21.15.32:0/3405584475 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:40.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.061+0000 7f8fd3400640 1 --1- 172.21.15.32:0/3405584475 >> v1:172.21.15.32:6789/0 conn(0x7f8fd4160350 0x7f8fd4160730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48748/0 (socket says 172.21.15.32:48748) 2024-04-17T10:00:40.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.061+0000 7f8fca000640 1 --2- 172.21.15.32:0/3405584475 >> v2:172.21.15.144:3300/0 conn(0x7f8fd40cf370 0x7f8fd40cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:40.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.061+0000 7f8fd2a00640 1 -- 172.21.15.32:0/3405584475 >> v2:172.21.15.144:3300/0 conn(0x7f8fd40cf370 msgr2=0x7f8fd40cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:40.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.061+0000 7f8fd2a00640 1 --2- 172.21.15.32:0/3405584475 >> v2:172.21.15.144:3300/0 conn(0x7f8fd40cf370 0x7f8fd40cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.061+0000 7f8fd2a00640 1 -- 172.21.15.32:0/3405584475 >> v1:172.21.15.32:6789/0 conn(0x7f8fd4160350 legacy=0x7f8fd4160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:40.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.061+0000 7f8fd2000640 1 -- 172.21.15.32:0/3405584475 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 67429272 0 0) 0x7f8fd406c350 con 0x7f8fd4160350 2024-04-17T10:00:40.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.061+0000 7f8fd2a00640 1 -- 172.21.15.32:0/3405584475 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd415d370 con 0x7f8fd406ba30 2024-04-17T10:00:40.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.061+0000 7f8fd2a00640 1 --2- 172.21.15.32:0/3405584475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fd406ba30 0x7f8fd406be10 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f8fd40afec0 tx=0x7f8fc401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607d5d8c0b1e1eb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:40.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.061+0000 7f8fd2000640 1 -- 172.21.15.32:0/3405584475 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc40b0070 con 0x7f8fd406ba30 2024-04-17T10:00:40.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.065+0000 7f8fd2000640 1 -- 172.21.15.32:0/3405584475 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fc40ab600 con 0x7f8fd406ba30 2024-04-17T10:00:40.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.065+0000 7f8fd2000640 1 -- 172.21.15.32:0/3405584475 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc40ab9d0 con 0x7f8fd406ba30 2024-04-17T10:00:40.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.065+0000 7f8fd9a00640 1 -- 172.21.15.32:0/3405584475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fd406ba30 msgr2=0x7f8fd406be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:40.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.065+0000 7f8fd9a00640 1 --2- 172.21.15.32:0/3405584475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fd406ba30 0x7f8fd406be10 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f8fd40afec0 tx=0x7f8fc401c490 comp rx=0 tx=0).stop 2024-04-17T10:00:40.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.065+0000 7f8fd9a00640 1 -- 172.21.15.32:0/3405584475 shutdown_connections 2024-04-17T10:00:40.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.065+0000 7f8fd9a00640 1 --2- 172.21.15.32:0/3405584475 >> v2:172.21.15.144:3300/0 conn(0x7f8fd40cf370 0x7f8fd40cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.065+0000 7f8fd9a00640 1 --2- 172.21.15.32:0/3405584475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fd406ba30 0x7f8fd406be10 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.065+0000 7f8fd9a00640 1 -- 172.21.15.32:0/3405584475 >> 172.21.15.32:0/3405584475 conn(0x7f8fd40b2120 msgr2=0x7f8fd40b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:40.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.065+0000 7f8fd9a00640 1 -- 172.21.15.32:0/3405584475 shutdown_connections 2024-04-17T10:00:40.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.065+0000 7f8fd9a00640 1 -- 172.21.15.32:0/3405584475 wait complete. 2024-04-17T10:00:40.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.065+0000 7f8fd9a00640 1 Processor -- start 2024-04-17T10:00:40.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.065+0000 7f8fd9a00640 1 -- start start 2024-04-17T10:00:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.065+0000 7f8fd9a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd406ba30 0x7f8fd415fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.065+0000 7f8fd9a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fd40cf370 0x7f8fd40b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.065+0000 7f8fd9a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fd4160350 0x7f8fd40b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.065+0000 7f8fd9a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd415dcf0 con 0x7f8fd4160350 2024-04-17T10:00:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.065+0000 7f8fd9a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd415db70 con 0x7f8fd406ba30 2024-04-17T10:00:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.065+0000 7f8fd9a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8fd4146230 con 0x7f8fd40cf370 2024-04-17T10:00:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.065+0000 7f8fd3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fd4160350 0x7f8fd40b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.065+0000 7f8fd3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fd4160350 0x7f8fd40b1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40176/0 (socket says 172.21.15.32:40176) 2024-04-17T10:00:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.065+0000 7f8fd3400640 1 -- 172.21.15.32:0/393246216 learned_addr learned my addr 172.21.15.32:0/393246216 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.065+0000 7f8fd2a00640 1 --2- 172.21.15.32:0/393246216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd406ba30 0x7f8fd415fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.065+0000 7f8fca000640 1 --2- 172.21.15.32:0/393246216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fd40cf370 0x7f8fd40b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:40.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.069+0000 7f8fd3400640 1 -- 172.21.15.32:0/393246216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fd40cf370 msgr2=0x7f8fd40b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:40.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.069+0000 7f8fd3400640 1 --2- 172.21.15.32:0/393246216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fd40cf370 0x7f8fd40b0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.069+0000 7f8fd3400640 1 -- 172.21.15.32:0/393246216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd406ba30 msgr2=0x7f8fd415fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:40.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.069+0000 7f8fd3400640 1 --2- 172.21.15.32:0/393246216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd406ba30 0x7f8fd415fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.069+0000 7f8fd3400640 1 -- 172.21.15.32:0/393246216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd415e110 con 0x7f8fd4160350 2024-04-17T10:00:40.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.069+0000 7f8fd3400640 1 --2- 172.21.15.32:0/393246216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fd4160350 0x7f8fd40b1520 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00c700 tx=0x7f8fcc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:40.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.069+0000 7f8fcb400640 1 -- 172.21.15.32:0/393246216 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc018070 con 0x7f8fd4160350 2024-04-17T10:00:40.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.069+0000 7f8fcb400640 1 -- 172.21.15.32:0/393246216 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fcc00ce10 con 0x7f8fd4160350 2024-04-17T10:00:40.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.069+0000 7f8fcb400640 1 -- 172.21.15.32:0/393246216 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc00a860 con 0x7f8fd4160350 2024-04-17T10:00:40.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.069+0000 7f8fd9a00640 1 -- 172.21.15.32:0/393246216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd415e3f0 con 0x7f8fd4160350 2024-04-17T10:00:40.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.069+0000 7f8fd9a00640 1 -- 172.21.15.32:0/393246216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd415e9b0 con 0x7f8fd4160350 2024-04-17T10:00:40.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.069+0000 7f8fd9a00640 1 -- 172.21.15.32:0/393246216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fd4068a90 con 0x7f8fd4160350 2024-04-17T10:00:40.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.069+0000 7f8fcb400640 1 -- 172.21.15.32:0/393246216 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8fcc00aa00 con 0x7f8fd4160350 2024-04-17T10:00:40.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.069+0000 7f8fcb400640 1 --2- 172.21.15.32:0/393246216 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8fb0068380 0x7f8fb006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:40.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.069+0000 7f8fcb400640 1 -- 172.21.15.32:0/393246216 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7f8fcc085f10 con 0x7f8fd4160350 2024-04-17T10:00:40.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.073+0000 7f8fd2a00640 1 --2- 172.21.15.32:0/393246216 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8fb0068380 0x7f8fb006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:40.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.073+0000 7f8fd2a00640 1 --2- 172.21.15.32:0/393246216 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8fb0068380 0x7f8fb006a840 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40ad000 tx=0x7f8fc40a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:40.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.077+0000 7f8fcb400640 1 -- 172.21.15.32:0/393246216 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f8fcc055e80 con 0x7f8fd4160350 2024-04-17T10:00:40.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.245+0000 7f8fd9a00640 1 -- 172.21.15.32:0/393246216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8fd406c850 con 0x7f8fd4160350 2024-04-17T10:00:40.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.245+0000 7f8fcb400640 1 -- 172.21.15.32:0/393246216 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) ==== 104+0+11 (secure 0 0 0) 0x7f8fcc05a180 con 0x7f8fd4160350 2024-04-17T10:00:40.242 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T10:00:40.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.245+0000 7f8fd9a00640 1 -- 172.21.15.32:0/393246216 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8fb0068380 msgr2=0x7f8fb006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:40.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.245+0000 7f8fd9a00640 1 --2- 172.21.15.32:0/393246216 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8fb0068380 0x7f8fb006a840 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40ad000 tx=0x7f8fc40a6000 comp rx=0 tx=0).stop 2024-04-17T10:00:40.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.245+0000 7f8fd9a00640 1 -- 172.21.15.32:0/393246216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fd4160350 msgr2=0x7f8fd40b1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:40.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.245+0000 7f8fd9a00640 1 --2- 172.21.15.32:0/393246216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fd4160350 0x7f8fd40b1520 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00c700 tx=0x7f8fcc00cbc0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.245+0000 7f8fd9a00640 1 -- 172.21.15.32:0/393246216 shutdown_connections 2024-04-17T10:00:40.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.245+0000 7f8fd9a00640 1 --2- 172.21.15.32:0/393246216 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8fb0068380 0x7f8fb006a840 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.245+0000 7f8fd9a00640 1 --2- 172.21.15.32:0/393246216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fd40cf370 0x7f8fd40b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.245+0000 7f8fd9a00640 1 --2- 172.21.15.32:0/393246216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fd4160350 0x7f8fd40b1520 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.245+0000 7f8fd9a00640 1 --2- 172.21.15.32:0/393246216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd406ba30 0x7f8fd415fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.245+0000 7f8fd9a00640 1 -- 172.21.15.32:0/393246216 >> 172.21.15.32:0/393246216 conn(0x7f8fd40b2120 msgr2=0x7f8fd4166390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:40.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.245+0000 7f8fd9a00640 1 -- 172.21.15.32:0/393246216 shutdown_connections 2024-04-17T10:00:40.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.245+0000 7f8fd9a00640 1 -- 172.21.15.32:0/393246216 wait complete. 2024-04-17T10:00:40.263 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:00:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.409+0000 7f09d0000640 1 Processor -- start 2024-04-17T10:00:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.409+0000 7f09d0000640 1 -- start start 2024-04-17T10:00:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.409+0000 7f09d0000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09c815ac30 0x7f09c806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.409+0000 7f09d0000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f09c815b010 0x7f09c806dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.409+0000 7f09d0000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09c806fcd0 con 0x7f09c80b3d80 2024-04-17T10:00:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.409+0000 7f09d0000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f09c806f300 con 0x7f09c815ac30 2024-04-17T10:00:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.409+0000 7f09d0000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f09c806e6f0 con 0x7f09c815b010 2024-04-17T10:00:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.409+0000 7f09cd600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09c815ac30 0x7f09c806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.409+0000 7f09cd600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09c815ac30 0x7f09c806f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55078/0 (socket says 172.21.15.32:55078) 2024-04-17T10:00:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.409+0000 7f09cd600640 1 -- 172.21.15.32:0/779505093 learned_addr learned my addr 172.21.15.32:0/779505093 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.409+0000 7f09ce000640 1 --1- 172.21.15.32:0/779505093 >> v1:172.21.15.32:6789/0 conn(0x7f09c80b3d80 0x7f09c8071e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44772/0 (socket says 172.21.15.32:44772) 2024-04-17T10:00:40.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.409+0000 7f09ccc00640 1 --2- 172.21.15.32:0/779505093 >> v2:172.21.15.144:3300/0 conn(0x7f09c815b010 0x7f09c806dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:40.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.409+0000 7f09cd600640 1 -- 172.21.15.32:0/779505093 >> v2:172.21.15.144:3300/0 conn(0x7f09c815b010 msgr2=0x7f09c806dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:40.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.409+0000 7f09cd600640 1 --2- 172.21.15.32:0/779505093 >> v2:172.21.15.144:3300/0 conn(0x7f09c815b010 0x7f09c806dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.409+0000 7f09cd600640 1 -- 172.21.15.32:0/779505093 >> v1:172.21.15.32:6789/0 conn(0x7f09c80b3d80 legacy=0x7f09c8071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:40.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.409+0000 7f09cd600640 1 -- 172.21.15.32:0/779505093 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c806e9d0 con 0x7f09c815ac30 2024-04-17T10:00:40.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.409+0000 7f09c7e00640 1 -- 172.21.15.32:0/779505093 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2545050990 0 0) 0x7f09c806fcd0 con 0x7f09c80b3d80 2024-04-17T10:00:40.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.413+0000 7f09cd600640 1 --2- 172.21.15.32:0/779505093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09c815ac30 0x7f09c806f790 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f09c806d7b0 tx=0x7f09bc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c99ebb4481c4123 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:40.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.413+0000 7f09c7e00640 1 -- 172.21.15.32:0/779505093 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09bc0ab610 con 0x7f09c815ac30 2024-04-17T10:00:40.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.413+0000 7f09c7e00640 1 -- 172.21.15.32:0/779505093 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09bc0b0070 con 0x7f09c815ac30 2024-04-17T10:00:40.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.413+0000 7f09c7e00640 1 -- 172.21.15.32:0/779505093 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09bc0ab9d0 con 0x7f09c815ac30 2024-04-17T10:00:40.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.413+0000 7f09d0000640 1 -- 172.21.15.32:0/779505093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09c815ac30 msgr2=0x7f09c806f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:40.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.413+0000 7f09d0000640 1 --2- 172.21.15.32:0/779505093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09c815ac30 0x7f09c806f790 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f09c806d7b0 tx=0x7f09bc01c520 comp rx=0 tx=0).stop 2024-04-17T10:00:40.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.413+0000 7f09d0000640 1 -- 172.21.15.32:0/779505093 shutdown_connections 2024-04-17T10:00:40.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.413+0000 7f09d0000640 1 --2- 172.21.15.32:0/779505093 >> v2:172.21.15.144:3300/0 conn(0x7f09c815b010 0x7f09c806dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.413+0000 7f09d0000640 1 --2- 172.21.15.32:0/779505093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09c815ac30 0x7f09c806f790 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.413+0000 7f09d0000640 1 -- 172.21.15.32:0/779505093 >> 172.21.15.32:0/779505093 conn(0x7f09c80b2120 msgr2=0x7f09c80b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:40.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.413+0000 7f09d0000640 1 -- 172.21.15.32:0/779505093 shutdown_connections 2024-04-17T10:00:40.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.413+0000 7f09d0000640 1 -- 172.21.15.32:0/779505093 wait complete. 2024-04-17T10:00:40.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09d0000640 1 Processor -- start 2024-04-17T10:00:40.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09d0000640 1 -- start start 2024-04-17T10:00:40.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09d0000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09c80b3d80 0x7f09c8155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:40.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09d0000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09c815ac30 0x7f09c80d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:40.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09d0000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09c815b010 0x7f09c8149e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:40.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09d0000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f09c80726f0 con 0x7f09c815b010 2024-04-17T10:00:40.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09d0000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f09c8072570 con 0x7f09c815ac30 2024-04-17T10:00:40.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09d0000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f09c80cd9c0 con 0x7f09c80b3d80 2024-04-17T10:00:40.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09ce000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09c815b010 0x7f09c8149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:40.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09ce000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09c815b010 0x7f09c8149e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55080/0 (socket says 172.21.15.32:55080) 2024-04-17T10:00:40.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09ce000640 1 -- 172.21.15.32:0/843705783 learned_addr learned my addr 172.21.15.32:0/843705783 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:40.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09ccc00640 1 --2- 172.21.15.32:0/843705783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09c815ac30 0x7f09c80d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:40.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09cd600640 1 --2- 172.21.15.32:0/843705783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09c80b3d80 0x7f09c8155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:40.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09ce000640 1 -- 172.21.15.32:0/843705783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09c80b3d80 msgr2=0x7f09c8155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:40.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09ce000640 1 --2- 172.21.15.32:0/843705783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09c80b3d80 0x7f09c8155630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09ce000640 1 -- 172.21.15.32:0/843705783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09c815ac30 msgr2=0x7f09c80d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:40.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09ce000640 1 --2- 172.21.15.32:0/843705783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09c815ac30 0x7f09c80d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09ce000640 1 -- 172.21.15.32:0/843705783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c8155f10 con 0x7f09c815b010 2024-04-17T10:00:40.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09cd600640 1 --2- 172.21.15.32:0/843705783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09c80b3d80 0x7f09c8155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:40.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09ce000640 1 --2- 172.21.15.32:0/843705783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09c815b010 0x7f09c8149e10 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f09c000c7a0 tx=0x7f09c000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:40.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09c5600640 1 -- 172.21.15.32:0/843705783 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c0018070 con 0x7f09c815b010 2024-04-17T10:00:40.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09c5600640 1 -- 172.21.15.32:0/843705783 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c000a640 con 0x7f09c815b010 2024-04-17T10:00:40.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09c5600640 1 -- 172.21.15.32:0/843705783 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c000aab0 con 0x7f09c815b010 2024-04-17T10:00:40.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09d0000640 1 -- 172.21.15.32:0/843705783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c81561f0 con 0x7f09c815b010 2024-04-17T10:00:40.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.417+0000 7f09d0000640 1 -- 172.21.15.32:0/843705783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c81567b0 con 0x7f09c815b010 2024-04-17T10:00:40.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.421+0000 7f09d0000640 1 -- 172.21.15.32:0/843705783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c8068a90 con 0x7f09c815b010 2024-04-17T10:00:40.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.421+0000 7f09c5600640 1 -- 172.21.15.32:0/843705783 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f09c001b070 con 0x7f09c815b010 2024-04-17T10:00:40.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.421+0000 7f09c5600640 1 --2- 172.21.15.32:0/843705783 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f09a8068380 0x7f09a806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:40.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.421+0000 7f09c5600640 1 -- 172.21.15.32:0/843705783 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7f09c0086090 con 0x7f09c815b010 2024-04-17T10:00:40.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.421+0000 7f09cd600640 1 --2- 172.21.15.32:0/843705783 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f09a8068380 0x7f09a806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:40.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.421+0000 7f09cd600640 1 --2- 172.21.15.32:0/843705783 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f09a8068380 0x7f09a806a840 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f09bc0af000 tx=0x7f09bc0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:40.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.425+0000 7f09c5600640 1 -- 172.21.15.32:0/843705783 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f09c0055ed0 con 0x7f09c815b010 2024-04-17T10:00:40.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.589+0000 7f09d0000640 1 -- 172.21.15.32:0/843705783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f09c8072a60 con 0x7f09c815b010 2024-04-17T10:00:40.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.593+0000 7f09c5600640 1 -- 172.21.15.32:0/843705783 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) ==== 100+0+11 (secure 0 0 0) 0x7f09c005a1d0 con 0x7f09c815b010 2024-04-17T10:00:40.589 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T10:00:40.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.593+0000 7f09d0000640 1 -- 172.21.15.32:0/843705783 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f09a8068380 msgr2=0x7f09a806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.593+0000 7f09d0000640 1 --2- 172.21.15.32:0/843705783 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f09a8068380 0x7f09a806a840 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f09bc0af000 tx=0x7f09bc0a7000 comp rx=0 tx=0).stop 2024-04-17T10:00:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.593+0000 7f09d0000640 1 -- 172.21.15.32:0/843705783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09c815b010 msgr2=0x7f09c8149e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.593+0000 7f09d0000640 1 --2- 172.21.15.32:0/843705783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09c815b010 0x7f09c8149e10 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f09c000c7a0 tx=0x7f09c000cc60 comp rx=0 tx=0).stop 2024-04-17T10:00:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.593+0000 7f09d0000640 1 -- 172.21.15.32:0/843705783 shutdown_connections 2024-04-17T10:00:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.593+0000 7f09d0000640 1 --2- 172.21.15.32:0/843705783 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f09a8068380 0x7f09a806a840 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.593+0000 7f09d0000640 1 --2- 172.21.15.32:0/843705783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09c815b010 0x7f09c8149e10 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.593+0000 7f09d0000640 1 --2- 172.21.15.32:0/843705783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09c815ac30 0x7f09c80d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.593+0000 7f09d0000640 1 --2- 172.21.15.32:0/843705783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09c80b3d80 0x7f09c8155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.593+0000 7f09d0000640 1 -- 172.21.15.32:0/843705783 >> 172.21.15.32:0/843705783 conn(0x7f09c80b2120 msgr2=0x7f09c80af8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.593+0000 7f09d0000640 1 -- 172.21.15.32:0/843705783 shutdown_connections 2024-04-17T10:00:40.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.593+0000 7f09d0000640 1 -- 172.21.15.32:0/843705783 wait complete. 2024-04-17T10:00:40.611 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:00:40.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.757+0000 7fe61e200640 1 Processor -- start 2024-04-17T10:00:40.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.757+0000 7fe61e200640 1 -- start start 2024-04-17T10:00:40.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.757+0000 7fe61e200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe61815c290 0x7fe6180d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:40.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.757+0000 7fe61e200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe618071eb0 con 0x7fe61815c670 2024-04-17T10:00:40.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.757+0000 7fe61e200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe618072190 con 0x7fe6180d00f0 2024-04-17T10:00:40.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.757+0000 7fe61e200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe618072470 con 0x7fe61815c290 2024-04-17T10:00:40.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.757+0000 7fe617400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe61815c290 0x7fe6180d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:40.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.757+0000 7fe617400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe61815c290 0x7fe6180d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55086/0 (socket says 172.21.15.32:55086) 2024-04-17T10:00:40.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.757+0000 7fe617400640 1 -- 172.21.15.32:0/3430592608 learned_addr learned my addr 172.21.15.32:0/3430592608 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:40.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.757+0000 7fe617400640 1 -- 172.21.15.32:0/3430592608 >> v1:172.21.15.144:6789/0 conn(0x7fe6180d00f0 legacy=0x7fe6180af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:40.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.757+0000 7fe617400640 1 -- 172.21.15.32:0/3430592608 >> v1:172.21.15.32:6789/0 conn(0x7fe61815c670 legacy=0x7fe6180b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:40.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.757+0000 7fe616a00640 1 -- 172.21.15.32:0/3430592608 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1261877241 0 0) 0x7fe618071eb0 con 0x7fe61815c670 2024-04-17T10:00:40.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.757+0000 7fe617400640 1 -- 172.21.15.32:0/3430592608 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe618072750 con 0x7fe61815c290 2024-04-17T10:00:40.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.761+0000 7fe617400640 1 --2- 172.21.15.32:0/3430592608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe61815c290 0x7fe6180d0720 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fe608009870 tx=0x7fe60801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30d7b1907e71b43 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:40.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.761+0000 7fe616a00640 1 -- 172.21.15.32:0/3430592608 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6080b0070 con 0x7fe61815c290 2024-04-17T10:00:40.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.761+0000 7fe616a00640 1 -- 172.21.15.32:0/3430592608 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6080ab600 con 0x7fe61815c290 2024-04-17T10:00:40.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.761+0000 7fe616a00640 1 -- 172.21.15.32:0/3430592608 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6080ab9d0 con 0x7fe61815c290 2024-04-17T10:00:40.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.761+0000 7fe61e200640 1 -- 172.21.15.32:0/3430592608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe61815c290 msgr2=0x7fe6180d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:40.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.761+0000 7fe61e200640 1 --2- 172.21.15.32:0/3430592608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe61815c290 0x7fe6180d0720 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fe608009870 tx=0x7fe60801c440 comp rx=0 tx=0).stop 2024-04-17T10:00:40.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.761+0000 7fe61e200640 1 -- 172.21.15.32:0/3430592608 shutdown_connections 2024-04-17T10:00:40.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.761+0000 7fe61e200640 1 --2- 172.21.15.32:0/3430592608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe61815c290 0x7fe6180d0720 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.761+0000 7fe61e200640 1 -- 172.21.15.32:0/3430592608 >> 172.21.15.32:0/3430592608 conn(0x7fe61806ba30 msgr2=0x7fe61806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:40.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.761+0000 7fe61e200640 1 -- 172.21.15.32:0/3430592608 shutdown_connections 2024-04-17T10:00:40.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.761+0000 7fe61e200640 1 -- 172.21.15.32:0/3430592608 wait complete. 2024-04-17T10:00:40.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.761+0000 7fe61e200640 1 Processor -- start 2024-04-17T10:00:40.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.761+0000 7fe61e200640 1 -- start start 2024-04-17T10:00:40.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe61e200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6180d00f0 0x7fe618152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:40.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe61e200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe61815c290 0x7fe618145400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:40.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe61e200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe61815c670 0x7fe618149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:40.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe61e200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe6180afbf0 con 0x7fe61815c670 2024-04-17T10:00:40.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe61e200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe6180afa70 con 0x7fe6180d00f0 2024-04-17T10:00:40.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe61e200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe61816c800 con 0x7fe61815c290 2024-04-17T10:00:40.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe617e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe61815c670 0x7fe618149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:40.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe617400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6180d00f0 0x7fe618152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:40.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe617e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe61815c670 0x7fe618149c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55102/0 (socket says 172.21.15.32:55102) 2024-04-17T10:00:40.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe617e00640 1 -- 172.21.15.32:0/1237124060 learned_addr learned my addr 172.21.15.32:0/1237124060 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:40.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe60fe00640 1 --2- 172.21.15.32:0/1237124060 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe61815c290 0x7fe618145400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:40.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe617e00640 1 -- 172.21.15.32:0/1237124060 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe61815c290 msgr2=0x7fe618145400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:40.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe617e00640 1 --2- 172.21.15.32:0/1237124060 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe61815c290 0x7fe618145400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe617e00640 1 -- 172.21.15.32:0/1237124060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6180d00f0 msgr2=0x7fe618152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:40.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe617e00640 1 --2- 172.21.15.32:0/1237124060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6180d00f0 0x7fe618152690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe617e00640 1 -- 172.21.15.32:0/1237124060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe618145940 con 0x7fe61815c670 2024-04-17T10:00:40.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe617400640 1 --2- 172.21.15.32:0/1237124060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6180d00f0 0x7fe618152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:00:40.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe617e00640 1 --2- 172.21.15.32:0/1237124060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe61815c670 0x7fe618149c90 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fe61000c7b0 tx=0x7fe61000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:40.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe60f400640 1 -- 172.21.15.32:0/1237124060 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61000ce60 con 0x7fe61815c670 2024-04-17T10:00:40.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe60f400640 1 -- 172.21.15.32:0/1237124060 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe610003db0 con 0x7fe61815c670 2024-04-17T10:00:40.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe61e200640 1 -- 172.21.15.32:0/1237124060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61814a670 con 0x7fe61815c670 2024-04-17T10:00:40.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe60f400640 1 -- 172.21.15.32:0/1237124060 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6100052b0 con 0x7fe61815c670 2024-04-17T10:00:40.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe61e200640 1 -- 172.21.15.32:0/1237124060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe618151e90 con 0x7fe61815c670 2024-04-17T10:00:40.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.765+0000 7fe61e200640 1 -- 172.21.15.32:0/1237124060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe618068a90 con 0x7fe61815c670 2024-04-17T10:00:40.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.769+0000 7fe60f400640 1 -- 172.21.15.32:0/1237124060 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe610005450 con 0x7fe61815c670 2024-04-17T10:00:40.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.769+0000 7fe60f400640 1 --2- 172.21.15.32:0/1237124060 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe5f40683b0 0x7fe5f406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:40.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.769+0000 7fe60f400640 1 -- 172.21.15.32:0/1237124060 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7fe610086780 con 0x7fe61815c670 2024-04-17T10:00:40.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.769+0000 7fe617400640 1 --2- 172.21.15.32:0/1237124060 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe5f40683b0 0x7fe5f406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:40.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.769+0000 7fe617400640 1 --2- 172.21.15.32:0/1237124060 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe5f40683b0 0x7fe5f406a870 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe6080ad000 tx=0x7fe608008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:40.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.773+0000 7fe60f400640 1 -- 172.21.15.32:0/1237124060 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fe610056650 con 0x7fe61815c670 2024-04-17T10:00:40.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.953+0000 7fe61e200640 1 -- 172.21.15.32:0/1237124060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe6180b0eb0 con 0x7fe61815c670 2024-04-17T10:00:40.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.953+0000 7fe60f400640 1 -- 172.21.15.32:0/1237124060 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v85) ==== 80+0+122 (secure 0 0 0) 0x7fe61005a950 con 0x7fe61815c670 2024-04-17T10:00:40.952 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6837/2656735817 2024-04-18T09:59:28.985107+0000 2024-04-17T10:00:40.952 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6835/2656735817 2024-04-18T09:59:28.985107+0000 2024-04-17T10:00:40.952 INFO:teuthology.orchestra.run.smithi032.stderr:listed 2 entries 2024-04-17T10:00:40.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.957+0000 7fe61e200640 1 -- 172.21.15.32:0/1237124060 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe5f40683b0 msgr2=0x7fe5f406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:40.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.957+0000 7fe61e200640 1 --2- 172.21.15.32:0/1237124060 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe5f40683b0 0x7fe5f406a870 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe6080ad000 tx=0x7fe608008040 comp rx=0 tx=0).stop 2024-04-17T10:00:40.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.957+0000 7fe61e200640 1 -- 172.21.15.32:0/1237124060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe61815c670 msgr2=0x7fe618149c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:40.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.957+0000 7fe61e200640 1 --2- 172.21.15.32:0/1237124060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe61815c670 0x7fe618149c90 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fe61000c7b0 tx=0x7fe61000cc70 comp rx=0 tx=0).stop 2024-04-17T10:00:40.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.957+0000 7fe61e200640 1 -- 172.21.15.32:0/1237124060 shutdown_connections 2024-04-17T10:00:40.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.957+0000 7fe61e200640 1 --2- 172.21.15.32:0/1237124060 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe5f40683b0 0x7fe5f406a870 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.957+0000 7fe61e200640 1 --2- 172.21.15.32:0/1237124060 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe61815c290 0x7fe618145400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.957+0000 7fe61e200640 1 --2- 172.21.15.32:0/1237124060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe61815c670 0x7fe618149c90 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.957+0000 7fe61e200640 1 --2- 172.21.15.32:0/1237124060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6180d00f0 0x7fe618152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:40.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.957+0000 7fe61e200640 1 -- 172.21.15.32:0/1237124060 >> 172.21.15.32:0/1237124060 conn(0x7fe61806ba30 msgr2=0x7fe6180b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:40.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.957+0000 7fe61e200640 1 -- 172.21.15.32:0/1237124060 shutdown_connections 2024-04-17T10:00:40.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:40.957+0000 7fe61e200640 1 -- 172.21.15.32:0/1237124060 wait complete. 2024-04-17T10:00:40.973 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:00:40.973 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:00:40.984 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:00:41.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.165+0000 7f04bee00640 1 Processor -- start 2024-04-17T10:00:41.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.165+0000 7f04bee00640 1 -- start start 2024-04-17T10:00:41.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.169+0000 7f04bee00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04b815baa0 0x7f04b815bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:41.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.169+0000 7f04bee00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f04b806ef60 0x7f04b806f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:41.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.169+0000 7f04bee00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04b8072bb0 con 0x7f04b8144e90 2024-04-17T10:00:41.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.169+0000 7f04bee00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f04b806f8f0 con 0x7f04b815baa0 2024-04-17T10:00:41.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.169+0000 7f04bee00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f04b8145270 con 0x7f04b806ef60 2024-04-17T10:00:41.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.169+0000 7f04af400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04b815baa0 0x7f04b815bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:41.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.169+0000 7f04af400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04b815baa0 0x7f04b815bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55118/0 (socket says 172.21.15.32:55118) 2024-04-17T10:00:41.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.169+0000 7f04af400640 1 -- 172.21.15.32:0/2026469546 learned_addr learned my addr 172.21.15.32:0/2026469546 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:41.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.169+0000 7f04bce00640 1 --2- 172.21.15.32:0/2026469546 >> v2:172.21.15.144:3300/0 conn(0x7f04b806ef60 0x7f04b806f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:41.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.169+0000 7f04af400640 1 -- 172.21.15.32:0/2026469546 >> v2:172.21.15.144:3300/0 conn(0x7f04b806ef60 msgr2=0x7f04b806f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:41.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.169+0000 7f04af400640 1 --2- 172.21.15.32:0/2026469546 >> v2:172.21.15.144:3300/0 conn(0x7f04b806ef60 0x7f04b806f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.169+0000 7f04af400640 1 -- 172.21.15.32:0/2026469546 >> v1:172.21.15.144:6789/0 conn(0x7f04b8144e90 legacy=0x7f04b80b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:41.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.169+0000 7f04af400640 1 -- 172.21.15.32:0/2026469546 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b80b3dc0 con 0x7f04b815baa0 2024-04-17T10:00:41.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.169+0000 7f04af400640 1 --2- 172.21.15.32:0/2026469546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04b815baa0 0x7f04b815bf30 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f04b80b3a00 tx=0x7f04a001ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6c4ad50b995276 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:41.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.173+0000 7f04b7400640 1 -- 172.21.15.32:0/2026469546 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a000b2d0 con 0x7f04b815baa0 2024-04-17T10:00:41.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.173+0000 7f04b7400640 1 -- 172.21.15.32:0/2026469546 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a000b470 con 0x7f04b815baa0 2024-04-17T10:00:41.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.173+0000 7f04b7400640 1 -- 172.21.15.32:0/2026469546 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a00a7bd0 con 0x7f04b815baa0 2024-04-17T10:00:41.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.173+0000 7f04bee00640 1 -- 172.21.15.32:0/2026469546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04b815baa0 msgr2=0x7f04b815bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:41.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.173+0000 7f04bee00640 1 --2- 172.21.15.32:0/2026469546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04b815baa0 0x7f04b815bf30 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f04b80b3a00 tx=0x7f04a001ebf0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.173+0000 7f04bee00640 1 -- 172.21.15.32:0/2026469546 shutdown_connections 2024-04-17T10:00:41.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.173+0000 7f04bee00640 1 --2- 172.21.15.32:0/2026469546 >> v2:172.21.15.144:3300/0 conn(0x7f04b806ef60 0x7f04b806f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.173+0000 7f04bee00640 1 --2- 172.21.15.32:0/2026469546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04b815baa0 0x7f04b815bf30 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.173+0000 7f04bee00640 1 -- 172.21.15.32:0/2026469546 >> 172.21.15.32:0/2026469546 conn(0x7f04b806ba30 msgr2=0x7f04b806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:41.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.173+0000 7f04bee00640 1 -- 172.21.15.32:0/2026469546 shutdown_connections 2024-04-17T10:00:41.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.173+0000 7f04bee00640 1 -- 172.21.15.32:0/2026469546 wait complete. 2024-04-17T10:00:41.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.173+0000 7f04bee00640 1 Processor -- start 2024-04-17T10:00:41.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.173+0000 7f04bee00640 1 -- start start 2024-04-17T10:00:41.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.173+0000 7f04bee00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04b806ef60 0x7f04b80c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:41.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.173+0000 7f04bee00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04b8144e90 0x7f04b81f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:41.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.173+0000 7f04bee00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04b815baa0 0x7f04b813bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:41.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.173+0000 7f04bee00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f04b80727f0 con 0x7f04b815baa0 2024-04-17T10:00:41.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.173+0000 7f04bee00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f04b8072670 con 0x7f04b8144e90 2024-04-17T10:00:41.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.173+0000 7f04bee00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f04b8072970 con 0x7f04b806ef60 2024-04-17T10:00:41.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.173+0000 7f04bce00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04b815baa0 0x7f04b813bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:41.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.173+0000 7f04bce00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04b815baa0 0x7f04b813bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55124/0 (socket says 172.21.15.32:55124) 2024-04-17T10:00:41.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.173+0000 7f04bce00640 1 -- 172.21.15.32:0/1910557950 learned_addr learned my addr 172.21.15.32:0/1910557950 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:41.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.173+0000 7f04af400640 1 --2- 172.21.15.32:0/1910557950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04b8144e90 0x7f04b81f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:41.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.177+0000 7f04b7e00640 1 --2- 172.21.15.32:0/1910557950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04b806ef60 0x7f04b80c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:41.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.177+0000 7f04bce00640 1 -- 172.21.15.32:0/1910557950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04b806ef60 msgr2=0x7f04b80c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:41.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.177+0000 7f04bce00640 1 --2- 172.21.15.32:0/1910557950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04b806ef60 0x7f04b80c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.177+0000 7f04bce00640 1 -- 172.21.15.32:0/1910557950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04b8144e90 msgr2=0x7f04b81f71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:41.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.177+0000 7f04bce00640 1 --2- 172.21.15.32:0/1910557950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04b8144e90 0x7f04b81f71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.177+0000 7f04bce00640 1 -- 172.21.15.32:0/1910557950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b8140dc0 con 0x7f04b815baa0 2024-04-17T10:00:41.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.177+0000 7f04bce00640 1 --2- 172.21.15.32:0/1910557950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04b815baa0 0x7f04b813bac0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f04b000e930 tx=0x7f04b000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:41.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.177+0000 7f04b4c00640 1 -- 172.21.15.32:0/1910557950 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0017070 con 0x7f04b815baa0 2024-04-17T10:00:41.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.177+0000 7f04b4c00640 1 -- 172.21.15.32:0/1910557950 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b000c9a0 con 0x7f04b815baa0 2024-04-17T10:00:41.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.177+0000 7f04bee00640 1 -- 172.21.15.32:0/1910557950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b8141080 con 0x7f04b815baa0 2024-04-17T10:00:41.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.177+0000 7f04b4c00640 1 -- 172.21.15.32:0/1910557950 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b000cd70 con 0x7f04b815baa0 2024-04-17T10:00:41.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.177+0000 7f04bee00640 1 -- 172.21.15.32:0/1910557950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b8141640 con 0x7f04b815baa0 2024-04-17T10:00:41.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.177+0000 7f04bee00640 1 -- 172.21.15.32:0/1910557950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04b8068a90 con 0x7f04b815baa0 2024-04-17T10:00:41.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.177+0000 7f04b4c00640 1 -- 172.21.15.32:0/1910557950 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f04b001a070 con 0x7f04b815baa0 2024-04-17T10:00:41.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.177+0000 7f04b4c00640 1 --2- 172.21.15.32:0/1910557950 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0494068380 0x7f049406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:41.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.177+0000 7f04b4c00640 1 -- 172.21.15.32:0/1910557950 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7f04b0086980 con 0x7f04b815baa0 2024-04-17T10:00:41.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.181+0000 7f04b7e00640 1 --2- 172.21.15.32:0/1910557950 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0494068380 0x7f049406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:41.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.181+0000 7f04b7e00640 1 --2- 172.21.15.32:0/1910557950 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0494068380 0x7f049406a840 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f04a8001810 tx=0x7f04a8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:41.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.181+0000 7f04b4c00640 1 -- 172.21.15.32:0/1910557950 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f04b0056920 con 0x7f04b815baa0 2024-04-17T10:00:41.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.353+0000 7f04bee00640 1 -- 172.21.15.32:0/1910557950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04b815c050 con 0x7f04b815baa0 2024-04-17T10:00:41.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.353+0000 7f04b4c00640 1 -- 172.21.15.32:0/1910557950 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16899 (secure 0 0 0) 0x7f04b005ac20 con 0x7f04b815baa0 2024-04-17T10:00:41.351 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:00:41.351 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":85,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T09:59:56.110989+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:59:03.223426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-17T09:59:03.780120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.11","mappings":[{"from":1,"to":3}]},{"pgid":"13.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6837/2656735817":"2024-04-18T09:59:28.985107+0000","172.21.15.32:6835/2656735817":"2024-04-18T09:59:28.985107+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:00:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.353+0000 7f04bee00640 1 -- 172.21.15.32:0/1910557950 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0494068380 msgr2=0x7f049406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.353+0000 7f04bee00640 1 --2- 172.21.15.32:0/1910557950 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0494068380 0x7f049406a840 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f04a8001810 tx=0x7f04a8009040 comp rx=0 tx=0).stop 2024-04-17T10:00:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.353+0000 7f04bee00640 1 -- 172.21.15.32:0/1910557950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04b815baa0 msgr2=0x7f04b813bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.353+0000 7f04bee00640 1 --2- 172.21.15.32:0/1910557950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04b815baa0 0x7f04b813bac0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f04b000e930 tx=0x7f04b000edf0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.353+0000 7f04bee00640 1 -- 172.21.15.32:0/1910557950 shutdown_connections 2024-04-17T10:00:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.353+0000 7f04bee00640 1 --2- 172.21.15.32:0/1910557950 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0494068380 0x7f049406a840 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.353+0000 7f04bee00640 1 --2- 172.21.15.32:0/1910557950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04b815baa0 0x7f04b813bac0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.353+0000 7f04bee00640 1 --2- 172.21.15.32:0/1910557950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04b8144e90 0x7f04b81f71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.353+0000 7f04bee00640 1 --2- 172.21.15.32:0/1910557950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04b806ef60 0x7f04b80c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.353+0000 7f04bee00640 1 -- 172.21.15.32:0/1910557950 >> 172.21.15.32:0/1910557950 conn(0x7f04b806ba30 msgr2=0x7f04b806be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.353+0000 7f04bee00640 1 -- 172.21.15.32:0/1910557950 shutdown_connections 2024-04-17T10:00:41.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.353+0000 7f04bee00640 1 -- 172.21.15.32:0/1910557950 wait complete. 2024-04-17T10:00:41.373 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:59:03.223426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:59:03.780120+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:00:41.373 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:00:41.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.521+0000 7fc886800640 1 Processor -- start 2024-04-17T10:00:41.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.521+0000 7fc886800640 1 -- start start 2024-04-17T10:00:41.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.525+0000 7fc886800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc88015c4c0 0x7fc8800adb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:41.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.525+0000 7fc886800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc88006c140 0x7fc8800b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:41.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.525+0000 7fc886800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8801629a0 con 0x7fc88015f9c0 2024-04-17T10:00:41.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.525+0000 7fc886800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc880162c80 con 0x7fc88006c140 2024-04-17T10:00:41.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.525+0000 7fc886800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc880162f60 con 0x7fc88015c4c0 2024-04-17T10:00:41.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.525+0000 7fc87f400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc88015c4c0 0x7fc8800adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:41.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.525+0000 7fc87fe00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc88006c140 0x7fc8800b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:41.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.525+0000 7fc87fe00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc88006c140 0x7fc8800b3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:45280/0 (socket says 172.21.15.32:45280) 2024-04-17T10:00:41.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.525+0000 7fc87fe00640 1 -- 172.21.15.32:0/869921252 learned_addr learned my addr 172.21.15.32:0/869921252 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:41.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.525+0000 7fc877e00640 1 --1- 172.21.15.32:0/869921252 >> v1:172.21.15.144:6789/0 conn(0x7fc88015f9c0 0x7fc88006ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:55054/0 (socket says 172.21.15.32:55054) 2024-04-17T10:00:41.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.525+0000 7fc87fe00640 1 -- 172.21.15.32:0/869921252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc88015c4c0 msgr2=0x7fc8800adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:41.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.525+0000 7fc87fe00640 1 --2- 172.21.15.32:0/869921252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc88015c4c0 0x7fc8800adb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.525+0000 7fc87fe00640 1 -- 172.21.15.32:0/869921252 >> v1:172.21.15.144:6789/0 conn(0x7fc88015f9c0 legacy=0x7fc88006ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:41.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.525+0000 7fc87fe00640 1 -- 172.21.15.32:0/869921252 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc880163240 con 0x7fc88006c140 2024-04-17T10:00:41.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.525+0000 7fc87f400640 1 --2- 172.21.15.32:0/869921252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc88015c4c0 0x7fc8800adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:00:41.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.529+0000 7fc87fe00640 1 --2- 172.21.15.32:0/869921252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc88006c140 0x7fc8800b3870 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc878009e50 tx=0x7fc87801ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64d4777dde917272 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:41.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.529+0000 7fc87ea00640 1 -- 172.21.15.32:0/869921252 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8780a5810 con 0x7fc88006c140 2024-04-17T10:00:41.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.529+0000 7fc87ea00640 1 -- 172.21.15.32:0/869921252 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8780a59b0 con 0x7fc88006c140 2024-04-17T10:00:41.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.529+0000 7fc87ea00640 1 -- 172.21.15.32:0/869921252 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8780a5d80 con 0x7fc88006c140 2024-04-17T10:00:41.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.529+0000 7fc886800640 1 -- 172.21.15.32:0/869921252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc88006c140 msgr2=0x7fc8800b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:41.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.529+0000 7fc886800640 1 --2- 172.21.15.32:0/869921252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc88006c140 0x7fc8800b3870 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc878009e50 tx=0x7fc87801ca30 comp rx=0 tx=0).stop 2024-04-17T10:00:41.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.529+0000 7fc886800640 1 -- 172.21.15.32:0/869921252 shutdown_connections 2024-04-17T10:00:41.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.529+0000 7fc886800640 1 --2- 172.21.15.32:0/869921252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc88006c140 0x7fc8800b3870 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.529+0000 7fc886800640 1 --2- 172.21.15.32:0/869921252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc88015c4c0 0x7fc8800adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.529+0000 7fc886800640 1 -- 172.21.15.32:0/869921252 >> 172.21.15.32:0/869921252 conn(0x7fc8800b2120 msgr2=0x7fc8800b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:41.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.529+0000 7fc886800640 1 -- 172.21.15.32:0/869921252 shutdown_connections 2024-04-17T10:00:41.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.529+0000 7fc886800640 1 -- 172.21.15.32:0/869921252 wait complete. 2024-04-17T10:00:41.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.529+0000 7fc886800640 1 Processor -- start 2024-04-17T10:00:41.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.529+0000 7fc886800640 1 -- start start 2024-04-17T10:00:41.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.533+0000 7fc886800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc88006c140 0x7fc8801538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:41.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.533+0000 7fc886800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc88015c4c0 0x7fc880153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:41.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.533+0000 7fc886800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc88015f9c0 0x7fc88015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:41.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.533+0000 7fc886800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc8801626b0 con 0x7fc88015c4c0 2024-04-17T10:00:41.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.533+0000 7fc886800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc880162530 con 0x7fc88006c140 2024-04-17T10:00:41.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.533+0000 7fc886800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc88016b6a0 con 0x7fc88015f9c0 2024-04-17T10:00:41.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.533+0000 7fc877e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc88015c4c0 0x7fc880153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:41.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.533+0000 7fc877e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc88015c4c0 0x7fc880153e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55136/0 (socket says 172.21.15.32:55136) 2024-04-17T10:00:41.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.533+0000 7fc877e00640 1 -- 172.21.15.32:0/2365064113 learned_addr learned my addr 172.21.15.32:0/2365064113 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:41.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.533+0000 7fc87f400640 1 --2- 172.21.15.32:0/2365064113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc88006c140 0x7fc8801538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:41.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.533+0000 7fc87fe00640 1 --2- 172.21.15.32:0/2365064113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc88015f9c0 0x7fc88015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:41.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.533+0000 7fc877e00640 1 -- 172.21.15.32:0/2365064113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc88015f9c0 msgr2=0x7fc88015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:41.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.533+0000 7fc877e00640 1 --2- 172.21.15.32:0/2365064113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc88015f9c0 0x7fc88015b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.533+0000 7fc877e00640 1 -- 172.21.15.32:0/2365064113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc88006c140 msgr2=0x7fc8801538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:41.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.533+0000 7fc877e00640 1 --2- 172.21.15.32:0/2365064113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc88006c140 0x7fc8801538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.533+0000 7fc877e00640 1 -- 172.21.15.32:0/2365064113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8800b1600 con 0x7fc88015c4c0 2024-04-17T10:00:41.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.533+0000 7fc877e00640 1 --2- 172.21.15.32:0/2365064113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc88015c4c0 0x7fc880153e20 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fc86800c370 tx=0x7fc86800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:41.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.533+0000 7fc877400640 1 -- 172.21.15.32:0/2365064113 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8680074e0 con 0x7fc88015c4c0 2024-04-17T10:00:41.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.533+0000 7fc877400640 1 -- 172.21.15.32:0/2365064113 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8680076a0 con 0x7fc88015c4c0 2024-04-17T10:00:41.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.533+0000 7fc877400640 1 -- 172.21.15.32:0/2365064113 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc868004a10 con 0x7fc88015c4c0 2024-04-17T10:00:41.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.533+0000 7fc886800640 1 -- 172.21.15.32:0/2365064113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8800b18e0 con 0x7fc88015c4c0 2024-04-17T10:00:41.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.533+0000 7fc886800640 1 -- 172.21.15.32:0/2365064113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8801549d0 con 0x7fc88015c4c0 2024-04-17T10:00:41.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.533+0000 7fc886800640 1 -- 172.21.15.32:0/2365064113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc880068a90 con 0x7fc88015c4c0 2024-04-17T10:00:41.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.537+0000 7fc877400640 1 -- 172.21.15.32:0/2365064113 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc86801a020 con 0x7fc88015c4c0 2024-04-17T10:00:41.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.537+0000 7fc877400640 1 --2- 172.21.15.32:0/2365064113 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc85c0683b0 0x7fc85c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:41.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.537+0000 7fc877400640 1 -- 172.21.15.32:0/2365064113 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7fc8680860b0 con 0x7fc88015c4c0 2024-04-17T10:00:41.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.537+0000 7fc87f400640 1 --2- 172.21.15.32:0/2365064113 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc85c0683b0 0x7fc85c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:41.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.537+0000 7fc87f400640 1 --2- 172.21.15.32:0/2365064113 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc85c0683b0 0x7fc85c06a870 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc880156df0 tx=0x7fc870009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:41.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.541+0000 7fc877400640 1 -- 172.21.15.32:0/2365064113 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc868055f00 con 0x7fc88015c4c0 2024-04-17T10:00:41.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.709+0000 7fc886800640 1 -- 172.21.15.32:0/2365064113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc8800b3870 con 0x7fc88015c4c0 2024-04-17T10:00:41.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.709+0000 7fc877400640 1 -- 172.21.15.32:0/2365064113 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7fc86805a200 con 0x7fc88015c4c0 2024-04-17T10:00:41.706 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T10:00:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.709+0000 7fc886800640 1 -- 172.21.15.32:0/2365064113 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc85c0683b0 msgr2=0x7fc85c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.709+0000 7fc886800640 1 --2- 172.21.15.32:0/2365064113 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc85c0683b0 0x7fc85c06a870 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc880156df0 tx=0x7fc870009190 comp rx=0 tx=0).stop 2024-04-17T10:00:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.709+0000 7fc886800640 1 -- 172.21.15.32:0/2365064113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc88015c4c0 msgr2=0x7fc880153e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.709+0000 7fc886800640 1 --2- 172.21.15.32:0/2365064113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc88015c4c0 0x7fc880153e20 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fc86800c370 tx=0x7fc86800c830 comp rx=0 tx=0).stop 2024-04-17T10:00:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.709+0000 7fc886800640 1 -- 172.21.15.32:0/2365064113 shutdown_connections 2024-04-17T10:00:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.709+0000 7fc886800640 1 --2- 172.21.15.32:0/2365064113 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc85c0683b0 0x7fc85c06a870 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.709+0000 7fc886800640 1 --2- 172.21.15.32:0/2365064113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc88015f9c0 0x7fc88015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.709+0000 7fc886800640 1 --2- 172.21.15.32:0/2365064113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc88015c4c0 0x7fc880153e20 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.709+0000 7fc886800640 1 --2- 172.21.15.32:0/2365064113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc88006c140 0x7fc8801538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.709+0000 7fc886800640 1 -- 172.21.15.32:0/2365064113 >> 172.21.15.32:0/2365064113 conn(0x7fc8800b2120 msgr2=0x7fc880168f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.709+0000 7fc886800640 1 -- 172.21.15.32:0/2365064113 shutdown_connections 2024-04-17T10:00:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.709+0000 7fc886800640 1 -- 172.21.15.32:0/2365064113 wait complete. 2024-04-17T10:00:41.729 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:00:41.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.873+0000 7f276e000640 1 Processor -- start 2024-04-17T10:00:41.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.873+0000 7f276e000640 1 -- start start 2024-04-17T10:00:41.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.877+0000 7f276e000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f276806c140 0x7f27680ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:41.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.877+0000 7f276e000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27680cc940 con 0x7f276815b7d0 2024-04-17T10:00:41.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.877+0000 7f276e000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27680ccc20 con 0x7f276806f7c0 2024-04-17T10:00:41.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.877+0000 7f276e000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f276806e0b0 con 0x7f276806c140 2024-04-17T10:00:41.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.877+0000 7f2767400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f276815b7d0 0x7f276806e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44790/0 (socket says 172.21.15.32:44790) 2024-04-17T10:00:41.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.877+0000 7f2767400640 1 -- 172.21.15.32:0/18274068 learned_addr learned my addr 172.21.15.32:0/18274068 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:00:41.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.877+0000 7f2767e00640 1 --2- 172.21.15.32:0/18274068 >> v2:172.21.15.144:3300/0 conn(0x7f276806c140 0x7f27680ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:41.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.877+0000 7f2766a00640 1 -- 172.21.15.32:0/18274068 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3593979094 0 0) 0x7f27680cc940 con 0x7f276815b7d0 2024-04-17T10:00:41.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.877+0000 7f2766a00640 1 -- 172.21.15.32:0/18274068 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f274c005740 con 0x7f276815b7d0 2024-04-17T10:00:41.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.877+0000 7f2767e00640 1 -- 172.21.15.32:0/18274068 >> v1:172.21.15.144:6789/0 conn(0x7f276806f7c0 legacy=0x7f276806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:41.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.877+0000 7f2767e00640 1 -- 172.21.15.32:0/18274068 >> v1:172.21.15.32:6789/0 conn(0x7f276815b7d0 legacy=0x7f276806e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:41.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.877+0000 7f2767e00640 1 -- 172.21.15.32:0/18274068 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276806e390 con 0x7f276806c140 2024-04-17T10:00:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.877+0000 7f2767e00640 1 --2- 172.21.15.32:0/18274068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f276806c140 0x7f27680ce290 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2760009d90 tx=0x7f276001c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de0b9ddca1272d23 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.881+0000 7f2766a00640 1 -- 172.21.15.32:0/18274068 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27600b2070 con 0x7f276806c140 2024-04-17T10:00:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.881+0000 7f2766a00640 1 -- 172.21.15.32:0/18274068 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27600a59c0 con 0x7f276806c140 2024-04-17T10:00:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.881+0000 7f2766a00640 1 -- 172.21.15.32:0/18274068 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27600a5d90 con 0x7f276806c140 2024-04-17T10:00:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.881+0000 7f276e000640 1 -- 172.21.15.32:0/18274068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f276806c140 msgr2=0x7f27680ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.881+0000 7f276e000640 1 --2- 172.21.15.32:0/18274068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f276806c140 0x7f27680ce290 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2760009d90 tx=0x7f276001c8e0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.881+0000 7f276e000640 1 -- 172.21.15.32:0/18274068 shutdown_connections 2024-04-17T10:00:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.881+0000 7f276e000640 1 --2- 172.21.15.32:0/18274068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f276806c140 0x7f27680ce290 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.881+0000 7f276e000640 1 -- 172.21.15.32:0/18274068 >> 172.21.15.32:0/18274068 conn(0x7f27680b2120 msgr2=0x7f27680b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.881+0000 7f276e000640 1 -- 172.21.15.32:0/18274068 shutdown_connections 2024-04-17T10:00:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.881+0000 7f276e000640 1 -- 172.21.15.32:0/18274068 wait complete. 2024-04-17T10:00:41.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.881+0000 7f276e000640 1 Processor -- start 2024-04-17T10:00:41.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.881+0000 7f276e000640 1 -- start start 2024-04-17T10:00:41.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.881+0000 7f276e000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f276806c140 0x7f276815c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:41.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.881+0000 7f276e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276806f7c0 0x7f276813e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:41.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.881+0000 7f276e000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f276815b7d0 0x7f276813ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:41.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.881+0000 7f276e000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2768072270 con 0x7f276806f7c0 2024-04-17T10:00:41.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.881+0000 7f276e000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f27680720f0 con 0x7f276815b7d0 2024-04-17T10:00:41.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.881+0000 7f276e000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f27680723f0 con 0x7f276806c140 2024-04-17T10:00:41.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.881+0000 7f275ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276806f7c0 0x7f276813e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:41.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.881+0000 7f275ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276806f7c0 0x7f276813e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55152/0 (socket says 172.21.15.32:55152) 2024-04-17T10:00:41.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.881+0000 7f275ea00640 1 -- 172.21.15.32:0/2278063409 learned_addr learned my addr 172.21.15.32:0/2278063409 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:41.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.881+0000 7f2767400640 1 --2- 172.21.15.32:0/2278063409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f276806c140 0x7f276815c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:41.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.885+0000 7f2767e00640 1 --2- 172.21.15.32:0/2278063409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f276815b7d0 0x7f276813ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:41.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.885+0000 7f275ea00640 1 -- 172.21.15.32:0/2278063409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f276806c140 msgr2=0x7f276815c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:41.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.885+0000 7f275ea00640 1 --2- 172.21.15.32:0/2278063409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f276806c140 0x7f276815c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.885+0000 7f275ea00640 1 -- 172.21.15.32:0/2278063409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f276815b7d0 msgr2=0x7f276813ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:41.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.885+0000 7f275ea00640 1 --2- 172.21.15.32:0/2278063409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f276815b7d0 0x7f276813ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:41.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.885+0000 7f275ea00640 1 -- 172.21.15.32:0/2278063409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27680aed50 con 0x7f276806f7c0 2024-04-17T10:00:41.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.885+0000 7f275ea00640 1 --2- 172.21.15.32:0/2278063409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276806f7c0 0x7f276813e930 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f275000c420 tx=0x7f275000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:41.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.885+0000 7f275fe00640 1 -- 172.21.15.32:0/2278063409 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27500052d0 con 0x7f276806f7c0 2024-04-17T10:00:41.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.885+0000 7f275fe00640 1 -- 172.21.15.32:0/2278063409 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2750005470 con 0x7f276806f7c0 2024-04-17T10:00:41.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.885+0000 7f276e000640 1 -- 172.21.15.32:0/2278063409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27680af030 con 0x7f276806f7c0 2024-04-17T10:00:41.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.885+0000 7f275fe00640 1 -- 172.21.15.32:0/2278063409 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2750007260 con 0x7f276806f7c0 2024-04-17T10:00:41.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.885+0000 7f276e000640 1 -- 172.21.15.32:0/2278063409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2768156a00 con 0x7f276806f7c0 2024-04-17T10:00:41.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.885+0000 7f276e000640 1 -- 172.21.15.32:0/2278063409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2768068a90 con 0x7f276806f7c0 2024-04-17T10:00:41.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.885+0000 7f275fe00640 1 -- 172.21.15.32:0/2278063409 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2750007400 con 0x7f276806f7c0 2024-04-17T10:00:41.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.885+0000 7f275fe00640 1 --2- 172.21.15.32:0/2278063409 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2744068380 0x7f274406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:41.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.885+0000 7f275fe00640 1 -- 172.21.15.32:0/2278063409 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7f2750085e10 con 0x7f276806f7c0 2024-04-17T10:00:41.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.889+0000 7f2767400640 1 --2- 172.21.15.32:0/2278063409 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2744068380 0x7f274406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:41.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.889+0000 7f2767400640 1 --2- 172.21.15.32:0/2278063409 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2744068380 0x7f274406a840 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f27580011f0 tx=0x7f2758007d40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:41.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:41.889+0000 7f275fe00640 1 -- 172.21.15.32:0/2278063409 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2750055d80 con 0x7f276806f7c0 2024-04-17T10:00:42.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.053+0000 7f276e000640 1 -- 172.21.15.32:0/2278063409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f27680afcd0 con 0x7f276806f7c0 2024-04-17T10:00:42.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.057+0000 7f275fe00640 1 -- 172.21.15.32:0/2278063409 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) ==== 104+0+11 (secure 0 0 0) 0x7f275005a080 con 0x7f276806f7c0 2024-04-17T10:00:42.054 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T10:00:42.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.057+0000 7f276e000640 1 -- 172.21.15.32:0/2278063409 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2744068380 msgr2=0x7f274406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.057+0000 7f276e000640 1 --2- 172.21.15.32:0/2278063409 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2744068380 0x7f274406a840 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f27580011f0 tx=0x7f2758007d40 comp rx=0 tx=0).stop 2024-04-17T10:00:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.057+0000 7f276e000640 1 -- 172.21.15.32:0/2278063409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276806f7c0 msgr2=0x7f276813e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.057+0000 7f276e000640 1 --2- 172.21.15.32:0/2278063409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276806f7c0 0x7f276813e930 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f275000c420 tx=0x7f275000c8e0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.057+0000 7f276e000640 1 -- 172.21.15.32:0/2278063409 shutdown_connections 2024-04-17T10:00:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.057+0000 7f276e000640 1 --2- 172.21.15.32:0/2278063409 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2744068380 0x7f274406a840 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.057+0000 7f276e000640 1 --2- 172.21.15.32:0/2278063409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f276815b7d0 0x7f276813ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.057+0000 7f276e000640 1 --2- 172.21.15.32:0/2278063409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276806f7c0 0x7f276813e930 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.057+0000 7f276e000640 1 --2- 172.21.15.32:0/2278063409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f276806c140 0x7f276815c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.057+0000 7f276e000640 1 -- 172.21.15.32:0/2278063409 >> 172.21.15.32:0/2278063409 conn(0x7f27680b2120 msgr2=0x7f27680cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.057+0000 7f276e000640 1 -- 172.21.15.32:0/2278063409 shutdown_connections 2024-04-17T10:00:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.057+0000 7f276e000640 1 -- 172.21.15.32:0/2278063409 wait complete. 2024-04-17T10:00:42.075 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:00:42.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.221+0000 7f32d4e00640 1 Processor -- start 2024-04-17T10:00:42.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.221+0000 7f32d4e00640 1 -- start start 2024-04-17T10:00:42.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.221+0000 7f32d4e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f32d006ba30 0x7f32d006be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:42.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.221+0000 7f32d4e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32d006c350 con 0x7f32d0071e60 2024-04-17T10:00:42.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.221+0000 7f32d4e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32d00d0130 con 0x7f32d00cf370 2024-04-17T10:00:42.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.221+0000 7f32d4e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f32d00cfef0 con 0x7f32d006ba30 2024-04-17T10:00:42.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.221+0000 7f32ce800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f32d0071e60 0x7f32d0072240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44792/0 (socket says 172.21.15.32:44792) 2024-04-17T10:00:42.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.221+0000 7f32ce800640 1 -- 172.21.15.32:0/3105867893 learned_addr learned my addr 172.21.15.32:0/3105867893 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:00:42.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.221+0000 7f32cde00640 1 --2- 172.21.15.32:0/3105867893 >> v2:172.21.15.144:3300/0 conn(0x7f32d006ba30 0x7f32d006be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:42.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.221+0000 7f32cd400640 1 -- 172.21.15.32:0/3105867893 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1546534220 0 0) 0x7f32d006c350 con 0x7f32d0071e60 2024-04-17T10:00:42.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.221+0000 7f32cd400640 1 -- 172.21.15.32:0/3105867893 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32b4003530 con 0x7f32d0071e60 2024-04-17T10:00:42.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.225+0000 7f32cde00640 1 -- 172.21.15.32:0/3105867893 >> v1:172.21.15.144:6789/0 conn(0x7f32d00cf370 legacy=0x7f32d00cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:42.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.225+0000 7f32cd400640 1 -- 172.21.15.32:0/3105867893 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2074419472 0 0) 0x7f32d00d0130 con 0x7f32d00cf370 2024-04-17T10:00:42.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.225+0000 7f32cde00640 1 -- 172.21.15.32:0/3105867893 >> v1:172.21.15.32:6789/0 conn(0x7f32d0071e60 legacy=0x7f32d0072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:42.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.225+0000 7f32cde00640 1 -- 172.21.15.32:0/3105867893 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d015d090 con 0x7f32d006ba30 2024-04-17T10:00:42.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.225+0000 7f32cde00640 1 --2- 172.21.15.32:0/3105867893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32d006ba30 0x7f32d006be10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f32c00097c0 tx=0x7f32c001c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caba1488524ee7bf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:42.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.229+0000 7f32cd400640 1 -- 172.21.15.32:0/3105867893 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c00b1070 con 0x7f32d006ba30 2024-04-17T10:00:42.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.229+0000 7f32cd400640 1 -- 172.21.15.32:0/3105867893 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32c00a8050 con 0x7f32d006ba30 2024-04-17T10:00:42.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.229+0000 7f32cd400640 1 -- 172.21.15.32:0/3105867893 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c00ac860 con 0x7f32d006ba30 2024-04-17T10:00:42.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.229+0000 7f32d4e00640 1 -- 172.21.15.32:0/3105867893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32d006ba30 msgr2=0x7f32d006be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:42.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.229+0000 7f32d4e00640 1 --2- 172.21.15.32:0/3105867893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32d006ba30 0x7f32d006be10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f32c00097c0 tx=0x7f32c001c3e0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.229+0000 7f32d4e00640 1 -- 172.21.15.32:0/3105867893 shutdown_connections 2024-04-17T10:00:42.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.229+0000 7f32d4e00640 1 --2- 172.21.15.32:0/3105867893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32d006ba30 0x7f32d006be10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.229+0000 7f32d4e00640 1 -- 172.21.15.32:0/3105867893 >> 172.21.15.32:0/3105867893 conn(0x7f32d00b2120 msgr2=0x7f32d00b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:42.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.229+0000 7f32d4e00640 1 -- 172.21.15.32:0/3105867893 shutdown_connections 2024-04-17T10:00:42.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.229+0000 7f32d4e00640 1 -- 172.21.15.32:0/3105867893 wait complete. 2024-04-17T10:00:42.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32d4e00640 1 Processor -- start 2024-04-17T10:00:42.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32d4e00640 1 -- start start 2024-04-17T10:00:42.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32d4e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32d006ba30 0x7f32d0140720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:42.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32d4e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32d0071e60 0x7f32d00b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:42.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32d4e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32d00cf370 0x7f32d00b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:42.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32d4e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f32d015db70 con 0x7f32d00cf370 2024-04-17T10:00:42.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32d4e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f32d015cd00 con 0x7f32d006ba30 2024-04-17T10:00:42.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32d4e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f32d015dcf0 con 0x7f32d0071e60 2024-04-17T10:00:42.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32ce800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32d00cf370 0x7f32d00b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:42.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32ce800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32d00cf370 0x7f32d00b3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55162/0 (socket says 172.21.15.32:55162) 2024-04-17T10:00:42.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32ce800640 1 -- 172.21.15.32:0/1121980925 learned_addr learned my addr 172.21.15.32:0/1121980925 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:42.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32c5400640 1 --2- 172.21.15.32:0/1121980925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32d0071e60 0x7f32d00b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:42.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32cde00640 1 --2- 172.21.15.32:0/1121980925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32d006ba30 0x7f32d0140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:42.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32ce800640 1 -- 172.21.15.32:0/1121980925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32d0071e60 msgr2=0x7f32d00b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:42.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32ce800640 1 --2- 172.21.15.32:0/1121980925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32d0071e60 0x7f32d00b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32ce800640 1 -- 172.21.15.32:0/1121980925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32d006ba30 msgr2=0x7f32d0140720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:42.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32ce800640 1 --2- 172.21.15.32:0/1121980925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32d006ba30 0x7f32d0140720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32ce800640 1 -- 172.21.15.32:0/1121980925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d00b3210 con 0x7f32d00cf370 2024-04-17T10:00:42.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32ce800640 1 --2- 172.21.15.32:0/1121980925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32d00cf370 0x7f32d00b3cb0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f32c800cb40 tx=0x7f32c800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:42.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32c6a00640 1 -- 172.21.15.32:0/1121980925 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8017070 con 0x7f32d00cf370 2024-04-17T10:00:42.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32c6a00640 1 -- 172.21.15.32:0/1121980925 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32c800aca0 con 0x7f32d00cf370 2024-04-17T10:00:42.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32c6a00640 1 -- 172.21.15.32:0/1121980925 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8012620 con 0x7f32d00cf370 2024-04-17T10:00:42.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32d4e00640 1 -- 172.21.15.32:0/1121980925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d00b4f40 con 0x7f32d00cf370 2024-04-17T10:00:42.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.233+0000 7f32d4e00640 1 -- 172.21.15.32:0/1121980925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d00b34b0 con 0x7f32d00cf370 2024-04-17T10:00:42.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.237+0000 7f32d4e00640 1 -- 172.21.15.32:0/1121980925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32d0068a90 con 0x7f32d00cf370 2024-04-17T10:00:42.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.237+0000 7f32c6a00640 1 -- 172.21.15.32:0/1121980925 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f32c801a070 con 0x7f32d00cf370 2024-04-17T10:00:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.237+0000 7f32c6a00640 1 --2- 172.21.15.32:0/1121980925 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f32ac068380 0x7f32ac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.237+0000 7f32c6a00640 1 -- 172.21.15.32:0/1121980925 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7f32c8086830 con 0x7f32d00cf370 2024-04-17T10:00:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.237+0000 7f32cde00640 1 --2- 172.21.15.32:0/1121980925 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f32ac068380 0x7f32ac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.237+0000 7f32cde00640 1 --2- 172.21.15.32:0/1121980925 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f32ac068380 0x7f32ac06a840 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f32c00ae000 tx=0x7f32c00a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:42.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.241+0000 7f32c6a00640 1 -- 172.21.15.32:0/1121980925 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f32c8056670 con 0x7f32d00cf370 2024-04-17T10:00:42.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.405+0000 7f32d4e00640 1 -- 172.21.15.32:0/1121980925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f32d006c850 con 0x7f32d00cf370 2024-04-17T10:00:42.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.405+0000 7f32c6a00640 1 -- 172.21.15.32:0/1121980925 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) ==== 100+0+11 (secure 0 0 0) 0x7f32c805a970 con 0x7f32d00cf370 2024-04-17T10:00:42.404 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T10:00:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.409+0000 7f32d4e00640 1 -- 172.21.15.32:0/1121980925 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f32ac068380 msgr2=0x7f32ac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.409+0000 7f32d4e00640 1 --2- 172.21.15.32:0/1121980925 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f32ac068380 0x7f32ac06a840 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f32c00ae000 tx=0x7f32c00a7000 comp rx=0 tx=0).stop 2024-04-17T10:00:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.409+0000 7f32d4e00640 1 -- 172.21.15.32:0/1121980925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32d00cf370 msgr2=0x7f32d00b3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.409+0000 7f32d4e00640 1 --2- 172.21.15.32:0/1121980925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32d00cf370 0x7f32d00b3cb0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f32c800cb40 tx=0x7f32c800a060 comp rx=0 tx=0).stop 2024-04-17T10:00:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.409+0000 7f32d4e00640 1 -- 172.21.15.32:0/1121980925 shutdown_connections 2024-04-17T10:00:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.409+0000 7f32d4e00640 1 --2- 172.21.15.32:0/1121980925 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f32ac068380 0x7f32ac06a840 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.409+0000 7f32d4e00640 1 --2- 172.21.15.32:0/1121980925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32d0071e60 0x7f32d00b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.409+0000 7f32d4e00640 1 --2- 172.21.15.32:0/1121980925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32d00cf370 0x7f32d00b3cb0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.409+0000 7f32d4e00640 1 --2- 172.21.15.32:0/1121980925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32d006ba30 0x7f32d0140720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.409+0000 7f32d4e00640 1 -- 172.21.15.32:0/1121980925 >> 172.21.15.32:0/1121980925 conn(0x7f32d00b2120 msgr2=0x7f32d00b08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.409+0000 7f32d4e00640 1 -- 172.21.15.32:0/1121980925 shutdown_connections 2024-04-17T10:00:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.409+0000 7f32d4e00640 1 -- 172.21.15.32:0/1121980925 wait complete. 2024-04-17T10:00:42.427 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:00:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.573+0000 7f8784600640 1 Processor -- start 2024-04-17T10:00:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.573+0000 7f8784600640 1 -- start start 2024-04-17T10:00:42.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.573+0000 7f8784600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f877c13f6e0 0x7f877c0cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:42.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.573+0000 7f8784600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f877c0cf370 0x7f877c0d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:42.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.573+0000 7f8784600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f877c1601e0 con 0x7f877c0d0820 2024-04-17T10:00:42.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.573+0000 7f8784600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f877c13fac0 con 0x7f877c13f6e0 2024-04-17T10:00:42.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.573+0000 7f8784600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f877c160480 con 0x7f877c0cf370 2024-04-17T10:00:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.573+0000 7f8781200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f877c0cf370 0x7f877c0d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.573+0000 7f8781c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f877c13f6e0 0x7f877c0cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.573+0000 7f8781200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f877c0cf370 0x7f877c0d02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:46698/0 (socket says 172.21.15.32:46698) 2024-04-17T10:00:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.573+0000 7f8781200640 1 -- 172.21.15.32:0/2953583181 learned_addr learned my addr 172.21.15.32:0/2953583181 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.573+0000 7f8781c00640 1 -- 172.21.15.32:0/2953583181 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f877c0cf370 msgr2=0x7f877c0d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.573+0000 7f8781c00640 1 --2- 172.21.15.32:0/2953583181 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f877c0cf370 0x7f877c0d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.573+0000 7f8781c00640 1 -- 172.21.15.32:0/2953583181 >> v1:172.21.15.144:6789/0 conn(0x7f877c0d0820 legacy=0x7f877c141d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.573+0000 7f8781c00640 1 -- 172.21.15.32:0/2953583181 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f877c160760 con 0x7f877c13f6e0 2024-04-17T10:00:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.573+0000 7f8781200640 1 --2- 172.21.15.32:0/2953583181 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f877c0cf370 0x7f877c0d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:00:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.577+0000 7f8781c00640 1 --2- 172.21.15.32:0/2953583181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f877c13f6e0 0x7f877c0cf850 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f87700097c0 tx=0x7f877001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9050916971b56492 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:42.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.577+0000 7f877be00640 1 -- 172.21.15.32:0/2953583181 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87700b0070 con 0x7f877c13f6e0 2024-04-17T10:00:42.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.577+0000 7f877be00640 1 -- 172.21.15.32:0/2953583181 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87700ab600 con 0x7f877c13f6e0 2024-04-17T10:00:42.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.577+0000 7f877be00640 1 -- 172.21.15.32:0/2953583181 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87700ab9d0 con 0x7f877c13f6e0 2024-04-17T10:00:42.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.577+0000 7f8784600640 1 -- 172.21.15.32:0/2953583181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f877c13f6e0 msgr2=0x7f877c0cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:42.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.577+0000 7f8784600640 1 --2- 172.21.15.32:0/2953583181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f877c13f6e0 0x7f877c0cf850 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f87700097c0 tx=0x7f877001c4a0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.577+0000 7f8784600640 1 -- 172.21.15.32:0/2953583181 shutdown_connections 2024-04-17T10:00:42.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.577+0000 7f8784600640 1 --2- 172.21.15.32:0/2953583181 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f877c0cf370 0x7f877c0d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.577+0000 7f8784600640 1 --2- 172.21.15.32:0/2953583181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f877c13f6e0 0x7f877c0cf850 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.577+0000 7f8784600640 1 -- 172.21.15.32:0/2953583181 >> 172.21.15.32:0/2953583181 conn(0x7f877c06ba30 msgr2=0x7f877c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:42.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.577+0000 7f8784600640 1 -- 172.21.15.32:0/2953583181 shutdown_connections 2024-04-17T10:00:42.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.577+0000 7f8784600640 1 -- 172.21.15.32:0/2953583181 wait complete. 2024-04-17T10:00:42.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8784600640 1 Processor -- start 2024-04-17T10:00:42.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8784600640 1 -- start start 2024-04-17T10:00:42.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8784600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f877c0cf370 0x7f877c13dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:42.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8784600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f877c0d0820 0x7f877c144de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:42.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8784600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f877c13f6e0 0x7f877c145320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:42.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8784600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f877c0cff10 con 0x7f877c0cf370 2024-04-17T10:00:42.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8784600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f877c0cfd90 con 0x7f877c0d0820 2024-04-17T10:00:42.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8784600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f877c169580 con 0x7f877c13f6e0 2024-04-17T10:00:42.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8781c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f877c0cf370 0x7f877c13dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:42.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8781c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f877c0cf370 0x7f877c13dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55178/0 (socket says 172.21.15.32:55178) 2024-04-17T10:00:42.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8781c00640 1 -- 172.21.15.32:0/1140016677 learned_addr learned my addr 172.21.15.32:0/1140016677 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:42.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8781200640 1 --2- 172.21.15.32:0/1140016677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f877c0d0820 0x7f877c144de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:42.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8782600640 1 --2- 172.21.15.32:0/1140016677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f877c13f6e0 0x7f877c145320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:42.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8781c00640 1 -- 172.21.15.32:0/1140016677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f877c13f6e0 msgr2=0x7f877c145320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8781c00640 1 --2- 172.21.15.32:0/1140016677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f877c13f6e0 0x7f877c145320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8781c00640 1 -- 172.21.15.32:0/1140016677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f877c0d0820 msgr2=0x7f877c144de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8781c00640 1 --2- 172.21.15.32:0/1140016677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f877c0d0820 0x7f877c144de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8781c00640 1 -- 172.21.15.32:0/1140016677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f877c142e80 con 0x7f877c0cf370 2024-04-17T10:00:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8781200640 1 --2- 172.21.15.32:0/1140016677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f877c0d0820 0x7f877c144de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:00:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8782600640 1 --2- 172.21.15.32:0/1140016677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f877c13f6e0 0x7f877c145320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:00:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8781c00640 1 --2- 172.21.15.32:0/1140016677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f877c0cf370 0x7f877c13dee0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f877001a0d0 tx=0x7f877001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8779600640 1 -- 172.21.15.32:0/1140016677 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87700b0050 con 0x7f877c0cf370 2024-04-17T10:00:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8779600640 1 -- 172.21.15.32:0/1140016677 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87700bc070 con 0x7f877c0cf370 2024-04-17T10:00:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8784600640 1 -- 172.21.15.32:0/1140016677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f877c143120 con 0x7f877c0cf370 2024-04-17T10:00:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8779600640 1 -- 172.21.15.32:0/1140016677 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87700b5830 con 0x7f877c0cf370 2024-04-17T10:00:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.581+0000 7f8784600640 1 -- 172.21.15.32:0/1140016677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f877c154280 con 0x7f877c0cf370 2024-04-17T10:00:42.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.585+0000 7f8784600640 1 -- 172.21.15.32:0/1140016677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f877c068a90 con 0x7f877c0cf370 2024-04-17T10:00:42.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.585+0000 7f8779600640 1 -- 172.21.15.32:0/1140016677 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f87700b59d0 con 0x7f877c0cf370 2024-04-17T10:00:42.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.585+0000 7f8779600640 1 --2- 172.21.15.32:0/1140016677 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f875c068380 0x7f875c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:42.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.585+0000 7f8779600640 1 -- 172.21.15.32:0/1140016677 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7f87701212c0 con 0x7f877c0cf370 2024-04-17T10:00:42.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.585+0000 7f8781200640 1 --2- 172.21.15.32:0/1140016677 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f875c068380 0x7f875c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:42.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.585+0000 7f8781200640 1 --2- 172.21.15.32:0/1140016677 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f875c068380 0x7f875c06a840 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f877c0cfb20 tx=0x7f8768008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:42.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.589+0000 7f8779600640 1 -- 172.21.15.32:0/1140016677 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f8770016090 con 0x7f877c0cf370 2024-04-17T10:00:42.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.765+0000 7f8784600640 1 -- 172.21.15.32:0/1140016677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f877c141800 con 0x7f877c0cf370 2024-04-17T10:00:42.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.765+0000 7f8779600640 1 -- 172.21.15.32:0/1140016677 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v85) ==== 80+0+122 (secure 0 0 0) 0x7f87700f1870 con 0x7f877c0cf370 2024-04-17T10:00:42.765 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6837/2656735817 2024-04-18T09:59:28.985107+0000 2024-04-17T10:00:42.765 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6835/2656735817 2024-04-18T09:59:28.985107+0000 2024-04-17T10:00:42.765 INFO:teuthology.orchestra.run.smithi032.stderr:listed 2 entries 2024-04-17T10:00:42.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.769+0000 7f8784600640 1 -- 172.21.15.32:0/1140016677 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f875c068380 msgr2=0x7f875c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:42.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.769+0000 7f8784600640 1 --2- 172.21.15.32:0/1140016677 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f875c068380 0x7f875c06a840 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f877c0cfb20 tx=0x7f8768008040 comp rx=0 tx=0).stop 2024-04-17T10:00:42.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.769+0000 7f8784600640 1 -- 172.21.15.32:0/1140016677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f877c0cf370 msgr2=0x7f877c13dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:42.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.769+0000 7f8784600640 1 --2- 172.21.15.32:0/1140016677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f877c0cf370 0x7f877c13dee0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f877001a0d0 tx=0x7f877001ace0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.769+0000 7f8784600640 1 -- 172.21.15.32:0/1140016677 shutdown_connections 2024-04-17T10:00:42.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.769+0000 7f8784600640 1 --2- 172.21.15.32:0/1140016677 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f875c068380 0x7f875c06a840 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.769+0000 7f8784600640 1 --2- 172.21.15.32:0/1140016677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f877c13f6e0 0x7f877c145320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.769+0000 7f8784600640 1 --2- 172.21.15.32:0/1140016677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f877c0d0820 0x7f877c144de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.769+0000 7f8784600640 1 --2- 172.21.15.32:0/1140016677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f877c0cf370 0x7f877c13dee0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:42.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.769+0000 7f8784600640 1 -- 172.21.15.32:0/1140016677 >> 172.21.15.32:0/1140016677 conn(0x7f877c06ba30 msgr2=0x7f877c06f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:42.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.769+0000 7f8784600640 1 -- 172.21.15.32:0/1140016677 shutdown_connections 2024-04-17T10:00:42.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:42.769+0000 7f8784600640 1 -- 172.21.15.32:0/1140016677 wait complete. 2024-04-17T10:00:42.786 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2024-04-17T10:00:42.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:00:42.786 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T10:00:42.804 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[45664]: fuse finished with error 0 and tester_r 0 2024-04-17T10:00:42.807 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:00:42.870 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T10:00:42.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:00:42.871 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T10:00:42.880 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:00:43.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.065+0000 7f83bfc00640 1 Processor -- start 2024-04-17T10:00:43.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.065+0000 7f83bfc00640 1 -- start start 2024-04-17T10:00:43.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.065+0000 7f83bfc00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f83b80d00b0 0x7f83b809a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.065+0000 7f83bfc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b815c9a0 0x7f83b809af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.065+0000 7f83bfc00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83b80d05e0 con 0x7f83b815c5c0 2024-04-17T10:00:43.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.065+0000 7f83bfc00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f83b80ce560 con 0x7f83b80d00b0 2024-04-17T10:00:43.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.065+0000 7f83bfc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f83b80d11f0 con 0x7f83b815c9a0 2024-04-17T10:00:43.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.065+0000 7f83bd200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f83b815c5c0 0x7f83b80cde50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44804/0 (socket says 172.21.15.32:44804) 2024-04-17T10:00:43.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.065+0000 7f83bd200640 1 -- 172.21.15.32:0/3780317054 learned_addr learned my addr 172.21.15.32:0/3780317054 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:00:43.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.065+0000 7f83afe00640 1 --2- 172.21.15.32:0/3780317054 >> v2:172.21.15.144:3300/0 conn(0x7f83b80d00b0 0x7f83b809a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.065+0000 7f83bdc00640 1 --2- 172.21.15.32:0/3780317054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b815c9a0 0x7f83b809af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.065+0000 7f83b7e00640 1 -- 172.21.15.32:0/3780317054 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1703764363 0 0) 0x7f83b80d05e0 con 0x7f83b815c5c0 2024-04-17T10:00:43.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.065+0000 7f83b7e00640 1 -- 172.21.15.32:0/3780317054 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f839c003530 con 0x7f83b815c5c0 2024-04-17T10:00:43.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.065+0000 7f83afe00640 1 -- 172.21.15.32:0/3780317054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b815c9a0 msgr2=0x7f83b809af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:43.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.065+0000 7f83afe00640 1 --2- 172.21.15.32:0/3780317054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b815c9a0 0x7f83b809af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.065+0000 7f83b7e00640 1 -- 172.21.15.32:0/3780317054 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2350494836 0 0) 0x7f839c003530 con 0x7f83b815c5c0 2024-04-17T10:00:43.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.065+0000 7f83afe00640 1 -- 172.21.15.32:0/3780317054 >> v1:172.21.15.32:6789/0 conn(0x7f83b815c5c0 legacy=0x7f83b80cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.065+0000 7f83afe00640 1 -- 172.21.15.32:0/3780317054 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b80d08c0 con 0x7f83b80d00b0 2024-04-17T10:00:43.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.069+0000 7f83afe00640 1 --2- 172.21.15.32:0/3780317054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b80d00b0 0x7f83b809a9f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f83a00097c0 tx=0x7f83a001c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee7b075ae4577810 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:43.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.069+0000 7f83b7e00640 1 -- 172.21.15.32:0/3780317054 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a00b0070 con 0x7f83b80d00b0 2024-04-17T10:00:43.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.069+0000 7f83b7e00640 1 -- 172.21.15.32:0/3780317054 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a00ab400 con 0x7f83b80d00b0 2024-04-17T10:00:43.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.069+0000 7f83b7e00640 1 -- 172.21.15.32:0/3780317054 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a8001680 con 0x7f83b80d00b0 2024-04-17T10:00:43.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.069+0000 7f83bfc00640 1 -- 172.21.15.32:0/3780317054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b80d00b0 msgr2=0x7f83b809a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.069+0000 7f83bfc00640 1 --2- 172.21.15.32:0/3780317054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b80d00b0 0x7f83b809a9f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f83a00097c0 tx=0x7f83a001c390 comp rx=0 tx=0).stop 2024-04-17T10:00:43.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.069+0000 7f83bfc00640 1 -- 172.21.15.32:0/3780317054 shutdown_connections 2024-04-17T10:00:43.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.069+0000 7f83bfc00640 1 --2- 172.21.15.32:0/3780317054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b815c9a0 0x7f83b809af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.069+0000 7f83bfc00640 1 --2- 172.21.15.32:0/3780317054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b80d00b0 0x7f83b809a9f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.069+0000 7f83bfc00640 1 -- 172.21.15.32:0/3780317054 >> 172.21.15.32:0/3780317054 conn(0x7f83b80b3fd0 msgr2=0x7f83b80b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:43.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.069+0000 7f83bfc00640 1 -- 172.21.15.32:0/3780317054 shutdown_connections 2024-04-17T10:00:43.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.069+0000 7f83bfc00640 1 -- 172.21.15.32:0/3780317054 wait complete. 2024-04-17T10:00:43.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.073+0000 7f83bfc00640 1 Processor -- start 2024-04-17T10:00:43.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.073+0000 7f83bfc00640 1 -- start start 2024-04-17T10:00:43.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.073+0000 7f83bfc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b80d00b0 0x7f83b80b6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.073+0000 7f83bfc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b815c5c0 0x7f83b814d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.073+0000 7f83bfc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b815c9a0 0x7f83b814cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.073+0000 7f83bfc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f83b80b3dd0 con 0x7f83b815c5c0 2024-04-17T10:00:43.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.073+0000 7f83bfc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f83b80b3c50 con 0x7f83b815c9a0 2024-04-17T10:00:43.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.073+0000 7f83bfc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f83b80b5330 con 0x7f83b80d00b0 2024-04-17T10:00:43.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.073+0000 7f83afe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b815c5c0 0x7f83b814d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.073+0000 7f83afe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b815c5c0 0x7f83b814d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55190/0 (socket says 172.21.15.32:55190) 2024-04-17T10:00:43.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.073+0000 7f83afe00640 1 -- 172.21.15.32:0/639545481 learned_addr learned my addr 172.21.15.32:0/639545481 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:43.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.073+0000 7f83bd200640 1 --2- 172.21.15.32:0/639545481 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b80d00b0 0x7f83b80b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.073+0000 7f83bdc00640 1 --2- 172.21.15.32:0/639545481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b815c9a0 0x7f83b814cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.073+0000 7f83afe00640 1 -- 172.21.15.32:0/639545481 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b80d00b0 msgr2=0x7f83b80b6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.073+0000 7f83afe00640 1 --2- 172.21.15.32:0/639545481 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b80d00b0 0x7f83b80b6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.073+0000 7f83afe00640 1 -- 172.21.15.32:0/639545481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b815c9a0 msgr2=0x7f83b814cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.073+0000 7f83afe00640 1 --2- 172.21.15.32:0/639545481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b815c9a0 0x7f83b814cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.073+0000 7f83afe00640 1 -- 172.21.15.32:0/639545481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b80b5790 con 0x7f83b815c5c0 2024-04-17T10:00:43.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.073+0000 7f83afe00640 1 --2- 172.21.15.32:0/639545481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b815c5c0 0x7f83b814d900 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f83a001aa40 tx=0x7f83a001abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:43.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.073+0000 7f83b5600640 1 -- 172.21.15.32:0/639545481 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a00b0050 con 0x7f83b815c5c0 2024-04-17T10:00:43.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.077+0000 7f83b5600640 1 -- 172.21.15.32:0/639545481 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a00bc070 con 0x7f83b815c5c0 2024-04-17T10:00:43.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.077+0000 7f83b5600640 1 -- 172.21.15.32:0/639545481 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a00b78b0 con 0x7f83b815c5c0 2024-04-17T10:00:43.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.077+0000 7f83bfc00640 1 -- 172.21.15.32:0/639545481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b80b5a70 con 0x7f83b815c5c0 2024-04-17T10:00:43.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.077+0000 7f83bfc00640 1 -- 172.21.15.32:0/639545481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b80b6030 con 0x7f83b815c5c0 2024-04-17T10:00:43.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.077+0000 7f83bfc00640 1 -- 172.21.15.32:0/639545481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83b8068a90 con 0x7f83b815c5c0 2024-04-17T10:00:43.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.077+0000 7f83b5600640 1 -- 172.21.15.32:0/639545481 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f83a00b7a50 con 0x7f83b815c5c0 2024-04-17T10:00:43.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.077+0000 7f83b5600640 1 --2- 172.21.15.32:0/639545481 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8394068380 0x7f839406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.077+0000 7f83b5600640 1 -- 172.21.15.32:0/639545481 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7f83a0121100 con 0x7f83b815c5c0 2024-04-17T10:00:43.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.077+0000 7f83bd200640 1 --2- 172.21.15.32:0/639545481 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8394068380 0x7f839406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.077+0000 7f83bd200640 1 --2- 172.21.15.32:0/639545481 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8394068380 0x7f839406a840 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f83a80013d0 tx=0x7f83a8001360 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:43.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.081+0000 7f83b5600640 1 -- 172.21.15.32:0/639545481 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f83a0015fb0 con 0x7f83b815c5c0 2024-04-17T10:00:43.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.285+0000 7f83bfc00640 1 -- 172.21.15.32:0/639545481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83b81542d0 con 0x7f83b815c5c0 2024-04-17T10:00:43.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.285+0000 7f83b5600640 1 -- 172.21.15.32:0/639545481 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5205 (secure 0 0 0) 0x7f83a00f0f40 con 0x7f83b815c5c0 2024-04-17T10:00:43.284 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:00:43.284 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6419,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3534108103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3534108103},{"type":"v1","addr":"172.21.15.144:6835","nonce":3534108103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7003,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/935945019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":935945019},{"type":"v1","addr":"172.21.15.144:6834","nonce":935945019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3044362393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3044362393},{"type":"v1","addr":"172.21.15.32:6836","nonce":3044362393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:59:04.834110+0000","modified":"2024-04-17T09:59:40.007746+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":6,"addr":"172.21.15.32:6837/1253497636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1253497636},{"type":"v1","addr":"172.21.15.32:6837","nonce":1253497636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7851,"qdb_cluster":[7851]},"id":6}]} 2024-04-17T10:00:43.284 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 66 2024-04-17T10:00:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.289+0000 7f83bfc00640 1 -- 172.21.15.32:0/639545481 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8394068380 msgr2=0x7f839406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.289+0000 7f83bfc00640 1 --2- 172.21.15.32:0/639545481 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8394068380 0x7f839406a840 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f83a80013d0 tx=0x7f83a8001360 comp rx=0 tx=0).stop 2024-04-17T10:00:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.289+0000 7f83bfc00640 1 -- 172.21.15.32:0/639545481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b815c5c0 msgr2=0x7f83b814d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.289+0000 7f83bfc00640 1 --2- 172.21.15.32:0/639545481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b815c5c0 0x7f83b814d900 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f83a001aa40 tx=0x7f83a001abd0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.289+0000 7f83bfc00640 1 -- 172.21.15.32:0/639545481 shutdown_connections 2024-04-17T10:00:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.289+0000 7f83bfc00640 1 --2- 172.21.15.32:0/639545481 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8394068380 0x7f839406a840 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.289+0000 7f83bfc00640 1 --2- 172.21.15.32:0/639545481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b815c9a0 0x7f83b814cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.289+0000 7f83bfc00640 1 --2- 172.21.15.32:0/639545481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b815c5c0 0x7f83b814d900 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.289+0000 7f83bfc00640 1 --2- 172.21.15.32:0/639545481 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b80d00b0 0x7f83b80b6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.289+0000 7f83bfc00640 1 -- 172.21.15.32:0/639545481 >> 172.21.15.32:0/639545481 conn(0x7f83b80b3fd0 msgr2=0x7f83b81547f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:43.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.289+0000 7f83bfc00640 1 -- 172.21.15.32:0/639545481 shutdown_connections 2024-04-17T10:00:43.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.289+0000 7f83bfc00640 1 -- 172.21.15.32:0/639545481 wait complete. 2024-04-17T10:00:43.305 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-17T10:00:43.479 INFO:teuthology.orchestra.run.smithi032.stdout:{"message":"","return_code":0} 2024-04-17T10:00:43.479 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-17T10:00:43.479 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T10:00:43.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.621+0000 7f6649c00640 1 Processor -- start 2024-04-17T10:00:43.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.621+0000 7f6649c00640 1 -- start start 2024-04-17T10:00:43.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.625+0000 7f6649c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f664413f6e0 0x7f66440cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.625+0000 7f6649c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66440cf370 0x7f66440d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.625+0000 7f6649c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66441601e0 con 0x7f66440d0820 2024-04-17T10:00:43.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.625+0000 7f6649c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f664413fac0 con 0x7f664413f6e0 2024-04-17T10:00:43.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.625+0000 7f6649c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6644160480 con 0x7f66440cf370 2024-04-17T10:00:43.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.625+0000 7f6642a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f664413f6e0 0x7f66440cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.625+0000 7f6642a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f664413f6e0 0x7f66440cf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55194/0 (socket says 172.21.15.32:55194) 2024-04-17T10:00:43.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.625+0000 7f6642a00640 1 -- 172.21.15.32:0/160242280 learned_addr learned my addr 172.21.15.32:0/160242280 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:43.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.625+0000 7f663a000640 1 --2- 172.21.15.32:0/160242280 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66440cf370 0x7f66440d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.625+0000 7f6642a00640 1 -- 172.21.15.32:0/160242280 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66440cf370 msgr2=0x7f66440d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:43.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.625+0000 7f6642a00640 1 --2- 172.21.15.32:0/160242280 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66440cf370 0x7f66440d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.625+0000 7f6642a00640 1 -- 172.21.15.32:0/160242280 >> v1:172.21.15.144:6789/0 conn(0x7f66440d0820 legacy=0x7f6644141d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:43.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.625+0000 7f6642a00640 1 -- 172.21.15.32:0/160242280 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6644160760 con 0x7f664413f6e0 2024-04-17T10:00:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.629+0000 7f6642a00640 1 --2- 172.21.15.32:0/160242280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f664413f6e0 0x7f66440cf850 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f6634009e50 tx=0x7f663401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59cab218ade5340 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.629+0000 7f6642000640 1 -- 172.21.15.32:0/160242280 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66340a5670 con 0x7f664413f6e0 2024-04-17T10:00:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.629+0000 7f6642000640 1 -- 172.21.15.32:0/160242280 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66340a5810 con 0x7f664413f6e0 2024-04-17T10:00:43.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.629+0000 7f6642000640 1 -- 172.21.15.32:0/160242280 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66340a5be0 con 0x7f664413f6e0 2024-04-17T10:00:43.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.629+0000 7f6649c00640 1 -- 172.21.15.32:0/160242280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f664413f6e0 msgr2=0x7f66440cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.629+0000 7f6649c00640 1 --2- 172.21.15.32:0/160242280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f664413f6e0 0x7f66440cf850 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f6634009e50 tx=0x7f663401c9f0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.629+0000 7f6649c00640 1 -- 172.21.15.32:0/160242280 shutdown_connections 2024-04-17T10:00:43.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.629+0000 7f6649c00640 1 --2- 172.21.15.32:0/160242280 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66440cf370 0x7f66440d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.629+0000 7f6649c00640 1 --2- 172.21.15.32:0/160242280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f664413f6e0 0x7f66440cf850 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.629+0000 7f6649c00640 1 -- 172.21.15.32:0/160242280 >> 172.21.15.32:0/160242280 conn(0x7f664406ba30 msgr2=0x7f664406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:43.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.629+0000 7f6649c00640 1 -- 172.21.15.32:0/160242280 shutdown_connections 2024-04-17T10:00:43.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.629+0000 7f6649c00640 1 -- 172.21.15.32:0/160242280 wait complete. 2024-04-17T10:00:43.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.629+0000 7f6649c00640 1 Processor -- start 2024-04-17T10:00:43.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.629+0000 7f6649c00640 1 -- start start 2024-04-17T10:00:43.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.629+0000 7f6649c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66440cf370 0x7f664415c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.629+0000 7f6649c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66440d0820 0x7f664413e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.629+0000 7f6649c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f664413f6e0 0x7f6644158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.629+0000 7f6649c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f66440cff10 con 0x7f66440cf370 2024-04-17T10:00:43.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.629+0000 7f6649c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f66440cfd90 con 0x7f66440d0820 2024-04-17T10:00:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.629+0000 7f6649c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6644169580 con 0x7f664413f6e0 2024-04-17T10:00:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.633+0000 7f6642a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66440cf370 0x7f664415c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.633+0000 7f6642a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66440cf370 0x7f664415c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55208/0 (socket says 172.21.15.32:55208) 2024-04-17T10:00:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.633+0000 7f6642a00640 1 -- 172.21.15.32:0/161136006 learned_addr learned my addr 172.21.15.32:0/161136006 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.633+0000 7f663a000640 1 --2- 172.21.15.32:0/161136006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66440d0820 0x7f664413e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.633+0000 7f6643400640 1 --2- 172.21.15.32:0/161136006 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f664413f6e0 0x7f6644158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.633+0000 7f6642a00640 1 -- 172.21.15.32:0/161136006 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f664413f6e0 msgr2=0x7f6644158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.633+0000 7f6642a00640 1 --2- 172.21.15.32:0/161136006 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f664413f6e0 0x7f6644158fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.633+0000 7f6642a00640 1 -- 172.21.15.32:0/161136006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66440d0820 msgr2=0x7f664413e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.633+0000 7f6642a00640 1 --2- 172.21.15.32:0/161136006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66440d0820 0x7f664413e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.633+0000 7f6642a00640 1 -- 172.21.15.32:0/161136006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664415b170 con 0x7f66440cf370 2024-04-17T10:00:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.633+0000 7f6642a00640 1 --2- 172.21.15.32:0/161136006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66440cf370 0x7f664415c210 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f663401b0a0 tx=0x7f6634019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.633+0000 7f663b400640 1 -- 172.21.15.32:0/161136006 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66340bc020 con 0x7f66440cf370 2024-04-17T10:00:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.633+0000 7f663b400640 1 -- 172.21.15.32:0/161136006 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66340b8670 con 0x7f66440cf370 2024-04-17T10:00:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.633+0000 7f663b400640 1 -- 172.21.15.32:0/161136006 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66340b8a40 con 0x7f66440cf370 2024-04-17T10:00:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.633+0000 7f6649c00640 1 -- 172.21.15.32:0/161136006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f664415b450 con 0x7f66440cf370 2024-04-17T10:00:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.633+0000 7f6649c00640 1 -- 172.21.15.32:0/161136006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f664415ba10 con 0x7f66440cf370 2024-04-17T10:00:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.633+0000 7f6649c00640 1 -- 172.21.15.32:0/161136006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6644068a90 con 0x7f66440cf370 2024-04-17T10:00:43.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.633+0000 7f663b400640 1 -- 172.21.15.32:0/161136006 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f66340bf020 con 0x7f66440cf370 2024-04-17T10:00:43.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.633+0000 7f663b400640 1 --2- 172.21.15.32:0/161136006 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6620068380 0x7f662006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.637+0000 7f663b400640 1 -- 172.21.15.32:0/161136006 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5853+0+0 (secure 0 0 0) 0x7f6634121a40 con 0x7f66440cf370 2024-04-17T10:00:43.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.637+0000 7f663a000640 1 --2- 172.21.15.32:0/161136006 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6620068380 0x7f662006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.637+0000 7f663a000640 1 --2- 172.21.15.32:0/161136006 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6620068380 0x7f662006a840 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6644146290 tx=0x7f662c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:43.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.641+0000 7f663b400640 1 -- 172.21.15.32:0/161136006 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f66340c6030 con 0x7f66440cf370 2024-04-17T10:00:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.837+0000 7f6649c00640 1 -- 172.21.15.32:0/161136006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f66440d1030 con 0x7f66440cf370 2024-04-17T10:00:43.838 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-17T10:00:43.841+0000 7f93c9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T10:00:43.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.841+0000 7f663b400640 1 -- 172.21.15.32:0/161136006 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v67) ==== 161+0+0 (secure 0 0 0) 0x7f66340165e0 con 0x7f66440cf370 2024-04-17T10:00:43.840 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T10:00:43.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.845+0000 7f6649c00640 1 -- 172.21.15.32:0/161136006 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6620068380 msgr2=0x7f662006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.845+0000 7f6649c00640 1 --2- 172.21.15.32:0/161136006 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6620068380 0x7f662006a840 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6644146290 tx=0x7f662c008040 comp rx=0 tx=0).stop 2024-04-17T10:00:43.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.845+0000 7f6649c00640 1 -- 172.21.15.32:0/161136006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66440cf370 msgr2=0x7f664415c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.845+0000 7f6649c00640 1 --2- 172.21.15.32:0/161136006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66440cf370 0x7f664415c210 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f663401b0a0 tx=0x7f6634019790 comp rx=0 tx=0).stop 2024-04-17T10:00:43.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.845+0000 7f6649c00640 1 -- 172.21.15.32:0/161136006 shutdown_connections 2024-04-17T10:00:43.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.845+0000 7f6649c00640 1 --2- 172.21.15.32:0/161136006 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6620068380 0x7f662006a840 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.845+0000 7f6649c00640 1 --2- 172.21.15.32:0/161136006 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f664413f6e0 0x7f6644158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.845+0000 7f6649c00640 1 --2- 172.21.15.32:0/161136006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66440d0820 0x7f664413e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.845+0000 7f6649c00640 1 --2- 172.21.15.32:0/161136006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66440cf370 0x7f664415c210 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.845+0000 7f6649c00640 1 -- 172.21.15.32:0/161136006 >> 172.21.15.32:0/161136006 conn(0x7f664406ba30 msgr2=0x7f664415ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:43.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.845+0000 7f6649c00640 1 -- 172.21.15.32:0/161136006 shutdown_connections 2024-04-17T10:00:43.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.845+0000 7f6649c00640 1 -- 172.21.15.32:0/161136006 wait complete. 2024-04-17T10:00:43.852 INFO:tasks.ceph.mds.c.smithi032.stderr: -4607> 2024-04-17T09:59:35.002+0000 7f3487a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:00:43.853 INFO:tasks.ceph.mds.c.smithi032.stderr: -4259> 2024-04-17T09:59:36.270+0000 7f3482a00640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-17T10:00:43.853 INFO:tasks.ceph.mds.c.smithi032.stderr: -4097> 2024-04-17T09:59:36.506+0000 7f3482a00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-04-17T09:59:18.000169+0000 1=1+0) n(v0 rc2024-04-17T09:59:18.115350+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x55d3b591a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-17T09:59:18.000169+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-17T09:59:18.115350+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-17T10:00:43.857 INFO:tasks.ceph.mds.c.smithi032.stderr: -2305> 2024-04-17T10:00:08.126+0000 7f3482a00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv17 ap=3 DIRTYPARENT f(v0 m2024-04-17T09:59:18.000169+0000 1=1+0)->f(v1 m2024-04-17T09:59:18.000169+0000) n(v0 rc2024-04-17T09:59:18.115350+0000 2=1+1)->n(v0 rc2024-04-17T09:59:18.115350+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55d3b591a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-17T09:59:18.000169+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-17T09:59:18.115350+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-17T10:00:43.864 DEBUG:teuthology.orchestra.run.smithi032:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-17T10:00:43.901 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-17T10:00:43.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.909+0000 7fb07fc228c0 1 -- start start 2024-04-17T10:00:43.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.909+0000 7fb07fc228c0 1 Processor -- start 2024-04-17T10:00:43.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.909+0000 7fb07fc228c0 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f31ae49800 0x55f31bc1a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.909+0000 7fb07fc228c0 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f31ae49400 0x55f31bc1a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.909+0000 7fb07fc228c0 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f31ae49000 0x55f31bc1ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.909+0000 7fb07fc228c0 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f31bc30180 con 0x55f31ae49800 2024-04-17T10:00:43.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.909+0000 7fb07fc228c0 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f31bc30300 con 0x55f31ae49000 2024-04-17T10:00:43.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.909+0000 7fb07fc228c0 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f31bc30480 con 0x55f31ae49400 2024-04-17T10:00:43.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f31ae49800 0x55f31bc1a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f31ae49800 0x55f31bc1a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55220/0 (socket says 172.21.15.32:55220) 2024-04-17T10:00:43.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07e600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f31ae49000 0x55f31bc1ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07dc00640 1 -- 172.21.15.32:0/913043941 learned_addr learned my addr 172.21.15.32:0/913043941 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:43.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07d200640 1 --2- 172.21.15.32:0/913043941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f31ae49400 0x55f31bc1a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07dc00640 1 -- 172.21.15.32:0/913043941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f31ae49400 msgr2=0x55f31bc1a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:43.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07dc00640 1 --2- 172.21.15.32:0/913043941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f31ae49400 0x55f31bc1a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07dc00640 1 -- 172.21.15.32:0/913043941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f31ae49000 msgr2=0x55f31bc1ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:43.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07dc00640 1 --2- 172.21.15.32:0/913043941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f31ae49000 0x55f31bc1ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07dc00640 1 -- 172.21.15.32:0/913043941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f31ae6e1e0 con 0x55f31ae49800 2024-04-17T10:00:43.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07d200640 1 --2- 172.21.15.32:0/913043941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f31ae49400 0x55f31bc1a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:43.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07e600640 1 --2- 172.21.15.32:0/913043941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f31ae49000 0x55f31bc1ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:00:43.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07dc00640 1 --2- 172.21.15.32:0/913043941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f31ae49800 0x55f31bc1a000 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x55f31bcad860 tx=0x55f31af2a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ac85620a226e89 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:43.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07aa00640 1 -- 172.21.15.32:0/913043941 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f31ae7e4e0 con 0x55f31ae49800 2024-04-17T10:00:43.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07aa00640 1 -- 172.21.15.32:0/913043941 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55f31ae94e00 con 0x55f31ae49800 2024-04-17T10:00:43.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07aa00640 1 -- 172.21.15.32:0/913043941 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f31ae7eb60 con 0x55f31ae49800 2024-04-17T10:00:43.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07fc228c0 1 -- 172.21.15.32:0/913043941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f31ae6f2c0 con 0x55f31ae49800 2024-04-17T10:00:43.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07fc228c0 1 -- 172.21.15.32:0/913043941 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55f31ae6fa40 con 0x55f31ae49800 2024-04-17T10:00:43.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07fc228c0 4 waiting for MDS map... 2024-04-17T10:00:43.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07aa00640 1 -- 172.21.15.32:0/913043941 <== mon.0 v2:172.21.15.32:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x55f31bd6c600 con 0x55f31ae49800 2024-04-17T10:00:43.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07aa00640 1 -- 172.21.15.32:0/913043941 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 67) ==== 2886+0+0 (secure 0 0 0) 0x55f31ae9f180 con 0x55f31ae49800 2024-04-17T10:00:43.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07fc228c0 4 Got MDS map 67 2024-04-17T10:00:43.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07fc228c0 4 datascan.main: Using metadata pool output 2024-04-17T10:00:43.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07fc228c0 4 datascan.main: connecting to RADOS... 2024-04-17T10:00:43.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07fc228c0 1 Processor -- start 2024-04-17T10:00:43.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07fc228c0 1 -- start start 2024-04-17T10:00:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07fc228c0 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f31bd71000 0x55f31bc1b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07fc228c0 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f31bd71400 0x55f31bc1b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07fc228c0 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f31bd71800 0x55f31bc1bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07fc228c0 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f31bc30600 con 0x55f31bd71000 2024-04-17T10:00:43.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07fc228c0 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f31bc30780 con 0x55f31bd71400 2024-04-17T10:00:43.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07fc228c0 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f31bc30900 con 0x55f31bd71800 2024-04-17T10:00:43.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07e600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f31bd71000 0x55f31bc1b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07e600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f31bd71000 0x55f31bc1b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55224/0 (socket says 172.21.15.32:55224) 2024-04-17T10:00:43.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07e600640 1 -- 172.21.15.32:0/3043156155 learned_addr learned my addr 172.21.15.32:0/3043156155 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:43.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07dc00640 1 --2- 172.21.15.32:0/3043156155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f31bd71400 0x55f31bc1b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07d200640 1 --2- 172.21.15.32:0/3043156155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f31bd71800 0x55f31bc1bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07e600640 1 -- 172.21.15.32:0/3043156155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f31bd71800 msgr2=0x55f31bc1bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:43.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07e600640 1 --2- 172.21.15.32:0/3043156155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f31bd71800 0x55f31bc1bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07e600640 1 -- 172.21.15.32:0/3043156155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f31bd71400 msgr2=0x55f31bc1b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:43.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07e600640 1 --2- 172.21.15.32:0/3043156155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f31bd71400 0x55f31bc1b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07e600640 1 -- 172.21.15.32:0/3043156155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f31ae6fc20 con 0x55f31bd71000 2024-04-17T10:00:43.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.913+0000 7fb07dc00640 1 --2- 172.21.15.32:0/3043156155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f31bd71400 0x55f31bc1b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:43.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07e600640 1 --2- 172.21.15.32:0/3043156155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f31bd71000 0x55f31bc1b080 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x55f31bd74c60 tx=0x55f31af26660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d9732ef8cd100b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:43.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb078200640 1 -- 172.21.15.32:0/3043156155 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f31ae7e9c0 con 0x55f31bd71000 2024-04-17T10:00:43.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb078200640 1 -- 172.21.15.32:0/3043156155 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55f31ae94c40 con 0x55f31bd71000 2024-04-17T10:00:43.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb078200640 1 -- 172.21.15.32:0/3043156155 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f31ae7e4e0 con 0x55f31bd71000 2024-04-17T10:00:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3043156155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f31bd71000 msgr2=0x55f31bc1b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07fc228c0 1 --2- 172.21.15.32:0/3043156155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f31bd71000 0x55f31bc1b080 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x55f31bd74c60 tx=0x55f31af26660 comp rx=0 tx=0).stop 2024-04-17T10:00:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3043156155 shutdown_connections 2024-04-17T10:00:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07fc228c0 1 --2- 172.21.15.32:0/3043156155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f31bd71800 0x55f31bc1bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07fc228c0 1 --2- 172.21.15.32:0/3043156155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f31bd71400 0x55f31bc1b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07fc228c0 1 --2- 172.21.15.32:0/3043156155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f31bd71000 0x55f31bc1b080 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3043156155 >> 172.21.15.32:0/3043156155 conn(0x55f31ae48c00 msgr2=0x55f31ae77800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3043156155 shutdown_connections 2024-04-17T10:00:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3043156155 wait complete. 2024-04-17T10:00:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07fc228c0 1 Processor -- start 2024-04-17T10:00:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07fc228c0 1 -- start start 2024-04-17T10:00:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07fc228c0 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f31bd71000 0x55f31bc1b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07fc228c0 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f31bd71400 0x55f31bc1b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07fc228c0 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f31bd71800 0x55f31bc1bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07fc228c0 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f31bc31b00 con 0x55f31bd71800 2024-04-17T10:00:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07fc228c0 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f31bc31980 con 0x55f31bd71000 2024-04-17T10:00:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07fc228c0 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f31bc31800 con 0x55f31bd71400 2024-04-17T10:00:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f31bd71800 0x55f31bc1bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f31bd71800 0x55f31bc1bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55232/0 (socket says 172.21.15.32:55232) 2024-04-17T10:00:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07d200640 1 -- 172.21.15.32:0/3076313822 learned_addr learned my addr 172.21.15.32:0/3076313822 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07dc00640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f31bd71400 0x55f31bc1b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07e600640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f31bd71000 0x55f31bc1b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07d200640 1 -- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f31bd71400 msgr2=0x55f31bc1b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07d200640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f31bd71400 0x55f31bc1b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07d200640 1 -- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f31bd71000 msgr2=0x55f31bc1b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07d200640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f31bd71000 0x55f31bc1b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07d200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f31ae6f4a0 con 0x55f31bd71800 2024-04-17T10:00:43.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07d200640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f31bd71800 0x55f31bc1bb80 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x55f31bd75a10 tx=0x55f31bd78c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:43.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb076400640 1 -- 172.21.15.32:0/3076313822 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f31ae7e9c0 con 0x55f31bd71800 2024-04-17T10:00:43.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb076400640 1 -- 172.21.15.32:0/3076313822 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55f31ae94540 con 0x55f31bd71800 2024-04-17T10:00:43.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f31ae6e3c0 con 0x55f31bd71800 2024-04-17T10:00:43.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb076400640 1 -- 172.21.15.32:0/3076313822 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f31ae7e4e0 con 0x55f31bd71800 2024-04-17T10:00:43.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f31ae6fc20 con 0x55f31bd71800 2024-04-17T10:00:43.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.917+0000 7fb07fc228c0 4 datascan.init: resolving metadata pool 12 2024-04-17T10:00:43.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.921+0000 7fb076400640 1 -- 172.21.15.32:0/3076313822 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x55f31bdd0c00 con 0x55f31bd71800 2024-04-17T10:00:43.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.921+0000 7fb076400640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55f31b14cc00 0x55f31bc1c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.921+0000 7fb076400640 1 -- 172.21.15.32:0/3076313822 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x55f31b14ea00 con 0x55f31bd71800 2024-04-17T10:00:43.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.921+0000 7fb07fc228c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-17T10:00:43.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.921+0000 7fb07fc228c0 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-04-17T10:00:43.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.921+0000 7fb07fc228c0 4 datascan.main: opening data pool 'cephfs_data' 2024-04-17T10:00:43.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.921+0000 7fb07fc228c0 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55f31bd71c00 0x55f31bc1c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.921+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b14d000 con 0x55f31bd71c00 2024-04-17T10:00:43.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.921+0000 7fb07e600640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55f31b14cc00 0x55f31bc1c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.921+0000 7fb07dc00640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55f31bd71c00 0x55f31bc1c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.921+0000 7fb07e600640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55f31b14cc00 0x55f31bc1c100 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x55f31bd74c60 tx=0x55f31af26a80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.921+0000 7fb07dc00640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55f31bd71c00 0x55f31bc1c680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:43.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.925+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.7 v2:172.21.15.144:6824/1746570140 1 ==== osd_op_reply(1 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31bd71c00 2024-04-17T10:00:43.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.925+0000 7fb077800640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55f31b23c000 0x55f31b242000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.925+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:2 13.20 13:04000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b23c400 con 0x55f31b23c000 2024-04-17T10:00:43.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.925+0000 7fb07e600640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55f31b23c000 0x55f31b242000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.925+0000 7fb07e600640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55f31b23c000 0x55f31b242000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:43.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.925+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.6 v2:172.21.15.144:6808/3198738696 1 ==== osd_op_reply(2 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b23c000 2024-04-17T10:00:43.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.925+0000 7fb078200640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55f31b23cc00 0x55f31b242580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.925+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:3 13.10 13:08000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b23d000 con 0x55f31b23cc00 2024-04-17T10:00:43.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.925+0000 7fb07dc00640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55f31b23cc00 0x55f31b242580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.929+0000 7fb07dc00640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55f31b23cc00 0x55f31b242580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:43.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.929+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.1 v2:172.21.15.32:6808/2454993455 1 ==== osd_op_reply(3 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31b23cc00 2024-04-17T10:00:43.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.929+0000 7fb077800640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55f31b23c800 0x55f31b242b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.929+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:4 13.30 13:0c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b23d800 con 0x55f31b23c800 2024-04-17T10:00:43.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.929+0000 7fb07d200640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55f31b23c800 0x55f31b242b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.929+0000 7fb07d200640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55f31b23c800 0x55f31b242b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:43.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.929+0000 7fb07d200640 1 -- 172.21.15.32:0/3076313822 <== osd.4 v2:172.21.15.144:6809/4076371964 1 ==== osd_op_reply(4 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16780 con 0x55f31b23c800 2024-04-17T10:00:43.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.929+0000 7fb078200640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55f31b23d400 0x55f31b243600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.929+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:5 13.8 13:10000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b256000 con 0x55f31b23d400 2024-04-17T10:00:43.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.929+0000 7fb07e600640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55f31b23d400 0x55f31b243600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.929+0000 7fb07e600640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55f31b23d400 0x55f31b243600 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:43.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.933+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.2 v2:172.21.15.32:6809/1954833462 1 ==== osd_op_reply(5 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b23d400 2024-04-17T10:00:43.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.933+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:6 13.28 13:14000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b23dc00 con 0x55f31b23d400 2024-04-17T10:00:43.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.933+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.2 v2:172.21.15.32:6809/1954833462 2 ==== osd_op_reply(6 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b23d400 2024-04-17T10:00:43.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.933+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:7 13.18 13:18000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b256400 con 0x55f31bd71c00 2024-04-17T10:00:43.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.933+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.7 v2:172.21.15.144:6824/1746570140 2 ==== osd_op_reply(7 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31bd71c00 2024-04-17T10:00:43.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.933+0000 7fb077800640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55f31b256c00 0x55f31b243080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.933+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:8 13.38 13:1c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b257000 con 0x55f31b256c00 2024-04-17T10:00:43.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.933+0000 7fb07dc00640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55f31b256c00 0x55f31b243080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.933+0000 7fb07dc00640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55f31b256c00 0x55f31b243080 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:43.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.937+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.5 v2:172.21.15.144:6800/2379578648 1 ==== osd_op_reply(8 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31b256c00 2024-04-17T10:00:43.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.937+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:9 13.4 13:20000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b256800 con 0x55f31b23c000 2024-04-17T10:00:43.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.937+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.6 v2:172.21.15.144:6808/3198738696 2 ==== osd_op_reply(9 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b23c000 2024-04-17T10:00:43.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.937+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:10 13.24 13:24000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b257400 con 0x55f31b23c000 2024-04-17T10:00:43.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.937+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.6 v2:172.21.15.144:6808/3198738696 3 ==== osd_op_reply(10 [pgnls start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b23c000 2024-04-17T10:00:43.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.937+0000 7fb078200640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55f31b23c400 0x55f31b243b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.937+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:11 13.14 13:28000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b14d000 con 0x55f31b23c400 2024-04-17T10:00:43.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.937+0000 7fb07d200640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55f31b23c400 0x55f31b243b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.937+0000 7fb07d200640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55f31b23c400 0x55f31b243b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:43.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.941+0000 7fb07d200640 1 -- 172.21.15.32:0/3076313822 <== osd.0 v2:172.21.15.32:6800/2392056658 1 ==== osd_op_reply(11 [pgnls start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16780 con 0x55f31b23c400 2024-04-17T10:00:43.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.941+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:12 13.34 13:2c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b257800 con 0x55f31b23c000 2024-04-17T10:00:43.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.941+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.6 v2:172.21.15.144:6808/3198738696 4 ==== osd_op_reply(12 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b23c000 2024-04-17T10:00:43.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.941+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:13 13.c 13:30000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31bd70800 con 0x55f31b23cc00 2024-04-17T10:00:43.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.941+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.1 v2:172.21.15.32:6808/2454993455 2 ==== osd_op_reply(13 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31b23cc00 2024-04-17T10:00:43.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.941+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:14 13.2c 13:34000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b256000 con 0x55f31b23d400 2024-04-17T10:00:43.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.941+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.2 v2:172.21.15.32:6809/1954833462 3 ==== osd_op_reply(14 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b23d400 2024-04-17T10:00:43.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.941+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:15 13.1c 13:38000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31bc3d400 con 0x55f31b23c000 2024-04-17T10:00:43.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.941+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.6 v2:172.21.15.144:6808/3198738696 5 ==== osd_op_reply(15 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b23c000 2024-04-17T10:00:43.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.941+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:16 13.3c 13:3c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b256800 con 0x55f31b23d400 2024-04-17T10:00:43.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.941+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.2 v2:172.21.15.32:6809/1954833462 4 ==== osd_op_reply(16 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b23d400 2024-04-17T10:00:43.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.941+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:17 13.2 13:40000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b257000 con 0x55f31b256c00 2024-04-17T10:00:43.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.941+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.5 v2:172.21.15.144:6800/2379578648 2 ==== osd_op_reply(17 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31b256c00 2024-04-17T10:00:43.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.941+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:18 13.22 13:44000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b256400 con 0x55f31b23cc00 2024-04-17T10:00:43.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.941+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.1 v2:172.21.15.32:6808/2454993455 3 ==== osd_op_reply(18 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31b23cc00 2024-04-17T10:00:43.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.941+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:19 13.12 13:48000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b257c00 con 0x55f31b23c400 2024-04-17T10:00:43.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.945+0000 7fb07d200640 1 -- 172.21.15.32:0/3076313822 <== osd.0 v2:172.21.15.32:6800/2392056658 2 ==== osd_op_reply(19 [pgnls start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16780 con 0x55f31b23c400 2024-04-17T10:00:43.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.945+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:20 13.32 13:4c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b23dc00 con 0x55f31b23d400 2024-04-17T10:00:43.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.945+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.2 v2:172.21.15.32:6809/1954833462 5 ==== osd_op_reply(20 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b23d400 2024-04-17T10:00:43.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.945+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:21 13.a 13:50000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b23dc00 con 0x55f31b23d400 2024-04-17T10:00:43.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.945+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.2 v2:172.21.15.32:6809/1954833462 6 ==== osd_op_reply(21 [pgnls start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b23d400 2024-04-17T10:00:43.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.945+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:22 13.2a 13:54000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b23d000 con 0x55f31b23cc00 2024-04-17T10:00:43.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.945+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.1 v2:172.21.15.32:6808/2454993455 4 ==== osd_op_reply(22 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31b23cc00 2024-04-17T10:00:43.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.945+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:23 13.1a 13:58000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b257400 con 0x55f31b23cc00 2024-04-17T10:00:43.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.945+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.1 v2:172.21.15.32:6808/2454993455 5 ==== osd_op_reply(23 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31b23cc00 2024-04-17T10:00:43.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.945+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:24 13.3a 13:5c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b264000 con 0x55f31b256c00 2024-04-17T10:00:43.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.945+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.5 v2:172.21.15.144:6800/2379578648 3 ==== osd_op_reply(24 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31b256c00 2024-04-17T10:00:43.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.945+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:25 13.6 13:60000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b257800 con 0x55f31b23c800 2024-04-17T10:00:43.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.945+0000 7fb07d200640 1 -- 172.21.15.32:0/3076313822 <== osd.4 v2:172.21.15.144:6809/4076371964 2 ==== osd_op_reply(25 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16780 con 0x55f31b23c800 2024-04-17T10:00:43.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.945+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:26 13.26 13:64000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b264400 con 0x55f31bd71c00 2024-04-17T10:00:43.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.945+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.7 v2:172.21.15.144:6824/1746570140 3 ==== osd_op_reply(26 [pgnls start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31bd71c00 2024-04-17T10:00:43.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.945+0000 7fb078200640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55f31b256000 0x55f31b244100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:43.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.945+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:27 13.16 13:68000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b256800 con 0x55f31b256000 2024-04-17T10:00:43.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.945+0000 7fb07e600640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55f31b256000 0x55f31b244100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:43.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.945+0000 7fb07e600640 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55f31b256000 0x55f31b244100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:43.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.949+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.3 v2:172.21.15.32:6824/2182899298 1 ==== osd_op_reply(27 [pgnls start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b256000 2024-04-17T10:00:43.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.949+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:28 13.36 13:6c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b264800 con 0x55f31b23c000 2024-04-17T10:00:43.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.949+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.6 v2:172.21.15.144:6808/3198738696 6 ==== osd_op_reply(28 [pgnls start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b23c000 2024-04-17T10:00:43.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.949+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:29 13.e 13:70000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31bc3d400 con 0x55f31bd71c00 2024-04-17T10:00:43.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.949+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.7 v2:172.21.15.144:6824/1746570140 4 ==== osd_op_reply(29 [pgnls start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31bd71c00 2024-04-17T10:00:43.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.949+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:30 13.2e 13:74000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b264c00 con 0x55f31b23d400 2024-04-17T10:00:43.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.949+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.2 v2:172.21.15.32:6809/1954833462 7 ==== osd_op_reply(30 [pgnls start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b23d400 2024-04-17T10:00:43.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.949+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:31 13.1e 13:78000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b257c00 con 0x55f31b23c800 2024-04-17T10:00:43.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.949+0000 7fb07d200640 1 -- 172.21.15.32:0/3076313822 <== osd.4 v2:172.21.15.144:6809/4076371964 3 ==== osd_op_reply(31 [pgnls start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16780 con 0x55f31b23c800 2024-04-17T10:00:43.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.949+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:32 13.3e 13:7c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b265000 con 0x55f31b23cc00 2024-04-17T10:00:43.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.953+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.1 v2:172.21.15.32:6808/2454993455 6 ==== osd_op_reply(32 [pgnls start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31b23cc00 2024-04-17T10:00:43.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.953+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:33 13.1 13:80000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b257800 con 0x55f31b23c000 2024-04-17T10:00:43.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.953+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.6 v2:172.21.15.144:6808/3198738696 7 ==== osd_op_reply(33 [pgnls start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b23c000 2024-04-17T10:00:43.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.953+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:34 13.21 13:84000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b265400 con 0x55f31b23c800 2024-04-17T10:00:43.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.953+0000 7fb07d200640 1 -- 172.21.15.32:0/3076313822 <== osd.4 v2:172.21.15.144:6809/4076371964 4 ==== osd_op_reply(34 [pgnls start_epoch 86 out=49b] v82'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16780 con 0x55f31b23c800 2024-04-17T10:00:43.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.953+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:35 13.11 13:88000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b257000 con 0x55f31b256000 2024-04-17T10:00:43.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.953+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.3 v2:172.21.15.32:6824/2182899298 2 ==== osd_op_reply(35 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b256000 2024-04-17T10:00:43.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.953+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:36 13.31 13:8c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b264000 con 0x55f31b256000 2024-04-17T10:00:43.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.953+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.3 v2:172.21.15.32:6824/2182899298 3 ==== osd_op_reply(36 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b256000 2024-04-17T10:00:43.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.953+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:37 13.9 13:90000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b23d800 con 0x55f31b256c00 2024-04-17T10:00:43.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.953+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.5 v2:172.21.15.144:6800/2379578648 4 ==== osd_op_reply(37 [pgnls start_epoch 86 out=49b] v82'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31b256c00 2024-04-17T10:00:43.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.953+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:38 13.29 13:94000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b265800 con 0x55f31b23c000 2024-04-17T10:00:43.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.953+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.6 v2:172.21.15.144:6808/3198738696 8 ==== osd_op_reply(38 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b23c000 2024-04-17T10:00:43.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.953+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:39 13.19 13:98000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b23d000 con 0x55f31b256000 2024-04-17T10:00:43.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.953+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.3 v2:172.21.15.32:6824/2182899298 4 ==== osd_op_reply(39 [pgnls start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b256000 2024-04-17T10:00:43.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.953+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:40 13.39 13:9c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b265c00 con 0x55f31b23d400 2024-04-17T10:00:43.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.953+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.2 v2:172.21.15.32:6809/1954833462 8 ==== osd_op_reply(40 [pgnls start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b23d400 2024-04-17T10:00:43.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.953+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:41 13.5 13:a0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31bd70800 con 0x55f31b23c800 2024-04-17T10:00:43.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.953+0000 7fb07d200640 1 -- 172.21.15.32:0/3076313822 <== osd.4 v2:172.21.15.144:6809/4076371964 5 ==== osd_op_reply(41 [pgnls start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16780 con 0x55f31b23c800 2024-04-17T10:00:43.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.953+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:42 13.25 13:a4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b256400 con 0x55f31b23c800 2024-04-17T10:00:43.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.957+0000 7fb07d200640 1 -- 172.21.15.32:0/3076313822 <== osd.4 v2:172.21.15.144:6809/4076371964 6 ==== osd_op_reply(42 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16780 con 0x55f31b23c800 2024-04-17T10:00:43.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.957+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:43 13.15 13:a8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b14d000 con 0x55f31b23c800 2024-04-17T10:00:43.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.957+0000 7fb07d200640 1 -- 172.21.15.32:0/3076313822 <== osd.4 v2:172.21.15.144:6809/4076371964 7 ==== osd_op_reply(43 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16780 con 0x55f31b23c800 2024-04-17T10:00:43.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.957+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:44 13.35 13:ac000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b257400 con 0x55f31b23cc00 2024-04-17T10:00:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.957+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.1 v2:172.21.15.32:6808/2454993455 7 ==== osd_op_reply(44 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31b23cc00 2024-04-17T10:00:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.957+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:45 13.d 13:b0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b257c00 con 0x55f31b23c800 2024-04-17T10:00:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.957+0000 7fb07d200640 1 -- 172.21.15.32:0/3076313822 <== osd.4 v2:172.21.15.144:6809/4076371964 8 ==== osd_op_reply(45 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16780 con 0x55f31b23c800 2024-04-17T10:00:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.957+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:46 13.2d 13:b4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b265400 con 0x55f31b23c400 2024-04-17T10:00:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.957+0000 7fb07d200640 1 -- 172.21.15.32:0/3076313822 <== osd.0 v2:172.21.15.32:6800/2392056658 3 ==== osd_op_reply(46 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16780 con 0x55f31b23c400 2024-04-17T10:00:43.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.957+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:47 13.1d 13:b8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b256400 con 0x55f31bd71c00 2024-04-17T10:00:43.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.957+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.7 v2:172.21.15.144:6824/1746570140 5 ==== osd_op_reply(47 [pgnls start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31bd71c00 2024-04-17T10:00:43.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.957+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:48 13.3d 13:bc000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b264800 con 0x55f31bd71c00 2024-04-17T10:00:43.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.957+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.7 v2:172.21.15.144:6824/1746570140 6 ==== osd_op_reply(48 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31bd71c00 2024-04-17T10:00:43.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.957+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:49 13.3 13:c0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b256800 con 0x55f31b23c400 2024-04-17T10:00:43.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.957+0000 7fb07d200640 1 -- 172.21.15.32:0/3076313822 <== osd.0 v2:172.21.15.32:6800/2392056658 4 ==== osd_op_reply(49 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16780 con 0x55f31b23c400 2024-04-17T10:00:43.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.957+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:50 13.23 13:c4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b264c00 con 0x55f31b23cc00 2024-04-17T10:00:43.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.957+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.1 v2:172.21.15.32:6808/2454993455 8 ==== osd_op_reply(50 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31b23cc00 2024-04-17T10:00:43.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.957+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:51 13.13 13:c8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b257800 con 0x55f31bd71c00 2024-04-17T10:00:43.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.957+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.7 v2:172.21.15.144:6824/1746570140 7 ==== osd_op_reply(51 [pgnls start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31bd71c00 2024-04-17T10:00:43.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.957+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:52 13.33 13:cc000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b264000 con 0x55f31bd71c00 2024-04-17T10:00:43.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.957+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.7 v2:172.21.15.144:6824/1746570140 8 ==== osd_op_reply(52 [pgnls start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31bd71c00 2024-04-17T10:00:43.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.957+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:53 13.b 13:d0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b257000 con 0x55f31b256c00 2024-04-17T10:00:43.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.961+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.5 v2:172.21.15.144:6800/2379578648 5 ==== osd_op_reply(53 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31b256c00 2024-04-17T10:00:43.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.961+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:54 13.2b 13:d4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b264400 con 0x55f31b23d400 2024-04-17T10:00:43.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.961+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.2 v2:172.21.15.32:6809/1954833462 9 ==== osd_op_reply(54 [pgnls start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b23d400 2024-04-17T10:00:43.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.961+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:55 13.1b 13:d8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b23dc00 con 0x55f31b23c800 2024-04-17T10:00:43.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.961+0000 7fb07d200640 1 -- 172.21.15.32:0/3076313822 <== osd.4 v2:172.21.15.144:6809/4076371964 9 ==== osd_op_reply(55 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16780 con 0x55f31b23c800 2024-04-17T10:00:43.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.961+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:56 13.3b 13:dc000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b265000 con 0x55f31b23c800 2024-04-17T10:00:43.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.961+0000 7fb07d200640 1 -- 172.21.15.32:0/3076313822 <== osd.4 v2:172.21.15.144:6809/4076371964 10 ==== osd_op_reply(56 [pgnls start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16780 con 0x55f31b23c800 2024-04-17T10:00:43.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.961+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:57 13.7 13:e0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b14d000 con 0x55f31bd71c00 2024-04-17T10:00:43.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.961+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.7 v2:172.21.15.144:6824/1746570140 9 ==== osd_op_reply(57 [pgnls start_epoch 86 out=81b] v82'6 uv6 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55f31af16000 con 0x55f31bd71c00 2024-04-17T10:00:43.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.961+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:58 13.7 13:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f31b14d400 con 0x55f31bd71c00 2024-04-17T10:00:43.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.961+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.7 v2:172.21.15.144:6824/1746570140 10 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55f31af16000 con 0x55f31bd71c00 2024-04-17T10:00:43.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.961+0000 7fb07fc228c0 10 datascan.operator(): handling object 1099511627777.0 2024-04-17T10:00:43.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.961+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:59 13.7 13:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x55f31b14d800 con 0x55f31bd71c00 2024-04-17T10:00:43.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.7 v2:172.21.15.144:6824/1746570140 11 ==== osd_op_reply(59 10000000001.00000000 [call] v86'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55f31af16000 con 0x55f31bd71c00 2024-04-17T10:00:43.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:60 13.27 13:e4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b14dc00 con 0x55f31b23cc00 2024-04-17T10:00:43.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.1 v2:172.21.15.32:6808/2454993455 9 ==== osd_op_reply(60 [pgnls start_epoch 86 out=49b] v82'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31b23cc00 2024-04-17T10:00:43.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:61 13.17 13:e8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31bd70800 con 0x55f31b256c00 2024-04-17T10:00:43.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.5 v2:172.21.15.144:6800/2379578648 6 ==== osd_op_reply(61 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31b256c00 2024-04-17T10:00:43.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:62 13.37 13:ec000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b264800 con 0x55f31b256c00 2024-04-17T10:00:43.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07dc00640 1 -- 172.21.15.32:0/3076313822 <== osd.5 v2:172.21.15.144:6800/2379578648 7 ==== osd_op_reply(62 [pgnls start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16000 con 0x55f31b256c00 2024-04-17T10:00:43.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:63 13.f 13:f0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b14d000 con 0x55f31b23c000 2024-04-17T10:00:43.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.6 v2:172.21.15.144:6808/3198738696 9 ==== osd_op_reply(63 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b23c000 2024-04-17T10:00:43.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:64 13.2f 13:f4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b257400 con 0x55f31b23c800 2024-04-17T10:00:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07d200640 1 -- 172.21.15.32:0/3076313822 <== osd.4 v2:172.21.15.144:6809/4076371964 11 ==== osd_op_reply(64 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16780 con 0x55f31b23c800 2024-04-17T10:00:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb078200640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:65 13.1f 13:f8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b14d400 con 0x55f31b23c000 2024-04-17T10:00:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.6 v2:172.21.15.144:6808/3198738696 10 ==== osd_op_reply(65 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b23c000 2024-04-17T10:00:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb077800640 1 -- 172.21.15.32:0/3076313822 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:66 13.3f 13:fc000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f31b256400 con 0x55f31b23c000 2024-04-17T10:00:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07e600640 1 -- 172.21.15.32:0/3076313822 <== osd.6 v2:172.21.15.144:6808/3198738696 11 ==== osd_op_reply(66 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f31af16500 con 0x55f31b23c000 2024-04-17T10:00:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55f31b23c400 msgr2=0x55f31b243b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55f31b23c400 0x55f31b243b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55f31b23cc00 msgr2=0x55f31b242580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55f31b23cc00 0x55f31b242580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55f31b23d400 msgr2=0x55f31b243600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55f31b23d400 0x55f31b243600 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55f31b256000 msgr2=0x55f31b244100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55f31b256000 0x55f31b244100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55f31b23c800 msgr2=0x55f31b242b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55f31b23c800 0x55f31b242b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55f31b256c00 msgr2=0x55f31b243080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55f31b256c00 0x55f31b243080 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55f31b23c000 msgr2=0x55f31b242000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55f31b23c000 0x55f31b242000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55f31bd71c00 msgr2=0x55f31bc1c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55f31bd71c00 0x55f31bc1c680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07d200640 1 -- 172.21.15.32:0/3076313822 reap_dead start 2024-04-17T10:00:43.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07d200640 1 -- 172.21.15.32:0/3076313822 reap_dead start 2024-04-17T10:00:43.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07d200640 1 -- 172.21.15.32:0/3076313822 reap_dead start 2024-04-17T10:00:43.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55f31b14cc00 msgr2=0x55f31bc1c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55f31b14cc00 0x55f31bc1c100 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x55f31bd74c60 tx=0x55f31af26a80 comp rx=0 tx=0).stop 2024-04-17T10:00:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f31bd71800 msgr2=0x55f31bc1bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f31bd71800 0x55f31bc1bb80 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x55f31bd75a10 tx=0x55f31bd78c00 comp rx=0 tx=0).stop 2024-04-17T10:00:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3076313822 shutdown_connections 2024-04-17T10:00:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55f31b14cc00 0x55f31bc1c100 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 --2- 172.21.15.32:0/3076313822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f31bd71800 0x55f31bc1bb80 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3076313822 >> 172.21.15.32:0/3076313822 conn(0x55f31ae48c00 msgr2=0x55f31ae77800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3076313822 shutdown_connections 2024-04-17T10:00:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 -- 172.21.15.32:0/3076313822 wait complete. 2024-04-17T10:00:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 -- 172.21.15.32:0/913043941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f31ae49800 msgr2=0x55f31bc1a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 --2- 172.21.15.32:0/913043941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f31ae49800 0x55f31bc1a000 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x55f31bcad860 tx=0x55f31af2a120 comp rx=0 tx=0).stop 2024-04-17T10:00:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 -- 172.21.15.32:0/913043941 shutdown_connections 2024-04-17T10:00:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 --2- 172.21.15.32:0/913043941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f31ae49400 0x55f31bc1a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 --2- 172.21.15.32:0/913043941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f31ae49000 0x55f31bc1ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 --2- 172.21.15.32:0/913043941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f31ae49800 0x55f31bc1a000 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 -- 172.21.15.32:0/913043941 >> 172.21.15.32:0/913043941 conn(0x55f31ae49c00 msgr2=0x55f31ae77000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 -- 172.21.15.32:0/913043941 shutdown_connections 2024-04-17T10:00:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:43.965+0000 7fb07fc228c0 1 -- 172.21.15.32:0/913043941 wait complete. 2024-04-17T10:00:43.970 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.106452, result: 2024-04-17T10:00:43.970 DEBUG:teuthology.orchestra.run.smithi032:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-04-17T10:00:44.022 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:00:44.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f1151c028c0 1 -- start start 2024-04-17T10:00:44.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f1151c028c0 1 Processor -- start 2024-04-17T10:00:44.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f1151c028c0 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e5bc84f000 0x55e5bd626000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f1151c028c0 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e5bc84f400 0x55e5bd626580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f1151c028c0 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5bc84f800 0x55e5bd626b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f1151c028c0 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e5bd764180 con 0x55e5bc84f000 2024-04-17T10:00:44.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f1151c028c0 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e5bd764300 con 0x55e5bc84f800 2024-04-17T10:00:44.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f1151c028c0 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e5bd764480 con 0x55e5bc84f400 2024-04-17T10:00:44.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f114fc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e5bc84f000 0x55e5bd626000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f114fc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e5bc84f000 0x55e5bd626000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55264/0 (socket says 172.21.15.32:55264) 2024-04-17T10:00:44.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f114fc00640 1 -- 172.21.15.32:0/3163904938 learned_addr learned my addr 172.21.15.32:0/3163904938 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:44.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f114f200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e5bc84f400 0x55e5bd626580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f1150600640 1 --2- 172.21.15.32:0/3163904938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5bc84f800 0x55e5bd626b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.065 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:00:44.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f114fc00640 1 -- 172.21.15.32:0/3163904938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e5bc84f400 msgr2=0x55e5bd626580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:44.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f114fc00640 1 --2- 172.21.15.32:0/3163904938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e5bc84f400 0x55e5bd626580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f114fc00640 1 -- 172.21.15.32:0/3163904938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5bc84f800 msgr2=0x55e5bd626b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:44.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f114fc00640 1 --2- 172.21.15.32:0/3163904938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5bc84f800 0x55e5bd626b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f114fc00640 1 -- 172.21.15.32:0/3163904938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5bc8741e0 con 0x55e5bc84f000 2024-04-17T10:00:44.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f114f200640 1 --2- 172.21.15.32:0/3163904938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e5bc84f400 0x55e5bd626580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:00:44.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f114fc00640 1 --2- 172.21.15.32:0/3163904938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e5bc84f000 0x55e5bd626000 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x55e5bd6af4a0 tx=0x55e5bc930fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389fff423ddf8e1b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f114ca00640 1 -- 172.21.15.32:0/3163904938 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e5bc8844e0 con 0x55e5bc84f000 2024-04-17T10:00:44.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f114ca00640 1 -- 172.21.15.32:0/3163904938 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e5bc89a540 con 0x55e5bc84f000 2024-04-17T10:00:44.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f114ca00640 1 -- 172.21.15.32:0/3163904938 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e5bc884b60 con 0x55e5bc84f000 2024-04-17T10:00:44.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f1151c028c0 1 -- 172.21.15.32:0/3163904938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5bc8752c0 con 0x55e5bc84f000 2024-04-17T10:00:44.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f1151c028c0 1 -- 172.21.15.32:0/3163904938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55e5bc875a40 con 0x55e5bc84f000 2024-04-17T10:00:44.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f1151c028c0 4 waiting for MDS map... 2024-04-17T10:00:44.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f114ca00640 1 -- 172.21.15.32:0/3163904938 <== mon.0 v2:172.21.15.32:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x55e5bc953000 con 0x55e5bc84f000 2024-04-17T10:00:44.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f114ca00640 1 -- 172.21.15.32:0/3163904938 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 67) ==== 2886+0+0 (secure 0 0 0) 0x55e5bc8a5180 con 0x55e5bc84f000 2024-04-17T10:00:44.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f1151c028c0 4 Got MDS map 67 2024-04-17T10:00:44.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f1151c028c0 4 datascan.main: Using metadata pool output 2024-04-17T10:00:44.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.065+0000 7f1151c028c0 4 datascan.main: connecting to RADOS... 2024-04-17T10:00:44.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1151c028c0 1 Processor -- start 2024-04-17T10:00:44.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1151c028c0 1 -- start start 2024-04-17T10:00:44.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1151c028c0 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e5bd639400 0x55e5bd627080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1151c028c0 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5bd639800 0x55e5bd627600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1151c028c0 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e5bd639c00 0x55e5bd627b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1151c028c0 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e5bd764600 con 0x55e5bd639400 2024-04-17T10:00:44.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1151c028c0 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e5bd764780 con 0x55e5bd639800 2024-04-17T10:00:44.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1151c028c0 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e5bd764900 con 0x55e5bd639c00 2024-04-17T10:00:44.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1150600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e5bd639400 0x55e5bd627080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1150600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e5bd639400 0x55e5bd627080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55278/0 (socket says 172.21.15.32:55278) 2024-04-17T10:00:44.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1150600640 1 -- 172.21.15.32:0/1632706238 learned_addr learned my addr 172.21.15.32:0/1632706238 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:44.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f114fc00640 1 --2- 172.21.15.32:0/1632706238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5bd639800 0x55e5bd627600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f114f200640 1 --2- 172.21.15.32:0/1632706238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e5bd639c00 0x55e5bd627b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1150600640 1 -- 172.21.15.32:0/1632706238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e5bd639c00 msgr2=0x55e5bd627b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1150600640 1 --2- 172.21.15.32:0/1632706238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e5bd639c00 0x55e5bd627b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1150600640 1 -- 172.21.15.32:0/1632706238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5bd639800 msgr2=0x55e5bd627600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1150600640 1 --2- 172.21.15.32:0/1632706238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5bd639800 0x55e5bd627600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1150600640 1 -- 172.21.15.32:0/1632706238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5bc875c20 con 0x55e5bd639400 2024-04-17T10:00:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1150600640 1 --2- 172.21.15.32:0/1632706238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e5bd639400 0x55e5bd627080 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x55e5bd78c660 tx=0x55e5bd790c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52176c30372330ad server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f114a200640 1 -- 172.21.15.32:0/1632706238 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e5bc8844e0 con 0x55e5bd639400 2024-04-17T10:00:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f114a200640 1 -- 172.21.15.32:0/1632706238 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e5bc89ae00 con 0x55e5bd639400 2024-04-17T10:00:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f114a200640 1 -- 172.21.15.32:0/1632706238 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e5bc884b60 con 0x55e5bd639400 2024-04-17T10:00:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1151c028c0 1 -- 172.21.15.32:0/1632706238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e5bd639400 msgr2=0x55e5bd627080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1151c028c0 1 --2- 172.21.15.32:0/1632706238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e5bd639400 0x55e5bd627080 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x55e5bd78c660 tx=0x55e5bd790c00 comp rx=0 tx=0).stop 2024-04-17T10:00:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1151c028c0 1 -- 172.21.15.32:0/1632706238 shutdown_connections 2024-04-17T10:00:44.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1151c028c0 1 --2- 172.21.15.32:0/1632706238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e5bd639c00 0x55e5bd627b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1151c028c0 1 --2- 172.21.15.32:0/1632706238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5bd639800 0x55e5bd627600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1151c028c0 1 --2- 172.21.15.32:0/1632706238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e5bd639400 0x55e5bd627080 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1151c028c0 1 -- 172.21.15.32:0/1632706238 >> 172.21.15.32:0/1632706238 conn(0x55e5bd639000 msgr2=0x55e5bc87c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1151c028c0 1 -- 172.21.15.32:0/1632706238 shutdown_connections 2024-04-17T10:00:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1151c028c0 1 -- 172.21.15.32:0/1632706238 wait complete. 2024-04-17T10:00:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1151c028c0 1 Processor -- start 2024-04-17T10:00:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.069+0000 7f1151c028c0 1 -- start start 2024-04-17T10:00:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f1151c028c0 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5bd639400 0x55e5bd627080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f1151c028c0 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e5bd639800 0x55e5bd627600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f1151c028c0 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e5bd639c00 0x55e5bd627b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f1151c028c0 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e5bd765c80 con 0x55e5bd639800 2024-04-17T10:00:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f1151c028c0 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e5bd765b00 con 0x55e5bd639400 2024-04-17T10:00:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f1151c028c0 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e5bd765980 con 0x55e5bd639c00 2024-04-17T10:00:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f114fc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e5bd639800 0x55e5bd627600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f114fc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e5bd639800 0x55e5bd627600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55282/0 (socket says 172.21.15.32:55282) 2024-04-17T10:00:44.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 learned_addr learned my addr 172.21.15.32:0/1001015886 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:44.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f1150600640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5bd639400 0x55e5bd627080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f114f200640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e5bd639c00 0x55e5bd627b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e5bd639c00 msgr2=0x55e5bd627b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f114fc00640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e5bd639c00 0x55e5bd627b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5bd639400 msgr2=0x55e5bd627080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f114fc00640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5bd639400 0x55e5bd627080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5bc8741e0 con 0x55e5bd639800 2024-04-17T10:00:44.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f114fc00640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e5bd639800 0x55e5bd627600 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x55e5bd78d980 tx=0x55e5bc930f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f1148400640 1 -- 172.21.15.32:0/1001015886 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e5bc884340 con 0x55e5bd639800 2024-04-17T10:00:44.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f1148400640 1 -- 172.21.15.32:0/1001015886 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e5bc89ac40 con 0x55e5bd639800 2024-04-17T10:00:44.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e5bc8752c0 con 0x55e5bd639800 2024-04-17T10:00:44.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f1148400640 1 -- 172.21.15.32:0/1001015886 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e5bc8844e0 con 0x55e5bd639800 2024-04-17T10:00:44.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5bc875e00 con 0x55e5bd639800 2024-04-17T10:00:44.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f1151c028c0 4 datascan.init: resolving metadata pool 12 2024-04-17T10:00:44.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.073+0000 7f1148400640 1 -- 172.21.15.32:0/1001015886 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x55e5bcb5ec00 con 0x55e5bd639800 2024-04-17T10:00:44.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.077+0000 7f1148400640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55e5bd7d7c00 0x55e5bd628100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.077+0000 7f1148400640 1 -- 172.21.15.32:0/1001015886 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x55e5bcb54c00 con 0x55e5bd639800 2024-04-17T10:00:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.077+0000 7f1151c028c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-17T10:00:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.077+0000 7f1151c028c0 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-04-17T10:00:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.077+0000 7f1151c028c0 4 datascan.main: opening data pool 'cephfs_data' 2024-04-17T10:00:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.077+0000 7f1151c028c0 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55e5bcc4e000 0x55e5bd628680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.077+0000 7f1150600640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55e5bd7d7c00 0x55e5bd628100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.077+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc4e400 con 0x55e5bcc4e000 2024-04-17T10:00:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.077+0000 7f114f200640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55e5bcc4e000 0x55e5bd628680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.077+0000 7f1150600640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55e5bd7d7c00 0x55e5bd628100 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x55e5bd78c660 tx=0x55e5bc930060 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.077+0000 7f114f200640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55e5bcc4e000 0x55e5bd628680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.077+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.5 v2:172.21.15.144:6800/2379578648 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv25 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4e000 2024-04-17T10:00:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.077+0000 7f1151c028c0 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55e5bcc4e800 0x55e5bd628c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.077+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:2 12.1f 12:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc4ec00 con 0x55e5bcc4e800 2024-04-17T10:00:44.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.077+0000 7f1150600640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55e5bcc4e800 0x55e5bd628c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.077+0000 7f1150600640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55e5bcc4e800 0x55e5bd628c00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.077+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.6 v2:172.21.15.144:6808/3198738696 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc4e800 2024-04-17T10:00:44.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.077+0000 7f1151c028c0 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55e5bcc4f000 0x55e5bd629180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.077+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:3 13.0 13:00000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc4f400 con 0x55e5bcc4f000 2024-04-17T10:00:44.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.077+0000 7f114fc00640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55e5bcc4f000 0x55e5bd629180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.077+0000 7f114fc00640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55e5bcc4f000 0x55e5bd629180 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.081+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.7 v2:172.21.15.144:6824/1746570140 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc4f000 2024-04-17T10:00:44.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.081+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:4 13.20 13:04000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc58000 con 0x55e5bcc4e800 2024-04-17T10:00:44.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.081+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.6 v2:172.21.15.144:6808/3198738696 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc4e800 2024-04-17T10:00:44.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.081+0000 7f114a200640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55e5bcc58400 0x55e5bcc5e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.081+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:5 13.10 13:08000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc58800 con 0x55e5bcc58400 2024-04-17T10:00:44.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.081+0000 7f114f200640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55e5bcc58400 0x55e5bcc5e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.081+0000 7f114f200640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55e5bcc58400 0x55e5bcc5e000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.081+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.1 v2:172.21.15.32:6808/2454993455 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc58400 2024-04-17T10:00:44.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.081+0000 7f1149800640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55e5bd643000 0x55e5bcc5e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.081+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:6 13.30 13:0c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd642c00 con 0x55e5bd643000 2024-04-17T10:00:44.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.081+0000 7f1150600640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55e5bd643000 0x55e5bcc5e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.081+0000 7f1150600640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55e5bd643000 0x55e5bcc5e580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.081+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.4 v2:172.21.15.144:6809/4076371964 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bd643000 2024-04-17T10:00:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.081+0000 7f114a200640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55e5bcc58c00 0x55e5bcc5eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.081+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:7 13.8 13:10000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd642c00 con 0x55e5bcc58c00 2024-04-17T10:00:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.081+0000 7f114fc00640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55e5bcc58c00 0x55e5bcc5eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.085+0000 7f114fc00640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55e5bcc58c00 0x55e5bcc5eb00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.085+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.2 v2:172.21.15.32:6809/1954833462 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc58c00 2024-04-17T10:00:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.085+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:8 13.28 13:14000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc58000 con 0x55e5bcc58c00 2024-04-17T10:00:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.085+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.2 v2:172.21.15.32:6809/1954833462 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc58c00 2024-04-17T10:00:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.085+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:9 13.18 13:18000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd642400 con 0x55e5bcc4f000 2024-04-17T10:00:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.085+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.7 v2:172.21.15.144:6824/1746570140 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc4f000 2024-04-17T10:00:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.085+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:10 13.38 13:1c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc4ec00 con 0x55e5bcc4e000 2024-04-17T10:00:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.085+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.5 v2:172.21.15.144:6800/2379578648 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4e000 2024-04-17T10:00:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.085+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:11 13.4 13:20000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd7d6800 con 0x55e5bcc4e800 2024-04-17T10:00:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.085+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.6 v2:172.21.15.144:6808/3198738696 3 ==== osd_op_reply(11 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc4e800 2024-04-17T10:00:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.085+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:12 13.24 13:24000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc4e400 con 0x55e5bcc4e800 2024-04-17T10:00:44.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.085+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.6 v2:172.21.15.144:6808/3198738696 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc4e800 2024-04-17T10:00:44.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.085+0000 7f114a200640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55e5bcc4f400 0x55e5bcc5f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.085+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:13 13.14 13:28000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd7d7800 con 0x55e5bcc4f400 2024-04-17T10:00:44.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.085+0000 7f114f200640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55e5bcc4f400 0x55e5bcc5f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.085+0000 7f114f200640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55e5bcc4f400 0x55e5bcc5f080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.089+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.0 v2:172.21.15.32:6800/2392056658 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4f400 2024-04-17T10:00:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.089+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:14 13.34 13:2c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc58000 con 0x55e5bcc4e800 2024-04-17T10:00:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.089+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.6 v2:172.21.15.144:6808/3198738696 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc4e800 2024-04-17T10:00:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.089+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:15 13.c 13:30000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd7d7400 con 0x55e5bcc58400 2024-04-17T10:00:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.089+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.1 v2:172.21.15.32:6808/2454993455 2 ==== osd_op_reply(15 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc58400 2024-04-17T10:00:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.089+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:16 13.2c 13:34000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc58800 con 0x55e5bcc58c00 2024-04-17T10:00:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.089+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.2 v2:172.21.15.32:6809/1954833462 3 ==== osd_op_reply(16 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc58c00 2024-04-17T10:00:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.089+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:17 13.1c 13:38000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd7d7000 con 0x55e5bcc4e800 2024-04-17T10:00:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.089+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.6 v2:172.21.15.144:6808/3198738696 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc4e800 2024-04-17T10:00:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.089+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:18 13.3c 13:3c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc59000 con 0x55e5bcc58c00 2024-04-17T10:00:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.089+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.2 v2:172.21.15.32:6809/1954833462 4 ==== osd_op_reply(18 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc58c00 2024-04-17T10:00:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.089+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:19 13.2 13:40000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc59400 con 0x55e5bcc4e000 2024-04-17T10:00:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.089+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.5 v2:172.21.15.144:6800/2379578648 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4e000 2024-04-17T10:00:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.089+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:20 13.22 13:44000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd642c00 con 0x55e5bcc58400 2024-04-17T10:00:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.089+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.1 v2:172.21.15.32:6808/2454993455 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc58400 2024-04-17T10:00:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.089+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:21 13.12 13:48000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc59800 con 0x55e5bcc4f400 2024-04-17T10:00:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.089+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.0 v2:172.21.15.32:6800/2392056658 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4f400 2024-04-17T10:00:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.089+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:22 13.32 13:4c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc59400 con 0x55e5bcc58c00 2024-04-17T10:00:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.089+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.2 v2:172.21.15.32:6809/1954833462 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc58c00 2024-04-17T10:00:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.089+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:23 13.a 13:50000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc59c00 con 0x55e5bcc58c00 2024-04-17T10:00:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.089+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.2 v2:172.21.15.32:6809/1954833462 6 ==== osd_op_reply(23 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc58c00 2024-04-17T10:00:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.089+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:24 13.2a 13:54000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd642c00 con 0x55e5bcc58400 2024-04-17T10:00:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.093+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.1 v2:172.21.15.32:6808/2454993455 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc58400 2024-04-17T10:00:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.093+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:25 13.1a 13:58000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc59800 con 0x55e5bcc58400 2024-04-17T10:00:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.093+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.1 v2:172.21.15.32:6808/2454993455 5 ==== osd_op_reply(25 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc58400 2024-04-17T10:00:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.093+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:26 13.3a 13:5c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd7d7800 con 0x55e5bcc4e000 2024-04-17T10:00:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.093+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.5 v2:172.21.15.144:6800/2379578648 4 ==== osd_op_reply(26 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4e000 2024-04-17T10:00:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.093+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:27 13.6 13:60000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc76000 con 0x55e5bd643000 2024-04-17T10:00:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.093+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.4 v2:172.21.15.144:6809/4076371964 2 ==== osd_op_reply(27 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bd643000 2024-04-17T10:00:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.093+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:28 13.26 13:64000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd7d7400 con 0x55e5bcc4f000 2024-04-17T10:00:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.093+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.7 v2:172.21.15.144:6824/1746570140 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc4f000 2024-04-17T10:00:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.093+0000 7f114a200640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55e5bcc76400 0x55e5bcc5f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.093+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:29 13.16 13:68000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc76800 con 0x55e5bcc76400 2024-04-17T10:00:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.093+0000 7f1150600640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55e5bcc76400 0x55e5bcc5f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.093+0000 7f1150600640 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55e5bcc76400 0x55e5bcc5f600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.093+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.3 v2:172.21.15.32:6824/2182899298 1 ==== osd_op_reply(29 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc76400 2024-04-17T10:00:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.093+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:30 13.36 13:6c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc4ec00 con 0x55e5bcc4e800 2024-04-17T10:00:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.093+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.6 v2:172.21.15.144:6808/3198738696 7 ==== osd_op_reply(30 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc4e800 2024-04-17T10:00:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.093+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:31 13.e 13:70000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc76c00 con 0x55e5bcc4f000 2024-04-17T10:00:44.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.093+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.7 v2:172.21.15.144:6824/1746570140 4 ==== osd_op_reply(31 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc4f000 2024-04-17T10:00:44.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.093+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:32 13.2e 13:74000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc58800 con 0x55e5bcc58c00 2024-04-17T10:00:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.093+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.2 v2:172.21.15.32:6809/1954833462 7 ==== osd_op_reply(32 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc58c00 2024-04-17T10:00:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.093+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:33 13.1e 13:78000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc77000 con 0x55e5bd643000 2024-04-17T10:00:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.097+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.4 v2:172.21.15.144:6809/4076371964 3 ==== osd_op_reply(33 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bd643000 2024-04-17T10:00:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.097+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:34 13.3e 13:7c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc59000 con 0x55e5bcc58400 2024-04-17T10:00:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.097+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.1 v2:172.21.15.32:6808/2454993455 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc58400 2024-04-17T10:00:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.097+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:35 13.1 13:80000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc77400 con 0x55e5bcc4e800 2024-04-17T10:00:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.097+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.6 v2:172.21.15.144:6808/3198738696 8 ==== osd_op_reply(35 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc4e800 2024-04-17T10:00:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.097+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:36 13.21 13:84000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc59400 con 0x55e5bd643000 2024-04-17T10:00:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.097+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.4 v2:172.21.15.144:6809/4076371964 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 86 out=49b] v82'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bd643000 2024-04-17T10:00:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.097+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:37 13.11 13:88000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc77800 con 0x55e5bcc76400 2024-04-17T10:00:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.097+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.3 v2:172.21.15.32:6824/2182899298 2 ==== osd_op_reply(37 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc76400 2024-04-17T10:00:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.097+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:38 13.31 13:8c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc59c00 con 0x55e5bcc76400 2024-04-17T10:00:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.097+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.3 v2:172.21.15.32:6824/2182899298 3 ==== osd_op_reply(38 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc76400 2024-04-17T10:00:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.097+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:39 13.9 13:90000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc76800 con 0x55e5bcc4e000 2024-04-17T10:00:44.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.097+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.5 v2:172.21.15.144:6800/2379578648 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 86 out=49b] v82'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4e000 2024-04-17T10:00:44.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.097+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:40 13.29 13:94000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc4e400 con 0x55e5bcc4e800 2024-04-17T10:00:44.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.097+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.6 v2:172.21.15.144:6808/3198738696 9 ==== osd_op_reply(40 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc4e800 2024-04-17T10:00:44.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.097+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:41 13.19 13:98000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc77000 con 0x55e5bcc76400 2024-04-17T10:00:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.097+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.3 v2:172.21.15.32:6824/2182899298 4 ==== osd_op_reply(41 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc76400 2024-04-17T10:00:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.097+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:42 13.39 13:9c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd642400 con 0x55e5bcc58c00 2024-04-17T10:00:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.097+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.2 v2:172.21.15.32:6809/1954833462 8 ==== osd_op_reply(42 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc58c00 2024-04-17T10:00:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.097+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:43 13.5 13:a0000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd7d7400 con 0x55e5bd643000 2024-04-17T10:00:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.097+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.4 v2:172.21.15.144:6809/4076371964 5 ==== osd_op_reply(43 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bd643000 2024-04-17T10:00:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.097+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:44 13.25 13:a4000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc77400 con 0x55e5bd643000 2024-04-17T10:00:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.101+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.4 v2:172.21.15.144:6809/4076371964 6 ==== osd_op_reply(44 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bd643000 2024-04-17T10:00:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.101+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:45 13.15 13:a8000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd7d6800 con 0x55e5bd643000 2024-04-17T10:00:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.101+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.4 v2:172.21.15.144:6809/4076371964 7 ==== osd_op_reply(45 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bd643000 2024-04-17T10:00:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.101+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:46 13.35 13:ac000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc77800 con 0x55e5bcc58400 2024-04-17T10:00:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.101+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.1 v2:172.21.15.32:6808/2454993455 7 ==== osd_op_reply(46 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc58400 2024-04-17T10:00:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.101+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:47 13.d 13:b0000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd7d7000 con 0x55e5bd643000 2024-04-17T10:00:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.101+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.4 v2:172.21.15.144:6809/4076371964 8 ==== osd_op_reply(47 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bd643000 2024-04-17T10:00:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.101+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:48 13.2d 13:b4000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc76c00 con 0x55e5bcc4f400 2024-04-17T10:00:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.101+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.0 v2:172.21.15.32:6800/2392056658 3 ==== osd_op_reply(48 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4f400 2024-04-17T10:00:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.101+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:49 13.1d 13:b8000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc59400 con 0x55e5bcc4f000 2024-04-17T10:00:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.101+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.7 v2:172.21.15.144:6824/1746570140 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc4f000 2024-04-17T10:00:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.101+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:50 13.3d 13:bc000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc76000 con 0x55e5bcc4f000 2024-04-17T10:00:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.101+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.7 v2:172.21.15.144:6824/1746570140 6 ==== osd_op_reply(50 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc4f000 2024-04-17T10:00:44.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.101+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:51 13.3 13:c0000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc58000 con 0x55e5bcc4f400 2024-04-17T10:00:44.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.101+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.0 v2:172.21.15.32:6800/2392056658 4 ==== osd_op_reply(51 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4f400 2024-04-17T10:00:44.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.101+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:52 13.23 13:c4000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc77c00 con 0x55e5bcc58400 2024-04-17T10:00:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.101+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.1 v2:172.21.15.32:6808/2454993455 8 ==== osd_op_reply(52 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc58400 2024-04-17T10:00:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.101+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:53 13.13 13:c8000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc4ec00 con 0x55e5bcc4f000 2024-04-17T10:00:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.101+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.7 v2:172.21.15.144:6824/1746570140 7 ==== osd_op_reply(53 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc4f000 2024-04-17T10:00:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.101+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:54 13.33 13:cc000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc76800 con 0x55e5bcc4f000 2024-04-17T10:00:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.101+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.7 v2:172.21.15.144:6824/1746570140 8 ==== osd_op_reply(54 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc4f000 2024-04-17T10:00:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.101+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:55 13.b 13:d0000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd7d7800 con 0x55e5bcc4e000 2024-04-17T10:00:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.105+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.5 v2:172.21.15.144:6800/2379578648 6 ==== osd_op_reply(55 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4e000 2024-04-17T10:00:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.105+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:56 13.2b 13:d4000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc77800 con 0x55e5bcc58c00 2024-04-17T10:00:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.105+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.2 v2:172.21.15.32:6809/1954833462 9 ==== osd_op_reply(56 [pgnls-filter start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc58c00 2024-04-17T10:00:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.105+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:57 13.1b 13:d8000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd7d7400 con 0x55e5bd643000 2024-04-17T10:00:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.105+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.4 v2:172.21.15.144:6809/4076371964 9 ==== osd_op_reply(57 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bd643000 2024-04-17T10:00:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.105+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:58 13.3b 13:dc000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc76c00 con 0x55e5bd643000 2024-04-17T10:00:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.105+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.4 v2:172.21.15.144:6809/4076371964 10 ==== osd_op_reply(58 [pgnls-filter start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bd643000 2024-04-17T10:00:44.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.105+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:59 13.7 13:e0000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd7d6800 con 0x55e5bcc4f000 2024-04-17T10:00:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.105+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.7 v2:172.21.15.144:6824/1746570140 9 ==== osd_op_reply(59 [pgnls-filter start_epoch 86 out=81b] v86'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc4f000 2024-04-17T10:00:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.105+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:60 13.0 13:00000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc4f800 con 0x55e5bcc4f000 2024-04-17T10:00:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.105+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.7 v2:172.21.15.144:6824/1746570140 10 ==== osd_op_reply(60 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc4f000 2024-04-17T10:00:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.105+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:61 13.20 13:04000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd7d7000 con 0x55e5bcc4e800 2024-04-17T10:00:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.105+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.6 v2:172.21.15.144:6808/3198738696 10 ==== osd_op_reply(61 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc4e800 2024-04-17T10:00:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.105+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:62 13.10 13:08000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc4e400 con 0x55e5bcc58400 2024-04-17T10:00:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.105+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.1 v2:172.21.15.32:6808/2454993455 9 ==== osd_op_reply(62 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc58400 2024-04-17T10:00:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.105+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:63 13.30 13:0c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc77000 con 0x55e5bd643000 2024-04-17T10:00:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.105+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.4 v2:172.21.15.144:6809/4076371964 11 ==== osd_op_reply(63 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bd643000 2024-04-17T10:00:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.105+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:64 13.8 13:10000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc59800 con 0x55e5bcc58c00 2024-04-17T10:00:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.109+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.2 v2:172.21.15.32:6809/1954833462 10 ==== osd_op_reply(64 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc58c00 2024-04-17T10:00:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.109+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:65 13.28 13:14000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc77400 con 0x55e5bcc58c00 2024-04-17T10:00:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.109+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.2 v2:172.21.15.32:6809/1954833462 11 ==== osd_op_reply(65 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc58c00 2024-04-17T10:00:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.109+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:66 13.18 13:18000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc59000 con 0x55e5bcc4f000 2024-04-17T10:00:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.109+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.7 v2:172.21.15.144:6824/1746570140 11 ==== osd_op_reply(66 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc4f000 2024-04-17T10:00:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.109+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:67 13.38 13:1c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd7d6800 con 0x55e5bcc4e000 2024-04-17T10:00:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.109+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.5 v2:172.21.15.144:6800/2379578648 7 ==== osd_op_reply(67 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4e000 2024-04-17T10:00:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.109+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:68 13.4 13:20000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc59c00 con 0x55e5bcc4e800 2024-04-17T10:00:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.109+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.6 v2:172.21.15.144:6808/3198738696 11 ==== osd_op_reply(68 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc4e800 2024-04-17T10:00:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.109+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:69 13.24 13:24000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc76000 con 0x55e5bcc4e800 2024-04-17T10:00:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.109+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.6 v2:172.21.15.144:6808/3198738696 12 ==== osd_op_reply(69 [pgnls-filter start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc4e800 2024-04-17T10:00:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.109+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:70 13.14 13:28000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd642c00 con 0x55e5bcc4f400 2024-04-17T10:00:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.109+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.0 v2:172.21.15.32:6800/2392056658 5 ==== osd_op_reply(70 [pgnls-filter start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4f400 2024-04-17T10:00:44.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.109+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:71 13.34 13:2c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc76800 con 0x55e5bcc4e800 2024-04-17T10:00:44.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.109+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.6 v2:172.21.15.144:6808/3198738696 13 ==== osd_op_reply(71 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc4e800 2024-04-17T10:00:44.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.109+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:72 13.c 13:30000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc84000 con 0x55e5bcc58400 2024-04-17T10:00:44.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.109+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.1 v2:172.21.15.32:6808/2454993455 10 ==== osd_op_reply(72 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc58400 2024-04-17T10:00:44.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.109+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:73 13.2c 13:34000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc77800 con 0x55e5bcc58c00 2024-04-17T10:00:44.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.109+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.2 v2:172.21.15.32:6809/1954833462 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc58c00 2024-04-17T10:00:44.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.109+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:74 13.1c 13:38000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc84400 con 0x55e5bcc4e800 2024-04-17T10:00:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.109+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.6 v2:172.21.15.144:6808/3198738696 14 ==== osd_op_reply(74 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc4e800 2024-04-17T10:00:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.109+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:75 13.3c 13:3c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc4ec00 con 0x55e5bcc58c00 2024-04-17T10:00:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.113+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.2 v2:172.21.15.32:6809/1954833462 13 ==== osd_op_reply(75 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc58c00 2024-04-17T10:00:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.113+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:76 13.2 13:40000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc84800 con 0x55e5bcc4e000 2024-04-17T10:00:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.113+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.5 v2:172.21.15.144:6800/2379578648 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4e000 2024-04-17T10:00:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.113+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:77 13.22 13:44000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc58800 con 0x55e5bcc58400 2024-04-17T10:00:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.113+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.1 v2:172.21.15.32:6808/2454993455 11 ==== osd_op_reply(77 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc58400 2024-04-17T10:00:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.113+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:78 13.12 13:48000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc84c00 con 0x55e5bcc4f400 2024-04-17T10:00:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.113+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.0 v2:172.21.15.32:6800/2392056658 6 ==== osd_op_reply(78 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4f400 2024-04-17T10:00:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.113+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:79 13.32 13:4c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc59400 con 0x55e5bcc58c00 2024-04-17T10:00:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.113+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.2 v2:172.21.15.32:6809/1954833462 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc58c00 2024-04-17T10:00:44.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.113+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:80 13.a 13:50000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc85000 con 0x55e5bcc58c00 2024-04-17T10:00:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.113+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.2 v2:172.21.15.32:6809/1954833462 15 ==== osd_op_reply(80 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc58c00 2024-04-17T10:00:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.113+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:81 13.2a 13:54000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd642400 con 0x55e5bcc58400 2024-04-17T10:00:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.113+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.1 v2:172.21.15.32:6808/2454993455 12 ==== osd_op_reply(81 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc58400 2024-04-17T10:00:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.113+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:82 13.1a 13:58000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc85400 con 0x55e5bcc58400 2024-04-17T10:00:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.113+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.1 v2:172.21.15.32:6808/2454993455 13 ==== osd_op_reply(82 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc58400 2024-04-17T10:00:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.113+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:83 13.3a 13:5c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc77c00 con 0x55e5bcc4e000 2024-04-17T10:00:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.113+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.5 v2:172.21.15.144:6800/2379578648 9 ==== osd_op_reply(83 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4e000 2024-04-17T10:00:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.113+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:84 13.6 13:60000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc85800 con 0x55e5bd643000 2024-04-17T10:00:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.113+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.4 v2:172.21.15.144:6809/4076371964 12 ==== osd_op_reply(84 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bd643000 2024-04-17T10:00:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.113+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:85 13.26 13:64000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc76c00 con 0x55e5bcc4f000 2024-04-17T10:00:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.113+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.7 v2:172.21.15.144:6824/1746570140 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc4f000 2024-04-17T10:00:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.113+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:86 13.16 13:68000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc85c00 con 0x55e5bcc76400 2024-04-17T10:00:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.117+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.3 v2:172.21.15.32:6824/2182899298 5 ==== osd_op_reply(86 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc76400 2024-04-17T10:00:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.117+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:87 13.36 13:6c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc77000 con 0x55e5bcc4e800 2024-04-17T10:00:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.117+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.6 v2:172.21.15.144:6808/3198738696 15 ==== osd_op_reply(87 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc4e800 2024-04-17T10:00:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.117+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:88 13.e 13:70000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc77400 con 0x55e5bcc4f000 2024-04-17T10:00:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.117+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.7 v2:172.21.15.144:6824/1746570140 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc4f000 2024-04-17T10:00:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.117+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:89 13.2e 13:74000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc58000 con 0x55e5bcc58c00 2024-04-17T10:00:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.117+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.2 v2:172.21.15.32:6809/1954833462 16 ==== osd_op_reply(89 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc58c00 2024-04-17T10:00:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.117+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:90 13.1e 13:78000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc77800 con 0x55e5bd643000 2024-04-17T10:00:44.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.117+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.4 v2:172.21.15.144:6809/4076371964 13 ==== osd_op_reply(90 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bd643000 2024-04-17T10:00:44.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.117+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:91 13.3e 13:7c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc59c00 con 0x55e5bcc58400 2024-04-17T10:00:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.117+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.1 v2:172.21.15.32:6808/2454993455 14 ==== osd_op_reply(91 [pgnls-filter start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc58400 2024-04-17T10:00:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.117+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:92 13.1 13:80000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc76c00 con 0x55e5bcc4e800 2024-04-17T10:00:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.117+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.6 v2:172.21.15.144:6808/3198738696 16 ==== osd_op_reply(92 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc4e800 2024-04-17T10:00:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.117+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:93 13.21 13:84000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd7d7800 con 0x55e5bd643000 2024-04-17T10:00:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.117+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.4 v2:172.21.15.144:6809/4076371964 14 ==== osd_op_reply(93 [pgnls-filter start_epoch 86 out=49b] v82'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bd643000 2024-04-17T10:00:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.117+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:94 13.11 13:88000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc76000 con 0x55e5bcc76400 2024-04-17T10:00:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.117+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.3 v2:172.21.15.32:6824/2182899298 6 ==== osd_op_reply(94 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc76400 2024-04-17T10:00:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.117+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:95 13.31 13:8c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd7d6800 con 0x55e5bcc76400 2024-04-17T10:00:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.117+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.3 v2:172.21.15.32:6824/2182899298 7 ==== osd_op_reply(95 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc76400 2024-04-17T10:00:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.117+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:96 13.9 13:90000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc76800 con 0x55e5bcc4e000 2024-04-17T10:00:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.117+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.5 v2:172.21.15.144:6800/2379578648 10 ==== osd_op_reply(96 [pgnls-filter start_epoch 86 out=49b] v82'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4e000 2024-04-17T10:00:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.117+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:97 13.29 13:94000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd7d7400 con 0x55e5bcc4e800 2024-04-17T10:00:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.121+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.6 v2:172.21.15.144:6808/3198738696 17 ==== osd_op_reply(97 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc4e800 2024-04-17T10:00:44.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.121+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:98 13.19 13:98000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc59800 con 0x55e5bcc76400 2024-04-17T10:00:44.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.121+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.3 v2:172.21.15.32:6824/2182899298 8 ==== osd_op_reply(98 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc76400 2024-04-17T10:00:44.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.121+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:99 13.39 13:9c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd7d7000 con 0x55e5bcc58c00 2024-04-17T10:00:44.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.121+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.2 v2:172.21.15.32:6809/1954833462 17 ==== osd_op_reply(99 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc58c00 2024-04-17T10:00:44.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.121+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:100 13.5 13:a0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc59000 con 0x55e5bd643000 2024-04-17T10:00:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.121+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.4 v2:172.21.15.144:6809/4076371964 15 ==== osd_op_reply(100 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bd643000 2024-04-17T10:00:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.121+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:101 13.25 13:a4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd642c00 con 0x55e5bd643000 2024-04-17T10:00:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.121+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.4 v2:172.21.15.144:6809/4076371964 16 ==== osd_op_reply(101 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bd643000 2024-04-17T10:00:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.121+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:102 13.15 13:a8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc59400 con 0x55e5bd643000 2024-04-17T10:00:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.121+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.4 v2:172.21.15.144:6809/4076371964 17 ==== osd_op_reply(102 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bd643000 2024-04-17T10:00:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.121+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:103 13.35 13:ac000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc4e400 con 0x55e5bcc58400 2024-04-17T10:00:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.121+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.1 v2:172.21.15.32:6808/2454993455 15 ==== osd_op_reply(103 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc58400 2024-04-17T10:00:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.121+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:104 13.d 13:b0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc4f800 con 0x55e5bd643000 2024-04-17T10:00:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.121+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.4 v2:172.21.15.144:6809/4076371964 18 ==== osd_op_reply(104 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bd643000 2024-04-17T10:00:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.121+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:105 13.2d 13:b4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc85800 con 0x55e5bcc4f400 2024-04-17T10:00:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.121+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.0 v2:172.21.15.32:6800/2392056658 7 ==== osd_op_reply(105 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4f400 2024-04-17T10:00:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.121+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:106 13.1d 13:b8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc4ec00 con 0x55e5bcc4f000 2024-04-17T10:00:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.121+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.7 v2:172.21.15.144:6824/1746570140 14 ==== osd_op_reply(106 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc4f000 2024-04-17T10:00:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.121+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:107 13.3d 13:bc000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc85c00 con 0x55e5bcc4f000 2024-04-17T10:00:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.125+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.7 v2:172.21.15.144:6824/1746570140 15 ==== osd_op_reply(107 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc4f000 2024-04-17T10:00:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.125+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:108 13.3 13:c0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc84c00 con 0x55e5bcc4f400 2024-04-17T10:00:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.125+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.0 v2:172.21.15.32:6800/2392056658 8 ==== osd_op_reply(108 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4f400 2024-04-17T10:00:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.125+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:109 13.23 13:c4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc85000 con 0x55e5bcc58400 2024-04-17T10:00:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.125+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.1 v2:172.21.15.32:6808/2454993455 16 ==== osd_op_reply(109 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc58400 2024-04-17T10:00:44.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.125+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:110 13.13 13:c8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc85400 con 0x55e5bcc4f000 2024-04-17T10:00:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.125+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.7 v2:172.21.15.144:6824/1746570140 16 ==== osd_op_reply(110 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc4f000 2024-04-17T10:00:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.125+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:111 13.33 13:cc000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc84000 con 0x55e5bcc4f000 2024-04-17T10:00:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.125+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.7 v2:172.21.15.144:6824/1746570140 17 ==== osd_op_reply(111 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc4f000 2024-04-17T10:00:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.125+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:112 13.b 13:d0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc85800 con 0x55e5bcc4e000 2024-04-17T10:00:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.125+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.5 v2:172.21.15.144:6800/2379578648 11 ==== osd_op_reply(112 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4e000 2024-04-17T10:00:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.125+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:113 13.2b 13:d4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc84800 con 0x55e5bcc58c00 2024-04-17T10:00:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.125+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.2 v2:172.21.15.32:6809/1954833462 18 ==== osd_op_reply(113 [pgnls-filter start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc58c00 2024-04-17T10:00:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.125+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:114 13.1b 13:d8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc84400 con 0x55e5bd643000 2024-04-17T10:00:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.125+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.4 v2:172.21.15.144:6809/4076371964 19 ==== osd_op_reply(114 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bd643000 2024-04-17T10:00:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.125+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:115 13.3b 13:dc000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd7d7800 con 0x55e5bd643000 2024-04-17T10:00:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.125+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.4 v2:172.21.15.144:6809/4076371964 20 ==== osd_op_reply(115 [pgnls-filter start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bd643000 2024-04-17T10:00:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.125+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:116 13.7 13:e0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc4e400 con 0x55e5bcc4f000 2024-04-17T10:00:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.125+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.7 v2:172.21.15.144:6824/1746570140 18 ==== osd_op_reply(116 [pgnls-filter start_epoch 86 out=81b] v86'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc4f000 2024-04-17T10:00:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.125+0000 7f1151c028c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-04-17T10:00:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.125+0000 7f1151c028c0 10 datascan.operator(): handling object 10000000001.0 2024-04-17T10:00:44.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.125+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:117 13.7 13:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc94000 con 0x55e5bcc4f000 2024-04-17T10:00:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.129+0000 7f114fc00640 1 -- 172.21.15.32:0/1001015886 <== osd.7 v2:172.21.15.144:6824/1746570140 19 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) ==== 416+0+157 (crc 0 0 0) 0x55e5bc964780 con 0x55e5bcc4f000 2024-04-17T10:00:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.129+0000 7f1151c028c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-04-17T10:00:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.129+0000 7f1151c028c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-04-17T10:00:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.129+0000 7f1151c028c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-04-17T10:00:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.129+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:118 12.10 12:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc94400 con 0x55e5bcc4f400 2024-04-17T10:00:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.129+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.0 v2:172.21.15.32:6800/2392056658 9 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) ==== 164+0+59 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4f400 2024-04-17T10:00:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.129+0000 7f1151c028c0 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-04-17T10:00:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.129+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc94800 con 0x55e5bcc4e800 2024-04-17T10:00:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.129+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.6 v2:172.21.15.144:6808/3198738696 18 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=520b] v0'0 uv9 ondisk = 0) ==== 154+0+520 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc4e800 2024-04-17T10:00:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.129+0000 7f1151c028c0 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-04-17T10:00:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.129+0000 7f1151c028c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-04-17T10:00:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.129+0000 7f1151c028c0 20 datascan.get_frag_of: frag is * 2024-04-17T10:00:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.129+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:120 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc94c00 con 0x55e5bcc4f400 2024-04-17T10:00:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.129+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.0 v2:172.21.15.32:6800/2392056658 10 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4f400 2024-04-17T10:00:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.129+0000 7f1151c028c0 10 datascan.find_or_create_dirfrag: read_version = 17 2024-04-17T10:00:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.129+0000 7f1151c028c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-04-17T10:00:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.129+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:121 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc95000 con 0x55e5bcc4f400 2024-04-17T10:00:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.129+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.0 v2:172.21.15.32:6800/2392056658 11 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4f400 2024-04-17T10:00:44.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.129+0000 7f1151c028c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-17T10:00:44.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.129+0000 7f1151c028c0 20 datascan.read_dentry: file_head not found in result 2024-04-17T10:00:44.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.129+0000 7f1151c028c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-04-17T10:00:44.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.129+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:122 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc95400 con 0x55e5bcc4f400 2024-04-17T10:00:44.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.0 v2:172.21.15.32:6800/2392056658 12 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v86'18 uv18 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4f400 2024-04-17T10:00:44.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-17T10:00:44.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:123 13.27 13:e4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc95800 con 0x55e5bcc58400 2024-04-17T10:00:44.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.1 v2:172.21.15.32:6808/2454993455 17 ==== osd_op_reply(123 [pgnls-filter start_epoch 86 out=49b] v82'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc58400 2024-04-17T10:00:44.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:124 13.17 13:e8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc77c00 con 0x55e5bcc4e000 2024-04-17T10:00:44.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.5 v2:172.21.15.144:6800/2379578648 12 ==== osd_op_reply(124 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4e000 2024-04-17T10:00:44.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:125 13.37 13:ec000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd7d6800 con 0x55e5bcc4e000 2024-04-17T10:00:44.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 <== osd.5 v2:172.21.15.144:6800/2379578648 13 ==== osd_op_reply(125 [pgnls-filter start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964000 con 0x55e5bcc4e000 2024-04-17T10:00:44.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:126 13.f 13:f0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc76800 con 0x55e5bcc4e800 2024-04-17T10:00:44.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.6 v2:172.21.15.144:6808/3198738696 19 ==== osd_op_reply(126 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc4e800 2024-04-17T10:00:44.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:127 13.2f 13:f4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bd7d7400 con 0x55e5bd643000 2024-04-17T10:00:44.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.4 v2:172.21.15.144:6809/4076371964 21 ==== osd_op_reply(127 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bd643000 2024-04-17T10:00:44.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1149800640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:128 13.1f 13:f8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc58800 con 0x55e5bcc4e800 2024-04-17T10:00:44.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.6 v2:172.21.15.144:6808/3198738696 20 ==== osd_op_reply(128 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc4e800 2024-04-17T10:00:44.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f114a200640 1 -- 172.21.15.32:0/1001015886 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:129 13.3f 13:fc000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e5bcc77000 con 0x55e5bcc4e800 2024-04-17T10:00:44.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1150600640 1 -- 172.21.15.32:0/1001015886 <== osd.6 v2:172.21.15.144:6808/3198738696 21 ==== osd_op_reply(129 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e5bc964500 con 0x55e5bcc4e800 2024-04-17T10:00:44.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55e5bcc4f400 msgr2=0x55e5bcc5f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55e5bcc4f400 0x55e5bcc5f080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55e5bcc58400 msgr2=0x55e5bcc5e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55e5bcc58400 0x55e5bcc5e000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55e5bcc58c00 msgr2=0x55e5bcc5eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55e5bcc58c00 0x55e5bcc5eb00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55e5bcc76400 msgr2=0x55e5bcc5f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55e5bcc76400 0x55e5bcc5f600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55e5bd643000 msgr2=0x55e5bcc5e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55e5bd643000 0x55e5bcc5e580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55e5bcc4e000 msgr2=0x55e5bd628680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55e5bcc4e000 0x55e5bd628680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55e5bcc4e800 msgr2=0x55e5bd628c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55e5bcc4e800 0x55e5bd628c00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55e5bcc4f000 msgr2=0x55e5bd629180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55e5bcc4f000 0x55e5bd629180 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 reap_dead start 2024-04-17T10:00:44.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 reap_dead start 2024-04-17T10:00:44.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f114f200640 1 -- 172.21.15.32:0/1001015886 reap_dead start 2024-04-17T10:00:44.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55e5bd7d7c00 msgr2=0x55e5bd628100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55e5bd7d7c00 0x55e5bd628100 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x55e5bd78c660 tx=0x55e5bc930060 comp rx=0 tx=0).stop 2024-04-17T10:00:44.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e5bd639800 msgr2=0x55e5bd627600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e5bd639800 0x55e5bd627600 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x55e5bd78d980 tx=0x55e5bc930f00 comp rx=0 tx=0).stop 2024-04-17T10:00:44.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 shutdown_connections 2024-04-17T10:00:44.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55e5bd7d7c00 0x55e5bd628100 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 --2- 172.21.15.32:0/1001015886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e5bd639800 0x55e5bd627600 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 >> 172.21.15.32:0/1001015886 conn(0x55e5bd639000 msgr2=0x55e5bc87c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:44.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 shutdown_connections 2024-04-17T10:00:44.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 -- 172.21.15.32:0/1001015886 wait complete. 2024-04-17T10:00:44.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 -- 172.21.15.32:0/3163904938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e5bc84f000 msgr2=0x55e5bd626000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.133+0000 7f1151c028c0 1 --2- 172.21.15.32:0/3163904938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e5bc84f000 0x55e5bd626000 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x55e5bd6af4a0 tx=0x55e5bc930fc0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.137+0000 7f1151c028c0 1 -- 172.21.15.32:0/3163904938 shutdown_connections 2024-04-17T10:00:44.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.137+0000 7f1151c028c0 1 --2- 172.21.15.32:0/3163904938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e5bc84f400 0x55e5bd626580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.137+0000 7f1151c028c0 1 --2- 172.21.15.32:0/3163904938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5bc84f800 0x55e5bd626b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.137+0000 7f1151c028c0 1 --2- 172.21.15.32:0/3163904938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e5bc84f000 0x55e5bd626000 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.137+0000 7f1151c028c0 1 -- 172.21.15.32:0/3163904938 >> 172.21.15.32:0/3163904938 conn(0x55e5bc84ec00 msgr2=0x55e5bc87b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:44.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.137+0000 7f1151c028c0 1 -- 172.21.15.32:0/3163904938 shutdown_connections 2024-04-17T10:00:44.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.137+0000 7f1151c028c0 1 -- 172.21.15.32:0/3163904938 wait complete. 2024-04-17T10:00:44.138 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.167859, result: 2024-04-17T10:00:44.138 DEBUG:teuthology.orchestra.run.smithi032:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-17T10:00:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae96668c0 1 -- start start 2024-04-17T10:00:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae96668c0 1 Processor -- start 2024-04-17T10:00:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae96668c0 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e8b2219800 0x55e8b2fe2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae96668c0 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8b2219400 0x55e8b2fe2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae96668c0 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8b2219000 0x55e8b2fe2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae96668c0 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e8b231c180 con 0x55e8b2219800 2024-04-17T10:00:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae96668c0 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e8b2278900 con 0x55e8b2219000 2024-04-17T10:00:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae96668c0 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e8b2278480 con 0x55e8b2219400 2024-04-17T10:00:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae7800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e8b2219800 0x55e8b2fe2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae7800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e8b2219800 0x55e8b2fe2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55300/0 (socket says 172.21.15.32:55300) 2024-04-17T10:00:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae7800640 1 -- 172.21.15.32:0/2836447593 learned_addr learned my addr 172.21.15.32:0/2836447593 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae6e00640 1 --2- 172.21.15.32:0/2836447593 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8b2219400 0x55e8b2fe2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae8200640 1 --2- 172.21.15.32:0/2836447593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8b2219000 0x55e8b2fe2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae7800640 1 -- 172.21.15.32:0/2836447593 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8b2219400 msgr2=0x55e8b2fe2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae7800640 1 --2- 172.21.15.32:0/2836447593 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8b2219400 0x55e8b2fe2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae7800640 1 -- 172.21.15.32:0/2836447593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8b2219000 msgr2=0x55e8b2fe2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae7800640 1 --2- 172.21.15.32:0/2836447593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8b2219000 0x55e8b2fe2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae7800640 1 -- 172.21.15.32:0/2836447593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8b223e1e0 con 0x55e8b2219800 2024-04-17T10:00:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae7800640 1 --2- 172.21.15.32:0/2836447593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e8b2219800 0x55e8b2fe2000 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x55e8b3073c50 tx=0x55e8b22fb500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd8b6cc9a2fba002 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae4600640 1 -- 172.21.15.32:0/2836447593 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e8b224e4e0 con 0x55e8b2219800 2024-04-17T10:00:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae4600640 1 -- 172.21.15.32:0/2836447593 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e8b22641c0 con 0x55e8b2219800 2024-04-17T10:00:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae4600640 1 -- 172.21.15.32:0/2836447593 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e8b224eb60 con 0x55e8b2219800 2024-04-17T10:00:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae96668c0 1 -- 172.21.15.32:0/2836447593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8b223f2c0 con 0x55e8b2219800 2024-04-17T10:00:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae96668c0 1 -- 172.21.15.32:0/2836447593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55e8b223fa40 con 0x55e8b2219800 2024-04-17T10:00:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae96668c0 4 waiting for MDS map... 2024-04-17T10:00:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.229+0000 7faae4600640 1 -- 172.21.15.32:0/2836447593 <== mon.0 v2:172.21.15.32:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x55e8b3140600 con 0x55e8b2219800 2024-04-17T10:00:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae4600640 1 -- 172.21.15.32:0/2836447593 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 67) ==== 2886+0+0 (secure 0 0 0) 0x55e8b226f180 con 0x55e8b2219800 2024-04-17T10:00:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae96668c0 4 Got MDS map 67 2024-04-17T10:00:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae96668c0 4 datascan.main: Using metadata pool output 2024-04-17T10:00:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae96668c0 4 datascan.main: connecting to RADOS... 2024-04-17T10:00:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae96668c0 1 Processor -- start 2024-04-17T10:00:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae96668c0 1 -- start start 2024-04-17T10:00:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae96668c0 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8b3137000 0x55e8b2fe3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae96668c0 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8b3137400 0x55e8b2fe3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae96668c0 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e8b3137800 0x55e8b2fe3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae96668c0 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e8b2278180 con 0x55e8b3137800 2024-04-17T10:00:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae96668c0 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e8b231cd80 con 0x55e8b3137400 2024-04-17T10:00:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae96668c0 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e8b231cf00 con 0x55e8b3137000 2024-04-17T10:00:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae6e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e8b3137800 0x55e8b2fe3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae6e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e8b3137800 0x55e8b2fe3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55314/0 (socket says 172.21.15.32:55314) 2024-04-17T10:00:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae6e00640 1 -- 172.21.15.32:0/419829570 learned_addr learned my addr 172.21.15.32:0/419829570 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae8200640 1 --2- 172.21.15.32:0/419829570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8b3137000 0x55e8b2fe3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae7800640 1 --2- 172.21.15.32:0/419829570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8b3137400 0x55e8b2fe3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae6e00640 1 -- 172.21.15.32:0/419829570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8b3137000 msgr2=0x55e8b2fe3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae6e00640 1 --2- 172.21.15.32:0/419829570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8b3137000 0x55e8b2fe3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae6e00640 1 -- 172.21.15.32:0/419829570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8b3137400 msgr2=0x55e8b2fe3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae6e00640 1 --2- 172.21.15.32:0/419829570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8b3137400 0x55e8b2fe3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae6e00640 1 -- 172.21.15.32:0/419829570 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8b223fc20 con 0x55e8b3137800 2024-04-17T10:00:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae7800640 1 --2- 172.21.15.32:0/419829570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8b3137400 0x55e8b2fe3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae6e00640 1 --2- 172.21.15.32:0/419829570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e8b3137800 0x55e8b2fe3b80 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x55e8b3073140 tx=0x55e8b22cf380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fdb2500b94a52bf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae1e00640 1 -- 172.21.15.32:0/419829570 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e8b224e9c0 con 0x55e8b3137800 2024-04-17T10:00:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae1e00640 1 -- 172.21.15.32:0/419829570 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e8b2264e00 con 0x55e8b3137800 2024-04-17T10:00:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.233+0000 7faae1e00640 1 -- 172.21.15.32:0/419829570 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e8b224e4e0 con 0x55e8b3137800 2024-04-17T10:00:44.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae96668c0 1 -- 172.21.15.32:0/419829570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e8b3137800 msgr2=0x55e8b2fe3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae96668c0 1 --2- 172.21.15.32:0/419829570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e8b3137800 0x55e8b2fe3b80 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x55e8b3073140 tx=0x55e8b22cf380 comp rx=0 tx=0).stop 2024-04-17T10:00:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae96668c0 1 -- 172.21.15.32:0/419829570 shutdown_connections 2024-04-17T10:00:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae96668c0 1 --2- 172.21.15.32:0/419829570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e8b3137800 0x55e8b2fe3b80 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae96668c0 1 --2- 172.21.15.32:0/419829570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8b3137400 0x55e8b2fe3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae96668c0 1 --2- 172.21.15.32:0/419829570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8b3137000 0x55e8b2fe3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae96668c0 1 -- 172.21.15.32:0/419829570 >> 172.21.15.32:0/419829570 conn(0x55e8b2218c00 msgr2=0x55e8b2247800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae96668c0 1 -- 172.21.15.32:0/419829570 shutdown_connections 2024-04-17T10:00:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae96668c0 1 -- 172.21.15.32:0/419829570 wait complete. 2024-04-17T10:00:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae96668c0 1 Processor -- start 2024-04-17T10:00:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae96668c0 1 -- start start 2024-04-17T10:00:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae96668c0 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8b3137000 0x55e8b2fe3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae96668c0 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8b3137400 0x55e8b2fe3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae96668c0 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e8b3137800 0x55e8b2fe3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae96668c0 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e8b231d200 con 0x55e8b3137800 2024-04-17T10:00:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae96668c0 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e8b231d080 con 0x55e8b3137000 2024-04-17T10:00:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae96668c0 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e8b231d380 con 0x55e8b3137400 2024-04-17T10:00:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae6e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e8b3137800 0x55e8b2fe3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae6e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e8b3137800 0x55e8b2fe3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55322/0 (socket says 172.21.15.32:55322) 2024-04-17T10:00:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 learned_addr learned my addr 172.21.15.32:0/2041825383 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae8200640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8b3137000 0x55e8b2fe3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae7800640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8b3137400 0x55e8b2fe3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8b3137400 msgr2=0x55e8b2fe3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae6e00640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8b3137400 0x55e8b2fe3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8b3137000 msgr2=0x55e8b2fe3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae6e00640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8b3137000 0x55e8b2fe3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8b223f4a0 con 0x55e8b3137800 2024-04-17T10:00:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae6e00640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e8b3137800 0x55e8b2fe3b80 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x55e8b3133980 tx=0x55e8b22f6540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae0000640 1 -- 172.21.15.32:0/2041825383 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e8b224eb60 con 0x55e8b3137800 2024-04-17T10:00:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae0000640 1 -- 172.21.15.32:0/2041825383 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e8b2264c40 con 0x55e8b3137800 2024-04-17T10:00:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae0000640 1 -- 172.21.15.32:0/2041825383 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e8b224e000 con 0x55e8b3137800 2024-04-17T10:00:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e8b223e3c0 con 0x55e8b3137800 2024-04-17T10:00:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8b223fc20 con 0x55e8b3137800 2024-04-17T10:00:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.237+0000 7faae96668c0 4 datascan.init: resolving metadata pool 12 2024-04-17T10:00:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.241+0000 7faae0000640 1 -- 172.21.15.32:0/2041825383 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x55e8b2518c00 con 0x55e8b3137800 2024-04-17T10:00:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.241+0000 7faae0000640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55e8b2602000 0x55e8b2fe4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.241+0000 7faae0000640 1 -- 172.21.15.32:0/2041825383 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x55e8b2522400 con 0x55e8b3137800 2024-04-17T10:00:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.241+0000 7faae96668c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-17T10:00:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.241+0000 7faae96668c0 4 datascan.main: resolving metadata pool 12 2024-04-17T10:00:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.241+0000 7faae96668c0 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55e8b3137c00 0x55e8b2fe4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.241+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:1 12.0 12:00000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b2602400 con 0x55e8b3137c00 2024-04-17T10:00:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.241+0000 7faae8200640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55e8b2602000 0x55e8b2fe4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.241+0000 7faae7800640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55e8b3137c00 0x55e8b2fe4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.241+0000 7faae8200640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55e8b2602000 0x55e8b2fe4100 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x55e8b22f8f30 tx=0x55e8b3150f60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.241+0000 7faae7800640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55e8b3137c00 0x55e8b2fe4680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.241+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 <== osd.4 v2:172.21.15.144:6809/4076371964 1 ==== osd_op_reply(1 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e000 con 0x55e8b3137c00 2024-04-17T10:00:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.241+0000 7faae7800640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55e8b2602400 0x55e8b2612000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.241+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:2 12.10 12:08000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b3136800 con 0x55e8b2602400 2024-04-17T10:00:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.241+0000 7faae8200640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55e8b2602400 0x55e8b2612000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.245+0000 7faae8200640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55e8b2602400 0x55e8b2612000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.245+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 1 ==== osd_op_reply(2 [pgnls start_epoch 86 out=177b] v86'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.245+0000 7faae8200640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55e8b3136800 0x55e8b2612580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.245+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:3 12.8 12:10000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b260a800 con 0x55e8b3136800 2024-04-17T10:00:44.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.245+0000 7faae7800640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55e8b3136800 0x55e8b2612580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.245+0000 7faae7800640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55e8b3136800 0x55e8b2612580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.245+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 <== osd.3 v2:172.21.15.32:6824/2182899298 1 ==== osd_op_reply(3 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e000 con 0x55e8b3136800 2024-04-17T10:00:44.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.245+0000 7faae7800640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55e8b260a800 0x55e8b2612b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.245+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:4 12.18 12:18000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b260ac00 con 0x55e8b260a800 2024-04-17T10:00:44.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.245+0000 7faae6e00640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55e8b260a800 0x55e8b2612b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.245+0000 7faae6e00640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55e8b260a800 0x55e8b2612b00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.245+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 <== osd.1 v2:172.21.15.32:6808/2454993455 1 ==== osd_op_reply(4 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e780 con 0x55e8b260a800 2024-04-17T10:00:44.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.245+0000 7faae6e00640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55e8b260ac00 0x55e8b2613600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.245+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:5 12.4 12:20000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b2fef400 con 0x55e8b260ac00 2024-04-17T10:00:44.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.245+0000 7faae8200640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55e8b260ac00 0x55e8b2613600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.245+0000 7faae8200640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55e8b260ac00 0x55e8b2613600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.249+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.6 v2:172.21.15.144:6808/3198738696 1 ==== osd_op_reply(5 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b260ac00 2024-04-17T10:00:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.249+0000 7faae8200640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55e8b2fef400 0x55e8b2613b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.249+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:6 12.14 12:28000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b260bc00 con 0x55e8b2fef400 2024-04-17T10:00:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.249+0000 7faae7800640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55e8b2fef400 0x55e8b2613b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.249+0000 7faae7800640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55e8b2fef400 0x55e8b2613b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.249+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 <== osd.5 v2:172.21.15.144:6800/2379578648 1 ==== osd_op_reply(6 [pgnls start_epoch 86 out=149b] v85'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x55e8b232e000 con 0x55e8b2fef400 2024-04-17T10:00:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.249+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:7 12.c 12:30000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b260bc00 con 0x55e8b2602400 2024-04-17T10:00:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.249+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 2 ==== osd_op_reply(7 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.249+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:8 12.1c 12:38000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b260bc00 con 0x55e8b2602400 2024-04-17T10:00:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.249+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 3 ==== osd_op_reply(8 [pgnls start_epoch 86 out=73b] v85'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.249+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:9 12.2 12:40000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b260bc00 con 0x55e8b3137c00 2024-04-17T10:00:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.249+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 <== osd.4 v2:172.21.15.144:6809/4076371964 2 ==== osd_op_reply(9 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e000 con 0x55e8b3137c00 2024-04-17T10:00:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.249+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:10 12.12 12:48000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b260bc00 con 0x55e8b3136800 2024-04-17T10:00:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.249+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 <== osd.3 v2:172.21.15.32:6824/2182899298 2 ==== osd_op_reply(10 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e000 con 0x55e8b3136800 2024-04-17T10:00:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.249+0000 7faae7800640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55e8b260bc00 0x55e8b2613080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.249+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:11 12.a 12:50000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b260b000 con 0x55e8b260bc00 2024-04-17T10:00:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.249+0000 7faae6e00640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55e8b260bc00 0x55e8b2613080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.249+0000 7faae6e00640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55e8b260bc00 0x55e8b2613080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.249+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 <== osd.2 v2:172.21.15.32:6809/1954833462 1 ==== osd_op_reply(11 [pgnls start_epoch 86 out=81b] v85'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55e8b232e780 con 0x55e8b260bc00 2024-04-17T10:00:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.249+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:12 12.1a 12:58000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b260b000 con 0x55e8b3137c00 2024-04-17T10:00:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.253+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 <== osd.4 v2:172.21.15.144:6809/4076371964 3 ==== osd_op_reply(12 [pgnls start_epoch 86 out=49b] v83'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e000 con 0x55e8b3137c00 2024-04-17T10:00:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.253+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:13 12.6 12:60000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b260b000 con 0x55e8b260a800 2024-04-17T10:00:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.253+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 <== osd.1 v2:172.21.15.32:6808/2454993455 2 ==== osd_op_reply(13 [pgnls start_epoch 86 out=77b] v85'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55e8b232e780 con 0x55e8b260a800 2024-04-17T10:00:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.253+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:14 12.16 12:68000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b260b000 con 0x55e8b260ac00 2024-04-17T10:00:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.253+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.6 v2:172.21.15.144:6808/3198738696 2 ==== osd_op_reply(14 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b260ac00 2024-04-17T10:00:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.253+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:15 12.e 12:70000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b260b000 con 0x55e8b2602400 2024-04-17T10:00:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.253+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 4 ==== osd_op_reply(15 [pgnls start_epoch 86 out=73b] v85'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.253+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:16 12.1e 12:78000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b260b000 con 0x55e8b3137c00 2024-04-17T10:00:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.253+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 <== osd.4 v2:172.21.15.144:6809/4076371964 4 ==== osd_op_reply(16 [pgnls start_epoch 86 out=49b] v83'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e000 con 0x55e8b3137c00 2024-04-17T10:00:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.253+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:17 12.1 12:80000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b260b000 con 0x55e8b260ac00 2024-04-17T10:00:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.253+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.6 v2:172.21.15.144:6808/3198738696 3 ==== osd_op_reply(17 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b260ac00 2024-04-17T10:00:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.253+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:18 12.11 12:88000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b260b000 con 0x55e8b2fef400 2024-04-17T10:00:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.253+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 <== osd.5 v2:172.21.15.144:6800/2379578648 2 ==== osd_op_reply(18 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e000 con 0x55e8b2fef400 2024-04-17T10:00:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.253+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:19 12.9 12:90000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b260b000 con 0x55e8b260bc00 2024-04-17T10:00:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.253+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 <== osd.2 v2:172.21.15.32:6809/1954833462 2 ==== osd_op_reply(19 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e780 con 0x55e8b260bc00 2024-04-17T10:00:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.253+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:20 12.19 12:98000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b260b000 con 0x55e8b2602400 2024-04-17T10:00:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.253+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 5 ==== osd_op_reply(20 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.253+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:21 12.5 12:a0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b260b000 con 0x55e8b260ac00 2024-04-17T10:00:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.253+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.6 v2:172.21.15.144:6808/3198738696 4 ==== osd_op_reply(21 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b260ac00 2024-04-17T10:00:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.253+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:22 12.15 12:a8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b260b000 con 0x55e8b260ac00 2024-04-17T10:00:44.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.253+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.6 v2:172.21.15.144:6808/3198738696 5 ==== osd_op_reply(22 [pgnls start_epoch 86 out=97b] v85'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b260ac00 2024-04-17T10:00:44.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.253+0000 7faae8200640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55e8b260b000 0x55e8b2614100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.253+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:23 12.d 12:b0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262e000 con 0x55e8b260b000 2024-04-17T10:00:44.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.257+0000 7faae8200640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55e8b260b000 0x55e8b2614100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.257+0000 7faae8200640 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55e8b260b000 0x55e8b2614100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.257+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.7 v2:172.21.15.144:6824/1746570140 1 ==== osd_op_reply(23 [pgnls start_epoch 86 out=74b] v81'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b260b000 2024-04-17T10:00:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.257+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:24 12.1d 12:b8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262e000 con 0x55e8b2602400 2024-04-17T10:00:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.257+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 6 ==== osd_op_reply(24 [pgnls start_epoch 86 out=73b] v85'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.257+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:25 12.3 12:c0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262e000 con 0x55e8b2602400 2024-04-17T10:00:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.257+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 7 ==== osd_op_reply(25 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.257+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:26 12.13 12:c8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262e000 con 0x55e8b260a800 2024-04-17T10:00:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.257+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 <== osd.1 v2:172.21.15.32:6808/2454993455 3 ==== osd_op_reply(26 [pgnls start_epoch 86 out=98b] v85'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55e8b232e780 con 0x55e8b260a800 2024-04-17T10:00:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.257+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:27 12.b 12:d0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262e000 con 0x55e8b3136800 2024-04-17T10:00:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.257+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 <== osd.3 v2:172.21.15.32:6824/2182899298 3 ==== osd_op_reply(27 [pgnls start_epoch 86 out=76b] v85'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55e8b232e000 con 0x55e8b3136800 2024-04-17T10:00:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.257+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:28 12.1b 12:d8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262e000 con 0x55e8b3137c00 2024-04-17T10:00:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.257+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 <== osd.4 v2:172.21.15.144:6809/4076371964 5 ==== osd_op_reply(28 [pgnls start_epoch 86 out=73b] v85'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55e8b232e000 con 0x55e8b3137c00 2024-04-17T10:00:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.257+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:29 12.7 12:e0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262e000 con 0x55e8b260a800 2024-04-17T10:00:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.257+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 <== osd.1 v2:172.21.15.32:6808/2454993455 4 ==== osd_op_reply(29 [pgnls start_epoch 86 out=73b] v85'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55e8b232e780 con 0x55e8b260a800 2024-04-17T10:00:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.257+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:30 12.17 12:e8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262e000 con 0x55e8b260ac00 2024-04-17T10:00:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.257+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.6 v2:172.21.15.144:6808/3198738696 6 ==== osd_op_reply(30 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b260ac00 2024-04-17T10:00:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.257+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:31 12.f 12:f0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262e000 con 0x55e8b260ac00 2024-04-17T10:00:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.257+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.6 v2:172.21.15.144:6808/3198738696 7 ==== osd_op_reply(31 [pgnls start_epoch 86 out=73b] v81'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b260ac00 2024-04-17T10:00:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.257+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:32 12.1f 12:f8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262e000 con 0x55e8b260ac00 2024-04-17T10:00:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.6 v2:172.21.15.144:6808/3198738696 8 ==== osd_op_reply(32 [pgnls start_epoch 86 out=101b] v85'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b260ac00 2024-04-17T10:00:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-17T10:00:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:33 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b2602800 con 0x55e8b2602400 2024-04-17T10:00:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 8 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-17T10:00:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:34 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b2602c00 con 0x55e8b2602400 2024-04-17T10:00:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 9 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-17T10:00:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:35 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b2603000 con 0x55e8b2602400 2024-04-17T10:00:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 10 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 20 datascan.scan_links: injected ino detected 2024-04-17T10:00:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-17T10:00:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:36 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b2603400 con 0x55e8b2602400 2024-04-17T10:00:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 11 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-17T10:00:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:37 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b2603800 con 0x55e8b2602400 2024-04-17T10:00:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 12 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-17T10:00:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-17T10:00:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-17T10:00:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-17T10:00:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-17T10:00:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-17T10:00:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-17T10:00:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:38 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b2603c00 con 0x55e8b2fef400 2024-04-17T10:00:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 <== osd.5 v2:172.21.15.144:6800/2379578648 3 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55e8b232e000 con 0x55e8b2fef400 2024-04-17T10:00:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-17T10:00:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:39 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b2602800 con 0x55e8b2602400 2024-04-17T10:00:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 13 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-04-17T10:00:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:40 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b2602c00 con 0x55e8b260bc00 2024-04-17T10:00:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 <== osd.2 v2:172.21.15.32:6809/1954833462 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x55e8b232e780 con 0x55e8b260bc00 2024-04-17T10:00:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-17T10:00:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-17T10:00:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-17T10:00:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:41 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262e000 con 0x55e8b2602400 2024-04-17T10:00:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 14 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-17T10:00:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:42 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262e400 con 0x55e8b260ac00 2024-04-17T10:00:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.6 v2:172.21.15.144:6808/3198738696 9 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b260ac00 2024-04-17T10:00:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-17T10:00:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-17T10:00:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-17T10:00:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-17T10:00:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-17T10:00:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.261+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:43 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262e800 con 0x55e8b2602400 2024-04-17T10:00:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 15 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-17T10:00:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae96668c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-17T10:00:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-17T10:00:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:44 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262ec00 con 0x55e8b260a800 2024-04-17T10:00:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 <== osd.1 v2:172.21.15.32:6808/2454993455 5 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x55e8b232e780 con 0x55e8b260a800 2024-04-17T10:00:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-17T10:00:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae96668c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-17T10:00:44.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object 200.00000002 2024-04-17T10:00:44.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae96668c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-17T10:00:44.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-17T10:00:44.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:45 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f000 con 0x55e8b260a800 2024-04-17T10:00:44.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 <== osd.1 v2:172.21.15.32:6808/2454993455 6 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55e8b232e780 con 0x55e8b260a800 2024-04-17T10:00:44.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-17T10:00:44.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae96668c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-17T10:00:44.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-17T10:00:44.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae96668c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-17T10:00:44.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae96668c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-17T10:00:44.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:46 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f400 con 0x55e8b260ac00 2024-04-17T10:00:44.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.6 v2:172.21.15.144:6808/3198738696 10 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) ==== 154+0+1041 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b260ac00 2024-04-17T10:00:44.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:47 12.0 12:00000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b3137c00 2024-04-17T10:00:44.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 <== osd.4 v2:172.21.15.144:6809/4076371964 6 ==== osd_op_reply(47 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e000 con 0x55e8b3137c00 2024-04-17T10:00:44.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:48 12.10 12:08000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b2602400 2024-04-17T10:00:44.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 16 ==== osd_op_reply(48 [pgnls start_epoch 86 out=177b] v86'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:49 12.8 12:10000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b3136800 2024-04-17T10:00:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 <== osd.3 v2:172.21.15.32:6824/2182899298 4 ==== osd_op_reply(49 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e000 con 0x55e8b3136800 2024-04-17T10:00:44.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:50 12.18 12:18000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b260a800 2024-04-17T10:00:44.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 <== osd.1 v2:172.21.15.32:6808/2454993455 7 ==== osd_op_reply(50 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e780 con 0x55e8b260a800 2024-04-17T10:00:44.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:51 12.4 12:20000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b260ac00 2024-04-17T10:00:44.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.6 v2:172.21.15.144:6808/3198738696 11 ==== osd_op_reply(51 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b260ac00 2024-04-17T10:00:44.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:52 12.14 12:28000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b2fef400 2024-04-17T10:00:44.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 <== osd.5 v2:172.21.15.144:6800/2379578648 4 ==== osd_op_reply(52 [pgnls start_epoch 86 out=149b] v85'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x55e8b232e000 con 0x55e8b2fef400 2024-04-17T10:00:44.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.265+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:53 12.c 12:30000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b2602400 2024-04-17T10:00:44.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 17 ==== osd_op_reply(53 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:54 12.1c 12:38000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b2602400 2024-04-17T10:00:44.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 18 ==== osd_op_reply(54 [pgnls start_epoch 86 out=73b] v85'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:55 12.2 12:40000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b3137c00 2024-04-17T10:00:44.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 <== osd.4 v2:172.21.15.144:6809/4076371964 7 ==== osd_op_reply(55 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e000 con 0x55e8b3137c00 2024-04-17T10:00:44.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:56 12.12 12:48000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b3136800 2024-04-17T10:00:44.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 <== osd.3 v2:172.21.15.32:6824/2182899298 5 ==== osd_op_reply(56 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e000 con 0x55e8b3136800 2024-04-17T10:00:44.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:57 12.a 12:50000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b260bc00 2024-04-17T10:00:44.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 <== osd.2 v2:172.21.15.32:6809/1954833462 4 ==== osd_op_reply(57 [pgnls start_epoch 86 out=81b] v85'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55e8b232e780 con 0x55e8b260bc00 2024-04-17T10:00:44.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:58 12.1a 12:58000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b3137c00 2024-04-17T10:00:44.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 <== osd.4 v2:172.21.15.144:6809/4076371964 8 ==== osd_op_reply(58 [pgnls start_epoch 86 out=49b] v83'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e000 con 0x55e8b3137c00 2024-04-17T10:00:44.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:59 12.6 12:60000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b260a800 2024-04-17T10:00:44.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 <== osd.1 v2:172.21.15.32:6808/2454993455 8 ==== osd_op_reply(59 [pgnls start_epoch 86 out=77b] v85'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55e8b232e780 con 0x55e8b260a800 2024-04-17T10:00:44.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:60 12.16 12:68000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b260ac00 2024-04-17T10:00:44.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.6 v2:172.21.15.144:6808/3198738696 12 ==== osd_op_reply(60 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b260ac00 2024-04-17T10:00:44.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:61 12.e 12:70000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b2602400 2024-04-17T10:00:44.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 19 ==== osd_op_reply(61 [pgnls start_epoch 86 out=73b] v85'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:62 12.1e 12:78000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b3137c00 2024-04-17T10:00:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 <== osd.4 v2:172.21.15.144:6809/4076371964 9 ==== osd_op_reply(62 [pgnls start_epoch 86 out=49b] v83'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e000 con 0x55e8b3137c00 2024-04-17T10:00:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:63 12.1 12:80000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b260ac00 2024-04-17T10:00:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.6 v2:172.21.15.144:6808/3198738696 13 ==== osd_op_reply(63 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b260ac00 2024-04-17T10:00:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:64 12.11 12:88000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b2fef400 2024-04-17T10:00:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 <== osd.5 v2:172.21.15.144:6800/2379578648 5 ==== osd_op_reply(64 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e000 con 0x55e8b2fef400 2024-04-17T10:00:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.269+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:65 12.9 12:90000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b260bc00 2024-04-17T10:00:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 <== osd.2 v2:172.21.15.32:6809/1954833462 5 ==== osd_op_reply(65 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e780 con 0x55e8b260bc00 2024-04-17T10:00:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:66 12.19 12:98000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b2602400 2024-04-17T10:00:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 20 ==== osd_op_reply(66 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:67 12.5 12:a0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b260ac00 2024-04-17T10:00:44.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.6 v2:172.21.15.144:6808/3198738696 14 ==== osd_op_reply(67 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b260ac00 2024-04-17T10:00:44.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:68 12.15 12:a8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b260ac00 2024-04-17T10:00:44.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.6 v2:172.21.15.144:6808/3198738696 15 ==== osd_op_reply(68 [pgnls start_epoch 86 out=97b] v85'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b260ac00 2024-04-17T10:00:44.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:69 12.d 12:b0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b260b000 2024-04-17T10:00:44.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.7 v2:172.21.15.144:6824/1746570140 2 ==== osd_op_reply(69 [pgnls start_epoch 86 out=74b] v81'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b260b000 2024-04-17T10:00:44.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:70 12.1d 12:b8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b2602400 2024-04-17T10:00:44.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 21 ==== osd_op_reply(70 [pgnls start_epoch 86 out=73b] v85'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:71 12.3 12:c0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b2602400 2024-04-17T10:00:44.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 22 ==== osd_op_reply(71 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:72 12.13 12:c8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b260a800 2024-04-17T10:00:44.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 <== osd.1 v2:172.21.15.32:6808/2454993455 9 ==== osd_op_reply(72 [pgnls start_epoch 86 out=98b] v85'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55e8b232e780 con 0x55e8b260a800 2024-04-17T10:00:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:73 12.b 12:d0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b3136800 2024-04-17T10:00:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 <== osd.3 v2:172.21.15.32:6824/2182899298 6 ==== osd_op_reply(73 [pgnls start_epoch 86 out=76b] v85'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55e8b232e000 con 0x55e8b3136800 2024-04-17T10:00:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:74 12.1b 12:d8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b3137c00 2024-04-17T10:00:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 <== osd.4 v2:172.21.15.144:6809/4076371964 10 ==== osd_op_reply(74 [pgnls start_epoch 86 out=73b] v85'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55e8b232e000 con 0x55e8b3137c00 2024-04-17T10:00:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:75 12.7 12:e0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b260a800 2024-04-17T10:00:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 <== osd.1 v2:172.21.15.32:6808/2454993455 10 ==== osd_op_reply(75 [pgnls start_epoch 86 out=73b] v85'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55e8b232e780 con 0x55e8b260a800 2024-04-17T10:00:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:76 12.17 12:e8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b260ac00 2024-04-17T10:00:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.6 v2:172.21.15.144:6808/3198738696 16 ==== osd_op_reply(76 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b260ac00 2024-04-17T10:00:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.273+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:77 12.f 12:f0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b260ac00 2024-04-17T10:00:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.6 v2:172.21.15.144:6808/3198738696 17 ==== osd_op_reply(77 [pgnls start_epoch 86 out=73b] v81'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b260ac00 2024-04-17T10:00:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:78 12.1f 12:f8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b260ac00 2024-04-17T10:00:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.6 v2:172.21.15.144:6808/3198738696 18 ==== osd_op_reply(78 [pgnls start_epoch 86 out=101b] v85'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b260ac00 2024-04-17T10:00:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-17T10:00:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:79 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262fc00 con 0x55e8b2602400 2024-04-17T10:00:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 23 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-17T10:00:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:80 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262e000 con 0x55e8b2602400 2024-04-17T10:00:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 24 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-17T10:00:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:81 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262e400 con 0x55e8b2602400 2024-04-17T10:00:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 25 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 20 datascan.scan_links: injected ino detected 2024-04-17T10:00:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-04-17T10:00:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-04-17T10:00:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-17T10:00:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:82 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262e800 con 0x55e8b2602400 2024-04-17T10:00:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 26 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-17T10:00:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:83 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b2603000 con 0x55e8b2602400 2024-04-17T10:00:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 27 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-17T10:00:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-17T10:00:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-17T10:00:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-17T10:00:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-17T10:00:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-17T10:00:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-17T10:00:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:84 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b2603400 con 0x55e8b2fef400 2024-04-17T10:00:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae7800640 1 -- 172.21.15.32:0/2041825383 <== osd.5 v2:172.21.15.144:6800/2379578648 6 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55e8b232e000 con 0x55e8b2fef400 2024-04-17T10:00:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-17T10:00:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:85 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b2603800 con 0x55e8b2602400 2024-04-17T10:00:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 28 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-04-17T10:00:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:86 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b2602800 con 0x55e8b260bc00 2024-04-17T10:00:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 <== osd.2 v2:172.21.15.32:6809/1954833462 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x55e8b232e780 con 0x55e8b260bc00 2024-04-17T10:00:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-17T10:00:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-17T10:00:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-17T10:00:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:87 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b2603000 con 0x55e8b2602400 2024-04-17T10:00:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 29 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-17T10:00:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:88 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b2603800 con 0x55e8b260ac00 2024-04-17T10:00:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.6 v2:172.21.15.144:6808/3198738696 19 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b260ac00 2024-04-17T10:00:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-17T10:00:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-17T10:00:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-17T10:00:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-17T10:00:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-17T10:00:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f400 con 0x55e8b2602400 2024-04-17T10:00:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 30 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-17T10:00:44.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-17T10:00:44.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-17T10:00:44.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.277+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:90 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b260a800 2024-04-17T10:00:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 <== osd.1 v2:172.21.15.32:6808/2454993455 11 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x55e8b232e780 con 0x55e8b260a800 2024-04-17T10:00:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-17T10:00:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-17T10:00:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object 200.00000002 2024-04-17T10:00:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-17T10:00:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-17T10:00:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:91 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262fc00 con 0x55e8b260a800 2024-04-17T10:00:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 <== osd.1 v2:172.21.15.32:6808/2454993455 12 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55e8b232e780 con 0x55e8b260a800 2024-04-17T10:00:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-17T10:00:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-17T10:00:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-17T10:00:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-17T10:00:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-17T10:00:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:92 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262e000 con 0x55e8b260ac00 2024-04-17T10:00:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.6 v2:172.21.15.144:6808/3198738696 20 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) ==== 154+0+1041 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b260ac00 2024-04-17T10:00:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-04-17T10:00:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-17T10:00:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-04-17T10:00:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-04-17T10:00:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:93 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262e400 con 0x55e8b2602400 2024-04-17T10:00:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 31 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-17T10:00:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:94 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262e800 con 0x55e8b2602400 2024-04-17T10:00:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 32 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v86'19 uv19 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-17T10:00:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 10 datascan.scan_links: processing 1 injected_inos 2024-04-17T10:00:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-04-17T10:00:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-04-17T10:00:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:95 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262ec00 con 0x55e8b2602400 2024-04-17T10:00:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 33 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-17T10:00:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 20 datascan.scan_links: first is now 2 2024-04-17T10:00:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.281+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:96 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f000 con 0x55e8b2602400 2024-04-17T10:00:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.0 v2:172.21.15.32:6800/2392056658 34 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v86'20 uv20 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55e8b232e500 con 0x55e8b2602400 2024-04-17T10:00:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-17T10:00:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 10 datascan.scan_links: updating inotable 2024-04-17T10:00:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:97 12.13 12:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b262f800 con 0x55e8b260a800 2024-04-17T10:00:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 <== osd.1 v2:172.21.15.32:6808/2454993455 13 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x55e8b232e780 con 0x55e8b260a800 2024-04-17T10:00:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-04-17T10:00:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-04-17T10:00:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:98 12.13 12:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x55e8b2602c00 con 0x55e8b260a800 2024-04-17T10:00:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 <== osd.1 v2:172.21.15.32:6808/2454993455 14 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v86'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x55e8b232e780 con 0x55e8b260a800 2024-04-17T10:00:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 10 datascan.scan_links: updating snaptable 2024-04-17T10:00:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:99 12.d 12:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55e8b2602800 con 0x55e8b260b000 2024-04-17T10:00:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae8200640 1 -- 172.21.15.32:0/2041825383 <== osd.7 v2:172.21.15.144:6824/1746570140 3 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x55e8b232e280 con 0x55e8b260b000 2024-04-17T10:00:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55e8b2602400 msgr2=0x55e8b2612000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55e8b2602400 0x55e8b2612000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55e8b260a800 msgr2=0x55e8b2612b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55e8b260a800 0x55e8b2612b00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55e8b260bc00 msgr2=0x55e8b2613080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55e8b260bc00 0x55e8b2613080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55e8b3136800 msgr2=0x55e8b2612580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55e8b3136800 0x55e8b2612580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55e8b3137c00 msgr2=0x55e8b2fe4680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55e8b3137c00 0x55e8b2fe4680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55e8b2fef400 msgr2=0x55e8b2613b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55e8b2fef400 0x55e8b2613b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55e8b260ac00 msgr2=0x55e8b2613600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55e8b260ac00 0x55e8b2613600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55e8b260b000 msgr2=0x55e8b2614100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55e8b260b000 0x55e8b2614100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 reap_dead start 2024-04-17T10:00:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae6e00640 1 -- 172.21.15.32:0/2041825383 reap_dead start 2024-04-17T10:00:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55e8b2602000 msgr2=0x55e8b2fe4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55e8b2602000 0x55e8b2fe4100 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x55e8b22f8f30 tx=0x55e8b3150f60 comp rx=0 tx=0).stop 2024-04-17T10:00:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e8b3137800 msgr2=0x55e8b2fe3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e8b3137800 0x55e8b2fe3b80 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x55e8b3133980 tx=0x55e8b22f6540 comp rx=0 tx=0).stop 2024-04-17T10:00:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 shutdown_connections 2024-04-17T10:00:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55e8b2602000 0x55e8b2fe4100 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 --2- 172.21.15.32:0/2041825383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e8b3137800 0x55e8b2fe3b80 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.285+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 >> 172.21.15.32:0/2041825383 conn(0x55e8b2218c00 msgr2=0x55e8b2247800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.289+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 shutdown_connections 2024-04-17T10:00:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.289+0000 7faae96668c0 1 -- 172.21.15.32:0/2041825383 wait complete. 2024-04-17T10:00:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.289+0000 7faae96668c0 1 -- 172.21.15.32:0/2836447593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e8b2219800 msgr2=0x55e8b2fe2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.289+0000 7faae96668c0 1 --2- 172.21.15.32:0/2836447593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e8b2219800 0x55e8b2fe2000 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x55e8b3073c50 tx=0x55e8b22fb500 comp rx=0 tx=0).stop 2024-04-17T10:00:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.289+0000 7faae96668c0 1 -- 172.21.15.32:0/2836447593 shutdown_connections 2024-04-17T10:00:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.289+0000 7faae96668c0 1 --2- 172.21.15.32:0/2836447593 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8b2219400 0x55e8b2fe2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.289+0000 7faae96668c0 1 --2- 172.21.15.32:0/2836447593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8b2219000 0x55e8b2fe2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.289+0000 7faae96668c0 1 --2- 172.21.15.32:0/2836447593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e8b2219800 0x55e8b2fe2000 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.289+0000 7faae96668c0 1 -- 172.21.15.32:0/2836447593 >> 172.21.15.32:0/2836447593 conn(0x55e8b2219c00 msgr2=0x55e8b2247000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.289+0000 7faae96668c0 1 -- 172.21.15.32:0/2836447593 shutdown_connections 2024-04-17T10:00:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.289+0000 7faae96668c0 1 -- 172.21.15.32:0/2836447593 wait complete. 2024-04-17T10:00:44.297 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.159032, result: 2024-04-17T10:00:44.298 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-17T10:00:44.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.485+0000 7fec09400640 1 Processor -- start 2024-04-17T10:00:44.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.485+0000 7fec09400640 1 -- start start 2024-04-17T10:00:44.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.489+0000 7fec09400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fec0415ac30 0x7fec0406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.489+0000 7fec09400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fec040b3d80 0x7fec04071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.489+0000 7fec09400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec0406fcd0 con 0x7fec0415b010 2024-04-17T10:00:44.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.489+0000 7fec09400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fec0406f300 con 0x7fec0415ac30 2024-04-17T10:00:44.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.489+0000 7fec09400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fec0406e6f0 con 0x7fec040b3d80 2024-04-17T10:00:44.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.489+0000 7fec02800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fec0415ac30 0x7fec0406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.489+0000 7febf9e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fec0415b010 0x7fec0406dea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44826/0 (socket says 172.21.15.32:44826) 2024-04-17T10:00:44.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.489+0000 7febf9e00640 1 -- 172.21.15.32:0/1793408846 learned_addr learned my addr 172.21.15.32:0/1793408846 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:00:44.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.489+0000 7fec03200640 1 --2- 172.21.15.32:0/1793408846 >> v2:172.21.15.144:3300/0 conn(0x7fec040b3d80 0x7fec04071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.489+0000 7fec02800640 1 -- 172.21.15.32:0/1793408846 >> v2:172.21.15.144:3300/0 conn(0x7fec040b3d80 msgr2=0x7fec04071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:44.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.489+0000 7fec02800640 1 --2- 172.21.15.32:0/1793408846 >> v2:172.21.15.144:3300/0 conn(0x7fec040b3d80 0x7fec04071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.489+0000 7fec02800640 1 -- 172.21.15.32:0/1793408846 >> v1:172.21.15.32:6789/0 conn(0x7fec0415b010 legacy=0x7fec0406dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.489+0000 7fec02800640 1 -- 172.21.15.32:0/1793408846 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0406e9d0 con 0x7fec0415ac30 2024-04-17T10:00:44.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.493+0000 7fec02800640 1 --2- 172.21.15.32:0/1793408846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec0415ac30 0x7fec0406f790 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fec040732e0 tx=0x7febec01e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7fa5f1dad42e28 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.493+0000 7fec01e00640 1 -- 172.21.15.32:0/1793408846 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec0ad070 con 0x7fec0415ac30 2024-04-17T10:00:44.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.493+0000 7fec01e00640 1 -- 172.21.15.32:0/1793408846 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febec0a7870 con 0x7fec0415ac30 2024-04-17T10:00:44.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.493+0000 7fec01e00640 1 -- 172.21.15.32:0/1793408846 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec0a7c40 con 0x7fec0415ac30 2024-04-17T10:00:44.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.493+0000 7fec09400640 1 -- 172.21.15.32:0/1793408846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec0415ac30 msgr2=0x7fec0406f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.493+0000 7fec09400640 1 --2- 172.21.15.32:0/1793408846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec0415ac30 0x7fec0406f790 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fec040732e0 tx=0x7febec01e910 comp rx=0 tx=0).stop 2024-04-17T10:00:44.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.493+0000 7fec09400640 1 -- 172.21.15.32:0/1793408846 shutdown_connections 2024-04-17T10:00:44.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.493+0000 7fec09400640 1 --2- 172.21.15.32:0/1793408846 >> v2:172.21.15.144:3300/0 conn(0x7fec040b3d80 0x7fec04071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.493+0000 7fec09400640 1 --2- 172.21.15.32:0/1793408846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec0415ac30 0x7fec0406f790 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.493+0000 7fec09400640 1 -- 172.21.15.32:0/1793408846 >> 172.21.15.32:0/1793408846 conn(0x7fec040b2120 msgr2=0x7fec040b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:44.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.493+0000 7fec09400640 1 -- 172.21.15.32:0/1793408846 shutdown_connections 2024-04-17T10:00:44.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.493+0000 7fec09400640 1 -- 172.21.15.32:0/1793408846 wait complete. 2024-04-17T10:00:44.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.493+0000 7fec09400640 1 Processor -- start 2024-04-17T10:00:44.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.493+0000 7fec09400640 1 -- start start 2024-04-17T10:00:44.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.493+0000 7fec09400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec040b3d80 0x7fec0414e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.493+0000 7fec09400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec0415ac30 0x7fec0415b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.493+0000 7fec09400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec0415b010 0x7fec0415b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.493+0000 7fec09400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fec040cdd50 con 0x7fec040b3d80 2024-04-17T10:00:44.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.493+0000 7fec09400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec040cdbd0 con 0x7fec0415ac30 2024-04-17T10:00:44.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.493+0000 7fec09400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fec040cded0 con 0x7fec0415b010 2024-04-17T10:00:44.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.493+0000 7fec02800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec040b3d80 0x7fec0414e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.497+0000 7fec02800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec040b3d80 0x7fec0414e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55342/0 (socket says 172.21.15.32:55342) 2024-04-17T10:00:44.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.497+0000 7fec02800640 1 -- 172.21.15.32:0/2905973811 learned_addr learned my addr 172.21.15.32:0/2905973811 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.497+0000 7febf9e00640 1 --2- 172.21.15.32:0/2905973811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec0415ac30 0x7fec0415b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.497+0000 7fec03200640 1 --2- 172.21.15.32:0/2905973811 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec0415b010 0x7fec0415b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.497+0000 7fec02800640 1 -- 172.21.15.32:0/2905973811 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec0415b010 msgr2=0x7fec0415b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.497+0000 7fec02800640 1 --2- 172.21.15.32:0/2905973811 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec0415b010 0x7fec0415b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.497+0000 7fec02800640 1 -- 172.21.15.32:0/2905973811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec0415ac30 msgr2=0x7fec0415b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.497+0000 7fec02800640 1 --2- 172.21.15.32:0/2905973811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec0415ac30 0x7fec0415b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.497+0000 7fec02800640 1 -- 172.21.15.32:0/2905973811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0415bfb0 con 0x7fec040b3d80 2024-04-17T10:00:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.497+0000 7febf9e00640 1 --2- 172.21.15.32:0/2905973811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec0415ac30 0x7fec0415b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:00:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.497+0000 7fec03200640 1 --2- 172.21.15.32:0/2905973811 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec0415b010 0x7fec0415b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:00:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.497+0000 7fec02800640 1 --2- 172.21.15.32:0/2905973811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec040b3d80 0x7fec0414e0b0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7febec01b520 tx=0x7febec01b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.497+0000 7febfb400640 1 -- 172.21.15.32:0/2905973811 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec0ad040 con 0x7fec040b3d80 2024-04-17T10:00:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.497+0000 7febfb400640 1 -- 172.21.15.32:0/2905973811 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febec0b1690 con 0x7fec040b3d80 2024-04-17T10:00:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.497+0000 7febfb400640 1 -- 172.21.15.32:0/2905973811 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec0b1a80 con 0x7fec040b3d80 2024-04-17T10:00:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.497+0000 7fec09400640 1 -- 172.21.15.32:0/2905973811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec0415c290 con 0x7fec040b3d80 2024-04-17T10:00:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.497+0000 7fec09400640 1 -- 172.21.15.32:0/2905973811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec0415c850 con 0x7fec040b3d80 2024-04-17T10:00:44.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.497+0000 7fec09400640 1 -- 172.21.15.32:0/2905973811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec04068a90 con 0x7fec040b3d80 2024-04-17T10:00:44.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.497+0000 7febfb400640 1 -- 172.21.15.32:0/2905973811 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7febec0c3060 con 0x7fec040b3d80 2024-04-17T10:00:44.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.497+0000 7febfb400640 1 --2- 172.21.15.32:0/2905973811 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7febe0068380 0x7febe006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:44.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.497+0000 7febfb400640 1 -- 172.21.15.32:0/2905973811 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7febec120660 con 0x7fec040b3d80 2024-04-17T10:00:44.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.501+0000 7febf9e00640 1 --2- 172.21.15.32:0/2905973811 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7febe0068380 0x7febe006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:44.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.501+0000 7febf9e00640 1 --2- 172.21.15.32:0/2905973811 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7febe0068380 0x7febe006a840 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7febf4007fd0 tx=0x7febf4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:44.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.501+0000 7febfb400640 1 -- 172.21.15.32:0/2905973811 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7febec018500 con 0x7fec040b3d80 2024-04-17T10:00:44.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.705+0000 7fec09400640 1 -- 172.21.15.32:0/2905973811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fec04072a60 con 0x7fec040b3d80 2024-04-17T10:00:44.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.845+0000 7febfb400640 1 -- 172.21.15.32:0/2905973811 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v68) ==== 162+0+0 (secure 0 0 0) 0x7febec11ee20 con 0x7fec040b3d80 2024-04-17T10:00:44.842 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-17T10:00:44.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.845+0000 7fec09400640 1 -- 172.21.15.32:0/2905973811 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7febe0068380 msgr2=0x7febe006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.845+0000 7fec09400640 1 --2- 172.21.15.32:0/2905973811 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7febe0068380 0x7febe006a840 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7febf4007fd0 tx=0x7febf4009040 comp rx=0 tx=0).stop 2024-04-17T10:00:44.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.845+0000 7fec09400640 1 -- 172.21.15.32:0/2905973811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec040b3d80 msgr2=0x7fec0414e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:44.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.845+0000 7fec09400640 1 --2- 172.21.15.32:0/2905973811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec040b3d80 0x7fec0414e0b0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7febec01b520 tx=0x7febec01b6b0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.845+0000 7fec09400640 1 -- 172.21.15.32:0/2905973811 shutdown_connections 2024-04-17T10:00:44.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.845+0000 7fec09400640 1 --2- 172.21.15.32:0/2905973811 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7febe0068380 0x7febe006a840 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.845+0000 7fec09400640 1 --2- 172.21.15.32:0/2905973811 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec0415b010 0x7fec0415b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.845+0000 7fec09400640 1 --2- 172.21.15.32:0/2905973811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec0415ac30 0x7fec0415b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.845+0000 7fec09400640 1 --2- 172.21.15.32:0/2905973811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec040b3d80 0x7fec0414e0b0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:44.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.845+0000 7fec09400640 1 -- 172.21.15.32:0/2905973811 >> 172.21.15.32:0/2905973811 conn(0x7fec040b2120 msgr2=0x7fec0406dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:44.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.845+0000 7fec09400640 1 -- 172.21.15.32:0/2905973811 shutdown_connections 2024-04-17T10:00:44.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:44.845+0000 7fec09400640 1 -- 172.21.15.32:0/2905973811 wait complete. 2024-04-17T10:00:44.867 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:00:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.009+0000 7f6528600640 1 Processor -- start 2024-04-17T10:00:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.009+0000 7f6528600640 1 -- start start 2024-04-17T10:00:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.013+0000 7f6528600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6520072500 0x7f65200cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.013+0000 7f6528600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f652015d8d0 0x7f65200cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.013+0000 7f6528600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65200cdac0 con 0x7f6520145f30 2024-04-17T10:00:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.013+0000 7f6528600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f652006f400 con 0x7f6520072500 2024-04-17T10:00:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.013+0000 7f6528600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6520146310 con 0x7f652015d8d0 2024-04-17T10:00:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.013+0000 7f6526600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f652015d8d0 0x7f65200cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.013+0000 7f6525200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6520072500 0x7f65200cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.013+0000 7f6525200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6520072500 0x7f65200cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:45520/0 (socket says 172.21.15.32:45520) 2024-04-17T10:00:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.013+0000 7f6525200640 1 -- 172.21.15.32:0/3491685969 learned_addr learned my addr 172.21.15.32:0/3491685969 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.013+0000 7f6526600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f652015d8d0 0x7f65200cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:46894/0 (socket says 172.21.15.32:46894) 2024-04-17T10:00:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.013+0000 7f6525200640 1 -- 172.21.15.32:0/3491685969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f652015d8d0 msgr2=0x7f65200cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.013+0000 7f6525200640 1 --2- 172.21.15.32:0/3491685969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f652015d8d0 0x7f65200cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.013+0000 7f6525200640 1 -- 172.21.15.32:0/3491685969 >> v1:172.21.15.144:6789/0 conn(0x7f6520145f30 legacy=0x7f6520071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.013+0000 7f6525200640 1 -- 172.21.15.32:0/3491685969 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652015cdf0 con 0x7f6520072500 2024-04-17T10:00:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.017+0000 7f6525200640 1 --2- 172.21.15.32:0/3491685969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6520072500 0x7f65200cce70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f650c0098a0 tx=0x7f650c01ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fe5cd33d0715c97 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:45.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.017+0000 7f651fe00640 1 -- 172.21.15.32:0/3491685969 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c0b5070 con 0x7f6520072500 2024-04-17T10:00:45.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.017+0000 7f651fe00640 1 -- 172.21.15.32:0/3491685969 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f650c0a8a30 con 0x7f6520072500 2024-04-17T10:00:45.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.017+0000 7f651fe00640 1 -- 172.21.15.32:0/3491685969 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c0a8e00 con 0x7f6520072500 2024-04-17T10:00:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.017+0000 7f6528600640 1 -- 172.21.15.32:0/3491685969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6520072500 msgr2=0x7f65200cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.017+0000 7f6528600640 1 --2- 172.21.15.32:0/3491685969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6520072500 0x7f65200cce70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f650c0098a0 tx=0x7f650c01ed30 comp rx=0 tx=0).stop 2024-04-17T10:00:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.017+0000 7f6528600640 1 -- 172.21.15.32:0/3491685969 shutdown_connections 2024-04-17T10:00:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.017+0000 7f6528600640 1 --2- 172.21.15.32:0/3491685969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f652015d8d0 0x7f65200cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.017+0000 7f6528600640 1 --2- 172.21.15.32:0/3491685969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6520072500 0x7f65200cce70 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.017+0000 7f6528600640 1 -- 172.21.15.32:0/3491685969 >> 172.21.15.32:0/3491685969 conn(0x7f652006ba30 msgr2=0x7f652006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.017+0000 7f6528600640 1 -- 172.21.15.32:0/3491685969 shutdown_connections 2024-04-17T10:00:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.017+0000 7f6528600640 1 -- 172.21.15.32:0/3491685969 wait complete. 2024-04-17T10:00:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.017+0000 7f6528600640 1 Processor -- start 2024-04-17T10:00:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.017+0000 7f6528600640 1 -- start start 2024-04-17T10:00:45.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f6528600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6520072500 0x7f652015c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f6528600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6520145f30 0x7f65200b4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f6528600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f652015d8d0 0x7f65200cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f6528600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f652015cb90 con 0x7f6520145f30 2024-04-17T10:00:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f6528600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f652015d440 con 0x7f6520072500 2024-04-17T10:00:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f6528600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f65201601d0 con 0x7f652015d8d0 2024-04-17T10:00:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f6525200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6520145f30 0x7f65200b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f6525200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6520145f30 0x7f65200b4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55348/0 (socket says 172.21.15.32:55348) 2024-04-17T10:00:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f6525200640 1 -- 172.21.15.32:0/1620326940 learned_addr learned my addr 172.21.15.32:0/1620326940 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f6525c00640 1 --2- 172.21.15.32:0/1620326940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6520072500 0x7f652015c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f6526600640 1 --2- 172.21.15.32:0/1620326940 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f652015d8d0 0x7f65200cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f6525200640 1 -- 172.21.15.32:0/1620326940 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f652015d8d0 msgr2=0x7f65200cbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f6525200640 1 --2- 172.21.15.32:0/1620326940 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f652015d8d0 0x7f65200cbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f6525200640 1 -- 172.21.15.32:0/1620326940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6520072500 msgr2=0x7f652015c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f6525200640 1 --2- 172.21.15.32:0/1620326940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6520072500 0x7f652015c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f6525200640 1 -- 172.21.15.32:0/1620326940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65200b49c0 con 0x7f6520145f30 2024-04-17T10:00:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f6525200640 1 --2- 172.21.15.32:0/1620326940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6520145f30 0x7f65200b4480 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f650c0b3040 tx=0x7f650c01bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f651d600640 1 -- 172.21.15.32:0/1620326940 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c0b5070 con 0x7f6520145f30 2024-04-17T10:00:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f651d600640 1 -- 172.21.15.32:0/1620326940 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f650c0b0a70 con 0x7f6520145f30 2024-04-17T10:00:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f6528600640 1 -- 172.21.15.32:0/1620326940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65200b4c90 con 0x7f6520145f30 2024-04-17T10:00:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f651d600640 1 -- 172.21.15.32:0/1620326940 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c0b0e60 con 0x7f6520145f30 2024-04-17T10:00:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f6528600640 1 -- 172.21.15.32:0/1620326940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65200c9da0 con 0x7f6520145f30 2024-04-17T10:00:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f6528600640 1 -- 172.21.15.32:0/1620326940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65200b51b0 con 0x7f6520145f30 2024-04-17T10:00:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.021+0000 7f651d600640 1 -- 172.21.15.32:0/1620326940 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f650c0c2060 con 0x7f6520145f30 2024-04-17T10:00:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.025+0000 7f651d600640 1 --2- 172.21.15.32:0/1620326940 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6500068380 0x7f650006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.025+0000 7f651d600640 1 -- 172.21.15.32:0/1620326940 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f650c123aa0 con 0x7f6520145f30 2024-04-17T10:00:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.025+0000 7f6525c00640 1 --2- 172.21.15.32:0/1620326940 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6500068380 0x7f650006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.025+0000 7f6525c00640 1 --2- 172.21.15.32:0/1620326940 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6500068380 0x7f650006a840 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6514007fd0 tx=0x7f6514009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:45.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.029+0000 7f651d600640 1 -- 172.21.15.32:0/1620326940 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f65200b51b0 con 0x7f6520145f30 2024-04-17T10:00:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.225+0000 7f6528600640 1 -- 172.21.15.32:0/1620326940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6520068a90 con 0x7f6520145f30 2024-04-17T10:00:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.229+0000 7f651d600640 1 -- 172.21.15.32:0/1620326940 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5198 (secure 0 0 0) 0x7f650c018890 con 0x7f6520145f30 2024-04-17T10:00:45.226 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:00:45.226 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":69,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6419,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3534108103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3534108103},{"type":"v1","addr":"172.21.15.144:6835","nonce":3534108103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7003,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/935945019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":935945019},{"type":"v1","addr":"172.21.15.144:6834","nonce":935945019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3044362393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3044362393},{"type":"v1","addr":"172.21.15.32:6836","nonce":3044362393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:59:04.834110+0000","modified":"2024-04-17T10:00:44.849678+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8196":{"gid":8196,"name":"c","rank":0,"incarnation":69,"state":"up:replay","state_seq":1,"addr":"172.21.15.32:6837/2058904361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2058904361},{"type":"v1","addr":"172.21.15.32:6837","nonce":2058904361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-17T10:00:45.226 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 69 2024-04-17T10:00:45.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.229+0000 7f6528600640 1 -- 172.21.15.32:0/1620326940 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6500068380 msgr2=0x7f650006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:45.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.229+0000 7f6528600640 1 --2- 172.21.15.32:0/1620326940 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6500068380 0x7f650006a840 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6514007fd0 tx=0x7f6514009040 comp rx=0 tx=0).stop 2024-04-17T10:00:45.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.229+0000 7f6528600640 1 -- 172.21.15.32:0/1620326940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6520145f30 msgr2=0x7f65200b4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:45.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.229+0000 7f6528600640 1 --2- 172.21.15.32:0/1620326940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6520145f30 0x7f65200b4480 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f650c0b3040 tx=0x7f650c01bad0 comp rx=0 tx=0).stop 2024-04-17T10:00:45.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.229+0000 7f6528600640 1 -- 172.21.15.32:0/1620326940 shutdown_connections 2024-04-17T10:00:45.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.229+0000 7f6528600640 1 --2- 172.21.15.32:0/1620326940 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6500068380 0x7f650006a840 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:45.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.229+0000 7f6528600640 1 --2- 172.21.15.32:0/1620326940 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f652015d8d0 0x7f65200cbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:45.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.229+0000 7f6528600640 1 --2- 172.21.15.32:0/1620326940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6520145f30 0x7f65200b4480 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:45.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.229+0000 7f6528600640 1 --2- 172.21.15.32:0/1620326940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6520072500 0x7f652015c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:45.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.229+0000 7f6528600640 1 -- 172.21.15.32:0/1620326940 >> 172.21.15.32:0/1620326940 conn(0x7f652006ba30 msgr2=0x7f65200b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:45.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.229+0000 7f6528600640 1 -- 172.21.15.32:0/1620326940 shutdown_connections 2024-04-17T10:00:45.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:45.229+0000 7f6528600640 1 -- 172.21.15.32:0/1620326940 wait complete. 2024-04-17T10:00:45.650 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T09:59:04.834110+0000', 'modified': '2024-04-17T10:00:44.849678+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 86, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8196}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8196': {'gid': 8196, 'name': 'c', 'rank': 0, 'incarnation': 69, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.32:6837/2058904361', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6835', 'nonce': 2058904361}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 2058904361}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-17T10:00:45.650 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8196:up:replay 2024-04-17T10:00:46.652 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:00:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.809+0000 7f6400a00640 1 Processor -- start 2024-04-17T10:00:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.809+0000 7f6400a00640 1 -- start start 2024-04-17T10:00:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.809+0000 7f6400a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63fc144e90 0x7f63fc0b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.809+0000 7f6400a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f63fc0b4de0 0x7f63fc15baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.809+0000 7f6400a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63fc072bb0 con 0x7f63fc15bfe0 2024-04-17T10:00:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.809+0000 7f6400a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f63fc15c430 con 0x7f63fc0b4de0 2024-04-17T10:00:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.809+0000 7f6400a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f63fc145270 con 0x7f63fc144e90 2024-04-17T10:00:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.809+0000 7f63f9400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f63fc0b4de0 0x7f63fc15baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.809+0000 7f63f9400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f63fc0b4de0 0x7f63fc15baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55356/0 (socket says 172.21.15.32:55356) 2024-04-17T10:00:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.809+0000 7f63f9400640 1 -- 172.21.15.32:0/45631384 learned_addr learned my addr 172.21.15.32:0/45631384 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:46.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.809+0000 7f63f9e00640 1 --2- 172.21.15.32:0/45631384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63fc144e90 0x7f63fc0b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:46.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.809+0000 7f63f9400640 1 -- 172.21.15.32:0/45631384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63fc144e90 msgr2=0x7f63fc0b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:46.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.809+0000 7f63f9400640 1 --2- 172.21.15.32:0/45631384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63fc144e90 0x7f63fc0b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:46.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.809+0000 7f63f9400640 1 -- 172.21.15.32:0/45631384 >> v1:172.21.15.144:6789/0 conn(0x7f63fc15bfe0 legacy=0x7f63fc06ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:46.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.809+0000 7f63f9400640 1 -- 172.21.15.32:0/45631384 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63fc0b3dc0 con 0x7f63fc0b4de0 2024-04-17T10:00:46.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.813+0000 7f63f9400640 1 --2- 172.21.15.32:0/45631384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63fc0b4de0 0x7f63fc15baa0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f63e80098a0 tx=0x7f63e801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1188db141d261 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:46.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.813+0000 7f63f8a00640 1 -- 172.21.15.32:0/45631384 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e80aa670 con 0x7f63fc0b4de0 2024-04-17T10:00:46.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.813+0000 7f63f8a00640 1 -- 172.21.15.32:0/45631384 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e80af070 con 0x7f63fc0b4de0 2024-04-17T10:00:46.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.813+0000 7f63f8a00640 1 -- 172.21.15.32:0/45631384 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e80aaa30 con 0x7f63fc0b4de0 2024-04-17T10:00:46.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.813+0000 7f6400a00640 1 -- 172.21.15.32:0/45631384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63fc0b4de0 msgr2=0x7f63fc15baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:46.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.813+0000 7f6400a00640 1 --2- 172.21.15.32:0/45631384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63fc0b4de0 0x7f63fc15baa0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f63e80098a0 tx=0x7f63e801c510 comp rx=0 tx=0).stop 2024-04-17T10:00:46.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.813+0000 7f6400a00640 1 -- 172.21.15.32:0/45631384 shutdown_connections 2024-04-17T10:00:46.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.813+0000 7f6400a00640 1 --2- 172.21.15.32:0/45631384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63fc0b4de0 0x7f63fc15baa0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:46.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.813+0000 7f6400a00640 1 --2- 172.21.15.32:0/45631384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63fc144e90 0x7f63fc0b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:46.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.813+0000 7f6400a00640 1 -- 172.21.15.32:0/45631384 >> 172.21.15.32:0/45631384 conn(0x7f63fc06ba30 msgr2=0x7f63fc06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:46.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.813+0000 7f6400a00640 1 -- 172.21.15.32:0/45631384 shutdown_connections 2024-04-17T10:00:46.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.813+0000 7f6400a00640 1 -- 172.21.15.32:0/45631384 wait complete. 2024-04-17T10:00:46.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.813+0000 7f6400a00640 1 Processor -- start 2024-04-17T10:00:46.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.813+0000 7f6400a00640 1 -- start start 2024-04-17T10:00:46.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.817+0000 7f6400a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63fc0b4de0 0x7f63fc134700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.817+0000 7f6400a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63fc144e90 0x7f63fc15e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.817+0000 7f6400a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63fc15bfe0 0x7f63fc15ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.817+0000 7f6400a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc072670 con 0x7f63fc15bfe0 2024-04-17T10:00:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.817+0000 7f6400a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc06f880 con 0x7f63fc0b4de0 2024-04-17T10:00:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.817+0000 7f6400a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f63fc0727f0 con 0x7f63fc144e90 2024-04-17T10:00:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.817+0000 7f63fa800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63fc15bfe0 0x7f63fc15ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.817+0000 7f63fa800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63fc15bfe0 0x7f63fc15ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55364/0 (socket says 172.21.15.32:55364) 2024-04-17T10:00:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.817+0000 7f63fa800640 1 -- 172.21.15.32:0/2358612735 learned_addr learned my addr 172.21.15.32:0/2358612735 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.817+0000 7f63f9e00640 1 --2- 172.21.15.32:0/2358612735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63fc0b4de0 0x7f63fc134700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.817+0000 7f63f9400640 1 --2- 172.21.15.32:0/2358612735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63fc144e90 0x7f63fc15e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.817+0000 7f63fa800640 1 -- 172.21.15.32:0/2358612735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63fc144e90 msgr2=0x7f63fc15e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.817+0000 7f63fa800640 1 --2- 172.21.15.32:0/2358612735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63fc144e90 0x7f63fc15e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.817+0000 7f63fa800640 1 -- 172.21.15.32:0/2358612735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63fc0b4de0 msgr2=0x7f63fc134700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:46.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.817+0000 7f63fa800640 1 --2- 172.21.15.32:0/2358612735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63fc0b4de0 0x7f63fc134700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:46.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.817+0000 7f63fa800640 1 -- 172.21.15.32:0/2358612735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63fc15e600 con 0x7f63fc15bfe0 2024-04-17T10:00:46.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.817+0000 7f63fa800640 1 --2- 172.21.15.32:0/2358612735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63fc15bfe0 0x7f63fc15ad30 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f63f400c770 tx=0x7f63f400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:46.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.817+0000 7f63f2000640 1 -- 172.21.15.32:0/2358612735 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4003b80 con 0x7f63fc15bfe0 2024-04-17T10:00:46.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.817+0000 7f63f2000640 1 -- 172.21.15.32:0/2358612735 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f4003d20 con 0x7f63fc15bfe0 2024-04-17T10:00:46.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.817+0000 7f63f2000640 1 -- 172.21.15.32:0/2358612735 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f400aa90 con 0x7f63fc15bfe0 2024-04-17T10:00:46.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.817+0000 7f6400a00640 1 -- 172.21.15.32:0/2358612735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63fc15cab0 con 0x7f63fc15bfe0 2024-04-17T10:00:46.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.817+0000 7f6400a00640 1 -- 172.21.15.32:0/2358612735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63fc0adb40 con 0x7f63fc15bfe0 2024-04-17T10:00:46.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.817+0000 7f6400a00640 1 -- 172.21.15.32:0/2358612735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63fc068a90 con 0x7f63fc15bfe0 2024-04-17T10:00:46.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.821+0000 7f63f2000640 1 -- 172.21.15.32:0/2358612735 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f63f400ac30 con 0x7f63fc15bfe0 2024-04-17T10:00:46.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.821+0000 7f63f2000640 1 --2- 172.21.15.32:0/2358612735 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f63dc068380 0x7f63dc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:46.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.821+0000 7f63f2000640 1 -- 172.21.15.32:0/2358612735 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f63f4086870 con 0x7f63fc15bfe0 2024-04-17T10:00:46.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.821+0000 7f63f9e00640 1 --2- 172.21.15.32:0/2358612735 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f63dc068380 0x7f63dc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:46.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.821+0000 7f63f9e00640 1 --2- 172.21.15.32:0/2358612735 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f63dc068380 0x7f63dc06a840 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f63ec006aa0 tx=0x7f63ec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:46.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:46.825+0000 7f63f2000640 1 -- 172.21.15.32:0/2358612735 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f63fc15e600 con 0x7f63fc15bfe0 2024-04-17T10:00:47.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:47.025+0000 7f6400a00640 1 -- 172.21.15.32:0/2358612735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63fc141680 con 0x7f63fc15bfe0 2024-04-17T10:00:47.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:47.025+0000 7f63f2000640 1 -- 172.21.15.32:0/2358612735 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5198 (secure 0 0 0) 0x7f63f40566e0 con 0x7f63fc15bfe0 2024-04-17T10:00:47.023 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:00:47.023 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":71,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6419,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3534108103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3534108103},{"type":"v1","addr":"172.21.15.144:6835","nonce":3534108103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7003,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/935945019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":935945019},{"type":"v1","addr":"172.21.15.144:6834","nonce":935945019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3044362393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3044362393},{"type":"v1","addr":"172.21.15.32:6836","nonce":3044362393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:59:04.834110+0000","modified":"2024-04-17T10:00:45.856391+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8196":{"gid":8196,"name":"c","rank":0,"incarnation":69,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.32:6837/2058904361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2058904361},{"type":"v1","addr":"172.21.15.32:6837","nonce":2058904361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-17T10:00:47.023 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 71 2024-04-17T10:00:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:47.025+0000 7f6400a00640 1 -- 172.21.15.32:0/2358612735 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f63dc068380 msgr2=0x7f63dc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:47.025+0000 7f6400a00640 1 --2- 172.21.15.32:0/2358612735 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f63dc068380 0x7f63dc06a840 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f63ec006aa0 tx=0x7f63ec008040 comp rx=0 tx=0).stop 2024-04-17T10:00:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:47.025+0000 7f6400a00640 1 -- 172.21.15.32:0/2358612735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63fc15bfe0 msgr2=0x7f63fc15ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:47.025+0000 7f6400a00640 1 --2- 172.21.15.32:0/2358612735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63fc15bfe0 0x7f63fc15ad30 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f63f400c770 tx=0x7f63f400cc30 comp rx=0 tx=0).stop 2024-04-17T10:00:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:47.025+0000 7f6400a00640 1 -- 172.21.15.32:0/2358612735 shutdown_connections 2024-04-17T10:00:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:47.025+0000 7f6400a00640 1 --2- 172.21.15.32:0/2358612735 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f63dc068380 0x7f63dc06a840 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:47.025+0000 7f6400a00640 1 --2- 172.21.15.32:0/2358612735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63fc144e90 0x7f63fc15e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:47.025+0000 7f6400a00640 1 --2- 172.21.15.32:0/2358612735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63fc15bfe0 0x7f63fc15ad30 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:47.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:47.025+0000 7f6400a00640 1 --2- 172.21.15.32:0/2358612735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63fc0b4de0 0x7f63fc134700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:47.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:47.025+0000 7f6400a00640 1 -- 172.21.15.32:0/2358612735 >> 172.21.15.32:0/2358612735 conn(0x7f63fc06ba30 msgr2=0x7f63fc06be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:47.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:47.025+0000 7f6400a00640 1 -- 172.21.15.32:0/2358612735 shutdown_connections 2024-04-17T10:00:47.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:47.025+0000 7f6400a00640 1 -- 172.21.15.32:0/2358612735 wait complete. 2024-04-17T10:00:47.447 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T09:59:04.834110+0000', 'modified': '2024-04-17T10:00:45.856391+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 86, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8196}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8196': {'gid': 8196, 'name': 'c', 'rank': 0, 'incarnation': 69, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.32:6837/2058904361', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6835', 'nonce': 2058904361}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 2058904361}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-17T10:00:47.447 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8196:up:rejoin 2024-04-17T10:00:48.449 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:00:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.605+0000 7fc577200640 1 Processor -- start 2024-04-17T10:00:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.605+0000 7fc577200640 1 -- start start 2024-04-17T10:00:48.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.605+0000 7fc577200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5780b37c0 0x7fc578162300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:48.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.605+0000 7fc577200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc578162840 0x7fc578162ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.605+0000 7fc577200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5780d4f50 con 0x7fc5780b18f0 2024-04-17T10:00:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.605+0000 7fc577200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc5780b3ba0 con 0x7fc578162840 2024-04-17T10:00:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.605+0000 7fc577200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc5780d3500 con 0x7fc5780b37c0 2024-04-17T10:00:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.605+0000 7fc576800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc5780b18f0 0x7fc5780b1d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44836/0 (socket says 172.21.15.32:44836) 2024-04-17T10:00:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.605+0000 7fc576800640 1 -- 172.21.15.32:0/931056697 learned_addr learned my addr 172.21.15.32:0/931056697 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:00:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.605+0000 7fc575e00640 1 --2- 172.21.15.32:0/931056697 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5780b37c0 0x7fc578162300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:48.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.605+0000 7fc56d400640 1 --2- 172.21.15.32:0/931056697 >> v2:172.21.15.144:3300/0 conn(0x7fc578162840 0x7fc578162ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:48.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.605+0000 7fc575400640 1 -- 172.21.15.32:0/931056697 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 786032355 0 0) 0x7fc5780d4f50 con 0x7fc5780b18f0 2024-04-17T10:00:48.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.605+0000 7fc575400640 1 -- 172.21.15.32:0/931056697 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc55c003530 con 0x7fc5780b18f0 2024-04-17T10:00:48.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.605+0000 7fc575e00640 1 -- 172.21.15.32:0/931056697 >> v2:172.21.15.144:3300/0 conn(0x7fc578162840 msgr2=0x7fc578162ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:48.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.605+0000 7fc575e00640 1 --2- 172.21.15.32:0/931056697 >> v2:172.21.15.144:3300/0 conn(0x7fc578162840 0x7fc578162ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:48.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.605+0000 7fc575e00640 1 -- 172.21.15.32:0/931056697 >> v1:172.21.15.32:6789/0 conn(0x7fc5780b18f0 legacy=0x7fc5780b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:48.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.605+0000 7fc575e00640 1 -- 172.21.15.32:0/931056697 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5780d37e0 con 0x7fc5780b37c0 2024-04-17T10:00:48.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.605+0000 7fc56d400640 1 --2- 172.21.15.32:0/931056697 >> v2:172.21.15.144:3300/0 conn(0x7fc578162840 0x7fc578162ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:00:48.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.609+0000 7fc575e00640 1 --2- 172.21.15.32:0/931056697 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5780b37c0 0x7fc578162300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc568009d90 tx=0x7fc56801c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3114c617d1604095 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:48.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.609+0000 7fc575400640 1 -- 172.21.15.32:0/931056697 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5680b2070 con 0x7fc5780b37c0 2024-04-17T10:00:48.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.609+0000 7fc575400640 1 -- 172.21.15.32:0/931056697 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5680a57a0 con 0x7fc5780b37c0 2024-04-17T10:00:48.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.609+0000 7fc575400640 1 -- 172.21.15.32:0/931056697 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5680a5b70 con 0x7fc5780b37c0 2024-04-17T10:00:48.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.609+0000 7fc577200640 1 -- 172.21.15.32:0/931056697 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5780b37c0 msgr2=0x7fc578162300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:48.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.609+0000 7fc577200640 1 --2- 172.21.15.32:0/931056697 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5780b37c0 0x7fc578162300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc568009d90 tx=0x7fc56801c8e0 comp rx=0 tx=0).stop 2024-04-17T10:00:48.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.609+0000 7fc577200640 1 -- 172.21.15.32:0/931056697 shutdown_connections 2024-04-17T10:00:48.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.609+0000 7fc577200640 1 --2- 172.21.15.32:0/931056697 >> v2:172.21.15.144:3300/0 conn(0x7fc578162840 0x7fc578162ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:48.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.609+0000 7fc577200640 1 --2- 172.21.15.32:0/931056697 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5780b37c0 0x7fc578162300 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:48.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.609+0000 7fc577200640 1 -- 172.21.15.32:0/931056697 >> 172.21.15.32:0/931056697 conn(0x7fc57808bf80 msgr2=0x7fc5780d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:48.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.609+0000 7fc577200640 1 -- 172.21.15.32:0/931056697 shutdown_connections 2024-04-17T10:00:48.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.609+0000 7fc577200640 1 -- 172.21.15.32:0/931056697 wait complete. 2024-04-17T10:00:48.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.609+0000 7fc577200640 1 Processor -- start 2024-04-17T10:00:48.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.609+0000 7fc577200640 1 -- start start 2024-04-17T10:00:48.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.613+0000 7fc577200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5780b18f0 0x7fc57815ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:48.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.613+0000 7fc577200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5780b37c0 0x7fc57815f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:48.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.613+0000 7fc577200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc578162840 0x7fc5780c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:48.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.613+0000 7fc577200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc57816b9d0 con 0x7fc5780b37c0 2024-04-17T10:00:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.613+0000 7fc577200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc57816b850 con 0x7fc5780b18f0 2024-04-17T10:00:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.613+0000 7fc577200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc57816bb50 con 0x7fc578162840 2024-04-17T10:00:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.613+0000 7fc56d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5780b37c0 0x7fc57815f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.613+0000 7fc56d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5780b37c0 0x7fc57815f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55378/0 (socket says 172.21.15.32:55378) 2024-04-17T10:00:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.613+0000 7fc56d400640 1 -- 172.21.15.32:0/15714831 learned_addr learned my addr 172.21.15.32:0/15714831 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.613+0000 7fc575e00640 1 --2- 172.21.15.32:0/15714831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5780b18f0 0x7fc57815ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.613+0000 7fc576800640 1 --2- 172.21.15.32:0/15714831 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc578162840 0x7fc5780c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.613+0000 7fc56d400640 1 -- 172.21.15.32:0/15714831 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc578162840 msgr2=0x7fc5780c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.613+0000 7fc56d400640 1 --2- 172.21.15.32:0/15714831 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc578162840 0x7fc5780c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.613+0000 7fc56d400640 1 -- 172.21.15.32:0/15714831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5780b18f0 msgr2=0x7fc57815ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.613+0000 7fc56d400640 1 --2- 172.21.15.32:0/15714831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5780b18f0 0x7fc57815ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.613+0000 7fc56d400640 1 -- 172.21.15.32:0/15714831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5781603d0 con 0x7fc5780b37c0 2024-04-17T10:00:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.613+0000 7fc56d400640 1 --2- 172.21.15.32:0/15714831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5780b37c0 0x7fc57815f9f0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fc56000ebe0 tx=0x7fc56000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.613+0000 7fc56ea00640 1 -- 172.21.15.32:0/15714831 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560017070 con 0x7fc5780b37c0 2024-04-17T10:00:48.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.613+0000 7fc577200640 1 -- 172.21.15.32:0/15714831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5781606b0 con 0x7fc5780b37c0 2024-04-17T10:00:48.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.613+0000 7fc56ea00640 1 -- 172.21.15.32:0/15714831 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc56000cc30 con 0x7fc5780b37c0 2024-04-17T10:00:48.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.613+0000 7fc56ea00640 1 -- 172.21.15.32:0/15714831 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560013470 con 0x7fc5780b37c0 2024-04-17T10:00:48.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.613+0000 7fc577200640 1 -- 172.21.15.32:0/15714831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc578160c70 con 0x7fc5780b37c0 2024-04-17T10:00:48.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.613+0000 7fc577200640 1 -- 172.21.15.32:0/15714831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc57804fd00 con 0x7fc5780b37c0 2024-04-17T10:00:48.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.617+0000 7fc56ea00640 1 -- 172.21.15.32:0/15714831 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc56001a070 con 0x7fc5780b37c0 2024-04-17T10:00:48.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.617+0000 7fc56ea00640 1 --2- 172.21.15.32:0/15714831 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc554068380 0x7fc55406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:48.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.617+0000 7fc56ea00640 1 -- 172.21.15.32:0/15714831 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7fc560086e40 con 0x7fc5780b37c0 2024-04-17T10:00:48.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.617+0000 7fc575e00640 1 --2- 172.21.15.32:0/15714831 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc554068380 0x7fc55406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:48.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.617+0000 7fc575e00640 1 --2- 172.21.15.32:0/15714831 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc554068380 0x7fc55406a840 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc5680af000 tx=0x7fc5680a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:48.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.621+0000 7fc56ea00640 1 -- 172.21.15.32:0/15714831 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc560056c30 con 0x7fc5780b37c0 2024-04-17T10:00:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.821+0000 7fc577200640 1 -- 172.21.15.32:0/15714831 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5780b2270 con 0x7fc5780b37c0 2024-04-17T10:00:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.821+0000 7fc56ea00640 1 -- 172.21.15.32:0/15714831 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5205 (secure 0 0 0) 0x7fc56005af30 con 0x7fc5780b37c0 2024-04-17T10:00:48.818 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:00:48.818 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":72,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6419,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3534108103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3534108103},{"type":"v1","addr":"172.21.15.144:6835","nonce":3534108103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7003,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/935945019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":935945019},{"type":"v1","addr":"172.21.15.144:6834","nonce":935945019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3044362393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3044362393},{"type":"v1","addr":"172.21.15.32:6836","nonce":3044362393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:59:04.834110+0000","modified":"2024-04-17T10:00:47.860710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8196":{"gid":8196,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6837/2058904361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2058904361},{"type":"v1","addr":"172.21.15.32:6837","nonce":2058904361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8196,"qdb_cluster":[8196]},"id":6}]} 2024-04-17T10:00:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 72 2024-04-17T10:00:48.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.821+0000 7fc577200640 1 -- 172.21.15.32:0/15714831 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc554068380 msgr2=0x7fc55406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:48.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.821+0000 7fc577200640 1 --2- 172.21.15.32:0/15714831 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc554068380 0x7fc55406a840 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc5680af000 tx=0x7fc5680a7070 comp rx=0 tx=0).stop 2024-04-17T10:00:48.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.821+0000 7fc577200640 1 -- 172.21.15.32:0/15714831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5780b37c0 msgr2=0x7fc57815f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:48.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.821+0000 7fc577200640 1 --2- 172.21.15.32:0/15714831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5780b37c0 0x7fc57815f9f0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fc56000ebe0 tx=0x7fc56000c520 comp rx=0 tx=0).stop 2024-04-17T10:00:48.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.821+0000 7fc577200640 1 -- 172.21.15.32:0/15714831 shutdown_connections 2024-04-17T10:00:48.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.821+0000 7fc577200640 1 --2- 172.21.15.32:0/15714831 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc554068380 0x7fc55406a840 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:48.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.821+0000 7fc577200640 1 --2- 172.21.15.32:0/15714831 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc578162840 0x7fc5780c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:48.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.821+0000 7fc577200640 1 --2- 172.21.15.32:0/15714831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5780b37c0 0x7fc57815f9f0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:48.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.821+0000 7fc577200640 1 --2- 172.21.15.32:0/15714831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5780b18f0 0x7fc57815ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:48.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.821+0000 7fc577200640 1 -- 172.21.15.32:0/15714831 >> 172.21.15.32:0/15714831 conn(0x7fc57808bf80 msgr2=0x7fc57815ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:48.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.821+0000 7fc577200640 1 -- 172.21.15.32:0/15714831 shutdown_connections 2024-04-17T10:00:48.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:48.821+0000 7fc577200640 1 -- 172.21.15.32:0/15714831 wait complete. 2024-04-17T10:00:49.242 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T09:59:04.834110+0000', 'modified': '2024-04-17T10:00:47.860710+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 86, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8196}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8196': {'gid': 8196, 'name': 'c', 'rank': 0, 'incarnation': 69, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.32:6837/2058904361', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6835', 'nonce': 2058904361}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 2058904361}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8196, 'qdb_cluster': [8196]} 2024-04-17T10:00:49.243 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T10:00:49.243 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-17T10:00:49.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.389+0000 7f18c6600640 1 Processor -- start 2024-04-17T10:00:49.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.389+0000 7f18c6600640 1 -- start start 2024-04-17T10:00:49.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.389+0000 7f18c6600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f18c015c790 0x7f18c006dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.389+0000 7f18c6600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f18c00cd230 0x7f18c00ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.389+0000 7f18c6600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18c00afa10 con 0x7f18c00b3d30 2024-04-17T10:00:49.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.389+0000 7f18c6600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f18c00b41c0 con 0x7f18c015c790 2024-04-17T10:00:49.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.389+0000 7f18c6600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f18c00b4410 con 0x7f18c00cd230 2024-04-17T10:00:49.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.393+0000 7f18c5200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f18c015c790 0x7f18c006dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.393+0000 7f18c5200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f18c015c790 0x7f18c006dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55384/0 (socket says 172.21.15.32:55384) 2024-04-17T10:00:49.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.393+0000 7f18c5200640 1 -- 172.21.15.32:0/1638621127 learned_addr learned my addr 172.21.15.32:0/1638621127 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:49.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.393+0000 7f18bfe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f18c00b3d30 0x7f18c00ccb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44852/0 (socket says 172.21.15.32:44852) 2024-04-17T10:00:49.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.393+0000 7f18c5c00640 1 --2- 172.21.15.32:0/1638621127 >> v2:172.21.15.144:3300/0 conn(0x7f18c00cd230 0x7f18c00ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.393+0000 7f18c5200640 1 -- 172.21.15.32:0/1638621127 >> v2:172.21.15.144:3300/0 conn(0x7f18c00cd230 msgr2=0x7f18c00ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:49.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.393+0000 7f18c5200640 1 --2- 172.21.15.32:0/1638621127 >> v2:172.21.15.144:3300/0 conn(0x7f18c00cd230 0x7f18c00ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.393+0000 7f18c5200640 1 -- 172.21.15.32:0/1638621127 >> v1:172.21.15.32:6789/0 conn(0x7f18c00b3d30 legacy=0x7f18c00ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:49.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.393+0000 7f18c5200640 1 -- 172.21.15.32:0/1638621127 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c00b46f0 con 0x7f18c015c790 2024-04-17T10:00:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.393+0000 7f18c5200640 1 --2- 172.21.15.32:0/1638621127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c015c790 0x7f18c006dee0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f18c00ca710 tx=0x7f18b401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e255ecc7667920 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.393+0000 7f18bf400640 1 -- 172.21.15.32:0/1638621127 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b40aa650 con 0x7f18c015c790 2024-04-17T10:00:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.393+0000 7f18bf400640 1 -- 172.21.15.32:0/1638621127 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b40af070 con 0x7f18c015c790 2024-04-17T10:00:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.393+0000 7f18bf400640 1 -- 172.21.15.32:0/1638621127 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b40aaa10 con 0x7f18c015c790 2024-04-17T10:00:49.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c6600640 1 -- 172.21.15.32:0/1638621127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c015c790 msgr2=0x7f18c006dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:49.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c6600640 1 --2- 172.21.15.32:0/1638621127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c015c790 0x7f18c006dee0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f18c00ca710 tx=0x7f18b401c4f0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c6600640 1 -- 172.21.15.32:0/1638621127 shutdown_connections 2024-04-17T10:00:49.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c6600640 1 --2- 172.21.15.32:0/1638621127 >> v2:172.21.15.144:3300/0 conn(0x7f18c00cd230 0x7f18c00ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c6600640 1 --2- 172.21.15.32:0/1638621127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c015c790 0x7f18c006dee0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c6600640 1 -- 172.21.15.32:0/1638621127 >> 172.21.15.32:0/1638621127 conn(0x7f18c006bc00 msgr2=0x7f18c006c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:49.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c6600640 1 -- 172.21.15.32:0/1638621127 shutdown_connections 2024-04-17T10:00:49.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c6600640 1 -- 172.21.15.32:0/1638621127 wait complete. 2024-04-17T10:00:49.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c6600640 1 Processor -- start 2024-04-17T10:00:49.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c6600640 1 -- start start 2024-04-17T10:00:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c6600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18c00b3d30 0x7f18c0156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c6600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18c00cd230 0x7f18c00cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c6600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c00cbdb0 0x7f18c015e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c6600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f18c00cf450 con 0x7f18c00cbdb0 2024-04-17T10:00:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c6600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f18c00cf2d0 con 0x7f18c00b3d30 2024-04-17T10:00:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c6600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f18c00cf5d0 con 0x7f18c00cd230 2024-04-17T10:00:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c5c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c00cbdb0 0x7f18c015e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c5c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c00cbdb0 0x7f18c015e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55394/0 (socket says 172.21.15.32:55394) 2024-04-17T10:00:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c5c00640 1 -- 172.21.15.32:0/142529008 learned_addr learned my addr 172.21.15.32:0/142529008 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c5200640 1 --2- 172.21.15.32:0/142529008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18c00b3d30 0x7f18c0156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18bfe00640 1 --2- 172.21.15.32:0/142529008 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18c00cd230 0x7f18c00cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c5c00640 1 -- 172.21.15.32:0/142529008 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18c00cd230 msgr2=0x7f18c00cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:49.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c5c00640 1 --2- 172.21.15.32:0/142529008 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18c00cd230 0x7f18c00cb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c5c00640 1 -- 172.21.15.32:0/142529008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18c00b3d30 msgr2=0x7f18c0156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:49.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c5c00640 1 --2- 172.21.15.32:0/142529008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18c00b3d30 0x7f18c0156a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c5c00640 1 -- 172.21.15.32:0/142529008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c015f0a0 con 0x7f18c00cbdb0 2024-04-17T10:00:49.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.397+0000 7f18c5c00640 1 --2- 172.21.15.32:0/142529008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c00cbdb0 0x7f18c015e190 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f18b8007e60 tx=0x7f18b800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:49.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.401+0000 7f18bcc00640 1 -- 172.21.15.32:0/142529008 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8014620 con 0x7f18c00cbdb0 2024-04-17T10:00:49.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.401+0000 7f18bcc00640 1 -- 172.21.15.32:0/142529008 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b8019070 con 0x7f18c00cbdb0 2024-04-17T10:00:49.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.401+0000 7f18bcc00640 1 -- 172.21.15.32:0/142529008 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8014a00 con 0x7f18c00cbdb0 2024-04-17T10:00:49.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.401+0000 7f18c6600640 1 -- 172.21.15.32:0/142529008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c015f380 con 0x7f18c00cbdb0 2024-04-17T10:00:49.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.401+0000 7f18c6600640 1 -- 172.21.15.32:0/142529008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c015f940 con 0x7f18c00cbdb0 2024-04-17T10:00:49.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.401+0000 7f18bcc00640 1 -- 172.21.15.32:0/142529008 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f18b8014ba0 con 0x7f18c00cbdb0 2024-04-17T10:00:49.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.401+0000 7f18bcc00640 1 --2- 172.21.15.32:0/142529008 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f18a0068380 0x7f18a006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.401+0000 7f18bcc00640 1 -- 172.21.15.32:0/142529008 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f18b8087390 con 0x7f18c00cbdb0 2024-04-17T10:00:49.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.401+0000 7f18c5200640 1 --2- 172.21.15.32:0/142529008 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f18a0068380 0x7f18a006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.401+0000 7f18cb162000 1 Processor -- start 2024-04-17T10:00:49.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.401+0000 7f18cb162000 1 -- start start 2024-04-17T10:00:49.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18c5200640 1 --2- 172.21.15.32:0/142529008 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f18a0068380 0x7f18a006a840 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f18b40ae000 tx=0x7f18b40a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:49.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18cb162000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5dbe05b50 0x55c5dbe05f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18cb162000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5dbe06470 0x55c5dbdbb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18cb162000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbdbc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18cb162000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c5dbc094d0 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18cb162000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c5dbba3bf0 con 0x55c5dbe06470 2024-04-17T10:00:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18cb162000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55c5dbc05490 con 0x55c5dbe05b50 2024-04-17T10:00:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18c5c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbdbc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18c5c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbdbc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55396/0 (socket says 172.21.15.32:55396) 2024-04-17T10:00:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18c5c00640 1 -- 172.21.15.32:0/2706755972 learned_addr learned my addr 172.21.15.32:0/2706755972 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18bfe00640 1 --2- 172.21.15.32:0/2706755972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5dbe06470 0x55c5dbdbb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18c5200640 1 --2- 172.21.15.32:0/2706755972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5dbe05b50 0x55c5dbe05f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18c5c00640 1 -- 172.21.15.32:0/2706755972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5dbe05b50 msgr2=0x55c5dbe05f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18c5c00640 1 --2- 172.21.15.32:0/2706755972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5dbe05b50 0x55c5dbe05f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18c5c00640 1 -- 172.21.15.32:0/2706755972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5dbe06470 msgr2=0x55c5dbdbb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18c5c00640 1 --2- 172.21.15.32:0/2706755972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5dbe06470 0x55c5dbdbb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18c5c00640 1 -- 172.21.15.32:0/2706755972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5dbb8fe80 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18bfe00640 1 --2- 172.21.15.32:0/2706755972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5dbe06470 0x55c5dbdbb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18c5200640 1 --2- 172.21.15.32:0/2706755972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5dbe05b50 0x55c5dbe05f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:49.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18c5c00640 1 --2- 172.21.15.32:0/2706755972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbdbc0d0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f18b8053540 tx=0x7f18b8057220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407a5cd140e1a0b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:49.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18b1600640 1 -- 172.21.15.32:0/2706755972 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b806f020 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18b1600640 1 -- 172.21.15.32:0/2706755972 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b80553e0 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18b1600640 1 -- 172.21.15.32:0/2706755972 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b805b070 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18cb162000 1 -- 172.21.15.32:0/2706755972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 msgr2=0x55c5dbdbc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:49.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18cb162000 1 --2- 172.21.15.32:0/2706755972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbdbc0d0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f18b8053540 tx=0x7f18b8057220 comp rx=0 tx=0).stop 2024-04-17T10:00:49.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18cb162000 1 -- 172.21.15.32:0/2706755972 shutdown_connections 2024-04-17T10:00:49.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18cb162000 1 --2- 172.21.15.32:0/2706755972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbdbc0d0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18cb162000 1 --2- 172.21.15.32:0/2706755972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5dbe06470 0x55c5dbdbb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18cb162000 1 --2- 172.21.15.32:0/2706755972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5dbe05b50 0x55c5dbe05f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18cb162000 1 -- 172.21.15.32:0/2706755972 >> 172.21.15.32:0/2706755972 conn(0x55c5dbdc1470 msgr2=0x55c5dbdba090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:49.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18cb162000 1 -- 172.21.15.32:0/2706755972 shutdown_connections 2024-04-17T10:00:49.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18cb162000 1 -- 172.21.15.32:0/2706755972 wait complete. 2024-04-17T10:00:49.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18cb162000 1 -- start start 2024-04-17T10:00:49.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.405+0000 7f18cb162000 1 Processor -- start 2024-04-17T10:00:49.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18cb162000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbdc07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18cb162000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5dbe05b50 0x55c5dbdf3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18cb162000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5dbe06470 0x55c5dbdf3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18cb162000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c5dbbc6a60 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18cb162000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c5dbbade80 con 0x55c5dbe06470 2024-04-17T10:00:49.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18cb162000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55c5dbbc4cf0 con 0x55c5dbe05b50 2024-04-17T10:00:49.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18cb162000 10 client.0 _ll_register_callbacks cb 0x55c5dbda6a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:00:49.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18c5200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbdc07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18c5200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbdc07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55412/0 (socket says 172.21.15.32:55412) 2024-04-17T10:00:49.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18c5200640 1 -- 172.21.15.32:0/3753620668 learned_addr learned my addr 172.21.15.32:0/3753620668 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:49.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18bfe00640 1 --2- 172.21.15.32:0/3753620668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5dbe05b50 0x55c5dbdf3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18c5c00640 1 --2- 172.21.15.32:0/3753620668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5dbe06470 0x55c5dbdf3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18c5200640 1 -- 172.21.15.32:0/3753620668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5dbe05b50 msgr2=0x55c5dbdf3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:49.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18c5200640 1 --2- 172.21.15.32:0/3753620668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5dbe05b50 0x55c5dbdf3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18c5200640 1 -- 172.21.15.32:0/3753620668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5dbe06470 msgr2=0x55c5dbdf3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:49.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18c5200640 1 --2- 172.21.15.32:0/3753620668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5dbe06470 0x55c5dbdf3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18c5200640 1 -- 172.21.15.32:0/3753620668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5dbba3e40 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18bfe00640 1 --2- 172.21.15.32:0/3753620668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5dbe05b50 0x55c5dbdf3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:49.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18c5c00640 1 --2- 172.21.15.32:0/3753620668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5dbe06470 0x55c5dbdf3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:49.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18c5200640 1 --2- 172.21.15.32:0/3753620668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbdc07e0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009630 tx=0x7f18b401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabc347d094693fd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:49.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18a6000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:00:49.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18a6000640 1 -- 172.21.15.32:0/3753620668 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b40cd060 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18a6000640 1 -- 172.21.15.32:0/3753620668 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f18b40c0820 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18cb162000 1 -- 172.21.15.32:0/3753620668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18b80553e0 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18a6000640 1 -- 172.21.15.32:0/3753620668 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f18b40c0a20 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18cb162000 10 client.8268 fetch_fsmap learned FSMap version 72 2024-04-17T10:00:49.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18cb162000 1 -- 172.21.15.32:0/3753620668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55c5dbbd4e00 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18a6000640 1 -- 172.21.15.32:0/3753620668 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b80553e0 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18a6000640 1 -- 172.21.15.32:0/3753620668 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 72) ==== 3495+0+0 (secure 0 0 0) 0x7f18b40c7030 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18cb162000 10 client.8268 fetch_fsmap finished waiting for FSMap version 72 2024-04-17T10:00:49.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18cb162000 10 client.8268 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T10:00:49.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18cb162000 20 client.8268 populate_metadata read hostname 'smithi032' 2024-04-17T10:00:49.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18cb162000 1 --2- 172.21.15.32:0/3753620668 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55c5dbe150b0 0x55c5dbe174a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18cb162000 4 client.8268 mds_command: new command op to 8196 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:00:49.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18cb162000 1 -- 172.21.15.32:0/3753620668 --> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5dbba2bc0 con 0x55c5dbe150b0 2024-04-17T10:00:49.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18bfe00640 1 --2- 172.21.15.32:0/3753620668 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55c5dbe150b0 0x55c5dbe174a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18bfe00640 1 --2- 172.21.15.32:0/3753620668 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55c5dbe150b0 0x55c5dbe174a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24c56bc3a738a5a server_cookie=5158e7d3825597ad in_seq=0 out_seq=0 2024-04-17T10:00:49.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.409+0000 7f18a6000640 10 client.8268 ms_handle_connect on v2:172.21.15.32:6835/2058904361 2024-04-17T10:00:49.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.413+0000 7f18a6000640 1 -- 172.21.15.32:0/3753620668 <== mds.0 v2:172.21.15.32:6835/2058904361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c5dbba2bc0 con 0x55c5dbe150b0 2024-04-17T10:00:49.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.413+0000 7f18a6000640 10 client.8268 handle_command_reply: tid=0 2024-04-17T10:00:49.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.413+0000 7f18cb162000 1 client.8268 shutdown 2024-04-17T10:00:49.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.413+0000 7f18cb162000 1 -- 172.21.15.32:0/3753620668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 msgr2=0x55c5dbdc07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:49.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.413+0000 7f18cb162000 1 --2- 172.21.15.32:0/3753620668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbdc07e0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009630 tx=0x7f18b401ad30 comp rx=0 tx=0).stop 2024-04-17T10:00:49.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.413+0000 7f18cb162000 1 -- 172.21.15.32:0/3753620668 shutdown_connections 2024-04-17T10:00:49.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.413+0000 7f18cb162000 1 --2- 172.21.15.32:0/3753620668 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55c5dbe150b0 0x55c5dbe174a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.413+0000 7f18cb162000 1 --2- 172.21.15.32:0/3753620668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5dbe05b50 0x55c5dbdf3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.413+0000 7f18cb162000 1 --2- 172.21.15.32:0/3753620668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5dbe06470 0x55c5dbdf3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.413+0000 7f18a6000640 0 client.8268 ms_handle_reset on v2:172.21.15.32:6835/2058904361 2024-04-17T10:00:49.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.413+0000 7f18cb162000 1 --2- 172.21.15.32:0/3753620668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbdc07e0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.413+0000 7f18cb162000 1 -- 172.21.15.32:0/3753620668 >> 172.21.15.32:0/3753620668 conn(0x55c5dbdc1470 msgr2=0x55c5dbdba7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:49.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.413+0000 7f18cb162000 1 -- 172.21.15.32:0/3753620668 shutdown_connections 2024-04-17T10:00:49.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.413+0000 7f18cb162000 1 -- 172.21.15.32:0/3753620668 wait complete. 2024-04-17T10:00:49.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.413+0000 7f18cb162000 20 client.8268 trim_cache size 0 max 16384 2024-04-17T10:00:49.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.429+0000 7f18cb162000 1 Processor -- start 2024-04-17T10:00:49.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.429+0000 7f18cb162000 1 -- start start 2024-04-17T10:00:49.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5dbdc1470 0x55c5dbe164a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbe169e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5dbe05b50 0x55c5dbe16f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f18c0165a00 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f18c00cb480 con 0x55c5dbe05b50 2024-04-17T10:00:49.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f18c01653b0 con 0x55c5dbdc1470 2024-04-17T10:00:49.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18bfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbe169e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18bfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbe169e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55420/0 (socket says 172.21.15.32:55420) 2024-04-17T10:00:49.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18bfe00640 1 -- 172.21.15.32:0/327256531 learned_addr learned my addr 172.21.15.32:0/327256531 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:49.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18c5200640 1 --2- 172.21.15.32:0/327256531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5dbdc1470 0x55c5dbe164a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18c5c00640 1 --2- 172.21.15.32:0/327256531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5dbe05b50 0x55c5dbe16f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18bfe00640 1 -- 172.21.15.32:0/327256531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5dbdc1470 msgr2=0x55c5dbe164a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:49.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18bfe00640 1 --2- 172.21.15.32:0/327256531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5dbdc1470 0x55c5dbe164a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18bfe00640 1 -- 172.21.15.32:0/327256531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5dbe05b50 msgr2=0x55c5dbe16f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:49.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18bfe00640 1 --2- 172.21.15.32:0/327256531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5dbe05b50 0x55c5dbe16f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18bfe00640 1 -- 172.21.15.32:0/327256531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5dbc8ad30 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18c5c00640 1 --2- 172.21.15.32:0/327256531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5dbe05b50 0x55c5dbe16f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:49.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18bfe00640 1 --2- 172.21.15.32:0/327256531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbe169e0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f18ac000f20 tx=0x7f18ac009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df322af3b07ebbf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:49.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18b0c00640 1 -- 172.21.15.32:0/327256531 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac01b630 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18b0c00640 1 -- 172.21.15.32:0/327256531 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18ac01b7d0 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18b0c00640 1 -- 172.21.15.32:0/327256531 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac016660 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 1 -- 172.21.15.32:0/327256531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 msgr2=0x55c5dbe169e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:49.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 1 --2- 172.21.15.32:0/327256531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbe169e0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f18ac000f20 tx=0x7f18ac009040 comp rx=0 tx=0).stop 2024-04-17T10:00:49.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 1 -- 172.21.15.32:0/327256531 shutdown_connections 2024-04-17T10:00:49.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 1 --2- 172.21.15.32:0/327256531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5dbe05b50 0x55c5dbe16f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 1 --2- 172.21.15.32:0/327256531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbe169e0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 1 --2- 172.21.15.32:0/327256531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5dbdc1470 0x55c5dbe164a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 1 -- 172.21.15.32:0/327256531 >> 172.21.15.32:0/327256531 conn(0x55c5dbe053d0 msgr2=0x55c5dbdf2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 1 -- 172.21.15.32:0/327256531 shutdown_connections 2024-04-17T10:00:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 1 -- 172.21.15.32:0/327256531 wait complete. 2024-04-17T10:00:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 1 -- start start 2024-04-17T10:00:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 1 Processor -- start 2024-04-17T10:00:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbe1a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5dbdc1470 0x55c5dbe31030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5dbe05b50 0x55c5dbe31570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c5dbc0db00 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c5dbc087e0 con 0x55c5dbe05b50 2024-04-17T10:00:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55c5dbc166e0 con 0x55c5dbdc1470 2024-04-17T10:00:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.433+0000 7f18cb162000 10 client.0 _ll_register_callbacks cb 0x7f18b4006ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:00:49.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18c5200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbe1a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18c5200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbe1a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55430/0 (socket says 172.21.15.32:55430) 2024-04-17T10:00:49.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18c5200640 1 -- 172.21.15.32:0/1602529574 learned_addr learned my addr 172.21.15.32:0/1602529574 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:49.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18c5c00640 1 --2- 172.21.15.32:0/1602529574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5dbe05b50 0x55c5dbe31570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18bfe00640 1 --2- 172.21.15.32:0/1602529574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5dbdc1470 0x55c5dbe31030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18c5200640 1 -- 172.21.15.32:0/1602529574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5dbdc1470 msgr2=0x55c5dbe31030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:49.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18c5200640 1 --2- 172.21.15.32:0/1602529574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5dbdc1470 0x55c5dbe31030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18c5200640 1 -- 172.21.15.32:0/1602529574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5dbe05b50 msgr2=0x55c5dbe31570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:49.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18c5200640 1 --2- 172.21.15.32:0/1602529574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5dbe05b50 0x55c5dbe31570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18c5200640 1 -- 172.21.15.32:0/1602529574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5dbb9b7c0 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18c5200640 1 --2- 172.21.15.32:0/1602529574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbe1a070 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f18b4004710 tx=0x7f18b4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e1a1589ecba366 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:49.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18a6000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:00:49.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18a6000640 1 -- 172.21.15.32:0/1602529574 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b40d7640 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18a6000640 1 -- 172.21.15.32:0/1602529574 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f18b40cd910 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18cb162000 1 -- 172.21.15.32:0/1602529574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18ac01b7d0 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18a6000640 1 -- 172.21.15.32:0/1602529574 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f18b40cdb10 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18cb162000 10 client.8274 fetch_fsmap learned FSMap version 72 2024-04-17T10:00:49.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18cb162000 1 -- 172.21.15.32:0/1602529574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55c5dbbae650 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18a6000640 1 -- 172.21.15.32:0/1602529574 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18ac01b7d0 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18a6000640 1 -- 172.21.15.32:0/1602529574 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 72) ==== 3495+0+0 (secure 0 0 0) 0x7f18b40c7040 con 0x55c5dbdbbcf0 2024-04-17T10:00:49.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18cb162000 10 client.8274 fetch_fsmap finished waiting for FSMap version 72 2024-04-17T10:00:49.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18cb162000 10 client.8274 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T10:00:49.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18cb162000 20 client.8274 populate_metadata read hostname 'smithi032' 2024-04-17T10:00:49.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18cb162000 1 --2- 172.21.15.32:0/1602529574 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55c5dbe06470 0x55c5dbe74a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18cb162000 4 client.8274 mds_command: new command op to 8196 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T10:00:49.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18cb162000 1 -- 172.21.15.32:0/1602529574 --> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] -- command(tid 0: {"prefix": "status"}) -- 0x55c5dbc16ef0 con 0x55c5dbe06470 2024-04-17T10:00:49.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18bfe00640 1 --2- 172.21.15.32:0/1602529574 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55c5dbe06470 0x55c5dbe74a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18bfe00640 1 --2- 172.21.15.32:0/1602529574 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55c5dbe06470 0x55c5dbe74a90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=670aa0693929cb4a server_cookie=f735dec786964be1 in_seq=0 out_seq=0 2024-04-17T10:00:49.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18a6000640 10 client.8274 ms_handle_connect on v2:172.21.15.32:6835/2058904361 2024-04-17T10:00:49.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18a6000640 1 -- 172.21.15.32:0/1602529574 <== mds.0 v2:172.21.15.32:6835/2058904361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55c5dbc16ef0 con 0x55c5dbe06470 2024-04-17T10:00:49.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18a6000640 10 client.8274 handle_command_reply: tid=0 2024-04-17T10:00:49.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18cb162000 1 client.8274 shutdown 2024-04-17T10:00:49.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18cb162000 1 -- 172.21.15.32:0/1602529574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 msgr2=0x55c5dbe1a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:49.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.437+0000 7f18cb162000 1 --2- 172.21.15.32:0/1602529574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbe1a070 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f18b4004710 tx=0x7f18b4008040 comp rx=0 tx=0).stop 2024-04-17T10:00:49.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.441+0000 7f18cb162000 1 -- 172.21.15.32:0/1602529574 shutdown_connections 2024-04-17T10:00:49.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.441+0000 7f18cb162000 1 --2- 172.21.15.32:0/1602529574 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55c5dbe06470 0x55c5dbe74a90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.441+0000 7f18cb162000 1 --2- 172.21.15.32:0/1602529574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5dbdc1470 0x55c5dbe31030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.441+0000 7f18cb162000 1 --2- 172.21.15.32:0/1602529574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5dbe05b50 0x55c5dbe31570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.441+0000 7f18cb162000 1 --2- 172.21.15.32:0/1602529574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c5dbdbbcf0 0x55c5dbe1a070 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.441+0000 7f18cb162000 1 -- 172.21.15.32:0/1602529574 >> 172.21.15.32:0/1602529574 conn(0x55c5dbe053d0 msgr2=0x55c5dbdf3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:49.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.441+0000 7f18a6000640 0 client.8274 ms_handle_reset on v2:172.21.15.32:6835/2058904361 2024-04-17T10:00:49.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.441+0000 7f18cb162000 1 -- 172.21.15.32:0/1602529574 shutdown_connections 2024-04-17T10:00:49.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.441+0000 7f18cb162000 1 -- 172.21.15.32:0/1602529574 wait complete. 2024-04-17T10:00:49.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.441+0000 7f18cb162000 20 client.8274 trim_cache size 0 max 16384 2024-04-17T10:00:49.440 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:00:49.440 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "ff178df8-4c06-4367-a1e9-2e3a66d870d0", 2024-04-17T10:00:49.441 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-17T10:00:49.441 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 8196, 2024-04-17T10:00:49.441 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-17T10:00:49.441 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-17T10:00:49.441 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-17T10:00:49.441 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 4.5872251100000003, 2024-04-17T10:00:49.441 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 72, 2024-04-17T10:00:49.441 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 86, 2024-04-17T10:00:49.441 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 86, 2024-04-17T10:00:49.441 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 5.5318381130000001 2024-04-17T10:00:49.441 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:00:49.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.441+0000 7f18c6600640 1 -- 172.21.15.32:0/142529008 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f18a0068380 msgr2=0x7f18a006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:49.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.441+0000 7f18c6600640 1 --2- 172.21.15.32:0/142529008 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f18a0068380 0x7f18a006a840 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f18b40ae000 tx=0x7f18b40a7000 comp rx=0 tx=0).stop 2024-04-17T10:00:49.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.441+0000 7f18c6600640 1 -- 172.21.15.32:0/142529008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c00cbdb0 msgr2=0x7f18c015e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:49.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.441+0000 7f18c6600640 1 --2- 172.21.15.32:0/142529008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c00cbdb0 0x7f18c015e190 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f18b8007e60 tx=0x7f18b800e4b0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.441+0000 7f18c6600640 1 -- 172.21.15.32:0/142529008 shutdown_connections 2024-04-17T10:00:49.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.441+0000 7f18c6600640 1 --2- 172.21.15.32:0/142529008 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f18a0068380 0x7f18a006a840 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.441+0000 7f18c6600640 1 --2- 172.21.15.32:0/142529008 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18c00cd230 0x7f18c00cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.441+0000 7f18c6600640 1 --2- 172.21.15.32:0/142529008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c00cbdb0 0x7f18c015e190 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.441+0000 7f18c6600640 1 --2- 172.21.15.32:0/142529008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18c00b3d30 0x7f18c0156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.441+0000 7f18c6600640 1 -- 172.21.15.32:0/142529008 >> 172.21.15.32:0/142529008 conn(0x7f18c006bc00 msgr2=0x7f18c00aecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:49.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.441+0000 7f18c6600640 1 -- 172.21.15.32:0/142529008 shutdown_connections 2024-04-17T10:00:49.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.441+0000 7f18c6600640 1 -- 172.21.15.32:0/142529008 wait complete. 2024-04-17T10:00:49.459 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T10:00:49.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.601+0000 7f8837e00640 1 Processor -- start 2024-04-17T10:00:49.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.601+0000 7f8837e00640 1 -- start start 2024-04-17T10:00:49.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.601+0000 7f8837e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8830145f30 0x7f8830071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.601+0000 7f8837e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f883015d8d0 0x7f8830072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.601+0000 7f8837e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88300cdac0 con 0x7f88300cce70 2024-04-17T10:00:49.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.601+0000 7f8837e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f883006f400 con 0x7f883015d8d0 2024-04-17T10:00:49.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.601+0000 7f8837e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8830146310 con 0x7f8830145f30 2024-04-17T10:00:49.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.601+0000 7f8835e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f88300cce70 0x7f88300cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44868/0 (socket says 172.21.15.32:44868) 2024-04-17T10:00:49.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.601+0000 7f8835e00640 1 -- 172.21.15.32:0/1993592448 learned_addr learned my addr 172.21.15.32:0/1993592448 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:00:49.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.601+0000 7f8834a00640 1 --2- 172.21.15.32:0/1993592448 >> v2:172.21.15.144:3300/0 conn(0x7f883015d8d0 0x7f8830072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.601+0000 7f8835400640 1 --2- 172.21.15.32:0/1993592448 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8830145f30 0x7f8830071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.605+0000 7f882fe00640 1 -- 172.21.15.32:0/1993592448 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 951777556 0 0) 0x7f88300cdac0 con 0x7f88300cce70 2024-04-17T10:00:49.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.605+0000 7f882fe00640 1 -- 172.21.15.32:0/1993592448 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8818003530 con 0x7f88300cce70 2024-04-17T10:00:49.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.605+0000 7f8834a00640 1 -- 172.21.15.32:0/1993592448 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8830145f30 msgr2=0x7f8830071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:49.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.605+0000 7f8834a00640 1 --2- 172.21.15.32:0/1993592448 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8830145f30 0x7f8830071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.605+0000 7f8834a00640 1 -- 172.21.15.32:0/1993592448 >> v1:172.21.15.32:6789/0 conn(0x7f88300cce70 legacy=0x7f88300cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:49.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.605+0000 7f8834a00640 1 -- 172.21.15.32:0/1993592448 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883015cdf0 con 0x7f883015d8d0 2024-04-17T10:00:49.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.605+0000 7f8834a00640 1 --2- 172.21.15.32:0/1993592448 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f883015d8d0 0x7f8830072330 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f881c0097c0 tx=0x7f881c01ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9876d5a4e1af98a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:49.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.605+0000 7f882fe00640 1 -- 172.21.15.32:0/1993592448 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c0b5070 con 0x7f883015d8d0 2024-04-17T10:00:49.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f882fe00640 1 -- 172.21.15.32:0/1993592448 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f881c0a8990 con 0x7f883015d8d0 2024-04-17T10:00:49.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f882fe00640 1 -- 172.21.15.32:0/1993592448 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c0a8d60 con 0x7f883015d8d0 2024-04-17T10:00:49.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8837e00640 1 -- 172.21.15.32:0/1993592448 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f883015d8d0 msgr2=0x7f8830072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:49.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8837e00640 1 --2- 172.21.15.32:0/1993592448 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f883015d8d0 0x7f8830072330 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f881c0097c0 tx=0x7f881c01ec90 comp rx=0 tx=0).stop 2024-04-17T10:00:49.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8837e00640 1 -- 172.21.15.32:0/1993592448 shutdown_connections 2024-04-17T10:00:49.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8837e00640 1 --2- 172.21.15.32:0/1993592448 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f883015d8d0 0x7f8830072330 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8837e00640 1 --2- 172.21.15.32:0/1993592448 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8830145f30 0x7f8830071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8837e00640 1 -- 172.21.15.32:0/1993592448 >> 172.21.15.32:0/1993592448 conn(0x7f883006ba30 msgr2=0x7f883006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:49.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8837e00640 1 -- 172.21.15.32:0/1993592448 shutdown_connections 2024-04-17T10:00:49.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8837e00640 1 -- 172.21.15.32:0/1993592448 wait complete. 2024-04-17T10:00:49.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8837e00640 1 Processor -- start 2024-04-17T10:00:49.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8837e00640 1 -- start start 2024-04-17T10:00:49.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8837e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88300cce70 0x7f88301f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8837e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8830145f30 0x7f88300b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8837e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f883015d8d0 0x7f88300b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8837e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f883015cb90 con 0x7f8830145f30 2024-04-17T10:00:49.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8837e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f883015d440 con 0x7f883015d8d0 2024-04-17T10:00:49.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8837e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f88301601d0 con 0x7f88300cce70 2024-04-17T10:00:49.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8834a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8830145f30 0x7f88300b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8834a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8830145f30 0x7f88300b3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55436/0 (socket says 172.21.15.32:55436) 2024-04-17T10:00:49.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8834a00640 1 -- 172.21.15.32:0/994270102 learned_addr learned my addr 172.21.15.32:0/994270102 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:49.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8835400640 1 --2- 172.21.15.32:0/994270102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88300cce70 0x7f88301f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8835e00640 1 --2- 172.21.15.32:0/994270102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f883015d8d0 0x7f88300b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8834a00640 1 -- 172.21.15.32:0/994270102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88300cce70 msgr2=0x7f88301f5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:49.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8834a00640 1 --2- 172.21.15.32:0/994270102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88300cce70 0x7f88301f5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8834a00640 1 -- 172.21.15.32:0/994270102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f883015d8d0 msgr2=0x7f88300b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:49.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8834a00640 1 --2- 172.21.15.32:0/994270102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f883015d8d0 0x7f88300b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.609+0000 7f8834a00640 1 -- 172.21.15.32:0/994270102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88300c9940 con 0x7f8830145f30 2024-04-17T10:00:49.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.613+0000 7f8834a00640 1 --2- 172.21.15.32:0/994270102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8830145f30 0x7f88300b3fa0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f881c0b3040 tx=0x7f881c01ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:49.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.613+0000 7f882d600640 1 -- 172.21.15.32:0/994270102 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c0bf070 con 0x7f8830145f30 2024-04-17T10:00:49.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.613+0000 7f8837e00640 1 -- 172.21.15.32:0/994270102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88300c9c20 con 0x7f8830145f30 2024-04-17T10:00:49.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.613+0000 7f882d600640 1 -- 172.21.15.32:0/994270102 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f881c0b0660 con 0x7f8830145f30 2024-04-17T10:00:49.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.613+0000 7f882d600640 1 -- 172.21.15.32:0/994270102 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c0b5040 con 0x7f8830145f30 2024-04-17T10:00:49.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.613+0000 7f8837e00640 1 -- 172.21.15.32:0/994270102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88300ca1e0 con 0x7f8830145f30 2024-04-17T10:00:49.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.613+0000 7f8837e00640 1 -- 172.21.15.32:0/994270102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8830068a90 con 0x7f8830145f30 2024-04-17T10:00:49.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.613+0000 7f882d600640 1 -- 172.21.15.32:0/994270102 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f881c0b0a50 con 0x7f8830145f30 2024-04-17T10:00:49.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.613+0000 7f882d600640 1 --2- 172.21.15.32:0/994270102 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8810068380 0x7f881006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:49.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.613+0000 7f882d600640 1 -- 172.21.15.32:0/994270102 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f881c1232e0 con 0x7f8830145f30 2024-04-17T10:00:49.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.613+0000 7f8835400640 1 --2- 172.21.15.32:0/994270102 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8810068380 0x7f881006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.617+0000 7f8835400640 1 --2- 172.21.15.32:0/994270102 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8810068380 0x7f881006a840 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f88300b1680 tx=0x7f8824009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:49.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.617+0000 7f882d600640 1 -- 172.21.15.32:0/994270102 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f881c0187f0 con 0x7f8830145f30 2024-04-17T10:00:49.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.821+0000 7f8837e00640 1 -- 172.21.15.32:0/994270102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8830072f90 con 0x7f8830145f30 2024-04-17T10:00:49.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.821+0000 7f882d600640 1 -- 172.21.15.32:0/994270102 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v72) ==== 53+0+73 (secure 0 0 0) 0x7f881c0f30d0 con 0x7f8830145f30 2024-04-17T10:00:49.818 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T10:00:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.821+0000 7f8837e00640 1 -- 172.21.15.32:0/994270102 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8810068380 msgr2=0x7f881006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.821+0000 7f8837e00640 1 --2- 172.21.15.32:0/994270102 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8810068380 0x7f881006a840 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f88300b1680 tx=0x7f8824009190 comp rx=0 tx=0).stop 2024-04-17T10:00:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.821+0000 7f8837e00640 1 -- 172.21.15.32:0/994270102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8830145f30 msgr2=0x7f88300b3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.821+0000 7f8837e00640 1 --2- 172.21.15.32:0/994270102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8830145f30 0x7f88300b3fa0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f881c0b3040 tx=0x7f881c01ba30 comp rx=0 tx=0).stop 2024-04-17T10:00:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.821+0000 7f8837e00640 1 -- 172.21.15.32:0/994270102 shutdown_connections 2024-04-17T10:00:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.821+0000 7f8837e00640 1 --2- 172.21.15.32:0/994270102 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8810068380 0x7f881006a840 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.821+0000 7f8837e00640 1 --2- 172.21.15.32:0/994270102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f883015d8d0 0x7f88300b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.821+0000 7f8837e00640 1 --2- 172.21.15.32:0/994270102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8830145f30 0x7f88300b3fa0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.821+0000 7f8837e00640 1 --2- 172.21.15.32:0/994270102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88300cce70 0x7f88301f5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.821+0000 7f8837e00640 1 -- 172.21.15.32:0/994270102 >> 172.21.15.32:0/994270102 conn(0x7f883006ba30 msgr2=0x7f883015e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.821+0000 7f8837e00640 1 -- 172.21.15.32:0/994270102 shutdown_connections 2024-04-17T10:00:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:49.821+0000 7f8837e00640 1 -- 172.21.15.32:0/994270102 wait complete. 2024-04-17T10:00:49.840 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T10:00:49.840 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T10:00:49.840 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-17T10:00:49.841 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T10:00:49.841 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T10:00:49.841 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T10:00:49.841 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T10:00:49.841 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T10:00:49.841 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:00:49.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:00:49.841 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-17T10:00:49.850 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:00:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:00:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:00:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:00:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-17T10:00:49.850 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:00:49.851 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:00:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-17T10:00:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-17T10:00:49.851 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T10:00:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:00:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-17T10:00:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-17T10:00:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42004sec preferred_lft 42004sec 2024-04-17T10:00:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2024-04-17T10:00:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:00:49.851 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:00:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-17T10:00:49.851 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-17T10:00:49.851 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:00:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:00:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-17T10:00:49.852 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T10:00:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:cd:f4:aa:5b brd ff:ff:ff:ff:ff:ff 2024-04-17T10:00:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T10:00:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:00:49.852 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T10:00:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T10:00:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T10:00:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:00:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T10:00:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:00:49.852 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:00:49.852 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:00:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T10:00:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:00:49.853 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:00:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:00:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T10:00:49.853 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:00:49.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:00:49.854 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T10:00:49.854 INFO:tasks.ceph.mds.c.smithi032.stderr:2024-04-17T10:00:49.853+0000 7f6c64200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:00:49.919 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-17T10:00:49.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:00:49.921 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-17T10:00:49.974 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-17T10:00:49.974 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T10:00:49.974 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-17T10:00:49.974 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T10:00:49.974 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-17T10:00:49.974 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T10:00:49.974 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T10:00:49.975 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T10:00:49.975 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-17T10:00:49.975 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-17T10:00:49.975 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-17T10:00:49.975 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T10:00:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T10:00:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:00:50.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:00:50.113 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-17T10:00:50.122 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T10:00:50.122 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T10:00:50.122 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:00:50.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:00:50.122 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-17T10:00:50.188 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T10:00:50.189 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:00:50.189 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T10:00:50.241 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:00:50.241 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:00:50.241 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:00:50.292 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T10:00:50.354 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:00:50.406 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T10:00:50.406 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T10:00:50.407 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T10:00:50.407 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T10:00:50.407 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T10:00:50.407 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T10:00:50.407 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T10:00:50.407 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T10:00:50.407 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T10:00:50.407 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T10:00:50.407 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T10:00:50.407 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T10:00:50.407 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:00:50.407 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T10:00:50.408 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T10:00:50.408 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T10:00:50.408 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T10:00:50.408 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T10:00:50.408 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T10:00:50.408 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T10:00:50.408 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:00:50.408 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T10:00:50.408 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T10:00:50.409 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T10:00:50.409 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:00:50.409 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:00:50.409 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:00:50.409 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:00:50.409 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T10:00:50.409 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:00:50.409 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:00:50.409 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:00:50.409 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:00:50.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:00:50.410 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T10:00:50.461 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T10:00:50.461 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T10:00:50.508 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T10:00:50.531 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:00:50.583 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T10:00:50.583 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T10:00:50.583 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T10:00:50.583 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T10:00:50.583 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T10:00:50.583 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T10:00:50.584 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T10:00:50.584 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T10:00:50.585 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T10:00:50.585 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T10:00:50.585 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T10:00:50.585 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T10:00:50.585 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:00:50.585 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T10:00:50.585 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T10:00:50.585 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T10:00:50.586 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T10:00:50.586 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T10:00:50.586 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T10:00:50.586 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T10:00:50.586 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:00:50.586 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T10:00:50.586 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T10:00:50.586 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T10:00:50.586 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:00:50.586 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:00:50.586 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:00:50.586 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:00:50.586 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T10:00:50.586 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:00:50.587 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:00:50.587 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:00:50.587 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:00:50.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:00:50.588 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T10:00:50.604 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2024-04-17T10:00:50.605+0000 7fc7f48d1580 -1 init, newargv = 0x558a34b07e70 newargc=15 2024-04-17T10:00:50.605 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[46849]: starting ceph client 2024-04-17T10:00:50.612 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[46849]: starting fuse 2024-04-17T10:00:50.614 INFO:teuthology.orchestra.run.smithi032.stdout:44 2024-04-17T10:00:50.614 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-17T10:00:50.614 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2024-04-17T10:00:50.614 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2024-04-17T10:00:50.615 DEBUG:teuthology.orchestra.run.smithi032:> import re 2024-04-17T10:00:50.615 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:00:50.615 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2024-04-17T10:00:50.615 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:00:50.615 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2024-04-17T10:00:50.615 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T10:00:50.615 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2024-04-17T10:00:50.615 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T10:00:50.615 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:00:50.615 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2024-04-17T10:00:50.615 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2024-04-17T10:00:50.615 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2024-04-17T10:00:50.615 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2024-04-17T10:00:50.615 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:00:50.615 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2024-04-17T10:00:50.616 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T10:00:50.616 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T10:00:50.616 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T10:00:50.616 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2024-04-17T10:00:50.616 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2024-04-17T10:00:50.616 DEBUG:teuthology.orchestra.run.smithi032:> return f 2024-04-17T10:00:50.616 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T10:00:50.616 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:00:50.616 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2024-04-17T10:00:50.616 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:00:50.739 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.46849.asok 2024-04-17T10:00:50.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:00:50.751 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46849.asok 2024-04-17T10:00:50.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:00:50.752 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46849.asok status 2024-04-17T10:00:50.926 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:00:50.926 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2024-04-17T10:00:50.926 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-17T10:00:50.926 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T10:00:50.926 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2024-04-17T10:00:50.926 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2024-04-17T10:00:50.926 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T10:00:50.926 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "46849", 2024-04-17T10:00:50.927 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2024-04-17T10:00:50.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:00:50.927 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2024-04-17T10:00:50.927 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2024-04-17T10:00:50.927 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 8289, 2024-04-17T10:00:50.927 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2024-04-17T10:00:50.927 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2024-04-17T10:00:50.927 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2024-04-17T10:00:50.927 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 8289 2024-04-17T10:00:50.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:00:50.928 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T10:00:50.928 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T10:00:50.928 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:00:50.928 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 866962976 2024-04-17T10:00:50.928 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:00:50.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:00:50.928 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T10:00:50.928 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T10:00:50.928 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:00:50.928 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 866962976 2024-04-17T10:00:50.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:00:50.928 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.8289 192.168.0.1:0/866962976", 2024-04-17T10:00:50.928 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/866962976", 2024-04-17T10:00:50.928 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2024-04-17T10:00:50.928 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 72, 2024-04-17T10:00:50.929 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 86, 2024-04-17T10:00:50.929 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2024-04-17T10:00:50.929 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2024-04-17T10:00:50.929 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs" 2024-04-17T10:00:50.929 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:00:50.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:00:50.941 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2024-04-17T10:00:50.941 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T10:00:50.950 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2024-04-17T10:00:50.951 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T10:00:50.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:00:50.951 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:00:51.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:00:51.012 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-17T10:00:51.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.197+0000 7f4fc8400640 1 Processor -- start 2024-04-17T10:00:51.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.197+0000 7f4fc8400640 1 -- start start 2024-04-17T10:00:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.197+0000 7f4fc8400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4fc00d00e0 0x7f4fc00d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.197+0000 7f4fc8400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4fc00d0a80 0x7f4fc00c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.197+0000 7f4fc8400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fc00b3b70 con 0x7f4fc015fd20 2024-04-17T10:00:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.197+0000 7f4fc8400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f4fc009baf0 con 0x7f4fc00d0a80 2024-04-17T10:00:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.197+0000 7f4fc8400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f4fc00d0ff0 con 0x7f4fc00d00e0 2024-04-17T10:00:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.201+0000 7f4fc6400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4fc00d0a80 0x7f4fc00c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.201+0000 7f4fc6400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4fc00d0a80 0x7f4fc00c6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35588/0 (socket says 172.21.15.32:35588) 2024-04-17T10:00:51.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.201+0000 7f4fc6400640 1 -- 172.21.15.32:0/3945146506 learned_addr learned my addr 172.21.15.32:0/3945146506 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:51.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.201+0000 7f4fc5000640 1 --2- 172.21.15.32:0/3945146506 >> v2:172.21.15.144:3300/0 conn(0x7f4fc00d00e0 0x7f4fc00d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.201+0000 7f4fc6400640 1 -- 172.21.15.32:0/3945146506 >> v2:172.21.15.144:3300/0 conn(0x7f4fc00d00e0 msgr2=0x7f4fc00d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:51.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.201+0000 7f4fc6400640 1 --2- 172.21.15.32:0/3945146506 >> v2:172.21.15.144:3300/0 conn(0x7f4fc00d00e0 0x7f4fc00d0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.201+0000 7f4fc6400640 1 -- 172.21.15.32:0/3945146506 >> v1:172.21.15.144:6789/0 conn(0x7f4fc015fd20 legacy=0x7f4fc00b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:51.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.201+0000 7f4fc6400640 1 -- 172.21.15.32:0/3945146506 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc00d12d0 con 0x7f4fc00d0a80 2024-04-17T10:00:51.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.201+0000 7f4fc6400640 1 --2- 172.21.15.32:0/3945146506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fc00d0a80 0x7f4fc00c6060 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f4fc00b5100 tx=0x7f4fb801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e267c9e68e550a80 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:51.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.201+0000 7f4fbfe00640 1 -- 172.21.15.32:0/3945146506 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb80a57f0 con 0x7f4fc00d0a80 2024-04-17T10:00:51.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.201+0000 7f4fbfe00640 1 -- 172.21.15.32:0/3945146506 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb80a5990 con 0x7f4fc00d0a80 2024-04-17T10:00:51.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.201+0000 7f4fbfe00640 1 -- 172.21.15.32:0/3945146506 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb80a5d60 con 0x7f4fc00d0a80 2024-04-17T10:00:51.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.205+0000 7f4fc8400640 1 -- 172.21.15.32:0/3945146506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fc00d0a80 msgr2=0x7f4fc00c6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:51.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.205+0000 7f4fc8400640 1 --2- 172.21.15.32:0/3945146506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fc00d0a80 0x7f4fc00c6060 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f4fc00b5100 tx=0x7f4fb801c930 comp rx=0 tx=0).stop 2024-04-17T10:00:51.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.205+0000 7f4fc8400640 1 -- 172.21.15.32:0/3945146506 shutdown_connections 2024-04-17T10:00:51.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.205+0000 7f4fc8400640 1 --2- 172.21.15.32:0/3945146506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fc00d0a80 0x7f4fc00c6060 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.205+0000 7f4fc8400640 1 --2- 172.21.15.32:0/3945146506 >> v2:172.21.15.144:3300/0 conn(0x7f4fc00d00e0 0x7f4fc00d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.205+0000 7f4fc8400640 1 -- 172.21.15.32:0/3945146506 >> 172.21.15.32:0/3945146506 conn(0x7f4fc00b4000 msgr2=0x7f4fc00b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:51.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.205+0000 7f4fc8400640 1 -- 172.21.15.32:0/3945146506 shutdown_connections 2024-04-17T10:00:51.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.205+0000 7f4fc8400640 1 -- 172.21.15.32:0/3945146506 wait complete. 2024-04-17T10:00:51.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.205+0000 7f4fc8400640 1 Processor -- start 2024-04-17T10:00:51.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.205+0000 7f4fc8400640 1 -- start start 2024-04-17T10:00:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.205+0000 7f4fc8400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fc00d00e0 0x7f4fc0159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.205+0000 7f4fc8400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fc00d0a80 0x7f4fc0165d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.205+0000 7f4fc8400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fc015fd20 0x7f4fc006ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.205+0000 7f4fc8400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc00c6f30 con 0x7f4fc015fd20 2024-04-17T10:00:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.205+0000 7f4fc8400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc00c6db0 con 0x7f4fc00d0a80 2024-04-17T10:00:51.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.205+0000 7f4fc8400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4fc00c70b0 con 0x7f4fc00d00e0 2024-04-17T10:00:51.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.205+0000 7f4fc6400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fc015fd20 0x7f4fc006ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.205+0000 7f4fc6400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fc015fd20 0x7f4fc006ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35602/0 (socket says 172.21.15.32:35602) 2024-04-17T10:00:51.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.205+0000 7f4fc6400640 1 -- 172.21.15.32:0/942014663 learned_addr learned my addr 172.21.15.32:0/942014663 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:51.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.205+0000 7f4fc5000640 1 --2- 172.21.15.32:0/942014663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fc00d0a80 0x7f4fc0165d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.205+0000 7f4fc5a00640 1 --2- 172.21.15.32:0/942014663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fc00d00e0 0x7f4fc0159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.209+0000 7f4fc6400640 1 -- 172.21.15.32:0/942014663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fc00d00e0 msgr2=0x7f4fc0159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:51.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.209+0000 7f4fc6400640 1 --2- 172.21.15.32:0/942014663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fc00d00e0 0x7f4fc0159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.209+0000 7f4fc6400640 1 -- 172.21.15.32:0/942014663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fc00d0a80 msgr2=0x7f4fc0165d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:51.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.209+0000 7f4fc6400640 1 --2- 172.21.15.32:0/942014663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fc00d0a80 0x7f4fc0165d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.209+0000 7f4fc6400640 1 -- 172.21.15.32:0/942014663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc0159d00 con 0x7f4fc015fd20 2024-04-17T10:00:51.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.209+0000 7f4fc5000640 1 --2- 172.21.15.32:0/942014663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fc00d0a80 0x7f4fc0165d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:00:51.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.209+0000 7f4fc6400640 1 --2- 172.21.15.32:0/942014663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fc015fd20 0x7f4fc006ddf0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8019540 tx=0x7f4fb80196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.209+0000 7f4fbd600640 1 -- 172.21.15.32:0/942014663 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb80bc020 con 0x7f4fc015fd20 2024-04-17T10:00:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.209+0000 7f4fbd600640 1 -- 172.21.15.32:0/942014663 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb80b8690 con 0x7f4fc015fd20 2024-04-17T10:00:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.209+0000 7f4fbd600640 1 -- 172.21.15.32:0/942014663 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8009510 con 0x7f4fc015fd20 2024-04-17T10:00:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.209+0000 7f4fc8400640 1 -- 172.21.15.32:0/942014663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc0159fe0 con 0x7f4fc015fd20 2024-04-17T10:00:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.209+0000 7f4fc8400640 1 -- 172.21.15.32:0/942014663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc015a5a0 con 0x7f4fc015fd20 2024-04-17T10:00:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.209+0000 7f4fbd600640 1 -- 172.21.15.32:0/942014663 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4fb80bf020 con 0x7f4fc015fd20 2024-04-17T10:00:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.209+0000 7f4fbd600640 1 --2- 172.21.15.32:0/942014663 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4fa0068380 0x7f4fa006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.209+0000 7f4fbd600640 1 -- 172.21.15.32:0/942014663 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f4fb8121890 con 0x7f4fc015fd20 2024-04-17T10:00:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fc5a00640 1 --2- 172.21.15.32:0/942014663 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4fa0068380 0x7f4fa006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fcb7f7000 1 Processor -- start 2024-04-17T10:00:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fcb7f7000 1 -- start start 2024-04-17T10:00:51.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fc5a00640 1 --2- 172.21.15.32:0/942014663 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4fa0068380 0x7f4fa006a840 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4001810 tx=0x7f4fb4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fcb7f7000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e49f7a7700 0x55e49f7a7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fcb7f7000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f7a8020 0x55e49f75a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fcb7f7000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e49f75aa10 0x55e49f75adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fcb7f7000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e49f5ac5f0 con 0x55e49f7a8020 2024-04-17T10:00:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fcb7f7000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e49f5657a0 con 0x55e49f75aa10 2024-04-17T10:00:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fcb7f7000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e49f5a85b0 con 0x55e49f7a7700 2024-04-17T10:00:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fc5000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f7a8020 0x55e49f75a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fc5000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f7a8020 0x55e49f75a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35604/0 (socket says 172.21.15.32:35604) 2024-04-17T10:00:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fc5000640 1 -- 172.21.15.32:0/2868588960 learned_addr learned my addr 172.21.15.32:0/2868588960 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fc5000640 1 -- 172.21.15.32:0/2868588960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e49f7a7700 msgr2=0x55e49f7a7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:51.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fc5000640 1 --2- 172.21.15.32:0/2868588960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e49f7a7700 0x55e49f7a7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fc5000640 1 -- 172.21.15.32:0/2868588960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e49f75aa10 msgr2=0x55e49f75adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:51.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fc6400640 1 --2- 172.21.15.32:0/2868588960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e49f75aa10 0x55e49f75adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fc5000640 1 --2- 172.21.15.32:0/2868588960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e49f75aa10 0x55e49f75adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fc5000640 1 -- 172.21.15.32:0/2868588960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e49f6536e0 con 0x55e49f7a8020 2024-04-17T10:00:51.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fc6400640 1 --2- 172.21.15.32:0/2868588960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e49f75aa10 0x55e49f75adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:00:51.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fc5000640 1 --2- 172.21.15.32:0/2868588960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f7a8020 0x55e49f75a4d0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f4fac009f40 tx=0x7f4fac011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19f041c17153da9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fb2000640 1 -- 172.21.15.32:0/2868588960 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac004020 con 0x55e49f7a8020 2024-04-17T10:00:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fb2000640 1 -- 172.21.15.32:0/2868588960 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fac0041c0 con 0x55e49f7a8020 2024-04-17T10:00:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fb2000640 1 -- 172.21.15.32:0/2868588960 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac014650 con 0x55e49f7a8020 2024-04-17T10:00:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/2868588960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f7a8020 msgr2=0x55e49f75a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fcb7f7000 1 --2- 172.21.15.32:0/2868588960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f7a8020 0x55e49f75a4d0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f4fac009f40 tx=0x7f4fac011750 comp rx=0 tx=0).stop 2024-04-17T10:00:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/2868588960 shutdown_connections 2024-04-17T10:00:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fcb7f7000 1 --2- 172.21.15.32:0/2868588960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e49f75aa10 0x55e49f75adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fcb7f7000 1 --2- 172.21.15.32:0/2868588960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f7a8020 0x55e49f75a4d0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fcb7f7000 1 --2- 172.21.15.32:0/2868588960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e49f7a7700 0x55e49f7a7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/2868588960 >> 172.21.15.32:0/2868588960 conn(0x55e49f788290 msgr2=0x55e49f78a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/2868588960 shutdown_connections 2024-04-17T10:00:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.213+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/2868588960 wait complete. 2024-04-17T10:00:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fcb7f7000 1 -- start start 2024-04-17T10:00:51.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fcb7f7000 1 Processor -- start 2024-04-17T10:00:51.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fcb7f7000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f75aa10 0x55e49f7625c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fcb7f7000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e49f7a7700 0x55e49f762b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fcb7f7000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e49f7a8020 0x55e49f75f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fcb7f7000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e49f5b7520 con 0x55e49f75aa10 2024-04-17T10:00:51.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fcb7f7000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e49f544ea0 con 0x55e49f7a8020 2024-04-17T10:00:51.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fcb7f7000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e49f56b500 con 0x55e49f7a7700 2024-04-17T10:00:51.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fcb7f7000 10 client.0 _ll_register_callbacks cb 0x55e49f748170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:00:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fc5a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f75aa10 0x55e49f7625c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fc5a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f75aa10 0x55e49f7625c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35616/0 (socket says 172.21.15.32:35616) 2024-04-17T10:00:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fc5a00640 1 -- 172.21.15.32:0/68366117 learned_addr learned my addr 172.21.15.32:0/68366117 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fc6400640 1 --2- 172.21.15.32:0/68366117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e49f7a8020 0x55e49f75f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fc5a00640 1 -- 172.21.15.32:0/68366117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e49f7a7700 msgr2=0x55e49f762b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fc5000640 1 --2- 172.21.15.32:0/68366117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e49f7a7700 0x55e49f762b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fc5a00640 1 --2- 172.21.15.32:0/68366117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e49f7a7700 0x55e49f762b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fc5a00640 1 -- 172.21.15.32:0/68366117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e49f7a8020 msgr2=0x55e49f75f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fc5a00640 1 --2- 172.21.15.32:0/68366117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e49f7a8020 0x55e49f75f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fc5a00640 1 -- 172.21.15.32:0/68366117 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e49f585540 con 0x55e49f75aa10 2024-04-17T10:00:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fc6400640 1 --2- 172.21.15.32:0/68366117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e49f7a8020 0x55e49f75f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:00:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fc5a00640 1 --2- 172.21.15.32:0/68366117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f75aa10 0x55e49f7625c0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4015770 tx=0x7f4fb4007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d93e5a03b42e87b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fa6a00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:00:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fa6a00640 1 -- 172.21.15.32:0/68366117 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4025060 con 0x55e49f75aa10 2024-04-17T10:00:51.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fa6a00640 1 -- 172.21.15.32:0/68366117 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f4fb4012620 con 0x55e49f75aa10 2024-04-17T10:00:51.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/68366117 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fac0041c0 con 0x55e49f75aa10 2024-04-17T10:00:51.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fa6a00640 1 -- 172.21.15.32:0/68366117 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f4fb402b050 con 0x55e49f75aa10 2024-04-17T10:00:51.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fcb7f7000 10 client.8301 fetch_fsmap learned FSMap version 72 2024-04-17T10:00:51.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/68366117 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55e49f5f0ba0 con 0x55e49f75aa10 2024-04-17T10:00:51.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fa6a00640 1 -- 172.21.15.32:0/68366117 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fac0041c0 con 0x55e49f75aa10 2024-04-17T10:00:51.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fa6a00640 1 -- 172.21.15.32:0/68366117 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 72) ==== 3495+0+0 (secure 0 0 0) 0x7f4fb402e020 con 0x55e49f75aa10 2024-04-17T10:00:51.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fcb7f7000 10 client.8301 fetch_fsmap finished waiting for FSMap version 72 2024-04-17T10:00:51.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fcb7f7000 10 client.8301 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T10:00:51.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fcb7f7000 20 client.8301 populate_metadata read hostname 'smithi032' 2024-04-17T10:00:51.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fcb7f7000 1 --2- 172.21.15.32:0/68366117 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55e49f7b69a0 0x55e49f7b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fcb7f7000 4 client.8301 mds_command: new command op to 8196 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:00:51.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/68366117 --> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e49f665450 con 0x55e49f7b69a0 2024-04-17T10:00:51.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fc5000640 1 --2- 172.21.15.32:0/68366117 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55e49f7b69a0 0x55e49f7b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fc5000640 1 --2- 172.21.15.32:0/68366117 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55e49f7b69a0 0x55e49f7b8d90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44fe8781b0ebbd4d server_cookie=a6ea3aca1bce1b7 in_seq=0 out_seq=0 2024-04-17T10:00:51.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.217+0000 7f4fa6a00640 10 client.8301 ms_handle_connect on v2:172.21.15.32:6835/2058904361 2024-04-17T10:00:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.221+0000 7f4fa6a00640 1 -- 172.21.15.32:0/68366117 <== mds.0 v2:172.21.15.32:6835/2058904361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e49f665450 con 0x55e49f7b69a0 2024-04-17T10:00:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.221+0000 7f4fa6a00640 10 client.8301 handle_command_reply: tid=0 2024-04-17T10:00:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.221+0000 7f4fcb7f7000 1 client.8301 shutdown 2024-04-17T10:00:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.221+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/68366117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f75aa10 msgr2=0x55e49f7625c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.221+0000 7f4fcb7f7000 1 --2- 172.21.15.32:0/68366117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f75aa10 0x55e49f7625c0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4015770 tx=0x7f4fb4007e60 comp rx=0 tx=0).stop 2024-04-17T10:00:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.221+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/68366117 shutdown_connections 2024-04-17T10:00:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.221+0000 7f4fcb7f7000 1 --2- 172.21.15.32:0/68366117 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55e49f7b69a0 0x55e49f7b8d90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.221+0000 7f4fcb7f7000 1 --2- 172.21.15.32:0/68366117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e49f7a7700 0x55e49f762b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.221+0000 7f4fcb7f7000 1 --2- 172.21.15.32:0/68366117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e49f7a8020 0x55e49f75f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.221+0000 7f4fcb7f7000 1 --2- 172.21.15.32:0/68366117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f75aa10 0x55e49f7625c0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.221+0000 7f4fa6a00640 0 client.8301 ms_handle_reset on v2:172.21.15.32:6835/2058904361 2024-04-17T10:00:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.221+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/68366117 >> 172.21.15.32:0/68366117 conn(0x55e49f788290 msgr2=0x55e49f78a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:51.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.221+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/68366117 shutdown_connections 2024-04-17T10:00:51.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.221+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/68366117 wait complete. 2024-04-17T10:00:51.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.221+0000 7f4fcb7f7000 20 client.8301 trim_cache size 0 max 16384 2024-04-17T10:00:51.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.237+0000 7f4fcb7f7000 1 Processor -- start 2024-04-17T10:00:51.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.237+0000 7f4fcb7f7000 1 -- start start 2024-04-17T10:00:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.237+0000 7f4fcb7f7000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e49f75aa10 0x55e49f75f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.237+0000 7f4fcb7f7000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f7a7700 0x55e49f75f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.237+0000 7f4fcb7f7000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e49f7a8020 0x55e49f75e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fcb7f7000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc0202e30 con 0x55e49f7a7700 2024-04-17T10:00:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fcb7f7000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc00b6a00 con 0x55e49f75aa10 2024-04-17T10:00:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fcb7f7000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4fc00b7050 con 0x55e49f7a8020 2024-04-17T10:00:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fc5000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f7a7700 0x55e49f75f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fc5000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f7a7700 0x55e49f75f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35624/0 (socket says 172.21.15.32:35624) 2024-04-17T10:00:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fc5000640 1 -- 172.21.15.32:0/659181096 learned_addr learned my addr 172.21.15.32:0/659181096 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fc6400640 1 --2- 172.21.15.32:0/659181096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e49f7a8020 0x55e49f75e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fc5000640 1 -- 172.21.15.32:0/659181096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e49f7a8020 msgr2=0x55e49f75e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fc5a00640 1 --2- 172.21.15.32:0/659181096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e49f75aa10 0x55e49f75f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fc5000640 1 --2- 172.21.15.32:0/659181096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e49f7a8020 0x55e49f75e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fc5000640 1 -- 172.21.15.32:0/659181096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e49f75aa10 msgr2=0x55e49f75f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fc5000640 1 --2- 172.21.15.32:0/659181096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e49f75aa10 0x55e49f75f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fc5000640 1 -- 172.21.15.32:0/659181096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e49f5b8f70 con 0x55e49f7a7700 2024-04-17T10:00:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fc5a00640 1 --2- 172.21.15.32:0/659181096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e49f75aa10 0x55e49f75f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:00:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fc6400640 1 --2- 172.21.15.32:0/659181096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e49f7a8020 0x55e49f75e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:00:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fc5000640 1 --2- 172.21.15.32:0/659181096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f7a7700 0x55e49f75f650 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f4fac009e60 tx=0x7f4fac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc60bc11f46383c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:51.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fb1600640 1 -- 172.21.15.32:0/659181096 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac033070 con 0x55e49f7a7700 2024-04-17T10:00:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fb1600640 1 -- 172.21.15.32:0/659181096 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fac014650 con 0x55e49f7a7700 2024-04-17T10:00:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fb1600640 1 -- 172.21.15.32:0/659181096 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac014a20 con 0x55e49f7a7700 2024-04-17T10:00:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/659181096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f7a7700 msgr2=0x55e49f75f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fcb7f7000 1 --2- 172.21.15.32:0/659181096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f7a7700 0x55e49f75f650 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f4fac009e60 tx=0x7f4fac016000 comp rx=0 tx=0).stop 2024-04-17T10:00:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/659181096 shutdown_connections 2024-04-17T10:00:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fcb7f7000 1 --2- 172.21.15.32:0/659181096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e49f7a8020 0x55e49f75e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fcb7f7000 1 --2- 172.21.15.32:0/659181096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f7a7700 0x55e49f75f650 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fcb7f7000 1 --2- 172.21.15.32:0/659181096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e49f75aa10 0x55e49f75f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/659181096 >> 172.21.15.32:0/659181096 conn(0x55e49f788290 msgr2=0x55e49f761a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/659181096 shutdown_connections 2024-04-17T10:00:51.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/659181096 wait complete. 2024-04-17T10:00:51.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fcb7f7000 1 -- start start 2024-04-17T10:00:51.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fcb7f7000 1 Processor -- start 2024-04-17T10:00:51.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fcb7f7000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e49f75aa10 0x55e49f75f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fcb7f7000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f7a7700 0x55e49f7b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fcb7f7000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e49f7a8020 0x55e49f7ba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fcb7f7000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e49f5b0c20 con 0x55e49f7a7700 2024-04-17T10:00:51.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fcb7f7000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e49f5ab900 con 0x55e49f7a8020 2024-04-17T10:00:51.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fcb7f7000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e49f5b9800 con 0x55e49f75aa10 2024-04-17T10:00:51.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fcb7f7000 10 client.0 _ll_register_callbacks cb 0x7f4fb4001450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:00:51.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fc5000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f7a7700 0x55e49f7b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fc5000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f7a7700 0x55e49f7b9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35626/0 (socket says 172.21.15.32:35626) 2024-04-17T10:00:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.241+0000 7f4fc5000640 1 -- 172.21.15.32:0/2493527364 learned_addr learned my addr 172.21.15.32:0/2493527364 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fc6400640 1 --2- 172.21.15.32:0/2493527364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e49f7a8020 0x55e49f7ba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fc5a00640 1 --2- 172.21.15.32:0/2493527364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e49f75aa10 0x55e49f75f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fc5000640 1 -- 172.21.15.32:0/2493527364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e49f75aa10 msgr2=0x55e49f75f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fc5000640 1 --2- 172.21.15.32:0/2493527364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e49f75aa10 0x55e49f75f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fc5000640 1 -- 172.21.15.32:0/2493527364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e49f7a8020 msgr2=0x55e49f7ba320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fc5000640 1 --2- 172.21.15.32:0/2493527364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e49f7a8020 0x55e49f7ba320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fc5000640 1 -- 172.21.15.32:0/2493527364 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e49f5659f0 con 0x55e49f7a7700 2024-04-17T10:00:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fc5000640 1 --2- 172.21.15.32:0/2493527364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f7a7700 0x55e49f7b9de0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x55e49f65ea90 tx=0x7f4fac004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227dc219d2adce8c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fa6a00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:00:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fa6a00640 1 -- 172.21.15.32:0/2493527364 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac03c060 con 0x55e49f7a7700 2024-04-17T10:00:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fa6a00640 1 -- 172.21.15.32:0/2493527364 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f4fac014800 con 0x55e49f7a7700 2024-04-17T10:00:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/2493527364 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fac014650 con 0x55e49f7a7700 2024-04-17T10:00:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fa6a00640 1 -- 172.21.15.32:0/2493527364 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f4fac042050 con 0x55e49f7a7700 2024-04-17T10:00:51.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fcb7f7000 10 client.8307 fetch_fsmap learned FSMap version 72 2024-04-17T10:00:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/2493527364 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55e49f551660 con 0x55e49f7a7700 2024-04-17T10:00:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fa6a00640 1 -- 172.21.15.32:0/2493527364 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fac014650 con 0x55e49f7a7700 2024-04-17T10:00:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fa6a00640 1 -- 172.21.15.32:0/2493527364 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 72) ==== 3495+0+0 (secure 0 0 0) 0x7f4fac045020 con 0x55e49f7a7700 2024-04-17T10:00:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fcb7f7000 10 client.8307 fetch_fsmap finished waiting for FSMap version 72 2024-04-17T10:00:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fcb7f7000 10 client.8307 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T10:00:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fcb7f7000 20 client.8307 populate_metadata read hostname 'smithi032' 2024-04-17T10:00:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fcb7f7000 1 --2- 172.21.15.32:0/2493527364 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55e49f7ba860 0x55e49f816bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fcb7f7000 4 client.8307 mds_command: new command op to 8196 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-17T10:00:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/2493527364 --> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x55e49f545ed0 con 0x55e49f7ba860 2024-04-17T10:00:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fc5a00640 1 --2- 172.21.15.32:0/2493527364 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55e49f7ba860 0x55e49f816bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fc5a00640 1 --2- 172.21.15.32:0/2493527364 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55e49f7ba860 0x55e49f816bb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4964250df5ea31ed server_cookie=164decaa52a20151 in_seq=0 out_seq=0 2024-04-17T10:00:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fa6a00640 10 client.8307 ms_handle_connect on v2:172.21.15.32:6835/2058904361 2024-04-17T10:00:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fa6a00640 1 -- 172.21.15.32:0/2493527364 <== mds.0 v2:172.21.15.32:6835/2058904361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e49f545ed0 con 0x55e49f7ba860 2024-04-17T10:00:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fa6a00640 10 client.8307 handle_command_reply: tid=0 2024-04-17T10:00:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fcb7f7000 1 client.8307 shutdown 2024-04-17T10:00:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/2493527364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f7a7700 msgr2=0x55e49f7b9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fcb7f7000 1 --2- 172.21.15.32:0/2493527364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f7a7700 0x55e49f7b9de0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x55e49f65ea90 tx=0x7f4fac004740 comp rx=0 tx=0).stop 2024-04-17T10:00:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/2493527364 shutdown_connections 2024-04-17T10:00:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fcb7f7000 1 --2- 172.21.15.32:0/2493527364 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55e49f7ba860 0x55e49f816bb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fcb7f7000 1 --2- 172.21.15.32:0/2493527364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e49f7a8020 0x55e49f7ba320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fcb7f7000 1 --2- 172.21.15.32:0/2493527364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e49f7a7700 0x55e49f7b9de0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fcb7f7000 1 --2- 172.21.15.32:0/2493527364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e49f75aa10 0x55e49f75f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fa6a00640 0 client.8307 ms_handle_reset on v2:172.21.15.32:6835/2058904361 2024-04-17T10:00:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/2493527364 >> 172.21.15.32:0/2493527364 conn(0x55e49f788290 msgr2=0x55e49f762320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/2493527364 shutdown_connections 2024-04-17T10:00:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fcb7f7000 1 -- 172.21.15.32:0/2493527364 wait complete. 2024-04-17T10:00:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.245+0000 7f4fcb7f7000 20 client.8307 trim_cache size 0 max 16384 2024-04-17T10:00:51.250 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:00:51.250 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-17T10:00:51.250 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "c881fe31-2642-4d09-88bc-644a57ed84b3", 2024-04-17T10:00:51.250 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-17T10:00:51.250 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:00:51.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.249+0000 7f4fc8400640 1 -- 172.21.15.32:0/942014663 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4fa0068380 msgr2=0x7f4fa006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:51.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.249+0000 7f4fc8400640 1 --2- 172.21.15.32:0/942014663 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4fa0068380 0x7f4fa006a840 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4001810 tx=0x7f4fb4009040 comp rx=0 tx=0).stop 2024-04-17T10:00:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.249+0000 7f4fc8400640 1 -- 172.21.15.32:0/942014663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fc015fd20 msgr2=0x7f4fc006ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.249+0000 7f4fc8400640 1 --2- 172.21.15.32:0/942014663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fc015fd20 0x7f4fc006ddf0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8019540 tx=0x7f4fb80196d0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.249+0000 7f4fc8400640 1 -- 172.21.15.32:0/942014663 shutdown_connections 2024-04-17T10:00:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.249+0000 7f4fc8400640 1 --2- 172.21.15.32:0/942014663 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4fa0068380 0x7f4fa006a840 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.249+0000 7f4fc8400640 1 --2- 172.21.15.32:0/942014663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fc015fd20 0x7f4fc006ddf0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.249+0000 7f4fc8400640 1 --2- 172.21.15.32:0/942014663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fc00d0a80 0x7f4fc0165d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.249+0000 7f4fc8400640 1 --2- 172.21.15.32:0/942014663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fc00d00e0 0x7f4fc0159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.249+0000 7f4fc8400640 1 -- 172.21.15.32:0/942014663 >> 172.21.15.32:0/942014663 conn(0x7f4fc00b4000 msgr2=0x7f4fc006ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.249+0000 7f4fc8400640 1 -- 172.21.15.32:0/942014663 shutdown_connections 2024-04-17T10:00:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.249+0000 7f4fc8400640 1 -- 172.21.15.32:0/942014663 wait complete. 2024-04-17T10:00:51.252 INFO:tasks.ceph.mds.c.smithi032.stderr:2024-04-17T10:00:51.249+0000 7f6c5f200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x55ad3f175180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-17T10:00:51.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:00:51.265 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-17T10:00:51.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.409+0000 7f4e17e00640 1 Processor -- start 2024-04-17T10:00:51.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.409+0000 7f4e17e00640 1 -- start start 2024-04-17T10:00:51.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.413+0000 7f4e17e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e101656d0 0x7f4e10167ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.413+0000 7f4e17e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e100d0130 con 0x7f4e1006ba30 2024-04-17T10:00:51.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.413+0000 7f4e17e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e1015d090 con 0x7f4e100cf370 2024-04-17T10:00:51.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.413+0000 7f4e17e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f4e100cfef0 con 0x7f4e101656d0 2024-04-17T10:00:51.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.413+0000 7f4e15e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e101656d0 0x7f4e10167ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.413+0000 7f4e15400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4e1006ba30 0x7f4e1006be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46668/0 (socket says 172.21.15.32:46668) 2024-04-17T10:00:51.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.413+0000 7f4e15400640 1 -- 172.21.15.32:0/2882076274 learned_addr learned my addr 172.21.15.32:0/2882076274 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:00:51.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.413+0000 7f4e15e00640 1 -- 172.21.15.32:0/2882076274 >> v1:172.21.15.144:6789/0 conn(0x7f4e100cf370 legacy=0x7f4e100cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:51.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.413+0000 7f4e15e00640 1 -- 172.21.15.32:0/2882076274 >> v1:172.21.15.32:6789/0 conn(0x7f4e1006ba30 legacy=0x7f4e1006be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:51.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.413+0000 7f4e0fe00640 1 -- 172.21.15.32:0/2882076274 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1091965901 0 0) 0x7f4e100d0130 con 0x7f4e1006ba30 2024-04-17T10:00:51.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.413+0000 7f4e15e00640 1 -- 172.21.15.32:0/2882076274 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1015d370 con 0x7f4e101656d0 2024-04-17T10:00:51.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.417+0000 7f4e15e00640 1 --2- 172.21.15.32:0/2882076274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e101656d0 0x7f4e10167ac0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f4e08009d90 tx=0x7f4e0801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc568f477c5a73ab server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:51.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.417+0000 7f4e0fe00640 1 -- 172.21.15.32:0/2882076274 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e080b2070 con 0x7f4e101656d0 2024-04-17T10:00:51.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.417+0000 7f4e0fe00640 1 -- 172.21.15.32:0/2882076274 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e080a58b0 con 0x7f4e101656d0 2024-04-17T10:00:51.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.417+0000 7f4e0fe00640 1 -- 172.21.15.32:0/2882076274 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e080a5c80 con 0x7f4e101656d0 2024-04-17T10:00:51.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.417+0000 7f4e17e00640 1 -- 172.21.15.32:0/2882076274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e101656d0 msgr2=0x7f4e10167ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:51.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.417+0000 7f4e17e00640 1 --2- 172.21.15.32:0/2882076274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e101656d0 0x7f4e10167ac0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f4e08009d90 tx=0x7f4e0801c8e0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.417+0000 7f4e17e00640 1 -- 172.21.15.32:0/2882076274 shutdown_connections 2024-04-17T10:00:51.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.417+0000 7f4e17e00640 1 --2- 172.21.15.32:0/2882076274 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e101656d0 0x7f4e10167ac0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.417+0000 7f4e17e00640 1 -- 172.21.15.32:0/2882076274 >> 172.21.15.32:0/2882076274 conn(0x7f4e100b2120 msgr2=0x7f4e100b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:51.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.417+0000 7f4e17e00640 1 -- 172.21.15.32:0/2882076274 shutdown_connections 2024-04-17T10:00:51.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.417+0000 7f4e17e00640 1 -- 172.21.15.32:0/2882076274 wait complete. 2024-04-17T10:00:51.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.417+0000 7f4e17e00640 1 Processor -- start 2024-04-17T10:00:51.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.417+0000 7f4e17e00640 1 -- start start 2024-04-17T10:00:51.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.417+0000 7f4e17e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e1006ba30 0x7f4e1015b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.417+0000 7f4e17e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e100cf370 0x7f4e100aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.417+0000 7f4e17e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e101656d0 0x7f4e100b3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.417+0000 7f4e17e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4e101463b0 con 0x7f4e100cf370 2024-04-17T10:00:51.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.417+0000 7f4e17e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4e10146230 con 0x7f4e1006ba30 2024-04-17T10:00:51.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.417+0000 7f4e17e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4e1016a820 con 0x7f4e101656d0 2024-04-17T10:00:51.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.417+0000 7f4e14a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e100cf370 0x7f4e100aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.417+0000 7f4e14a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e100cf370 0x7f4e100aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35636/0 (socket says 172.21.15.32:35636) 2024-04-17T10:00:51.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.417+0000 7f4e14a00640 1 -- 172.21.15.32:0/681168094 learned_addr learned my addr 172.21.15.32:0/681168094 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:51.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.421+0000 7f4e15e00640 1 --2- 172.21.15.32:0/681168094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e101656d0 0x7f4e100b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.421+0000 7f4e15400640 1 --2- 172.21.15.32:0/681168094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e1006ba30 0x7f4e1015b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.421+0000 7f4e14a00640 1 -- 172.21.15.32:0/681168094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e101656d0 msgr2=0x7f4e100b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:51.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.421+0000 7f4e14a00640 1 --2- 172.21.15.32:0/681168094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e101656d0 0x7f4e100b3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.421+0000 7f4e14a00640 1 -- 172.21.15.32:0/681168094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e1006ba30 msgr2=0x7f4e1015b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:51.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.421+0000 7f4e14a00640 1 --2- 172.21.15.32:0/681168094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e1006ba30 0x7f4e1015b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.421+0000 7f4e14a00640 1 -- 172.21.15.32:0/681168094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1015bb20 con 0x7f4e100cf370 2024-04-17T10:00:51.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.421+0000 7f4e15400640 1 --2- 172.21.15.32:0/681168094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e1006ba30 0x7f4e1015b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:00:51.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.421+0000 7f4e14a00640 1 --2- 172.21.15.32:0/681168094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e100cf370 0x7f4e100aeb10 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00c500 tx=0x7f4dfc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:51.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.421+0000 7f4e0d600640 1 -- 172.21.15.32:0/681168094 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc016020 con 0x7f4e100cf370 2024-04-17T10:00:51.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.421+0000 7f4e0d600640 1 -- 172.21.15.32:0/681168094 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dfc005220 con 0x7f4e100cf370 2024-04-17T10:00:51.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.421+0000 7f4e0d600640 1 -- 172.21.15.32:0/681168094 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc007030 con 0x7f4e100cf370 2024-04-17T10:00:51.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.421+0000 7f4e17e00640 1 -- 172.21.15.32:0/681168094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e10167040 con 0x7f4e100cf370 2024-04-17T10:00:51.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.421+0000 7f4e17e00640 1 -- 172.21.15.32:0/681168094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e100ae5f0 con 0x7f4e100cf370 2024-04-17T10:00:51.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.421+0000 7f4e0d600640 1 -- 172.21.15.32:0/681168094 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4dfc0071f0 con 0x7f4e100cf370 2024-04-17T10:00:51.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.421+0000 7f4e0d600640 1 --2- 172.21.15.32:0/681168094 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4df0068380 0x7f4df006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.421+0000 7f4e0d600640 1 -- 172.21.15.32:0/681168094 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f4dfc085ea0 con 0x7f4e100cf370 2024-04-17T10:00:51.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e15400640 1 --2- 172.21.15.32:0/681168094 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4df0068380 0x7f4df006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e1b233000 1 Processor -- start 2024-04-17T10:00:51.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e1b233000 1 -- start start 2024-04-17T10:00:51.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e15400640 1 --2- 172.21.15.32:0/681168094 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4df0068380 0x7f4df006a840 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4e04000f70 tx=0x7f4e04007c60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:51.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e1b233000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52260 0x55eb7df52640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e1b233000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb7df52b80 0x55eb7df05460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e1b233000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eb7df059a0 0x55eb7df05d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e1b233000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55eb7dd574f0 con 0x55eb7df52260 2024-04-17T10:00:51.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e1b233000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55eb7dd106c0 con 0x55eb7df52b80 2024-04-17T10:00:51.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e1b233000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55eb7dd534b0 con 0x55eb7df059a0 2024-04-17T10:00:51.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e15400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52260 0x55eb7df52640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e15400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52260 0x55eb7df52640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35650/0 (socket says 172.21.15.32:35650) 2024-04-17T10:00:51.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e15400640 1 -- 172.21.15.32:0/1287957023 learned_addr learned my addr 172.21.15.32:0/1287957023 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:51.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e15e00640 1 --2- 172.21.15.32:0/1287957023 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eb7df059a0 0x55eb7df05d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e14a00640 1 --2- 172.21.15.32:0/1287957023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb7df52b80 0x55eb7df05460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e15400640 1 -- 172.21.15.32:0/1287957023 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eb7df059a0 msgr2=0x55eb7df05d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:51.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e15400640 1 --2- 172.21.15.32:0/1287957023 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eb7df059a0 0x55eb7df05d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e15400640 1 -- 172.21.15.32:0/1287957023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb7df52b80 msgr2=0x55eb7df05460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:51.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e15400640 1 --2- 172.21.15.32:0/1287957023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb7df52b80 0x55eb7df05460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e15400640 1 -- 172.21.15.32:0/1287957023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb7dd9c8e0 con 0x55eb7df52260 2024-04-17T10:00:51.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e15400640 1 --2- 172.21.15.32:0/1287957023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52260 0x55eb7df52640 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f4e04005d70 tx=0x7f4e04012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0fbde488b9123c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:51.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e02000640 1 -- 172.21.15.32:0/1287957023 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e04015110 con 0x55eb7df52260 2024-04-17T10:00:51.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e02000640 1 -- 172.21.15.32:0/1287957023 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e040159c0 con 0x55eb7df52260 2024-04-17T10:00:51.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e02000640 1 -- 172.21.15.32:0/1287957023 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e04015d70 con 0x55eb7df52260 2024-04-17T10:00:51.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e1b233000 1 -- 172.21.15.32:0/1287957023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52260 msgr2=0x55eb7df52640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:51.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e1b233000 1 --2- 172.21.15.32:0/1287957023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52260 0x55eb7df52640 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f4e04005d70 tx=0x7f4e04012000 comp rx=0 tx=0).stop 2024-04-17T10:00:51.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e1b233000 1 -- 172.21.15.32:0/1287957023 shutdown_connections 2024-04-17T10:00:51.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e1b233000 1 --2- 172.21.15.32:0/1287957023 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eb7df059a0 0x55eb7df05d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e1b233000 1 --2- 172.21.15.32:0/1287957023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb7df52b80 0x55eb7df05460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e1b233000 1 --2- 172.21.15.32:0/1287957023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52260 0x55eb7df52640 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e1b233000 1 -- 172.21.15.32:0/1287957023 >> 172.21.15.32:0/1287957023 conn(0x55eb7df33000 msgr2=0x55eb7df353f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:51.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e1b233000 1 -- 172.21.15.32:0/1287957023 shutdown_connections 2024-04-17T10:00:51.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.425+0000 7f4e1b233000 1 -- 172.21.15.32:0/1287957023 wait complete. 2024-04-17T10:00:51.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e1b233000 1 -- start start 2024-04-17T10:00:51.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e1b233000 1 Processor -- start 2024-04-17T10:00:51.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e1b233000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb7df059a0 0x55eb7df0b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e1b233000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52260 0x55eb7df0b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e1b233000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eb7df52b80 0x55eb7df0baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e1b233000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55eb7ddd87b0 con 0x55eb7df52260 2024-04-17T10:00:51.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e1b233000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55eb7de12d00 con 0x55eb7df059a0 2024-04-17T10:00:51.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e1b233000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55eb7dd1c2c0 con 0x55eb7df52b80 2024-04-17T10:00:51.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e1b233000 10 client.0 _ll_register_callbacks cb 0x55eb7dd2c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:00:51.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e14a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52260 0x55eb7df0b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e14a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52260 0x55eb7df0b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35654/0 (socket says 172.21.15.32:35654) 2024-04-17T10:00:51.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e14a00640 1 -- 172.21.15.32:0/55001950 learned_addr learned my addr 172.21.15.32:0/55001950 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:51.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e15400640 1 --2- 172.21.15.32:0/55001950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb7df059a0 0x55eb7df0b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e15e00640 1 --2- 172.21.15.32:0/55001950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eb7df52b80 0x55eb7df0baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e14a00640 1 -- 172.21.15.32:0/55001950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eb7df52b80 msgr2=0x55eb7df0baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:51.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e14a00640 1 --2- 172.21.15.32:0/55001950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eb7df52b80 0x55eb7df0baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e14a00640 1 -- 172.21.15.32:0/55001950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb7df059a0 msgr2=0x55eb7df0b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:51.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e14a00640 1 --2- 172.21.15.32:0/55001950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb7df059a0 0x55eb7df0b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e14a00640 1 -- 172.21.15.32:0/55001950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb7dd2f490 con 0x55eb7df52260 2024-04-17T10:00:51.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e14a00640 1 --2- 172.21.15.32:0/55001950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52260 0x55eb7df0b560 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00cec0 tx=0x7f4dfc055dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ffef3f156b1999 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:51.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4df6a00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:00:51.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4df6a00640 1 -- 172.21.15.32:0/55001950 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc054050 con 0x55eb7df52260 2024-04-17T10:00:51.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4df6a00640 1 -- 172.21.15.32:0/55001950 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f4dfc0550e0 con 0x55eb7df52260 2024-04-17T10:00:51.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e1b233000 1 -- 172.21.15.32:0/55001950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e040159c0 con 0x55eb7df52260 2024-04-17T10:00:51.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4df6a00640 1 -- 172.21.15.32:0/55001950 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f4dfc082050 con 0x55eb7df52260 2024-04-17T10:00:51.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e1b233000 10 client.8322 fetch_fsmap learned FSMap version 72 2024-04-17T10:00:51.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e1b233000 1 -- 172.21.15.32:0/55001950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55eb7dd22eb0 con 0x55eb7df52260 2024-04-17T10:00:51.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4df6a00640 1 -- 172.21.15.32:0/55001950 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e040159c0 con 0x55eb7df52260 2024-04-17T10:00:51.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4df6a00640 1 -- 172.21.15.32:0/55001950 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 72) ==== 3495+0+0 (secure 0 0 0) 0x7f4dfc05c1a0 con 0x55eb7df52260 2024-04-17T10:00:51.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e1b233000 10 client.8322 fetch_fsmap finished waiting for FSMap version 72 2024-04-17T10:00:51.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e1b233000 10 client.8322 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T10:00:51.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e1b233000 20 client.8322 populate_metadata read hostname 'smithi032' 2024-04-17T10:00:51.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e1b233000 1 --2- 172.21.15.32:0/55001950 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55eb7df61500 0x55eb7df638f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e1b233000 4 client.8322 mds_command: new command op to 8196 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:00:51.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e1b233000 1 -- 172.21.15.32:0/55001950 --> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb7dcf0dd0 con 0x55eb7df61500 2024-04-17T10:00:51.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.429+0000 7f4e15400640 1 --2- 172.21.15.32:0/55001950 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55eb7df61500 0x55eb7df638f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.433+0000 7f4e15400640 1 --2- 172.21.15.32:0/55001950 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55eb7df61500 0x55eb7df638f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2ac193db6add0fb server_cookie=e227f3f35165975b in_seq=0 out_seq=0 2024-04-17T10:00:51.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.433+0000 7f4df6a00640 10 client.8322 ms_handle_connect on v2:172.21.15.32:6835/2058904361 2024-04-17T10:00:51.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.433+0000 7f4df6a00640 1 -- 172.21.15.32:0/55001950 <== mds.0 v2:172.21.15.32:6835/2058904361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb7dcf0dd0 con 0x55eb7df61500 2024-04-17T10:00:51.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.433+0000 7f4df6a00640 10 client.8322 handle_command_reply: tid=0 2024-04-17T10:00:51.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.433+0000 7f4e1b233000 1 client.8322 shutdown 2024-04-17T10:00:51.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.433+0000 7f4e1b233000 1 -- 172.21.15.32:0/55001950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52260 msgr2=0x55eb7df0b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:51.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.433+0000 7f4e1b233000 1 --2- 172.21.15.32:0/55001950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52260 0x55eb7df0b560 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00cec0 tx=0x7f4dfc055dd0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.433+0000 7f4e1b233000 1 -- 172.21.15.32:0/55001950 shutdown_connections 2024-04-17T10:00:51.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.433+0000 7f4e1b233000 1 --2- 172.21.15.32:0/55001950 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55eb7df61500 0x55eb7df638f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.433+0000 7f4e1b233000 1 --2- 172.21.15.32:0/55001950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eb7df52b80 0x55eb7df0baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.433+0000 7f4e1b233000 1 --2- 172.21.15.32:0/55001950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52260 0x55eb7df0b560 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.433+0000 7f4e1b233000 1 --2- 172.21.15.32:0/55001950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb7df059a0 0x55eb7df0b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.433+0000 7f4df6a00640 0 client.8322 ms_handle_reset on v2:172.21.15.32:6835/2058904361 2024-04-17T10:00:51.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.433+0000 7f4e1b233000 1 -- 172.21.15.32:0/55001950 >> 172.21.15.32:0/55001950 conn(0x55eb7df33000 msgr2=0x55eb7df353f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:51.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.433+0000 7f4e1b233000 1 -- 172.21.15.32:0/55001950 shutdown_connections 2024-04-17T10:00:51.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.433+0000 7f4e1b233000 1 -- 172.21.15.32:0/55001950 wait complete. 2024-04-17T10:00:51.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.433+0000 7f4e1b233000 20 client.8322 trim_cache size 0 max 16384 2024-04-17T10:00:51.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.449+0000 7f4e1b233000 1 Processor -- start 2024-04-17T10:00:51.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.449+0000 7f4e1b233000 1 -- start start 2024-04-17T10:00:51.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.449+0000 7f4e1b233000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb7df059a0 0x55eb7df04b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.449+0000 7f4e1b233000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eb7df52260 0x55eb7df050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.449+0000 7f4e1b233000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52b80 0x55eb7df638f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.449+0000 7f4e1b233000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4e10202e40 con 0x55eb7df52b80 2024-04-17T10:00:51.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.449+0000 7f4e1b233000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4e102021a0 con 0x55eb7df059a0 2024-04-17T10:00:51.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.449+0000 7f4e1b233000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4e102027f0 con 0x55eb7df52260 2024-04-17T10:00:51.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.449+0000 7f4e15e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52b80 0x55eb7df638f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.449+0000 7f4e15e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52b80 0x55eb7df638f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35658/0 (socket says 172.21.15.32:35658) 2024-04-17T10:00:51.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.449+0000 7f4e15e00640 1 -- 172.21.15.32:0/2948663628 learned_addr learned my addr 172.21.15.32:0/2948663628 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:51.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e14a00640 1 --2- 172.21.15.32:0/2948663628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eb7df52260 0x55eb7df050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e15400640 1 --2- 172.21.15.32:0/2948663628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb7df059a0 0x55eb7df04b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e15e00640 1 -- 172.21.15.32:0/2948663628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eb7df52260 msgr2=0x55eb7df050a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:51.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e15e00640 1 --2- 172.21.15.32:0/2948663628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eb7df52260 0x55eb7df050a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e15e00640 1 -- 172.21.15.32:0/2948663628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb7df059a0 msgr2=0x55eb7df04b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:51.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e15e00640 1 --2- 172.21.15.32:0/2948663628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb7df059a0 0x55eb7df04b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e15e00640 1 -- 172.21.15.32:0/2948663628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb7dd11a20 con 0x55eb7df52b80 2024-04-17T10:00:51.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e15400640 1 --2- 172.21.15.32:0/2948663628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb7df059a0 0x55eb7df04b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:00:51.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e15e00640 1 --2- 172.21.15.32:0/2948663628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52b80 0x55eb7df638f0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f4e08009d90 tx=0x7f4e08004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee50b0be2dd1948 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:51.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e01600640 1 -- 172.21.15.32:0/2948663628 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e080c7070 con 0x55eb7df52b80 2024-04-17T10:00:51.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e01600640 1 -- 172.21.15.32:0/2948663628 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e080a58a0 con 0x55eb7df52b80 2024-04-17T10:00:51.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e01600640 1 -- 172.21.15.32:0/2948663628 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e080b2040 con 0x55eb7df52b80 2024-04-17T10:00:51.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e1b233000 1 -- 172.21.15.32:0/2948663628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52b80 msgr2=0x55eb7df638f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:51.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e1b233000 1 --2- 172.21.15.32:0/2948663628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52b80 0x55eb7df638f0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f4e08009d90 tx=0x7f4e08004ac0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e1b233000 1 -- 172.21.15.32:0/2948663628 shutdown_connections 2024-04-17T10:00:51.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e1b233000 1 --2- 172.21.15.32:0/2948663628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eb7df52260 0x55eb7df050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e1b233000 1 --2- 172.21.15.32:0/2948663628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52b80 0x55eb7df638f0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e1b233000 1 --2- 172.21.15.32:0/2948663628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb7df059a0 0x55eb7df04b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e1b233000 1 -- 172.21.15.32:0/2948663628 >> 172.21.15.32:0/2948663628 conn(0x55eb7df33000 msgr2=0x55eb7df04450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:51.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e1b233000 1 -- 172.21.15.32:0/2948663628 shutdown_connections 2024-04-17T10:00:51.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e1b233000 1 -- 172.21.15.32:0/2948663628 wait complete. 2024-04-17T10:00:51.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e1b233000 1 -- start start 2024-04-17T10:00:51.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e1b233000 1 Processor -- start 2024-04-17T10:00:51.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e1b233000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eb7df059a0 0x55eb7df66970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e1b233000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52260 0x55eb7df7d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e1b233000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb7df52b80 0x55eb7df7d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e1b233000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55eb7dcfb050 con 0x55eb7df52260 2024-04-17T10:00:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e1b233000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55eb7dd19570 con 0x55eb7df52b80 2024-04-17T10:00:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e1b233000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55eb7dd56800 con 0x55eb7df059a0 2024-04-17T10:00:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e1b233000 10 client.0 _ll_register_callbacks cb 0x7f4dfc055a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:00:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e14a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52260 0x55eb7df7d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e14a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52260 0x55eb7df7d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35674/0 (socket says 172.21.15.32:35674) 2024-04-17T10:00:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e14a00640 1 -- 172.21.15.32:0/265562939 learned_addr learned my addr 172.21.15.32:0/265562939 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:00:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e15400640 1 --2- 172.21.15.32:0/265562939 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eb7df059a0 0x55eb7df66970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.453+0000 7f4e15e00640 1 --2- 172.21.15.32:0/265562939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb7df52b80 0x55eb7df7d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e14a00640 1 -- 172.21.15.32:0/265562939 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eb7df059a0 msgr2=0x55eb7df66970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e14a00640 1 --2- 172.21.15.32:0/265562939 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eb7df059a0 0x55eb7df66970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e14a00640 1 -- 172.21.15.32:0/265562939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb7df52b80 msgr2=0x55eb7df7d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:00:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e14a00640 1 --2- 172.21.15.32:0/265562939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb7df52b80 0x55eb7df7d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e14a00640 1 -- 172.21.15.32:0/265562939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb7ddd8c40 con 0x55eb7df52260 2024-04-17T10:00:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e14a00640 1 --2- 172.21.15.32:0/265562939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52260 0x55eb7df7d480 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc0520f0 tx=0x7f4dfc0841a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ef5ed622b9a15a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:00:51.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4df6a00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:00:51.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4df6a00640 1 -- 172.21.15.32:0/265562939 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc0555f0 con 0x55eb7df52260 2024-04-17T10:00:51.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4df6a00640 1 -- 172.21.15.32:0/265562939 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f4dfc074af0 con 0x55eb7df52260 2024-04-17T10:00:51.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e1b233000 1 -- 172.21.15.32:0/265562939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e080a58a0 con 0x55eb7df52260 2024-04-17T10:00:51.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4df6a00640 1 -- 172.21.15.32:0/265562939 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f4dfc054050 con 0x55eb7df52260 2024-04-17T10:00:51.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e1b233000 10 client.8328 fetch_fsmap learned FSMap version 72 2024-04-17T10:00:51.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e1b233000 1 -- 172.21.15.32:0/265562939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55eb7dcfc600 con 0x55eb7df52260 2024-04-17T10:00:51.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4df6a00640 1 -- 172.21.15.32:0/265562939 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e080a58a0 con 0x55eb7df52260 2024-04-17T10:00:51.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4df6a00640 1 -- 172.21.15.32:0/265562939 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 72) ==== 3495+0+0 (secure 0 0 0) 0x7f4dfc0519e0 con 0x55eb7df52260 2024-04-17T10:00:51.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e1b233000 10 client.8328 fetch_fsmap finished waiting for FSMap version 72 2024-04-17T10:00:51.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e1b233000 10 client.8328 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T10:00:51.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e1b233000 20 client.8328 populate_metadata read hostname 'smithi032' 2024-04-17T10:00:51.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e1b233000 1 --2- 172.21.15.32:0/265562939 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55eb7dfc1300 0x55eb7dfc1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:00:51.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e1b233000 4 client.8328 mds_command: new command op to 8196 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:00:51.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e1b233000 1 -- 172.21.15.32:0/265562939 --> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eb7dd64f10 con 0x55eb7dfc1300 2024-04-17T10:00:51.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e15400640 1 --2- 172.21.15.32:0/265562939 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55eb7dfc1300 0x55eb7dfc1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:00:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e15400640 1 --2- 172.21.15.32:0/265562939 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55eb7dfc1300 0x55eb7dfc1710 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12aedc1be12e48e9 server_cookie=d4ec123a5994e10 in_seq=0 out_seq=0 2024-04-17T10:00:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4df6a00640 10 client.8328 ms_handle_connect on v2:172.21.15.32:6835/2058904361 2024-04-17T10:00:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4df6a00640 1 -- 172.21.15.32:0/265562939 <== mds.0 v2:172.21.15.32:6835/2058904361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x55eb7dd64f10 con 0x55eb7dfc1300 2024-04-17T10:00:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4df6a00640 10 client.8328 handle_command_reply: tid=0 2024-04-17T10:00:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e1b233000 1 client.8328 shutdown 2024-04-17T10:00:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e1b233000 1 -- 172.21.15.32:0/265562939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52260 msgr2=0x55eb7df7d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e1b233000 1 --2- 172.21.15.32:0/265562939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52260 0x55eb7df7d480 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc0520f0 tx=0x7f4dfc0841a0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e1b233000 1 -- 172.21.15.32:0/265562939 shutdown_connections 2024-04-17T10:00:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e1b233000 1 --2- 172.21.15.32:0/265562939 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55eb7dfc1300 0x55eb7dfc1710 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e1b233000 1 --2- 172.21.15.32:0/265562939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb7df52b80 0x55eb7df7d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e1b233000 1 --2- 172.21.15.32:0/265562939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eb7df52260 0x55eb7df7d480 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e1b233000 1 --2- 172.21.15.32:0/265562939 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55eb7df059a0 0x55eb7df66970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4df6a00640 0 client.8328 ms_handle_reset on v2:172.21.15.32:6835/2058904361 2024-04-17T10:00:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e1b233000 1 -- 172.21.15.32:0/265562939 >> 172.21.15.32:0/265562939 conn(0x55eb7df33000 msgr2=0x55eb7df04b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e1b233000 1 -- 172.21.15.32:0/265562939 shutdown_connections 2024-04-17T10:00:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e1b233000 1 -- 172.21.15.32:0/265562939 wait complete. 2024-04-17T10:00:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e1b233000 20 client.8328 trim_cache size 0 max 16384 2024-04-17T10:00:51.460 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:00:51.460 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T10:00:51.460 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-17T10:00:51.461 INFO:teuthology.orchestra.run.smithi032.stdout: "c881fe31-2642-4d09-88bc-644a57ed84b3": { 2024-04-17T10:00:51.461 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/dir", 2024-04-17T10:00:51.461 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "c881fe31-2642-4d09-88bc-644a57ed84b3", 2024-04-17T10:00:51.461 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,repair" 2024-04-17T10:00:51.461 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:00:51.461 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:00:51.461 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:00:51.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e17e00640 1 -- 172.21.15.32:0/681168094 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4df0068380 msgr2=0x7f4df006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:51.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e17e00640 1 --2- 172.21.15.32:0/681168094 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4df0068380 0x7f4df006a840 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4e04000f70 tx=0x7f4e04007c60 comp rx=0 tx=0).stop 2024-04-17T10:00:51.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e17e00640 1 -- 172.21.15.32:0/681168094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e100cf370 msgr2=0x7f4e100aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:00:51.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e17e00640 1 --2- 172.21.15.32:0/681168094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e100cf370 0x7f4e100aeb10 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00c500 tx=0x7f4dfc00c9c0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e17e00640 1 -- 172.21.15.32:0/681168094 shutdown_connections 2024-04-17T10:00:51.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e17e00640 1 --2- 172.21.15.32:0/681168094 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4df0068380 0x7f4df006a840 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e17e00640 1 --2- 172.21.15.32:0/681168094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e101656d0 0x7f4e100b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e17e00640 1 --2- 172.21.15.32:0/681168094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e100cf370 0x7f4e100aeb10 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e17e00640 1 --2- 172.21.15.32:0/681168094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e1006ba30 0x7f4e1015b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:00:51.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e17e00640 1 -- 172.21.15.32:0/681168094 >> 172.21.15.32:0/681168094 conn(0x7f4e100b2120 msgr2=0x7f4e100b1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:00:51.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e17e00640 1 -- 172.21.15.32:0/681168094 shutdown_connections 2024-04-17T10:00:51.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:00:51.457+0000 7f4e17e00640 1 -- 172.21.15.32:0/681168094 wait complete. 2024-04-17T10:00:51.476 INFO:tasks.cephfs.filesystem:scrub status for tag:c881fe31-2642-4d09-88bc-644a57ed84b3 - {'path': '/dir', 'tag': 'c881fe31-2642-4d09-88bc-644a57ed84b3', 'options': 'recursive,repair'} 2024-04-17T10:01:14.122 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:01:14.126 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:01:21.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:01:21.477 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-17T10:01:21.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.637+0000 7f8efb600640 1 Processor -- start 2024-04-17T10:01:21.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.637+0000 7f8efb600640 1 -- start start 2024-04-17T10:01:21.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.641+0000 7f8efb600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ef406ba30 0x7f8ef406be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.641+0000 7f8efb600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8ef40cf370 0x7f8ef40cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.641+0000 7f8efb600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ef406c350 con 0x7f8ef4160350 2024-04-17T10:01:21.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.641+0000 7f8efb600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f8ef40d0130 con 0x7f8ef40cf370 2024-04-17T10:01:21.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.641+0000 7f8efb600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef415d090 con 0x7f8ef406ba30 2024-04-17T10:01:21.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.641+0000 7f8ef8c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ef406ba30 0x7f8ef406be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.641+0000 7f8eebe00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8ef40cf370 0x7f8ef40cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.641+0000 7f8ef8c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ef406ba30 0x7f8ef406be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:35674/0 (socket says 172.21.15.32:35674) 2024-04-17T10:01:21.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.641+0000 7f8ef8c00640 1 -- 172.21.15.32:0/3313780070 learned_addr learned my addr 172.21.15.32:0/3313780070 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:21.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.641+0000 7f8ef8c00640 1 -- 172.21.15.32:0/3313780070 >> v2:172.21.15.144:3300/0 conn(0x7f8ef40cf370 msgr2=0x7f8ef40cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:21.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.641+0000 7f8ef8c00640 1 --2- 172.21.15.32:0/3313780070 >> v2:172.21.15.144:3300/0 conn(0x7f8ef40cf370 0x7f8ef40cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.641+0000 7f8ef8c00640 1 -- 172.21.15.32:0/3313780070 >> v1:172.21.15.144:6789/0 conn(0x7f8ef4160350 legacy=0x7f8ef41607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:21.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.641+0000 7f8ef8c00640 1 -- 172.21.15.32:0/3313780070 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef415d370 con 0x7f8ef406ba30 2024-04-17T10:01:21.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.645+0000 7f8ef8c00640 1 --2- 172.21.15.32:0/3313780070 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ef406ba30 0x7f8ef406be10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4009870 tx=0x7f8ee401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0908c6921a5a75d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:21.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.645+0000 7f8ef3e00640 1 -- 172.21.15.32:0/3313780070 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee40a7070 con 0x7f8ef406ba30 2024-04-17T10:01:21.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.645+0000 7f8ef3e00640 1 -- 172.21.15.32:0/3313780070 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ee40b0070 con 0x7f8ef406ba30 2024-04-17T10:01:21.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.645+0000 7f8ef3e00640 1 -- 172.21.15.32:0/3313780070 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee40ab890 con 0x7f8ef406ba30 2024-04-17T10:01:21.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.645+0000 7f8efb600640 1 -- 172.21.15.32:0/3313780070 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ef406ba30 msgr2=0x7f8ef406be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:21.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.645+0000 7f8efb600640 1 --2- 172.21.15.32:0/3313780070 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ef406ba30 0x7f8ef406be10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4009870 tx=0x7f8ee401c560 comp rx=0 tx=0).stop 2024-04-17T10:01:21.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.645+0000 7f8efb600640 1 -- 172.21.15.32:0/3313780070 shutdown_connections 2024-04-17T10:01:21.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.645+0000 7f8efb600640 1 --2- 172.21.15.32:0/3313780070 >> v2:172.21.15.144:3300/0 conn(0x7f8ef40cf370 0x7f8ef40cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.645+0000 7f8efb600640 1 --2- 172.21.15.32:0/3313780070 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ef406ba30 0x7f8ef406be10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.645+0000 7f8efb600640 1 -- 172.21.15.32:0/3313780070 >> 172.21.15.32:0/3313780070 conn(0x7f8ef40b2120 msgr2=0x7f8ef40b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:21.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.645+0000 7f8efb600640 1 -- 172.21.15.32:0/3313780070 shutdown_connections 2024-04-17T10:01:21.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.645+0000 7f8efb600640 1 -- 172.21.15.32:0/3313780070 wait complete. 2024-04-17T10:01:21.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.645+0000 7f8efb600640 1 Processor -- start 2024-04-17T10:01:21.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.645+0000 7f8efb600640 1 -- start start 2024-04-17T10:01:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.645+0000 7f8efb600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ef406ba30 0x7f8ef415e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.645+0000 7f8efb600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ef40cf370 0x7f8ef41fae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.645+0000 7f8efb600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ef4160350 0x7f8ef4166230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.645+0000 7f8efb600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef415dcf0 con 0x7f8ef40cf370 2024-04-17T10:01:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.645+0000 7f8efb600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef415db70 con 0x7f8ef406ba30 2024-04-17T10:01:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.645+0000 7f8efb600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef4146230 con 0x7f8ef4160350 2024-04-17T10:01:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.649+0000 7f8eebe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ef40cf370 0x7f8ef41fae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.649+0000 7f8eebe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ef40cf370 0x7f8ef41fae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53198/0 (socket says 172.21.15.32:53198) 2024-04-17T10:01:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.649+0000 7f8eebe00640 1 -- 172.21.15.32:0/2596337356 learned_addr learned my addr 172.21.15.32:0/2596337356 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.649+0000 7f8ef8c00640 1 --2- 172.21.15.32:0/2596337356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ef406ba30 0x7f8ef415e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.649+0000 7f8ef9600640 1 --2- 172.21.15.32:0/2596337356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ef4160350 0x7f8ef4166230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.649+0000 7f8eebe00640 1 -- 172.21.15.32:0/2596337356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ef4160350 msgr2=0x7f8ef4166230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.649+0000 7f8eebe00640 1 --2- 172.21.15.32:0/2596337356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ef4160350 0x7f8ef4166230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.649+0000 7f8eebe00640 1 -- 172.21.15.32:0/2596337356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ef406ba30 msgr2=0x7f8ef415e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.649+0000 7f8eebe00640 1 --2- 172.21.15.32:0/2596337356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ef406ba30 0x7f8ef415e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.649+0000 7f8eebe00640 1 -- 172.21.15.32:0/2596337356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef41fb8a0 con 0x7f8ef40cf370 2024-04-17T10:01:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.649+0000 7f8eebe00640 1 --2- 172.21.15.32:0/2596337356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ef40cf370 0x7f8ef41fae20 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00e960 tx=0x7f8edc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.649+0000 7f8ef1600640 1 -- 172.21.15.32:0/2596337356 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc017070 con 0x7f8ef40cf370 2024-04-17T10:01:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.649+0000 7f8ef1600640 1 -- 172.21.15.32:0/2596337356 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8edc00ca00 con 0x7f8ef40cf370 2024-04-17T10:01:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.649+0000 7f8ef1600640 1 -- 172.21.15.32:0/2596337356 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc00cdb0 con 0x7f8ef40cf370 2024-04-17T10:01:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.649+0000 7f8efb600640 1 -- 172.21.15.32:0/2596337356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef41fbb80 con 0x7f8ef40cf370 2024-04-17T10:01:21.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.649+0000 7f8efb600640 1 -- 172.21.15.32:0/2596337356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef413d0d0 con 0x7f8ef40cf370 2024-04-17T10:01:21.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.649+0000 7f8ef1600640 1 -- 172.21.15.32:0/2596337356 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8edc01a070 con 0x7f8ef40cf370 2024-04-17T10:01:21.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8ef1600640 1 --2- 172.21.15.32:0/2596337356 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8ed0068380 0x7f8ed006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8ef1600640 1 -- 172.21.15.32:0/2596337356 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f8edc086a10 con 0x7f8ef40cf370 2024-04-17T10:01:21.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8ef8c00640 1 --2- 172.21.15.32:0/2596337356 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8ed0068380 0x7f8ed006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8efe9cf000 1 Processor -- start 2024-04-17T10:01:21.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8efe9cf000 1 -- start start 2024-04-17T10:01:21.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8ef8c00640 1 --2- 172.21.15.32:0/2596337356 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8ed0068380 0x7f8ed006a840 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8ee40af000 tx=0x7f8ee40a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:21.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8efe9cf000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f0112c260 0x564f0112c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8efe9cf000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f0112cb80 0x564f010df460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8efe9cf000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f010df9a0 0x564f010dfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8efe9cf000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564f00f314f0 con 0x564f0112cb80 2024-04-17T10:01:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8efe9cf000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564f00eea6c0 con 0x564f010df9a0 2024-04-17T10:01:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8efe9cf000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x564f00f2d4b0 con 0x564f0112c260 2024-04-17T10:01:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8ef8c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f0112c260 0x564f0112c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8eebe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f0112cb80 0x564f010df460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8ef8c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f0112c260 0x564f0112c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:35686/0 (socket says 172.21.15.32:35686) 2024-04-17T10:01:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8ef8c00640 1 -- 172.21.15.32:0/620389974 learned_addr learned my addr 172.21.15.32:0/620389974 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:21.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8ef9600640 1 --2- 172.21.15.32:0/620389974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f010df9a0 0x564f010dfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8eebe00640 1 -- 172.21.15.32:0/620389974 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f0112c260 msgr2=0x564f0112c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:21.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8eebe00640 1 --2- 172.21.15.32:0/620389974 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f0112c260 0x564f0112c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8eebe00640 1 -- 172.21.15.32:0/620389974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f010df9a0 msgr2=0x564f010dfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:21.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8eebe00640 1 --2- 172.21.15.32:0/620389974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f010df9a0 0x564f010dfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8eebe00640 1 -- 172.21.15.32:0/620389974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f00f768e0 con 0x564f0112cb80 2024-04-17T10:01:21.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8eebe00640 1 --2- 172.21.15.32:0/620389974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f0112cb80 0x564f010df460 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f8edc084e40 tx=0x7f8edc056940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f404c6e1fb2aefc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:21.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8ee9600640 1 -- 172.21.15.32:0/620389974 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc021070 con 0x564f0112cb80 2024-04-17T10:01:21.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8ee9600640 1 -- 172.21.15.32:0/620389974 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8edc05d0c0 con 0x564f0112cb80 2024-04-17T10:01:21.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8ee9600640 1 -- 172.21.15.32:0/620389974 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc052ef0 con 0x564f0112cb80 2024-04-17T10:01:21.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8efe9cf000 1 -- 172.21.15.32:0/620389974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f0112cb80 msgr2=0x564f010df460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:21.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8efe9cf000 1 --2- 172.21.15.32:0/620389974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f0112cb80 0x564f010df460 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f8edc084e40 tx=0x7f8edc056940 comp rx=0 tx=0).stop 2024-04-17T10:01:21.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8efe9cf000 1 -- 172.21.15.32:0/620389974 shutdown_connections 2024-04-17T10:01:21.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8efe9cf000 1 --2- 172.21.15.32:0/620389974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f010df9a0 0x564f010dfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8efe9cf000 1 --2- 172.21.15.32:0/620389974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f0112cb80 0x564f010df460 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8efe9cf000 1 --2- 172.21.15.32:0/620389974 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f0112c260 0x564f0112c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8efe9cf000 1 -- 172.21.15.32:0/620389974 >> 172.21.15.32:0/620389974 conn(0x564f0110d000 msgr2=0x564f0110f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:21.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8efe9cf000 1 -- 172.21.15.32:0/620389974 shutdown_connections 2024-04-17T10:01:21.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.653+0000 7f8efe9cf000 1 -- 172.21.15.32:0/620389974 wait complete. 2024-04-17T10:01:21.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8efe9cf000 1 -- start start 2024-04-17T10:01:21.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8efe9cf000 1 Processor -- start 2024-04-17T10:01:21.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8efe9cf000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f010df9a0 0x564f010e5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8efe9cf000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f0112c260 0x564f010e5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8efe9cf000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f0112cb80 0x564f010e64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8efe9cf000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564f00fb27b0 con 0x564f0112cb80 2024-04-17T10:01:21.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8efe9cf000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564f00fecd00 con 0x564f010df9a0 2024-04-17T10:01:21.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8efe9cf000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x564f00ef62c0 con 0x564f0112c260 2024-04-17T10:01:21.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8efe9cf000 10 client.0 _ll_register_callbacks cb 0x564f00f06030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:01:21.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8ef9600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f0112cb80 0x564f010e64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8ef9600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f0112cb80 0x564f010e64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53222/0 (socket says 172.21.15.32:53222) 2024-04-17T10:01:21.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8ef9600640 1 -- 172.21.15.32:0/2937691276 learned_addr learned my addr 172.21.15.32:0/2937691276 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:21.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8eebe00640 1 --2- 172.21.15.32:0/2937691276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f0112c260 0x564f010e5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8ef8c00640 1 --2- 172.21.15.32:0/2937691276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f010df9a0 0x564f010e5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8ef9600640 1 -- 172.21.15.32:0/2937691276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f0112c260 msgr2=0x564f010e5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8ef9600640 1 --2- 172.21.15.32:0/2937691276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f0112c260 0x564f010e5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8ef9600640 1 -- 172.21.15.32:0/2937691276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f010df9a0 msgr2=0x564f010e5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8ef9600640 1 --2- 172.21.15.32:0/2937691276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f010df9a0 0x564f010e5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8ef9600640 1 -- 172.21.15.32:0/2937691276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f00f09490 con 0x564f0112cb80 2024-04-17T10:01:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8eebe00640 1 --2- 172.21.15.32:0/2937691276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f0112c260 0x564f010e5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8ef8c00640 1 --2- 172.21.15.32:0/2937691276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f010df9a0 0x564f010e5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8ef9600640 1 --2- 172.21.15.32:0/2937691276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f0112cb80 0x564f010e64d0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f8eec012630 tx=0x7f8eec012af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90eeab8406c4805d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8ee2000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:01:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8ee2000640 1 -- 172.21.15.32:0/2937691276 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec019020 con 0x564f0112cb80 2024-04-17T10:01:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8ee2000640 1 -- 172.21.15.32:0/2937691276 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f8eec003b90 con 0x564f0112cb80 2024-04-17T10:01:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8efe9cf000 1 -- 172.21.15.32:0/2937691276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564f00ecadd0 con 0x564f0112cb80 2024-04-17T10:01:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8ee2000640 1 -- 172.21.15.32:0/2937691276 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f8eec023050 con 0x564f0112cb80 2024-04-17T10:01:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8efe9cf000 10 client.8337 fetch_fsmap learned FSMap version 72 2024-04-17T10:01:21.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8efe9cf000 1 -- 172.21.15.32:0/2937691276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x564f00efceb0 con 0x564f0112cb80 2024-04-17T10:01:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8ee2000640 1 -- 172.21.15.32:0/2937691276 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564f00ecadd0 con 0x564f0112cb80 2024-04-17T10:01:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8ee2000640 1 -- 172.21.15.32:0/2937691276 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 72) ==== 3495+0+0 (secure 0 0 0) 0x7f8eec026020 con 0x564f0112cb80 2024-04-17T10:01:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8efe9cf000 10 client.8337 fetch_fsmap finished waiting for FSMap version 72 2024-04-17T10:01:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8efe9cf000 10 client.8337 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T10:01:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8efe9cf000 20 client.8337 populate_metadata read hostname 'smithi032' 2024-04-17T10:01:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8efe9cf000 1 --2- 172.21.15.32:0/2937691276 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x564f010e6ee0 0x564f0113e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8efe9cf000 4 client.8337 mds_command: new command op to 8196 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:01:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8efe9cf000 1 -- 172.21.15.32:0/2937691276 --> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f00f3ef10 con 0x564f010e6ee0 2024-04-17T10:01:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.657+0000 7f8ef8c00640 1 --2- 172.21.15.32:0/2937691276 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x564f010e6ee0 0x564f0113e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.661+0000 7f8ef8c00640 1 --2- 172.21.15.32:0/2937691276 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x564f010e6ee0 0x564f0113e520 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68e6415e2603b304 server_cookie=4c5010ed07cf8062 in_seq=0 out_seq=0 2024-04-17T10:01:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.661+0000 7f8ee2000640 10 client.8337 ms_handle_connect on v2:172.21.15.32:6835/2058904361 2024-04-17T10:01:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.661+0000 7f8ee2000640 1 -- 172.21.15.32:0/2937691276 <== mds.0 v2:172.21.15.32:6835/2058904361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f00f3ef10 con 0x564f010e6ee0 2024-04-17T10:01:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.661+0000 7f8ee2000640 10 client.8337 handle_command_reply: tid=0 2024-04-17T10:01:21.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.661+0000 7f8efe9cf000 1 client.8337 shutdown 2024-04-17T10:01:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.661+0000 7f8efe9cf000 1 -- 172.21.15.32:0/2937691276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f0112cb80 msgr2=0x564f010e64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.661+0000 7f8efe9cf000 1 --2- 172.21.15.32:0/2937691276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f0112cb80 0x564f010e64d0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f8eec012630 tx=0x7f8eec012af0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.661+0000 7f8efe9cf000 1 -- 172.21.15.32:0/2937691276 shutdown_connections 2024-04-17T10:01:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.661+0000 7f8efe9cf000 1 --2- 172.21.15.32:0/2937691276 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x564f010e6ee0 0x564f0113e520 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.661+0000 7f8efe9cf000 1 --2- 172.21.15.32:0/2937691276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f0112c260 0x564f010e5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.661+0000 7f8efe9cf000 1 --2- 172.21.15.32:0/2937691276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f0112cb80 0x564f010e64d0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.661+0000 7f8ee2000640 0 client.8337 ms_handle_reset on v2:172.21.15.32:6835/2058904361 2024-04-17T10:01:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.661+0000 7f8efe9cf000 1 --2- 172.21.15.32:0/2937691276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f010df9a0 0x564f010e5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.661+0000 7f8efe9cf000 1 -- 172.21.15.32:0/2937691276 >> 172.21.15.32:0/2937691276 conn(0x564f0110d000 msgr2=0x564f010deb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.661+0000 7f8efe9cf000 1 -- 172.21.15.32:0/2937691276 shutdown_connections 2024-04-17T10:01:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.661+0000 7f8efe9cf000 1 -- 172.21.15.32:0/2937691276 wait complete. 2024-04-17T10:01:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.661+0000 7f8efe9cf000 20 client.8337 trim_cache size 0 max 16384 2024-04-17T10:01:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.677+0000 7f8efe9cf000 1 Processor -- start 2024-04-17T10:01:21.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.677+0000 7f8efe9cf000 1 -- start start 2024-04-17T10:01:21.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f010df9a0 0x564f010debd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f0112c260 0x564f010df110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0112cb80 0x564f0113e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef42032d0 con 0x564f010df9a0 2024-04-17T10:01:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef413e7b0 con 0x564f0112cb80 2024-04-17T10:01:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef40b1c00 con 0x564f0112c260 2024-04-17T10:01:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8ef8c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f010df9a0 0x564f010debd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8ef8c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f010df9a0 0x564f010debd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53236/0 (socket says 172.21.15.32:53236) 2024-04-17T10:01:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8ef8c00640 1 -- 172.21.15.32:0/2290668635 learned_addr learned my addr 172.21.15.32:0/2290668635 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8eebe00640 1 --2- 172.21.15.32:0/2290668635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f0112c260 0x564f010df110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8ef9600640 1 --2- 172.21.15.32:0/2290668635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0112cb80 0x564f0113e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8ef8c00640 1 -- 172.21.15.32:0/2290668635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f0112c260 msgr2=0x564f010df110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8ef8c00640 1 --2- 172.21.15.32:0/2290668635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f0112c260 0x564f010df110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8ef8c00640 1 -- 172.21.15.32:0/2290668635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0112cb80 msgr2=0x564f0113e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:21.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8ef8c00640 1 --2- 172.21.15.32:0/2290668635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0112cb80 0x564f0113e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8ef8c00640 1 -- 172.21.15.32:0/2290668635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f00eeba20 con 0x564f010df9a0 2024-04-17T10:01:21.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8ef9600640 1 --2- 172.21.15.32:0/2290668635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0112cb80 0x564f0113e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:21.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8ef8c00640 1 --2- 172.21.15.32:0/2290668635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f010df9a0 0x564f010debd0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f8ee401ac10 tx=0x7f8ee4009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4a2a4d0d15fde0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:21.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8ee8c00640 1 -- 172.21.15.32:0/2290668635 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee40d2070 con 0x564f010df9a0 2024-04-17T10:01:21.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8ee8c00640 1 -- 172.21.15.32:0/2290668635 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ee40b0040 con 0x564f010df9a0 2024-04-17T10:01:21.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8ee8c00640 1 -- 172.21.15.32:0/2290668635 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee40cc850 con 0x564f010df9a0 2024-04-17T10:01:21.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 1 -- 172.21.15.32:0/2290668635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f010df9a0 msgr2=0x564f010debd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:21.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 1 --2- 172.21.15.32:0/2290668635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f010df9a0 0x564f010debd0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f8ee401ac10 tx=0x7f8ee4009240 comp rx=0 tx=0).stop 2024-04-17T10:01:21.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 1 -- 172.21.15.32:0/2290668635 shutdown_connections 2024-04-17T10:01:21.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 1 --2- 172.21.15.32:0/2290668635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f0112c260 0x564f010df110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 1 --2- 172.21.15.32:0/2290668635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0112cb80 0x564f0113e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 1 --2- 172.21.15.32:0/2290668635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f010df9a0 0x564f010debd0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 1 -- 172.21.15.32:0/2290668635 >> 172.21.15.32:0/2290668635 conn(0x564f0110d000 msgr2=0x564f0112a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:21.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 1 -- 172.21.15.32:0/2290668635 shutdown_connections 2024-04-17T10:01:21.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 1 -- 172.21.15.32:0/2290668635 wait complete. 2024-04-17T10:01:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 1 -- start start 2024-04-17T10:01:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 1 Processor -- start 2024-04-17T10:01:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f010df9a0 0x564f011415a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0112c260 0x564f011580b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f0112cb80 0x564f011585f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564f00ed5050 con 0x564f0112cb80 2024-04-17T10:01:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564f00ef3570 con 0x564f0112c260 2024-04-17T10:01:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x564f00f30800 con 0x564f010df9a0 2024-04-17T10:01:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.681+0000 7f8efe9cf000 10 client.0 _ll_register_callbacks cb 0x7f8eec0122c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:01:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8ef9600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f0112cb80 0x564f011585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8ef8c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f010df9a0 0x564f011415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8ef9600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f0112cb80 0x564f011585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53242/0 (socket says 172.21.15.32:53242) 2024-04-17T10:01:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8ef9600640 1 -- 172.21.15.32:0/3703155176 learned_addr learned my addr 172.21.15.32:0/3703155176 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:21.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8eebe00640 1 --2- 172.21.15.32:0/3703155176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0112c260 0x564f011580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8ef9600640 1 -- 172.21.15.32:0/3703155176 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f010df9a0 msgr2=0x564f011415a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8ef9600640 1 --2- 172.21.15.32:0/3703155176 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f010df9a0 0x564f011415a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8ef9600640 1 -- 172.21.15.32:0/3703155176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0112c260 msgr2=0x564f011580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8ef9600640 1 --2- 172.21.15.32:0/3703155176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0112c260 0x564f011580b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8ef9600640 1 -- 172.21.15.32:0/3703155176 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f00fb2c40 con 0x564f0112cb80 2024-04-17T10:01:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8ef9600640 1 --2- 172.21.15.32:0/3703155176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f0112cb80 0x564f011585f0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f8eec001b60 tx=0x7f8eec0014b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f705d47b63847b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8ee2000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:01:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8ee2000640 1 -- 172.21.15.32:0/3703155176 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec019050 con 0x564f0112cb80 2024-04-17T10:01:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8ee2000640 1 -- 172.21.15.32:0/3703155176 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f8eec010540 con 0x564f0112cb80 2024-04-17T10:01:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8efe9cf000 1 -- 172.21.15.32:0/3703155176 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ee40b0040 con 0x564f0112cb80 2024-04-17T10:01:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8ee2000640 1 -- 172.21.15.32:0/3703155176 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f8eec034050 con 0x564f0112cb80 2024-04-17T10:01:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8efe9cf000 10 client.8343 fetch_fsmap learned FSMap version 72 2024-04-17T10:01:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8efe9cf000 1 -- 172.21.15.32:0/3703155176 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x564f00ed6600 con 0x564f0112cb80 2024-04-17T10:01:21.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8ee2000640 1 -- 172.21.15.32:0/3703155176 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ee40b0040 con 0x564f0112cb80 2024-04-17T10:01:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8ee2000640 1 -- 172.21.15.32:0/3703155176 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 72) ==== 3495+0+0 (secure 0 0 0) 0x7f8eec037020 con 0x564f0112cb80 2024-04-17T10:01:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8efe9cf000 10 client.8343 fetch_fsmap finished waiting for FSMap version 72 2024-04-17T10:01:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8efe9cf000 10 client.8343 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-17T10:01:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8efe9cf000 20 client.8343 populate_metadata read hostname 'smithi032' 2024-04-17T10:01:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8efe9cf000 1 --2- 172.21.15.32:0/3703155176 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x564f0119bf30 0x564f0119c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8efe9cf000 4 client.8343 mds_command: new command op to 8196 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:01:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8efe9cf000 1 -- 172.21.15.32:0/3703155176 --> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564f00f38d80 con 0x564f0119bf30 2024-04-17T10:01:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8ef8c00640 1 --2- 172.21.15.32:0/3703155176 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x564f0119bf30 0x564f0119c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8ef8c00640 1 --2- 172.21.15.32:0/3703155176 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x564f0119bf30 0x564f0119c340 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af89ad31316ee469 server_cookie=afc17cca50cc7382 in_seq=0 out_seq=0 2024-04-17T10:01:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8ee2000640 10 client.8343 ms_handle_connect on v2:172.21.15.32:6835/2058904361 2024-04-17T10:01:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8ee2000640 1 -- 172.21.15.32:0/3703155176 <== mds.0 v2:172.21.15.32:6835/2058904361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564f00f38d80 con 0x564f0119bf30 2024-04-17T10:01:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8ee2000640 10 client.8343 handle_command_reply: tid=0 2024-04-17T10:01:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8efe9cf000 1 client.8343 shutdown 2024-04-17T10:01:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8efe9cf000 1 -- 172.21.15.32:0/3703155176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f0112cb80 msgr2=0x564f011585f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8efe9cf000 1 --2- 172.21.15.32:0/3703155176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f0112cb80 0x564f011585f0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f8eec001b60 tx=0x7f8eec0014b0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8efe9cf000 1 -- 172.21.15.32:0/3703155176 shutdown_connections 2024-04-17T10:01:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8efe9cf000 1 --2- 172.21.15.32:0/3703155176 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x564f0119bf30 0x564f0119c340 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8efe9cf000 1 --2- 172.21.15.32:0/3703155176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f0112cb80 0x564f011585f0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8efe9cf000 1 --2- 172.21.15.32:0/3703155176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564f0112c260 0x564f011580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8efe9cf000 1 --2- 172.21.15.32:0/3703155176 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564f010df9a0 0x564f011415a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8ee2000640 0 client.8343 ms_handle_reset on v2:172.21.15.32:6835/2058904361 2024-04-17T10:01:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8efe9cf000 1 -- 172.21.15.32:0/3703155176 >> 172.21.15.32:0/3703155176 conn(0x564f0110d000 msgr2=0x564f0112aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8efe9cf000 1 -- 172.21.15.32:0/3703155176 shutdown_connections 2024-04-17T10:01:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8efe9cf000 1 -- 172.21.15.32:0/3703155176 wait complete. 2024-04-17T10:01:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8efe9cf000 20 client.8343 trim_cache size 0 max 16384 2024-04-17T10:01:21.688 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:01:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-17T10:01:21.688 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-17T10:01:21.688 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:01:21.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.685+0000 7f8efb600640 1 -- 172.21.15.32:0/2596337356 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8ed0068380 msgr2=0x7f8ed006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:21.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.689+0000 7f8efb600640 1 --2- 172.21.15.32:0/2596337356 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8ed0068380 0x7f8ed006a840 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8ee40af000 tx=0x7f8ee40a7000 comp rx=0 tx=0).stop 2024-04-17T10:01:21.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.689+0000 7f8efb600640 1 -- 172.21.15.32:0/2596337356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ef40cf370 msgr2=0x7f8ef41fae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:21.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.689+0000 7f8efb600640 1 --2- 172.21.15.32:0/2596337356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ef40cf370 0x7f8ef41fae20 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00e960 tx=0x7f8edc00ee20 comp rx=0 tx=0).stop 2024-04-17T10:01:21.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.689+0000 7f8efb600640 1 -- 172.21.15.32:0/2596337356 shutdown_connections 2024-04-17T10:01:21.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.689+0000 7f8efb600640 1 --2- 172.21.15.32:0/2596337356 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8ed0068380 0x7f8ed006a840 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.689+0000 7f8efb600640 1 --2- 172.21.15.32:0/2596337356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ef4160350 0x7f8ef4166230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.689+0000 7f8efb600640 1 --2- 172.21.15.32:0/2596337356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ef40cf370 0x7f8ef41fae20 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.689+0000 7f8efb600640 1 --2- 172.21.15.32:0/2596337356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ef406ba30 0x7f8ef415e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.689+0000 7f8efb600640 1 -- 172.21.15.32:0/2596337356 >> 172.21.15.32:0/2596337356 conn(0x7f8ef40b2120 msgr2=0x7f8ef40ce0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:21.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.689+0000 7f8efb600640 1 -- 172.21.15.32:0/2596337356 shutdown_connections 2024-04-17T10:01:21.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.689+0000 7f8efb600640 1 -- 172.21.15.32:0/2596337356 wait complete. 2024-04-17T10:01:21.703 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T10:01:21.704 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:01:21.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.849+0000 7f1259400640 1 Processor -- start 2024-04-17T10:01:21.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.849+0000 7f1259400640 1 -- start start 2024-04-17T10:01:21.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.849+0000 7f1259400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12540c7ce0 0x7f12540cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.849+0000 7f1259400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12540b0d00 con 0x7f125415c760 2024-04-17T10:01:21.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.849+0000 7f1259400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12540b0fe0 con 0x7f12540b3980 2024-04-17T10:01:21.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.849+0000 7f1259400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f12540b1bf0 con 0x7f12540c7ce0 2024-04-17T10:01:21.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.849+0000 7f1253e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f125415c760 0x7f12540c75d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50348/0 (socket says 172.21.15.32:50348) 2024-04-17T10:01:21.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.849+0000 7f1253e00640 1 -- 172.21.15.32:0/612353244 learned_addr learned my addr 172.21.15.32:0/612353244 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:21.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.849+0000 7f1258a00640 1 --2- 172.21.15.32:0/612353244 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12540c7ce0 0x7f12540cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.849+0000 7f1252a00640 1 -- 172.21.15.32:0/612353244 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1744592979 0 0) 0x7f12540b0d00 con 0x7f125415c760 2024-04-17T10:01:21.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.849+0000 7f1252a00640 1 -- 172.21.15.32:0/612353244 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1238003530 con 0x7f125415c760 2024-04-17T10:01:21.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.849+0000 7f1252a00640 1 -- 172.21.15.32:0/612353244 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 963429121 0 0) 0x7f12540b0fe0 con 0x7f12540b3980 2024-04-17T10:01:21.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.849+0000 7f1258a00640 1 -- 172.21.15.32:0/612353244 >> v1:172.21.15.144:6789/0 conn(0x7f12540b3980 legacy=0x7f12540cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:21.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.849+0000 7f1258a00640 1 -- 172.21.15.32:0/612353244 >> v1:172.21.15.32:6789/0 conn(0x7f125415c760 legacy=0x7f12540c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:21.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.849+0000 7f1258a00640 1 -- 172.21.15.32:0/612353244 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12540b12c0 con 0x7f12540c7ce0 2024-04-17T10:01:21.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.853+0000 7f1258a00640 1 --2- 172.21.15.32:0/612353244 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12540c7ce0 0x7f12540cb890 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f12540b1d70 tx=0x7f124c01c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46a63030f6b4503b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:21.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.853+0000 7f1252a00640 1 -- 172.21.15.32:0/612353244 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c0b2070 con 0x7f12540c7ce0 2024-04-17T10:01:21.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.853+0000 7f1252a00640 1 -- 172.21.15.32:0/612353244 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f124c0a55c0 con 0x7f12540c7ce0 2024-04-17T10:01:21.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.853+0000 7f1252a00640 1 -- 172.21.15.32:0/612353244 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244001680 con 0x7f12540c7ce0 2024-04-17T10:01:21.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.853+0000 7f1259400640 1 -- 172.21.15.32:0/612353244 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12540c7ce0 msgr2=0x7f12540cb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:21.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.853+0000 7f1259400640 1 --2- 172.21.15.32:0/612353244 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12540c7ce0 0x7f12540cb890 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f12540b1d70 tx=0x7f124c01c740 comp rx=0 tx=0).stop 2024-04-17T10:01:21.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.853+0000 7f1259400640 1 -- 172.21.15.32:0/612353244 shutdown_connections 2024-04-17T10:01:21.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.853+0000 7f1259400640 1 --2- 172.21.15.32:0/612353244 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12540c7ce0 0x7f12540cb890 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.853+0000 7f1259400640 1 -- 172.21.15.32:0/612353244 >> 172.21.15.32:0/612353244 conn(0x7f12540b2e90 msgr2=0x7f12540b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:21.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.853+0000 7f1259400640 1 -- 172.21.15.32:0/612353244 shutdown_connections 2024-04-17T10:01:21.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.853+0000 7f1259400640 1 -- 172.21.15.32:0/612353244 wait complete. 2024-04-17T10:01:21.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.853+0000 7f1259400640 1 Processor -- start 2024-04-17T10:01:21.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f1259400640 1 -- start start 2024-04-17T10:01:21.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f1259400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12540b3980 0x7f125406f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f1259400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12540c7ce0 0x7f125406f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f1259400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f125415c760 0x7f12540c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f1259400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f12540cb650 con 0x7f12540b3980 2024-04-17T10:01:21.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f1259400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f12540cb4d0 con 0x7f12540c7ce0 2024-04-17T10:01:21.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f1259400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f12540ce920 con 0x7f125415c760 2024-04-17T10:01:21.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f1253e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12540b3980 0x7f125406f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f1253e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12540b3980 0x7f125406f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53254/0 (socket says 172.21.15.32:53254) 2024-04-17T10:01:21.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f1253e00640 1 -- 172.21.15.32:0/1398697527 learned_addr learned my addr 172.21.15.32:0/1398697527 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:21.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f1258a00640 1 --2- 172.21.15.32:0/1398697527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f125415c760 0x7f12540c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f1253400640 1 --2- 172.21.15.32:0/1398697527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12540c7ce0 0x7f125406f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f1253e00640 1 -- 172.21.15.32:0/1398697527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f125415c760 msgr2=0x7f12540c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f1253e00640 1 --2- 172.21.15.32:0/1398697527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f125415c760 0x7f12540c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f1253e00640 1 -- 172.21.15.32:0/1398697527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12540c7ce0 msgr2=0x7f125406f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f1253e00640 1 --2- 172.21.15.32:0/1398697527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12540c7ce0 0x7f125406f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f1253e00640 1 -- 172.21.15.32:0/1398697527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12540b4150 con 0x7f12540b3980 2024-04-17T10:01:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f1253e00640 1 --2- 172.21.15.32:0/1398697527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12540b3980 0x7f125406f260 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f124400c790 tx=0x7f124400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f124be00640 1 -- 172.21.15.32:0/1398697527 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244018070 con 0x7f12540b3980 2024-04-17T10:01:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f124be00640 1 -- 172.21.15.32:0/1398697527 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12440074e0 con 0x7f12540b3980 2024-04-17T10:01:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f1259400640 1 -- 172.21.15.32:0/1398697527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12540c9a10 con 0x7f12540b3980 2024-04-17T10:01:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f124be00640 1 -- 172.21.15.32:0/1398697527 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244004a10 con 0x7f12540b3980 2024-04-17T10:01:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f1259400640 1 -- 172.21.15.32:0/1398697527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12540c9fd0 con 0x7f12540b3980 2024-04-17T10:01:21.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.857+0000 7f1259400640 1 -- 172.21.15.32:0/1398697527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f125404fd00 con 0x7f12540b3980 2024-04-17T10:01:21.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.861+0000 7f124be00640 1 -- 172.21.15.32:0/1398697527 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f124401b070 con 0x7f12540b3980 2024-04-17T10:01:21.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.861+0000 7f124be00640 1 --2- 172.21.15.32:0/1398697527 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1234068380 0x7f123406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:21.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.861+0000 7f124be00640 1 -- 172.21.15.32:0/1398697527 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f1244086080 con 0x7f12540b3980 2024-04-17T10:01:21.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.861+0000 7f1253400640 1 --2- 172.21.15.32:0/1398697527 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1234068380 0x7f123406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:21.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.861+0000 7f1253400640 1 --2- 172.21.15.32:0/1398697527 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1234068380 0x7f123406a840 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f12541508f0 tx=0x7f1240009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:21.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:21.869+0000 7f124be00640 1 -- 172.21.15.32:0/1398697527 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f1244055ef0 con 0x7f12540b3980 2024-04-17T10:01:22.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.069+0000 7f1259400640 1 -- 172.21.15.32:0/1398697527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f125406dc70 con 0x7f12540b3980 2024-04-17T10:01:22.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.069+0000 7f124be00640 1 -- 172.21.15.32:0/1398697527 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5205 (secure 0 0 0) 0x7f124405a1f0 con 0x7f12540b3980 2024-04-17T10:01:22.068 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:22.068 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":72,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6419,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3534108103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3534108103},{"type":"v1","addr":"172.21.15.144:6835","nonce":3534108103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7003,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/935945019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":935945019},{"type":"v1","addr":"172.21.15.144:6834","nonce":935945019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3044362393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3044362393},{"type":"v1","addr":"172.21.15.32:6836","nonce":3044362393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:59:04.834110+0000","modified":"2024-04-17T10:00:47.860710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8196":{"gid":8196,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6837/2058904361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2058904361},{"type":"v1","addr":"172.21.15.32:6837","nonce":2058904361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8196,"qdb_cluster":[8196]},"id":6}]} 2024-04-17T10:01:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 72 2024-04-17T10:01:22.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.073+0000 7f1259400640 1 -- 172.21.15.32:0/1398697527 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1234068380 msgr2=0x7f123406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:22.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.073+0000 7f1259400640 1 --2- 172.21.15.32:0/1398697527 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1234068380 0x7f123406a840 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f12541508f0 tx=0x7f1240009040 comp rx=0 tx=0).stop 2024-04-17T10:01:22.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.073+0000 7f1259400640 1 -- 172.21.15.32:0/1398697527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12540b3980 msgr2=0x7f125406f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.073+0000 7f1259400640 1 --2- 172.21.15.32:0/1398697527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12540b3980 0x7f125406f260 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f124400c790 tx=0x7f124400cc50 comp rx=0 tx=0).stop 2024-04-17T10:01:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.073+0000 7f1259400640 1 -- 172.21.15.32:0/1398697527 shutdown_connections 2024-04-17T10:01:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.073+0000 7f1259400640 1 --2- 172.21.15.32:0/1398697527 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1234068380 0x7f123406a840 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.073+0000 7f1259400640 1 --2- 172.21.15.32:0/1398697527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f125415c760 0x7f12540c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.073+0000 7f1259400640 1 --2- 172.21.15.32:0/1398697527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12540c7ce0 0x7f125406f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.073+0000 7f1259400640 1 --2- 172.21.15.32:0/1398697527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12540b3980 0x7f125406f260 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.073+0000 7f1259400640 1 -- 172.21.15.32:0/1398697527 >> 172.21.15.32:0/1398697527 conn(0x7f12540b2e90 msgr2=0x7f12541525a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.073+0000 7f1259400640 1 -- 172.21.15.32:0/1398697527 shutdown_connections 2024-04-17T10:01:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.073+0000 7f1259400640 1 -- 172.21.15.32:0/1398697527 wait complete. 2024-04-17T10:01:22.091 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-17T10:01:22.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.237+0000 7f01ab400640 1 Processor -- start 2024-04-17T10:01:22.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.237+0000 7f01ab400640 1 -- start start 2024-04-17T10:01:22.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.237+0000 7f01ab400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01a415b7d0 0x7f01a406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.237+0000 7f01ab400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01a406f7c0 0x7f01a406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.237+0000 7f01ab400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01a40cc940 con 0x7f01a406f270 2024-04-17T10:01:22.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.237+0000 7f01ab400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f01a40ccc20 con 0x7f01a406f7c0 2024-04-17T10:01:22.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.237+0000 7f01ab400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f01a406e0b0 con 0x7f01a415b7d0 2024-04-17T10:01:22.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.237+0000 7f019be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01a406f7c0 0x7f01a406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.237+0000 7f01a9400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f01a406f270 0x7f01a40ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50358/0 (socket says 172.21.15.32:50358) 2024-04-17T10:01:22.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.237+0000 7f019be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01a406f7c0 0x7f01a406ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53258/0 (socket says 172.21.15.32:53258) 2024-04-17T10:01:22.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.237+0000 7f019be00640 1 -- 172.21.15.32:0/218387919 learned_addr learned my addr 172.21.15.32:0/218387919 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:22.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.237+0000 7f01a8a00640 1 --2- 172.21.15.32:0/218387919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01a415b7d0 0x7f01a406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.237+0000 7f019be00640 1 -- 172.21.15.32:0/218387919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01a415b7d0 msgr2=0x7f01a406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:22.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.237+0000 7f019be00640 1 --2- 172.21.15.32:0/218387919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01a415b7d0 0x7f01a406e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.237+0000 7f019be00640 1 -- 172.21.15.32:0/218387919 >> v1:172.21.15.32:6789/0 conn(0x7f01a406f270 legacy=0x7f01a40ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:22.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.237+0000 7f019be00640 1 -- 172.21.15.32:0/218387919 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a406e390 con 0x7f01a406f7c0 2024-04-17T10:01:22.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.237+0000 7f01a8a00640 1 --2- 172.21.15.32:0/218387919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01a415b7d0 0x7f01a406e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:22.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.241+0000 7f019be00640 1 --2- 172.21.15.32:0/218387919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a406f7c0 0x7f01a406ba30 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f018c009870 tx=0x7f018c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e37d47d2de1c08e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:22.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.241+0000 7f01a3e00640 1 -- 172.21.15.32:0/218387919 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c0aa650 con 0x7f01a406f7c0 2024-04-17T10:01:22.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.241+0000 7f01a3e00640 1 -- 172.21.15.32:0/218387919 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f018c0af070 con 0x7f01a406f7c0 2024-04-17T10:01:22.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.241+0000 7f01a3e00640 1 -- 172.21.15.32:0/218387919 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c0aaa10 con 0x7f01a406f7c0 2024-04-17T10:01:22.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.241+0000 7f01ab400640 1 -- 172.21.15.32:0/218387919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a406f7c0 msgr2=0x7f01a406ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:22.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.241+0000 7f01ab400640 1 --2- 172.21.15.32:0/218387919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a406f7c0 0x7f01a406ba30 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f018c009870 tx=0x7f018c01c4d0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.241+0000 7f01ab400640 1 -- 172.21.15.32:0/218387919 shutdown_connections 2024-04-17T10:01:22.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.241+0000 7f01ab400640 1 --2- 172.21.15.32:0/218387919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a406f7c0 0x7f01a406ba30 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.241+0000 7f01ab400640 1 --2- 172.21.15.32:0/218387919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01a415b7d0 0x7f01a406e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.241+0000 7f01ab400640 1 -- 172.21.15.32:0/218387919 >> 172.21.15.32:0/218387919 conn(0x7f01a40b2120 msgr2=0x7f01a40b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:22.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.241+0000 7f01ab400640 1 -- 172.21.15.32:0/218387919 shutdown_connections 2024-04-17T10:01:22.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.241+0000 7f01ab400640 1 -- 172.21.15.32:0/218387919 wait complete. 2024-04-17T10:01:22.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f01ab400640 1 Processor -- start 2024-04-17T10:01:22.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f01ab400640 1 -- start start 2024-04-17T10:01:22.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f01ab400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01a406f270 0x7f01a40af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f01ab400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01a406f7c0 0x7f01a414ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f01ab400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a415b7d0 0x7f01a40b3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f01ab400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f01a4071e60 con 0x7f01a415b7d0 2024-04-17T10:01:22.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f01ab400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f01a406c380 con 0x7f01a406f270 2024-04-17T10:01:22.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f01ab400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f01a4071fe0 con 0x7f01a406f7c0 2024-04-17T10:01:22.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f01a9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a415b7d0 0x7f01a40b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f019be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01a406f7c0 0x7f01a414ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f01a9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a415b7d0 0x7f01a40b3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53274/0 (socket says 172.21.15.32:53274) 2024-04-17T10:01:22.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f01a9400640 1 -- 172.21.15.32:0/2885025308 learned_addr learned my addr 172.21.15.32:0/2885025308 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:22.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f01a8a00640 1 --2- 172.21.15.32:0/2885025308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01a406f270 0x7f01a40af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f01a9400640 1 -- 172.21.15.32:0/2885025308 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01a406f7c0 msgr2=0x7f01a414ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:22.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f01a9400640 1 --2- 172.21.15.32:0/2885025308 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01a406f7c0 0x7f01a414ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f01a9400640 1 -- 172.21.15.32:0/2885025308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01a406f270 msgr2=0x7f01a40af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:22.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f01a9400640 1 --2- 172.21.15.32:0/2885025308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01a406f270 0x7f01a40af0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f01a9400640 1 -- 172.21.15.32:0/2885025308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a4156860 con 0x7f01a415b7d0 2024-04-17T10:01:22.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f01a9400640 1 --2- 172.21.15.32:0/2885025308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a415b7d0 0x7f01a40b3210 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f019c00c7a0 tx=0x7f019c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:22.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f01a1600640 1 -- 172.21.15.32:0/2885025308 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c017070 con 0x7f01a415b7d0 2024-04-17T10:01:22.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f01a1600640 1 -- 172.21.15.32:0/2885025308 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f019c00a640 con 0x7f01a415b7d0 2024-04-17T10:01:22.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f01ab400640 1 -- 172.21.15.32:0/2885025308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a40ae4c0 con 0x7f01a415b7d0 2024-04-17T10:01:22.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f01a1600640 1 -- 172.21.15.32:0/2885025308 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c00a9f0 con 0x7f01a415b7d0 2024-04-17T10:01:22.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.245+0000 7f01ab400640 1 -- 172.21.15.32:0/2885025308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a40aea80 con 0x7f01a415b7d0 2024-04-17T10:01:22.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.249+0000 7f01a1600640 1 -- 172.21.15.32:0/2885025308 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f019c01a070 con 0x7f01a415b7d0 2024-04-17T10:01:22.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.249+0000 7f01a1600640 1 --2- 172.21.15.32:0/2885025308 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f01800683b0 0x7f018006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.249+0000 7f01a1600640 1 -- 172.21.15.32:0/2885025308 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f019c086070 con 0x7f01a415b7d0 2024-04-17T10:01:22.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.249+0000 7f01a8a00640 1 --2- 172.21.15.32:0/2885025308 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f01800683b0 0x7f018006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.249+0000 7f01ae7ba000 1 Processor -- start 2024-04-17T10:01:22.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.249+0000 7f01ae7ba000 1 -- start start 2024-04-17T10:01:22.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.249+0000 7f01a8a00640 1 --2- 172.21.15.32:0/2885025308 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f01800683b0 0x7f018006a870 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0194006fd0 tx=0x7f0194008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:22.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.249+0000 7f01ae7ba000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f18dfaba50 0x55f18dfabe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.249+0000 7f01ae7ba000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18dfac370 0x55f18df5ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.249+0000 7f01ae7ba000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f18df5ef70 0x55f18df5f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.249+0000 7f01ae7ba000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f18ddb0590 con 0x55f18dfac370 2024-04-17T10:01:22.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.249+0000 7f01ae7ba000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f18dd69580 con 0x55f18dfaba50 2024-04-17T10:01:22.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.249+0000 7f01ae7ba000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f18ddac550 con 0x55f18df5ef70 2024-04-17T10:01:22.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.249+0000 7f019be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18dfac370 0x55f18df5ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.249+0000 7f019be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18dfac370 0x55f18df5ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53288/0 (socket says 172.21.15.32:53288) 2024-04-17T10:01:22.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.249+0000 7f019be00640 1 -- 172.21.15.32:0/2215545858 learned_addr learned my addr 172.21.15.32:0/2215545858 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:22.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.249+0000 7f01a8a00640 1 --2- 172.21.15.32:0/2215545858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f18dfaba50 0x55f18dfabe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.249+0000 7f01a9400640 1 --2- 172.21.15.32:0/2215545858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f18df5ef70 0x55f18df5f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.249+0000 7f019be00640 1 -- 172.21.15.32:0/2215545858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f18df5ef70 msgr2=0x55f18df5f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:22.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.249+0000 7f019be00640 1 --2- 172.21.15.32:0/2215545858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f18df5ef70 0x55f18df5f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.249+0000 7f019be00640 1 -- 172.21.15.32:0/2215545858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f18dfaba50 msgr2=0x55f18dfabe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:22.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.249+0000 7f019be00640 1 --2- 172.21.15.32:0/2215545858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f18dfaba50 0x55f18dfabe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.249+0000 7f019be00640 1 -- 172.21.15.32:0/2215545858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f18de57b90 con 0x55f18dfac370 2024-04-17T10:01:22.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f019be00640 1 --2- 172.21.15.32:0/2215545858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18dfac370 0x55f18df5ea30 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f018c01ab80 tx=0x7f018c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2967f2c829493395 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f0199600640 1 -- 172.21.15.32:0/2215545858 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c0aab20 con 0x55f18dfac370 2024-04-17T10:01:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f0199600640 1 -- 172.21.15.32:0/2215545858 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f018c0af040 con 0x55f18dfac370 2024-04-17T10:01:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f0199600640 1 -- 172.21.15.32:0/2215545858 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c0bd3f0 con 0x55f18dfac370 2024-04-17T10:01:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2215545858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18dfac370 msgr2=0x55f18df5ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01ae7ba000 1 --2- 172.21.15.32:0/2215545858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18dfac370 0x55f18df5ea30 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f018c01ab80 tx=0x7f018c004740 comp rx=0 tx=0).stop 2024-04-17T10:01:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2215545858 shutdown_connections 2024-04-17T10:01:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01ae7ba000 1 --2- 172.21.15.32:0/2215545858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f18df5ef70 0x55f18df5f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01ae7ba000 1 --2- 172.21.15.32:0/2215545858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18dfac370 0x55f18df5ea30 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01ae7ba000 1 --2- 172.21.15.32:0/2215545858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f18dfaba50 0x55f18dfabe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2215545858 >> 172.21.15.32:0/2215545858 conn(0x55f18df8c7f0 msgr2=0x55f18df8ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2215545858 shutdown_connections 2024-04-17T10:01:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2215545858 wait complete. 2024-04-17T10:01:22.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01ae7ba000 1 -- start start 2024-04-17T10:01:22.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01ae7ba000 1 Processor -- start 2024-04-17T10:01:22.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01ae7ba000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f18df5ef70 0x55f18dfaadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01ae7ba000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f18dfaba50 0x55f18dfab330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01ae7ba000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18dfac370 0x55f18df633b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01ae7ba000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f18ddbb4c0 con 0x55f18dfac370 2024-04-17T10:01:22.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01ae7ba000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f18dd50c80 con 0x55f18df5ef70 2024-04-17T10:01:22.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01ae7ba000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f18dd7b930 con 0x55f18dfaba50 2024-04-17T10:01:22.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01ae7ba000 10 client.0 _ll_register_callbacks cb 0x55f18df4c6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:01:22.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01a9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18dfac370 0x55f18df633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01a9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18dfac370 0x55f18df633b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53296/0 (socket says 172.21.15.32:53296) 2024-04-17T10:01:22.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01a9400640 1 -- 172.21.15.32:0/2777741814 learned_addr learned my addr 172.21.15.32:0/2777741814 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:22.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f019be00640 1 --2- 172.21.15.32:0/2777741814 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f18dfaba50 0x55f18dfab330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01a8a00640 1 --2- 172.21.15.32:0/2777741814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f18df5ef70 0x55f18dfaadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01a9400640 1 -- 172.21.15.32:0/2777741814 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f18dfaba50 msgr2=0x55f18dfab330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01a9400640 1 --2- 172.21.15.32:0/2777741814 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f18dfaba50 0x55f18dfab330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01a9400640 1 -- 172.21.15.32:0/2777741814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f18df5ef70 msgr2=0x55f18dfaadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01a9400640 1 --2- 172.21.15.32:0/2777741814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f18df5ef70 0x55f18dfaadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01a9400640 1 -- 172.21.15.32:0/2777741814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f18dd89b20 con 0x55f18dfac370 2024-04-17T10:01:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f01a9400640 1 --2- 172.21.15.32:0/2777741814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18dfac370 0x55f18df633b0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f019c005fd0 tx=0x7f019c05a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adeef6610f299aa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.253+0000 7f0192000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:01:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f0192000640 1 -- 172.21.15.32:0/2777741814 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c06e020 con 0x55f18dfac370 2024-04-17T10:01:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f0192000640 1 -- 172.21.15.32:0/2777741814 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f019c05e4f0 con 0x55f18dfac370 2024-04-17T10:01:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2777741814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f018c0af040 con 0x55f18dfac370 2024-04-17T10:01:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f0192000640 1 -- 172.21.15.32:0/2777741814 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f019c059070 con 0x55f18dfac370 2024-04-17T10:01:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f01ae7ba000 10 client.8364 fetch_fsmap learned FSMap version 72 2024-04-17T10:01:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2777741814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55f18ddf4610 con 0x55f18dfac370 2024-04-17T10:01:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f0192000640 1 -- 172.21.15.32:0/2777741814 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f018c0af040 con 0x55f18dfac370 2024-04-17T10:01:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f0192000640 1 -- 172.21.15.32:0/2777741814 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 72) ==== 3495+0+0 (secure 0 0 0) 0x7f019c00ab90 con 0x55f18dfac370 2024-04-17T10:01:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f01ae7ba000 10 client.8364 fetch_fsmap finished waiting for FSMap version 72 2024-04-17T10:01:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f01ae7ba000 10 client.8364 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T10:01:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f01ae7ba000 20 client.8364 populate_metadata read hostname 'smithi032' 2024-04-17T10:01:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f01ae7ba000 1 --2- 172.21.15.32:0/2777741814 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55f18df63dc0 0x55f18dfbcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f01ae7ba000 4 client.8364 mds_command: new command op to 8196 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:01:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2777741814 --> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f18dd49e40 con 0x55f18df63dc0 2024-04-17T10:01:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f01a8a00640 1 --2- 172.21.15.32:0/2777741814 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55f18df63dc0 0x55f18dfbcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f01a8a00640 1 --2- 172.21.15.32:0/2777741814 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55f18df63dc0 0x55f18dfbcd00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b429af5968e21bb server_cookie=a7485a97e1486f12 in_seq=0 out_seq=0 2024-04-17T10:01:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f0192000640 10 client.8364 ms_handle_connect on v2:172.21.15.32:6835/2058904361 2024-04-17T10:01:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f0192000640 1 -- 172.21.15.32:0/2777741814 <== mds.0 v2:172.21.15.32:6835/2058904361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f18dd49e40 con 0x55f18df63dc0 2024-04-17T10:01:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f0192000640 10 client.8364 handle_command_reply: tid=0 2024-04-17T10:01:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f01ae7ba000 1 client.8364 shutdown 2024-04-17T10:01:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2777741814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18dfac370 msgr2=0x55f18df633b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f01ae7ba000 1 --2- 172.21.15.32:0/2777741814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18dfac370 0x55f18df633b0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f019c005fd0 tx=0x7f019c05a150 comp rx=0 tx=0).stop 2024-04-17T10:01:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2777741814 shutdown_connections 2024-04-17T10:01:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f01ae7ba000 1 --2- 172.21.15.32:0/2777741814 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55f18df63dc0 0x55f18dfbcd00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f01ae7ba000 1 --2- 172.21.15.32:0/2777741814 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f18dfaba50 0x55f18dfab330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f01ae7ba000 1 --2- 172.21.15.32:0/2777741814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18dfac370 0x55f18df633b0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f01ae7ba000 1 --2- 172.21.15.32:0/2777741814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f18df5ef70 0x55f18dfaadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f0192000640 0 client.8364 ms_handle_reset on v2:172.21.15.32:6835/2058904361 2024-04-17T10:01:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2777741814 >> 172.21.15.32:0/2777741814 conn(0x55f18df8c7f0 msgr2=0x55f18df8ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2777741814 shutdown_connections 2024-04-17T10:01:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2777741814 wait complete. 2024-04-17T10:01:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.257+0000 7f01ae7ba000 20 client.8364 trim_cache size 0 max 16384 2024-04-17T10:01:22.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f01ae7ba000 1 Processor -- start 2024-04-17T10:01:22.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f01ae7ba000 1 -- start start 2024-04-17T10:01:22.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f01ae7ba000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f18df5ef70 0x55f18dfab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f01ae7ba000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18dfaba50 0x55f18df5d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f01ae7ba000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f18dfac370 0x55f18df5d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f01ae7ba000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f18ddbd7a0 con 0x55f18dfaba50 2024-04-17T10:01:22.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f01ae7ba000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f18ddaf8a0 con 0x55f18dfac370 2024-04-17T10:01:22.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f01ae7ba000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f18dd48e10 con 0x55f18df5ef70 2024-04-17T10:01:22.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f019be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18dfaba50 0x55f18df5d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f019be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18dfaba50 0x55f18df5d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53306/0 (socket says 172.21.15.32:53306) 2024-04-17T10:01:22.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f019be00640 1 -- 172.21.15.32:0/2353300087 learned_addr learned my addr 172.21.15.32:0/2353300087 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:22.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f01a9400640 1 --2- 172.21.15.32:0/2353300087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f18dfac370 0x55f18df5d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f01a8a00640 1 --2- 172.21.15.32:0/2353300087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f18df5ef70 0x55f18dfab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f019be00640 1 -- 172.21.15.32:0/2353300087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f18df5ef70 msgr2=0x55f18dfab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:22.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f019be00640 1 --2- 172.21.15.32:0/2353300087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f18df5ef70 0x55f18dfab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f019be00640 1 -- 172.21.15.32:0/2353300087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f18dfac370 msgr2=0x55f18df5d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:22.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f019be00640 1 --2- 172.21.15.32:0/2353300087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f18dfac370 0x55f18df5d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f019be00640 1 -- 172.21.15.32:0/2353300087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f18dd86190 con 0x55f18dfaba50 2024-04-17T10:01:22.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f01a9400640 1 --2- 172.21.15.32:0/2353300087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f18dfac370 0x55f18df5d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:22.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f01a8a00640 1 --2- 172.21.15.32:0/2353300087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f18df5ef70 0x55f18dfab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:22.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f019be00640 1 --2- 172.21.15.32:0/2353300087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18dfaba50 0x55f18df5d190 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f018c01ab80 tx=0x7f018c009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2f35ada41ec14 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:22.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f0198c00640 1 -- 172.21.15.32:0/2353300087 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c004240 con 0x55f18dfaba50 2024-04-17T10:01:22.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f0198c00640 1 -- 172.21.15.32:0/2353300087 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f018c0af040 con 0x55f18dfaba50 2024-04-17T10:01:22.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f0198c00640 1 -- 172.21.15.32:0/2353300087 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c0aa920 con 0x55f18dfaba50 2024-04-17T10:01:22.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2353300087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18dfaba50 msgr2=0x55f18df5d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:22.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f01ae7ba000 1 --2- 172.21.15.32:0/2353300087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18dfaba50 0x55f18df5d190 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f018c01ab80 tx=0x7f018c009240 comp rx=0 tx=0).stop 2024-04-17T10:01:22.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2353300087 shutdown_connections 2024-04-17T10:01:22.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f01ae7ba000 1 --2- 172.21.15.32:0/2353300087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f18dfac370 0x55f18df5d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f01ae7ba000 1 --2- 172.21.15.32:0/2353300087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18dfaba50 0x55f18df5d190 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f01ae7ba000 1 --2- 172.21.15.32:0/2353300087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f18df5ef70 0x55f18dfab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2353300087 >> 172.21.15.32:0/2353300087 conn(0x55f18df8c7f0 msgr2=0x55f18df61f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:22.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2353300087 shutdown_connections 2024-04-17T10:01:22.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2353300087 wait complete. 2024-04-17T10:01:22.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.273+0000 7f01ae7ba000 1 -- start start 2024-04-17T10:01:22.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01ae7ba000 1 Processor -- start 2024-04-17T10:01:22.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01ae7ba000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18df5ef70 0x55f18df5e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01ae7ba000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f18dfaba50 0x55f18dfbdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01ae7ba000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f18dfac370 0x55f18dfbe290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01ae7ba000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f18dd6ea90 con 0x55f18df5ef70 2024-04-17T10:01:22.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01ae7ba000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f18dd55230 con 0x55f18dfac370 2024-04-17T10:01:22.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01ae7ba000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f18ddb4bc0 con 0x55f18dfaba50 2024-04-17T10:01:22.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01ae7ba000 10 client.0 _ll_register_callbacks cb 0x7f019400a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:01:22.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01a8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18df5ef70 0x55f18df5e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01a8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18df5ef70 0x55f18df5e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53320/0 (socket says 172.21.15.32:53320) 2024-04-17T10:01:22.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01a8a00640 1 -- 172.21.15.32:0/2035866216 learned_addr learned my addr 172.21.15.32:0/2035866216 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:22.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f019be00640 1 --2- 172.21.15.32:0/2035866216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f18dfaba50 0x55f18dfbdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01a9400640 1 --2- 172.21.15.32:0/2035866216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f18dfac370 0x55f18dfbe290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01a8a00640 1 -- 172.21.15.32:0/2035866216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f18dfaba50 msgr2=0x55f18dfbdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:22.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01a8a00640 1 --2- 172.21.15.32:0/2035866216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f18dfaba50 0x55f18dfbdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01a8a00640 1 -- 172.21.15.32:0/2035866216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f18dfac370 msgr2=0x55f18dfbe290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:22.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01a8a00640 1 --2- 172.21.15.32:0/2035866216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f18dfac370 0x55f18dfbe290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01a8a00640 1 -- 172.21.15.32:0/2035866216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f18dd697d0 con 0x55f18df5ef70 2024-04-17T10:01:22.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01a8a00640 1 --2- 172.21.15.32:0/2035866216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18df5ef70 0x55f18df5e590 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x55f18ddb4de0 tx=0x7f0194004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4c9217a6e5e93d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:22.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f0192000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:01:22.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f0192000640 1 -- 172.21.15.32:0/2035866216 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019402f060 con 0x55f18df5ef70 2024-04-17T10:01:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f0192000640 1 -- 172.21.15.32:0/2035866216 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f0194006710 con 0x55f18df5ef70 2024-04-17T10:01:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2035866216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f018c0af040 con 0x55f18df5ef70 2024-04-17T10:01:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f0192000640 1 -- 172.21.15.32:0/2035866216 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f0194035050 con 0x55f18df5ef70 2024-04-17T10:01:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01ae7ba000 10 client.8370 fetch_fsmap learned FSMap version 72 2024-04-17T10:01:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2035866216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55f18dd55440 con 0x55f18df5ef70 2024-04-17T10:01:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f0192000640 1 -- 172.21.15.32:0/2035866216 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f018c0af040 con 0x55f18df5ef70 2024-04-17T10:01:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f0192000640 1 -- 172.21.15.32:0/2035866216 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 72) ==== 3495+0+0 (secure 0 0 0) 0x7f0194038020 con 0x55f18df5ef70 2024-04-17T10:01:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01ae7ba000 10 client.8370 fetch_fsmap finished waiting for FSMap version 72 2024-04-17T10:01:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01ae7ba000 10 client.8370 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T10:01:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01ae7ba000 20 client.8370 populate_metadata read hostname 'smithi032' 2024-04-17T10:01:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01ae7ba000 1 --2- 172.21.15.32:0/2035866216 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55f18e01a0d0 0x55f18e01a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01ae7ba000 4 client.8370 mds_command: new command op to 8196 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-17T10:01:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2035866216 --> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55f18ddbdfb0 con 0x55f18e01a0d0 2024-04-17T10:01:22.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f019be00640 1 --2- 172.21.15.32:0/2035866216 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55f18e01a0d0 0x55f18e01a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f0192000640 10 client.8370 ms_handle_connect on v2:172.21.15.32:6835/2058904361 2024-04-17T10:01:22.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f019be00640 1 --2- 172.21.15.32:0/2035866216 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55f18e01a0d0 0x55f18e01a4b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc57b720d414a784 server_cookie=fcc41713e881bd31 in_seq=0 out_seq=0 2024-04-17T10:01:22.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f0192000640 1 -- 172.21.15.32:0/2035866216 <== mds.0 v2:172.21.15.32:6835/2058904361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f18ddbdfb0 con 0x55f18e01a0d0 2024-04-17T10:01:22.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f0192000640 10 client.8370 handle_command_reply: tid=0 2024-04-17T10:01:22.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.277+0000 7f01ae7ba000 1 client.8370 shutdown 2024-04-17T10:01:22.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2035866216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18df5ef70 msgr2=0x55f18df5e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f01ae7ba000 1 --2- 172.21.15.32:0/2035866216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18df5ef70 0x55f18df5e590 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x55f18ddb4de0 tx=0x7f0194004740 comp rx=0 tx=0).stop 2024-04-17T10:01:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2035866216 shutdown_connections 2024-04-17T10:01:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f01ae7ba000 1 --2- 172.21.15.32:0/2035866216 >> [v2:172.21.15.32:6835/2058904361,v1:172.21.15.32:6837/2058904361] conn(0x55f18e01a0d0 0x55f18e01a4b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f01ae7ba000 1 --2- 172.21.15.32:0/2035866216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f18dfaba50 0x55f18dfbdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f01ae7ba000 1 --2- 172.21.15.32:0/2035866216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f18dfac370 0x55f18dfbe290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f0192000640 0 client.8370 ms_handle_reset on v2:172.21.15.32:6835/2058904361 2024-04-17T10:01:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f01ae7ba000 1 --2- 172.21.15.32:0/2035866216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18df5ef70 0x55f18df5e590 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2035866216 >> 172.21.15.32:0/2035866216 conn(0x55f18df8c7f0 msgr2=0x55f18df627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2035866216 shutdown_connections 2024-04-17T10:01:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f01ae7ba000 1 -- 172.21.15.32:0/2035866216 wait complete. 2024-04-17T10:01:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f01ae7ba000 20 client.8370 trim_cache size 0 max 16384 2024-04-17T10:01:22.284 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:22.284 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2024-04-17T10:01:22.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f01ab400640 1 -- 172.21.15.32:0/2885025308 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f01800683b0 msgr2=0x7f018006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:22.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f01ab400640 1 --2- 172.21.15.32:0/2885025308 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f01800683b0 0x7f018006a870 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0194006fd0 tx=0x7f0194008040 comp rx=0 tx=0).stop 2024-04-17T10:01:22.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f01ab400640 1 -- 172.21.15.32:0/2885025308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a415b7d0 msgr2=0x7f01a40b3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:22.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f01ab400640 1 --2- 172.21.15.32:0/2885025308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a415b7d0 0x7f01a40b3210 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f019c00c7a0 tx=0x7f019c00cc60 comp rx=0 tx=0).stop 2024-04-17T10:01:22.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f01ab400640 1 -- 172.21.15.32:0/2885025308 shutdown_connections 2024-04-17T10:01:22.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f01ab400640 1 --2- 172.21.15.32:0/2885025308 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f01800683b0 0x7f018006a870 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f01ab400640 1 --2- 172.21.15.32:0/2885025308 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01a406f7c0 0x7f01a414ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f01ab400640 1 --2- 172.21.15.32:0/2885025308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a415b7d0 0x7f01a40b3210 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f01ab400640 1 --2- 172.21.15.32:0/2885025308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01a406f270 0x7f01a40af0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f01ab400640 1 -- 172.21.15.32:0/2885025308 >> 172.21.15.32:0/2885025308 conn(0x7f01a40b2120 msgr2=0x7f01a4072300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:22.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f01ab400640 1 -- 172.21.15.32:0/2885025308 shutdown_connections 2024-04-17T10:01:22.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.281+0000 7f01ab400640 1 -- 172.21.15.32:0/2885025308 wait complete. 2024-04-17T10:01:22.297 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-17T10:01:22.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.441+0000 7f265f000640 1 Processor -- start 2024-04-17T10:01:22.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.441+0000 7f265f000640 1 -- start start 2024-04-17T10:01:22.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.441+0000 7f265f000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265809b100 0x7f2658169960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.441+0000 7f265f000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f265815ef10 con 0x7f265815eb30 2024-04-17T10:01:22.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.441+0000 7f265f000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26580d0db0 con 0x7f26580d0890 2024-04-17T10:01:22.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.441+0000 7f265f000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f265806dd60 con 0x7f265809b100 2024-04-17T10:01:22.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.441+0000 7f2657e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f265815eb30 0x7f265809b770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50370/0 (socket says 172.21.15.32:50370) 2024-04-17T10:01:22.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.441+0000 7f2657e00640 1 -- 172.21.15.32:0/3671681112 learned_addr learned my addr 172.21.15.32:0/3671681112 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:22.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.441+0000 7f265d000640 1 --2- 172.21.15.32:0/3671681112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265809b100 0x7f2658169960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.441+0000 7f2657400640 1 -- 172.21.15.32:0/3671681112 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2120711051 0 0) 0x7f265815ef10 con 0x7f265815eb30 2024-04-17T10:01:22.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.441+0000 7f2657400640 1 -- 172.21.15.32:0/3671681112 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f263c003530 con 0x7f265815eb30 2024-04-17T10:01:22.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.441+0000 7f2657400640 1 -- 172.21.15.32:0/3671681112 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3610873857 0 0) 0x7f26580d0db0 con 0x7f26580d0890 2024-04-17T10:01:22.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.441+0000 7f2657400640 1 -- 172.21.15.32:0/3671681112 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f265815ef10 con 0x7f26580d0890 2024-04-17T10:01:22.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.441+0000 7f2657400640 1 -- 172.21.15.32:0/3671681112 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4028067550 0 0) 0x7f263c003530 con 0x7f265815eb30 2024-04-17T10:01:22.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.441+0000 7f265d000640 1 -- 172.21.15.32:0/3671681112 >> v1:172.21.15.144:6789/0 conn(0x7f26580d0890 legacy=0x7f265809a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:22.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.441+0000 7f265d000640 1 -- 172.21.15.32:0/3671681112 >> v1:172.21.15.32:6789/0 conn(0x7f265815eb30 legacy=0x7f265809b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:22.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.441+0000 7f265d000640 1 -- 172.21.15.32:0/3671681112 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26580d1090 con 0x7f265809b100 2024-04-17T10:01:22.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.445+0000 7f265d000640 1 --2- 172.21.15.32:0/3671681112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265809b100 0x7f2658169960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2650009dc0 tx=0x7f265001f0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=532467fd29bde203 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:22.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.445+0000 7f2657400640 1 -- 172.21.15.32:0/3671681112 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26500b5070 con 0x7f265809b100 2024-04-17T10:01:22.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.445+0000 7f2657400640 1 -- 172.21.15.32:0/3671681112 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26500a8c30 con 0x7f265809b100 2024-04-17T10:01:22.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.445+0000 7f2657400640 1 -- 172.21.15.32:0/3671681112 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648001680 con 0x7f265809b100 2024-04-17T10:01:22.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.445+0000 7f265f000640 1 -- 172.21.15.32:0/3671681112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265809b100 msgr2=0x7f2658169960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:22.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.445+0000 7f265f000640 1 --2- 172.21.15.32:0/3671681112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265809b100 0x7f2658169960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2650009dc0 tx=0x7f265001f0a0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.445+0000 7f265f000640 1 -- 172.21.15.32:0/3671681112 shutdown_connections 2024-04-17T10:01:22.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.445+0000 7f265f000640 1 --2- 172.21.15.32:0/3671681112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265809b100 0x7f2658169960 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.445+0000 7f265f000640 1 -- 172.21.15.32:0/3671681112 >> 172.21.15.32:0/3671681112 conn(0x7f26580b3fd0 msgr2=0x7f26580b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:22.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.445+0000 7f265f000640 1 -- 172.21.15.32:0/3671681112 shutdown_connections 2024-04-17T10:01:22.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.445+0000 7f265f000640 1 -- 172.21.15.32:0/3671681112 wait complete. 2024-04-17T10:01:22.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f265f000640 1 Processor -- start 2024-04-17T10:01:22.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f265f000640 1 -- start start 2024-04-17T10:01:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f265f000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f265809b100 0x7f265815da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f265f000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26580d0890 0x7f2658160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f265f000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265815eb30 0x7f2658160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f265f000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f26580d1930 con 0x7f26580d0890 2024-04-17T10:01:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f265f000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f26580d1650 con 0x7f265809b100 2024-04-17T10:01:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f265f000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f26580cdb40 con 0x7f265815eb30 2024-04-17T10:01:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f264f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26580d0890 0x7f2658160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f264f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26580d0890 0x7f2658160680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53334/0 (socket says 172.21.15.32:53334) 2024-04-17T10:01:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f264f400640 1 -- 172.21.15.32:0/3452630991 learned_addr learned my addr 172.21.15.32:0/3452630991 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f265d000640 1 --2- 172.21.15.32:0/3452630991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265815eb30 0x7f2658160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f2657e00640 1 --2- 172.21.15.32:0/3452630991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f265809b100 0x7f265815da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f264f400640 1 -- 172.21.15.32:0/3452630991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265815eb30 msgr2=0x7f2658160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f264f400640 1 --2- 172.21.15.32:0/3452630991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265815eb30 0x7f2658160bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f264f400640 1 -- 172.21.15.32:0/3452630991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f265809b100 msgr2=0x7f265815da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:22.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f264f400640 1 --2- 172.21.15.32:0/3452630991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f265809b100 0x7f265815da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f264f400640 1 -- 172.21.15.32:0/3452630991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26580c82d0 con 0x7f26580d0890 2024-04-17T10:01:22.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f264f400640 1 --2- 172.21.15.32:0/3452630991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26580d0890 0x7f2658160680 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f264000c410 tx=0x7f264000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:22.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f2654c00640 1 -- 172.21.15.32:0/3452630991 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640017070 con 0x7f26580d0890 2024-04-17T10:01:22.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f2654c00640 1 -- 172.21.15.32:0/3452630991 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26400052f0 con 0x7f26580d0890 2024-04-17T10:01:22.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f2654c00640 1 -- 172.21.15.32:0/3452630991 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640003840 con 0x7f26580d0890 2024-04-17T10:01:22.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f265f000640 1 -- 172.21.15.32:0/3452630991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26580c85b0 con 0x7f26580d0890 2024-04-17T10:01:22.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.449+0000 7f265f000640 1 -- 172.21.15.32:0/3452630991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26580c8b70 con 0x7f26580d0890 2024-04-17T10:01:22.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.453+0000 7f265f000640 1 -- 172.21.15.32:0/3452630991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2658068a90 con 0x7f26580d0890 2024-04-17T10:01:22.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.453+0000 7f2654c00640 1 -- 172.21.15.32:0/3452630991 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f264001a070 con 0x7f26580d0890 2024-04-17T10:01:22.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.453+0000 7f2654c00640 1 --2- 172.21.15.32:0/3452630991 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2634068380 0x7f263406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:22.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.453+0000 7f2654c00640 1 -- 172.21.15.32:0/3452630991 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f2640085e40 con 0x7f26580d0890 2024-04-17T10:01:22.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.453+0000 7f2657e00640 1 --2- 172.21.15.32:0/3452630991 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2634068380 0x7f263406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:22.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.453+0000 7f2657e00640 1 --2- 172.21.15.32:0/3452630991 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2634068380 0x7f263406a840 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f26480011f0 tx=0x7f2648009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:22.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.457+0000 7f2654c00640 1 -- 172.21.15.32:0/3452630991 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2640055cb0 con 0x7f26580d0890 2024-04-17T10:01:22.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.657+0000 7f265f000640 1 -- 172.21.15.32:0/3452630991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f26580b4c30 con 0x7f26580d0890 2024-04-17T10:01:22.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.657+0000 7f2654c00640 1 -- 172.21.15.32:0/3452630991 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f2640059fb0 con 0x7f26580d0890 2024-04-17T10:01:22.656 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:22.657 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-17T10:01:22.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.661+0000 7f265f000640 1 -- 172.21.15.32:0/3452630991 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2634068380 msgr2=0x7f263406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:22.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.661+0000 7f265f000640 1 --2- 172.21.15.32:0/3452630991 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2634068380 0x7f263406a840 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f26480011f0 tx=0x7f2648009040 comp rx=0 tx=0).stop 2024-04-17T10:01:22.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.661+0000 7f265f000640 1 -- 172.21.15.32:0/3452630991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26580d0890 msgr2=0x7f2658160680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:22.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.661+0000 7f265f000640 1 --2- 172.21.15.32:0/3452630991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26580d0890 0x7f2658160680 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f264000c410 tx=0x7f264000c8d0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.661+0000 7f265f000640 1 -- 172.21.15.32:0/3452630991 shutdown_connections 2024-04-17T10:01:22.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.661+0000 7f265f000640 1 --2- 172.21.15.32:0/3452630991 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2634068380 0x7f263406a840 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.661+0000 7f265f000640 1 --2- 172.21.15.32:0/3452630991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265815eb30 0x7f2658160bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.661+0000 7f265f000640 1 --2- 172.21.15.32:0/3452630991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26580d0890 0x7f2658160680 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.661+0000 7f265f000640 1 --2- 172.21.15.32:0/3452630991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f265809b100 0x7f265815da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:22.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.661+0000 7f265f000640 1 -- 172.21.15.32:0/3452630991 >> 172.21.15.32:0/3452630991 conn(0x7f26580b3fd0 msgr2=0x7f265806cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:22.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.661+0000 7f265f000640 1 -- 172.21.15.32:0/3452630991 shutdown_connections 2024-04-17T10:01:22.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:22.661+0000 7f265f000640 1 -- 172.21.15.32:0/3452630991 wait complete. 2024-04-17T10:01:22.678 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-17T10:01:22.678 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T10:01:22.678 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:01:22.763 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:01:22.855 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T10:01:22.855 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T10:01:22.855 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:01:22.855 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:01:22.865 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:01:23.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.053+0000 7fa7e6400640 1 Processor -- start 2024-04-17T10:01:23.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.053+0000 7fa7e6400640 1 -- start start 2024-04-17T10:01:23.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.057+0000 7fa7e6400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7e0144e90 0x7fa7e00b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:23.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.057+0000 7fa7e6400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa7e00b4de0 0x7fa7e015baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:23.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.057+0000 7fa7e6400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7e0072bb0 con 0x7fa7e015bfe0 2024-04-17T10:01:23.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.057+0000 7fa7e6400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa7e015c430 con 0x7fa7e00b4de0 2024-04-17T10:01:23.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.057+0000 7fa7e6400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e0145270 con 0x7fa7e0144e90 2024-04-17T10:01:23.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.057+0000 7fa7d6a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa7e00b4de0 0x7fa7e015baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:23.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.057+0000 7fa7d6a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa7e00b4de0 0x7fa7e015baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53346/0 (socket says 172.21.15.32:53346) 2024-04-17T10:01:23.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.057+0000 7fa7d6a00640 1 -- 172.21.15.32:0/3865865663 learned_addr learned my addr 172.21.15.32:0/3865865663 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:23.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.057+0000 7fa7df400640 1 --2- 172.21.15.32:0/3865865663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7e0144e90 0x7fa7e00b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:23.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.057+0000 7fa7d6a00640 1 -- 172.21.15.32:0/3865865663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7e0144e90 msgr2=0x7fa7e00b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:23.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.057+0000 7fa7d6a00640 1 --2- 172.21.15.32:0/3865865663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7e0144e90 0x7fa7e00b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.057+0000 7fa7d6a00640 1 -- 172.21.15.32:0/3865865663 >> v1:172.21.15.144:6789/0 conn(0x7fa7e015bfe0 legacy=0x7fa7e006ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:23.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.057+0000 7fa7d6a00640 1 -- 172.21.15.32:0/3865865663 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e00b3dc0 con 0x7fa7e00b4de0 2024-04-17T10:01:23.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.057+0000 7fa7d6a00640 1 --2- 172.21.15.32:0/3865865663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7e00b4de0 0x7fa7e015baa0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80098a0 tx=0x7fa7c801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404448b684f3b0c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:23.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.057+0000 7fa7dea00640 1 -- 172.21.15.32:0/3865865663 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c80aa670 con 0x7fa7e00b4de0 2024-04-17T10:01:23.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.061+0000 7fa7dea00640 1 -- 172.21.15.32:0/3865865663 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7c80af070 con 0x7fa7e00b4de0 2024-04-17T10:01:23.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.061+0000 7fa7dea00640 1 -- 172.21.15.32:0/3865865663 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c80aaa30 con 0x7fa7e00b4de0 2024-04-17T10:01:23.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.061+0000 7fa7e6400640 1 -- 172.21.15.32:0/3865865663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7e00b4de0 msgr2=0x7fa7e015baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:23.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.061+0000 7fa7e6400640 1 --2- 172.21.15.32:0/3865865663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7e00b4de0 0x7fa7e015baa0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80098a0 tx=0x7fa7c801c510 comp rx=0 tx=0).stop 2024-04-17T10:01:23.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.061+0000 7fa7e6400640 1 -- 172.21.15.32:0/3865865663 shutdown_connections 2024-04-17T10:01:23.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.061+0000 7fa7e6400640 1 --2- 172.21.15.32:0/3865865663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7e00b4de0 0x7fa7e015baa0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.061+0000 7fa7e6400640 1 --2- 172.21.15.32:0/3865865663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7e0144e90 0x7fa7e00b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.061+0000 7fa7e6400640 1 -- 172.21.15.32:0/3865865663 >> 172.21.15.32:0/3865865663 conn(0x7fa7e006ba30 msgr2=0x7fa7e006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:23.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.061+0000 7fa7e6400640 1 -- 172.21.15.32:0/3865865663 shutdown_connections 2024-04-17T10:01:23.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.061+0000 7fa7e6400640 1 -- 172.21.15.32:0/3865865663 wait complete. 2024-04-17T10:01:23.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.061+0000 7fa7e6400640 1 Processor -- start 2024-04-17T10:01:23.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.061+0000 7fa7e6400640 1 -- start start 2024-04-17T10:01:23.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.061+0000 7fa7e6400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7e00b4de0 0x7fa7e015a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:23.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.061+0000 7fa7e6400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7e0144e90 0x7fa7e01382f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:23.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.061+0000 7fa7e6400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7e015bfe0 0x7fa7e0138830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:23.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.061+0000 7fa7e6400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e0072670 con 0x7fa7e015bfe0 2024-04-17T10:01:23.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.061+0000 7fa7e6400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e006f880 con 0x7fa7e00b4de0 2024-04-17T10:01:23.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.061+0000 7fa7e6400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e00727f0 con 0x7fa7e0144e90 2024-04-17T10:01:23.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.061+0000 7fa7dfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7e015bfe0 0x7fa7e0138830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:23.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.061+0000 7fa7dfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7e015bfe0 0x7fa7e0138830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53356/0 (socket says 172.21.15.32:53356) 2024-04-17T10:01:23.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.061+0000 7fa7dfe00640 1 -- 172.21.15.32:0/1212455286 learned_addr learned my addr 172.21.15.32:0/1212455286 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:23.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.061+0000 7fa7d6a00640 1 --2- 172.21.15.32:0/1212455286 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7e0144e90 0x7fa7e01382f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:23.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.061+0000 7fa7df400640 1 --2- 172.21.15.32:0/1212455286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7e00b4de0 0x7fa7e015a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:23.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.065+0000 7fa7dfe00640 1 -- 172.21.15.32:0/1212455286 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7e0144e90 msgr2=0x7fa7e01382f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:23.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.065+0000 7fa7dfe00640 1 --2- 172.21.15.32:0/1212455286 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7e0144e90 0x7fa7e01382f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.065+0000 7fa7dfe00640 1 -- 172.21.15.32:0/1212455286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7e00b4de0 msgr2=0x7fa7e015a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:23.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.065+0000 7fa7dfe00640 1 --2- 172.21.15.32:0/1212455286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7e00b4de0 0x7fa7e015a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.065+0000 7fa7dfe00640 1 -- 172.21.15.32:0/1212455286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e0138f20 con 0x7fa7e015bfe0 2024-04-17T10:01:23.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.065+0000 7fa7dfe00640 1 --2- 172.21.15.32:0/1212455286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7e015bfe0 0x7fa7e0138830 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800c770 tx=0x7fa7d800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:23.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.065+0000 7fa7d7e00640 1 -- 172.21.15.32:0/1212455286 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8003b80 con 0x7fa7e015bfe0 2024-04-17T10:01:23.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.065+0000 7fa7d7e00640 1 -- 172.21.15.32:0/1212455286 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d8003d20 con 0x7fa7e015bfe0 2024-04-17T10:01:23.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.065+0000 7fa7d7e00640 1 -- 172.21.15.32:0/1212455286 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d800aa90 con 0x7fa7e015bfe0 2024-04-17T10:01:23.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.065+0000 7fa7e6400640 1 -- 172.21.15.32:0/1212455286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e0139200 con 0x7fa7e015bfe0 2024-04-17T10:01:23.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.065+0000 7fa7e6400640 1 -- 172.21.15.32:0/1212455286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e01397c0 con 0x7fa7e015bfe0 2024-04-17T10:01:23.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.065+0000 7fa7e6400640 1 -- 172.21.15.32:0/1212455286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7e0068a90 con 0x7fa7e015bfe0 2024-04-17T10:01:23.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.065+0000 7fa7d7e00640 1 -- 172.21.15.32:0/1212455286 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa7d800ac30 con 0x7fa7e015bfe0 2024-04-17T10:01:23.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.065+0000 7fa7d7e00640 1 --2- 172.21.15.32:0/1212455286 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa7bc068380 0x7fa7bc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:23.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.065+0000 7fa7d7e00640 1 -- 172.21.15.32:0/1212455286 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7fa7d8086900 con 0x7fa7e015bfe0 2024-04-17T10:01:23.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.069+0000 7fa7df400640 1 --2- 172.21.15.32:0/1212455286 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa7bc068380 0x7fa7bc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:23.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.069+0000 7fa7df400640 1 --2- 172.21.15.32:0/1212455286 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa7bc068380 0x7fa7bc06a840 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00079c0 tx=0x7fa7d0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:23.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.073+0000 7fa7d7e00640 1 -- 172.21.15.32:0/1212455286 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa7d80566e0 con 0x7fa7e015bfe0 2024-04-17T10:01:23.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.237+0000 7fa7e6400640 1 -- 172.21.15.32:0/1212455286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7e00b4a30 con 0x7fa7e015bfe0 2024-04-17T10:01:23.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.241+0000 7fa7d7e00640 1 -- 172.21.15.32:0/1212455286 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+17029 (secure 0 0 0) 0x7fa7d805a9e0 con 0x7fa7e015bfe0 2024-04-17T10:01:23.238 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:23.238 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":86,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:00:43.842900+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:59:03.223426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-17T09:59:03.780120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.11","mappings":[{"from":1,"to":3}]},{"pgid":"13.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6837/1253497636":"2024-04-18T10:00:43.842876+0000","172.21.15.32:6835/1253497636":"2024-04-18T10:00:43.842876+0000","172.21.15.32:6837/2656735817":"2024-04-18T09:59:28.985107+0000","172.21.15.32:6835/2656735817":"2024-04-18T09:59:28.985107+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:01:23.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.241+0000 7fa7e6400640 1 -- 172.21.15.32:0/1212455286 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa7bc068380 msgr2=0x7fa7bc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:23.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.241+0000 7fa7e6400640 1 --2- 172.21.15.32:0/1212455286 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa7bc068380 0x7fa7bc06a840 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00079c0 tx=0x7fa7d0008040 comp rx=0 tx=0).stop 2024-04-17T10:01:23.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.241+0000 7fa7e6400640 1 -- 172.21.15.32:0/1212455286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7e015bfe0 msgr2=0x7fa7e0138830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:23.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.241+0000 7fa7e6400640 1 --2- 172.21.15.32:0/1212455286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7e015bfe0 0x7fa7e0138830 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800c770 tx=0x7fa7d800cc30 comp rx=0 tx=0).stop 2024-04-17T10:01:23.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.241+0000 7fa7e6400640 1 -- 172.21.15.32:0/1212455286 shutdown_connections 2024-04-17T10:01:23.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.241+0000 7fa7e6400640 1 --2- 172.21.15.32:0/1212455286 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa7bc068380 0x7fa7bc06a840 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.241+0000 7fa7e6400640 1 --2- 172.21.15.32:0/1212455286 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7e0144e90 0x7fa7e01382f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.241+0000 7fa7e6400640 1 --2- 172.21.15.32:0/1212455286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7e015bfe0 0x7fa7e0138830 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.241+0000 7fa7e6400640 1 --2- 172.21.15.32:0/1212455286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7e00b4de0 0x7fa7e015a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.241+0000 7fa7e6400640 1 -- 172.21.15.32:0/1212455286 >> 172.21.15.32:0/1212455286 conn(0x7fa7e006ba30 msgr2=0x7fa7e0143e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:23.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.241+0000 7fa7e6400640 1 -- 172.21.15.32:0/1212455286 shutdown_connections 2024-04-17T10:01:23.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.241+0000 7fa7e6400640 1 -- 172.21.15.32:0/1212455286 wait complete. 2024-04-17T10:01:23.260 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:59:03.223426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:59:03.780120+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:01:23.261 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:01:23.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.409+0000 7f35ba000640 1 Processor -- start 2024-04-17T10:01:23.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.409+0000 7f35ba000640 1 -- start start 2024-04-17T10:01:23.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.409+0000 7f35ba000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f35b40cf370 0x7f35b40cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:23.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.409+0000 7f35ba000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f35b4071e60 0x7f35b4072270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:23.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.409+0000 7f35ba000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35b40d0130 con 0x7f35b406ba30 2024-04-17T10:01:23.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.409+0000 7f35ba000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f35b415d090 con 0x7f35b4071e60 2024-04-17T10:01:23.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.409+0000 7f35ba000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f35b415d370 con 0x7f35b40cf370 2024-04-17T10:01:23.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.409+0000 7f35b3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f35b4071e60 0x7f35b4072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:23.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.409+0000 7f35b3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f35b4071e60 0x7f35b4072270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53366/0 (socket says 172.21.15.32:53366) 2024-04-17T10:01:23.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.409+0000 7f35b3e00640 1 -- 172.21.15.32:0/1776683466 learned_addr learned my addr 172.21.15.32:0/1776683466 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:23.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.409+0000 7f35abe00640 1 --2- 172.21.15.32:0/1776683466 >> v2:172.21.15.144:3300/0 conn(0x7f35b40cf370 0x7f35b40cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:23.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.409+0000 7f35b3e00640 1 -- 172.21.15.32:0/1776683466 >> v2:172.21.15.144:3300/0 conn(0x7f35b40cf370 msgr2=0x7f35b40cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:23.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.409+0000 7f35b3e00640 1 --2- 172.21.15.32:0/1776683466 >> v2:172.21.15.144:3300/0 conn(0x7f35b40cf370 0x7f35b40cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.409+0000 7f35b3e00640 1 -- 172.21.15.32:0/1776683466 >> v1:172.21.15.144:6789/0 conn(0x7f35b406ba30 legacy=0x7f35b406be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:23.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.409+0000 7f35b3e00640 1 -- 172.21.15.32:0/1776683466 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b415d650 con 0x7f35b4071e60 2024-04-17T10:01:23.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.413+0000 7f35b3e00640 1 --2- 172.21.15.32:0/1776683466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35b4071e60 0x7f35b4072270 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f35b40cc7c0 tx=0x7f35ac01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34dcabe733c1e0f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.413+0000 7f35b2a00640 1 -- 172.21.15.32:0/1776683466 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac0a5890 con 0x7f35b4071e60 2024-04-17T10:01:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.413+0000 7f35b2a00640 1 -- 172.21.15.32:0/1776683466 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35ac0a5a30 con 0x7f35b4071e60 2024-04-17T10:01:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.413+0000 7f35b2a00640 1 -- 172.21.15.32:0/1776683466 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac0a5e00 con 0x7f35b4071e60 2024-04-17T10:01:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.413+0000 7f35ba000640 1 -- 172.21.15.32:0/1776683466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35b4071e60 msgr2=0x7f35b4072270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.413+0000 7f35ba000640 1 --2- 172.21.15.32:0/1776683466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35b4071e60 0x7f35b4072270 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f35b40cc7c0 tx=0x7f35ac01c9b0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.413+0000 7f35ba000640 1 -- 172.21.15.32:0/1776683466 shutdown_connections 2024-04-17T10:01:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.413+0000 7f35ba000640 1 --2- 172.21.15.32:0/1776683466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35b4071e60 0x7f35b4072270 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.413+0000 7f35ba000640 1 --2- 172.21.15.32:0/1776683466 >> v2:172.21.15.144:3300/0 conn(0x7f35b40cf370 0x7f35b40cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.413+0000 7f35ba000640 1 -- 172.21.15.32:0/1776683466 >> 172.21.15.32:0/1776683466 conn(0x7f35b40b2120 msgr2=0x7f35b40b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:23.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.413+0000 7f35ba000640 1 -- 172.21.15.32:0/1776683466 shutdown_connections 2024-04-17T10:01:23.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.413+0000 7f35ba000640 1 -- 172.21.15.32:0/1776683466 wait complete. 2024-04-17T10:01:23.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.413+0000 7f35ba000640 1 Processor -- start 2024-04-17T10:01:23.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35ba000640 1 -- start start 2024-04-17T10:01:23.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35ba000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35b406ba30 0x7f35b412ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:23.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35ba000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35b4071e60 0x7f35b4129400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:23.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35ba000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35b40cf370 0x7f35b4140410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:23.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35ba000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f35b4146230 con 0x7f35b4071e60 2024-04-17T10:01:23.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35ba000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35b415dd80 con 0x7f35b406ba30 2024-04-17T10:01:23.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35ba000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f35b41463b0 con 0x7f35b40cf370 2024-04-17T10:01:23.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35abe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35b4071e60 0x7f35b4129400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:23.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35abe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35b4071e60 0x7f35b4129400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53382/0 (socket says 172.21.15.32:53382) 2024-04-17T10:01:23.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35abe00640 1 -- 172.21.15.32:0/3170289551 learned_addr learned my addr 172.21.15.32:0/3170289551 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:23.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35b3e00640 1 --2- 172.21.15.32:0/3170289551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35b40cf370 0x7f35b4140410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:23.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35b3400640 1 --2- 172.21.15.32:0/3170289551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35b406ba30 0x7f35b412ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:23.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35abe00640 1 -- 172.21.15.32:0/3170289551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35b40cf370 msgr2=0x7f35b4140410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:23.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35abe00640 1 --2- 172.21.15.32:0/3170289551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35b40cf370 0x7f35b4140410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35abe00640 1 -- 172.21.15.32:0/3170289551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35b406ba30 msgr2=0x7f35b412ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:23.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35abe00640 1 --2- 172.21.15.32:0/3170289551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35b406ba30 0x7f35b412ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35abe00640 1 -- 172.21.15.32:0/3170289551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b4129d00 con 0x7f35b4071e60 2024-04-17T10:01:23.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35abe00640 1 --2- 172.21.15.32:0/3170289551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35b4071e60 0x7f35b4129400 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f359c00d620 tx=0x7f359c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:23.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35ab400640 1 -- 172.21.15.32:0/3170289551 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c00b620 con 0x7f35b4071e60 2024-04-17T10:01:23.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35ab400640 1 -- 172.21.15.32:0/3170289551 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f359c019070 con 0x7f35b4071e60 2024-04-17T10:01:23.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35ab400640 1 -- 172.21.15.32:0/3170289551 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c00b9e0 con 0x7f35b4071e60 2024-04-17T10:01:23.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35ba000640 1 -- 172.21.15.32:0/3170289551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b4129fe0 con 0x7f35b4071e60 2024-04-17T10:01:23.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35ba000640 1 -- 172.21.15.32:0/3170289551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b412a5a0 con 0x7f35b4071e60 2024-04-17T10:01:23.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.417+0000 7f35ba000640 1 -- 172.21.15.32:0/3170289551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35b4068a90 con 0x7f35b4071e60 2024-04-17T10:01:23.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.421+0000 7f35ab400640 1 -- 172.21.15.32:0/3170289551 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f359c00bb80 con 0x7f35b4071e60 2024-04-17T10:01:23.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.421+0000 7f35ab400640 1 --2- 172.21.15.32:0/3170289551 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3590068380 0x7f359006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:23.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.421+0000 7f35ab400640 1 -- 172.21.15.32:0/3170289551 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f359c086e90 con 0x7f35b4071e60 2024-04-17T10:01:23.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.421+0000 7f35b3400640 1 --2- 172.21.15.32:0/3170289551 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3590068380 0x7f359006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:23.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.421+0000 7f35b3400640 1 --2- 172.21.15.32:0/3170289551 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3590068380 0x7f359006a840 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f35a4004e30 tx=0x7f35a4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:23.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.425+0000 7f35ab400640 1 -- 172.21.15.32:0/3170289551 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f359c0573f0 con 0x7f35b4071e60 2024-04-17T10:01:23.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.593+0000 7f35ba000640 1 -- 172.21.15.32:0/3170289551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f35b406c850 con 0x7f35b4071e60 2024-04-17T10:01:23.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.593+0000 7f35ab400640 1 -- 172.21.15.32:0/3170289551 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7f359c056cf0 con 0x7f35b4071e60 2024-04-17T10:01:23.590 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T10:01:23.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.593+0000 7f35ba000640 1 -- 172.21.15.32:0/3170289551 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3590068380 msgr2=0x7f359006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:23.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.593+0000 7f35ba000640 1 --2- 172.21.15.32:0/3170289551 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3590068380 0x7f359006a840 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f35a4004e30 tx=0x7f35a4009040 comp rx=0 tx=0).stop 2024-04-17T10:01:23.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.593+0000 7f35ba000640 1 -- 172.21.15.32:0/3170289551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35b4071e60 msgr2=0x7f35b4129400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:23.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.593+0000 7f35ba000640 1 --2- 172.21.15.32:0/3170289551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35b4071e60 0x7f35b4129400 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f359c00d620 tx=0x7f359c00dae0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.593+0000 7f35ba000640 1 -- 172.21.15.32:0/3170289551 shutdown_connections 2024-04-17T10:01:23.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.593+0000 7f35ba000640 1 --2- 172.21.15.32:0/3170289551 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3590068380 0x7f359006a840 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.593+0000 7f35ba000640 1 --2- 172.21.15.32:0/3170289551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35b40cf370 0x7f35b4140410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.593+0000 7f35ba000640 1 --2- 172.21.15.32:0/3170289551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35b4071e60 0x7f35b4129400 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.593+0000 7f35ba000640 1 --2- 172.21.15.32:0/3170289551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35b406ba30 0x7f35b412ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.593+0000 7f35ba000640 1 -- 172.21.15.32:0/3170289551 >> 172.21.15.32:0/3170289551 conn(0x7f35b40b2120 msgr2=0x7f35b40b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:23.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.593+0000 7f35ba000640 1 -- 172.21.15.32:0/3170289551 shutdown_connections 2024-04-17T10:01:23.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.593+0000 7f35ba000640 1 -- 172.21.15.32:0/3170289551 wait complete. 2024-04-17T10:01:23.612 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:01:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.757+0000 7f81cac00640 1 Processor -- start 2024-04-17T10:01:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.757+0000 7f81cac00640 1 -- start start 2024-04-17T10:01:23.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.761+0000 7f81cac00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81c406ba30 0x7f81c406be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:23.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.761+0000 7f81cac00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f81c4071e60 0x7f81c40722b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:23.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.761+0000 7f81cac00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81c406c350 con 0x7f81c40cf370 2024-04-17T10:01:23.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.761+0000 7f81cac00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f81c40cfef0 con 0x7f81c4071e60 2024-04-17T10:01:23.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.761+0000 7f81cac00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f81c40d0130 con 0x7f81c406ba30 2024-04-17T10:01:23.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.761+0000 7f81bb400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f81c40cf370 0x7f81c40cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50372/0 (socket says 172.21.15.32:50372) 2024-04-17T10:01:23.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.761+0000 7f81bb400640 1 -- 172.21.15.32:0/3825005798 learned_addr learned my addr 172.21.15.32:0/3825005798 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:23.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.761+0000 7f81c8c00640 1 --2- 172.21.15.32:0/3825005798 >> v2:172.21.15.144:3300/0 conn(0x7f81c4071e60 0x7f81c40722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:23.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.761+0000 7f81c3e00640 1 --2- 172.21.15.32:0/3825005798 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81c406ba30 0x7f81c406be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:23.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.761+0000 7f81c3400640 1 -- 172.21.15.32:0/3825005798 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3771720724 0 0) 0x7f81c406c350 con 0x7f81c40cf370 2024-04-17T10:01:23.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.761+0000 7f81c3400640 1 -- 172.21.15.32:0/3825005798 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81a8003530 con 0x7f81c40cf370 2024-04-17T10:01:23.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.761+0000 7f81c8c00640 1 -- 172.21.15.32:0/3825005798 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81c406ba30 msgr2=0x7f81c406be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:23.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.761+0000 7f81c8c00640 1 --2- 172.21.15.32:0/3825005798 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81c406ba30 0x7f81c406be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.761+0000 7f81c8c00640 1 -- 172.21.15.32:0/3825005798 >> v1:172.21.15.32:6789/0 conn(0x7f81c40cf370 legacy=0x7f81c40cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:23.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.761+0000 7f81c8c00640 1 -- 172.21.15.32:0/3825005798 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c415d090 con 0x7f81c4071e60 2024-04-17T10:01:23.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.765+0000 7f81c8c00640 1 --2- 172.21.15.32:0/3825005798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81c4071e60 0x7f81c40722b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f81bc009d90 tx=0x7f81bc01c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58a84b103f1085cf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:23.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.765+0000 7f81c3400640 1 -- 172.21.15.32:0/3825005798 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc0b2070 con 0x7f81c4071e60 2024-04-17T10:01:23.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.765+0000 7f81c3400640 1 -- 172.21.15.32:0/3825005798 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81bc0a5a00 con 0x7f81c4071e60 2024-04-17T10:01:23.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.765+0000 7f81c3400640 1 -- 172.21.15.32:0/3825005798 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc0a5dd0 con 0x7f81c4071e60 2024-04-17T10:01:23.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.765+0000 7f81cac00640 1 -- 172.21.15.32:0/3825005798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81c4071e60 msgr2=0x7f81c40722b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:23.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.765+0000 7f81cac00640 1 --2- 172.21.15.32:0/3825005798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81c4071e60 0x7f81c40722b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f81bc009d90 tx=0x7f81bc01c940 comp rx=0 tx=0).stop 2024-04-17T10:01:23.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.765+0000 7f81cac00640 1 -- 172.21.15.32:0/3825005798 shutdown_connections 2024-04-17T10:01:23.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.765+0000 7f81cac00640 1 --2- 172.21.15.32:0/3825005798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81c4071e60 0x7f81c40722b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.765+0000 7f81cac00640 1 --2- 172.21.15.32:0/3825005798 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81c406ba30 0x7f81c406be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.765+0000 7f81cac00640 1 -- 172.21.15.32:0/3825005798 >> 172.21.15.32:0/3825005798 conn(0x7f81c40b2120 msgr2=0x7f81c40b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:23.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.765+0000 7f81cac00640 1 -- 172.21.15.32:0/3825005798 shutdown_connections 2024-04-17T10:01:23.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.765+0000 7f81cac00640 1 -- 172.21.15.32:0/3825005798 wait complete. 2024-04-17T10:01:23.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.765+0000 7f81cac00640 1 Processor -- start 2024-04-17T10:01:23.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.765+0000 7f81cac00640 1 -- start start 2024-04-17T10:01:23.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81cac00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81c406ba30 0x7f81c4159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:23.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81cac00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81c4071e60 0x7f81c41fabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:23.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81cac00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81c40cf370 0x7f81c41fb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:23.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81cac00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f81c415dcf0 con 0x7f81c40cf370 2024-04-17T10:01:23.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81cac00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f81c415db70 con 0x7f81c4071e60 2024-04-17T10:01:23.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81cac00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f81c4146230 con 0x7f81c406ba30 2024-04-17T10:01:23.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81c8c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81c40cf370 0x7f81c41fb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:23.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81c8c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81c40cf370 0x7f81c41fb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53388/0 (socket says 172.21.15.32:53388) 2024-04-17T10:01:23.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81c8c00640 1 -- 172.21.15.32:0/2279726447 learned_addr learned my addr 172.21.15.32:0/2279726447 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:23.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81bb400640 1 --2- 172.21.15.32:0/2279726447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81c4071e60 0x7f81c41fabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:23.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81c3e00640 1 --2- 172.21.15.32:0/2279726447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81c406ba30 0x7f81c4159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:23.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81c8c00640 1 -- 172.21.15.32:0/2279726447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81c406ba30 msgr2=0x7f81c4159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:23.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81c8c00640 1 --2- 172.21.15.32:0/2279726447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81c406ba30 0x7f81c4159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81c8c00640 1 -- 172.21.15.32:0/2279726447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81c4071e60 msgr2=0x7f81c41fabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:23.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81c8c00640 1 --2- 172.21.15.32:0/2279726447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81c4071e60 0x7f81c41fabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81c8c00640 1 -- 172.21.15.32:0/2279726447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c40b4fc0 con 0x7f81c40cf370 2024-04-17T10:01:23.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81c8c00640 1 --2- 172.21.15.32:0/2279726447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81c40cf370 0x7f81c41fb100 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f81bc01aff0 tx=0x7f81bc0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:23.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81c0c00640 1 -- 172.21.15.32:0/2279726447 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc0b2050 con 0x7f81c40cf370 2024-04-17T10:01:23.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81c0c00640 1 -- 172.21.15.32:0/2279726447 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81bc0be070 con 0x7f81c40cf370 2024-04-17T10:01:23.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81cac00640 1 -- 172.21.15.32:0/2279726447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c40b52a0 con 0x7f81c40cf370 2024-04-17T10:01:23.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81c0c00640 1 -- 172.21.15.32:0/2279726447 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc0ada50 con 0x7f81c40cf370 2024-04-17T10:01:23.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81cac00640 1 -- 172.21.15.32:0/2279726447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c415a250 con 0x7f81c40cf370 2024-04-17T10:01:23.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81cac00640 1 -- 172.21.15.32:0/2279726447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c4068a90 con 0x7f81c40cf370 2024-04-17T10:01:23.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.769+0000 7f81c0c00640 1 -- 172.21.15.32:0/2279726447 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f81bc0adbf0 con 0x7f81c40cf370 2024-04-17T10:01:23.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.773+0000 7f81c0c00640 1 --2- 172.21.15.32:0/2279726447 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f81a0068380 0x7f81a006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:23.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.773+0000 7f81c0c00640 1 -- 172.21.15.32:0/2279726447 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f81bc121ed0 con 0x7f81c40cf370 2024-04-17T10:01:23.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.773+0000 7f81c3e00640 1 --2- 172.21.15.32:0/2279726447 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f81a0068380 0x7f81a006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:23.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.773+0000 7f81c3e00640 1 --2- 172.21.15.32:0/2279726447 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f81a0068380 0x7f81a006a840 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f81c4145c50 tx=0x7f81b4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:23.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.777+0000 7f81c0c00640 1 -- 172.21.15.32:0/2279726447 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f81bc016560 con 0x7f81c40cf370 2024-04-17T10:01:23.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.941+0000 7f81cac00640 1 -- 172.21.15.32:0/2279726447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f81c406c850 con 0x7f81c40cf370 2024-04-17T10:01:23.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.941+0000 7f81c0c00640 1 -- 172.21.15.32:0/2279726447 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) ==== 104+0+11 (secure 0 0 0) 0x7f81bc0f2390 con 0x7f81c40cf370 2024-04-17T10:01:23.941 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T10:01:23.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.945+0000 7f81cac00640 1 -- 172.21.15.32:0/2279726447 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f81a0068380 msgr2=0x7f81a006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:23.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.945+0000 7f81cac00640 1 --2- 172.21.15.32:0/2279726447 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f81a0068380 0x7f81a006a840 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f81c4145c50 tx=0x7f81b4008040 comp rx=0 tx=0).stop 2024-04-17T10:01:23.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.945+0000 7f81cac00640 1 -- 172.21.15.32:0/2279726447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81c40cf370 msgr2=0x7f81c41fb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:23.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.945+0000 7f81cac00640 1 --2- 172.21.15.32:0/2279726447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81c40cf370 0x7f81c41fb100 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f81bc01aff0 tx=0x7f81bc0196e0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.945+0000 7f81cac00640 1 -- 172.21.15.32:0/2279726447 shutdown_connections 2024-04-17T10:01:23.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.945+0000 7f81cac00640 1 --2- 172.21.15.32:0/2279726447 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f81a0068380 0x7f81a006a840 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.945+0000 7f81cac00640 1 --2- 172.21.15.32:0/2279726447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81c40cf370 0x7f81c41fb100 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.945+0000 7f81cac00640 1 --2- 172.21.15.32:0/2279726447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81c4071e60 0x7f81c41fabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.945+0000 7f81cac00640 1 --2- 172.21.15.32:0/2279726447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81c406ba30 0x7f81c4159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:23.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.945+0000 7f81cac00640 1 -- 172.21.15.32:0/2279726447 >> 172.21.15.32:0/2279726447 conn(0x7f81c40b2120 msgr2=0x7f81c4165890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:23.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.945+0000 7f81cac00640 1 -- 172.21.15.32:0/2279726447 shutdown_connections 2024-04-17T10:01:23.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:23.945+0000 7f81cac00640 1 -- 172.21.15.32:0/2279726447 wait complete. 2024-04-17T10:01:23.963 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:01:24.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.109+0000 7fe814200640 1 Processor -- start 2024-04-17T10:01:24.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.109+0000 7fe814200640 1 -- start start 2024-04-17T10:01:24.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.109+0000 7fe814200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe80c0cf370 0x7fe80c0cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:24.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.109+0000 7fe814200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80c071e60 0x7fe80c072270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:24.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.109+0000 7fe814200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe80c0d0130 con 0x7fe80c06ba30 2024-04-17T10:01:24.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.109+0000 7fe814200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe80c15d090 con 0x7fe80c0cf370 2024-04-17T10:01:24.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.109+0000 7fe814200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe80c15d370 con 0x7fe80c071e60 2024-04-17T10:01:24.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.109+0000 7fe810e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe80c0cf370 0x7fe80c0cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.109+0000 7fe812200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80c071e60 0x7fe80c072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.109+0000 7fe812200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80c071e60 0x7fe80c072270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:35880/0 (socket says 172.21.15.32:35880) 2024-04-17T10:01:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.109+0000 7fe812200640 1 -- 172.21.15.32:0/2445488193 learned_addr learned my addr 172.21.15.32:0/2445488193 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.109+0000 7fe811800640 1 --1- 172.21.15.32:0/2445488193 >> v1:172.21.15.144:6789/0 conn(0x7fe80c06ba30 0x7fe80c06be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:44798/0 (socket says 172.21.15.32:44798) 2024-04-17T10:01:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.113+0000 7fe812200640 1 -- 172.21.15.32:0/2445488193 >> v2:172.21.15.144:3300/0 conn(0x7fe80c0cf370 msgr2=0x7fe80c0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.113+0000 7fe812200640 1 --2- 172.21.15.32:0/2445488193 >> v2:172.21.15.144:3300/0 conn(0x7fe80c0cf370 0x7fe80c0cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.113+0000 7fe812200640 1 -- 172.21.15.32:0/2445488193 >> v1:172.21.15.144:6789/0 conn(0x7fe80c06ba30 legacy=0x7fe80c06be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.113+0000 7fe812200640 1 -- 172.21.15.32:0/2445488193 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80c15d650 con 0x7fe80c071e60 2024-04-17T10:01:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.113+0000 7fe812200640 1 --2- 172.21.15.32:0/2445488193 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80c071e60 0x7fe80c072270 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe804009e50 tx=0x7fe80401ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78ca16ea5426636e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.113+0000 7fe80be00640 1 -- 172.21.15.32:0/2445488193 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8040a5920 con 0x7fe80c071e60 2024-04-17T10:01:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.113+0000 7fe80be00640 1 -- 172.21.15.32:0/2445488193 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8040a5ac0 con 0x7fe80c071e60 2024-04-17T10:01:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.113+0000 7fe80be00640 1 -- 172.21.15.32:0/2445488193 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8040ad3f0 con 0x7fe80c071e60 2024-04-17T10:01:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.113+0000 7fe814200640 1 -- 172.21.15.32:0/2445488193 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80c071e60 msgr2=0x7fe80c072270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.113+0000 7fe814200640 1 --2- 172.21.15.32:0/2445488193 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80c071e60 0x7fe80c072270 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe804009e50 tx=0x7fe80401ca30 comp rx=0 tx=0).stop 2024-04-17T10:01:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe814200640 1 -- 172.21.15.32:0/2445488193 shutdown_connections 2024-04-17T10:01:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe814200640 1 --2- 172.21.15.32:0/2445488193 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80c071e60 0x7fe80c072270 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe814200640 1 --2- 172.21.15.32:0/2445488193 >> v2:172.21.15.144:3300/0 conn(0x7fe80c0cf370 0x7fe80c0cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe814200640 1 -- 172.21.15.32:0/2445488193 >> 172.21.15.32:0/2445488193 conn(0x7fe80c0b2120 msgr2=0x7fe80c0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe814200640 1 -- 172.21.15.32:0/2445488193 shutdown_connections 2024-04-17T10:01:24.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe814200640 1 -- 172.21.15.32:0/2445488193 wait complete. 2024-04-17T10:01:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe814200640 1 Processor -- start 2024-04-17T10:01:24.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe814200640 1 -- start start 2024-04-17T10:01:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe814200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80c06ba30 0x7fe80c129e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe814200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe80c071e60 0x7fe80c1f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe814200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe80c0cf370 0x7fe80c13f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe814200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c146230 con 0x7fe80c0cf370 2024-04-17T10:01:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe814200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c15dd80 con 0x7fe80c071e60 2024-04-17T10:01:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe814200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe80c1463b0 con 0x7fe80c06ba30 2024-04-17T10:01:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe812200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe80c0cf370 0x7fe80c13f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe810e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe80c071e60 0x7fe80c1f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe812200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe80c0cf370 0x7fe80c13f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53396/0 (socket says 172.21.15.32:53396) 2024-04-17T10:01:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe812200640 1 -- 172.21.15.32:0/1469428693 learned_addr learned my addr 172.21.15.32:0/1469428693 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe811800640 1 --2- 172.21.15.32:0/1469428693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80c06ba30 0x7fe80c129e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe812200640 1 -- 172.21.15.32:0/1469428693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80c06ba30 msgr2=0x7fe80c129e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe812200640 1 --2- 172.21.15.32:0/1469428693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80c06ba30 0x7fe80c129e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe812200640 1 -- 172.21.15.32:0/1469428693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe80c071e60 msgr2=0x7fe80c1f7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe812200640 1 --2- 172.21.15.32:0/1469428693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe80c071e60 0x7fe80c1f7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe812200640 1 -- 172.21.15.32:0/1469428693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80c068300 con 0x7fe80c0cf370 2024-04-17T10:01:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.117+0000 7fe812200640 1 --2- 172.21.15.32:0/1469428693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe80c0cf370 0x7fe80c13f830 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fe8040af040 tx=0x7fe8040197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.121+0000 7fe809600640 1 -- 172.21.15.32:0/1469428693 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8040a5920 con 0x7fe80c0cf370 2024-04-17T10:01:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.121+0000 7fe809600640 1 -- 172.21.15.32:0/1469428693 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8040be070 con 0x7fe80c0cf370 2024-04-17T10:01:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.121+0000 7fe809600640 1 -- 172.21.15.32:0/1469428693 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804009510 con 0x7fe80c0cf370 2024-04-17T10:01:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.121+0000 7fe814200640 1 -- 172.21.15.32:0/1469428693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe80c13db40 con 0x7fe80c0cf370 2024-04-17T10:01:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.121+0000 7fe814200640 1 -- 172.21.15.32:0/1469428693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe80c0aed20 con 0x7fe80c0cf370 2024-04-17T10:01:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.121+0000 7fe814200640 1 -- 172.21.15.32:0/1469428693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe80c068a90 con 0x7fe80c0cf370 2024-04-17T10:01:24.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.121+0000 7fe809600640 1 -- 172.21.15.32:0/1469428693 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe8040039a0 con 0x7fe80c0cf370 2024-04-17T10:01:24.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.121+0000 7fe809600640 1 --2- 172.21.15.32:0/1469428693 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe7ec0683b0 0x7fe7ec06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:24.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.121+0000 7fe809600640 1 -- 172.21.15.32:0/1469428693 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7fe8041219d0 con 0x7fe80c0cf370 2024-04-17T10:01:24.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.121+0000 7fe811800640 1 --2- 172.21.15.32:0/1469428693 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe7ec0683b0 0x7fe7ec06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:24.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.121+0000 7fe811800640 1 --2- 172.21.15.32:0/1469428693 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe7ec0683b0 0x7fe7ec06a870 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe800000f70 tx=0x7fe800007e30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:24.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.125+0000 7fe809600640 1 -- 172.21.15.32:0/1469428693 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fe8040c7020 con 0x7fe80c0cf370 2024-04-17T10:01:24.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.289+0000 7fe814200640 1 -- 172.21.15.32:0/1469428693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe80c0b2600 con 0x7fe80c0cf370 2024-04-17T10:01:24.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.289+0000 7fe809600640 1 -- 172.21.15.32:0/1469428693 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) ==== 100+0+11 (secure 0 0 0) 0x7fe804016620 con 0x7fe80c0cf370 2024-04-17T10:01:24.288 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T10:01:24.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.293+0000 7fe814200640 1 -- 172.21.15.32:0/1469428693 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe7ec0683b0 msgr2=0x7fe7ec06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:24.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.293+0000 7fe814200640 1 --2- 172.21.15.32:0/1469428693 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe7ec0683b0 0x7fe7ec06a870 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe800000f70 tx=0x7fe800007e30 comp rx=0 tx=0).stop 2024-04-17T10:01:24.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.293+0000 7fe814200640 1 -- 172.21.15.32:0/1469428693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe80c0cf370 msgr2=0x7fe80c13f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:24.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.293+0000 7fe814200640 1 --2- 172.21.15.32:0/1469428693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe80c0cf370 0x7fe80c13f830 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fe8040af040 tx=0x7fe8040197d0 comp rx=0 tx=0).stop 2024-04-17T10:01:24.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.293+0000 7fe814200640 1 -- 172.21.15.32:0/1469428693 shutdown_connections 2024-04-17T10:01:24.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.293+0000 7fe814200640 1 --2- 172.21.15.32:0/1469428693 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe7ec0683b0 0x7fe7ec06a870 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:24.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.293+0000 7fe814200640 1 --2- 172.21.15.32:0/1469428693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe80c0cf370 0x7fe80c13f830 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:24.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.293+0000 7fe814200640 1 --2- 172.21.15.32:0/1469428693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe80c071e60 0x7fe80c1f7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:24.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.293+0000 7fe814200640 1 --2- 172.21.15.32:0/1469428693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80c06ba30 0x7fe80c129e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:24.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.293+0000 7fe814200640 1 -- 172.21.15.32:0/1469428693 >> 172.21.15.32:0/1469428693 conn(0x7fe80c0b2120 msgr2=0x7fe80c0aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:24.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.293+0000 7fe814200640 1 -- 172.21.15.32:0/1469428693 shutdown_connections 2024-04-17T10:01:24.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.293+0000 7fe814200640 1 -- 172.21.15.32:0/1469428693 wait complete. 2024-04-17T10:01:24.309 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:01:24.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.453+0000 7fa539200640 1 Processor -- start 2024-04-17T10:01:24.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.453+0000 7fa539200640 1 -- start start 2024-04-17T10:01:24.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.453+0000 7fa539200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa53413f6e0 0x7fa5340cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:24.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.457+0000 7fa539200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5340cf370 0x7fa5340d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:24.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.457+0000 7fa539200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5340d0820 0x7fa534141d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:24.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.457+0000 7fa539200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa53413fac0 con 0x7fa5340cf370 2024-04-17T10:01:24.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.457+0000 7fa539200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa5341601e0 con 0x7fa53413f6e0 2024-04-17T10:01:24.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.457+0000 7fa539200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa534160480 con 0x7fa5340d0820 2024-04-17T10:01:24.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.457+0000 7fa52be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5340cf370 0x7fa5340d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:24.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.457+0000 7fa52be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5340cf370 0x7fa5340d02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53410/0 (socket says 172.21.15.32:53410) 2024-04-17T10:01:24.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.457+0000 7fa52be00640 1 -- 172.21.15.32:0/2114691550 learned_addr learned my addr 172.21.15.32:0/2114691550 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:24.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.457+0000 7fa532800640 1 --2- 172.21.15.32:0/2114691550 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5340d0820 0x7fa534141d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:24.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.457+0000 7fa531e00640 1 --2- 172.21.15.32:0/2114691550 >> v2:172.21.15.144:3300/0 conn(0x7fa53413f6e0 0x7fa5340cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:24.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.457+0000 7fa52be00640 1 -- 172.21.15.32:0/2114691550 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5340d0820 msgr2=0x7fa534141d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:24.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.457+0000 7fa52be00640 1 --2- 172.21.15.32:0/2114691550 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5340d0820 0x7fa534141d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:24.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.457+0000 7fa52be00640 1 -- 172.21.15.32:0/2114691550 >> v2:172.21.15.144:3300/0 conn(0x7fa53413f6e0 msgr2=0x7fa5340cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:24.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.457+0000 7fa52be00640 1 --2- 172.21.15.32:0/2114691550 >> v2:172.21.15.144:3300/0 conn(0x7fa53413f6e0 0x7fa5340cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:24.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.457+0000 7fa52be00640 1 -- 172.21.15.32:0/2114691550 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa534160760 con 0x7fa5340cf370 2024-04-17T10:01:24.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.457+0000 7fa52be00640 1 --2- 172.21.15.32:0/2114691550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5340cf370 0x7fa5340d02e0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fa5340b3100 tx=0x7fa51c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae008e3714f20092 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:24.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.457+0000 7fa531400640 1 -- 172.21.15.32:0/2114691550 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0b1070 con 0x7fa5340cf370 2024-04-17T10:01:24.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa531400640 1 -- 172.21.15.32:0/2114691550 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa51c0ac630 con 0x7fa5340cf370 2024-04-17T10:01:24.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa531400640 1 -- 172.21.15.32:0/2114691550 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0a8050 con 0x7fa5340cf370 2024-04-17T10:01:24.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa539200640 1 -- 172.21.15.32:0/2114691550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5340cf370 msgr2=0x7fa5340d02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:24.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa539200640 1 --2- 172.21.15.32:0/2114691550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5340cf370 0x7fa5340d02e0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fa5340b3100 tx=0x7fa51c01c520 comp rx=0 tx=0).stop 2024-04-17T10:01:24.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa539200640 1 -- 172.21.15.32:0/2114691550 shutdown_connections 2024-04-17T10:01:24.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa539200640 1 --2- 172.21.15.32:0/2114691550 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5340d0820 0x7fa534141d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:24.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa539200640 1 --2- 172.21.15.32:0/2114691550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5340cf370 0x7fa5340d02e0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:24.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa539200640 1 --2- 172.21.15.32:0/2114691550 >> v2:172.21.15.144:3300/0 conn(0x7fa53413f6e0 0x7fa5340cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:24.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa539200640 1 -- 172.21.15.32:0/2114691550 >> 172.21.15.32:0/2114691550 conn(0x7fa53406ba30 msgr2=0x7fa53406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:24.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa539200640 1 -- 172.21.15.32:0/2114691550 shutdown_connections 2024-04-17T10:01:24.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa539200640 1 -- 172.21.15.32:0/2114691550 wait complete. 2024-04-17T10:01:24.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa539200640 1 Processor -- start 2024-04-17T10:01:24.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa539200640 1 -- start start 2024-04-17T10:01:24.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa539200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5340cf370 0x7fa534143ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:24.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa539200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5340d0820 0x7fa534144980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:24.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa539200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa53413f6e0 0x7fa5340b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:24.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa539200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa5340cff10 con 0x7fa5340cf370 2024-04-17T10:01:24.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa539200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa5340cfd90 con 0x7fa53413f6e0 2024-04-17T10:01:24.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa539200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa534071d80 con 0x7fa5340d0820 2024-04-17T10:01:24.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa531e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5340cf370 0x7fa534143ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa531e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5340cf370 0x7fa534143ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53418/0 (socket says 172.21.15.32:53418) 2024-04-17T10:01:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa531e00640 1 -- 172.21.15.32:0/1437108241 learned_addr learned my addr 172.21.15.32:0/1437108241 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa532800640 1 --2- 172.21.15.32:0/1437108241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa53413f6e0 0x7fa5340b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa52be00640 1 --2- 172.21.15.32:0/1437108241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5340d0820 0x7fa534144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa531e00640 1 -- 172.21.15.32:0/1437108241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5340d0820 msgr2=0x7fa534144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa531e00640 1 --2- 172.21.15.32:0/1437108241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5340d0820 0x7fa534144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa531e00640 1 -- 172.21.15.32:0/1437108241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa53413f6e0 msgr2=0x7fa5340b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa531e00640 1 --2- 172.21.15.32:0/1437108241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa53413f6e0 0x7fa5340b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa531e00640 1 -- 172.21.15.32:0/1437108241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5340b26a0 con 0x7fa5340cf370 2024-04-17T10:01:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.461+0000 7fa52be00640 1 --2- 172.21.15.32:0/1437108241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5340d0820 0x7fa534144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.465+0000 7fa531e00640 1 --2- 172.21.15.32:0/1437108241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5340cf370 0x7fa534143ad0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fa52400e870 tx=0x7fa52400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.465+0000 7fa52a000640 1 -- 172.21.15.32:0/1437108241 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524017070 con 0x7fa5340cf370 2024-04-17T10:01:24.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.465+0000 7fa52a000640 1 -- 172.21.15.32:0/1437108241 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa52400c940 con 0x7fa5340cf370 2024-04-17T10:01:24.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.465+0000 7fa52a000640 1 -- 172.21.15.32:0/1437108241 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52400ccf0 con 0x7fa5340cf370 2024-04-17T10:01:24.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.465+0000 7fa539200640 1 -- 172.21.15.32:0/1437108241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa534142ff0 con 0x7fa5340cf370 2024-04-17T10:01:24.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.465+0000 7fa539200640 1 -- 172.21.15.32:0/1437108241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa534144010 con 0x7fa5340cf370 2024-04-17T10:01:24.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.465+0000 7fa539200640 1 -- 172.21.15.32:0/1437108241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa534142de0 con 0x7fa5340cf370 2024-04-17T10:01:24.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.465+0000 7fa52a000640 1 -- 172.21.15.32:0/1437108241 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa52401a070 con 0x7fa5340cf370 2024-04-17T10:01:24.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.465+0000 7fa52a000640 1 --2- 172.21.15.32:0/1437108241 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa510068380 0x7fa51006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:24.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.465+0000 7fa52a000640 1 -- 172.21.15.32:0/1437108241 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7fa524086980 con 0x7fa5340cf370 2024-04-17T10:01:24.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.469+0000 7fa52be00640 1 --2- 172.21.15.32:0/1437108241 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa510068380 0x7fa51006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:24.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.469+0000 7fa52be00640 1 --2- 172.21.15.32:0/1437108241 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa510068380 0x7fa51006a840 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0ae000 tx=0x7fa51c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:24.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.469+0000 7fa52a000640 1 -- 172.21.15.32:0/1437108241 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa5240566d0 con 0x7fa5340cf370 2024-04-17T10:01:24.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.649+0000 7fa539200640 1 -- 172.21.15.32:0/1437108241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa534068a90 con 0x7fa5340cf370 2024-04-17T10:01:24.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.649+0000 7fa52a000640 1 -- 172.21.15.32:0/1437108241 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v86) ==== 80+0+244 (secure 0 0 0) 0x7fa52405a9d0 con 0x7fa5340cf370 2024-04-17T10:01:24.646 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6837/1253497636 2024-04-18T10:00:43.842876+0000 2024-04-17T10:01:24.646 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6835/1253497636 2024-04-18T10:00:43.842876+0000 2024-04-17T10:01:24.646 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6837/2656735817 2024-04-18T09:59:28.985107+0000 2024-04-17T10:01:24.646 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6835/2656735817 2024-04-18T09:59:28.985107+0000 2024-04-17T10:01:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:listed 4 entries 2024-04-17T10:01:24.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.649+0000 7fa539200640 1 -- 172.21.15.32:0/1437108241 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa510068380 msgr2=0x7fa51006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:24.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.649+0000 7fa539200640 1 --2- 172.21.15.32:0/1437108241 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa510068380 0x7fa51006a840 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0ae000 tx=0x7fa51c008040 comp rx=0 tx=0).stop 2024-04-17T10:01:24.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.649+0000 7fa539200640 1 -- 172.21.15.32:0/1437108241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5340cf370 msgr2=0x7fa534143ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:24.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.649+0000 7fa539200640 1 --2- 172.21.15.32:0/1437108241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5340cf370 0x7fa534143ad0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fa52400e870 tx=0x7fa52400ec70 comp rx=0 tx=0).stop 2024-04-17T10:01:24.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.649+0000 7fa539200640 1 -- 172.21.15.32:0/1437108241 shutdown_connections 2024-04-17T10:01:24.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.649+0000 7fa539200640 1 --2- 172.21.15.32:0/1437108241 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa510068380 0x7fa51006a840 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:24.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.649+0000 7fa539200640 1 --2- 172.21.15.32:0/1437108241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5340d0820 0x7fa534144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:24.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.649+0000 7fa539200640 1 --2- 172.21.15.32:0/1437108241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa53413f6e0 0x7fa5340b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:24.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.649+0000 7fa539200640 1 --2- 172.21.15.32:0/1437108241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5340cf370 0x7fa534143ad0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:24.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.649+0000 7fa539200640 1 -- 172.21.15.32:0/1437108241 >> 172.21.15.32:0/1437108241 conn(0x7fa53406ba30 msgr2=0x7fa534072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:24.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.649+0000 7fa539200640 1 -- 172.21.15.32:0/1437108241 shutdown_connections 2024-04-17T10:01:24.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:24.649+0000 7fa539200640 1 -- 172.21.15.32:0/1437108241 wait complete. 2024-04-17T10:01:24.666 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2024-04-17T10:01:24.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:01:24.666 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T10:01:24.684 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[46849]: fuse finished with error 0 and tester_r 0 2024-04-17T10:01:24.687 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T10:01:24.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:01:24.687 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T10:01:24.835 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T10:01:24.835 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T10:01:24.835 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:01:24.845 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:01:24.845 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T10:01:24.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:01:24.845 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-17T10:01:24.896 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-17T10:01:24.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:01:24.897 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T10:01:24.897 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:01:24.897 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:01:24.906 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:01:24.906 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T10:01:24.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:01:24.906 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-17T10:01:24.956 INFO:teuthology.orchestra.run.smithi032.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-17T10:01:24.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:01:24.957 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T10:01:24.957 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T10:01:24.957 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:01:24.966 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:01:24.966 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T10:01:24.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:01:24.966 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-17T10:01:25.015 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-17T10:01:25.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:01:25.017 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:01:25.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.161+0000 7fa68d200640 1 Processor -- start 2024-04-17T10:01:25.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.161+0000 7fa68d200640 1 -- start start 2024-04-17T10:01:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.161+0000 7fa68d200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa68815eb10 0x7fa68815f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.161+0000 7fa68d200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6880b2550 0x7fa6880d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.161+0000 7fa68d200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6880b2ac0 con 0x7fa68806c640 2024-04-17T10:01:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.161+0000 7fa68d200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa68815ff10 con 0x7fa68815eb10 2024-04-17T10:01:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.161+0000 7fa68d200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa6880b2da0 con 0x7fa6880b2550 2024-04-17T10:01:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.161+0000 7fa686800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa68806c640 0x7fa6881603c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50382/0 (socket says 172.21.15.32:50382) 2024-04-17T10:01:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.161+0000 7fa686800640 1 -- 172.21.15.32:0/283796049 learned_addr learned my addr 172.21.15.32:0/283796049 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:25.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.161+0000 7fa67d400640 1 --2- 172.21.15.32:0/283796049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6880b2550 0x7fa6880d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:25.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.161+0000 7fa685e00640 1 --2- 172.21.15.32:0/283796049 >> v2:172.21.15.144:3300/0 conn(0x7fa68815eb10 0x7fa68815f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:25.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.161+0000 7fa685400640 1 -- 172.21.15.32:0/283796049 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4241671238 0 0) 0x7fa6880b2ac0 con 0x7fa68806c640 2024-04-17T10:01:25.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.161+0000 7fa685400640 1 -- 172.21.15.32:0/283796049 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa66c003530 con 0x7fa68806c640 2024-04-17T10:01:25.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.161+0000 7fa685400640 1 -- 172.21.15.32:0/283796049 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1008944525 0 0) 0x7fa66c003530 con 0x7fa68806c640 2024-04-17T10:01:25.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.161+0000 7fa685400640 1 -- 172.21.15.32:0/283796049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6880b2550 msgr2=0x7fa6880d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:25.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.161+0000 7fa685400640 1 --2- 172.21.15.32:0/283796049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6880b2550 0x7fa6880d0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.161+0000 7fa685400640 1 -- 172.21.15.32:0/283796049 >> v2:172.21.15.144:3300/0 conn(0x7fa68815eb10 msgr2=0x7fa68815f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:25.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.161+0000 7fa685400640 1 --2- 172.21.15.32:0/283796049 >> v2:172.21.15.144:3300/0 conn(0x7fa68815eb10 0x7fa68815f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.161+0000 7fa685400640 1 -- 172.21.15.32:0/283796049 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68806bb60 con 0x7fa68806c640 2024-04-17T10:01:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.161+0000 7fa685400640 1 -- 172.21.15.32:0/283796049 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7fa680001680 con 0x7fa68806c640 2024-04-17T10:01:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.165+0000 7fa685400640 1 -- 172.21.15.32:0/283796049 >> v1:172.21.15.32:6789/0 conn(0x7fa68806c640 legacy=0x7fa6881603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.165+0000 7fa685400640 1 --2- 172.21.15.32:0/283796049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66c0039c0 0x7fa66c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.165+0000 7fa685400640 1 --2- 172.21.15.32:0/283796049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6880b2550 0x7fa66c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.165+0000 7fa685400640 1 --2- 172.21.15.32:0/283796049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa66c004d50 0x7fa66c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.165+0000 7fa685400640 1 -- 172.21.15.32:0/283796049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa68815ff10 con 0x7fa66c0039c0 2024-04-17T10:01:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.165+0000 7fa685400640 1 -- 172.21.15.32:0/283796049 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa6880b2da0 con 0x7fa66c004d50 2024-04-17T10:01:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.165+0000 7fa685400640 1 -- 172.21.15.32:0/283796049 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa66c00ba80 con 0x7fa6880b2550 2024-04-17T10:01:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.165+0000 7fa685e00640 1 --2- 172.21.15.32:0/283796049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66c0039c0 0x7fa66c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.165+0000 7fa67d400640 1 --2- 172.21.15.32:0/283796049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6880b2550 0x7fa66c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.165+0000 7fa686800640 1 --2- 172.21.15.32:0/283796049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa66c004d50 0x7fa66c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.165+0000 7fa685e00640 1 -- 172.21.15.32:0/283796049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6880b2550 msgr2=0x7fa66c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.165+0000 7fa685e00640 1 --2- 172.21.15.32:0/283796049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6880b2550 0x7fa66c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.165+0000 7fa685e00640 1 -- 172.21.15.32:0/283796049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa66c004d50 msgr2=0x7fa66c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.165+0000 7fa685e00640 1 --2- 172.21.15.32:0/283796049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa66c004d50 0x7fa66c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.165+0000 7fa685e00640 1 -- 172.21.15.32:0/283796049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66c003530 con 0x7fa66c0039c0 2024-04-17T10:01:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.165+0000 7fa67d400640 1 --2- 172.21.15.32:0/283796049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6880b2550 0x7fa66c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.165+0000 7fa686800640 1 --2- 172.21.15.32:0/283796049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa66c004d50 0x7fa66c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:25.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.165+0000 7fa685e00640 1 --2- 172.21.15.32:0/283796049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66c0039c0 0x7fa66c003de0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fa678009ae0 tx=0x7fa67801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9044a422d3d7baa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:25.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.165+0000 7fa685400640 1 -- 172.21.15.32:0/283796049 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6780b2070 con 0x7fa66c0039c0 2024-04-17T10:01:25.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.165+0000 7fa685400640 1 -- 172.21.15.32:0/283796049 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6780a5570 con 0x7fa66c0039c0 2024-04-17T10:01:25.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.165+0000 7fa685400640 1 -- 172.21.15.32:0/283796049 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6780096b0 con 0x7fa66c0039c0 2024-04-17T10:01:25.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa68d200640 1 -- 172.21.15.32:0/283796049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66c0039c0 msgr2=0x7fa66c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:25.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa68d200640 1 --2- 172.21.15.32:0/283796049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66c0039c0 0x7fa66c003de0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fa678009ae0 tx=0x7fa67801cab0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa686800640 1 -- 172.21.15.32:0/283796049 reap_dead start 2024-04-17T10:01:25.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa68d200640 1 -- 172.21.15.32:0/283796049 shutdown_connections 2024-04-17T10:01:25.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa68d200640 1 -- 172.21.15.32:0/283796049 >> 172.21.15.32:0/283796049 conn(0x7fa68809a9f0 msgr2=0x7fa6880b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:25.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa68d200640 1 -- 172.21.15.32:0/283796049 shutdown_connections 2024-04-17T10:01:25.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa68d200640 1 -- 172.21.15.32:0/283796049 wait complete. 2024-04-17T10:01:25.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa68d200640 1 Processor -- start 2024-04-17T10:01:25.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa68d200640 1 -- start start 2024-04-17T10:01:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa68d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66c0039c0 0x7fa6881587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa68d200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6881ef060 0x7fa6880ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa68d200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa688158010 0x7fa6881635b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa68d200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa6880d0ba0 con 0x7fa66c0039c0 2024-04-17T10:01:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa68d200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa6880d0a20 con 0x7fa6881ef060 2024-04-17T10:01:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa68d200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa6880d0d20 con 0x7fa688158010 2024-04-17T10:01:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa685e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66c0039c0 0x7fa6881587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa685e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66c0039c0 0x7fa6881587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53446/0 (socket says 172.21.15.32:53446) 2024-04-17T10:01:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa685e00640 1 -- 172.21.15.32:0/3340362044 learned_addr learned my addr 172.21.15.32:0/3340362044 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa685e00640 1 -- 172.21.15.32:0/3340362044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa688158010 msgr2=0x7fa6881635b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa685e00640 1 --2- 172.21.15.32:0/3340362044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa688158010 0x7fa6881635b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa685e00640 1 -- 172.21.15.32:0/3340362044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6881ef060 msgr2=0x7fa6880ae3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa685e00640 1 --2- 172.21.15.32:0/3340362044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6881ef060 0x7fa6880ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa685e00640 1 -- 172.21.15.32:0/3340362044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6881ec550 con 0x7fa66c0039c0 2024-04-17T10:01:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.169+0000 7fa685e00640 1 --2- 172.21.15.32:0/3340362044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66c0039c0 0x7fa6881587f0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fa67801c8e0 tx=0x7fa6881612e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:25.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.173+0000 7fa67ea00640 1 -- 172.21.15.32:0/3340362044 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6780b2040 con 0x7fa66c0039c0 2024-04-17T10:01:25.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.173+0000 7fa67ea00640 1 -- 172.21.15.32:0/3340362044 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa67800f000 con 0x7fa66c0039c0 2024-04-17T10:01:25.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.173+0000 7fa67ea00640 1 -- 172.21.15.32:0/3340362044 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa678004020 con 0x7fa66c0039c0 2024-04-17T10:01:25.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.173+0000 7fa68d200640 1 -- 172.21.15.32:0/3340362044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6881ec830 con 0x7fa66c0039c0 2024-04-17T10:01:25.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.173+0000 7fa68d200640 1 -- 172.21.15.32:0/3340362044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6881ecdf0 con 0x7fa66c0039c0 2024-04-17T10:01:25.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.173+0000 7fa68d200640 1 -- 172.21.15.32:0/3340362044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa688068a90 con 0x7fa66c0039c0 2024-04-17T10:01:25.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.173+0000 7fa67ea00640 1 -- 172.21.15.32:0/3340362044 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa6780bf020 con 0x7fa66c0039c0 2024-04-17T10:01:25.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.173+0000 7fa67ea00640 1 --2- 172.21.15.32:0/3340362044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa664068380 0x7fa66406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:25.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.173+0000 7fa67ea00640 1 -- 172.21.15.32:0/3340362044 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7fa678120890 con 0x7fa66c0039c0 2024-04-17T10:01:25.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.173+0000 7fa67d400640 1 --2- 172.21.15.32:0/3340362044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa664068380 0x7fa66406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:25.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.177+0000 7fa67d400640 1 --2- 172.21.15.32:0/3340362044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa664068380 0x7fa66406a840 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa6880b1f40 tx=0x7fa670009750 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:25.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.177+0000 7fa67ea00640 1 -- 172.21.15.32:0/3340362044 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa6780167c0 con 0x7fa66c0039c0 2024-04-17T10:01:25.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.377+0000 7fa68d200640 1 -- 172.21.15.32:0/3340362044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6881655a0 con 0x7fa66c0039c0 2024-04-17T10:01:25.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.377+0000 7fa67ea00640 1 -- 172.21.15.32:0/3340362044 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5205 (secure 0 0 0) 0x7fa6780f0cc0 con 0x7fa66c0039c0 2024-04-17T10:01:25.376 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:25.376 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":72,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6419,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3534108103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3534108103},{"type":"v1","addr":"172.21.15.144:6835","nonce":3534108103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7003,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/935945019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":935945019},{"type":"v1","addr":"172.21.15.144:6834","nonce":935945019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3044362393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3044362393},{"type":"v1","addr":"172.21.15.32:6836","nonce":3044362393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:59:04.834110+0000","modified":"2024-04-17T10:00:47.860710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8196":{"gid":8196,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6837/2058904361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2058904361},{"type":"v1","addr":"172.21.15.32:6837","nonce":2058904361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8196,"qdb_cluster":[8196]},"id":6}]} 2024-04-17T10:01:25.376 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 72 2024-04-17T10:01:25.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.381+0000 7fa68d200640 1 -- 172.21.15.32:0/3340362044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa664068380 msgr2=0x7fa66406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:25.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.381+0000 7fa68d200640 1 --2- 172.21.15.32:0/3340362044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa664068380 0x7fa66406a840 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa6880b1f40 tx=0x7fa670009750 comp rx=0 tx=0).stop 2024-04-17T10:01:25.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.381+0000 7fa68d200640 1 -- 172.21.15.32:0/3340362044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66c0039c0 msgr2=0x7fa6881587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:25.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.381+0000 7fa68d200640 1 --2- 172.21.15.32:0/3340362044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66c0039c0 0x7fa6881587f0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fa67801c8e0 tx=0x7fa6881612e0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.381+0000 7fa68d200640 1 -- 172.21.15.32:0/3340362044 shutdown_connections 2024-04-17T10:01:25.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.381+0000 7fa68d200640 1 --2- 172.21.15.32:0/3340362044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa664068380 0x7fa66406a840 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.381+0000 7fa68d200640 1 --2- 172.21.15.32:0/3340362044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa688158010 0x7fa6881635b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.381+0000 7fa68d200640 1 --2- 172.21.15.32:0/3340362044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6881ef060 0x7fa6880ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.381+0000 7fa68d200640 1 --2- 172.21.15.32:0/3340362044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa66c0039c0 0x7fa6881587f0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.381+0000 7fa68d200640 1 -- 172.21.15.32:0/3340362044 >> 172.21.15.32:0/3340362044 conn(0x7fa68809a9f0 msgr2=0x7fa688070250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:25.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.381+0000 7fa68d200640 1 -- 172.21.15.32:0/3340362044 shutdown_connections 2024-04-17T10:01:25.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.381+0000 7fa68d200640 1 -- 172.21.15.32:0/3340362044 wait complete. 2024-04-17T10:01:25.397 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:01:25.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.541+0000 7f030e800640 1 Processor -- start 2024-04-17T10:01:25.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.541+0000 7f030e800640 1 -- start start 2024-04-17T10:01:25.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.541+0000 7f030e800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0308145f30 0x7f0308071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:25.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.541+0000 7f030e800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f030815d8d0 0x7f0308072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:25.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.541+0000 7f030e800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03080cdac0 con 0x7f03080cce70 2024-04-17T10:01:25.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.541+0000 7f030e800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f030806f400 con 0x7f0308145f30 2024-04-17T10:01:25.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.541+0000 7f030e800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0308146310 con 0x7f030815d8d0 2024-04-17T10:01:25.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.541+0000 7f0307400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0308145f30 0x7f0308071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:25.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.541+0000 7f0307400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0308145f30 0x7f0308071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53454/0 (socket says 172.21.15.32:53454) 2024-04-17T10:01:25.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.541+0000 7f0307400640 1 -- 172.21.15.32:0/3628505195 learned_addr learned my addr 172.21.15.32:0/3628505195 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:25.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.541+0000 7f02fea00640 1 --2- 172.21.15.32:0/3628505195 >> v2:172.21.15.144:3300/0 conn(0x7f030815d8d0 0x7f0308072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:25.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.541+0000 7f0307400640 1 -- 172.21.15.32:0/3628505195 >> v2:172.21.15.144:3300/0 conn(0x7f030815d8d0 msgr2=0x7f0308072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:25.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.541+0000 7f0307400640 1 --2- 172.21.15.32:0/3628505195 >> v2:172.21.15.144:3300/0 conn(0x7f030815d8d0 0x7f0308072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.541+0000 7f0307400640 1 -- 172.21.15.32:0/3628505195 >> v1:172.21.15.144:6789/0 conn(0x7f03080cce70 legacy=0x7f03080cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:25.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.541+0000 7f0307400640 1 -- 172.21.15.32:0/3628505195 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030815cdf0 con 0x7f0308145f30 2024-04-17T10:01:25.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.545+0000 7f0307400640 1 --2- 172.21.15.32:0/3628505195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0308145f30 0x7f0308071df0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f03080702a0 tx=0x7f02f801ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788a186f026da0f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.545+0000 7f0306a00640 1 -- 172.21.15.32:0/3628505195 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f80b5070 con 0x7f0308145f30 2024-04-17T10:01:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.545+0000 7f0306a00640 1 -- 172.21.15.32:0/3628505195 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02f80a89d0 con 0x7f0308145f30 2024-04-17T10:01:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.545+0000 7f0306a00640 1 -- 172.21.15.32:0/3628505195 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f80a8da0 con 0x7f0308145f30 2024-04-17T10:01:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.545+0000 7f030e800640 1 -- 172.21.15.32:0/3628505195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0308145f30 msgr2=0x7f0308071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.545+0000 7f030e800640 1 --2- 172.21.15.32:0/3628505195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0308145f30 0x7f0308071df0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f03080702a0 tx=0x7f02f801ecd0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.545+0000 7f030e800640 1 -- 172.21.15.32:0/3628505195 shutdown_connections 2024-04-17T10:01:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.545+0000 7f030e800640 1 --2- 172.21.15.32:0/3628505195 >> v2:172.21.15.144:3300/0 conn(0x7f030815d8d0 0x7f0308072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.545+0000 7f030e800640 1 --2- 172.21.15.32:0/3628505195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0308145f30 0x7f0308071df0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.545+0000 7f030e800640 1 -- 172.21.15.32:0/3628505195 >> 172.21.15.32:0/3628505195 conn(0x7f030806ba30 msgr2=0x7f030806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.545+0000 7f030e800640 1 -- 172.21.15.32:0/3628505195 shutdown_connections 2024-04-17T10:01:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.545+0000 7f030e800640 1 -- 172.21.15.32:0/3628505195 wait complete. 2024-04-17T10:01:25.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f030e800640 1 Processor -- start 2024-04-17T10:01:25.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f030e800640 1 -- start start 2024-04-17T10:01:25.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f030e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03080cce70 0x7f030815b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:25.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f030e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0308145f30 0x7f030815dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:25.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f030e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f030815d8d0 0x7f030815e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:25.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f030e800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f030815cb90 con 0x7f03080cce70 2024-04-17T10:01:25.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f030e800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f030815d440 con 0x7f0308145f30 2024-04-17T10:01:25.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f030e800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f03081601d0 con 0x7f030815d8d0 2024-04-17T10:01:25.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f0307400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03080cce70 0x7f030815b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:25.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f0307400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03080cce70 0x7f030815b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53460/0 (socket says 172.21.15.32:53460) 2024-04-17T10:01:25.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f0307400640 1 -- 172.21.15.32:0/1331970371 learned_addr learned my addr 172.21.15.32:0/1331970371 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:25.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f02fea00640 1 --2- 172.21.15.32:0/1331970371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0308145f30 0x7f030815dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:25.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f0307e00640 1 --2- 172.21.15.32:0/1331970371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f030815d8d0 0x7f030815e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:25.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f0307400640 1 -- 172.21.15.32:0/1331970371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f030815d8d0 msgr2=0x7f030815e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:25.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f0307400640 1 --2- 172.21.15.32:0/1331970371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f030815d8d0 0x7f030815e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f0307400640 1 -- 172.21.15.32:0/1331970371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0308145f30 msgr2=0x7f030815dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:25.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f0307400640 1 --2- 172.21.15.32:0/1331970371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0308145f30 0x7f030815dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f0307400640 1 -- 172.21.15.32:0/1331970371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03080c6e20 con 0x7f03080cce70 2024-04-17T10:01:25.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f0307e00640 1 --2- 172.21.15.32:0/1331970371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f030815d8d0 0x7f030815e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:25.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f0307400640 1 --2- 172.21.15.32:0/1331970371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03080cce70 0x7f030815b810 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f02f80b3040 tx=0x7f02f801ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:25.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f02ffe00640 1 -- 172.21.15.32:0/1331970371 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f80c0070 con 0x7f03080cce70 2024-04-17T10:01:25.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f02ffe00640 1 -- 172.21.15.32:0/1331970371 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02f80b0a50 con 0x7f03080cce70 2024-04-17T10:01:25.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f02ffe00640 1 -- 172.21.15.32:0/1331970371 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f80b5070 con 0x7f03080cce70 2024-04-17T10:01:25.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f030e800640 1 -- 172.21.15.32:0/1331970371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03080cc9e0 con 0x7f03080cce70 2024-04-17T10:01:25.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f030e800640 1 -- 172.21.15.32:0/1331970371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03080c73c0 con 0x7f03080cce70 2024-04-17T10:01:25.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.549+0000 7f030e800640 1 -- 172.21.15.32:0/1331970371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0308068a90 con 0x7f03080cce70 2024-04-17T10:01:25.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.553+0000 7f02ffe00640 1 -- 172.21.15.32:0/1331970371 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f02f80c3070 con 0x7f03080cce70 2024-04-17T10:01:25.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.553+0000 7f02ffe00640 1 --2- 172.21.15.32:0/1331970371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f02e4068380 0x7f02e406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:25.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.553+0000 7f02ffe00640 1 -- 172.21.15.32:0/1331970371 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f02f8123c00 con 0x7f03080cce70 2024-04-17T10:01:25.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.553+0000 7f02fea00640 1 --2- 172.21.15.32:0/1331970371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f02e4068380 0x7f02e406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:25.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.553+0000 7f02fea00640 1 --2- 172.21.15.32:0/1331970371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f02e4068380 0x7f02e406a840 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f030813de80 tx=0x7f02f0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:25.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.557+0000 7f02ffe00640 1 -- 172.21.15.32:0/1331970371 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f02f8018830 con 0x7f03080cce70 2024-04-17T10:01:25.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.721+0000 7f030e800640 1 -- 172.21.15.32:0/1331970371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0308072f90 con 0x7f03080cce70 2024-04-17T10:01:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.721+0000 7f02ffe00640 1 -- 172.21.15.32:0/1331970371 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+17029 (secure 0 0 0) 0x7f02f80f4030 con 0x7f03080cce70 2024-04-17T10:01:25.720 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:25.721 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":86,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:00:43.842900+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:59:03.223426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-17T09:59:03.780120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.11","mappings":[{"from":1,"to":3}]},{"pgid":"13.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6837/1253497636":"2024-04-18T10:00:43.842876+0000","172.21.15.32:6835/1253497636":"2024-04-18T10:00:43.842876+0000","172.21.15.32:6837/2656735817":"2024-04-18T09:59:28.985107+0000","172.21.15.32:6835/2656735817":"2024-04-18T09:59:28.985107+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:01:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.725+0000 7f030e800640 1 -- 172.21.15.32:0/1331970371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f02e4068380 msgr2=0x7f02e406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.725+0000 7f030e800640 1 --2- 172.21.15.32:0/1331970371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f02e4068380 0x7f02e406a840 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f030813de80 tx=0x7f02f0008040 comp rx=0 tx=0).stop 2024-04-17T10:01:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.725+0000 7f030e800640 1 -- 172.21.15.32:0/1331970371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03080cce70 msgr2=0x7f030815b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.725+0000 7f030e800640 1 --2- 172.21.15.32:0/1331970371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03080cce70 0x7f030815b810 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f02f80b3040 tx=0x7f02f801ba70 comp rx=0 tx=0).stop 2024-04-17T10:01:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.725+0000 7f030e800640 1 -- 172.21.15.32:0/1331970371 shutdown_connections 2024-04-17T10:01:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.725+0000 7f030e800640 1 --2- 172.21.15.32:0/1331970371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f02e4068380 0x7f02e406a840 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.725+0000 7f030e800640 1 --2- 172.21.15.32:0/1331970371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f030815d8d0 0x7f030815e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.725+0000 7f030e800640 1 --2- 172.21.15.32:0/1331970371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0308145f30 0x7f030815dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.725+0000 7f030e800640 1 --2- 172.21.15.32:0/1331970371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03080cce70 0x7f030815b810 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.725+0000 7f030e800640 1 -- 172.21.15.32:0/1331970371 >> 172.21.15.32:0/1331970371 conn(0x7f030806ba30 msgr2=0x7f030815f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.725+0000 7f030e800640 1 -- 172.21.15.32:0/1331970371 shutdown_connections 2024-04-17T10:01:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.725+0000 7f030e800640 1 -- 172.21.15.32:0/1331970371 wait complete. 2024-04-17T10:01:25.743 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T09:59:03.223426+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T09:59:03.780120+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:01:25.744 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:01:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.889+0000 7fc7cfe00640 1 Processor -- start 2024-04-17T10:01:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.889+0000 7fc7cfe00640 1 -- start start 2024-04-17T10:01:25.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.889+0000 7fc7cfe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7c815b7d0 0x7fc7c806e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:25.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.889+0000 7fc7cfe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7c806f7c0 0x7fc7c806ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:25.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.889+0000 7fc7cfe00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7c80cc940 con 0x7fc7c806f270 2024-04-17T10:01:25.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.889+0000 7fc7cfe00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc7c80ccc20 con 0x7fc7c815b7d0 2024-04-17T10:01:25.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.889+0000 7fc7cfe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c806e0b0 con 0x7fc7c806f7c0 2024-04-17T10:01:25.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.889+0000 7fc7cd400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7c815b7d0 0x7fc7c806e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:25.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.889+0000 7fc7cde00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc7c806f270 0x7fc7c80ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50394/0 (socket says 172.21.15.32:50394) 2024-04-17T10:01:25.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.889+0000 7fc7cde00640 1 -- 172.21.15.32:0/1348042926 learned_addr learned my addr 172.21.15.32:0/1348042926 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:25.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.893+0000 7fc7cca00640 1 --2- 172.21.15.32:0/1348042926 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7c806f7c0 0x7fc7c806ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:25.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.893+0000 7fc7cd400640 1 -- 172.21.15.32:0/1348042926 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7c806f7c0 msgr2=0x7fc7c806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:25.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.893+0000 7fc7cd400640 1 --2- 172.21.15.32:0/1348042926 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7c806f7c0 0x7fc7c806ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.893+0000 7fc7cd400640 1 -- 172.21.15.32:0/1348042926 >> v1:172.21.15.32:6789/0 conn(0x7fc7c806f270 legacy=0x7fc7c80ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:25.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.893+0000 7fc7cd400640 1 -- 172.21.15.32:0/1348042926 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c806e390 con 0x7fc7c815b7d0 2024-04-17T10:01:25.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.893+0000 7fc7c7e00640 1 -- 172.21.15.32:0/1348042926 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3538268867 0 0) 0x7fc7c80cc940 con 0x7fc7c806f270 2024-04-17T10:01:25.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.893+0000 7fc7cca00640 1 --2- 172.21.15.32:0/1348042926 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7c806f7c0 0x7fc7c806ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:25.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.893+0000 7fc7cd400640 1 --2- 172.21.15.32:0/1348042926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7c815b7d0 0x7fc7c806e8b0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0097c0 tx=0x7fc7bc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bcc457ed09ddaf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:25.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.893+0000 7fc7c7e00640 1 -- 172.21.15.32:0/1348042926 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc0b0070 con 0x7fc7c815b7d0 2024-04-17T10:01:25.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.893+0000 7fc7c7e00640 1 -- 172.21.15.32:0/1348042926 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7bc0ab600 con 0x7fc7c815b7d0 2024-04-17T10:01:25.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.893+0000 7fc7c7e00640 1 -- 172.21.15.32:0/1348042926 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc0ab9d0 con 0x7fc7c815b7d0 2024-04-17T10:01:25.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.893+0000 7fc7cfe00640 1 -- 172.21.15.32:0/1348042926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7c815b7d0 msgr2=0x7fc7c806e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.893+0000 7fc7cfe00640 1 --2- 172.21.15.32:0/1348042926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7c815b7d0 0x7fc7c806e8b0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0097c0 tx=0x7fc7bc01c410 comp rx=0 tx=0).stop 2024-04-17T10:01:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.893+0000 7fc7cfe00640 1 -- 172.21.15.32:0/1348042926 shutdown_connections 2024-04-17T10:01:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.893+0000 7fc7cfe00640 1 --2- 172.21.15.32:0/1348042926 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7c806f7c0 0x7fc7c806ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.893+0000 7fc7cfe00640 1 --2- 172.21.15.32:0/1348042926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7c815b7d0 0x7fc7c806e8b0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.893+0000 7fc7cfe00640 1 -- 172.21.15.32:0/1348042926 >> 172.21.15.32:0/1348042926 conn(0x7fc7c80b2120 msgr2=0x7fc7c80b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.897+0000 7fc7cfe00640 1 -- 172.21.15.32:0/1348042926 shutdown_connections 2024-04-17T10:01:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.897+0000 7fc7cfe00640 1 -- 172.21.15.32:0/1348042926 wait complete. 2024-04-17T10:01:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.897+0000 7fc7cfe00640 1 Processor -- start 2024-04-17T10:01:25.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.897+0000 7fc7cfe00640 1 -- start start 2024-04-17T10:01:25.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.897+0000 7fc7cfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7c806f270 0x7fc7c813f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:25.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.897+0000 7fc7cfe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7c806f7c0 0x7fc7c80ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:25.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.897+0000 7fc7cfe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7c815b7d0 0x7fc7c813cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:25.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.897+0000 7fc7cfe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c8168ab0 con 0x7fc7c806f270 2024-04-17T10:01:25.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.897+0000 7fc7cfe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c806c380 con 0x7fc7c815b7d0 2024-04-17T10:01:25.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.897+0000 7fc7cfe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c8168c30 con 0x7fc7c806f7c0 2024-04-17T10:01:25.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.897+0000 7fc7cd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7c806f270 0x7fc7c813f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:25.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.897+0000 7fc7cd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7c806f270 0x7fc7c813f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53468/0 (socket says 172.21.15.32:53468) 2024-04-17T10:01:25.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.897+0000 7fc7cd400640 1 -- 172.21.15.32:0/2359838650 learned_addr learned my addr 172.21.15.32:0/2359838650 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:25.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.897+0000 7fc7cd400640 1 -- 172.21.15.32:0/2359838650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7c806f7c0 msgr2=0x7fc7c80ae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:25.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.897+0000 7fc7cca00640 1 --2- 172.21.15.32:0/2359838650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7c806f7c0 0x7fc7c80ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:25.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.897+0000 7fc7cde00640 1 --2- 172.21.15.32:0/2359838650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7c815b7d0 0x7fc7c813cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:25.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.897+0000 7fc7cd400640 1 --2- 172.21.15.32:0/2359838650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7c806f7c0 0x7fc7c80ae5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.897+0000 7fc7cd400640 1 -- 172.21.15.32:0/2359838650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7c815b7d0 msgr2=0x7fc7c813cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:25.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.897+0000 7fc7cd400640 1 --2- 172.21.15.32:0/2359838650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7c815b7d0 0x7fc7c813cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:25.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.897+0000 7fc7cd400640 1 -- 172.21.15.32:0/2359838650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c81404a0 con 0x7fc7c806f270 2024-04-17T10:01:25.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.897+0000 7fc7cde00640 1 --2- 172.21.15.32:0/2359838650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7c815b7d0 0x7fc7c813cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:25.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.897+0000 7fc7cd400640 1 --2- 172.21.15.32:0/2359838650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7c806f270 0x7fc7c813f540 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0ae040 tx=0x7fc7bc01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:25.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.897+0000 7fc7c5600640 1 -- 172.21.15.32:0/2359838650 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc0b0050 con 0x7fc7c806f270 2024-04-17T10:01:25.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.901+0000 7fc7c5600640 1 -- 172.21.15.32:0/2359838650 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7bc0b1040 con 0x7fc7c806f270 2024-04-17T10:01:25.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.901+0000 7fc7c5600640 1 -- 172.21.15.32:0/2359838650 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc004220 con 0x7fc7c806f270 2024-04-17T10:01:25.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.901+0000 7fc7cfe00640 1 -- 172.21.15.32:0/2359838650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c813fc00 con 0x7fc7c806f270 2024-04-17T10:01:25.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.901+0000 7fc7cfe00640 1 -- 172.21.15.32:0/2359838650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c81401c0 con 0x7fc7c806f270 2024-04-17T10:01:25.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.901+0000 7fc7cfe00640 1 -- 172.21.15.32:0/2359838650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c8068a90 con 0x7fc7c806f270 2024-04-17T10:01:25.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.901+0000 7fc7c5600640 1 -- 172.21.15.32:0/2359838650 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc7bc0be060 con 0x7fc7c806f270 2024-04-17T10:01:25.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.901+0000 7fc7c5600640 1 --2- 172.21.15.32:0/2359838650 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc7a8068380 0x7fc7a806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:25.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.901+0000 7fc7c5600640 1 -- 172.21.15.32:0/2359838650 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7fc7bc1204a0 con 0x7fc7c806f270 2024-04-17T10:01:25.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.901+0000 7fc7cca00640 1 --2- 172.21.15.32:0/2359838650 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc7a8068380 0x7fc7a806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:25.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.905+0000 7fc7cca00640 1 --2- 172.21.15.32:0/2359838650 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc7a8068380 0x7fc7a806a840 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4006fd0 tx=0x7fc7b4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:25.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:25.905+0000 7fc7c5600640 1 -- 172.21.15.32:0/2359838650 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc7bc016090 con 0x7fc7c806f270 2024-04-17T10:01:26.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.073+0000 7fc7cfe00640 1 -- 172.21.15.32:0/2359838650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc7c80cd0b0 con 0x7fc7c806f270 2024-04-17T10:01:26.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.073+0000 7fc7c5600640 1 -- 172.21.15.32:0/2359838650 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7fc7bc0f0960 con 0x7fc7c806f270 2024-04-17T10:01:26.071 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T10:01:26.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.077+0000 7fc7cfe00640 1 -- 172.21.15.32:0/2359838650 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc7a8068380 msgr2=0x7fc7a806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:26.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.077+0000 7fc7cfe00640 1 --2- 172.21.15.32:0/2359838650 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc7a8068380 0x7fc7a806a840 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4006fd0 tx=0x7fc7b4008040 comp rx=0 tx=0).stop 2024-04-17T10:01:26.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.077+0000 7fc7cfe00640 1 -- 172.21.15.32:0/2359838650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7c806f270 msgr2=0x7fc7c813f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:26.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.077+0000 7fc7cfe00640 1 --2- 172.21.15.32:0/2359838650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7c806f270 0x7fc7c813f540 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0ae040 tx=0x7fc7bc01ac50 comp rx=0 tx=0).stop 2024-04-17T10:01:26.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.077+0000 7fc7cfe00640 1 -- 172.21.15.32:0/2359838650 shutdown_connections 2024-04-17T10:01:26.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.077+0000 7fc7cfe00640 1 --2- 172.21.15.32:0/2359838650 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc7a8068380 0x7fc7a806a840 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.077+0000 7fc7cfe00640 1 --2- 172.21.15.32:0/2359838650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7c806f7c0 0x7fc7c80ae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.077+0000 7fc7cfe00640 1 --2- 172.21.15.32:0/2359838650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7c815b7d0 0x7fc7c813cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.077+0000 7fc7cfe00640 1 --2- 172.21.15.32:0/2359838650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7c806f270 0x7fc7c813f540 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.077+0000 7fc7cfe00640 1 -- 172.21.15.32:0/2359838650 >> 172.21.15.32:0/2359838650 conn(0x7fc7c80b2120 msgr2=0x7fc7c80b4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:26.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.077+0000 7fc7cfe00640 1 -- 172.21.15.32:0/2359838650 shutdown_connections 2024-04-17T10:01:26.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.077+0000 7fc7cfe00640 1 -- 172.21.15.32:0/2359838650 wait complete. 2024-04-17T10:01:26.092 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:01:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.237+0000 7f6c00e00640 1 Processor -- start 2024-04-17T10:01:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.237+0000 7f6c00e00640 1 -- start start 2024-04-17T10:01:26.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.237+0000 7f6c00e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bfc06e160 0x7f6bfc0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:26.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.237+0000 7f6c00e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6bfc09b100 0x7f6bfc15e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:26.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.237+0000 7f6c00e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bfc0cdd90 con 0x7f6bfc0cd8d0 2024-04-17T10:01:26.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.237+0000 7f6c00e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6bfc0ce070 con 0x7f6bfc09b100 2024-04-17T10:01:26.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.237+0000 7f6c00e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6bfc0ce350 con 0x7f6bfc06e160 2024-04-17T10:01:26.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.237+0000 7f6bfbe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6bfc09b100 0x7f6bfc15e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:26.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.237+0000 7f6bfbe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6bfc09b100 0x7f6bfc15e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53476/0 (socket says 172.21.15.32:53476) 2024-04-17T10:01:26.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.237+0000 7f6bfbe00640 1 -- 172.21.15.32:0/2072396100 learned_addr learned my addr 172.21.15.32:0/2072396100 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:26.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.237+0000 7f6bf2a00640 1 --1- 172.21.15.32:0/2072396100 >> v1:172.21.15.32:6789/0 conn(0x7f6bfc0cd8d0 0x7f6bfc09a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50400/0 (socket says 172.21.15.32:50400) 2024-04-17T10:01:26.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.237+0000 7f6bfb400640 1 --2- 172.21.15.32:0/2072396100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bfc06e160 0x7f6bfc0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:26.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.237+0000 7f6bfbe00640 1 -- 172.21.15.32:0/2072396100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bfc06e160 msgr2=0x7f6bfc0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:26.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.237+0000 7f6bfbe00640 1 --2- 172.21.15.32:0/2072396100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bfc06e160 0x7f6bfc0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.237+0000 7f6bfbe00640 1 -- 172.21.15.32:0/2072396100 >> v1:172.21.15.32:6789/0 conn(0x7f6bfc0cd8d0 legacy=0x7f6bfc09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:26.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.237+0000 7f6bfbe00640 1 -- 172.21.15.32:0/2072396100 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bfc0ce630 con 0x7f6bfc09b100 2024-04-17T10:01:26.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.237+0000 7f6bfaa00640 1 -- 172.21.15.32:0/2072396100 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1095592289 0 0) 0x7f6bfc0cdd90 con 0x7f6bfc0cd8d0 2024-04-17T10:01:26.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.241+0000 7f6bfbe00640 1 --2- 172.21.15.32:0/2072396100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bfc09b100 0x7f6bfc15e180 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009870 tx=0x7f6bf401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c19340644ef879 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:26.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.241+0000 7f6bfaa00640 1 -- 172.21.15.32:0/2072396100 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40a7070 con 0x7f6bfc09b100 2024-04-17T10:01:26.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.241+0000 7f6bfaa00640 1 -- 172.21.15.32:0/2072396100 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bf40b0070 con 0x7f6bfc09b100 2024-04-17T10:01:26.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.241+0000 7f6bfaa00640 1 -- 172.21.15.32:0/2072396100 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40ab890 con 0x7f6bfc09b100 2024-04-17T10:01:26.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.241+0000 7f6c00e00640 1 -- 172.21.15.32:0/2072396100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bfc09b100 msgr2=0x7f6bfc15e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:26.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.241+0000 7f6c00e00640 1 --2- 172.21.15.32:0/2072396100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bfc09b100 0x7f6bfc15e180 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009870 tx=0x7f6bf401c560 comp rx=0 tx=0).stop 2024-04-17T10:01:26.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.241+0000 7f6c00e00640 1 -- 172.21.15.32:0/2072396100 shutdown_connections 2024-04-17T10:01:26.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.241+0000 7f6c00e00640 1 --2- 172.21.15.32:0/2072396100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bfc09b100 0x7f6bfc15e180 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.241+0000 7f6c00e00640 1 --2- 172.21.15.32:0/2072396100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bfc06e160 0x7f6bfc0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.241+0000 7f6c00e00640 1 -- 172.21.15.32:0/2072396100 >> 172.21.15.32:0/2072396100 conn(0x7f6bfc0b4000 msgr2=0x7f6bfc0b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:26.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.241+0000 7f6c00e00640 1 -- 172.21.15.32:0/2072396100 shutdown_connections 2024-04-17T10:01:26.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.241+0000 7f6c00e00640 1 -- 172.21.15.32:0/2072396100 wait complete. 2024-04-17T10:01:26.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6c00e00640 1 Processor -- start 2024-04-17T10:01:26.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6c00e00640 1 -- start start 2024-04-17T10:01:26.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6c00e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bfc06e160 0x7f6bfc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:26.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6c00e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bfc09b100 0x7f6bfc158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:26.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6c00e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bfc0cd8d0 0x7f6bfc158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:26.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6c00e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6bfc06cc10 con 0x7f6bfc0cd8d0 2024-04-17T10:01:26.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6c00e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6bfc06ca90 con 0x7f6bfc09b100 2024-04-17T10:01:26.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6c00e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6bfc06cd90 con 0x7f6bfc06e160 2024-04-17T10:01:26.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6bfbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bfc0cd8d0 0x7f6bfc158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:26.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6bfbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bfc0cd8d0 0x7f6bfc158c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53488/0 (socket says 172.21.15.32:53488) 2024-04-17T10:01:26.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6bfbe00640 1 -- 172.21.15.32:0/1265757644 learned_addr learned my addr 172.21.15.32:0/1265757644 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:26.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6bfb400640 1 --2- 172.21.15.32:0/1265757644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bfc06e160 0x7f6bfc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:26.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6bf2a00640 1 --2- 172.21.15.32:0/1265757644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bfc09b100 0x7f6bfc158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:26.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6bfbe00640 1 -- 172.21.15.32:0/1265757644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bfc06e160 msgr2=0x7f6bfc0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:26.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6bfbe00640 1 --2- 172.21.15.32:0/1265757644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bfc06e160 0x7f6bfc0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6bfbe00640 1 -- 172.21.15.32:0/1265757644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bfc09b100 msgr2=0x7f6bfc158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:26.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6bfbe00640 1 --2- 172.21.15.32:0/1265757644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bfc09b100 0x7f6bfc158730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6bfbe00640 1 -- 172.21.15.32:0/1265757644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bfc0b58e0 con 0x7f6bfc0cd8d0 2024-04-17T10:01:26.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6bf2a00640 1 --2- 172.21.15.32:0/1265757644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bfc09b100 0x7f6bfc158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:26.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6bfbe00640 1 --2- 172.21.15.32:0/1265757644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bfc0cd8d0 0x7f6bfc158c70 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f6bf40ad040 tx=0x7f6bf401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:26.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6bf3e00640 1 -- 172.21.15.32:0/1265757644 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40abe60 con 0x7f6bfc0cd8d0 2024-04-17T10:01:26.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6bf3e00640 1 -- 172.21.15.32:0/1265757644 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bf40be5f0 con 0x7f6bfc0cd8d0 2024-04-17T10:01:26.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6bf3e00640 1 -- 172.21.15.32:0/1265757644 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40be9e0 con 0x7f6bfc0cd8d0 2024-04-17T10:01:26.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6c00e00640 1 -- 172.21.15.32:0/1265757644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bfc0b6d30 con 0x7f6bfc0cd8d0 2024-04-17T10:01:26.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.245+0000 7f6c00e00640 1 -- 172.21.15.32:0/1265757644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bfc0b72f0 con 0x7f6bfc0cd8d0 2024-04-17T10:01:26.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.249+0000 7f6c00e00640 1 -- 172.21.15.32:0/1265757644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bfc04fda0 con 0x7f6bfc0cd8d0 2024-04-17T10:01:26.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.249+0000 7f6bf3e00640 1 -- 172.21.15.32:0/1265757644 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6bf40b43f0 con 0x7f6bfc0cd8d0 2024-04-17T10:01:26.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.249+0000 7f6bf3e00640 1 --2- 172.21.15.32:0/1265757644 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6bd8068380 0x7f6bd806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:26.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.249+0000 7f6bf3e00640 1 -- 172.21.15.32:0/1265757644 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f6bf4120ff0 con 0x7f6bfc0cd8d0 2024-04-17T10:01:26.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.249+0000 7f6bfb400640 1 --2- 172.21.15.32:0/1265757644 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6bd8068380 0x7f6bd806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:26.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.249+0000 7f6bfb400640 1 --2- 172.21.15.32:0/1265757644 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6bd8068380 0x7f6bd806a840 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0113e0 tx=0x7f6bec00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:26.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.253+0000 7f6bf3e00640 1 -- 172.21.15.32:0/1265757644 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f6bf40c4060 con 0x7f6bfc0cd8d0 2024-04-17T10:01:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.417+0000 7f6c00e00640 1 -- 172.21.15.32:0/1265757644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6bfc15f100 con 0x7f6bfc0cd8d0 2024-04-17T10:01:26.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.417+0000 7f6bf3e00640 1 -- 172.21.15.32:0/1265757644 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) ==== 104+0+11 (secure 0 0 0) 0x7f6bf4016150 con 0x7f6bfc0cd8d0 2024-04-17T10:01:26.416 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T10:01:26.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.421+0000 7f6c00e00640 1 -- 172.21.15.32:0/1265757644 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6bd8068380 msgr2=0x7f6bd806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:26.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.421+0000 7f6c00e00640 1 --2- 172.21.15.32:0/1265757644 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6bd8068380 0x7f6bd806a840 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0113e0 tx=0x7f6bec00b040 comp rx=0 tx=0).stop 2024-04-17T10:01:26.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.421+0000 7f6c00e00640 1 -- 172.21.15.32:0/1265757644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bfc0cd8d0 msgr2=0x7f6bfc158c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:26.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.421+0000 7f6c00e00640 1 --2- 172.21.15.32:0/1265757644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bfc0cd8d0 0x7f6bfc158c70 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f6bf40ad040 tx=0x7f6bf401ada0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.421+0000 7f6c00e00640 1 -- 172.21.15.32:0/1265757644 shutdown_connections 2024-04-17T10:01:26.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.421+0000 7f6c00e00640 1 --2- 172.21.15.32:0/1265757644 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6bd8068380 0x7f6bd806a840 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.421+0000 7f6c00e00640 1 --2- 172.21.15.32:0/1265757644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bfc0cd8d0 0x7f6bfc158c70 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.421+0000 7f6c00e00640 1 --2- 172.21.15.32:0/1265757644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bfc09b100 0x7f6bfc158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.421+0000 7f6c00e00640 1 --2- 172.21.15.32:0/1265757644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bfc06e160 0x7f6bfc0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.421+0000 7f6c00e00640 1 -- 172.21.15.32:0/1265757644 >> 172.21.15.32:0/1265757644 conn(0x7f6bfc0b4000 msgr2=0x7f6bfc071640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:26.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.421+0000 7f6c00e00640 1 -- 172.21.15.32:0/1265757644 shutdown_connections 2024-04-17T10:01:26.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.421+0000 7f6c00e00640 1 -- 172.21.15.32:0/1265757644 wait complete. 2024-04-17T10:01:26.437 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:01:26.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb727e00640 1 Processor -- start 2024-04-17T10:01:26.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb727e00640 1 -- start start 2024-04-17T10:01:26.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb727e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb72015b7d0 0x7fb72006e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:26.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb727e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7200cc940 con 0x7fb72006f7c0 2024-04-17T10:01:26.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb727e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7200ccc20 con 0x7fb72006c140 2024-04-17T10:01:26.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb727e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb72006e0b0 con 0x7fb72015b7d0 2024-04-17T10:01:26.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb724a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb72006f7c0 0x7fb72006ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50404/0 (socket says 172.21.15.32:50404) 2024-04-17T10:01:26.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb724a00640 1 -- 172.21.15.32:0/3000698307 learned_addr learned my addr 172.21.15.32:0/3000698307 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:26.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb725400640 1 --2- 172.21.15.32:0/3000698307 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb72015b7d0 0x7fb72006e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:26.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb71fe00640 1 -- 172.21.15.32:0/3000698307 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 485405877 0 0) 0x7fb7200cc940 con 0x7fb72006f7c0 2024-04-17T10:01:26.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb71fe00640 1 -- 172.21.15.32:0/3000698307 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb708003530 con 0x7fb72006f7c0 2024-04-17T10:01:26.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb71fe00640 1 -- 172.21.15.32:0/3000698307 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 471309237 0 0) 0x7fb7200ccc20 con 0x7fb72006c140 2024-04-17T10:01:26.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb71fe00640 1 -- 172.21.15.32:0/3000698307 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7200cc940 con 0x7fb72006c140 2024-04-17T10:01:26.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb71fe00640 1 -- 172.21.15.32:0/3000698307 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1593902085 0 0) 0x7fb708003530 con 0x7fb72006f7c0 2024-04-17T10:01:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb71fe00640 1 -- 172.21.15.32:0/3000698307 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb72015b7d0 msgr2=0x7fb72006e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb71fe00640 1 --2- 172.21.15.32:0/3000698307 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb72015b7d0 0x7fb72006e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb71fe00640 1 -- 172.21.15.32:0/3000698307 >> v1:172.21.15.144:6789/0 conn(0x7fb72006c140 legacy=0x7fb7200ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb71fe00640 1 -- 172.21.15.32:0/3000698307 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72006e390 con 0x7fb72006f7c0 2024-04-17T10:01:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb71fe00640 1 -- 172.21.15.32:0/3000698307 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7fb70c001680 con 0x7fb72006f7c0 2024-04-17T10:01:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb71fe00640 1 -- 172.21.15.32:0/3000698307 >> v1:172.21.15.32:6789/0 conn(0x7fb72006f7c0 legacy=0x7fb72006ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb71fe00640 1 --2- 172.21.15.32:0/3000698307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb708003b80 0x7fb708003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb71fe00640 1 --2- 172.21.15.32:0/3000698307 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb72015b7d0 0x7fb708004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb71fe00640 1 --2- 172.21.15.32:0/3000698307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb708004d50 0x7fb7080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb71fe00640 1 -- 172.21.15.32:0/3000698307 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb72006e0b0 con 0x7fb708003b80 2024-04-17T10:01:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb71fe00640 1 -- 172.21.15.32:0/3000698307 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb70800bb60 con 0x7fb708004d50 2024-04-17T10:01:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb71fe00640 1 -- 172.21.15.32:0/3000698307 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb70800bd30 con 0x7fb72015b7d0 2024-04-17T10:01:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb725400640 1 --2- 172.21.15.32:0/3000698307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb708003b80 0x7fb708003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:26.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb724a00640 1 --2- 172.21.15.32:0/3000698307 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb72015b7d0 0x7fb708004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:26.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb725e00640 1 --2- 172.21.15.32:0/3000698307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb708004d50 0x7fb7080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:26.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb725400640 1 -- 172.21.15.32:0/3000698307 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb72015b7d0 msgr2=0x7fb708004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:26.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb725400640 1 --2- 172.21.15.32:0/3000698307 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb72015b7d0 0x7fb708004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb725400640 1 -- 172.21.15.32:0/3000698307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb708004d50 msgr2=0x7fb7080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:26.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb725400640 1 --2- 172.21.15.32:0/3000698307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb708004d50 0x7fb7080051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.585+0000 7fb725400640 1 -- 172.21.15.32:0/3000698307 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb708003530 con 0x7fb708003b80 2024-04-17T10:01:26.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.589+0000 7fb725400640 1 --2- 172.21.15.32:0/3000698307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb708003b80 0x7fb708003f80 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fb71400e170 tx=0x7fb71401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e0ef528d802348 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:26.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.589+0000 7fb71fe00640 1 -- 172.21.15.32:0/3000698307 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7140b2070 con 0x7fb708003b80 2024-04-17T10:01:26.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.589+0000 7fb71fe00640 1 -- 172.21.15.32:0/3000698307 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7140a5c70 con 0x7fb708003b80 2024-04-17T10:01:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.589+0000 7fb71fe00640 1 -- 172.21.15.32:0/3000698307 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714009c10 con 0x7fb708003b80 2024-04-17T10:01:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.589+0000 7fb727e00640 1 -- 172.21.15.32:0/3000698307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb708003b80 msgr2=0x7fb708003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.589+0000 7fb727e00640 1 --2- 172.21.15.32:0/3000698307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb708003b80 0x7fb708003f80 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fb71400e170 tx=0x7fb71401cc60 comp rx=0 tx=0).stop 2024-04-17T10:01:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.589+0000 7fb725e00640 1 -- 172.21.15.32:0/3000698307 reap_dead start 2024-04-17T10:01:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.589+0000 7fb727e00640 1 -- 172.21.15.32:0/3000698307 shutdown_connections 2024-04-17T10:01:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.589+0000 7fb727e00640 1 -- 172.21.15.32:0/3000698307 >> 172.21.15.32:0/3000698307 conn(0x7fb7200b2120 msgr2=0x7fb7200b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.589+0000 7fb727e00640 1 -- 172.21.15.32:0/3000698307 shutdown_connections 2024-04-17T10:01:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.589+0000 7fb727e00640 1 -- 172.21.15.32:0/3000698307 wait complete. 2024-04-17T10:01:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb727e00640 1 Processor -- start 2024-04-17T10:01:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb727e00640 1 -- start start 2024-04-17T10:01:26.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb727e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb708003b80 0x7fb7201461c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:26.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb727e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb720145b50 0x7fb7200b3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:26.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb727e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7200b3750 0x7fb7201519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:26.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb727e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb72006ec80 con 0x7fb708003b80 2024-04-17T10:01:26.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb727e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb72006eb00 con 0x7fb7200b3750 2024-04-17T10:01:26.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb727e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb72006ee00 con 0x7fb720145b50 2024-04-17T10:01:26.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb725400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb708003b80 0x7fb7201461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:26.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb725400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb708003b80 0x7fb7201461c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53504/0 (socket says 172.21.15.32:53504) 2024-04-17T10:01:26.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb725400640 1 -- 172.21.15.32:0/2429147215 learned_addr learned my addr 172.21.15.32:0/2429147215 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:26.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb725e00640 1 --2- 172.21.15.32:0/2429147215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7200b3750 0x7fb7201519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:26.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb724a00640 1 --2- 172.21.15.32:0/2429147215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb720145b50 0x7fb7200b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:26.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb725400640 1 -- 172.21.15.32:0/2429147215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb720145b50 msgr2=0x7fb7200b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:26.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb725400640 1 --2- 172.21.15.32:0/2429147215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb720145b50 0x7fb7200b3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb725400640 1 -- 172.21.15.32:0/2429147215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7200b3750 msgr2=0x7fb7201519a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:26.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb725400640 1 --2- 172.21.15.32:0/2429147215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7200b3750 0x7fb7201519a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb725400640 1 -- 172.21.15.32:0/2429147215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb720145fd0 con 0x7fb708003b80 2024-04-17T10:01:26.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb724a00640 1 --2- 172.21.15.32:0/2429147215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb720145b50 0x7fb7200b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:26.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb725400640 1 --2- 172.21.15.32:0/2429147215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb708003b80 0x7fb7201461c0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fb7140b0040 tx=0x7fb714009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:26.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb71d600640 1 -- 172.21.15.32:0/2429147215 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7140ba020 con 0x7fb708003b80 2024-04-17T10:01:26.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb71d600640 1 -- 172.21.15.32:0/2429147215 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7140ada50 con 0x7fb708003b80 2024-04-17T10:01:26.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb71d600640 1 -- 172.21.15.32:0/2429147215 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7140b2040 con 0x7fb708003b80 2024-04-17T10:01:26.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb727e00640 1 -- 172.21.15.32:0/2429147215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb720157270 con 0x7fb708003b80 2024-04-17T10:01:26.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.593+0000 7fb727e00640 1 -- 172.21.15.32:0/2429147215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb720157830 con 0x7fb708003b80 2024-04-17T10:01:26.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.597+0000 7fb727e00640 1 -- 172.21.15.32:0/2429147215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb720068a90 con 0x7fb708003b80 2024-04-17T10:01:26.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.597+0000 7fb71d600640 1 -- 172.21.15.32:0/2429147215 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb7140bd020 con 0x7fb708003b80 2024-04-17T10:01:26.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.597+0000 7fb71d600640 1 --2- 172.21.15.32:0/2429147215 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb700068380 0x7fb70006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:26.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.597+0000 7fb71d600640 1 -- 172.21.15.32:0/2429147215 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7fb7141211d0 con 0x7fb708003b80 2024-04-17T10:01:26.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.597+0000 7fb724a00640 1 --2- 172.21.15.32:0/2429147215 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb700068380 0x7fb70006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:26.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.597+0000 7fb724a00640 1 --2- 172.21.15.32:0/2429147215 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb700068380 0x7fb70006a840 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb70c001210 tx=0x7fb70c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:26.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.601+0000 7fb71d600640 1 -- 172.21.15.32:0/2429147215 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fb7140c4030 con 0x7fb708003b80 2024-04-17T10:01:26.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.769+0000 7fb727e00640 1 -- 172.21.15.32:0/2429147215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb7200b3b30 con 0x7fb708003b80 2024-04-17T10:01:26.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.769+0000 7fb71d600640 1 -- 172.21.15.32:0/2429147215 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) ==== 100+0+11 (secure 0 0 0) 0x7fb7140a5e20 con 0x7fb708003b80 2024-04-17T10:01:26.767 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T10:01:26.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.769+0000 7fb727e00640 1 -- 172.21.15.32:0/2429147215 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb700068380 msgr2=0x7fb70006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:26.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.769+0000 7fb727e00640 1 --2- 172.21.15.32:0/2429147215 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb700068380 0x7fb70006a840 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb70c001210 tx=0x7fb70c009040 comp rx=0 tx=0).stop 2024-04-17T10:01:26.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.769+0000 7fb727e00640 1 -- 172.21.15.32:0/2429147215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb708003b80 msgr2=0x7fb7201461c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:26.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.769+0000 7fb727e00640 1 --2- 172.21.15.32:0/2429147215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb708003b80 0x7fb7201461c0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fb7140b0040 tx=0x7fb714009750 comp rx=0 tx=0).stop 2024-04-17T10:01:26.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.769+0000 7fb727e00640 1 -- 172.21.15.32:0/2429147215 shutdown_connections 2024-04-17T10:01:26.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.769+0000 7fb727e00640 1 --2- 172.21.15.32:0/2429147215 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb700068380 0x7fb70006a840 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.769+0000 7fb727e00640 1 --2- 172.21.15.32:0/2429147215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb720145b50 0x7fb7200b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.769+0000 7fb727e00640 1 --2- 172.21.15.32:0/2429147215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7200b3750 0x7fb7201519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.769+0000 7fb727e00640 1 --2- 172.21.15.32:0/2429147215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb708003b80 0x7fb7201461c0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.769+0000 7fb727e00640 1 -- 172.21.15.32:0/2429147215 >> 172.21.15.32:0/2429147215 conn(0x7fb7200b2120 msgr2=0x7fb7200d0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:26.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.769+0000 7fb727e00640 1 -- 172.21.15.32:0/2429147215 shutdown_connections 2024-04-17T10:01:26.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.769+0000 7fb727e00640 1 -- 172.21.15.32:0/2429147215 wait complete. 2024-04-17T10:01:26.788 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:01:26.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.933+0000 7ff2c7600640 1 Processor -- start 2024-04-17T10:01:26.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.933+0000 7ff2c7600640 1 -- start start 2024-04-17T10:01:26.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.933+0000 7ff2c7600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff2c0144e90 0x7ff2c00b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:26.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.933+0000 7ff2c7600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2c015c1b0 0x7ff2c006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:26.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.933+0000 7ff2c7600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c0072bb0 con 0x7ff2c00b4de0 2024-04-17T10:01:26.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.933+0000 7ff2c7600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff2c0145270 con 0x7ff2c0144e90 2024-04-17T10:01:26.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.933+0000 7ff2c7600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c00b3dc0 con 0x7ff2c015c1b0 2024-04-17T10:01:26.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.933+0000 7ff2c4c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff2c0144e90 0x7ff2c00b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:26.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.933+0000 7ff2c4c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff2c0144e90 0x7ff2c00b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53520/0 (socket says 172.21.15.32:53520) 2024-04-17T10:01:26.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.933+0000 7ff2c4c00640 1 -- 172.21.15.32:0/85377204 learned_addr learned my addr 172.21.15.32:0/85377204 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:26.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.933+0000 7ff2c4c00640 1 -- 172.21.15.32:0/85377204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2c015c1b0 msgr2=0x7ff2c006ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:26.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.933+0000 7ff2c4c00640 1 --2- 172.21.15.32:0/85377204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2c015c1b0 0x7ff2c006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.933+0000 7ff2c4c00640 1 -- 172.21.15.32:0/85377204 >> v1:172.21.15.144:6789/0 conn(0x7ff2c00b4de0 legacy=0x7ff2c015baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:26.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.933+0000 7ff2c4c00640 1 -- 172.21.15.32:0/85377204 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c00b40a0 con 0x7ff2c0144e90 2024-04-17T10:01:26.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.937+0000 7ff2c4c00640 1 --2- 172.21.15.32:0/85377204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c0144e90 0x7ff2c00b48a0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009870 tx=0x7ff2b401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9049c4737503c15 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:26.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.937+0000 7ff2bf400640 1 -- 172.21.15.32:0/85377204 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b40b1070 con 0x7ff2c0144e90 2024-04-17T10:01:26.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.937+0000 7ff2bf400640 1 -- 172.21.15.32:0/85377204 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b40ac620 con 0x7ff2c0144e90 2024-04-17T10:01:26.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.937+0000 7ff2bf400640 1 -- 172.21.15.32:0/85377204 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b40ac9f0 con 0x7ff2c0144e90 2024-04-17T10:01:26.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.937+0000 7ff2c7600640 1 -- 172.21.15.32:0/85377204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c0144e90 msgr2=0x7ff2c00b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:26.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.937+0000 7ff2c7600640 1 --2- 172.21.15.32:0/85377204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c0144e90 0x7ff2c00b48a0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009870 tx=0x7ff2b401c4f0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.937+0000 7ff2c7600640 1 -- 172.21.15.32:0/85377204 shutdown_connections 2024-04-17T10:01:26.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.937+0000 7ff2c7600640 1 --2- 172.21.15.32:0/85377204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2c015c1b0 0x7ff2c006ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.937+0000 7ff2c7600640 1 --2- 172.21.15.32:0/85377204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c0144e90 0x7ff2c00b48a0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.937+0000 7ff2c7600640 1 -- 172.21.15.32:0/85377204 >> 172.21.15.32:0/85377204 conn(0x7ff2c006ba30 msgr2=0x7ff2c006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:26.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.937+0000 7ff2c7600640 1 -- 172.21.15.32:0/85377204 shutdown_connections 2024-04-17T10:01:26.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.937+0000 7ff2c7600640 1 -- 172.21.15.32:0/85377204 wait complete. 2024-04-17T10:01:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.941+0000 7ff2c7600640 1 Processor -- start 2024-04-17T10:01:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.941+0000 7ff2c7600640 1 -- start start 2024-04-17T10:01:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.941+0000 7ff2c7600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2c00b4de0 0x7ff2c015acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.941+0000 7ff2c7600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2c0144e90 0x7ff2c006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.941+0000 7ff2c7600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c015c1b0 0x7ff2c006e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.941+0000 7ff2c7600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c0166100 con 0x7ff2c015c1b0 2024-04-17T10:01:26.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.941+0000 7ff2c7600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c00728c0 con 0x7ff2c00b4de0 2024-04-17T10:01:26.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.941+0000 7ff2c7600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c0166280 con 0x7ff2c0144e90 2024-04-17T10:01:26.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.941+0000 7ff2c5600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c015c1b0 0x7ff2c006e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:26.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.941+0000 7ff2c5600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c015c1b0 0x7ff2c006e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53536/0 (socket says 172.21.15.32:53536) 2024-04-17T10:01:26.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.941+0000 7ff2c5600640 1 -- 172.21.15.32:0/2106522026 learned_addr learned my addr 172.21.15.32:0/2106522026 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:26.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.941+0000 7ff2bfe00640 1 --2- 172.21.15.32:0/2106522026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2c0144e90 0x7ff2c006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:26.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.941+0000 7ff2c4c00640 1 --2- 172.21.15.32:0/2106522026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2c00b4de0 0x7ff2c015acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:26.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.941+0000 7ff2c5600640 1 -- 172.21.15.32:0/2106522026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2c0144e90 msgr2=0x7ff2c006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:26.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.941+0000 7ff2c5600640 1 --2- 172.21.15.32:0/2106522026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2c0144e90 0x7ff2c006de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.941+0000 7ff2c5600640 1 -- 172.21.15.32:0/2106522026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2c00b4de0 msgr2=0x7ff2c015acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:26.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.941+0000 7ff2c5600640 1 --2- 172.21.15.32:0/2106522026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2c00b4de0 0x7ff2c015acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:26.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.941+0000 7ff2c5600640 1 -- 172.21.15.32:0/2106522026 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c013f540 con 0x7ff2c015c1b0 2024-04-17T10:01:26.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.941+0000 7ff2c5600640 1 --2- 172.21.15.32:0/2106522026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c015c1b0 0x7ff2c006e390 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800b970 tx=0x7ff2b800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:26.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.941+0000 7ff2bcc00640 1 -- 172.21.15.32:0/2106522026 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8019070 con 0x7ff2c015c1b0 2024-04-17T10:01:26.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.941+0000 7ff2bcc00640 1 -- 172.21.15.32:0/2106522026 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b800cb60 con 0x7ff2c015c1b0 2024-04-17T10:01:26.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.941+0000 7ff2bcc00640 1 -- 172.21.15.32:0/2106522026 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8014440 con 0x7ff2c015c1b0 2024-04-17T10:01:26.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.941+0000 7ff2c7600640 1 -- 172.21.15.32:0/2106522026 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c013f820 con 0x7ff2c015c1b0 2024-04-17T10:01:26.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.945+0000 7ff2c7600640 1 -- 172.21.15.32:0/2106522026 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c013fde0 con 0x7ff2c015c1b0 2024-04-17T10:01:26.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.945+0000 7ff2c7600640 1 -- 172.21.15.32:0/2106522026 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c0068a90 con 0x7ff2c015c1b0 2024-04-17T10:01:26.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.945+0000 7ff2bcc00640 1 -- 172.21.15.32:0/2106522026 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff2b801c070 con 0x7ff2c015c1b0 2024-04-17T10:01:26.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.945+0000 7ff2bcc00640 1 --2- 172.21.15.32:0/2106522026 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff2a0068380 0x7ff2a006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:26.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.945+0000 7ff2bcc00640 1 -- 172.21.15.32:0/2106522026 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7ff2b8086870 con 0x7ff2c015c1b0 2024-04-17T10:01:26.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.945+0000 7ff2c4c00640 1 --2- 172.21.15.32:0/2106522026 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff2a0068380 0x7ff2a006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:26.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.945+0000 7ff2c4c00640 1 --2- 172.21.15.32:0/2106522026 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff2a0068380 0x7ff2a006a840 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff2b40ae000 tx=0x7ff2b40a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:26.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:26.949+0000 7ff2bcc00640 1 -- 172.21.15.32:0/2106522026 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff2b8056650 con 0x7ff2c015c1b0 2024-04-17T10:01:27.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.149+0000 7ff2c7600640 1 -- 172.21.15.32:0/2106522026 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2c006cdd0 con 0x7ff2c015c1b0 2024-04-17T10:01:27.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.149+0000 7ff2bcc00640 1 -- 172.21.15.32:0/2106522026 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5205 (secure 0 0 0) 0x7ff2b805a950 con 0x7ff2c015c1b0 2024-04-17T10:01:27.148 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:27.148 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":72,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6419,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3534108103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3534108103},{"type":"v1","addr":"172.21.15.144:6835","nonce":3534108103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7003,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/935945019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":935945019},{"type":"v1","addr":"172.21.15.144:6834","nonce":935945019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3044362393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3044362393},{"type":"v1","addr":"172.21.15.32:6836","nonce":3044362393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:59:04.834110+0000","modified":"2024-04-17T10:00:47.860710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8196":{"gid":8196,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6837/2058904361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2058904361},{"type":"v1","addr":"172.21.15.32:6837","nonce":2058904361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8196,"qdb_cluster":[8196]},"id":6}]} 2024-04-17T10:01:27.149 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 72 2024-04-17T10:01:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.153+0000 7ff2c7600640 1 -- 172.21.15.32:0/2106522026 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff2a0068380 msgr2=0x7ff2a006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.153+0000 7ff2c7600640 1 --2- 172.21.15.32:0/2106522026 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff2a0068380 0x7ff2a006a840 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff2b40ae000 tx=0x7ff2b40a7000 comp rx=0 tx=0).stop 2024-04-17T10:01:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.153+0000 7ff2c7600640 1 -- 172.21.15.32:0/2106522026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c015c1b0 msgr2=0x7ff2c006e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.153+0000 7ff2c7600640 1 --2- 172.21.15.32:0/2106522026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c015c1b0 0x7ff2c006e390 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800b970 tx=0x7ff2b800be30 comp rx=0 tx=0).stop 2024-04-17T10:01:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.153+0000 7ff2c7600640 1 -- 172.21.15.32:0/2106522026 shutdown_connections 2024-04-17T10:01:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.153+0000 7ff2c7600640 1 --2- 172.21.15.32:0/2106522026 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff2a0068380 0x7ff2a006a840 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.153+0000 7ff2c7600640 1 --2- 172.21.15.32:0/2106522026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2c0144e90 0x7ff2c006de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.153+0000 7ff2c7600640 1 --2- 172.21.15.32:0/2106522026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c015c1b0 0x7ff2c006e390 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.153+0000 7ff2c7600640 1 --2- 172.21.15.32:0/2106522026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2c00b4de0 0x7ff2c015acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.153+0000 7ff2c7600640 1 -- 172.21.15.32:0/2106522026 >> 172.21.15.32:0/2106522026 conn(0x7ff2c006ba30 msgr2=0x7ff2c00cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:27.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.153+0000 7ff2c7600640 1 -- 172.21.15.32:0/2106522026 shutdown_connections 2024-04-17T10:01:27.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.153+0000 7ff2c7600640 1 -- 172.21.15.32:0/2106522026 wait complete. 2024-04-17T10:01:27.169 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:01:27.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.313+0000 7fface600640 1 Processor -- start 2024-04-17T10:01:27.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.313+0000 7fface600640 1 -- start start 2024-04-17T10:01:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.313+0000 7fface600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ffac81604f0 0x7ffac809ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.313+0000 7fface600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffac809a9f0 con 0x7ffac815e3a0 2024-04-17T10:01:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.313+0000 7fface600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffac809acd0 con 0x7ffac81608d0 2024-04-17T10:01:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.313+0000 7fface600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ffac809b600 con 0x7ffac81604f0 2024-04-17T10:01:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.313+0000 7ffac7e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ffac815e3a0 0x7ffac815e810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50414/0 (socket says 172.21.15.32:50414) 2024-04-17T10:01:27.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.313+0000 7ffac7e00640 1 -- 172.21.15.32:0/3602272032 learned_addr learned my addr 172.21.15.32:0/3602272032 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:27.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.313+0000 7ffac7400640 1 --2- 172.21.15.32:0/3602272032 >> v2:172.21.15.144:3300/0 conn(0x7ffac81604f0 0x7ffac809ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:27.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.313+0000 7ffac6a00640 1 -- 172.21.15.32:0/3602272032 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2306677921 0 0) 0x7ffac809a9f0 con 0x7ffac815e3a0 2024-04-17T10:01:27.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.313+0000 7ffac6a00640 1 -- 172.21.15.32:0/3602272032 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffaac003530 con 0x7ffac815e3a0 2024-04-17T10:01:27.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.313+0000 7ffac6a00640 1 -- 172.21.15.32:0/3602272032 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3977982943 0 0) 0x7ffaac003530 con 0x7ffac815e3a0 2024-04-17T10:01:27.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.313+0000 7ffac6a00640 1 -- 172.21.15.32:0/3602272032 >> v2:172.21.15.144:3300/0 conn(0x7ffac81604f0 msgr2=0x7ffac809ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:27.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.313+0000 7ffac6a00640 1 --2- 172.21.15.32:0/3602272032 >> v2:172.21.15.144:3300/0 conn(0x7ffac81604f0 0x7ffac809ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.313+0000 7ffac6a00640 1 -- 172.21.15.32:0/3602272032 >> v1:172.21.15.144:6789/0 conn(0x7ffac81608d0 legacy=0x7ffac80cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:27.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.313+0000 7ffac6a00640 1 -- 172.21.15.32:0/3602272032 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac809afb0 con 0x7ffac815e3a0 2024-04-17T10:01:27.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.313+0000 7ffac6a00640 1 -- 172.21.15.32:0/3602272032 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7ffac0001a00 con 0x7ffac815e3a0 2024-04-17T10:01:27.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.313+0000 7ffac6a00640 1 -- 172.21.15.32:0/3602272032 >> v1:172.21.15.32:6789/0 conn(0x7ffac815e3a0 legacy=0x7ffac815e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:27.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.313+0000 7ffac6a00640 1 --2- 172.21.15.32:0/3602272032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffaac0039c0 0x7ffaac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:27.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.313+0000 7ffac6a00640 1 --2- 172.21.15.32:0/3602272032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffaac004810 0x7ffaac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:27.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.317+0000 7ffac6a00640 1 --2- 172.21.15.32:0/3602272032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaac0051d0 0x7ffaac00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:27.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.317+0000 7ffac6a00640 1 -- 172.21.15.32:0/3602272032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffac809b600 con 0x7ffaac0051d0 2024-04-17T10:01:27.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.317+0000 7ffac6a00640 1 -- 172.21.15.32:0/3602272032 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac00c0a0 con 0x7ffaac0039c0 2024-04-17T10:01:27.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.317+0000 7ffac6a00640 1 -- 172.21.15.32:0/3602272032 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffaac00c2a0 con 0x7ffaac004810 2024-04-17T10:01:27.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.317+0000 7ffac7e00640 1 --2- 172.21.15.32:0/3602272032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaac0051d0 0x7ffaac00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:27.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.317+0000 7ffac7400640 1 --2- 172.21.15.32:0/3602272032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffaac0039c0 0x7ffaac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:27.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.317+0000 7ffabfe00640 1 --2- 172.21.15.32:0/3602272032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffaac004810 0x7ffaac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:27.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.317+0000 7ffac7e00640 1 -- 172.21.15.32:0/3602272032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffaac004810 msgr2=0x7ffaac004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:27.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.317+0000 7ffac7e00640 1 --2- 172.21.15.32:0/3602272032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffaac004810 0x7ffaac004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.317+0000 7ffac7e00640 1 -- 172.21.15.32:0/3602272032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffaac0039c0 msgr2=0x7ffaac003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:27.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.317+0000 7ffac7e00640 1 --2- 172.21.15.32:0/3602272032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffaac0039c0 0x7ffaac003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.317+0000 7ffac7e00640 1 -- 172.21.15.32:0/3602272032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaac003530 con 0x7ffaac0051d0 2024-04-17T10:01:27.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.317+0000 7ffabfe00640 1 --2- 172.21.15.32:0/3602272032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffaac004810 0x7ffaac004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:27.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.317+0000 7ffac7e00640 1 -- 172.21.15.32:0/3602272032 reap_dead start 2024-04-17T10:01:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.317+0000 7ffac7e00640 1 --2- 172.21.15.32:0/3602272032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaac0051d0 0x7ffaac00b9d0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7ffac000a850 tx=0x7ffac001d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ad2b768b4e8971 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.317+0000 7ffac6a00640 1 -- 172.21.15.32:0/3602272032 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac000a4a0 con 0x7ffaac0051d0 2024-04-17T10:01:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.317+0000 7ffac6a00640 1 -- 172.21.15.32:0/3602272032 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffac00ad650 con 0x7ffaac0051d0 2024-04-17T10:01:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.317+0000 7ffac6a00640 1 -- 172.21.15.32:0/3602272032 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac00ada40 con 0x7ffaac0051d0 2024-04-17T10:01:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7fface600640 1 -- 172.21.15.32:0/3602272032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaac0051d0 msgr2=0x7ffaac00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7fface600640 1 --2- 172.21.15.32:0/3602272032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaac0051d0 0x7ffaac00b9d0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7ffac000a850 tx=0x7ffac001d3a0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7fface600640 1 -- 172.21.15.32:0/3602272032 shutdown_connections 2024-04-17T10:01:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7fface600640 1 --2- 172.21.15.32:0/3602272032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaac0051d0 0x7ffaac00b9d0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7fface600640 1 -- 172.21.15.32:0/3602272032 >> 172.21.15.32:0/3602272032 conn(0x7ffac80b4010 msgr2=0x7ffac80b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7fface600640 1 -- 172.21.15.32:0/3602272032 shutdown_connections 2024-04-17T10:01:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7fface600640 1 -- 172.21.15.32:0/3602272032 wait complete. 2024-04-17T10:01:27.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7fface600640 1 Processor -- start 2024-04-17T10:01:27.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7fface600640 1 -- start start 2024-04-17T10:01:27.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7fface600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffaac0051d0 0x7ffac815ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:27.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7fface600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffac81f4f90 0x7ffac80c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:27.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7fface600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffac8071370 0x7ffac81f7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:27.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7fface600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ffac815de80 con 0x7ffac8071370 2024-04-17T10:01:27.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7fface600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffac8161260 con 0x7ffac81f4f90 2024-04-17T10:01:27.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7fface600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffac81f5410 con 0x7ffaac0051d0 2024-04-17T10:01:27.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7ffac7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffac8071370 0x7ffac81f7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:27.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7ffac7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffac8071370 0x7ffac81f7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53544/0 (socket says 172.21.15.32:53544) 2024-04-17T10:01:27.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7ffac7e00640 1 -- 172.21.15.32:0/4089351499 learned_addr learned my addr 172.21.15.32:0/4089351499 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:27.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7ffac7400640 1 --2- 172.21.15.32:0/4089351499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffaac0051d0 0x7ffac815ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:27.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7ffabfe00640 1 --2- 172.21.15.32:0/4089351499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffac81f4f90 0x7ffac80c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:27.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7ffac7e00640 1 -- 172.21.15.32:0/4089351499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffaac0051d0 msgr2=0x7ffac815ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:27.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7ffac7e00640 1 --2- 172.21.15.32:0/4089351499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffaac0051d0 0x7ffac815ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7ffac7e00640 1 -- 172.21.15.32:0/4089351499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffac81f4f90 msgr2=0x7ffac80c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:27.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7ffac7e00640 1 --2- 172.21.15.32:0/4089351499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffac81f4f90 0x7ffac80c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.321+0000 7ffac7e00640 1 -- 172.21.15.32:0/4089351499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac80c72e0 con 0x7ffac8071370 2024-04-17T10:01:27.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.325+0000 7ffac7e00640 1 --2- 172.21.15.32:0/4089351499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffac8071370 0x7ffac81f7a00 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7ffac001d1d0 tx=0x7ffaac005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:27.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.325+0000 7ffabf400640 1 -- 172.21.15.32:0/4089351499 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac00019c0 con 0x7ffac8071370 2024-04-17T10:01:27.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.325+0000 7ffabf400640 1 -- 172.21.15.32:0/4089351499 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffac00b4070 con 0x7ffac8071370 2024-04-17T10:01:27.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.325+0000 7ffabf400640 1 -- 172.21.15.32:0/4089351499 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac000a480 con 0x7ffac8071370 2024-04-17T10:01:27.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.325+0000 7fface600640 1 -- 172.21.15.32:0/4089351499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac80c7840 con 0x7ffac8071370 2024-04-17T10:01:27.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.325+0000 7fface600640 1 -- 172.21.15.32:0/4089351499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac815a380 con 0x7ffac8071370 2024-04-17T10:01:27.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.325+0000 7fface600640 1 -- 172.21.15.32:0/4089351499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac8068a90 con 0x7ffac8071370 2024-04-17T10:01:27.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.325+0000 7ffabf400640 1 -- 172.21.15.32:0/4089351499 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ffac00bf060 con 0x7ffac8071370 2024-04-17T10:01:27.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.325+0000 7ffabf400640 1 --2- 172.21.15.32:0/4089351499 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffaa40683b0 0x7ffaa406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:27.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.325+0000 7ffabf400640 1 -- 172.21.15.32:0/4089351499 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7ffac0120a50 con 0x7ffac8071370 2024-04-17T10:01:27.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.325+0000 7ffac7400640 1 --2- 172.21.15.32:0/4089351499 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffaa40683b0 0x7ffaa406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:27.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.329+0000 7ffac7400640 1 --2- 172.21.15.32:0/4089351499 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffaa40683b0 0x7ffaa406a870 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ffac8071d20 tx=0x7ffab8009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:27.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.329+0000 7ffabf400640 1 -- 172.21.15.32:0/4089351499 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ffac80c72e0 con 0x7ffac8071370 2024-04-17T10:01:27.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.497+0000 7fface600640 1 -- 172.21.15.32:0/4089351499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffac80b4c70 con 0x7ffac8071370 2024-04-17T10:01:27.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.501+0000 7ffabf400640 1 -- 172.21.15.32:0/4089351499 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+17029 (secure 0 0 0) 0x7ffac00170b0 con 0x7ffac8071370 2024-04-17T10:01:27.497 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:27.497 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":86,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:00:43.842900+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:59:03.223426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-17T09:59:03.780120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.11","mappings":[{"from":1,"to":3}]},{"pgid":"13.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6837/1253497636":"2024-04-18T10:00:43.842876+0000","172.21.15.32:6835/1253497636":"2024-04-18T10:00:43.842876+0000","172.21.15.32:6837/2656735817":"2024-04-18T09:59:28.985107+0000","172.21.15.32:6835/2656735817":"2024-04-18T09:59:28.985107+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:01:27.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.501+0000 7fface600640 1 -- 172.21.15.32:0/4089351499 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffaa40683b0 msgr2=0x7ffaa406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:27.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.501+0000 7fface600640 1 --2- 172.21.15.32:0/4089351499 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffaa40683b0 0x7ffaa406a870 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ffac8071d20 tx=0x7ffab8009190 comp rx=0 tx=0).stop 2024-04-17T10:01:27.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.501+0000 7fface600640 1 -- 172.21.15.32:0/4089351499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffac8071370 msgr2=0x7ffac81f7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:27.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.501+0000 7fface600640 1 --2- 172.21.15.32:0/4089351499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffac8071370 0x7ffac81f7a00 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7ffac001d1d0 tx=0x7ffaac005840 comp rx=0 tx=0).stop 2024-04-17T10:01:27.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.501+0000 7fface600640 1 -- 172.21.15.32:0/4089351499 shutdown_connections 2024-04-17T10:01:27.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.501+0000 7fface600640 1 --2- 172.21.15.32:0/4089351499 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ffaa40683b0 0x7ffaa406a870 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.501+0000 7fface600640 1 --2- 172.21.15.32:0/4089351499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffac8071370 0x7ffac81f7a00 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.501+0000 7fface600640 1 --2- 172.21.15.32:0/4089351499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffac81f4f90 0x7ffac80c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.501+0000 7fface600640 1 --2- 172.21.15.32:0/4089351499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffaac0051d0 0x7ffac815ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.501+0000 7fface600640 1 -- 172.21.15.32:0/4089351499 >> 172.21.15.32:0/4089351499 conn(0x7ffac80b4010 msgr2=0x7ffac80701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:27.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.501+0000 7fface600640 1 -- 172.21.15.32:0/4089351499 shutdown_connections 2024-04-17T10:01:27.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.501+0000 7fface600640 1 -- 172.21.15.32:0/4089351499 wait complete. 2024-04-17T10:01:27.519 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T10:01:27.519 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:01:27.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.661+0000 7f08d3e00640 1 Processor -- start 2024-04-17T10:01:27.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.661+0000 7f08d3e00640 1 -- start start 2024-04-17T10:01:27.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.661+0000 7f08d3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08cc144e90 0x7f08cc0b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:27.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.661+0000 7f08d3e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08cc15c1b0 0x7f08cc06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:27.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.661+0000 7f08d3e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08cc072bb0 con 0x7f08cc0b4de0 2024-04-17T10:01:27.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.661+0000 7f08d3e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f08cc145270 con 0x7f08cc144e90 2024-04-17T10:01:27.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.661+0000 7f08d3e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f08cc0b3dc0 con 0x7f08cc15c1b0 2024-04-17T10:01:27.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.661+0000 7f08d1400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08cc144e90 0x7f08cc0b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:27.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.665+0000 7f08d1400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08cc144e90 0x7f08cc0b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53552/0 (socket says 172.21.15.32:53552) 2024-04-17T10:01:27.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.665+0000 7f08d1400640 1 -- 172.21.15.32:0/1024638631 learned_addr learned my addr 172.21.15.32:0/1024638631 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:27.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.665+0000 7f08d1e00640 1 --2- 172.21.15.32:0/1024638631 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08cc15c1b0 0x7f08cc06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:27.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.665+0000 7f08d1400640 1 -- 172.21.15.32:0/1024638631 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08cc15c1b0 msgr2=0x7f08cc06ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:27.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.665+0000 7f08d1400640 1 --2- 172.21.15.32:0/1024638631 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08cc15c1b0 0x7f08cc06ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.665+0000 7f08d1400640 1 -- 172.21.15.32:0/1024638631 >> v1:172.21.15.144:6789/0 conn(0x7f08cc0b4de0 legacy=0x7f08cc15baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:27.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.665+0000 7f08d1400640 1 -- 172.21.15.32:0/1024638631 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08cc0b40a0 con 0x7f08cc144e90 2024-04-17T10:01:27.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.665+0000 7f08d1e00640 1 --2- 172.21.15.32:0/1024638631 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08cc15c1b0 0x7f08cc06ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:27.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.665+0000 7f08d1400640 1 --2- 172.21.15.32:0/1024638631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08cc144e90 0x7f08cc0b48a0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009870 tx=0x7f08c001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2d48747d2c54be server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:27.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.665+0000 7f08cbe00640 1 -- 172.21.15.32:0/1024638631 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c00ab640 con 0x7f08cc144e90 2024-04-17T10:01:27.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.665+0000 7f08cbe00640 1 -- 172.21.15.32:0/1024638631 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c00b0070 con 0x7f08cc144e90 2024-04-17T10:01:27.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.665+0000 7f08cbe00640 1 -- 172.21.15.32:0/1024638631 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c00aba00 con 0x7f08cc144e90 2024-04-17T10:01:27.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d3e00640 1 -- 172.21.15.32:0/1024638631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08cc144e90 msgr2=0x7f08cc0b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:27.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d3e00640 1 --2- 172.21.15.32:0/1024638631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08cc144e90 0x7f08cc0b48a0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009870 tx=0x7f08c001c530 comp rx=0 tx=0).stop 2024-04-17T10:01:27.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d3e00640 1 -- 172.21.15.32:0/1024638631 shutdown_connections 2024-04-17T10:01:27.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d3e00640 1 --2- 172.21.15.32:0/1024638631 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08cc15c1b0 0x7f08cc06ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d3e00640 1 --2- 172.21.15.32:0/1024638631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08cc144e90 0x7f08cc0b48a0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d3e00640 1 -- 172.21.15.32:0/1024638631 >> 172.21.15.32:0/1024638631 conn(0x7f08cc06ba30 msgr2=0x7f08cc06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:27.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d3e00640 1 -- 172.21.15.32:0/1024638631 shutdown_connections 2024-04-17T10:01:27.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d3e00640 1 -- 172.21.15.32:0/1024638631 wait complete. 2024-04-17T10:01:27.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d3e00640 1 Processor -- start 2024-04-17T10:01:27.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d3e00640 1 -- start start 2024-04-17T10:01:27.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08cc0b4de0 0x7f08cc1faf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:27.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08cc144e90 0x7f08cc1f6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:27.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d3e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08cc15c1b0 0x7f08cc0b0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:27.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d3e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc168110 con 0x7f08cc0b4de0 2024-04-17T10:01:27.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d3e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc072880 con 0x7f08cc144e90 2024-04-17T10:01:27.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d3e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f08cc168290 con 0x7f08cc15c1b0 2024-04-17T10:01:27.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d1400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08cc0b4de0 0x7f08cc1faf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:27.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d1400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08cc0b4de0 0x7f08cc1faf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53564/0 (socket says 172.21.15.32:53564) 2024-04-17T10:01:27.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d1400640 1 -- 172.21.15.32:0/3481061351 learned_addr learned my addr 172.21.15.32:0/3481061351 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:27.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d1e00640 1 --2- 172.21.15.32:0/3481061351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08cc15c1b0 0x7f08cc0b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:27.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d0a00640 1 --2- 172.21.15.32:0/3481061351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08cc144e90 0x7f08cc1f6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:27.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d1400640 1 -- 172.21.15.32:0/3481061351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08cc15c1b0 msgr2=0x7f08cc0b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:27.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d1400640 1 --2- 172.21.15.32:0/3481061351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08cc15c1b0 0x7f08cc0b0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d1400640 1 -- 172.21.15.32:0/3481061351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08cc144e90 msgr2=0x7f08cc1f6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:27.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d1400640 1 --2- 172.21.15.32:0/3481061351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08cc144e90 0x7f08cc1f6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d1400640 1 -- 172.21.15.32:0/3481061351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08cc1fb450 con 0x7f08cc0b4de0 2024-04-17T10:01:27.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d0a00640 1 --2- 172.21.15.32:0/3481061351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08cc144e90 0x7f08cc1f6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:27.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.669+0000 7f08d1400640 1 --2- 172.21.15.32:0/3481061351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08cc0b4de0 0x7f08cc1faf10 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f08c00ad040 tx=0x7f08c001ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:27.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.673+0000 7f08c9600640 1 -- 172.21.15.32:0/3481061351 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c00b4800 con 0x7f08cc0b4de0 2024-04-17T10:01:27.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.673+0000 7f08c9600640 1 -- 172.21.15.32:0/3481061351 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c00b49c0 con 0x7f08cc0b4de0 2024-04-17T10:01:27.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.673+0000 7f08d3e00640 1 -- 172.21.15.32:0/3481061351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08cc13e2d0 con 0x7f08cc0b4de0 2024-04-17T10:01:27.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.673+0000 7f08c9600640 1 -- 172.21.15.32:0/3481061351 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c00b4db0 con 0x7f08cc0b4de0 2024-04-17T10:01:27.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.673+0000 7f08d3e00640 1 -- 172.21.15.32:0/3481061351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08cc13e890 con 0x7f08cc0b4de0 2024-04-17T10:01:27.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.673+0000 7f08d3e00640 1 -- 172.21.15.32:0/3481061351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08cc068a90 con 0x7f08cc0b4de0 2024-04-17T10:01:27.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.673+0000 7f08c9600640 1 -- 172.21.15.32:0/3481061351 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f08c00c7440 con 0x7f08cc0b4de0 2024-04-17T10:01:27.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.673+0000 7f08c9600640 1 --2- 172.21.15.32:0/3481061351 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f08ac068380 0x7f08ac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:27.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.673+0000 7f08c9600640 1 -- 172.21.15.32:0/3481061351 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f08c01214a0 con 0x7f08cc0b4de0 2024-04-17T10:01:27.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.673+0000 7f08d0a00640 1 --2- 172.21.15.32:0/3481061351 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f08ac068380 0x7f08ac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:27.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.673+0000 7f08d0a00640 1 --2- 172.21.15.32:0/3481061351 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f08ac068380 0x7f08ac06a840 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0ccc90 tx=0x7f08b8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:27.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.677+0000 7f08c9600640 1 -- 172.21.15.32:0/3481061351 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f08c0016150 con 0x7f08cc0b4de0 2024-04-17T10:01:27.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.873+0000 7f08d3e00640 1 -- 172.21.15.32:0/3481061351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08cc13fd70 con 0x7f08cc0b4de0 2024-04-17T10:01:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.873+0000 7f08c9600640 1 -- 172.21.15.32:0/3481061351 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5205 (secure 0 0 0) 0x7f08c00f1310 con 0x7f08cc0b4de0 2024-04-17T10:01:27.872 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:27.872 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":72,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6419,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3534108103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3534108103},{"type":"v1","addr":"172.21.15.144:6835","nonce":3534108103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7003,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/935945019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":935945019},{"type":"v1","addr":"172.21.15.144:6834","nonce":935945019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3044362393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3044362393},{"type":"v1","addr":"172.21.15.32:6836","nonce":3044362393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:59:04.834110+0000","modified":"2024-04-17T10:00:47.860710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8196":{"gid":8196,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6837/2058904361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2058904361},{"type":"v1","addr":"172.21.15.32:6837","nonce":2058904361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8196,"qdb_cluster":[8196]},"id":6}]} 2024-04-17T10:01:27.873 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 72 2024-04-17T10:01:27.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.877+0000 7f08d3e00640 1 -- 172.21.15.32:0/3481061351 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f08ac068380 msgr2=0x7f08ac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:27.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.877+0000 7f08d3e00640 1 --2- 172.21.15.32:0/3481061351 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f08ac068380 0x7f08ac06a840 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0ccc90 tx=0x7f08b8009040 comp rx=0 tx=0).stop 2024-04-17T10:01:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.877+0000 7f08d3e00640 1 -- 172.21.15.32:0/3481061351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08cc0b4de0 msgr2=0x7f08cc1faf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.877+0000 7f08d3e00640 1 --2- 172.21.15.32:0/3481061351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08cc0b4de0 0x7f08cc1faf10 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f08c00ad040 tx=0x7f08c001ad70 comp rx=0 tx=0).stop 2024-04-17T10:01:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.877+0000 7f08d3e00640 1 -- 172.21.15.32:0/3481061351 shutdown_connections 2024-04-17T10:01:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.877+0000 7f08d3e00640 1 --2- 172.21.15.32:0/3481061351 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f08ac068380 0x7f08ac06a840 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.877+0000 7f08d3e00640 1 --2- 172.21.15.32:0/3481061351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08cc15c1b0 0x7f08cc0b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.877+0000 7f08d3e00640 1 --2- 172.21.15.32:0/3481061351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08cc144e90 0x7f08cc1f6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.877+0000 7f08d3e00640 1 --2- 172.21.15.32:0/3481061351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08cc0b4de0 0x7f08cc1faf10 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.877+0000 7f08d3e00640 1 -- 172.21.15.32:0/3481061351 >> 172.21.15.32:0/3481061351 conn(0x7f08cc06ba30 msgr2=0x7f08cc0b51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.877+0000 7f08d3e00640 1 -- 172.21.15.32:0/3481061351 shutdown_connections 2024-04-17T10:01:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:27.877+0000 7f08d3e00640 1 -- 172.21.15.32:0/3481061351 wait complete. 2024-04-17T10:01:27.893 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:01:28.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.037+0000 7f2fd6c00640 1 Processor -- start 2024-04-17T10:01:28.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.037+0000 7f2fd6c00640 1 -- start start 2024-04-17T10:01:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.037+0000 7f2fd6c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fd015e870 0x7f2fd0162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.037+0000 7f2fd6c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2fd00b0320 0x7f2fd00d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.037+0000 7f2fd6c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fd0163340 con 0x7f2fd00d0780 2024-04-17T10:01:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.037+0000 7f2fd6c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2fd00cf810 con 0x7f2fd00b0320 2024-04-17T10:01:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.037+0000 7f2fd6c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd0067dd0 con 0x7f2fd015e870 2024-04-17T10:01:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.037+0000 7f2fc7400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2fd00b0320 0x7f2fd00d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.037+0000 7f2fd4c00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2fd00d0780 0x7f2fd006f7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50418/0 (socket says 172.21.15.32:50418) 2024-04-17T10:01:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.037+0000 7f2fd4c00640 1 -- 172.21.15.32:0/2974908256 learned_addr learned my addr 172.21.15.32:0/2974908256 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.037+0000 7f2fcfe00640 1 --2- 172.21.15.32:0/2974908256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fd015e870 0x7f2fd0162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.037+0000 7f2fc7400640 1 -- 172.21.15.32:0/2974908256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fd015e870 msgr2=0x7f2fd0162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.037+0000 7f2fc7400640 1 --2- 172.21.15.32:0/2974908256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fd015e870 0x7f2fd0162e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.037+0000 7f2fc7400640 1 -- 172.21.15.32:0/2974908256 >> v1:172.21.15.32:6789/0 conn(0x7f2fd00d0780 legacy=0x7f2fd006f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.037+0000 7f2fcf400640 1 -- 172.21.15.32:0/2974908256 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1789649115 0 0) 0x7f2fd0163340 con 0x7f2fd00d0780 2024-04-17T10:01:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.037+0000 7f2fc7400640 1 -- 172.21.15.32:0/2974908256 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd015ec50 con 0x7f2fd00b0320 2024-04-17T10:01:28.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.041+0000 7f2fc7400640 1 --2- 172.21.15.32:0/2974908256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fd00b0320 0x7f2fd00d0240 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f2fb80097c0 tx=0x7f2fb801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f223440c8a07db9f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:28.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.041+0000 7f2fcf400640 1 -- 172.21.15.32:0/2974908256 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb80b2070 con 0x7f2fd00b0320 2024-04-17T10:01:28.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.041+0000 7f2fcf400640 1 -- 172.21.15.32:0/2974908256 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb80ad640 con 0x7f2fd00b0320 2024-04-17T10:01:28.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.041+0000 7f2fcf400640 1 -- 172.21.15.32:0/2974908256 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb80ada10 con 0x7f2fd00b0320 2024-04-17T10:01:28.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.041+0000 7f2fd6c00640 1 -- 172.21.15.32:0/2974908256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fd00b0320 msgr2=0x7f2fd00d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:28.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.041+0000 7f2fd6c00640 1 --2- 172.21.15.32:0/2974908256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fd00b0320 0x7f2fd00d0240 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f2fb80097c0 tx=0x7f2fb801c4a0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.041+0000 7f2fd6c00640 1 -- 172.21.15.32:0/2974908256 shutdown_connections 2024-04-17T10:01:28.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.041+0000 7f2fd6c00640 1 --2- 172.21.15.32:0/2974908256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fd00b0320 0x7f2fd00d0240 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.041+0000 7f2fd6c00640 1 --2- 172.21.15.32:0/2974908256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fd015e870 0x7f2fd0162e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.041+0000 7f2fd6c00640 1 -- 172.21.15.32:0/2974908256 >> 172.21.15.32:0/2974908256 conn(0x7f2fd00d0c00 msgr2=0x7f2fd00d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:28.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.041+0000 7f2fd6c00640 1 -- 172.21.15.32:0/2974908256 shutdown_connections 2024-04-17T10:01:28.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.041+0000 7f2fd6c00640 1 -- 172.21.15.32:0/2974908256 wait complete. 2024-04-17T10:01:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fd6c00640 1 Processor -- start 2024-04-17T10:01:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fd6c00640 1 -- start start 2024-04-17T10:01:28.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fd6c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fd00b0320 0x7f2fd0150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:28.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fd6c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fd015e870 0x7f2fd014ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:28.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fd6c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fd014f350 0x7f2fd00adf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:28.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fd6c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd006c3c0 con 0x7f2fd014f350 2024-04-17T10:01:28.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fd6c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd006c240 con 0x7f2fd00b0320 2024-04-17T10:01:28.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fd6c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd006fed0 con 0x7f2fd015e870 2024-04-17T10:01:28.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fd4c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fd014f350 0x7f2fd00adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:28.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fd4c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fd014f350 0x7f2fd00adf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53592/0 (socket says 172.21.15.32:53592) 2024-04-17T10:01:28.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fd4c00640 1 -- 172.21.15.32:0/3788877878 learned_addr learned my addr 172.21.15.32:0/3788877878 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:28.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fc7400640 1 --2- 172.21.15.32:0/3788877878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fd015e870 0x7f2fd014ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:28.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fcfe00640 1 --2- 172.21.15.32:0/3788877878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fd00b0320 0x7f2fd0150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:28.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fd4c00640 1 -- 172.21.15.32:0/3788877878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fd015e870 msgr2=0x7f2fd014ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:28.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fd4c00640 1 --2- 172.21.15.32:0/3788877878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fd015e870 0x7f2fd014ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fd4c00640 1 -- 172.21.15.32:0/3788877878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fd00b0320 msgr2=0x7f2fd0150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:28.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fd4c00640 1 --2- 172.21.15.32:0/3788877878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fd00b0320 0x7f2fd0150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fd4c00640 1 -- 172.21.15.32:0/3788877878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd01592e0 con 0x7f2fd014f350 2024-04-17T10:01:28.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fc7400640 1 --2- 172.21.15.32:0/3788877878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fd015e870 0x7f2fd014ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:28.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fd4c00640 1 --2- 172.21.15.32:0/3788877878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fd014f350 0x7f2fd00adf80 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800c7a0 tx=0x7f2fc800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:28.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fccc00640 1 -- 172.21.15.32:0/3788877878 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8018070 con 0x7f2fd014f350 2024-04-17T10:01:28.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fccc00640 1 -- 172.21.15.32:0/3788877878 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc800a640 con 0x7f2fd014f350 2024-04-17T10:01:28.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fccc00640 1 -- 172.21.15.32:0/3788877878 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc800aab0 con 0x7f2fd014f350 2024-04-17T10:01:28.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fd6c00640 1 -- 172.21.15.32:0/3788877878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd014f930 con 0x7f2fd014f350 2024-04-17T10:01:28.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.045+0000 7f2fd6c00640 1 -- 172.21.15.32:0/3788877878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd0150950 con 0x7f2fd014f350 2024-04-17T10:01:28.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.049+0000 7f2fd6c00640 1 -- 172.21.15.32:0/3788877878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd0068a90 con 0x7f2fd014f350 2024-04-17T10:01:28.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.049+0000 7f2fccc00640 1 -- 172.21.15.32:0/3788877878 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2fc800ac50 con 0x7f2fd014f350 2024-04-17T10:01:28.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.049+0000 7f2fccc00640 1 --2- 172.21.15.32:0/3788877878 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2fac068380 0x7f2fac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:28.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.049+0000 7f2fccc00640 1 -- 172.21.15.32:0/3788877878 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f2fc8086100 con 0x7f2fd014f350 2024-04-17T10:01:28.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.049+0000 7f2fcfe00640 1 --2- 172.21.15.32:0/3788877878 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2fac068380 0x7f2fac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:28.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.049+0000 7f2fcfe00640 1 --2- 172.21.15.32:0/3788877878 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2fac068380 0x7f2fac06a840 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009a50 tx=0x7f2fc0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:28.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.057+0000 7f2fccc00640 1 -- 172.21.15.32:0/3788877878 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2fc8056660 con 0x7f2fd014f350 2024-04-17T10:01:28.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.253+0000 7f2fd6c00640 1 -- 172.21.15.32:0/3788877878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fd0073690 con 0x7f2fd014f350 2024-04-17T10:01:28.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.253+0000 7f2fccc00640 1 -- 172.21.15.32:0/3788877878 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5205 (secure 0 0 0) 0x7f2fc8055f60 con 0x7f2fd014f350 2024-04-17T10:01:28.253 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:28.253 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":72,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6419,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3534108103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3534108103},{"type":"v1","addr":"172.21.15.144:6835","nonce":3534108103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7003,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/935945019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":935945019},{"type":"v1","addr":"172.21.15.144:6834","nonce":935945019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3044362393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3044362393},{"type":"v1","addr":"172.21.15.32:6836","nonce":3044362393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T09:59:04.834110+0000","modified":"2024-04-17T10:00:47.860710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8196":{"gid":8196,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6837/2058904361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2058904361},{"type":"v1","addr":"172.21.15.32:6837","nonce":2058904361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8196,"qdb_cluster":[8196]},"id":6}]} 2024-04-17T10:01:28.253 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 72 2024-04-17T10:01:28.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.257+0000 7f2fd6c00640 1 -- 172.21.15.32:0/3788877878 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2fac068380 msgr2=0x7f2fac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:28.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.257+0000 7f2fd6c00640 1 --2- 172.21.15.32:0/3788877878 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2fac068380 0x7f2fac06a840 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009a50 tx=0x7f2fc0008040 comp rx=0 tx=0).stop 2024-04-17T10:01:28.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.257+0000 7f2fd6c00640 1 -- 172.21.15.32:0/3788877878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fd014f350 msgr2=0x7f2fd00adf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:28.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.257+0000 7f2fd6c00640 1 --2- 172.21.15.32:0/3788877878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fd014f350 0x7f2fd00adf80 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800c7a0 tx=0x7f2fc800cc60 comp rx=0 tx=0).stop 2024-04-17T10:01:28.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.257+0000 7f2fd6c00640 1 -- 172.21.15.32:0/3788877878 shutdown_connections 2024-04-17T10:01:28.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.257+0000 7f2fd6c00640 1 --2- 172.21.15.32:0/3788877878 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2fac068380 0x7f2fac06a840 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.257+0000 7f2fd6c00640 1 --2- 172.21.15.32:0/3788877878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fd015e870 0x7f2fd014ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.257+0000 7f2fd6c00640 1 --2- 172.21.15.32:0/3788877878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fd014f350 0x7f2fd00adf80 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.257+0000 7f2fd6c00640 1 --2- 172.21.15.32:0/3788877878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fd00b0320 0x7f2fd0150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.257+0000 7f2fd6c00640 1 -- 172.21.15.32:0/3788877878 >> 172.21.15.32:0/3788877878 conn(0x7f2fd00d0c00 msgr2=0x7f2fd00d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:28.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.257+0000 7f2fd6c00640 1 -- 172.21.15.32:0/3788877878 shutdown_connections 2024-04-17T10:01:28.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.257+0000 7f2fd6c00640 1 -- 172.21.15.32:0/3788877878 wait complete. 2024-04-17T10:01:28.275 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:01:28.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.417+0000 7f97ef600640 1 Processor -- start 2024-04-17T10:01:28.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.417+0000 7f97ef600640 1 -- start start 2024-04-17T10:01:28.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.421+0000 7f97ef600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f97e8144e90 0x7f97e80b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:28.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.421+0000 7f97ef600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97e815c1b0 0x7f97e806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:28.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.421+0000 7f97ef600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97e8072bb0 con 0x7f97e80b4de0 2024-04-17T10:01:28.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.421+0000 7f97ef600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f97e8145270 con 0x7f97e8144e90 2024-04-17T10:01:28.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.421+0000 7f97ef600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f97e80b3dc0 con 0x7f97e815c1b0 2024-04-17T10:01:28.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.421+0000 7f97ecc00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f97e8144e90 0x7f97e80b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:28.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.421+0000 7f97ecc00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f97e8144e90 0x7f97e80b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53600/0 (socket says 172.21.15.32:53600) 2024-04-17T10:01:28.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.421+0000 7f97ecc00640 1 -- 172.21.15.32:0/3967802723 learned_addr learned my addr 172.21.15.32:0/3967802723 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:28.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.421+0000 7f97ed600640 1 --2- 172.21.15.32:0/3967802723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97e815c1b0 0x7f97e806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:28.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.421+0000 7f97ecc00640 1 -- 172.21.15.32:0/3967802723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97e815c1b0 msgr2=0x7f97e806ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:28.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.421+0000 7f97ecc00640 1 --2- 172.21.15.32:0/3967802723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97e815c1b0 0x7f97e806ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.421+0000 7f97ecc00640 1 -- 172.21.15.32:0/3967802723 >> v1:172.21.15.144:6789/0 conn(0x7f97e80b4de0 legacy=0x7f97e815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:28.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.421+0000 7f97ecc00640 1 -- 172.21.15.32:0/3967802723 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e80b40a0 con 0x7f97e8144e90 2024-04-17T10:01:28.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.421+0000 7f97ed600640 1 --2- 172.21.15.32:0/3967802723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97e815c1b0 0x7f97e806ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:28.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.421+0000 7f97ecc00640 1 --2- 172.21.15.32:0/3967802723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97e8144e90 0x7f97e80b48a0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009870 tx=0x7f97dc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b668c8e159183dfb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:28.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97e7400640 1 -- 172.21.15.32:0/3967802723 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc0b1070 con 0x7f97e8144e90 2024-04-17T10:01:28.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97e7400640 1 -- 172.21.15.32:0/3967802723 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97dc0ac620 con 0x7f97e8144e90 2024-04-17T10:01:28.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97e7400640 1 -- 172.21.15.32:0/3967802723 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc0ac9f0 con 0x7f97e8144e90 2024-04-17T10:01:28.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97ef600640 1 -- 172.21.15.32:0/3967802723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97e8144e90 msgr2=0x7f97e80b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:28.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97ef600640 1 --2- 172.21.15.32:0/3967802723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97e8144e90 0x7f97e80b48a0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009870 tx=0x7f97dc01c4f0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97ef600640 1 -- 172.21.15.32:0/3967802723 shutdown_connections 2024-04-17T10:01:28.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97ef600640 1 --2- 172.21.15.32:0/3967802723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97e815c1b0 0x7f97e806ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97ef600640 1 --2- 172.21.15.32:0/3967802723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97e8144e90 0x7f97e80b48a0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97ef600640 1 -- 172.21.15.32:0/3967802723 >> 172.21.15.32:0/3967802723 conn(0x7f97e806ba30 msgr2=0x7f97e806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:28.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97ef600640 1 -- 172.21.15.32:0/3967802723 shutdown_connections 2024-04-17T10:01:28.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97ef600640 1 -- 172.21.15.32:0/3967802723 wait complete. 2024-04-17T10:01:28.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97ef600640 1 Processor -- start 2024-04-17T10:01:28.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97ef600640 1 -- start start 2024-04-17T10:01:28.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97ef600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97e80b4de0 0x7f97e8165b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:28.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97ef600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97e8144e90 0x7f97e8142c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:28.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97ef600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97e815c1b0 0x7f97e8162c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:28.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97ef600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f97e8168110 con 0x7f97e80b4de0 2024-04-17T10:01:28.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97ef600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f97e8072880 con 0x7f97e8144e90 2024-04-17T10:01:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97ef600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f97e8168290 con 0x7f97e815c1b0 2024-04-17T10:01:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97ecc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97e80b4de0 0x7f97e8165b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97ecc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97e80b4de0 0x7f97e8165b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53604/0 (socket says 172.21.15.32:53604) 2024-04-17T10:01:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97ecc00640 1 -- 172.21.15.32:0/3403042922 learned_addr learned my addr 172.21.15.32:0/3403042922 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97e7e00640 1 --2- 172.21.15.32:0/3403042922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97e8144e90 0x7f97e8142c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.425+0000 7f97ed600640 1 --2- 172.21.15.32:0/3403042922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97e815c1b0 0x7f97e8162c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.429+0000 7f97ecc00640 1 -- 172.21.15.32:0/3403042922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97e815c1b0 msgr2=0x7f97e8162c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:28.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.429+0000 7f97ecc00640 1 --2- 172.21.15.32:0/3403042922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97e815c1b0 0x7f97e8162c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.429+0000 7f97ecc00640 1 -- 172.21.15.32:0/3403042922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97e8144e90 msgr2=0x7f97e8142c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:28.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.429+0000 7f97ecc00640 1 --2- 172.21.15.32:0/3403042922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97e8144e90 0x7f97e8142c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.429+0000 7f97ecc00640 1 -- 172.21.15.32:0/3403042922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e8143190 con 0x7f97e80b4de0 2024-04-17T10:01:28.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.429+0000 7f97ecc00640 1 --2- 172.21.15.32:0/3403042922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97e80b4de0 0x7f97e8165b40 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f97dc01aba0 tx=0x7f97dc019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:28.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.429+0000 7f97e4c00640 1 -- 172.21.15.32:0/3403042922 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc0b1040 con 0x7f97e80b4de0 2024-04-17T10:01:28.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.429+0000 7f97e4c00640 1 -- 172.21.15.32:0/3403042922 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97dc0ba070 con 0x7f97e80b4de0 2024-04-17T10:01:28.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.429+0000 7f97e4c00640 1 -- 172.21.15.32:0/3403042922 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc0b5880 con 0x7f97e80b4de0 2024-04-17T10:01:28.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.429+0000 7f97ef600640 1 -- 172.21.15.32:0/3403042922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97e8143460 con 0x7f97e80b4de0 2024-04-17T10:01:28.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.429+0000 7f97ef600640 1 -- 172.21.15.32:0/3403042922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97e8143a20 con 0x7f97e80b4de0 2024-04-17T10:01:28.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.429+0000 7f97ef600640 1 -- 172.21.15.32:0/3403042922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97e8068a90 con 0x7f97e80b4de0 2024-04-17T10:01:28.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.429+0000 7f97e4c00640 1 -- 172.21.15.32:0/3403042922 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f97dc0ac4e0 con 0x7f97e80b4de0 2024-04-17T10:01:28.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.429+0000 7f97e4c00640 1 --2- 172.21.15.32:0/3403042922 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f97c8068380 0x7f97c806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:28.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.429+0000 7f97e4c00640 1 -- 172.21.15.32:0/3403042922 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7f97dc1209c0 con 0x7f97e80b4de0 2024-04-17T10:01:28.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.429+0000 7f97e7e00640 1 --2- 172.21.15.32:0/3403042922 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f97c8068380 0x7f97c806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:28.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.433+0000 7f97e7e00640 1 --2- 172.21.15.32:0/3403042922 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f97c8068380 0x7f97c806a840 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f97e81fade0 tx=0x7f97d4007e40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:28.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.433+0000 7f97e4c00640 1 -- 172.21.15.32:0/3403042922 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f97dc016110 con 0x7f97e80b4de0 2024-04-17T10:01:28.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.605+0000 7f97ef600640 1 -- 172.21.15.32:0/3403042922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97e806e900 con 0x7f97e80b4de0 2024-04-17T10:01:28.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.605+0000 7f97e4c00640 1 -- 172.21.15.32:0/3403042922 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+17029 (secure 0 0 0) 0x7f97dc0f07a0 con 0x7f97e80b4de0 2024-04-17T10:01:28.602 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:28.602 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":86,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:00:43.842900+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-17T09:59:03.223426+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-17T09:59:03.780120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.11","mappings":[{"from":1,"to":3}]},{"pgid":"13.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6837/1253497636":"2024-04-18T10:00:43.842876+0000","172.21.15.32:6835/1253497636":"2024-04-18T10:00:43.842876+0000","172.21.15.32:6837/2656735817":"2024-04-18T09:59:28.985107+0000","172.21.15.32:6835/2656735817":"2024-04-18T09:59:28.985107+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:01:28.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.605+0000 7f97ef600640 1 -- 172.21.15.32:0/3403042922 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f97c8068380 msgr2=0x7f97c806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:28.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.605+0000 7f97ef600640 1 --2- 172.21.15.32:0/3403042922 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f97c8068380 0x7f97c806a840 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f97e81fade0 tx=0x7f97d4007e40 comp rx=0 tx=0).stop 2024-04-17T10:01:28.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.605+0000 7f97ef600640 1 -- 172.21.15.32:0/3403042922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97e80b4de0 msgr2=0x7f97e8165b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:28.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.605+0000 7f97ef600640 1 --2- 172.21.15.32:0/3403042922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97e80b4de0 0x7f97e8165b40 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f97dc01aba0 tx=0x7f97dc019290 comp rx=0 tx=0).stop 2024-04-17T10:01:28.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.605+0000 7f97ef600640 1 -- 172.21.15.32:0/3403042922 shutdown_connections 2024-04-17T10:01:28.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.605+0000 7f97ef600640 1 --2- 172.21.15.32:0/3403042922 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f97c8068380 0x7f97c806a840 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.605+0000 7f97ef600640 1 --2- 172.21.15.32:0/3403042922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97e815c1b0 0x7f97e8162c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.605+0000 7f97ef600640 1 --2- 172.21.15.32:0/3403042922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97e8144e90 0x7f97e8142c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.605+0000 7f97ef600640 1 --2- 172.21.15.32:0/3403042922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97e80b4de0 0x7f97e8165b40 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.605+0000 7f97ef600640 1 -- 172.21.15.32:0/3403042922 >> 172.21.15.32:0/3403042922 conn(0x7f97e806ba30 msgr2=0x7f97e80ae4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:28.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.605+0000 7f97ef600640 1 -- 172.21.15.32:0/3403042922 shutdown_connections 2024-04-17T10:01:28.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.605+0000 7f97ef600640 1 -- 172.21.15.32:0/3403042922 wait complete. 2024-04-17T10:01:28.624 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T10:01:28.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.769+0000 7fc23b000640 1 Processor -- start 2024-04-17T10:01:28.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.769+0000 7fc23b000640 1 -- start start 2024-04-17T10:01:28.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.769+0000 7fc23b000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc23413f6e0 0x7fc2340cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:28.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.769+0000 7fc23b000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2340cf370 0x7fc2340d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:28.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.769+0000 7fc23b000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2340d0820 0x7fc234141d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.769+0000 7fc23b000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc23413fac0 con 0x7fc2340d0820 2024-04-17T10:01:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.769+0000 7fc23b000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc2341601e0 con 0x7fc23413f6e0 2024-04-17T10:01:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.769+0000 7fc23b000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc234160480 con 0x7fc2340cf370 2024-04-17T10:01:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.769+0000 7fc239000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2340d0820 0x7fc234141d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.769+0000 7fc239000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2340d0820 0x7fc234141d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53606/0 (socket says 172.21.15.32:53606) 2024-04-17T10:01:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.769+0000 7fc239000640 1 -- 172.21.15.32:0/3347989182 learned_addr learned my addr 172.21.15.32:0/3347989182 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.769+0000 7fc233e00640 1 --2- 172.21.15.32:0/3347989182 >> v2:172.21.15.144:3300/0 conn(0x7fc23413f6e0 0x7fc2340cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.769+0000 7fc22b400640 1 --2- 172.21.15.32:0/3347989182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2340cf370 0x7fc2340d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.769+0000 7fc239000640 1 -- 172.21.15.32:0/3347989182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2340cf370 msgr2=0x7fc2340d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.769+0000 7fc239000640 1 --2- 172.21.15.32:0/3347989182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2340cf370 0x7fc2340d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.769+0000 7fc239000640 1 -- 172.21.15.32:0/3347989182 >> v2:172.21.15.144:3300/0 conn(0x7fc23413f6e0 msgr2=0x7fc2340cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.769+0000 7fc239000640 1 --2- 172.21.15.32:0/3347989182 >> v2:172.21.15.144:3300/0 conn(0x7fc23413f6e0 0x7fc2340cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.769+0000 7fc239000640 1 -- 172.21.15.32:0/3347989182 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc234160760 con 0x7fc2340d0820 2024-04-17T10:01:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.769+0000 7fc22b400640 1 --2- 172.21.15.32:0/3347989182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2340cf370 0x7fc2340d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:28.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.773+0000 7fc239000640 1 --2- 172.21.15.32:0/3347989182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2340d0820 0x7fc234141d70 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fc2340b3100 tx=0x7fc22c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b722c3f7ba5f8b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:28.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.773+0000 7fc233400640 1 -- 172.21.15.32:0/3347989182 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c0a5760 con 0x7fc2340d0820 2024-04-17T10:01:28.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.773+0000 7fc233400640 1 -- 172.21.15.32:0/3347989182 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc22c0a5900 con 0x7fc2340d0820 2024-04-17T10:01:28.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.773+0000 7fc233400640 1 -- 172.21.15.32:0/3347989182 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c0a5cd0 con 0x7fc2340d0820 2024-04-17T10:01:28.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.773+0000 7fc23b000640 1 -- 172.21.15.32:0/3347989182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2340d0820 msgr2=0x7fc234141d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:28.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.773+0000 7fc23b000640 1 --2- 172.21.15.32:0/3347989182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2340d0820 0x7fc234141d70 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fc2340b3100 tx=0x7fc22c01c9f0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.773+0000 7fc23b000640 1 -- 172.21.15.32:0/3347989182 shutdown_connections 2024-04-17T10:01:28.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.773+0000 7fc23b000640 1 --2- 172.21.15.32:0/3347989182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2340d0820 0x7fc234141d70 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.773+0000 7fc23b000640 1 --2- 172.21.15.32:0/3347989182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2340cf370 0x7fc2340d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.773+0000 7fc23b000640 1 --2- 172.21.15.32:0/3347989182 >> v2:172.21.15.144:3300/0 conn(0x7fc23413f6e0 0x7fc2340cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.773+0000 7fc23b000640 1 -- 172.21.15.32:0/3347989182 >> 172.21.15.32:0/3347989182 conn(0x7fc23406ba30 msgr2=0x7fc23406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.773+0000 7fc23b000640 1 -- 172.21.15.32:0/3347989182 shutdown_connections 2024-04-17T10:01:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.773+0000 7fc23b000640 1 -- 172.21.15.32:0/3347989182 wait complete. 2024-04-17T10:01:28.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc23b000640 1 Processor -- start 2024-04-17T10:01:28.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc23b000640 1 -- start start 2024-04-17T10:01:28.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc23b000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2340cf370 0x7fc23413cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:28.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc23b000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2340d0820 0x7fc234157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:28.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc23b000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc23413f6e0 0x7fc2341581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc23b000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc2340cff10 con 0x7fc2340cf370 2024-04-17T10:01:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc23b000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc2340cfd90 con 0x7fc2340d0820 2024-04-17T10:01:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc23b000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc234071d80 con 0x7fc23413f6e0 2024-04-17T10:01:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc233e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2340cf370 0x7fc23413cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc233e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2340cf370 0x7fc23413cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53614/0 (socket says 172.21.15.32:53614) 2024-04-17T10:01:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc233e00640 1 -- 172.21.15.32:0/2703547041 learned_addr learned my addr 172.21.15.32:0/2703547041 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc239000640 1 --2- 172.21.15.32:0/2703547041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc23413f6e0 0x7fc2341581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc22b400640 1 --2- 172.21.15.32:0/2703547041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2340d0820 0x7fc234157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc233e00640 1 -- 172.21.15.32:0/2703547041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc23413f6e0 msgr2=0x7fc2341581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc233e00640 1 --2- 172.21.15.32:0/2703547041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc23413f6e0 0x7fc2341581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc233e00640 1 -- 172.21.15.32:0/2703547041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2340d0820 msgr2=0x7fc234157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc233e00640 1 --2- 172.21.15.32:0/2703547041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2340d0820 0x7fc234157cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc233e00640 1 -- 172.21.15.32:0/2703547041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23413d8e0 con 0x7fc2340cf370 2024-04-17T10:01:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc239000640 1 --2- 172.21.15.32:0/2703547041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc23413f6e0 0x7fc2341581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:28.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc22b400640 1 --2- 172.21.15.32:0/2703547041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2340d0820 0x7fc234157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:28.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc233e00640 1 --2- 172.21.15.32:0/2703547041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2340cf370 0x7fc23413cf50 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fc22400e870 tx=0x7fc22400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:28.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc230c00640 1 -- 172.21.15.32:0/2703547041 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc224017070 con 0x7fc2340cf370 2024-04-17T10:01:28.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc230c00640 1 -- 172.21.15.32:0/2703547041 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc22400c940 con 0x7fc2340cf370 2024-04-17T10:01:28.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc230c00640 1 -- 172.21.15.32:0/2703547041 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22400ccf0 con 0x7fc2340cf370 2024-04-17T10:01:28.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc23b000640 1 -- 172.21.15.32:0/2703547041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc23413dbc0 con 0x7fc2340cf370 2024-04-17T10:01:28.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc23b000640 1 -- 172.21.15.32:0/2703547041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc23413e180 con 0x7fc2340cf370 2024-04-17T10:01:28.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.777+0000 7fc23b000640 1 -- 172.21.15.32:0/2703547041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc234068a90 con 0x7fc2340cf370 2024-04-17T10:01:28.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.781+0000 7fc230c00640 1 -- 172.21.15.32:0/2703547041 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc22401a070 con 0x7fc2340cf370 2024-04-17T10:01:28.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.781+0000 7fc230c00640 1 --2- 172.21.15.32:0/2703547041 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc210068380 0x7fc21006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:28.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.781+0000 7fc230c00640 1 -- 172.21.15.32:0/2703547041 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5939+0+0 (secure 0 0 0) 0x7fc224086a70 con 0x7fc2340cf370 2024-04-17T10:01:28.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.781+0000 7fc22b400640 1 --2- 172.21.15.32:0/2703547041 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc210068380 0x7fc21006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:28.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.781+0000 7fc22b400640 1 --2- 172.21.15.32:0/2703547041 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc210068380 0x7fc21006a840 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc234158b20 tx=0x7fc21c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:28.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.785+0000 7fc230c00640 1 -- 172.21.15.32:0/2703547041 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc224056ea0 con 0x7fc2340cf370 2024-04-17T10:01:28.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.985+0000 7fc23b000640 1 -- 172.21.15.32:0/2703547041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc2340d1030 con 0x7fc2340cf370 2024-04-17T10:01:28.984 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-17T10:01:28.985+0000 7f93c9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T10:01:28.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.989+0000 7fc230c00640 1 -- 172.21.15.32:0/2703547041 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v73) ==== 161+0+0 (secure 0 0 0) 0x7fc2240567a0 con 0x7fc2340cf370 2024-04-17T10:01:28.986 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T10:01:28.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.989+0000 7fc23b000640 1 -- 172.21.15.32:0/2703547041 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc210068380 msgr2=0x7fc21006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:28.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.989+0000 7fc23b000640 1 --2- 172.21.15.32:0/2703547041 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc210068380 0x7fc21006a840 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc234158b20 tx=0x7fc21c008040 comp rx=0 tx=0).stop 2024-04-17T10:01:28.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.989+0000 7fc23b000640 1 -- 172.21.15.32:0/2703547041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2340cf370 msgr2=0x7fc23413cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:28.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.989+0000 7fc23b000640 1 --2- 172.21.15.32:0/2703547041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2340cf370 0x7fc23413cf50 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fc22400e870 tx=0x7fc22400ec70 comp rx=0 tx=0).stop 2024-04-17T10:01:28.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.993+0000 7fc23b000640 1 -- 172.21.15.32:0/2703547041 shutdown_connections 2024-04-17T10:01:28.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.993+0000 7fc23b000640 1 --2- 172.21.15.32:0/2703547041 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc210068380 0x7fc21006a840 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.993+0000 7fc23b000640 1 --2- 172.21.15.32:0/2703547041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc23413f6e0 0x7fc2341581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.993+0000 7fc23b000640 1 --2- 172.21.15.32:0/2703547041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2340d0820 0x7fc234157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.993+0000 7fc23b000640 1 --2- 172.21.15.32:0/2703547041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2340cf370 0x7fc23413cf50 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:28.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.993+0000 7fc23b000640 1 -- 172.21.15.32:0/2703547041 >> 172.21.15.32:0/2703547041 conn(0x7fc23406ba30 msgr2=0x7fc234158e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:28.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.993+0000 7fc23b000640 1 -- 172.21.15.32:0/2703547041 shutdown_connections 2024-04-17T10:01:28.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:28.993+0000 7fc23b000640 1 -- 172.21.15.32:0/2703547041 wait complete. 2024-04-17T10:01:28.996 INFO:tasks.ceph.mds.c.smithi032.stderr: -2362> 2024-04-17T10:00:49.853+0000 7f6c64200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:01:28.997 INFO:tasks.ceph.mds.c.smithi032.stderr: -1974> 2024-04-17T10:00:51.249+0000 7f6c5f200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x55ad3f175180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-17T10:01:29.011 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-17T10:01:29.042 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-17T10:01:29.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.157+0000 7fc918000640 1 Processor -- start 2024-04-17T10:01:29.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.157+0000 7fc918000640 1 -- start start 2024-04-17T10:01:29.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.157+0000 7fc918000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc91006ba30 0x7fc91006be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:29.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.161+0000 7fc918000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc9100cf370 0x7fc9100cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:29.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.161+0000 7fc918000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc910160350 0x7fc9101607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:29.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.161+0000 7fc918000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc91006c350 con 0x7fc910160350 2024-04-17T10:01:29.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.161+0000 7fc918000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc9100d0130 con 0x7fc9100cf370 2024-04-17T10:01:29.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.161+0000 7fc918000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc91015d090 con 0x7fc91006ba30 2024-04-17T10:01:29.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.161+0000 7fc916000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc910160350 0x7fc9101607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.161+0000 7fc916000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc910160350 0x7fc9101607a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53648/0 (socket says 172.21.15.32:53648) 2024-04-17T10:01:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.161+0000 7fc916000640 1 -- 172.21.15.32:0/4253999092 learned_addr learned my addr 172.21.15.32:0/4253999092 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.161+0000 7fc915600640 1 --2- 172.21.15.32:0/4253999092 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc91006ba30 0x7fc91006be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.161+0000 7fc914c00640 1 --2- 172.21.15.32:0/4253999092 >> v2:172.21.15.144:3300/0 conn(0x7fc9100cf370 0x7fc9100cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.161+0000 7fc916000640 1 -- 172.21.15.32:0/4253999092 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc91006ba30 msgr2=0x7fc91006be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.161+0000 7fc916000640 1 --2- 172.21.15.32:0/4253999092 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc91006ba30 0x7fc91006be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.161+0000 7fc916000640 1 -- 172.21.15.32:0/4253999092 >> v2:172.21.15.144:3300/0 conn(0x7fc9100cf370 msgr2=0x7fc9100cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.161+0000 7fc916000640 1 --2- 172.21.15.32:0/4253999092 >> v2:172.21.15.144:3300/0 conn(0x7fc9100cf370 0x7fc9100cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.161+0000 7fc916000640 1 -- 172.21.15.32:0/4253999092 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc91015d370 con 0x7fc910160350 2024-04-17T10:01:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.161+0000 7fc915600640 1 --2- 172.21.15.32:0/4253999092 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc91006ba30 0x7fc91006be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.161+0000 7fc916000640 1 --2- 172.21.15.32:0/4253999092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc910160350 0x7fc9101607a0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fc9100b57b0 tx=0x7fc90801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107c33ed5279ddb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.161+0000 7fc90fe00640 1 -- 172.21.15.32:0/4253999092 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9080a5800 con 0x7fc910160350 2024-04-17T10:01:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.161+0000 7fc90fe00640 1 -- 172.21.15.32:0/4253999092 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9080a59a0 con 0x7fc910160350 2024-04-17T10:01:29.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.161+0000 7fc90fe00640 1 -- 172.21.15.32:0/4253999092 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9080a5d70 con 0x7fc910160350 2024-04-17T10:01:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc918000640 1 -- 172.21.15.32:0/4253999092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc910160350 msgr2=0x7fc9101607a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc918000640 1 --2- 172.21.15.32:0/4253999092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc910160350 0x7fc9101607a0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fc9100b57b0 tx=0x7fc90801c9c0 comp rx=0 tx=0).stop 2024-04-17T10:01:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc918000640 1 -- 172.21.15.32:0/4253999092 shutdown_connections 2024-04-17T10:01:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc918000640 1 --2- 172.21.15.32:0/4253999092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc910160350 0x7fc9101607a0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc918000640 1 --2- 172.21.15.32:0/4253999092 >> v2:172.21.15.144:3300/0 conn(0x7fc9100cf370 0x7fc9100cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc918000640 1 --2- 172.21.15.32:0/4253999092 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc91006ba30 0x7fc91006be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc918000640 1 -- 172.21.15.32:0/4253999092 >> 172.21.15.32:0/4253999092 conn(0x7fc9100b2120 msgr2=0x7fc9100b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc918000640 1 -- 172.21.15.32:0/4253999092 shutdown_connections 2024-04-17T10:01:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc918000640 1 -- 172.21.15.32:0/4253999092 wait complete. 2024-04-17T10:01:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc918000640 1 Processor -- start 2024-04-17T10:01:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc918000640 1 -- start start 2024-04-17T10:01:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc918000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc91006ba30 0x7fc91015ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc918000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9100cf370 0x7fc910166930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc918000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc910160350 0x7fc910166e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc918000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc9100cfea0 con 0x7fc91006ba30 2024-04-17T10:01:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc918000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc9100cfd20 con 0x7fc910160350 2024-04-17T10:01:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc918000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc91015db70 con 0x7fc9100cf370 2024-04-17T10:01:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc915600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc91006ba30 0x7fc91015ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc915600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc91006ba30 0x7fc91015ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53658/0 (socket says 172.21.15.32:53658) 2024-04-17T10:01:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc915600640 1 -- 172.21.15.32:0/2453331880 learned_addr learned my addr 172.21.15.32:0/2453331880 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc914c00640 1 --2- 172.21.15.32:0/2453331880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9100cf370 0x7fc910166930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc916000640 1 --2- 172.21.15.32:0/2453331880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc910160350 0x7fc910166e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc915600640 1 -- 172.21.15.32:0/2453331880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9100cf370 msgr2=0x7fc910166930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc915600640 1 --2- 172.21.15.32:0/2453331880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9100cf370 0x7fc910166930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc915600640 1 -- 172.21.15.32:0/2453331880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc910160350 msgr2=0x7fc910166e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc915600640 1 --2- 172.21.15.32:0/2453331880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc910160350 0x7fc910166e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.165+0000 7fc915600640 1 -- 172.21.15.32:0/2453331880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc91015f290 con 0x7fc91006ba30 2024-04-17T10:01:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.169+0000 7fc915600640 1 --2- 172.21.15.32:0/2453331880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc91006ba30 0x7fc91015ea40 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fc90400e810 tx=0x7fc90400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.169+0000 7fc90d600640 1 -- 172.21.15.32:0/2453331880 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90400f040 con 0x7fc91006ba30 2024-04-17T10:01:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.169+0000 7fc90d600640 1 -- 172.21.15.32:0/2453331880 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc904004030 con 0x7fc91006ba30 2024-04-17T10:01:29.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.169+0000 7fc90d600640 1 -- 172.21.15.32:0/2453331880 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc904004400 con 0x7fc91006ba30 2024-04-17T10:01:29.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.169+0000 7fc918000640 1 -- 172.21.15.32:0/2453331880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc91015f570 con 0x7fc91006ba30 2024-04-17T10:01:29.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.169+0000 7fc918000640 1 -- 172.21.15.32:0/2453331880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc91015fb30 con 0x7fc91006ba30 2024-04-17T10:01:29.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.169+0000 7fc918000640 1 -- 172.21.15.32:0/2453331880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc910068a90 con 0x7fc91006ba30 2024-04-17T10:01:29.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.169+0000 7fc90d600640 1 -- 172.21.15.32:0/2453331880 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc90400c6d0 con 0x7fc91006ba30 2024-04-17T10:01:29.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.169+0000 7fc90d600640 1 --2- 172.21.15.32:0/2453331880 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc8f0068380 0x7fc8f006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:29.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.169+0000 7fc90d600640 1 -- 172.21.15.32:0/2453331880 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6025+0+0 (secure 0 0 0) 0x7fc904086780 con 0x7fc91006ba30 2024-04-17T10:01:29.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.169+0000 7fc914c00640 1 --2- 172.21.15.32:0/2453331880 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc8f0068380 0x7fc8f006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:29.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.173+0000 7fc914c00640 1 --2- 172.21.15.32:0/2453331880 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc8f0068380 0x7fc8f006a840 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc9100a9f40 tx=0x7fc8fc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:29.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.173+0000 7fc90d600640 1 -- 172.21.15.32:0/2453331880 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc904017020 con 0x7fc91006ba30 2024-04-17T10:01:29.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.373+0000 7fc918000640 1 -- 172.21.15.32:0/2453331880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc91006c850 con 0x7fc91006ba30 2024-04-17T10:01:29.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.989+0000 7fc90d600640 1 -- 172.21.15.32:0/2453331880 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v74) ==== 104+0+0 (secure 0 0 0) 0x7fc904056510 con 0x7fc91006ba30 2024-04-17T10:01:29.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.993+0000 7fc918000640 1 -- 172.21.15.32:0/2453331880 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc8f0068380 msgr2=0x7fc8f006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:29.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.993+0000 7fc918000640 1 --2- 172.21.15.32:0/2453331880 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc8f0068380 0x7fc8f006a840 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc9100a9f40 tx=0x7fc8fc008040 comp rx=0 tx=0).stop 2024-04-17T10:01:29.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.993+0000 7fc918000640 1 -- 172.21.15.32:0/2453331880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc91006ba30 msgr2=0x7fc91015ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:29.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.993+0000 7fc918000640 1 --2- 172.21.15.32:0/2453331880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc91006ba30 0x7fc91015ea40 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fc90400e810 tx=0x7fc90400ecd0 comp rx=0 tx=0).stop 2024-04-17T10:01:29.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.993+0000 7fc918000640 1 -- 172.21.15.32:0/2453331880 shutdown_connections 2024-04-17T10:01:29.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.993+0000 7fc918000640 1 --2- 172.21.15.32:0/2453331880 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc8f0068380 0x7fc8f006a840 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:29.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.993+0000 7fc918000640 1 --2- 172.21.15.32:0/2453331880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9100cf370 0x7fc910166930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:29.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.993+0000 7fc918000640 1 --2- 172.21.15.32:0/2453331880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc910160350 0x7fc910166e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:29.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.993+0000 7fc918000640 1 --2- 172.21.15.32:0/2453331880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc91006ba30 0x7fc91015ea40 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:29.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.993+0000 7fc918000640 1 -- 172.21.15.32:0/2453331880 >> 172.21.15.32:0/2453331880 conn(0x7fc9100b2120 msgr2=0x7fc91006f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:29.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.993+0000 7fc918000640 1 -- 172.21.15.32:0/2453331880 shutdown_connections 2024-04-17T10:01:29.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:29.993+0000 7fc918000640 1 -- 172.21.15.32:0/2453331880 wait complete. 2024-04-17T10:01:30.014 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-17T10:01:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.209+0000 7fe687400640 1 Processor -- start 2024-04-17T10:01:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.209+0000 7fe687400640 1 -- start start 2024-04-17T10:01:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.209+0000 7fe687400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe6801479e0 0x7fe68006e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.209+0000 7fe687400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6801663d0 0x7fe6801687c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.209+0000 7fe687400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6800aeb40 con 0x7fe680071db0 2024-04-17T10:01:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.209+0000 7fe687400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe680147830 con 0x7fe6801479e0 2024-04-17T10:01:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.209+0000 7fe687400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe6800aede0 con 0x7fe6801663d0 2024-04-17T10:01:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.213+0000 7fe677e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe6801479e0 0x7fe68006e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.213+0000 7fe677e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe6801479e0 0x7fe68006e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53672/0 (socket says 172.21.15.32:53672) 2024-04-17T10:01:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.213+0000 7fe677e00640 1 -- 172.21.15.32:0/1424617158 learned_addr learned my addr 172.21.15.32:0/1424617158 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.213+0000 7fe685400640 1 --2- 172.21.15.32:0/1424617158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6801663d0 0x7fe6801687c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.213+0000 7fe677e00640 1 -- 172.21.15.32:0/1424617158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6801663d0 msgr2=0x7fe6801687c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.213+0000 7fe677e00640 1 --2- 172.21.15.32:0/1424617158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6801663d0 0x7fe6801687c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.213+0000 7fe677e00640 1 -- 172.21.15.32:0/1424617158 >> v1:172.21.15.144:6789/0 conn(0x7fe680071db0 legacy=0x7fe680072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.213+0000 7fe677e00640 1 -- 172.21.15.32:0/1424617158 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6800af0c0 con 0x7fe6801479e0 2024-04-17T10:01:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.213+0000 7fe677e00640 1 --2- 172.21.15.32:0/1424617158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6801479e0 0x7fe68006e6a0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fe668006fd0 tx=0x7fe66801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fe5a9c11e6a71 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.213+0000 7fe67fe00640 1 -- 172.21.15.32:0/1424617158 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6680b1070 con 0x7fe6801479e0 2024-04-17T10:01:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.213+0000 7fe67fe00640 1 -- 172.21.15.32:0/1424617158 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6680ac410 con 0x7fe6801479e0 2024-04-17T10:01:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.213+0000 7fe67fe00640 1 -- 172.21.15.32:0/1424617158 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6680ac7e0 con 0x7fe6801479e0 2024-04-17T10:01:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe687400640 1 -- 172.21.15.32:0/1424617158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6801479e0 msgr2=0x7fe68006e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe687400640 1 --2- 172.21.15.32:0/1424617158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6801479e0 0x7fe68006e6a0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fe668006fd0 tx=0x7fe66801c500 comp rx=0 tx=0).stop 2024-04-17T10:01:30.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe687400640 1 -- 172.21.15.32:0/1424617158 shutdown_connections 2024-04-17T10:01:30.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe687400640 1 --2- 172.21.15.32:0/1424617158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6801663d0 0x7fe6801687c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:30.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe687400640 1 --2- 172.21.15.32:0/1424617158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6801479e0 0x7fe68006e6a0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:30.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe687400640 1 -- 172.21.15.32:0/1424617158 >> 172.21.15.32:0/1424617158 conn(0x7fe6800b2190 msgr2=0x7fe6800b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:30.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe687400640 1 -- 172.21.15.32:0/1424617158 shutdown_connections 2024-04-17T10:01:30.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe687400640 1 -- 172.21.15.32:0/1424617158 wait complete. 2024-04-17T10:01:30.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe687400640 1 Processor -- start 2024-04-17T10:01:30.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe687400640 1 -- start start 2024-04-17T10:01:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe687400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe680071db0 0x7fe680143370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe687400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6801479e0 0x7fe680142350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe687400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6801663d0 0x7fe680161650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe687400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe6800afd20 con 0x7fe6801479e0 2024-04-17T10:01:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe687400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe6800afba0 con 0x7fe6801663d0 2024-04-17T10:01:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe687400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe68016b520 con 0x7fe680071db0 2024-04-17T10:01:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe677e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6801479e0 0x7fe680142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe677e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6801479e0 0x7fe680142350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53680/0 (socket says 172.21.15.32:53680) 2024-04-17T10:01:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe677e00640 1 -- 172.21.15.32:0/2451418356 learned_addr learned my addr 172.21.15.32:0/2451418356 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe685400640 1 --2- 172.21.15.32:0/2451418356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6801663d0 0x7fe680161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe684a00640 1 --2- 172.21.15.32:0/2451418356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe680071db0 0x7fe680143370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe677e00640 1 -- 172.21.15.32:0/2451418356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe680071db0 msgr2=0x7fe680143370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:30.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe677e00640 1 --2- 172.21.15.32:0/2451418356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe680071db0 0x7fe680143370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:30.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe677e00640 1 -- 172.21.15.32:0/2451418356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6801663d0 msgr2=0x7fe680161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:30.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe677e00640 1 --2- 172.21.15.32:0/2451418356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6801663d0 0x7fe680161650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:30.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe677e00640 1 -- 172.21.15.32:0/2451418356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6801418b0 con 0x7fe6801479e0 2024-04-17T10:01:30.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.217+0000 7fe685400640 1 --2- 172.21.15.32:0/2451418356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6801663d0 0x7fe680161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:30.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.221+0000 7fe677e00640 1 --2- 172.21.15.32:0/2451418356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6801479e0 0x7fe680142350 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fe668006fd0 tx=0x7fe6680192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:30.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.221+0000 7fe67d600640 1 -- 172.21.15.32:0/2451418356 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6680b9020 con 0x7fe6801479e0 2024-04-17T10:01:30.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.221+0000 7fe67d600640 1 -- 172.21.15.32:0/2451418356 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6680059a0 con 0x7fe6801479e0 2024-04-17T10:01:30.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.221+0000 7fe67d600640 1 -- 172.21.15.32:0/2451418356 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6680b1040 con 0x7fe6801479e0 2024-04-17T10:01:30.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.221+0000 7fe687400640 1 -- 172.21.15.32:0/2451418356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe680144e30 con 0x7fe6801479e0 2024-04-17T10:01:30.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.221+0000 7fe687400640 1 -- 172.21.15.32:0/2451418356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6801453f0 con 0x7fe6801479e0 2024-04-17T10:01:30.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.221+0000 7fe687400640 1 -- 172.21.15.32:0/2451418356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6801455c0 con 0x7fe6801479e0 2024-04-17T10:01:30.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.221+0000 7fe67d600640 1 -- 172.21.15.32:0/2451418356 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe668003720 con 0x7fe6801479e0 2024-04-17T10:01:30.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.221+0000 7fe67d600640 1 --2- 172.21.15.32:0/2451418356 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe65c068380 0x7fe65c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:30.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.221+0000 7fe67d600640 1 -- 172.21.15.32:0/2451418356 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6025+0+0 (secure 0 0 0) 0x7fe668120350 con 0x7fe6801479e0 2024-04-17T10:01:30.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.221+0000 7fe684a00640 1 --2- 172.21.15.32:0/2451418356 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe65c068380 0x7fe65c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:30.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.225+0000 7fe684a00640 1 --2- 172.21.15.32:0/2451418356 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe65c068380 0x7fe65c06a840 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe670004e30 tx=0x7fe670009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:30.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.225+0000 7fe67d600640 1 -- 172.21.15.32:0/2451418356 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fe6680c3020 con 0x7fe6801479e0 2024-04-17T10:01:30.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:30.393+0000 7fe687400640 1 -- 172.21.15.32:0/2451418356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe68006fac0 con 0x7fe6801479e0 2024-04-17T10:01:31.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.065+0000 7fe67d600640 1 -- 172.21.15.32:0/2451418356 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v88) ==== 181+0+0 (secure 0 0 0) 0x7fe6680160f0 con 0x7fe6801479e0 2024-04-17T10:01:31.063 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-04-17T10:01:31.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.069+0000 7fe687400640 1 -- 172.21.15.32:0/2451418356 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe65c068380 msgr2=0x7fe65c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:31.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.069+0000 7fe687400640 1 --2- 172.21.15.32:0/2451418356 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe65c068380 0x7fe65c06a840 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe670004e30 tx=0x7fe670009040 comp rx=0 tx=0).stop 2024-04-17T10:01:31.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.069+0000 7fe687400640 1 -- 172.21.15.32:0/2451418356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6801479e0 msgr2=0x7fe680142350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:31.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.069+0000 7fe687400640 1 --2- 172.21.15.32:0/2451418356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6801479e0 0x7fe680142350 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fe668006fd0 tx=0x7fe6680192a0 comp rx=0 tx=0).stop 2024-04-17T10:01:31.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.069+0000 7fe687400640 1 -- 172.21.15.32:0/2451418356 shutdown_connections 2024-04-17T10:01:31.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.069+0000 7fe687400640 1 --2- 172.21.15.32:0/2451418356 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe65c068380 0x7fe65c06a840 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:31.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.069+0000 7fe687400640 1 --2- 172.21.15.32:0/2451418356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6801663d0 0x7fe680161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:31.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.069+0000 7fe687400640 1 --2- 172.21.15.32:0/2451418356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6801479e0 0x7fe680142350 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:31.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.069+0000 7fe687400640 1 --2- 172.21.15.32:0/2451418356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe680071db0 0x7fe680143370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:31.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.069+0000 7fe687400640 1 -- 172.21.15.32:0/2451418356 >> 172.21.15.32:0/2451418356 conn(0x7fe6800b2190 msgr2=0x7fe6800b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:31.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.069+0000 7fe687400640 1 -- 172.21.15.32:0/2451418356 shutdown_connections 2024-04-17T10:01:31.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.069+0000 7fe687400640 1 -- 172.21.15.32:0/2451418356 wait complete. 2024-04-17T10:01:31.092 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-17T10:01:31.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.245+0000 7fbdfc400640 1 Processor -- start 2024-04-17T10:01:31.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.245+0000 7fbdfc400640 1 -- start start 2024-04-17T10:01:31.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.245+0000 7fbdfc400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbdf41460a0 0x7fbdf40ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:31.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.245+0000 7fbdfc400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdf406e6f0 0x7fbdf4071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:31.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.245+0000 7fbdfc400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdf406eeb0 con 0x7fbdf40722f0 2024-04-17T10:01:31.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.245+0000 7fbdfc400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fbdf415de00 con 0x7fbdf41460a0 2024-04-17T10:01:31.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.245+0000 7fbdfc400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbdf406f190 con 0x7fbdf406e6f0 2024-04-17T10:01:31.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.245+0000 7fbdfa400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbdf40722f0 0x7fbdf415cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36058/0 (socket says 172.21.15.32:36058) 2024-04-17T10:01:31.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.245+0000 7fbdfa400640 1 -- 172.21.15.32:0/3997287955 learned_addr learned my addr 172.21.15.32:0/3997287955 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:31.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.245+0000 7fbdf9a00640 1 --2- 172.21.15.32:0/3997287955 >> v2:172.21.15.144:3300/0 conn(0x7fbdf41460a0 0x7fbdf40ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:31.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.245+0000 7fbdf9000640 1 --2- 172.21.15.32:0/3997287955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdf406e6f0 0x7fbdf4071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:31.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.245+0000 7fbdf3e00640 1 -- 172.21.15.32:0/3997287955 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 668235128 0 0) 0x7fbdf406eeb0 con 0x7fbdf40722f0 2024-04-17T10:01:31.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.245+0000 7fbdf3e00640 1 -- 172.21.15.32:0/3997287955 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbddc003530 con 0x7fbdf40722f0 2024-04-17T10:01:31.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.249+0000 7fbdf9a00640 1 -- 172.21.15.32:0/3997287955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdf406e6f0 msgr2=0x7fbdf4071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:31.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.249+0000 7fbdf9a00640 1 --2- 172.21.15.32:0/3997287955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdf406e6f0 0x7fbdf4071db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:31.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.249+0000 7fbdf9a00640 1 -- 172.21.15.32:0/3997287955 >> v1:172.21.15.32:6789/0 conn(0x7fbdf40722f0 legacy=0x7fbdf415cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:31.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.249+0000 7fbdf9a00640 1 -- 172.21.15.32:0/3997287955 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf406f470 con 0x7fbdf41460a0 2024-04-17T10:01:31.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.249+0000 7fbdf9a00640 1 --2- 172.21.15.32:0/3997287955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdf41460a0 0x7fbdf40ceb40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbde80097c0 tx=0x7fbde801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dce001d479be3b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:31.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.249+0000 7fbdf3e00640 1 -- 172.21.15.32:0/3997287955 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde80b0070 con 0x7fbdf41460a0 2024-04-17T10:01:31.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.249+0000 7fbdf3e00640 1 -- 172.21.15.32:0/3997287955 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdf415db50 con 0x7fbdf41460a0 2024-04-17T10:01:31.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.249+0000 7fbdf3e00640 1 -- 172.21.15.32:0/3997287955 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde80ab820 con 0x7fbdf41460a0 2024-04-17T10:01:31.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.249+0000 7fbdfc400640 1 -- 172.21.15.32:0/3997287955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdf41460a0 msgr2=0x7fbdf40ceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.249+0000 7fbdfc400640 1 --2- 172.21.15.32:0/3997287955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdf41460a0 0x7fbdf40ceb40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbde80097c0 tx=0x7fbde801c4a0 comp rx=0 tx=0).stop 2024-04-17T10:01:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.249+0000 7fbdfc400640 1 -- 172.21.15.32:0/3997287955 shutdown_connections 2024-04-17T10:01:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.249+0000 7fbdfc400640 1 --2- 172.21.15.32:0/3997287955 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdf406e6f0 0x7fbdf4071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.249+0000 7fbdfc400640 1 --2- 172.21.15.32:0/3997287955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdf41460a0 0x7fbdf40ceb40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.249+0000 7fbdfc400640 1 -- 172.21.15.32:0/3997287955 >> 172.21.15.32:0/3997287955 conn(0x7fbdf40b2190 msgr2=0x7fbdf40b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.249+0000 7fbdfc400640 1 -- 172.21.15.32:0/3997287955 shutdown_connections 2024-04-17T10:01:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdfc400640 1 -- 172.21.15.32:0/3997287955 wait complete. 2024-04-17T10:01:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdfc400640 1 Processor -- start 2024-04-17T10:01:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdfc400640 1 -- start start 2024-04-17T10:01:31.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdfc400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdf406e6f0 0x7fbdf415b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:31.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdfc400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf40722f0 0x7fbdf406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:31.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdfc400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdf41460a0 0x7fbdf41f8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:31.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdfc400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf4169610 con 0x7fbdf40722f0 2024-04-17T10:01:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdfc400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf4169490 con 0x7fbdf406e6f0 2024-04-17T10:01:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdfc400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbdf4169790 con 0x7fbdf41460a0 2024-04-17T10:01:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdf9000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf40722f0 0x7fbdf406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdf9000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf40722f0 0x7fbdf406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53880/0 (socket says 172.21.15.32:53880) 2024-04-17T10:01:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdf9000640 1 -- 172.21.15.32:0/2288894993 learned_addr learned my addr 172.21.15.32:0/2288894993 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdf9a00640 1 --2- 172.21.15.32:0/2288894993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdf406e6f0 0x7fbdf415b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdfa400640 1 --2- 172.21.15.32:0/2288894993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdf41460a0 0x7fbdf41f8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdf9000640 1 -- 172.21.15.32:0/2288894993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdf41460a0 msgr2=0x7fbdf41f8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdf9000640 1 --2- 172.21.15.32:0/2288894993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdf41460a0 0x7fbdf41f8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdf9000640 1 -- 172.21.15.32:0/2288894993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdf406e6f0 msgr2=0x7fbdf415b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdf9000640 1 --2- 172.21.15.32:0/2288894993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdf406e6f0 0x7fbdf415b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdf9000640 1 -- 172.21.15.32:0/2288894993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf41591f0 con 0x7fbdf40722f0 2024-04-17T10:01:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdf9a00640 1 --2- 172.21.15.32:0/2288894993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdf406e6f0 0x7fbdf415b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdf9000640 1 --2- 172.21.15.32:0/2288894993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf40722f0 0x7fbdf406bc90 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fbde000e9b0 tx=0x7fbde000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdf1600640 1 -- 172.21.15.32:0/2288894993 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0017070 con 0x7fbdf40722f0 2024-04-17T10:01:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdf1600640 1 -- 172.21.15.32:0/2288894993 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbde000c9a0 con 0x7fbdf40722f0 2024-04-17T10:01:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdfc400640 1 -- 172.21.15.32:0/2288894993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf41594d0 con 0x7fbdf40722f0 2024-04-17T10:01:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdf1600640 1 -- 172.21.15.32:0/2288894993 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde000cd50 con 0x7fbdf40722f0 2024-04-17T10:01:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.253+0000 7fbdfc400640 1 -- 172.21.15.32:0/2288894993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf4159a90 con 0x7fbdf40722f0 2024-04-17T10:01:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.257+0000 7fbdfc400640 1 -- 172.21.15.32:0/2288894993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdf4068a90 con 0x7fbdf40722f0 2024-04-17T10:01:31.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.257+0000 7fbdf1600640 1 -- 172.21.15.32:0/2288894993 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbde001a070 con 0x7fbdf40722f0 2024-04-17T10:01:31.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.257+0000 7fbdf1600640 1 --2- 172.21.15.32:0/2288894993 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbdd4068380 0x7fbdd406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:31.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.257+0000 7fbdf1600640 1 -- 172.21.15.32:0/2288894993 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5587+0+0 (secure 0 0 0) 0x7fbde0086dc0 con 0x7fbdf40722f0 2024-04-17T10:01:31.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.257+0000 7fbdf9a00640 1 --2- 172.21.15.32:0/2288894993 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbdd4068380 0x7fbdd406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:31.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.257+0000 7fbdf9a00640 1 --2- 172.21.15.32:0/2288894993 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbdd4068380 0x7fbdd406a840 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbde80ad000 tx=0x7fbde80a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:31.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.261+0000 7fbdf1600640 1 -- 172.21.15.32:0/2288894993 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fbde00573e0 con 0x7fbdf40722f0 2024-04-17T10:01:31.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:31.437+0000 7fbdfc400640 1 -- 172.21.15.32:0/2288894993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbdf41401f0 con 0x7fbdf40722f0 2024-04-17T10:01:32.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.081+0000 7fbdf1600640 1 -- 172.21.15.32:0/2288894993 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v89) ==== 169+0+0 (secure 0 0 0) 0x7fbde0056ce0 con 0x7fbdf40722f0 2024-04-17T10:01:32.079 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-04-17T10:01:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.085+0000 7fbdfc400640 1 -- 172.21.15.32:0/2288894993 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbdd4068380 msgr2=0x7fbdd406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.085+0000 7fbdfc400640 1 --2- 172.21.15.32:0/2288894993 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbdd4068380 0x7fbdd406a840 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbde80ad000 tx=0x7fbde80a6000 comp rx=0 tx=0).stop 2024-04-17T10:01:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.085+0000 7fbdfc400640 1 -- 172.21.15.32:0/2288894993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf40722f0 msgr2=0x7fbdf406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.085+0000 7fbdfc400640 1 --2- 172.21.15.32:0/2288894993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf40722f0 0x7fbdf406bc90 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fbde000e9b0 tx=0x7fbde000ee70 comp rx=0 tx=0).stop 2024-04-17T10:01:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.085+0000 7fbdfc400640 1 -- 172.21.15.32:0/2288894993 shutdown_connections 2024-04-17T10:01:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.085+0000 7fbdfc400640 1 --2- 172.21.15.32:0/2288894993 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbdd4068380 0x7fbdd406a840 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.085+0000 7fbdfc400640 1 --2- 172.21.15.32:0/2288894993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbdf41460a0 0x7fbdf41f8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.085+0000 7fbdfc400640 1 --2- 172.21.15.32:0/2288894993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf40722f0 0x7fbdf406bc90 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.085+0000 7fbdfc400640 1 --2- 172.21.15.32:0/2288894993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdf406e6f0 0x7fbdf415b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.085+0000 7fbdfc400640 1 -- 172.21.15.32:0/2288894993 >> 172.21.15.32:0/2288894993 conn(0x7fbdf40b2190 msgr2=0x7fbdf406dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.085+0000 7fbdfc400640 1 -- 172.21.15.32:0/2288894993 shutdown_connections 2024-04-17T10:01:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.085+0000 7fbdfc400640 1 -- 172.21.15.32:0/2288894993 wait complete. 2024-04-17T10:01:32.109 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-17T10:01:32.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.265+0000 7fa5ad600640 1 Processor -- start 2024-04-17T10:01:32.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.265+0000 7fa5ad600640 1 -- start start 2024-04-17T10:01:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.269+0000 7fa5ad600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5a806ba30 0x7fa5a806be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.269+0000 7fa5ad600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa5a8071e60 0x7fa5a8072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.269+0000 7fa5ad600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a806c350 con 0x7fa5a80cf370 2024-04-17T10:01:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.269+0000 7fa5ad600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa5a80cfef0 con 0x7fa5a806ba30 2024-04-17T10:01:32.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.269+0000 7fa5ad600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa5a80d0130 con 0x7fa5a8071e60 2024-04-17T10:01:32.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.269+0000 7fa5a6a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5a806ba30 0x7fa5a806be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:32.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.269+0000 7fa59fe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa5a80cf370 0x7fa5a80cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36062/0 (socket says 172.21.15.32:36062) 2024-04-17T10:01:32.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.269+0000 7fa59fe00640 1 -- 172.21.15.32:0/2261727127 learned_addr learned my addr 172.21.15.32:0/2261727127 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:32.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.269+0000 7fa5a7400640 1 --2- 172.21.15.32:0/2261727127 >> v2:172.21.15.144:3300/0 conn(0x7fa5a8071e60 0x7fa5a8072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:32.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.269+0000 7fa5a6a00640 1 -- 172.21.15.32:0/2261727127 >> v2:172.21.15.144:3300/0 conn(0x7fa5a8071e60 msgr2=0x7fa5a8072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:32.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.269+0000 7fa5a6a00640 1 --2- 172.21.15.32:0/2261727127 >> v2:172.21.15.144:3300/0 conn(0x7fa5a8071e60 0x7fa5a8072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:32.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.269+0000 7fa5a6a00640 1 -- 172.21.15.32:0/2261727127 >> v1:172.21.15.32:6789/0 conn(0x7fa5a80cf370 legacy=0x7fa5a80cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:32.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.269+0000 7fa5a6a00640 1 -- 172.21.15.32:0/2261727127 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a815d090 con 0x7fa5a806ba30 2024-04-17T10:01:32.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.269+0000 7fa5a6000640 1 -- 172.21.15.32:0/2261727127 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1309898011 0 0) 0x7fa5a806c350 con 0x7fa5a80cf370 2024-04-17T10:01:32.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.273+0000 7fa5a6a00640 1 --2- 172.21.15.32:0/2261727127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a806ba30 0x7fa5a806be10 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80cd7e0 tx=0x7fa59801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80173531e6f003bd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:32.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.273+0000 7fa5a6000640 1 -- 172.21.15.32:0/2261727127 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5980b0070 con 0x7fa5a806ba30 2024-04-17T10:01:32.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.273+0000 7fa5a6000640 1 -- 172.21.15.32:0/2261727127 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5980ab600 con 0x7fa5a806ba30 2024-04-17T10:01:32.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.273+0000 7fa5a6000640 1 -- 172.21.15.32:0/2261727127 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5980ab9d0 con 0x7fa5a806ba30 2024-04-17T10:01:32.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.273+0000 7fa5ad600640 1 -- 172.21.15.32:0/2261727127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a806ba30 msgr2=0x7fa5a806be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:32.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.273+0000 7fa5ad600640 1 --2- 172.21.15.32:0/2261727127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a806ba30 0x7fa5a806be10 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80cd7e0 tx=0x7fa59801c390 comp rx=0 tx=0).stop 2024-04-17T10:01:32.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.273+0000 7fa5ad600640 1 -- 172.21.15.32:0/2261727127 shutdown_connections 2024-04-17T10:01:32.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.273+0000 7fa5ad600640 1 --2- 172.21.15.32:0/2261727127 >> v2:172.21.15.144:3300/0 conn(0x7fa5a8071e60 0x7fa5a8072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:32.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.273+0000 7fa5ad600640 1 --2- 172.21.15.32:0/2261727127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a806ba30 0x7fa5a806be10 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:32.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.273+0000 7fa5ad600640 1 -- 172.21.15.32:0/2261727127 >> 172.21.15.32:0/2261727127 conn(0x7fa5a80b2120 msgr2=0x7fa5a80b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:32.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.273+0000 7fa5ad600640 1 -- 172.21.15.32:0/2261727127 shutdown_connections 2024-04-17T10:01:32.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.273+0000 7fa5ad600640 1 -- 172.21.15.32:0/2261727127 wait complete. 2024-04-17T10:01:32.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.273+0000 7fa5ad600640 1 Processor -- start 2024-04-17T10:01:32.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.273+0000 7fa5ad600640 1 -- start start 2024-04-17T10:01:32.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa5ad600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a806ba30 0x7fa5a806ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:32.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa5ad600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a8071e60 0x7fa5a81441d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:32.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa5ad600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a80cf370 0x7fa5a8142670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:32.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa5ad600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a815dcf0 con 0x7fa5a806ba30 2024-04-17T10:01:32.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa5ad600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a815db70 con 0x7fa5a80cf370 2024-04-17T10:01:32.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa5ad600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a8146230 con 0x7fa5a8071e60 2024-04-17T10:01:32.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa5a6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a806ba30 0x7fa5a806ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:32.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa5a6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a806ba30 0x7fa5a806ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53888/0 (socket says 172.21.15.32:53888) 2024-04-17T10:01:32.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa5a6a00640 1 -- 172.21.15.32:0/530999409 learned_addr learned my addr 172.21.15.32:0/530999409 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:32.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa5a7400640 1 --2- 172.21.15.32:0/530999409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a80cf370 0x7fa5a8142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:32.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa59fe00640 1 --2- 172.21.15.32:0/530999409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a8071e60 0x7fa5a81441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:32.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa5a6a00640 1 -- 172.21.15.32:0/530999409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a8071e60 msgr2=0x7fa5a81441d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:32.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa5a6a00640 1 --2- 172.21.15.32:0/530999409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a8071e60 0x7fa5a81441d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:32.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa5a6a00640 1 -- 172.21.15.32:0/530999409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a80cf370 msgr2=0x7fa5a8142670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:32.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa5a6a00640 1 --2- 172.21.15.32:0/530999409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a80cf370 0x7fa5a8142670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:32.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa5a6a00640 1 -- 172.21.15.32:0/530999409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a8143130 con 0x7fa5a806ba30 2024-04-17T10:01:32.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa5a6a00640 1 --2- 172.21.15.32:0/530999409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a806ba30 0x7fa5a806ed90 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fa598019fc0 tx=0x7fa59801abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:32.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa59ea00640 1 -- 172.21.15.32:0/530999409 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5980b0050 con 0x7fa5a806ba30 2024-04-17T10:01:32.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa59ea00640 1 -- 172.21.15.32:0/530999409 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5980b1040 con 0x7fa5a806ba30 2024-04-17T10:01:32.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa5ad600640 1 -- 172.21.15.32:0/530999409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a8143410 con 0x7fa5a806ba30 2024-04-17T10:01:32.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa59ea00640 1 -- 172.21.15.32:0/530999409 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598004240 con 0x7fa5a806ba30 2024-04-17T10:01:32.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa5ad600640 1 -- 172.21.15.32:0/530999409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a81439d0 con 0x7fa5a806ba30 2024-04-17T10:01:32.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa5ad600640 1 -- 172.21.15.32:0/530999409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a8068a90 con 0x7fa5a806ba30 2024-04-17T10:01:32.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.277+0000 7fa59ea00640 1 -- 172.21.15.32:0/530999409 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa5980be060 con 0x7fa5a806ba30 2024-04-17T10:01:32.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.281+0000 7fa59ea00640 1 --2- 172.21.15.32:0/530999409 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa584068380 0x7fa58406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:32.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.281+0000 7fa59ea00640 1 -- 172.21.15.32:0/530999409 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5115+0+0 (secure 0 0 0) 0x7fa5981200c0 con 0x7fa5a806ba30 2024-04-17T10:01:32.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.281+0000 7fa59fe00640 1 --2- 172.21.15.32:0/530999409 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa584068380 0x7fa58406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:32.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.281+0000 7fa59fe00640 1 --2- 172.21.15.32:0/530999409 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa584068380 0x7fa58406a840 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa5a81401a0 tx=0x7fa590007df0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:32.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.285+0000 7fa59ea00640 1 -- 172.21.15.32:0/530999409 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa598016070 con 0x7fa5a806ba30 2024-04-17T10:01:32.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.449+0000 7fa5ad600640 1 -- 172.21.15.32:0/530999409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fa5a806c850 con 0x7fa5a806ba30 2024-04-17T10:01:32.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.453+0000 7fa59ea00640 1 -- 172.21.15.32:0/530999409 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7fa5980f0270 con 0x7fa5a806ba30 2024-04-17T10:01:32.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.453+0000 7fa5ad600640 1 -- 172.21.15.32:0/530999409 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa584068380 msgr2=0x7fa58406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:32.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.453+0000 7fa5ad600640 1 --2- 172.21.15.32:0/530999409 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa584068380 0x7fa58406a840 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa5a81401a0 tx=0x7fa590007df0 comp rx=0 tx=0).stop 2024-04-17T10:01:32.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.453+0000 7fa5ad600640 1 -- 172.21.15.32:0/530999409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a806ba30 msgr2=0x7fa5a806ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.453+0000 7fa5ad600640 1 --2- 172.21.15.32:0/530999409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a806ba30 0x7fa5a806ed90 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fa598019fc0 tx=0x7fa59801abd0 comp rx=0 tx=0).stop 2024-04-17T10:01:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.453+0000 7fa5ad600640 1 -- 172.21.15.32:0/530999409 shutdown_connections 2024-04-17T10:01:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.453+0000 7fa5ad600640 1 --2- 172.21.15.32:0/530999409 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa584068380 0x7fa58406a840 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.453+0000 7fa5ad600640 1 --2- 172.21.15.32:0/530999409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a8071e60 0x7fa5a81441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.453+0000 7fa5ad600640 1 --2- 172.21.15.32:0/530999409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a80cf370 0x7fa5a8142670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.453+0000 7fa5ad600640 1 --2- 172.21.15.32:0/530999409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a806ba30 0x7fa5a806ed90 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.453+0000 7fa5ad600640 1 -- 172.21.15.32:0/530999409 >> 172.21.15.32:0/530999409 conn(0x7fa5a80b2120 msgr2=0x7fa5a813d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.453+0000 7fa5ad600640 1 -- 172.21.15.32:0/530999409 shutdown_connections 2024-04-17T10:01:32.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.453+0000 7fa5ad600640 1 -- 172.21.15.32:0/530999409 wait complete. 2024-04-17T10:01:32.471 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-17T10:01:32.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.613+0000 7fef72e00640 1 Processor -- start 2024-04-17T10:01:32.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.613+0000 7fef72e00640 1 -- start start 2024-04-17T10:01:32.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.617+0000 7fef72e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef6c145f30 0x7fef6c071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:32.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.617+0000 7fef72e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fef6c15d8d0 0x7fef6c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:32.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.617+0000 7fef72e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef6c0cdac0 con 0x7fef6c0cce70 2024-04-17T10:01:32.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.617+0000 7fef72e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fef6c06f400 con 0x7fef6c15d8d0 2024-04-17T10:01:32.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.617+0000 7fef72e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fef6c146310 con 0x7fef6c145f30 2024-04-17T10:01:32.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.617+0000 7fef63400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fef6c15d8d0 0x7fef6c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:32.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.617+0000 7fef63400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fef6c15d8d0 0x7fef6c072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:39430/0 (socket says 172.21.15.32:39430) 2024-04-17T10:01:32.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.617+0000 7fef63400640 1 -- 172.21.15.32:0/443940738 learned_addr learned my addr 172.21.15.32:0/443940738 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:32.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.617+0000 7fef70e00640 1 --1- 172.21.15.32:0/443940738 >> v1:172.21.15.32:6789/0 conn(0x7fef6c0cce70 0x7fef6c0cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36076/0 (socket says 172.21.15.32:36076) 2024-04-17T10:01:32.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.617+0000 7fef6be00640 1 --2- 172.21.15.32:0/443940738 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef6c145f30 0x7fef6c071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:32.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.617+0000 7fef63400640 1 -- 172.21.15.32:0/443940738 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef6c145f30 msgr2=0x7fef6c071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:32.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.617+0000 7fef63400640 1 --2- 172.21.15.32:0/443940738 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef6c145f30 0x7fef6c071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:32.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.617+0000 7fef63400640 1 -- 172.21.15.32:0/443940738 >> v1:172.21.15.32:6789/0 conn(0x7fef6c0cce70 legacy=0x7fef6c0cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:32.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.617+0000 7fef63400640 1 -- 172.21.15.32:0/443940738 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6c15cdf0 con 0x7fef6c15d8d0 2024-04-17T10:01:32.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.617+0000 7fef6b400640 1 -- 172.21.15.32:0/443940738 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2694968760 0 0) 0x7fef6c0cdac0 con 0x7fef6c0cce70 2024-04-17T10:01:32.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.621+0000 7fef63400640 1 --2- 172.21.15.32:0/443940738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef6c15d8d0 0x7fef6c072330 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fef54009870 tx=0x7fef5401ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e61b651e88ad947c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:32.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.621+0000 7fef6b400640 1 -- 172.21.15.32:0/443940738 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef540b5070 con 0x7fef6c15d8d0 2024-04-17T10:01:32.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.621+0000 7fef6b400640 1 -- 172.21.15.32:0/443940738 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef540a8a50 con 0x7fef6c15d8d0 2024-04-17T10:01:32.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.621+0000 7fef6b400640 1 -- 172.21.15.32:0/443940738 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef540a8e20 con 0x7fef6c15d8d0 2024-04-17T10:01:32.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.621+0000 7fef72e00640 1 -- 172.21.15.32:0/443940738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef6c15d8d0 msgr2=0x7fef6c072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:32.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.621+0000 7fef72e00640 1 --2- 172.21.15.32:0/443940738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef6c15d8d0 0x7fef6c072330 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fef54009870 tx=0x7fef5401ed50 comp rx=0 tx=0).stop 2024-04-17T10:01:32.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.621+0000 7fef72e00640 1 -- 172.21.15.32:0/443940738 shutdown_connections 2024-04-17T10:01:32.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.621+0000 7fef72e00640 1 --2- 172.21.15.32:0/443940738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef6c15d8d0 0x7fef6c072330 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:32.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.621+0000 7fef72e00640 1 --2- 172.21.15.32:0/443940738 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef6c145f30 0x7fef6c071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:32.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.621+0000 7fef72e00640 1 -- 172.21.15.32:0/443940738 >> 172.21.15.32:0/443940738 conn(0x7fef6c06ba30 msgr2=0x7fef6c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:32.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.621+0000 7fef72e00640 1 -- 172.21.15.32:0/443940738 shutdown_connections 2024-04-17T10:01:32.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.621+0000 7fef72e00640 1 -- 172.21.15.32:0/443940738 wait complete. 2024-04-17T10:01:32.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.621+0000 7fef72e00640 1 Processor -- start 2024-04-17T10:01:32.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.621+0000 7fef72e00640 1 -- start start 2024-04-17T10:01:32.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef72e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef6c0cce70 0x7fef6c142df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:32.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef72e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef6c145f30 0x7fef6c15c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:32.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef72e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef6c15d8d0 0x7fef6c1e7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:32.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef72e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c15cb90 con 0x7fef6c0cce70 2024-04-17T10:01:32.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef72e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c15d440 con 0x7fef6c145f30 2024-04-17T10:01:32.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef72e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fef6c1601d0 con 0x7fef6c15d8d0 2024-04-17T10:01:32.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef6be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef6c0cce70 0x7fef6c142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef6be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef6c0cce70 0x7fef6c142df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53890/0 (socket says 172.21.15.32:53890) 2024-04-17T10:01:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef6be00640 1 -- 172.21.15.32:0/3207149021 learned_addr learned my addr 172.21.15.32:0/3207149021 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef63400640 1 --2- 172.21.15.32:0/3207149021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef6c145f30 0x7fef6c15c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef70e00640 1 --2- 172.21.15.32:0/3207149021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef6c15d8d0 0x7fef6c1e7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef6be00640 1 -- 172.21.15.32:0/3207149021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef6c15d8d0 msgr2=0x7fef6c1e7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef6be00640 1 --2- 172.21.15.32:0/3207149021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef6c15d8d0 0x7fef6c1e7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef6be00640 1 -- 172.21.15.32:0/3207149021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef6c145f30 msgr2=0x7fef6c15c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef6be00640 1 --2- 172.21.15.32:0/3207149021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef6c145f30 0x7fef6c15c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef6be00640 1 -- 172.21.15.32:0/3207149021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6c144850 con 0x7fef6c0cce70 2024-04-17T10:01:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef6be00640 1 --2- 172.21.15.32:0/3207149021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef6c0cce70 0x7fef6c142df0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00e930 tx=0x7fef5c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef68c00640 1 -- 172.21.15.32:0/3207149021 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c017070 con 0x7fef6c0cce70 2024-04-17T10:01:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef68c00640 1 -- 172.21.15.32:0/3207149021 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef5c00ca00 con 0x7fef6c0cce70 2024-04-17T10:01:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef68c00640 1 -- 172.21.15.32:0/3207149021 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c00cdb0 con 0x7fef6c0cce70 2024-04-17T10:01:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef72e00640 1 -- 172.21.15.32:0/3207149021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef6c142310 con 0x7fef6c0cce70 2024-04-17T10:01:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef72e00640 1 -- 172.21.15.32:0/3207149021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef6c1428d0 con 0x7fef6c0cce70 2024-04-17T10:01:32.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef72e00640 1 -- 172.21.15.32:0/3207149021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef6c068a90 con 0x7fef6c0cce70 2024-04-17T10:01:32.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.625+0000 7fef68c00640 1 -- 172.21.15.32:0/3207149021 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fef5c01a070 con 0x7fef6c0cce70 2024-04-17T10:01:32.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.629+0000 7fef68c00640 1 --2- 172.21.15.32:0/3207149021 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fef48068380 0x7fef4806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:32.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.629+0000 7fef68c00640 1 -- 172.21.15.32:0/3207149021 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5115+0+0 (secure 0 0 0) 0x7fef5c086aa0 con 0x7fef6c0cce70 2024-04-17T10:01:32.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.629+0000 7fef63400640 1 --2- 172.21.15.32:0/3207149021 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fef48068380 0x7fef4806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:32.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.629+0000 7fef63400640 1 --2- 172.21.15.32:0/3207149021 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fef48068380 0x7fef4806a840 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fef540b2000 tx=0x7fef54009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:32.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.633+0000 7fef68c00640 1 -- 172.21.15.32:0/3207149021 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fef5c056d10 con 0x7fef6c0cce70 2024-04-17T10:01:32.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:32.853+0000 7fef72e00640 1 -- 172.21.15.32:0/3207149021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7fef6c0b4b30 con 0x7fef6c0cce70 2024-04-17T10:01:33.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.085+0000 7fef68c00640 1 -- 172.21.15.32:0/3207149021 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v396) ==== 167+0+0 (secure 0 0 0) 0x7fef5c05b010 con 0x7fef6c0cce70 2024-04-17T10:01:33.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.085+0000 7fef72e00640 1 -- 172.21.15.32:0/3207149021 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fef48068380 msgr2=0x7fef4806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:33.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.085+0000 7fef72e00640 1 --2- 172.21.15.32:0/3207149021 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fef48068380 0x7fef4806a840 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fef540b2000 tx=0x7fef54009040 comp rx=0 tx=0).stop 2024-04-17T10:01:33.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.085+0000 7fef72e00640 1 -- 172.21.15.32:0/3207149021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef6c0cce70 msgr2=0x7fef6c142df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.085+0000 7fef72e00640 1 --2- 172.21.15.32:0/3207149021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef6c0cce70 0x7fef6c142df0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00e930 tx=0x7fef5c00edf0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.085+0000 7fef72e00640 1 -- 172.21.15.32:0/3207149021 shutdown_connections 2024-04-17T10:01:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.085+0000 7fef72e00640 1 --2- 172.21.15.32:0/3207149021 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fef48068380 0x7fef4806a840 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.085+0000 7fef72e00640 1 --2- 172.21.15.32:0/3207149021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef6c15d8d0 0x7fef6c1e7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.085+0000 7fef72e00640 1 --2- 172.21.15.32:0/3207149021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef6c145f30 0x7fef6c15c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.085+0000 7fef72e00640 1 --2- 172.21.15.32:0/3207149021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef6c0cce70 0x7fef6c142df0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.085+0000 7fef72e00640 1 -- 172.21.15.32:0/3207149021 >> 172.21.15.32:0/3207149021 conn(0x7fef6c06ba30 msgr2=0x7fef6c15dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.085+0000 7fef72e00640 1 -- 172.21.15.32:0/3207149021 shutdown_connections 2024-04-17T10:01:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.085+0000 7fef72e00640 1 -- 172.21.15.32:0/3207149021 wait complete. 2024-04-17T10:01:33.103 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-17T10:01:33.103 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-17T10:01:33.104 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:01:33.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.249+0000 7f6845e00640 1 Processor -- start 2024-04-17T10:01:33.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.249+0000 7f6845e00640 1 -- start start 2024-04-17T10:01:33.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.249+0000 7f6845e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6840072500 0x7f68400cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:33.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.249+0000 7f6845e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684015d8d0 0x7f68400cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:33.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.249+0000 7f6845e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68400cdac0 con 0x7f6840145f30 2024-04-17T10:01:33.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.249+0000 7f6845e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f684006f400 con 0x7f6840072500 2024-04-17T10:01:33.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.249+0000 7f6845e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6840146310 con 0x7f684015d8d0 2024-04-17T10:01:33.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.249+0000 7f6836000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6840072500 0x7f68400cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:33.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.249+0000 7f683f400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684015d8d0 0x7f68400cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:33.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.249+0000 7f683f400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684015d8d0 0x7f68400cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:44896/0 (socket says 172.21.15.32:44896) 2024-04-17T10:01:33.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.249+0000 7f6836000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6840072500 0x7f68400cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:39446/0 (socket says 172.21.15.32:39446) 2024-04-17T10:01:33.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.249+0000 7f683f400640 1 -- 172.21.15.32:0/1840255720 learned_addr learned my addr 172.21.15.32:0/1840255720 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:33.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.249+0000 7f683ea00640 1 --1- 172.21.15.32:0/1840255720 >> v1:172.21.15.144:6789/0 conn(0x7f6840145f30 0x7f6840071df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:38970/0 (socket says 172.21.15.32:38970) 2024-04-17T10:01:33.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.249+0000 7f6836000640 1 -- 172.21.15.32:0/1840255720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684015d8d0 msgr2=0x7f68400cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:33.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.249+0000 7f6836000640 1 --2- 172.21.15.32:0/1840255720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684015d8d0 0x7f68400cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.249+0000 7f6836000640 1 -- 172.21.15.32:0/1840255720 >> v1:172.21.15.144:6789/0 conn(0x7f6840145f30 legacy=0x7f6840071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:33.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.249+0000 7f6836000640 1 -- 172.21.15.32:0/1840255720 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f684015cdf0 con 0x7f6840072500 2024-04-17T10:01:33.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.249+0000 7f683f400640 1 --2- 172.21.15.32:0/1840255720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684015d8d0 0x7f68400cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:01:33.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.253+0000 7f6836000640 1 --2- 172.21.15.32:0/1840255720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6840072500 0x7f68400cce70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f68280097f0 tx=0x7f682801ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2b5bf8f0992406d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:33.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.253+0000 7f683e000640 1 -- 172.21.15.32:0/1840255720 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68280b5070 con 0x7f6840072500 2024-04-17T10:01:33.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.253+0000 7f683e000640 1 -- 172.21.15.32:0/1840255720 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68280a8990 con 0x7f6840072500 2024-04-17T10:01:33.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.253+0000 7f683e000640 1 -- 172.21.15.32:0/1840255720 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68280a8d60 con 0x7f6840072500 2024-04-17T10:01:33.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.253+0000 7f6845e00640 1 -- 172.21.15.32:0/1840255720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6840072500 msgr2=0x7f68400cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:33.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.253+0000 7f6845e00640 1 --2- 172.21.15.32:0/1840255720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6840072500 0x7f68400cce70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f68280097f0 tx=0x7f682801ec90 comp rx=0 tx=0).stop 2024-04-17T10:01:33.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.253+0000 7f6845e00640 1 -- 172.21.15.32:0/1840255720 shutdown_connections 2024-04-17T10:01:33.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.253+0000 7f6845e00640 1 --2- 172.21.15.32:0/1840255720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684015d8d0 0x7f68400cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.253+0000 7f6845e00640 1 --2- 172.21.15.32:0/1840255720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6840072500 0x7f68400cce70 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.253+0000 7f6845e00640 1 -- 172.21.15.32:0/1840255720 >> 172.21.15.32:0/1840255720 conn(0x7f684006ba30 msgr2=0x7f684006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:33.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.253+0000 7f6845e00640 1 -- 172.21.15.32:0/1840255720 shutdown_connections 2024-04-17T10:01:33.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.253+0000 7f6845e00640 1 -- 172.21.15.32:0/1840255720 wait complete. 2024-04-17T10:01:33.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f6845e00640 1 Processor -- start 2024-04-17T10:01:33.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f6845e00640 1 -- start start 2024-04-17T10:01:33.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f6845e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6840072500 0x7f6840144580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:33.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f6845e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6840145f30 0x7f684013e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:33.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f6845e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684015d8d0 0x7f684013eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:33.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f6845e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f684015cb90 con 0x7f684015d8d0 2024-04-17T10:01:33.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f6845e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f684015d440 con 0x7f6840072500 2024-04-17T10:01:33.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f6845e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f68401601d0 con 0x7f6840145f30 2024-04-17T10:01:33.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f683f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684015d8d0 0x7f684013eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:33.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f683f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684015d8d0 0x7f684013eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53896/0 (socket says 172.21.15.32:53896) 2024-04-17T10:01:33.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f683f400640 1 -- 172.21.15.32:0/4219884589 learned_addr learned my addr 172.21.15.32:0/4219884589 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:33.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f6836000640 1 --2- 172.21.15.32:0/4219884589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6840145f30 0x7f684013e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:33.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f683ea00640 1 --2- 172.21.15.32:0/4219884589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6840072500 0x7f6840144580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:33.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f683f400640 1 -- 172.21.15.32:0/4219884589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6840145f30 msgr2=0x7f684013e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:33.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f683f400640 1 --2- 172.21.15.32:0/4219884589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6840145f30 0x7f684013e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f683f400640 1 -- 172.21.15.32:0/4219884589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6840072500 msgr2=0x7f6840144580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:33.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f683f400640 1 --2- 172.21.15.32:0/4219884589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6840072500 0x7f6840144580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f683f400640 1 -- 172.21.15.32:0/4219884589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f684013f1a0 con 0x7f684015d8d0 2024-04-17T10:01:33.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f683f400640 1 --2- 172.21.15.32:0/4219884589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684015d8d0 0x7f684013eb20 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f683800e960 tx=0x7f683800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:33.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f6837e00640 1 -- 172.21.15.32:0/4219884589 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6838017070 con 0x7f684015d8d0 2024-04-17T10:01:33.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f6837e00640 1 -- 172.21.15.32:0/4219884589 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f683800ca00 con 0x7f684015d8d0 2024-04-17T10:01:33.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f6837e00640 1 -- 172.21.15.32:0/4219884589 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683800cdd0 con 0x7f684015d8d0 2024-04-17T10:01:33.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f6845e00640 1 -- 172.21.15.32:0/4219884589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f684013f480 con 0x7f684015d8d0 2024-04-17T10:01:33.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.257+0000 7f6845e00640 1 -- 172.21.15.32:0/4219884589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f684013fa40 con 0x7f684015d8d0 2024-04-17T10:01:33.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.261+0000 7f6845e00640 1 -- 172.21.15.32:0/4219884589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6840068a90 con 0x7f684015d8d0 2024-04-17T10:01:33.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.261+0000 7f6837e00640 1 -- 172.21.15.32:0/4219884589 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6838013470 con 0x7f684015d8d0 2024-04-17T10:01:33.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.261+0000 7f6837e00640 1 --2- 172.21.15.32:0/4219884589 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f681c068380 0x7f681c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:33.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.261+0000 7f6837e00640 1 -- 172.21.15.32:0/4219884589 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5115+0+0 (secure 0 0 0) 0x7f6838086990 con 0x7f684015d8d0 2024-04-17T10:01:33.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.261+0000 7f683ea00640 1 --2- 172.21.15.32:0/4219884589 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f681c068380 0x7f681c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:33.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.261+0000 7f683ea00640 1 --2- 172.21.15.32:0/4219884589 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f681c068380 0x7f681c06a840 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f68300012f0 tx=0x7f6830005b80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:33.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.265+0000 7f6837e00640 1 -- 172.21.15.32:0/4219884589 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f6838057100 con 0x7f684015d8d0 2024-04-17T10:01:33.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.433+0000 7f6845e00640 1 -- 172.21.15.32:0/4219884589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68401f4870 con 0x7f684015d8d0 2024-04-17T10:01:33.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.433+0000 7f6837e00640 1 -- 172.21.15.32:0/4219884589 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+13293 (secure 0 0 0) 0x7f6838056a00 con 0x7f684015d8d0 2024-04-17T10:01:33.431 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:33.431 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":89,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:01:32.078033+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6837/2058904361":"2024-04-18T10:01:28.990081+0000","172.21.15.32:6837/1253497636":"2024-04-18T10:00:43.842876+0000","172.21.15.32:6835/2058904361":"2024-04-18T10:01:28.990081+0000","172.21.15.32:6835/1253497636":"2024-04-18T10:00:43.842876+0000","172.21.15.32:6837/2656735817":"2024-04-18T09:59:28.985107+0000","172.21.15.32:6835/2656735817":"2024-04-18T09:59:28.985107+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:01:33.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.433+0000 7f6845e00640 1 -- 172.21.15.32:0/4219884589 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f681c068380 msgr2=0x7f681c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:33.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.433+0000 7f6845e00640 1 --2- 172.21.15.32:0/4219884589 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f681c068380 0x7f681c06a840 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f68300012f0 tx=0x7f6830005b80 comp rx=0 tx=0).stop 2024-04-17T10:01:33.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.433+0000 7f6845e00640 1 -- 172.21.15.32:0/4219884589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684015d8d0 msgr2=0x7f684013eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:33.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.433+0000 7f6845e00640 1 --2- 172.21.15.32:0/4219884589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684015d8d0 0x7f684013eb20 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f683800e960 tx=0x7f683800ee20 comp rx=0 tx=0).stop 2024-04-17T10:01:33.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.437+0000 7f6845e00640 1 -- 172.21.15.32:0/4219884589 shutdown_connections 2024-04-17T10:01:33.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.437+0000 7f6845e00640 1 --2- 172.21.15.32:0/4219884589 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f681c068380 0x7f681c06a840 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.437+0000 7f6845e00640 1 --2- 172.21.15.32:0/4219884589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6840145f30 0x7f684013e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.437+0000 7f6845e00640 1 --2- 172.21.15.32:0/4219884589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f684015d8d0 0x7f684013eb20 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.437+0000 7f6845e00640 1 --2- 172.21.15.32:0/4219884589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6840072500 0x7f6840144580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.437+0000 7f6845e00640 1 -- 172.21.15.32:0/4219884589 >> 172.21.15.32:0/4219884589 conn(0x7f684006ba30 msgr2=0x7f6840160600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:33.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.437+0000 7f6845e00640 1 -- 172.21.15.32:0/4219884589 shutdown_connections 2024-04-17T10:01:33.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.437+0000 7f6845e00640 1 -- 172.21.15.32:0/4219884589 wait complete. 2024-04-17T10:01:33.453 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:01:33.453 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:01:33.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.597+0000 7f5bb5600640 1 Processor -- start 2024-04-17T10:01:33.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.601+0000 7f5bb5600640 1 -- start start 2024-04-17T10:01:33.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.601+0000 7f5bb5600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5bb00cf370 0x7f5bb00cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:33.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.601+0000 7f5bb5600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bb0071e60 0x7f5bb0072270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:33.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.601+0000 7f5bb5600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bb00d0130 con 0x7f5bb006ba30 2024-04-17T10:01:33.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.601+0000 7f5bb5600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5bb015d090 con 0x7f5bb00cf370 2024-04-17T10:01:33.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.601+0000 7f5bb5600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5bb015d370 con 0x7f5bb0071e60 2024-04-17T10:01:33.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.601+0000 7f5baea00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5bb006ba30 0x7f5bb006be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36092/0 (socket says 172.21.15.32:36092) 2024-04-17T10:01:33.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.601+0000 7f5baea00640 1 -- 172.21.15.32:0/1014041812 learned_addr learned my addr 172.21.15.32:0/1014041812 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:33.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.601+0000 7f5bae000640 1 --2- 172.21.15.32:0/1014041812 >> v2:172.21.15.32:3300/0 conn(0x7f5bb00cf370 0x7f5bb00cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:33.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.601+0000 7f5baf400640 1 --2- 172.21.15.32:0/1014041812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bb0071e60 0x7f5bb0072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:33.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.601+0000 7f5bad600640 1 -- 172.21.15.32:0/1014041812 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2631009845 0 0) 0x7f5bb00d0130 con 0x7f5bb006ba30 2024-04-17T10:01:33.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.601+0000 7f5bad600640 1 -- 172.21.15.32:0/1014041812 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b94003530 con 0x7f5bb006ba30 2024-04-17T10:01:33.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.601+0000 7f5bae000640 1 -- 172.21.15.32:0/1014041812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bb0071e60 msgr2=0x7f5bb0072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:33.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.601+0000 7f5bae000640 1 --2- 172.21.15.32:0/1014041812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bb0071e60 0x7f5bb0072270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.601+0000 7f5bae000640 1 -- 172.21.15.32:0/1014041812 >> v1:172.21.15.32:6789/0 conn(0x7f5bb006ba30 legacy=0x7f5bb006be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:33.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.601+0000 7f5bae000640 1 -- 172.21.15.32:0/1014041812 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb015d650 con 0x7f5bb00cf370 2024-04-17T10:01:33.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.605+0000 7f5bae000640 1 --2- 172.21.15.32:0/1014041812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb00cf370 0x7f5bb00cf7e0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c0097c0 tx=0x7f5b9c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32000667f749c8a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:33.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.605+0000 7f5bad600640 1 -- 172.21.15.32:0/1014041812 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c0aa620 con 0x7f5bb00cf370 2024-04-17T10:01:33.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.605+0000 7f5bad600640 1 -- 172.21.15.32:0/1014041812 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b9c0af070 con 0x7f5bb00cf370 2024-04-17T10:01:33.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.605+0000 7f5bad600640 1 -- 172.21.15.32:0/1014041812 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c0aa9c0 con 0x7f5bb00cf370 2024-04-17T10:01:33.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.605+0000 7f5bb5600640 1 -- 172.21.15.32:0/1014041812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb00cf370 msgr2=0x7f5bb00cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:33.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.605+0000 7f5bb5600640 1 --2- 172.21.15.32:0/1014041812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb00cf370 0x7f5bb00cf7e0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c0097c0 tx=0x7f5b9c01c4a0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.605+0000 7f5bb5600640 1 -- 172.21.15.32:0/1014041812 shutdown_connections 2024-04-17T10:01:33.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.605+0000 7f5bb5600640 1 --2- 172.21.15.32:0/1014041812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bb0071e60 0x7f5bb0072270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.605+0000 7f5bb5600640 1 --2- 172.21.15.32:0/1014041812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb00cf370 0x7f5bb00cf7e0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.605+0000 7f5bb5600640 1 -- 172.21.15.32:0/1014041812 >> 172.21.15.32:0/1014041812 conn(0x7f5bb00b2120 msgr2=0x7f5bb00b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:33.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.605+0000 7f5bb5600640 1 -- 172.21.15.32:0/1014041812 shutdown_connections 2024-04-17T10:01:33.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.605+0000 7f5bb5600640 1 -- 172.21.15.32:0/1014041812 wait complete. 2024-04-17T10:01:33.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.605+0000 7f5bb5600640 1 Processor -- start 2024-04-17T10:01:33.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.605+0000 7f5bb5600640 1 -- start start 2024-04-17T10:01:33.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5bb5600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bb006ba30 0x7f5bb015b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:33.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5bb5600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bb0071e60 0x7f5bb013ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:33.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5bb5600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb00cf370 0x7f5bb013fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:33.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5bb5600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb0146230 con 0x7f5bb00cf370 2024-04-17T10:01:33.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5bb5600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb015dd80 con 0x7f5bb006ba30 2024-04-17T10:01:33.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5bb5600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5bb01463b0 con 0x7f5bb0071e60 2024-04-17T10:01:33.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5baf400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb00cf370 0x7f5bb013fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:33.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5baf400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb00cf370 0x7f5bb013fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53914/0 (socket says 172.21.15.32:53914) 2024-04-17T10:01:33.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5baf400640 1 -- 172.21.15.32:0/1163642745 learned_addr learned my addr 172.21.15.32:0/1163642745 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:33.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5bae000640 1 --2- 172.21.15.32:0/1163642745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bb0071e60 0x7f5bb013ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:33.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5baea00640 1 --2- 172.21.15.32:0/1163642745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bb006ba30 0x7f5bb015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:33.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5baf400640 1 -- 172.21.15.32:0/1163642745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bb0071e60 msgr2=0x7f5bb013ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:33.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5baf400640 1 --2- 172.21.15.32:0/1163642745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bb0071e60 0x7f5bb013ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5baf400640 1 -- 172.21.15.32:0/1163642745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bb006ba30 msgr2=0x7f5bb015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:33.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5baf400640 1 --2- 172.21.15.32:0/1163642745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bb006ba30 0x7f5bb015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5baf400640 1 -- 172.21.15.32:0/1163642745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb013cf70 con 0x7f5bb00cf370 2024-04-17T10:01:33.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5baf400640 1 --2- 172.21.15.32:0/1163642745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb00cf370 0x7f5bb013fbb0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8007e70 tx=0x7f5ba800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:33.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5ba6a00640 1 -- 172.21.15.32:0/1163642745 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8014620 con 0x7f5bb00cf370 2024-04-17T10:01:33.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5ba6a00640 1 -- 172.21.15.32:0/1163642745 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ba8019070 con 0x7f5bb00cf370 2024-04-17T10:01:33.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5ba6a00640 1 -- 172.21.15.32:0/1163642745 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8014a20 con 0x7f5bb00cf370 2024-04-17T10:01:33.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5bb5600640 1 -- 172.21.15.32:0/1163642745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb015a160 con 0x7f5bb00cf370 2024-04-17T10:01:33.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5bb5600640 1 -- 172.21.15.32:0/1163642745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb015a720 con 0x7f5bb00cf370 2024-04-17T10:01:33.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5bb5600640 1 -- 172.21.15.32:0/1163642745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb0068a90 con 0x7f5bb00cf370 2024-04-17T10:01:33.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.609+0000 7f5ba6a00640 1 -- 172.21.15.32:0/1163642745 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5ba8014bc0 con 0x7f5bb00cf370 2024-04-17T10:01:33.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.613+0000 7f5ba6a00640 1 --2- 172.21.15.32:0/1163642745 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5b90068380 0x7f5b9006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:33.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.613+0000 7f5ba6a00640 1 -- 172.21.15.32:0/1163642745 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5115+0+0 (secure 0 0 0) 0x7f5ba8086fd0 con 0x7f5bb00cf370 2024-04-17T10:01:33.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.613+0000 7f5baea00640 1 --2- 172.21.15.32:0/1163642745 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5b90068380 0x7f5b9006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:33.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.613+0000 7f5baea00640 1 --2- 172.21.15.32:0/1163642745 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5b90068380 0x7f5b9006a840 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5bb01fab70 tx=0x7f5ba0000bd0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:33.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.617+0000 7f5ba6a00640 1 -- 172.21.15.32:0/1163642745 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5ba80570f0 con 0x7f5bb00cf370 2024-04-17T10:01:33.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.781+0000 7f5bb5600640 1 -- 172.21.15.32:0/1163642745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5bb006c850 con 0x7f5bb00cf370 2024-04-17T10:01:33.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.781+0000 7f5ba6a00640 1 -- 172.21.15.32:0/1163642745 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f5ba805b3f0 con 0x7f5bb00cf370 2024-04-17T10:01:33.777 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T10:01:33.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.781+0000 7f5bb5600640 1 -- 172.21.15.32:0/1163642745 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5b90068380 msgr2=0x7f5b9006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:33.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.781+0000 7f5bb5600640 1 --2- 172.21.15.32:0/1163642745 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5b90068380 0x7f5b9006a840 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5bb01fab70 tx=0x7f5ba0000bd0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.781+0000 7f5bb5600640 1 -- 172.21.15.32:0/1163642745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb00cf370 msgr2=0x7f5bb013fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:33.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.781+0000 7f5bb5600640 1 --2- 172.21.15.32:0/1163642745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb00cf370 0x7f5bb013fbb0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8007e70 tx=0x7f5ba800e4b0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.781+0000 7f5bb5600640 1 -- 172.21.15.32:0/1163642745 shutdown_connections 2024-04-17T10:01:33.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.781+0000 7f5bb5600640 1 --2- 172.21.15.32:0/1163642745 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5b90068380 0x7f5b9006a840 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.781+0000 7f5bb5600640 1 --2- 172.21.15.32:0/1163642745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bb0071e60 0x7f5bb013ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.781+0000 7f5bb5600640 1 --2- 172.21.15.32:0/1163642745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb00cf370 0x7f5bb013fbb0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.781+0000 7f5bb5600640 1 --2- 172.21.15.32:0/1163642745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bb006ba30 0x7f5bb015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.781+0000 7f5bb5600640 1 -- 172.21.15.32:0/1163642745 >> 172.21.15.32:0/1163642745 conn(0x7f5bb00b2120 msgr2=0x7f5bb00aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:33.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.781+0000 7f5bb5600640 1 -- 172.21.15.32:0/1163642745 shutdown_connections 2024-04-17T10:01:33.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.781+0000 7f5bb5600640 1 -- 172.21.15.32:0/1163642745 wait complete. 2024-04-17T10:01:33.799 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-17T10:01:33.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.941+0000 7f7376800640 1 Processor -- start 2024-04-17T10:01:33.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.941+0000 7f7376800640 1 -- start start 2024-04-17T10:01:33.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.945+0000 7f7376800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7370144e90 0x7f73700b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:33.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.945+0000 7f7376800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f737015c1b0 0x7f737006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:33.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.945+0000 7f7376800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7370072bb0 con 0x7f73700b4de0 2024-04-17T10:01:33.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.945+0000 7f7376800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7370145270 con 0x7f7370144e90 2024-04-17T10:01:33.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.945+0000 7f7376800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f73700b3dc0 con 0x7f737015c1b0 2024-04-17T10:01:33.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.945+0000 7f736f400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7370144e90 0x7f73700b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:33.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.945+0000 7f736f400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7370144e90 0x7f73700b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53926/0 (socket says 172.21.15.32:53926) 2024-04-17T10:01:33.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.945+0000 7f736f400640 1 -- 172.21.15.32:0/3535956443 learned_addr learned my addr 172.21.15.32:0/3535956443 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:33.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.945+0000 7f736fe00640 1 --2- 172.21.15.32:0/3535956443 >> v2:172.21.15.144:3300/0 conn(0x7f737015c1b0 0x7f737006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:33.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.945+0000 7f736f400640 1 -- 172.21.15.32:0/3535956443 >> v2:172.21.15.144:3300/0 conn(0x7f737015c1b0 msgr2=0x7f737006ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:33.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.945+0000 7f736f400640 1 --2- 172.21.15.32:0/3535956443 >> v2:172.21.15.144:3300/0 conn(0x7f737015c1b0 0x7f737006ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.945+0000 7f736f400640 1 -- 172.21.15.32:0/3535956443 >> v1:172.21.15.144:6789/0 conn(0x7f73700b4de0 legacy=0x7f737015baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:33.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.945+0000 7f736f400640 1 -- 172.21.15.32:0/3535956443 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73700b40a0 con 0x7f7370144e90 2024-04-17T10:01:33.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.945+0000 7f736fe00640 1 --2- 172.21.15.32:0/3535956443 >> v2:172.21.15.144:3300/0 conn(0x7f737015c1b0 0x7f737006ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:33.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.949+0000 7f736f400640 1 --2- 172.21.15.32:0/3535956443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7370144e90 0x7f73700b48a0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f737006ff30 tx=0x7f736001eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712b775d74a715db server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:33.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.949+0000 7f736ea00640 1 -- 172.21.15.32:0/3535956443 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73600b4070 con 0x7f7370144e90 2024-04-17T10:01:33.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.949+0000 7f736ea00640 1 -- 172.21.15.32:0/3535956443 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73600a7b40 con 0x7f7370144e90 2024-04-17T10:01:33.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.949+0000 7f736ea00640 1 -- 172.21.15.32:0/3535956443 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73600af440 con 0x7f7370144e90 2024-04-17T10:01:33.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.949+0000 7f7376800640 1 -- 172.21.15.32:0/3535956443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7370144e90 msgr2=0x7f73700b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:33.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.949+0000 7f7376800640 1 --2- 172.21.15.32:0/3535956443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7370144e90 0x7f73700b48a0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f737006ff30 tx=0x7f736001eb60 comp rx=0 tx=0).stop 2024-04-17T10:01:33.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.949+0000 7f7376800640 1 -- 172.21.15.32:0/3535956443 shutdown_connections 2024-04-17T10:01:33.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.949+0000 7f7376800640 1 --2- 172.21.15.32:0/3535956443 >> v2:172.21.15.144:3300/0 conn(0x7f737015c1b0 0x7f737006ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.949+0000 7f7376800640 1 --2- 172.21.15.32:0/3535956443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7370144e90 0x7f73700b48a0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.949+0000 7f7376800640 1 -- 172.21.15.32:0/3535956443 >> 172.21.15.32:0/3535956443 conn(0x7f737006ba30 msgr2=0x7f737006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:33.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.949+0000 7f7376800640 1 -- 172.21.15.32:0/3535956443 shutdown_connections 2024-04-17T10:01:33.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.949+0000 7f7376800640 1 -- 172.21.15.32:0/3535956443 wait complete. 2024-04-17T10:01:33.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.949+0000 7f7376800640 1 Processor -- start 2024-04-17T10:01:33.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.949+0000 7f7376800640 1 -- start start 2024-04-17T10:01:33.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f7376800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73700b4de0 0x7f737013a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:33.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f7376800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7370144e90 0x7f7370158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:33.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f7376800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f737015c1b0 0x7f7370141ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:33.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f7376800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f737006de50 con 0x7f737015c1b0 2024-04-17T10:01:33.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f7376800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7370072880 con 0x7f7370144e90 2024-04-17T10:01:33.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f7376800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f737006dfd0 con 0x7f73700b4de0 2024-04-17T10:01:33.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f736fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f737015c1b0 0x7f7370141ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:33.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f736fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f737015c1b0 0x7f7370141ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53930/0 (socket says 172.21.15.32:53930) 2024-04-17T10:01:33.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f736fe00640 1 -- 172.21.15.32:0/2772258568 learned_addr learned my addr 172.21.15.32:0/2772258568 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:33.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f7367e00640 1 --2- 172.21.15.32:0/2772258568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7370144e90 0x7f7370158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:33.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f736f400640 1 --2- 172.21.15.32:0/2772258568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73700b4de0 0x7f737013a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:33.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f736fe00640 1 -- 172.21.15.32:0/2772258568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73700b4de0 msgr2=0x7f737013a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:33.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f736fe00640 1 --2- 172.21.15.32:0/2772258568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73700b4de0 0x7f737013a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f736fe00640 1 -- 172.21.15.32:0/2772258568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7370144e90 msgr2=0x7f7370158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:33.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f736fe00640 1 --2- 172.21.15.32:0/2772258568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7370144e90 0x7f7370158a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:33.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f736fe00640 1 -- 172.21.15.32:0/2772258568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7370141c50 con 0x7f737015c1b0 2024-04-17T10:01:33.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f736fe00640 1 --2- 172.21.15.32:0/2772258568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f737015c1b0 0x7f7370141ff0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f736800cd80 tx=0x7f736800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:33.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f7367400640 1 -- 172.21.15.32:0/2772258568 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736800f680 con 0x7f737015c1b0 2024-04-17T10:01:33.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f7367400640 1 -- 172.21.15.32:0/2772258568 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f736800f840 con 0x7f737015c1b0 2024-04-17T10:01:33.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f7367400640 1 -- 172.21.15.32:0/2772258568 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736800fc30 con 0x7f737015c1b0 2024-04-17T10:01:33.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f7376800640 1 -- 172.21.15.32:0/2772258568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7370141040 con 0x7f737015c1b0 2024-04-17T10:01:33.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f7376800640 1 -- 172.21.15.32:0/2772258568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7370141600 con 0x7f737015c1b0 2024-04-17T10:01:33.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f7376800640 1 -- 172.21.15.32:0/2772258568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7370068a90 con 0x7f737015c1b0 2024-04-17T10:01:33.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.953+0000 7f7367400640 1 -- 172.21.15.32:0/2772258568 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f73680213f0 con 0x7f737015c1b0 2024-04-17T10:01:33.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.957+0000 7f7367400640 1 --2- 172.21.15.32:0/2772258568 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f734c0683b0 0x7f734c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:33.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.957+0000 7f7367400640 1 -- 172.21.15.32:0/2772258568 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5115+0+0 (secure 0 0 0) 0x7f73680874f0 con 0x7f737015c1b0 2024-04-17T10:01:33.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.957+0000 7f736f400640 1 --2- 172.21.15.32:0/2772258568 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f734c0683b0 0x7f734c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:33.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.957+0000 7f736f400640 1 --2- 172.21.15.32:0/2772258568 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f734c0683b0 0x7f734c06a870 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f73600b1000 tx=0x7f73600a9070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:33.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:33.961+0000 7f7367400640 1 -- 172.21.15.32:0/2772258568 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f7368014030 con 0x7f737015c1b0 2024-04-17T10:01:34.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:34.181+0000 7f7376800640 1 -- 172.21.15.32:0/2772258568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7f73701f6cb0 con 0x7f737015c1b0 2024-04-17T10:01:35.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.089+0000 7f7367400640 1 -- 172.21.15.32:0/2772258568 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v398) ==== 149+0+0 (secure 0 0 0) 0x7f7368057c60 con 0x7f737015c1b0 2024-04-17T10:01:35.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.093+0000 7f7376800640 1 -- 172.21.15.32:0/2772258568 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f734c0683b0 msgr2=0x7f734c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:35.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.093+0000 7f7376800640 1 --2- 172.21.15.32:0/2772258568 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f734c0683b0 0x7f734c06a870 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f73600b1000 tx=0x7f73600a9070 comp rx=0 tx=0).stop 2024-04-17T10:01:35.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.093+0000 7f7376800640 1 -- 172.21.15.32:0/2772258568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f737015c1b0 msgr2=0x7f7370141ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:35.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.093+0000 7f7376800640 1 --2- 172.21.15.32:0/2772258568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f737015c1b0 0x7f7370141ff0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f736800cd80 tx=0x7f736800a650 comp rx=0 tx=0).stop 2024-04-17T10:01:35.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.093+0000 7f7376800640 1 -- 172.21.15.32:0/2772258568 shutdown_connections 2024-04-17T10:01:35.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.093+0000 7f7376800640 1 --2- 172.21.15.32:0/2772258568 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f734c0683b0 0x7f734c06a870 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:35.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.093+0000 7f7376800640 1 --2- 172.21.15.32:0/2772258568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f737015c1b0 0x7f7370141ff0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:35.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.093+0000 7f7376800640 1 --2- 172.21.15.32:0/2772258568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7370144e90 0x7f7370158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:35.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.093+0000 7f7376800640 1 --2- 172.21.15.32:0/2772258568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73700b4de0 0x7f737013a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:35.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.093+0000 7f7376800640 1 -- 172.21.15.32:0/2772258568 >> 172.21.15.32:0/2772258568 conn(0x7f737006ba30 msgr2=0x7f73701403f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:35.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.093+0000 7f7376800640 1 -- 172.21.15.32:0/2772258568 shutdown_connections 2024-04-17T10:01:35.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.093+0000 7f7376800640 1 -- 172.21.15.32:0/2772258568 wait complete. 2024-04-17T10:01:35.114 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-17T10:01:35.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.265+0000 7f2cf0a00640 1 Processor -- start 2024-04-17T10:01:35.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.265+0000 7f2cf0a00640 1 -- start start 2024-04-17T10:01:35.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.265+0000 7f2cf0a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2cec0cf370 0x7f2cec0cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:35.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.265+0000 7f2cf0a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2cec071e60 0x7f2cec072270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:35.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.265+0000 7f2cf0a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cec0d0130 con 0x7f2cec06ba30 2024-04-17T10:01:35.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.265+0000 7f2cf0a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2cec15d090 con 0x7f2cec0cf370 2024-04-17T10:01:35.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.265+0000 7f2cf0a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f2cec15d370 con 0x7f2cec071e60 2024-04-17T10:01:35.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.265+0000 7f2ce3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2cec0cf370 0x7f2cec0cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:35.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.265+0000 7f2ce3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2cec0cf370 0x7f2cec0cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53934/0 (socket says 172.21.15.32:53934) 2024-04-17T10:01:35.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.265+0000 7f2ce3e00640 1 -- 172.21.15.32:0/1492114845 learned_addr learned my addr 172.21.15.32:0/1492114845 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:35.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.265+0000 7f2cea800640 1 --2- 172.21.15.32:0/1492114845 >> v2:172.21.15.144:3300/0 conn(0x7f2cec071e60 0x7f2cec072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:35.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.265+0000 7f2ce3e00640 1 -- 172.21.15.32:0/1492114845 >> v2:172.21.15.144:3300/0 conn(0x7f2cec071e60 msgr2=0x7f2cec072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:35.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.265+0000 7f2ce3e00640 1 --2- 172.21.15.32:0/1492114845 >> v2:172.21.15.144:3300/0 conn(0x7f2cec071e60 0x7f2cec072270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:35.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.265+0000 7f2ce3e00640 1 -- 172.21.15.32:0/1492114845 >> v1:172.21.15.144:6789/0 conn(0x7f2cec06ba30 legacy=0x7f2cec06be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:35.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.265+0000 7f2ce3e00640 1 -- 172.21.15.32:0/1492114845 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cec15d650 con 0x7f2cec0cf370 2024-04-17T10:01:35.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.269+0000 7f2ce3e00640 1 --2- 172.21.15.32:0/1492114845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cec0cf370 0x7f2cec0cf7e0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f2cec15a950 tx=0x7f2cd401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9277930b86d1c59b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:35.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.269+0000 7f2ce9400640 1 -- 172.21.15.32:0/1492114845 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd40a9400 con 0x7f2cec0cf370 2024-04-17T10:01:35.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.269+0000 7f2ce9400640 1 -- 172.21.15.32:0/1492114845 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cd40a95a0 con 0x7f2cec0cf370 2024-04-17T10:01:35.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.269+0000 7f2ce9400640 1 -- 172.21.15.32:0/1492114845 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd40a9970 con 0x7f2cec0cf370 2024-04-17T10:01:35.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.269+0000 7f2cf0a00640 1 -- 172.21.15.32:0/1492114845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cec0cf370 msgr2=0x7f2cec0cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:35.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.269+0000 7f2cf0a00640 1 --2- 172.21.15.32:0/1492114845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cec0cf370 0x7f2cec0cf7e0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f2cec15a950 tx=0x7f2cd401c520 comp rx=0 tx=0).stop 2024-04-17T10:01:35.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.269+0000 7f2cf0a00640 1 -- 172.21.15.32:0/1492114845 shutdown_connections 2024-04-17T10:01:35.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.269+0000 7f2cf0a00640 1 --2- 172.21.15.32:0/1492114845 >> v2:172.21.15.144:3300/0 conn(0x7f2cec071e60 0x7f2cec072270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:35.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.269+0000 7f2cf0a00640 1 --2- 172.21.15.32:0/1492114845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cec0cf370 0x7f2cec0cf7e0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:35.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.269+0000 7f2cf0a00640 1 -- 172.21.15.32:0/1492114845 >> 172.21.15.32:0/1492114845 conn(0x7f2cec0b2120 msgr2=0x7f2cec0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:35.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.269+0000 7f2cf0a00640 1 -- 172.21.15.32:0/1492114845 shutdown_connections 2024-04-17T10:01:35.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.269+0000 7f2cf0a00640 1 -- 172.21.15.32:0/1492114845 wait complete. 2024-04-17T10:01:35.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2cf0a00640 1 Processor -- start 2024-04-17T10:01:35.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2cf0a00640 1 -- start start 2024-04-17T10:01:35.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2cf0a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cec06ba30 0x7f2cec159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:35.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2cf0a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec071e60 0x7f2cec15b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:35.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2cf0a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cec0cf370 0x7f2cec15bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:35.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2cf0a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2cec146230 con 0x7f2cec0cf370 2024-04-17T10:01:35.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2cf0a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2cec15dd80 con 0x7f2cec071e60 2024-04-17T10:01:35.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2cf0a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2cec1463b0 con 0x7f2cec06ba30 2024-04-17T10:01:35.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2cea800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cec0cf370 0x7f2cec15bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2cea800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cec0cf370 0x7f2cec15bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53940/0 (socket says 172.21.15.32:53940) 2024-04-17T10:01:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2cea800640 1 -- 172.21.15.32:0/875677621 learned_addr learned my addr 172.21.15.32:0/875677621 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2ce9e00640 1 --2- 172.21.15.32:0/875677621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cec06ba30 0x7f2cec159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2ce3e00640 1 --2- 172.21.15.32:0/875677621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec071e60 0x7f2cec15b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2cea800640 1 -- 172.21.15.32:0/875677621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cec06ba30 msgr2=0x7f2cec159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2cea800640 1 --2- 172.21.15.32:0/875677621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cec06ba30 0x7f2cec159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2cea800640 1 -- 172.21.15.32:0/875677621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec071e60 msgr2=0x7f2cec15b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2cea800640 1 --2- 172.21.15.32:0/875677621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec071e60 0x7f2cec15b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2cea800640 1 -- 172.21.15.32:0/875677621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cec0b4810 con 0x7f2cec0cf370 2024-04-17T10:01:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2cea800640 1 --2- 172.21.15.32:0/875677621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cec0cf370 0x7f2cec15bed0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400efc0 tx=0x7f2ce400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2ce2000640 1 -- 172.21.15.32:0/875677621 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4018020 con 0x7f2cec0cf370 2024-04-17T10:01:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2ce2000640 1 -- 172.21.15.32:0/875677621 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ce4014480 con 0x7f2cec0cf370 2024-04-17T10:01:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2ce2000640 1 -- 172.21.15.32:0/875677621 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4014870 con 0x7f2cec0cf370 2024-04-17T10:01:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2cf0a00640 1 -- 172.21.15.32:0/875677621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cec0b4af0 con 0x7f2cec0cf370 2024-04-17T10:01:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.273+0000 7f2cf0a00640 1 -- 172.21.15.32:0/875677621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cec159e40 con 0x7f2cec0cf370 2024-04-17T10:01:35.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.277+0000 7f2cf0a00640 1 -- 172.21.15.32:0/875677621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cec068a90 con 0x7f2cec0cf370 2024-04-17T10:01:35.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.277+0000 7f2ce2000640 1 -- 172.21.15.32:0/875677621 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2ce401b020 con 0x7f2cec0cf370 2024-04-17T10:01:35.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.277+0000 7f2ce2000640 1 --2- 172.21.15.32:0/875677621 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2cc8068380 0x7f2cc806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:35.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.277+0000 7f2ce2000640 1 -- 172.21.15.32:0/875677621 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5115+0+0 (secure 0 0 0) 0x7f2ce4087150 con 0x7f2cec0cf370 2024-04-17T10:01:35.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.277+0000 7f2ce9e00640 1 --2- 172.21.15.32:0/875677621 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2cc8068380 0x7f2cc806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:35.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.277+0000 7f2ce9e00640 1 --2- 172.21.15.32:0/875677621 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2cc8068380 0x7f2cc806a840 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc001810 tx=0x7f2cdc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:35.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.281+0000 7f2ce2000640 1 -- 172.21.15.32:0/875677621 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2ce4022070 con 0x7f2cec0cf370 2024-04-17T10:01:35.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.449+0000 7f2cf0a00640 1 -- 172.21.15.32:0/875677621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2cec06c850 con 0x7f2cec0cf370 2024-04-17T10:01:35.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.453+0000 7f2ce2000640 1 -- 172.21.15.32:0/875677621 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f2ce4010040 con 0x7f2cec0cf370 2024-04-17T10:01:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.453+0000 7f2cf0a00640 1 -- 172.21.15.32:0/875677621 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2cc8068380 msgr2=0x7f2cc806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:35.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.453+0000 7f2cf0a00640 1 --2- 172.21.15.32:0/875677621 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2cc8068380 0x7f2cc806a840 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc001810 tx=0x7f2cdc009040 comp rx=0 tx=0).stop 2024-04-17T10:01:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.453+0000 7f2cf0a00640 1 -- 172.21.15.32:0/875677621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cec0cf370 msgr2=0x7f2cec15bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.453+0000 7f2cf0a00640 1 --2- 172.21.15.32:0/875677621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cec0cf370 0x7f2cec15bed0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400efc0 tx=0x7f2ce400c490 comp rx=0 tx=0).stop 2024-04-17T10:01:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.453+0000 7f2cf0a00640 1 -- 172.21.15.32:0/875677621 shutdown_connections 2024-04-17T10:01:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.453+0000 7f2cf0a00640 1 --2- 172.21.15.32:0/875677621 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2cc8068380 0x7f2cc806a840 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.453+0000 7f2cf0a00640 1 --2- 172.21.15.32:0/875677621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cec0cf370 0x7f2cec15bed0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.453+0000 7f2cf0a00640 1 --2- 172.21.15.32:0/875677621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec071e60 0x7f2cec15b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.453+0000 7f2cf0a00640 1 --2- 172.21.15.32:0/875677621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cec06ba30 0x7f2cec159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.453+0000 7f2cf0a00640 1 -- 172.21.15.32:0/875677621 >> 172.21.15.32:0/875677621 conn(0x7f2cec0b2120 msgr2=0x7f2cec13e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.453+0000 7f2cf0a00640 1 -- 172.21.15.32:0/875677621 shutdown_connections 2024-04-17T10:01:35.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.453+0000 7f2cf0a00640 1 -- 172.21.15.32:0/875677621 wait complete. 2024-04-17T10:01:35.472 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T10:01:35.472 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:01:35.571 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:01:35.671 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T10:01:35.671 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:01:35.671 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:01:35.680 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:01:35.680 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T10:01:35.681 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:01:35.690 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:01:35.690 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:01:35.690 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:01:35.736 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:01:35.737 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T10:01:35.737 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:01:35.746 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:01:35.746 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-17T10:01:35.746 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:01:35.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.921+0000 7f28baa00640 1 Processor -- start 2024-04-17T10:01:35.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.921+0000 7f28baa00640 1 -- start start 2024-04-17T10:01:35.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.925+0000 7f28baa00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28b4144e90 0x7f28b40b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:35.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.925+0000 7f28baa00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f28b40b4de0 0x7f28b415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:35.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.925+0000 7f28baa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28b415bfe0 0x7f28b406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:35.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.925+0000 7f28baa00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f28b415c430 con 0x7f28b4144e90 2024-04-17T10:01:35.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.925+0000 7f28baa00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f28b4145270 con 0x7f28b40b4de0 2024-04-17T10:01:35.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.925+0000 7f28baa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f28b4072bb0 con 0x7f28b415bfe0 2024-04-17T10:01:35.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.925+0000 7f28b3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28b4144e90 0x7f28b40b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:35.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.925+0000 7f28b3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28b4144e90 0x7f28b40b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53956/0 (socket says 172.21.15.32:53956) 2024-04-17T10:01:35.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.925+0000 7f28b3e00640 1 -- 172.21.15.32:0/2185464183 learned_addr learned my addr 172.21.15.32:0/2185464183 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:35.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.925+0000 7f28abe00640 1 --2- 172.21.15.32:0/2185464183 >> v2:172.21.15.144:3300/0 conn(0x7f28b40b4de0 0x7f28b415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:35.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.925+0000 7f28b3e00640 1 -- 172.21.15.32:0/2185464183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28b415bfe0 msgr2=0x7f28b406ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:35.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.925+0000 7f28b8a00640 1 --2- 172.21.15.32:0/2185464183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28b415bfe0 0x7f28b406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:35.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.925+0000 7f28b3e00640 1 --2- 172.21.15.32:0/2185464183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28b415bfe0 0x7f28b406ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:35.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.925+0000 7f28b3e00640 1 -- 172.21.15.32:0/2185464183 >> v2:172.21.15.144:3300/0 conn(0x7f28b40b4de0 msgr2=0x7f28b415baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:35.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.925+0000 7f28b3e00640 1 --2- 172.21.15.32:0/2185464183 >> v2:172.21.15.144:3300/0 conn(0x7f28b40b4de0 0x7f28b415baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:35.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.925+0000 7f28b3e00640 1 -- 172.21.15.32:0/2185464183 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b40b3dc0 con 0x7f28b4144e90 2024-04-17T10:01:35.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.925+0000 7f28abe00640 1 --2- 172.21.15.32:0/2185464183 >> v2:172.21.15.144:3300/0 conn(0x7f28b40b4de0 0x7f28b415baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:35.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.925+0000 7f28b8a00640 1 --2- 172.21.15.32:0/2185464183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28b415bfe0 0x7f28b406ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:35.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.929+0000 7f28b3e00640 1 --2- 172.21.15.32:0/2185464183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28b4144e90 0x7f28b40b48a0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f28b406fb40 tx=0x7f28a401ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6137b0bf99c739b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:35.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.929+0000 7f28b3400640 1 -- 172.21.15.32:0/2185464183 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a40b4070 con 0x7f28b4144e90 2024-04-17T10:01:35.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.929+0000 7f28b3400640 1 -- 172.21.15.32:0/2185464183 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28a40a7b60 con 0x7f28b4144e90 2024-04-17T10:01:35.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.929+0000 7f28b3400640 1 -- 172.21.15.32:0/2185464183 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a40af440 con 0x7f28b4144e90 2024-04-17T10:01:35.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.929+0000 7f28baa00640 1 -- 172.21.15.32:0/2185464183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28b4144e90 msgr2=0x7f28b40b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:35.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.929+0000 7f28baa00640 1 --2- 172.21.15.32:0/2185464183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28b4144e90 0x7f28b40b48a0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f28b406fb40 tx=0x7f28a401ebc0 comp rx=0 tx=0).stop 2024-04-17T10:01:35.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.929+0000 7f28baa00640 1 -- 172.21.15.32:0/2185464183 shutdown_connections 2024-04-17T10:01:35.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.929+0000 7f28baa00640 1 --2- 172.21.15.32:0/2185464183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28b415bfe0 0x7f28b406ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:35.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.929+0000 7f28baa00640 1 --2- 172.21.15.32:0/2185464183 >> v2:172.21.15.144:3300/0 conn(0x7f28b40b4de0 0x7f28b415baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:35.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.929+0000 7f28baa00640 1 --2- 172.21.15.32:0/2185464183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28b4144e90 0x7f28b40b48a0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:35.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.929+0000 7f28baa00640 1 -- 172.21.15.32:0/2185464183 >> 172.21.15.32:0/2185464183 conn(0x7f28b406ba30 msgr2=0x7f28b406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:35.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.929+0000 7f28baa00640 1 -- 172.21.15.32:0/2185464183 shutdown_connections 2024-04-17T10:01:35.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.929+0000 7f28baa00640 1 -- 172.21.15.32:0/2185464183 wait complete. 2024-04-17T10:01:35.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.929+0000 7f28baa00640 1 Processor -- start 2024-04-17T10:01:35.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.929+0000 7f28baa00640 1 -- start start 2024-04-17T10:01:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28baa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28b40b4de0 0x7f28b40c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28baa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28b4144e90 0x7f28b415a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28baa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28b415bfe0 0x7f28b4141fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28baa00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f28b406de50 con 0x7f28b415bfe0 2024-04-17T10:01:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28baa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f28b4072880 con 0x7f28b4144e90 2024-04-17T10:01:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28baa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f28b406dfd0 con 0x7f28b40b4de0 2024-04-17T10:01:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28b8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28b415bfe0 0x7f28b4141fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28b8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28b415bfe0 0x7f28b4141fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53972/0 (socket says 172.21.15.32:53972) 2024-04-17T10:01:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28b8a00640 1 -- 172.21.15.32:0/1353364418 learned_addr learned my addr 172.21.15.32:0/1353364418 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28abe00640 1 --2- 172.21.15.32:0/1353364418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28b4144e90 0x7f28b415a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28b3e00640 1 --2- 172.21.15.32:0/1353364418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28b40b4de0 0x7f28b40c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28b8a00640 1 -- 172.21.15.32:0/1353364418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28b40b4de0 msgr2=0x7f28b40c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28b8a00640 1 --2- 172.21.15.32:0/1353364418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28b40b4de0 0x7f28b40c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28b8a00640 1 -- 172.21.15.32:0/1353364418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28b4144e90 msgr2=0x7f28b415a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28b8a00640 1 --2- 172.21.15.32:0/1353364418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28b4144e90 0x7f28b415a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28b8a00640 1 -- 172.21.15.32:0/1353364418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b413fa50 con 0x7f28b415bfe0 2024-04-17T10:01:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28b8a00640 1 --2- 172.21.15.32:0/1353364418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28b415bfe0 0x7f28b4141fe0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00cd50 tx=0x7f28ac00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28b0c00640 1 -- 172.21.15.32:0/1353364418 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac00f680 con 0x7f28b415bfe0 2024-04-17T10:01:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28b0c00640 1 -- 172.21.15.32:0/1353364418 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28ac00f840 con 0x7f28b415bfe0 2024-04-17T10:01:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28b0c00640 1 -- 172.21.15.32:0/1353364418 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac00fc30 con 0x7f28b415bfe0 2024-04-17T10:01:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28baa00640 1 -- 172.21.15.32:0/1353364418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b413df40 con 0x7f28b415bfe0 2024-04-17T10:01:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28baa00640 1 -- 172.21.15.32:0/1353364418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b413cf10 con 0x7f28b415bfe0 2024-04-17T10:01:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28baa00640 1 -- 172.21.15.32:0/1353364418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b4068a90 con 0x7f28b415bfe0 2024-04-17T10:01:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.933+0000 7f28b0c00640 1 -- 172.21.15.32:0/1353364418 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f28ac0213f0 con 0x7f28b415bfe0 2024-04-17T10:01:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.937+0000 7f28b0c00640 1 --2- 172.21.15.32:0/1353364418 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f28900683b0 0x7f289006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.937+0000 7f28b0c00640 1 -- 172.21.15.32:0/1353364418 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5115+0+0 (secure 0 0 0) 0x7f28ac0871f0 con 0x7f28b415bfe0 2024-04-17T10:01:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.937+0000 7f28b3e00640 1 --2- 172.21.15.32:0/1353364418 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f28900683b0 0x7f289006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.937+0000 7f28b3e00640 1 --2- 172.21.15.32:0/1353364418 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f28900683b0 0x7f289006a870 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f28a40b1000 tx=0x7f28a4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:35.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:35.941+0000 7f28b0c00640 1 -- 172.21.15.32:0/1353364418 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f28ac014030 con 0x7f28b415bfe0 2024-04-17T10:01:36.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.141+0000 7f28baa00640 1 -- 172.21.15.32:0/1353364418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28b40cd8c0 con 0x7f28b415bfe0 2024-04-17T10:01:36.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.141+0000 7f28b0c00640 1 -- 172.21.15.32:0/1353364418 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+3869 (secure 0 0 0) 0x7f28ac057440 con 0x7f28b415bfe0 2024-04-17T10:01:36.139 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:36.139 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":74,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6419,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3534108103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3534108103},{"type":"v1","addr":"172.21.15.144:6835","nonce":3534108103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7003,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/935945019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":935945019},{"type":"v1","addr":"172.21.15.144:6834","nonce":935945019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7554,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3044362393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3044362393},{"type":"v1","addr":"172.21.15.32:6836","nonce":3044362393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/550219120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":550219120},{"type":"v1","addr":"172.21.15.32:6837","nonce":550219120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74}],"filesystems":[]} 2024-04-17T10:01:36.139 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 74 2024-04-17T10:01:36.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.141+0000 7f28baa00640 1 -- 172.21.15.32:0/1353364418 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f28900683b0 msgr2=0x7f289006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:36.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.141+0000 7f28baa00640 1 --2- 172.21.15.32:0/1353364418 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f28900683b0 0x7f289006a870 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f28a40b1000 tx=0x7f28a4008040 comp rx=0 tx=0).stop 2024-04-17T10:01:36.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.141+0000 7f28baa00640 1 -- 172.21.15.32:0/1353364418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28b415bfe0 msgr2=0x7f28b4141fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:36.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.141+0000 7f28baa00640 1 --2- 172.21.15.32:0/1353364418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28b415bfe0 0x7f28b4141fe0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00cd50 tx=0x7f28ac00a650 comp rx=0 tx=0).stop 2024-04-17T10:01:36.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.141+0000 7f28baa00640 1 -- 172.21.15.32:0/1353364418 shutdown_connections 2024-04-17T10:01:36.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.141+0000 7f28baa00640 1 --2- 172.21.15.32:0/1353364418 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f28900683b0 0x7f289006a870 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:36.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.141+0000 7f28baa00640 1 --2- 172.21.15.32:0/1353364418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28b415bfe0 0x7f28b4141fe0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:36.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.141+0000 7f28baa00640 1 --2- 172.21.15.32:0/1353364418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28b4144e90 0x7f28b415a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:36.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.141+0000 7f28baa00640 1 --2- 172.21.15.32:0/1353364418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28b40b4de0 0x7f28b40c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:36.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.141+0000 7f28baa00640 1 -- 172.21.15.32:0/1353364418 >> 172.21.15.32:0/1353364418 conn(0x7f28b406ba30 msgr2=0x7f28b40b51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:36.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.141+0000 7f28baa00640 1 -- 172.21.15.32:0/1353364418 shutdown_connections 2024-04-17T10:01:36.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.141+0000 7f28baa00640 1 -- 172.21.15.32:0/1353364418 wait complete. 2024-04-17T10:01:36.160 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-17T10:01:36.160 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-17T10:01:36.161 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-17T10:01:36.161 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:01:36.161 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-17T10:01:36.161 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-17T10:01:36.161 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-17T10:01:36.161 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:01:36.161 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-17T10:01:36.161 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-17T10:01:36.162 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-17T10:01:36.162 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:01:36.162 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-17T10:01:36.162 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-17T10:01:36.162 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-17T10:01:36.162 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:01:36.172 INFO:tasks.ceph.mds.a:Stopped 2024-04-17T10:01:36.172 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-17T10:01:36.173 INFO:tasks.ceph.mds.c:Stopped 2024-04-17T10:01:36.173 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-17T10:01:36.175 INFO:tasks.ceph.mds.d:Stopped 2024-04-17T10:01:36.175 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-17T10:01:36.176 INFO:tasks.ceph.mds.b:Stopped 2024-04-17T10:01:36.176 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-17T10:01:36.218 INFO:tasks.ceph.mds.d:Started 2024-04-17T10:01:36.218 DEBUG:teuthology.parallel:result is None 2024-04-17T10:01:36.220 INFO:tasks.ceph.mds.b:Started 2024-04-17T10:01:36.220 DEBUG:teuthology.parallel:result is None 2024-04-17T10:01:36.223 INFO:tasks.ceph.mds.a:Started 2024-04-17T10:01:36.223 DEBUG:teuthology.parallel:result is None 2024-04-17T10:01:36.224 INFO:tasks.ceph.mds.c:Started 2024-04-17T10:01:36.224 DEBUG:teuthology.parallel:result is None 2024-04-17T10:01:36.225 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-17T10:01:36.309 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2024-04-17T10:01:36.310 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2024-04-17T10:01:36.311 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-17T10:01:36.313 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-17T10:01:36.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.373+0000 7ff59aa00640 1 Processor -- start 2024-04-17T10:01:36.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.373+0000 7ff59aa00640 1 -- start start 2024-04-17T10:01:36.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.373+0000 7ff59aa00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff59413f6e0 0x7ff5940cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:36.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.373+0000 7ff59aa00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5941601e0 con 0x7ff5940d09f0 2024-04-17T10:01:36.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.373+0000 7ff59aa00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff594160480 con 0x7ff5940cf370 2024-04-17T10:01:36.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.373+0000 7ff59aa00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff59413fac0 con 0x7ff59413f6e0 2024-04-17T10:01:36.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.373+0000 7ff598a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff5940d09f0 0x7ff594071d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36100/0 (socket says 172.21.15.32:36100) 2024-04-17T10:01:36.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.377+0000 7ff598a00640 1 -- 172.21.15.32:0/3382714366 learned_addr learned my addr 172.21.15.32:0/3382714366 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:36.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.377+0000 7ff593e00640 1 --2- 172.21.15.32:0/3382714366 >> v2:172.21.15.144:3300/0 conn(0x7ff59413f6e0 0x7ff5940cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:36.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.377+0000 7ff593400640 1 -- 172.21.15.32:0/3382714366 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 428457866 0 0) 0x7ff5941601e0 con 0x7ff5940d09f0 2024-04-17T10:01:36.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.377+0000 7ff593400640 1 -- 172.21.15.32:0/3382714366 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff578003530 con 0x7ff5940d09f0 2024-04-17T10:01:36.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.377+0000 7ff593e00640 1 -- 172.21.15.32:0/3382714366 >> v1:172.21.15.144:6789/0 conn(0x7ff5940cf370 legacy=0x7ff5940d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:36.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.377+0000 7ff593400640 1 -- 172.21.15.32:0/3382714366 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2510794545 0 0) 0x7ff594160480 con 0x7ff5940cf370 2024-04-17T10:01:36.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.377+0000 7ff593e00640 1 -- 172.21.15.32:0/3382714366 >> v1:172.21.15.32:6789/0 conn(0x7ff5940d09f0 legacy=0x7ff594071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:36.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.377+0000 7ff593e00640 1 -- 172.21.15.32:0/3382714366 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff594160760 con 0x7ff59413f6e0 2024-04-17T10:01:36.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.377+0000 7ff593e00640 1 --2- 172.21.15.32:0/3382714366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff59413f6e0 0x7ff5940cf850 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff5840097c0 tx=0x7ff58401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e802cb2afec97b8d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:36.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.381+0000 7ff593400640 1 -- 172.21.15.32:0/3382714366 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5840a7070 con 0x7ff59413f6e0 2024-04-17T10:01:36.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.381+0000 7ff593400640 1 -- 172.21.15.32:0/3382714366 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5840b0070 con 0x7ff59413f6e0 2024-04-17T10:01:36.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.381+0000 7ff593400640 1 -- 172.21.15.32:0/3382714366 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5840ab890 con 0x7ff59413f6e0 2024-04-17T10:01:36.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.381+0000 7ff59aa00640 1 -- 172.21.15.32:0/3382714366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff59413f6e0 msgr2=0x7ff5940cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:36.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.381+0000 7ff59aa00640 1 --2- 172.21.15.32:0/3382714366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff59413f6e0 0x7ff5940cf850 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff5840097c0 tx=0x7ff58401c470 comp rx=0 tx=0).stop 2024-04-17T10:01:36.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.381+0000 7ff59aa00640 1 -- 172.21.15.32:0/3382714366 shutdown_connections 2024-04-17T10:01:36.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.381+0000 7ff59aa00640 1 --2- 172.21.15.32:0/3382714366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff59413f6e0 0x7ff5940cf850 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:36.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.381+0000 7ff59aa00640 1 -- 172.21.15.32:0/3382714366 >> 172.21.15.32:0/3382714366 conn(0x7ff59406ba30 msgr2=0x7ff59406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:36.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.381+0000 7ff59aa00640 1 -- 172.21.15.32:0/3382714366 shutdown_connections 2024-04-17T10:01:36.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.381+0000 7ff59aa00640 1 -- 172.21.15.32:0/3382714366 wait complete. 2024-04-17T10:01:36.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.381+0000 7ff59aa00640 1 Processor -- start 2024-04-17T10:01:36.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.381+0000 7ff59aa00640 1 -- start start 2024-04-17T10:01:36.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.381+0000 7ff59aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5940cf370 0x7ff594140280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:36.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.381+0000 7ff59aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5940d09f0 0x7ff594155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:36.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.381+0000 7ff59aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff59413f6e0 0x7ff594155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:36.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.381+0000 7ff59aa00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff5940cff10 con 0x7ff5940d09f0 2024-04-17T10:01:36.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.381+0000 7ff59aa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff5940cfd90 con 0x7ff59413f6e0 2024-04-17T10:01:36.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.381+0000 7ff59aa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff59416a590 con 0x7ff5940cf370 2024-04-17T10:01:36.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.381+0000 7ff58b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5940d09f0 0x7ff594155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:36.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.381+0000 7ff593e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5940cf370 0x7ff594140280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:36.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.381+0000 7ff58b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5940d09f0 0x7ff594155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54020/0 (socket says 172.21.15.32:54020) 2024-04-17T10:01:36.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.381+0000 7ff58b400640 1 -- 172.21.15.32:0/2369831975 learned_addr learned my addr 172.21.15.32:0/2369831975 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:36.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.381+0000 7ff598a00640 1 --2- 172.21.15.32:0/2369831975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff59413f6e0 0x7ff594155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:36.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.385+0000 7ff593e00640 1 -- 172.21.15.32:0/2369831975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff59413f6e0 msgr2=0x7ff594155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:36.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.385+0000 7ff593e00640 1 --2- 172.21.15.32:0/2369831975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff59413f6e0 0x7ff594155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:36.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.385+0000 7ff593e00640 1 -- 172.21.15.32:0/2369831975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5940d09f0 msgr2=0x7ff594155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:36.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.385+0000 7ff593e00640 1 --2- 172.21.15.32:0/2369831975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5940d09f0 0x7ff594155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:36.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.385+0000 7ff593e00640 1 -- 172.21.15.32:0/2369831975 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff594135950 con 0x7ff5940cf370 2024-04-17T10:01:36.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.385+0000 7ff593e00640 1 --2- 172.21.15.32:0/2369831975 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5940cf370 0x7ff594140280 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff5840ac000 tx=0x7ff58401acb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:36.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.385+0000 7ff590c00640 1 -- 172.21.15.32:0/2369831975 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584004220 con 0x7ff5940cf370 2024-04-17T10:01:36.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.385+0000 7ff590c00640 1 -- 172.21.15.32:0/2369831975 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5840043c0 con 0x7ff5940cf370 2024-04-17T10:01:36.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.385+0000 7ff59aa00640 1 -- 172.21.15.32:0/2369831975 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff594135bd0 con 0x7ff5940cf370 2024-04-17T10:01:36.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.385+0000 7ff590c00640 1 -- 172.21.15.32:0/2369831975 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584003840 con 0x7ff5940cf370 2024-04-17T10:01:36.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.385+0000 7ff59aa00640 1 -- 172.21.15.32:0/2369831975 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff594136190 con 0x7ff5940cf370 2024-04-17T10:01:36.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.385+0000 7ff59aa00640 1 -- 172.21.15.32:0/2369831975 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff594068a90 con 0x7ff5940cf370 2024-04-17T10:01:36.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.385+0000 7ff590c00640 1 -- 172.21.15.32:0/2369831975 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff5840bd030 con 0x7ff5940cf370 2024-04-17T10:01:36.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.385+0000 7ff590c00640 1 --2- 172.21.15.32:0/2369831975 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff570068380 0x7ff57006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:36.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.385+0000 7ff590c00640 1 -- 172.21.15.32:0/2369831975 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 5115+0+0 (secure 0 0 0) 0x7ff584120b70 con 0x7ff5940cf370 2024-04-17T10:01:36.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.385+0000 7ff58b400640 1 --2- 172.21.15.32:0/2369831975 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff570068380 0x7ff57006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:36.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.389+0000 7ff58b400640 1 --2- 172.21.15.32:0/2369831975 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff570068380 0x7ff57006a840 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff594140ea0 tx=0x7ff57c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.393+0000 7ff590c00640 1 -- 172.21.15.32:0/2369831975 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff5840c4030 con 0x7ff5940cf370 2024-04-17T10:01:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:36.569+0000 7ff59aa00640 1 -- 172.21.15.32:0/2369831975 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff5940d1030 con 0x7ff5940cf370 2024-04-17T10:01:37.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.117+0000 7ff590c00640 1 -- 172.21.15.32:0/2369831975 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v91) ==== 97+0+0 (secure 0 0 0) 0x7ff5840f7030 con 0x7ff5940cf370 2024-04-17T10:01:37.114 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2024-04-17T10:01:37.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.117+0000 7ff59aa00640 1 -- 172.21.15.32:0/2369831975 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff570068380 msgr2=0x7ff57006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:37.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.117+0000 7ff59aa00640 1 --2- 172.21.15.32:0/2369831975 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff570068380 0x7ff57006a840 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff594140ea0 tx=0x7ff57c009040 comp rx=0 tx=0).stop 2024-04-17T10:01:37.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.117+0000 7ff59aa00640 1 -- 172.21.15.32:0/2369831975 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5940cf370 msgr2=0x7ff594140280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:37.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.117+0000 7ff59aa00640 1 --2- 172.21.15.32:0/2369831975 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5940cf370 0x7ff594140280 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff5840ac000 tx=0x7ff58401acb0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.117+0000 7ff59aa00640 1 -- 172.21.15.32:0/2369831975 shutdown_connections 2024-04-17T10:01:37.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.117+0000 7ff59aa00640 1 --2- 172.21.15.32:0/2369831975 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff570068380 0x7ff57006a840 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.117+0000 7ff59aa00640 1 --2- 172.21.15.32:0/2369831975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff59413f6e0 0x7ff594155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.117+0000 7ff59aa00640 1 --2- 172.21.15.32:0/2369831975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5940d09f0 0x7ff594155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.117+0000 7ff59aa00640 1 --2- 172.21.15.32:0/2369831975 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5940cf370 0x7ff594140280 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.121+0000 7ff59aa00640 1 -- 172.21.15.32:0/2369831975 >> 172.21.15.32:0/2369831975 conn(0x7ff59406ba30 msgr2=0x7ff594071d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:37.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.121+0000 7ff59aa00640 1 -- 172.21.15.32:0/2369831975 shutdown_connections 2024-04-17T10:01:37.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.121+0000 7ff59aa00640 1 -- 172.21.15.32:0/2369831975 wait complete. 2024-04-17T10:01:37.141 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-17T10:01:37.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.289+0000 7f4086200640 1 Processor -- start 2024-04-17T10:01:37.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.289+0000 7f4086200640 1 -- start start 2024-04-17T10:01:37.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.289+0000 7f4086200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f408015eb30 0x7f40800d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:37.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.289+0000 7f4086200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f40800b4af0 0x7f408009a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:37.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.289+0000 7f4086200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40800ccbd0 con 0x7f408009af30 2024-04-17T10:01:37.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.289+0000 7f4086200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f40800ce9a0 con 0x7f408015eb30 2024-04-17T10:01:37.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.289+0000 7f4086200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f40800cceb0 con 0x7f40800b4af0 2024-04-17T10:01:37.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.289+0000 7f407f400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f408015eb30 0x7f40800d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:37.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.289+0000 7f407f400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f408015eb30 0x7f40800d06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54028/0 (socket says 172.21.15.32:54028) 2024-04-17T10:01:37.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.289+0000 7f407f400640 1 -- 172.21.15.32:0/3583148537 learned_addr learned my addr 172.21.15.32:0/3583148537 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:37.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.289+0000 7f4077e00640 1 --2- 172.21.15.32:0/3583148537 >> v2:172.21.15.144:3300/0 conn(0x7f40800b4af0 0x7f408009a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:37.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.289+0000 7f407f400640 1 -- 172.21.15.32:0/3583148537 >> v2:172.21.15.144:3300/0 conn(0x7f40800b4af0 msgr2=0x7f408009a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:37.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.289+0000 7f407f400640 1 --2- 172.21.15.32:0/3583148537 >> v2:172.21.15.144:3300/0 conn(0x7f40800b4af0 0x7f408009a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.289+0000 7f407f400640 1 -- 172.21.15.32:0/3583148537 >> v1:172.21.15.144:6789/0 conn(0x7f408009af30 legacy=0x7f408006cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:37.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.289+0000 7f407f400640 1 -- 172.21.15.32:0/3583148537 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40800cd190 con 0x7f408015eb30 2024-04-17T10:01:37.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.289+0000 7f4077e00640 1 --2- 172.21.15.32:0/3583148537 >> v2:172.21.15.144:3300/0 conn(0x7f40800b4af0 0x7f408009a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:37.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.293+0000 7f407f400640 1 --2- 172.21.15.32:0/3583148537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f408015eb30 0x7f40800d06b0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f40800733a0 tx=0x7f407001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9072bb42782f29 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:37.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.293+0000 7f407ea00640 1 -- 172.21.15.32:0/3583148537 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40700b5070 con 0x7f408015eb30 2024-04-17T10:01:37.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.293+0000 7f407ea00640 1 -- 172.21.15.32:0/3583148537 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40700a8a10 con 0x7f408015eb30 2024-04-17T10:01:37.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.293+0000 7f407ea00640 1 -- 172.21.15.32:0/3583148537 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40700a8de0 con 0x7f408015eb30 2024-04-17T10:01:37.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.293+0000 7f4086200640 1 -- 172.21.15.32:0/3583148537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f408015eb30 msgr2=0x7f40800d06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:37.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.293+0000 7f4086200640 1 --2- 172.21.15.32:0/3583148537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f408015eb30 0x7f40800d06b0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f40800733a0 tx=0x7f407001ed10 comp rx=0 tx=0).stop 2024-04-17T10:01:37.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.293+0000 7f4086200640 1 -- 172.21.15.32:0/3583148537 shutdown_connections 2024-04-17T10:01:37.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.293+0000 7f4086200640 1 --2- 172.21.15.32:0/3583148537 >> v2:172.21.15.144:3300/0 conn(0x7f40800b4af0 0x7f408009a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.293+0000 7f4086200640 1 --2- 172.21.15.32:0/3583148537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f408015eb30 0x7f40800d06b0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.293+0000 7f4086200640 1 -- 172.21.15.32:0/3583148537 >> 172.21.15.32:0/3583148537 conn(0x7f40800b3fd0 msgr2=0x7f40800b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:37.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.293+0000 7f4086200640 1 -- 172.21.15.32:0/3583148537 shutdown_connections 2024-04-17T10:01:37.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.293+0000 7f4086200640 1 -- 172.21.15.32:0/3583148537 wait complete. 2024-04-17T10:01:37.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.293+0000 7f4086200640 1 Processor -- start 2024-04-17T10:01:37.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.293+0000 7f4086200640 1 -- start start 2024-04-17T10:01:37.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.297+0000 7f4086200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f408009af30 0x7f4080160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:37.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.297+0000 7f4086200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40800b4af0 0x7f408009f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:37.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.297+0000 7f4086200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f408015eb30 0x7f408015cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:37.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.297+0000 7f4086200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f40800d1680 con 0x7f408009af30 2024-04-17T10:01:37.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.297+0000 7f4086200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f408006de50 con 0x7f40800b4af0 2024-04-17T10:01:37.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.297+0000 7f4086200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f40800af170 con 0x7f408015eb30 2024-04-17T10:01:37.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.297+0000 7f407f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f408009af30 0x7f4080160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:37.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.297+0000 7f407f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f408009af30 0x7f4080160280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54044/0 (socket says 172.21.15.32:54044) 2024-04-17T10:01:37.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.297+0000 7f407f400640 1 -- 172.21.15.32:0/2612400370 learned_addr learned my addr 172.21.15.32:0/2612400370 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:37.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.297+0000 7f4077e00640 1 --2- 172.21.15.32:0/2612400370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40800b4af0 0x7f408009f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:37.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.297+0000 7f407fe00640 1 --2- 172.21.15.32:0/2612400370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f408015eb30 0x7f408015cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:37.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.297+0000 7f407f400640 1 -- 172.21.15.32:0/2612400370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f408015eb30 msgr2=0x7f408015cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:37.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.297+0000 7f407f400640 1 --2- 172.21.15.32:0/2612400370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f408015eb30 0x7f408015cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.297+0000 7f407f400640 1 -- 172.21.15.32:0/2612400370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40800b4af0 msgr2=0x7f408009f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:37.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.297+0000 7f407f400640 1 --2- 172.21.15.32:0/2612400370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40800b4af0 0x7f408009f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.297+0000 7f407f400640 1 -- 172.21.15.32:0/2612400370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408015f2f0 con 0x7f408009af30 2024-04-17T10:01:37.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.297+0000 7f407f400640 1 --2- 172.21.15.32:0/2612400370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f408009af30 0x7f4080160280 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f407001b920 tx=0x7f407001bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:37.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.297+0000 7f4077400640 1 -- 172.21.15.32:0/2612400370 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40700b5070 con 0x7f408009af30 2024-04-17T10:01:37.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.297+0000 7f4077400640 1 -- 172.21.15.32:0/2612400370 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40700b0b30 con 0x7f408009af30 2024-04-17T10:01:37.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.297+0000 7f4077400640 1 -- 172.21.15.32:0/2612400370 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40700bb410 con 0x7f408009af30 2024-04-17T10:01:37.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.297+0000 7f4086200640 1 -- 172.21.15.32:0/2612400370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408015f5d0 con 0x7f408009af30 2024-04-17T10:01:37.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.297+0000 7f4086200640 1 -- 172.21.15.32:0/2612400370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f408015fb90 con 0x7f408009af30 2024-04-17T10:01:37.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.297+0000 7f4086200640 1 -- 172.21.15.32:0/2612400370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4080068a90 con 0x7f408009af30 2024-04-17T10:01:37.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.301+0000 7f4077400640 1 -- 172.21.15.32:0/2612400370 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f40700c2060 con 0x7f408009af30 2024-04-17T10:01:37.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.301+0000 7f4077400640 1 --2- 172.21.15.32:0/2612400370 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f405c0683b0 0x7f405c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:37.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.301+0000 7f4077400640 1 -- 172.21.15.32:0/2612400370 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4857+0+0 (secure 0 0 0) 0x7f4070123680 con 0x7f408009af30 2024-04-17T10:01:37.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.301+0000 7f4077e00640 1 --2- 172.21.15.32:0/2612400370 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f405c0683b0 0x7f405c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:37.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.301+0000 7f4077e00640 1 --2- 172.21.15.32:0/2612400370 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f405c0683b0 0x7f405c06a870 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f40800afc70 tx=0x7f4068008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:37.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.305+0000 7f4077400640 1 -- 172.21.15.32:0/2612400370 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f4070018870 con 0x7f408009af30 2024-04-17T10:01:37.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.529+0000 7f4086200640 1 -- 172.21.15.32:0/2612400370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f408006e430 con 0x7f408009af30 2024-04-17T10:01:37.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.529+0000 7f4077400640 1 -- 172.21.15.32:0/2612400370 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7f40700f3f80 con 0x7f408009af30 2024-04-17T10:01:37.528 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:37.528 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:01:37.528 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2024-04-17T10:01:37.528 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:01:37.528 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2024-04-17T10:01:37.528 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mzM1KExAAn0WH+Gf6gGV/dJonXzAgPQ==", 2024-04-17T10:01:37.528 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:01:37.528 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T10:01:37.528 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T10:01:37.529 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T10:01:37.529 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:01:37.529 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:01:37.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:01:37.529 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:01:37.529 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2024-04-17T10:01:37.529 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mQnJ4HxAAT0rFHhdAj7PMxZGiqEyWbg==", 2024-04-17T10:01:37.529 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:01:37.529 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T10:01:37.529 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T10:01:37.529 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T10:01:37.529 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:01:37.530 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:01:37.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:01:37.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:01:37.530 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2024-04-17T10:01:37.530 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mWWi5GhAA3RAdRxK6GRwuVCH4DHoMeA==", 2024-04-17T10:01:37.530 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:01:37.530 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T10:01:37.530 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T10:01:37.530 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T10:01:37.530 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:01:37.530 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:01:37.530 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:01:37.530 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:01:37.530 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2024-04-17T10:01:37.531 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9m9TMAJxAAbmUp05GHRE1AqCmq83scVQ==", 2024-04-17T10:01:37.531 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:01:37.531 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T10:01:37.531 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T10:01:37.531 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T10:01:37.531 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:01:37.531 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:01:37.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:01:37.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:01:37.531 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2024-04-17T10:01:37.531 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDRmx9mnPwyDBAAxD10jhxptVH5zhupengElg==", 2024-04-17T10:01:37.531 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:01:37.531 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T10:01:37.531 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T10:01:37.532 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:01:37.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:01:37.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:01:37.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:01:37.532 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2024-04-17T10:01:37.532 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUmx9mh3iuGBAAqvWSRyNX3u/TjEH6TJpqdA==", 2024-04-17T10:01:37.532 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:01:37.532 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T10:01:37.532 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T10:01:37.532 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:01:37.532 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:01:37.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:01:37.532 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:01:37.533 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2024-04-17T10:01:37.533 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXmx9mbe1CIRAA1pmzAl04tGkTu6QLpmCkag==", 2024-04-17T10:01:37.533 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:01:37.533 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T10:01:37.533 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T10:01:37.533 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:01:37.533 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:01:37.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:01:37.533 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:01:37.533 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2024-04-17T10:01:37.533 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDamx9mIwXoKxAAWPTFaYdp4lIZhuOU7F0x8g==", 2024-04-17T10:01:37.533 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:01:37.533 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T10:01:37.533 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T10:01:37.533 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:01:37.534 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:01:37.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:01:37.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:01:37.534 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2024-04-17T10:01:37.534 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDgmx9mp0s/JxAAv+usQqwrxgKINRlkDhhydQ==", 2024-04-17T10:01:37.534 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:01:37.534 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T10:01:37.534 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T10:01:37.534 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:01:37.534 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:01:37.534 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:01:37.534 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:01:37.535 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2024-04-17T10:01:37.535 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDjmx9mMVStMhAA/n8jk+LVDosWhziXk/lLRQ==", 2024-04-17T10:01:37.535 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:01:37.535 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T10:01:37.535 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T10:01:37.535 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:01:37.535 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:01:37.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:01:37.535 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:01:37.535 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2024-04-17T10:01:37.535 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDnmx9m/YQlABAAOqfWzVtgonD7b+WlR95mnA==", 2024-04-17T10:01:37.535 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:01:37.535 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T10:01:37.535 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T10:01:37.536 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:01:37.536 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:01:37.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:01:37.536 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:01:37.536 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2024-04-17T10:01:37.536 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDqmx9m572mCBAAqBib/yeQOBno+0B1K1dLxA==", 2024-04-17T10:01:37.536 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:01:37.536 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T10:01:37.536 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T10:01:37.536 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:01:37.536 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:01:37.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:01:37.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:01:37.537 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2024-04-17T10:01:37.537 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mUXoLKxAANV7zhLvL0DqHngk4TtsdBQ==", 2024-04-17T10:01:37.537 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:01:37.537 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T10:01:37.537 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2024-04-17T10:01:37.537 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-17T10:01:37.537 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:01:37.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:01:37.537 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:01:37.537 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2024-04-17T10:01:37.538 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mZBeRNBAAQg5K950mp6+XND7NCABAKQ==", 2024-04-17T10:01:37.538 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:01:37.538 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T10:01:37.538 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-17T10:01:37.538 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-17T10:01:37.538 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-17T10:01:37.538 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:01:37.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:01:37.538 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:01:37.538 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2024-04-17T10:01:37.538 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mWKstMRAAp18Hbnk02w+hT0sYZ3tSLA==", 2024-04-17T10:01:37.538 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:01:37.538 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T10:01:37.538 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-17T10:01:37.539 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-17T10:01:37.539 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-17T10:01:37.539 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:01:37.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:01:37.539 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:01:37.539 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2024-04-17T10:01:37.540 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mRQ2wOhAAzlWXFtP0WaXWmneqxnZ5/w==", 2024-04-17T10:01:37.540 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:01:37.540 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T10:01:37.540 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-17T10:01:37.540 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-17T10:01:37.540 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-17T10:01:37.540 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:01:37.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:01:37.540 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:01:37.540 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2024-04-17T10:01:37.540 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDOmx9maDStMxAAMCa36HfycwV3WbLg6Voo9Q==", 2024-04-17T10:01:37.540 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:01:37.540 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-17T10:01:37.541 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T10:01:37.541 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T10:01:37.541 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:01:37.541 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:01:37.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:01:37.541 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:01:37.541 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2024-04-17T10:01:37.541 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9myy0oDxAAPD2BTjIP09hiU5L7i6gUMg==", 2024-04-17T10:01:37.541 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:01:37.541 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-17T10:01:37.542 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-17T10:01:37.542 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:01:37.542 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:01:37.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:01:37.542 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:01:37.542 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2024-04-17T10:01:37.542 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9m3FmACxAApri5AI5PeR0+/HLev8DKcw==", 2024-04-17T10:01:37.542 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:01:37.542 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-17T10:01:37.542 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-17T10:01:37.543 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:01:37.543 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:01:37.543 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:01:37.543 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-17T10:01:37.543 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:01:37.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.533+0000 7f4086200640 1 -- 172.21.15.32:0/2612400370 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f405c0683b0 msgr2=0x7f405c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:37.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.533+0000 7f4086200640 1 --2- 172.21.15.32:0/2612400370 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f405c0683b0 0x7f405c06a870 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f40800afc70 tx=0x7f4068008040 comp rx=0 tx=0).stop 2024-04-17T10:01:37.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.533+0000 7f4086200640 1 -- 172.21.15.32:0/2612400370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f408009af30 msgr2=0x7f4080160280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:37.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.533+0000 7f4086200640 1 --2- 172.21.15.32:0/2612400370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f408009af30 0x7f4080160280 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f407001b920 tx=0x7f407001bab0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.533+0000 7f4086200640 1 -- 172.21.15.32:0/2612400370 shutdown_connections 2024-04-17T10:01:37.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.533+0000 7f4086200640 1 --2- 172.21.15.32:0/2612400370 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f405c0683b0 0x7f405c06a870 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.533+0000 7f4086200640 1 --2- 172.21.15.32:0/2612400370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f408015eb30 0x7f408015cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.533+0000 7f4086200640 1 --2- 172.21.15.32:0/2612400370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40800b4af0 0x7f408009f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.533+0000 7f4086200640 1 --2- 172.21.15.32:0/2612400370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f408009af30 0x7f4080160280 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.533+0000 7f4086200640 1 -- 172.21.15.32:0/2612400370 >> 172.21.15.32:0/2612400370 conn(0x7f40800b3fd0 msgr2=0x7f40800ce120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:37.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.533+0000 7f4086200640 1 -- 172.21.15.32:0/2612400370 shutdown_connections 2024-04-17T10:01:37.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.533+0000 7f4086200640 1 -- 172.21.15.32:0/2612400370 wait complete. 2024-04-17T10:01:37.550 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:01:37.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.693+0000 7f80d9200640 1 Processor -- start 2024-04-17T10:01:37.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.693+0000 7f80d9200640 1 -- start start 2024-04-17T10:01:37.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.693+0000 7f80d9200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f80d415eba0 0x7f80d406cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:37.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.693+0000 7f80d9200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80d409b3c0 0x7f80d406d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:37.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.693+0000 7f80d9200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80d409bd60 con 0x7f80d406da90 2024-04-17T10:01:37.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.693+0000 7f80d9200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f80d406dec0 con 0x7f80d415eba0 2024-04-17T10:01:37.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.693+0000 7f80d9200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f80d40b1e40 con 0x7f80d409b3c0 2024-04-17T10:01:37.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.693+0000 7f80d1400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80d409b3c0 0x7f80d406d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:37.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.693+0000 7f80d1e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f80d415eba0 0x7f80d406cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:37.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.693+0000 7f80d2800640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f80d406da90 0x7f80d40b2ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:39026/0 (socket says 172.21.15.32:39026) 2024-04-17T10:01:37.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.693+0000 7f80d2800640 1 -- 172.21.15.32:0/822058397 learned_addr learned my addr 172.21.15.32:0/822058397 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:37.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.697+0000 7f80d1e00640 1 -- 172.21.15.32:0/822058397 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80d409b3c0 msgr2=0x7f80d406d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:37.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.697+0000 7f80d1e00640 1 --2- 172.21.15.32:0/822058397 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80d409b3c0 0x7f80d406d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.697+0000 7f80d1e00640 1 -- 172.21.15.32:0/822058397 >> v1:172.21.15.144:6789/0 conn(0x7f80d406da90 legacy=0x7f80d40b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:37.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.697+0000 7f80d1e00640 1 -- 172.21.15.32:0/822058397 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d40b2120 con 0x7f80d415eba0 2024-04-17T10:01:37.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.697+0000 7f80d0a00640 1 -- 172.21.15.32:0/822058397 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2137618186 0 0) 0x7f80d409bd60 con 0x7f80d406da90 2024-04-17T10:01:37.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.697+0000 7f80d1400640 1 --2- 172.21.15.32:0/822058397 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80d409b3c0 0x7f80d406d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:01:37.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.697+0000 7f80d1e00640 1 --2- 172.21.15.32:0/822058397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80d415eba0 0x7f80d406cd50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f80c40097c0 tx=0x7f80c401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ed5b4a7e538e3e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:37.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.697+0000 7f80d0a00640 1 -- 172.21.15.32:0/822058397 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c40b0070 con 0x7f80d415eba0 2024-04-17T10:01:37.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.697+0000 7f80d0a00640 1 -- 172.21.15.32:0/822058397 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c40ab600 con 0x7f80d415eba0 2024-04-17T10:01:37.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.697+0000 7f80d0a00640 1 -- 172.21.15.32:0/822058397 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c40ab9d0 con 0x7f80d415eba0 2024-04-17T10:01:37.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.697+0000 7f80d9200640 1 -- 172.21.15.32:0/822058397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80d415eba0 msgr2=0x7f80d406cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:37.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.697+0000 7f80d9200640 1 --2- 172.21.15.32:0/822058397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80d415eba0 0x7f80d406cd50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f80c40097c0 tx=0x7f80c401c410 comp rx=0 tx=0).stop 2024-04-17T10:01:37.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.697+0000 7f80d9200640 1 -- 172.21.15.32:0/822058397 shutdown_connections 2024-04-17T10:01:37.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.697+0000 7f80d9200640 1 --2- 172.21.15.32:0/822058397 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80d409b3c0 0x7f80d406d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.697+0000 7f80d9200640 1 --2- 172.21.15.32:0/822058397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80d415eba0 0x7f80d406cd50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.697+0000 7f80d9200640 1 -- 172.21.15.32:0/822058397 >> 172.21.15.32:0/822058397 conn(0x7f80d40b3fd0 msgr2=0x7f80d40b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:37.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d9200640 1 -- 172.21.15.32:0/822058397 shutdown_connections 2024-04-17T10:01:37.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d9200640 1 -- 172.21.15.32:0/822058397 wait complete. 2024-04-17T10:01:37.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d9200640 1 Processor -- start 2024-04-17T10:01:37.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d9200640 1 -- start start 2024-04-17T10:01:37.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d9200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80d406da90 0x7f80d4158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:37.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d9200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80d409b3c0 0x7f80d40c7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:37.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d9200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d415eba0 0x7f80d40cded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:37.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d9200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f80d40cccb0 con 0x7f80d415eba0 2024-04-17T10:01:37.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d9200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f80d40ccb30 con 0x7f80d409b3c0 2024-04-17T10:01:37.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d9200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f80d416b580 con 0x7f80d406da90 2024-04-17T10:01:37.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d415eba0 0x7f80d40cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:37.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d1e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80d406da90 0x7f80d4158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:37.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d1400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80d409b3c0 0x7f80d40c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:37.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d415eba0 0x7f80d40cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54054/0 (socket says 172.21.15.32:54054) 2024-04-17T10:01:37.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d1400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80d409b3c0 0x7f80d40c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:39606/0 (socket says 172.21.15.32:39606) 2024-04-17T10:01:37.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d1400640 1 -- 172.21.15.32:0/895730238 learned_addr learned my addr 172.21.15.32:0/895730238 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:37.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d1e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80d406da90 0x7f80d4158710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:45018/0 (socket says 172.21.15.32:45018) 2024-04-17T10:01:37.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d2800640 1 -- 172.21.15.32:0/895730238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80d406da90 msgr2=0x7f80d4158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:37.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d2800640 1 --2- 172.21.15.32:0/895730238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80d406da90 0x7f80d4158710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d2800640 1 -- 172.21.15.32:0/895730238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80d409b3c0 msgr2=0x7f80d40c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:37.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d2800640 1 --2- 172.21.15.32:0/895730238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80d409b3c0 0x7f80d40c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d2800640 1 -- 172.21.15.32:0/895730238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d40c95b0 con 0x7f80d415eba0 2024-04-17T10:01:37.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d1400640 1 --2- 172.21.15.32:0/895730238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80d409b3c0 0x7f80d40c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:01:37.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.701+0000 7f80d2800640 1 --2- 172.21.15.32:0/895730238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d415eba0 0x7f80d40cded0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00c780 tx=0x7f80cc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:37.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.705+0000 7f80ca000640 1 -- 172.21.15.32:0/895730238 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc018070 con 0x7f80d415eba0 2024-04-17T10:01:37.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.705+0000 7f80ca000640 1 -- 172.21.15.32:0/895730238 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80cc00ce30 con 0x7f80d415eba0 2024-04-17T10:01:37.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.705+0000 7f80ca000640 1 -- 172.21.15.32:0/895730238 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc00a7e0 con 0x7f80d415eba0 2024-04-17T10:01:37.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.705+0000 7f80d9200640 1 -- 172.21.15.32:0/895730238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d415f3e0 con 0x7f80d415eba0 2024-04-17T10:01:37.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.705+0000 7f80d9200640 1 -- 172.21.15.32:0/895730238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d415fc20 con 0x7f80d415eba0 2024-04-17T10:01:37.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.705+0000 7f80d9200640 1 -- 172.21.15.32:0/895730238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80d4068a90 con 0x7f80d415eba0 2024-04-17T10:01:37.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.705+0000 7f80ca000640 1 -- 172.21.15.32:0/895730238 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f80cc00a980 con 0x7f80d415eba0 2024-04-17T10:01:37.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.705+0000 7f80ca000640 1 --2- 172.21.15.32:0/895730238 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f80b4068380 0x7f80b406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:37.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.705+0000 7f80ca000640 1 -- 172.21.15.32:0/895730238 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4857+0+0 (secure 0 0 0) 0x7f80cc085b10 con 0x7f80d415eba0 2024-04-17T10:01:37.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.705+0000 7f80d1e00640 1 --2- 172.21.15.32:0/895730238 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f80b4068380 0x7f80b406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:37.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.709+0000 7f80d1e00640 1 --2- 172.21.15.32:0/895730238 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f80b4068380 0x7f80b406a840 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f80c40ad000 tx=0x7f80c40a4f90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:37.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.709+0000 7f80ca000640 1 -- 172.21.15.32:0/895730238 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f80d415f3e0 con 0x7f80d415eba0 2024-04-17T10:01:37.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.877+0000 7f80d9200640 1 -- 172.21.15.32:0/895730238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80d40af960 con 0x7f80d415eba0 2024-04-17T10:01:37.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.877+0000 7f80ca000640 1 -- 172.21.15.32:0/895730238 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+12904 (secure 0 0 0) 0x7f80cc055dc0 con 0x7f80d415eba0 2024-04-17T10:01:37.876 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:37.876 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":93,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:01:37.123866+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:01:37.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.881+0000 7f80d9200640 1 -- 172.21.15.32:0/895730238 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f80b4068380 msgr2=0x7f80b406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:37.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.881+0000 7f80d9200640 1 --2- 172.21.15.32:0/895730238 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f80b4068380 0x7f80b406a840 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f80c40ad000 tx=0x7f80c40a4f90 comp rx=0 tx=0).stop 2024-04-17T10:01:37.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.881+0000 7f80d9200640 1 -- 172.21.15.32:0/895730238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d415eba0 msgr2=0x7f80d40cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:37.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.881+0000 7f80d9200640 1 --2- 172.21.15.32:0/895730238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d415eba0 0x7f80d40cded0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00c780 tx=0x7f80cc00cc40 comp rx=0 tx=0).stop 2024-04-17T10:01:37.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.881+0000 7f80d9200640 1 -- 172.21.15.32:0/895730238 shutdown_connections 2024-04-17T10:01:37.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.881+0000 7f80d9200640 1 --2- 172.21.15.32:0/895730238 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f80b4068380 0x7f80b406a840 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.881+0000 7f80d9200640 1 --2- 172.21.15.32:0/895730238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d415eba0 0x7f80d40cded0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.881+0000 7f80d9200640 1 --2- 172.21.15.32:0/895730238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80d409b3c0 0x7f80d40c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.881+0000 7f80d9200640 1 --2- 172.21.15.32:0/895730238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80d406da90 0x7f80d4158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:37.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.881+0000 7f80d9200640 1 -- 172.21.15.32:0/895730238 >> 172.21.15.32:0/895730238 conn(0x7f80d40b3fd0 msgr2=0x7f80d4074aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:37.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.881+0000 7f80d9200640 1 -- 172.21.15.32:0/895730238 shutdown_connections 2024-04-17T10:01:37.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:37.881+0000 7f80d9200640 1 -- 172.21.15.32:0/895730238 wait complete. 2024-04-17T10:01:37.896 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:01:37.897 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:01:38.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.041+0000 7f54e2000640 1 Processor -- start 2024-04-17T10:01:38.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.041+0000 7f54e2000640 1 -- start start 2024-04-17T10:01:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.045+0000 7f54e2000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54dc15c4c0 0x7f54dc0adb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.045+0000 7f54e2000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54dc06c140 0x7f54dc0b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.045+0000 7f54e2000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54dc1629a0 con 0x7f54dc15f9c0 2024-04-17T10:01:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.045+0000 7f54e2000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f54dc162c80 con 0x7f54dc06c140 2024-04-17T10:01:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.045+0000 7f54e2000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f54dc162f60 con 0x7f54dc15c4c0 2024-04-17T10:01:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.045+0000 7f54dbe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54dc06c140 0x7f54dc0b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.045+0000 7f54dbe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54dc06c140 0x7f54dc0b3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54056/0 (socket says 172.21.15.32:54056) 2024-04-17T10:01:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.045+0000 7f54dbe00640 1 -- 172.21.15.32:0/2840882785 learned_addr learned my addr 172.21.15.32:0/2840882785 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.045+0000 7f54db400640 1 --2- 172.21.15.32:0/2840882785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54dc15c4c0 0x7f54dc0adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.045+0000 7f54dbe00640 1 -- 172.21.15.32:0/2840882785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54dc15c4c0 msgr2=0x7f54dc0adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.045+0000 7f54dbe00640 1 --2- 172.21.15.32:0/2840882785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54dc15c4c0 0x7f54dc0adb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.045+0000 7f54dbe00640 1 -- 172.21.15.32:0/2840882785 >> v1:172.21.15.144:6789/0 conn(0x7f54dc15f9c0 legacy=0x7f54dc06ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.045+0000 7f54dbe00640 1 -- 172.21.15.32:0/2840882785 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54dc163240 con 0x7f54dc06c140 2024-04-17T10:01:38.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.049+0000 7f54dbe00640 1 --2- 172.21.15.32:0/2840882785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54dc06c140 0x7f54dc0b3870 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009e50 tx=0x7f54d401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ddc855199f7f066 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:38.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.049+0000 7f54daa00640 1 -- 172.21.15.32:0/2840882785 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d40a5810 con 0x7f54dc06c140 2024-04-17T10:01:38.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.049+0000 7f54daa00640 1 -- 172.21.15.32:0/2840882785 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54d40a59b0 con 0x7f54dc06c140 2024-04-17T10:01:38.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.049+0000 7f54daa00640 1 -- 172.21.15.32:0/2840882785 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d40a5d80 con 0x7f54dc06c140 2024-04-17T10:01:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.049+0000 7f54e2000640 1 -- 172.21.15.32:0/2840882785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54dc06c140 msgr2=0x7f54dc0b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.049+0000 7f54e2000640 1 --2- 172.21.15.32:0/2840882785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54dc06c140 0x7f54dc0b3870 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009e50 tx=0x7f54d401ca30 comp rx=0 tx=0).stop 2024-04-17T10:01:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.049+0000 7f54e2000640 1 -- 172.21.15.32:0/2840882785 shutdown_connections 2024-04-17T10:01:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.049+0000 7f54e2000640 1 --2- 172.21.15.32:0/2840882785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54dc06c140 0x7f54dc0b3870 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.049+0000 7f54e2000640 1 --2- 172.21.15.32:0/2840882785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54dc15c4c0 0x7f54dc0adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.049+0000 7f54e2000640 1 -- 172.21.15.32:0/2840882785 >> 172.21.15.32:0/2840882785 conn(0x7f54dc0b2120 msgr2=0x7f54dc0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.049+0000 7f54e2000640 1 -- 172.21.15.32:0/2840882785 shutdown_connections 2024-04-17T10:01:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.049+0000 7f54e2000640 1 -- 172.21.15.32:0/2840882785 wait complete. 2024-04-17T10:01:38.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.049+0000 7f54e2000640 1 Processor -- start 2024-04-17T10:01:38.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.049+0000 7f54e2000640 1 -- start start 2024-04-17T10:01:38.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.049+0000 7f54e2000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54dc06c140 0x7f54dc1538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:38.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.049+0000 7f54e2000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54dc15c4c0 0x7f54dc153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:38.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.049+0000 7f54e2000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54dc15f9c0 0x7f54dc15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:38.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.049+0000 7f54e2000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f54dc1626b0 con 0x7f54dc06c140 2024-04-17T10:01:38.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.049+0000 7f54e2000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f54dc162530 con 0x7f54dc15c4c0 2024-04-17T10:01:38.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.049+0000 7f54e2000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f54dc16b6a0 con 0x7f54dc15f9c0 2024-04-17T10:01:38.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.053+0000 7f54db400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54dc06c140 0x7f54dc1538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.053+0000 7f54db400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54dc06c140 0x7f54dc1538e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54066/0 (socket says 172.21.15.32:54066) 2024-04-17T10:01:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.053+0000 7f54db400640 1 -- 172.21.15.32:0/3890021346 learned_addr learned my addr 172.21.15.32:0/3890021346 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.053+0000 7f54d3e00640 1 --2- 172.21.15.32:0/3890021346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54dc15c4c0 0x7f54dc153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.053+0000 7f54dbe00640 1 --2- 172.21.15.32:0/3890021346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54dc15f9c0 0x7f54dc15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.053+0000 7f54db400640 1 -- 172.21.15.32:0/3890021346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54dc15f9c0 msgr2=0x7f54dc15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.053+0000 7f54db400640 1 --2- 172.21.15.32:0/3890021346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54dc15f9c0 0x7f54dc15b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.053+0000 7f54db400640 1 -- 172.21.15.32:0/3890021346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54dc15c4c0 msgr2=0x7f54dc153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.053+0000 7f54db400640 1 --2- 172.21.15.32:0/3890021346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54dc15c4c0 0x7f54dc153e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.053+0000 7f54db400640 1 -- 172.21.15.32:0/3890021346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54dc0b1600 con 0x7f54dc06c140 2024-04-17T10:01:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.053+0000 7f54d3e00640 1 --2- 172.21.15.32:0/3890021346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54dc15c4c0 0x7f54dc153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.053+0000 7f54db400640 1 --2- 172.21.15.32:0/3890021346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54dc06c140 0x7f54dc1538e0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00d800 tx=0x7f54cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.053+0000 7f54d3400640 1 -- 172.21.15.32:0/3890021346 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc00b6c0 con 0x7f54dc06c140 2024-04-17T10:01:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.053+0000 7f54d3400640 1 -- 172.21.15.32:0/3890021346 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54cc019070 con 0x7f54dc06c140 2024-04-17T10:01:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.053+0000 7f54d3400640 1 -- 172.21.15.32:0/3890021346 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc00ba60 con 0x7f54dc06c140 2024-04-17T10:01:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.053+0000 7f54e2000640 1 -- 172.21.15.32:0/3890021346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54dc0b18e0 con 0x7f54dc06c140 2024-04-17T10:01:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.053+0000 7f54e2000640 1 -- 172.21.15.32:0/3890021346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54dc1549d0 con 0x7f54dc06c140 2024-04-17T10:01:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.053+0000 7f54e2000640 1 -- 172.21.15.32:0/3890021346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54dc068a90 con 0x7f54dc06c140 2024-04-17T10:01:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.053+0000 7f54d3400640 1 -- 172.21.15.32:0/3890021346 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f54cc00bc00 con 0x7f54dc06c140 2024-04-17T10:01:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.053+0000 7f54d3400640 1 --2- 172.21.15.32:0/3890021346 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f54b8068400 0x7f54b806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.057+0000 7f54d3400640 1 -- 172.21.15.32:0/3890021346 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4857+0+0 (secure 0 0 0) 0x7f54cc086900 con 0x7f54dc06c140 2024-04-17T10:01:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.057+0000 7f54d3e00640 1 --2- 172.21.15.32:0/3890021346 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f54b8068400 0x7f54b806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.057+0000 7f54d3e00640 1 --2- 172.21.15.32:0/3890021346 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f54b8068400 0x7f54b806a8c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f54c4000f70 tx=0x7f54c4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:38.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.061+0000 7f54d3400640 1 -- 172.21.15.32:0/3890021346 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f54cc057380 con 0x7f54dc06c140 2024-04-17T10:01:38.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.233+0000 7f54e2000640 1 -- 172.21.15.32:0/3890021346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f54dc0b3870 con 0x7f54dc06c140 2024-04-17T10:01:38.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.233+0000 7f54d3400640 1 -- 172.21.15.32:0/3890021346 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f54cc056c80 con 0x7f54dc06c140 2024-04-17T10:01:38.230 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T10:01:38.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.233+0000 7f54e2000640 1 -- 172.21.15.32:0/3890021346 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f54b8068400 msgr2=0x7f54b806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:38.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.233+0000 7f54e2000640 1 --2- 172.21.15.32:0/3890021346 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f54b8068400 0x7f54b806a8c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f54c4000f70 tx=0x7f54c4009040 comp rx=0 tx=0).stop 2024-04-17T10:01:38.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.233+0000 7f54e2000640 1 -- 172.21.15.32:0/3890021346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54dc06c140 msgr2=0x7f54dc1538e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:38.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.233+0000 7f54e2000640 1 --2- 172.21.15.32:0/3890021346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54dc06c140 0x7f54dc1538e0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00d800 tx=0x7f54cc00dcc0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.233+0000 7f54e2000640 1 -- 172.21.15.32:0/3890021346 shutdown_connections 2024-04-17T10:01:38.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.233+0000 7f54e2000640 1 --2- 172.21.15.32:0/3890021346 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f54b8068400 0x7f54b806a8c0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.233+0000 7f54e2000640 1 --2- 172.21.15.32:0/3890021346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54dc15f9c0 0x7f54dc15b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.233+0000 7f54e2000640 1 --2- 172.21.15.32:0/3890021346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54dc15c4c0 0x7f54dc153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.233+0000 7f54e2000640 1 --2- 172.21.15.32:0/3890021346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54dc06c140 0x7f54dc1538e0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.233+0000 7f54e2000640 1 -- 172.21.15.32:0/3890021346 >> 172.21.15.32:0/3890021346 conn(0x7f54dc0b2120 msgr2=0x7f54dc168f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:38.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.233+0000 7f54e2000640 1 -- 172.21.15.32:0/3890021346 shutdown_connections 2024-04-17T10:01:38.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.237+0000 7f54e2000640 1 -- 172.21.15.32:0/3890021346 wait complete. 2024-04-17T10:01:38.252 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T10:01:38.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.393+0000 7ff60fe00640 1 Processor -- start 2024-04-17T10:01:38.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.393+0000 7ff60fe00640 1 -- start start 2024-04-17T10:01:38.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.397+0000 7ff60fe00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff608072500 0x7ff6080cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.397+0000 7ff60fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff60815d8d0 0x7ff6080cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.397+0000 7ff60fe00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6080cdac0 con 0x7ff608145f30 2024-04-17T10:01:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.397+0000 7ff60fe00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff60806f400 con 0x7ff608072500 2024-04-17T10:01:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.397+0000 7ff60fe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff608146310 con 0x7ff60815d8d0 2024-04-17T10:01:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.397+0000 7ff60d400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff608145f30 0x7ff608071df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36110/0 (socket says 172.21.15.32:36110) 2024-04-17T10:01:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.397+0000 7ff60d400640 1 -- 172.21.15.32:0/3929359595 learned_addr learned my addr 172.21.15.32:0/3929359595 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.397+0000 7ff60de00640 1 --2- 172.21.15.32:0/3929359595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff60815d8d0 0x7ff6080cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.397+0000 7ff60ca00640 1 --2- 172.21.15.32:0/3929359595 >> v2:172.21.15.144:3300/0 conn(0x7ff608072500 0x7ff6080cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:38.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.397+0000 7ff607e00640 1 -- 172.21.15.32:0/3929359595 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3871954350 0 0) 0x7ff6080cdac0 con 0x7ff608145f30 2024-04-17T10:01:38.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.397+0000 7ff607e00640 1 -- 172.21.15.32:0/3929359595 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5f0003530 con 0x7ff608145f30 2024-04-17T10:01:38.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.397+0000 7ff60de00640 1 -- 172.21.15.32:0/3929359595 >> v2:172.21.15.144:3300/0 conn(0x7ff608072500 msgr2=0x7ff6080cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:38.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.397+0000 7ff60de00640 1 --2- 172.21.15.32:0/3929359595 >> v2:172.21.15.144:3300/0 conn(0x7ff608072500 0x7ff6080cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.397+0000 7ff60de00640 1 -- 172.21.15.32:0/3929359595 >> v1:172.21.15.32:6789/0 conn(0x7ff608145f30 legacy=0x7ff608071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:38.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.397+0000 7ff607e00640 1 -- 172.21.15.32:0/3929359595 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4021620012 0 0) 0x7ff5f0003530 con 0x7ff608145f30 2024-04-17T10:01:38.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.397+0000 7ff60de00640 1 -- 172.21.15.32:0/3929359595 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60815cdf0 con 0x7ff60815d8d0 2024-04-17T10:01:38.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.401+0000 7ff60de00640 1 --2- 172.21.15.32:0/3929359595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff60815d8d0 0x7ff6080cd3b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff600009d90 tx=0x7ff60001f080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=152e1139b2220025 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:38.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.401+0000 7ff607e00640 1 -- 172.21.15.32:0/3929359595 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6000b5070 con 0x7ff60815d8d0 2024-04-17T10:01:38.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.401+0000 7ff607e00640 1 -- 172.21.15.32:0/3929359595 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6000a8bd0 con 0x7ff60815d8d0 2024-04-17T10:01:38.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.401+0000 7ff607e00640 1 -- 172.21.15.32:0/3929359595 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc0016a0 con 0x7ff60815d8d0 2024-04-17T10:01:38.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.401+0000 7ff60fe00640 1 -- 172.21.15.32:0/3929359595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff60815d8d0 msgr2=0x7ff6080cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:38.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.401+0000 7ff60fe00640 1 --2- 172.21.15.32:0/3929359595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff60815d8d0 0x7ff6080cd3b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff600009d90 tx=0x7ff60001f080 comp rx=0 tx=0).stop 2024-04-17T10:01:38.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.401+0000 7ff60fe00640 1 -- 172.21.15.32:0/3929359595 shutdown_connections 2024-04-17T10:01:38.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.401+0000 7ff60fe00640 1 --2- 172.21.15.32:0/3929359595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff60815d8d0 0x7ff6080cd3b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.401+0000 7ff60fe00640 1 --2- 172.21.15.32:0/3929359595 >> v2:172.21.15.144:3300/0 conn(0x7ff608072500 0x7ff6080cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.401+0000 7ff60fe00640 1 -- 172.21.15.32:0/3929359595 >> 172.21.15.32:0/3929359595 conn(0x7ff60806ba30 msgr2=0x7ff60806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:38.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.401+0000 7ff60fe00640 1 -- 172.21.15.32:0/3929359595 shutdown_connections 2024-04-17T10:01:38.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.401+0000 7ff60fe00640 1 -- 172.21.15.32:0/3929359595 wait complete. 2024-04-17T10:01:38.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.401+0000 7ff60fe00640 1 Processor -- start 2024-04-17T10:01:38.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.401+0000 7ff60fe00640 1 -- start start 2024-04-17T10:01:38.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff60fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff608072500 0x7ff608136c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:38.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff60fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff608145f30 0x7ff6080cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:38.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff60fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff60815d8d0 0x7ff6080a94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:38.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff60fe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff60815cb90 con 0x7ff608072500 2024-04-17T10:01:38.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff60fe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff60815d440 con 0x7ff608145f30 2024-04-17T10:01:38.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff60fe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff6081601d0 con 0x7ff60815d8d0 2024-04-17T10:01:38.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff60d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff608072500 0x7ff608136c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:38.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff60d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff608072500 0x7ff608136c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54072/0 (socket says 172.21.15.32:54072) 2024-04-17T10:01:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff60d400640 1 -- 172.21.15.32:0/1621946502 learned_addr learned my addr 172.21.15.32:0/1621946502 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff60ca00640 1 --2- 172.21.15.32:0/1621946502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff608145f30 0x7ff6080cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff60de00640 1 --2- 172.21.15.32:0/1621946502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff60815d8d0 0x7ff6080a94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff60d400640 1 -- 172.21.15.32:0/1621946502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff60815d8d0 msgr2=0x7ff6080a94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff60d400640 1 --2- 172.21.15.32:0/1621946502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff60815d8d0 0x7ff6080a94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff60d400640 1 -- 172.21.15.32:0/1621946502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff608145f30 msgr2=0x7ff6080cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff60d400640 1 --2- 172.21.15.32:0/1621946502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff608145f30 0x7ff6080cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff60d400640 1 -- 172.21.15.32:0/1621946502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff608145b00 con 0x7ff608072500 2024-04-17T10:01:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff60de00640 1 --2- 172.21.15.32:0/1621946502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff60815d8d0 0x7ff6080a94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff60ca00640 1 --2- 172.21.15.32:0/1621946502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff608145f30 0x7ff6080cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff60d400640 1 --2- 172.21.15.32:0/1621946502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff608072500 0x7ff608136c90 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00c730 tx=0x7ff5fc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:38.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff605600640 1 -- 172.21.15.32:0/1621946502 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc018070 con 0x7ff608072500 2024-04-17T10:01:38.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff605600640 1 -- 172.21.15.32:0/1621946502 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5fc0072f0 con 0x7ff608072500 2024-04-17T10:01:38.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff60fe00640 1 -- 172.21.15.32:0/1621946502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff60815ff90 con 0x7ff608072500 2024-04-17T10:01:38.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff605600640 1 -- 172.21.15.32:0/1621946502 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc0047f0 con 0x7ff608072500 2024-04-17T10:01:38.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff60fe00640 1 -- 172.21.15.32:0/1621946502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6080cf510 con 0x7ff608072500 2024-04-17T10:01:38.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.405+0000 7ff60fe00640 1 -- 172.21.15.32:0/1621946502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff608068a90 con 0x7ff608072500 2024-04-17T10:01:38.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.409+0000 7ff605600640 1 -- 172.21.15.32:0/1621946502 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff5fc01b070 con 0x7ff608072500 2024-04-17T10:01:38.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.409+0000 7ff605600640 1 --2- 172.21.15.32:0/1621946502 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff5e8068380 0x7ff5e806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:38.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.409+0000 7ff605600640 1 -- 172.21.15.32:0/1621946502 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4857+0+0 (secure 0 0 0) 0x7ff5fc085be0 con 0x7ff608072500 2024-04-17T10:01:38.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.409+0000 7ff60ca00640 1 --2- 172.21.15.32:0/1621946502 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff5e8068380 0x7ff5e806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:38.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.409+0000 7ff60ca00640 1 --2- 172.21.15.32:0/1621946502 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff5e8068380 0x7ff5e806a840 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff608072050 tx=0x7ff5f4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:38.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.413+0000 7ff605600640 1 -- 172.21.15.32:0/1621946502 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff5fc0563c0 con 0x7ff608072500 2024-04-17T10:01:38.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.577+0000 7ff60fe00640 1 -- 172.21.15.32:0/1621946502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff608072f90 con 0x7ff608072500 2024-04-17T10:01:38.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.577+0000 7ff605600640 1 -- 172.21.15.32:0/1621946502 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v93) ==== 84+0+65 (secure 0 0 0) 0x7ff5fc055cc0 con 0x7ff608072500 2024-04-17T10:01:38.576 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:38.576 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-17T10:01:38.576 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:01:38.576 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-04-17T10:01:38.576 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-04-17T10:01:38.576 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:01:38.576 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-17T10:01:38.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.581+0000 7ff60fe00640 1 -- 172.21.15.32:0/1621946502 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff5e8068380 msgr2=0x7ff5e806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:38.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.581+0000 7ff60fe00640 1 --2- 172.21.15.32:0/1621946502 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff5e8068380 0x7ff5e806a840 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff608072050 tx=0x7ff5f4008040 comp rx=0 tx=0).stop 2024-04-17T10:01:38.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.581+0000 7ff60fe00640 1 -- 172.21.15.32:0/1621946502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff608072500 msgr2=0x7ff608136c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:38.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.581+0000 7ff60fe00640 1 --2- 172.21.15.32:0/1621946502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff608072500 0x7ff608136c90 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00c730 tx=0x7ff5fc00cbf0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.581+0000 7ff60fe00640 1 -- 172.21.15.32:0/1621946502 shutdown_connections 2024-04-17T10:01:38.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.581+0000 7ff60fe00640 1 --2- 172.21.15.32:0/1621946502 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff5e8068380 0x7ff5e806a840 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.581+0000 7ff60fe00640 1 --2- 172.21.15.32:0/1621946502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff60815d8d0 0x7ff6080a94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.581+0000 7ff60fe00640 1 --2- 172.21.15.32:0/1621946502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff608145f30 0x7ff6080cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.581+0000 7ff60fe00640 1 --2- 172.21.15.32:0/1621946502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff608072500 0x7ff608136c90 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.581+0000 7ff60fe00640 1 -- 172.21.15.32:0/1621946502 >> 172.21.15.32:0/1621946502 conn(0x7ff60806ba30 msgr2=0x7ff6080ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:38.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.581+0000 7ff60fe00640 1 -- 172.21.15.32:0/1621946502 shutdown_connections 2024-04-17T10:01:38.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.581+0000 7ff60fe00640 1 -- 172.21.15.32:0/1621946502 wait complete. 2024-04-17T10:01:38.597 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T10:01:38.598 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T10:01:38.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.745+0000 7f37e8600640 1 Processor -- start 2024-04-17T10:01:38.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.745+0000 7f37e8600640 1 -- start start 2024-04-17T10:01:38.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.745+0000 7f37e8600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37e00ce440 0x7f37e009a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:38.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.745+0000 7f37e8600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37e00d1e50 0x7f37e009af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:38.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.745+0000 7f37e8600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37e00d0db0 con 0x7f37e006ce00 2024-04-17T10:01:38.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.745+0000 7f37e8600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f37e00d1090 con 0x7f37e00ce440 2024-04-17T10:01:38.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.745+0000 7f37e8600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f37e00d1370 con 0x7f37e00d1e50 2024-04-17T10:01:38.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.745+0000 7f37e5c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37e00ce440 0x7f37e009a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:38.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.745+0000 7f37e5200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f37e006ce00 0x7f37e015e160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36120/0 (socket says 172.21.15.32:36120) 2024-04-17T10:01:38.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.745+0000 7f37e5200640 1 -- 172.21.15.32:0/1484897922 learned_addr learned my addr 172.21.15.32:0/1484897922 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:38.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.745+0000 7f37e6600640 1 --2- 172.21.15.32:0/1484897922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37e00d1e50 0x7f37e009af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:38.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.745+0000 7f37e5c00640 1 -- 172.21.15.32:0/1484897922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37e00d1e50 msgr2=0x7f37e009af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:38.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.745+0000 7f37e5c00640 1 --2- 172.21.15.32:0/1484897922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37e00d1e50 0x7f37e009af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.745+0000 7f37e5c00640 1 -- 172.21.15.32:0/1484897922 >> v1:172.21.15.32:6789/0 conn(0x7f37e006ce00 legacy=0x7f37e015e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:38.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.745+0000 7f37e5c00640 1 -- 172.21.15.32:0/1484897922 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e00d1650 con 0x7f37e00ce440 2024-04-17T10:01:38.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.745+0000 7f37dfe00640 1 -- 172.21.15.32:0/1484897922 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3273274892 0 0) 0x7f37e00d0db0 con 0x7f37e006ce00 2024-04-17T10:01:38.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.749+0000 7f37e5c00640 1 --2- 172.21.15.32:0/1484897922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e00ce440 0x7f37e009a9f0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f37d40097c0 tx=0x7f37d401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9554c6dd0c46f79a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:38.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.749+0000 7f37dfe00640 1 -- 172.21.15.32:0/1484897922 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e00cea30 con 0x7f37e00ce440 2024-04-17T10:01:38.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.749+0000 7f37dfe00640 1 -- 172.21.15.32:0/1484897922 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d40af070 con 0x7f37e00ce440 2024-04-17T10:01:38.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.749+0000 7f37dfe00640 1 -- 172.21.15.32:0/1484897922 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d40aa810 con 0x7f37e00ce440 2024-04-17T10:01:38.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.749+0000 7f37e8600640 1 -- 172.21.15.32:0/1484897922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e00ce440 msgr2=0x7f37e009a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:38.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.749+0000 7f37e8600640 1 --2- 172.21.15.32:0/1484897922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e00ce440 0x7f37e009a9f0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f37d40097c0 tx=0x7f37d401c470 comp rx=0 tx=0).stop 2024-04-17T10:01:38.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.749+0000 7f37e8600640 1 -- 172.21.15.32:0/1484897922 shutdown_connections 2024-04-17T10:01:38.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.749+0000 7f37e8600640 1 --2- 172.21.15.32:0/1484897922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37e00d1e50 0x7f37e009af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.749+0000 7f37e8600640 1 --2- 172.21.15.32:0/1484897922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e00ce440 0x7f37e009a9f0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.749+0000 7f37e8600640 1 -- 172.21.15.32:0/1484897922 >> 172.21.15.32:0/1484897922 conn(0x7f37e00b3fe0 msgr2=0x7f37e00b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:38.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.749+0000 7f37e8600640 1 -- 172.21.15.32:0/1484897922 shutdown_connections 2024-04-17T10:01:38.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.749+0000 7f37e8600640 1 -- 172.21.15.32:0/1484897922 wait complete. 2024-04-17T10:01:38.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e8600640 1 Processor -- start 2024-04-17T10:01:38.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e8600640 1 -- start start 2024-04-17T10:01:38.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e8600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37e006ce00 0x7f37e014f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:38.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e8600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e00ce440 0x7f37e0155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:38.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e8600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37e00d1e50 0x7f37e0155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:38.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e8600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f37e00cced0 con 0x7f37e00ce440 2024-04-17T10:01:38.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e8600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f37e00ccd50 con 0x7f37e00d1e50 2024-04-17T10:01:38.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e8600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f37e00cd050 con 0x7f37e006ce00 2024-04-17T10:01:38.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e5200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e00ce440 0x7f37e0155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:38.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e5200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e00ce440 0x7f37e0155130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54084/0 (socket says 172.21.15.32:54084) 2024-04-17T10:01:38.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e5200640 1 -- 172.21.15.32:0/891932348 learned_addr learned my addr 172.21.15.32:0/891932348 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:38.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e5c00640 1 --2- 172.21.15.32:0/891932348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37e006ce00 0x7f37e014f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:38.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e5200640 1 -- 172.21.15.32:0/891932348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37e006ce00 msgr2=0x7f37e014f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:38.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e5200640 1 --2- 172.21.15.32:0/891932348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37e006ce00 0x7f37e014f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e5200640 1 -- 172.21.15.32:0/891932348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37e00d1e50 msgr2=0x7f37e0155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:38.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e6600640 1 --2- 172.21.15.32:0/891932348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37e00d1e50 0x7f37e0155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:38.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e5200640 1 --2- 172.21.15.32:0/891932348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37e00d1e50 0x7f37e0155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:38.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e5200640 1 -- 172.21.15.32:0/891932348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e01513e0 con 0x7f37e00ce440 2024-04-17T10:01:38.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e5c00640 1 --2- 172.21.15.32:0/891932348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37e006ce00 0x7f37e014f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:38.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e6600640 1 --2- 172.21.15.32:0/891932348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37e00d1e50 0x7f37e0155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:38.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e5200640 1 --2- 172.21.15.32:0/891932348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e00ce440 0x7f37e0155130 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f37cc0011f0 tx=0x7f37cc00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:38.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37dd600640 1 -- 172.21.15.32:0/891932348 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc018070 con 0x7f37e00ce440 2024-04-17T10:01:38.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37dd600640 1 -- 172.21.15.32:0/891932348 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37cc0074e0 con 0x7f37e00ce440 2024-04-17T10:01:38.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e8600640 1 -- 172.21.15.32:0/891932348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e01516c0 con 0x7f37e00ce440 2024-04-17T10:01:38.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37dd600640 1 -- 172.21.15.32:0/891932348 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc004a10 con 0x7f37e00ce440 2024-04-17T10:01:38.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e8600640 1 -- 172.21.15.32:0/891932348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e0151c80 con 0x7f37e00ce440 2024-04-17T10:01:38.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.753+0000 7f37e8600640 1 -- 172.21.15.32:0/891932348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e0068a90 con 0x7f37e00ce440 2024-04-17T10:01:38.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.757+0000 7f37dd600640 1 -- 172.21.15.32:0/891932348 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f37cc01b070 con 0x7f37e00ce440 2024-04-17T10:01:38.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.757+0000 7f37dd600640 1 --2- 172.21.15.32:0/891932348 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f37c0068380 0x7f37c006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:38.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.757+0000 7f37dd600640 1 -- 172.21.15.32:0/891932348 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4857+0+0 (secure 0 0 0) 0x7f37cc085cc0 con 0x7f37e00ce440 2024-04-17T10:01:38.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.757+0000 7f37e5c00640 1 --2- 172.21.15.32:0/891932348 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f37c0068380 0x7f37c006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:38.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.757+0000 7f37e5c00640 1 --2- 172.21.15.32:0/891932348 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f37c0068380 0x7f37c006a840 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f37d40ae000 tx=0x7f37d40a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:38.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.761+0000 7f37dd600640 1 -- 172.21.15.32:0/891932348 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f37cc0566c0 con 0x7f37e00ce440 2024-04-17T10:01:38.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:38.925+0000 7f37e8600640 1 -- 172.21.15.32:0/891932348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f37e0159e80 con 0x7f37e00ce440 2024-04-17T10:01:39.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.133+0000 7f37dd600640 1 -- 172.21.15.32:0/891932348 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v94) ==== 138+0+0 (secure 0 0 0) 0x7f37cc055fc0 con 0x7f37e00ce440 2024-04-17T10:01:39.129 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-04-17T10:01:39.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.133+0000 7f37e8600640 1 -- 172.21.15.32:0/891932348 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f37c0068380 msgr2=0x7f37c006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:39.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.133+0000 7f37e8600640 1 --2- 172.21.15.32:0/891932348 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f37c0068380 0x7f37c006a840 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f37d40ae000 tx=0x7f37d40a7000 comp rx=0 tx=0).stop 2024-04-17T10:01:39.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.133+0000 7f37e8600640 1 -- 172.21.15.32:0/891932348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e00ce440 msgr2=0x7f37e0155130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:39.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.133+0000 7f37e8600640 1 --2- 172.21.15.32:0/891932348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e00ce440 0x7f37e0155130 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f37cc0011f0 tx=0x7f37cc00c9b0 comp rx=0 tx=0).stop 2024-04-17T10:01:39.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.133+0000 7f37e8600640 1 -- 172.21.15.32:0/891932348 shutdown_connections 2024-04-17T10:01:39.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.133+0000 7f37e8600640 1 --2- 172.21.15.32:0/891932348 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f37c0068380 0x7f37c006a840 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:39.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.133+0000 7f37e8600640 1 --2- 172.21.15.32:0/891932348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37e00d1e50 0x7f37e0155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:39.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.133+0000 7f37e8600640 1 --2- 172.21.15.32:0/891932348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e00ce440 0x7f37e0155130 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:39.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.133+0000 7f37e8600640 1 --2- 172.21.15.32:0/891932348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37e006ce00 0x7f37e014f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:39.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.133+0000 7f37e8600640 1 -- 172.21.15.32:0/891932348 >> 172.21.15.32:0/891932348 conn(0x7f37e00b3fe0 msgr2=0x7f37e015a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:39.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.133+0000 7f37e8600640 1 -- 172.21.15.32:0/891932348 shutdown_connections 2024-04-17T10:01:39.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.133+0000 7f37e8600640 1 -- 172.21.15.32:0/891932348 wait complete. 2024-04-17T10:01:39.157 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-17T10:01:39.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.309+0000 7f66dfa00640 1 Processor -- start 2024-04-17T10:01:39.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.309+0000 7f66dfa00640 1 -- start start 2024-04-17T10:01:39.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.309+0000 7f66dfa00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f66d815b7d0 0x7f66d806e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:39.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.309+0000 7f66dfa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66d806f7c0 0x7f66d806ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:39.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.309+0000 7f66dfa00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66d80cc940 con 0x7f66d806f270 2024-04-17T10:01:39.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.309+0000 7f66dfa00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f66d80ccc20 con 0x7f66d815b7d0 2024-04-17T10:01:39.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.309+0000 7f66dfa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f66d806e0b0 con 0x7f66d806f7c0 2024-04-17T10:01:39.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.309+0000 7f66dd000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f66d815b7d0 0x7f66d806e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:39.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.309+0000 7f66dd000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f66d815b7d0 0x7f66d806e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54092/0 (socket says 172.21.15.32:54092) 2024-04-17T10:01:39.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.309+0000 7f66dd000640 1 -- 172.21.15.32:0/1791907420 learned_addr learned my addr 172.21.15.32:0/1791907420 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:39.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.309+0000 7f66d7e00640 1 --2- 172.21.15.32:0/1791907420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66d806f7c0 0x7f66d806ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:39.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.309+0000 7f66dd000640 1 -- 172.21.15.32:0/1791907420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66d806f7c0 msgr2=0x7f66d806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:39.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.309+0000 7f66dd000640 1 --2- 172.21.15.32:0/1791907420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66d806f7c0 0x7f66d806ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:39.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.309+0000 7f66dd000640 1 -- 172.21.15.32:0/1791907420 >> v1:172.21.15.144:6789/0 conn(0x7f66d806f270 legacy=0x7f66d80ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:39.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.309+0000 7f66dd000640 1 -- 172.21.15.32:0/1791907420 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d806e390 con 0x7f66d815b7d0 2024-04-17T10:01:39.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.309+0000 7f66d7e00640 1 --2- 172.21.15.32:0/1791907420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66d806f7c0 0x7f66d806ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:39.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.313+0000 7f66dd000640 1 --2- 172.21.15.32:0/1791907420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d815b7d0 0x7f66d806e8b0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009870 tx=0x7f66cc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5119ecc5b5d5389 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:39.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.313+0000 7f66d7400640 1 -- 172.21.15.32:0/1791907420 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc0b2070 con 0x7f66d815b7d0 2024-04-17T10:01:39.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.313+0000 7f66d7400640 1 -- 172.21.15.32:0/1791907420 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66cc0a9070 con 0x7f66d815b7d0 2024-04-17T10:01:39.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.313+0000 7f66d7400640 1 -- 172.21.15.32:0/1791907420 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc0ad880 con 0x7f66d815b7d0 2024-04-17T10:01:39.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.313+0000 7f66dfa00640 1 -- 172.21.15.32:0/1791907420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d815b7d0 msgr2=0x7f66d806e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:39.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.313+0000 7f66dfa00640 1 --2- 172.21.15.32:0/1791907420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d815b7d0 0x7f66d806e8b0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009870 tx=0x7f66cc01c490 comp rx=0 tx=0).stop 2024-04-17T10:01:39.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.313+0000 7f66dfa00640 1 -- 172.21.15.32:0/1791907420 shutdown_connections 2024-04-17T10:01:39.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.313+0000 7f66dfa00640 1 --2- 172.21.15.32:0/1791907420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66d806f7c0 0x7f66d806ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:39.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.313+0000 7f66dfa00640 1 --2- 172.21.15.32:0/1791907420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d815b7d0 0x7f66d806e8b0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:39.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.313+0000 7f66dfa00640 1 -- 172.21.15.32:0/1791907420 >> 172.21.15.32:0/1791907420 conn(0x7f66d80b2120 msgr2=0x7f66d80b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:39.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.313+0000 7f66dfa00640 1 -- 172.21.15.32:0/1791907420 shutdown_connections 2024-04-17T10:01:39.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.313+0000 7f66dfa00640 1 -- 172.21.15.32:0/1791907420 wait complete. 2024-04-17T10:01:39.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66dfa00640 1 Processor -- start 2024-04-17T10:01:39.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66dfa00640 1 -- start start 2024-04-17T10:01:39.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66dfa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66d806f270 0x7f66d8143d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:39.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66dfa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66d806f7c0 0x7f66d8151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:39.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66dfa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d815b7d0 0x7f66d815cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:39.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66dfa00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f66d8168ab0 con 0x7f66d815b7d0 2024-04-17T10:01:39.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66dfa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f66d806c380 con 0x7f66d806f270 2024-04-17T10:01:39.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66dfa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f66d8168c30 con 0x7f66d806f7c0 2024-04-17T10:01:39.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66dd000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66d806f270 0x7f66d8143d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:39.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66dda00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d815b7d0 0x7f66d815cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:39.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66dd000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66d806f270 0x7f66d8143d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:39656/0 (socket says 172.21.15.32:39656) 2024-04-17T10:01:39.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66dd000640 1 -- 172.21.15.32:0/308235788 learned_addr learned my addr 172.21.15.32:0/308235788 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:39.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66d7e00640 1 --2- 172.21.15.32:0/308235788 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66d806f7c0 0x7f66d8151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:39.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66dda00640 1 -- 172.21.15.32:0/308235788 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66d806f7c0 msgr2=0x7f66d8151060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:39.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66dda00640 1 --2- 172.21.15.32:0/308235788 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66d806f7c0 0x7f66d8151060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:39.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66dda00640 1 -- 172.21.15.32:0/308235788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66d806f270 msgr2=0x7f66d8143d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:39.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66dda00640 1 --2- 172.21.15.32:0/308235788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66d806f270 0x7f66d8143d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:39.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66dda00640 1 -- 172.21.15.32:0/308235788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d80ae310 con 0x7f66d815b7d0 2024-04-17T10:01:39.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66dd000640 1 --2- 172.21.15.32:0/308235788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66d806f270 0x7f66d8143d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:01:39.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66dda00640 1 --2- 172.21.15.32:0/308235788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d815b7d0 0x7f66d815cd80 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f66d000be00 tx=0x7f66d000bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:39.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66d4c00640 1 -- 172.21.15.32:0/308235788 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0003980 con 0x7f66d815b7d0 2024-04-17T10:01:39.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66d4c00640 1 -- 172.21.15.32:0/308235788 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d0003b20 con 0x7f66d815b7d0 2024-04-17T10:01:39.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66d4c00640 1 -- 172.21.15.32:0/308235788 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0004020 con 0x7f66d815b7d0 2024-04-17T10:01:39.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66dfa00640 1 -- 172.21.15.32:0/308235788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d80ae5f0 con 0x7f66d815b7d0 2024-04-17T10:01:39.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66dfa00640 1 -- 172.21.15.32:0/308235788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d80aebb0 con 0x7f66d815b7d0 2024-04-17T10:01:39.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.317+0000 7f66dfa00640 1 -- 172.21.15.32:0/308235788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d8068a90 con 0x7f66d815b7d0 2024-04-17T10:01:39.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.321+0000 7f66d4c00640 1 -- 172.21.15.32:0/308235788 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f66d0006d10 con 0x7f66d815b7d0 2024-04-17T10:01:39.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.321+0000 7f66d4c00640 1 --2- 172.21.15.32:0/308235788 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f66b8068380 0x7f66b806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:39.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.321+0000 7f66d4c00640 1 -- 172.21.15.32:0/308235788 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5227+0+0 (secure 0 0 0) 0x7f66d0086e00 con 0x7f66d815b7d0 2024-04-17T10:01:39.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.321+0000 7f66dd000640 1 --2- 172.21.15.32:0/308235788 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f66b8068380 0x7f66b806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:39.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.321+0000 7f66dd000640 1 --2- 172.21.15.32:0/308235788 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f66b8068380 0x7f66b806a840 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f66cc01c4d0 tx=0x7f66cc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:39.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.325+0000 7f66d4c00640 1 -- 172.21.15.32:0/308235788 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f66d0056f30 con 0x7f66d815b7d0 2024-04-17T10:01:39.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:39.493+0000 7f66dfa00640 1 -- 172.21.15.32:0/308235788 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f66d80b0890 con 0x7f66d815b7d0 2024-04-17T10:01:40.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.137+0000 7f66d4c00640 1 -- 172.21.15.32:0/308235788 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v95) ==== 170+0+0 (secure 0 0 0) 0x7f66d005b230 con 0x7f66d815b7d0 2024-04-17T10:01:40.137 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-04-17T10:01:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.145+0000 7f66dfa00640 1 -- 172.21.15.32:0/308235788 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f66b8068380 msgr2=0x7f66b806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.145+0000 7f66dfa00640 1 --2- 172.21.15.32:0/308235788 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f66b8068380 0x7f66b806a840 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f66cc01c4d0 tx=0x7f66cc008040 comp rx=0 tx=0).stop 2024-04-17T10:01:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.145+0000 7f66dfa00640 1 -- 172.21.15.32:0/308235788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d815b7d0 msgr2=0x7f66d815cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.145+0000 7f66dfa00640 1 --2- 172.21.15.32:0/308235788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d815b7d0 0x7f66d815cd80 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f66d000be00 tx=0x7f66d000bef0 comp rx=0 tx=0).stop 2024-04-17T10:01:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.145+0000 7f66dfa00640 1 -- 172.21.15.32:0/308235788 shutdown_connections 2024-04-17T10:01:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.145+0000 7f66dfa00640 1 --2- 172.21.15.32:0/308235788 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f66b8068380 0x7f66b806a840 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.145+0000 7f66dfa00640 1 --2- 172.21.15.32:0/308235788 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66d806f7c0 0x7f66d8151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.145+0000 7f66dfa00640 1 --2- 172.21.15.32:0/308235788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66d815b7d0 0x7f66d815cd80 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.145+0000 7f66dfa00640 1 --2- 172.21.15.32:0/308235788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66d806f270 0x7f66d8143d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:40.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.145+0000 7f66dfa00640 1 -- 172.21.15.32:0/308235788 >> 172.21.15.32:0/308235788 conn(0x7f66d80b2120 msgr2=0x7f66d80b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:40.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.145+0000 7f66dfa00640 1 -- 172.21.15.32:0/308235788 shutdown_connections 2024-04-17T10:01:40.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.149+0000 7f66dfa00640 1 -- 172.21.15.32:0/308235788 wait complete. 2024-04-17T10:01:40.181 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T10:01:40.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.329+0000 7f2fe6800640 1 Processor -- start 2024-04-17T10:01:40.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.329+0000 7f2fe6800640 1 -- start start 2024-04-17T10:01:40.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.329+0000 7f2fe6800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2fe015b7d0 0x7f2fe006e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:40.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.329+0000 7f2fe6800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fe00cc940 con 0x7f2fe006c140 2024-04-17T10:01:40.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.329+0000 7f2fe6800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fe00ccc20 con 0x7f2fe006f7c0 2024-04-17T10:01:40.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.329+0000 7f2fe6800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2fe006e0b0 con 0x7f2fe015b7d0 2024-04-17T10:01:40.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.333+0000 7f2fdf400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2fe015b7d0 0x7f2fe006e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:40.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.333+0000 7f2fdfe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2fe006c140 0x7f2fe00ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50440/0 (socket says 172.21.15.32:50440) 2024-04-17T10:01:40.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.333+0000 7f2fdfe00640 1 -- 172.21.15.32:0/103366388 learned_addr learned my addr 172.21.15.32:0/103366388 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:40.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.333+0000 7f2fdf400640 1 -- 172.21.15.32:0/103366388 >> v1:172.21.15.144:6789/0 conn(0x7f2fe006f7c0 legacy=0x7f2fe006ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:40.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.333+0000 7f2fdf400640 1 -- 172.21.15.32:0/103366388 >> v1:172.21.15.32:6789/0 conn(0x7f2fe006c140 legacy=0x7f2fe00ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:40.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.333+0000 7f2fdea00640 1 -- 172.21.15.32:0/103366388 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1621074379 0 0) 0x7f2fe00cc940 con 0x7f2fe006c140 2024-04-17T10:01:40.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.333+0000 7f2fdf400640 1 -- 172.21.15.32:0/103366388 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe006e390 con 0x7f2fe015b7d0 2024-04-17T10:01:40.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.333+0000 7f2fdf400640 1 --2- 172.21.15.32:0/103366388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fe015b7d0 0x7f2fe006e8b0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00097c0 tx=0x7f2fd001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b07ef387fb5bf45 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:40.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.333+0000 7f2fdea00640 1 -- 172.21.15.32:0/103366388 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd00ab430 con 0x7f2fe015b7d0 2024-04-17T10:01:40.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.333+0000 7f2fdea00640 1 -- 172.21.15.32:0/103366388 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd00b0070 con 0x7f2fe015b7d0 2024-04-17T10:01:40.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.333+0000 7f2fdea00640 1 -- 172.21.15.32:0/103366388 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd00ab7f0 con 0x7f2fe015b7d0 2024-04-17T10:01:40.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fe6800640 1 -- 172.21.15.32:0/103366388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fe015b7d0 msgr2=0x7f2fe006e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:40.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fe6800640 1 --2- 172.21.15.32:0/103366388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fe015b7d0 0x7f2fe006e8b0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00097c0 tx=0x7f2fd001c410 comp rx=0 tx=0).stop 2024-04-17T10:01:40.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fe6800640 1 -- 172.21.15.32:0/103366388 shutdown_connections 2024-04-17T10:01:40.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fe6800640 1 --2- 172.21.15.32:0/103366388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fe015b7d0 0x7f2fe006e8b0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:40.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fe6800640 1 -- 172.21.15.32:0/103366388 >> 172.21.15.32:0/103366388 conn(0x7f2fe00b2120 msgr2=0x7f2fe00b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:40.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fe6800640 1 -- 172.21.15.32:0/103366388 shutdown_connections 2024-04-17T10:01:40.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fe6800640 1 -- 172.21.15.32:0/103366388 wait complete. 2024-04-17T10:01:40.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fe6800640 1 Processor -- start 2024-04-17T10:01:40.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fe6800640 1 -- start start 2024-04-17T10:01:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fe6800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fe006c140 0x7f2fe01f8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fe6800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fe006f7c0 0x7f2fe01465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fe6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fe015b7d0 0x7f2fe01fb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fe6800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe0169c40 con 0x7f2fe015b7d0 2024-04-17T10:01:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fe6800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe0169ac0 con 0x7f2fe006c140 2024-04-17T10:01:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fe6800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2fe0169dc0 con 0x7f2fe006f7c0 2024-04-17T10:01:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fdfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fe015b7d0 0x7f2fe01fb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fdfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fe015b7d0 0x7f2fe01fb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52756/0 (socket says 172.21.15.32:52756) 2024-04-17T10:01:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fdfe00640 1 -- 172.21.15.32:0/184699981 learned_addr learned my addr 172.21.15.32:0/184699981 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fdf400640 1 --2- 172.21.15.32:0/184699981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fe006c140 0x7f2fe01f8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:40.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fd7e00640 1 --2- 172.21.15.32:0/184699981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fe006f7c0 0x7f2fe01465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:40.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fdfe00640 1 -- 172.21.15.32:0/184699981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fe006f7c0 msgr2=0x7f2fe01465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:40.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fdfe00640 1 --2- 172.21.15.32:0/184699981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fe006f7c0 0x7f2fe01465d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:40.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fdfe00640 1 -- 172.21.15.32:0/184699981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fe006c140 msgr2=0x7f2fe01f8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:40.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fdfe00640 1 --2- 172.21.15.32:0/184699981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fe006c140 0x7f2fe01f8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:40.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fdfe00640 1 -- 172.21.15.32:0/184699981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe0148670 con 0x7f2fe015b7d0 2024-04-17T10:01:40.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.337+0000 7f2fdfe00640 1 --2- 172.21.15.32:0/184699981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fe015b7d0 0x7f2fe01fb870 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800c6a0 tx=0x7f2fd800cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:40.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.341+0000 7f2fd7400640 1 -- 172.21.15.32:0/184699981 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8018070 con 0x7f2fe015b7d0 2024-04-17T10:01:40.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.341+0000 7f2fd7400640 1 -- 172.21.15.32:0/184699981 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd8004d70 con 0x7f2fe015b7d0 2024-04-17T10:01:40.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.341+0000 7f2fd7400640 1 -- 172.21.15.32:0/184699981 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd800a880 con 0x7f2fe015b7d0 2024-04-17T10:01:40.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.341+0000 7f2fe6800640 1 -- 172.21.15.32:0/184699981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe0148950 con 0x7f2fe015b7d0 2024-04-17T10:01:40.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.341+0000 7f2fe6800640 1 -- 172.21.15.32:0/184699981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe01fb350 con 0x7f2fe015b7d0 2024-04-17T10:01:40.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.341+0000 7f2fe6800640 1 -- 172.21.15.32:0/184699981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fe0068a90 con 0x7f2fe015b7d0 2024-04-17T10:01:40.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.341+0000 7f2fd7400640 1 -- 172.21.15.32:0/184699981 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2fd800aa20 con 0x7f2fe015b7d0 2024-04-17T10:01:40.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.345+0000 7f2fd7400640 1 --2- 172.21.15.32:0/184699981 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2fbc068380 0x7f2fbc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:40.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.345+0000 7f2fd7400640 1 -- 172.21.15.32:0/184699981 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5609+0+0 (secure 0 0 0) 0x7f2fd8085d40 con 0x7f2fe015b7d0 2024-04-17T10:01:40.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.345+0000 7f2fdf400640 1 --2- 172.21.15.32:0/184699981 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2fbc068380 0x7f2fbc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:40.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.345+0000 7f2fdf400640 1 --2- 172.21.15.32:0/184699981 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2fbc068380 0x7f2fbc06a840 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00af000 tx=0x7f2fd00a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:40.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.349+0000 7f2fd7400640 1 -- 172.21.15.32:0/184699981 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2fd8055d10 con 0x7f2fe015b7d0 2024-04-17T10:01:40.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:40.553+0000 7f2fe6800640 1 -- 172.21.15.32:0/184699981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2fe0157540 con 0x7f2fe015b7d0 2024-04-17T10:01:41.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.141+0000 7f2fd7400640 1 -- 172.21.15.32:0/184699981 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v77) ==== 174+0+0 (secure 0 0 0) 0x7f2fd805a010 con 0x7f2fe015b7d0 2024-04-17T10:01:41.139 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 14 and data pool 15 2024-04-17T10:01:41.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.145+0000 7f2fe6800640 1 -- 172.21.15.32:0/184699981 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2fbc068380 msgr2=0x7f2fbc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:41.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.145+0000 7f2fe6800640 1 --2- 172.21.15.32:0/184699981 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2fbc068380 0x7f2fbc06a840 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00af000 tx=0x7f2fd00a7000 comp rx=0 tx=0).stop 2024-04-17T10:01:41.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.145+0000 7f2fe6800640 1 -- 172.21.15.32:0/184699981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fe015b7d0 msgr2=0x7f2fe01fb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:41.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.145+0000 7f2fe6800640 1 --2- 172.21.15.32:0/184699981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fe015b7d0 0x7f2fe01fb870 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800c6a0 tx=0x7f2fd800cb60 comp rx=0 tx=0).stop 2024-04-17T10:01:41.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.145+0000 7f2fe6800640 1 -- 172.21.15.32:0/184699981 shutdown_connections 2024-04-17T10:01:41.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.145+0000 7f2fe6800640 1 --2- 172.21.15.32:0/184699981 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2fbc068380 0x7f2fbc06a840 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:41.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.145+0000 7f2fe6800640 1 --2- 172.21.15.32:0/184699981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fe006f7c0 0x7f2fe01465d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:41.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.145+0000 7f2fe6800640 1 --2- 172.21.15.32:0/184699981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fe015b7d0 0x7f2fe01fb870 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:41.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.145+0000 7f2fe6800640 1 --2- 172.21.15.32:0/184699981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fe006c140 0x7f2fe01f8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:41.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.145+0000 7f2fe6800640 1 -- 172.21.15.32:0/184699981 >> 172.21.15.32:0/184699981 conn(0x7f2fe00b2120 msgr2=0x7f2fe0157a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:41.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.149+0000 7f2fe6800640 1 -- 172.21.15.32:0/184699981 shutdown_connections 2024-04-17T10:01:41.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.149+0000 7f2fe6800640 1 -- 172.21.15.32:0/184699981 wait complete. 2024-04-17T10:01:41.199 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:01:41.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.345+0000 7f4a0c600640 1 Processor -- start 2024-04-17T10:01:41.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.345+0000 7f4a0c600640 1 -- start start 2024-04-17T10:01:41.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.345+0000 7f4a0c600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4a04072500 0x7f4a040cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:41.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.345+0000 7f4a0c600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a040cdac0 con 0x7f4a0415d8d0 2024-04-17T10:01:41.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.345+0000 7f4a0c600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a0415cdf0 con 0x7f4a04145f30 2024-04-17T10:01:41.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.345+0000 7f4a0c600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f4a0406f400 con 0x7f4a04072500 2024-04-17T10:01:41.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.345+0000 7f4a09200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4a04072500 0x7f4a040cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:41.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.345+0000 7f4a0a600640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4a0415d8d0 0x7f4a041601d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50456/0 (socket says 172.21.15.32:50456) 2024-04-17T10:01:41.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.345+0000 7f4a0a600640 1 -- 172.21.15.32:0/3815159254 learned_addr learned my addr 172.21.15.32:0/3815159254 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:41.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.345+0000 7f4a09200640 1 -- 172.21.15.32:0/3815159254 >> v1:172.21.15.144:6789/0 conn(0x7f4a04145f30 legacy=0x7f4a04071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:41.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.345+0000 7f4a09200640 1 -- 172.21.15.32:0/3815159254 >> v1:172.21.15.32:6789/0 conn(0x7f4a0415d8d0 legacy=0x7f4a041601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:41.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.345+0000 7f4a09200640 1 -- 172.21.15.32:0/3815159254 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0415d0d0 con 0x7f4a04072500 2024-04-17T10:01:41.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.349+0000 7f4a09200640 1 --2- 172.21.15.32:0/3815159254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a04072500 0x7f4a040cce70 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f49f00097c0 tx=0x7f49f001ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c1211d2bc8d000 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:41.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.349+0000 7f4a03e00640 1 -- 172.21.15.32:0/3815159254 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f00b5070 con 0x7f4a04072500 2024-04-17T10:01:41.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.349+0000 7f4a03e00640 1 -- 172.21.15.32:0/3815159254 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49f00a8990 con 0x7f4a04072500 2024-04-17T10:01:41.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.349+0000 7f4a03e00640 1 -- 172.21.15.32:0/3815159254 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f00a8d60 con 0x7f4a04072500 2024-04-17T10:01:41.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.349+0000 7f4a0c600640 1 -- 172.21.15.32:0/3815159254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a04072500 msgr2=0x7f4a040cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:41.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.349+0000 7f4a0c600640 1 --2- 172.21.15.32:0/3815159254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a04072500 0x7f4a040cce70 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f49f00097c0 tx=0x7f49f001ec90 comp rx=0 tx=0).stop 2024-04-17T10:01:41.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.349+0000 7f4a0c600640 1 -- 172.21.15.32:0/3815159254 shutdown_connections 2024-04-17T10:01:41.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.349+0000 7f4a0c600640 1 --2- 172.21.15.32:0/3815159254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a04072500 0x7f4a040cce70 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:41.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.349+0000 7f4a0c600640 1 -- 172.21.15.32:0/3815159254 >> 172.21.15.32:0/3815159254 conn(0x7f4a0406ba30 msgr2=0x7f4a0406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:41.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.349+0000 7f4a0c600640 1 -- 172.21.15.32:0/3815159254 shutdown_connections 2024-04-17T10:01:41.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.349+0000 7f4a0c600640 1 -- 172.21.15.32:0/3815159254 wait complete. 2024-04-17T10:01:41.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a0c600640 1 Processor -- start 2024-04-17T10:01:41.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a0c600640 1 -- start start 2024-04-17T10:01:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a0c600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a04072500 0x7f4a041e6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a0c600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a04145f30 0x7f4a04140aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a0c600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a0415d8d0 0x7f4a041f5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a0c600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4a040cd800 con 0x7f4a04145f30 2024-04-17T10:01:41.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a0c600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4a040cd680 con 0x7f4a04072500 2024-04-17T10:01:41.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a0c600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4a04146310 con 0x7f4a0415d8d0 2024-04-17T10:01:41.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a09200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a04145f30 0x7f4a04140aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:41.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a09200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a04145f30 0x7f4a04140aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52766/0 (socket says 172.21.15.32:52766) 2024-04-17T10:01:41.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a09200640 1 -- 172.21.15.32:0/165821487 learned_addr learned my addr 172.21.15.32:0/165821487 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:41.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a0a600640 1 --2- 172.21.15.32:0/165821487 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a0415d8d0 0x7f4a041f5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:41.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a09c00640 1 --2- 172.21.15.32:0/165821487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a04072500 0x7f4a041e6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:41.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a09200640 1 -- 172.21.15.32:0/165821487 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a0415d8d0 msgr2=0x7f4a041f5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:41.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a09200640 1 --2- 172.21.15.32:0/165821487 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a0415d8d0 0x7f4a041f5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:41.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a09200640 1 -- 172.21.15.32:0/165821487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a04072500 msgr2=0x7f4a041e6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:41.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a09200640 1 --2- 172.21.15.32:0/165821487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a04072500 0x7f4a041e6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:41.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a09200640 1 -- 172.21.15.32:0/165821487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a04069330 con 0x7f4a04145f30 2024-04-17T10:01:41.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a09200640 1 --2- 172.21.15.32:0/165821487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a04145f30 0x7f4a04140aa0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f49f00b3040 tx=0x7f49f001ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:41.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a01600640 1 -- 172.21.15.32:0/165821487 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f00bf070 con 0x7f4a04145f30 2024-04-17T10:01:41.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a01600640 1 -- 172.21.15.32:0/165821487 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49f00b0900 con 0x7f4a04145f30 2024-04-17T10:01:41.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a01600640 1 -- 172.21.15.32:0/165821487 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f00b5040 con 0x7f4a04145f30 2024-04-17T10:01:41.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a0c600640 1 -- 172.21.15.32:0/165821487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a041f6c70 con 0x7f4a04145f30 2024-04-17T10:01:41.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.353+0000 7f4a0c600640 1 -- 172.21.15.32:0/165821487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a041e6490 con 0x7f4a04145f30 2024-04-17T10:01:41.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.357+0000 7f4a0c600640 1 -- 172.21.15.32:0/165821487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a04068a90 con 0x7f4a04145f30 2024-04-17T10:01:41.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.357+0000 7f4a01600640 1 -- 172.21.15.32:0/165821487 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f49f00b0cf0 con 0x7f4a04145f30 2024-04-17T10:01:41.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.357+0000 7f4a01600640 1 --2- 172.21.15.32:0/165821487 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f49e40683b0 0x7f49e406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:41.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.357+0000 7f4a01600640 1 -- 172.21.15.32:0/165821487 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f49f0123390 con 0x7f4a04145f30 2024-04-17T10:01:41.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.357+0000 7f4a09c00640 1 --2- 172.21.15.32:0/165821487 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f49e40683b0 0x7f49e406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:41.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.357+0000 7f4a09c00640 1 --2- 172.21.15.32:0/165821487 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f49e40683b0 0x7f49e406a870 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f49f8001810 tx=0x7f49f8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:41.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.361+0000 7f4a01600640 1 -- 172.21.15.32:0/165821487 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f49f00187f0 con 0x7f4a04145f30 2024-04-17T10:01:41.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.529+0000 7f4a0c600640 1 -- 172.21.15.32:0/165821487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a04072f90 con 0x7f4a04145f30 2024-04-17T10:01:41.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.529+0000 7f4a01600640 1 -- 172.21.15.32:0/165821487 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+16633 (secure 0 0 0) 0x7f49f00f3260 con 0x7f4a04145f30 2024-04-17T10:01:41.527 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:41.527 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":96,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:01:40.557244+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:01:38.938460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-17T10:01:39.505171+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:01:41.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.533+0000 7f4a0c600640 1 -- 172.21.15.32:0/165821487 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f49e40683b0 msgr2=0x7f49e406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:41.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.533+0000 7f4a0c600640 1 --2- 172.21.15.32:0/165821487 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f49e40683b0 0x7f49e406a870 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f49f8001810 tx=0x7f49f8009040 comp rx=0 tx=0).stop 2024-04-17T10:01:41.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.533+0000 7f4a0c600640 1 -- 172.21.15.32:0/165821487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a04145f30 msgr2=0x7f4a04140aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:41.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.533+0000 7f4a0c600640 1 --2- 172.21.15.32:0/165821487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a04145f30 0x7f4a04140aa0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f49f00b3040 tx=0x7f49f001ba30 comp rx=0 tx=0).stop 2024-04-17T10:01:41.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.533+0000 7f4a0c600640 1 -- 172.21.15.32:0/165821487 shutdown_connections 2024-04-17T10:01:41.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.533+0000 7f4a0c600640 1 --2- 172.21.15.32:0/165821487 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f49e40683b0 0x7f49e406a870 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:41.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.533+0000 7f4a0c600640 1 --2- 172.21.15.32:0/165821487 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a0415d8d0 0x7f4a041f5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:41.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.533+0000 7f4a0c600640 1 --2- 172.21.15.32:0/165821487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a04145f30 0x7f4a04140aa0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:41.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.533+0000 7f4a0c600640 1 --2- 172.21.15.32:0/165821487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a04072500 0x7f4a041e6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:41.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.533+0000 7f4a0c600640 1 -- 172.21.15.32:0/165821487 >> 172.21.15.32:0/165821487 conn(0x7f4a0406ba30 msgr2=0x7f4a04160860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:41.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.533+0000 7f4a0c600640 1 -- 172.21.15.32:0/165821487 shutdown_connections 2024-04-17T10:01:41.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.533+0000 7f4a0c600640 1 -- 172.21.15.32:0/165821487 wait complete. 2024-04-17T10:01:41.549 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:01:41.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.693+0000 7f952a200640 1 Processor -- start 2024-04-17T10:01:41.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.693+0000 7f952a200640 1 -- start start 2024-04-17T10:01:41.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.693+0000 7f952a200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f952406ddf0 0x7f9524070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:41.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.693+0000 7f952a200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9524070dc0 0x7f952409add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:41.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.693+0000 7f952a200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f952406eb40 con 0x7f952409a9f0 2024-04-17T10:01:41.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.693+0000 7f952a200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f95240b4970 con 0x7f9524070dc0 2024-04-17T10:01:41.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.693+0000 7f952a200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f95240b4c50 con 0x7f952406ddf0 2024-04-17T10:01:41.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.693+0000 7f9529800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9524070dc0 0x7f952409add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:41.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.693+0000 7f9529800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9524070dc0 0x7f952409add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52782/0 (socket says 172.21.15.32:52782) 2024-04-17T10:01:41.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.693+0000 7f9529800640 1 -- 172.21.15.32:0/3072834854 learned_addr learned my addr 172.21.15.32:0/3072834854 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:41.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.693+0000 7f951be00640 1 --2- 172.21.15.32:0/3072834854 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f952406ddf0 0x7f9524070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:41.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.693+0000 7f9529800640 1 -- 172.21.15.32:0/3072834854 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f952406ddf0 msgr2=0x7f9524070880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:41.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.693+0000 7f9529800640 1 --2- 172.21.15.32:0/3072834854 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f952406ddf0 0x7f9524070880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:41.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.693+0000 7f9529800640 1 -- 172.21.15.32:0/3072834854 >> v1:172.21.15.144:6789/0 conn(0x7f952409a9f0 legacy=0x7f95240d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:41.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.693+0000 7f9529800640 1 -- 172.21.15.32:0/3072834854 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952406d550 con 0x7f9524070dc0 2024-04-17T10:01:41.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.697+0000 7f9529800640 1 --2- 172.21.15.32:0/3072834854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9524070dc0 0x7f952409add0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f951c009e50 tx=0x7f951c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a612d6c1a559cff9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:41.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.697+0000 7f9523e00640 1 -- 172.21.15.32:0/3072834854 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c0a5700 con 0x7f9524070dc0 2024-04-17T10:01:41.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.697+0000 7f9523e00640 1 -- 172.21.15.32:0/3072834854 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f951c0a58a0 con 0x7f9524070dc0 2024-04-17T10:01:41.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.697+0000 7f9523e00640 1 -- 172.21.15.32:0/3072834854 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c0a5c70 con 0x7f9524070dc0 2024-04-17T10:01:41.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.697+0000 7f952a200640 1 -- 172.21.15.32:0/3072834854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9524070dc0 msgr2=0x7f952409add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:41.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.697+0000 7f952a200640 1 --2- 172.21.15.32:0/3072834854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9524070dc0 0x7f952409add0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f951c009e50 tx=0x7f951c01ca30 comp rx=0 tx=0).stop 2024-04-17T10:01:41.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.697+0000 7f952a200640 1 -- 172.21.15.32:0/3072834854 shutdown_connections 2024-04-17T10:01:41.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.697+0000 7f952a200640 1 --2- 172.21.15.32:0/3072834854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9524070dc0 0x7f952409add0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:41.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.697+0000 7f952a200640 1 --2- 172.21.15.32:0/3072834854 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f952406ddf0 0x7f9524070880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:41.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.697+0000 7f952a200640 1 -- 172.21.15.32:0/3072834854 >> 172.21.15.32:0/3072834854 conn(0x7f95240b3e50 msgr2=0x7f95240b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:41.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.697+0000 7f952a200640 1 -- 172.21.15.32:0/3072834854 shutdown_connections 2024-04-17T10:01:41.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.697+0000 7f952a200640 1 -- 172.21.15.32:0/3072834854 wait complete. 2024-04-17T10:01:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f952a200640 1 Processor -- start 2024-04-17T10:01:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f952a200640 1 -- start start 2024-04-17T10:01:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f952a200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f952406ddf0 0x7f95241535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f952a200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9524070dc0 0x7f9524159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f952a200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f952409a9f0 0x7f952415bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f952a200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f95240704f0 con 0x7f9524070dc0 2024-04-17T10:01:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f952a200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9524070370 con 0x7f952406ddf0 2024-04-17T10:01:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f952a200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f952416bf50 con 0x7f952409a9f0 2024-04-17T10:01:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f951be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9524070dc0 0x7f9524159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f951be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9524070dc0 0x7f9524159000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52790/0 (socket says 172.21.15.32:52790) 2024-04-17T10:01:41.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f951be00640 1 -- 172.21.15.32:0/853812103 learned_addr learned my addr 172.21.15.32:0/853812103 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:41.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f9528e00640 1 --2- 172.21.15.32:0/853812103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f952406ddf0 0x7f95241535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:41.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f9529800640 1 --2- 172.21.15.32:0/853812103 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f952409a9f0 0x7f952415bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:41.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f951be00640 1 -- 172.21.15.32:0/853812103 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f952409a9f0 msgr2=0x7f952415bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:41.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f951be00640 1 --2- 172.21.15.32:0/853812103 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f952409a9f0 0x7f952415bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:41.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f951be00640 1 -- 172.21.15.32:0/853812103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f952406ddf0 msgr2=0x7f95241535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:41.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f951be00640 1 --2- 172.21.15.32:0/853812103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f952406ddf0 0x7f95241535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:41.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f951be00640 1 -- 172.21.15.32:0/853812103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9524159a80 con 0x7f9524070dc0 2024-04-17T10:01:41.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f951be00640 1 --2- 172.21.15.32:0/853812103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9524070dc0 0x7f9524159000 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f950c00e870 tx=0x7f950c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:41.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f9521600640 1 -- 172.21.15.32:0/853812103 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c017070 con 0x7f9524070dc0 2024-04-17T10:01:41.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f9521600640 1 -- 172.21.15.32:0/853812103 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f950c00c940 con 0x7f9524070dc0 2024-04-17T10:01:41.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f952a200640 1 -- 172.21.15.32:0/853812103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95241546d0 con 0x7f9524070dc0 2024-04-17T10:01:41.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f9521600640 1 -- 172.21.15.32:0/853812103 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c00ccf0 con 0x7f9524070dc0 2024-04-17T10:01:41.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.701+0000 7f952a200640 1 -- 172.21.15.32:0/853812103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95241540c0 con 0x7f9524070dc0 2024-04-17T10:01:41.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.705+0000 7f952a200640 1 -- 172.21.15.32:0/853812103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f952404fd00 con 0x7f9524070dc0 2024-04-17T10:01:41.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.705+0000 7f9521600640 1 -- 172.21.15.32:0/853812103 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f950c01a070 con 0x7f9524070dc0 2024-04-17T10:01:41.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.705+0000 7f9521600640 1 --2- 172.21.15.32:0/853812103 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9500068380 0x7f950006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:41.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.705+0000 7f9521600640 1 -- 172.21.15.32:0/853812103 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f950c086c80 con 0x7f9524070dc0 2024-04-17T10:01:41.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.705+0000 7f9528e00640 1 --2- 172.21.15.32:0/853812103 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9500068380 0x7f950006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:41.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.705+0000 7f9528e00640 1 --2- 172.21.15.32:0/853812103 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9500068380 0x7f950006a840 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9514001770 tx=0x7f9514009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:41.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.709+0000 7f9521600640 1 -- 172.21.15.32:0/853812103 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f950c056b50 con 0x7f9524070dc0 2024-04-17T10:01:41.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.877+0000 7f952a200640 1 -- 172.21.15.32:0/853812103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95240d20d0 con 0x7f9524070dc0 2024-04-17T10:01:41.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.877+0000 7f9521600640 1 -- 172.21.15.32:0/853812103 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+16633 (secure 0 0 0) 0x7f950c05ae50 con 0x7f9524070dc0 2024-04-17T10:01:41.874 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:41.874 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":96,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:01:40.557244+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:01:38.938460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-17T10:01:39.505171+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:01:41.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.877+0000 7f952a200640 1 -- 172.21.15.32:0/853812103 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9500068380 msgr2=0x7f950006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:41.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.877+0000 7f952a200640 1 --2- 172.21.15.32:0/853812103 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9500068380 0x7f950006a840 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9514001770 tx=0x7f9514009040 comp rx=0 tx=0).stop 2024-04-17T10:01:41.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.877+0000 7f952a200640 1 -- 172.21.15.32:0/853812103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9524070dc0 msgr2=0x7f9524159000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:41.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.877+0000 7f952a200640 1 --2- 172.21.15.32:0/853812103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9524070dc0 0x7f9524159000 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f950c00e870 tx=0x7f950c00ec70 comp rx=0 tx=0).stop 2024-04-17T10:01:41.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.877+0000 7f952a200640 1 -- 172.21.15.32:0/853812103 shutdown_connections 2024-04-17T10:01:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.877+0000 7f952a200640 1 --2- 172.21.15.32:0/853812103 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9500068380 0x7f950006a840 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.877+0000 7f952a200640 1 --2- 172.21.15.32:0/853812103 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f952409a9f0 0x7f952415bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.877+0000 7f952a200640 1 --2- 172.21.15.32:0/853812103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9524070dc0 0x7f9524159000 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.877+0000 7f952a200640 1 --2- 172.21.15.32:0/853812103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f952406ddf0 0x7f95241535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.877+0000 7f952a200640 1 -- 172.21.15.32:0/853812103 >> 172.21.15.32:0/853812103 conn(0x7f95240b3e50 msgr2=0x7f9524159e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.877+0000 7f952a200640 1 -- 172.21.15.32:0/853812103 shutdown_connections 2024-04-17T10:01:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:41.877+0000 7f952a200640 1 -- 172.21.15.32:0/853812103 wait complete. 2024-04-17T10:01:41.895 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T10:01:42.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.041+0000 7f988a000640 1 Processor -- start 2024-04-17T10:01:42.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.041+0000 7f988a000640 1 -- start start 2024-04-17T10:01:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.041+0000 7f988a000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f988406ba30 0x7f988406be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.041+0000 7f988a000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9884071e60 0x7f98840722b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.041+0000 7f988a000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f988406c350 con 0x7f98840cf370 2024-04-17T10:01:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.041+0000 7f988a000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f98840cfef0 con 0x7f9884071e60 2024-04-17T10:01:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.041+0000 7f988a000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f98840d0130 con 0x7f988406ba30 2024-04-17T10:01:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.045+0000 7f9883e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9884071e60 0x7f98840722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.045+0000 7f9883e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9884071e60 0x7f98840722b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52794/0 (socket says 172.21.15.32:52794) 2024-04-17T10:01:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.045+0000 7f9883e00640 1 -- 172.21.15.32:0/3402268851 learned_addr learned my addr 172.21.15.32:0/3402268851 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.045+0000 7f9883400640 1 --2- 172.21.15.32:0/3402268851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f988406ba30 0x7f988406be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.045+0000 7f9883e00640 1 -- 172.21.15.32:0/3402268851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f988406ba30 msgr2=0x7f988406be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.045+0000 7f9883e00640 1 --2- 172.21.15.32:0/3402268851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f988406ba30 0x7f988406be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.045+0000 7f9883e00640 1 -- 172.21.15.32:0/3402268851 >> v1:172.21.15.144:6789/0 conn(0x7f98840cf370 legacy=0x7f98840cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.045+0000 7f9883e00640 1 -- 172.21.15.32:0/3402268851 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f988415d090 con 0x7f9884071e60 2024-04-17T10:01:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.045+0000 7f9883400640 1 --2- 172.21.15.32:0/3402268851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f988406ba30 0x7f988406be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.045+0000 7f9883e00640 1 --2- 172.21.15.32:0/3402268851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9884071e60 0x7f98840722b0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f987c009e50 tx=0x7f987c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36a74c9368d1d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.045+0000 7f9882a00640 1 -- 172.21.15.32:0/3402268851 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c0a5840 con 0x7f9884071e60 2024-04-17T10:01:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.045+0000 7f9882a00640 1 -- 172.21.15.32:0/3402268851 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f987c0a59e0 con 0x7f9884071e60 2024-04-17T10:01:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.045+0000 7f9882a00640 1 -- 172.21.15.32:0/3402268851 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c0a5db0 con 0x7f9884071e60 2024-04-17T10:01:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f988a000640 1 -- 172.21.15.32:0/3402268851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9884071e60 msgr2=0x7f98840722b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f988a000640 1 --2- 172.21.15.32:0/3402268851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9884071e60 0x7f98840722b0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f987c009e50 tx=0x7f987c01c9c0 comp rx=0 tx=0).stop 2024-04-17T10:01:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f988a000640 1 -- 172.21.15.32:0/3402268851 shutdown_connections 2024-04-17T10:01:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f988a000640 1 --2- 172.21.15.32:0/3402268851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9884071e60 0x7f98840722b0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f988a000640 1 --2- 172.21.15.32:0/3402268851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f988406ba30 0x7f988406be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f988a000640 1 -- 172.21.15.32:0/3402268851 >> 172.21.15.32:0/3402268851 conn(0x7f98840b2120 msgr2=0x7f98840b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f988a000640 1 -- 172.21.15.32:0/3402268851 shutdown_connections 2024-04-17T10:01:42.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f988a000640 1 -- 172.21.15.32:0/3402268851 wait complete. 2024-04-17T10:01:42.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f988a000640 1 Processor -- start 2024-04-17T10:01:42.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f988a000640 1 -- start start 2024-04-17T10:01:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f988a000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f988406ba30 0x7f9884159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f988a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9884071e60 0x7f9884144da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f988a000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98840cf370 0x7f988413d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f988a000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f988415dcf0 con 0x7f9884071e60 2024-04-17T10:01:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f988a000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f988415db70 con 0x7f988406ba30 2024-04-17T10:01:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f988a000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9884146230 con 0x7f98840cf370 2024-04-17T10:01:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f987be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9884071e60 0x7f9884144da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f987be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9884071e60 0x7f9884144da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52796/0 (socket says 172.21.15.32:52796) 2024-04-17T10:01:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f987be00640 1 -- 172.21.15.32:0/42736362 learned_addr learned my addr 172.21.15.32:0/42736362 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f9883400640 1 --2- 172.21.15.32:0/42736362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f988406ba30 0x7f9884159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f9883e00640 1 --2- 172.21.15.32:0/42736362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98840cf370 0x7f988413d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f987be00640 1 -- 172.21.15.32:0/42736362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98840cf370 msgr2=0x7f988413d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f987be00640 1 --2- 172.21.15.32:0/42736362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98840cf370 0x7f988413d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f987be00640 1 -- 172.21.15.32:0/42736362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f988406ba30 msgr2=0x7f9884159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f987be00640 1 --2- 172.21.15.32:0/42736362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f988406ba30 0x7f9884159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.049+0000 7f987be00640 1 -- 172.21.15.32:0/42736362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9884159f60 con 0x7f9884071e60 2024-04-17T10:01:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.053+0000 7f987be00640 1 --2- 172.21.15.32:0/42736362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9884071e60 0x7f9884144da0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f986c00b410 tx=0x7f986c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:42.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.053+0000 7f987b400640 1 -- 172.21.15.32:0/42736362 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c015030 con 0x7f9884071e60 2024-04-17T10:01:42.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.053+0000 7f987b400640 1 -- 172.21.15.32:0/42736362 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f986c004020 con 0x7f9884071e60 2024-04-17T10:01:42.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.053+0000 7f988a000640 1 -- 172.21.15.32:0/42736362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f988415a240 con 0x7f9884071e60 2024-04-17T10:01:42.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.053+0000 7f987b400640 1 -- 172.21.15.32:0/42736362 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c004450 con 0x7f9884071e60 2024-04-17T10:01:42.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.053+0000 7f988a000640 1 -- 172.21.15.32:0/42736362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f988415e120 con 0x7f9884071e60 2024-04-17T10:01:42.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.053+0000 7f988a000640 1 -- 172.21.15.32:0/42736362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9884068a90 con 0x7f9884071e60 2024-04-17T10:01:42.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.053+0000 7f987b400640 1 -- 172.21.15.32:0/42736362 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f986c018070 con 0x7f9884071e60 2024-04-17T10:01:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.053+0000 7f987b400640 1 --2- 172.21.15.32:0/42736362 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9860068380 0x7f986006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.053+0000 7f987b400640 1 -- 172.21.15.32:0/42736362 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f986c0864b0 con 0x7f9884071e60 2024-04-17T10:01:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.057+0000 7f9883400640 1 --2- 172.21.15.32:0/42736362 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9860068380 0x7f986006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:42.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.057+0000 7f9883400640 1 --2- 172.21.15.32:0/42736362 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9860068380 0x7f986006a840 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f987400ab60 tx=0x7f9874008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.057+0000 7f987b400640 1 -- 172.21.15.32:0/42736362 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f986c056380 con 0x7f9884071e60 2024-04-17T10:01:42.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:42.265+0000 7f988a000640 1 -- 172.21.15.32:0/42736362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f98840b1f00 con 0x7f9884071e60 2024-04-17T10:01:43.113 INFO:tasks.ceph.mds.c.smithi032.stderr:2024-04-17T10:01:43.117+0000 7fd06f000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:01:43.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.169+0000 7f987b400640 1 -- 172.21.15.32:0/42736362 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v80) ==== 118+0+0 (secure 0 0 0) 0x7f986c05a680 con 0x7f9884071e60 2024-04-17T10:01:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.173+0000 7f988a000640 1 -- 172.21.15.32:0/42736362 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9860068380 msgr2=0x7f986006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.173+0000 7f988a000640 1 --2- 172.21.15.32:0/42736362 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9860068380 0x7f986006a840 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f987400ab60 tx=0x7f9874008040 comp rx=0 tx=0).stop 2024-04-17T10:01:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.173+0000 7f988a000640 1 -- 172.21.15.32:0/42736362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9884071e60 msgr2=0x7f9884144da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.173+0000 7f988a000640 1 --2- 172.21.15.32:0/42736362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9884071e60 0x7f9884144da0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f986c00b410 tx=0x7f986c00b8d0 comp rx=0 tx=0).stop 2024-04-17T10:01:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.173+0000 7f988a000640 1 -- 172.21.15.32:0/42736362 shutdown_connections 2024-04-17T10:01:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.173+0000 7f988a000640 1 --2- 172.21.15.32:0/42736362 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9860068380 0x7f986006a840 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.173+0000 7f988a000640 1 --2- 172.21.15.32:0/42736362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98840cf370 0x7f988413d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.173+0000 7f988a000640 1 --2- 172.21.15.32:0/42736362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9884071e60 0x7f9884144da0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.173+0000 7f988a000640 1 --2- 172.21.15.32:0/42736362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f988406ba30 0x7f9884159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.173+0000 7f988a000640 1 -- 172.21.15.32:0/42736362 >> 172.21.15.32:0/42736362 conn(0x7f98840b2120 msgr2=0x7f98840cdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.173+0000 7f988a000640 1 -- 172.21.15.32:0/42736362 shutdown_connections 2024-04-17T10:01:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.173+0000 7f988a000640 1 -- 172.21.15.32:0/42736362 wait complete. 2024-04-17T10:01:43.193 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-17T10:01:43.193 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-17T10:01:43.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.349+0000 7f6845a00640 1 Processor -- start 2024-04-17T10:01:43.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.349+0000 7f6845a00640 1 -- start start 2024-04-17T10:01:43.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.349+0000 7f6845a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f684015ac30 0x7f684006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:43.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.349+0000 7f6845a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684015b010 0x7f684006dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:43.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.349+0000 7f6845a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f684006fcd0 con 0x7f68400b3d80 2024-04-17T10:01:43.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.349+0000 7f6845a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f684006f300 con 0x7f684015ac30 2024-04-17T10:01:43.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.349+0000 7f6845a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f684006e6f0 con 0x7f684015b010 2024-04-17T10:01:43.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.349+0000 7f683ea00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f684015ac30 0x7f684006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:43.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.349+0000 7f683ea00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f684015ac30 0x7f684006f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:47060/0 (socket says 172.21.15.32:47060) 2024-04-17T10:01:43.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.349+0000 7f6836000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684015b010 0x7f684006dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:43.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.349+0000 7f683ea00640 1 -- 172.21.15.32:0/71300383 learned_addr learned my addr 172.21.15.32:0/71300383 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:43.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.349+0000 7f683f400640 1 --1- 172.21.15.32:0/71300383 >> v1:172.21.15.144:6789/0 conn(0x7f68400b3d80 0x7f6840071e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:55454/0 (socket says 172.21.15.32:55454) 2024-04-17T10:01:43.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.349+0000 7f6836000640 1 -- 172.21.15.32:0/71300383 >> v2:172.21.15.144:3300/0 conn(0x7f684015ac30 msgr2=0x7f684006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:43.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.349+0000 7f6836000640 1 --2- 172.21.15.32:0/71300383 >> v2:172.21.15.144:3300/0 conn(0x7f684015ac30 0x7f684006f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:43.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.349+0000 7f6836000640 1 -- 172.21.15.32:0/71300383 >> v1:172.21.15.144:6789/0 conn(0x7f68400b3d80 legacy=0x7f6840071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:43.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.349+0000 7f6836000640 1 -- 172.21.15.32:0/71300383 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f684006e9d0 con 0x7f684015b010 2024-04-17T10:01:43.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.349+0000 7f683e000640 1 -- 172.21.15.32:0/71300383 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2655686211 0 0) 0x7f684006fcd0 con 0x7f68400b3d80 2024-04-17T10:01:43.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.349+0000 7f683ea00640 1 --2- 172.21.15.32:0/71300383 >> v2:172.21.15.144:3300/0 conn(0x7f684015ac30 0x7f684006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:01:43.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.353+0000 7f6836000640 1 --2- 172.21.15.32:0/71300383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684015b010 0x7f684006dea0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f68280097c0 tx=0x7f682801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=980f7b076070ce30 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:43.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.353+0000 7f683e000640 1 -- 172.21.15.32:0/71300383 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68280aa600 con 0x7f684015b010 2024-04-17T10:01:43.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.353+0000 7f683e000640 1 -- 172.21.15.32:0/71300383 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68280af070 con 0x7f684015b010 2024-04-17T10:01:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.353+0000 7f683e000640 1 -- 172.21.15.32:0/71300383 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68280aa9c0 con 0x7f684015b010 2024-04-17T10:01:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.353+0000 7f6845a00640 1 -- 172.21.15.32:0/71300383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684015b010 msgr2=0x7f684006dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.353+0000 7f6845a00640 1 --2- 172.21.15.32:0/71300383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684015b010 0x7f684006dea0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f68280097c0 tx=0x7f682801c4a0 comp rx=0 tx=0).stop 2024-04-17T10:01:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.353+0000 7f6845a00640 1 -- 172.21.15.32:0/71300383 shutdown_connections 2024-04-17T10:01:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.353+0000 7f6845a00640 1 --2- 172.21.15.32:0/71300383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684015b010 0x7f684006dea0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.353+0000 7f6845a00640 1 --2- 172.21.15.32:0/71300383 >> v2:172.21.15.144:3300/0 conn(0x7f684015ac30 0x7f684006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.353+0000 7f6845a00640 1 -- 172.21.15.32:0/71300383 >> 172.21.15.32:0/71300383 conn(0x7f68400b2120 msgr2=0x7f68400b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.353+0000 7f6845a00640 1 -- 172.21.15.32:0/71300383 shutdown_connections 2024-04-17T10:01:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.353+0000 7f6845a00640 1 -- 172.21.15.32:0/71300383 wait complete. 2024-04-17T10:01:43.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f6845a00640 1 Processor -- start 2024-04-17T10:01:43.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f6845a00640 1 -- start start 2024-04-17T10:01:43.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f6845a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68400b3d80 0x7f6840157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:43.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f6845a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684015ac30 0x7f68400ad090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:43.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f6845a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684015b010 0x7f68400ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:43.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f6845a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f68400726f0 con 0x7f68400b3d80 2024-04-17T10:01:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f6845a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6840072570 con 0x7f684015b010 2024-04-17T10:01:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f6845a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f68400ce1d0 con 0x7f684015ac30 2024-04-17T10:01:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f683ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68400b3d80 0x7f6840157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f683ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68400b3d80 0x7f6840157790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52804/0 (socket says 172.21.15.32:52804) 2024-04-17T10:01:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f683ea00640 1 -- 172.21.15.32:0/769086114 learned_addr learned my addr 172.21.15.32:0/769086114 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f6836000640 1 --2- 172.21.15.32:0/769086114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684015ac30 0x7f68400ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f683f400640 1 --2- 172.21.15.32:0/769086114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684015b010 0x7f68400ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f683ea00640 1 -- 172.21.15.32:0/769086114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684015ac30 msgr2=0x7f68400ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f683ea00640 1 --2- 172.21.15.32:0/769086114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684015ac30 0x7f68400ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f683ea00640 1 -- 172.21.15.32:0/769086114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684015b010 msgr2=0x7f68400ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f683ea00640 1 --2- 172.21.15.32:0/769086114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684015b010 0x7f68400ad5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f683ea00640 1 -- 172.21.15.32:0/769086114 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6840157cd0 con 0x7f68400b3d80 2024-04-17T10:01:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f683ea00640 1 --2- 172.21.15.32:0/769086114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68400b3d80 0x7f6840157790 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f683000e9d0 tx=0x7f683000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f6837400640 1 -- 172.21.15.32:0/769086114 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6830017070 con 0x7f68400b3d80 2024-04-17T10:01:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f6837400640 1 -- 172.21.15.32:0/769086114 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f683000ca00 con 0x7f68400b3d80 2024-04-17T10:01:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f6837400640 1 -- 172.21.15.32:0/769086114 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683000cdb0 con 0x7f68400b3d80 2024-04-17T10:01:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f6845a00640 1 -- 172.21.15.32:0/769086114 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68400adbc0 con 0x7f68400b3d80 2024-04-17T10:01:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.357+0000 7f6845a00640 1 -- 172.21.15.32:0/769086114 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68400ae180 con 0x7f68400b3d80 2024-04-17T10:01:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.361+0000 7f6845a00640 1 -- 172.21.15.32:0/769086114 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6840068a90 con 0x7f68400b3d80 2024-04-17T10:01:43.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.361+0000 7f6837400640 1 -- 172.21.15.32:0/769086114 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f683001a070 con 0x7f68400b3d80 2024-04-17T10:01:43.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.361+0000 7f6837400640 1 --2- 172.21.15.32:0/769086114 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f681c068380 0x7f681c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:43.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.361+0000 7f6837400640 1 -- 172.21.15.32:0/769086114 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f6830086830 con 0x7f68400b3d80 2024-04-17T10:01:43.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.361+0000 7f6836000640 1 --2- 172.21.15.32:0/769086114 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f681c068380 0x7f681c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:43.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.361+0000 7f6836000640 1 --2- 172.21.15.32:0/769086114 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f681c068380 0x7f681c06a840 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f68280ae000 tx=0x7f68280a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:43.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.365+0000 7f6837400640 1 -- 172.21.15.32:0/769086114 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f6830056790 con 0x7f68400b3d80 2024-04-17T10:01:43.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:43.565+0000 7f6845a00640 1 -- 172.21.15.32:0/769086114 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f6840072a60 con 0x7f68400b3d80 2024-04-17T10:01:44.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.177+0000 7f6837400640 1 -- 172.21.15.32:0/769086114 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v81) ==== 122+0+0 (secure 0 0 0) 0x7f683005aa90 con 0x7f68400b3d80 2024-04-17T10:01:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.181+0000 7f6845a00640 1 -- 172.21.15.32:0/769086114 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f681c068380 msgr2=0x7f681c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.181+0000 7f6845a00640 1 --2- 172.21.15.32:0/769086114 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f681c068380 0x7f681c06a840 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f68280ae000 tx=0x7f68280a7000 comp rx=0 tx=0).stop 2024-04-17T10:01:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.181+0000 7f6845a00640 1 -- 172.21.15.32:0/769086114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68400b3d80 msgr2=0x7f6840157790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.181+0000 7f6845a00640 1 --2- 172.21.15.32:0/769086114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68400b3d80 0x7f6840157790 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f683000e9d0 tx=0x7f683000ee90 comp rx=0 tx=0).stop 2024-04-17T10:01:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.181+0000 7f6845a00640 1 -- 172.21.15.32:0/769086114 shutdown_connections 2024-04-17T10:01:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.181+0000 7f6845a00640 1 --2- 172.21.15.32:0/769086114 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f681c068380 0x7f681c06a840 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.181+0000 7f6845a00640 1 --2- 172.21.15.32:0/769086114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684015ac30 0x7f68400ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.181+0000 7f6845a00640 1 --2- 172.21.15.32:0/769086114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684015b010 0x7f68400ad5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.181+0000 7f6845a00640 1 --2- 172.21.15.32:0/769086114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68400b3d80 0x7f6840157790 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.181+0000 7f6845a00640 1 -- 172.21.15.32:0/769086114 >> 172.21.15.32:0/769086114 conn(0x7f68400b2120 msgr2=0x7f68400b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.181+0000 7f6845a00640 1 -- 172.21.15.32:0/769086114 shutdown_connections 2024-04-17T10:01:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.181+0000 7f6845a00640 1 -- 172.21.15.32:0/769086114 wait complete. 2024-04-17T10:01:44.191 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:01:44.194 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:01:44.202 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-17T10:01:44.205 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:01:44.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.433+0000 7ff291c00640 1 Processor -- start 2024-04-17T10:01:44.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.433+0000 7ff291c00640 1 -- start start 2024-04-17T10:01:44.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.433+0000 7ff291c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff28c144f20 0x7ff28c06e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:44.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.433+0000 7ff291c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff28c06eda0 0x7ff28c071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:44.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.433+0000 7ff291c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff28c1481e0 0x7ff28c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:44.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.433+0000 7ff291c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff28c0aed20 con 0x7ff28c1481e0 2024-04-17T10:01:44.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.433+0000 7ff291c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff28c0af000 con 0x7ff28c06eda0 2024-04-17T10:01:44.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.433+0000 7ff291c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff28c0af2e0 con 0x7ff28c144f20 2024-04-17T10:01:44.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.437+0000 7ff28b400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff28c1481e0 0x7ff28c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:44.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.437+0000 7ff28b400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff28c1481e0 0x7ff28c072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52812/0 (socket says 172.21.15.32:52812) 2024-04-17T10:01:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.437+0000 7ff28b400640 1 -- 172.21.15.32:0/2080110036 learned_addr learned my addr 172.21.15.32:0/2080110036 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.437+0000 7ff283e00640 1 --2- 172.21.15.32:0/2080110036 >> v2:172.21.15.144:3300/0 conn(0x7ff28c06eda0 0x7ff28c071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.437+0000 7ff28aa00640 1 --2- 172.21.15.32:0/2080110036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff28c144f20 0x7ff28c06e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.437+0000 7ff28b400640 1 -- 172.21.15.32:0/2080110036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff28c144f20 msgr2=0x7ff28c06e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.437+0000 7ff28b400640 1 --2- 172.21.15.32:0/2080110036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff28c144f20 0x7ff28c06e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.437+0000 7ff28b400640 1 -- 172.21.15.32:0/2080110036 >> v2:172.21.15.144:3300/0 conn(0x7ff28c06eda0 msgr2=0x7ff28c071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.437+0000 7ff28b400640 1 --2- 172.21.15.32:0/2080110036 >> v2:172.21.15.144:3300/0 conn(0x7ff28c06eda0 0x7ff28c071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.437+0000 7ff28b400640 1 -- 172.21.15.32:0/2080110036 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28c0af5c0 con 0x7ff28c1481e0 2024-04-17T10:01:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.437+0000 7ff283e00640 1 --2- 172.21.15.32:0/2080110036 >> v2:172.21.15.144:3300/0 conn(0x7ff28c06eda0 0x7ff28c071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:44.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.437+0000 7ff28aa00640 1 --2- 172.21.15.32:0/2080110036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff28c144f20 0x7ff28c06e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:44.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.437+0000 7ff28b400640 1 --2- 172.21.15.32:0/2080110036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff28c1481e0 0x7ff28c072330 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7ff28c06cac0 tx=0x7ff28401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b468eca9d7795b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:44.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.437+0000 7ff28a000640 1 -- 172.21.15.32:0/2080110036 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2840a5680 con 0x7ff28c1481e0 2024-04-17T10:01:44.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.437+0000 7ff28a000640 1 -- 172.21.15.32:0/2080110036 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2840a5820 con 0x7ff28c1481e0 2024-04-17T10:01:44.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.437+0000 7ff28a000640 1 -- 172.21.15.32:0/2080110036 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2840a5bf0 con 0x7ff28c1481e0 2024-04-17T10:01:44.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff291c00640 1 -- 172.21.15.32:0/2080110036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff28c1481e0 msgr2=0x7ff28c072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:44.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff291c00640 1 --2- 172.21.15.32:0/2080110036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff28c1481e0 0x7ff28c072330 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7ff28c06cac0 tx=0x7ff28401c9f0 comp rx=0 tx=0).stop 2024-04-17T10:01:44.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff291c00640 1 -- 172.21.15.32:0/2080110036 shutdown_connections 2024-04-17T10:01:44.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff291c00640 1 --2- 172.21.15.32:0/2080110036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff28c1481e0 0x7ff28c072330 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:44.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff291c00640 1 --2- 172.21.15.32:0/2080110036 >> v2:172.21.15.144:3300/0 conn(0x7ff28c06eda0 0x7ff28c071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:44.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff291c00640 1 --2- 172.21.15.32:0/2080110036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff28c144f20 0x7ff28c06e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:44.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff291c00640 1 -- 172.21.15.32:0/2080110036 >> 172.21.15.32:0/2080110036 conn(0x7ff28c06ba30 msgr2=0x7ff28c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:44.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff291c00640 1 -- 172.21.15.32:0/2080110036 shutdown_connections 2024-04-17T10:01:44.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff291c00640 1 -- 172.21.15.32:0/2080110036 wait complete. 2024-04-17T10:01:44.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff291c00640 1 Processor -- start 2024-04-17T10:01:44.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff291c00640 1 -- start start 2024-04-17T10:01:44.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff291c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff28c06eda0 0x7ff28c141a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:44.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff291c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff28c144f20 0x7ff28c0b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:44.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff291c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff28c1481e0 0x7ff28c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:44.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff291c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c162fe0 con 0x7ff28c1481e0 2024-04-17T10:01:44.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff291c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c162e60 con 0x7ff28c144f20 2024-04-17T10:01:44.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff291c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff28c163160 con 0x7ff28c06eda0 2024-04-17T10:01:44.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff28b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff28c1481e0 0x7ff28c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:44.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff28b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff28c1481e0 0x7ff28c0b4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52818/0 (socket says 172.21.15.32:52818) 2024-04-17T10:01:44.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff28b400640 1 -- 172.21.15.32:0/852226622 learned_addr learned my addr 172.21.15.32:0/852226622 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:44.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff28aa00640 1 --2- 172.21.15.32:0/852226622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff28c06eda0 0x7ff28c141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:44.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff283e00640 1 --2- 172.21.15.32:0/852226622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff28c144f20 0x7ff28c0b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:44.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff28b400640 1 -- 172.21.15.32:0/852226622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff28c06eda0 msgr2=0x7ff28c141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:44.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff28b400640 1 --2- 172.21.15.32:0/852226622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff28c06eda0 0x7ff28c141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:44.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff28b400640 1 -- 172.21.15.32:0/852226622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff28c144f20 msgr2=0x7ff28c0b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:44.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff28b400640 1 --2- 172.21.15.32:0/852226622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff28c144f20 0x7ff28c0b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:44.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff28b400640 1 -- 172.21.15.32:0/852226622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28c157d50 con 0x7ff28c1481e0 2024-04-17T10:01:44.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.441+0000 7ff28aa00640 1 --2- 172.21.15.32:0/852226622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff28c06eda0 0x7ff28c141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:44.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.445+0000 7ff28b400640 1 --2- 172.21.15.32:0/852226622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff28c1481e0 0x7ff28c0b4d00 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7ff28401d290 tx=0x7ff284019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:44.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.445+0000 7ff282a00640 1 -- 172.21.15.32:0/852226622 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2840b8670 con 0x7ff28c1481e0 2024-04-17T10:01:44.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.445+0000 7ff282a00640 1 -- 172.21.15.32:0/852226622 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2840be070 con 0x7ff28c1481e0 2024-04-17T10:01:44.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.445+0000 7ff291c00640 1 -- 172.21.15.32:0/852226622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28c158030 con 0x7ff28c1481e0 2024-04-17T10:01:44.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.445+0000 7ff282a00640 1 -- 172.21.15.32:0/852226622 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2840b8a90 con 0x7ff28c1481e0 2024-04-17T10:01:44.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.445+0000 7ff291c00640 1 -- 172.21.15.32:0/852226622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff28c1585f0 con 0x7ff28c1481e0 2024-04-17T10:01:44.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.445+0000 7ff291c00640 1 -- 172.21.15.32:0/852226622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff28c068a90 con 0x7ff28c1481e0 2024-04-17T10:01:44.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.445+0000 7ff282a00640 1 -- 172.21.15.32:0/852226622 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff2840b8c30 con 0x7ff28c1481e0 2024-04-17T10:01:44.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.445+0000 7ff282a00640 1 --2- 172.21.15.32:0/852226622 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff2680683b0 0x7ff26806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:44.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.445+0000 7ff282a00640 1 -- 172.21.15.32:0/852226622 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7ff284121b90 con 0x7ff28c1481e0 2024-04-17T10:01:44.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.445+0000 7ff28aa00640 1 --2- 172.21.15.32:0/852226622 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff2680683b0 0x7ff26806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:44.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.449+0000 7ff28aa00640 1 --2- 172.21.15.32:0/852226622 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff2680683b0 0x7ff26806a870 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff27c006fd0 tx=0x7ff27c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:44.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.449+0000 7ff282a00640 1 -- 172.21.15.32:0/852226622 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff2840165e0 con 0x7ff28c1481e0 2024-04-17T10:01:44.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.649+0000 7ff291c00640 1 -- 172.21.15.32:0/852226622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff28c06f530 con 0x7ff28c1481e0 2024-04-17T10:01:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.649+0000 7ff282a00640 1 -- 172.21.15.32:0/852226622 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5207 (secure 0 0 0) 0x7ff2840ac3a0 con 0x7ff28c1481e0 2024-04-17T10:01:44.646 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:44.646 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":81,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2635109588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2635109588},{"type":"v1","addr":"172.21.15.144:6835","nonce":2635109588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7789,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3258455939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3258455939},{"type":"v1","addr":"172.21.15.144:6834","nonce":3258455939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":8574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3390468332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3390468332},{"type":"v1","addr":"172.21.15.32:6836","nonce":3390468332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:01:40.557208+0000","modified":"2024-04-17T10:01:43.570474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8577":{"gid":8577,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/1235126741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1235126741},{"type":"v1","addr":"172.21.15.32:6837","nonce":1235126741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8577,"qdb_cluster":[8577]},"id":7}]} 2024-04-17T10:01:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 81 2024-04-17T10:01:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.649+0000 7ff291c00640 1 -- 172.21.15.32:0/852226622 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff2680683b0 msgr2=0x7ff26806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.649+0000 7ff291c00640 1 --2- 172.21.15.32:0/852226622 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff2680683b0 0x7ff26806a870 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff27c006fd0 tx=0x7ff27c008040 comp rx=0 tx=0).stop 2024-04-17T10:01:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.649+0000 7ff291c00640 1 -- 172.21.15.32:0/852226622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff28c1481e0 msgr2=0x7ff28c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.649+0000 7ff291c00640 1 --2- 172.21.15.32:0/852226622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff28c1481e0 0x7ff28c0b4d00 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7ff28401d290 tx=0x7ff284019790 comp rx=0 tx=0).stop 2024-04-17T10:01:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.649+0000 7ff291c00640 1 -- 172.21.15.32:0/852226622 shutdown_connections 2024-04-17T10:01:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.649+0000 7ff291c00640 1 --2- 172.21.15.32:0/852226622 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff2680683b0 0x7ff26806a870 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.649+0000 7ff291c00640 1 --2- 172.21.15.32:0/852226622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff28c1481e0 0x7ff28c0b4d00 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.649+0000 7ff291c00640 1 --2- 172.21.15.32:0/852226622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff28c144f20 0x7ff28c0b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.649+0000 7ff291c00640 1 --2- 172.21.15.32:0/852226622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff28c06eda0 0x7ff28c141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.649+0000 7ff291c00640 1 -- 172.21.15.32:0/852226622 >> 172.21.15.32:0/852226622 conn(0x7ff28c06ba30 msgr2=0x7ff28c1434f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.649+0000 7ff291c00640 1 -- 172.21.15.32:0/852226622 shutdown_connections 2024-04-17T10:01:44.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.649+0000 7ff291c00640 1 -- 172.21.15.32:0/852226622 wait complete. 2024-04-17T10:01:44.668 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:01:44.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.809+0000 7f695c000640 1 Processor -- start 2024-04-17T10:01:44.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.809+0000 7f695c000640 1 -- start start 2024-04-17T10:01:44.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.813+0000 7f695c000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69540b2430 0x7f69540b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:44.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.813+0000 7f695c000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f69540b3900 0x7f695406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:44.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.813+0000 7f695c000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f695406c800 con 0x7f695415d670 2024-04-17T10:01:44.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.813+0000 7f695c000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f69540b2d50 con 0x7f69540b3900 2024-04-17T10:01:44.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.813+0000 7f695c000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f695406bf70 con 0x7f69540b2430 2024-04-17T10:01:44.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.813+0000 7f6958c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f69540b3900 0x7f695406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:44.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.813+0000 7f6959600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69540b2430 0x7f69540b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:44.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.813+0000 7f6958c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f69540b3900 0x7f695406ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:47090/0 (socket says 172.21.15.32:47090) 2024-04-17T10:01:44.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.813+0000 7f6958c00640 1 -- 172.21.15.32:0/1007029488 learned_addr learned my addr 172.21.15.32:0/1007029488 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:44.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.813+0000 7f6959600640 1 -- 172.21.15.32:0/1007029488 >> v2:172.21.15.144:3300/0 conn(0x7f69540b3900 msgr2=0x7f695406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:44.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.813+0000 7f6959600640 1 --2- 172.21.15.32:0/1007029488 >> v2:172.21.15.144:3300/0 conn(0x7f69540b3900 0x7f695406ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:44.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.813+0000 7f6959600640 1 -- 172.21.15.32:0/1007029488 >> v1:172.21.15.144:6789/0 conn(0x7f695415d670 legacy=0x7f69541602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:44.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.813+0000 7f6959600640 1 -- 172.21.15.32:0/1007029488 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695406c240 con 0x7f69540b2430 2024-04-17T10:01:44.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.813+0000 7f6958c00640 1 --2- 172.21.15.32:0/1007029488 >> v2:172.21.15.144:3300/0 conn(0x7f69540b3900 0x7f695406ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:01:44.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.817+0000 7f6959600640 1 --2- 172.21.15.32:0/1007029488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69540b2430 0x7f69540b2810 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f69480097c0 tx=0x7f694801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=760be4e535935c2c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:44.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.817+0000 7f6953e00640 1 -- 172.21.15.32:0/1007029488 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69480b2070 con 0x7f69540b2430 2024-04-17T10:01:44.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.817+0000 7f6953e00640 1 -- 172.21.15.32:0/1007029488 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69480ad640 con 0x7f69540b2430 2024-04-17T10:01:44.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.817+0000 7f6953e00640 1 -- 172.21.15.32:0/1007029488 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69480ada10 con 0x7f69540b2430 2024-04-17T10:01:44.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.817+0000 7f695c000640 1 -- 172.21.15.32:0/1007029488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69540b2430 msgr2=0x7f69540b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:44.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.817+0000 7f695c000640 1 --2- 172.21.15.32:0/1007029488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69540b2430 0x7f69540b2810 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f69480097c0 tx=0x7f694801c4a0 comp rx=0 tx=0).stop 2024-04-17T10:01:44.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.817+0000 7f695c000640 1 -- 172.21.15.32:0/1007029488 shutdown_connections 2024-04-17T10:01:44.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.817+0000 7f695c000640 1 --2- 172.21.15.32:0/1007029488 >> v2:172.21.15.144:3300/0 conn(0x7f69540b3900 0x7f695406ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:44.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.817+0000 7f695c000640 1 --2- 172.21.15.32:0/1007029488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69540b2430 0x7f69540b2810 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:44.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.817+0000 7f695c000640 1 -- 172.21.15.32:0/1007029488 >> 172.21.15.32:0/1007029488 conn(0x7f695409a9f0 msgr2=0x7f69540b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:44.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.817+0000 7f695c000640 1 -- 172.21.15.32:0/1007029488 shutdown_connections 2024-04-17T10:01:44.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.817+0000 7f695c000640 1 -- 172.21.15.32:0/1007029488 wait complete. 2024-04-17T10:01:44.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.817+0000 7f695c000640 1 Processor -- start 2024-04-17T10:01:44.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.817+0000 7f695c000640 1 -- start start 2024-04-17T10:01:44.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f695c000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69540b2430 0x7f6954166fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:44.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f695c000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69540b3900 0x7f6954155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:44.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f695c000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f695415d670 0x7f695415de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:44.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f695c000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f69540cdb30 con 0x7f69540b2430 2024-04-17T10:01:44.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f695c000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6954160e00 con 0x7f695415d670 2024-04-17T10:01:44.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f695c000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f69540cdcb0 con 0x7f69540b3900 2024-04-17T10:01:44.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f6959600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69540b2430 0x7f6954166fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:44.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f6959600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69540b2430 0x7f6954166fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52826/0 (socket says 172.21.15.32:52826) 2024-04-17T10:01:44.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f6959600640 1 -- 172.21.15.32:0/4117306315 learned_addr learned my addr 172.21.15.32:0/4117306315 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:44.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f695a000640 1 --2- 172.21.15.32:0/4117306315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f695415d670 0x7f695415de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:44.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f6958c00640 1 --2- 172.21.15.32:0/4117306315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69540b3900 0x7f6954155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:44.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f6959600640 1 -- 172.21.15.32:0/4117306315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69540b3900 msgr2=0x7f6954155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:44.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f6959600640 1 --2- 172.21.15.32:0/4117306315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69540b3900 0x7f6954155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:44.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f6959600640 1 -- 172.21.15.32:0/4117306315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f695415d670 msgr2=0x7f695415de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:44.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f6959600640 1 --2- 172.21.15.32:0/4117306315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f695415d670 0x7f695415de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:44.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f6959600640 1 -- 172.21.15.32:0/4117306315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69540b6700 con 0x7f69540b2430 2024-04-17T10:01:44.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f6959600640 1 --2- 172.21.15.32:0/4117306315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69540b2430 0x7f6954166fa0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f694801a0d0 tx=0x7f6948019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:44.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f6951600640 1 -- 172.21.15.32:0/4117306315 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69480ba070 con 0x7f69540b2430 2024-04-17T10:01:44.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f6951600640 1 -- 172.21.15.32:0/4117306315 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69480b6410 con 0x7f69540b2430 2024-04-17T10:01:44.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f695c000640 1 -- 172.21.15.32:0/4117306315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69540b4ce0 con 0x7f69540b2430 2024-04-17T10:01:44.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f6951600640 1 -- 172.21.15.32:0/4117306315 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69480b2040 con 0x7f69540b2430 2024-04-17T10:01:44.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f695c000640 1 -- 172.21.15.32:0/4117306315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69540b6c80 con 0x7f69540b2430 2024-04-17T10:01:44.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f695c000640 1 -- 172.21.15.32:0/4117306315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6954068a90 con 0x7f69540b2430 2024-04-17T10:01:44.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.821+0000 7f6951600640 1 -- 172.21.15.32:0/4117306315 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f69480bd070 con 0x7f69540b2430 2024-04-17T10:01:44.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.825+0000 7f6951600640 1 --2- 172.21.15.32:0/4117306315 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f69340683b0 0x7f693406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:44.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.825+0000 7f6951600640 1 -- 172.21.15.32:0/4117306315 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f6948120910 con 0x7f69540b2430 2024-04-17T10:01:44.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.825+0000 7f6958c00640 1 --2- 172.21.15.32:0/4117306315 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f69340683b0 0x7f693406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:44.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.825+0000 7f6958c00640 1 --2- 172.21.15.32:0/4117306315 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f69340683b0 0x7f693406a870 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f695415faf0 tx=0x7f6940008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:44.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.829+0000 7f6951600640 1 -- 172.21.15.32:0/4117306315 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f6948016090 con 0x7f69540b2430 2024-04-17T10:01:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.997+0000 7f695c000640 1 -- 172.21.15.32:0/4117306315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69540b36e0 con 0x7f69540b2430 2024-04-17T10:01:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:44.997+0000 7f6951600640 1 -- 172.21.15.32:0/4117306315 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+16620 (secure 0 0 0) 0x7f69480f0ec0 con 0x7f69540b2430 2024-04-17T10:01:44.996 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:44.997 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":97,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:01:42.155683+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:01:38.938460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-17T10:01:39.505171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:01:44.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.001+0000 7f695c000640 1 -- 172.21.15.32:0/4117306315 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f69340683b0 msgr2=0x7f693406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:44.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.001+0000 7f695c000640 1 --2- 172.21.15.32:0/4117306315 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f69340683b0 0x7f693406a870 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f695415faf0 tx=0x7f6940008040 comp rx=0 tx=0).stop 2024-04-17T10:01:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.001+0000 7f695c000640 1 -- 172.21.15.32:0/4117306315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69540b2430 msgr2=0x7f6954166fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.001+0000 7f695c000640 1 --2- 172.21.15.32:0/4117306315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69540b2430 0x7f6954166fa0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f694801a0d0 tx=0x7f6948019240 comp rx=0 tx=0).stop 2024-04-17T10:01:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.001+0000 7f695c000640 1 -- 172.21.15.32:0/4117306315 shutdown_connections 2024-04-17T10:01:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.001+0000 7f695c000640 1 --2- 172.21.15.32:0/4117306315 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f69340683b0 0x7f693406a870 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:45.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.001+0000 7f695c000640 1 --2- 172.21.15.32:0/4117306315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69540b3900 0x7f6954155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:45.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.001+0000 7f695c000640 1 --2- 172.21.15.32:0/4117306315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f695415d670 0x7f695415de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:45.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.001+0000 7f695c000640 1 --2- 172.21.15.32:0/4117306315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69540b2430 0x7f6954166fa0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:45.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.001+0000 7f695c000640 1 -- 172.21.15.32:0/4117306315 >> 172.21.15.32:0/4117306315 conn(0x7f695409a9f0 msgr2=0x7f69540cfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:45.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.001+0000 7f695c000640 1 -- 172.21.15.32:0/4117306315 shutdown_connections 2024-04-17T10:01:45.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.001+0000 7f695c000640 1 -- 172.21.15.32:0/4117306315 wait complete. 2024-04-17T10:01:45.165 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T10:01:45.165 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:01:45.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.309+0000 7f7e30400640 1 Processor -- start 2024-04-17T10:01:45.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.309+0000 7f7e30400640 1 -- start start 2024-04-17T10:01:45.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.309+0000 7f7e30400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e2815baa0 0x7f7e2815bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:45.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.309+0000 7f7e30400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e28072bb0 con 0x7f7e2806ef60 2024-04-17T10:01:45.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.309+0000 7f7e30400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e280b3dc0 con 0x7f7e28144e90 2024-04-17T10:01:45.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.309+0000 7f7e30400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7e2815c440 con 0x7f7e2815baa0 2024-04-17T10:01:45.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.313+0000 7f7e2e400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7e2806ef60 0x7f7e28071e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50458/0 (socket says 172.21.15.32:50458) 2024-04-17T10:01:45.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.313+0000 7f7e2e400640 1 -- 172.21.15.32:0/4156460155 learned_addr learned my addr 172.21.15.32:0/4156460155 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:45.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.313+0000 7f7e2d000640 1 --2- 172.21.15.32:0/4156460155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e2815baa0 0x7f7e2815bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:45.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.313+0000 7f7e27e00640 1 -- 172.21.15.32:0/4156460155 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2157076809 0 0) 0x7f7e28072bb0 con 0x7f7e2806ef60 2024-04-17T10:01:45.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.313+0000 7f7e27e00640 1 -- 172.21.15.32:0/4156460155 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e10005740 con 0x7f7e2806ef60 2024-04-17T10:01:45.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.313+0000 7f7e27e00640 1 -- 172.21.15.32:0/4156460155 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 524337308 0 0) 0x7f7e280b3dc0 con 0x7f7e28144e90 2024-04-17T10:01:45.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.313+0000 7f7e27e00640 1 -- 172.21.15.32:0/4156460155 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e28072bb0 con 0x7f7e28144e90 2024-04-17T10:01:45.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.313+0000 7f7e2d000640 1 -- 172.21.15.32:0/4156460155 >> v1:172.21.15.144:6789/0 conn(0x7f7e28144e90 legacy=0x7f7e280b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:45.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.313+0000 7f7e2d000640 1 -- 172.21.15.32:0/4156460155 >> v1:172.21.15.32:6789/0 conn(0x7f7e2806ef60 legacy=0x7f7e28071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:45.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.313+0000 7f7e2d000640 1 -- 172.21.15.32:0/4156460155 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e280b40a0 con 0x7f7e2815baa0 2024-04-17T10:01:45.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.313+0000 7f7e2d000640 1 --2- 172.21.15.32:0/4156460155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e2815baa0 0x7f7e2815bf00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7e140097f0 tx=0x7f7e1401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd03c0e44ece22bf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:45.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.313+0000 7f7e27e00640 1 -- 172.21.15.32:0/4156460155 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e140b0070 con 0x7f7e2815baa0 2024-04-17T10:01:45.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e27e00640 1 -- 172.21.15.32:0/4156460155 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e140ab600 con 0x7f7e2815baa0 2024-04-17T10:01:45.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e27e00640 1 -- 172.21.15.32:0/4156460155 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e140ab9d0 con 0x7f7e2815baa0 2024-04-17T10:01:45.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e30400640 1 -- 172.21.15.32:0/4156460155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e2815baa0 msgr2=0x7f7e2815bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:45.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e30400640 1 --2- 172.21.15.32:0/4156460155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e2815baa0 0x7f7e2815bf00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7e140097f0 tx=0x7f7e1401c4e0 comp rx=0 tx=0).stop 2024-04-17T10:01:45.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e30400640 1 -- 172.21.15.32:0/4156460155 shutdown_connections 2024-04-17T10:01:45.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e30400640 1 --2- 172.21.15.32:0/4156460155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e2815baa0 0x7f7e2815bf00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:45.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e30400640 1 -- 172.21.15.32:0/4156460155 >> 172.21.15.32:0/4156460155 conn(0x7f7e2806ba30 msgr2=0x7f7e2806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:45.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e30400640 1 -- 172.21.15.32:0/4156460155 shutdown_connections 2024-04-17T10:01:45.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e30400640 1 -- 172.21.15.32:0/4156460155 wait complete. 2024-04-17T10:01:45.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e30400640 1 Processor -- start 2024-04-17T10:01:45.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e30400640 1 -- start start 2024-04-17T10:01:45.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e30400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e2806ef60 0x7f7e28135ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:45.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e30400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e28144e90 0x7f7e280b1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:45.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e30400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e2815baa0 0x7f7e280b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:45.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e30400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2806e7e0 con 0x7f7e2815baa0 2024-04-17T10:01:45.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e30400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2806e660 con 0x7f7e28144e90 2024-04-17T10:01:45.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e30400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7e2806e960 con 0x7f7e2806ef60 2024-04-17T10:01:45.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e2e400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e2815baa0 0x7f7e280b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:45.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e2e400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e2815baa0 0x7f7e280b1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52834/0 (socket says 172.21.15.32:52834) 2024-04-17T10:01:45.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e2e400640 1 -- 172.21.15.32:0/995865524 learned_addr learned my addr 172.21.15.32:0/995865524 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:45.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e2d000640 1 --2- 172.21.15.32:0/995865524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e28144e90 0x7f7e280b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:45.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e2da00640 1 --2- 172.21.15.32:0/995865524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e2806ef60 0x7f7e28135ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:45.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e2e400640 1 -- 172.21.15.32:0/995865524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e2806ef60 msgr2=0x7f7e28135ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e2e400640 1 --2- 172.21.15.32:0/995865524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e2806ef60 0x7f7e28135ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e2e400640 1 -- 172.21.15.32:0/995865524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e28144e90 msgr2=0x7f7e280b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e2e400640 1 --2- 172.21.15.32:0/995865524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e28144e90 0x7f7e280b1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e2e400640 1 -- 172.21.15.32:0/995865524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2815ddc0 con 0x7f7e2815baa0 2024-04-17T10:01:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.317+0000 7f7e2da00640 1 --2- 172.21.15.32:0/995865524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e2806ef60 0x7f7e28135ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.321+0000 7f7e2e400640 1 --2- 172.21.15.32:0/995865524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e2815baa0 0x7f7e280b1cb0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000cbc0 tx=0x7f7e2000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.321+0000 7f7e25600640 1 -- 172.21.15.32:0/995865524 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e20017070 con 0x7f7e2815baa0 2024-04-17T10:01:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.321+0000 7f7e25600640 1 -- 172.21.15.32:0/995865524 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e2000ace0 con 0x7f7e2815baa0 2024-04-17T10:01:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.321+0000 7f7e30400640 1 -- 172.21.15.32:0/995865524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2815e0a0 con 0x7f7e2815baa0 2024-04-17T10:01:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.321+0000 7f7e25600640 1 -- 172.21.15.32:0/995865524 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e20012600 con 0x7f7e2815baa0 2024-04-17T10:01:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.321+0000 7f7e30400640 1 -- 172.21.15.32:0/995865524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e281363d0 con 0x7f7e2815baa0 2024-04-17T10:01:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.321+0000 7f7e30400640 1 -- 172.21.15.32:0/995865524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e28068a90 con 0x7f7e2815baa0 2024-04-17T10:01:45.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.321+0000 7f7e25600640 1 -- 172.21.15.32:0/995865524 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7e2001a070 con 0x7f7e2815baa0 2024-04-17T10:01:45.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.321+0000 7f7e25600640 1 --2- 172.21.15.32:0/995865524 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7e080683b0 0x7f7e0806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:45.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.321+0000 7f7e25600640 1 -- 172.21.15.32:0/995865524 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f7e20086800 con 0x7f7e2815baa0 2024-04-17T10:01:45.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.321+0000 7f7e2da00640 1 --2- 172.21.15.32:0/995865524 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7e080683b0 0x7f7e0806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:45.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.325+0000 7f7e2da00640 1 --2- 172.21.15.32:0/995865524 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7e080683b0 0x7f7e0806a870 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0016e0 tx=0x7f7e1c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:45.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.325+0000 7f7e25600640 1 -- 172.21.15.32:0/995865524 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f7e20056650 con 0x7f7e2815baa0 2024-04-17T10:01:45.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.489+0000 7f7e30400640 1 -- 172.21.15.32:0/995865524 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e281446c0 con 0x7f7e080683b0 2024-04-17T10:01:45.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.497+0000 7f7e25600640 1 -- 172.21.15.32:0/995865524 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252086 (secure 0 0 0) 0x7f7e281446c0 con 0x7f7e080683b0 2024-04-17T10:01:45.498 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:45.500 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T10:01:45.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.501+0000 7f7e30400640 1 -- 172.21.15.32:0/995865524 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7e080683b0 msgr2=0x7f7e0806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:45.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.501+0000 7f7e30400640 1 --2- 172.21.15.32:0/995865524 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7e080683b0 0x7f7e0806a870 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0016e0 tx=0x7f7e1c009040 comp rx=0 tx=0).stop 2024-04-17T10:01:45.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.501+0000 7f7e30400640 1 -- 172.21.15.32:0/995865524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e2815baa0 msgr2=0x7f7e280b1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:45.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.501+0000 7f7e30400640 1 --2- 172.21.15.32:0/995865524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e2815baa0 0x7f7e280b1cb0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000cbc0 tx=0x7f7e2000a430 comp rx=0 tx=0).stop 2024-04-17T10:01:45.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.501+0000 7f7e30400640 1 -- 172.21.15.32:0/995865524 shutdown_connections 2024-04-17T10:01:45.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.501+0000 7f7e30400640 1 --2- 172.21.15.32:0/995865524 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7e080683b0 0x7f7e0806a870 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:45.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.501+0000 7f7e30400640 1 --2- 172.21.15.32:0/995865524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e2815baa0 0x7f7e280b1cb0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:45.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.501+0000 7f7e30400640 1 --2- 172.21.15.32:0/995865524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e28144e90 0x7f7e280b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:45.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.501+0000 7f7e30400640 1 --2- 172.21.15.32:0/995865524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e2806ef60 0x7f7e28135ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:45.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.501+0000 7f7e30400640 1 -- 172.21.15.32:0/995865524 >> 172.21.15.32:0/995865524 conn(0x7f7e2806ba30 msgr2=0x7f7e2806be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:45.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.501+0000 7f7e30400640 1 -- 172.21.15.32:0/995865524 shutdown_connections 2024-04-17T10:01:45.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.501+0000 7f7e30400640 1 -- 172.21.15.32:0/995865524 wait complete. 2024-04-17T10:01:45.518 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-04-17T10:01:45.404898+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264152,"kb_used_data":10536,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754710568,"statfs":{"total":773094113280,"available":772823621632,"internally_reserved":0,"allocated":10788864,"data_stored":5293148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000998"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.635087+0000","last_change":"2024-04-17T10:01:41.169709+0000","last_active":"2024-04-17T10:01:42.635087+0000","last_peered":"2024-04-17T10:01:42.635087+0000","last_clean":"2024-04-17T10:01:42.635087+0000","last_became_active":"2024-04-17T10:01:41.169404+0000","last_became_peered":"2024-04-17T10:01:41.169404+0000","last_unstale":"2024-04-17T10:01:42.635087+0000","last_undegraded":"2024-04-17T10:01:42.635087+0000","last_fullsized":"2024-04-17T10:01:42.635087+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:57:14.455552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164709+0000","last_change":"2024-04-17T10:01:41.172326+0000","last_active":"2024-04-17T10:01:42.164709+0000","last_peered":"2024-04-17T10:01:42.164709+0000","last_clean":"2024-04-17T10:01:42.164709+0000","last_became_active":"2024-04-17T10:01:41.171845+0000","last_became_peered":"2024-04-17T10:01:41.171845+0000","last_unstale":"2024-04-17T10:01:42.164709+0000","last_undegraded":"2024-04-17T10:01:42.164709+0000","last_fullsized":"2024-04-17T10:01:42.164709+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:15:42.342477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.634905+0000","last_change":"2024-04-17T10:01:41.171183+0000","last_active":"2024-04-17T10:01:42.634905+0000","last_peered":"2024-04-17T10:01:42.634905+0000","last_clean":"2024-04-17T10:01:42.634905+0000","last_became_active":"2024-04-17T10:01:41.170897+0000","last_became_peered":"2024-04-17T10:01:41.170897+0000","last_unstale":"2024-04-17T10:01:42.634905+0000","last_undegraded":"2024-04-17T10:01:42.634905+0000","last_fullsized":"2024-04-17T10:01:42.634905+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:30:16.652994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.236450+0000","last_change":"2024-04-17T10:01:41.172134+0000","last_active":"2024-04-17T10:01:43.236450+0000","last_peered":"2024-04-17T10:01:43.236450+0000","last_clean":"2024-04-17T10:01:43.236450+0000","last_became_active":"2024-04-17T10:01:41.171865+0000","last_became_peered":"2024-04-17T10:01:41.171865+0000","last_unstale":"2024-04-17T10:01:43.236450+0000","last_undegraded":"2024-04-17T10:01:43.236450+0000","last_fullsized":"2024-04-17T10:01:43.236450+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:43.497208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168540+0000","last_change":"2024-04-17T10:01:41.171370+0000","last_active":"2024-04-17T10:01:42.168540+0000","last_peered":"2024-04-17T10:01:42.168540+0000","last_clean":"2024-04-17T10:01:42.168540+0000","last_became_active":"2024-04-17T10:01:41.170117+0000","last_became_peered":"2024-04-17T10:01:41.170117+0000","last_unstale":"2024-04-17T10:01:42.168540+0000","last_undegraded":"2024-04-17T10:01:42.168540+0000","last_fullsized":"2024-04-17T10:01:42.168540+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:38:39.994633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.102024+0000","last_change":"2024-04-17T10:01:41.172657+0000","last_active":"2024-04-17T10:01:43.102024+0000","last_peered":"2024-04-17T10:01:43.102024+0000","last_clean":"2024-04-17T10:01:43.102024+0000","last_became_active":"2024-04-17T10:01:41.172400+0000","last_became_peered":"2024-04-17T10:01:41.172400+0000","last_unstale":"2024-04-17T10:01:43.102024+0000","last_undegraded":"2024-04-17T10:01:43.102024+0000","last_fullsized":"2024-04-17T10:01:43.102024+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:19:33.143963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.101906+0000","last_change":"2024-04-17T10:01:41.174356+0000","last_active":"2024-04-17T10:01:43.101906+0000","last_peered":"2024-04-17T10:01:43.101906+0000","last_clean":"2024-04-17T10:01:43.101906+0000","last_became_active":"2024-04-17T10:01:41.174027+0000","last_became_peered":"2024-04-17T10:01:41.174027+0000","last_unstale":"2024-04-17T10:01:43.101906+0000","last_undegraded":"2024-04-17T10:01:43.101906+0000","last_fullsized":"2024-04-17T10:01:43.101906+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:28:28.088441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164269+0000","last_change":"2024-04-17T10:01:41.176238+0000","last_active":"2024-04-17T10:01:42.164269+0000","last_peered":"2024-04-17T10:01:42.164269+0000","last_clean":"2024-04-17T10:01:42.164269+0000","last_became_active":"2024-04-17T10:01:41.175737+0000","last_became_peered":"2024-04-17T10:01:41.175737+0000","last_unstale":"2024-04-17T10:01:42.164269+0000","last_undegraded":"2024-04-17T10:01:42.164269+0000","last_fullsized":"2024-04-17T10:01:42.164269+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:08:31.141647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.167741+0000","last_change":"2024-04-17T10:01:41.181864+0000","last_active":"2024-04-17T10:01:42.167741+0000","last_peered":"2024-04-17T10:01:42.167741+0000","last_clean":"2024-04-17T10:01:42.167741+0000","last_became_active":"2024-04-17T10:01:41.181570+0000","last_became_peered":"2024-04-17T10:01:41.181570+0000","last_unstale":"2024-04-17T10:01:42.167741+0000","last_undegraded":"2024-04-17T10:01:42.167741+0000","last_fullsized":"2024-04-17T10:01:42.167741+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:35:45.509525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168683+0000","last_change":"2024-04-17T10:01:41.178356+0000","last_active":"2024-04-17T10:01:42.168683+0000","last_peered":"2024-04-17T10:01:42.168683+0000","last_clean":"2024-04-17T10:01:42.168683+0000","last_became_active":"2024-04-17T10:01:41.178050+0000","last_became_peered":"2024-04-17T10:01:41.178050+0000","last_unstale":"2024-04-17T10:01:42.168683+0000","last_undegraded":"2024-04-17T10:01:42.168683+0000","last_fullsized":"2024-04-17T10:01:42.168683+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:05:38.614052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.101847+0000","last_change":"2024-04-17T10:01:41.168135+0000","last_active":"2024-04-17T10:01:43.101847+0000","last_peered":"2024-04-17T10:01:43.101847+0000","last_clean":"2024-04-17T10:01:43.101847+0000","last_became_active":"2024-04-17T10:01:41.167852+0000","last_became_peered":"2024-04-17T10:01:41.167852+0000","last_unstale":"2024-04-17T10:01:43.101847+0000","last_undegraded":"2024-04-17T10:01:43.101847+0000","last_fullsized":"2024-04-17T10:01:43.101847+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:18.842280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.167603+0000","last_change":"2024-04-17T10:01:41.185788+0000","last_active":"2024-04-17T10:01:42.167603+0000","last_peered":"2024-04-17T10:01:42.167603+0000","last_clean":"2024-04-17T10:01:42.167603+0000","last_became_active":"2024-04-17T10:01:41.185569+0000","last_became_peered":"2024-04-17T10:01:41.185569+0000","last_unstale":"2024-04-17T10:01:42.167603+0000","last_undegraded":"2024-04-17T10:01:42.167603+0000","last_fullsized":"2024-04-17T10:01:42.167603+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:59:20.847648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.236148+0000","last_change":"2024-04-17T10:01:41.170165+0000","last_active":"2024-04-17T10:01:43.236148+0000","last_peered":"2024-04-17T10:01:43.236148+0000","last_clean":"2024-04-17T10:01:43.236148+0000","last_became_active":"2024-04-17T10:01:41.169909+0000","last_became_peered":"2024-04-17T10:01:41.169909+0000","last_unstale":"2024-04-17T10:01:43.236148+0000","last_undegraded":"2024-04-17T10:01:43.236148+0000","last_fullsized":"2024-04-17T10:01:43.236148+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:52:26.949592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704621+0000","last_change":"2024-04-17T10:01:41.162991+0000","last_active":"2024-04-17T10:01:42.704621+0000","last_peered":"2024-04-17T10:01:42.704621+0000","last_clean":"2024-04-17T10:01:42.704621+0000","last_became_active":"2024-04-17T10:01:41.162698+0000","last_became_peered":"2024-04-17T10:01:41.162698+0000","last_unstale":"2024-04-17T10:01:42.704621+0000","last_undegraded":"2024-04-17T10:01:42.704621+0000","last_fullsized":"2024-04-17T10:01:42.704621+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:44:20.048815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.236018+0000","last_change":"2024-04-17T10:01:41.182220+0000","last_active":"2024-04-17T10:01:43.236018+0000","last_peered":"2024-04-17T10:01:43.236018+0000","last_clean":"2024-04-17T10:01:43.236018+0000","last_became_active":"2024-04-17T10:01:41.181934+0000","last_became_peered":"2024-04-17T10:01:41.181934+0000","last_unstale":"2024-04-17T10:01:43.236018+0000","last_undegraded":"2024-04-17T10:01:43.236018+0000","last_fullsized":"2024-04-17T10:01:43.236018+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:05:38.614052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.101931+0000","last_change":"2024-04-17T10:01:41.173558+0000","last_active":"2024-04-17T10:01:43.101931+0000","last_peered":"2024-04-17T10:01:43.101931+0000","last_clean":"2024-04-17T10:01:43.101931+0000","last_became_active":"2024-04-17T10:01:41.173031+0000","last_became_peered":"2024-04-17T10:01:41.173031+0000","last_unstale":"2024-04-17T10:01:43.101931+0000","last_undegraded":"2024-04-17T10:01:43.101931+0000","last_fullsized":"2024-04-17T10:01:43.101931+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:43.497208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164827+0000","last_change":"2024-04-17T10:01:41.176858+0000","last_active":"2024-04-17T10:01:42.164827+0000","last_peered":"2024-04-17T10:01:42.164827+0000","last_clean":"2024-04-17T10:01:42.164827+0000","last_became_active":"2024-04-17T10:01:41.176549+0000","last_became_peered":"2024-04-17T10:01:41.176549+0000","last_unstale":"2024-04-17T10:01:42.164827+0000","last_undegraded":"2024-04-17T10:01:42.164827+0000","last_fullsized":"2024-04-17T10:01:42.164827+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:15:09.143192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.236621+0000","last_change":"2024-04-17T10:01:41.169886+0000","last_active":"2024-04-17T10:01:43.236621+0000","last_peered":"2024-04-17T10:01:43.236621+0000","last_clean":"2024-04-17T10:01:43.236621+0000","last_became_active":"2024-04-17T10:01:41.169583+0000","last_became_peered":"2024-04-17T10:01:41.169583+0000","last_unstale":"2024-04-17T10:01:43.236621+0000","last_undegraded":"2024-04-17T10:01:43.236621+0000","last_fullsized":"2024-04-17T10:01:43.236621+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:41.159393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704953+0000","last_change":"2024-04-17T10:01:41.174709+0000","last_active":"2024-04-17T10:01:42.704953+0000","last_peered":"2024-04-17T10:01:42.704953+0000","last_clean":"2024-04-17T10:01:42.704953+0000","last_became_active":"2024-04-17T10:01:41.174455+0000","last_became_peered":"2024-04-17T10:01:41.174455+0000","last_unstale":"2024-04-17T10:01:42.704953+0000","last_undegraded":"2024-04-17T10:01:42.704953+0000","last_fullsized":"2024-04-17T10:01:42.704953+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:53:48.731198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.167786+0000","last_change":"2024-04-17T10:01:41.181864+0000","last_active":"2024-04-17T10:01:42.167786+0000","last_peered":"2024-04-17T10:01:42.167786+0000","last_clean":"2024-04-17T10:01:42.167786+0000","last_became_active":"2024-04-17T10:01:41.181359+0000","last_became_peered":"2024-04-17T10:01:41.181359+0000","last_unstale":"2024-04-17T10:01:42.167786+0000","last_undegraded":"2024-04-17T10:01:42.167786+0000","last_fullsized":"2024-04-17T10:01:42.167786+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:59:16.039278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704705+0000","last_change":"2024-04-17T10:01:41.170969+0000","last_active":"2024-04-17T10:01:42.704705+0000","last_peered":"2024-04-17T10:01:42.704705+0000","last_clean":"2024-04-17T10:01:42.704705+0000","last_became_active":"2024-04-17T10:01:41.170599+0000","last_became_peered":"2024-04-17T10:01:41.170599+0000","last_unstale":"2024-04-17T10:01:42.704705+0000","last_undegraded":"2024-04-17T10:01:42.704705+0000","last_fullsized":"2024-04-17T10:01:42.704705+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:21:23.020174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164258+0000","last_change":"2024-04-17T10:01:41.172796+0000","last_active":"2024-04-17T10:01:42.164258+0000","last_peered":"2024-04-17T10:01:42.164258+0000","last_clean":"2024-04-17T10:01:42.164258+0000","last_became_active":"2024-04-17T10:01:41.172533+0000","last_became_peered":"2024-04-17T10:01:41.172533+0000","last_unstale":"2024-04-17T10:01:42.164258+0000","last_undegraded":"2024-04-17T10:01:42.164258+0000","last_fullsized":"2024-04-17T10:01:42.164258+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:26:28.367326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164288+0000","last_change":"2024-04-17T10:01:41.168024+0000","last_active":"2024-04-17T10:01:42.164288+0000","last_peered":"2024-04-17T10:01:42.164288+0000","last_clean":"2024-04-17T10:01:42.164288+0000","last_became_active":"2024-04-17T10:01:41.167699+0000","last_became_peered":"2024-04-17T10:01:41.167699+0000","last_unstale":"2024-04-17T10:01:42.164288+0000","last_undegraded":"2024-04-17T10:01:42.164288+0000","last_fullsized":"2024-04-17T10:01:42.164288+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:16:22.394543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.101899+0000","last_change":"2024-04-17T10:01:41.170278+0000","last_active":"2024-04-17T10:01:43.101899+0000","last_peered":"2024-04-17T10:01:43.101899+0000","last_clean":"2024-04-17T10:01:43.101899+0000","last_became_active":"2024-04-17T10:01:41.170026+0000","last_became_peered":"2024-04-17T10:01:41.170026+0000","last_unstale":"2024-04-17T10:01:43.101899+0000","last_undegraded":"2024-04-17T10:01:43.101899+0000","last_fullsized":"2024-04-17T10:01:43.101899+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:52:26.949592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168830+0000","last_change":"2024-04-17T10:01:41.167439+0000","last_active":"2024-04-17T10:01:42.168830+0000","last_peered":"2024-04-17T10:01:42.168830+0000","last_clean":"2024-04-17T10:01:42.168830+0000","last_became_active":"2024-04-17T10:01:41.167170+0000","last_became_peered":"2024-04-17T10:01:41.167170+0000","last_unstale":"2024-04-17T10:01:42.168830+0000","last_undegraded":"2024-04-17T10:01:42.168830+0000","last_fullsized":"2024-04-17T10:01:42.168830+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:52:26.949592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164835+0000","last_change":"2024-04-17T10:01:41.182427+0000","last_active":"2024-04-17T10:01:42.164835+0000","last_peered":"2024-04-17T10:01:42.164835+0000","last_clean":"2024-04-17T10:01:42.164835+0000","last_became_active":"2024-04-17T10:01:41.182143+0000","last_became_peered":"2024-04-17T10:01:41.182143+0000","last_unstale":"2024-04-17T10:01:42.164835+0000","last_undegraded":"2024-04-17T10:01:42.164835+0000","last_fullsized":"2024-04-17T10:01:42.164835+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:36:57.579353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.101966+0000","last_change":"2024-04-17T10:01:41.176088+0000","last_active":"2024-04-17T10:01:43.101966+0000","last_peered":"2024-04-17T10:01:43.101966+0000","last_clean":"2024-04-17T10:01:43.101966+0000","last_became_active":"2024-04-17T10:01:41.175823+0000","last_became_peered":"2024-04-17T10:01:41.175823+0000","last_unstale":"2024-04-17T10:01:43.101966+0000","last_undegraded":"2024-04-17T10:01:43.101966+0000","last_fullsized":"2024-04-17T10:01:43.101966+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:41.159393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168576+0000","last_change":"2024-04-17T10:01:41.181453+0000","last_active":"2024-04-17T10:01:42.168576+0000","last_peered":"2024-04-17T10:01:42.168576+0000","last_clean":"2024-04-17T10:01:42.168576+0000","last_became_active":"2024-04-17T10:01:41.181081+0000","last_became_peered":"2024-04-17T10:01:41.181081+0000","last_unstale":"2024-04-17T10:01:42.168576+0000","last_undegraded":"2024-04-17T10:01:42.168576+0000","last_fullsized":"2024-04-17T10:01:42.168576+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:31:54.380850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164483+0000","last_change":"2024-04-17T10:01:41.181208+0000","last_active":"2024-04-17T10:01:42.164483+0000","last_peered":"2024-04-17T10:01:42.164483+0000","last_clean":"2024-04-17T10:01:42.164483+0000","last_became_active":"2024-04-17T10:01:41.180929+0000","last_became_peered":"2024-04-17T10:01:41.180929+0000","last_unstale":"2024-04-17T10:01:42.164483+0000","last_undegraded":"2024-04-17T10:01:42.164483+0000","last_fullsized":"2024-04-17T10:01:42.164483+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:21:57.358333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704622+0000","last_change":"2024-04-17T10:01:41.187968+0000","last_active":"2024-04-17T10:01:42.704622+0000","last_peered":"2024-04-17T10:01:42.704622+0000","last_clean":"2024-04-17T10:01:42.704622+0000","last_became_active":"2024-04-17T10:01:41.187361+0000","last_became_peered":"2024-04-17T10:01:41.187361+0000","last_unstale":"2024-04-17T10:01:42.704622+0000","last_undegraded":"2024-04-17T10:01:42.704622+0000","last_fullsized":"2024-04-17T10:01:42.704622+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:10:49.494694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.236162+0000","last_change":"2024-04-17T10:01:41.168988+0000","last_active":"2024-04-17T10:01:43.236162+0000","last_peered":"2024-04-17T10:01:43.236162+0000","last_clean":"2024-04-17T10:01:43.236162+0000","last_became_active":"2024-04-17T10:01:41.168719+0000","last_became_peered":"2024-04-17T10:01:41.168719+0000","last_unstale":"2024-04-17T10:01:43.236162+0000","last_undegraded":"2024-04-17T10:01:43.236162+0000","last_fullsized":"2024-04-17T10:01:43.236162+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:18.842280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.634715+0000","last_change":"2024-04-17T10:01:41.173014+0000","last_active":"2024-04-17T10:01:42.634715+0000","last_peered":"2024-04-17T10:01:42.634715+0000","last_clean":"2024-04-17T10:01:42.634715+0000","last_became_active":"2024-04-17T10:01:41.172766+0000","last_became_peered":"2024-04-17T10:01:41.172766+0000","last_unstale":"2024-04-17T10:01:42.634715+0000","last_undegraded":"2024-04-17T10:01:42.634715+0000","last_fullsized":"2024-04-17T10:01:42.634715+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:19:29.782799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.705127+0000","last_change":"2024-04-17T10:01:41.169918+0000","last_active":"2024-04-17T10:01:42.705127+0000","last_peered":"2024-04-17T10:01:42.705127+0000","last_clean":"2024-04-17T10:01:42.705127+0000","last_became_active":"2024-04-17T10:01:41.169610+0000","last_became_peered":"2024-04-17T10:01:41.169610+0000","last_unstale":"2024-04-17T10:01:42.705127+0000","last_undegraded":"2024-04-17T10:01:42.705127+0000","last_fullsized":"2024-04-17T10:01:42.705127+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:59:54.345845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"96'4","reported_seq":18,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164778+0000","last_change":"2024-04-17T10:01:40.160121+0000","last_active":"2024-04-17T10:01:42.164778+0000","last_peered":"2024-04-17T10:01:42.164778+0000","last_clean":"2024-04-17T10:01:42.164778+0000","last_became_active":"2024-04-17T10:01:40.159857+0000","last_became_peered":"2024-04-17T10:01:40.159857+0000","last_unstale":"2024-04-17T10:01:42.164778+0000","last_undegraded":"2024-04-17T10:01:42.164778+0000","last_fullsized":"2024-04-17T10:01:42.164778+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:32:31.209263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.236630+0000","last_change":"2024-04-17T10:01:41.174097+0000","last_active":"2024-04-17T10:01:43.236630+0000","last_peered":"2024-04-17T10:01:43.236630+0000","last_clean":"2024-04-17T10:01:43.236630+0000","last_became_active":"2024-04-17T10:01:41.173670+0000","last_became_peered":"2024-04-17T10:01:41.173670+0000","last_unstale":"2024-04-17T10:01:43.236630+0000","last_undegraded":"2024-04-17T10:01:43.236630+0000","last_fullsized":"2024-04-17T10:01:43.236630+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:28:28.088441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.102149+0000","last_change":"2024-04-17T10:01:40.170107+0000","last_active":"2024-04-17T10:01:43.102149+0000","last_peered":"2024-04-17T10:01:43.102149+0000","last_clean":"2024-04-17T10:01:43.102149+0000","last_became_active":"2024-04-17T10:01:40.169840+0000","last_became_peered":"2024-04-17T10:01:40.169840+0000","last_unstale":"2024-04-17T10:01:43.102149+0000","last_undegraded":"2024-04-17T10:01:43.102149+0000","last_fullsized":"2024-04-17T10:01:43.102149+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:57:04.089659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.634810+0000","last_change":"2024-04-17T10:01:41.187137+0000","last_active":"2024-04-17T10:01:42.634810+0000","last_peered":"2024-04-17T10:01:42.634810+0000","last_clean":"2024-04-17T10:01:42.634810+0000","last_became_active":"2024-04-17T10:01:41.186816+0000","last_became_peered":"2024-04-17T10:01:41.186816+0000","last_unstale":"2024-04-17T10:01:42.634810+0000","last_undegraded":"2024-04-17T10:01:42.634810+0000","last_fullsized":"2024-04-17T10:01:42.634810+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:19:49.382185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164571+0000","last_change":"2024-04-17T10:01:40.164316+0000","last_active":"2024-04-17T10:01:42.164571+0000","last_peered":"2024-04-17T10:01:42.164571+0000","last_clean":"2024-04-17T10:01:42.164571+0000","last_became_active":"2024-04-17T10:01:40.164034+0000","last_became_peered":"2024-04-17T10:01:40.164034+0000","last_unstale":"2024-04-17T10:01:42.164571+0000","last_undegraded":"2024-04-17T10:01:42.164571+0000","last_fullsized":"2024-04-17T10:01:42.164571+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:00:40.154429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.167986+0000","last_change":"2024-04-17T10:01:41.169139+0000","last_active":"2024-04-17T10:01:42.167986+0000","last_peered":"2024-04-17T10:01:42.167986+0000","last_clean":"2024-04-17T10:01:42.167986+0000","last_became_active":"2024-04-17T10:01:41.168831+0000","last_became_peered":"2024-04-17T10:01:41.168831+0000","last_unstale":"2024-04-17T10:01:42.167986+0000","last_undegraded":"2024-04-17T10:01:42.167986+0000","last_fullsized":"2024-04-17T10:01:42.167986+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:15:27.271094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164423+0000","last_change":"2024-04-17T10:01:40.155092+0000","last_active":"2024-04-17T10:01:42.164423+0000","last_peered":"2024-04-17T10:01:42.164423+0000","last_clean":"2024-04-17T10:01:42.164423+0000","last_became_active":"2024-04-17T10:01:40.154580+0000","last_became_peered":"2024-04-17T10:01:40.154580+0000","last_unstale":"2024-04-17T10:01:42.164423+0000","last_undegraded":"2024-04-17T10:01:42.164423+0000","last_fullsized":"2024-04-17T10:01:42.164423+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:15:26.268344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.234919+0000","last_change":"2024-04-17T10:01:41.166037+0000","last_active":"2024-04-17T10:01:43.234919+0000","last_peered":"2024-04-17T10:01:43.234919+0000","last_clean":"2024-04-17T10:01:43.234919+0000","last_became_active":"2024-04-17T10:01:41.161924+0000","last_became_peered":"2024-04-17T10:01:41.161924+0000","last_unstale":"2024-04-17T10:01:43.234919+0000","last_undegraded":"2024-04-17T10:01:43.234919+0000","last_fullsized":"2024-04-17T10:01:43.234919+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:59:54.345845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.634777+0000","last_change":"2024-04-17T10:01:40.167949+0000","last_active":"2024-04-17T10:01:42.634777+0000","last_peered":"2024-04-17T10:01:42.634777+0000","last_clean":"2024-04-17T10:01:42.634777+0000","last_became_active":"2024-04-17T10:01:40.167631+0000","last_became_peered":"2024-04-17T10:01:40.167631+0000","last_unstale":"2024-04-17T10:01:42.634777+0000","last_undegraded":"2024-04-17T10:01:42.634777+0000","last_fullsized":"2024-04-17T10:01:42.634777+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:34.040735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.101853+0000","last_change":"2024-04-17T10:01:41.169292+0000","last_active":"2024-04-17T10:01:43.101853+0000","last_peered":"2024-04-17T10:01:43.101853+0000","last_clean":"2024-04-17T10:01:43.101853+0000","last_became_active":"2024-04-17T10:01:41.166770+0000","last_became_peered":"2024-04-17T10:01:41.166770+0000","last_unstale":"2024-04-17T10:01:43.101853+0000","last_undegraded":"2024-04-17T10:01:43.101853+0000","last_fullsized":"2024-04-17T10:01:43.101853+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:02:42.771231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164167+0000","last_change":"2024-04-17T10:01:40.167854+0000","last_active":"2024-04-17T10:01:42.164167+0000","last_peered":"2024-04-17T10:01:42.164167+0000","last_clean":"2024-04-17T10:01:42.164167+0000","last_became_active":"2024-04-17T10:01:40.167597+0000","last_became_peered":"2024-04-17T10:01:40.167597+0000","last_unstale":"2024-04-17T10:01:42.164167+0000","last_undegraded":"2024-04-17T10:01:42.164167+0000","last_fullsized":"2024-04-17T10:01:42.164167+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:19:56.536862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.236021+0000","last_change":"2024-04-17T10:01:41.173071+0000","last_active":"2024-04-17T10:01:43.236021+0000","last_peered":"2024-04-17T10:01:43.236021+0000","last_clean":"2024-04-17T10:01:43.236021+0000","last_became_active":"2024-04-17T10:01:41.172795+0000","last_became_peered":"2024-04-17T10:01:41.172795+0000","last_unstale":"2024-04-17T10:01:43.236021+0000","last_undegraded":"2024-04-17T10:01:43.236021+0000","last_fullsized":"2024-04-17T10:01:43.236021+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:31:54.380850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168144+0000","last_change":"2024-04-17T10:01:40.153897+0000","last_active":"2024-04-17T10:01:42.168144+0000","last_peered":"2024-04-17T10:01:42.168144+0000","last_clean":"2024-04-17T10:01:42.168144+0000","last_became_active":"2024-04-17T10:01:40.153600+0000","last_became_peered":"2024-04-17T10:01:40.153600+0000","last_unstale":"2024-04-17T10:01:42.168144+0000","last_undegraded":"2024-04-17T10:01:42.168144+0000","last_fullsized":"2024-04-17T10:01:42.168144+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:53:54.003854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.167613+0000","last_change":"2024-04-17T10:01:41.163703+0000","last_active":"2024-04-17T10:01:42.167613+0000","last_peered":"2024-04-17T10:01:42.167613+0000","last_clean":"2024-04-17T10:01:42.167613+0000","last_became_active":"2024-04-17T10:01:41.163430+0000","last_became_peered":"2024-04-17T10:01:41.163430+0000","last_unstale":"2024-04-17T10:01:42.167613+0000","last_undegraded":"2024-04-17T10:01:42.167613+0000","last_fullsized":"2024-04-17T10:01:42.167613+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:43.483061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.634662+0000","last_change":"2024-04-17T10:01:40.170053+0000","last_active":"2024-04-17T10:01:42.634662+0000","last_peered":"2024-04-17T10:01:42.634662+0000","last_clean":"2024-04-17T10:01:42.634662+0000","last_became_active":"2024-04-17T10:01:40.169679+0000","last_became_peered":"2024-04-17T10:01:40.169679+0000","last_unstale":"2024-04-17T10:01:42.634662+0000","last_undegraded":"2024-04-17T10:01:42.634662+0000","last_fullsized":"2024-04-17T10:01:42.634662+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:46:58.755726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164852+0000","last_change":"2024-04-17T10:01:41.165063+0000","last_active":"2024-04-17T10:01:42.164852+0000","last_peered":"2024-04-17T10:01:42.164852+0000","last_clean":"2024-04-17T10:01:42.164852+0000","last_became_active":"2024-04-17T10:01:41.164795+0000","last_became_peered":"2024-04-17T10:01:41.164795+0000","last_unstale":"2024-04-17T10:01:42.164852+0000","last_undegraded":"2024-04-17T10:01:42.164852+0000","last_fullsized":"2024-04-17T10:01:42.164852+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:12:27.567410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.635161+0000","last_change":"2024-04-17T10:01:40.168480+0000","last_active":"2024-04-17T10:01:42.635161+0000","last_peered":"2024-04-17T10:01:42.635161+0000","last_clean":"2024-04-17T10:01:42.635161+0000","last_became_active":"2024-04-17T10:01:40.168224+0000","last_became_peered":"2024-04-17T10:01:40.168224+0000","last_unstale":"2024-04-17T10:01:42.635161+0000","last_undegraded":"2024-04-17T10:01:42.635161+0000","last_fullsized":"2024-04-17T10:01:42.635161+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:40:03.594724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.167896+0000","last_change":"2024-04-17T10:01:41.167874+0000","last_active":"2024-04-17T10:01:42.167896+0000","last_peered":"2024-04-17T10:01:42.167896+0000","last_clean":"2024-04-17T10:01:42.167896+0000","last_became_active":"2024-04-17T10:01:41.166968+0000","last_became_peered":"2024-04-17T10:01:41.166968+0000","last_unstale":"2024-04-17T10:01:42.167896+0000","last_undegraded":"2024-04-17T10:01:42.167896+0000","last_fullsized":"2024-04-17T10:01:42.167896+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:19:57.539613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.236610+0000","last_change":"2024-04-17T10:01:40.166485+0000","last_active":"2024-04-17T10:01:43.236610+0000","last_peered":"2024-04-17T10:01:43.236610+0000","last_clean":"2024-04-17T10:01:43.236610+0000","last_became_active":"2024-04-17T10:01:40.166130+0000","last_became_peered":"2024-04-17T10:01:40.166130+0000","last_unstale":"2024-04-17T10:01:43.236610+0000","last_undegraded":"2024-04-17T10:01:43.236610+0000","last_fullsized":"2024-04-17T10:01:43.236610+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:03:52.105693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168713+0000","last_change":"2024-04-17T10:01:41.168427+0000","last_active":"2024-04-17T10:01:42.168713+0000","last_peered":"2024-04-17T10:01:42.168713+0000","last_clean":"2024-04-17T10:01:42.168713+0000","last_became_active":"2024-04-17T10:01:41.167896+0000","last_became_peered":"2024-04-17T10:01:41.167896+0000","last_unstale":"2024-04-17T10:01:42.168713+0000","last_undegraded":"2024-04-17T10:01:42.168713+0000","last_fullsized":"2024-04-17T10:01:42.168713+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:09:34.902074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"96'3","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.102198+0000","last_change":"2024-04-17T10:01:40.162282+0000","last_active":"2024-04-17T10:01:43.102198+0000","last_peered":"2024-04-17T10:01:43.102198+0000","last_clean":"2024-04-17T10:01:43.102198+0000","last_became_active":"2024-04-17T10:01:40.161926+0000","last_became_peered":"2024-04-17T10:01:40.161926+0000","last_unstale":"2024-04-17T10:01:43.102198+0000","last_undegraded":"2024-04-17T10:01:43.102198+0000","last_fullsized":"2024-04-17T10:01:43.102198+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:30:15.436996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.102075+0000","last_change":"2024-04-17T10:01:41.172100+0000","last_active":"2024-04-17T10:01:43.102075+0000","last_peered":"2024-04-17T10:01:43.102075+0000","last_clean":"2024-04-17T10:01:43.102075+0000","last_became_active":"2024-04-17T10:01:41.171610+0000","last_became_peered":"2024-04-17T10:01:41.171610+0000","last_unstale":"2024-04-17T10:01:43.102075+0000","last_undegraded":"2024-04-17T10:01:43.102075+0000","last_fullsized":"2024-04-17T10:01:43.102075+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:57:04.605370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"96'6","reported_seq":20,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.167839+0000","last_change":"2024-04-17T10:01:40.165072+0000","last_active":"2024-04-17T10:01:42.167839+0000","last_peered":"2024-04-17T10:01:42.167839+0000","last_clean":"2024-04-17T10:01:42.167839+0000","last_became_active":"2024-04-17T10:01:40.164539+0000","last_became_peered":"2024-04-17T10:01:40.164539+0000","last_unstale":"2024-04-17T10:01:42.167839+0000","last_undegraded":"2024-04-17T10:01:42.167839+0000","last_fullsized":"2024-04-17T10:01:42.167839+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:14:29.431599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.236107+0000","last_change":"2024-04-17T10:01:41.175752+0000","last_active":"2024-04-17T10:01:43.236107+0000","last_peered":"2024-04-17T10:01:43.236107+0000","last_clean":"2024-04-17T10:01:43.236107+0000","last_became_active":"2024-04-17T10:01:41.175491+0000","last_became_peered":"2024-04-17T10:01:41.175491+0000","last_unstale":"2024-04-17T10:01:43.236107+0000","last_undegraded":"2024-04-17T10:01:43.236107+0000","last_fullsized":"2024-04-17T10:01:43.236107+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:09:34.902074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704990+0000","last_change":"2024-04-17T10:01:40.161384+0000","last_active":"2024-04-17T10:01:42.704990+0000","last_peered":"2024-04-17T10:01:42.704990+0000","last_clean":"2024-04-17T10:01:42.704990+0000","last_became_active":"2024-04-17T10:01:40.160765+0000","last_became_peered":"2024-04-17T10:01:40.160765+0000","last_unstale":"2024-04-17T10:01:42.704990+0000","last_undegraded":"2024-04-17T10:01:42.704990+0000","last_fullsized":"2024-04-17T10:01:42.704990+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:52.376439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.634724+0000","last_change":"2024-04-17T10:01:41.178046+0000","last_active":"2024-04-17T10:01:42.634724+0000","last_peered":"2024-04-17T10:01:42.634724+0000","last_clean":"2024-04-17T10:01:42.634724+0000","last_became_active":"2024-04-17T10:01:41.177758+0000","last_became_peered":"2024-04-17T10:01:41.177758+0000","last_unstale":"2024-04-17T10:01:42.634724+0000","last_undegraded":"2024-04-17T10:01:42.634724+0000","last_fullsized":"2024-04-17T10:01:42.634724+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:59:54.345845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704941+0000","last_change":"2024-04-17T10:01:40.167316+0000","last_active":"2024-04-17T10:01:42.704941+0000","last_peered":"2024-04-17T10:01:42.704941+0000","last_clean":"2024-04-17T10:01:42.704941+0000","last_became_active":"2024-04-17T10:01:40.167059+0000","last_became_peered":"2024-04-17T10:01:40.167059+0000","last_unstale":"2024-04-17T10:01:42.704941+0000","last_undegraded":"2024-04-17T10:01:42.704941+0000","last_fullsized":"2024-04-17T10:01:42.704941+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:59:53.861492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.167207+0000","last_change":"2024-04-17T10:01:41.164492+0000","last_active":"2024-04-17T10:01:42.167207+0000","last_peered":"2024-04-17T10:01:42.167207+0000","last_clean":"2024-04-17T10:01:42.167207+0000","last_became_active":"2024-04-17T10:01:41.164221+0000","last_became_peered":"2024-04-17T10:01:41.164221+0000","last_unstale":"2024-04-17T10:01:42.167207+0000","last_undegraded":"2024-04-17T10:01:42.167207+0000","last_fullsized":"2024-04-17T10:01:42.167207+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:52:24.999028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.167290+0000","last_change":"2024-04-17T10:01:40.155904+0000","last_active":"2024-04-17T10:01:42.167290+0000","last_peered":"2024-04-17T10:01:42.167290+0000","last_clean":"2024-04-17T10:01:42.167290+0000","last_became_active":"2024-04-17T10:01:40.155636+0000","last_became_peered":"2024-04-17T10:01:40.155636+0000","last_unstale":"2024-04-17T10:01:42.167290+0000","last_undegraded":"2024-04-17T10:01:42.167290+0000","last_fullsized":"2024-04-17T10:01:42.167290+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:15:38.090676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704596+0000","last_change":"2024-04-17T10:01:41.164364+0000","last_active":"2024-04-17T10:01:42.704596+0000","last_peered":"2024-04-17T10:01:42.704596+0000","last_clean":"2024-04-17T10:01:42.704596+0000","last_became_active":"2024-04-17T10:01:41.164053+0000","last_became_peered":"2024-04-17T10:01:41.164053+0000","last_unstale":"2024-04-17T10:01:42.704596+0000","last_undegraded":"2024-04-17T10:01:42.704596+0000","last_fullsized":"2024-04-17T10:01:42.704596+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:46:59.758476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"96'8","reported_seq":22,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.634795+0000","last_change":"2024-04-17T10:01:40.167677+0000","last_active":"2024-04-17T10:01:42.634795+0000","last_peered":"2024-04-17T10:01:42.634795+0000","last_clean":"2024-04-17T10:01:42.634795+0000","last_became_active":"2024-04-17T10:01:40.167413+0000","last_became_peered":"2024-04-17T10:01:40.167413+0000","last_unstale":"2024-04-17T10:01:42.634795+0000","last_undegraded":"2024-04-17T10:01:42.634795+0000","last_fullsized":"2024-04-17T10:01:42.634795+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:51.603821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.169011+0000","last_change":"2024-04-17T10:01:41.158705+0000","last_active":"2024-04-17T10:01:42.169011+0000","last_peered":"2024-04-17T10:01:42.169011+0000","last_clean":"2024-04-17T10:01:42.169011+0000","last_became_active":"2024-04-17T10:01:41.158394+0000","last_became_peered":"2024-04-17T10:01:41.158394+0000","last_unstale":"2024-04-17T10:01:42.169011+0000","last_undegraded":"2024-04-17T10:01:42.169011+0000","last_fullsized":"2024-04-17T10:01:42.169011+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:43.497208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.167329+0000","last_change":"2024-04-17T10:01:40.164421+0000","last_active":"2024-04-17T10:01:42.167329+0000","last_peered":"2024-04-17T10:01:42.167329+0000","last_clean":"2024-04-17T10:01:42.167329+0000","last_became_active":"2024-04-17T10:01:40.164012+0000","last_became_peered":"2024-04-17T10:01:40.164012+0000","last_unstale":"2024-04-17T10:01:42.167329+0000","last_undegraded":"2024-04-17T10:01:42.167329+0000","last_fullsized":"2024-04-17T10:01:42.167329+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:23.887703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.234955+0000","last_change":"2024-04-17T10:01:41.161054+0000","last_active":"2024-04-17T10:01:43.234955+0000","last_peered":"2024-04-17T10:01:43.234955+0000","last_clean":"2024-04-17T10:01:43.234955+0000","last_became_active":"2024-04-17T10:01:41.160747+0000","last_became_peered":"2024-04-17T10:01:41.160747+0000","last_unstale":"2024-04-17T10:01:43.234955+0000","last_undegraded":"2024-04-17T10:01:43.234955+0000","last_fullsized":"2024-04-17T10:01:43.234955+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:19:49.382185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168859+0000","last_change":"2024-04-17T10:01:40.161166+0000","last_active":"2024-04-17T10:01:42.168859+0000","last_peered":"2024-04-17T10:01:42.168859+0000","last_clean":"2024-04-17T10:01:42.168859+0000","last_became_active":"2024-04-17T10:01:40.160730+0000","last_became_peered":"2024-04-17T10:01:40.160730+0000","last_unstale":"2024-04-17T10:01:42.168859+0000","last_undegraded":"2024-04-17T10:01:42.168859+0000","last_fullsized":"2024-04-17T10:01:42.168859+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:48:07.562422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.102173+0000","last_change":"2024-04-17T10:01:41.172348+0000","last_active":"2024-04-17T10:01:43.102173+0000","last_peered":"2024-04-17T10:01:43.102173+0000","last_clean":"2024-04-17T10:01:43.102173+0000","last_became_active":"2024-04-17T10:01:41.171216+0000","last_became_peered":"2024-04-17T10:01:41.171216+0000","last_unstale":"2024-04-17T10:01:43.102173+0000","last_undegraded":"2024-04-17T10:01:43.102173+0000","last_fullsized":"2024-04-17T10:01:43.102173+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:15:07.463450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164504+0000","last_change":"2024-04-17T10:01:40.155254+0000","last_active":"2024-04-17T10:01:42.164504+0000","last_peered":"2024-04-17T10:01:42.164504+0000","last_clean":"2024-04-17T10:01:42.164504+0000","last_became_active":"2024-04-17T10:01:40.154981+0000","last_became_peered":"2024-04-17T10:01:40.154981+0000","last_unstale":"2024-04-17T10:01:42.164504+0000","last_undegraded":"2024-04-17T10:01:42.164504+0000","last_fullsized":"2024-04-17T10:01:42.164504+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:48:24.647136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704836+0000","last_change":"2024-04-17T10:01:41.174056+0000","last_active":"2024-04-17T10:01:42.704836+0000","last_peered":"2024-04-17T10:01:42.704836+0000","last_clean":"2024-04-17T10:01:42.704836+0000","last_became_active":"2024-04-17T10:01:41.173788+0000","last_became_peered":"2024-04-17T10:01:41.173788+0000","last_unstale":"2024-04-17T10:01:42.704836+0000","last_undegraded":"2024-04-17T10:01:42.704836+0000","last_fullsized":"2024-04-17T10:01:42.704836+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:44:23.708967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164378+0000","last_change":"2024-04-17T10:01:40.160732+0000","last_active":"2024-04-17T10:01:42.164378+0000","last_peered":"2024-04-17T10:01:42.164378+0000","last_clean":"2024-04-17T10:01:42.164378+0000","last_became_active":"2024-04-17T10:01:40.160460+0000","last_became_peered":"2024-04-17T10:01:40.160460+0000","last_unstale":"2024-04-17T10:01:42.164378+0000","last_undegraded":"2024-04-17T10:01:42.164378+0000","last_fullsized":"2024-04-17T10:01:42.164378+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:49:47.387563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704803+0000","last_change":"2024-04-17T10:01:41.166663+0000","last_active":"2024-04-17T10:01:42.704803+0000","last_peered":"2024-04-17T10:01:42.704803+0000","last_clean":"2024-04-17T10:01:42.704803+0000","last_became_active":"2024-04-17T10:01:41.166404+0000","last_became_peered":"2024-04-17T10:01:41.166404+0000","last_unstale":"2024-04-17T10:01:42.704803+0000","last_undegraded":"2024-04-17T10:01:42.704803+0000","last_fullsized":"2024-04-17T10:01:42.704803+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:35.043485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.234871+0000","last_change":"2024-04-17T10:01:40.165482+0000","last_active":"2024-04-17T10:01:43.234871+0000","last_peered":"2024-04-17T10:01:43.234871+0000","last_clean":"2024-04-17T10:01:43.234871+0000","last_became_active":"2024-04-17T10:01:40.165137+0000","last_became_peered":"2024-04-17T10:01:40.165137+0000","last_unstale":"2024-04-17T10:01:43.234871+0000","last_undegraded":"2024-04-17T10:01:43.234871+0000","last_fullsized":"2024-04-17T10:01:43.234871+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:21:22.017423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704905+0000","last_change":"2024-04-17T10:01:41.183818+0000","last_active":"2024-04-17T10:01:42.704905+0000","last_peered":"2024-04-17T10:01:42.704905+0000","last_clean":"2024-04-17T10:01:42.704905+0000","last_became_active":"2024-04-17T10:01:41.183587+0000","last_became_peered":"2024-04-17T10:01:41.183587+0000","last_unstale":"2024-04-17T10:01:42.704905+0000","last_undegraded":"2024-04-17T10:01:42.704905+0000","last_fullsized":"2024-04-17T10:01:42.704905+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:48:28.471564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.234935+0000","last_change":"2024-04-17T10:01:40.168344+0000","last_active":"2024-04-17T10:01:43.234935+0000","last_peered":"2024-04-17T10:01:43.234935+0000","last_clean":"2024-04-17T10:01:43.234935+0000","last_became_active":"2024-04-17T10:01:40.168019+0000","last_became_peered":"2024-04-17T10:01:40.168019+0000","last_unstale":"2024-04-17T10:01:43.234935+0000","last_undegraded":"2024-04-17T10:01:43.234935+0000","last_fullsized":"2024-04-17T10:01:43.234935+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:34.040735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.167884+0000","last_change":"2024-04-17T10:01:41.178076+0000","last_active":"2024-04-17T10:01:42.167884+0000","last_peered":"2024-04-17T10:01:42.167884+0000","last_clean":"2024-04-17T10:01:42.167884+0000","last_became_active":"2024-04-17T10:01:41.177676+0000","last_became_peered":"2024-04-17T10:01:41.177676+0000","last_unstale":"2024-04-17T10:01:42.167884+0000","last_undegraded":"2024-04-17T10:01:42.167884+0000","last_fullsized":"2024-04-17T10:01:42.167884+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:12:11.496720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164165+0000","last_change":"2024-04-17T10:01:40.170539+0000","last_active":"2024-04-17T10:01:42.164165+0000","last_peered":"2024-04-17T10:01:42.164165+0000","last_clean":"2024-04-17T10:01:42.164165+0000","last_became_active":"2024-04-17T10:01:40.170292+0000","last_became_peered":"2024-04-17T10:01:40.170292+0000","last_unstale":"2024-04-17T10:01:42.164165+0000","last_undegraded":"2024-04-17T10:01:42.164165+0000","last_fullsized":"2024-04-17T10:01:42.164165+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:59:15.036528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164979+0000","last_change":"2024-04-17T10:01:41.172112+0000","last_active":"2024-04-17T10:01:42.164979+0000","last_peered":"2024-04-17T10:01:42.164979+0000","last_clean":"2024-04-17T10:01:42.164979+0000","last_became_active":"2024-04-17T10:01:41.171452+0000","last_became_peered":"2024-04-17T10:01:41.171452+0000","last_unstale":"2024-04-17T10:01:42.164979+0000","last_undegraded":"2024-04-17T10:01:42.164979+0000","last_fullsized":"2024-04-17T10:01:42.164979+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:05:54.006175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.235194+0000","last_change":"2024-04-17T10:01:40.163629+0000","last_active":"2024-04-17T10:01:43.235194+0000","last_peered":"2024-04-17T10:01:43.235194+0000","last_clean":"2024-04-17T10:01:43.235194+0000","last_became_active":"2024-04-17T10:01:40.162662+0000","last_became_peered":"2024-04-17T10:01:40.162662+0000","last_unstale":"2024-04-17T10:01:43.235194+0000","last_undegraded":"2024-04-17T10:01:43.235194+0000","last_fullsized":"2024-04-17T10:01:43.235194+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:44:22.706217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.635221+0000","last_change":"2024-04-17T10:01:41.174084+0000","last_active":"2024-04-17T10:01:42.635221+0000","last_peered":"2024-04-17T10:01:42.635221+0000","last_clean":"2024-04-17T10:01:42.635221+0000","last_became_active":"2024-04-17T10:01:41.173775+0000","last_became_peered":"2024-04-17T10:01:41.173775+0000","last_unstale":"2024-04-17T10:01:42.635221+0000","last_undegraded":"2024-04-17T10:01:42.635221+0000","last_fullsized":"2024-04-17T10:01:42.635221+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:31.381171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"97'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164948+0000","last_change":"2024-04-17T10:01:40.163268+0000","last_active":"2024-04-17T10:01:42.164948+0000","last_peered":"2024-04-17T10:01:42.164948+0000","last_clean":"2024-04-17T10:01:42.164948+0000","last_became_active":"2024-04-17T10:01:40.162834+0000","last_became_peered":"2024-04-17T10:01:40.162834+0000","last_unstale":"2024-04-17T10:01:42.164948+0000","last_undegraded":"2024-04-17T10:01:42.164948+0000","last_fullsized":"2024-04-17T10:01:42.164948+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:08:09.524060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.634883+0000","last_change":"2024-04-17T10:01:41.178579+0000","last_active":"2024-04-17T10:01:42.634883+0000","last_peered":"2024-04-17T10:01:42.634883+0000","last_clean":"2024-04-17T10:01:42.634883+0000","last_became_active":"2024-04-17T10:01:41.172877+0000","last_became_peered":"2024-04-17T10:01:41.172877+0000","last_unstale":"2024-04-17T10:01:42.634883+0000","last_undegraded":"2024-04-17T10:01:42.634883+0000","last_fullsized":"2024-04-17T10:01:42.634883+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:13:41.747902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164627+0000","last_change":"2024-04-17T10:01:40.156477+0000","last_active":"2024-04-17T10:01:42.164627+0000","last_peered":"2024-04-17T10:01:42.164627+0000","last_clean":"2024-04-17T10:01:42.164627+0000","last_became_active":"2024-04-17T10:01:40.155966+0000","last_became_peered":"2024-04-17T10:01:40.155966+0000","last_unstale":"2024-04-17T10:01:42.164627+0000","last_undegraded":"2024-04-17T10:01:42.164627+0000","last_fullsized":"2024-04-17T10:01:42.164627+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:15.977614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168806+0000","last_change":"2024-04-17T10:01:41.169791+0000","last_active":"2024-04-17T10:01:42.168806+0000","last_peered":"2024-04-17T10:01:42.168806+0000","last_clean":"2024-04-17T10:01:42.168806+0000","last_became_active":"2024-04-17T10:01:41.169505+0000","last_became_peered":"2024-04-17T10:01:41.169505+0000","last_unstale":"2024-04-17T10:01:42.168806+0000","last_undegraded":"2024-04-17T10:01:42.168806+0000","last_fullsized":"2024-04-17T10:01:42.168806+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:18.842280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168699+0000","last_change":"2024-04-17T10:01:40.157674+0000","last_active":"2024-04-17T10:01:42.168699+0000","last_peered":"2024-04-17T10:01:42.168699+0000","last_clean":"2024-04-17T10:01:42.168699+0000","last_became_active":"2024-04-17T10:01:40.157376+0000","last_became_peered":"2024-04-17T10:01:40.157376+0000","last_unstale":"2024-04-17T10:01:42.168699+0000","last_undegraded":"2024-04-17T10:01:42.168699+0000","last_fullsized":"2024-04-17T10:01:42.168699+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:47:41.412490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704877+0000","last_change":"2024-04-17T10:01:41.164938+0000","last_active":"2024-04-17T10:01:42.704877+0000","last_peered":"2024-04-17T10:01:42.704877+0000","last_clean":"2024-04-17T10:01:42.704877+0000","last_became_active":"2024-04-17T10:01:41.162664+0000","last_became_peered":"2024-04-17T10:01:41.162664+0000","last_unstale":"2024-04-17T10:01:42.704877+0000","last_undegraded":"2024-04-17T10:01:42.704877+0000","last_fullsized":"2024-04-17T10:01:42.704877+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:40:04.597475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164491+0000","last_change":"2024-04-17T10:01:40.168554+0000","last_active":"2024-04-17T10:01:42.164491+0000","last_peered":"2024-04-17T10:01:42.164491+0000","last_clean":"2024-04-17T10:01:42.164491+0000","last_became_active":"2024-04-17T10:01:40.166642+0000","last_became_peered":"2024-04-17T10:01:40.166642+0000","last_unstale":"2024-04-17T10:01:42.164491+0000","last_undegraded":"2024-04-17T10:01:42.164491+0000","last_fullsized":"2024-04-17T10:01:42.164491+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:25.681331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168110+0000","last_change":"2024-04-17T10:01:41.159859+0000","last_active":"2024-04-17T10:01:42.168110+0000","last_peered":"2024-04-17T10:01:42.168110+0000","last_clean":"2024-04-17T10:01:42.168110+0000","last_became_active":"2024-04-17T10:01:41.157954+0000","last_became_peered":"2024-04-17T10:01:41.157954+0000","last_unstale":"2024-04-17T10:01:42.168110+0000","last_undegraded":"2024-04-17T10:01:42.168110+0000","last_fullsized":"2024-04-17T10:01:42.168110+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:32:32.212013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168264+0000","last_change":"2024-04-17T10:01:40.162475+0000","last_active":"2024-04-17T10:01:42.168264+0000","last_peered":"2024-04-17T10:01:42.168264+0000","last_clean":"2024-04-17T10:01:42.168264+0000","last_became_active":"2024-04-17T10:01:40.162118+0000","last_became_peered":"2024-04-17T10:01:40.162118+0000","last_unstale":"2024-04-17T10:01:42.168264+0000","last_undegraded":"2024-04-17T10:01:42.168264+0000","last_fullsized":"2024-04-17T10:01:42.168264+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:48.875169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704621+0000","last_change":"2024-04-17T10:01:41.165484+0000","last_active":"2024-04-17T10:01:42.704621+0000","last_peered":"2024-04-17T10:01:42.704621+0000","last_clean":"2024-04-17T10:01:42.704621+0000","last_became_active":"2024-04-17T10:01:41.163612+0000","last_became_peered":"2024-04-17T10:01:41.163612+0000","last_unstale":"2024-04-17T10:01:42.704621+0000","last_undegraded":"2024-04-17T10:01:42.704621+0000","last_fullsized":"2024-04-17T10:01:42.704621+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:52.606571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.236139+0000","last_change":"2024-04-17T10:01:40.167187+0000","last_active":"2024-04-17T10:01:43.236139+0000","last_peered":"2024-04-17T10:01:43.236139+0000","last_clean":"2024-04-17T10:01:43.236139+0000","last_became_active":"2024-04-17T10:01:40.166910+0000","last_became_peered":"2024-04-17T10:01:40.166910+0000","last_unstale":"2024-04-17T10:01:43.236139+0000","last_undegraded":"2024-04-17T10:01:43.236139+0000","last_fullsized":"2024-04-17T10:01:43.236139+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:57:04.089659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168036+0000","last_change":"2024-04-17T10:01:41.181687+0000","last_active":"2024-04-17T10:01:42.168036+0000","last_peered":"2024-04-17T10:01:42.168036+0000","last_clean":"2024-04-17T10:01:42.168036+0000","last_became_active":"2024-04-17T10:01:41.181396+0000","last_became_peered":"2024-04-17T10:01:41.181396+0000","last_unstale":"2024-04-17T10:01:42.168036+0000","last_undegraded":"2024-04-17T10:01:42.168036+0000","last_fullsized":"2024-04-17T10:01:42.168036+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:22:55.041149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704742+0000","last_change":"2024-04-17T10:01:40.171334+0000","last_active":"2024-04-17T10:01:42.704742+0000","last_peered":"2024-04-17T10:01:42.704742+0000","last_clean":"2024-04-17T10:01:42.704742+0000","last_became_active":"2024-04-17T10:01:40.171038+0000","last_became_peered":"2024-04-17T10:01:40.171038+0000","last_unstale":"2024-04-17T10:01:42.704742+0000","last_undegraded":"2024-04-17T10:01:42.704742+0000","last_fullsized":"2024-04-17T10:01:42.704742+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:39:23.806066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.705060+0000","last_change":"2024-04-17T10:01:41.175011+0000","last_active":"2024-04-17T10:01:42.705060+0000","last_peered":"2024-04-17T10:01:42.705060+0000","last_clean":"2024-04-17T10:01:42.705060+0000","last_became_active":"2024-04-17T10:01:41.174584+0000","last_became_peered":"2024-04-17T10:01:41.174584+0000","last_unstale":"2024-04-17T10:01:42.705060+0000","last_undegraded":"2024-04-17T10:01:42.705060+0000","last_fullsized":"2024-04-17T10:01:42.705060+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:19:49.382185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.101917+0000","last_change":"2024-04-17T10:01:40.168498+0000","last_active":"2024-04-17T10:01:43.101917+0000","last_peered":"2024-04-17T10:01:43.101917+0000","last_clean":"2024-04-17T10:01:43.101917+0000","last_became_active":"2024-04-17T10:01:40.168146+0000","last_became_peered":"2024-04-17T10:01:40.168146+0000","last_unstale":"2024-04-17T10:01:43.101917+0000","last_undegraded":"2024-04-17T10:01:43.101917+0000","last_fullsized":"2024-04-17T10:01:43.101917+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:03:52.105693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":313,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.102095+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T10:01:43.102095+0000","last_peered":"2024-04-17T10:01:43.102095+0000","last_clean":"2024-04-17T10:01:43.102095+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T10:01:43.102095+0000","last_undegraded":"2024-04-17T10:01:43.102095+0000","last_fullsized":"2024-04-17T10:01:43.102095+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509548,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32936,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338904,"statfs":{"total":96636764160,"available":96603037696,"internally_reserved":0,"allocated":1794048,"data_stored":1103929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509549,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":514101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509548,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1196032,"data_stored":513519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509548,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1208320,"data_stored":515083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509549,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1196032,"data_stored":513483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509548,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1191936,"data_stored":513449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542252,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32368,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339472,"statfs":{"total":96636764160,"available":96603619328,"internally_reserved":0,"allocated":1212416,"data_stored":515185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542252,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37096,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334744,"statfs":{"total":96636764160,"available":96598777856,"internally_reserved":0,"allocated":1789952,"data_stored":1104399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:01:45.523 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:01:45.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.705+0000 7fb4b7400640 1 Processor -- start 2024-04-17T10:01:45.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.705+0000 7fb4b7400640 1 -- start start 2024-04-17T10:01:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.705+0000 7fb4b7400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb4b015c750 0x7fb4b015cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.705+0000 7fb4b7400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b00b2880 0x7fb4b00cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.705+0000 7fb4b7400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4b00cd100 con 0x7fb4b0160ce0 2024-04-17T10:01:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.705+0000 7fb4b7400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb4b00b3c40 con 0x7fb4b015c750 2024-04-17T10:01:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.705+0000 7fb4b7400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b015d1a0 con 0x7fb4b00b2880 2024-04-17T10:01:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.705+0000 7fb4a7e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb4b0160ce0 0x7fb4b00b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50460/0 (socket says 172.21.15.32:50460) 2024-04-17T10:01:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.705+0000 7fb4a7e00640 1 -- 172.21.15.32:0/2292000771 learned_addr learned my addr 172.21.15.32:0/2292000771 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.705+0000 7fb4b5400640 1 --2- 172.21.15.32:0/2292000771 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b00b2880 0x7fb4b00cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.705+0000 7fb4b4a00640 1 --2- 172.21.15.32:0/2292000771 >> v2:172.21.15.144:3300/0 conn(0x7fb4b015c750 0x7fb4b015cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.705+0000 7fb4afe00640 1 -- 172.21.15.32:0/2292000771 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3208756713 0 0) 0x7fb4b00cd100 con 0x7fb4b0160ce0 2024-04-17T10:01:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.705+0000 7fb4afe00640 1 -- 172.21.15.32:0/2292000771 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb494003530 con 0x7fb4b0160ce0 2024-04-17T10:01:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.705+0000 7fb4b5400640 1 -- 172.21.15.32:0/2292000771 >> v2:172.21.15.144:3300/0 conn(0x7fb4b015c750 msgr2=0x7fb4b015cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.705+0000 7fb4b5400640 1 --2- 172.21.15.32:0/2292000771 >> v2:172.21.15.144:3300/0 conn(0x7fb4b015c750 0x7fb4b015cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.705+0000 7fb4b5400640 1 -- 172.21.15.32:0/2292000771 >> v1:172.21.15.32:6789/0 conn(0x7fb4b0160ce0 legacy=0x7fb4b00b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.705+0000 7fb4b5400640 1 -- 172.21.15.32:0/2292000771 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b015d480 con 0x7fb4b00b2880 2024-04-17T10:01:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.705+0000 7fb4b4a00640 1 --2- 172.21.15.32:0/2292000771 >> v2:172.21.15.144:3300/0 conn(0x7fb4b015c750 0x7fb4b015cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:01:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.709+0000 7fb4b5400640 1 --2- 172.21.15.32:0/2292000771 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b00b2880 0x7fb4b00cd5a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009d90 tx=0x7fb4a801c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad12aeb296cf5ad server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.709+0000 7fb4afe00640 1 -- 172.21.15.32:0/2292000771 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a80b2070 con 0x7fb4b00b2880 2024-04-17T10:01:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.709+0000 7fb4afe00640 1 -- 172.21.15.32:0/2292000771 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4a80a5970 con 0x7fb4b00b2880 2024-04-17T10:01:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.709+0000 7fb4afe00640 1 -- 172.21.15.32:0/2292000771 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a80a5d40 con 0x7fb4b00b2880 2024-04-17T10:01:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.709+0000 7fb4b7400640 1 -- 172.21.15.32:0/2292000771 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b00b2880 msgr2=0x7fb4b00cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.709+0000 7fb4b7400640 1 --2- 172.21.15.32:0/2292000771 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b00b2880 0x7fb4b00cd5a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009d90 tx=0x7fb4a801c8b0 comp rx=0 tx=0).stop 2024-04-17T10:01:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.709+0000 7fb4b7400640 1 -- 172.21.15.32:0/2292000771 shutdown_connections 2024-04-17T10:01:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.709+0000 7fb4b7400640 1 --2- 172.21.15.32:0/2292000771 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b00b2880 0x7fb4b00cd5a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.709+0000 7fb4b7400640 1 --2- 172.21.15.32:0/2292000771 >> v2:172.21.15.144:3300/0 conn(0x7fb4b015c750 0x7fb4b015cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.709+0000 7fb4b7400640 1 -- 172.21.15.32:0/2292000771 >> 172.21.15.32:0/2292000771 conn(0x7fb4b009a9f0 msgr2=0x7fb4b00b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.709+0000 7fb4b7400640 1 -- 172.21.15.32:0/2292000771 shutdown_connections 2024-04-17T10:01:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.709+0000 7fb4b7400640 1 -- 172.21.15.32:0/2292000771 wait complete. 2024-04-17T10:01:45.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4b7400640 1 Processor -- start 2024-04-17T10:01:45.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4b7400640 1 -- start start 2024-04-17T10:01:45.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4b7400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4b00b2880 0x7fb4b00d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:45.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4b7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b015c750 0x7fb4b0166120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:45.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4b7400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b0160ce0 0x7fb4b0158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:45.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4b7400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b00ce470 con 0x7fb4b015c750 2024-04-17T10:01:45.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4b7400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b00ce2f0 con 0x7fb4b00b2880 2024-04-17T10:01:45.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4b7400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b00ce5f0 con 0x7fb4b0160ce0 2024-04-17T10:01:45.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4a7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b015c750 0x7fb4b0166120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:45.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4a7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b015c750 0x7fb4b0166120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52840/0 (socket says 172.21.15.32:52840) 2024-04-17T10:01:45.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4a7e00640 1 -- 172.21.15.32:0/3765574604 learned_addr learned my addr 172.21.15.32:0/3765574604 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:45.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4b4a00640 1 --2- 172.21.15.32:0/3765574604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4b00b2880 0x7fb4b00d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:45.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4b5400640 1 --2- 172.21.15.32:0/3765574604 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b0160ce0 0x7fb4b0158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:45.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4a7e00640 1 -- 172.21.15.32:0/3765574604 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b0160ce0 msgr2=0x7fb4b0158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:45.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4a7e00640 1 --2- 172.21.15.32:0/3765574604 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b0160ce0 0x7fb4b0158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:45.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4a7e00640 1 -- 172.21.15.32:0/3765574604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4b00b2880 msgr2=0x7fb4b00d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:45.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4a7e00640 1 --2- 172.21.15.32:0/3765574604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4b00b2880 0x7fb4b00d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:45.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4a7e00640 1 -- 172.21.15.32:0/3765574604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b00c95f0 con 0x7fb4b015c750 2024-04-17T10:01:45.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4a7e00640 1 --2- 172.21.15.32:0/3765574604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b015c750 0x7fb4b0166120 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fb49800c400 tx=0x7fb49800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:45.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4ad600640 1 -- 172.21.15.32:0/3765574604 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498017070 con 0x7fb4b015c750 2024-04-17T10:01:45.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4ad600640 1 -- 172.21.15.32:0/3765574604 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb498005d10 con 0x7fb4b015c750 2024-04-17T10:01:45.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4ad600640 1 -- 172.21.15.32:0/3765574604 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498007310 con 0x7fb4b015c750 2024-04-17T10:01:45.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4b7400640 1 -- 172.21.15.32:0/3765574604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b0166970 con 0x7fb4b015c750 2024-04-17T10:01:45.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.713+0000 7fb4b7400640 1 -- 172.21.15.32:0/3765574604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b0166f30 con 0x7fb4b015c750 2024-04-17T10:01:45.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.717+0000 7fb4b7400640 1 -- 172.21.15.32:0/3765574604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4b0068a90 con 0x7fb4b015c750 2024-04-17T10:01:45.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.717+0000 7fb4ad600640 1 -- 172.21.15.32:0/3765574604 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb49801a070 con 0x7fb4b015c750 2024-04-17T10:01:45.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.717+0000 7fb4ad600640 1 --2- 172.21.15.32:0/3765574604 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb48c0683b0 0x7fb48c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:45.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.717+0000 7fb4ad600640 1 -- 172.21.15.32:0/3765574604 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fb498085d50 con 0x7fb4b015c750 2024-04-17T10:01:45.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.717+0000 7fb4b4a00640 1 --2- 172.21.15.32:0/3765574604 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb48c0683b0 0x7fb48c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.717+0000 7fb4b4a00640 1 --2- 172.21.15.32:0/3765574604 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb48c0683b0 0x7fb48c06a870 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009580 tx=0x7fb4a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:45.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.721+0000 7fb4ad600640 1 -- 172.21.15.32:0/3765574604 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fb498056300 con 0x7fb4b015c750 2024-04-17T10:01:45.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.893+0000 7fb4b7400640 1 -- 172.21.15.32:0/3765574604 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb4b00b7450 con 0x7fb48c0683b0 2024-04-17T10:01:45.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.901+0000 7fb4ad600640 1 -- 172.21.15.32:0/3765574604 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252086 (secure 0 0 0) 0x7fb4b00b7450 con 0x7fb48c0683b0 2024-04-17T10:01:45.899 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:45.900 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T10:01:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.905+0000 7fb4b7400640 1 -- 172.21.15.32:0/3765574604 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb48c0683b0 msgr2=0x7fb48c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.905+0000 7fb4b7400640 1 --2- 172.21.15.32:0/3765574604 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb48c0683b0 0x7fb48c06a870 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009580 tx=0x7fb4a0008040 comp rx=0 tx=0).stop 2024-04-17T10:01:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.905+0000 7fb4b7400640 1 -- 172.21.15.32:0/3765574604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b015c750 msgr2=0x7fb4b0166120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.905+0000 7fb4b7400640 1 --2- 172.21.15.32:0/3765574604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b015c750 0x7fb4b0166120 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fb49800c400 tx=0x7fb49800c8c0 comp rx=0 tx=0).stop 2024-04-17T10:01:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.905+0000 7fb4b7400640 1 -- 172.21.15.32:0/3765574604 shutdown_connections 2024-04-17T10:01:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.905+0000 7fb4b7400640 1 --2- 172.21.15.32:0/3765574604 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb48c0683b0 0x7fb48c06a870 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.905+0000 7fb4b7400640 1 --2- 172.21.15.32:0/3765574604 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b0160ce0 0x7fb4b0158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.905+0000 7fb4b7400640 1 --2- 172.21.15.32:0/3765574604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b015c750 0x7fb4b0166120 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.905+0000 7fb4b7400640 1 --2- 172.21.15.32:0/3765574604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4b00b2880 0x7fb4b00d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.905+0000 7fb4b7400640 1 -- 172.21.15.32:0/3765574604 >> 172.21.15.32:0/3765574604 conn(0x7fb4b009a9f0 msgr2=0x7fb4b015e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.905+0000 7fb4b7400640 1 -- 172.21.15.32:0/3765574604 shutdown_connections 2024-04-17T10:01:45.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:45.905+0000 7fb4b7400640 1 -- 172.21.15.32:0/3765574604 wait complete. 2024-04-17T10:01:45.920 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-04-17T10:01:45.404898+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264152,"kb_used_data":10536,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754710568,"statfs":{"total":773094113280,"available":772823621632,"internally_reserved":0,"allocated":10788864,"data_stored":5293148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000998"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.635087+0000","last_change":"2024-04-17T10:01:41.169709+0000","last_active":"2024-04-17T10:01:42.635087+0000","last_peered":"2024-04-17T10:01:42.635087+0000","last_clean":"2024-04-17T10:01:42.635087+0000","last_became_active":"2024-04-17T10:01:41.169404+0000","last_became_peered":"2024-04-17T10:01:41.169404+0000","last_unstale":"2024-04-17T10:01:42.635087+0000","last_undegraded":"2024-04-17T10:01:42.635087+0000","last_fullsized":"2024-04-17T10:01:42.635087+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:57:14.455552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164709+0000","last_change":"2024-04-17T10:01:41.172326+0000","last_active":"2024-04-17T10:01:42.164709+0000","last_peered":"2024-04-17T10:01:42.164709+0000","last_clean":"2024-04-17T10:01:42.164709+0000","last_became_active":"2024-04-17T10:01:41.171845+0000","last_became_peered":"2024-04-17T10:01:41.171845+0000","last_unstale":"2024-04-17T10:01:42.164709+0000","last_undegraded":"2024-04-17T10:01:42.164709+0000","last_fullsized":"2024-04-17T10:01:42.164709+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:15:42.342477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.634905+0000","last_change":"2024-04-17T10:01:41.171183+0000","last_active":"2024-04-17T10:01:42.634905+0000","last_peered":"2024-04-17T10:01:42.634905+0000","last_clean":"2024-04-17T10:01:42.634905+0000","last_became_active":"2024-04-17T10:01:41.170897+0000","last_became_peered":"2024-04-17T10:01:41.170897+0000","last_unstale":"2024-04-17T10:01:42.634905+0000","last_undegraded":"2024-04-17T10:01:42.634905+0000","last_fullsized":"2024-04-17T10:01:42.634905+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:30:16.652994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.236450+0000","last_change":"2024-04-17T10:01:41.172134+0000","last_active":"2024-04-17T10:01:43.236450+0000","last_peered":"2024-04-17T10:01:43.236450+0000","last_clean":"2024-04-17T10:01:43.236450+0000","last_became_active":"2024-04-17T10:01:41.171865+0000","last_became_peered":"2024-04-17T10:01:41.171865+0000","last_unstale":"2024-04-17T10:01:43.236450+0000","last_undegraded":"2024-04-17T10:01:43.236450+0000","last_fullsized":"2024-04-17T10:01:43.236450+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:43.497208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168540+0000","last_change":"2024-04-17T10:01:41.171370+0000","last_active":"2024-04-17T10:01:42.168540+0000","last_peered":"2024-04-17T10:01:42.168540+0000","last_clean":"2024-04-17T10:01:42.168540+0000","last_became_active":"2024-04-17T10:01:41.170117+0000","last_became_peered":"2024-04-17T10:01:41.170117+0000","last_unstale":"2024-04-17T10:01:42.168540+0000","last_undegraded":"2024-04-17T10:01:42.168540+0000","last_fullsized":"2024-04-17T10:01:42.168540+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:38:39.994633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.102024+0000","last_change":"2024-04-17T10:01:41.172657+0000","last_active":"2024-04-17T10:01:43.102024+0000","last_peered":"2024-04-17T10:01:43.102024+0000","last_clean":"2024-04-17T10:01:43.102024+0000","last_became_active":"2024-04-17T10:01:41.172400+0000","last_became_peered":"2024-04-17T10:01:41.172400+0000","last_unstale":"2024-04-17T10:01:43.102024+0000","last_undegraded":"2024-04-17T10:01:43.102024+0000","last_fullsized":"2024-04-17T10:01:43.102024+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:19:33.143963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.101906+0000","last_change":"2024-04-17T10:01:41.174356+0000","last_active":"2024-04-17T10:01:43.101906+0000","last_peered":"2024-04-17T10:01:43.101906+0000","last_clean":"2024-04-17T10:01:43.101906+0000","last_became_active":"2024-04-17T10:01:41.174027+0000","last_became_peered":"2024-04-17T10:01:41.174027+0000","last_unstale":"2024-04-17T10:01:43.101906+0000","last_undegraded":"2024-04-17T10:01:43.101906+0000","last_fullsized":"2024-04-17T10:01:43.101906+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:28:28.088441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164269+0000","last_change":"2024-04-17T10:01:41.176238+0000","last_active":"2024-04-17T10:01:42.164269+0000","last_peered":"2024-04-17T10:01:42.164269+0000","last_clean":"2024-04-17T10:01:42.164269+0000","last_became_active":"2024-04-17T10:01:41.175737+0000","last_became_peered":"2024-04-17T10:01:41.175737+0000","last_unstale":"2024-04-17T10:01:42.164269+0000","last_undegraded":"2024-04-17T10:01:42.164269+0000","last_fullsized":"2024-04-17T10:01:42.164269+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:08:31.141647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.167741+0000","last_change":"2024-04-17T10:01:41.181864+0000","last_active":"2024-04-17T10:01:42.167741+0000","last_peered":"2024-04-17T10:01:42.167741+0000","last_clean":"2024-04-17T10:01:42.167741+0000","last_became_active":"2024-04-17T10:01:41.181570+0000","last_became_peered":"2024-04-17T10:01:41.181570+0000","last_unstale":"2024-04-17T10:01:42.167741+0000","last_undegraded":"2024-04-17T10:01:42.167741+0000","last_fullsized":"2024-04-17T10:01:42.167741+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:35:45.509525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168683+0000","last_change":"2024-04-17T10:01:41.178356+0000","last_active":"2024-04-17T10:01:42.168683+0000","last_peered":"2024-04-17T10:01:42.168683+0000","last_clean":"2024-04-17T10:01:42.168683+0000","last_became_active":"2024-04-17T10:01:41.178050+0000","last_became_peered":"2024-04-17T10:01:41.178050+0000","last_unstale":"2024-04-17T10:01:42.168683+0000","last_undegraded":"2024-04-17T10:01:42.168683+0000","last_fullsized":"2024-04-17T10:01:42.168683+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:05:38.614052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.101847+0000","last_change":"2024-04-17T10:01:41.168135+0000","last_active":"2024-04-17T10:01:43.101847+0000","last_peered":"2024-04-17T10:01:43.101847+0000","last_clean":"2024-04-17T10:01:43.101847+0000","last_became_active":"2024-04-17T10:01:41.167852+0000","last_became_peered":"2024-04-17T10:01:41.167852+0000","last_unstale":"2024-04-17T10:01:43.101847+0000","last_undegraded":"2024-04-17T10:01:43.101847+0000","last_fullsized":"2024-04-17T10:01:43.101847+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:18.842280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.167603+0000","last_change":"2024-04-17T10:01:41.185788+0000","last_active":"2024-04-17T10:01:42.167603+0000","last_peered":"2024-04-17T10:01:42.167603+0000","last_clean":"2024-04-17T10:01:42.167603+0000","last_became_active":"2024-04-17T10:01:41.185569+0000","last_became_peered":"2024-04-17T10:01:41.185569+0000","last_unstale":"2024-04-17T10:01:42.167603+0000","last_undegraded":"2024-04-17T10:01:42.167603+0000","last_fullsized":"2024-04-17T10:01:42.167603+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:59:20.847648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.236148+0000","last_change":"2024-04-17T10:01:41.170165+0000","last_active":"2024-04-17T10:01:43.236148+0000","last_peered":"2024-04-17T10:01:43.236148+0000","last_clean":"2024-04-17T10:01:43.236148+0000","last_became_active":"2024-04-17T10:01:41.169909+0000","last_became_peered":"2024-04-17T10:01:41.169909+0000","last_unstale":"2024-04-17T10:01:43.236148+0000","last_undegraded":"2024-04-17T10:01:43.236148+0000","last_fullsized":"2024-04-17T10:01:43.236148+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:52:26.949592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704621+0000","last_change":"2024-04-17T10:01:41.162991+0000","last_active":"2024-04-17T10:01:42.704621+0000","last_peered":"2024-04-17T10:01:42.704621+0000","last_clean":"2024-04-17T10:01:42.704621+0000","last_became_active":"2024-04-17T10:01:41.162698+0000","last_became_peered":"2024-04-17T10:01:41.162698+0000","last_unstale":"2024-04-17T10:01:42.704621+0000","last_undegraded":"2024-04-17T10:01:42.704621+0000","last_fullsized":"2024-04-17T10:01:42.704621+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:44:20.048815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.236018+0000","last_change":"2024-04-17T10:01:41.182220+0000","last_active":"2024-04-17T10:01:43.236018+0000","last_peered":"2024-04-17T10:01:43.236018+0000","last_clean":"2024-04-17T10:01:43.236018+0000","last_became_active":"2024-04-17T10:01:41.181934+0000","last_became_peered":"2024-04-17T10:01:41.181934+0000","last_unstale":"2024-04-17T10:01:43.236018+0000","last_undegraded":"2024-04-17T10:01:43.236018+0000","last_fullsized":"2024-04-17T10:01:43.236018+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:05:38.614052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.101931+0000","last_change":"2024-04-17T10:01:41.173558+0000","last_active":"2024-04-17T10:01:43.101931+0000","last_peered":"2024-04-17T10:01:43.101931+0000","last_clean":"2024-04-17T10:01:43.101931+0000","last_became_active":"2024-04-17T10:01:41.173031+0000","last_became_peered":"2024-04-17T10:01:41.173031+0000","last_unstale":"2024-04-17T10:01:43.101931+0000","last_undegraded":"2024-04-17T10:01:43.101931+0000","last_fullsized":"2024-04-17T10:01:43.101931+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:43.497208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164827+0000","last_change":"2024-04-17T10:01:41.176858+0000","last_active":"2024-04-17T10:01:42.164827+0000","last_peered":"2024-04-17T10:01:42.164827+0000","last_clean":"2024-04-17T10:01:42.164827+0000","last_became_active":"2024-04-17T10:01:41.176549+0000","last_became_peered":"2024-04-17T10:01:41.176549+0000","last_unstale":"2024-04-17T10:01:42.164827+0000","last_undegraded":"2024-04-17T10:01:42.164827+0000","last_fullsized":"2024-04-17T10:01:42.164827+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:15:09.143192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.236621+0000","last_change":"2024-04-17T10:01:41.169886+0000","last_active":"2024-04-17T10:01:43.236621+0000","last_peered":"2024-04-17T10:01:43.236621+0000","last_clean":"2024-04-17T10:01:43.236621+0000","last_became_active":"2024-04-17T10:01:41.169583+0000","last_became_peered":"2024-04-17T10:01:41.169583+0000","last_unstale":"2024-04-17T10:01:43.236621+0000","last_undegraded":"2024-04-17T10:01:43.236621+0000","last_fullsized":"2024-04-17T10:01:43.236621+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:41.159393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704953+0000","last_change":"2024-04-17T10:01:41.174709+0000","last_active":"2024-04-17T10:01:42.704953+0000","last_peered":"2024-04-17T10:01:42.704953+0000","last_clean":"2024-04-17T10:01:42.704953+0000","last_became_active":"2024-04-17T10:01:41.174455+0000","last_became_peered":"2024-04-17T10:01:41.174455+0000","last_unstale":"2024-04-17T10:01:42.704953+0000","last_undegraded":"2024-04-17T10:01:42.704953+0000","last_fullsized":"2024-04-17T10:01:42.704953+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:53:48.731198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.167786+0000","last_change":"2024-04-17T10:01:41.181864+0000","last_active":"2024-04-17T10:01:42.167786+0000","last_peered":"2024-04-17T10:01:42.167786+0000","last_clean":"2024-04-17T10:01:42.167786+0000","last_became_active":"2024-04-17T10:01:41.181359+0000","last_became_peered":"2024-04-17T10:01:41.181359+0000","last_unstale":"2024-04-17T10:01:42.167786+0000","last_undegraded":"2024-04-17T10:01:42.167786+0000","last_fullsized":"2024-04-17T10:01:42.167786+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:59:16.039278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704705+0000","last_change":"2024-04-17T10:01:41.170969+0000","last_active":"2024-04-17T10:01:42.704705+0000","last_peered":"2024-04-17T10:01:42.704705+0000","last_clean":"2024-04-17T10:01:42.704705+0000","last_became_active":"2024-04-17T10:01:41.170599+0000","last_became_peered":"2024-04-17T10:01:41.170599+0000","last_unstale":"2024-04-17T10:01:42.704705+0000","last_undegraded":"2024-04-17T10:01:42.704705+0000","last_fullsized":"2024-04-17T10:01:42.704705+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:21:23.020174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164258+0000","last_change":"2024-04-17T10:01:41.172796+0000","last_active":"2024-04-17T10:01:42.164258+0000","last_peered":"2024-04-17T10:01:42.164258+0000","last_clean":"2024-04-17T10:01:42.164258+0000","last_became_active":"2024-04-17T10:01:41.172533+0000","last_became_peered":"2024-04-17T10:01:41.172533+0000","last_unstale":"2024-04-17T10:01:42.164258+0000","last_undegraded":"2024-04-17T10:01:42.164258+0000","last_fullsized":"2024-04-17T10:01:42.164258+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:26:28.367326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164288+0000","last_change":"2024-04-17T10:01:41.168024+0000","last_active":"2024-04-17T10:01:42.164288+0000","last_peered":"2024-04-17T10:01:42.164288+0000","last_clean":"2024-04-17T10:01:42.164288+0000","last_became_active":"2024-04-17T10:01:41.167699+0000","last_became_peered":"2024-04-17T10:01:41.167699+0000","last_unstale":"2024-04-17T10:01:42.164288+0000","last_undegraded":"2024-04-17T10:01:42.164288+0000","last_fullsized":"2024-04-17T10:01:42.164288+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:16:22.394543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.101899+0000","last_change":"2024-04-17T10:01:41.170278+0000","last_active":"2024-04-17T10:01:43.101899+0000","last_peered":"2024-04-17T10:01:43.101899+0000","last_clean":"2024-04-17T10:01:43.101899+0000","last_became_active":"2024-04-17T10:01:41.170026+0000","last_became_peered":"2024-04-17T10:01:41.170026+0000","last_unstale":"2024-04-17T10:01:43.101899+0000","last_undegraded":"2024-04-17T10:01:43.101899+0000","last_fullsized":"2024-04-17T10:01:43.101899+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:52:26.949592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168830+0000","last_change":"2024-04-17T10:01:41.167439+0000","last_active":"2024-04-17T10:01:42.168830+0000","last_peered":"2024-04-17T10:01:42.168830+0000","last_clean":"2024-04-17T10:01:42.168830+0000","last_became_active":"2024-04-17T10:01:41.167170+0000","last_became_peered":"2024-04-17T10:01:41.167170+0000","last_unstale":"2024-04-17T10:01:42.168830+0000","last_undegraded":"2024-04-17T10:01:42.168830+0000","last_fullsized":"2024-04-17T10:01:42.168830+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:52:26.949592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164835+0000","last_change":"2024-04-17T10:01:41.182427+0000","last_active":"2024-04-17T10:01:42.164835+0000","last_peered":"2024-04-17T10:01:42.164835+0000","last_clean":"2024-04-17T10:01:42.164835+0000","last_became_active":"2024-04-17T10:01:41.182143+0000","last_became_peered":"2024-04-17T10:01:41.182143+0000","last_unstale":"2024-04-17T10:01:42.164835+0000","last_undegraded":"2024-04-17T10:01:42.164835+0000","last_fullsized":"2024-04-17T10:01:42.164835+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:36:57.579353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.101966+0000","last_change":"2024-04-17T10:01:41.176088+0000","last_active":"2024-04-17T10:01:43.101966+0000","last_peered":"2024-04-17T10:01:43.101966+0000","last_clean":"2024-04-17T10:01:43.101966+0000","last_became_active":"2024-04-17T10:01:41.175823+0000","last_became_peered":"2024-04-17T10:01:41.175823+0000","last_unstale":"2024-04-17T10:01:43.101966+0000","last_undegraded":"2024-04-17T10:01:43.101966+0000","last_fullsized":"2024-04-17T10:01:43.101966+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:50:41.159393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168576+0000","last_change":"2024-04-17T10:01:41.181453+0000","last_active":"2024-04-17T10:01:42.168576+0000","last_peered":"2024-04-17T10:01:42.168576+0000","last_clean":"2024-04-17T10:01:42.168576+0000","last_became_active":"2024-04-17T10:01:41.181081+0000","last_became_peered":"2024-04-17T10:01:41.181081+0000","last_unstale":"2024-04-17T10:01:42.168576+0000","last_undegraded":"2024-04-17T10:01:42.168576+0000","last_fullsized":"2024-04-17T10:01:42.168576+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:31:54.380850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164483+0000","last_change":"2024-04-17T10:01:41.181208+0000","last_active":"2024-04-17T10:01:42.164483+0000","last_peered":"2024-04-17T10:01:42.164483+0000","last_clean":"2024-04-17T10:01:42.164483+0000","last_became_active":"2024-04-17T10:01:41.180929+0000","last_became_peered":"2024-04-17T10:01:41.180929+0000","last_unstale":"2024-04-17T10:01:42.164483+0000","last_undegraded":"2024-04-17T10:01:42.164483+0000","last_fullsized":"2024-04-17T10:01:42.164483+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:21:57.358333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704622+0000","last_change":"2024-04-17T10:01:41.187968+0000","last_active":"2024-04-17T10:01:42.704622+0000","last_peered":"2024-04-17T10:01:42.704622+0000","last_clean":"2024-04-17T10:01:42.704622+0000","last_became_active":"2024-04-17T10:01:41.187361+0000","last_became_peered":"2024-04-17T10:01:41.187361+0000","last_unstale":"2024-04-17T10:01:42.704622+0000","last_undegraded":"2024-04-17T10:01:42.704622+0000","last_fullsized":"2024-04-17T10:01:42.704622+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:10:49.494694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.236162+0000","last_change":"2024-04-17T10:01:41.168988+0000","last_active":"2024-04-17T10:01:43.236162+0000","last_peered":"2024-04-17T10:01:43.236162+0000","last_clean":"2024-04-17T10:01:43.236162+0000","last_became_active":"2024-04-17T10:01:41.168719+0000","last_became_peered":"2024-04-17T10:01:41.168719+0000","last_unstale":"2024-04-17T10:01:43.236162+0000","last_undegraded":"2024-04-17T10:01:43.236162+0000","last_fullsized":"2024-04-17T10:01:43.236162+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:18.842280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.634715+0000","last_change":"2024-04-17T10:01:41.173014+0000","last_active":"2024-04-17T10:01:42.634715+0000","last_peered":"2024-04-17T10:01:42.634715+0000","last_clean":"2024-04-17T10:01:42.634715+0000","last_became_active":"2024-04-17T10:01:41.172766+0000","last_became_peered":"2024-04-17T10:01:41.172766+0000","last_unstale":"2024-04-17T10:01:42.634715+0000","last_undegraded":"2024-04-17T10:01:42.634715+0000","last_fullsized":"2024-04-17T10:01:42.634715+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:19:29.782799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.705127+0000","last_change":"2024-04-17T10:01:41.169918+0000","last_active":"2024-04-17T10:01:42.705127+0000","last_peered":"2024-04-17T10:01:42.705127+0000","last_clean":"2024-04-17T10:01:42.705127+0000","last_became_active":"2024-04-17T10:01:41.169610+0000","last_became_peered":"2024-04-17T10:01:41.169610+0000","last_unstale":"2024-04-17T10:01:42.705127+0000","last_undegraded":"2024-04-17T10:01:42.705127+0000","last_fullsized":"2024-04-17T10:01:42.705127+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:59:54.345845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"96'4","reported_seq":18,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164778+0000","last_change":"2024-04-17T10:01:40.160121+0000","last_active":"2024-04-17T10:01:42.164778+0000","last_peered":"2024-04-17T10:01:42.164778+0000","last_clean":"2024-04-17T10:01:42.164778+0000","last_became_active":"2024-04-17T10:01:40.159857+0000","last_became_peered":"2024-04-17T10:01:40.159857+0000","last_unstale":"2024-04-17T10:01:42.164778+0000","last_undegraded":"2024-04-17T10:01:42.164778+0000","last_fullsized":"2024-04-17T10:01:42.164778+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:32:31.209263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.236630+0000","last_change":"2024-04-17T10:01:41.174097+0000","last_active":"2024-04-17T10:01:43.236630+0000","last_peered":"2024-04-17T10:01:43.236630+0000","last_clean":"2024-04-17T10:01:43.236630+0000","last_became_active":"2024-04-17T10:01:41.173670+0000","last_became_peered":"2024-04-17T10:01:41.173670+0000","last_unstale":"2024-04-17T10:01:43.236630+0000","last_undegraded":"2024-04-17T10:01:43.236630+0000","last_fullsized":"2024-04-17T10:01:43.236630+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:28:28.088441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.102149+0000","last_change":"2024-04-17T10:01:40.170107+0000","last_active":"2024-04-17T10:01:43.102149+0000","last_peered":"2024-04-17T10:01:43.102149+0000","last_clean":"2024-04-17T10:01:43.102149+0000","last_became_active":"2024-04-17T10:01:40.169840+0000","last_became_peered":"2024-04-17T10:01:40.169840+0000","last_unstale":"2024-04-17T10:01:43.102149+0000","last_undegraded":"2024-04-17T10:01:43.102149+0000","last_fullsized":"2024-04-17T10:01:43.102149+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:57:04.089659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.634810+0000","last_change":"2024-04-17T10:01:41.187137+0000","last_active":"2024-04-17T10:01:42.634810+0000","last_peered":"2024-04-17T10:01:42.634810+0000","last_clean":"2024-04-17T10:01:42.634810+0000","last_became_active":"2024-04-17T10:01:41.186816+0000","last_became_peered":"2024-04-17T10:01:41.186816+0000","last_unstale":"2024-04-17T10:01:42.634810+0000","last_undegraded":"2024-04-17T10:01:42.634810+0000","last_fullsized":"2024-04-17T10:01:42.634810+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:19:49.382185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164571+0000","last_change":"2024-04-17T10:01:40.164316+0000","last_active":"2024-04-17T10:01:42.164571+0000","last_peered":"2024-04-17T10:01:42.164571+0000","last_clean":"2024-04-17T10:01:42.164571+0000","last_became_active":"2024-04-17T10:01:40.164034+0000","last_became_peered":"2024-04-17T10:01:40.164034+0000","last_unstale":"2024-04-17T10:01:42.164571+0000","last_undegraded":"2024-04-17T10:01:42.164571+0000","last_fullsized":"2024-04-17T10:01:42.164571+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:00:40.154429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.167986+0000","last_change":"2024-04-17T10:01:41.169139+0000","last_active":"2024-04-17T10:01:42.167986+0000","last_peered":"2024-04-17T10:01:42.167986+0000","last_clean":"2024-04-17T10:01:42.167986+0000","last_became_active":"2024-04-17T10:01:41.168831+0000","last_became_peered":"2024-04-17T10:01:41.168831+0000","last_unstale":"2024-04-17T10:01:42.167986+0000","last_undegraded":"2024-04-17T10:01:42.167986+0000","last_fullsized":"2024-04-17T10:01:42.167986+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:15:27.271094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164423+0000","last_change":"2024-04-17T10:01:40.155092+0000","last_active":"2024-04-17T10:01:42.164423+0000","last_peered":"2024-04-17T10:01:42.164423+0000","last_clean":"2024-04-17T10:01:42.164423+0000","last_became_active":"2024-04-17T10:01:40.154580+0000","last_became_peered":"2024-04-17T10:01:40.154580+0000","last_unstale":"2024-04-17T10:01:42.164423+0000","last_undegraded":"2024-04-17T10:01:42.164423+0000","last_fullsized":"2024-04-17T10:01:42.164423+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:15:26.268344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.234919+0000","last_change":"2024-04-17T10:01:41.166037+0000","last_active":"2024-04-17T10:01:43.234919+0000","last_peered":"2024-04-17T10:01:43.234919+0000","last_clean":"2024-04-17T10:01:43.234919+0000","last_became_active":"2024-04-17T10:01:41.161924+0000","last_became_peered":"2024-04-17T10:01:41.161924+0000","last_unstale":"2024-04-17T10:01:43.234919+0000","last_undegraded":"2024-04-17T10:01:43.234919+0000","last_fullsized":"2024-04-17T10:01:43.234919+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:59:54.345845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.634777+0000","last_change":"2024-04-17T10:01:40.167949+0000","last_active":"2024-04-17T10:01:42.634777+0000","last_peered":"2024-04-17T10:01:42.634777+0000","last_clean":"2024-04-17T10:01:42.634777+0000","last_became_active":"2024-04-17T10:01:40.167631+0000","last_became_peered":"2024-04-17T10:01:40.167631+0000","last_unstale":"2024-04-17T10:01:42.634777+0000","last_undegraded":"2024-04-17T10:01:42.634777+0000","last_fullsized":"2024-04-17T10:01:42.634777+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:34.040735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.101853+0000","last_change":"2024-04-17T10:01:41.169292+0000","last_active":"2024-04-17T10:01:43.101853+0000","last_peered":"2024-04-17T10:01:43.101853+0000","last_clean":"2024-04-17T10:01:43.101853+0000","last_became_active":"2024-04-17T10:01:41.166770+0000","last_became_peered":"2024-04-17T10:01:41.166770+0000","last_unstale":"2024-04-17T10:01:43.101853+0000","last_undegraded":"2024-04-17T10:01:43.101853+0000","last_fullsized":"2024-04-17T10:01:43.101853+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:02:42.771231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164167+0000","last_change":"2024-04-17T10:01:40.167854+0000","last_active":"2024-04-17T10:01:42.164167+0000","last_peered":"2024-04-17T10:01:42.164167+0000","last_clean":"2024-04-17T10:01:42.164167+0000","last_became_active":"2024-04-17T10:01:40.167597+0000","last_became_peered":"2024-04-17T10:01:40.167597+0000","last_unstale":"2024-04-17T10:01:42.164167+0000","last_undegraded":"2024-04-17T10:01:42.164167+0000","last_fullsized":"2024-04-17T10:01:42.164167+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:19:56.536862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.236021+0000","last_change":"2024-04-17T10:01:41.173071+0000","last_active":"2024-04-17T10:01:43.236021+0000","last_peered":"2024-04-17T10:01:43.236021+0000","last_clean":"2024-04-17T10:01:43.236021+0000","last_became_active":"2024-04-17T10:01:41.172795+0000","last_became_peered":"2024-04-17T10:01:41.172795+0000","last_unstale":"2024-04-17T10:01:43.236021+0000","last_undegraded":"2024-04-17T10:01:43.236021+0000","last_fullsized":"2024-04-17T10:01:43.236021+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:31:54.380850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168144+0000","last_change":"2024-04-17T10:01:40.153897+0000","last_active":"2024-04-17T10:01:42.168144+0000","last_peered":"2024-04-17T10:01:42.168144+0000","last_clean":"2024-04-17T10:01:42.168144+0000","last_became_active":"2024-04-17T10:01:40.153600+0000","last_became_peered":"2024-04-17T10:01:40.153600+0000","last_unstale":"2024-04-17T10:01:42.168144+0000","last_undegraded":"2024-04-17T10:01:42.168144+0000","last_fullsized":"2024-04-17T10:01:42.168144+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:53:54.003854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.167613+0000","last_change":"2024-04-17T10:01:41.163703+0000","last_active":"2024-04-17T10:01:42.167613+0000","last_peered":"2024-04-17T10:01:42.167613+0000","last_clean":"2024-04-17T10:01:42.167613+0000","last_became_active":"2024-04-17T10:01:41.163430+0000","last_became_peered":"2024-04-17T10:01:41.163430+0000","last_unstale":"2024-04-17T10:01:42.167613+0000","last_undegraded":"2024-04-17T10:01:42.167613+0000","last_fullsized":"2024-04-17T10:01:42.167613+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:43.483061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.634662+0000","last_change":"2024-04-17T10:01:40.170053+0000","last_active":"2024-04-17T10:01:42.634662+0000","last_peered":"2024-04-17T10:01:42.634662+0000","last_clean":"2024-04-17T10:01:42.634662+0000","last_became_active":"2024-04-17T10:01:40.169679+0000","last_became_peered":"2024-04-17T10:01:40.169679+0000","last_unstale":"2024-04-17T10:01:42.634662+0000","last_undegraded":"2024-04-17T10:01:42.634662+0000","last_fullsized":"2024-04-17T10:01:42.634662+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:46:58.755726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164852+0000","last_change":"2024-04-17T10:01:41.165063+0000","last_active":"2024-04-17T10:01:42.164852+0000","last_peered":"2024-04-17T10:01:42.164852+0000","last_clean":"2024-04-17T10:01:42.164852+0000","last_became_active":"2024-04-17T10:01:41.164795+0000","last_became_peered":"2024-04-17T10:01:41.164795+0000","last_unstale":"2024-04-17T10:01:42.164852+0000","last_undegraded":"2024-04-17T10:01:42.164852+0000","last_fullsized":"2024-04-17T10:01:42.164852+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:12:27.567410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.635161+0000","last_change":"2024-04-17T10:01:40.168480+0000","last_active":"2024-04-17T10:01:42.635161+0000","last_peered":"2024-04-17T10:01:42.635161+0000","last_clean":"2024-04-17T10:01:42.635161+0000","last_became_active":"2024-04-17T10:01:40.168224+0000","last_became_peered":"2024-04-17T10:01:40.168224+0000","last_unstale":"2024-04-17T10:01:42.635161+0000","last_undegraded":"2024-04-17T10:01:42.635161+0000","last_fullsized":"2024-04-17T10:01:42.635161+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:40:03.594724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.167896+0000","last_change":"2024-04-17T10:01:41.167874+0000","last_active":"2024-04-17T10:01:42.167896+0000","last_peered":"2024-04-17T10:01:42.167896+0000","last_clean":"2024-04-17T10:01:42.167896+0000","last_became_active":"2024-04-17T10:01:41.166968+0000","last_became_peered":"2024-04-17T10:01:41.166968+0000","last_unstale":"2024-04-17T10:01:42.167896+0000","last_undegraded":"2024-04-17T10:01:42.167896+0000","last_fullsized":"2024-04-17T10:01:42.167896+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:19:57.539613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.236610+0000","last_change":"2024-04-17T10:01:40.166485+0000","last_active":"2024-04-17T10:01:43.236610+0000","last_peered":"2024-04-17T10:01:43.236610+0000","last_clean":"2024-04-17T10:01:43.236610+0000","last_became_active":"2024-04-17T10:01:40.166130+0000","last_became_peered":"2024-04-17T10:01:40.166130+0000","last_unstale":"2024-04-17T10:01:43.236610+0000","last_undegraded":"2024-04-17T10:01:43.236610+0000","last_fullsized":"2024-04-17T10:01:43.236610+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:03:52.105693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168713+0000","last_change":"2024-04-17T10:01:41.168427+0000","last_active":"2024-04-17T10:01:42.168713+0000","last_peered":"2024-04-17T10:01:42.168713+0000","last_clean":"2024-04-17T10:01:42.168713+0000","last_became_active":"2024-04-17T10:01:41.167896+0000","last_became_peered":"2024-04-17T10:01:41.167896+0000","last_unstale":"2024-04-17T10:01:42.168713+0000","last_undegraded":"2024-04-17T10:01:42.168713+0000","last_fullsized":"2024-04-17T10:01:42.168713+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:09:34.902074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"96'3","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.102198+0000","last_change":"2024-04-17T10:01:40.162282+0000","last_active":"2024-04-17T10:01:43.102198+0000","last_peered":"2024-04-17T10:01:43.102198+0000","last_clean":"2024-04-17T10:01:43.102198+0000","last_became_active":"2024-04-17T10:01:40.161926+0000","last_became_peered":"2024-04-17T10:01:40.161926+0000","last_unstale":"2024-04-17T10:01:43.102198+0000","last_undegraded":"2024-04-17T10:01:43.102198+0000","last_fullsized":"2024-04-17T10:01:43.102198+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:30:15.436996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.102075+0000","last_change":"2024-04-17T10:01:41.172100+0000","last_active":"2024-04-17T10:01:43.102075+0000","last_peered":"2024-04-17T10:01:43.102075+0000","last_clean":"2024-04-17T10:01:43.102075+0000","last_became_active":"2024-04-17T10:01:41.171610+0000","last_became_peered":"2024-04-17T10:01:41.171610+0000","last_unstale":"2024-04-17T10:01:43.102075+0000","last_undegraded":"2024-04-17T10:01:43.102075+0000","last_fullsized":"2024-04-17T10:01:43.102075+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:57:04.605370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"96'6","reported_seq":20,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.167839+0000","last_change":"2024-04-17T10:01:40.165072+0000","last_active":"2024-04-17T10:01:42.167839+0000","last_peered":"2024-04-17T10:01:42.167839+0000","last_clean":"2024-04-17T10:01:42.167839+0000","last_became_active":"2024-04-17T10:01:40.164539+0000","last_became_peered":"2024-04-17T10:01:40.164539+0000","last_unstale":"2024-04-17T10:01:42.167839+0000","last_undegraded":"2024-04-17T10:01:42.167839+0000","last_fullsized":"2024-04-17T10:01:42.167839+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:14:29.431599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.236107+0000","last_change":"2024-04-17T10:01:41.175752+0000","last_active":"2024-04-17T10:01:43.236107+0000","last_peered":"2024-04-17T10:01:43.236107+0000","last_clean":"2024-04-17T10:01:43.236107+0000","last_became_active":"2024-04-17T10:01:41.175491+0000","last_became_peered":"2024-04-17T10:01:41.175491+0000","last_unstale":"2024-04-17T10:01:43.236107+0000","last_undegraded":"2024-04-17T10:01:43.236107+0000","last_fullsized":"2024-04-17T10:01:43.236107+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:09:34.902074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704990+0000","last_change":"2024-04-17T10:01:40.161384+0000","last_active":"2024-04-17T10:01:42.704990+0000","last_peered":"2024-04-17T10:01:42.704990+0000","last_clean":"2024-04-17T10:01:42.704990+0000","last_became_active":"2024-04-17T10:01:40.160765+0000","last_became_peered":"2024-04-17T10:01:40.160765+0000","last_unstale":"2024-04-17T10:01:42.704990+0000","last_undegraded":"2024-04-17T10:01:42.704990+0000","last_fullsized":"2024-04-17T10:01:42.704990+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:52.376439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.634724+0000","last_change":"2024-04-17T10:01:41.178046+0000","last_active":"2024-04-17T10:01:42.634724+0000","last_peered":"2024-04-17T10:01:42.634724+0000","last_clean":"2024-04-17T10:01:42.634724+0000","last_became_active":"2024-04-17T10:01:41.177758+0000","last_became_peered":"2024-04-17T10:01:41.177758+0000","last_unstale":"2024-04-17T10:01:42.634724+0000","last_undegraded":"2024-04-17T10:01:42.634724+0000","last_fullsized":"2024-04-17T10:01:42.634724+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:59:54.345845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704941+0000","last_change":"2024-04-17T10:01:40.167316+0000","last_active":"2024-04-17T10:01:42.704941+0000","last_peered":"2024-04-17T10:01:42.704941+0000","last_clean":"2024-04-17T10:01:42.704941+0000","last_became_active":"2024-04-17T10:01:40.167059+0000","last_became_peered":"2024-04-17T10:01:40.167059+0000","last_unstale":"2024-04-17T10:01:42.704941+0000","last_undegraded":"2024-04-17T10:01:42.704941+0000","last_fullsized":"2024-04-17T10:01:42.704941+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:59:53.861492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.167207+0000","last_change":"2024-04-17T10:01:41.164492+0000","last_active":"2024-04-17T10:01:42.167207+0000","last_peered":"2024-04-17T10:01:42.167207+0000","last_clean":"2024-04-17T10:01:42.167207+0000","last_became_active":"2024-04-17T10:01:41.164221+0000","last_became_peered":"2024-04-17T10:01:41.164221+0000","last_unstale":"2024-04-17T10:01:42.167207+0000","last_undegraded":"2024-04-17T10:01:42.167207+0000","last_fullsized":"2024-04-17T10:01:42.167207+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:52:24.999028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.167290+0000","last_change":"2024-04-17T10:01:40.155904+0000","last_active":"2024-04-17T10:01:42.167290+0000","last_peered":"2024-04-17T10:01:42.167290+0000","last_clean":"2024-04-17T10:01:42.167290+0000","last_became_active":"2024-04-17T10:01:40.155636+0000","last_became_peered":"2024-04-17T10:01:40.155636+0000","last_unstale":"2024-04-17T10:01:42.167290+0000","last_undegraded":"2024-04-17T10:01:42.167290+0000","last_fullsized":"2024-04-17T10:01:42.167290+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:15:38.090676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704596+0000","last_change":"2024-04-17T10:01:41.164364+0000","last_active":"2024-04-17T10:01:42.704596+0000","last_peered":"2024-04-17T10:01:42.704596+0000","last_clean":"2024-04-17T10:01:42.704596+0000","last_became_active":"2024-04-17T10:01:41.164053+0000","last_became_peered":"2024-04-17T10:01:41.164053+0000","last_unstale":"2024-04-17T10:01:42.704596+0000","last_undegraded":"2024-04-17T10:01:42.704596+0000","last_fullsized":"2024-04-17T10:01:42.704596+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:46:59.758476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"96'8","reported_seq":22,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.634795+0000","last_change":"2024-04-17T10:01:40.167677+0000","last_active":"2024-04-17T10:01:42.634795+0000","last_peered":"2024-04-17T10:01:42.634795+0000","last_clean":"2024-04-17T10:01:42.634795+0000","last_became_active":"2024-04-17T10:01:40.167413+0000","last_became_peered":"2024-04-17T10:01:40.167413+0000","last_unstale":"2024-04-17T10:01:42.634795+0000","last_undegraded":"2024-04-17T10:01:42.634795+0000","last_fullsized":"2024-04-17T10:01:42.634795+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:51.603821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.169011+0000","last_change":"2024-04-17T10:01:41.158705+0000","last_active":"2024-04-17T10:01:42.169011+0000","last_peered":"2024-04-17T10:01:42.169011+0000","last_clean":"2024-04-17T10:01:42.169011+0000","last_became_active":"2024-04-17T10:01:41.158394+0000","last_became_peered":"2024-04-17T10:01:41.158394+0000","last_unstale":"2024-04-17T10:01:42.169011+0000","last_undegraded":"2024-04-17T10:01:42.169011+0000","last_fullsized":"2024-04-17T10:01:42.169011+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:43.497208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.167329+0000","last_change":"2024-04-17T10:01:40.164421+0000","last_active":"2024-04-17T10:01:42.167329+0000","last_peered":"2024-04-17T10:01:42.167329+0000","last_clean":"2024-04-17T10:01:42.167329+0000","last_became_active":"2024-04-17T10:01:40.164012+0000","last_became_peered":"2024-04-17T10:01:40.164012+0000","last_unstale":"2024-04-17T10:01:42.167329+0000","last_undegraded":"2024-04-17T10:01:42.167329+0000","last_fullsized":"2024-04-17T10:01:42.167329+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:23.887703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.234955+0000","last_change":"2024-04-17T10:01:41.161054+0000","last_active":"2024-04-17T10:01:43.234955+0000","last_peered":"2024-04-17T10:01:43.234955+0000","last_clean":"2024-04-17T10:01:43.234955+0000","last_became_active":"2024-04-17T10:01:41.160747+0000","last_became_peered":"2024-04-17T10:01:41.160747+0000","last_unstale":"2024-04-17T10:01:43.234955+0000","last_undegraded":"2024-04-17T10:01:43.234955+0000","last_fullsized":"2024-04-17T10:01:43.234955+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:19:49.382185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168859+0000","last_change":"2024-04-17T10:01:40.161166+0000","last_active":"2024-04-17T10:01:42.168859+0000","last_peered":"2024-04-17T10:01:42.168859+0000","last_clean":"2024-04-17T10:01:42.168859+0000","last_became_active":"2024-04-17T10:01:40.160730+0000","last_became_peered":"2024-04-17T10:01:40.160730+0000","last_unstale":"2024-04-17T10:01:42.168859+0000","last_undegraded":"2024-04-17T10:01:42.168859+0000","last_fullsized":"2024-04-17T10:01:42.168859+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:48:07.562422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.102173+0000","last_change":"2024-04-17T10:01:41.172348+0000","last_active":"2024-04-17T10:01:43.102173+0000","last_peered":"2024-04-17T10:01:43.102173+0000","last_clean":"2024-04-17T10:01:43.102173+0000","last_became_active":"2024-04-17T10:01:41.171216+0000","last_became_peered":"2024-04-17T10:01:41.171216+0000","last_unstale":"2024-04-17T10:01:43.102173+0000","last_undegraded":"2024-04-17T10:01:43.102173+0000","last_fullsized":"2024-04-17T10:01:43.102173+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:15:07.463450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164504+0000","last_change":"2024-04-17T10:01:40.155254+0000","last_active":"2024-04-17T10:01:42.164504+0000","last_peered":"2024-04-17T10:01:42.164504+0000","last_clean":"2024-04-17T10:01:42.164504+0000","last_became_active":"2024-04-17T10:01:40.154981+0000","last_became_peered":"2024-04-17T10:01:40.154981+0000","last_unstale":"2024-04-17T10:01:42.164504+0000","last_undegraded":"2024-04-17T10:01:42.164504+0000","last_fullsized":"2024-04-17T10:01:42.164504+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:48:24.647136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704836+0000","last_change":"2024-04-17T10:01:41.174056+0000","last_active":"2024-04-17T10:01:42.704836+0000","last_peered":"2024-04-17T10:01:42.704836+0000","last_clean":"2024-04-17T10:01:42.704836+0000","last_became_active":"2024-04-17T10:01:41.173788+0000","last_became_peered":"2024-04-17T10:01:41.173788+0000","last_unstale":"2024-04-17T10:01:42.704836+0000","last_undegraded":"2024-04-17T10:01:42.704836+0000","last_fullsized":"2024-04-17T10:01:42.704836+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:44:23.708967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164378+0000","last_change":"2024-04-17T10:01:40.160732+0000","last_active":"2024-04-17T10:01:42.164378+0000","last_peered":"2024-04-17T10:01:42.164378+0000","last_clean":"2024-04-17T10:01:42.164378+0000","last_became_active":"2024-04-17T10:01:40.160460+0000","last_became_peered":"2024-04-17T10:01:40.160460+0000","last_unstale":"2024-04-17T10:01:42.164378+0000","last_undegraded":"2024-04-17T10:01:42.164378+0000","last_fullsized":"2024-04-17T10:01:42.164378+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:49:47.387563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704803+0000","last_change":"2024-04-17T10:01:41.166663+0000","last_active":"2024-04-17T10:01:42.704803+0000","last_peered":"2024-04-17T10:01:42.704803+0000","last_clean":"2024-04-17T10:01:42.704803+0000","last_became_active":"2024-04-17T10:01:41.166404+0000","last_became_peered":"2024-04-17T10:01:41.166404+0000","last_unstale":"2024-04-17T10:01:42.704803+0000","last_undegraded":"2024-04-17T10:01:42.704803+0000","last_fullsized":"2024-04-17T10:01:42.704803+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:35.043485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.234871+0000","last_change":"2024-04-17T10:01:40.165482+0000","last_active":"2024-04-17T10:01:43.234871+0000","last_peered":"2024-04-17T10:01:43.234871+0000","last_clean":"2024-04-17T10:01:43.234871+0000","last_became_active":"2024-04-17T10:01:40.165137+0000","last_became_peered":"2024-04-17T10:01:40.165137+0000","last_unstale":"2024-04-17T10:01:43.234871+0000","last_undegraded":"2024-04-17T10:01:43.234871+0000","last_fullsized":"2024-04-17T10:01:43.234871+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:21:22.017423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704905+0000","last_change":"2024-04-17T10:01:41.183818+0000","last_active":"2024-04-17T10:01:42.704905+0000","last_peered":"2024-04-17T10:01:42.704905+0000","last_clean":"2024-04-17T10:01:42.704905+0000","last_became_active":"2024-04-17T10:01:41.183587+0000","last_became_peered":"2024-04-17T10:01:41.183587+0000","last_unstale":"2024-04-17T10:01:42.704905+0000","last_undegraded":"2024-04-17T10:01:42.704905+0000","last_fullsized":"2024-04-17T10:01:42.704905+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:48:28.471564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.234935+0000","last_change":"2024-04-17T10:01:40.168344+0000","last_active":"2024-04-17T10:01:43.234935+0000","last_peered":"2024-04-17T10:01:43.234935+0000","last_clean":"2024-04-17T10:01:43.234935+0000","last_became_active":"2024-04-17T10:01:40.168019+0000","last_became_peered":"2024-04-17T10:01:40.168019+0000","last_unstale":"2024-04-17T10:01:43.234935+0000","last_undegraded":"2024-04-17T10:01:43.234935+0000","last_fullsized":"2024-04-17T10:01:43.234935+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:34.040735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.167884+0000","last_change":"2024-04-17T10:01:41.178076+0000","last_active":"2024-04-17T10:01:42.167884+0000","last_peered":"2024-04-17T10:01:42.167884+0000","last_clean":"2024-04-17T10:01:42.167884+0000","last_became_active":"2024-04-17T10:01:41.177676+0000","last_became_peered":"2024-04-17T10:01:41.177676+0000","last_unstale":"2024-04-17T10:01:42.167884+0000","last_undegraded":"2024-04-17T10:01:42.167884+0000","last_fullsized":"2024-04-17T10:01:42.167884+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:12:11.496720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164165+0000","last_change":"2024-04-17T10:01:40.170539+0000","last_active":"2024-04-17T10:01:42.164165+0000","last_peered":"2024-04-17T10:01:42.164165+0000","last_clean":"2024-04-17T10:01:42.164165+0000","last_became_active":"2024-04-17T10:01:40.170292+0000","last_became_peered":"2024-04-17T10:01:40.170292+0000","last_unstale":"2024-04-17T10:01:42.164165+0000","last_undegraded":"2024-04-17T10:01:42.164165+0000","last_fullsized":"2024-04-17T10:01:42.164165+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:59:15.036528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164979+0000","last_change":"2024-04-17T10:01:41.172112+0000","last_active":"2024-04-17T10:01:42.164979+0000","last_peered":"2024-04-17T10:01:42.164979+0000","last_clean":"2024-04-17T10:01:42.164979+0000","last_became_active":"2024-04-17T10:01:41.171452+0000","last_became_peered":"2024-04-17T10:01:41.171452+0000","last_unstale":"2024-04-17T10:01:42.164979+0000","last_undegraded":"2024-04-17T10:01:42.164979+0000","last_fullsized":"2024-04-17T10:01:42.164979+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:05:54.006175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.235194+0000","last_change":"2024-04-17T10:01:40.163629+0000","last_active":"2024-04-17T10:01:43.235194+0000","last_peered":"2024-04-17T10:01:43.235194+0000","last_clean":"2024-04-17T10:01:43.235194+0000","last_became_active":"2024-04-17T10:01:40.162662+0000","last_became_peered":"2024-04-17T10:01:40.162662+0000","last_unstale":"2024-04-17T10:01:43.235194+0000","last_undegraded":"2024-04-17T10:01:43.235194+0000","last_fullsized":"2024-04-17T10:01:43.235194+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:44:22.706217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.635221+0000","last_change":"2024-04-17T10:01:41.174084+0000","last_active":"2024-04-17T10:01:42.635221+0000","last_peered":"2024-04-17T10:01:42.635221+0000","last_clean":"2024-04-17T10:01:42.635221+0000","last_became_active":"2024-04-17T10:01:41.173775+0000","last_became_peered":"2024-04-17T10:01:41.173775+0000","last_unstale":"2024-04-17T10:01:42.635221+0000","last_undegraded":"2024-04-17T10:01:42.635221+0000","last_fullsized":"2024-04-17T10:01:42.635221+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:31.381171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"97'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164948+0000","last_change":"2024-04-17T10:01:40.163268+0000","last_active":"2024-04-17T10:01:42.164948+0000","last_peered":"2024-04-17T10:01:42.164948+0000","last_clean":"2024-04-17T10:01:42.164948+0000","last_became_active":"2024-04-17T10:01:40.162834+0000","last_became_peered":"2024-04-17T10:01:40.162834+0000","last_unstale":"2024-04-17T10:01:42.164948+0000","last_undegraded":"2024-04-17T10:01:42.164948+0000","last_fullsized":"2024-04-17T10:01:42.164948+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:08:09.524060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.634883+0000","last_change":"2024-04-17T10:01:41.178579+0000","last_active":"2024-04-17T10:01:42.634883+0000","last_peered":"2024-04-17T10:01:42.634883+0000","last_clean":"2024-04-17T10:01:42.634883+0000","last_became_active":"2024-04-17T10:01:41.172877+0000","last_became_peered":"2024-04-17T10:01:41.172877+0000","last_unstale":"2024-04-17T10:01:42.634883+0000","last_undegraded":"2024-04-17T10:01:42.634883+0000","last_fullsized":"2024-04-17T10:01:42.634883+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:13:41.747902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164627+0000","last_change":"2024-04-17T10:01:40.156477+0000","last_active":"2024-04-17T10:01:42.164627+0000","last_peered":"2024-04-17T10:01:42.164627+0000","last_clean":"2024-04-17T10:01:42.164627+0000","last_became_active":"2024-04-17T10:01:40.155966+0000","last_became_peered":"2024-04-17T10:01:40.155966+0000","last_unstale":"2024-04-17T10:01:42.164627+0000","last_undegraded":"2024-04-17T10:01:42.164627+0000","last_fullsized":"2024-04-17T10:01:42.164627+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:15.977614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168806+0000","last_change":"2024-04-17T10:01:41.169791+0000","last_active":"2024-04-17T10:01:42.168806+0000","last_peered":"2024-04-17T10:01:42.168806+0000","last_clean":"2024-04-17T10:01:42.168806+0000","last_became_active":"2024-04-17T10:01:41.169505+0000","last_became_peered":"2024-04-17T10:01:41.169505+0000","last_unstale":"2024-04-17T10:01:42.168806+0000","last_undegraded":"2024-04-17T10:01:42.168806+0000","last_fullsized":"2024-04-17T10:01:42.168806+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:18.842280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168699+0000","last_change":"2024-04-17T10:01:40.157674+0000","last_active":"2024-04-17T10:01:42.168699+0000","last_peered":"2024-04-17T10:01:42.168699+0000","last_clean":"2024-04-17T10:01:42.168699+0000","last_became_active":"2024-04-17T10:01:40.157376+0000","last_became_peered":"2024-04-17T10:01:40.157376+0000","last_unstale":"2024-04-17T10:01:42.168699+0000","last_undegraded":"2024-04-17T10:01:42.168699+0000","last_fullsized":"2024-04-17T10:01:42.168699+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:47:41.412490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704877+0000","last_change":"2024-04-17T10:01:41.164938+0000","last_active":"2024-04-17T10:01:42.704877+0000","last_peered":"2024-04-17T10:01:42.704877+0000","last_clean":"2024-04-17T10:01:42.704877+0000","last_became_active":"2024-04-17T10:01:41.162664+0000","last_became_peered":"2024-04-17T10:01:41.162664+0000","last_unstale":"2024-04-17T10:01:42.704877+0000","last_undegraded":"2024-04-17T10:01:42.704877+0000","last_fullsized":"2024-04-17T10:01:42.704877+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:40:04.597475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.164491+0000","last_change":"2024-04-17T10:01:40.168554+0000","last_active":"2024-04-17T10:01:42.164491+0000","last_peered":"2024-04-17T10:01:42.164491+0000","last_clean":"2024-04-17T10:01:42.164491+0000","last_became_active":"2024-04-17T10:01:40.166642+0000","last_became_peered":"2024-04-17T10:01:40.166642+0000","last_unstale":"2024-04-17T10:01:42.164491+0000","last_undegraded":"2024-04-17T10:01:42.164491+0000","last_fullsized":"2024-04-17T10:01:42.164491+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:25.681331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168110+0000","last_change":"2024-04-17T10:01:41.159859+0000","last_active":"2024-04-17T10:01:42.168110+0000","last_peered":"2024-04-17T10:01:42.168110+0000","last_clean":"2024-04-17T10:01:42.168110+0000","last_became_active":"2024-04-17T10:01:41.157954+0000","last_became_peered":"2024-04-17T10:01:41.157954+0000","last_unstale":"2024-04-17T10:01:42.168110+0000","last_undegraded":"2024-04-17T10:01:42.168110+0000","last_fullsized":"2024-04-17T10:01:42.168110+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:32:32.212013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168264+0000","last_change":"2024-04-17T10:01:40.162475+0000","last_active":"2024-04-17T10:01:42.168264+0000","last_peered":"2024-04-17T10:01:42.168264+0000","last_clean":"2024-04-17T10:01:42.168264+0000","last_became_active":"2024-04-17T10:01:40.162118+0000","last_became_peered":"2024-04-17T10:01:40.162118+0000","last_unstale":"2024-04-17T10:01:42.168264+0000","last_undegraded":"2024-04-17T10:01:42.168264+0000","last_fullsized":"2024-04-17T10:01:42.168264+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:48.875169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704621+0000","last_change":"2024-04-17T10:01:41.165484+0000","last_active":"2024-04-17T10:01:42.704621+0000","last_peered":"2024-04-17T10:01:42.704621+0000","last_clean":"2024-04-17T10:01:42.704621+0000","last_became_active":"2024-04-17T10:01:41.163612+0000","last_became_peered":"2024-04-17T10:01:41.163612+0000","last_unstale":"2024-04-17T10:01:42.704621+0000","last_undegraded":"2024-04-17T10:01:42.704621+0000","last_fullsized":"2024-04-17T10:01:42.704621+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:52.606571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.236139+0000","last_change":"2024-04-17T10:01:40.167187+0000","last_active":"2024-04-17T10:01:43.236139+0000","last_peered":"2024-04-17T10:01:43.236139+0000","last_clean":"2024-04-17T10:01:43.236139+0000","last_became_active":"2024-04-17T10:01:40.166910+0000","last_became_peered":"2024-04-17T10:01:40.166910+0000","last_unstale":"2024-04-17T10:01:43.236139+0000","last_undegraded":"2024-04-17T10:01:43.236139+0000","last_fullsized":"2024-04-17T10:01:43.236139+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:57:04.089659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.168036+0000","last_change":"2024-04-17T10:01:41.181687+0000","last_active":"2024-04-17T10:01:42.168036+0000","last_peered":"2024-04-17T10:01:42.168036+0000","last_clean":"2024-04-17T10:01:42.168036+0000","last_became_active":"2024-04-17T10:01:41.181396+0000","last_became_peered":"2024-04-17T10:01:41.181396+0000","last_unstale":"2024-04-17T10:01:42.168036+0000","last_undegraded":"2024-04-17T10:01:42.168036+0000","last_fullsized":"2024-04-17T10:01:42.168036+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:22:55.041149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.704742+0000","last_change":"2024-04-17T10:01:40.171334+0000","last_active":"2024-04-17T10:01:42.704742+0000","last_peered":"2024-04-17T10:01:42.704742+0000","last_clean":"2024-04-17T10:01:42.704742+0000","last_became_active":"2024-04-17T10:01:40.171038+0000","last_became_peered":"2024-04-17T10:01:40.171038+0000","last_unstale":"2024-04-17T10:01:42.704742+0000","last_undegraded":"2024-04-17T10:01:42.704742+0000","last_fullsized":"2024-04-17T10:01:42.704742+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:39:23.806066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:42.705060+0000","last_change":"2024-04-17T10:01:41.175011+0000","last_active":"2024-04-17T10:01:42.705060+0000","last_peered":"2024-04-17T10:01:42.705060+0000","last_clean":"2024-04-17T10:01:42.705060+0000","last_became_active":"2024-04-17T10:01:41.174584+0000","last_became_peered":"2024-04-17T10:01:41.174584+0000","last_unstale":"2024-04-17T10:01:42.705060+0000","last_undegraded":"2024-04-17T10:01:42.705060+0000","last_fullsized":"2024-04-17T10:01:42.705060+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:40.136044+0000","last_clean_scrub_stamp":"2024-04-17T10:01:40.136044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:19:49.382185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.101917+0000","last_change":"2024-04-17T10:01:40.168498+0000","last_active":"2024-04-17T10:01:43.101917+0000","last_peered":"2024-04-17T10:01:43.101917+0000","last_clean":"2024-04-17T10:01:43.101917+0000","last_became_active":"2024-04-17T10:01:40.168146+0000","last_became_peered":"2024-04-17T10:01:40.168146+0000","last_unstale":"2024-04-17T10:01:43.101917+0000","last_undegraded":"2024-04-17T10:01:43.101917+0000","last_fullsized":"2024-04-17T10:01:43.101917+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:01:39.133294+0000","last_clean_scrub_stamp":"2024-04-17T10:01:39.133294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:03:52.105693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":313,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-17T10:01:43.102095+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T10:01:43.102095+0000","last_peered":"2024-04-17T10:01:43.102095+0000","last_clean":"2024-04-17T10:01:43.102095+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T10:01:43.102095+0000","last_undegraded":"2024-04-17T10:01:43.102095+0000","last_fullsized":"2024-04-17T10:01:43.102095+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509548,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32936,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338904,"statfs":{"total":96636764160,"available":96603037696,"internally_reserved":0,"allocated":1794048,"data_stored":1103929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509549,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":514101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509548,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1196032,"data_stored":513519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509548,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1208320,"data_stored":515083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509549,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1196032,"data_stored":513483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509548,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1191936,"data_stored":513449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542252,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32368,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339472,"statfs":{"total":96636764160,"available":96603619328,"internally_reserved":0,"allocated":1212416,"data_stored":515185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542252,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37096,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334744,"statfs":{"total":96636764160,"available":96598777856,"internally_reserved":0,"allocated":1789952,"data_stored":1104399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:01:45.923 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T10:01:45.924 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-17T10:01:46.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.113+0000 7f27a6200640 1 Processor -- start 2024-04-17T10:01:46.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.113+0000 7f27a6200640 1 -- start start 2024-04-17T10:01:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.117+0000 7f27a6200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f27a006ba30 0x7f27a006be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.117+0000 7f27a6200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27a0071e60 0x7f27a0072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:46.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.117+0000 7f27a6200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27a006c350 con 0x7f27a00cf370 2024-04-17T10:01:46.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.117+0000 7f27a6200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f27a00cfef0 con 0x7f27a0071e60 2024-04-17T10:01:46.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.117+0000 7f27a6200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f27a00d0130 con 0x7f27a006ba30 2024-04-17T10:01:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.117+0000 7f2797e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f27a00cf370 0x7f27a00cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50476/0 (socket says 172.21.15.32:50476) 2024-04-17T10:01:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.117+0000 7f2797e00640 1 -- 172.21.15.32:0/2704515761 learned_addr learned my addr 172.21.15.32:0/2704515761 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.117+0000 7f279fe00640 1 --2- 172.21.15.32:0/2704515761 >> v2:172.21.15.32:3300/0 conn(0x7f27a0071e60 0x7f27a0072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.117+0000 7f279f400640 1 --2- 172.21.15.32:0/2704515761 >> v2:172.21.15.144:3300/0 conn(0x7f27a006ba30 0x7f27a006be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.117+0000 7f279fe00640 1 -- 172.21.15.32:0/2704515761 >> v2:172.21.15.144:3300/0 conn(0x7f27a006ba30 msgr2=0x7f27a006be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.117+0000 7f279fe00640 1 --2- 172.21.15.32:0/2704515761 >> v2:172.21.15.144:3300/0 conn(0x7f27a006ba30 0x7f27a006be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.117+0000 7f279fe00640 1 -- 172.21.15.32:0/2704515761 >> v1:172.21.15.32:6789/0 conn(0x7f27a00cf370 legacy=0x7f27a00cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.117+0000 7f279fe00640 1 -- 172.21.15.32:0/2704515761 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a015d090 con 0x7f27a0071e60 2024-04-17T10:01:46.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.117+0000 7f279ea00640 1 -- 172.21.15.32:0/2704515761 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 422411309 0 0) 0x7f27a006c350 con 0x7f27a00cf370 2024-04-17T10:01:46.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.121+0000 7f279fe00640 1 --2- 172.21.15.32:0/2704515761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a0071e60 0x7f27a0072240 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f27a00b5730 tx=0x7f279801c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c56089c3746f175 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:46.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.121+0000 7f279ea00640 1 -- 172.21.15.32:0/2704515761 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27980b2070 con 0x7f27a0071e60 2024-04-17T10:01:46.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.121+0000 7f279ea00640 1 -- 172.21.15.32:0/2704515761 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27980a57f0 con 0x7f27a0071e60 2024-04-17T10:01:46.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.121+0000 7f279ea00640 1 -- 172.21.15.32:0/2704515761 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27980a5bc0 con 0x7f27a0071e60 2024-04-17T10:01:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.121+0000 7f27a6200640 1 -- 172.21.15.32:0/2704515761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a0071e60 msgr2=0x7f27a0072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.121+0000 7f27a6200640 1 --2- 172.21.15.32:0/2704515761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a0071e60 0x7f27a0072240 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f27a00b5730 tx=0x7f279801c840 comp rx=0 tx=0).stop 2024-04-17T10:01:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.121+0000 7f27a6200640 1 -- 172.21.15.32:0/2704515761 shutdown_connections 2024-04-17T10:01:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.121+0000 7f27a6200640 1 --2- 172.21.15.32:0/2704515761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a0071e60 0x7f27a0072240 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.121+0000 7f27a6200640 1 --2- 172.21.15.32:0/2704515761 >> v2:172.21.15.144:3300/0 conn(0x7f27a006ba30 0x7f27a006be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.121+0000 7f27a6200640 1 -- 172.21.15.32:0/2704515761 >> 172.21.15.32:0/2704515761 conn(0x7f27a00b2120 msgr2=0x7f27a00b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.121+0000 7f27a6200640 1 -- 172.21.15.32:0/2704515761 shutdown_connections 2024-04-17T10:01:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.121+0000 7f27a6200640 1 -- 172.21.15.32:0/2704515761 wait complete. 2024-04-17T10:01:46.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.121+0000 7f27a6200640 1 Processor -- start 2024-04-17T10:01:46.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.121+0000 7f27a6200640 1 -- start start 2024-04-17T10:01:46.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.121+0000 7f27a6200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27a006ba30 0x7f27a0158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:46.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.121+0000 7f27a6200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27a0071e60 0x7f27a013cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:46.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.121+0000 7f27a6200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a00cf370 0x7f27a00b1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:46.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.121+0000 7f27a6200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f27a015dcf0 con 0x7f27a00cf370 2024-04-17T10:01:46.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.121+0000 7f27a6200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f27a015db70 con 0x7f27a0071e60 2024-04-17T10:01:46.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.121+0000 7f27a6200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f27a0146230 con 0x7f27a006ba30 2024-04-17T10:01:46.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.121+0000 7f279fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a00cf370 0x7f27a00b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:46.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.121+0000 7f279fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a00cf370 0x7f27a00b1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52858/0 (socket says 172.21.15.32:52858) 2024-04-17T10:01:46.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.121+0000 7f279fe00640 1 -- 172.21.15.32:0/844983209 learned_addr learned my addr 172.21.15.32:0/844983209 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:46.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.125+0000 7f2797e00640 1 --2- 172.21.15.32:0/844983209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27a0071e60 0x7f27a013cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.125+0000 7f279f400640 1 --2- 172.21.15.32:0/844983209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27a006ba30 0x7f27a0158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.125+0000 7f279fe00640 1 -- 172.21.15.32:0/844983209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27a006ba30 msgr2=0x7f27a0158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.125+0000 7f279fe00640 1 --2- 172.21.15.32:0/844983209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27a006ba30 0x7f27a0158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.125+0000 7f279fe00640 1 -- 172.21.15.32:0/844983209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27a0071e60 msgr2=0x7f27a013cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.125+0000 7f279fe00640 1 --2- 172.21.15.32:0/844983209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27a0071e60 0x7f27a013cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.125+0000 7f279fe00640 1 -- 172.21.15.32:0/844983209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a0157980 con 0x7f27a00cf370 2024-04-17T10:01:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.125+0000 7f279f400640 1 --2- 172.21.15.32:0/844983209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27a006ba30 0x7f27a0158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.125+0000 7f279fe00640 1 --2- 172.21.15.32:0/844983209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a00cf370 0x7f27a00b1360 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f279801a470 tx=0x7f27980195e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:46.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.125+0000 7f2797400640 1 -- 172.21.15.32:0/844983209 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27980b2050 con 0x7f27a00cf370 2024-04-17T10:01:46.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.125+0000 7f2797400640 1 -- 172.21.15.32:0/844983209 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27980be070 con 0x7f27a00cf370 2024-04-17T10:01:46.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.125+0000 7f27a6200640 1 -- 172.21.15.32:0/844983209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a0157c60 con 0x7f27a00cf370 2024-04-17T10:01:46.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.125+0000 7f2797400640 1 -- 172.21.15.32:0/844983209 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27980ad890 con 0x7f27a00cf370 2024-04-17T10:01:46.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.125+0000 7f27a6200640 1 -- 172.21.15.32:0/844983209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a0158220 con 0x7f27a00cf370 2024-04-17T10:01:46.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.125+0000 7f27a6200640 1 -- 172.21.15.32:0/844983209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27a0068a90 con 0x7f27a00cf370 2024-04-17T10:01:46.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.125+0000 7f2797400640 1 -- 172.21.15.32:0/844983209 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f27980ada30 con 0x7f27a00cf370 2024-04-17T10:01:46.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.125+0000 7f2797400640 1 --2- 172.21.15.32:0/844983209 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f277c068380 0x7f277c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:46.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.125+0000 7f2797400640 1 -- 172.21.15.32:0/844983209 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f2798121bb0 con 0x7f27a00cf370 2024-04-17T10:01:46.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.125+0000 7f279f400640 1 --2- 172.21.15.32:0/844983209 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f277c068380 0x7f277c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:46.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.129+0000 7f279f400640 1 --2- 172.21.15.32:0/844983209 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f277c068380 0x7f277c06a840 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2790009bb0 tx=0x7f2790008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:46.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.133+0000 7f2797400640 1 -- 172.21.15.32:0/844983209 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2798016520 con 0x7f27a00cf370 2024-04-17T10:01:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.357+0000 7f27a6200640 1 -- 172.21.15.32:0/844983209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f27a015a730 con 0x7f27a00cf370 2024-04-17T10:01:46.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.361+0000 7f2797400640 1 -- 172.21.15.32:0/844983209 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7f27980f21f0 con 0x7f27a00cf370 2024-04-17T10:01:46.357 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-17T10:01:46.357 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDPmx9mUXoLKxAANV7zhLvL0DqHngk4TtsdBQ== 2024-04-17T10:01:46.357 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2024-04-17T10:01:46.357 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2024-04-17T10:01:46.358 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-17T10:01:46.358 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2024-04-17T10:01:46.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.361+0000 7f27a6200640 1 -- 172.21.15.32:0/844983209 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f277c068380 msgr2=0x7f277c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:46.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.361+0000 7f27a6200640 1 --2- 172.21.15.32:0/844983209 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f277c068380 0x7f277c06a840 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2790009bb0 tx=0x7f2790008040 comp rx=0 tx=0).stop 2024-04-17T10:01:46.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.361+0000 7f27a6200640 1 -- 172.21.15.32:0/844983209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a00cf370 msgr2=0x7f27a00b1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:46.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.361+0000 7f27a6200640 1 --2- 172.21.15.32:0/844983209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a00cf370 0x7f27a00b1360 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f279801a470 tx=0x7f27980195e0 comp rx=0 tx=0).stop 2024-04-17T10:01:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.361+0000 7f27a6200640 1 -- 172.21.15.32:0/844983209 shutdown_connections 2024-04-17T10:01:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.361+0000 7f27a6200640 1 --2- 172.21.15.32:0/844983209 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f277c068380 0x7f277c06a840 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.361+0000 7f27a6200640 1 --2- 172.21.15.32:0/844983209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27a00cf370 0x7f27a00b1360 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.361+0000 7f27a6200640 1 --2- 172.21.15.32:0/844983209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27a0071e60 0x7f27a013cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.361+0000 7f27a6200640 1 --2- 172.21.15.32:0/844983209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27a006ba30 0x7f27a0158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.361+0000 7f27a6200640 1 -- 172.21.15.32:0/844983209 >> 172.21.15.32:0/844983209 conn(0x7f27a00b2120 msgr2=0x7f27a006e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.361+0000 7f27a6200640 1 -- 172.21.15.32:0/844983209 shutdown_connections 2024-04-17T10:01:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.361+0000 7f27a6200640 1 -- 172.21.15.32:0/844983209 wait complete. 2024-04-17T10:01:46.379 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:01:46.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.521+0000 7f4f03e00640 1 Processor -- start 2024-04-17T10:01:46.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.521+0000 7f4f03e00640 1 -- start start 2024-04-17T10:01:46.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.521+0000 7f4f03e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4efc144e90 0x7f4efc0b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:46.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.521+0000 7f4f03e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4efc0b4de0 0x7f4efc15baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:46.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.521+0000 7f4f03e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4efc072bb0 con 0x7f4efc15bfe0 2024-04-17T10:01:46.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.521+0000 7f4f03e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f4efc15c430 con 0x7f4efc144e90 2024-04-17T10:01:46.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.521+0000 7f4f03e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4efc145270 con 0x7f4efc0b4de0 2024-04-17T10:01:46.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.525+0000 7f4f01400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4efc144e90 0x7f4efc0b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:46.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.525+0000 7f4f01400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4efc144e90 0x7f4efc0b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52868/0 (socket says 172.21.15.32:52868) 2024-04-17T10:01:46.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.525+0000 7f4f01400640 1 -- 172.21.15.32:0/3991717483 learned_addr learned my addr 172.21.15.32:0/3991717483 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:46.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.525+0000 7f4f00a00640 1 --2- 172.21.15.32:0/3991717483 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4efc0b4de0 0x7f4efc15baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:46.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.525+0000 7f4f01400640 1 -- 172.21.15.32:0/3991717483 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4efc0b4de0 msgr2=0x7f4efc15baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:46.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.525+0000 7f4f01400640 1 --2- 172.21.15.32:0/3991717483 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4efc0b4de0 0x7f4efc15baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:46.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.525+0000 7f4f01400640 1 -- 172.21.15.32:0/3991717483 >> v1:172.21.15.144:6789/0 conn(0x7f4efc15bfe0 legacy=0x7f4efc06ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:46.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.525+0000 7f4f01400640 1 -- 172.21.15.32:0/3991717483 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4efc0b3dc0 con 0x7f4efc144e90 2024-04-17T10:01:46.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.525+0000 7f4f01400640 1 --2- 172.21.15.32:0/3991717483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4efc144e90 0x7f4efc0b48a0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00098a0 tx=0x7f4ef001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec51d94fb1e04405 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:46.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.525+0000 7f4efbe00640 1 -- 172.21.15.32:0/3991717483 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef00a95f0 con 0x7f4efc144e90 2024-04-17T10:01:46.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.525+0000 7f4efbe00640 1 -- 172.21.15.32:0/3991717483 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef00a9790 con 0x7f4efc144e90 2024-04-17T10:01:46.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.525+0000 7f4efbe00640 1 -- 172.21.15.32:0/3991717483 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef00a9b60 con 0x7f4efc144e90 2024-04-17T10:01:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.525+0000 7f4f03e00640 1 -- 172.21.15.32:0/3991717483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4efc144e90 msgr2=0x7f4efc0b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.525+0000 7f4f03e00640 1 --2- 172.21.15.32:0/3991717483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4efc144e90 0x7f4efc0b48a0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00098a0 tx=0x7f4ef001c510 comp rx=0 tx=0).stop 2024-04-17T10:01:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f03e00640 1 -- 172.21.15.32:0/3991717483 shutdown_connections 2024-04-17T10:01:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f03e00640 1 --2- 172.21.15.32:0/3991717483 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4efc0b4de0 0x7f4efc15baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f03e00640 1 --2- 172.21.15.32:0/3991717483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4efc144e90 0x7f4efc0b48a0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f03e00640 1 -- 172.21.15.32:0/3991717483 >> 172.21.15.32:0/3991717483 conn(0x7f4efc06ba30 msgr2=0x7f4efc06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f03e00640 1 -- 172.21.15.32:0/3991717483 shutdown_connections 2024-04-17T10:01:46.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f03e00640 1 -- 172.21.15.32:0/3991717483 wait complete. 2024-04-17T10:01:46.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f03e00640 1 Processor -- start 2024-04-17T10:01:46.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f03e00640 1 -- start start 2024-04-17T10:01:46.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f03e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4efc0b4de0 0x7f4efc1595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:46.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f03e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4efc144e90 0x7f4efc13c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:46.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f03e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4efc15bfe0 0x7f4efc13dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:46.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f03e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc072670 con 0x7f4efc0b4de0 2024-04-17T10:01:46.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f03e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc06f8c0 con 0x7f4efc144e90 2024-04-17T10:01:46.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f03e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4efc0727f0 con 0x7f4efc15bfe0 2024-04-17T10:01:46.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f01400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4efc0b4de0 0x7f4efc1595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f01400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4efc0b4de0 0x7f4efc1595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52880/0 (socket says 172.21.15.32:52880) 2024-04-17T10:01:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f01400640 1 -- 172.21.15.32:0/2834957889 learned_addr learned my addr 172.21.15.32:0/2834957889 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f00a00640 1 --2- 172.21.15.32:0/2834957889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4efc144e90 0x7f4efc13c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:46.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f01e00640 1 --2- 172.21.15.32:0/2834957889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4efc15bfe0 0x7f4efc13dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f01400640 1 -- 172.21.15.32:0/2834957889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4efc15bfe0 msgr2=0x7f4efc13dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f01400640 1 --2- 172.21.15.32:0/2834957889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4efc15bfe0 0x7f4efc13dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f01400640 1 -- 172.21.15.32:0/2834957889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4efc144e90 msgr2=0x7f4efc13c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f01400640 1 --2- 172.21.15.32:0/2834957889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4efc144e90 0x7f4efc13c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f01400640 1 -- 172.21.15.32:0/2834957889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4efc13cce0 con 0x7f4efc0b4de0 2024-04-17T10:01:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.529+0000 7f4f01400640 1 --2- 172.21.15.32:0/2834957889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4efc0b4de0 0x7f4efc1595b0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f4ef001a140 tx=0x7f4ef001ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.533+0000 7f4ef9600640 1 -- 172.21.15.32:0/2834957889 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef00b1800 con 0x7f4efc0b4de0 2024-04-17T10:01:46.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.533+0000 7f4ef9600640 1 -- 172.21.15.32:0/2834957889 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef00b19a0 con 0x7f4efc0b4de0 2024-04-17T10:01:46.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.533+0000 7f4ef9600640 1 -- 172.21.15.32:0/2834957889 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef00b1d90 con 0x7f4efc0b4de0 2024-04-17T10:01:46.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.533+0000 7f4f03e00640 1 -- 172.21.15.32:0/2834957889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4efc13cfc0 con 0x7f4efc0b4de0 2024-04-17T10:01:46.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.533+0000 7f4f03e00640 1 -- 172.21.15.32:0/2834957889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4efc13d580 con 0x7f4efc0b4de0 2024-04-17T10:01:46.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.533+0000 7f4f03e00640 1 -- 172.21.15.32:0/2834957889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4efc068a90 con 0x7f4efc0b4de0 2024-04-17T10:01:46.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.533+0000 7f4ef9600640 1 -- 172.21.15.32:0/2834957889 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4ef00bc450 con 0x7f4efc0b4de0 2024-04-17T10:01:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.533+0000 7f4ef9600640 1 --2- 172.21.15.32:0/2834957889 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4edc068380 0x7f4edc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.533+0000 7f4ef9600640 1 -- 172.21.15.32:0/2834957889 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f4ef0120f50 con 0x7f4efc0b4de0 2024-04-17T10:01:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.533+0000 7f4f00a00640 1 --2- 172.21.15.32:0/2834957889 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4edc068380 0x7f4edc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:46.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.533+0000 7f4f00a00640 1 --2- 172.21.15.32:0/2834957889 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4edc068380 0x7f4edc06a840 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4efc1f8580 tx=0x7f4ee8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:46.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.537+0000 7f4ef9600640 1 -- 172.21.15.32:0/2834957889 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f4ef00c3030 con 0x7f4efc0b4de0 2024-04-17T10:01:46.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.737+0000 7f4f03e00640 1 -- 172.21.15.32:0/2834957889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4efc0ccf80 con 0x7f4efc0b4de0 2024-04-17T10:01:46.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.741+0000 7f4ef9600640 1 -- 172.21.15.32:0/2834957889 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5207 (secure 0 0 0) 0x7f4ef00ad020 con 0x7f4efc0b4de0 2024-04-17T10:01:46.737 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:46.737 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":81,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2635109588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2635109588},{"type":"v1","addr":"172.21.15.144:6835","nonce":2635109588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7789,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3258455939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3258455939},{"type":"v1","addr":"172.21.15.144:6834","nonce":3258455939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":8574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3390468332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3390468332},{"type":"v1","addr":"172.21.15.32:6836","nonce":3390468332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:01:40.557208+0000","modified":"2024-04-17T10:01:43.570474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8577":{"gid":8577,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/1235126741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1235126741},{"type":"v1","addr":"172.21.15.32:6837","nonce":1235126741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8577,"qdb_cluster":[8577]},"id":7}]} 2024-04-17T10:01:46.737 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 81 2024-04-17T10:01:46.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.741+0000 7f4f03e00640 1 -- 172.21.15.32:0/2834957889 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4edc068380 msgr2=0x7f4edc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:46.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.741+0000 7f4f03e00640 1 --2- 172.21.15.32:0/2834957889 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4edc068380 0x7f4edc06a840 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4efc1f8580 tx=0x7f4ee8008040 comp rx=0 tx=0).stop 2024-04-17T10:01:46.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.741+0000 7f4f03e00640 1 -- 172.21.15.32:0/2834957889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4efc0b4de0 msgr2=0x7f4efc1595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:46.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.741+0000 7f4f03e00640 1 --2- 172.21.15.32:0/2834957889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4efc0b4de0 0x7f4efc1595b0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f4ef001a140 tx=0x7f4ef001ad50 comp rx=0 tx=0).stop 2024-04-17T10:01:46.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.741+0000 7f4f03e00640 1 -- 172.21.15.32:0/2834957889 shutdown_connections 2024-04-17T10:01:46.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.741+0000 7f4f03e00640 1 --2- 172.21.15.32:0/2834957889 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4edc068380 0x7f4edc06a840 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:46.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.741+0000 7f4f03e00640 1 --2- 172.21.15.32:0/2834957889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4efc15bfe0 0x7f4efc13dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:46.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.741+0000 7f4f03e00640 1 --2- 172.21.15.32:0/2834957889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4efc144e90 0x7f4efc13c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:46.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.741+0000 7f4f03e00640 1 --2- 172.21.15.32:0/2834957889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4efc0b4de0 0x7f4efc1595b0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:46.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.741+0000 7f4f03e00640 1 -- 172.21.15.32:0/2834957889 >> 172.21.15.32:0/2834957889 conn(0x7f4efc06ba30 msgr2=0x7f4efc06be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:46.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.741+0000 7f4f03e00640 1 -- 172.21.15.32:0/2834957889 shutdown_connections 2024-04-17T10:01:46.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:46.741+0000 7f4f03e00640 1 -- 172.21.15.32:0/2834957889 wait complete. 2024-04-17T10:01:47.161 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 81, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:01:40.557208+0000', 'modified': '2024-04-17T10:01:43.570474+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8577}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8577': {'gid': 8577, 'name': 'c', 'rank': 0, 'incarnation': 77, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6837/1235126741', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6835', 'nonce': 1235126741}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 1235126741}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8577, 'qdb_cluster': [8577]} 2024-04-17T10:01:47.161 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T10:01:47.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-17T10:01:47.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.309+0000 7f1b62000640 1 Processor -- start 2024-04-17T10:01:47.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.309+0000 7f1b62000640 1 -- start start 2024-04-17T10:01:47.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.309+0000 7f1b62000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1b5c155e50 0x7f1b5c15f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.309+0000 7f1b62000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b5c15f870 con 0x7f1b5c0b3340 2024-04-17T10:01:47.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.309+0000 7f1b62000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b5c15fb50 con 0x7f1b5c155a70 2024-04-17T10:01:47.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.309+0000 7f1b62000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1b5c0b42a0 con 0x7f1b5c155e50 2024-04-17T10:01:47.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.309+0000 7f1b5be00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1b5c0b3340 0x7f1b5c15c260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50480/0 (socket says 172.21.15.32:50480) 2024-04-17T10:01:47.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.309+0000 7f1b5be00640 1 -- 172.21.15.32:0/3111093230 learned_addr learned my addr 172.21.15.32:0/3111093230 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:47.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.309+0000 7f1b52a00640 1 --2- 172.21.15.32:0/3111093230 >> v2:172.21.15.144:3300/0 conn(0x7f1b5c155e50 0x7f1b5c15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:47.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.313+0000 7f1b5aa00640 1 -- 172.21.15.32:0/3111093230 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 85698225 0 0) 0x7f1b5c15f870 con 0x7f1b5c0b3340 2024-04-17T10:01:47.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.313+0000 7f1b5aa00640 1 -- 172.21.15.32:0/3111093230 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b40003530 con 0x7f1b5c0b3340 2024-04-17T10:01:47.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.313+0000 7f1b5aa00640 1 -- 172.21.15.32:0/3111093230 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2909712545 0 0) 0x7f1b40003530 con 0x7f1b5c0b3340 2024-04-17T10:01:47.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.313+0000 7f1b52a00640 1 -- 172.21.15.32:0/3111093230 >> v1:172.21.15.144:6789/0 conn(0x7f1b5c155a70 legacy=0x7f1b5c15d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:47.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.313+0000 7f1b52a00640 1 -- 172.21.15.32:0/3111093230 >> v1:172.21.15.32:6789/0 conn(0x7f1b5c0b3340 legacy=0x7f1b5c15c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:47.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.313+0000 7f1b52a00640 1 -- 172.21.15.32:0/3111093230 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5c15fe30 con 0x7f1b5c155e50 2024-04-17T10:01:47.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.313+0000 7f1b52a00640 1 --2- 172.21.15.32:0/3111093230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b5c155e50 0x7f1b5c15f190 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c0b4420 tx=0x7f1b4401c290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=201f5223d0168d02 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.313+0000 7f1b5aa00640 1 -- 172.21.15.32:0/3111093230 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b440b0070 con 0x7f1b5c155e50 2024-04-17T10:01:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.313+0000 7f1b5aa00640 1 -- 172.21.15.32:0/3111093230 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b440ab400 con 0x7f1b5c155e50 2024-04-17T10:01:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.313+0000 7f1b5aa00640 1 -- 172.21.15.32:0/3111093230 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b54001680 con 0x7f1b5c155e50 2024-04-17T10:01:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b62000640 1 -- 172.21.15.32:0/3111093230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b5c155e50 msgr2=0x7f1b5c15f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b62000640 1 --2- 172.21.15.32:0/3111093230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b5c155e50 0x7f1b5c15f190 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c0b4420 tx=0x7f1b4401c290 comp rx=0 tx=0).stop 2024-04-17T10:01:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b62000640 1 -- 172.21.15.32:0/3111093230 shutdown_connections 2024-04-17T10:01:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b62000640 1 --2- 172.21.15.32:0/3111093230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b5c155e50 0x7f1b5c15f190 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b62000640 1 -- 172.21.15.32:0/3111093230 >> 172.21.15.32:0/3111093230 conn(0x7f1b5c06ba30 msgr2=0x7f1b5c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b62000640 1 -- 172.21.15.32:0/3111093230 shutdown_connections 2024-04-17T10:01:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b62000640 1 -- 172.21.15.32:0/3111093230 wait complete. 2024-04-17T10:01:47.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b62000640 1 Processor -- start 2024-04-17T10:01:47.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b62000640 1 -- start start 2024-04-17T10:01:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b62000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b5c0b3340 0x7f1b5c146e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b62000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b5c155a70 0x7f1b5c0ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b62000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b5c155e50 0x7f1b5c150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b62000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c0d0170 con 0x7f1b5c155a70 2024-04-17T10:01:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b62000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c0d02f0 con 0x7f1b5c155e50 2024-04-17T10:01:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b62000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5c0d0470 con 0x7f1b5c0b3340 2024-04-17T10:01:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b52a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b5c155a70 0x7f1b5c0ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b52a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b5c155a70 0x7f1b5c0ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52896/0 (socket says 172.21.15.32:52896) 2024-04-17T10:01:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b52a00640 1 -- 172.21.15.32:0/2504225032 learned_addr learned my addr 172.21.15.32:0/2504225032 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:47.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b5b400640 1 --2- 172.21.15.32:0/2504225032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b5c0b3340 0x7f1b5c146e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:47.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b5be00640 1 --2- 172.21.15.32:0/2504225032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b5c155e50 0x7f1b5c150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:47.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b52a00640 1 -- 172.21.15.32:0/2504225032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b5c0b3340 msgr2=0x7f1b5c146e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:47.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b52a00640 1 --2- 172.21.15.32:0/2504225032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b5c0b3340 0x7f1b5c146e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b52a00640 1 -- 172.21.15.32:0/2504225032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b5c155e50 msgr2=0x7f1b5c150590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:47.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b52a00640 1 --2- 172.21.15.32:0/2504225032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b5c155e50 0x7f1b5c150590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.317+0000 7f1b52a00640 1 -- 172.21.15.32:0/2504225032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5c0ce880 con 0x7f1b5c155a70 2024-04-17T10:01:47.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.321+0000 7f1b52a00640 1 --2- 172.21.15.32:0/2504225032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b5c155a70 0x7f1b5c0ce050 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f1b44000c10 tx=0x7f1b4401aad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:47.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.321+0000 7f1b53e00640 1 -- 172.21.15.32:0/2504225032 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b440b0050 con 0x7f1b5c155a70 2024-04-17T10:01:47.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.321+0000 7f1b53e00640 1 -- 172.21.15.32:0/2504225032 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b440bc070 con 0x7f1b5c155a70 2024-04-17T10:01:47.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.321+0000 7f1b53e00640 1 -- 172.21.15.32:0/2504225032 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b440b7850 con 0x7f1b5c155a70 2024-04-17T10:01:47.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.321+0000 7f1b62000640 1 -- 172.21.15.32:0/2504225032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b5c0ceb60 con 0x7f1b5c155a70 2024-04-17T10:01:47.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.321+0000 7f1b62000640 1 -- 172.21.15.32:0/2504225032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b5c0ada00 con 0x7f1b5c155a70 2024-04-17T10:01:47.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.321+0000 7f1b53e00640 1 -- 172.21.15.32:0/2504225032 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1b440b79f0 con 0x7f1b5c155a70 2024-04-17T10:01:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.321+0000 7f1b53e00640 1 --2- 172.21.15.32:0/2504225032 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1b38068380 0x7f1b3806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.321+0000 7f1b53e00640 1 -- 172.21.15.32:0/2504225032 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f1b44121270 con 0x7f1b5c155a70 2024-04-17T10:01:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.321+0000 7f1b5b400640 1 --2- 172.21.15.32:0/2504225032 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1b38068380 0x7f1b3806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.321+0000 7f1b65453000 1 Processor -- start 2024-04-17T10:01:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b65453000 1 -- start start 2024-04-17T10:01:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b5b400640 1 --2- 172.21.15.32:0/2504225032 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1b38068380 0x7f1b3806a840 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0012f0 tx=0x7f1b4c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b65453000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad20eb50 0x5638ad20ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b65453000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638ad20f470 0x5638ad1c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b65453000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ad1c4cf0 0x5638ad1c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b65453000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5638ad0124d0 con 0x5638ad20eb50 2024-04-17T10:01:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b65453000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5638acfacbf0 con 0x5638ad1c4cf0 2024-04-17T10:01:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b65453000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5638ad00e490 con 0x5638ad20f470 2024-04-17T10:01:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b5b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad20eb50 0x5638ad20ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b5b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad20eb50 0x5638ad20ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52904/0 (socket says 172.21.15.32:52904) 2024-04-17T10:01:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b5b400640 1 -- 172.21.15.32:0/1590957217 learned_addr learned my addr 172.21.15.32:0/1590957217 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b5b400640 1 -- 172.21.15.32:0/1590957217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638ad20f470 msgr2=0x5638ad1c47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b5b400640 1 --2- 172.21.15.32:0/1590957217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638ad20f470 0x5638ad1c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b5b400640 1 -- 172.21.15.32:0/1590957217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ad1c4cf0 msgr2=0x5638ad1c50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:47.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b5b400640 1 --2- 172.21.15.32:0/1590957217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ad1c4cf0 0x5638ad1c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b5b400640 1 -- 172.21.15.32:0/1590957217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638acf98e80 con 0x5638ad20eb50 2024-04-17T10:01:47.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b5b400640 1 --2- 172.21.15.32:0/1590957217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad20eb50 0x5638ad20ef30 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c001d10 tx=0x7f1b4c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea047f097252576d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b4be00640 1 -- 172.21.15.32:0/1590957217 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c0053a0 con 0x5638ad20eb50 2024-04-17T10:01:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b4be00640 1 -- 172.21.15.32:0/1590957217 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b4c005540 con 0x5638ad20eb50 2024-04-17T10:01:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b4be00640 1 -- 172.21.15.32:0/1590957217 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c021450 con 0x5638ad20eb50 2024-04-17T10:01:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b65453000 1 -- 172.21.15.32:0/1590957217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad20eb50 msgr2=0x5638ad20ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b65453000 1 --2- 172.21.15.32:0/1590957217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad20eb50 0x5638ad20ef30 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c001d10 tx=0x7f1b4c011000 comp rx=0 tx=0).stop 2024-04-17T10:01:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b65453000 1 -- 172.21.15.32:0/1590957217 shutdown_connections 2024-04-17T10:01:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b65453000 1 --2- 172.21.15.32:0/1590957217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638ad20f470 0x5638ad1c47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b65453000 1 --2- 172.21.15.32:0/1590957217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ad1c4cf0 0x5638ad1c50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b65453000 1 --2- 172.21.15.32:0/1590957217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad20eb50 0x5638ad20ef30 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b65453000 1 -- 172.21.15.32:0/1590957217 >> 172.21.15.32:0/1590957217 conn(0x5638ad1ca470 msgr2=0x5638ad1c3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b65453000 1 -- 172.21.15.32:0/1590957217 shutdown_connections 2024-04-17T10:01:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b65453000 1 -- 172.21.15.32:0/1590957217 wait complete. 2024-04-17T10:01:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b65453000 1 -- start start 2024-04-17T10:01:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.325+0000 7f1b65453000 1 Processor -- start 2024-04-17T10:01:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b65453000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ad1c4cf0 0x5638ad1c9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b65453000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad20eb50 0x5638ad1fc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b65453000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638ad20f470 0x5638ad1fc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b65453000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5638acfcfa60 con 0x5638ad20eb50 2024-04-17T10:01:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b65453000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5638acfb6e80 con 0x5638ad1c4cf0 2024-04-17T10:01:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b65453000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5638acfcdcf0 con 0x5638ad20f470 2024-04-17T10:01:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b65453000 10 client.0 _ll_register_callbacks cb 0x5638ad1afa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:01:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b52a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad20eb50 0x5638ad1fc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b52a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad20eb50 0x5638ad1fc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52920/0 (socket says 172.21.15.32:52920) 2024-04-17T10:01:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b52a00640 1 -- 172.21.15.32:0/4233385807 learned_addr learned my addr 172.21.15.32:0/4233385807 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b5b400640 1 --2- 172.21.15.32:0/4233385807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ad1c4cf0 0x5638ad1c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b5be00640 1 --2- 172.21.15.32:0/4233385807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638ad20f470 0x5638ad1fc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b52a00640 1 -- 172.21.15.32:0/4233385807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638ad20f470 msgr2=0x5638ad1fc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b52a00640 1 --2- 172.21.15.32:0/4233385807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638ad20f470 0x5638ad1fc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b52a00640 1 -- 172.21.15.32:0/4233385807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ad1c4cf0 msgr2=0x5638ad1c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b52a00640 1 --2- 172.21.15.32:0/4233385807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ad1c4cf0 0x5638ad1c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b52a00640 1 -- 172.21.15.32:0/4233385807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638acface40 con 0x5638ad20eb50 2024-04-17T10:01:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b52a00640 1 --2- 172.21.15.32:0/4233385807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad20eb50 0x5638ad1fc440 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f1b440f1020 tx=0x7f1b440f09f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39bec58ed03e06de server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b48c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:01:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b48c00640 1 -- 172.21.15.32:0/4233385807 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b440f1090 con 0x5638ad20eb50 2024-04-17T10:01:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b48c00640 1 -- 172.21.15.32:0/4233385807 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f1b440f5020 con 0x5638ad20eb50 2024-04-17T10:01:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b65453000 1 -- 172.21.15.32:0/4233385807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b4c005540 con 0x5638ad20eb50 2024-04-17T10:01:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b48c00640 1 -- 172.21.15.32:0/4233385807 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f1b440f5240 con 0x5638ad20eb50 2024-04-17T10:01:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b65453000 10 client.8700 fetch_fsmap learned FSMap version 81 2024-04-17T10:01:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b65453000 1 -- 172.21.15.32:0/4233385807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x5638acfdde00 con 0x5638ad20eb50 2024-04-17T10:01:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b48c00640 1 -- 172.21.15.32:0/4233385807 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b4c005540 con 0x5638ad20eb50 2024-04-17T10:01:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b48c00640 1 -- 172.21.15.32:0/4233385807 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 81) ==== 3495+0+0 (secure 0 0 0) 0x7f1b440b7ce0 con 0x5638ad20eb50 2024-04-17T10:01:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b65453000 10 client.8700 fetch_fsmap finished waiting for FSMap version 81 2024-04-17T10:01:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b65453000 10 client.8700 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T10:01:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b65453000 20 client.8700 populate_metadata read hostname 'smithi032' 2024-04-17T10:01:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b65453000 1 --2- 172.21.15.32:0/4233385807 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x5638ad21e0b0 0x5638ad2204a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b65453000 4 client.8700 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:01:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b65453000 1 -- 172.21.15.32:0/4233385807 --> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638acfabbc0 con 0x5638ad21e0b0 2024-04-17T10:01:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b5b400640 1 --2- 172.21.15.32:0/4233385807 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x5638ad21e0b0 0x5638ad2204a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b5b400640 1 --2- 172.21.15.32:0/4233385807 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x5638ad21e0b0 0x5638ad2204a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=794d8db33fef950f server_cookie=77a3d085e9109bb3 in_seq=0 out_seq=0 2024-04-17T10:01:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b48c00640 10 client.8700 ms_handle_connect on v2:172.21.15.32:6835/1235126741 2024-04-17T10:01:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b48c00640 1 -- 172.21.15.32:0/4233385807 <== mds.0 v2:172.21.15.32:6835/1235126741 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5638acfabbc0 con 0x5638ad21e0b0 2024-04-17T10:01:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.329+0000 7f1b48c00640 10 client.8700 handle_command_reply: tid=0 2024-04-17T10:01:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.333+0000 7f1b65453000 1 client.8700 shutdown 2024-04-17T10:01:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.333+0000 7f1b65453000 1 -- 172.21.15.32:0/4233385807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad20eb50 msgr2=0x5638ad1fc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.333+0000 7f1b65453000 1 --2- 172.21.15.32:0/4233385807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad20eb50 0x5638ad1fc440 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f1b440f1020 tx=0x7f1b440f09f0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.333+0000 7f1b65453000 1 -- 172.21.15.32:0/4233385807 shutdown_connections 2024-04-17T10:01:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.333+0000 7f1b65453000 1 --2- 172.21.15.32:0/4233385807 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x5638ad21e0b0 0x5638ad2204a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.333+0000 7f1b65453000 1 --2- 172.21.15.32:0/4233385807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638ad20f470 0x5638ad1fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.333+0000 7f1b65453000 1 --2- 172.21.15.32:0/4233385807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad20eb50 0x5638ad1fc440 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.333+0000 7f1b48c00640 0 client.8700 ms_handle_reset on v2:172.21.15.32:6835/1235126741 2024-04-17T10:01:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.333+0000 7f1b65453000 1 --2- 172.21.15.32:0/4233385807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ad1c4cf0 0x5638ad1c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.333+0000 7f1b65453000 1 -- 172.21.15.32:0/4233385807 >> 172.21.15.32:0/4233385807 conn(0x5638ad1ca470 msgr2=0x5638ad1c37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.333+0000 7f1b65453000 1 -- 172.21.15.32:0/4233385807 shutdown_connections 2024-04-17T10:01:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.333+0000 7f1b65453000 1 -- 172.21.15.32:0/4233385807 wait complete. 2024-04-17T10:01:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.333+0000 7f1b65453000 20 client.8700 trim_cache size 0 max 16384 2024-04-17T10:01:47.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.349+0000 7f1b65453000 1 Processor -- start 2024-04-17T10:01:47.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.349+0000 7f1b65453000 1 -- start start 2024-04-17T10:01:47.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.349+0000 7f1b65453000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638ad1c4cf0 0x5638ad1fc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.349+0000 7f1b65453000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ad20eb50 0x5638ad1fc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.349+0000 7f1b65453000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad20f470 0x5638ad21f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.349+0000 7f1b65453000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c202d70 con 0x5638ad20f470 2024-04-17T10:01:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.349+0000 7f1b65453000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c2020d0 con 0x5638ad20eb50 2024-04-17T10:01:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.349+0000 7f1b65453000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5c202720 con 0x5638ad1c4cf0 2024-04-17T10:01:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.349+0000 7f1b5be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad20f470 0x5638ad21f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.349+0000 7f1b5be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad20f470 0x5638ad21f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52922/0 (socket says 172.21.15.32:52922) 2024-04-17T10:01:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.349+0000 7f1b5be00640 1 -- 172.21.15.32:0/1399503204 learned_addr learned my addr 172.21.15.32:0/1399503204 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.349+0000 7f1b5b400640 1 --2- 172.21.15.32:0/1399503204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638ad1c4cf0 0x5638ad1fc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.349+0000 7f1b52a00640 1 --2- 172.21.15.32:0/1399503204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ad20eb50 0x5638ad1fc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.349+0000 7f1b5be00640 1 -- 172.21.15.32:0/1399503204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638ad1c4cf0 msgr2=0x5638ad1fc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:47.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.349+0000 7f1b5be00640 1 --2- 172.21.15.32:0/1399503204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638ad1c4cf0 0x5638ad1fc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.349+0000 7f1b5be00640 1 -- 172.21.15.32:0/1399503204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ad20eb50 msgr2=0x5638ad1fc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:47.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.349+0000 7f1b5be00640 1 --2- 172.21.15.32:0/1399503204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ad20eb50 0x5638ad1fc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.349+0000 7f1b5be00640 1 -- 172.21.15.32:0/1399503204 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638ad093d30 con 0x5638ad20f470 2024-04-17T10:01:47.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b5be00640 1 --2- 172.21.15.32:0/1399503204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad20f470 0x5638ad21f4a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f1b540017b0 tx=0x7f1b54000f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ddf58e1ed0dc4cf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:47.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b4b400640 1 -- 172.21.15.32:0/1399503204 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b540109c0 con 0x5638ad20f470 2024-04-17T10:01:47.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b4b400640 1 -- 172.21.15.32:0/1399503204 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b54019070 con 0x5638ad20f470 2024-04-17T10:01:47.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b4b400640 1 -- 172.21.15.32:0/1399503204 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b54010da0 con 0x5638ad20f470 2024-04-17T10:01:47.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b65453000 1 -- 172.21.15.32:0/1399503204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad20f470 msgr2=0x5638ad21f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:47.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b65453000 1 --2- 172.21.15.32:0/1399503204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad20f470 0x5638ad21f4a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f1b540017b0 tx=0x7f1b54000f90 comp rx=0 tx=0).stop 2024-04-17T10:01:47.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b65453000 1 -- 172.21.15.32:0/1399503204 shutdown_connections 2024-04-17T10:01:47.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b65453000 1 --2- 172.21.15.32:0/1399503204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad20f470 0x5638ad21f4a0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b65453000 1 --2- 172.21.15.32:0/1399503204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ad20eb50 0x5638ad1fc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b65453000 1 --2- 172.21.15.32:0/1399503204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638ad1c4cf0 0x5638ad1fc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b65453000 1 -- 172.21.15.32:0/1399503204 >> 172.21.15.32:0/1399503204 conn(0x5638ad1ca470 msgr2=0x5638ad1fb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:47.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b65453000 1 -- 172.21.15.32:0/1399503204 shutdown_connections 2024-04-17T10:01:47.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b65453000 1 -- 172.21.15.32:0/1399503204 wait complete. 2024-04-17T10:01:47.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b65453000 1 -- start start 2024-04-17T10:01:47.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b65453000 1 Processor -- start 2024-04-17T10:01:47.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b65453000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad1c4cf0 0x5638ad222ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b65453000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638ad20eb50 0x5638ad23a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b65453000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ad20f470 0x5638ad23a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b65453000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5638ad016b00 con 0x5638ad1c4cf0 2024-04-17T10:01:47.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b65453000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5638ad0117e0 con 0x5638ad20f470 2024-04-17T10:01:47.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b65453000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5638ad01f6e0 con 0x5638ad20eb50 2024-04-17T10:01:47.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b65453000 10 client.0 _ll_register_callbacks cb 0x7f1b44006c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:01:47.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b5b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad1c4cf0 0x5638ad222ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:47.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b5b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad1c4cf0 0x5638ad222ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52930/0 (socket says 172.21.15.32:52930) 2024-04-17T10:01:47.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b5b400640 1 -- 172.21.15.32:0/1799096056 learned_addr learned my addr 172.21.15.32:0/1799096056 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:47.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b5be00640 1 --2- 172.21.15.32:0/1799096056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ad20f470 0x5638ad23a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:47.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b52a00640 1 --2- 172.21.15.32:0/1799096056 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638ad20eb50 0x5638ad23a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:47.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b5b400640 1 -- 172.21.15.32:0/1799096056 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638ad20eb50 msgr2=0x5638ad23a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:47.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b5b400640 1 --2- 172.21.15.32:0/1799096056 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638ad20eb50 0x5638ad23a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b5b400640 1 -- 172.21.15.32:0/1799096056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ad20f470 msgr2=0x5638ad23a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:47.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b5b400640 1 --2- 172.21.15.32:0/1799096056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ad20f470 0x5638ad23a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b5b400640 1 -- 172.21.15.32:0/1799096056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638acfa47c0 con 0x5638ad1c4cf0 2024-04-17T10:01:47.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b5be00640 1 --2- 172.21.15.32:0/1799096056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ad20f470 0x5638ad23a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:47.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b5b400640 1 --2- 172.21.15.32:0/1799096056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad1c4cf0 0x5638ad222ac0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x5638ad1908c0 tx=0x7f1b4c001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24172827483e3897 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:47.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b48c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:01:47.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.353+0000 7f1b48c00640 1 -- 172.21.15.32:0/1799096056 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c037060 con 0x5638ad1c4cf0 2024-04-17T10:01:47.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b48c00640 1 -- 172.21.15.32:0/1799096056 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f1b4c010040 con 0x5638ad1c4cf0 2024-04-17T10:01:47.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b65453000 1 -- 172.21.15.32:0/1799096056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b54019070 con 0x5638ad1c4cf0 2024-04-17T10:01:47.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b48c00640 1 -- 172.21.15.32:0/1799096056 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f1b4c0310e0 con 0x5638ad1c4cf0 2024-04-17T10:01:47.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b65453000 10 client.8706 fetch_fsmap learned FSMap version 81 2024-04-17T10:01:47.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b65453000 1 -- 172.21.15.32:0/1799096056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x5638acfb7650 con 0x5638ad1c4cf0 2024-04-17T10:01:47.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b48c00640 1 -- 172.21.15.32:0/1799096056 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b54019070 con 0x5638ad1c4cf0 2024-04-17T10:01:47.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b48c00640 1 -- 172.21.15.32:0/1799096056 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 81) ==== 3495+0+0 (secure 0 0 0) 0x7f1b4c02c070 con 0x5638ad1c4cf0 2024-04-17T10:01:47.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b65453000 10 client.8706 fetch_fsmap finished waiting for FSMap version 81 2024-04-17T10:01:47.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b65453000 10 client.8706 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T10:01:47.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b65453000 20 client.8706 populate_metadata read hostname 'smithi032' 2024-04-17T10:01:47.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b65453000 1 --2- 172.21.15.32:0/1799096056 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x5638ad27d670 0x5638ad27da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b65453000 4 client.8706 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T10:01:47.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b65453000 1 -- 172.21.15.32:0/1799096056 --> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] -- command(tid 0: {"prefix": "status"}) -- 0x5638ad01fef0 con 0x5638ad27d670 2024-04-17T10:01:47.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b52a00640 1 --2- 172.21.15.32:0/1799096056 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x5638ad27d670 0x5638ad27da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:47.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b52a00640 1 --2- 172.21.15.32:0/1799096056 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x5638ad27d670 0x5638ad27da50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50427f493a5f0f9 server_cookie=d5f183d7ee4b1061 in_seq=0 out_seq=0 2024-04-17T10:01:47.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b48c00640 10 client.8706 ms_handle_connect on v2:172.21.15.32:6835/1235126741 2024-04-17T10:01:47.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b48c00640 1 -- 172.21.15.32:0/1799096056 <== mds.0 v2:172.21.15.32:6835/1235126741 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5638ad01fef0 con 0x5638ad27d670 2024-04-17T10:01:47.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b48c00640 10 client.8706 handle_command_reply: tid=0 2024-04-17T10:01:47.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b65453000 1 client.8706 shutdown 2024-04-17T10:01:47.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b65453000 1 -- 172.21.15.32:0/1799096056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad1c4cf0 msgr2=0x5638ad222ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:47.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b65453000 1 --2- 172.21.15.32:0/1799096056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad1c4cf0 0x5638ad222ac0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x5638ad1908c0 tx=0x7f1b4c001260 comp rx=0 tx=0).stop 2024-04-17T10:01:47.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b65453000 1 -- 172.21.15.32:0/1799096056 shutdown_connections 2024-04-17T10:01:47.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b65453000 1 --2- 172.21.15.32:0/1799096056 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x5638ad27d670 0x5638ad27da50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b65453000 1 --2- 172.21.15.32:0/1799096056 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638ad20eb50 0x5638ad23a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b65453000 1 --2- 172.21.15.32:0/1799096056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ad20f470 0x5638ad23a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b65453000 1 --2- 172.21.15.32:0/1799096056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5638ad1c4cf0 0x5638ad222ac0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b65453000 1 -- 172.21.15.32:0/1799096056 >> 172.21.15.32:0/1799096056 conn(0x5638ad1ca470 msgr2=0x5638ad1c6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:47.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b48c00640 0 client.8706 ms_handle_reset on v2:172.21.15.32:6835/1235126741 2024-04-17T10:01:47.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b65453000 1 -- 172.21.15.32:0/1799096056 shutdown_connections 2024-04-17T10:01:47.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b65453000 1 -- 172.21.15.32:0/1799096056 wait complete. 2024-04-17T10:01:47.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b65453000 20 client.8706 trim_cache size 0 max 16384 2024-04-17T10:01:47.359 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:01:47.359 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "ff178df8-4c06-4367-a1e9-2e3a66d870d0", 2024-04-17T10:01:47.359 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-17T10:01:47.359 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 8577, 2024-04-17T10:01:47.359 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-17T10:01:47.360 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-17T10:01:47.360 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-17T10:01:47.360 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 6.2141433599999996, 2024-04-17T10:01:47.360 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 81, 2024-04-17T10:01:47.360 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 97, 2024-04-17T10:01:47.360 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 97, 2024-04-17T10:01:47.360 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 11.035818372 2024-04-17T10:01:47.360 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:01:47.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b62000640 1 -- 172.21.15.32:0/2504225032 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1b38068380 msgr2=0x7f1b3806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:47.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b62000640 1 --2- 172.21.15.32:0/2504225032 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1b38068380 0x7f1b3806a840 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0012f0 tx=0x7f1b4c009040 comp rx=0 tx=0).stop 2024-04-17T10:01:47.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b62000640 1 -- 172.21.15.32:0/2504225032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b5c155a70 msgr2=0x7f1b5c0ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:47.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b62000640 1 --2- 172.21.15.32:0/2504225032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b5c155a70 0x7f1b5c0ce050 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f1b44000c10 tx=0x7f1b4401aad0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b62000640 1 -- 172.21.15.32:0/2504225032 shutdown_connections 2024-04-17T10:01:47.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b62000640 1 --2- 172.21.15.32:0/2504225032 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1b38068380 0x7f1b3806a840 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b62000640 1 --2- 172.21.15.32:0/2504225032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b5c155e50 0x7f1b5c150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b62000640 1 --2- 172.21.15.32:0/2504225032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b5c155a70 0x7f1b5c0ce050 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b62000640 1 --2- 172.21.15.32:0/2504225032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b5c0b3340 0x7f1b5c146e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b62000640 1 -- 172.21.15.32:0/2504225032 >> 172.21.15.32:0/2504225032 conn(0x7f1b5c06ba30 msgr2=0x7f1b5c15e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:47.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b62000640 1 -- 172.21.15.32:0/2504225032 shutdown_connections 2024-04-17T10:01:47.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.357+0000 7f1b62000640 1 -- 172.21.15.32:0/2504225032 wait complete. 2024-04-17T10:01:47.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T10:01:47.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.517+0000 7fb427200640 1 Processor -- start 2024-04-17T10:01:47.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.517+0000 7fb427200640 1 -- start start 2024-04-17T10:01:47.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.517+0000 7fb427200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb420145f30 0x7fb420071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.517+0000 7fb427200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4200cd580 0x7fb420072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.517+0000 7fb427200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4200cdac0 con 0x7fb42015d8d0 2024-04-17T10:01:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.517+0000 7fb427200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb42006f400 con 0x7fb4200cd580 2024-04-17T10:01:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.517+0000 7fb427200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb420146310 con 0x7fb420145f30 2024-04-17T10:01:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.517+0000 7fb425200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4200cd580 0x7fb420072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.517+0000 7fb425200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4200cd580 0x7fb420072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52932/0 (socket says 172.21.15.32:52932) 2024-04-17T10:01:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.517+0000 7fb425200640 1 -- 172.21.15.32:0/2289930485 learned_addr learned my addr 172.21.15.32:0/2289930485 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.517+0000 7fb41fe00640 1 --2- 172.21.15.32:0/2289930485 >> v2:172.21.15.144:3300/0 conn(0x7fb420145f30 0x7fb420071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:47.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.517+0000 7fb425200640 1 -- 172.21.15.32:0/2289930485 >> v2:172.21.15.144:3300/0 conn(0x7fb420145f30 msgr2=0x7fb420071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:47.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.517+0000 7fb425200640 1 --2- 172.21.15.32:0/2289930485 >> v2:172.21.15.144:3300/0 conn(0x7fb420145f30 0x7fb420071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.517+0000 7fb425200640 1 -- 172.21.15.32:0/2289930485 >> v1:172.21.15.144:6789/0 conn(0x7fb42015d8d0 legacy=0x7fb4200cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:47.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.517+0000 7fb425200640 1 -- 172.21.15.32:0/2289930485 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42015cdf0 con 0x7fb4200cd580 2024-04-17T10:01:47.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.521+0000 7fb425200640 1 --2- 172.21.15.32:0/2289930485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4200cd580 0x7fb420072330 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fb42006ff00 tx=0x7fb41801f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8805c00e431d552 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:47.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.521+0000 7fb41f400640 1 -- 172.21.15.32:0/2289930485 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4180b4070 con 0x7fb4200cd580 2024-04-17T10:01:47.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.521+0000 7fb41f400640 1 -- 172.21.15.32:0/2289930485 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4180af400 con 0x7fb4200cd580 2024-04-17T10:01:47.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.521+0000 7fb41f400640 1 -- 172.21.15.32:0/2289930485 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4180af7d0 con 0x7fb4200cd580 2024-04-17T10:01:47.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.521+0000 7fb427200640 1 -- 172.21.15.32:0/2289930485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4200cd580 msgr2=0x7fb420072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:47.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.521+0000 7fb427200640 1 --2- 172.21.15.32:0/2289930485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4200cd580 0x7fb420072330 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fb42006ff00 tx=0x7fb41801f1e0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.521+0000 7fb427200640 1 -- 172.21.15.32:0/2289930485 shutdown_connections 2024-04-17T10:01:47.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.521+0000 7fb427200640 1 --2- 172.21.15.32:0/2289930485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4200cd580 0x7fb420072330 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.521+0000 7fb427200640 1 --2- 172.21.15.32:0/2289930485 >> v2:172.21.15.144:3300/0 conn(0x7fb420145f30 0x7fb420071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.521+0000 7fb427200640 1 -- 172.21.15.32:0/2289930485 >> 172.21.15.32:0/2289930485 conn(0x7fb42006ba30 msgr2=0x7fb42006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:47.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.521+0000 7fb427200640 1 -- 172.21.15.32:0/2289930485 shutdown_connections 2024-04-17T10:01:47.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.521+0000 7fb427200640 1 -- 172.21.15.32:0/2289930485 wait complete. 2024-04-17T10:01:47.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb427200640 1 Processor -- start 2024-04-17T10:01:47.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb427200640 1 -- start start 2024-04-17T10:01:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb427200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4200cd580 0x7fb42015bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb427200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb420145f30 0x7fb420131620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb427200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb42015d8d0 0x7fb4201f7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb427200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb42015cb90 con 0x7fb42015d8d0 2024-04-17T10:01:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb427200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb42015d440 con 0x7fb4200cd580 2024-04-17T10:01:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb427200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb4201601d0 con 0x7fb420145f30 2024-04-17T10:01:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb425200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb42015d8d0 0x7fb4201f7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb425200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb42015d8d0 0x7fb4201f7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52940/0 (socket says 172.21.15.32:52940) 2024-04-17T10:01:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb425200640 1 -- 172.21.15.32:0/1312102124 learned_addr learned my addr 172.21.15.32:0/1312102124 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb417e00640 1 --2- 172.21.15.32:0/1312102124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb420145f30 0x7fb420131620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb41fe00640 1 --2- 172.21.15.32:0/1312102124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4200cd580 0x7fb42015bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb425200640 1 -- 172.21.15.32:0/1312102124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb420145f30 msgr2=0x7fb420131620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb425200640 1 --2- 172.21.15.32:0/1312102124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb420145f30 0x7fb420131620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb425200640 1 -- 172.21.15.32:0/1312102124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4200cd580 msgr2=0x7fb42015bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb425200640 1 --2- 172.21.15.32:0/1312102124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4200cd580 0x7fb42015bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb425200640 1 -- 172.21.15.32:0/1312102124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4200b5090 con 0x7fb42015d8d0 2024-04-17T10:01:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb425200640 1 --2- 172.21.15.32:0/1312102124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb42015d8d0 0x7fb4201f7fd0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fb4180b2040 tx=0x7fb41801bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb41cc00640 1 -- 172.21.15.32:0/1312102124 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4180b4040 con 0x7fb42015d8d0 2024-04-17T10:01:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb41cc00640 1 -- 172.21.15.32:0/1312102124 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4180c0070 con 0x7fb42015d8d0 2024-04-17T10:01:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb41cc00640 1 -- 172.21.15.32:0/1312102124 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb418009510 con 0x7fb42015d8d0 2024-04-17T10:01:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb427200640 1 -- 172.21.15.32:0/1312102124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb420131fe0 con 0x7fb42015d8d0 2024-04-17T10:01:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.525+0000 7fb427200640 1 -- 172.21.15.32:0/1312102124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4201325a0 con 0x7fb42015d8d0 2024-04-17T10:01:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.529+0000 7fb427200640 1 -- 172.21.15.32:0/1312102124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb420068a90 con 0x7fb42015d8d0 2024-04-17T10:01:47.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.529+0000 7fb41cc00640 1 -- 172.21.15.32:0/1312102124 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb4180c5460 con 0x7fb42015d8d0 2024-04-17T10:01:47.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.529+0000 7fb41cc00640 1 --2- 172.21.15.32:0/1312102124 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb3fc068380 0x7fb3fc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:47.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.529+0000 7fb41cc00640 1 -- 172.21.15.32:0/1312102124 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fb418124d70 con 0x7fb42015d8d0 2024-04-17T10:01:47.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.529+0000 7fb41fe00640 1 --2- 172.21.15.32:0/1312102124 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb3fc068380 0x7fb3fc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:47.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.529+0000 7fb41fe00640 1 --2- 172.21.15.32:0/1312102124 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb3fc068380 0x7fb3fc06a840 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb4200b28b0 tx=0x7fb410008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:47.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.533+0000 7fb41cc00640 1 -- 172.21.15.32:0/1312102124 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fb418018d40 con 0x7fb42015d8d0 2024-04-17T10:01:47.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.733+0000 7fb427200640 1 -- 172.21.15.32:0/1312102124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb420072f90 con 0x7fb42015d8d0 2024-04-17T10:01:47.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.733+0000 7fb41cc00640 1 -- 172.21.15.32:0/1312102124 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v81) ==== 53+0+73 (secure 0 0 0) 0x7fb4180f5290 con 0x7fb42015d8d0 2024-04-17T10:01:47.731 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T10:01:47.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.737+0000 7fb427200640 1 -- 172.21.15.32:0/1312102124 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb3fc068380 msgr2=0x7fb3fc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.737+0000 7fb427200640 1 --2- 172.21.15.32:0/1312102124 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb3fc068380 0x7fb3fc06a840 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb4200b28b0 tx=0x7fb410008040 comp rx=0 tx=0).stop 2024-04-17T10:01:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.737+0000 7fb427200640 1 -- 172.21.15.32:0/1312102124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb42015d8d0 msgr2=0x7fb4201f7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.737+0000 7fb427200640 1 --2- 172.21.15.32:0/1312102124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb42015d8d0 0x7fb4201f7fd0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fb4180b2040 tx=0x7fb41801bf80 comp rx=0 tx=0).stop 2024-04-17T10:01:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.737+0000 7fb427200640 1 -- 172.21.15.32:0/1312102124 shutdown_connections 2024-04-17T10:01:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.737+0000 7fb427200640 1 --2- 172.21.15.32:0/1312102124 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb3fc068380 0x7fb3fc06a840 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.737+0000 7fb427200640 1 --2- 172.21.15.32:0/1312102124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb420145f30 0x7fb420131620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.737+0000 7fb427200640 1 --2- 172.21.15.32:0/1312102124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb42015d8d0 0x7fb4201f7fd0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.737+0000 7fb427200640 1 --2- 172.21.15.32:0/1312102124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4200cd580 0x7fb42015bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.737+0000 7fb427200640 1 -- 172.21.15.32:0/1312102124 >> 172.21.15.32:0/1312102124 conn(0x7fb42006ba30 msgr2=0x7fb42015f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.737+0000 7fb427200640 1 -- 172.21.15.32:0/1312102124 shutdown_connections 2024-04-17T10:01:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47.737+0000 7fb427200640 1 -- 172.21.15.32:0/1312102124 wait complete. 2024-04-17T10:01:47.751 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T10:01:47.751 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T10:01:47.751 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-17T10:01:47.751 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T10:01:47.752 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T10:01:47.752 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T10:01:47.752 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T10:01:47.752 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T10:01:47.752 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:01:47.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:01:47.752 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-17T10:01:47.761 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:01:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:01:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:01:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:01:47.761 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-17T10:01:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:01:47.762 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:01:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-17T10:01:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-17T10:01:47.762 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T10:01:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:01:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-17T10:01:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-17T10:01:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 41946sec preferred_lft 41946sec 2024-04-17T10:01:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2024-04-17T10:01:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:01:47.762 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:01:47.762 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-17T10:01:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-17T10:01:47.763 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:01:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:01:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-17T10:01:47.763 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T10:01:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:cd:f4:aa:5b brd ff:ff:ff:ff:ff:ff 2024-04-17T10:01:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T10:01:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:01:47.763 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T10:01:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T10:01:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T10:01:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:01:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T10:01:47.763 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:01:47.764 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:01:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:01:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T10:01:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:01:47.764 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:01:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:01:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T10:01:47.764 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:01:47.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:01:47.765 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T10:01:47.826 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-17T10:01:47.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:01:47.828 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-17T10:01:47.881 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-17T10:01:47.881 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T10:01:47.881 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-17T10:01:47.881 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T10:01:47.882 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-17T10:01:47.882 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T10:01:47.882 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T10:01:47.882 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T10:01:47.882 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-17T10:01:47.882 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-17T10:01:47.882 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-17T10:01:47.882 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T10:01:47.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T10:01:48.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:01:48.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:01:48.022 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-17T10:01:48.030 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T10:01:48.030 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T10:01:48.031 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:01:48.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:01:48.031 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-17T10:01:48.092 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T10:01:48.092 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:01:48.092 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T10:01:48.144 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:01:48.144 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:01:48.144 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:01:48.195 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T10:01:48.257 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:01:48.310 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T10:01:48.310 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T10:01:48.310 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T10:01:48.310 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T10:01:48.310 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T10:01:48.310 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T10:01:48.310 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T10:01:48.310 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T10:01:48.310 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T10:01:48.310 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T10:01:48.310 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T10:01:48.311 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T10:01:48.311 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:01:48.311 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T10:01:48.311 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T10:01:48.311 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T10:01:48.311 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T10:01:48.311 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T10:01:48.311 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T10:01:48.311 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T10:01:48.311 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:01:48.311 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T10:01:48.311 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T10:01:48.311 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T10:01:48.312 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:01:48.312 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:01:48.312 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:01:48.312 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:01:48.312 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T10:01:48.312 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:01:48.312 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:01:48.312 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:01:48.312 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:01:48.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:01:48.313 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T10:01:48.368 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T10:01:48.368 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T10:01:48.411 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T10:01:48.432 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:01:48.485 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T10:01:48.486 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T10:01:48.486 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T10:01:48.486 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T10:01:48.486 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T10:01:48.486 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T10:01:48.486 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T10:01:48.486 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T10:01:48.486 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T10:01:48.486 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T10:01:48.486 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T10:01:48.486 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T10:01:48.486 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:01:48.486 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T10:01:48.487 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T10:01:48.487 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T10:01:48.487 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T10:01:48.487 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T10:01:48.487 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T10:01:48.487 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T10:01:48.487 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:01:48.487 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T10:01:48.487 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T10:01:48.487 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T10:01:48.487 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:01:48.487 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:01:48.487 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:01:48.487 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:01:48.488 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T10:01:48.488 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:01:48.488 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:01:48.488 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:01:48.488 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:01:48.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:01:48.489 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T10:01:48.506 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2024-04-17T10:01:48.509+0000 7f3547047580 -1 init, newargv = 0x55dbb2fe0db0 newargc=15 2024-04-17T10:01:48.506 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[48511]: starting ceph client 2024-04-17T10:01:48.513 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[48511]: starting fuse 2024-04-17T10:01:48.515 INFO:teuthology.orchestra.run.smithi032.stdout:44 2024-04-17T10:01:48.515 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-17T10:01:48.515 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2024-04-17T10:01:48.515 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2024-04-17T10:01:48.515 DEBUG:teuthology.orchestra.run.smithi032:> import re 2024-04-17T10:01:48.515 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:01:48.515 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2024-04-17T10:01:48.516 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:01:48.516 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2024-04-17T10:01:48.516 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T10:01:48.516 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2024-04-17T10:01:48.516 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T10:01:48.516 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:01:48.516 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2024-04-17T10:01:48.516 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2024-04-17T10:01:48.516 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2024-04-17T10:01:48.516 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2024-04-17T10:01:48.516 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:01:48.516 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2024-04-17T10:01:48.516 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T10:01:48.516 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T10:01:48.516 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T10:01:48.516 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2024-04-17T10:01:48.517 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2024-04-17T10:01:48.517 DEBUG:teuthology.orchestra.run.smithi032:> return f 2024-04-17T10:01:48.517 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T10:01:48.517 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:01:48.517 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2024-04-17T10:01:48.517 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:01:48.639 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.48511.asok 2024-04-17T10:01:48.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:01:48.651 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48511.asok 2024-04-17T10:01:48.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:01:48.651 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48511.asok status 2024-04-17T10:01:48.828 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:01:48.828 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2024-04-17T10:01:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-17T10:01:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T10:01:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2024-04-17T10:01:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2024-04-17T10:01:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T10:01:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "48511", 2024-04-17T10:01:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2024-04-17T10:01:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:01:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2024-04-17T10:01:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2024-04-17T10:01:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 8721, 2024-04-17T10:01:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2024-04-17T10:01:48.829 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2024-04-17T10:01:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2024-04-17T10:01:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 8721 2024-04-17T10:01:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:01:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T10:01:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T10:01:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:01:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3860833184 2024-04-17T10:01:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:01:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:01:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T10:01:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T10:01:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:01:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3860833184 2024-04-17T10:01:48.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:01:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.8721 192.168.0.1:0/3860833184", 2024-04-17T10:01:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/3860833184", 2024-04-17T10:01:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2024-04-17T10:01:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 81, 2024-04-17T10:01:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 97, 2024-04-17T10:01:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2024-04-17T10:01:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2024-04-17T10:01:48.831 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs" 2024-04-17T10:01:48.831 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:01:48.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:01:48.844 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2024-04-17T10:01:48.844 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T10:01:48.853 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2024-04-17T10:01:48.853 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T10:01:48.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:01:48.854 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:01:48.923 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:01:48.923 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:01:48.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:01:48.977 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-17T10:01:49.114 INFO:teuthology.orchestra.run.smithi032.stderr:6+0 records in 2024-04-17T10:01:49.114 INFO:teuthology.orchestra.run.smithi032.stderr:6+0 records out 2024-04-17T10:01:49.114 INFO:teuthology.orchestra.run.smithi032.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0787531 s, 79.9 MB/s 2024-04-17T10:01:49.115 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:01:49.115 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:01:49.124 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:01:49.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.317+0000 7fd723200640 1 Processor -- start 2024-04-17T10:01:49.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.317+0000 7fd723200640 1 -- start start 2024-04-17T10:01:49.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.317+0000 7fd723200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd71c144e90 0x7fd71c0b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:49.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.317+0000 7fd723200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd71c0b4de0 0x7fd71c15baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:49.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.317+0000 7fd723200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd71c15bfe0 0x7fd71c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:49.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.317+0000 7fd723200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd71c15c430 con 0x7fd71c144e90 2024-04-17T10:01:49.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.317+0000 7fd723200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd71c145270 con 0x7fd71c15bfe0 2024-04-17T10:01:49.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.317+0000 7fd723200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd71c072bb0 con 0x7fd71c0b4de0 2024-04-17T10:01:49.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.317+0000 7fd71be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd71c144e90 0x7fd71c0b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:49.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.317+0000 7fd71be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd71c144e90 0x7fd71c0b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52950/0 (socket says 172.21.15.32:52950) 2024-04-17T10:01:49.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.317+0000 7fd71be00640 1 -- 172.21.15.32:0/3802784692 learned_addr learned my addr 172.21.15.32:0/3802784692 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:49.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.317+0000 7fd721200640 1 --2- 172.21.15.32:0/3802784692 >> v2:172.21.15.144:3300/0 conn(0x7fd71c15bfe0 0x7fd71c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:49.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.317+0000 7fd71b400640 1 --2- 172.21.15.32:0/3802784692 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd71c0b4de0 0x7fd71c15baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:49.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.317+0000 7fd71be00640 1 -- 172.21.15.32:0/3802784692 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd71c0b4de0 msgr2=0x7fd71c15baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:49.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.317+0000 7fd71be00640 1 --2- 172.21.15.32:0/3802784692 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd71c0b4de0 0x7fd71c15baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:49.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.317+0000 7fd71be00640 1 -- 172.21.15.32:0/3802784692 >> v2:172.21.15.144:3300/0 conn(0x7fd71c15bfe0 msgr2=0x7fd71c06ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:49.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.317+0000 7fd71be00640 1 --2- 172.21.15.32:0/3802784692 >> v2:172.21.15.144:3300/0 conn(0x7fd71c15bfe0 0x7fd71c06ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:49.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.317+0000 7fd71be00640 1 -- 172.21.15.32:0/3802784692 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd71c0b3dc0 con 0x7fd71c144e90 2024-04-17T10:01:49.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.317+0000 7fd71b400640 1 --2- 172.21.15.32:0/3802784692 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd71c0b4de0 0x7fd71c15baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:49.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.321+0000 7fd71be00640 1 --2- 172.21.15.32:0/3802784692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71c144e90 0x7fd71c0b48a0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fd71c070150 tx=0x7fd70c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816bbf2f0bfca0e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:49.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.321+0000 7fd71aa00640 1 -- 172.21.15.32:0/3802784692 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c0aa430 con 0x7fd71c144e90 2024-04-17T10:01:49.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.321+0000 7fd71aa00640 1 -- 172.21.15.32:0/3802784692 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd70c0af070 con 0x7fd71c144e90 2024-04-17T10:01:49.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.321+0000 7fd71aa00640 1 -- 172.21.15.32:0/3802784692 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c0aa7f0 con 0x7fd71c144e90 2024-04-17T10:01:49.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.321+0000 7fd723200640 1 -- 172.21.15.32:0/3802784692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71c144e90 msgr2=0x7fd71c0b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:49.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.321+0000 7fd723200640 1 --2- 172.21.15.32:0/3802784692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71c144e90 0x7fd71c0b48a0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fd71c070150 tx=0x7fd70c01c4f0 comp rx=0 tx=0).stop 2024-04-17T10:01:49.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.321+0000 7fd723200640 1 -- 172.21.15.32:0/3802784692 shutdown_connections 2024-04-17T10:01:49.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.321+0000 7fd723200640 1 --2- 172.21.15.32:0/3802784692 >> v2:172.21.15.144:3300/0 conn(0x7fd71c15bfe0 0x7fd71c06ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:49.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.321+0000 7fd723200640 1 --2- 172.21.15.32:0/3802784692 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd71c0b4de0 0x7fd71c15baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:49.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.321+0000 7fd723200640 1 --2- 172.21.15.32:0/3802784692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71c144e90 0x7fd71c0b48a0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:49.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.321+0000 7fd723200640 1 -- 172.21.15.32:0/3802784692 >> 172.21.15.32:0/3802784692 conn(0x7fd71c06ba30 msgr2=0x7fd71c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:49.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.321+0000 7fd723200640 1 -- 172.21.15.32:0/3802784692 shutdown_connections 2024-04-17T10:01:49.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.321+0000 7fd723200640 1 -- 172.21.15.32:0/3802784692 wait complete. 2024-04-17T10:01:49.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.321+0000 7fd723200640 1 Processor -- start 2024-04-17T10:01:49.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.321+0000 7fd723200640 1 -- start start 2024-04-17T10:01:49.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd723200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71c0b4de0 0x7fd71c0aac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:49.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd723200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd71c144e90 0x7fd71c1f6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:49.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd723200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd71c15bfe0 0x7fd71c0a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:49.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd723200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c06f8b0 con 0x7fd71c0b4de0 2024-04-17T10:01:49.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd723200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c072880 con 0x7fd71c144e90 2024-04-17T10:01:49.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd723200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd71c0cf2b0 con 0x7fd71c15bfe0 2024-04-17T10:01:49.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd71be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71c0b4de0 0x7fd71c0aac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:49.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd71be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71c0b4de0 0x7fd71c0aac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52956/0 (socket says 172.21.15.32:52956) 2024-04-17T10:01:49.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd71be00640 1 -- 172.21.15.32:0/794002725 learned_addr learned my addr 172.21.15.32:0/794002725 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:49.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd721200640 1 --2- 172.21.15.32:0/794002725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd71c15bfe0 0x7fd71c0a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:49.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd71be00640 1 -- 172.21.15.32:0/794002725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd71c15bfe0 msgr2=0x7fd71c0a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:49.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd71b400640 1 --2- 172.21.15.32:0/794002725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd71c144e90 0x7fd71c1f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:49.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd71be00640 1 --2- 172.21.15.32:0/794002725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd71c15bfe0 0x7fd71c0a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:49.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd71be00640 1 -- 172.21.15.32:0/794002725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd71c144e90 msgr2=0x7fd71c1f6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:49.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd71be00640 1 --2- 172.21.15.32:0/794002725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd71c144e90 0x7fd71c1f6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:49.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd71be00640 1 -- 172.21.15.32:0/794002725 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd71c1f67e0 con 0x7fd71c0b4de0 2024-04-17T10:01:49.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd71b400640 1 --2- 172.21.15.32:0/794002725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd71c144e90 0x7fd71c1f6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd721200640 1 --2- 172.21.15.32:0/794002725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd71c15bfe0 0x7fd71c0a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd71be00640 1 --2- 172.21.15.32:0/794002725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71c0b4de0 0x7fd71c0aac10 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0ac040 tx=0x7fd70c01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd713e00640 1 -- 172.21.15.32:0/794002725 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c0bd5f0 con 0x7fd71c0b4de0 2024-04-17T10:01:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd713e00640 1 -- 172.21.15.32:0/794002725 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd70c0bd790 con 0x7fd71c0b4de0 2024-04-17T10:01:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd723200640 1 -- 172.21.15.32:0/794002725 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd71c1f6ac0 con 0x7fd71c0b4de0 2024-04-17T10:01:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd713e00640 1 -- 172.21.15.32:0/794002725 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c0bdb80 con 0x7fd71c0b4de0 2024-04-17T10:01:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd723200640 1 -- 172.21.15.32:0/794002725 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd71c1f7080 con 0x7fd71c0b4de0 2024-04-17T10:01:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd723200640 1 -- 172.21.15.32:0/794002725 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd71c068a90 con 0x7fd71c0b4de0 2024-04-17T10:01:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.325+0000 7fd713e00640 1 -- 172.21.15.32:0/794002725 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd70c0b33f0 con 0x7fd71c0b4de0 2024-04-17T10:01:49.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.329+0000 7fd713e00640 1 --2- 172.21.15.32:0/794002725 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6fc068380 0x7fd6fc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:49.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.329+0000 7fd713e00640 1 -- 172.21.15.32:0/794002725 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fd70c121400 con 0x7fd71c0b4de0 2024-04-17T10:01:49.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.329+0000 7fd71b400640 1 --2- 172.21.15.32:0/794002725 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6fc068380 0x7fd6fc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:49.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.329+0000 7fd71b400640 1 --2- 172.21.15.32:0/794002725 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6fc068380 0x7fd6fc06a840 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd71c1444a0 tx=0x7fd708008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:49.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.333+0000 7fd713e00640 1 -- 172.21.15.32:0/794002725 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd70c0c3060 con 0x7fd71c0b4de0 2024-04-17T10:01:49.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.501+0000 7fd723200640 1 -- 172.21.15.32:0/794002725 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd71c143e30 con 0x7fd71c0b4de0 2024-04-17T10:01:49.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.501+0000 7fd713e00640 1 -- 172.21.15.32:0/794002725 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+16620 (secure 0 0 0) 0x7fd70c016110 con 0x7fd71c0b4de0 2024-04-17T10:01:49.498 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:49.499 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":97,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:01:42.155683+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:01:38.938460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-17T10:01:39.505171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:01:49.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.501+0000 7fd723200640 1 -- 172.21.15.32:0/794002725 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6fc068380 msgr2=0x7fd6fc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:49.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.501+0000 7fd723200640 1 --2- 172.21.15.32:0/794002725 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6fc068380 0x7fd6fc06a840 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd71c1444a0 tx=0x7fd708008040 comp rx=0 tx=0).stop 2024-04-17T10:01:49.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.501+0000 7fd723200640 1 -- 172.21.15.32:0/794002725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71c0b4de0 msgr2=0x7fd71c0aac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:49.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.501+0000 7fd723200640 1 --2- 172.21.15.32:0/794002725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71c0b4de0 0x7fd71c0aac10 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0ac040 tx=0x7fd70c01ad30 comp rx=0 tx=0).stop 2024-04-17T10:01:49.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.505+0000 7fd723200640 1 -- 172.21.15.32:0/794002725 shutdown_connections 2024-04-17T10:01:49.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.505+0000 7fd723200640 1 --2- 172.21.15.32:0/794002725 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6fc068380 0x7fd6fc06a840 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:49.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.505+0000 7fd723200640 1 --2- 172.21.15.32:0/794002725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd71c15bfe0 0x7fd71c0a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:49.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.505+0000 7fd723200640 1 --2- 172.21.15.32:0/794002725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd71c144e90 0x7fd71c1f6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:49.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.505+0000 7fd723200640 1 --2- 172.21.15.32:0/794002725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd71c0b4de0 0x7fd71c0aac10 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:49.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.505+0000 7fd723200640 1 -- 172.21.15.32:0/794002725 >> 172.21.15.32:0/794002725 conn(0x7fd71c06ba30 msgr2=0x7fd71c0cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:49.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.505+0000 7fd723200640 1 -- 172.21.15.32:0/794002725 shutdown_connections 2024-04-17T10:01:49.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.505+0000 7fd723200640 1 -- 172.21.15.32:0/794002725 wait complete. 2024-04-17T10:01:49.520 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:01:38.938460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:01:39.505171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:01:49.521 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:01:49.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.669+0000 7fd485600640 1 Processor -- start 2024-04-17T10:01:49.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.669+0000 7fd485600640 1 -- start start 2024-04-17T10:01:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.669+0000 7fd485600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd48015b7d0 0x7fd48006e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.669+0000 7fd485600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd48006c140 0x7fd4800ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.669+0000 7fd485600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4800cc940 con 0x7fd48006f7c0 2024-04-17T10:01:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.669+0000 7fd485600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd4800ccc20 con 0x7fd48015b7d0 2024-04-17T10:01:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.669+0000 7fd485600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd48006e0b0 con 0x7fd48006c140 2024-04-17T10:01:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.669+0000 7fd477e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd48006f7c0 0x7fd48006ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50490/0 (socket says 172.21.15.32:50490) 2024-04-17T10:01:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.669+0000 7fd477e00640 1 -- 172.21.15.32:0/3386823927 learned_addr learned my addr 172.21.15.32:0/3386823927 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.669+0000 7fd47f400640 1 --2- 172.21.15.32:0/3386823927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd48006c140 0x7fd4800ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:49.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.669+0000 7fd47ea00640 1 --2- 172.21.15.32:0/3386823927 >> v2:172.21.15.144:3300/0 conn(0x7fd48015b7d0 0x7fd48006e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:49.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.669+0000 7fd47e000640 1 -- 172.21.15.32:0/3386823927 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2876529286 0 0) 0x7fd4800cc940 con 0x7fd48006f7c0 2024-04-17T10:01:49.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.669+0000 7fd47e000640 1 -- 172.21.15.32:0/3386823927 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd464003530 con 0x7fd48006f7c0 2024-04-17T10:01:49.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.669+0000 7fd47ea00640 1 -- 172.21.15.32:0/3386823927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd48006c140 msgr2=0x7fd4800ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:49.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.669+0000 7fd47ea00640 1 --2- 172.21.15.32:0/3386823927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd48006c140 0x7fd4800ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:49.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.669+0000 7fd47ea00640 1 -- 172.21.15.32:0/3386823927 >> v1:172.21.15.32:6789/0 conn(0x7fd48006f7c0 legacy=0x7fd48006ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:49.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.669+0000 7fd47ea00640 1 -- 172.21.15.32:0/3386823927 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48006e390 con 0x7fd48015b7d0 2024-04-17T10:01:49.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.669+0000 7fd47f400640 1 --2- 172.21.15.32:0/3386823927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd48006c140 0x7fd4800ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:01:49.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.673+0000 7fd47ea00640 1 --2- 172.21.15.32:0/3386823927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd48015b7d0 0x7fd48006e8b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd4700097c0 tx=0x7fd47001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2593c12f7891d013 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:49.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.673+0000 7fd47e000640 1 -- 172.21.15.32:0/3386823927 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4700ab430 con 0x7fd48015b7d0 2024-04-17T10:01:49.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.673+0000 7fd47e000640 1 -- 172.21.15.32:0/3386823927 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4700b0070 con 0x7fd48015b7d0 2024-04-17T10:01:49.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.673+0000 7fd47e000640 1 -- 172.21.15.32:0/3386823927 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4700ab7d0 con 0x7fd48015b7d0 2024-04-17T10:01:49.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.673+0000 7fd485600640 1 -- 172.21.15.32:0/3386823927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd48015b7d0 msgr2=0x7fd48006e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:49.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.673+0000 7fd485600640 1 --2- 172.21.15.32:0/3386823927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd48015b7d0 0x7fd48006e8b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd4700097c0 tx=0x7fd47001c410 comp rx=0 tx=0).stop 2024-04-17T10:01:49.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.673+0000 7fd485600640 1 -- 172.21.15.32:0/3386823927 shutdown_connections 2024-04-17T10:01:49.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.673+0000 7fd485600640 1 --2- 172.21.15.32:0/3386823927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd48006c140 0x7fd4800ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:49.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.673+0000 7fd485600640 1 --2- 172.21.15.32:0/3386823927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd48015b7d0 0x7fd48006e8b0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:49.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.673+0000 7fd485600640 1 -- 172.21.15.32:0/3386823927 >> 172.21.15.32:0/3386823927 conn(0x7fd4800b2120 msgr2=0x7fd4800b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:49.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.673+0000 7fd485600640 1 -- 172.21.15.32:0/3386823927 shutdown_connections 2024-04-17T10:01:49.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.673+0000 7fd485600640 1 -- 172.21.15.32:0/3386823927 wait complete. 2024-04-17T10:01:49.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd485600640 1 Processor -- start 2024-04-17T10:01:49.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd485600640 1 -- start start 2024-04-17T10:01:49.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd485600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd48006c140 0x7fd48014b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:49.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd485600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd48006f7c0 0x7fd48015e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:49.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd485600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd48015b7d0 0x7fd4801512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:49.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd485600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd480168630 con 0x7fd48015b7d0 2024-04-17T10:01:49.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd485600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801684b0 con 0x7fd48006f7c0 2024-04-17T10:01:49.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd485600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd4801687b0 con 0x7fd48006c140 2024-04-17T10:01:49.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd47f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd48015b7d0 0x7fd4801512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:49.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd47f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd48015b7d0 0x7fd4801512d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52962/0 (socket says 172.21.15.32:52962) 2024-04-17T10:01:49.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd47f400640 1 -- 172.21.15.32:0/1146412923 learned_addr learned my addr 172.21.15.32:0/1146412923 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:49.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd47ea00640 1 --2- 172.21.15.32:0/1146412923 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd48006c140 0x7fd48014b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:49.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd477e00640 1 --2- 172.21.15.32:0/1146412923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd48006f7c0 0x7fd48015e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:49.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd47f400640 1 -- 172.21.15.32:0/1146412923 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd48006c140 msgr2=0x7fd48014b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:49.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd47f400640 1 --2- 172.21.15.32:0/1146412923 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd48006c140 0x7fd48014b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:49.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd47f400640 1 -- 172.21.15.32:0/1146412923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd48006f7c0 msgr2=0x7fd48015e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:49.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd47f400640 1 --2- 172.21.15.32:0/1146412923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd48006f7c0 0x7fd48015e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:49.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd47f400640 1 -- 172.21.15.32:0/1146412923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48015cf60 con 0x7fd48015b7d0 2024-04-17T10:01:49.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd47f400640 1 --2- 172.21.15.32:0/1146412923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd48015b7d0 0x7fd4801512d0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fd478009e10 tx=0x7fd47800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:49.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd476a00640 1 -- 172.21.15.32:0/1146412923 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478018020 con 0x7fd48015b7d0 2024-04-17T10:01:49.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd476a00640 1 -- 172.21.15.32:0/1146412923 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd478014480 con 0x7fd48015b7d0 2024-04-17T10:01:49.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd476a00640 1 -- 172.21.15.32:0/1146412923 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478014850 con 0x7fd48015b7d0 2024-04-17T10:01:49.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd485600640 1 -- 172.21.15.32:0/1146412923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48015c6c0 con 0x7fd48015b7d0 2024-04-17T10:01:49.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd485600640 1 -- 172.21.15.32:0/1146412923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48015cc80 con 0x7fd48015b7d0 2024-04-17T10:01:49.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.677+0000 7fd485600640 1 -- 172.21.15.32:0/1146412923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd480068a90 con 0x7fd48015b7d0 2024-04-17T10:01:49.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.681+0000 7fd476a00640 1 -- 172.21.15.32:0/1146412923 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd47801b020 con 0x7fd48015b7d0 2024-04-17T10:01:49.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.681+0000 7fd476a00640 1 --2- 172.21.15.32:0/1146412923 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd45c068380 0x7fd45c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:49.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.681+0000 7fd476a00640 1 -- 172.21.15.32:0/1146412923 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fd478087710 con 0x7fd48015b7d0 2024-04-17T10:01:49.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.681+0000 7fd47ea00640 1 --2- 172.21.15.32:0/1146412923 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd45c068380 0x7fd45c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:49.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.681+0000 7fd47ea00640 1 --2- 172.21.15.32:0/1146412923 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd45c068380 0x7fd45c06a840 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd4700af000 tx=0x7fd470008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:49.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.685+0000 7fd476a00640 1 -- 172.21.15.32:0/1146412923 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd478022030 con 0x7fd48015b7d0 2024-04-17T10:01:49.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.849+0000 7fd485600640 1 -- 172.21.15.32:0/1146412923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd48015aed0 con 0x7fd48015b7d0 2024-04-17T10:01:49.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.849+0000 7fd476a00640 1 -- 172.21.15.32:0/1146412923 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7fd478010040 con 0x7fd48015b7d0 2024-04-17T10:01:49.846 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T10:01:49.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.849+0000 7fd485600640 1 -- 172.21.15.32:0/1146412923 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd45c068380 msgr2=0x7fd45c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:49.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.849+0000 7fd485600640 1 --2- 172.21.15.32:0/1146412923 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd45c068380 0x7fd45c06a840 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd4700af000 tx=0x7fd470008040 comp rx=0 tx=0).stop 2024-04-17T10:01:49.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.849+0000 7fd485600640 1 -- 172.21.15.32:0/1146412923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd48015b7d0 msgr2=0x7fd4801512d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:49.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.849+0000 7fd485600640 1 --2- 172.21.15.32:0/1146412923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd48015b7d0 0x7fd4801512d0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fd478009e10 tx=0x7fd47800c490 comp rx=0 tx=0).stop 2024-04-17T10:01:49.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.849+0000 7fd485600640 1 -- 172.21.15.32:0/1146412923 shutdown_connections 2024-04-17T10:01:49.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.849+0000 7fd485600640 1 --2- 172.21.15.32:0/1146412923 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd45c068380 0x7fd45c06a840 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:49.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.849+0000 7fd485600640 1 --2- 172.21.15.32:0/1146412923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd48015b7d0 0x7fd4801512d0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:49.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.849+0000 7fd485600640 1 --2- 172.21.15.32:0/1146412923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd48006f7c0 0x7fd48015e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:49.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.849+0000 7fd485600640 1 --2- 172.21.15.32:0/1146412923 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd48006c140 0x7fd48014b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:49.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.849+0000 7fd485600640 1 -- 172.21.15.32:0/1146412923 >> 172.21.15.32:0/1146412923 conn(0x7fd4800b2120 msgr2=0x7fd4800b0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:49.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.849+0000 7fd485600640 1 -- 172.21.15.32:0/1146412923 shutdown_connections 2024-04-17T10:01:49.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:49.849+0000 7fd485600640 1 -- 172.21.15.32:0/1146412923 wait complete. 2024-04-17T10:01:49.868 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:01:50.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.017+0000 7f18f1a00640 1 Processor -- start 2024-04-17T10:01:50.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.017+0000 7f18f1a00640 1 -- start start 2024-04-17T10:01:50.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.017+0000 7f18f1a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f18ec06f7c0 0x7f18ec06ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:50.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.017+0000 7f18f1a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18ec06f270 0x7f18ec06bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:50.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.017+0000 7f18f1a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18ec0cc940 con 0x7f18ec15b7d0 2024-04-17T10:01:50.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.017+0000 7f18f1a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f18ec0ccc20 con 0x7f18ec06f7c0 2024-04-17T10:01:50.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.017+0000 7f18f1a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f18ec06e0b0 con 0x7f18ec06f270 2024-04-17T10:01:50.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.017+0000 7f18e2000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f18ec06f7c0 0x7f18ec06ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:50.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.017+0000 7f18eb400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18ec06f270 0x7f18ec06bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:50.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.017+0000 7f18e2000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f18ec06f7c0 0x7f18ec06ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:47258/0 (socket says 172.21.15.32:47258) 2024-04-17T10:01:50.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.017+0000 7f18e2000640 1 -- 172.21.15.32:0/1626163211 learned_addr learned my addr 172.21.15.32:0/1626163211 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:50.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.017+0000 7f18eaa00640 1 --1- 172.21.15.32:0/1626163211 >> v1:172.21.15.144:6789/0 conn(0x7f18ec15b7d0 0x7f18ec06e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:55504/0 (socket says 172.21.15.32:55504) 2024-04-17T10:01:50.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.017+0000 7f18e2000640 1 -- 172.21.15.32:0/1626163211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18ec06f270 msgr2=0x7f18ec06bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:50.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.017+0000 7f18e2000640 1 --2- 172.21.15.32:0/1626163211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18ec06f270 0x7f18ec06bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.017+0000 7f18e2000640 1 -- 172.21.15.32:0/1626163211 >> v1:172.21.15.144:6789/0 conn(0x7f18ec15b7d0 legacy=0x7f18ec06e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:50.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.021+0000 7f18e2000640 1 -- 172.21.15.32:0/1626163211 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ec06e390 con 0x7f18ec06f7c0 2024-04-17T10:01:50.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.021+0000 7f18e2000640 1 --2- 172.21.15.32:0/1626163211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18ec06f7c0 0x7f18ec06ba30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f18d400bb10 tx=0x7f18d401eba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9734000ed11217fa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:50.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.021+0000 7f18ea000640 1 -- 172.21.15.32:0/1626163211 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d40b4070 con 0x7f18ec06f7c0 2024-04-17T10:01:50.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.021+0000 7f18ea000640 1 -- 172.21.15.32:0/1626163211 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18d40a7c90 con 0x7f18ec06f7c0 2024-04-17T10:01:50.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.021+0000 7f18ea000640 1 -- 172.21.15.32:0/1626163211 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d40af660 con 0x7f18ec06f7c0 2024-04-17T10:01:50.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.021+0000 7f18f1a00640 1 -- 172.21.15.32:0/1626163211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18ec06f7c0 msgr2=0x7f18ec06ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:50.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.021+0000 7f18f1a00640 1 --2- 172.21.15.32:0/1626163211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18ec06f7c0 0x7f18ec06ba30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f18d400bb10 tx=0x7f18d401eba0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.021+0000 7f18f1a00640 1 -- 172.21.15.32:0/1626163211 shutdown_connections 2024-04-17T10:01:50.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.021+0000 7f18f1a00640 1 --2- 172.21.15.32:0/1626163211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18ec06f270 0x7f18ec06bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.021+0000 7f18f1a00640 1 --2- 172.21.15.32:0/1626163211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18ec06f7c0 0x7f18ec06ba30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.021+0000 7f18f1a00640 1 -- 172.21.15.32:0/1626163211 >> 172.21.15.32:0/1626163211 conn(0x7f18ec0b2120 msgr2=0x7f18ec0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:50.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.021+0000 7f18f1a00640 1 -- 172.21.15.32:0/1626163211 shutdown_connections 2024-04-17T10:01:50.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.021+0000 7f18f1a00640 1 -- 172.21.15.32:0/1626163211 wait complete. 2024-04-17T10:01:50.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.025+0000 7f18f1a00640 1 Processor -- start 2024-04-17T10:01:50.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.025+0000 7f18f1a00640 1 -- start start 2024-04-17T10:01:50.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.025+0000 7f18f1a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ec06f270 0x7f18ec142590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:50.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.025+0000 7f18f1a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18ec06f7c0 0x7f18ec15a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:50.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.025+0000 7f18f1a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18ec15b7d0 0x7f18ec15a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:50.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.025+0000 7f18f1a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec0ceeb0 con 0x7f18ec06f270 2024-04-17T10:01:50.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.025+0000 7f18f1a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec0ced30 con 0x7f18ec15b7d0 2024-04-17T10:01:50.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.025+0000 7f18f1a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f18ec071e60 con 0x7f18ec06f7c0 2024-04-17T10:01:50.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.025+0000 7f18eaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ec06f270 0x7f18ec142590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:50.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.025+0000 7f18eaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ec06f270 0x7f18ec142590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52968/0 (socket says 172.21.15.32:52968) 2024-04-17T10:01:50.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.025+0000 7f18eaa00640 1 -- 172.21.15.32:0/4096262189 learned_addr learned my addr 172.21.15.32:0/4096262189 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:50.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.025+0000 7f18e2000640 1 --2- 172.21.15.32:0/4096262189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18ec06f7c0 0x7f18ec15a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:50.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.025+0000 7f18eb400640 1 --2- 172.21.15.32:0/4096262189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18ec15b7d0 0x7f18ec15a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:50.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.025+0000 7f18eaa00640 1 -- 172.21.15.32:0/4096262189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18ec06f7c0 msgr2=0x7f18ec15a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:50.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.025+0000 7f18eaa00640 1 --2- 172.21.15.32:0/4096262189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18ec06f7c0 0x7f18ec15a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.025+0000 7f18eaa00640 1 -- 172.21.15.32:0/4096262189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18ec15b7d0 msgr2=0x7f18ec15a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:50.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.025+0000 7f18eaa00640 1 --2- 172.21.15.32:0/4096262189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18ec15b7d0 0x7f18ec15a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.025+0000 7f18eaa00640 1 -- 172.21.15.32:0/4096262189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ec1fa3b0 con 0x7f18ec06f270 2024-04-17T10:01:50.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.025+0000 7f18eb400640 1 --2- 172.21.15.32:0/4096262189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18ec15b7d0 0x7f18ec15a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:50.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.025+0000 7f18eaa00640 1 --2- 172.21.15.32:0/4096262189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ec06f270 0x7f18ec142590 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00c370 tx=0x7f18dc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:50.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.025+0000 7f18e3400640 1 -- 172.21.15.32:0/4096262189 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc0074e0 con 0x7f18ec06f270 2024-04-17T10:01:50.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.025+0000 7f18e3400640 1 -- 172.21.15.32:0/4096262189 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18dc007680 con 0x7f18ec06f270 2024-04-17T10:01:50.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.025+0000 7f18e3400640 1 -- 172.21.15.32:0/4096262189 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc004a30 con 0x7f18ec06f270 2024-04-17T10:01:50.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.029+0000 7f18f1a00640 1 -- 172.21.15.32:0/4096262189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18ec1fa650 con 0x7f18ec06f270 2024-04-17T10:01:50.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.029+0000 7f18f1a00640 1 -- 172.21.15.32:0/4096262189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18ec1fac10 con 0x7f18ec06f270 2024-04-17T10:01:50.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.029+0000 7f18f1a00640 1 -- 172.21.15.32:0/4096262189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18ec068a90 con 0x7f18ec06f270 2024-04-17T10:01:50.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.029+0000 7f18e3400640 1 -- 172.21.15.32:0/4096262189 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f18dc01a020 con 0x7f18ec06f270 2024-04-17T10:01:50.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.029+0000 7f18e3400640 1 --2- 172.21.15.32:0/4096262189 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f18c8068380 0x7f18c806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:50.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.029+0000 7f18e3400640 1 -- 172.21.15.32:0/4096262189 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f18dc086180 con 0x7f18ec06f270 2024-04-17T10:01:50.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.029+0000 7f18e2000640 1 --2- 172.21.15.32:0/4096262189 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f18c8068380 0x7f18c806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:50.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.029+0000 7f18e2000640 1 --2- 172.21.15.32:0/4096262189 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f18c8068380 0x7f18c806a840 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f18d40b1000 tx=0x7f18d4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:50.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.033+0000 7f18e3400640 1 -- 172.21.15.32:0/4096262189 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f18dc0566a0 con 0x7f18ec06f270 2024-04-17T10:01:50.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.201+0000 7f18f1a00640 1 -- 172.21.15.32:0/4096262189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f18ec06ce40 con 0x7f18ec06f270 2024-04-17T10:01:50.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.201+0000 7f18e3400640 1 -- 172.21.15.32:0/4096262189 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f18dc055fa0 con 0x7f18ec06f270 2024-04-17T10:01:50.199 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T10:01:50.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.205+0000 7f18f1a00640 1 -- 172.21.15.32:0/4096262189 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f18c8068380 msgr2=0x7f18c806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:50.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.205+0000 7f18f1a00640 1 --2- 172.21.15.32:0/4096262189 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f18c8068380 0x7f18c806a840 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f18d40b1000 tx=0x7f18d4008040 comp rx=0 tx=0).stop 2024-04-17T10:01:50.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.205+0000 7f18f1a00640 1 -- 172.21.15.32:0/4096262189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ec06f270 msgr2=0x7f18ec142590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:50.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.205+0000 7f18f1a00640 1 --2- 172.21.15.32:0/4096262189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ec06f270 0x7f18ec142590 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00c370 tx=0x7f18dc00c830 comp rx=0 tx=0).stop 2024-04-17T10:01:50.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.205+0000 7f18f1a00640 1 -- 172.21.15.32:0/4096262189 shutdown_connections 2024-04-17T10:01:50.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.205+0000 7f18f1a00640 1 --2- 172.21.15.32:0/4096262189 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f18c8068380 0x7f18c806a840 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.205+0000 7f18f1a00640 1 --2- 172.21.15.32:0/4096262189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18ec06f7c0 0x7f18ec15a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.205+0000 7f18f1a00640 1 --2- 172.21.15.32:0/4096262189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18ec15b7d0 0x7f18ec15a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.205+0000 7f18f1a00640 1 --2- 172.21.15.32:0/4096262189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ec06f270 0x7f18ec142590 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.205+0000 7f18f1a00640 1 -- 172.21.15.32:0/4096262189 >> 172.21.15.32:0/4096262189 conn(0x7f18ec0b2120 msgr2=0x7f18ec072370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:50.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.205+0000 7f18f1a00640 1 -- 172.21.15.32:0/4096262189 shutdown_connections 2024-04-17T10:01:50.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.205+0000 7f18f1a00640 1 -- 172.21.15.32:0/4096262189 wait complete. 2024-04-17T10:01:50.221 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:01:50.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ef0a00640 1 Processor -- start 2024-04-17T10:01:50.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ef0a00640 1 -- start start 2024-04-17T10:01:50.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ef0a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2eec06ba30 0x7f2eec06be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:50.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ef0a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2eec06c350 con 0x7f2eec071e60 2024-04-17T10:01:50.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ef0a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2eec0d0130 con 0x7f2eec0cf370 2024-04-17T10:01:50.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ef0a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f2eec0cfef0 con 0x7f2eec06ba30 2024-04-17T10:01:50.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2eea800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2eec071e60 0x7f2eec072240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33618/0 (socket says 172.21.15.32:33618) 2024-04-17T10:01:50.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2eea800640 1 -- 172.21.15.32:0/826166747 learned_addr learned my addr 172.21.15.32:0/826166747 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:50.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ee9e00640 1 --2- 172.21.15.32:0/826166747 >> v2:172.21.15.144:3300/0 conn(0x7f2eec06ba30 0x7f2eec06be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:50.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ee8a00640 1 -- 172.21.15.32:0/826166747 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3784981131 0 0) 0x7f2eec06c350 con 0x7f2eec071e60 2024-04-17T10:01:50.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ee8a00640 1 -- 172.21.15.32:0/826166747 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ed0003530 con 0x7f2eec071e60 2024-04-17T10:01:50.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ee8a00640 1 -- 172.21.15.32:0/826166747 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1888153716 0 0) 0x7f2ed0003530 con 0x7f2eec071e60 2024-04-17T10:01:50.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ee8a00640 1 -- 172.21.15.32:0/826166747 >> v2:172.21.15.144:3300/0 conn(0x7f2eec06ba30 msgr2=0x7f2eec06be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:50.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ee8a00640 1 --2- 172.21.15.32:0/826166747 >> v2:172.21.15.144:3300/0 conn(0x7f2eec06ba30 0x7f2eec06be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ee8a00640 1 -- 172.21.15.32:0/826166747 >> v1:172.21.15.144:6789/0 conn(0x7f2eec0cf370 legacy=0x7f2eec0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:50.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ee8a00640 1 -- 172.21.15.32:0/826166747 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eec15d090 con 0x7f2eec071e60 2024-04-17T10:01:50.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ee8a00640 1 -- 172.21.15.32:0/826166747 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f2ee4001a00 con 0x7f2eec071e60 2024-04-17T10:01:50.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ee8a00640 1 -- 172.21.15.32:0/826166747 >> v1:172.21.15.32:6789/0 conn(0x7f2eec071e60 legacy=0x7f2eec072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:50.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ee8a00640 1 --2- 172.21.15.32:0/826166747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ed00039c0 0x7f2ed0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:50.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ee8a00640 1 --2- 172.21.15.32:0/826166747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ed0004810 0x7f2ed0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:50.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ee8a00640 1 --2- 172.21.15.32:0/826166747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ed00051d0 0x7f2ed000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:50.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ee8a00640 1 -- 172.21.15.32:0/826166747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec0cfef0 con 0x7f2ed00051d0 2024-04-17T10:01:50.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ee8a00640 1 -- 172.21.15.32:0/826166747 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed000c170 con 0x7f2ed0004810 2024-04-17T10:01:50.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ee8a00640 1 -- 172.21.15.32:0/826166747 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed000c370 con 0x7f2ed00039c0 2024-04-17T10:01:50.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2eea800640 1 --2- 172.21.15.32:0/826166747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ed00051d0 0x7f2ed000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:50.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ee9400640 1 --2- 172.21.15.32:0/826166747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ed0004810 0x7f2ed0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:50.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2ee9e00640 1 --2- 172.21.15.32:0/826166747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ed00039c0 0x7f2ed0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:50.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2eea800640 1 -- 172.21.15.32:0/826166747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ed00039c0 msgr2=0x7f2ed0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:50.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2eea800640 1 --2- 172.21.15.32:0/826166747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ed00039c0 0x7f2ed0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2eea800640 1 -- 172.21.15.32:0/826166747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ed0004810 msgr2=0x7f2ed0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:50.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2eea800640 1 --2- 172.21.15.32:0/826166747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ed0004810 0x7f2ed0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.369+0000 7f2eea800640 1 -- 172.21.15.32:0/826166747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed0003530 con 0x7f2ed00051d0 2024-04-17T10:01:50.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.373+0000 7f2eea800640 1 -- 172.21.15.32:0/826166747 reap_dead start 2024-04-17T10:01:50.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.373+0000 7f2eea800640 1 --2- 172.21.15.32:0/826166747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ed00051d0 0x7f2ed000b9d0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400a860 tx=0x7f2ee401fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb7f2cf4e6eba5e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:50.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.373+0000 7f2ee8a00640 1 -- 172.21.15.32:0/826166747 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee400a480 con 0x7f2ed00051d0 2024-04-17T10:01:50.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.373+0000 7f2ee8a00640 1 -- 172.21.15.32:0/826166747 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ee40aa510 con 0x7f2ed00051d0 2024-04-17T10:01:50.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.373+0000 7f2ee8a00640 1 -- 172.21.15.32:0/826166747 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee40aa8e0 con 0x7f2ed00051d0 2024-04-17T10:01:50.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.373+0000 7f2ef0a00640 1 -- 172.21.15.32:0/826166747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ed00051d0 msgr2=0x7f2ed000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:50.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.373+0000 7f2ef0a00640 1 --2- 172.21.15.32:0/826166747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ed00051d0 0x7f2ed000b9d0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400a860 tx=0x7f2ee401fae0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.373+0000 7f2ef0a00640 1 -- 172.21.15.32:0/826166747 shutdown_connections 2024-04-17T10:01:50.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.373+0000 7f2ef0a00640 1 --2- 172.21.15.32:0/826166747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ed00051d0 0x7f2ed000b9d0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.373+0000 7f2ef0a00640 1 -- 172.21.15.32:0/826166747 >> 172.21.15.32:0/826166747 conn(0x7f2eec0b2120 msgr2=0x7f2eec0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:50.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.373+0000 7f2ef0a00640 1 -- 172.21.15.32:0/826166747 shutdown_connections 2024-04-17T10:01:50.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.373+0000 7f2ef0a00640 1 -- 172.21.15.32:0/826166747 wait complete. 2024-04-17T10:01:50.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.377+0000 7f2ef0a00640 1 Processor -- start 2024-04-17T10:01:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.377+0000 7f2ef0a00640 1 -- start start 2024-04-17T10:01:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.377+0000 7f2ef0a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ed00051d0 0x7f2eec141050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.377+0000 7f2ef0a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eec144860 0x7f2eec144ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.377+0000 7f2ef0a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2eec1e3a10 0x7f2eec1e3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.377+0000 7f2ef0a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec15cd00 con 0x7f2ed00051d0 2024-04-17T10:01:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.377+0000 7f2ef0a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec15d650 con 0x7f2eec144860 2024-04-17T10:01:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.377+0000 7f2ef0a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2eec15db70 con 0x7f2eec1e3a10 2024-04-17T10:01:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.377+0000 7f2ee9e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ed00051d0 0x7f2eec141050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.377+0000 7f2ee9e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ed00051d0 0x7f2eec141050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33108/0 (socket says 172.21.15.32:33108) 2024-04-17T10:01:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.377+0000 7f2ee9e00640 1 -- 172.21.15.32:0/1593645381 learned_addr learned my addr 172.21.15.32:0/1593645381 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.377+0000 7f2ee9e00640 1 -- 172.21.15.32:0/1593645381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2eec1e3a10 msgr2=0x7f2eec1e3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.377+0000 7f2ee9e00640 1 --2- 172.21.15.32:0/1593645381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2eec1e3a10 0x7f2eec1e3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.377+0000 7f2ee9e00640 1 -- 172.21.15.32:0/1593645381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eec144860 msgr2=0x7f2eec144ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.377+0000 7f2ee9e00640 1 --2- 172.21.15.32:0/1593645381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eec144860 0x7f2eec144ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.377+0000 7f2ee9e00640 1 -- 172.21.15.32:0/1593645381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eec141590 con 0x7f2ed00051d0 2024-04-17T10:01:50.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.377+0000 7f2ee9e00640 1 --2- 172.21.15.32:0/1593645381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ed00051d0 0x7f2eec141050 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f2edc011960 tx=0x7f2edc011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:50.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.377+0000 7f2ee2000640 1 -- 172.21.15.32:0/1593645381 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc00fa90 con 0x7f2ed00051d0 2024-04-17T10:01:50.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.377+0000 7f2ee2000640 1 -- 172.21.15.32:0/1593645381 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2edc00fc30 con 0x7f2ed00051d0 2024-04-17T10:01:50.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.377+0000 7f2ee2000640 1 -- 172.21.15.32:0/1593645381 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc014440 con 0x7f2ed00051d0 2024-04-17T10:01:50.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.377+0000 7f2ef0a00640 1 -- 172.21.15.32:0/1593645381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eec1e4790 con 0x7f2ed00051d0 2024-04-17T10:01:50.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.377+0000 7f2ef0a00640 1 -- 172.21.15.32:0/1593645381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eec1e4d50 con 0x7f2ed00051d0 2024-04-17T10:01:50.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.381+0000 7f2ef0a00640 1 -- 172.21.15.32:0/1593645381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eec145220 con 0x7f2ed00051d0 2024-04-17T10:01:50.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.381+0000 7f2ee2000640 1 -- 172.21.15.32:0/1593645381 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2edc012040 con 0x7f2ed00051d0 2024-04-17T10:01:50.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.381+0000 7f2ee2000640 1 --2- 172.21.15.32:0/1593645381 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2ecc0683b0 0x7f2ecc06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:50.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.381+0000 7f2ee2000640 1 -- 172.21.15.32:0/1593645381 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f2edc086f00 con 0x7f2ed00051d0 2024-04-17T10:01:50.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.381+0000 7f2ee9400640 1 --2- 172.21.15.32:0/1593645381 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2ecc0683b0 0x7f2ecc06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:50.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.381+0000 7f2ee9400640 1 --2- 172.21.15.32:0/1593645381 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2ecc0683b0 0x7f2ecc06a870 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8000c10 tx=0x7f2ed8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:50.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.385+0000 7f2ee2000640 1 -- 172.21.15.32:0/1593645381 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2edc019030 con 0x7f2ed00051d0 2024-04-17T10:01:50.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.553+0000 7f2ef0a00640 1 -- 172.21.15.32:0/1593645381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2eec068a90 con 0x7f2ed00051d0 2024-04-17T10:01:50.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.553+0000 7f2ee2000640 1 -- 172.21.15.32:0/1593645381 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f2edc056e60 con 0x7f2ed00051d0 2024-04-17T10:01:50.550 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T10:01:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.553+0000 7f2ef0a00640 1 -- 172.21.15.32:0/1593645381 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2ecc0683b0 msgr2=0x7f2ecc06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.553+0000 7f2ef0a00640 1 --2- 172.21.15.32:0/1593645381 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2ecc0683b0 0x7f2ecc06a870 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8000c10 tx=0x7f2ed8009040 comp rx=0 tx=0).stop 2024-04-17T10:01:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.553+0000 7f2ef0a00640 1 -- 172.21.15.32:0/1593645381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ed00051d0 msgr2=0x7f2eec141050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.553+0000 7f2ef0a00640 1 --2- 172.21.15.32:0/1593645381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ed00051d0 0x7f2eec141050 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f2edc011960 tx=0x7f2edc011e20 comp rx=0 tx=0).stop 2024-04-17T10:01:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.557+0000 7f2ef0a00640 1 -- 172.21.15.32:0/1593645381 shutdown_connections 2024-04-17T10:01:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.557+0000 7f2ef0a00640 1 --2- 172.21.15.32:0/1593645381 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2ecc0683b0 0x7f2ecc06a870 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.557+0000 7f2ef0a00640 1 --2- 172.21.15.32:0/1593645381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2eec1e3a10 0x7f2eec1e3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.557+0000 7f2ef0a00640 1 --2- 172.21.15.32:0/1593645381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eec144860 0x7f2eec144ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.557+0000 7f2ef0a00640 1 --2- 172.21.15.32:0/1593645381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ed00051d0 0x7f2eec141050 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.557+0000 7f2ef0a00640 1 -- 172.21.15.32:0/1593645381 >> 172.21.15.32:0/1593645381 conn(0x7f2eec0b2120 msgr2=0x7f2eec15bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:50.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.557+0000 7f2ef0a00640 1 -- 172.21.15.32:0/1593645381 shutdown_connections 2024-04-17T10:01:50.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.557+0000 7f2ef0a00640 1 -- 172.21.15.32:0/1593645381 wait complete. 2024-04-17T10:01:50.573 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:01:50.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.717+0000 7f4b94800640 1 Processor -- start 2024-04-17T10:01:50.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.717+0000 7f4b94800640 1 -- start start 2024-04-17T10:01:50.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.717+0000 7f4b94800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b8c0cf370 0x7f4b8c0d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:50.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.717+0000 7f4b94800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4b8c0d0820 0x7f4b8c071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:50.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.717+0000 7f4b94800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b8c0cff60 con 0x7f4b8c13f6e0 2024-04-17T10:01:50.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.717+0000 7f4b94800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f4b8c13fac0 con 0x7f4b8c0d0820 2024-04-17T10:01:50.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.717+0000 7f4b94800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4b8c1601e0 con 0x7f4b8c0cf370 2024-04-17T10:01:50.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.717+0000 7f4b91e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4b8c13f6e0 0x7f4b8c0cf850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33634/0 (socket says 172.21.15.32:33634) 2024-04-17T10:01:50.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.717+0000 7f4b91e00640 1 -- 172.21.15.32:0/684615267 learned_addr learned my addr 172.21.15.32:0/684615267 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:50.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.721+0000 7f4b92800640 1 --2- 172.21.15.32:0/684615267 >> v2:172.21.15.144:3300/0 conn(0x7f4b8c0d0820 0x7f4b8c071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:50.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.721+0000 7f4b91400640 1 --2- 172.21.15.32:0/684615267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b8c0cf370 0x7f4b8c0d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:50.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.721+0000 7f4b90a00640 1 -- 172.21.15.32:0/684615267 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2122277148 0 0) 0x7f4b8c0cff60 con 0x7f4b8c13f6e0 2024-04-17T10:01:50.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.721+0000 7f4b90a00640 1 -- 172.21.15.32:0/684615267 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b74003530 con 0x7f4b8c13f6e0 2024-04-17T10:01:50.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.721+0000 7f4b92800640 1 -- 172.21.15.32:0/684615267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b8c0cf370 msgr2=0x7f4b8c0d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:50.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.721+0000 7f4b92800640 1 --2- 172.21.15.32:0/684615267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b8c0cf370 0x7f4b8c0d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.721+0000 7f4b92800640 1 -- 172.21.15.32:0/684615267 >> v1:172.21.15.32:6789/0 conn(0x7f4b8c13f6e0 legacy=0x7f4b8c0cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:50.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.721+0000 7f4b92800640 1 -- 172.21.15.32:0/684615267 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8c160480 con 0x7f4b8c0d0820 2024-04-17T10:01:50.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.721+0000 7f4b92800640 1 --2- 172.21.15.32:0/684615267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b8c0d0820 0x7f4b8c071d80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009d90 tx=0x7f4b8801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed93ed345f283482 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:50.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.721+0000 7f4b90a00640 1 -- 172.21.15.32:0/684615267 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b880b2070 con 0x7f4b8c0d0820 2024-04-17T10:01:50.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.721+0000 7f4b90a00640 1 -- 172.21.15.32:0/684615267 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b880a5920 con 0x7f4b8c0d0820 2024-04-17T10:01:50.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.721+0000 7f4b90a00640 1 -- 172.21.15.32:0/684615267 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b880a5cf0 con 0x7f4b8c0d0820 2024-04-17T10:01:50.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b94800640 1 -- 172.21.15.32:0/684615267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b8c0d0820 msgr2=0x7f4b8c071d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:50.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b94800640 1 --2- 172.21.15.32:0/684615267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b8c0d0820 0x7f4b8c071d80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009d90 tx=0x7f4b8801c970 comp rx=0 tx=0).stop 2024-04-17T10:01:50.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b94800640 1 -- 172.21.15.32:0/684615267 shutdown_connections 2024-04-17T10:01:50.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b94800640 1 --2- 172.21.15.32:0/684615267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b8c0d0820 0x7f4b8c071d80 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b94800640 1 --2- 172.21.15.32:0/684615267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b8c0cf370 0x7f4b8c0d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b94800640 1 -- 172.21.15.32:0/684615267 >> 172.21.15.32:0/684615267 conn(0x7f4b8c06ba30 msgr2=0x7f4b8c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:50.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b94800640 1 -- 172.21.15.32:0/684615267 shutdown_connections 2024-04-17T10:01:50.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b94800640 1 -- 172.21.15.32:0/684615267 wait complete. 2024-04-17T10:01:50.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b94800640 1 Processor -- start 2024-04-17T10:01:50.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b94800640 1 -- start start 2024-04-17T10:01:50.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b94800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b8c0cf370 0x7f4b8c0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:50.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b94800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b8c0d0820 0x7f4b8c158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:50.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b94800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b8c13f6e0 0x7f4b8c139d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:50.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b94800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c072ad0 con 0x7f4b8c0d0820 2024-04-17T10:01:50.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b94800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c13fd40 con 0x7f4b8c13f6e0 2024-04-17T10:01:50.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b94800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4b8c072c50 con 0x7f4b8c0cf370 2024-04-17T10:01:50.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b91400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b8c0d0820 0x7f4b8c158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:50.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b91400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b8c0d0820 0x7f4b8c158370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33114/0 (socket says 172.21.15.32:33114) 2024-04-17T10:01:50.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b91400640 1 -- 172.21.15.32:0/4010515413 learned_addr learned my addr 172.21.15.32:0/4010515413 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:50.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b91e00640 1 --2- 172.21.15.32:0/4010515413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b8c0cf370 0x7f4b8c0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:50.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b92800640 1 --2- 172.21.15.32:0/4010515413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b8c13f6e0 0x7f4b8c139d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:50.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b91400640 1 -- 172.21.15.32:0/4010515413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b8c0cf370 msgr2=0x7f4b8c0affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:50.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b91400640 1 --2- 172.21.15.32:0/4010515413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b8c0cf370 0x7f4b8c0affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b91400640 1 -- 172.21.15.32:0/4010515413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b8c13f6e0 msgr2=0x7f4b8c139d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:50.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b91400640 1 --2- 172.21.15.32:0/4010515413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b8c13f6e0 0x7f4b8c139d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b91400640 1 -- 172.21.15.32:0/4010515413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8c144080 con 0x7f4b8c0d0820 2024-04-17T10:01:50.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.725+0000 7f4b92800640 1 --2- 172.21.15.32:0/4010515413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b8c13f6e0 0x7f4b8c139d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:50.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.729+0000 7f4b91400640 1 --2- 172.21.15.32:0/4010515413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b8c0d0820 0x7f4b8c158370 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00e870 tx=0x7f4b7c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:50.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.729+0000 7f4b86000640 1 -- 172.21.15.32:0/4010515413 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c017070 con 0x7f4b8c0d0820 2024-04-17T10:01:50.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.729+0000 7f4b86000640 1 -- 172.21.15.32:0/4010515413 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b7c00c940 con 0x7f4b8c0d0820 2024-04-17T10:01:50.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.729+0000 7f4b86000640 1 -- 172.21.15.32:0/4010515413 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c00cd30 con 0x7f4b8c0d0820 2024-04-17T10:01:50.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.729+0000 7f4b94800640 1 -- 172.21.15.32:0/4010515413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b8c144360 con 0x7f4b8c0d0820 2024-04-17T10:01:50.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.729+0000 7f4b94800640 1 -- 172.21.15.32:0/4010515413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b8c155a70 con 0x7f4b8c0d0820 2024-04-17T10:01:50.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.729+0000 7f4b94800640 1 -- 172.21.15.32:0/4010515413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b8c068a90 con 0x7f4b8c0d0820 2024-04-17T10:01:50.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.729+0000 7f4b86000640 1 -- 172.21.15.32:0/4010515413 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4b7c01a070 con 0x7f4b8c0d0820 2024-04-17T10:01:50.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.729+0000 7f4b86000640 1 --2- 172.21.15.32:0/4010515413 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4b700683b0 0x7f4b7006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:50.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.729+0000 7f4b86000640 1 -- 172.21.15.32:0/4010515413 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f4b7c086c10 con 0x7f4b8c0d0820 2024-04-17T10:01:50.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.729+0000 7f4b91e00640 1 --2- 172.21.15.32:0/4010515413 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4b700683b0 0x7f4b7006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:50.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.733+0000 7f4b91e00640 1 --2- 172.21.15.32:0/4010515413 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4b700683b0 0x7f4b7006a870 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4b80005890 tx=0x7f4b80009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:50.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.733+0000 7f4b86000640 1 -- 172.21.15.32:0/4010515413 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f4b7c057130 con 0x7f4b8c0d0820 2024-04-17T10:01:50.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.913+0000 7f4b94800640 1 -- 172.21.15.32:0/4010515413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4b8c0d1030 con 0x7f4b8c0d0820 2024-04-17T10:01:50.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.913+0000 7f4b86000640 1 -- 172.21.15.32:0/4010515413 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v97) ==== 80+0+0 (secure 0 0 0) 0x7f4b7c056a30 con 0x7f4b8c0d0820 2024-04-17T10:01:50.911 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T10:01:50.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.917+0000 7f4b94800640 1 -- 172.21.15.32:0/4010515413 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4b700683b0 msgr2=0x7f4b7006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:50.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.917+0000 7f4b94800640 1 --2- 172.21.15.32:0/4010515413 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4b700683b0 0x7f4b7006a870 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4b80005890 tx=0x7f4b80009040 comp rx=0 tx=0).stop 2024-04-17T10:01:50.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.917+0000 7f4b94800640 1 -- 172.21.15.32:0/4010515413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b8c0d0820 msgr2=0x7f4b8c158370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:50.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.917+0000 7f4b94800640 1 --2- 172.21.15.32:0/4010515413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b8c0d0820 0x7f4b8c158370 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00e870 tx=0x7f4b7c00ec70 comp rx=0 tx=0).stop 2024-04-17T10:01:50.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.917+0000 7f4b94800640 1 -- 172.21.15.32:0/4010515413 shutdown_connections 2024-04-17T10:01:50.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.917+0000 7f4b94800640 1 --2- 172.21.15.32:0/4010515413 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4b700683b0 0x7f4b7006a870 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.917+0000 7f4b94800640 1 --2- 172.21.15.32:0/4010515413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b8c13f6e0 0x7f4b8c139d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.917+0000 7f4b94800640 1 --2- 172.21.15.32:0/4010515413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b8c0d0820 0x7f4b8c158370 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.917+0000 7f4b94800640 1 --2- 172.21.15.32:0/4010515413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b8c0cf370 0x7f4b8c0affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:50.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.917+0000 7f4b94800640 1 -- 172.21.15.32:0/4010515413 >> 172.21.15.32:0/4010515413 conn(0x7f4b8c06ba30 msgr2=0x7f4b8c06ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:50.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.917+0000 7f4b94800640 1 -- 172.21.15.32:0/4010515413 shutdown_connections 2024-04-17T10:01:50.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:50.917+0000 7f4b94800640 1 -- 172.21.15.32:0/4010515413 wait complete. 2024-04-17T10:01:50.932 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:01:50.932 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:01:50.941 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:01:51.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.125+0000 7f3e15600640 1 Processor -- start 2024-04-17T10:01:51.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.125+0000 7f3e15600640 1 -- start start 2024-04-17T10:01:51.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e15600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3e1015baa0 0x7f3e1015bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:51.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e15600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e1006ef60 0x7f3e1006f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:51.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e15600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e10072bb0 con 0x7f3e10144e90 2024-04-17T10:01:51.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e15600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f3e1006f8f0 con 0x7f3e1015baa0 2024-04-17T10:01:51.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e15600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3e10145270 con 0x7f3e1006ef60 2024-04-17T10:01:51.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e0ea00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3e10144e90 0x7f3e100b48a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33650/0 (socket says 172.21.15.32:33650) 2024-04-17T10:01:51.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e0ea00640 1 -- 172.21.15.32:0/2601673785 learned_addr learned my addr 172.21.15.32:0/2601673785 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e0e000640 1 -- 172.21.15.32:0/2601673785 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 662899822 0 0) 0x7f3e10072bb0 con 0x7f3e10144e90 2024-04-17T10:01:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e0f400640 1 --2- 172.21.15.32:0/2601673785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e1006ef60 0x7f3e1006f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e0e000640 1 -- 172.21.15.32:0/2601673785 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3df4005740 con 0x7f3e10144e90 2024-04-17T10:01:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e06000640 1 --2- 172.21.15.32:0/2601673785 >> v2:172.21.15.144:3300/0 conn(0x7f3e1015baa0 0x7f3e1015bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e0e000640 1 -- 172.21.15.32:0/2601673785 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3663554971 0 0) 0x7f3df4005740 con 0x7f3e10144e90 2024-04-17T10:01:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e0e000640 1 -- 172.21.15.32:0/2601673785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e1006ef60 msgr2=0x7f3e1006f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e0e000640 1 --2- 172.21.15.32:0/2601673785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e1006ef60 0x7f3e1006f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e0e000640 1 -- 172.21.15.32:0/2601673785 >> v2:172.21.15.144:3300/0 conn(0x7f3e1015baa0 msgr2=0x7f3e1015bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e0e000640 1 --2- 172.21.15.32:0/2601673785 >> v2:172.21.15.144:3300/0 conn(0x7f3e1015baa0 0x7f3e1015bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e0e000640 1 -- 172.21.15.32:0/2601673785 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e100b3dc0 con 0x7f3e10144e90 2024-04-17T10:01:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e0e000640 1 -- 172.21.15.32:0/2601673785 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f3e000016a0 con 0x7f3e10144e90 2024-04-17T10:01:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e06000640 1 --2- 172.21.15.32:0/2601673785 >> v2:172.21.15.144:3300/0 conn(0x7f3e1015baa0 0x7f3e1015bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e0e000640 1 -- 172.21.15.32:0/2601673785 >> v1:172.21.15.32:6789/0 conn(0x7f3e10144e90 legacy=0x7f3e100b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:51.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e0e000640 1 --2- 172.21.15.32:0/2601673785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3df4005c00 0x7f3df4006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:51.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e0e000640 1 --2- 172.21.15.32:0/2601673785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df4006a20 0x7f3df4006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:51.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e0e000640 1 --2- 172.21.15.32:0/2601673785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e1006ef60 0x7f3df40073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:51.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e0e000640 1 -- 172.21.15.32:0/2601673785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1006f8f0 con 0x7f3df4006a20 2024-04-17T10:01:51.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e0e000640 1 -- 172.21.15.32:0/2601673785 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3e10145270 con 0x7f3df4005c00 2024-04-17T10:01:51.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e0e000640 1 -- 172.21.15.32:0/2601673785 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3df400dc60 con 0x7f3e1006ef60 2024-04-17T10:01:51.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e06000640 1 --2- 172.21.15.32:0/2601673785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df4006a20 0x7f3df4006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:51.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e0ea00640 1 --2- 172.21.15.32:0/2601673785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3df4005c00 0x7f3df4006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:51.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e0f400640 1 --2- 172.21.15.32:0/2601673785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e1006ef60 0x7f3df40073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:51.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e06000640 1 -- 172.21.15.32:0/2601673785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e1006ef60 msgr2=0x7f3df40073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:51.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e06000640 1 --2- 172.21.15.32:0/2601673785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e1006ef60 0x7f3df40073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e06000640 1 -- 172.21.15.32:0/2601673785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3df4005c00 msgr2=0x7f3df4006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:51.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e06000640 1 --2- 172.21.15.32:0/2601673785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3df4005c00 0x7f3df4006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.129+0000 7f3e06000640 1 -- 172.21.15.32:0/2601673785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df4005740 con 0x7f3df4006a20 2024-04-17T10:01:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.133+0000 7f3e06000640 1 --2- 172.21.15.32:0/2601673785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df4006a20 0x7f3df4006ea0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f3df80098d0 tx=0x7f3df801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81997afc804f12f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.133+0000 7f3e0e000640 1 -- 172.21.15.32:0/2601673785 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df80b1070 con 0x7f3df4006a20 2024-04-17T10:01:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.133+0000 7f3e0e000640 1 -- 172.21.15.32:0/2601673785 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3df80a4a90 con 0x7f3df4006a20 2024-04-17T10:01:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.133+0000 7f3e0e000640 1 -- 172.21.15.32:0/2601673785 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df800f040 con 0x7f3df4006a20 2024-04-17T10:01:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.133+0000 7f3e15600640 1 -- 172.21.15.32:0/2601673785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df4006a20 msgr2=0x7f3df4006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:51.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.133+0000 7f3e15600640 1 --2- 172.21.15.32:0/2601673785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df4006a20 0x7f3df4006ea0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f3df80098d0 tx=0x7f3df801c330 comp rx=0 tx=0).stop 2024-04-17T10:01:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.133+0000 7f3e0f400640 1 -- 172.21.15.32:0/2601673785 reap_dead start 2024-04-17T10:01:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.133+0000 7f3e15600640 1 -- 172.21.15.32:0/2601673785 shutdown_connections 2024-04-17T10:01:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.133+0000 7f3e15600640 1 -- 172.21.15.32:0/2601673785 >> 172.21.15.32:0/2601673785 conn(0x7f3e1006ba30 msgr2=0x7f3e1006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.133+0000 7f3e15600640 1 -- 172.21.15.32:0/2601673785 shutdown_connections 2024-04-17T10:01:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.133+0000 7f3e15600640 1 -- 172.21.15.32:0/2601673785 wait complete. 2024-04-17T10:01:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e15600640 1 Processor -- start 2024-04-17T10:01:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e15600640 1 -- start start 2024-04-17T10:01:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e15600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df4006a20 0x7f3e1013f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e15600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e100b1410 0x7f3e100b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e15600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e1015d6c0 0x7f3e1015db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e15600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3e100b4fb0 con 0x7f3df4006a20 2024-04-17T10:01:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e15600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3e100b4380 con 0x7f3e100b1410 2024-04-17T10:01:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e15600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3e100b5130 con 0x7f3e1015d6c0 2024-04-17T10:01:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e0ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df4006a20 0x7f3e1013f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e0ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df4006a20 0x7f3e1013f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33128/0 (socket says 172.21.15.32:33128) 2024-04-17T10:01:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e0ea00640 1 -- 172.21.15.32:0/2635427958 learned_addr learned my addr 172.21.15.32:0/2635427958 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e0f400640 1 --2- 172.21.15.32:0/2635427958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e1015d6c0 0x7f3e1015db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e06000640 1 --2- 172.21.15.32:0/2635427958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e100b1410 0x7f3e100b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e0ea00640 1 -- 172.21.15.32:0/2635427958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e1015d6c0 msgr2=0x7f3e1015db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e0ea00640 1 --2- 172.21.15.32:0/2635427958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e1015d6c0 0x7f3e1015db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e0ea00640 1 -- 172.21.15.32:0/2635427958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e100b1410 msgr2=0x7f3e100b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e0ea00640 1 --2- 172.21.15.32:0/2635427958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e100b1410 0x7f3e100b1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e0ea00640 1 -- 172.21.15.32:0/2635427958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1015e420 con 0x7f3df4006a20 2024-04-17T10:01:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e0f400640 1 --2- 172.21.15.32:0/2635427958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e1015d6c0 0x7f3e1015db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e06000640 1 --2- 172.21.15.32:0/2635427958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e100b1410 0x7f3e100b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:01:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e0ea00640 1 --2- 172.21.15.32:0/2635427958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df4006a20 0x7f3e1013f860 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f3e00012540 tx=0x7f3e00012a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e07400640 1 -- 172.21.15.32:0/2635427958 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0001c070 con 0x7f3df4006a20 2024-04-17T10:01:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e07400640 1 -- 172.21.15.32:0/2635427958 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e00014810 con 0x7f3df4006a20 2024-04-17T10:01:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e07400640 1 -- 172.21.15.32:0/2635427958 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e00014bc0 con 0x7f3df4006a20 2024-04-17T10:01:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e15600640 1 -- 172.21.15.32:0/2635427958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e1015e700 con 0x7f3df4006a20 2024-04-17T10:01:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e15600640 1 -- 172.21.15.32:0/2635427958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e1015ecc0 con 0x7f3df4006a20 2024-04-17T10:01:51.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.137+0000 7f3e15600640 1 -- 172.21.15.32:0/2635427958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e10068a90 con 0x7f3df4006a20 2024-04-17T10:01:51.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.141+0000 7f3e07400640 1 -- 172.21.15.32:0/2635427958 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3e0001f070 con 0x7f3df4006a20 2024-04-17T10:01:51.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.141+0000 7f3e07400640 1 --2- 172.21.15.32:0/2635427958 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3dec068380 0x7f3dec06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:51.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.141+0000 7f3e07400640 1 -- 172.21.15.32:0/2635427958 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f3e00089450 con 0x7f3df4006a20 2024-04-17T10:01:51.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.141+0000 7f3e06000640 1 --2- 172.21.15.32:0/2635427958 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3dec068380 0x7f3dec06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:51.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.141+0000 7f3e06000640 1 --2- 172.21.15.32:0/2635427958 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3dec068380 0x7f3dec06a840 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3df80ae000 tx=0x7f3df80046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:51.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.145+0000 7f3e07400640 1 -- 172.21.15.32:0/2635427958 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3e00059a30 con 0x7f3df4006a20 2024-04-17T10:01:51.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.313+0000 7f3e15600640 1 -- 172.21.15.32:0/2635427958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e100cd010 con 0x7f3df4006a20 2024-04-17T10:01:51.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.313+0000 7f3e07400640 1 -- 172.21.15.32:0/2635427958 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+16620 (secure 0 0 0) 0x7f3e00059330 con 0x7f3df4006a20 2024-04-17T10:01:51.312 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:51.312 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":97,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:01:42.155683+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:01:38.938460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-17T10:01:39.505171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:01:51.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.317+0000 7f3e15600640 1 -- 172.21.15.32:0/2635427958 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3dec068380 msgr2=0x7f3dec06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:51.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.317+0000 7f3e15600640 1 --2- 172.21.15.32:0/2635427958 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3dec068380 0x7f3dec06a840 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3df80ae000 tx=0x7f3df80046d0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.317+0000 7f3e15600640 1 -- 172.21.15.32:0/2635427958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df4006a20 msgr2=0x7f3e1013f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:51.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.317+0000 7f3e15600640 1 --2- 172.21.15.32:0/2635427958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df4006a20 0x7f3e1013f860 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f3e00012540 tx=0x7f3e00012a00 comp rx=0 tx=0).stop 2024-04-17T10:01:51.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.317+0000 7f3e15600640 1 -- 172.21.15.32:0/2635427958 shutdown_connections 2024-04-17T10:01:51.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.317+0000 7f3e15600640 1 --2- 172.21.15.32:0/2635427958 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3dec068380 0x7f3dec06a840 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.317+0000 7f3e15600640 1 --2- 172.21.15.32:0/2635427958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e1015d6c0 0x7f3e1015db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.317+0000 7f3e15600640 1 --2- 172.21.15.32:0/2635427958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e100b1410 0x7f3e100b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.317+0000 7f3e15600640 1 --2- 172.21.15.32:0/2635427958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df4006a20 0x7f3e1013f860 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.317+0000 7f3e15600640 1 -- 172.21.15.32:0/2635427958 >> 172.21.15.32:0/2635427958 conn(0x7f3e1006ba30 msgr2=0x7f3e1006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:51.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.317+0000 7f3e15600640 1 -- 172.21.15.32:0/2635427958 shutdown_connections 2024-04-17T10:01:51.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.317+0000 7f3e15600640 1 -- 172.21.15.32:0/2635427958 wait complete. 2024-04-17T10:01:51.334 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:01:38.938460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:01:39.505171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:01:51.335 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:01:51.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.481+0000 7f0a81000640 1 Processor -- start 2024-04-17T10:01:51.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.481+0000 7f0a81000640 1 -- start start 2024-04-17T10:01:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.481+0000 7f0a81000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0a7c15e880 0x7f0a7c0b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.481+0000 7f0a81000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0a7c06c140 0x7f0a7c0cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.485+0000 7f0a81000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a7c0d0690 con 0x7f0a7c0b0a30 2024-04-17T10:01:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.485+0000 7f0a81000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0a7c0d11d0 con 0x7f0a7c15e880 2024-04-17T10:01:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.485+0000 7f0a81000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0a7c0d0930 con 0x7f0a7c06c140 2024-04-17T10:01:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.485+0000 7f0a79e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0a7c15e880 0x7f0a7c0b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.485+0000 7f0a79e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0a7c15e880 0x7f0a7c0b04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33140/0 (socket says 172.21.15.32:33140) 2024-04-17T10:01:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.485+0000 7f0a79e00640 1 -- 172.21.15.32:0/295949132 learned_addr learned my addr 172.21.15.32:0/295949132 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.485+0000 7f0a7a800640 1 --2- 172.21.15.32:0/295949132 >> v2:172.21.15.144:3300/0 conn(0x7f0a7c06c140 0x7f0a7c0cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.485+0000 7f0a79e00640 1 -- 172.21.15.32:0/295949132 >> v2:172.21.15.144:3300/0 conn(0x7f0a7c06c140 msgr2=0x7f0a7c0cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:51.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.485+0000 7f0a79e00640 1 --2- 172.21.15.32:0/295949132 >> v2:172.21.15.144:3300/0 conn(0x7f0a7c06c140 0x7f0a7c0cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.485+0000 7f0a79e00640 1 -- 172.21.15.32:0/295949132 >> v1:172.21.15.144:6789/0 conn(0x7f0a7c0b0a30 legacy=0x7f0a7c06ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:51.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.485+0000 7f0a79e00640 1 -- 172.21.15.32:0/295949132 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7c0d0c10 con 0x7f0a7c15e880 2024-04-17T10:01:51.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.485+0000 7f0a79e00640 1 --2- 172.21.15.32:0/295949132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a7c15e880 0x7f0a7c0b04f0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0733b0 tx=0x7f0a6c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9abb3a108ad98c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:51.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.485+0000 7f0a79400640 1 -- 172.21.15.32:0/295949132 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c0ab620 con 0x7f0a7c15e880 2024-04-17T10:01:51.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.485+0000 7f0a79400640 1 -- 172.21.15.32:0/295949132 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a6c0b0070 con 0x7f0a7c15e880 2024-04-17T10:01:51.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.485+0000 7f0a79400640 1 -- 172.21.15.32:0/295949132 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c0ab9e0 con 0x7f0a7c15e880 2024-04-17T10:01:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a81000640 1 -- 172.21.15.32:0/295949132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a7c15e880 msgr2=0x7f0a7c0b04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a81000640 1 --2- 172.21.15.32:0/295949132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a7c15e880 0x7f0a7c0b04f0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0733b0 tx=0x7f0a6c01c460 comp rx=0 tx=0).stop 2024-04-17T10:01:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a81000640 1 -- 172.21.15.32:0/295949132 shutdown_connections 2024-04-17T10:01:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a81000640 1 --2- 172.21.15.32:0/295949132 >> v2:172.21.15.144:3300/0 conn(0x7f0a7c06c140 0x7f0a7c0cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a81000640 1 --2- 172.21.15.32:0/295949132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a7c15e880 0x7f0a7c0b04f0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a81000640 1 -- 172.21.15.32:0/295949132 >> 172.21.15.32:0/295949132 conn(0x7f0a7c09a9f0 msgr2=0x7f0a7c0b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a81000640 1 -- 172.21.15.32:0/295949132 shutdown_connections 2024-04-17T10:01:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a81000640 1 -- 172.21.15.32:0/295949132 wait complete. 2024-04-17T10:01:51.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a81000640 1 Processor -- start 2024-04-17T10:01:51.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a81000640 1 -- start start 2024-04-17T10:01:51.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a81000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a7c06c140 0x7f0a7c152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:51.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a81000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a7c0b0a30 0x7f0a7c15d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a81000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a7c15e880 0x7f0a7c15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a81000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c0cbc20 con 0x7f0a7c15e880 2024-04-17T10:01:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a81000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c0cbaa0 con 0x7f0a7c06c140 2024-04-17T10:01:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a81000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0a7c16b490 con 0x7f0a7c0b0a30 2024-04-17T10:01:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a7a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a7c15e880 0x7f0a7c15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a7a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a7c15e880 0x7f0a7c15dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33150/0 (socket says 172.21.15.32:33150) 2024-04-17T10:01:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a7a800640 1 -- 172.21.15.32:0/3612383599 learned_addr learned my addr 172.21.15.32:0/3612383599 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a71400640 1 --2- 172.21.15.32:0/3612383599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a7c0b0a30 0x7f0a7c15d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a79e00640 1 --2- 172.21.15.32:0/3612383599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a7c06c140 0x7f0a7c152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a7a800640 1 -- 172.21.15.32:0/3612383599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a7c0b0a30 msgr2=0x7f0a7c15d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a7a800640 1 --2- 172.21.15.32:0/3612383599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a7c0b0a30 0x7f0a7c15d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a7a800640 1 -- 172.21.15.32:0/3612383599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a7c06c140 msgr2=0x7f0a7c152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a7a800640 1 --2- 172.21.15.32:0/3612383599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a7c06c140 0x7f0a7c152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a7a800640 1 -- 172.21.15.32:0/3612383599 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7c0afe40 con 0x7f0a7c15e880 2024-04-17T10:01:51.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.489+0000 7f0a79e00640 1 --2- 172.21.15.32:0/3612383599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a7c06c140 0x7f0a7c152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:51.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.493+0000 7f0a7a800640 1 --2- 172.21.15.32:0/3612383599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a7c15e880 0x7f0a7c15dcb0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f0a74007e90 tx=0x7f0a7400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:51.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.493+0000 7f0a72a00640 1 -- 172.21.15.32:0/3612383599 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a74014420 con 0x7f0a7c15e880 2024-04-17T10:01:51.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.493+0000 7f0a72a00640 1 -- 172.21.15.32:0/3612383599 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a74019070 con 0x7f0a7c15e880 2024-04-17T10:01:51.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.493+0000 7f0a72a00640 1 -- 172.21.15.32:0/3612383599 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a74014800 con 0x7f0a7c15e880 2024-04-17T10:01:51.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.493+0000 7f0a81000640 1 -- 172.21.15.32:0/3612383599 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7c0b0120 con 0x7f0a7c15e880 2024-04-17T10:01:51.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.493+0000 7f0a81000640 1 -- 172.21.15.32:0/3612383599 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a7c0c5260 con 0x7f0a7c15e880 2024-04-17T10:01:51.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.493+0000 7f0a81000640 1 -- 172.21.15.32:0/3612383599 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a7c068a90 con 0x7f0a7c15e880 2024-04-17T10:01:51.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.493+0000 7f0a72a00640 1 -- 172.21.15.32:0/3612383599 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0a740149a0 con 0x7f0a7c15e880 2024-04-17T10:01:51.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.493+0000 7f0a72a00640 1 --2- 172.21.15.32:0/3612383599 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0a580683d0 0x7f0a5806a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:51.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.493+0000 7f0a72a00640 1 -- 172.21.15.32:0/3612383599 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f0a74087140 con 0x7f0a7c15e880 2024-04-17T10:01:51.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.493+0000 7f0a79e00640 1 --2- 172.21.15.32:0/3612383599 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0a580683d0 0x7f0a5806a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.497+0000 7f0a79e00640 1 --2- 172.21.15.32:0/3612383599 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0a580683d0 0x7f0a5806a890 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0af000 tx=0x7f0a6c0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.497+0000 7f0a72a00640 1 -- 172.21.15.32:0/3612383599 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0a740576e0 con 0x7f0a7c15e880 2024-04-17T10:01:51.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.665+0000 7f0a81000640 1 -- 172.21.15.32:0/3612383599 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0a7c169130 con 0x7f0a7c15e880 2024-04-17T10:01:51.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.665+0000 7f0a72a00640 1 -- 172.21.15.32:0/3612383599 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f0a74056fe0 con 0x7f0a7c15e880 2024-04-17T10:01:51.663 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T10:01:51.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.669+0000 7f0a81000640 1 -- 172.21.15.32:0/3612383599 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0a580683d0 msgr2=0x7f0a5806a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:51.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.669+0000 7f0a81000640 1 --2- 172.21.15.32:0/3612383599 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0a580683d0 0x7f0a5806a890 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0af000 tx=0x7f0a6c0a7000 comp rx=0 tx=0).stop 2024-04-17T10:01:51.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.669+0000 7f0a81000640 1 -- 172.21.15.32:0/3612383599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a7c15e880 msgr2=0x7f0a7c15dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:51.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.669+0000 7f0a81000640 1 --2- 172.21.15.32:0/3612383599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a7c15e880 0x7f0a7c15dcb0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f0a74007e90 tx=0x7f0a7400e4b0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.669+0000 7f0a81000640 1 -- 172.21.15.32:0/3612383599 shutdown_connections 2024-04-17T10:01:51.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.669+0000 7f0a81000640 1 --2- 172.21.15.32:0/3612383599 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0a580683d0 0x7f0a5806a890 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.669+0000 7f0a81000640 1 --2- 172.21.15.32:0/3612383599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a7c0b0a30 0x7f0a7c15d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.669+0000 7f0a81000640 1 --2- 172.21.15.32:0/3612383599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a7c15e880 0x7f0a7c15dcb0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.669+0000 7f0a81000640 1 --2- 172.21.15.32:0/3612383599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a7c06c140 0x7f0a7c152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.669+0000 7f0a81000640 1 -- 172.21.15.32:0/3612383599 >> 172.21.15.32:0/3612383599 conn(0x7f0a7c09a9f0 msgr2=0x7f0a7c0cc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:51.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.669+0000 7f0a81000640 1 -- 172.21.15.32:0/3612383599 shutdown_connections 2024-04-17T10:01:51.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.669+0000 7f0a81000640 1 -- 172.21.15.32:0/3612383599 wait complete. 2024-04-17T10:01:51.684 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:01:51.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.829+0000 7f9080c00640 1 Processor -- start 2024-04-17T10:01:51.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.829+0000 7f9080c00640 1 -- start start 2024-04-17T10:01:51.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.833+0000 7f9080c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f907c06f7c0 0x7f907c06ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:51.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.833+0000 7f9080c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f907c0cc940 con 0x7f907c06f270 2024-04-17T10:01:51.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.833+0000 7f9080c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f907c0ccc20 con 0x7f907c15b7d0 2024-04-17T10:01:51.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.833+0000 7f9080c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f907c06e0b0 con 0x7f907c06f7c0 2024-04-17T10:01:51.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.833+0000 7f907a800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f907c06f270 0x7f907c0ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33652/0 (socket says 172.21.15.32:33652) 2024-04-17T10:01:51.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.833+0000 7f907a800640 1 -- 172.21.15.32:0/562013866 learned_addr learned my addr 172.21.15.32:0/562013866 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:51.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.833+0000 7f9079400640 1 --2- 172.21.15.32:0/562013866 >> v2:172.21.15.144:3300/0 conn(0x7f907c06f7c0 0x7f907c06ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:51.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.833+0000 7f9078a00640 1 -- 172.21.15.32:0/562013866 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2528419205 0 0) 0x7f907c0cc940 con 0x7f907c06f270 2024-04-17T10:01:51.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.833+0000 7f9078a00640 1 -- 172.21.15.32:0/562013866 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9060003530 con 0x7f907c06f270 2024-04-17T10:01:51.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.833+0000 7f9078a00640 1 -- 172.21.15.32:0/562013866 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1292274918 0 0) 0x7f907c0ccc20 con 0x7f907c15b7d0 2024-04-17T10:01:51.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.833+0000 7f9078a00640 1 -- 172.21.15.32:0/562013866 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f907c0cc940 con 0x7f907c15b7d0 2024-04-17T10:01:51.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.833+0000 7f9079400640 1 -- 172.21.15.32:0/562013866 >> v1:172.21.15.144:6789/0 conn(0x7f907c15b7d0 legacy=0x7f907c06e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:51.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.833+0000 7f9079400640 1 -- 172.21.15.32:0/562013866 >> v1:172.21.15.32:6789/0 conn(0x7f907c06f270 legacy=0x7f907c0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:51.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.833+0000 7f9079400640 1 -- 172.21.15.32:0/562013866 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907c06e390 con 0x7f907c06f7c0 2024-04-17T10:01:51.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f9079400640 1 --2- 172.21.15.32:0/562013866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f907c06f7c0 0x7f907c06ba30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f90680097c0 tx=0x7f906801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1084a0b1249158c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:51.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f9078a00640 1 -- 172.21.15.32:0/562013866 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90680ab430 con 0x7f907c06f7c0 2024-04-17T10:01:51.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f9078a00640 1 -- 172.21.15.32:0/562013866 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90680b0070 con 0x7f907c06f7c0 2024-04-17T10:01:51.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f9078a00640 1 -- 172.21.15.32:0/562013866 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90680ab7f0 con 0x7f907c06f7c0 2024-04-17T10:01:51.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f9080c00640 1 -- 172.21.15.32:0/562013866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f907c06f7c0 msgr2=0x7f907c06ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:51.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f9080c00640 1 --2- 172.21.15.32:0/562013866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f907c06f7c0 0x7f907c06ba30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f90680097c0 tx=0x7f906801c410 comp rx=0 tx=0).stop 2024-04-17T10:01:51.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f9080c00640 1 -- 172.21.15.32:0/562013866 shutdown_connections 2024-04-17T10:01:51.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f9080c00640 1 --2- 172.21.15.32:0/562013866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f907c06f7c0 0x7f907c06ba30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f9080c00640 1 -- 172.21.15.32:0/562013866 >> 172.21.15.32:0/562013866 conn(0x7f907c0b2120 msgr2=0x7f907c0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:51.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f9080c00640 1 -- 172.21.15.32:0/562013866 shutdown_connections 2024-04-17T10:01:51.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f9080c00640 1 -- 172.21.15.32:0/562013866 wait complete. 2024-04-17T10:01:51.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f9080c00640 1 Processor -- start 2024-04-17T10:01:51.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f9080c00640 1 -- start start 2024-04-17T10:01:51.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f9080c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f907c06f270 0x7f907c1505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:51.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f9080c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f907c06f7c0 0x7f907c1570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:51.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f9080c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f907c15b7d0 0x7f907c157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:51.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f9080c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f907c169cd0 con 0x7f907c06f270 2024-04-17T10:01:51.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f9080c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f907c06c380 con 0x7f907c15b7d0 2024-04-17T10:01:51.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f9080c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f907c169e50 con 0x7f907c06f7c0 2024-04-17T10:01:51.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f9079e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f907c06f270 0x7f907c1505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:51.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f9079e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f907c06f270 0x7f907c1505c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33166/0 (socket says 172.21.15.32:33166) 2024-04-17T10:01:51.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f9079e00640 1 -- 172.21.15.32:0/3834966821 learned_addr learned my addr 172.21.15.32:0/3834966821 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:51.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f907a800640 1 --2- 172.21.15.32:0/3834966821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f907c15b7d0 0x7f907c157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:51.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.837+0000 7f9079400640 1 --2- 172.21.15.32:0/3834966821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f907c06f7c0 0x7f907c1570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:51.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.841+0000 7f9079e00640 1 -- 172.21.15.32:0/3834966821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f907c06f7c0 msgr2=0x7f907c1570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:51.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.841+0000 7f9079e00640 1 --2- 172.21.15.32:0/3834966821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f907c06f7c0 0x7f907c1570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.841+0000 7f9079e00640 1 -- 172.21.15.32:0/3834966821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f907c15b7d0 msgr2=0x7f907c157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:51.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.841+0000 7f9079e00640 1 --2- 172.21.15.32:0/3834966821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f907c15b7d0 0x7f907c157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:51.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.841+0000 7f9079e00640 1 -- 172.21.15.32:0/3834966821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907c150df0 con 0x7f907c06f270 2024-04-17T10:01:51.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.841+0000 7f9079e00640 1 --2- 172.21.15.32:0/3834966821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f907c06f270 0x7f907c1505c0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f906c00c490 tx=0x7f906c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:51.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.841+0000 7f9072000640 1 -- 172.21.15.32:0/3834966821 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c0052d0 con 0x7f907c06f270 2024-04-17T10:01:51.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.841+0000 7f9072000640 1 -- 172.21.15.32:0/3834966821 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f906c005470 con 0x7f907c06f270 2024-04-17T10:01:51.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.841+0000 7f9072000640 1 -- 172.21.15.32:0/3834966821 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c012870 con 0x7f907c06f270 2024-04-17T10:01:51.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.841+0000 7f9080c00640 1 -- 172.21.15.32:0/3834966821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907c1510d0 con 0x7f907c06f270 2024-04-17T10:01:51.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.841+0000 7f9080c00640 1 -- 172.21.15.32:0/3834966821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f907c151690 con 0x7f907c06f270 2024-04-17T10:01:51.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.841+0000 7f9080c00640 1 -- 172.21.15.32:0/3834966821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f907c068a90 con 0x7f907c06f270 2024-04-17T10:01:51.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.841+0000 7f9072000640 1 -- 172.21.15.32:0/3834966821 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f906c012a10 con 0x7f907c06f270 2024-04-17T10:01:51.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.841+0000 7f9072000640 1 --2- 172.21.15.32:0/3834966821 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f905c068380 0x7f905c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:51.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.841+0000 7f9072000640 1 -- 172.21.15.32:0/3834966821 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f906c0864f0 con 0x7f907c06f270 2024-04-17T10:01:51.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.841+0000 7f9079400640 1 --2- 172.21.15.32:0/3834966821 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f905c068380 0x7f905c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:51.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.845+0000 7f9079400640 1 --2- 172.21.15.32:0/3834966821 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f905c068380 0x7f905c06a840 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f90680af000 tx=0x7f90680a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:51.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:51.849+0000 7f9072000640 1 -- 172.21.15.32:0/3834966821 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f906c056460 con 0x7f907c06f270 2024-04-17T10:01:52.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.009+0000 7f9080c00640 1 -- 172.21.15.32:0/3834966821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f907c15a820 con 0x7f907c06f270 2024-04-17T10:01:52.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.013+0000 7f9072000640 1 -- 172.21.15.32:0/3834966821 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f906c05a760 con 0x7f907c06f270 2024-04-17T10:01:52.009 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T10:01:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.013+0000 7f9080c00640 1 -- 172.21.15.32:0/3834966821 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f905c068380 msgr2=0x7f905c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.013+0000 7f9080c00640 1 --2- 172.21.15.32:0/3834966821 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f905c068380 0x7f905c06a840 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f90680af000 tx=0x7f90680a7000 comp rx=0 tx=0).stop 2024-04-17T10:01:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.013+0000 7f9080c00640 1 -- 172.21.15.32:0/3834966821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f907c06f270 msgr2=0x7f907c1505c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.013+0000 7f9080c00640 1 --2- 172.21.15.32:0/3834966821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f907c06f270 0x7f907c1505c0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f906c00c490 tx=0x7f906c00c950 comp rx=0 tx=0).stop 2024-04-17T10:01:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.013+0000 7f9080c00640 1 -- 172.21.15.32:0/3834966821 shutdown_connections 2024-04-17T10:01:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.013+0000 7f9080c00640 1 --2- 172.21.15.32:0/3834966821 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f905c068380 0x7f905c06a840 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.013+0000 7f9080c00640 1 --2- 172.21.15.32:0/3834966821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f907c06f7c0 0x7f907c1570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.013+0000 7f9080c00640 1 --2- 172.21.15.32:0/3834966821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f907c15b7d0 0x7f907c157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.013+0000 7f9080c00640 1 --2- 172.21.15.32:0/3834966821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f907c06f270 0x7f907c1505c0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.013+0000 7f9080c00640 1 -- 172.21.15.32:0/3834966821 >> 172.21.15.32:0/3834966821 conn(0x7f907c0b2120 msgr2=0x7f907c15a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.013+0000 7f9080c00640 1 -- 172.21.15.32:0/3834966821 shutdown_connections 2024-04-17T10:01:52.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.013+0000 7f9080c00640 1 -- 172.21.15.32:0/3834966821 wait complete. 2024-04-17T10:01:52.030 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:01:52.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.177+0000 7fb88ce00640 1 Processor -- start 2024-04-17T10:01:52.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.177+0000 7fb88ce00640 1 -- start start 2024-04-17T10:01:52.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.177+0000 7fb88ce00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb88806c140 0x7fb8880ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:52.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.177+0000 7fb88ce00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8880cc940 con 0x7fb88806f7c0 2024-04-17T10:01:52.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.177+0000 7fb88ce00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8880ccc20 con 0x7fb88815b7d0 2024-04-17T10:01:52.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.177+0000 7fb88ce00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb88806e0b0 con 0x7fb88806c140 2024-04-17T10:01:52.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.177+0000 7fb885400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb88806f7c0 0x7fb88806ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33662/0 (socket says 172.21.15.32:33662) 2024-04-17T10:01:52.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.177+0000 7fb885400640 1 -- 172.21.15.32:0/1003273720 learned_addr learned my addr 172.21.15.32:0/1003273720 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:52.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.177+0000 7fb886800640 1 --2- 172.21.15.32:0/1003273720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb88806c140 0x7fb8880ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:52.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.177+0000 7fb884a00640 1 -- 172.21.15.32:0/1003273720 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3653281873 0 0) 0x7fb8880cc940 con 0x7fb88806f7c0 2024-04-17T10:01:52.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.177+0000 7fb884a00640 1 -- 172.21.15.32:0/1003273720 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb86c003530 con 0x7fb88806f7c0 2024-04-17T10:01:52.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.177+0000 7fb884a00640 1 -- 172.21.15.32:0/1003273720 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1766884034 0 0) 0x7fb8880ccc20 con 0x7fb88815b7d0 2024-04-17T10:01:52.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.177+0000 7fb884a00640 1 -- 172.21.15.32:0/1003273720 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8880cc940 con 0x7fb88815b7d0 2024-04-17T10:01:52.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.177+0000 7fb886800640 1 -- 172.21.15.32:0/1003273720 >> v1:172.21.15.144:6789/0 conn(0x7fb88815b7d0 legacy=0x7fb88806e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:52.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.177+0000 7fb886800640 1 -- 172.21.15.32:0/1003273720 >> v1:172.21.15.32:6789/0 conn(0x7fb88806f7c0 legacy=0x7fb88806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:52.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.177+0000 7fb886800640 1 -- 172.21.15.32:0/1003273720 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88806e390 con 0x7fb88806c140 2024-04-17T10:01:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.181+0000 7fb886800640 1 --2- 172.21.15.32:0/1003273720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb88806c140 0x7fb8880ce290 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb880009dc0 tx=0x7fb88001c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48d0ce6065f85aaf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:52.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.181+0000 7fb884a00640 1 -- 172.21.15.32:0/1003273720 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8800b2070 con 0x7fb88806c140 2024-04-17T10:01:52.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.181+0000 7fb884a00640 1 -- 172.21.15.32:0/1003273720 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8800a59e0 con 0x7fb88806c140 2024-04-17T10:01:52.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.181+0000 7fb884a00640 1 -- 172.21.15.32:0/1003273720 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8800a5db0 con 0x7fb88806c140 2024-04-17T10:01:52.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.181+0000 7fb88ce00640 1 -- 172.21.15.32:0/1003273720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb88806c140 msgr2=0x7fb8880ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:52.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.181+0000 7fb88ce00640 1 --2- 172.21.15.32:0/1003273720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb88806c140 0x7fb8880ce290 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb880009dc0 tx=0x7fb88001c900 comp rx=0 tx=0).stop 2024-04-17T10:01:52.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.181+0000 7fb88ce00640 1 -- 172.21.15.32:0/1003273720 shutdown_connections 2024-04-17T10:01:52.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.181+0000 7fb88ce00640 1 --2- 172.21.15.32:0/1003273720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb88806c140 0x7fb8880ce290 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.181+0000 7fb88ce00640 1 -- 172.21.15.32:0/1003273720 >> 172.21.15.32:0/1003273720 conn(0x7fb8880b2120 msgr2=0x7fb8880b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:52.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.181+0000 7fb88ce00640 1 -- 172.21.15.32:0/1003273720 shutdown_connections 2024-04-17T10:01:52.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.181+0000 7fb88ce00640 1 -- 172.21.15.32:0/1003273720 wait complete. 2024-04-17T10:01:52.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb88ce00640 1 Processor -- start 2024-04-17T10:01:52.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb88ce00640 1 -- start start 2024-04-17T10:01:52.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb88ce00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb88806c140 0x7fb8881549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:52.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb88ce00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb88806f7c0 0x7fb888157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:52.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb88ce00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb88815b7d0 0x7fb88815a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:52.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb88ce00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb888169640 con 0x7fb88815b7d0 2024-04-17T10:01:52.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb88ce00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8881694c0 con 0x7fb88806c140 2024-04-17T10:01:52.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb88ce00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb8881697c0 con 0x7fb88806f7c0 2024-04-17T10:01:52.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb886800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb88815b7d0 0x7fb88815a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:52.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb886800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb88815b7d0 0x7fb88815a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33168/0 (socket says 172.21.15.32:33168) 2024-04-17T10:01:52.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb886800640 1 -- 172.21.15.32:0/3591415020 learned_addr learned my addr 172.21.15.32:0/3591415020 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:52.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb885400640 1 --2- 172.21.15.32:0/3591415020 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb88806f7c0 0x7fb888157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:52.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb885e00640 1 --2- 172.21.15.32:0/3591415020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb88806c140 0x7fb8881549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:52.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb886800640 1 -- 172.21.15.32:0/3591415020 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb88806f7c0 msgr2=0x7fb888157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:52.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb886800640 1 --2- 172.21.15.32:0/3591415020 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb88806f7c0 0x7fb888157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb886800640 1 -- 172.21.15.32:0/3591415020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb88806c140 msgr2=0x7fb8881549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:52.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb886800640 1 --2- 172.21.15.32:0/3591415020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb88806c140 0x7fb8881549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb886800640 1 -- 172.21.15.32:0/3591415020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8880b2e20 con 0x7fb88815b7d0 2024-04-17T10:01:52.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb885e00640 1 --2- 172.21.15.32:0/3591415020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb88806c140 0x7fb8881549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:52.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb886800640 1 --2- 172.21.15.32:0/3591415020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb88815b7d0 0x7fb88815a300 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fb880019510 tx=0x7fb8800196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:52.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb87e000640 1 -- 172.21.15.32:0/3591415020 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8800b2050 con 0x7fb88815b7d0 2024-04-17T10:01:52.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb87e000640 1 -- 172.21.15.32:0/3591415020 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8800be070 con 0x7fb88815b7d0 2024-04-17T10:01:52.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb87e000640 1 -- 172.21.15.32:0/3591415020 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8800adc40 con 0x7fb88815b7d0 2024-04-17T10:01:52.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb88ce00640 1 -- 172.21.15.32:0/3591415020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb888154fb0 con 0x7fb88815b7d0 2024-04-17T10:01:52.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb88ce00640 1 -- 172.21.15.32:0/3591415020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb888155570 con 0x7fb88815b7d0 2024-04-17T10:01:52.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.185+0000 7fb88ce00640 1 -- 172.21.15.32:0/3591415020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb888068a90 con 0x7fb88815b7d0 2024-04-17T10:01:52.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.189+0000 7fb87e000640 1 -- 172.21.15.32:0/3591415020 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb8800c3460 con 0x7fb88815b7d0 2024-04-17T10:01:52.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.189+0000 7fb87e000640 1 --2- 172.21.15.32:0/3591415020 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb868068380 0x7fb86806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:52.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.189+0000 7fb87e000640 1 -- 172.21.15.32:0/3591415020 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fb8801218c0 con 0x7fb88815b7d0 2024-04-17T10:01:52.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.189+0000 7fb885e00640 1 --2- 172.21.15.32:0/3591415020 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb868068380 0x7fb86806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:52.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.189+0000 7fb885e00640 1 --2- 172.21.15.32:0/3591415020 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb868068380 0x7fb86806a840 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb8780016a0 tx=0x7fb878009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:52.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.193+0000 7fb87e000640 1 -- 172.21.15.32:0/3591415020 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fb8800c8020 con 0x7fb88815b7d0 2024-04-17T10:01:52.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.357+0000 7fb88ce00640 1 -- 172.21.15.32:0/3591415020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb888159a60 con 0x7fb88815b7d0 2024-04-17T10:01:52.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.361+0000 7fb87e000640 1 -- 172.21.15.32:0/3591415020 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7fb8801210a0 con 0x7fb88815b7d0 2024-04-17T10:01:52.357 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T10:01:52.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.361+0000 7fb88ce00640 1 -- 172.21.15.32:0/3591415020 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb868068380 msgr2=0x7fb86806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:52.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.361+0000 7fb88ce00640 1 --2- 172.21.15.32:0/3591415020 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb868068380 0x7fb86806a840 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb8780016a0 tx=0x7fb878009040 comp rx=0 tx=0).stop 2024-04-17T10:01:52.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.361+0000 7fb88ce00640 1 -- 172.21.15.32:0/3591415020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb88815b7d0 msgr2=0x7fb88815a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:52.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.361+0000 7fb88ce00640 1 --2- 172.21.15.32:0/3591415020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb88815b7d0 0x7fb88815a300 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fb880019510 tx=0x7fb8800196a0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.361+0000 7fb88ce00640 1 -- 172.21.15.32:0/3591415020 shutdown_connections 2024-04-17T10:01:52.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.361+0000 7fb88ce00640 1 --2- 172.21.15.32:0/3591415020 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb868068380 0x7fb86806a840 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.361+0000 7fb88ce00640 1 --2- 172.21.15.32:0/3591415020 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb88806f7c0 0x7fb888157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.361+0000 7fb88ce00640 1 --2- 172.21.15.32:0/3591415020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb88815b7d0 0x7fb88815a300 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.361+0000 7fb88ce00640 1 --2- 172.21.15.32:0/3591415020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb88806c140 0x7fb8881549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.361+0000 7fb88ce00640 1 -- 172.21.15.32:0/3591415020 >> 172.21.15.32:0/3591415020 conn(0x7fb8880b2120 msgr2=0x7fb888152a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:52.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.361+0000 7fb88ce00640 1 -- 172.21.15.32:0/3591415020 shutdown_connections 2024-04-17T10:01:52.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.361+0000 7fb88ce00640 1 -- 172.21.15.32:0/3591415020 wait complete. 2024-04-17T10:01:52.378 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:01:52.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.521+0000 7f232b200640 1 Processor -- start 2024-04-17T10:01:52.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.521+0000 7f232b200640 1 -- start start 2024-04-17T10:01:52.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.525+0000 7f232b200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f232c15e8a0 0x7f232c0d0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:52.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.525+0000 7f232b200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f232c06cb40 0x7f232c0d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:52.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.525+0000 7f232b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f232c0d0ff0 0x7f232c09a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:52.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.525+0000 7f232b200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f232c0d3200 con 0x7f232c06cb40 2024-04-17T10:01:52.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.525+0000 7f232b200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f232c15ec80 con 0x7f232c15e8a0 2024-04-17T10:01:52.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.525+0000 7f232b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f232c0ccad0 con 0x7f232c0d0ff0 2024-04-17T10:01:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.525+0000 7f2321400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f232c06cb40 0x7f232c0d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.525+0000 7f2321400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f232c06cb40 0x7f232c0d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33172/0 (socket says 172.21.15.32:33172) 2024-04-17T10:01:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.525+0000 7f2321400640 1 -- 172.21.15.32:0/1570081342 learned_addr learned my addr 172.21.15.32:0/1570081342 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.525+0000 7f2329e00640 1 --2- 172.21.15.32:0/1570081342 >> v2:172.21.15.144:3300/0 conn(0x7f232c15e8a0 0x7f232c0d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.525+0000 7f232a800640 1 --2- 172.21.15.32:0/1570081342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f232c0d0ff0 0x7f232c09a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.525+0000 7f2321400640 1 -- 172.21.15.32:0/1570081342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f232c0d0ff0 msgr2=0x7f232c09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.525+0000 7f2321400640 1 --2- 172.21.15.32:0/1570081342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f232c0d0ff0 0x7f232c09a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.525+0000 7f2321400640 1 -- 172.21.15.32:0/1570081342 >> v2:172.21.15.144:3300/0 conn(0x7f232c15e8a0 msgr2=0x7f232c0d0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.525+0000 7f2321400640 1 --2- 172.21.15.32:0/1570081342 >> v2:172.21.15.144:3300/0 conn(0x7f232c15e8a0 0x7f232c0d0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.525+0000 7f2321400640 1 -- 172.21.15.32:0/1570081342 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c06cfd0 con 0x7f232c06cb40 2024-04-17T10:01:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.525+0000 7f232a800640 1 --2- 172.21.15.32:0/1570081342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f232c0d0ff0 0x7f232c09a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:52.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.525+0000 7f2321400640 1 --2- 172.21.15.32:0/1570081342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f232c06cb40 0x7f232c0d0ab0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f232c073410 tx=0x7f231401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a42a7e2f331838 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:52.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f2329400640 1 -- 172.21.15.32:0/1570081342 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23140aa610 con 0x7f232c06cb40 2024-04-17T10:01:52.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f2329400640 1 -- 172.21.15.32:0/1570081342 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23140af070 con 0x7f232c06cb40 2024-04-17T10:01:52.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f2329400640 1 -- 172.21.15.32:0/1570081342 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23140aa9d0 con 0x7f232c06cb40 2024-04-17T10:01:52.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f232b200640 1 -- 172.21.15.32:0/1570081342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f232c06cb40 msgr2=0x7f232c0d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:52.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f232b200640 1 --2- 172.21.15.32:0/1570081342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f232c06cb40 0x7f232c0d0ab0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f232c073410 tx=0x7f231401c4f0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f232b200640 1 -- 172.21.15.32:0/1570081342 shutdown_connections 2024-04-17T10:01:52.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f232b200640 1 --2- 172.21.15.32:0/1570081342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f232c0d0ff0 0x7f232c09a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f232b200640 1 --2- 172.21.15.32:0/1570081342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f232c06cb40 0x7f232c0d0ab0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f232b200640 1 --2- 172.21.15.32:0/1570081342 >> v2:172.21.15.144:3300/0 conn(0x7f232c15e8a0 0x7f232c0d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f232b200640 1 -- 172.21.15.32:0/1570081342 >> 172.21.15.32:0/1570081342 conn(0x7f232c0d23e0 msgr2=0x7f232c0d27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:52.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f232b200640 1 -- 172.21.15.32:0/1570081342 shutdown_connections 2024-04-17T10:01:52.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f232b200640 1 -- 172.21.15.32:0/1570081342 wait complete. 2024-04-17T10:01:52.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f232b200640 1 Processor -- start 2024-04-17T10:01:52.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f232b200640 1 -- start start 2024-04-17T10:01:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f232b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f232c0d0ff0 0x7f232c15dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f232b200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f232c15e8a0 0x7f232c0c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f232b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f232c0c7da0 0x7f232c0c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f232b200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f232c09b740 con 0x7f232c0d0ff0 2024-04-17T10:01:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f232b200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f232c0d1400 con 0x7f232c15e8a0 2024-04-17T10:01:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f232b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f232c09b8c0 con 0x7f232c0c7da0 2024-04-17T10:01:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f2329e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f232c0d0ff0 0x7f232c15dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f2329e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f232c0d0ff0 0x7f232c15dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33182/0 (socket says 172.21.15.32:33182) 2024-04-17T10:01:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f2329e00640 1 -- 172.21.15.32:0/4257733391 learned_addr learned my addr 172.21.15.32:0/4257733391 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f2321400640 1 --2- 172.21.15.32:0/4257733391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f232c15e8a0 0x7f232c0c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.529+0000 7f232a800640 1 --2- 172.21.15.32:0/4257733391 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f232c0c7da0 0x7f232c0c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.533+0000 7f2329e00640 1 -- 172.21.15.32:0/4257733391 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f232c0c7da0 msgr2=0x7f232c0c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.533+0000 7f2329e00640 1 --2- 172.21.15.32:0/4257733391 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f232c0c7da0 0x7f232c0c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.533+0000 7f2329e00640 1 -- 172.21.15.32:0/4257733391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f232c15e8a0 msgr2=0x7f232c0c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.533+0000 7f2329e00640 1 --2- 172.21.15.32:0/4257733391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f232c15e8a0 0x7f232c0c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.533+0000 7f2329e00640 1 -- 172.21.15.32:0/4257733391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c0c63f0 con 0x7f232c0d0ff0 2024-04-17T10:01:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.533+0000 7f2329e00640 1 --2- 172.21.15.32:0/4257733391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f232c0d0ff0 0x7f232c15dd70 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f231c00e930 tx=0x7f231c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.533+0000 7f2322a00640 1 -- 172.21.15.32:0/4257733391 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c017070 con 0x7f232c0d0ff0 2024-04-17T10:01:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.533+0000 7f2322a00640 1 -- 172.21.15.32:0/4257733391 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f231c00ca00 con 0x7f232c0d0ff0 2024-04-17T10:01:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.533+0000 7f232b200640 1 -- 172.21.15.32:0/4257733391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232c0c88c0 con 0x7f232c0d0ff0 2024-04-17T10:01:52.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.533+0000 7f2322a00640 1 -- 172.21.15.32:0/4257733391 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c00cdb0 con 0x7f232c0d0ff0 2024-04-17T10:01:52.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.533+0000 7f232b200640 1 -- 172.21.15.32:0/4257733391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f232c15f600 con 0x7f232c0d0ff0 2024-04-17T10:01:52.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.533+0000 7f232b200640 1 -- 172.21.15.32:0/4257733391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f232c0c8de0 con 0x7f232c0d0ff0 2024-04-17T10:01:52.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.533+0000 7f2322a00640 1 -- 172.21.15.32:0/4257733391 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f231c01a070 con 0x7f232c0d0ff0 2024-04-17T10:01:52.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.533+0000 7f2322a00640 1 --2- 172.21.15.32:0/4257733391 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2308068380 0x7f230806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:52.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.533+0000 7f2322a00640 1 -- 172.21.15.32:0/4257733391 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f231c086a10 con 0x7f232c0d0ff0 2024-04-17T10:01:52.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.533+0000 7f2321400640 1 --2- 172.21.15.32:0/4257733391 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2308068380 0x7f230806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:52.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.537+0000 7f2321400640 1 --2- 172.21.15.32:0/4257733391 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2308068380 0x7f230806a840 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f23140ae000 tx=0x7f23140a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:52.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.537+0000 7f2322a00640 1 -- 172.21.15.32:0/4257733391 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f231c0569a0 con 0x7f232c0d0ff0 2024-04-17T10:01:52.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.717+0000 7f232b200640 1 -- 172.21.15.32:0/4257733391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f232c04fd00 con 0x7f232c0d0ff0 2024-04-17T10:01:52.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.717+0000 7f2322a00640 1 -- 172.21.15.32:0/4257733391 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v97) ==== 80+0+0 (secure 0 0 0) 0x7f231c05aca0 con 0x7f232c0d0ff0 2024-04-17T10:01:52.713 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T10:01:52.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.717+0000 7f232b200640 1 -- 172.21.15.32:0/4257733391 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2308068380 msgr2=0x7f230806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:52.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.717+0000 7f232b200640 1 --2- 172.21.15.32:0/4257733391 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2308068380 0x7f230806a840 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f23140ae000 tx=0x7f23140a7000 comp rx=0 tx=0).stop 2024-04-17T10:01:52.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.717+0000 7f232b200640 1 -- 172.21.15.32:0/4257733391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f232c0d0ff0 msgr2=0x7f232c15dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:52.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.717+0000 7f232b200640 1 --2- 172.21.15.32:0/4257733391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f232c0d0ff0 0x7f232c15dd70 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f231c00e930 tx=0x7f231c00edf0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.717+0000 7f232b200640 1 -- 172.21.15.32:0/4257733391 shutdown_connections 2024-04-17T10:01:52.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.717+0000 7f232b200640 1 --2- 172.21.15.32:0/4257733391 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2308068380 0x7f230806a840 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.717+0000 7f232b200640 1 --2- 172.21.15.32:0/4257733391 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f232c0c7da0 0x7f232c0c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.717+0000 7f232b200640 1 --2- 172.21.15.32:0/4257733391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f232c15e8a0 0x7f232c0c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.717+0000 7f232b200640 1 --2- 172.21.15.32:0/4257733391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f232c0d0ff0 0x7f232c15dd70 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:52.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.717+0000 7f232b200640 1 -- 172.21.15.32:0/4257733391 >> 172.21.15.32:0/4257733391 conn(0x7f232c0d23e0 msgr2=0x7f232c071310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:52.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.717+0000 7f232b200640 1 -- 172.21.15.32:0/4257733391 shutdown_connections 2024-04-17T10:01:52.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:52.717+0000 7f232b200640 1 -- 172.21.15.32:0/4257733391 wait complete. 2024-04-17T10:01:52.734 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2024-04-17T10:01:52.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:01:52.735 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T10:01:52.752 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[48511]: fuse finished with error 0 and tester_r 0 2024-04-17T10:01:52.756 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:01:58.858 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T10:01:58.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:01:58.859 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T10:01:58.871 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:01:59.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.064+0000 7f387dc00640 1 Processor -- start 2024-04-17T10:01:59.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.064+0000 7f387dc00640 1 -- start start 2024-04-17T10:01:59.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.068+0000 7f387dc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3878144e90 0x7f38780b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:59.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.068+0000 7f387dc00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f387815c1b0 0x7f387806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:59.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.068+0000 7f387dc00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3878072bb0 con 0x7f38780b4de0 2024-04-17T10:01:59.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.068+0000 7f387dc00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3878145270 con 0x7f387815c1b0 2024-04-17T10:01:59.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.068+0000 7f387dc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f38780b3dc0 con 0x7f3878144e90 2024-04-17T10:01:59.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.068+0000 7f386e000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f38780b4de0 0x7f387815baa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33672/0 (socket says 172.21.15.32:33672) 2024-04-17T10:01:59.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.068+0000 7f386e000640 1 -- 172.21.15.32:0/1375303538 learned_addr learned my addr 172.21.15.32:0/1375303538 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:59.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.068+0000 7f3877400640 1 --2- 172.21.15.32:0/1375303538 >> v2:172.21.15.32:3300/0 conn(0x7f387815c1b0 0x7f387806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:59.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.068+0000 7f3876a00640 1 --2- 172.21.15.32:0/1375303538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3878144e90 0x7f38780b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:59.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.068+0000 7f3876000640 1 -- 172.21.15.32:0/1375303538 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2886540741 0 0) 0x7f3878072bb0 con 0x7f38780b4de0 2024-04-17T10:01:59.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.068+0000 7f3876000640 1 -- 172.21.15.32:0/1375303538 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f385c005740 con 0x7f38780b4de0 2024-04-17T10:01:59.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.068+0000 7f3877400640 1 -- 172.21.15.32:0/1375303538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3878144e90 msgr2=0x7f38780b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:59.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.068+0000 7f3877400640 1 --2- 172.21.15.32:0/1375303538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3878144e90 0x7f38780b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.068+0000 7f3877400640 1 -- 172.21.15.32:0/1375303538 >> v1:172.21.15.32:6789/0 conn(0x7f38780b4de0 legacy=0x7f387815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:59.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.068+0000 7f3877400640 1 -- 172.21.15.32:0/1375303538 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38780b40a0 con 0x7f387815c1b0 2024-04-17T10:01:59.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.068+0000 7f3877400640 1 --2- 172.21.15.32:0/1375303538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f387815c1b0 0x7f387806ef60 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f3870009d90 tx=0x7f387001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd2dbfc2985d069c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:59.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.072+0000 7f3876000640 1 -- 172.21.15.32:0/1375303538 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38700b2070 con 0x7f387815c1b0 2024-04-17T10:01:59.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.072+0000 7f3876000640 1 -- 172.21.15.32:0/1375303538 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38700a5a20 con 0x7f387815c1b0 2024-04-17T10:01:59.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.072+0000 7f3876000640 1 -- 172.21.15.32:0/1375303538 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38700a5dd0 con 0x7f387815c1b0 2024-04-17T10:01:59.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.072+0000 7f387dc00640 1 -- 172.21.15.32:0/1375303538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f387815c1b0 msgr2=0x7f387806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:59.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.072+0000 7f387dc00640 1 --2- 172.21.15.32:0/1375303538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f387815c1b0 0x7f387806ef60 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f3870009d90 tx=0x7f387001c940 comp rx=0 tx=0).stop 2024-04-17T10:01:59.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.072+0000 7f387dc00640 1 -- 172.21.15.32:0/1375303538 shutdown_connections 2024-04-17T10:01:59.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.072+0000 7f387dc00640 1 --2- 172.21.15.32:0/1375303538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f387815c1b0 0x7f387806ef60 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.072+0000 7f387dc00640 1 --2- 172.21.15.32:0/1375303538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3878144e90 0x7f38780b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.072+0000 7f387dc00640 1 -- 172.21.15.32:0/1375303538 >> 172.21.15.32:0/1375303538 conn(0x7f387806ba30 msgr2=0x7f387806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:59.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.072+0000 7f387dc00640 1 -- 172.21.15.32:0/1375303538 shutdown_connections 2024-04-17T10:01:59.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.072+0000 7f387dc00640 1 -- 172.21.15.32:0/1375303538 wait complete. 2024-04-17T10:01:59.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.072+0000 7f387dc00640 1 Processor -- start 2024-04-17T10:01:59.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.072+0000 7f387dc00640 1 -- start start 2024-04-17T10:01:59.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.072+0000 7f387dc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38780b4de0 0x7f3878160d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:59.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.072+0000 7f387dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3878144e90 0x7f387815fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:59.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.072+0000 7f387dc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f387815c1b0 0x7f38780adc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:59.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.072+0000 7f387dc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f387806f8b0 con 0x7f3878144e90 2024-04-17T10:01:59.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.072+0000 7f387dc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3878072880 con 0x7f38780b4de0 2024-04-17T10:01:59.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.072+0000 7f387dc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f387806de50 con 0x7f387815c1b0 2024-04-17T10:01:59.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.072+0000 7f386e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3878144e90 0x7f387815fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:59.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.072+0000 7f386e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3878144e90 0x7f387815fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33204/0 (socket says 172.21.15.32:33204) 2024-04-17T10:01:59.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.072+0000 7f386e000640 1 -- 172.21.15.32:0/838390298 learned_addr learned my addr 172.21.15.32:0/838390298 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:59.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.076+0000 7f3877400640 1 --2- 172.21.15.32:0/838390298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f387815c1b0 0x7f38780adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:59.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.076+0000 7f3876a00640 1 --2- 172.21.15.32:0/838390298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38780b4de0 0x7f3878160d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:59.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.076+0000 7f386e000640 1 -- 172.21.15.32:0/838390298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f387815c1b0 msgr2=0x7f38780adc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:59.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.076+0000 7f386e000640 1 --2- 172.21.15.32:0/838390298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f387815c1b0 0x7f38780adc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.076+0000 7f386e000640 1 -- 172.21.15.32:0/838390298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38780b4de0 msgr2=0x7f3878160d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:59.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.076+0000 7f386e000640 1 --2- 172.21.15.32:0/838390298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38780b4de0 0x7f3878160d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.076+0000 7f386e000640 1 -- 172.21.15.32:0/838390298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38780b0110 con 0x7f3878144e90 2024-04-17T10:01:59.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.076+0000 7f3877400640 1 --2- 172.21.15.32:0/838390298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f387815c1b0 0x7f38780adc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:59.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.076+0000 7f386e000640 1 --2- 172.21.15.32:0/838390298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3878144e90 0x7f387815fa60 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f38600010f0 tx=0x7f386000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:59.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.076+0000 7f386f400640 1 -- 172.21.15.32:0/838390298 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860017070 con 0x7f3878144e90 2024-04-17T10:01:59.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.076+0000 7f386f400640 1 -- 172.21.15.32:0/838390298 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3860005d10 con 0x7f3878144e90 2024-04-17T10:01:59.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.076+0000 7f386f400640 1 -- 172.21.15.32:0/838390298 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860007310 con 0x7f3878144e90 2024-04-17T10:01:59.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.076+0000 7f387dc00640 1 -- 172.21.15.32:0/838390298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38780b03f0 con 0x7f3878144e90 2024-04-17T10:01:59.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.076+0000 7f387dc00640 1 -- 172.21.15.32:0/838390298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38780b09b0 con 0x7f3878144e90 2024-04-17T10:01:59.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.076+0000 7f387dc00640 1 -- 172.21.15.32:0/838390298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3878068a90 con 0x7f3878144e90 2024-04-17T10:01:59.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.076+0000 7f386f400640 1 -- 172.21.15.32:0/838390298 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f386001a070 con 0x7f3878144e90 2024-04-17T10:01:59.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.076+0000 7f386f400640 1 --2- 172.21.15.32:0/838390298 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3854068380 0x7f385406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:59.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.076+0000 7f386f400640 1 -- 172.21.15.32:0/838390298 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f3860085ee0 con 0x7f3878144e90 2024-04-17T10:01:59.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.080+0000 7f3876a00640 1 --2- 172.21.15.32:0/838390298 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3854068380 0x7f385406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:59.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.080+0000 7f3876a00640 1 --2- 172.21.15.32:0/838390298 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3854068380 0x7f385406a840 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3868009af0 tx=0x7f3868008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:59.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.084+0000 7f386f400640 1 -- 172.21.15.32:0/838390298 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3860056400 con 0x7f3878144e90 2024-04-17T10:01:59.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.280+0000 7f387dc00640 1 -- 172.21.15.32:0/838390298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38781428b0 con 0x7f3878144e90 2024-04-17T10:01:59.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.280+0000 7f386f400640 1 -- 172.21.15.32:0/838390298 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5207 (secure 0 0 0) 0x7f3860055d00 con 0x7f3878144e90 2024-04-17T10:01:59.280 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:59.280 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":81,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2635109588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2635109588},{"type":"v1","addr":"172.21.15.144:6835","nonce":2635109588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7789,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3258455939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3258455939},{"type":"v1","addr":"172.21.15.144:6834","nonce":3258455939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":8574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3390468332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3390468332},{"type":"v1","addr":"172.21.15.32:6836","nonce":3390468332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:01:40.557208+0000","modified":"2024-04-17T10:01:43.570474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8577":{"gid":8577,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/1235126741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1235126741},{"type":"v1","addr":"172.21.15.32:6837","nonce":1235126741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8577,"qdb_cluster":[8577]},"id":7}]} 2024-04-17T10:01:59.280 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 81 2024-04-17T10:01:59.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.284+0000 7f387dc00640 1 -- 172.21.15.32:0/838390298 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3854068380 msgr2=0x7f385406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:59.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.284+0000 7f387dc00640 1 --2- 172.21.15.32:0/838390298 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3854068380 0x7f385406a840 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3868009af0 tx=0x7f3868008040 comp rx=0 tx=0).stop 2024-04-17T10:01:59.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.284+0000 7f387dc00640 1 -- 172.21.15.32:0/838390298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3878144e90 msgr2=0x7f387815fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:59.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.284+0000 7f387dc00640 1 --2- 172.21.15.32:0/838390298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3878144e90 0x7f387815fa60 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f38600010f0 tx=0x7f386000c8f0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.284+0000 7f387dc00640 1 -- 172.21.15.32:0/838390298 shutdown_connections 2024-04-17T10:01:59.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.284+0000 7f387dc00640 1 --2- 172.21.15.32:0/838390298 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3854068380 0x7f385406a840 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.284+0000 7f387dc00640 1 --2- 172.21.15.32:0/838390298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f387815c1b0 0x7f38780adc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.284+0000 7f387dc00640 1 --2- 172.21.15.32:0/838390298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3878144e90 0x7f387815fa60 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.284+0000 7f387dc00640 1 --2- 172.21.15.32:0/838390298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38780b4de0 0x7f3878160d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.284+0000 7f387dc00640 1 -- 172.21.15.32:0/838390298 >> 172.21.15.32:0/838390298 conn(0x7f387806ba30 msgr2=0x7f38780af6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:59.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.284+0000 7f387dc00640 1 -- 172.21.15.32:0/838390298 shutdown_connections 2024-04-17T10:01:59.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.284+0000 7f387dc00640 1 -- 172.21.15.32:0/838390298 wait complete. 2024-04-17T10:01:59.302 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-17T10:01:59.473 INFO:teuthology.orchestra.run.smithi032.stdout:{"message":"","return_code":0} 2024-04-17T10:01:59.474 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-17T10:01:59.474 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:01:59.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.616+0000 7f98d6800640 1 Processor -- start 2024-04-17T10:01:59.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.616+0000 7f98d6800640 1 -- start start 2024-04-17T10:01:59.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.616+0000 7f98d6800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f98d0151db0 0x7f98d00b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:59.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.616+0000 7f98d6800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f98d006f630 0x7f98d0072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:59.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.616+0000 7f98d6800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98d00b3490 con 0x7f98d0072bc0 2024-04-17T10:01:59.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.616+0000 7f98d6800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f98d00b44f0 con 0x7f98d006f630 2024-04-17T10:01:59.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.616+0000 7f98d6800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f98d0073010 con 0x7f98d0151db0 2024-04-17T10:01:59.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.616+0000 7f98cfe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f98d0072bc0 0x7f98d015e7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33678/0 (socket says 172.21.15.32:33678) 2024-04-17T10:01:59.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.616+0000 7f98cfe00640 1 -- 172.21.15.32:0/794961293 learned_addr learned my addr 172.21.15.32:0/794961293 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:01:59.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.616+0000 7f98c6a00640 1 --2- 172.21.15.32:0/794961293 >> v2:172.21.15.32:3300/0 conn(0x7f98d006f630 0x7f98d0072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:59.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.620+0000 7f98cf400640 1 --2- 172.21.15.32:0/794961293 >> v2:172.21.15.144:3300/0 conn(0x7f98d0151db0 0x7f98d00b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:59.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.620+0000 7f98cea00640 1 -- 172.21.15.32:0/794961293 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1099769383 0 0) 0x7f98d00b3490 con 0x7f98d0072bc0 2024-04-17T10:01:59.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.620+0000 7f98cea00640 1 -- 172.21.15.32:0/794961293 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98b4005740 con 0x7f98d0072bc0 2024-04-17T10:01:59.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.620+0000 7f98c6a00640 1 -- 172.21.15.32:0/794961293 >> v2:172.21.15.144:3300/0 conn(0x7f98d0151db0 msgr2=0x7f98d00b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:59.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.620+0000 7f98c6a00640 1 --2- 172.21.15.32:0/794961293 >> v2:172.21.15.144:3300/0 conn(0x7f98d0151db0 0x7f98d00b51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.620+0000 7f98c6a00640 1 -- 172.21.15.32:0/794961293 >> v1:172.21.15.32:6789/0 conn(0x7f98d0072bc0 legacy=0x7f98d015e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:59.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.620+0000 7f98c6a00640 1 -- 172.21.15.32:0/794961293 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d00cea40 con 0x7f98d006f630 2024-04-17T10:01:59.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.620+0000 7f98cf400640 1 --2- 172.21.15.32:0/794961293 >> v2:172.21.15.144:3300/0 conn(0x7f98d0151db0 0x7f98d00b51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:01:59.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.620+0000 7f98c6a00640 1 --2- 172.21.15.32:0/794961293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98d006f630 0x7f98d0072680 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f98d0073260 tx=0x7f98c001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737c2b8bec2ef92c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:59.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.620+0000 7f98cea00640 1 -- 172.21.15.32:0/794961293 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c00b0070 con 0x7f98d006f630 2024-04-17T10:01:59.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.620+0000 7f98cea00640 1 -- 172.21.15.32:0/794961293 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98c00ab600 con 0x7f98d006f630 2024-04-17T10:01:59.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.620+0000 7f98cea00640 1 -- 172.21.15.32:0/794961293 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c00ab9d0 con 0x7f98d006f630 2024-04-17T10:01:59.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.620+0000 7f98d6800640 1 -- 172.21.15.32:0/794961293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98d006f630 msgr2=0x7f98d0072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:59.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.620+0000 7f98d6800640 1 --2- 172.21.15.32:0/794961293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98d006f630 0x7f98d0072680 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f98d0073260 tx=0x7f98c001c480 comp rx=0 tx=0).stop 2024-04-17T10:01:59.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98d6800640 1 -- 172.21.15.32:0/794961293 shutdown_connections 2024-04-17T10:01:59.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98d6800640 1 --2- 172.21.15.32:0/794961293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98d006f630 0x7f98d0072680 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98d6800640 1 --2- 172.21.15.32:0/794961293 >> v2:172.21.15.144:3300/0 conn(0x7f98d0151db0 0x7f98d00b51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98d6800640 1 -- 172.21.15.32:0/794961293 >> 172.21.15.32:0/794961293 conn(0x7f98d006ba30 msgr2=0x7f98d006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:59.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98d6800640 1 -- 172.21.15.32:0/794961293 shutdown_connections 2024-04-17T10:01:59.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98d6800640 1 -- 172.21.15.32:0/794961293 wait complete. 2024-04-17T10:01:59.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98d6800640 1 Processor -- start 2024-04-17T10:01:59.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98d6800640 1 -- start start 2024-04-17T10:01:59.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98d6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98d006f630 0x7f98d0149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:59.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98d6800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98d0072bc0 0x7f98d014c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:59.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98d6800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98d0151db0 0x7f98d0147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:59.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98d6800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f98d00b5700 con 0x7f98d006f630 2024-04-17T10:01:59.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98d6800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f98d00b3fd0 con 0x7f98d0072bc0 2024-04-17T10:01:59.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98d6800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f98d00b5880 con 0x7f98d0151db0 2024-04-17T10:01:59.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98cf400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98d006f630 0x7f98d0149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:59.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98cf400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98d006f630 0x7f98d0149ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33218/0 (socket says 172.21.15.32:33218) 2024-04-17T10:01:59.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98cf400640 1 -- 172.21.15.32:0/2473511621 learned_addr learned my addr 172.21.15.32:0/2473511621 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:59.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98c6a00640 1 --2- 172.21.15.32:0/2473511621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98d0072bc0 0x7f98d014c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:59.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98cfe00640 1 --2- 172.21.15.32:0/2473511621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98d0151db0 0x7f98d0147fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:59.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98cf400640 1 -- 172.21.15.32:0/2473511621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98d0151db0 msgr2=0x7f98d0147fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:59.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98cf400640 1 --2- 172.21.15.32:0/2473511621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98d0151db0 0x7f98d0147fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98cf400640 1 -- 172.21.15.32:0/2473511621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98d0072bc0 msgr2=0x7f98d014c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:59.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98cf400640 1 --2- 172.21.15.32:0/2473511621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98d0072bc0 0x7f98d014c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98cf400640 1 -- 172.21.15.32:0/2473511621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d0148e00 con 0x7f98d006f630 2024-04-17T10:01:59.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98cf400640 1 --2- 172.21.15.32:0/2473511621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98d006f630 0x7f98d0149ea0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f98b800e840 tx=0x7f98b800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:59.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98c7e00640 1 -- 172.21.15.32:0/2473511621 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b8017070 con 0x7f98d006f630 2024-04-17T10:01:59.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98c7e00640 1 -- 172.21.15.32:0/2473511621 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b800c8d0 con 0x7f98d006f630 2024-04-17T10:01:59.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98c7e00640 1 -- 172.21.15.32:0/2473511621 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b800f040 con 0x7f98d006f630 2024-04-17T10:01:59.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.624+0000 7f98d6800640 1 -- 172.21.15.32:0/2473511621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d01490e0 con 0x7f98d006f630 2024-04-17T10:01:59.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.628+0000 7f98d6800640 1 -- 172.21.15.32:0/2473511621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d01496a0 con 0x7f98d006f630 2024-04-17T10:01:59.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.628+0000 7f98d6800640 1 -- 172.21.15.32:0/2473511621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98d0068a90 con 0x7f98d006f630 2024-04-17T10:01:59.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.628+0000 7f98c7e00640 1 -- 172.21.15.32:0/2473511621 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f98b801a070 con 0x7f98d006f630 2024-04-17T10:01:59.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.628+0000 7f98c7e00640 1 --2- 172.21.15.32:0/2473511621 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f98ac068380 0x7f98ac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:59.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.628+0000 7f98c7e00640 1 -- 172.21.15.32:0/2473511621 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f98b80865a0 con 0x7f98d006f630 2024-04-17T10:01:59.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.628+0000 7f98c6a00640 1 --2- 172.21.15.32:0/2473511621 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f98ac068380 0x7f98ac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:59.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.632+0000 7f98c6a00640 1 --2- 172.21.15.32:0/2473511621 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f98ac068380 0x7f98ac06a840 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f98c00ad000 tx=0x7f98c0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:59.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.632+0000 7f98c7e00640 1 -- 172.21.15.32:0/2473511621 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f98b8056500 con 0x7f98d006f630 2024-04-17T10:01:59.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.832+0000 7f98d6800640 1 -- 172.21.15.32:0/2473511621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98d00b1f00 con 0x7f98d006f630 2024-04-17T10:01:59.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.832+0000 7f98c7e00640 1 -- 172.21.15.32:0/2473511621 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5207 (secure 0 0 0) 0x7f98b805a800 con 0x7f98d006f630 2024-04-17T10:01:59.830 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:01:59.830 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":81,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2635109588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2635109588},{"type":"v1","addr":"172.21.15.144:6835","nonce":2635109588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7789,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3258455939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3258455939},{"type":"v1","addr":"172.21.15.144:6834","nonce":3258455939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":8574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3390468332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3390468332},{"type":"v1","addr":"172.21.15.32:6836","nonce":3390468332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:01:40.557208+0000","modified":"2024-04-17T10:01:43.570474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8577":{"gid":8577,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/1235126741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1235126741},{"type":"v1","addr":"172.21.15.32:6837","nonce":1235126741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8577,"qdb_cluster":[8577]},"id":7}]} 2024-04-17T10:01:59.831 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 81 2024-04-17T10:01:59.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.832+0000 7f98d6800640 1 -- 172.21.15.32:0/2473511621 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f98ac068380 msgr2=0x7f98ac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:59.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.832+0000 7f98d6800640 1 --2- 172.21.15.32:0/2473511621 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f98ac068380 0x7f98ac06a840 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f98c00ad000 tx=0x7f98c0008040 comp rx=0 tx=0).stop 2024-04-17T10:01:59.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.832+0000 7f98d6800640 1 -- 172.21.15.32:0/2473511621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98d006f630 msgr2=0x7f98d0149ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:59.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.832+0000 7f98d6800640 1 --2- 172.21.15.32:0/2473511621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98d006f630 0x7f98d0149ea0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f98b800e840 tx=0x7f98b800ed00 comp rx=0 tx=0).stop 2024-04-17T10:01:59.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.832+0000 7f98d6800640 1 -- 172.21.15.32:0/2473511621 shutdown_connections 2024-04-17T10:01:59.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.832+0000 7f98d6800640 1 --2- 172.21.15.32:0/2473511621 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f98ac068380 0x7f98ac06a840 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.832+0000 7f98d6800640 1 --2- 172.21.15.32:0/2473511621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98d0151db0 0x7f98d0147fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.832+0000 7f98d6800640 1 --2- 172.21.15.32:0/2473511621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98d0072bc0 0x7f98d014c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.832+0000 7f98d6800640 1 --2- 172.21.15.32:0/2473511621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98d006f630 0x7f98d0149ea0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.832+0000 7f98d6800640 1 -- 172.21.15.32:0/2473511621 >> 172.21.15.32:0/2473511621 conn(0x7f98d006ba30 msgr2=0x7f98d006be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.832+0000 7f98d6800640 1 -- 172.21.15.32:0/2473511621 shutdown_connections 2024-04-17T10:01:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.832+0000 7f98d6800640 1 -- 172.21.15.32:0/2473511621 wait complete. 2024-04-17T10:01:59.852 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-04-17T10:01:59.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.900+0000 7f93110c8080 1 Processor -- start 2024-04-17T10:01:59.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.900+0000 7f93110c8080 1 -- start start 2024-04-17T10:01:59.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.900+0000 7f93110c8080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48e63f230 0x55e48e6ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:59.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.900+0000 7f93110c8080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e48e63f610 0x55e48e639bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:59.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.900+0000 7f93110c8080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e48e6cc270 0x55e48e63a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:59.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.900+0000 7f93110c8080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e48e59bf90 con 0x55e48e63f230 2024-04-17T10:01:59.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.900+0000 7f93110c8080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e48e59ffd0 con 0x55e48e63f610 2024-04-17T10:01:59.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.900+0000 7f93110c8080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e48e6ccd60 con 0x55e48e6cc270 2024-04-17T10:01:59.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.900+0000 7f930e200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48e63f230 0x55e48e6ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:59.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.900+0000 7f930e200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48e63f230 0x55e48e6ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33222/0 (socket says 172.21.15.32:33222) 2024-04-17T10:01:59.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.900+0000 7f930e200640 1 -- 172.21.15.32:0/213809168 learned_addr learned my addr 172.21.15.32:0/213809168 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:59.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.900+0000 7f930ec00640 1 --2- 172.21.15.32:0/213809168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e48e6cc270 0x55e48e63a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:59.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.900+0000 7f930d800640 1 --2- 172.21.15.32:0/213809168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e48e63f610 0x55e48e639bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:59.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f930d800640 1 -- 172.21.15.32:0/213809168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e48e6cc270 msgr2=0x55e48e63a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:59.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f930d800640 1 --2- 172.21.15.32:0/213809168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e48e6cc270 0x55e48e63a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f930d800640 1 -- 172.21.15.32:0/213809168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48e63f230 msgr2=0x55e48e6ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:01:59.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f930d800640 1 --2- 172.21.15.32:0/213809168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48e63f230 0x55e48e6ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f930d800640 1 -- 172.21.15.32:0/213809168 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e48e6cb830 con 0x55e48e63f610 2024-04-17T10:01:59.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f930ec00640 1 --2- 172.21.15.32:0/213809168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e48e6cc270 0x55e48e63a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:01:59.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f930d800640 1 --2- 172.21.15.32:0/213809168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e48e63f610 0x55e48e639bb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x55e48e5fd1f0 tx=0x7f92fc003030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91132d8a3fdb69e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:59.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f9307e00640 1 -- 172.21.15.32:0/213809168 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc0af040 con 0x55e48e63f610 2024-04-17T10:01:59.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f9307e00640 1 -- 172.21.15.32:0/213809168 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92fc0b9070 con 0x55e48e63f610 2024-04-17T10:01:59.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f9307e00640 1 -- 172.21.15.32:0/213809168 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc0aa9d0 con 0x55e48e63f610 2024-04-17T10:01:59.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f93110c8080 1 -- 172.21.15.32:0/213809168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e48e63f610 msgr2=0x55e48e639bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:59.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f93110c8080 1 --2- 172.21.15.32:0/213809168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e48e63f610 0x55e48e639bb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x55e48e5fd1f0 tx=0x7f92fc003030 comp rx=0 tx=0).stop 2024-04-17T10:01:59.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f93110c8080 1 -- 172.21.15.32:0/213809168 shutdown_connections 2024-04-17T10:01:59.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f93110c8080 1 --2- 172.21.15.32:0/213809168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e48e6cc270 0x55e48e63a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f93110c8080 1 --2- 172.21.15.32:0/213809168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e48e63f610 0x55e48e639bb0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f93110c8080 1 --2- 172.21.15.32:0/213809168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48e63f230 0x55e48e6ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f93110c8080 1 -- 172.21.15.32:0/213809168 >> 172.21.15.32:0/213809168 conn(0x55e48e5f5620 msgr2=0x55e48e6266d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:59.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f93110c8080 1 -- 172.21.15.32:0/213809168 shutdown_connections 2024-04-17T10:01:59.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f93110c8080 1 -- 172.21.15.32:0/213809168 wait complete. 2024-04-17T10:01:59.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f93110c8080 1 Processor -- start 2024-04-17T10:01:59.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f93110c8080 1 -- start start 2024-04-17T10:01:59.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f93110c8080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e48e63f230 0x55e48e61a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:59.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f93110c8080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e48e63f610 0x55e48e6ab070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:59.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f93110c8080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48e6cc270 0x55e48e63c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:59.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f93110c8080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55e48e6ac140 con 0x55e48e6cc270 2024-04-17T10:01:59.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f93110c8080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e48e6abee0 con 0x55e48e63f230 2024-04-17T10:01:59.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f93110c8080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e48e6ac3e0 con 0x55e48e63f610 2024-04-17T10:01:59.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f930ec00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48e6cc270 0x55e48e63c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:59.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f930ec00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48e6cc270 0x55e48e63c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33228/0 (socket says 172.21.15.32:33228) 2024-04-17T10:01:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f930ec00640 1 -- 172.21.15.32:0/1639993208 learned_addr learned my addr 172.21.15.32:0/1639993208 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:01:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f930d800640 1 --2- 172.21.15.32:0/1639993208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e48e63f610 0x55e48e6ab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.904+0000 7f930e200640 1 --2- 172.21.15.32:0/1639993208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e48e63f230 0x55e48e61a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.908+0000 7f930ec00640 1 -- 172.21.15.32:0/1639993208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e48e63f610 msgr2=0x55e48e6ab070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.908+0000 7f930ec00640 1 --2- 172.21.15.32:0/1639993208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e48e63f610 0x55e48e6ab070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.908+0000 7f930ec00640 1 -- 172.21.15.32:0/1639993208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e48e63f230 msgr2=0x55e48e61a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.908+0000 7f930ec00640 1 --2- 172.21.15.32:0/1639993208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e48e63f230 0x55e48e61a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.908+0000 7f930ec00640 1 -- 172.21.15.32:0/1639993208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e48e622350 con 0x55e48e6cc270 2024-04-17T10:01:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.908+0000 7f930ec00640 1 --2- 172.21.15.32:0/1639993208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48e6cc270 0x55e48e63c280 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f930800d470 tx=0x7f930800d930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.908+0000 7f9306a00640 1 -- 172.21.15.32:0/1639993208 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93080125f0 con 0x55e48e6cc270 2024-04-17T10:01:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.908+0000 7f9306a00640 1 -- 172.21.15.32:0/1639993208 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9308012790 con 0x55e48e6cc270 2024-04-17T10:01:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.908+0000 7f9306a00640 1 -- 172.21.15.32:0/1639993208 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308012980 con 0x55e48e6cc270 2024-04-17T10:01:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.908+0000 7f93110c8080 1 -- 172.21.15.32:0/1639993208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e48e6225f0 con 0x55e48e6cc270 2024-04-17T10:01:59.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.908+0000 7f93110c8080 1 -- 172.21.15.32:0/1639993208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e48e6239c0 con 0x55e48e6cc270 2024-04-17T10:01:59.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.908+0000 7f9306a00640 1 -- 172.21.15.32:0/1639993208 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f930801a020 con 0x55e48e6cc270 2024-04-17T10:01:59.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.908+0000 7f9306a00640 1 --2- 172.21.15.32:0/1639993208 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f92f0066d60 0x7f92f0069220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:59.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.908+0000 7f9306a00640 1 -- 172.21.15.32:0/1639993208 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f9308084ec0 con 0x55e48e6cc270 2024-04-17T10:01:59.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.908+0000 7f930e200640 1 --2- 172.21.15.32:0/1639993208 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f92f0066d60 0x7f92f0069220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:59.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.908+0000 7f93110c8080 1 --2- 172.21.15.32:0/1639993208 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55e48e63c7c0 0x55e48e63a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:01:59.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.908+0000 7f93110c8080 1 -- 172.21.15.32:0/1639993208 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55e48e63b890 con 0x55e48e63c7c0 2024-04-17T10:01:59.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.912+0000 7f930d800640 1 --2- 172.21.15.32:0/1639993208 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55e48e63c7c0 0x55e48e63a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:01:59.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.912+0000 7f930e200640 1 --2- 172.21.15.32:0/1639993208 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f92f0066d60 0x7f92f0069220 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f93000010f0 tx=0x7f9300010040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:59.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.912+0000 7f930d800640 1 --2- 172.21.15.32:0/1639993208 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55e48e63c7c0 0x55e48e63a6c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:01:59.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.912+0000 7f930d800640 1 -- 172.21.15.32:0/1639993208 <== osd.2 v2:172.21.15.32:6809/1954833462 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7f92fc0b2040 con 0x55e48e63c7c0 2024-04-17T10:01:59.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.912+0000 7f93110c8080 1 -- 172.21.15.32:0/1639993208 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55e48e63c7c0 msgr2=0x55e48e63a6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:59.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.912+0000 7f93110c8080 1 --2- 172.21.15.32:0/1639993208 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55e48e63c7c0 0x55e48e63a6c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:01:59.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.912+0000 7f93110c8080 1 -- 172.21.15.32:0/1639993208 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f92f0066d60 msgr2=0x7f92f0069220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:59.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.912+0000 7f93110c8080 1 --2- 172.21.15.32:0/1639993208 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f92f0066d60 0x7f92f0069220 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f93000010f0 tx=0x7f9300010040 comp rx=0 tx=0).stop 2024-04-17T10:01:59.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.912+0000 7f93110c8080 1 -- 172.21.15.32:0/1639993208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48e6cc270 msgr2=0x55e48e63c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:01:59.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.912+0000 7f93110c8080 1 --2- 172.21.15.32:0/1639993208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e48e6cc270 0x55e48e63c280 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f930800d470 tx=0x7f930800d930 comp rx=0 tx=0).stop 2024-04-17T10:01:59.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.912+0000 7f930ec00640 1 -- 172.21.15.32:0/1639993208 reap_dead start 2024-04-17T10:01:59.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.912+0000 7f93110c8080 1 -- 172.21.15.32:0/1639993208 shutdown_connections 2024-04-17T10:01:59.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.912+0000 7f93110c8080 1 -- 172.21.15.32:0/1639993208 >> 172.21.15.32:0/1639993208 conn(0x55e48e5f5620 msgr2=0x55e48e637fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:01:59.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.912+0000 7f93110c8080 1 -- 172.21.15.32:0/1639993208 shutdown_connections 2024-04-17T10:01:59.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:01:59.912+0000 7f93110c8080 1 -- 172.21.15.32:0/1639993208 wait complete. 2024-04-17T10:01:59.914 INFO:teuthology.orchestra.run.smithi032.stdout:����� 2024-04-17T10:01:59.914 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T10:02:00.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.096+0000 7fbb11c00640 1 Processor -- start 2024-04-17T10:02:00.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.096+0000 7fbb11c00640 1 -- start start 2024-04-17T10:02:00.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.100+0000 7fbb11c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbb0c0b2430 0x7fbb0c0b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:00.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.100+0000 7fbb11c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbb0c0cdb10 0x7fbb0c0cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:00.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.100+0000 7fbb11c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb0c0b2d50 con 0x7fbb0c0d0890 2024-04-17T10:02:00.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.100+0000 7fbb11c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fbb0c162620 con 0x7fbb0c0b2430 2024-04-17T10:02:00.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.100+0000 7fbb11c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fbb0c162900 con 0x7fbb0c0cdb10 2024-04-17T10:02:00.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.100+0000 7fbb0aa00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbb0c0b2430 0x7fbb0c0b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:00.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.100+0000 7fbb02000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbb0c0d0890 0x7fbb0c0d0d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33698/0 (socket says 172.21.15.32:33698) 2024-04-17T10:02:00.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.100+0000 7fbb0aa00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbb0c0b2430 0x7fbb0c0b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33244/0 (socket says 172.21.15.32:33244) 2024-04-17T10:02:00.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.100+0000 7fbb02000640 1 -- 172.21.15.32:0/2110648827 learned_addr learned my addr 172.21.15.32:0/2110648827 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:02:00.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.100+0000 7fbb0b400640 1 --2- 172.21.15.32:0/2110648827 >> v2:172.21.15.144:3300/0 conn(0x7fbb0c0cdb10 0x7fbb0c0cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:00.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.100+0000 7fbb0aa00640 1 -- 172.21.15.32:0/2110648827 >> v2:172.21.15.144:3300/0 conn(0x7fbb0c0cdb10 msgr2=0x7fbb0c0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:00.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.100+0000 7fbb0aa00640 1 --2- 172.21.15.32:0/2110648827 >> v2:172.21.15.144:3300/0 conn(0x7fbb0c0cdb10 0x7fbb0c0cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:00.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.100+0000 7fbb0aa00640 1 -- 172.21.15.32:0/2110648827 >> v1:172.21.15.32:6789/0 conn(0x7fbb0c0d0890 legacy=0x7fbb0c0d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:00.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.100+0000 7fbb0aa00640 1 -- 172.21.15.32:0/2110648827 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0c162be0 con 0x7fbb0c0b2430 2024-04-17T10:02:00.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.104+0000 7fbb0aa00640 1 --2- 172.21.15.32:0/2110648827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb0c0b2430 0x7fbb0c0b2810 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c06ed70 tx=0x7fbafc01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b699b2bf6f10b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:00.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.104+0000 7fbb0a000640 1 -- 172.21.15.32:0/2110648827 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc0b0070 con 0x7fbb0c0b2430 2024-04-17T10:02:00.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.104+0000 7fbb0a000640 1 -- 172.21.15.32:0/2110648827 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbafc0ab600 con 0x7fbb0c0b2430 2024-04-17T10:02:00.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.104+0000 7fbb0a000640 1 -- 172.21.15.32:0/2110648827 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc0ab9d0 con 0x7fbb0c0b2430 2024-04-17T10:02:00.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.104+0000 7fbb11c00640 1 -- 172.21.15.32:0/2110648827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb0c0b2430 msgr2=0x7fbb0c0b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:00.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.104+0000 7fbb11c00640 1 --2- 172.21.15.32:0/2110648827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb0c0b2430 0x7fbb0c0b2810 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c06ed70 tx=0x7fbafc01c360 comp rx=0 tx=0).stop 2024-04-17T10:02:00.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.104+0000 7fbb11c00640 1 -- 172.21.15.32:0/2110648827 shutdown_connections 2024-04-17T10:02:00.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.104+0000 7fbb11c00640 1 --2- 172.21.15.32:0/2110648827 >> v2:172.21.15.144:3300/0 conn(0x7fbb0c0cdb10 0x7fbb0c0cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:00.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.104+0000 7fbb11c00640 1 --2- 172.21.15.32:0/2110648827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb0c0b2430 0x7fbb0c0b2810 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:00.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.104+0000 7fbb11c00640 1 -- 172.21.15.32:0/2110648827 >> 172.21.15.32:0/2110648827 conn(0x7fbb0c09a9f0 msgr2=0x7fbb0c0b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:00.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.104+0000 7fbb11c00640 1 -- 172.21.15.32:0/2110648827 shutdown_connections 2024-04-17T10:02:00.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.104+0000 7fbb11c00640 1 -- 172.21.15.32:0/2110648827 wait complete. 2024-04-17T10:02:00.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.104+0000 7fbb11c00640 1 Processor -- start 2024-04-17T10:02:00.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.104+0000 7fbb11c00640 1 -- start start 2024-04-17T10:02:00.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb11c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb0c0b2430 0x7fbb0c15b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:00.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb11c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb0c0cdb10 0x7fbb0c14d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:00.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb11c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb0c0d0890 0x7fbb0c06ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:00.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb11c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c16bc20 con 0x7fbb0c0b2430 2024-04-17T10:02:00.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb11c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c16baa0 con 0x7fbb0c0cdb10 2024-04-17T10:02:00.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb11c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbb0c16bda0 con 0x7fbb0c0d0890 2024-04-17T10:02:00.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb0aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb0c0b2430 0x7fbb0c15b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:00.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb0aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb0c0b2430 0x7fbb0c15b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33246/0 (socket says 172.21.15.32:33246) 2024-04-17T10:02:00.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb0aa00640 1 -- 172.21.15.32:0/3552464956 learned_addr learned my addr 172.21.15.32:0/3552464956 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:00.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb02000640 1 --2- 172.21.15.32:0/3552464956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb0c0cdb10 0x7fbb0c14d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:00.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb0b400640 1 --2- 172.21.15.32:0/3552464956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb0c0d0890 0x7fbb0c06ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:00.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb0aa00640 1 -- 172.21.15.32:0/3552464956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb0c0d0890 msgr2=0x7fbb0c06ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:00.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb0aa00640 1 --2- 172.21.15.32:0/3552464956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb0c0d0890 0x7fbb0c06ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:00.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb0aa00640 1 -- 172.21.15.32:0/3552464956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb0c0cdb10 msgr2=0x7fbb0c14d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:00.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb0aa00640 1 --2- 172.21.15.32:0/3552464956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb0c0cdb10 0x7fbb0c14d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:00.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb0aa00640 1 -- 172.21.15.32:0/3552464956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0c14de60 con 0x7fbb0c0b2430 2024-04-17T10:02:00.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb0aa00640 1 --2- 172.21.15.32:0/3552464956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb0c0b2430 0x7fbb0c15b860 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0ae040 tx=0x7fbafc019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:00.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb03400640 1 -- 172.21.15.32:0/3552464956 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc0b8020 con 0x7fbb0c0b2430 2024-04-17T10:02:00.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb03400640 1 -- 172.21.15.32:0/3552464956 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbafc0b4410 con 0x7fbb0c0b2430 2024-04-17T10:02:00.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb11c00640 1 -- 172.21.15.32:0/3552464956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0c15be40 con 0x7fbb0c0b2430 2024-04-17T10:02:00.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb03400640 1 -- 172.21.15.32:0/3552464956 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc0b0040 con 0x7fbb0c0b2430 2024-04-17T10:02:00.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb11c00640 1 -- 172.21.15.32:0/3552464956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb0c15c400 con 0x7fbb0c0b2430 2024-04-17T10:02:00.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb11c00640 1 -- 172.21.15.32:0/3552464956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb0c068a90 con 0x7fbb0c0b2430 2024-04-17T10:02:00.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.108+0000 7fbb03400640 1 -- 172.21.15.32:0/3552464956 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbafc0bb020 con 0x7fbb0c0b2430 2024-04-17T10:02:00.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.112+0000 7fbb03400640 1 --2- 172.21.15.32:0/3552464956 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbae8068380 0x7fbae806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:00.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.112+0000 7fbb03400640 1 -- 172.21.15.32:0/3552464956 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fbafc1204d0 con 0x7fbb0c0b2430 2024-04-17T10:02:00.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.112+0000 7fbb02000640 1 --2- 172.21.15.32:0/3552464956 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbae8068380 0x7fbae806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:00.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.112+0000 7fbb02000640 1 --2- 172.21.15.32:0/3552464956 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbae8068380 0x7fbae806a840 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c160fe0 tx=0x7fbaf4007e90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:00.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.116+0000 7fbb03400640 1 -- 172.21.15.32:0/3552464956 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fbafc0c2030 con 0x7fbb0c0b2430 2024-04-17T10:02:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.320+0000 7fbb11c00640 1 -- 172.21.15.32:0/3552464956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbb0c09af00 con 0x7fbb0c0b2430 2024-04-17T10:02:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.320+0000 7fbb03400640 1 -- 172.21.15.32:0/3552464956 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v81) ==== 53+0+73 (secure 0 0 0) 0x7fbafc016070 con 0x7fbb0c0b2430 2024-04-17T10:02:00.318 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T10:02:00.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.320+0000 7fbb11c00640 1 -- 172.21.15.32:0/3552464956 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbae8068380 msgr2=0x7fbae806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:00.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.320+0000 7fbb11c00640 1 --2- 172.21.15.32:0/3552464956 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbae8068380 0x7fbae806a840 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c160fe0 tx=0x7fbaf4007e90 comp rx=0 tx=0).stop 2024-04-17T10:02:00.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.320+0000 7fbb11c00640 1 -- 172.21.15.32:0/3552464956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb0c0b2430 msgr2=0x7fbb0c15b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:00.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.320+0000 7fbb11c00640 1 --2- 172.21.15.32:0/3552464956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb0c0b2430 0x7fbb0c15b860 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0ae040 tx=0x7fbafc019100 comp rx=0 tx=0).stop 2024-04-17T10:02:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.320+0000 7fbb11c00640 1 -- 172.21.15.32:0/3552464956 shutdown_connections 2024-04-17T10:02:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.320+0000 7fbb11c00640 1 --2- 172.21.15.32:0/3552464956 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbae8068380 0x7fbae806a840 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.320+0000 7fbb11c00640 1 --2- 172.21.15.32:0/3552464956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb0c0d0890 0x7fbb0c06ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.320+0000 7fbb11c00640 1 --2- 172.21.15.32:0/3552464956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb0c0cdb10 0x7fbb0c14d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.320+0000 7fbb11c00640 1 --2- 172.21.15.32:0/3552464956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb0c0b2430 0x7fbb0c15b860 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.320+0000 7fbb11c00640 1 -- 172.21.15.32:0/3552464956 >> 172.21.15.32:0/3552464956 conn(0x7fbb0c09a9f0 msgr2=0x7fbb0c0d0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.324+0000 7fbb11c00640 1 -- 172.21.15.32:0/3552464956 shutdown_connections 2024-04-17T10:02:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00.324+0000 7fbb11c00640 1 -- 172.21.15.32:0/3552464956 wait complete. 2024-04-17T10:02:00.340 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T10:02:00.340 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T10:02:00.340 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-17T10:02:00.340 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T10:02:00.340 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T10:02:00.340 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T10:02:00.340 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T10:02:00.341 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T10:02:00.341 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:02:00.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:02:00.341 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-17T10:02:00.350 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:02:00.350 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:02:00.350 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:02:00.350 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:02:00.350 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-17T10:02:00.350 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:02:00.350 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:02:00.350 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-17T10:02:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-17T10:02:00.351 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T10:02:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:02:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-17T10:02:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-17T10:02:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 41933sec preferred_lft 41933sec 2024-04-17T10:02:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2024-04-17T10:02:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:02:00.351 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:02:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-17T10:02:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-17T10:02:00.351 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:02:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:02:00.351 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-17T10:02:00.352 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T10:02:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:cd:f4:aa:5b brd ff:ff:ff:ff:ff:ff 2024-04-17T10:02:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T10:02:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:02:00.352 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T10:02:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T10:02:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T10:02:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:02:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T10:02:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:02:00.352 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:02:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:02:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T10:02:00.352 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:02:00.353 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:02:00.353 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:02:00.353 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T10:02:00.353 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:02:00.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:02:00.354 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T10:02:00.414 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-17T10:02:00.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:02:00.416 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-17T10:02:00.469 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-17T10:02:00.470 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T10:02:00.470 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-17T10:02:00.470 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T10:02:00.470 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-17T10:02:00.470 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T10:02:00.470 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T10:02:00.470 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T10:02:00.471 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-17T10:02:00.471 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-17T10:02:00.471 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-17T10:02:00.471 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T10:02:00.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T10:02:00.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:02:00.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:02:00.608 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-17T10:02:00.663 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T10:02:00.663 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T10:02:00.664 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:02:00.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:02:00.665 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-17T10:02:00.734 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T10:02:00.734 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:02:00.735 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T10:02:00.790 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:02:00.790 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:02:00.790 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:02:00.840 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T10:02:00.905 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:02:00.959 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T10:02:00.960 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T10:02:00.960 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T10:02:00.960 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T10:02:00.960 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T10:02:00.960 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T10:02:00.960 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T10:02:00.960 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T10:02:00.960 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T10:02:00.960 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T10:02:00.961 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T10:02:00.961 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T10:02:00.961 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:02:00.961 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T10:02:00.961 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T10:02:00.961 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T10:02:00.961 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T10:02:00.961 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T10:02:00.961 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T10:02:00.961 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T10:02:00.962 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:02:00.962 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T10:02:00.962 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T10:02:00.962 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T10:02:00.962 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:02:00.962 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:02:00.962 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:02:00.962 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:02:00.962 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T10:02:00.962 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:02:00.962 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:02:00.963 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:02:00.963 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:02:00.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:02:00.963 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T10:02:01.015 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T10:02:01.015 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T10:02:01.063 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T10:02:01.084 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:02:01.141 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T10:02:01.141 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T10:02:01.142 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T10:02:01.142 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T10:02:01.142 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T10:02:01.142 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T10:02:01.142 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T10:02:01.142 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T10:02:01.142 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T10:02:01.142 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T10:02:01.142 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T10:02:01.142 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T10:02:01.142 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:02:01.142 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T10:02:01.142 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T10:02:01.142 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T10:02:01.143 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T10:02:01.143 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T10:02:01.143 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T10:02:01.143 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T10:02:01.143 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:02:01.143 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T10:02:01.143 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T10:02:01.143 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T10:02:01.143 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:02:01.143 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:02:01.143 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:02:01.143 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:02:01.143 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T10:02:01.143 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:02:01.143 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:02:01.144 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:02:01.144 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:02:01.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:02:01.145 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T10:02:01.159 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2024-04-17T10:02:01.160+0000 7fc8cbf38580 -1 init, newargv = 0x561e070efe90 newargc=15 2024-04-17T10:02:01.159 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[48965]: starting ceph client 2024-04-17T10:02:01.166 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[48965]: starting fuse 2024-04-17T10:02:01.168 INFO:teuthology.orchestra.run.smithi032.stdout:44 2024-04-17T10:02:01.168 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-17T10:02:01.169 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2024-04-17T10:02:01.169 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2024-04-17T10:02:01.169 DEBUG:teuthology.orchestra.run.smithi032:> import re 2024-04-17T10:02:01.169 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:02:01.169 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2024-04-17T10:02:01.169 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:02:01.169 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2024-04-17T10:02:01.169 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T10:02:01.169 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2024-04-17T10:02:01.169 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T10:02:01.169 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:02:01.169 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2024-04-17T10:02:01.170 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2024-04-17T10:02:01.170 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2024-04-17T10:02:01.170 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2024-04-17T10:02:01.170 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:02:01.170 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2024-04-17T10:02:01.170 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T10:02:01.170 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T10:02:01.170 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T10:02:01.170 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2024-04-17T10:02:01.170 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2024-04-17T10:02:01.170 DEBUG:teuthology.orchestra.run.smithi032:> return f 2024-04-17T10:02:01.170 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T10:02:01.170 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:02:01.170 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2024-04-17T10:02:01.170 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:02:01.295 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.48965.asok 2024-04-17T10:02:01.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:02:01.307 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48965.asok 2024-04-17T10:02:01.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:02:01.307 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48965.asok status 2024-04-17T10:02:01.484 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:02:01.484 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2024-04-17T10:02:01.484 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-17T10:02:01.484 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T10:02:01.484 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2024-04-17T10:02:01.484 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2024-04-17T10:02:01.484 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T10:02:01.484 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "48965", 2024-04-17T10:02:01.484 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2024-04-17T10:02:01.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:02:01.485 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2024-04-17T10:02:01.485 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2024-04-17T10:02:01.485 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 8817, 2024-04-17T10:02:01.485 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2024-04-17T10:02:01.485 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2024-04-17T10:02:01.485 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2024-04-17T10:02:01.485 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 8817 2024-04-17T10:02:01.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:02:01.485 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T10:02:01.485 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T10:02:01.485 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:02:01.485 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3112364504 2024-04-17T10:02:01.485 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:02:01.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:02:01.485 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T10:02:01.486 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T10:02:01.486 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:02:01.486 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3112364504 2024-04-17T10:02:01.486 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:02:01.486 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.8817 192.168.0.1:0/3112364504", 2024-04-17T10:02:01.486 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/3112364504", 2024-04-17T10:02:01.486 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2024-04-17T10:02:01.486 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 81, 2024-04-17T10:02:01.486 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 97, 2024-04-17T10:02:01.486 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2024-04-17T10:02:01.486 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2024-04-17T10:02:01.486 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs" 2024-04-17T10:02:01.486 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:02:01.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:02:01.499 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2024-04-17T10:02:01.499 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T10:02:01.508 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2024-04-17T10:02:01.508 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T10:02:01.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:02:01.509 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:02:01.575 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:02:01.575 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:02:01.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:02:01.628 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-17T10:02:01.755 INFO:teuthology.orchestra.run.smithi032.stderr:6+0 records in 2024-04-17T10:02:01.756 INFO:teuthology.orchestra.run.smithi032.stderr:6+0 records out 2024-04-17T10:02:01.756 INFO:teuthology.orchestra.run.smithi032.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0707024 s, 89.0 MB/s 2024-04-17T10:02:01.757 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:02:01.757 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:02:01.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:02:01.765 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-17T10:02:01.901 INFO:teuthology.orchestra.run.smithi032.stderr:6+0 records in 2024-04-17T10:02:01.901 INFO:teuthology.orchestra.run.smithi032.stderr:6+0 records out 2024-04-17T10:02:01.901 INFO:teuthology.orchestra.run.smithi032.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.078056 s, 80.6 MB/s 2024-04-17T10:02:01.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:02:01.902 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-17T10:02:01.915 INFO:teuthology.orchestra.run.smithi032.stdout:./ 2024-04-17T10:02:01.915 INFO:teuthology.orchestra.run.smithi032.stdout:./file2_sixmegs 2024-04-17T10:02:01.915 INFO:teuthology.orchestra.run.smithi032.stdout:./file3_sixmegs 2024-04-17T10:02:01.915 INFO:teuthology.orchestra.run.smithi032.stdout:./file1_sixmegs 2024-04-17T10:02:01.916 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:02:01.916 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:02:01.917 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:02:01.917 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:02:01.917 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-17T10:02:01.917 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:02:02.026 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-17T10:02:02.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:02:02.038 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:02:02.038 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:02:02.038 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:02:02.038 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:02:02.038 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-04-17T10:02:02.038 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:02:02.150 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627777 2024-04-17T10:02:02.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:02:02.159 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:02:02.160 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:02:02.160 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:02:02.160 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:02:02.160 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-04-17T10:02:02.160 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:02:02.270 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627778 2024-04-17T10:02:02.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:02:02.279 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:02:02.279 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:02:02.279 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:02:02.279 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:02:02.279 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-04-17T10:02:02.279 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:02:02.390 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627776 2024-04-17T10:02:02.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:02:02.399 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:02:02.399 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:02:02.452 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:02:02.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.632+0000 7f5bf0e00640 1 Processor -- start 2024-04-17T10:02:02.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.632+0000 7f5bf0e00640 1 -- start start 2024-04-17T10:02:02.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.636+0000 7f5bf0e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bec15baa0 0x7f5bec15bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:02.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.636+0000 7f5bf0e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5bec06ef60 0x7f5bec06f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:02.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.636+0000 7f5bf0e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bec072bb0 con 0x7f5bec144e90 2024-04-17T10:02:02.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.636+0000 7f5bf0e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5bec15c440 con 0x7f5bec06ef60 2024-04-17T10:02:02.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.636+0000 7f5bf0e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5bec06f8f0 con 0x7f5bec15baa0 2024-04-17T10:02:02.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.636+0000 7f5be9e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5bec144e90 0x7f5bec0b48a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60402/0 (socket says 172.21.15.32:60402) 2024-04-17T10:02:02.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.636+0000 7f5be9e00640 1 -- 172.21.15.32:0/1508035734 learned_addr learned my addr 172.21.15.32:0/1508035734 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:02:02.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.636+0000 7f5bea800640 1 --2- 172.21.15.32:0/1508035734 >> v2:172.21.15.32:3300/0 conn(0x7f5bec06ef60 0x7f5bec06f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:02.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.636+0000 7f5be3e00640 1 --2- 172.21.15.32:0/1508035734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bec15baa0 0x7f5bec15bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:02.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.636+0000 7f5be9400640 1 -- 172.21.15.32:0/1508035734 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1406636724 0 0) 0x7f5bec072bb0 con 0x7f5bec144e90 2024-04-17T10:02:02.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.636+0000 7f5bea800640 1 -- 172.21.15.32:0/1508035734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bec15baa0 msgr2=0x7f5bec15bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:02.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.636+0000 7f5bea800640 1 --2- 172.21.15.32:0/1508035734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bec15baa0 0x7f5bec15bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:02.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.636+0000 7f5bea800640 1 -- 172.21.15.32:0/1508035734 >> v1:172.21.15.32:6789/0 conn(0x7f5bec144e90 legacy=0x7f5bec0b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:02.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.636+0000 7f5bea800640 1 -- 172.21.15.32:0/1508035734 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bec0b3dc0 con 0x7f5bec06ef60 2024-04-17T10:02:02.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.640+0000 7f5bea800640 1 --2- 172.21.15.32:0/1508035734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bec06ef60 0x7f5bec06f3b0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f5be4009d90 tx=0x7f5be401eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3af6dff560ad6b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:02.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.640+0000 7f5be9400640 1 -- 172.21.15.32:0/1508035734 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be40b4070 con 0x7f5bec06ef60 2024-04-17T10:02:02.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.640+0000 7f5be9400640 1 -- 172.21.15.32:0/1508035734 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be40a7b40 con 0x7f5bec06ef60 2024-04-17T10:02:02.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.640+0000 7f5be9400640 1 -- 172.21.15.32:0/1508035734 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be40af440 con 0x7f5bec06ef60 2024-04-17T10:02:02.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.640+0000 7f5bf0e00640 1 -- 172.21.15.32:0/1508035734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bec06ef60 msgr2=0x7f5bec06f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:02.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.640+0000 7f5bf0e00640 1 --2- 172.21.15.32:0/1508035734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bec06ef60 0x7f5bec06f3b0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f5be4009d90 tx=0x7f5be401eb80 comp rx=0 tx=0).stop 2024-04-17T10:02:02.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.640+0000 7f5bf0e00640 1 -- 172.21.15.32:0/1508035734 shutdown_connections 2024-04-17T10:02:02.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.640+0000 7f5bf0e00640 1 --2- 172.21.15.32:0/1508035734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bec06ef60 0x7f5bec06f3b0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:02.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.640+0000 7f5bf0e00640 1 --2- 172.21.15.32:0/1508035734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bec15baa0 0x7f5bec15bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:02.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.640+0000 7f5bf0e00640 1 -- 172.21.15.32:0/1508035734 >> 172.21.15.32:0/1508035734 conn(0x7f5bec06ba30 msgr2=0x7f5bec06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:02.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.640+0000 7f5bf0e00640 1 -- 172.21.15.32:0/1508035734 shutdown_connections 2024-04-17T10:02:02.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.640+0000 7f5bf0e00640 1 -- 172.21.15.32:0/1508035734 wait complete. 2024-04-17T10:02:02.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.640+0000 7f5bf0e00640 1 Processor -- start 2024-04-17T10:02:02.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.640+0000 7f5bf0e00640 1 -- start start 2024-04-17T10:02:02.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5bf0e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bec06ef60 0x7f5bec1411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:02.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5bf0e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bec144e90 0x7f5bec0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:02.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5bf0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bec15baa0 0x7f5bec15aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:02.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5bf0e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec0727f0 con 0x7f5bec15baa0 2024-04-17T10:02:02.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5bf0e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec072670 con 0x7f5bec144e90 2024-04-17T10:02:02.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5bf0e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5bec072970 con 0x7f5bec06ef60 2024-04-17T10:02:02.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5bea800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bec15baa0 0x7f5bec15aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:02.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5bea800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bec15baa0 0x7f5bec15aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41720/0 (socket says 172.21.15.32:41720) 2024-04-17T10:02:02.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5bea800640 1 -- 172.21.15.32:0/342486663 learned_addr learned my addr 172.21.15.32:0/342486663 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:02.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5be9e00640 1 --2- 172.21.15.32:0/342486663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bec06ef60 0x7f5bec1411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:02.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5be3e00640 1 --2- 172.21.15.32:0/342486663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bec144e90 0x7f5bec0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:02.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5bea800640 1 -- 172.21.15.32:0/342486663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bec06ef60 msgr2=0x7f5bec1411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:02.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5bea800640 1 --2- 172.21.15.32:0/342486663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bec06ef60 0x7f5bec1411e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:02.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5bea800640 1 -- 172.21.15.32:0/342486663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bec144e90 msgr2=0x7f5bec0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:02.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5bea800640 1 --2- 172.21.15.32:0/342486663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bec144e90 0x7f5bec0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:02.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5bea800640 1 -- 172.21.15.32:0/342486663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bec141720 con 0x7f5bec15baa0 2024-04-17T10:02:02.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5be9e00640 1 --2- 172.21.15.32:0/342486663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bec06ef60 0x7f5bec1411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:02:02.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5be3e00640 1 --2- 172.21.15.32:0/342486663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bec144e90 0x7f5bec0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:02:02.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5bea800640 1 --2- 172.21.15.32:0/342486663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bec15baa0 0x7f5bec15aa40 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f5be40b2040 tx=0x7f5be401b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:02.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5be2000640 1 -- 172.21.15.32:0/342486663 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be40bc020 con 0x7f5bec15baa0 2024-04-17T10:02:02.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5be2000640 1 -- 172.21.15.32:0/342486663 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be400e080 con 0x7f5bec15baa0 2024-04-17T10:02:02.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5be2000640 1 -- 172.21.15.32:0/342486663 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be40b4040 con 0x7f5bec15baa0 2024-04-17T10:02:02.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5bf0e00640 1 -- 172.21.15.32:0/342486663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bec1419c0 con 0x7f5bec15baa0 2024-04-17T10:02:02.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5bf0e00640 1 -- 172.21.15.32:0/342486663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bec141f80 con 0x7f5bec15baa0 2024-04-17T10:02:02.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5bf0e00640 1 -- 172.21.15.32:0/342486663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bec068a90 con 0x7f5bec15baa0 2024-04-17T10:02:02.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.644+0000 7f5be2000640 1 -- 172.21.15.32:0/342486663 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5be40039a0 con 0x7f5bec15baa0 2024-04-17T10:02:02.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.648+0000 7f5be2000640 1 --2- 172.21.15.32:0/342486663 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5bc8068380 0x7f5bc806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:02.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.648+0000 7f5be2000640 1 -- 172.21.15.32:0/342486663 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f5be4120f40 con 0x7f5bec15baa0 2024-04-17T10:02:02.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.648+0000 7f5be9e00640 1 --2- 172.21.15.32:0/342486663 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5bc8068380 0x7f5bc806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:02.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.648+0000 7f5be9e00640 1 --2- 172.21.15.32:0/342486663 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5bc8068380 0x7f5bc806a840 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc007fd0 tx=0x7f5bdc007c20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:02.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.652+0000 7f5be2000640 1 -- 172.21.15.32:0/342486663 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5be40c6060 con 0x7f5bec15baa0 2024-04-17T10:02:02.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.816+0000 7f5bf0e00640 1 -- 172.21.15.32:0/342486663 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5bec15ea20 con 0x7f5bec15baa0 2024-04-17T10:02:02.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.820+0000 7f5be2000640 1 -- 172.21.15.32:0/342486663 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+16620 (secure 0 0 0) 0x7f5be4018770 con 0x7f5bec15baa0 2024-04-17T10:02:02.816 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:02:02.817 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":97,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:01:42.155683+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:01:38.938460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-17T10:01:39.505171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:02:02.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.820+0000 7f5bf0e00640 1 -- 172.21.15.32:0/342486663 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5bc8068380 msgr2=0x7f5bc806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:02.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.820+0000 7f5bf0e00640 1 --2- 172.21.15.32:0/342486663 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5bc8068380 0x7f5bc806a840 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc007fd0 tx=0x7f5bdc007c20 comp rx=0 tx=0).stop 2024-04-17T10:02:02.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.820+0000 7f5bf0e00640 1 -- 172.21.15.32:0/342486663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bec15baa0 msgr2=0x7f5bec15aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:02.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.820+0000 7f5bf0e00640 1 --2- 172.21.15.32:0/342486663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bec15baa0 0x7f5bec15aa40 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f5be40b2040 tx=0x7f5be401b920 comp rx=0 tx=0).stop 2024-04-17T10:02:02.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.820+0000 7f5bf0e00640 1 -- 172.21.15.32:0/342486663 shutdown_connections 2024-04-17T10:02:02.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.820+0000 7f5bf0e00640 1 --2- 172.21.15.32:0/342486663 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5bc8068380 0x7f5bc806a840 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:02.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.820+0000 7f5bf0e00640 1 --2- 172.21.15.32:0/342486663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bec15baa0 0x7f5bec15aa40 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:02.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.820+0000 7f5bf0e00640 1 --2- 172.21.15.32:0/342486663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bec144e90 0x7f5bec0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:02.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.820+0000 7f5bf0e00640 1 --2- 172.21.15.32:0/342486663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bec06ef60 0x7f5bec1411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:02.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.820+0000 7f5bf0e00640 1 -- 172.21.15.32:0/342486663 >> 172.21.15.32:0/342486663 conn(0x7f5bec06ba30 msgr2=0x7f5bec143ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:02.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.820+0000 7f5bf0e00640 1 -- 172.21.15.32:0/342486663 shutdown_connections 2024-04-17T10:02:02.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.820+0000 7f5bf0e00640 1 -- 172.21.15.32:0/342486663 wait complete. 2024-04-17T10:02:02.838 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:01:38.938460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:01:39.505171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:02:02.838 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:02:02.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.984+0000 7f3d62a00640 1 Processor -- start 2024-04-17T10:02:02.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.984+0000 7f3d62a00640 1 -- start start 2024-04-17T10:02:02.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.984+0000 7f3d62a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3d5c06ba30 0x7f3d5c06be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:02.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.984+0000 7f3d62a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d5c06c350 con 0x7f3d5c071e60 2024-04-17T10:02:02.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.984+0000 7f3d62a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d5c0d0130 con 0x7f3d5c0cf370 2024-04-17T10:02:02.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.984+0000 7f3d62a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3d5c0cfef0 con 0x7f3d5c06ba30 2024-04-17T10:02:02.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.984+0000 7f3d5be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3d5c06ba30 0x7f3d5c06be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:02.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.984+0000 7f3d5be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3d5c06ba30 0x7f3d5c06be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41730/0 (socket says 172.21.15.32:41730) 2024-04-17T10:02:02.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.984+0000 7f3d5be00640 1 -- 172.21.15.32:0/2846256613 learned_addr learned my addr 172.21.15.32:0/2846256613 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:02.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.984+0000 7f3d5be00640 1 -- 172.21.15.32:0/2846256613 >> v1:172.21.15.144:6789/0 conn(0x7f3d5c0cf370 legacy=0x7f3d5c0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:02.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.984+0000 7f3d5be00640 1 -- 172.21.15.32:0/2846256613 >> v1:172.21.15.32:6789/0 conn(0x7f3d5c071e60 legacy=0x7f3d5c072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:02.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.984+0000 7f3d5be00640 1 -- 172.21.15.32:0/2846256613 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5c15d090 con 0x7f3d5c06ba30 2024-04-17T10:02:02.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.984+0000 7f3d5b400640 1 -- 172.21.15.32:0/2846256613 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1907930714 0 0) 0x7f3d5c06c350 con 0x7f3d5c071e60 2024-04-17T10:02:02.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.988+0000 7f3d5be00640 1 --2- 172.21.15.32:0/2846256613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d5c06ba30 0x7f3d5c06be10 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009870 tx=0x7f3d4c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd8b8aca325e8da7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:02.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.988+0000 7f3d5b400640 1 -- 172.21.15.32:0/2846256613 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0aa600 con 0x7f3d5c06ba30 2024-04-17T10:02:02.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.988+0000 7f3d5b400640 1 -- 172.21.15.32:0/2846256613 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d4c0af070 con 0x7f3d5c06ba30 2024-04-17T10:02:02.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.988+0000 7f3d5b400640 1 -- 172.21.15.32:0/2846256613 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0aa9c0 con 0x7f3d5c06ba30 2024-04-17T10:02:02.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.988+0000 7f3d62a00640 1 -- 172.21.15.32:0/2846256613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d5c06ba30 msgr2=0x7f3d5c06be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:02.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.988+0000 7f3d62a00640 1 --2- 172.21.15.32:0/2846256613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d5c06ba30 0x7f3d5c06be10 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009870 tx=0x7f3d4c01c4a0 comp rx=0 tx=0).stop 2024-04-17T10:02:02.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.988+0000 7f3d62a00640 1 -- 172.21.15.32:0/2846256613 shutdown_connections 2024-04-17T10:02:02.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.988+0000 7f3d62a00640 1 --2- 172.21.15.32:0/2846256613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d5c06ba30 0x7f3d5c06be10 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:02.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.988+0000 7f3d62a00640 1 -- 172.21.15.32:0/2846256613 >> 172.21.15.32:0/2846256613 conn(0x7f3d5c0b2120 msgr2=0x7f3d5c0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:02.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.988+0000 7f3d62a00640 1 -- 172.21.15.32:0/2846256613 shutdown_connections 2024-04-17T10:02:02.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.988+0000 7f3d62a00640 1 -- 172.21.15.32:0/2846256613 wait complete. 2024-04-17T10:02:02.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d62a00640 1 Processor -- start 2024-04-17T10:02:02.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d62a00640 1 -- start start 2024-04-17T10:02:02.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d62a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d5c06ba30 0x7f3d5c0b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:02.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d62a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d5c071e60 0x7f3d5c141370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:02.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d62a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d5c0cf370 0x7f3d5c159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:02.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d62a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c15db70 con 0x7f3d5c0cf370 2024-04-17T10:02:02.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d62a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c15cd00 con 0x7f3d5c06ba30 2024-04-17T10:02:02.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d62a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3d5c15dcf0 con 0x7f3d5c071e60 2024-04-17T10:02:02.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d60a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d5c0cf370 0x7f3d5c159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:02.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d60a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d5c0cf370 0x7f3d5c159b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41746/0 (socket says 172.21.15.32:41746) 2024-04-17T10:02:02.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d60a00640 1 -- 172.21.15.32:0/1297935584 learned_addr learned my addr 172.21.15.32:0/1297935584 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:02.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d53400640 1 --2- 172.21.15.32:0/1297935584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d5c071e60 0x7f3d5c141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:02.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d5be00640 1 --2- 172.21.15.32:0/1297935584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d5c06ba30 0x7f3d5c0b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:02.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d60a00640 1 -- 172.21.15.32:0/1297935584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d5c071e60 msgr2=0x7f3d5c141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:02.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d60a00640 1 --2- 172.21.15.32:0/1297935584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d5c071e60 0x7f3d5c141370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:02.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d60a00640 1 -- 172.21.15.32:0/1297935584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d5c06ba30 msgr2=0x7f3d5c0b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:02.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d60a00640 1 --2- 172.21.15.32:0/1297935584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d5c06ba30 0x7f3d5c0b3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:02.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d60a00640 1 -- 172.21.15.32:0/1297935584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5c13c620 con 0x7f3d5c0cf370 2024-04-17T10:02:02.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d60a00640 1 --2- 172.21.15.32:0/1297935584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d5c0cf370 0x7f3d5c159b50 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400c780 tx=0x7f3d5400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:02.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d58c00640 1 -- 172.21.15.32:0/1297935584 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d54003db0 con 0x7f3d5c0cf370 2024-04-17T10:02:02.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d58c00640 1 -- 172.21.15.32:0/1297935584 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d54005070 con 0x7f3d5c0cf370 2024-04-17T10:02:02.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d62a00640 1 -- 172.21.15.32:0/1297935584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d5c13c900 con 0x7f3d5c0cf370 2024-04-17T10:02:02.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d58c00640 1 -- 172.21.15.32:0/1297935584 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d54005460 con 0x7f3d5c0cf370 2024-04-17T10:02:02.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.992+0000 7f3d62a00640 1 -- 172.21.15.32:0/1297935584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d5c13cec0 con 0x7f3d5c0cf370 2024-04-17T10:02:02.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.996+0000 7f3d62a00640 1 -- 172.21.15.32:0/1297935584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d5c068a90 con 0x7f3d5c0cf370 2024-04-17T10:02:02.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.996+0000 7f3d58c00640 1 -- 172.21.15.32:0/1297935584 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3d54005600 con 0x7f3d5c0cf370 2024-04-17T10:02:02.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.996+0000 7f3d58c00640 1 --2- 172.21.15.32:0/1297935584 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3d38068380 0x7f3d3806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:02.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.996+0000 7f3d58c00640 1 -- 172.21.15.32:0/1297935584 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f3d54086960 con 0x7f3d5c0cf370 2024-04-17T10:02:02.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.996+0000 7f3d5be00640 1 --2- 172.21.15.32:0/1297935584 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3d38068380 0x7f3d3806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:02.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:02.996+0000 7f3d5be00640 1 --2- 172.21.15.32:0/1297935584 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3d38068380 0x7f3d3806a840 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0ae000 tx=0x7f3d4c0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:02.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.000+0000 7f3d58c00640 1 -- 172.21.15.32:0/1297935584 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3d54056e80 con 0x7f3d5c0cf370 2024-04-17T10:02:03.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.164+0000 7f3d62a00640 1 -- 172.21.15.32:0/1297935584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3d5c06c850 con 0x7f3d5c0cf370 2024-04-17T10:02:03.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.164+0000 7f3d58c00640 1 -- 172.21.15.32:0/1297935584 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f3d54056780 con 0x7f3d5c0cf370 2024-04-17T10:02:03.164 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T10:02:03.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.168+0000 7f3d62a00640 1 -- 172.21.15.32:0/1297935584 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3d38068380 msgr2=0x7f3d3806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:03.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.168+0000 7f3d62a00640 1 --2- 172.21.15.32:0/1297935584 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3d38068380 0x7f3d3806a840 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0ae000 tx=0x7f3d4c0a7000 comp rx=0 tx=0).stop 2024-04-17T10:02:03.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.168+0000 7f3d62a00640 1 -- 172.21.15.32:0/1297935584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d5c0cf370 msgr2=0x7f3d5c159b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:03.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.168+0000 7f3d62a00640 1 --2- 172.21.15.32:0/1297935584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d5c0cf370 0x7f3d5c159b50 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400c780 tx=0x7f3d5400cc40 comp rx=0 tx=0).stop 2024-04-17T10:02:03.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.168+0000 7f3d62a00640 1 -- 172.21.15.32:0/1297935584 shutdown_connections 2024-04-17T10:02:03.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.168+0000 7f3d62a00640 1 --2- 172.21.15.32:0/1297935584 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3d38068380 0x7f3d3806a840 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.168+0000 7f3d62a00640 1 --2- 172.21.15.32:0/1297935584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d5c071e60 0x7f3d5c141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.168+0000 7f3d62a00640 1 --2- 172.21.15.32:0/1297935584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d5c0cf370 0x7f3d5c159b50 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.168+0000 7f3d62a00640 1 --2- 172.21.15.32:0/1297935584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d5c06ba30 0x7f3d5c0b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.168+0000 7f3d62a00640 1 -- 172.21.15.32:0/1297935584 >> 172.21.15.32:0/1297935584 conn(0x7f3d5c0b2120 msgr2=0x7f3d5c15c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:03.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.168+0000 7f3d62a00640 1 -- 172.21.15.32:0/1297935584 shutdown_connections 2024-04-17T10:02:03.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.168+0000 7f3d62a00640 1 -- 172.21.15.32:0/1297935584 wait complete. 2024-04-17T10:02:03.186 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:02:03.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.332+0000 7fbf03600640 1 Processor -- start 2024-04-17T10:02:03.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.332+0000 7fbf03600640 1 -- start start 2024-04-17T10:02:03.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.332+0000 7fbf03600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbefc15b7d0 0x7fbefc06e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:03.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.332+0000 7fbf03600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbefc06c140 0x7fbefc0ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:03.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.332+0000 7fbf03600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbefc0cc940 con 0x7fbefc06f7c0 2024-04-17T10:02:03.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.332+0000 7fbf03600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fbefc0ccc20 con 0x7fbefc15b7d0 2024-04-17T10:02:03.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.332+0000 7fbf03600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbefc06e0b0 con 0x7fbefc06c140 2024-04-17T10:02:03.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.332+0000 7fbf01600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbefc06c140 0x7fbefc0ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:03.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.332+0000 7fbf00c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbefc15b7d0 0x7fbefc06e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:03.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.332+0000 7fbf00c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbefc15b7d0 0x7fbefc06e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:35770/0 (socket says 172.21.15.32:35770) 2024-04-17T10:02:03.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.332+0000 7fbf01600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbefc06c140 0x7fbefc0ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:43158/0 (socket says 172.21.15.32:43158) 2024-04-17T10:02:03.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.332+0000 7fbf00c00640 1 -- 172.21.15.32:0/972411983 learned_addr learned my addr 172.21.15.32:0/972411983 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:03.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.332+0000 7fbf00c00640 1 -- 172.21.15.32:0/972411983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbefc06c140 msgr2=0x7fbefc0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:03.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.332+0000 7fbf00c00640 1 --2- 172.21.15.32:0/972411983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbefc06c140 0x7fbefc0ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.332+0000 7fbf00c00640 1 -- 172.21.15.32:0/972411983 >> v1:172.21.15.144:6789/0 conn(0x7fbefc06f7c0 legacy=0x7fbefc06ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:03.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.332+0000 7fbf00c00640 1 -- 172.21.15.32:0/972411983 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbefc06e390 con 0x7fbefc15b7d0 2024-04-17T10:02:03.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.336+0000 7fbf00c00640 1 --2- 172.21.15.32:0/972411983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbefc15b7d0 0x7fbefc06e8b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbeec009870 tx=0x7fbeec01c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdd86c61a0084a41 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:03.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.336+0000 7fbefbe00640 1 -- 172.21.15.32:0/972411983 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec0aa650 con 0x7fbefc15b7d0 2024-04-17T10:02:03.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.336+0000 7fbefbe00640 1 -- 172.21.15.32:0/972411983 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeec0af070 con 0x7fbefc15b7d0 2024-04-17T10:02:03.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.336+0000 7fbefbe00640 1 -- 172.21.15.32:0/972411983 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec0aaa10 con 0x7fbefc15b7d0 2024-04-17T10:02:03.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.336+0000 7fbf03600640 1 -- 172.21.15.32:0/972411983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbefc15b7d0 msgr2=0x7fbefc06e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:03.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.336+0000 7fbf03600640 1 --2- 172.21.15.32:0/972411983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbefc15b7d0 0x7fbefc06e8b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbeec009870 tx=0x7fbeec01c4d0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.336+0000 7fbf03600640 1 -- 172.21.15.32:0/972411983 shutdown_connections 2024-04-17T10:02:03.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.336+0000 7fbf03600640 1 --2- 172.21.15.32:0/972411983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbefc06c140 0x7fbefc0ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.336+0000 7fbf03600640 1 --2- 172.21.15.32:0/972411983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbefc15b7d0 0x7fbefc06e8b0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.336+0000 7fbf03600640 1 -- 172.21.15.32:0/972411983 >> 172.21.15.32:0/972411983 conn(0x7fbefc0b2120 msgr2=0x7fbefc0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:03.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.336+0000 7fbf03600640 1 -- 172.21.15.32:0/972411983 shutdown_connections 2024-04-17T10:02:03.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.336+0000 7fbf03600640 1 -- 172.21.15.32:0/972411983 wait complete. 2024-04-17T10:02:03.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbf03600640 1 Processor -- start 2024-04-17T10:02:03.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbf03600640 1 -- start start 2024-04-17T10:02:03.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbf03600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbefc06c140 0x7fbefc15e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:03.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbf03600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbefc06f7c0 0x7fbefc13b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:03.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbf03600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbefc15b7d0 0x7fbefc1384f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:03.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbf03600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc071fe0 con 0x7fbefc15b7d0 2024-04-17T10:02:03.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbf03600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc071e60 con 0x7fbefc06f7c0 2024-04-17T10:02:03.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbf03600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbefc072160 con 0x7fbefc06c140 2024-04-17T10:02:03.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbf01600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbefc15b7d0 0x7fbefc1384f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:03.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbf01600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbefc15b7d0 0x7fbefc1384f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41756/0 (socket says 172.21.15.32:41756) 2024-04-17T10:02:03.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbf01600640 1 -- 172.21.15.32:0/1928777440 learned_addr learned my addr 172.21.15.32:0/1928777440 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:03.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbf00c00640 1 --2- 172.21.15.32:0/1928777440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbefc06c140 0x7fbefc15e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:03.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbef3e00640 1 --2- 172.21.15.32:0/1928777440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbefc06f7c0 0x7fbefc13b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:03.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbf01600640 1 -- 172.21.15.32:0/1928777440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbefc06c140 msgr2=0x7fbefc15e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:03.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbf01600640 1 --2- 172.21.15.32:0/1928777440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbefc06c140 0x7fbefc15e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbf01600640 1 -- 172.21.15.32:0/1928777440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbefc06f7c0 msgr2=0x7fbefc13b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:03.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbf01600640 1 --2- 172.21.15.32:0/1928777440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbefc06f7c0 0x7fbefc13b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbf01600640 1 -- 172.21.15.32:0/1928777440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbefc13bb00 con 0x7fbefc15b7d0 2024-04-17T10:02:03.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbf01600640 1 --2- 172.21.15.32:0/1928777440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbefc15b7d0 0x7fbefc1384f0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009e30 tx=0x7fbef400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:03.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbef9600640 1 -- 172.21.15.32:0/1928777440 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4018020 con 0x7fbefc15b7d0 2024-04-17T10:02:03.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbef9600640 1 -- 172.21.15.32:0/1928777440 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbef4014480 con 0x7fbefc15b7d0 2024-04-17T10:02:03.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbef9600640 1 -- 172.21.15.32:0/1928777440 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4014870 con 0x7fbefc15b7d0 2024-04-17T10:02:03.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbf03600640 1 -- 172.21.15.32:0/1928777440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbefc13bde0 con 0x7fbefc15b7d0 2024-04-17T10:02:03.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.340+0000 7fbf03600640 1 -- 172.21.15.32:0/1928777440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbefc15c540 con 0x7fbefc15b7d0 2024-04-17T10:02:03.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.344+0000 7fbf03600640 1 -- 172.21.15.32:0/1928777440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbefc068a90 con 0x7fbefc15b7d0 2024-04-17T10:02:03.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.344+0000 7fbef9600640 1 -- 172.21.15.32:0/1928777440 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbef401b020 con 0x7fbefc15b7d0 2024-04-17T10:02:03.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.344+0000 7fbef9600640 1 --2- 172.21.15.32:0/1928777440 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbed8068380 0x7fbed806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:03.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.344+0000 7fbef9600640 1 -- 172.21.15.32:0/1928777440 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fbef40877d0 con 0x7fbefc15b7d0 2024-04-17T10:02:03.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.344+0000 7fbf00c00640 1 --2- 172.21.15.32:0/1928777440 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbed8068380 0x7fbed806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:03.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.344+0000 7fbf00c00640 1 --2- 172.21.15.32:0/1928777440 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbed8068380 0x7fbed806a840 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fbeec0ae000 tx=0x7fbeec0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:03.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.348+0000 7fbef9600640 1 -- 172.21.15.32:0/1928777440 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fbef4022020 con 0x7fbefc15b7d0 2024-04-17T10:02:03.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.516+0000 7fbf03600640 1 -- 172.21.15.32:0/1928777440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbefc15da60 con 0x7fbefc15b7d0 2024-04-17T10:02:03.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.516+0000 7fbef9600640 1 -- 172.21.15.32:0/1928777440 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7fbef4010040 con 0x7fbefc15b7d0 2024-04-17T10:02:03.513 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T10:02:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.516+0000 7fbf03600640 1 -- 172.21.15.32:0/1928777440 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbed8068380 msgr2=0x7fbed806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.516+0000 7fbf03600640 1 --2- 172.21.15.32:0/1928777440 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbed8068380 0x7fbed806a840 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fbeec0ae000 tx=0x7fbeec0a7000 comp rx=0 tx=0).stop 2024-04-17T10:02:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.516+0000 7fbf03600640 1 -- 172.21.15.32:0/1928777440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbefc15b7d0 msgr2=0x7fbefc1384f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.516+0000 7fbf03600640 1 --2- 172.21.15.32:0/1928777440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbefc15b7d0 0x7fbefc1384f0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009e30 tx=0x7fbef400c490 comp rx=0 tx=0).stop 2024-04-17T10:02:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.516+0000 7fbf03600640 1 -- 172.21.15.32:0/1928777440 shutdown_connections 2024-04-17T10:02:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.516+0000 7fbf03600640 1 --2- 172.21.15.32:0/1928777440 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbed8068380 0x7fbed806a840 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.516+0000 7fbf03600640 1 --2- 172.21.15.32:0/1928777440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbefc15b7d0 0x7fbefc1384f0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.516+0000 7fbf03600640 1 --2- 172.21.15.32:0/1928777440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbefc06f7c0 0x7fbefc13b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.516+0000 7fbf03600640 1 --2- 172.21.15.32:0/1928777440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbefc06c140 0x7fbefc15e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.516+0000 7fbf03600640 1 -- 172.21.15.32:0/1928777440 >> 172.21.15.32:0/1928777440 conn(0x7fbefc0b2120 msgr2=0x7fbefc0b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.516+0000 7fbf03600640 1 -- 172.21.15.32:0/1928777440 shutdown_connections 2024-04-17T10:02:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.516+0000 7fbf03600640 1 -- 172.21.15.32:0/1928777440 wait complete. 2024-04-17T10:02:03.534 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:02:03.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.680+0000 7f0861800640 1 Processor -- start 2024-04-17T10:02:03.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.680+0000 7f0861800640 1 -- start start 2024-04-17T10:02:03.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.680+0000 7f0861800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f085c15b7d0 0x7f085c06e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:03.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.680+0000 7f0861800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f085c06c140 0x7f085c0ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:03.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.680+0000 7f0861800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f085c0cc940 con 0x7f085c06f7c0 2024-04-17T10:02:03.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.680+0000 7f0861800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f085c0ccc20 con 0x7f085c06c140 2024-04-17T10:02:03.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.680+0000 7f0861800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f085c06e0b0 con 0x7f085c15b7d0 2024-04-17T10:02:03.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.680+0000 7f0852000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f085c06f7c0 0x7f085c06ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60416/0 (socket says 172.21.15.32:60416) 2024-04-17T10:02:03.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.680+0000 7f0852000640 1 -- 172.21.15.32:0/3784360815 learned_addr learned my addr 172.21.15.32:0/3784360815 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:02:03.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.680+0000 7f085aa00640 1 --2- 172.21.15.32:0/3784360815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f085c15b7d0 0x7f085c06e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:03.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.680+0000 7f085b400640 1 --2- 172.21.15.32:0/3784360815 >> v2:172.21.15.144:3300/0 conn(0x7f085c06c140 0x7f085c0ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:03.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.680+0000 7f085a000640 1 -- 172.21.15.32:0/3784360815 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3051845724 0 0) 0x7f085c0cc940 con 0x7f085c06f7c0 2024-04-17T10:02:03.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.680+0000 7f085a000640 1 -- 172.21.15.32:0/3784360815 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0840003530 con 0x7f085c06f7c0 2024-04-17T10:02:03.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.680+0000 7f085b400640 1 -- 172.21.15.32:0/3784360815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f085c15b7d0 msgr2=0x7f085c06e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:03.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.680+0000 7f085b400640 1 --2- 172.21.15.32:0/3784360815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f085c15b7d0 0x7f085c06e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.680+0000 7f085b400640 1 -- 172.21.15.32:0/3784360815 >> v1:172.21.15.32:6789/0 conn(0x7f085c06f7c0 legacy=0x7f085c06ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:03.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.680+0000 7f085b400640 1 -- 172.21.15.32:0/3784360815 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085c06e390 con 0x7f085c06c140 2024-04-17T10:02:03.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.684+0000 7f085b400640 1 --2- 172.21.15.32:0/3784360815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f085c06c140 0x7f085c0ce290 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0854009d90 tx=0x7f085401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ef014498bc985d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:03.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.684+0000 7f085a000640 1 -- 172.21.15.32:0/3784360815 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08540b2070 con 0x7f085c06c140 2024-04-17T10:02:03.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.684+0000 7f085a000640 1 -- 172.21.15.32:0/3784360815 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08540a59c0 con 0x7f085c06c140 2024-04-17T10:02:03.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.684+0000 7f085a000640 1 -- 172.21.15.32:0/3784360815 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08540a5d70 con 0x7f085c06c140 2024-04-17T10:02:03.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.684+0000 7f0861800640 1 -- 172.21.15.32:0/3784360815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f085c06c140 msgr2=0x7f085c0ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:03.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.684+0000 7f0861800640 1 --2- 172.21.15.32:0/3784360815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f085c06c140 0x7f085c0ce290 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0854009d90 tx=0x7f085401c8e0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.684+0000 7f0861800640 1 -- 172.21.15.32:0/3784360815 shutdown_connections 2024-04-17T10:02:03.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.684+0000 7f0861800640 1 --2- 172.21.15.32:0/3784360815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f085c06c140 0x7f085c0ce290 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.684+0000 7f0861800640 1 --2- 172.21.15.32:0/3784360815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f085c15b7d0 0x7f085c06e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.684+0000 7f0861800640 1 -- 172.21.15.32:0/3784360815 >> 172.21.15.32:0/3784360815 conn(0x7f085c0b2120 msgr2=0x7f085c0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:03.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.684+0000 7f0861800640 1 -- 172.21.15.32:0/3784360815 shutdown_connections 2024-04-17T10:02:03.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.684+0000 7f0861800640 1 -- 172.21.15.32:0/3784360815 wait complete. 2024-04-17T10:02:03.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f0861800640 1 Processor -- start 2024-04-17T10:02:03.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f0861800640 1 -- start start 2024-04-17T10:02:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f0861800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f085c06c140 0x7f085c15e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f0861800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f085c06f7c0 0x7f085c1595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f0861800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f085c15b7d0 0x7f085c148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f0861800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f085c071fe0 con 0x7f085c06f7c0 2024-04-17T10:02:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f0861800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f085c071e60 con 0x7f085c15b7d0 2024-04-17T10:02:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f0861800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f085c072160 con 0x7f085c06c140 2024-04-17T10:02:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f0852000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f085c06f7c0 0x7f085c1595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f0852000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f085c06f7c0 0x7f085c1595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41758/0 (socket says 172.21.15.32:41758) 2024-04-17T10:02:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f0852000640 1 -- 172.21.15.32:0/1258970243 learned_addr learned my addr 172.21.15.32:0/1258970243 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f085aa00640 1 --2- 172.21.15.32:0/1258970243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f085c06c140 0x7f085c15e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f085b400640 1 --2- 172.21.15.32:0/1258970243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f085c15b7d0 0x7f085c148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f0852000640 1 -- 172.21.15.32:0/1258970243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f085c06c140 msgr2=0x7f085c15e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f0852000640 1 --2- 172.21.15.32:0/1258970243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f085c06c140 0x7f085c15e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f0852000640 1 -- 172.21.15.32:0/1258970243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f085c15b7d0 msgr2=0x7f085c148e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f0852000640 1 --2- 172.21.15.32:0/1258970243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f085c15b7d0 0x7f085c148e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f0852000640 1 -- 172.21.15.32:0/1258970243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085c156860 con 0x7f085c06f7c0 2024-04-17T10:02:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f0852000640 1 --2- 172.21.15.32:0/1258970243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f085c06f7c0 0x7f085c1595b0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f084400c450 tx=0x7f084400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f0853400640 1 -- 172.21.15.32:0/1258970243 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844017070 con 0x7f085c06f7c0 2024-04-17T10:02:03.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f0853400640 1 -- 172.21.15.32:0/1258970243 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0844005cf0 con 0x7f085c06f7c0 2024-04-17T10:02:03.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f0853400640 1 -- 172.21.15.32:0/1258970243 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844007290 con 0x7f085c06f7c0 2024-04-17T10:02:03.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f0861800640 1 -- 172.21.15.32:0/1258970243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085c156b40 con 0x7f085c06f7c0 2024-04-17T10:02:03.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f0861800640 1 -- 172.21.15.32:0/1258970243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f085c15c7c0 con 0x7f085c06f7c0 2024-04-17T10:02:03.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.688+0000 7f0861800640 1 -- 172.21.15.32:0/1258970243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f085c068a90 con 0x7f085c06f7c0 2024-04-17T10:02:03.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.692+0000 7f0853400640 1 -- 172.21.15.32:0/1258970243 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f084401a070 con 0x7f085c06f7c0 2024-04-17T10:02:03.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.692+0000 7f0853400640 1 --2- 172.21.15.32:0/1258970243 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0838068380 0x7f083806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:03.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.692+0000 7f0853400640 1 -- 172.21.15.32:0/1258970243 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f0844085d80 con 0x7f085c06f7c0 2024-04-17T10:02:03.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.692+0000 7f085aa00640 1 --2- 172.21.15.32:0/1258970243 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0838068380 0x7f083806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:03.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.692+0000 7f085aa00640 1 --2- 172.21.15.32:0/1258970243 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0838068380 0x7f083806a840 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f084c009ac0 tx=0x7f084c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:03.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.696+0000 7f0853400640 1 -- 172.21.15.32:0/1258970243 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0844055c50 con 0x7f085c06f7c0 2024-04-17T10:02:03.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.860+0000 7f0861800640 1 -- 172.21.15.32:0/1258970243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f085c072bb0 con 0x7f085c06f7c0 2024-04-17T10:02:03.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.860+0000 7f0853400640 1 -- 172.21.15.32:0/1258970243 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f0844059f50 con 0x7f085c06f7c0 2024-04-17T10:02:03.859 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T10:02:03.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.864+0000 7f0861800640 1 -- 172.21.15.32:0/1258970243 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0838068380 msgr2=0x7f083806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:03.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.864+0000 7f0861800640 1 --2- 172.21.15.32:0/1258970243 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0838068380 0x7f083806a840 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f084c009ac0 tx=0x7f084c009190 comp rx=0 tx=0).stop 2024-04-17T10:02:03.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.864+0000 7f0861800640 1 -- 172.21.15.32:0/1258970243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f085c06f7c0 msgr2=0x7f085c1595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:03.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.864+0000 7f0861800640 1 --2- 172.21.15.32:0/1258970243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f085c06f7c0 0x7f085c1595b0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f084400c450 tx=0x7f084400c910 comp rx=0 tx=0).stop 2024-04-17T10:02:03.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.864+0000 7f0861800640 1 -- 172.21.15.32:0/1258970243 shutdown_connections 2024-04-17T10:02:03.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.864+0000 7f0861800640 1 --2- 172.21.15.32:0/1258970243 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0838068380 0x7f083806a840 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.864+0000 7f0861800640 1 --2- 172.21.15.32:0/1258970243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f085c15b7d0 0x7f085c148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.864+0000 7f0861800640 1 --2- 172.21.15.32:0/1258970243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f085c06f7c0 0x7f085c1595b0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.864+0000 7f0861800640 1 --2- 172.21.15.32:0/1258970243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f085c06c140 0x7f085c15e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:03.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.864+0000 7f0861800640 1 -- 172.21.15.32:0/1258970243 >> 172.21.15.32:0/1258970243 conn(0x7f085c0b2120 msgr2=0x7f085c152420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:03.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.864+0000 7f0861800640 1 -- 172.21.15.32:0/1258970243 shutdown_connections 2024-04-17T10:02:03.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:03.864+0000 7f0861800640 1 -- 172.21.15.32:0/1258970243 wait complete. 2024-04-17T10:02:03.881 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:02:04.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.024+0000 7f5886600640 1 Processor -- start 2024-04-17T10:02:04.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.024+0000 7f5886600640 1 -- start start 2024-04-17T10:02:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.024+0000 7f5886600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5880155a70 0x7f588015f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.024+0000 7f5886600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5880155e50 0x7f588015d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.024+0000 7f5886600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f588015d5d0 0x7f588015c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.024+0000 7f5886600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f58800b42a0 con 0x7f588015d5d0 2024-04-17T10:02:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.024+0000 7f5886600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f58800d1250 con 0x7f5880155e50 2024-04-17T10:02:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.024+0000 7f5886600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5880160110 con 0x7f5880155a70 2024-04-17T10:02:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.028+0000 7f587fe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f588015d5d0 0x7f588015c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.028+0000 7f587fe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f588015d5d0 0x7f588015c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41770/0 (socket says 172.21.15.32:41770) 2024-04-17T10:02:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.028+0000 7f587fe00640 1 -- 172.21.15.32:0/1158070873 learned_addr learned my addr 172.21.15.32:0/1158070873 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.028+0000 7f5876a00640 1 --2- 172.21.15.32:0/1158070873 >> v2:172.21.15.144:3300/0 conn(0x7f5880155e50 0x7f588015d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.028+0000 7f587f400640 1 --2- 172.21.15.32:0/1158070873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5880155a70 0x7f588015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.028+0000 7f587fe00640 1 -- 172.21.15.32:0/1158070873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5880155a70 msgr2=0x7f588015f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:04.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.028+0000 7f587fe00640 1 --2- 172.21.15.32:0/1158070873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5880155a70 0x7f588015f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.028+0000 7f587fe00640 1 -- 172.21.15.32:0/1158070873 >> v2:172.21.15.144:3300/0 conn(0x7f5880155e50 msgr2=0x7f588015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:04.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.028+0000 7f587fe00640 1 --2- 172.21.15.32:0/1158070873 >> v2:172.21.15.144:3300/0 conn(0x7f5880155e50 0x7f588015d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.028+0000 7f587fe00640 1 -- 172.21.15.32:0/1158070873 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588015f870 con 0x7f588015d5d0 2024-04-17T10:02:04.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.028+0000 7f587f400640 1 --2- 172.21.15.32:0/1158070873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5880155a70 0x7f588015f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:02:04.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.028+0000 7f587fe00640 1 --2- 172.21.15.32:0/1158070873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f588015d5d0 0x7f588015c260 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f5880160290 tx=0x7f587801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14abeae372e4a71e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.028+0000 7f587ea00640 1 -- 172.21.15.32:0/1158070873 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58780aa610 con 0x7f588015d5d0 2024-04-17T10:02:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.028+0000 7f587ea00640 1 -- 172.21.15.32:0/1158070873 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58780af070 con 0x7f588015d5d0 2024-04-17T10:02:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.028+0000 7f587ea00640 1 -- 172.21.15.32:0/1158070873 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58780aa9d0 con 0x7f588015d5d0 2024-04-17T10:02:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5886600640 1 -- 172.21.15.32:0/1158070873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f588015d5d0 msgr2=0x7f588015c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5886600640 1 --2- 172.21.15.32:0/1158070873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f588015d5d0 0x7f588015c260 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f5880160290 tx=0x7f587801c4f0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5886600640 1 -- 172.21.15.32:0/1158070873 shutdown_connections 2024-04-17T10:02:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5886600640 1 --2- 172.21.15.32:0/1158070873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f588015d5d0 0x7f588015c260 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5886600640 1 --2- 172.21.15.32:0/1158070873 >> v2:172.21.15.144:3300/0 conn(0x7f5880155e50 0x7f588015d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5886600640 1 --2- 172.21.15.32:0/1158070873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5880155a70 0x7f588015f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5886600640 1 -- 172.21.15.32:0/1158070873 >> 172.21.15.32:0/1158070873 conn(0x7f588006ba30 msgr2=0x7f588006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5886600640 1 -- 172.21.15.32:0/1158070873 shutdown_connections 2024-04-17T10:02:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5886600640 1 -- 172.21.15.32:0/1158070873 wait complete. 2024-04-17T10:02:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5886600640 1 Processor -- start 2024-04-17T10:02:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5886600640 1 -- start start 2024-04-17T10:02:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5886600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5880155a70 0x7f58801529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5886600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5880155e50 0x7f588014a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5886600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f588015d5d0 0x7f5880154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5886600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f58800b5320 con 0x7f5880155e50 2024-04-17T10:02:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5886600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f58800b51a0 con 0x7f588015d5d0 2024-04-17T10:02:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5886600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f58800d0170 con 0x7f5880155a70 2024-04-17T10:02:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5876a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5880155e50 0x7f588014a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5876a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5880155e50 0x7f588014a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41778/0 (socket says 172.21.15.32:41778) 2024-04-17T10:02:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5876a00640 1 -- 172.21.15.32:0/2877245409 learned_addr learned my addr 172.21.15.32:0/2877245409 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f587f400640 1 --2- 172.21.15.32:0/2877245409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5880155a70 0x7f58801529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f587fe00640 1 --2- 172.21.15.32:0/2877245409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f588015d5d0 0x7f5880154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5876a00640 1 -- 172.21.15.32:0/2877245409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5880155a70 msgr2=0x7f58801529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5876a00640 1 --2- 172.21.15.32:0/2877245409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5880155a70 0x7f58801529f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5876a00640 1 -- 172.21.15.32:0/2877245409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f588015d5d0 msgr2=0x7f5880154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5876a00640 1 --2- 172.21.15.32:0/2877245409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f588015d5d0 0x7f5880154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f5876a00640 1 -- 172.21.15.32:0/2877245409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5880156400 con 0x7f5880155e50 2024-04-17T10:02:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f587f400640 1 --2- 172.21.15.32:0/2877245409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5880155a70 0x7f58801529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:02:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.032+0000 7f587fe00640 1 --2- 172.21.15.32:0/2877245409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f588015d5d0 0x7f5880154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:02:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.036+0000 7f5876a00640 1 --2- 172.21.15.32:0/2877245409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5880155e50 0x7f588014a220 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f5868007ec0 tx=0x7f586800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.036+0000 7f5877e00640 1 -- 172.21.15.32:0/2877245409 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868014620 con 0x7f5880155e50 2024-04-17T10:02:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.036+0000 7f5877e00640 1 -- 172.21.15.32:0/2877245409 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5868019070 con 0x7f5880155e50 2024-04-17T10:02:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.036+0000 7f5877e00640 1 -- 172.21.15.32:0/2877245409 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868014a00 con 0x7f5880155e50 2024-04-17T10:02:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.036+0000 7f5886600640 1 -- 172.21.15.32:0/2877245409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5880153080 con 0x7f5880155e50 2024-04-17T10:02:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.036+0000 7f5886600640 1 -- 172.21.15.32:0/2877245409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58801566a0 con 0x7f5880155e50 2024-04-17T10:02:04.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.036+0000 7f5886600640 1 -- 172.21.15.32:0/2877245409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5880068a90 con 0x7f5880155e50 2024-04-17T10:02:04.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.036+0000 7f5877e00640 1 -- 172.21.15.32:0/2877245409 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5868014ba0 con 0x7f5880155e50 2024-04-17T10:02:04.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.036+0000 7f5877e00640 1 --2- 172.21.15.32:0/2877245409 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f585c068380 0x7f585c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:04.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.036+0000 7f5877e00640 1 -- 172.21.15.32:0/2877245409 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f5868087250 con 0x7f5880155e50 2024-04-17T10:02:04.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.036+0000 7f587f400640 1 --2- 172.21.15.32:0/2877245409 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f585c068380 0x7f585c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:04.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.040+0000 7f587f400640 1 --2- 172.21.15.32:0/2877245409 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f585c068380 0x7f585c06a840 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5870008930 tx=0x7f5870008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:04.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.040+0000 7f5877e00640 1 -- 172.21.15.32:0/2877245409 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5868057800 con 0x7f5880155e50 2024-04-17T10:02:04.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.220+0000 7f5886600640 1 -- 172.21.15.32:0/2877245409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f58801535a0 con 0x7f5880155e50 2024-04-17T10:02:04.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.220+0000 7f5877e00640 1 -- 172.21.15.32:0/2877245409 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v97) ==== 80+0+0 (secure 0 0 0) 0x7f5868057100 con 0x7f5880155e50 2024-04-17T10:02:04.219 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T10:02:04.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.224+0000 7f5886600640 1 -- 172.21.15.32:0/2877245409 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f585c068380 msgr2=0x7f585c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:04.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.224+0000 7f5886600640 1 --2- 172.21.15.32:0/2877245409 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f585c068380 0x7f585c06a840 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5870008930 tx=0x7f5870008040 comp rx=0 tx=0).stop 2024-04-17T10:02:04.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.224+0000 7f5886600640 1 -- 172.21.15.32:0/2877245409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5880155e50 msgr2=0x7f588014a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:04.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.224+0000 7f5886600640 1 --2- 172.21.15.32:0/2877245409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5880155e50 0x7f588014a220 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f5868007ec0 tx=0x7f586800e4b0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.224+0000 7f5886600640 1 -- 172.21.15.32:0/2877245409 shutdown_connections 2024-04-17T10:02:04.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.224+0000 7f5886600640 1 --2- 172.21.15.32:0/2877245409 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f585c068380 0x7f585c06a840 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.224+0000 7f5886600640 1 --2- 172.21.15.32:0/2877245409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f588015d5d0 0x7f5880154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.224+0000 7f5886600640 1 --2- 172.21.15.32:0/2877245409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5880155e50 0x7f588014a220 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.224+0000 7f5886600640 1 --2- 172.21.15.32:0/2877245409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5880155a70 0x7f58801529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.224+0000 7f5886600640 1 -- 172.21.15.32:0/2877245409 >> 172.21.15.32:0/2877245409 conn(0x7f588006ba30 msgr2=0x7f58801597c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:04.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.224+0000 7f5886600640 1 -- 172.21.15.32:0/2877245409 shutdown_connections 2024-04-17T10:02:04.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.224+0000 7f5886600640 1 -- 172.21.15.32:0/2877245409 wait complete. 2024-04-17T10:02:04.240 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:02:04.240 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:02:04.250 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:02:04.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.436+0000 7f515c400640 1 Processor -- start 2024-04-17T10:02:04.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.436+0000 7f515c400640 1 -- start start 2024-04-17T10:02:04.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.440+0000 7f515c400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5154159010 0x7f515415ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:04.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.440+0000 7f515c400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51540d05a0 0x7f51540d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:04.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.440+0000 7f515c400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f51540b4200 0x7f51540b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:04.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.440+0000 7f515c400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f515406e390 con 0x7f51540d05a0 2024-04-17T10:02:04.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.440+0000 7f515c400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f515406e670 con 0x7f51540b4200 2024-04-17T10:02:04.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.440+0000 7f515c400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f515406e950 con 0x7f5154159010 2024-04-17T10:02:04.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.440+0000 7f5159000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51540d05a0 0x7f51540d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:04.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.440+0000 7f5159000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51540d05a0 0x7f51540d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41790/0 (socket says 172.21.15.32:41790) 2024-04-17T10:02:04.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.440+0000 7f5159000640 1 -- 172.21.15.32:0/701463029 learned_addr learned my addr 172.21.15.32:0/701463029 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:04.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.440+0000 7f515a400640 1 --2- 172.21.15.32:0/701463029 >> v2:172.21.15.144:3300/0 conn(0x7f51540b4200 0x7f51540b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:04.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.440+0000 7f5159a00640 1 --2- 172.21.15.32:0/701463029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5154159010 0x7f515415ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:04.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.440+0000 7f5159000640 1 -- 172.21.15.32:0/701463029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5154159010 msgr2=0x7f515415ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:04.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.440+0000 7f5159000640 1 --2- 172.21.15.32:0/701463029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5154159010 0x7f515415ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.440+0000 7f5159000640 1 -- 172.21.15.32:0/701463029 >> v2:172.21.15.144:3300/0 conn(0x7f51540b4200 msgr2=0x7f51540b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:04.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.440+0000 7f5159000640 1 --2- 172.21.15.32:0/701463029 >> v2:172.21.15.144:3300/0 conn(0x7f51540b4200 0x7f51540b46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.440+0000 7f5159000640 1 -- 172.21.15.32:0/701463029 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515406ec30 con 0x7f51540d05a0 2024-04-17T10:02:04.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.440+0000 7f515a400640 1 --2- 172.21.15.32:0/701463029 >> v2:172.21.15.144:3300/0 conn(0x7f51540b4200 0x7f51540b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:02:04.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.444+0000 7f5159000640 1 --2- 172.21.15.32:0/701463029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51540d05a0 0x7f51540d0bd0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f5154073510 tx=0x7f514001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da57b58fff469c14 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:04.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.444+0000 7f5153e00640 1 -- 172.21.15.32:0/701463029 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51400b0070 con 0x7f51540d05a0 2024-04-17T10:02:04.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.444+0000 7f5153e00640 1 -- 172.21.15.32:0/701463029 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51400ab600 con 0x7f51540d05a0 2024-04-17T10:02:04.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.444+0000 7f5153e00640 1 -- 172.21.15.32:0/701463029 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51400ab9d0 con 0x7f51540d05a0 2024-04-17T10:02:04.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.444+0000 7f515c400640 1 -- 172.21.15.32:0/701463029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51540d05a0 msgr2=0x7f51540d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:04.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.444+0000 7f515c400640 1 --2- 172.21.15.32:0/701463029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51540d05a0 0x7f51540d0bd0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f5154073510 tx=0x7f514001c490 comp rx=0 tx=0).stop 2024-04-17T10:02:04.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.444+0000 7f515c400640 1 -- 172.21.15.32:0/701463029 shutdown_connections 2024-04-17T10:02:04.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.444+0000 7f515c400640 1 --2- 172.21.15.32:0/701463029 >> v2:172.21.15.144:3300/0 conn(0x7f51540b4200 0x7f51540b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.444+0000 7f515c400640 1 --2- 172.21.15.32:0/701463029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51540d05a0 0x7f51540d0bd0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.444+0000 7f515c400640 1 --2- 172.21.15.32:0/701463029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5154159010 0x7f515415ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.444+0000 7f515c400640 1 -- 172.21.15.32:0/701463029 >> 172.21.15.32:0/701463029 conn(0x7f515406ba30 msgr2=0x7f515406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:04.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.444+0000 7f515c400640 1 -- 172.21.15.32:0/701463029 shutdown_connections 2024-04-17T10:02:04.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.444+0000 7f515c400640 1 -- 172.21.15.32:0/701463029 wait complete. 2024-04-17T10:02:04.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.444+0000 7f515c400640 1 Processor -- start 2024-04-17T10:02:04.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.444+0000 7f515c400640 1 -- start start 2024-04-17T10:02:04.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f515c400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51540b4200 0x7f51540af740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:04.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f515c400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51540d05a0 0x7f5154150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:04.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f515c400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5154159010 0x7f51541557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:04.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f515c400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f515406ffd0 con 0x7f51540b4200 2024-04-17T10:02:04.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f515c400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f515406fe50 con 0x7f5154159010 2024-04-17T10:02:04.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f515c400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f51540721b0 con 0x7f51540d05a0 2024-04-17T10:02:04.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f5159a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51540b4200 0x7f51540af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:04.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f5159a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51540b4200 0x7f51540af740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41802/0 (socket says 172.21.15.32:41802) 2024-04-17T10:02:04.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f5159a00640 1 -- 172.21.15.32:0/600959592 learned_addr learned my addr 172.21.15.32:0/600959592 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:04.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f5159000640 1 --2- 172.21.15.32:0/600959592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51540d05a0 0x7f5154150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:04.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f515a400640 1 --2- 172.21.15.32:0/600959592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5154159010 0x7f51541557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:04.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f5159a00640 1 -- 172.21.15.32:0/600959592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51540d05a0 msgr2=0x7f5154150230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:04.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f5159a00640 1 --2- 172.21.15.32:0/600959592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51540d05a0 0x7f5154150230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f5159a00640 1 -- 172.21.15.32:0/600959592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5154159010 msgr2=0x7f51541557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:04.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f5159a00640 1 --2- 172.21.15.32:0/600959592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5154159010 0x7f51541557c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f5159a00640 1 -- 172.21.15.32:0/600959592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51541489b0 con 0x7f51540b4200 2024-04-17T10:02:04.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f5159a00640 1 --2- 172.21.15.32:0/600959592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51540b4200 0x7f51540af740 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f514800d620 tx=0x7f514800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:04.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f5151600640 1 -- 172.21.15.32:0/600959592 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514800b620 con 0x7f51540b4200 2024-04-17T10:02:04.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f5151600640 1 -- 172.21.15.32:0/600959592 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5148019070 con 0x7f51540b4200 2024-04-17T10:02:04.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f515c400640 1 -- 172.21.15.32:0/600959592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5154148c90 con 0x7f51540b4200 2024-04-17T10:02:04.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f5151600640 1 -- 172.21.15.32:0/600959592 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514800b9e0 con 0x7f51540b4200 2024-04-17T10:02:04.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f515c400640 1 -- 172.21.15.32:0/600959592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5154149250 con 0x7f51540b4200 2024-04-17T10:02:04.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f515c400640 1 -- 172.21.15.32:0/600959592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5154068a90 con 0x7f51540b4200 2024-04-17T10:02:04.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.448+0000 7f5151600640 1 -- 172.21.15.32:0/600959592 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f514800bb80 con 0x7f51540b4200 2024-04-17T10:02:04.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.452+0000 7f5151600640 1 --2- 172.21.15.32:0/600959592 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5134068380 0x7f513406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:04.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.452+0000 7f5151600640 1 -- 172.21.15.32:0/600959592 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f5148086cd0 con 0x7f51540b4200 2024-04-17T10:02:04.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.452+0000 7f5159000640 1 --2- 172.21.15.32:0/600959592 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5134068380 0x7f513406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:04.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.452+0000 7f5159000640 1 --2- 172.21.15.32:0/600959592 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5134068380 0x7f513406a840 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f51400ad000 tx=0x7f5140008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:04.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.456+0000 7f5151600640 1 -- 172.21.15.32:0/600959592 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5148056ba0 con 0x7f51540b4200 2024-04-17T10:02:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.620+0000 7f515c400640 1 -- 172.21.15.32:0/600959592 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51540b20a0 con 0x7f51540b4200 2024-04-17T10:02:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.624+0000 7f5151600640 1 -- 172.21.15.32:0/600959592 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+16620 (secure 0 0 0) 0x7f514805aea0 con 0x7f51540b4200 2024-04-17T10:02:04.620 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:02:04.621 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":97,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:01:42.155683+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:01:38.938460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-17T10:01:39.505171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:02:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.624+0000 7f515c400640 1 -- 172.21.15.32:0/600959592 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5134068380 msgr2=0x7f513406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.624+0000 7f515c400640 1 --2- 172.21.15.32:0/600959592 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5134068380 0x7f513406a840 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f51400ad000 tx=0x7f5140008040 comp rx=0 tx=0).stop 2024-04-17T10:02:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.624+0000 7f515c400640 1 -- 172.21.15.32:0/600959592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51540b4200 msgr2=0x7f51540af740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.624+0000 7f515c400640 1 --2- 172.21.15.32:0/600959592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51540b4200 0x7f51540af740 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f514800d620 tx=0x7f514800dae0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.624+0000 7f515c400640 1 -- 172.21.15.32:0/600959592 shutdown_connections 2024-04-17T10:02:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.624+0000 7f515c400640 1 --2- 172.21.15.32:0/600959592 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5134068380 0x7f513406a840 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.624+0000 7f515c400640 1 --2- 172.21.15.32:0/600959592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51540d05a0 0x7f5154150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.624+0000 7f515c400640 1 --2- 172.21.15.32:0/600959592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5154159010 0x7f51541557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.624+0000 7f515c400640 1 --2- 172.21.15.32:0/600959592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51540b4200 0x7f51540af740 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.624+0000 7f515c400640 1 -- 172.21.15.32:0/600959592 >> 172.21.15.32:0/600959592 conn(0x7f515406ba30 msgr2=0x7f51540ae4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.624+0000 7f515c400640 1 -- 172.21.15.32:0/600959592 shutdown_connections 2024-04-17T10:02:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.624+0000 7f515c400640 1 -- 172.21.15.32:0/600959592 wait complete. 2024-04-17T10:02:04.642 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:01:38.938460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:01:39.505171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:02:04.642 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:02:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.788+0000 7f10a9a00640 1 Processor -- start 2024-04-17T10:02:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.788+0000 7f10a9a00640 1 -- start start 2024-04-17T10:02:04.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.788+0000 7f10a9a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f10a40cf370 0x7f10a40cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:04.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.788+0000 7f10a9a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10a4071e60 0x7f10a4072270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:04.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.788+0000 7f10a9a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10a40d0130 con 0x7f10a406ba30 2024-04-17T10:02:04.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.788+0000 7f10a9a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f10a415d090 con 0x7f10a40cf370 2024-04-17T10:02:04.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.788+0000 7f10a9a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f10a415d370 con 0x7f10a4071e60 2024-04-17T10:02:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.788+0000 7f109a000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f10a40cf370 0x7f10a40cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.788+0000 7f10a3400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10a4071e60 0x7f10a4072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.788+0000 7f10a2a00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f10a406ba30 0x7f10a406be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:51526/0 (socket says 172.21.15.32:51526) 2024-04-17T10:02:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.788+0000 7f10a2a00640 1 -- 172.21.15.32:0/1294278591 learned_addr learned my addr 172.21.15.32:0/1294278591 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:02:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.792+0000 7f109a000640 1 -- 172.21.15.32:0/1294278591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10a4071e60 msgr2=0x7f10a4072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.792+0000 7f109a000640 1 --2- 172.21.15.32:0/1294278591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10a4071e60 0x7f10a4072270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.792+0000 7f109a000640 1 -- 172.21.15.32:0/1294278591 >> v1:172.21.15.144:6789/0 conn(0x7f10a406ba30 legacy=0x7f10a406be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.792+0000 7f10a2000640 1 -- 172.21.15.32:0/1294278591 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1224127737 0 0) 0x7f10a40d0130 con 0x7f10a406ba30 2024-04-17T10:02:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.792+0000 7f109a000640 1 -- 172.21.15.32:0/1294278591 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a415d650 con 0x7f10a40cf370 2024-04-17T10:02:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.792+0000 7f109a000640 1 --2- 172.21.15.32:0/1294278591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10a40cf370 0x7f10a40cf7e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f108c0097c0 tx=0x7f108c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1887052b2ead1be3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:04.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.792+0000 7f10a2000640 1 -- 172.21.15.32:0/1294278591 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c0aa620 con 0x7f10a40cf370 2024-04-17T10:02:04.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.792+0000 7f10a2000640 1 -- 172.21.15.32:0/1294278591 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f108c0af070 con 0x7f10a40cf370 2024-04-17T10:02:04.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.792+0000 7f10a2000640 1 -- 172.21.15.32:0/1294278591 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c0aa9e0 con 0x7f10a40cf370 2024-04-17T10:02:04.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a9a00640 1 -- 172.21.15.32:0/1294278591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10a40cf370 msgr2=0x7f10a40cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:04.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a9a00640 1 --2- 172.21.15.32:0/1294278591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10a40cf370 0x7f10a40cf7e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f108c0097c0 tx=0x7f108c01c4a0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a9a00640 1 -- 172.21.15.32:0/1294278591 shutdown_connections 2024-04-17T10:02:04.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a9a00640 1 --2- 172.21.15.32:0/1294278591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10a4071e60 0x7f10a4072270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a9a00640 1 --2- 172.21.15.32:0/1294278591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10a40cf370 0x7f10a40cf7e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a9a00640 1 -- 172.21.15.32:0/1294278591 >> 172.21.15.32:0/1294278591 conn(0x7f10a40b2120 msgr2=0x7f10a40b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:04.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a9a00640 1 -- 172.21.15.32:0/1294278591 shutdown_connections 2024-04-17T10:02:04.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a9a00640 1 -- 172.21.15.32:0/1294278591 wait complete. 2024-04-17T10:02:04.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a9a00640 1 Processor -- start 2024-04-17T10:02:04.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a9a00640 1 -- start start 2024-04-17T10:02:04.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a9a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10a406ba30 0x7f10a41f7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:04.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a9a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10a4071e60 0x7f10a413ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:04.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a9a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10a40cf370 0x7f10a4142530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:04.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a9a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f10a4146230 con 0x7f10a406ba30 2024-04-17T10:02:04.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a9a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f10a415dd80 con 0x7f10a4071e60 2024-04-17T10:02:04.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a9a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f10a41463b0 con 0x7f10a40cf370 2024-04-17T10:02:04.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10a406ba30 0x7f10a41f7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:04.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10a406ba30 0x7f10a41f7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41814/0 (socket says 172.21.15.32:41814) 2024-04-17T10:02:04.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a2a00640 1 -- 172.21.15.32:0/2639379166 learned_addr learned my addr 172.21.15.32:0/2639379166 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:04.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a3400640 1 --2- 172.21.15.32:0/2639379166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10a40cf370 0x7f10a4142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:04.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f109a000640 1 --2- 172.21.15.32:0/2639379166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10a4071e60 0x7f10a413ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a2a00640 1 -- 172.21.15.32:0/2639379166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10a40cf370 msgr2=0x7f10a4142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a2a00640 1 --2- 172.21.15.32:0/2639379166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10a40cf370 0x7f10a4142530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a2a00640 1 -- 172.21.15.32:0/2639379166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10a4071e60 msgr2=0x7f10a413ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a2a00640 1 --2- 172.21.15.32:0/2639379166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10a4071e60 0x7f10a413ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.796+0000 7f10a2a00640 1 -- 172.21.15.32:0/2639379166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a41f8150 con 0x7f10a406ba30 2024-04-17T10:02:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.800+0000 7f10a2a00640 1 --2- 172.21.15.32:0/2639379166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10a406ba30 0x7f10a41f7c10 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f109400cc10 tx=0x7f109400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.800+0000 7f109b400640 1 -- 172.21.15.32:0/2639379166 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1094017070 con 0x7f10a406ba30 2024-04-17T10:02:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.800+0000 7f109b400640 1 -- 172.21.15.32:0/2639379166 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f109400ce20 con 0x7f10a406ba30 2024-04-17T10:02:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.800+0000 7f109b400640 1 -- 172.21.15.32:0/2639379166 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1094012640 con 0x7f10a406ba30 2024-04-17T10:02:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.800+0000 7f10a9a00640 1 -- 172.21.15.32:0/2639379166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a413b380 con 0x7f10a406ba30 2024-04-17T10:02:04.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.800+0000 7f10a9a00640 1 -- 172.21.15.32:0/2639379166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a413cb60 con 0x7f10a406ba30 2024-04-17T10:02:04.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.800+0000 7f10a9a00640 1 -- 172.21.15.32:0/2639379166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a4068a90 con 0x7f10a406ba30 2024-04-17T10:02:04.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.800+0000 7f109b400640 1 -- 172.21.15.32:0/2639379166 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f109401a070 con 0x7f10a406ba30 2024-04-17T10:02:04.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.800+0000 7f109b400640 1 --2- 172.21.15.32:0/2639379166 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1080068380 0x7f108006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:04.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.800+0000 7f109b400640 1 -- 172.21.15.32:0/2639379166 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f10940868a0 con 0x7f10a406ba30 2024-04-17T10:02:04.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.800+0000 7f109a000640 1 --2- 172.21.15.32:0/2639379166 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1080068380 0x7f108006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:04.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.804+0000 7f109a000640 1 --2- 172.21.15.32:0/2639379166 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1080068380 0x7f108006a840 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f108c0ae000 tx=0x7f108c0b5040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:04.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.804+0000 7f109b400640 1 -- 172.21.15.32:0/2639379166 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f10940566e0 con 0x7f10a406ba30 2024-04-17T10:02:04.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.972+0000 7f10a9a00640 1 -- 172.21.15.32:0/2639379166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f10a406c850 con 0x7f10a406ba30 2024-04-17T10:02:04.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.972+0000 7f109b400640 1 -- 172.21.15.32:0/2639379166 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f109405a9e0 con 0x7f10a406ba30 2024-04-17T10:02:04.969 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T10:02:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.972+0000 7f10a9a00640 1 -- 172.21.15.32:0/2639379166 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1080068380 msgr2=0x7f108006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.972+0000 7f10a9a00640 1 --2- 172.21.15.32:0/2639379166 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1080068380 0x7f108006a840 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f108c0ae000 tx=0x7f108c0b5040 comp rx=0 tx=0).stop 2024-04-17T10:02:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.972+0000 7f10a9a00640 1 -- 172.21.15.32:0/2639379166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10a406ba30 msgr2=0x7f10a41f7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.972+0000 7f10a9a00640 1 --2- 172.21.15.32:0/2639379166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10a406ba30 0x7f10a41f7c10 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f109400cc10 tx=0x7f109400a430 comp rx=0 tx=0).stop 2024-04-17T10:02:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.972+0000 7f10a9a00640 1 -- 172.21.15.32:0/2639379166 shutdown_connections 2024-04-17T10:02:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.972+0000 7f10a9a00640 1 --2- 172.21.15.32:0/2639379166 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1080068380 0x7f108006a840 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.972+0000 7f10a9a00640 1 --2- 172.21.15.32:0/2639379166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10a40cf370 0x7f10a4142530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.972+0000 7f10a9a00640 1 --2- 172.21.15.32:0/2639379166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10a4071e60 0x7f10a413ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.972+0000 7f10a9a00640 1 --2- 172.21.15.32:0/2639379166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10a406ba30 0x7f10a41f7c10 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.972+0000 7f10a9a00640 1 -- 172.21.15.32:0/2639379166 >> 172.21.15.32:0/2639379166 conn(0x7f10a40b2120 msgr2=0x7f10a40b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.972+0000 7f10a9a00640 1 -- 172.21.15.32:0/2639379166 shutdown_connections 2024-04-17T10:02:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:04.972+0000 7f10a9a00640 1 -- 172.21.15.32:0/2639379166 wait complete. 2024-04-17T10:02:04.990 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:02:05.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c17600640 1 Processor -- start 2024-04-17T10:02:05.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c17600640 1 -- start start 2024-04-17T10:02:05.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c17600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2c100cf370 0x7f2c100cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c17600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c10071e60 0x7f2c10072270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c17600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c100d0130 con 0x7f2c1006ba30 2024-04-17T10:02:05.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c17600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f2c1015d090 con 0x7f2c100cf370 2024-04-17T10:02:05.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c17600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2c1015d370 con 0x7f2c10071e60 2024-04-17T10:02:05.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c14c00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2c1006ba30 0x7f2c1006be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60422/0 (socket says 172.21.15.32:60422) 2024-04-17T10:02:05.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c14c00640 1 -- 172.21.15.32:0/61092619 learned_addr learned my addr 172.21.15.32:0/61092619 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:02:05.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c0fe00640 1 --2- 172.21.15.32:0/61092619 >> v2:172.21.15.144:3300/0 conn(0x7f2c100cf370 0x7f2c100cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:05.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c15600640 1 --2- 172.21.15.32:0/61092619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c10071e60 0x7f2c10072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:05.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c0f400640 1 -- 172.21.15.32:0/61092619 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3366213678 0 0) 0x7f2c100d0130 con 0x7f2c1006ba30 2024-04-17T10:02:05.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c0f400640 1 -- 172.21.15.32:0/61092619 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bf8003530 con 0x7f2c1006ba30 2024-04-17T10:02:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c0f400640 1 -- 172.21.15.32:0/61092619 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2393750457 0 0) 0x7f2bf8003530 con 0x7f2c1006ba30 2024-04-17T10:02:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c0f400640 1 -- 172.21.15.32:0/61092619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c10071e60 msgr2=0x7f2c10072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c0f400640 1 --2- 172.21.15.32:0/61092619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c10071e60 0x7f2c10072270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c0f400640 1 -- 172.21.15.32:0/61092619 >> v2:172.21.15.144:3300/0 conn(0x7f2c100cf370 msgr2=0x7f2c100cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c0f400640 1 --2- 172.21.15.32:0/61092619 >> v2:172.21.15.144:3300/0 conn(0x7f2c100cf370 0x7f2c100cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c0f400640 1 -- 172.21.15.32:0/61092619 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1015d650 con 0x7f2c1006ba30 2024-04-17T10:02:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c0f400640 1 -- 172.21.15.32:0/61092619 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f2c040016a0 con 0x7f2c1006ba30 2024-04-17T10:02:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c0f400640 1 -- 172.21.15.32:0/61092619 >> v1:172.21.15.32:6789/0 conn(0x7f2c1006ba30 legacy=0x7f2c1006be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c0f400640 1 --2- 172.21.15.32:0/61092619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c10071e60 0x7f2bf80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c0f400640 1 --2- 172.21.15.32:0/61092619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bf8003f40 0x7f2bf8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c0f400640 1 --2- 172.21.15.32:0/61092619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bf8004d50 0x7f2bf80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c0f400640 1 -- 172.21.15.32:0/61092619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1015d090 con 0x7f2bf8003f40 2024-04-17T10:02:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c0f400640 1 -- 172.21.15.32:0/61092619 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1015d370 con 0x7f2bf8004d50 2024-04-17T10:02:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c0f400640 1 -- 172.21.15.32:0/61092619 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf800b9d0 con 0x7f2c10071e60 2024-04-17T10:02:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c0fe00640 1 --2- 172.21.15.32:0/61092619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bf8003f40 0x7f2bf8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:05.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c15600640 1 --2- 172.21.15.32:0/61092619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bf8004d50 0x7f2bf80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:05.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c14c00640 1 --2- 172.21.15.32:0/61092619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c10071e60 0x7f2bf80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:05.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c0fe00640 1 -- 172.21.15.32:0/61092619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c10071e60 msgr2=0x7f2bf80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:05.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c0fe00640 1 --2- 172.21.15.32:0/61092619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c10071e60 0x7f2bf80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c0fe00640 1 -- 172.21.15.32:0/61092619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bf8004d50 msgr2=0x7f2bf80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:05.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c0fe00640 1 --2- 172.21.15.32:0/61092619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bf8004d50 0x7f2bf80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c0fe00640 1 -- 172.21.15.32:0/61092619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1015d650 con 0x7f2bf8003f40 2024-04-17T10:02:05.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.136+0000 7f2c15600640 1 --2- 172.21.15.32:0/61092619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bf8004d50 0x7f2bf80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:02:05.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.140+0000 7f2c0fe00640 1 --2- 172.21.15.32:0/61092619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bf8003f40 0x7f2bf8004810 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009ae0 tx=0x7f2bfc01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477ee44a9983389 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:05.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.140+0000 7f2c0f400640 1 -- 172.21.15.32:0/61092619 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0b2070 con 0x7f2bf8003f40 2024-04-17T10:02:05.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.140+0000 7f2c0f400640 1 -- 172.21.15.32:0/61092619 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bfc0a56a0 con 0x7f2bf8003f40 2024-04-17T10:02:05.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.140+0000 7f2c0f400640 1 -- 172.21.15.32:0/61092619 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0096b0 con 0x7f2bf8003f40 2024-04-17T10:02:05.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.140+0000 7f2c17600640 1 -- 172.21.15.32:0/61092619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bf8003f40 msgr2=0x7f2bf8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:05.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.140+0000 7f2c17600640 1 --2- 172.21.15.32:0/61092619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bf8003f40 0x7f2bf8004810 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009ae0 tx=0x7f2bfc01cab0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.140+0000 7f2c15600640 1 -- 172.21.15.32:0/61092619 reap_dead start 2024-04-17T10:02:05.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.140+0000 7f2c17600640 1 -- 172.21.15.32:0/61092619 shutdown_connections 2024-04-17T10:02:05.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.140+0000 7f2c17600640 1 -- 172.21.15.32:0/61092619 >> 172.21.15.32:0/61092619 conn(0x7f2c100b2120 msgr2=0x7f2c100b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:05.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.140+0000 7f2c17600640 1 -- 172.21.15.32:0/61092619 shutdown_connections 2024-04-17T10:02:05.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.140+0000 7f2c17600640 1 -- 172.21.15.32:0/61092619 wait complete. 2024-04-17T10:02:05.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.144+0000 7f2c17600640 1 Processor -- start 2024-04-17T10:02:05.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.144+0000 7f2c17600640 1 -- start start 2024-04-17T10:02:05.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.144+0000 7f2c17600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bf8003f40 0x7f2c101f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.144+0000 7f2c17600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c10135140 0x7f2c101f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.144+0000 7f2c17600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c100a9df0 0x7f2c101340e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.144+0000 7f2c17600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2c100cfea0 con 0x7f2c10135140 2024-04-17T10:02:05.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.144+0000 7f2c17600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2c100cfd20 con 0x7f2c100a9df0 2024-04-17T10:02:05.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.144+0000 7f2c17600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2c1015db70 con 0x7f2bf8003f40 2024-04-17T10:02:05.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.144+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c10135140 0x7f2c101f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:05.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.144+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c10135140 0x7f2c101f55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41838/0 (socket says 172.21.15.32:41838) 2024-04-17T10:02:05.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.144+0000 7f2c0fe00640 1 -- 172.21.15.32:0/660626291 learned_addr learned my addr 172.21.15.32:0/660626291 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:05.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.144+0000 7f2c0fe00640 1 -- 172.21.15.32:0/660626291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bf8003f40 msgr2=0x7f2c101f27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:05.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.144+0000 7f2c0fe00640 1 --2- 172.21.15.32:0/660626291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bf8003f40 0x7f2c101f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.144+0000 7f2c0fe00640 1 -- 172.21.15.32:0/660626291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c100a9df0 msgr2=0x7f2c101340e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:05.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.144+0000 7f2c0fe00640 1 --2- 172.21.15.32:0/660626291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c100a9df0 0x7f2c101340e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.144+0000 7f2c0fe00640 1 -- 172.21.15.32:0/660626291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c101f5af0 con 0x7f2c10135140 2024-04-17T10:02:05.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.144+0000 7f2c0fe00640 1 --2- 172.21.15.32:0/660626291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c10135140 0x7f2c101f55b0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc01c8e0 tx=0x7f2bf8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:05.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.144+0000 7f2c0cc00640 1 -- 172.21.15.32:0/660626291 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0bf070 con 0x7f2c10135140 2024-04-17T10:02:05.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.144+0000 7f2c0cc00640 1 -- 172.21.15.32:0/660626291 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bfc004030 con 0x7f2c10135140 2024-04-17T10:02:05.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.144+0000 7f2c17600640 1 -- 172.21.15.32:0/660626291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c10133040 con 0x7f2c10135140 2024-04-17T10:02:05.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.144+0000 7f2c0cc00640 1 -- 172.21.15.32:0/660626291 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0b2040 con 0x7f2c10135140 2024-04-17T10:02:05.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.144+0000 7f2c17600640 1 -- 172.21.15.32:0/660626291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c10133600 con 0x7f2c10135140 2024-04-17T10:02:05.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.148+0000 7f2c17600640 1 -- 172.21.15.32:0/660626291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c10068a90 con 0x7f2c10135140 2024-04-17T10:02:05.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.148+0000 7f2c0cc00640 1 -- 172.21.15.32:0/660626291 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2bfc003620 con 0x7f2c10135140 2024-04-17T10:02:05.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.148+0000 7f2c0cc00640 1 --2- 172.21.15.32:0/660626291 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2bf0068380 0x7f2bf006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.148+0000 7f2c0cc00640 1 -- 172.21.15.32:0/660626291 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f2bfc120640 con 0x7f2c10135140 2024-04-17T10:02:05.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.148+0000 7f2c14c00640 1 --2- 172.21.15.32:0/660626291 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2bf0068380 0x7f2bf006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:05.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.148+0000 7f2c14c00640 1 --2- 172.21.15.32:0/660626291 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2bf0068380 0x7f2bf006a840 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2c100cd810 tx=0x7f2c04000ee0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:05.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.152+0000 7f2c0cc00640 1 -- 172.21.15.32:0/660626291 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2bfc0167c0 con 0x7f2c10135140 2024-04-17T10:02:05.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.324+0000 7f2c17600640 1 -- 172.21.15.32:0/660626291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2c1006e150 con 0x7f2c10135140 2024-04-17T10:02:05.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.324+0000 7f2c0cc00640 1 -- 172.21.15.32:0/660626291 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f2bfc0f0510 con 0x7f2c10135140 2024-04-17T10:02:05.321 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T10:02:05.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.324+0000 7f2c17600640 1 -- 172.21.15.32:0/660626291 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2bf0068380 msgr2=0x7f2bf006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:05.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.324+0000 7f2c17600640 1 --2- 172.21.15.32:0/660626291 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2bf0068380 0x7f2bf006a840 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2c100cd810 tx=0x7f2c04000ee0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.324+0000 7f2c17600640 1 -- 172.21.15.32:0/660626291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c10135140 msgr2=0x7f2c101f55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:05.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.324+0000 7f2c17600640 1 --2- 172.21.15.32:0/660626291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c10135140 0x7f2c101f55b0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc01c8e0 tx=0x7f2bf8005840 comp rx=0 tx=0).stop 2024-04-17T10:02:05.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.324+0000 7f2c17600640 1 -- 172.21.15.32:0/660626291 shutdown_connections 2024-04-17T10:02:05.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.324+0000 7f2c17600640 1 --2- 172.21.15.32:0/660626291 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2bf0068380 0x7f2bf006a840 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.324+0000 7f2c17600640 1 --2- 172.21.15.32:0/660626291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c100a9df0 0x7f2c101340e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.324+0000 7f2c17600640 1 --2- 172.21.15.32:0/660626291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c10135140 0x7f2c101f55b0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.324+0000 7f2c17600640 1 --2- 172.21.15.32:0/660626291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bf8003f40 0x7f2c101f27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.324+0000 7f2c17600640 1 -- 172.21.15.32:0/660626291 >> 172.21.15.32:0/660626291 conn(0x7f2c100b2120 msgr2=0x7f2c100d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:05.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.324+0000 7f2c17600640 1 -- 172.21.15.32:0/660626291 shutdown_connections 2024-04-17T10:02:05.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.324+0000 7f2c17600640 1 -- 172.21.15.32:0/660626291 wait complete. 2024-04-17T10:02:05.342 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:02:05.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f147ba00640 1 Processor -- start 2024-04-17T10:02:05.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f147ba00640 1 -- start start 2024-04-17T10:02:05.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f147ba00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f14741656d0 0x7f1474167ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f147ba00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14740d0130 con 0x7f147406ba30 2024-04-17T10:02:05.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f147ba00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f147415d090 con 0x7f14740cf370 2024-04-17T10:02:05.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f147ba00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f14740cfef0 con 0x7f14741656d0 2024-04-17T10:02:05.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f1479000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f147406ba30 0x7f147406be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60432/0 (socket says 172.21.15.32:60432) 2024-04-17T10:02:05.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f1479000640 1 -- 172.21.15.32:0/719390718 learned_addr learned my addr 172.21.15.32:0/719390718 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:02:05.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f1479a00640 1 --2- 172.21.15.32:0/719390718 >> v2:172.21.15.144:3300/0 conn(0x7f14741656d0 0x7f1474167ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:05.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f1473400640 1 -- 172.21.15.32:0/719390718 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1049520369 0 0) 0x7f14740d0130 con 0x7f147406ba30 2024-04-17T10:02:05.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f1473400640 1 -- 172.21.15.32:0/719390718 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f145c003530 con 0x7f147406ba30 2024-04-17T10:02:05.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f1473400640 1 -- 172.21.15.32:0/719390718 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 775685613 0 0) 0x7f145c003530 con 0x7f147406ba30 2024-04-17T10:02:05.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f1473400640 1 -- 172.21.15.32:0/719390718 >> v2:172.21.15.144:3300/0 conn(0x7f14741656d0 msgr2=0x7f1474167ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:05.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f1473400640 1 --2- 172.21.15.32:0/719390718 >> v2:172.21.15.144:3300/0 conn(0x7f14741656d0 0x7f1474167ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f1473400640 1 -- 172.21.15.32:0/719390718 >> v1:172.21.15.144:6789/0 conn(0x7f14740cf370 legacy=0x7f14740cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:05.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f1473400640 1 -- 172.21.15.32:0/719390718 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147415d370 con 0x7f147406ba30 2024-04-17T10:02:05.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f1473400640 1 -- 172.21.15.32:0/719390718 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f1468001680 con 0x7f147406ba30 2024-04-17T10:02:05.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f1473400640 1 -- 172.21.15.32:0/719390718 >> v1:172.21.15.32:6789/0 conn(0x7f147406ba30 legacy=0x7f147406be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:05.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f1473400640 1 --2- 172.21.15.32:0/719390718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f145c0039c0 0x7f145c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f1473400640 1 --2- 172.21.15.32:0/719390718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f145c004810 0x7f145c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f1473400640 1 --2- 172.21.15.32:0/719390718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c0051d0 0x7f145c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f1473400640 1 -- 172.21.15.32:0/719390718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f14740cfef0 con 0x7f145c0051d0 2024-04-17T10:02:05.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f1473400640 1 -- 172.21.15.32:0/719390718 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f145c00c170 con 0x7f145c004810 2024-04-17T10:02:05.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f1473400640 1 -- 172.21.15.32:0/719390718 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f145c00c370 con 0x7f145c0039c0 2024-04-17T10:02:05.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f1479a00640 1 --2- 172.21.15.32:0/719390718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c0051d0 0x7f145c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:05.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.492+0000 7f1479000640 1 --2- 172.21.15.32:0/719390718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f145c0039c0 0x7f145c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:05.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.488+0000 7f1473e00640 1 --2- 172.21.15.32:0/719390718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f145c004810 0x7f145c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:05.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.492+0000 7f1479a00640 1 -- 172.21.15.32:0/719390718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f145c0039c0 msgr2=0x7f145c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:05.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.492+0000 7f1479a00640 1 --2- 172.21.15.32:0/719390718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f145c0039c0 0x7f145c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.492+0000 7f1479a00640 1 -- 172.21.15.32:0/719390718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f145c004810 msgr2=0x7f145c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:05.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.492+0000 7f1479a00640 1 --2- 172.21.15.32:0/719390718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f145c004810 0x7f145c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.492+0000 7f1479a00640 1 -- 172.21.15.32:0/719390718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145c003530 con 0x7f145c0051d0 2024-04-17T10:02:05.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.492+0000 7f1473e00640 1 --2- 172.21.15.32:0/719390718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f145c004810 0x7f145c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:02:05.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.492+0000 7f1479a00640 1 -- 172.21.15.32:0/719390718 reap_dead start 2024-04-17T10:02:05.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.492+0000 7f1479a00640 1 --2- 172.21.15.32:0/719390718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c0051d0 0x7f145c00b9d0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f146c009f90 tx=0x7f146c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fec1fe00e928163 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:05.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.492+0000 7f1473400640 1 -- 172.21.15.32:0/719390718 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146c0b2070 con 0x7f145c0051d0 2024-04-17T10:02:05.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.492+0000 7f1473400640 1 -- 172.21.15.32:0/719390718 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f146c0a5b80 con 0x7f145c0051d0 2024-04-17T10:02:05.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.492+0000 7f1473400640 1 -- 172.21.15.32:0/719390718 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146c009c10 con 0x7f145c0051d0 2024-04-17T10:02:05.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.492+0000 7f147ba00640 1 -- 172.21.15.32:0/719390718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c0051d0 msgr2=0x7f145c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:05.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.492+0000 7f147ba00640 1 --2- 172.21.15.32:0/719390718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c0051d0 0x7f145c00b9d0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f146c009f90 tx=0x7f146c01cc80 comp rx=0 tx=0).stop 2024-04-17T10:02:05.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.492+0000 7f147ba00640 1 -- 172.21.15.32:0/719390718 shutdown_connections 2024-04-17T10:02:05.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.492+0000 7f147ba00640 1 --2- 172.21.15.32:0/719390718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c0051d0 0x7f145c00b9d0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.492+0000 7f147ba00640 1 -- 172.21.15.32:0/719390718 >> 172.21.15.32:0/719390718 conn(0x7f14740b2120 msgr2=0x7f14740b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:05.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.496+0000 7f147ba00640 1 -- 172.21.15.32:0/719390718 shutdown_connections 2024-04-17T10:02:05.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.496+0000 7f147ba00640 1 -- 172.21.15.32:0/719390718 wait complete. 2024-04-17T10:02:05.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.496+0000 7f147ba00640 1 Processor -- start 2024-04-17T10:02:05.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.496+0000 7f147ba00640 1 -- start start 2024-04-17T10:02:05.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.496+0000 7f147ba00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f145c0051d0 0x7f1474129ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.496+0000 7f147ba00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14740ac010 0x7f1474072790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.496+0000 7f147ba00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1474072cd0 0x7f147412c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.496+0000 7f147ba00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f147415db70 con 0x7f14740ac010 2024-04-17T10:02:05.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.496+0000 7f147ba00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f147415cd00 con 0x7f145c0051d0 2024-04-17T10:02:05.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.496+0000 7f147ba00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f147415dcf0 con 0x7f1474072cd0 2024-04-17T10:02:05.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.496+0000 7f1473e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14740ac010 0x7f1474072790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:05.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.496+0000 7f1473e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14740ac010 0x7f1474072790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41852/0 (socket says 172.21.15.32:41852) 2024-04-17T10:02:05.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.496+0000 7f1473e00640 1 -- 172.21.15.32:0/2689107923 learned_addr learned my addr 172.21.15.32:0/2689107923 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:05.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.496+0000 7f1479000640 1 --2- 172.21.15.32:0/2689107923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f145c0051d0 0x7f1474129ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:05.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.496+0000 7f1479a00640 1 --2- 172.21.15.32:0/2689107923 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1474072cd0 0x7f147412c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:05.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.496+0000 7f1473e00640 1 -- 172.21.15.32:0/2689107923 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1474072cd0 msgr2=0x7f147412c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:05.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.496+0000 7f1473e00640 1 --2- 172.21.15.32:0/2689107923 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1474072cd0 0x7f147412c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.496+0000 7f1473e00640 1 -- 172.21.15.32:0/2689107923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f145c0051d0 msgr2=0x7f1474129ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:05.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.496+0000 7f1473e00640 1 --2- 172.21.15.32:0/2689107923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f145c0051d0 0x7f1474129ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.496+0000 7f1473e00640 1 -- 172.21.15.32:0/2689107923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147412d730 con 0x7f14740ac010 2024-04-17T10:02:05.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.496+0000 7f1473e00640 1 --2- 172.21.15.32:0/2689107923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14740ac010 0x7f1474072790 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f146000e4b0 tx=0x7f146000e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:05.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.496+0000 7f1470c00640 1 -- 172.21.15.32:0/2689107923 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460019070 con 0x7f14740ac010 2024-04-17T10:02:05.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.500+0000 7f1470c00640 1 -- 172.21.15.32:0/2689107923 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14600052e0 con 0x7f14740ac010 2024-04-17T10:02:05.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.500+0000 7f147ba00640 1 -- 172.21.15.32:0/2689107923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f147412d450 con 0x7f14740ac010 2024-04-17T10:02:05.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.500+0000 7f1470c00640 1 -- 172.21.15.32:0/2689107923 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460003820 con 0x7f14740ac010 2024-04-17T10:02:05.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.500+0000 7f147ba00640 1 -- 172.21.15.32:0/2689107923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14741297f0 con 0x7f14740ac010 2024-04-17T10:02:05.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.500+0000 7f147ba00640 1 -- 172.21.15.32:0/2689107923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14740730e0 con 0x7f14740ac010 2024-04-17T10:02:05.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.500+0000 7f1470c00640 1 -- 172.21.15.32:0/2689107923 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f146001c070 con 0x7f14740ac010 2024-04-17T10:02:05.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.500+0000 7f1470c00640 1 --2- 172.21.15.32:0/2689107923 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f14540683b0 0x7f145406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.500+0000 7f1470c00640 1 -- 172.21.15.32:0/2689107923 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f1460085ea0 con 0x7f14740ac010 2024-04-17T10:02:05.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.500+0000 7f1479000640 1 --2- 172.21.15.32:0/2689107923 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f14540683b0 0x7f145406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:05.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.500+0000 7f1479000640 1 --2- 172.21.15.32:0/2689107923 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f14540683b0 0x7f145406a870 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1468001060 tx=0x7f1468004840 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:05.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.504+0000 7f1470c00640 1 -- 172.21.15.32:0/2689107923 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f14740730e0 con 0x7f14740ac010 2024-04-17T10:02:05.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.672+0000 7f147ba00640 1 -- 172.21.15.32:0/2689107923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1474068a90 con 0x7f14740ac010 2024-04-17T10:02:05.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.672+0000 7f1470c00640 1 -- 172.21.15.32:0/2689107923 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f1460055e00 con 0x7f14740ac010 2024-04-17T10:02:05.670 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T10:02:05.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.672+0000 7f147ba00640 1 -- 172.21.15.32:0/2689107923 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f14540683b0 msgr2=0x7f145406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:05.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.672+0000 7f147ba00640 1 --2- 172.21.15.32:0/2689107923 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f14540683b0 0x7f145406a870 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1468001060 tx=0x7f1468004840 comp rx=0 tx=0).stop 2024-04-17T10:02:05.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.672+0000 7f147ba00640 1 -- 172.21.15.32:0/2689107923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14740ac010 msgr2=0x7f1474072790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:05.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.672+0000 7f147ba00640 1 --2- 172.21.15.32:0/2689107923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14740ac010 0x7f1474072790 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f146000e4b0 tx=0x7f146000e8b0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.672+0000 7f147ba00640 1 -- 172.21.15.32:0/2689107923 shutdown_connections 2024-04-17T10:02:05.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.672+0000 7f147ba00640 1 --2- 172.21.15.32:0/2689107923 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f14540683b0 0x7f145406a870 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.672+0000 7f147ba00640 1 --2- 172.21.15.32:0/2689107923 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1474072cd0 0x7f147412c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.672+0000 7f147ba00640 1 --2- 172.21.15.32:0/2689107923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14740ac010 0x7f1474072790 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.672+0000 7f147ba00640 1 --2- 172.21.15.32:0/2689107923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f145c0051d0 0x7f1474129ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.672+0000 7f147ba00640 1 -- 172.21.15.32:0/2689107923 >> 172.21.15.32:0/2689107923 conn(0x7f14740b2120 msgr2=0x7f147415a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:05.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.672+0000 7f147ba00640 1 -- 172.21.15.32:0/2689107923 shutdown_connections 2024-04-17T10:02:05.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.672+0000 7f147ba00640 1 -- 172.21.15.32:0/2689107923 wait complete. 2024-04-17T10:02:05.692 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:02:05.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.836+0000 7fbcf7a00640 1 Processor -- start 2024-04-17T10:02:05.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.836+0000 7fbcf7a00640 1 -- start start 2024-04-17T10:02:05.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.836+0000 7fbcf7a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbcf015fc60 0x7fbcf01602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.836+0000 7fbcf7a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbcf009bbc0 0x7fbcf0160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.836+0000 7fbcf7a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbcf00cdb20 0x7fbcf00cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.836+0000 7fbcf7a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fbcf0160040 con 0x7fbcf009bbc0 2024-04-17T10:02:05.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.836+0000 7fbcf7a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fbcf009ab20 con 0x7fbcf00cdb20 2024-04-17T10:02:05.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.836+0000 7fbcf7a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf009ae00 con 0x7fbcf015fc60 2024-04-17T10:02:05.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.836+0000 7fbce7e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbcf009bbc0 0x7fbcf0160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:05.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.836+0000 7fbce7e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbcf009bbc0 0x7fbcf0160820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41856/0 (socket says 172.21.15.32:41856) 2024-04-17T10:02:05.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.836+0000 7fbce7e00640 1 -- 172.21.15.32:0/1219293965 learned_addr learned my addr 172.21.15.32:0/1219293965 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:05.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.836+0000 7fbcf5000640 1 --2- 172.21.15.32:0/1219293965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbcf015fc60 0x7fbcf01602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:05.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.836+0000 7fbcf5a00640 1 --2- 172.21.15.32:0/1219293965 >> v2:172.21.15.144:3300/0 conn(0x7fbcf00cdb20 0x7fbcf00cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:05.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.836+0000 7fbce7e00640 1 -- 172.21.15.32:0/1219293965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbcf015fc60 msgr2=0x7fbcf01602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:05.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.836+0000 7fbce7e00640 1 --2- 172.21.15.32:0/1219293965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbcf015fc60 0x7fbcf01602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.836+0000 7fbce7e00640 1 -- 172.21.15.32:0/1219293965 >> v2:172.21.15.144:3300/0 conn(0x7fbcf00cdb20 msgr2=0x7fbcf00cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:05.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.836+0000 7fbce7e00640 1 --2- 172.21.15.32:0/1219293965 >> v2:172.21.15.144:3300/0 conn(0x7fbcf00cdb20 0x7fbcf00cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.836+0000 7fbce7e00640 1 -- 172.21.15.32:0/1219293965 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf009b0e0 con 0x7fbcf009bbc0 2024-04-17T10:02:05.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.840+0000 7fbce7e00640 1 --2- 172.21.15.32:0/1219293965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf009bbc0 0x7fbcf0160820 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fbcf006f8d0 tx=0x7fbcd801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1fac923476f191c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:05.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.840+0000 7fbcefe00640 1 -- 172.21.15.32:0/1219293965 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd80b0070 con 0x7fbcf009bbc0 2024-04-17T10:02:05.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.840+0000 7fbcefe00640 1 -- 172.21.15.32:0/1219293965 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcd80a4e50 con 0x7fbcf009bbc0 2024-04-17T10:02:05.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.840+0000 7fbcefe00640 1 -- 172.21.15.32:0/1219293965 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd80ab820 con 0x7fbcf009bbc0 2024-04-17T10:02:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.840+0000 7fbcf7a00640 1 -- 172.21.15.32:0/1219293965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf009bbc0 msgr2=0x7fbcf0160820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.840+0000 7fbcf7a00640 1 --2- 172.21.15.32:0/1219293965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf009bbc0 0x7fbcf0160820 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fbcf006f8d0 tx=0x7fbcd801c400 comp rx=0 tx=0).stop 2024-04-17T10:02:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.840+0000 7fbcf7a00640 1 -- 172.21.15.32:0/1219293965 shutdown_connections 2024-04-17T10:02:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.840+0000 7fbcf7a00640 1 --2- 172.21.15.32:0/1219293965 >> v2:172.21.15.144:3300/0 conn(0x7fbcf00cdb20 0x7fbcf00cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.840+0000 7fbcf7a00640 1 --2- 172.21.15.32:0/1219293965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf009bbc0 0x7fbcf0160820 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.840+0000 7fbcf7a00640 1 --2- 172.21.15.32:0/1219293965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbcf015fc60 0x7fbcf01602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.840+0000 7fbcf7a00640 1 -- 172.21.15.32:0/1219293965 >> 172.21.15.32:0/1219293965 conn(0x7fbcf00b3ff0 msgr2=0x7fbcf00b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.840+0000 7fbcf7a00640 1 -- 172.21.15.32:0/1219293965 shutdown_connections 2024-04-17T10:02:05.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.840+0000 7fbcf7a00640 1 -- 172.21.15.32:0/1219293965 wait complete. 2024-04-17T10:02:05.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbcf7a00640 1 Processor -- start 2024-04-17T10:02:05.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbcf7a00640 1 -- start start 2024-04-17T10:02:05.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbcf7a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf009bbc0 0x7fbcf0154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbcf7a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcf00cdb20 0x7fbcf0149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbcf7a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbcf015fc60 0x7fbcf00cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbcf7a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf00d0c70 con 0x7fbcf009bbc0 2024-04-17T10:02:05.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbcf7a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf00d0af0 con 0x7fbcf00cdb20 2024-04-17T10:02:05.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbcf7a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf00d0df0 con 0x7fbcf015fc60 2024-04-17T10:02:05.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbcf5000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf009bbc0 0x7fbcf0154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:05.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbcf5000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf009bbc0 0x7fbcf0154350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41868/0 (socket says 172.21.15.32:41868) 2024-04-17T10:02:05.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbcf5000640 1 -- 172.21.15.32:0/859865476 learned_addr learned my addr 172.21.15.32:0/859865476 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:05.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbce7e00640 1 --2- 172.21.15.32:0/859865476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcf00cdb20 0x7fbcf0149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:05.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbcf5a00640 1 --2- 172.21.15.32:0/859865476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbcf015fc60 0x7fbcf00cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:05.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbcf5000640 1 -- 172.21.15.32:0/859865476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbcf015fc60 msgr2=0x7fbcf00cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:05.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbcf5000640 1 --2- 172.21.15.32:0/859865476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbcf015fc60 0x7fbcf00cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbcf5000640 1 -- 172.21.15.32:0/859865476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcf00cdb20 msgr2=0x7fbcf0149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:05.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbcf5000640 1 --2- 172.21.15.32:0/859865476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcf00cdb20 0x7fbcf0149890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:05.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbcf5000640 1 -- 172.21.15.32:0/859865476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf006cd00 con 0x7fbcf009bbc0 2024-04-17T10:02:05.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbcf5000640 1 --2- 172.21.15.32:0/859865476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf009bbc0 0x7fbcf0154350 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fbce000e870 tx=0x7fbce000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:05.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbced600640 1 -- 172.21.15.32:0/859865476 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0017070 con 0x7fbcf009bbc0 2024-04-17T10:02:05.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbced600640 1 -- 172.21.15.32:0/859865476 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbce000c940 con 0x7fbcf009bbc0 2024-04-17T10:02:05.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbced600640 1 -- 172.21.15.32:0/859865476 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce000ccf0 con 0x7fbcf009bbc0 2024-04-17T10:02:05.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbcf7a00640 1 -- 172.21.15.32:0/859865476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf006cfe0 con 0x7fbcf009bbc0 2024-04-17T10:02:05.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.844+0000 7fbcf7a00640 1 -- 172.21.15.32:0/859865476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf006d5a0 con 0x7fbcf009bbc0 2024-04-17T10:02:05.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.848+0000 7fbcf7a00640 1 -- 172.21.15.32:0/859865476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf0068a90 con 0x7fbcf009bbc0 2024-04-17T10:02:05.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.848+0000 7fbced600640 1 -- 172.21.15.32:0/859865476 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbce001a070 con 0x7fbcf009bbc0 2024-04-17T10:02:05.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.848+0000 7fbced600640 1 --2- 172.21.15.32:0/859865476 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbccc0683b0 0x7fbccc06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:05.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.848+0000 7fbced600640 1 -- 172.21.15.32:0/859865476 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fbce0086c40 con 0x7fbcf009bbc0 2024-04-17T10:02:05.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.848+0000 7fbce7e00640 1 --2- 172.21.15.32:0/859865476 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbccc0683b0 0x7fbccc06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:05.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.848+0000 7fbce7e00640 1 --2- 172.21.15.32:0/859865476 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbccc0683b0 0x7fbccc06a870 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80ad000 tx=0x7fbcd80a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:05.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:05.852+0000 7fbced600640 1 -- 172.21.15.32:0/859865476 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fbce0057160 con 0x7fbcf009bbc0 2024-04-17T10:02:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:06.032+0000 7fbcf7a00640 1 -- 172.21.15.32:0/859865476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbcf00cec80 con 0x7fbcf009bbc0 2024-04-17T10:02:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:06.032+0000 7fbced600640 1 -- 172.21.15.32:0/859865476 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v97) ==== 80+0+0 (secure 0 0 0) 0x7fbce0056a60 con 0x7fbcf009bbc0 2024-04-17T10:02:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T10:02:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:06.036+0000 7fbcf7a00640 1 -- 172.21.15.32:0/859865476 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbccc0683b0 msgr2=0x7fbccc06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:06.036+0000 7fbcf7a00640 1 --2- 172.21.15.32:0/859865476 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbccc0683b0 0x7fbccc06a870 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80ad000 tx=0x7fbcd80a6000 comp rx=0 tx=0).stop 2024-04-17T10:02:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:06.036+0000 7fbcf7a00640 1 -- 172.21.15.32:0/859865476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf009bbc0 msgr2=0x7fbcf0154350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:06.036+0000 7fbcf7a00640 1 --2- 172.21.15.32:0/859865476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf009bbc0 0x7fbcf0154350 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fbce000e870 tx=0x7fbce000ec70 comp rx=0 tx=0).stop 2024-04-17T10:02:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:06.036+0000 7fbcf7a00640 1 -- 172.21.15.32:0/859865476 shutdown_connections 2024-04-17T10:02:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:06.036+0000 7fbcf7a00640 1 --2- 172.21.15.32:0/859865476 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbccc0683b0 0x7fbccc06a870 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:06.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:06.036+0000 7fbcf7a00640 1 --2- 172.21.15.32:0/859865476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbcf015fc60 0x7fbcf00cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:06.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:06.036+0000 7fbcf7a00640 1 --2- 172.21.15.32:0/859865476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcf00cdb20 0x7fbcf0149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:06.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:06.036+0000 7fbcf7a00640 1 --2- 172.21.15.32:0/859865476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcf009bbc0 0x7fbcf0154350 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:06.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:06.036+0000 7fbcf7a00640 1 -- 172.21.15.32:0/859865476 >> 172.21.15.32:0/859865476 conn(0x7fbcf00b3ff0 msgr2=0x7fbcf00b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:06.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:06.036+0000 7fbcf7a00640 1 -- 172.21.15.32:0/859865476 shutdown_connections 2024-04-17T10:02:06.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:06.036+0000 7fbcf7a00640 1 -- 172.21.15.32:0/859865476 wait complete. 2024-04-17T10:02:06.053 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2024-04-17T10:02:06.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:02:06.053 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T10:02:06.077 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[48965]: fuse finished with error 0 and tester_r 0 2024-04-17T10:02:06.082 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:02:12.184 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T10:02:12.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:02:12.185 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T10:02:12.196 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:02:12.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.392+0000 7fc2c4200640 1 Processor -- start 2024-04-17T10:02:12.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.392+0000 7fc2c4200640 1 -- start start 2024-04-17T10:02:12.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.396+0000 7fc2c4200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2bc1481e0 0x7fc2bc071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.396+0000 7fc2c4200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2bc06ef70 con 0x7fc2bc072330 2024-04-17T10:02:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.396+0000 7fc2c4200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2bc0aed20 con 0x7fc2bc144f20 2024-04-17T10:02:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.396+0000 7fc2c4200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc2bc0727a0 con 0x7fc2bc1481e0 2024-04-17T10:02:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.396+0000 7fc2c0e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2bc1481e0 0x7fc2bc071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.396+0000 7fc2c2200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc2bc072330 0x7fc2bc0b3490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57456/0 (socket says 172.21.15.32:57456) 2024-04-17T10:02:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.396+0000 7fc2c2200640 1 -- 172.21.15.32:0/214827565 learned_addr learned my addr 172.21.15.32:0/214827565 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:02:12.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.396+0000 7fc2c0e00640 1 -- 172.21.15.32:0/214827565 >> v1:172.21.15.144:6789/0 conn(0x7fc2bc144f20 legacy=0x7fc2bc06e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:12.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.396+0000 7fc2bbe00640 1 -- 172.21.15.32:0/214827565 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3844385422 0 0) 0x7fc2bc06ef70 con 0x7fc2bc072330 2024-04-17T10:02:12.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.396+0000 7fc2c0e00640 1 -- 172.21.15.32:0/214827565 >> v1:172.21.15.32:6789/0 conn(0x7fc2bc072330 legacy=0x7fc2bc0b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:12.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.396+0000 7fc2c0e00640 1 -- 172.21.15.32:0/214827565 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2bc0af000 con 0x7fc2bc1481e0 2024-04-17T10:02:12.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.396+0000 7fc2c0e00640 1 --2- 172.21.15.32:0/214827565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2bc1481e0 0x7fc2bc071df0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fc2a80097c0 tx=0x7fc2a801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30877e795d0e440 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:12.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.400+0000 7fc2bbe00640 1 -- 172.21.15.32:0/214827565 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a80b1070 con 0x7fc2bc1481e0 2024-04-17T10:02:12.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.400+0000 7fc2bbe00640 1 -- 172.21.15.32:0/214827565 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a80ac5f0 con 0x7fc2bc1481e0 2024-04-17T10:02:12.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.400+0000 7fc2bbe00640 1 -- 172.21.15.32:0/214827565 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a80ac9c0 con 0x7fc2bc1481e0 2024-04-17T10:02:12.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.400+0000 7fc2c4200640 1 -- 172.21.15.32:0/214827565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2bc1481e0 msgr2=0x7fc2bc071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:12.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.400+0000 7fc2c4200640 1 --2- 172.21.15.32:0/214827565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2bc1481e0 0x7fc2bc071df0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fc2a80097c0 tx=0x7fc2a801c4a0 comp rx=0 tx=0).stop 2024-04-17T10:02:12.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.400+0000 7fc2c4200640 1 -- 172.21.15.32:0/214827565 shutdown_connections 2024-04-17T10:02:12.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.400+0000 7fc2c4200640 1 --2- 172.21.15.32:0/214827565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2bc1481e0 0x7fc2bc071df0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.400+0000 7fc2c4200640 1 -- 172.21.15.32:0/214827565 >> 172.21.15.32:0/214827565 conn(0x7fc2bc06ba30 msgr2=0x7fc2bc06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.400+0000 7fc2c4200640 1 -- 172.21.15.32:0/214827565 shutdown_connections 2024-04-17T10:02:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.400+0000 7fc2c4200640 1 -- 172.21.15.32:0/214827565 wait complete. 2024-04-17T10:02:12.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.400+0000 7fc2c4200640 1 Processor -- start 2024-04-17T10:02:12.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.400+0000 7fc2c4200640 1 -- start start 2024-04-17T10:02:12.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.400+0000 7fc2c4200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2bc072330 0x7fc2bc0b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:12.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.400+0000 7fc2c4200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2bc144f20 0x7fc2bc136eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:12.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.400+0000 7fc2c4200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2bc1481e0 0x7fc2bc1569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:12.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.400+0000 7fc2c4200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc0afdc0 con 0x7fc2bc144f20 2024-04-17T10:02:12.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.400+0000 7fc2c4200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc16c5e0 con 0x7fc2bc1481e0 2024-04-17T10:02:12.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.400+0000 7fc2c4200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc2bc0b2c30 con 0x7fc2bc072330 2024-04-17T10:02:12.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.400+0000 7fc2c0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2bc144f20 0x7fc2bc136eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:12.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.400+0000 7fc2c0e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2bc144f20 0x7fc2bc136eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36468/0 (socket says 172.21.15.32:36468) 2024-04-17T10:02:12.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.400+0000 7fc2c0e00640 1 -- 172.21.15.32:0/2006968459 learned_addr learned my addr 172.21.15.32:0/2006968459 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:12.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.400+0000 7fc2c1800640 1 --2- 172.21.15.32:0/2006968459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2bc072330 0x7fc2bc0b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:12.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.404+0000 7fc2c2200640 1 --2- 172.21.15.32:0/2006968459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2bc1481e0 0x7fc2bc1569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:12.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.404+0000 7fc2c0e00640 1 -- 172.21.15.32:0/2006968459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2bc072330 msgr2=0x7fc2bc0b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:12.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.404+0000 7fc2c0e00640 1 --2- 172.21.15.32:0/2006968459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2bc072330 0x7fc2bc0b00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:12.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.404+0000 7fc2c0e00640 1 -- 172.21.15.32:0/2006968459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2bc1481e0 msgr2=0x7fc2bc1569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:12.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.404+0000 7fc2c0e00640 1 --2- 172.21.15.32:0/2006968459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2bc1481e0 0x7fc2bc1569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:12.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.404+0000 7fc2c0e00640 1 -- 172.21.15.32:0/2006968459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2bc0cf640 con 0x7fc2bc144f20 2024-04-17T10:02:12.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.404+0000 7fc2c0e00640 1 --2- 172.21.15.32:0/2006968459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2bc144f20 0x7fc2bc136eb0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fc2a80097c0 tx=0x7fc2a801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:12.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.404+0000 7fc2b9600640 1 -- 172.21.15.32:0/2006968459 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a80bd070 con 0x7fc2bc144f20 2024-04-17T10:02:12.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.404+0000 7fc2b9600640 1 -- 172.21.15.32:0/2006968459 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a80b2040 con 0x7fc2bc144f20 2024-04-17T10:02:12.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.404+0000 7fc2c4200640 1 -- 172.21.15.32:0/2006968459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2bc139ee0 con 0x7fc2bc144f20 2024-04-17T10:02:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.404+0000 7fc2b9600640 1 -- 172.21.15.32:0/2006968459 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a80b1050 con 0x7fc2bc144f20 2024-04-17T10:02:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.404+0000 7fc2c4200640 1 -- 172.21.15.32:0/2006968459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2bc1373f0 con 0x7fc2bc144f20 2024-04-17T10:02:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.404+0000 7fc2c4200640 1 -- 172.21.15.32:0/2006968459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2bc068a90 con 0x7fc2bc144f20 2024-04-17T10:02:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.404+0000 7fc2b9600640 1 -- 172.21.15.32:0/2006968459 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc2a80c0070 con 0x7fc2bc144f20 2024-04-17T10:02:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.404+0000 7fc2b9600640 1 --2- 172.21.15.32:0/2006968459 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc29c068380 0x7fc29c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.404+0000 7fc2b9600640 1 -- 172.21.15.32:0/2006968459 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fc2a81204f0 con 0x7fc2bc144f20 2024-04-17T10:02:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.404+0000 7fc2c1800640 1 --2- 172.21.15.32:0/2006968459 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc29c068380 0x7fc29c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:12.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.408+0000 7fc2c1800640 1 --2- 172.21.15.32:0/2006968459 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc29c068380 0x7fc29c06a840 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0000f70 tx=0x7fc2b0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:12.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.412+0000 7fc2b9600640 1 -- 172.21.15.32:0/2006968459 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc2a80a9070 con 0x7fc2bc144f20 2024-04-17T10:02:12.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.608+0000 7fc2c4200640 1 -- 172.21.15.32:0/2006968459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2bc0d01c0 con 0x7fc2bc144f20 2024-04-17T10:02:12.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.608+0000 7fc2b9600640 1 -- 172.21.15.32:0/2006968459 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5207 (secure 0 0 0) 0x7fc2a8016090 con 0x7fc2bc144f20 2024-04-17T10:02:12.606 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:02:12.606 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":81,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2635109588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2635109588},{"type":"v1","addr":"172.21.15.144:6835","nonce":2635109588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7789,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3258455939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3258455939},{"type":"v1","addr":"172.21.15.144:6834","nonce":3258455939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":8574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3390468332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3390468332},{"type":"v1","addr":"172.21.15.32:6836","nonce":3390468332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:01:40.557208+0000","modified":"2024-04-17T10:01:43.570474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8577":{"gid":8577,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/1235126741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1235126741},{"type":"v1","addr":"172.21.15.32:6837","nonce":1235126741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8577,"qdb_cluster":[8577]},"id":7}]} 2024-04-17T10:02:12.607 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 81 2024-04-17T10:02:12.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.608+0000 7fc2c4200640 1 -- 172.21.15.32:0/2006968459 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc29c068380 msgr2=0x7fc29c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:12.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.608+0000 7fc2c4200640 1 --2- 172.21.15.32:0/2006968459 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc29c068380 0x7fc29c06a840 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0000f70 tx=0x7fc2b0009040 comp rx=0 tx=0).stop 2024-04-17T10:02:12.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.608+0000 7fc2c4200640 1 -- 172.21.15.32:0/2006968459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2bc144f20 msgr2=0x7fc2bc136eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:12.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.608+0000 7fc2c4200640 1 --2- 172.21.15.32:0/2006968459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2bc144f20 0x7fc2bc136eb0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fc2a80097c0 tx=0x7fc2a801ace0 comp rx=0 tx=0).stop 2024-04-17T10:02:12.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.608+0000 7fc2c4200640 1 -- 172.21.15.32:0/2006968459 shutdown_connections 2024-04-17T10:02:12.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.612+0000 7fc2c4200640 1 --2- 172.21.15.32:0/2006968459 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc29c068380 0x7fc29c06a840 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:12.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.612+0000 7fc2c4200640 1 --2- 172.21.15.32:0/2006968459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2bc1481e0 0x7fc2bc1569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:12.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.612+0000 7fc2c4200640 1 --2- 172.21.15.32:0/2006968459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2bc144f20 0x7fc2bc136eb0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:12.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.612+0000 7fc2c4200640 1 --2- 172.21.15.32:0/2006968459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2bc072330 0x7fc2bc0b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:12.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.612+0000 7fc2c4200640 1 -- 172.21.15.32:0/2006968459 >> 172.21.15.32:0/2006968459 conn(0x7fc2bc06ba30 msgr2=0x7fc2bc06de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:12.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.612+0000 7fc2c4200640 1 -- 172.21.15.32:0/2006968459 shutdown_connections 2024-04-17T10:02:12.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:12.612+0000 7fc2c4200640 1 -- 172.21.15.32:0/2006968459 wait complete. 2024-04-17T10:02:12.628 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-17T10:02:12.800 INFO:teuthology.orchestra.run.smithi032.stdout:{"message":"","return_code":0} 2024-04-17T10:02:12.801 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-17T10:02:12.801 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:02:12.801 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:02:12.810 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:02:12.810 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi032.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-04-17T10:02:12.810 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T10:02:12.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:02:12.810 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T10:02:12.863 INFO:teuthology.orchestra.run.smithi032.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-17T10:02:12.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:02:12.864 DEBUG:teuthology.orchestra.run.smithi032:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-17T10:02:12.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:02:12.910 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-17T10:02:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.056+0000 7fdebc000640 1 Processor -- start 2024-04-17T10:02:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.056+0000 7fdebc000640 1 -- start start 2024-04-17T10:02:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.060+0000 7fdebc000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdeb415e9d0 0x7fdeb40b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.060+0000 7fdebc000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdeb40b2800 0x7fdeb406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.060+0000 7fdebc000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdeb40b4300 con 0x7fdeb406bf70 2024-04-17T10:02:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.060+0000 7fdebc000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fdeb40b2100 con 0x7fdeb415e9d0 2024-04-17T10:02:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.060+0000 7fdebc000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fdeb40b45a0 con 0x7fdeb40b2800 2024-04-17T10:02:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.060+0000 7fdeb9600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdeb415e9d0 0x7fdeb40b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.060+0000 7fdeb9600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdeb415e9d0 0x7fdeb40b61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36480/0 (socket says 172.21.15.32:36480) 2024-04-17T10:02:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.060+0000 7fdeb9600640 1 -- 172.21.15.32:0/140667887 learned_addr learned my addr 172.21.15.32:0/140667887 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.060+0000 7fdeb8c00640 1 --2- 172.21.15.32:0/140667887 >> v2:172.21.15.144:3300/0 conn(0x7fdeb40b2800 0x7fdeb406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.060+0000 7fdeb9600640 1 -- 172.21.15.32:0/140667887 >> v2:172.21.15.144:3300/0 conn(0x7fdeb40b2800 msgr2=0x7fdeb406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.060+0000 7fdeb9600640 1 --2- 172.21.15.32:0/140667887 >> v2:172.21.15.144:3300/0 conn(0x7fdeb40b2800 0x7fdeb406ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.060+0000 7fdeb9600640 1 -- 172.21.15.32:0/140667887 >> v1:172.21.15.144:6789/0 conn(0x7fdeb406bf70 legacy=0x7fdeb40aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.060+0000 7fdeb9600640 1 -- 172.21.15.32:0/140667887 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb40b4880 con 0x7fdeb415e9d0 2024-04-17T10:02:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.064+0000 7fdeb9600640 1 --2- 172.21.15.32:0/140667887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb415e9d0 0x7fdeb40b61c0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fdeb406cb10 tx=0x7fdea801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1f476a229682c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.064+0000 7fdeb3e00640 1 -- 172.21.15.32:0/140667887 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea80b0070 con 0x7fdeb415e9d0 2024-04-17T10:02:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.064+0000 7fdeb3e00640 1 -- 172.21.15.32:0/140667887 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea80ab600 con 0x7fdeb415e9d0 2024-04-17T10:02:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.064+0000 7fdeb3e00640 1 -- 172.21.15.32:0/140667887 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea80ab9d0 con 0x7fdeb415e9d0 2024-04-17T10:02:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.064+0000 7fdebc000640 1 -- 172.21.15.32:0/140667887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb415e9d0 msgr2=0x7fdeb40b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.064+0000 7fdebc000640 1 --2- 172.21.15.32:0/140667887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb415e9d0 0x7fdeb40b61c0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fdeb406cb10 tx=0x7fdea801c4c0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.064+0000 7fdebc000640 1 -- 172.21.15.32:0/140667887 shutdown_connections 2024-04-17T10:02:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.064+0000 7fdebc000640 1 --2- 172.21.15.32:0/140667887 >> v2:172.21.15.144:3300/0 conn(0x7fdeb40b2800 0x7fdeb406ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.064+0000 7fdebc000640 1 --2- 172.21.15.32:0/140667887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb415e9d0 0x7fdeb40b61c0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.064+0000 7fdebc000640 1 -- 172.21.15.32:0/140667887 >> 172.21.15.32:0/140667887 conn(0x7fdeb40b6860 msgr2=0x7fdeb40b6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.064+0000 7fdebc000640 1 -- 172.21.15.32:0/140667887 shutdown_connections 2024-04-17T10:02:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.064+0000 7fdebc000640 1 -- 172.21.15.32:0/140667887 wait complete. 2024-04-17T10:02:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.064+0000 7fdebc000640 1 Processor -- start 2024-04-17T10:02:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.064+0000 7fdebc000640 1 -- start start 2024-04-17T10:02:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.064+0000 7fdebc000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb406bf70 0x7fdeb4160810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.064+0000 7fdebc000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdeb40b2800 0x7fdeb409dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.064+0000 7fdebc000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdeb415e9d0 0x7fdeb409e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.064+0000 7fdebc000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb40b2be0 con 0x7fdeb406bf70 2024-04-17T10:02:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.064+0000 7fdebc000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb40b0570 con 0x7fdeb40b2800 2024-04-17T10:02:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.064+0000 7fdebc000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdeb40b2d60 con 0x7fdeb415e9d0 2024-04-17T10:02:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.068+0000 7fdeb9600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb406bf70 0x7fdeb4160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.068+0000 7fdeb9600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb406bf70 0x7fdeb4160810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36490/0 (socket says 172.21.15.32:36490) 2024-04-17T10:02:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.068+0000 7fdeb9600640 1 -- 172.21.15.32:0/550314895 learned_addr learned my addr 172.21.15.32:0/550314895 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.068+0000 7fdeb8c00640 1 --2- 172.21.15.32:0/550314895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdeb40b2800 0x7fdeb409dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.068+0000 7fdeba000640 1 --2- 172.21.15.32:0/550314895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdeb415e9d0 0x7fdeb409e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.068+0000 7fdeb9600640 1 -- 172.21.15.32:0/550314895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdeb415e9d0 msgr2=0x7fdeb409e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.068+0000 7fdeb9600640 1 --2- 172.21.15.32:0/550314895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdeb415e9d0 0x7fdeb409e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.068+0000 7fdeb9600640 1 -- 172.21.15.32:0/550314895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdeb40b2800 msgr2=0x7fdeb409dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.068+0000 7fdeb9600640 1 --2- 172.21.15.32:0/550314895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdeb40b2800 0x7fdeb409dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.068+0000 7fdeb9600640 1 -- 172.21.15.32:0/550314895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb415dd40 con 0x7fdeb406bf70 2024-04-17T10:02:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.068+0000 7fdeb9600640 1 --2- 172.21.15.32:0/550314895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb406bf70 0x7fdeb4160810 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fdea80ae040 tx=0x7fdea801ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.068+0000 7fdeb1600640 1 -- 172.21.15.32:0/550314895 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea80ba020 con 0x7fdeb406bf70 2024-04-17T10:02:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.068+0000 7fdeb1600640 1 -- 172.21.15.32:0/550314895 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea80b6420 con 0x7fdeb406bf70 2024-04-17T10:02:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.068+0000 7fdeb1600640 1 -- 172.21.15.32:0/550314895 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea80b0040 con 0x7fdeb406bf70 2024-04-17T10:02:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.068+0000 7fdebc000640 1 -- 172.21.15.32:0/550314895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb409d150 con 0x7fdeb406bf70 2024-04-17T10:02:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.068+0000 7fdebc000640 1 -- 172.21.15.32:0/550314895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb409d3f0 con 0x7fdeb406bf70 2024-04-17T10:02:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.068+0000 7fdeb1600640 1 -- 172.21.15.32:0/550314895 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdea80bd020 con 0x7fdeb406bf70 2024-04-17T10:02:13.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdeb1600640 1 --2- 172.21.15.32:0/550314895 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fde94068380 0x7fde9406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdeb1600640 1 -- 172.21.15.32:0/550314895 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fdea8120b80 con 0x7fdeb406bf70 2024-04-17T10:02:13.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdeb8c00640 1 --2- 172.21.15.32:0/550314895 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fde94068380 0x7fde9406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdebf4a0000 1 Processor -- start 2024-04-17T10:02:13.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdebf4a0000 1 -- start start 2024-04-17T10:02:13.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdeb8c00640 1 --2- 172.21.15.32:0/550314895 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fde94068380 0x7fde9406a840 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fdeb415b8a0 tx=0x7fdea0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdebf4a0000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccfdcc9700 0x55ccfdcc9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdebf4a0000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ccfdcca020 0x55ccfdc7c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdebf4a0000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdc7ca10 0x55ccfdc7cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdebf4a0000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ccfdace5f0 con 0x55ccfdc7ca10 2024-04-17T10:02:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdebf4a0000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ccfda877a0 con 0x55ccfdcc9700 2024-04-17T10:02:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdebf4a0000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ccfdaca5b0 con 0x55ccfdcca020 2024-04-17T10:02:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdeba000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdc7ca10 0x55ccfdc7cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdeba000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdc7ca10 0x55ccfdc7cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36502/0 (socket says 172.21.15.32:36502) 2024-04-17T10:02:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdeba000640 1 -- 172.21.15.32:0/516082891 learned_addr learned my addr 172.21.15.32:0/516082891 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdeb9600640 1 --2- 172.21.15.32:0/516082891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccfdcc9700 0x55ccfdcc9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdeb8c00640 1 --2- 172.21.15.32:0/516082891 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ccfdcca020 0x55ccfdc7c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdeba000640 1 -- 172.21.15.32:0/516082891 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ccfdcca020 msgr2=0x55ccfdc7c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:13.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdeba000640 1 --2- 172.21.15.32:0/516082891 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ccfdcca020 0x55ccfdc7c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdeba000640 1 -- 172.21.15.32:0/516082891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccfdcc9700 msgr2=0x55ccfdcc9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:13.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdeba000640 1 --2- 172.21.15.32:0/516082891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccfdcc9700 0x55ccfdcc9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdeba000640 1 -- 172.21.15.32:0/516082891 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccfdb756e0 con 0x55ccfdc7ca10 2024-04-17T10:02:13.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdeba000640 1 --2- 172.21.15.32:0/516082891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdc7ca10 0x55ccfdc7cdf0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fdeac005420 tx=0x7fdeac0058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abc626400cff6c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:13.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdea6000640 1 -- 172.21.15.32:0/516082891 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac006d70 con 0x55ccfdc7ca10 2024-04-17T10:02:13.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdea6000640 1 -- 172.21.15.32:0/516082891 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeac018070 con 0x55ccfdc7ca10 2024-04-17T10:02:13.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdea6000640 1 -- 172.21.15.32:0/516082891 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac00eb00 con 0x55ccfdc7ca10 2024-04-17T10:02:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdebf4a0000 1 -- 172.21.15.32:0/516082891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdc7ca10 msgr2=0x55ccfdc7cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdebf4a0000 1 --2- 172.21.15.32:0/516082891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdc7ca10 0x55ccfdc7cdf0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fdeac005420 tx=0x7fdeac0058e0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdebf4a0000 1 -- 172.21.15.32:0/516082891 shutdown_connections 2024-04-17T10:02:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdebf4a0000 1 --2- 172.21.15.32:0/516082891 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ccfdcca020 0x55ccfdc7c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdebf4a0000 1 --2- 172.21.15.32:0/516082891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdc7ca10 0x55ccfdc7cdf0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdebf4a0000 1 --2- 172.21.15.32:0/516082891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccfdcc9700 0x55ccfdcc9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.072+0000 7fdebf4a0000 1 -- 172.21.15.32:0/516082891 >> 172.21.15.32:0/516082891 conn(0x55ccfdcaa290 msgr2=0x55ccfdcac680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdebf4a0000 1 -- 172.21.15.32:0/516082891 shutdown_connections 2024-04-17T10:02:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdebf4a0000 1 -- 172.21.15.32:0/516082891 wait complete. 2024-04-17T10:02:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdebf4a0000 1 -- start start 2024-04-17T10:02:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdebf4a0000 1 Processor -- start 2024-04-17T10:02:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdebf4a0000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccfdc7ca10 0x55ccfdc81650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdebf4a0000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ccfdcc9700 0x55ccfdc81b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdebf4a0000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdcca020 0x55ccfdc820d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdebf4a0000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ccfdad9520 con 0x55ccfdcca020 2024-04-17T10:02:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdebf4a0000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ccfda66ea0 con 0x55ccfdc7ca10 2024-04-17T10:02:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdebf4a0000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ccfda8d500 con 0x55ccfdcc9700 2024-04-17T10:02:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdebf4a0000 10 client.0 _ll_register_callbacks cb 0x55ccfdc6a170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:02:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdeba000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdcca020 0x55ccfdc820d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdeba000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdcca020 0x55ccfdc820d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36518/0 (socket says 172.21.15.32:36518) 2024-04-17T10:02:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdeba000640 1 -- 172.21.15.32:0/1239841079 learned_addr learned my addr 172.21.15.32:0/1239841079 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdeb9600640 1 --2- 172.21.15.32:0/1239841079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccfdc7ca10 0x55ccfdc81650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdeb8c00640 1 --2- 172.21.15.32:0/1239841079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ccfdcc9700 0x55ccfdc81b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdeba000640 1 -- 172.21.15.32:0/1239841079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ccfdcc9700 msgr2=0x55ccfdc81b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdeba000640 1 --2- 172.21.15.32:0/1239841079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ccfdcc9700 0x55ccfdc81b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdeba000640 1 -- 172.21.15.32:0/1239841079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccfdc7ca10 msgr2=0x55ccfdc81650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdeba000640 1 --2- 172.21.15.32:0/1239841079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccfdc7ca10 0x55ccfdc81650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdeba000640 1 -- 172.21.15.32:0/1239841079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccfdaa7540 con 0x55ccfdcca020 2024-04-17T10:02:13.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdeba000640 1 --2- 172.21.15.32:0/1239841079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdcca020 0x55ccfdc820d0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fdeac005920 tx=0x7fdeac003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0239824a1287082 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:13.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fde9aa00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:02:13.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fde9aa00640 1 -- 172.21.15.32:0/1239841079 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac00ec70 con 0x55ccfdcca020 2024-04-17T10:02:13.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fde9aa00640 1 -- 172.21.15.32:0/1239841079 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fdeac00ee10 con 0x55ccfdcca020 2024-04-17T10:02:13.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdebf4a0000 1 -- 172.21.15.32:0/1239841079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeac018070 con 0x55ccfdcca020 2024-04-17T10:02:13.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fde9aa00640 1 -- 172.21.15.32:0/1239841079 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fdeac013420 con 0x55ccfdcca020 2024-04-17T10:02:13.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdebf4a0000 10 client.8898 fetch_fsmap learned FSMap version 81 2024-04-17T10:02:13.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdebf4a0000 1 -- 172.21.15.32:0/1239841079 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x55ccfdb12ba0 con 0x55ccfdcca020 2024-04-17T10:02:13.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fde9aa00640 1 -- 172.21.15.32:0/1239841079 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeac018070 con 0x55ccfdcca020 2024-04-17T10:02:13.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fde9aa00640 1 -- 172.21.15.32:0/1239841079 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 81) ==== 3495+0+0 (secure 0 0 0) 0x7fdeac02e050 con 0x55ccfdcca020 2024-04-17T10:02:13.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdebf4a0000 10 client.8898 fetch_fsmap finished waiting for FSMap version 81 2024-04-17T10:02:13.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.076+0000 7fdebf4a0000 10 client.8898 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-17T10:02:13.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.080+0000 7fdebf4a0000 20 client.8898 populate_metadata read hostname 'smithi032' 2024-04-17T10:02:13.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.080+0000 7fdebf4a0000 1 --2- 172.21.15.32:0/1239841079 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x55ccfdcd8790 0x55ccfdcdab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.080+0000 7fdebf4a0000 4 client.8898 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:02:13.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.080+0000 7fdebf4a0000 1 -- 172.21.15.32:0/1239841079 --> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccfdb87450 con 0x55ccfdcd8790 2024-04-17T10:02:13.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.080+0000 7fdeb9600640 1 --2- 172.21.15.32:0/1239841079 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x55ccfdcd8790 0x55ccfdcdab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.080+0000 7fdeb9600640 1 --2- 172.21.15.32:0/1239841079 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x55ccfdcd8790 0x55ccfdcdab80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb5dee6e9a2d3b01 server_cookie=f86d1eefa664daa in_seq=0 out_seq=0 2024-04-17T10:02:13.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.080+0000 7fde9aa00640 10 client.8898 ms_handle_connect on v2:172.21.15.32:6835/1235126741 2024-04-17T10:02:13.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.080+0000 7fde9aa00640 1 -- 172.21.15.32:0/1239841079 <== mds.0 v2:172.21.15.32:6835/1235126741 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ccfdb87450 con 0x55ccfdcd8790 2024-04-17T10:02:13.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.080+0000 7fde9aa00640 10 client.8898 handle_command_reply: tid=0 2024-04-17T10:02:13.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.080+0000 7fdebf4a0000 1 client.8898 shutdown 2024-04-17T10:02:13.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.080+0000 7fdebf4a0000 1 -- 172.21.15.32:0/1239841079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdcca020 msgr2=0x55ccfdc820d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:13.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.080+0000 7fdebf4a0000 1 --2- 172.21.15.32:0/1239841079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdcca020 0x55ccfdc820d0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fdeac005920 tx=0x7fdeac003a80 comp rx=0 tx=0).stop 2024-04-17T10:02:13.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.080+0000 7fdebf4a0000 1 -- 172.21.15.32:0/1239841079 shutdown_connections 2024-04-17T10:02:13.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.080+0000 7fdebf4a0000 1 --2- 172.21.15.32:0/1239841079 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x55ccfdcd8790 0x55ccfdcdab80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.080+0000 7fdebf4a0000 1 --2- 172.21.15.32:0/1239841079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ccfdcc9700 0x55ccfdc81b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.080+0000 7fdebf4a0000 1 --2- 172.21.15.32:0/1239841079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdcca020 0x55ccfdc820d0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.080+0000 7fdebf4a0000 1 --2- 172.21.15.32:0/1239841079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccfdc7ca10 0x55ccfdc81650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.080+0000 7fde9aa00640 0 client.8898 ms_handle_reset on v2:172.21.15.32:6835/1235126741 2024-04-17T10:02:13.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.080+0000 7fdebf4a0000 1 -- 172.21.15.32:0/1239841079 >> 172.21.15.32:0/1239841079 conn(0x55ccfdcaa290 msgr2=0x55ccfdcac680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:13.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.080+0000 7fdebf4a0000 1 -- 172.21.15.32:0/1239841079 shutdown_connections 2024-04-17T10:02:13.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.080+0000 7fdebf4a0000 1 -- 172.21.15.32:0/1239841079 wait complete. 2024-04-17T10:02:13.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.080+0000 7fdebf4a0000 20 client.8898 trim_cache size 0 max 16384 2024-04-17T10:02:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.084+0000 7f8c0b600640 1 Processor -- start 2024-04-17T10:02:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.084+0000 7f8c0b600640 1 -- start start 2024-04-17T10:02:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.084+0000 7f8c0b600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8c04162ed0 0x7f8c04169af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.084+0000 7f8c0b600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c0409aa90 con 0x7f8c0415ec20 2024-04-17T10:02:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.084+0000 7f8c0b600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c0409ad70 con 0x7f8c0409bb30 2024-04-17T10:02:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.084+0000 7f8c0b600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8c0409b6a0 con 0x7f8c04162ed0 2024-04-17T10:02:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.084+0000 7f8c08c00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8c0415ec20 0x7f8c0406cb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57464/0 (socket says 172.21.15.32:57464) 2024-04-17T10:02:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.084+0000 7f8c08c00640 1 -- 172.21.15.32:0/2277023411 learned_addr learned my addr 172.21.15.32:0/2277023411 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:02:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.084+0000 7f8c09600640 1 --2- 172.21.15.32:0/2277023411 >> v2:172.21.15.32:3300/0 conn(0x7f8c04162ed0 0x7f8c04169af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.084+0000 7f8c09600640 1 -- 172.21.15.32:0/2277023411 >> v1:172.21.15.144:6789/0 conn(0x7f8c0409bb30 legacy=0x7f8c041627c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.084+0000 7f8c09600640 1 -- 172.21.15.32:0/2277023411 >> v1:172.21.15.32:6789/0 conn(0x7f8c0415ec20 legacy=0x7f8c0406cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.084+0000 7f8c09600640 1 -- 172.21.15.32:0/2277023411 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0409b050 con 0x7f8c04162ed0 2024-04-17T10:02:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.084+0000 7f8c03400640 1 -- 172.21.15.32:0/2277023411 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2377825717 0 0) 0x7f8c0409ad70 con 0x7f8c0409bb30 2024-04-17T10:02:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.088+0000 7f8c09600640 1 --2- 172.21.15.32:0/2277023411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c04162ed0 0x7f8c04169af0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009d90 tx=0x7f8bfc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aac25080cb79263 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.088+0000 7f8c03400640 1 -- 172.21.15.32:0/2277023411 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc0b2070 con 0x7f8c04162ed0 2024-04-17T10:02:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.088+0000 7f8c03400640 1 -- 172.21.15.32:0/2277023411 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bfc0a5940 con 0x7f8c04162ed0 2024-04-17T10:02:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.088+0000 7f8c03400640 1 -- 172.21.15.32:0/2277023411 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc0a5d10 con 0x7f8c04162ed0 2024-04-17T10:02:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.088+0000 7f8c0b600640 1 -- 172.21.15.32:0/2277023411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c04162ed0 msgr2=0x7f8c04169af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:13.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.088+0000 7f8c0b600640 1 --2- 172.21.15.32:0/2277023411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c04162ed0 0x7f8c04169af0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009d90 tx=0x7f8bfc01c970 comp rx=0 tx=0).stop 2024-04-17T10:02:13.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.088+0000 7f8c0b600640 1 -- 172.21.15.32:0/2277023411 shutdown_connections 2024-04-17T10:02:13.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.088+0000 7f8c0b600640 1 --2- 172.21.15.32:0/2277023411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c04162ed0 0x7f8c04169af0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.088+0000 7f8c0b600640 1 -- 172.21.15.32:0/2277023411 >> 172.21.15.32:0/2277023411 conn(0x7f8c040d25d0 msgr2=0x7f8c040d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:13.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.088+0000 7f8c0b600640 1 -- 172.21.15.32:0/2277023411 shutdown_connections 2024-04-17T10:02:13.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.088+0000 7f8c0b600640 1 -- 172.21.15.32:0/2277023411 wait complete. 2024-04-17T10:02:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.088+0000 7f8c0b600640 1 Processor -- start 2024-04-17T10:02:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.088+0000 7f8c0b600640 1 -- start start 2024-04-17T10:02:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.092+0000 7f8c0b600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c0409bb30 0x7f8c0414a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.092+0000 7f8c0b600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c0415ec20 0x7f8c040b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.092+0000 7f8c0b600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c04162ed0 0x7f8c040c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.092+0000 7f8c0b600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8c040b3890 con 0x7f8c0409bb30 2024-04-17T10:02:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.092+0000 7f8c0b600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8c04160010 con 0x7f8c04162ed0 2024-04-17T10:02:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.092+0000 7f8c0b600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8c040b3a10 con 0x7f8c0415ec20 2024-04-17T10:02:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.092+0000 7f8c08c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c0409bb30 0x7f8c0414a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.092+0000 7f8c08c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c0409bb30 0x7f8c0414a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36534/0 (socket says 172.21.15.32:36534) 2024-04-17T10:02:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.092+0000 7f8c08c00640 1 -- 172.21.15.32:0/3298135584 learned_addr learned my addr 172.21.15.32:0/3298135584 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.092+0000 7f8c09600640 1 --2- 172.21.15.32:0/3298135584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c04162ed0 0x7f8c040c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.092+0000 7f8c03e00640 1 --2- 172.21.15.32:0/3298135584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c0415ec20 0x7f8c040b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.092+0000 7f8c08c00640 1 -- 172.21.15.32:0/3298135584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c0415ec20 msgr2=0x7f8c040b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:13.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.092+0000 7f8c08c00640 1 --2- 172.21.15.32:0/3298135584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c0415ec20 0x7f8c040b5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.092+0000 7f8c08c00640 1 -- 172.21.15.32:0/3298135584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c04162ed0 msgr2=0x7f8c040c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:13.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.092+0000 7f8c08c00640 1 --2- 172.21.15.32:0/3298135584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c04162ed0 0x7f8c040c8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.092+0000 7f8c08c00640 1 -- 172.21.15.32:0/3298135584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c040b7080 con 0x7f8c0409bb30 2024-04-17T10:02:13.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.092+0000 7f8c08c00640 1 --2- 172.21.15.32:0/3298135584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c0409bb30 0x7f8c0414a9c0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800c420 tx=0x7f8bf800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:13.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.092+0000 7f8c00c00640 1 -- 172.21.15.32:0/3298135584 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8017070 con 0x7f8c0409bb30 2024-04-17T10:02:13.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.092+0000 7f8c00c00640 1 -- 172.21.15.32:0/3298135584 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf8005cf0 con 0x7f8c0409bb30 2024-04-17T10:02:13.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.092+0000 7f8c0b600640 1 -- 172.21.15.32:0/3298135584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c0414af00 con 0x7f8c0409bb30 2024-04-17T10:02:13.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.092+0000 7f8c00c00640 1 -- 172.21.15.32:0/3298135584 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf80072f0 con 0x7f8c0409bb30 2024-04-17T10:02:13.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.092+0000 7f8c0b600640 1 -- 172.21.15.32:0/3298135584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c040b67e0 con 0x7f8c0409bb30 2024-04-17T10:02:13.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.092+0000 7f8c0b600640 1 -- 172.21.15.32:0/3298135584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f8c04068a90 con 0x7f8c0409bb30 2024-04-17T10:02:13.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.096+0000 7f8c00c00640 1 -- 172.21.15.32:0/3298135584 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8bf801a070 con 0x7f8c0409bb30 2024-04-17T10:02:13.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.096+0000 7f8c00c00640 1 --2- 172.21.15.32:0/3298135584 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8be40683b0 0x7f8be406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.096+0000 7f8c00c00640 1 -- 172.21.15.32:0/3298135584 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f8bf8085cb0 con 0x7f8c0409bb30 2024-04-17T10:02:13.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.096+0000 7f8c03e00640 1 --2- 172.21.15.32:0/3298135584 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8be40683b0 0x7f8be406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.096+0000 7f8c00c00640 1 -- 172.21.15.32:0/3298135584 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+511 (secure 0 0 0) 0x7f8bf8088250 con 0x7f8c0409bb30 2024-04-17T10:02:13.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.096+0000 7f8c0b600640 1 -- 172.21.15.32:0/3298135584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f8c040d34e0 con 0x7f8c0409bb30 2024-04-17T10:02:13.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.096+0000 7f8c03e00640 1 --2- 172.21.15.32:0/3298135584 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8be40683b0 0x7f8be406a870 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8c040b74a0 tx=0x7f8bf0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:13.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.100+0000 7fdebf4a0000 1 Processor -- start 2024-04-17T10:02:13.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.100+0000 7fdebf4a0000 1 -- start start 2024-04-17T10:02:13.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.100+0000 7fdebf4a0000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ccfdc7ca10 0x55ccfdca8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.100+0000 7fdebf4a0000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdcc9700 0x55ccfdca8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.100+0000 7fdebf4a0000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccfdcca020 0x55ccfdcdab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.100+0000 7fdebf4a0000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb4202bf0 con 0x55ccfdcc9700 2024-04-17T10:02:13.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.100+0000 7fdebf4a0000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb4201f80 con 0x55ccfdcca020 2024-04-17T10:02:13.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.100+0000 7fdebf4a0000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdeb42025a0 con 0x55ccfdc7ca10 2024-04-17T10:02:13.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.100+0000 7fdeb8c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdcc9700 0x55ccfdca8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.100+0000 7fdeb8c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdcc9700 0x55ccfdca8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36546/0 (socket says 172.21.15.32:36546) 2024-04-17T10:02:13.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.100+0000 7fdeb8c00640 1 -- 172.21.15.32:0/1506710344 learned_addr learned my addr 172.21.15.32:0/1506710344 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:13.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.100+0000 7fdeb9600640 1 --2- 172.21.15.32:0/1506710344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ccfdc7ca10 0x55ccfdca8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.100+0000 7fdeba000640 1 --2- 172.21.15.32:0/1506710344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccfdcca020 0x55ccfdcdab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.100+0000 7fdeb8c00640 1 -- 172.21.15.32:0/1506710344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ccfdc7ca10 msgr2=0x55ccfdca8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:13.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.100+0000 7fdeb8c00640 1 --2- 172.21.15.32:0/1506710344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ccfdc7ca10 0x55ccfdca8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.100+0000 7fdeb8c00640 1 -- 172.21.15.32:0/1506710344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccfdcca020 msgr2=0x55ccfdcdab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:13.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.100+0000 7fdeb8c00640 1 --2- 172.21.15.32:0/1506710344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccfdcca020 0x55ccfdcdab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdeb8c00640 1 -- 172.21.15.32:0/1506710344 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccfdadaf70 con 0x55ccfdcc9700 2024-04-17T10:02:13.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdeba000640 1 --2- 172.21.15.32:0/1506710344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccfdcca020 0x55ccfdcdab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:02:13.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdeb8c00640 1 --2- 172.21.15.32:0/1506710344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdcc9700 0x55ccfdca8c80 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fdea000fa50 tx=0x7fdea0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0273cba7f6ec5e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:13.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdea5600640 1 -- 172.21.15.32:0/1506710344 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea001b040 con 0x55ccfdcc9700 2024-04-17T10:02:13.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdea5600640 1 -- 172.21.15.32:0/1506710344 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea001c040 con 0x55ccfdcc9700 2024-04-17T10:02:13.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdea5600640 1 -- 172.21.15.32:0/1506710344 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0015410 con 0x55ccfdcc9700 2024-04-17T10:02:13.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdebf4a0000 1 -- 172.21.15.32:0/1506710344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdcc9700 msgr2=0x55ccfdca8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:13.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdebf4a0000 1 --2- 172.21.15.32:0/1506710344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdcc9700 0x55ccfdca8c80 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fdea000fa50 tx=0x7fdea0015000 comp rx=0 tx=0).stop 2024-04-17T10:02:13.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdebf4a0000 1 -- 172.21.15.32:0/1506710344 shutdown_connections 2024-04-17T10:02:13.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdebf4a0000 1 --2- 172.21.15.32:0/1506710344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccfdcca020 0x55ccfdcdab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdebf4a0000 1 --2- 172.21.15.32:0/1506710344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdcc9700 0x55ccfdca8c80 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdebf4a0000 1 --2- 172.21.15.32:0/1506710344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ccfdc7ca10 0x55ccfdca8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdebf4a0000 1 -- 172.21.15.32:0/1506710344 >> 172.21.15.32:0/1506710344 conn(0x55ccfdcaa290 msgr2=0x55ccfdc83700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:13.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdebf4a0000 1 -- 172.21.15.32:0/1506710344 shutdown_connections 2024-04-17T10:02:13.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdebf4a0000 1 -- 172.21.15.32:0/1506710344 wait complete. 2024-04-17T10:02:13.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdebf4a0000 1 -- start start 2024-04-17T10:02:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdebf4a0000 1 Processor -- start 2024-04-17T10:02:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdebf4a0000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ccfdc7ca10 0x55ccfdcddd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdebf4a0000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdcc9700 0x55ccfdcf4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdebf4a0000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccfdcca020 0x55ccfdcf4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdebf4a0000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ccfdad2c20 con 0x55ccfdcc9700 2024-04-17T10:02:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdebf4a0000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ccfdacd900 con 0x55ccfdcca020 2024-04-17T10:02:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdebf4a0000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ccfdadb800 con 0x55ccfdc7ca10 2024-04-17T10:02:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdebf4a0000 10 client.0 _ll_register_callbacks cb 0x7fdea80094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:02:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdeb8c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdcc9700 0x55ccfdcf4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdeb8c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdcc9700 0x55ccfdcf4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36558/0 (socket says 172.21.15.32:36558) 2024-04-17T10:02:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdeb8c00640 1 -- 172.21.15.32:0/2527811662 learned_addr learned my addr 172.21.15.32:0/2527811662 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdeba000640 1 --2- 172.21.15.32:0/2527811662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccfdcca020 0x55ccfdcf4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdeb9600640 1 --2- 172.21.15.32:0/2527811662 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ccfdc7ca10 0x55ccfdcddd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdeb8c00640 1 -- 172.21.15.32:0/2527811662 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ccfdc7ca10 msgr2=0x55ccfdcddd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:13.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdeb8c00640 1 --2- 172.21.15.32:0/2527811662 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ccfdc7ca10 0x55ccfdcddd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdeb8c00640 1 -- 172.21.15.32:0/2527811662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccfdcca020 msgr2=0x55ccfdcf4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:13.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdeb8c00640 1 --2- 172.21.15.32:0/2527811662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccfdcca020 0x55ccfdcf4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.104+0000 7fdeb8c00640 1 -- 172.21.15.32:0/2527811662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccfda879f0 con 0x55ccfdcc9700 2024-04-17T10:02:13.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdeb8c00640 1 --2- 172.21.15.32:0/2527811662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdcc9700 0x55ccfdcf4710 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fdea000b7b0 tx=0x7fdea0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a978679619d26e62 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:13.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fde9aa00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:02:13.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fde9aa00640 1 -- 172.21.15.32:0/2527811662 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0037060 con 0x55ccfdcc9700 2024-04-17T10:02:13.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fde9aa00640 1 -- 172.21.15.32:0/2527811662 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fdea0015c50 con 0x55ccfdcc9700 2024-04-17T10:02:13.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebf4a0000 1 -- 172.21.15.32:0/2527811662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdea001c040 con 0x55ccfdcc9700 2024-04-17T10:02:13.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fde9aa00640 1 -- 172.21.15.32:0/2527811662 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fdea003d050 con 0x55ccfdcc9700 2024-04-17T10:02:13.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebf4a0000 10 client.8913 fetch_fsmap learned FSMap version 81 2024-04-17T10:02:13.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebf4a0000 1 -- 172.21.15.32:0/2527811662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x55ccfda73660 con 0x55ccfdcc9700 2024-04-17T10:02:13.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fde9aa00640 1 -- 172.21.15.32:0/2527811662 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea001c040 con 0x55ccfdcc9700 2024-04-17T10:02:13.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fde9aa00640 1 -- 172.21.15.32:0/2527811662 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 81) ==== 3495+0+0 (secure 0 0 0) 0x7fdea0040020 con 0x55ccfdcc9700 2024-04-17T10:02:13.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebf4a0000 10 client.8913 fetch_fsmap finished waiting for FSMap version 81 2024-04-17T10:02:13.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebf4a0000 10 client.8913 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-17T10:02:13.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebf4a0000 20 client.8913 populate_metadata read hostname 'smithi032' 2024-04-17T10:02:13.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebf4a0000 1 --2- 172.21.15.32:0/2527811662 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x55ccfdd389b0 0x55ccfdd38dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebf4a0000 4 client.8913 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-17T10:02:13.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebf4a0000 1 -- 172.21.15.32:0/2527811662 --> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x55ccfda67ed0 con 0x55ccfdd389b0 2024-04-17T10:02:13.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdeb9600640 1 --2- 172.21.15.32:0/2527811662 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x55ccfdd389b0 0x55ccfdd38dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdeb9600640 1 --2- 172.21.15.32:0/2527811662 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x55ccfdd389b0 0x55ccfdd38dc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5195f6e252b09f42 server_cookie=ac44c3297bc16464 in_seq=0 out_seq=0 2024-04-17T10:02:13.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fde9aa00640 10 client.8913 ms_handle_connect on v2:172.21.15.32:6835/1235126741 2024-04-17T10:02:13.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fde9aa00640 1 -- 172.21.15.32:0/2527811662 <== mds.0 v2:172.21.15.32:6835/1235126741 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ccfda67ed0 con 0x55ccfdd389b0 2024-04-17T10:02:13.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fde9aa00640 10 client.8913 handle_command_reply: tid=0 2024-04-17T10:02:13.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebf4a0000 1 client.8913 shutdown 2024-04-17T10:02:13.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebf4a0000 1 -- 172.21.15.32:0/2527811662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdcc9700 msgr2=0x55ccfdcf4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:13.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebf4a0000 1 --2- 172.21.15.32:0/2527811662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdcc9700 0x55ccfdcf4710 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fdea000b7b0 tx=0x7fdea0012000 comp rx=0 tx=0).stop 2024-04-17T10:02:13.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebf4a0000 1 -- 172.21.15.32:0/2527811662 shutdown_connections 2024-04-17T10:02:13.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebf4a0000 1 --2- 172.21.15.32:0/2527811662 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x55ccfdd389b0 0x55ccfdd38dc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebf4a0000 1 --2- 172.21.15.32:0/2527811662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccfdcca020 0x55ccfdcf4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebf4a0000 1 --2- 172.21.15.32:0/2527811662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ccfdcc9700 0x55ccfdcf4710 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebf4a0000 1 --2- 172.21.15.32:0/2527811662 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ccfdc7ca10 0x55ccfdcddd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fde9aa00640 0 client.8913 ms_handle_reset on v2:172.21.15.32:6835/1235126741 2024-04-17T10:02:13.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebf4a0000 1 -- 172.21.15.32:0/2527811662 >> 172.21.15.32:0/2527811662 conn(0x55ccfdcaa290 msgr2=0x55ccfdca7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:13.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebf4a0000 1 -- 172.21.15.32:0/2527811662 shutdown_connections 2024-04-17T10:02:13.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebf4a0000 1 -- 172.21.15.32:0/2527811662 wait complete. 2024-04-17T10:02:13.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebf4a0000 20 client.8913 trim_cache size 0 max 16384 2024-04-17T10:02:13.111 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:02:13.111 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-17T10:02:13.111 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "0d36c4c5-e395-4395-afb9-7c59710f27c1", 2024-04-17T10:02:13.111 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-17T10:02:13.111 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:02:13.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebc000640 1 -- 172.21.15.32:0/550314895 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fde94068380 msgr2=0x7fde9406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:13.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebc000640 1 --2- 172.21.15.32:0/550314895 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fde94068380 0x7fde9406a840 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fdeb415b8a0 tx=0x7fdea0008040 comp rx=0 tx=0).stop 2024-04-17T10:02:13.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebc000640 1 -- 172.21.15.32:0/550314895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb406bf70 msgr2=0x7fdeb4160810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:13.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.108+0000 7fdebc000640 1 --2- 172.21.15.32:0/550314895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb406bf70 0x7fdeb4160810 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fdea80ae040 tx=0x7fdea801ad00 comp rx=0 tx=0).stop 2024-04-17T10:02:13.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.112+0000 7fdebc000640 1 -- 172.21.15.32:0/550314895 shutdown_connections 2024-04-17T10:02:13.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.112+0000 7fdebc000640 1 --2- 172.21.15.32:0/550314895 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fde94068380 0x7fde9406a840 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.112+0000 7fdebc000640 1 --2- 172.21.15.32:0/550314895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdeb415e9d0 0x7fdeb409e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.112+0000 7fdebc000640 1 --2- 172.21.15.32:0/550314895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdeb40b2800 0x7fdeb409dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.112+0000 7fdebc000640 1 --2- 172.21.15.32:0/550314895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb406bf70 0x7fdeb4160810 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.112+0000 7fdebc000640 1 -- 172.21.15.32:0/550314895 >> 172.21.15.32:0/550314895 conn(0x7fdeb40b6860 msgr2=0x7fdeb40b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:13.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.112+0000 7fdebc000640 1 -- 172.21.15.32:0/550314895 shutdown_connections 2024-04-17T10:02:13.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.112+0000 7fdebc000640 1 -- 172.21.15.32:0/550314895 wait complete. 2024-04-17T10:02:13.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:02:13.128 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-17T10:02:13.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.276+0000 7fd1dde00640 1 Processor -- start 2024-04-17T10:02:13.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.276+0000 7fd1dde00640 1 -- start start 2024-04-17T10:02:13.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.276+0000 7fd1dde00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1d806ba30 0x7fd1d806be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.276+0000 7fd1dde00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1d80cf370 0x7fd1d80cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.276+0000 7fd1dde00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd1d8160350 0x7fd1d81607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.276+0000 7fd1dde00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd1d806c350 con 0x7fd1d806ba30 2024-04-17T10:02:13.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.276+0000 7fd1dde00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd1d80d0130 con 0x7fd1d8160350 2024-04-17T10:02:13.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.276+0000 7fd1dde00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd1d815d090 con 0x7fd1d80cf370 2024-04-17T10:02:13.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.276+0000 7fd1d6a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1d806ba30 0x7fd1d806be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.276+0000 7fd1d6a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1d806ba30 0x7fd1d806be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36560/0 (socket says 172.21.15.32:36560) 2024-04-17T10:02:13.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.276+0000 7fd1d6a00640 1 -- 172.21.15.32:0/1155654903 learned_addr learned my addr 172.21.15.32:0/1155654903 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:13.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.276+0000 7fd1d6000640 1 --2- 172.21.15.32:0/1155654903 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1d80cf370 0x7fd1d80cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.276+0000 7fd1d7400640 1 --2- 172.21.15.32:0/1155654903 >> v2:172.21.15.144:3300/0 conn(0x7fd1d8160350 0x7fd1d81607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.276+0000 7fd1d6a00640 1 -- 172.21.15.32:0/1155654903 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1d80cf370 msgr2=0x7fd1d80cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:13.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.276+0000 7fd1d6a00640 1 --2- 172.21.15.32:0/1155654903 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1d80cf370 0x7fd1d80cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.276+0000 7fd1d6a00640 1 -- 172.21.15.32:0/1155654903 >> v2:172.21.15.144:3300/0 conn(0x7fd1d8160350 msgr2=0x7fd1d81607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:13.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.276+0000 7fd1d6a00640 1 --2- 172.21.15.32:0/1155654903 >> v2:172.21.15.144:3300/0 conn(0x7fd1d8160350 0x7fd1d81607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.276+0000 7fd1d6a00640 1 -- 172.21.15.32:0/1155654903 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d815d370 con 0x7fd1d806ba30 2024-04-17T10:02:13.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.276+0000 7fd1d6000640 1 --2- 172.21.15.32:0/1155654903 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1d80cf370 0x7fd1d80cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:02:13.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.280+0000 7fd1d6a00640 1 --2- 172.21.15.32:0/1155654903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1d806ba30 0x7fd1d806be10 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80b58d0 tx=0x7fd1c801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa11f088d7481c98 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:13.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.280+0000 7fd1d5600640 1 -- 172.21.15.32:0/1155654903 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c80a7070 con 0x7fd1d806ba30 2024-04-17T10:02:13.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.280+0000 7fd1d5600640 1 -- 172.21.15.32:0/1155654903 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c80b0070 con 0x7fd1d806ba30 2024-04-17T10:02:13.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.280+0000 7fd1d5600640 1 -- 172.21.15.32:0/1155654903 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c80ab890 con 0x7fd1d806ba30 2024-04-17T10:02:13.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.280+0000 7fd1dde00640 1 -- 172.21.15.32:0/1155654903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1d806ba30 msgr2=0x7fd1d806be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:13.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.280+0000 7fd1dde00640 1 --2- 172.21.15.32:0/1155654903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1d806ba30 0x7fd1d806be10 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80b58d0 tx=0x7fd1c801c4f0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.280+0000 7fd1dde00640 1 -- 172.21.15.32:0/1155654903 shutdown_connections 2024-04-17T10:02:13.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.280+0000 7fd1dde00640 1 --2- 172.21.15.32:0/1155654903 >> v2:172.21.15.144:3300/0 conn(0x7fd1d8160350 0x7fd1d81607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.280+0000 7fd1dde00640 1 --2- 172.21.15.32:0/1155654903 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1d80cf370 0x7fd1d80cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.280+0000 7fd1dde00640 1 --2- 172.21.15.32:0/1155654903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1d806ba30 0x7fd1d806be10 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.280+0000 7fd1dde00640 1 -- 172.21.15.32:0/1155654903 >> 172.21.15.32:0/1155654903 conn(0x7fd1d80b2120 msgr2=0x7fd1d80b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:13.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.280+0000 7fd1dde00640 1 -- 172.21.15.32:0/1155654903 shutdown_connections 2024-04-17T10:02:13.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.280+0000 7fd1dde00640 1 -- 172.21.15.32:0/1155654903 wait complete. 2024-04-17T10:02:13.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1dde00640 1 Processor -- start 2024-04-17T10:02:13.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1dde00640 1 -- start start 2024-04-17T10:02:13.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1dde00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1d806ba30 0x7fd1d815f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1dde00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1d80cf370 0x7fd1d81fb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1dde00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1d8160350 0x7fd1d81fbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1dde00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d80cfea0 con 0x7fd1d8160350 2024-04-17T10:02:13.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1dde00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d80cfd20 con 0x7fd1d806ba30 2024-04-17T10:02:13.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1dde00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd1d815db70 con 0x7fd1d80cf370 2024-04-17T10:02:13.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1d7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1d8160350 0x7fd1d81fbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1d7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1d8160350 0x7fd1d81fbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36572/0 (socket says 172.21.15.32:36572) 2024-04-17T10:02:13.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1d7400640 1 -- 172.21.15.32:0/1753150317 learned_addr learned my addr 172.21.15.32:0/1753150317 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:13.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1d6a00640 1 --2- 172.21.15.32:0/1753150317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1d806ba30 0x7fd1d815f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1d6000640 1 --2- 172.21.15.32:0/1753150317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1d80cf370 0x7fd1d81fb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1d7400640 1 -- 172.21.15.32:0/1753150317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1d80cf370 msgr2=0x7fd1d81fb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:13.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1d7400640 1 --2- 172.21.15.32:0/1753150317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1d80cf370 0x7fd1d81fb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1d7400640 1 -- 172.21.15.32:0/1753150317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1d806ba30 msgr2=0x7fd1d815f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:13.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1d7400640 1 --2- 172.21.15.32:0/1753150317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1d806ba30 0x7fd1d815f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1d7400640 1 -- 172.21.15.32:0/1753150317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d806c0a0 con 0x7fd1d8160350 2024-04-17T10:02:13.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1d7400640 1 --2- 172.21.15.32:0/1753150317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1d8160350 0x7fd1d81fbb80 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0007e60 tx=0x7fd1d000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:13.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1cf400640 1 -- 172.21.15.32:0/1753150317 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0014420 con 0x7fd1d8160350 2024-04-17T10:02:13.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1cf400640 1 -- 172.21.15.32:0/1753150317 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1d0019070 con 0x7fd1d8160350 2024-04-17T10:02:13.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1cf400640 1 -- 172.21.15.32:0/1753150317 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0014820 con 0x7fd1d8160350 2024-04-17T10:02:13.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1dde00640 1 -- 172.21.15.32:0/1753150317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d81430f0 con 0x7fd1d8160350 2024-04-17T10:02:13.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.284+0000 7fd1dde00640 1 -- 172.21.15.32:0/1753150317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d81436b0 con 0x7fd1d8160350 2024-04-17T10:02:13.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1cf400640 1 -- 172.21.15.32:0/1753150317 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd1d00149c0 con 0x7fd1d8160350 2024-04-17T10:02:13.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1cf400640 1 --2- 172.21.15.32:0/1753150317 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd1b8068380 0x7fd1b806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1e12cc000 1 Processor -- start 2024-04-17T10:02:13.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1cf400640 1 -- 172.21.15.32:0/1753150317 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fd1d0087140 con 0x7fd1d8160350 2024-04-17T10:02:13.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1d6a00640 1 --2- 172.21.15.32:0/1753150317 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd1b8068380 0x7fd1b806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1e12cc000 1 -- start start 2024-04-17T10:02:13.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1d6a00640 1 --2- 172.21.15.32:0/1753150317 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd1b8068380 0x7fd1b806a840 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80af000 tx=0x7fd1c80a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:13.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1e12cc000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b04c9260 0x5576b04c9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1e12cc000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b04c9b80 0x5576b047c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1e12cc000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5576b047c9a0 0x5576b047cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1e12cc000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5576b02ce4f0 con 0x5576b04c9b80 2024-04-17T10:02:13.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1e12cc000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5576b02876c0 con 0x5576b04c9260 2024-04-17T10:02:13.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1e12cc000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5576b02ca4b0 con 0x5576b047c9a0 2024-04-17T10:02:13.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1d6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b04c9b80 0x5576b047c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1d6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b04c9b80 0x5576b047c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36576/0 (socket says 172.21.15.32:36576) 2024-04-17T10:02:13.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1d6000640 1 -- 172.21.15.32:0/3941953185 learned_addr learned my addr 172.21.15.32:0/3941953185 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:13.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1d6a00640 1 --2- 172.21.15.32:0/3941953185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b04c9260 0x5576b04c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1d7400640 1 --2- 172.21.15.32:0/3941953185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5576b047c9a0 0x5576b047cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1d6000640 1 -- 172.21.15.32:0/3941953185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5576b047c9a0 msgr2=0x5576b047cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:13.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1d6000640 1 --2- 172.21.15.32:0/3941953185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5576b047c9a0 0x5576b047cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1d6000640 1 -- 172.21.15.32:0/3941953185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b04c9260 msgr2=0x5576b04c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:13.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1d6000640 1 --2- 172.21.15.32:0/3941953185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b04c9260 0x5576b04c9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1d6000640 1 -- 172.21.15.32:0/3941953185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576b03138e0 con 0x5576b04c9b80 2024-04-17T10:02:13.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1d6a00640 1 --2- 172.21.15.32:0/3941953185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b04c9260 0x5576b04c9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:02:13.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1d7400640 1 --2- 172.21.15.32:0/3941953185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5576b047c9a0 0x5576b047cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:02:13.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.288+0000 7fd1d6000640 1 --2- 172.21.15.32:0/3941953185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b04c9b80 0x5576b047c460 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40098a0 tx=0x7fd1c4009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8be862665bb4552 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:13.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1c3e00640 1 -- 172.21.15.32:0/3941953185 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4019020 con 0x5576b04c9b80 2024-04-17T10:02:13.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1c3e00640 1 -- 172.21.15.32:0/3941953185 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c40042f0 con 0x5576b04c9b80 2024-04-17T10:02:13.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1c3e00640 1 -- 172.21.15.32:0/3941953185 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c400c070 con 0x5576b04c9b80 2024-04-17T10:02:13.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1e12cc000 1 -- 172.21.15.32:0/3941953185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b04c9b80 msgr2=0x5576b047c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:13.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1e12cc000 1 --2- 172.21.15.32:0/3941953185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b04c9b80 0x5576b047c460 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40098a0 tx=0x7fd1c4009d60 comp rx=0 tx=0).stop 2024-04-17T10:02:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1e12cc000 1 -- 172.21.15.32:0/3941953185 shutdown_connections 2024-04-17T10:02:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1e12cc000 1 --2- 172.21.15.32:0/3941953185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5576b047c9a0 0x5576b047cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1e12cc000 1 --2- 172.21.15.32:0/3941953185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b04c9b80 0x5576b047c460 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1e12cc000 1 --2- 172.21.15.32:0/3941953185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b04c9260 0x5576b04c9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1e12cc000 1 -- 172.21.15.32:0/3941953185 >> 172.21.15.32:0/3941953185 conn(0x5576b04aa000 msgr2=0x5576b04ac3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1e12cc000 1 -- 172.21.15.32:0/3941953185 shutdown_connections 2024-04-17T10:02:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1e12cc000 1 -- 172.21.15.32:0/3941953185 wait complete. 2024-04-17T10:02:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1e12cc000 1 -- start start 2024-04-17T10:02:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1e12cc000 1 Processor -- start 2024-04-17T10:02:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1e12cc000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b047c9a0 0x5576b0481fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1e12cc000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5576b04c9260 0x5576b0482520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1e12cc000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b04c9b80 0x5576b0482a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1e12cc000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5576b034f7b0 con 0x5576b047c9a0 2024-04-17T10:02:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1e12cc000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5576b0389d00 con 0x5576b04c9b80 2024-04-17T10:02:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1e12cc000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5576b02932c0 con 0x5576b04c9260 2024-04-17T10:02:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1e12cc000 10 client.0 _ll_register_callbacks cb 0x5576b02a3030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:02:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1d6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b047c9a0 0x5576b0481fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1d6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b047c9a0 0x5576b0481fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36586/0 (socket says 172.21.15.32:36586) 2024-04-17T10:02:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1d6a00640 1 -- 172.21.15.32:0/730095622 learned_addr learned my addr 172.21.15.32:0/730095622 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1d6000640 1 --2- 172.21.15.32:0/730095622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5576b04c9260 0x5576b0482520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1d7400640 1 --2- 172.21.15.32:0/730095622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b04c9b80 0x5576b0482a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1d6a00640 1 -- 172.21.15.32:0/730095622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5576b04c9260 msgr2=0x5576b0482520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1d6a00640 1 --2- 172.21.15.32:0/730095622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5576b04c9260 0x5576b0482520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1d6a00640 1 -- 172.21.15.32:0/730095622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b04c9b80 msgr2=0x5576b0482a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1d6a00640 1 --2- 172.21.15.32:0/730095622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b04c9b80 0x5576b0482a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1d6a00640 1 -- 172.21.15.32:0/730095622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576b02a6490 con 0x5576b047c9a0 2024-04-17T10:02:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1d7400640 1 --2- 172.21.15.32:0/730095622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b04c9b80 0x5576b0482a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:02:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1d6a00640 1 --2- 172.21.15.32:0/730095622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b047c9a0 0x5576b0481fe0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fd1c801c9d0 tx=0x7fd1c801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221672b7d731b5ce server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:13.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:02:13.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1c0c00640 1 -- 172.21.15.32:0/730095622 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c80cd060 con 0x5576b047c9a0 2024-04-17T10:02:13.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1c0c00640 1 -- 172.21.15.32:0/730095622 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fd1c80c0c20 con 0x5576b047c9a0 2024-04-17T10:02:13.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.292+0000 7fd1e12cc000 1 -- 172.21.15.32:0/730095622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1c40042f0 con 0x5576b047c9a0 2024-04-17T10:02:13.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1c0c00640 1 -- 172.21.15.32:0/730095622 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fd1c80d3050 con 0x5576b047c9a0 2024-04-17T10:02:13.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1e12cc000 10 client.8925 fetch_fsmap learned FSMap version 81 2024-04-17T10:02:13.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1e12cc000 1 -- 172.21.15.32:0/730095622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x5576b0299eb0 con 0x5576b047c9a0 2024-04-17T10:02:13.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1c0c00640 1 -- 172.21.15.32:0/730095622 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c40042f0 con 0x5576b047c9a0 2024-04-17T10:02:13.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1c0c00640 1 -- 172.21.15.32:0/730095622 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 81) ==== 3495+0+0 (secure 0 0 0) 0x7fd1c80d6020 con 0x5576b047c9a0 2024-04-17T10:02:13.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1e12cc000 10 client.8925 fetch_fsmap finished waiting for FSMap version 81 2024-04-17T10:02:13.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1e12cc000 10 client.8925 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-17T10:02:13.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1e12cc000 20 client.8925 populate_metadata read hostname 'smithi032' 2024-04-17T10:02:13.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1e12cc000 1 --2- 172.21.15.32:0/730095622 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x5576b04d8500 0x5576b04da8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1e12cc000 4 client.8925 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:02:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1e12cc000 1 -- 172.21.15.32:0/730095622 --> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576b0267dd0 con 0x5576b04d8500 2024-04-17T10:02:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1d6000640 1 --2- 172.21.15.32:0/730095622 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x5576b04d8500 0x5576b04da8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1d6000640 1 --2- 172.21.15.32:0/730095622 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x5576b04d8500 0x5576b04da8f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10400ef07a29b557 server_cookie=511d48e16ec3cf7c in_seq=0 out_seq=0 2024-04-17T10:02:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1c0c00640 10 client.8925 ms_handle_connect on v2:172.21.15.32:6835/1235126741 2024-04-17T10:02:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1c0c00640 1 -- 172.21.15.32:0/730095622 <== mds.0 v2:172.21.15.32:6835/1235126741 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5576b0267dd0 con 0x5576b04d8500 2024-04-17T10:02:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1c0c00640 10 client.8925 handle_command_reply: tid=0 2024-04-17T10:02:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1e12cc000 1 client.8925 shutdown 2024-04-17T10:02:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1e12cc000 1 -- 172.21.15.32:0/730095622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b047c9a0 msgr2=0x5576b0481fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1e12cc000 1 --2- 172.21.15.32:0/730095622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b047c9a0 0x5576b0481fe0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fd1c801c9d0 tx=0x7fd1c801ad30 comp rx=0 tx=0).stop 2024-04-17T10:02:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1e12cc000 1 -- 172.21.15.32:0/730095622 shutdown_connections 2024-04-17T10:02:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1e12cc000 1 --2- 172.21.15.32:0/730095622 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x5576b04d8500 0x5576b04da8f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1e12cc000 1 --2- 172.21.15.32:0/730095622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5576b04c9260 0x5576b0482520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1e12cc000 1 --2- 172.21.15.32:0/730095622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b04c9b80 0x5576b0482a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1e12cc000 1 --2- 172.21.15.32:0/730095622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b047c9a0 0x5576b0481fe0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1c0c00640 0 client.8925 ms_handle_reset on v2:172.21.15.32:6835/1235126741 2024-04-17T10:02:13.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1e12cc000 1 -- 172.21.15.32:0/730095622 >> 172.21.15.32:0/730095622 conn(0x5576b04aa000 msgr2=0x5576b04ac3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:13.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1e12cc000 1 -- 172.21.15.32:0/730095622 shutdown_connections 2024-04-17T10:02:13.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1e12cc000 1 -- 172.21.15.32:0/730095622 wait complete. 2024-04-17T10:02:13.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.296+0000 7fd1e12cc000 20 client.8925 trim_cache size 0 max 16384 2024-04-17T10:02:13.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.312+0000 7fd1e12cc000 1 Processor -- start 2024-04-17T10:02:13.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1e12cc000 1 -- start start 2024-04-17T10:02:13.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1e12cc000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b047c9a0 0x5576b04c8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1e12cc000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b04c9260 0x5576b04c8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1e12cc000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5576b04c9b80 0x5576b04da8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1e12cc000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d81f8de0 con 0x5576b04c9260 2024-04-17T10:02:13.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1e12cc000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d81f8140 con 0x5576b047c9a0 2024-04-17T10:02:13.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1e12cc000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd1d81f8790 con 0x5576b04c9b80 2024-04-17T10:02:13.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1d6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b04c9260 0x5576b04c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1d6000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b04c9260 0x5576b04c8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36592/0 (socket says 172.21.15.32:36592) 2024-04-17T10:02:13.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1d6000640 1 -- 172.21.15.32:0/3065913501 learned_addr learned my addr 172.21.15.32:0/3065913501 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:13.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1d6000640 1 -- 172.21.15.32:0/3065913501 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5576b04c9b80 msgr2=0x5576b04da8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:13.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1d6000640 1 --2- 172.21.15.32:0/3065913501 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5576b04c9b80 0x5576b04da8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1d6000640 1 -- 172.21.15.32:0/3065913501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b047c9a0 msgr2=0x5576b04c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:13.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1d6a00640 1 --2- 172.21.15.32:0/3065913501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b047c9a0 0x5576b04c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1d6000640 1 --2- 172.21.15.32:0/3065913501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b047c9a0 0x5576b04c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1d6000640 1 -- 172.21.15.32:0/3065913501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576b0288a20 con 0x5576b04c9260 2024-04-17T10:02:13.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1d6a00640 1 --2- 172.21.15.32:0/3065913501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b047c9a0 0x5576b04c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:02:13.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1d6000640 1 --2- 172.21.15.32:0/3065913501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b04c9260 0x5576b04c8860 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4018040 tx=0x7fd1c400b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb269ffebd47b64 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:13.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1c3400640 1 -- 172.21.15.32:0/3065913501 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4019040 con 0x5576b04c9260 2024-04-17T10:02:13.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1c3400640 1 -- 172.21.15.32:0/3065913501 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c4011840 con 0x5576b04c9260 2024-04-17T10:02:13.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1c3400640 1 -- 172.21.15.32:0/3065913501 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c400f040 con 0x5576b04c9260 2024-04-17T10:02:13.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1e12cc000 1 -- 172.21.15.32:0/3065913501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b04c9260 msgr2=0x5576b04c8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:13.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1e12cc000 1 --2- 172.21.15.32:0/3065913501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b04c9260 0x5576b04c8860 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4018040 tx=0x7fd1c400b000 comp rx=0 tx=0).stop 2024-04-17T10:02:13.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1e12cc000 1 -- 172.21.15.32:0/3065913501 shutdown_connections 2024-04-17T10:02:13.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1e12cc000 1 --2- 172.21.15.32:0/3065913501 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5576b04c9b80 0x5576b04da8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1e12cc000 1 --2- 172.21.15.32:0/3065913501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b04c9260 0x5576b04c8860 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1e12cc000 1 --2- 172.21.15.32:0/3065913501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b047c9a0 0x5576b04c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1e12cc000 1 -- 172.21.15.32:0/3065913501 >> 172.21.15.32:0/3065913501 conn(0x5576b04aa000 msgr2=0x5576b047b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:13.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1e12cc000 1 -- 172.21.15.32:0/3065913501 shutdown_connections 2024-04-17T10:02:13.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1e12cc000 1 -- 172.21.15.32:0/3065913501 wait complete. 2024-04-17T10:02:13.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1e12cc000 1 -- start start 2024-04-17T10:02:13.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.316+0000 7fd1e12cc000 1 Processor -- start 2024-04-17T10:02:13.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b047c9a0 0x5576b04dd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b04c9260 0x5576b04f4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5576b04c9b80 0x5576b04f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5576b0272050 con 0x5576b047c9a0 2024-04-17T10:02:13.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5576b0290570 con 0x5576b04c9260 2024-04-17T10:02:13.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5576b02cd800 con 0x5576b04c9b80 2024-04-17T10:02:13.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 10 client.0 _ll_register_callbacks cb 0x7fd1c80092b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:02:13.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1d6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b047c9a0 0x5576b04dd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1d6a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b047c9a0 0x5576b04dd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36596/0 (socket says 172.21.15.32:36596) 2024-04-17T10:02:13.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1d6a00640 1 -- 172.21.15.32:0/1455872497 learned_addr learned my addr 172.21.15.32:0/1455872497 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:13.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1d6000640 1 --2- 172.21.15.32:0/1455872497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b04c9260 0x5576b04f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1d7400640 1 --2- 172.21.15.32:0/1455872497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5576b04c9b80 0x5576b04f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1d6a00640 1 -- 172.21.15.32:0/1455872497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5576b04c9b80 msgr2=0x5576b04f49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:13.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1d6a00640 1 --2- 172.21.15.32:0/1455872497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5576b04c9b80 0x5576b04f49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1d6a00640 1 -- 172.21.15.32:0/1455872497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b04c9260 msgr2=0x5576b04f4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:13.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1d6a00640 1 --2- 172.21.15.32:0/1455872497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b04c9260 0x5576b04f4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1d6a00640 1 -- 172.21.15.32:0/1455872497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576b034fc40 con 0x5576b047c9a0 2024-04-17T10:02:13.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1d6000640 1 --2- 172.21.15.32:0/1455872497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b04c9260 0x5576b04f4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:02:13.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1d7400640 1 --2- 172.21.15.32:0/1455872497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5576b04c9b80 0x5576b04f49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:02:13.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1d6a00640 1 --2- 172.21.15.32:0/1455872497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b047c9a0 0x5576b04dd970 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fd1c801a120 tx=0x7fd1c8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5047987fd7e6fc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:13.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:02:13.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1c0c00640 1 -- 172.21.15.32:0/1455872497 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c80c8830 con 0x5576b047c9a0 2024-04-17T10:02:13.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1c0c00640 1 -- 172.21.15.32:0/1455872497 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fd1c80d8aa0 con 0x5576b047c9a0 2024-04-17T10:02:13.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 1 -- 172.21.15.32:0/1455872497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1c4011840 con 0x5576b047c9a0 2024-04-17T10:02:13.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1c0c00640 1 -- 172.21.15.32:0/1455872497 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fd1c80cd030 con 0x5576b047c9a0 2024-04-17T10:02:13.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 10 client.8931 fetch_fsmap learned FSMap version 81 2024-04-17T10:02:13.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 1 -- 172.21.15.32:0/1455872497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x5576b0273600 con 0x5576b047c9a0 2024-04-17T10:02:13.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1c0c00640 1 -- 172.21.15.32:0/1455872497 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c4011840 con 0x5576b047c9a0 2024-04-17T10:02:13.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1c0c00640 1 -- 172.21.15.32:0/1455872497 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 81) ==== 3495+0+0 (secure 0 0 0) 0x7fd1c80d0070 con 0x5576b047c9a0 2024-04-17T10:02:13.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 10 client.8931 fetch_fsmap finished waiting for FSMap version 81 2024-04-17T10:02:13.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 10 client.8931 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-17T10:02:13.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 20 client.8931 populate_metadata read hostname 'smithi032' 2024-04-17T10:02:13.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 1 --2- 172.21.15.32:0/1455872497 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x5576b0538300 0x5576b0538710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:13.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 4 client.8931 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:02:13.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 1 -- 172.21.15.32:0/1455872497 --> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5576b0485020 con 0x5576b0538300 2024-04-17T10:02:13.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1d6000640 1 --2- 172.21.15.32:0/1455872497 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x5576b0538300 0x5576b0538710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:13.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1d6000640 1 --2- 172.21.15.32:0/1455872497 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x5576b0538300 0x5576b0538710 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ed95b31247f121b server_cookie=d6b67ee2f3b24359 in_seq=0 out_seq=0 2024-04-17T10:02:13.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1c0c00640 10 client.8931 ms_handle_connect on v2:172.21.15.32:6835/1235126741 2024-04-17T10:02:13.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1c0c00640 1 -- 172.21.15.32:0/1455872497 <== mds.0 v2:172.21.15.32:6835/1235126741 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x5576b0485020 con 0x5576b0538300 2024-04-17T10:02:13.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1c0c00640 10 client.8931 handle_command_reply: tid=0 2024-04-17T10:02:13.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 1 client.8931 shutdown 2024-04-17T10:02:13.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 1 -- 172.21.15.32:0/1455872497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b047c9a0 msgr2=0x5576b04dd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:13.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 1 --2- 172.21.15.32:0/1455872497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b047c9a0 0x5576b04dd970 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fd1c801a120 tx=0x7fd1c8008040 comp rx=0 tx=0).stop 2024-04-17T10:02:13.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 1 -- 172.21.15.32:0/1455872497 shutdown_connections 2024-04-17T10:02:13.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 1 --2- 172.21.15.32:0/1455872497 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x5576b0538300 0x5576b0538710 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 1 --2- 172.21.15.32:0/1455872497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5576b04c9b80 0x5576b04f49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 1 --2- 172.21.15.32:0/1455872497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576b04c9260 0x5576b04f4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 1 --2- 172.21.15.32:0/1455872497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576b047c9a0 0x5576b04dd970 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1c0c00640 0 client.8931 ms_handle_reset on v2:172.21.15.32:6835/1235126741 2024-04-17T10:02:13.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 1 -- 172.21.15.32:0/1455872497 >> 172.21.15.32:0/1455872497 conn(0x5576b04aa000 msgr2=0x5576b047bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:13.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 1 -- 172.21.15.32:0/1455872497 shutdown_connections 2024-04-17T10:02:13.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 1 -- 172.21.15.32:0/1455872497 wait complete. 2024-04-17T10:02:13.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.320+0000 7fd1e12cc000 20 client.8931 trim_cache size 0 max 16384 2024-04-17T10:02:13.325 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:02:13.325 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T10:02:13.325 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-17T10:02:13.325 INFO:teuthology.orchestra.run.smithi032.stdout: "0d36c4c5-e395-4395-afb9-7c59710f27c1": { 2024-04-17T10:02:13.325 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-04-17T10:02:13.325 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "0d36c4c5-e395-4395-afb9-7c59710f27c1", 2024-04-17T10:02:13.325 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,repair" 2024-04-17T10:02:13.325 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:02:13.325 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:02:13.325 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:02:13.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.324+0000 7fd1dde00640 1 -- 172.21.15.32:0/1753150317 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd1b8068380 msgr2=0x7fd1b806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:13.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.324+0000 7fd1dde00640 1 --2- 172.21.15.32:0/1753150317 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd1b8068380 0x7fd1b806a840 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80af000 tx=0x7fd1c80a7000 comp rx=0 tx=0).stop 2024-04-17T10:02:13.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.324+0000 7fd1dde00640 1 -- 172.21.15.32:0/1753150317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1d8160350 msgr2=0x7fd1d81fbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:13.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.324+0000 7fd1dde00640 1 --2- 172.21.15.32:0/1753150317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1d8160350 0x7fd1d81fbb80 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0007e60 tx=0x7fd1d000e4b0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.324+0000 7fd1dde00640 1 -- 172.21.15.32:0/1753150317 shutdown_connections 2024-04-17T10:02:13.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.324+0000 7fd1dde00640 1 --2- 172.21.15.32:0/1753150317 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd1b8068380 0x7fd1b806a840 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.324+0000 7fd1dde00640 1 --2- 172.21.15.32:0/1753150317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1d80cf370 0x7fd1d81fb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.324+0000 7fd1dde00640 1 --2- 172.21.15.32:0/1753150317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1d8160350 0x7fd1d81fbb80 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.324+0000 7fd1dde00640 1 --2- 172.21.15.32:0/1753150317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1d806ba30 0x7fd1d815f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:13.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.324+0000 7fd1dde00640 1 -- 172.21.15.32:0/1753150317 >> 172.21.15.32:0/1753150317 conn(0x7fd1d80b2120 msgr2=0x7fd1d80ccfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:13.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.324+0000 7fd1dde00640 1 -- 172.21.15.32:0/1753150317 shutdown_connections 2024-04-17T10:02:13.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.324+0000 7fd1dde00640 1 -- 172.21.15.32:0/1753150317 wait complete. 2024-04-17T10:02:13.339 INFO:tasks.cephfs.filesystem:scrub status for tag:0d36c4c5-e395-4395-afb9-7c59710f27c1 - {'path': '/', 'tag': '0d36c4c5-e395-4395-afb9-7c59710f27c1', 'options': 'recursive,repair'} 2024-04-17T10:02:13.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:13.468+0000 7f8c00c00640 1 -- 172.21.15.32:0/3298135584 <== mon.0 v2:172.21.15.32:3300/0 7 ==== log(4 entries from seq 1 at 2024-04-17T10:02:13.112927+0000) ==== 695+0+0 (secure 0 0 0) 0x7f8bf8053cb0 con 0x7f8c0409bb30 2024-04-17T10:02:13.466 INFO:teuthology.orchestra.run.smithi032.stdout: cluster: 2024-04-17T10:02:13.466 INFO:teuthology.orchestra.run.smithi032.stdout: id: ff178df8-4c06-4367-a1e9-2e3a66d870d0 2024-04-17T10:02:13.466 INFO:teuthology.orchestra.run.smithi032.stdout: health: HEALTH_OK 2024-04-17T10:02:13.466 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:02:13.466 INFO:teuthology.orchestra.run.smithi032.stdout: services: 2024-04-17T10:02:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: mon: 3 daemons, quorum a,b,c (age 9m) 2024-04-17T10:02:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: mgr: y(active, since 9m), standbys: x 2024-04-17T10:02:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: mds: 1/1 daemons up, 3 standby 2024-04-17T10:02:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 9m) 2024-04-17T10:02:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:02:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: data: 2024-04-17T10:02:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: volumes: 1/1 healthy 2024-04-17T10:02:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: pools: 3 pools, 97 pgs 2024-04-17T10:02:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: objects: 30 objects, 19 MiB 2024-04-17T10:02:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: usage: 294 MiB used, 720 GiB / 720 GiB avail 2024-04-17T10:02:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: pgs: 97 active+clean 2024-04-17T10:02:13.467 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:02:13.468 INFO:teuthology.orchestra.run.smithi032.stdout: io: 2024-04-17T10:02:13.468 INFO:teuthology.orchestra.run.smithi032.stdout: client: 85 B/s rd, 1.0 MiB/s wr, 0 op/s rd, 2 op/s wr 2024-04-17T10:02:13.468 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:02:13.468 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:02:13.468 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T10:02:13.112927+0000 mds.c [INF] scrub queued for path: / 2024-04-17T10:02:13.468 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T10:02:13.112930+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-17T10:02:13.468 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T10:02:13.112934+0000 mds.c [INF] scrub summary: active paths [/] 2024-04-17T10:02:13.468 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T10:02:13.114435+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-17T10:02:14.271 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:02:14.274 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:02:16.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:16.472+0000 7f8c00c00640 1 -- 172.21.15.32:0/3298135584 <== mon.0 v2:172.21.15.32:3300/0 8 ==== log(1 entries from seq 5 at 2024-04-17T10:02:16.328177+0000) ==== 189+0+0 (secure 0 0 0) 0x7f8bf8053f10 con 0x7f8c0409bb30 2024-04-17T10:02:16.472 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T10:02:16.328177+0000 mds.c [INF] scrub summary: idle 2024-04-17T10:02:43.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:02:43.341 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-17T10:02:43.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.500+0000 7f1209000640 1 Processor -- start 2024-04-17T10:02:43.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.500+0000 7f1209000640 1 -- start start 2024-04-17T10:02:43.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.500+0000 7f1209000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f120406c140 0x7f12040adb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.500+0000 7f1209000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f120415f9c0 0x7f12040b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.500+0000 7f1209000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12041629a0 con 0x7f120415c4c0 2024-04-17T10:02:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.500+0000 7f1209000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f1204162c80 con 0x7f120415f9c0 2024-04-17T10:02:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.500+0000 7f1209000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1204162f60 con 0x7f120406c140 2024-04-17T10:02:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.500+0000 7f1202800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f120415f9c0 0x7f12040b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.500+0000 7f1202800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f120415f9c0 0x7f12040b3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44086/0 (socket says 172.21.15.32:44086) 2024-04-17T10:02:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.500+0000 7f1202800640 1 -- 172.21.15.32:0/749226333 learned_addr learned my addr 172.21.15.32:0/749226333 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.500+0000 7f11fbe00640 1 --2- 172.21.15.32:0/749226333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f120406c140 0x7f12040adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.500+0000 7f1202800640 1 -- 172.21.15.32:0/749226333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f120406c140 msgr2=0x7f12040adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.500+0000 7f1202800640 1 --2- 172.21.15.32:0/749226333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f120406c140 0x7f12040adb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.500+0000 7f1202800640 1 -- 172.21.15.32:0/749226333 >> v1:172.21.15.144:6789/0 conn(0x7f120415c4c0 legacy=0x7f120406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.500+0000 7f1202800640 1 -- 172.21.15.32:0/749226333 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1204163240 con 0x7f120415f9c0 2024-04-17T10:02:43.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.504+0000 7f1202800640 1 --2- 172.21.15.32:0/749226333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f120415f9c0 0x7f12040b3870 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f11fc009e50 tx=0x7f11fc01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a826691d42e6f30d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:43.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.504+0000 7f1201400640 1 -- 172.21.15.32:0/749226333 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc0a5810 con 0x7f120415f9c0 2024-04-17T10:02:43.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.504+0000 7f1201400640 1 -- 172.21.15.32:0/749226333 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11fc0a59b0 con 0x7f120415f9c0 2024-04-17T10:02:43.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.504+0000 7f1201400640 1 -- 172.21.15.32:0/749226333 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc0a5d80 con 0x7f120415f9c0 2024-04-17T10:02:43.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.504+0000 7f1209000640 1 -- 172.21.15.32:0/749226333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f120415f9c0 msgr2=0x7f12040b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:43.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.504+0000 7f1209000640 1 --2- 172.21.15.32:0/749226333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f120415f9c0 0x7f12040b3870 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f11fc009e50 tx=0x7f11fc01ca30 comp rx=0 tx=0).stop 2024-04-17T10:02:43.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.504+0000 7f1209000640 1 -- 172.21.15.32:0/749226333 shutdown_connections 2024-04-17T10:02:43.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.504+0000 7f1209000640 1 --2- 172.21.15.32:0/749226333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f120415f9c0 0x7f12040b3870 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.504+0000 7f1209000640 1 --2- 172.21.15.32:0/749226333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f120406c140 0x7f12040adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.504+0000 7f1209000640 1 -- 172.21.15.32:0/749226333 >> 172.21.15.32:0/749226333 conn(0x7f12040b2120 msgr2=0x7f12040b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:43.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.504+0000 7f1209000640 1 -- 172.21.15.32:0/749226333 shutdown_connections 2024-04-17T10:02:43.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.504+0000 7f1209000640 1 -- 172.21.15.32:0/749226333 wait complete. 2024-04-17T10:02:43.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f1209000640 1 Processor -- start 2024-04-17T10:02:43.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f1209000640 1 -- start start 2024-04-17T10:02:43.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f1209000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f120406c140 0x7f120414d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f1209000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f120415c4c0 0x7f1204153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f1209000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f120415f9c0 0x7f120414d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f1209000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f12041626b0 con 0x7f120406c140 2024-04-17T10:02:43.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f1209000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1204162530 con 0x7f120415c4c0 2024-04-17T10:02:43.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f1209000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f120416b6a0 con 0x7f120415f9c0 2024-04-17T10:02:43.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f1201e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f120406c140 0x7f120414d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f1201e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f120406c140 0x7f120414d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44092/0 (socket says 172.21.15.32:44092) 2024-04-17T10:02:43.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f1201e00640 1 -- 172.21.15.32:0/2646253072 learned_addr learned my addr 172.21.15.32:0/2646253072 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:43.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f1202800640 1 --2- 172.21.15.32:0/2646253072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f120415f9c0 0x7f120414d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f11fbe00640 1 --2- 172.21.15.32:0/2646253072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f120415c4c0 0x7f1204153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f1201e00640 1 -- 172.21.15.32:0/2646253072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f120415f9c0 msgr2=0x7f120414d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:43.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f1201e00640 1 --2- 172.21.15.32:0/2646253072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f120415f9c0 0x7f120414d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f1201e00640 1 -- 172.21.15.32:0/2646253072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f120415c4c0 msgr2=0x7f1204153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:43.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f1201e00640 1 --2- 172.21.15.32:0/2646253072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f120415c4c0 0x7f1204153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f1201e00640 1 -- 172.21.15.32:0/2646253072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120414d7e0 con 0x7f120406c140 2024-04-17T10:02:43.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f1201e00640 1 --2- 172.21.15.32:0/2646253072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f120406c140 0x7f120414d2a0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f11f400c420 tx=0x7f11f400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:43.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f11fa000640 1 -- 172.21.15.32:0/2646253072 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f40052d0 con 0x7f120406c140 2024-04-17T10:02:43.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f11fa000640 1 -- 172.21.15.32:0/2646253072 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11f4005470 con 0x7f120406c140 2024-04-17T10:02:43.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f11fa000640 1 -- 172.21.15.32:0/2646253072 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f4007260 con 0x7f120406c140 2024-04-17T10:02:43.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f1209000640 1 -- 172.21.15.32:0/2646253072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120415b410 con 0x7f120406c140 2024-04-17T10:02:43.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.508+0000 7f1209000640 1 -- 172.21.15.32:0/2646253072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f120415ae50 con 0x7f120406c140 2024-04-17T10:02:43.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.512+0000 7f11fa000640 1 -- 172.21.15.32:0/2646253072 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f11f4007400 con 0x7f120406c140 2024-04-17T10:02:43.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.512+0000 7f11fa000640 1 --2- 172.21.15.32:0/2646253072 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f11e0068380 0x7f11e006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.512+0000 7f11fa000640 1 -- 172.21.15.32:0/2646253072 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f11f4085ed0 con 0x7f120406c140 2024-04-17T10:02:43.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.512+0000 7f11fbe00640 1 --2- 172.21.15.32:0/2646253072 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f11e0068380 0x7f11e006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.512+0000 7f120c3f9000 1 Processor -- start 2024-04-17T10:02:43.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.512+0000 7f120c3f9000 1 -- start start 2024-04-17T10:02:43.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.512+0000 7f11fbe00640 1 --2- 172.21.15.32:0/2646253072 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f11e0068380 0x7f11e006a840 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f12040b1fc0 tx=0x7f11ec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:43.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.512+0000 7f120c3f9000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf9261260 0x55aaf9261640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.512+0000 7f120c3f9000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261b80 0x55aaf9214460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.512+0000 7f120c3f9000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf92149a0 0x55aaf9214d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.512+0000 7f120c3f9000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55aaf90664f0 con 0x55aaf9261b80 2024-04-17T10:02:43.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.512+0000 7f120c3f9000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55aaf901f6c0 con 0x55aaf92149a0 2024-04-17T10:02:43.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.512+0000 7f120c3f9000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55aaf90624b0 con 0x55aaf9261260 2024-04-17T10:02:43.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.512+0000 7f11fbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261b80 0x55aaf9214460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.512+0000 7f11fbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261b80 0x55aaf9214460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44106/0 (socket says 172.21.15.32:44106) 2024-04-17T10:02:43.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.512+0000 7f11fbe00640 1 -- 172.21.15.32:0/4028017296 learned_addr learned my addr 172.21.15.32:0/4028017296 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:43.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f1202800640 1 --2- 172.21.15.32:0/4028017296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf92149a0 0x55aaf9214d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f1201e00640 1 --2- 172.21.15.32:0/4028017296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf9261260 0x55aaf9261640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f11fbe00640 1 -- 172.21.15.32:0/4028017296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf9261260 msgr2=0x55aaf9261640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:43.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f11fbe00640 1 --2- 172.21.15.32:0/4028017296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf9261260 0x55aaf9261640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f11fbe00640 1 -- 172.21.15.32:0/4028017296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf92149a0 msgr2=0x55aaf9214d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:43.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f11fbe00640 1 --2- 172.21.15.32:0/4028017296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf92149a0 0x55aaf9214d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f11fbe00640 1 -- 172.21.15.32:0/4028017296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaf90ab8e0 con 0x55aaf9261b80 2024-04-17T10:02:43.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f11fbe00640 1 --2- 172.21.15.32:0/4028017296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261b80 0x55aaf9214460 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f11ec004150 tx=0x7f11ec013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3733777e4b0701fd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:43.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f11f2a00640 1 -- 172.21.15.32:0/4028017296 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec027070 con 0x55aaf9261b80 2024-04-17T10:02:43.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f11f2a00640 1 -- 172.21.15.32:0/4028017296 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11ec003830 con 0x55aaf9261b80 2024-04-17T10:02:43.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f11f2a00640 1 -- 172.21.15.32:0/4028017296 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec003c20 con 0x55aaf9261b80 2024-04-17T10:02:43.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f120c3f9000 1 -- 172.21.15.32:0/4028017296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261b80 msgr2=0x55aaf9214460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:43.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f120c3f9000 1 --2- 172.21.15.32:0/4028017296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261b80 0x55aaf9214460 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f11ec004150 tx=0x7f11ec013000 comp rx=0 tx=0).stop 2024-04-17T10:02:43.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f120c3f9000 1 -- 172.21.15.32:0/4028017296 shutdown_connections 2024-04-17T10:02:43.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f120c3f9000 1 --2- 172.21.15.32:0/4028017296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf92149a0 0x55aaf9214d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f120c3f9000 1 --2- 172.21.15.32:0/4028017296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261b80 0x55aaf9214460 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f120c3f9000 1 --2- 172.21.15.32:0/4028017296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf9261260 0x55aaf9261640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f120c3f9000 1 -- 172.21.15.32:0/4028017296 >> 172.21.15.32:0/4028017296 conn(0x55aaf9242000 msgr2=0x55aaf92443f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:43.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f120c3f9000 1 -- 172.21.15.32:0/4028017296 shutdown_connections 2024-04-17T10:02:43.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f120c3f9000 1 -- 172.21.15.32:0/4028017296 wait complete. 2024-04-17T10:02:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f120c3f9000 1 -- start start 2024-04-17T10:02:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f120c3f9000 1 Processor -- start 2024-04-17T10:02:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f120c3f9000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf92149a0 0x55aaf921aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f120c3f9000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf9261260 0x55aaf921afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f120c3f9000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261b80 0x55aaf921b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f120c3f9000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55aaf90e77b0 con 0x55aaf9261b80 2024-04-17T10:02:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f120c3f9000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55aaf9121d00 con 0x55aaf9261260 2024-04-17T10:02:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f120c3f9000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55aaf902b2c0 con 0x55aaf92149a0 2024-04-17T10:02:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f120c3f9000 10 client.0 _ll_register_callbacks cb 0x55aaf903b030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:02:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f1202800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261b80 0x55aaf921b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f1202800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261b80 0x55aaf921b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44120/0 (socket says 172.21.15.32:44120) 2024-04-17T10:02:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.516+0000 7f1202800640 1 -- 172.21.15.32:0/1420692225 learned_addr learned my addr 172.21.15.32:0/1420692225 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f1201e00640 1 --2- 172.21.15.32:0/1420692225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf92149a0 0x55aaf921aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f11fbe00640 1 --2- 172.21.15.32:0/1420692225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf9261260 0x55aaf921afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f1202800640 1 -- 172.21.15.32:0/1420692225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf92149a0 msgr2=0x55aaf921aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:43.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f1202800640 1 --2- 172.21.15.32:0/1420692225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf92149a0 0x55aaf921aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f1202800640 1 -- 172.21.15.32:0/1420692225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf9261260 msgr2=0x55aaf921afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:43.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f1202800640 1 --2- 172.21.15.32:0/1420692225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf9261260 0x55aaf921afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f1202800640 1 -- 172.21.15.32:0/1420692225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaf903e490 con 0x55aaf9261b80 2024-04-17T10:02:43.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f1202800640 1 --2- 172.21.15.32:0/1420692225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261b80 0x55aaf921b520 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f11fc0af040 tx=0x7f11fc004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71943eab30ed5b2d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:43.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f11e7400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:02:43.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f11e7400640 1 -- 172.21.15.32:0/1420692225 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc0a5ac0 con 0x55aaf9261b80 2024-04-17T10:02:43.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f11e7400640 1 -- 172.21.15.32:0/1420692225 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f11fc0a5c60 con 0x55aaf9261b80 2024-04-17T10:02:43.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f120c3f9000 1 -- 172.21.15.32:0/1420692225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11ec003830 con 0x55aaf9261b80 2024-04-17T10:02:43.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f11e7400640 1 -- 172.21.15.32:0/1420692225 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f11fc0a5e60 con 0x55aaf9261b80 2024-04-17T10:02:43.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f120c3f9000 10 client.8943 fetch_fsmap learned FSMap version 81 2024-04-17T10:02:43.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f120c3f9000 1 -- 172.21.15.32:0/1420692225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x55aaf9031eb0 con 0x55aaf9261b80 2024-04-17T10:02:43.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f11e7400640 1 -- 172.21.15.32:0/1420692225 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11ec003830 con 0x55aaf9261b80 2024-04-17T10:02:43.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f11e7400640 1 -- 172.21.15.32:0/1420692225 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 81) ==== 3495+0+0 (secure 0 0 0) 0x7f11fc0c9050 con 0x55aaf9261b80 2024-04-17T10:02:43.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f120c3f9000 10 client.8943 fetch_fsmap finished waiting for FSMap version 81 2024-04-17T10:02:43.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f120c3f9000 10 client.8943 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-17T10:02:43.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f120c3f9000 20 client.8943 populate_metadata read hostname 'smithi032' 2024-04-17T10:02:43.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f120c3f9000 1 --2- 172.21.15.32:0/1420692225 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x55aaf921bf30 0x55aaf9273520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f120c3f9000 4 client.8943 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:02:43.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f120c3f9000 1 -- 172.21.15.32:0/1420692225 --> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aaf8fffdd0 con 0x55aaf921bf30 2024-04-17T10:02:43.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f1201e00640 1 --2- 172.21.15.32:0/1420692225 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x55aaf921bf30 0x55aaf9273520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f1201e00640 1 --2- 172.21.15.32:0/1420692225 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x55aaf921bf30 0x55aaf9273520 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eabef8799479f7e server_cookie=f80a6caa88a27959 in_seq=0 out_seq=0 2024-04-17T10:02:43.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f11e7400640 10 client.8943 ms_handle_connect on v2:172.21.15.32:6835/1235126741 2024-04-17T10:02:43.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f11e7400640 1 -- 172.21.15.32:0/1420692225 <== mds.0 v2:172.21.15.32:6835/1235126741 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aaf8fffdd0 con 0x55aaf921bf30 2024-04-17T10:02:43.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f11e7400640 10 client.8943 handle_command_reply: tid=0 2024-04-17T10:02:43.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f120c3f9000 1 client.8943 shutdown 2024-04-17T10:02:43.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f120c3f9000 1 -- 172.21.15.32:0/1420692225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261b80 msgr2=0x55aaf921b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:43.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f120c3f9000 1 --2- 172.21.15.32:0/1420692225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261b80 0x55aaf921b520 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f11fc0af040 tx=0x7f11fc004a50 comp rx=0 tx=0).stop 2024-04-17T10:02:43.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f120c3f9000 1 -- 172.21.15.32:0/1420692225 shutdown_connections 2024-04-17T10:02:43.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f120c3f9000 1 --2- 172.21.15.32:0/1420692225 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x55aaf921bf30 0x55aaf9273520 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f120c3f9000 1 --2- 172.21.15.32:0/1420692225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261b80 0x55aaf921b520 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f120c3f9000 1 --2- 172.21.15.32:0/1420692225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf9261260 0x55aaf921afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f11e7400640 0 client.8943 ms_handle_reset on v2:172.21.15.32:6835/1235126741 2024-04-17T10:02:43.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f120c3f9000 1 --2- 172.21.15.32:0/1420692225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf92149a0 0x55aaf921aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f120c3f9000 1 -- 172.21.15.32:0/1420692225 >> 172.21.15.32:0/1420692225 conn(0x55aaf9242000 msgr2=0x55aaf9213b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:43.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f120c3f9000 1 -- 172.21.15.32:0/1420692225 shutdown_connections 2024-04-17T10:02:43.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f120c3f9000 1 -- 172.21.15.32:0/1420692225 wait complete. 2024-04-17T10:02:43.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.520+0000 7f120c3f9000 20 client.8943 trim_cache size 0 max 16384 2024-04-17T10:02:43.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.540+0000 7f120c3f9000 1 Processor -- start 2024-04-17T10:02:43.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.540+0000 7f120c3f9000 1 -- start start 2024-04-17T10:02:43.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.540+0000 7f120c3f9000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf92149a0 0x55aaf925f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.540+0000 7f120c3f9000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf9261260 0x55aaf925f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.540+0000 7f120c3f9000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261b80 0x55aaf9273520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.540+0000 7f120c3f9000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1204202ed0 con 0x55aaf9261b80 2024-04-17T10:02:43.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.540+0000 7f120c3f9000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1204160f20 con 0x55aaf9261260 2024-04-17T10:02:43.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.540+0000 7f120c3f9000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1204161570 con 0x55aaf92149a0 2024-04-17T10:02:43.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.540+0000 7f1202800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261b80 0x55aaf9273520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.540+0000 7f1202800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261b80 0x55aaf9273520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44124/0 (socket says 172.21.15.32:44124) 2024-04-17T10:02:43.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.540+0000 7f1202800640 1 -- 172.21.15.32:0/2883846189 learned_addr learned my addr 172.21.15.32:0/2883846189 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:43.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.540+0000 7f11fbe00640 1 --2- 172.21.15.32:0/2883846189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf9261260 0x55aaf925f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.540+0000 7f1201e00640 1 --2- 172.21.15.32:0/2883846189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf92149a0 0x55aaf925f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.540+0000 7f1202800640 1 -- 172.21.15.32:0/2883846189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf92149a0 msgr2=0x55aaf925f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:43.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.540+0000 7f1202800640 1 --2- 172.21.15.32:0/2883846189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf92149a0 0x55aaf925f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.540+0000 7f1202800640 1 -- 172.21.15.32:0/2883846189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf9261260 msgr2=0x55aaf925f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:43.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.540+0000 7f1202800640 1 --2- 172.21.15.32:0/2883846189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf9261260 0x55aaf925f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.540+0000 7f1202800640 1 -- 172.21.15.32:0/2883846189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaf9020a20 con 0x55aaf9261b80 2024-04-17T10:02:43.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.540+0000 7f1202800640 1 --2- 172.21.15.32:0/2883846189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261b80 0x55aaf9273520 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f11fc01d2d0 tx=0x7f11fc0a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e989fd351e9cb73 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:43.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.540+0000 7f11f2000640 1 -- 172.21.15.32:0/2883846189 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc0a5810 con 0x55aaf9261b80 2024-04-17T10:02:43.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.540+0000 7f11f2000640 1 -- 172.21.15.32:0/2883846189 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11fc0a5ac0 con 0x55aaf9261b80 2024-04-17T10:02:43.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.540+0000 7f11f2000640 1 -- 172.21.15.32:0/2883846189 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc003ba0 con 0x55aaf9261b80 2024-04-17T10:02:43.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f120c3f9000 1 -- 172.21.15.32:0/2883846189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261b80 msgr2=0x55aaf9273520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:43.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f120c3f9000 1 --2- 172.21.15.32:0/2883846189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261b80 0x55aaf9273520 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f11fc01d2d0 tx=0x7f11fc0a7000 comp rx=0 tx=0).stop 2024-04-17T10:02:43.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f120c3f9000 1 -- 172.21.15.32:0/2883846189 shutdown_connections 2024-04-17T10:02:43.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f120c3f9000 1 --2- 172.21.15.32:0/2883846189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261b80 0x55aaf9273520 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f120c3f9000 1 --2- 172.21.15.32:0/2883846189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf9261260 0x55aaf925f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f120c3f9000 1 --2- 172.21.15.32:0/2883846189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf92149a0 0x55aaf925f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f120c3f9000 1 -- 172.21.15.32:0/2883846189 >> 172.21.15.32:0/2883846189 conn(0x55aaf9242000 msgr2=0x55aaf9276770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:43.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f120c3f9000 1 -- 172.21.15.32:0/2883846189 shutdown_connections 2024-04-17T10:02:43.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f120c3f9000 1 -- 172.21.15.32:0/2883846189 wait complete. 2024-04-17T10:02:43.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f120c3f9000 1 -- start start 2024-04-17T10:02:43.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f120c3f9000 1 Processor -- start 2024-04-17T10:02:43.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f120c3f9000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf92149a0 0x55aaf9276140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f120c3f9000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261260 0x55aaf928d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f120c3f9000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf9261b80 0x55aaf928d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f120c3f9000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55aaf900a050 con 0x55aaf9261260 2024-04-17T10:02:43.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f120c3f9000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55aaf9028570 con 0x55aaf92149a0 2024-04-17T10:02:43.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f120c3f9000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55aaf9065800 con 0x55aaf9261b80 2024-04-17T10:02:43.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f120c3f9000 10 client.0 _ll_register_callbacks cb 0x7f11fc009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:02:43.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f11fbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261260 0x55aaf928d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f11fbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261260 0x55aaf928d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44136/0 (socket says 172.21.15.32:44136) 2024-04-17T10:02:43.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f11fbe00640 1 -- 172.21.15.32:0/3814792328 learned_addr learned my addr 172.21.15.32:0/3814792328 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:43.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f1202800640 1 --2- 172.21.15.32:0/3814792328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf9261b80 0x55aaf928d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f1201e00640 1 --2- 172.21.15.32:0/3814792328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf92149a0 0x55aaf9276140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f11fbe00640 1 -- 172.21.15.32:0/3814792328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf9261b80 msgr2=0x55aaf928d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:43.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f11fbe00640 1 --2- 172.21.15.32:0/3814792328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf9261b80 0x55aaf928d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f11fbe00640 1 -- 172.21.15.32:0/3814792328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf92149a0 msgr2=0x55aaf9276140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:43.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f11fbe00640 1 --2- 172.21.15.32:0/3814792328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf92149a0 0x55aaf9276140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f11fbe00640 1 -- 172.21.15.32:0/3814792328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaf90e7c40 con 0x55aaf9261260 2024-04-17T10:02:43.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f1202800640 1 --2- 172.21.15.32:0/3814792328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf9261b80 0x55aaf928d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:02:43.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f1201e00640 1 --2- 172.21.15.32:0/3814792328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf92149a0 0x55aaf9276140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:02:43.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f11fbe00640 1 --2- 172.21.15.32:0/3814792328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261260 0x55aaf928d0b0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x55aaf90254b0 tx=0x7f11ec004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5c1f03f683c7bd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:43.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f11e7400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:02:43.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f11e7400640 1 -- 172.21.15.32:0/3814792328 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec027040 con 0x55aaf9261260 2024-04-17T10:02:43.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f11e7400640 1 -- 172.21.15.32:0/3814792328 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f11ec016690 con 0x55aaf9261260 2024-04-17T10:02:43.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f120c3f9000 1 -- 172.21.15.32:0/3814792328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11fc0a5ac0 con 0x55aaf9261260 2024-04-17T10:02:43.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f11e7400640 1 -- 172.21.15.32:0/3814792328 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f11ec016890 con 0x55aaf9261260 2024-04-17T10:02:43.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f120c3f9000 10 client.8949 fetch_fsmap learned FSMap version 81 2024-04-17T10:02:43.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f120c3f9000 1 -- 172.21.15.32:0/3814792328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x55aaf900b600 con 0x55aaf9261260 2024-04-17T10:02:43.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.544+0000 7f11e7400640 1 -- 172.21.15.32:0/3814792328 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11fc0a5ac0 con 0x55aaf9261260 2024-04-17T10:02:43.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f11e7400640 1 -- 172.21.15.32:0/3814792328 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 81) ==== 3495+0+0 (secure 0 0 0) 0x7f11ec03e080 con 0x55aaf9261260 2024-04-17T10:02:43.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f120c3f9000 10 client.8949 fetch_fsmap finished waiting for FSMap version 81 2024-04-17T10:02:43.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f120c3f9000 10 client.8949 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-17T10:02:43.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f120c3f9000 20 client.8949 populate_metadata read hostname 'smithi032' 2024-04-17T10:02:43.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f120c3f9000 1 --2- 172.21.15.32:0/3814792328 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x55aaf92d0f30 0x55aaf92d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f120c3f9000 4 client.8949 mds_command: new command op to 8577 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:02:43.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f120c3f9000 1 -- 172.21.15.32:0/3814792328 --> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aaf9073f10 con 0x55aaf92d0f30 2024-04-17T10:02:43.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f1201e00640 1 --2- 172.21.15.32:0/3814792328 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x55aaf92d0f30 0x55aaf92d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f1201e00640 1 --2- 172.21.15.32:0/3814792328 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x55aaf92d0f30 0x55aaf92d1310 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf4a7a26e80c15ae server_cookie=af182089f5a2c6af in_seq=0 out_seq=0 2024-04-17T10:02:43.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f11e7400640 10 client.8949 ms_handle_connect on v2:172.21.15.32:6835/1235126741 2024-04-17T10:02:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f11e7400640 1 -- 172.21.15.32:0/3814792328 <== mds.0 v2:172.21.15.32:6835/1235126741 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55aaf9073f10 con 0x55aaf92d0f30 2024-04-17T10:02:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f11e7400640 10 client.8949 handle_command_reply: tid=0 2024-04-17T10:02:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f120c3f9000 1 client.8949 shutdown 2024-04-17T10:02:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f120c3f9000 1 -- 172.21.15.32:0/3814792328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261260 msgr2=0x55aaf928d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f120c3f9000 1 --2- 172.21.15.32:0/3814792328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261260 0x55aaf928d0b0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x55aaf90254b0 tx=0x7f11ec004740 comp rx=0 tx=0).stop 2024-04-17T10:02:43.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f120c3f9000 1 -- 172.21.15.32:0/3814792328 shutdown_connections 2024-04-17T10:02:43.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f120c3f9000 1 --2- 172.21.15.32:0/3814792328 >> [v2:172.21.15.32:6835/1235126741,v1:172.21.15.32:6837/1235126741] conn(0x55aaf92d0f30 0x55aaf92d1310 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f120c3f9000 1 --2- 172.21.15.32:0/3814792328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf9261b80 0x55aaf928d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f120c3f9000 1 --2- 172.21.15.32:0/3814792328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aaf9261260 0x55aaf928d0b0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f120c3f9000 1 --2- 172.21.15.32:0/3814792328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf92149a0 0x55aaf9276140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f11e7400640 0 client.8949 ms_handle_reset on v2:172.21.15.32:6835/1235126741 2024-04-17T10:02:43.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f120c3f9000 1 -- 172.21.15.32:0/3814792328 >> 172.21.15.32:0/3814792328 conn(0x55aaf9242000 msgr2=0x55aaf9260050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:43.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f120c3f9000 1 -- 172.21.15.32:0/3814792328 shutdown_connections 2024-04-17T10:02:43.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f120c3f9000 1 -- 172.21.15.32:0/3814792328 wait complete. 2024-04-17T10:02:43.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f120c3f9000 20 client.8949 trim_cache size 0 max 16384 2024-04-17T10:02:43.549 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:02:43.550 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-17T10:02:43.550 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-17T10:02:43.550 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:02:43.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f1209000640 1 -- 172.21.15.32:0/2646253072 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f11e0068380 msgr2=0x7f11e006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f1209000640 1 --2- 172.21.15.32:0/2646253072 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f11e0068380 0x7f11e006a840 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f12040b1fc0 tx=0x7f11ec008040 comp rx=0 tx=0).stop 2024-04-17T10:02:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f1209000640 1 -- 172.21.15.32:0/2646253072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f120406c140 msgr2=0x7f120414d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f1209000640 1 --2- 172.21.15.32:0/2646253072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f120406c140 0x7f120414d2a0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f11f400c420 tx=0x7f11f400c8e0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f1209000640 1 -- 172.21.15.32:0/2646253072 shutdown_connections 2024-04-17T10:02:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f1209000640 1 --2- 172.21.15.32:0/2646253072 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f11e0068380 0x7f11e006a840 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f1209000640 1 --2- 172.21.15.32:0/2646253072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f120415f9c0 0x7f120414d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f1209000640 1 --2- 172.21.15.32:0/2646253072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f120415c4c0 0x7f1204153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f1209000640 1 --2- 172.21.15.32:0/2646253072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f120406c140 0x7f120414d2a0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f1209000640 1 -- 172.21.15.32:0/2646253072 >> 172.21.15.32:0/2646253072 conn(0x7f12040b2120 msgr2=0x7f120406ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f1209000640 1 -- 172.21.15.32:0/2646253072 shutdown_connections 2024-04-17T10:02:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.548+0000 7f1209000640 1 -- 172.21.15.32:0/2646253072 wait complete. 2024-04-17T10:02:43.565 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T10:02:43.666 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T10:02:43.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.808+0000 7f3faca00640 1 Processor -- start 2024-04-17T10:02:43.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.808+0000 7f3faca00640 1 -- start start 2024-04-17T10:02:43.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.812+0000 7f3faca00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fa813f6e0 0x7f3fa80cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.812+0000 7f3faca00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3fa80cf370 0x7f3fa80d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.812+0000 7f3faca00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3fa80d0820 0x7f3fa8141d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.812+0000 7f3faca00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3fa813fac0 con 0x7f3fa80cf370 2024-04-17T10:02:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.812+0000 7f3faca00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f3fa81601e0 con 0x7f3fa80d0820 2024-04-17T10:02:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.812+0000 7f3faca00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3fa8160480 con 0x7f3fa813f6e0 2024-04-17T10:02:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.812+0000 7f3f9d400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3fa80cf370 0x7f3fa80d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.812+0000 7f3f9d400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3fa80cf370 0x7f3fa80d02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44140/0 (socket says 172.21.15.32:44140) 2024-04-17T10:02:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.812+0000 7f3f9d400640 1 -- 172.21.15.32:0/2811994361 learned_addr learned my addr 172.21.15.32:0/2811994361 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.812+0000 7f3f9d400640 1 -- 172.21.15.32:0/2811994361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fa813f6e0 msgr2=0x7f3fa80cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.812+0000 7f3fa5e00640 1 --2- 172.21.15.32:0/2811994361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fa813f6e0 0x7f3fa80cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.812+0000 7f3fa6800640 1 --2- 172.21.15.32:0/2811994361 >> v2:172.21.15.144:3300/0 conn(0x7f3fa80d0820 0x7f3fa8141d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.812+0000 7f3f9d400640 1 --2- 172.21.15.32:0/2811994361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fa813f6e0 0x7f3fa80cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.812+0000 7f3f9d400640 1 -- 172.21.15.32:0/2811994361 >> v2:172.21.15.144:3300/0 conn(0x7f3fa80d0820 msgr2=0x7f3fa8141d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.812+0000 7f3f9d400640 1 --2- 172.21.15.32:0/2811994361 >> v2:172.21.15.144:3300/0 conn(0x7f3fa80d0820 0x7f3fa8141d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.812+0000 7f3f9d400640 1 -- 172.21.15.32:0/2811994361 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa8160760 con 0x7f3fa80cf370 2024-04-17T10:02:43.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.812+0000 7f3fa5e00640 1 --2- 172.21.15.32:0/2811994361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fa813f6e0 0x7f3fa80cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:02:43.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.812+0000 7f3fa6800640 1 --2- 172.21.15.32:0/2811994361 >> v2:172.21.15.144:3300/0 conn(0x7f3fa80d0820 0x7f3fa8141d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:02:43.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.816+0000 7f3f9d400640 1 --2- 172.21.15.32:0/2811994361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fa80cf370 0x7f3fa80d02e0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f3fa806db20 tx=0x7f3f9001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675c518a1e6ae228 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:43.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.816+0000 7f3fa5400640 1 -- 172.21.15.32:0/2811994361 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f900b1070 con 0x7f3fa80cf370 2024-04-17T10:02:43.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.816+0000 7f3fa5400640 1 -- 172.21.15.32:0/2811994361 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f900ac630 con 0x7f3fa80cf370 2024-04-17T10:02:43.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.816+0000 7f3fa5400640 1 -- 172.21.15.32:0/2811994361 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f900a8050 con 0x7f3fa80cf370 2024-04-17T10:02:43.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.816+0000 7f3faca00640 1 -- 172.21.15.32:0/2811994361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fa80cf370 msgr2=0x7f3fa80d02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:43.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.816+0000 7f3faca00640 1 --2- 172.21.15.32:0/2811994361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fa80cf370 0x7f3fa80d02e0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f3fa806db20 tx=0x7f3f9001c520 comp rx=0 tx=0).stop 2024-04-17T10:02:43.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.816+0000 7f3faca00640 1 -- 172.21.15.32:0/2811994361 shutdown_connections 2024-04-17T10:02:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.816+0000 7f3faca00640 1 --2- 172.21.15.32:0/2811994361 >> v2:172.21.15.144:3300/0 conn(0x7f3fa80d0820 0x7f3fa8141d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.816+0000 7f3faca00640 1 --2- 172.21.15.32:0/2811994361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fa80cf370 0x7f3fa80d02e0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.816+0000 7f3faca00640 1 --2- 172.21.15.32:0/2811994361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fa813f6e0 0x7f3fa80cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.816+0000 7f3faca00640 1 -- 172.21.15.32:0/2811994361 >> 172.21.15.32:0/2811994361 conn(0x7f3fa806ba30 msgr2=0x7f3fa806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.816+0000 7f3faca00640 1 -- 172.21.15.32:0/2811994361 shutdown_connections 2024-04-17T10:02:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.816+0000 7f3faca00640 1 -- 172.21.15.32:0/2811994361 wait complete. 2024-04-17T10:02:43.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.816+0000 7f3faca00640 1 Processor -- start 2024-04-17T10:02:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.816+0000 7f3faca00640 1 -- start start 2024-04-17T10:02:43.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3faca00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fa80cf370 0x7f3fa813ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3faca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fa80d0820 0x7f3fa80b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3faca00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fa813f6e0 0x7f3fa80b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3faca00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa80cff10 con 0x7f3fa80d0820 2024-04-17T10:02:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3faca00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa80cfd90 con 0x7f3fa813f6e0 2024-04-17T10:02:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3faca00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3fa8071d80 con 0x7f3fa80cf370 2024-04-17T10:02:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3f9d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fa80d0820 0x7f3fa80b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3f9d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fa80d0820 0x7f3fa80b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44142/0 (socket says 172.21.15.32:44142) 2024-04-17T10:02:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3f9d400640 1 -- 172.21.15.32:0/2290141386 learned_addr learned my addr 172.21.15.32:0/2290141386 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3fa5e00640 1 --2- 172.21.15.32:0/2290141386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fa80cf370 0x7f3fa813ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3fa6800640 1 --2- 172.21.15.32:0/2290141386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fa813f6e0 0x7f3fa80b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3f9d400640 1 -- 172.21.15.32:0/2290141386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fa80cf370 msgr2=0x7f3fa813ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3f9d400640 1 --2- 172.21.15.32:0/2290141386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fa80cf370 0x7f3fa813ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3f9d400640 1 -- 172.21.15.32:0/2290141386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fa813f6e0 msgr2=0x7f3fa80b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3f9d400640 1 --2- 172.21.15.32:0/2290141386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fa813f6e0 0x7f3fa80b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:43.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3f9d400640 1 -- 172.21.15.32:0/2290141386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa80b1ee0 con 0x7f3fa80d0820 2024-04-17T10:02:43.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3f9d400640 1 --2- 172.21.15.32:0/2290141386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fa80d0820 0x7f3fa80b1820 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f3f900af040 tx=0x7f3f900192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:43.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3f9ea00640 1 -- 172.21.15.32:0/2290141386 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f900b1070 con 0x7f3fa80d0820 2024-04-17T10:02:43.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3f9ea00640 1 -- 172.21.15.32:0/2290141386 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f900ab800 con 0x7f3fa80d0820 2024-04-17T10:02:43.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3f9ea00640 1 -- 172.21.15.32:0/2290141386 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f900a8040 con 0x7f3fa80d0820 2024-04-17T10:02:43.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3faca00640 1 -- 172.21.15.32:0/2290141386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa80b21c0 con 0x7f3fa80d0820 2024-04-17T10:02:43.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3faca00640 1 -- 172.21.15.32:0/2290141386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa80b2780 con 0x7f3fa80d0820 2024-04-17T10:02:43.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3faca00640 1 -- 172.21.15.32:0/2290141386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fa8068a90 con 0x7f3fa80d0820 2024-04-17T10:02:43.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.820+0000 7f3f9ea00640 1 -- 172.21.15.32:0/2290141386 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3f900be060 con 0x7f3fa80d0820 2024-04-17T10:02:43.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.824+0000 7f3f9ea00640 1 --2- 172.21.15.32:0/2290141386 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3f84068380 0x7f3f8406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:43.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.824+0000 7f3f9ea00640 1 -- 172.21.15.32:0/2290141386 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f3f9011ff70 con 0x7f3fa80d0820 2024-04-17T10:02:43.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.824+0000 7f3fa5e00640 1 --2- 172.21.15.32:0/2290141386 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3f84068380 0x7f3f8406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:43.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.824+0000 7f3fa5e00640 1 --2- 172.21.15.32:0/2290141386 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3f84068380 0x7f3f8406a840 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3f98006fd0 tx=0x7f3f98008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:43.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:43.828+0000 7f3f9ea00640 1 -- 172.21.15.32:0/2290141386 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3f90016110 con 0x7f3fa80d0820 2024-04-17T10:02:44.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.032+0000 7f3faca00640 1 -- 172.21.15.32:0/2290141386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3fa8159140 con 0x7f3fa80d0820 2024-04-17T10:02:44.029 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-17T10:02:44.032+0000 7f93c9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T10:02:44.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.032+0000 7f3f9ea00640 1 -- 172.21.15.32:0/2290141386 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v82) ==== 161+0+0 (secure 0 0 0) 0x7f3f900efe40 con 0x7f3fa80d0820 2024-04-17T10:02:44.031 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T10:02:44.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.036+0000 7f3faca00640 1 -- 172.21.15.32:0/2290141386 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3f84068380 msgr2=0x7f3f8406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.036+0000 7f3faca00640 1 --2- 172.21.15.32:0/2290141386 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3f84068380 0x7f3f8406a840 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3f98006fd0 tx=0x7f3f98008040 comp rx=0 tx=0).stop 2024-04-17T10:02:44.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.036+0000 7f3faca00640 1 -- 172.21.15.32:0/2290141386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fa80d0820 msgr2=0x7f3fa80b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.036+0000 7f3faca00640 1 --2- 172.21.15.32:0/2290141386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fa80d0820 0x7f3fa80b1820 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f3f900af040 tx=0x7f3f900192c0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.036+0000 7f3faca00640 1 -- 172.21.15.32:0/2290141386 shutdown_connections 2024-04-17T10:02:44.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.036+0000 7f3faca00640 1 --2- 172.21.15.32:0/2290141386 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3f84068380 0x7f3f8406a840 secure :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3f98006fd0 tx=0x7f3f98008040 comp rx=0 tx=0).stop 2024-04-17T10:02:44.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.036+0000 7f3faca00640 1 --2- 172.21.15.32:0/2290141386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fa813f6e0 0x7f3fa80b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.036+0000 7f3faca00640 1 --2- 172.21.15.32:0/2290141386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fa80d0820 0x7f3fa80b1820 secure :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f3f900af040 tx=0x7f3f900192c0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.036+0000 7f3faca00640 1 --2- 172.21.15.32:0/2290141386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fa80cf370 0x7f3fa813ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.036+0000 7f3faca00640 1 -- 172.21.15.32:0/2290141386 >> 172.21.15.32:0/2290141386 conn(0x7f3fa806ba30 msgr2=0x7f3fa806ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:44.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.036+0000 7f3faca00640 1 -- 172.21.15.32:0/2290141386 shutdown_connections 2024-04-17T10:02:44.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.036+0000 7f3faca00640 1 -- 172.21.15.32:0/2290141386 wait complete. 2024-04-17T10:02:44.044 INFO:tasks.ceph.mds.c.smithi032.stderr: -6471> 2024-04-17T10:01:43.117+0000 7fd06f000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:02:44.055 DEBUG:teuthology.orchestra.run.smithi032:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-04-17T10:02:44.095 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-17T10:02:44.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f683af7e400 1 -- start start 2024-04-17T10:02:44.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f683af7e400 1 Processor -- start 2024-04-17T10:02:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f683af7e400 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beed685000 0x55beee454000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f683af7e400 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55beed685400 0x55beee454580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f683af7e400 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55beed685800 0x55beee454b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f683af7e400 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55beee478780 con 0x55beed685400 2024-04-17T10:02:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f683af7e400 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55beee478600 con 0x55beed685000 2024-04-17T10:02:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f683af7e400 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55beee478480 con 0x55beed685800 2024-04-17T10:02:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f6838600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55beed685400 0x55beee454580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f6838600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55beed685400 0x55beee454580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44180/0 (socket says 172.21.15.32:44180) 2024-04-17T10:02:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f6838600640 1 -- 172.21.15.32:0/1988629188 learned_addr learned my addr 172.21.15.32:0/1988629188 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f6839000640 1 --2- 172.21.15.32:0/1988629188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beed685000 0x55beee454000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f6839a00640 1 --2- 172.21.15.32:0/1988629188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55beed685800 0x55beee454b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f6838600640 1 -- 172.21.15.32:0/1988629188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55beed685800 msgr2=0x55beee454b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f6838600640 1 --2- 172.21.15.32:0/1988629188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55beed685800 0x55beee454b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f6838600640 1 -- 172.21.15.32:0/1988629188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beed685000 msgr2=0x55beee454000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f6838600640 1 --2- 172.21.15.32:0/1988629188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beed685000 0x55beee454000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f6838600640 1 -- 172.21.15.32:0/1988629188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beed7f8b40 con 0x55beed685400 2024-04-17T10:02:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f6839000640 1 --2- 172.21.15.32:0/1988629188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beed685000 0x55beee454000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:02:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f6839a00640 1 --2- 172.21.15.32:0/1988629188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55beed685800 0x55beee454b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:02:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f6838600640 1 --2- 172.21.15.32:0/1988629188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55beed685400 0x55beee454580 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x55beee4ff470 tx=0x55beed760f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a28cd87f0fa6afc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f6835e00640 1 -- 172.21.15.32:0/1988629188 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55beed6ba4e0 con 0x55beed685400 2024-04-17T10:02:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f6835e00640 1 -- 172.21.15.32:0/1988629188 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55beed6d01c0 con 0x55beed685400 2024-04-17T10:02:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f6835e00640 1 -- 172.21.15.32:0/1988629188 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55beed6ba9c0 con 0x55beed685400 2024-04-17T10:02:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f683af7e400 1 -- 172.21.15.32:0/1988629188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beed7f8f00 con 0x55beed685400 2024-04-17T10:02:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f683af7e400 1 -- 172.21.15.32:0/1988629188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55beed7f90e0 con 0x55beed685400 2024-04-17T10:02:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f683af7e400 4 waiting for MDS map... 2024-04-17T10:02:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f6835e00640 1 -- 172.21.15.32:0/1988629188 <== mon.0 v2:172.21.15.32:3300/0 4 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x55beed7f7c00 con 0x55beed685400 2024-04-17T10:02:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f6835e00640 1 -- 172.21.15.32:0/1988629188 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 82) ==== 2886+0+0 (secure 0 0 0) 0x55beed6db180 con 0x55beed685400 2024-04-17T10:02:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f683af7e400 4 Got MDS map 82 2024-04-17T10:02:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f683af7e400 10 main: JournalTool::main 2024-04-17T10:02:44.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f683af7e400 4 main: JournalTool: connecting to RADOS... 2024-04-17T10:02:44.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f683af7e400 1 Processor -- start 2024-04-17T10:02:44.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.108+0000 7f683af7e400 1 -- start start 2024-04-17T10:02:44.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f683af7e400 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beee5a5400 0x55beee455080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f683af7e400 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55beee5a5800 0x55beee455600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f683af7e400 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55beee5a5c00 0x55beee455b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f683af7e400 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55beee478300 con 0x55beee5a5c00 2024-04-17T10:02:44.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f683af7e400 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55beee478000 con 0x55beee5a5400 2024-04-17T10:02:44.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f683af7e400 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55beed6e4900 con 0x55beee5a5800 2024-04-17T10:02:44.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f6838600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55beee5a5c00 0x55beee455b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f6838600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55beee5a5c00 0x55beee455b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44186/0 (socket says 172.21.15.32:44186) 2024-04-17T10:02:44.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f6838600640 1 -- 172.21.15.32:0/1422559372 learned_addr learned my addr 172.21.15.32:0/1422559372 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:44.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f6839a00640 1 --2- 172.21.15.32:0/1422559372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beee5a5400 0x55beee455080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f6839000640 1 --2- 172.21.15.32:0/1422559372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55beee5a5800 0x55beee455600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f6838600640 1 -- 172.21.15.32:0/1422559372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55beee5a5800 msgr2=0x55beee455600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f6838600640 1 --2- 172.21.15.32:0/1422559372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55beee5a5800 0x55beee455600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f6838600640 1 -- 172.21.15.32:0/1422559372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beee5a5400 msgr2=0x55beee455080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f6838600640 1 --2- 172.21.15.32:0/1422559372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beee5a5400 0x55beee455080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f6838600640 1 -- 172.21.15.32:0/1422559372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beed7f8b40 con 0x55beee5a5c00 2024-04-17T10:02:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f6839a00640 1 --2- 172.21.15.32:0/1422559372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beee5a5400 0x55beee455080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:02:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f6838600640 1 --2- 172.21.15.32:0/1422559372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55beee5a5c00 0x55beee455b80 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x55beee4ff860 tx=0x55beed760ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7269b0bd27ffccf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f6833600640 1 -- 172.21.15.32:0/1422559372 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55beed6ba4e0 con 0x55beee5a5c00 2024-04-17T10:02:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f6833600640 1 -- 172.21.15.32:0/1422559372 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55beed6d0e00 con 0x55beee5a5c00 2024-04-17T10:02:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f6833600640 1 -- 172.21.15.32:0/1422559372 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55beed6ba9c0 con 0x55beee5a5c00 2024-04-17T10:02:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f683af7e400 1 -- 172.21.15.32:0/1422559372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55beee5a5c00 msgr2=0x55beee455b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f683af7e400 1 --2- 172.21.15.32:0/1422559372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55beee5a5c00 0x55beee455b80 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x55beee4ff860 tx=0x55beed760ea0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f683af7e400 1 -- 172.21.15.32:0/1422559372 shutdown_connections 2024-04-17T10:02:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f683af7e400 1 --2- 172.21.15.32:0/1422559372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55beee5a5800 0x55beee455600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f683af7e400 1 --2- 172.21.15.32:0/1422559372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55beee5a5c00 0x55beee455b80 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f683af7e400 1 --2- 172.21.15.32:0/1422559372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beee5a5400 0x55beee455080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f683af7e400 1 -- 172.21.15.32:0/1422559372 >> 172.21.15.32:0/1422559372 conn(0x55beee5a5000 msgr2=0x55beed6b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f683af7e400 1 -- 172.21.15.32:0/1422559372 shutdown_connections 2024-04-17T10:02:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f683af7e400 1 -- 172.21.15.32:0/1422559372 wait complete. 2024-04-17T10:02:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f683af7e400 1 Processor -- start 2024-04-17T10:02:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f683af7e400 1 -- start start 2024-04-17T10:02:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f683af7e400 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beee5a5400 0x55beee455080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f683af7e400 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55beee5a5800 0x55beee455600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f683af7e400 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55beee5a5c00 0x55beee455b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f683af7e400 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55beee479500 con 0x55beee5a5c00 2024-04-17T10:02:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f683af7e400 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55beee479680 con 0x55beee5a5400 2024-04-17T10:02:44.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f683af7e400 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55beee478000 con 0x55beee5a5800 2024-04-17T10:02:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f6838600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55beee5a5c00 0x55beee455b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f6838600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55beee5a5c00 0x55beee455b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44196/0 (socket says 172.21.15.32:44196) 2024-04-17T10:02:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f6838600640 1 -- 172.21.15.32:0/665646036 learned_addr learned my addr 172.21.15.32:0/665646036 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f6839a00640 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beee5a5400 0x55beee455080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.112+0000 7f6839000640 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55beee5a5800 0x55beee455600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.116+0000 7f6838600640 1 -- 172.21.15.32:0/665646036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55beee5a5800 msgr2=0x55beee455600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.116+0000 7f6838600640 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55beee5a5800 0x55beee455600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.116+0000 7f6838600640 1 -- 172.21.15.32:0/665646036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beee5a5400 msgr2=0x55beee455080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.116+0000 7f6838600640 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beee5a5400 0x55beee455080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.116+0000 7f6838600640 1 -- 172.21.15.32:0/665646036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beed7f9a40 con 0x55beee5a5c00 2024-04-17T10:02:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.116+0000 7f6839a00640 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beee5a5400 0x55beee455080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:02:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.116+0000 7f6838600640 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55beee5a5c00 0x55beee455b80 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x55beee5b7440 tx=0x55beee5c2c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.116+0000 7f6831800640 1 -- 172.21.15.32:0/665646036 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55beed6bab60 con 0x55beee5a5c00 2024-04-17T10:02:44.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.116+0000 7f6831800640 1 -- 172.21.15.32:0/665646036 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55beed6d0c40 con 0x55beee5a5c00 2024-04-17T10:02:44.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.116+0000 7f6831800640 1 -- 172.21.15.32:0/665646036 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55beed6ba4e0 con 0x55beee5a5c00 2024-04-17T10:02:44.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.116+0000 7f683af7e400 1 -- 172.21.15.32:0/665646036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55beed7f9c20 con 0x55beee5a5c00 2024-04-17T10:02:44.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.116+0000 7f683af7e400 1 -- 172.21.15.32:0/665646036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beed976000 con 0x55beee5a5c00 2024-04-17T10:02:44.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.116+0000 7f683af7e400 4 main: JournalTool: resolving pool 14 2024-04-17T10:02:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.116+0000 7f6831800640 1 -- 172.21.15.32:0/665646036 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x55beee602c00 con 0x55beee5a5c00 2024-04-17T10:02:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.116+0000 7f6831800640 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55beed973000 0x55beee456100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.116+0000 7f6831800640 1 -- 172.21.15.32:0/665646036 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x55beed974200 con 0x55beee5a5c00 2024-04-17T10:02:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.116+0000 7f683af7e400 4 main: JournalTool: creating IoCtx.. 2024-04-17T10:02:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.116+0000 7f683af7e400 4 main: Executing for rank 0 2024-04-17T10:02:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.116+0000 7f683af7e400 4 Filtering by inode '1099511627777' 2024-04-17T10:02:44.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.116+0000 7f683af7e400 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55beed973400 0x55beee456680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.116+0000 7f683af7e400 1 -- 172.21.15.32:0/665646036 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) -- 0x55beed973800 con 0x55beed973400 2024-04-17T10:02:44.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.116+0000 7f6839a00640 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55beed973000 0x55beee456100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.116+0000 7f6839000640 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55beed973400 0x55beee456680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.120+0000 7f6839a00640 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55beed973000 0x55beee456100 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x55beed77a090 tx=0x55beed760300 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:44.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.120+0000 7f6839000640 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55beed973400 0x55beee456680 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:44.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.120+0000 7f6839000640 1 -- 172.21.15.32:0/665646036 <== osd.7 v2:172.21.15.144:6824/1746570140 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x55beee408000 con 0x55beed973400 2024-04-17T10:02:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.120+0000 7f683af7e400 4 Pointer 400.00000000 is readable 2024-04-17T10:02:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.120+0000 7f683af7e400 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-17T10:02:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.120+0000 7f683af7e400 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55beed973c00 0x55beee456c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.120+0000 7f683af7e400 1 -- 172.21.15.32:0/665646036 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) -- 0x55beeda7a000 con 0x55beed973c00 2024-04-17T10:02:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.120+0000 7f6839a00640 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55beed973c00 0x55beee456c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.124+0000 7f6839a00640 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55beed973c00 0x55beee456c00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:44.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.124+0000 7f6839a00640 1 -- 172.21.15.32:0/665646036 <== osd.4 v2:172.21.15.144:6809/4076371964 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x55beee408500 con 0x55beed973c00 2024-04-17T10:02:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.124+0000 7f683af7e400 10 Header 0x400000 0x405f0e 0x40628a 2024-04-17T10:02:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.124+0000 7f683af7e400 10 Starting journal scan from offset 0x405f0e 2024-04-17T10:02:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.124+0000 7f683af7e400 1 -- 172.21.15.32:0/665646036 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 24334~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) -- 0x55beeda7a400 con 0x55beed973c00 2024-04-17T10:02:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.124+0000 7f6839a00640 1 -- 172.21.15.32:0/665646036 <== osd.4 v2:172.21.15.144:6809/4076371964 2 ==== osd_op_reply(3 200.00000001 [read 24334~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x55beee408500 con 0x55beed973c00 2024-04-17T10:02:44.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.124+0000 7f683af7e400 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-17T10:02:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.124+0000 7f683af7e400 10 Parsing data, 0x37c bytes available 2024-04-17T10:02:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.124+0000 7f683af7e400 10 Attempting decode at 0x405f0e 2024-04-17T10:02:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.124+0000 7f683af7e400 10 Consumed 0x37c bytes 2024-04-17T10:02:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.124+0000 7f683af7e400 10 Valid entry at 0x405f0e 2024-04-17T10:02:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.124+0000 7f683af7e400 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55beeda7a800 0x55beee457180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.124+0000 7f683af7e400 1 -- 172.21.15.32:0/665646036 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) -- 0x55beeda7ac00 con 0x55beeda7a800 2024-04-17T10:02:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.124+0000 7f6839000640 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55beeda7a800 0x55beee457180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.124+0000 7f6839000640 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55beeda7a800 0x55beee457180 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f6839000640 1 -- 172.21.15.32:0/665646036 <== osd.1 v2:172.21.15.32:6808/2454993455 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55beee408000 con 0x55beeda7a800 2024-04-17T10:02:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 4 Reached end of journal objects 2024-04-17T10:02:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 4 Scanned objects, 0 missing, 1 valid 2024-04-17T10:02:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 4 Events scanned, 0 gaps 2024-04-17T10:02:44.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 4 Found 1 valid events 2024-04-17T10:02:44.125 INFO:teuthology.orchestra.run.smithi032.stdout:Events by type: 2024-04-17T10:02:44.125 INFO:teuthology.orchestra.run.smithi032.stdout:Errors: 0 2024-04-17T10:02:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 4 Selected 0 events events for processing 2024-04-17T10:02:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 4 0 events 2024-04-17T10:02:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 -- 172.21.15.32:0/665646036 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55beeda7a800 msgr2=0x55beee457180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55beeda7a800 0x55beee457180 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 -- 172.21.15.32:0/665646036 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55beed973c00 msgr2=0x55beee456c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55beed973c00 0x55beee456c00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 -- 172.21.15.32:0/665646036 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55beed973400 msgr2=0x55beee456680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55beed973400 0x55beee456680 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f6839000640 1 -- 172.21.15.32:0/665646036 reap_dead start 2024-04-17T10:02:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 -- 172.21.15.32:0/665646036 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55beed973000 msgr2=0x55beee456100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55beed973000 0x55beee456100 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x55beed77a090 tx=0x55beed760300 comp rx=0 tx=0).stop 2024-04-17T10:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 -- 172.21.15.32:0/665646036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55beee5a5c00 msgr2=0x55beee455b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55beee5a5c00 0x55beee455b80 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x55beee5b7440 tx=0x55beee5c2c00 comp rx=0 tx=0).stop 2024-04-17T10:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 -- 172.21.15.32:0/665646036 shutdown_connections 2024-04-17T10:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55beed973000 0x55beee456100 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 --2- 172.21.15.32:0/665646036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55beee5a5c00 0x55beee455b80 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 -- 172.21.15.32:0/665646036 >> 172.21.15.32:0/665646036 conn(0x55beee5a5000 msgr2=0x55beed6b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 -- 172.21.15.32:0/665646036 shutdown_connections 2024-04-17T10:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 -- 172.21.15.32:0/665646036 wait complete. 2024-04-17T10:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 -- 172.21.15.32:0/1988629188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55beed685400 msgr2=0x55beee454580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 --2- 172.21.15.32:0/1988629188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55beed685400 0x55beee454580 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x55beee4ff470 tx=0x55beed760f60 comp rx=0 tx=0).stop 2024-04-17T10:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 -- 172.21.15.32:0/1988629188 shutdown_connections 2024-04-17T10:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 --2- 172.21.15.32:0/1988629188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55beed685800 0x55beee454b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 --2- 172.21.15.32:0/1988629188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55beed685400 0x55beee454580 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 --2- 172.21.15.32:0/1988629188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beed685000 0x55beee454000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 -- 172.21.15.32:0/1988629188 >> 172.21.15.32:0/1988629188 conn(0x55beed684c00 msgr2=0x55beed6b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:44.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 -- 172.21.15.32:0/1988629188 shutdown_connections 2024-04-17T10:02:44.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.128+0000 7f683af7e400 1 -- 172.21.15.32:0/1988629188 wait complete. 2024-04-17T10:02:44.131 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.075206, result: Events by type: Errors: 0 2024-04-17T10:02:44.131 DEBUG:teuthology.orchestra.run.smithi032:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-04-17T10:02:44.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c8ba08400 1 -- start start 2024-04-17T10:02:44.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c8ba08400 1 Processor -- start 2024-04-17T10:02:44.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c8ba08400 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56419f36d800 0x5641a0132000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c8ba08400 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56419f36d400 0x5641a0132580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c8ba08400 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56419f36d000 0x5641a0132b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c8ba08400 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5641a0152180 con 0x56419f36d400 2024-04-17T10:02:44.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c8ba08400 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56419f3cc900 con 0x56419f36d000 2024-04-17T10:02:44.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c8ba08400 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x56419f3cc480 con 0x56419f36d800 2024-04-17T10:02:44.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c89000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56419f36d400 0x5641a0132580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c89000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56419f36d400 0x5641a0132580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44214/0 (socket says 172.21.15.32:44214) 2024-04-17T10:02:44.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c89000640 1 -- 172.21.15.32:0/1494711468 learned_addr learned my addr 172.21.15.32:0/1494711468 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:44.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c89a00640 1 --2- 172.21.15.32:0/1494711468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56419f36d800 0x5641a0132000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c8a400640 1 --2- 172.21.15.32:0/1494711468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56419f36d000 0x5641a0132b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c89000640 1 -- 172.21.15.32:0/1494711468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56419f36d800 msgr2=0x5641a0132000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:44.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c89000640 1 --2- 172.21.15.32:0/1494711468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56419f36d800 0x5641a0132000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c89000640 1 -- 172.21.15.32:0/1494711468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56419f36d000 msgr2=0x5641a0132b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:44.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c89000640 1 --2- 172.21.15.32:0/1494711468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56419f36d000 0x5641a0132b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c89000640 1 -- 172.21.15.32:0/1494711468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641a00dcb40 con 0x56419f36d400 2024-04-17T10:02:44.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c8a400640 1 --2- 172.21.15.32:0/1494711468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56419f36d000 0x5641a0132b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:02:44.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c89a00640 1 --2- 172.21.15.32:0/1494711468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56419f36d800 0x5641a0132000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:02:44.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c89000640 1 --2- 172.21.15.32:0/1494711468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56419f36d400 0x5641a0132580 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x5641a01dea20 tx=0x56419f423380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f800b29a0bc274b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:44.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c86800640 1 -- 172.21.15.32:0/1494711468 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56419f3a29c0 con 0x56419f36d400 2024-04-17T10:02:44.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c86800640 1 -- 172.21.15.32:0/1494711468 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56419f3b81c0 con 0x56419f36d400 2024-04-17T10:02:44.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c86800640 1 -- 172.21.15.32:0/1494711468 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56419f3a24e0 con 0x56419f36d400 2024-04-17T10:02:44.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c8ba08400 1 -- 172.21.15.32:0/1494711468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641a00dcd20 con 0x56419f36d400 2024-04-17T10:02:44.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c8ba08400 1 -- 172.21.15.32:0/1494711468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5641a00dcf00 con 0x56419f36d400 2024-04-17T10:02:44.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c8ba08400 4 waiting for MDS map... 2024-04-17T10:02:44.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c86800640 1 -- 172.21.15.32:0/1494711468 <== mon.0 v2:172.21.15.32:3300/0 4 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x5641a028e200 con 0x56419f36d400 2024-04-17T10:02:44.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c86800640 1 -- 172.21.15.32:0/1494711468 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 82) ==== 2886+0+0 (secure 0 0 0) 0x56419f3c3180 con 0x56419f36d400 2024-04-17T10:02:44.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c8ba08400 4 Got MDS map 82 2024-04-17T10:02:44.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c8ba08400 10 main: JournalTool::main 2024-04-17T10:02:44.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.220+0000 7f2c8ba08400 4 main: JournalTool: connecting to RADOS... 2024-04-17T10:02:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8ba08400 1 Processor -- start 2024-04-17T10:02:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8ba08400 1 -- start start 2024-04-17T10:02:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8ba08400 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641a0159c00 0x5641a0133080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8ba08400 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5641a02f2000 0x5641a0133600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8ba08400 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641a02f2400 0x5641a0133b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8ba08400 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56419f3cc180 con 0x5641a0159c00 2024-04-17T10:02:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8ba08400 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5641a0152d80 con 0x5641a02f2400 2024-04-17T10:02:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8ba08400 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5641a0152f00 con 0x5641a02f2000 2024-04-17T10:02:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8a400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641a0159c00 0x5641a0133080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8a400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641a0159c00 0x5641a0133080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44216/0 (socket says 172.21.15.32:44216) 2024-04-17T10:02:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8a400640 1 -- 172.21.15.32:0/1740960444 learned_addr learned my addr 172.21.15.32:0/1740960444 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c89a00640 1 --2- 172.21.15.32:0/1740960444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5641a02f2000 0x5641a0133600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c89000640 1 --2- 172.21.15.32:0/1740960444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641a02f2400 0x5641a0133b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8a400640 1 -- 172.21.15.32:0/1740960444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5641a02f2000 msgr2=0x5641a0133600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8a400640 1 --2- 172.21.15.32:0/1740960444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5641a02f2000 0x5641a0133600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8a400640 1 -- 172.21.15.32:0/1740960444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641a02f2400 msgr2=0x5641a0133b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8a400640 1 --2- 172.21.15.32:0/1740960444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641a02f2400 0x5641a0133b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8a400640 1 -- 172.21.15.32:0/1740960444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641a00dc1e0 con 0x5641a0159c00 2024-04-17T10:02:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c89a00640 1 --2- 172.21.15.32:0/1740960444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5641a02f2000 0x5641a0133600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:02:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c89000640 1 --2- 172.21.15.32:0/1740960444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641a02f2400 0x5641a0133b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:02:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8a400640 1 --2- 172.21.15.32:0/1740960444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641a0159c00 0x5641a0133080 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x5641a0294870 tx=0x5641a02a0c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b15a218c634b832 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c84000640 1 -- 172.21.15.32:0/1740960444 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56419f3a29c0 con 0x5641a0159c00 2024-04-17T10:02:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c84000640 1 -- 172.21.15.32:0/1740960444 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56419f3b8c40 con 0x5641a0159c00 2024-04-17T10:02:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c84000640 1 -- 172.21.15.32:0/1740960444 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56419f3a24e0 con 0x5641a0159c00 2024-04-17T10:02:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8ba08400 1 -- 172.21.15.32:0/1740960444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641a0159c00 msgr2=0x5641a0133080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8ba08400 1 --2- 172.21.15.32:0/1740960444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641a0159c00 0x5641a0133080 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x5641a0294870 tx=0x5641a02a0c00 comp rx=0 tx=0).stop 2024-04-17T10:02:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8ba08400 1 -- 172.21.15.32:0/1740960444 shutdown_connections 2024-04-17T10:02:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8ba08400 1 --2- 172.21.15.32:0/1740960444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5641a02f2000 0x5641a0133600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8ba08400 1 --2- 172.21.15.32:0/1740960444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641a02f2400 0x5641a0133b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8ba08400 1 --2- 172.21.15.32:0/1740960444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641a0159c00 0x5641a0133080 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8ba08400 1 -- 172.21.15.32:0/1740960444 >> 172.21.15.32:0/1740960444 conn(0x56419f36cc00 msgr2=0x56419f39b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8ba08400 1 -- 172.21.15.32:0/1740960444 shutdown_connections 2024-04-17T10:02:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8ba08400 1 -- 172.21.15.32:0/1740960444 wait complete. 2024-04-17T10:02:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8ba08400 1 Processor -- start 2024-04-17T10:02:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.224+0000 7f2c8ba08400 1 -- start start 2024-04-17T10:02:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c8ba08400 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5641a0159c00 0x5641a0133080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c8ba08400 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641a02f3800 0x5641a0133600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c8ba08400 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641a02f3400 0x5641a0133b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c8ba08400 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5641a0153200 con 0x5641a02f3800 2024-04-17T10:02:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c8ba08400 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5641a0153080 con 0x5641a02f3400 2024-04-17T10:02:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c8ba08400 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5641a0153380 con 0x5641a0159c00 2024-04-17T10:02:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c89a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641a02f3800 0x5641a0133600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c89a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641a02f3800 0x5641a0133600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44218/0 (socket says 172.21.15.32:44218) 2024-04-17T10:02:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c89a00640 1 -- 172.21.15.32:0/3519626043 learned_addr learned my addr 172.21.15.32:0/3519626043 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c8a400640 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5641a0159c00 0x5641a0133080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c89000640 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641a02f3400 0x5641a0133b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c89a00640 1 -- 172.21.15.32:0/3519626043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5641a0159c00 msgr2=0x5641a0133080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c89a00640 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5641a0159c00 0x5641a0133080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c89a00640 1 -- 172.21.15.32:0/3519626043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641a02f3400 msgr2=0x5641a0133b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c89a00640 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641a02f3400 0x5641a0133b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c89a00640 1 -- 172.21.15.32:0/3519626043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641a00dda40 con 0x5641a02f3800 2024-04-17T10:02:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c89a00640 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641a02f3800 0x5641a0133600 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x5641a0295560 tx=0x5641a02a0d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c82200640 1 -- 172.21.15.32:0/3519626043 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56419f3a29c0 con 0x5641a02f3800 2024-04-17T10:02:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c82200640 1 -- 172.21.15.32:0/3519626043 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56419f3b8540 con 0x5641a02f3800 2024-04-17T10:02:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c8ba08400 1 -- 172.21.15.32:0/3519626043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5641a00ddc20 con 0x5641a02f3800 2024-04-17T10:02:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c82200640 1 -- 172.21.15.32:0/3519626043 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56419f3a24e0 con 0x5641a02f3800 2024-04-17T10:02:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c8ba08400 1 -- 172.21.15.32:0/3519626043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641a00dda40 con 0x5641a02f3800 2024-04-17T10:02:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c8ba08400 4 main: JournalTool: resolving pool 14 2024-04-17T10:02:44.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c82200640 1 -- 172.21.15.32:0/3519626043 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x5641a02f0c00 con 0x5641a02f3800 2024-04-17T10:02:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c82200640 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x56419f661400 0x5641a0134100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c82200640 1 -- 172.21.15.32:0/3519626043 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x56419f670a00 con 0x5641a02f3800 2024-04-17T10:02:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c8ba08400 4 main: JournalTool: creating IoCtx.. 2024-04-17T10:02:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c8ba08400 4 main: Executing for rank 0 2024-04-17T10:02:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c8ba08400 4 Filtering by inode '1099511627778' 2024-04-17T10:02:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.228+0000 7f2c8ba08400 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x5641a02f3000 0x5641a0134680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c8a400640 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x56419f661400 0x5641a0134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c8ba08400 1 -- 172.21.15.32:0/3519626043 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) -- 0x5641a02f2c00 con 0x5641a02f3000 2024-04-17T10:02:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c89000640 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x5641a02f3000 0x5641a0134680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c8a400640 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x56419f661400 0x5641a0134100 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x5641a0294870 tx=0x56419f447f80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c89000640 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x5641a02f3000 0x5641a0134680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c89000640 1 -- 172.21.15.32:0/3519626043 <== osd.7 v2:172.21.15.144:6824/1746570140 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x5641a00f0000 con 0x5641a02f3000 2024-04-17T10:02:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c8ba08400 4 Pointer 400.00000000 is readable 2024-04-17T10:02:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c8ba08400 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-17T10:02:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c8ba08400 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x5641a02f2800 0x5641a0134c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c8ba08400 1 -- 172.21.15.32:0/3519626043 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) -- 0x5641a02f2400 con 0x5641a02f2800 2024-04-17T10:02:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c8a400640 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x5641a02f2800 0x5641a0134c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c8a400640 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x5641a02f2800 0x5641a0134c00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c8a400640 1 -- 172.21.15.32:0/3519626043 <== osd.4 v2:172.21.15.144:6809/4076371964 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x5641a00f0500 con 0x5641a02f2800 2024-04-17T10:02:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c8ba08400 10 Header 0x400000 0x405f0e 0x40628a 2024-04-17T10:02:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c8ba08400 10 Starting journal scan from offset 0x405f0e 2024-04-17T10:02:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c8ba08400 1 -- 172.21.15.32:0/3519626043 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 24334~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) -- 0x5641a02f2000 con 0x5641a02f2800 2024-04-17T10:02:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c8a400640 1 -- 172.21.15.32:0/3519626043 <== osd.4 v2:172.21.15.144:6809/4076371964 2 ==== osd_op_reply(3 200.00000001 [read 24334~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x5641a00f0500 con 0x5641a02f2800 2024-04-17T10:02:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c8ba08400 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-17T10:02:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c8ba08400 10 Parsing data, 0x37c bytes available 2024-04-17T10:02:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c8ba08400 10 Attempting decode at 0x405f0e 2024-04-17T10:02:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c8ba08400 10 Consumed 0x37c bytes 2024-04-17T10:02:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c8ba08400 10 Valid entry at 0x405f0e 2024-04-17T10:02:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c8ba08400 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x56419f661800 0x5641a0135180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c8ba08400 1 -- 172.21.15.32:0/3519626043 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) -- 0x56419f661c00 con 0x56419f661800 2024-04-17T10:02:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.232+0000 7f2c89a00640 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x56419f661800 0x5641a0135180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c89a00640 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x56419f661800 0x5641a0135180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c89a00640 1 -- 172.21.15.32:0/3519626043 <== osd.1 v2:172.21.15.32:6808/2454993455 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5641a00f0780 con 0x56419f661800 2024-04-17T10:02:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 4 Reached end of journal objects 2024-04-17T10:02:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 4 Scanned objects, 0 missing, 1 valid 2024-04-17T10:02:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 4 Events scanned, 0 gaps 2024-04-17T10:02:44.235 INFO:teuthology.orchestra.run.smithi032.stdout:Events by type: 2024-04-17T10:02:44.235 INFO:teuthology.orchestra.run.smithi032.stdout:Errors: 0 2024-04-17T10:02:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 4 Found 1 valid events 2024-04-17T10:02:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 4 Selected 0 events events for processing 2024-04-17T10:02:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 4 0 events 2024-04-17T10:02:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 -- 172.21.15.32:0/3519626043 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x56419f661800 msgr2=0x5641a0135180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x56419f661800 0x5641a0135180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 -- 172.21.15.32:0/3519626043 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x5641a02f2800 msgr2=0x5641a0134c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x5641a02f2800 0x5641a0134c00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 -- 172.21.15.32:0/3519626043 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x5641a02f3000 msgr2=0x5641a0134680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x5641a02f3000 0x5641a0134680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c89a00640 1 -- 172.21.15.32:0/3519626043 reap_dead start 2024-04-17T10:02:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 -- 172.21.15.32:0/3519626043 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x56419f661400 msgr2=0x5641a0134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x56419f661400 0x5641a0134100 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x5641a0294870 tx=0x56419f447f80 comp rx=0 tx=0).stop 2024-04-17T10:02:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 -- 172.21.15.32:0/3519626043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641a02f3800 msgr2=0x5641a0133600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641a02f3800 0x5641a0133600 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x5641a0295560 tx=0x5641a02a0d20 comp rx=0 tx=0).stop 2024-04-17T10:02:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 -- 172.21.15.32:0/3519626043 shutdown_connections 2024-04-17T10:02:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x56419f661400 0x5641a0134100 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 --2- 172.21.15.32:0/3519626043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641a02f3800 0x5641a0133600 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 -- 172.21.15.32:0/3519626043 >> 172.21.15.32:0/3519626043 conn(0x56419f36cc00 msgr2=0x56419f39b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 -- 172.21.15.32:0/3519626043 shutdown_connections 2024-04-17T10:02:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 -- 172.21.15.32:0/3519626043 wait complete. 2024-04-17T10:02:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 -- 172.21.15.32:0/1494711468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56419f36d400 msgr2=0x5641a0132580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 --2- 172.21.15.32:0/1494711468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56419f36d400 0x5641a0132580 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x5641a01dea20 tx=0x56419f423380 comp rx=0 tx=0).stop 2024-04-17T10:02:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 -- 172.21.15.32:0/1494711468 shutdown_connections 2024-04-17T10:02:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 --2- 172.21.15.32:0/1494711468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56419f36d000 0x5641a0132b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 --2- 172.21.15.32:0/1494711468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56419f36d400 0x5641a0132580 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 --2- 172.21.15.32:0/1494711468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56419f36d800 0x5641a0132000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 -- 172.21.15.32:0/1494711468 >> 172.21.15.32:0/1494711468 conn(0x56419f36dc00 msgr2=0x56419f39b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 -- 172.21.15.32:0/1494711468 shutdown_connections 2024-04-17T10:02:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.236+0000 7f2c8ba08400 1 -- 172.21.15.32:0/1494711468 wait complete. 2024-04-17T10:02:44.240 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.108996, result: Events by type: Errors: 0 2024-04-17T10:02:44.240 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-04-17T10:02:44.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.332+0000 7f7f06648080 1 Processor -- start 2024-04-17T10:02:44.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.332+0000 7f7f06648080 1 -- start start 2024-04-17T10:02:44.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.332+0000 7f7f06648080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c012ce230 0x564c012c7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.332+0000 7f7f06648080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c012ce610 0x564c012b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.332+0000 7f7f06648080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c0135b270 0x564c013f0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.332+0000 7f7f06648080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564c012b5a70 con 0x564c0135b270 2024-04-17T10:02:44.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.332+0000 7f7f06648080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564c012d15f0 con 0x564c012ce230 2024-04-17T10:02:44.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.332+0000 7f7f06648080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x564c012b5bf0 con 0x564c012ce610 2024-04-17T10:02:44.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.332+0000 7f7f03600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c0135b270 0x564c013f0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.332+0000 7f7f03600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c0135b270 0x564c013f0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44236/0 (socket says 172.21.15.32:44236) 2024-04-17T10:02:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.332+0000 7f7f03600640 1 -- 172.21.15.32:0/654928122 learned_addr learned my addr 172.21.15.32:0/654928122 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.332+0000 7f7f02200640 1 --2- 172.21.15.32:0/654928122 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c012ce610 0x564c012b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.332+0000 7f7f02c00640 1 --2- 172.21.15.32:0/654928122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c012ce230 0x564c012c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.332+0000 7f7f03600640 1 -- 172.21.15.32:0/654928122 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c012ce610 msgr2=0x564c012b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.332+0000 7f7f03600640 1 --2- 172.21.15.32:0/654928122 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c012ce610 0x564c012b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.332+0000 7f7f03600640 1 -- 172.21.15.32:0/654928122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c012ce230 msgr2=0x564c012c7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.332+0000 7f7f03600640 1 --2- 172.21.15.32:0/654928122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c012ce230 0x564c012c7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.332+0000 7f7f03600640 1 -- 172.21.15.32:0/654928122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c011d3940 con 0x564c0135b270 2024-04-17T10:02:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.332+0000 7f7f02c00640 1 --2- 172.21.15.32:0/654928122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c012ce230 0x564c012c7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:02:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.332+0000 7f7f03600640 1 --2- 172.21.15.32:0/654928122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c0135b270 0x564c013f0170 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00a4b0 tx=0x7f7efc0033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dcc069a445bbad9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.332+0000 7f7f00e00640 1 -- 172.21.15.32:0/654928122 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc0b9070 con 0x564c0135b270 2024-04-17T10:02:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.332+0000 7f7f00e00640 1 -- 172.21.15.32:0/654928122 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7efc0a4e50 con 0x564c0135b270 2024-04-17T10:02:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.332+0000 7f7f00e00640 1 -- 172.21.15.32:0/654928122 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc0ab610 con 0x564c0135b270 2024-04-17T10:02:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f06648080 1 -- 172.21.15.32:0/654928122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c0135b270 msgr2=0x564c013f0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f06648080 1 --2- 172.21.15.32:0/654928122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c0135b270 0x564c013f0170 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00a4b0 tx=0x7f7efc0033b0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f06648080 1 -- 172.21.15.32:0/654928122 shutdown_connections 2024-04-17T10:02:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f06648080 1 --2- 172.21.15.32:0/654928122 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c012ce610 0x564c012b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f06648080 1 --2- 172.21.15.32:0/654928122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c0135b270 0x564c013f0170 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f06648080 1 --2- 172.21.15.32:0/654928122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c012ce230 0x564c012c7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f06648080 1 -- 172.21.15.32:0/654928122 >> 172.21.15.32:0/654928122 conn(0x564c01284620 msgr2=0x564c013571c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f06648080 1 -- 172.21.15.32:0/654928122 shutdown_connections 2024-04-17T10:02:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f06648080 1 -- 172.21.15.32:0/654928122 wait complete. 2024-04-17T10:02:44.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f06648080 1 Processor -- start 2024-04-17T10:02:44.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f06648080 1 -- start start 2024-04-17T10:02:44.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f06648080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c012ce230 0x564c012836f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f06648080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c012ce610 0x564c0133d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f06648080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c0135b270 0x564c0133d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f06648080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564c012c91f0 con 0x564c012ce610 2024-04-17T10:02:44.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f06648080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564c012c8f50 con 0x564c012ce230 2024-04-17T10:02:44.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f06648080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x564c012c9490 con 0x564c0135b270 2024-04-17T10:02:44.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f02200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c012ce610 0x564c0133d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f02200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c012ce610 0x564c0133d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44252/0 (socket says 172.21.15.32:44252) 2024-04-17T10:02:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f02200640 1 -- 172.21.15.32:0/3946967851 learned_addr learned my addr 172.21.15.32:0/3946967851 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f02c00640 1 --2- 172.21.15.32:0/3946967851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c012ce230 0x564c012836f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f03600640 1 --2- 172.21.15.32:0/3946967851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c0135b270 0x564c0133d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f02200640 1 -- 172.21.15.32:0/3946967851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c0135b270 msgr2=0x564c0133d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f02200640 1 --2- 172.21.15.32:0/3946967851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c0135b270 0x564c0133d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f02200640 1 -- 172.21.15.32:0/3946967851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c012ce230 msgr2=0x564c012836f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f02200640 1 --2- 172.21.15.32:0/3946967851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c012ce230 0x564c012836f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f02200640 1 -- 172.21.15.32:0/3946967851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c011af950 con 0x564c012ce610 2024-04-17T10:02:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f02200640 1 --2- 172.21.15.32:0/3946967851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c012ce610 0x564c0133d1f0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00087e0 tx=0x7f7ef0008ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7efb400640 1 -- 172.21.15.32:0/3946967851 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0006d80 con 0x564c012ce610 2024-04-17T10:02:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7efb400640 1 -- 172.21.15.32:0/3946967851 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ef001a070 con 0x564c012ce610 2024-04-17T10:02:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f06648080 1 -- 172.21.15.32:0/3946967851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564c0135a3b0 con 0x564c012ce610 2024-04-17T10:02:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7efb400640 1 -- 172.21.15.32:0/3946967851 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0015660 con 0x564c012ce610 2024-04-17T10:02:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.336+0000 7f7f06648080 1 -- 172.21.15.32:0/3946967851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c0133e270 con 0x564c012ce610 2024-04-17T10:02:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.340+0000 7f7efb400640 1 -- 172.21.15.32:0/3946967851 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7ef0013030 con 0x564c012ce610 2024-04-17T10:02:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.340+0000 7f7efb400640 1 --2- 172.21.15.32:0/3946967851 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7ee4066dc0 0x7f7ee4069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.340+0000 7f7efb400640 1 -- 172.21.15.32:0/3946967851 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f7ef0085f30 con 0x564c012ce610 2024-04-17T10:02:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.340+0000 7f7f02c00640 1 --2- 172.21.15.32:0/3946967851 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7ee4066dc0 0x7f7ee4069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.340+0000 7f7f06648080 1 --2- 172.21.15.32:0/3946967851 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x564c01283c30 0x564c013eefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.340+0000 7f7f06648080 1 -- 172.21.15.32:0/3946967851 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x564c013ef4f0 con 0x564c01283c30 2024-04-17T10:02:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.340+0000 7f7f03600640 1 --2- 172.21.15.32:0/3946967851 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x564c01283c30 0x564c013eefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.340+0000 7f7f02c00640 1 --2- 172.21.15.32:0/3946967851 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7ee4066dc0 0x7f7ee4069280 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009690 tx=0x7f7ef400c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.340+0000 7f7f03600640 1 --2- 172.21.15.32:0/3946967851 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x564c01283c30 0x564c013eefb0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:44.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.344+0000 7f7f03600640 1 -- 172.21.15.32:0/3946967851 <== osd.2 v2:172.21.15.32:6809/1954833462 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v98'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7efc0bc040 con 0x564c01283c30 2024-04-17T10:02:44.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.344+0000 7f7f06648080 1 -- 172.21.15.32:0/3946967851 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x564c01283c30 msgr2=0x564c013eefb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.344+0000 7f7f06648080 1 --2- 172.21.15.32:0/3946967851 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x564c01283c30 0x564c013eefb0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.344+0000 7f7f06648080 1 -- 172.21.15.32:0/3946967851 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7ee4066dc0 msgr2=0x7f7ee4069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.344+0000 7f7f06648080 1 --2- 172.21.15.32:0/3946967851 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7ee4066dc0 0x7f7ee4069280 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009690 tx=0x7f7ef400c040 comp rx=0 tx=0).stop 2024-04-17T10:02:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.344+0000 7f7f06648080 1 -- 172.21.15.32:0/3946967851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c012ce610 msgr2=0x564c0133d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.344+0000 7f7f06648080 1 --2- 172.21.15.32:0/3946967851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564c012ce610 0x564c0133d1f0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00087e0 tx=0x7f7ef0008ca0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.344+0000 7f7f03600640 1 -- 172.21.15.32:0/3946967851 reap_dead start 2024-04-17T10:02:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.344+0000 7f7f06648080 1 -- 172.21.15.32:0/3946967851 shutdown_connections 2024-04-17T10:02:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.344+0000 7f7f06648080 1 -- 172.21.15.32:0/3946967851 >> 172.21.15.32:0/3946967851 conn(0x564c01284620 msgr2=0x564c012c7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.344+0000 7f7f06648080 1 -- 172.21.15.32:0/3946967851 shutdown_connections 2024-04-17T10:02:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.344+0000 7f7f06648080 1 -- 172.21.15.32:0/3946967851 wait complete. 2024-04-17T10:02:44.348 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-17T10:02:44.354 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:02:44.396 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:02:44.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.540+0000 7f92ba800640 1 Processor -- start 2024-04-17T10:02:44.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.540+0000 7f92ba800640 1 -- start start 2024-04-17T10:02:44.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.540+0000 7f92ba800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92b415a410 0x7f92b40cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.540+0000 7f92ba800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b415dce0 con 0x7f92b406c6f0 2024-04-17T10:02:44.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.540+0000 7f92ba800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b41603c0 con 0x7f92b40cd6d0 2024-04-17T10:02:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.540+0000 7f92ba800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f92b41606a0 con 0x7f92b415a410 2024-04-17T10:02:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.540+0000 7f92b3400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92b415a410 0x7f92b40cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.540+0000 7f92b2a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f92b406c6f0 0x7f92b40b23d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46186/0 (socket says 172.21.15.32:46186) 2024-04-17T10:02:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.540+0000 7f92b2a00640 1 -- 172.21.15.32:0/1031047357 learned_addr learned my addr 172.21.15.32:0/1031047357 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:02:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.540+0000 7f92b3400640 1 -- 172.21.15.32:0/1031047357 >> v1:172.21.15.144:6789/0 conn(0x7f92b40cd6d0 legacy=0x7f92b40cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.540+0000 7f92b3400640 1 -- 172.21.15.32:0/1031047357 >> v1:172.21.15.32:6789/0 conn(0x7f92b406c6f0 legacy=0x7f92b40b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.540+0000 7f92b3400640 1 -- 172.21.15.32:0/1031047357 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b4160980 con 0x7f92b415a410 2024-04-17T10:02:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.544+0000 7f92b3400640 1 --2- 172.21.15.32:0/1031047357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b415a410 0x7f92b40cd190 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f92a40097c0 tx=0x7f92a401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4797364312a7e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.544+0000 7f92b2000640 1 -- 172.21.15.32:0/1031047357 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a40b0070 con 0x7f92b415a410 2024-04-17T10:02:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.544+0000 7f92b2000640 1 -- 172.21.15.32:0/1031047357 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92a40ab600 con 0x7f92b415a410 2024-04-17T10:02:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.544+0000 7f92b2000640 1 -- 172.21.15.32:0/1031047357 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a40ab9d0 con 0x7f92b415a410 2024-04-17T10:02:44.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.544+0000 7f92ba800640 1 -- 172.21.15.32:0/1031047357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b415a410 msgr2=0x7f92b40cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.544+0000 7f92ba800640 1 --2- 172.21.15.32:0/1031047357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b415a410 0x7f92b40cd190 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f92a40097c0 tx=0x7f92a401c4a0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.544+0000 7f92ba800640 1 -- 172.21.15.32:0/1031047357 shutdown_connections 2024-04-17T10:02:44.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.544+0000 7f92ba800640 1 --2- 172.21.15.32:0/1031047357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b415a410 0x7f92b40cd190 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.544+0000 7f92ba800640 1 -- 172.21.15.32:0/1031047357 >> 172.21.15.32:0/1031047357 conn(0x7f92b40b3000 msgr2=0x7f92b40b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:44.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.544+0000 7f92ba800640 1 -- 172.21.15.32:0/1031047357 shutdown_connections 2024-04-17T10:02:44.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.544+0000 7f92ba800640 1 -- 172.21.15.32:0/1031047357 wait complete. 2024-04-17T10:02:44.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.544+0000 7f92ba800640 1 Processor -- start 2024-04-17T10:02:44.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.544+0000 7f92ba800640 1 -- start start 2024-04-17T10:02:44.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.548+0000 7f92ba800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b406c6f0 0x7f92b40cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.548+0000 7f92ba800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92b40cd6d0 0x7f92b40d0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.548+0000 7f92ba800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92b415a410 0x7f92b40c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.548+0000 7f92ba800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f92b416aa60 con 0x7f92b406c6f0 2024-04-17T10:02:44.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.548+0000 7f92ba800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f92b416a8e0 con 0x7f92b415a410 2024-04-17T10:02:44.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.548+0000 7f92ba800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f92b416abe0 con 0x7f92b40cd6d0 2024-04-17T10:02:44.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.548+0000 7f92b3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b406c6f0 0x7f92b40cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.548+0000 7f92b3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b406c6f0 0x7f92b40cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44264/0 (socket says 172.21.15.32:44264) 2024-04-17T10:02:44.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.548+0000 7f92b3400640 1 -- 172.21.15.32:0/3353195349 learned_addr learned my addr 172.21.15.32:0/3353195349 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:44.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.548+0000 7f92b3400640 1 -- 172.21.15.32:0/3353195349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92b40cd6d0 msgr2=0x7f92b40d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.548+0000 7f92b3400640 1 --2- 172.21.15.32:0/3353195349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92b40cd6d0 0x7f92b40d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.548+0000 7f92b3400640 1 -- 172.21.15.32:0/3353195349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92b415a410 msgr2=0x7f92b40c5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:44.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.548+0000 7f92b3400640 1 --2- 172.21.15.32:0/3353195349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92b415a410 0x7f92b40c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:44.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.548+0000 7f92b3400640 1 -- 172.21.15.32:0/3353195349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b40b3de0 con 0x7f92b406c6f0 2024-04-17T10:02:44.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.548+0000 7f92b3400640 1 --2- 172.21.15.32:0/3353195349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b406c6f0 0x7f92b40cfad0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f92a40097c0 tx=0x7f92a4019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:44.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.548+0000 7f92ab400640 1 -- 172.21.15.32:0/3353195349 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a40b0040 con 0x7f92b406c6f0 2024-04-17T10:02:44.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.548+0000 7f92ab400640 1 -- 172.21.15.32:0/3353195349 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92a40b4410 con 0x7f92b406c6f0 2024-04-17T10:02:44.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.548+0000 7f92ba800640 1 -- 172.21.15.32:0/3353195349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b40d0620 con 0x7f92b406c6f0 2024-04-17T10:02:44.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.548+0000 7f92ab400640 1 -- 172.21.15.32:0/3353195349 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a40b4800 con 0x7f92b406c6f0 2024-04-17T10:02:44.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.548+0000 7f92ba800640 1 -- 172.21.15.32:0/3353195349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b40c5620 con 0x7f92b406c6f0 2024-04-17T10:02:44.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.548+0000 7f92ba800640 1 -- 172.21.15.32:0/3353195349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b40d0b40 con 0x7f92b406c6f0 2024-04-17T10:02:44.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.552+0000 7f92ab400640 1 -- 172.21.15.32:0/3353195349 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f92a40bc020 con 0x7f92b406c6f0 2024-04-17T10:02:44.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.552+0000 7f92ab400640 1 --2- 172.21.15.32:0/3353195349 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f92940683b0 0x7f929406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:44.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.552+0000 7f92ab400640 1 -- 172.21.15.32:0/3353195349 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f92a411fed0 con 0x7f92b406c6f0 2024-04-17T10:02:44.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.552+0000 7f92b2a00640 1 --2- 172.21.15.32:0/3353195349 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f92940683b0 0x7f929406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:44.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.552+0000 7f92b2a00640 1 --2- 172.21.15.32:0/3353195349 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f92940683b0 0x7f929406a870 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f92b41442c0 tx=0x7f92a0007a50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:44.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.556+0000 7f92ab400640 1 -- 172.21.15.32:0/3353195349 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f92b40d0b40 con 0x7f92b406c6f0 2024-04-17T10:02:44.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:44.756+0000 7f92ba800640 1 -- 172.21.15.32:0/3353195349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f92b4068a90 con 0x7f92b406c6f0 2024-04-17T10:02:45.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.036+0000 7f92ab400640 1 -- 172.21.15.32:0/3353195349 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v83) ==== 162+0+0 (secure 0 0 0) 0x7f92a40c3020 con 0x7f92b406c6f0 2024-04-17T10:02:45.033 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-17T10:02:45.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.036+0000 7f92ba800640 1 -- 172.21.15.32:0/3353195349 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f92940683b0 msgr2=0x7f929406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:45.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.036+0000 7f92ba800640 1 --2- 172.21.15.32:0/3353195349 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f92940683b0 0x7f929406a870 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f92b41442c0 tx=0x7f92a0007a50 comp rx=0 tx=0).stop 2024-04-17T10:02:45.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.036+0000 7f92ba800640 1 -- 172.21.15.32:0/3353195349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b406c6f0 msgr2=0x7f92b40cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:45.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.036+0000 7f92ba800640 1 --2- 172.21.15.32:0/3353195349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b406c6f0 0x7f92b40cfad0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f92a40097c0 tx=0x7f92a4019240 comp rx=0 tx=0).stop 2024-04-17T10:02:45.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.036+0000 7f92ba800640 1 -- 172.21.15.32:0/3353195349 shutdown_connections 2024-04-17T10:02:45.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.036+0000 7f92ba800640 1 --2- 172.21.15.32:0/3353195349 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f92940683b0 0x7f929406a870 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:45.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.036+0000 7f92ba800640 1 --2- 172.21.15.32:0/3353195349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92b40cd6d0 0x7f92b40d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:45.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.036+0000 7f92ba800640 1 --2- 172.21.15.32:0/3353195349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92b415a410 0x7f92b40c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:45.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.036+0000 7f92ba800640 1 --2- 172.21.15.32:0/3353195349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b406c6f0 0x7f92b40cfad0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:45.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.036+0000 7f92ba800640 1 -- 172.21.15.32:0/3353195349 >> 172.21.15.32:0/3353195349 conn(0x7f92b40b3000 msgr2=0x7f92b40ca530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:45.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.036+0000 7f92ba800640 1 -- 172.21.15.32:0/3353195349 shutdown_connections 2024-04-17T10:02:45.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.036+0000 7f92ba800640 1 -- 172.21.15.32:0/3353195349 wait complete. 2024-04-17T10:02:45.061 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:02:45.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.208+0000 7f562d200640 1 Processor -- start 2024-04-17T10:02:45.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.208+0000 7f562d200640 1 -- start start 2024-04-17T10:02:45.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.208+0000 7f562d200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5628145f30 0x7f5628071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:45.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.208+0000 7f562d200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f562815d8d0 0x7f5628072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:45.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.208+0000 7f562d200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56280cdac0 con 0x7f56280cce70 2024-04-17T10:02:45.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.208+0000 7f562d200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f562806f400 con 0x7f5628145f30 2024-04-17T10:02:45.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.208+0000 7f562d200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f5628146310 con 0x7f562815d8d0 2024-04-17T10:02:45.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.208+0000 7f5625e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5628145f30 0x7f5628071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:45.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.208+0000 7f5625e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5628145f30 0x7f5628071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44268/0 (socket says 172.21.15.32:44268) 2024-04-17T10:02:45.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.208+0000 7f5625e00640 1 -- 172.21.15.32:0/315430332 learned_addr learned my addr 172.21.15.32:0/315430332 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:45.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.208+0000 7f561fe00640 1 --2- 172.21.15.32:0/315430332 >> v2:172.21.15.144:3300/0 conn(0x7f562815d8d0 0x7f5628072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:45.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.208+0000 7f5625e00640 1 -- 172.21.15.32:0/315430332 >> v2:172.21.15.144:3300/0 conn(0x7f562815d8d0 msgr2=0x7f5628072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:45.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.208+0000 7f5625e00640 1 --2- 172.21.15.32:0/315430332 >> v2:172.21.15.144:3300/0 conn(0x7f562815d8d0 0x7f5628072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:45.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.208+0000 7f5625e00640 1 -- 172.21.15.32:0/315430332 >> v1:172.21.15.144:6789/0 conn(0x7f56280cce70 legacy=0x7f56280cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:45.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.208+0000 7f5625e00640 1 -- 172.21.15.32:0/315430332 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562815cdf0 con 0x7f5628145f30 2024-04-17T10:02:45.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.212+0000 7f5625e00640 1 --2- 172.21.15.32:0/315430332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5628145f30 0x7f5628071df0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f56280702a0 tx=0x7f561801ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1689b3237a794b88 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:45.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.212+0000 7f5625400640 1 -- 172.21.15.32:0/315430332 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56180b5070 con 0x7f5628145f30 2024-04-17T10:02:45.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.212+0000 7f5625400640 1 -- 172.21.15.32:0/315430332 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56180a89d0 con 0x7f5628145f30 2024-04-17T10:02:45.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.212+0000 7f5625400640 1 -- 172.21.15.32:0/315430332 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56180a8da0 con 0x7f5628145f30 2024-04-17T10:02:45.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.212+0000 7f562d200640 1 -- 172.21.15.32:0/315430332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5628145f30 msgr2=0x7f5628071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:45.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.212+0000 7f562d200640 1 --2- 172.21.15.32:0/315430332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5628145f30 0x7f5628071df0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f56280702a0 tx=0x7f561801ecd0 comp rx=0 tx=0).stop 2024-04-17T10:02:45.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.212+0000 7f562d200640 1 -- 172.21.15.32:0/315430332 shutdown_connections 2024-04-17T10:02:45.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.212+0000 7f562d200640 1 --2- 172.21.15.32:0/315430332 >> v2:172.21.15.144:3300/0 conn(0x7f562815d8d0 0x7f5628072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:45.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.212+0000 7f562d200640 1 --2- 172.21.15.32:0/315430332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5628145f30 0x7f5628071df0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:45.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.212+0000 7f562d200640 1 -- 172.21.15.32:0/315430332 >> 172.21.15.32:0/315430332 conn(0x7f562806ba30 msgr2=0x7f562806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:45.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.212+0000 7f562d200640 1 -- 172.21.15.32:0/315430332 shutdown_connections 2024-04-17T10:02:45.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.212+0000 7f562d200640 1 -- 172.21.15.32:0/315430332 wait complete. 2024-04-17T10:02:45.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f562d200640 1 Processor -- start 2024-04-17T10:02:45.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f562d200640 1 -- start start 2024-04-17T10:02:45.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f562d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56280cce70 0x7f562815b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:45.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f562d200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5628145f30 0x7f562815dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:45.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f562d200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f562815d8d0 0x7f562815e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:45.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f562d200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f562815cb90 con 0x7f56280cce70 2024-04-17T10:02:45.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f562d200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f562815d440 con 0x7f562815d8d0 2024-04-17T10:02:45.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f562d200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f56281601d0 con 0x7f5628145f30 2024-04-17T10:02:45.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f5625e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56280cce70 0x7f562815b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:45.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f5625e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56280cce70 0x7f562815b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44272/0 (socket says 172.21.15.32:44272) 2024-04-17T10:02:45.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f5625e00640 1 -- 172.21.15.32:0/134157147 learned_addr learned my addr 172.21.15.32:0/134157147 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:45.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f5626800640 1 --2- 172.21.15.32:0/134157147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f562815d8d0 0x7f562815e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:45.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f561fe00640 1 --2- 172.21.15.32:0/134157147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5628145f30 0x7f562815dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:45.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f5625e00640 1 -- 172.21.15.32:0/134157147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5628145f30 msgr2=0x7f562815dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:45.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f5625e00640 1 --2- 172.21.15.32:0/134157147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5628145f30 0x7f562815dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:45.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f5625e00640 1 -- 172.21.15.32:0/134157147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f562815d8d0 msgr2=0x7f562815e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:45.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f5625e00640 1 --2- 172.21.15.32:0/134157147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f562815d8d0 0x7f562815e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:45.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f5625e00640 1 -- 172.21.15.32:0/134157147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56280c6e20 con 0x7f56280cce70 2024-04-17T10:02:45.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f5625e00640 1 --2- 172.21.15.32:0/134157147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56280cce70 0x7f562815b810 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f56180b3040 tx=0x7f561801ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:45.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f561e000640 1 -- 172.21.15.32:0/134157147 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56180c0070 con 0x7f56280cce70 2024-04-17T10:02:45.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f561e000640 1 -- 172.21.15.32:0/134157147 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56180b0a50 con 0x7f56280cce70 2024-04-17T10:02:45.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f561e000640 1 -- 172.21.15.32:0/134157147 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56180b5070 con 0x7f56280cce70 2024-04-17T10:02:45.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f562d200640 1 -- 172.21.15.32:0/134157147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56280cc9e0 con 0x7f56280cce70 2024-04-17T10:02:45.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.216+0000 7f562d200640 1 -- 172.21.15.32:0/134157147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56280c73c0 con 0x7f56280cce70 2024-04-17T10:02:45.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.220+0000 7f562d200640 1 -- 172.21.15.32:0/134157147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5628068a90 con 0x7f56280cce70 2024-04-17T10:02:45.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.220+0000 7f561e000640 1 -- 172.21.15.32:0/134157147 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f56180c3070 con 0x7f56280cce70 2024-04-17T10:02:45.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.220+0000 7f561e000640 1 --2- 172.21.15.32:0/134157147 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5604068380 0x7f560406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:45.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.220+0000 7f561e000640 1 -- 172.21.15.32:0/134157147 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f5618123b70 con 0x7f56280cce70 2024-04-17T10:02:45.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.220+0000 7f561fe00640 1 --2- 172.21.15.32:0/134157147 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5604068380 0x7f560406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:45.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.220+0000 7f561fe00640 1 --2- 172.21.15.32:0/134157147 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5604068380 0x7f560406a840 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f562813de80 tx=0x7f5610008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:45.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.224+0000 7f561e000640 1 -- 172.21.15.32:0/134157147 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5618018830 con 0x7f56280cce70 2024-04-17T10:02:45.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.424+0000 7f562d200640 1 -- 172.21.15.32:0/134157147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5628072f90 con 0x7f56280cce70 2024-04-17T10:02:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.428+0000 7f561e000640 1 -- 172.21.15.32:0/134157147 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5201 (secure 0 0 0) 0x7f56180f4030 con 0x7f56280cce70 2024-04-17T10:02:45.424 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:02:45.424 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":84,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2635109588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2635109588},{"type":"v1","addr":"172.21.15.144:6835","nonce":2635109588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7789,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3258455939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3258455939},{"type":"v1","addr":"172.21.15.144:6834","nonce":3258455939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":8574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3390468332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3390468332},{"type":"v1","addr":"172.21.15.32:6836","nonce":3390468332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:01:40.557208+0000","modified":"2024-04-17T10:02:45.042456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8964":{"gid":8964,"name":"c","rank":0,"incarnation":84,"state":"up:replay","state_seq":1,"addr":"172.21.15.32:6837/3078068380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3078068380},{"type":"v1","addr":"172.21.15.32:6837","nonce":3078068380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-17T10:02:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 84 2024-04-17T10:02:45.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.428+0000 7f562d200640 1 -- 172.21.15.32:0/134157147 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5604068380 msgr2=0x7f560406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:45.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.428+0000 7f562d200640 1 --2- 172.21.15.32:0/134157147 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5604068380 0x7f560406a840 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f562813de80 tx=0x7f5610008040 comp rx=0 tx=0).stop 2024-04-17T10:02:45.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.428+0000 7f562d200640 1 -- 172.21.15.32:0/134157147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56280cce70 msgr2=0x7f562815b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:45.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.428+0000 7f562d200640 1 --2- 172.21.15.32:0/134157147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56280cce70 0x7f562815b810 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f56180b3040 tx=0x7f561801ba70 comp rx=0 tx=0).stop 2024-04-17T10:02:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.428+0000 7f562d200640 1 -- 172.21.15.32:0/134157147 shutdown_connections 2024-04-17T10:02:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.428+0000 7f562d200640 1 --2- 172.21.15.32:0/134157147 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5604068380 0x7f560406a840 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.428+0000 7f562d200640 1 --2- 172.21.15.32:0/134157147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5628145f30 0x7f562815dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.428+0000 7f562d200640 1 --2- 172.21.15.32:0/134157147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f562815d8d0 0x7f562815e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.428+0000 7f562d200640 1 --2- 172.21.15.32:0/134157147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56280cce70 0x7f562815b810 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.428+0000 7f562d200640 1 -- 172.21.15.32:0/134157147 >> 172.21.15.32:0/134157147 conn(0x7f562806ba30 msgr2=0x7f562815f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.428+0000 7f562d200640 1 -- 172.21.15.32:0/134157147 shutdown_connections 2024-04-17T10:02:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:45.428+0000 7f562d200640 1 -- 172.21.15.32:0/134157147 wait complete. 2024-04-17T10:02:45.849 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 84, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:01:40.557208+0000', 'modified': '2024-04-17T10:02:45.042456+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 98, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8964}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8964': {'gid': 8964, 'name': 'c', 'rank': 0, 'incarnation': 84, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.32:6837/3078068380', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6835', 'nonce': 3078068380}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 3078068380}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-17T10:02:45.850 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8964:up:replay 2024-04-17T10:02:46.851 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:02:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.004+0000 7f4bf3a00640 1 Processor -- start 2024-04-17T10:02:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.004+0000 7f4bf3a00640 1 -- start start 2024-04-17T10:02:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.008+0000 7f4bf3a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4bec144e90 0x7f4bec0b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.008+0000 7f4bf3a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4bec15c1b0 0x7f4bec06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.008+0000 7f4bf3a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bec072bb0 con 0x7f4bec0b4de0 2024-04-17T10:02:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.008+0000 7f4bf3a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f4bec145270 con 0x7f4bec144e90 2024-04-17T10:02:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.008+0000 7f4bf3a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4bec0b3dc0 con 0x7f4bec15c1b0 2024-04-17T10:02:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.008+0000 7f4bebe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4bec0b4de0 0x7f4bec15baa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46200/0 (socket says 172.21.15.32:46200) 2024-04-17T10:02:47.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.008+0000 7f4bebe00640 1 -- 172.21.15.32:0/2930889175 learned_addr learned my addr 172.21.15.32:0/2930889175 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:02:47.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.008+0000 7f4bf1000640 1 --2- 172.21.15.32:0/2930889175 >> v2:172.21.15.144:3300/0 conn(0x7f4bec144e90 0x7f4bec0b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:47.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.008+0000 7f4bf1a00640 1 --2- 172.21.15.32:0/2930889175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4bec15c1b0 0x7f4bec06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:47.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.008+0000 7f4beb400640 1 -- 172.21.15.32:0/2930889175 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 327564561 0 0) 0x7f4bec072bb0 con 0x7f4bec0b4de0 2024-04-17T10:02:47.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.008+0000 7f4beb400640 1 -- 172.21.15.32:0/2930889175 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4bd4003530 con 0x7f4bec0b4de0 2024-04-17T10:02:47.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.008+0000 7f4bf1a00640 1 -- 172.21.15.32:0/2930889175 >> v2:172.21.15.144:3300/0 conn(0x7f4bec144e90 msgr2=0x7f4bec0b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:47.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.008+0000 7f4bf1a00640 1 --2- 172.21.15.32:0/2930889175 >> v2:172.21.15.144:3300/0 conn(0x7f4bec144e90 0x7f4bec0b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:47.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.008+0000 7f4bf1a00640 1 -- 172.21.15.32:0/2930889175 >> v1:172.21.15.32:6789/0 conn(0x7f4bec0b4de0 legacy=0x7f4bec15baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:47.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.008+0000 7f4bf1a00640 1 -- 172.21.15.32:0/2930889175 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bec0b40a0 con 0x7f4bec15c1b0 2024-04-17T10:02:47.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.008+0000 7f4bf1000640 1 --2- 172.21.15.32:0/2930889175 >> v2:172.21.15.144:3300/0 conn(0x7f4bec144e90 0x7f4bec0b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:02:47.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.012+0000 7f4bf1a00640 1 --2- 172.21.15.32:0/2930889175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4bec15c1b0 0x7f4bec06ef60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009d90 tx=0x7f4be401c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ec7f14036422c8f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:47.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.012+0000 7f4beb400640 1 -- 172.21.15.32:0/2930889175 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be40b2070 con 0x7f4bec15c1b0 2024-04-17T10:02:47.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.012+0000 7f4beb400640 1 -- 172.21.15.32:0/2930889175 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4be40a5910 con 0x7f4bec15c1b0 2024-04-17T10:02:47.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.012+0000 7f4beb400640 1 -- 172.21.15.32:0/2930889175 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be40a5ce0 con 0x7f4bec15c1b0 2024-04-17T10:02:47.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.012+0000 7f4bf3a00640 1 -- 172.21.15.32:0/2930889175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4bec15c1b0 msgr2=0x7f4bec06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:47.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.012+0000 7f4bf3a00640 1 --2- 172.21.15.32:0/2930889175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4bec15c1b0 0x7f4bec06ef60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009d90 tx=0x7f4be401c940 comp rx=0 tx=0).stop 2024-04-17T10:02:47.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.012+0000 7f4bf3a00640 1 -- 172.21.15.32:0/2930889175 shutdown_connections 2024-04-17T10:02:47.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.012+0000 7f4bf3a00640 1 --2- 172.21.15.32:0/2930889175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4bec15c1b0 0x7f4bec06ef60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:47.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.012+0000 7f4bf3a00640 1 --2- 172.21.15.32:0/2930889175 >> v2:172.21.15.144:3300/0 conn(0x7f4bec144e90 0x7f4bec0b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:47.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.012+0000 7f4bf3a00640 1 -- 172.21.15.32:0/2930889175 >> 172.21.15.32:0/2930889175 conn(0x7f4bec06ba30 msgr2=0x7f4bec06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:47.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.012+0000 7f4bf3a00640 1 -- 172.21.15.32:0/2930889175 shutdown_connections 2024-04-17T10:02:47.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.012+0000 7f4bf3a00640 1 -- 172.21.15.32:0/2930889175 wait complete. 2024-04-17T10:02:47.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.012+0000 7f4bf3a00640 1 Processor -- start 2024-04-17T10:02:47.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4bf3a00640 1 -- start start 2024-04-17T10:02:47.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4bf3a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bec0b4de0 0x7f4bec15e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:47.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4bf3a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bec144e90 0x7f4bec157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:47.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4bf3a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4bec15c1b0 0x7f4bec138f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:47.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4bf3a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec166100 con 0x7f4bec0b4de0 2024-04-17T10:02:47.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4bf3a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec0728c0 con 0x7f4bec144e90 2024-04-17T10:02:47.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4bf3a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4bec166280 con 0x7f4bec15c1b0 2024-04-17T10:02:47.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4bf1000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bec0b4de0 0x7f4bec15e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:47.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4bf1000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bec0b4de0 0x7f4bec15e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44286/0 (socket says 172.21.15.32:44286) 2024-04-17T10:02:47.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4bf1000640 1 -- 172.21.15.32:0/2848736410 learned_addr learned my addr 172.21.15.32:0/2848736410 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:47.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4bf1a00640 1 --2- 172.21.15.32:0/2848736410 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4bec15c1b0 0x7f4bec138f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:47.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4bebe00640 1 --2- 172.21.15.32:0/2848736410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bec144e90 0x7f4bec157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:47.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4bf1000640 1 -- 172.21.15.32:0/2848736410 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4bec15c1b0 msgr2=0x7f4bec138f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:47.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4bf1000640 1 --2- 172.21.15.32:0/2848736410 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4bec15c1b0 0x7f4bec138f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:47.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4bf1000640 1 -- 172.21.15.32:0/2848736410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bec144e90 msgr2=0x7f4bec157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:47.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4bf1000640 1 --2- 172.21.15.32:0/2848736410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bec144e90 0x7f4bec157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:47.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4bf1000640 1 -- 172.21.15.32:0/2848736410 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bec0b1880 con 0x7f4bec0b4de0 2024-04-17T10:02:47.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4bf1a00640 1 --2- 172.21.15.32:0/2848736410 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4bec15c1b0 0x7f4bec138f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:02:47.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4bf1000640 1 --2- 172.21.15.32:0/2848736410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bec0b4de0 0x7f4bec15e1c0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f4be000ec10 tx=0x7f4be000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:47.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4be8c00640 1 -- 172.21.15.32:0/2848736410 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be0017070 con 0x7f4bec0b4de0 2024-04-17T10:02:47.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4be8c00640 1 -- 172.21.15.32:0/2848736410 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4be000cc30 con 0x7f4bec0b4de0 2024-04-17T10:02:47.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4be8c00640 1 -- 172.21.15.32:0/2848736410 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be0013470 con 0x7f4bec0b4de0 2024-04-17T10:02:47.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4bf3a00640 1 -- 172.21.15.32:0/2848736410 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bec0b1b60 con 0x7f4bec0b4de0 2024-04-17T10:02:47.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4bf3a00640 1 -- 172.21.15.32:0/2848736410 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bec0b2120 con 0x7f4bec0b4de0 2024-04-17T10:02:47.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.016+0000 7f4bf3a00640 1 -- 172.21.15.32:0/2848736410 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bec068a90 con 0x7f4bec0b4de0 2024-04-17T10:02:47.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.020+0000 7f4be8c00640 1 -- 172.21.15.32:0/2848736410 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4be001a070 con 0x7f4bec0b4de0 2024-04-17T10:02:47.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.020+0000 7f4be8c00640 1 --2- 172.21.15.32:0/2848736410 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4bcc068380 0x7f4bcc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:47.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.020+0000 7f4be8c00640 1 -- 172.21.15.32:0/2848736410 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f4be0086e40 con 0x7f4bec0b4de0 2024-04-17T10:02:47.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.020+0000 7f4bebe00640 1 --2- 172.21.15.32:0/2848736410 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4bcc068380 0x7f4bcc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:47.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.020+0000 7f4bebe00640 1 --2- 172.21.15.32:0/2848736410 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4bcc068380 0x7f4bcc06a840 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4bec15ef00 tx=0x7f4bd8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:47.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.024+0000 7f4be8c00640 1 -- 172.21.15.32:0/2848736410 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f4be0057300 con 0x7f4bec0b4de0 2024-04-17T10:02:47.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.228+0000 7f4bf3a00640 1 -- 172.21.15.32:0/2848736410 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4bec13afc0 con 0x7f4bec0b4de0 2024-04-17T10:02:47.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.228+0000 7f4be8c00640 1 -- 172.21.15.32:0/2848736410 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5201 (secure 0 0 0) 0x7f4be0056c00 con 0x7f4bec0b4de0 2024-04-17T10:02:47.227 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:02:47.227 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":86,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2635109588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2635109588},{"type":"v1","addr":"172.21.15.144:6835","nonce":2635109588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7789,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3258455939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3258455939},{"type":"v1","addr":"172.21.15.144:6834","nonce":3258455939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":8574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3390468332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3390468332},{"type":"v1","addr":"172.21.15.32:6836","nonce":3390468332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:01:40.557208+0000","modified":"2024-04-17T10:02:46.053370+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8964":{"gid":8964,"name":"c","rank":0,"incarnation":84,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.32:6837/3078068380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3078068380},{"type":"v1","addr":"172.21.15.32:6837","nonce":3078068380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-17T10:02:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 86 2024-04-17T10:02:47.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.232+0000 7f4bf3a00640 1 -- 172.21.15.32:0/2848736410 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4bcc068380 msgr2=0x7f4bcc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:47.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.232+0000 7f4bf3a00640 1 --2- 172.21.15.32:0/2848736410 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4bcc068380 0x7f4bcc06a840 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4bec15ef00 tx=0x7f4bd8009040 comp rx=0 tx=0).stop 2024-04-17T10:02:47.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.232+0000 7f4bf3a00640 1 -- 172.21.15.32:0/2848736410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bec0b4de0 msgr2=0x7f4bec15e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:47.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.232+0000 7f4bf3a00640 1 --2- 172.21.15.32:0/2848736410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bec0b4de0 0x7f4bec15e1c0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f4be000ec10 tx=0x7f4be000c520 comp rx=0 tx=0).stop 2024-04-17T10:02:47.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.232+0000 7f4bf3a00640 1 -- 172.21.15.32:0/2848736410 shutdown_connections 2024-04-17T10:02:47.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.232+0000 7f4bf3a00640 1 --2- 172.21.15.32:0/2848736410 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4bcc068380 0x7f4bcc06a840 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:47.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.232+0000 7f4bf3a00640 1 --2- 172.21.15.32:0/2848736410 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4bec15c1b0 0x7f4bec138f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:47.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.232+0000 7f4bf3a00640 1 --2- 172.21.15.32:0/2848736410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bec144e90 0x7f4bec157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:47.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.232+0000 7f4bf3a00640 1 --2- 172.21.15.32:0/2848736410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bec0b4de0 0x7f4bec15e1c0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:47.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.232+0000 7f4bf3a00640 1 -- 172.21.15.32:0/2848736410 >> 172.21.15.32:0/2848736410 conn(0x7f4bec06ba30 msgr2=0x7f4bec0ccee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:47.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.232+0000 7f4bf3a00640 1 -- 172.21.15.32:0/2848736410 shutdown_connections 2024-04-17T10:02:47.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:47.232+0000 7f4bf3a00640 1 -- 172.21.15.32:0/2848736410 wait complete. 2024-04-17T10:02:47.652 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:01:40.557208+0000', 'modified': '2024-04-17T10:02:46.053370+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 98, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8964}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8964': {'gid': 8964, 'name': 'c', 'rank': 0, 'incarnation': 84, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.32:6837/3078068380', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6835', 'nonce': 3078068380}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 3078068380}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-17T10:02:47.652 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8964:up:rejoin 2024-04-17T10:02:48.653 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:02:48.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.808+0000 7f361b600640 1 Processor -- start 2024-04-17T10:02:48.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.808+0000 7f361b600640 1 -- start start 2024-04-17T10:02:48.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.808+0000 7f361b600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3614145f30 0x7f3614071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:48.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.808+0000 7f361b600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f361415d8d0 0x7f3614072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:48.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.808+0000 7f361b600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36140cdac0 con 0x7f36140cce70 2024-04-17T10:02:48.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.808+0000 7f361b600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f361406f400 con 0x7f3614145f30 2024-04-17T10:02:48.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.808+0000 7f361b600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3614146310 con 0x7f361415d8d0 2024-04-17T10:02:48.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.808+0000 7f3619600640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f36140cce70 0x7f36140cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46212/0 (socket says 172.21.15.32:46212) 2024-04-17T10:02:48.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.808+0000 7f3619600640 1 -- 172.21.15.32:0/182563434 learned_addr learned my addr 172.21.15.32:0/182563434 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:02:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f360be00640 1 --2- 172.21.15.32:0/182563434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f361415d8d0 0x7f3614072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3618c00640 1 --2- 172.21.15.32:0/182563434 >> v2:172.21.15.144:3300/0 conn(0x7f3614145f30 0x7f3614071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3613e00640 1 -- 172.21.15.32:0/182563434 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1651461444 0 0) 0x7f36140cdac0 con 0x7f36140cce70 2024-04-17T10:02:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3613e00640 1 -- 172.21.15.32:0/182563434 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35f8003530 con 0x7f36140cce70 2024-04-17T10:02:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3613e00640 1 -- 172.21.15.32:0/182563434 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2674112305 0 0) 0x7f35f8003530 con 0x7f36140cce70 2024-04-17T10:02:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3613e00640 1 -- 172.21.15.32:0/182563434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f361415d8d0 msgr2=0x7f3614072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3613e00640 1 --2- 172.21.15.32:0/182563434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f361415d8d0 0x7f3614072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3613e00640 1 -- 172.21.15.32:0/182563434 >> v2:172.21.15.144:3300/0 conn(0x7f3614145f30 msgr2=0x7f3614071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3613e00640 1 --2- 172.21.15.32:0/182563434 >> v2:172.21.15.144:3300/0 conn(0x7f3614145f30 0x7f3614071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3613e00640 1 -- 172.21.15.32:0/182563434 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361415cdf0 con 0x7f36140cce70 2024-04-17T10:02:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3613e00640 1 -- 172.21.15.32:0/182563434 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f360c001a00 con 0x7f36140cce70 2024-04-17T10:02:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3613e00640 1 -- 172.21.15.32:0/182563434 >> v1:172.21.15.32:6789/0 conn(0x7f36140cce70 legacy=0x7f36140cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3613e00640 1 --2- 172.21.15.32:0/182563434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f80039c0 0x7f35f8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3613e00640 1 --2- 172.21.15.32:0/182563434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35f8004810 0x7f35f8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3613e00640 1 --2- 172.21.15.32:0/182563434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f361415d8d0 0x7f35f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3613e00640 1 -- 172.21.15.32:0/182563434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f361406f400 con 0x7f35f80039c0 2024-04-17T10:02:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3613e00640 1 -- 172.21.15.32:0/182563434 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3614146310 con 0x7f35f8004810 2024-04-17T10:02:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3613e00640 1 -- 172.21.15.32:0/182563434 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f35f800ba00 con 0x7f361415d8d0 2024-04-17T10:02:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3618c00640 1 --2- 172.21.15.32:0/182563434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f80039c0 0x7f35f8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f360be00640 1 --2- 172.21.15.32:0/182563434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35f8004810 0x7f35f8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:48.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3619600640 1 --2- 172.21.15.32:0/182563434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f361415d8d0 0x7f35f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:48.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3618c00640 1 -- 172.21.15.32:0/182563434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f361415d8d0 msgr2=0x7f35f80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:48.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3618c00640 1 --2- 172.21.15.32:0/182563434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f361415d8d0 0x7f35f80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:48.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3618c00640 1 -- 172.21.15.32:0/182563434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35f8004810 msgr2=0x7f35f8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:48.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3618c00640 1 --2- 172.21.15.32:0/182563434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35f8004810 0x7f35f8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:48.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.812+0000 7f3618c00640 1 -- 172.21.15.32:0/182563434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f8003530 con 0x7f35f80039c0 2024-04-17T10:02:48.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.816+0000 7f3618c00640 1 --2- 172.21.15.32:0/182563434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f80039c0 0x7f35f8003de0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f3604009ae0 tx=0x7f360401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db2c2ed8f6c82ea server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:48.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.816+0000 7f3613e00640 1 -- 172.21.15.32:0/182563434 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36040096b0 con 0x7f35f80039c0 2024-04-17T10:02:48.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.816+0000 7f3613e00640 1 -- 172.21.15.32:0/182563434 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36040a64c0 con 0x7f35f80039c0 2024-04-17T10:02:48.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.816+0000 7f3613e00640 1 -- 172.21.15.32:0/182563434 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36040a68b0 con 0x7f35f80039c0 2024-04-17T10:02:48.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.816+0000 7f361b600640 1 -- 172.21.15.32:0/182563434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f80039c0 msgr2=0x7f35f8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:48.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.816+0000 7f361b600640 1 --2- 172.21.15.32:0/182563434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35f80039c0 0x7f35f8003de0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f3604009ae0 tx=0x7f360401cab0 comp rx=0 tx=0).stop 2024-04-17T10:02:48.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.816+0000 7f3619600640 1 -- 172.21.15.32:0/182563434 reap_dead start 2024-04-17T10:02:48.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.816+0000 7f361b600640 1 -- 172.21.15.32:0/182563434 shutdown_connections 2024-04-17T10:02:48.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.816+0000 7f361b600640 1 -- 172.21.15.32:0/182563434 >> 172.21.15.32:0/182563434 conn(0x7f361406ba30 msgr2=0x7f361406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:48.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.816+0000 7f361b600640 1 -- 172.21.15.32:0/182563434 shutdown_connections 2024-04-17T10:02:48.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.816+0000 7f361b600640 1 -- 172.21.15.32:0/182563434 wait complete. 2024-04-17T10:02:48.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.816+0000 7f361b600640 1 Processor -- start 2024-04-17T10:02:48.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.816+0000 7f361b600640 1 -- start start 2024-04-17T10:02:48.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.816+0000 7f361b600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35f80039c0 0x7f361415c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:48.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.816+0000 7f361b600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36141e6bb0 0x7f36141e7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:48.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.820+0000 7f361b600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36141e7e90 0x7f36140b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.820+0000 7f361b600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f361415cb90 con 0x7f36141e7e90 2024-04-17T10:02:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.820+0000 7f361b600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f361415d440 con 0x7f35f80039c0 2024-04-17T10:02:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.820+0000 7f361b600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3614072330 con 0x7f36141e6bb0 2024-04-17T10:02:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.820+0000 7f3619600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36141e7e90 0x7f36140b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.820+0000 7f3619600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36141e7e90 0x7f36140b3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44300/0 (socket says 172.21.15.32:44300) 2024-04-17T10:02:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.820+0000 7f3619600640 1 -- 172.21.15.32:0/1316399676 learned_addr learned my addr 172.21.15.32:0/1316399676 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.820+0000 7f360be00640 1 --2- 172.21.15.32:0/1316399676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36141e6bb0 0x7f36141e7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.820+0000 7f3618c00640 1 --2- 172.21.15.32:0/1316399676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35f80039c0 0x7f361415c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.820+0000 7f3619600640 1 -- 172.21.15.32:0/1316399676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36141e6bb0 msgr2=0x7f36141e7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.820+0000 7f3619600640 1 --2- 172.21.15.32:0/1316399676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36141e6bb0 0x7f36141e7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.820+0000 7f3619600640 1 -- 172.21.15.32:0/1316399676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35f80039c0 msgr2=0x7f361415c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.820+0000 7f3619600640 1 --2- 172.21.15.32:0/1316399676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35f80039c0 0x7f361415c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.820+0000 7f3619600640 1 -- 172.21.15.32:0/1316399676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36141e8680 con 0x7f36141e7e90 2024-04-17T10:02:48.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.820+0000 7f3619600640 1 --2- 172.21.15.32:0/1316399676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36141e7e90 0x7f36140b3e90 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f361415dd30 tx=0x7f360c011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.820+0000 7f3611600640 1 -- 172.21.15.32:0/1316399676 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c010040 con 0x7f36141e7e90 2024-04-17T10:02:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.820+0000 7f3611600640 1 -- 172.21.15.32:0/1316399676 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f360c01a070 con 0x7f36141e7e90 2024-04-17T10:02:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.820+0000 7f3611600640 1 -- 172.21.15.32:0/1316399676 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c00fb40 con 0x7f36141e7e90 2024-04-17T10:02:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.820+0000 7f361b600640 1 -- 172.21.15.32:0/1316399676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f361415c590 con 0x7f36141e7e90 2024-04-17T10:02:48.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.820+0000 7f361b600640 1 -- 172.21.15.32:0/1316399676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36141e60d0 con 0x7f36141e7e90 2024-04-17T10:02:48.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.820+0000 7f361b600640 1 -- 172.21.15.32:0/1316399676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3614068a90 con 0x7f36141e7e90 2024-04-17T10:02:48.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.820+0000 7f3611600640 1 -- 172.21.15.32:0/1316399676 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f360c00fce0 con 0x7f36141e7e90 2024-04-17T10:02:48.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.824+0000 7f3611600640 1 --2- 172.21.15.32:0/1316399676 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f35f0068380 0x7f35f006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:48.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.824+0000 7f3611600640 1 -- 172.21.15.32:0/1316399676 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f360c089730 con 0x7f36141e7e90 2024-04-17T10:02:48.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.824+0000 7f3618c00640 1 --2- 172.21.15.32:0/1316399676 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f35f0068380 0x7f35f006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:48.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.824+0000 7f3618c00640 1 --2- 172.21.15.32:0/1316399676 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f35f0068380 0x7f35f006a840 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f36040af000 tx=0x7f36040046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:48.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:48.828+0000 7f3611600640 1 -- 172.21.15.32:0/1316399676 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f360c059640 con 0x7f36141e7e90 2024-04-17T10:02:49.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.028+0000 7f361b600640 1 -- 172.21.15.32:0/1316399676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3614072f90 con 0x7f36141e7e90 2024-04-17T10:02:49.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.032+0000 7f3611600640 1 -- 172.21.15.32:0/1316399676 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5208 (secure 0 0 0) 0x7f360c05d940 con 0x7f36141e7e90 2024-04-17T10:02:49.028 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:02:49.029 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":87,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2635109588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2635109588},{"type":"v1","addr":"172.21.15.144:6835","nonce":2635109588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7789,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3258455939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3258455939},{"type":"v1","addr":"172.21.15.144:6834","nonce":3258455939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":8574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3390468332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3390468332},{"type":"v1","addr":"172.21.15.32:6836","nonce":3390468332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:01:40.557208+0000","modified":"2024-04-17T10:02:48.058401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8964":{"gid":8964,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6837/3078068380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3078068380},{"type":"v1","addr":"172.21.15.32:6837","nonce":3078068380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8964,"qdb_cluster":[8964]},"id":7}]} 2024-04-17T10:02:49.029 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 87 2024-04-17T10:02:49.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.032+0000 7f361b600640 1 -- 172.21.15.32:0/1316399676 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f35f0068380 msgr2=0x7f35f006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.032+0000 7f361b600640 1 --2- 172.21.15.32:0/1316399676 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f35f0068380 0x7f35f006a840 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f36040af000 tx=0x7f36040046d0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.032+0000 7f361b600640 1 -- 172.21.15.32:0/1316399676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36141e7e90 msgr2=0x7f36140b3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.032+0000 7f361b600640 1 --2- 172.21.15.32:0/1316399676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36141e7e90 0x7f36140b3e90 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f361415dd30 tx=0x7f360c011c20 comp rx=0 tx=0).stop 2024-04-17T10:02:49.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.032+0000 7f361b600640 1 -- 172.21.15.32:0/1316399676 shutdown_connections 2024-04-17T10:02:49.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.032+0000 7f361b600640 1 --2- 172.21.15.32:0/1316399676 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f35f0068380 0x7f35f006a840 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.032+0000 7f361b600640 1 --2- 172.21.15.32:0/1316399676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36141e6bb0 0x7f36141e7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.032+0000 7f361b600640 1 --2- 172.21.15.32:0/1316399676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36141e7e90 0x7f36140b3e90 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.032+0000 7f361b600640 1 --2- 172.21.15.32:0/1316399676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35f80039c0 0x7f361415c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.032+0000 7f361b600640 1 -- 172.21.15.32:0/1316399676 >> 172.21.15.32:0/1316399676 conn(0x7f361406ba30 msgr2=0x7f36140b2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:49.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.032+0000 7f361b600640 1 -- 172.21.15.32:0/1316399676 shutdown_connections 2024-04-17T10:02:49.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.032+0000 7f361b600640 1 -- 172.21.15.32:0/1316399676 wait complete. 2024-04-17T10:02:49.454 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:01:40.557208+0000', 'modified': '2024-04-17T10:02:48.058401+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 98, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8964}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8964': {'gid': 8964, 'name': 'c', 'rank': 0, 'incarnation': 84, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.32:6837/3078068380', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6835', 'nonce': 3078068380}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 3078068380}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8964, 'qdb_cluster': [8964]} 2024-04-17T10:02:49.454 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T10:02:49.455 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-17T10:02:49.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.600+0000 7f1671200640 1 Processor -- start 2024-04-17T10:02:49.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.600+0000 7f1671200640 1 -- start start 2024-04-17T10:02:49.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.600+0000 7f1671200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f166c13f6e0 0x7f166c0cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.600+0000 7f1671200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f166c0d09f0 0x7f166c071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.600+0000 7f1671200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f166c1601e0 con 0x7f166c0cf370 2024-04-17T10:02:49.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.600+0000 7f1671200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f166c13fac0 con 0x7f166c0d09f0 2024-04-17T10:02:49.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.600+0000 7f1671200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f166c160480 con 0x7f166c13f6e0 2024-04-17T10:02:49.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.600+0000 7f1661400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f166c0cf370 0x7f166c0d02e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46216/0 (socket says 172.21.15.32:46216) 2024-04-17T10:02:49.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.600+0000 7f1661400640 1 -- 172.21.15.32:0/4294531504 learned_addr learned my addr 172.21.15.32:0/4294531504 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:02:49.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.600+0000 7f166a800640 1 --2- 172.21.15.32:0/4294531504 >> v2:172.21.15.32:3300/0 conn(0x7f166c0d09f0 0x7f166c071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.600+0000 7f1669e00640 1 --2- 172.21.15.32:0/4294531504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f166c13f6e0 0x7f166c0cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.600+0000 7f1669400640 1 -- 172.21.15.32:0/4294531504 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3993011313 0 0) 0x7f166c1601e0 con 0x7f166c0cf370 2024-04-17T10:02:49.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.600+0000 7f1669400640 1 -- 172.21.15.32:0/4294531504 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1650003530 con 0x7f166c0cf370 2024-04-17T10:02:49.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.604+0000 7f166a800640 1 -- 172.21.15.32:0/4294531504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f166c13f6e0 msgr2=0x7f166c0cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:49.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.604+0000 7f166a800640 1 --2- 172.21.15.32:0/4294531504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f166c13f6e0 0x7f166c0cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.604+0000 7f166a800640 1 -- 172.21.15.32:0/4294531504 >> v1:172.21.15.32:6789/0 conn(0x7f166c0cf370 legacy=0x7f166c0d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.604+0000 7f166a800640 1 -- 172.21.15.32:0/4294531504 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c160760 con 0x7f166c0d09f0 2024-04-17T10:02:49.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.604+0000 7f166a800640 1 --2- 172.21.15.32:0/4294531504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166c0d09f0 0x7f166c071d80 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f1664009d90 tx=0x7f166401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aaf444fc6a357d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:49.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.604+0000 7f1669400640 1 -- 172.21.15.32:0/4294531504 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16640b2070 con 0x7f166c0d09f0 2024-04-17T10:02:49.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.604+0000 7f1669400640 1 -- 172.21.15.32:0/4294531504 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16640a5a00 con 0x7f166c0d09f0 2024-04-17T10:02:49.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.604+0000 7f1669400640 1 -- 172.21.15.32:0/4294531504 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16640a5dd0 con 0x7f166c0d09f0 2024-04-17T10:02:49.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.604+0000 7f1671200640 1 -- 172.21.15.32:0/4294531504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166c0d09f0 msgr2=0x7f166c071d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.604+0000 7f1671200640 1 --2- 172.21.15.32:0/4294531504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166c0d09f0 0x7f166c071d80 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f1664009d90 tx=0x7f166401c940 comp rx=0 tx=0).stop 2024-04-17T10:02:49.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.604+0000 7f1671200640 1 -- 172.21.15.32:0/4294531504 shutdown_connections 2024-04-17T10:02:49.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.604+0000 7f1671200640 1 --2- 172.21.15.32:0/4294531504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166c0d09f0 0x7f166c071d80 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.604+0000 7f1671200640 1 --2- 172.21.15.32:0/4294531504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f166c13f6e0 0x7f166c0cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.604+0000 7f1671200640 1 -- 172.21.15.32:0/4294531504 >> 172.21.15.32:0/4294531504 conn(0x7f166c06ba30 msgr2=0x7f166c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:49.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.604+0000 7f1671200640 1 -- 172.21.15.32:0/4294531504 shutdown_connections 2024-04-17T10:02:49.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.604+0000 7f1671200640 1 -- 172.21.15.32:0/4294531504 wait complete. 2024-04-17T10:02:49.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f1671200640 1 Processor -- start 2024-04-17T10:02:49.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f1671200640 1 -- start start 2024-04-17T10:02:49.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f1671200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f166c0cf370 0x7f166c13a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f1671200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f166c0d09f0 0x7f166c0b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f1671200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166c13f6e0 0x7f166c1f67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f1671200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f166c0cff10 con 0x7f166c13f6e0 2024-04-17T10:02:49.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f1671200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f166c0cfd90 con 0x7f166c0cf370 2024-04-17T10:02:49.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f1671200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f166c072ad0 con 0x7f166c0d09f0 2024-04-17T10:02:49.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f166a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166c13f6e0 0x7f166c1f67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f166a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166c13f6e0 0x7f166c1f67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44318/0 (socket says 172.21.15.32:44318) 2024-04-17T10:02:49.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f166a800640 1 -- 172.21.15.32:0/1084841187 learned_addr learned my addr 172.21.15.32:0/1084841187 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:49.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f1661400640 1 --2- 172.21.15.32:0/1084841187 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f166c0d09f0 0x7f166c0b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f1669e00640 1 --2- 172.21.15.32:0/1084841187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f166c0cf370 0x7f166c13a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f166a800640 1 -- 172.21.15.32:0/1084841187 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f166c0d09f0 msgr2=0x7f166c0b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f166a800640 1 --2- 172.21.15.32:0/1084841187 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f166c0d09f0 0x7f166c0b14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f166a800640 1 -- 172.21.15.32:0/1084841187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f166c0cf370 msgr2=0x7f166c13a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f166a800640 1 --2- 172.21.15.32:0/1084841187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f166c0cf370 0x7f166c13a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f166a800640 1 -- 172.21.15.32:0/1084841187 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c0b5080 con 0x7f166c13f6e0 2024-04-17T10:02:49.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f1661400640 1 --2- 172.21.15.32:0/1084841187 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f166c0d09f0 0x7f166c0b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:02:49.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f1669e00640 1 --2- 172.21.15.32:0/1084841187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f166c0cf370 0x7f166c13a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:02:49.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f166a800640 1 --2- 172.21.15.32:0/1084841187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166c13f6e0 0x7f166c1f67a0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f1664009d90 tx=0x7f16640196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:49.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f1662a00640 1 -- 172.21.15.32:0/1084841187 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16640b2050 con 0x7f166c13f6e0 2024-04-17T10:02:49.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f1662a00640 1 -- 172.21.15.32:0/1084841187 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16640be070 con 0x7f166c13f6e0 2024-04-17T10:02:49.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f1671200640 1 -- 172.21.15.32:0/1084841187 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166c0b5360 con 0x7f166c13f6e0 2024-04-17T10:02:49.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f1662a00640 1 -- 172.21.15.32:0/1084841187 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16640ada90 con 0x7f166c13f6e0 2024-04-17T10:02:49.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.608+0000 7f1671200640 1 -- 172.21.15.32:0/1084841187 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f166c0b36f0 con 0x7f166c13f6e0 2024-04-17T10:02:49.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.612+0000 7f1662a00640 1 -- 172.21.15.32:0/1084841187 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f16640adc30 con 0x7f166c13f6e0 2024-04-17T10:02:49.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.612+0000 7f1662a00640 1 --2- 172.21.15.32:0/1084841187 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1648068380 0x7f164806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.612+0000 7f1662a00640 1 -- 172.21.15.32:0/1084841187 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f1664121ef0 con 0x7f166c13f6e0 2024-04-17T10:02:49.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.612+0000 7f1669e00640 1 --2- 172.21.15.32:0/1084841187 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1648068380 0x7f164806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.612+0000 7f16745ef000 1 Processor -- start 2024-04-17T10:02:49.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.612+0000 7f16745ef000 1 -- start start 2024-04-17T10:02:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.612+0000 7f1669e00640 1 --2- 172.21.15.32:0/1084841187 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1648068380 0x7f164806a840 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f165c009bb0 tx=0x7f165c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.612+0000 7f16745ef000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563efe307b50 0x563efe307f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.612+0000 7f16745ef000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe308470 0x563efe2bd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.612+0000 7f16745ef000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563efe2bdcf0 0x563efe2be0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.612+0000 7f16745ef000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563efe10b4d0 con 0x563efe308470 2024-04-17T10:02:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.612+0000 7f16745ef000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563efe0a5bf0 con 0x563efe307b50 2024-04-17T10:02:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.612+0000 7f16745ef000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x563efe107490 con 0x563efe2bdcf0 2024-04-17T10:02:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f1661400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe308470 0x563efe2bd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f1661400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe308470 0x563efe2bd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44320/0 (socket says 172.21.15.32:44320) 2024-04-17T10:02:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f1661400640 1 -- 172.21.15.32:0/293841930 learned_addr learned my addr 172.21.15.32:0/293841930 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f1669e00640 1 --2- 172.21.15.32:0/293841930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563efe307b50 0x563efe307f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f166a800640 1 --2- 172.21.15.32:0/293841930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563efe2bdcf0 0x563efe2be0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f1661400640 1 -- 172.21.15.32:0/293841930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563efe2bdcf0 msgr2=0x563efe2be0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f1661400640 1 --2- 172.21.15.32:0/293841930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563efe2bdcf0 0x563efe2be0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f1661400640 1 -- 172.21.15.32:0/293841930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563efe307b50 msgr2=0x563efe307f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f1661400640 1 --2- 172.21.15.32:0/293841930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563efe307b50 0x563efe307f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f1661400640 1 -- 172.21.15.32:0/293841930 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563efe091e80 con 0x563efe308470 2024-04-17T10:02:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f1661400640 1 --2- 172.21.15.32:0/293841930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe308470 0x563efe2bd7b0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f16540015e0 tx=0x7f1654000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83335bd534ff95a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f165aa00640 1 -- 172.21.15.32:0/293841930 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654019070 con 0x563efe308470 2024-04-17T10:02:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f165aa00640 1 -- 172.21.15.32:0/293841930 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16540058c0 con 0x563efe308470 2024-04-17T10:02:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f165aa00640 1 -- 172.21.15.32:0/293841930 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654005cb0 con 0x563efe308470 2024-04-17T10:02:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f16745ef000 1 -- 172.21.15.32:0/293841930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe308470 msgr2=0x563efe2bd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f16745ef000 1 --2- 172.21.15.32:0/293841930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe308470 0x563efe2bd7b0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f16540015e0 tx=0x7f1654000bd0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f16745ef000 1 -- 172.21.15.32:0/293841930 shutdown_connections 2024-04-17T10:02:49.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f16745ef000 1 --2- 172.21.15.32:0/293841930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563efe2bdcf0 0x563efe2be0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f16745ef000 1 --2- 172.21.15.32:0/293841930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe308470 0x563efe2bd7b0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f16745ef000 1 --2- 172.21.15.32:0/293841930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563efe307b50 0x563efe307f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f16745ef000 1 -- 172.21.15.32:0/293841930 >> 172.21.15.32:0/293841930 conn(0x563efe2c3470 msgr2=0x563efe2bc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:49.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f16745ef000 1 -- 172.21.15.32:0/293841930 shutdown_connections 2024-04-17T10:02:49.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f16745ef000 1 -- 172.21.15.32:0/293841930 wait complete. 2024-04-17T10:02:49.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f16745ef000 1 -- start start 2024-04-17T10:02:49.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f16745ef000 1 Processor -- start 2024-04-17T10:02:49.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f16745ef000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563efe2bdcf0 0x563efe2c2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f16745ef000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563efe307b50 0x563efe2f5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f16745ef000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe308470 0x563efe2f5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f16745ef000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563efe0c8a60 con 0x563efe308470 2024-04-17T10:02:49.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f16745ef000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563efe0afe80 con 0x563efe307b50 2024-04-17T10:02:49.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f16745ef000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x563efe0c6cf0 con 0x563efe2bdcf0 2024-04-17T10:02:49.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f16745ef000 10 client.0 _ll_register_callbacks cb 0x563efe2a8a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:02:49.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.616+0000 7f166a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe308470 0x563efe2f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f1669e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563efe2bdcf0 0x563efe2c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f1661400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563efe307b50 0x563efe2f5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f1661400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563efe307b50 0x563efe2f5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:49630/0 (socket says 172.21.15.32:49630) 2024-04-17T10:02:49.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f1661400640 1 -- 172.21.15.32:0/2049317101 learned_addr learned my addr 172.21.15.32:0/2049317101 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:49.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f166a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe308470 0x563efe2f5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44322/0 (socket says 172.21.15.32:44322) 2024-04-17T10:02:49.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f1669e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563efe2bdcf0 0x563efe2c2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:53036/0 (socket says 172.21.15.32:53036) 2024-04-17T10:02:49.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f166a800640 1 -- 172.21.15.32:0/2049317101 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563efe2bdcf0 msgr2=0x563efe2c2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:49.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f166a800640 1 --2- 172.21.15.32:0/2049317101 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563efe2bdcf0 0x563efe2c2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f166a800640 1 -- 172.21.15.32:0/2049317101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563efe307b50 msgr2=0x563efe2f5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:49.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f166a800640 1 --2- 172.21.15.32:0/2049317101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563efe307b50 0x563efe2f5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f166a800640 1 -- 172.21.15.32:0/2049317101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563efe0a5e40 con 0x563efe308470 2024-04-17T10:02:49.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f166a800640 1 --2- 172.21.15.32:0/2049317101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe308470 0x563efe2f5980 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f166401ceb0 tx=0x7f16640c3000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4981d09cad47b4bc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:49.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f164f400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:02:49.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f164f400640 1 -- 172.21.15.32:0/2049317101 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16640f5070 con 0x563efe308470 2024-04-17T10:02:49.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f164f400640 1 -- 172.21.15.32:0/2049317101 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f16640fa720 con 0x563efe308470 2024-04-17T10:02:49.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f16745ef000 1 -- 172.21.15.32:0/2049317101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16540058c0 con 0x563efe308470 2024-04-17T10:02:49.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f164f400640 1 -- 172.21.15.32:0/2049317101 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f16640fa9b0 con 0x563efe308470 2024-04-17T10:02:49.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f16745ef000 10 client.9036 fetch_fsmap learned FSMap version 87 2024-04-17T10:02:49.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f16745ef000 1 -- 172.21.15.32:0/2049317101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x563efe0d6e00 con 0x563efe308470 2024-04-17T10:02:49.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f164f400640 1 -- 172.21.15.32:0/2049317101 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16540058c0 con 0x563efe308470 2024-04-17T10:02:49.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f164f400640 1 -- 172.21.15.32:0/2049317101 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 87) ==== 3495+0+0 (secure 0 0 0) 0x7f16640f12d0 con 0x563efe308470 2024-04-17T10:02:49.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f16745ef000 10 client.9036 fetch_fsmap finished waiting for FSMap version 87 2024-04-17T10:02:49.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f16745ef000 10 client.9036 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T10:02:49.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f16745ef000 20 client.9036 populate_metadata read hostname 'smithi032' 2024-04-17T10:02:49.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f16745ef000 1 --2- 172.21.15.32:0/2049317101 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x563efe3170b0 0x563efe3194a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f16745ef000 4 client.9036 mds_command: new command op to 8964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:02:49.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f16745ef000 1 -- 172.21.15.32:0/2049317101 --> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563efe0a4bc0 con 0x563efe3170b0 2024-04-17T10:02:49.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f1669e00640 1 --2- 172.21.15.32:0/2049317101 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x563efe3170b0 0x563efe3194a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f164f400640 10 client.9036 ms_handle_connect on v2:172.21.15.32:6835/3078068380 2024-04-17T10:02:49.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f1669e00640 1 --2- 172.21.15.32:0/2049317101 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x563efe3170b0 0x563efe3194a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0b4b8cdaed1ea0f server_cookie=99150035ff301a2d in_seq=0 out_seq=0 2024-04-17T10:02:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f164f400640 1 -- 172.21.15.32:0/2049317101 <== mds.0 v2:172.21.15.32:6835/3078068380 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563efe0a4bc0 con 0x563efe3170b0 2024-04-17T10:02:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f164f400640 10 client.9036 handle_command_reply: tid=0 2024-04-17T10:02:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f16745ef000 1 client.9036 shutdown 2024-04-17T10:02:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f16745ef000 1 -- 172.21.15.32:0/2049317101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe308470 msgr2=0x563efe2f5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f16745ef000 1 --2- 172.21.15.32:0/2049317101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe308470 0x563efe2f5980 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f166401ceb0 tx=0x7f16640c3000 comp rx=0 tx=0).stop 2024-04-17T10:02:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f16745ef000 1 -- 172.21.15.32:0/2049317101 shutdown_connections 2024-04-17T10:02:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f16745ef000 1 --2- 172.21.15.32:0/2049317101 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x563efe3170b0 0x563efe3194a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f16745ef000 1 --2- 172.21.15.32:0/2049317101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe308470 0x563efe2f5980 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f16745ef000 1 --2- 172.21.15.32:0/2049317101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563efe307b50 0x563efe2f5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f164f400640 0 client.9036 ms_handle_reset on v2:172.21.15.32:6835/3078068380 2024-04-17T10:02:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f16745ef000 1 --2- 172.21.15.32:0/2049317101 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563efe2bdcf0 0x563efe2c2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f16745ef000 1 -- 172.21.15.32:0/2049317101 >> 172.21.15.32:0/2049317101 conn(0x563efe2c3470 msgr2=0x563efe2bc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f16745ef000 1 -- 172.21.15.32:0/2049317101 shutdown_connections 2024-04-17T10:02:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f16745ef000 1 -- 172.21.15.32:0/2049317101 wait complete. 2024-04-17T10:02:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.620+0000 7f16745ef000 20 client.9036 trim_cache size 0 max 16384 2024-04-17T10:02:49.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.640+0000 7f16745ef000 1 Processor -- start 2024-04-17T10:02:49.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.640+0000 7f16745ef000 1 -- start start 2024-04-17T10:02:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.640+0000 7f16745ef000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe2bdcf0 0x563efe2f5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.640+0000 7f16745ef000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563efe307b50 0x563efe2f5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.640+0000 7f16745ef000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563efe308470 0x563efe3184a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.640+0000 7f16745ef000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f166c2029b0 con 0x563efe2bdcf0 2024-04-17T10:02:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.640+0000 7f16745ef000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f166c1fa860 con 0x563efe307b50 2024-04-17T10:02:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.640+0000 7f16745ef000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f166c1faeb0 con 0x563efe308470 2024-04-17T10:02:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.640+0000 7f1669e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe2bdcf0 0x563efe2f5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.640+0000 7f1669e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe2bdcf0 0x563efe2f5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44338/0 (socket says 172.21.15.32:44338) 2024-04-17T10:02:49.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.640+0000 7f1669e00640 1 -- 172.21.15.32:0/3918821001 learned_addr learned my addr 172.21.15.32:0/3918821001 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:49.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.640+0000 7f1669e00640 1 -- 172.21.15.32:0/3918821001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563efe308470 msgr2=0x563efe3184a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:49.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.640+0000 7f1669e00640 1 --2- 172.21.15.32:0/3918821001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563efe308470 0x563efe3184a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.640+0000 7f1669e00640 1 -- 172.21.15.32:0/3918821001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563efe307b50 msgr2=0x563efe2f5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:49.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.640+0000 7f1661400640 1 --2- 172.21.15.32:0/3918821001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563efe307b50 0x563efe2f5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.640+0000 7f1669e00640 1 --2- 172.21.15.32:0/3918821001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563efe307b50 0x563efe2f5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.640+0000 7f1669e00640 1 -- 172.21.15.32:0/3918821001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563efe18cd30 con 0x563efe2bdcf0 2024-04-17T10:02:49.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.640+0000 7f1661400640 1 --2- 172.21.15.32:0/3918821001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563efe307b50 0x563efe2f5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:02:49.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.640+0000 7f1669e00640 1 --2- 172.21.15.32:0/3918821001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe2bdcf0 0x563efe2f5020 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f165c011eb0 tx=0x7f165c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbed85e01e2e067a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:49.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f165a000640 1 -- 172.21.15.32:0/3918821001 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c032070 con 0x563efe2bdcf0 2024-04-17T10:02:49.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f165a000640 1 -- 172.21.15.32:0/3918821001 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f165c01d040 con 0x563efe2bdcf0 2024-04-17T10:02:49.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f165a000640 1 -- 172.21.15.32:0/3918821001 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c02c3d0 con 0x563efe2bdcf0 2024-04-17T10:02:49.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f16745ef000 1 -- 172.21.15.32:0/3918821001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe2bdcf0 msgr2=0x563efe2f5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f16745ef000 1 --2- 172.21.15.32:0/3918821001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe2bdcf0 0x563efe2f5020 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f165c011eb0 tx=0x7f165c014000 comp rx=0 tx=0).stop 2024-04-17T10:02:49.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f16745ef000 1 -- 172.21.15.32:0/3918821001 shutdown_connections 2024-04-17T10:02:49.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f16745ef000 1 --2- 172.21.15.32:0/3918821001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563efe308470 0x563efe3184a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f16745ef000 1 --2- 172.21.15.32:0/3918821001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563efe307b50 0x563efe2f5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f16745ef000 1 --2- 172.21.15.32:0/3918821001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe2bdcf0 0x563efe2f5020 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f16745ef000 1 -- 172.21.15.32:0/3918821001 >> 172.21.15.32:0/3918821001 conn(0x563efe2c3470 msgr2=0x563efe2f4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:49.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f16745ef000 1 -- 172.21.15.32:0/3918821001 shutdown_connections 2024-04-17T10:02:49.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f16745ef000 1 -- 172.21.15.32:0/3918821001 wait complete. 2024-04-17T10:02:49.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f16745ef000 1 -- start start 2024-04-17T10:02:49.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f16745ef000 1 Processor -- start 2024-04-17T10:02:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f16745ef000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563efe2bdcf0 0x563efe31bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f16745ef000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe307b50 0x563efe33b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f16745ef000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563efe308470 0x563efe33bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f16745ef000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563efe10fb00 con 0x563efe307b50 2024-04-17T10:02:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f16745ef000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563efe10a7e0 con 0x563efe2bdcf0 2024-04-17T10:02:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f16745ef000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x563efe1186e0 con 0x563efe308470 2024-04-17T10:02:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f16745ef000 10 client.0 _ll_register_callbacks cb 0x7f16640f1c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:02:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f1661400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe307b50 0x563efe33b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f1661400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe307b50 0x563efe33b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44350/0 (socket says 172.21.15.32:44350) 2024-04-17T10:02:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f1661400640 1 -- 172.21.15.32:0/3808796515 learned_addr learned my addr 172.21.15.32:0/3808796515 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f1669e00640 1 --2- 172.21.15.32:0/3808796515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563efe2bdcf0 0x563efe31bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f166a800640 1 --2- 172.21.15.32:0/3808796515 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563efe308470 0x563efe33bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f1661400640 1 -- 172.21.15.32:0/3808796515 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563efe308470 msgr2=0x563efe33bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f1661400640 1 --2- 172.21.15.32:0/3808796515 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563efe308470 0x563efe33bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f1661400640 1 -- 172.21.15.32:0/3808796515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563efe2bdcf0 msgr2=0x563efe31bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f1661400640 1 --2- 172.21.15.32:0/3808796515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563efe2bdcf0 0x563efe31bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f1661400640 1 -- 172.21.15.32:0/3808796515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563efe09d7c0 con 0x563efe307b50 2024-04-17T10:02:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f166a800640 1 --2- 172.21.15.32:0/3808796515 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563efe308470 0x563efe33bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:02:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f1669e00640 1 --2- 172.21.15.32:0/3808796515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563efe2bdcf0 0x563efe31bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:02:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f1661400640 1 --2- 172.21.15.32:0/3808796515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe307b50 0x563efe33b840 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f1654017040 tx=0x7f1654000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aba77f0e548c4cc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f164f400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:02:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f164f400640 1 -- 172.21.15.32:0/3808796515 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654024020 con 0x563efe307b50 2024-04-17T10:02:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f164f400640 1 -- 172.21.15.32:0/3808796515 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f16540070d0 con 0x563efe307b50 2024-04-17T10:02:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f16745ef000 1 -- 172.21.15.32:0/3808796515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f165c01d040 con 0x563efe307b50 2024-04-17T10:02:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f164f400640 1 -- 172.21.15.32:0/3808796515 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f165402e050 con 0x563efe307b50 2024-04-17T10:02:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f16745ef000 10 client.9042 fetch_fsmap learned FSMap version 87 2024-04-17T10:02:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.644+0000 7f16745ef000 1 -- 172.21.15.32:0/3808796515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x563efe0b0650 con 0x563efe307b50 2024-04-17T10:02:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f164f400640 1 -- 172.21.15.32:0/3808796515 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f165c01d040 con 0x563efe307b50 2024-04-17T10:02:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f164f400640 1 -- 172.21.15.32:0/3808796515 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 87) ==== 3495+0+0 (secure 0 0 0) 0x7f1654031020 con 0x563efe307b50 2024-04-17T10:02:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f16745ef000 10 client.9042 fetch_fsmap finished waiting for FSMap version 87 2024-04-17T10:02:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f16745ef000 10 client.9042 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-17T10:02:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f16745ef000 20 client.9042 populate_metadata read hostname 'smithi032' 2024-04-17T10:02:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f16745ef000 1 --2- 172.21.15.32:0/3808796515 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x563efe37ee80 0x563efe37f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f16745ef000 4 client.9042 mds_command: new command op to 8964 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T10:02:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f16745ef000 1 -- 172.21.15.32:0/3808796515 --> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] -- command(tid 0: {"prefix": "status"}) -- 0x563efe118ef0 con 0x563efe37ee80 2024-04-17T10:02:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f1669e00640 1 --2- 172.21.15.32:0/3808796515 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x563efe37ee80 0x563efe37f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f164f400640 10 client.9042 ms_handle_connect on v2:172.21.15.32:6835/3078068380 2024-04-17T10:02:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f1669e00640 1 --2- 172.21.15.32:0/3808796515 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x563efe37ee80 0x563efe37f260 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56e28677084eba4 server_cookie=62635550c56087ef in_seq=0 out_seq=0 2024-04-17T10:02:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f164f400640 1 -- 172.21.15.32:0/3808796515 <== mds.0 v2:172.21.15.32:6835/3078068380 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x563efe118ef0 con 0x563efe37ee80 2024-04-17T10:02:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f164f400640 10 client.9042 handle_command_reply: tid=0 2024-04-17T10:02:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f16745ef000 1 client.9042 shutdown 2024-04-17T10:02:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f16745ef000 1 -- 172.21.15.32:0/3808796515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe307b50 msgr2=0x563efe33b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f16745ef000 1 --2- 172.21.15.32:0/3808796515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe307b50 0x563efe33b840 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f1654017040 tx=0x7f1654000ee0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f16745ef000 1 -- 172.21.15.32:0/3808796515 shutdown_connections 2024-04-17T10:02:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f16745ef000 1 --2- 172.21.15.32:0/3808796515 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563efe308470 0x563efe33bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f16745ef000 1 --2- 172.21.15.32:0/3808796515 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x563efe37ee80 0x563efe37f260 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f16745ef000 1 --2- 172.21.15.32:0/3808796515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563efe307b50 0x563efe33b840 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f16745ef000 1 --2- 172.21.15.32:0/3808796515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563efe2bdcf0 0x563efe31bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f164f400640 0 client.9042 ms_handle_reset on v2:172.21.15.32:6835/3078068380 2024-04-17T10:02:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f16745ef000 1 -- 172.21.15.32:0/3808796515 >> 172.21.15.32:0/3808796515 conn(0x563efe2c3470 msgr2=0x563efe2bf070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f16745ef000 1 -- 172.21.15.32:0/3808796515 shutdown_connections 2024-04-17T10:02:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f16745ef000 1 -- 172.21.15.32:0/3808796515 wait complete. 2024-04-17T10:02:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f16745ef000 20 client.9042 trim_cache size 0 max 16384 2024-04-17T10:02:49.650 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:02:49.650 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "ff178df8-4c06-4367-a1e9-2e3a66d870d0", 2024-04-17T10:02:49.650 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-17T10:02:49.650 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 8964, 2024-04-17T10:02:49.650 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-17T10:02:49.650 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-17T10:02:49.650 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-17T10:02:49.650 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 4.6038778840000001, 2024-04-17T10:02:49.650 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 87, 2024-04-17T10:02:49.650 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 98, 2024-04-17T10:02:49.650 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 98, 2024-04-17T10:02:49.651 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 5.5459437119999997 2024-04-17T10:02:49.651 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:02:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f1671200640 1 -- 172.21.15.32:0/1084841187 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1648068380 msgr2=0x7f164806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f1671200640 1 --2- 172.21.15.32:0/1084841187 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1648068380 0x7f164806a840 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f165c009bb0 tx=0x7f165c008040 comp rx=0 tx=0).stop 2024-04-17T10:02:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f1671200640 1 -- 172.21.15.32:0/1084841187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166c13f6e0 msgr2=0x7f166c1f67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f1671200640 1 --2- 172.21.15.32:0/1084841187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166c13f6e0 0x7f166c1f67a0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f1664009d90 tx=0x7f16640196e0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f1671200640 1 -- 172.21.15.32:0/1084841187 shutdown_connections 2024-04-17T10:02:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f1671200640 1 --2- 172.21.15.32:0/1084841187 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1648068380 0x7f164806a840 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f1671200640 1 --2- 172.21.15.32:0/1084841187 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f166c0d09f0 0x7f166c0b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f1671200640 1 --2- 172.21.15.32:0/1084841187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f166c13f6e0 0x7f166c1f67a0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f1671200640 1 --2- 172.21.15.32:0/1084841187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f166c0cf370 0x7f166c13a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f1671200640 1 -- 172.21.15.32:0/1084841187 >> 172.21.15.32:0/1084841187 conn(0x7f166c06ba30 msgr2=0x7f166c06be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f1671200640 1 -- 172.21.15.32:0/1084841187 shutdown_connections 2024-04-17T10:02:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.648+0000 7f1671200640 1 -- 172.21.15.32:0/1084841187 wait complete. 2024-04-17T10:02:49.664 DEBUG:teuthology.orchestra.run.smithi032:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-17T10:02:49.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:02:49.668 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-17T10:02:49.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.816+0000 7f9f67600640 1 Processor -- start 2024-04-17T10:02:49.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.816+0000 7f9f67600640 1 -- start start 2024-04-17T10:02:49.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.816+0000 7f9f67600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9f600cd870 0x7f9f601661a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.816+0000 7f9f67600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f6015f200 con 0x7f9f6006f500 2024-04-17T10:02:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.816+0000 7f9f67600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f6015f4e0 con 0x7f9f600cfc70 2024-04-17T10:02:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.816+0000 7f9f67600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9f6015fb30 con 0x7f9f600cd870 2024-04-17T10:02:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.816+0000 7f9f65600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9f600cd870 0x7f9f601661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.816+0000 7f9f5fe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9f6006f500 0x7f9f600cd160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46220/0 (socket says 172.21.15.32:46220) 2024-04-17T10:02:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.816+0000 7f9f5fe00640 1 -- 172.21.15.32:0/4229939185 learned_addr learned my addr 172.21.15.32:0/4229939185 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:02:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.816+0000 7f9f5f400640 1 -- 172.21.15.32:0/4229939185 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3816253931 0 0) 0x7f9f6015f200 con 0x7f9f6006f500 2024-04-17T10:02:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.816+0000 7f9f65600640 1 -- 172.21.15.32:0/4229939185 >> v1:172.21.15.144:6789/0 conn(0x7f9f600cfc70 legacy=0x7f9f6006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.816+0000 7f9f65600640 1 -- 172.21.15.32:0/4229939185 >> v1:172.21.15.32:6789/0 conn(0x7f9f6006f500 legacy=0x7f9f600cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.816+0000 7f9f65600640 1 -- 172.21.15.32:0/4229939185 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6015f7c0 con 0x7f9f600cd870 2024-04-17T10:02:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.824+0000 7f9f65600640 1 --2- 172.21.15.32:0/4229939185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f600cd870 0x7f9f601661a0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009d90 tx=0x7f9f5801c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1ec62cde5110d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.824+0000 7f9f5f400640 1 -- 172.21.15.32:0/4229939185 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f580b2070 con 0x7f9f600cd870 2024-04-17T10:02:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.824+0000 7f9f5f400640 1 -- 172.21.15.32:0/4229939185 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f580a5770 con 0x7f9f600cd870 2024-04-17T10:02:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.824+0000 7f9f5f400640 1 -- 172.21.15.32:0/4229939185 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f580a5b40 con 0x7f9f600cd870 2024-04-17T10:02:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.824+0000 7f9f67600640 1 -- 172.21.15.32:0/4229939185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f600cd870 msgr2=0x7f9f601661a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.824+0000 7f9f67600640 1 --2- 172.21.15.32:0/4229939185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f600cd870 0x7f9f601661a0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009d90 tx=0x7f9f5801c820 comp rx=0 tx=0).stop 2024-04-17T10:02:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.824+0000 7f9f67600640 1 -- 172.21.15.32:0/4229939185 shutdown_connections 2024-04-17T10:02:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.824+0000 7f9f67600640 1 --2- 172.21.15.32:0/4229939185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f600cd870 0x7f9f601661a0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.824+0000 7f9f67600640 1 -- 172.21.15.32:0/4229939185 >> 172.21.15.32:0/4229939185 conn(0x7f9f600b2190 msgr2=0x7f9f600b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:49.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.824+0000 7f9f67600640 1 -- 172.21.15.32:0/4229939185 shutdown_connections 2024-04-17T10:02:49.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.824+0000 7f9f67600640 1 -- 172.21.15.32:0/4229939185 wait complete. 2024-04-17T10:02:49.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.824+0000 7f9f67600640 1 Processor -- start 2024-04-17T10:02:49.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.824+0000 7f9f67600640 1 -- start start 2024-04-17T10:02:49.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.828+0000 7f9f67600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f6006f500 0x7f9f6015ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.828+0000 7f9f67600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f600cd870 0x7f9f60142bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.828+0000 7f9f67600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f600cfc70 0x7f9f60153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.828+0000 7f9f67600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9f60169110 con 0x7f9f6006f500 2024-04-17T10:02:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.828+0000 7f9f67600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9f600d0260 con 0x7f9f600cd870 2024-04-17T10:02:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.828+0000 7f9f67600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9f60169290 con 0x7f9f600cfc70 2024-04-17T10:02:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.828+0000 7f9f64c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f6006f500 0x7f9f6015ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.828+0000 7f9f64c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f6006f500 0x7f9f6015ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44372/0 (socket says 172.21.15.32:44372) 2024-04-17T10:02:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.828+0000 7f9f64c00640 1 -- 172.21.15.32:0/1051529948 learned_addr learned my addr 172.21.15.32:0/1051529948 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.828+0000 7f9f65600640 1 --2- 172.21.15.32:0/1051529948 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f600cfc70 0x7f9f60153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.828+0000 7f9f5fe00640 1 --2- 172.21.15.32:0/1051529948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f600cd870 0x7f9f60142bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.828+0000 7f9f64c00640 1 -- 172.21.15.32:0/1051529948 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f600cfc70 msgr2=0x7f9f60153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.828+0000 7f9f64c00640 1 --2- 172.21.15.32:0/1051529948 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f600cfc70 0x7f9f60153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.828+0000 7f9f64c00640 1 -- 172.21.15.32:0/1051529948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f600cd870 msgr2=0x7f9f60142bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.828+0000 7f9f64c00640 1 --2- 172.21.15.32:0/1051529948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f600cd870 0x7f9f60142bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.828+0000 7f9f64c00640 1 -- 172.21.15.32:0/1051529948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6015d870 con 0x7f9f6006f500 2024-04-17T10:02:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.828+0000 7f9f64c00640 1 --2- 172.21.15.32:0/1051529948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f6006f500 0x7f9f6015ab60 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400c750 tx=0x7f9f5400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.828+0000 7f9f5cc00640 1 -- 172.21.15.32:0/1051529948 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f54017070 con 0x7f9f6006f500 2024-04-17T10:02:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.828+0000 7f9f5cc00640 1 -- 172.21.15.32:0/1051529948 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f54005270 con 0x7f9f6006f500 2024-04-17T10:02:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.828+0000 7f9f5cc00640 1 -- 172.21.15.32:0/1051529948 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5400a640 con 0x7f9f6006f500 2024-04-17T10:02:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.828+0000 7f9f67600640 1 -- 172.21.15.32:0/1051529948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f6015a080 con 0x7f9f6006f500 2024-04-17T10:02:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.828+0000 7f9f67600640 1 -- 172.21.15.32:0/1051529948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f6015db50 con 0x7f9f6006f500 2024-04-17T10:02:49.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f5cc00640 1 -- 172.21.15.32:0/1051529948 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9f5401a070 con 0x7f9f6006f500 2024-04-17T10:02:49.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f5cc00640 1 --2- 172.21.15.32:0/1051529948 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9f40068380 0x7f9f4006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f5cc00640 1 -- 172.21.15.32:0/1051529948 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f9f54085fa0 con 0x7f9f6006f500 2024-04-17T10:02:49.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f5fe00640 1 --2- 172.21.15.32:0/1051529948 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9f40068380 0x7f9f4006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f6ab57000 1 Processor -- start 2024-04-17T10:02:49.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f6ab57000 1 -- start start 2024-04-17T10:02:49.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f5fe00640 1 --2- 172.21.15.32:0/1051529948 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9f40068380 0x7f9f4006a840 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9f60155620 tx=0x7f9f4c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:49.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f6ab57000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d06524700 0x564d06524ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f6ab57000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06525020 0x564d064d74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f6ab57000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d064d7a10 0x564d064d7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f6ab57000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564d063295f0 con 0x564d06525020 2024-04-17T10:02:49.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f6ab57000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564d062e27a0 con 0x564d06524700 2024-04-17T10:02:49.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f6ab57000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x564d063255b0 con 0x564d064d7a10 2024-04-17T10:02:49.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f5fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06525020 0x564d064d74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f5fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06525020 0x564d064d74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44378/0 (socket says 172.21.15.32:44378) 2024-04-17T10:02:49.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f5fe00640 1 -- 172.21.15.32:0/2312325857 learned_addr learned my addr 172.21.15.32:0/2312325857 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:49.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f64c00640 1 --2- 172.21.15.32:0/2312325857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d06524700 0x564d06524ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f65600640 1 --2- 172.21.15.32:0/2312325857 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d064d7a10 0x564d064d7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f5fe00640 1 -- 172.21.15.32:0/2312325857 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d064d7a10 msgr2=0x564d064d7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:49.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f5fe00640 1 --2- 172.21.15.32:0/2312325857 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d064d7a10 0x564d064d7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f5fe00640 1 -- 172.21.15.32:0/2312325857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d06524700 msgr2=0x564d06524ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:49.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f5fe00640 1 --2- 172.21.15.32:0/2312325857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d06524700 0x564d06524ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f5fe00640 1 -- 172.21.15.32:0/2312325857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d063d06e0 con 0x564d06525020 2024-04-17T10:02:49.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.832+0000 7f9f5fe00640 1 --2- 172.21.15.32:0/2312325857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06525020 0x564d064d74d0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c001e00 tx=0x7f9f4c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76af58dc092a0594 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:49.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f51600640 1 -- 172.21.15.32:0/2312325857 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0074a0 con 0x564d06525020 2024-04-17T10:02:49.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f51600640 1 -- 172.21.15.32:0/2312325857 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f4c00e7b0 con 0x564d06525020 2024-04-17T10:02:49.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f51600640 1 -- 172.21.15.32:0/2312325857 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c00eba0 con 0x564d06525020 2024-04-17T10:02:49.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f6ab57000 1 -- 172.21.15.32:0/2312325857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06525020 msgr2=0x564d064d74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f6ab57000 1 --2- 172.21.15.32:0/2312325857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06525020 0x564d064d74d0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c001e00 tx=0x7f9f4c011000 comp rx=0 tx=0).stop 2024-04-17T10:02:49.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f6ab57000 1 -- 172.21.15.32:0/2312325857 shutdown_connections 2024-04-17T10:02:49.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f6ab57000 1 --2- 172.21.15.32:0/2312325857 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d064d7a10 0x564d064d7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f6ab57000 1 --2- 172.21.15.32:0/2312325857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06525020 0x564d064d74d0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f6ab57000 1 --2- 172.21.15.32:0/2312325857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d06524700 0x564d06524ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f6ab57000 1 -- 172.21.15.32:0/2312325857 >> 172.21.15.32:0/2312325857 conn(0x564d06505290 msgr2=0x564d06507680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:49.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f6ab57000 1 -- 172.21.15.32:0/2312325857 shutdown_connections 2024-04-17T10:02:49.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f6ab57000 1 -- 172.21.15.32:0/2312325857 wait complete. 2024-04-17T10:02:49.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f6ab57000 1 -- start start 2024-04-17T10:02:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f6ab57000 1 Processor -- start 2024-04-17T10:02:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f6ab57000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d064d7a10 0x564d064ee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f6ab57000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06524700 0x564d064eee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f6ab57000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d06525020 0x564d064ef370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f6ab57000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564d06334520 con 0x564d06524700 2024-04-17T10:02:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f6ab57000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564d062c1ea0 con 0x564d06525020 2024-04-17T10:02:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f6ab57000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x564d062e8500 con 0x564d064d7a10 2024-04-17T10:02:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f6ab57000 10 client.0 _ll_register_callbacks cb 0x564d064c5170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:02:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f5fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06524700 0x564d064eee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f5fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06524700 0x564d064eee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44382/0 (socket says 172.21.15.32:44382) 2024-04-17T10:02:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f5fe00640 1 -- 172.21.15.32:0/2000542870 learned_addr learned my addr 172.21.15.32:0/2000542870 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f64c00640 1 --2- 172.21.15.32:0/2000542870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d064d7a10 0x564d064ee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f65600640 1 --2- 172.21.15.32:0/2000542870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d06525020 0x564d064ef370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f5fe00640 1 -- 172.21.15.32:0/2000542870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d064d7a10 msgr2=0x564d064ee8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f5fe00640 1 --2- 172.21.15.32:0/2000542870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d064d7a10 0x564d064ee8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f5fe00640 1 -- 172.21.15.32:0/2000542870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d06525020 msgr2=0x564d064ef370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f5fe00640 1 --2- 172.21.15.32:0/2000542870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d06525020 0x564d064ef370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f5fe00640 1 -- 172.21.15.32:0/2000542870 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d06302540 con 0x564d06524700 2024-04-17T10:02:49.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f5fe00640 1 --2- 172.21.15.32:0/2000542870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06524700 0x564d064eee30 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c007fa0 tx=0x7f9f4c007850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab4adec6d3d1891 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:49.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.836+0000 7f9f46000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:02:49.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f46000640 1 -- 172.21.15.32:0/2000542870 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c00eb70 con 0x564d06524700 2024-04-17T10:02:49.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f46000640 1 -- 172.21.15.32:0/2000542870 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f9f4c010050 con 0x564d06524700 2024-04-17T10:02:49.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f6ab57000 1 -- 172.21.15.32:0/2000542870 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f4c00e7b0 con 0x564d06524700 2024-04-17T10:02:49.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f46000640 1 -- 172.21.15.32:0/2000542870 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f9f4c032050 con 0x564d06524700 2024-04-17T10:02:49.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f6ab57000 10 client.9057 fetch_fsmap learned FSMap version 87 2024-04-17T10:02:49.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f6ab57000 1 -- 172.21.15.32:0/2000542870 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x564d0636dba0 con 0x564d06524700 2024-04-17T10:02:49.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f46000640 1 -- 172.21.15.32:0/2000542870 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f4c00e7b0 con 0x564d06524700 2024-04-17T10:02:49.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f46000640 1 -- 172.21.15.32:0/2000542870 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 87) ==== 3495+0+0 (secure 0 0 0) 0x7f9f4c035020 con 0x564d06524700 2024-04-17T10:02:49.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f6ab57000 10 client.9057 fetch_fsmap finished waiting for FSMap version 87 2024-04-17T10:02:49.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f6ab57000 10 client.9057 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-17T10:02:49.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f6ab57000 20 client.9057 populate_metadata read hostname 'smithi032' 2024-04-17T10:02:49.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f6ab57000 1 --2- 172.21.15.32:0/2000542870 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x564d064efd80 0x564d065359b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f6ab57000 4 client.9057 mds_command: new command op to 8964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:02:49.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f6ab57000 1 -- 172.21.15.32:0/2000542870 --> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d063e2450 con 0x564d064efd80 2024-04-17T10:02:49.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f64c00640 1 --2- 172.21.15.32:0/2000542870 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x564d064efd80 0x564d065359b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f64c00640 1 --2- 172.21.15.32:0/2000542870 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x564d064efd80 0x564d065359b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d7d0d1fc3def437 server_cookie=cc33c63385670756 in_seq=0 out_seq=0 2024-04-17T10:02:49.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f46000640 10 client.9057 ms_handle_connect on v2:172.21.15.32:6835/3078068380 2024-04-17T10:02:49.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f46000640 1 -- 172.21.15.32:0/2000542870 <== mds.0 v2:172.21.15.32:6835/3078068380 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d063e2450 con 0x564d064efd80 2024-04-17T10:02:49.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f46000640 10 client.9057 handle_command_reply: tid=0 2024-04-17T10:02:49.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f6ab57000 1 client.9057 shutdown 2024-04-17T10:02:49.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f6ab57000 1 -- 172.21.15.32:0/2000542870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06524700 msgr2=0x564d064eee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f6ab57000 1 --2- 172.21.15.32:0/2000542870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06524700 0x564d064eee30 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c007fa0 tx=0x7f9f4c007850 comp rx=0 tx=0).stop 2024-04-17T10:02:49.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f6ab57000 1 -- 172.21.15.32:0/2000542870 shutdown_connections 2024-04-17T10:02:49.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f6ab57000 1 --2- 172.21.15.32:0/2000542870 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x564d064efd80 0x564d065359b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f6ab57000 1 --2- 172.21.15.32:0/2000542870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d06525020 0x564d064ef370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f6ab57000 1 --2- 172.21.15.32:0/2000542870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06524700 0x564d064eee30 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f6ab57000 1 --2- 172.21.15.32:0/2000542870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d064d7a10 0x564d064ee8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f6ab57000 1 -- 172.21.15.32:0/2000542870 >> 172.21.15.32:0/2000542870 conn(0x564d06505290 msgr2=0x564d06507680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:49.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f46000640 0 client.9057 ms_handle_reset on v2:172.21.15.32:6835/3078068380 2024-04-17T10:02:49.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f6ab57000 1 -- 172.21.15.32:0/2000542870 shutdown_connections 2024-04-17T10:02:49.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f6ab57000 1 -- 172.21.15.32:0/2000542870 wait complete. 2024-04-17T10:02:49.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f9f6ab57000 20 client.9057 trim_cache size 0 max 16384 2024-04-17T10:02:49.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f7a07400640 1 Processor -- start 2024-04-17T10:02:49.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.840+0000 7f7a07400640 1 -- start start 2024-04-17T10:02:49.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.844+0000 7f7a07400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a000b25d0 0x7f7a00162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.844+0000 7f7a07400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a000b29b0 0x7f7a00162fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.844+0000 7f7a07400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7a001634e0 0x7f7a000cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.844+0000 7f7a07400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7a0006ba30 con 0x7f7a000b25d0 2024-04-17T10:02:49.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.844+0000 7f7a07400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f7a0006bcd0 con 0x7f7a001634e0 2024-04-17T10:02:49.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.844+0000 7f7a07400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7a0006bfb0 con 0x7f7a000b29b0 2024-04-17T10:02:49.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.844+0000 7f7a04a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a000b25d0 0x7f7a00162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.844+0000 7f7a04a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a000b25d0 0x7f7a00162a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44390/0 (socket says 172.21.15.32:44390) 2024-04-17T10:02:49.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.844+0000 7f7a04a00640 1 -- 172.21.15.32:0/2696078553 learned_addr learned my addr 172.21.15.32:0/2696078553 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:49.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.844+0000 7f79ffe00640 1 --2- 172.21.15.32:0/2696078553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a000b29b0 0x7f7a00162fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.844+0000 7f7a05400640 1 --2- 172.21.15.32:0/2696078553 >> v2:172.21.15.144:3300/0 conn(0x7f7a001634e0 0x7f7a000cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.844+0000 7f7a04a00640 1 -- 172.21.15.32:0/2696078553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a000b29b0 msgr2=0x7f7a00162fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:49.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.844+0000 7f7a04a00640 1 --2- 172.21.15.32:0/2696078553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a000b29b0 0x7f7a00162fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.844+0000 7f7a04a00640 1 -- 172.21.15.32:0/2696078553 >> v2:172.21.15.144:3300/0 conn(0x7f7a001634e0 msgr2=0x7f7a000cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:49.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.844+0000 7f7a04a00640 1 --2- 172.21.15.32:0/2696078553 >> v2:172.21.15.144:3300/0 conn(0x7f7a001634e0 0x7f7a000cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.844+0000 7f7a04a00640 1 -- 172.21.15.32:0/2696078553 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a0006c290 con 0x7f7a000b25d0 2024-04-17T10:02:49.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.848+0000 7f7a04a00640 1 --2- 172.21.15.32:0/2696078553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a000b25d0 0x7f7a00162a60 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f7a00072ce0 tx=0x7f79f401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e5781e7ecda5b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:49.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.848+0000 7f79ff400640 1 -- 172.21.15.32:0/2696078553 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f40a5800 con 0x7f7a000b25d0 2024-04-17T10:02:49.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.848+0000 7f79ff400640 1 -- 172.21.15.32:0/2696078553 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79f40a59a0 con 0x7f7a000b25d0 2024-04-17T10:02:49.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.848+0000 7f79ff400640 1 -- 172.21.15.32:0/2696078553 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f40a5d70 con 0x7f7a000b25d0 2024-04-17T10:02:49.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.848+0000 7f7a07400640 1 -- 172.21.15.32:0/2696078553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a000b25d0 msgr2=0x7f7a00162a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.848+0000 7f7a07400640 1 --2- 172.21.15.32:0/2696078553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a000b25d0 0x7f7a00162a60 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f7a00072ce0 tx=0x7f79f401c960 comp rx=0 tx=0).stop 2024-04-17T10:02:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.848+0000 7f7a07400640 1 -- 172.21.15.32:0/2696078553 shutdown_connections 2024-04-17T10:02:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.848+0000 7f7a07400640 1 --2- 172.21.15.32:0/2696078553 >> v2:172.21.15.144:3300/0 conn(0x7f7a001634e0 0x7f7a000cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.848+0000 7f7a07400640 1 --2- 172.21.15.32:0/2696078553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a000b29b0 0x7f7a00162fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.848+0000 7f7a07400640 1 --2- 172.21.15.32:0/2696078553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a000b25d0 0x7f7a00162a60 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.848+0000 7f7a07400640 1 -- 172.21.15.32:0/2696078553 >> 172.21.15.32:0/2696078553 conn(0x7f7a000d1400 msgr2=0x7f7a000d1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.848+0000 7f7a07400640 1 -- 172.21.15.32:0/2696078553 shutdown_connections 2024-04-17T10:02:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.848+0000 7f7a07400640 1 -- 172.21.15.32:0/2696078553 wait complete. 2024-04-17T10:02:49.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.848+0000 7f7a07400640 1 Processor -- start 2024-04-17T10:02:49.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.848+0000 7f7a07400640 1 -- start start 2024-04-17T10:02:49.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.852+0000 7f7a07400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a000b25d0 0x7f7a000b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.852+0000 7f7a07400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a000b29b0 0x7f7a000cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.852+0000 7f7a07400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a001634e0 0x7f7a000b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.852+0000 7f7a07400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7a000ccd00 con 0x7f7a000b29b0 2024-04-17T10:02:49.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.852+0000 7f7a07400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7a000ccb80 con 0x7f7a000b25d0 2024-04-17T10:02:49.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.852+0000 7f7a07400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7a000cce80 con 0x7f7a001634e0 2024-04-17T10:02:49.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.852+0000 7f79ffe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a000b29b0 0x7f7a000cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.852+0000 7f79ffe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a000b29b0 0x7f7a000cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44394/0 (socket says 172.21.15.32:44394) 2024-04-17T10:02:49.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.852+0000 7f79ffe00640 1 -- 172.21.15.32:0/791571353 learned_addr learned my addr 172.21.15.32:0/791571353 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:49.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.852+0000 7f7a04a00640 1 --2- 172.21.15.32:0/791571353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a000b25d0 0x7f7a000b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.852+0000 7f7a05400640 1 --2- 172.21.15.32:0/791571353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a001634e0 0x7f7a000b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.852+0000 7f79ffe00640 1 -- 172.21.15.32:0/791571353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a001634e0 msgr2=0x7f7a000b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.852+0000 7f79ffe00640 1 --2- 172.21.15.32:0/791571353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a001634e0 0x7f7a000b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.852+0000 7f79ffe00640 1 -- 172.21.15.32:0/791571353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a000b25d0 msgr2=0x7f7a000b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.852+0000 7f79ffe00640 1 --2- 172.21.15.32:0/791571353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a000b25d0 0x7f7a000b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.852+0000 7f79ffe00640 1 -- 172.21.15.32:0/791571353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a000cb2f0 con 0x7f7a000b29b0 2024-04-17T10:02:49.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.852+0000 7f79ffe00640 1 --2- 172.21.15.32:0/791571353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a000b29b0 0x7f7a000cadb0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00e930 tx=0x7f79ec00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:49.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.852+0000 7f79fcc00640 1 -- 172.21.15.32:0/791571353 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec017070 con 0x7f7a000b29b0 2024-04-17T10:02:49.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.852+0000 7f79fcc00640 1 -- 172.21.15.32:0/791571353 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79ec00c940 con 0x7f7a000b29b0 2024-04-17T10:02:49.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.852+0000 7f79fcc00640 1 -- 172.21.15.32:0/791571353 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec00ccf0 con 0x7f7a000b29b0 2024-04-17T10:02:49.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.852+0000 7f7a07400640 1 -- 172.21.15.32:0/791571353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a000c9210 con 0x7f7a000b29b0 2024-04-17T10:02:49.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.852+0000 7f7a07400640 1 -- 172.21.15.32:0/791571353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a0015b1c0 con 0x7f7a000b29b0 2024-04-17T10:02:49.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.852+0000 7f7a07400640 1 -- 172.21.15.32:0/791571353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f7a0015b8d0 con 0x7f7a000b29b0 2024-04-17T10:02:49.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.856+0000 7f79fcc00640 1 -- 172.21.15.32:0/791571353 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f79ec01a070 con 0x7f7a000b29b0 2024-04-17T10:02:49.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.856+0000 7f79fcc00640 1 --2- 172.21.15.32:0/791571353 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f79e0068380 0x7f79e006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.856+0000 7f79fcc00640 1 -- 172.21.15.32:0/791571353 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f79ec0868c0 con 0x7f7a000b29b0 2024-04-17T10:02:49.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.856+0000 7f7a04a00640 1 --2- 172.21.15.32:0/791571353 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f79e0068380 0x7f79e006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.856+0000 7f79fcc00640 1 -- 172.21.15.32:0/791571353 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+512 (secure 0 0 0) 0x7f7a000cb2f0 con 0x7f7a000b29b0 2024-04-17T10:02:49.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.856+0000 7f7a07400640 1 -- 172.21.15.32:0/791571353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f7a00068a90 con 0x7f7a000b29b0 2024-04-17T10:02:49.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.856+0000 7f79fcc00640 1 -- 172.21.15.32:0/791571353 <== mon.0 v2:172.21.15.32:3300/0 7 ==== log(2 entries from seq 1186 at 2024-04-17T10:02:48.058465+0000) ==== 389+0+0 (secure 0 0 0) 0x7f79ec08d070 con 0x7f7a000b29b0 2024-04-17T10:02:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: cluster: 2024-04-17T10:02:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: id: ff178df8-4c06-4367-a1e9-2e3a66d870d0 2024-04-17T10:02:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: health: HEALTH_OK 2024-04-17T10:02:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:02:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: services: 2024-04-17T10:02:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: mon: 3 daemons, quorum a,b,c (age 10m) 2024-04-17T10:02:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: mgr: y(active, since 9m), standbys: x 2024-04-17T10:02:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: mds: 1/1 daemons up, 3 standby 2024-04-17T10:02:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 9m) 2024-04-17T10:02:49.854 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:02:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: data: 2024-04-17T10:02:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: volumes: 1/1 healthy 2024-04-17T10:02:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: pools: 3 pools, 97 pgs 2024-04-17T10:02:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: objects: 30 objects, 19 MiB 2024-04-17T10:02:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: usage: 294 MiB used, 720 GiB / 720 GiB avail 2024-04-17T10:02:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: pgs: 97 active+clean 2024-04-17T10:02:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:02:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: io: 2024-04-17T10:02:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: client: 1023 B/s rd, 102 B/s wr, 0 op/s rd, 0 op/s wr 2024-04-17T10:02:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:02:49.855 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:02:49.855 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T10:02:48.058465+0000 mon.a [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-17T10:02:49.855 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T10:02:48.058490+0000 mon.a [INF] Cluster is now healthy 2024-04-17T10:02:49.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.856+0000 7f7a04a00640 1 --2- 172.21.15.32:0/791571353 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f79e0068380 0x7f79e006a840 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f79f40ae000 tx=0x7f79f40a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:49.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.860+0000 7f9f6ab57000 1 Processor -- start 2024-04-17T10:02:49.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.860+0000 7f9f6ab57000 1 -- start start 2024-04-17T10:02:49.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.860+0000 7f9f6ab57000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d064d7a10 0x564d064d6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.860+0000 7f9f6ab57000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d06524700 0x564d065359b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.860+0000 7f9f6ab57000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06525020 0x564d06535ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.860+0000 7f9f6ab57000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6015ec00 con 0x564d06525020 2024-04-17T10:02:49.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.860+0000 7f9f6ab57000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6015df60 con 0x564d064d7a10 2024-04-17T10:02:49.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.860+0000 7f9f6ab57000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9f6015e5b0 con 0x564d06524700 2024-04-17T10:02:49.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.860+0000 7f9f65600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06525020 0x564d06535ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.860+0000 7f9f65600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06525020 0x564d06535ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44398/0 (socket says 172.21.15.32:44398) 2024-04-17T10:02:49.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.860+0000 7f9f65600640 1 -- 172.21.15.32:0/3258262362 learned_addr learned my addr 172.21.15.32:0/3258262362 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:49.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.860+0000 7f9f64c00640 1 --2- 172.21.15.32:0/3258262362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d064d7a10 0x564d064d6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.860+0000 7f9f5fe00640 1 --2- 172.21.15.32:0/3258262362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d06524700 0x564d065359b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f65600640 1 -- 172.21.15.32:0/3258262362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d06524700 msgr2=0x564d065359b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:49.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f65600640 1 --2- 172.21.15.32:0/3258262362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d06524700 0x564d065359b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f65600640 1 -- 172.21.15.32:0/3258262362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d064d7a10 msgr2=0x564d064d6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:49.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f65600640 1 --2- 172.21.15.32:0/3258262362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d064d7a10 0x564d064d6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f65600640 1 -- 172.21.15.32:0/3258262362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d064df500 con 0x564d06525020 2024-04-17T10:02:49.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f65600640 1 --2- 172.21.15.32:0/3258262362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06525020 0x564d06535ef0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f9f58019430 tx=0x7f9f58004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1370c73dd48ea784 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:49.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f50c00640 1 -- 172.21.15.32:0/3258262362 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f580b2040 con 0x564d06525020 2024-04-17T10:02:49.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f50c00640 1 -- 172.21.15.32:0/3258262362 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f580a5a90 con 0x564d06525020 2024-04-17T10:02:49.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f50c00640 1 -- 172.21.15.32:0/3258262362 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f580a5e40 con 0x564d06525020 2024-04-17T10:02:49.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f6ab57000 1 -- 172.21.15.32:0/3258262362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06525020 msgr2=0x564d06535ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f6ab57000 1 --2- 172.21.15.32:0/3258262362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06525020 0x564d06535ef0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f9f58019430 tx=0x7f9f58004ac0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f6ab57000 1 -- 172.21.15.32:0/3258262362 shutdown_connections 2024-04-17T10:02:49.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f6ab57000 1 --2- 172.21.15.32:0/3258262362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d06524700 0x564d065359b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f6ab57000 1 --2- 172.21.15.32:0/3258262362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06525020 0x564d06535ef0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f6ab57000 1 --2- 172.21.15.32:0/3258262362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d064d7a10 0x564d064d6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f6ab57000 1 -- 172.21.15.32:0/3258262362 >> 172.21.15.32:0/3258262362 conn(0x564d06505290 msgr2=0x564d064d5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:49.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f6ab57000 1 -- 172.21.15.32:0/3258262362 shutdown_connections 2024-04-17T10:02:49.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f6ab57000 1 -- 172.21.15.32:0/3258262362 wait complete. 2024-04-17T10:02:49.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f6ab57000 1 -- start start 2024-04-17T10:02:49.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f6ab57000 1 Processor -- start 2024-04-17T10:02:49.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f6ab57000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d064d7a10 0x564d065942e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f6ab57000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06524700 0x564d06594820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f6ab57000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d06525020 0x564d0654f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f6ab57000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x564d0632dc20 con 0x564d06524700 2024-04-17T10:02:49.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f6ab57000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564d06328900 con 0x564d064d7a10 2024-04-17T10:02:49.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f6ab57000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x564d06336800 con 0x564d06525020 2024-04-17T10:02:49.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f6ab57000 10 client.0 _ll_register_callbacks cb 0x7f9f54055b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:02:49.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f5fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06524700 0x564d06594820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f64c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d064d7a10 0x564d065942e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f5fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06524700 0x564d06594820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44400/0 (socket says 172.21.15.32:44400) 2024-04-17T10:02:49.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f5fe00640 1 -- 172.21.15.32:0/3747949173 learned_addr learned my addr 172.21.15.32:0/3747949173 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:49.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.864+0000 7f9f65600640 1 --2- 172.21.15.32:0/3747949173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d06525020 0x564d0654f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f5fe00640 1 -- 172.21.15.32:0/3747949173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d06525020 msgr2=0x564d0654f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:49.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f5fe00640 1 --2- 172.21.15.32:0/3747949173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d06525020 0x564d0654f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f5fe00640 1 -- 172.21.15.32:0/3747949173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d064d7a10 msgr2=0x564d065942e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:49.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f5fe00640 1 --2- 172.21.15.32:0/3747949173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d064d7a10 0x564d065942e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f5fe00640 1 -- 172.21.15.32:0/3747949173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d06335f70 con 0x564d06524700 2024-04-17T10:02:49.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f64c00640 1 --2- 172.21.15.32:0/3747949173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d064d7a10 0x564d065942e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:02:49.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f65600640 1 --2- 172.21.15.32:0/3747949173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d06525020 0x564d0654f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:02:49.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f5fe00640 1 --2- 172.21.15.32:0/3747949173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06524700 0x564d06594820 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c007fa0 tx=0x7f9f4c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55332ffdd283320 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:49.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f46000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:02:49.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f46000640 1 -- 172.21.15.32:0/3747949173 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c00eb70 con 0x564d06524700 2024-04-17T10:02:49.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f46000640 1 -- 172.21.15.32:0/3747949173 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f9f4c028040 con 0x564d06524700 2024-04-17T10:02:49.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f6ab57000 1 -- 172.21.15.32:0/3747949173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f580a5a90 con 0x564d06524700 2024-04-17T10:02:49.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f46000640 1 -- 172.21.15.32:0/3747949173 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f9f4c040050 con 0x564d06524700 2024-04-17T10:02:49.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f6ab57000 10 client.9069 fetch_fsmap learned FSMap version 87 2024-04-17T10:02:49.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f6ab57000 1 -- 172.21.15.32:0/3747949173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x564d062e29f0 con 0x564d06524700 2024-04-17T10:02:49.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f46000640 1 -- 172.21.15.32:0/3747949173 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f580a5a90 con 0x564d06524700 2024-04-17T10:02:49.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f46000640 1 -- 172.21.15.32:0/3747949173 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 87) ==== 3495+0+0 (secure 0 0 0) 0x7f9f4c043020 con 0x564d06524700 2024-04-17T10:02:49.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f6ab57000 10 client.9069 fetch_fsmap finished waiting for FSMap version 87 2024-04-17T10:02:49.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f6ab57000 10 client.9069 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-17T10:02:49.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f6ab57000 20 client.9069 populate_metadata read hostname 'smithi032' 2024-04-17T10:02:49.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f6ab57000 1 --2- 172.21.15.32:0/3747949173 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x564d0654ff60 0x564d06550370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:49.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f6ab57000 4 client.9069 mds_command: new command op to 8964 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-17T10:02:49.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f6ab57000 1 -- 172.21.15.32:0/3747949173 --> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x564d062c2ed0 con 0x564d0654ff60 2024-04-17T10:02:49.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f64c00640 1 --2- 172.21.15.32:0/3747949173 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x564d0654ff60 0x564d06550370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:49.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f46000640 10 client.9069 ms_handle_connect on v2:172.21.15.32:6835/3078068380 2024-04-17T10:02:49.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f64c00640 1 --2- 172.21.15.32:0/3747949173 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x564d0654ff60 0x564d06550370 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7121b41472e647b7 server_cookie=5dc0565285c6921c in_seq=0 out_seq=0 2024-04-17T10:02:49.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f46000640 1 -- 172.21.15.32:0/3747949173 <== mds.0 v2:172.21.15.32:6835/3078068380 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564d062c2ed0 con 0x564d0654ff60 2024-04-17T10:02:49.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f46000640 10 client.9069 handle_command_reply: tid=0 2024-04-17T10:02:49.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f6ab57000 1 client.9069 shutdown 2024-04-17T10:02:49.869 INFO:tasks.ceph.mds.c.smithi032.stderr:2024-04-17T10:02:49.868+0000 7fd583400640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-17T10:02:49.869 INFO:tasks.ceph.mds.c.smithi032.stderr:2024-04-17T10:02:49.868+0000 7fd583400640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-17T10:02:49.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f6ab57000 1 -- 172.21.15.32:0/3747949173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06524700 msgr2=0x564d06594820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f6ab57000 1 --2- 172.21.15.32:0/3747949173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06524700 0x564d06594820 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c007fa0 tx=0x7f9f4c01d000 comp rx=0 tx=0).stop 2024-04-17T10:02:49.870 INFO:tasks.ceph.mds.c.smithi032.stderr:2024-04-17T10:02:49.868+0000 7fd583400640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-17T10:02:49.870 INFO:tasks.ceph.mds.c.smithi032.stderr:2024-04-17T10:02:49.868+0000 7fd583400640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-17T10:02:49.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f6ab57000 1 -- 172.21.15.32:0/3747949173 shutdown_connections 2024-04-17T10:02:49.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f6ab57000 1 --2- 172.21.15.32:0/3747949173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564d06525020 0x564d0654f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f6ab57000 1 --2- 172.21.15.32:0/3747949173 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x564d0654ff60 0x564d06550370 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f6ab57000 1 --2- 172.21.15.32:0/3747949173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564d06524700 0x564d06594820 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f6ab57000 1 --2- 172.21.15.32:0/3747949173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d064d7a10 0x564d065942e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f6ab57000 1 -- 172.21.15.32:0/3747949173 >> 172.21.15.32:0/3747949173 conn(0x564d06505290 msgr2=0x564d064eef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:49.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.868+0000 7f9f46000640 0 client.9069 ms_handle_reset on v2:172.21.15.32:6835/3078068380 2024-04-17T10:02:49.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.872+0000 7f9f6ab57000 1 -- 172.21.15.32:0/3747949173 shutdown_connections 2024-04-17T10:02:49.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.872+0000 7f9f6ab57000 1 -- 172.21.15.32:0/3747949173 wait complete. 2024-04-17T10:02:49.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.872+0000 7f9f6ab57000 20 client.9069 trim_cache size 0 max 16384 2024-04-17T10:02:49.872 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:02:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2024-04-17T10:02:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "6309f98b-ba71-41b3-a13d-82d033c07f04", 2024-04-17T10:02:49.872 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2024-04-17T10:02:49.872 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:02:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.872+0000 7f9f67600640 1 -- 172.21.15.32:0/1051529948 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9f40068380 msgr2=0x7f9f4006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.872+0000 7f9f67600640 1 --2- 172.21.15.32:0/1051529948 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9f40068380 0x7f9f4006a840 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9f60155620 tx=0x7f9f4c009040 comp rx=0 tx=0).stop 2024-04-17T10:02:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.872+0000 7f9f67600640 1 -- 172.21.15.32:0/1051529948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f6006f500 msgr2=0x7f9f6015ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.872+0000 7f9f67600640 1 --2- 172.21.15.32:0/1051529948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f6006f500 0x7f9f6015ab60 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400c750 tx=0x7f9f5400cc10 comp rx=0 tx=0).stop 2024-04-17T10:02:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.872+0000 7f9f67600640 1 -- 172.21.15.32:0/1051529948 shutdown_connections 2024-04-17T10:02:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.872+0000 7f9f67600640 1 --2- 172.21.15.32:0/1051529948 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9f40068380 0x7f9f4006a840 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.872+0000 7f9f67600640 1 --2- 172.21.15.32:0/1051529948 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f600cfc70 0x7f9f60153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.872+0000 7f9f67600640 1 --2- 172.21.15.32:0/1051529948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f600cd870 0x7f9f60142bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.872+0000 7f9f67600640 1 --2- 172.21.15.32:0/1051529948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f6006f500 0x7f9f6015ab60 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.872+0000 7f9f67600640 1 -- 172.21.15.32:0/1051529948 >> 172.21.15.32:0/1051529948 conn(0x7f9f600b2190 msgr2=0x7f9f600b4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.872+0000 7f9f67600640 1 -- 172.21.15.32:0/1051529948 shutdown_connections 2024-04-17T10:02:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:49.872+0000 7f9f67600640 1 -- 172.21.15.32:0/1051529948 wait complete. 2024-04-17T10:02:49.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:02:49.888 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-17T10:02:50.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.032+0000 7fd98f800640 1 Processor -- start 2024-04-17T10:02:50.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.032+0000 7fd98f800640 1 -- start start 2024-04-17T10:02:50.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.032+0000 7fd98f800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd98806ba30 0x7fd98806be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:50.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.036+0000 7fd98f800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd98806c350 con 0x7fd988071e60 2024-04-17T10:02:50.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.036+0000 7fd98f800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9880d0130 con 0x7fd9880cf370 2024-04-17T10:02:50.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.036+0000 7fd98f800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd9880cfef0 con 0x7fd98806ba30 2024-04-17T10:02:50.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.036+0000 7fd98d800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd988071e60 0x7fd988072240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46222/0 (socket says 172.21.15.32:46222) 2024-04-17T10:02:50.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.036+0000 7fd98d800640 1 -- 172.21.15.32:0/2046746586 learned_addr learned my addr 172.21.15.32:0/2046746586 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:02:50.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.036+0000 7fd98ce00640 1 --2- 172.21.15.32:0/2046746586 >> v2:172.21.15.32:3300/0 conn(0x7fd98806ba30 0x7fd98806be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:50.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.036+0000 7fd98ce00640 1 -- 172.21.15.32:0/2046746586 >> v1:172.21.15.144:6789/0 conn(0x7fd9880cf370 legacy=0x7fd9880cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:50.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.036+0000 7fd98ce00640 1 -- 172.21.15.32:0/2046746586 >> v1:172.21.15.32:6789/0 conn(0x7fd988071e60 legacy=0x7fd988072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:50.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.036+0000 7fd98ce00640 1 -- 172.21.15.32:0/2046746586 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd98815d090 con 0x7fd98806ba30 2024-04-17T10:02:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.036+0000 7fd98ce00640 1 --2- 172.21.15.32:0/2046746586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98806ba30 0x7fd98806be10 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0097c0 tx=0x7fd97c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f7e1623dff01b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.036+0000 7fd987400640 1 -- 172.21.15.32:0/2046746586 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0b1070 con 0x7fd98806ba30 2024-04-17T10:02:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.036+0000 7fd987400640 1 -- 172.21.15.32:0/2046746586 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd97c0a8050 con 0x7fd98806ba30 2024-04-17T10:02:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.036+0000 7fd987400640 1 -- 172.21.15.32:0/2046746586 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0ac860 con 0x7fd98806ba30 2024-04-17T10:02:50.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd98f800640 1 -- 172.21.15.32:0/2046746586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98806ba30 msgr2=0x7fd98806be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:50.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd98f800640 1 --2- 172.21.15.32:0/2046746586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98806ba30 0x7fd98806be10 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0097c0 tx=0x7fd97c01c3e0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd98f800640 1 -- 172.21.15.32:0/2046746586 shutdown_connections 2024-04-17T10:02:50.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd98f800640 1 --2- 172.21.15.32:0/2046746586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98806ba30 0x7fd98806be10 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd98f800640 1 -- 172.21.15.32:0/2046746586 >> 172.21.15.32:0/2046746586 conn(0x7fd9880b2120 msgr2=0x7fd9880b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:50.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd98f800640 1 -- 172.21.15.32:0/2046746586 shutdown_connections 2024-04-17T10:02:50.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd98f800640 1 -- 172.21.15.32:0/2046746586 wait complete. 2024-04-17T10:02:50.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd98f800640 1 Processor -- start 2024-04-17T10:02:50.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd98f800640 1 -- start start 2024-04-17T10:02:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd98f800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98806ba30 0x7fd98813d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd98f800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd988071e60 0x7fd9880b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd98f800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9880cf370 0x7fd988159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd98f800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd98815db70 con 0x7fd98806ba30 2024-04-17T10:02:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd98f800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd98815cd00 con 0x7fd988071e60 2024-04-17T10:02:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd98f800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd98815dcf0 con 0x7fd9880cf370 2024-04-17T10:02:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd98ce00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98806ba30 0x7fd98813d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd98ce00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98806ba30 0x7fd98813d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44412/0 (socket says 172.21.15.32:44412) 2024-04-17T10:02:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd98ce00640 1 -- 172.21.15.32:0/642738252 learned_addr learned my addr 172.21.15.32:0/642738252 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd987e00640 1 --2- 172.21.15.32:0/642738252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd988071e60 0x7fd9880b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd98ce00640 1 -- 172.21.15.32:0/642738252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9880cf370 msgr2=0x7fd988159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd98ce00640 1 --2- 172.21.15.32:0/642738252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9880cf370 0x7fd988159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd98ce00640 1 -- 172.21.15.32:0/642738252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd988071e60 msgr2=0x7fd9880b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd98ce00640 1 --2- 172.21.15.32:0/642738252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd988071e60 0x7fd9880b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.040+0000 7fd98ce00640 1 -- 172.21.15.32:0/642738252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd988159fd0 con 0x7fd98806ba30 2024-04-17T10:02:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.044+0000 7fd98ce00640 1 --2- 172.21.15.32:0/642738252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98806ba30 0x7fd98813d940 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0af040 tx=0x7fd97c019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.044+0000 7fd984c00640 1 -- 172.21.15.32:0/642738252 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0b9020 con 0x7fd98806ba30 2024-04-17T10:02:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.044+0000 7fd984c00640 1 -- 172.21.15.32:0/642738252 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd97c0b5410 con 0x7fd98806ba30 2024-04-17T10:02:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.044+0000 7fd984c00640 1 -- 172.21.15.32:0/642738252 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0b1040 con 0x7fd98806ba30 2024-04-17T10:02:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.044+0000 7fd98f800640 1 -- 172.21.15.32:0/642738252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd98815a2b0 con 0x7fd98806ba30 2024-04-17T10:02:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.044+0000 7fd98f800640 1 -- 172.21.15.32:0/642738252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd98813de80 con 0x7fd98806ba30 2024-04-17T10:02:50.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.044+0000 7fd984c00640 1 -- 172.21.15.32:0/642738252 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd97c0bc020 con 0x7fd98806ba30 2024-04-17T10:02:50.043 INFO:tasks.ceph.mds.c.smithi032.stderr:2024-04-17T10:02:50.044+0000 7fd588400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:02:50.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd984c00640 1 --2- 172.21.15.32:0/642738252 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd9680683b0 0x7fd96806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:50.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd992ced000 1 Processor -- start 2024-04-17T10:02:50.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd984c00640 1 -- 172.21.15.32:0/642738252 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7fd97c1207d0 con 0x7fd98806ba30 2024-04-17T10:02:50.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd987e00640 1 --2- 172.21.15.32:0/642738252 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd9680683b0 0x7fd96806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:50.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd992ced000 1 -- start start 2024-04-17T10:02:50.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd987e00640 1 --2- 172.21.15.32:0/642738252 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd9680683b0 0x7fd96806a870 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd9881fb380 tx=0x7fd974009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:50.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd992ced000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013260 0x55a67e013640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:50.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd992ced000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67e013b80 0x55a67dfc6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:50.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd992ced000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67dfc69a0 0x55a67dfc6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:50.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd992ced000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a67de184f0 con 0x55a67e013260 2024-04-17T10:02:50.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd992ced000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a67ddd16c0 con 0x55a67e013b80 2024-04-17T10:02:50.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd992ced000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55a67de144b0 con 0x55a67dfc69a0 2024-04-17T10:02:50.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd98ce00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013260 0x55a67e013640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:50.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd98ce00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013260 0x55a67e013640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44420/0 (socket says 172.21.15.32:44420) 2024-04-17T10:02:50.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd98ce00640 1 -- 172.21.15.32:0/1317275104 learned_addr learned my addr 172.21.15.32:0/1317275104 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:50.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd987e00640 1 --2- 172.21.15.32:0/1317275104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67e013b80 0x55a67dfc6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:50.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd98d800640 1 --2- 172.21.15.32:0/1317275104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67dfc69a0 0x55a67dfc6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:50.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd98ce00640 1 -- 172.21.15.32:0/1317275104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67dfc69a0 msgr2=0x55a67dfc6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:50.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd98ce00640 1 --2- 172.21.15.32:0/1317275104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67dfc69a0 0x55a67dfc6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd98ce00640 1 -- 172.21.15.32:0/1317275104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67e013b80 msgr2=0x55a67dfc6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:50.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd98ce00640 1 --2- 172.21.15.32:0/1317275104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67e013b80 0x55a67dfc6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd98ce00640 1 -- 172.21.15.32:0/1317275104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a67de5d8e0 con 0x55a67e013260 2024-04-17T10:02:50.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd98ce00640 1 --2- 172.21.15.32:0/1317275104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013260 0x55a67e013640 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fd97c006cf0 tx=0x7fd97c0acf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca83ae7fec87d5b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:50.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd979600640 1 -- 172.21.15.32:0/1317275104 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0f9200 con 0x55a67e013260 2024-04-17T10:02:50.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd979600640 1 -- 172.21.15.32:0/1317275104 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd97c0a8070 con 0x55a67e013260 2024-04-17T10:02:50.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd979600640 1 -- 172.21.15.32:0/1317275104 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0ecc90 con 0x55a67e013260 2024-04-17T10:02:50.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd992ced000 1 -- 172.21.15.32:0/1317275104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013260 msgr2=0x55a67e013640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:50.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd992ced000 1 --2- 172.21.15.32:0/1317275104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013260 0x55a67e013640 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fd97c006cf0 tx=0x7fd97c0acf90 comp rx=0 tx=0).stop 2024-04-17T10:02:50.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd992ced000 1 -- 172.21.15.32:0/1317275104 shutdown_connections 2024-04-17T10:02:50.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd992ced000 1 --2- 172.21.15.32:0/1317275104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67dfc69a0 0x55a67dfc6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd992ced000 1 --2- 172.21.15.32:0/1317275104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67e013b80 0x55a67dfc6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd992ced000 1 --2- 172.21.15.32:0/1317275104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013260 0x55a67e013640 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd992ced000 1 -- 172.21.15.32:0/1317275104 >> 172.21.15.32:0/1317275104 conn(0x55a67dff4000 msgr2=0x55a67dff63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:50.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd992ced000 1 -- 172.21.15.32:0/1317275104 shutdown_connections 2024-04-17T10:02:50.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.048+0000 7fd992ced000 1 -- 172.21.15.32:0/1317275104 wait complete. 2024-04-17T10:02:50.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd992ced000 1 -- start start 2024-04-17T10:02:50.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd992ced000 1 Processor -- start 2024-04-17T10:02:50.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd992ced000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67dfc69a0 0x55a67dfcc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:50.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd992ced000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67e013260 0x55a67dfcc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:50.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd992ced000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013b80 0x55a67dfccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:50.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd992ced000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a67de997b0 con 0x55a67e013b80 2024-04-17T10:02:50.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd992ced000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a67ded3d00 con 0x55a67dfc69a0 2024-04-17T10:02:50.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd992ced000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55a67dddd2c0 con 0x55a67e013260 2024-04-17T10:02:50.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd992ced000 10 client.0 _ll_register_callbacks cb 0x55a67dded030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:02:50.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd98d800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013b80 0x55a67dfccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:50.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd98d800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013b80 0x55a67dfccae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44434/0 (socket says 172.21.15.32:44434) 2024-04-17T10:02:50.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd98d800640 1 -- 172.21.15.32:0/189362213 learned_addr learned my addr 172.21.15.32:0/189362213 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:50.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd98ce00640 1 --2- 172.21.15.32:0/189362213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67dfc69a0 0x55a67dfcc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:50.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd987e00640 1 --2- 172.21.15.32:0/189362213 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67e013260 0x55a67dfcc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:50.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd98d800640 1 -- 172.21.15.32:0/189362213 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67e013260 msgr2=0x55a67dfcc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:50.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd98d800640 1 --2- 172.21.15.32:0/189362213 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67e013260 0x55a67dfcc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd98d800640 1 -- 172.21.15.32:0/189362213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67dfc69a0 msgr2=0x55a67dfcc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:50.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd98d800640 1 --2- 172.21.15.32:0/189362213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67dfc69a0 0x55a67dfcc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd98d800640 1 -- 172.21.15.32:0/189362213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a67ddf0490 con 0x55a67e013b80 2024-04-17T10:02:50.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd98ce00640 1 --2- 172.21.15.32:0/189362213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67dfc69a0 0x55a67dfcc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:02:50.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd98d800640 1 --2- 172.21.15.32:0/189362213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013b80 0x55a67dfccae0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fd98000e7b0 tx=0x7fd98000ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4029455faa6f3484 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:50.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd96e000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:02:50.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd96e000640 1 -- 172.21.15.32:0/189362213 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98000ee00 con 0x55a67e013b80 2024-04-17T10:02:50.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd96e000640 1 -- 172.21.15.32:0/189362213 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7fd98000c8d0 con 0x55a67e013b80 2024-04-17T10:02:50.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd992ced000 1 -- 172.21.15.32:0/189362213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd97c0a8070 con 0x55a67e013b80 2024-04-17T10:02:50.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd96e000640 1 -- 172.21.15.32:0/189362213 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fd980020050 con 0x55a67e013b80 2024-04-17T10:02:50.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd992ced000 10 client.9084 fetch_fsmap learned FSMap version 87 2024-04-17T10:02:50.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd992ced000 1 -- 172.21.15.32:0/189362213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x55a67dde3eb0 con 0x55a67e013b80 2024-04-17T10:02:50.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd96e000640 1 -- 172.21.15.32:0/189362213 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd97c0a8070 con 0x55a67e013b80 2024-04-17T10:02:50.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd96e000640 1 -- 172.21.15.32:0/189362213 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 87) ==== 3495+0+0 (secure 0 0 0) 0x7fd980023020 con 0x55a67e013b80 2024-04-17T10:02:50.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd992ced000 10 client.9084 fetch_fsmap finished waiting for FSMap version 87 2024-04-17T10:02:50.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd992ced000 10 client.9084 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-17T10:02:50.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd992ced000 20 client.9084 populate_metadata read hostname 'smithi032' 2024-04-17T10:02:50.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd992ced000 1 --2- 172.21.15.32:0/189362213 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x55a67e022500 0x55a67e0248f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:50.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd992ced000 4 client.9084 mds_command: new command op to 8964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:02:50.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd992ced000 1 -- 172.21.15.32:0/189362213 --> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a67ddb1dd0 con 0x55a67e022500 2024-04-17T10:02:50.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd98ce00640 1 --2- 172.21.15.32:0/189362213 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x55a67e022500 0x55a67e0248f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:50.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd96e000640 10 client.9084 ms_handle_connect on v2:172.21.15.32:6835/3078068380 2024-04-17T10:02:50.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.052+0000 7fd98ce00640 1 --2- 172.21.15.32:0/189362213 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x55a67e022500 0x55a67e0248f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2b50f0e6f0ddc41 server_cookie=5f483bf2818fd3ee in_seq=0 out_seq=0 2024-04-17T10:02:50.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.056+0000 7fd96e000640 1 -- 172.21.15.32:0/189362213 <== mds.0 v2:172.21.15.32:6835/3078068380 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a67ddb1dd0 con 0x55a67e022500 2024-04-17T10:02:50.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.056+0000 7fd96e000640 10 client.9084 handle_command_reply: tid=0 2024-04-17T10:02:50.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.056+0000 7fd992ced000 1 client.9084 shutdown 2024-04-17T10:02:50.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.056+0000 7fd992ced000 1 -- 172.21.15.32:0/189362213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013b80 msgr2=0x55a67dfccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:50.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.056+0000 7fd992ced000 1 --2- 172.21.15.32:0/189362213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013b80 0x55a67dfccae0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fd98000e7b0 tx=0x7fd98000ebb0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.056+0000 7fd992ced000 1 -- 172.21.15.32:0/189362213 shutdown_connections 2024-04-17T10:02:50.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.056+0000 7fd992ced000 1 --2- 172.21.15.32:0/189362213 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67e013260 0x55a67dfcc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.056+0000 7fd992ced000 1 --2- 172.21.15.32:0/189362213 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x55a67e022500 0x55a67e0248f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.056+0000 7fd992ced000 1 --2- 172.21.15.32:0/189362213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013b80 0x55a67dfccae0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.056+0000 7fd992ced000 1 --2- 172.21.15.32:0/189362213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67dfc69a0 0x55a67dfcc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.056+0000 7fd96e000640 0 client.9084 ms_handle_reset on v2:172.21.15.32:6835/3078068380 2024-04-17T10:02:50.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.056+0000 7fd992ced000 1 -- 172.21.15.32:0/189362213 >> 172.21.15.32:0/189362213 conn(0x55a67dff4000 msgr2=0x55a67dff63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:50.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.056+0000 7fd992ced000 1 -- 172.21.15.32:0/189362213 shutdown_connections 2024-04-17T10:02:50.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.056+0000 7fd992ced000 1 -- 172.21.15.32:0/189362213 wait complete. 2024-04-17T10:02:50.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.056+0000 7fd992ced000 20 client.9084 trim_cache size 0 max 16384 2024-04-17T10:02:50.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.072+0000 7fd992ced000 1 Processor -- start 2024-04-17T10:02:50.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.072+0000 7fd992ced000 1 -- start start 2024-04-17T10:02:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd992ced000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67dfc69a0 0x55a67e07dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd992ced000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67e013260 0x55a67e07e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd992ced000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013b80 0x55a67e07e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd992ced000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd988203120 con 0x55a67e013b80 2024-04-17T10:02:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd992ced000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd9882024b0 con 0x55a67dfc69a0 2024-04-17T10:02:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd992ced000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd988202b00 con 0x55a67e013260 2024-04-17T10:02:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd98d800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013b80 0x55a67e07e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd98d800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013b80 0x55a67e07e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44440/0 (socket says 172.21.15.32:44440) 2024-04-17T10:02:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd98d800640 1 -- 172.21.15.32:0/3026612545 learned_addr learned my addr 172.21.15.32:0/3026612545 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd98ce00640 1 --2- 172.21.15.32:0/3026612545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67dfc69a0 0x55a67e07dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd987e00640 1 --2- 172.21.15.32:0/3026612545 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67e013260 0x55a67e07e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:50.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd98d800640 1 -- 172.21.15.32:0/3026612545 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67e013260 msgr2=0x55a67e07e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:50.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd98d800640 1 --2- 172.21.15.32:0/3026612545 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67e013260 0x55a67e07e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd98d800640 1 -- 172.21.15.32:0/3026612545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67dfc69a0 msgr2=0x55a67e07dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:50.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd98d800640 1 --2- 172.21.15.32:0/3026612545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67dfc69a0 0x55a67e07dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd98d800640 1 -- 172.21.15.32:0/3026612545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a67ddd2a20 con 0x55a67e013b80 2024-04-17T10:02:50.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd98d800640 1 --2- 172.21.15.32:0/3026612545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013b80 0x55a67e07e710 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fd980001230 tx=0x7fd980012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da8b1ee65d2e7b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:50.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd978c00640 1 -- 172.21.15.32:0/3026612545 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98000c4b0 con 0x55a67e013b80 2024-04-17T10:02:50.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd978c00640 1 -- 172.21.15.32:0/3026612545 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd980028030 con 0x55a67e013b80 2024-04-17T10:02:50.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd978c00640 1 -- 172.21.15.32:0/3026612545 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9800016f0 con 0x55a67e013b80 2024-04-17T10:02:50.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd992ced000 1 -- 172.21.15.32:0/3026612545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013b80 msgr2=0x55a67e07e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:50.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd992ced000 1 --2- 172.21.15.32:0/3026612545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013b80 0x55a67e07e710 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fd980001230 tx=0x7fd980012000 comp rx=0 tx=0).stop 2024-04-17T10:02:50.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd992ced000 1 -- 172.21.15.32:0/3026612545 shutdown_connections 2024-04-17T10:02:50.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd992ced000 1 --2- 172.21.15.32:0/3026612545 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67e013260 0x55a67e07e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd992ced000 1 --2- 172.21.15.32:0/3026612545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013b80 0x55a67e07e710 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd992ced000 1 --2- 172.21.15.32:0/3026612545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67dfc69a0 0x55a67e07dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd992ced000 1 -- 172.21.15.32:0/3026612545 >> 172.21.15.32:0/3026612545 conn(0x55a67dff4000 msgr2=0x55a67dfc4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:50.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd992ced000 1 -- 172.21.15.32:0/3026612545 shutdown_connections 2024-04-17T10:02:50.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd992ced000 1 -- 172.21.15.32:0/3026612545 wait complete. 2024-04-17T10:02:50.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd992ced000 1 -- start start 2024-04-17T10:02:50.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.076+0000 7fd992ced000 1 Processor -- start 2024-04-17T10:02:50.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67dfc69a0 0x55a67e027000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67e013260 0x55a67e03e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013b80 0x55a67e03e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a67ddbc050 con 0x55a67e013b80 2024-04-17T10:02:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a67ddda570 con 0x55a67e013260 2024-04-17T10:02:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55a67de17800 con 0x55a67dfc69a0 2024-04-17T10:02:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 10 client.0 _ll_register_callbacks cb 0x7fd980001a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:02:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd98d800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013b80 0x55a67e03e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd98d800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013b80 0x55a67e03e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44446/0 (socket says 172.21.15.32:44446) 2024-04-17T10:02:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd98d800640 1 -- 172.21.15.32:0/2845751137 learned_addr learned my addr 172.21.15.32:0/2845751137 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:02:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd98ce00640 1 --2- 172.21.15.32:0/2845751137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67dfc69a0 0x55a67e027000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd987e00640 1 --2- 172.21.15.32:0/2845751137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67e013260 0x55a67e03e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd98d800640 1 -- 172.21.15.32:0/2845751137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67dfc69a0 msgr2=0x55a67e027000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd98d800640 1 --2- 172.21.15.32:0/2845751137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67dfc69a0 0x55a67e027000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd98d800640 1 -- 172.21.15.32:0/2845751137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67e013260 msgr2=0x55a67e03e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:02:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd98d800640 1 --2- 172.21.15.32:0/2845751137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67e013260 0x55a67e03e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd98d800640 1 -- 172.21.15.32:0/2845751137 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a67de99c40 con 0x55a67e013b80 2024-04-17T10:02:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd98d800640 1 --2- 172.21.15.32:0/2845751137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013b80 0x55a67e03e9c0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fd98000cbf0 tx=0x7fd980005b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5984715c264c7ca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:02:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd96e000640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:02:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd96e000640 1 -- 172.21.15.32:0/2845751137 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98000c8d0 con 0x55a67e013b80 2024-04-17T10:02:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd96e000640 1 -- 172.21.15.32:0/2845751137 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7fd98000ee00 con 0x55a67e013b80 2024-04-17T10:02:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 1 -- 172.21.15.32:0/2845751137 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd980028030 con 0x55a67e013b80 2024-04-17T10:02:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd96e000640 1 -- 172.21.15.32:0/2845751137 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fd98003e050 con 0x55a67e013b80 2024-04-17T10:02:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 10 client.9090 fetch_fsmap learned FSMap version 87 2024-04-17T10:02:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 1 -- 172.21.15.32:0/2845751137 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x55a67ddbd600 con 0x55a67e013b80 2024-04-17T10:02:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd96e000640 1 -- 172.21.15.32:0/2845751137 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd980028030 con 0x55a67e013b80 2024-04-17T10:02:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd96e000640 1 -- 172.21.15.32:0/2845751137 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 87) ==== 3495+0+0 (secure 0 0 0) 0x7fd980041020 con 0x55a67e013b80 2024-04-17T10:02:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 10 client.9090 fetch_fsmap finished waiting for FSMap version 87 2024-04-17T10:02:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 10 client.9090 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-17T10:02:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 20 client.9090 populate_metadata read hostname 'smithi032' 2024-04-17T10:02:50.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 1 --2- 172.21.15.32:0/2845751137 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x55a67e0818b0 0x55a67e081cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:02:50.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 4 client.9090 mds_command: new command op to 8964 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:02:50.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 1 -- 172.21.15.32:0/2845751137 --> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a67dff3c40 con 0x55a67e0818b0 2024-04-17T10:02:50.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd98ce00640 1 --2- 172.21.15.32:0/2845751137 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x55a67e0818b0 0x55a67e081cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:02:50.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd98ce00640 1 --2- 172.21.15.32:0/2845751137 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x55a67e0818b0 0x55a67e081cc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efc6189d2ddf3c77 server_cookie=20497f27f1509c55 in_seq=0 out_seq=0 2024-04-17T10:02:50.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd96e000640 10 client.9090 ms_handle_connect on v2:172.21.15.32:6835/3078068380 2024-04-17T10:02:50.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd96e000640 1 -- 172.21.15.32:0/2845751137 <== mds.0 v2:172.21.15.32:6835/3078068380 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55a67dff3c40 con 0x55a67e0818b0 2024-04-17T10:02:50.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd96e000640 10 client.9090 handle_command_reply: tid=0 2024-04-17T10:02:50.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 1 client.9090 shutdown 2024-04-17T10:02:50.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 1 -- 172.21.15.32:0/2845751137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013b80 msgr2=0x55a67e03e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:50.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 1 --2- 172.21.15.32:0/2845751137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013b80 0x55a67e03e9c0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fd98000cbf0 tx=0x7fd980005b80 comp rx=0 tx=0).stop 2024-04-17T10:02:50.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 1 -- 172.21.15.32:0/2845751137 shutdown_connections 2024-04-17T10:02:50.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 1 --2- 172.21.15.32:0/2845751137 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x55a67e0818b0 0x55a67e081cc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 1 --2- 172.21.15.32:0/2845751137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a67e013b80 0x55a67e03e9c0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 1 --2- 172.21.15.32:0/2845751137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a67e013260 0x55a67e03e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd96e000640 0 client.9090 ms_handle_reset on v2:172.21.15.32:6835/3078068380 2024-04-17T10:02:50.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 1 --2- 172.21.15.32:0/2845751137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a67dfc69a0 0x55a67e027000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 1 -- 172.21.15.32:0/2845751137 >> 172.21.15.32:0/2845751137 conn(0x55a67dff4000 msgr2=0x55a67dfc5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:50.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.080+0000 7fd992ced000 1 -- 172.21.15.32:0/2845751137 shutdown_connections 2024-04-17T10:02:50.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.084+0000 7fd992ced000 1 -- 172.21.15.32:0/2845751137 wait complete. 2024-04-17T10:02:50.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.084+0000 7fd992ced000 20 client.9090 trim_cache size 0 max 16384 2024-04-17T10:02:50.083 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:02:50.083 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T10:02:50.083 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2024-04-17T10:02:50.083 INFO:teuthology.orchestra.run.smithi032.stdout: "6309f98b-ba71-41b3-a13d-82d033c07f04": { 2024-04-17T10:02:50.084 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2024-04-17T10:02:50.084 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "6309f98b-ba71-41b3-a13d-82d033c07f04", 2024-04-17T10:02:50.084 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,repair" 2024-04-17T10:02:50.084 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:02:50.084 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:02:50.084 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:02:50.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.084+0000 7fd98f800640 1 -- 172.21.15.32:0/642738252 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd9680683b0 msgr2=0x7fd96806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:50.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.084+0000 7fd98f800640 1 --2- 172.21.15.32:0/642738252 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd9680683b0 0x7fd96806a870 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd9881fb380 tx=0x7fd974009040 comp rx=0 tx=0).stop 2024-04-17T10:02:50.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.084+0000 7fd98f800640 1 -- 172.21.15.32:0/642738252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98806ba30 msgr2=0x7fd98813d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:02:50.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.084+0000 7fd98f800640 1 --2- 172.21.15.32:0/642738252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98806ba30 0x7fd98813d940 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0af040 tx=0x7fd97c019180 comp rx=0 tx=0).stop 2024-04-17T10:02:50.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.084+0000 7fd98f800640 1 -- 172.21.15.32:0/642738252 shutdown_connections 2024-04-17T10:02:50.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.084+0000 7fd98f800640 1 --2- 172.21.15.32:0/642738252 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd9680683b0 0x7fd96806a870 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.084+0000 7fd98f800640 1 --2- 172.21.15.32:0/642738252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9880cf370 0x7fd988159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.084+0000 7fd98f800640 1 --2- 172.21.15.32:0/642738252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd988071e60 0x7fd9880b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.084+0000 7fd98f800640 1 --2- 172.21.15.32:0/642738252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98806ba30 0x7fd98813d940 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:02:50.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.084+0000 7fd98f800640 1 -- 172.21.15.32:0/642738252 >> 172.21.15.32:0/642738252 conn(0x7fd9880b2120 msgr2=0x7fd9880b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:02:50.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.084+0000 7fd98f800640 1 -- 172.21.15.32:0/642738252 shutdown_connections 2024-04-17T10:02:50.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:50.084+0000 7fd98f800640 1 -- 172.21.15.32:0/642738252 wait complete. 2024-04-17T10:02:50.099 INFO:tasks.cephfs.filesystem:scrub status for tag:6309f98b-ba71-41b3-a13d-82d033c07f04 - {'path': '/', 'tag': '6309f98b-ba71-41b3-a13d-82d033c07f04', 'options': 'recursive,repair'} 2024-04-17T10:02:51.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:51.072+0000 7f79fcc00640 1 -- 172.21.15.32:0/791571353 <== mon.0 v2:172.21.15.32:3300/0 8 ==== log(8 entries from seq 1 at 2024-04-17T10:02:49.873503+0000) ==== 1407+0+0 (secure 0 0 0) 0x7f79ec054920 con 0x7f7a000b29b0 2024-04-17T10:02:51.069 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T10:02:49.873503+0000 mds.c [INF] scrub queued for path: / 2024-04-17T10:02:51.069 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T10:02:49.873506+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-17T10:02:51.069 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T10:02:49.873510+0000 mds.c [INF] scrub summary: active paths [/] 2024-04-17T10:02:51.069 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T10:02:49.874462+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-04-17T10:02:51.069 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T10:02:49.874469+0000 mds.c [ERR] inode table repaired for inode: 0x10000000001 2024-04-17T10:02:51.070 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T10:02:49.874542+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-04-17T10:02:51.070 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T10:02:49.874548+0000 mds.c [ERR] inode table repaired for inode: 0x10000000002 2024-04-17T10:02:51.070 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T10:02:49.875215+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-17T10:02:56.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:02:56.084+0000 7f79fcc00640 1 -- 172.21.15.32:0/791571353 <== mon.0 v2:172.21.15.32:3300/0 9 ==== log(1 entries from seq 9 at 2024-04-17T10:02:54.109361+0000) ==== 189+0+0 (secure 0 0 0) 0x7f79ec084e00 con 0x7f7a000b29b0 2024-04-17T10:02:56.083 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T10:02:54.109361+0000 mds.c [INF] scrub summary: idle 2024-04-17T10:03:14.455 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:03:14.460 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:03:20.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:20.101 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-17T10:03:20.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c9e00640 1 Processor -- start 2024-04-17T10:03:20.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c9e00640 1 -- start start 2024-04-17T10:03:20.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c9e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0c415eae0 0x7fe0c40d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c9e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe0c406ce50 0x7fe0c406d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c9e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0c409ac40 con 0x7fe0c40d1750 2024-04-17T10:03:20.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c9e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe0c409a9f0 con 0x7fe0c406ce50 2024-04-17T10:03:20.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c9e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c409b570 con 0x7fe0c415eae0 2024-04-17T10:03:20.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c3400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe0c40d1750 0x7fe0c40ccbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58190/0 (socket says 172.21.15.32:58190) 2024-04-17T10:03:20.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c3400640 1 -- 172.21.15.32:0/1134471148 learned_addr learned my addr 172.21.15.32:0/1134471148 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:20.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c2a00640 1 --2- 172.21.15.32:0/1134471148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0c415eae0 0x7fe0c40d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0ba000640 1 --2- 172.21.15.32:0/1134471148 >> v2:172.21.15.144:3300/0 conn(0x7fe0c406ce50 0x7fe0c406d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c2000640 1 -- 172.21.15.32:0/1134471148 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1705515870 0 0) 0x7fe0c409ac40 con 0x7fe0c40d1750 2024-04-17T10:03:20.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c2000640 1 -- 172.21.15.32:0/1134471148 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0a8003530 con 0x7fe0c40d1750 2024-04-17T10:03:20.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c2000640 1 -- 172.21.15.32:0/1134471148 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3074094732 0 0) 0x7fe0a8003530 con 0x7fe0c40d1750 2024-04-17T10:03:20.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c2000640 1 -- 172.21.15.32:0/1134471148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0c415eae0 msgr2=0x7fe0c40d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:20.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c2000640 1 --2- 172.21.15.32:0/1134471148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0c415eae0 0x7fe0c40d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c2000640 1 -- 172.21.15.32:0/1134471148 >> v2:172.21.15.144:3300/0 conn(0x7fe0c406ce50 msgr2=0x7fe0c406d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:20.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c2000640 1 --2- 172.21.15.32:0/1134471148 >> v2:172.21.15.144:3300/0 conn(0x7fe0c406ce50 0x7fe0c406d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c2000640 1 -- 172.21.15.32:0/1134471148 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c409af20 con 0x7fe0c40d1750 2024-04-17T10:03:20.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c2000640 1 -- 172.21.15.32:0/1134471148 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7fe0bc001a00 con 0x7fe0c40d1750 2024-04-17T10:03:20.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c2000640 1 -- 172.21.15.32:0/1134471148 >> v1:172.21.15.32:6789/0 conn(0x7fe0c40d1750 legacy=0x7fe0c40ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:20.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c2000640 1 --2- 172.21.15.32:0/1134471148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0c415eae0 0x7fe0a80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c2000640 1 --2- 172.21.15.32:0/1134471148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a8003f90 0x7fe0a8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c2000640 1 --2- 172.21.15.32:0/1134471148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0a8004d50 0x7fe0a80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c2000640 1 -- 172.21.15.32:0/1134471148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c409a9f0 con 0x7fe0a8003f90 2024-04-17T10:03:20.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c2000640 1 -- 172.21.15.32:0/1134471148 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c409b570 con 0x7fe0a8004d50 2024-04-17T10:03:20.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c2000640 1 -- 172.21.15.32:0/1134471148 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe0a800bb00 con 0x7fe0c415eae0 2024-04-17T10:03:20.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0ba000640 1 --2- 172.21.15.32:0/1134471148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a8003f90 0x7fe0a8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c2a00640 1 --2- 172.21.15.32:0/1134471148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0c415eae0 0x7fe0a80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.260+0000 7fe0c3400640 1 --2- 172.21.15.32:0/1134471148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0a8004d50 0x7fe0a80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.264+0000 7fe0ba000640 1 -- 172.21.15.32:0/1134471148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0c415eae0 msgr2=0x7fe0a80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:20.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.264+0000 7fe0ba000640 1 --2- 172.21.15.32:0/1134471148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0c415eae0 0x7fe0a80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.264+0000 7fe0ba000640 1 -- 172.21.15.32:0/1134471148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0a8004d50 msgr2=0x7fe0a80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:20.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.264+0000 7fe0ba000640 1 --2- 172.21.15.32:0/1134471148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0a8004d50 0x7fe0a80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.264+0000 7fe0ba000640 1 -- 172.21.15.32:0/1134471148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a8003530 con 0x7fe0a8003f90 2024-04-17T10:03:20.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.264+0000 7fe0c3400640 1 --2- 172.21.15.32:0/1134471148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0a8004d50 0x7fe0a80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:20.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.264+0000 7fe0ba000640 1 --2- 172.21.15.32:0/1134471148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a8003f90 0x7fe0a8004810 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009ae0 tx=0x7fe0ac01ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a65088954943791 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:20.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.264+0000 7fe0c2000640 1 -- 172.21.15.32:0/1134471148 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0b2070 con 0x7fe0a8003f90 2024-04-17T10:03:20.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.264+0000 7fe0c2000640 1 -- 172.21.15.32:0/1134471148 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0ac0a5670 con 0x7fe0a8003f90 2024-04-17T10:03:20.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.264+0000 7fe0c2000640 1 -- 172.21.15.32:0/1134471148 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0096b0 con 0x7fe0a8003f90 2024-04-17T10:03:20.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.264+0000 7fe0c9e00640 1 -- 172.21.15.32:0/1134471148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a8003f90 msgr2=0x7fe0a8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:20.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.264+0000 7fe0c9e00640 1 --2- 172.21.15.32:0/1134471148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a8003f90 0x7fe0a8004810 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009ae0 tx=0x7fe0ac01ca80 comp rx=0 tx=0).stop 2024-04-17T10:03:20.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.264+0000 7fe0c3400640 1 -- 172.21.15.32:0/1134471148 reap_dead start 2024-04-17T10:03:20.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.264+0000 7fe0c9e00640 1 -- 172.21.15.32:0/1134471148 shutdown_connections 2024-04-17T10:03:20.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.264+0000 7fe0c9e00640 1 -- 172.21.15.32:0/1134471148 >> 172.21.15.32:0/1134471148 conn(0x7fe0c40b4080 msgr2=0x7fe0c40b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:20.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0c9e00640 1 -- 172.21.15.32:0/1134471148 shutdown_connections 2024-04-17T10:03:20.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0c9e00640 1 -- 172.21.15.32:0/1134471148 wait complete. 2024-04-17T10:03:20.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0c9e00640 1 Processor -- start 2024-04-17T10:03:20.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0c9e00640 1 -- start start 2024-04-17T10:03:20.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0c9e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a8003f90 0x7fe0c4168300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0c9e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0c4158760 0x7fe0c4158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0c9e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0c4159100 0x7fe0c41668a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0c9e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c406de20 con 0x7fe0a8003f90 2024-04-17T10:03:20.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0c9e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c406dca0 con 0x7fe0c4159100 2024-04-17T10:03:20.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0c9e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c4169b20 con 0x7fe0c4158760 2024-04-17T10:03:20.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0c2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a8003f90 0x7fe0c4168300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0c2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a8003f90 0x7fe0c4168300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38914/0 (socket says 172.21.15.32:38914) 2024-04-17T10:03:20.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0c2a00640 1 -- 172.21.15.32:0/3328393297 learned_addr learned my addr 172.21.15.32:0/3328393297 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:20.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0ba000640 1 --2- 172.21.15.32:0/3328393297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0c4158760 0x7fe0c4158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0c3400640 1 --2- 172.21.15.32:0/3328393297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0c4159100 0x7fe0c41668a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0c2a00640 1 -- 172.21.15.32:0/3328393297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0c4158760 msgr2=0x7fe0c4158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:20.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0c2a00640 1 --2- 172.21.15.32:0/3328393297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0c4158760 0x7fe0c4158bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0c2a00640 1 -- 172.21.15.32:0/3328393297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0c4159100 msgr2=0x7fe0c41668a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:20.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0c2a00640 1 --2- 172.21.15.32:0/3328393297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0c4159100 0x7fe0c41668a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0c2a00640 1 -- 172.21.15.32:0/3328393297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c4159580 con 0x7fe0a8003f90 2024-04-17T10:03:20.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0c2a00640 1 --2- 172.21.15.32:0/3328393297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a8003f90 0x7fe0c4168300 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4011a00 tx=0x7fe0b4011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:20.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0bbe00640 1 -- 172.21.15.32:0/3328393297 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b400fb50 con 0x7fe0a8003f90 2024-04-17T10:03:20.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0bbe00640 1 -- 172.21.15.32:0/3328393297 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b400fcf0 con 0x7fe0a8003f90 2024-04-17T10:03:20.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0bbe00640 1 -- 172.21.15.32:0/3328393297 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b400fea0 con 0x7fe0a8003f90 2024-04-17T10:03:20.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.268+0000 7fe0c9e00640 1 -- 172.21.15.32:0/3328393297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c4165b00 con 0x7fe0a8003f90 2024-04-17T10:03:20.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.272+0000 7fe0c9e00640 1 -- 172.21.15.32:0/3328393297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c4165dc0 con 0x7fe0a8003f90 2024-04-17T10:03:20.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.272+0000 7fe0bbe00640 1 -- 172.21.15.32:0/3328393297 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe0b4012040 con 0x7fe0a8003f90 2024-04-17T10:03:20.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.272+0000 7fe0bbe00640 1 --2- 172.21.15.32:0/3328393297 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0a00683b0 0x7fe0a006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.272+0000 7fe0bbe00640 1 -- 172.21.15.32:0/3328393297 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7fe0b4086f20 con 0x7fe0a8003f90 2024-04-17T10:03:20.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.272+0000 7fe0ba000640 1 --2- 172.21.15.32:0/3328393297 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0a00683b0 0x7fe0a006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0cd3ab000 1 Processor -- start 2024-04-17T10:03:20.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0cd3ab000 1 -- start start 2024-04-17T10:03:20.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0ba000640 1 --2- 172.21.15.32:0/3328393297 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0a00683b0 0x7fe0a006a870 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0af000 tx=0x7fe0ac0046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:20.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0cd3ab000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555d21596260 0x555d21596640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0cd3ab000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d21596b80 0x555d21549460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0cd3ab000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d215499a0 0x555d21549d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0cd3ab000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555d2139b4f0 con 0x555d215499a0 2024-04-17T10:03:20.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0cd3ab000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555d213546c0 con 0x555d21596b80 2024-04-17T10:03:20.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0cd3ab000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x555d213974b0 con 0x555d21596260 2024-04-17T10:03:20.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0c3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d215499a0 0x555d21549d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0c3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d215499a0 0x555d21549d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38928/0 (socket says 172.21.15.32:38928) 2024-04-17T10:03:20.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0c3400640 1 -- 172.21.15.32:0/4229164415 learned_addr learned my addr 172.21.15.32:0/4229164415 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:20.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0c2a00640 1 --2- 172.21.15.32:0/4229164415 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555d21596260 0x555d21596640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0ba000640 1 --2- 172.21.15.32:0/4229164415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d21596b80 0x555d21549460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0c3400640 1 -- 172.21.15.32:0/4229164415 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555d21596260 msgr2=0x555d21596640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:20.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0c3400640 1 --2- 172.21.15.32:0/4229164415 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555d21596260 0x555d21596640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0c3400640 1 -- 172.21.15.32:0/4229164415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d21596b80 msgr2=0x555d21549460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:20.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0c3400640 1 --2- 172.21.15.32:0/4229164415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d21596b80 0x555d21549460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0c3400640 1 -- 172.21.15.32:0/4229164415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d213e08e0 con 0x555d215499a0 2024-04-17T10:03:20.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0c3400640 1 --2- 172.21.15.32:0/4229164415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d215499a0 0x555d21549d80 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc0014b0 tx=0x7fe0bc001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887827c66f186a68 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:20.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0b3e00640 1 -- 172.21.15.32:0/4229164415 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc007700 con 0x555d215499a0 2024-04-17T10:03:20.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0b3e00640 1 -- 172.21.15.32:0/4229164415 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0bc019070 con 0x555d215499a0 2024-04-17T10:03:20.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0b3e00640 1 -- 172.21.15.32:0/4229164415 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc007ac0 con 0x555d215499a0 2024-04-17T10:03:20.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/4229164415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d215499a0 msgr2=0x555d21549d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:20.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0cd3ab000 1 --2- 172.21.15.32:0/4229164415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d215499a0 0x555d21549d80 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc0014b0 tx=0x7fe0bc001b90 comp rx=0 tx=0).stop 2024-04-17T10:03:20.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/4229164415 shutdown_connections 2024-04-17T10:03:20.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0cd3ab000 1 --2- 172.21.15.32:0/4229164415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d215499a0 0x555d21549d80 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0cd3ab000 1 --2- 172.21.15.32:0/4229164415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d21596b80 0x555d21549460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0cd3ab000 1 --2- 172.21.15.32:0/4229164415 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555d21596260 0x555d21596640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/4229164415 >> 172.21.15.32:0/4229164415 conn(0x555d21577000 msgr2=0x555d215793f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:20.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/4229164415 shutdown_connections 2024-04-17T10:03:20.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/4229164415 wait complete. 2024-04-17T10:03:20.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0cd3ab000 1 -- start start 2024-04-17T10:03:20.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.276+0000 7fe0cd3ab000 1 Processor -- start 2024-04-17T10:03:20.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0cd3ab000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555d215499a0 0x555d2154f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0cd3ab000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d21596260 0x555d2154f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0cd3ab000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d21596b80 0x555d2154fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0cd3ab000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555d2141c7b0 con 0x555d21596b80 2024-04-17T10:03:20.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0cd3ab000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555d21456d00 con 0x555d21596260 2024-04-17T10:03:20.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0cd3ab000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x555d213602c0 con 0x555d215499a0 2024-04-17T10:03:20.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0cd3ab000 10 client.0 _ll_register_callbacks cb 0x555d21370030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:03:20.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0c3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d21596b80 0x555d2154fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0c3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d21596b80 0x555d2154fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38930/0 (socket says 172.21.15.32:38930) 2024-04-17T10:03:20.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0c3400640 1 -- 172.21.15.32:0/691831321 learned_addr learned my addr 172.21.15.32:0/691831321 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:20.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0c2a00640 1 --2- 172.21.15.32:0/691831321 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555d215499a0 0x555d2154f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0ba000640 1 --2- 172.21.15.32:0/691831321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d21596260 0x555d2154f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0c3400640 1 -- 172.21.15.32:0/691831321 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555d215499a0 msgr2=0x555d2154f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:20.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0c3400640 1 --2- 172.21.15.32:0/691831321 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555d215499a0 0x555d2154f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0c3400640 1 -- 172.21.15.32:0/691831321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d21596260 msgr2=0x555d2154f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:20.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0c3400640 1 --2- 172.21.15.32:0/691831321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d21596260 0x555d2154f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0c3400640 1 -- 172.21.15.32:0/691831321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d21373490 con 0x555d21596b80 2024-04-17T10:03:20.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0c3400640 1 --2- 172.21.15.32:0/691831321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d21596b80 0x555d2154fab0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fe0c409ab70 tx=0x7fe0bc001d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f480c34492ff1f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:20.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:03:20.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0b0c00640 1 -- 172.21.15.32:0/691831321 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc01f620 con 0x555d21596b80 2024-04-17T10:03:20.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0b0c00640 1 -- 172.21.15.32:0/691831321 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7fe0bc01f7c0 con 0x555d21596b80 2024-04-17T10:03:20.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/691831321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0bc019070 con 0x555d21596b80 2024-04-17T10:03:20.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0b0c00640 1 -- 172.21.15.32:0/691831321 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fe0bc02e050 con 0x555d21596b80 2024-04-17T10:03:20.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0cd3ab000 10 client.9102 fetch_fsmap learned FSMap version 87 2024-04-17T10:03:20.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/691831321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x555d21366eb0 con 0x555d21596b80 2024-04-17T10:03:20.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0b0c00640 1 -- 172.21.15.32:0/691831321 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0bc019070 con 0x555d21596b80 2024-04-17T10:03:20.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0b0c00640 1 -- 172.21.15.32:0/691831321 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 87) ==== 3495+0+0 (secure 0 0 0) 0x7fe0bc031020 con 0x555d21596b80 2024-04-17T10:03:20.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0cd3ab000 10 client.9102 fetch_fsmap finished waiting for FSMap version 87 2024-04-17T10:03:20.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0cd3ab000 10 client.9102 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-17T10:03:20.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0cd3ab000 20 client.9102 populate_metadata read hostname 'smithi032' 2024-04-17T10:03:20.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0cd3ab000 1 --2- 172.21.15.32:0/691831321 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x555d215a5500 0x555d215a78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0cd3ab000 4 client.9102 mds_command: new command op to 8964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:03:20.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/691831321 --> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d21334dd0 con 0x555d215a5500 2024-04-17T10:03:20.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0c2a00640 1 --2- 172.21.15.32:0/691831321 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x555d215a5500 0x555d215a78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0c2a00640 1 --2- 172.21.15.32:0/691831321 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x555d215a5500 0x555d215a78f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2415e96da4f0ea0 server_cookie=ae14f35b0a98a90f in_seq=0 out_seq=0 2024-04-17T10:03:20.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.280+0000 7fe0b0c00640 10 client.9102 ms_handle_connect on v2:172.21.15.32:6835/3078068380 2024-04-17T10:03:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.284+0000 7fe0b0c00640 1 -- 172.21.15.32:0/691831321 <== mds.0 v2:172.21.15.32:6835/3078068380 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555d21334dd0 con 0x555d215a5500 2024-04-17T10:03:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.284+0000 7fe0b0c00640 10 client.9102 handle_command_reply: tid=0 2024-04-17T10:03:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.284+0000 7fe0cd3ab000 1 client.9102 shutdown 2024-04-17T10:03:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.284+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/691831321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d21596b80 msgr2=0x555d2154fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.284+0000 7fe0cd3ab000 1 --2- 172.21.15.32:0/691831321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d21596b80 0x555d2154fab0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fe0c409ab70 tx=0x7fe0bc001d70 comp rx=0 tx=0).stop 2024-04-17T10:03:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.284+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/691831321 shutdown_connections 2024-04-17T10:03:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.284+0000 7fe0cd3ab000 1 --2- 172.21.15.32:0/691831321 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x555d215a5500 0x555d215a78f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.284+0000 7fe0cd3ab000 1 --2- 172.21.15.32:0/691831321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d21596b80 0x555d2154fab0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.284+0000 7fe0cd3ab000 1 --2- 172.21.15.32:0/691831321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d21596260 0x555d2154f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.284+0000 7fe0cd3ab000 1 --2- 172.21.15.32:0/691831321 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555d215499a0 0x555d2154f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.284+0000 7fe0b0c00640 0 client.9102 ms_handle_reset on v2:172.21.15.32:6835/3078068380 2024-04-17T10:03:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.284+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/691831321 >> 172.21.15.32:0/691831321 conn(0x555d21577000 msgr2=0x555d215793f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.284+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/691831321 shutdown_connections 2024-04-17T10:03:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.284+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/691831321 wait complete. 2024-04-17T10:03:20.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.284+0000 7fe0cd3ab000 20 client.9102 trim_cache size 0 max 16384 2024-04-17T10:03:20.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.300+0000 7fe0cd3ab000 1 Processor -- start 2024-04-17T10:03:20.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.300+0000 7fe0cd3ab000 1 -- start start 2024-04-17T10:03:20.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.300+0000 7fe0cd3ab000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555d215499a0 0x555d21549140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.300+0000 7fe0cd3ab000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d21596260 0x555d21547bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.300+0000 7fe0cd3ab000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d21596b80 0x555d21548100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.300+0000 7fe0cd3ab000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c4203e60 con 0x555d21596260 2024-04-17T10:03:20.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.300+0000 7fe0cd3ab000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c40c8910 con 0x555d21596b80 2024-04-17T10:03:20.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.300+0000 7fe0cd3ab000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c42038e0 con 0x555d215499a0 2024-04-17T10:03:20.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.300+0000 7fe0ba000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d21596260 0x555d21547bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.300+0000 7fe0ba000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d21596260 0x555d21547bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38934/0 (socket says 172.21.15.32:38934) 2024-04-17T10:03:20.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.300+0000 7fe0ba000640 1 -- 172.21.15.32:0/3549290613 learned_addr learned my addr 172.21.15.32:0/3549290613 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:20.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.300+0000 7fe0ba000640 1 -- 172.21.15.32:0/3549290613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555d215499a0 msgr2=0x555d21549140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:20.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.300+0000 7fe0ba000640 1 --2- 172.21.15.32:0/3549290613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555d215499a0 0x555d21549140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.300+0000 7fe0ba000640 1 -- 172.21.15.32:0/3549290613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d21596b80 msgr2=0x555d21548100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.300+0000 7fe0ba000640 1 --2- 172.21.15.32:0/3549290613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d21596b80 0x555d21548100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.300+0000 7fe0ba000640 1 -- 172.21.15.32:0/3549290613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d21355a20 con 0x555d21596260 2024-04-17T10:03:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0ba000640 1 --2- 172.21.15.32:0/3549290613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d21596260 0x555d21547bc0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009ae0 tx=0x7fe0ac004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77aea6c8f80a5f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0b3400640 1 -- 172.21.15.32:0/3549290613 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0b2040 con 0x555d21596260 2024-04-17T10:03:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0b3400640 1 -- 172.21.15.32:0/3549290613 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0ac0c8070 con 0x555d21596260 2024-04-17T10:03:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0b3400640 1 -- 172.21.15.32:0/3549290613 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0a5db0 con 0x555d21596260 2024-04-17T10:03:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/3549290613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d21596260 msgr2=0x555d21547bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0cd3ab000 1 --2- 172.21.15.32:0/3549290613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d21596260 0x555d21547bc0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009ae0 tx=0x7fe0ac004740 comp rx=0 tx=0).stop 2024-04-17T10:03:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/3549290613 shutdown_connections 2024-04-17T10:03:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0cd3ab000 1 --2- 172.21.15.32:0/3549290613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d21596b80 0x555d21548100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0cd3ab000 1 --2- 172.21.15.32:0/3549290613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d21596260 0x555d21547bc0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0cd3ab000 1 --2- 172.21.15.32:0/3549290613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555d215499a0 0x555d21549140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/3549290613 >> 172.21.15.32:0/3549290613 conn(0x555d21577000 msgr2=0x555d21594310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:20.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/3549290613 shutdown_connections 2024-04-17T10:03:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/3549290613 wait complete. 2024-04-17T10:03:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0cd3ab000 1 -- start start 2024-04-17T10:03:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0cd3ab000 1 Processor -- start 2024-04-17T10:03:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0cd3ab000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555d215499a0 0x555d215aa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0cd3ab000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d21596260 0x555d215c1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0cd3ab000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d21596b80 0x555d215c19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0cd3ab000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x555d2133f050 con 0x555d21596b80 2024-04-17T10:03:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0cd3ab000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555d2135d570 con 0x555d21596260 2024-04-17T10:03:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0cd3ab000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x555d2139a800 con 0x555d215499a0 2024-04-17T10:03:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0cd3ab000 10 client.0 _ll_register_callbacks cb 0x7fe0b40530d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:03:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0c3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d21596b80 0x555d215c19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0c3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d21596b80 0x555d215c19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43016/0 (socket says 172.21.15.32:43016) 2024-04-17T10:03:20.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0c3400640 1 -- 172.21.15.32:0/4150776710 learned_addr learned my addr 172.21.15.32:0/4150776710 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0ba000640 1 --2- 172.21.15.32:0/4150776710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d21596260 0x555d215c1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0c2a00640 1 --2- 172.21.15.32:0/4150776710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555d215499a0 0x555d215aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0c3400640 1 -- 172.21.15.32:0/4150776710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555d215499a0 msgr2=0x555d215aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0c3400640 1 --2- 172.21.15.32:0/4150776710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555d215499a0 0x555d215aa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0c3400640 1 -- 172.21.15.32:0/4150776710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d21596260 msgr2=0x555d215c1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0c3400640 1 --2- 172.21.15.32:0/4150776710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d21596260 0x555d215c1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0c3400640 1 -- 172.21.15.32:0/4150776710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d2141cc40 con 0x555d21596b80 2024-04-17T10:03:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0c2a00640 1 --2- 172.21.15.32:0/4150776710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555d215499a0 0x555d215aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.304+0000 7fe0c3400640 1 --2- 172.21.15.32:0/4150776710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d21596b80 0x555d215c19c0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc0079e0 tx=0x7fe0bc001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0cbf5ccd9eb3ea server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:03:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0b0c00640 1 -- 172.21.15.32:0/4150776710 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc01fde0 con 0x555d21596b80 2024-04-17T10:03:20.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/4150776710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0ac0c8070 con 0x555d21596b80 2024-04-17T10:03:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0b0c00640 1 -- 172.21.15.32:0/4150776710 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7fe0bc014400 con 0x555d21596b80 2024-04-17T10:03:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0b0c00640 1 -- 172.21.15.32:0/4150776710 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fe0bc040050 con 0x555d21596b80 2024-04-17T10:03:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0cd3ab000 10 client.9108 fetch_fsmap learned FSMap version 87 2024-04-17T10:03:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/4150776710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x555d21340600 con 0x555d21596b80 2024-04-17T10:03:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0b0c00640 1 -- 172.21.15.32:0/4150776710 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0ac0c8070 con 0x555d21596b80 2024-04-17T10:03:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0b0c00640 1 -- 172.21.15.32:0/4150776710 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 87) ==== 3495+0+0 (secure 0 0 0) 0x7fe0bc043020 con 0x555d21596b80 2024-04-17T10:03:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0cd3ab000 10 client.9108 fetch_fsmap finished waiting for FSMap version 87 2024-04-17T10:03:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0cd3ab000 10 client.9108 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-17T10:03:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0cd3ab000 20 client.9108 populate_metadata read hostname 'smithi032' 2024-04-17T10:03:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0cd3ab000 1 --2- 172.21.15.32:0/4150776710 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x555d21605300 0x555d216056e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0cd3ab000 4 client.9108 mds_command: new command op to 8964 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T10:03:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/4150776710 --> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555d21576c40 con 0x555d21605300 2024-04-17T10:03:20.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0c2a00640 1 --2- 172.21.15.32:0/4150776710 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x555d21605300 0x555d216056e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0c2a00640 1 --2- 172.21.15.32:0/4150776710 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x555d21605300 0x555d216056e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80045a773b8f5a41 server_cookie=7fb8a312e6d3a7bf in_seq=0 out_seq=0 2024-04-17T10:03:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0b0c00640 10 client.9108 ms_handle_connect on v2:172.21.15.32:6835/3078068380 2024-04-17T10:03:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0b0c00640 1 -- 172.21.15.32:0/4150776710 <== mds.0 v2:172.21.15.32:6835/3078068380 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555d21576c40 con 0x555d21605300 2024-04-17T10:03:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0b0c00640 10 client.9108 handle_command_reply: tid=0 2024-04-17T10:03:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0cd3ab000 1 client.9108 shutdown 2024-04-17T10:03:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/4150776710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d21596b80 msgr2=0x555d215c19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0cd3ab000 1 --2- 172.21.15.32:0/4150776710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d21596b80 0x555d215c19c0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc0079e0 tx=0x7fe0bc001d00 comp rx=0 tx=0).stop 2024-04-17T10:03:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/4150776710 shutdown_connections 2024-04-17T10:03:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0cd3ab000 1 --2- 172.21.15.32:0/4150776710 >> [v2:172.21.15.32:6835/3078068380,v1:172.21.15.32:6837/3078068380] conn(0x555d21605300 0x555d216056e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0cd3ab000 1 --2- 172.21.15.32:0/4150776710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555d21596b80 0x555d215c19c0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0cd3ab000 1 --2- 172.21.15.32:0/4150776710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d21596260 0x555d215c1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0b0c00640 0 client.9108 ms_handle_reset on v2:172.21.15.32:6835/3078068380 2024-04-17T10:03:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0cd3ab000 1 --2- 172.21.15.32:0/4150776710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555d215499a0 0x555d215aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/4150776710 >> 172.21.15.32:0/4150776710 conn(0x555d21577000 msgr2=0x555d21594ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/4150776710 shutdown_connections 2024-04-17T10:03:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0cd3ab000 1 -- 172.21.15.32:0/4150776710 wait complete. 2024-04-17T10:03:20.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0cd3ab000 20 client.9108 trim_cache size 0 max 16384 2024-04-17T10:03:20.313 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:03:20.313 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2024-04-17T10:03:20.313 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2024-04-17T10:03:20.313 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:03:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0c9e00640 1 -- 172.21.15.32:0/3328393297 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0a00683b0 msgr2=0x7fe0a006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0c9e00640 1 --2- 172.21.15.32:0/3328393297 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0a00683b0 0x7fe0a006a870 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0af000 tx=0x7fe0ac0046d0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0c9e00640 1 -- 172.21.15.32:0/3328393297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a8003f90 msgr2=0x7fe0c4168300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0c9e00640 1 --2- 172.21.15.32:0/3328393297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a8003f90 0x7fe0c4168300 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4011a00 tx=0x7fe0b4011ec0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0c9e00640 1 -- 172.21.15.32:0/3328393297 shutdown_connections 2024-04-17T10:03:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0c9e00640 1 --2- 172.21.15.32:0/3328393297 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0a00683b0 0x7fe0a006a870 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0c9e00640 1 --2- 172.21.15.32:0/3328393297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0c4158760 0x7fe0c4158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0c9e00640 1 --2- 172.21.15.32:0/3328393297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0c4159100 0x7fe0c41668a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0c9e00640 1 --2- 172.21.15.32:0/3328393297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a8003f90 0x7fe0c4168300 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0c9e00640 1 -- 172.21.15.32:0/3328393297 >> 172.21.15.32:0/3328393297 conn(0x7fe0c40b4080 msgr2=0x7fe0c40b7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:20.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0c9e00640 1 -- 172.21.15.32:0/3328393297 shutdown_connections 2024-04-17T10:03:20.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.308+0000 7fe0c9e00640 1 -- 172.21.15.32:0/3328393297 wait complete. 2024-04-17T10:03:20.325 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T10:03:20.426 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T10:03:20.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.568+0000 7f8030c00640 1 Processor -- start 2024-04-17T10:03:20.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.568+0000 7f8030c00640 1 -- start start 2024-04-17T10:03:20.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.572+0000 7f8030c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f802c13f6e0 0x7f802c0cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.572+0000 7f8030c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f802c1601e0 con 0x7f802c0cf370 2024-04-17T10:03:20.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.572+0000 7f8030c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f802c160480 con 0x7f802c0d09f0 2024-04-17T10:03:20.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.572+0000 7f8030c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f802c13fac0 con 0x7f802c13f6e0 2024-04-17T10:03:20.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.572+0000 7f8029e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f802c13f6e0 0x7f802c0cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.572+0000 7f8023e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f802c0cf370 0x7f802c0d02e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53194/0 (socket says 172.21.15.32:53194) 2024-04-17T10:03:20.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.572+0000 7f8023e00640 1 -- 172.21.15.32:0/944823902 learned_addr learned my addr 172.21.15.32:0/944823902 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:20.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.572+0000 7f8029e00640 1 -- 172.21.15.32:0/944823902 >> v1:172.21.15.144:6789/0 conn(0x7f802c0d09f0 legacy=0x7f802c071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:20.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.572+0000 7f8029e00640 1 -- 172.21.15.32:0/944823902 >> v1:172.21.15.32:6789/0 conn(0x7f802c0cf370 legacy=0x7f802c0d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:20.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.572+0000 7f8029400640 1 -- 172.21.15.32:0/944823902 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1036592640 0 0) 0x7f802c1601e0 con 0x7f802c0cf370 2024-04-17T10:03:20.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.572+0000 7f8029e00640 1 -- 172.21.15.32:0/944823902 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802c160760 con 0x7f802c13f6e0 2024-04-17T10:03:20.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.576+0000 7f8029e00640 1 --2- 172.21.15.32:0/944823902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802c13f6e0 0x7f802c0cf850 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f801c0097c0 tx=0x7f801c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbdb96984b94630 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:20.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.576+0000 7f8029400640 1 -- 172.21.15.32:0/944823902 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c0a7070 con 0x7f802c13f6e0 2024-04-17T10:03:20.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.576+0000 7f8029400640 1 -- 172.21.15.32:0/944823902 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f801c0b0070 con 0x7f802c13f6e0 2024-04-17T10:03:20.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.576+0000 7f8029400640 1 -- 172.21.15.32:0/944823902 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c0ab890 con 0x7f802c13f6e0 2024-04-17T10:03:20.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.576+0000 7f8030c00640 1 -- 172.21.15.32:0/944823902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802c13f6e0 msgr2=0x7f802c0cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:20.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.576+0000 7f8030c00640 1 --2- 172.21.15.32:0/944823902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802c13f6e0 0x7f802c0cf850 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f801c0097c0 tx=0x7f801c01c470 comp rx=0 tx=0).stop 2024-04-17T10:03:20.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.576+0000 7f8030c00640 1 -- 172.21.15.32:0/944823902 shutdown_connections 2024-04-17T10:03:20.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.576+0000 7f8030c00640 1 --2- 172.21.15.32:0/944823902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802c13f6e0 0x7f802c0cf850 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.576+0000 7f8030c00640 1 -- 172.21.15.32:0/944823902 >> 172.21.15.32:0/944823902 conn(0x7f802c06ba30 msgr2=0x7f802c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:20.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.576+0000 7f8030c00640 1 -- 172.21.15.32:0/944823902 shutdown_connections 2024-04-17T10:03:20.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.576+0000 7f8030c00640 1 -- 172.21.15.32:0/944823902 wait complete. 2024-04-17T10:03:20.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.576+0000 7f8030c00640 1 Processor -- start 2024-04-17T10:03:20.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.576+0000 7f8030c00640 1 -- start start 2024-04-17T10:03:20.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f8030c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f802c0cf370 0x7f802c0a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f8030c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802c0d09f0 0x7f802c13ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f8030c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f802c13f6e0 0x7f802c0b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f8030c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f802c0cff10 con 0x7f802c0d09f0 2024-04-17T10:03:20.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f8030c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f802c0cfd90 con 0x7f802c0cf370 2024-04-17T10:03:20.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f8030c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f802c16a590 con 0x7f802c13f6e0 2024-04-17T10:03:20.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f8023e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802c0d09f0 0x7f802c13ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f8023e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802c0d09f0 0x7f802c13ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43038/0 (socket says 172.21.15.32:43038) 2024-04-17T10:03:20.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f8023e00640 1 -- 172.21.15.32:0/4260949940 learned_addr learned my addr 172.21.15.32:0/4260949940 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:20.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f8029e00640 1 --2- 172.21.15.32:0/4260949940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f802c0cf370 0x7f802c0a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f802a800640 1 --2- 172.21.15.32:0/4260949940 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f802c13f6e0 0x7f802c0b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f8023e00640 1 -- 172.21.15.32:0/4260949940 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f802c13f6e0 msgr2=0x7f802c0b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:20.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f8023e00640 1 --2- 172.21.15.32:0/4260949940 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f802c13f6e0 0x7f802c0b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f8023e00640 1 -- 172.21.15.32:0/4260949940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f802c0cf370 msgr2=0x7f802c0a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:20.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f8023e00640 1 --2- 172.21.15.32:0/4260949940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f802c0cf370 0x7f802c0a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f8023e00640 1 -- 172.21.15.32:0/4260949940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802c13f270 con 0x7f802c0d09f0 2024-04-17T10:03:20.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f8023e00640 1 --2- 172.21.15.32:0/4260949940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802c0d09f0 0x7f802c13ed30 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f801400c480 tx=0x7f801400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:20.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f8022000640 1 -- 172.21.15.32:0/4260949940 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014005cf0 con 0x7f802c0d09f0 2024-04-17T10:03:20.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f8022000640 1 -- 172.21.15.32:0/4260949940 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8014007070 con 0x7f802c0d09f0 2024-04-17T10:03:20.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f8022000640 1 -- 172.21.15.32:0/4260949940 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80140074a0 con 0x7f802c0d09f0 2024-04-17T10:03:20.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f8030c00640 1 -- 172.21.15.32:0/4260949940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f802c157f50 con 0x7f802c0d09f0 2024-04-17T10:03:20.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f8030c00640 1 -- 172.21.15.32:0/4260949940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f802c158510 con 0x7f802c0d09f0 2024-04-17T10:03:20.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f8030c00640 1 -- 172.21.15.32:0/4260949940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f802c068a90 con 0x7f802c0d09f0 2024-04-17T10:03:20.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.580+0000 7f8022000640 1 -- 172.21.15.32:0/4260949940 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f801401b400 con 0x7f802c0d09f0 2024-04-17T10:03:20.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.584+0000 7f8022000640 1 --2- 172.21.15.32:0/4260949940 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8008068380 0x7f800806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.584+0000 7f8022000640 1 -- 172.21.15.32:0/4260949940 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f8014086690 con 0x7f802c0d09f0 2024-04-17T10:03:20.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.584+0000 7f8029e00640 1 --2- 172.21.15.32:0/4260949940 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8008068380 0x7f800806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.584+0000 7f8029e00640 1 --2- 172.21.15.32:0/4260949940 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8008068380 0x7f800806a840 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f801c0af000 tx=0x7f801c0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:20.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.588+0000 7f8022000640 1 -- 172.21.15.32:0/4260949940 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f8014056c80 con 0x7f802c0d09f0 2024-04-17T10:03:20.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.784+0000 7f8030c00640 1 -- 172.21.15.32:0/4260949940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f802c0d1030 con 0x7f802c0d09f0 2024-04-17T10:03:20.783 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-17T10:03:20.784+0000 7f93c9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T10:03:20.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.788+0000 7f8022000640 1 -- 172.21.15.32:0/4260949940 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v88) ==== 161+0+0 (secure 0 0 0) 0x7f8014056580 con 0x7f802c0d09f0 2024-04-17T10:03:20.785 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T10:03:20.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.792+0000 7f8030c00640 1 -- 172.21.15.32:0/4260949940 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8008068380 msgr2=0x7f800806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:20.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.792+0000 7f8030c00640 1 --2- 172.21.15.32:0/4260949940 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8008068380 0x7f800806a840 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f801c0af000 tx=0x7f801c0a7000 comp rx=0 tx=0).stop 2024-04-17T10:03:20.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.792+0000 7f8030c00640 1 -- 172.21.15.32:0/4260949940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802c0d09f0 msgr2=0x7f802c13ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:20.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.792+0000 7f8030c00640 1 --2- 172.21.15.32:0/4260949940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802c0d09f0 0x7f802c13ed30 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f801400c480 tx=0x7f801400c940 comp rx=0 tx=0).stop 2024-04-17T10:03:20.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.792+0000 7f8030c00640 1 -- 172.21.15.32:0/4260949940 shutdown_connections 2024-04-17T10:03:20.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.792+0000 7f8030c00640 1 --2- 172.21.15.32:0/4260949940 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8008068380 0x7f800806a840 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.792+0000 7f8030c00640 1 --2- 172.21.15.32:0/4260949940 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f802c13f6e0 0x7f802c0b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.792+0000 7f8030c00640 1 --2- 172.21.15.32:0/4260949940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f802c0d09f0 0x7f802c13ed30 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.792+0000 7f8030c00640 1 --2- 172.21.15.32:0/4260949940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f802c0cf370 0x7f802c0a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.792+0000 7f8030c00640 1 -- 172.21.15.32:0/4260949940 >> 172.21.15.32:0/4260949940 conn(0x7f802c06ba30 msgr2=0x7f802c071d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:20.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.792+0000 7f8030c00640 1 -- 172.21.15.32:0/4260949940 shutdown_connections 2024-04-17T10:03:20.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.792+0000 7f8030c00640 1 -- 172.21.15.32:0/4260949940 wait complete. 2024-04-17T10:03:20.794 INFO:tasks.ceph.mds.c.smithi032.stderr: -1153> 2024-04-17T10:02:49.868+0000 7fd583400640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-17T10:03:20.794 INFO:tasks.ceph.mds.c.smithi032.stderr: -1150> 2024-04-17T10:02:49.868+0000 7fd583400640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-17T10:03:20.794 INFO:tasks.ceph.mds.c.smithi032.stderr: -1133> 2024-04-17T10:02:49.868+0000 7fd583400640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-17T10:03:20.794 INFO:tasks.ceph.mds.c.smithi032.stderr: -1130> 2024-04-17T10:02:49.868+0000 7fd583400640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-17T10:03:20.794 INFO:tasks.ceph.mds.c.smithi032.stderr: -1036> 2024-04-17T10:02:50.044+0000 7fd588400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:03:20.809 DEBUG:teuthology.orchestra.run.smithi032:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-04-17T10:03:20.836 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-17T10:03:20.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.856+0000 7f7254476400 1 -- start start 2024-04-17T10:03:20.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.856+0000 7f7254476400 1 Processor -- start 2024-04-17T10:03:20.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.856+0000 7f7254476400 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610b05b1800 0x5610b1380000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.856+0000 7f7254476400 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5610b05b1400 0x5610b1380580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.856+0000 7f7254476400 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610b05b1000 0x5610b1380b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.856+0000 7f7254476400 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5610b13a6600 con 0x5610b05b1800 2024-04-17T10:03:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.856+0000 7f7254476400 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5610b13a6480 con 0x5610b05b1000 2024-04-17T10:03:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.856+0000 7f7254476400 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5610b13a6300 con 0x5610b05b1400 2024-04-17T10:03:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.856+0000 7f7252600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610b05b1800 0x5610b1380000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.856+0000 7f7252600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610b05b1800 0x5610b1380000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43064/0 (socket says 172.21.15.32:43064) 2024-04-17T10:03:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.856+0000 7f7252600640 1 -- 172.21.15.32:0/3969029464 learned_addr learned my addr 172.21.15.32:0/3969029464 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.856+0000 7f7253000640 1 --2- 172.21.15.32:0/3969029464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610b05b1000 0x5610b1380b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.856+0000 7f7251c00640 1 --2- 172.21.15.32:0/3969029464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5610b05b1400 0x5610b1380580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.856+0000 7f7252600640 1 -- 172.21.15.32:0/3969029464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5610b05b1400 msgr2=0x5610b1380580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.856+0000 7f7252600640 1 --2- 172.21.15.32:0/3969029464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5610b05b1400 0x5610b1380580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.856+0000 7f7252600640 1 -- 172.21.15.32:0/3969029464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610b05b1000 msgr2=0x5610b1380b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.856+0000 7f7252600640 1 --2- 172.21.15.32:0/3969029464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610b05b1000 0x5610b1380b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.856+0000 7f7252600640 1 -- 172.21.15.32:0/3969029464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b137e960 con 0x5610b05b1800 2024-04-17T10:03:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.856+0000 7f7251c00640 1 --2- 172.21.15.32:0/3969029464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5610b05b1400 0x5610b1380580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7252600640 1 --2- 172.21.15.32:0/3969029464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610b05b1800 0x5610b1380000 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x5610b14331a0 tx=0x5610b068b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c88b2c973018c23 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:20.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f724f400640 1 -- 172.21.15.32:0/3969029464 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5610b05e64e0 con 0x5610b05b1800 2024-04-17T10:03:20.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f724f400640 1 -- 172.21.15.32:0/3969029464 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5610b05fc1c0 con 0x5610b05b1800 2024-04-17T10:03:20.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7254476400 1 -- 172.21.15.32:0/3969029464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610b137ed20 con 0x5610b05b1800 2024-04-17T10:03:20.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f724f400640 1 -- 172.21.15.32:0/3969029464 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5610b05e6b60 con 0x5610b05b1800 2024-04-17T10:03:20.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7254476400 1 -- 172.21.15.32:0/3969029464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5610b137ef00 con 0x5610b05b1800 2024-04-17T10:03:20.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7254476400 4 waiting for MDS map... 2024-04-17T10:03:20.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f724f400640 1 -- 172.21.15.32:0/3969029464 <== mon.0 v2:172.21.15.32:3300/0 4 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x5610b14ce600 con 0x5610b05b1800 2024-04-17T10:03:20.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f724f400640 1 -- 172.21.15.32:0/3969029464 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 88) ==== 2886+0+0 (secure 0 0 0) 0x5610b0607180 con 0x5610b05b1800 2024-04-17T10:03:20.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7254476400 4 Got MDS map 88 2024-04-17T10:03:20.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7254476400 10 main: main 2024-04-17T10:03:20.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7254476400 4 main: connecting to RADOS... 2024-04-17T10:03:20.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7254476400 1 Processor -- start 2024-04-17T10:03:20.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7254476400 1 -- start start 2024-04-17T10:03:20.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7254476400 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610b14d3000 0x5610b1381080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7254476400 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610b14d3400 0x5610b1381600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7254476400 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5610b14d3800 0x5610b1381b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7254476400 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5610b13a6180 con 0x5610b14d3400 2024-04-17T10:03:20.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7254476400 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5610b13a6900 con 0x5610b14d3000 2024-04-17T10:03:20.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7254476400 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5610b0610900 con 0x5610b14d3800 2024-04-17T10:03:20.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7252600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610b14d3400 0x5610b1381600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7252600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610b14d3400 0x5610b1381600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43068/0 (socket says 172.21.15.32:43068) 2024-04-17T10:03:20.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7252600640 1 -- 172.21.15.32:0/1316880358 learned_addr learned my addr 172.21.15.32:0/1316880358 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:20.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7253000640 1 --2- 172.21.15.32:0/1316880358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610b14d3000 0x5610b1381080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7252600640 1 -- 172.21.15.32:0/1316880358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5610b14d3800 msgr2=0x5610b1381b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:20.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7251c00640 1 --2- 172.21.15.32:0/1316880358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5610b14d3800 0x5610b1381b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7252600640 1 --2- 172.21.15.32:0/1316880358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5610b14d3800 0x5610b1381b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7252600640 1 -- 172.21.15.32:0/1316880358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610b14d3000 msgr2=0x5610b1381080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:20.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7252600640 1 --2- 172.21.15.32:0/1316880358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610b14d3000 0x5610b1381080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7252600640 1 -- 172.21.15.32:0/1316880358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b137ed20 con 0x5610b14d3400 2024-04-17T10:03:20.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7253000640 1 --2- 172.21.15.32:0/1316880358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610b14d3000 0x5610b1381080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:03:20.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7251c00640 1 --2- 172.21.15.32:0/1316880358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5610b14d3800 0x5610b1381b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:03:20.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f7252600640 1 --2- 172.21.15.32:0/1316880358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610b14d3400 0x5610b1381600 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x5610b1538030 tx=0x5610b1338ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570c442d70705d8a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:20.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f724cc00640 1 -- 172.21.15.32:0/1316880358 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5610b05e6340 con 0x5610b14d3400 2024-04-17T10:03:20.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f724cc00640 1 -- 172.21.15.32:0/1316880358 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5610b05fce00 con 0x5610b14d3400 2024-04-17T10:03:20.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.860+0000 7f724cc00640 1 -- 172.21.15.32:0/1316880358 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5610b05e64e0 con 0x5610b14d3400 2024-04-17T10:03:20.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7254476400 1 -- 172.21.15.32:0/1316880358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610b14d3400 msgr2=0x5610b1381600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7254476400 1 --2- 172.21.15.32:0/1316880358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610b14d3400 0x5610b1381600 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x5610b1538030 tx=0x5610b1338ae0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7254476400 1 -- 172.21.15.32:0/1316880358 shutdown_connections 2024-04-17T10:03:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7254476400 1 --2- 172.21.15.32:0/1316880358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5610b14d3800 0x5610b1381b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7254476400 1 --2- 172.21.15.32:0/1316880358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610b14d3400 0x5610b1381600 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7254476400 1 --2- 172.21.15.32:0/1316880358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610b14d3000 0x5610b1381080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7254476400 1 -- 172.21.15.32:0/1316880358 >> 172.21.15.32:0/1316880358 conn(0x5610b05b0c00 msgr2=0x5610b05df800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7254476400 1 -- 172.21.15.32:0/1316880358 shutdown_connections 2024-04-17T10:03:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7254476400 1 -- 172.21.15.32:0/1316880358 wait complete. 2024-04-17T10:03:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7254476400 1 Processor -- start 2024-04-17T10:03:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7254476400 1 -- start start 2024-04-17T10:03:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7254476400 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610b14d3000 0x5610b1381080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7254476400 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5610b14d3400 0x5610b1381600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7254476400 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610b14d3800 0x5610b1381b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7254476400 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5610b13a7680 con 0x5610b14d3000 2024-04-17T10:03:20.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7254476400 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5610b13a7800 con 0x5610b14d3800 2024-04-17T10:03:20.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7254476400 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5610b13a7500 con 0x5610b14d3400 2024-04-17T10:03:20.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7253000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610b14d3000 0x5610b1381080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7253000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610b14d3000 0x5610b1381080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43070/0 (socket says 172.21.15.32:43070) 2024-04-17T10:03:20.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7253000640 1 -- 172.21.15.32:0/1428653813 learned_addr learned my addr 172.21.15.32:0/1428653813 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:20.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7251c00640 1 --2- 172.21.15.32:0/1428653813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610b14d3800 0x5610b1381b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7252600640 1 --2- 172.21.15.32:0/1428653813 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5610b14d3400 0x5610b1381600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7253000640 1 -- 172.21.15.32:0/1428653813 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5610b14d3400 msgr2=0x5610b1381600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:20.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7253000640 1 --2- 172.21.15.32:0/1428653813 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5610b14d3400 0x5610b1381600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7253000640 1 -- 172.21.15.32:0/1428653813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610b14d3800 msgr2=0x5610b1381b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:20.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7253000640 1 --2- 172.21.15.32:0/1428653813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610b14d3800 0x5610b1381b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7253000640 1 -- 172.21.15.32:0/1428653813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b137fc20 con 0x5610b14d3000 2024-04-17T10:03:20.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7253000640 1 --2- 172.21.15.32:0/1428653813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610b14d3000 0x5610b1381080 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x5610b14329f0 tx=0x5610b1338a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:20.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f724ae00640 1 -- 172.21.15.32:0/1428653813 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5610b05e69c0 con 0x5610b14d3000 2024-04-17T10:03:20.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f724ae00640 1 -- 172.21.15.32:0/1428653813 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5610b05fca80 con 0x5610b14d3000 2024-04-17T10:03:20.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7254476400 1 -- 172.21.15.32:0/1428653813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5610b137fe00 con 0x5610b14d3000 2024-04-17T10:03:20.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f724ae00640 1 -- 172.21.15.32:0/1428653813 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5610b05e6340 con 0x5610b14d3000 2024-04-17T10:03:20.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7254476400 1 -- 172.21.15.32:0/1428653813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610b137e1e0 con 0x5610b14d3000 2024-04-17T10:03:20.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.864+0000 7f7254476400 4 main: resolving pool 14 2024-04-17T10:03:20.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.868+0000 7f724ae00640 1 -- 172.21.15.32:0/1428653813 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x5610b0898c00 con 0x5610b14d3000 2024-04-17T10:03:20.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.868+0000 7f724ae00640 1 --2- 172.21.15.32:0/1428653813 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x5610b0897000 0x5610b1382100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.868+0000 7f724ae00640 1 -- 172.21.15.32:0/1428653813 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x5610b08a6600 con 0x5610b14d3000 2024-04-17T10:03:20.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.868+0000 7f7254476400 4 main: creating IoCtx.. 2024-04-17T10:03:20.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.868+0000 7f7254476400 1 --2- 172.21.15.32:0/1428653813 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x5610b14d3c00 0x5610b1382680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:20.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.868+0000 7f7254476400 1 -- 172.21.15.32:0/1428653813 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) -- 0x5610b0897400 con 0x5610b14d3c00 2024-04-17T10:03:20.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.868+0000 7f7252600640 1 --2- 172.21.15.32:0/1428653813 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x5610b0897000 0x5610b1382100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.868+0000 7f7251c00640 1 --2- 172.21.15.32:0/1428653813 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x5610b14d3c00 0x5610b1382680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:20.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.868+0000 7f7252600640 1 --2- 172.21.15.32:0/1428653813 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x5610b0897000 0x5610b1382100 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x5610b1538030 tx=0x5610b14e0c00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:20.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.868+0000 7f7251c00640 1 --2- 172.21.15.32:0/1428653813 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x5610b14d3c00 0x5610b1382680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:20.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.872+0000 7f7251c00640 1 -- 172.21.15.32:0/1428653813 <== osd.2 v2:172.21.15.32:6809/1954833462 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x5610b1334000 con 0x5610b14d3c00 2024-04-17T10:03:20.870 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:03:20.870 INFO:teuthology.orchestra.run.smithi032.stdout: "0": { 2024-04-17T10:03:20.870 INFO:teuthology.orchestra.run.smithi032.stdout: "data": { 2024-04-17T10:03:20.870 INFO:teuthology.orchestra.run.smithi032.stdout: "version": 5, 2024-04-17T10:03:20.870 INFO:teuthology.orchestra.run.smithi032.stdout: "inotable": { 2024-04-17T10:03:20.870 INFO:teuthology.orchestra.run.smithi032.stdout: "projected_free": [ 2024-04-17T10:03:20.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:03:20.870 INFO:teuthology.orchestra.run.smithi032.stdout: "start": 1099511627779, 2024-04-17T10:03:20.870 INFO:teuthology.orchestra.run.smithi032.stdout: "len": 1099511627773 2024-04-17T10:03:20.870 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-04-17T10:03:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: "free": [ 2024-04-17T10:03:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:03:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: "start": 1099511627779, 2024-04-17T10:03:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: "len": 1099511627773 2024-04-17T10:03:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-17T10:03:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: "result": 0 2024-04-17T10:03:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:20.871 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:03:20.871 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:03:20.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.872+0000 7f7254476400 1 -- 172.21.15.32:0/1428653813 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x5610b14d3c00 msgr2=0x5610b1382680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.872+0000 7f7254476400 1 --2- 172.21.15.32:0/1428653813 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x5610b14d3c00 0x5610b1382680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.872+0000 7f7254476400 1 -- 172.21.15.32:0/1428653813 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x5610b0897000 msgr2=0x5610b1382100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.872+0000 7f7254476400 1 --2- 172.21.15.32:0/1428653813 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x5610b0897000 0x5610b1382100 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x5610b1538030 tx=0x5610b14e0c00 comp rx=0 tx=0).stop 2024-04-17T10:03:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.872+0000 7f7254476400 1 -- 172.21.15.32:0/1428653813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610b14d3000 msgr2=0x5610b1381080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.872+0000 7f7254476400 1 --2- 172.21.15.32:0/1428653813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610b14d3000 0x5610b1381080 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x5610b14329f0 tx=0x5610b1338a80 comp rx=0 tx=0).stop 2024-04-17T10:03:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.872+0000 7f7251c00640 1 -- 172.21.15.32:0/1428653813 reap_dead start 2024-04-17T10:03:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.872+0000 7f7254476400 1 -- 172.21.15.32:0/1428653813 shutdown_connections 2024-04-17T10:03:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.872+0000 7f7254476400 1 -- 172.21.15.32:0/1428653813 >> 172.21.15.32:0/1428653813 conn(0x5610b05b0c00 msgr2=0x5610b05df800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.872+0000 7f7254476400 1 -- 172.21.15.32:0/1428653813 shutdown_connections 2024-04-17T10:03:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.872+0000 7f7254476400 1 -- 172.21.15.32:0/1428653813 wait complete. 2024-04-17T10:03:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.872+0000 7f7254476400 1 -- 172.21.15.32:0/3969029464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610b05b1800 msgr2=0x5610b1380000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.872+0000 7f7254476400 1 --2- 172.21.15.32:0/3969029464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610b05b1800 0x5610b1380000 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x5610b14331a0 tx=0x5610b068b500 comp rx=0 tx=0).stop 2024-04-17T10:03:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.872+0000 7f7254476400 1 -- 172.21.15.32:0/3969029464 shutdown_connections 2024-04-17T10:03:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.872+0000 7f7254476400 1 --2- 172.21.15.32:0/3969029464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5610b05b1400 0x5610b1380580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.872+0000 7f7254476400 1 --2- 172.21.15.32:0/3969029464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610b05b1000 0x5610b1380b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.872+0000 7f7254476400 1 --2- 172.21.15.32:0/3969029464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5610b05b1800 0x5610b1380000 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.872+0000 7f7254476400 1 -- 172.21.15.32:0/3969029464 >> 172.21.15.32:0/3969029464 conn(0x5610b05b1c00 msgr2=0x5610b05df000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.872+0000 7f7254476400 1 -- 172.21.15.32:0/3969029464 shutdown_connections 2024-04-17T10:03:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:20.872+0000 7f7254476400 1 -- 172.21.15.32:0/3969029464 wait complete. 2024-04-17T10:03:20.875 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.065982, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-04-17T10:03:20.875 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T10:03:20.875 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:03:21.034 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:03:21.126 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T10:03:21.126 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T10:03:21.126 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:03:21.126 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:03:21.135 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:03:21.136 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T10:03:21.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:21.136 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T10:03:21.187 INFO:teuthology.orchestra.run.smithi032.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-17T10:03:21.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:03:21.188 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T10:03:21.188 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T10:03:21.188 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:03:21.197 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:03:21.197 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T10:03:21.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:21.197 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-17T10:03:21.246 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-17T10:03:21.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:03:21.247 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T10:03:21.248 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:03:21.248 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:03:21.256 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:03:21.256 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T10:03:21.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:21.257 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-17T10:03:21.307 INFO:teuthology.orchestra.run.smithi032.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-17T10:03:21.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:03:21.308 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T10:03:21.308 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T10:03:21.308 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:03:21.316 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:03:21.316 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T10:03:21.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:21.316 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-17T10:03:21.370 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-17T10:03:21.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:03:21.372 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:03:21.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.516+0000 7fc3f9e00640 1 Processor -- start 2024-04-17T10:03:21.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.516+0000 7fc3f9e00640 1 -- start start 2024-04-17T10:03:21.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.516+0000 7fc3f9e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3f415baa0 0x7fc3f415bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:21.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.516+0000 7fc3f9e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc3f406ef60 0x7fc3f406f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:21.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.516+0000 7fc3f9e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3f4072bb0 con 0x7fc3f4144e90 2024-04-17T10:03:21.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.516+0000 7fc3f9e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc3f415c440 con 0x7fc3f406ef60 2024-04-17T10:03:21.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.516+0000 7fc3f9e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc3f406f8f0 con 0x7fc3f415baa0 2024-04-17T10:03:21.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.516+0000 7fc3f2a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc3f4144e90 0x7fc3f40b48a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53206/0 (socket says 172.21.15.32:53206) 2024-04-17T10:03:21.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.516+0000 7fc3f2a00640 1 -- 172.21.15.32:0/3690435693 learned_addr learned my addr 172.21.15.32:0/3690435693 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:21.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.520+0000 7fc3f3400640 1 --2- 172.21.15.32:0/3690435693 >> v2:172.21.15.32:3300/0 conn(0x7fc3f406ef60 0x7fc3f406f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:21.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.520+0000 7fc3ea000640 1 --2- 172.21.15.32:0/3690435693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3f415baa0 0x7fc3f415bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:21.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.520+0000 7fc3f2000640 1 -- 172.21.15.32:0/3690435693 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 648361470 0 0) 0x7fc3f4072bb0 con 0x7fc3f4144e90 2024-04-17T10:03:21.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.520+0000 7fc3f2000640 1 -- 172.21.15.32:0/3690435693 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3d8003530 con 0x7fc3f4144e90 2024-04-17T10:03:21.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.520+0000 7fc3f3400640 1 -- 172.21.15.32:0/3690435693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3f415baa0 msgr2=0x7fc3f415bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:21.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.520+0000 7fc3f3400640 1 --2- 172.21.15.32:0/3690435693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3f415baa0 0x7fc3f415bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:21.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.520+0000 7fc3f3400640 1 -- 172.21.15.32:0/3690435693 >> v1:172.21.15.32:6789/0 conn(0x7fc3f4144e90 legacy=0x7fc3f40b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:21.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.520+0000 7fc3f3400640 1 -- 172.21.15.32:0/3690435693 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f40b3dc0 con 0x7fc3f406ef60 2024-04-17T10:03:21.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.520+0000 7fc3f3400640 1 --2- 172.21.15.32:0/3690435693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3f406ef60 0x7fc3f406f3b0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec009d90 tx=0x7fc3ec01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563983aedb03fca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:21.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.520+0000 7fc3f2000640 1 -- 172.21.15.32:0/3690435693 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0b2070 con 0x7fc3f406ef60 2024-04-17T10:03:21.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.524+0000 7fc3f2000640 1 -- 172.21.15.32:0/3690435693 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3ec0a58a0 con 0x7fc3f406ef60 2024-04-17T10:03:21.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.524+0000 7fc3f2000640 1 -- 172.21.15.32:0/3690435693 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0a5c50 con 0x7fc3f406ef60 2024-04-17T10:03:21.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.524+0000 7fc3f9e00640 1 -- 172.21.15.32:0/3690435693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3f406ef60 msgr2=0x7fc3f406f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:21.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.524+0000 7fc3f9e00640 1 --2- 172.21.15.32:0/3690435693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3f406ef60 0x7fc3f406f3b0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec009d90 tx=0x7fc3ec01c930 comp rx=0 tx=0).stop 2024-04-17T10:03:21.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.524+0000 7fc3f9e00640 1 -- 172.21.15.32:0/3690435693 shutdown_connections 2024-04-17T10:03:21.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.524+0000 7fc3f9e00640 1 --2- 172.21.15.32:0/3690435693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3f406ef60 0x7fc3f406f3b0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:21.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.524+0000 7fc3f9e00640 1 --2- 172.21.15.32:0/3690435693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3f415baa0 0x7fc3f415bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:21.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.524+0000 7fc3f9e00640 1 -- 172.21.15.32:0/3690435693 >> 172.21.15.32:0/3690435693 conn(0x7fc3f406ba30 msgr2=0x7fc3f406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:21.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.524+0000 7fc3f9e00640 1 -- 172.21.15.32:0/3690435693 shutdown_connections 2024-04-17T10:03:21.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.524+0000 7fc3f9e00640 1 -- 172.21.15.32:0/3690435693 wait complete. 2024-04-17T10:03:21.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.524+0000 7fc3f9e00640 1 Processor -- start 2024-04-17T10:03:21.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.524+0000 7fc3f9e00640 1 -- start start 2024-04-17T10:03:21.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.524+0000 7fc3f9e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3f406ef60 0x7fc3f415a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:21.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.524+0000 7fc3f9e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3f4144e90 0x7fc3f40aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:21.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.524+0000 7fc3f9e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3f415baa0 0x7fc3f415c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:21.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.524+0000 7fc3f9e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f40727f0 con 0x7fc3f406ef60 2024-04-17T10:03:21.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.524+0000 7fc3f9e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f4072670 con 0x7fc3f4144e90 2024-04-17T10:03:21.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.524+0000 7fc3f9e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc3f4072970 con 0x7fc3f415baa0 2024-04-17T10:03:21.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.524+0000 7fc3f2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3f406ef60 0x7fc3f415a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:21.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.524+0000 7fc3f2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3f406ef60 0x7fc3f415a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43086/0 (socket says 172.21.15.32:43086) 2024-04-17T10:03:21.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.524+0000 7fc3f2a00640 1 -- 172.21.15.32:0/3302093666 learned_addr learned my addr 172.21.15.32:0/3302093666 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:21.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.524+0000 7fc3ea000640 1 --2- 172.21.15.32:0/3302093666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3f4144e90 0x7fc3f40aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:21.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.524+0000 7fc3f3400640 1 --2- 172.21.15.32:0/3302093666 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3f415baa0 0x7fc3f415c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:21.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.528+0000 7fc3f2a00640 1 -- 172.21.15.32:0/3302093666 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3f415baa0 msgr2=0x7fc3f415c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:21.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.528+0000 7fc3f2a00640 1 --2- 172.21.15.32:0/3302093666 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3f415baa0 0x7fc3f415c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:21.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.528+0000 7fc3f2a00640 1 -- 172.21.15.32:0/3302093666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3f4144e90 msgr2=0x7fc3f40aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:21.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.528+0000 7fc3f2a00640 1 --2- 172.21.15.32:0/3302093666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3f4144e90 0x7fc3f40aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:21.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.528+0000 7fc3f2a00640 1 -- 172.21.15.32:0/3302093666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f415b160 con 0x7fc3f406ef60 2024-04-17T10:03:21.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.528+0000 7fc3ea000640 1 --2- 172.21.15.32:0/3302093666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3f4144e90 0x7fc3f40aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:21.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.528+0000 7fc3f2a00640 1 --2- 172.21.15.32:0/3302093666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3f406ef60 0x7fc3f415a350 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fc3e400c550 tx=0x7fc3e400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:21.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.528+0000 7fc3ebe00640 1 -- 172.21.15.32:0/3302093666 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4018070 con 0x7fc3f406ef60 2024-04-17T10:03:21.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.528+0000 7fc3ebe00640 1 -- 172.21.15.32:0/3302093666 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3e40074e0 con 0x7fc3f406ef60 2024-04-17T10:03:21.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.528+0000 7fc3ebe00640 1 -- 172.21.15.32:0/3302093666 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4004a30 con 0x7fc3f406ef60 2024-04-17T10:03:21.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.528+0000 7fc3f9e00640 1 -- 172.21.15.32:0/3302093666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f415a9d0 con 0x7fc3f406ef60 2024-04-17T10:03:21.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.528+0000 7fc3f9e00640 1 -- 172.21.15.32:0/3302093666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f40ae680 con 0x7fc3f406ef60 2024-04-17T10:03:21.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.528+0000 7fc3f9e00640 1 -- 172.21.15.32:0/3302093666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f40ae880 con 0x7fc3f406ef60 2024-04-17T10:03:21.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.528+0000 7fc3ebe00640 1 -- 172.21.15.32:0/3302093666 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc3e4004bd0 con 0x7fc3f406ef60 2024-04-17T10:03:21.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.528+0000 7fc3ebe00640 1 --2- 172.21.15.32:0/3302093666 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc3d00683b0 0x7fc3d006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:21.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.528+0000 7fc3ebe00640 1 -- 172.21.15.32:0/3302093666 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x7fc3e4085cf0 con 0x7fc3f406ef60 2024-04-17T10:03:21.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.528+0000 7fc3ea000640 1 --2- 172.21.15.32:0/3302093666 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc3d00683b0 0x7fc3d006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:21.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.532+0000 7fc3ea000640 1 --2- 172.21.15.32:0/3302093666 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc3d00683b0 0x7fc3d006a870 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40b5520 tx=0x7fc3dc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:21.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.532+0000 7fc3ebe00640 1 -- 172.21.15.32:0/3302093666 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc3f40ae880 con 0x7fc3f406ef60 2024-04-17T10:03:21.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.732+0000 7fc3f9e00640 1 -- 172.21.15.32:0/3302093666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3f4068a90 con 0x7fc3f406ef60 2024-04-17T10:03:21.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.732+0000 7fc3ebe00640 1 -- 172.21.15.32:0/3302093666 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+4368 (secure 0 0 0) 0x7fc3e4056e20 con 0x7fc3f406ef60 2024-04-17T10:03:21.730 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:03:21.730 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":88,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2635109588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2635109588},{"type":"v1","addr":"172.21.15.144:6835","nonce":2635109588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7789,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3258455939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3258455939},{"type":"v1","addr":"172.21.15.144:6834","nonce":3258455939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":8574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3390468332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3390468332},{"type":"v1","addr":"172.21.15.32:6836","nonce":3390468332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":88,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:01:40.557208+0000","modified":"2024-04-17T10:03:20.790439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-17T10:03:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 88 2024-04-17T10:03:21.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.736+0000 7fc3f9e00640 1 -- 172.21.15.32:0/3302093666 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc3d00683b0 msgr2=0x7fc3d006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:21.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.736+0000 7fc3f9e00640 1 --2- 172.21.15.32:0/3302093666 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc3d00683b0 0x7fc3d006a870 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40b5520 tx=0x7fc3dc008040 comp rx=0 tx=0).stop 2024-04-17T10:03:21.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.736+0000 7fc3f9e00640 1 -- 172.21.15.32:0/3302093666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3f406ef60 msgr2=0x7fc3f415a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:21.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.736+0000 7fc3f9e00640 1 --2- 172.21.15.32:0/3302093666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3f406ef60 0x7fc3f415a350 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fc3e400c550 tx=0x7fc3e400ca10 comp rx=0 tx=0).stop 2024-04-17T10:03:21.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.736+0000 7fc3f9e00640 1 -- 172.21.15.32:0/3302093666 shutdown_connections 2024-04-17T10:03:21.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.736+0000 7fc3f9e00640 1 --2- 172.21.15.32:0/3302093666 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc3d00683b0 0x7fc3d006a870 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:21.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.736+0000 7fc3f9e00640 1 --2- 172.21.15.32:0/3302093666 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3f415baa0 0x7fc3f415c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:21.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.736+0000 7fc3f9e00640 1 --2- 172.21.15.32:0/3302093666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3f4144e90 0x7fc3f40aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:21.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.736+0000 7fc3f9e00640 1 --2- 172.21.15.32:0/3302093666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3f406ef60 0x7fc3f415a350 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:21.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.736+0000 7fc3f9e00640 1 -- 172.21.15.32:0/3302093666 >> 172.21.15.32:0/3302093666 conn(0x7fc3f406ba30 msgr2=0x7fc3f406be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:21.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.736+0000 7fc3f9e00640 1 -- 172.21.15.32:0/3302093666 shutdown_connections 2024-04-17T10:03:21.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.736+0000 7fc3f9e00640 1 -- 172.21.15.32:0/3302093666 wait complete. 2024-04-17T10:03:21.753 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:03:21.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.896+0000 7fe317600640 1 Processor -- start 2024-04-17T10:03:21.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.896+0000 7fe317600640 1 -- start start 2024-04-17T10:03:21.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe317600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe310072500 0x7fe3100cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:21.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe317600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3100cdac0 con 0x7fe31015d8d0 2024-04-17T10:03:21.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe317600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe31015cdf0 con 0x7fe310145f30 2024-04-17T10:03:21.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe317600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe31006f400 con 0x7fe310072500 2024-04-17T10:03:21.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe315600640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe31015d8d0 0x7fe3101601d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53208/0 (socket says 172.21.15.32:53208) 2024-04-17T10:03:21.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe315600640 1 -- 172.21.15.32:0/2442218329 learned_addr learned my addr 172.21.15.32:0/2442218329 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:21.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe30fe00640 1 --2- 172.21.15.32:0/2442218329 >> v2:172.21.15.144:3300/0 conn(0x7fe310072500 0x7fe3100cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:21.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe30f400640 1 -- 172.21.15.32:0/2442218329 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1041249434 0 0) 0x7fe3100cdac0 con 0x7fe31015d8d0 2024-04-17T10:03:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe30f400640 1 -- 172.21.15.32:0/2442218329 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2f8003530 con 0x7fe31015d8d0 2024-04-17T10:03:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe30f400640 1 -- 172.21.15.32:0/2442218329 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3901864393 0 0) 0x7fe2f8003530 con 0x7fe31015d8d0 2024-04-17T10:03:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe30f400640 1 -- 172.21.15.32:0/2442218329 >> v2:172.21.15.144:3300/0 conn(0x7fe310072500 msgr2=0x7fe3100cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe30f400640 1 --2- 172.21.15.32:0/2442218329 >> v2:172.21.15.144:3300/0 conn(0x7fe310072500 0x7fe3100cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe30f400640 1 -- 172.21.15.32:0/2442218329 >> v1:172.21.15.144:6789/0 conn(0x7fe310145f30 legacy=0x7fe310071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe30f400640 1 -- 172.21.15.32:0/2442218329 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31015d0d0 con 0x7fe31015d8d0 2024-04-17T10:03:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe30f400640 1 -- 172.21.15.32:0/2442218329 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7fe308001a00 con 0x7fe31015d8d0 2024-04-17T10:03:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe30f400640 1 -- 172.21.15.32:0/2442218329 >> v1:172.21.15.32:6789/0 conn(0x7fe31015d8d0 legacy=0x7fe3101601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe30f400640 1 --2- 172.21.15.32:0/2442218329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2f80039c0 0x7fe2f8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe30f400640 1 --2- 172.21.15.32:0/2442218329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2f8004810 0x7fe2f8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe30f400640 1 --2- 172.21.15.32:0/2442218329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f80051d0 0x7fe2f800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe30f400640 1 -- 172.21.15.32:0/2442218329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe31006f400 con 0x7fe2f80051d0 2024-04-17T10:03:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe30f400640 1 -- 172.21.15.32:0/2442218329 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f800c080 con 0x7fe2f80039c0 2024-04-17T10:03:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe30f400640 1 -- 172.21.15.32:0/2442218329 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe2f800c280 con 0x7fe2f8004810 2024-04-17T10:03:21.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe314c00640 1 --2- 172.21.15.32:0/2442218329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2f80039c0 0x7fe2f8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:21.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe315600640 1 --2- 172.21.15.32:0/2442218329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f80051d0 0x7fe2f800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:21.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe30fe00640 1 --2- 172.21.15.32:0/2442218329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2f8004810 0x7fe2f8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:21.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe315600640 1 -- 172.21.15.32:0/2442218329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2f8004810 msgr2=0x7fe2f8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:21.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe315600640 1 --2- 172.21.15.32:0/2442218329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2f8004810 0x7fe2f8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:21.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe315600640 1 -- 172.21.15.32:0/2442218329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2f80039c0 msgr2=0x7fe2f8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:21.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe315600640 1 --2- 172.21.15.32:0/2442218329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2f80039c0 0x7fe2f8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:21.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.900+0000 7fe315600640 1 -- 172.21.15.32:0/2442218329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f8003530 con 0x7fe2f80051d0 2024-04-17T10:03:21.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.904+0000 7fe315600640 1 -- 172.21.15.32:0/2442218329 reap_dead start 2024-04-17T10:03:21.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.904+0000 7fe315600640 1 --2- 172.21.15.32:0/2442218329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f80051d0 0x7fe2f800b9d0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fe30800a880 tx=0x7fe30801d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79cb7627d24864bd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:21.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.904+0000 7fe30f400640 1 -- 172.21.15.32:0/2442218329 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30800a4a0 con 0x7fe2f80051d0 2024-04-17T10:03:21.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.904+0000 7fe30f400640 1 -- 172.21.15.32:0/2442218329 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3080ad650 con 0x7fe2f80051d0 2024-04-17T10:03:21.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.904+0000 7fe30f400640 1 -- 172.21.15.32:0/2442218329 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3080ada40 con 0x7fe2f80051d0 2024-04-17T10:03:21.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.904+0000 7fe317600640 1 -- 172.21.15.32:0/2442218329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f80051d0 msgr2=0x7fe2f800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:21.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.904+0000 7fe317600640 1 --2- 172.21.15.32:0/2442218329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f80051d0 0x7fe2f800b9d0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fe30800a880 tx=0x7fe30801d3a0 comp rx=0 tx=0).stop 2024-04-17T10:03:21.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.904+0000 7fe317600640 1 -- 172.21.15.32:0/2442218329 shutdown_connections 2024-04-17T10:03:21.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.904+0000 7fe317600640 1 --2- 172.21.15.32:0/2442218329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f80051d0 0x7fe2f800b9d0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:21.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.904+0000 7fe317600640 1 -- 172.21.15.32:0/2442218329 >> 172.21.15.32:0/2442218329 conn(0x7fe31006ba30 msgr2=0x7fe31006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:21.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.904+0000 7fe317600640 1 -- 172.21.15.32:0/2442218329 shutdown_connections 2024-04-17T10:03:21.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.904+0000 7fe317600640 1 -- 172.21.15.32:0/2442218329 wait complete. 2024-04-17T10:03:21.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe317600640 1 Processor -- start 2024-04-17T10:03:21.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe317600640 1 -- start start 2024-04-17T10:03:21.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe317600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2f80051d0 0x7fe31015bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:21.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe317600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3100abde0 0x7fe3100b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:21.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe317600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3100ac240 0x7fe3100b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:21.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe317600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe310072180 con 0x7fe3100abde0 2024-04-17T10:03:21.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe317600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe310072000 con 0x7fe3100ac240 2024-04-17T10:03:21.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe317600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe310072300 con 0x7fe2f80051d0 2024-04-17T10:03:21.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe30fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3100abde0 0x7fe3100b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:21.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe30fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3100abde0 0x7fe3100b3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43102/0 (socket says 172.21.15.32:43102) 2024-04-17T10:03:21.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe30fe00640 1 -- 172.21.15.32:0/1714996428 learned_addr learned my addr 172.21.15.32:0/1714996428 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:21.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe314c00640 1 --2- 172.21.15.32:0/1714996428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2f80051d0 0x7fe31015bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:21.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe315600640 1 --2- 172.21.15.32:0/1714996428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3100ac240 0x7fe3100b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:21.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe30fe00640 1 -- 172.21.15.32:0/1714996428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2f80051d0 msgr2=0x7fe31015bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:21.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe30fe00640 1 --2- 172.21.15.32:0/1714996428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2f80051d0 0x7fe31015bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:21.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe30fe00640 1 -- 172.21.15.32:0/1714996428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3100ac240 msgr2=0x7fe3100b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:21.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe30fe00640 1 --2- 172.21.15.32:0/1714996428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3100ac240 0x7fe3100b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:21.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe30fe00640 1 -- 172.21.15.32:0/1714996428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3101384b0 con 0x7fe3100abde0 2024-04-17T10:03:21.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe30fe00640 1 --2- 172.21.15.32:0/1714996428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3100abde0 0x7fe3100b3cb0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc011990 tx=0x7fe2fc011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:21.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe30cc00640 1 -- 172.21.15.32:0/1714996428 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc00fb30 con 0x7fe3100abde0 2024-04-17T10:03:21.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe30cc00640 1 -- 172.21.15.32:0/1714996428 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2fc00fcd0 con 0x7fe3100abde0 2024-04-17T10:03:21.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe317600640 1 -- 172.21.15.32:0/1714996428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe310138790 con 0x7fe3100abde0 2024-04-17T10:03:21.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe30cc00640 1 -- 172.21.15.32:0/1714996428 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc014660 con 0x7fe3100abde0 2024-04-17T10:03:21.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe317600640 1 -- 172.21.15.32:0/1714996428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe310138d50 con 0x7fe3100abde0 2024-04-17T10:03:21.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.908+0000 7fe317600640 1 -- 172.21.15.32:0/1714996428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe310068a90 con 0x7fe3100abde0 2024-04-17T10:03:21.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.912+0000 7fe30cc00640 1 -- 172.21.15.32:0/1714996428 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe2fc012040 con 0x7fe3100abde0 2024-04-17T10:03:21.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.912+0000 7fe30cc00640 1 --2- 172.21.15.32:0/1714996428 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe2f0068380 0x7fe2f006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:21.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.912+0000 7fe30cc00640 1 -- 172.21.15.32:0/1714996428 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x7fe2fc087080 con 0x7fe3100abde0 2024-04-17T10:03:21.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.912+0000 7fe314c00640 1 --2- 172.21.15.32:0/1714996428 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe2f0068380 0x7fe2f006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:21.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.912+0000 7fe314c00640 1 --2- 172.21.15.32:0/1714996428 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe2f0068380 0x7fe2f006a840 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe304000c10 tx=0x7fe304009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:21.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:21.916+0000 7fe30cc00640 1 -- 172.21.15.32:0/1714996428 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fe2fc019030 con 0x7fe3100abde0 2024-04-17T10:03:22.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.080+0000 7fe317600640 1 -- 172.21.15.32:0/1714996428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe310072f90 con 0x7fe3100abde0 2024-04-17T10:03:22.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.080+0000 7fe30cc00640 1 -- 172.21.15.32:0/1714996428 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16879 (secure 0 0 0) 0x7fe2fc056e10 con 0x7fe3100abde0 2024-04-17T10:03:22.079 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:03:22.079 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":99,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:03:20.790225+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:01:38.938460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-17T10:01:39.505171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6835/3078068380":"2024-04-18T10:03:20.790196+0000","172.21.15.32:6837/3078068380":"2024-04-18T10:03:20.790196+0000","172.21.15.32:6837/1235126741":"2024-04-18T10:02:44.035918+0000","172.21.15.32:6835/1235126741":"2024-04-18T10:02:44.035918+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:03:22.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.084+0000 7fe317600640 1 -- 172.21.15.32:0/1714996428 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe2f0068380 msgr2=0x7fe2f006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.084+0000 7fe317600640 1 --2- 172.21.15.32:0/1714996428 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe2f0068380 0x7fe2f006a840 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe304000c10 tx=0x7fe304009040 comp rx=0 tx=0).stop 2024-04-17T10:03:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.084+0000 7fe317600640 1 -- 172.21.15.32:0/1714996428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3100abde0 msgr2=0x7fe3100b3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.084+0000 7fe317600640 1 --2- 172.21.15.32:0/1714996428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3100abde0 0x7fe3100b3cb0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc011990 tx=0x7fe2fc011e50 comp rx=0 tx=0).stop 2024-04-17T10:03:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.084+0000 7fe317600640 1 -- 172.21.15.32:0/1714996428 shutdown_connections 2024-04-17T10:03:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.084+0000 7fe317600640 1 --2- 172.21.15.32:0/1714996428 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe2f0068380 0x7fe2f006a840 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.084+0000 7fe317600640 1 --2- 172.21.15.32:0/1714996428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3100ac240 0x7fe3100b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.084+0000 7fe317600640 1 --2- 172.21.15.32:0/1714996428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3100abde0 0x7fe3100b3cb0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.084+0000 7fe317600640 1 --2- 172.21.15.32:0/1714996428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2f80051d0 0x7fe31015bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.084+0000 7fe317600640 1 -- 172.21.15.32:0/1714996428 >> 172.21.15.32:0/1714996428 conn(0x7fe31006ba30 msgr2=0x7fe3100cdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.084+0000 7fe317600640 1 -- 172.21.15.32:0/1714996428 shutdown_connections 2024-04-17T10:03:22.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.084+0000 7fe317600640 1 -- 172.21.15.32:0/1714996428 wait complete. 2024-04-17T10:03:22.100 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:01:38.938460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:01:39.505171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:03:22.100 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:03:22.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.248+0000 7fc155400640 1 Processor -- start 2024-04-17T10:03:22.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.248+0000 7fc155400640 1 -- start start 2024-04-17T10:03:22.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.248+0000 7fc155400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc1500d1930 0x7fc15006d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:22.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.248+0000 7fc155400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc15006da50 0x7fc1500cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:22.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.248+0000 7fc155400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc15006d250 con 0x7fc15015e7b0 2024-04-17T10:03:22.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.248+0000 7fc155400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc1500d14a0 con 0x7fc1500d1930 2024-04-17T10:03:22.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.248+0000 7fc155400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc1500d0b70 con 0x7fc15006da50 2024-04-17T10:03:22.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.248+0000 7fc145e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc1500d1930 0x7fc15006d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:22.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.248+0000 7fc145e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc1500d1930 0x7fc15006d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43112/0 (socket says 172.21.15.32:43112) 2024-04-17T10:03:22.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.248+0000 7fc145e00640 1 -- 172.21.15.32:0/2523704349 learned_addr learned my addr 172.21.15.32:0/2523704349 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:22.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.248+0000 7fc14f200640 1 --2- 172.21.15.32:0/2523704349 >> v2:172.21.15.144:3300/0 conn(0x7fc15006da50 0x7fc1500cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:22.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.248+0000 7fc145e00640 1 -- 172.21.15.32:0/2523704349 >> v2:172.21.15.144:3300/0 conn(0x7fc15006da50 msgr2=0x7fc1500cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:22.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.248+0000 7fc145e00640 1 --2- 172.21.15.32:0/2523704349 >> v2:172.21.15.144:3300/0 conn(0x7fc15006da50 0x7fc1500cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.248+0000 7fc145e00640 1 -- 172.21.15.32:0/2523704349 >> v1:172.21.15.144:6789/0 conn(0x7fc15015e7b0 legacy=0x7fc15006cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:22.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.248+0000 7fc145e00640 1 -- 172.21.15.32:0/2523704349 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1500d0e50 con 0x7fc1500d1930 2024-04-17T10:03:22.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.252+0000 7fc145e00640 1 --2- 172.21.15.32:0/2523704349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1500d1930 0x7fc15006d510 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fc138009870 tx=0x7fc13801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14525fe706e34445 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:22.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.252+0000 7fc14de00640 1 -- 172.21.15.32:0/2523704349 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1380b0070 con 0x7fc1500d1930 2024-04-17T10:03:22.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.252+0000 7fc14de00640 1 -- 172.21.15.32:0/2523704349 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1380ab600 con 0x7fc1500d1930 2024-04-17T10:03:22.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.252+0000 7fc14de00640 1 -- 172.21.15.32:0/2523704349 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1380ab9d0 con 0x7fc1500d1930 2024-04-17T10:03:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.252+0000 7fc155400640 1 -- 172.21.15.32:0/2523704349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1500d1930 msgr2=0x7fc15006d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.252+0000 7fc155400640 1 --2- 172.21.15.32:0/2523704349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1500d1930 0x7fc15006d510 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fc138009870 tx=0x7fc13801c520 comp rx=0 tx=0).stop 2024-04-17T10:03:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.252+0000 7fc155400640 1 -- 172.21.15.32:0/2523704349 shutdown_connections 2024-04-17T10:03:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.252+0000 7fc155400640 1 --2- 172.21.15.32:0/2523704349 >> v2:172.21.15.144:3300/0 conn(0x7fc15006da50 0x7fc1500cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.252+0000 7fc155400640 1 --2- 172.21.15.32:0/2523704349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1500d1930 0x7fc15006d510 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.252+0000 7fc155400640 1 -- 172.21.15.32:0/2523704349 >> 172.21.15.32:0/2523704349 conn(0x7fc1500b4080 msgr2=0x7fc1500b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:22.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.252+0000 7fc155400640 1 -- 172.21.15.32:0/2523704349 shutdown_connections 2024-04-17T10:03:22.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.252+0000 7fc155400640 1 -- 172.21.15.32:0/2523704349 wait complete. 2024-04-17T10:03:22.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc155400640 1 Processor -- start 2024-04-17T10:03:22.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc155400640 1 -- start start 2024-04-17T10:03:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc155400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc15006da50 0x7fc1500b7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc155400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1500d1930 0x7fc1500b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc155400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15015e7b0 0x7fc1500b2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc155400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc15009b940 con 0x7fc1500d1930 2024-04-17T10:03:22.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc155400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc15015eda0 con 0x7fc15015e7b0 2024-04-17T10:03:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc155400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc15009bac0 con 0x7fc15006da50 2024-04-17T10:03:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc145e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1500d1930 0x7fc1500b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc145e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1500d1930 0x7fc1500b5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43126/0 (socket says 172.21.15.32:43126) 2024-04-17T10:03:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc145e00640 1 -- 172.21.15.32:0/515269241 learned_addr learned my addr 172.21.15.32:0/515269241 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc14f200640 1 --2- 172.21.15.32:0/515269241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15015e7b0 0x7fc1500b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc14e800640 1 --2- 172.21.15.32:0/515269241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc15006da50 0x7fc1500b7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc145e00640 1 -- 172.21.15.32:0/515269241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc15006da50 msgr2=0x7fc1500b7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc145e00640 1 --2- 172.21.15.32:0/515269241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc15006da50 0x7fc1500b7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc145e00640 1 -- 172.21.15.32:0/515269241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15015e7b0 msgr2=0x7fc1500b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:22.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc145e00640 1 --2- 172.21.15.32:0/515269241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15015e7b0 0x7fc1500b2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc145e00640 1 -- 172.21.15.32:0/515269241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1500b2860 con 0x7fc1500d1930 2024-04-17T10:03:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc14e800640 1 --2- 172.21.15.32:0/515269241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc15006da50 0x7fc1500b7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc145e00640 1 --2- 172.21.15.32:0/515269241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1500d1930 0x7fc1500b5c90 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fc13801abd0 tx=0x7fc13801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc147400640 1 -- 172.21.15.32:0/515269241 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1380b0040 con 0x7fc1500d1930 2024-04-17T10:03:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc147400640 1 -- 172.21.15.32:0/515269241 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1380bc070 con 0x7fc1500d1930 2024-04-17T10:03:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc155400640 1 -- 172.21.15.32:0/515269241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc15015c8d0 con 0x7fc1500d1930 2024-04-17T10:03:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc147400640 1 -- 172.21.15.32:0/515269241 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1380b6660 con 0x7fc1500d1930 2024-04-17T10:03:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.256+0000 7fc155400640 1 -- 172.21.15.32:0/515269241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1500b2550 con 0x7fc1500d1930 2024-04-17T10:03:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.260+0000 7fc155400640 1 -- 172.21.15.32:0/515269241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc150068a90 con 0x7fc1500d1930 2024-04-17T10:03:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.260+0000 7fc147400640 1 -- 172.21.15.32:0/515269241 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc1380b6800 con 0x7fc1500d1930 2024-04-17T10:03:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.260+0000 7fc147400640 1 --2- 172.21.15.32:0/515269241 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc12c068380 0x7fc12c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.260+0000 7fc147400640 1 -- 172.21.15.32:0/515269241 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x7fc138120c90 con 0x7fc1500d1930 2024-04-17T10:03:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.260+0000 7fc14e800640 1 --2- 172.21.15.32:0/515269241 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc12c068380 0x7fc12c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.260+0000 7fc14e800640 1 --2- 172.21.15.32:0/515269241 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc12c068380 0x7fc12c06a840 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc140001710 tx=0x7fc140009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:22.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.264+0000 7fc147400640 1 -- 172.21.15.32:0/515269241 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc138016110 con 0x7fc1500d1930 2024-04-17T10:03:22.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.428+0000 7fc155400640 1 -- 172.21.15.32:0/515269241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc1500b4cb0 con 0x7fc1500d1930 2024-04-17T10:03:22.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.428+0000 7fc147400640 1 -- 172.21.15.32:0/515269241 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7fc1380f1190 con 0x7fc1500d1930 2024-04-17T10:03:22.427 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T10:03:22.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.432+0000 7fc155400640 1 -- 172.21.15.32:0/515269241 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc12c068380 msgr2=0x7fc12c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:22.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.432+0000 7fc155400640 1 --2- 172.21.15.32:0/515269241 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc12c068380 0x7fc12c06a840 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc140001710 tx=0x7fc140009040 comp rx=0 tx=0).stop 2024-04-17T10:03:22.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.432+0000 7fc155400640 1 -- 172.21.15.32:0/515269241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1500d1930 msgr2=0x7fc1500b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:22.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.432+0000 7fc155400640 1 --2- 172.21.15.32:0/515269241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1500d1930 0x7fc1500b5c90 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fc13801abd0 tx=0x7fc13801ad60 comp rx=0 tx=0).stop 2024-04-17T10:03:22.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.432+0000 7fc155400640 1 -- 172.21.15.32:0/515269241 shutdown_connections 2024-04-17T10:03:22.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.432+0000 7fc155400640 1 --2- 172.21.15.32:0/515269241 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc12c068380 0x7fc12c06a840 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.432+0000 7fc155400640 1 --2- 172.21.15.32:0/515269241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15015e7b0 0x7fc1500b2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.432+0000 7fc155400640 1 --2- 172.21.15.32:0/515269241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1500d1930 0x7fc1500b5c90 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.432+0000 7fc155400640 1 --2- 172.21.15.32:0/515269241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc15006da50 0x7fc1500b7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.432+0000 7fc155400640 1 -- 172.21.15.32:0/515269241 >> 172.21.15.32:0/515269241 conn(0x7fc1500b4080 msgr2=0x7fc1500cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:22.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.432+0000 7fc155400640 1 -- 172.21.15.32:0/515269241 shutdown_connections 2024-04-17T10:03:22.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.432+0000 7fc155400640 1 -- 172.21.15.32:0/515269241 wait complete. 2024-04-17T10:03:22.448 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:03:22.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dc8400640 1 Processor -- start 2024-04-17T10:03:22.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dc8400640 1 -- start start 2024-04-17T10:03:22.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dc8400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9dc015b7d0 0x7f9dc006e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:22.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dc8400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dc00cc940 con 0x7f9dc006c140 2024-04-17T10:03:22.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dc8400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dc00ccc20 con 0x7f9dc006f7c0 2024-04-17T10:03:22.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dc8400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f9dc006e0b0 con 0x7f9dc015b7d0 2024-04-17T10:03:22.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dc6400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9dc006c140 0x7f9dc00ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53222/0 (socket says 172.21.15.32:53222) 2024-04-17T10:03:22.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dc6400640 1 -- 172.21.15.32:0/45297588 learned_addr learned my addr 172.21.15.32:0/45297588 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:22.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dc5a00640 1 --2- 172.21.15.32:0/45297588 >> v2:172.21.15.144:3300/0 conn(0x7f9dc015b7d0 0x7f9dc006e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:22.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dbfe00640 1 -- 172.21.15.32:0/45297588 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 53264194 0 0) 0x7f9dc00cc940 con 0x7f9dc006c140 2024-04-17T10:03:22.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dbfe00640 1 -- 172.21.15.32:0/45297588 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9da8003530 con 0x7f9dc006c140 2024-04-17T10:03:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dbfe00640 1 -- 172.21.15.32:0/45297588 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 869151489 0 0) 0x7f9da8003530 con 0x7f9dc006c140 2024-04-17T10:03:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dbfe00640 1 -- 172.21.15.32:0/45297588 >> v2:172.21.15.144:3300/0 conn(0x7f9dc015b7d0 msgr2=0x7f9dc006e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dbfe00640 1 --2- 172.21.15.32:0/45297588 >> v2:172.21.15.144:3300/0 conn(0x7f9dc015b7d0 0x7f9dc006e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dbfe00640 1 -- 172.21.15.32:0/45297588 >> v1:172.21.15.144:6789/0 conn(0x7f9dc006f7c0 legacy=0x7f9dc006ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dbfe00640 1 -- 172.21.15.32:0/45297588 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc006e390 con 0x7f9dc006c140 2024-04-17T10:03:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dbfe00640 1 -- 172.21.15.32:0/45297588 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f9db8001a00 con 0x7f9dc006c140 2024-04-17T10:03:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dbfe00640 1 -- 172.21.15.32:0/45297588 >> v1:172.21.15.32:6789/0 conn(0x7f9dc006c140 legacy=0x7f9dc00ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dbfe00640 1 --2- 172.21.15.32:0/45297588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80039c0 0x7f9da8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dbfe00640 1 --2- 172.21.15.32:0/45297588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9da8004810 0x7f9da8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dbfe00640 1 --2- 172.21.15.32:0/45297588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da80051d0 0x7f9da800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dbfe00640 1 -- 172.21.15.32:0/45297588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc006e0b0 con 0x7f9da80039c0 2024-04-17T10:03:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dbfe00640 1 -- 172.21.15.32:0/45297588 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9da800c0d0 con 0x7f9da80051d0 2024-04-17T10:03:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dbfe00640 1 -- 172.21.15.32:0/45297588 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9da800c2d0 con 0x7f9da8004810 2024-04-17T10:03:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dc5a00640 1 --2- 172.21.15.32:0/45297588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80039c0 0x7f9da8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dc5000640 1 --2- 172.21.15.32:0/45297588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9da8004810 0x7f9da8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dc6400640 1 --2- 172.21.15.32:0/45297588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da80051d0 0x7f9da800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dc5a00640 1 -- 172.21.15.32:0/45297588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9da8004810 msgr2=0x7f9da8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dc5a00640 1 --2- 172.21.15.32:0/45297588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9da8004810 0x7f9da8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dc5a00640 1 -- 172.21.15.32:0/45297588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da80051d0 msgr2=0x7f9da800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dc5a00640 1 --2- 172.21.15.32:0/45297588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da80051d0 0x7f9da800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dc5a00640 1 -- 172.21.15.32:0/45297588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da8003530 con 0x7f9da80039c0 2024-04-17T10:03:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.596+0000 7f9dc6400640 1 -- 172.21.15.32:0/45297588 reap_dead start 2024-04-17T10:03:22.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.600+0000 7f9dc5a00640 1 --2- 172.21.15.32:0/45297588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80039c0 0x7f9da8003da0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009ae0 tx=0x7f9db401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef80ef66b5ca6eef server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:22.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.600+0000 7f9dbfe00640 1 -- 172.21.15.32:0/45297588 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db40b2070 con 0x7f9da80039c0 2024-04-17T10:03:22.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.600+0000 7f9dbfe00640 1 -- 172.21.15.32:0/45297588 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9db40a56f0 con 0x7f9da80039c0 2024-04-17T10:03:22.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.600+0000 7f9dbfe00640 1 -- 172.21.15.32:0/45297588 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db40096b0 con 0x7f9da80039c0 2024-04-17T10:03:22.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.600+0000 7f9dc8400640 1 -- 172.21.15.32:0/45297588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80039c0 msgr2=0x7f9da8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:22.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.600+0000 7f9dc8400640 1 --2- 172.21.15.32:0/45297588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80039c0 0x7f9da8003da0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009ae0 tx=0x7f9db401ca70 comp rx=0 tx=0).stop 2024-04-17T10:03:22.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.600+0000 7f9dc8400640 1 -- 172.21.15.32:0/45297588 shutdown_connections 2024-04-17T10:03:22.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.600+0000 7f9dc8400640 1 --2- 172.21.15.32:0/45297588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80039c0 0x7f9da8003da0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.600+0000 7f9dc8400640 1 -- 172.21.15.32:0/45297588 >> 172.21.15.32:0/45297588 conn(0x7f9dc00b2120 msgr2=0x7f9dc00b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:22.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.600+0000 7f9dc8400640 1 -- 172.21.15.32:0/45297588 shutdown_connections 2024-04-17T10:03:22.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.600+0000 7f9dc8400640 1 -- 172.21.15.32:0/45297588 wait complete. 2024-04-17T10:03:22.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dc8400640 1 Processor -- start 2024-04-17T10:03:22.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dc8400640 1 -- start start 2024-04-17T10:03:22.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dc8400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80039c0 0x7f9dc0153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:22.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dc8400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dc014ea10 0x7f9dc014cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:22.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dc8400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dc0151050 0x7f9dc01526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:22.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dc8400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc00ced60 con 0x7f9da80039c0 2024-04-17T10:03:22.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dc8400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc00cebe0 con 0x7f9dc014ea10 2024-04-17T10:03:22.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dc8400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9dc00ceee0 con 0x7f9dc0151050 2024-04-17T10:03:22.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dc5a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80039c0 0x7f9dc0153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:22.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dc5a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80039c0 0x7f9dc0153ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43150/0 (socket says 172.21.15.32:43150) 2024-04-17T10:03:22.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dc5a00640 1 -- 172.21.15.32:0/1664192405 learned_addr learned my addr 172.21.15.32:0/1664192405 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:22.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dc5000640 1 --2- 172.21.15.32:0/1664192405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dc014ea10 0x7f9dc014cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:22.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dc6400640 1 --2- 172.21.15.32:0/1664192405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dc0151050 0x7f9dc01526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:22.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dc5a00640 1 -- 172.21.15.32:0/1664192405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dc0151050 msgr2=0x7f9dc01526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:22.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dc5a00640 1 --2- 172.21.15.32:0/1664192405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dc0151050 0x7f9dc01526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dc5a00640 1 -- 172.21.15.32:0/1664192405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dc014ea10 msgr2=0x7f9dc014cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:22.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dc5a00640 1 --2- 172.21.15.32:0/1664192405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dc014ea10 0x7f9dc014cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dc5a00640 1 -- 172.21.15.32:0/1664192405 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc004fe30 con 0x7f9da80039c0 2024-04-17T10:03:22.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dc5a00640 1 --2- 172.21.15.32:0/1664192405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80039c0 0x7f9dc0153ca0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f9db401c8a0 tx=0x7f9db4019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:22.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dbd600640 1 -- 172.21.15.32:0/1664192405 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db40b2040 con 0x7f9da80039c0 2024-04-17T10:03:22.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dbd600640 1 -- 172.21.15.32:0/1664192405 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9db400f000 con 0x7f9da80039c0 2024-04-17T10:03:22.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dbd600640 1 -- 172.21.15.32:0/1664192405 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db40096b0 con 0x7f9da80039c0 2024-04-17T10:03:22.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dc8400640 1 -- 172.21.15.32:0/1664192405 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc0150730 con 0x7f9da80039c0 2024-04-17T10:03:22.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.604+0000 7f9dc8400640 1 -- 172.21.15.32:0/1664192405 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc01521a0 con 0x7f9da80039c0 2024-04-17T10:03:22.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.608+0000 7f9dc8400640 1 -- 172.21.15.32:0/1664192405 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc0068a90 con 0x7f9da80039c0 2024-04-17T10:03:22.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.608+0000 7f9dbd600640 1 -- 172.21.15.32:0/1664192405 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9db40bf030 con 0x7f9da80039c0 2024-04-17T10:03:22.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.608+0000 7f9dbd600640 1 --2- 172.21.15.32:0/1664192405 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9da0068380 0x7f9da006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:22.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.608+0000 7f9dbd600640 1 -- 172.21.15.32:0/1664192405 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x7f9db41206a0 con 0x7f9da80039c0 2024-04-17T10:03:22.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.608+0000 7f9dc5000640 1 --2- 172.21.15.32:0/1664192405 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9da0068380 0x7f9da006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:22.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.608+0000 7f9dc5000640 1 --2- 172.21.15.32:0/1664192405 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9da0068380 0x7f9da006a840 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9dc006bcc0 tx=0x7f9dac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:22.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.612+0000 7f9dbd600640 1 -- 172.21.15.32:0/1664192405 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f9db40c6060 con 0x7f9da80039c0 2024-04-17T10:03:22.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.776+0000 7f9dc8400640 1 -- 172.21.15.32:0/1664192405 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9dc0151f90 con 0x7f9da80039c0 2024-04-17T10:03:22.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.780+0000 7f9dbd600640 1 -- 172.21.15.32:0/1664192405 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7f9db4016780 con 0x7f9da80039c0 2024-04-17T10:03:22.776 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T10:03:22.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.780+0000 7f9dc8400640 1 -- 172.21.15.32:0/1664192405 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9da0068380 msgr2=0x7f9da006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:22.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.780+0000 7f9dc8400640 1 --2- 172.21.15.32:0/1664192405 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9da0068380 0x7f9da006a840 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9dc006bcc0 tx=0x7f9dac009040 comp rx=0 tx=0).stop 2024-04-17T10:03:22.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.780+0000 7f9dc8400640 1 -- 172.21.15.32:0/1664192405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80039c0 msgr2=0x7f9dc0153ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:22.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.780+0000 7f9dc8400640 1 --2- 172.21.15.32:0/1664192405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80039c0 0x7f9dc0153ca0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f9db401c8a0 tx=0x7f9db4019810 comp rx=0 tx=0).stop 2024-04-17T10:03:22.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.780+0000 7f9dc8400640 1 -- 172.21.15.32:0/1664192405 shutdown_connections 2024-04-17T10:03:22.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.780+0000 7f9dc8400640 1 --2- 172.21.15.32:0/1664192405 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9da0068380 0x7f9da006a840 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.780+0000 7f9dc8400640 1 --2- 172.21.15.32:0/1664192405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dc0151050 0x7f9dc01526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.780+0000 7f9dc8400640 1 --2- 172.21.15.32:0/1664192405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dc014ea10 0x7f9dc014cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.780+0000 7f9dc8400640 1 --2- 172.21.15.32:0/1664192405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9da80039c0 0x7f9dc0153ca0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.780+0000 7f9dc8400640 1 -- 172.21.15.32:0/1664192405 >> 172.21.15.32:0/1664192405 conn(0x7f9dc00b2120 msgr2=0x7f9dc00af6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.780+0000 7f9dc8400640 1 -- 172.21.15.32:0/1664192405 shutdown_connections 2024-04-17T10:03:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.780+0000 7f9dc8400640 1 -- 172.21.15.32:0/1664192405 wait complete. 2024-04-17T10:03:22.798 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:03:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.944+0000 7f0f5a000640 1 Processor -- start 2024-04-17T10:03:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.944+0000 7f0f5a000640 1 -- start start 2024-04-17T10:03:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.944+0000 7f0f5a000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f5406ba30 0x7f0f5406be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.944+0000 7f0f5a000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0f54071e60 0x7f0f540722b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.944+0000 7f0f5a000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f5406c350 con 0x7f0f540cf370 2024-04-17T10:03:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.944+0000 7f0f5a000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0f540cfef0 con 0x7f0f54071e60 2024-04-17T10:03:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.944+0000 7f0f5a000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0f540d0130 con 0x7f0f5406ba30 2024-04-17T10:03:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.944+0000 7f0f53e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0f54071e60 0x7f0f540722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.944+0000 7f0f53e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0f54071e60 0x7f0f540722b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43158/0 (socket says 172.21.15.32:43158) 2024-04-17T10:03:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.944+0000 7f0f53e00640 1 -- 172.21.15.32:0/2505832896 learned_addr learned my addr 172.21.15.32:0/2505832896 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.944+0000 7f0f53400640 1 --2- 172.21.15.32:0/2505832896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f5406ba30 0x7f0f5406be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.944+0000 7f0f53e00640 1 -- 172.21.15.32:0/2505832896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f5406ba30 msgr2=0x7f0f5406be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.944+0000 7f0f53e00640 1 --2- 172.21.15.32:0/2505832896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f5406ba30 0x7f0f5406be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.944+0000 7f0f53e00640 1 -- 172.21.15.32:0/2505832896 >> v1:172.21.15.144:6789/0 conn(0x7f0f540cf370 legacy=0x7f0f540cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.944+0000 7f0f53e00640 1 -- 172.21.15.32:0/2505832896 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5415d090 con 0x7f0f54071e60 2024-04-17T10:03:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.944+0000 7f0f53400640 1 --2- 172.21.15.32:0/2505832896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f5406ba30 0x7f0f5406be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:03:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.948+0000 7f0f53e00640 1 --2- 172.21.15.32:0/2505832896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f54071e60 0x7f0f540722b0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009e50 tx=0x7f0f4c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c0236ba5ac4346 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.948+0000 7f0f52000640 1 -- 172.21.15.32:0/2505832896 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0a5840 con 0x7f0f54071e60 2024-04-17T10:03:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.948+0000 7f0f52000640 1 -- 172.21.15.32:0/2505832896 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f4c0a59e0 con 0x7f0f54071e60 2024-04-17T10:03:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.948+0000 7f0f52000640 1 -- 172.21.15.32:0/2505832896 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0a5db0 con 0x7f0f54071e60 2024-04-17T10:03:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.948+0000 7f0f5a000640 1 -- 172.21.15.32:0/2505832896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f54071e60 msgr2=0x7f0f540722b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.948+0000 7f0f5a000640 1 --2- 172.21.15.32:0/2505832896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f54071e60 0x7f0f540722b0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009e50 tx=0x7f0f4c01c9c0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.948+0000 7f0f5a000640 1 -- 172.21.15.32:0/2505832896 shutdown_connections 2024-04-17T10:03:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.948+0000 7f0f5a000640 1 --2- 172.21.15.32:0/2505832896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f54071e60 0x7f0f540722b0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.948+0000 7f0f5a000640 1 --2- 172.21.15.32:0/2505832896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f5406ba30 0x7f0f5406be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.948+0000 7f0f5a000640 1 -- 172.21.15.32:0/2505832896 >> 172.21.15.32:0/2505832896 conn(0x7f0f540b2120 msgr2=0x7f0f540b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:22.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.948+0000 7f0f5a000640 1 -- 172.21.15.32:0/2505832896 shutdown_connections 2024-04-17T10:03:22.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.948+0000 7f0f5a000640 1 -- 172.21.15.32:0/2505832896 wait complete. 2024-04-17T10:03:22.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f5a000640 1 Processor -- start 2024-04-17T10:03:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f5a000640 1 -- start start 2024-04-17T10:03:22.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f5a000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f5406ba30 0x7f0f54141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:22.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f5a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f54071e60 0x7f0f5415b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:22.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f5a000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f540cf370 0x7f0f5415bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:22.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f5a000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5415dcf0 con 0x7f0f54071e60 2024-04-17T10:03:22.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f5a000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5415db70 con 0x7f0f5406ba30 2024-04-17T10:03:22.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f5a000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0f54146230 con 0x7f0f540cf370 2024-04-17T10:03:22.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f52a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f54071e60 0x7f0f5415b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:22.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f52a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f54071e60 0x7f0f5415b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43168/0 (socket says 172.21.15.32:43168) 2024-04-17T10:03:22.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f52a00640 1 -- 172.21.15.32:0/1464002585 learned_addr learned my addr 172.21.15.32:0/1464002585 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:22.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f53400640 1 --2- 172.21.15.32:0/1464002585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f5406ba30 0x7f0f54141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:22.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f53e00640 1 --2- 172.21.15.32:0/1464002585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f540cf370 0x7f0f5415bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:22.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f52a00640 1 -- 172.21.15.32:0/1464002585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f540cf370 msgr2=0x7f0f5415bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:22.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f52a00640 1 --2- 172.21.15.32:0/1464002585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f540cf370 0x7f0f5415bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f52a00640 1 -- 172.21.15.32:0/1464002585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f5406ba30 msgr2=0x7f0f54141130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:22.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f52a00640 1 --2- 172.21.15.32:0/1464002585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f5406ba30 0x7f0f54141130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:22.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f52a00640 1 -- 172.21.15.32:0/1464002585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f541f78d0 con 0x7f0f54071e60 2024-04-17T10:03:22.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f52a00640 1 --2- 172.21.15.32:0/1464002585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f54071e60 0x7f0f5415b9d0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f0f4000b410 tx=0x7f0f4000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:22.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f4b400640 1 -- 172.21.15.32:0/1464002585 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f40015030 con 0x7f0f54071e60 2024-04-17T10:03:22.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f4b400640 1 -- 172.21.15.32:0/1464002585 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f40004020 con 0x7f0f54071e60 2024-04-17T10:03:22.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f4b400640 1 -- 172.21.15.32:0/1464002585 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f40004450 con 0x7f0f54071e60 2024-04-17T10:03:22.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f5a000640 1 -- 172.21.15.32:0/1464002585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f541f7bb0 con 0x7f0f54071e60 2024-04-17T10:03:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f5a000640 1 -- 172.21.15.32:0/1464002585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f541f8170 con 0x7f0f54071e60 2024-04-17T10:03:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.952+0000 7f0f5a000640 1 -- 172.21.15.32:0/1464002585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f54068a90 con 0x7f0f54071e60 2024-04-17T10:03:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.956+0000 7f0f4b400640 1 -- 172.21.15.32:0/1464002585 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0f40018070 con 0x7f0f54071e60 2024-04-17T10:03:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.956+0000 7f0f4b400640 1 --2- 172.21.15.32:0/1464002585 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0f34068380 0x7f0f3406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.956+0000 7f0f4b400640 1 -- 172.21.15.32:0/1464002585 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x7f0f40086500 con 0x7f0f54071e60 2024-04-17T10:03:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.956+0000 7f0f53400640 1 --2- 172.21.15.32:0/1464002585 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0f34068380 0x7f0f3406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:22.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.956+0000 7f0f53400640 1 --2- 172.21.15.32:0/1464002585 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0f34068380 0x7f0f3406a840 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400ab60 tx=0x7f0f44008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:22.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:22.960+0000 7f0f4b400640 1 -- 172.21.15.32:0/1464002585 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0f40056320 con 0x7f0f54071e60 2024-04-17T10:03:23.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.124+0000 7f0f5a000640 1 -- 172.21.15.32:0/1464002585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0f5406c850 con 0x7f0f54071e60 2024-04-17T10:03:23.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.124+0000 7f0f4b400640 1 -- 172.21.15.32:0/1464002585 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7f0f4005a620 con 0x7f0f54071e60 2024-04-17T10:03:23.123 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T10:03:23.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.128+0000 7f0f5a000640 1 -- 172.21.15.32:0/1464002585 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0f34068380 msgr2=0x7f0f3406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:23.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.128+0000 7f0f5a000640 1 --2- 172.21.15.32:0/1464002585 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0f34068380 0x7f0f3406a840 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400ab60 tx=0x7f0f44008040 comp rx=0 tx=0).stop 2024-04-17T10:03:23.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.128+0000 7f0f5a000640 1 -- 172.21.15.32:0/1464002585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f54071e60 msgr2=0x7f0f5415b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:23.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.128+0000 7f0f5a000640 1 --2- 172.21.15.32:0/1464002585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f54071e60 0x7f0f5415b9d0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f0f4000b410 tx=0x7f0f4000b8d0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.128+0000 7f0f5a000640 1 -- 172.21.15.32:0/1464002585 shutdown_connections 2024-04-17T10:03:23.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.128+0000 7f0f5a000640 1 --2- 172.21.15.32:0/1464002585 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0f34068380 0x7f0f3406a840 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.128+0000 7f0f5a000640 1 --2- 172.21.15.32:0/1464002585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f540cf370 0x7f0f5415bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.128+0000 7f0f5a000640 1 --2- 172.21.15.32:0/1464002585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f54071e60 0x7f0f5415b9d0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.128+0000 7f0f5a000640 1 --2- 172.21.15.32:0/1464002585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f5406ba30 0x7f0f54141130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.128+0000 7f0f5a000640 1 -- 172.21.15.32:0/1464002585 >> 172.21.15.32:0/1464002585 conn(0x7f0f540b2120 msgr2=0x7f0f54165890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:23.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.128+0000 7f0f5a000640 1 -- 172.21.15.32:0/1464002585 shutdown_connections 2024-04-17T10:03:23.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.128+0000 7f0f5a000640 1 -- 172.21.15.32:0/1464002585 wait complete. 2024-04-17T10:03:23.144 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:03:23.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.288+0000 7f07b9200640 1 Processor -- start 2024-04-17T10:03:23.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.288+0000 7f07b9200640 1 -- start start 2024-04-17T10:03:23.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.288+0000 7f07b9200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f07b4072500 0x7f07b40cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:23.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.288+0000 7f07b9200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07b415d8d0 0x7f07b40cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:23.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.288+0000 7f07b9200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07b40cdac0 con 0x7f07b4145f30 2024-04-17T10:03:23.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.288+0000 7f07b9200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f07b406f400 con 0x7f07b4072500 2024-04-17T10:03:23.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.288+0000 7f07b9200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f07b4146310 con 0x7f07b415d8d0 2024-04-17T10:03:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.288+0000 7f07a9400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f07b4072500 0x7f07b40cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.288+0000 7f07b2800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07b415d8d0 0x7f07b40cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.288+0000 7f07b2800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07b415d8d0 0x7f07b40cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:54472/0 (socket says 172.21.15.32:54472) 2024-04-17T10:03:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.288+0000 7f07b2800640 1 -- 172.21.15.32:0/1168638877 learned_addr learned my addr 172.21.15.32:0/1168638877 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.288+0000 7f07a9400640 1 -- 172.21.15.32:0/1168638877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07b415d8d0 msgr2=0x7f07b40cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.288+0000 7f07a9400640 1 --2- 172.21.15.32:0/1168638877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07b415d8d0 0x7f07b40cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.288+0000 7f07a9400640 1 -- 172.21.15.32:0/1168638877 >> v1:172.21.15.144:6789/0 conn(0x7f07b4145f30 legacy=0x7f07b4071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.288+0000 7f07a9400640 1 -- 172.21.15.32:0/1168638877 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b415cdf0 con 0x7f07b4072500 2024-04-17T10:03:23.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.288+0000 7f07b2800640 1 --2- 172.21.15.32:0/1168638877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07b415d8d0 0x7f07b40cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:03:23.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.292+0000 7f07a9400640 1 --2- 172.21.15.32:0/1168638877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07b4072500 0x7f07b40cce70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f079c0097f0 tx=0x7f079c01ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd97464e36b74ddb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:23.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.292+0000 7f07b1400640 1 -- 172.21.15.32:0/1168638877 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c0b5070 con 0x7f07b4072500 2024-04-17T10:03:23.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.292+0000 7f07b1400640 1 -- 172.21.15.32:0/1168638877 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f079c0a8990 con 0x7f07b4072500 2024-04-17T10:03:23.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.292+0000 7f07b1400640 1 -- 172.21.15.32:0/1168638877 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c0a8d60 con 0x7f07b4072500 2024-04-17T10:03:23.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.292+0000 7f07b9200640 1 -- 172.21.15.32:0/1168638877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07b4072500 msgr2=0x7f07b40cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:23.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.292+0000 7f07b9200640 1 --2- 172.21.15.32:0/1168638877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07b4072500 0x7f07b40cce70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f079c0097f0 tx=0x7f079c01ec90 comp rx=0 tx=0).stop 2024-04-17T10:03:23.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.292+0000 7f07b9200640 1 -- 172.21.15.32:0/1168638877 shutdown_connections 2024-04-17T10:03:23.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.292+0000 7f07b9200640 1 --2- 172.21.15.32:0/1168638877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07b415d8d0 0x7f07b40cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.292+0000 7f07b9200640 1 --2- 172.21.15.32:0/1168638877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07b4072500 0x7f07b40cce70 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.292+0000 7f07b9200640 1 -- 172.21.15.32:0/1168638877 >> 172.21.15.32:0/1168638877 conn(0x7f07b406ba30 msgr2=0x7f07b406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:23.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.292+0000 7f07b9200640 1 -- 172.21.15.32:0/1168638877 shutdown_connections 2024-04-17T10:03:23.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.292+0000 7f07b9200640 1 -- 172.21.15.32:0/1168638877 wait complete. 2024-04-17T10:03:23.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07b9200640 1 Processor -- start 2024-04-17T10:03:23.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07b9200640 1 -- start start 2024-04-17T10:03:23.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07b9200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b4072500 0x7f07b40b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:23.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07b9200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07b4145f30 0x7f07b4142e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:23.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07b9200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07b415d8d0 0x7f07b415f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:23.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07b9200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f07b415cb90 con 0x7f07b4072500 2024-04-17T10:03:23.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07b9200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f07b415d440 con 0x7f07b4145f30 2024-04-17T10:03:23.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07b9200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f07b41601d0 con 0x7f07b415d8d0 2024-04-17T10:03:23.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07b1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b4072500 0x7f07b40b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:23.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07b1e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b4072500 0x7f07b40b4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43184/0 (socket says 172.21.15.32:43184) 2024-04-17T10:03:23.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07b1e00640 1 -- 172.21.15.32:0/1399289340 learned_addr learned my addr 172.21.15.32:0/1399289340 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:23.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07a9400640 1 --2- 172.21.15.32:0/1399289340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07b4145f30 0x7f07b4142e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:23.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07b2800640 1 --2- 172.21.15.32:0/1399289340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07b415d8d0 0x7f07b415f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:23.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07b1e00640 1 -- 172.21.15.32:0/1399289340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07b415d8d0 msgr2=0x7f07b415f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:23.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07b1e00640 1 --2- 172.21.15.32:0/1399289340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07b415d8d0 0x7f07b415f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07b1e00640 1 -- 172.21.15.32:0/1399289340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07b4145f30 msgr2=0x7f07b4142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:23.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07b1e00640 1 --2- 172.21.15.32:0/1399289340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07b4145f30 0x7f07b4142e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07b1e00640 1 -- 172.21.15.32:0/1399289340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b41f8510 con 0x7f07b4072500 2024-04-17T10:03:23.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07b1e00640 1 --2- 172.21.15.32:0/1399289340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b4072500 0x7f07b40b4940 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f07a400c400 tx=0x7f07a400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:23.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07aaa00640 1 -- 172.21.15.32:0/1399289340 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4017070 con 0x7f07b4072500 2024-04-17T10:03:23.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07aaa00640 1 -- 172.21.15.32:0/1399289340 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a4005310 con 0x7f07b4072500 2024-04-17T10:03:23.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07aaa00640 1 -- 172.21.15.32:0/1399289340 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a40054c0 con 0x7f07b4072500 2024-04-17T10:03:23.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07b9200640 1 -- 172.21.15.32:0/1399289340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b41f87f0 con 0x7f07b4072500 2024-04-17T10:03:23.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.296+0000 7f07b9200640 1 -- 172.21.15.32:0/1399289340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b41f8db0 con 0x7f07b4072500 2024-04-17T10:03:23.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.300+0000 7f07b9200640 1 -- 172.21.15.32:0/1399289340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07b4068a90 con 0x7f07b4072500 2024-04-17T10:03:23.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.300+0000 7f07aaa00640 1 -- 172.21.15.32:0/1399289340 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f07a401a070 con 0x7f07b4072500 2024-04-17T10:03:23.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.300+0000 7f07aaa00640 1 --2- 172.21.15.32:0/1399289340 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0790068380 0x7f079006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:23.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.300+0000 7f07aaa00640 1 -- 172.21.15.32:0/1399289340 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x7f07a4085ce0 con 0x7f07b4072500 2024-04-17T10:03:23.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.300+0000 7f07a9400640 1 --2- 172.21.15.32:0/1399289340 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0790068380 0x7f079006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:23.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.300+0000 7f07a9400640 1 --2- 172.21.15.32:0/1399289340 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0790068380 0x7f079006a840 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f079c0b2000 tx=0x7f079c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:23.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.304+0000 7f07aaa00640 1 -- 172.21.15.32:0/1399289340 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f07a4055b90 con 0x7f07b4072500 2024-04-17T10:03:23.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.504+0000 7f07b9200640 1 -- 172.21.15.32:0/1399289340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07b4072f90 con 0x7f07b4072500 2024-04-17T10:03:23.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.504+0000 7f07aaa00640 1 -- 172.21.15.32:0/1399289340 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5190 (secure 0 0 0) 0x7f07a4059e90 con 0x7f07b4072500 2024-04-17T10:03:23.503 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:03:23.503 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":89,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2635109588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2635109588},{"type":"v1","addr":"172.21.15.144:6835","nonce":2635109588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7789,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3258455939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3258455939},{"type":"v1","addr":"172.21.15.144:6834","nonce":3258455939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":8574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3390468332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3390468332},{"type":"v1","addr":"172.21.15.32:6836","nonce":3390468332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":9123,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/374146310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":374146310},{"type":"v1","addr":"172.21.15.32:6837","nonce":374146310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":88,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:01:40.557208+0000","modified":"2024-04-17T10:03:20.790439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-17T10:03:23.503 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 89 2024-04-17T10:03:23.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.508+0000 7f07b9200640 1 -- 172.21.15.32:0/1399289340 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0790068380 msgr2=0x7f079006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.508+0000 7f07b9200640 1 --2- 172.21.15.32:0/1399289340 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0790068380 0x7f079006a840 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f079c0b2000 tx=0x7f079c008040 comp rx=0 tx=0).stop 2024-04-17T10:03:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.508+0000 7f07b9200640 1 -- 172.21.15.32:0/1399289340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b4072500 msgr2=0x7f07b40b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.508+0000 7f07b9200640 1 --2- 172.21.15.32:0/1399289340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b4072500 0x7f07b40b4940 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f07a400c400 tx=0x7f07a400c8c0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.508+0000 7f07b9200640 1 -- 172.21.15.32:0/1399289340 shutdown_connections 2024-04-17T10:03:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.508+0000 7f07b9200640 1 --2- 172.21.15.32:0/1399289340 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0790068380 0x7f079006a840 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.508+0000 7f07b9200640 1 --2- 172.21.15.32:0/1399289340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07b415d8d0 0x7f07b415f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.508+0000 7f07b9200640 1 --2- 172.21.15.32:0/1399289340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07b4145f30 0x7f07b4142e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.508+0000 7f07b9200640 1 --2- 172.21.15.32:0/1399289340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b4072500 0x7f07b40b4940 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.508+0000 7f07b9200640 1 -- 172.21.15.32:0/1399289340 >> 172.21.15.32:0/1399289340 conn(0x7f07b406ba30 msgr2=0x7f07b406be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.508+0000 7f07b9200640 1 -- 172.21.15.32:0/1399289340 shutdown_connections 2024-04-17T10:03:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.508+0000 7f07b9200640 1 -- 172.21.15.32:0/1399289340 wait complete. 2024-04-17T10:03:23.524 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:03:23.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.668+0000 7faafea00640 1 Processor -- start 2024-04-17T10:03:23.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.668+0000 7faafea00640 1 -- start start 2024-04-17T10:03:23.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.668+0000 7faafea00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faaf8151db0 0x7faaf80b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:23.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.668+0000 7faafea00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faaf806f630 0x7faaf8072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:23.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.668+0000 7faafea00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faaf8072bc0 0x7faaf815e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:23.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.668+0000 7faafea00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7faaf80b44f0 con 0x7faaf806f630 2024-04-17T10:03:23.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.668+0000 7faafea00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7faaf80b3490 con 0x7faaf8151db0 2024-04-17T10:03:23.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.668+0000 7faafea00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7faaf80cea40 con 0x7faaf8072bc0 2024-04-17T10:03:23.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.668+0000 7faaefe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faaf806f630 0x7faaf8072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:23.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.668+0000 7faaefe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faaf806f630 0x7faaf8072680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43186/0 (socket says 172.21.15.32:43186) 2024-04-17T10:03:23.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.668+0000 7faaefe00640 1 -- 172.21.15.32:0/1265996594 learned_addr learned my addr 172.21.15.32:0/1265996594 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:23.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.668+0000 7faaf7e00640 1 --2- 172.21.15.32:0/1265996594 >> v2:172.21.15.144:3300/0 conn(0x7faaf8151db0 0x7faaf80b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:23.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.668+0000 7faafca00640 1 --2- 172.21.15.32:0/1265996594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faaf8072bc0 0x7faaf815e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:23.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.668+0000 7faaefe00640 1 -- 172.21.15.32:0/1265996594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faaf8072bc0 msgr2=0x7faaf815e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:23.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.668+0000 7faaefe00640 1 --2- 172.21.15.32:0/1265996594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faaf8072bc0 0x7faaf815e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.668+0000 7faaefe00640 1 -- 172.21.15.32:0/1265996594 >> v2:172.21.15.144:3300/0 conn(0x7faaf8151db0 msgr2=0x7faaf80b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:23.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.668+0000 7faaefe00640 1 --2- 172.21.15.32:0/1265996594 >> v2:172.21.15.144:3300/0 conn(0x7faaf8151db0 0x7faaf80b51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.668+0000 7faaefe00640 1 -- 172.21.15.32:0/1265996594 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf80b3730 con 0x7faaf806f630 2024-04-17T10:03:23.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.672+0000 7faaefe00640 1 --2- 172.21.15.32:0/1265996594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf806f630 0x7faaf8072680 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7faaf8073260 tx=0x7faae001e9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b482c2171739c79 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:23.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.672+0000 7faaf7400640 1 -- 172.21.15.32:0/1265996594 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae00ad070 con 0x7faaf806f630 2024-04-17T10:03:23.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.672+0000 7faaf7400640 1 -- 172.21.15.32:0/1265996594 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faae0004230 con 0x7faaf806f630 2024-04-17T10:03:23.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.672+0000 7faaf7400640 1 -- 172.21.15.32:0/1265996594 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae0003620 con 0x7faaf806f630 2024-04-17T10:03:23.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.672+0000 7faafea00640 1 -- 172.21.15.32:0/1265996594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf806f630 msgr2=0x7faaf8072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:23.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.672+0000 7faafea00640 1 --2- 172.21.15.32:0/1265996594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf806f630 0x7faaf8072680 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7faaf8073260 tx=0x7faae001e9d0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.672+0000 7faafea00640 1 -- 172.21.15.32:0/1265996594 shutdown_connections 2024-04-17T10:03:23.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.672+0000 7faafea00640 1 --2- 172.21.15.32:0/1265996594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faaf8072bc0 0x7faaf815e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.672+0000 7faafea00640 1 --2- 172.21.15.32:0/1265996594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf806f630 0x7faaf8072680 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.672+0000 7faafea00640 1 --2- 172.21.15.32:0/1265996594 >> v2:172.21.15.144:3300/0 conn(0x7faaf8151db0 0x7faaf80b51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.672+0000 7faafea00640 1 -- 172.21.15.32:0/1265996594 >> 172.21.15.32:0/1265996594 conn(0x7faaf806ba30 msgr2=0x7faaf806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:23.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.672+0000 7faafea00640 1 -- 172.21.15.32:0/1265996594 shutdown_connections 2024-04-17T10:03:23.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.672+0000 7faafea00640 1 -- 172.21.15.32:0/1265996594 wait complete. 2024-04-17T10:03:23.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faafea00640 1 Processor -- start 2024-04-17T10:03:23.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faafea00640 1 -- start start 2024-04-17T10:03:23.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faafea00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faaf806f630 0x7faaf8147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:23.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faafea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaf8072bc0 0x7faaf814b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:23.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faafea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf8151db0 0x7faaf815c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:23.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faafea00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faaf80b5880 con 0x7faaf8151db0 2024-04-17T10:03:23.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faafea00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faaf80b5700 con 0x7faaf8072bc0 2024-04-17T10:03:23.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faafea00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7faaf815ed00 con 0x7faaf806f630 2024-04-17T10:03:23.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faafca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf8151db0 0x7faaf815c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:23.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faaefe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaf8072bc0 0x7faaf814b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:23.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faafca00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf8151db0 0x7faaf815c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43200/0 (socket says 172.21.15.32:43200) 2024-04-17T10:03:23.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faafca00640 1 -- 172.21.15.32:0/1613042505 learned_addr learned my addr 172.21.15.32:0/1613042505 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:23.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faaf7e00640 1 --2- 172.21.15.32:0/1613042505 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faaf806f630 0x7faaf8147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:23.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faafca00640 1 -- 172.21.15.32:0/1613042505 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faaf806f630 msgr2=0x7faaf8147ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:23.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faafca00640 1 --2- 172.21.15.32:0/1613042505 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faaf806f630 0x7faaf8147ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faafca00640 1 -- 172.21.15.32:0/1613042505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaf8072bc0 msgr2=0x7faaf814b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:23.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faafca00640 1 --2- 172.21.15.32:0/1613042505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaf8072bc0 0x7faaf814b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faafca00640 1 -- 172.21.15.32:0/1613042505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf815dcf0 con 0x7faaf8151db0 2024-04-17T10:03:23.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faafca00640 1 --2- 172.21.15.32:0/1613042505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf8151db0 0x7faaf815c7b0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7faaf0007e90 tx=0x7faaf000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:23.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faaf4c00640 1 -- 172.21.15.32:0/1613042505 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0014420 con 0x7faaf8151db0 2024-04-17T10:03:23.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faaf4c00640 1 -- 172.21.15.32:0/1613042505 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaf0019070 con 0x7faaf8151db0 2024-04-17T10:03:23.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faafea00640 1 -- 172.21.15.32:0/1613042505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf815dfd0 con 0x7faaf8151db0 2024-04-17T10:03:23.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faaf4c00640 1 -- 172.21.15.32:0/1613042505 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0014820 con 0x7faaf8151db0 2024-04-17T10:03:23.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.676+0000 7faafea00640 1 -- 172.21.15.32:0/1613042505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf81fa9b0 con 0x7faaf8151db0 2024-04-17T10:03:23.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.680+0000 7faafea00640 1 -- 172.21.15.32:0/1613042505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf8068a90 con 0x7faaf8151db0 2024-04-17T10:03:23.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.680+0000 7faaf4c00640 1 -- 172.21.15.32:0/1613042505 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7faaf00149c0 con 0x7faaf8151db0 2024-04-17T10:03:23.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.680+0000 7faaf4c00640 1 --2- 172.21.15.32:0/1613042505 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faad4068380 0x7faad406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:23.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.680+0000 7faaf4c00640 1 -- 172.21.15.32:0/1613042505 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x7faaf00871c0 con 0x7faaf8151db0 2024-04-17T10:03:23.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.680+0000 7faaf7e00640 1 --2- 172.21.15.32:0/1613042505 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faad4068380 0x7faad406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:23.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.680+0000 7faaf7e00640 1 --2- 172.21.15.32:0/1613042505 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faad4068380 0x7faad406a840 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7faae80079c0 tx=0x7faae8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:23.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.684+0000 7faaf4c00640 1 -- 172.21.15.32:0/1613042505 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7faaf0057070 con 0x7faaf8151db0 2024-04-17T10:03:23.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.852+0000 7faafea00640 1 -- 172.21.15.32:0/1613042505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faaf815cf20 con 0x7faaf8151db0 2024-04-17T10:03:23.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.856+0000 7faaf4c00640 1 -- 172.21.15.32:0/1613042505 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16879 (secure 0 0 0) 0x7faaf005b370 con 0x7faaf8151db0 2024-04-17T10:03:23.852 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:03:23.852 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":99,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:03:20.790225+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:01:38.938460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-17T10:01:39.505171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6835/3078068380":"2024-04-18T10:03:20.790196+0000","172.21.15.32:6837/3078068380":"2024-04-18T10:03:20.790196+0000","172.21.15.32:6837/1235126741":"2024-04-18T10:02:44.035918+0000","172.21.15.32:6835/1235126741":"2024-04-18T10:02:44.035918+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:03:23.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.856+0000 7faafea00640 1 -- 172.21.15.32:0/1613042505 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faad4068380 msgr2=0x7faad406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:23.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.856+0000 7faafea00640 1 --2- 172.21.15.32:0/1613042505 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faad4068380 0x7faad406a840 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7faae80079c0 tx=0x7faae8008040 comp rx=0 tx=0).stop 2024-04-17T10:03:23.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.856+0000 7faafea00640 1 -- 172.21.15.32:0/1613042505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf8151db0 msgr2=0x7faaf815c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:23.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.856+0000 7faafea00640 1 --2- 172.21.15.32:0/1613042505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf8151db0 0x7faaf815c7b0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7faaf0007e90 tx=0x7faaf000e4b0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.856+0000 7faafea00640 1 -- 172.21.15.32:0/1613042505 shutdown_connections 2024-04-17T10:03:23.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.856+0000 7faafea00640 1 --2- 172.21.15.32:0/1613042505 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7faad4068380 0x7faad406a840 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.856+0000 7faafea00640 1 --2- 172.21.15.32:0/1613042505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf8151db0 0x7faaf815c7b0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.856+0000 7faafea00640 1 --2- 172.21.15.32:0/1613042505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaf8072bc0 0x7faaf814b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.856+0000 7faafea00640 1 --2- 172.21.15.32:0/1613042505 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faaf806f630 0x7faaf8147ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:23.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.856+0000 7faafea00640 1 -- 172.21.15.32:0/1613042505 >> 172.21.15.32:0/1613042505 conn(0x7faaf806ba30 msgr2=0x7faaf80b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:23.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.856+0000 7faafea00640 1 -- 172.21.15.32:0/1613042505 shutdown_connections 2024-04-17T10:03:23.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:23.856+0000 7faafea00640 1 -- 172.21.15.32:0/1613042505 wait complete. 2024-04-17T10:03:23.874 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T10:03:23.874 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:03:24.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.016+0000 7fd658a00640 1 Processor -- start 2024-04-17T10:03:24.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.016+0000 7fd658a00640 1 -- start start 2024-04-17T10:03:24.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.020+0000 7fd658a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd654144e90 0x7fd6540b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:24.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.020+0000 7fd658a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd6540b4de0 0x7fd65415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:24.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.020+0000 7fd658a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd654072bb0 con 0x7fd65415bfe0 2024-04-17T10:03:24.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.020+0000 7fd658a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd65415c430 con 0x7fd6540b4de0 2024-04-17T10:03:24.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.020+0000 7fd658a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd654145270 con 0x7fd654144e90 2024-04-17T10:03:24.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.020+0000 7fd64be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd6540b4de0 0x7fd65415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:24.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.020+0000 7fd64be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd6540b4de0 0x7fd65415baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43216/0 (socket says 172.21.15.32:43216) 2024-04-17T10:03:24.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.020+0000 7fd64be00640 1 -- 172.21.15.32:0/2586677275 learned_addr learned my addr 172.21.15.32:0/2586677275 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:24.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.020+0000 7fd651e00640 1 --2- 172.21.15.32:0/2586677275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd654144e90 0x7fd6540b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:24.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.020+0000 7fd64be00640 1 -- 172.21.15.32:0/2586677275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd654144e90 msgr2=0x7fd6540b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:24.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.020+0000 7fd64be00640 1 --2- 172.21.15.32:0/2586677275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd654144e90 0x7fd6540b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.020+0000 7fd64be00640 1 -- 172.21.15.32:0/2586677275 >> v1:172.21.15.144:6789/0 conn(0x7fd65415bfe0 legacy=0x7fd65406ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:24.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.020+0000 7fd64be00640 1 -- 172.21.15.32:0/2586677275 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6540b3dc0 con 0x7fd6540b4de0 2024-04-17T10:03:24.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.024+0000 7fd64be00640 1 --2- 172.21.15.32:0/2586677275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6540b4de0 0x7fd65415baa0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0098a0 tx=0x7fd63c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2eaab1ded68e773 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:24.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.024+0000 7fd651400640 1 -- 172.21.15.32:0/2586677275 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0aa670 con 0x7fd6540b4de0 2024-04-17T10:03:24.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.024+0000 7fd651400640 1 -- 172.21.15.32:0/2586677275 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd63c0af070 con 0x7fd6540b4de0 2024-04-17T10:03:24.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.024+0000 7fd651400640 1 -- 172.21.15.32:0/2586677275 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0aaa30 con 0x7fd6540b4de0 2024-04-17T10:03:24.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.024+0000 7fd658a00640 1 -- 172.21.15.32:0/2586677275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6540b4de0 msgr2=0x7fd65415baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:24.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.024+0000 7fd658a00640 1 --2- 172.21.15.32:0/2586677275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6540b4de0 0x7fd65415baa0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0098a0 tx=0x7fd63c01c510 comp rx=0 tx=0).stop 2024-04-17T10:03:24.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.024+0000 7fd658a00640 1 -- 172.21.15.32:0/2586677275 shutdown_connections 2024-04-17T10:03:24.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.024+0000 7fd658a00640 1 --2- 172.21.15.32:0/2586677275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6540b4de0 0x7fd65415baa0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.024+0000 7fd658a00640 1 --2- 172.21.15.32:0/2586677275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd654144e90 0x7fd6540b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.024+0000 7fd658a00640 1 -- 172.21.15.32:0/2586677275 >> 172.21.15.32:0/2586677275 conn(0x7fd65406ba30 msgr2=0x7fd65406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:24.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.024+0000 7fd658a00640 1 -- 172.21.15.32:0/2586677275 shutdown_connections 2024-04-17T10:03:24.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.024+0000 7fd658a00640 1 -- 172.21.15.32:0/2586677275 wait complete. 2024-04-17T10:03:24.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.024+0000 7fd658a00640 1 Processor -- start 2024-04-17T10:03:24.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.024+0000 7fd658a00640 1 -- start start 2024-04-17T10:03:24.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.024+0000 7fd658a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6540b4de0 0x7fd6540b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:24.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.028+0000 7fd658a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd654144e90 0x7fd6540b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:24.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.028+0000 7fd658a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65415bfe0 0x7fd65415a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:24.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.028+0000 7fd658a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd654072670 con 0x7fd65415bfe0 2024-04-17T10:03:24.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.028+0000 7fd658a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd65406f880 con 0x7fd6540b4de0 2024-04-17T10:03:24.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.028+0000 7fd658a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd6540727f0 con 0x7fd654144e90 2024-04-17T10:03:24.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.028+0000 7fd652800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65415bfe0 0x7fd65415a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:24.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.028+0000 7fd651e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6540b4de0 0x7fd6540b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:24.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.028+0000 7fd64be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd654144e90 0x7fd6540b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:24.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.028+0000 7fd652800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65415bfe0 0x7fd65415a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43232/0 (socket says 172.21.15.32:43232) 2024-04-17T10:03:24.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.028+0000 7fd652800640 1 -- 172.21.15.32:0/1029261049 learned_addr learned my addr 172.21.15.32:0/1029261049 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:24.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.028+0000 7fd64be00640 1 -- 172.21.15.32:0/1029261049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6540b4de0 msgr2=0x7fd6540b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:24.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.028+0000 7fd64be00640 1 --2- 172.21.15.32:0/1029261049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6540b4de0 0x7fd6540b1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.028+0000 7fd64be00640 1 -- 172.21.15.32:0/1029261049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65415bfe0 msgr2=0x7fd65415a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:24.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.028+0000 7fd64be00640 1 --2- 172.21.15.32:0/1029261049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65415bfe0 0x7fd65415a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.028+0000 7fd64be00640 1 -- 172.21.15.32:0/1029261049 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6540b21b0 con 0x7fd654144e90 2024-04-17T10:03:24.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.028+0000 7fd652800640 1 --2- 172.21.15.32:0/1029261049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65415bfe0 0x7fd65415a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:03:24.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.028+0000 7fd64be00640 1 --2- 172.21.15.32:0/1029261049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd654144e90 0x7fd6540b2b10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0ab000 tx=0x7fd63c01ad50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:24.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.028+0000 7fd64a000640 1 -- 172.21.15.32:0/1029261049 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0bd5f0 con 0x7fd654144e90 2024-04-17T10:03:24.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.028+0000 7fd64a000640 1 -- 172.21.15.32:0/1029261049 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd63c0bd7b0 con 0x7fd654144e90 2024-04-17T10:03:24.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.028+0000 7fd64a000640 1 -- 172.21.15.32:0/1029261049 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0bdba0 con 0x7fd654144e90 2024-04-17T10:03:24.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.028+0000 7fd658a00640 1 -- 172.21.15.32:0/1029261049 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd65415d320 con 0x7fd654144e90 2024-04-17T10:03:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.028+0000 7fd658a00640 1 -- 172.21.15.32:0/1029261049 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd65415d8e0 con 0x7fd654144e90 2024-04-17T10:03:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.028+0000 7fd658a00640 1 -- 172.21.15.32:0/1029261049 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6540b3050 con 0x7fd654144e90 2024-04-17T10:03:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.032+0000 7fd64a000640 1 -- 172.21.15.32:0/1029261049 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd63c0b33f0 con 0x7fd654144e90 2024-04-17T10:03:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.032+0000 7fd64a000640 1 --2- 172.21.15.32:0/1029261049 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6300683b0 0x7fd63006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.032+0000 7fd64a000640 1 -- 172.21.15.32:0/1029261049 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x7fd63c120e90 con 0x7fd654144e90 2024-04-17T10:03:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.032+0000 7fd651e00640 1 --2- 172.21.15.32:0/1029261049 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6300683b0 0x7fd63006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:24.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.032+0000 7fd651e00640 1 --2- 172.21.15.32:0/1029261049 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6300683b0 0x7fd63006a870 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd644009b60 tx=0x7fd644009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:24.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.036+0000 7fd64a000640 1 -- 172.21.15.32:0/1029261049 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd63c016130 con 0x7fd654144e90 2024-04-17T10:03:24.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.240+0000 7fd658a00640 1 -- 172.21.15.32:0/1029261049 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd654068a90 con 0x7fd654144e90 2024-04-17T10:03:24.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.240+0000 7fd64a000640 1 -- 172.21.15.32:0/1029261049 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5190 (secure 0 0 0) 0x7fd63c0f0dc0 con 0x7fd654144e90 2024-04-17T10:03:24.238 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:03:24.238 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":89,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2635109588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2635109588},{"type":"v1","addr":"172.21.15.144:6835","nonce":2635109588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7789,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3258455939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3258455939},{"type":"v1","addr":"172.21.15.144:6834","nonce":3258455939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":8574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3390468332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3390468332},{"type":"v1","addr":"172.21.15.32:6836","nonce":3390468332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":9123,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/374146310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":374146310},{"type":"v1","addr":"172.21.15.32:6837","nonce":374146310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":88,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:01:40.557208+0000","modified":"2024-04-17T10:03:20.790439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-17T10:03:24.238 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 89 2024-04-17T10:03:24.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.244+0000 7fd658a00640 1 -- 172.21.15.32:0/1029261049 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6300683b0 msgr2=0x7fd63006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:24.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.244+0000 7fd658a00640 1 --2- 172.21.15.32:0/1029261049 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6300683b0 0x7fd63006a870 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd644009b60 tx=0x7fd644009190 comp rx=0 tx=0).stop 2024-04-17T10:03:24.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.244+0000 7fd658a00640 1 -- 172.21.15.32:0/1029261049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd654144e90 msgr2=0x7fd6540b2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:24.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.244+0000 7fd658a00640 1 --2- 172.21.15.32:0/1029261049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd654144e90 0x7fd6540b2b10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0ab000 tx=0x7fd63c01ad50 comp rx=0 tx=0).stop 2024-04-17T10:03:24.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.244+0000 7fd658a00640 1 -- 172.21.15.32:0/1029261049 shutdown_connections 2024-04-17T10:03:24.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.244+0000 7fd658a00640 1 --2- 172.21.15.32:0/1029261049 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd6300683b0 0x7fd63006a870 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.244+0000 7fd658a00640 1 --2- 172.21.15.32:0/1029261049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd654144e90 0x7fd6540b2b10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.244+0000 7fd658a00640 1 --2- 172.21.15.32:0/1029261049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd65415bfe0 0x7fd65415a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.244+0000 7fd658a00640 1 --2- 172.21.15.32:0/1029261049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6540b4de0 0x7fd6540b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.244+0000 7fd658a00640 1 -- 172.21.15.32:0/1029261049 >> 172.21.15.32:0/1029261049 conn(0x7fd65406ba30 msgr2=0x7fd654142d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:24.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.244+0000 7fd658a00640 1 -- 172.21.15.32:0/1029261049 shutdown_connections 2024-04-17T10:03:24.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.244+0000 7fd658a00640 1 -- 172.21.15.32:0/1029261049 wait complete. 2024-04-17T10:03:24.259 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:03:24.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.404+0000 7f44be400640 1 Processor -- start 2024-04-17T10:03:24.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.404+0000 7f44be400640 1 -- start start 2024-04-17T10:03:24.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.404+0000 7f44be400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f44b80b4af0 0x7f44b80d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:24.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.404+0000 7f44be400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44b80ccbd0 con 0x7f44b809b100 2024-04-17T10:03:24.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.404+0000 7f44be400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44b80cceb0 con 0x7f44b815eb30 2024-04-17T10:03:24.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.404+0000 7f44be400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f44b80cd190 con 0x7f44b80b4af0 2024-04-17T10:03:24.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.404+0000 7f44aea00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f44b809b100 0x7f44b80cd990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53238/0 (socket says 172.21.15.32:53238) 2024-04-17T10:03:24.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.404+0000 7f44b7e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f44b80b4af0 0x7f44b80d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:24.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.404+0000 7f44aea00640 1 -- 172.21.15.32:0/4222828716 learned_addr learned my addr 172.21.15.32:0/4222828716 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:24.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.404+0000 7f44b7e00640 1 -- 172.21.15.32:0/4222828716 >> v1:172.21.15.144:6789/0 conn(0x7f44b815eb30 legacy=0x7f44b809a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:24.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.404+0000 7f44b6a00640 1 -- 172.21.15.32:0/4222828716 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3001652082 0 0) 0x7f44b80ccbd0 con 0x7f44b809b100 2024-04-17T10:03:24.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.404+0000 7f44b7e00640 1 -- 172.21.15.32:0/4222828716 >> v1:172.21.15.32:6789/0 conn(0x7f44b809b100 legacy=0x7f44b80cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:24.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.404+0000 7f44b7e00640 1 -- 172.21.15.32:0/4222828716 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b80cd470 con 0x7f44b80b4af0 2024-04-17T10:03:24.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.408+0000 7f44b7e00640 1 --2- 172.21.15.32:0/4222828716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44b80b4af0 0x7f44b80d06b0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009d90 tx=0x7f44b001f120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce7cc37b88271f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:24.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.408+0000 7f44b6a00640 1 -- 172.21.15.32:0/4222828716 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b00b4070 con 0x7f44b80b4af0 2024-04-17T10:03:24.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.408+0000 7f44b6a00640 1 -- 172.21.15.32:0/4222828716 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44b00af600 con 0x7f44b80b4af0 2024-04-17T10:03:24.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.408+0000 7f44b6a00640 1 -- 172.21.15.32:0/4222828716 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b00af9d0 con 0x7f44b80b4af0 2024-04-17T10:03:24.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.408+0000 7f44be400640 1 -- 172.21.15.32:0/4222828716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44b80b4af0 msgr2=0x7f44b80d06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:24.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.408+0000 7f44be400640 1 --2- 172.21.15.32:0/4222828716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44b80b4af0 0x7f44b80d06b0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009d90 tx=0x7f44b001f120 comp rx=0 tx=0).stop 2024-04-17T10:03:24.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.408+0000 7f44be400640 1 -- 172.21.15.32:0/4222828716 shutdown_connections 2024-04-17T10:03:24.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.408+0000 7f44be400640 1 --2- 172.21.15.32:0/4222828716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44b80b4af0 0x7f44b80d06b0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.408+0000 7f44be400640 1 -- 172.21.15.32:0/4222828716 >> 172.21.15.32:0/4222828716 conn(0x7f44b80b3fd0 msgr2=0x7f44b80b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:24.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.408+0000 7f44be400640 1 -- 172.21.15.32:0/4222828716 shutdown_connections 2024-04-17T10:03:24.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.408+0000 7f44be400640 1 -- 172.21.15.32:0/4222828716 wait complete. 2024-04-17T10:03:24.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44be400640 1 Processor -- start 2024-04-17T10:03:24.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44be400640 1 -- start start 2024-04-17T10:03:24.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44be400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44b809b100 0x7f44b80b6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:24.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44be400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44b80b4af0 0x7f44b815a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:24.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44be400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44b815eb30 0x7f44b809fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44be400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f44b80af170 con 0x7f44b80b4af0 2024-04-17T10:03:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44be400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f44b80d1680 con 0x7f44b809b100 2024-04-17T10:03:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44be400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f44b80d19b0 con 0x7f44b815eb30 2024-04-17T10:03:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44aea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44b80b4af0 0x7f44b815a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44aea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44b80b4af0 0x7f44b815a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43246/0 (socket says 172.21.15.32:43246) 2024-04-17T10:03:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44aea00640 1 -- 172.21.15.32:0/1051828903 learned_addr learned my addr 172.21.15.32:0/1051828903 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44b7e00640 1 --2- 172.21.15.32:0/1051828903 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44b815eb30 0x7f44b809fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44b7400640 1 --2- 172.21.15.32:0/1051828903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44b809b100 0x7f44b80b6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44aea00640 1 -- 172.21.15.32:0/1051828903 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44b815eb30 msgr2=0x7f44b809fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44aea00640 1 --2- 172.21.15.32:0/1051828903 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44b815eb30 0x7f44b809fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44aea00640 1 -- 172.21.15.32:0/1051828903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44b809b100 msgr2=0x7f44b80b6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44aea00640 1 --2- 172.21.15.32:0/1051828903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44b809b100 0x7f44b80b6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44aea00640 1 -- 172.21.15.32:0/1051828903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b80b2000 con 0x7f44b80b4af0 2024-04-17T10:03:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44b7400640 1 --2- 172.21.15.32:0/1051828903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44b809b100 0x7f44b80b6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44b7e00640 1 --2- 172.21.15.32:0/1051828903 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44b815eb30 0x7f44b809fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:24.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44aea00640 1 --2- 172.21.15.32:0/1051828903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44b80b4af0 0x7f44b815a9f0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f44a000c520 tx=0x7f44a000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:24.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44afe00640 1 -- 172.21.15.32:0/1051828903 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0018070 con 0x7f44b80b4af0 2024-04-17T10:03:24.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44afe00640 1 -- 172.21.15.32:0/1051828903 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a00074e0 con 0x7f44b80b4af0 2024-04-17T10:03:24.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44afe00640 1 -- 172.21.15.32:0/1051828903 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a00049f0 con 0x7f44b80b4af0 2024-04-17T10:03:24.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44be400640 1 -- 172.21.15.32:0/1051828903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b81f8060 con 0x7f44b80b4af0 2024-04-17T10:03:24.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.412+0000 7f44be400640 1 -- 172.21.15.32:0/1051828903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b81f8620 con 0x7f44b80b4af0 2024-04-17T10:03:24.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.416+0000 7f44be400640 1 -- 172.21.15.32:0/1051828903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b80b21d0 con 0x7f44b80b4af0 2024-04-17T10:03:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.416+0000 7f44afe00640 1 -- 172.21.15.32:0/1051828903 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f44a0004b90 con 0x7f44b80b4af0 2024-04-17T10:03:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.416+0000 7f44afe00640 1 --2- 172.21.15.32:0/1051828903 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4494068380 0x7f449406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.416+0000 7f44afe00640 1 -- 172.21.15.32:0/1051828903 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x7f44a0085cb0 con 0x7f44b80b4af0 2024-04-17T10:03:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.416+0000 7f44b7400640 1 --2- 172.21.15.32:0/1051828903 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4494068380 0x7f449406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.416+0000 7f44b7400640 1 --2- 172.21.15.32:0/1051828903 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4494068380 0x7f449406a840 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f44a80098b0 tx=0x7f44a8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:24.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.420+0000 7f44afe00640 1 -- 172.21.15.32:0/1051828903 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f44a0055c00 con 0x7f44b80b4af0 2024-04-17T10:03:24.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.620+0000 7f44be400640 1 -- 172.21.15.32:0/1051828903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44b8068a90 con 0x7f44b80b4af0 2024-04-17T10:03:24.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.620+0000 7f44afe00640 1 -- 172.21.15.32:0/1051828903 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5190 (secure 0 0 0) 0x7f44a0059f00 con 0x7f44b80b4af0 2024-04-17T10:03:24.617 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:03:24.617 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":89,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2635109588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2635109588},{"type":"v1","addr":"172.21.15.144:6835","nonce":2635109588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7789,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3258455939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3258455939},{"type":"v1","addr":"172.21.15.144:6834","nonce":3258455939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":8574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3390468332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3390468332},{"type":"v1","addr":"172.21.15.32:6836","nonce":3390468332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":9123,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/374146310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":374146310},{"type":"v1","addr":"172.21.15.32:6837","nonce":374146310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":88,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:01:40.557208+0000","modified":"2024-04-17T10:03:20.790439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-17T10:03:24.617 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 89 2024-04-17T10:03:24.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.620+0000 7f44be400640 1 -- 172.21.15.32:0/1051828903 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4494068380 msgr2=0x7f449406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:24.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.620+0000 7f44be400640 1 --2- 172.21.15.32:0/1051828903 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4494068380 0x7f449406a840 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f44a80098b0 tx=0x7f44a8008040 comp rx=0 tx=0).stop 2024-04-17T10:03:24.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.620+0000 7f44be400640 1 -- 172.21.15.32:0/1051828903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44b80b4af0 msgr2=0x7f44b815a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:24.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.620+0000 7f44be400640 1 --2- 172.21.15.32:0/1051828903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44b80b4af0 0x7f44b815a9f0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f44a000c520 tx=0x7f44a000c9e0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.620+0000 7f44be400640 1 -- 172.21.15.32:0/1051828903 shutdown_connections 2024-04-17T10:03:24.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.620+0000 7f44be400640 1 --2- 172.21.15.32:0/1051828903 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4494068380 0x7f449406a840 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.620+0000 7f44be400640 1 --2- 172.21.15.32:0/1051828903 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44b815eb30 0x7f44b809fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.620+0000 7f44be400640 1 --2- 172.21.15.32:0/1051828903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44b80b4af0 0x7f44b815a9f0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.620+0000 7f44be400640 1 --2- 172.21.15.32:0/1051828903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44b809b100 0x7f44b80b6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.620+0000 7f44be400640 1 -- 172.21.15.32:0/1051828903 >> 172.21.15.32:0/1051828903 conn(0x7f44b80b3fd0 msgr2=0x7f44b80b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:24.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.620+0000 7f44be400640 1 -- 172.21.15.32:0/1051828903 shutdown_connections 2024-04-17T10:03:24.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.620+0000 7f44be400640 1 -- 172.21.15.32:0/1051828903 wait complete. 2024-04-17T10:03:24.638 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:03:24.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.780+0000 7fa87fc00640 1 Processor -- start 2024-04-17T10:03:24.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.780+0000 7fa87fc00640 1 -- start start 2024-04-17T10:03:24.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.780+0000 7fa87fc00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa878072500 0x7fa8780cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:24.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.780+0000 7fa87fc00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8780cdac0 con 0x7fa87815d8d0 2024-04-17T10:03:24.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.780+0000 7fa87fc00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa87815cdf0 con 0x7fa878145f30 2024-04-17T10:03:24.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.780+0000 7fa87fc00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa87806f400 con 0x7fa878072500 2024-04-17T10:03:24.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.784+0000 7fa86fe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa878072500 0x7fa8780cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:24.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.784+0000 7fa87dc00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa87815d8d0 0x7fa8781601d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53254/0 (socket says 172.21.15.32:53254) 2024-04-17T10:03:24.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.784+0000 7fa87dc00640 1 -- 172.21.15.32:0/4081864837 learned_addr learned my addr 172.21.15.32:0/4081864837 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:24.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.784+0000 7fa86fe00640 1 -- 172.21.15.32:0/4081864837 >> v1:172.21.15.144:6789/0 conn(0x7fa878145f30 legacy=0x7fa878071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:24.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.784+0000 7fa86fe00640 1 -- 172.21.15.32:0/4081864837 >> v1:172.21.15.32:6789/0 conn(0x7fa87815d8d0 legacy=0x7fa8781601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:24.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.784+0000 7fa86fe00640 1 -- 172.21.15.32:0/4081864837 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87815d0d0 con 0x7fa878072500 2024-04-17T10:03:24.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.784+0000 7fa877e00640 1 -- 172.21.15.32:0/4081864837 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3940390614 0 0) 0x7fa8780cdac0 con 0x7fa87815d8d0 2024-04-17T10:03:24.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.784+0000 7fa86fe00640 1 --2- 172.21.15.32:0/4081864837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa878072500 0x7fa8780cce70 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fa8600097c0 tx=0x7fa86001ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4e41852e285cf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:24.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.784+0000 7fa877e00640 1 -- 172.21.15.32:0/4081864837 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8600b5070 con 0x7fa878072500 2024-04-17T10:03:24.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.784+0000 7fa877e00640 1 -- 172.21.15.32:0/4081864837 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8600a8990 con 0x7fa878072500 2024-04-17T10:03:24.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.784+0000 7fa877e00640 1 -- 172.21.15.32:0/4081864837 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8600a8d60 con 0x7fa878072500 2024-04-17T10:03:24.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa87fc00640 1 -- 172.21.15.32:0/4081864837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa878072500 msgr2=0x7fa8780cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:24.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa87fc00640 1 --2- 172.21.15.32:0/4081864837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa878072500 0x7fa8780cce70 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fa8600097c0 tx=0x7fa86001ec90 comp rx=0 tx=0).stop 2024-04-17T10:03:24.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa87fc00640 1 -- 172.21.15.32:0/4081864837 shutdown_connections 2024-04-17T10:03:24.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa87fc00640 1 --2- 172.21.15.32:0/4081864837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa878072500 0x7fa8780cce70 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa87fc00640 1 -- 172.21.15.32:0/4081864837 >> 172.21.15.32:0/4081864837 conn(0x7fa87806ba30 msgr2=0x7fa87806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:24.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa87fc00640 1 -- 172.21.15.32:0/4081864837 shutdown_connections 2024-04-17T10:03:24.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa87fc00640 1 -- 172.21.15.32:0/4081864837 wait complete. 2024-04-17T10:03:24.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa87fc00640 1 Processor -- start 2024-04-17T10:03:24.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa87fc00640 1 -- start start 2024-04-17T10:03:24.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa87fc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa878072500 0x7fa8781f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:24.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa87fc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa878145f30 0x7fa8781e6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:24.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa87fc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa87815d8d0 0x7fa8780b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:24.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa87fc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa8780cd800 con 0x7fa878145f30 2024-04-17T10:03:24.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa87fc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa8780cd680 con 0x7fa878072500 2024-04-17T10:03:24.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa87fc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa878146310 con 0x7fa87815d8d0 2024-04-17T10:03:24.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa86fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa878145f30 0x7fa8781e6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:24.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa86fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa878145f30 0x7fa8781e6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43252/0 (socket says 172.21.15.32:43252) 2024-04-17T10:03:24.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa86fe00640 1 -- 172.21.15.32:0/2161019767 learned_addr learned my addr 172.21.15.32:0/2161019767 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:24.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa87d200640 1 --2- 172.21.15.32:0/2161019767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa878072500 0x7fa8781f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:24.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa87dc00640 1 --2- 172.21.15.32:0/2161019767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa87815d8d0 0x7fa8780b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:24.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa86fe00640 1 -- 172.21.15.32:0/2161019767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa87815d8d0 msgr2=0x7fa8780b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:24.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa86fe00640 1 --2- 172.21.15.32:0/2161019767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa87815d8d0 0x7fa8780b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa86fe00640 1 -- 172.21.15.32:0/2161019767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa878072500 msgr2=0x7fa8781f6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:24.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa86fe00640 1 --2- 172.21.15.32:0/2161019767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa878072500 0x7fa8781f6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.788+0000 7fa86fe00640 1 -- 172.21.15.32:0/2161019767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8781e7390 con 0x7fa878145f30 2024-04-17T10:03:24.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.792+0000 7fa86fe00640 1 --2- 172.21.15.32:0/2161019767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa878145f30 0x7fa8781e6a40 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa86001b8a0 tx=0x7fa86001ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:24.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.792+0000 7fa875600640 1 -- 172.21.15.32:0/2161019767 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8600b5040 con 0x7fa878145f30 2024-04-17T10:03:24.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.792+0000 7fa875600640 1 -- 172.21.15.32:0/2161019767 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8600bf070 con 0x7fa878145f30 2024-04-17T10:03:24.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.792+0000 7fa875600640 1 -- 172.21.15.32:0/2161019767 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8600b0b60 con 0x7fa878145f30 2024-04-17T10:03:24.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.792+0000 7fa87fc00640 1 -- 172.21.15.32:0/2161019767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8781e7670 con 0x7fa878145f30 2024-04-17T10:03:24.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.792+0000 7fa87fc00640 1 -- 172.21.15.32:0/2161019767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8781e7c30 con 0x7fa878145f30 2024-04-17T10:03:24.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.792+0000 7fa87fc00640 1 -- 172.21.15.32:0/2161019767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa878068a90 con 0x7fa878145f30 2024-04-17T10:03:24.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.792+0000 7fa875600640 1 -- 172.21.15.32:0/2161019767 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa8600b0d00 con 0x7fa878145f30 2024-04-17T10:03:24.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.792+0000 7fa875600640 1 --2- 172.21.15.32:0/2161019767 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa854068380 0x7fa85406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:24.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.792+0000 7fa875600640 1 -- 172.21.15.32:0/2161019767 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x7fa860123f50 con 0x7fa878145f30 2024-04-17T10:03:24.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.796+0000 7fa87d200640 1 --2- 172.21.15.32:0/2161019767 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa854068380 0x7fa85406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:24.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.796+0000 7fa87d200640 1 --2- 172.21.15.32:0/2161019767 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa854068380 0x7fa85406a840 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa868007fd0 tx=0x7fa868009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:24.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.796+0000 7fa875600640 1 -- 172.21.15.32:0/2161019767 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa8600187f0 con 0x7fa878145f30 2024-04-17T10:03:24.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.964+0000 7fa87fc00640 1 -- 172.21.15.32:0/2161019767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa878072f90 con 0x7fa878145f30 2024-04-17T10:03:24.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.964+0000 7fa875600640 1 -- 172.21.15.32:0/2161019767 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16879 (secure 0 0 0) 0x7fa8600f3d70 con 0x7fa878145f30 2024-04-17T10:03:24.961 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:03:24.962 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":99,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:03:20.790225+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:01:38.938460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-17T10:01:39.505171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6835/3078068380":"2024-04-18T10:03:20.790196+0000","172.21.15.32:6837/3078068380":"2024-04-18T10:03:20.790196+0000","172.21.15.32:6837/1235126741":"2024-04-18T10:02:44.035918+0000","172.21.15.32:6835/1235126741":"2024-04-18T10:02:44.035918+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:03:24.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.964+0000 7fa87fc00640 1 -- 172.21.15.32:0/2161019767 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa854068380 msgr2=0x7fa85406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:24.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.964+0000 7fa87fc00640 1 --2- 172.21.15.32:0/2161019767 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa854068380 0x7fa85406a840 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa868007fd0 tx=0x7fa868009040 comp rx=0 tx=0).stop 2024-04-17T10:03:24.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.964+0000 7fa87fc00640 1 -- 172.21.15.32:0/2161019767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa878145f30 msgr2=0x7fa8781e6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:24.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.964+0000 7fa87fc00640 1 --2- 172.21.15.32:0/2161019767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa878145f30 0x7fa8781e6a40 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa86001b8a0 tx=0x7fa86001ba30 comp rx=0 tx=0).stop 2024-04-17T10:03:24.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.964+0000 7fa87fc00640 1 -- 172.21.15.32:0/2161019767 shutdown_connections 2024-04-17T10:03:24.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.964+0000 7fa87fc00640 1 --2- 172.21.15.32:0/2161019767 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa854068380 0x7fa85406a840 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.964+0000 7fa87fc00640 1 --2- 172.21.15.32:0/2161019767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa87815d8d0 0x7fa8780b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.964+0000 7fa87fc00640 1 --2- 172.21.15.32:0/2161019767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa878145f30 0x7fa8781e6a40 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.964+0000 7fa87fc00640 1 --2- 172.21.15.32:0/2161019767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa878072500 0x7fa8781f6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:24.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.968+0000 7fa87fc00640 1 -- 172.21.15.32:0/2161019767 >> 172.21.15.32:0/2161019767 conn(0x7fa87806ba30 msgr2=0x7fa87806be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:24.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.968+0000 7fa87fc00640 1 -- 172.21.15.32:0/2161019767 shutdown_connections 2024-04-17T10:03:24.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:24.968+0000 7fa87fc00640 1 -- 172.21.15.32:0/2161019767 wait complete. 2024-04-17T10:03:24.983 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T10:03:25.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f9400640 1 Processor -- start 2024-04-17T10:03:25.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f9400640 1 -- start start 2024-04-17T10:03:25.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f9400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f40ce580 0x7f46f4169390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:25.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f9400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46f415ec80 con 0x7f46f415e8a0 2024-04-17T10:03:25.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f9400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46f40ccad0 con 0x7f46f406cb40 2024-04-17T10:03:25.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f9400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f46f40d3200 con 0x7f46f40ce580 2024-04-17T10:03:25.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f3e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f46f415e8a0 0x7f46f40d0570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53258/0 (socket says 172.21.15.32:53258) 2024-04-17T10:03:25.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f3e00640 1 -- 172.21.15.32:0/3267887685 learned_addr learned my addr 172.21.15.32:0/3267887685 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:25.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f8a00640 1 --2- 172.21.15.32:0/3267887685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f40ce580 0x7f46f4169390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:25.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f3400640 1 -- 172.21.15.32:0/3267887685 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2506554842 0 0) 0x7f46f415ec80 con 0x7f46f415e8a0 2024-04-17T10:03:25.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f3400640 1 -- 172.21.15.32:0/3267887685 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46d8003530 con 0x7f46f415e8a0 2024-04-17T10:03:25.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f3400640 1 -- 172.21.15.32:0/3267887685 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 560745164 0 0) 0x7f46d8003530 con 0x7f46f415e8a0 2024-04-17T10:03:25.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f3400640 1 -- 172.21.15.32:0/3267887685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f40ce580 msgr2=0x7f46f4169390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:25.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f3400640 1 --2- 172.21.15.32:0/3267887685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f40ce580 0x7f46f4169390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:25.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f3400640 1 -- 172.21.15.32:0/3267887685 >> v1:172.21.15.144:6789/0 conn(0x7f46f406cb40 legacy=0x7f46f40d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:25.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f3400640 1 -- 172.21.15.32:0/3267887685 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f406cfd0 con 0x7f46f415e8a0 2024-04-17T10:03:25.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f3400640 1 -- 172.21.15.32:0/3267887685 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f46e40016a0 con 0x7f46f415e8a0 2024-04-17T10:03:25.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f8a00640 1 --2- 172.21.15.32:0/3267887685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f40ce580 0x7f46f4169390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:03:25.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f3400640 1 -- 172.21.15.32:0/3267887685 >> v1:172.21.15.32:6789/0 conn(0x7f46f415e8a0 legacy=0x7f46f40d0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:25.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f3400640 1 --2- 172.21.15.32:0/3267887685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46d80039f0 0x7f46d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:25.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f3400640 1 --2- 172.21.15.32:0/3267887685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46d80043a0 0x7f46d8004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:25.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f3400640 1 --2- 172.21.15.32:0/3267887685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f40ce580 0x7f46d8005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:25.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f3400640 1 -- 172.21.15.32:0/3267887685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f46d800b9d0 con 0x7f46d80039f0 2024-04-17T10:03:25.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f3400640 1 -- 172.21.15.32:0/3267887685 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f46d800bb50 con 0x7f46d80043a0 2024-04-17T10:03:25.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f3400640 1 -- 172.21.15.32:0/3267887685 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f46d800bd00 con 0x7f46f40ce580 2024-04-17T10:03:25.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.128+0000 7f46f3e00640 1 --2- 172.21.15.32:0/3267887685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46d80039f0 0x7f46d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:25.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.132+0000 7f46ebe00640 1 --2- 172.21.15.32:0/3267887685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46d80043a0 0x7f46d8004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:25.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.132+0000 7f46f8a00640 1 --2- 172.21.15.32:0/3267887685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f40ce580 0x7f46d8005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:25.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.132+0000 7f46f3e00640 1 -- 172.21.15.32:0/3267887685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f40ce580 msgr2=0x7f46d8005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:25.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.132+0000 7f46f3e00640 1 --2- 172.21.15.32:0/3267887685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f40ce580 0x7f46d8005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:25.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.132+0000 7f46f3e00640 1 -- 172.21.15.32:0/3267887685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46d80043a0 msgr2=0x7f46d8004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:25.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.132+0000 7f46f3e00640 1 --2- 172.21.15.32:0/3267887685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46d80043a0 0x7f46d8004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:25.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.132+0000 7f46f3e00640 1 -- 172.21.15.32:0/3267887685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d8003530 con 0x7f46d80039f0 2024-04-17T10:03:25.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.132+0000 7f46f3e00640 1 --2- 172.21.15.32:0/3267887685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46d80039f0 0x7f46d8003dd0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f46e400c210 tx=0x7f46e401cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6027df1d32b40042 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:25.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.132+0000 7f46f3400640 1 -- 172.21.15.32:0/3267887685 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4007d70 con 0x7f46d80039f0 2024-04-17T10:03:25.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.132+0000 7f46f3400640 1 -- 172.21.15.32:0/3267887685 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e40a9070 con 0x7f46d80039f0 2024-04-17T10:03:25.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.132+0000 7f46f3400640 1 -- 172.21.15.32:0/3267887685 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e40a5d90 con 0x7f46d80039f0 2024-04-17T10:03:25.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.132+0000 7f46f9400640 1 -- 172.21.15.32:0/3267887685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46d80039f0 msgr2=0x7f46d8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:25.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.132+0000 7f46f9400640 1 --2- 172.21.15.32:0/3267887685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46d80039f0 0x7f46d8003dd0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f46e400c210 tx=0x7f46e401cc70 comp rx=0 tx=0).stop 2024-04-17T10:03:25.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.132+0000 7f46f8a00640 1 -- 172.21.15.32:0/3267887685 reap_dead start 2024-04-17T10:03:25.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.132+0000 7f46f9400640 1 -- 172.21.15.32:0/3267887685 shutdown_connections 2024-04-17T10:03:25.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.132+0000 7f46f9400640 1 -- 172.21.15.32:0/3267887685 >> 172.21.15.32:0/3267887685 conn(0x7f46f40d23e0 msgr2=0x7f46f40d27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:25.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.132+0000 7f46f9400640 1 -- 172.21.15.32:0/3267887685 shutdown_connections 2024-04-17T10:03:25.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46f9400640 1 -- 172.21.15.32:0/3267887685 wait complete. 2024-04-17T10:03:25.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46f9400640 1 Processor -- start 2024-04-17T10:03:25.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46f9400640 1 -- start start 2024-04-17T10:03:25.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46f9400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46d80039f0 0x7f46f40b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:25.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46f9400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46f41571b0 0x7f46f40b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46f9400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46f40b5500 0x7f46f415ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46f9400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f46f40d1390 con 0x7f46f40b5500 2024-04-17T10:03:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46f9400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f46f406d2b0 con 0x7f46f41571b0 2024-04-17T10:03:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46f9400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f46f4169ba0 con 0x7f46d80039f0 2024-04-17T10:03:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46f8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46f40b5500 0x7f46f415ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46f8a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46f40b5500 0x7f46f415ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43278/0 (socket says 172.21.15.32:43278) 2024-04-17T10:03:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46f3e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46d80039f0 0x7f46f40b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46f8a00640 1 -- 172.21.15.32:0/164848743 learned_addr learned my addr 172.21.15.32:0/164848743 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46ebe00640 1 --2- 172.21.15.32:0/164848743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46f41571b0 0x7f46f40b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46f8a00640 1 -- 172.21.15.32:0/164848743 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46d80039f0 msgr2=0x7f46f40b7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46f8a00640 1 --2- 172.21.15.32:0/164848743 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46d80039f0 0x7f46f40b7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46f8a00640 1 -- 172.21.15.32:0/164848743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46f41571b0 msgr2=0x7f46f40b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46f8a00640 1 --2- 172.21.15.32:0/164848743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46f41571b0 0x7f46f40b4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:25.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46f8a00640 1 -- 172.21.15.32:0/164848743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f40b6810 con 0x7f46f40b5500 2024-04-17T10:03:25.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46f8a00640 1 --2- 172.21.15.32:0/164848743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46f40b5500 0x7f46f415ec80 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009ed0 tx=0x7f46ec0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:25.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46f0c00640 1 -- 172.21.15.32:0/164848743 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec011ea0 con 0x7f46f40b5500 2024-04-17T10:03:25.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46f0c00640 1 -- 172.21.15.32:0/164848743 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ec01b070 con 0x7f46f40b5500 2024-04-17T10:03:25.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46f0c00640 1 -- 172.21.15.32:0/164848743 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec0039a0 con 0x7f46f40b5500 2024-04-17T10:03:25.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.136+0000 7f46f9400640 1 -- 172.21.15.32:0/164848743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f40b6af0 con 0x7f46f40b5500 2024-04-17T10:03:25.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.140+0000 7f46f9400640 1 -- 172.21.15.32:0/164848743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f40b70b0 con 0x7f46f40b5500 2024-04-17T10:03:25.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.140+0000 7f46f9400640 1 -- 172.21.15.32:0/164848743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46f404fd00 con 0x7f46f40b5500 2024-04-17T10:03:25.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.140+0000 7f46f0c00640 1 -- 172.21.15.32:0/164848743 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f46ec003b40 con 0x7f46f40b5500 2024-04-17T10:03:25.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.140+0000 7f46f0c00640 1 --2- 172.21.15.32:0/164848743 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f46d0068380 0x7f46d006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:25.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.140+0000 7f46f0c00640 1 -- 172.21.15.32:0/164848743 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x7f46ec087210 con 0x7f46f40b5500 2024-04-17T10:03:25.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.140+0000 7f46f3e00640 1 --2- 172.21.15.32:0/164848743 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f46d0068380 0x7f46d006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:25.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.140+0000 7f46f3e00640 1 --2- 172.21.15.32:0/164848743 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f46d0068380 0x7f46d006a840 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f46e40ab000 tx=0x7f46e40a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:25.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.144+0000 7f46f0c00640 1 -- 172.21.15.32:0/164848743 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f46ec025020 con 0x7f46f40b5500 2024-04-17T10:03:25.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:25.352+0000 7f46f9400640 1 -- 172.21.15.32:0/164848743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f46f4070450 con 0x7f46f40b5500 2024-04-17T10:03:26.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.152+0000 7f46f0c00640 1 -- 172.21.15.32:0/164848743 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v90) ==== 161+0+0 (secure 0 0 0) 0x7f46ec057030 con 0x7f46f40b5500 2024-04-17T10:03:26.151 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T10:03:26.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.156+0000 7f46f9400640 1 -- 172.21.15.32:0/164848743 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f46d0068380 msgr2=0x7f46d006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:26.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.156+0000 7f46f9400640 1 --2- 172.21.15.32:0/164848743 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f46d0068380 0x7f46d006a840 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f46e40ab000 tx=0x7f46e40a7070 comp rx=0 tx=0).stop 2024-04-17T10:03:26.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.156+0000 7f46f9400640 1 -- 172.21.15.32:0/164848743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46f40b5500 msgr2=0x7f46f415ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:26.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.156+0000 7f46f9400640 1 --2- 172.21.15.32:0/164848743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46f40b5500 0x7f46f415ec80 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009ed0 tx=0x7f46ec0115a0 comp rx=0 tx=0).stop 2024-04-17T10:03:26.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.156+0000 7f46f9400640 1 -- 172.21.15.32:0/164848743 shutdown_connections 2024-04-17T10:03:26.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.156+0000 7f46f9400640 1 --2- 172.21.15.32:0/164848743 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f46d0068380 0x7f46d006a840 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:26.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.156+0000 7f46f9400640 1 --2- 172.21.15.32:0/164848743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46f40b5500 0x7f46f415ec80 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:26.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.156+0000 7f46f9400640 1 --2- 172.21.15.32:0/164848743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46f41571b0 0x7f46f40b4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:26.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.156+0000 7f46f9400640 1 --2- 172.21.15.32:0/164848743 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46d80039f0 0x7f46f40b7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:26.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.156+0000 7f46f9400640 1 -- 172.21.15.32:0/164848743 >> 172.21.15.32:0/164848743 conn(0x7f46f40d23e0 msgr2=0x7f46f409a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:26.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.156+0000 7f46f9400640 1 -- 172.21.15.32:0/164848743 shutdown_connections 2024-04-17T10:03:26.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.156+0000 7f46f9400640 1 -- 172.21.15.32:0/164848743 wait complete. 2024-04-17T10:03:26.175 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-17T10:03:26.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.324+0000 7f43a7a00640 1 Processor -- start 2024-04-17T10:03:26.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.324+0000 7f43a7a00640 1 -- start start 2024-04-17T10:03:26.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.328+0000 7f43a7a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43a015b7d0 0x7f43a006e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:26.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.328+0000 7f43a7a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43a006f7c0 0x7f43a006ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:26.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.328+0000 7f43a7a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43a00cc940 con 0x7f43a006f270 2024-04-17T10:03:26.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.328+0000 7f43a7a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f43a00ccc20 con 0x7f43a006f7c0 2024-04-17T10:03:26.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.328+0000 7f43a7a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f43a006e0b0 con 0x7f43a015b7d0 2024-04-17T10:03:26.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.328+0000 7f4397e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43a006f7c0 0x7f43a006ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:26.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.328+0000 7f4397e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43a006f7c0 0x7f43a006ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43282/0 (socket says 172.21.15.32:43282) 2024-04-17T10:03:26.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.328+0000 7f4397e00640 1 -- 172.21.15.32:0/427112838 learned_addr learned my addr 172.21.15.32:0/427112838 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:26.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.328+0000 7f43a5000640 1 --2- 172.21.15.32:0/427112838 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43a015b7d0 0x7f43a006e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:26.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.328+0000 7f4397e00640 1 -- 172.21.15.32:0/427112838 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43a015b7d0 msgr2=0x7f43a006e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:26.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.328+0000 7f4397e00640 1 --2- 172.21.15.32:0/427112838 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43a015b7d0 0x7f43a006e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:26.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.328+0000 7f4397e00640 1 -- 172.21.15.32:0/427112838 >> v1:172.21.15.144:6789/0 conn(0x7f43a006f270 legacy=0x7f43a00ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:26.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.328+0000 7f4397e00640 1 -- 172.21.15.32:0/427112838 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a006e390 con 0x7f43a006f7c0 2024-04-17T10:03:26.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.332+0000 7f4397e00640 1 --2- 172.21.15.32:0/427112838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43a006f7c0 0x7f43a006ba30 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f4388009870 tx=0x7f438801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f682f1efc3ab8e5f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:26.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.332+0000 7f439fe00640 1 -- 172.21.15.32:0/427112838 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43880aa650 con 0x7f43a006f7c0 2024-04-17T10:03:26.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.332+0000 7f439fe00640 1 -- 172.21.15.32:0/427112838 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43880af070 con 0x7f43a006f7c0 2024-04-17T10:03:26.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.332+0000 7f439fe00640 1 -- 172.21.15.32:0/427112838 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43880aaa10 con 0x7f43a006f7c0 2024-04-17T10:03:26.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.332+0000 7f43a7a00640 1 -- 172.21.15.32:0/427112838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43a006f7c0 msgr2=0x7f43a006ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:26.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.332+0000 7f43a7a00640 1 --2- 172.21.15.32:0/427112838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43a006f7c0 0x7f43a006ba30 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f4388009870 tx=0x7f438801c4d0 comp rx=0 tx=0).stop 2024-04-17T10:03:26.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.332+0000 7f43a7a00640 1 -- 172.21.15.32:0/427112838 shutdown_connections 2024-04-17T10:03:26.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.332+0000 7f43a7a00640 1 --2- 172.21.15.32:0/427112838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43a006f7c0 0x7f43a006ba30 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:26.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.332+0000 7f43a7a00640 1 --2- 172.21.15.32:0/427112838 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43a015b7d0 0x7f43a006e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:26.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.332+0000 7f43a7a00640 1 -- 172.21.15.32:0/427112838 >> 172.21.15.32:0/427112838 conn(0x7f43a00b2120 msgr2=0x7f43a00b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:26.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.332+0000 7f43a7a00640 1 -- 172.21.15.32:0/427112838 shutdown_connections 2024-04-17T10:03:26.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.332+0000 7f43a7a00640 1 -- 172.21.15.32:0/427112838 wait complete. 2024-04-17T10:03:26.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.332+0000 7f43a7a00640 1 Processor -- start 2024-04-17T10:03:26.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.332+0000 7f43a7a00640 1 -- start start 2024-04-17T10:03:26.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f43a7a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43a006f270 0x7f43a0169630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:26.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f43a7a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43a006f7c0 0x7f43a0149fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:26.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f43a7a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43a015b7d0 0x7f43a00aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:26.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f43a7a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f43a0071e60 con 0x7f43a006f270 2024-04-17T10:03:26.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f43a7a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f43a006c380 con 0x7f43a006f7c0 2024-04-17T10:03:26.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f43a7a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f43a0071fe0 con 0x7f43a015b7d0 2024-04-17T10:03:26.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f43a5000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43a006f270 0x7f43a0169630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:26.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f43a5000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43a006f270 0x7f43a0169630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43292/0 (socket says 172.21.15.32:43292) 2024-04-17T10:03:26.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f43a5000640 1 -- 172.21.15.32:0/966154522 learned_addr learned my addr 172.21.15.32:0/966154522 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:26.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f43a5a00640 1 --2- 172.21.15.32:0/966154522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43a015b7d0 0x7f43a00aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:26.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f4397e00640 1 --2- 172.21.15.32:0/966154522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43a006f7c0 0x7f43a0149fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:26.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f43a5000640 1 -- 172.21.15.32:0/966154522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43a015b7d0 msgr2=0x7f43a00aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:26.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f43a5000640 1 --2- 172.21.15.32:0/966154522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43a015b7d0 0x7f43a00aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:26.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f43a5000640 1 -- 172.21.15.32:0/966154522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43a006f7c0 msgr2=0x7f43a0149fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:26.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f43a5000640 1 --2- 172.21.15.32:0/966154522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43a006f7c0 0x7f43a0149fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:26.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f43a5000640 1 -- 172.21.15.32:0/966154522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a00afa10 con 0x7f43a006f270 2024-04-17T10:03:26.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f43a5a00640 1 --2- 172.21.15.32:0/966154522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43a015b7d0 0x7f43a00aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:03:26.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f4397e00640 1 --2- 172.21.15.32:0/966154522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43a006f7c0 0x7f43a0149fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:03:26.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f43a5000640 1 --2- 172.21.15.32:0/966154522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43a006f270 0x7f43a0169630 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f439000e870 tx=0x7f439000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:26.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f439d600640 1 -- 172.21.15.32:0/966154522 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390017070 con 0x7f43a006f270 2024-04-17T10:03:26.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f439d600640 1 -- 172.21.15.32:0/966154522 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f439000c940 con 0x7f43a006f270 2024-04-17T10:03:26.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f439d600640 1 -- 172.21.15.32:0/966154522 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439000ccf0 con 0x7f43a006f270 2024-04-17T10:03:26.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f43a7a00640 1 -- 172.21.15.32:0/966154522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a0168b50 con 0x7f43a006f270 2024-04-17T10:03:26.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f43a7a00640 1 -- 172.21.15.32:0/966154522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a00afcf0 con 0x7f43a006f270 2024-04-17T10:03:26.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.336+0000 7f43a7a00640 1 -- 172.21.15.32:0/966154522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43a0068a90 con 0x7f43a006f270 2024-04-17T10:03:26.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.340+0000 7f439d600640 1 -- 172.21.15.32:0/966154522 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f439001a070 con 0x7f43a006f270 2024-04-17T10:03:26.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.340+0000 7f439d600640 1 --2- 172.21.15.32:0/966154522 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f437c068380 0x7f437c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:26.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.340+0000 7f439d600640 1 -- 172.21.15.32:0/966154522 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x7f4390086ce0 con 0x7f43a006f270 2024-04-17T10:03:26.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.340+0000 7f4397e00640 1 --2- 172.21.15.32:0/966154522 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f437c068380 0x7f437c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:26.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.340+0000 7f4397e00640 1 --2- 172.21.15.32:0/966154522 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f437c068380 0x7f437c06a840 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f43880ae000 tx=0x7f4388008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:26.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.344+0000 7f439d600640 1 -- 172.21.15.32:0/966154522 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f43900571e0 con 0x7f43a006f270 2024-04-17T10:03:26.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:26.548+0000 7f43a7a00640 1 -- 172.21.15.32:0/966154522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f43a00b4ff0 con 0x7f43a006f270 2024-04-17T10:03:27.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.156+0000 7f439d600640 1 -- 172.21.15.32:0/966154522 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v91) ==== 104+0+0 (secure 0 0 0) 0x7f4390056ae0 con 0x7f43a006f270 2024-04-17T10:03:27.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.156+0000 7f43a7a00640 1 -- 172.21.15.32:0/966154522 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f437c068380 msgr2=0x7f437c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:27.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.156+0000 7f43a7a00640 1 --2- 172.21.15.32:0/966154522 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f437c068380 0x7f437c06a840 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f43880ae000 tx=0x7f4388008040 comp rx=0 tx=0).stop 2024-04-17T10:03:27.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.156+0000 7f43a7a00640 1 -- 172.21.15.32:0/966154522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43a006f270 msgr2=0x7f43a0169630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:27.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.156+0000 7f43a7a00640 1 --2- 172.21.15.32:0/966154522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43a006f270 0x7f43a0169630 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f439000e870 tx=0x7f439000ec70 comp rx=0 tx=0).stop 2024-04-17T10:03:27.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.156+0000 7f43a7a00640 1 -- 172.21.15.32:0/966154522 shutdown_connections 2024-04-17T10:03:27.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.156+0000 7f43a7a00640 1 --2- 172.21.15.32:0/966154522 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f437c068380 0x7f437c06a840 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:27.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.156+0000 7f43a7a00640 1 --2- 172.21.15.32:0/966154522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43a015b7d0 0x7f43a00aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:27.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.156+0000 7f43a7a00640 1 --2- 172.21.15.32:0/966154522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43a006f7c0 0x7f43a0149fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:27.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.156+0000 7f43a7a00640 1 --2- 172.21.15.32:0/966154522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43a006f270 0x7f43a0169630 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:27.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.156+0000 7f43a7a00640 1 -- 172.21.15.32:0/966154522 >> 172.21.15.32:0/966154522 conn(0x7f43a00b2120 msgr2=0x7f43a0072300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:27.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.156+0000 7f43a7a00640 1 -- 172.21.15.32:0/966154522 shutdown_connections 2024-04-17T10:03:27.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.156+0000 7f43a7a00640 1 -- 172.21.15.32:0/966154522 wait complete. 2024-04-17T10:03:27.180 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-17T10:03:27.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.332+0000 7f5d57200640 1 Processor -- start 2024-04-17T10:03:27.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.332+0000 7f5d57200640 1 -- start start 2024-04-17T10:03:27.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.332+0000 7f5d57200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d580b8050 0x7f5d5806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:27.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.332+0000 7f5d57200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d580ccac0 con 0x7f5d5809b100 2024-04-17T10:03:27.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.332+0000 7f5d57200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d580ccd30 con 0x7f5d5806d020 2024-04-17T10:03:27.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.332+0000 7f5d57200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5d5806d440 con 0x7f5d580b8050 2024-04-17T10:03:27.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.332+0000 7f5d56800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5d5809b100 0x7f5d580cd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53270/0 (socket says 172.21.15.32:53270) 2024-04-17T10:03:27.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.332+0000 7f5d56800640 1 -- 172.21.15.32:0/3198897154 learned_addr learned my addr 172.21.15.32:0/3198897154 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:27.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.332+0000 7f5d55e00640 1 --2- 172.21.15.32:0/3198897154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d580b8050 0x7f5d5806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:27.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.332+0000 7f5d55400640 1 -- 172.21.15.32:0/3198897154 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1426786721 0 0) 0x7f5d580ccac0 con 0x7f5d5809b100 2024-04-17T10:03:27.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.332+0000 7f5d55400640 1 -- 172.21.15.32:0/3198897154 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d3c003530 con 0x7f5d5809b100 2024-04-17T10:03:27.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.332+0000 7f5d55400640 1 -- 172.21.15.32:0/3198897154 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2620071635 0 0) 0x7f5d580ccd30 con 0x7f5d5806d020 2024-04-17T10:03:27.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.332+0000 7f5d55400640 1 -- 172.21.15.32:0/3198897154 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d580ccac0 con 0x7f5d5806d020 2024-04-17T10:03:27.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.332+0000 7f5d55e00640 1 -- 172.21.15.32:0/3198897154 >> v1:172.21.15.144:6789/0 conn(0x7f5d5806d020 legacy=0x7f5d5809a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:27.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.336+0000 7f5d55e00640 1 -- 172.21.15.32:0/3198897154 >> v1:172.21.15.32:6789/0 conn(0x7f5d5809b100 legacy=0x7f5d580cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:27.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.336+0000 7f5d55e00640 1 -- 172.21.15.32:0/3198897154 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d580cd010 con 0x7f5d580b8050 2024-04-17T10:03:27.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.336+0000 7f5d55e00640 1 --2- 172.21.15.32:0/3198897154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d580b8050 0x7f5d5806e000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009df0 tx=0x7f5d4801f180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3081134b122fa9bb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:27.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.336+0000 7f5d55400640 1 -- 172.21.15.32:0/3198897154 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d480b4070 con 0x7f5d580b8050 2024-04-17T10:03:27.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.336+0000 7f5d55400640 1 -- 172.21.15.32:0/3198897154 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d480af600 con 0x7f5d580b8050 2024-04-17T10:03:27.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.336+0000 7f5d55400640 1 -- 172.21.15.32:0/3198897154 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d480af9d0 con 0x7f5d580b8050 2024-04-17T10:03:27.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.336+0000 7f5d57200640 1 -- 172.21.15.32:0/3198897154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d580b8050 msgr2=0x7f5d5806e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:27.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.336+0000 7f5d57200640 1 --2- 172.21.15.32:0/3198897154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d580b8050 0x7f5d5806e000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009df0 tx=0x7f5d4801f180 comp rx=0 tx=0).stop 2024-04-17T10:03:27.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.336+0000 7f5d57200640 1 -- 172.21.15.32:0/3198897154 shutdown_connections 2024-04-17T10:03:27.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.336+0000 7f5d57200640 1 --2- 172.21.15.32:0/3198897154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d580b8050 0x7f5d5806e000 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:27.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.336+0000 7f5d57200640 1 -- 172.21.15.32:0/3198897154 >> 172.21.15.32:0/3198897154 conn(0x7f5d580b3ec0 msgr2=0x7f5d580b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:27.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.336+0000 7f5d57200640 1 -- 172.21.15.32:0/3198897154 shutdown_connections 2024-04-17T10:03:27.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.336+0000 7f5d57200640 1 -- 172.21.15.32:0/3198897154 wait complete. 2024-04-17T10:03:27.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d57200640 1 Processor -- start 2024-04-17T10:03:27.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d57200640 1 -- start start 2024-04-17T10:03:27.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d57200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d5806d020 0x7f5d580c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:27.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d57200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d5809b100 0x7f5d58148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:27.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d57200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d580b8050 0x7f5d5815f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:27.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d57200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5d580b5220 con 0x7f5d580b8050 2024-04-17T10:03:27.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d57200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5809bc50 con 0x7f5d5809b100 2024-04-17T10:03:27.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d57200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5d5806e600 con 0x7f5d5806d020 2024-04-17T10:03:27.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d56800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d580b8050 0x7f5d5815f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:27.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d56800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d580b8050 0x7f5d5815f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43308/0 (socket says 172.21.15.32:43308) 2024-04-17T10:03:27.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d56800640 1 -- 172.21.15.32:0/441810791 learned_addr learned my addr 172.21.15.32:0/441810791 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:27.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d55e00640 1 --2- 172.21.15.32:0/441810791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d5806d020 0x7f5d580c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:27.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d4d400640 1 --2- 172.21.15.32:0/441810791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d5809b100 0x7f5d58148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:27.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d56800640 1 -- 172.21.15.32:0/441810791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d5806d020 msgr2=0x7f5d580c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:27.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d56800640 1 --2- 172.21.15.32:0/441810791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d5806d020 0x7f5d580c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:27.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d56800640 1 -- 172.21.15.32:0/441810791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d5809b100 msgr2=0x7f5d58148ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:27.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d56800640 1 --2- 172.21.15.32:0/441810791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d5809b100 0x7f5d58148ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:27.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d56800640 1 -- 172.21.15.32:0/441810791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d580c8880 con 0x7f5d580b8050 2024-04-17T10:03:27.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d55e00640 1 --2- 172.21.15.32:0/441810791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d5806d020 0x7f5d580c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:27.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d56800640 1 --2- 172.21.15.32:0/441810791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d580b8050 0x7f5d5815f420 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000c480 tx=0x7f5d5000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:27.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d4ea00640 1 -- 172.21.15.32:0/441810791 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d50017070 con 0x7f5d580b8050 2024-04-17T10:03:27.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d4ea00640 1 -- 172.21.15.32:0/441810791 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d50005db0 con 0x7f5d580b8050 2024-04-17T10:03:27.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d4ea00640 1 -- 172.21.15.32:0/441810791 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d50007290 con 0x7f5d580b8050 2024-04-17T10:03:27.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.340+0000 7f5d57200640 1 -- 172.21.15.32:0/441810791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d580c72a0 con 0x7f5d580b8050 2024-04-17T10:03:27.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.344+0000 7f5d57200640 1 -- 172.21.15.32:0/441810791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d580c7860 con 0x7f5d580b8050 2024-04-17T10:03:27.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.344+0000 7f5d57200640 1 -- 172.21.15.32:0/441810791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d5804fd00 con 0x7f5d580b8050 2024-04-17T10:03:27.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.344+0000 7f5d4ea00640 1 -- 172.21.15.32:0/441810791 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5d5001a070 con 0x7f5d580b8050 2024-04-17T10:03:27.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.344+0000 7f5d4ea00640 1 --2- 172.21.15.32:0/441810791 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5d34068380 0x7f5d3406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:27.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.344+0000 7f5d4ea00640 1 -- 172.21.15.32:0/441810791 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x7f5d50085e30 con 0x7f5d580b8050 2024-04-17T10:03:27.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.344+0000 7f5d55e00640 1 --2- 172.21.15.32:0/441810791 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5d34068380 0x7f5d3406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:27.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.344+0000 7f5d55e00640 1 --2- 172.21.15.32:0/441810791 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5d34068380 0x7f5d3406a840 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5d480b1000 tx=0x7f5d480aa000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:27.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.348+0000 7f5d4ea00640 1 -- 172.21.15.32:0/441810791 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5d50056330 con 0x7f5d580b8050 2024-04-17T10:03:27.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:27.520+0000 7f5d57200640 1 -- 172.21.15.32:0/441810791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5d5809b620 con 0x7f5d580b8050 2024-04-17T10:03:28.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.168+0000 7f5d4ea00640 1 -- 172.21.15.32:0/441810791 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v100) ==== 181+0+0 (secure 0 0 0) 0x7f5d50055c30 con 0x7f5d580b8050 2024-04-17T10:03:28.164 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-04-17T10:03:28.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.168+0000 7f5d57200640 1 -- 172.21.15.32:0/441810791 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5d34068380 msgr2=0x7f5d3406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:28.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.168+0000 7f5d57200640 1 --2- 172.21.15.32:0/441810791 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5d34068380 0x7f5d3406a840 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5d480b1000 tx=0x7f5d480aa000 comp rx=0 tx=0).stop 2024-04-17T10:03:28.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.168+0000 7f5d57200640 1 -- 172.21.15.32:0/441810791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d580b8050 msgr2=0x7f5d5815f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:28.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.168+0000 7f5d57200640 1 --2- 172.21.15.32:0/441810791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d580b8050 0x7f5d5815f420 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000c480 tx=0x7f5d5000c940 comp rx=0 tx=0).stop 2024-04-17T10:03:28.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.168+0000 7f5d57200640 1 -- 172.21.15.32:0/441810791 shutdown_connections 2024-04-17T10:03:28.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.168+0000 7f5d57200640 1 --2- 172.21.15.32:0/441810791 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5d34068380 0x7f5d3406a840 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:28.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.168+0000 7f5d57200640 1 --2- 172.21.15.32:0/441810791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d580b8050 0x7f5d5815f420 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:28.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.168+0000 7f5d57200640 1 --2- 172.21.15.32:0/441810791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d5809b100 0x7f5d58148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:28.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.168+0000 7f5d57200640 1 --2- 172.21.15.32:0/441810791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d5806d020 0x7f5d580c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:28.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.168+0000 7f5d57200640 1 -- 172.21.15.32:0/441810791 >> 172.21.15.32:0/441810791 conn(0x7f5d580b3ec0 msgr2=0x7f5d5809a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:28.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.172+0000 7f5d57200640 1 -- 172.21.15.32:0/441810791 shutdown_connections 2024-04-17T10:03:28.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.172+0000 7f5d57200640 1 -- 172.21.15.32:0/441810791 wait complete. 2024-04-17T10:03:28.192 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-17T10:03:28.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.344+0000 7f8cb7600640 1 Processor -- start 2024-04-17T10:03:28.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.344+0000 7f8cb7600640 1 -- start start 2024-04-17T10:03:28.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.344+0000 7f8cb7600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cb00ceb40 0x7f8cb0071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:28.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.348+0000 7f8cb7600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cb00cefa0 con 0x7f8cb00722f0 2024-04-17T10:03:28.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.348+0000 7f8cb7600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cb006eeb0 con 0x7f8cb01460a0 2024-04-17T10:03:28.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.348+0000 7f8cb7600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8cb015de00 con 0x7f8cb00ceb40 2024-04-17T10:03:28.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.348+0000 7f8cb5600640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8cb00722f0 0x7f8cb015e550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53284/0 (socket says 172.21.15.32:53284) 2024-04-17T10:03:28.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.348+0000 7f8cb5600640 1 -- 172.21.15.32:0/2811498633 learned_addr learned my addr 172.21.15.32:0/2811498633 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:28.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.348+0000 7f8ca7e00640 1 --2- 172.21.15.32:0/2811498633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cb00ceb40 0x7f8cb0071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:28.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.348+0000 7f8cafe00640 1 -- 172.21.15.32:0/2811498633 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3797898374 0 0) 0x7f8cb00cefa0 con 0x7f8cb00722f0 2024-04-17T10:03:28.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.348+0000 7f8cafe00640 1 -- 172.21.15.32:0/2811498633 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c94003530 con 0x7f8cb00722f0 2024-04-17T10:03:28.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.348+0000 7f8cafe00640 1 -- 172.21.15.32:0/2811498633 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3155758790 0 0) 0x7f8cb006eeb0 con 0x7f8cb01460a0 2024-04-17T10:03:28.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.348+0000 7f8cafe00640 1 -- 172.21.15.32:0/2811498633 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cb00cefa0 con 0x7f8cb01460a0 2024-04-17T10:03:28.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.348+0000 7f8ca7e00640 1 -- 172.21.15.32:0/2811498633 >> v1:172.21.15.144:6789/0 conn(0x7f8cb01460a0 legacy=0x7f8cb006e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:28.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.348+0000 7f8ca7e00640 1 -- 172.21.15.32:0/2811498633 >> v1:172.21.15.32:6789/0 conn(0x7f8cb00722f0 legacy=0x7f8cb015e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:28.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.348+0000 7f8ca7e00640 1 -- 172.21.15.32:0/2811498633 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb006f190 con 0x7f8cb00ceb40 2024-04-17T10:03:28.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.352+0000 7f8ca7e00640 1 --2- 172.21.15.32:0/2811498633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cb00ceb40 0x7f8cb0071db0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8c980097d0 tx=0x7f8c9801c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1ad35321e4d4d9b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:28.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.352+0000 7f8cafe00640 1 -- 172.21.15.32:0/2811498633 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c980b0070 con 0x7f8cb00ceb40 2024-04-17T10:03:28.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.352+0000 7f8cafe00640 1 -- 172.21.15.32:0/2811498633 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c980ab600 con 0x7f8cb00ceb40 2024-04-17T10:03:28.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.352+0000 7f8cafe00640 1 -- 172.21.15.32:0/2811498633 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c980ab9b0 con 0x7f8cb00ceb40 2024-04-17T10:03:28.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.352+0000 7f8cb7600640 1 -- 172.21.15.32:0/2811498633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cb00ceb40 msgr2=0x7f8cb0071db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:28.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.352+0000 7f8cb7600640 1 --2- 172.21.15.32:0/2811498633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cb00ceb40 0x7f8cb0071db0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8c980097d0 tx=0x7f8c9801c4c0 comp rx=0 tx=0).stop 2024-04-17T10:03:28.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.352+0000 7f8cb7600640 1 -- 172.21.15.32:0/2811498633 shutdown_connections 2024-04-17T10:03:28.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.352+0000 7f8cb7600640 1 --2- 172.21.15.32:0/2811498633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cb00ceb40 0x7f8cb0071db0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:28.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.352+0000 7f8cb7600640 1 -- 172.21.15.32:0/2811498633 >> 172.21.15.32:0/2811498633 conn(0x7f8cb00b2190 msgr2=0x7f8cb00b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:28.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.352+0000 7f8cb7600640 1 -- 172.21.15.32:0/2811498633 shutdown_connections 2024-04-17T10:03:28.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.352+0000 7f8cb7600640 1 -- 172.21.15.32:0/2811498633 wait complete. 2024-04-17T10:03:28.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.352+0000 7f8cb7600640 1 Processor -- start 2024-04-17T10:03:28.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.352+0000 7f8cb7600640 1 -- start start 2024-04-17T10:03:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.352+0000 7f8cb7600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cb00722f0 0x7f8cb015b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.352+0000 7f8cb7600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cb00ceb40 0x7f8cb00b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.352+0000 7f8cb7600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cb01460a0 0x7f8cb013b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.352+0000 7f8cb7600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb016a290 con 0x7f8cb00722f0 2024-04-17T10:03:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.352+0000 7f8cb7600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb015f740 con 0x7f8cb01460a0 2024-04-17T10:03:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.352+0000 7f8cb7600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8cb016a410 con 0x7f8cb00ceb40 2024-04-17T10:03:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.352+0000 7f8cb4c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cb00722f0 0x7f8cb015b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.352+0000 7f8cb4c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cb00722f0 0x7f8cb015b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43312/0 (socket says 172.21.15.32:43312) 2024-04-17T10:03:28.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.352+0000 7f8cb4c00640 1 -- 172.21.15.32:0/2372498371 learned_addr learned my addr 172.21.15.32:0/2372498371 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:28.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.352+0000 7f8ca7e00640 1 --2- 172.21.15.32:0/2372498371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cb00ceb40 0x7f8cb00b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:28.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.356+0000 7f8cb5600640 1 --2- 172.21.15.32:0/2372498371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cb01460a0 0x7f8cb013b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:28.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.356+0000 7f8cb4c00640 1 -- 172.21.15.32:0/2372498371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cb00ceb40 msgr2=0x7f8cb00b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:28.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.356+0000 7f8cb4c00640 1 --2- 172.21.15.32:0/2372498371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cb00ceb40 0x7f8cb00b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:28.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.356+0000 7f8cb4c00640 1 -- 172.21.15.32:0/2372498371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cb01460a0 msgr2=0x7f8cb013b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:28.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.356+0000 7f8cb4c00640 1 --2- 172.21.15.32:0/2372498371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cb01460a0 0x7f8cb013b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:28.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.356+0000 7f8cb4c00640 1 -- 172.21.15.32:0/2372498371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb015bcd0 con 0x7f8cb00722f0 2024-04-17T10:03:28.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.356+0000 7f8cb4c00640 1 --2- 172.21.15.32:0/2372498371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cb00722f0 0x7f8cb015b790 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000c430 tx=0x7f8ca000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:28.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.356+0000 7f8cad600640 1 -- 172.21.15.32:0/2372498371 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0017070 con 0x7f8cb00722f0 2024-04-17T10:03:28.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.356+0000 7f8cad600640 1 -- 172.21.15.32:0/2372498371 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ca00052f0 con 0x7f8cb00722f0 2024-04-17T10:03:28.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.356+0000 7f8cad600640 1 -- 172.21.15.32:0/2372498371 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0003840 con 0x7f8cb00722f0 2024-04-17T10:03:28.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.356+0000 7f8cb7600640 1 -- 172.21.15.32:0/2372498371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb00b0a70 con 0x7f8cb00722f0 2024-04-17T10:03:28.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.356+0000 7f8cb7600640 1 -- 172.21.15.32:0/2372498371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb00b1030 con 0x7f8cb00722f0 2024-04-17T10:03:28.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.356+0000 7f8cb7600640 1 -- 172.21.15.32:0/2372498371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cb0068a90 con 0x7f8cb00722f0 2024-04-17T10:03:28.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.356+0000 7f8cad600640 1 -- 172.21.15.32:0/2372498371 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8ca001a070 con 0x7f8cb00722f0 2024-04-17T10:03:28.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.356+0000 7f8cad600640 1 --2- 172.21.15.32:0/2372498371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8c8c068380 0x7f8c8c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:28.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.356+0000 7f8cad600640 1 -- 172.21.15.32:0/2372498371 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5443+0+0 (secure 0 0 0) 0x7f8ca0085c30 con 0x7f8cb00722f0 2024-04-17T10:03:28.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.360+0000 7f8ca7e00640 1 --2- 172.21.15.32:0/2372498371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8c8c068380 0x7f8c8c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:28.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.360+0000 7f8ca7e00640 1 --2- 172.21.15.32:0/2372498371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8c8c068380 0x7f8c8c06a840 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8c980ad000 tx=0x7f8c980a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:28.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.364+0000 7f8cad600640 1 -- 172.21.15.32:0/2372498371 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f8ca0055c00 con 0x7f8cb00722f0 2024-04-17T10:03:28.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:28.532+0000 7f8cb7600640 1 -- 172.21.15.32:0/2372498371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8cb00b2850 con 0x7f8cb00722f0 2024-04-17T10:03:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.168+0000 7f8cad600640 1 -- 172.21.15.32:0/2372498371 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v101) ==== 169+0+0 (secure 0 0 0) 0x7f8ca0059f00 con 0x7f8cb00722f0 2024-04-17T10:03:29.166 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-04-17T10:03:29.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.172+0000 7f8cb7600640 1 -- 172.21.15.32:0/2372498371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8c8c068380 msgr2=0x7f8c8c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:29.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.172+0000 7f8cb7600640 1 --2- 172.21.15.32:0/2372498371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8c8c068380 0x7f8c8c06a840 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8c980ad000 tx=0x7f8c980a6000 comp rx=0 tx=0).stop 2024-04-17T10:03:29.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.172+0000 7f8cb7600640 1 -- 172.21.15.32:0/2372498371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cb00722f0 msgr2=0x7f8cb015b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:29.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.172+0000 7f8cb7600640 1 --2- 172.21.15.32:0/2372498371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cb00722f0 0x7f8cb015b790 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000c430 tx=0x7f8ca000c8f0 comp rx=0 tx=0).stop 2024-04-17T10:03:29.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.172+0000 7f8cb7600640 1 -- 172.21.15.32:0/2372498371 shutdown_connections 2024-04-17T10:03:29.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.172+0000 7f8cb7600640 1 --2- 172.21.15.32:0/2372498371 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8c8c068380 0x7f8c8c06a840 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:29.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.172+0000 7f8cb7600640 1 --2- 172.21.15.32:0/2372498371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cb00ceb40 0x7f8cb00b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:29.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.172+0000 7f8cb7600640 1 --2- 172.21.15.32:0/2372498371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cb01460a0 0x7f8cb013b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:29.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.172+0000 7f8cb7600640 1 --2- 172.21.15.32:0/2372498371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cb00722f0 0x7f8cb015b790 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:29.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.172+0000 7f8cb7600640 1 -- 172.21.15.32:0/2372498371 >> 172.21.15.32:0/2372498371 conn(0x7f8cb00b2190 msgr2=0x7f8cb006e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:29.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.172+0000 7f8cb7600640 1 -- 172.21.15.32:0/2372498371 shutdown_connections 2024-04-17T10:03:29.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.172+0000 7f8cb7600640 1 -- 172.21.15.32:0/2372498371 wait complete. 2024-04-17T10:03:29.197 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-17T10:03:29.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.348+0000 7f5ce8600640 1 Processor -- start 2024-04-17T10:03:29.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.348+0000 7f5ce8600640 1 -- start start 2024-04-17T10:03:29.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.348+0000 7f5ce8600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ce015b7d0 0x7f5ce006e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:29.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.348+0000 7f5ce8600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ce00cc940 con 0x7f5ce006c140 2024-04-17T10:03:29.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.348+0000 7f5ce8600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ce00ccc20 con 0x7f5ce006f7c0 2024-04-17T10:03:29.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.348+0000 7f5ce8600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5ce006e0b0 con 0x7f5ce015b7d0 2024-04-17T10:03:29.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.348+0000 7f5ce6600640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5ce006c140 0x7f5ce00ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53296/0 (socket says 172.21.15.32:53296) 2024-04-17T10:03:29.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.348+0000 7f5ce6600640 1 -- 172.21.15.32:0/964018314 learned_addr learned my addr 172.21.15.32:0/964018314 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:29.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.348+0000 7f5ce5c00640 1 --2- 172.21.15.32:0/964018314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ce015b7d0 0x7f5ce006e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:29.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.348+0000 7f5cdfe00640 1 -- 172.21.15.32:0/964018314 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 951826702 0 0) 0x7f5ce00cc940 con 0x7f5ce006c140 2024-04-17T10:03:29.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.348+0000 7f5cdfe00640 1 -- 172.21.15.32:0/964018314 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5cc8003530 con 0x7f5ce006c140 2024-04-17T10:03:29.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.352+0000 7f5ce5c00640 1 -- 172.21.15.32:0/964018314 >> v1:172.21.15.144:6789/0 conn(0x7f5ce006f7c0 legacy=0x7f5ce006ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:29.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.352+0000 7f5cdfe00640 1 -- 172.21.15.32:0/964018314 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1071010576 0 0) 0x7f5ce00ccc20 con 0x7f5ce006f7c0 2024-04-17T10:03:29.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.352+0000 7f5ce5c00640 1 -- 172.21.15.32:0/964018314 >> v1:172.21.15.32:6789/0 conn(0x7f5ce006c140 legacy=0x7f5ce00ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:29.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.352+0000 7f5ce5c00640 1 -- 172.21.15.32:0/964018314 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce006e390 con 0x7f5ce015b7d0 2024-04-17T10:03:29.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.352+0000 7f5ce5c00640 1 --2- 172.21.15.32:0/964018314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ce015b7d0 0x7f5ce006e8b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40097f0 tx=0x7f5cd401c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a74496f4732bc86 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:29.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.352+0000 7f5cdfe00640 1 -- 172.21.15.32:0/964018314 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd40b1070 con 0x7f5ce015b7d0 2024-04-17T10:03:29.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.352+0000 7f5cdfe00640 1 -- 172.21.15.32:0/964018314 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cd40ac640 con 0x7f5ce015b7d0 2024-04-17T10:03:29.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.352+0000 7f5cdfe00640 1 -- 172.21.15.32:0/964018314 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd40ac9f0 con 0x7f5ce015b7d0 2024-04-17T10:03:29.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.352+0000 7f5ce8600640 1 -- 172.21.15.32:0/964018314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ce015b7d0 msgr2=0x7f5ce006e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:29.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.352+0000 7f5ce8600640 1 --2- 172.21.15.32:0/964018314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ce015b7d0 0x7f5ce006e8b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40097f0 tx=0x7f5cd401c370 comp rx=0 tx=0).stop 2024-04-17T10:03:29.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce8600640 1 -- 172.21.15.32:0/964018314 shutdown_connections 2024-04-17T10:03:29.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce8600640 1 --2- 172.21.15.32:0/964018314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ce015b7d0 0x7f5ce006e8b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:29.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce8600640 1 -- 172.21.15.32:0/964018314 >> 172.21.15.32:0/964018314 conn(0x7f5ce00b2120 msgr2=0x7f5ce00b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:29.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce8600640 1 -- 172.21.15.32:0/964018314 shutdown_connections 2024-04-17T10:03:29.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce8600640 1 -- 172.21.15.32:0/964018314 wait complete. 2024-04-17T10:03:29.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce8600640 1 Processor -- start 2024-04-17T10:03:29.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce8600640 1 -- start start 2024-04-17T10:03:29.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce8600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ce006c140 0x7f5ce0154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:29.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce8600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ce006f7c0 0x7f5ce0153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:29.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce8600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce015b7d0 0x7f5ce01fb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:29.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce8600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce0169c40 con 0x7f5ce015b7d0 2024-04-17T10:03:29.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce8600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce0169ac0 con 0x7f5ce006f7c0 2024-04-17T10:03:29.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce8600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5ce0169dc0 con 0x7f5ce006c140 2024-04-17T10:03:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce6600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce015b7d0 0x7f5ce01fb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce6600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce015b7d0 0x7f5ce01fb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43320/0 (socket says 172.21.15.32:43320) 2024-04-17T10:03:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce6600640 1 -- 172.21.15.32:0/897224854 learned_addr learned my addr 172.21.15.32:0/897224854 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce5c00640 1 --2- 172.21.15.32:0/897224854 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ce006c140 0x7f5ce0154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce5200640 1 --2- 172.21.15.32:0/897224854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ce006f7c0 0x7f5ce0153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce6600640 1 -- 172.21.15.32:0/897224854 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ce006c140 msgr2=0x7f5ce0154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce6600640 1 --2- 172.21.15.32:0/897224854 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ce006c140 0x7f5ce0154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce6600640 1 -- 172.21.15.32:0/897224854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ce006f7c0 msgr2=0x7f5ce0153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce6600640 1 --2- 172.21.15.32:0/897224854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ce006f7c0 0x7f5ce0153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce6600640 1 -- 172.21.15.32:0/897224854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce0156590 con 0x7f5ce015b7d0 2024-04-17T10:03:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce6600640 1 --2- 172.21.15.32:0/897224854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce015b7d0 0x7f5ce01fb250 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8002030 tx=0x7f5cd800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5cdd600640 1 -- 172.21.15.32:0/897224854 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8012400 con 0x7f5ce015b7d0 2024-04-17T10:03:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5cdd600640 1 -- 172.21.15.32:0/897224854 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cd80125a0 con 0x7f5ce015b7d0 2024-04-17T10:03:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5cdd600640 1 -- 172.21.15.32:0/897224854 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd80129b0 con 0x7f5ce015b7d0 2024-04-17T10:03:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.356+0000 7f5ce8600640 1 -- 172.21.15.32:0/897224854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce0156870 con 0x7f5ce015b7d0 2024-04-17T10:03:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.360+0000 7f5ce8600640 1 -- 172.21.15.32:0/897224854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce014d1a0 con 0x7f5ce015b7d0 2024-04-17T10:03:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.360+0000 7f5ce8600640 1 -- 172.21.15.32:0/897224854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ce0068a90 con 0x7f5ce015b7d0 2024-04-17T10:03:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.360+0000 7f5cdd600640 1 -- 172.21.15.32:0/897224854 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5cd8012b50 con 0x7f5ce015b7d0 2024-04-17T10:03:29.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.360+0000 7f5cdd600640 1 --2- 172.21.15.32:0/897224854 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5cc0068380 0x7f5cc006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:29.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.360+0000 7f5cdd600640 1 -- 172.21.15.32:0/897224854 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5029+0+0 (secure 0 0 0) 0x7f5cd8087090 con 0x7f5ce015b7d0 2024-04-17T10:03:29.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.360+0000 7f5ce5c00640 1 --2- 172.21.15.32:0/897224854 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5cc0068380 0x7f5cc006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:29.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.360+0000 7f5ce5c00640 1 --2- 172.21.15.32:0/897224854 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5cc0068380 0x7f5cc006a840 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40ae000 tx=0x7f5cd40a6070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:29.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.364+0000 7f5cdd600640 1 -- 172.21.15.32:0/897224854 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5cd80578e0 con 0x7f5ce015b7d0 2024-04-17T10:03:29.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.528+0000 7f5ce8600640 1 -- 172.21.15.32:0/897224854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f5ce0157160 con 0x7f5ce015b7d0 2024-04-17T10:03:29.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.532+0000 7f5cdd600640 1 -- 172.21.15.32:0/897224854 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f5cd80571e0 con 0x7f5ce015b7d0 2024-04-17T10:03:29.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.536+0000 7f5ce8600640 1 -- 172.21.15.32:0/897224854 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5cc0068380 msgr2=0x7f5cc006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:29.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.536+0000 7f5ce8600640 1 --2- 172.21.15.32:0/897224854 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5cc0068380 0x7f5cc006a840 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40ae000 tx=0x7f5cd40a6070 comp rx=0 tx=0).stop 2024-04-17T10:03:29.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.536+0000 7f5ce8600640 1 -- 172.21.15.32:0/897224854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce015b7d0 msgr2=0x7f5ce01fb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:29.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.536+0000 7f5ce8600640 1 --2- 172.21.15.32:0/897224854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce015b7d0 0x7f5ce01fb250 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8002030 tx=0x7f5cd800a430 comp rx=0 tx=0).stop 2024-04-17T10:03:29.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.536+0000 7f5ce8600640 1 -- 172.21.15.32:0/897224854 shutdown_connections 2024-04-17T10:03:29.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.536+0000 7f5ce8600640 1 --2- 172.21.15.32:0/897224854 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5cc0068380 0x7f5cc006a840 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:29.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.536+0000 7f5ce8600640 1 --2- 172.21.15.32:0/897224854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce015b7d0 0x7f5ce01fb250 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:29.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.536+0000 7f5ce8600640 1 --2- 172.21.15.32:0/897224854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ce006f7c0 0x7f5ce0153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:29.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.536+0000 7f5ce8600640 1 --2- 172.21.15.32:0/897224854 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ce006c140 0x7f5ce0154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:29.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.536+0000 7f5ce8600640 1 -- 172.21.15.32:0/897224854 >> 172.21.15.32:0/897224854 conn(0x7f5ce00b2120 msgr2=0x7f5ce00b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:29.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.536+0000 7f5ce8600640 1 -- 172.21.15.32:0/897224854 shutdown_connections 2024-04-17T10:03:29.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.536+0000 7f5ce8600640 1 -- 172.21.15.32:0/897224854 wait complete. 2024-04-17T10:03:29.553 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-17T10:03:29.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.708+0000 7f25c9800640 1 Processor -- start 2024-04-17T10:03:29.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.708+0000 7f25c9800640 1 -- start start 2024-04-17T10:03:29.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.708+0000 7f25c9800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25c4072500 0x7f25c40cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:29.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.708+0000 7f25c9800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f25c415d8d0 0x7f25c40cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:29.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.708+0000 7f25c9800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25c40cdac0 con 0x7f25c4145f30 2024-04-17T10:03:29.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.708+0000 7f25c9800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f25c406f400 con 0x7f25c415d8d0 2024-04-17T10:03:29.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.708+0000 7f25c9800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f25c4146310 con 0x7f25c4072500 2024-04-17T10:03:29.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.708+0000 7f25c2a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f25c4145f30 0x7f25c4071df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53306/0 (socket says 172.21.15.32:53306) 2024-04-17T10:03:29.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.708+0000 7f25c3400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f25c415d8d0 0x7f25c40cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:29.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.708+0000 7f25c2a00640 1 -- 172.21.15.32:0/3740701810 learned_addr learned my addr 172.21.15.32:0/3740701810 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:29.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.708+0000 7f25c2000640 1 --2- 172.21.15.32:0/3740701810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25c4072500 0x7f25c40cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:29.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.708+0000 7f25c1600640 1 -- 172.21.15.32:0/3740701810 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4035566431 0 0) 0x7f25c40cdac0 con 0x7f25c4145f30 2024-04-17T10:03:29.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.708+0000 7f25c1600640 1 -- 172.21.15.32:0/3740701810 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25a8003530 con 0x7f25c4145f30 2024-04-17T10:03:29.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.708+0000 7f25c3400640 1 -- 172.21.15.32:0/3740701810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25c4072500 msgr2=0x7f25c40cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:29.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.708+0000 7f25c3400640 1 --2- 172.21.15.32:0/3740701810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25c4072500 0x7f25c40cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:29.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.708+0000 7f25c3400640 1 -- 172.21.15.32:0/3740701810 >> v1:172.21.15.32:6789/0 conn(0x7f25c4145f30 legacy=0x7f25c4071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:29.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.708+0000 7f25c3400640 1 -- 172.21.15.32:0/3740701810 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c415cdf0 con 0x7f25c415d8d0 2024-04-17T10:03:29.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.708+0000 7f25c2000640 1 --2- 172.21.15.32:0/3740701810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25c4072500 0x7f25c40cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:29.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.712+0000 7f25c3400640 1 --2- 172.21.15.32:0/3740701810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25c415d8d0 0x7f25c40cd3b0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009d90 tx=0x7f25bc01f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5eaab102b89ad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:29.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.712+0000 7f25c1600640 1 -- 172.21.15.32:0/3740701810 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc0b4070 con 0x7f25c415d8d0 2024-04-17T10:03:29.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.712+0000 7f25c1600640 1 -- 172.21.15.32:0/3740701810 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25bc0af600 con 0x7f25c415d8d0 2024-04-17T10:03:29.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.712+0000 7f25c1600640 1 -- 172.21.15.32:0/3740701810 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc0af9d0 con 0x7f25c415d8d0 2024-04-17T10:03:29.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.712+0000 7f25c9800640 1 -- 172.21.15.32:0/3740701810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25c415d8d0 msgr2=0x7f25c40cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:29.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.712+0000 7f25c9800640 1 --2- 172.21.15.32:0/3740701810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25c415d8d0 0x7f25c40cd3b0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009d90 tx=0x7f25bc01f150 comp rx=0 tx=0).stop 2024-04-17T10:03:29.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.712+0000 7f25c9800640 1 -- 172.21.15.32:0/3740701810 shutdown_connections 2024-04-17T10:03:29.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.712+0000 7f25c9800640 1 --2- 172.21.15.32:0/3740701810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25c415d8d0 0x7f25c40cd3b0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:29.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.712+0000 7f25c9800640 1 --2- 172.21.15.32:0/3740701810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25c4072500 0x7f25c40cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:29.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.712+0000 7f25c9800640 1 -- 172.21.15.32:0/3740701810 >> 172.21.15.32:0/3740701810 conn(0x7f25c406ba30 msgr2=0x7f25c406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:29.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.712+0000 7f25c9800640 1 -- 172.21.15.32:0/3740701810 shutdown_connections 2024-04-17T10:03:29.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.712+0000 7f25c9800640 1 -- 172.21.15.32:0/3740701810 wait complete. 2024-04-17T10:03:29.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.712+0000 7f25c9800640 1 Processor -- start 2024-04-17T10:03:29.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.712+0000 7f25c9800640 1 -- start start 2024-04-17T10:03:29.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.716+0000 7f25c9800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25c4072500 0x7f25c4140fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:29.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.716+0000 7f25c9800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c4145f30 0x7f25c40a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:29.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.716+0000 7f25c9800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25c415d8d0 0x7f25c40a99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:29.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.716+0000 7f25c9800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f25c415cb90 con 0x7f25c4072500 2024-04-17T10:03:29.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.716+0000 7f25c9800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f25c415d440 con 0x7f25c4145f30 2024-04-17T10:03:29.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.716+0000 7f25c9800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f25c41601d0 con 0x7f25c415d8d0 2024-04-17T10:03:29.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.716+0000 7f25c2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25c4072500 0x7f25c4140fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:29.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.716+0000 7f25c2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25c4072500 0x7f25c4140fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43336/0 (socket says 172.21.15.32:43336) 2024-04-17T10:03:29.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.716+0000 7f25c2a00640 1 -- 172.21.15.32:0/2820354923 learned_addr learned my addr 172.21.15.32:0/2820354923 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:29.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.716+0000 7f25c2000640 1 --2- 172.21.15.32:0/2820354923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c4145f30 0x7f25c40a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:29.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.716+0000 7f25c3400640 1 --2- 172.21.15.32:0/2820354923 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25c415d8d0 0x7f25c40a99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:29.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.716+0000 7f25c2a00640 1 -- 172.21.15.32:0/2820354923 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25c415d8d0 msgr2=0x7f25c40a99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:29.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.716+0000 7f25c2a00640 1 --2- 172.21.15.32:0/2820354923 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25c415d8d0 0x7f25c40a99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:29.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.716+0000 7f25c2a00640 1 -- 172.21.15.32:0/2820354923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c4145f30 msgr2=0x7f25c40a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:29.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.716+0000 7f25c2a00640 1 --2- 172.21.15.32:0/2820354923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c4145f30 0x7f25c40a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:29.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.716+0000 7f25c2a00640 1 -- 172.21.15.32:0/2820354923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c41425c0 con 0x7f25c4072500 2024-04-17T10:03:29.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.716+0000 7f25c2a00640 1 --2- 172.21.15.32:0/2820354923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25c4072500 0x7f25c4140fd0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f25b400c450 tx=0x7f25b400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:29.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.716+0000 7f25baa00640 1 -- 172.21.15.32:0/2820354923 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b4017070 con 0x7f25c4072500 2024-04-17T10:03:29.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.716+0000 7f25baa00640 1 -- 172.21.15.32:0/2820354923 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25b4005cf0 con 0x7f25c4072500 2024-04-17T10:03:29.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.716+0000 7f25baa00640 1 -- 172.21.15.32:0/2820354923 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b4007290 con 0x7f25c4072500 2024-04-17T10:03:29.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.716+0000 7f25c9800640 1 -- 172.21.15.32:0/2820354923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c41428a0 con 0x7f25c4072500 2024-04-17T10:03:29.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.716+0000 7f25c9800640 1 -- 172.21.15.32:0/2820354923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c4142e60 con 0x7f25c4072500 2024-04-17T10:03:29.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.716+0000 7f25c9800640 1 -- 172.21.15.32:0/2820354923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25c4068a90 con 0x7f25c4072500 2024-04-17T10:03:29.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.720+0000 7f25baa00640 1 -- 172.21.15.32:0/2820354923 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f25b401a070 con 0x7f25c4072500 2024-04-17T10:03:29.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.720+0000 7f25baa00640 1 --2- 172.21.15.32:0/2820354923 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f25a4068380 0x7f25a406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:29.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.720+0000 7f25baa00640 1 -- 172.21.15.32:0/2820354923 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5029+0+0 (secure 0 0 0) 0x7f25b4085a90 con 0x7f25c4072500 2024-04-17T10:03:29.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.720+0000 7f25c2000640 1 --2- 172.21.15.32:0/2820354923 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f25a4068380 0x7f25a406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:29.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.720+0000 7f25c2000640 1 --2- 172.21.15.32:0/2820354923 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f25a4068380 0x7f25a406a840 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f25c4144f00 tx=0x7f25b0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:29.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.724+0000 7f25baa00640 1 -- 172.21.15.32:0/2820354923 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f25b40562e0 con 0x7f25c4072500 2024-04-17T10:03:29.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:29.944+0000 7f25c9800640 1 -- 172.21.15.32:0/2820354923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7f25c4072f90 con 0x7f25c4072500 2024-04-17T10:03:30.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.180+0000 7f25baa00640 1 -- 172.21.15.32:0/2820354923 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v476) ==== 146+0+0 (secure 0 0 0) 0x7f25b4055be0 con 0x7f25c4072500 2024-04-17T10:03:30.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.184+0000 7f25c9800640 1 -- 172.21.15.32:0/2820354923 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f25a4068380 msgr2=0x7f25a406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:30.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.184+0000 7f25c9800640 1 --2- 172.21.15.32:0/2820354923 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f25a4068380 0x7f25a406a840 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f25c4144f00 tx=0x7f25b0008040 comp rx=0 tx=0).stop 2024-04-17T10:03:30.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.184+0000 7f25c9800640 1 -- 172.21.15.32:0/2820354923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25c4072500 msgr2=0x7f25c4140fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:30.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.184+0000 7f25c9800640 1 --2- 172.21.15.32:0/2820354923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25c4072500 0x7f25c4140fd0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f25b400c450 tx=0x7f25b400c910 comp rx=0 tx=0).stop 2024-04-17T10:03:30.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.184+0000 7f25c9800640 1 -- 172.21.15.32:0/2820354923 shutdown_connections 2024-04-17T10:03:30.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.184+0000 7f25c9800640 1 --2- 172.21.15.32:0/2820354923 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f25a4068380 0x7f25a406a840 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.184+0000 7f25c9800640 1 --2- 172.21.15.32:0/2820354923 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25c415d8d0 0x7f25c40a99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.184+0000 7f25c9800640 1 --2- 172.21.15.32:0/2820354923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c4145f30 0x7f25c40a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.184+0000 7f25c9800640 1 --2- 172.21.15.32:0/2820354923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25c4072500 0x7f25c4140fd0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.184+0000 7f25c9800640 1 -- 172.21.15.32:0/2820354923 >> 172.21.15.32:0/2820354923 conn(0x7f25c406ba30 msgr2=0x7f25c415f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:30.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.184+0000 7f25c9800640 1 -- 172.21.15.32:0/2820354923 shutdown_connections 2024-04-17T10:03:30.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.184+0000 7f25c9800640 1 -- 172.21.15.32:0/2820354923 wait complete. 2024-04-17T10:03:30.201 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-17T10:03:30.201 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-17T10:03:30.202 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:03:30.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.344+0000 7f9684c00640 1 Processor -- start 2024-04-17T10:03:30.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.344+0000 7f9684c00640 1 -- start start 2024-04-17T10:03:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.344+0000 7f9684c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9680145f30 0x7f9680071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.344+0000 7f9684c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96800cd580 0x7f9680072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.344+0000 7f9684c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96800cdac0 con 0x7f968015d8d0 2024-04-17T10:03:30.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.344+0000 7f9684c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f968006f400 con 0x7f9680145f30 2024-04-17T10:03:30.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.344+0000 7f9684c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9680146310 con 0x7f96800cd580 2024-04-17T10:03:30.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.344+0000 7f967de00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9680145f30 0x7f9680071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:30.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.344+0000 7f967e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96800cd580 0x7f9680072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:30.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.344+0000 7f967de00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9680145f30 0x7f9680071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:51140/0 (socket says 172.21.15.32:51140) 2024-04-17T10:03:30.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.344+0000 7f967e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96800cd580 0x7f9680072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:50880/0 (socket says 172.21.15.32:50880) 2024-04-17T10:03:30.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.344+0000 7f967e800640 1 -- 172.21.15.32:0/3637110107 learned_addr learned my addr 172.21.15.32:0/3637110107 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:30.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.344+0000 7f9675400640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f968015d8d0 0x7f96800cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:59970/0 (socket says 172.21.15.32:59970) 2024-04-17T10:03:30.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.348+0000 7f967e800640 1 -- 172.21.15.32:0/3637110107 >> v2:172.21.15.144:3300/0 conn(0x7f9680145f30 msgr2=0x7f9680071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:30.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.348+0000 7f967e800640 1 --2- 172.21.15.32:0/3637110107 >> v2:172.21.15.144:3300/0 conn(0x7f9680145f30 0x7f9680071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.348+0000 7f967e800640 1 -- 172.21.15.32:0/3637110107 >> v1:172.21.15.144:6789/0 conn(0x7f968015d8d0 legacy=0x7f96800cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:30.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.348+0000 7f967e800640 1 -- 172.21.15.32:0/3637110107 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968015cdf0 con 0x7f96800cd580 2024-04-17T10:03:30.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.348+0000 7f967de00640 1 --2- 172.21.15.32:0/3637110107 >> v2:172.21.15.144:3300/0 conn(0x7f9680145f30 0x7f9680071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:03:30.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.348+0000 7f967e800640 1 --2- 172.21.15.32:0/3637110107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96800cd580 0x7f9680072330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9678009e50 tx=0x7f967801f210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=530a6faad82a2435 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:30.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.348+0000 7f967d400640 1 -- 172.21.15.32:0/3637110107 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96780b4070 con 0x7f96800cd580 2024-04-17T10:03:30.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.348+0000 7f967d400640 1 -- 172.21.15.32:0/3637110107 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96780af400 con 0x7f96800cd580 2024-04-17T10:03:30.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.348+0000 7f967d400640 1 -- 172.21.15.32:0/3637110107 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96780af7d0 con 0x7f96800cd580 2024-04-17T10:03:30.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f9684c00640 1 -- 172.21.15.32:0/3637110107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96800cd580 msgr2=0x7f9680072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:30.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f9684c00640 1 --2- 172.21.15.32:0/3637110107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96800cd580 0x7f9680072330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9678009e50 tx=0x7f967801f210 comp rx=0 tx=0).stop 2024-04-17T10:03:30.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f9684c00640 1 -- 172.21.15.32:0/3637110107 shutdown_connections 2024-04-17T10:03:30.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f9684c00640 1 --2- 172.21.15.32:0/3637110107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96800cd580 0x7f9680072330 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f9684c00640 1 --2- 172.21.15.32:0/3637110107 >> v2:172.21.15.144:3300/0 conn(0x7f9680145f30 0x7f9680071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f9684c00640 1 -- 172.21.15.32:0/3637110107 >> 172.21.15.32:0/3637110107 conn(0x7f968006ba30 msgr2=0x7f968006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:30.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f9684c00640 1 -- 172.21.15.32:0/3637110107 shutdown_connections 2024-04-17T10:03:30.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f9684c00640 1 -- 172.21.15.32:0/3637110107 wait complete. 2024-04-17T10:03:30.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f9684c00640 1 Processor -- start 2024-04-17T10:03:30.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f9684c00640 1 -- start start 2024-04-17T10:03:30.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f9684c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96800cd580 0x7f96801435e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:30.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f9684c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9680145f30 0x7f96800b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:30.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f9684c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f968015d8d0 0x7f96800aa050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:30.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f9684c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f968015cb90 con 0x7f968015d8d0 2024-04-17T10:03:30.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f9684c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f968015d440 con 0x7f9680145f30 2024-04-17T10:03:30.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f9684c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f96801601d0 con 0x7f96800cd580 2024-04-17T10:03:30.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f967e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f968015d8d0 0x7f96800aa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:30.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f967e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f968015d8d0 0x7f96800aa050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58610/0 (socket says 172.21.15.32:58610) 2024-04-17T10:03:30.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f967e800640 1 -- 172.21.15.32:0/4132761293 learned_addr learned my addr 172.21.15.32:0/4132761293 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:30.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f967de00640 1 --2- 172.21.15.32:0/4132761293 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96800cd580 0x7f96801435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:30.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f9675400640 1 --2- 172.21.15.32:0/4132761293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9680145f30 0x7f96800b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:30.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f967e800640 1 -- 172.21.15.32:0/4132761293 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96800cd580 msgr2=0x7f96801435e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:30.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f967e800640 1 --2- 172.21.15.32:0/4132761293 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96800cd580 0x7f96801435e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f967e800640 1 -- 172.21.15.32:0/4132761293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9680145f30 msgr2=0x7f96800b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:30.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f967e800640 1 --2- 172.21.15.32:0/4132761293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9680145f30 0x7f96800b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.352+0000 7f967e800640 1 -- 172.21.15.32:0/4132761293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96800cc0e0 con 0x7f968015d8d0 2024-04-17T10:03:30.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.356+0000 7f967e800640 1 --2- 172.21.15.32:0/4132761293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f968015d8d0 0x7f96800aa050 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f967801ce40 tx=0x7f967801bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:30.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.356+0000 7f9676a00640 1 -- 172.21.15.32:0/4132761293 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96780b4040 con 0x7f968015d8d0 2024-04-17T10:03:30.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.356+0000 7f9676a00640 1 -- 172.21.15.32:0/4132761293 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96780c0070 con 0x7f968015d8d0 2024-04-17T10:03:30.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.356+0000 7f9684c00640 1 -- 172.21.15.32:0/4132761293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96800cc3c0 con 0x7f968015d8d0 2024-04-17T10:03:30.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.356+0000 7f9676a00640 1 -- 172.21.15.32:0/4132761293 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678009510 con 0x7f968015d8d0 2024-04-17T10:03:30.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.356+0000 7f9684c00640 1 -- 172.21.15.32:0/4132761293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96800cc980 con 0x7f968015d8d0 2024-04-17T10:03:30.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.356+0000 7f9684c00640 1 -- 172.21.15.32:0/4132761293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9680068a90 con 0x7f968015d8d0 2024-04-17T10:03:30.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.356+0000 7f9676a00640 1 -- 172.21.15.32:0/4132761293 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f96780a8d10 con 0x7f968015d8d0 2024-04-17T10:03:30.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.356+0000 7f9676a00640 1 --2- 172.21.15.32:0/4132761293 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f965c068380 0x7f965c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:30.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.356+0000 7f9676a00640 1 -- 172.21.15.32:0/4132761293 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5029+0+0 (secure 0 0 0) 0x7f9678124550 con 0x7f968015d8d0 2024-04-17T10:03:30.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.356+0000 7f967de00640 1 --2- 172.21.15.32:0/4132761293 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f965c068380 0x7f965c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:30.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.360+0000 7f967de00640 1 --2- 172.21.15.32:0/4132761293 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f965c068380 0x7f965c06a840 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9670009580 tx=0x7f9670009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.360+0000 7f9676a00640 1 -- 172.21.15.32:0/4132761293 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f9678018d70 con 0x7f968015d8d0 2024-04-17T10:03:30.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.528+0000 7f9684c00640 1 -- 172.21.15.32:0/4132761293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9680072f90 con 0x7f968015d8d0 2024-04-17T10:03:30.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.528+0000 7f9676a00640 1 -- 172.21.15.32:0/4132761293 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+13164 (secure 0 0 0) 0x7f96780f48a0 con 0x7f968015d8d0 2024-04-17T10:03:30.526 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:03:30.526 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":101,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:03:29.170833+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6835/3078068380":"2024-04-18T10:03:20.790196+0000","172.21.15.32:6837/3078068380":"2024-04-18T10:03:20.790196+0000","172.21.15.32:6837/1235126741":"2024-04-18T10:02:44.035918+0000","172.21.15.32:6835/1235126741":"2024-04-18T10:02:44.035918+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:03:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.532+0000 7f9684c00640 1 -- 172.21.15.32:0/4132761293 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f965c068380 msgr2=0x7f965c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.532+0000 7f9684c00640 1 --2- 172.21.15.32:0/4132761293 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f965c068380 0x7f965c06a840 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9670009580 tx=0x7f9670009190 comp rx=0 tx=0).stop 2024-04-17T10:03:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.532+0000 7f9684c00640 1 -- 172.21.15.32:0/4132761293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f968015d8d0 msgr2=0x7f96800aa050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.532+0000 7f9684c00640 1 --2- 172.21.15.32:0/4132761293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f968015d8d0 0x7f96800aa050 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f967801ce40 tx=0x7f967801bfb0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.532+0000 7f9684c00640 1 -- 172.21.15.32:0/4132761293 shutdown_connections 2024-04-17T10:03:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.532+0000 7f9684c00640 1 --2- 172.21.15.32:0/4132761293 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f965c068380 0x7f965c06a840 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.532+0000 7f9684c00640 1 --2- 172.21.15.32:0/4132761293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f968015d8d0 0x7f96800aa050 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.532+0000 7f9684c00640 1 --2- 172.21.15.32:0/4132761293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9680145f30 0x7f96800b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.532+0000 7f9684c00640 1 --2- 172.21.15.32:0/4132761293 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96800cd580 0x7f96801435e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.532+0000 7f9684c00640 1 -- 172.21.15.32:0/4132761293 >> 172.21.15.32:0/4132761293 conn(0x7f968006ba30 msgr2=0x7f968015f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:30.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.532+0000 7f9684c00640 1 -- 172.21.15.32:0/4132761293 shutdown_connections 2024-04-17T10:03:30.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.532+0000 7f9684c00640 1 -- 172.21.15.32:0/4132761293 wait complete. 2024-04-17T10:03:30.547 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:03:30.547 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:03:30.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.692+0000 7fe108c00640 1 Processor -- start 2024-04-17T10:03:30.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.696+0000 7fe108c00640 1 -- start start 2024-04-17T10:03:30.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.696+0000 7fe108c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe10415b7d0 0x7fe10406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:30.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.696+0000 7fe108c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe10406c140 0x7fe1040ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.696+0000 7fe108c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1040cc940 con 0x7fe10406f7c0 2024-04-17T10:03:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.696+0000 7fe108c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe1040ccc20 con 0x7fe10415b7d0 2024-04-17T10:03:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.696+0000 7fe108c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe10406e0b0 con 0x7fe10406c140 2024-04-17T10:03:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.696+0000 7fe0f9400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe10406f7c0 0x7fe10406ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57508/0 (socket says 172.21.15.32:57508) 2024-04-17T10:03:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.696+0000 7fe0f9400640 1 -- 172.21.15.32:0/3207787184 learned_addr learned my addr 172.21.15.32:0/3207787184 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.696+0000 7fe101e00640 1 --2- 172.21.15.32:0/3207787184 >> v2:172.21.15.32:3300/0 conn(0x7fe10415b7d0 0x7fe10406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.696+0000 7fe102800640 1 --2- 172.21.15.32:0/3207787184 >> v2:172.21.15.144:3300/0 conn(0x7fe10406c140 0x7fe1040ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.696+0000 7fe101400640 1 -- 172.21.15.32:0/3207787184 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1982542109 0 0) 0x7fe1040cc940 con 0x7fe10406f7c0 2024-04-17T10:03:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.696+0000 7fe101400640 1 -- 172.21.15.32:0/3207787184 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0e8003530 con 0x7fe10406f7c0 2024-04-17T10:03:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.696+0000 7fe101e00640 1 -- 172.21.15.32:0/3207787184 >> v2:172.21.15.144:3300/0 conn(0x7fe10406c140 msgr2=0x7fe1040ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.696+0000 7fe101e00640 1 --2- 172.21.15.32:0/3207787184 >> v2:172.21.15.144:3300/0 conn(0x7fe10406c140 0x7fe1040ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.696+0000 7fe101e00640 1 -- 172.21.15.32:0/3207787184 >> v1:172.21.15.32:6789/0 conn(0x7fe10406f7c0 legacy=0x7fe10406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:30.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.696+0000 7fe101e00640 1 -- 172.21.15.32:0/3207787184 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe10406e390 con 0x7fe10415b7d0 2024-04-17T10:03:30.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.696+0000 7fe102800640 1 --2- 172.21.15.32:0/3207787184 >> v2:172.21.15.144:3300/0 conn(0x7fe10406c140 0x7fe1040ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:30.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.700+0000 7fe101e00640 1 --2- 172.21.15.32:0/3207787184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe10415b7d0 0x7fe10406e8b0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fe1040b20b0 tx=0x7fe0f401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce343dae56b700f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.700+0000 7fe101400640 1 -- 172.21.15.32:0/3207787184 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f40ab610 con 0x7fe10415b7d0 2024-04-17T10:03:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.700+0000 7fe101400640 1 -- 172.21.15.32:0/3207787184 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0f40b0070 con 0x7fe10415b7d0 2024-04-17T10:03:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.700+0000 7fe101400640 1 -- 172.21.15.32:0/3207787184 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f40ab9b0 con 0x7fe10415b7d0 2024-04-17T10:03:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.700+0000 7fe108c00640 1 -- 172.21.15.32:0/3207787184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe10415b7d0 msgr2=0x7fe10406e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:30.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.700+0000 7fe108c00640 1 --2- 172.21.15.32:0/3207787184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe10415b7d0 0x7fe10406e8b0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fe1040b20b0 tx=0x7fe0f401c3f0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.700+0000 7fe108c00640 1 -- 172.21.15.32:0/3207787184 shutdown_connections 2024-04-17T10:03:30.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.700+0000 7fe108c00640 1 --2- 172.21.15.32:0/3207787184 >> v2:172.21.15.144:3300/0 conn(0x7fe10406c140 0x7fe1040ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.700+0000 7fe108c00640 1 --2- 172.21.15.32:0/3207787184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe10415b7d0 0x7fe10406e8b0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.700+0000 7fe108c00640 1 -- 172.21.15.32:0/3207787184 >> 172.21.15.32:0/3207787184 conn(0x7fe1040b2120 msgr2=0x7fe1040b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:30.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.700+0000 7fe108c00640 1 -- 172.21.15.32:0/3207787184 shutdown_connections 2024-04-17T10:03:30.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.700+0000 7fe108c00640 1 -- 172.21.15.32:0/3207787184 wait complete. 2024-04-17T10:03:30.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.700+0000 7fe108c00640 1 Processor -- start 2024-04-17T10:03:30.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.700+0000 7fe108c00640 1 -- start start 2024-04-17T10:03:30.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.704+0000 7fe108c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe10406c140 0x7fe1040af630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:30.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.704+0000 7fe108c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe10406f7c0 0x7fe10415e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:30.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.704+0000 7fe108c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe10415b7d0 0x7fe10415cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:30.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.704+0000 7fe108c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe104166620 con 0x7fe10406c140 2024-04-17T10:03:30.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.704+0000 7fe108c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe1041664a0 con 0x7fe10406f7c0 2024-04-17T10:03:30.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.704+0000 7fe108c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe1041667a0 con 0x7fe10415b7d0 2024-04-17T10:03:30.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.704+0000 7fe101e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe10406c140 0x7fe1040af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:30.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.704+0000 7fe101e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe10406c140 0x7fe1040af630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58628/0 (socket says 172.21.15.32:58628) 2024-04-17T10:03:30.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.704+0000 7fe101e00640 1 -- 172.21.15.32:0/2407208534 learned_addr learned my addr 172.21.15.32:0/2407208534 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:30.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.704+0000 7fe0f9400640 1 --2- 172.21.15.32:0/2407208534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe10406f7c0 0x7fe10415e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:30.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.704+0000 7fe102800640 1 --2- 172.21.15.32:0/2407208534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe10415b7d0 0x7fe10415cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.704+0000 7fe101e00640 1 -- 172.21.15.32:0/2407208534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe10415b7d0 msgr2=0x7fe10415cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.704+0000 7fe101e00640 1 --2- 172.21.15.32:0/2407208534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe10415b7d0 0x7fe10415cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.704+0000 7fe101e00640 1 -- 172.21.15.32:0/2407208534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe10406f7c0 msgr2=0x7fe10415e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.704+0000 7fe101e00640 1 --2- 172.21.15.32:0/2407208534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe10406f7c0 0x7fe10415e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.704+0000 7fe101e00640 1 -- 172.21.15.32:0/2407208534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe10415dd20 con 0x7fe10406c140 2024-04-17T10:03:30.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.704+0000 7fe101e00640 1 --2- 172.21.15.32:0/2407208534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe10406c140 0x7fe1040af630 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fe0f401aaa0 tx=0x7fe0f401ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:30.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.704+0000 7fe0faa00640 1 -- 172.21.15.32:0/2407208534 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4006e20 con 0x7fe10406c140 2024-04-17T10:03:30.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.704+0000 7fe0faa00640 1 -- 172.21.15.32:0/2407208534 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0f40bf610 con 0x7fe10406c140 2024-04-17T10:03:30.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.704+0000 7fe108c00640 1 -- 172.21.15.32:0/2407208534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe10415e000 con 0x7fe10406c140 2024-04-17T10:03:30.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.704+0000 7fe0faa00640 1 -- 172.21.15.32:0/2407208534 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f40bf9c0 con 0x7fe10406c140 2024-04-17T10:03:30.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.704+0000 7fe108c00640 1 -- 172.21.15.32:0/2407208534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe104150b50 con 0x7fe10406c140 2024-04-17T10:03:30.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.704+0000 7fe108c00640 1 -- 172.21.15.32:0/2407208534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe104150e80 con 0x7fe10406c140 2024-04-17T10:03:30.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.708+0000 7fe0faa00640 1 -- 172.21.15.32:0/2407208534 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe0f40bd020 con 0x7fe10406c140 2024-04-17T10:03:30.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.708+0000 7fe0faa00640 1 --2- 172.21.15.32:0/2407208534 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0e0068380 0x7fe0e006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:30.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.708+0000 7fe0faa00640 1 -- 172.21.15.32:0/2407208534 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5029+0+0 (secure 0 0 0) 0x7fe0f4120da0 con 0x7fe10406c140 2024-04-17T10:03:30.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.708+0000 7fe0f9400640 1 --2- 172.21.15.32:0/2407208534 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0e0068380 0x7fe0e006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:30.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.708+0000 7fe0f9400640 1 --2- 172.21.15.32:0/2407208534 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0e0068380 0x7fe0e006a840 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec007fd0 tx=0x7fe0ec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.712+0000 7fe0faa00640 1 -- 172.21.15.32:0/2407208534 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fe0f40a8050 con 0x7fe10406c140 2024-04-17T10:03:30.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.876+0000 7fe108c00640 1 -- 172.21.15.32:0/2407208534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe104068a90 con 0x7fe10406c140 2024-04-17T10:03:30.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.876+0000 7fe0faa00640 1 -- 172.21.15.32:0/2407208534 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7fe0f4016070 con 0x7fe10406c140 2024-04-17T10:03:30.874 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T10:03:30.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.880+0000 7fe108c00640 1 -- 172.21.15.32:0/2407208534 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0e0068380 msgr2=0x7fe0e006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:30.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.880+0000 7fe108c00640 1 --2- 172.21.15.32:0/2407208534 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0e0068380 0x7fe0e006a840 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec007fd0 tx=0x7fe0ec009040 comp rx=0 tx=0).stop 2024-04-17T10:03:30.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.880+0000 7fe108c00640 1 -- 172.21.15.32:0/2407208534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe10406c140 msgr2=0x7fe1040af630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:30.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.880+0000 7fe108c00640 1 --2- 172.21.15.32:0/2407208534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe10406c140 0x7fe1040af630 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fe0f401aaa0 tx=0x7fe0f401ac30 comp rx=0 tx=0).stop 2024-04-17T10:03:30.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.880+0000 7fe108c00640 1 -- 172.21.15.32:0/2407208534 shutdown_connections 2024-04-17T10:03:30.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.880+0000 7fe108c00640 1 --2- 172.21.15.32:0/2407208534 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0e0068380 0x7fe0e006a840 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.880+0000 7fe108c00640 1 --2- 172.21.15.32:0/2407208534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe10415b7d0 0x7fe10415cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.880+0000 7fe108c00640 1 --2- 172.21.15.32:0/2407208534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe10406f7c0 0x7fe10415e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.880+0000 7fe108c00640 1 --2- 172.21.15.32:0/2407208534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe10406c140 0x7fe1040af630 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:30.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.880+0000 7fe108c00640 1 -- 172.21.15.32:0/2407208534 >> 172.21.15.32:0/2407208534 conn(0x7fe1040b2120 msgr2=0x7fe1040b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:30.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.880+0000 7fe108c00640 1 -- 172.21.15.32:0/2407208534 shutdown_connections 2024-04-17T10:03:30.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:30.880+0000 7fe108c00640 1 -- 172.21.15.32:0/2407208534 wait complete. 2024-04-17T10:03:30.895 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-17T10:03:31.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.036+0000 7f146e200640 1 Processor -- start 2024-04-17T10:03:31.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.036+0000 7f146e200640 1 -- start start 2024-04-17T10:03:31.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.040+0000 7f146e200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1468145f30 0x7f1468071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:31.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.040+0000 7f146e200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14680cd580 0x7f1468072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:31.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.040+0000 7f146e200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14680cdac0 con 0x7f146815d8d0 2024-04-17T10:03:31.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.040+0000 7f146e200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f146806f400 con 0x7f14680cd580 2024-04-17T10:03:31.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.040+0000 7f146e200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1468146310 con 0x7f1468145f30 2024-04-17T10:03:31.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.040+0000 7f1467e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14680cd580 0x7f1468072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:31.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.040+0000 7f1467e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14680cd580 0x7f1468072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58632/0 (socket says 172.21.15.32:58632) 2024-04-17T10:03:31.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.040+0000 7f1467e00640 1 -- 172.21.15.32:0/1055244402 learned_addr learned my addr 172.21.15.32:0/1055244402 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:31.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.040+0000 7f1467e00640 1 -- 172.21.15.32:0/1055244402 >> v2:172.21.15.144:3300/0 conn(0x7f1468145f30 msgr2=0x7f1468071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:31.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.040+0000 7f1467400640 1 --2- 172.21.15.32:0/1055244402 >> v2:172.21.15.144:3300/0 conn(0x7f1468145f30 0x7f1468071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:31.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.040+0000 7f1467e00640 1 --2- 172.21.15.32:0/1055244402 >> v2:172.21.15.144:3300/0 conn(0x7f1468145f30 0x7f1468071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:31.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.040+0000 7f1467e00640 1 -- 172.21.15.32:0/1055244402 >> v1:172.21.15.144:6789/0 conn(0x7f146815d8d0 legacy=0x7f14680cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:31.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.040+0000 7f1467e00640 1 -- 172.21.15.32:0/1055244402 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146815cdf0 con 0x7f14680cd580 2024-04-17T10:03:31.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.040+0000 7f1467400640 1 --2- 172.21.15.32:0/1055244402 >> v2:172.21.15.144:3300/0 conn(0x7f1468145f30 0x7f1468071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:03:31.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.044+0000 7f1467e00640 1 --2- 172.21.15.32:0/1055244402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14680cd580 0x7f1468072330 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f146806ff00 tx=0x7f146001f1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75612430a3c599d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:31.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.044+0000 7f1466a00640 1 -- 172.21.15.32:0/1055244402 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14600b4070 con 0x7f14680cd580 2024-04-17T10:03:31.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.044+0000 7f1466a00640 1 -- 172.21.15.32:0/1055244402 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14600af400 con 0x7f14680cd580 2024-04-17T10:03:31.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.044+0000 7f1466a00640 1 -- 172.21.15.32:0/1055244402 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14600af7d0 con 0x7f14680cd580 2024-04-17T10:03:31.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.044+0000 7f146e200640 1 -- 172.21.15.32:0/1055244402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14680cd580 msgr2=0x7f1468072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:31.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.044+0000 7f146e200640 1 --2- 172.21.15.32:0/1055244402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14680cd580 0x7f1468072330 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f146806ff00 tx=0x7f146001f1a0 comp rx=0 tx=0).stop 2024-04-17T10:03:31.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.044+0000 7f146e200640 1 -- 172.21.15.32:0/1055244402 shutdown_connections 2024-04-17T10:03:31.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.044+0000 7f146e200640 1 --2- 172.21.15.32:0/1055244402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14680cd580 0x7f1468072330 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:31.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.044+0000 7f146e200640 1 --2- 172.21.15.32:0/1055244402 >> v2:172.21.15.144:3300/0 conn(0x7f1468145f30 0x7f1468071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:31.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.044+0000 7f146e200640 1 -- 172.21.15.32:0/1055244402 >> 172.21.15.32:0/1055244402 conn(0x7f146806ba30 msgr2=0x7f146806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:31.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.044+0000 7f146e200640 1 -- 172.21.15.32:0/1055244402 shutdown_connections 2024-04-17T10:03:31.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.044+0000 7f146e200640 1 -- 172.21.15.32:0/1055244402 wait complete. 2024-04-17T10:03:31.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.044+0000 7f146e200640 1 Processor -- start 2024-04-17T10:03:31.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.044+0000 7f146e200640 1 -- start start 2024-04-17T10:03:31.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.044+0000 7f146e200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14680cd580 0x7f14680cae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:31.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.044+0000 7f146e200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1468145f30 0x7f14680cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:31.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.044+0000 7f146e200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f146815d8d0 0x7f14680cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:31.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.044+0000 7f146e200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f146815cb90 con 0x7f1468145f30 2024-04-17T10:03:31.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.044+0000 7f146e200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f146815d440 con 0x7f146815d8d0 2024-04-17T10:03:31.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.044+0000 7f146e200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f14681601d0 con 0x7f14680cd580 2024-04-17T10:03:31.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.048+0000 7f145fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1468145f30 0x7f14680cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:31.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.048+0000 7f145fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1468145f30 0x7f14680cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58636/0 (socket says 172.21.15.32:58636) 2024-04-17T10:03:31.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.048+0000 7f145fe00640 1 -- 172.21.15.32:0/1839329454 learned_addr learned my addr 172.21.15.32:0/1839329454 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:31.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.048+0000 7f1467400640 1 --2- 172.21.15.32:0/1839329454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14680cd580 0x7f14680cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:31.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.048+0000 7f1467e00640 1 --2- 172.21.15.32:0/1839329454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f146815d8d0 0x7f14680cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:31.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.048+0000 7f145fe00640 1 -- 172.21.15.32:0/1839329454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14680cd580 msgr2=0x7f14680cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:31.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.048+0000 7f145fe00640 1 --2- 172.21.15.32:0/1839329454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14680cd580 0x7f14680cae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:31.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.048+0000 7f145fe00640 1 -- 172.21.15.32:0/1839329454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f146815d8d0 msgr2=0x7f14680cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:31.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.048+0000 7f145fe00640 1 --2- 172.21.15.32:0/1839329454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f146815d8d0 0x7f14680cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:31.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.048+0000 7f145fe00640 1 -- 172.21.15.32:0/1839329454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14680b4170 con 0x7f1468145f30 2024-04-17T10:03:31.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.048+0000 7f145fe00640 1 --2- 172.21.15.32:0/1839329454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1468145f30 0x7f14680cb8b0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f145000b410 tx=0x7f145000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:31.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.048+0000 7f145f400640 1 -- 172.21.15.32:0/1839329454 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1450015030 con 0x7f1468145f30 2024-04-17T10:03:31.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.048+0000 7f145f400640 1 -- 172.21.15.32:0/1839329454 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1450004020 con 0x7f1468145f30 2024-04-17T10:03:31.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.048+0000 7f145f400640 1 -- 172.21.15.32:0/1839329454 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1450004410 con 0x7f1468145f30 2024-04-17T10:03:31.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.048+0000 7f146e200640 1 -- 172.21.15.32:0/1839329454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14680b4450 con 0x7f1468145f30 2024-04-17T10:03:31.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.048+0000 7f146e200640 1 -- 172.21.15.32:0/1839329454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14680b4a10 con 0x7f1468145f30 2024-04-17T10:03:31.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.048+0000 7f146e200640 1 -- 172.21.15.32:0/1839329454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1468068a90 con 0x7f1468145f30 2024-04-17T10:03:31.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.048+0000 7f145f400640 1 -- 172.21.15.32:0/1839329454 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1450018070 con 0x7f1468145f30 2024-04-17T10:03:31.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.052+0000 7f145f400640 1 --2- 172.21.15.32:0/1839329454 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1444068380 0x7f144406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:31.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.052+0000 7f145f400640 1 -- 172.21.15.32:0/1839329454 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5029+0+0 (secure 0 0 0) 0x7f1450086160 con 0x7f1468145f30 2024-04-17T10:03:31.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.052+0000 7f1467400640 1 --2- 172.21.15.32:0/1839329454 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1444068380 0x7f144406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:31.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.052+0000 7f1467400640 1 --2- 172.21.15.32:0/1839329454 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1444068380 0x7f144406a840 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f145800ab60 tx=0x7f1458008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:31.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.056+0000 7f145f400640 1 -- 172.21.15.32:0/1839329454 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f1450056360 con 0x7f1468145f30 2024-04-17T10:03:31.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:31.280+0000 7f146e200640 1 -- 172.21.15.32:0/1839329454 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f14680ccc90 con 0x7f1468145f30 2024-04-17T10:03:32.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.187+0000 7f145f400640 1 -- 172.21.15.32:0/1839329454 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v478) ==== 147+0+0 (secure 0 0 0) 0x7f14680ccc90 con 0x7f1468145f30 2024-04-17T10:03:32.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.191+0000 7f146e200640 1 -- 172.21.15.32:0/1839329454 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1444068380 msgr2=0x7f144406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:32.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.191+0000 7f146e200640 1 --2- 172.21.15.32:0/1839329454 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1444068380 0x7f144406a840 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f145800ab60 tx=0x7f1458008040 comp rx=0 tx=0).stop 2024-04-17T10:03:32.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.191+0000 7f146e200640 1 -- 172.21.15.32:0/1839329454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1468145f30 msgr2=0x7f14680cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:32.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.191+0000 7f146e200640 1 --2- 172.21.15.32:0/1839329454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1468145f30 0x7f14680cb8b0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f145000b410 tx=0x7f145000b8d0 comp rx=0 tx=0).stop 2024-04-17T10:03:32.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.191+0000 7f146e200640 1 -- 172.21.15.32:0/1839329454 shutdown_connections 2024-04-17T10:03:32.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.191+0000 7f146e200640 1 --2- 172.21.15.32:0/1839329454 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1444068380 0x7f144406a840 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:32.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.191+0000 7f146e200640 1 --2- 172.21.15.32:0/1839329454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f146815d8d0 0x7f14680cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:32.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.191+0000 7f146e200640 1 --2- 172.21.15.32:0/1839329454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1468145f30 0x7f14680cb8b0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:32.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.191+0000 7f146e200640 1 --2- 172.21.15.32:0/1839329454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14680cd580 0x7f14680cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:32.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.191+0000 7f146e200640 1 -- 172.21.15.32:0/1839329454 >> 172.21.15.32:0/1839329454 conn(0x7f146806ba30 msgr2=0x7f146815f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:32.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.191+0000 7f146e200640 1 -- 172.21.15.32:0/1839329454 shutdown_connections 2024-04-17T10:03:32.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.191+0000 7f146e200640 1 -- 172.21.15.32:0/1839329454 wait complete. 2024-04-17T10:03:32.216 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-17T10:03:32.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.371+0000 7fb961200640 1 Processor -- start 2024-04-17T10:03:32.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.371+0000 7fb961200640 1 -- start start 2024-04-17T10:03:32.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.371+0000 7fb961200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb95c06ba30 0x7fb95c06be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:32.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.371+0000 7fb961200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb95c06c350 con 0x7fb95c0cf370 2024-04-17T10:03:32.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.371+0000 7fb961200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb95c0d0130 con 0x7fb95c071e60 2024-04-17T10:03:32.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.371+0000 7fb961200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb95c0cfef0 con 0x7fb95c06ba30 2024-04-17T10:03:32.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.371+0000 7fb959400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb95c0cf370 0x7fb95c0cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57524/0 (socket says 172.21.15.32:57524) 2024-04-17T10:03:32.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.371+0000 7fb959400640 1 -- 172.21.15.32:0/591523888 learned_addr learned my addr 172.21.15.32:0/591523888 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:32.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.371+0000 7fb959e00640 1 --2- 172.21.15.32:0/591523888 >> v2:172.21.15.144:3300/0 conn(0x7fb95c06ba30 0x7fb95c06be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:32.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.371+0000 7fb958a00640 1 -- 172.21.15.32:0/591523888 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2757128881 0 0) 0x7fb95c06c350 con 0x7fb95c0cf370 2024-04-17T10:03:32.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.371+0000 7fb958a00640 1 -- 172.21.15.32:0/591523888 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb940003530 con 0x7fb95c0cf370 2024-04-17T10:03:32.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.371+0000 7fb959e00640 1 -- 172.21.15.32:0/591523888 >> v1:172.21.15.144:6789/0 conn(0x7fb95c071e60 legacy=0x7fb95c072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:32.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.371+0000 7fb959e00640 1 -- 172.21.15.32:0/591523888 >> v1:172.21.15.32:6789/0 conn(0x7fb95c0cf370 legacy=0x7fb95c0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:32.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.371+0000 7fb959e00640 1 -- 172.21.15.32:0/591523888 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95c15d090 con 0x7fb95c06ba30 2024-04-17T10:03:32.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.375+0000 7fb959e00640 1 --2- 172.21.15.32:0/591523888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb95c06ba30 0x7fb95c06be10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0097c0 tx=0x7fb94c01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c31050303bc31673 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:32.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.375+0000 7fb958a00640 1 -- 172.21.15.32:0/591523888 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c0b1070 con 0x7fb95c06ba30 2024-04-17T10:03:32.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.375+0000 7fb958a00640 1 -- 172.21.15.32:0/591523888 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb94c0a8050 con 0x7fb95c06ba30 2024-04-17T10:03:32.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.375+0000 7fb958a00640 1 -- 172.21.15.32:0/591523888 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c0ac860 con 0x7fb95c06ba30 2024-04-17T10:03:32.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.375+0000 7fb961200640 1 -- 172.21.15.32:0/591523888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb95c06ba30 msgr2=0x7fb95c06be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:32.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.375+0000 7fb961200640 1 --2- 172.21.15.32:0/591523888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb95c06ba30 0x7fb95c06be10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0097c0 tx=0x7fb94c01c3e0 comp rx=0 tx=0).stop 2024-04-17T10:03:32.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.375+0000 7fb961200640 1 -- 172.21.15.32:0/591523888 shutdown_connections 2024-04-17T10:03:32.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.375+0000 7fb961200640 1 --2- 172.21.15.32:0/591523888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb95c06ba30 0x7fb95c06be10 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:32.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.375+0000 7fb961200640 1 -- 172.21.15.32:0/591523888 >> 172.21.15.32:0/591523888 conn(0x7fb95c0b2120 msgr2=0x7fb95c0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:32.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.375+0000 7fb961200640 1 -- 172.21.15.32:0/591523888 shutdown_connections 2024-04-17T10:03:32.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.375+0000 7fb961200640 1 -- 172.21.15.32:0/591523888 wait complete. 2024-04-17T10:03:32.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb961200640 1 Processor -- start 2024-04-17T10:03:32.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb961200640 1 -- start start 2024-04-17T10:03:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb961200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb95c06ba30 0x7fb95c159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb961200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb95c071e60 0x7fb95c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb961200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb95c0cf370 0x7fb95c06f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb961200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c15db70 con 0x7fb95c071e60 2024-04-17T10:03:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb961200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c15cd00 con 0x7fb95c06ba30 2024-04-17T10:03:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb961200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb95c15dcf0 con 0x7fb95c0cf370 2024-04-17T10:03:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb959400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb95c071e60 0x7fb95c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb959400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb95c071e60 0x7fb95c06ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58644/0 (socket says 172.21.15.32:58644) 2024-04-17T10:03:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb959400640 1 -- 172.21.15.32:0/478230497 learned_addr learned my addr 172.21.15.32:0/478230497 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb959e00640 1 --2- 172.21.15.32:0/478230497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb95c06ba30 0x7fb95c159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb95a800640 1 --2- 172.21.15.32:0/478230497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb95c0cf370 0x7fb95c06f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb959400640 1 -- 172.21.15.32:0/478230497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb95c0cf370 msgr2=0x7fb95c06f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb959400640 1 --2- 172.21.15.32:0/478230497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb95c0cf370 0x7fb95c06f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb959400640 1 -- 172.21.15.32:0/478230497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb95c06ba30 msgr2=0x7fb95c159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb959400640 1 --2- 172.21.15.32:0/478230497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb95c06ba30 0x7fb95c159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb959400640 1 -- 172.21.15.32:0/478230497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95c13d120 con 0x7fb95c071e60 2024-04-17T10:03:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb959e00640 1 --2- 172.21.15.32:0/478230497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb95c06ba30 0x7fb95c159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:03:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb95a800640 1 --2- 172.21.15.32:0/478230497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb95c0cf370 0x7fb95c06f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:03:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb959400640 1 --2- 172.21.15.32:0/478230497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb95c071e60 0x7fb95c06ee80 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fb94800c450 tx=0x7fb94800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb952000640 1 -- 172.21.15.32:0/478230497 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948017070 con 0x7fb95c071e60 2024-04-17T10:03:32.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb952000640 1 -- 172.21.15.32:0/478230497 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb948005cf0 con 0x7fb95c071e60 2024-04-17T10:03:32.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb961200640 1 -- 172.21.15.32:0/478230497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95c13d400 con 0x7fb95c071e60 2024-04-17T10:03:32.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb952000640 1 -- 172.21.15.32:0/478230497 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948007270 con 0x7fb95c071e60 2024-04-17T10:03:32.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb961200640 1 -- 172.21.15.32:0/478230497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb95c13d9c0 con 0x7fb95c071e60 2024-04-17T10:03:32.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.379+0000 7fb961200640 1 -- 172.21.15.32:0/478230497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb95c068a90 con 0x7fb95c071e60 2024-04-17T10:03:32.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.383+0000 7fb952000640 1 -- 172.21.15.32:0/478230497 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb94801a070 con 0x7fb95c071e60 2024-04-17T10:03:32.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.383+0000 7fb952000640 1 --2- 172.21.15.32:0/478230497 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb93c0683b0 0x7fb93c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:32.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.383+0000 7fb952000640 1 -- 172.21.15.32:0/478230497 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5029+0+0 (secure 0 0 0) 0x7fb948085990 con 0x7fb95c071e60 2024-04-17T10:03:32.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.383+0000 7fb959e00640 1 --2- 172.21.15.32:0/478230497 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb93c0683b0 0x7fb93c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:32.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.383+0000 7fb959e00640 1 --2- 172.21.15.32:0/478230497 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb93c0683b0 0x7fb93c06a870 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0ae000 tx=0x7fb94c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:32.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.387+0000 7fb952000640 1 -- 172.21.15.32:0/478230497 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fb948055b00 con 0x7fb95c071e60 2024-04-17T10:03:32.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.551+0000 7fb961200640 1 -- 172.21.15.32:0/478230497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb95c06c850 con 0x7fb95c071e60 2024-04-17T10:03:32.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.555+0000 7fb952000640 1 -- 172.21.15.32:0/478230497 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7fb948059e00 con 0x7fb95c071e60 2024-04-17T10:03:32.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.559+0000 7fb961200640 1 -- 172.21.15.32:0/478230497 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb93c0683b0 msgr2=0x7fb93c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:32.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.559+0000 7fb961200640 1 --2- 172.21.15.32:0/478230497 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb93c0683b0 0x7fb93c06a870 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0ae000 tx=0x7fb94c008040 comp rx=0 tx=0).stop 2024-04-17T10:03:32.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.559+0000 7fb961200640 1 -- 172.21.15.32:0/478230497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb95c071e60 msgr2=0x7fb95c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:32.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.559+0000 7fb961200640 1 --2- 172.21.15.32:0/478230497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb95c071e60 0x7fb95c06ee80 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fb94800c450 tx=0x7fb94800c910 comp rx=0 tx=0).stop 2024-04-17T10:03:32.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.559+0000 7fb961200640 1 -- 172.21.15.32:0/478230497 shutdown_connections 2024-04-17T10:03:32.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.559+0000 7fb961200640 1 --2- 172.21.15.32:0/478230497 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb93c0683b0 0x7fb93c06a870 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:32.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.559+0000 7fb961200640 1 --2- 172.21.15.32:0/478230497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb95c0cf370 0x7fb95c06f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:32.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.559+0000 7fb961200640 1 --2- 172.21.15.32:0/478230497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb95c071e60 0x7fb95c06ee80 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:32.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.559+0000 7fb961200640 1 --2- 172.21.15.32:0/478230497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb95c06ba30 0x7fb95c159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:32.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.559+0000 7fb961200640 1 -- 172.21.15.32:0/478230497 >> 172.21.15.32:0/478230497 conn(0x7fb95c0b2120 msgr2=0x7fb95c13e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:32.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.559+0000 7fb961200640 1 -- 172.21.15.32:0/478230497 shutdown_connections 2024-04-17T10:03:32.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:32.559+0000 7fb961200640 1 -- 172.21.15.32:0/478230497 wait complete. 2024-04-17T10:03:32.576 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T10:03:32.576 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:03:32.686 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:03:32.778 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T10:03:32.778 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:03:32.779 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:03:32.787 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:03:32.787 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T10:03:32.787 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:03:32.796 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:03:32.796 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:03:32.797 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:03:32.839 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:03:32.839 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T10:03:32.839 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:03:32.848 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:03:32.848 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-17T10:03:32.848 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:03:33.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.023+0000 7fc0bf600640 1 Processor -- start 2024-04-17T10:03:33.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.027+0000 7fc0bf600640 1 -- start start 2024-04-17T10:03:33.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.027+0000 7fc0bf600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc0b815eb10 0x7fc0b815f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:33.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.027+0000 7fc0bf600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0b80b2550 0x7fc0b80d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:33.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.027+0000 7fc0bf600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0b806c640 0x7fc0b80d0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:33.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.027+0000 7fc0bf600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc0b80d1290 con 0x7fc0b806c640 2024-04-17T10:03:33.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.027+0000 7fc0bf600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc0b815ff10 con 0x7fc0b815eb10 2024-04-17T10:03:33.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.027+0000 7fc0bf600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc0b80b2da0 con 0x7fc0b80b2550 2024-04-17T10:03:33.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.027+0000 7fc0bd600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0b806c640 0x7fc0b80d0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:33.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.027+0000 7fc0bd600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0b806c640 0x7fc0b80d0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58652/0 (socket says 172.21.15.32:58652) 2024-04-17T10:03:33.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.027+0000 7fc0bd600640 1 -- 172.21.15.32:0/1959597346 learned_addr learned my addr 172.21.15.32:0/1959597346 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:33.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.027+0000 7fc0bd600640 1 -- 172.21.15.32:0/1959597346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0b80b2550 msgr2=0x7fc0b80d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:33.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.027+0000 7fc0bd600640 1 --2- 172.21.15.32:0/1959597346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0b80b2550 0x7fc0b80d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:33.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.027+0000 7fc0bd600640 1 -- 172.21.15.32:0/1959597346 >> v2:172.21.15.144:3300/0 conn(0x7fc0b815eb10 msgr2=0x7fc0b815f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:33.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.027+0000 7fc0bd600640 1 --2- 172.21.15.32:0/1959597346 >> v2:172.21.15.144:3300/0 conn(0x7fc0b815eb10 0x7fc0b815f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:33.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.027+0000 7fc0bd600640 1 -- 172.21.15.32:0/1959597346 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b80b2ac0 con 0x7fc0b806c640 2024-04-17T10:03:33.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.031+0000 7fc0bd600640 1 --2- 172.21.15.32:0/1959597346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b806c640 0x7fc0b80d0d50 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80cefe0 tx=0x7fc0b001c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6997f8854a1b9a00 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:33.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.031+0000 7fc0b7400640 1 -- 172.21.15.32:0/1959597346 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b00a5640 con 0x7fc0b806c640 2024-04-17T10:03:33.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.031+0000 7fc0b7400640 1 -- 172.21.15.32:0/1959597346 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b00a57e0 con 0x7fc0b806c640 2024-04-17T10:03:33.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.031+0000 7fc0b7400640 1 -- 172.21.15.32:0/1959597346 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b00a5bb0 con 0x7fc0b806c640 2024-04-17T10:03:33.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.031+0000 7fc0bf600640 1 -- 172.21.15.32:0/1959597346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b806c640 msgr2=0x7fc0b80d0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.031+0000 7fc0bf600640 1 --2- 172.21.15.32:0/1959597346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b806c640 0x7fc0b80d0d50 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80cefe0 tx=0x7fc0b001c8d0 comp rx=0 tx=0).stop 2024-04-17T10:03:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.031+0000 7fc0bf600640 1 -- 172.21.15.32:0/1959597346 shutdown_connections 2024-04-17T10:03:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.031+0000 7fc0bf600640 1 --2- 172.21.15.32:0/1959597346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b806c640 0x7fc0b80d0d50 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.031+0000 7fc0bf600640 1 --2- 172.21.15.32:0/1959597346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0b80b2550 0x7fc0b80d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.031+0000 7fc0bf600640 1 --2- 172.21.15.32:0/1959597346 >> v2:172.21.15.144:3300/0 conn(0x7fc0b815eb10 0x7fc0b815f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.031+0000 7fc0bf600640 1 -- 172.21.15.32:0/1959597346 >> 172.21.15.32:0/1959597346 conn(0x7fc0b809a9f0 msgr2=0x7fc0b80b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.031+0000 7fc0bf600640 1 -- 172.21.15.32:0/1959597346 shutdown_connections 2024-04-17T10:03:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.031+0000 7fc0bf600640 1 -- 172.21.15.32:0/1959597346 wait complete. 2024-04-17T10:03:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.031+0000 7fc0bf600640 1 Processor -- start 2024-04-17T10:03:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.031+0000 7fc0bf600640 1 -- start start 2024-04-17T10:03:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0bf600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b806c640 0x7fc0b80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0bf600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0b80b2550 0x7fc0b8156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0bf600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0b815eb10 0x7fc0b8070120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0bf600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b8160d50 con 0x7fc0b806c640 2024-04-17T10:03:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0bf600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b8160bd0 con 0x7fc0b815eb10 2024-04-17T10:03:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0bf600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc0b8160ed0 con 0x7fc0b80b2550 2024-04-17T10:03:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0bcc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b806c640 0x7fc0b80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0bcc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b806c640 0x7fc0b80b4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58662/0 (socket says 172.21.15.32:58662) 2024-04-17T10:03:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0bcc00640 1 -- 172.21.15.32:0/4259332707 learned_addr learned my addr 172.21.15.32:0/4259332707 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0b7e00640 1 --2- 172.21.15.32:0/4259332707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0b80b2550 0x7fc0b8156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0bd600640 1 --2- 172.21.15.32:0/4259332707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0b815eb10 0x7fc0b8070120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0bcc00640 1 -- 172.21.15.32:0/4259332707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0b80b2550 msgr2=0x7fc0b8156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0bcc00640 1 --2- 172.21.15.32:0/4259332707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0b80b2550 0x7fc0b8156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0bcc00640 1 -- 172.21.15.32:0/4259332707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0b815eb10 msgr2=0x7fc0b8070120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0bcc00640 1 --2- 172.21.15.32:0/4259332707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0b815eb10 0x7fc0b8070120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:33.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0bcc00640 1 -- 172.21.15.32:0/4259332707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b80b5470 con 0x7fc0b806c640 2024-04-17T10:03:33.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0bcc00640 1 --2- 172.21.15.32:0/4259332707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b806c640 0x7fc0b80b4f30 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00b440 tx=0x7fc0ac00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:33.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0b4c00640 1 -- 172.21.15.32:0/4259332707 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac015020 con 0x7fc0b806c640 2024-04-17T10:03:33.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0b4c00640 1 -- 172.21.15.32:0/4259332707 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0ac011850 con 0x7fc0b806c640 2024-04-17T10:03:33.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0b4c00640 1 -- 172.21.15.32:0/4259332707 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac011c40 con 0x7fc0b806c640 2024-04-17T10:03:33.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0bf600640 1 -- 172.21.15.32:0/4259332707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b81575a0 con 0x7fc0b806c640 2024-04-17T10:03:33.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0bf600640 1 -- 172.21.15.32:0/4259332707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b8157b60 con 0x7fc0b806c640 2024-04-17T10:03:33.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0bf600640 1 -- 172.21.15.32:0/4259332707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0b8068a90 con 0x7fc0b806c640 2024-04-17T10:03:33.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.035+0000 7fc0b4c00640 1 -- 172.21.15.32:0/4259332707 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc0ac018020 con 0x7fc0b806c640 2024-04-17T10:03:33.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.039+0000 7fc0b4c00640 1 --2- 172.21.15.32:0/4259332707 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc098068380 0x7fc09806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:33.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.039+0000 7fc0b4c00640 1 -- 172.21.15.32:0/4259332707 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5029+0+0 (secure 0 0 0) 0x7fc0ac0864f0 con 0x7fc0b806c640 2024-04-17T10:03:33.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.039+0000 7fc0b7e00640 1 --2- 172.21.15.32:0/4259332707 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc098068380 0x7fc09806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:33.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.039+0000 7fc0b7e00640 1 --2- 172.21.15.32:0/4259332707 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc098068380 0x7fc09806a840 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80cbf20 tx=0x7fc0a4009110 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:33.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.043+0000 7fc0b4c00640 1 -- 172.21.15.32:0/4259332707 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc0ac01f020 con 0x7fc0b806c640 2024-04-17T10:03:33.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.243+0000 7fc0bf600640 1 -- 172.21.15.32:0/4259332707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0b8158260 con 0x7fc0b806c640 2024-04-17T10:03:33.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.243+0000 7fc0b4c00640 1 -- 172.21.15.32:0/4259332707 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+3872 (secure 0 0 0) 0x7fc0ac0566e0 con 0x7fc0b806c640 2024-04-17T10:03:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:03:33.241 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":91,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2635109588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2635109588},{"type":"v1","addr":"172.21.15.144:6835","nonce":2635109588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7789,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3258455939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3258455939},{"type":"v1","addr":"172.21.15.144:6834","nonce":3258455939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":8574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3390468332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3390468332},{"type":"v1","addr":"172.21.15.32:6836","nonce":3390468332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":9123,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/374146310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":374146310},{"type":"v1","addr":"172.21.15.32:6837","nonce":374146310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[]} 2024-04-17T10:03:33.242 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 91 2024-04-17T10:03:33.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.243+0000 7fc0bf600640 1 -- 172.21.15.32:0/4259332707 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc098068380 msgr2=0x7fc09806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:33.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.243+0000 7fc0bf600640 1 --2- 172.21.15.32:0/4259332707 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc098068380 0x7fc09806a840 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80cbf20 tx=0x7fc0a4009110 comp rx=0 tx=0).stop 2024-04-17T10:03:33.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.243+0000 7fc0bf600640 1 -- 172.21.15.32:0/4259332707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b806c640 msgr2=0x7fc0b80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:33.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.243+0000 7fc0bf600640 1 --2- 172.21.15.32:0/4259332707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b806c640 0x7fc0b80b4f30 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00b440 tx=0x7fc0ac00b900 comp rx=0 tx=0).stop 2024-04-17T10:03:33.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.243+0000 7fc0bf600640 1 -- 172.21.15.32:0/4259332707 shutdown_connections 2024-04-17T10:03:33.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.243+0000 7fc0bf600640 1 --2- 172.21.15.32:0/4259332707 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc098068380 0x7fc09806a840 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:33.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.243+0000 7fc0bf600640 1 --2- 172.21.15.32:0/4259332707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0b80b2550 0x7fc0b8156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:33.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.243+0000 7fc0bf600640 1 --2- 172.21.15.32:0/4259332707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0b815eb10 0x7fc0b8070120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:33.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.243+0000 7fc0bf600640 1 --2- 172.21.15.32:0/4259332707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b806c640 0x7fc0b80b4f30 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:33.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.247+0000 7fc0bf600640 1 -- 172.21.15.32:0/4259332707 >> 172.21.15.32:0/4259332707 conn(0x7fc0b809a9f0 msgr2=0x7fc0b8161300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:33.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.247+0000 7fc0bf600640 1 -- 172.21.15.32:0/4259332707 shutdown_connections 2024-04-17T10:03:33.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.247+0000 7fc0bf600640 1 -- 172.21.15.32:0/4259332707 wait complete. 2024-04-17T10:03:33.263 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-17T10:03:33.263 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-17T10:03:33.263 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-17T10:03:33.263 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:03:33.264 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-17T10:03:33.264 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-17T10:03:33.264 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-17T10:03:33.264 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:03:33.264 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-17T10:03:33.264 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-17T10:03:33.264 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-17T10:03:33.264 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:03:33.265 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-17T10:03:33.265 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-17T10:03:33.265 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-17T10:03:33.265 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:03:33.276 INFO:tasks.ceph.mds.a:Stopped 2024-04-17T10:03:33.276 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-17T10:03:33.277 INFO:tasks.ceph.mds.c:Stopped 2024-04-17T10:03:33.277 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-17T10:03:33.279 INFO:tasks.ceph.mds.b:Stopped 2024-04-17T10:03:33.279 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-17T10:03:33.279 INFO:tasks.ceph.mds.d:Stopped 2024-04-17T10:03:33.280 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-17T10:03:33.326 INFO:tasks.ceph.mds.a:Started 2024-04-17T10:03:33.326 DEBUG:teuthology.parallel:result is None 2024-04-17T10:03:33.327 INFO:tasks.ceph.mds.c:Started 2024-04-17T10:03:33.327 DEBUG:teuthology.parallel:result is None 2024-04-17T10:03:33.330 INFO:tasks.ceph.mds.b:Started 2024-04-17T10:03:33.331 DEBUG:teuthology.parallel:result is None 2024-04-17T10:03:33.332 INFO:tasks.ceph.mds.d:Started 2024-04-17T10:03:33.333 DEBUG:teuthology.parallel:result is None 2024-04-17T10:03:33.333 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-17T10:03:33.411 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2024-04-17T10:03:33.414 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-17T10:03:33.420 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2024-04-17T10:03:33.421 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2024-04-17T10:03:33.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.483+0000 7ff60ec00640 1 Processor -- start 2024-04-17T10:03:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.483+0000 7ff60ec00640 1 -- start start 2024-04-17T10:03:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.483+0000 7ff60ec00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff60815e880 0x7ff6080cb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.483+0000 7ff60ec00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff6080702c0 0x7ff608070710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.483+0000 7ff60ec00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6080d0dc0 con 0x7ff6080b3530 2024-04-17T10:03:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.483+0000 7ff60ec00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff608070c50 con 0x7ff60815e880 2024-04-17T10:03:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.483+0000 7ff60ec00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff6080d10a0 con 0x7ff6080702c0 2024-04-17T10:03:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.483+0000 7ff60d800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff60815e880 0x7ff6080cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.483+0000 7ff60d800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff60815e880 0x7ff6080cb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58706/0 (socket says 172.21.15.32:58706) 2024-04-17T10:03:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.483+0000 7ff60d800640 1 -- 172.21.15.32:0/1393113938 learned_addr learned my addr 172.21.15.32:0/1393113938 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.483+0000 7ff60e200640 1 --2- 172.21.15.32:0/1393113938 >> v2:172.21.15.144:3300/0 conn(0x7ff6080702c0 0x7ff608070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.483+0000 7ff60d800640 1 -- 172.21.15.32:0/1393113938 >> v2:172.21.15.144:3300/0 conn(0x7ff6080702c0 msgr2=0x7ff608070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.483+0000 7ff60d800640 1 --2- 172.21.15.32:0/1393113938 >> v2:172.21.15.144:3300/0 conn(0x7ff6080702c0 0x7ff608070710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.483+0000 7ff60d800640 1 -- 172.21.15.32:0/1393113938 >> v1:172.21.15.144:6789/0 conn(0x7ff6080b3530 legacy=0x7ff6080d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.483+0000 7ff60d800640 1 -- 172.21.15.32:0/1393113938 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6080cd570 con 0x7ff60815e880 2024-04-17T10:03:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.483+0000 7ff607e00640 1 -- 172.21.15.32:0/1393113938 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2113494236 0 0) 0x7ff6080d0dc0 con 0x7ff6080b3530 2024-04-17T10:03:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.487+0000 7ff60d800640 1 --2- 172.21.15.32:0/1393113938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff60815e880 0x7ff6080cb900 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7ff6080c8310 tx=0x7ff5fc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b615a966b7f574bd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.487+0000 7ff607e00640 1 -- 172.21.15.32:0/1393113938 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc0b1070 con 0x7ff60815e880 2024-04-17T10:03:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.487+0000 7ff607e00640 1 -- 172.21.15.32:0/1393113938 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5fc0ac600 con 0x7ff60815e880 2024-04-17T10:03:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.487+0000 7ff607e00640 1 -- 172.21.15.32:0/1393113938 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc0ac9d0 con 0x7ff60815e880 2024-04-17T10:03:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.487+0000 7ff60ec00640 1 -- 172.21.15.32:0/1393113938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff60815e880 msgr2=0x7ff6080cb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.487+0000 7ff60ec00640 1 --2- 172.21.15.32:0/1393113938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff60815e880 0x7ff6080cb900 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7ff6080c8310 tx=0x7ff5fc01c490 comp rx=0 tx=0).stop 2024-04-17T10:03:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.487+0000 7ff60ec00640 1 -- 172.21.15.32:0/1393113938 shutdown_connections 2024-04-17T10:03:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.487+0000 7ff60ec00640 1 --2- 172.21.15.32:0/1393113938 >> v2:172.21.15.144:3300/0 conn(0x7ff6080702c0 0x7ff608070710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.487+0000 7ff60ec00640 1 --2- 172.21.15.32:0/1393113938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff60815e880 0x7ff6080cb900 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.487+0000 7ff60ec00640 1 -- 172.21.15.32:0/1393113938 >> 172.21.15.32:0/1393113938 conn(0x7ff60809a9f0 msgr2=0x7ff6080b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.487+0000 7ff60ec00640 1 -- 172.21.15.32:0/1393113938 shutdown_connections 2024-04-17T10:03:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.487+0000 7ff60ec00640 1 -- 172.21.15.32:0/1393113938 wait complete. 2024-04-17T10:03:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff60ec00640 1 Processor -- start 2024-04-17T10:03:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff60ec00640 1 -- start start 2024-04-17T10:03:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff60ec00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6080702c0 0x7ff60815a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff60ec00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6080b3530 0x7ff608149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff60ec00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff60815e880 0x7ff608149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff60ec00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff60815ede0 con 0x7ff6080b3530 2024-04-17T10:03:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff60ec00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff60815ec60 con 0x7ff6080702c0 2024-04-17T10:03:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff60ec00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff6080ce860 con 0x7ff60815e880 2024-04-17T10:03:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff60ce00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6080b3530 0x7ff608149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff60ce00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6080b3530 0x7ff608149780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58716/0 (socket says 172.21.15.32:58716) 2024-04-17T10:03:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff60ce00640 1 -- 172.21.15.32:0/1000020653 learned_addr learned my addr 172.21.15.32:0/1000020653 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff60d800640 1 --2- 172.21.15.32:0/1000020653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6080702c0 0x7ff60815a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff60e200640 1 --2- 172.21.15.32:0/1000020653 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff60815e880 0x7ff608149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff60ce00640 1 -- 172.21.15.32:0/1000020653 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff60815e880 msgr2=0x7ff608149cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff60ce00640 1 --2- 172.21.15.32:0/1000020653 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff60815e880 0x7ff608149cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff60ce00640 1 -- 172.21.15.32:0/1000020653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6080702c0 msgr2=0x7ff60815a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff60ce00640 1 --2- 172.21.15.32:0/1000020653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6080702c0 0x7ff60815a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff60ce00640 1 -- 172.21.15.32:0/1000020653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6081504f0 con 0x7ff6080b3530 2024-04-17T10:03:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff60e200640 1 --2- 172.21.15.32:0/1000020653 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff60815e880 0x7ff608149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff60ce00640 1 --2- 172.21.15.32:0/1000020653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6080b3530 0x7ff608149780 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400c4d0 tx=0x7ff5f400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff605600640 1 -- 172.21.15.32:0/1000020653 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4017070 con 0x7ff6080b3530 2024-04-17T10:03:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff605600640 1 -- 172.21.15.32:0/1000020653 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5f4005480 con 0x7ff6080b3530 2024-04-17T10:03:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff605600640 1 -- 172.21.15.32:0/1000020653 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4003840 con 0x7ff6080b3530 2024-04-17T10:03:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff60ec00640 1 -- 172.21.15.32:0/1000020653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff60814f730 con 0x7ff6080b3530 2024-04-17T10:03:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.491+0000 7ff60ec00640 1 -- 172.21.15.32:0/1000020653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff60814fcf0 con 0x7ff6080b3530 2024-04-17T10:03:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.495+0000 7ff60ec00640 1 -- 172.21.15.32:0/1000020653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff60804fd00 con 0x7ff6080b3530 2024-04-17T10:03:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.495+0000 7ff605600640 1 -- 172.21.15.32:0/1000020653 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff5f401a070 con 0x7ff6080b3530 2024-04-17T10:03:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.495+0000 7ff605600640 1 --2- 172.21.15.32:0/1000020653 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff5e80683b0 0x7ff5e806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:33.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.495+0000 7ff605600640 1 -- 172.21.15.32:0/1000020653 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5029+0+0 (secure 0 0 0) 0x7ff5f4085a70 con 0x7ff6080b3530 2024-04-17T10:03:33.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.495+0000 7ff60d800640 1 --2- 172.21.15.32:0/1000020653 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff5e80683b0 0x7ff5e806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:33.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.495+0000 7ff60d800640 1 --2- 172.21.15.32:0/1000020653 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff5e80683b0 0x7ff5e806a870 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0ae000 tx=0x7ff5fc0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.499+0000 7ff605600640 1 -- 172.21.15.32:0/1000020653 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff5f4055d30 con 0x7ff6080b3530 2024-04-17T10:03:33.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:33.675+0000 7ff60ec00640 1 -- 172.21.15.32:0/1000020653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff6080cb4d0 con 0x7ff6080b3530 2024-04-17T10:03:34.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.199+0000 7ff605600640 1 -- 172.21.15.32:0/1000020653 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v103) ==== 97+0+0 (secure 0 0 0) 0x7ff5f405a030 con 0x7ff6080b3530 2024-04-17T10:03:34.199 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2024-04-17T10:03:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.203+0000 7ff60ec00640 1 -- 172.21.15.32:0/1000020653 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff5e80683b0 msgr2=0x7ff5e806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.203+0000 7ff60ec00640 1 --2- 172.21.15.32:0/1000020653 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff5e80683b0 0x7ff5e806a870 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0ae000 tx=0x7ff5fc0a7000 comp rx=0 tx=0).stop 2024-04-17T10:03:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.203+0000 7ff60ec00640 1 -- 172.21.15.32:0/1000020653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6080b3530 msgr2=0x7ff608149780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.203+0000 7ff60ec00640 1 --2- 172.21.15.32:0/1000020653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6080b3530 0x7ff608149780 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400c4d0 tx=0x7ff5f400c990 comp rx=0 tx=0).stop 2024-04-17T10:03:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.203+0000 7ff60ec00640 1 -- 172.21.15.32:0/1000020653 shutdown_connections 2024-04-17T10:03:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.203+0000 7ff60ec00640 1 --2- 172.21.15.32:0/1000020653 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff5e80683b0 0x7ff5e806a870 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.203+0000 7ff60ec00640 1 --2- 172.21.15.32:0/1000020653 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff60815e880 0x7ff608149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.203+0000 7ff60ec00640 1 --2- 172.21.15.32:0/1000020653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6080b3530 0x7ff608149780 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.203+0000 7ff60ec00640 1 --2- 172.21.15.32:0/1000020653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6080702c0 0x7ff60815a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.203+0000 7ff60ec00640 1 -- 172.21.15.32:0/1000020653 >> 172.21.15.32:0/1000020653 conn(0x7ff60809a9f0 msgr2=0x7ff6080b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.203+0000 7ff60ec00640 1 -- 172.21.15.32:0/1000020653 shutdown_connections 2024-04-17T10:03:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.203+0000 7ff60ec00640 1 -- 172.21.15.32:0/1000020653 wait complete. 2024-04-17T10:03:34.227 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-17T10:03:34.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.371+0000 7fcaaea00640 1 Processor -- start 2024-04-17T10:03:34.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.371+0000 7fcaaea00640 1 -- start start 2024-04-17T10:03:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.371+0000 7fcaaea00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcaa81481e0 0x7fcaa8071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.371+0000 7fcaaea00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcaa8072330 0x7fcaa80b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.371+0000 7fcaaea00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaa806ef70 con 0x7fcaa8144f20 2024-04-17T10:03:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.371+0000 7fcaaea00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fcaa80aed20 con 0x7fcaa8072330 2024-04-17T10:03:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.371+0000 7fcaaea00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcaa80af000 con 0x7fcaa81481e0 2024-04-17T10:03:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.375+0000 7fca9fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcaa81481e0 0x7fcaa8071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.375+0000 7fcaaca00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcaa8072330 0x7fcaa80b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.375+0000 7fcaa7e00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fcaa8144f20 0x7fcaa806e860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:60036/0 (socket says 172.21.15.32:60036) 2024-04-17T10:03:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.375+0000 7fcaa7e00640 1 -- 172.21.15.32:0/18886339 learned_addr learned my addr 172.21.15.32:0/18886339 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.375+0000 7fcaaca00640 1 -- 172.21.15.32:0/18886339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcaa81481e0 msgr2=0x7fcaa8071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.375+0000 7fcaaca00640 1 --2- 172.21.15.32:0/18886339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcaa81481e0 0x7fcaa8071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.375+0000 7fcaaca00640 1 -- 172.21.15.32:0/18886339 >> v1:172.21.15.144:6789/0 conn(0x7fcaa8144f20 legacy=0x7fcaa806e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:34.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.375+0000 7fcaaca00640 1 -- 172.21.15.32:0/18886339 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa80af2e0 con 0x7fcaa8072330 2024-04-17T10:03:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.375+0000 7fcaaca00640 1 --2- 172.21.15.32:0/18886339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaa8072330 0x7fcaa80b3490 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009d90 tx=0x7fcaa001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6023cbbc1b9bc334 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:34.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.375+0000 7fcaa7400640 1 -- 172.21.15.32:0/18886339 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00b2070 con 0x7fcaa8072330 2024-04-17T10:03:34.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.375+0000 7fcaa7400640 1 -- 172.21.15.32:0/18886339 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaa00a5830 con 0x7fcaa8072330 2024-04-17T10:03:34.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.375+0000 7fcaa7400640 1 -- 172.21.15.32:0/18886339 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00a5c00 con 0x7fcaa8072330 2024-04-17T10:03:34.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fcaaea00640 1 -- 172.21.15.32:0/18886339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaa8072330 msgr2=0x7fcaa80b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:34.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fcaaea00640 1 --2- 172.21.15.32:0/18886339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaa8072330 0x7fcaa80b3490 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009d90 tx=0x7fcaa001c970 comp rx=0 tx=0).stop 2024-04-17T10:03:34.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fcaaea00640 1 -- 172.21.15.32:0/18886339 shutdown_connections 2024-04-17T10:03:34.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fcaaea00640 1 --2- 172.21.15.32:0/18886339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaa8072330 0x7fcaa80b3490 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:34.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fcaaea00640 1 --2- 172.21.15.32:0/18886339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcaa81481e0 0x7fcaa8071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:34.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fcaaea00640 1 -- 172.21.15.32:0/18886339 >> 172.21.15.32:0/18886339 conn(0x7fcaa806ba30 msgr2=0x7fcaa806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:34.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fcaaea00640 1 -- 172.21.15.32:0/18886339 shutdown_connections 2024-04-17T10:03:34.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fcaaea00640 1 -- 172.21.15.32:0/18886339 wait complete. 2024-04-17T10:03:34.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fcaaea00640 1 Processor -- start 2024-04-17T10:03:34.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fcaaea00640 1 -- start start 2024-04-17T10:03:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fcaaea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa8072330 0x7fcaa813f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fcaaea00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcaa8144f20 0x7fcaa8141020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fcaaea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaa81481e0 0x7fcaa80b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fcaaea00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa80b4360 con 0x7fcaa8072330 2024-04-17T10:03:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fcaaea00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa80b41e0 con 0x7fcaa81481e0 2024-04-17T10:03:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fcaaea00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcaa816b5d0 con 0x7fcaa8144f20 2024-04-17T10:03:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fcaa7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa8072330 0x7fcaa813f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fca9fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcaa8144f20 0x7fcaa8141020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fcaa7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa8072330 0x7fcaa813f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58718/0 (socket says 172.21.15.32:58718) 2024-04-17T10:03:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fcaa7e00640 1 -- 172.21.15.32:0/1570986704 learned_addr learned my addr 172.21.15.32:0/1570986704 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fcaaca00640 1 --2- 172.21.15.32:0/1570986704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaa81481e0 0x7fcaa80b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fca9fe00640 1 -- 172.21.15.32:0/1570986704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaa81481e0 msgr2=0x7fcaa80b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fca9fe00640 1 --2- 172.21.15.32:0/1570986704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaa81481e0 0x7fcaa80b1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fca9fe00640 1 -- 172.21.15.32:0/1570986704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa8072330 msgr2=0x7fcaa813f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fca9fe00640 1 --2- 172.21.15.32:0/1570986704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa8072330 0x7fcaa813f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.379+0000 7fca9fe00640 1 -- 172.21.15.32:0/1570986704 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa813ff80 con 0x7fcaa8144f20 2024-04-17T10:03:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.383+0000 7fca9fe00640 1 --2- 172.21.15.32:0/1570986704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcaa8144f20 0x7fcaa8141020 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fca9000e840 tx=0x7fca9000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:34.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.383+0000 7fcaa4c00640 1 -- 172.21.15.32:0/1570986704 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90017070 con 0x7fcaa8144f20 2024-04-17T10:03:34.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.383+0000 7fcaa4c00640 1 -- 172.21.15.32:0/1570986704 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca9000c940 con 0x7fcaa8144f20 2024-04-17T10:03:34.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.383+0000 7fcaaea00640 1 -- 172.21.15.32:0/1570986704 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa8140260 con 0x7fcaa8144f20 2024-04-17T10:03:34.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.383+0000 7fcaa4c00640 1 -- 172.21.15.32:0/1570986704 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9000ccf0 con 0x7fcaa8144f20 2024-04-17T10:03:34.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.383+0000 7fcaaea00640 1 -- 172.21.15.32:0/1570986704 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa8140820 con 0x7fcaa8144f20 2024-04-17T10:03:34.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.383+0000 7fcaaea00640 1 -- 172.21.15.32:0/1570986704 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaa8068a90 con 0x7fcaa8144f20 2024-04-17T10:03:34.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.383+0000 7fcaa4c00640 1 -- 172.21.15.32:0/1570986704 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fca9001a070 con 0x7fcaa8144f20 2024-04-17T10:03:34.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.383+0000 7fcaa4c00640 1 --2- 172.21.15.32:0/1570986704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fca840683b0 0x7fca8406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:34.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.383+0000 7fcaa4c00640 1 -- 172.21.15.32:0/1570986704 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(105..105 src has 1..105) ==== 4857+0+0 (secure 0 0 0) 0x7fca90086910 con 0x7fcaa8144f20 2024-04-17T10:03:34.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.383+0000 7fcaa7e00640 1 --2- 172.21.15.32:0/1570986704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fca840683b0 0x7fca8406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:34.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.387+0000 7fcaa7e00640 1 --2- 172.21.15.32:0/1570986704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fca840683b0 0x7fca8406a870 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fcaa815e620 tx=0x7fca98007f20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:34.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.387+0000 7fcaa4c00640 1 -- 172.21.15.32:0/1570986704 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fca90056bb0 con 0x7fcaa8144f20 2024-04-17T10:03:34.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.607+0000 7fcaaea00640 1 -- 172.21.15.32:0/1570986704 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fcaa806c200 con 0x7fcaa8144f20 2024-04-17T10:03:34.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.607+0000 7fcaa4c00640 1 -- 172.21.15.32:0/1570986704 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) ==== 80+0+5304 (secure 0 0 0) 0x7fca9005aeb0 con 0x7fcaa8144f20 2024-04-17T10:03:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:03:34.606 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:03:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2024-04-17T10:03:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:03:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2024-04-17T10:03:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mzM1KExAAn0WH+Gf6gGV/dJonXzAgPQ==", 2024-04-17T10:03:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:03:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T10:03:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T10:03:34.606 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T10:03:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:03:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:03:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2024-04-17T10:03:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mQnJ4HxAAT0rFHhdAj7PMxZGiqEyWbg==", 2024-04-17T10:03:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:03:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T10:03:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T10:03:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T10:03:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:03:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:03:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2024-04-17T10:03:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mWWi5GhAA3RAdRxK6GRwuVCH4DHoMeA==", 2024-04-17T10:03:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:03:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T10:03:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T10:03:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T10:03:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:03:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:03:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2024-04-17T10:03:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9m9TMAJxAAbmUp05GHRE1AqCmq83scVQ==", 2024-04-17T10:03:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:03:34.608 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T10:03:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T10:03:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T10:03:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:03:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:03:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2024-04-17T10:03:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDRmx9mnPwyDBAAxD10jhxptVH5zhupengElg==", 2024-04-17T10:03:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:03:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T10:03:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T10:03:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:03:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:34.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:03:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2024-04-17T10:03:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDUmx9mh3iuGBAAqvWSRyNX3u/TjEH6TJpqdA==", 2024-04-17T10:03:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:03:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T10:03:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T10:03:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:03:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:03:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2024-04-17T10:03:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDXmx9mbe1CIRAA1pmzAl04tGkTu6QLpmCkag==", 2024-04-17T10:03:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:03:34.610 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T10:03:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T10:03:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:03:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:03:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2024-04-17T10:03:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDamx9mIwXoKxAAWPTFaYdp4lIZhuOU7F0x8g==", 2024-04-17T10:03:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:03:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T10:03:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T10:03:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:03:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:03:34.611 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2024-04-17T10:03:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDgmx9mp0s/JxAAv+usQqwrxgKINRlkDhhydQ==", 2024-04-17T10:03:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:03:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T10:03:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T10:03:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:03:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:03:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2024-04-17T10:03:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDjmx9mMVStMhAA/n8jk+LVDosWhziXk/lLRQ==", 2024-04-17T10:03:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:03:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T10:03:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T10:03:34.612 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:03:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:03:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2024-04-17T10:03:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDnmx9m/YQlABAAOqfWzVtgonD7b+WlR95mnA==", 2024-04-17T10:03:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:03:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T10:03:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T10:03:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:03:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:03:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2024-04-17T10:03:34.613 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDqmx9m572mCBAAqBib/yeQOBno+0B1K1dLxA==", 2024-04-17T10:03:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:03:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2024-04-17T10:03:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2024-04-17T10:03:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:03:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:03:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2024-04-17T10:03:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mUXoLKxAANV7zhLvL0DqHngk4TtsdBQ==", 2024-04-17T10:03:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:03:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T10:03:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2024-04-17T10:03:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-17T10:03:34.614 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:03:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2024-04-17T10:03:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mZBeRNBAAQg5K950mp6+XND7NCABAKQ==", 2024-04-17T10:03:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:03:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T10:03:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-17T10:03:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-17T10:03:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-17T10:03:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:03:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2024-04-17T10:03:34.615 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mWKstMRAAp18Hbnk02w+hT0sYZ3tSLA==", 2024-04-17T10:03:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:03:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T10:03:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-17T10:03:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-17T10:03:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-17T10:03:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:03:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2024-04-17T10:03:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9mRQ2wOhAAzlWXFtP0WaXWmneqxnZ5/w==", 2024-04-17T10:03:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:03:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2024-04-17T10:03:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2024-04-17T10:03:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2024-04-17T10:03:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2024-04-17T10:03:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:03:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2024-04-17T10:03:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDOmx9maDStMxAAMCa36HfycwV3WbLg6Voo9Q==", 2024-04-17T10:03:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:03:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-17T10:03:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2024-04-17T10:03:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2024-04-17T10:03:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:03:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:03:34.617 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2024-04-17T10:03:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9myy0oDxAAPD2BTjIP09hiU5L7i6gUMg==", 2024-04-17T10:03:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:03:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-17T10:03:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-17T10:03:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:03:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:03:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2024-04-17T10:03:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQDPmx9m3FmACxAApri5AI5PeR0+/HLev8DKcw==", 2024-04-17T10:03:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2024-04-17T10:03:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2024-04-17T10:03:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2024-04-17T10:03:34.618 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2024-04-17T10:03:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:34.619 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-04-17T10:03:34.619 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:03:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.611+0000 7fcaaea00640 1 -- 172.21.15.32:0/1570986704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fca840683b0 msgr2=0x7fca8406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.611+0000 7fcaaea00640 1 --2- 172.21.15.32:0/1570986704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fca840683b0 0x7fca8406a870 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fcaa815e620 tx=0x7fca98007f20 comp rx=0 tx=0).stop 2024-04-17T10:03:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.611+0000 7fcaaea00640 1 -- 172.21.15.32:0/1570986704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcaa8144f20 msgr2=0x7fcaa8141020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.611+0000 7fcaaea00640 1 --2- 172.21.15.32:0/1570986704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcaa8144f20 0x7fcaa8141020 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fca9000e840 tx=0x7fca9000ed00 comp rx=0 tx=0).stop 2024-04-17T10:03:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.611+0000 7fcaaea00640 1 -- 172.21.15.32:0/1570986704 shutdown_connections 2024-04-17T10:03:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.611+0000 7fcaaea00640 1 --2- 172.21.15.32:0/1570986704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fca840683b0 0x7fca8406a870 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.611+0000 7fcaaea00640 1 --2- 172.21.15.32:0/1570986704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcaa8144f20 0x7fcaa8141020 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.611+0000 7fcaaea00640 1 --2- 172.21.15.32:0/1570986704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaa81481e0 0x7fcaa80b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.611+0000 7fcaaea00640 1 --2- 172.21.15.32:0/1570986704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa8072330 0x7fcaa813f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.611+0000 7fcaaea00640 1 -- 172.21.15.32:0/1570986704 >> 172.21.15.32:0/1570986704 conn(0x7fcaa806ba30 msgr2=0x7fcaa80b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.611+0000 7fcaaea00640 1 -- 172.21.15.32:0/1570986704 shutdown_connections 2024-04-17T10:03:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.611+0000 7fcaaea00640 1 -- 172.21.15.32:0/1570986704 wait complete. 2024-04-17T10:03:34.627 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:03:34.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.771+0000 7fd95dc00640 1 Processor -- start 2024-04-17T10:03:34.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.771+0000 7fd95dc00640 1 -- start start 2024-04-17T10:03:34.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.771+0000 7fd95dc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd958144e90 0x7fd9580b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:34.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.771+0000 7fd95dc00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd95815c1b0 0x7fd95806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:34.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.771+0000 7fd95dc00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd958072bb0 con 0x7fd9580b4de0 2024-04-17T10:03:34.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.771+0000 7fd95dc00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd958145270 con 0x7fd95815c1b0 2024-04-17T10:03:34.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.771+0000 7fd95dc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd9580b3dc0 con 0x7fd958144e90 2024-04-17T10:03:34.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.771+0000 7fd957400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd95815c1b0 0x7fd95806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:34.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.771+0000 7fd94e000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd9580b4de0 0x7fd95815baa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57552/0 (socket says 172.21.15.32:57552) 2024-04-17T10:03:34.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.771+0000 7fd94e000640 1 -- 172.21.15.32:0/3136856121 learned_addr learned my addr 172.21.15.32:0/3136856121 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:34.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.771+0000 7fd956a00640 1 --2- 172.21.15.32:0/3136856121 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd958144e90 0x7fd9580b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:34.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.771+0000 7fd957400640 1 -- 172.21.15.32:0/3136856121 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd958144e90 msgr2=0x7fd9580b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:34.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.771+0000 7fd957400640 1 --2- 172.21.15.32:0/3136856121 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd958144e90 0x7fd9580b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:34.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.771+0000 7fd957400640 1 -- 172.21.15.32:0/3136856121 >> v1:172.21.15.32:6789/0 conn(0x7fd9580b4de0 legacy=0x7fd95815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:34.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.771+0000 7fd957400640 1 -- 172.21.15.32:0/3136856121 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9580b40a0 con 0x7fd95815c1b0 2024-04-17T10:03:34.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.775+0000 7fd957400640 1 --2- 172.21.15.32:0/3136856121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd95815c1b0 0x7fd95806ef60 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fd950009d90 tx=0x7fd95001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93fa264dc7eb5fe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:34.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.775+0000 7fd956000640 1 -- 172.21.15.32:0/3136856121 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9500b2070 con 0x7fd95815c1b0 2024-04-17T10:03:34.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.775+0000 7fd956000640 1 -- 172.21.15.32:0/3136856121 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9500a5a20 con 0x7fd95815c1b0 2024-04-17T10:03:34.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.775+0000 7fd956000640 1 -- 172.21.15.32:0/3136856121 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9500a5df0 con 0x7fd95815c1b0 2024-04-17T10:03:34.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.775+0000 7fd95dc00640 1 -- 172.21.15.32:0/3136856121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd95815c1b0 msgr2=0x7fd95806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:34.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.775+0000 7fd95dc00640 1 --2- 172.21.15.32:0/3136856121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd95815c1b0 0x7fd95806ef60 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fd950009d90 tx=0x7fd95001c940 comp rx=0 tx=0).stop 2024-04-17T10:03:34.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.775+0000 7fd95dc00640 1 -- 172.21.15.32:0/3136856121 shutdown_connections 2024-04-17T10:03:34.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.775+0000 7fd95dc00640 1 --2- 172.21.15.32:0/3136856121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd95815c1b0 0x7fd95806ef60 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:34.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.775+0000 7fd95dc00640 1 --2- 172.21.15.32:0/3136856121 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd958144e90 0x7fd9580b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:34.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.775+0000 7fd95dc00640 1 -- 172.21.15.32:0/3136856121 >> 172.21.15.32:0/3136856121 conn(0x7fd95806ba30 msgr2=0x7fd95806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:34.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.775+0000 7fd95dc00640 1 -- 172.21.15.32:0/3136856121 shutdown_connections 2024-04-17T10:03:34.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.775+0000 7fd95dc00640 1 -- 172.21.15.32:0/3136856121 wait complete. 2024-04-17T10:03:34.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd95dc00640 1 Processor -- start 2024-04-17T10:03:34.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd95dc00640 1 -- start start 2024-04-17T10:03:34.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd95dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9580b4de0 0x7fd9580ae400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:34.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd95dc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd958144e90 0x7fd95815da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:34.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd95dc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd95815c1b0 0x7fd95815a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:34.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd95dc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd95806f8b0 con 0x7fd9580b4de0 2024-04-17T10:03:34.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd95dc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd958072880 con 0x7fd95815c1b0 2024-04-17T10:03:34.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd95dc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd958168110 con 0x7fd958144e90 2024-04-17T10:03:34.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd956a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9580b4de0 0x7fd9580ae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:34.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd956a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9580b4de0 0x7fd9580ae400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58724/0 (socket says 172.21.15.32:58724) 2024-04-17T10:03:34.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd956a00640 1 -- 172.21.15.32:0/4172183977 learned_addr learned my addr 172.21.15.32:0/4172183977 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:34.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd957400640 1 --2- 172.21.15.32:0/4172183977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd95815c1b0 0x7fd95815a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:34.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd94e000640 1 --2- 172.21.15.32:0/4172183977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd958144e90 0x7fd95815da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:34.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd956a00640 1 -- 172.21.15.32:0/4172183977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd958144e90 msgr2=0x7fd95815da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:34.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd956a00640 1 --2- 172.21.15.32:0/4172183977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd958144e90 0x7fd95815da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:34.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd956a00640 1 -- 172.21.15.32:0/4172183977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd95815c1b0 msgr2=0x7fd95815a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:34.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd956a00640 1 --2- 172.21.15.32:0/4172183977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd95815c1b0 0x7fd95815a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:34.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd956a00640 1 -- 172.21.15.32:0/4172183977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9581f7790 con 0x7fd9580b4de0 2024-04-17T10:03:34.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd957400640 1 --2- 172.21.15.32:0/4172183977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd95815c1b0 0x7fd95815a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:03:34.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd94e000640 1 --2- 172.21.15.32:0/4172183977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd958144e90 0x7fd95815da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:03:34.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd956a00640 1 --2- 172.21.15.32:0/4172183977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9580b4de0 0x7fd9580ae400 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fd94800d830 tx=0x7fd94800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:34.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd94f400640 1 -- 172.21.15.32:0/4172183977 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94800b6c0 con 0x7fd9580b4de0 2024-04-17T10:03:34.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd94f400640 1 -- 172.21.15.32:0/4172183977 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd948019070 con 0x7fd9580b4de0 2024-04-17T10:03:34.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd94f400640 1 -- 172.21.15.32:0/4172183977 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94800ba60 con 0x7fd9580b4de0 2024-04-17T10:03:34.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd95dc00640 1 -- 172.21.15.32:0/4172183977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd95815af10 con 0x7fd9580b4de0 2024-04-17T10:03:34.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.779+0000 7fd95dc00640 1 -- 172.21.15.32:0/4172183977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9580aed70 con 0x7fd9580b4de0 2024-04-17T10:03:34.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.783+0000 7fd95dc00640 1 -- 172.21.15.32:0/4172183977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd958068a90 con 0x7fd9580b4de0 2024-04-17T10:03:34.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.783+0000 7fd94f400640 1 -- 172.21.15.32:0/4172183977 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd94801e460 con 0x7fd9580b4de0 2024-04-17T10:03:34.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.783+0000 7fd94f400640 1 --2- 172.21.15.32:0/4172183977 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd934068380 0x7fd93406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:34.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.783+0000 7fd94f400640 1 -- 172.21.15.32:0/4172183977 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 4857+0+0 (secure 0 0 0) 0x7fd948087890 con 0x7fd9580b4de0 2024-04-17T10:03:34.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.783+0000 7fd94e000640 1 --2- 172.21.15.32:0/4172183977 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd934068380 0x7fd93406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:34.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.787+0000 7fd94e000640 1 --2- 172.21.15.32:0/4172183977 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd934068380 0x7fd93406a840 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd9581faa50 tx=0x7fd940009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:34.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.787+0000 7fd94f400640 1 -- 172.21.15.32:0/4172183977 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd948058180 con 0x7fd9580b4de0 2024-04-17T10:03:34.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.959+0000 7fd95dc00640 1 -- 172.21.15.32:0/4172183977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9581f7550 con 0x7fd9580b4de0 2024-04-17T10:03:34.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.959+0000 7fd94f400640 1 -- 172.21.15.32:0/4172183977 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+12905 (secure 0 0 0) 0x7fd948057a80 con 0x7fd9580b4de0 2024-04-17T10:03:34.957 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:03:34.957 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":105,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:03:34.213945+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:03:34.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.959+0000 7fd95dc00640 1 -- 172.21.15.32:0/4172183977 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd934068380 msgr2=0x7fd93406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:34.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.959+0000 7fd95dc00640 1 --2- 172.21.15.32:0/4172183977 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd934068380 0x7fd93406a840 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd9581faa50 tx=0x7fd940009040 comp rx=0 tx=0).stop 2024-04-17T10:03:34.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.959+0000 7fd95dc00640 1 -- 172.21.15.32:0/4172183977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9580b4de0 msgr2=0x7fd9580ae400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:34.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.959+0000 7fd95dc00640 1 --2- 172.21.15.32:0/4172183977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9580b4de0 0x7fd9580ae400 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fd94800d830 tx=0x7fd94800dcf0 comp rx=0 tx=0).stop 2024-04-17T10:03:34.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.959+0000 7fd95dc00640 1 -- 172.21.15.32:0/4172183977 shutdown_connections 2024-04-17T10:03:34.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.959+0000 7fd95dc00640 1 --2- 172.21.15.32:0/4172183977 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd934068380 0x7fd93406a840 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:34.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.959+0000 7fd95dc00640 1 --2- 172.21.15.32:0/4172183977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd958144e90 0x7fd95815da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:34.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.959+0000 7fd95dc00640 1 --2- 172.21.15.32:0/4172183977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd95815c1b0 0x7fd95815a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:34.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.959+0000 7fd95dc00640 1 --2- 172.21.15.32:0/4172183977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9580b4de0 0x7fd9580ae400 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:34.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.959+0000 7fd95dc00640 1 -- 172.21.15.32:0/4172183977 >> 172.21.15.32:0/4172183977 conn(0x7fd95806ba30 msgr2=0x7fd9580b5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:34.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.959+0000 7fd95dc00640 1 -- 172.21.15.32:0/4172183977 shutdown_connections 2024-04-17T10:03:34.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:34.959+0000 7fd95dc00640 1 -- 172.21.15.32:0/4172183977 wait complete. 2024-04-17T10:03:34.979 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:03:34.979 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:03:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.123+0000 7f3ce3400640 1 Processor -- start 2024-04-17T10:03:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.123+0000 7f3ce3400640 1 -- start start 2024-04-17T10:03:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.123+0000 7f3ce3400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3cdc06f7c0 0x7f3cdc06ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.123+0000 7f3ce3400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3cdc06f270 0x7f3cdc06bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.123+0000 7f3ce3400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cdc0cc940 con 0x7f3cdc15b7d0 2024-04-17T10:03:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.123+0000 7f3ce3400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3cdc0ccc20 con 0x7f3cdc06f7c0 2024-04-17T10:03:35.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.123+0000 7f3ce3400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f3cdc06e0b0 con 0x7f3cdc06f270 2024-04-17T10:03:35.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.127+0000 7f3cd3e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3cdc06f7c0 0x7f3cdc06ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:35.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.127+0000 7f3ce0a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3cdc15b7d0 0x7f3cdc06e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57566/0 (socket says 172.21.15.32:57566) 2024-04-17T10:03:35.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.127+0000 7f3ce0a00640 1 -- 172.21.15.32:0/26084201 learned_addr learned my addr 172.21.15.32:0/26084201 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:35.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.127+0000 7f3ce1400640 1 --2- 172.21.15.32:0/26084201 >> v2:172.21.15.144:3300/0 conn(0x7f3cdc06f270 0x7f3cdc06bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:35.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.127+0000 7f3cd3e00640 1 -- 172.21.15.32:0/26084201 >> v2:172.21.15.144:3300/0 conn(0x7f3cdc06f270 msgr2=0x7f3cdc06bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:35.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.127+0000 7f3cd3e00640 1 --2- 172.21.15.32:0/26084201 >> v2:172.21.15.144:3300/0 conn(0x7f3cdc06f270 0x7f3cdc06bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.127+0000 7f3cd3e00640 1 -- 172.21.15.32:0/26084201 >> v1:172.21.15.32:6789/0 conn(0x7f3cdc15b7d0 legacy=0x7f3cdc06e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:35.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.127+0000 7f3cd3e00640 1 -- 172.21.15.32:0/26084201 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cdc06e390 con 0x7f3cdc06f7c0 2024-04-17T10:03:35.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.127+0000 7f3cdbe00640 1 -- 172.21.15.32:0/26084201 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2351041508 0 0) 0x7f3cdc0cc940 con 0x7f3cdc15b7d0 2024-04-17T10:03:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.127+0000 7f3cd3e00640 1 --2- 172.21.15.32:0/26084201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cdc06f7c0 0x7f3cdc06ba30 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0b20b0 tx=0x7f3cc401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b729c355511483e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.127+0000 7f3cdbe00640 1 -- 172.21.15.32:0/26084201 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc40ab610 con 0x7f3cdc06f7c0 2024-04-17T10:03:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.127+0000 7f3cdbe00640 1 -- 172.21.15.32:0/26084201 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc40b0070 con 0x7f3cdc06f7c0 2024-04-17T10:03:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.127+0000 7f3cdbe00640 1 -- 172.21.15.32:0/26084201 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc40ab9d0 con 0x7f3cdc06f7c0 2024-04-17T10:03:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce3400640 1 -- 172.21.15.32:0/26084201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cdc06f7c0 msgr2=0x7f3cdc06ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce3400640 1 --2- 172.21.15.32:0/26084201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cdc06f7c0 0x7f3cdc06ba30 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0b20b0 tx=0x7f3cc401c3f0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce3400640 1 -- 172.21.15.32:0/26084201 shutdown_connections 2024-04-17T10:03:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce3400640 1 --2- 172.21.15.32:0/26084201 >> v2:172.21.15.144:3300/0 conn(0x7f3cdc06f270 0x7f3cdc06bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce3400640 1 --2- 172.21.15.32:0/26084201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cdc06f7c0 0x7f3cdc06ba30 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce3400640 1 -- 172.21.15.32:0/26084201 >> 172.21.15.32:0/26084201 conn(0x7f3cdc0b2120 msgr2=0x7f3cdc0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce3400640 1 -- 172.21.15.32:0/26084201 shutdown_connections 2024-04-17T10:03:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce3400640 1 -- 172.21.15.32:0/26084201 wait complete. 2024-04-17T10:03:35.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce3400640 1 Processor -- start 2024-04-17T10:03:35.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce3400640 1 -- start start 2024-04-17T10:03:35.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cdc06f270 0x7f3cdc15e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:35.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce3400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cdc06f7c0 0x7f3cdc151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:35.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce3400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cdc15b7d0 0x7f3cdc151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:35.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce3400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc0cee30 con 0x7f3cdc06f270 2024-04-17T10:03:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce3400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc0cecb0 con 0x7f3cdc15b7d0 2024-04-17T10:03:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce3400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3cdc1682a0 con 0x7f3cdc06f7c0 2024-04-17T10:03:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce0a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cdc06f270 0x7f3cdc15e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce0a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cdc06f270 0x7f3cdc15e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58742/0 (socket says 172.21.15.32:58742) 2024-04-17T10:03:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce0a00640 1 -- 172.21.15.32:0/2888794342 learned_addr learned my addr 172.21.15.32:0/2888794342 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce1400640 1 --2- 172.21.15.32:0/2888794342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cdc15b7d0 0x7f3cdc151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3cd3e00640 1 --2- 172.21.15.32:0/2888794342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cdc06f7c0 0x7f3cdc151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce0a00640 1 -- 172.21.15.32:0/2888794342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cdc06f7c0 msgr2=0x7f3cdc151580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce0a00640 1 --2- 172.21.15.32:0/2888794342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cdc06f7c0 0x7f3cdc151580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce0a00640 1 -- 172.21.15.32:0/2888794342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cdc15b7d0 msgr2=0x7f3cdc151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce0a00640 1 --2- 172.21.15.32:0/2888794342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cdc15b7d0 0x7f3cdc151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce0a00640 1 -- 172.21.15.32:0/2888794342 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cdc15a9e0 con 0x7f3cdc06f270 2024-04-17T10:03:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.131+0000 7f3ce0a00640 1 --2- 172.21.15.32:0/2888794342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cdc06f270 0x7f3cdc15e4e0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00c4b0 tx=0x7f3ccc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:35.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.135+0000 7f3cd9600640 1 -- 172.21.15.32:0/2888794342 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc018070 con 0x7f3cdc06f270 2024-04-17T10:03:35.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.135+0000 7f3cd9600640 1 -- 172.21.15.32:0/2888794342 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ccc007500 con 0x7f3cdc06f270 2024-04-17T10:03:35.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.135+0000 7f3cd9600640 1 -- 172.21.15.32:0/2888794342 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc004a30 con 0x7f3cdc06f270 2024-04-17T10:03:35.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.135+0000 7f3ce3400640 1 -- 172.21.15.32:0/2888794342 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cdc0aeed0 con 0x7f3cdc06f270 2024-04-17T10:03:35.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.135+0000 7f3ce3400640 1 -- 172.21.15.32:0/2888794342 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cdc0af490 con 0x7f3cdc06f270 2024-04-17T10:03:35.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.135+0000 7f3ce3400640 1 -- 172.21.15.32:0/2888794342 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cdc068a90 con 0x7f3cdc06f270 2024-04-17T10:03:35.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.135+0000 7f3cd9600640 1 -- 172.21.15.32:0/2888794342 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3ccc01b070 con 0x7f3cdc06f270 2024-04-17T10:03:35.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.135+0000 7f3cd9600640 1 --2- 172.21.15.32:0/2888794342 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3cb8068380 0x7f3cb806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:35.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.135+0000 7f3cd9600640 1 -- 172.21.15.32:0/2888794342 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 4857+0+0 (secure 0 0 0) 0x7f3ccc085870 con 0x7f3cdc06f270 2024-04-17T10:03:35.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.135+0000 7f3cd3e00640 1 --2- 172.21.15.32:0/2888794342 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3cb8068380 0x7f3cb806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:35.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.139+0000 7f3cd3e00640 1 --2- 172.21.15.32:0/2888794342 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3cb8068380 0x7f3cb806a840 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3cc40af000 tx=0x7f3cc40a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:35.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.139+0000 7f3cd9600640 1 -- 172.21.15.32:0/2888794342 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3ccc056210 con 0x7f3cdc06f270 2024-04-17T10:03:35.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.307+0000 7f3ce3400640 1 -- 172.21.15.32:0/2888794342 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3cdc06ce40 con 0x7f3cdc06f270 2024-04-17T10:03:35.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.307+0000 7f3cd9600640 1 -- 172.21.15.32:0/2888794342 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7f3ccc055b10 con 0x7f3cdc06f270 2024-04-17T10:03:35.307 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T10:03:35.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.311+0000 7f3ce3400640 1 -- 172.21.15.32:0/2888794342 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3cb8068380 msgr2=0x7f3cb806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:35.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.311+0000 7f3ce3400640 1 --2- 172.21.15.32:0/2888794342 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3cb8068380 0x7f3cb806a840 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3cc40af000 tx=0x7f3cc40a7000 comp rx=0 tx=0).stop 2024-04-17T10:03:35.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.311+0000 7f3ce3400640 1 -- 172.21.15.32:0/2888794342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cdc06f270 msgr2=0x7f3cdc15e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.311+0000 7f3ce3400640 1 --2- 172.21.15.32:0/2888794342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cdc06f270 0x7f3cdc15e4e0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00c4b0 tx=0x7f3ccc00c970 comp rx=0 tx=0).stop 2024-04-17T10:03:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.311+0000 7f3ce3400640 1 -- 172.21.15.32:0/2888794342 shutdown_connections 2024-04-17T10:03:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.311+0000 7f3ce3400640 1 --2- 172.21.15.32:0/2888794342 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3cb8068380 0x7f3cb806a840 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.311+0000 7f3ce3400640 1 --2- 172.21.15.32:0/2888794342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cdc06f7c0 0x7f3cdc151580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.311+0000 7f3ce3400640 1 --2- 172.21.15.32:0/2888794342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cdc15b7d0 0x7f3cdc151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.311+0000 7f3ce3400640 1 --2- 172.21.15.32:0/2888794342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cdc06f270 0x7f3cdc15e4e0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.311+0000 7f3ce3400640 1 -- 172.21.15.32:0/2888794342 >> 172.21.15.32:0/2888794342 conn(0x7f3cdc0b2120 msgr2=0x7f3cdc153250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.311+0000 7f3ce3400640 1 -- 172.21.15.32:0/2888794342 shutdown_connections 2024-04-17T10:03:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.311+0000 7f3ce3400640 1 -- 172.21.15.32:0/2888794342 wait complete. 2024-04-17T10:03:35.329 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T10:03:35.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.471+0000 7fc14e400640 1 Processor -- start 2024-04-17T10:03:35.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.471+0000 7fc14e400640 1 -- start start 2024-04-17T10:03:35.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.475+0000 7fc14e400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc148144e90 0x7fc1480b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:35.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.475+0000 7fc14e400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc14815c1b0 0x7fc14806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:35.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.475+0000 7fc14e400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc148072bb0 con 0x7fc1480b4de0 2024-04-17T10:03:35.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.475+0000 7fc14e400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc148145270 con 0x7fc14815c1b0 2024-04-17T10:03:35.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.475+0000 7fc14e400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc1480b3dc0 con 0x7fc148144e90 2024-04-17T10:03:35.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.475+0000 7fc147400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc148144e90 0x7fc1480b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:35.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.475+0000 7fc13ea00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fc1480b4de0 0x7fc14815baa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:60038/0 (socket says 172.21.15.32:60038) 2024-04-17T10:03:35.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.475+0000 7fc147e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc14815c1b0 0x7fc14806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:35.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.475+0000 7fc13ea00640 1 -- 172.21.15.32:0/1092014738 learned_addr learned my addr 172.21.15.32:0/1092014738 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:35.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.475+0000 7fc146a00640 1 -- 172.21.15.32:0/1092014738 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1889061070 0 0) 0x7fc148072bb0 con 0x7fc1480b4de0 2024-04-17T10:03:35.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.475+0000 7fc146a00640 1 -- 172.21.15.32:0/1092014738 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc12c003530 con 0x7fc1480b4de0 2024-04-17T10:03:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.475+0000 7fc147400640 1 -- 172.21.15.32:0/1092014738 >> v2:172.21.15.144:3300/0 conn(0x7fc14815c1b0 msgr2=0x7fc14806ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.475+0000 7fc147400640 1 --2- 172.21.15.32:0/1092014738 >> v2:172.21.15.144:3300/0 conn(0x7fc14815c1b0 0x7fc14806ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.475+0000 7fc147400640 1 -- 172.21.15.32:0/1092014738 >> v1:172.21.15.144:6789/0 conn(0x7fc1480b4de0 legacy=0x7fc14815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.475+0000 7fc147400640 1 -- 172.21.15.32:0/1092014738 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1480b40a0 con 0x7fc148144e90 2024-04-17T10:03:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.475+0000 7fc147e00640 1 --2- 172.21.15.32:0/1092014738 >> v2:172.21.15.144:3300/0 conn(0x7fc14815c1b0 0x7fc14806ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:03:35.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.479+0000 7fc147400640 1 --2- 172.21.15.32:0/1092014738 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc148144e90 0x7fc1480b48a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc1380097c0 tx=0x7fc13801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=541481b0cc4c6d02 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:35.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.479+0000 7fc146a00640 1 -- 172.21.15.32:0/1092014738 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1380aa5f0 con 0x7fc148144e90 2024-04-17T10:03:35.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.479+0000 7fc146a00640 1 -- 172.21.15.32:0/1092014738 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1380af070 con 0x7fc148144e90 2024-04-17T10:03:35.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.479+0000 7fc146a00640 1 -- 172.21.15.32:0/1092014738 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1380aa990 con 0x7fc148144e90 2024-04-17T10:03:35.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.479+0000 7fc14e400640 1 -- 172.21.15.32:0/1092014738 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc148144e90 msgr2=0x7fc1480b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:35.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.479+0000 7fc14e400640 1 --2- 172.21.15.32:0/1092014738 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc148144e90 0x7fc1480b48a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc1380097c0 tx=0x7fc13801c470 comp rx=0 tx=0).stop 2024-04-17T10:03:35.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.479+0000 7fc14e400640 1 -- 172.21.15.32:0/1092014738 shutdown_connections 2024-04-17T10:03:35.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.479+0000 7fc14e400640 1 --2- 172.21.15.32:0/1092014738 >> v2:172.21.15.144:3300/0 conn(0x7fc14815c1b0 0x7fc14806ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.479+0000 7fc14e400640 1 --2- 172.21.15.32:0/1092014738 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc148144e90 0x7fc1480b48a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.479+0000 7fc14e400640 1 -- 172.21.15.32:0/1092014738 >> 172.21.15.32:0/1092014738 conn(0x7fc14806ba30 msgr2=0x7fc14806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:35.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.479+0000 7fc14e400640 1 -- 172.21.15.32:0/1092014738 shutdown_connections 2024-04-17T10:03:35.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.479+0000 7fc14e400640 1 -- 172.21.15.32:0/1092014738 wait complete. 2024-04-17T10:03:35.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.479+0000 7fc14e400640 1 Processor -- start 2024-04-17T10:03:35.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.479+0000 7fc14e400640 1 -- start start 2024-04-17T10:03:35.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc14e400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1480b4de0 0x7fc148143d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:35.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc14e400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc148144e90 0x7fc148142530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:35.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc14e400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc14815c1b0 0x7fc14813c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:35.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc14e400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc14806f8b0 con 0x7fc1480b4de0 2024-04-17T10:03:35.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc14e400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc148072880 con 0x7fc148144e90 2024-04-17T10:03:35.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc14e400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc148168110 con 0x7fc14815c1b0 2024-04-17T10:03:35.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc147400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1480b4de0 0x7fc148143d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:35.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc147400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1480b4de0 0x7fc148143d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58752/0 (socket says 172.21.15.32:58752) 2024-04-17T10:03:35.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc147400640 1 -- 172.21.15.32:0/1543809785 learned_addr learned my addr 172.21.15.32:0/1543809785 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:35.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc13ea00640 1 --2- 172.21.15.32:0/1543809785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc148144e90 0x7fc148142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:35.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc147e00640 1 --2- 172.21.15.32:0/1543809785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc14815c1b0 0x7fc14813c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc147400640 1 -- 172.21.15.32:0/1543809785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc14815c1b0 msgr2=0x7fc14813c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc147400640 1 --2- 172.21.15.32:0/1543809785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc14815c1b0 0x7fc14813c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc147400640 1 -- 172.21.15.32:0/1543809785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc148144e90 msgr2=0x7fc148142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc147400640 1 --2- 172.21.15.32:0/1543809785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc148144e90 0x7fc148142530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc147400640 1 -- 172.21.15.32:0/1543809785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1480cdc80 con 0x7fc1480b4de0 2024-04-17T10:03:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc147400640 1 --2- 172.21.15.32:0/1543809785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1480b4de0 0x7fc148143d60 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fc13801ab20 tx=0x7fc13801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc13fe00640 1 -- 172.21.15.32:0/1543809785 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138006e20 con 0x7fc1480b4de0 2024-04-17T10:03:35.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc13fe00640 1 -- 172.21.15.32:0/1543809785 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1380be610 con 0x7fc1480b4de0 2024-04-17T10:03:35.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc13fe00640 1 -- 172.21.15.32:0/1543809785 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1380be9e0 con 0x7fc1480b4de0 2024-04-17T10:03:35.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc14e400640 1 -- 172.21.15.32:0/1543809785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc148142cc0 con 0x7fc1480b4de0 2024-04-17T10:03:35.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc14e400640 1 -- 172.21.15.32:0/1543809785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc148143280 con 0x7fc1480b4de0 2024-04-17T10:03:35.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc14e400640 1 -- 172.21.15.32:0/1543809785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc148068a90 con 0x7fc1480b4de0 2024-04-17T10:03:35.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.483+0000 7fc13fe00640 1 -- 172.21.15.32:0/1543809785 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc1380bc020 con 0x7fc1480b4de0 2024-04-17T10:03:35.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.487+0000 7fc13fe00640 1 --2- 172.21.15.32:0/1543809785 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc1240683b0 0x7fc12406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:35.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.487+0000 7fc13fe00640 1 -- 172.21.15.32:0/1543809785 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 4857+0+0 (secure 0 0 0) 0x7fc138120a90 con 0x7fc1480b4de0 2024-04-17T10:03:35.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.487+0000 7fc13ea00640 1 --2- 172.21.15.32:0/1543809785 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc1240683b0 0x7fc12406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:35.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.487+0000 7fc13ea00640 1 --2- 172.21.15.32:0/1543809785 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc1240683b0 0x7fc12406a870 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc14806f470 tx=0x7fc130000bd0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:35.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.491+0000 7fc13fe00640 1 -- 172.21.15.32:0/1543809785 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc138016090 con 0x7fc1480b4de0 2024-04-17T10:03:35.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.655+0000 7fc14e400640 1 -- 172.21.15.32:0/1543809785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc1480af880 con 0x7fc1480b4de0 2024-04-17T10:03:35.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.655+0000 7fc13fe00640 1 -- 172.21.15.32:0/1543809785 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v105) ==== 84+0+65 (secure 0 0 0) 0x7fc1380f0d50 con 0x7fc1480b4de0 2024-04-17T10:03:35.655 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:03:35.655 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-04-17T10:03:35.655 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-04-17T10:03:35.655 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-04-17T10:03:35.655 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-04-17T10:03:35.655 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:35.655 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-04-17T10:03:35.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.659+0000 7fc14e400640 1 -- 172.21.15.32:0/1543809785 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc1240683b0 msgr2=0x7fc12406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:35.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.659+0000 7fc14e400640 1 --2- 172.21.15.32:0/1543809785 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc1240683b0 0x7fc12406a870 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc14806f470 tx=0x7fc130000bd0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.659+0000 7fc14e400640 1 -- 172.21.15.32:0/1543809785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1480b4de0 msgr2=0x7fc148143d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:35.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.659+0000 7fc14e400640 1 --2- 172.21.15.32:0/1543809785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1480b4de0 0x7fc148143d60 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fc13801ab20 tx=0x7fc13801acb0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.659+0000 7fc14e400640 1 -- 172.21.15.32:0/1543809785 shutdown_connections 2024-04-17T10:03:35.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.659+0000 7fc14e400640 1 --2- 172.21.15.32:0/1543809785 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc1240683b0 0x7fc12406a870 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.659+0000 7fc14e400640 1 --2- 172.21.15.32:0/1543809785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc14815c1b0 0x7fc14813c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.659+0000 7fc14e400640 1 --2- 172.21.15.32:0/1543809785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc148144e90 0x7fc148142530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.659+0000 7fc14e400640 1 --2- 172.21.15.32:0/1543809785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1480b4de0 0x7fc148143d60 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.659+0000 7fc14e400640 1 -- 172.21.15.32:0/1543809785 >> 172.21.15.32:0/1543809785 conn(0x7fc14806ba30 msgr2=0x7fc1480cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:35.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.659+0000 7fc14e400640 1 -- 172.21.15.32:0/1543809785 shutdown_connections 2024-04-17T10:03:35.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.659+0000 7fc14e400640 1 -- 172.21.15.32:0/1543809785 wait complete. 2024-04-17T10:03:35.676 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T10:03:35.677 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T10:03:35.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.823+0000 7f5bcf000640 1 Processor -- start 2024-04-17T10:03:35.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.823+0000 7f5bcf000640 1 -- start start 2024-04-17T10:03:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.823+0000 7f5bcf000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bc806f500 0x7f5bc80cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.823+0000 7f5bcf000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5bc80cd6a0 0x7f5bc815ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.823+0000 7f5bcf000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bc815f200 con 0x7f5bc80cfc70 2024-04-17T10:03:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.823+0000 7f5bcf000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f5bc815fb30 con 0x7f5bc80cd6a0 2024-04-17T10:03:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.823+0000 7f5bcf000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5bc815f4e0 con 0x7f5bc806f500 2024-04-17T10:03:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.827+0000 7f5bc7e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5bc80cfc70 0x7f5bc806edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57572/0 (socket says 172.21.15.32:57572) 2024-04-17T10:03:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.827+0000 7f5bcd000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5bc80cd6a0 0x7f5bc815ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.827+0000 7f5bc7e00640 1 -- 172.21.15.32:0/2662494215 learned_addr learned my addr 172.21.15.32:0/2662494215 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:35.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.827+0000 7f5bbf400640 1 --2- 172.21.15.32:0/2662494215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bc806f500 0x7f5bc80cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:35.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.827+0000 7f5bcd000640 1 -- 172.21.15.32:0/2662494215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bc806f500 msgr2=0x7f5bc80cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:35.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.827+0000 7f5bcd000640 1 --2- 172.21.15.32:0/2662494215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bc806f500 0x7f5bc80cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.827+0000 7f5bcd000640 1 -- 172.21.15.32:0/2662494215 >> v1:172.21.15.32:6789/0 conn(0x7f5bc80cfc70 legacy=0x7f5bc806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:35.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.827+0000 7f5bcd000640 1 -- 172.21.15.32:0/2662494215 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc815f7c0 con 0x7f5bc80cd6a0 2024-04-17T10:03:35.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.827+0000 7f5bbf400640 1 --2- 172.21.15.32:0/2662494215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bc806f500 0x7f5bc80cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:35.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.827+0000 7f5bc7400640 1 -- 172.21.15.32:0/2662494215 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 465005566 0 0) 0x7f5bc815f200 con 0x7f5bc80cfc70 2024-04-17T10:03:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.827+0000 7f5bcd000640 1 --2- 172.21.15.32:0/2662494215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bc80cd6a0 0x7f5bc815ba70 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009d90 tx=0x7f5bc001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72880ef8bb392fa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.827+0000 7f5bc7400640 1 -- 172.21.15.32:0/2662494215 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00b2070 con 0x7f5bc80cd6a0 2024-04-17T10:03:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.827+0000 7f5bc7400640 1 -- 172.21.15.32:0/2662494215 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bc00a5830 con 0x7f5bc80cd6a0 2024-04-17T10:03:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.827+0000 7f5bc7400640 1 -- 172.21.15.32:0/2662494215 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00a5c00 con 0x7f5bc80cd6a0 2024-04-17T10:03:35.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bcf000640 1 -- 172.21.15.32:0/2662494215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bc80cd6a0 msgr2=0x7f5bc815ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:35.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bcf000640 1 --2- 172.21.15.32:0/2662494215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bc80cd6a0 0x7f5bc815ba70 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009d90 tx=0x7f5bc001c970 comp rx=0 tx=0).stop 2024-04-17T10:03:35.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bcf000640 1 -- 172.21.15.32:0/2662494215 shutdown_connections 2024-04-17T10:03:35.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bcf000640 1 --2- 172.21.15.32:0/2662494215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bc80cd6a0 0x7f5bc815ba70 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bcf000640 1 --2- 172.21.15.32:0/2662494215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bc806f500 0x7f5bc80cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bcf000640 1 -- 172.21.15.32:0/2662494215 >> 172.21.15.32:0/2662494215 conn(0x7f5bc80b2190 msgr2=0x7f5bc80b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:35.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bcf000640 1 -- 172.21.15.32:0/2662494215 shutdown_connections 2024-04-17T10:03:35.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bcf000640 1 -- 172.21.15.32:0/2662494215 wait complete. 2024-04-17T10:03:35.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bcf000640 1 Processor -- start 2024-04-17T10:03:35.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bcf000640 1 -- start start 2024-04-17T10:03:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bcf000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bc806f500 0x7f5bc80a96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bcf000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bc80cd6a0 0x7f5bc8164580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bcf000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bc80cfc70 0x7f5bc8148840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bcf000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81681b0 con 0x7f5bc80cd6a0 2024-04-17T10:03:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bcf000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc80d0260 con 0x7f5bc80cfc70 2024-04-17T10:03:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bcf000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5bc8168330 con 0x7f5bc806f500 2024-04-17T10:03:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bbf400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bc80cd6a0 0x7f5bc8164580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bbf400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bc80cd6a0 0x7f5bc8164580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58772/0 (socket says 172.21.15.32:58772) 2024-04-17T10:03:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bbf400640 1 -- 172.21.15.32:0/1313044194 learned_addr learned my addr 172.21.15.32:0/1313044194 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bc7e00640 1 --2- 172.21.15.32:0/1313044194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bc806f500 0x7f5bc80a96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bcd000640 1 --2- 172.21.15.32:0/1313044194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bc80cfc70 0x7f5bc8148840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bbf400640 1 -- 172.21.15.32:0/1313044194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bc806f500 msgr2=0x7f5bc80a96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bbf400640 1 --2- 172.21.15.32:0/1313044194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bc806f500 0x7f5bc80a96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bbf400640 1 -- 172.21.15.32:0/1313044194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bc80cfc70 msgr2=0x7f5bc8148840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bbf400640 1 --2- 172.21.15.32:0/1313044194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bc80cfc70 0x7f5bc8148840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bbf400640 1 -- 172.21.15.32:0/1313044194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc8165040 con 0x7f5bc80cd6a0 2024-04-17T10:03:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.831+0000 7f5bc7e00640 1 --2- 172.21.15.32:0/1313044194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bc806f500 0x7f5bc80a96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.835+0000 7f5bbf400640 1 --2- 172.21.15.32:0/1313044194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bc80cd6a0 0x7f5bc8164580 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000e730 tx=0x7f5bb000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.835+0000 7f5bc4c00640 1 -- 172.21.15.32:0/1313044194 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb000f040 con 0x7f5bc80cd6a0 2024-04-17T10:03:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.835+0000 7f5bc4c00640 1 -- 172.21.15.32:0/1313044194 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb0005ba0 con 0x7f5bc80cd6a0 2024-04-17T10:03:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.835+0000 7f5bcf000640 1 -- 172.21.15.32:0/1313044194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc8165320 con 0x7f5bc80cd6a0 2024-04-17T10:03:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.835+0000 7f5bc4c00640 1 -- 172.21.15.32:0/1313044194 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb000ee40 con 0x7f5bc80cd6a0 2024-04-17T10:03:35.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.835+0000 7f5bcf000640 1 -- 172.21.15.32:0/1313044194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc81658e0 con 0x7f5bc80cd6a0 2024-04-17T10:03:35.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.835+0000 7f5bcf000640 1 -- 172.21.15.32:0/1313044194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bc8068a90 con 0x7f5bc80cd6a0 2024-04-17T10:03:35.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.835+0000 7f5bc4c00640 1 -- 172.21.15.32:0/1313044194 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5bb0004030 con 0x7f5bc80cd6a0 2024-04-17T10:03:35.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.835+0000 7f5bc4c00640 1 --2- 172.21.15.32:0/1313044194 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ba4068380 0x7f5ba406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:35.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.835+0000 7f5bc4c00640 1 -- 172.21.15.32:0/1313044194 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 4857+0+0 (secure 0 0 0) 0x7f5bb00867c0 con 0x7f5bc80cd6a0 2024-04-17T10:03:35.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.835+0000 7f5bc7e00640 1 --2- 172.21.15.32:0/1313044194 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ba4068380 0x7f5ba406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:35.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.839+0000 7f5bc7e00640 1 --2- 172.21.15.32:0/1313044194 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ba4068380 0x7f5ba406a840 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8000f70 tx=0x7f5bb8007cb0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:35.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:35.839+0000 7f5bc4c00640 1 -- 172.21.15.32:0/1313044194 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5bb0020020 con 0x7f5bc80cd6a0 2024-04-17T10:03:36.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.007+0000 7f5bcf000640 1 -- 172.21.15.32:0/1313044194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5bc80706e0 con 0x7f5bc80cd6a0 2024-04-17T10:03:36.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.207+0000 7f5bc4c00640 1 -- 172.21.15.32:0/1313044194 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v106) ==== 138+0+0 (secure 0 0 0) 0x7f5bb0017030 con 0x7f5bc80cd6a0 2024-04-17T10:03:36.204 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-04-17T10:03:36.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.207+0000 7f5bcf000640 1 -- 172.21.15.32:0/1313044194 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ba4068380 msgr2=0x7f5ba406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:36.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.207+0000 7f5bcf000640 1 --2- 172.21.15.32:0/1313044194 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ba4068380 0x7f5ba406a840 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8000f70 tx=0x7f5bb8007cb0 comp rx=0 tx=0).stop 2024-04-17T10:03:36.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.207+0000 7f5bcf000640 1 -- 172.21.15.32:0/1313044194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bc80cd6a0 msgr2=0x7f5bc8164580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:36.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.207+0000 7f5bcf000640 1 --2- 172.21.15.32:0/1313044194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bc80cd6a0 0x7f5bc8164580 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000e730 tx=0x7f5bb000ebf0 comp rx=0 tx=0).stop 2024-04-17T10:03:36.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.207+0000 7f5bcf000640 1 -- 172.21.15.32:0/1313044194 shutdown_connections 2024-04-17T10:03:36.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.207+0000 7f5bcf000640 1 --2- 172.21.15.32:0/1313044194 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5ba4068380 0x7f5ba406a840 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:36.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.207+0000 7f5bcf000640 1 --2- 172.21.15.32:0/1313044194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bc80cfc70 0x7f5bc8148840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:36.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.207+0000 7f5bcf000640 1 --2- 172.21.15.32:0/1313044194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bc80cd6a0 0x7f5bc8164580 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:36.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.207+0000 7f5bcf000640 1 --2- 172.21.15.32:0/1313044194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bc806f500 0x7f5bc80a96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:36.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.207+0000 7f5bcf000640 1 -- 172.21.15.32:0/1313044194 >> 172.21.15.32:0/1313044194 conn(0x7f5bc80b2190 msgr2=0x7f5bc80b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:36.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.207+0000 7f5bcf000640 1 -- 172.21.15.32:0/1313044194 shutdown_connections 2024-04-17T10:03:36.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.207+0000 7f5bcf000640 1 -- 172.21.15.32:0/1313044194 wait complete. 2024-04-17T10:03:36.231 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-17T10:03:36.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.379+0000 7ff996a00640 1 Processor -- start 2024-04-17T10:03:36.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.379+0000 7ff996a00640 1 -- start start 2024-04-17T10:03:36.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.383+0000 7ff996a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff99013f630 0x7ff990071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:36.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.383+0000 7ff996a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff990072330 0x7ff990160320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:36.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.383+0000 7ff996a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff99015db00 con 0x7ff990160860 2024-04-17T10:03:36.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.383+0000 7ff996a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff99013fa10 con 0x7ff990072330 2024-04-17T10:03:36.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.383+0000 7ff996a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff990072770 con 0x7ff99013f630 2024-04-17T10:03:36.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.383+0000 7ff996000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff990160860 0x7ff990141d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57584/0 (socket says 172.21.15.32:57584) 2024-04-17T10:03:36.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.383+0000 7ff996000640 1 -- 172.21.15.32:0/1593578107 learned_addr learned my addr 172.21.15.32:0/1593578107 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:36.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.383+0000 7ff994c00640 1 --2- 172.21.15.32:0/1593578107 >> v2:172.21.15.144:3300/0 conn(0x7ff990072330 0x7ff990160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:36.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.383+0000 7ff995600640 1 --2- 172.21.15.32:0/1593578107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff99013f630 0x7ff990071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:36.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.383+0000 7ff98fe00640 1 -- 172.21.15.32:0/1593578107 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 767662987 0 0) 0x7ff99015db00 con 0x7ff990160860 2024-04-17T10:03:36.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.383+0000 7ff98fe00640 1 -- 172.21.15.32:0/1593578107 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff978003530 con 0x7ff990160860 2024-04-17T10:03:36.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.383+0000 7ff994c00640 1 -- 172.21.15.32:0/1593578107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff99013f630 msgr2=0x7ff990071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:36.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.383+0000 7ff994c00640 1 --2- 172.21.15.32:0/1593578107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff99013f630 0x7ff990071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:36.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.383+0000 7ff98fe00640 1 -- 172.21.15.32:0/1593578107 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2185311962 0 0) 0x7ff978003530 con 0x7ff990160860 2024-04-17T10:03:36.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.383+0000 7ff994c00640 1 -- 172.21.15.32:0/1593578107 >> v1:172.21.15.32:6789/0 conn(0x7ff990160860 legacy=0x7ff990141d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:36.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.383+0000 7ff994c00640 1 -- 172.21.15.32:0/1593578107 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9901610c0 con 0x7ff990072330 2024-04-17T10:03:36.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.383+0000 7ff995600640 1 --2- 172.21.15.32:0/1593578107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff99013f630 0x7ff990071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:03:36.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.387+0000 7ff994c00640 1 --2- 172.21.15.32:0/1593578107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff990072330 0x7ff990160320 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0097c0 tx=0x7ff97c01c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=120a4ddffcd476e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:36.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.387+0000 7ff98fe00640 1 -- 172.21.15.32:0/1593578107 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c0b0070 con 0x7ff990072330 2024-04-17T10:03:36.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.387+0000 7ff98fe00640 1 -- 172.21.15.32:0/1593578107 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff97c0ab400 con 0x7ff990072330 2024-04-17T10:03:36.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.387+0000 7ff98fe00640 1 -- 172.21.15.32:0/1593578107 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988001a00 con 0x7ff990072330 2024-04-17T10:03:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.387+0000 7ff996a00640 1 -- 172.21.15.32:0/1593578107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff990072330 msgr2=0x7ff990160320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.387+0000 7ff996a00640 1 --2- 172.21.15.32:0/1593578107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff990072330 0x7ff990160320 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0097c0 tx=0x7ff97c01c390 comp rx=0 tx=0).stop 2024-04-17T10:03:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.387+0000 7ff996a00640 1 -- 172.21.15.32:0/1593578107 shutdown_connections 2024-04-17T10:03:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.387+0000 7ff996a00640 1 --2- 172.21.15.32:0/1593578107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff990072330 0x7ff990160320 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.387+0000 7ff996a00640 1 --2- 172.21.15.32:0/1593578107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff99013f630 0x7ff990071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.387+0000 7ff996a00640 1 -- 172.21.15.32:0/1593578107 >> 172.21.15.32:0/1593578107 conn(0x7ff990067d70 msgr2=0x7ff99006ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.387+0000 7ff996a00640 1 -- 172.21.15.32:0/1593578107 shutdown_connections 2024-04-17T10:03:36.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.387+0000 7ff996a00640 1 -- 172.21.15.32:0/1593578107 wait complete. 2024-04-17T10:03:36.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.387+0000 7ff996a00640 1 Processor -- start 2024-04-17T10:03:36.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.387+0000 7ff996a00640 1 -- start start 2024-04-17T10:03:36.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.391+0000 7ff996a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff990072330 0x7ff9901f8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:36.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.391+0000 7ff996a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff99013f630 0x7ff9901562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:36.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.391+0000 7ff996a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff990160860 0x7ff990156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:36.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.391+0000 7ff996a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff990160cf0 con 0x7ff990160860 2024-04-17T10:03:36.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.391+0000 7ff996a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff99015d5e0 con 0x7ff99013f630 2024-04-17T10:03:36.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.391+0000 7ff996a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff990160e70 con 0x7ff990072330 2024-04-17T10:03:36.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.391+0000 7ff996000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff990160860 0x7ff990156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:36.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.391+0000 7ff996000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff990160860 0x7ff990156810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58776/0 (socket says 172.21.15.32:58776) 2024-04-17T10:03:36.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.391+0000 7ff996000640 1 -- 172.21.15.32:0/147322746 learned_addr learned my addr 172.21.15.32:0/147322746 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:36.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.391+0000 7ff995600640 1 --2- 172.21.15.32:0/147322746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff990072330 0x7ff9901f8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:36.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.391+0000 7ff994c00640 1 --2- 172.21.15.32:0/147322746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff99013f630 0x7ff9901562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:36.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.391+0000 7ff996000640 1 -- 172.21.15.32:0/147322746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff990072330 msgr2=0x7ff9901f8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:36.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.391+0000 7ff996000640 1 --2- 172.21.15.32:0/147322746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff990072330 0x7ff9901f8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:36.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.391+0000 7ff996000640 1 -- 172.21.15.32:0/147322746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff99013f630 msgr2=0x7ff9901562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:36.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.391+0000 7ff996000640 1 --2- 172.21.15.32:0/147322746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff99013f630 0x7ff9901562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.391+0000 7ff996000640 1 -- 172.21.15.32:0/147322746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9901549d0 con 0x7ff990160860 2024-04-17T10:03:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.391+0000 7ff996000640 1 --2- 172.21.15.32:0/147322746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff990160860 0x7ff990156810 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7ff98800cc90 tx=0x7ff98800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.391+0000 7ff98d600640 1 -- 172.21.15.32:0/147322746 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98800ce40 con 0x7ff990160860 2024-04-17T10:03:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.391+0000 7ff996a00640 1 -- 172.21.15.32:0/147322746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff990154cb0 con 0x7ff990160860 2024-04-17T10:03:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.391+0000 7ff98d600640 1 -- 172.21.15.32:0/147322746 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff988012400 con 0x7ff990160860 2024-04-17T10:03:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.391+0000 7ff98d600640 1 -- 172.21.15.32:0/147322746 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9880127f0 con 0x7ff990160860 2024-04-17T10:03:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.391+0000 7ff996a00640 1 -- 172.21.15.32:0/147322746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff990155270 con 0x7ff990160860 2024-04-17T10:03:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.391+0000 7ff996a00640 1 -- 172.21.15.32:0/147322746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff99004fda0 con 0x7ff990160860 2024-04-17T10:03:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.395+0000 7ff98d600640 1 -- 172.21.15.32:0/147322746 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff988012990 con 0x7ff990160860 2024-04-17T10:03:36.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.395+0000 7ff98d600640 1 --2- 172.21.15.32:0/147322746 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff970068380 0x7ff97006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:36.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.395+0000 7ff98d600640 1 -- 172.21.15.32:0/147322746 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5227+0+0 (secure 0 0 0) 0x7ff988087980 con 0x7ff990160860 2024-04-17T10:03:36.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.395+0000 7ff995600640 1 --2- 172.21.15.32:0/147322746 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff970068380 0x7ff97006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:36.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.395+0000 7ff995600640 1 --2- 172.21.15.32:0/147322746 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff970068380 0x7ff97006a840 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff984009580 tx=0x7ff984009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:36.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.399+0000 7ff98d600640 1 -- 172.21.15.32:0/147322746 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff988057ad0 con 0x7ff990160860 2024-04-17T10:03:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:36.567+0000 7ff996a00640 1 -- 172.21.15.32:0/147322746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff990153a40 con 0x7ff990160860 2024-04-17T10:03:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.207+0000 7ff98d600640 1 -- 172.21.15.32:0/147322746 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v107) ==== 170+0+0 (secure 0 0 0) 0x7ff98805bdd0 con 0x7ff990160860 2024-04-17T10:03:37.207 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-04-17T10:03:37.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.211+0000 7ff996a00640 1 -- 172.21.15.32:0/147322746 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff970068380 msgr2=0x7ff97006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:37.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.211+0000 7ff996a00640 1 --2- 172.21.15.32:0/147322746 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff970068380 0x7ff97006a840 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff984009580 tx=0x7ff984009190 comp rx=0 tx=0).stop 2024-04-17T10:03:37.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.211+0000 7ff996a00640 1 -- 172.21.15.32:0/147322746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff990160860 msgr2=0x7ff990156810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:37.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.211+0000 7ff996a00640 1 --2- 172.21.15.32:0/147322746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff990160860 0x7ff990156810 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7ff98800cc90 tx=0x7ff98800a430 comp rx=0 tx=0).stop 2024-04-17T10:03:37.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.211+0000 7ff996a00640 1 -- 172.21.15.32:0/147322746 shutdown_connections 2024-04-17T10:03:37.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.211+0000 7ff996a00640 1 --2- 172.21.15.32:0/147322746 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff970068380 0x7ff97006a840 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:37.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.211+0000 7ff996a00640 1 --2- 172.21.15.32:0/147322746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff990160860 0x7ff990156810 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:37.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.211+0000 7ff996a00640 1 --2- 172.21.15.32:0/147322746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff99013f630 0x7ff9901562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:37.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.211+0000 7ff996a00640 1 --2- 172.21.15.32:0/147322746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff990072330 0x7ff9901f8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:37.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.211+0000 7ff996a00640 1 -- 172.21.15.32:0/147322746 >> 172.21.15.32:0/147322746 conn(0x7ff990067d70 msgr2=0x7ff9900b17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:37.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.211+0000 7ff996a00640 1 -- 172.21.15.32:0/147322746 shutdown_connections 2024-04-17T10:03:37.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.211+0000 7ff996a00640 1 -- 172.21.15.32:0/147322746 wait complete. 2024-04-17T10:03:37.250 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T10:03:37.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.395+0000 7f53f5000640 1 Processor -- start 2024-04-17T10:03:37.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.395+0000 7f53f5000640 1 -- start start 2024-04-17T10:03:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.395+0000 7f53f5000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53f015b7d0 0x7f53f006e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.395+0000 7f53f5000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53f006c140 0x7f53f00ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.395+0000 7f53f5000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53f00cc940 con 0x7f53f006f7c0 2024-04-17T10:03:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.395+0000 7f53f5000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f53f00ccc20 con 0x7f53f015b7d0 2024-04-17T10:03:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.395+0000 7f53f5000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f53f006e0b0 con 0x7f53f006c140 2024-04-17T10:03:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.395+0000 7f53ede00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53f015b7d0 0x7f53f006e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.395+0000 7f53e7e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f53f006f7c0 0x7f53f006ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57600/0 (socket says 172.21.15.32:57600) 2024-04-17T10:03:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.395+0000 7f53e7e00640 1 -- 172.21.15.32:0/3512474341 learned_addr learned my addr 172.21.15.32:0/3512474341 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.395+0000 7f53ee800640 1 --2- 172.21.15.32:0/3512474341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53f006c140 0x7f53f00ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.395+0000 7f53ede00640 1 -- 172.21.15.32:0/3512474341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53f006c140 msgr2=0x7f53f00ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.395+0000 7f53ede00640 1 --2- 172.21.15.32:0/3512474341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53f006c140 0x7f53f00ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.395+0000 7f53ede00640 1 -- 172.21.15.32:0/3512474341 >> v1:172.21.15.32:6789/0 conn(0x7f53f006f7c0 legacy=0x7f53f006ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.395+0000 7f53ede00640 1 -- 172.21.15.32:0/3512474341 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f006e390 con 0x7f53f015b7d0 2024-04-17T10:03:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.399+0000 7f53ede00640 1 --2- 172.21.15.32:0/3512474341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f015b7d0 0x7f53f006e8b0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f53e00097c0 tx=0x7f53e001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129fc834a5a3116e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.399+0000 7f53ed400640 1 -- 172.21.15.32:0/3512474341 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e00ab430 con 0x7f53f015b7d0 2024-04-17T10:03:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.399+0000 7f53ed400640 1 -- 172.21.15.32:0/3512474341 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e00b0070 con 0x7f53f015b7d0 2024-04-17T10:03:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.399+0000 7f53ed400640 1 -- 172.21.15.32:0/3512474341 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e00ab7f0 con 0x7f53f015b7d0 2024-04-17T10:03:37.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.399+0000 7f53f5000640 1 -- 172.21.15.32:0/3512474341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f015b7d0 msgr2=0x7f53f006e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:37.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.399+0000 7f53f5000640 1 --2- 172.21.15.32:0/3512474341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f015b7d0 0x7f53f006e8b0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f53e00097c0 tx=0x7f53e001c410 comp rx=0 tx=0).stop 2024-04-17T10:03:37.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.399+0000 7f53f5000640 1 -- 172.21.15.32:0/3512474341 shutdown_connections 2024-04-17T10:03:37.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.399+0000 7f53f5000640 1 --2- 172.21.15.32:0/3512474341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53f006c140 0x7f53f00ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:37.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.399+0000 7f53f5000640 1 --2- 172.21.15.32:0/3512474341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f015b7d0 0x7f53f006e8b0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:37.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.399+0000 7f53f5000640 1 -- 172.21.15.32:0/3512474341 >> 172.21.15.32:0/3512474341 conn(0x7f53f00b2120 msgr2=0x7f53f00b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:37.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.399+0000 7f53f5000640 1 -- 172.21.15.32:0/3512474341 shutdown_connections 2024-04-17T10:03:37.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.399+0000 7f53f5000640 1 -- 172.21.15.32:0/3512474341 wait complete. 2024-04-17T10:03:37.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.403+0000 7f53f5000640 1 Processor -- start 2024-04-17T10:03:37.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.403+0000 7f53f5000640 1 -- start start 2024-04-17T10:03:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.403+0000 7f53f5000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f006c140 0x7f53f0141d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.403+0000 7f53f5000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53f006f7c0 0x7f53f0157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.403+0000 7f53f5000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53f015b7d0 0x7f53f0151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.403+0000 7f53f5000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f53f0168630 con 0x7f53f006c140 2024-04-17T10:03:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.403+0000 7f53f5000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f53f01684b0 con 0x7f53f006f7c0 2024-04-17T10:03:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.403+0000 7f53f5000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f53f01687b0 con 0x7f53f015b7d0 2024-04-17T10:03:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.403+0000 7f53ede00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f006c140 0x7f53f0141d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.403+0000 7f53ede00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f006c140 0x7f53f0141d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58806/0 (socket says 172.21.15.32:58806) 2024-04-17T10:03:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.403+0000 7f53ede00640 1 -- 172.21.15.32:0/3863349242 learned_addr learned my addr 172.21.15.32:0/3863349242 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.403+0000 7f53ede00640 1 -- 172.21.15.32:0/3863349242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53f015b7d0 msgr2=0x7f53f0151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.403+0000 7f53ede00640 1 --2- 172.21.15.32:0/3863349242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53f015b7d0 0x7f53f0151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.403+0000 7f53ede00640 1 -- 172.21.15.32:0/3863349242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53f006f7c0 msgr2=0x7f53f0157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.403+0000 7f53ede00640 1 --2- 172.21.15.32:0/3863349242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53f006f7c0 0x7f53f0157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.403+0000 7f53ede00640 1 -- 172.21.15.32:0/3863349242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f0142370 con 0x7f53f006c140 2024-04-17T10:03:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.403+0000 7f53ede00640 1 --2- 172.21.15.32:0/3863349242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f006c140 0x7f53f0141d90 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f53e00ad040 tx=0x7f53e00191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.403+0000 7f53e6000640 1 -- 172.21.15.32:0/3863349242 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e00bc420 con 0x7f53f006c140 2024-04-17T10:03:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.403+0000 7f53e6000640 1 -- 172.21.15.32:0/3863349242 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e00bc5e0 con 0x7f53f006c140 2024-04-17T10:03:37.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.403+0000 7f53f5000640 1 -- 172.21.15.32:0/3863349242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f0142650 con 0x7f53f006c140 2024-04-17T10:03:37.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.403+0000 7f53e6000640 1 -- 172.21.15.32:0/3863349242 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e00bc9d0 con 0x7f53f006c140 2024-04-17T10:03:37.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.403+0000 7f53f5000640 1 -- 172.21.15.32:0/3863349242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f01552f0 con 0x7f53f006c140 2024-04-17T10:03:37.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.407+0000 7f53f5000640 1 -- 172.21.15.32:0/3863349242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f0068a90 con 0x7f53f006c140 2024-04-17T10:03:37.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.407+0000 7f53e6000640 1 -- 172.21.15.32:0/3863349242 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f53e00c3060 con 0x7f53f006c140 2024-04-17T10:03:37.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.407+0000 7f53e6000640 1 --2- 172.21.15.32:0/3863349242 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f53cc068380 0x7f53cc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:37.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.407+0000 7f53e6000640 1 -- 172.21.15.32:0/3863349242 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5609+0+0 (secure 0 0 0) 0x7f53e0120e80 con 0x7f53f006c140 2024-04-17T10:03:37.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.407+0000 7f53e7e00640 1 --2- 172.21.15.32:0/3863349242 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f53cc068380 0x7f53cc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:37.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.407+0000 7f53e7e00640 1 --2- 172.21.15.32:0/3863349242 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f53cc068380 0x7f53cc06a840 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f53d8000f70 tx=0x7f53d8007a90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:37.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.411+0000 7f53e6000640 1 -- 172.21.15.32:0/3863349242 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f53e0016090 con 0x7f53f006c140 2024-04-17T10:03:37.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:37.615+0000 7f53f5000640 1 -- 172.21.15.32:0/3863349242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f53f00b0510 con 0x7f53f006c140 2024-04-17T10:03:38.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.239+0000 7f53e6000640 1 -- 172.21.15.32:0/3863349242 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v94) ==== 174+0+0 (secure 0 0 0) 0x7f53e00aa390 con 0x7f53f006c140 2024-04-17T10:03:38.239 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 16 and data pool 17 2024-04-17T10:03:38.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.251+0000 7f53f5000640 1 -- 172.21.15.32:0/3863349242 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f53cc068380 msgr2=0x7f53cc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:38.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.251+0000 7f53f5000640 1 --2- 172.21.15.32:0/3863349242 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f53cc068380 0x7f53cc06a840 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f53d8000f70 tx=0x7f53d8007a90 comp rx=0 tx=0).stop 2024-04-17T10:03:38.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.251+0000 7f53f5000640 1 -- 172.21.15.32:0/3863349242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f006c140 msgr2=0x7f53f0141d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:38.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.251+0000 7f53f5000640 1 --2- 172.21.15.32:0/3863349242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f006c140 0x7f53f0141d90 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f53e00ad040 tx=0x7f53e00191b0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.255+0000 7f53f5000640 1 -- 172.21.15.32:0/3863349242 shutdown_connections 2024-04-17T10:03:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.255+0000 7f53f5000640 1 --2- 172.21.15.32:0/3863349242 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f53cc068380 0x7f53cc06a840 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.255+0000 7f53f5000640 1 --2- 172.21.15.32:0/3863349242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53f015b7d0 0x7f53f0151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.255+0000 7f53f5000640 1 --2- 172.21.15.32:0/3863349242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53f006f7c0 0x7f53f0157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.255+0000 7f53f5000640 1 --2- 172.21.15.32:0/3863349242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f006c140 0x7f53f0141d90 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.255+0000 7f53f5000640 1 -- 172.21.15.32:0/3863349242 >> 172.21.15.32:0/3863349242 conn(0x7f53f00b2120 msgr2=0x7f53f00b0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:38.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.259+0000 7f53f5000640 1 -- 172.21.15.32:0/3863349242 shutdown_connections 2024-04-17T10:03:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.259+0000 7f53f5000640 1 -- 172.21.15.32:0/3863349242 wait complete. 2024-04-17T10:03:38.287 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:03:38.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.431+0000 7f7c1aa00640 1 Processor -- start 2024-04-17T10:03:38.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.431+0000 7f7c1aa00640 1 -- start start 2024-04-17T10:03:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.431+0000 7f7c1aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c14072500 0x7f7c140cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.431+0000 7f7c1aa00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7c1415d8d0 0x7f7c140cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.431+0000 7f7c1aa00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c140cdac0 con 0x7f7c14145f30 2024-04-17T10:03:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.431+0000 7f7c1aa00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7c1406f400 con 0x7f7c1415d8d0 2024-04-17T10:03:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.431+0000 7f7c1aa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7c14146310 con 0x7f7c14072500 2024-04-17T10:03:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.431+0000 7f7c18a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7c1415d8d0 0x7f7c140cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.431+0000 7f7c13e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7c14145f30 0x7f7c14071df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57602/0 (socket says 172.21.15.32:57602) 2024-04-17T10:03:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.431+0000 7f7c13e00640 1 -- 172.21.15.32:0/2013494053 learned_addr learned my addr 172.21.15.32:0/2013494053 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.431+0000 7f7c0be00640 1 --2- 172.21.15.32:0/2013494053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c14072500 0x7f7c140cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.431+0000 7f7c18a00640 1 -- 172.21.15.32:0/2013494053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c14072500 msgr2=0x7f7c140cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.431+0000 7f7c18a00640 1 --2- 172.21.15.32:0/2013494053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c14072500 0x7f7c140cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.431+0000 7f7c18a00640 1 -- 172.21.15.32:0/2013494053 >> v1:172.21.15.32:6789/0 conn(0x7f7c14145f30 legacy=0x7f7c14071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:38.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.431+0000 7f7c18a00640 1 -- 172.21.15.32:0/2013494053 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1415cdf0 con 0x7f7c1415d8d0 2024-04-17T10:03:38.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.431+0000 7f7c0be00640 1 --2- 172.21.15.32:0/2013494053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c14072500 0x7f7c140cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:38.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.435+0000 7f7c18a00640 1 --2- 172.21.15.32:0/2013494053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c1415d8d0 0x7f7c140cd3b0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009d90 tx=0x7f7c0c01f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5756d0382ba67a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:38.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.435+0000 7f7c13400640 1 -- 172.21.15.32:0/2013494053 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c0b4070 con 0x7f7c1415d8d0 2024-04-17T10:03:38.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.435+0000 7f7c13400640 1 -- 172.21.15.32:0/2013494053 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c0c0af600 con 0x7f7c1415d8d0 2024-04-17T10:03:38.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.435+0000 7f7c13400640 1 -- 172.21.15.32:0/2013494053 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c0af9d0 con 0x7f7c1415d8d0 2024-04-17T10:03:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.435+0000 7f7c1aa00640 1 -- 172.21.15.32:0/2013494053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c1415d8d0 msgr2=0x7f7c140cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.435+0000 7f7c1aa00640 1 --2- 172.21.15.32:0/2013494053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c1415d8d0 0x7f7c140cd3b0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009d90 tx=0x7f7c0c01f150 comp rx=0 tx=0).stop 2024-04-17T10:03:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.435+0000 7f7c1aa00640 1 -- 172.21.15.32:0/2013494053 shutdown_connections 2024-04-17T10:03:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.435+0000 7f7c1aa00640 1 --2- 172.21.15.32:0/2013494053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c1415d8d0 0x7f7c140cd3b0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.435+0000 7f7c1aa00640 1 --2- 172.21.15.32:0/2013494053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c14072500 0x7f7c140cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.435+0000 7f7c1aa00640 1 -- 172.21.15.32:0/2013494053 >> 172.21.15.32:0/2013494053 conn(0x7f7c1406ba30 msgr2=0x7f7c1406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.435+0000 7f7c1aa00640 1 -- 172.21.15.32:0/2013494053 shutdown_connections 2024-04-17T10:03:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.435+0000 7f7c1aa00640 1 -- 172.21.15.32:0/2013494053 wait complete. 2024-04-17T10:03:38.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c1aa00640 1 Processor -- start 2024-04-17T10:03:38.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c1aa00640 1 -- start start 2024-04-17T10:03:38.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c1aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c14072500 0x7f7c140c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:38.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c1aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c14145f30 0x7f7c1415c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:38.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c1aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c1415d8d0 0x7f7c14141170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:38.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c1aa00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1415cb90 con 0x7f7c14072500 2024-04-17T10:03:38.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c1aa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1415d440 con 0x7f7c14145f30 2024-04-17T10:03:38.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c1aa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7c141601d0 con 0x7f7c1415d8d0 2024-04-17T10:03:38.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c13e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c14072500 0x7f7c140c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:38.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c13e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c14072500 0x7f7c140c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58836/0 (socket says 172.21.15.32:58836) 2024-04-17T10:03:38.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c13e00640 1 -- 172.21.15.32:0/785143773 learned_addr learned my addr 172.21.15.32:0/785143773 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:38.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c0be00640 1 --2- 172.21.15.32:0/785143773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c14145f30 0x7f7c1415c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:38.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c18a00640 1 --2- 172.21.15.32:0/785143773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c1415d8d0 0x7f7c14141170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c13e00640 1 -- 172.21.15.32:0/785143773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c1415d8d0 msgr2=0x7f7c14141170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c13e00640 1 --2- 172.21.15.32:0/785143773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c1415d8d0 0x7f7c14141170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c13e00640 1 -- 172.21.15.32:0/785143773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c14145f30 msgr2=0x7f7c1415c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c13e00640 1 --2- 172.21.15.32:0/785143773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c14145f30 0x7f7c1415c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c13e00640 1 -- 172.21.15.32:0/785143773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c141416b0 con 0x7f7c14072500 2024-04-17T10:03:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c18a00640 1 --2- 172.21.15.32:0/785143773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c1415d8d0 0x7f7c14141170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:03:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c13e00640 1 --2- 172.21.15.32:0/785143773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c14072500 0x7f7c140c5b80 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400c420 tx=0x7f7c0400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c10c00640 1 -- 172.21.15.32:0/785143773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04017070 con 0x7f7c14072500 2024-04-17T10:03:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c10c00640 1 -- 172.21.15.32:0/785143773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c04005cf0 con 0x7f7c14072500 2024-04-17T10:03:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c10c00640 1 -- 172.21.15.32:0/785143773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c040072f0 con 0x7f7c14072500 2024-04-17T10:03:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c1aa00640 1 -- 172.21.15.32:0/785143773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c140c6160 con 0x7f7c14072500 2024-04-17T10:03:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.439+0000 7f7c1aa00640 1 -- 172.21.15.32:0/785143773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c141f6420 con 0x7f7c14072500 2024-04-17T10:03:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.443+0000 7f7c1aa00640 1 -- 172.21.15.32:0/785143773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c14068a90 con 0x7f7c14072500 2024-04-17T10:03:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.443+0000 7f7c10c00640 1 -- 172.21.15.32:0/785143773 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7c0401a070 con 0x7f7c14072500 2024-04-17T10:03:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.443+0000 7f7c10c00640 1 --2- 172.21.15.32:0/785143773 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7bf00683b0 0x7f7bf006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.443+0000 7f7c10c00640 1 -- 172.21.15.32:0/785143773 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5709+0+0 (secure 0 0 0) 0x7f7c04085df0 con 0x7f7c14072500 2024-04-17T10:03:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.443+0000 7f7c0be00640 1 --2- 172.21.15.32:0/785143773 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7bf00683b0 0x7f7bf006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.443+0000 7f7c0be00640 1 --2- 172.21.15.32:0/785143773 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7bf00683b0 0x7f7bf006a870 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f7c140ce0c0 tx=0x7f7bfc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:38.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.447+0000 7f7c10c00640 1 -- 172.21.15.32:0/785143773 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f7c04056310 con 0x7f7c14072500 2024-04-17T10:03:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.615+0000 7f7c1aa00640 1 -- 172.21.15.32:0/785143773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c1415e9d0 con 0x7f7c14072500 2024-04-17T10:03:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.615+0000 7f7c10c00640 1 -- 172.21.15.32:0/785143773 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+16652 (secure 0 0 0) 0x7f7c04055c10 con 0x7f7c14072500 2024-04-17T10:03:38.614 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:03:38.614 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":108,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:03:37.619826+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:03:36.021322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-17T10:03:36.581134+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:03:38.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.619+0000 7f7c1aa00640 1 -- 172.21.15.32:0/785143773 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7bf00683b0 msgr2=0x7f7bf006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:38.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.619+0000 7f7c1aa00640 1 --2- 172.21.15.32:0/785143773 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7bf00683b0 0x7f7bf006a870 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f7c140ce0c0 tx=0x7f7bfc008040 comp rx=0 tx=0).stop 2024-04-17T10:03:38.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.619+0000 7f7c1aa00640 1 -- 172.21.15.32:0/785143773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c14072500 msgr2=0x7f7c140c5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:38.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.619+0000 7f7c1aa00640 1 --2- 172.21.15.32:0/785143773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c14072500 0x7f7c140c5b80 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400c420 tx=0x7f7c0400c8e0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.619+0000 7f7c1aa00640 1 -- 172.21.15.32:0/785143773 shutdown_connections 2024-04-17T10:03:38.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.619+0000 7f7c1aa00640 1 --2- 172.21.15.32:0/785143773 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7bf00683b0 0x7f7bf006a870 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.619+0000 7f7c1aa00640 1 --2- 172.21.15.32:0/785143773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c1415d8d0 0x7f7c14141170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.619+0000 7f7c1aa00640 1 --2- 172.21.15.32:0/785143773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c14145f30 0x7f7c1415c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.619+0000 7f7c1aa00640 1 --2- 172.21.15.32:0/785143773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c14072500 0x7f7c140c5b80 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.619+0000 7f7c1aa00640 1 -- 172.21.15.32:0/785143773 >> 172.21.15.32:0/785143773 conn(0x7f7c1406ba30 msgr2=0x7f7c14143d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:38.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.619+0000 7f7c1aa00640 1 -- 172.21.15.32:0/785143773 shutdown_connections 2024-04-17T10:03:38.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.619+0000 7f7c1aa00640 1 -- 172.21.15.32:0/785143773 wait complete. 2024-04-17T10:03:38.636 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:03:38.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.775+0000 7f4106400640 1 Processor -- start 2024-04-17T10:03:38.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.775+0000 7f4106400640 1 -- start start 2024-04-17T10:03:38.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.779+0000 7f4106400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f410015baa0 0x7f410015bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:38.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.779+0000 7f4106400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f410006ef60 0x7f410006f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:38.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.779+0000 7f4106400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4100072bb0 con 0x7f4100144e90 2024-04-17T10:03:38.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.779+0000 7f4106400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f410015c440 con 0x7f410006ef60 2024-04-17T10:03:38.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.779+0000 7f4106400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f410006f8f0 con 0x7f410015baa0 2024-04-17T10:03:38.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.779+0000 7f40ffe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f410006ef60 0x7f410006f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:38.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.779+0000 7f40ff400640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f4100144e90 0x7f41000b48a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:60046/0 (socket says 172.21.15.32:60046) 2024-04-17T10:03:38.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.779+0000 7f40f7e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f410015baa0 0x7f410015bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:38.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.779+0000 7f40ff400640 1 -- 172.21.15.32:0/4174872645 learned_addr learned my addr 172.21.15.32:0/4174872645 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:38.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.779+0000 7f40ffe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f410006ef60 0x7f410006f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58848/0 (socket says 172.21.15.32:58848) 2024-04-17T10:03:38.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.779+0000 7f40ffe00640 1 -- 172.21.15.32:0/4174872645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f410015baa0 msgr2=0x7f410015bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:38.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.779+0000 7f40ffe00640 1 --2- 172.21.15.32:0/4174872645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f410015baa0 0x7f410015bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.779+0000 7f40ffe00640 1 -- 172.21.15.32:0/4174872645 >> v1:172.21.15.144:6789/0 conn(0x7f4100144e90 legacy=0x7f41000b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:38.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.779+0000 7f40ffe00640 1 -- 172.21.15.32:0/4174872645 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41000b3dc0 con 0x7f410006ef60 2024-04-17T10:03:38.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.783+0000 7f40ffe00640 1 --2- 172.21.15.32:0/4174872645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f410006ef60 0x7f410006f3b0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f40f8009d90 tx=0x7f40f801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fcdff8ebb77efc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:38.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.783+0000 7f40fea00640 1 -- 172.21.15.32:0/4174872645 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f80b2070 con 0x7f410006ef60 2024-04-17T10:03:38.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.783+0000 7f40fea00640 1 -- 172.21.15.32:0/4174872645 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f80a5920 con 0x7f410006ef60 2024-04-17T10:03:38.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.783+0000 7f40fea00640 1 -- 172.21.15.32:0/4174872645 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f80a5cf0 con 0x7f410006ef60 2024-04-17T10:03:38.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.783+0000 7f4106400640 1 -- 172.21.15.32:0/4174872645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f410006ef60 msgr2=0x7f410006f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:38.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.783+0000 7f4106400640 1 --2- 172.21.15.32:0/4174872645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f410006ef60 0x7f410006f3b0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f40f8009d90 tx=0x7f40f801c970 comp rx=0 tx=0).stop 2024-04-17T10:03:38.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.783+0000 7f4106400640 1 -- 172.21.15.32:0/4174872645 shutdown_connections 2024-04-17T10:03:38.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.783+0000 7f4106400640 1 --2- 172.21.15.32:0/4174872645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f410006ef60 0x7f410006f3b0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.783+0000 7f4106400640 1 --2- 172.21.15.32:0/4174872645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f410015baa0 0x7f410015bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.783+0000 7f4106400640 1 -- 172.21.15.32:0/4174872645 >> 172.21.15.32:0/4174872645 conn(0x7f410006ba30 msgr2=0x7f410006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:38.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.783+0000 7f4106400640 1 -- 172.21.15.32:0/4174872645 shutdown_connections 2024-04-17T10:03:38.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.783+0000 7f4106400640 1 -- 172.21.15.32:0/4174872645 wait complete. 2024-04-17T10:03:38.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.783+0000 7f4106400640 1 Processor -- start 2024-04-17T10:03:38.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.783+0000 7f4106400640 1 -- start start 2024-04-17T10:03:38.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f4106400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f410006ef60 0x7f410013fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:38.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f4106400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4100144e90 0x7f410015d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:38.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f4106400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f410015baa0 0x7f410015a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:38.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f4106400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f41000727f0 con 0x7f4100144e90 2024-04-17T10:03:38.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f4106400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4100072670 con 0x7f410015baa0 2024-04-17T10:03:38.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f4106400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4100072970 con 0x7f410006ef60 2024-04-17T10:03:38.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f40f7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4100144e90 0x7f410015d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:38.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f40f7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4100144e90 0x7f410015d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58860/0 (socket says 172.21.15.32:58860) 2024-04-17T10:03:38.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f40f7e00640 1 -- 172.21.15.32:0/496238427 learned_addr learned my addr 172.21.15.32:0/496238427 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:38.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f40ff400640 1 --2- 172.21.15.32:0/496238427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f410006ef60 0x7f410013fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:38.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f40ffe00640 1 --2- 172.21.15.32:0/496238427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f410015baa0 0x7f410015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:38.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f40f7e00640 1 -- 172.21.15.32:0/496238427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f410006ef60 msgr2=0x7f410013fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:38.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f40f7e00640 1 --2- 172.21.15.32:0/496238427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f410006ef60 0x7f410013fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f40f7e00640 1 -- 172.21.15.32:0/496238427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f410015baa0 msgr2=0x7f410015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:38.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f40f7e00640 1 --2- 172.21.15.32:0/496238427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f410015baa0 0x7f410015a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f40f7e00640 1 -- 172.21.15.32:0/496238427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4100142e80 con 0x7f4100144e90 2024-04-17T10:03:38.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f40f7e00640 1 --2- 172.21.15.32:0/496238427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4100144e90 0x7f410015d960 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f40e800e870 tx=0x7f40e800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:38.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f40f7400640 1 -- 172.21.15.32:0/496238427 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8017070 con 0x7f4100144e90 2024-04-17T10:03:38.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f40f7400640 1 -- 172.21.15.32:0/496238427 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e800c940 con 0x7f4100144e90 2024-04-17T10:03:38.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f40f7400640 1 -- 172.21.15.32:0/496238427 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e800ccf0 con 0x7f4100144e90 2024-04-17T10:03:38.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f4106400640 1 -- 172.21.15.32:0/496238427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f410015a5d0 con 0x7f4100144e90 2024-04-17T10:03:38.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f4106400640 1 -- 172.21.15.32:0/496238427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41001fad90 con 0x7f4100144e90 2024-04-17T10:03:38.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f4106400640 1 -- 172.21.15.32:0/496238427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4100068a90 con 0x7f4100144e90 2024-04-17T10:03:38.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.787+0000 7f40f7400640 1 -- 172.21.15.32:0/496238427 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f40e801a070 con 0x7f4100144e90 2024-04-17T10:03:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.791+0000 7f40f7400640 1 --2- 172.21.15.32:0/496238427 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40dc068380 0x7f40dc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.791+0000 7f40f7400640 1 -- 172.21.15.32:0/496238427 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5709+0+0 (secure 0 0 0) 0x7f40e8086890 con 0x7f4100144e90 2024-04-17T10:03:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.791+0000 7f40ff400640 1 --2- 172.21.15.32:0/496238427 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40dc068380 0x7f40dc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.791+0000 7f40ff400640 1 --2- 172.21.15.32:0/496238427 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40dc068380 0x7f40dc06a840 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f40f0000f70 tx=0x7f40f0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:38.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.795+0000 7f40f7400640 1 -- 172.21.15.32:0/496238427 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f40e8056e50 con 0x7f4100144e90 2024-04-17T10:03:38.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.959+0000 7f4106400640 1 -- 172.21.15.32:0/496238427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f410006f6b0 con 0x7f4100144e90 2024-04-17T10:03:38.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.959+0000 7f40f7400640 1 -- 172.21.15.32:0/496238427 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+16652 (secure 0 0 0) 0x7f40e8056750 con 0x7f4100144e90 2024-04-17T10:03:38.959 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:03:38.959 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":108,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:03:37.619826+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:03:36.021322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-17T10:03:36.581134+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:03:38.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.963+0000 7f4106400640 1 -- 172.21.15.32:0/496238427 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40dc068380 msgr2=0x7f40dc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:38.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.963+0000 7f4106400640 1 --2- 172.21.15.32:0/496238427 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40dc068380 0x7f40dc06a840 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f40f0000f70 tx=0x7f40f0009040 comp rx=0 tx=0).stop 2024-04-17T10:03:38.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.963+0000 7f4106400640 1 -- 172.21.15.32:0/496238427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4100144e90 msgr2=0x7f410015d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:38.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.963+0000 7f4106400640 1 --2- 172.21.15.32:0/496238427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4100144e90 0x7f410015d960 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f40e800e870 tx=0x7f40e800ec70 comp rx=0 tx=0).stop 2024-04-17T10:03:38.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.963+0000 7f4106400640 1 -- 172.21.15.32:0/496238427 shutdown_connections 2024-04-17T10:03:38.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.963+0000 7f4106400640 1 --2- 172.21.15.32:0/496238427 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40dc068380 0x7f40dc06a840 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.963+0000 7f4106400640 1 --2- 172.21.15.32:0/496238427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f410015baa0 0x7f410015a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.963+0000 7f4106400640 1 --2- 172.21.15.32:0/496238427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4100144e90 0x7f410015d960 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.963+0000 7f4106400640 1 --2- 172.21.15.32:0/496238427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f410006ef60 0x7f410013fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:38.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.963+0000 7f4106400640 1 -- 172.21.15.32:0/496238427 >> 172.21.15.32:0/496238427 conn(0x7f410006ba30 msgr2=0x7f410015e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:38.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.963+0000 7f4106400640 1 -- 172.21.15.32:0/496238427 shutdown_connections 2024-04-17T10:03:38.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:38.963+0000 7f4106400640 1 -- 172.21.15.32:0/496238427 wait complete. 2024-04-17T10:03:38.980 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T10:03:39.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.127+0000 7f32cd400640 1 Processor -- start 2024-04-17T10:03:39.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.127+0000 7f32cd400640 1 -- start start 2024-04-17T10:03:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.127+0000 7f32cd400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32c815b7d0 0x7f32c806e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.127+0000 7f32cd400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f32c806f7c0 0x7f32c806ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.127+0000 7f32cd400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f32c806f270 0x7f32c806bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.127+0000 7f32cd400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f32c80cc940 con 0x7f32c806f270 2024-04-17T10:03:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.127+0000 7f32cd400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f32c80ccc20 con 0x7f32c806f7c0 2024-04-17T10:03:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.127+0000 7f32cd400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f32c806e0b0 con 0x7f32c815b7d0 2024-04-17T10:03:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.127+0000 7f32c7200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f32c806f270 0x7f32c806bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.127+0000 7f32c7200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f32c806f270 0x7f32c806bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58870/0 (socket says 172.21.15.32:58870) 2024-04-17T10:03:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.127+0000 7f32c7200640 1 -- 172.21.15.32:0/3097001877 learned_addr learned my addr 172.21.15.32:0/3097001877 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:39.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.127+0000 7f32c6800640 1 --2- 172.21.15.32:0/3097001877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32c815b7d0 0x7f32c806e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.127+0000 7f32bde00640 1 --2- 172.21.15.32:0/3097001877 >> v2:172.21.15.144:3300/0 conn(0x7f32c806f7c0 0x7f32c806ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.127+0000 7f32c7200640 1 -- 172.21.15.32:0/3097001877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32c815b7d0 msgr2=0x7f32c806e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.127+0000 7f32c7200640 1 --2- 172.21.15.32:0/3097001877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32c815b7d0 0x7f32c806e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.127+0000 7f32c7200640 1 -- 172.21.15.32:0/3097001877 >> v2:172.21.15.144:3300/0 conn(0x7f32c806f7c0 msgr2=0x7f32c806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.127+0000 7f32c7200640 1 --2- 172.21.15.32:0/3097001877 >> v2:172.21.15.144:3300/0 conn(0x7f32c806f7c0 0x7f32c806ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:39.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.127+0000 7f32c7200640 1 -- 172.21.15.32:0/3097001877 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c806e390 con 0x7f32c806f270 2024-04-17T10:03:39.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.131+0000 7f32c7200640 1 --2- 172.21.15.32:0/3097001877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32c806f270 0x7f32c806bf70 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f32c8072ff0 tx=0x7f32c001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe0667cab51107d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:39.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.131+0000 7f32c5e00640 1 -- 172.21.15.32:0/3097001877 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c00b2070 con 0x7f32c806f270 2024-04-17T10:03:39.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.131+0000 7f32c5e00640 1 -- 172.21.15.32:0/3097001877 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32c00a9070 con 0x7f32c806f270 2024-04-17T10:03:39.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.131+0000 7f32c5e00640 1 -- 172.21.15.32:0/3097001877 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c00ad880 con 0x7f32c806f270 2024-04-17T10:03:39.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.131+0000 7f32cd400640 1 -- 172.21.15.32:0/3097001877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32c806f270 msgr2=0x7f32c806bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:39.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.131+0000 7f32cd400640 1 --2- 172.21.15.32:0/3097001877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32c806f270 0x7f32c806bf70 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f32c8072ff0 tx=0x7f32c001c490 comp rx=0 tx=0).stop 2024-04-17T10:03:39.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.131+0000 7f32cd400640 1 -- 172.21.15.32:0/3097001877 shutdown_connections 2024-04-17T10:03:39.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.131+0000 7f32cd400640 1 --2- 172.21.15.32:0/3097001877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32c806f270 0x7f32c806bf70 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:39.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.131+0000 7f32cd400640 1 --2- 172.21.15.32:0/3097001877 >> v2:172.21.15.144:3300/0 conn(0x7f32c806f7c0 0x7f32c806ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:39.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.131+0000 7f32cd400640 1 --2- 172.21.15.32:0/3097001877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32c815b7d0 0x7f32c806e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:39.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.131+0000 7f32cd400640 1 -- 172.21.15.32:0/3097001877 >> 172.21.15.32:0/3097001877 conn(0x7f32c80b2120 msgr2=0x7f32c80b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:39.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.131+0000 7f32cd400640 1 -- 172.21.15.32:0/3097001877 shutdown_connections 2024-04-17T10:03:39.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.131+0000 7f32cd400640 1 -- 172.21.15.32:0/3097001877 wait complete. 2024-04-17T10:03:39.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.131+0000 7f32cd400640 1 Processor -- start 2024-04-17T10:03:39.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.131+0000 7f32cd400640 1 -- start start 2024-04-17T10:03:39.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32cd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32c806f270 0x7f32c81464f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:39.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32cd400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32c806f7c0 0x7f32c815d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:39.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32cd400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32c815b7d0 0x7f32c80b0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:39.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32cd400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f32c80cee30 con 0x7f32c806f270 2024-04-17T10:03:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32cd400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f32c80cecb0 con 0x7f32c806f7c0 2024-04-17T10:03:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32cd400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f32c80b3420 con 0x7f32c815b7d0 2024-04-17T10:03:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32c6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32c806f270 0x7f32c81464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32c6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32c806f270 0x7f32c81464f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58884/0 (socket says 172.21.15.32:58884) 2024-04-17T10:03:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32c6800640 1 -- 172.21.15.32:0/3908356704 learned_addr learned my addr 172.21.15.32:0/3908356704 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32bde00640 1 --2- 172.21.15.32:0/3908356704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32c806f7c0 0x7f32c815d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32c7200640 1 --2- 172.21.15.32:0/3908356704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32c815b7d0 0x7f32c80b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32c6800640 1 -- 172.21.15.32:0/3908356704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32c815b7d0 msgr2=0x7f32c80b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32c6800640 1 --2- 172.21.15.32:0/3908356704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32c815b7d0 0x7f32c80b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32c6800640 1 -- 172.21.15.32:0/3908356704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32c806f7c0 msgr2=0x7f32c815d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32c6800640 1 --2- 172.21.15.32:0/3908356704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32c806f7c0 0x7f32c815d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32c6800640 1 -- 172.21.15.32:0/3908356704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c815dd90 con 0x7f32c806f270 2024-04-17T10:03:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32bde00640 1 --2- 172.21.15.32:0/3908356704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32c806f7c0 0x7f32c815d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32c7200640 1 --2- 172.21.15.32:0/3908356704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32c815b7d0 0x7f32c80b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:39.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32c6800640 1 --2- 172.21.15.32:0/3908356704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32c806f270 0x7f32c81464f0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009d10 tx=0x7f32b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:39.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32bf400640 1 -- 172.21.15.32:0/3908356704 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8018020 con 0x7f32c806f270 2024-04-17T10:03:39.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32bf400640 1 -- 172.21.15.32:0/3908356704 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32b8014480 con 0x7f32c806f270 2024-04-17T10:03:39.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32cd400640 1 -- 172.21.15.32:0/3908356704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c8150fd0 con 0x7f32c806f270 2024-04-17T10:03:39.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32bf400640 1 -- 172.21.15.32:0/3908356704 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8014830 con 0x7f32c806f270 2024-04-17T10:03:39.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32cd400640 1 -- 172.21.15.32:0/3908356704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c815da80 con 0x7f32c806f270 2024-04-17T10:03:39.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32cd400640 1 -- 172.21.15.32:0/3908356704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c8068a90 con 0x7f32c806f270 2024-04-17T10:03:39.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.135+0000 7f32bf400640 1 -- 172.21.15.32:0/3908356704 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f32b801b020 con 0x7f32c806f270 2024-04-17T10:03:39.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.139+0000 7f32bf400640 1 --2- 172.21.15.32:0/3908356704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f32a40683b0 0x7f32a406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:39.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.139+0000 7f32bf400640 1 -- 172.21.15.32:0/3908356704 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5709+0+0 (secure 0 0 0) 0x7f32b8087350 con 0x7f32c806f270 2024-04-17T10:03:39.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.139+0000 7f32bde00640 1 --2- 172.21.15.32:0/3908356704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f32a40683b0 0x7f32a406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:39.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.139+0000 7f32bde00640 1 --2- 172.21.15.32:0/3908356704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f32a40683b0 0x7f32a406a870 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f32c815e8c0 tx=0x7f32b0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:39.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.143+0000 7f32bf400640 1 -- 172.21.15.32:0/3908356704 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f32b8022070 con 0x7f32c806f270 2024-04-17T10:03:39.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:39.343+0000 7f32cd400640 1 -- 172.21.15.32:0/3908356704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f32c80691a0 con 0x7f32c806f270 2024-04-17T10:03:40.202 INFO:tasks.ceph.mds.a.smithi032.stderr:2024-04-17T10:03:40.203+0000 7f229ec00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:03:40.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.247+0000 7f32bf400640 1 -- 172.21.15.32:0/3908356704 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v97) ==== 118+0+0 (secure 0 0 0) 0x7f32b8010040 con 0x7f32c806f270 2024-04-17T10:03:40.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.251+0000 7f32cd400640 1 -- 172.21.15.32:0/3908356704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f32a40683b0 msgr2=0x7f32a406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:40.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.251+0000 7f32cd400640 1 --2- 172.21.15.32:0/3908356704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f32a40683b0 0x7f32a406a870 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f32c815e8c0 tx=0x7f32b0008040 comp rx=0 tx=0).stop 2024-04-17T10:03:40.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.251+0000 7f32cd400640 1 -- 172.21.15.32:0/3908356704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32c806f270 msgr2=0x7f32c81464f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:40.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.251+0000 7f32cd400640 1 --2- 172.21.15.32:0/3908356704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32c806f270 0x7f32c81464f0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009d10 tx=0x7f32b800c490 comp rx=0 tx=0).stop 2024-04-17T10:03:40.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.251+0000 7f32cd400640 1 -- 172.21.15.32:0/3908356704 shutdown_connections 2024-04-17T10:03:40.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.251+0000 7f32cd400640 1 --2- 172.21.15.32:0/3908356704 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f32a40683b0 0x7f32a406a870 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:40.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.251+0000 7f32cd400640 1 --2- 172.21.15.32:0/3908356704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32c815b7d0 0x7f32c80b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:40.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.251+0000 7f32cd400640 1 --2- 172.21.15.32:0/3908356704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32c806f7c0 0x7f32c815d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:40.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.251+0000 7f32cd400640 1 --2- 172.21.15.32:0/3908356704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32c806f270 0x7f32c81464f0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:40.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.251+0000 7f32cd400640 1 -- 172.21.15.32:0/3908356704 >> 172.21.15.32:0/3908356704 conn(0x7f32c80b2120 msgr2=0x7f32c80b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:40.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.251+0000 7f32cd400640 1 -- 172.21.15.32:0/3908356704 shutdown_connections 2024-04-17T10:03:40.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.251+0000 7f32cd400640 1 -- 172.21.15.32:0/3908356704 wait complete. 2024-04-17T10:03:40.272 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-17T10:03:40.272 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-17T10:03:40.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.423+0000 7fd09d000640 1 Processor -- start 2024-04-17T10:03:40.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.423+0000 7fd09d000640 1 -- start start 2024-04-17T10:03:40.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.423+0000 7fd09d000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd09815b7d0 0x7fd09806e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:40.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.423+0000 7fd09d000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd09806f7c0 0x7fd09806ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:40.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.423+0000 7fd09d000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0980cc940 con 0x7fd09806f270 2024-04-17T10:03:40.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.423+0000 7fd09d000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd0980ccc20 con 0x7fd09806f7c0 2024-04-17T10:03:40.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.423+0000 7fd09d000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd09806e0b0 con 0x7fd09815b7d0 2024-04-17T10:03:40.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.423+0000 7fd08d400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd09806f7c0 0x7fd09806ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:40.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.423+0000 7fd08d400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd09806f7c0 0x7fd09806ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49220/0 (socket says 172.21.15.32:49220) 2024-04-17T10:03:40.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.423+0000 7fd08d400640 1 -- 172.21.15.32:0/2173456678 learned_addr learned my addr 172.21.15.32:0/2173456678 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:40.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.423+0000 7fd095e00640 1 --2- 172.21.15.32:0/2173456678 >> v2:172.21.15.144:3300/0 conn(0x7fd09815b7d0 0x7fd09806e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:40.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.423+0000 7fd08d400640 1 -- 172.21.15.32:0/2173456678 >> v2:172.21.15.144:3300/0 conn(0x7fd09815b7d0 msgr2=0x7fd09806e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:40.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.423+0000 7fd08d400640 1 --2- 172.21.15.32:0/2173456678 >> v2:172.21.15.144:3300/0 conn(0x7fd09815b7d0 0x7fd09806e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:40.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.423+0000 7fd08d400640 1 -- 172.21.15.32:0/2173456678 >> v1:172.21.15.144:6789/0 conn(0x7fd09806f270 legacy=0x7fd0980ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:40.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.423+0000 7fd08d400640 1 -- 172.21.15.32:0/2173456678 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09806e390 con 0x7fd09806f7c0 2024-04-17T10:03:40.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.427+0000 7fd08d400640 1 --2- 172.21.15.32:0/2173456678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd09806f7c0 0x7fd09806ba30 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fd0980d1530 tx=0x7fd08001eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320100d677b8b818 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:40.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.427+0000 7fd095400640 1 -- 172.21.15.32:0/2173456678 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0800b4070 con 0x7fd09806f7c0 2024-04-17T10:03:40.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.427+0000 7fd095400640 1 -- 172.21.15.32:0/2173456678 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0800a7b50 con 0x7fd09806f7c0 2024-04-17T10:03:40.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.427+0000 7fd095400640 1 -- 172.21.15.32:0/2173456678 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0800af440 con 0x7fd09806f7c0 2024-04-17T10:03:40.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.427+0000 7fd09d000640 1 -- 172.21.15.32:0/2173456678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd09806f7c0 msgr2=0x7fd09806ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:40.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.427+0000 7fd09d000640 1 --2- 172.21.15.32:0/2173456678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd09806f7c0 0x7fd09806ba30 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fd0980d1530 tx=0x7fd08001eb60 comp rx=0 tx=0).stop 2024-04-17T10:03:40.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.427+0000 7fd09d000640 1 -- 172.21.15.32:0/2173456678 shutdown_connections 2024-04-17T10:03:40.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.427+0000 7fd09d000640 1 --2- 172.21.15.32:0/2173456678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd09806f7c0 0x7fd09806ba30 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:40.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.427+0000 7fd09d000640 1 --2- 172.21.15.32:0/2173456678 >> v2:172.21.15.144:3300/0 conn(0x7fd09815b7d0 0x7fd09806e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:40.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.427+0000 7fd09d000640 1 -- 172.21.15.32:0/2173456678 >> 172.21.15.32:0/2173456678 conn(0x7fd0980b2120 msgr2=0x7fd0980b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:40.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.427+0000 7fd09d000640 1 -- 172.21.15.32:0/2173456678 shutdown_connections 2024-04-17T10:03:40.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.427+0000 7fd09d000640 1 -- 172.21.15.32:0/2173456678 wait complete. 2024-04-17T10:03:40.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd09d000640 1 Processor -- start 2024-04-17T10:03:40.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd09d000640 1 -- start start 2024-04-17T10:03:40.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd09d000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd09806f270 0x7fd098136db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:40.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd09d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd09806f7c0 0x7fd098156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:40.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd09d000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd09815b7d0 0x7fd098152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:40.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd09d000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd098072670 con 0x7fd09806f7c0 2024-04-17T10:03:40.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd09d000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd09806c380 con 0x7fd09815b7d0 2024-04-17T10:03:40.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd09d000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd0980727f0 con 0x7fd09806f270 2024-04-17T10:03:40.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd08d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd09806f7c0 0x7fd098156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:40.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd08d400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd09806f7c0 0x7fd098156a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49230/0 (socket says 172.21.15.32:49230) 2024-04-17T10:03:40.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd08d400640 1 -- 172.21.15.32:0/2952336725 learned_addr learned my addr 172.21.15.32:0/2952336725 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:40.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd096800640 1 --2- 172.21.15.32:0/2952336725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd09815b7d0 0x7fd098152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:40.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd095e00640 1 --2- 172.21.15.32:0/2952336725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd09806f270 0x7fd098136db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:40.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd08d400640 1 -- 172.21.15.32:0/2952336725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd09806f270 msgr2=0x7fd098136db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:40.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd08d400640 1 --2- 172.21.15.32:0/2952336725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd09806f270 0x7fd098136db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:40.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd08d400640 1 -- 172.21.15.32:0/2952336725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd09815b7d0 msgr2=0x7fd098152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:40.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd08d400640 1 --2- 172.21.15.32:0/2952336725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd09815b7d0 0x7fd098152b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:40.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd08d400640 1 -- 172.21.15.32:0/2952336725 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd098137680 con 0x7fd09806f7c0 2024-04-17T10:03:40.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd096800640 1 --2- 172.21.15.32:0/2952336725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd09815b7d0 0x7fd098152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:03:40.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd095e00640 1 --2- 172.21.15.32:0/2952336725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd09806f270 0x7fd098136db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:03:40.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd08d400640 1 --2- 172.21.15.32:0/2952336725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd09806f7c0 0x7fd098156a10 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fd08000b2c0 tx=0x7fd08001b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:40.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd08ea00640 1 -- 172.21.15.32:0/2952336725 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0800bf070 con 0x7fd09806f7c0 2024-04-17T10:03:40.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd08ea00640 1 -- 172.21.15.32:0/2952336725 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0800b6050 con 0x7fd09806f7c0 2024-04-17T10:03:40.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd08ea00640 1 -- 172.21.15.32:0/2952336725 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0800b4030 con 0x7fd09806f7c0 2024-04-17T10:03:40.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd09d000640 1 -- 172.21.15.32:0/2952336725 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0981372f0 con 0x7fd09806f7c0 2024-04-17T10:03:40.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd09d000640 1 -- 172.21.15.32:0/2952336725 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd098137bd0 con 0x7fd09806f7c0 2024-04-17T10:03:40.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.431+0000 7fd09d000640 1 -- 172.21.15.32:0/2952336725 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd098068a90 con 0x7fd09806f7c0 2024-04-17T10:03:40.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.435+0000 7fd08ea00640 1 -- 172.21.15.32:0/2952336725 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd0800c2360 con 0x7fd09806f7c0 2024-04-17T10:03:40.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.435+0000 7fd08ea00640 1 --2- 172.21.15.32:0/2952336725 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd074068380 0x7fd07406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:40.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.435+0000 7fd08ea00640 1 -- 172.21.15.32:0/2952336725 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fd080120b50 con 0x7fd09806f7c0 2024-04-17T10:03:40.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.435+0000 7fd095e00640 1 --2- 172.21.15.32:0/2952336725 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd074068380 0x7fd07406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:40.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.435+0000 7fd095e00640 1 --2- 172.21.15.32:0/2952336725 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd074068380 0x7fd07406a840 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd0880097a0 tx=0x7fd088008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:40.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.439+0000 7fd08ea00640 1 -- 172.21.15.32:0/2952336725 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd0800187e0 con 0x7fd09806f7c0 2024-04-17T10:03:40.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:40.639+0000 7fd09d000640 1 -- 172.21.15.32:0/2952336725 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd0980cf820 con 0x7fd09806f7c0 2024-04-17T10:03:41.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.251+0000 7fd08ea00640 1 -- 172.21.15.32:0/2952336725 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v98) ==== 122+0+0 (secure 0 0 0) 0x7fd0800f1190 con 0x7fd09806f7c0 2024-04-17T10:03:41.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.251+0000 7fd09d000640 1 -- 172.21.15.32:0/2952336725 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd074068380 msgr2=0x7fd07406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:41.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.251+0000 7fd09d000640 1 --2- 172.21.15.32:0/2952336725 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd074068380 0x7fd07406a840 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd0880097a0 tx=0x7fd088008040 comp rx=0 tx=0).stop 2024-04-17T10:03:41.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.251+0000 7fd09d000640 1 -- 172.21.15.32:0/2952336725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd09806f7c0 msgr2=0x7fd098156a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:41.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.251+0000 7fd09d000640 1 --2- 172.21.15.32:0/2952336725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd09806f7c0 0x7fd098156a10 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fd08000b2c0 tx=0x7fd08001b900 comp rx=0 tx=0).stop 2024-04-17T10:03:41.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.251+0000 7fd09d000640 1 -- 172.21.15.32:0/2952336725 shutdown_connections 2024-04-17T10:03:41.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.251+0000 7fd09d000640 1 --2- 172.21.15.32:0/2952336725 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd074068380 0x7fd07406a840 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.251+0000 7fd09d000640 1 --2- 172.21.15.32:0/2952336725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd09815b7d0 0x7fd098152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.251+0000 7fd09d000640 1 --2- 172.21.15.32:0/2952336725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd09806f7c0 0x7fd098156a10 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.251+0000 7fd09d000640 1 --2- 172.21.15.32:0/2952336725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd09806f270 0x7fd098136db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.251+0000 7fd09d000640 1 -- 172.21.15.32:0/2952336725 >> 172.21.15.32:0/2952336725 conn(0x7fd0980b2120 msgr2=0x7fd098159de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:41.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.251+0000 7fd09d000640 1 -- 172.21.15.32:0/2952336725 shutdown_connections 2024-04-17T10:03:41.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.251+0000 7fd09d000640 1 -- 172.21.15.32:0/2952336725 wait complete. 2024-04-17T10:03:41.272 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-17T10:03:41.272 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:03:41.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.415+0000 7f6200600640 1 Processor -- start 2024-04-17T10:03:41.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.415+0000 7f6200600640 1 -- start start 2024-04-17T10:03:41.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.419+0000 7f6200600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f61f81481e0 0x7f61f8071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:41.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.419+0000 7f6200600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61f8072330 0x7f61f80b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:41.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.419+0000 7f6200600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61f806ef70 con 0x7f61f8144f20 2024-04-17T10:03:41.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.419+0000 7f6200600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f61f80aed20 con 0x7f61f81481e0 2024-04-17T10:03:41.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.419+0000 7f6200600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f61f80af000 con 0x7f61f8072330 2024-04-17T10:03:41.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.419+0000 7f61fd200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f61f81481e0 0x7f61f8071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:41.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.419+0000 7f61fdc00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f61f8144f20 0x7f61f806e860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41492/0 (socket says 172.21.15.32:41492) 2024-04-17T10:03:41.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.419+0000 7f61fdc00640 1 -- 172.21.15.32:0/2917678180 learned_addr learned my addr 172.21.15.32:0/2917678180 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:41.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.419+0000 7f61fe600640 1 --2- 172.21.15.32:0/2917678180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61f8072330 0x7f61f80b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:41.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.419+0000 7f61f7e00640 1 -- 172.21.15.32:0/2917678180 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2046628383 0 0) 0x7f61f806ef70 con 0x7f61f8144f20 2024-04-17T10:03:41.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.419+0000 7f61fd200640 1 -- 172.21.15.32:0/2917678180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61f8072330 msgr2=0x7f61f80b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:41.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.419+0000 7f61fd200640 1 --2- 172.21.15.32:0/2917678180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61f8072330 0x7f61f80b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.419+0000 7f61fd200640 1 -- 172.21.15.32:0/2917678180 >> v1:172.21.15.32:6789/0 conn(0x7f61f8144f20 legacy=0x7f61f806e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:41.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.419+0000 7f61fd200640 1 -- 172.21.15.32:0/2917678180 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f80af2e0 con 0x7f61f81481e0 2024-04-17T10:03:41.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.423+0000 7f61fd200640 1 --2- 172.21.15.32:0/2917678180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61f81481e0 0x7f61f8071df0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f61e40097c0 tx=0x7f61e401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e947f4cbc114a6bf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:41.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.423+0000 7f61f7e00640 1 -- 172.21.15.32:0/2917678180 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e40b0070 con 0x7f61f81481e0 2024-04-17T10:03:41.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.423+0000 7f61f7e00640 1 -- 172.21.15.32:0/2917678180 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61e40ab600 con 0x7f61f81481e0 2024-04-17T10:03:41.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.423+0000 7f61f7e00640 1 -- 172.21.15.32:0/2917678180 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e40ab9d0 con 0x7f61f81481e0 2024-04-17T10:03:41.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.423+0000 7f6200600640 1 -- 172.21.15.32:0/2917678180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61f81481e0 msgr2=0x7f61f8071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:41.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.423+0000 7f6200600640 1 --2- 172.21.15.32:0/2917678180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61f81481e0 0x7f61f8071df0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f61e40097c0 tx=0x7f61e401c4a0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.423+0000 7f6200600640 1 -- 172.21.15.32:0/2917678180 shutdown_connections 2024-04-17T10:03:41.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.423+0000 7f6200600640 1 --2- 172.21.15.32:0/2917678180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61f8072330 0x7f61f80b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.423+0000 7f6200600640 1 --2- 172.21.15.32:0/2917678180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61f81481e0 0x7f61f8071df0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.423+0000 7f6200600640 1 -- 172.21.15.32:0/2917678180 >> 172.21.15.32:0/2917678180 conn(0x7f61f806ba30 msgr2=0x7f61f806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:41.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.423+0000 7f6200600640 1 -- 172.21.15.32:0/2917678180 shutdown_connections 2024-04-17T10:03:41.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.423+0000 7f6200600640 1 -- 172.21.15.32:0/2917678180 wait complete. 2024-04-17T10:03:41.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.423+0000 7f6200600640 1 Processor -- start 2024-04-17T10:03:41.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.423+0000 7f6200600640 1 -- start start 2024-04-17T10:03:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f6200600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61f8072330 0x7f61f8140750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f6200600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61f8144f20 0x7f61f8139ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f6200600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61f81481e0 0x7f61f813a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f6200600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f61f80b4360 con 0x7f61f81481e0 2024-04-17T10:03:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f6200600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f61f80b41e0 con 0x7f61f8072330 2024-04-17T10:03:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f6200600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f61f816b5d0 con 0x7f61f8144f20 2024-04-17T10:03:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f61fe600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61f81481e0 0x7f61f813a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f61fe600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61f81481e0 0x7f61f813a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49256/0 (socket says 172.21.15.32:49256) 2024-04-17T10:03:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f61fe600640 1 -- 172.21.15.32:0/3189650711 learned_addr learned my addr 172.21.15.32:0/3189650711 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f61fd200640 1 --2- 172.21.15.32:0/3189650711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61f8144f20 0x7f61f8139ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f61fdc00640 1 --2- 172.21.15.32:0/3189650711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61f8072330 0x7f61f8140750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f61fe600640 1 -- 172.21.15.32:0/3189650711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61f8144f20 msgr2=0x7f61f8139ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f61fe600640 1 --2- 172.21.15.32:0/3189650711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61f8144f20 0x7f61f8139ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f61fe600640 1 -- 172.21.15.32:0/3189650711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61f8072330 msgr2=0x7f61f8140750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f61fe600640 1 --2- 172.21.15.32:0/3189650711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61f8072330 0x7f61f8140750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f61fe600640 1 -- 172.21.15.32:0/3189650711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f81402d0 con 0x7f61f81481e0 2024-04-17T10:03:41.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f61fe600640 1 --2- 172.21.15.32:0/3189650711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61f81481e0 0x7f61f813a400 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f61f000efc0 tx=0x7f61f000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:41.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f61f5600640 1 -- 172.21.15.32:0/3189650711 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f0018020 con 0x7f61f81481e0 2024-04-17T10:03:41.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f61f5600640 1 -- 172.21.15.32:0/3189650711 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61f0014480 con 0x7f61f81481e0 2024-04-17T10:03:41.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f61f5600640 1 -- 172.21.15.32:0/3189650711 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f0014870 con 0x7f61f81481e0 2024-04-17T10:03:41.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f6200600640 1 -- 172.21.15.32:0/3189650711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f8137850 con 0x7f61f81481e0 2024-04-17T10:03:41.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f6200600640 1 -- 172.21.15.32:0/3189650711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f8137e10 con 0x7f61f81481e0 2024-04-17T10:03:41.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f6200600640 1 -- 172.21.15.32:0/3189650711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61f81384d0 con 0x7f61f81481e0 2024-04-17T10:03:41.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.427+0000 7f61f5600640 1 -- 172.21.15.32:0/3189650711 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f61f001b020 con 0x7f61f81481e0 2024-04-17T10:03:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.431+0000 7f61f5600640 1 --2- 172.21.15.32:0/3189650711 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f61d8068380 0x7f61d806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.431+0000 7f61f5600640 1 -- 172.21.15.32:0/3189650711 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f61f00874a0 con 0x7f61f81481e0 2024-04-17T10:03:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.431+0000 7f61fdc00640 1 --2- 172.21.15.32:0/3189650711 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f61d8068380 0x7f61d806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.431+0000 7f61fdc00640 1 --2- 172.21.15.32:0/3189650711 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f61d8068380 0x7f61d806a840 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f61ec000f70 tx=0x7f61ec007b60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:41.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.435+0000 7f61f5600640 1 -- 172.21.15.32:0/3189650711 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f61f81384d0 con 0x7f61f81481e0 2024-04-17T10:03:41.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.639+0000 7f6200600640 1 -- 172.21.15.32:0/3189650711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61f8068a90 con 0x7f61f81481e0 2024-04-17T10:03:41.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.639+0000 7f61f5600640 1 -- 172.21.15.32:0/3189650711 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5204 (secure 0 0 0) 0x7f61f0022070 con 0x7f61f81481e0 2024-04-17T10:03:41.639 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:03:41.640 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":98,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7586,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3601627312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3601627312},{"type":"v1","addr":"172.21.15.144:6834","nonce":3601627312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2382359917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2382359917},{"type":"v1","addr":"172.21.15.144:6835","nonce":2382359917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1857552160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1857552160},{"type":"v1","addr":"172.21.15.32:6835","nonce":1857552160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:03:37.619785+0000","modified":"2024-04-17T10:03:40.645595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9285":{"gid":9285,"name":"a","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/901447746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":901447746},{"type":"v1","addr":"172.21.15.32:6837","nonce":901447746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9285,"qdb_cluster":[9285]},"id":8}]} 2024-04-17T10:03:41.640 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 98 2024-04-17T10:03:41.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.643+0000 7f6200600640 1 -- 172.21.15.32:0/3189650711 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f61d8068380 msgr2=0x7f61d806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:41.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.643+0000 7f6200600640 1 --2- 172.21.15.32:0/3189650711 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f61d8068380 0x7f61d806a840 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f61ec000f70 tx=0x7f61ec007b60 comp rx=0 tx=0).stop 2024-04-17T10:03:41.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.643+0000 7f6200600640 1 -- 172.21.15.32:0/3189650711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61f81481e0 msgr2=0x7f61f813a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:41.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.643+0000 7f6200600640 1 --2- 172.21.15.32:0/3189650711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61f81481e0 0x7f61f813a400 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f61f000efc0 tx=0x7f61f000c490 comp rx=0 tx=0).stop 2024-04-17T10:03:41.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.643+0000 7f6200600640 1 -- 172.21.15.32:0/3189650711 shutdown_connections 2024-04-17T10:03:41.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.643+0000 7f6200600640 1 --2- 172.21.15.32:0/3189650711 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f61d8068380 0x7f61d806a840 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.643+0000 7f6200600640 1 --2- 172.21.15.32:0/3189650711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61f8144f20 0x7f61f8139ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.643+0000 7f6200600640 1 --2- 172.21.15.32:0/3189650711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61f81481e0 0x7f61f813a400 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.643+0000 7f6200600640 1 --2- 172.21.15.32:0/3189650711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61f8072330 0x7f61f8140750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.643+0000 7f6200600640 1 -- 172.21.15.32:0/3189650711 >> 172.21.15.32:0/3189650711 conn(0x7f61f806ba30 msgr2=0x7f61f81425b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:41.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.643+0000 7f6200600640 1 -- 172.21.15.32:0/3189650711 shutdown_connections 2024-04-17T10:03:41.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.643+0000 7f6200600640 1 -- 172.21.15.32:0/3189650711 wait complete. 2024-04-17T10:03:41.661 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:03:41.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.803+0000 7f4240600640 1 Processor -- start 2024-04-17T10:03:41.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.803+0000 7f4240600640 1 -- start start 2024-04-17T10:03:41.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.803+0000 7f4240600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4238072500 0x7f42380cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:41.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.803+0000 7f4240600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f423815d8d0 0x7f42380cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:41.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.803+0000 7f4240600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42380cdac0 con 0x7f4238145f30 2024-04-17T10:03:41.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.803+0000 7f4240600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f423806f400 con 0x7f423815d8d0 2024-04-17T10:03:41.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.803+0000 7f4240600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f4238146310 con 0x7f4238072500 2024-04-17T10:03:41.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.803+0000 7f423dc00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4238145f30 0x7f4238071df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41496/0 (socket says 172.21.15.32:41496) 2024-04-17T10:03:41.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.803+0000 7f423e600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f423815d8d0 0x7f42380cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:41.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.803+0000 7f423dc00640 1 -- 172.21.15.32:0/4031253097 learned_addr learned my addr 172.21.15.32:0/4031253097 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:41.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.803+0000 7f423d200640 1 --2- 172.21.15.32:0/4031253097 >> v2:172.21.15.144:3300/0 conn(0x7f4238072500 0x7f42380cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:41.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.803+0000 7f4237e00640 1 -- 172.21.15.32:0/4031253097 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1108136301 0 0) 0x7f42380cdac0 con 0x7f4238145f30 2024-04-17T10:03:41.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.803+0000 7f4237e00640 1 -- 172.21.15.32:0/4031253097 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4220003530 con 0x7f4238145f30 2024-04-17T10:03:41.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.803+0000 7f423e600640 1 -- 172.21.15.32:0/4031253097 >> v2:172.21.15.144:3300/0 conn(0x7f4238072500 msgr2=0x7f42380cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:41.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.803+0000 7f423e600640 1 --2- 172.21.15.32:0/4031253097 >> v2:172.21.15.144:3300/0 conn(0x7f4238072500 0x7f42380cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.803+0000 7f423e600640 1 -- 172.21.15.32:0/4031253097 >> v1:172.21.15.32:6789/0 conn(0x7f4238145f30 legacy=0x7f4238071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:41.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.803+0000 7f423e600640 1 -- 172.21.15.32:0/4031253097 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423815cdf0 con 0x7f423815d8d0 2024-04-17T10:03:41.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.807+0000 7f423e600640 1 --2- 172.21.15.32:0/4031253097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423815d8d0 0x7f42380cd3b0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f42380702a0 tx=0x7f423001f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf046b9b1a7ba3b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:41.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.807+0000 7f4237e00640 1 -- 172.21.15.32:0/4031253097 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42300b4070 con 0x7f423815d8d0 2024-04-17T10:03:41.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.807+0000 7f4237e00640 1 -- 172.21.15.32:0/4031253097 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42300af600 con 0x7f423815d8d0 2024-04-17T10:03:41.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.807+0000 7f4237e00640 1 -- 172.21.15.32:0/4031253097 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42300af9d0 con 0x7f423815d8d0 2024-04-17T10:03:41.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.807+0000 7f4240600640 1 -- 172.21.15.32:0/4031253097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423815d8d0 msgr2=0x7f42380cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:41.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.807+0000 7f4240600640 1 --2- 172.21.15.32:0/4031253097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423815d8d0 0x7f42380cd3b0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f42380702a0 tx=0x7f423001f110 comp rx=0 tx=0).stop 2024-04-17T10:03:41.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.807+0000 7f4240600640 1 -- 172.21.15.32:0/4031253097 shutdown_connections 2024-04-17T10:03:41.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.807+0000 7f4240600640 1 --2- 172.21.15.32:0/4031253097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423815d8d0 0x7f42380cd3b0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.807+0000 7f4240600640 1 --2- 172.21.15.32:0/4031253097 >> v2:172.21.15.144:3300/0 conn(0x7f4238072500 0x7f42380cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.807+0000 7f4240600640 1 -- 172.21.15.32:0/4031253097 >> 172.21.15.32:0/4031253097 conn(0x7f423806ba30 msgr2=0x7f423806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:41.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.807+0000 7f4240600640 1 -- 172.21.15.32:0/4031253097 shutdown_connections 2024-04-17T10:03:41.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.807+0000 7f4240600640 1 -- 172.21.15.32:0/4031253097 wait complete. 2024-04-17T10:03:41.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f4240600640 1 Processor -- start 2024-04-17T10:03:41.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f4240600640 1 -- start start 2024-04-17T10:03:41.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f4240600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4238072500 0x7f423815bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:41.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f4240600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4238145f30 0x7f42380cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:41.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f4240600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423815d8d0 0x7f42381e8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:41.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f4240600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f423815cb90 con 0x7f423815d8d0 2024-04-17T10:03:41.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f4240600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f423815d440 con 0x7f4238145f30 2024-04-17T10:03:41.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f4240600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f42381601d0 con 0x7f4238072500 2024-04-17T10:03:41.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f423e600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423815d8d0 0x7f42381e8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:41.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f423e600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423815d8d0 0x7f42381e8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49268/0 (socket says 172.21.15.32:49268) 2024-04-17T10:03:41.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f423e600640 1 -- 172.21.15.32:0/4073243887 learned_addr learned my addr 172.21.15.32:0/4073243887 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:41.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f423dc00640 1 --2- 172.21.15.32:0/4073243887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4238072500 0x7f423815bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:41.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f423d200640 1 --2- 172.21.15.32:0/4073243887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4238145f30 0x7f42380cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:41.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f423e600640 1 -- 172.21.15.32:0/4073243887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4238072500 msgr2=0x7f423815bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:41.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f423e600640 1 --2- 172.21.15.32:0/4073243887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4238072500 0x7f423815bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f423e600640 1 -- 172.21.15.32:0/4073243887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4238145f30 msgr2=0x7f42380cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:41.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f423e600640 1 --2- 172.21.15.32:0/4073243887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4238145f30 0x7f42380cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f423e600640 1 -- 172.21.15.32:0/4073243887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42380c9d80 con 0x7f423815d8d0 2024-04-17T10:03:41.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f423e600640 1 --2- 172.21.15.32:0/4073243887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423815d8d0 0x7f42381e8200 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f42300b2040 tx=0x7f423001beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:41.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f4235600640 1 -- 172.21.15.32:0/4073243887 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42300b4040 con 0x7f423815d8d0 2024-04-17T10:03:41.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f4235600640 1 -- 172.21.15.32:0/4073243887 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42300c1070 con 0x7f423815d8d0 2024-04-17T10:03:41.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f4235600640 1 -- 172.21.15.32:0/4073243887 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230009450 con 0x7f423815d8d0 2024-04-17T10:03:41.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f4240600640 1 -- 172.21.15.32:0/4073243887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42380ca060 con 0x7f423815d8d0 2024-04-17T10:03:41.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.811+0000 7f4240600640 1 -- 172.21.15.32:0/4073243887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42380ca620 con 0x7f423815d8d0 2024-04-17T10:03:41.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.815+0000 7f4240600640 1 -- 172.21.15.32:0/4073243887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4238068a90 con 0x7f423815d8d0 2024-04-17T10:03:41.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.815+0000 7f4235600640 1 -- 172.21.15.32:0/4073243887 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f42300ba6b0 con 0x7f423815d8d0 2024-04-17T10:03:41.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.815+0000 7f4235600640 1 --2- 172.21.15.32:0/4073243887 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4218068380 0x7f421806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:41.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.815+0000 7f4235600640 1 -- 172.21.15.32:0/4073243887 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f4230125300 con 0x7f423815d8d0 2024-04-17T10:03:41.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.815+0000 7f423dc00640 1 --2- 172.21.15.32:0/4073243887 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4218068380 0x7f421806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:41.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.815+0000 7f423dc00640 1 --2- 172.21.15.32:0/4073243887 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4218068380 0x7f421806a840 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f422c005890 tx=0x7f422c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.819+0000 7f4235600640 1 -- 172.21.15.32:0/4073243887 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f4230018c70 con 0x7f423815d8d0 2024-04-17T10:03:41.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.983+0000 7f4240600640 1 -- 172.21.15.32:0/4073243887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4238072f90 con 0x7f423815d8d0 2024-04-17T10:03:41.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.987+0000 7f4235600640 1 -- 172.21.15.32:0/4073243887 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+16639 (secure 0 0 0) 0x7f42300f5940 con 0x7f423815d8d0 2024-04-17T10:03:41.984 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:03:41.984 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":109,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:03:39.241637+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:03:36.021322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-17T10:03:36.581134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:03:41.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.987+0000 7f4240600640 1 -- 172.21.15.32:0/4073243887 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4218068380 msgr2=0x7f421806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:41.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.987+0000 7f4240600640 1 --2- 172.21.15.32:0/4073243887 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4218068380 0x7f421806a840 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f422c005890 tx=0x7f422c009040 comp rx=0 tx=0).stop 2024-04-17T10:03:41.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.987+0000 7f4240600640 1 -- 172.21.15.32:0/4073243887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423815d8d0 msgr2=0x7f42381e8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:41.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.987+0000 7f4240600640 1 --2- 172.21.15.32:0/4073243887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423815d8d0 0x7f42381e8200 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f42300b2040 tx=0x7f423001beb0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.987+0000 7f4240600640 1 -- 172.21.15.32:0/4073243887 shutdown_connections 2024-04-17T10:03:41.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.987+0000 7f4240600640 1 --2- 172.21.15.32:0/4073243887 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4218068380 0x7f421806a840 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.987+0000 7f4240600640 1 --2- 172.21.15.32:0/4073243887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423815d8d0 0x7f42381e8200 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.987+0000 7f4240600640 1 --2- 172.21.15.32:0/4073243887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4238145f30 0x7f42380cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.987+0000 7f4240600640 1 --2- 172.21.15.32:0/4073243887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4238072500 0x7f423815bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:41.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.987+0000 7f4240600640 1 -- 172.21.15.32:0/4073243887 >> 172.21.15.32:0/4073243887 conn(0x7f423806ba30 msgr2=0x7f42381451e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:41.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.987+0000 7f4240600640 1 -- 172.21.15.32:0/4073243887 shutdown_connections 2024-04-17T10:03:41.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:41.987+0000 7f4240600640 1 -- 172.21.15.32:0/4073243887 wait complete. 2024-04-17T10:03:42.005 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T10:03:42.005 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:03:42.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.147+0000 7f16eae00640 1 Processor -- start 2024-04-17T10:03:42.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.147+0000 7f16eae00640 1 -- start start 2024-04-17T10:03:42.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.147+0000 7f16eae00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16e4144e90 0x7f16e40b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:42.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.147+0000 7f16eae00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f16e415c1b0 0x7f16e406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:42.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.147+0000 7f16eae00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16e4072bb0 con 0x7f16e40b4de0 2024-04-17T10:03:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.147+0000 7f16eae00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f16e4145270 con 0x7f16e415c1b0 2024-04-17T10:03:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.147+0000 7f16eae00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f16e40b3dc0 con 0x7f16e4144e90 2024-04-17T10:03:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.147+0000 7f16db400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f16e40b4de0 0x7f16e415baa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41508/0 (socket says 172.21.15.32:41508) 2024-04-17T10:03:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.147+0000 7f16db400640 1 -- 172.21.15.32:0/4264131478 learned_addr learned my addr 172.21.15.32:0/4264131478 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.147+0000 7f16e3400640 1 -- 172.21.15.32:0/4264131478 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2867354860 0 0) 0x7f16e4072bb0 con 0x7f16e40b4de0 2024-04-17T10:03:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.147+0000 7f16e3e00640 1 --2- 172.21.15.32:0/4264131478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16e4144e90 0x7f16e40b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.147+0000 7f16e8e00640 1 --2- 172.21.15.32:0/4264131478 >> v2:172.21.15.144:3300/0 conn(0x7f16e415c1b0 0x7f16e406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.147+0000 7f16e3400640 1 -- 172.21.15.32:0/4264131478 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16c8003530 con 0x7f16e40b4de0 2024-04-17T10:03:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16e3400640 1 -- 172.21.15.32:0/4264131478 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3673295522 0 0) 0x7f16c8003530 con 0x7f16e40b4de0 2024-04-17T10:03:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16e3400640 1 -- 172.21.15.32:0/4264131478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16e4144e90 msgr2=0x7f16e40b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16e3400640 1 --2- 172.21.15.32:0/4264131478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16e4144e90 0x7f16e40b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16e3400640 1 -- 172.21.15.32:0/4264131478 >> v2:172.21.15.144:3300/0 conn(0x7f16e415c1b0 msgr2=0x7f16e406ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16e3400640 1 --2- 172.21.15.32:0/4264131478 >> v2:172.21.15.144:3300/0 conn(0x7f16e415c1b0 0x7f16e406ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16e3400640 1 -- 172.21.15.32:0/4264131478 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e40b40a0 con 0x7f16e40b4de0 2024-04-17T10:03:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16e3400640 1 -- 172.21.15.32:0/4264131478 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f16cc0016a0 con 0x7f16e40b4de0 2024-04-17T10:03:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16e3400640 1 -- 172.21.15.32:0/4264131478 >> v1:172.21.15.32:6789/0 conn(0x7f16e40b4de0 legacy=0x7f16e415baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16e3400640 1 --2- 172.21.15.32:0/4264131478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16e4144e90 0x7f16c80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16e3400640 1 --2- 172.21.15.32:0/4264131478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16c8003f40 0x7f16c8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16e3400640 1 --2- 172.21.15.32:0/4264131478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16c8004d50 0x7f16c80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16e3400640 1 -- 172.21.15.32:0/4264131478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f16e4145270 con 0x7f16c8003f40 2024-04-17T10:03:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16e3400640 1 -- 172.21.15.32:0/4264131478 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f16e40b3dc0 con 0x7f16c8004d50 2024-04-17T10:03:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16e3400640 1 -- 172.21.15.32:0/4264131478 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f16c800b9d0 con 0x7f16e4144e90 2024-04-17T10:03:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16db400640 1 --2- 172.21.15.32:0/4264131478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16c8003f40 0x7f16c8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16e8e00640 1 --2- 172.21.15.32:0/4264131478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16c8004d50 0x7f16c80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16e3e00640 1 --2- 172.21.15.32:0/4264131478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16e4144e90 0x7f16c80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16db400640 1 -- 172.21.15.32:0/4264131478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16e4144e90 msgr2=0x7f16c80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16db400640 1 --2- 172.21.15.32:0/4264131478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16e4144e90 0x7f16c80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16db400640 1 -- 172.21.15.32:0/4264131478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16c8004d50 msgr2=0x7f16c80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16db400640 1 --2- 172.21.15.32:0/4264131478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16c8004d50 0x7f16c80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16db400640 1 -- 172.21.15.32:0/4264131478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c8003530 con 0x7f16c8003f40 2024-04-17T10:03:42.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16e8e00640 1 --2- 172.21.15.32:0/4264131478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16c8004d50 0x7f16c80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:42.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16e3e00640 1 --2- 172.21.15.32:0/4264131478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16e4144e90 0x7f16c80039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16db400640 1 --2- 172.21.15.32:0/4264131478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16c8003f40 0x7f16c8004810 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f16cc00c210 tx=0x7f16cc01f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a6259d2f483d0e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16e3400640 1 -- 172.21.15.32:0/4264131478 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc007d80 con 0x7f16c8003f40 2024-04-17T10:03:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16e3400640 1 -- 172.21.15.32:0/4264131478 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16cc005b00 con 0x7f16c8003f40 2024-04-17T10:03:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.151+0000 7f16e3400640 1 -- 172.21.15.32:0/4264131478 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc007070 con 0x7f16c8003f40 2024-04-17T10:03:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16eae00640 1 -- 172.21.15.32:0/4264131478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16c8003f40 msgr2=0x7f16c8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16eae00640 1 --2- 172.21.15.32:0/4264131478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16c8003f40 0x7f16c8004810 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f16cc00c210 tx=0x7f16cc01f390 comp rx=0 tx=0).stop 2024-04-17T10:03:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16e8e00640 1 -- 172.21.15.32:0/4264131478 reap_dead start 2024-04-17T10:03:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16eae00640 1 -- 172.21.15.32:0/4264131478 shutdown_connections 2024-04-17T10:03:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16eae00640 1 -- 172.21.15.32:0/4264131478 >> 172.21.15.32:0/4264131478 conn(0x7f16e406ba30 msgr2=0x7f16e406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:42.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16eae00640 1 -- 172.21.15.32:0/4264131478 shutdown_connections 2024-04-17T10:03:42.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16eae00640 1 -- 172.21.15.32:0/4264131478 wait complete. 2024-04-17T10:03:42.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16eae00640 1 Processor -- start 2024-04-17T10:03:42.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16eae00640 1 -- start start 2024-04-17T10:03:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16eae00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16c8003f40 0x7f16e41f3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16eae00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16e415c590 0x7f16e415ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16eae00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e413ede0 0x7f16e413f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16eae00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f16e40b4a20 con 0x7f16e415c590 2024-04-17T10:03:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16eae00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f16e40b48a0 con 0x7f16e413ede0 2024-04-17T10:03:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16eae00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f16e40b4ba0 con 0x7f16c8003f40 2024-04-17T10:03:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16db400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16e415c590 0x7f16e415ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16db400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16e415c590 0x7f16e415ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49290/0 (socket says 172.21.15.32:49290) 2024-04-17T10:03:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16db400640 1 -- 172.21.15.32:0/2730161808 learned_addr learned my addr 172.21.15.32:0/2730161808 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16e3e00640 1 --2- 172.21.15.32:0/2730161808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16c8003f40 0x7f16e41f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16e8e00640 1 --2- 172.21.15.32:0/2730161808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e413ede0 0x7f16e413f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16db400640 1 -- 172.21.15.32:0/2730161808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16c8003f40 msgr2=0x7f16e41f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16db400640 1 --2- 172.21.15.32:0/2730161808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16c8003f40 0x7f16e41f3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16db400640 1 -- 172.21.15.32:0/2730161808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e413ede0 msgr2=0x7f16e413f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16db400640 1 --2- 172.21.15.32:0/2730161808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e413ede0 0x7f16e413f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.155+0000 7f16db400640 1 -- 172.21.15.32:0/2730161808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e41f3970 con 0x7f16e415c590 2024-04-17T10:03:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.159+0000 7f16db400640 1 --2- 172.21.15.32:0/2730161808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16e415c590 0x7f16e415ca10 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f16cc01f1c0 tx=0x7f16c8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.159+0000 7f16e0c00640 1 -- 172.21.15.32:0/2730161808 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc005cb0 con 0x7f16e415c590 2024-04-17T10:03:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.159+0000 7f16e0c00640 1 -- 172.21.15.32:0/2730161808 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16cc005e50 con 0x7f16e415c590 2024-04-17T10:03:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.159+0000 7f16e0c00640 1 -- 172.21.15.32:0/2730161808 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc0049f0 con 0x7f16e415c590 2024-04-17T10:03:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.159+0000 7f16eae00640 1 -- 172.21.15.32:0/2730161808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e415d030 con 0x7f16e415c590 2024-04-17T10:03:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.159+0000 7f16eae00640 1 -- 172.21.15.32:0/2730161808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e415d5f0 con 0x7f16e415c590 2024-04-17T10:03:42.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.159+0000 7f16eae00640 1 -- 172.21.15.32:0/2730161808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e415b720 con 0x7f16e415c590 2024-04-17T10:03:42.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.159+0000 7f16e0c00640 1 -- 172.21.15.32:0/2730161808 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f16cc0ba030 con 0x7f16e415c590 2024-04-17T10:03:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.159+0000 7f16e0c00640 1 --2- 172.21.15.32:0/2730161808 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f16c0068350 0x7f16c006a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.159+0000 7f16e0c00640 1 -- 172.21.15.32:0/2730161808 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f16cc1233c0 con 0x7f16e415c590 2024-04-17T10:03:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.159+0000 7f16e3e00640 1 --2- 172.21.15.32:0/2730161808 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f16c0068350 0x7f16c006a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.163+0000 7f16e3e00640 1 --2- 172.21.15.32:0/2730161808 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f16c0068350 0x7f16c006a810 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f16e415e3f0 tx=0x7f16d4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.163+0000 7f16e0c00640 1 -- 172.21.15.32:0/2730161808 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f16cc019100 con 0x7f16e415c590 2024-04-17T10:03:42.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.327+0000 7f16eae00640 1 -- 172.21.15.32:0/2730161808 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f16e4068a90 con 0x7f16c0068350 2024-04-17T10:03:42.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.335+0000 7f16e0c00640 1 -- 172.21.15.32:0/2730161808 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252467 (secure 0 0 0) 0x7f16e4068a90 con 0x7f16c0068350 2024-04-17T10:03:42.334 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:03:42.336 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T10:03:42.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.335+0000 7f16eae00640 1 -- 172.21.15.32:0/2730161808 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f16c0068350 msgr2=0x7f16c006a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:42.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.335+0000 7f16eae00640 1 --2- 172.21.15.32:0/2730161808 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f16c0068350 0x7f16c006a810 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f16e415e3f0 tx=0x7f16d4008040 comp rx=0 tx=0).stop 2024-04-17T10:03:42.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.335+0000 7f16eae00640 1 -- 172.21.15.32:0/2730161808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16e415c590 msgr2=0x7f16e415ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:42.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.335+0000 7f16eae00640 1 --2- 172.21.15.32:0/2730161808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16e415c590 0x7f16e415ca10 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f16cc01f1c0 tx=0x7f16c8005840 comp rx=0 tx=0).stop 2024-04-17T10:03:42.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.335+0000 7f16eae00640 1 -- 172.21.15.32:0/2730161808 shutdown_connections 2024-04-17T10:03:42.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.335+0000 7f16eae00640 1 --2- 172.21.15.32:0/2730161808 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f16c0068350 0x7f16c006a810 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.335+0000 7f16eae00640 1 --2- 172.21.15.32:0/2730161808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e413ede0 0x7f16e413f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.335+0000 7f16eae00640 1 --2- 172.21.15.32:0/2730161808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16e415c590 0x7f16e415ca10 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.335+0000 7f16eae00640 1 --2- 172.21.15.32:0/2730161808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16c8003f40 0x7f16e41f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.335+0000 7f16eae00640 1 -- 172.21.15.32:0/2730161808 >> 172.21.15.32:0/2730161808 conn(0x7f16e406ba30 msgr2=0x7f16e4142860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.339+0000 7f16eae00640 1 -- 172.21.15.32:0/2730161808 shutdown_connections 2024-04-17T10:03:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.339+0000 7f16eae00640 1 -- 172.21.15.32:0/2730161808 wait complete. 2024-04-17T10:03:42.355 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-04-17T10:03:41.433341+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265344,"kb_used_data":11704,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754709376,"statfs":{"total":773094113280,"available":772822401024,"internally_reserved":0,"allocated":11984896,"data_stored":5825508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.221510"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.256233+0000","last_change":"2024-04-17T10:03:38.241978+0000","last_active":"2024-04-17T10:03:38.256233+0000","last_peered":"2024-04-17T10:03:38.256233+0000","last_clean":"2024-04-17T10:03:38.256233+0000","last_became_active":"2024-04-17T10:03:38.241715+0000","last_became_peered":"2024-04-17T10:03:38.241715+0000","last_unstale":"2024-04-17T10:03:38.256233+0000","last_undegraded":"2024-04-17T10:03:38.256233+0000","last_fullsized":"2024-04-17T10:03:38.256233+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:25.842390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249635+0000","last_change":"2024-04-17T10:03:38.242768+0000","last_active":"2024-04-17T10:03:39.249635+0000","last_peered":"2024-04-17T10:03:39.249635+0000","last_clean":"2024-04-17T10:03:39.249635+0000","last_became_active":"2024-04-17T10:03:38.242480+0000","last_became_peered":"2024-04-17T10:03:38.242480+0000","last_unstale":"2024-04-17T10:03:39.249635+0000","last_undegraded":"2024-04-17T10:03:39.249635+0000","last_fullsized":"2024-04-17T10:03:39.249635+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:38:00.474515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.035734+0000","last_change":"2024-04-17T10:03:38.253724+0000","last_active":"2024-04-17T10:03:40.035734+0000","last_peered":"2024-04-17T10:03:40.035734+0000","last_clean":"2024-04-17T10:03:40.035734+0000","last_became_active":"2024-04-17T10:03:38.253444+0000","last_became_peered":"2024-04-17T10:03:38.253444+0000","last_unstale":"2024-04-17T10:03:40.035734+0000","last_undegraded":"2024-04-17T10:03:40.035734+0000","last_fullsized":"2024-04-17T10:03:40.035734+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:43:25.504563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.261294+0000","last_change":"2024-04-17T10:03:38.254487+0000","last_active":"2024-04-17T10:03:38.261294+0000","last_peered":"2024-04-17T10:03:38.261294+0000","last_clean":"2024-04-17T10:03:38.261294+0000","last_became_active":"2024-04-17T10:03:38.253818+0000","last_became_peered":"2024-04-17T10:03:38.253818+0000","last_unstale":"2024-04-17T10:03:38.261294+0000","last_undegraded":"2024-04-17T10:03:38.261294+0000","last_fullsized":"2024-04-17T10:03:38.261294+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:09:28.208134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249129+0000","last_change":"2024-04-17T10:03:38.263442+0000","last_active":"2024-04-17T10:03:39.249129+0000","last_peered":"2024-04-17T10:03:39.249129+0000","last_clean":"2024-04-17T10:03:39.249129+0000","last_became_active":"2024-04-17T10:03:38.262179+0000","last_became_peered":"2024-04-17T10:03:38.262179+0000","last_unstale":"2024-04-17T10:03:39.249129+0000","last_undegraded":"2024-04-17T10:03:39.249129+0000","last_fullsized":"2024-04-17T10:03:39.249129+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:56.070036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.950205+0000","last_change":"2024-04-17T10:03:38.253899+0000","last_active":"2024-04-17T10:03:39.950205+0000","last_peered":"2024-04-17T10:03:39.950205+0000","last_clean":"2024-04-17T10:03:39.950205+0000","last_became_active":"2024-04-17T10:03:38.253644+0000","last_became_peered":"2024-04-17T10:03:38.253644+0000","last_unstale":"2024-04-17T10:03:39.950205+0000","last_undegraded":"2024-04-17T10:03:39.950205+0000","last_fullsized":"2024-04-17T10:03:39.950205+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:58:46.428563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.252551+0000","last_change":"2024-04-17T10:03:38.254343+0000","last_active":"2024-04-17T10:03:39.252551+0000","last_peered":"2024-04-17T10:03:39.252551+0000","last_clean":"2024-04-17T10:03:39.252551+0000","last_became_active":"2024-04-17T10:03:38.253868+0000","last_became_peered":"2024-04-17T10:03:38.253868+0000","last_unstale":"2024-04-17T10:03:39.252551+0000","last_undegraded":"2024-04-17T10:03:39.252551+0000","last_fullsized":"2024-04-17T10:03:39.252551+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:40:44.504640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.035484+0000","last_change":"2024-04-17T10:03:38.254947+0000","last_active":"2024-04-17T10:03:40.035484+0000","last_peered":"2024-04-17T10:03:40.035484+0000","last_clean":"2024-04-17T10:03:40.035484+0000","last_became_active":"2024-04-17T10:03:38.254681+0000","last_became_peered":"2024-04-17T10:03:38.254681+0000","last_unstale":"2024-04-17T10:03:40.035484+0000","last_undegraded":"2024-04-17T10:03:40.035484+0000","last_fullsized":"2024-04-17T10:03:40.035484+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:35.405701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.259940+0000","last_change":"2024-04-17T10:03:38.256106+0000","last_active":"2024-04-17T10:03:38.259940+0000","last_peered":"2024-04-17T10:03:38.259940+0000","last_clean":"2024-04-17T10:03:38.259940+0000","last_became_active":"2024-04-17T10:03:38.254920+0000","last_became_peered":"2024-04-17T10:03:38.254920+0000","last_unstale":"2024-04-17T10:03:38.259940+0000","last_undegraded":"2024-04-17T10:03:38.259940+0000","last_fullsized":"2024-04-17T10:03:38.259940+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:05:46.153121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.260934+0000","last_change":"2024-04-17T10:03:38.257961+0000","last_active":"2024-04-17T10:03:38.260934+0000","last_peered":"2024-04-17T10:03:38.260934+0000","last_clean":"2024-04-17T10:03:38.260934+0000","last_became_active":"2024-04-17T10:03:38.257615+0000","last_became_peered":"2024-04-17T10:03:38.257615+0000","last_unstale":"2024-04-17T10:03:38.260934+0000","last_undegraded":"2024-04-17T10:03:38.260934+0000","last_fullsized":"2024-04-17T10:03:38.260934+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:09.795387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249592+0000","last_change":"2024-04-17T10:03:38.241600+0000","last_active":"2024-04-17T10:03:39.249592+0000","last_peered":"2024-04-17T10:03:39.249592+0000","last_clean":"2024-04-17T10:03:39.249592+0000","last_became_active":"2024-04-17T10:03:38.241308+0000","last_became_peered":"2024-04-17T10:03:38.241308+0000","last_unstale":"2024-04-17T10:03:39.249592+0000","last_undegraded":"2024-04-17T10:03:39.249592+0000","last_fullsized":"2024-04-17T10:03:39.249592+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:58:23.683077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.261337+0000","last_change":"2024-04-17T10:03:38.256550+0000","last_active":"2024-04-17T10:03:38.261337+0000","last_peered":"2024-04-17T10:03:38.261337+0000","last_clean":"2024-04-17T10:03:38.261337+0000","last_became_active":"2024-04-17T10:03:38.256284+0000","last_became_peered":"2024-04-17T10:03:38.256284+0000","last_unstale":"2024-04-17T10:03:38.261337+0000","last_undegraded":"2024-04-17T10:03:38.261337+0000","last_fullsized":"2024-04-17T10:03:38.261337+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:25.387774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.035989+0000","last_change":"2024-04-17T10:03:38.250852+0000","last_active":"2024-04-17T10:03:40.035989+0000","last_peered":"2024-04-17T10:03:40.035989+0000","last_clean":"2024-04-17T10:03:40.035989+0000","last_became_active":"2024-04-17T10:03:38.249938+0000","last_became_peered":"2024-04-17T10:03:38.249938+0000","last_unstale":"2024-04-17T10:03:40.035989+0000","last_undegraded":"2024-04-17T10:03:40.035989+0000","last_fullsized":"2024-04-17T10:03:40.035989+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:46:32.849083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.035724+0000","last_change":"2024-04-17T10:03:38.253982+0000","last_active":"2024-04-17T10:03:40.035724+0000","last_peered":"2024-04-17T10:03:40.035724+0000","last_clean":"2024-04-17T10:03:40.035724+0000","last_became_active":"2024-04-17T10:03:38.252903+0000","last_became_peered":"2024-04-17T10:03:38.252903+0000","last_unstale":"2024-04-17T10:03:40.035724+0000","last_undegraded":"2024-04-17T10:03:40.035724+0000","last_fullsized":"2024-04-17T10:03:40.035724+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:08:40.728849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.950171+0000","last_change":"2024-04-17T10:03:38.251999+0000","last_active":"2024-04-17T10:03:39.950171+0000","last_peered":"2024-04-17T10:03:39.950171+0000","last_clean":"2024-04-17T10:03:39.950171+0000","last_became_active":"2024-04-17T10:03:38.251689+0000","last_became_peered":"2024-04-17T10:03:38.251689+0000","last_unstale":"2024-04-17T10:03:39.950171+0000","last_undegraded":"2024-04-17T10:03:39.950171+0000","last_fullsized":"2024-04-17T10:03:39.950171+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:04:44.056356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.257138+0000","last_change":"2024-04-17T10:03:38.252510+0000","last_active":"2024-04-17T10:03:38.257138+0000","last_peered":"2024-04-17T10:03:38.257138+0000","last_clean":"2024-04-17T10:03:38.257138+0000","last_became_active":"2024-04-17T10:03:38.250422+0000","last_became_peered":"2024-04-17T10:03:38.250422+0000","last_unstale":"2024-04-17T10:03:38.257138+0000","last_undegraded":"2024-04-17T10:03:38.257138+0000","last_fullsized":"2024-04-17T10:03:38.257138+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:50:09.412443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249899+0000","last_change":"2024-04-17T10:03:38.241946+0000","last_active":"2024-04-17T10:03:39.249899+0000","last_peered":"2024-04-17T10:03:39.249899+0000","last_clean":"2024-04-17T10:03:39.249899+0000","last_became_active":"2024-04-17T10:03:38.241561+0000","last_became_peered":"2024-04-17T10:03:38.241561+0000","last_unstale":"2024-04-17T10:03:39.249899+0000","last_undegraded":"2024-04-17T10:03:39.249899+0000","last_fullsized":"2024-04-17T10:03:39.249899+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:20:23.111219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248646+0000","last_change":"2024-04-17T10:03:38.242368+0000","last_active":"2024-04-17T10:03:39.248646+0000","last_peered":"2024-04-17T10:03:39.248646+0000","last_clean":"2024-04-17T10:03:39.248646+0000","last_became_active":"2024-04-17T10:03:38.242084+0000","last_became_peered":"2024-04-17T10:03:38.242084+0000","last_unstale":"2024-04-17T10:03:39.248646+0000","last_undegraded":"2024-04-17T10:03:39.248646+0000","last_fullsized":"2024-04-17T10:03:39.248646+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:27:02.945101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.036116+0000","last_change":"2024-04-17T10:03:38.255202+0000","last_active":"2024-04-17T10:03:40.036116+0000","last_peered":"2024-04-17T10:03:40.036116+0000","last_clean":"2024-04-17T10:03:40.036116+0000","last_became_active":"2024-04-17T10:03:38.254410+0000","last_became_peered":"2024-04-17T10:03:38.254410+0000","last_unstale":"2024-04-17T10:03:40.036116+0000","last_undegraded":"2024-04-17T10:03:40.036116+0000","last_fullsized":"2024-04-17T10:03:40.036116+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:49:26.025529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249463+0000","last_change":"2024-04-17T10:03:38.247529+0000","last_active":"2024-04-17T10:03:39.249463+0000","last_peered":"2024-04-17T10:03:39.249463+0000","last_clean":"2024-04-17T10:03:39.249463+0000","last_became_active":"2024-04-17T10:03:38.246836+0000","last_became_peered":"2024-04-17T10:03:38.246836+0000","last_unstale":"2024-04-17T10:03:39.249463+0000","last_undegraded":"2024-04-17T10:03:39.249463+0000","last_fullsized":"2024-04-17T10:03:39.249463+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:27:18.939260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249099+0000","last_change":"2024-04-17T10:03:38.243162+0000","last_active":"2024-04-17T10:03:39.249099+0000","last_peered":"2024-04-17T10:03:39.249099+0000","last_clean":"2024-04-17T10:03:39.249099+0000","last_became_active":"2024-04-17T10:03:38.241641+0000","last_became_peered":"2024-04-17T10:03:38.241641+0000","last_unstale":"2024-04-17T10:03:39.249099+0000","last_undegraded":"2024-04-17T10:03:39.249099+0000","last_fullsized":"2024-04-17T10:03:39.249099+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:52:10.948082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743440+0000","last_change":"2024-04-17T10:03:38.246858+0000","last_active":"2024-04-17T10:03:39.743440+0000","last_peered":"2024-04-17T10:03:39.743440+0000","last_clean":"2024-04-17T10:03:39.743440+0000","last_became_active":"2024-04-17T10:03:38.243321+0000","last_became_peered":"2024-04-17T10:03:38.243321+0000","last_unstale":"2024-04-17T10:03:39.743440+0000","last_undegraded":"2024-04-17T10:03:39.743440+0000","last_fullsized":"2024-04-17T10:03:39.743440+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:28:52.125194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248903+0000","last_change":"2024-04-17T10:03:38.248166+0000","last_active":"2024-04-17T10:03:39.248903+0000","last_peered":"2024-04-17T10:03:39.248903+0000","last_clean":"2024-04-17T10:03:39.248903+0000","last_became_active":"2024-04-17T10:03:38.246440+0000","last_became_peered":"2024-04-17T10:03:38.246440+0000","last_unstale":"2024-04-17T10:03:39.248903+0000","last_undegraded":"2024-04-17T10:03:39.248903+0000","last_fullsized":"2024-04-17T10:03:39.248903+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:42:54.852740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743343+0000","last_change":"2024-04-17T10:03:38.252625+0000","last_active":"2024-04-17T10:03:39.743343+0000","last_peered":"2024-04-17T10:03:39.743343+0000","last_clean":"2024-04-17T10:03:39.743343+0000","last_became_active":"2024-04-17T10:03:38.252197+0000","last_became_peered":"2024-04-17T10:03:38.252197+0000","last_unstale":"2024-04-17T10:03:39.743343+0000","last_undegraded":"2024-04-17T10:03:39.743343+0000","last_fullsized":"2024-04-17T10:03:39.743343+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:35:32.567676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249900+0000","last_change":"2024-04-17T10:03:38.241660+0000","last_active":"2024-04-17T10:03:39.249900+0000","last_peered":"2024-04-17T10:03:39.249900+0000","last_clean":"2024-04-17T10:03:39.249900+0000","last_became_active":"2024-04-17T10:03:38.240632+0000","last_became_peered":"2024-04-17T10:03:38.240632+0000","last_unstale":"2024-04-17T10:03:39.249900+0000","last_undegraded":"2024-04-17T10:03:39.249900+0000","last_fullsized":"2024-04-17T10:03:39.249900+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:23.356537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.258585+0000","last_change":"2024-04-17T10:03:38.251376+0000","last_active":"2024-04-17T10:03:38.258585+0000","last_peered":"2024-04-17T10:03:38.258585+0000","last_clean":"2024-04-17T10:03:38.258585+0000","last_became_active":"2024-04-17T10:03:38.251124+0000","last_became_peered":"2024-04-17T10:03:38.251124+0000","last_unstale":"2024-04-17T10:03:38.258585+0000","last_undegraded":"2024-04-17T10:03:38.258585+0000","last_fullsized":"2024-04-17T10:03:38.258585+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:59:12.444356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.950417+0000","last_change":"2024-04-17T10:03:38.256626+0000","last_active":"2024-04-17T10:03:39.950417+0000","last_peered":"2024-04-17T10:03:39.950417+0000","last_clean":"2024-04-17T10:03:39.950417+0000","last_became_active":"2024-04-17T10:03:38.256451+0000","last_became_peered":"2024-04-17T10:03:38.256451+0000","last_unstale":"2024-04-17T10:03:39.950417+0000","last_undegraded":"2024-04-17T10:03:39.950417+0000","last_fullsized":"2024-04-17T10:03:39.950417+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:08:42.534344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743712+0000","last_change":"2024-04-17T10:03:38.251236+0000","last_active":"2024-04-17T10:03:39.743712+0000","last_peered":"2024-04-17T10:03:39.743712+0000","last_clean":"2024-04-17T10:03:39.743712+0000","last_became_active":"2024-04-17T10:03:38.250794+0000","last_became_peered":"2024-04-17T10:03:38.250794+0000","last_unstale":"2024-04-17T10:03:39.743712+0000","last_undegraded":"2024-04-17T10:03:39.743712+0000","last_fullsized":"2024-04-17T10:03:39.743712+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:35:21.820720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248625+0000","last_change":"2024-04-17T10:03:38.246470+0000","last_active":"2024-04-17T10:03:39.248625+0000","last_peered":"2024-04-17T10:03:39.248625+0000","last_clean":"2024-04-17T10:03:39.248625+0000","last_became_active":"2024-04-17T10:03:38.246159+0000","last_became_peered":"2024-04-17T10:03:38.246159+0000","last_unstale":"2024-04-17T10:03:39.248625+0000","last_undegraded":"2024-04-17T10:03:39.248625+0000","last_fullsized":"2024-04-17T10:03:39.248625+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:38:53.396120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.252619+0000","last_change":"2024-04-17T10:03:38.251399+0000","last_active":"2024-04-17T10:03:39.252619+0000","last_peered":"2024-04-17T10:03:39.252619+0000","last_clean":"2024-04-17T10:03:39.252619+0000","last_became_active":"2024-04-17T10:03:38.251107+0000","last_became_peered":"2024-04-17T10:03:38.251107+0000","last_unstale":"2024-04-17T10:03:39.252619+0000","last_undegraded":"2024-04-17T10:03:39.252619+0000","last_fullsized":"2024-04-17T10:03:39.252619+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:43:29.151231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.259080+0000","last_change":"2024-04-17T10:03:38.254531+0000","last_active":"2024-04-17T10:03:38.259080+0000","last_peered":"2024-04-17T10:03:38.259080+0000","last_clean":"2024-04-17T10:03:38.259080+0000","last_became_active":"2024-04-17T10:03:38.254245+0000","last_became_peered":"2024-04-17T10:03:38.254245+0000","last_unstale":"2024-04-17T10:03:38.259080+0000","last_undegraded":"2024-04-17T10:03:38.259080+0000","last_fullsized":"2024-04-17T10:03:38.259080+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:30:29.968238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.950122+0000","last_change":"2024-04-17T10:03:38.253622+0000","last_active":"2024-04-17T10:03:39.950122+0000","last_peered":"2024-04-17T10:03:39.950122+0000","last_clean":"2024-04-17T10:03:39.950122+0000","last_became_active":"2024-04-17T10:03:38.253362+0000","last_became_peered":"2024-04-17T10:03:38.253362+0000","last_unstale":"2024-04-17T10:03:39.950122+0000","last_undegraded":"2024-04-17T10:03:39.950122+0000","last_fullsized":"2024-04-17T10:03:39.950122+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:51:15.865296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.950446+0000","last_change":"2024-04-17T10:03:38.268315+0000","last_active":"2024-04-17T10:03:39.950446+0000","last_peered":"2024-04-17T10:03:39.950446+0000","last_clean":"2024-04-17T10:03:39.950446+0000","last_became_active":"2024-04-17T10:03:38.268050+0000","last_became_peered":"2024-04-17T10:03:38.268050+0000","last_unstale":"2024-04-17T10:03:39.950446+0000","last_undegraded":"2024-04-17T10:03:39.950446+0000","last_fullsized":"2024-04-17T10:03:39.950446+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:06:41.194573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"108'4","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.252795+0000","last_change":"2024-04-17T10:03:37.232654+0000","last_active":"2024-04-17T10:03:39.252795+0000","last_peered":"2024-04-17T10:03:39.252795+0000","last_clean":"2024-04-17T10:03:39.252795+0000","last_became_active":"2024-04-17T10:03:37.232152+0000","last_became_peered":"2024-04-17T10:03:37.232152+0000","last_unstale":"2024-04-17T10:03:39.252795+0000","last_undegraded":"2024-04-17T10:03:39.252795+0000","last_fullsized":"2024-04-17T10:03:39.252795+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:08:28.143714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248869+0000","last_change":"2024-04-17T10:03:38.245365+0000","last_active":"2024-04-17T10:03:39.248869+0000","last_peered":"2024-04-17T10:03:39.248869+0000","last_clean":"2024-04-17T10:03:39.248869+0000","last_became_active":"2024-04-17T10:03:38.245043+0000","last_became_peered":"2024-04-17T10:03:38.245043+0000","last_unstale":"2024-04-17T10:03:39.248869+0000","last_undegraded":"2024-04-17T10:03:39.248869+0000","last_fullsized":"2024-04-17T10:03:39.248869+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:02:28.560561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.035752+0000","last_change":"2024-04-17T10:03:37.233688+0000","last_active":"2024-04-17T10:03:40.035752+0000","last_peered":"2024-04-17T10:03:40.035752+0000","last_clean":"2024-04-17T10:03:40.035752+0000","last_became_active":"2024-04-17T10:03:37.233444+0000","last_became_peered":"2024-04-17T10:03:37.233444+0000","last_unstale":"2024-04-17T10:03:40.035752+0000","last_undegraded":"2024-04-17T10:03:40.035752+0000","last_fullsized":"2024-04-17T10:03:40.035752+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:38.021524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.950391+0000","last_change":"2024-04-17T10:03:38.255263+0000","last_active":"2024-04-17T10:03:39.950391+0000","last_peered":"2024-04-17T10:03:39.950391+0000","last_clean":"2024-04-17T10:03:39.950391+0000","last_became_active":"2024-04-17T10:03:38.254431+0000","last_became_peered":"2024-04-17T10:03:38.254431+0000","last_unstale":"2024-04-17T10:03:39.950391+0000","last_undegraded":"2024-04-17T10:03:39.950391+0000","last_fullsized":"2024-04-17T10:03:39.950391+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:21:30.110101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249228+0000","last_change":"2024-04-17T10:03:37.239592+0000","last_active":"2024-04-17T10:03:39.249228+0000","last_peered":"2024-04-17T10:03:39.249228+0000","last_clean":"2024-04-17T10:03:39.249228+0000","last_became_active":"2024-04-17T10:03:37.239054+0000","last_became_peered":"2024-04-17T10:03:37.239054+0000","last_unstale":"2024-04-17T10:03:39.249228+0000","last_undegraded":"2024-04-17T10:03:39.249228+0000","last_fullsized":"2024-04-17T10:03:39.249228+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:12:20.347453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.035990+0000","last_change":"2024-04-17T10:03:38.255412+0000","last_active":"2024-04-17T10:03:40.035990+0000","last_peered":"2024-04-17T10:03:40.035990+0000","last_clean":"2024-04-17T10:03:40.035990+0000","last_became_active":"2024-04-17T10:03:38.254792+0000","last_became_peered":"2024-04-17T10:03:38.254792+0000","last_unstale":"2024-04-17T10:03:40.035990+0000","last_undegraded":"2024-04-17T10:03:40.035990+0000","last_fullsized":"2024-04-17T10:03:40.035990+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:28:52.125194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248995+0000","last_change":"2024-04-17T10:03:37.245295+0000","last_active":"2024-04-17T10:03:39.248995+0000","last_peered":"2024-04-17T10:03:39.248995+0000","last_clean":"2024-04-17T10:03:39.248995+0000","last_became_active":"2024-04-17T10:03:37.245002+0000","last_became_peered":"2024-04-17T10:03:37.245002+0000","last_unstale":"2024-04-17T10:03:39.248995+0000","last_undegraded":"2024-04-17T10:03:39.248995+0000","last_fullsized":"2024-04-17T10:03:39.248995+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:36:16.847890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.259850+0000","last_change":"2024-04-17T10:03:38.244171+0000","last_active":"2024-04-17T10:03:38.259850+0000","last_peered":"2024-04-17T10:03:38.259850+0000","last_clean":"2024-04-17T10:03:38.259850+0000","last_became_active":"2024-04-17T10:03:38.242809+0000","last_became_peered":"2024-04-17T10:03:38.242809+0000","last_unstale":"2024-04-17T10:03:38.259850+0000","last_undegraded":"2024-04-17T10:03:38.259850+0000","last_fullsized":"2024-04-17T10:03:38.259850+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:47.458921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.252421+0000","last_change":"2024-04-17T10:03:37.242282+0000","last_active":"2024-04-17T10:03:39.252421+0000","last_peered":"2024-04-17T10:03:39.252421+0000","last_clean":"2024-04-17T10:03:39.252421+0000","last_became_active":"2024-04-17T10:03:37.242048+0000","last_became_peered":"2024-04-17T10:03:37.242048+0000","last_unstale":"2024-04-17T10:03:39.252421+0000","last_undegraded":"2024-04-17T10:03:39.252421+0000","last_fullsized":"2024-04-17T10:03:39.252421+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:12:20.347453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248783+0000","last_change":"2024-04-17T10:03:38.243205+0000","last_active":"2024-04-17T10:03:39.248783+0000","last_peered":"2024-04-17T10:03:39.248783+0000","last_clean":"2024-04-17T10:03:39.248783+0000","last_became_active":"2024-04-17T10:03:38.242951+0000","last_became_peered":"2024-04-17T10:03:38.242951+0000","last_unstale":"2024-04-17T10:03:39.248783+0000","last_undegraded":"2024-04-17T10:03:39.248783+0000","last_fullsized":"2024-04-17T10:03:39.248783+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:14:58.265930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743572+0000","last_change":"2024-04-17T10:03:37.230409+0000","last_active":"2024-04-17T10:03:39.743572+0000","last_peered":"2024-04-17T10:03:39.743572+0000","last_clean":"2024-04-17T10:03:39.743572+0000","last_became_active":"2024-04-17T10:03:37.229938+0000","last_became_peered":"2024-04-17T10:03:37.229938+0000","last_unstale":"2024-04-17T10:03:39.743572+0000","last_undegraded":"2024-04-17T10:03:39.743572+0000","last_fullsized":"2024-04-17T10:03:39.743572+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:28:55.097624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.257116+0000","last_change":"2024-04-17T10:03:38.252087+0000","last_active":"2024-04-17T10:03:38.257116+0000","last_peered":"2024-04-17T10:03:38.257116+0000","last_clean":"2024-04-17T10:03:38.257116+0000","last_became_active":"2024-04-17T10:03:38.251786+0000","last_became_peered":"2024-04-17T10:03:38.251786+0000","last_unstale":"2024-04-17T10:03:38.257116+0000","last_undegraded":"2024-04-17T10:03:38.257116+0000","last_fullsized":"2024-04-17T10:03:38.257116+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:47:20.161956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248578+0000","last_change":"2024-04-17T10:03:37.236403+0000","last_active":"2024-04-17T10:03:39.248578+0000","last_peered":"2024-04-17T10:03:39.248578+0000","last_clean":"2024-04-17T10:03:39.248578+0000","last_became_active":"2024-04-17T10:03:37.236065+0000","last_became_peered":"2024-04-17T10:03:37.236065+0000","last_unstale":"2024-04-17T10:03:39.248578+0000","last_undegraded":"2024-04-17T10:03:39.248578+0000","last_fullsized":"2024-04-17T10:03:39.248578+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:36:01.630765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.252611+0000","last_change":"2024-04-17T10:03:38.251313+0000","last_active":"2024-04-17T10:03:39.252611+0000","last_peered":"2024-04-17T10:03:39.252611+0000","last_clean":"2024-04-17T10:03:39.252611+0000","last_became_active":"2024-04-17T10:03:38.251036+0000","last_became_peered":"2024-04-17T10:03:38.251036+0000","last_unstale":"2024-04-17T10:03:39.252611+0000","last_undegraded":"2024-04-17T10:03:39.252611+0000","last_fullsized":"2024-04-17T10:03:39.252611+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:56.070036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.233137+0000","last_change":"2024-04-17T10:03:37.240607+0000","last_active":"2024-04-17T10:03:38.233137+0000","last_peered":"2024-04-17T10:03:38.233137+0000","last_clean":"2024-04-17T10:03:38.233137+0000","last_became_active":"2024-04-17T10:03:37.240316+0000","last_became_peered":"2024-04-17T10:03:37.240316+0000","last_unstale":"2024-04-17T10:03:38.233137+0000","last_undegraded":"2024-04-17T10:03:38.233137+0000","last_fullsized":"2024-04-17T10:03:38.233137+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:18.787118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.252766+0000","last_change":"2024-04-17T10:03:38.267558+0000","last_active":"2024-04-17T10:03:39.252766+0000","last_peered":"2024-04-17T10:03:39.252766+0000","last_clean":"2024-04-17T10:03:39.252766+0000","last_became_active":"2024-04-17T10:03:38.267178+0000","last_became_peered":"2024-04-17T10:03:38.267178+0000","last_unstale":"2024-04-17T10:03:39.252766+0000","last_undegraded":"2024-04-17T10:03:39.252766+0000","last_fullsized":"2024-04-17T10:03:39.252766+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:51:15.865296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249223+0000","last_change":"2024-04-17T10:03:37.245656+0000","last_active":"2024-04-17T10:03:39.249223+0000","last_peered":"2024-04-17T10:03:39.249223+0000","last_clean":"2024-04-17T10:03:39.249223+0000","last_became_active":"2024-04-17T10:03:37.245110+0000","last_became_peered":"2024-04-17T10:03:37.245110+0000","last_unstale":"2024-04-17T10:03:39.249223+0000","last_undegraded":"2024-04-17T10:03:39.249223+0000","last_fullsized":"2024-04-17T10:03:39.249223+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:09:11.212786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249570+0000","last_change":"2024-04-17T10:03:38.252737+0000","last_active":"2024-04-17T10:03:39.249570+0000","last_peered":"2024-04-17T10:03:39.249570+0000","last_clean":"2024-04-17T10:03:39.249570+0000","last_became_active":"2024-04-17T10:03:38.252196+0000","last_became_peered":"2024-04-17T10:03:38.252196+0000","last_unstale":"2024-04-17T10:03:39.249570+0000","last_undegraded":"2024-04-17T10:03:39.249570+0000","last_fullsized":"2024-04-17T10:03:39.249570+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:51:08.063670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743676+0000","last_change":"2024-04-17T10:03:37.230141+0000","last_active":"2024-04-17T10:03:39.743676+0000","last_peered":"2024-04-17T10:03:39.743676+0000","last_clean":"2024-04-17T10:03:39.743676+0000","last_became_active":"2024-04-17T10:03:37.229031+0000","last_became_peered":"2024-04-17T10:03:37.229031+0000","last_unstale":"2024-04-17T10:03:39.743676+0000","last_undegraded":"2024-04-17T10:03:39.743676+0000","last_fullsized":"2024-04-17T10:03:39.743676+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:21:18.665522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743697+0000","last_change":"2024-04-17T10:03:38.246592+0000","last_active":"2024-04-17T10:03:39.743697+0000","last_peered":"2024-04-17T10:03:39.743697+0000","last_clean":"2024-04-17T10:03:39.743697+0000","last_became_active":"2024-04-17T10:03:38.246267+0000","last_became_peered":"2024-04-17T10:03:38.246267+0000","last_unstale":"2024-04-17T10:03:39.743697+0000","last_undegraded":"2024-04-17T10:03:39.743697+0000","last_fullsized":"2024-04-17T10:03:39.743697+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:17:36.152018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"108'3","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.271228+0000","last_change":"2024-04-17T10:03:37.242610+0000","last_active":"2024-04-17T10:03:38.271228+0000","last_peered":"2024-04-17T10:03:38.271228+0000","last_clean":"2024-04-17T10:03:38.271228+0000","last_became_active":"2024-04-17T10:03:37.242275+0000","last_became_peered":"2024-04-17T10:03:37.242275+0000","last_unstale":"2024-04-17T10:03:38.271228+0000","last_undegraded":"2024-04-17T10:03:38.271228+0000","last_fullsized":"2024-04-17T10:03:38.271228+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:33:19.570754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.252680+0000","last_change":"2024-04-17T10:03:38.248842+0000","last_active":"2024-04-17T10:03:39.252680+0000","last_peered":"2024-04-17T10:03:39.252680+0000","last_clean":"2024-04-17T10:03:39.252680+0000","last_became_active":"2024-04-17T10:03:38.248542+0000","last_became_peered":"2024-04-17T10:03:38.248542+0000","last_unstale":"2024-04-17T10:03:39.252680+0000","last_undegraded":"2024-04-17T10:03:39.252680+0000","last_fullsized":"2024-04-17T10:03:39.252680+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:21:46.569995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"108'6","reported_seq":18,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.278319+0000","last_change":"2024-04-17T10:03:37.239417+0000","last_active":"2024-04-17T10:03:38.278319+0000","last_peered":"2024-04-17T10:03:38.278319+0000","last_clean":"2024-04-17T10:03:38.278319+0000","last_became_active":"2024-04-17T10:03:37.239151+0000","last_became_peered":"2024-04-17T10:03:37.239151+0000","last_unstale":"2024-04-17T10:03:38.278319+0000","last_undegraded":"2024-04-17T10:03:38.278319+0000","last_fullsized":"2024-04-17T10:03:38.278319+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:41:51.899402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743519+0000","last_change":"2024-04-17T10:03:38.254700+0000","last_active":"2024-04-17T10:03:39.743519+0000","last_peered":"2024-04-17T10:03:39.743519+0000","last_clean":"2024-04-17T10:03:39.743519+0000","last_became_active":"2024-04-17T10:03:38.254243+0000","last_became_peered":"2024-04-17T10:03:38.254243+0000","last_unstale":"2024-04-17T10:03:39.743519+0000","last_undegraded":"2024-04-17T10:03:39.743519+0000","last_fullsized":"2024-04-17T10:03:39.743519+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:49:26.025529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.950163+0000","last_change":"2024-04-17T10:03:37.243002+0000","last_active":"2024-04-17T10:03:39.950163+0000","last_peered":"2024-04-17T10:03:39.950163+0000","last_clean":"2024-04-17T10:03:39.950163+0000","last_became_active":"2024-04-17T10:03:37.241917+0000","last_became_peered":"2024-04-17T10:03:37.241917+0000","last_unstale":"2024-04-17T10:03:39.950163+0000","last_undegraded":"2024-04-17T10:03:39.950163+0000","last_fullsized":"2024-04-17T10:03:39.950163+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:09:11.212786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743550+0000","last_change":"2024-04-17T10:03:38.251692+0000","last_active":"2024-04-17T10:03:39.743550+0000","last_peered":"2024-04-17T10:03:39.743550+0000","last_clean":"2024-04-17T10:03:39.743550+0000","last_became_active":"2024-04-17T10:03:38.251248+0000","last_became_peered":"2024-04-17T10:03:38.251248+0000","last_unstale":"2024-04-17T10:03:39.743550+0000","last_undegraded":"2024-04-17T10:03:39.743550+0000","last_fullsized":"2024-04-17T10:03:39.743550+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:07:33.201501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248867+0000","last_change":"2024-04-17T10:03:37.228382+0000","last_active":"2024-04-17T10:03:39.248867+0000","last_peered":"2024-04-17T10:03:39.248867+0000","last_clean":"2024-04-17T10:03:39.248867+0000","last_became_active":"2024-04-17T10:03:37.227982+0000","last_became_peered":"2024-04-17T10:03:37.227982+0000","last_unstale":"2024-04-17T10:03:39.248867+0000","last_undegraded":"2024-04-17T10:03:39.248867+0000","last_fullsized":"2024-04-17T10:03:39.248867+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:38.021524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743320+0000","last_change":"2024-04-17T10:03:38.244720+0000","last_active":"2024-04-17T10:03:39.743320+0000","last_peered":"2024-04-17T10:03:39.743320+0000","last_clean":"2024-04-17T10:03:39.743320+0000","last_became_active":"2024-04-17T10:03:38.243299+0000","last_became_peered":"2024-04-17T10:03:38.243299+0000","last_unstale":"2024-04-17T10:03:39.743320+0000","last_undegraded":"2024-04-17T10:03:39.743320+0000","last_fullsized":"2024-04-17T10:03:39.743320+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:43:25.504563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"108'8","reported_seq":22,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248915+0000","last_change":"2024-04-17T10:03:37.237198+0000","last_active":"2024-04-17T10:03:39.248915+0000","last_peered":"2024-04-17T10:03:39.248915+0000","last_clean":"2024-04-17T10:03:39.248915+0000","last_became_active":"2024-04-17T10:03:37.236898+0000","last_became_peered":"2024-04-17T10:03:37.236898+0000","last_unstale":"2024-04-17T10:03:39.248915+0000","last_undegraded":"2024-04-17T10:03:39.248915+0000","last_fullsized":"2024-04-17T10:03:39.248915+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:02:55.973258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249199+0000","last_change":"2024-04-17T10:03:38.262479+0000","last_active":"2024-04-17T10:03:39.249199+0000","last_peered":"2024-04-17T10:03:39.249199+0000","last_clean":"2024-04-17T10:03:39.249199+0000","last_became_active":"2024-04-17T10:03:38.262132+0000","last_became_peered":"2024-04-17T10:03:38.262132+0000","last_unstale":"2024-04-17T10:03:39.249199+0000","last_undegraded":"2024-04-17T10:03:39.249199+0000","last_fullsized":"2024-04-17T10:03:39.249199+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:40:44.504640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"108'1","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.269634+0000","last_change":"2024-04-17T10:03:37.235798+0000","last_active":"2024-04-17T10:03:38.269634+0000","last_peered":"2024-04-17T10:03:38.269634+0000","last_clean":"2024-04-17T10:03:38.269634+0000","last_became_active":"2024-04-17T10:03:37.234528+0000","last_became_peered":"2024-04-17T10:03:37.234528+0000","last_unstale":"2024-04-17T10:03:38.269634+0000","last_undegraded":"2024-04-17T10:03:38.269634+0000","last_fullsized":"2024-04-17T10:03:38.269634+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:54:44.214011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.950463+0000","last_change":"2024-04-17T10:03:38.258644+0000","last_active":"2024-04-17T10:03:39.950463+0000","last_peered":"2024-04-17T10:03:39.950463+0000","last_clean":"2024-04-17T10:03:39.950463+0000","last_became_active":"2024-04-17T10:03:38.258142+0000","last_became_peered":"2024-04-17T10:03:38.258142+0000","last_unstale":"2024-04-17T10:03:39.950463+0000","last_undegraded":"2024-04-17T10:03:39.950463+0000","last_fullsized":"2024-04-17T10:03:39.950463+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:08.816357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249264+0000","last_change":"2024-04-17T10:03:37.237574+0000","last_active":"2024-04-17T10:03:39.249264+0000","last_peered":"2024-04-17T10:03:39.249264+0000","last_clean":"2024-04-17T10:03:39.249264+0000","last_became_active":"2024-04-17T10:03:37.237251+0000","last_became_peered":"2024-04-17T10:03:37.237251+0000","last_unstale":"2024-04-17T10:03:39.249264+0000","last_undegraded":"2024-04-17T10:03:39.249264+0000","last_fullsized":"2024-04-17T10:03:39.249264+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:17:59.650782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.035853+0000","last_change":"2024-04-17T10:03:38.257666+0000","last_active":"2024-04-17T10:03:40.035853+0000","last_peered":"2024-04-17T10:03:40.035853+0000","last_clean":"2024-04-17T10:03:40.035853+0000","last_became_active":"2024-04-17T10:03:38.257338+0000","last_became_peered":"2024-04-17T10:03:38.257338+0000","last_unstale":"2024-04-17T10:03:40.035853+0000","last_undegraded":"2024-04-17T10:03:40.035853+0000","last_fullsized":"2024-04-17T10:03:40.035853+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:04:32.865505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249243+0000","last_change":"2024-04-17T10:03:37.236556+0000","last_active":"2024-04-17T10:03:39.249243+0000","last_peered":"2024-04-17T10:03:39.249243+0000","last_clean":"2024-04-17T10:03:39.249243+0000","last_became_active":"2024-04-17T10:03:37.236187+0000","last_became_peered":"2024-04-17T10:03:37.236187+0000","last_unstale":"2024-04-17T10:03:39.249243+0000","last_undegraded":"2024-04-17T10:03:39.249243+0000","last_fullsized":"2024-04-17T10:03:39.249243+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:09.258658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.036131+0000","last_change":"2024-04-17T10:03:38.253107+0000","last_active":"2024-04-17T10:03:40.036131+0000","last_peered":"2024-04-17T10:03:40.036131+0000","last_clean":"2024-04-17T10:03:40.036131+0000","last_became_active":"2024-04-17T10:03:38.252753+0000","last_became_peered":"2024-04-17T10:03:38.252753+0000","last_unstale":"2024-04-17T10:03:40.036131+0000","last_undegraded":"2024-04-17T10:03:40.036131+0000","last_fullsized":"2024-04-17T10:03:40.036131+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:07:33.201501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.252556+0000","last_change":"2024-04-17T10:03:37.239507+0000","last_active":"2024-04-17T10:03:39.252556+0000","last_peered":"2024-04-17T10:03:39.252556+0000","last_clean":"2024-04-17T10:03:39.252556+0000","last_became_active":"2024-04-17T10:03:37.239209+0000","last_became_peered":"2024-04-17T10:03:37.239209+0000","last_unstale":"2024-04-17T10:03:39.252556+0000","last_undegraded":"2024-04-17T10:03:39.252556+0000","last_fullsized":"2024-04-17T10:03:39.252556+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:09:18.401838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.035603+0000","last_change":"2024-04-17T10:03:38.253662+0000","last_active":"2024-04-17T10:03:40.035603+0000","last_peered":"2024-04-17T10:03:40.035603+0000","last_clean":"2024-04-17T10:03:40.035603+0000","last_became_active":"2024-04-17T10:03:38.253374+0000","last_became_peered":"2024-04-17T10:03:38.253374+0000","last_unstale":"2024-04-17T10:03:40.035603+0000","last_undegraded":"2024-04-17T10:03:40.035603+0000","last_fullsized":"2024-04-17T10:03:40.035603+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:09:00.962555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.950195+0000","last_change":"2024-04-17T10:03:37.246255+0000","last_active":"2024-04-17T10:03:39.950195+0000","last_peered":"2024-04-17T10:03:39.950195+0000","last_clean":"2024-04-17T10:03:39.950195+0000","last_became_active":"2024-04-17T10:03:37.245983+0000","last_became_peered":"2024-04-17T10:03:37.245983+0000","last_unstale":"2024-04-17T10:03:39.950195+0000","last_undegraded":"2024-04-17T10:03:39.950195+0000","last_fullsized":"2024-04-17T10:03:39.950195+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:17:59.650782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.255869+0000","last_change":"2024-04-17T10:03:38.247680+0000","last_active":"2024-04-17T10:03:38.255869+0000","last_peered":"2024-04-17T10:03:38.255869+0000","last_clean":"2024-04-17T10:03:38.255869+0000","last_became_active":"2024-04-17T10:03:38.247288+0000","last_became_peered":"2024-04-17T10:03:38.247288+0000","last_unstale":"2024-04-17T10:03:38.255869+0000","last_undegraded":"2024-04-17T10:03:38.255869+0000","last_fullsized":"2024-04-17T10:03:38.255869+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:42:29.599286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248857+0000","last_change":"2024-04-17T10:03:37.245202+0000","last_active":"2024-04-17T10:03:39.248857+0000","last_peered":"2024-04-17T10:03:39.248857+0000","last_clean":"2024-04-17T10:03:39.248857+0000","last_became_active":"2024-04-17T10:03:37.244890+0000","last_became_peered":"2024-04-17T10:03:37.244890+0000","last_unstale":"2024-04-17T10:03:39.248857+0000","last_undegraded":"2024-04-17T10:03:39.248857+0000","last_fullsized":"2024-04-17T10:03:39.248857+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:11:24.185185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.035558+0000","last_change":"2024-04-17T10:03:38.250351+0000","last_active":"2024-04-17T10:03:40.035558+0000","last_peered":"2024-04-17T10:03:40.035558+0000","last_clean":"2024-04-17T10:03:40.035558+0000","last_became_active":"2024-04-17T10:03:38.250026+0000","last_became_peered":"2024-04-17T10:03:38.250026+0000","last_unstale":"2024-04-17T10:03:40.035558+0000","last_undegraded":"2024-04-17T10:03:40.035558+0000","last_fullsized":"2024-04-17T10:03:40.035558+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:40:21.521207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.950233+0000","last_change":"2024-04-17T10:03:37.240462+0000","last_active":"2024-04-17T10:03:39.950233+0000","last_peered":"2024-04-17T10:03:39.950233+0000","last_clean":"2024-04-17T10:03:39.950233+0000","last_became_active":"2024-04-17T10:03:37.240202+0000","last_became_peered":"2024-04-17T10:03:37.240202+0000","last_unstale":"2024-04-17T10:03:39.950233+0000","last_undegraded":"2024-04-17T10:03:39.950233+0000","last_fullsized":"2024-04-17T10:03:39.950233+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:08:28.143714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743361+0000","last_change":"2024-04-17T10:03:38.251702+0000","last_active":"2024-04-17T10:03:39.743361+0000","last_peered":"2024-04-17T10:03:39.743361+0000","last_clean":"2024-04-17T10:03:39.743361+0000","last_became_active":"2024-04-17T10:03:38.250714+0000","last_became_peered":"2024-04-17T10:03:38.250714+0000","last_unstale":"2024-04-17T10:03:39.743361+0000","last_undegraded":"2024-04-17T10:03:39.743361+0000","last_fullsized":"2024-04-17T10:03:39.743361+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:04:32.865505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.233589+0000","last_change":"2024-04-17T10:03:37.238515+0000","last_active":"2024-04-17T10:03:38.233589+0000","last_peered":"2024-04-17T10:03:38.233589+0000","last_clean":"2024-04-17T10:03:38.233589+0000","last_became_active":"2024-04-17T10:03:37.238237+0000","last_became_peered":"2024-04-17T10:03:37.238237+0000","last_unstale":"2024-04-17T10:03:38.233589+0000","last_undegraded":"2024-04-17T10:03:38.233589+0000","last_fullsized":"2024-04-17T10:03:38.233589+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:42:33.549870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248812+0000","last_change":"2024-04-17T10:03:38.258042+0000","last_active":"2024-04-17T10:03:39.248812+0000","last_peered":"2024-04-17T10:03:39.248812+0000","last_clean":"2024-04-17T10:03:39.248812+0000","last_became_active":"2024-04-17T10:03:38.257665+0000","last_became_peered":"2024-04-17T10:03:38.257665+0000","last_unstale":"2024-04-17T10:03:39.248812+0000","last_undegraded":"2024-04-17T10:03:39.248812+0000","last_fullsized":"2024-04-17T10:03:39.248812+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:03:29.409571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743773+0000","last_change":"2024-04-17T10:03:37.238418+0000","last_active":"2024-04-17T10:03:39.743773+0000","last_peered":"2024-04-17T10:03:39.743773+0000","last_clean":"2024-04-17T10:03:39.743773+0000","last_became_active":"2024-04-17T10:03:37.233451+0000","last_became_peered":"2024-04-17T10:03:37.233451+0000","last_unstale":"2024-04-17T10:03:39.743773+0000","last_undegraded":"2024-04-17T10:03:39.743773+0000","last_fullsized":"2024-04-17T10:03:39.743773+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:36:43.078937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.259922+0000","last_change":"2024-04-17T10:03:38.254895+0000","last_active":"2024-04-17T10:03:38.259922+0000","last_peered":"2024-04-17T10:03:38.259922+0000","last_clean":"2024-04-17T10:03:38.259922+0000","last_became_active":"2024-04-17T10:03:38.250334+0000","last_became_peered":"2024-04-17T10:03:38.250334+0000","last_unstale":"2024-04-17T10:03:38.259922+0000","last_undegraded":"2024-04-17T10:03:38.259922+0000","last_fullsized":"2024-04-17T10:03:38.259922+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:10:52.153884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"109'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.251213+0000","last_change":"2024-04-17T10:03:37.232534+0000","last_active":"2024-04-17T10:03:39.251213+0000","last_peered":"2024-04-17T10:03:39.251213+0000","last_clean":"2024-04-17T10:03:39.251213+0000","last_became_active":"2024-04-17T10:03:37.232218+0000","last_became_peered":"2024-04-17T10:03:37.232218+0000","last_unstale":"2024-04-17T10:03:39.251213+0000","last_undegraded":"2024-04-17T10:03:39.251213+0000","last_fullsized":"2024-04-17T10:03:39.251213+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:28:55.097624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.261318+0000","last_change":"2024-04-17T10:03:38.254757+0000","last_active":"2024-04-17T10:03:38.261318+0000","last_peered":"2024-04-17T10:03:38.261318+0000","last_clean":"2024-04-17T10:03:38.261318+0000","last_became_active":"2024-04-17T10:03:38.254131+0000","last_became_peered":"2024-04-17T10:03:38.254131+0000","last_unstale":"2024-04-17T10:03:38.261318+0000","last_undegraded":"2024-04-17T10:03:38.261318+0000","last_fullsized":"2024-04-17T10:03:38.261318+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:00:04.333061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.234459+0000","last_change":"2024-04-17T10:03:37.237993+0000","last_active":"2024-04-17T10:03:38.234459+0000","last_peered":"2024-04-17T10:03:38.234459+0000","last_clean":"2024-04-17T10:03:38.234459+0000","last_became_active":"2024-04-17T10:03:37.237716+0000","last_became_peered":"2024-04-17T10:03:37.237716+0000","last_unstale":"2024-04-17T10:03:38.234459+0000","last_undegraded":"2024-04-17T10:03:38.234459+0000","last_fullsized":"2024-04-17T10:03:38.234459+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:08.897487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249180+0000","last_change":"2024-04-17T10:03:38.248687+0000","last_active":"2024-04-17T10:03:39.249180+0000","last_peered":"2024-04-17T10:03:39.249180+0000","last_clean":"2024-04-17T10:03:39.249180+0000","last_became_active":"2024-04-17T10:03:38.248330+0000","last_became_peered":"2024-04-17T10:03:38.248330+0000","last_unstale":"2024-04-17T10:03:39.249180+0000","last_undegraded":"2024-04-17T10:03:39.249180+0000","last_fullsized":"2024-04-17T10:03:39.249180+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:45:12.303860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743751+0000","last_change":"2024-04-17T10:03:37.237005+0000","last_active":"2024-04-17T10:03:39.743751+0000","last_peered":"2024-04-17T10:03:39.743751+0000","last_clean":"2024-04-17T10:03:39.743751+0000","last_became_active":"2024-04-17T10:03:37.236755+0000","last_became_peered":"2024-04-17T10:03:37.236755+0000","last_unstale":"2024-04-17T10:03:39.743751+0000","last_undegraded":"2024-04-17T10:03:39.743751+0000","last_fullsized":"2024-04-17T10:03:39.743751+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:16:32.705548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248419+0000","last_change":"2024-04-17T10:03:38.251135+0000","last_active":"2024-04-17T10:03:39.248419+0000","last_peered":"2024-04-17T10:03:39.248419+0000","last_clean":"2024-04-17T10:03:39.248419+0000","last_became_active":"2024-04-17T10:03:38.250878+0000","last_became_peered":"2024-04-17T10:03:38.250878+0000","last_unstale":"2024-04-17T10:03:39.248419+0000","last_undegraded":"2024-04-17T10:03:39.248419+0000","last_fullsized":"2024-04-17T10:03:39.248419+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:05:01.156946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.252598+0000","last_change":"2024-04-17T10:03:37.232899+0000","last_active":"2024-04-17T10:03:39.252598+0000","last_peered":"2024-04-17T10:03:39.252598+0000","last_clean":"2024-04-17T10:03:39.252598+0000","last_became_active":"2024-04-17T10:03:37.232637+0000","last_became_peered":"2024-04-17T10:03:37.232637+0000","last_unstale":"2024-04-17T10:03:39.252598+0000","last_undegraded":"2024-04-17T10:03:39.252598+0000","last_fullsized":"2024-04-17T10:03:39.252598+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:02:55.973258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248910+0000","last_change":"2024-04-17T10:03:38.254711+0000","last_active":"2024-04-17T10:03:39.248910+0000","last_peered":"2024-04-17T10:03:39.248910+0000","last_clean":"2024-04-17T10:03:39.248910+0000","last_became_active":"2024-04-17T10:03:38.252650+0000","last_became_peered":"2024-04-17T10:03:38.252650+0000","last_unstale":"2024-04-17T10:03:39.248910+0000","last_undegraded":"2024-04-17T10:03:39.248910+0000","last_fullsized":"2024-04-17T10:03:39.248910+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:19:08.029031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248400+0000","last_change":"2024-04-17T10:03:37.233986+0000","last_active":"2024-04-17T10:03:39.248400+0000","last_peered":"2024-04-17T10:03:39.248400+0000","last_clean":"2024-04-17T10:03:39.248400+0000","last_became_active":"2024-04-17T10:03:37.233746+0000","last_became_peered":"2024-04-17T10:03:37.233746+0000","last_unstale":"2024-04-17T10:03:39.248400+0000","last_undegraded":"2024-04-17T10:03:39.248400+0000","last_fullsized":"2024-04-17T10:03:39.248400+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:14:24.245875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.035796+0000","last_change":"2024-04-17T10:03:38.251661+0000","last_active":"2024-04-17T10:03:40.035796+0000","last_peered":"2024-04-17T10:03:40.035796+0000","last_clean":"2024-04-17T10:03:40.035796+0000","last_became_active":"2024-04-17T10:03:38.251311+0000","last_became_peered":"2024-04-17T10:03:38.251311+0000","last_unstale":"2024-04-17T10:03:40.035796+0000","last_undegraded":"2024-04-17T10:03:40.035796+0000","last_fullsized":"2024-04-17T10:03:40.035796+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:35:32.567676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248848+0000","last_change":"2024-04-17T10:03:37.245414+0000","last_active":"2024-04-17T10:03:39.248848+0000","last_peered":"2024-04-17T10:03:39.248848+0000","last_clean":"2024-04-17T10:03:39.248848+0000","last_became_active":"2024-04-17T10:03:37.245152+0000","last_became_peered":"2024-04-17T10:03:37.245152+0000","last_unstale":"2024-04-17T10:03:39.248848+0000","last_undegraded":"2024-04-17T10:03:39.248848+0000","last_fullsized":"2024-04-17T10:03:39.248848+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:08:28.143714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.259059+0000","last_change":"2024-04-17T10:03:38.251077+0000","last_active":"2024-04-17T10:03:38.259059+0000","last_peered":"2024-04-17T10:03:38.259059+0000","last_clean":"2024-04-17T10:03:38.259059+0000","last_became_active":"2024-04-17T10:03:38.250777+0000","last_became_peered":"2024-04-17T10:03:38.250777+0000","last_unstale":"2024-04-17T10:03:38.259059+0000","last_undegraded":"2024-04-17T10:03:38.259059+0000","last_fullsized":"2024-04-17T10:03:38.259059+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:10:48.549970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.233395+0000","last_change":"2024-04-17T10:03:37.238164+0000","last_active":"2024-04-17T10:03:38.233395+0000","last_peered":"2024-04-17T10:03:38.233395+0000","last_clean":"2024-04-17T10:03:38.233395+0000","last_became_active":"2024-04-17T10:03:37.237818+0000","last_became_peered":"2024-04-17T10:03:37.237818+0000","last_unstale":"2024-04-17T10:03:38.233395+0000","last_undegraded":"2024-04-17T10:03:38.233395+0000","last_fullsized":"2024-04-17T10:03:38.233395+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:31.528731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743407+0000","last_change":"2024-04-17T10:03:38.252456+0000","last_active":"2024-04-17T10:03:39.743407+0000","last_peered":"2024-04-17T10:03:39.743407+0000","last_clean":"2024-04-17T10:03:39.743407+0000","last_became_active":"2024-04-17T10:03:38.251750+0000","last_became_peered":"2024-04-17T10:03:38.251750+0000","last_unstale":"2024-04-17T10:03:39.743407+0000","last_undegraded":"2024-04-17T10:03:39.743407+0000","last_fullsized":"2024-04-17T10:03:39.743407+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:46:32.849083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.252761+0000","last_change":"2024-04-17T10:03:37.234733+0000","last_active":"2024-04-17T10:03:39.252761+0000","last_peered":"2024-04-17T10:03:39.252761+0000","last_clean":"2024-04-17T10:03:39.252761+0000","last_became_active":"2024-04-17T10:03:37.234471+0000","last_became_peered":"2024-04-17T10:03:37.234471+0000","last_unstale":"2024-04-17T10:03:39.252761+0000","last_undegraded":"2024-04-17T10:03:39.252761+0000","last_fullsized":"2024-04-17T10:03:39.252761+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:09:11.212786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":336,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.035907+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T10:03:40.035907+0000","last_peered":"2024-04-17T10:03:40.035907+0000","last_clean":"2024-04-17T10:03:40.035907+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T10:03:40.035907+0000","last_undegraded":"2024-04-17T10:03:40.035907+0000","last_fullsized":"2024-04-17T10:03:40.035907+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509571,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33072,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338768,"statfs":{"total":96636764160,"available":96602898432,"internally_reserved":0,"allocated":1937408,"data_stored":1167828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509572,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1351680,"data_stored":583214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509571,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1355776,"data_stored":583748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509571,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1351680,"data_stored":578526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509572,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1347584,"data_stored":583144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509571,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1355776,"data_stored":583268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542275,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1335296,"data_stored":576844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542275,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37248,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334592,"statfs":{"total":96636764160,"available":96598622208,"internally_reserved":0,"allocated":1949696,"data_stored":1168936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:03:42.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:03:42.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.543+0000 7fca99800640 1 Processor -- start 2024-04-17T10:03:42.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.543+0000 7fca99800640 1 -- start start 2024-04-17T10:03:42.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.547+0000 7fca99800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca94145f30 0x7fca94071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:42.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.547+0000 7fca99800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fca9415d8d0 0x7fca94072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:42.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.547+0000 7fca99800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fca940cce70 0x7fca940cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:42.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.547+0000 7fca99800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fca9406f400 con 0x7fca9415d8d0 2024-04-17T10:03:42.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.547+0000 7fca99800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fca94146310 con 0x7fca940cce70 2024-04-17T10:03:42.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.547+0000 7fca99800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fca9415d440 con 0x7fca94145f30 2024-04-17T10:03:42.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.547+0000 7fca8a000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fca9415d8d0 0x7fca94072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:42.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.547+0000 7fca8a000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fca9415d8d0 0x7fca94072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49306/0 (socket says 172.21.15.32:49306) 2024-04-17T10:03:42.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.547+0000 7fca8a000640 1 -- 172.21.15.32:0/2823087408 learned_addr learned my addr 172.21.15.32:0/2823087408 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:42.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.547+0000 7fca92a00640 1 --2- 172.21.15.32:0/2823087408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca94145f30 0x7fca94071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:42.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.547+0000 7fca93400640 1 --2- 172.21.15.32:0/2823087408 >> v2:172.21.15.144:3300/0 conn(0x7fca940cce70 0x7fca940cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:42.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.547+0000 7fca8a000640 1 -- 172.21.15.32:0/2823087408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca94145f30 msgr2=0x7fca94071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:42.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.547+0000 7fca8a000640 1 --2- 172.21.15.32:0/2823087408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca94145f30 0x7fca94071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.547+0000 7fca8a000640 1 -- 172.21.15.32:0/2823087408 >> v2:172.21.15.144:3300/0 conn(0x7fca940cce70 msgr2=0x7fca940cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:42.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.547+0000 7fca8a000640 1 --2- 172.21.15.32:0/2823087408 >> v2:172.21.15.144:3300/0 conn(0x7fca940cce70 0x7fca940cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.547+0000 7fca8a000640 1 -- 172.21.15.32:0/2823087408 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca940cd840 con 0x7fca9415d8d0 2024-04-17T10:03:42.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.547+0000 7fca92a00640 1 --2- 172.21.15.32:0/2823087408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca94145f30 0x7fca94071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:42.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.547+0000 7fca93400640 1 --2- 172.21.15.32:0/2823087408 >> v2:172.21.15.144:3300/0 conn(0x7fca940cce70 0x7fca940cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:42.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.551+0000 7fca8a000640 1 --2- 172.21.15.32:0/2823087408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca9415d8d0 0x7fca94072330 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fca940b5a10 tx=0x7fca7c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c9d753c189f94c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:42.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.551+0000 7fca92000640 1 -- 172.21.15.32:0/2823087408 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c0b5070 con 0x7fca9415d8d0 2024-04-17T10:03:42.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.551+0000 7fca92000640 1 -- 172.21.15.32:0/2823087408 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca7c0a89d0 con 0x7fca9415d8d0 2024-04-17T10:03:42.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.551+0000 7fca92000640 1 -- 172.21.15.32:0/2823087408 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c0a8da0 con 0x7fca9415d8d0 2024-04-17T10:03:42.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.551+0000 7fca99800640 1 -- 172.21.15.32:0/2823087408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca9415d8d0 msgr2=0x7fca94072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:42.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.551+0000 7fca99800640 1 --2- 172.21.15.32:0/2823087408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca9415d8d0 0x7fca94072330 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fca940b5a10 tx=0x7fca7c01ed10 comp rx=0 tx=0).stop 2024-04-17T10:03:42.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.551+0000 7fca99800640 1 -- 172.21.15.32:0/2823087408 shutdown_connections 2024-04-17T10:03:42.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.551+0000 7fca99800640 1 --2- 172.21.15.32:0/2823087408 >> v2:172.21.15.144:3300/0 conn(0x7fca940cce70 0x7fca940cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.551+0000 7fca99800640 1 --2- 172.21.15.32:0/2823087408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca9415d8d0 0x7fca94072330 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.551+0000 7fca99800640 1 --2- 172.21.15.32:0/2823087408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca94145f30 0x7fca94071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.551+0000 7fca99800640 1 -- 172.21.15.32:0/2823087408 >> 172.21.15.32:0/2823087408 conn(0x7fca9406ba30 msgr2=0x7fca9406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:42.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.551+0000 7fca99800640 1 -- 172.21.15.32:0/2823087408 shutdown_connections 2024-04-17T10:03:42.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.551+0000 7fca99800640 1 -- 172.21.15.32:0/2823087408 wait complete. 2024-04-17T10:03:42.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.551+0000 7fca99800640 1 Processor -- start 2024-04-17T10:03:42.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.551+0000 7fca99800640 1 -- start start 2024-04-17T10:03:42.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.551+0000 7fca99800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca940cce70 0x7fca941e6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:42.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.551+0000 7fca99800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca94145f30 0x7fca9415bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:42.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.551+0000 7fca99800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca9415d8d0 0x7fca94133450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:42.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.555+0000 7fca99800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fca9415cdf0 con 0x7fca940cce70 2024-04-17T10:03:42.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.555+0000 7fca99800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fca9415cb90 con 0x7fca9415d8d0 2024-04-17T10:03:42.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.555+0000 7fca99800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fca9415d0d0 con 0x7fca94145f30 2024-04-17T10:03:42.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.555+0000 7fca92a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca940cce70 0x7fca941e6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:42.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.555+0000 7fca92a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca940cce70 0x7fca941e6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49314/0 (socket says 172.21.15.32:49314) 2024-04-17T10:03:42.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.555+0000 7fca92a00640 1 -- 172.21.15.32:0/3444286256 learned_addr learned my addr 172.21.15.32:0/3444286256 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:42.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.555+0000 7fca93400640 1 --2- 172.21.15.32:0/3444286256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca9415d8d0 0x7fca94133450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:42.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.555+0000 7fca8a000640 1 --2- 172.21.15.32:0/3444286256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca94145f30 0x7fca9415bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:42.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.555+0000 7fca92a00640 1 -- 172.21.15.32:0/3444286256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca94145f30 msgr2=0x7fca9415bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:42.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.555+0000 7fca92a00640 1 --2- 172.21.15.32:0/3444286256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca94145f30 0x7fca9415bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.555+0000 7fca92a00640 1 -- 172.21.15.32:0/3444286256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca9415d8d0 msgr2=0x7fca94133450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:42.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.555+0000 7fca92a00640 1 --2- 172.21.15.32:0/3444286256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca9415d8d0 0x7fca94133450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.555+0000 7fca92a00640 1 -- 172.21.15.32:0/3444286256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca941e7500 con 0x7fca940cce70 2024-04-17T10:03:42.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.555+0000 7fca8a000640 1 --2- 172.21.15.32:0/3444286256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca94145f30 0x7fca9415bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:42.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.555+0000 7fca92a00640 1 --2- 172.21.15.32:0/3444286256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca940cce70 0x7fca941e6c80 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fca84009fc0 tx=0x7fca8400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:42.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.555+0000 7fca8b400640 1 -- 172.21.15.32:0/3444286256 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8401a070 con 0x7fca940cce70 2024-04-17T10:03:42.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.555+0000 7fca8b400640 1 -- 172.21.15.32:0/3444286256 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca84003bc0 con 0x7fca940cce70 2024-04-17T10:03:42.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.555+0000 7fca8b400640 1 -- 172.21.15.32:0/3444286256 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca84004030 con 0x7fca940cce70 2024-04-17T10:03:42.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.555+0000 7fca99800640 1 -- 172.21.15.32:0/3444286256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca941e77e0 con 0x7fca940cce70 2024-04-17T10:03:42.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.555+0000 7fca99800640 1 -- 172.21.15.32:0/3444286256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca941e71c0 con 0x7fca940cce70 2024-04-17T10:03:42.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.555+0000 7fca99800640 1 -- 172.21.15.32:0/3444286256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca941e7d00 con 0x7fca940cce70 2024-04-17T10:03:42.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.555+0000 7fca8b400640 1 -- 172.21.15.32:0/3444286256 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fca8401d070 con 0x7fca940cce70 2024-04-17T10:03:42.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.559+0000 7fca8b400640 1 --2- 172.21.15.32:0/3444286256 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fca70068380 0x7fca7006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:42.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.559+0000 7fca8b400640 1 -- 172.21.15.32:0/3444286256 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fca84086b50 con 0x7fca940cce70 2024-04-17T10:03:42.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.559+0000 7fca8a000640 1 --2- 172.21.15.32:0/3444286256 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fca70068380 0x7fca7006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:42.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.559+0000 7fca8a000640 1 --2- 172.21.15.32:0/3444286256 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fca70068380 0x7fca7006a840 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0b2000 tx=0x7fca7c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:42.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.563+0000 7fca8b400640 1 -- 172.21.15.32:0/3444286256 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fca84010070 con 0x7fca940cce70 2024-04-17T10:03:42.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.719+0000 7fca99800640 1 -- 172.21.15.32:0/3444286256 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fca94068a90 con 0x7fca70068380 2024-04-17T10:03:42.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.727+0000 7fca8b400640 1 -- 172.21.15.32:0/3444286256 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252467 (secure 0 0 0) 0x7fca94068a90 con 0x7fca70068380 2024-04-17T10:03:42.728 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:03:42.729 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T10:03:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.731+0000 7fca99800640 1 -- 172.21.15.32:0/3444286256 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fca70068380 msgr2=0x7fca7006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.731+0000 7fca99800640 1 --2- 172.21.15.32:0/3444286256 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fca70068380 0x7fca7006a840 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0b2000 tx=0x7fca7c008040 comp rx=0 tx=0).stop 2024-04-17T10:03:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.731+0000 7fca99800640 1 -- 172.21.15.32:0/3444286256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca940cce70 msgr2=0x7fca941e6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.731+0000 7fca99800640 1 --2- 172.21.15.32:0/3444286256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca940cce70 0x7fca941e6c80 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fca84009fc0 tx=0x7fca8400d4a0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.731+0000 7fca99800640 1 -- 172.21.15.32:0/3444286256 shutdown_connections 2024-04-17T10:03:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.731+0000 7fca99800640 1 --2- 172.21.15.32:0/3444286256 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fca70068380 0x7fca7006a840 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.731+0000 7fca99800640 1 --2- 172.21.15.32:0/3444286256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca94145f30 0x7fca9415bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.731+0000 7fca99800640 1 --2- 172.21.15.32:0/3444286256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca9415d8d0 0x7fca94133450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.731+0000 7fca99800640 1 --2- 172.21.15.32:0/3444286256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca940cce70 0x7fca941e6c80 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.731+0000 7fca99800640 1 -- 172.21.15.32:0/3444286256 >> 172.21.15.32:0/3444286256 conn(0x7fca9406ba30 msgr2=0x7fca9415ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.731+0000 7fca99800640 1 -- 172.21.15.32:0/3444286256 shutdown_connections 2024-04-17T10:03:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.731+0000 7fca99800640 1 -- 172.21.15.32:0/3444286256 wait complete. 2024-04-17T10:03:42.748 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-04-17T10:03:41.433341+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265344,"kb_used_data":11704,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754709376,"statfs":{"total":773094113280,"available":772822401024,"internally_reserved":0,"allocated":11984896,"data_stored":5825508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.221510"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.256233+0000","last_change":"2024-04-17T10:03:38.241978+0000","last_active":"2024-04-17T10:03:38.256233+0000","last_peered":"2024-04-17T10:03:38.256233+0000","last_clean":"2024-04-17T10:03:38.256233+0000","last_became_active":"2024-04-17T10:03:38.241715+0000","last_became_peered":"2024-04-17T10:03:38.241715+0000","last_unstale":"2024-04-17T10:03:38.256233+0000","last_undegraded":"2024-04-17T10:03:38.256233+0000","last_fullsized":"2024-04-17T10:03:38.256233+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:25.842390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249635+0000","last_change":"2024-04-17T10:03:38.242768+0000","last_active":"2024-04-17T10:03:39.249635+0000","last_peered":"2024-04-17T10:03:39.249635+0000","last_clean":"2024-04-17T10:03:39.249635+0000","last_became_active":"2024-04-17T10:03:38.242480+0000","last_became_peered":"2024-04-17T10:03:38.242480+0000","last_unstale":"2024-04-17T10:03:39.249635+0000","last_undegraded":"2024-04-17T10:03:39.249635+0000","last_fullsized":"2024-04-17T10:03:39.249635+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:38:00.474515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.035734+0000","last_change":"2024-04-17T10:03:38.253724+0000","last_active":"2024-04-17T10:03:40.035734+0000","last_peered":"2024-04-17T10:03:40.035734+0000","last_clean":"2024-04-17T10:03:40.035734+0000","last_became_active":"2024-04-17T10:03:38.253444+0000","last_became_peered":"2024-04-17T10:03:38.253444+0000","last_unstale":"2024-04-17T10:03:40.035734+0000","last_undegraded":"2024-04-17T10:03:40.035734+0000","last_fullsized":"2024-04-17T10:03:40.035734+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:43:25.504563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.261294+0000","last_change":"2024-04-17T10:03:38.254487+0000","last_active":"2024-04-17T10:03:38.261294+0000","last_peered":"2024-04-17T10:03:38.261294+0000","last_clean":"2024-04-17T10:03:38.261294+0000","last_became_active":"2024-04-17T10:03:38.253818+0000","last_became_peered":"2024-04-17T10:03:38.253818+0000","last_unstale":"2024-04-17T10:03:38.261294+0000","last_undegraded":"2024-04-17T10:03:38.261294+0000","last_fullsized":"2024-04-17T10:03:38.261294+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:09:28.208134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249129+0000","last_change":"2024-04-17T10:03:38.263442+0000","last_active":"2024-04-17T10:03:39.249129+0000","last_peered":"2024-04-17T10:03:39.249129+0000","last_clean":"2024-04-17T10:03:39.249129+0000","last_became_active":"2024-04-17T10:03:38.262179+0000","last_became_peered":"2024-04-17T10:03:38.262179+0000","last_unstale":"2024-04-17T10:03:39.249129+0000","last_undegraded":"2024-04-17T10:03:39.249129+0000","last_fullsized":"2024-04-17T10:03:39.249129+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:56.070036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.950205+0000","last_change":"2024-04-17T10:03:38.253899+0000","last_active":"2024-04-17T10:03:39.950205+0000","last_peered":"2024-04-17T10:03:39.950205+0000","last_clean":"2024-04-17T10:03:39.950205+0000","last_became_active":"2024-04-17T10:03:38.253644+0000","last_became_peered":"2024-04-17T10:03:38.253644+0000","last_unstale":"2024-04-17T10:03:39.950205+0000","last_undegraded":"2024-04-17T10:03:39.950205+0000","last_fullsized":"2024-04-17T10:03:39.950205+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:58:46.428563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.252551+0000","last_change":"2024-04-17T10:03:38.254343+0000","last_active":"2024-04-17T10:03:39.252551+0000","last_peered":"2024-04-17T10:03:39.252551+0000","last_clean":"2024-04-17T10:03:39.252551+0000","last_became_active":"2024-04-17T10:03:38.253868+0000","last_became_peered":"2024-04-17T10:03:38.253868+0000","last_unstale":"2024-04-17T10:03:39.252551+0000","last_undegraded":"2024-04-17T10:03:39.252551+0000","last_fullsized":"2024-04-17T10:03:39.252551+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:40:44.504640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.035484+0000","last_change":"2024-04-17T10:03:38.254947+0000","last_active":"2024-04-17T10:03:40.035484+0000","last_peered":"2024-04-17T10:03:40.035484+0000","last_clean":"2024-04-17T10:03:40.035484+0000","last_became_active":"2024-04-17T10:03:38.254681+0000","last_became_peered":"2024-04-17T10:03:38.254681+0000","last_unstale":"2024-04-17T10:03:40.035484+0000","last_undegraded":"2024-04-17T10:03:40.035484+0000","last_fullsized":"2024-04-17T10:03:40.035484+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:35.405701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.259940+0000","last_change":"2024-04-17T10:03:38.256106+0000","last_active":"2024-04-17T10:03:38.259940+0000","last_peered":"2024-04-17T10:03:38.259940+0000","last_clean":"2024-04-17T10:03:38.259940+0000","last_became_active":"2024-04-17T10:03:38.254920+0000","last_became_peered":"2024-04-17T10:03:38.254920+0000","last_unstale":"2024-04-17T10:03:38.259940+0000","last_undegraded":"2024-04-17T10:03:38.259940+0000","last_fullsized":"2024-04-17T10:03:38.259940+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:05:46.153121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.260934+0000","last_change":"2024-04-17T10:03:38.257961+0000","last_active":"2024-04-17T10:03:38.260934+0000","last_peered":"2024-04-17T10:03:38.260934+0000","last_clean":"2024-04-17T10:03:38.260934+0000","last_became_active":"2024-04-17T10:03:38.257615+0000","last_became_peered":"2024-04-17T10:03:38.257615+0000","last_unstale":"2024-04-17T10:03:38.260934+0000","last_undegraded":"2024-04-17T10:03:38.260934+0000","last_fullsized":"2024-04-17T10:03:38.260934+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:09.795387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249592+0000","last_change":"2024-04-17T10:03:38.241600+0000","last_active":"2024-04-17T10:03:39.249592+0000","last_peered":"2024-04-17T10:03:39.249592+0000","last_clean":"2024-04-17T10:03:39.249592+0000","last_became_active":"2024-04-17T10:03:38.241308+0000","last_became_peered":"2024-04-17T10:03:38.241308+0000","last_unstale":"2024-04-17T10:03:39.249592+0000","last_undegraded":"2024-04-17T10:03:39.249592+0000","last_fullsized":"2024-04-17T10:03:39.249592+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:58:23.683077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.261337+0000","last_change":"2024-04-17T10:03:38.256550+0000","last_active":"2024-04-17T10:03:38.261337+0000","last_peered":"2024-04-17T10:03:38.261337+0000","last_clean":"2024-04-17T10:03:38.261337+0000","last_became_active":"2024-04-17T10:03:38.256284+0000","last_became_peered":"2024-04-17T10:03:38.256284+0000","last_unstale":"2024-04-17T10:03:38.261337+0000","last_undegraded":"2024-04-17T10:03:38.261337+0000","last_fullsized":"2024-04-17T10:03:38.261337+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:25.387774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.035989+0000","last_change":"2024-04-17T10:03:38.250852+0000","last_active":"2024-04-17T10:03:40.035989+0000","last_peered":"2024-04-17T10:03:40.035989+0000","last_clean":"2024-04-17T10:03:40.035989+0000","last_became_active":"2024-04-17T10:03:38.249938+0000","last_became_peered":"2024-04-17T10:03:38.249938+0000","last_unstale":"2024-04-17T10:03:40.035989+0000","last_undegraded":"2024-04-17T10:03:40.035989+0000","last_fullsized":"2024-04-17T10:03:40.035989+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:46:32.849083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.035724+0000","last_change":"2024-04-17T10:03:38.253982+0000","last_active":"2024-04-17T10:03:40.035724+0000","last_peered":"2024-04-17T10:03:40.035724+0000","last_clean":"2024-04-17T10:03:40.035724+0000","last_became_active":"2024-04-17T10:03:38.252903+0000","last_became_peered":"2024-04-17T10:03:38.252903+0000","last_unstale":"2024-04-17T10:03:40.035724+0000","last_undegraded":"2024-04-17T10:03:40.035724+0000","last_fullsized":"2024-04-17T10:03:40.035724+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:08:40.728849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.950171+0000","last_change":"2024-04-17T10:03:38.251999+0000","last_active":"2024-04-17T10:03:39.950171+0000","last_peered":"2024-04-17T10:03:39.950171+0000","last_clean":"2024-04-17T10:03:39.950171+0000","last_became_active":"2024-04-17T10:03:38.251689+0000","last_became_peered":"2024-04-17T10:03:38.251689+0000","last_unstale":"2024-04-17T10:03:39.950171+0000","last_undegraded":"2024-04-17T10:03:39.950171+0000","last_fullsized":"2024-04-17T10:03:39.950171+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:04:44.056356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.257138+0000","last_change":"2024-04-17T10:03:38.252510+0000","last_active":"2024-04-17T10:03:38.257138+0000","last_peered":"2024-04-17T10:03:38.257138+0000","last_clean":"2024-04-17T10:03:38.257138+0000","last_became_active":"2024-04-17T10:03:38.250422+0000","last_became_peered":"2024-04-17T10:03:38.250422+0000","last_unstale":"2024-04-17T10:03:38.257138+0000","last_undegraded":"2024-04-17T10:03:38.257138+0000","last_fullsized":"2024-04-17T10:03:38.257138+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:50:09.412443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249899+0000","last_change":"2024-04-17T10:03:38.241946+0000","last_active":"2024-04-17T10:03:39.249899+0000","last_peered":"2024-04-17T10:03:39.249899+0000","last_clean":"2024-04-17T10:03:39.249899+0000","last_became_active":"2024-04-17T10:03:38.241561+0000","last_became_peered":"2024-04-17T10:03:38.241561+0000","last_unstale":"2024-04-17T10:03:39.249899+0000","last_undegraded":"2024-04-17T10:03:39.249899+0000","last_fullsized":"2024-04-17T10:03:39.249899+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:20:23.111219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248646+0000","last_change":"2024-04-17T10:03:38.242368+0000","last_active":"2024-04-17T10:03:39.248646+0000","last_peered":"2024-04-17T10:03:39.248646+0000","last_clean":"2024-04-17T10:03:39.248646+0000","last_became_active":"2024-04-17T10:03:38.242084+0000","last_became_peered":"2024-04-17T10:03:38.242084+0000","last_unstale":"2024-04-17T10:03:39.248646+0000","last_undegraded":"2024-04-17T10:03:39.248646+0000","last_fullsized":"2024-04-17T10:03:39.248646+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:27:02.945101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.036116+0000","last_change":"2024-04-17T10:03:38.255202+0000","last_active":"2024-04-17T10:03:40.036116+0000","last_peered":"2024-04-17T10:03:40.036116+0000","last_clean":"2024-04-17T10:03:40.036116+0000","last_became_active":"2024-04-17T10:03:38.254410+0000","last_became_peered":"2024-04-17T10:03:38.254410+0000","last_unstale":"2024-04-17T10:03:40.036116+0000","last_undegraded":"2024-04-17T10:03:40.036116+0000","last_fullsized":"2024-04-17T10:03:40.036116+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:49:26.025529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249463+0000","last_change":"2024-04-17T10:03:38.247529+0000","last_active":"2024-04-17T10:03:39.249463+0000","last_peered":"2024-04-17T10:03:39.249463+0000","last_clean":"2024-04-17T10:03:39.249463+0000","last_became_active":"2024-04-17T10:03:38.246836+0000","last_became_peered":"2024-04-17T10:03:38.246836+0000","last_unstale":"2024-04-17T10:03:39.249463+0000","last_undegraded":"2024-04-17T10:03:39.249463+0000","last_fullsized":"2024-04-17T10:03:39.249463+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:27:18.939260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249099+0000","last_change":"2024-04-17T10:03:38.243162+0000","last_active":"2024-04-17T10:03:39.249099+0000","last_peered":"2024-04-17T10:03:39.249099+0000","last_clean":"2024-04-17T10:03:39.249099+0000","last_became_active":"2024-04-17T10:03:38.241641+0000","last_became_peered":"2024-04-17T10:03:38.241641+0000","last_unstale":"2024-04-17T10:03:39.249099+0000","last_undegraded":"2024-04-17T10:03:39.249099+0000","last_fullsized":"2024-04-17T10:03:39.249099+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:52:10.948082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743440+0000","last_change":"2024-04-17T10:03:38.246858+0000","last_active":"2024-04-17T10:03:39.743440+0000","last_peered":"2024-04-17T10:03:39.743440+0000","last_clean":"2024-04-17T10:03:39.743440+0000","last_became_active":"2024-04-17T10:03:38.243321+0000","last_became_peered":"2024-04-17T10:03:38.243321+0000","last_unstale":"2024-04-17T10:03:39.743440+0000","last_undegraded":"2024-04-17T10:03:39.743440+0000","last_fullsized":"2024-04-17T10:03:39.743440+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:28:52.125194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248903+0000","last_change":"2024-04-17T10:03:38.248166+0000","last_active":"2024-04-17T10:03:39.248903+0000","last_peered":"2024-04-17T10:03:39.248903+0000","last_clean":"2024-04-17T10:03:39.248903+0000","last_became_active":"2024-04-17T10:03:38.246440+0000","last_became_peered":"2024-04-17T10:03:38.246440+0000","last_unstale":"2024-04-17T10:03:39.248903+0000","last_undegraded":"2024-04-17T10:03:39.248903+0000","last_fullsized":"2024-04-17T10:03:39.248903+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:42:54.852740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743343+0000","last_change":"2024-04-17T10:03:38.252625+0000","last_active":"2024-04-17T10:03:39.743343+0000","last_peered":"2024-04-17T10:03:39.743343+0000","last_clean":"2024-04-17T10:03:39.743343+0000","last_became_active":"2024-04-17T10:03:38.252197+0000","last_became_peered":"2024-04-17T10:03:38.252197+0000","last_unstale":"2024-04-17T10:03:39.743343+0000","last_undegraded":"2024-04-17T10:03:39.743343+0000","last_fullsized":"2024-04-17T10:03:39.743343+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:35:32.567676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249900+0000","last_change":"2024-04-17T10:03:38.241660+0000","last_active":"2024-04-17T10:03:39.249900+0000","last_peered":"2024-04-17T10:03:39.249900+0000","last_clean":"2024-04-17T10:03:39.249900+0000","last_became_active":"2024-04-17T10:03:38.240632+0000","last_became_peered":"2024-04-17T10:03:38.240632+0000","last_unstale":"2024-04-17T10:03:39.249900+0000","last_undegraded":"2024-04-17T10:03:39.249900+0000","last_fullsized":"2024-04-17T10:03:39.249900+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:23.356537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.258585+0000","last_change":"2024-04-17T10:03:38.251376+0000","last_active":"2024-04-17T10:03:38.258585+0000","last_peered":"2024-04-17T10:03:38.258585+0000","last_clean":"2024-04-17T10:03:38.258585+0000","last_became_active":"2024-04-17T10:03:38.251124+0000","last_became_peered":"2024-04-17T10:03:38.251124+0000","last_unstale":"2024-04-17T10:03:38.258585+0000","last_undegraded":"2024-04-17T10:03:38.258585+0000","last_fullsized":"2024-04-17T10:03:38.258585+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:59:12.444356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.950417+0000","last_change":"2024-04-17T10:03:38.256626+0000","last_active":"2024-04-17T10:03:39.950417+0000","last_peered":"2024-04-17T10:03:39.950417+0000","last_clean":"2024-04-17T10:03:39.950417+0000","last_became_active":"2024-04-17T10:03:38.256451+0000","last_became_peered":"2024-04-17T10:03:38.256451+0000","last_unstale":"2024-04-17T10:03:39.950417+0000","last_undegraded":"2024-04-17T10:03:39.950417+0000","last_fullsized":"2024-04-17T10:03:39.950417+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:08:42.534344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743712+0000","last_change":"2024-04-17T10:03:38.251236+0000","last_active":"2024-04-17T10:03:39.743712+0000","last_peered":"2024-04-17T10:03:39.743712+0000","last_clean":"2024-04-17T10:03:39.743712+0000","last_became_active":"2024-04-17T10:03:38.250794+0000","last_became_peered":"2024-04-17T10:03:38.250794+0000","last_unstale":"2024-04-17T10:03:39.743712+0000","last_undegraded":"2024-04-17T10:03:39.743712+0000","last_fullsized":"2024-04-17T10:03:39.743712+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:35:21.820720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248625+0000","last_change":"2024-04-17T10:03:38.246470+0000","last_active":"2024-04-17T10:03:39.248625+0000","last_peered":"2024-04-17T10:03:39.248625+0000","last_clean":"2024-04-17T10:03:39.248625+0000","last_became_active":"2024-04-17T10:03:38.246159+0000","last_became_peered":"2024-04-17T10:03:38.246159+0000","last_unstale":"2024-04-17T10:03:39.248625+0000","last_undegraded":"2024-04-17T10:03:39.248625+0000","last_fullsized":"2024-04-17T10:03:39.248625+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:38:53.396120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.252619+0000","last_change":"2024-04-17T10:03:38.251399+0000","last_active":"2024-04-17T10:03:39.252619+0000","last_peered":"2024-04-17T10:03:39.252619+0000","last_clean":"2024-04-17T10:03:39.252619+0000","last_became_active":"2024-04-17T10:03:38.251107+0000","last_became_peered":"2024-04-17T10:03:38.251107+0000","last_unstale":"2024-04-17T10:03:39.252619+0000","last_undegraded":"2024-04-17T10:03:39.252619+0000","last_fullsized":"2024-04-17T10:03:39.252619+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:43:29.151231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.259080+0000","last_change":"2024-04-17T10:03:38.254531+0000","last_active":"2024-04-17T10:03:38.259080+0000","last_peered":"2024-04-17T10:03:38.259080+0000","last_clean":"2024-04-17T10:03:38.259080+0000","last_became_active":"2024-04-17T10:03:38.254245+0000","last_became_peered":"2024-04-17T10:03:38.254245+0000","last_unstale":"2024-04-17T10:03:38.259080+0000","last_undegraded":"2024-04-17T10:03:38.259080+0000","last_fullsized":"2024-04-17T10:03:38.259080+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:30:29.968238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.950122+0000","last_change":"2024-04-17T10:03:38.253622+0000","last_active":"2024-04-17T10:03:39.950122+0000","last_peered":"2024-04-17T10:03:39.950122+0000","last_clean":"2024-04-17T10:03:39.950122+0000","last_became_active":"2024-04-17T10:03:38.253362+0000","last_became_peered":"2024-04-17T10:03:38.253362+0000","last_unstale":"2024-04-17T10:03:39.950122+0000","last_undegraded":"2024-04-17T10:03:39.950122+0000","last_fullsized":"2024-04-17T10:03:39.950122+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:51:15.865296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.950446+0000","last_change":"2024-04-17T10:03:38.268315+0000","last_active":"2024-04-17T10:03:39.950446+0000","last_peered":"2024-04-17T10:03:39.950446+0000","last_clean":"2024-04-17T10:03:39.950446+0000","last_became_active":"2024-04-17T10:03:38.268050+0000","last_became_peered":"2024-04-17T10:03:38.268050+0000","last_unstale":"2024-04-17T10:03:39.950446+0000","last_undegraded":"2024-04-17T10:03:39.950446+0000","last_fullsized":"2024-04-17T10:03:39.950446+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:06:41.194573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"108'4","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.252795+0000","last_change":"2024-04-17T10:03:37.232654+0000","last_active":"2024-04-17T10:03:39.252795+0000","last_peered":"2024-04-17T10:03:39.252795+0000","last_clean":"2024-04-17T10:03:39.252795+0000","last_became_active":"2024-04-17T10:03:37.232152+0000","last_became_peered":"2024-04-17T10:03:37.232152+0000","last_unstale":"2024-04-17T10:03:39.252795+0000","last_undegraded":"2024-04-17T10:03:39.252795+0000","last_fullsized":"2024-04-17T10:03:39.252795+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:08:28.143714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248869+0000","last_change":"2024-04-17T10:03:38.245365+0000","last_active":"2024-04-17T10:03:39.248869+0000","last_peered":"2024-04-17T10:03:39.248869+0000","last_clean":"2024-04-17T10:03:39.248869+0000","last_became_active":"2024-04-17T10:03:38.245043+0000","last_became_peered":"2024-04-17T10:03:38.245043+0000","last_unstale":"2024-04-17T10:03:39.248869+0000","last_undegraded":"2024-04-17T10:03:39.248869+0000","last_fullsized":"2024-04-17T10:03:39.248869+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:02:28.560561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.035752+0000","last_change":"2024-04-17T10:03:37.233688+0000","last_active":"2024-04-17T10:03:40.035752+0000","last_peered":"2024-04-17T10:03:40.035752+0000","last_clean":"2024-04-17T10:03:40.035752+0000","last_became_active":"2024-04-17T10:03:37.233444+0000","last_became_peered":"2024-04-17T10:03:37.233444+0000","last_unstale":"2024-04-17T10:03:40.035752+0000","last_undegraded":"2024-04-17T10:03:40.035752+0000","last_fullsized":"2024-04-17T10:03:40.035752+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:38.021524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.950391+0000","last_change":"2024-04-17T10:03:38.255263+0000","last_active":"2024-04-17T10:03:39.950391+0000","last_peered":"2024-04-17T10:03:39.950391+0000","last_clean":"2024-04-17T10:03:39.950391+0000","last_became_active":"2024-04-17T10:03:38.254431+0000","last_became_peered":"2024-04-17T10:03:38.254431+0000","last_unstale":"2024-04-17T10:03:39.950391+0000","last_undegraded":"2024-04-17T10:03:39.950391+0000","last_fullsized":"2024-04-17T10:03:39.950391+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:21:30.110101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249228+0000","last_change":"2024-04-17T10:03:37.239592+0000","last_active":"2024-04-17T10:03:39.249228+0000","last_peered":"2024-04-17T10:03:39.249228+0000","last_clean":"2024-04-17T10:03:39.249228+0000","last_became_active":"2024-04-17T10:03:37.239054+0000","last_became_peered":"2024-04-17T10:03:37.239054+0000","last_unstale":"2024-04-17T10:03:39.249228+0000","last_undegraded":"2024-04-17T10:03:39.249228+0000","last_fullsized":"2024-04-17T10:03:39.249228+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:12:20.347453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.035990+0000","last_change":"2024-04-17T10:03:38.255412+0000","last_active":"2024-04-17T10:03:40.035990+0000","last_peered":"2024-04-17T10:03:40.035990+0000","last_clean":"2024-04-17T10:03:40.035990+0000","last_became_active":"2024-04-17T10:03:38.254792+0000","last_became_peered":"2024-04-17T10:03:38.254792+0000","last_unstale":"2024-04-17T10:03:40.035990+0000","last_undegraded":"2024-04-17T10:03:40.035990+0000","last_fullsized":"2024-04-17T10:03:40.035990+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:28:52.125194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248995+0000","last_change":"2024-04-17T10:03:37.245295+0000","last_active":"2024-04-17T10:03:39.248995+0000","last_peered":"2024-04-17T10:03:39.248995+0000","last_clean":"2024-04-17T10:03:39.248995+0000","last_became_active":"2024-04-17T10:03:37.245002+0000","last_became_peered":"2024-04-17T10:03:37.245002+0000","last_unstale":"2024-04-17T10:03:39.248995+0000","last_undegraded":"2024-04-17T10:03:39.248995+0000","last_fullsized":"2024-04-17T10:03:39.248995+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:36:16.847890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.259850+0000","last_change":"2024-04-17T10:03:38.244171+0000","last_active":"2024-04-17T10:03:38.259850+0000","last_peered":"2024-04-17T10:03:38.259850+0000","last_clean":"2024-04-17T10:03:38.259850+0000","last_became_active":"2024-04-17T10:03:38.242809+0000","last_became_peered":"2024-04-17T10:03:38.242809+0000","last_unstale":"2024-04-17T10:03:38.259850+0000","last_undegraded":"2024-04-17T10:03:38.259850+0000","last_fullsized":"2024-04-17T10:03:38.259850+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:47.458921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.252421+0000","last_change":"2024-04-17T10:03:37.242282+0000","last_active":"2024-04-17T10:03:39.252421+0000","last_peered":"2024-04-17T10:03:39.252421+0000","last_clean":"2024-04-17T10:03:39.252421+0000","last_became_active":"2024-04-17T10:03:37.242048+0000","last_became_peered":"2024-04-17T10:03:37.242048+0000","last_unstale":"2024-04-17T10:03:39.252421+0000","last_undegraded":"2024-04-17T10:03:39.252421+0000","last_fullsized":"2024-04-17T10:03:39.252421+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:12:20.347453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248783+0000","last_change":"2024-04-17T10:03:38.243205+0000","last_active":"2024-04-17T10:03:39.248783+0000","last_peered":"2024-04-17T10:03:39.248783+0000","last_clean":"2024-04-17T10:03:39.248783+0000","last_became_active":"2024-04-17T10:03:38.242951+0000","last_became_peered":"2024-04-17T10:03:38.242951+0000","last_unstale":"2024-04-17T10:03:39.248783+0000","last_undegraded":"2024-04-17T10:03:39.248783+0000","last_fullsized":"2024-04-17T10:03:39.248783+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:14:58.265930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743572+0000","last_change":"2024-04-17T10:03:37.230409+0000","last_active":"2024-04-17T10:03:39.743572+0000","last_peered":"2024-04-17T10:03:39.743572+0000","last_clean":"2024-04-17T10:03:39.743572+0000","last_became_active":"2024-04-17T10:03:37.229938+0000","last_became_peered":"2024-04-17T10:03:37.229938+0000","last_unstale":"2024-04-17T10:03:39.743572+0000","last_undegraded":"2024-04-17T10:03:39.743572+0000","last_fullsized":"2024-04-17T10:03:39.743572+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:28:55.097624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.257116+0000","last_change":"2024-04-17T10:03:38.252087+0000","last_active":"2024-04-17T10:03:38.257116+0000","last_peered":"2024-04-17T10:03:38.257116+0000","last_clean":"2024-04-17T10:03:38.257116+0000","last_became_active":"2024-04-17T10:03:38.251786+0000","last_became_peered":"2024-04-17T10:03:38.251786+0000","last_unstale":"2024-04-17T10:03:38.257116+0000","last_undegraded":"2024-04-17T10:03:38.257116+0000","last_fullsized":"2024-04-17T10:03:38.257116+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:47:20.161956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248578+0000","last_change":"2024-04-17T10:03:37.236403+0000","last_active":"2024-04-17T10:03:39.248578+0000","last_peered":"2024-04-17T10:03:39.248578+0000","last_clean":"2024-04-17T10:03:39.248578+0000","last_became_active":"2024-04-17T10:03:37.236065+0000","last_became_peered":"2024-04-17T10:03:37.236065+0000","last_unstale":"2024-04-17T10:03:39.248578+0000","last_undegraded":"2024-04-17T10:03:39.248578+0000","last_fullsized":"2024-04-17T10:03:39.248578+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:36:01.630765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.252611+0000","last_change":"2024-04-17T10:03:38.251313+0000","last_active":"2024-04-17T10:03:39.252611+0000","last_peered":"2024-04-17T10:03:39.252611+0000","last_clean":"2024-04-17T10:03:39.252611+0000","last_became_active":"2024-04-17T10:03:38.251036+0000","last_became_peered":"2024-04-17T10:03:38.251036+0000","last_unstale":"2024-04-17T10:03:39.252611+0000","last_undegraded":"2024-04-17T10:03:39.252611+0000","last_fullsized":"2024-04-17T10:03:39.252611+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:56.070036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.233137+0000","last_change":"2024-04-17T10:03:37.240607+0000","last_active":"2024-04-17T10:03:38.233137+0000","last_peered":"2024-04-17T10:03:38.233137+0000","last_clean":"2024-04-17T10:03:38.233137+0000","last_became_active":"2024-04-17T10:03:37.240316+0000","last_became_peered":"2024-04-17T10:03:37.240316+0000","last_unstale":"2024-04-17T10:03:38.233137+0000","last_undegraded":"2024-04-17T10:03:38.233137+0000","last_fullsized":"2024-04-17T10:03:38.233137+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:18.787118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.252766+0000","last_change":"2024-04-17T10:03:38.267558+0000","last_active":"2024-04-17T10:03:39.252766+0000","last_peered":"2024-04-17T10:03:39.252766+0000","last_clean":"2024-04-17T10:03:39.252766+0000","last_became_active":"2024-04-17T10:03:38.267178+0000","last_became_peered":"2024-04-17T10:03:38.267178+0000","last_unstale":"2024-04-17T10:03:39.252766+0000","last_undegraded":"2024-04-17T10:03:39.252766+0000","last_fullsized":"2024-04-17T10:03:39.252766+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:51:15.865296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249223+0000","last_change":"2024-04-17T10:03:37.245656+0000","last_active":"2024-04-17T10:03:39.249223+0000","last_peered":"2024-04-17T10:03:39.249223+0000","last_clean":"2024-04-17T10:03:39.249223+0000","last_became_active":"2024-04-17T10:03:37.245110+0000","last_became_peered":"2024-04-17T10:03:37.245110+0000","last_unstale":"2024-04-17T10:03:39.249223+0000","last_undegraded":"2024-04-17T10:03:39.249223+0000","last_fullsized":"2024-04-17T10:03:39.249223+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:09:11.212786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249570+0000","last_change":"2024-04-17T10:03:38.252737+0000","last_active":"2024-04-17T10:03:39.249570+0000","last_peered":"2024-04-17T10:03:39.249570+0000","last_clean":"2024-04-17T10:03:39.249570+0000","last_became_active":"2024-04-17T10:03:38.252196+0000","last_became_peered":"2024-04-17T10:03:38.252196+0000","last_unstale":"2024-04-17T10:03:39.249570+0000","last_undegraded":"2024-04-17T10:03:39.249570+0000","last_fullsized":"2024-04-17T10:03:39.249570+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:51:08.063670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743676+0000","last_change":"2024-04-17T10:03:37.230141+0000","last_active":"2024-04-17T10:03:39.743676+0000","last_peered":"2024-04-17T10:03:39.743676+0000","last_clean":"2024-04-17T10:03:39.743676+0000","last_became_active":"2024-04-17T10:03:37.229031+0000","last_became_peered":"2024-04-17T10:03:37.229031+0000","last_unstale":"2024-04-17T10:03:39.743676+0000","last_undegraded":"2024-04-17T10:03:39.743676+0000","last_fullsized":"2024-04-17T10:03:39.743676+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:21:18.665522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743697+0000","last_change":"2024-04-17T10:03:38.246592+0000","last_active":"2024-04-17T10:03:39.743697+0000","last_peered":"2024-04-17T10:03:39.743697+0000","last_clean":"2024-04-17T10:03:39.743697+0000","last_became_active":"2024-04-17T10:03:38.246267+0000","last_became_peered":"2024-04-17T10:03:38.246267+0000","last_unstale":"2024-04-17T10:03:39.743697+0000","last_undegraded":"2024-04-17T10:03:39.743697+0000","last_fullsized":"2024-04-17T10:03:39.743697+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:17:36.152018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"108'3","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.271228+0000","last_change":"2024-04-17T10:03:37.242610+0000","last_active":"2024-04-17T10:03:38.271228+0000","last_peered":"2024-04-17T10:03:38.271228+0000","last_clean":"2024-04-17T10:03:38.271228+0000","last_became_active":"2024-04-17T10:03:37.242275+0000","last_became_peered":"2024-04-17T10:03:37.242275+0000","last_unstale":"2024-04-17T10:03:38.271228+0000","last_undegraded":"2024-04-17T10:03:38.271228+0000","last_fullsized":"2024-04-17T10:03:38.271228+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:33:19.570754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.252680+0000","last_change":"2024-04-17T10:03:38.248842+0000","last_active":"2024-04-17T10:03:39.252680+0000","last_peered":"2024-04-17T10:03:39.252680+0000","last_clean":"2024-04-17T10:03:39.252680+0000","last_became_active":"2024-04-17T10:03:38.248542+0000","last_became_peered":"2024-04-17T10:03:38.248542+0000","last_unstale":"2024-04-17T10:03:39.252680+0000","last_undegraded":"2024-04-17T10:03:39.252680+0000","last_fullsized":"2024-04-17T10:03:39.252680+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:21:46.569995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"108'6","reported_seq":18,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.278319+0000","last_change":"2024-04-17T10:03:37.239417+0000","last_active":"2024-04-17T10:03:38.278319+0000","last_peered":"2024-04-17T10:03:38.278319+0000","last_clean":"2024-04-17T10:03:38.278319+0000","last_became_active":"2024-04-17T10:03:37.239151+0000","last_became_peered":"2024-04-17T10:03:37.239151+0000","last_unstale":"2024-04-17T10:03:38.278319+0000","last_undegraded":"2024-04-17T10:03:38.278319+0000","last_fullsized":"2024-04-17T10:03:38.278319+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:41:51.899402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743519+0000","last_change":"2024-04-17T10:03:38.254700+0000","last_active":"2024-04-17T10:03:39.743519+0000","last_peered":"2024-04-17T10:03:39.743519+0000","last_clean":"2024-04-17T10:03:39.743519+0000","last_became_active":"2024-04-17T10:03:38.254243+0000","last_became_peered":"2024-04-17T10:03:38.254243+0000","last_unstale":"2024-04-17T10:03:39.743519+0000","last_undegraded":"2024-04-17T10:03:39.743519+0000","last_fullsized":"2024-04-17T10:03:39.743519+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:49:26.025529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.950163+0000","last_change":"2024-04-17T10:03:37.243002+0000","last_active":"2024-04-17T10:03:39.950163+0000","last_peered":"2024-04-17T10:03:39.950163+0000","last_clean":"2024-04-17T10:03:39.950163+0000","last_became_active":"2024-04-17T10:03:37.241917+0000","last_became_peered":"2024-04-17T10:03:37.241917+0000","last_unstale":"2024-04-17T10:03:39.950163+0000","last_undegraded":"2024-04-17T10:03:39.950163+0000","last_fullsized":"2024-04-17T10:03:39.950163+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:09:11.212786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743550+0000","last_change":"2024-04-17T10:03:38.251692+0000","last_active":"2024-04-17T10:03:39.743550+0000","last_peered":"2024-04-17T10:03:39.743550+0000","last_clean":"2024-04-17T10:03:39.743550+0000","last_became_active":"2024-04-17T10:03:38.251248+0000","last_became_peered":"2024-04-17T10:03:38.251248+0000","last_unstale":"2024-04-17T10:03:39.743550+0000","last_undegraded":"2024-04-17T10:03:39.743550+0000","last_fullsized":"2024-04-17T10:03:39.743550+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:07:33.201501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248867+0000","last_change":"2024-04-17T10:03:37.228382+0000","last_active":"2024-04-17T10:03:39.248867+0000","last_peered":"2024-04-17T10:03:39.248867+0000","last_clean":"2024-04-17T10:03:39.248867+0000","last_became_active":"2024-04-17T10:03:37.227982+0000","last_became_peered":"2024-04-17T10:03:37.227982+0000","last_unstale":"2024-04-17T10:03:39.248867+0000","last_undegraded":"2024-04-17T10:03:39.248867+0000","last_fullsized":"2024-04-17T10:03:39.248867+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:38.021524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743320+0000","last_change":"2024-04-17T10:03:38.244720+0000","last_active":"2024-04-17T10:03:39.743320+0000","last_peered":"2024-04-17T10:03:39.743320+0000","last_clean":"2024-04-17T10:03:39.743320+0000","last_became_active":"2024-04-17T10:03:38.243299+0000","last_became_peered":"2024-04-17T10:03:38.243299+0000","last_unstale":"2024-04-17T10:03:39.743320+0000","last_undegraded":"2024-04-17T10:03:39.743320+0000","last_fullsized":"2024-04-17T10:03:39.743320+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:43:25.504563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"108'8","reported_seq":22,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248915+0000","last_change":"2024-04-17T10:03:37.237198+0000","last_active":"2024-04-17T10:03:39.248915+0000","last_peered":"2024-04-17T10:03:39.248915+0000","last_clean":"2024-04-17T10:03:39.248915+0000","last_became_active":"2024-04-17T10:03:37.236898+0000","last_became_peered":"2024-04-17T10:03:37.236898+0000","last_unstale":"2024-04-17T10:03:39.248915+0000","last_undegraded":"2024-04-17T10:03:39.248915+0000","last_fullsized":"2024-04-17T10:03:39.248915+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:02:55.973258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249199+0000","last_change":"2024-04-17T10:03:38.262479+0000","last_active":"2024-04-17T10:03:39.249199+0000","last_peered":"2024-04-17T10:03:39.249199+0000","last_clean":"2024-04-17T10:03:39.249199+0000","last_became_active":"2024-04-17T10:03:38.262132+0000","last_became_peered":"2024-04-17T10:03:38.262132+0000","last_unstale":"2024-04-17T10:03:39.249199+0000","last_undegraded":"2024-04-17T10:03:39.249199+0000","last_fullsized":"2024-04-17T10:03:39.249199+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T21:40:44.504640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"108'1","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.269634+0000","last_change":"2024-04-17T10:03:37.235798+0000","last_active":"2024-04-17T10:03:38.269634+0000","last_peered":"2024-04-17T10:03:38.269634+0000","last_clean":"2024-04-17T10:03:38.269634+0000","last_became_active":"2024-04-17T10:03:37.234528+0000","last_became_peered":"2024-04-17T10:03:37.234528+0000","last_unstale":"2024-04-17T10:03:38.269634+0000","last_undegraded":"2024-04-17T10:03:38.269634+0000","last_fullsized":"2024-04-17T10:03:38.269634+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:54:44.214011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.950463+0000","last_change":"2024-04-17T10:03:38.258644+0000","last_active":"2024-04-17T10:03:39.950463+0000","last_peered":"2024-04-17T10:03:39.950463+0000","last_clean":"2024-04-17T10:03:39.950463+0000","last_became_active":"2024-04-17T10:03:38.258142+0000","last_became_peered":"2024-04-17T10:03:38.258142+0000","last_unstale":"2024-04-17T10:03:39.950463+0000","last_undegraded":"2024-04-17T10:03:39.950463+0000","last_fullsized":"2024-04-17T10:03:39.950463+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:08.816357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249264+0000","last_change":"2024-04-17T10:03:37.237574+0000","last_active":"2024-04-17T10:03:39.249264+0000","last_peered":"2024-04-17T10:03:39.249264+0000","last_clean":"2024-04-17T10:03:39.249264+0000","last_became_active":"2024-04-17T10:03:37.237251+0000","last_became_peered":"2024-04-17T10:03:37.237251+0000","last_unstale":"2024-04-17T10:03:39.249264+0000","last_undegraded":"2024-04-17T10:03:39.249264+0000","last_fullsized":"2024-04-17T10:03:39.249264+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:17:59.650782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.035853+0000","last_change":"2024-04-17T10:03:38.257666+0000","last_active":"2024-04-17T10:03:40.035853+0000","last_peered":"2024-04-17T10:03:40.035853+0000","last_clean":"2024-04-17T10:03:40.035853+0000","last_became_active":"2024-04-17T10:03:38.257338+0000","last_became_peered":"2024-04-17T10:03:38.257338+0000","last_unstale":"2024-04-17T10:03:40.035853+0000","last_undegraded":"2024-04-17T10:03:40.035853+0000","last_fullsized":"2024-04-17T10:03:40.035853+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:04:32.865505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249243+0000","last_change":"2024-04-17T10:03:37.236556+0000","last_active":"2024-04-17T10:03:39.249243+0000","last_peered":"2024-04-17T10:03:39.249243+0000","last_clean":"2024-04-17T10:03:39.249243+0000","last_became_active":"2024-04-17T10:03:37.236187+0000","last_became_peered":"2024-04-17T10:03:37.236187+0000","last_unstale":"2024-04-17T10:03:39.249243+0000","last_undegraded":"2024-04-17T10:03:39.249243+0000","last_fullsized":"2024-04-17T10:03:39.249243+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T22:02:09.258658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.036131+0000","last_change":"2024-04-17T10:03:38.253107+0000","last_active":"2024-04-17T10:03:40.036131+0000","last_peered":"2024-04-17T10:03:40.036131+0000","last_clean":"2024-04-17T10:03:40.036131+0000","last_became_active":"2024-04-17T10:03:38.252753+0000","last_became_peered":"2024-04-17T10:03:38.252753+0000","last_unstale":"2024-04-17T10:03:40.036131+0000","last_undegraded":"2024-04-17T10:03:40.036131+0000","last_fullsized":"2024-04-17T10:03:40.036131+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:07:33.201501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.252556+0000","last_change":"2024-04-17T10:03:37.239507+0000","last_active":"2024-04-17T10:03:39.252556+0000","last_peered":"2024-04-17T10:03:39.252556+0000","last_clean":"2024-04-17T10:03:39.252556+0000","last_became_active":"2024-04-17T10:03:37.239209+0000","last_became_peered":"2024-04-17T10:03:37.239209+0000","last_unstale":"2024-04-17T10:03:39.252556+0000","last_undegraded":"2024-04-17T10:03:39.252556+0000","last_fullsized":"2024-04-17T10:03:39.252556+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:09:18.401838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.035603+0000","last_change":"2024-04-17T10:03:38.253662+0000","last_active":"2024-04-17T10:03:40.035603+0000","last_peered":"2024-04-17T10:03:40.035603+0000","last_clean":"2024-04-17T10:03:40.035603+0000","last_became_active":"2024-04-17T10:03:38.253374+0000","last_became_peered":"2024-04-17T10:03:38.253374+0000","last_unstale":"2024-04-17T10:03:40.035603+0000","last_undegraded":"2024-04-17T10:03:40.035603+0000","last_fullsized":"2024-04-17T10:03:40.035603+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:09:00.962555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.950195+0000","last_change":"2024-04-17T10:03:37.246255+0000","last_active":"2024-04-17T10:03:39.950195+0000","last_peered":"2024-04-17T10:03:39.950195+0000","last_clean":"2024-04-17T10:03:39.950195+0000","last_became_active":"2024-04-17T10:03:37.245983+0000","last_became_peered":"2024-04-17T10:03:37.245983+0000","last_unstale":"2024-04-17T10:03:39.950195+0000","last_undegraded":"2024-04-17T10:03:39.950195+0000","last_fullsized":"2024-04-17T10:03:39.950195+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:17:59.650782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.255869+0000","last_change":"2024-04-17T10:03:38.247680+0000","last_active":"2024-04-17T10:03:38.255869+0000","last_peered":"2024-04-17T10:03:38.255869+0000","last_clean":"2024-04-17T10:03:38.255869+0000","last_became_active":"2024-04-17T10:03:38.247288+0000","last_became_peered":"2024-04-17T10:03:38.247288+0000","last_unstale":"2024-04-17T10:03:38.255869+0000","last_undegraded":"2024-04-17T10:03:38.255869+0000","last_fullsized":"2024-04-17T10:03:38.255869+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:42:29.599286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248857+0000","last_change":"2024-04-17T10:03:37.245202+0000","last_active":"2024-04-17T10:03:39.248857+0000","last_peered":"2024-04-17T10:03:39.248857+0000","last_clean":"2024-04-17T10:03:39.248857+0000","last_became_active":"2024-04-17T10:03:37.244890+0000","last_became_peered":"2024-04-17T10:03:37.244890+0000","last_unstale":"2024-04-17T10:03:39.248857+0000","last_undegraded":"2024-04-17T10:03:39.248857+0000","last_fullsized":"2024-04-17T10:03:39.248857+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:11:24.185185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.035558+0000","last_change":"2024-04-17T10:03:38.250351+0000","last_active":"2024-04-17T10:03:40.035558+0000","last_peered":"2024-04-17T10:03:40.035558+0000","last_clean":"2024-04-17T10:03:40.035558+0000","last_became_active":"2024-04-17T10:03:38.250026+0000","last_became_peered":"2024-04-17T10:03:38.250026+0000","last_unstale":"2024-04-17T10:03:40.035558+0000","last_undegraded":"2024-04-17T10:03:40.035558+0000","last_fullsized":"2024-04-17T10:03:40.035558+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:40:21.521207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.950233+0000","last_change":"2024-04-17T10:03:37.240462+0000","last_active":"2024-04-17T10:03:39.950233+0000","last_peered":"2024-04-17T10:03:39.950233+0000","last_clean":"2024-04-17T10:03:39.950233+0000","last_became_active":"2024-04-17T10:03:37.240202+0000","last_became_peered":"2024-04-17T10:03:37.240202+0000","last_unstale":"2024-04-17T10:03:39.950233+0000","last_undegraded":"2024-04-17T10:03:39.950233+0000","last_fullsized":"2024-04-17T10:03:39.950233+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:08:28.143714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743361+0000","last_change":"2024-04-17T10:03:38.251702+0000","last_active":"2024-04-17T10:03:39.743361+0000","last_peered":"2024-04-17T10:03:39.743361+0000","last_clean":"2024-04-17T10:03:39.743361+0000","last_became_active":"2024-04-17T10:03:38.250714+0000","last_became_peered":"2024-04-17T10:03:38.250714+0000","last_unstale":"2024-04-17T10:03:39.743361+0000","last_undegraded":"2024-04-17T10:03:39.743361+0000","last_fullsized":"2024-04-17T10:03:39.743361+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:04:32.865505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.233589+0000","last_change":"2024-04-17T10:03:37.238515+0000","last_active":"2024-04-17T10:03:38.233589+0000","last_peered":"2024-04-17T10:03:38.233589+0000","last_clean":"2024-04-17T10:03:38.233589+0000","last_became_active":"2024-04-17T10:03:37.238237+0000","last_became_peered":"2024-04-17T10:03:37.238237+0000","last_unstale":"2024-04-17T10:03:38.233589+0000","last_undegraded":"2024-04-17T10:03:38.233589+0000","last_fullsized":"2024-04-17T10:03:38.233589+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:42:33.549870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248812+0000","last_change":"2024-04-17T10:03:38.258042+0000","last_active":"2024-04-17T10:03:39.248812+0000","last_peered":"2024-04-17T10:03:39.248812+0000","last_clean":"2024-04-17T10:03:39.248812+0000","last_became_active":"2024-04-17T10:03:38.257665+0000","last_became_peered":"2024-04-17T10:03:38.257665+0000","last_unstale":"2024-04-17T10:03:39.248812+0000","last_undegraded":"2024-04-17T10:03:39.248812+0000","last_fullsized":"2024-04-17T10:03:39.248812+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:03:29.409571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743773+0000","last_change":"2024-04-17T10:03:37.238418+0000","last_active":"2024-04-17T10:03:39.743773+0000","last_peered":"2024-04-17T10:03:39.743773+0000","last_clean":"2024-04-17T10:03:39.743773+0000","last_became_active":"2024-04-17T10:03:37.233451+0000","last_became_peered":"2024-04-17T10:03:37.233451+0000","last_unstale":"2024-04-17T10:03:39.743773+0000","last_undegraded":"2024-04-17T10:03:39.743773+0000","last_fullsized":"2024-04-17T10:03:39.743773+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T20:36:43.078937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.259922+0000","last_change":"2024-04-17T10:03:38.254895+0000","last_active":"2024-04-17T10:03:38.259922+0000","last_peered":"2024-04-17T10:03:38.259922+0000","last_clean":"2024-04-17T10:03:38.259922+0000","last_became_active":"2024-04-17T10:03:38.250334+0000","last_became_peered":"2024-04-17T10:03:38.250334+0000","last_unstale":"2024-04-17T10:03:38.259922+0000","last_undegraded":"2024-04-17T10:03:38.259922+0000","last_fullsized":"2024-04-17T10:03:38.259922+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:10:52.153884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"109'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.251213+0000","last_change":"2024-04-17T10:03:37.232534+0000","last_active":"2024-04-17T10:03:39.251213+0000","last_peered":"2024-04-17T10:03:39.251213+0000","last_clean":"2024-04-17T10:03:39.251213+0000","last_became_active":"2024-04-17T10:03:37.232218+0000","last_became_peered":"2024-04-17T10:03:37.232218+0000","last_unstale":"2024-04-17T10:03:39.251213+0000","last_undegraded":"2024-04-17T10:03:39.251213+0000","last_fullsized":"2024-04-17T10:03:39.251213+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:28:55.097624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.261318+0000","last_change":"2024-04-17T10:03:38.254757+0000","last_active":"2024-04-17T10:03:38.261318+0000","last_peered":"2024-04-17T10:03:38.261318+0000","last_clean":"2024-04-17T10:03:38.261318+0000","last_became_active":"2024-04-17T10:03:38.254131+0000","last_became_peered":"2024-04-17T10:03:38.254131+0000","last_unstale":"2024-04-17T10:03:38.261318+0000","last_undegraded":"2024-04-17T10:03:38.261318+0000","last_fullsized":"2024-04-17T10:03:38.261318+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:00:04.333061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.234459+0000","last_change":"2024-04-17T10:03:37.237993+0000","last_active":"2024-04-17T10:03:38.234459+0000","last_peered":"2024-04-17T10:03:38.234459+0000","last_clean":"2024-04-17T10:03:38.234459+0000","last_became_active":"2024-04-17T10:03:37.237716+0000","last_became_peered":"2024-04-17T10:03:37.237716+0000","last_unstale":"2024-04-17T10:03:38.234459+0000","last_undegraded":"2024-04-17T10:03:38.234459+0000","last_fullsized":"2024-04-17T10:03:38.234459+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:08.897487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.249180+0000","last_change":"2024-04-17T10:03:38.248687+0000","last_active":"2024-04-17T10:03:39.249180+0000","last_peered":"2024-04-17T10:03:39.249180+0000","last_clean":"2024-04-17T10:03:39.249180+0000","last_became_active":"2024-04-17T10:03:38.248330+0000","last_became_peered":"2024-04-17T10:03:38.248330+0000","last_unstale":"2024-04-17T10:03:39.249180+0000","last_undegraded":"2024-04-17T10:03:39.249180+0000","last_fullsized":"2024-04-17T10:03:39.249180+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:45:12.303860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743751+0000","last_change":"2024-04-17T10:03:37.237005+0000","last_active":"2024-04-17T10:03:39.743751+0000","last_peered":"2024-04-17T10:03:39.743751+0000","last_clean":"2024-04-17T10:03:39.743751+0000","last_became_active":"2024-04-17T10:03:37.236755+0000","last_became_peered":"2024-04-17T10:03:37.236755+0000","last_unstale":"2024-04-17T10:03:39.743751+0000","last_undegraded":"2024-04-17T10:03:39.743751+0000","last_fullsized":"2024-04-17T10:03:39.743751+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:16:32.705548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248419+0000","last_change":"2024-04-17T10:03:38.251135+0000","last_active":"2024-04-17T10:03:39.248419+0000","last_peered":"2024-04-17T10:03:39.248419+0000","last_clean":"2024-04-17T10:03:39.248419+0000","last_became_active":"2024-04-17T10:03:38.250878+0000","last_became_peered":"2024-04-17T10:03:38.250878+0000","last_unstale":"2024-04-17T10:03:39.248419+0000","last_undegraded":"2024-04-17T10:03:39.248419+0000","last_fullsized":"2024-04-17T10:03:39.248419+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:05:01.156946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.252598+0000","last_change":"2024-04-17T10:03:37.232899+0000","last_active":"2024-04-17T10:03:39.252598+0000","last_peered":"2024-04-17T10:03:39.252598+0000","last_clean":"2024-04-17T10:03:39.252598+0000","last_became_active":"2024-04-17T10:03:37.232637+0000","last_became_peered":"2024-04-17T10:03:37.232637+0000","last_unstale":"2024-04-17T10:03:39.252598+0000","last_undegraded":"2024-04-17T10:03:39.252598+0000","last_fullsized":"2024-04-17T10:03:39.252598+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:02:55.973258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248910+0000","last_change":"2024-04-17T10:03:38.254711+0000","last_active":"2024-04-17T10:03:39.248910+0000","last_peered":"2024-04-17T10:03:39.248910+0000","last_clean":"2024-04-17T10:03:39.248910+0000","last_became_active":"2024-04-17T10:03:38.252650+0000","last_became_peered":"2024-04-17T10:03:38.252650+0000","last_unstale":"2024-04-17T10:03:39.248910+0000","last_undegraded":"2024-04-17T10:03:39.248910+0000","last_fullsized":"2024-04-17T10:03:39.248910+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:19:08.029031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248400+0000","last_change":"2024-04-17T10:03:37.233986+0000","last_active":"2024-04-17T10:03:39.248400+0000","last_peered":"2024-04-17T10:03:39.248400+0000","last_clean":"2024-04-17T10:03:39.248400+0000","last_became_active":"2024-04-17T10:03:37.233746+0000","last_became_peered":"2024-04-17T10:03:37.233746+0000","last_unstale":"2024-04-17T10:03:39.248400+0000","last_undegraded":"2024-04-17T10:03:39.248400+0000","last_fullsized":"2024-04-17T10:03:39.248400+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:14:24.245875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.035796+0000","last_change":"2024-04-17T10:03:38.251661+0000","last_active":"2024-04-17T10:03:40.035796+0000","last_peered":"2024-04-17T10:03:40.035796+0000","last_clean":"2024-04-17T10:03:40.035796+0000","last_became_active":"2024-04-17T10:03:38.251311+0000","last_became_peered":"2024-04-17T10:03:38.251311+0000","last_unstale":"2024-04-17T10:03:40.035796+0000","last_undegraded":"2024-04-17T10:03:40.035796+0000","last_fullsized":"2024-04-17T10:03:40.035796+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:35:32.567676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.248848+0000","last_change":"2024-04-17T10:03:37.245414+0000","last_active":"2024-04-17T10:03:39.248848+0000","last_peered":"2024-04-17T10:03:39.248848+0000","last_clean":"2024-04-17T10:03:39.248848+0000","last_became_active":"2024-04-17T10:03:37.245152+0000","last_became_peered":"2024-04-17T10:03:37.245152+0000","last_unstale":"2024-04-17T10:03:39.248848+0000","last_undegraded":"2024-04-17T10:03:39.248848+0000","last_fullsized":"2024-04-17T10:03:39.248848+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:08:28.143714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.259059+0000","last_change":"2024-04-17T10:03:38.251077+0000","last_active":"2024-04-17T10:03:38.259059+0000","last_peered":"2024-04-17T10:03:38.259059+0000","last_clean":"2024-04-17T10:03:38.259059+0000","last_became_active":"2024-04-17T10:03:38.250777+0000","last_became_peered":"2024-04-17T10:03:38.250777+0000","last_unstale":"2024-04-17T10:03:38.259059+0000","last_undegraded":"2024-04-17T10:03:38.259059+0000","last_fullsized":"2024-04-17T10:03:38.259059+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:10:48.549970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-17T10:03:38.233395+0000","last_change":"2024-04-17T10:03:37.238164+0000","last_active":"2024-04-17T10:03:38.233395+0000","last_peered":"2024-04-17T10:03:38.233395+0000","last_clean":"2024-04-17T10:03:38.233395+0000","last_became_active":"2024-04-17T10:03:37.237818+0000","last_became_peered":"2024-04-17T10:03:37.237818+0000","last_unstale":"2024-04-17T10:03:38.233395+0000","last_undegraded":"2024-04-17T10:03:38.233395+0000","last_fullsized":"2024-04-17T10:03:38.233395+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:31.528731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.743407+0000","last_change":"2024-04-17T10:03:38.252456+0000","last_active":"2024-04-17T10:03:39.743407+0000","last_peered":"2024-04-17T10:03:39.743407+0000","last_clean":"2024-04-17T10:03:39.743407+0000","last_became_active":"2024-04-17T10:03:38.251750+0000","last_became_peered":"2024-04-17T10:03:38.251750+0000","last_unstale":"2024-04-17T10:03:39.743407+0000","last_undegraded":"2024-04-17T10:03:39.743407+0000","last_fullsized":"2024-04-17T10:03:39.743407+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:37.211216+0000","last_clean_scrub_stamp":"2024-04-17T10:03:37.211216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:46:32.849083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:39.252761+0000","last_change":"2024-04-17T10:03:37.234733+0000","last_active":"2024-04-17T10:03:39.252761+0000","last_peered":"2024-04-17T10:03:39.252761+0000","last_clean":"2024-04-17T10:03:39.252761+0000","last_became_active":"2024-04-17T10:03:37.234471+0000","last_became_peered":"2024-04-17T10:03:37.234471+0000","last_unstale":"2024-04-17T10:03:39.252761+0000","last_undegraded":"2024-04-17T10:03:39.252761+0000","last_fullsized":"2024-04-17T10:03:39.252761+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T10:03:36.203872+0000","last_clean_scrub_stamp":"2024-04-17T10:03:36.203872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:09:11.212786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":336,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-17T10:03:40.035907+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T10:03:40.035907+0000","last_peered":"2024-04-17T10:03:40.035907+0000","last_clean":"2024-04-17T10:03:40.035907+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T10:03:40.035907+0000","last_undegraded":"2024-04-17T10:03:40.035907+0000","last_fullsized":"2024-04-17T10:03:40.035907+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509571,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33072,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338768,"statfs":{"total":96636764160,"available":96602898432,"internally_reserved":0,"allocated":1937408,"data_stored":1167828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509572,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1351680,"data_stored":583214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509571,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1355776,"data_stored":583748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509571,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1351680,"data_stored":578526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509572,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1347584,"data_stored":583144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509571,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1355776,"data_stored":583268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542275,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1335296,"data_stored":576844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542275,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37248,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334592,"statfs":{"total":96636764160,"available":96598622208,"internally_reserved":0,"allocated":1949696,"data_stored":1168936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:03:42.752 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T10:03:42.753 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-17T10:03:42.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.943+0000 7fa2be400640 1 Processor -- start 2024-04-17T10:03:42.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.943+0000 7fa2be400640 1 -- start start 2024-04-17T10:03:42.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.943+0000 7fa2be400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2b806ba30 0x7fa2b806be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:42.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.943+0000 7fa2be400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa2b8071e60 0x7fa2b80722b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:42.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.943+0000 7fa2be400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2b806c350 con 0x7fa2b80cf370 2024-04-17T10:03:42.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.943+0000 7fa2be400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa2b80cfef0 con 0x7fa2b8071e60 2024-04-17T10:03:42.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.943+0000 7fa2be400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b80d0130 con 0x7fa2b806ba30 2024-04-17T10:03:42.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.947+0000 7fa2b7400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2b806ba30 0x7fa2b806be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:42.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.947+0000 7fa2afe00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fa2b80cf370 0x7fa2b80cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:37114/0 (socket says 172.21.15.32:37114) 2024-04-17T10:03:42.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.947+0000 7fa2afe00640 1 -- 172.21.15.32:0/724425800 learned_addr learned my addr 172.21.15.32:0/724425800 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:42.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.947+0000 7fa2b7e00640 1 --2- 172.21.15.32:0/724425800 >> v2:172.21.15.144:3300/0 conn(0x7fa2b8071e60 0x7fa2b80722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:42.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.947+0000 7fa2b7400640 1 -- 172.21.15.32:0/724425800 >> v2:172.21.15.144:3300/0 conn(0x7fa2b8071e60 msgr2=0x7fa2b80722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:42.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.947+0000 7fa2b7400640 1 --2- 172.21.15.32:0/724425800 >> v2:172.21.15.144:3300/0 conn(0x7fa2b8071e60 0x7fa2b80722b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.947+0000 7fa2b7400640 1 -- 172.21.15.32:0/724425800 >> v1:172.21.15.144:6789/0 conn(0x7fa2b80cf370 legacy=0x7fa2b80cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:42.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.947+0000 7fa2b7400640 1 -- 172.21.15.32:0/724425800 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b815d090 con 0x7fa2b806ba30 2024-04-17T10:03:42.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.947+0000 7fa2b7400640 1 --2- 172.21.15.32:0/724425800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2b806ba30 0x7fa2b806be10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80097c0 tx=0x7fa2a801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aef4a01bc0ac7dc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:42.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.951+0000 7fa2b6a00640 1 -- 172.21.15.32:0/724425800 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80a7070 con 0x7fa2b806ba30 2024-04-17T10:03:42.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.951+0000 7fa2b6a00640 1 -- 172.21.15.32:0/724425800 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2a80b0070 con 0x7fa2b806ba30 2024-04-17T10:03:42.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.951+0000 7fa2b6a00640 1 -- 172.21.15.32:0/724425800 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80ab890 con 0x7fa2b806ba30 2024-04-17T10:03:42.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.951+0000 7fa2be400640 1 -- 172.21.15.32:0/724425800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2b806ba30 msgr2=0x7fa2b806be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:42.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.951+0000 7fa2be400640 1 --2- 172.21.15.32:0/724425800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2b806ba30 0x7fa2b806be10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80097c0 tx=0x7fa2a801c470 comp rx=0 tx=0).stop 2024-04-17T10:03:42.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.951+0000 7fa2be400640 1 -- 172.21.15.32:0/724425800 shutdown_connections 2024-04-17T10:03:42.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.951+0000 7fa2be400640 1 --2- 172.21.15.32:0/724425800 >> v2:172.21.15.144:3300/0 conn(0x7fa2b8071e60 0x7fa2b80722b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.951+0000 7fa2be400640 1 --2- 172.21.15.32:0/724425800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2b806ba30 0x7fa2b806be10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.951+0000 7fa2be400640 1 -- 172.21.15.32:0/724425800 >> 172.21.15.32:0/724425800 conn(0x7fa2b80b2120 msgr2=0x7fa2b80b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:42.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.951+0000 7fa2be400640 1 -- 172.21.15.32:0/724425800 shutdown_connections 2024-04-17T10:03:42.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.951+0000 7fa2be400640 1 -- 172.21.15.32:0/724425800 wait complete. 2024-04-17T10:03:42.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.951+0000 7fa2be400640 1 Processor -- start 2024-04-17T10:03:42.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.951+0000 7fa2be400640 1 -- start start 2024-04-17T10:03:42.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.951+0000 7fa2be400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b806ba30 0x7fa2b8139c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:42.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.951+0000 7fa2be400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b8071e60 0x7fa2b813a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:42.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.951+0000 7fa2be400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2b80cf370 0x7fa2b815df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:42.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.951+0000 7fa2be400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b815dcf0 con 0x7fa2b806ba30 2024-04-17T10:03:42.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.951+0000 7fa2be400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b815db70 con 0x7fa2b8071e60 2024-04-17T10:03:42.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.951+0000 7fa2be400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b8146230 con 0x7fa2b80cf370 2024-04-17T10:03:42.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.951+0000 7fa2b7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b806ba30 0x7fa2b8139c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:42.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.951+0000 7fa2b7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b806ba30 0x7fa2b8139c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49326/0 (socket says 172.21.15.32:49326) 2024-04-17T10:03:42.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.951+0000 7fa2b7400640 1 -- 172.21.15.32:0/557893453 learned_addr learned my addr 172.21.15.32:0/557893453 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:42.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.955+0000 7fa2b7e00640 1 --2- 172.21.15.32:0/557893453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2b80cf370 0x7fa2b815df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:42.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.955+0000 7fa2afe00640 1 --2- 172.21.15.32:0/557893453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b8071e60 0x7fa2b813a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:42.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.955+0000 7fa2b7400640 1 -- 172.21.15.32:0/557893453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2b80cf370 msgr2=0x7fa2b815df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:42.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.955+0000 7fa2b7400640 1 --2- 172.21.15.32:0/557893453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2b80cf370 0x7fa2b815df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.955+0000 7fa2b7400640 1 -- 172.21.15.32:0/557893453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b8071e60 msgr2=0x7fa2b813a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:42.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.955+0000 7fa2b7400640 1 --2- 172.21.15.32:0/557893453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b8071e60 0x7fa2b813a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:42.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.955+0000 7fa2b7400640 1 -- 172.21.15.32:0/557893453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b815f8a0 con 0x7fa2b806ba30 2024-04-17T10:03:42.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.955+0000 7fa2b7400640 1 --2- 172.21.15.32:0/557893453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b806ba30 0x7fa2b8139c10 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80ad040 tx=0x7fa2a801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:42.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.955+0000 7fa2af400640 1 -- 172.21.15.32:0/557893453 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80bf610 con 0x7fa2b806ba30 2024-04-17T10:03:42.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.955+0000 7fa2af400640 1 -- 172.21.15.32:0/557893453 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2a80bf7d0 con 0x7fa2b806ba30 2024-04-17T10:03:42.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.955+0000 7fa2af400640 1 -- 172.21.15.32:0/557893453 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80bfbc0 con 0x7fa2b806ba30 2024-04-17T10:03:42.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.955+0000 7fa2be400640 1 -- 172.21.15.32:0/557893453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b815fb80 con 0x7fa2b806ba30 2024-04-17T10:03:42.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.955+0000 7fa2be400640 1 -- 172.21.15.32:0/557893453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b8160140 con 0x7fa2b806ba30 2024-04-17T10:03:42.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.955+0000 7fa2be400640 1 -- 172.21.15.32:0/557893453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b8068a90 con 0x7fa2b806ba30 2024-04-17T10:03:42.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.955+0000 7fa2af400640 1 -- 172.21.15.32:0/557893453 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa2a80bd020 con 0x7fa2b806ba30 2024-04-17T10:03:42.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.955+0000 7fa2af400640 1 --2- 172.21.15.32:0/557893453 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa294068380 0x7fa29406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:42.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.959+0000 7fa2af400640 1 -- 172.21.15.32:0/557893453 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fa2a8120ce0 con 0x7fa2b806ba30 2024-04-17T10:03:42.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.959+0000 7fa2afe00640 1 --2- 172.21.15.32:0/557893453 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa294068380 0x7fa29406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:42.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.959+0000 7fa2afe00640 1 --2- 172.21.15.32:0/557893453 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa294068380 0x7fa29406a840 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa2b813f570 tx=0x7fa2a0007e90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:42.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:42.959+0000 7fa2af400640 1 -- 172.21.15.32:0/557893453 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa2a80c6030 con 0x7fa2b806ba30 2024-04-17T10:03:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.183+0000 7fa2be400640 1 -- 172.21.15.32:0/557893453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa2b806e150 con 0x7fa2b806ba30 2024-04-17T10:03:43.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.183+0000 7fa2af400640 1 -- 172.21.15.32:0/557893453 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) ==== 190+0+147 (secure 0 0 0) 0x7fa2a8016090 con 0x7fa2b806ba30 2024-04-17T10:03:43.183 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-04-17T10:03:43.184 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDPmx9mUXoLKxAANV7zhLvL0DqHngk4TtsdBQ== 2024-04-17T10:03:43.184 INFO:teuthology.orchestra.run.smithi032.stdout: caps mds = "allow" 2024-04-17T10:03:43.184 INFO:teuthology.orchestra.run.smithi032.stdout: caps mon = "allow r" 2024-04-17T10:03:43.184 INFO:teuthology.orchestra.run.smithi032.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-17T10:03:43.184 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2024-04-17T10:03:43.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.187+0000 7fa2be400640 1 -- 172.21.15.32:0/557893453 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa294068380 msgr2=0x7fa29406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:43.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.187+0000 7fa2be400640 1 --2- 172.21.15.32:0/557893453 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa294068380 0x7fa29406a840 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa2b813f570 tx=0x7fa2a0007e90 comp rx=0 tx=0).stop 2024-04-17T10:03:43.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.187+0000 7fa2be400640 1 -- 172.21.15.32:0/557893453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b806ba30 msgr2=0x7fa2b8139c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:43.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.187+0000 7fa2be400640 1 --2- 172.21.15.32:0/557893453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b806ba30 0x7fa2b8139c10 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80ad040 tx=0x7fa2a801acb0 comp rx=0 tx=0).stop 2024-04-17T10:03:43.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.187+0000 7fa2be400640 1 -- 172.21.15.32:0/557893453 shutdown_connections 2024-04-17T10:03:43.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.187+0000 7fa2be400640 1 --2- 172.21.15.32:0/557893453 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa294068380 0x7fa29406a840 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:43.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.187+0000 7fa2be400640 1 --2- 172.21.15.32:0/557893453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2b80cf370 0x7fa2b815df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:43.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.187+0000 7fa2be400640 1 --2- 172.21.15.32:0/557893453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b8071e60 0x7fa2b813a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:43.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.187+0000 7fa2be400640 1 --2- 172.21.15.32:0/557893453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2b806ba30 0x7fa2b8139c10 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:43.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.187+0000 7fa2be400640 1 -- 172.21.15.32:0/557893453 >> 172.21.15.32:0/557893453 conn(0x7fa2b80b2120 msgr2=0x7fa2b80b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:43.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.187+0000 7fa2be400640 1 -- 172.21.15.32:0/557893453 shutdown_connections 2024-04-17T10:03:43.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.187+0000 7fa2be400640 1 -- 172.21.15.32:0/557893453 wait complete. 2024-04-17T10:03:43.205 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:03:43.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.347+0000 7f2989c00640 1 Processor -- start 2024-04-17T10:03:43.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.347+0000 7f2989c00640 1 -- start start 2024-04-17T10:03:43.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2989c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f298415baa0 0x7f298415bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:43.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2989c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f298406ef60 0x7f298406f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:43.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2989c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2984072bb0 con 0x7f2984144e90 2024-04-17T10:03:43.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2989c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f298415c440 con 0x7f298406ef60 2024-04-17T10:03:43.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2989c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f298406f8f0 con 0x7f298415baa0 2024-04-17T10:03:43.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2982a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2984144e90 0x7f29840b48a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41520/0 (socket says 172.21.15.32:41520) 2024-04-17T10:03:43.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2982a00640 1 -- 172.21.15.32:0/3882856461 learned_addr learned my addr 172.21.15.32:0/3882856461 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:43.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2983400640 1 --2- 172.21.15.32:0/3882856461 >> v2:172.21.15.144:3300/0 conn(0x7f298406ef60 0x7f298406f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f297be00640 1 --2- 172.21.15.32:0/3882856461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f298415baa0 0x7f298415bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2982000640 1 -- 172.21.15.32:0/3882856461 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4031346646 0 0) 0x7f2984072bb0 con 0x7f2984144e90 2024-04-17T10:03:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2982000640 1 -- 172.21.15.32:0/3882856461 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2968003530 con 0x7f2984144e90 2024-04-17T10:03:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2982000640 1 -- 172.21.15.32:0/3882856461 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3178866642 0 0) 0x7f2968003530 con 0x7f2984144e90 2024-04-17T10:03:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2982000640 1 -- 172.21.15.32:0/3882856461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f298415baa0 msgr2=0x7f298415bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2982000640 1 --2- 172.21.15.32:0/3882856461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f298415baa0 0x7f298415bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2982000640 1 -- 172.21.15.32:0/3882856461 >> v2:172.21.15.144:3300/0 conn(0x7f298406ef60 msgr2=0x7f298406f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2982000640 1 --2- 172.21.15.32:0/3882856461 >> v2:172.21.15.144:3300/0 conn(0x7f298406ef60 0x7f298406f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2982000640 1 -- 172.21.15.32:0/3882856461 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29840b3dc0 con 0x7f2984144e90 2024-04-17T10:03:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2982000640 1 -- 172.21.15.32:0/3882856461 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f2974001680 con 0x7f2984144e90 2024-04-17T10:03:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2982000640 1 -- 172.21.15.32:0/3882856461 >> v1:172.21.15.32:6789/0 conn(0x7f2984144e90 legacy=0x7f29840b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2982000640 1 --2- 172.21.15.32:0/3882856461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29680039c0 0x7f2968003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2982000640 1 --2- 172.21.15.32:0/3882856461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f298415baa0 0x7f2968004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2982000640 1 --2- 172.21.15.32:0/3882856461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2968004d50 0x7f29680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2982000640 1 -- 172.21.15.32:0/3882856461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f298406f8f0 con 0x7f2968004d50 2024-04-17T10:03:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2982000640 1 -- 172.21.15.32:0/3882856461 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f296800b9d0 con 0x7f29680039c0 2024-04-17T10:03:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2982000640 1 -- 172.21.15.32:0/3882856461 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f296800bb50 con 0x7f298415baa0 2024-04-17T10:03:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2983400640 1 --2- 172.21.15.32:0/3882856461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2968004d50 0x7f29680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2982a00640 1 --2- 172.21.15.32:0/3882856461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29680039c0 0x7f2968003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f297be00640 1 --2- 172.21.15.32:0/3882856461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f298415baa0 0x7f2968004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2983400640 1 -- 172.21.15.32:0/3882856461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f298415baa0 msgr2=0x7f2968004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2983400640 1 --2- 172.21.15.32:0/3882856461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f298415baa0 0x7f2968004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2983400640 1 -- 172.21.15.32:0/3882856461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29680039c0 msgr2=0x7f2968003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2983400640 1 --2- 172.21.15.32:0/3882856461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29680039c0 0x7f2968003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.351+0000 7f2983400640 1 -- 172.21.15.32:0/3882856461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2968003530 con 0x7f2968004d50 2024-04-17T10:03:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.355+0000 7f2983400640 1 --2- 172.21.15.32:0/3882856461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2968004d50 0x7f29680051d0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f297c009fc0 tx=0x7f297c01eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d80b12a61f5fa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.355+0000 7f2982000640 1 -- 172.21.15.32:0/3882856461 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c0b4070 con 0x7f2968004d50 2024-04-17T10:03:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.355+0000 7f2982000640 1 -- 172.21.15.32:0/3882856461 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f297c0af440 con 0x7f2968004d50 2024-04-17T10:03:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.355+0000 7f2982000640 1 -- 172.21.15.32:0/3882856461 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c009c10 con 0x7f2968004d50 2024-04-17T10:03:43.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.355+0000 7f2989c00640 1 -- 172.21.15.32:0/3882856461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2968004d50 msgr2=0x7f29680051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:43.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.355+0000 7f2989c00640 1 --2- 172.21.15.32:0/3882856461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2968004d50 0x7f29680051d0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f297c009fc0 tx=0x7f297c01eed0 comp rx=0 tx=0).stop 2024-04-17T10:03:43.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.355+0000 7f2983400640 1 -- 172.21.15.32:0/3882856461 reap_dead start 2024-04-17T10:03:43.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.355+0000 7f2989c00640 1 -- 172.21.15.32:0/3882856461 shutdown_connections 2024-04-17T10:03:43.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.355+0000 7f2989c00640 1 -- 172.21.15.32:0/3882856461 >> 172.21.15.32:0/3882856461 conn(0x7f298406ba30 msgr2=0x7f298406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:43.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.355+0000 7f2989c00640 1 -- 172.21.15.32:0/3882856461 shutdown_connections 2024-04-17T10:03:43.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.355+0000 7f2989c00640 1 -- 172.21.15.32:0/3882856461 wait complete. 2024-04-17T10:03:43.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f2989c00640 1 Processor -- start 2024-04-17T10:03:43.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f2989c00640 1 -- start start 2024-04-17T10:03:43.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f2989c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2968004d50 0x7f2984141f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:43.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f2989c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29840b2680 0x7f29840b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:43.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f2989c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29841e41d0 0x7f29841e4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:43.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f2989c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f29840b4380 con 0x7f29841e41d0 2024-04-17T10:03:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f2989c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2984145270 con 0x7f2968004d50 2024-04-17T10:03:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f2989c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f29840b4fb0 con 0x7f29840b2680 2024-04-17T10:03:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f2983400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29841e41d0 0x7f29841e4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f2983400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29841e41d0 0x7f29841e4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49336/0 (socket says 172.21.15.32:49336) 2024-04-17T10:03:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f2983400640 1 -- 172.21.15.32:0/1588624282 learned_addr learned my addr 172.21.15.32:0/1588624282 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f2982a00640 1 --2- 172.21.15.32:0/1588624282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2968004d50 0x7f2984141f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f297be00640 1 --2- 172.21.15.32:0/1588624282 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29840b2680 0x7f29840b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f2983400640 1 -- 172.21.15.32:0/1588624282 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29840b2680 msgr2=0x7f29840b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f2983400640 1 --2- 172.21.15.32:0/1588624282 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29840b2680 0x7f29840b2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f2983400640 1 -- 172.21.15.32:0/1588624282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2968004d50 msgr2=0x7f2984141f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f2983400640 1 --2- 172.21.15.32:0/1588624282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2968004d50 0x7f2984141f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f2983400640 1 -- 172.21.15.32:0/1588624282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29841f4360 con 0x7f29841e41d0 2024-04-17T10:03:43.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f2983400640 1 --2- 172.21.15.32:0/1588624282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29841e41d0 0x7f29841e4650 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f297c025400 tx=0x7f297c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f297aa00640 1 -- 172.21.15.32:0/1588624282 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c0be070 con 0x7f29841e41d0 2024-04-17T10:03:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f297aa00640 1 -- 172.21.15.32:0/1588624282 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f297c0039a0 con 0x7f29841e41d0 2024-04-17T10:03:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f297aa00640 1 -- 172.21.15.32:0/1588624282 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c0b4070 con 0x7f29841e41d0 2024-04-17T10:03:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f2989c00640 1 -- 172.21.15.32:0/1588624282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29841f4640 con 0x7f29841e41d0 2024-04-17T10:03:43.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f2989c00640 1 -- 172.21.15.32:0/1588624282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29841f4c00 con 0x7f29841e41d0 2024-04-17T10:03:43.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.359+0000 7f2989c00640 1 -- 172.21.15.32:0/1588624282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2984068a90 con 0x7f29841e41d0 2024-04-17T10:03:43.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.363+0000 7f297aa00640 1 -- 172.21.15.32:0/1588624282 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f297c0b8070 con 0x7f29841e41d0 2024-04-17T10:03:43.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.363+0000 7f297aa00640 1 --2- 172.21.15.32:0/1588624282 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2960068380 0x7f296006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:43.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.363+0000 7f297aa00640 1 -- 172.21.15.32:0/1588624282 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f297c120e40 con 0x7f29841e41d0 2024-04-17T10:03:43.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.363+0000 7f2982a00640 1 --2- 172.21.15.32:0/1588624282 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2960068380 0x7f296006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:43.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.363+0000 7f2982a00640 1 --2- 172.21.15.32:0/1588624282 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2960068380 0x7f296006a840 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f2974000c10 tx=0x7f2974001e20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:43.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.367+0000 7f297aa00640 1 -- 172.21.15.32:0/1588624282 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f297c010040 con 0x7f29841e41d0 2024-04-17T10:03:43.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.587+0000 7f2989c00640 1 -- 172.21.15.32:0/1588624282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29840adca0 con 0x7f29841e41d0 2024-04-17T10:03:43.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.587+0000 7f297aa00640 1 -- 172.21.15.32:0/1588624282 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5204 (secure 0 0 0) 0x7f297c018be0 con 0x7f29841e41d0 2024-04-17T10:03:43.586 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:03:43.586 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":98,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7586,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3601627312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3601627312},{"type":"v1","addr":"172.21.15.144:6834","nonce":3601627312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2382359917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2382359917},{"type":"v1","addr":"172.21.15.144:6835","nonce":2382359917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1857552160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1857552160},{"type":"v1","addr":"172.21.15.32:6835","nonce":1857552160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:03:37.619785+0000","modified":"2024-04-17T10:03:40.645595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9285":{"gid":9285,"name":"a","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/901447746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":901447746},{"type":"v1","addr":"172.21.15.32:6837","nonce":901447746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9285,"qdb_cluster":[9285]},"id":8}]} 2024-04-17T10:03:43.586 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 98 2024-04-17T10:03:43.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.591+0000 7f2989c00640 1 -- 172.21.15.32:0/1588624282 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2960068380 msgr2=0x7f296006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:43.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.591+0000 7f2989c00640 1 --2- 172.21.15.32:0/1588624282 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2960068380 0x7f296006a840 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f2974000c10 tx=0x7f2974001e20 comp rx=0 tx=0).stop 2024-04-17T10:03:43.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.591+0000 7f2989c00640 1 -- 172.21.15.32:0/1588624282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29841e41d0 msgr2=0x7f29841e4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:43.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.591+0000 7f2989c00640 1 --2- 172.21.15.32:0/1588624282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29841e41d0 0x7f29841e4650 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f297c025400 tx=0x7f297c009750 comp rx=0 tx=0).stop 2024-04-17T10:03:43.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.591+0000 7f2989c00640 1 -- 172.21.15.32:0/1588624282 shutdown_connections 2024-04-17T10:03:43.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.591+0000 7f2989c00640 1 --2- 172.21.15.32:0/1588624282 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2960068380 0x7f296006a840 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:43.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.591+0000 7f2989c00640 1 --2- 172.21.15.32:0/1588624282 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29840b2680 0x7f29840b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:43.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.591+0000 7f2989c00640 1 --2- 172.21.15.32:0/1588624282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29841e41d0 0x7f29841e4650 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:43.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.591+0000 7f2989c00640 1 --2- 172.21.15.32:0/1588624282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2968004d50 0x7f2984141f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:43.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.591+0000 7f2989c00640 1 -- 172.21.15.32:0/1588624282 >> 172.21.15.32:0/1588624282 conn(0x7f298406ba30 msgr2=0x7f2984143e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:43.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.591+0000 7f2989c00640 1 -- 172.21.15.32:0/1588624282 shutdown_connections 2024-04-17T10:03:43.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:43.591+0000 7f2989c00640 1 -- 172.21.15.32:0/1588624282 wait complete. 2024-04-17T10:03:44.010 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 98, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:03:37.619785+0000', 'modified': '2024-04-17T10:03:40.645595+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9285': {'gid': 9285, 'name': 'a', 'rank': 0, 'incarnation': 94, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6837/901447746', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6836', 'nonce': 901447746}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 901447746}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9285, 'qdb_cluster': [9285]} 2024-04-17T10:03:44.010 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T10:03:44.010 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-17T10:03:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.155+0000 7fce2d200640 1 Processor -- start 2024-04-17T10:03:44.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.155+0000 7fce2d200640 1 -- start start 2024-04-17T10:03:44.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.159+0000 7fce2d200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fce28155a70 0x7fce2815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.159+0000 7fce2d200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fce280b3340 0x7fce2815c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.159+0000 7fce2d200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce2815f870 con 0x7fce28155e50 2024-04-17T10:03:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.159+0000 7fce2d200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fce280b42a0 con 0x7fce280b3340 2024-04-17T10:03:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.159+0000 7fce2d200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fce280d1250 con 0x7fce28155a70 2024-04-17T10:03:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.159+0000 7fce26800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fce280b3340 0x7fce2815c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.159+0000 7fce26800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fce280b3340 0x7fce2815c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49338/0 (socket says 172.21.15.32:49338) 2024-04-17T10:03:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.159+0000 7fce26800640 1 -- 172.21.15.32:0/998695857 learned_addr learned my addr 172.21.15.32:0/998695857 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.159+0000 7fce25e00640 1 --2- 172.21.15.32:0/998695857 >> v2:172.21.15.144:3300/0 conn(0x7fce28155a70 0x7fce2815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.159+0000 7fce26800640 1 -- 172.21.15.32:0/998695857 >> v2:172.21.15.144:3300/0 conn(0x7fce28155a70 msgr2=0x7fce2815f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.159+0000 7fce26800640 1 --2- 172.21.15.32:0/998695857 >> v2:172.21.15.144:3300/0 conn(0x7fce28155a70 0x7fce2815f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.159+0000 7fce26800640 1 -- 172.21.15.32:0/998695857 >> v1:172.21.15.144:6789/0 conn(0x7fce28155e50 legacy=0x7fce2815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.159+0000 7fce26800640 1 -- 172.21.15.32:0/998695857 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2815fb50 con 0x7fce280b3340 2024-04-17T10:03:44.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.159+0000 7fce25e00640 1 --2- 172.21.15.32:0/998695857 >> v2:172.21.15.144:3300/0 conn(0x7fce28155a70 0x7fce2815f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:03:44.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.159+0000 7fce26800640 1 --2- 172.21.15.32:0/998695857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce280b3340 0x7fce2815c260 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fce2000d150 tx=0x7fce2001ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d150fdbc7344352f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:44.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.159+0000 7fce25400640 1 -- 172.21.15.32:0/998695857 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2000b2d0 con 0x7fce280b3340 2024-04-17T10:03:44.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.163+0000 7fce25400640 1 -- 172.21.15.32:0/998695857 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce2000b470 con 0x7fce280b3340 2024-04-17T10:03:44.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.163+0000 7fce25400640 1 -- 172.21.15.32:0/998695857 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce200a7aa0 con 0x7fce280b3340 2024-04-17T10:03:44.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.163+0000 7fce2d200640 1 -- 172.21.15.32:0/998695857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce280b3340 msgr2=0x7fce2815c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:44.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.163+0000 7fce2d200640 1 --2- 172.21.15.32:0/998695857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce280b3340 0x7fce2815c260 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fce2000d150 tx=0x7fce2001ebb0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.163+0000 7fce2d200640 1 -- 172.21.15.32:0/998695857 shutdown_connections 2024-04-17T10:03:44.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.163+0000 7fce2d200640 1 --2- 172.21.15.32:0/998695857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce280b3340 0x7fce2815c260 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.163+0000 7fce2d200640 1 --2- 172.21.15.32:0/998695857 >> v2:172.21.15.144:3300/0 conn(0x7fce28155a70 0x7fce2815f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.163+0000 7fce2d200640 1 -- 172.21.15.32:0/998695857 >> 172.21.15.32:0/998695857 conn(0x7fce2806ba30 msgr2=0x7fce2806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:44.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.163+0000 7fce2d200640 1 -- 172.21.15.32:0/998695857 shutdown_connections 2024-04-17T10:03:44.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.163+0000 7fce2d200640 1 -- 172.21.15.32:0/998695857 wait complete. 2024-04-17T10:03:44.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.163+0000 7fce2d200640 1 Processor -- start 2024-04-17T10:03:44.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.163+0000 7fce2d200640 1 -- start start 2024-04-17T10:03:44.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.163+0000 7fce2d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce280b3340 0x7fce28150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.163+0000 7fce2d200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce28155a70 0x7fce28152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.163+0000 7fce2d200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce28155e50 0x7fce2813ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.163+0000 7fce2d200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fce280d02f0 con 0x7fce280b3340 2024-04-17T10:03:44.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.163+0000 7fce2d200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fce280d0170 con 0x7fce28155a70 2024-04-17T10:03:44.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.163+0000 7fce2d200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fce280d0470 con 0x7fce28155e50 2024-04-17T10:03:44.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.163+0000 7fce25e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce280b3340 0x7fce28150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.163+0000 7fce25e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce280b3340 0x7fce28150620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49342/0 (socket says 172.21.15.32:49342) 2024-04-17T10:03:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.163+0000 7fce25e00640 1 -- 172.21.15.32:0/1152154758 learned_addr learned my addr 172.21.15.32:0/1152154758 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.163+0000 7fce1fe00640 1 --2- 172.21.15.32:0/1152154758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce28155a70 0x7fce28152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.163+0000 7fce26800640 1 --2- 172.21.15.32:0/1152154758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce28155e50 0x7fce2813ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.167+0000 7fce25e00640 1 -- 172.21.15.32:0/1152154758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce28155e50 msgr2=0x7fce2813ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.167+0000 7fce25e00640 1 --2- 172.21.15.32:0/1152154758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce28155e50 0x7fce2813ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.167+0000 7fce25e00640 1 -- 172.21.15.32:0/1152154758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce28155a70 msgr2=0x7fce28152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.167+0000 7fce25e00640 1 --2- 172.21.15.32:0/1152154758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce28155a70 0x7fce28152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.167+0000 7fce25e00640 1 -- 172.21.15.32:0/1152154758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce28150c70 con 0x7fce280b3340 2024-04-17T10:03:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.167+0000 7fce1fe00640 1 --2- 172.21.15.32:0/1152154758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce28155a70 0x7fce28152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:03:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.167+0000 7fce26800640 1 --2- 172.21.15.32:0/1152154758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce28155e50 0x7fce2813ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:03:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.167+0000 7fce25e00640 1 --2- 172.21.15.32:0/1152154758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce280b3340 0x7fce28150620 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fce1800c7c0 tx=0x7fce1800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.167+0000 7fce1e000640 1 -- 172.21.15.32:0/1152154758 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18007a30 con 0x7fce280b3340 2024-04-17T10:03:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.167+0000 7fce1e000640 1 -- 172.21.15.32:0/1152154758 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce18007bd0 con 0x7fce280b3340 2024-04-17T10:03:44.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.167+0000 7fce1e000640 1 -- 172.21.15.32:0/1152154758 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce180113f0 con 0x7fce280b3340 2024-04-17T10:03:44.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.167+0000 7fce2d200640 1 -- 172.21.15.32:0/1152154758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce2813f8d0 con 0x7fce280b3340 2024-04-17T10:03:44.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.167+0000 7fce2d200640 1 -- 172.21.15.32:0/1152154758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce2813fe90 con 0x7fce280b3340 2024-04-17T10:03:44.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.167+0000 7fce1e000640 1 -- 172.21.15.32:0/1152154758 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fce18011590 con 0x7fce280b3340 2024-04-17T10:03:44.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.167+0000 7fce1e000640 1 --2- 172.21.15.32:0/1152154758 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fce04068380 0x7fce0406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.167+0000 7fce1e000640 1 -- 172.21.15.32:0/1152154758 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fce18086b80 con 0x7fce280b3340 2024-04-17T10:03:44.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.167+0000 7fce1fe00640 1 --2- 172.21.15.32:0/1152154758 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fce04068380 0x7fce0406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.167+0000 7fce305fe000 1 Processor -- start 2024-04-17T10:03:44.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce305fe000 1 -- start start 2024-04-17T10:03:44.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce1fe00640 1 --2- 172.21.15.32:0/1152154758 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fce04068380 0x7fce0406a840 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fce281672f0 tx=0x7fce10008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:44.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce305fe000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562178911b50 0x562178911f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce305fe000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562178912470 0x5621788c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce305fe000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5621788c7cf0 0x5621788c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce305fe000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5621787154d0 con 0x562178911b50 2024-04-17T10:03:44.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce305fe000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5621786afbf0 con 0x562178912470 2024-04-17T10:03:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce305fe000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x562178711490 con 0x5621788c7cf0 2024-04-17T10:03:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce25e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562178911b50 0x562178911f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce25e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562178911b50 0x562178911f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49350/0 (socket says 172.21.15.32:49350) 2024-04-17T10:03:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce25e00640 1 -- 172.21.15.32:0/1099665842 learned_addr learned my addr 172.21.15.32:0/1099665842 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce1fe00640 1 --2- 172.21.15.32:0/1099665842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562178912470 0x5621788c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce26800640 1 --2- 172.21.15.32:0/1099665842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5621788c7cf0 0x5621788c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce25e00640 1 -- 172.21.15.32:0/1099665842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5621788c7cf0 msgr2=0x5621788c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce25e00640 1 --2- 172.21.15.32:0/1099665842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5621788c7cf0 0x5621788c80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce25e00640 1 -- 172.21.15.32:0/1099665842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562178912470 msgr2=0x5621788c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce25e00640 1 --2- 172.21.15.32:0/1099665842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562178912470 0x5621788c77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce25e00640 1 -- 172.21.15.32:0/1099665842 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56217869be80 con 0x562178911b50 2024-04-17T10:03:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce1fe00640 1 --2- 172.21.15.32:0/1099665842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562178912470 0x5621788c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce25e00640 1 --2- 172.21.15.32:0/1099665842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562178911b50 0x562178911f30 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fce1800afd0 tx=0x7fce18056a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7019a38006157e3b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:44.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce16a00640 1 -- 172.21.15.32:0/1099665842 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1801e040 con 0x562178911b50 2024-04-17T10:03:44.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce16a00640 1 -- 172.21.15.32:0/1099665842 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce18054d30 con 0x562178911b50 2024-04-17T10:03:44.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce16a00640 1 -- 172.21.15.32:0/1099665842 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1805d070 con 0x562178911b50 2024-04-17T10:03:44.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce305fe000 1 -- 172.21.15.32:0/1099665842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562178911b50 msgr2=0x562178911f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:44.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce305fe000 1 --2- 172.21.15.32:0/1099665842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562178911b50 0x562178911f30 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fce1800afd0 tx=0x7fce18056a30 comp rx=0 tx=0).stop 2024-04-17T10:03:44.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce305fe000 1 -- 172.21.15.32:0/1099665842 shutdown_connections 2024-04-17T10:03:44.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce305fe000 1 --2- 172.21.15.32:0/1099665842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5621788c7cf0 0x5621788c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce305fe000 1 --2- 172.21.15.32:0/1099665842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562178912470 0x5621788c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce305fe000 1 --2- 172.21.15.32:0/1099665842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562178911b50 0x562178911f30 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce305fe000 1 -- 172.21.15.32:0/1099665842 >> 172.21.15.32:0/1099665842 conn(0x5621788cd470 msgr2=0x5621788c6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:44.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce305fe000 1 -- 172.21.15.32:0/1099665842 shutdown_connections 2024-04-17T10:03:44.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce305fe000 1 -- 172.21.15.32:0/1099665842 wait complete. 2024-04-17T10:03:44.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce305fe000 1 -- start start 2024-04-17T10:03:44.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.171+0000 7fce305fe000 1 Processor -- start 2024-04-17T10:03:44.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce305fe000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5621788c7cf0 0x5621788cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce305fe000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562178911b50 0x5621788ff440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce305fe000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562178912470 0x5621788ff980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce305fe000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5621786d2a60 con 0x5621788c7cf0 2024-04-17T10:03:44.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce305fe000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5621786b9e80 con 0x562178911b50 2024-04-17T10:03:44.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce305fe000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5621786d0cf0 con 0x562178912470 2024-04-17T10:03:44.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce305fe000 10 client.0 _ll_register_callbacks cb 0x5621788b2a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:03:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce25e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5621788c7cf0 0x5621788cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce25e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5621788c7cf0 0x5621788cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49360/0 (socket says 172.21.15.32:49360) 2024-04-17T10:03:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce25e00640 1 -- 172.21.15.32:0/109166486 learned_addr learned my addr 172.21.15.32:0/109166486 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce1fe00640 1 --2- 172.21.15.32:0/109166486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562178911b50 0x5621788ff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce26800640 1 --2- 172.21.15.32:0/109166486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562178912470 0x5621788ff980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce25e00640 1 -- 172.21.15.32:0/109166486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562178912470 msgr2=0x5621788ff980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce25e00640 1 --2- 172.21.15.32:0/109166486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562178912470 0x5621788ff980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce25e00640 1 -- 172.21.15.32:0/109166486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562178911b50 msgr2=0x5621788ff440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce25e00640 1 --2- 172.21.15.32:0/109166486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562178911b50 0x5621788ff440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce25e00640 1 -- 172.21.15.32:0/109166486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621786afe40 con 0x5621788c7cf0 2024-04-17T10:03:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce25e00640 1 --2- 172.21.15.32:0/109166486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5621788c7cf0 0x5621788cc7e0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fce18056970 tx=0x7fce18000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b025b2dd356f8cc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce0b400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:03:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce0b400640 1 -- 172.21.15.32:0/109166486 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1801e070 con 0x5621788c7cf0 2024-04-17T10:03:44.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce0b400640 1 -- 172.21.15.32:0/109166486 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fce18055e80 con 0x5621788c7cf0 2024-04-17T10:03:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce305fe000 1 -- 172.21.15.32:0/109166486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce18054d30 con 0x5621788c7cf0 2024-04-17T10:03:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce0b400640 1 -- 172.21.15.32:0/109166486 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7fce180560c0 con 0x5621788c7cf0 2024-04-17T10:03:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce305fe000 10 client.9423 fetch_fsmap learned FSMap version 98 2024-04-17T10:03:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce305fe000 1 -- 172.21.15.32:0/109166486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x5621786e0e00 con 0x5621788c7cf0 2024-04-17T10:03:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce0b400640 1 -- 172.21.15.32:0/109166486 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce18054d30 con 0x5621788c7cf0 2024-04-17T10:03:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce0b400640 1 -- 172.21.15.32:0/109166486 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 98) ==== 3495+0+0 (secure 0 0 0) 0x7fce18072020 con 0x5621788c7cf0 2024-04-17T10:03:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce305fe000 10 client.9423 fetch_fsmap finished waiting for FSMap version 98 2024-04-17T10:03:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce305fe000 10 client.9423 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T10:03:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce305fe000 20 client.9423 populate_metadata read hostname 'smithi032' 2024-04-17T10:03:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce305fe000 1 --2- 172.21.15.32:0/109166486 >> [v2:172.21.15.32:6836/901447746,v1:172.21.15.32:6837/901447746] conn(0x5621789210b0 0x5621789234a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce305fe000 4 client.9423 mds_command: new command op to 9285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:03:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce305fe000 1 -- 172.21.15.32:0/109166486 --> [v2:172.21.15.32:6836/901447746,v1:172.21.15.32:6837/901447746] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621786aebc0 con 0x5621789210b0 2024-04-17T10:03:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce1fe00640 1 --2- 172.21.15.32:0/109166486 >> [v2:172.21.15.32:6836/901447746,v1:172.21.15.32:6837/901447746] conn(0x5621789210b0 0x5621789234a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce1fe00640 1 --2- 172.21.15.32:0/109166486 >> [v2:172.21.15.32:6836/901447746,v1:172.21.15.32:6837/901447746] conn(0x5621789210b0 0x5621789234a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3271b0f6c26c4371 server_cookie=31a2adb817ec1566 in_seq=0 out_seq=0 2024-04-17T10:03:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.175+0000 7fce0b400640 10 client.9423 ms_handle_connect on v2:172.21.15.32:6836/901447746 2024-04-17T10:03:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.179+0000 7fce0b400640 1 -- 172.21.15.32:0/109166486 <== mds.0 v2:172.21.15.32:6836/901447746 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5621786aebc0 con 0x5621789210b0 2024-04-17T10:03:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.179+0000 7fce0b400640 10 client.9423 handle_command_reply: tid=0 2024-04-17T10:03:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.179+0000 7fce305fe000 1 client.9423 shutdown 2024-04-17T10:03:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.179+0000 7fce305fe000 1 -- 172.21.15.32:0/109166486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5621788c7cf0 msgr2=0x5621788cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.179+0000 7fce305fe000 1 --2- 172.21.15.32:0/109166486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5621788c7cf0 0x5621788cc7e0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fce18056970 tx=0x7fce18000bd0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.179+0000 7fce305fe000 1 -- 172.21.15.32:0/109166486 shutdown_connections 2024-04-17T10:03:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.179+0000 7fce305fe000 1 --2- 172.21.15.32:0/109166486 >> [v2:172.21.15.32:6836/901447746,v1:172.21.15.32:6837/901447746] conn(0x5621789210b0 0x5621789234a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.179+0000 7fce305fe000 1 --2- 172.21.15.32:0/109166486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562178912470 0x5621788ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.179+0000 7fce305fe000 1 --2- 172.21.15.32:0/109166486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562178911b50 0x5621788ff440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.179+0000 7fce0b400640 0 client.9423 ms_handle_reset on v2:172.21.15.32:6836/901447746 2024-04-17T10:03:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.179+0000 7fce305fe000 1 --2- 172.21.15.32:0/109166486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5621788c7cf0 0x5621788cc7e0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.179+0000 7fce305fe000 1 -- 172.21.15.32:0/109166486 >> 172.21.15.32:0/109166486 conn(0x5621788cd470 msgr2=0x5621788c67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.179+0000 7fce305fe000 1 -- 172.21.15.32:0/109166486 shutdown_connections 2024-04-17T10:03:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.179+0000 7fce305fe000 1 -- 172.21.15.32:0/109166486 wait complete. 2024-04-17T10:03:44.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.179+0000 7fce305fe000 20 client.9423 trim_cache size 0 max 16384 2024-04-17T10:03:44.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.195+0000 7fce305fe000 1 Processor -- start 2024-04-17T10:03:44.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.195+0000 7fce305fe000 1 -- start start 2024-04-17T10:03:44.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.195+0000 7fce305fe000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621788c7cf0 0x5621788ff020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.195+0000 7fce305fe000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562178911b50 0x5621788ff560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.195+0000 7fce305fe000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562178912470 0x5621789224a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.195+0000 7fce305fe000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fce2813e4d0 con 0x562178911b50 2024-04-17T10:03:44.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.195+0000 7fce305fe000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fce2813d830 con 0x5621788c7cf0 2024-04-17T10:03:44.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.195+0000 7fce305fe000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fce2813de80 con 0x562178912470 2024-04-17T10:03:44.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.195+0000 7fce1fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562178911b50 0x5621788ff560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.195+0000 7fce1fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562178911b50 0x5621788ff560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49362/0 (socket says 172.21.15.32:49362) 2024-04-17T10:03:44.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.195+0000 7fce1fe00640 1 -- 172.21.15.32:0/56868990 learned_addr learned my addr 172.21.15.32:0/56868990 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:44.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.195+0000 7fce26800640 1 --2- 172.21.15.32:0/56868990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562178912470 0x5621789224a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce25e00640 1 --2- 172.21.15.32:0/56868990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621788c7cf0 0x5621788ff020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce1fe00640 1 -- 172.21.15.32:0/56868990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562178912470 msgr2=0x5621789224a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:44.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce1fe00640 1 --2- 172.21.15.32:0/56868990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562178912470 0x5621789224a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce1fe00640 1 -- 172.21.15.32:0/56868990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621788c7cf0 msgr2=0x5621788ff020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce1fe00640 1 --2- 172.21.15.32:0/56868990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621788c7cf0 0x5621788ff020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce1fe00640 1 -- 172.21.15.32:0/56868990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562178796d30 con 0x562178911b50 2024-04-17T10:03:44.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce1fe00640 1 --2- 172.21.15.32:0/56868990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562178911b50 0x5621788ff560 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fce1000cbf0 tx=0x7fce10012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1cf446d1625362 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce16000640 1 -- 172.21.15.32:0/56868990 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1000cd20 con 0x562178911b50 2024-04-17T10:03:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce16000640 1 -- 172.21.15.32:0/56868990 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce1002b030 con 0x562178911b50 2024-04-17T10:03:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce16000640 1 -- 172.21.15.32:0/56868990 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce10022a90 con 0x562178911b50 2024-04-17T10:03:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce305fe000 1 -- 172.21.15.32:0/56868990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562178911b50 msgr2=0x5621788ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce305fe000 1 --2- 172.21.15.32:0/56868990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562178911b50 0x5621788ff560 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fce1000cbf0 tx=0x7fce10012000 comp rx=0 tx=0).stop 2024-04-17T10:03:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce305fe000 1 -- 172.21.15.32:0/56868990 shutdown_connections 2024-04-17T10:03:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce305fe000 1 --2- 172.21.15.32:0/56868990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562178912470 0x5621789224a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce305fe000 1 --2- 172.21.15.32:0/56868990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562178911b50 0x5621788ff560 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce305fe000 1 --2- 172.21.15.32:0/56868990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621788c7cf0 0x5621788ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce305fe000 1 -- 172.21.15.32:0/56868990 >> 172.21.15.32:0/56868990 conn(0x5621788cd470 msgr2=0x5621788fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:44.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce305fe000 1 -- 172.21.15.32:0/56868990 shutdown_connections 2024-04-17T10:03:44.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce305fe000 1 -- 172.21.15.32:0/56868990 wait complete. 2024-04-17T10:03:44.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce305fe000 1 -- start start 2024-04-17T10:03:44.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce305fe000 1 Processor -- start 2024-04-17T10:03:44.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce305fe000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5621788c7cf0 0x5621789806b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce305fe000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562178911b50 0x562178980bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce305fe000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562178912470 0x562178925270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce305fe000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562178719b00 con 0x562178912470 2024-04-17T10:03:44.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce305fe000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5621787147e0 con 0x562178911b50 2024-04-17T10:03:44.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce305fe000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5621787226e0 con 0x5621788c7cf0 2024-04-17T10:03:44.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce305fe000 10 client.0 _ll_register_callbacks cb 0x7fce2000d050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:03:44.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce26800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562178912470 0x562178925270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce26800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562178912470 0x562178925270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49376/0 (socket says 172.21.15.32:49376) 2024-04-17T10:03:44.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce26800640 1 -- 172.21.15.32:0/1073992764 learned_addr learned my addr 172.21.15.32:0/1073992764 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:44.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce1fe00640 1 --2- 172.21.15.32:0/1073992764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562178911b50 0x562178980bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce25e00640 1 --2- 172.21.15.32:0/1073992764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5621788c7cf0 0x5621789806b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce26800640 1 -- 172.21.15.32:0/1073992764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5621788c7cf0 msgr2=0x5621789806b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:44.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce26800640 1 --2- 172.21.15.32:0/1073992764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5621788c7cf0 0x5621789806b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce26800640 1 -- 172.21.15.32:0/1073992764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562178911b50 msgr2=0x562178980bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:44.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce26800640 1 --2- 172.21.15.32:0/1073992764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562178911b50 0x562178980bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.199+0000 7fce26800640 1 -- 172.21.15.32:0/1073992764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621786a77c0 con 0x562178912470 2024-04-17T10:03:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce26800640 1 --2- 172.21.15.32:0/1073992764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562178912470 0x562178925270 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fce2000e090 tx=0x7fce2001d3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e57d2fc567d00a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce0b400640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:03:44.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce0b400640 1 -- 172.21.15.32:0/1073992764 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce200a7a90 con 0x562178912470 2024-04-17T10:03:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce0b400640 1 -- 172.21.15.32:0/1073992764 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fce200a7c30 con 0x562178912470 2024-04-17T10:03:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce305fe000 1 -- 172.21.15.32:0/1073992764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce1002b030 con 0x562178912470 2024-04-17T10:03:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce0b400640 1 -- 172.21.15.32:0/1073992764 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7fce200a7e30 con 0x562178912470 2024-04-17T10:03:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce305fe000 10 client.9429 fetch_fsmap learned FSMap version 98 2024-04-17T10:03:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce305fe000 1 -- 172.21.15.32:0/1073992764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x5621786ba650 con 0x562178912470 2024-04-17T10:03:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce0b400640 1 -- 172.21.15.32:0/1073992764 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce1002b030 con 0x562178912470 2024-04-17T10:03:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce0b400640 1 -- 172.21.15.32:0/1073992764 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 98) ==== 3495+0+0 (secure 0 0 0) 0x7fce200c1070 con 0x562178912470 2024-04-17T10:03:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce305fe000 10 client.9429 fetch_fsmap finished waiting for FSMap version 98 2024-04-17T10:03:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce305fe000 10 client.9429 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T10:03:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce305fe000 20 client.9429 populate_metadata read hostname 'smithi032' 2024-04-17T10:03:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce305fe000 1 --2- 172.21.15.32:0/1073992764 >> [v2:172.21.15.32:6836/901447746,v1:172.21.15.32:6837/901447746] conn(0x562178987510 0x562178925ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce305fe000 4 client.9429 mds_command: new command op to 9285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T10:03:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce305fe000 1 -- 172.21.15.32:0/1073992764 --> [v2:172.21.15.32:6836/901447746,v1:172.21.15.32:6837/901447746] -- command(tid 0: {"prefix": "status"}) -- 0x562178722ef0 con 0x562178987510 2024-04-17T10:03:44.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce25e00640 1 --2- 172.21.15.32:0/1073992764 >> [v2:172.21.15.32:6836/901447746,v1:172.21.15.32:6837/901447746] conn(0x562178987510 0x562178925ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce25e00640 1 --2- 172.21.15.32:0/1073992764 >> [v2:172.21.15.32:6836/901447746,v1:172.21.15.32:6837/901447746] conn(0x562178987510 0x562178925ea0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6813d6a2bc92e579 server_cookie=945dc9b2615508d4 in_seq=0 out_seq=0 2024-04-17T10:03:44.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce0b400640 10 client.9429 ms_handle_connect on v2:172.21.15.32:6836/901447746 2024-04-17T10:03:44.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce0b400640 1 -- 172.21.15.32:0/1073992764 <== mds.0 v2:172.21.15.32:6836/901447746 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562178722ef0 con 0x562178987510 2024-04-17T10:03:44.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce0b400640 10 client.9429 handle_command_reply: tid=0 2024-04-17T10:03:44.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce305fe000 1 client.9429 shutdown 2024-04-17T10:03:44.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce305fe000 1 -- 172.21.15.32:0/1073992764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562178912470 msgr2=0x562178925270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:44.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce305fe000 1 --2- 172.21.15.32:0/1073992764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562178912470 0x562178925270 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fce2000e090 tx=0x7fce2001d3f0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce305fe000 1 -- 172.21.15.32:0/1073992764 shutdown_connections 2024-04-17T10:03:44.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce305fe000 1 --2- 172.21.15.32:0/1073992764 >> [v2:172.21.15.32:6836/901447746,v1:172.21.15.32:6837/901447746] conn(0x562178987510 0x562178925ea0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce305fe000 1 --2- 172.21.15.32:0/1073992764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562178912470 0x562178925270 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce305fe000 1 --2- 172.21.15.32:0/1073992764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562178911b50 0x562178980bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce0b400640 0 client.9429 ms_handle_reset on v2:172.21.15.32:6836/901447746 2024-04-17T10:03:44.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce305fe000 1 --2- 172.21.15.32:0/1073992764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5621788c7cf0 0x5621789806b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce305fe000 1 -- 172.21.15.32:0/1073992764 >> 172.21.15.32:0/1073992764 conn(0x5621788cd470 msgr2=0x5621788c9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:44.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce305fe000 1 -- 172.21.15.32:0/1073992764 shutdown_connections 2024-04-17T10:03:44.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce305fe000 1 -- 172.21.15.32:0/1073992764 wait complete. 2024-04-17T10:03:44.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce305fe000 20 client.9429 trim_cache size 0 max 16384 2024-04-17T10:03:44.206 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:03:44.206 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "ff178df8-4c06-4367-a1e9-2e3a66d870d0", 2024-04-17T10:03:44.206 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-17T10:03:44.206 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 9285, 2024-04-17T10:03:44.206 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-17T10:03:44.206 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-17T10:03:44.206 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-17T10:03:44.207 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.9656798520000001, 2024-04-17T10:03:44.207 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 98, 2024-04-17T10:03:44.207 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 109, 2024-04-17T10:03:44.207 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 109, 2024-04-17T10:03:44.207 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 10.782461174 2024-04-17T10:03:44.207 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:03:44.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce2d200640 1 -- 172.21.15.32:0/1152154758 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fce04068380 msgr2=0x7fce0406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:44.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce2d200640 1 --2- 172.21.15.32:0/1152154758 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fce04068380 0x7fce0406a840 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fce281672f0 tx=0x7fce10008040 comp rx=0 tx=0).stop 2024-04-17T10:03:44.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce2d200640 1 -- 172.21.15.32:0/1152154758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce280b3340 msgr2=0x7fce28150620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:44.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce2d200640 1 --2- 172.21.15.32:0/1152154758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce280b3340 0x7fce28150620 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fce1800c7c0 tx=0x7fce1800cc80 comp rx=0 tx=0).stop 2024-04-17T10:03:44.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce2d200640 1 -- 172.21.15.32:0/1152154758 shutdown_connections 2024-04-17T10:03:44.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce2d200640 1 --2- 172.21.15.32:0/1152154758 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fce04068380 0x7fce0406a840 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce2d200640 1 --2- 172.21.15.32:0/1152154758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce28155e50 0x7fce2813ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce2d200640 1 --2- 172.21.15.32:0/1152154758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce28155a70 0x7fce28152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce2d200640 1 --2- 172.21.15.32:0/1152154758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce280b3340 0x7fce28150620 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce2d200640 1 -- 172.21.15.32:0/1152154758 >> 172.21.15.32:0/1152154758 conn(0x7fce2806ba30 msgr2=0x7fce28155100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:44.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce2d200640 1 -- 172.21.15.32:0/1152154758 shutdown_connections 2024-04-17T10:03:44.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.203+0000 7fce2d200640 1 -- 172.21.15.32:0/1152154758 wait complete. 2024-04-17T10:03:44.222 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T10:03:44.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.367+0000 7fa44dc00640 1 Processor -- start 2024-04-17T10:03:44.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.367+0000 7fa44dc00640 1 -- start start 2024-04-17T10:03:44.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.367+0000 7fa44dc00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa448144e90 0x7fa4480b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.367+0000 7fa44dc00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa4480b4de0 0x7fa44815baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.367+0000 7fa44dc00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa448072bb0 con 0x7fa44815bfe0 2024-04-17T10:03:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.367+0000 7fa44dc00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa44815c430 con 0x7fa4480b4de0 2024-04-17T10:03:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.367+0000 7fa44dc00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa448145270 con 0x7fa448144e90 2024-04-17T10:03:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.367+0000 7fa43e000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa4480b4de0 0x7fa44815baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.367+0000 7fa43e000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa4480b4de0 0x7fa44815baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49384/0 (socket says 172.21.15.32:49384) 2024-04-17T10:03:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.367+0000 7fa43e000640 1 -- 172.21.15.32:0/3068223271 learned_addr learned my addr 172.21.15.32:0/3068223271 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.367+0000 7fa446a00640 1 --2- 172.21.15.32:0/3068223271 >> v2:172.21.15.144:3300/0 conn(0x7fa448144e90 0x7fa4480b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.367+0000 7fa43e000640 1 -- 172.21.15.32:0/3068223271 >> v2:172.21.15.144:3300/0 conn(0x7fa448144e90 msgr2=0x7fa4480b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.367+0000 7fa43e000640 1 --2- 172.21.15.32:0/3068223271 >> v2:172.21.15.144:3300/0 conn(0x7fa448144e90 0x7fa4480b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.367+0000 7fa43e000640 1 -- 172.21.15.32:0/3068223271 >> v1:172.21.15.144:6789/0 conn(0x7fa44815bfe0 legacy=0x7fa44806ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.367+0000 7fa43e000640 1 -- 172.21.15.32:0/3068223271 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4480b3dc0 con 0x7fa4480b4de0 2024-04-17T10:03:44.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.371+0000 7fa43e000640 1 --2- 172.21.15.32:0/3068223271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4480b4de0 0x7fa44815baa0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fa44806ff30 tx=0x7fa43001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1242c0216fd77a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:44.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.371+0000 7fa446000640 1 -- 172.21.15.32:0/3068223271 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4300aa650 con 0x7fa4480b4de0 2024-04-17T10:03:44.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.371+0000 7fa446000640 1 -- 172.21.15.32:0/3068223271 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4300af070 con 0x7fa4480b4de0 2024-04-17T10:03:44.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.371+0000 7fa446000640 1 -- 172.21.15.32:0/3068223271 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4300aaa10 con 0x7fa4480b4de0 2024-04-17T10:03:44.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.371+0000 7fa44dc00640 1 -- 172.21.15.32:0/3068223271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4480b4de0 msgr2=0x7fa44815baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:44.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.371+0000 7fa44dc00640 1 --2- 172.21.15.32:0/3068223271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4480b4de0 0x7fa44815baa0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fa44806ff30 tx=0x7fa43001c4f0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.371+0000 7fa44dc00640 1 -- 172.21.15.32:0/3068223271 shutdown_connections 2024-04-17T10:03:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.371+0000 7fa44dc00640 1 --2- 172.21.15.32:0/3068223271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4480b4de0 0x7fa44815baa0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.371+0000 7fa44dc00640 1 --2- 172.21.15.32:0/3068223271 >> v2:172.21.15.144:3300/0 conn(0x7fa448144e90 0x7fa4480b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.371+0000 7fa44dc00640 1 -- 172.21.15.32:0/3068223271 >> 172.21.15.32:0/3068223271 conn(0x7fa44806ba30 msgr2=0x7fa44806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.371+0000 7fa44dc00640 1 -- 172.21.15.32:0/3068223271 shutdown_connections 2024-04-17T10:03:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.371+0000 7fa44dc00640 1 -- 172.21.15.32:0/3068223271 wait complete. 2024-04-17T10:03:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa44dc00640 1 Processor -- start 2024-04-17T10:03:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa44dc00640 1 -- start start 2024-04-17T10:03:44.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa44dc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4480b4de0 0x7fa4480cba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa44dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa448144e90 0x7fa448136d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa44dc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa44815bfe0 0x7fa44813d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa44dc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa448072670 con 0x7fa448144e90 2024-04-17T10:03:44.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa44dc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa44806f880 con 0x7fa4480b4de0 2024-04-17T10:03:44.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa44dc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa4480727f0 con 0x7fa44815bfe0 2024-04-17T10:03:44.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa43e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa448144e90 0x7fa448136d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa43e000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa448144e90 0x7fa448136d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49398/0 (socket says 172.21.15.32:49398) 2024-04-17T10:03:44.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa43e000640 1 -- 172.21.15.32:0/441117135 learned_addr learned my addr 172.21.15.32:0/441117135 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:44.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa446a00640 1 --2- 172.21.15.32:0/441117135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4480b4de0 0x7fa4480cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa447400640 1 --2- 172.21.15.32:0/441117135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa44815bfe0 0x7fa44813d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa43e000640 1 -- 172.21.15.32:0/441117135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa44815bfe0 msgr2=0x7fa44813d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:44.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa43e000640 1 --2- 172.21.15.32:0/441117135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa44815bfe0 0x7fa44813d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa43e000640 1 -- 172.21.15.32:0/441117135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4480b4de0 msgr2=0x7fa4480cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:44.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa43e000640 1 --2- 172.21.15.32:0/441117135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4480b4de0 0x7fa4480cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa43e000640 1 -- 172.21.15.32:0/441117135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa448138560 con 0x7fa448144e90 2024-04-17T10:03:44.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa447400640 1 --2- 172.21.15.32:0/441117135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa44815bfe0 0x7fa44813d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:44.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa43e000640 1 --2- 172.21.15.32:0/441117135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa448144e90 0x7fa448136d20 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fa430019100 tx=0x7fa43001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:44.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa43f400640 1 -- 172.21.15.32:0/441117135 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4300bd5f0 con 0x7fa448144e90 2024-04-17T10:03:44.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa43f400640 1 -- 172.21.15.32:0/441117135 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4300a6040 con 0x7fa448144e90 2024-04-17T10:03:44.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa44dc00640 1 -- 172.21.15.32:0/441117135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa448138840 con 0x7fa448144e90 2024-04-17T10:03:44.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa43f400640 1 -- 172.21.15.32:0/441117135 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4300bd9d0 con 0x7fa448144e90 2024-04-17T10:03:44.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.375+0000 7fa44dc00640 1 -- 172.21.15.32:0/441117135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa448138e00 con 0x7fa448144e90 2024-04-17T10:03:44.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.379+0000 7fa44dc00640 1 -- 172.21.15.32:0/441117135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa448068a90 con 0x7fa448144e90 2024-04-17T10:03:44.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.379+0000 7fa43f400640 1 -- 172.21.15.32:0/441117135 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa4300b33f0 con 0x7fa448144e90 2024-04-17T10:03:44.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.379+0000 7fa43f400640 1 --2- 172.21.15.32:0/441117135 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa424068380 0x7fa42406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:44.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.379+0000 7fa43f400640 1 -- 172.21.15.32:0/441117135 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fa430120e10 con 0x7fa448144e90 2024-04-17T10:03:44.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.379+0000 7fa446a00640 1 --2- 172.21.15.32:0/441117135 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa424068380 0x7fa42406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:44.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.379+0000 7fa446a00640 1 --2- 172.21.15.32:0/441117135 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa424068380 0x7fa42406a840 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa438009b60 tx=0x7fa438009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:44.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.383+0000 7fa43f400640 1 -- 172.21.15.32:0/441117135 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa4300c3060 con 0x7fa448144e90 2024-04-17T10:03:44.518 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:03:44.521 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:03:44.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.583+0000 7fa44dc00640 1 -- 172.21.15.32:0/441117135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa4480cff40 con 0x7fa448144e90 2024-04-17T10:03:44.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.587+0000 7fa43f400640 1 -- 172.21.15.32:0/441117135 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v98) ==== 53+0+73 (secure 0 0 0) 0x7fa430016110 con 0x7fa448144e90 2024-04-17T10:03:44.583 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T10:03:44.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.587+0000 7fa44dc00640 1 -- 172.21.15.32:0/441117135 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa424068380 msgr2=0x7fa42406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:44.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.587+0000 7fa44dc00640 1 --2- 172.21.15.32:0/441117135 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa424068380 0x7fa42406a840 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa438009b60 tx=0x7fa438009190 comp rx=0 tx=0).stop 2024-04-17T10:03:44.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.587+0000 7fa44dc00640 1 -- 172.21.15.32:0/441117135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa448144e90 msgr2=0x7fa448136d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:44.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.587+0000 7fa44dc00640 1 --2- 172.21.15.32:0/441117135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa448144e90 0x7fa448136d20 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fa430019100 tx=0x7fa43001ad30 comp rx=0 tx=0).stop 2024-04-17T10:03:44.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.587+0000 7fa44dc00640 1 -- 172.21.15.32:0/441117135 shutdown_connections 2024-04-17T10:03:44.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.587+0000 7fa44dc00640 1 --2- 172.21.15.32:0/441117135 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa424068380 0x7fa42406a840 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.587+0000 7fa44dc00640 1 --2- 172.21.15.32:0/441117135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa44815bfe0 0x7fa44813d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.587+0000 7fa44dc00640 1 --2- 172.21.15.32:0/441117135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa448144e90 0x7fa448136d20 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.587+0000 7fa44dc00640 1 --2- 172.21.15.32:0/441117135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4480b4de0 0x7fa4480cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:44.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.587+0000 7fa44dc00640 1 -- 172.21.15.32:0/441117135 >> 172.21.15.32:0/441117135 conn(0x7fa44806ba30 msgr2=0x7fa4480cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:44.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.587+0000 7fa44dc00640 1 -- 172.21.15.32:0/441117135 shutdown_connections 2024-04-17T10:03:44.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44.587+0000 7fa44dc00640 1 -- 172.21.15.32:0/441117135 wait complete. 2024-04-17T10:03:44.604 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T10:03:44.604 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T10:03:44.604 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-17T10:03:44.605 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T10:03:44.605 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T10:03:44.605 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T10:03:44.605 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T10:03:44.605 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T10:03:44.605 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:03:44.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:44.605 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-17T10:03:44.660 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:03:44.660 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:03:44.660 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:03:44.660 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:03:44.661 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-17T10:03:44.661 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:03:44.661 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:03:44.661 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-17T10:03:44.661 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-17T10:03:44.661 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T10:03:44.661 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:03:44.661 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-17T10:03:44.661 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-17T10:03:44.661 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 41829sec preferred_lft 41829sec 2024-04-17T10:03:44.661 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2024-04-17T10:03:44.661 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:03:44.661 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:03:44.661 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-17T10:03:44.662 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-17T10:03:44.662 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:03:44.662 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:03:44.662 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-17T10:03:44.662 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T10:03:44.662 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:cd:f4:aa:5b brd ff:ff:ff:ff:ff:ff 2024-04-17T10:03:44.662 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T10:03:44.662 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:03:44.662 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T10:03:44.662 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T10:03:44.662 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T10:03:44.662 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:03:44.662 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T10:03:44.663 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:03:44.663 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:03:44.663 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:03:44.663 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T10:03:44.663 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:03:44.663 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:03:44.663 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:03:44.663 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T10:03:44.663 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:03:44.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:44.664 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T10:03:44.721 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-17T10:03:44.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:44.723 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-17T10:03:44.776 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-17T10:03:44.777 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T10:03:44.777 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-17T10:03:44.777 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T10:03:44.777 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-17T10:03:44.777 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T10:03:44.777 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T10:03:44.777 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T10:03:44.777 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-17T10:03:44.777 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-17T10:03:44.778 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-17T10:03:44.778 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T10:03:44.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T10:03:44.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:03:44.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:44.915 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-17T10:03:44.924 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T10:03:44.924 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T10:03:44.925 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:03:44.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:44.925 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-17T10:03:44.991 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T10:03:44.991 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:03:44.991 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T10:03:45.039 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:03:45.040 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:03:45.040 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:03:45.091 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T10:03:45.155 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:03:45.208 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T10:03:45.209 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T10:03:45.209 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T10:03:45.209 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T10:03:45.209 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T10:03:45.209 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T10:03:45.209 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T10:03:45.209 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T10:03:45.210 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T10:03:45.210 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T10:03:45.210 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T10:03:45.210 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T10:03:45.210 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:03:45.210 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T10:03:45.210 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T10:03:45.210 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T10:03:45.210 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T10:03:45.210 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T10:03:45.210 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T10:03:45.210 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T10:03:45.211 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:03:45.211 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T10:03:45.211 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T10:03:45.211 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T10:03:45.211 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:03:45.211 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:03:45.211 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:03:45.211 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:03:45.211 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T10:03:45.211 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:03:45.211 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:03:45.211 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:03:45.211 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:03:45.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:45.212 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T10:03:45.267 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T10:03:45.268 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T10:03:45.318 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T10:03:45.338 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:03:45.388 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T10:03:45.389 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T10:03:45.389 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T10:03:45.389 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T10:03:45.389 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T10:03:45.389 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T10:03:45.389 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T10:03:45.389 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T10:03:45.389 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T10:03:45.389 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T10:03:45.389 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T10:03:45.389 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T10:03:45.389 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:03:45.389 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T10:03:45.389 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T10:03:45.390 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T10:03:45.390 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T10:03:45.390 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T10:03:45.390 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T10:03:45.390 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T10:03:45.390 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:03:45.390 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T10:03:45.390 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T10:03:45.390 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T10:03:45.390 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:03:45.390 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:03:45.390 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:03:45.390 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:03:45.390 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T10:03:45.391 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:03:45.391 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:03:45.391 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:03:45.391 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:03:45.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:45.392 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T10:03:45.413 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2024-04-17T10:03:45.415+0000 7f7e73e1a580 -1 init, newargv = 0x561363f63d80 newargc=15 2024-04-17T10:03:45.413 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[51253]: starting ceph client 2024-04-17T10:03:45.421 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[51253]: starting fuse 2024-04-17T10:03:45.422 INFO:teuthology.orchestra.run.smithi032.stdout:44 2024-04-17T10:03:45.423 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-17T10:03:45.423 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2024-04-17T10:03:45.423 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2024-04-17T10:03:45.423 DEBUG:teuthology.orchestra.run.smithi032:> import re 2024-04-17T10:03:45.423 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:03:45.423 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2024-04-17T10:03:45.423 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:03:45.423 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2024-04-17T10:03:45.423 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T10:03:45.423 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2024-04-17T10:03:45.423 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T10:03:45.423 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:03:45.423 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2024-04-17T10:03:45.424 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2024-04-17T10:03:45.424 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2024-04-17T10:03:45.424 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2024-04-17T10:03:45.424 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:03:45.424 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2024-04-17T10:03:45.424 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T10:03:45.424 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T10:03:45.424 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T10:03:45.424 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2024-04-17T10:03:45.424 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2024-04-17T10:03:45.424 DEBUG:teuthology.orchestra.run.smithi032:> return f 2024-04-17T10:03:45.424 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T10:03:45.424 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:03:45.424 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2024-04-17T10:03:45.425 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:03:45.550 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.51253.asok 2024-04-17T10:03:45.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:03:45.562 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51253.asok 2024-04-17T10:03:45.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:45.563 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51253.asok status 2024-04-17T10:03:45.735 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:03:45.735 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2024-04-17T10:03:45.735 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-17T10:03:45.735 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T10:03:45.735 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2024-04-17T10:03:45.735 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2024-04-17T10:03:45.735 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T10:03:45.735 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "51253", 2024-04-17T10:03:45.735 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2024-04-17T10:03:45.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:45.736 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2024-04-17T10:03:45.736 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2024-04-17T10:03:45.736 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 9441, 2024-04-17T10:03:45.736 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2024-04-17T10:03:45.736 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2024-04-17T10:03:45.736 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2024-04-17T10:03:45.736 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 9441 2024-04-17T10:03:45.736 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:45.736 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T10:03:45.736 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T10:03:45.736 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:03:45.737 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3774284600 2024-04-17T10:03:45.737 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:45.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:45.737 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T10:03:45.737 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T10:03:45.737 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:03:45.737 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3774284600 2024-04-17T10:03:45.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:45.737 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.9441 192.168.0.1:0/3774284600", 2024-04-17T10:03:45.737 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/3774284600", 2024-04-17T10:03:45.738 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2024-04-17T10:03:45.738 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 98, 2024-04-17T10:03:45.738 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 109, 2024-04-17T10:03:45.738 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2024-04-17T10:03:45.738 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2024-04-17T10:03:45.738 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs" 2024-04-17T10:03:45.738 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:03:45.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:45.751 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2024-04-17T10:03:45.751 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T10:03:45.761 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2024-04-17T10:03:45.762 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T10:03:45.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:45.762 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:03:45.823 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:03:45.823 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:03:45.823 DEBUG:teuthology.orchestra.run.smithi032:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T10:03:45.823 DEBUG:teuthology.orchestra.run.smithi032:> parent = os.path.join(mount_point, "parent") 2024-04-17T10:03:45.823 DEBUG:teuthology.orchestra.run.smithi032:> os.mkdir(parent) 2024-04-17T10:03:45.823 DEBUG:teuthology.orchestra.run.smithi032:> flushed = os.path.join(parent, "flushed") 2024-04-17T10:03:45.823 DEBUG:teuthology.orchestra.run.smithi032:> os.mkdir(flushed) 2024-04-17T10:03:45.824 DEBUG:teuthology.orchestra.run.smithi032:> for f in ["alpha", "bravo", "charlie"]: 2024-04-17T10:03:45.824 DEBUG:teuthology.orchestra.run.smithi032:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-04-17T10:03:45.824 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:03:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:03:45.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:45.960 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-17T10:03:46.024 INFO:teuthology.orchestra.run.smithi032.stdout:./ 2024-04-17T10:03:46.025 INFO:teuthology.orchestra.run.smithi032.stdout:./parent 2024-04-17T10:03:46.025 INFO:teuthology.orchestra.run.smithi032.stdout:./parent/flushed 2024-04-17T10:03:46.025 INFO:teuthology.orchestra.run.smithi032.stdout:./parent/flushed/charlie 2024-04-17T10:03:46.025 INFO:teuthology.orchestra.run.smithi032.stdout:./parent/flushed/alpha 2024-04-17T10:03:46.025 INFO:teuthology.orchestra.run.smithi032.stdout:./parent/flushed/bravo 2024-04-17T10:03:46.026 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:03:46.026 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:03:46.026 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:03:46.026 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:03:46.026 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-17T10:03:46.026 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:03:46.137 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-17T10:03:46.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:03:46.145 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:03:46.146 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:03:46.146 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:03:46.146 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:03:46.146 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-17T10:03:46.146 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:03:46.256 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627776 2024-04-17T10:03:46.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:03:46.265 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:03:46.266 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:03:46.266 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:03:46.266 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:03:46.266 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-17T10:03:46.266 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:03:46.373 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627777 2024-04-17T10:03:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:03:46.382 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:03:46.382 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:03:46.382 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:03:46.382 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:03:46.383 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-17T10:03:46.383 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:03:46.489 INFO:teuthology.orchestra.run.smithi032.stdout:1099511628280 2024-04-17T10:03:46.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:03:46.499 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:03:46.499 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:03:46.499 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:03:46.499 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:03:46.499 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-17T10:03:46.499 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:03:46.605 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627778 2024-04-17T10:03:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:03:46.615 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:03:46.615 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:03:46.615 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:03:46.615 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:03:46.615 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-17T10:03:46.615 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:03:46.725 INFO:teuthology.orchestra.run.smithi032.stdout:1099511628279 2024-04-17T10:03:46.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:03:46.734 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:03:46.734 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:03:46.788 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:03:46.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.975+0000 7f46b3400640 1 Processor -- start 2024-04-17T10:03:46.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.975+0000 7f46b3400640 1 -- start start 2024-04-17T10:03:46.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.979+0000 7f46b3400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46ac072500 0x7f46ac1695c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:46.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.979+0000 7f46b3400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46ac06ef70 con 0x7f46ac144f20 2024-04-17T10:03:46.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.979+0000 7f46b3400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46ac0aed20 con 0x7f46ac1481e0 2024-04-17T10:03:46.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.979+0000 7f46b3400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f46ac0b44a0 con 0x7f46ac072500 2024-04-17T10:03:46.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.979+0000 7f46b0a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f46ac144f20 0x7f46ac06e860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41530/0 (socket says 172.21.15.32:41530) 2024-04-17T10:03:46.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.979+0000 7f46b0a00640 1 -- 172.21.15.32:0/2107851539 learned_addr learned my addr 172.21.15.32:0/2107851539 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:46.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.979+0000 7f46b1400640 1 --2- 172.21.15.32:0/2107851539 >> v2:172.21.15.32:3300/0 conn(0x7f46ac072500 0x7f46ac1695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:46.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.979+0000 7f46b1400640 1 -- 172.21.15.32:0/2107851539 >> v1:172.21.15.144:6789/0 conn(0x7f46ac1481e0 legacy=0x7f46ac071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:46.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.979+0000 7f46b1400640 1 -- 172.21.15.32:0/2107851539 >> v1:172.21.15.32:6789/0 conn(0x7f46ac144f20 legacy=0x7f46ac06e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:46.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.979+0000 7f46b1400640 1 -- 172.21.15.32:0/2107851539 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ac0af000 con 0x7f46ac072500 2024-04-17T10:03:46.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.979+0000 7f46ab400640 1 -- 172.21.15.32:0/2107851539 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3158019645 0 0) 0x7f46ac06ef70 con 0x7f46ac144f20 2024-04-17T10:03:46.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.983+0000 7f46b1400640 1 --2- 172.21.15.32:0/2107851539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ac072500 0x7f46ac1695c0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009d90 tx=0x7f46a401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5558cddb9d043217 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:46.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.983+0000 7f46ab400640 1 -- 172.21.15.32:0/2107851539 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a40b2070 con 0x7f46ac072500 2024-04-17T10:03:46.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.983+0000 7f46ab400640 1 -- 172.21.15.32:0/2107851539 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46a40a58e0 con 0x7f46ac072500 2024-04-17T10:03:46.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.983+0000 7f46ab400640 1 -- 172.21.15.32:0/2107851539 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a40a5cb0 con 0x7f46ac072500 2024-04-17T10:03:46.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.983+0000 7f46b3400640 1 -- 172.21.15.32:0/2107851539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ac072500 msgr2=0x7f46ac1695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:46.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.983+0000 7f46b3400640 1 --2- 172.21.15.32:0/2107851539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ac072500 0x7f46ac1695c0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009d90 tx=0x7f46a401c930 comp rx=0 tx=0).stop 2024-04-17T10:03:46.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.983+0000 7f46b3400640 1 -- 172.21.15.32:0/2107851539 shutdown_connections 2024-04-17T10:03:46.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.983+0000 7f46b3400640 1 --2- 172.21.15.32:0/2107851539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ac072500 0x7f46ac1695c0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:46.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.983+0000 7f46b3400640 1 -- 172.21.15.32:0/2107851539 >> 172.21.15.32:0/2107851539 conn(0x7f46ac06ba30 msgr2=0x7f46ac06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:46.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.983+0000 7f46b3400640 1 -- 172.21.15.32:0/2107851539 shutdown_connections 2024-04-17T10:03:46.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.983+0000 7f46b3400640 1 -- 172.21.15.32:0/2107851539 wait complete. 2024-04-17T10:03:46.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.983+0000 7f46b3400640 1 Processor -- start 2024-04-17T10:03:46.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.983+0000 7f46b3400640 1 -- start start 2024-04-17T10:03:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.983+0000 7f46b3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ac072500 0x7f46ac13eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.983+0000 7f46b3400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46ac144f20 0x7f46ac15ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:46.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.983+0000 7f46b3400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46ac1481e0 0x7f46ac0cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:46.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.983+0000 7f46b3400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac0afdc0 con 0x7f46ac072500 2024-04-17T10:03:46.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.983+0000 7f46b3400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac16c730 con 0x7f46ac144f20 2024-04-17T10:03:46.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.983+0000 7f46b3400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f46ac0cfb10 con 0x7f46ac1481e0 2024-04-17T10:03:46.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.983+0000 7f46b0a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ac072500 0x7f46ac13eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:46.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.983+0000 7f46b0a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ac072500 0x7f46ac13eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49414/0 (socket says 172.21.15.32:49414) 2024-04-17T10:03:46.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.983+0000 7f46b0a00640 1 -- 172.21.15.32:0/2827185222 learned_addr learned my addr 172.21.15.32:0/2827185222 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:46.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.987+0000 7f46b0a00640 1 -- 172.21.15.32:0/2827185222 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46ac1481e0 msgr2=0x7f46ac0cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:46.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.987+0000 7f46b0a00640 1 --2- 172.21.15.32:0/2827185222 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46ac1481e0 0x7f46ac0cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:46.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.987+0000 7f46b0a00640 1 -- 172.21.15.32:0/2827185222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46ac144f20 msgr2=0x7f46ac15ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:46.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.987+0000 7f46b0a00640 1 --2- 172.21.15.32:0/2827185222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46ac144f20 0x7f46ac15ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:46.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.987+0000 7f46b0a00640 1 -- 172.21.15.32:0/2827185222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ac1415e0 con 0x7f46ac072500 2024-04-17T10:03:46.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.987+0000 7f46b0a00640 1 --2- 172.21.15.32:0/2827185222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ac072500 0x7f46ac13eba0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f46a00011f0 tx=0x7f46a000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:46.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.987+0000 7f46a8c00640 1 -- 172.21.15.32:0/2827185222 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0018070 con 0x7f46ac072500 2024-04-17T10:03:46.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.987+0000 7f46a8c00640 1 -- 172.21.15.32:0/2827185222 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46a00074e0 con 0x7f46ac072500 2024-04-17T10:03:46.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.987+0000 7f46a8c00640 1 -- 172.21.15.32:0/2827185222 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0004a10 con 0x7f46ac072500 2024-04-17T10:03:46.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.987+0000 7f46b3400640 1 -- 172.21.15.32:0/2827185222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ac1418c0 con 0x7f46ac072500 2024-04-17T10:03:46.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.987+0000 7f46b3400640 1 -- 172.21.15.32:0/2827185222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ac141e80 con 0x7f46ac072500 2024-04-17T10:03:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.987+0000 7f46b3400640 1 -- 172.21.15.32:0/2827185222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46ac068a90 con 0x7f46ac072500 2024-04-17T10:03:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.987+0000 7f46a8c00640 1 -- 172.21.15.32:0/2827185222 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f46a0004bb0 con 0x7f46ac072500 2024-04-17T10:03:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.987+0000 7f46a8c00640 1 --2- 172.21.15.32:0/2827185222 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f468c0683b0 0x7f468c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:46.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.987+0000 7f46a8c00640 1 -- 172.21.15.32:0/2827185222 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f46a0085c30 con 0x7f46ac072500 2024-04-17T10:03:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.991+0000 7f46abe00640 1 --2- 172.21.15.32:0/2827185222 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f468c0683b0 0x7f468c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:46.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.991+0000 7f46abe00640 1 --2- 172.21.15.32:0/2827185222 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f468c0683b0 0x7f468c06a870 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f46ac1693d0 tx=0x7f4698006bd0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:46.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:46.995+0000 7f46a8c00640 1 -- 172.21.15.32:0/2827185222 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f46a00561e0 con 0x7f46ac072500 2024-04-17T10:03:47.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.159+0000 7f46b3400640 1 -- 172.21.15.32:0/2827185222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f46ac15e870 con 0x7f46ac072500 2024-04-17T10:03:47.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.163+0000 7f46a8c00640 1 -- 172.21.15.32:0/2827185222 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+16639 (secure 0 0 0) 0x7f46a0055ae0 con 0x7f46ac072500 2024-04-17T10:03:47.160 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:03:47.160 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":109,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:03:39.241637+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:03:36.021322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-17T10:03:36.581134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:03:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.163+0000 7f46b3400640 1 -- 172.21.15.32:0/2827185222 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f468c0683b0 msgr2=0x7f468c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.163+0000 7f46b3400640 1 --2- 172.21.15.32:0/2827185222 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f468c0683b0 0x7f468c06a870 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f46ac1693d0 tx=0x7f4698006bd0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.163+0000 7f46b3400640 1 -- 172.21.15.32:0/2827185222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ac072500 msgr2=0x7f46ac13eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.163+0000 7f46b3400640 1 --2- 172.21.15.32:0/2827185222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ac072500 0x7f46ac13eba0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f46a00011f0 tx=0x7f46a000c9b0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.163+0000 7f46b3400640 1 -- 172.21.15.32:0/2827185222 shutdown_connections 2024-04-17T10:03:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.163+0000 7f46b3400640 1 --2- 172.21.15.32:0/2827185222 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f468c0683b0 0x7f468c06a870 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.163+0000 7f46b3400640 1 --2- 172.21.15.32:0/2827185222 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46ac1481e0 0x7f46ac0cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.163+0000 7f46b3400640 1 --2- 172.21.15.32:0/2827185222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46ac144f20 0x7f46ac15ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.163+0000 7f46b3400640 1 --2- 172.21.15.32:0/2827185222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46ac072500 0x7f46ac13eba0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.163+0000 7f46b3400640 1 -- 172.21.15.32:0/2827185222 >> 172.21.15.32:0/2827185222 conn(0x7f46ac06ba30 msgr2=0x7f46ac0b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:47.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.163+0000 7f46b3400640 1 -- 172.21.15.32:0/2827185222 shutdown_connections 2024-04-17T10:03:47.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.163+0000 7f46b3400640 1 -- 172.21.15.32:0/2827185222 wait complete. 2024-04-17T10:03:47.181 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:03:36.021322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:03:36.581134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:03:47.181 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:03:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.327+0000 7f6c30e00640 1 Processor -- start 2024-04-17T10:03:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.327+0000 7f6c30e00640 1 -- start start 2024-04-17T10:03:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.327+0000 7f6c30e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c2c15e7b0 0x7f6c2c06cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.327+0000 7f6c30e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c2c0d0b70 con 0x7f6c2c06dc20 2024-04-17T10:03:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.327+0000 7f6c30e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c2c0d0e50 con 0x7f6c2c0d1930 2024-04-17T10:03:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.327+0000 7f6c30e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6c2c0d14a0 con 0x7f6c2c15e7b0 2024-04-17T10:03:47.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.331+0000 7f6c29e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c2c15e7b0 0x7f6c2c06cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:47.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.331+0000 7f6c29e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c2c15e7b0 0x7f6c2c06cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49428/0 (socket says 172.21.15.32:49428) 2024-04-17T10:03:47.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.331+0000 7f6c29e00640 1 -- 172.21.15.32:0/2995090203 learned_addr learned my addr 172.21.15.32:0/2995090203 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:47.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.331+0000 7f6c2a800640 1 --1- 172.21.15.32:0/2995090203 >> v1:172.21.15.32:6789/0 conn(0x7f6c2c06dc20 0x7f6c2c0cdec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41544/0 (socket says 172.21.15.32:41544) 2024-04-17T10:03:47.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.331+0000 7f6c29e00640 1 -- 172.21.15.32:0/2995090203 >> v1:172.21.15.144:6789/0 conn(0x7f6c2c0d1930 legacy=0x7f6c2c06d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:47.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.331+0000 7f6c29e00640 1 -- 172.21.15.32:0/2995090203 >> v1:172.21.15.32:6789/0 conn(0x7f6c2c06dc20 legacy=0x7f6c2c0cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:47.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.331+0000 7f6c29e00640 1 -- 172.21.15.32:0/2995090203 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2c0d1130 con 0x7f6c2c15e7b0 2024-04-17T10:03:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.331+0000 7f6c29400640 1 -- 172.21.15.32:0/2995090203 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 239334473 0 0) 0x7f6c2c0d0b70 con 0x7f6c2c06dc20 2024-04-17T10:03:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.331+0000 7f6c29e00640 1 --2- 172.21.15.32:0/2995090203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c2c15e7b0 0x7f6c2c06cb40 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009870 tx=0x7f6c1c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237e27afd0c6eeef server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.331+0000 7f6c29400640 1 -- 172.21.15.32:0/2995090203 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c0a9400 con 0x7f6c2c15e7b0 2024-04-17T10:03:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.331+0000 7f6c29400640 1 -- 172.21.15.32:0/2995090203 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c1c0a95a0 con 0x7f6c2c15e7b0 2024-04-17T10:03:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.331+0000 7f6c29400640 1 -- 172.21.15.32:0/2995090203 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c0a9970 con 0x7f6c2c15e7b0 2024-04-17T10:03:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c30e00640 1 -- 172.21.15.32:0/2995090203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c2c15e7b0 msgr2=0x7f6c2c06cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c30e00640 1 --2- 172.21.15.32:0/2995090203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c2c15e7b0 0x7f6c2c06cb40 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009870 tx=0x7f6c1c01c520 comp rx=0 tx=0).stop 2024-04-17T10:03:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c30e00640 1 -- 172.21.15.32:0/2995090203 shutdown_connections 2024-04-17T10:03:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c30e00640 1 --2- 172.21.15.32:0/2995090203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c2c15e7b0 0x7f6c2c06cb40 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c30e00640 1 -- 172.21.15.32:0/2995090203 >> 172.21.15.32:0/2995090203 conn(0x7f6c2c0b4080 msgr2=0x7f6c2c0b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c30e00640 1 -- 172.21.15.32:0/2995090203 shutdown_connections 2024-04-17T10:03:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c30e00640 1 -- 172.21.15.32:0/2995090203 wait complete. 2024-04-17T10:03:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c30e00640 1 Processor -- start 2024-04-17T10:03:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c30e00640 1 -- start start 2024-04-17T10:03:47.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c30e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c2c06dc20 0x7f6c2c15a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:47.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c30e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c2c0d1930 0x7f6c2c1f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:47.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c30e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c2c15e7b0 0x7f6c2c1545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:47.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c30e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c06d200 con 0x7f6c2c0d1930 2024-04-17T10:03:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c30e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c06d080 con 0x7f6c2c15e7b0 2024-04-17T10:03:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c30e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6c2c09b940 con 0x7f6c2c06dc20 2024-04-17T10:03:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c21400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c2c0d1930 0x7f6c2c1f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c21400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c2c0d1930 0x7f6c2c1f8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49434/0 (socket says 172.21.15.32:49434) 2024-04-17T10:03:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c21400640 1 -- 172.21.15.32:0/3905373372 learned_addr learned my addr 172.21.15.32:0/3905373372 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c2a800640 1 --2- 172.21.15.32:0/3905373372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c2c15e7b0 0x7f6c2c1545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c29e00640 1 --2- 172.21.15.32:0/3905373372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c2c06dc20 0x7f6c2c15a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c21400640 1 -- 172.21.15.32:0/3905373372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c2c06dc20 msgr2=0x7f6c2c15a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c21400640 1 --2- 172.21.15.32:0/3905373372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c2c06dc20 0x7f6c2c15a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c21400640 1 -- 172.21.15.32:0/3905373372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c2c15e7b0 msgr2=0x7f6c2c1545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c21400640 1 --2- 172.21.15.32:0/3905373372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c2c15e7b0 0x7f6c2c1545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c21400640 1 -- 172.21.15.32:0/3905373372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2c15b010 con 0x7f6c2c0d1930 2024-04-17T10:03:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.335+0000 7f6c21400640 1 --2- 172.21.15.32:0/3905373372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c2c0d1930 0x7f6c2c1f8000 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400c440 tx=0x7f6c1400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.339+0000 7f6c22a00640 1 -- 172.21.15.32:0/3905373372 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c14016020 con 0x7f6c2c0d1930 2024-04-17T10:03:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.339+0000 7f6c22a00640 1 -- 172.21.15.32:0/3905373372 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c14012830 con 0x7f6c2c0d1930 2024-04-17T10:03:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.339+0000 7f6c22a00640 1 -- 172.21.15.32:0/3905373372 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c14012c20 con 0x7f6c2c0d1930 2024-04-17T10:03:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.339+0000 7f6c30e00640 1 -- 172.21.15.32:0/3905373372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c2c15b2f0 con 0x7f6c2c0d1930 2024-04-17T10:03:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.339+0000 7f6c30e00640 1 -- 172.21.15.32:0/3905373372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c2c15b8b0 con 0x7f6c2c0d1930 2024-04-17T10:03:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.339+0000 7f6c30e00640 1 -- 172.21.15.32:0/3905373372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c2c068a90 con 0x7f6c2c0d1930 2024-04-17T10:03:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.339+0000 7f6c22a00640 1 -- 172.21.15.32:0/3905373372 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6c14019020 con 0x7f6c2c0d1930 2024-04-17T10:03:47.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.339+0000 7f6c22a00640 1 --2- 172.21.15.32:0/3905373372 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6c08068380 0x7f6c0806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:47.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.339+0000 7f6c22a00640 1 -- 172.21.15.32:0/3905373372 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f6c14086a80 con 0x7f6c2c0d1930 2024-04-17T10:03:47.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.339+0000 7f6c29e00640 1 --2- 172.21.15.32:0/3905373372 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6c08068380 0x7f6c0806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:47.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.339+0000 7f6c29e00640 1 --2- 172.21.15.32:0/3905373372 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6c08068380 0x7f6c0806a840 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0ae000 tx=0x7f6c1c0b3040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:47.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.343+0000 7f6c22a00640 1 -- 172.21.15.32:0/3905373372 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f6c14088e20 con 0x7f6c2c0d1930 2024-04-17T10:03:47.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.515+0000 7f6c30e00640 1 -- 172.21.15.32:0/3905373372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6c2c0b6400 con 0x7f6c2c0d1930 2024-04-17T10:03:47.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.515+0000 7f6c22a00640 1 -- 172.21.15.32:0/3905373372 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) ==== 93+0+10 (secure 0 0 0) 0x7f6c14020020 con 0x7f6c2c0d1930 2024-04-17T10:03:47.513 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T10:03:47.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.515+0000 7f6c30e00640 1 -- 172.21.15.32:0/3905373372 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6c08068380 msgr2=0x7f6c0806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:47.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.515+0000 7f6c30e00640 1 --2- 172.21.15.32:0/3905373372 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6c08068380 0x7f6c0806a840 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0ae000 tx=0x7f6c1c0b3040 comp rx=0 tx=0).stop 2024-04-17T10:03:47.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.515+0000 7f6c30e00640 1 -- 172.21.15.32:0/3905373372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c2c0d1930 msgr2=0x7f6c2c1f8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:47.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.515+0000 7f6c30e00640 1 --2- 172.21.15.32:0/3905373372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c2c0d1930 0x7f6c2c1f8000 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400c440 tx=0x7f6c1400c900 comp rx=0 tx=0).stop 2024-04-17T10:03:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.515+0000 7f6c30e00640 1 -- 172.21.15.32:0/3905373372 shutdown_connections 2024-04-17T10:03:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.515+0000 7f6c30e00640 1 --2- 172.21.15.32:0/3905373372 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6c08068380 0x7f6c0806a840 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.515+0000 7f6c30e00640 1 --2- 172.21.15.32:0/3905373372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c2c15e7b0 0x7f6c2c1545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.515+0000 7f6c30e00640 1 --2- 172.21.15.32:0/3905373372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c2c0d1930 0x7f6c2c1f8000 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.515+0000 7f6c30e00640 1 --2- 172.21.15.32:0/3905373372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c2c06dc20 0x7f6c2c15a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.515+0000 7f6c30e00640 1 -- 172.21.15.32:0/3905373372 >> 172.21.15.32:0/3905373372 conn(0x7f6c2c0b4080 msgr2=0x7f6c2c15ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.515+0000 7f6c30e00640 1 -- 172.21.15.32:0/3905373372 shutdown_connections 2024-04-17T10:03:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.515+0000 7f6c30e00640 1 -- 172.21.15.32:0/3905373372 wait complete. 2024-04-17T10:03:47.534 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:03:47.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.679+0000 7f0bee000640 1 Processor -- start 2024-04-17T10:03:47.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.679+0000 7f0bee000640 1 -- start start 2024-04-17T10:03:47.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.679+0000 7f0bee000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0be815b7d0 0x7f0be806e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:47.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.679+0000 7f0bee000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0be806f7c0 0x7f0be806ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:47.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.679+0000 7f0bee000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0be806f270 0x7f0be806bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:47.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.679+0000 7f0bee000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0be80cc940 con 0x7f0be815b7d0 2024-04-17T10:03:47.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.679+0000 7f0bee000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0be80ccc20 con 0x7f0be806f270 2024-04-17T10:03:47.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.679+0000 7f0bee000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0be806e0b0 con 0x7f0be806f7c0 2024-04-17T10:03:47.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.683+0000 7f0be7400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0be815b7d0 0x7f0be806e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:47.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.683+0000 7f0be7400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0be815b7d0 0x7f0be806e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49438/0 (socket says 172.21.15.32:49438) 2024-04-17T10:03:47.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.683+0000 7f0be7400640 1 -- 172.21.15.32:0/3174794132 learned_addr learned my addr 172.21.15.32:0/3174794132 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:47.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.683+0000 7f0be7e00640 1 --2- 172.21.15.32:0/3174794132 >> v2:172.21.15.144:3300/0 conn(0x7f0be806f270 0x7f0be806bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:47.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.683+0000 7f0bdea00640 1 --2- 172.21.15.32:0/3174794132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0be806f7c0 0x7f0be806ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:47.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.683+0000 7f0be7400640 1 -- 172.21.15.32:0/3174794132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0be806f7c0 msgr2=0x7f0be806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:47.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.683+0000 7f0be7400640 1 --2- 172.21.15.32:0/3174794132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0be806f7c0 0x7f0be806ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.683+0000 7f0be7400640 1 -- 172.21.15.32:0/3174794132 >> v2:172.21.15.144:3300/0 conn(0x7f0be806f270 msgr2=0x7f0be806bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:47.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.683+0000 7f0be7400640 1 --2- 172.21.15.32:0/3174794132 >> v2:172.21.15.144:3300/0 conn(0x7f0be806f270 0x7f0be806bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.683+0000 7f0be7400640 1 -- 172.21.15.32:0/3174794132 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be806e390 con 0x7f0be815b7d0 2024-04-17T10:03:47.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.683+0000 7f0be7400640 1 --2- 172.21.15.32:0/3174794132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0be815b7d0 0x7f0be806e8b0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f0be80b44e0 tx=0x7f0bd801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f738167929dbd1b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:47.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.683+0000 7f0be6a00640 1 -- 172.21.15.32:0/3174794132 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd80a7070 con 0x7f0be815b7d0 2024-04-17T10:03:47.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.683+0000 7f0be6a00640 1 -- 172.21.15.32:0/3174794132 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bd80b0070 con 0x7f0be815b7d0 2024-04-17T10:03:47.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.683+0000 7f0be6a00640 1 -- 172.21.15.32:0/3174794132 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd80ab890 con 0x7f0be815b7d0 2024-04-17T10:03:47.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0bee000640 1 -- 172.21.15.32:0/3174794132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0be815b7d0 msgr2=0x7f0be806e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:47.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0bee000640 1 --2- 172.21.15.32:0/3174794132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0be815b7d0 0x7f0be806e8b0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f0be80b44e0 tx=0x7f0bd801c490 comp rx=0 tx=0).stop 2024-04-17T10:03:47.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0bee000640 1 -- 172.21.15.32:0/3174794132 shutdown_connections 2024-04-17T10:03:47.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0bee000640 1 --2- 172.21.15.32:0/3174794132 >> v2:172.21.15.144:3300/0 conn(0x7f0be806f270 0x7f0be806bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0bee000640 1 --2- 172.21.15.32:0/3174794132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0be806f7c0 0x7f0be806ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0bee000640 1 --2- 172.21.15.32:0/3174794132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0be815b7d0 0x7f0be806e8b0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0bee000640 1 -- 172.21.15.32:0/3174794132 >> 172.21.15.32:0/3174794132 conn(0x7f0be80b2120 msgr2=0x7f0be80b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:47.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0bee000640 1 -- 172.21.15.32:0/3174794132 shutdown_connections 2024-04-17T10:03:47.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0bee000640 1 -- 172.21.15.32:0/3174794132 wait complete. 2024-04-17T10:03:47.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0bee000640 1 Processor -- start 2024-04-17T10:03:47.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0bee000640 1 -- start start 2024-04-17T10:03:47.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0bee000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0be806f270 0x7f0be813bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:47.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0bee000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0be806f7c0 0x7f0be8157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:47.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0bee000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0be815b7d0 0x7f0be815e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:47.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0bee000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0be80cee70 con 0x7f0be815b7d0 2024-04-17T10:03:47.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0bee000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0be80cecf0 con 0x7f0be806f270 2024-04-17T10:03:47.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0bee000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0be80b3420 con 0x7f0be806f7c0 2024-04-17T10:03:47.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0be7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0be815b7d0 0x7f0be815e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:47.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0be7e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0be815b7d0 0x7f0be815e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49442/0 (socket says 172.21.15.32:49442) 2024-04-17T10:03:47.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0be7e00640 1 -- 172.21.15.32:0/3604940034 learned_addr learned my addr 172.21.15.32:0/3604940034 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:47.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0be7400640 1 --2- 172.21.15.32:0/3604940034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0be806f270 0x7f0be813bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:47.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0bdea00640 1 --2- 172.21.15.32:0/3604940034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0be806f7c0 0x7f0be8157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:47.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0be7e00640 1 -- 172.21.15.32:0/3604940034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0be806f7c0 msgr2=0x7f0be8157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:47.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0be7e00640 1 --2- 172.21.15.32:0/3604940034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0be806f7c0 0x7f0be8157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0be7e00640 1 -- 172.21.15.32:0/3604940034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0be806f270 msgr2=0x7f0be813bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:47.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0be7e00640 1 --2- 172.21.15.32:0/3604940034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0be806f270 0x7f0be813bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.687+0000 7f0be7e00640 1 -- 172.21.15.32:0/3604940034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be815d9d0 con 0x7f0be815b7d0 2024-04-17T10:03:47.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.691+0000 7f0be7e00640 1 --2- 172.21.15.32:0/3604940034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0be815b7d0 0x7f0be815e470 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f0be0007e60 tx=0x7f0be000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:47.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.691+0000 7f0bdfe00640 1 -- 172.21.15.32:0/3604940034 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be0014620 con 0x7f0be815b7d0 2024-04-17T10:03:47.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.691+0000 7f0bdfe00640 1 -- 172.21.15.32:0/3604940034 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0be0019070 con 0x7f0be815b7d0 2024-04-17T10:03:47.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.691+0000 7f0bdfe00640 1 -- 172.21.15.32:0/3604940034 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be0014a20 con 0x7f0be815b7d0 2024-04-17T10:03:47.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.691+0000 7f0bee000640 1 -- 172.21.15.32:0/3604940034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be8158280 con 0x7f0be815b7d0 2024-04-17T10:03:47.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.691+0000 7f0bee000640 1 -- 172.21.15.32:0/3604940034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be815bfb0 con 0x7f0be815b7d0 2024-04-17T10:03:47.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.691+0000 7f0bee000640 1 -- 172.21.15.32:0/3604940034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0be8068a90 con 0x7f0be815b7d0 2024-04-17T10:03:47.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.691+0000 7f0bdfe00640 1 -- 172.21.15.32:0/3604940034 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0be0014bc0 con 0x7f0be815b7d0 2024-04-17T10:03:47.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.691+0000 7f0bdfe00640 1 --2- 172.21.15.32:0/3604940034 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0bc4068380 0x7f0bc406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:47.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.691+0000 7f0bdfe00640 1 -- 172.21.15.32:0/3604940034 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f0be00870f0 con 0x7f0be815b7d0 2024-04-17T10:03:47.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.691+0000 7f0be7400640 1 --2- 172.21.15.32:0/3604940034 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0bc4068380 0x7f0bc406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:47.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.695+0000 7f0be7400640 1 --2- 172.21.15.32:0/3604940034 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0bc4068380 0x7f0bc406a840 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0bd80af000 tx=0x7f0bd80a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:47.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.695+0000 7f0bdfe00640 1 -- 172.21.15.32:0/3604940034 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0be0057050 con 0x7f0be815b7d0 2024-04-17T10:03:47.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.859+0000 7f0bee000640 1 -- 172.21.15.32:0/3604940034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0be8154ad0 con 0x7f0be815b7d0 2024-04-17T10:03:47.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.863+0000 7f0bdfe00640 1 -- 172.21.15.32:0/3604940034 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) ==== 104+0+11 (secure 0 0 0) 0x7f0be005b350 con 0x7f0be815b7d0 2024-04-17T10:03:47.860 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T10:03:47.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.863+0000 7f0bee000640 1 -- 172.21.15.32:0/3604940034 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0bc4068380 msgr2=0x7f0bc406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:47.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.863+0000 7f0bee000640 1 --2- 172.21.15.32:0/3604940034 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0bc4068380 0x7f0bc406a840 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0bd80af000 tx=0x7f0bd80a7000 comp rx=0 tx=0).stop 2024-04-17T10:03:47.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.863+0000 7f0bee000640 1 -- 172.21.15.32:0/3604940034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0be815b7d0 msgr2=0x7f0be815e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:47.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.863+0000 7f0bee000640 1 --2- 172.21.15.32:0/3604940034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0be815b7d0 0x7f0be815e470 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f0be0007e60 tx=0x7f0be000e4b0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.863+0000 7f0bee000640 1 -- 172.21.15.32:0/3604940034 shutdown_connections 2024-04-17T10:03:47.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.863+0000 7f0bee000640 1 --2- 172.21.15.32:0/3604940034 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0bc4068380 0x7f0bc406a840 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.863+0000 7f0bee000640 1 --2- 172.21.15.32:0/3604940034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0be806f7c0 0x7f0be8157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.863+0000 7f0bee000640 1 --2- 172.21.15.32:0/3604940034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0be815b7d0 0x7f0be815e470 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.863+0000 7f0bee000640 1 --2- 172.21.15.32:0/3604940034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0be806f270 0x7f0be813bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:47.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.863+0000 7f0bee000640 1 -- 172.21.15.32:0/3604940034 >> 172.21.15.32:0/3604940034 conn(0x7f0be80b2120 msgr2=0x7f0be80cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:47.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.863+0000 7f0bee000640 1 -- 172.21.15.32:0/3604940034 shutdown_connections 2024-04-17T10:03:47.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:47.863+0000 7f0bee000640 1 -- 172.21.15.32:0/3604940034 wait complete. 2024-04-17T10:03:47.881 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:03:48.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.027+0000 7f0b16600640 1 Processor -- start 2024-04-17T10:03:48.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.027+0000 7f0b16600640 1 -- start start 2024-04-17T10:03:48.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.027+0000 7f0b16600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b1006ba30 0x7f0b1006be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:48.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.027+0000 7f0b16600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0b100cf370 0x7f0b100cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:48.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.027+0000 7f0b16600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b1006c350 con 0x7f0b10160350 2024-04-17T10:03:48.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.027+0000 7f0b16600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0b100d0130 con 0x7f0b100cf370 2024-04-17T10:03:48.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.027+0000 7f0b16600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0b1015d090 con 0x7f0b1006ba30 2024-04-17T10:03:48.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.027+0000 7f0b0ea00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0b100cf370 0x7f0b100cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:48.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.027+0000 7f0b0ea00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0b100cf370 0x7f0b100cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:59188/0 (socket says 172.21.15.32:59188) 2024-04-17T10:03:48.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.027+0000 7f0b0ea00640 1 -- 172.21.15.32:0/2849002786 learned_addr learned my addr 172.21.15.32:0/2849002786 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.027+0000 7f0b0f400640 1 --2- 172.21.15.32:0/2849002786 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b1006ba30 0x7f0b1006be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.027+0000 7f0b0ea00640 1 -- 172.21.15.32:0/2849002786 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b1006ba30 msgr2=0x7f0b1006be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.027+0000 7f0b0ea00640 1 --2- 172.21.15.32:0/2849002786 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b1006ba30 0x7f0b1006be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.027+0000 7f0b0ea00640 1 -- 172.21.15.32:0/2849002786 >> v1:172.21.15.144:6789/0 conn(0x7f0b10160350 legacy=0x7f0b101607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.027+0000 7f0b0ea00640 1 -- 172.21.15.32:0/2849002786 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1015d370 con 0x7f0b100cf370 2024-04-17T10:03:48.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.031+0000 7f0b0ea00640 1 --2- 172.21.15.32:0/2849002786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b100cf370 0x7f0b100cf7e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009870 tx=0x7f0afc01c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a1af1bf757f8e58 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:48.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.031+0000 7f0b0e000640 1 -- 172.21.15.32:0/2849002786 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc0a7070 con 0x7f0b100cf370 2024-04-17T10:03:48.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.031+0000 7f0b0e000640 1 -- 172.21.15.32:0/2849002786 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0afc0b0070 con 0x7f0b100cf370 2024-04-17T10:03:48.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.031+0000 7f0b0e000640 1 -- 172.21.15.32:0/2849002786 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc0ab890 con 0x7f0b100cf370 2024-04-17T10:03:48.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.031+0000 7f0b16600640 1 -- 172.21.15.32:0/2849002786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b100cf370 msgr2=0x7f0b100cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:48.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.031+0000 7f0b16600640 1 --2- 172.21.15.32:0/2849002786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b100cf370 0x7f0b100cf7e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009870 tx=0x7f0afc01c560 comp rx=0 tx=0).stop 2024-04-17T10:03:48.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.031+0000 7f0b16600640 1 -- 172.21.15.32:0/2849002786 shutdown_connections 2024-04-17T10:03:48.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.031+0000 7f0b16600640 1 --2- 172.21.15.32:0/2849002786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b100cf370 0x7f0b100cf7e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.031+0000 7f0b16600640 1 --2- 172.21.15.32:0/2849002786 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b1006ba30 0x7f0b1006be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.031+0000 7f0b16600640 1 -- 172.21.15.32:0/2849002786 >> 172.21.15.32:0/2849002786 conn(0x7f0b100b2120 msgr2=0x7f0b100b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:48.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.031+0000 7f0b16600640 1 -- 172.21.15.32:0/2849002786 shutdown_connections 2024-04-17T10:03:48.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.031+0000 7f0b16600640 1 -- 172.21.15.32:0/2849002786 wait complete. 2024-04-17T10:03:48.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.035+0000 7f0b16600640 1 Processor -- start 2024-04-17T10:03:48.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.035+0000 7f0b16600640 1 -- start start 2024-04-17T10:03:48.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.035+0000 7f0b16600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b1006ba30 0x7f0b1013ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:48.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.035+0000 7f0b16600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b100cf370 0x7f0b1013df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:48.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.035+0000 7f0b16600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b10160350 0x7f0b1015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:48.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.035+0000 7f0b16600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1015dcf0 con 0x7f0b100cf370 2024-04-17T10:03:48.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.035+0000 7f0b16600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1015db70 con 0x7f0b1006ba30 2024-04-17T10:03:48.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.035+0000 7f0b16600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0b10146230 con 0x7f0b10160350 2024-04-17T10:03:48.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.035+0000 7f0b0ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b100cf370 0x7f0b1013df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:48.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.035+0000 7f0b0ea00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b100cf370 0x7f0b1013df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49454/0 (socket says 172.21.15.32:49454) 2024-04-17T10:03:48.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.035+0000 7f0b0ea00640 1 -- 172.21.15.32:0/1682040907 learned_addr learned my addr 172.21.15.32:0/1682040907 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:48.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.035+0000 7f0b0ea00640 1 -- 172.21.15.32:0/1682040907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b10160350 msgr2=0x7f0b1015ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:48.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.035+0000 7f0b0ea00640 1 --2- 172.21.15.32:0/1682040907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b10160350 0x7f0b1015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.035+0000 7f0b0ea00640 1 -- 172.21.15.32:0/1682040907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b1006ba30 msgr2=0x7f0b1013ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:48.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.035+0000 7f0b0ea00640 1 --2- 172.21.15.32:0/1682040907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b1006ba30 0x7f0b1013ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.035+0000 7f0b0ea00640 1 -- 172.21.15.32:0/1682040907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1013f260 con 0x7f0b100cf370 2024-04-17T10:03:48.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.035+0000 7f0b0ea00640 1 --2- 172.21.15.32:0/1682040907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b100cf370 0x7f0b1013df90 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f0afc0ad040 tx=0x7f0afc01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:48.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.035+0000 7f0b07400640 1 -- 172.21.15.32:0/1682040907 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc0be630 con 0x7f0b100cf370 2024-04-17T10:03:48.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.035+0000 7f0b07400640 1 -- 172.21.15.32:0/1682040907 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0afc0be7f0 con 0x7f0b100cf370 2024-04-17T10:03:48.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.035+0000 7f0b16600640 1 -- 172.21.15.32:0/1682040907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b1013d3a0 con 0x7f0b100cf370 2024-04-17T10:03:48.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.035+0000 7f0b07400640 1 -- 172.21.15.32:0/1682040907 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc0bebe0 con 0x7f0b100cf370 2024-04-17T10:03:48.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.035+0000 7f0b16600640 1 -- 172.21.15.32:0/1682040907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b101400e0 con 0x7f0b100cf370 2024-04-17T10:03:48.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.035+0000 7f0b16600640 1 -- 172.21.15.32:0/1682040907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b10068a90 con 0x7f0b100cf370 2024-04-17T10:03:48.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.039+0000 7f0b07400640 1 -- 172.21.15.32:0/1682040907 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0afc0b43f0 con 0x7f0b100cf370 2024-04-17T10:03:48.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.039+0000 7f0b07400640 1 --2- 172.21.15.32:0/1682040907 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0af0068380 0x7f0af006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:48.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.039+0000 7f0b07400640 1 -- 172.21.15.32:0/1682040907 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f0afc120db0 con 0x7f0b100cf370 2024-04-17T10:03:48.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.039+0000 7f0b0f400640 1 --2- 172.21.15.32:0/1682040907 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0af0068380 0x7f0af006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:48.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.039+0000 7f0b0f400640 1 --2- 172.21.15.32:0/1682040907 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0af0068380 0x7f0af006a840 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0b00006aa0 tx=0x7f0b00006a30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:48.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.043+0000 7f0b07400640 1 -- 172.21.15.32:0/1682040907 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0afc125070 con 0x7f0b100cf370 2024-04-17T10:03:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.207+0000 7f0b16600640 1 -- 172.21.15.32:0/1682040907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0b1006c850 con 0x7f0b100cf370 2024-04-17T10:03:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.207+0000 7f0b07400640 1 -- 172.21.15.32:0/1682040907 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) ==== 100+0+11 (secure 0 0 0) 0x7f0afc016150 con 0x7f0b100cf370 2024-04-17T10:03:48.205 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T10:03:48.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.207+0000 7f0b16600640 1 -- 172.21.15.32:0/1682040907 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0af0068380 msgr2=0x7f0af006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:48.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.207+0000 7f0b16600640 1 --2- 172.21.15.32:0/1682040907 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0af0068380 0x7f0af006a840 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0b00006aa0 tx=0x7f0b00006a30 comp rx=0 tx=0).stop 2024-04-17T10:03:48.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.207+0000 7f0b16600640 1 -- 172.21.15.32:0/1682040907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b100cf370 msgr2=0x7f0b1013df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.207+0000 7f0b16600640 1 --2- 172.21.15.32:0/1682040907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b100cf370 0x7f0b1013df90 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f0afc0ad040 tx=0x7f0afc01ada0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.207+0000 7f0b16600640 1 -- 172.21.15.32:0/1682040907 shutdown_connections 2024-04-17T10:03:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.207+0000 7f0b16600640 1 --2- 172.21.15.32:0/1682040907 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0af0068380 0x7f0af006a840 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.207+0000 7f0b16600640 1 --2- 172.21.15.32:0/1682040907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b10160350 0x7f0b1015ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.207+0000 7f0b16600640 1 --2- 172.21.15.32:0/1682040907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b100cf370 0x7f0b1013df90 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.207+0000 7f0b16600640 1 --2- 172.21.15.32:0/1682040907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b1006ba30 0x7f0b1013ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.207+0000 7f0b16600640 1 -- 172.21.15.32:0/1682040907 >> 172.21.15.32:0/1682040907 conn(0x7f0b100b2120 msgr2=0x7f0b1006f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.207+0000 7f0b16600640 1 -- 172.21.15.32:0/1682040907 shutdown_connections 2024-04-17T10:03:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.207+0000 7f0b16600640 1 -- 172.21.15.32:0/1682040907 wait complete. 2024-04-17T10:03:48.227 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:03:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.371+0000 7fa141800640 1 Processor -- start 2024-04-17T10:03:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.371+0000 7fa141800640 1 -- start start 2024-04-17T10:03:48.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.371+0000 7fa141800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa13c155a70 0x7fa13c15f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:48.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.371+0000 7fa141800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa13c15f870 con 0x7fa13c155e50 2024-04-17T10:03:48.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.371+0000 7fa141800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa13c15fb50 con 0x7fa13c0b3340 2024-04-17T10:03:48.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.371+0000 7fa141800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa13c0b42a0 con 0x7fa13c155a70 2024-04-17T10:03:48.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.371+0000 7fa132000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa13c155e50 0x7fa13c15d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41548/0 (socket says 172.21.15.32:41548) 2024-04-17T10:03:48.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa132000640 1 -- 172.21.15.32:0/48940154 learned_addr learned my addr 172.21.15.32:0/48940154 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa13aa00640 1 --2- 172.21.15.32:0/48940154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa13c155a70 0x7fa13c15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa13a000640 1 -- 172.21.15.32:0/48940154 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2329335821 0 0) 0x7fa13c15f870 con 0x7fa13c155e50 2024-04-17T10:03:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa13a000640 1 -- 172.21.15.32:0/48940154 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa120003530 con 0x7fa13c155e50 2024-04-17T10:03:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa13a000640 1 -- 172.21.15.32:0/48940154 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3110430444 0 0) 0x7fa13c15fb50 con 0x7fa13c0b3340 2024-04-17T10:03:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa13a000640 1 -- 172.21.15.32:0/48940154 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa13c15f870 con 0x7fa13c0b3340 2024-04-17T10:03:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa13a000640 1 -- 172.21.15.32:0/48940154 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1002167431 0 0) 0x7fa120003530 con 0x7fa13c155e50 2024-04-17T10:03:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa13a000640 1 -- 172.21.15.32:0/48940154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa13c155a70 msgr2=0x7fa13c15f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa13a000640 1 --2- 172.21.15.32:0/48940154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa13c155a70 0x7fa13c15f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa13a000640 1 -- 172.21.15.32:0/48940154 >> v1:172.21.15.144:6789/0 conn(0x7fa13c0b3340 legacy=0x7fa13c15c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa13a000640 1 -- 172.21.15.32:0/48940154 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13c15fe30 con 0x7fa13c155e50 2024-04-17T10:03:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa13a000640 1 -- 172.21.15.32:0/48940154 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7fa12c001680 con 0x7fa13c155e50 2024-04-17T10:03:48.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa13a000640 1 -- 172.21.15.32:0/48940154 >> v1:172.21.15.32:6789/0 conn(0x7fa13c155e50 legacy=0x7fa13c15d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:48.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa13a000640 1 --2- 172.21.15.32:0/48940154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa120003b80 0x7fa120003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:48.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa13a000640 1 --2- 172.21.15.32:0/48940154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa120004810 0x7fa120004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:48.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa13a000640 1 --2- 172.21.15.32:0/48940154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa13c155a70 0x7fa1200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:48.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa13a000640 1 -- 172.21.15.32:0/48940154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c0b42a0 con 0x7fa120004810 2024-04-17T10:03:48.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa13a000640 1 -- 172.21.15.32:0/48940154 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa12000bab0 con 0x7fa120003b80 2024-04-17T10:03:48.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa13a000640 1 -- 172.21.15.32:0/48940154 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa12000bcb0 con 0x7fa13c155a70 2024-04-17T10:03:48.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa132000640 1 --2- 172.21.15.32:0/48940154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa120004810 0x7fa120004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:48.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa13aa00640 1 --2- 172.21.15.32:0/48940154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa120003b80 0x7fa120003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:48.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa13b400640 1 --2- 172.21.15.32:0/48940154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa13c155a70 0x7fa1200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:48.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa132000640 1 -- 172.21.15.32:0/48940154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa13c155a70 msgr2=0x7fa1200051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:48.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa132000640 1 --2- 172.21.15.32:0/48940154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa13c155a70 0x7fa1200051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa132000640 1 -- 172.21.15.32:0/48940154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa120003b80 msgr2=0x7fa120003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:48.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa132000640 1 --2- 172.21.15.32:0/48940154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa120003b80 0x7fa120003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.375+0000 7fa132000640 1 -- 172.21.15.32:0/48940154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa120003530 con 0x7fa120004810 2024-04-17T10:03:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.379+0000 7fa132000640 1 --2- 172.21.15.32:0/48940154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa120004810 0x7fa120004c90 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fa13c156230 tx=0x7fa12c01cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4511eee426d783 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.379+0000 7fa13a000640 1 -- 172.21.15.32:0/48940154 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c007c10 con 0x7fa120004810 2024-04-17T10:03:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.379+0000 7fa13a000640 1 -- 172.21.15.32:0/48940154 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa12c0a9070 con 0x7fa120004810 2024-04-17T10:03:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.379+0000 7fa13a000640 1 -- 172.21.15.32:0/48940154 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c005d20 con 0x7fa120004810 2024-04-17T10:03:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.379+0000 7fa141800640 1 -- 172.21.15.32:0/48940154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa120004810 msgr2=0x7fa120004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.379+0000 7fa141800640 1 --2- 172.21.15.32:0/48940154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa120004810 0x7fa120004c90 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fa13c156230 tx=0x7fa12c01cac0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.379+0000 7fa13b400640 1 -- 172.21.15.32:0/48940154 reap_dead start 2024-04-17T10:03:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.379+0000 7fa141800640 1 -- 172.21.15.32:0/48940154 shutdown_connections 2024-04-17T10:03:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.379+0000 7fa141800640 1 -- 172.21.15.32:0/48940154 >> 172.21.15.32:0/48940154 conn(0x7fa13c06ba30 msgr2=0x7fa13c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.379+0000 7fa141800640 1 -- 172.21.15.32:0/48940154 shutdown_connections 2024-04-17T10:03:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.379+0000 7fa141800640 1 -- 172.21.15.32:0/48940154 wait complete. 2024-04-17T10:03:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.379+0000 7fa141800640 1 Processor -- start 2024-04-17T10:03:48.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.379+0000 7fa141800640 1 -- start start 2024-04-17T10:03:48.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.379+0000 7fa141800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa120004810 0x7fa13c0b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:48.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.379+0000 7fa141800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa13c1e5170 0x7fa13c13e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:48.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.379+0000 7fa141800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13c15a640 0x7fa13c15aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:48.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.383+0000 7fa141800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c160110 con 0x7fa13c15a640 2024-04-17T10:03:48.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.383+0000 7fa141800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c0d1250 con 0x7fa13c1e5170 2024-04-17T10:03:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.383+0000 7fa141800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa13c15d7a0 con 0x7fa120004810 2024-04-17T10:03:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.383+0000 7fa13b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13c15a640 0x7fa13c15aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.383+0000 7fa13aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa120004810 0x7fa13c0b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.383+0000 7fa13b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13c15a640 0x7fa13c15aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49478/0 (socket says 172.21.15.32:49478) 2024-04-17T10:03:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.383+0000 7fa13b400640 1 -- 172.21.15.32:0/2738882019 learned_addr learned my addr 172.21.15.32:0/2738882019 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.383+0000 7fa132000640 1 --2- 172.21.15.32:0/2738882019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa13c1e5170 0x7fa13c13e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.383+0000 7fa13b400640 1 -- 172.21.15.32:0/2738882019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa120004810 msgr2=0x7fa13c0b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.383+0000 7fa13b400640 1 --2- 172.21.15.32:0/2738882019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa120004810 0x7fa13c0b2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.383+0000 7fa13b400640 1 -- 172.21.15.32:0/2738882019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa13c1e5170 msgr2=0x7fa13c13e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.383+0000 7fa13b400640 1 --2- 172.21.15.32:0/2738882019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa13c1e5170 0x7fa13c13e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.383+0000 7fa13b400640 1 -- 172.21.15.32:0/2738882019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13c0b3150 con 0x7fa13c15a640 2024-04-17T10:03:48.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.383+0000 7fa13b400640 1 --2- 172.21.15.32:0/2738882019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13c15a640 0x7fa13c15aac0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0cd8b0 tx=0x7fa134011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:48.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.383+0000 7fa133400640 1 -- 172.21.15.32:0/2738882019 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa134010040 con 0x7fa13c15a640 2024-04-17T10:03:48.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.383+0000 7fa133400640 1 -- 172.21.15.32:0/2738882019 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa13401a070 con 0x7fa13c15a640 2024-04-17T10:03:48.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.383+0000 7fa133400640 1 -- 172.21.15.32:0/2738882019 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa134015410 con 0x7fa13c15a640 2024-04-17T10:03:48.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.383+0000 7fa141800640 1 -- 172.21.15.32:0/2738882019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa13c13ee40 con 0x7fa13c15a640 2024-04-17T10:03:48.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.383+0000 7fa141800640 1 -- 172.21.15.32:0/2738882019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa13c141e70 con 0x7fa13c15a640 2024-04-17T10:03:48.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.383+0000 7fa141800640 1 -- 172.21.15.32:0/2738882019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa13c15b000 con 0x7fa13c15a640 2024-04-17T10:03:48.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.383+0000 7fa133400640 1 -- 172.21.15.32:0/2738882019 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa1340155b0 con 0x7fa13c15a640 2024-04-17T10:03:48.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.387+0000 7fa133400640 1 --2- 172.21.15.32:0/2738882019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa1180683b0 0x7fa11806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:48.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.387+0000 7fa133400640 1 -- 172.21.15.32:0/2738882019 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fa134089cc0 con 0x7fa13c15a640 2024-04-17T10:03:48.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.387+0000 7fa13aa00640 1 --2- 172.21.15.32:0/2738882019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa1180683b0 0x7fa11806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:48.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.387+0000 7fa13aa00640 1 --2- 172.21.15.32:0/2738882019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa1180683b0 0x7fa11806a870 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa124009b00 tx=0x7fa124009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:48.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.391+0000 7fa133400640 1 -- 172.21.15.32:0/2738882019 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa134090080 con 0x7fa13c15a640 2024-04-17T10:03:48.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.567+0000 7fa141800640 1 -- 172.21.15.32:0/2738882019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa13c068a90 con 0x7fa13c15a640 2024-04-17T10:03:48.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.567+0000 7fa133400640 1 -- 172.21.15.32:0/2738882019 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v109) ==== 80+0+0 (secure 0 0 0) 0x7fa13405a1e0 con 0x7fa13c15a640 2024-04-17T10:03:48.565 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T10:03:48.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.567+0000 7fa141800640 1 -- 172.21.15.32:0/2738882019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa1180683b0 msgr2=0x7fa11806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:48.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.567+0000 7fa141800640 1 --2- 172.21.15.32:0/2738882019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa1180683b0 0x7fa11806a870 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa124009b00 tx=0x7fa124009190 comp rx=0 tx=0).stop 2024-04-17T10:03:48.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.567+0000 7fa141800640 1 -- 172.21.15.32:0/2738882019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13c15a640 msgr2=0x7fa13c15aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:48.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.567+0000 7fa141800640 1 --2- 172.21.15.32:0/2738882019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13c15a640 0x7fa13c15aac0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0cd8b0 tx=0x7fa134011f40 comp rx=0 tx=0).stop 2024-04-17T10:03:48.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.567+0000 7fa141800640 1 -- 172.21.15.32:0/2738882019 shutdown_connections 2024-04-17T10:03:48.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.567+0000 7fa141800640 1 --2- 172.21.15.32:0/2738882019 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa1180683b0 0x7fa11806a870 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.567+0000 7fa141800640 1 --2- 172.21.15.32:0/2738882019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13c15a640 0x7fa13c15aac0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.567+0000 7fa141800640 1 --2- 172.21.15.32:0/2738882019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa13c1e5170 0x7fa13c13e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.567+0000 7fa141800640 1 --2- 172.21.15.32:0/2738882019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa120004810 0x7fa13c0b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.567+0000 7fa141800640 1 -- 172.21.15.32:0/2738882019 >> 172.21.15.32:0/2738882019 conn(0x7fa13c06ba30 msgr2=0x7fa13c15d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:48.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.567+0000 7fa141800640 1 -- 172.21.15.32:0/2738882019 shutdown_connections 2024-04-17T10:03:48.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.567+0000 7fa141800640 1 -- 172.21.15.32:0/2738882019 wait complete. 2024-04-17T10:03:48.587 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:03:48.587 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:03:48.596 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:03:48.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.783+0000 7efca3c00640 1 Processor -- start 2024-04-17T10:03:48.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.783+0000 7efca3c00640 1 -- start start 2024-04-17T10:03:48.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.787+0000 7efca3c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efc9c145f30 0x7efc9c071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:48.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.787+0000 7efca3c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc9c15d8d0 0x7efc9c072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:48.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.787+0000 7efca3c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc9c0cdac0 con 0x7efc9c0cce70 2024-04-17T10:03:48.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.787+0000 7efca3c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7efc9c06f400 con 0x7efc9c145f30 2024-04-17T10:03:48.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.787+0000 7efca3c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efc9c146310 con 0x7efc9c15d8d0 2024-04-17T10:03:48.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.787+0000 7efc93e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc9c15d8d0 0x7efc9c072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:48.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.787+0000 7efca1200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efc9c145f30 0x7efc9c071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:48.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.787+0000 7efca1c00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7efc9c0cce70 0x7efc9c0cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:37174/0 (socket says 172.21.15.32:37174) 2024-04-17T10:03:48.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.787+0000 7efc93e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc9c15d8d0 0x7efc9c072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:36314/0 (socket says 172.21.15.32:36314) 2024-04-17T10:03:48.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.787+0000 7efca1c00640 1 -- 172.21.15.32:0/2173148879 learned_addr learned my addr 172.21.15.32:0/2173148879 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:48.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.787+0000 7efc93e00640 1 -- 172.21.15.32:0/2173148879 >> v2:172.21.15.144:3300/0 conn(0x7efc9c145f30 msgr2=0x7efc9c071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:48.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.787+0000 7efc93e00640 1 --2- 172.21.15.32:0/2173148879 >> v2:172.21.15.144:3300/0 conn(0x7efc9c145f30 0x7efc9c071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.787+0000 7efc93e00640 1 -- 172.21.15.32:0/2173148879 >> v1:172.21.15.144:6789/0 conn(0x7efc9c0cce70 legacy=0x7efc9c0cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:48.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.787+0000 7efc93e00640 1 -- 172.21.15.32:0/2173148879 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9c15cdf0 con 0x7efc9c15d8d0 2024-04-17T10:03:48.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.791+0000 7efc93e00640 1 --2- 172.21.15.32:0/2173148879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc9c15d8d0 0x7efc9c072330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efc840097c0 tx=0x7efc8401ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=590b96b72b7ad97b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:48.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.791+0000 7efc9be00640 1 -- 172.21.15.32:0/2173148879 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc840b5070 con 0x7efc9c15d8d0 2024-04-17T10:03:48.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.791+0000 7efc9be00640 1 -- 172.21.15.32:0/2173148879 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc840a8990 con 0x7efc9c15d8d0 2024-04-17T10:03:48.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.791+0000 7efc9be00640 1 -- 172.21.15.32:0/2173148879 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc840a8d60 con 0x7efc9c15d8d0 2024-04-17T10:03:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.791+0000 7efca3c00640 1 -- 172.21.15.32:0/2173148879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc9c15d8d0 msgr2=0x7efc9c072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.791+0000 7efca3c00640 1 --2- 172.21.15.32:0/2173148879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc9c15d8d0 0x7efc9c072330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efc840097c0 tx=0x7efc8401ec90 comp rx=0 tx=0).stop 2024-04-17T10:03:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.791+0000 7efca3c00640 1 -- 172.21.15.32:0/2173148879 shutdown_connections 2024-04-17T10:03:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.791+0000 7efca3c00640 1 --2- 172.21.15.32:0/2173148879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc9c15d8d0 0x7efc9c072330 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.791+0000 7efca3c00640 1 --2- 172.21.15.32:0/2173148879 >> v2:172.21.15.144:3300/0 conn(0x7efc9c145f30 0x7efc9c071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.791+0000 7efca3c00640 1 -- 172.21.15.32:0/2173148879 >> 172.21.15.32:0/2173148879 conn(0x7efc9c06ba30 msgr2=0x7efc9c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.791+0000 7efca3c00640 1 -- 172.21.15.32:0/2173148879 shutdown_connections 2024-04-17T10:03:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.791+0000 7efca3c00640 1 -- 172.21.15.32:0/2173148879 wait complete. 2024-04-17T10:03:48.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.791+0000 7efca3c00640 1 Processor -- start 2024-04-17T10:03:48.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.791+0000 7efca3c00640 1 -- start start 2024-04-17T10:03:48.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efca3c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc9c0cce70 0x7efc9c0b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:48.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efca3c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc9c145f30 0x7efc9c1e7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:48.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efca3c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc9c15d8d0 0x7efc9c0a94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:48.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efca3c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c15cb90 con 0x7efc9c145f30 2024-04-17T10:03:48.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efca3c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c15d440 con 0x7efc9c0cce70 2024-04-17T10:03:48.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efca3c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efc9c1601d0 con 0x7efc9c15d8d0 2024-04-17T10:03:48.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efc93e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc9c145f30 0x7efc9c1e7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:48.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efc93e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc9c145f30 0x7efc9c1e7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49490/0 (socket says 172.21.15.32:49490) 2024-04-17T10:03:48.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efc93e00640 1 -- 172.21.15.32:0/2970742192 learned_addr learned my addr 172.21.15.32:0/2970742192 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:48.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efca1200640 1 --2- 172.21.15.32:0/2970742192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc9c0cce70 0x7efc9c0b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:48.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efca1c00640 1 --2- 172.21.15.32:0/2970742192 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc9c15d8d0 0x7efc9c0a94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:48.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efc93e00640 1 -- 172.21.15.32:0/2970742192 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc9c15d8d0 msgr2=0x7efc9c0a94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:48.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efc93e00640 1 --2- 172.21.15.32:0/2970742192 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc9c15d8d0 0x7efc9c0a94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efc93e00640 1 -- 172.21.15.32:0/2970742192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc9c0cce70 msgr2=0x7efc9c0b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:48.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efc93e00640 1 --2- 172.21.15.32:0/2970742192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc9c0cce70 0x7efc9c0b14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efc93e00640 1 -- 172.21.15.32:0/2970742192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9c0abdb0 con 0x7efc9c145f30 2024-04-17T10:03:48.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efca1c00640 1 --2- 172.21.15.32:0/2970742192 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc9c15d8d0 0x7efc9c0a94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:48.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efc93e00640 1 --2- 172.21.15.32:0/2970742192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc9c145f30 0x7efc9c1e7840 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7efc840b3040 tx=0x7efc8401ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:48.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efc99600640 1 -- 172.21.15.32:0/2970742192 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc840bf070 con 0x7efc9c145f30 2024-04-17T10:03:48.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efc99600640 1 -- 172.21.15.32:0/2970742192 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc840b0720 con 0x7efc9c145f30 2024-04-17T10:03:48.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efc99600640 1 -- 172.21.15.32:0/2970742192 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc840b5040 con 0x7efc9c145f30 2024-04-17T10:03:48.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efca3c00640 1 -- 172.21.15.32:0/2970742192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc9c136900 con 0x7efc9c145f30 2024-04-17T10:03:48.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efca3c00640 1 -- 172.21.15.32:0/2970742192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc9c137140 con 0x7efc9c145f30 2024-04-17T10:03:48.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.795+0000 7efca3c00640 1 -- 172.21.15.32:0/2970742192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc9c068a90 con 0x7efc9c145f30 2024-04-17T10:03:48.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.799+0000 7efc99600640 1 -- 172.21.15.32:0/2970742192 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7efc840b0b10 con 0x7efc9c145f30 2024-04-17T10:03:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.799+0000 7efc99600640 1 --2- 172.21.15.32:0/2970742192 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efc78068380 0x7efc7806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.799+0000 7efc99600640 1 -- 172.21.15.32:0/2970742192 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7efc841231d0 con 0x7efc9c145f30 2024-04-17T10:03:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.799+0000 7efca1200640 1 --2- 172.21.15.32:0/2970742192 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efc78068380 0x7efc7806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.799+0000 7efca1200640 1 --2- 172.21.15.32:0/2970742192 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efc78068380 0x7efc7806a840 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7efc8c009be0 tx=0x7efc8c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:48.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.803+0000 7efc99600640 1 -- 172.21.15.32:0/2970742192 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7efc9c136900 con 0x7efc9c145f30 2024-04-17T10:03:48.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.967+0000 7efca3c00640 1 -- 172.21.15.32:0/2970742192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc9c072f90 con 0x7efc9c145f30 2024-04-17T10:03:48.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.967+0000 7efc99600640 1 -- 172.21.15.32:0/2970742192 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+16639 (secure 0 0 0) 0x7efc840187f0 con 0x7efc9c145f30 2024-04-17T10:03:48.966 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:03:48.966 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":109,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:03:39.241637+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:03:36.021322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-17T10:03:36.581134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:03:48.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.971+0000 7efca3c00640 1 -- 172.21.15.32:0/2970742192 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efc78068380 msgr2=0x7efc7806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:48.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.971+0000 7efca3c00640 1 --2- 172.21.15.32:0/2970742192 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efc78068380 0x7efc7806a840 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7efc8c009be0 tx=0x7efc8c009190 comp rx=0 tx=0).stop 2024-04-17T10:03:48.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.971+0000 7efca3c00640 1 -- 172.21.15.32:0/2970742192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc9c145f30 msgr2=0x7efc9c1e7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:48.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.971+0000 7efca3c00640 1 --2- 172.21.15.32:0/2970742192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc9c145f30 0x7efc9c1e7840 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7efc840b3040 tx=0x7efc8401ba30 comp rx=0 tx=0).stop 2024-04-17T10:03:48.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.971+0000 7efca3c00640 1 -- 172.21.15.32:0/2970742192 shutdown_connections 2024-04-17T10:03:48.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.971+0000 7efca3c00640 1 --2- 172.21.15.32:0/2970742192 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7efc78068380 0x7efc7806a840 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.971+0000 7efca3c00640 1 --2- 172.21.15.32:0/2970742192 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc9c15d8d0 0x7efc9c0a94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.971+0000 7efca3c00640 1 --2- 172.21.15.32:0/2970742192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc9c145f30 0x7efc9c1e7840 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.971+0000 7efca3c00640 1 --2- 172.21.15.32:0/2970742192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc9c0cce70 0x7efc9c0b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:48.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.971+0000 7efca3c00640 1 -- 172.21.15.32:0/2970742192 >> 172.21.15.32:0/2970742192 conn(0x7efc9c06ba30 msgr2=0x7efc9c15efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:48.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.971+0000 7efca3c00640 1 -- 172.21.15.32:0/2970742192 shutdown_connections 2024-04-17T10:03:48.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:48.971+0000 7efca3c00640 1 -- 172.21.15.32:0/2970742192 wait complete. 2024-04-17T10:03:48.988 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:03:36.021322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:03:36.581134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:03:48.989 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:03:49.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.135+0000 7ff0f2a00640 1 Processor -- start 2024-04-17T10:03:49.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.135+0000 7ff0f2a00640 1 -- start start 2024-04-17T10:03:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.135+0000 7ff0f2a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0ec15f560 0x7ff0ec06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.135+0000 7ff0f2a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0ec0b0780 con 0x7ff0ec144850 2024-04-17T10:03:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.135+0000 7ff0f2a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0ec0af9c0 con 0x7ff0ec144470 2024-04-17T10:03:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.135+0000 7ff0f2a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff0ec0b02f0 con 0x7ff0ec15f560 2024-04-17T10:03:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.135+0000 7ff0f0a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff0ec144850 0x7ff0ec0b3d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41562/0 (socket says 172.21.15.32:41562) 2024-04-17T10:03:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.135+0000 7ff0f0a00640 1 -- 172.21.15.32:0/1380448737 learned_addr learned my addr 172.21.15.32:0/1380448737 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.135+0000 7ff0e3400640 1 --2- 172.21.15.32:0/1380448737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0ec15f560 0x7ff0ec06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.135+0000 7ff0eb400640 1 -- 172.21.15.32:0/1380448737 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 951314142 0 0) 0x7ff0ec0b0780 con 0x7ff0ec144850 2024-04-17T10:03:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.135+0000 7ff0eb400640 1 -- 172.21.15.32:0/1380448737 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0d0003450 con 0x7ff0ec144850 2024-04-17T10:03:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.135+0000 7ff0eb400640 1 -- 172.21.15.32:0/1380448737 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3654085723 0 0) 0x7ff0ec0af9c0 con 0x7ff0ec144470 2024-04-17T10:03:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.135+0000 7ff0e3400640 1 -- 172.21.15.32:0/1380448737 >> v1:172.21.15.144:6789/0 conn(0x7ff0ec144470 legacy=0x7ff0ec0b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.135+0000 7ff0e3400640 1 -- 172.21.15.32:0/1380448737 >> v1:172.21.15.32:6789/0 conn(0x7ff0ec144850 legacy=0x7ff0ec0b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:49.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.135+0000 7ff0e3400640 1 -- 172.21.15.32:0/1380448737 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ec0afca0 con 0x7ff0ec15f560 2024-04-17T10:03:49.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.139+0000 7ff0e3400640 1 --2- 172.21.15.32:0/1380448737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0ec15f560 0x7ff0ec06ca40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40097f0 tx=0x7ff0d401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ac2d9b171b91ee9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:49.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.139+0000 7ff0eb400640 1 -- 172.21.15.32:0/1380448737 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d40b0070 con 0x7ff0ec15f560 2024-04-17T10:03:49.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.139+0000 7ff0eb400640 1 -- 172.21.15.32:0/1380448737 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0d40ab600 con 0x7ff0ec15f560 2024-04-17T10:03:49.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.139+0000 7ff0eb400640 1 -- 172.21.15.32:0/1380448737 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d40ab9d0 con 0x7ff0ec15f560 2024-04-17T10:03:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.139+0000 7ff0f2a00640 1 -- 172.21.15.32:0/1380448737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0ec15f560 msgr2=0x7ff0ec06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.139+0000 7ff0f2a00640 1 --2- 172.21.15.32:0/1380448737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0ec15f560 0x7ff0ec06ca40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40097f0 tx=0x7ff0d401c4e0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.139+0000 7ff0f2a00640 1 -- 172.21.15.32:0/1380448737 shutdown_connections 2024-04-17T10:03:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.139+0000 7ff0f2a00640 1 --2- 172.21.15.32:0/1380448737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0ec15f560 0x7ff0ec06ca40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.139+0000 7ff0f2a00640 1 -- 172.21.15.32:0/1380448737 >> 172.21.15.32:0/1380448737 conn(0x7ff0ec072670 msgr2=0x7ff0ec072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.139+0000 7ff0f2a00640 1 -- 172.21.15.32:0/1380448737 shutdown_connections 2024-04-17T10:03:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.139+0000 7ff0f2a00640 1 -- 172.21.15.32:0/1380448737 wait complete. 2024-04-17T10:03:49.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0f2a00640 1 Processor -- start 2024-04-17T10:03:49.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0f2a00640 1 -- start start 2024-04-17T10:03:49.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0f2a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0ec144470 0x7ff0ec141160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:49.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0f2a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0ec144850 0x7ff0ec154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:49.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0f2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0ec13f650 0x7ff0ec1434b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:49.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0f2a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec0b4830 con 0x7ff0ec13f650 2024-04-17T10:03:49.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0f2a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec0b46b0 con 0x7ff0ec144470 2024-04-17T10:03:49.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0f2a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff0ec0b49b0 con 0x7ff0ec144850 2024-04-17T10:03:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0f0a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0ec13f650 0x7ff0ec1434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0f0a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0ec13f650 0x7ff0ec1434b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49502/0 (socket says 172.21.15.32:49502) 2024-04-17T10:03:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0f0a00640 1 -- 172.21.15.32:0/3066167962 learned_addr learned my addr 172.21.15.32:0/3066167962 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0e3400640 1 --2- 172.21.15.32:0/3066167962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0ec144850 0x7ff0ec154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0ebe00640 1 --2- 172.21.15.32:0/3066167962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0ec144470 0x7ff0ec141160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0f0a00640 1 -- 172.21.15.32:0/3066167962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0ec144850 msgr2=0x7ff0ec154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0f0a00640 1 --2- 172.21.15.32:0/3066167962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0ec144850 0x7ff0ec154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0f0a00640 1 -- 172.21.15.32:0/3066167962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0ec144470 msgr2=0x7ff0ec141160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0f0a00640 1 --2- 172.21.15.32:0/3066167962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0ec144470 0x7ff0ec141160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0f0a00640 1 -- 172.21.15.32:0/3066167962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ec154060 con 0x7ff0ec13f650 2024-04-17T10:03:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0e3400640 1 --2- 172.21.15.32:0/3066167962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0ec144850 0x7ff0ec154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:03:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0ebe00640 1 --2- 172.21.15.32:0/3066167962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0ec144470 0x7ff0ec141160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:03:49.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0f0a00640 1 --2- 172.21.15.32:0/3066167962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0ec13f650 0x7ff0ec1434b0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400cbe0 tx=0x7ff0e400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:49.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0e8c00640 1 -- 172.21.15.32:0/3066167962 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4017070 con 0x7ff0ec13f650 2024-04-17T10:03:49.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0e8c00640 1 -- 172.21.15.32:0/3066167962 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0e400ad40 con 0x7ff0ec13f650 2024-04-17T10:03:49.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0e8c00640 1 -- 172.21.15.32:0/3066167962 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4012600 con 0x7ff0ec13f650 2024-04-17T10:03:49.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0f2a00640 1 -- 172.21.15.32:0/3066167962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ec154340 con 0x7ff0ec13f650 2024-04-17T10:03:49.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.143+0000 7ff0f2a00640 1 -- 172.21.15.32:0/3066167962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ec1439f0 con 0x7ff0ec13f650 2024-04-17T10:03:49.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.147+0000 7ff0f2a00640 1 -- 172.21.15.32:0/3066167962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0ec068a90 con 0x7ff0ec13f650 2024-04-17T10:03:49.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.147+0000 7ff0e8c00640 1 -- 172.21.15.32:0/3066167962 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff0e401a070 con 0x7ff0ec13f650 2024-04-17T10:03:49.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.147+0000 7ff0e8c00640 1 --2- 172.21.15.32:0/3066167962 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff0c8068380 0x7ff0c806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:49.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.147+0000 7ff0e8c00640 1 -- 172.21.15.32:0/3066167962 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7ff0e4086740 con 0x7ff0ec13f650 2024-04-17T10:03:49.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.147+0000 7ff0ebe00640 1 --2- 172.21.15.32:0/3066167962 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff0c8068380 0x7ff0c806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:49.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.147+0000 7ff0ebe00640 1 --2- 172.21.15.32:0/3066167962 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff0c8068380 0x7ff0c806a840 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc000f70 tx=0x7ff0dc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:49.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.151+0000 7ff0e8c00640 1 -- 172.21.15.32:0/3066167962 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff0e4056610 con 0x7ff0ec13f650 2024-04-17T10:03:49.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.319+0000 7ff0f2a00640 1 -- 172.21.15.32:0/3066167962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff0ec06fd30 con 0x7ff0ec13f650 2024-04-17T10:03:49.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.319+0000 7ff0e8c00640 1 -- 172.21.15.32:0/3066167962 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) ==== 93+0+10 (secure 0 0 0) 0x7ff0e405a910 con 0x7ff0ec13f650 2024-04-17T10:03:49.316 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T10:03:49.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.319+0000 7ff0f2a00640 1 -- 172.21.15.32:0/3066167962 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff0c8068380 msgr2=0x7ff0c806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:49.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.319+0000 7ff0f2a00640 1 --2- 172.21.15.32:0/3066167962 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff0c8068380 0x7ff0c806a840 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc000f70 tx=0x7ff0dc009040 comp rx=0 tx=0).stop 2024-04-17T10:03:49.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.319+0000 7ff0f2a00640 1 -- 172.21.15.32:0/3066167962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0ec13f650 msgr2=0x7ff0ec1434b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:49.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.319+0000 7ff0f2a00640 1 --2- 172.21.15.32:0/3066167962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0ec13f650 0x7ff0ec1434b0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400cbe0 tx=0x7ff0e400a430 comp rx=0 tx=0).stop 2024-04-17T10:03:49.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.319+0000 7ff0f2a00640 1 -- 172.21.15.32:0/3066167962 shutdown_connections 2024-04-17T10:03:49.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.319+0000 7ff0f2a00640 1 --2- 172.21.15.32:0/3066167962 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff0c8068380 0x7ff0c806a840 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.319+0000 7ff0f2a00640 1 --2- 172.21.15.32:0/3066167962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0ec144850 0x7ff0ec154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.319+0000 7ff0f2a00640 1 --2- 172.21.15.32:0/3066167962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0ec13f650 0x7ff0ec1434b0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.319+0000 7ff0f2a00640 1 --2- 172.21.15.32:0/3066167962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0ec144470 0x7ff0ec141160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.319+0000 7ff0f2a00640 1 -- 172.21.15.32:0/3066167962 >> 172.21.15.32:0/3066167962 conn(0x7ff0ec072670 msgr2=0x7ff0ec06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:49.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.319+0000 7ff0f2a00640 1 -- 172.21.15.32:0/3066167962 shutdown_connections 2024-04-17T10:03:49.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.319+0000 7ff0f2a00640 1 -- 172.21.15.32:0/3066167962 wait complete. 2024-04-17T10:03:49.338 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:03:49.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.483+0000 7fd4b5600640 1 Processor -- start 2024-04-17T10:03:49.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.483+0000 7fd4b5600640 1 -- start start 2024-04-17T10:03:49.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.483+0000 7fd4b5600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd4b006ba30 0x7fd4b006be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:49.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.483+0000 7fd4b5600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4b00cf370 0x7fd4b00cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:49.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4b5600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4b006c350 con 0x7fd4b0160350 2024-04-17T10:03:49.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4b5600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd4b00d0130 con 0x7fd4b006ba30 2024-04-17T10:03:49.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4b5600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd4b015d090 con 0x7fd4b00cf370 2024-04-17T10:03:49.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4af400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd4b0160350 0x7fd4b01607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41568/0 (socket says 172.21.15.32:41568) 2024-04-17T10:03:49.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4af400640 1 -- 172.21.15.32:0/4245517011 learned_addr learned my addr 172.21.15.32:0/4245517011 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4a6000640 1 --2- 172.21.15.32:0/4245517011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4b00cf370 0x7fd4b00cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4aea00640 1 --2- 172.21.15.32:0/4245517011 >> v2:172.21.15.144:3300/0 conn(0x7fd4b006ba30 0x7fd4b006be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4ae000640 1 -- 172.21.15.32:0/4245517011 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1131454353 0 0) 0x7fd4b006c350 con 0x7fd4b0160350 2024-04-17T10:03:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4ae000640 1 -- 172.21.15.32:0/4245517011 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd494003530 con 0x7fd4b0160350 2024-04-17T10:03:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4ae000640 1 -- 172.21.15.32:0/4245517011 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 668396360 0 0) 0x7fd494003530 con 0x7fd4b0160350 2024-04-17T10:03:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4ae000640 1 -- 172.21.15.32:0/4245517011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4b00cf370 msgr2=0x7fd4b00cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4ae000640 1 --2- 172.21.15.32:0/4245517011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4b00cf370 0x7fd4b00cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4ae000640 1 -- 172.21.15.32:0/4245517011 >> v2:172.21.15.144:3300/0 conn(0x7fd4b006ba30 msgr2=0x7fd4b006be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:49.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4ae000640 1 --2- 172.21.15.32:0/4245517011 >> v2:172.21.15.144:3300/0 conn(0x7fd4b006ba30 0x7fd4b006be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4ae000640 1 -- 172.21.15.32:0/4245517011 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b015d370 con 0x7fd4b0160350 2024-04-17T10:03:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4ae000640 1 -- 172.21.15.32:0/4245517011 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7fd4a8001680 con 0x7fd4b0160350 2024-04-17T10:03:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4ae000640 1 -- 172.21.15.32:0/4245517011 >> v1:172.21.15.32:6789/0 conn(0x7fd4b0160350 legacy=0x7fd4b01607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4ae000640 1 --2- 172.21.15.32:0/4245517011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4b00cf370 0x7fd4940039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4ae000640 1 --2- 172.21.15.32:0/4245517011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd494003f90 0x7fd494004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4ae000640 1 --2- 172.21.15.32:0/4245517011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd494004d50 0x7fd4940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4ae000640 1 -- 172.21.15.32:0/4245517011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b00d0130 con 0x7fd494003f90 2024-04-17T10:03:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4ae000640 1 -- 172.21.15.32:0/4245517011 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b015d090 con 0x7fd494004d50 2024-04-17T10:03:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4ae000640 1 -- 172.21.15.32:0/4245517011 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd49400b9d0 con 0x7fd4b00cf370 2024-04-17T10:03:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4a6000640 1 --2- 172.21.15.32:0/4245517011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd494003f90 0x7fd494004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4aea00640 1 --2- 172.21.15.32:0/4245517011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4b00cf370 0x7fd4940039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4af400640 1 --2- 172.21.15.32:0/4245517011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd494004d50 0x7fd4940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4a6000640 1 -- 172.21.15.32:0/4245517011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4b00cf370 msgr2=0x7fd4940039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:49.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4a6000640 1 --2- 172.21.15.32:0/4245517011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4b00cf370 0x7fd4940039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4a6000640 1 -- 172.21.15.32:0/4245517011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd494004d50 msgr2=0x7fd4940051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:49.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4a6000640 1 --2- 172.21.15.32:0/4245517011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd494004d50 0x7fd4940051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4a6000640 1 -- 172.21.15.32:0/4245517011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd494003530 con 0x7fd494003f90 2024-04-17T10:03:49.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.487+0000 7fd4af400640 1 --2- 172.21.15.32:0/4245517011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd494004d50 0x7fd4940051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:49.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.491+0000 7fd4a6000640 1 --2- 172.21.15.32:0/4245517011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd494003f90 0x7fd494004810 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fd498009b10 tx=0x7fd49801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abf36eeb4629b96 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:49.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.491+0000 7fd4ae000640 1 -- 172.21.15.32:0/4245517011 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4980b2070 con 0x7fd494003f90 2024-04-17T10:03:49.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.491+0000 7fd4ae000640 1 -- 172.21.15.32:0/4245517011 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4980a5550 con 0x7fd494003f90 2024-04-17T10:03:49.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.491+0000 7fd4ae000640 1 -- 172.21.15.32:0/4245517011 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4980096b0 con 0x7fd494003f90 2024-04-17T10:03:49.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.491+0000 7fd4b5600640 1 -- 172.21.15.32:0/4245517011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd494003f90 msgr2=0x7fd494004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:49.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.491+0000 7fd4b5600640 1 --2- 172.21.15.32:0/4245517011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd494003f90 0x7fd494004810 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fd498009b10 tx=0x7fd49801ca70 comp rx=0 tx=0).stop 2024-04-17T10:03:49.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.491+0000 7fd4af400640 1 -- 172.21.15.32:0/4245517011 reap_dead start 2024-04-17T10:03:49.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.491+0000 7fd4b5600640 1 -- 172.21.15.32:0/4245517011 shutdown_connections 2024-04-17T10:03:49.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.491+0000 7fd4b5600640 1 -- 172.21.15.32:0/4245517011 >> 172.21.15.32:0/4245517011 conn(0x7fd4b00b2120 msgr2=0x7fd4b00b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:49.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.491+0000 7fd4b5600640 1 -- 172.21.15.32:0/4245517011 shutdown_connections 2024-04-17T10:03:49.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.491+0000 7fd4b5600640 1 -- 172.21.15.32:0/4245517011 wait complete. 2024-04-17T10:03:49.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.491+0000 7fd4b5600640 1 Processor -- start 2024-04-17T10:03:49.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.491+0000 7fd4b5600640 1 -- start start 2024-04-17T10:03:49.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4b5600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd494003f90 0x7fd4b00b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:49.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4b5600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4b0138a70 0x7fd4b0138ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:49.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4b5600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4b013a830 0x7fd4b013acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:49.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4b5600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b015db70 con 0x7fd4b013a830 2024-04-17T10:03:49.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4b5600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b015cd00 con 0x7fd4b0138a70 2024-04-17T10:03:49.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4b5600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd4b015dcf0 con 0x7fd494003f90 2024-04-17T10:03:49.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4af400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4b013a830 0x7fd4b013acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:49.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4af400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4b013a830 0x7fd4b013acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49514/0 (socket says 172.21.15.32:49514) 2024-04-17T10:03:49.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4af400640 1 -- 172.21.15.32:0/1437720726 learned_addr learned my addr 172.21.15.32:0/1437720726 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:49.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4a6000640 1 --2- 172.21.15.32:0/1437720726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4b0138a70 0x7fd4b0138ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:49.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4aea00640 1 --2- 172.21.15.32:0/1437720726 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd494003f90 0x7fd4b00b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:49.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4af400640 1 -- 172.21.15.32:0/1437720726 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd494003f90 msgr2=0x7fd4b00b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:49.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4af400640 1 --2- 172.21.15.32:0/1437720726 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd494003f90 0x7fd4b00b13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4af400640 1 -- 172.21.15.32:0/1437720726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4b0138a70 msgr2=0x7fd4b0138ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:49.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4af400640 1 --2- 172.21.15.32:0/1437720726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4b0138a70 0x7fd4b0138ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4af400640 1 -- 172.21.15.32:0/1437720726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b013b620 con 0x7fd4b013a830 2024-04-17T10:03:49.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4a6000640 1 --2- 172.21.15.32:0/1437720726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4b0138a70 0x7fd4b0138ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:03:49.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4af400640 1 --2- 172.21.15.32:0/1437720726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4b013a830 0x7fd4b013acb0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0160730 tx=0x7fd4a8011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:49.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4a7400640 1 -- 172.21.15.32:0/1437720726 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8010040 con 0x7fd4b013a830 2024-04-17T10:03:49.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4a7400640 1 -- 172.21.15.32:0/1437720726 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a801a070 con 0x7fd4b013a830 2024-04-17T10:03:49.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4b5600640 1 -- 172.21.15.32:0/1437720726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b013b900 con 0x7fd4b013a830 2024-04-17T10:03:49.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4a7400640 1 -- 172.21.15.32:0/1437720726 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a800fb00 con 0x7fd4b013a830 2024-04-17T10:03:49.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4b5600640 1 -- 172.21.15.32:0/1437720726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b00b1920 con 0x7fd4b013a830 2024-04-17T10:03:49.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4b5600640 1 -- 172.21.15.32:0/1437720726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4b0068a90 con 0x7fd4b013a830 2024-04-17T10:03:49.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.495+0000 7fd4a7400640 1 -- 172.21.15.32:0/1437720726 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd4a800fca0 con 0x7fd4b013a830 2024-04-17T10:03:49.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.499+0000 7fd4a7400640 1 --2- 172.21.15.32:0/1437720726 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd48c0683d0 0x7fd48c06a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:49.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.499+0000 7fd4a7400640 1 -- 172.21.15.32:0/1437720726 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fd4a80897d0 con 0x7fd4b013a830 2024-04-17T10:03:49.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.499+0000 7fd4aea00640 1 --2- 172.21.15.32:0/1437720726 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd48c0683d0 0x7fd48c06a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:49.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.499+0000 7fd4aea00640 1 --2- 172.21.15.32:0/1437720726 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd48c0683d0 0x7fd48c06a890 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0014510 tx=0x7fd4a000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:49.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.503+0000 7fd4a7400640 1 -- 172.21.15.32:0/1437720726 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd4a80597b0 con 0x7fd4b013a830 2024-04-17T10:03:49.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.667+0000 7fd4b5600640 1 -- 172.21.15.32:0/1437720726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd4b015c860 con 0x7fd4b013a830 2024-04-17T10:03:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.667+0000 7fd4a7400640 1 -- 172.21.15.32:0/1437720726 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) ==== 104+0+11 (secure 0 0 0) 0x7fd4a805dab0 con 0x7fd4b013a830 2024-04-17T10:03:49.666 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T10:03:49.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.671+0000 7fd4b5600640 1 -- 172.21.15.32:0/1437720726 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd48c0683d0 msgr2=0x7fd48c06a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:49.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.671+0000 7fd4b5600640 1 --2- 172.21.15.32:0/1437720726 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd48c0683d0 0x7fd48c06a890 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0014510 tx=0x7fd4a000b040 comp rx=0 tx=0).stop 2024-04-17T10:03:49.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.671+0000 7fd4b5600640 1 -- 172.21.15.32:0/1437720726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4b013a830 msgr2=0x7fd4b013acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:49.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.671+0000 7fd4b5600640 1 --2- 172.21.15.32:0/1437720726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4b013a830 0x7fd4b013acb0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0160730 tx=0x7fd4a8011c20 comp rx=0 tx=0).stop 2024-04-17T10:03:49.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.671+0000 7fd4b5600640 1 -- 172.21.15.32:0/1437720726 shutdown_connections 2024-04-17T10:03:49.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.671+0000 7fd4b5600640 1 --2- 172.21.15.32:0/1437720726 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd48c0683d0 0x7fd48c06a890 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.671+0000 7fd4b5600640 1 --2- 172.21.15.32:0/1437720726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4b013a830 0x7fd4b013acb0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.671+0000 7fd4b5600640 1 --2- 172.21.15.32:0/1437720726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4b0138a70 0x7fd4b0138ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.671+0000 7fd4b5600640 1 --2- 172.21.15.32:0/1437720726 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd494003f90 0x7fd4b00b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.671+0000 7fd4b5600640 1 -- 172.21.15.32:0/1437720726 >> 172.21.15.32:0/1437720726 conn(0x7fd4b00b2120 msgr2=0x7fd4b00cf980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:49.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.671+0000 7fd4b5600640 1 -- 172.21.15.32:0/1437720726 shutdown_connections 2024-04-17T10:03:49.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.671+0000 7fd4b5600640 1 -- 172.21.15.32:0/1437720726 wait complete. 2024-04-17T10:03:49.687 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:03:49.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.831+0000 7fb416800640 1 Processor -- start 2024-04-17T10:03:49.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.831+0000 7fb416800640 1 -- start start 2024-04-17T10:03:49.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.831+0000 7fb416800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb4101662c0 0x7fb4100b4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:49.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.831+0000 7fb416800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb41006eb60 0x7fb41006f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:49.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.831+0000 7fb416800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb41006f290 con 0x7fb41006ba30 2024-04-17T10:03:49.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.831+0000 7fb416800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb41006efc0 con 0x7fb4101662c0 2024-04-17T10:03:49.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.831+0000 7fb416800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb4101666a0 con 0x7fb41006eb60 2024-04-17T10:03:49.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.831+0000 7fb40fe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb41006ba30 0x7fb41006be60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41580/0 (socket says 172.21.15.32:41580) 2024-04-17T10:03:49.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.831+0000 7fb40fe00640 1 -- 172.21.15.32:0/753882725 learned_addr learned my addr 172.21.15.32:0/753882725 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:49.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.831+0000 7fb40f400640 1 --2- 172.21.15.32:0/753882725 >> v2:172.21.15.144:3300/0 conn(0x7fb4101662c0 0x7fb4100b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:49.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.831+0000 7fb40ea00640 1 --2- 172.21.15.32:0/753882725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb41006eb60 0x7fb41006f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:49.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.835+0000 7fb40e000640 1 -- 172.21.15.32:0/753882725 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 707978050 0 0) 0x7fb41006f290 con 0x7fb41006ba30 2024-04-17T10:03:49.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.835+0000 7fb40e000640 1 -- 172.21.15.32:0/753882725 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3f4003530 con 0x7fb41006ba30 2024-04-17T10:03:49.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.835+0000 7fb40f400640 1 -- 172.21.15.32:0/753882725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb41006eb60 msgr2=0x7fb41006f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:49.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.835+0000 7fb40f400640 1 --2- 172.21.15.32:0/753882725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb41006eb60 0x7fb41006f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.835+0000 7fb40f400640 1 -- 172.21.15.32:0/753882725 >> v1:172.21.15.32:6789/0 conn(0x7fb41006ba30 legacy=0x7fb41006be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:49.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.835+0000 7fb40f400640 1 -- 172.21.15.32:0/753882725 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4100b34c0 con 0x7fb4101662c0 2024-04-17T10:03:49.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.835+0000 7fb40f400640 1 --2- 172.21.15.32:0/753882725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4101662c0 0x7fb4100b4280 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb4000097c0 tx=0x7fb40001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83da13b835232dfa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:49.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.835+0000 7fb40e000640 1 -- 172.21.15.32:0/753882725 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4000b2070 con 0x7fb4101662c0 2024-04-17T10:03:49.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.835+0000 7fb40e000640 1 -- 172.21.15.32:0/753882725 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4000ad640 con 0x7fb4101662c0 2024-04-17T10:03:49.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.835+0000 7fb40e000640 1 -- 172.21.15.32:0/753882725 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4000ada10 con 0x7fb4101662c0 2024-04-17T10:03:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.835+0000 7fb416800640 1 -- 172.21.15.32:0/753882725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4101662c0 msgr2=0x7fb4100b4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.835+0000 7fb416800640 1 --2- 172.21.15.32:0/753882725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4101662c0 0x7fb4100b4280 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb4000097c0 tx=0x7fb40001c4a0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb416800640 1 -- 172.21.15.32:0/753882725 shutdown_connections 2024-04-17T10:03:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb416800640 1 --2- 172.21.15.32:0/753882725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb41006eb60 0x7fb41006f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb416800640 1 --2- 172.21.15.32:0/753882725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4101662c0 0x7fb4100b4280 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb416800640 1 -- 172.21.15.32:0/753882725 >> 172.21.15.32:0/753882725 conn(0x7fb4100b2120 msgr2=0x7fb4100b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb416800640 1 -- 172.21.15.32:0/753882725 shutdown_connections 2024-04-17T10:03:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb416800640 1 -- 172.21.15.32:0/753882725 wait complete. 2024-04-17T10:03:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb416800640 1 Processor -- start 2024-04-17T10:03:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb416800640 1 -- start start 2024-04-17T10:03:49.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb416800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb41006ba30 0x7fb410153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:49.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb416800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb41006eb60 0x7fb410159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:49.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb416800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4101662c0 0x7fb410158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:49.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb416800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb4100cdb20 con 0x7fb41006ba30 2024-04-17T10:03:49.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb416800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb4100cd9a0 con 0x7fb41006eb60 2024-04-17T10:03:49.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb416800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb4100cdca0 con 0x7fb4101662c0 2024-04-17T10:03:49.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb40f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb41006ba30 0x7fb410153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:49.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb40f400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb41006ba30 0x7fb410153070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49522/0 (socket says 172.21.15.32:49522) 2024-04-17T10:03:49.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb40f400640 1 -- 172.21.15.32:0/1642484431 learned_addr learned my addr 172.21.15.32:0/1642484431 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:49.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb40ea00640 1 --2- 172.21.15.32:0/1642484431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb41006eb60 0x7fb410159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:49.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb40fe00640 1 --2- 172.21.15.32:0/1642484431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4101662c0 0x7fb410158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:49.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb40f400640 1 -- 172.21.15.32:0/1642484431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4101662c0 msgr2=0x7fb410158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:49.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb40f400640 1 --2- 172.21.15.32:0/1642484431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4101662c0 0x7fb410158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb40f400640 1 -- 172.21.15.32:0/1642484431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb41006eb60 msgr2=0x7fb410159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:49.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb40f400640 1 --2- 172.21.15.32:0/1642484431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb41006eb60 0x7fb410159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:49.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb40f400640 1 -- 172.21.15.32:0/1642484431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb410158a00 con 0x7fb41006ba30 2024-04-17T10:03:49.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb40ea00640 1 --2- 172.21.15.32:0/1642484431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb41006eb60 0x7fb410159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:49.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.839+0000 7fb40f400640 1 --2- 172.21.15.32:0/1642484431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb41006ba30 0x7fb410153070 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fb4000097c0 tx=0x7fb400019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:49.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.843+0000 7fb407400640 1 -- 172.21.15.32:0/1642484431 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4000ba020 con 0x7fb41006ba30 2024-04-17T10:03:49.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.843+0000 7fb407400640 1 -- 172.21.15.32:0/1642484431 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4000b6410 con 0x7fb41006ba30 2024-04-17T10:03:49.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.843+0000 7fb407400640 1 -- 172.21.15.32:0/1642484431 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4000b2040 con 0x7fb41006ba30 2024-04-17T10:03:49.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.843+0000 7fb416800640 1 -- 172.21.15.32:0/1642484431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb410158ce0 con 0x7fb41006ba30 2024-04-17T10:03:49.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.843+0000 7fb416800640 1 -- 172.21.15.32:0/1642484431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4101592a0 con 0x7fb41006ba30 2024-04-17T10:03:49.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.843+0000 7fb416800640 1 -- 172.21.15.32:0/1642484431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb410068a90 con 0x7fb41006ba30 2024-04-17T10:03:49.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.843+0000 7fb407400640 1 -- 172.21.15.32:0/1642484431 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb4000bd020 con 0x7fb41006ba30 2024-04-17T10:03:49.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.843+0000 7fb407400640 1 --2- 172.21.15.32:0/1642484431 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb3f0068380 0x7fb3f006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:49.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.843+0000 7fb407400640 1 -- 172.21.15.32:0/1642484431 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fb400120640 con 0x7fb41006ba30 2024-04-17T10:03:49.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.843+0000 7fb40ea00640 1 --2- 172.21.15.32:0/1642484431 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb3f0068380 0x7fb3f006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:49.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.843+0000 7fb40ea00640 1 --2- 172.21.15.32:0/1642484431 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb3f0068380 0x7fb3f006a840 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb4101feb10 tx=0x7fb3fc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:49.847+0000 7fb407400640 1 -- 172.21.15.32:0/1642484431 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fb4000c4030 con 0x7fb41006ba30 2024-04-17T10:03:50.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.015+0000 7fb416800640 1 -- 172.21.15.32:0/1642484431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb4100cca80 con 0x7fb41006ba30 2024-04-17T10:03:50.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.015+0000 7fb407400640 1 -- 172.21.15.32:0/1642484431 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) ==== 100+0+11 (secure 0 0 0) 0x7fb400016090 con 0x7fb41006ba30 2024-04-17T10:03:50.014 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T10:03:50.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.019+0000 7fb416800640 1 -- 172.21.15.32:0/1642484431 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb3f0068380 msgr2=0x7fb3f006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:50.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.019+0000 7fb416800640 1 --2- 172.21.15.32:0/1642484431 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb3f0068380 0x7fb3f006a840 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb4101feb10 tx=0x7fb3fc008040 comp rx=0 tx=0).stop 2024-04-17T10:03:50.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.019+0000 7fb416800640 1 -- 172.21.15.32:0/1642484431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb41006ba30 msgr2=0x7fb410153070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:50.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.019+0000 7fb416800640 1 --2- 172.21.15.32:0/1642484431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb41006ba30 0x7fb410153070 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fb4000097c0 tx=0x7fb400019240 comp rx=0 tx=0).stop 2024-04-17T10:03:50.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.019+0000 7fb416800640 1 -- 172.21.15.32:0/1642484431 shutdown_connections 2024-04-17T10:03:50.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.019+0000 7fb416800640 1 --2- 172.21.15.32:0/1642484431 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb3f0068380 0x7fb3f006a840 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:50.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.019+0000 7fb416800640 1 --2- 172.21.15.32:0/1642484431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4101662c0 0x7fb410158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:50.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.019+0000 7fb416800640 1 --2- 172.21.15.32:0/1642484431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb41006eb60 0x7fb410159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:50.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.019+0000 7fb416800640 1 --2- 172.21.15.32:0/1642484431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb41006ba30 0x7fb410153070 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:50.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.019+0000 7fb416800640 1 -- 172.21.15.32:0/1642484431 >> 172.21.15.32:0/1642484431 conn(0x7fb4100b2120 msgr2=0x7fb4100b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:50.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.019+0000 7fb416800640 1 -- 172.21.15.32:0/1642484431 shutdown_connections 2024-04-17T10:03:50.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.019+0000 7fb416800640 1 -- 172.21.15.32:0/1642484431 wait complete. 2024-04-17T10:03:50.035 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:03:50.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.179+0000 7f2e63c00640 1 Processor -- start 2024-04-17T10:03:50.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.179+0000 7f2e63c00640 1 -- start start 2024-04-17T10:03:50.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.179+0000 7f2e63c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e5c15eba0 0x7f2e5c0700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:50.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.179+0000 7f2e63c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e5c0ce950 0x7f2e5c0cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:50.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.179+0000 7f2e63c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2e5c0add30 0x7f2e5c0ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:50.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.179+0000 7f2e63c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2e5c06f0d0 con 0x7f2e5c0ce950 2024-04-17T10:03:50.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.179+0000 7f2e63c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f2e5c067d60 con 0x7f2e5c0add30 2024-04-17T10:03:50.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.179+0000 7f2e63c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2e5c06f320 con 0x7f2e5c15eba0 2024-04-17T10:03:50.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.179+0000 7f2e5be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e5c0ce950 0x7f2e5c0cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:50.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.179+0000 7f2e5be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e5c0ce950 0x7f2e5c0cedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49532/0 (socket says 172.21.15.32:49532) 2024-04-17T10:03:50.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.179+0000 7f2e5be00640 1 -- 172.21.15.32:0/2717703173 learned_addr learned my addr 172.21.15.32:0/2717703173 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:50.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.179+0000 7f2e61200640 1 --2- 172.21.15.32:0/2717703173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e5c15eba0 0x7f2e5c0700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:50.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.179+0000 7f2e61c00640 1 --2- 172.21.15.32:0/2717703173 >> v2:172.21.15.144:3300/0 conn(0x7f2e5c0add30 0x7f2e5c0ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:50.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.179+0000 7f2e5be00640 1 -- 172.21.15.32:0/2717703173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e5c15eba0 msgr2=0x7f2e5c0700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:50.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.179+0000 7f2e5be00640 1 --2- 172.21.15.32:0/2717703173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e5c15eba0 0x7f2e5c0700e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:50.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.179+0000 7f2e5be00640 1 -- 172.21.15.32:0/2717703173 >> v2:172.21.15.144:3300/0 conn(0x7f2e5c0add30 msgr2=0x7f2e5c0ae1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:50.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.179+0000 7f2e5be00640 1 --2- 172.21.15.32:0/2717703173 >> v2:172.21.15.144:3300/0 conn(0x7f2e5c0add30 0x7f2e5c0ae1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:50.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.179+0000 7f2e5be00640 1 -- 172.21.15.32:0/2717703173 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5c06f600 con 0x7f2e5c0ce950 2024-04-17T10:03:50.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.183+0000 7f2e5be00640 1 --2- 172.21.15.32:0/2717703173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e5c0ce950 0x7f2e5c0cedd0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0b28e0 tx=0x7f2e4801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb44ebfc06b18bd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:50.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.183+0000 7f2e5b400640 1 -- 172.21.15.32:0/2717703173 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e480b0070 con 0x7f2e5c0ce950 2024-04-17T10:03:50.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.183+0000 7f2e5b400640 1 -- 172.21.15.32:0/2717703173 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e480ab600 con 0x7f2e5c0ce950 2024-04-17T10:03:50.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.183+0000 7f2e5b400640 1 -- 172.21.15.32:0/2717703173 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e480ab9d0 con 0x7f2e5c0ce950 2024-04-17T10:03:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.183+0000 7f2e63c00640 1 -- 172.21.15.32:0/2717703173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e5c0ce950 msgr2=0x7f2e5c0cedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.183+0000 7f2e63c00640 1 --2- 172.21.15.32:0/2717703173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e5c0ce950 0x7f2e5c0cedd0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0b28e0 tx=0x7f2e4801c490 comp rx=0 tx=0).stop 2024-04-17T10:03:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.183+0000 7f2e63c00640 1 -- 172.21.15.32:0/2717703173 shutdown_connections 2024-04-17T10:03:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.183+0000 7f2e63c00640 1 --2- 172.21.15.32:0/2717703173 >> v2:172.21.15.144:3300/0 conn(0x7f2e5c0add30 0x7f2e5c0ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.183+0000 7f2e63c00640 1 --2- 172.21.15.32:0/2717703173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e5c0ce950 0x7f2e5c0cedd0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.183+0000 7f2e63c00640 1 --2- 172.21.15.32:0/2717703173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e5c15eba0 0x7f2e5c0700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.183+0000 7f2e63c00640 1 -- 172.21.15.32:0/2717703173 >> 172.21.15.32:0/2717703173 conn(0x7f2e5c0cd800 msgr2=0x7f2e5c0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.183+0000 7f2e63c00640 1 -- 172.21.15.32:0/2717703173 shutdown_connections 2024-04-17T10:03:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.183+0000 7f2e63c00640 1 -- 172.21.15.32:0/2717703173 wait complete. 2024-04-17T10:03:50.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e63c00640 1 Processor -- start 2024-04-17T10:03:50.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e63c00640 1 -- start start 2024-04-17T10:03:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e63c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e5c0add30 0x7f2e5c14c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e63c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e5c0ce950 0x7f2e5c151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e63c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e5c15eba0 0x7f2e5c0c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e63c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c06c5d0 con 0x7f2e5c0ce950 2024-04-17T10:03:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e63c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c06c450 con 0x7f2e5c15eba0 2024-04-17T10:03:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e63c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2e5c06c750 con 0x7f2e5c0add30 2024-04-17T10:03:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e5be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e5c0ce950 0x7f2e5c151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e5be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e5c0ce950 0x7f2e5c151200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49544/0 (socket says 172.21.15.32:49544) 2024-04-17T10:03:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e5be00640 1 -- 172.21.15.32:0/3002854656 learned_addr learned my addr 172.21.15.32:0/3002854656 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e61c00640 1 --2- 172.21.15.32:0/3002854656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e5c15eba0 0x7f2e5c0c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e61200640 1 --2- 172.21.15.32:0/3002854656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e5c0add30 0x7f2e5c14c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e5be00640 1 -- 172.21.15.32:0/3002854656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e5c0add30 msgr2=0x7f2e5c14c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e5be00640 1 --2- 172.21.15.32:0/3002854656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e5c0add30 0x7f2e5c14c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e5be00640 1 -- 172.21.15.32:0/3002854656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e5c15eba0 msgr2=0x7f2e5c0c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e5be00640 1 --2- 172.21.15.32:0/3002854656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e5c15eba0 0x7f2e5c0c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e5be00640 1 -- 172.21.15.32:0/3002854656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5c14cae0 con 0x7f2e5c0ce950 2024-04-17T10:03:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e61c00640 1 --2- 172.21.15.32:0/3002854656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e5c15eba0 0x7f2e5c0c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e61200640 1 --2- 172.21.15.32:0/3002854656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e5c0add30 0x7f2e5c14c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e5be00640 1 --2- 172.21.15.32:0/3002854656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e5c0ce950 0x7f2e5c151200 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f2e4801cd30 tx=0x7f2e48019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e58c00640 1 -- 172.21.15.32:0/3002854656 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e480b9070 con 0x7f2e5c0ce950 2024-04-17T10:03:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e58c00640 1 -- 172.21.15.32:0/3002854656 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e480b4420 con 0x7f2e5c0ce950 2024-04-17T10:03:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e58c00640 1 -- 172.21.15.32:0/3002854656 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e480b0040 con 0x7f2e5c0ce950 2024-04-17T10:03:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e63c00640 1 -- 172.21.15.32:0/3002854656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e5c14cdc0 con 0x7f2e5c0ce950 2024-04-17T10:03:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.187+0000 7f2e63c00640 1 -- 172.21.15.32:0/3002854656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e5c14d380 con 0x7f2e5c0ce950 2024-04-17T10:03:50.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.191+0000 7f2e63c00640 1 -- 172.21.15.32:0/3002854656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e5c068a90 con 0x7f2e5c0ce950 2024-04-17T10:03:50.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.191+0000 7f2e58c00640 1 -- 172.21.15.32:0/3002854656 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2e480aa3a0 con 0x7f2e5c0ce950 2024-04-17T10:03:50.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.191+0000 7f2e58c00640 1 --2- 172.21.15.32:0/3002854656 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e3c068380 0x7f2e3c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:50.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.191+0000 7f2e58c00640 1 -- 172.21.15.32:0/3002854656 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f2e4811fc40 con 0x7f2e5c0ce950 2024-04-17T10:03:50.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.191+0000 7f2e61200640 1 --2- 172.21.15.32:0/3002854656 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e3c068380 0x7f2e3c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:50.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.191+0000 7f2e61200640 1 --2- 172.21.15.32:0/3002854656 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e3c068380 0x7f2e3c06a840 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009b10 tx=0x7f2e50008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:50.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.195+0000 7f2e58c00640 1 -- 172.21.15.32:0/3002854656 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f2e480ace20 con 0x7f2e5c0ce950 2024-04-17T10:03:50.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.375+0000 7f2e63c00640 1 -- 172.21.15.32:0/3002854656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2e5c15e370 con 0x7f2e5c0ce950 2024-04-17T10:03:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.375+0000 7f2e58c00640 1 -- 172.21.15.32:0/3002854656 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v109) ==== 80+0+0 (secure 0 0 0) 0x7f2e48016110 con 0x7f2e5c0ce950 2024-04-17T10:03:50.376 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T10:03:50.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.379+0000 7f2e63c00640 1 -- 172.21.15.32:0/3002854656 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e3c068380 msgr2=0x7f2e3c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:50.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.379+0000 7f2e63c00640 1 --2- 172.21.15.32:0/3002854656 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e3c068380 0x7f2e3c06a840 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009b10 tx=0x7f2e50008040 comp rx=0 tx=0).stop 2024-04-17T10:03:50.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.379+0000 7f2e63c00640 1 -- 172.21.15.32:0/3002854656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e5c0ce950 msgr2=0x7f2e5c151200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:50.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.379+0000 7f2e63c00640 1 --2- 172.21.15.32:0/3002854656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e5c0ce950 0x7f2e5c151200 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f2e4801cd30 tx=0x7f2e48019230 comp rx=0 tx=0).stop 2024-04-17T10:03:50.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.379+0000 7f2e63c00640 1 -- 172.21.15.32:0/3002854656 shutdown_connections 2024-04-17T10:03:50.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.379+0000 7f2e63c00640 1 --2- 172.21.15.32:0/3002854656 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2e3c068380 0x7f2e3c06a840 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:50.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.379+0000 7f2e63c00640 1 --2- 172.21.15.32:0/3002854656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e5c15eba0 0x7f2e5c0c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:50.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.379+0000 7f2e63c00640 1 --2- 172.21.15.32:0/3002854656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e5c0ce950 0x7f2e5c151200 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:50.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.379+0000 7f2e63c00640 1 --2- 172.21.15.32:0/3002854656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e5c0add30 0x7f2e5c14c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:50.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.379+0000 7f2e63c00640 1 -- 172.21.15.32:0/3002854656 >> 172.21.15.32:0/3002854656 conn(0x7f2e5c0cd800 msgr2=0x7f2e5c072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:50.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.379+0000 7f2e63c00640 1 -- 172.21.15.32:0/3002854656 shutdown_connections 2024-04-17T10:03:50.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:50.379+0000 7f2e63c00640 1 -- 172.21.15.32:0/3002854656 wait complete. 2024-04-17T10:03:50.397 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2024-04-17T10:03:50.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:50.397 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T10:03:50.415 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[51253]: fuse finished with error 0 and tester_r 0 2024-04-17T10:03:50.418 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:03:56.519 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T10:03:56.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:56.520 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T10:03:56.530 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-17T10:03:56.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.727+0000 7fa639e00640 1 Processor -- start 2024-04-17T10:03:56.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.727+0000 7fa639e00640 1 -- start start 2024-04-17T10:03:56.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.727+0000 7fa639e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa63415b7d0 0x7fa63406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.727+0000 7fa639e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa63406f7c0 0x7fa63406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.727+0000 7fa639e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6340cc940 con 0x7fa63406f270 2024-04-17T10:03:56.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.727+0000 7fa639e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa6340ccc20 con 0x7fa63406f7c0 2024-04-17T10:03:56.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.727+0000 7fa639e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa63406e0b0 con 0x7fa63415b7d0 2024-04-17T10:03:56.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.727+0000 7fa633400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa63406f270 0x7fa6340ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45642/0 (socket says 172.21.15.32:45642) 2024-04-17T10:03:56.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.727+0000 7fa633400640 1 -- 172.21.15.32:0/836695124 learned_addr learned my addr 172.21.15.32:0/836695124 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:56.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.727+0000 7fa62a000640 1 --2- 172.21.15.32:0/836695124 >> v2:172.21.15.144:3300/0 conn(0x7fa63406f7c0 0x7fa63406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.727+0000 7fa632a00640 1 --2- 172.21.15.32:0/836695124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa63415b7d0 0x7fa63406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.731+0000 7fa632000640 1 -- 172.21.15.32:0/836695124 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3322356846 0 0) 0x7fa6340cc940 con 0x7fa63406f270 2024-04-17T10:03:56.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.731+0000 7fa632000640 1 -- 172.21.15.32:0/836695124 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa618003530 con 0x7fa63406f270 2024-04-17T10:03:56.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.731+0000 7fa62a000640 1 -- 172.21.15.32:0/836695124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa63415b7d0 msgr2=0x7fa63406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:56.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.731+0000 7fa62a000640 1 --2- 172.21.15.32:0/836695124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa63415b7d0 0x7fa63406e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.731+0000 7fa62a000640 1 -- 172.21.15.32:0/836695124 >> v1:172.21.15.32:6789/0 conn(0x7fa63406f270 legacy=0x7fa6340ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:56.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.731+0000 7fa62a000640 1 -- 172.21.15.32:0/836695124 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa63406e390 con 0x7fa63406f7c0 2024-04-17T10:03:56.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.731+0000 7fa632a00640 1 --2- 172.21.15.32:0/836695124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa63415b7d0 0x7fa63406e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:03:56.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.731+0000 7fa62a000640 1 --2- 172.21.15.32:0/836695124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa63406f7c0 0x7fa63406ba30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0097c0 tx=0x7fa61c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4d08a39fb077005 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:56.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.731+0000 7fa632000640 1 -- 172.21.15.32:0/836695124 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0ab430 con 0x7fa63406f7c0 2024-04-17T10:03:56.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.731+0000 7fa632000640 1 -- 172.21.15.32:0/836695124 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa61c0b0070 con 0x7fa63406f7c0 2024-04-17T10:03:56.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa632000640 1 -- 172.21.15.32:0/836695124 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0ab7f0 con 0x7fa63406f7c0 2024-04-17T10:03:56.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa639e00640 1 -- 172.21.15.32:0/836695124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa63406f7c0 msgr2=0x7fa63406ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:56.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa639e00640 1 --2- 172.21.15.32:0/836695124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa63406f7c0 0x7fa63406ba30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0097c0 tx=0x7fa61c01c410 comp rx=0 tx=0).stop 2024-04-17T10:03:56.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa639e00640 1 -- 172.21.15.32:0/836695124 shutdown_connections 2024-04-17T10:03:56.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa639e00640 1 --2- 172.21.15.32:0/836695124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa63406f7c0 0x7fa63406ba30 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa639e00640 1 --2- 172.21.15.32:0/836695124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa63415b7d0 0x7fa63406e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa639e00640 1 -- 172.21.15.32:0/836695124 >> 172.21.15.32:0/836695124 conn(0x7fa6340b2120 msgr2=0x7fa6340b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:56.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa639e00640 1 -- 172.21.15.32:0/836695124 shutdown_connections 2024-04-17T10:03:56.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa639e00640 1 -- 172.21.15.32:0/836695124 wait complete. 2024-04-17T10:03:56.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa639e00640 1 Processor -- start 2024-04-17T10:03:56.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa639e00640 1 -- start start 2024-04-17T10:03:56.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa639e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa63406f270 0x7fa634145110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa639e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63406f7c0 0x7fa634145650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa639e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa63415b7d0 0x7fa6340ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa639e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa634166aa0 con 0x7fa63406f7c0 2024-04-17T10:03:56.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa639e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa63406c380 con 0x7fa63415b7d0 2024-04-17T10:03:56.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa639e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa634166c20 con 0x7fa63406f270 2024-04-17T10:03:56.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa62a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63406f7c0 0x7fa634145650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa62a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63406f7c0 0x7fa634145650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36122/0 (socket says 172.21.15.32:36122) 2024-04-17T10:03:56.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa62a000640 1 -- 172.21.15.32:0/4136405887 learned_addr learned my addr 172.21.15.32:0/4136405887 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:56.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa632a00640 1 --2- 172.21.15.32:0/4136405887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa63406f270 0x7fa634145110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa633400640 1 --2- 172.21.15.32:0/4136405887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa63415b7d0 0x7fa6340ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa62a000640 1 -- 172.21.15.32:0/4136405887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa63406f270 msgr2=0x7fa634145110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:56.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa62a000640 1 --2- 172.21.15.32:0/4136405887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa63406f270 0x7fa634145110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa62a000640 1 -- 172.21.15.32:0/4136405887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa63415b7d0 msgr2=0x7fa6340ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:56.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa62a000640 1 --2- 172.21.15.32:0/4136405887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa63415b7d0 0x7fa6340ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa62a000640 1 -- 172.21.15.32:0/4136405887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6341551b0 con 0x7fa63406f7c0 2024-04-17T10:03:56.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa632a00640 1 --2- 172.21.15.32:0/4136405887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa63406f270 0x7fa634145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:03:56.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.735+0000 7fa633400640 1 --2- 172.21.15.32:0/4136405887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa63415b7d0 0x7fa6340ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:03:56.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.739+0000 7fa62a000640 1 --2- 172.21.15.32:0/4136405887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63406f7c0 0x7fa634145650 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fa61c01aac0 tx=0x7fa61c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:56.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.739+0000 7fa62be00640 1 -- 172.21.15.32:0/4136405887 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0abdd0 con 0x7fa63406f7c0 2024-04-17T10:03:56.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.739+0000 7fa62be00640 1 -- 172.21.15.32:0/4136405887 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa61c0bd480 con 0x7fa63406f7c0 2024-04-17T10:03:56.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.739+0000 7fa62be00640 1 -- 172.21.15.32:0/4136405887 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0bd870 con 0x7fa63406f7c0 2024-04-17T10:03:56.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.739+0000 7fa639e00640 1 -- 172.21.15.32:0/4136405887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa634155490 con 0x7fa63406f7c0 2024-04-17T10:03:56.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.739+0000 7fa639e00640 1 -- 172.21.15.32:0/4136405887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa634155a50 con 0x7fa63406f7c0 2024-04-17T10:03:56.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.739+0000 7fa62be00640 1 -- 172.21.15.32:0/4136405887 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa61c0b43f0 con 0x7fa63406f7c0 2024-04-17T10:03:56.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa62be00640 1 --2- 172.21.15.32:0/4136405887 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa610068380 0x7fa61006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa62be00640 1 -- 172.21.15.32:0/4136405887 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7fa61c1208b0 con 0x7fa63406f7c0 2024-04-17T10:03:56.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa632a00640 1 --2- 172.21.15.32:0/4136405887 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa610068380 0x7fa61006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa63d288000 1 Processor -- start 2024-04-17T10:03:56.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa63d288000 1 -- start start 2024-04-17T10:03:56.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa632a00640 1 --2- 172.21.15.32:0/4136405887 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa610068380 0x7fa61006a840 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa624007440 tx=0x7fa62400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:56.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa63d288000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8260 0x562eaa5a8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa63d288000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eaa5a8b80 0x562eaa55b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa63d288000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eaa55b9a0 0x562eaa55bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa63d288000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562eaa3ad4f0 con 0x562eaa5a8260 2024-04-17T10:03:56.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa63d288000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562eaa3666c0 con 0x562eaa5a8b80 2024-04-17T10:03:56.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa63d288000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x562eaa3a94b0 con 0x562eaa55b9a0 2024-04-17T10:03:56.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa632a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8260 0x562eaa5a8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa632a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8260 0x562eaa5a8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36130/0 (socket says 172.21.15.32:36130) 2024-04-17T10:03:56.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa632a00640 1 -- 172.21.15.32:0/3328771795 learned_addr learned my addr 172.21.15.32:0/3328771795 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:56.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa62a000640 1 --2- 172.21.15.32:0/3328771795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eaa5a8b80 0x562eaa55b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa633400640 1 --2- 172.21.15.32:0/3328771795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eaa55b9a0 0x562eaa55bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa632a00640 1 -- 172.21.15.32:0/3328771795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eaa55b9a0 msgr2=0x562eaa55bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:56.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa632a00640 1 --2- 172.21.15.32:0/3328771795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eaa55b9a0 0x562eaa55bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa632a00640 1 -- 172.21.15.32:0/3328771795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eaa5a8b80 msgr2=0x562eaa55b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:56.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa632a00640 1 --2- 172.21.15.32:0/3328771795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eaa5a8b80 0x562eaa55b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa632a00640 1 -- 172.21.15.32:0/3328771795 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eaa3f28e0 con 0x562eaa5a8260 2024-04-17T10:03:56.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa632a00640 1 --2- 172.21.15.32:0/3328771795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8260 0x562eaa5a8640 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fa624009ed0 tx=0x7fa624012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6307094db9cd06b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:56.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa623e00640 1 -- 172.21.15.32:0/3328771795 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624026070 con 0x562eaa5a8260 2024-04-17T10:03:56.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa623e00640 1 -- 172.21.15.32:0/3328771795 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6240149a0 con 0x562eaa5a8260 2024-04-17T10:03:56.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa623e00640 1 -- 172.21.15.32:0/3328771795 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624014d50 con 0x562eaa5a8260 2024-04-17T10:03:56.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa63d288000 1 -- 172.21.15.32:0/3328771795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8260 msgr2=0x562eaa5a8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:56.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa63d288000 1 --2- 172.21.15.32:0/3328771795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8260 0x562eaa5a8640 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fa624009ed0 tx=0x7fa624012000 comp rx=0 tx=0).stop 2024-04-17T10:03:56.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa63d288000 1 -- 172.21.15.32:0/3328771795 shutdown_connections 2024-04-17T10:03:56.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa63d288000 1 --2- 172.21.15.32:0/3328771795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eaa55b9a0 0x562eaa55bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa63d288000 1 --2- 172.21.15.32:0/3328771795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eaa5a8b80 0x562eaa55b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa63d288000 1 --2- 172.21.15.32:0/3328771795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8260 0x562eaa5a8640 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.743+0000 7fa63d288000 1 -- 172.21.15.32:0/3328771795 >> 172.21.15.32:0/3328771795 conn(0x562eaa589000 msgr2=0x562eaa58b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:56.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa63d288000 1 -- 172.21.15.32:0/3328771795 shutdown_connections 2024-04-17T10:03:56.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa63d288000 1 -- 172.21.15.32:0/3328771795 wait complete. 2024-04-17T10:03:56.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa63d288000 1 -- start start 2024-04-17T10:03:56.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa63d288000 1 Processor -- start 2024-04-17T10:03:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa63d288000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eaa55b9a0 0x562eaa561020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa63d288000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eaa5a8260 0x562eaa561560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa63d288000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8b80 0x562eaa561aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa63d288000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562eaa42e7b0 con 0x562eaa5a8b80 2024-04-17T10:03:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa63d288000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562eaa468d00 con 0x562eaa55b9a0 2024-04-17T10:03:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa63d288000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x562eaa3722c0 con 0x562eaa5a8260 2024-04-17T10:03:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa63d288000 10 client.0 _ll_register_callbacks cb 0x562eaa382030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:03:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa633400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8b80 0x562eaa561aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa633400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8b80 0x562eaa561aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36138/0 (socket says 172.21.15.32:36138) 2024-04-17T10:03:56.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa633400640 1 -- 172.21.15.32:0/2431301995 learned_addr learned my addr 172.21.15.32:0/2431301995 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:56.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa632a00640 1 --2- 172.21.15.32:0/2431301995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eaa55b9a0 0x562eaa561020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa62a000640 1 --2- 172.21.15.32:0/2431301995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eaa5a8260 0x562eaa561560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa633400640 1 -- 172.21.15.32:0/2431301995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eaa5a8260 msgr2=0x562eaa561560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:56.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa633400640 1 --2- 172.21.15.32:0/2431301995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eaa5a8260 0x562eaa561560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa633400640 1 -- 172.21.15.32:0/2431301995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eaa55b9a0 msgr2=0x562eaa561020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:56.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa633400640 1 --2- 172.21.15.32:0/2431301995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eaa55b9a0 0x562eaa561020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa633400640 1 -- 172.21.15.32:0/2431301995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eaa385490 con 0x562eaa5a8b80 2024-04-17T10:03:56.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa633400640 1 --2- 172.21.15.32:0/2431301995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8b80 0x562eaa561aa0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fa62c007650 tx=0x7fa62c0016b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee37be83e29e64a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:56.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa620c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:03:56.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa620c00640 1 -- 172.21.15.32:0/2431301995 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c0137d0 con 0x562eaa5a8b80 2024-04-17T10:03:56.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa620c00640 1 -- 172.21.15.32:0/2431301995 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7fa62c013970 con 0x562eaa5a8b80 2024-04-17T10:03:56.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa63d288000 1 -- 172.21.15.32:0/2431301995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6240149a0 con 0x562eaa5a8b80 2024-04-17T10:03:56.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa620c00640 1 -- 172.21.15.32:0/2431301995 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7fa62c007740 con 0x562eaa5a8b80 2024-04-17T10:03:56.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa63d288000 10 client.9513 fetch_fsmap learned FSMap version 98 2024-04-17T10:03:56.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa63d288000 1 -- 172.21.15.32:0/2431301995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x562eaa378eb0 con 0x562eaa5a8b80 2024-04-17T10:03:56.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa620c00640 1 -- 172.21.15.32:0/2431301995 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6240149a0 con 0x562eaa5a8b80 2024-04-17T10:03:56.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa620c00640 1 -- 172.21.15.32:0/2431301995 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 98) ==== 3495+0+0 (secure 0 0 0) 0x7fa62c013b70 con 0x562eaa5a8b80 2024-04-17T10:03:56.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa63d288000 10 client.9513 fetch_fsmap finished waiting for FSMap version 98 2024-04-17T10:03:56.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa63d288000 10 client.9513 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-17T10:03:56.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa63d288000 20 client.9513 populate_metadata read hostname 'smithi032' 2024-04-17T10:03:56.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa63d288000 1 --2- 172.21.15.32:0/2431301995 >> [v2:172.21.15.32:6836/901447746,v1:172.21.15.32:6837/901447746] conn(0x562eaa5b7500 0x562eaa5b98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa63d288000 4 client.9513 mds_command: new command op to 9285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:03:56.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.747+0000 7fa63d288000 1 -- 172.21.15.32:0/2431301995 --> [v2:172.21.15.32:6836/901447746,v1:172.21.15.32:6837/901447746] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562eaa346dd0 con 0x562eaa5b7500 2024-04-17T10:03:56.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.751+0000 7fa632a00640 1 --2- 172.21.15.32:0/2431301995 >> [v2:172.21.15.32:6836/901447746,v1:172.21.15.32:6837/901447746] conn(0x562eaa5b7500 0x562eaa5b98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.751+0000 7fa632a00640 1 --2- 172.21.15.32:0/2431301995 >> [v2:172.21.15.32:6836/901447746,v1:172.21.15.32:6837/901447746] conn(0x562eaa5b7500 0x562eaa5b98f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68fc3e1d6cb4c0f6 server_cookie=f9ac0773338e1420 in_seq=0 out_seq=0 2024-04-17T10:03:56.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.751+0000 7fa620c00640 10 client.9513 ms_handle_connect on v2:172.21.15.32:6836/901447746 2024-04-17T10:03:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.751+0000 7fa620c00640 1 -- 172.21.15.32:0/2431301995 <== mds.0 v2:172.21.15.32:6836/901447746 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562eaa346dd0 con 0x562eaa5b7500 2024-04-17T10:03:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.751+0000 7fa620c00640 10 client.9513 handle_command_reply: tid=0 2024-04-17T10:03:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.751+0000 7fa63d288000 1 client.9513 shutdown 2024-04-17T10:03:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.751+0000 7fa63d288000 1 -- 172.21.15.32:0/2431301995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8b80 msgr2=0x562eaa561aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.751+0000 7fa63d288000 1 --2- 172.21.15.32:0/2431301995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8b80 0x562eaa561aa0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fa62c007650 tx=0x7fa62c0016b0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.751+0000 7fa63d288000 1 -- 172.21.15.32:0/2431301995 shutdown_connections 2024-04-17T10:03:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.751+0000 7fa63d288000 1 --2- 172.21.15.32:0/2431301995 >> [v2:172.21.15.32:6836/901447746,v1:172.21.15.32:6837/901447746] conn(0x562eaa5b7500 0x562eaa5b98f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.751+0000 7fa63d288000 1 --2- 172.21.15.32:0/2431301995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eaa5a8260 0x562eaa561560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.751+0000 7fa63d288000 1 --2- 172.21.15.32:0/2431301995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8b80 0x562eaa561aa0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.751+0000 7fa63d288000 1 --2- 172.21.15.32:0/2431301995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eaa55b9a0 0x562eaa561020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.751+0000 7fa63d288000 1 -- 172.21.15.32:0/2431301995 >> 172.21.15.32:0/2431301995 conn(0x562eaa589000 msgr2=0x562eaa58b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.751+0000 7fa620c00640 0 client.9513 ms_handle_reset on v2:172.21.15.32:6836/901447746 2024-04-17T10:03:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.751+0000 7fa63d288000 1 -- 172.21.15.32:0/2431301995 shutdown_connections 2024-04-17T10:03:56.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.751+0000 7fa63d288000 1 -- 172.21.15.32:0/2431301995 wait complete. 2024-04-17T10:03:56.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.751+0000 7fa63d288000 20 client.9513 trim_cache size 0 max 16384 2024-04-17T10:03:56.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa63d288000 1 Processor -- start 2024-04-17T10:03:56.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa63d288000 1 -- start start 2024-04-17T10:03:56.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa63d288000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eaa55b9a0 0x562eaa6130b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa63d288000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eaa5a8260 0x562eaa6135f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa63d288000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8b80 0x562eaa613b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa63d288000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa634203360 con 0x562eaa5a8b80 2024-04-17T10:03:56.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa63d288000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa634202790 con 0x562eaa5a8260 2024-04-17T10:03:56.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa63d288000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa634202d10 con 0x562eaa55b9a0 2024-04-17T10:03:56.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa633400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8b80 0x562eaa613b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa633400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8b80 0x562eaa613b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36146/0 (socket says 172.21.15.32:36146) 2024-04-17T10:03:56.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa633400640 1 -- 172.21.15.32:0/3810047077 learned_addr learned my addr 172.21.15.32:0/3810047077 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:56.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa633400640 1 -- 172.21.15.32:0/3810047077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eaa55b9a0 msgr2=0x562eaa6130b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:56.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa633400640 1 --2- 172.21.15.32:0/3810047077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eaa55b9a0 0x562eaa6130b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa633400640 1 -- 172.21.15.32:0/3810047077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eaa5a8260 msgr2=0x562eaa6135f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:56.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa62a000640 1 --2- 172.21.15.32:0/3810047077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eaa5a8260 0x562eaa6135f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa633400640 1 --2- 172.21.15.32:0/3810047077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eaa5a8260 0x562eaa6135f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa633400640 1 -- 172.21.15.32:0/3810047077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eaa367a20 con 0x562eaa5a8b80 2024-04-17T10:03:56.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa62a000640 1 --2- 172.21.15.32:0/3810047077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eaa5a8260 0x562eaa6135f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:03:56.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa633400640 1 --2- 172.21.15.32:0/3810047077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8b80 0x562eaa613b30 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fa62c007e70 tx=0x7fa62c00d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492fe5e02a7b646b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:56.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa623400640 1 -- 172.21.15.32:0/3810047077 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c0049f0 con 0x562eaa5a8b80 2024-04-17T10:03:56.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa623400640 1 -- 172.21.15.32:0/3810047077 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa62c019070 con 0x562eaa5a8b80 2024-04-17T10:03:56.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa623400640 1 -- 172.21.15.32:0/3810047077 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c004df0 con 0x562eaa5a8b80 2024-04-17T10:03:56.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa63d288000 1 -- 172.21.15.32:0/3810047077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8b80 msgr2=0x562eaa613b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:56.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa63d288000 1 --2- 172.21.15.32:0/3810047077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8b80 0x562eaa613b30 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fa62c007e70 tx=0x7fa62c00d000 comp rx=0 tx=0).stop 2024-04-17T10:03:56.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa63d288000 1 -- 172.21.15.32:0/3810047077 shutdown_connections 2024-04-17T10:03:56.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa63d288000 1 --2- 172.21.15.32:0/3810047077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8b80 0x562eaa613b30 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa63d288000 1 --2- 172.21.15.32:0/3810047077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eaa5a8260 0x562eaa6135f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa63d288000 1 --2- 172.21.15.32:0/3810047077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eaa55b9a0 0x562eaa6130b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa63d288000 1 -- 172.21.15.32:0/3810047077 >> 172.21.15.32:0/3810047077 conn(0x562eaa589000 msgr2=0x562eaa5bcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:56.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa63d288000 1 -- 172.21.15.32:0/3810047077 shutdown_connections 2024-04-17T10:03:56.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.767+0000 7fa63d288000 1 -- 172.21.15.32:0/3810047077 wait complete. 2024-04-17T10:03:56.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa63d288000 1 -- start start 2024-04-17T10:03:56.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa63d288000 1 Processor -- start 2024-04-17T10:03:56.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa63d288000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eaa55b9a0 0x562eaa5bc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa63d288000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8260 0x562eaa5d3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa63d288000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eaa5a8b80 0x562eaa5d39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa63d288000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x562eaa351050 con 0x562eaa5a8260 2024-04-17T10:03:56.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa63d288000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562eaa36f570 con 0x562eaa55b9a0 2024-04-17T10:03:56.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa63d288000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x562eaa3ac800 con 0x562eaa5a8b80 2024-04-17T10:03:56.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa63d288000 10 client.0 _ll_register_callbacks cb 0x7fa62c006970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:03:56.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa62a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8260 0x562eaa5d3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa62a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8260 0x562eaa5d3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36160/0 (socket says 172.21.15.32:36160) 2024-04-17T10:03:56.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa62a000640 1 -- 172.21.15.32:0/225795920 learned_addr learned my addr 172.21.15.32:0/225795920 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:56.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa632a00640 1 --2- 172.21.15.32:0/225795920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eaa55b9a0 0x562eaa5bc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa633400640 1 --2- 172.21.15.32:0/225795920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eaa5a8b80 0x562eaa5d39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa62a000640 1 -- 172.21.15.32:0/225795920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eaa5a8b80 msgr2=0x562eaa5d39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa62a000640 1 --2- 172.21.15.32:0/225795920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eaa5a8b80 0x562eaa5d39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa62a000640 1 -- 172.21.15.32:0/225795920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eaa55b9a0 msgr2=0x562eaa5bc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa62a000640 1 --2- 172.21.15.32:0/225795920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eaa55b9a0 0x562eaa5bc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa62a000640 1 -- 172.21.15.32:0/225795920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eaa42ec40 con 0x562eaa5a8260 2024-04-17T10:03:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa62a000640 1 --2- 172.21.15.32:0/225795920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8260 0x562eaa5d3480 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0f0790 tx=0x7fa61c0f4cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329db2b3d4087b40 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa620c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:03:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa620c00640 1 -- 172.21.15.32:0/225795920 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0f8fa0 con 0x562eaa5a8260 2024-04-17T10:03:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa620c00640 1 -- 172.21.15.32:0/225795920 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7fa61c0f9140 con 0x562eaa5a8260 2024-04-17T10:03:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa63d288000 1 -- 172.21.15.32:0/225795920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa62c019070 con 0x562eaa5a8260 2024-04-17T10:03:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa620c00640 1 -- 172.21.15.32:0/225795920 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7fa61c0f1b80 con 0x562eaa5a8260 2024-04-17T10:03:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa63d288000 10 client.9519 fetch_fsmap learned FSMap version 98 2024-04-17T10:03:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa63d288000 1 -- 172.21.15.32:0/225795920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x562eaa352600 con 0x562eaa5a8260 2024-04-17T10:03:56.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa620c00640 1 -- 172.21.15.32:0/225795920 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa62c019070 con 0x562eaa5a8260 2024-04-17T10:03:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa620c00640 1 -- 172.21.15.32:0/225795920 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 98) ==== 3495+0+0 (secure 0 0 0) 0x7fa61c0aa070 con 0x562eaa5a8260 2024-04-17T10:03:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa63d288000 10 client.9519 fetch_fsmap finished waiting for FSMap version 98 2024-04-17T10:03:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa63d288000 10 client.9519 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-17T10:03:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa63d288000 20 client.9519 populate_metadata read hostname 'smithi032' 2024-04-17T10:03:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa63d288000 1 --2- 172.21.15.32:0/225795920 >> [v2:172.21.15.32:6836/901447746,v1:172.21.15.32:6837/901447746] conn(0x562eaa6168b0 0x562eaa616cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa63d288000 4 client.9519 mds_command: new command op to 9285 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T10:03:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa63d288000 1 -- 172.21.15.32:0/225795920 --> [v2:172.21.15.32:6836/901447746,v1:172.21.15.32:6837/901447746] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562eaa588c40 con 0x562eaa6168b0 2024-04-17T10:03:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa632a00640 1 --2- 172.21.15.32:0/225795920 >> [v2:172.21.15.32:6836/901447746,v1:172.21.15.32:6837/901447746] conn(0x562eaa6168b0 0x562eaa616cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa632a00640 1 --2- 172.21.15.32:0/225795920 >> [v2:172.21.15.32:6836/901447746,v1:172.21.15.32:6837/901447746] conn(0x562eaa6168b0 0x562eaa616cc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13783818a5751634 server_cookie=d9e0a830f1052124 in_seq=0 out_seq=0 2024-04-17T10:03:56.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.771+0000 7fa620c00640 10 client.9519 ms_handle_connect on v2:172.21.15.32:6836/901447746 2024-04-17T10:03:56.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa620c00640 1 -- 172.21.15.32:0/225795920 <== mds.0 v2:172.21.15.32:6836/901447746 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562eaa588c40 con 0x562eaa6168b0 2024-04-17T10:03:56.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa620c00640 10 client.9519 handle_command_reply: tid=0 2024-04-17T10:03:56.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa63d288000 1 client.9519 shutdown 2024-04-17T10:03:56.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa63d288000 1 -- 172.21.15.32:0/225795920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8260 msgr2=0x562eaa5d3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:56.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa63d288000 1 --2- 172.21.15.32:0/225795920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8260 0x562eaa5d3480 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0f0790 tx=0x7fa61c0f4cc0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa63d288000 1 -- 172.21.15.32:0/225795920 shutdown_connections 2024-04-17T10:03:56.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa63d288000 1 --2- 172.21.15.32:0/225795920 >> [v2:172.21.15.32:6836/901447746,v1:172.21.15.32:6837/901447746] conn(0x562eaa6168b0 0x562eaa616cc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa63d288000 1 --2- 172.21.15.32:0/225795920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562eaa5a8b80 0x562eaa5d39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa63d288000 1 --2- 172.21.15.32:0/225795920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562eaa5a8260 0x562eaa5d3480 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa63d288000 1 --2- 172.21.15.32:0/225795920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562eaa55b9a0 0x562eaa5bc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa620c00640 0 client.9519 ms_handle_reset on v2:172.21.15.32:6836/901447746 2024-04-17T10:03:56.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa63d288000 1 -- 172.21.15.32:0/225795920 >> 172.21.15.32:0/225795920 conn(0x562eaa589000 msgr2=0x562eaa55a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:56.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa63d288000 1 -- 172.21.15.32:0/225795920 shutdown_connections 2024-04-17T10:03:56.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa63d288000 1 -- 172.21.15.32:0/225795920 wait complete. 2024-04-17T10:03:56.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa63d288000 20 client.9519 trim_cache size 0 max 16384 2024-04-17T10:03:56.788 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:03:56.788 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "", 2024-04-17T10:03:56.788 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0 2024-04-17T10:03:56.788 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:03:56.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa639e00640 1 -- 172.21.15.32:0/4136405887 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa610068380 msgr2=0x7fa61006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:56.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa639e00640 1 --2- 172.21.15.32:0/4136405887 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa610068380 0x7fa61006a840 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa624007440 tx=0x7fa62400b040 comp rx=0 tx=0).stop 2024-04-17T10:03:56.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa639e00640 1 -- 172.21.15.32:0/4136405887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63406f7c0 msgr2=0x7fa634145650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:56.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa639e00640 1 --2- 172.21.15.32:0/4136405887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63406f7c0 0x7fa634145650 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fa61c01aac0 tx=0x7fa61c0191b0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa639e00640 1 -- 172.21.15.32:0/4136405887 shutdown_connections 2024-04-17T10:03:56.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa639e00640 1 --2- 172.21.15.32:0/4136405887 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa610068380 0x7fa61006a840 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa639e00640 1 --2- 172.21.15.32:0/4136405887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa63415b7d0 0x7fa6340ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa639e00640 1 --2- 172.21.15.32:0/4136405887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63406f7c0 0x7fa634145650 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa639e00640 1 --2- 172.21.15.32:0/4136405887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa63406f270 0x7fa634145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.787+0000 7fa639e00640 1 -- 172.21.15.32:0/4136405887 >> 172.21.15.32:0/4136405887 conn(0x7fa6340b2120 msgr2=0x7fa634072660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:56.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.791+0000 7fa639e00640 1 -- 172.21.15.32:0/4136405887 shutdown_connections 2024-04-17T10:03:56.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.791+0000 7fa639e00640 1 -- 172.21.15.32:0/4136405887 wait complete. 2024-04-17T10:03:56.805 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T10:03:56.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.947+0000 7fd8b9c00640 1 Processor -- start 2024-04-17T10:03:56.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.947+0000 7fd8b9c00640 1 -- start start 2024-04-17T10:03:56.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.947+0000 7fd8b9c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8b4144f20 0x7fd8b406e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.947+0000 7fd8b9c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd8b406eda0 0x7fd8b4071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.947+0000 7fd8b9c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8b40aed20 con 0x7fd8b41481e0 2024-04-17T10:03:56.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.947+0000 7fd8b9c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fd8b40b44a0 con 0x7fd8b4144f20 2024-04-17T10:03:56.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.947+0000 7fd8b9c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd8b40af000 con 0x7fd8b406eda0 2024-04-17T10:03:56.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.947+0000 7fd8b2a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8b4144f20 0x7fd8b406e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.947+0000 7fd8b3400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd8b41481e0 0x7fd8b4162440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45650/0 (socket says 172.21.15.32:45650) 2024-04-17T10:03:56.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.947+0000 7fd8b3400640 1 -- 172.21.15.32:0/3542943529 learned_addr learned my addr 172.21.15.32:0/3542943529 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:56.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.947+0000 7fd8aa000640 1 --2- 172.21.15.32:0/3542943529 >> v2:172.21.15.144:3300/0 conn(0x7fd8b406eda0 0x7fd8b4071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.947+0000 7fd8b2a00640 1 -- 172.21.15.32:0/3542943529 >> v2:172.21.15.144:3300/0 conn(0x7fd8b406eda0 msgr2=0x7fd8b4071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:56.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.947+0000 7fd8b2a00640 1 --2- 172.21.15.32:0/3542943529 >> v2:172.21.15.144:3300/0 conn(0x7fd8b406eda0 0x7fd8b4071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.947+0000 7fd8b2a00640 1 -- 172.21.15.32:0/3542943529 >> v1:172.21.15.32:6789/0 conn(0x7fd8b41481e0 legacy=0x7fd8b4162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:56.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.947+0000 7fd8b2a00640 1 -- 172.21.15.32:0/3542943529 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b40af2e0 con 0x7fd8b4144f20 2024-04-17T10:03:56.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.947+0000 7fd8b2000640 1 -- 172.21.15.32:0/3542943529 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3202726870 0 0) 0x7fd8b40aed20 con 0x7fd8b41481e0 2024-04-17T10:03:56.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.947+0000 7fd8aa000640 1 --2- 172.21.15.32:0/3542943529 >> v2:172.21.15.144:3300/0 conn(0x7fd8b406eda0 0x7fd8b4071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:03:56.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.951+0000 7fd8b2a00640 1 --2- 172.21.15.32:0/3542943529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b4144f20 0x7fd8b406e860 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40b3250 tx=0x7fd8a401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec7d419074e783a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:56.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.951+0000 7fd8b2000640 1 -- 172.21.15.32:0/3542943529 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a40b0070 con 0x7fd8b4144f20 2024-04-17T10:03:56.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.951+0000 7fd8b2000640 1 -- 172.21.15.32:0/3542943529 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a40ab600 con 0x7fd8b4144f20 2024-04-17T10:03:56.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.951+0000 7fd8b2000640 1 -- 172.21.15.32:0/3542943529 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a40ab9d0 con 0x7fd8b4144f20 2024-04-17T10:03:56.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.951+0000 7fd8b9c00640 1 -- 172.21.15.32:0/3542943529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b4144f20 msgr2=0x7fd8b406e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:56.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.951+0000 7fd8b9c00640 1 --2- 172.21.15.32:0/3542943529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b4144f20 0x7fd8b406e860 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40b3250 tx=0x7fd8a401c480 comp rx=0 tx=0).stop 2024-04-17T10:03:56.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.951+0000 7fd8b9c00640 1 -- 172.21.15.32:0/3542943529 shutdown_connections 2024-04-17T10:03:56.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.951+0000 7fd8b9c00640 1 --2- 172.21.15.32:0/3542943529 >> v2:172.21.15.144:3300/0 conn(0x7fd8b406eda0 0x7fd8b4071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.951+0000 7fd8b9c00640 1 --2- 172.21.15.32:0/3542943529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b4144f20 0x7fd8b406e860 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.951+0000 7fd8b9c00640 1 -- 172.21.15.32:0/3542943529 >> 172.21.15.32:0/3542943529 conn(0x7fd8b406ba30 msgr2=0x7fd8b406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:56.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.951+0000 7fd8b9c00640 1 -- 172.21.15.32:0/3542943529 shutdown_connections 2024-04-17T10:03:56.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.951+0000 7fd8b9c00640 1 -- 172.21.15.32:0/3542943529 wait complete. 2024-04-17T10:03:56.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8b9c00640 1 Processor -- start 2024-04-17T10:03:56.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8b9c00640 1 -- start start 2024-04-17T10:03:56.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8b9c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b406eda0 0x7fd8b413c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8b9c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8b4144f20 0x7fd8b414e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8b9c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b41481e0 0x7fd8b414ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8b9c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b416b750 con 0x7fd8b406eda0 2024-04-17T10:03:56.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8b9c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b416b5d0 con 0x7fd8b41481e0 2024-04-17T10:03:56.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8b9c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b416b8d0 con 0x7fd8b4144f20 2024-04-17T10:03:56.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8b2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b406eda0 0x7fd8b413c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8b2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b406eda0 0x7fd8b413c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36190/0 (socket says 172.21.15.32:36190) 2024-04-17T10:03:56.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8b2a00640 1 -- 172.21.15.32:0/938868054 learned_addr learned my addr 172.21.15.32:0/938868054 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:56.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8b3400640 1 --2- 172.21.15.32:0/938868054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b41481e0 0x7fd8b414ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8aa000640 1 --2- 172.21.15.32:0/938868054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8b4144f20 0x7fd8b414e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8b2a00640 1 -- 172.21.15.32:0/938868054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8b4144f20 msgr2=0x7fd8b414e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:56.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8b2a00640 1 --2- 172.21.15.32:0/938868054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8b4144f20 0x7fd8b414e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8b2a00640 1 -- 172.21.15.32:0/938868054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b41481e0 msgr2=0x7fd8b414ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:56.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8b2a00640 1 --2- 172.21.15.32:0/938868054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b41481e0 0x7fd8b414ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:56.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8b2a00640 1 -- 172.21.15.32:0/938868054 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b40cd980 con 0x7fd8b406eda0 2024-04-17T10:03:56.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8aa000640 1 --2- 172.21.15.32:0/938868054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8b4144f20 0x7fd8b414e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:03:56.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8b2a00640 1 --2- 172.21.15.32:0/938868054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b406eda0 0x7fd8b413c830 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4019090 tx=0x7fd8a401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:56.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8ab400640 1 -- 172.21.15.32:0/938868054 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a40b0050 con 0x7fd8b406eda0 2024-04-17T10:03:56.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8ab400640 1 -- 172.21.15.32:0/938868054 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a40bc070 con 0x7fd8b406eda0 2024-04-17T10:03:56.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8b9c00640 1 -- 172.21.15.32:0/938868054 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b40cdc60 con 0x7fd8b406eda0 2024-04-17T10:03:56.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8ab400640 1 -- 172.21.15.32:0/938868054 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a40b5810 con 0x7fd8b406eda0 2024-04-17T10:03:56.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.955+0000 7fd8b9c00640 1 -- 172.21.15.32:0/938868054 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b40ce220 con 0x7fd8b406eda0 2024-04-17T10:03:56.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.959+0000 7fd8b9c00640 1 -- 172.21.15.32:0/938868054 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b4068a90 con 0x7fd8b406eda0 2024-04-17T10:03:56.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.959+0000 7fd8ab400640 1 -- 172.21.15.32:0/938868054 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd8a40b59b0 con 0x7fd8b406eda0 2024-04-17T10:03:56.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.959+0000 7fd8ab400640 1 --2- 172.21.15.32:0/938868054 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd890068380 0x7fd89006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:56.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.959+0000 7fd8ab400640 1 -- 172.21.15.32:0/938868054 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7fd8a41213a0 con 0x7fd8b406eda0 2024-04-17T10:03:56.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.959+0000 7fd8aa000640 1 --2- 172.21.15.32:0/938868054 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd890068380 0x7fd89006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:56.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.959+0000 7fd8aa000640 1 --2- 172.21.15.32:0/938868054 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd890068380 0x7fd89006a840 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd8b415e8b0 tx=0x7fd89c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:56.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:56.963+0000 7fd8ab400640 1 -- 172.21.15.32:0/938868054 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd8a4016070 con 0x7fd8b406eda0 2024-04-17T10:03:57.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:57.163+0000 7fd8b9c00640 1 -- 172.21.15.32:0/938868054 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd8b40b5410 con 0x7fd8b406eda0 2024-04-17T10:03:57.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:57.163+0000 7fd8ab400640 1 -- 172.21.15.32:0/938868054 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v98) ==== 53+0+73 (secure 0 0 0) 0x7fd8a40f1880 con 0x7fd8b406eda0 2024-04-17T10:03:57.163 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T10:03:57.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:57.167+0000 7fd8b9c00640 1 -- 172.21.15.32:0/938868054 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd890068380 msgr2=0x7fd89006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:57.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:57.167+0000 7fd8b9c00640 1 --2- 172.21.15.32:0/938868054 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd890068380 0x7fd89006a840 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd8b415e8b0 tx=0x7fd89c008040 comp rx=0 tx=0).stop 2024-04-17T10:03:57.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:57.167+0000 7fd8b9c00640 1 -- 172.21.15.32:0/938868054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b406eda0 msgr2=0x7fd8b413c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:57.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:57.167+0000 7fd8b9c00640 1 --2- 172.21.15.32:0/938868054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b406eda0 0x7fd8b413c830 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4019090 tx=0x7fd8a401acc0 comp rx=0 tx=0).stop 2024-04-17T10:03:57.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:57.167+0000 7fd8b9c00640 1 -- 172.21.15.32:0/938868054 shutdown_connections 2024-04-17T10:03:57.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:57.167+0000 7fd8b9c00640 1 --2- 172.21.15.32:0/938868054 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd890068380 0x7fd89006a840 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:57.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:57.167+0000 7fd8b9c00640 1 --2- 172.21.15.32:0/938868054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8b4144f20 0x7fd8b414e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:57.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:57.167+0000 7fd8b9c00640 1 --2- 172.21.15.32:0/938868054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8b41481e0 0x7fd8b414ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:57.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:57.167+0000 7fd8b9c00640 1 --2- 172.21.15.32:0/938868054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b406eda0 0x7fd8b413c830 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:57.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:57.167+0000 7fd8b9c00640 1 -- 172.21.15.32:0/938868054 >> 172.21.15.32:0/938868054 conn(0x7fd8b406ba30 msgr2=0x7fd8b406be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:57.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:57.167+0000 7fd8b9c00640 1 -- 172.21.15.32:0/938868054 shutdown_connections 2024-04-17T10:03:57.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:57.167+0000 7fd8b9c00640 1 -- 172.21.15.32:0/938868054 wait complete. 2024-04-17T10:03:57.184 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T10:03:57.184 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T10:03:57.185 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-17T10:03:57.185 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T10:03:57.185 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T10:03:57.185 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T10:03:57.185 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T10:03:57.185 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T10:03:57.185 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:03:57.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:57.185 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-17T10:03:57.194 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:03:57.194 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:03:57.194 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:03:57.194 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:03:57.195 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-17T10:03:57.195 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:03:57.195 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:03:57.195 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-17T10:03:57.195 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-17T10:03:57.195 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T10:03:57.195 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:03:57.195 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-17T10:03:57.195 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-17T10:03:57.195 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 41817sec preferred_lft 41817sec 2024-04-17T10:03:57.195 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2024-04-17T10:03:57.195 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:03:57.195 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:03:57.196 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-17T10:03:57.196 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-17T10:03:57.196 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:03:57.196 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:03:57.196 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-17T10:03:57.196 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T10:03:57.196 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:cd:f4:aa:5b brd ff:ff:ff:ff:ff:ff 2024-04-17T10:03:57.196 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T10:03:57.196 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:03:57.196 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T10:03:57.196 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T10:03:57.196 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T10:03:57.196 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:03:57.196 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T10:03:57.197 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:03:57.197 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:03:57.197 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:03:57.197 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T10:03:57.197 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:03:57.197 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:03:57.197 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:03:57.197 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T10:03:57.197 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:03:57.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:57.198 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T10:03:57.261 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-17T10:03:57.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:57.263 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-17T10:03:57.316 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-17T10:03:57.316 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T10:03:57.316 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-17T10:03:57.316 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T10:03:57.316 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-17T10:03:57.316 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T10:03:57.317 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T10:03:57.317 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T10:03:57.317 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-17T10:03:57.317 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-17T10:03:57.317 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-17T10:03:57.317 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T10:03:57.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T10:03:57.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:03:57.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:57.460 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-17T10:03:57.470 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T10:03:57.470 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T10:03:57.470 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:03:57.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:57.471 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-17T10:03:57.531 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T10:03:57.531 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:03:57.531 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T10:03:57.587 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:03:57.587 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:03:57.587 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:03:57.643 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T10:03:57.707 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:03:57.765 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T10:03:57.765 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T10:03:57.765 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T10:03:57.765 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T10:03:57.765 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T10:03:57.765 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T10:03:57.765 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T10:03:57.765 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T10:03:57.765 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T10:03:57.765 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T10:03:57.766 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T10:03:57.766 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T10:03:57.766 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:03:57.766 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T10:03:57.766 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T10:03:57.766 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T10:03:57.766 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T10:03:57.766 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T10:03:57.766 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T10:03:57.766 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T10:03:57.766 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:03:57.767 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T10:03:57.767 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T10:03:57.767 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T10:03:57.767 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:03:57.767 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:03:57.767 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:03:57.767 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:03:57.767 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T10:03:57.767 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:03:57.767 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:03:57.767 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:03:57.767 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:03:57.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:57.768 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T10:03:57.819 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T10:03:57.819 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T10:03:57.866 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T10:03:57.886 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:03:57.936 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T10:03:57.937 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T10:03:57.937 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T10:03:57.937 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T10:03:57.937 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T10:03:57.937 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T10:03:57.937 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T10:03:57.937 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T10:03:57.937 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T10:03:57.937 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T10:03:57.937 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T10:03:57.937 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T10:03:57.937 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:03:57.938 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T10:03:57.938 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T10:03:57.938 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T10:03:57.938 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T10:03:57.938 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T10:03:57.938 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T10:03:57.938 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T10:03:57.938 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:03:57.938 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T10:03:57.938 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T10:03:57.938 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T10:03:57.938 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:03:57.938 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:03:57.938 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:03:57.939 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:03:57.939 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T10:03:57.939 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:03:57.939 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:03:57.939 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:03:57.939 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:03:57.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:57.940 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T10:03:57.961 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2024-04-17T10:03:57.963+0000 7f814300d580 -1 init, newargv = 0x555cfa0bfa40 newargc=15 2024-04-17T10:03:57.961 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[51693]: starting ceph client 2024-04-17T10:03:57.968 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[51693]: starting fuse 2024-04-17T10:03:57.970 INFO:teuthology.orchestra.run.smithi032.stdout:44 2024-04-17T10:03:57.970 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-17T10:03:57.971 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2024-04-17T10:03:57.971 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2024-04-17T10:03:57.971 DEBUG:teuthology.orchestra.run.smithi032:> import re 2024-04-17T10:03:57.971 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:03:57.971 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2024-04-17T10:03:57.971 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:03:57.971 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2024-04-17T10:03:57.971 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T10:03:57.971 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2024-04-17T10:03:57.971 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T10:03:57.971 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:03:57.971 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2024-04-17T10:03:57.971 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2024-04-17T10:03:57.971 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2024-04-17T10:03:57.972 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2024-04-17T10:03:57.972 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:03:57.972 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2024-04-17T10:03:57.972 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T10:03:57.972 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T10:03:57.972 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T10:03:57.972 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2024-04-17T10:03:57.972 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2024-04-17T10:03:57.972 DEBUG:teuthology.orchestra.run.smithi032:> return f 2024-04-17T10:03:57.972 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T10:03:57.972 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:03:57.972 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2024-04-17T10:03:57.972 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:03:58.093 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.51693.asok 2024-04-17T10:03:58.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:03:58.105 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51693.asok 2024-04-17T10:03:58.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:58.105 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51693.asok status 2024-04-17T10:03:58.283 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:03:58.284 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2024-04-17T10:03:58.284 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-17T10:03:58.284 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T10:03:58.284 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2024-04-17T10:03:58.284 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2024-04-17T10:03:58.284 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T10:03:58.284 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "51693", 2024-04-17T10:03:58.284 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2024-04-17T10:03:58.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:58.284 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2024-04-17T10:03:58.284 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2024-04-17T10:03:58.284 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 9537, 2024-04-17T10:03:58.284 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2024-04-17T10:03:58.285 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2024-04-17T10:03:58.285 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2024-04-17T10:03:58.285 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 9537 2024-04-17T10:03:58.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:58.285 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T10:03:58.285 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T10:03:58.285 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:03:58.285 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2215158985 2024-04-17T10:03:58.285 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:03:58.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:58.285 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T10:03:58.285 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T10:03:58.285 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:03:58.285 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2215158985 2024-04-17T10:03:58.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:03:58.286 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.9537 192.168.0.1:0/2215158985", 2024-04-17T10:03:58.286 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/2215158985", 2024-04-17T10:03:58.286 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2024-04-17T10:03:58.286 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 98, 2024-04-17T10:03:58.286 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 111, 2024-04-17T10:03:58.286 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2024-04-17T10:03:58.286 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2024-04-17T10:03:58.286 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs" 2024-04-17T10:03:58.286 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:03:58.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:58.299 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2024-04-17T10:03:58.300 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T10:03:58.310 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2024-04-17T10:03:58.310 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T10:03:58.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:58.311 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:03:58.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:58.371 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-04-17T10:03:58.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:03:58.425 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-04-17T10:03:58.541 INFO:teuthology.orchestra.run.smithi032.stderr:8+0 records in 2024-04-17T10:03:58.542 INFO:teuthology.orchestra.run.smithi032.stderr:8+0 records out 2024-04-17T10:03:58.542 INFO:teuthology.orchestra.run.smithi032.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0643345 s, 130 MB/s 2024-04-17T10:03:58.543 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:03:58.543 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:03:58.543 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:03:58.543 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:03:58.543 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-17T10:03:58.543 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:03:58.612 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627779 2024-04-17T10:03:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:03:58.621 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:03:58.621 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:03:58.621 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:03:58.621 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:03:58.621 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-17T10:03:58.621 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:03:58.729 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627780 2024-04-17T10:03:58.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:03:58.739 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:03:58.739 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:03:58.792 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:03:58.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.975+0000 7f385aa00640 1 Processor -- start 2024-04-17T10:03:58.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.975+0000 7f385aa00640 1 -- start start 2024-04-17T10:03:58.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.979+0000 7f385aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3854144e90 0x7f38540b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:58.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.979+0000 7f385aa00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f38540b4de0 0x7f385415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:58.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.979+0000 7f385aa00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3854072bb0 con 0x7f385415bfe0 2024-04-17T10:03:58.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.979+0000 7f385aa00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f385415c430 con 0x7f38540b4de0 2024-04-17T10:03:58.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.979+0000 7f385aa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3854145270 con 0x7f3854144e90 2024-04-17T10:03:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.979+0000 7f3853e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3854144e90 0x7f38540b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.979+0000 7f384b400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f38540b4de0 0x7f385415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.979+0000 7f3858a00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f385415bfe0 0x7f385406ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:39440/0 (socket says 172.21.15.32:39440) 2024-04-17T10:03:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.979+0000 7f3858a00640 1 -- 172.21.15.32:0/1897191035 learned_addr learned my addr 172.21.15.32:0/1897191035 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.979+0000 7f384b400640 1 -- 172.21.15.32:0/1897191035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3854144e90 msgr2=0x7f38540b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.979+0000 7f384b400640 1 --2- 172.21.15.32:0/1897191035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3854144e90 0x7f38540b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.979+0000 7f384b400640 1 -- 172.21.15.32:0/1897191035 >> v1:172.21.15.144:6789/0 conn(0x7f385415bfe0 legacy=0x7f385406ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.979+0000 7f384b400640 1 -- 172.21.15.32:0/1897191035 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38540b3dc0 con 0x7f38540b4de0 2024-04-17T10:03:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.983+0000 7f384b400640 1 --2- 172.21.15.32:0/1897191035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38540b4de0 0x7f385415baa0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f383c0097f0 tx=0x7f383c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f176674f633eb8d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:58.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.983+0000 7f3853400640 1 -- 172.21.15.32:0/1897191035 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c0a7070 con 0x7f38540b4de0 2024-04-17T10:03:58.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.983+0000 7f3853400640 1 -- 172.21.15.32:0/1897191035 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f383c0b0070 con 0x7f38540b4de0 2024-04-17T10:03:58.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.983+0000 7f3853400640 1 -- 172.21.15.32:0/1897191035 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c0ab890 con 0x7f38540b4de0 2024-04-17T10:03:58.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.983+0000 7f385aa00640 1 -- 172.21.15.32:0/1897191035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38540b4de0 msgr2=0x7f385415baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:58.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.983+0000 7f385aa00640 1 --2- 172.21.15.32:0/1897191035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38540b4de0 0x7f385415baa0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f383c0097f0 tx=0x7f383c01c470 comp rx=0 tx=0).stop 2024-04-17T10:03:58.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.983+0000 7f385aa00640 1 -- 172.21.15.32:0/1897191035 shutdown_connections 2024-04-17T10:03:58.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.983+0000 7f385aa00640 1 --2- 172.21.15.32:0/1897191035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38540b4de0 0x7f385415baa0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:58.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.983+0000 7f385aa00640 1 --2- 172.21.15.32:0/1897191035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3854144e90 0x7f38540b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:58.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.983+0000 7f385aa00640 1 -- 172.21.15.32:0/1897191035 >> 172.21.15.32:0/1897191035 conn(0x7f385406ba30 msgr2=0x7f385406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:58.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.983+0000 7f385aa00640 1 -- 172.21.15.32:0/1897191035 shutdown_connections 2024-04-17T10:03:58.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.983+0000 7f385aa00640 1 -- 172.21.15.32:0/1897191035 wait complete. 2024-04-17T10:03:58.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.983+0000 7f385aa00640 1 Processor -- start 2024-04-17T10:03:58.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.983+0000 7f385aa00640 1 -- start start 2024-04-17T10:03:58.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f385aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38540b4de0 0x7f3854159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:58.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f385aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3854144e90 0x7f3854141900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:58.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f385aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f385415bfe0 0x7f38540ab250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:58.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f385aa00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3854072670 con 0x7f3854144e90 2024-04-17T10:03:58.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f385aa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f385406f880 con 0x7f38540b4de0 2024-04-17T10:03:58.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f385aa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f38540727f0 con 0x7f385415bfe0 2024-04-17T10:03:58.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f384b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3854144e90 0x7f3854141900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:58.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f384b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3854144e90 0x7f3854141900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36192/0 (socket says 172.21.15.32:36192) 2024-04-17T10:03:58.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f384b400640 1 -- 172.21.15.32:0/2147242258 learned_addr learned my addr 172.21.15.32:0/2147242258 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:58.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f3858a00640 1 --2- 172.21.15.32:0/2147242258 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f385415bfe0 0x7f38540ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:58.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f3853e00640 1 --2- 172.21.15.32:0/2147242258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38540b4de0 0x7f3854159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:58.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f384b400640 1 -- 172.21.15.32:0/2147242258 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f385415bfe0 msgr2=0x7f38540ab250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:58.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f384b400640 1 --2- 172.21.15.32:0/2147242258 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f385415bfe0 0x7f38540ab250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:58.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f384b400640 1 -- 172.21.15.32:0/2147242258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38540b4de0 msgr2=0x7f3854159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:58.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f384b400640 1 --2- 172.21.15.32:0/2147242258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38540b4de0 0x7f3854159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:58.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f384b400640 1 -- 172.21.15.32:0/2147242258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38540cd8a0 con 0x7f3854144e90 2024-04-17T10:03:58.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f384b400640 1 --2- 172.21.15.32:0/2147242258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3854144e90 0x7f3854141900 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f383c01a0a0 tx=0x7f383c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:58.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f3850c00640 1 -- 172.21.15.32:0/2147242258 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c0bf610 con 0x7f3854144e90 2024-04-17T10:03:58.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f3850c00640 1 -- 172.21.15.32:0/2147242258 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f383c0bf7b0 con 0x7f3854144e90 2024-04-17T10:03:58.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f3850c00640 1 -- 172.21.15.32:0/2147242258 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c0bfba0 con 0x7f3854144e90 2024-04-17T10:03:58.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f385aa00640 1 -- 172.21.15.32:0/2147242258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38540abaa0 con 0x7f3854144e90 2024-04-17T10:03:58.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f385aa00640 1 -- 172.21.15.32:0/2147242258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3854141100 con 0x7f3854144e90 2024-04-17T10:03:58.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f385aa00640 1 -- 172.21.15.32:0/2147242258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3854068a90 con 0x7f3854144e90 2024-04-17T10:03:58.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.987+0000 7f3850c00640 1 -- 172.21.15.32:0/2147242258 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f383c0bd020 con 0x7f3854144e90 2024-04-17T10:03:58.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.991+0000 7f3850c00640 1 --2- 172.21.15.32:0/2147242258 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3830068380 0x7f383006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:58.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.991+0000 7f3850c00640 1 -- 172.21.15.32:0/2147242258 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7f383c120e20 con 0x7f3854144e90 2024-04-17T10:03:58.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.991+0000 7f3853e00640 1 --2- 172.21.15.32:0/2147242258 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3830068380 0x7f383006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:58.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.991+0000 7f3853e00640 1 --2- 172.21.15.32:0/2147242258 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3830068380 0x7f383006a840 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3844009a50 tx=0x7f3844008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:58.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:58.995+0000 7f3850c00640 1 -- 172.21.15.32:0/2147242258 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f383c0c6030 con 0x7f3854144e90 2024-04-17T10:03:59.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.159+0000 7f385aa00640 1 -- 172.21.15.32:0/2147242258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38540cd160 con 0x7f3854144e90 2024-04-17T10:03:59.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.159+0000 7f3850c00640 1 -- 172.21.15.32:0/2147242258 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+16736 (secure 0 0 0) 0x7f383c016090 con 0x7f3854144e90 2024-04-17T10:03:59.159 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:03:59.160 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":111,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:03:56.500014+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:03:36.021322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-17T10:03:36.581134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.5","mappings":[{"from":0,"to":5}]},{"pgid":"17.25","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:03:59.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.163+0000 7f385aa00640 1 -- 172.21.15.32:0/2147242258 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3830068380 msgr2=0x7f383006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:59.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.163+0000 7f385aa00640 1 --2- 172.21.15.32:0/2147242258 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3830068380 0x7f383006a840 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3844009a50 tx=0x7f3844008040 comp rx=0 tx=0).stop 2024-04-17T10:03:59.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.163+0000 7f385aa00640 1 -- 172.21.15.32:0/2147242258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3854144e90 msgr2=0x7f3854141900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:59.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.163+0000 7f385aa00640 1 --2- 172.21.15.32:0/2147242258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3854144e90 0x7f3854141900 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f383c01a0a0 tx=0x7f383c01acb0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.163+0000 7f385aa00640 1 -- 172.21.15.32:0/2147242258 shutdown_connections 2024-04-17T10:03:59.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.163+0000 7f385aa00640 1 --2- 172.21.15.32:0/2147242258 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3830068380 0x7f383006a840 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.163+0000 7f385aa00640 1 --2- 172.21.15.32:0/2147242258 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f385415bfe0 0x7f38540ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.163+0000 7f385aa00640 1 --2- 172.21.15.32:0/2147242258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3854144e90 0x7f3854141900 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.163+0000 7f385aa00640 1 --2- 172.21.15.32:0/2147242258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38540b4de0 0x7f3854159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.163+0000 7f385aa00640 1 -- 172.21.15.32:0/2147242258 >> 172.21.15.32:0/2147242258 conn(0x7f385406ba30 msgr2=0x7f3854140470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:59.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.163+0000 7f385aa00640 1 -- 172.21.15.32:0/2147242258 shutdown_connections 2024-04-17T10:03:59.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.163+0000 7f385aa00640 1 -- 172.21.15.32:0/2147242258 wait complete. 2024-04-17T10:03:59.181 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:03:36.021322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:03:36.581134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:03:59.181 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:03:59.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.327+0000 7fed13e00640 1 Processor -- start 2024-04-17T10:03:59.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.327+0000 7fed13e00640 1 -- start start 2024-04-17T10:03:59.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.327+0000 7fed13e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fed1406e160 0x7fed140cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:59.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.327+0000 7fed13e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fed1409b100 0x7fed1415e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:59.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.327+0000 7fed13e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed140cdd90 con 0x7fed140cd8d0 2024-04-17T10:03:59.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.327+0000 7fed13e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fed140ce070 con 0x7fed1406e160 2024-04-17T10:03:59.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.327+0000 7fed13e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fed140ce350 con 0x7fed1409b100 2024-04-17T10:03:59.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.327+0000 7fed12a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fed1406e160 0x7fed140cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:59.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.327+0000 7fed12a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fed1406e160 0x7fed140cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36200/0 (socket says 172.21.15.32:36200) 2024-04-17T10:03:59.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.327+0000 7fed12a00640 1 -- 172.21.15.32:0/3799714580 learned_addr learned my addr 172.21.15.32:0/3799714580 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:59.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.327+0000 7fed13400640 1 --2- 172.21.15.32:0/3799714580 >> v2:172.21.15.144:3300/0 conn(0x7fed1409b100 0x7fed1415e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:59.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.327+0000 7fed12a00640 1 -- 172.21.15.32:0/3799714580 >> v2:172.21.15.144:3300/0 conn(0x7fed1409b100 msgr2=0x7fed1415e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:59.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.327+0000 7fed12a00640 1 --2- 172.21.15.32:0/3799714580 >> v2:172.21.15.144:3300/0 conn(0x7fed1409b100 0x7fed1415e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.327+0000 7fed12a00640 1 -- 172.21.15.32:0/3799714580 >> v1:172.21.15.144:6789/0 conn(0x7fed140cd8d0 legacy=0x7fed1409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:59.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.327+0000 7fed12a00640 1 -- 172.21.15.32:0/3799714580 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed140ce630 con 0x7fed1406e160 2024-04-17T10:03:59.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.331+0000 7fed12a00640 1 --2- 172.21.15.32:0/3799714580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed1406e160 0x7fed140cd390 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fed140c5ff0 tx=0x7fed0401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346c25657e6678e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:59.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.331+0000 7fed12000640 1 -- 172.21.15.32:0/3799714580 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed040ab5f0 con 0x7fed1406e160 2024-04-17T10:03:59.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.331+0000 7fed12000640 1 -- 172.21.15.32:0/3799714580 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed040b0070 con 0x7fed1406e160 2024-04-17T10:03:59.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.331+0000 7fed12000640 1 -- 172.21.15.32:0/3799714580 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed040ab9b0 con 0x7fed1406e160 2024-04-17T10:03:59.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.331+0000 7fed13e00640 1 -- 172.21.15.32:0/3799714580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed1406e160 msgr2=0x7fed140cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:59.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.331+0000 7fed13e00640 1 --2- 172.21.15.32:0/3799714580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed1406e160 0x7fed140cd390 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fed140c5ff0 tx=0x7fed0401c4e0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.331+0000 7fed13e00640 1 -- 172.21.15.32:0/3799714580 shutdown_connections 2024-04-17T10:03:59.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.331+0000 7fed13e00640 1 --2- 172.21.15.32:0/3799714580 >> v2:172.21.15.144:3300/0 conn(0x7fed1409b100 0x7fed1415e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.331+0000 7fed13e00640 1 --2- 172.21.15.32:0/3799714580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed1406e160 0x7fed140cd390 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.331+0000 7fed13e00640 1 -- 172.21.15.32:0/3799714580 >> 172.21.15.32:0/3799714580 conn(0x7fed140b4000 msgr2=0x7fed140b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:59.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.331+0000 7fed13e00640 1 -- 172.21.15.32:0/3799714580 shutdown_connections 2024-04-17T10:03:59.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.331+0000 7fed13e00640 1 -- 172.21.15.32:0/3799714580 wait complete. 2024-04-17T10:03:59.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed13e00640 1 Processor -- start 2024-04-17T10:03:59.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed13e00640 1 -- start start 2024-04-17T10:03:59.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed13e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed1406e160 0x7fed14153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:59.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed13e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed1409b100 0x7fed141614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:59.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed13e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed140cd8d0 0x7fed14161a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:59.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed13e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fed1406cc90 con 0x7fed140cd8d0 2024-04-17T10:03:59.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed13e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fed1406cb10 con 0x7fed1409b100 2024-04-17T10:03:59.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed13e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fed1406ce10 con 0x7fed1406e160 2024-04-17T10:03:59.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed13400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed140cd8d0 0x7fed14161a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:59.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed13400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed140cd8d0 0x7fed14161a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36208/0 (socket says 172.21.15.32:36208) 2024-04-17T10:03:59.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed13400640 1 -- 172.21.15.32:0/2535809044 learned_addr learned my addr 172.21.15.32:0/2535809044 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:59.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed12a00640 1 --2- 172.21.15.32:0/2535809044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed1406e160 0x7fed14153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:59.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed0a000640 1 --2- 172.21.15.32:0/2535809044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed1409b100 0x7fed141614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:59.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed13400640 1 -- 172.21.15.32:0/2535809044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed1406e160 msgr2=0x7fed14153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:59.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed13400640 1 --2- 172.21.15.32:0/2535809044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed1406e160 0x7fed14153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed13400640 1 -- 172.21.15.32:0/2535809044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed1409b100 msgr2=0x7fed141614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:59.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed13400640 1 --2- 172.21.15.32:0/2535809044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed1409b100 0x7fed141614f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed13400640 1 -- 172.21.15.32:0/2535809044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1415ffc0 con 0x7fed140cd8d0 2024-04-17T10:03:59.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed13400640 1 --2- 172.21.15.32:0/2535809044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed140cd8d0 0x7fed14161a30 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fed0c007e40 tx=0x7fed0c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:59.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed0b400640 1 -- 172.21.15.32:0/2535809044 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c014620 con 0x7fed140cd8d0 2024-04-17T10:03:59.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed0b400640 1 -- 172.21.15.32:0/2535809044 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed0c019070 con 0x7fed140cd8d0 2024-04-17T10:03:59.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed0b400640 1 -- 172.21.15.32:0/2535809044 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c014a20 con 0x7fed140cd8d0 2024-04-17T10:03:59.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed13e00640 1 -- 172.21.15.32:0/2535809044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed141602a0 con 0x7fed140cd8d0 2024-04-17T10:03:59.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.335+0000 7fed13e00640 1 -- 172.21.15.32:0/2535809044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed14160860 con 0x7fed140cd8d0 2024-04-17T10:03:59.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.339+0000 7fed13e00640 1 -- 172.21.15.32:0/2535809044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed1404fda0 con 0x7fed140cd8d0 2024-04-17T10:03:59.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.339+0000 7fed0b400640 1 -- 172.21.15.32:0/2535809044 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fed0c014bc0 con 0x7fed140cd8d0 2024-04-17T10:03:59.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.339+0000 7fed0b400640 1 --2- 172.21.15.32:0/2535809044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fecf0068380 0x7fecf006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:59.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.339+0000 7fed0b400640 1 -- 172.21.15.32:0/2535809044 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7fed0c0871c0 con 0x7fed140cd8d0 2024-04-17T10:03:59.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.339+0000 7fed12a00640 1 --2- 172.21.15.32:0/2535809044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fecf0068380 0x7fecf006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:59.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.339+0000 7fed12a00640 1 --2- 172.21.15.32:0/2535809044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fecf0068380 0x7fecf006a840 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fed040af000 tx=0x7fed040a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:59.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.343+0000 7fed0b400640 1 -- 172.21.15.32:0/2535809044 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fed0c0570e0 con 0x7fed140cd8d0 2024-04-17T10:03:59.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.511+0000 7fed13e00640 1 -- 172.21.15.32:0/2535809044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fed140008e0 con 0x7fed140cd8d0 2024-04-17T10:03:59.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.515+0000 7fed0b400640 1 -- 172.21.15.32:0/2535809044 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) ==== 93+0+10 (secure 0 0 0) 0x7fed0c05b3e0 con 0x7fed140cd8d0 2024-04-17T10:03:59.512 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T10:03:59.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.515+0000 7fed13e00640 1 -- 172.21.15.32:0/2535809044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fecf0068380 msgr2=0x7fecf006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:59.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.515+0000 7fed13e00640 1 --2- 172.21.15.32:0/2535809044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fecf0068380 0x7fecf006a840 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fed040af000 tx=0x7fed040a7000 comp rx=0 tx=0).stop 2024-04-17T10:03:59.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.515+0000 7fed13e00640 1 -- 172.21.15.32:0/2535809044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed140cd8d0 msgr2=0x7fed14161a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:59.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.515+0000 7fed13e00640 1 --2- 172.21.15.32:0/2535809044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed140cd8d0 0x7fed14161a30 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fed0c007e40 tx=0x7fed0c00e4b0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.515+0000 7fed13e00640 1 -- 172.21.15.32:0/2535809044 shutdown_connections 2024-04-17T10:03:59.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.515+0000 7fed13e00640 1 --2- 172.21.15.32:0/2535809044 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fecf0068380 0x7fecf006a840 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.515+0000 7fed13e00640 1 --2- 172.21.15.32:0/2535809044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed140cd8d0 0x7fed14161a30 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.515+0000 7fed13e00640 1 --2- 172.21.15.32:0/2535809044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed1409b100 0x7fed141614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.515+0000 7fed13e00640 1 --2- 172.21.15.32:0/2535809044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed1406e160 0x7fed14153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.515+0000 7fed13e00640 1 -- 172.21.15.32:0/2535809044 >> 172.21.15.32:0/2535809044 conn(0x7fed140b4000 msgr2=0x7fed140b0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:59.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.515+0000 7fed13e00640 1 -- 172.21.15.32:0/2535809044 shutdown_connections 2024-04-17T10:03:59.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.515+0000 7fed13e00640 1 -- 172.21.15.32:0/2535809044 wait complete. 2024-04-17T10:03:59.535 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:03:59.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.711+0000 7f5dff800640 1 Processor -- start 2024-04-17T10:03:59.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.711+0000 7f5dff800640 1 -- start start 2024-04-17T10:03:59.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.711+0000 7f5dff800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5df80b2230 0x7f5df80b2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:59.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.711+0000 7f5dff800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5df80ccf50 0x7f5df80cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:59.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.711+0000 7f5dff800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5df80b2b50 con 0x7f5df806ba30 2024-04-17T10:03:59.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.711+0000 7f5dff800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f5df815d080 con 0x7f5df80ccf50 2024-04-17T10:03:59.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.711+0000 7f5dff800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5df80ce870 con 0x7f5df80b2230 2024-04-17T10:03:59.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.711+0000 7f5dfce00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5df80b2230 0x7f5df80b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:59.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.711+0000 7f5dfd800640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f5df806ba30 0x7f5df806bec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:39470/0 (socket says 172.21.15.32:39470) 2024-04-17T10:03:59.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.711+0000 7f5dfd800640 1 -- 172.21.15.32:0/2063305575 learned_addr learned my addr 172.21.15.32:0/2063305575 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:03:59.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.711+0000 7f5defe00640 1 --2- 172.21.15.32:0/2063305575 >> v2:172.21.15.144:3300/0 conn(0x7f5df80ccf50 0x7f5df80cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:59.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.711+0000 7f5dfce00640 1 -- 172.21.15.32:0/2063305575 >> v2:172.21.15.144:3300/0 conn(0x7f5df80ccf50 msgr2=0x7f5df80cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:03:59.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.711+0000 7f5dfce00640 1 --2- 172.21.15.32:0/2063305575 >> v2:172.21.15.144:3300/0 conn(0x7f5df80ccf50 0x7f5df80cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.711+0000 7f5dfce00640 1 -- 172.21.15.32:0/2063305575 >> v1:172.21.15.144:6789/0 conn(0x7f5df806ba30 legacy=0x7f5df806bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:59.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.711+0000 7f5dfce00640 1 -- 172.21.15.32:0/2063305575 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df80cdc60 con 0x7f5df80b2230 2024-04-17T10:03:59.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.715+0000 7f5dfce00640 1 --2- 172.21.15.32:0/2063305575 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5df80b2230 0x7f5df80b2610 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5de80097c0 tx=0x7f5de801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e75d0f85801aa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:59.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.715+0000 7f5df7e00640 1 -- 172.21.15.32:0/2063305575 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de80b0070 con 0x7f5df80b2230 2024-04-17T10:03:59.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.715+0000 7f5df7e00640 1 -- 172.21.15.32:0/2063305575 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de80ab600 con 0x7f5df80b2230 2024-04-17T10:03:59.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.715+0000 7f5df7e00640 1 -- 172.21.15.32:0/2063305575 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de80ab9d0 con 0x7f5df80b2230 2024-04-17T10:03:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.715+0000 7f5dff800640 1 -- 172.21.15.32:0/2063305575 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5df80b2230 msgr2=0x7f5df80b2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.715+0000 7f5dff800640 1 --2- 172.21.15.32:0/2063305575 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5df80b2230 0x7f5df80b2610 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5de80097c0 tx=0x7f5de801c410 comp rx=0 tx=0).stop 2024-04-17T10:03:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.715+0000 7f5dff800640 1 -- 172.21.15.32:0/2063305575 shutdown_connections 2024-04-17T10:03:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.715+0000 7f5dff800640 1 --2- 172.21.15.32:0/2063305575 >> v2:172.21.15.144:3300/0 conn(0x7f5df80ccf50 0x7f5df80cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.715+0000 7f5dff800640 1 --2- 172.21.15.32:0/2063305575 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5df80b2230 0x7f5df80b2610 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.715+0000 7f5dff800640 1 -- 172.21.15.32:0/2063305575 >> 172.21.15.32:0/2063305575 conn(0x7f5df80d1430 msgr2=0x7f5df80d1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.715+0000 7f5dff800640 1 -- 172.21.15.32:0/2063305575 shutdown_connections 2024-04-17T10:03:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.715+0000 7f5dff800640 1 -- 172.21.15.32:0/2063305575 wait complete. 2024-04-17T10:03:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5dff800640 1 Processor -- start 2024-04-17T10:03:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5dff800640 1 -- start start 2024-04-17T10:03:59.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5dff800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5df806ba30 0x7f5df80cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:59.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5dff800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5df80b2230 0x7f5df80c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:59.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5dff800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5df80ccf50 0x7f5df8155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:59.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5dff800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5df815ee60 con 0x7f5df806ba30 2024-04-17T10:03:59.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5dff800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5df80cd910 con 0x7f5df80b2230 2024-04-17T10:03:59.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5dff800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5df815efe0 con 0x7f5df80ccf50 2024-04-17T10:03:59.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5dfce00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5df806ba30 0x7f5df80cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:59.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5dfce00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5df806ba30 0x7f5df80cacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36222/0 (socket says 172.21.15.32:36222) 2024-04-17T10:03:59.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5dfce00640 1 -- 172.21.15.32:0/578613092 learned_addr learned my addr 172.21.15.32:0/578613092 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:03:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5defe00640 1 --2- 172.21.15.32:0/578613092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5df80b2230 0x7f5df80c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5dfd800640 1 --2- 172.21.15.32:0/578613092 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5df80ccf50 0x7f5df8155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5dfce00640 1 -- 172.21.15.32:0/578613092 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5df80ccf50 msgr2=0x7f5df8155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5dfce00640 1 --2- 172.21.15.32:0/578613092 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5df80ccf50 0x7f5df8155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5dfce00640 1 -- 172.21.15.32:0/578613092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5df80b2230 msgr2=0x7f5df80c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5dfce00640 1 --2- 172.21.15.32:0/578613092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5df80b2230 0x7f5df80c9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5dfce00640 1 -- 172.21.15.32:0/578613092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df80c5be0 con 0x7f5df806ba30 2024-04-17T10:03:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5dfce00640 1 --2- 172.21.15.32:0/578613092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5df806ba30 0x7f5df80cacf0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f5de801aac0 tx=0x7f5de801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5df5600640 1 -- 172.21.15.32:0/578613092 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de80b0050 con 0x7f5df806ba30 2024-04-17T10:03:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5df5600640 1 -- 172.21.15.32:0/578613092 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de80bc070 con 0x7f5df806ba30 2024-04-17T10:03:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5df5600640 1 -- 172.21.15.32:0/578613092 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de80b5830 con 0x7f5df806ba30 2024-04-17T10:03:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5dff800640 1 -- 172.21.15.32:0/578613092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df80c5250 con 0x7f5df806ba30 2024-04-17T10:03:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.719+0000 7f5dff800640 1 -- 172.21.15.32:0/578613092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df80c6060 con 0x7f5df806ba30 2024-04-17T10:03:59.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.723+0000 7f5dff800640 1 -- 172.21.15.32:0/578613092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df8068a90 con 0x7f5df806ba30 2024-04-17T10:03:59.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.723+0000 7f5df5600640 1 -- 172.21.15.32:0/578613092 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5de80b59d0 con 0x7f5df806ba30 2024-04-17T10:03:59.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.723+0000 7f5df5600640 1 --2- 172.21.15.32:0/578613092 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5dd4068380 0x7f5dd406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:03:59.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.723+0000 7f5df5600640 1 -- 172.21.15.32:0/578613092 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7f5de8121230 con 0x7f5df806ba30 2024-04-17T10:03:59.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.723+0000 7f5defe00640 1 --2- 172.21.15.32:0/578613092 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5dd4068380 0x7f5dd406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:03:59.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.723+0000 7f5defe00640 1 --2- 172.21.15.32:0/578613092 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5dd4068380 0x7f5dd406a840 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5df81fad50 tx=0x7f5de0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:03:59.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.727+0000 7f5df5600640 1 -- 172.21.15.32:0/578613092 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5de8016090 con 0x7f5df806ba30 2024-04-17T10:03:59.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.891+0000 7f5dff800640 1 -- 172.21.15.32:0/578613092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5df8156f10 con 0x7f5df806ba30 2024-04-17T10:03:59.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.891+0000 7f5df5600640 1 -- 172.21.15.32:0/578613092 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v111) ==== 104+0+11 (secure 0 0 0) 0x7f5de80f17a0 con 0x7f5df806ba30 2024-04-17T10:03:59.891 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T10:03:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.895+0000 7f5dff800640 1 -- 172.21.15.32:0/578613092 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5dd4068380 msgr2=0x7f5dd406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.895+0000 7f5dff800640 1 --2- 172.21.15.32:0/578613092 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5dd4068380 0x7f5dd406a840 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5df81fad50 tx=0x7f5de0008040 comp rx=0 tx=0).stop 2024-04-17T10:03:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.895+0000 7f5dff800640 1 -- 172.21.15.32:0/578613092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5df806ba30 msgr2=0x7f5df80cacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:03:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.895+0000 7f5dff800640 1 --2- 172.21.15.32:0/578613092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5df806ba30 0x7f5df80cacf0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f5de801aac0 tx=0x7f5de801ac50 comp rx=0 tx=0).stop 2024-04-17T10:03:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.895+0000 7f5dff800640 1 -- 172.21.15.32:0/578613092 shutdown_connections 2024-04-17T10:03:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.895+0000 7f5dff800640 1 --2- 172.21.15.32:0/578613092 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5dd4068380 0x7f5dd406a840 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.895+0000 7f5dff800640 1 --2- 172.21.15.32:0/578613092 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5df80ccf50 0x7f5df8155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.895+0000 7f5dff800640 1 --2- 172.21.15.32:0/578613092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5df80b2230 0x7f5df80c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.895+0000 7f5dff800640 1 --2- 172.21.15.32:0/578613092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5df806ba30 0x7f5df80cacf0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:03:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.895+0000 7f5dff800640 1 -- 172.21.15.32:0/578613092 >> 172.21.15.32:0/578613092 conn(0x7f5df80d1430 msgr2=0x7f5df80705b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:03:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.895+0000 7f5dff800640 1 -- 172.21.15.32:0/578613092 shutdown_connections 2024-04-17T10:03:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:03:59.895+0000 7f5dff800640 1 -- 172.21.15.32:0/578613092 wait complete. 2024-04-17T10:03:59.912 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:04:00.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.095+0000 7fcbf0c00640 1 Processor -- start 2024-04-17T10:04:00.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.095+0000 7fcbf0c00640 1 -- start start 2024-04-17T10:04:00.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.095+0000 7fcbf0c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcbec0cf370 0x7fcbec0cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:00.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.095+0000 7fcbf0c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbec071e60 0x7fcbec072270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:00.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.095+0000 7fcbf0c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbec0d0130 con 0x7fcbec06ba30 2024-04-17T10:04:00.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.095+0000 7fcbf0c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fcbec15d090 con 0x7fcbec0cf370 2024-04-17T10:04:00.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.095+0000 7fcbf0c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcbec15d370 con 0x7fcbec071e60 2024-04-17T10:04:00.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.095+0000 7fcbea800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbec071e60 0x7fcbec072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:00.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.095+0000 7fcbe3e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcbec0cf370 0x7fcbec0cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:00.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.095+0000 7fcbe9e00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fcbec06ba30 0x7fcbec06be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:39474/0 (socket says 172.21.15.32:39474) 2024-04-17T10:04:00.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.095+0000 7fcbe9e00640 1 -- 172.21.15.32:0/651175863 learned_addr learned my addr 172.21.15.32:0/651175863 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:00.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.099+0000 7fcbe3e00640 1 -- 172.21.15.32:0/651175863 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbec071e60 msgr2=0x7fcbec072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:00.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.099+0000 7fcbe3e00640 1 --2- 172.21.15.32:0/651175863 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbec071e60 0x7fcbec072270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:00.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.099+0000 7fcbe3e00640 1 -- 172.21.15.32:0/651175863 >> v1:172.21.15.144:6789/0 conn(0x7fcbec06ba30 legacy=0x7fcbec06be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:00.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.099+0000 7fcbe3e00640 1 -- 172.21.15.32:0/651175863 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbec15d650 con 0x7fcbec0cf370 2024-04-17T10:04:00.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.099+0000 7fcbe9400640 1 -- 172.21.15.32:0/651175863 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2210083977 0 0) 0x7fcbec0d0130 con 0x7fcbec06ba30 2024-04-17T10:04:00.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.099+0000 7fcbe3e00640 1 --2- 172.21.15.32:0/651175863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbec0cf370 0x7fcbec0cf7e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcbd40097c0 tx=0x7fcbd401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13e993d3482a4bb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:00.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.099+0000 7fcbe9400640 1 -- 172.21.15.32:0/651175863 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd40aa620 con 0x7fcbec0cf370 2024-04-17T10:04:00.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.099+0000 7fcbe9400640 1 -- 172.21.15.32:0/651175863 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbd40af070 con 0x7fcbec0cf370 2024-04-17T10:04:00.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.099+0000 7fcbe9400640 1 -- 172.21.15.32:0/651175863 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd40aa9e0 con 0x7fcbec0cf370 2024-04-17T10:04:00.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.103+0000 7fcbf0c00640 1 -- 172.21.15.32:0/651175863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbec0cf370 msgr2=0x7fcbec0cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:00.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.103+0000 7fcbf0c00640 1 --2- 172.21.15.32:0/651175863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbec0cf370 0x7fcbec0cf7e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcbd40097c0 tx=0x7fcbd401c4a0 comp rx=0 tx=0).stop 2024-04-17T10:04:00.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.103+0000 7fcbf0c00640 1 -- 172.21.15.32:0/651175863 shutdown_connections 2024-04-17T10:04:00.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.103+0000 7fcbf0c00640 1 --2- 172.21.15.32:0/651175863 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbec071e60 0x7fcbec072270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:00.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.103+0000 7fcbf0c00640 1 --2- 172.21.15.32:0/651175863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbec0cf370 0x7fcbec0cf7e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:00.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.103+0000 7fcbf0c00640 1 -- 172.21.15.32:0/651175863 >> 172.21.15.32:0/651175863 conn(0x7fcbec0b2120 msgr2=0x7fcbec0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:00.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.103+0000 7fcbf0c00640 1 -- 172.21.15.32:0/651175863 shutdown_connections 2024-04-17T10:04:00.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.103+0000 7fcbf0c00640 1 -- 172.21.15.32:0/651175863 wait complete. 2024-04-17T10:04:00.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.103+0000 7fcbf0c00640 1 Processor -- start 2024-04-17T10:04:00.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.103+0000 7fcbf0c00640 1 -- start start 2024-04-17T10:04:00.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.107+0000 7fcbf0c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbec06ba30 0x7fcbec13cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:00.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.107+0000 7fcbf0c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbec071e60 0x7fcbec0ae080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:00.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.107+0000 7fcbf0c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbec0cf370 0x7fcbec06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:00.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.107+0000 7fcbf0c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec146230 con 0x7fcbec06ba30 2024-04-17T10:04:00.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.107+0000 7fcbf0c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec15dd80 con 0x7fcbec0cf370 2024-04-17T10:04:00.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.107+0000 7fcbf0c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcbec1463b0 con 0x7fcbec071e60 2024-04-17T10:04:00.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.107+0000 7fcbe9e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbec06ba30 0x7fcbec13cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:00.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.107+0000 7fcbe9e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbec06ba30 0x7fcbec13cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36224/0 (socket says 172.21.15.32:36224) 2024-04-17T10:04:00.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.107+0000 7fcbe9e00640 1 -- 172.21.15.32:0/2010386140 learned_addr learned my addr 172.21.15.32:0/2010386140 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:00.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.107+0000 7fcbea800640 1 --2- 172.21.15.32:0/2010386140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbec0cf370 0x7fcbec06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:00.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.107+0000 7fcbe3e00640 1 --2- 172.21.15.32:0/2010386140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbec071e60 0x7fcbec0ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:00.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.107+0000 7fcbe9e00640 1 -- 172.21.15.32:0/2010386140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbec071e60 msgr2=0x7fcbec0ae080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:00.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.107+0000 7fcbe9e00640 1 --2- 172.21.15.32:0/2010386140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbec071e60 0x7fcbec0ae080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:00.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.107+0000 7fcbe9e00640 1 -- 172.21.15.32:0/2010386140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbec0cf370 msgr2=0x7fcbec06e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:00.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.107+0000 7fcbe9e00640 1 --2- 172.21.15.32:0/2010386140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbec0cf370 0x7fcbec06e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:00.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.107+0000 7fcbe9e00640 1 -- 172.21.15.32:0/2010386140 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbec158a90 con 0x7fcbec06ba30 2024-04-17T10:04:00.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.107+0000 7fcbe9e00640 1 --2- 172.21.15.32:0/2010386140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbec06ba30 0x7fcbec13cea0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc00c550 tx=0x7fcbdc00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:00.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.107+0000 7fcbe2000640 1 -- 172.21.15.32:0/2010386140 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc017070 con 0x7fcbec06ba30 2024-04-17T10:04:00.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.107+0000 7fcbe2000640 1 -- 172.21.15.32:0/2010386140 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbdc005d10 con 0x7fcbec06ba30 2024-04-17T10:04:00.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.107+0000 7fcbf0c00640 1 -- 172.21.15.32:0/2010386140 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbec158d70 con 0x7fcbec06ba30 2024-04-17T10:04:00.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.107+0000 7fcbe2000640 1 -- 172.21.15.32:0/2010386140 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc007330 con 0x7fcbec06ba30 2024-04-17T10:04:00.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.107+0000 7fcbf0c00640 1 -- 172.21.15.32:0/2010386140 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbec13ee20 con 0x7fcbec06ba30 2024-04-17T10:04:00.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.107+0000 7fcbf0c00640 1 -- 172.21.15.32:0/2010386140 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbec068a90 con 0x7fcbec06ba30 2024-04-17T10:04:00.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.111+0000 7fcbe2000640 1 -- 172.21.15.32:0/2010386140 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcbdc01a070 con 0x7fcbec06ba30 2024-04-17T10:04:00.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.111+0000 7fcbe2000640 1 --2- 172.21.15.32:0/2010386140 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcbc8068380 0x7fcbc806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:00.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.111+0000 7fcbe2000640 1 -- 172.21.15.32:0/2010386140 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7fcbdc0861b0 con 0x7fcbec06ba30 2024-04-17T10:04:00.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.111+0000 7fcbe3e00640 1 --2- 172.21.15.32:0/2010386140 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcbc8068380 0x7fcbc806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:00.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.111+0000 7fcbe3e00640 1 --2- 172.21.15.32:0/2010386140 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcbc8068380 0x7fcbc806a840 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fcbd40ae000 tx=0x7fcbd40b5040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:00.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.119+0000 7fcbe2000640 1 -- 172.21.15.32:0/2010386140 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fcbdc021020 con 0x7fcbec06ba30 2024-04-17T10:04:00.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.283+0000 7fcbf0c00640 1 -- 172.21.15.32:0/2010386140 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcbec0afcd0 con 0x7fcbec06ba30 2024-04-17T10:04:00.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.283+0000 7fcbe2000640 1 -- 172.21.15.32:0/2010386140 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v111) ==== 100+0+11 (secure 0 0 0) 0x7fcbdc056150 con 0x7fcbec06ba30 2024-04-17T10:04:00.281 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T10:04:00.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.283+0000 7fcbf0c00640 1 -- 172.21.15.32:0/2010386140 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcbc8068380 msgr2=0x7fcbc806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:00.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.283+0000 7fcbf0c00640 1 --2- 172.21.15.32:0/2010386140 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcbc8068380 0x7fcbc806a840 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fcbd40ae000 tx=0x7fcbd40b5040 comp rx=0 tx=0).stop 2024-04-17T10:04:00.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.283+0000 7fcbf0c00640 1 -- 172.21.15.32:0/2010386140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbec06ba30 msgr2=0x7fcbec13cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:00.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.283+0000 7fcbf0c00640 1 --2- 172.21.15.32:0/2010386140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbec06ba30 0x7fcbec13cea0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc00c550 tx=0x7fcbdc00ca10 comp rx=0 tx=0).stop 2024-04-17T10:04:00.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.283+0000 7fcbf0c00640 1 -- 172.21.15.32:0/2010386140 shutdown_connections 2024-04-17T10:04:00.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.283+0000 7fcbf0c00640 1 --2- 172.21.15.32:0/2010386140 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcbc8068380 0x7fcbc806a840 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:00.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.283+0000 7fcbf0c00640 1 --2- 172.21.15.32:0/2010386140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbec071e60 0x7fcbec0ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:00.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.283+0000 7fcbf0c00640 1 --2- 172.21.15.32:0/2010386140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbec0cf370 0x7fcbec06e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:00.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.283+0000 7fcbf0c00640 1 --2- 172.21.15.32:0/2010386140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbec06ba30 0x7fcbec13cea0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:00.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.283+0000 7fcbf0c00640 1 -- 172.21.15.32:0/2010386140 >> 172.21.15.32:0/2010386140 conn(0x7fcbec0b2120 msgr2=0x7fcbec0aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:00.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.283+0000 7fcbf0c00640 1 -- 172.21.15.32:0/2010386140 shutdown_connections 2024-04-17T10:04:00.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.283+0000 7fcbf0c00640 1 -- 172.21.15.32:0/2010386140 wait complete. 2024-04-17T10:04:00.302 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:04:00.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.447+0000 7f3479800640 1 Processor -- start 2024-04-17T10:04:00.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.447+0000 7f3479800640 1 -- start start 2024-04-17T10:04:00.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.447+0000 7f3479800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34740735d0 0x7f3474166320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:00.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.447+0000 7f3479800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34740ce740 con 0x7f347415b560 2024-04-17T10:04:00.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.447+0000 7f3479800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34740ce9f0 con 0x7f34740b3000 2024-04-17T10:04:00.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.447+0000 7f3479800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f34740cecd0 con 0x7f34740735d0 2024-04-17T10:04:00.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.447+0000 7f3478e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34740735d0 0x7f3474166320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:00.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.447+0000 7f3473e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f347415b560 0x7f34740ce030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35910/0 (socket says 172.21.15.32:35910) 2024-04-17T10:04:00.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.447+0000 7f3473e00640 1 -- 172.21.15.32:0/2828702960 learned_addr learned my addr 172.21.15.32:0/2828702960 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:00.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.447+0000 7f3478e00640 1 -- 172.21.15.32:0/2828702960 >> v1:172.21.15.144:6789/0 conn(0x7f34740b3000 legacy=0x7f34740b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:00.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.447+0000 7f3478e00640 1 -- 172.21.15.32:0/2828702960 >> v1:172.21.15.32:6789/0 conn(0x7f347415b560 legacy=0x7f34740ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:00.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.447+0000 7f3478e00640 1 -- 172.21.15.32:0/2828702960 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3474071dc0 con 0x7f34740735d0 2024-04-17T10:04:00.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.451+0000 7f3478e00640 1 --2- 172.21.15.32:0/2828702960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34740735d0 0x7f3474166320 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f346c009d90 tx=0x7f346c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780286dffbc16018 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:00.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.451+0000 7f3473400640 1 -- 172.21.15.32:0/2828702960 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c0b2070 con 0x7f34740735d0 2024-04-17T10:04:00.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.451+0000 7f3473400640 1 -- 172.21.15.32:0/2828702960 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f346c0a5830 con 0x7f34740735d0 2024-04-17T10:04:00.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.451+0000 7f3473400640 1 -- 172.21.15.32:0/2828702960 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c0a5c00 con 0x7f34740735d0 2024-04-17T10:04:00.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.451+0000 7f3479800640 1 -- 172.21.15.32:0/2828702960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34740735d0 msgr2=0x7f3474166320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:00.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.451+0000 7f3479800640 1 --2- 172.21.15.32:0/2828702960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34740735d0 0x7f3474166320 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f346c009d90 tx=0x7f346c01c970 comp rx=0 tx=0).stop 2024-04-17T10:04:00.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.451+0000 7f3479800640 1 -- 172.21.15.32:0/2828702960 shutdown_connections 2024-04-17T10:04:00.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.451+0000 7f3479800640 1 --2- 172.21.15.32:0/2828702960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34740735d0 0x7f3474166320 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:00.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.451+0000 7f3479800640 1 -- 172.21.15.32:0/2828702960 >> 172.21.15.32:0/2828702960 conn(0x7f347406bc00 msgr2=0x7f347406c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:00.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.451+0000 7f3479800640 1 -- 172.21.15.32:0/2828702960 shutdown_connections 2024-04-17T10:04:00.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.451+0000 7f3479800640 1 -- 172.21.15.32:0/2828702960 wait complete. 2024-04-17T10:04:00.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f3479800640 1 Processor -- start 2024-04-17T10:04:00.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f3479800640 1 -- start start 2024-04-17T10:04:00.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f3479800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34740735d0 0x7f34740c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:00.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f3479800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34740b3000 0x7f347409d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:00.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f3479800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f347415b560 0x7f3474159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:00.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f3479800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f34741696a0 con 0x7f34740b3000 2024-04-17T10:04:00.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f3479800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f34740cfef0 con 0x7f347415b560 2024-04-17T10:04:00.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f3479800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3474169820 con 0x7f34740735d0 2024-04-17T10:04:00.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f346be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34740b3000 0x7f347409d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:00.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f346be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34740b3000 0x7f347409d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40932/0 (socket says 172.21.15.32:40932) 2024-04-17T10:04:00.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f346be00640 1 -- 172.21.15.32:0/1285024596 learned_addr learned my addr 172.21.15.32:0/1285024596 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:00.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f3473e00640 1 --2- 172.21.15.32:0/1285024596 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34740735d0 0x7f34740c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:00.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f3478e00640 1 --2- 172.21.15.32:0/1285024596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f347415b560 0x7f3474159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:00.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f346be00640 1 -- 172.21.15.32:0/1285024596 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34740735d0 msgr2=0x7f34740c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:00.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f346be00640 1 --2- 172.21.15.32:0/1285024596 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34740735d0 0x7f34740c9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:00.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f346be00640 1 -- 172.21.15.32:0/1285024596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f347415b560 msgr2=0x7f3474159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:00.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f346be00640 1 --2- 172.21.15.32:0/1285024596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f347415b560 0x7f3474159360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:00.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f346be00640 1 -- 172.21.15.32:0/1285024596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34740ca060 con 0x7f34740b3000 2024-04-17T10:04:00.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f346be00640 1 --2- 172.21.15.32:0/1285024596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34740b3000 0x7f347409d7b0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f345c00c450 tx=0x7f345c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:00.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f3470c00640 1 -- 172.21.15.32:0/1285024596 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c016020 con 0x7f34740b3000 2024-04-17T10:04:00.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f3470c00640 1 -- 172.21.15.32:0/1285024596 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f345c0052d0 con 0x7f34740b3000 2024-04-17T10:04:00.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f3470c00640 1 -- 172.21.15.32:0/1285024596 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c007260 con 0x7f34740b3000 2024-04-17T10:04:00.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f3479800640 1 -- 172.21.15.32:0/1285024596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34740c3e90 con 0x7f34740b3000 2024-04-17T10:04:00.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f3479800640 1 -- 172.21.15.32:0/1285024596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34740c5f40 con 0x7f34740b3000 2024-04-17T10:04:00.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.455+0000 7f3479800640 1 -- 172.21.15.32:0/1285024596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f347404fd00 con 0x7f34740b3000 2024-04-17T10:04:00.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.459+0000 7f3470c00640 1 -- 172.21.15.32:0/1285024596 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f345c007400 con 0x7f34740b3000 2024-04-17T10:04:00.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.459+0000 7f3470c00640 1 --2- 172.21.15.32:0/1285024596 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3450068380 0x7f345006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:00.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.459+0000 7f3470c00640 1 -- 172.21.15.32:0/1285024596 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7f345c0862a0 con 0x7f34740b3000 2024-04-17T10:04:00.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.459+0000 7f3473e00640 1 --2- 172.21.15.32:0/1285024596 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3450068380 0x7f345006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:00.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.459+0000 7f3473e00640 1 --2- 172.21.15.32:0/1285024596 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3450068380 0x7f345006a840 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3464005890 tx=0x7f3464009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:00.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.463+0000 7f3470c00640 1 -- 172.21.15.32:0/1285024596 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f345c020020 con 0x7f34740b3000 2024-04-17T10:04:00.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.639+0000 7f3479800640 1 -- 172.21.15.32:0/1285024596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3474073ee0 con 0x7f34740b3000 2024-04-17T10:04:00.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.639+0000 7f3470c00640 1 -- 172.21.15.32:0/1285024596 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v111) ==== 80+0+0 (secure 0 0 0) 0x7f345c005480 con 0x7f34740b3000 2024-04-17T10:04:00.639 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T10:04:00.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.643+0000 7f3479800640 1 -- 172.21.15.32:0/1285024596 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3450068380 msgr2=0x7f345006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:00.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.643+0000 7f3479800640 1 --2- 172.21.15.32:0/1285024596 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3450068380 0x7f345006a840 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3464005890 tx=0x7f3464009040 comp rx=0 tx=0).stop 2024-04-17T10:04:00.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.643+0000 7f3479800640 1 -- 172.21.15.32:0/1285024596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34740b3000 msgr2=0x7f347409d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:00.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.643+0000 7f3479800640 1 --2- 172.21.15.32:0/1285024596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34740b3000 0x7f347409d7b0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f345c00c450 tx=0x7f345c00c910 comp rx=0 tx=0).stop 2024-04-17T10:04:00.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.643+0000 7f3479800640 1 -- 172.21.15.32:0/1285024596 shutdown_connections 2024-04-17T10:04:00.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.643+0000 7f3479800640 1 --2- 172.21.15.32:0/1285024596 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3450068380 0x7f345006a840 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:00.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.643+0000 7f3479800640 1 --2- 172.21.15.32:0/1285024596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f347415b560 0x7f3474159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:00.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.643+0000 7f3479800640 1 --2- 172.21.15.32:0/1285024596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34740b3000 0x7f347409d7b0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:00.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.643+0000 7f3479800640 1 --2- 172.21.15.32:0/1285024596 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34740735d0 0x7f34740c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:00.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.643+0000 7f3479800640 1 -- 172.21.15.32:0/1285024596 >> 172.21.15.32:0/1285024596 conn(0x7f347406bc00 msgr2=0x7f3474158b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:00.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.643+0000 7f3479800640 1 -- 172.21.15.32:0/1285024596 shutdown_connections 2024-04-17T10:04:00.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.643+0000 7f3479800640 1 -- 172.21.15.32:0/1285024596 wait complete. 2024-04-17T10:04:00.661 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:04:00.661 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:04:00.671 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:04:00.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.851+0000 7fc322000640 1 Processor -- start 2024-04-17T10:04:00.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.851+0000 7fc322000640 1 -- start start 2024-04-17T10:04:00.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.851+0000 7fc322000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc31c144e90 0x7fc31c0b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:00.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.851+0000 7fc322000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc31c0b4de0 0x7fc31c15baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:00.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.851+0000 7fc322000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc31c072bb0 con 0x7fc31c15bfe0 2024-04-17T10:04:00.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.851+0000 7fc322000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc31c15c430 con 0x7fc31c144e90 2024-04-17T10:04:00.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.851+0000 7fc322000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc31c145270 con 0x7fc31c0b4de0 2024-04-17T10:04:00.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.851+0000 7fc31be00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc31c15bfe0 0x7fc31c06ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35926/0 (socket says 172.21.15.32:35926) 2024-04-17T10:04:00.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.851+0000 7fc31be00640 1 -- 172.21.15.32:0/3403812127 learned_addr learned my addr 172.21.15.32:0/3403812127 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:00.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.855+0000 7fc313e00640 1 --2- 172.21.15.32:0/3403812127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc31c0b4de0 0x7fc31c15baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:00.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.855+0000 7fc31b400640 1 --2- 172.21.15.32:0/3403812127 >> v2:172.21.15.144:3300/0 conn(0x7fc31c144e90 0x7fc31c0b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:00.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.855+0000 7fc31aa00640 1 -- 172.21.15.32:0/3403812127 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1095951600 0 0) 0x7fc31c072bb0 con 0x7fc31c15bfe0 2024-04-17T10:04:00.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.855+0000 7fc31aa00640 1 -- 172.21.15.32:0/3403812127 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc300003530 con 0x7fc31c15bfe0 2024-04-17T10:04:00.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.855+0000 7fc313e00640 1 -- 172.21.15.32:0/3403812127 >> v2:172.21.15.144:3300/0 conn(0x7fc31c144e90 msgr2=0x7fc31c0b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:00.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.855+0000 7fc313e00640 1 --2- 172.21.15.32:0/3403812127 >> v2:172.21.15.144:3300/0 conn(0x7fc31c144e90 0x7fc31c0b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:00.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.855+0000 7fc313e00640 1 -- 172.21.15.32:0/3403812127 >> v1:172.21.15.32:6789/0 conn(0x7fc31c15bfe0 legacy=0x7fc31c06ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:00.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.855+0000 7fc313e00640 1 -- 172.21.15.32:0/3403812127 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc31c0b3dc0 con 0x7fc31c0b4de0 2024-04-17T10:04:00.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.855+0000 7fc31b400640 1 --2- 172.21.15.32:0/3403812127 >> v2:172.21.15.144:3300/0 conn(0x7fc31c144e90 0x7fc31c0b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:04:00.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.855+0000 7fc313e00640 1 --2- 172.21.15.32:0/3403812127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc31c0b4de0 0x7fc31c15baa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc3040097c0 tx=0x7fc30401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c18715811110914a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.855+0000 7fc31aa00640 1 -- 172.21.15.32:0/3403812127 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3040a7070 con 0x7fc31c0b4de0 2024-04-17T10:04:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.855+0000 7fc31aa00640 1 -- 172.21.15.32:0/3403812127 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3040b0070 con 0x7fc31c0b4de0 2024-04-17T10:04:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.855+0000 7fc31aa00640 1 -- 172.21.15.32:0/3403812127 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3040ab870 con 0x7fc31c0b4de0 2024-04-17T10:04:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.859+0000 7fc322000640 1 -- 172.21.15.32:0/3403812127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc31c0b4de0 msgr2=0x7fc31c15baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.859+0000 7fc322000640 1 --2- 172.21.15.32:0/3403812127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc31c0b4de0 0x7fc31c15baa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc3040097c0 tx=0x7fc30401c470 comp rx=0 tx=0).stop 2024-04-17T10:04:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.859+0000 7fc322000640 1 -- 172.21.15.32:0/3403812127 shutdown_connections 2024-04-17T10:04:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.859+0000 7fc322000640 1 --2- 172.21.15.32:0/3403812127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc31c0b4de0 0x7fc31c15baa0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.859+0000 7fc322000640 1 --2- 172.21.15.32:0/3403812127 >> v2:172.21.15.144:3300/0 conn(0x7fc31c144e90 0x7fc31c0b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.859+0000 7fc322000640 1 -- 172.21.15.32:0/3403812127 >> 172.21.15.32:0/3403812127 conn(0x7fc31c06ba30 msgr2=0x7fc31c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.859+0000 7fc322000640 1 -- 172.21.15.32:0/3403812127 shutdown_connections 2024-04-17T10:04:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.859+0000 7fc322000640 1 -- 172.21.15.32:0/3403812127 wait complete. 2024-04-17T10:04:00.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.859+0000 7fc322000640 1 Processor -- start 2024-04-17T10:04:00.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.859+0000 7fc322000640 1 -- start start 2024-04-17T10:04:00.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.859+0000 7fc322000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc31c0b4de0 0x7fc31c15a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:00.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.859+0000 7fc322000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc31c144e90 0x7fc31c1f7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:00.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.859+0000 7fc322000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc31c15bfe0 0x7fc31c15a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:00.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.859+0000 7fc322000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c072670 con 0x7fc31c0b4de0 2024-04-17T10:04:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.859+0000 7fc322000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c06f880 con 0x7fc31c144e90 2024-04-17T10:04:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.859+0000 7fc322000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc31c0727f0 con 0x7fc31c15bfe0 2024-04-17T10:04:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.859+0000 7fc31b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc31c0b4de0 0x7fc31c15a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.859+0000 7fc31b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc31c0b4de0 0x7fc31c15a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40934/0 (socket says 172.21.15.32:40934) 2024-04-17T10:04:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.859+0000 7fc31b400640 1 -- 172.21.15.32:0/1768476851 learned_addr learned my addr 172.21.15.32:0/1768476851 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.859+0000 7fc31be00640 1 --2- 172.21.15.32:0/1768476851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc31c15bfe0 0x7fc31c15a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.859+0000 7fc313e00640 1 --2- 172.21.15.32:0/1768476851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc31c144e90 0x7fc31c1f7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.859+0000 7fc31b400640 1 -- 172.21.15.32:0/1768476851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc31c15bfe0 msgr2=0x7fc31c15a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.863+0000 7fc31b400640 1 --2- 172.21.15.32:0/1768476851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc31c15bfe0 0x7fc31c15a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.863+0000 7fc31b400640 1 -- 172.21.15.32:0/1768476851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc31c144e90 msgr2=0x7fc31c1f7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.863+0000 7fc31b400640 1 --2- 172.21.15.32:0/1768476851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc31c144e90 0x7fc31c1f7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.863+0000 7fc31b400640 1 -- 172.21.15.32:0/1768476851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc31c0b1b30 con 0x7fc31c0b4de0 2024-04-17T10:04:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.863+0000 7fc313e00640 1 --2- 172.21.15.32:0/1768476851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc31c144e90 0x7fc31c1f7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.863+0000 7fc31be00640 1 --2- 172.21.15.32:0/1768476851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc31c15bfe0 0x7fc31c15a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.863+0000 7fc31b400640 1 --2- 172.21.15.32:0/1768476851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc31c0b4de0 0x7fc31c15a130 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00e9b0 tx=0x7fc30c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:00.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.863+0000 7fc313400640 1 -- 172.21.15.32:0/1768476851 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c017070 con 0x7fc31c0b4de0 2024-04-17T10:04:00.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.863+0000 7fc313400640 1 -- 172.21.15.32:0/1768476851 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc30c00c9a0 con 0x7fc31c0b4de0 2024-04-17T10:04:00.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.863+0000 7fc313400640 1 -- 172.21.15.32:0/1768476851 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c00cd70 con 0x7fc31c0b4de0 2024-04-17T10:04:00.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.863+0000 7fc322000640 1 -- 172.21.15.32:0/1768476851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc31c0ad900 con 0x7fc31c0b4de0 2024-04-17T10:04:00.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.863+0000 7fc322000640 1 -- 172.21.15.32:0/1768476851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc31c0b20b0 con 0x7fc31c0b4de0 2024-04-17T10:04:00.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.863+0000 7fc322000640 1 -- 172.21.15.32:0/1768476851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc31c068a90 con 0x7fc31c0b4de0 2024-04-17T10:04:00.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.863+0000 7fc313400640 1 -- 172.21.15.32:0/1768476851 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc30c01a070 con 0x7fc31c0b4de0 2024-04-17T10:04:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.863+0000 7fc313400640 1 --2- 172.21.15.32:0/1768476851 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc2f8068380 0x7fc2f806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.863+0000 7fc313400640 1 -- 172.21.15.32:0/1768476851 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7fc30c086890 con 0x7fc31c0b4de0 2024-04-17T10:04:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.863+0000 7fc313e00640 1 --2- 172.21.15.32:0/1768476851 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc2f8068380 0x7fc2f806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:00.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.867+0000 7fc313e00640 1 --2- 172.21.15.32:0/1768476851 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc2f8068380 0x7fc2f806a840 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc3040097c0 tx=0x7fc304008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:00.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:00.867+0000 7fc313400640 1 -- 172.21.15.32:0/1768476851 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fc30c056e00 con 0x7fc31c0b4de0 2024-04-17T10:04:01.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.031+0000 7fc322000640 1 -- 172.21.15.32:0/1768476851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc31c15a670 con 0x7fc31c0b4de0 2024-04-17T10:04:01.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.035+0000 7fc313400640 1 -- 172.21.15.32:0/1768476851 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+16736 (secure 0 0 0) 0x7fc30c056700 con 0x7fc31c0b4de0 2024-04-17T10:04:01.033 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:01.033 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":111,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:03:56.500014+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:03:36.021322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-17T10:03:36.581134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.5","mappings":[{"from":0,"to":5}]},{"pgid":"17.25","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:04:01.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.035+0000 7fc322000640 1 -- 172.21.15.32:0/1768476851 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc2f8068380 msgr2=0x7fc2f806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:01.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.035+0000 7fc322000640 1 --2- 172.21.15.32:0/1768476851 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc2f8068380 0x7fc2f806a840 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc3040097c0 tx=0x7fc304008040 comp rx=0 tx=0).stop 2024-04-17T10:04:01.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.035+0000 7fc322000640 1 -- 172.21.15.32:0/1768476851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc31c0b4de0 msgr2=0x7fc31c15a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:01.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.035+0000 7fc322000640 1 --2- 172.21.15.32:0/1768476851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc31c0b4de0 0x7fc31c15a130 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00e9b0 tx=0x7fc30c00ee70 comp rx=0 tx=0).stop 2024-04-17T10:04:01.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.035+0000 7fc322000640 1 -- 172.21.15.32:0/1768476851 shutdown_connections 2024-04-17T10:04:01.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.035+0000 7fc322000640 1 --2- 172.21.15.32:0/1768476851 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc2f8068380 0x7fc2f806a840 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.035+0000 7fc322000640 1 --2- 172.21.15.32:0/1768476851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc31c15bfe0 0x7fc31c15a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.035+0000 7fc322000640 1 --2- 172.21.15.32:0/1768476851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc31c144e90 0x7fc31c1f7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.035+0000 7fc322000640 1 --2- 172.21.15.32:0/1768476851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc31c0b4de0 0x7fc31c15a130 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.035+0000 7fc322000640 1 -- 172.21.15.32:0/1768476851 >> 172.21.15.32:0/1768476851 conn(0x7fc31c06ba30 msgr2=0x7fc31c06beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:01.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.035+0000 7fc322000640 1 -- 172.21.15.32:0/1768476851 shutdown_connections 2024-04-17T10:04:01.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.035+0000 7fc322000640 1 -- 172.21.15.32:0/1768476851 wait complete. 2024-04-17T10:04:01.054 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:03:36.021322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:03:36.581134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:04:01.055 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:04:01.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.199+0000 7f4d04c00640 1 Processor -- start 2024-04-17T10:04:01.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.199+0000 7f4d04c00640 1 -- start start 2024-04-17T10:04:01.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4d04c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4d0006ba30 0x7f4d0006be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:01.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4d04c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d0006c350 con 0x7f4d00071e60 2024-04-17T10:04:01.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4d04c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d000d0130 con 0x7f4d000cf370 2024-04-17T10:04:01.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4d04c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f4d000cfef0 con 0x7f4d0006ba30 2024-04-17T10:04:01.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfe800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4d00071e60 0x7f4d00072240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35934/0 (socket says 172.21.15.32:35934) 2024-04-17T10:04:01.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfe800640 1 -- 172.21.15.32:0/2463022296 learned_addr learned my addr 172.21.15.32:0/2463022296 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:01.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfde00640 1 --2- 172.21.15.32:0/2463022296 >> v2:172.21.15.144:3300/0 conn(0x7f4d0006ba30 0x7f4d0006be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:01.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfca00640 1 -- 172.21.15.32:0/2463022296 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2417157113 0 0) 0x7f4d0006c350 con 0x7f4d00071e60 2024-04-17T10:04:01.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfca00640 1 -- 172.21.15.32:0/2463022296 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ce4003530 con 0x7f4d00071e60 2024-04-17T10:04:01.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfca00640 1 -- 172.21.15.32:0/2463022296 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3525386374 0 0) 0x7f4ce4003530 con 0x7f4d00071e60 2024-04-17T10:04:01.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfca00640 1 -- 172.21.15.32:0/2463022296 >> v2:172.21.15.144:3300/0 conn(0x7f4d0006ba30 msgr2=0x7f4d0006be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:01.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfca00640 1 --2- 172.21.15.32:0/2463022296 >> v2:172.21.15.144:3300/0 conn(0x7f4d0006ba30 0x7f4d0006be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfca00640 1 -- 172.21.15.32:0/2463022296 >> v1:172.21.15.144:6789/0 conn(0x7f4d000cf370 legacy=0x7f4d000cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:01.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfca00640 1 -- 172.21.15.32:0/2463022296 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0015d090 con 0x7f4d00071e60 2024-04-17T10:04:01.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfca00640 1 -- 172.21.15.32:0/2463022296 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f4cf8001a00 con 0x7f4d00071e60 2024-04-17T10:04:01.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfde00640 1 --2- 172.21.15.32:0/2463022296 >> v2:172.21.15.144:3300/0 conn(0x7f4d0006ba30 0x7f4d0006be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:04:01.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfca00640 1 -- 172.21.15.32:0/2463022296 >> v1:172.21.15.32:6789/0 conn(0x7f4d00071e60 legacy=0x7f4d00072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:01.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfca00640 1 --2- 172.21.15.32:0/2463022296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ce40039c0 0x7f4ce4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:01.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfca00640 1 --2- 172.21.15.32:0/2463022296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ce40043c0 0x7f4ce4004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:01.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfca00640 1 --2- 172.21.15.32:0/2463022296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ce4005130 0x7f4ce400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:01.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfca00640 1 -- 172.21.15.32:0/2463022296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4d000cfef0 con 0x7f4ce40039c0 2024-04-17T10:04:01.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfca00640 1 -- 172.21.15.32:0/2463022296 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce400c070 con 0x7f4ce4005130 2024-04-17T10:04:01.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfca00640 1 -- 172.21.15.32:0/2463022296 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4ce400c270 con 0x7f4ce40043c0 2024-04-17T10:04:01.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfde00640 1 --2- 172.21.15.32:0/2463022296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ce40039c0 0x7f4ce4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:01.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfe800640 1 --2- 172.21.15.32:0/2463022296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ce4005130 0x7f4ce400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:01.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfd400640 1 --2- 172.21.15.32:0/2463022296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ce40043c0 0x7f4ce4004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:01.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfde00640 1 -- 172.21.15.32:0/2463022296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ce40043c0 msgr2=0x7f4ce4004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:01.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfde00640 1 --2- 172.21.15.32:0/2463022296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ce40043c0 0x7f4ce4004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfde00640 1 -- 172.21.15.32:0/2463022296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ce4005130 msgr2=0x7f4ce400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:01.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfde00640 1 --2- 172.21.15.32:0/2463022296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ce4005130 0x7f4ce400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfde00640 1 -- 172.21.15.32:0/2463022296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce4003530 con 0x7f4ce40039c0 2024-04-17T10:04:01.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.203+0000 7f4cfe800640 1 -- 172.21.15.32:0/2463022296 reap_dead start 2024-04-17T10:04:01.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.207+0000 7f4cfde00640 1 --2- 172.21.15.32:0/2463022296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ce40039c0 0x7f4ce4003da0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009cb0 tx=0x7f4cf001cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432b86613059c689 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.207+0000 7f4cfca00640 1 -- 172.21.15.32:0/2463022296 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf00b2070 con 0x7f4ce40039c0 2024-04-17T10:04:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.207+0000 7f4cfca00640 1 -- 172.21.15.32:0/2463022296 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cf00a5790 con 0x7f4ce40039c0 2024-04-17T10:04:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.207+0000 7f4cfca00640 1 -- 172.21.15.32:0/2463022296 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0009880 con 0x7f4ce40039c0 2024-04-17T10:04:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.207+0000 7f4d04c00640 1 -- 172.21.15.32:0/2463022296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ce40039c0 msgr2=0x7f4ce4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.207+0000 7f4d04c00640 1 --2- 172.21.15.32:0/2463022296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ce40039c0 0x7f4ce4003da0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009cb0 tx=0x7f4cf001cbe0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.207+0000 7f4d04c00640 1 -- 172.21.15.32:0/2463022296 shutdown_connections 2024-04-17T10:04:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.207+0000 7f4d04c00640 1 --2- 172.21.15.32:0/2463022296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ce40039c0 0x7f4ce4003da0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.207+0000 7f4d04c00640 1 -- 172.21.15.32:0/2463022296 >> 172.21.15.32:0/2463022296 conn(0x7f4d000b2120 msgr2=0x7f4d000b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.207+0000 7f4d04c00640 1 -- 172.21.15.32:0/2463022296 shutdown_connections 2024-04-17T10:04:01.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.207+0000 7f4d04c00640 1 -- 172.21.15.32:0/2463022296 wait complete. 2024-04-17T10:04:01.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.207+0000 7f4d04c00640 1 Processor -- start 2024-04-17T10:04:01.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.207+0000 7f4d04c00640 1 -- start start 2024-04-17T10:04:01.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4d04c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ce40039c0 0x7f4d001f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:01.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4d04c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d001f7c30 0x7f4d0006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:01.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4d04c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d0006e8b0 0x7f4d001f8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:01.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4d04c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4d000cfd10 con 0x7f4d001f7c30 2024-04-17T10:04:01.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4d04c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4d000cfb90 con 0x7f4d0006e8b0 2024-04-17T10:04:01.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4d04c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4d0015db70 con 0x7f4ce40039c0 2024-04-17T10:04:01.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4cfd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d001f7c30 0x7f4d0006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:01.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4cfd400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d001f7c30 0x7f4d0006e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40944/0 (socket says 172.21.15.32:40944) 2024-04-17T10:04:01.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4cfd400640 1 -- 172.21.15.32:0/447859155 learned_addr learned my addr 172.21.15.32:0/447859155 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:01.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4cfe800640 1 --2- 172.21.15.32:0/447859155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d0006e8b0 0x7f4d001f8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:01.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4cfde00640 1 --2- 172.21.15.32:0/447859155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ce40039c0 0x7f4d001f76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:01.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4cfd400640 1 -- 172.21.15.32:0/447859155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ce40039c0 msgr2=0x7f4d001f76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:01.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4cfd400640 1 --2- 172.21.15.32:0/447859155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ce40039c0 0x7f4d001f76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4cfd400640 1 -- 172.21.15.32:0/447859155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d0006e8b0 msgr2=0x7f4d001f8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:01.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4cfd400640 1 --2- 172.21.15.32:0/447859155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d0006e8b0 0x7f4d001f8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4cfd400640 1 -- 172.21.15.32:0/447859155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d001f90a0 con 0x7f4d001f7c30 2024-04-17T10:04:01.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4cfd400640 1 --2- 172.21.15.32:0/447859155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d001f7c30 0x7f4d0006e370 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f4cec00f800 tx=0x7f4cec00fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:01.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4cf6000640 1 -- 172.21.15.32:0/447859155 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec01a070 con 0x7f4d001f7c30 2024-04-17T10:04:01.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4cf6000640 1 -- 172.21.15.32:0/447859155 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cec005320 con 0x7f4d001f7c30 2024-04-17T10:04:01.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4cf6000640 1 -- 172.21.15.32:0/447859155 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec006a20 con 0x7f4d001f7c30 2024-04-17T10:04:01.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4d04c00640 1 -- 172.21.15.32:0/447859155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d001fb7c0 con 0x7f4d001f7c30 2024-04-17T10:04:01.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4d04c00640 1 -- 172.21.15.32:0/447859155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d001fbd80 con 0x7f4d001f7c30 2024-04-17T10:04:01.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4d04c00640 1 -- 172.21.15.32:0/447859155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d00068a90 con 0x7f4d001f7c30 2024-04-17T10:04:01.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.211+0000 7f4cf6000640 1 -- 172.21.15.32:0/447859155 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4cec01d070 con 0x7f4d001f7c30 2024-04-17T10:04:01.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.215+0000 7f4cf6000640 1 --2- 172.21.15.32:0/447859155 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4ce0068380 0x7f4ce006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:01.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.215+0000 7f4cf6000640 1 -- 172.21.15.32:0/447859155 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7f4cec086040 con 0x7f4d001f7c30 2024-04-17T10:04:01.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.215+0000 7f4cfde00640 1 --2- 172.21.15.32:0/447859155 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4ce0068380 0x7f4ce006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:01.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.215+0000 7f4cfde00640 1 --2- 172.21.15.32:0/447859155 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4ce0068380 0x7f4ce006a840 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00af000 tx=0x7f4cf0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:01.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.219+0000 7f4cf6000640 1 -- 172.21.15.32:0/447859155 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f4cec056630 con 0x7f4d001f7c30 2024-04-17T10:04:01.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.383+0000 7f4d04c00640 1 -- 172.21.15.32:0/447859155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4d0006ec90 con 0x7f4d001f7c30 2024-04-17T10:04:01.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.383+0000 7f4cf6000640 1 -- 172.21.15.32:0/447859155 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) ==== 93+0+10 (secure 0 0 0) 0x7f4cec055f30 con 0x7f4d001f7c30 2024-04-17T10:04:01.383 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T10:04:01.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.387+0000 7f4d04c00640 1 -- 172.21.15.32:0/447859155 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4ce0068380 msgr2=0x7f4ce006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:01.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.387+0000 7f4d04c00640 1 --2- 172.21.15.32:0/447859155 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4ce0068380 0x7f4ce006a840 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00af000 tx=0x7f4cf0008040 comp rx=0 tx=0).stop 2024-04-17T10:04:01.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.387+0000 7f4d04c00640 1 -- 172.21.15.32:0/447859155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d001f7c30 msgr2=0x7f4d0006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:01.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.387+0000 7f4d04c00640 1 --2- 172.21.15.32:0/447859155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d001f7c30 0x7f4d0006e370 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f4cec00f800 tx=0x7f4cec00fcc0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.387+0000 7f4d04c00640 1 -- 172.21.15.32:0/447859155 shutdown_connections 2024-04-17T10:04:01.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.387+0000 7f4d04c00640 1 --2- 172.21.15.32:0/447859155 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f4ce0068380 0x7f4ce006a840 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.387+0000 7f4d04c00640 1 --2- 172.21.15.32:0/447859155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d0006e8b0 0x7f4d001f8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.387+0000 7f4d04c00640 1 --2- 172.21.15.32:0/447859155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d001f7c30 0x7f4d0006e370 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.387+0000 7f4d04c00640 1 --2- 172.21.15.32:0/447859155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ce40039c0 0x7f4d001f76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.387+0000 7f4d04c00640 1 -- 172.21.15.32:0/447859155 >> 172.21.15.32:0/447859155 conn(0x7f4d000b2120 msgr2=0x7f4d00140ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:01.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.387+0000 7f4d04c00640 1 -- 172.21.15.32:0/447859155 shutdown_connections 2024-04-17T10:04:01.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.387+0000 7f4d04c00640 1 -- 172.21.15.32:0/447859155 wait complete. 2024-04-17T10:04:01.404 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:04:01.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.551+0000 7ff61c400640 1 Processor -- start 2024-04-17T10:04:01.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.551+0000 7ff61c400640 1 -- start start 2024-04-17T10:04:01.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.551+0000 7ff61c400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6140cf370 0x7ff6140cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:01.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.551+0000 7ff61c400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6140d0130 con 0x7ff61406ba30 2024-04-17T10:04:01.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.551+0000 7ff61c400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff61415d090 con 0x7ff614071e60 2024-04-17T10:04:01.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.551+0000 7ff61c400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7ff61415d370 con 0x7ff6140cf370 2024-04-17T10:04:01.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.551+0000 7ff619a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff61406ba30 0x7ff61406be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35948/0 (socket says 172.21.15.32:35948) 2024-04-17T10:04:01.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.551+0000 7ff619a00640 1 -- 172.21.15.32:0/4268579889 learned_addr learned my addr 172.21.15.32:0/4268579889 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:01.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.551+0000 7ff619000640 1 --2- 172.21.15.32:0/4268579889 >> v2:172.21.15.32:3300/0 conn(0x7ff6140cf370 0x7ff6140cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:01.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.551+0000 7ff613e00640 1 -- 172.21.15.32:0/4268579889 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2933713278 0 0) 0x7ff6140d0130 con 0x7ff61406ba30 2024-04-17T10:04:01.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.551+0000 7ff613e00640 1 -- 172.21.15.32:0/4268579889 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5fc003530 con 0x7ff61406ba30 2024-04-17T10:04:01.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.551+0000 7ff619000640 1 -- 172.21.15.32:0/4268579889 >> v1:172.21.15.144:6789/0 conn(0x7ff614071e60 legacy=0x7ff614072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:01.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.551+0000 7ff619000640 1 -- 172.21.15.32:0/4268579889 >> v1:172.21.15.32:6789/0 conn(0x7ff61406ba30 legacy=0x7ff61406be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:01.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.551+0000 7ff619000640 1 -- 172.21.15.32:0/4268579889 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61415d650 con 0x7ff6140cf370 2024-04-17T10:04:01.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.555+0000 7ff619000640 1 --2- 172.21.15.32:0/4268579889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6140cf370 0x7ff6140cf7e0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7ff6000097c0 tx=0x7ff60001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64085e1432911ca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:01.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.555+0000 7ff613e00640 1 -- 172.21.15.32:0/4268579889 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6000ab660 con 0x7ff6140cf370 2024-04-17T10:04:01.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.555+0000 7ff613e00640 1 -- 172.21.15.32:0/4268579889 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6000b0070 con 0x7ff6140cf370 2024-04-17T10:04:01.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.555+0000 7ff613e00640 1 -- 172.21.15.32:0/4268579889 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6000aba20 con 0x7ff6140cf370 2024-04-17T10:04:01.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.555+0000 7ff61c400640 1 -- 172.21.15.32:0/4268579889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6140cf370 msgr2=0x7ff6140cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:01.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.555+0000 7ff61c400640 1 --2- 172.21.15.32:0/4268579889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6140cf370 0x7ff6140cf7e0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7ff6000097c0 tx=0x7ff60001c460 comp rx=0 tx=0).stop 2024-04-17T10:04:01.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.555+0000 7ff61c400640 1 -- 172.21.15.32:0/4268579889 shutdown_connections 2024-04-17T10:04:01.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.555+0000 7ff61c400640 1 --2- 172.21.15.32:0/4268579889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6140cf370 0x7ff6140cf7e0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.555+0000 7ff61c400640 1 -- 172.21.15.32:0/4268579889 >> 172.21.15.32:0/4268579889 conn(0x7ff6140b2120 msgr2=0x7ff6140b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:01.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.555+0000 7ff61c400640 1 -- 172.21.15.32:0/4268579889 shutdown_connections 2024-04-17T10:04:01.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.555+0000 7ff61c400640 1 -- 172.21.15.32:0/4268579889 wait complete. 2024-04-17T10:04:01.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff61c400640 1 Processor -- start 2024-04-17T10:04:01.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff61c400640 1 -- start start 2024-04-17T10:04:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff61c400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61406ba30 0x7ff61413db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff61c400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff614071e60 0x7ff6140b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff61c400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6140cf370 0x7ff614157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff61c400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff61415dcf0 con 0x7ff61406ba30 2024-04-17T10:04:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff61c400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff61415db70 con 0x7ff614071e60 2024-04-17T10:04:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff61c400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff614146230 con 0x7ff6140cf370 2024-04-17T10:04:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff619a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61406ba30 0x7ff61413db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff619a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61406ba30 0x7ff61413db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40976/0 (socket says 172.21.15.32:40976) 2024-04-17T10:04:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff619a00640 1 -- 172.21.15.32:0/544170252 learned_addr learned my addr 172.21.15.32:0/544170252 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff619000640 1 --2- 172.21.15.32:0/544170252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff614071e60 0x7ff6140b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff61a400640 1 --2- 172.21.15.32:0/544170252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6140cf370 0x7ff614157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff619a00640 1 -- 172.21.15.32:0/544170252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6140cf370 msgr2=0x7ff614157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff619a00640 1 --2- 172.21.15.32:0/544170252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6140cf370 0x7ff614157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff619a00640 1 -- 172.21.15.32:0/544170252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff614071e60 msgr2=0x7ff6140b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff619a00640 1 --2- 172.21.15.32:0/544170252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff614071e60 0x7ff6140b1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff619a00640 1 -- 172.21.15.32:0/544170252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6140b01a0 con 0x7ff61406ba30 2024-04-17T10:04:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff619000640 1 --2- 172.21.15.32:0/544170252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff614071e60 0x7ff6140b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff619a00640 1 --2- 172.21.15.32:0/544170252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61406ba30 0x7ff61413db60 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7ff60800c480 tx=0x7ff60800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff611600640 1 -- 172.21.15.32:0/544170252 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608017070 con 0x7ff61406ba30 2024-04-17T10:04:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff611600640 1 -- 172.21.15.32:0/544170252 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff608005cf0 con 0x7ff61406ba30 2024-04-17T10:04:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff611600640 1 -- 172.21.15.32:0/544170252 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608007290 con 0x7ff61406ba30 2024-04-17T10:04:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff61c400640 1 -- 172.21.15.32:0/544170252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6140b0480 con 0x7ff61406ba30 2024-04-17T10:04:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff61c400640 1 -- 172.21.15.32:0/544170252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6140b0a40 con 0x7ff61406ba30 2024-04-17T10:04:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.559+0000 7ff61c400640 1 -- 172.21.15.32:0/544170252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff614068a90 con 0x7ff61406ba30 2024-04-17T10:04:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.563+0000 7ff611600640 1 -- 172.21.15.32:0/544170252 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff60801a070 con 0x7ff61406ba30 2024-04-17T10:04:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.563+0000 7ff611600640 1 --2- 172.21.15.32:0/544170252 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff5f4068380 0x7ff5f406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.563+0000 7ff611600640 1 -- 172.21.15.32:0/544170252 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7ff608085e70 con 0x7ff61406ba30 2024-04-17T10:04:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.563+0000 7ff619000640 1 --2- 172.21.15.32:0/544170252 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff5f4068380 0x7ff5f406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.563+0000 7ff619000640 1 --2- 172.21.15.32:0/544170252 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff5f4068380 0x7ff5f406a840 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff6000af000 tx=0x7ff6000a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:01.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.567+0000 7ff611600640 1 -- 172.21.15.32:0/544170252 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff6080563e0 con 0x7ff61406ba30 2024-04-17T10:04:01.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.731+0000 7ff61c400640 1 -- 172.21.15.32:0/544170252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff6140afef0 con 0x7ff61406ba30 2024-04-17T10:04:01.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.731+0000 7ff611600640 1 -- 172.21.15.32:0/544170252 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v111) ==== 104+0+11 (secure 0 0 0) 0x7ff608055ce0 con 0x7ff61406ba30 2024-04-17T10:04:01.728 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T10:04:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.731+0000 7ff61c400640 1 -- 172.21.15.32:0/544170252 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff5f4068380 msgr2=0x7ff5f406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.731+0000 7ff61c400640 1 --2- 172.21.15.32:0/544170252 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff5f4068380 0x7ff5f406a840 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff6000af000 tx=0x7ff6000a7000 comp rx=0 tx=0).stop 2024-04-17T10:04:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.731+0000 7ff61c400640 1 -- 172.21.15.32:0/544170252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61406ba30 msgr2=0x7ff61413db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.731+0000 7ff61c400640 1 --2- 172.21.15.32:0/544170252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61406ba30 0x7ff61413db60 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7ff60800c480 tx=0x7ff60800c940 comp rx=0 tx=0).stop 2024-04-17T10:04:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.731+0000 7ff61c400640 1 -- 172.21.15.32:0/544170252 shutdown_connections 2024-04-17T10:04:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.731+0000 7ff61c400640 1 --2- 172.21.15.32:0/544170252 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff5f4068380 0x7ff5f406a840 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.731+0000 7ff61c400640 1 --2- 172.21.15.32:0/544170252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6140cf370 0x7ff614157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.731+0000 7ff61c400640 1 --2- 172.21.15.32:0/544170252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff614071e60 0x7ff6140b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.731+0000 7ff61c400640 1 --2- 172.21.15.32:0/544170252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61406ba30 0x7ff61413db60 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.731+0000 7ff61c400640 1 -- 172.21.15.32:0/544170252 >> 172.21.15.32:0/544170252 conn(0x7ff6140b2120 msgr2=0x7ff61415b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:01.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.731+0000 7ff61c400640 1 -- 172.21.15.32:0/544170252 shutdown_connections 2024-04-17T10:04:01.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.731+0000 7ff61c400640 1 -- 172.21.15.32:0/544170252 wait complete. 2024-04-17T10:04:01.749 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:04:01.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.891+0000 7f0ee8600640 1 Processor -- start 2024-04-17T10:04:01.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.891+0000 7f0ee8600640 1 -- start start 2024-04-17T10:04:01.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.895+0000 7f0ee8600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ee015b7d0 0x7f0ee006e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:01.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.895+0000 7f0ee8600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ee006c140 0x7f0ee00ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:01.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.895+0000 7f0ee8600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ee00cc940 con 0x7f0ee006f7c0 2024-04-17T10:04:01.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.895+0000 7f0ee8600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0ee00ccc20 con 0x7f0ee015b7d0 2024-04-17T10:04:01.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.895+0000 7f0ee8600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0ee006e0b0 con 0x7f0ee006c140 2024-04-17T10:04:01.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.895+0000 7f0ee5c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ee015b7d0 0x7f0ee006e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:01.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.895+0000 7f0ee5200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0ee006f7c0 0x7f0ee006ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35954/0 (socket says 172.21.15.32:35954) 2024-04-17T10:04:01.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.895+0000 7f0ee5200640 1 -- 172.21.15.32:0/386316399 learned_addr learned my addr 172.21.15.32:0/386316399 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:01.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.895+0000 7f0ee6600640 1 --2- 172.21.15.32:0/386316399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ee006c140 0x7f0ee00ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:01.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.895+0000 7f0ee5c00640 1 -- 172.21.15.32:0/386316399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ee006c140 msgr2=0x7f0ee00ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:01.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.895+0000 7f0ee5c00640 1 --2- 172.21.15.32:0/386316399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ee006c140 0x7f0ee00ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.895+0000 7f0ee5c00640 1 -- 172.21.15.32:0/386316399 >> v1:172.21.15.32:6789/0 conn(0x7f0ee006f7c0 legacy=0x7f0ee006ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:01.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.895+0000 7f0ee5c00640 1 -- 172.21.15.32:0/386316399 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee006e390 con 0x7f0ee015b7d0 2024-04-17T10:04:01.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.895+0000 7f0edfe00640 1 -- 172.21.15.32:0/386316399 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3092610116 0 0) 0x7f0ee00cc940 con 0x7f0ee006f7c0 2024-04-17T10:04:01.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.899+0000 7f0ee5c00640 1 --2- 172.21.15.32:0/386316399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ee015b7d0 0x7f0ee006e8b0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40097c0 tx=0x7f0ed401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5807e5dafce8e333 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:01.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.899+0000 7f0edfe00640 1 -- 172.21.15.32:0/386316399 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed40ab430 con 0x7f0ee015b7d0 2024-04-17T10:04:01.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.899+0000 7f0edfe00640 1 -- 172.21.15.32:0/386316399 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ed40b0070 con 0x7f0ee015b7d0 2024-04-17T10:04:01.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.899+0000 7f0edfe00640 1 -- 172.21.15.32:0/386316399 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed40ab7f0 con 0x7f0ee015b7d0 2024-04-17T10:04:01.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.899+0000 7f0ee8600640 1 -- 172.21.15.32:0/386316399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ee015b7d0 msgr2=0x7f0ee006e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:01.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.899+0000 7f0ee8600640 1 --2- 172.21.15.32:0/386316399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ee015b7d0 0x7f0ee006e8b0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40097c0 tx=0x7f0ed401c410 comp rx=0 tx=0).stop 2024-04-17T10:04:01.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.899+0000 7f0ee8600640 1 -- 172.21.15.32:0/386316399 shutdown_connections 2024-04-17T10:04:01.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.899+0000 7f0ee8600640 1 --2- 172.21.15.32:0/386316399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ee006c140 0x7f0ee00ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.899+0000 7f0ee8600640 1 --2- 172.21.15.32:0/386316399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ee015b7d0 0x7f0ee006e8b0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.899+0000 7f0ee8600640 1 -- 172.21.15.32:0/386316399 >> 172.21.15.32:0/386316399 conn(0x7f0ee00b2120 msgr2=0x7f0ee00b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:01.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.899+0000 7f0ee8600640 1 -- 172.21.15.32:0/386316399 shutdown_connections 2024-04-17T10:04:01.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.899+0000 7f0ee8600640 1 -- 172.21.15.32:0/386316399 wait complete. 2024-04-17T10:04:01.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.899+0000 7f0ee8600640 1 Processor -- start 2024-04-17T10:04:01.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.899+0000 7f0ee8600640 1 -- start start 2024-04-17T10:04:01.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.899+0000 7f0ee8600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ee006c140 0x7f0ee0156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:01.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.899+0000 7f0ee8600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ee006f7c0 0x7f0ee01550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:01.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.899+0000 7f0ee8600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ee015b7d0 0x7f0ee0149b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:01.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.899+0000 7f0ee8600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee0168630 con 0x7f0ee006c140 2024-04-17T10:04:01.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.899+0000 7f0ee8600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee01684b0 con 0x7f0ee006f7c0 2024-04-17T10:04:01.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.899+0000 7f0ee8600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0ee01687b0 con 0x7f0ee015b7d0 2024-04-17T10:04:01.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.903+0000 7f0ee5c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ee006c140 0x7f0ee0156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:01.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.903+0000 7f0ee5c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ee006c140 0x7f0ee0156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40994/0 (socket says 172.21.15.32:40994) 2024-04-17T10:04:01.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.903+0000 7f0ee5c00640 1 -- 172.21.15.32:0/1340908913 learned_addr learned my addr 172.21.15.32:0/1340908913 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:01.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.903+0000 7f0ee6600640 1 --2- 172.21.15.32:0/1340908913 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ee015b7d0 0x7f0ee0149b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:01.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.903+0000 7f0ee5200640 1 --2- 172.21.15.32:0/1340908913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ee006f7c0 0x7f0ee01550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:01.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.903+0000 7f0ee5c00640 1 -- 172.21.15.32:0/1340908913 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ee015b7d0 msgr2=0x7f0ee0149b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:01.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.903+0000 7f0ee5c00640 1 --2- 172.21.15.32:0/1340908913 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ee015b7d0 0x7f0ee0149b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.903+0000 7f0ee5c00640 1 -- 172.21.15.32:0/1340908913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ee006f7c0 msgr2=0x7f0ee01550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:01.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.903+0000 7f0ee5c00640 1 --2- 172.21.15.32:0/1340908913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ee006f7c0 0x7f0ee01550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:01.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.903+0000 7f0ee5c00640 1 -- 172.21.15.32:0/1340908913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee014c8b0 con 0x7f0ee006c140 2024-04-17T10:04:01.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.903+0000 7f0ee5200640 1 --2- 172.21.15.32:0/1340908913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ee006f7c0 0x7f0ee01550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:01.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.903+0000 7f0ee5c00640 1 --2- 172.21.15.32:0/1340908913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ee006c140 0x7f0ee0156b90 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f0ed401a040 tx=0x7f0ed401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:01.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.903+0000 7f0edd600640 1 -- 172.21.15.32:0/1340908913 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4006e20 con 0x7f0ee006c140 2024-04-17T10:04:01.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.903+0000 7f0edd600640 1 -- 172.21.15.32:0/1340908913 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ed4004020 con 0x7f0ee006c140 2024-04-17T10:04:01.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.903+0000 7f0ee8600640 1 -- 172.21.15.32:0/1340908913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee0159250 con 0x7f0ee006c140 2024-04-17T10:04:01.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.903+0000 7f0edd600640 1 -- 172.21.15.32:0/1340908913 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed40043d0 con 0x7f0ee006c140 2024-04-17T10:04:01.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.903+0000 7f0ee8600640 1 -- 172.21.15.32:0/1340908913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ee014ce30 con 0x7f0ee006c140 2024-04-17T10:04:01.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.903+0000 7f0ee8600640 1 -- 172.21.15.32:0/1340908913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ee0159420 con 0x7f0ee006c140 2024-04-17T10:04:01.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.903+0000 7f0edd600640 1 -- 172.21.15.32:0/1340908913 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0ed4003620 con 0x7f0ee006c140 2024-04-17T10:04:01.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.907+0000 7f0edd600640 1 --2- 172.21.15.32:0/1340908913 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0ec00683b0 0x7f0ec006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:01.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.907+0000 7f0edd600640 1 -- 172.21.15.32:0/1340908913 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7f0ed41206f0 con 0x7f0ee006c140 2024-04-17T10:04:01.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.907+0000 7f0ee5200640 1 --2- 172.21.15.32:0/1340908913 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0ec00683b0 0x7f0ec006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:01.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.907+0000 7f0ee5200640 1 --2- 172.21.15.32:0/1340908913 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0ec00683b0 0x7f0ec006a870 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f0ee014b4b0 tx=0x7f0ecc007c40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:01.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:01.911+0000 7f0edd600640 1 -- 172.21.15.32:0/1340908913 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0ed4016090 con 0x7f0ee006c140 2024-04-17T10:04:02.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.071+0000 7f0ee8600640 1 -- 172.21.15.32:0/1340908913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0ee0068a90 con 0x7f0ee006c140 2024-04-17T10:04:02.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.071+0000 7f0edd600640 1 -- 172.21.15.32:0/1340908913 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v111) ==== 100+0+11 (secure 0 0 0) 0x7f0ed40f0610 con 0x7f0ee006c140 2024-04-17T10:04:02.071 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T10:04:02.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.075+0000 7f0ee8600640 1 -- 172.21.15.32:0/1340908913 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0ec00683b0 msgr2=0x7f0ec006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:02.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.075+0000 7f0ee8600640 1 --2- 172.21.15.32:0/1340908913 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0ec00683b0 0x7f0ec006a870 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f0ee014b4b0 tx=0x7f0ecc007c40 comp rx=0 tx=0).stop 2024-04-17T10:04:02.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.075+0000 7f0ee8600640 1 -- 172.21.15.32:0/1340908913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ee006c140 msgr2=0x7f0ee0156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:02.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.075+0000 7f0ee8600640 1 --2- 172.21.15.32:0/1340908913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ee006c140 0x7f0ee0156b90 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f0ed401a040 tx=0x7f0ed401ac50 comp rx=0 tx=0).stop 2024-04-17T10:04:02.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.075+0000 7f0ee8600640 1 -- 172.21.15.32:0/1340908913 shutdown_connections 2024-04-17T10:04:02.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.075+0000 7f0ee8600640 1 --2- 172.21.15.32:0/1340908913 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0ec00683b0 0x7f0ec006a870 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:02.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.075+0000 7f0ee8600640 1 --2- 172.21.15.32:0/1340908913 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ee015b7d0 0x7f0ee0149b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:02.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.075+0000 7f0ee8600640 1 --2- 172.21.15.32:0/1340908913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ee006f7c0 0x7f0ee01550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:02.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.075+0000 7f0ee8600640 1 --2- 172.21.15.32:0/1340908913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ee006c140 0x7f0ee0156b90 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:02.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.075+0000 7f0ee8600640 1 -- 172.21.15.32:0/1340908913 >> 172.21.15.32:0/1340908913 conn(0x7f0ee00b2120 msgr2=0x7f0ee0072490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:02.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.075+0000 7f0ee8600640 1 -- 172.21.15.32:0/1340908913 shutdown_connections 2024-04-17T10:04:02.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.075+0000 7f0ee8600640 1 -- 172.21.15.32:0/1340908913 wait complete. 2024-04-17T10:04:02.093 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:04:02.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.235+0000 7f873a800640 1 Processor -- start 2024-04-17T10:04:02.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.235+0000 7f873a800640 1 -- start start 2024-04-17T10:04:02.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.239+0000 7f873a800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f873413f6e0 0x7f87340cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:02.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.239+0000 7f873a800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f87340cf370 0x7f87340d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:02.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.239+0000 7f873a800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f87340d0820 0x7f8734141d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:02.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.239+0000 7f873a800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f873413fac0 con 0x7f87340cf370 2024-04-17T10:04:02.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.239+0000 7f873a800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f87341601e0 con 0x7f87340d0820 2024-04-17T10:04:02.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.239+0000 7f873a800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8734160480 con 0x7f873413f6e0 2024-04-17T10:04:02.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.239+0000 7f8733400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f873413f6e0 0x7f87340cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:02.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.239+0000 7f872be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f87340cf370 0x7f87340d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:02.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.239+0000 7f8733400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f873413f6e0 0x7f87340cf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:53066/0 (socket says 172.21.15.32:53066) 2024-04-17T10:04:02.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.239+0000 7f8733400640 1 -- 172.21.15.32:0/3608193566 learned_addr learned my addr 172.21.15.32:0/3608193566 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:02.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.239+0000 7f8733e00640 1 --2- 172.21.15.32:0/3608193566 >> v2:172.21.15.144:3300/0 conn(0x7f87340d0820 0x7f8734141d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:02.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.239+0000 7f872be00640 1 -- 172.21.15.32:0/3608193566 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f873413f6e0 msgr2=0x7f87340cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:02.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.239+0000 7f872be00640 1 --2- 172.21.15.32:0/3608193566 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f873413f6e0 0x7f87340cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:02.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.239+0000 7f872be00640 1 -- 172.21.15.32:0/3608193566 >> v2:172.21.15.144:3300/0 conn(0x7f87340d0820 msgr2=0x7f8734141d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:02.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.239+0000 7f872be00640 1 --2- 172.21.15.32:0/3608193566 >> v2:172.21.15.144:3300/0 conn(0x7f87340d0820 0x7f8734141d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:02.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.239+0000 7f872be00640 1 -- 172.21.15.32:0/3608193566 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8734160760 con 0x7f87340cf370 2024-04-17T10:04:02.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.239+0000 7f8733400640 1 --2- 172.21.15.32:0/3608193566 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f873413f6e0 0x7f87340cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:02.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.243+0000 7f872be00640 1 --2- 172.21.15.32:0/3608193566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87340cf370 0x7f87340d02e0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f873406db20 tx=0x7f871c020890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced32ef4da165676 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:02.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.243+0000 7f8732a00640 1 -- 172.21.15.32:0/3608193566 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c0b0070 con 0x7f87340cf370 2024-04-17T10:04:02.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.243+0000 7f8732a00640 1 -- 172.21.15.32:0/3608193566 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f871c0aa620 con 0x7f87340cf370 2024-04-17T10:04:02.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.243+0000 7f8732a00640 1 -- 172.21.15.32:0/3608193566 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c0aa9f0 con 0x7f87340cf370 2024-04-17T10:04:02.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.243+0000 7f873a800640 1 -- 172.21.15.32:0/3608193566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87340cf370 msgr2=0x7f87340d02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:02.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.243+0000 7f873a800640 1 --2- 172.21.15.32:0/3608193566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87340cf370 0x7f87340d02e0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f873406db20 tx=0x7f871c020890 comp rx=0 tx=0).stop 2024-04-17T10:04:02.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.243+0000 7f873a800640 1 -- 172.21.15.32:0/3608193566 shutdown_connections 2024-04-17T10:04:02.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.243+0000 7f873a800640 1 --2- 172.21.15.32:0/3608193566 >> v2:172.21.15.144:3300/0 conn(0x7f87340d0820 0x7f8734141d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:02.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.243+0000 7f873a800640 1 --2- 172.21.15.32:0/3608193566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87340cf370 0x7f87340d02e0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:02.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.243+0000 7f873a800640 1 --2- 172.21.15.32:0/3608193566 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f873413f6e0 0x7f87340cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:02.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.243+0000 7f873a800640 1 -- 172.21.15.32:0/3608193566 >> 172.21.15.32:0/3608193566 conn(0x7f873406ba30 msgr2=0x7f873406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:02.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.243+0000 7f873a800640 1 -- 172.21.15.32:0/3608193566 shutdown_connections 2024-04-17T10:04:02.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.243+0000 7f873a800640 1 -- 172.21.15.32:0/3608193566 wait complete. 2024-04-17T10:04:02.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.243+0000 7f873a800640 1 Processor -- start 2024-04-17T10:04:02.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.243+0000 7f873a800640 1 -- start start 2024-04-17T10:04:02.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.243+0000 7f873a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87340cf370 0x7f87340b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:02.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.243+0000 7f873a800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87340d0820 0x7f87341f3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:02.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.243+0000 7f873a800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f873413f6e0 0x7f87341f3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:02.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.247+0000 7f873a800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f87340cff10 con 0x7f87340cf370 2024-04-17T10:04:02.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.247+0000 7f873a800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f87340cfd90 con 0x7f87340d0820 2024-04-17T10:04:02.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.247+0000 7f873a800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8734071d80 con 0x7f873413f6e0 2024-04-17T10:04:02.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.247+0000 7f8733400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87340cf370 0x7f87340b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:02.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.247+0000 7f8733400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87340cf370 0x7f87340b2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41008/0 (socket says 172.21.15.32:41008) 2024-04-17T10:04:02.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.247+0000 7f8733400640 1 -- 172.21.15.32:0/1482276189 learned_addr learned my addr 172.21.15.32:0/1482276189 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:02.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.247+0000 7f8733e00640 1 --2- 172.21.15.32:0/1482276189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f873413f6e0 0x7f87341f3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:02.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.247+0000 7f872be00640 1 --2- 172.21.15.32:0/1482276189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87340d0820 0x7f87341f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:02.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.247+0000 7f8733400640 1 -- 172.21.15.32:0/1482276189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f873413f6e0 msgr2=0x7f87341f3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:02.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.247+0000 7f8733400640 1 --2- 172.21.15.32:0/1482276189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f873413f6e0 0x7f87341f3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:02.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.247+0000 7f8733400640 1 -- 172.21.15.32:0/1482276189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87340d0820 msgr2=0x7f87341f3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:02.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.247+0000 7f8733400640 1 --2- 172.21.15.32:0/1482276189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87340d0820 0x7f87341f3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:02.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.247+0000 7f8733400640 1 -- 172.21.15.32:0/1482276189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87341440d0 con 0x7f87340cf370 2024-04-17T10:04:02.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.247+0000 7f8733400640 1 --2- 172.21.15.32:0/1482276189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87340cf370 0x7f87340b2820 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f872400e810 tx=0x7f872400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:02.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.247+0000 7f872b400640 1 -- 172.21.15.32:0/1482276189 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872400f040 con 0x7f87340cf370 2024-04-17T10:04:02.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.247+0000 7f872b400640 1 -- 172.21.15.32:0/1482276189 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8724004230 con 0x7f87340cf370 2024-04-17T10:04:02.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.247+0000 7f872b400640 1 -- 172.21.15.32:0/1482276189 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872400c6d0 con 0x7f87340cf370 2024-04-17T10:04:02.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.247+0000 7f873a800640 1 -- 172.21.15.32:0/1482276189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8734143030 con 0x7f87340cf370 2024-04-17T10:04:02.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.247+0000 7f873a800640 1 -- 172.21.15.32:0/1482276189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87341435f0 con 0x7f87340cf370 2024-04-17T10:04:02.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.247+0000 7f873a800640 1 -- 172.21.15.32:0/1482276189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8734068a90 con 0x7f87340cf370 2024-04-17T10:04:02.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.251+0000 7f872b400640 1 -- 172.21.15.32:0/1482276189 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f872400c870 con 0x7f87340cf370 2024-04-17T10:04:02.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.251+0000 7f872b400640 1 --2- 172.21.15.32:0/1482276189 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8710068380 0x7f871006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:02.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.251+0000 7f872b400640 1 -- 172.21.15.32:0/1482276189 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7f87240866b0 con 0x7f87340cf370 2024-04-17T10:04:02.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.251+0000 7f872be00640 1 --2- 172.21.15.32:0/1482276189 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8710068380 0x7f871006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:02.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.251+0000 7f872be00640 1 --2- 172.21.15.32:0/1482276189 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8710068380 0x7f871006a840 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f871c0b2000 tx=0x7f871c0b7040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:02.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.255+0000 7f872b400640 1 -- 172.21.15.32:0/1482276189 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f8724017020 con 0x7f87340cf370 2024-04-17T10:04:02.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.435+0000 7f873a800640 1 -- 172.21.15.32:0/1482276189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f87340d1030 con 0x7f87340cf370 2024-04-17T10:04:02.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.435+0000 7f872b400640 1 -- 172.21.15.32:0/1482276189 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v111) ==== 80+0+0 (secure 0 0 0) 0x7f8724056540 con 0x7f87340cf370 2024-04-17T10:04:02.433 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2024-04-17T10:04:02.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.439+0000 7f873a800640 1 -- 172.21.15.32:0/1482276189 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8710068380 msgr2=0x7f871006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:02.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.439+0000 7f873a800640 1 --2- 172.21.15.32:0/1482276189 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8710068380 0x7f871006a840 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f871c0b2000 tx=0x7f871c0b7040 comp rx=0 tx=0).stop 2024-04-17T10:04:02.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.439+0000 7f873a800640 1 -- 172.21.15.32:0/1482276189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87340cf370 msgr2=0x7f87340b2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:02.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.439+0000 7f873a800640 1 --2- 172.21.15.32:0/1482276189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87340cf370 0x7f87340b2820 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f872400e810 tx=0x7f872400ecd0 comp rx=0 tx=0).stop 2024-04-17T10:04:02.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.439+0000 7f873a800640 1 -- 172.21.15.32:0/1482276189 shutdown_connections 2024-04-17T10:04:02.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.439+0000 7f873a800640 1 --2- 172.21.15.32:0/1482276189 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8710068380 0x7f871006a840 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:02.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.439+0000 7f873a800640 1 --2- 172.21.15.32:0/1482276189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f873413f6e0 0x7f87341f3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:02.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.439+0000 7f873a800640 1 --2- 172.21.15.32:0/1482276189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87340d0820 0x7f87341f3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:02.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.439+0000 7f873a800640 1 --2- 172.21.15.32:0/1482276189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87340cf370 0x7f87340b2820 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:02.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.439+0000 7f873a800640 1 -- 172.21.15.32:0/1482276189 >> 172.21.15.32:0/1482276189 conn(0x7f873406ba30 msgr2=0x7f873406f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:02.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.439+0000 7f873a800640 1 -- 172.21.15.32:0/1482276189 shutdown_connections 2024-04-17T10:04:02.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.439+0000 7f873a800640 1 -- 172.21.15.32:0/1482276189 wait complete. 2024-04-17T10:04:02.455 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2024-04-17T10:04:02.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:02.455 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T10:04:02.473 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[51693]: fuse finished with error 0 and tester_r 0 2024-04-17T10:04:02.478 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:04:02.548 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T10:04:02.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:02.548 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T10:04:02.557 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:04:02.557 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:04:02.611 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:04:02.611 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi032.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-04-17T10:04:02.611 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T10:04:02.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:02.611 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T10:04:02.667 INFO:teuthology.orchestra.run.smithi032.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-17T10:04:02.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:04:02.668 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T10:04:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.859+0000 7fe4b4200640 1 Processor -- start 2024-04-17T10:04:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.859+0000 7fe4b4200640 1 -- start start 2024-04-17T10:04:02.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.859+0000 7fe4b4200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4ac155a70 0x7fe4ac15f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:02.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.859+0000 7fe4b4200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe4ac0b3340 0x7fe4ac15c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:02.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.859+0000 7fe4b4200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4ac15f870 con 0x7fe4ac155e50 2024-04-17T10:04:02.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.859+0000 7fe4b4200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe4ac0b42a0 con 0x7fe4ac0b3340 2024-04-17T10:04:02.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.859+0000 7fe4b4200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe4ac0d1250 con 0x7fe4ac155a70 2024-04-17T10:04:02.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.859+0000 7fe4b0e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe4ac155e50 0x7fe4ac15d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35970/0 (socket says 172.21.15.32:35970) 2024-04-17T10:04:02.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.859+0000 7fe4b0e00640 1 -- 172.21.15.32:0/1895174696 learned_addr learned my addr 172.21.15.32:0/1895174696 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:02.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.859+0000 7fe4b2200640 1 --2- 172.21.15.32:0/1895174696 >> v2:172.21.15.32:3300/0 conn(0x7fe4ac0b3340 0x7fe4ac15c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:02.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.859+0000 7fe4b1800640 1 --2- 172.21.15.32:0/1895174696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4ac155a70 0x7fe4ac15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:02.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.859+0000 7fe4b2200640 1 -- 172.21.15.32:0/1895174696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4ac155a70 msgr2=0x7fe4ac15f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:02.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.859+0000 7fe4b2200640 1 --2- 172.21.15.32:0/1895174696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4ac155a70 0x7fe4ac15f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:02.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.859+0000 7fe4b2200640 1 -- 172.21.15.32:0/1895174696 >> v1:172.21.15.32:6789/0 conn(0x7fe4ac155e50 legacy=0x7fe4ac15d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:02.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.859+0000 7fe4b2200640 1 -- 172.21.15.32:0/1895174696 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4ac15fb50 con 0x7fe4ac0b3340 2024-04-17T10:04:02.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.863+0000 7fe4b2200640 1 --2- 172.21.15.32:0/1895174696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4ac0b3340 0x7fe4ac15c260 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009d90 tx=0x7fe4a401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66019f1c511d2c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:02.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.863+0000 7fe4abe00640 1 -- 172.21.15.32:0/1895174696 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40b2070 con 0x7fe4ac0b3340 2024-04-17T10:04:02.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.863+0000 7fe4abe00640 1 -- 172.21.15.32:0/1895174696 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4a40a5830 con 0x7fe4ac0b3340 2024-04-17T10:04:02.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.863+0000 7fe4abe00640 1 -- 172.21.15.32:0/1895174696 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40a5c00 con 0x7fe4ac0b3340 2024-04-17T10:04:02.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.863+0000 7fe4b4200640 1 -- 172.21.15.32:0/1895174696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4ac0b3340 msgr2=0x7fe4ac15c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:02.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.863+0000 7fe4b4200640 1 --2- 172.21.15.32:0/1895174696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4ac0b3340 0x7fe4ac15c260 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009d90 tx=0x7fe4a401c970 comp rx=0 tx=0).stop 2024-04-17T10:04:02.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.863+0000 7fe4b4200640 1 -- 172.21.15.32:0/1895174696 shutdown_connections 2024-04-17T10:04:02.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.863+0000 7fe4b4200640 1 --2- 172.21.15.32:0/1895174696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4ac0b3340 0x7fe4ac15c260 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:02.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.863+0000 7fe4b4200640 1 --2- 172.21.15.32:0/1895174696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4ac155a70 0x7fe4ac15f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:02.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.863+0000 7fe4b4200640 1 -- 172.21.15.32:0/1895174696 >> 172.21.15.32:0/1895174696 conn(0x7fe4ac06ba30 msgr2=0x7fe4ac06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:02.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.863+0000 7fe4b4200640 1 -- 172.21.15.32:0/1895174696 shutdown_connections 2024-04-17T10:04:02.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.863+0000 7fe4b4200640 1 -- 172.21.15.32:0/1895174696 wait complete. 2024-04-17T10:04:02.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4b4200640 1 Processor -- start 2024-04-17T10:04:02.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4b4200640 1 -- start start 2024-04-17T10:04:02.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4b4200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4ac0b3340 0x7fe4ac1fad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:02.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4b4200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4ac155a70 0x7fe4ac1f7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:02.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4b4200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4ac155e50 0x7fe4ac0cef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:02.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4b4200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac0d02f0 con 0x7fe4ac155e50 2024-04-17T10:04:02.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4b4200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac0d0170 con 0x7fe4ac155a70 2024-04-17T10:04:02.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4b4200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe4ac0d0470 con 0x7fe4ac0b3340 2024-04-17T10:04:02.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4b2200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4ac155e50 0x7fe4ac0cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:02.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4b2200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4ac155e50 0x7fe4ac0cef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41020/0 (socket says 172.21.15.32:41020) 2024-04-17T10:04:02.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4b2200640 1 -- 172.21.15.32:0/1068838893 learned_addr learned my addr 172.21.15.32:0/1068838893 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:02.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4b0e00640 1 --2- 172.21.15.32:0/1068838893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4ac155a70 0x7fe4ac1f7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:02.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4b1800640 1 --2- 172.21.15.32:0/1068838893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4ac0b3340 0x7fe4ac1fad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:02.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4b2200640 1 -- 172.21.15.32:0/1068838893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4ac0b3340 msgr2=0x7fe4ac1fad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:02.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4b2200640 1 --2- 172.21.15.32:0/1068838893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4ac0b3340 0x7fe4ac1fad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:02.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4b2200640 1 -- 172.21.15.32:0/1068838893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4ac155a70 msgr2=0x7fe4ac1f7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:02.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4b2200640 1 --2- 172.21.15.32:0/1068838893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4ac155a70 0x7fe4ac1f7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:02.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4b2200640 1 -- 172.21.15.32:0/1068838893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4ac1f8370 con 0x7fe4ac155e50 2024-04-17T10:04:02.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4b2200640 1 --2- 172.21.15.32:0/1068838893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4ac155e50 0x7fe4ac0cef30 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fe4a401b020 tx=0x7fe4a4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:02.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4a9600640 1 -- 172.21.15.32:0/1068838893 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40b2050 con 0x7fe4ac155e50 2024-04-17T10:04:02.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4a9600640 1 -- 172.21.15.32:0/1068838893 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4a40be070 con 0x7fe4ac155e50 2024-04-17T10:04:02.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4a9600640 1 -- 172.21.15.32:0/1068838893 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40ada50 con 0x7fe4ac155e50 2024-04-17T10:04:02.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4b4200640 1 -- 172.21.15.32:0/1068838893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4ac158cc0 con 0x7fe4ac155e50 2024-04-17T10:04:02.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4b4200640 1 -- 172.21.15.32:0/1068838893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4ac159f40 con 0x7fe4ac155e50 2024-04-17T10:04:02.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.867+0000 7fe4b4200640 1 -- 172.21.15.32:0/1068838893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4ac068a90 con 0x7fe4ac155e50 2024-04-17T10:04:02.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.871+0000 7fe4a9600640 1 -- 172.21.15.32:0/1068838893 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe4a40a56b0 con 0x7fe4ac155e50 2024-04-17T10:04:02.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.871+0000 7fe4a9600640 1 --2- 172.21.15.32:0/1068838893 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe48c068380 0x7fe48c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:02.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.871+0000 7fe4a9600640 1 -- 172.21.15.32:0/1068838893 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7fe4a4121e90 con 0x7fe4ac155e50 2024-04-17T10:04:02.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.871+0000 7fe4b1800640 1 --2- 172.21.15.32:0/1068838893 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe48c068380 0x7fe48c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:02.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.871+0000 7fe4b1800640 1 --2- 172.21.15.32:0/1068838893 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe48c068380 0x7fe48c06a840 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe4a000b9d0 tx=0x7fe4a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:02.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:02.875+0000 7fe4a9600640 1 -- 172.21.15.32:0/1068838893 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fe4ac1f8370 con 0x7fe4ac155e50 2024-04-17T10:04:03.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.075+0000 7fe4b4200640 1 -- 172.21.15.32:0/1068838893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe4ac06ec90 con 0x7fe4ac155e50 2024-04-17T10:04:03.073 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-17T10:04:03.075+0000 7f93c9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T10:04:03.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.075+0000 7fe4a9600640 1 -- 172.21.15.32:0/1068838893 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v99) ==== 161+0+0 (secure 0 0 0) 0x7fe4a4123090 con 0x7fe4ac155e50 2024-04-17T10:04:03.075 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T10:04:03.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.079+0000 7fe4b4200640 1 -- 172.21.15.32:0/1068838893 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe48c068380 msgr2=0x7fe48c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:03.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.079+0000 7fe4b4200640 1 --2- 172.21.15.32:0/1068838893 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe48c068380 0x7fe48c06a840 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe4a000b9d0 tx=0x7fe4a0008040 comp rx=0 tx=0).stop 2024-04-17T10:04:03.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.079+0000 7fe4b4200640 1 -- 172.21.15.32:0/1068838893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4ac155e50 msgr2=0x7fe4ac0cef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:03.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.079+0000 7fe4b4200640 1 --2- 172.21.15.32:0/1068838893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4ac155e50 0x7fe4ac0cef30 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fe4a401b020 tx=0x7fe4a4019710 comp rx=0 tx=0).stop 2024-04-17T10:04:03.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.079+0000 7fe4b4200640 1 -- 172.21.15.32:0/1068838893 shutdown_connections 2024-04-17T10:04:03.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.079+0000 7fe4b4200640 1 --2- 172.21.15.32:0/1068838893 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe48c068380 0x7fe48c06a840 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:03.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.079+0000 7fe4b4200640 1 --2- 172.21.15.32:0/1068838893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4ac155e50 0x7fe4ac0cef30 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:03.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.079+0000 7fe4b4200640 1 --2- 172.21.15.32:0/1068838893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4ac155a70 0x7fe4ac1f7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:03.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.079+0000 7fe4b4200640 1 --2- 172.21.15.32:0/1068838893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4ac0b3340 0x7fe4ac1fad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:03.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.079+0000 7fe4b4200640 1 -- 172.21.15.32:0/1068838893 >> 172.21.15.32:0/1068838893 conn(0x7fe4ac06ba30 msgr2=0x7fe4ac15d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:03.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.079+0000 7fe4b4200640 1 -- 172.21.15.32:0/1068838893 shutdown_connections 2024-04-17T10:04:03.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.079+0000 7fe4b4200640 1 -- 172.21.15.32:0/1068838893 wait complete. 2024-04-17T10:04:03.090 INFO:tasks.ceph.mds.a.smithi032.stderr: -7935> 2024-04-17T10:03:40.203+0000 7f229ec00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:04:03.100 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-17T10:04:03.103 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-17T10:04:03.141 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-17T10:04:03.145 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2024-04-17T10:04:03.145 INFO:teuthology.orchestra.run.smithi032.stdout: chdir = "" 2024-04-17T10:04:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-17T10:04:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: auth supported = cephx 2024-04-17T10:04:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: filestore xattr use omap = true 2024-04-17T10:04:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: mon clock drift allowed = 1.000 2024-04-17T10:04:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: osd crush chooseleaf type = 0 2024-04-17T10:04:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: auth debug = true 2024-04-17T10:04:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on old message = true 2024-04-17T10:04:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on bug = true 2024-04-17T10:04:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.146 INFO:teuthology.orchestra.run.smithi032.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-17T10:04:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: mon pg warn max object skew = 0 2024-04-17T10:04:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: # disable pg_autoscaler by default for new pools 2024-04-17T10:04:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-17T10:04:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default size = 2 2024-04-17T10:04:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow primary affinity = true 2024-04-17T10:04:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow pg remap = true 2024-04-17T10:04:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on legacy crush tunables = false 2024-04-17T10:04:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on crush straw calc version zero = false 2024-04-17T10:04:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on no sortbitwise = false 2024-04-17T10:04:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on osd down out interval zero = false 2024-04-17T10:04:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on too few osds = false 2024-04-17T10:04:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-17T10:04:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-17T10:04:03.147 INFO:teuthology.orchestra.run.smithi032.stdout: mon_allow_pool_size_one = true 2024-04-17T10:04:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-17T10:04:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: osd default data pool replay window = 5 2024-04-17T10:04:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: mon allow pool delete = true 2024-04-17T10:04:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: mon cluster log file level = debug 2024-04-17T10:04:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: debug asserts on shutdown = true 2024-04-17T10:04:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: mon health detail to clog = false 2024-04-17T10:04:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: mon host = "172.21.15.32,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2024-04-17T10:04:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = ff178df8-4c06-4367-a1e9-2e3a66d870d0 2024-04-17T10:04:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.148 INFO:teuthology.orchestra.run.smithi032.stdout:[osd] 2024-04-17T10:04:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: osd journal size = 100 2024-04-17T10:04:03.148 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub load threshold = 5.0 2024-04-17T10:04:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub max interval = 600 2024-04-17T10:04:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: osd mclock profile = high_recovery_ops 2024-04-17T10:04:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: osd recover clone overlap = true 2024-04-17T10:04:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: osd recovery max chunk = 1048576 2024-04-17T10:04:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug shutdown = true 2024-04-17T10:04:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug op order = true 2024-04-17T10:04:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug verify stray on activate = true 2024-04-17T10:04:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug trim objects = true 2024-04-17T10:04:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: osd open classes on start = true 2024-04-17T10:04:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug pg log writeout = true 2024-04-17T10:04:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: osd deep scrub update digest min age = 30 2024-04-17T10:04:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: osd map max advance = 10 2024-04-17T10:04:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: journal zero on create = true 2024-04-17T10:04:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: filestore ondisk finisher threads = 3 2024-04-17T10:04:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: filestore apply finisher threads = 3 2024-04-17T10:04:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: bdev debug aio = true 2024-04-17T10:04:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug misdirected ops = true 2024-04-17T10:04:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: bdev async discard = True 2024-04-17T10:04:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: bdev enable discard = True 2024-04-17T10:04:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore allocator = bitmap 2024-04-17T10:04:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore block size = 96636764160 2024-04-17T10:04:03.150 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore fsck on mount = True 2024-04-17T10:04:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluefs = 1/20 2024-04-17T10:04:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluestore = 1/20 2024-04-17T10:04:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T10:04:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: debug osd = 20 2024-04-17T10:04:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: debug rocksdb = 4/10 2024-04-17T10:04:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd backfillfull_ratio = 0.85 2024-04-17T10:04:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd full ratio = 0.9 2024-04-17T10:04:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd nearfull ratio = 0.8 2024-04-17T10:04:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: osd failsafe full ratio = 0.95 2024-04-17T10:04:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: osd objectstore = bluestore 2024-04-17T10:04:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2024-04-17T10:04:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.151 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr] 2024-04-17T10:04:03.151 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T10:04:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: debug mgr = 20 2024-04-17T10:04:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2024-04-17T10:04:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2024-04-17T10:04:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2024-04-17T10:04:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2024-04-17T10:04:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: mgr/telemetry/nag = false 2024-04-17T10:04:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2024-04-17T10:04:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.152 INFO:teuthology.orchestra.run.smithi032.stdout:[mon] 2024-04-17T10:04:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T10:04:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2024-04-17T10:04:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: debug paxos = 20 2024-04-17T10:04:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2024-04-17T10:04:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: mon data avail warn = 5 2024-04-17T10:04:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: mon mgr mkfs grace = 240 2024-04-17T10:04:03.152 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2024-04-17T10:04:03.153 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd reporter subtree level = osd 2024-04-17T10:04:03.153 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd prime pg temp = true 2024-04-17T10:04:03.153 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2024-04-17T10:04:03.153 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.153 INFO:teuthology.orchestra.run.smithi032.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-17T10:04:03.153 INFO:teuthology.orchestra.run.smithi032.stdout: auth mon ticket ttl = 660 # 11m 2024-04-17T10:04:03.153 INFO:teuthology.orchestra.run.smithi032.stdout: auth service ticket ttl = 240 # 4m 2024-04-17T10:04:03.153 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.153 INFO:teuthology.orchestra.run.smithi032.stdout: # don't complain about insecure global_id in the test suite 2024-04-17T10:04:03.153 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-17T10:04:03.153 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-17T10:04:03.153 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.153 INFO:teuthology.orchestra.run.smithi032.stdout: # 1m isn't quite enough 2024-04-17T10:04:03.153 INFO:teuthology.orchestra.run.smithi032.stdout: mon_down_mkfs_grace = 2m 2024-04-17T10:04:03.153 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.154 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_filestore_osds = false 2024-04-17T10:04:03.154 INFO:teuthology.orchestra.run.smithi032.stdout: mon down mkfs grace = 300 2024-04-17T10:04:03.154 INFO:teuthology.orchestra.run.smithi032.stdout: mon op complaint time = 120 2024-04-17T10:04:03.154 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.154 INFO:teuthology.orchestra.run.smithi032.stdout:[client] 2024-04-17T10:04:03.154 INFO:teuthology.orchestra.run.smithi032.stdout: rgw cache enabled = true 2024-04-17T10:04:03.154 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable ops log = true 2024-04-17T10:04:03.154 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable usage log = true 2024-04-17T10:04:03.154 INFO:teuthology.orchestra.run.smithi032.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-17T10:04:03.154 INFO:teuthology.orchestra.run.smithi032.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-17T10:04:03.154 INFO:teuthology.orchestra.run.smithi032.stdout: client mount timeout = 600 2024-04-17T10:04:03.154 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2024-04-17T10:04:03.154 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T10:04:03.154 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = "" 2024-04-17T10:04:03.155 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2024-04-17T10:04:03.155 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2024-04-17T10:04:03.155 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.a] 2024-04-17T10:04:03.155 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.b] 2024-04-17T10:04:03.155 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.c] 2024-04-17T10:04:03.155 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2024-04-17T10:04:03.155 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c] 2024-04-17T10:04:03.155 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2024-04-17T10:04:03.155 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.d] 2024-04-17T10:04:03.155 INFO:teuthology.orchestra.run.smithi032.stdout:[mds] 2024-04-17T10:04:03.155 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds = 20 2024-04-17T10:04:03.155 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds balancer = 20 2024-04-17T10:04:03.155 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T10:04:03.155 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug frag = True 2024-04-17T10:04:03.155 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug scatterstat = True 2024-04-17T10:04:03.156 INFO:teuthology.orchestra.run.smithi032.stdout: mds op complaint time = 180 2024-04-17T10:04:03.156 INFO:teuthology.orchestra.run.smithi032.stdout: mds verify scatter = False 2024-04-17T10:04:03.156 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2024-04-17T10:04:03.156 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2024-04-17T10:04:03.156 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2024-04-17T10:04:03.156 INFO:teuthology.orchestra.run.smithi144.stdout:[global] 2024-04-17T10:04:03.156 INFO:teuthology.orchestra.run.smithi144.stdout: chdir = "" 2024-04-17T10:04:03.156 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-17T10:04:03.156 INFO:teuthology.orchestra.run.smithi144.stdout: auth supported = cephx 2024-04-17T10:04:03.156 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.157 INFO:teuthology.orchestra.run.smithi144.stdout: filestore xattr use omap = true 2024-04-17T10:04:03.157 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.157 INFO:teuthology.orchestra.run.smithi144.stdout: mon clock drift allowed = 1.000 2024-04-17T10:04:03.157 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.157 INFO:teuthology.orchestra.run.smithi144.stdout: osd crush chooseleaf type = 0 2024-04-17T10:04:03.157 INFO:teuthology.orchestra.run.smithi144.stdout: auth debug = true 2024-04-17T10:04:03.157 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.157 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on old message = true 2024-04-17T10:04:03.157 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on bug = true 2024-04-17T10:04:03.157 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.157 INFO:teuthology.orchestra.run.smithi144.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-17T10:04:03.157 INFO:teuthology.orchestra.run.smithi144.stdout: mon pg warn max object skew = 0 2024-04-17T10:04:03.157 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.157 INFO:teuthology.orchestra.run.smithi144.stdout: # disable pg_autoscaler by default for new pools 2024-04-17T10:04:03.158 INFO:teuthology.orchestra.run.smithi144.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-17T10:04:03.158 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.158 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default size = 2 2024-04-17T10:04:03.158 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.158 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow primary affinity = true 2024-04-17T10:04:03.158 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow pg remap = true 2024-04-17T10:04:03.158 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on legacy crush tunables = false 2024-04-17T10:04:03.158 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on crush straw calc version zero = false 2024-04-17T10:04:03.158 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on no sortbitwise = false 2024-04-17T10:04:03.158 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on osd down out interval zero = false 2024-04-17T10:04:03.158 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on too few osds = false 2024-04-17T10:04:03.158 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-17T10:04:03.158 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-17T10:04:03.158 INFO:teuthology.orchestra.run.smithi144.stdout: mon_allow_pool_size_one = true 2024-04-17T10:04:03.159 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.159 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-17T10:04:03.159 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.159 INFO:teuthology.orchestra.run.smithi144.stdout: osd default data pool replay window = 5 2024-04-17T10:04:03.159 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.159 INFO:teuthology.orchestra.run.smithi144.stdout: mon allow pool delete = true 2024-04-17T10:04:03.159 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.159 INFO:teuthology.orchestra.run.smithi144.stdout: mon cluster log file level = debug 2024-04-17T10:04:03.159 INFO:teuthology.orchestra.run.smithi144.stdout: debug asserts on shutdown = true 2024-04-17T10:04:03.159 INFO:teuthology.orchestra.run.smithi144.stdout: mon health detail to clog = false 2024-04-17T10:04:03.159 INFO:teuthology.orchestra.run.smithi144.stdout: mon host = "172.21.15.32,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2024-04-17T10:04:03.159 INFO:teuthology.orchestra.run.smithi144.stdout: fsid = ff178df8-4c06-4367-a1e9-2e3a66d870d0 2024-04-17T10:04:03.159 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.159 INFO:teuthology.orchestra.run.smithi144.stdout:[osd] 2024-04-17T10:04:03.159 INFO:teuthology.orchestra.run.smithi144.stdout: osd journal size = 100 2024-04-17T10:04:03.159 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.160 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub load threshold = 5.0 2024-04-17T10:04:03.160 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub max interval = 600 2024-04-17T10:04:03.160 INFO:teuthology.orchestra.run.smithi144.stdout: osd mclock profile = high_recovery_ops 2024-04-17T10:04:03.160 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.160 INFO:teuthology.orchestra.run.smithi144.stdout: osd recover clone overlap = true 2024-04-17T10:04:03.160 INFO:teuthology.orchestra.run.smithi144.stdout: osd recovery max chunk = 1048576 2024-04-17T10:04:03.160 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.160 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug shutdown = true 2024-04-17T10:04:03.160 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug op order = true 2024-04-17T10:04:03.160 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug verify stray on activate = true 2024-04-17T10:04:03.160 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug trim objects = true 2024-04-17T10:04:03.160 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.160 INFO:teuthology.orchestra.run.smithi144.stdout: osd open classes on start = true 2024-04-17T10:04:03.160 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug pg log writeout = true 2024-04-17T10:04:03.160 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.160 INFO:teuthology.orchestra.run.smithi144.stdout: osd deep scrub update digest min age = 30 2024-04-17T10:04:03.161 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.161 INFO:teuthology.orchestra.run.smithi144.stdout: osd map max advance = 10 2024-04-17T10:04:03.161 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.161 INFO:teuthology.orchestra.run.smithi144.stdout: journal zero on create = true 2024-04-17T10:04:03.161 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.161 INFO:teuthology.orchestra.run.smithi144.stdout: filestore ondisk finisher threads = 3 2024-04-17T10:04:03.161 INFO:teuthology.orchestra.run.smithi144.stdout: filestore apply finisher threads = 3 2024-04-17T10:04:03.161 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.161 INFO:teuthology.orchestra.run.smithi144.stdout: bdev debug aio = true 2024-04-17T10:04:03.161 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug misdirected ops = true 2024-04-17T10:04:03.161 INFO:teuthology.orchestra.run.smithi144.stdout: bdev async discard = True 2024-04-17T10:04:03.161 INFO:teuthology.orchestra.run.smithi144.stdout: bdev enable discard = True 2024-04-17T10:04:03.161 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore allocator = bitmap 2024-04-17T10:04:03.161 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore block size = 96636764160 2024-04-17T10:04:03.161 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore fsck on mount = True 2024-04-17T10:04:03.161 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluefs = 1/20 2024-04-17T10:04:03.162 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluestore = 1/20 2024-04-17T10:04:03.162 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T10:04:03.162 INFO:teuthology.orchestra.run.smithi144.stdout: debug osd = 20 2024-04-17T10:04:03.162 INFO:teuthology.orchestra.run.smithi144.stdout: debug rocksdb = 4/10 2024-04-17T10:04:03.162 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd backfillfull_ratio = 0.85 2024-04-17T10:04:03.162 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd full ratio = 0.9 2024-04-17T10:04:03.162 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd nearfull ratio = 0.8 2024-04-17T10:04:03.162 INFO:teuthology.orchestra.run.smithi144.stdout: osd failsafe full ratio = 0.95 2024-04-17T10:04:03.162 INFO:teuthology.orchestra.run.smithi144.stdout: osd objectstore = bluestore 2024-04-17T10:04:03.162 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2024-04-17T10:04:03.162 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.162 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr] 2024-04-17T10:04:03.162 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T10:04:03.162 INFO:teuthology.orchestra.run.smithi144.stdout: debug mgr = 20 2024-04-17T10:04:03.163 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-04-17T10:04:03.163 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-04-17T10:04:03.163 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-04-17T10:04:03.163 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-04-17T10:04:03.163 INFO:teuthology.orchestra.run.smithi144.stdout: mgr/telemetry/nag = false 2024-04-17T10:04:03.163 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2024-04-17T10:04:03.163 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.163 INFO:teuthology.orchestra.run.smithi144.stdout:[mon] 2024-04-17T10:04:03.163 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T10:04:03.163 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-04-17T10:04:03.163 INFO:teuthology.orchestra.run.smithi144.stdout: debug paxos = 20 2024-04-17T10:04:03.163 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-04-17T10:04:03.163 INFO:teuthology.orchestra.run.smithi144.stdout: mon data avail warn = 5 2024-04-17T10:04:03.163 INFO:teuthology.orchestra.run.smithi144.stdout: mon mgr mkfs grace = 240 2024-04-17T10:04:03.163 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-04-17T10:04:03.163 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd reporter subtree level = osd 2024-04-17T10:04:03.164 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd prime pg temp = true 2024-04-17T10:04:03.164 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-04-17T10:04:03.164 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.164 INFO:teuthology.orchestra.run.smithi144.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-17T10:04:03.164 INFO:teuthology.orchestra.run.smithi144.stdout: auth mon ticket ttl = 660 # 11m 2024-04-17T10:04:03.164 INFO:teuthology.orchestra.run.smithi144.stdout: auth service ticket ttl = 240 # 4m 2024-04-17T10:04:03.164 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.164 INFO:teuthology.orchestra.run.smithi144.stdout: # don't complain about insecure global_id in the test suite 2024-04-17T10:04:03.164 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-17T10:04:03.164 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-17T10:04:03.164 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.164 INFO:teuthology.orchestra.run.smithi144.stdout: # 1m isn't quite enough 2024-04-17T10:04:03.164 INFO:teuthology.orchestra.run.smithi144.stdout: mon_down_mkfs_grace = 2m 2024-04-17T10:04:03.164 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.164 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_filestore_osds = false 2024-04-17T10:04:03.164 INFO:teuthology.orchestra.run.smithi144.stdout: mon down mkfs grace = 300 2024-04-17T10:04:03.165 INFO:teuthology.orchestra.run.smithi144.stdout: mon op complaint time = 120 2024-04-17T10:04:03.165 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.165 INFO:teuthology.orchestra.run.smithi144.stdout:[client] 2024-04-17T10:04:03.165 INFO:teuthology.orchestra.run.smithi144.stdout: rgw cache enabled = true 2024-04-17T10:04:03.165 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable ops log = true 2024-04-17T10:04:03.165 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable usage log = true 2024-04-17T10:04:03.165 INFO:teuthology.orchestra.run.smithi144.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-17T10:04:03.165 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-17T10:04:03.165 INFO:teuthology.orchestra.run.smithi144.stdout: client mount timeout = 600 2024-04-17T10:04:03.165 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2024-04-17T10:04:03.165 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T10:04:03.165 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = "" 2024-04-17T10:04:03.165 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2024-04-17T10:04:03.165 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2024-04-17T10:04:03.165 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.a] 2024-04-17T10:04:03.166 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.b] 2024-04-17T10:04:03.166 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.c] 2024-04-17T10:04:03.166 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.a] 2024-04-17T10:04:03.166 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.c] 2024-04-17T10:04:03.166 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.b] 2024-04-17T10:04:03.166 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.d] 2024-04-17T10:04:03.166 INFO:teuthology.orchestra.run.smithi144.stdout:[mds] 2024-04-17T10:04:03.166 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds = 20 2024-04-17T10:04:03.166 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds balancer = 20 2024-04-17T10:04:03.166 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T10:04:03.166 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug frag = True 2024-04-17T10:04:03.166 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug scatterstat = True 2024-04-17T10:04:03.166 INFO:teuthology.orchestra.run.smithi144.stdout: mds op complaint time = 180 2024-04-17T10:04:03.166 INFO:teuthology.orchestra.run.smithi144.stdout: mds verify scatter = False 2024-04-17T10:04:03.166 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2024-04-17T10:04:03.166 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2024-04-17T10:04:03.167 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2024-04-17T10:04:03.168 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-17T10:04:03.214 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-17T10:04:03.255 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2024-04-17T10:04:03.255 INFO:teuthology.orchestra.run.smithi032.stdout: chdir = "" 2024-04-17T10:04:03.255 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-17T10:04:03.255 INFO:teuthology.orchestra.run.smithi032.stdout: auth supported = cephx 2024-04-17T10:04:03.255 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.255 INFO:teuthology.orchestra.run.smithi032.stdout: filestore xattr use omap = true 2024-04-17T10:04:03.255 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.255 INFO:teuthology.orchestra.run.smithi032.stdout: mon clock drift allowed = 1.000 2024-04-17T10:04:03.255 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.255 INFO:teuthology.orchestra.run.smithi032.stdout: osd crush chooseleaf type = 0 2024-04-17T10:04:03.256 INFO:teuthology.orchestra.run.smithi032.stdout: auth debug = true 2024-04-17T10:04:03.256 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.256 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on old message = true 2024-04-17T10:04:03.256 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on bug = true 2024-04-17T10:04:03.256 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.256 INFO:teuthology.orchestra.run.smithi032.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-17T10:04:03.256 INFO:teuthology.orchestra.run.smithi032.stdout: mon pg warn max object skew = 0 2024-04-17T10:04:03.256 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.256 INFO:teuthology.orchestra.run.smithi032.stdout: # disable pg_autoscaler by default for new pools 2024-04-17T10:04:03.256 INFO:teuthology.orchestra.run.smithi032.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-17T10:04:03.256 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.256 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default size = 2 2024-04-17T10:04:03.256 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.256 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow primary affinity = true 2024-04-17T10:04:03.256 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow pg remap = true 2024-04-17T10:04:03.256 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on legacy crush tunables = false 2024-04-17T10:04:03.257 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on crush straw calc version zero = false 2024-04-17T10:04:03.257 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on no sortbitwise = false 2024-04-17T10:04:03.257 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on osd down out interval zero = false 2024-04-17T10:04:03.257 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on too few osds = false 2024-04-17T10:04:03.257 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-17T10:04:03.257 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-17T10:04:03.257 INFO:teuthology.orchestra.run.smithi032.stdout: mon_allow_pool_size_one = true 2024-04-17T10:04:03.257 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.257 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-17T10:04:03.257 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.257 INFO:teuthology.orchestra.run.smithi032.stdout: osd default data pool replay window = 5 2024-04-17T10:04:03.257 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.257 INFO:teuthology.orchestra.run.smithi032.stdout: mon allow pool delete = true 2024-04-17T10:04:03.257 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.257 INFO:teuthology.orchestra.run.smithi032.stdout: mon cluster log file level = debug 2024-04-17T10:04:03.258 INFO:teuthology.orchestra.run.smithi032.stdout: debug asserts on shutdown = true 2024-04-17T10:04:03.258 INFO:teuthology.orchestra.run.smithi032.stdout: mon health detail to clog = false 2024-04-17T10:04:03.258 INFO:teuthology.orchestra.run.smithi032.stdout: mon host = "172.21.15.32,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2024-04-17T10:04:03.258 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = ff178df8-4c06-4367-a1e9-2e3a66d870d0 2024-04-17T10:04:03.258 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.258 INFO:teuthology.orchestra.run.smithi032.stdout:[osd] 2024-04-17T10:04:03.258 INFO:teuthology.orchestra.run.smithi032.stdout: osd journal size = 100 2024-04-17T10:04:03.258 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.258 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub load threshold = 5.0 2024-04-17T10:04:03.258 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub max interval = 600 2024-04-17T10:04:03.258 INFO:teuthology.orchestra.run.smithi032.stdout: osd mclock profile = high_recovery_ops 2024-04-17T10:04:03.258 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.258 INFO:teuthology.orchestra.run.smithi032.stdout: osd recover clone overlap = true 2024-04-17T10:04:03.258 INFO:teuthology.orchestra.run.smithi032.stdout: osd recovery max chunk = 1048576 2024-04-17T10:04:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug shutdown = true 2024-04-17T10:04:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug op order = true 2024-04-17T10:04:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug verify stray on activate = true 2024-04-17T10:04:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug trim objects = true 2024-04-17T10:04:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: osd open classes on start = true 2024-04-17T10:04:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug pg log writeout = true 2024-04-17T10:04:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: osd deep scrub update digest min age = 30 2024-04-17T10:04:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: osd map max advance = 10 2024-04-17T10:04:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: journal zero on create = true 2024-04-17T10:04:03.259 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: filestore ondisk finisher threads = 3 2024-04-17T10:04:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: filestore apply finisher threads = 3 2024-04-17T10:04:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: bdev debug aio = true 2024-04-17T10:04:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug misdirected ops = true 2024-04-17T10:04:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: bdev async discard = True 2024-04-17T10:04:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: bdev enable discard = True 2024-04-17T10:04:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore allocator = bitmap 2024-04-17T10:04:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore block size = 96636764160 2024-04-17T10:04:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore fsck on mount = True 2024-04-17T10:04:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluefs = 1/20 2024-04-17T10:04:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluestore = 1/20 2024-04-17T10:04:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T10:04:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: debug osd = 20 2024-04-17T10:04:03.260 INFO:teuthology.orchestra.run.smithi032.stdout: debug rocksdb = 4/10 2024-04-17T10:04:03.261 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd backfillfull_ratio = 0.85 2024-04-17T10:04:03.261 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd full ratio = 0.9 2024-04-17T10:04:03.261 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd nearfull ratio = 0.8 2024-04-17T10:04:03.261 INFO:teuthology.orchestra.run.smithi032.stdout: osd failsafe full ratio = 0.95 2024-04-17T10:04:03.261 INFO:teuthology.orchestra.run.smithi032.stdout: osd objectstore = bluestore 2024-04-17T10:04:03.261 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2024-04-17T10:04:03.261 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.261 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr] 2024-04-17T10:04:03.261 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T10:04:03.261 INFO:teuthology.orchestra.run.smithi032.stdout: debug mgr = 20 2024-04-17T10:04:03.261 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2024-04-17T10:04:03.261 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2024-04-17T10:04:03.261 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2024-04-17T10:04:03.261 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2024-04-17T10:04:03.261 INFO:teuthology.orchestra.run.smithi032.stdout: mgr/telemetry/nag = false 2024-04-17T10:04:03.262 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2024-04-17T10:04:03.262 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.262 INFO:teuthology.orchestra.run.smithi032.stdout:[mon] 2024-04-17T10:04:03.262 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T10:04:03.262 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2024-04-17T10:04:03.262 INFO:teuthology.orchestra.run.smithi032.stdout: debug paxos = 20 2024-04-17T10:04:03.262 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2024-04-17T10:04:03.262 INFO:teuthology.orchestra.run.smithi032.stdout: mon data avail warn = 5 2024-04-17T10:04:03.262 INFO:teuthology.orchestra.run.smithi032.stdout: mon mgr mkfs grace = 240 2024-04-17T10:04:03.262 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2024-04-17T10:04:03.262 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd reporter subtree level = osd 2024-04-17T10:04:03.262 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd prime pg temp = true 2024-04-17T10:04:03.262 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2024-04-17T10:04:03.262 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-17T10:04:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: auth mon ticket ttl = 660 # 11m 2024-04-17T10:04:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: auth service ticket ttl = 240 # 4m 2024-04-17T10:04:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: # don't complain about insecure global_id in the test suite 2024-04-17T10:04:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-17T10:04:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-17T10:04:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: # 1m isn't quite enough 2024-04-17T10:04:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: mon_down_mkfs_grace = 2m 2024-04-17T10:04:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_filestore_osds = false 2024-04-17T10:04:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: mon down mkfs grace = 300 2024-04-17T10:04:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: mon op complaint time = 120 2024-04-17T10:04:03.263 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:03.263 INFO:teuthology.orchestra.run.smithi032.stdout:[client] 2024-04-17T10:04:03.264 INFO:teuthology.orchestra.run.smithi032.stdout: rgw cache enabled = true 2024-04-17T10:04:03.264 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable ops log = true 2024-04-17T10:04:03.264 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable usage log = true 2024-04-17T10:04:03.264 INFO:teuthology.orchestra.run.smithi032.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-17T10:04:03.264 INFO:teuthology.orchestra.run.smithi032.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-17T10:04:03.264 INFO:teuthology.orchestra.run.smithi032.stdout: client mount timeout = 600 2024-04-17T10:04:03.264 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2024-04-17T10:04:03.264 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T10:04:03.264 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = "" 2024-04-17T10:04:03.264 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2024-04-17T10:04:03.264 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2024-04-17T10:04:03.264 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.a] 2024-04-17T10:04:03.264 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.b] 2024-04-17T10:04:03.264 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.c] 2024-04-17T10:04:03.264 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2024-04-17T10:04:03.265 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c] 2024-04-17T10:04:03.265 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2024-04-17T10:04:03.265 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.d] 2024-04-17T10:04:03.265 INFO:teuthology.orchestra.run.smithi032.stdout:[mds] 2024-04-17T10:04:03.265 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds = 20 2024-04-17T10:04:03.265 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds balancer = 20 2024-04-17T10:04:03.265 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T10:04:03.265 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug frag = True 2024-04-17T10:04:03.265 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug scatterstat = False 2024-04-17T10:04:03.265 INFO:teuthology.orchestra.run.smithi032.stdout: mds op complaint time = 180 2024-04-17T10:04:03.265 INFO:teuthology.orchestra.run.smithi032.stdout: mds verify scatter = False 2024-04-17T10:04:03.265 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2024-04-17T10:04:03.265 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2024-04-17T10:04:03.265 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2024-04-17T10:04:03.266 INFO:teuthology.orchestra.run.smithi144.stdout:[global] 2024-04-17T10:04:03.266 INFO:teuthology.orchestra.run.smithi144.stdout: chdir = "" 2024-04-17T10:04:03.266 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-17T10:04:03.266 INFO:teuthology.orchestra.run.smithi144.stdout: auth supported = cephx 2024-04-17T10:04:03.266 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.266 INFO:teuthology.orchestra.run.smithi144.stdout: filestore xattr use omap = true 2024-04-17T10:04:03.266 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.266 INFO:teuthology.orchestra.run.smithi144.stdout: mon clock drift allowed = 1.000 2024-04-17T10:04:03.266 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.267 INFO:teuthology.orchestra.run.smithi144.stdout: osd crush chooseleaf type = 0 2024-04-17T10:04:03.267 INFO:teuthology.orchestra.run.smithi144.stdout: auth debug = true 2024-04-17T10:04:03.267 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.267 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on old message = true 2024-04-17T10:04:03.267 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on bug = true 2024-04-17T10:04:03.267 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.267 INFO:teuthology.orchestra.run.smithi144.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-17T10:04:03.267 INFO:teuthology.orchestra.run.smithi144.stdout: mon pg warn max object skew = 0 2024-04-17T10:04:03.267 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.267 INFO:teuthology.orchestra.run.smithi144.stdout: # disable pg_autoscaler by default for new pools 2024-04-17T10:04:03.267 INFO:teuthology.orchestra.run.smithi144.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-17T10:04:03.267 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.267 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default size = 2 2024-04-17T10:04:03.267 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.267 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow primary affinity = true 2024-04-17T10:04:03.267 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow pg remap = true 2024-04-17T10:04:03.268 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on legacy crush tunables = false 2024-04-17T10:04:03.268 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on crush straw calc version zero = false 2024-04-17T10:04:03.268 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on no sortbitwise = false 2024-04-17T10:04:03.268 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on osd down out interval zero = false 2024-04-17T10:04:03.268 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on too few osds = false 2024-04-17T10:04:03.268 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-17T10:04:03.268 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-17T10:04:03.268 INFO:teuthology.orchestra.run.smithi144.stdout: mon_allow_pool_size_one = true 2024-04-17T10:04:03.268 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.268 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-17T10:04:03.268 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.268 INFO:teuthology.orchestra.run.smithi144.stdout: osd default data pool replay window = 5 2024-04-17T10:04:03.268 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.268 INFO:teuthology.orchestra.run.smithi144.stdout: mon allow pool delete = true 2024-04-17T10:04:03.268 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: mon cluster log file level = debug 2024-04-17T10:04:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: debug asserts on shutdown = true 2024-04-17T10:04:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: mon health detail to clog = false 2024-04-17T10:04:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: mon host = "172.21.15.32,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2024-04-17T10:04:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: fsid = ff178df8-4c06-4367-a1e9-2e3a66d870d0 2024-04-17T10:04:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.269 INFO:teuthology.orchestra.run.smithi144.stdout:[osd] 2024-04-17T10:04:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: osd journal size = 100 2024-04-17T10:04:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub load threshold = 5.0 2024-04-17T10:04:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub max interval = 600 2024-04-17T10:04:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: osd mclock profile = high_recovery_ops 2024-04-17T10:04:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: osd recover clone overlap = true 2024-04-17T10:04:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: osd recovery max chunk = 1048576 2024-04-17T10:04:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug shutdown = true 2024-04-17T10:04:03.270 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug op order = true 2024-04-17T10:04:03.270 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug verify stray on activate = true 2024-04-17T10:04:03.270 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug trim objects = true 2024-04-17T10:04:03.270 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.270 INFO:teuthology.orchestra.run.smithi144.stdout: osd open classes on start = true 2024-04-17T10:04:03.270 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug pg log writeout = true 2024-04-17T10:04:03.270 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.270 INFO:teuthology.orchestra.run.smithi144.stdout: osd deep scrub update digest min age = 30 2024-04-17T10:04:03.270 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.270 INFO:teuthology.orchestra.run.smithi144.stdout: osd map max advance = 10 2024-04-17T10:04:03.270 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.270 INFO:teuthology.orchestra.run.smithi144.stdout: journal zero on create = true 2024-04-17T10:04:03.270 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.270 INFO:teuthology.orchestra.run.smithi144.stdout: filestore ondisk finisher threads = 3 2024-04-17T10:04:03.270 INFO:teuthology.orchestra.run.smithi144.stdout: filestore apply finisher threads = 3 2024-04-17T10:04:03.271 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.271 INFO:teuthology.orchestra.run.smithi144.stdout: bdev debug aio = true 2024-04-17T10:04:03.271 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug misdirected ops = true 2024-04-17T10:04:03.271 INFO:teuthology.orchestra.run.smithi144.stdout: bdev async discard = True 2024-04-17T10:04:03.271 INFO:teuthology.orchestra.run.smithi144.stdout: bdev enable discard = True 2024-04-17T10:04:03.271 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore allocator = bitmap 2024-04-17T10:04:03.271 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore block size = 96636764160 2024-04-17T10:04:03.271 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore fsck on mount = True 2024-04-17T10:04:03.271 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluefs = 1/20 2024-04-17T10:04:03.271 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluestore = 1/20 2024-04-17T10:04:03.271 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T10:04:03.271 INFO:teuthology.orchestra.run.smithi144.stdout: debug osd = 20 2024-04-17T10:04:03.271 INFO:teuthology.orchestra.run.smithi144.stdout: debug rocksdb = 4/10 2024-04-17T10:04:03.271 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd backfillfull_ratio = 0.85 2024-04-17T10:04:03.272 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd full ratio = 0.9 2024-04-17T10:04:03.272 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd nearfull ratio = 0.8 2024-04-17T10:04:03.272 INFO:teuthology.orchestra.run.smithi144.stdout: osd failsafe full ratio = 0.95 2024-04-17T10:04:03.272 INFO:teuthology.orchestra.run.smithi144.stdout: osd objectstore = bluestore 2024-04-17T10:04:03.272 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2024-04-17T10:04:03.272 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.272 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr] 2024-04-17T10:04:03.272 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T10:04:03.272 INFO:teuthology.orchestra.run.smithi144.stdout: debug mgr = 20 2024-04-17T10:04:03.272 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-04-17T10:04:03.272 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-04-17T10:04:03.272 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-04-17T10:04:03.272 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-04-17T10:04:03.272 INFO:teuthology.orchestra.run.smithi144.stdout: mgr/telemetry/nag = false 2024-04-17T10:04:03.273 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2024-04-17T10:04:03.273 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.273 INFO:teuthology.orchestra.run.smithi144.stdout:[mon] 2024-04-17T10:04:03.273 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T10:04:03.273 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-04-17T10:04:03.273 INFO:teuthology.orchestra.run.smithi144.stdout: debug paxos = 20 2024-04-17T10:04:03.273 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-04-17T10:04:03.273 INFO:teuthology.orchestra.run.smithi144.stdout: mon data avail warn = 5 2024-04-17T10:04:03.273 INFO:teuthology.orchestra.run.smithi144.stdout: mon mgr mkfs grace = 240 2024-04-17T10:04:03.273 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-04-17T10:04:03.273 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd reporter subtree level = osd 2024-04-17T10:04:03.273 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd prime pg temp = true 2024-04-17T10:04:03.273 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-04-17T10:04:03.273 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.273 INFO:teuthology.orchestra.run.smithi144.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-17T10:04:03.273 INFO:teuthology.orchestra.run.smithi144.stdout: auth mon ticket ttl = 660 # 11m 2024-04-17T10:04:03.274 INFO:teuthology.orchestra.run.smithi144.stdout: auth service ticket ttl = 240 # 4m 2024-04-17T10:04:03.274 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.274 INFO:teuthology.orchestra.run.smithi144.stdout: # don't complain about insecure global_id in the test suite 2024-04-17T10:04:03.274 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-17T10:04:03.274 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-17T10:04:03.274 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.274 INFO:teuthology.orchestra.run.smithi144.stdout: # 1m isn't quite enough 2024-04-17T10:04:03.274 INFO:teuthology.orchestra.run.smithi144.stdout: mon_down_mkfs_grace = 2m 2024-04-17T10:04:03.274 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.274 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_filestore_osds = false 2024-04-17T10:04:03.274 INFO:teuthology.orchestra.run.smithi144.stdout: mon down mkfs grace = 300 2024-04-17T10:04:03.274 INFO:teuthology.orchestra.run.smithi144.stdout: mon op complaint time = 120 2024-04-17T10:04:03.274 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:03.274 INFO:teuthology.orchestra.run.smithi144.stdout:[client] 2024-04-17T10:04:03.274 INFO:teuthology.orchestra.run.smithi144.stdout: rgw cache enabled = true 2024-04-17T10:04:03.274 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable ops log = true 2024-04-17T10:04:03.275 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable usage log = true 2024-04-17T10:04:03.275 INFO:teuthology.orchestra.run.smithi144.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-17T10:04:03.275 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-17T10:04:03.275 INFO:teuthology.orchestra.run.smithi144.stdout: client mount timeout = 600 2024-04-17T10:04:03.275 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2024-04-17T10:04:03.275 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T10:04:03.275 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = "" 2024-04-17T10:04:03.275 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2024-04-17T10:04:03.275 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2024-04-17T10:04:03.275 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.a] 2024-04-17T10:04:03.275 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.b] 2024-04-17T10:04:03.275 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.c] 2024-04-17T10:04:03.275 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.a] 2024-04-17T10:04:03.275 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.c] 2024-04-17T10:04:03.276 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.b] 2024-04-17T10:04:03.276 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.d] 2024-04-17T10:04:03.276 INFO:teuthology.orchestra.run.smithi144.stdout:[mds] 2024-04-17T10:04:03.276 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds = 20 2024-04-17T10:04:03.276 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds balancer = 20 2024-04-17T10:04:03.276 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T10:04:03.276 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug frag = True 2024-04-17T10:04:03.276 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug scatterstat = False 2024-04-17T10:04:03.276 INFO:teuthology.orchestra.run.smithi144.stdout: mds op complaint time = 180 2024-04-17T10:04:03.276 INFO:teuthology.orchestra.run.smithi144.stdout: mds verify scatter = False 2024-04-17T10:04:03.276 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2024-04-17T10:04:03.276 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2024-04-17T10:04:03.276 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2024-04-17T10:04:03.278 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-04-17T10:04:03.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.367+0000 7ff753255080 1 Processor -- start 2024-04-17T10:04:03.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.367+0000 7ff753255080 1 -- start start 2024-04-17T10:04:03.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.367+0000 7ff753255080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559439f3a230 0x559439fc4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:03.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.367+0000 7ff753255080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559439f3a610 0x559439f1eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:03.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.367+0000 7ff753255080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559439fc7270 0x559439fc1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:03.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.367+0000 7ff753255080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559439f3d5f0 con 0x559439fc7270 2024-04-17T10:04:03.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.367+0000 7ff753255080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559439e96f90 con 0x559439f3a230 2024-04-17T10:04:03.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.367+0000 7ff753255080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x559439f237e0 con 0x559439f3a610 2024-04-17T10:04:03.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.367+0000 7ff750e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559439fc7270 0x559439fc1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:03.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.367+0000 7ff750e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559439fc7270 0x559439fc1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41042/0 (socket says 172.21.15.32:41042) 2024-04-17T10:04:03.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.367+0000 7ff750e00640 1 -- 172.21.15.32:0/92110589 learned_addr learned my addr 172.21.15.32:0/92110589 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:03.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.367+0000 7ff74fa00640 1 --2- 172.21.15.32:0/92110589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559439f3a610 0x559439f1eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:03.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.367+0000 7ff750400640 1 --2- 172.21.15.32:0/92110589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559439f3a230 0x559439fc4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:03.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.367+0000 7ff750e00640 1 -- 172.21.15.32:0/92110589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559439f3a610 msgr2=0x559439f1eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:03.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.367+0000 7ff750e00640 1 --2- 172.21.15.32:0/92110589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559439f3a610 0x559439f1eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:03.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.367+0000 7ff750e00640 1 -- 172.21.15.32:0/92110589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559439f3a230 msgr2=0x559439fc4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:03.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.367+0000 7ff750e00640 1 --2- 172.21.15.32:0/92110589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559439f3a230 0x559439fc4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:03.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.367+0000 7ff750e00640 1 -- 172.21.15.32:0/92110589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559439e1b950 con 0x559439fc7270 2024-04-17T10:04:03.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.367+0000 7ff750400640 1 --2- 172.21.15.32:0/92110589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559439f3a230 0x559439fc4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:03.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff750e00640 1 --2- 172.21.15.32:0/92110589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559439fc7270 0x559439fc1800 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7ff7480ae040 tx=0x7ff7480033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985823fca3e745f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:03.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff74e600640 1 -- 172.21.15.32:0/92110589 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7480b9070 con 0x559439fc7270 2024-04-17T10:04:03.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff74e600640 1 -- 172.21.15.32:0/92110589 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7480ab640 con 0x559439fc7270 2024-04-17T10:04:03.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff74e600640 1 -- 172.21.15.32:0/92110589 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7480b0040 con 0x559439fc7270 2024-04-17T10:04:03.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff753255080 1 -- 172.21.15.32:0/92110589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559439fc7270 msgr2=0x559439fc1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:03.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff753255080 1 --2- 172.21.15.32:0/92110589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559439fc7270 0x559439fc1800 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7ff7480ae040 tx=0x7ff7480033b0 comp rx=0 tx=0).stop 2024-04-17T10:04:03.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff753255080 1 -- 172.21.15.32:0/92110589 shutdown_connections 2024-04-17T10:04:03.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff753255080 1 --2- 172.21.15.32:0/92110589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559439f3a610 0x559439f1eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:03.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff753255080 1 --2- 172.21.15.32:0/92110589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559439fc7270 0x559439fc1800 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:03.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff753255080 1 --2- 172.21.15.32:0/92110589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559439f3a230 0x559439fc4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:03.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff753255080 1 -- 172.21.15.32:0/92110589 >> 172.21.15.32:0/92110589 conn(0x559439ef0620 msgr2=0x559439ef0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:03.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff753255080 1 -- 172.21.15.32:0/92110589 shutdown_connections 2024-04-17T10:04:03.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff753255080 1 -- 172.21.15.32:0/92110589 wait complete. 2024-04-17T10:04:03.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff753255080 1 Processor -- start 2024-04-17T10:04:03.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff753255080 1 -- start start 2024-04-17T10:04:03.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff753255080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559439f3a230 0x559439fc1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:03.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff753255080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559439f3a610 0x559439fbe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:03.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff753255080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559439fc7270 0x559439fba110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:03.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff753255080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x559439fc3770 con 0x559439f3a230 2024-04-17T10:04:03.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff753255080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559439fc34d0 con 0x559439fc7270 2024-04-17T10:04:03.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff753255080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x559439fc3a10 con 0x559439f3a610 2024-04-17T10:04:03.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff750400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559439f3a230 0x559439fc1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:03.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff750400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559439f3a230 0x559439fc1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41052/0 (socket says 172.21.15.32:41052) 2024-04-17T10:04:03.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff750400640 1 -- 172.21.15.32:0/545953328 learned_addr learned my addr 172.21.15.32:0/545953328 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:03.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff750e00640 1 --2- 172.21.15.32:0/545953328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559439fc7270 0x559439fba110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:03.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff74fa00640 1 --2- 172.21.15.32:0/545953328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559439f3a610 0x559439fbe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:03.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff750400640 1 -- 172.21.15.32:0/545953328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559439f3a610 msgr2=0x559439fbe6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:03.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff750400640 1 --2- 172.21.15.32:0/545953328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559439f3a610 0x559439fbe6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:03.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff750400640 1 -- 172.21.15.32:0/545953328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559439fc7270 msgr2=0x559439fba110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:03.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff750400640 1 --2- 172.21.15.32:0/545953328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559439fc7270 0x559439fba110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:03.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff750400640 1 -- 172.21.15.32:0/545953328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559439fc3090 con 0x559439f3a230 2024-04-17T10:04:03.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff750400640 1 --2- 172.21.15.32:0/545953328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559439f3a230 0x559439fc1fc0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7ff74000dd60 tx=0x7ff74000b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:03.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff74d200640 1 -- 172.21.15.32:0/545953328 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740017470 con 0x559439f3a230 2024-04-17T10:04:03.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff74d200640 1 -- 172.21.15.32:0/545953328 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff740012070 con 0x559439f3a230 2024-04-17T10:04:03.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.371+0000 7ff74d200640 1 -- 172.21.15.32:0/545953328 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740017810 con 0x559439f3a230 2024-04-17T10:04:03.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.375+0000 7ff753255080 1 -- 172.21.15.32:0/545953328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559439f34790 con 0x559439f3a230 2024-04-17T10:04:03.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.375+0000 7ff753255080 1 -- 172.21.15.32:0/545953328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559439f34cd0 con 0x559439f3a230 2024-04-17T10:04:03.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.375+0000 7ff74d200640 1 -- 172.21.15.32:0/545953328 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff740009070 con 0x559439f3a230 2024-04-17T10:04:03.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.375+0000 7ff74d200640 1 --2- 172.21.15.32:0/545953328 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff730066d90 0x7ff730069250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:03.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.375+0000 7ff74d200640 1 -- 172.21.15.32:0/545953328 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7ff7400858e0 con 0x559439f3a230 2024-04-17T10:04:03.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.375+0000 7ff74fa00640 1 --2- 172.21.15.32:0/545953328 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff730066d90 0x7ff730069250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:03.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.375+0000 7ff753255080 1 --2- 172.21.15.32:0/545953328 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x559439fbebf0 0x559439fc0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:03.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.375+0000 7ff753255080 1 -- 172.21.15.32:0/545953328 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:1 16.7 16:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x559439fc0ff0 con 0x559439fbebf0 2024-04-17T10:04:03.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.375+0000 7ff750e00640 1 --2- 172.21.15.32:0/545953328 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x559439fbebf0 0x559439fc0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:03.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.375+0000 7ff74fa00640 1 --2- 172.21.15.32:0/545953328 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff730066d90 0x7ff730069250 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009940 tx=0x7ff73c00c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:03.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.375+0000 7ff750e00640 1 --2- 172.21.15.32:0/545953328 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x559439fbebf0 0x559439fc0ab0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:03.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.379+0000 7ff750e00640 1 -- 172.21.15.32:0/545953328 <== osd.3 v2:172.21.15.32:6824/2182899298 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v112'5 uv5 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff7480ab7f0 con 0x559439fbebf0 2024-04-17T10:04:03.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.379+0000 7ff753255080 1 -- 172.21.15.32:0/545953328 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x559439fbebf0 msgr2=0x559439fc0ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:03.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.379+0000 7ff753255080 1 --2- 172.21.15.32:0/545953328 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x559439fbebf0 0x559439fc0ab0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:03.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.379+0000 7ff753255080 1 -- 172.21.15.32:0/545953328 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff730066d90 msgr2=0x7ff730069250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:03.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.379+0000 7ff753255080 1 --2- 172.21.15.32:0/545953328 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff730066d90 0x7ff730069250 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009940 tx=0x7ff73c00c040 comp rx=0 tx=0).stop 2024-04-17T10:04:03.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.379+0000 7ff753255080 1 -- 172.21.15.32:0/545953328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559439f3a230 msgr2=0x559439fc1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:03.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.379+0000 7ff753255080 1 --2- 172.21.15.32:0/545953328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559439f3a230 0x559439fc1fc0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7ff74000dd60 tx=0x7ff74000b6d0 comp rx=0 tx=0).stop 2024-04-17T10:04:03.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.379+0000 7ff750e00640 1 -- 172.21.15.32:0/545953328 reap_dead start 2024-04-17T10:04:03.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.379+0000 7ff753255080 1 -- 172.21.15.32:0/545953328 shutdown_connections 2024-04-17T10:04:03.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.379+0000 7ff753255080 1 -- 172.21.15.32:0/545953328 >> 172.21.15.32:0/545953328 conn(0x559439ef0620 msgr2=0x559439fd1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:03.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.379+0000 7ff753255080 1 -- 172.21.15.32:0/545953328 shutdown_connections 2024-04-17T10:04:03.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.379+0000 7ff753255080 1 -- 172.21.15.32:0/545953328 wait complete. 2024-04-17T10:04:03.383 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-17T10:04:03.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.567+0000 7fa117e00640 1 Processor -- start 2024-04-17T10:04:03.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.567+0000 7fa117e00640 1 -- start start 2024-04-17T10:04:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.571+0000 7fa117e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa11015e7c0 0x7fa11015eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.571+0000 7fa117e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1100b4490 0x7fa1100b5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.571+0000 7fa117e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1100b56d0 con 0x7fa1100aef90 2024-04-17T10:04:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.571+0000 7fa117e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa1100b39b0 con 0x7fa11015e7c0 2024-04-17T10:04:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.571+0000 7fa117e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa1100b3c90 con 0x7fa1100b4490 2024-04-17T10:04:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.571+0000 7fa114a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1100b4490 0x7fa1100b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.571+0000 7fa115e00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fa1100aef90 0x7fa110072260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:33874/0 (socket says 172.21.15.32:33874) 2024-04-17T10:04:03.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.571+0000 7fa115e00640 1 -- 172.21.15.32:0/3622187410 learned_addr learned my addr 172.21.15.32:0/3622187410 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:03.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.571+0000 7fa115400640 1 --2- 172.21.15.32:0/3622187410 >> v2:172.21.15.144:3300/0 conn(0x7fa11015e7c0 0x7fa11015eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:03.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.571+0000 7fa114a00640 1 -- 172.21.15.32:0/3622187410 >> v2:172.21.15.144:3300/0 conn(0x7fa11015e7c0 msgr2=0x7fa11015eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:03.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.571+0000 7fa114a00640 1 --2- 172.21.15.32:0/3622187410 >> v2:172.21.15.144:3300/0 conn(0x7fa11015e7c0 0x7fa11015eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:03.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.571+0000 7fa10fe00640 1 -- 172.21.15.32:0/3622187410 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 470673158 0 0) 0x7fa1100b56d0 con 0x7fa1100aef90 2024-04-17T10:04:03.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.571+0000 7fa114a00640 1 -- 172.21.15.32:0/3622187410 >> v1:172.21.15.144:6789/0 conn(0x7fa1100aef90 legacy=0x7fa110072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:03.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.571+0000 7fa114a00640 1 -- 172.21.15.32:0/3622187410 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1100b3f70 con 0x7fa1100b4490 2024-04-17T10:04:03.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.575+0000 7fa114a00640 1 --2- 172.21.15.32:0/3622187410 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1100b4490 0x7fa1100b5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0097c0 tx=0x7fa0fc01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32190d094cea57ef server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:03.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.575+0000 7fa10fe00640 1 -- 172.21.15.32:0/3622187410 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc0ab430 con 0x7fa1100b4490 2024-04-17T10:04:03.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.575+0000 7fa10fe00640 1 -- 172.21.15.32:0/3622187410 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0fc0b0070 con 0x7fa1100b4490 2024-04-17T10:04:03.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.575+0000 7fa10fe00640 1 -- 172.21.15.32:0/3622187410 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc0ab7f0 con 0x7fa1100b4490 2024-04-17T10:04:03.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.575+0000 7fa117e00640 1 -- 172.21.15.32:0/3622187410 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1100b4490 msgr2=0x7fa1100b5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:03.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.575+0000 7fa117e00640 1 --2- 172.21.15.32:0/3622187410 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1100b4490 0x7fa1100b5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0097c0 tx=0x7fa0fc01c410 comp rx=0 tx=0).stop 2024-04-17T10:04:03.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.575+0000 7fa117e00640 1 -- 172.21.15.32:0/3622187410 shutdown_connections 2024-04-17T10:04:03.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.575+0000 7fa117e00640 1 --2- 172.21.15.32:0/3622187410 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1100b4490 0x7fa1100b5040 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:03.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.575+0000 7fa117e00640 1 --2- 172.21.15.32:0/3622187410 >> v2:172.21.15.144:3300/0 conn(0x7fa11015e7c0 0x7fa11015eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:03.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.575+0000 7fa117e00640 1 -- 172.21.15.32:0/3622187410 >> 172.21.15.32:0/3622187410 conn(0x7fa1100b2510 msgr2=0x7fa1100b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:03.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.575+0000 7fa117e00640 1 -- 172.21.15.32:0/3622187410 shutdown_connections 2024-04-17T10:04:03.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.575+0000 7fa117e00640 1 -- 172.21.15.32:0/3622187410 wait complete. 2024-04-17T10:04:03.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.575+0000 7fa117e00640 1 Processor -- start 2024-04-17T10:04:03.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.575+0000 7fa117e00640 1 -- start start 2024-04-17T10:04:03.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa117e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1100aef90 0x7fa110156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:03.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa117e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1100b4490 0x7fa110154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:03.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa117e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa11015e7c0 0x7fa110155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:03.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa117e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa11006f1b0 con 0x7fa1100aef90 2024-04-17T10:04:03.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa117e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa11006f030 con 0x7fa11015e7c0 2024-04-17T10:04:03.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa117e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa11006f330 con 0x7fa1100b4490 2024-04-17T10:04:03.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa115400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1100aef90 0x7fa110156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:03.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa115400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1100aef90 0x7fa110156aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41062/0 (socket says 172.21.15.32:41062) 2024-04-17T10:04:03.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa115400640 1 -- 172.21.15.32:0/870522035 learned_addr learned my addr 172.21.15.32:0/870522035 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:03.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa115e00640 1 --2- 172.21.15.32:0/870522035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa11015e7c0 0x7fa110155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:03.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa114a00640 1 --2- 172.21.15.32:0/870522035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1100b4490 0x7fa110154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:03.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa115400640 1 -- 172.21.15.32:0/870522035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1100b4490 msgr2=0x7fa110154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:03.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa115400640 1 --2- 172.21.15.32:0/870522035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1100b4490 0x7fa110154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:03.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa115400640 1 -- 172.21.15.32:0/870522035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa11015e7c0 msgr2=0x7fa110155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:03.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa115400640 1 --2- 172.21.15.32:0/870522035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa11015e7c0 0x7fa110155270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:03.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa115400640 1 -- 172.21.15.32:0/870522035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa110155de0 con 0x7fa1100aef90 2024-04-17T10:04:03.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa114a00640 1 --2- 172.21.15.32:0/870522035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1100b4490 0x7fa110154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:04:03.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa115400640 1 --2- 172.21.15.32:0/870522035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1100aef90 0x7fa110156aa0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fa10400e930 tx=0x7fa10400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa10d600640 1 -- 172.21.15.32:0/870522035 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104017070 con 0x7fa1100aef90 2024-04-17T10:04:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa10d600640 1 -- 172.21.15.32:0/870522035 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa10400ca00 con 0x7fa1100aef90 2024-04-17T10:04:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa117e00640 1 -- 172.21.15.32:0/870522035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa110156080 con 0x7fa1100aef90 2024-04-17T10:04:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa10d600640 1 -- 172.21.15.32:0/870522035 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10400cdb0 con 0x7fa1100aef90 2024-04-17T10:04:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa117e00640 1 -- 172.21.15.32:0/870522035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa110156640 con 0x7fa1100aef90 2024-04-17T10:04:03.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa117e00640 1 -- 172.21.15.32:0/870522035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa110068a90 con 0x7fa1100aef90 2024-04-17T10:04:03.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.579+0000 7fa10d600640 1 -- 172.21.15.32:0/870522035 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa10401a070 con 0x7fa1100aef90 2024-04-17T10:04:03.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.583+0000 7fa10d600640 1 --2- 172.21.15.32:0/870522035 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa0f0068380 0x7fa0f006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:03.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.583+0000 7fa10d600640 1 -- 172.21.15.32:0/870522035 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7fa104086a50 con 0x7fa1100aef90 2024-04-17T10:04:03.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.583+0000 7fa114a00640 1 --2- 172.21.15.32:0/870522035 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa0f0068380 0x7fa0f006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:03.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.583+0000 7fa114a00640 1 --2- 172.21.15.32:0/870522035 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa0f0068380 0x7fa0f006a840 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0af000 tx=0x7fa0fc0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:03.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.587+0000 7fa10d600640 1 -- 172.21.15.32:0/870522035 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa104056950 con 0x7fa1100aef90 2024-04-17T10:04:03.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:03.787+0000 7fa117e00640 1 -- 172.21.15.32:0/870522035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fa110073060 con 0x7fa1100aef90 2024-04-17T10:04:04.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.079+0000 7fa10d600640 1 -- 172.21.15.32:0/870522035 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v100) ==== 162+0+0 (secure 0 0 0) 0x7fa10405ac50 con 0x7fa1100aef90 2024-04-17T10:04:04.080 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-17T10:04:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.083+0000 7fa117e00640 1 -- 172.21.15.32:0/870522035 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa0f0068380 msgr2=0x7fa0f006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.083+0000 7fa117e00640 1 --2- 172.21.15.32:0/870522035 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa0f0068380 0x7fa0f006a840 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0af000 tx=0x7fa0fc0a7000 comp rx=0 tx=0).stop 2024-04-17T10:04:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.083+0000 7fa117e00640 1 -- 172.21.15.32:0/870522035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1100aef90 msgr2=0x7fa110156aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.083+0000 7fa117e00640 1 --2- 172.21.15.32:0/870522035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1100aef90 0x7fa110156aa0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fa10400e930 tx=0x7fa10400edf0 comp rx=0 tx=0).stop 2024-04-17T10:04:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.083+0000 7fa117e00640 1 -- 172.21.15.32:0/870522035 shutdown_connections 2024-04-17T10:04:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.083+0000 7fa117e00640 1 --2- 172.21.15.32:0/870522035 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa0f0068380 0x7fa0f006a840 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.083+0000 7fa117e00640 1 --2- 172.21.15.32:0/870522035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1100b4490 0x7fa110154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.083+0000 7fa117e00640 1 --2- 172.21.15.32:0/870522035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa11015e7c0 0x7fa110155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.083+0000 7fa117e00640 1 --2- 172.21.15.32:0/870522035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1100aef90 0x7fa110156aa0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.083+0000 7fa117e00640 1 -- 172.21.15.32:0/870522035 >> 172.21.15.32:0/870522035 conn(0x7fa1100b2510 msgr2=0x7fa11015a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.083+0000 7fa117e00640 1 -- 172.21.15.32:0/870522035 shutdown_connections 2024-04-17T10:04:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.083+0000 7fa117e00640 1 -- 172.21.15.32:0/870522035 wait complete. 2024-04-17T10:04:04.107 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:04:04.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.251+0000 7fb44c400640 1 Processor -- start 2024-04-17T10:04:04.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.251+0000 7fb44c400640 1 -- start start 2024-04-17T10:04:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.251+0000 7fb44c400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb444144e90 0x7fb4440b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.251+0000 7fb44c400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4440b4de0 0x7fb44415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.251+0000 7fb44c400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb444072bb0 con 0x7fb44415bfe0 2024-04-17T10:04:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.251+0000 7fb44c400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb44415c430 con 0x7fb4440b4de0 2024-04-17T10:04:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.251+0000 7fb44c400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb444145270 con 0x7fb444144e90 2024-04-17T10:04:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.251+0000 7fb449000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4440b4de0 0x7fb44415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.251+0000 7fb44a400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb44415bfe0 0x7fb44406ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35996/0 (socket says 172.21.15.32:35996) 2024-04-17T10:04:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.251+0000 7fb44a400640 1 -- 172.21.15.32:0/1404464197 learned_addr learned my addr 172.21.15.32:0/1404464197 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:04.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.251+0000 7fb449a00640 1 --2- 172.21.15.32:0/1404464197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb444144e90 0x7fb4440b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:04.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.251+0000 7fb449000640 1 -- 172.21.15.32:0/1404464197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb444144e90 msgr2=0x7fb4440b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:04.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.251+0000 7fb449000640 1 --2- 172.21.15.32:0/1404464197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb444144e90 0x7fb4440b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:04.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.251+0000 7fb449000640 1 -- 172.21.15.32:0/1404464197 >> v1:172.21.15.32:6789/0 conn(0x7fb44415bfe0 legacy=0x7fb44406ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:04.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.251+0000 7fb449000640 1 -- 172.21.15.32:0/1404464197 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4440b3dc0 con 0x7fb4440b4de0 2024-04-17T10:04:04.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.255+0000 7fb449000640 1 --2- 172.21.15.32:0/1404464197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4440b4de0 0x7fb44415baa0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fb4300097f0 tx=0x7fb43001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8fe0e781ae2a6d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:04.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.255+0000 7fb443e00640 1 -- 172.21.15.32:0/1404464197 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4300a7070 con 0x7fb4440b4de0 2024-04-17T10:04:04.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.255+0000 7fb443e00640 1 -- 172.21.15.32:0/1404464197 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4300b0070 con 0x7fb4440b4de0 2024-04-17T10:04:04.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.255+0000 7fb443e00640 1 -- 172.21.15.32:0/1404464197 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4300ab890 con 0x7fb4440b4de0 2024-04-17T10:04:04.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.255+0000 7fb44c400640 1 -- 172.21.15.32:0/1404464197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4440b4de0 msgr2=0x7fb44415baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:04.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.255+0000 7fb44c400640 1 --2- 172.21.15.32:0/1404464197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4440b4de0 0x7fb44415baa0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fb4300097f0 tx=0x7fb43001c470 comp rx=0 tx=0).stop 2024-04-17T10:04:04.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.255+0000 7fb44c400640 1 -- 172.21.15.32:0/1404464197 shutdown_connections 2024-04-17T10:04:04.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.255+0000 7fb44c400640 1 --2- 172.21.15.32:0/1404464197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4440b4de0 0x7fb44415baa0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:04.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.255+0000 7fb44c400640 1 --2- 172.21.15.32:0/1404464197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb444144e90 0x7fb4440b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:04.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.255+0000 7fb44c400640 1 -- 172.21.15.32:0/1404464197 >> 172.21.15.32:0/1404464197 conn(0x7fb44406ba30 msgr2=0x7fb44406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:04.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.255+0000 7fb44c400640 1 -- 172.21.15.32:0/1404464197 shutdown_connections 2024-04-17T10:04:04.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.255+0000 7fb44c400640 1 -- 172.21.15.32:0/1404464197 wait complete. 2024-04-17T10:04:04.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb44c400640 1 Processor -- start 2024-04-17T10:04:04.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb44c400640 1 -- start start 2024-04-17T10:04:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb44c400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4440b4de0 0x7fb44413dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb44c400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb444144e90 0x7fb44415c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb44c400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb44415bfe0 0x7fb44415cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb44c400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb444072670 con 0x7fb4440b4de0 2024-04-17T10:04:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb44c400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb44406f880 con 0x7fb444144e90 2024-04-17T10:04:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb44c400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb4440727f0 con 0x7fb44415bfe0 2024-04-17T10:04:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb449a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4440b4de0 0x7fb44413dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb449a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4440b4de0 0x7fb44413dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41088/0 (socket says 172.21.15.32:41088) 2024-04-17T10:04:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb449a00640 1 -- 172.21.15.32:0/2759786477 learned_addr learned my addr 172.21.15.32:0/2759786477 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb449000640 1 --2- 172.21.15.32:0/2759786477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb444144e90 0x7fb44415c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb44a400640 1 --2- 172.21.15.32:0/2759786477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb44415bfe0 0x7fb44415cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb449a00640 1 -- 172.21.15.32:0/2759786477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb44415bfe0 msgr2=0x7fb44415cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb449a00640 1 --2- 172.21.15.32:0/2759786477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb44415bfe0 0x7fb44415cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb449a00640 1 -- 172.21.15.32:0/2759786477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb444144e90 msgr2=0x7fb44415c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb449a00640 1 --2- 172.21.15.32:0/2759786477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb444144e90 0x7fb44415c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb449a00640 1 -- 172.21.15.32:0/2759786477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44415d260 con 0x7fb4440b4de0 2024-04-17T10:04:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb44a400640 1 --2- 172.21.15.32:0/2759786477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb44415bfe0 0x7fb44415cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb449a00640 1 --2- 172.21.15.32:0/2759786477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4440b4de0 0x7fb44413dc90 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fb43800e930 tx=0x7fb43800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb441600640 1 -- 172.21.15.32:0/2759786477 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb438017070 con 0x7fb4440b4de0 2024-04-17T10:04:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb441600640 1 -- 172.21.15.32:0/2759786477 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb43800ca00 con 0x7fb4440b4de0 2024-04-17T10:04:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb441600640 1 -- 172.21.15.32:0/2759786477 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43800cdb0 con 0x7fb4440b4de0 2024-04-17T10:04:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb44c400640 1 -- 172.21.15.32:0/2759786477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb44415d540 con 0x7fb4440b4de0 2024-04-17T10:04:04.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb44c400640 1 -- 172.21.15.32:0/2759786477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb44415db00 con 0x7fb4440b4de0 2024-04-17T10:04:04.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.259+0000 7fb44c400640 1 -- 172.21.15.32:0/2759786477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb444068a90 con 0x7fb4440b4de0 2024-04-17T10:04:04.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.263+0000 7fb441600640 1 -- 172.21.15.32:0/2759786477 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb43801a070 con 0x7fb4440b4de0 2024-04-17T10:04:04.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.263+0000 7fb441600640 1 --2- 172.21.15.32:0/2759786477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb424068380 0x7fb42406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:04.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.263+0000 7fb441600640 1 -- 172.21.15.32:0/2759786477 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7fb438086ec0 con 0x7fb4440b4de0 2024-04-17T10:04:04.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.263+0000 7fb449000640 1 --2- 172.21.15.32:0/2759786477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb424068380 0x7fb42406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:04.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.263+0000 7fb449000640 1 --2- 172.21.15.32:0/2759786477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb424068380 0x7fb42406a840 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fb4300af000 tx=0x7fb4300a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:04.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.267+0000 7fb441600640 1 -- 172.21.15.32:0/2759786477 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fb438057410 con 0x7fb4440b4de0 2024-04-17T10:04:04.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.467+0000 7fb44c400640 1 -- 172.21.15.32:0/2759786477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4440af880 con 0x7fb4440b4de0 2024-04-17T10:04:04.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.467+0000 7fb441600640 1 -- 172.21.15.32:0/2759786477 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5202 (secure 0 0 0) 0x7fb438056d10 con 0x7fb4440b4de0 2024-04-17T10:04:04.464 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:04.465 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":101,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7586,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3601627312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3601627312},{"type":"v1","addr":"172.21.15.144:6834","nonce":3601627312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2382359917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2382359917},{"type":"v1","addr":"172.21.15.144:6835","nonce":2382359917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1857552160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1857552160},{"type":"v1","addr":"172.21.15.32:6835","nonce":1857552160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:03:37.619785+0000","modified":"2024-04-17T10:04:04.088565+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9615},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9615":{"gid":9615,"name":"a","rank":0,"incarnation":101,"state":"up:replay","state_seq":1,"addr":"172.21.15.32:6837/578643390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":578643390},{"type":"v1","addr":"172.21.15.32:6837","nonce":578643390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-17T10:04:04.465 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 101 2024-04-17T10:04:04.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.467+0000 7fb44c400640 1 -- 172.21.15.32:0/2759786477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb424068380 msgr2=0x7fb42406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:04.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.467+0000 7fb44c400640 1 --2- 172.21.15.32:0/2759786477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb424068380 0x7fb42406a840 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fb4300af000 tx=0x7fb4300a7000 comp rx=0 tx=0).stop 2024-04-17T10:04:04.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.467+0000 7fb44c400640 1 -- 172.21.15.32:0/2759786477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4440b4de0 msgr2=0x7fb44413dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:04.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.467+0000 7fb44c400640 1 --2- 172.21.15.32:0/2759786477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4440b4de0 0x7fb44413dc90 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fb43800e930 tx=0x7fb43800edf0 comp rx=0 tx=0).stop 2024-04-17T10:04:04.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.467+0000 7fb44c400640 1 -- 172.21.15.32:0/2759786477 shutdown_connections 2024-04-17T10:04:04.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.467+0000 7fb44c400640 1 --2- 172.21.15.32:0/2759786477 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb424068380 0x7fb42406a840 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:04.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.467+0000 7fb44c400640 1 --2- 172.21.15.32:0/2759786477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb44415bfe0 0x7fb44415cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:04.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.467+0000 7fb44c400640 1 --2- 172.21.15.32:0/2759786477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb444144e90 0x7fb44415c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:04.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.467+0000 7fb44c400640 1 --2- 172.21.15.32:0/2759786477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4440b4de0 0x7fb44413dc90 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:04.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.467+0000 7fb44c400640 1 -- 172.21.15.32:0/2759786477 >> 172.21.15.32:0/2759786477 conn(0x7fb44406ba30 msgr2=0x7fb44406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:04.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.467+0000 7fb44c400640 1 -- 172.21.15.32:0/2759786477 shutdown_connections 2024-04-17T10:04:04.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:04.467+0000 7fb44c400640 1 -- 172.21.15.32:0/2759786477 wait complete. 2024-04-17T10:04:04.888 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 101, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:03:37.619785+0000', 'modified': '2024-04-17T10:04:04.088565+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9615}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9615': {'gid': 9615, 'name': 'a', 'rank': 0, 'incarnation': 101, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.32:6837/578643390', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6836', 'nonce': 578643390}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 578643390}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-17T10:04:04.888 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9615:up:replay 2024-04-17T10:04:05.890 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:04:06.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.043+0000 7f57b9200640 1 Processor -- start 2024-04-17T10:04:06.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.043+0000 7f57b9200640 1 -- start start 2024-04-17T10:04:06.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.043+0000 7f57b9200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f57b415eb60 0x7f57b40b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:06.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.043+0000 7f57b9200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57b409b100 0x7f57b40cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:06.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.043+0000 7f57b9200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57b40d0c70 con 0x7f57b40b4970 2024-04-17T10:04:06.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.043+0000 7f57b9200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f57b415ef40 con 0x7f57b415eb60 2024-04-17T10:04:06.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.043+0000 7f57b9200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f57b406d480 con 0x7f57b409b100 2024-04-17T10:04:06.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.047+0000 7f57b3400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f57b415eb60 0x7f57b40b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:06.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.047+0000 7f57aaa00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f57b40b4970 0x7f57b409a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36008/0 (socket says 172.21.15.32:36008) 2024-04-17T10:04:06.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.047+0000 7f57aaa00640 1 -- 172.21.15.32:0/1805719835 learned_addr learned my addr 172.21.15.32:0/1805719835 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:06.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.047+0000 7f57b3e00640 1 --2- 172.21.15.32:0/1805719835 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57b409b100 0x7f57b40cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:06.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.047+0000 7f57b3400640 1 -- 172.21.15.32:0/1805719835 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57b409b100 msgr2=0x7f57b40cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:06.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.047+0000 7f57b3400640 1 --2- 172.21.15.32:0/1805719835 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57b409b100 0x7f57b40cc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:06.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.047+0000 7f57b3400640 1 -- 172.21.15.32:0/1805719835 >> v1:172.21.15.32:6789/0 conn(0x7f57b40b4970 legacy=0x7f57b409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:06.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.047+0000 7f57b3400640 1 -- 172.21.15.32:0/1805719835 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b40d0f50 con 0x7f57b415eb60 2024-04-17T10:04:06.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.047+0000 7f57b2a00640 1 -- 172.21.15.32:0/1805719835 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 654219350 0 0) 0x7f57b40d0c70 con 0x7f57b40b4970 2024-04-17T10:04:06.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.047+0000 7f57b3400640 1 --2- 172.21.15.32:0/1805719835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57b415eb60 0x7f57b40b13b0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f57a40097f0 tx=0x7f57a401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b903bac2668785 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:06.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.047+0000 7f57b2a00640 1 -- 172.21.15.32:0/1805719835 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a40b1070 con 0x7f57b415eb60 2024-04-17T10:04:06.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.047+0000 7f57b2a00640 1 -- 172.21.15.32:0/1805719835 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57a40ac600 con 0x7f57b415eb60 2024-04-17T10:04:06.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.047+0000 7f57b2a00640 1 -- 172.21.15.32:0/1805719835 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a40ac9d0 con 0x7f57b415eb60 2024-04-17T10:04:06.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57b9200640 1 -- 172.21.15.32:0/1805719835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57b415eb60 msgr2=0x7f57b40b13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:06.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57b9200640 1 --2- 172.21.15.32:0/1805719835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57b415eb60 0x7f57b40b13b0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f57a40097f0 tx=0x7f57a401c3e0 comp rx=0 tx=0).stop 2024-04-17T10:04:06.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57b9200640 1 -- 172.21.15.32:0/1805719835 shutdown_connections 2024-04-17T10:04:06.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57b9200640 1 --2- 172.21.15.32:0/1805719835 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57b409b100 0x7f57b40cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:06.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57b9200640 1 --2- 172.21.15.32:0/1805719835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57b415eb60 0x7f57b40b13b0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:06.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57b9200640 1 -- 172.21.15.32:0/1805719835 >> 172.21.15.32:0/1805719835 conn(0x7f57b40b3e50 msgr2=0x7f57b40b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:06.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57b9200640 1 -- 172.21.15.32:0/1805719835 shutdown_connections 2024-04-17T10:04:06.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57b9200640 1 -- 172.21.15.32:0/1805719835 wait complete. 2024-04-17T10:04:06.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57b9200640 1 Processor -- start 2024-04-17T10:04:06.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57b9200640 1 -- start start 2024-04-17T10:04:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57b9200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57b409b100 0x7f57b41470c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57b9200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57b40b4970 0x7f57b40c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57b9200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57b415eb60 0x7f57b40b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57b9200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f57b406e2a0 con 0x7f57b40b4970 2024-04-17T10:04:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57b9200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f57b406e120 con 0x7f57b409b100 2024-04-17T10:04:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57b9200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f57b406e420 con 0x7f57b415eb60 2024-04-17T10:04:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57aaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57b40b4970 0x7f57b40c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57aaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57b40b4970 0x7f57b40c7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41110/0 (socket says 172.21.15.32:41110) 2024-04-17T10:04:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57aaa00640 1 -- 172.21.15.32:0/3118836192 learned_addr learned my addr 172.21.15.32:0/3118836192 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57b3400640 1 --2- 172.21.15.32:0/3118836192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57b409b100 0x7f57b41470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57b3e00640 1 --2- 172.21.15.32:0/3118836192 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57b415eb60 0x7f57b40b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:06.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57aaa00640 1 -- 172.21.15.32:0/3118836192 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57b415eb60 msgr2=0x7f57b40b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:06.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57aaa00640 1 --2- 172.21.15.32:0/3118836192 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57b415eb60 0x7f57b40b6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:06.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57aaa00640 1 -- 172.21.15.32:0/3118836192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57b409b100 msgr2=0x7f57b41470c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:06.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57aaa00640 1 --2- 172.21.15.32:0/3118836192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57b409b100 0x7f57b41470c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:06.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57aaa00640 1 -- 172.21.15.32:0/3118836192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b40c86a0 con 0x7f57b40b4970 2024-04-17T10:04:06.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.051+0000 7f57b3400640 1 --2- 172.21.15.32:0/3118836192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57b409b100 0x7f57b41470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:06.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.055+0000 7f57aaa00640 1 --2- 172.21.15.32:0/3118836192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57b40b4970 0x7f57b40c7ca0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f579c00c890 tx=0x7f579c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:06.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.055+0000 7f57abe00640 1 -- 172.21.15.32:0/3118836192 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c018070 con 0x7f57b40b4970 2024-04-17T10:04:06.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.055+0000 7f57abe00640 1 -- 172.21.15.32:0/3118836192 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f579c00a640 con 0x7f57b40b4970 2024-04-17T10:04:06.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.055+0000 7f57b9200640 1 -- 172.21.15.32:0/3118836192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b40c8980 con 0x7f57b40b4970 2024-04-17T10:04:06.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.055+0000 7f57abe00640 1 -- 172.21.15.32:0/3118836192 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c00aa30 con 0x7f57b40b4970 2024-04-17T10:04:06.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.055+0000 7f57b9200640 1 -- 172.21.15.32:0/3118836192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b40c8f40 con 0x7f57b40b4970 2024-04-17T10:04:06.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.055+0000 7f57b9200640 1 -- 172.21.15.32:0/3118836192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57b404fd00 con 0x7f57b40b4970 2024-04-17T10:04:06.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.055+0000 7f57abe00640 1 -- 172.21.15.32:0/3118836192 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f579c01b070 con 0x7f57b40b4970 2024-04-17T10:04:06.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.055+0000 7f57abe00640 1 --2- 172.21.15.32:0/3118836192 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f57900683b0 0x7f579006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.055+0000 7f57abe00640 1 -- 172.21.15.32:0/3118836192 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7f579c085fe0 con 0x7f57b40b4970 2024-04-17T10:04:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.055+0000 7f57b3400640 1 --2- 172.21.15.32:0/3118836192 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f57900683b0 0x7f579006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.055+0000 7f57b3400640 1 --2- 172.21.15.32:0/3118836192 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f57900683b0 0x7f579006a870 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f57a40ae000 tx=0x7f57a40a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.059+0000 7f57abe00640 1 -- 172.21.15.32:0/3118836192 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f579c055f10 con 0x7f57b40b4970 2024-04-17T10:04:06.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.263+0000 7f57b9200640 1 -- 172.21.15.32:0/3118836192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57b40b82e0 con 0x7f57b40b4970 2024-04-17T10:04:06.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.263+0000 7f57abe00640 1 -- 172.21.15.32:0/3118836192 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5202 (secure 0 0 0) 0x7f579c05a210 con 0x7f57b40b4970 2024-04-17T10:04:06.262 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:06.262 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":103,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7586,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3601627312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3601627312},{"type":"v1","addr":"172.21.15.144:6834","nonce":3601627312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2382359917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2382359917},{"type":"v1","addr":"172.21.15.144:6835","nonce":2382359917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1857552160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1857552160},{"type":"v1","addr":"172.21.15.32:6835","nonce":1857552160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:03:37.619785+0000","modified":"2024-04-17T10:04:05.094241+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9615},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9615":{"gid":9615,"name":"a","rank":0,"incarnation":101,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.32:6837/578643390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":578643390},{"type":"v1","addr":"172.21.15.32:6837","nonce":578643390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-17T10:04:06.263 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 103 2024-04-17T10:04:06.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.267+0000 7f57b9200640 1 -- 172.21.15.32:0/3118836192 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f57900683b0 msgr2=0x7f579006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:06.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.267+0000 7f57b9200640 1 --2- 172.21.15.32:0/3118836192 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f57900683b0 0x7f579006a870 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f57a40ae000 tx=0x7f57a40a7000 comp rx=0 tx=0).stop 2024-04-17T10:04:06.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.267+0000 7f57b9200640 1 -- 172.21.15.32:0/3118836192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57b40b4970 msgr2=0x7f57b40c7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:06.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.267+0000 7f57b9200640 1 --2- 172.21.15.32:0/3118836192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57b40b4970 0x7f57b40c7ca0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f579c00c890 tx=0x7f579c00cd50 comp rx=0 tx=0).stop 2024-04-17T10:04:06.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.267+0000 7f57b9200640 1 -- 172.21.15.32:0/3118836192 shutdown_connections 2024-04-17T10:04:06.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.267+0000 7f57b9200640 1 --2- 172.21.15.32:0/3118836192 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f57900683b0 0x7f579006a870 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:06.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.267+0000 7f57b9200640 1 --2- 172.21.15.32:0/3118836192 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57b415eb60 0x7f57b40b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:06.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.267+0000 7f57b9200640 1 --2- 172.21.15.32:0/3118836192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57b40b4970 0x7f57b40c7ca0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:06.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.267+0000 7f57b9200640 1 --2- 172.21.15.32:0/3118836192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57b409b100 0x7f57b41470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:06.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.267+0000 7f57b9200640 1 -- 172.21.15.32:0/3118836192 >> 172.21.15.32:0/3118836192 conn(0x7f57b40b3e50 msgr2=0x7f57b40b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:06.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.267+0000 7f57b9200640 1 -- 172.21.15.32:0/3118836192 shutdown_connections 2024-04-17T10:04:06.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:06.267+0000 7f57b9200640 1 -- 172.21.15.32:0/3118836192 wait complete. 2024-04-17T10:04:06.686 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 103, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:03:37.619785+0000', 'modified': '2024-04-17T10:04:05.094241+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9615}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9615': {'gid': 9615, 'name': 'a', 'rank': 0, 'incarnation': 101, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.32:6837/578643390', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6836', 'nonce': 578643390}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 578643390}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-17T10:04:06.686 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9615:up:rejoin 2024-04-17T10:04:07.688 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:04:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.843+0000 7f6b7ce00640 1 Processor -- start 2024-04-17T10:04:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.843+0000 7f6b7ce00640 1 -- start start 2024-04-17T10:04:07.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.843+0000 7f6b7ce00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b7815eb10 0x7f6b7815f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:07.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.843+0000 7f6b7ce00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b780d0f20 0x7f6b781603c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:07.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.843+0000 7f6b7ce00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b780b2ac0 con 0x7f6b780b2550 2024-04-17T10:04:07.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.843+0000 7f6b7ce00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f6b7815ff10 con 0x7f6b7815eb10 2024-04-17T10:04:07.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.843+0000 7f6b7ce00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6b780b2da0 con 0x7f6b780d0f20 2024-04-17T10:04:07.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.843+0000 7f6b75e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b7815eb10 0x7f6b7815f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:07.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.843+0000 7f6b75e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b7815eb10 0x7f6b7815f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41112/0 (socket says 172.21.15.32:41112) 2024-04-17T10:04:07.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.843+0000 7f6b75e00640 1 -- 172.21.15.32:0/2554856805 learned_addr learned my addr 172.21.15.32:0/2554856805 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:07.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.843+0000 7f6b76800640 1 --2- 172.21.15.32:0/2554856805 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b780d0f20 0x7f6b781603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:07.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.843+0000 7f6b75e00640 1 -- 172.21.15.32:0/2554856805 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b780d0f20 msgr2=0x7f6b781603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:07.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.843+0000 7f6b75e00640 1 --2- 172.21.15.32:0/2554856805 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b780d0f20 0x7f6b781603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:07.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.843+0000 7f6b75e00640 1 -- 172.21.15.32:0/2554856805 >> v1:172.21.15.144:6789/0 conn(0x7f6b780b2550 legacy=0x7f6b780d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:07.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.843+0000 7f6b75e00640 1 -- 172.21.15.32:0/2554856805 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7806bb60 con 0x7f6b7815eb10 2024-04-17T10:04:07.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.847+0000 7f6b75e00640 1 --2- 172.21.15.32:0/2554856805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b7815eb10 0x7f6b7815f9d0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009870 tx=0x7f6b6801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789bae81874e79e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:07.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.847+0000 7f6b74a00640 1 -- 172.21.15.32:0/2554856805 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b680b1070 con 0x7f6b7815eb10 2024-04-17T10:04:07.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.847+0000 7f6b74a00640 1 -- 172.21.15.32:0/2554856805 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b680a8050 con 0x7f6b7815eb10 2024-04-17T10:04:07.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.847+0000 7f6b74a00640 1 -- 172.21.15.32:0/2554856805 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b680ac860 con 0x7f6b7815eb10 2024-04-17T10:04:07.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.847+0000 7f6b7ce00640 1 -- 172.21.15.32:0/2554856805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b7815eb10 msgr2=0x7f6b7815f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:07.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.847+0000 7f6b7ce00640 1 --2- 172.21.15.32:0/2554856805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b7815eb10 0x7f6b7815f9d0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009870 tx=0x7f6b6801c4d0 comp rx=0 tx=0).stop 2024-04-17T10:04:07.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.847+0000 7f6b7ce00640 1 -- 172.21.15.32:0/2554856805 shutdown_connections 2024-04-17T10:04:07.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.847+0000 7f6b7ce00640 1 --2- 172.21.15.32:0/2554856805 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b780d0f20 0x7f6b781603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:07.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.847+0000 7f6b7ce00640 1 --2- 172.21.15.32:0/2554856805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b7815eb10 0x7f6b7815f9d0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:07.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.847+0000 7f6b7ce00640 1 -- 172.21.15.32:0/2554856805 >> 172.21.15.32:0/2554856805 conn(0x7f6b7809a9f0 msgr2=0x7f6b780b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:07.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.847+0000 7f6b7ce00640 1 -- 172.21.15.32:0/2554856805 shutdown_connections 2024-04-17T10:04:07.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.847+0000 7f6b7ce00640 1 -- 172.21.15.32:0/2554856805 wait complete. 2024-04-17T10:04:07.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.847+0000 7f6b7ce00640 1 Processor -- start 2024-04-17T10:04:07.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.847+0000 7f6b7ce00640 1 -- start start 2024-04-17T10:04:07.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.851+0000 7f6b7ce00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b780b2550 0x7f6b78153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:07.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.851+0000 7f6b7ce00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b780d0f20 0x7f6b780c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:07.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.851+0000 7f6b7ce00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b7815eb10 0x7f6b780c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:07.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.851+0000 7f6b7ce00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7816bbf0 con 0x7f6b780d0f20 2024-04-17T10:04:07.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.851+0000 7f6b7ce00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7816ba70 con 0x7f6b7815eb10 2024-04-17T10:04:07.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.851+0000 7f6b7ce00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6b7816bd70 con 0x7f6b780b2550 2024-04-17T10:04:07.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.851+0000 7f6b75400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b780d0f20 0x7f6b780c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:07.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.851+0000 7f6b75400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b780d0f20 0x7f6b780c85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41116/0 (socket says 172.21.15.32:41116) 2024-04-17T10:04:07.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.851+0000 7f6b75400640 1 -- 172.21.15.32:0/264337021 learned_addr learned my addr 172.21.15.32:0/264337021 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:07.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.851+0000 7f6b75e00640 1 --2- 172.21.15.32:0/264337021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b780b2550 0x7f6b78153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:07.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.851+0000 7f6b76800640 1 --2- 172.21.15.32:0/264337021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b7815eb10 0x7f6b780c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:07.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.851+0000 7f6b75400640 1 -- 172.21.15.32:0/264337021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b780b2550 msgr2=0x7f6b78153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:07.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.851+0000 7f6b75400640 1 --2- 172.21.15.32:0/264337021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b780b2550 0x7f6b78153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:07.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.851+0000 7f6b75400640 1 -- 172.21.15.32:0/264337021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b7815eb10 msgr2=0x7f6b780c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:07.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.851+0000 7f6b75400640 1 --2- 172.21.15.32:0/264337021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b7815eb10 0x7f6b780c8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:07.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.851+0000 7f6b75400640 1 -- 172.21.15.32:0/264337021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b781546b0 con 0x7f6b780d0f20 2024-04-17T10:04:07.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.851+0000 7f6b75400640 1 --2- 172.21.15.32:0/264337021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b780d0f20 0x7f6b780c85a0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400c450 tx=0x7f6b6400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:07.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.851+0000 7f6b6e000640 1 -- 172.21.15.32:0/264337021 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64016020 con 0x7f6b780d0f20 2024-04-17T10:04:07.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.851+0000 7f6b6e000640 1 -- 172.21.15.32:0/264337021 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b640052d0 con 0x7f6b780d0f20 2024-04-17T10:04:07.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.851+0000 7f6b6e000640 1 -- 172.21.15.32:0/264337021 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64007260 con 0x7f6b780d0f20 2024-04-17T10:04:07.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.851+0000 7f6b7ce00640 1 -- 172.21.15.32:0/264337021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b780cb190 con 0x7f6b780d0f20 2024-04-17T10:04:07.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.851+0000 7f6b7ce00640 1 -- 172.21.15.32:0/264337021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b780b4a60 con 0x7f6b780d0f20 2024-04-17T10:04:07.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.851+0000 7f6b7ce00640 1 -- 172.21.15.32:0/264337021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b780ce550 con 0x7f6b780d0f20 2024-04-17T10:04:07.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.855+0000 7f6b6e000640 1 -- 172.21.15.32:0/264337021 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6b64007400 con 0x7f6b780d0f20 2024-04-17T10:04:07.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.855+0000 7f6b6e000640 1 --2- 172.21.15.32:0/264337021 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6b58068380 0x7f6b5806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:07.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.855+0000 7f6b6e000640 1 -- 172.21.15.32:0/264337021 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7f6b640862f0 con 0x7f6b780d0f20 2024-04-17T10:04:07.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.855+0000 7f6b75e00640 1 --2- 172.21.15.32:0/264337021 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6b58068380 0x7f6b5806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:07.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.855+0000 7f6b75e00640 1 --2- 172.21.15.32:0/264337021 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6b58068380 0x7f6b5806a840 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6b680ae000 tx=0x7f6b680a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:07.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:07.859+0000 7f6b6e000640 1 -- 172.21.15.32:0/264337021 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f6b780ce550 con 0x7f6b780d0f20 2024-04-17T10:04:08.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.059+0000 7f6b7ce00640 1 -- 172.21.15.32:0/264337021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b78068a90 con 0x7f6b780d0f20 2024-04-17T10:04:08.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.059+0000 7f6b6e000640 1 -- 172.21.15.32:0/264337021 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5209 (secure 0 0 0) 0x7f6b64020020 con 0x7f6b780d0f20 2024-04-17T10:04:08.057 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:08.057 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":104,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7586,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3601627312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3601627312},{"type":"v1","addr":"172.21.15.144:6834","nonce":3601627312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2382359917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2382359917},{"type":"v1","addr":"172.21.15.144:6835","nonce":2382359917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1857552160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1857552160},{"type":"v1","addr":"172.21.15.32:6835","nonce":1857552160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:03:37.619785+0000","modified":"2024-04-17T10:04:07.100059+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9615},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9615":{"gid":9615,"name":"a","rank":0,"incarnation":101,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6837/578643390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":578643390},{"type":"v1","addr":"172.21.15.32:6837","nonce":578643390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9615,"qdb_cluster":[9615]},"id":8}]} 2024-04-17T10:04:08.057 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 104 2024-04-17T10:04:08.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.059+0000 7f6b7ce00640 1 -- 172.21.15.32:0/264337021 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6b58068380 msgr2=0x7f6b5806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:08.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.059+0000 7f6b7ce00640 1 --2- 172.21.15.32:0/264337021 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6b58068380 0x7f6b5806a840 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6b680ae000 tx=0x7f6b680a7000 comp rx=0 tx=0).stop 2024-04-17T10:04:08.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.059+0000 7f6b7ce00640 1 -- 172.21.15.32:0/264337021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b780d0f20 msgr2=0x7f6b780c85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:08.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.059+0000 7f6b7ce00640 1 --2- 172.21.15.32:0/264337021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b780d0f20 0x7f6b780c85a0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400c450 tx=0x7f6b6400c910 comp rx=0 tx=0).stop 2024-04-17T10:04:08.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.059+0000 7f6b7ce00640 1 -- 172.21.15.32:0/264337021 shutdown_connections 2024-04-17T10:04:08.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.059+0000 7f6b7ce00640 1 --2- 172.21.15.32:0/264337021 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6b58068380 0x7f6b5806a840 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.059+0000 7f6b7ce00640 1 --2- 172.21.15.32:0/264337021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b7815eb10 0x7f6b780c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.059+0000 7f6b7ce00640 1 --2- 172.21.15.32:0/264337021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b780d0f20 0x7f6b780c85a0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.059+0000 7f6b7ce00640 1 --2- 172.21.15.32:0/264337021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b780b2550 0x7f6b78153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.059+0000 7f6b7ce00640 1 -- 172.21.15.32:0/264337021 >> 172.21.15.32:0/264337021 conn(0x7f6b7809a9f0 msgr2=0x7f6b780ae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:08.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.059+0000 7f6b7ce00640 1 -- 172.21.15.32:0/264337021 shutdown_connections 2024-04-17T10:04:08.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.059+0000 7f6b7ce00640 1 -- 172.21.15.32:0/264337021 wait complete. 2024-04-17T10:04:08.481 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:03:37.619785+0000', 'modified': '2024-04-17T10:04:07.100059+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9615}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9615': {'gid': 9615, 'name': 'a', 'rank': 0, 'incarnation': 101, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.32:6837/578643390', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6836', 'nonce': 578643390}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 578643390}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9615, 'qdb_cluster': [9615]} 2024-04-17T10:04:08.482 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T10:04:08.482 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-17T10:04:08.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.627+0000 7f1188e00640 1 Processor -- start 2024-04-17T10:04:08.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.627+0000 7f1188e00640 1 -- start start 2024-04-17T10:04:08.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.627+0000 7f1188e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11840ce020 0x7f1184164100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.627+0000 7f1188e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11840d0600 con 0x7f11840d0f40 2024-04-17T10:04:08.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.627+0000 7f1188e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f118415cd00 con 0x7f11840d0b60 2024-04-17T10:04:08.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.627+0000 7f1188e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f11840b3c90 con 0x7f11840ce020 2024-04-17T10:04:08.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.627+0000 7f1183e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f11840d0f40 0x7f1184166650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36018/0 (socket says 172.21.15.32:36018) 2024-04-17T10:04:08.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.627+0000 7f1183e00640 1 -- 172.21.15.32:0/1531479567 learned_addr learned my addr 172.21.15.32:0/1531479567 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:08.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.627+0000 7f117be00640 1 --2- 172.21.15.32:0/1531479567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11840ce020 0x7f1184164100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.627+0000 7f1182a00640 1 -- 172.21.15.32:0/1531479567 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2723508645 0 0) 0x7f11840d0600 con 0x7f11840d0f40 2024-04-17T10:04:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.627+0000 7f1182a00640 1 -- 172.21.15.32:0/1531479567 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1168003530 con 0x7f11840d0f40 2024-04-17T10:04:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.627+0000 7f1182a00640 1 -- 172.21.15.32:0/1531479567 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1512546836 0 0) 0x7f118415cd00 con 0x7f11840d0b60 2024-04-17T10:04:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f1182a00640 1 -- 172.21.15.32:0/1531479567 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11840d0600 con 0x7f11840d0b60 2024-04-17T10:04:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f1182a00640 1 -- 172.21.15.32:0/1531479567 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1476864554 0 0) 0x7f1168003530 con 0x7f11840d0f40 2024-04-17T10:04:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f1182a00640 1 -- 172.21.15.32:0/1531479567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11840ce020 msgr2=0x7f1184164100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f1182a00640 1 --2- 172.21.15.32:0/1531479567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11840ce020 0x7f1184164100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f1182a00640 1 -- 172.21.15.32:0/1531479567 >> v1:172.21.15.144:6789/0 conn(0x7f11840d0b60 legacy=0x7f11840cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f1182a00640 1 -- 172.21.15.32:0/1531479567 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118415cfe0 con 0x7f11840d0f40 2024-04-17T10:04:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f1182a00640 1 -- 172.21.15.32:0/1531479567 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f117c001a00 con 0x7f11840d0f40 2024-04-17T10:04:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f1182a00640 1 -- 172.21.15.32:0/1531479567 >> v1:172.21.15.32:6789/0 conn(0x7f11840d0f40 legacy=0x7f1184166650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f1182a00640 1 --2- 172.21.15.32:0/1531479567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11840ce020 0x7f1168003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f1182a00640 1 --2- 172.21.15.32:0/1531479567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11680040e0 0x7f1168004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f1182a00640 1 --2- 172.21.15.32:0/1531479567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1168004d50 0x7f11680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f1182a00640 1 -- 172.21.15.32:0/1531479567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f11840b3c90 con 0x7f11680040e0 2024-04-17T10:04:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f1182a00640 1 -- 172.21.15.32:0/1531479567 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f116800bab0 con 0x7f1168004d50 2024-04-17T10:04:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f1182a00640 1 -- 172.21.15.32:0/1531479567 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f116800bcb0 con 0x7f11840ce020 2024-04-17T10:04:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f117be00640 1 --2- 172.21.15.32:0/1531479567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11680040e0 0x7f1168004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f1183400640 1 --2- 172.21.15.32:0/1531479567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11840ce020 0x7f1168003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f1183e00640 1 --2- 172.21.15.32:0/1531479567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1168004d50 0x7f11680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f117be00640 1 -- 172.21.15.32:0/1531479567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11840ce020 msgr2=0x7f1168003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f117be00640 1 --2- 172.21.15.32:0/1531479567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11840ce020 0x7f1168003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f117be00640 1 -- 172.21.15.32:0/1531479567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1168004d50 msgr2=0x7f11680051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f117be00640 1 --2- 172.21.15.32:0/1531479567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1168004d50 0x7f11680051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f117be00640 1 -- 172.21.15.32:0/1531479567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1168003530 con 0x7f11680040e0 2024-04-17T10:04:08.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f117be00640 1 --2- 172.21.15.32:0/1531479567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11680040e0 0x7f1168004810 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f116c009ae0 tx=0x7f116c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b45fd5132f1114 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:08.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f1182a00640 1 -- 172.21.15.32:0/1531479567 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c0b2070 con 0x7f11680040e0 2024-04-17T10:04:08.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f1182a00640 1 -- 172.21.15.32:0/1531479567 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f116c0a5660 con 0x7f11680040e0 2024-04-17T10:04:08.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.631+0000 7f1182a00640 1 -- 172.21.15.32:0/1531479567 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c0096b0 con 0x7f11680040e0 2024-04-17T10:04:08.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1188e00640 1 -- 172.21.15.32:0/1531479567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11680040e0 msgr2=0x7f1168004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:08.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1188e00640 1 --2- 172.21.15.32:0/1531479567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11680040e0 0x7f1168004810 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f116c009ae0 tx=0x7f116c01ca70 comp rx=0 tx=0).stop 2024-04-17T10:04:08.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1183e00640 1 -- 172.21.15.32:0/1531479567 reap_dead start 2024-04-17T10:04:08.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1188e00640 1 -- 172.21.15.32:0/1531479567 shutdown_connections 2024-04-17T10:04:08.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1188e00640 1 -- 172.21.15.32:0/1531479567 >> 172.21.15.32:0/1531479567 conn(0x7f11840b3e10 msgr2=0x7f11840b4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:08.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1188e00640 1 -- 172.21.15.32:0/1531479567 shutdown_connections 2024-04-17T10:04:08.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1188e00640 1 -- 172.21.15.32:0/1531479567 wait complete. 2024-04-17T10:04:08.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1188e00640 1 Processor -- start 2024-04-17T10:04:08.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1188e00640 1 -- start start 2024-04-17T10:04:08.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1188e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11680040e0 0x7f118414e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1188e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11840c6ea0 0x7f11841572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1188e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11840c7320 0x7f1184157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1188e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f118415d880 con 0x7f11840c7320 2024-04-17T10:04:08.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1188e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f118415d5a0 con 0x7f11840c6ea0 2024-04-17T10:04:08.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1188e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f118409b910 con 0x7f11680040e0 2024-04-17T10:04:08.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1183e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11840c7320 0x7f1184157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1183e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11840c7320 0x7f1184157800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41124/0 (socket says 172.21.15.32:41124) 2024-04-17T10:04:08.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1183e00640 1 -- 172.21.15.32:0/744073961 learned_addr learned my addr 172.21.15.32:0/744073961 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:08.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1183400640 1 --2- 172.21.15.32:0/744073961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11680040e0 0x7f118414e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f117be00640 1 --2- 172.21.15.32:0/744073961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11840c6ea0 0x7f11841572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1183e00640 1 -- 172.21.15.32:0/744073961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11680040e0 msgr2=0x7f118414e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:08.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1183e00640 1 --2- 172.21.15.32:0/744073961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11680040e0 0x7f118414e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1183e00640 1 -- 172.21.15.32:0/744073961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11840c6ea0 msgr2=0x7f11841572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:08.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1183e00640 1 --2- 172.21.15.32:0/744073961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11840c6ea0 0x7f11841572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.635+0000 7f1183e00640 1 -- 172.21.15.32:0/744073961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118414e600 con 0x7f11840c7320 2024-04-17T10:04:08.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.639+0000 7f1183e00640 1 --2- 172.21.15.32:0/744073961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11840c7320 0x7f1184157800 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f11840b8330 tx=0x7f117c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:08.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.639+0000 7f117b400640 1 -- 172.21.15.32:0/744073961 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c0019a0 con 0x7f11840c7320 2024-04-17T10:04:08.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.639+0000 7f117b400640 1 -- 172.21.15.32:0/744073961 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f117c00ee20 con 0x7f11840c7320 2024-04-17T10:04:08.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.639+0000 7f1188e00640 1 -- 172.21.15.32:0/744073961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11840c78d0 con 0x7f11840c7320 2024-04-17T10:04:08.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.639+0000 7f117b400640 1 -- 172.21.15.32:0/744073961 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c014670 con 0x7f11840c7320 2024-04-17T10:04:08.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.639+0000 7f1188e00640 1 -- 172.21.15.32:0/744073961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11840c7e90 con 0x7f11840c7320 2024-04-17T10:04:08.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.639+0000 7f117b400640 1 -- 172.21.15.32:0/744073961 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f117c014810 con 0x7f11840c7320 2024-04-17T10:04:08.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.639+0000 7f117b400640 1 --2- 172.21.15.32:0/744073961 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1160068380 0x7f116006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.639+0000 7f117b400640 1 -- 172.21.15.32:0/744073961 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7f117c086830 con 0x7f11840c7320 2024-04-17T10:04:08.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.639+0000 7f1183400640 1 --2- 172.21.15.32:0/744073961 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1160068380 0x7f116006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f118d871000 1 Processor -- start 2024-04-17T10:04:08.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f1183400640 1 --2- 172.21.15.32:0/744073961 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1160068380 0x7f116006a840 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f1174007af0 tx=0x7f1174007290 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:08.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f118d871000 1 -- start start 2024-04-17T10:04:08.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f118d871000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a497873b50 0x55a497873f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f118d871000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a497874470 0x55a4978297b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f118d871000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497829cf0 0x55a49782a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f118d871000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a4976774d0 con 0x55a497829cf0 2024-04-17T10:04:08.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f118d871000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a497611bf0 con 0x55a497873b50 2024-04-17T10:04:08.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f118d871000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55a497673490 con 0x55a497874470 2024-04-17T10:04:08.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f1183e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497829cf0 0x55a49782a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f1183e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497829cf0 0x55a49782a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41132/0 (socket says 172.21.15.32:41132) 2024-04-17T10:04:08.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f1183e00640 1 -- 172.21.15.32:0/434266528 learned_addr learned my addr 172.21.15.32:0/434266528 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:08.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f1183400640 1 --2- 172.21.15.32:0/434266528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a497873b50 0x55a497873f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f117be00640 1 --2- 172.21.15.32:0/434266528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a497874470 0x55a4978297b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f1183e00640 1 -- 172.21.15.32:0/434266528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a497874470 msgr2=0x55a4978297b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:08.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f1183e00640 1 --2- 172.21.15.32:0/434266528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a497874470 0x55a4978297b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f1183e00640 1 -- 172.21.15.32:0/434266528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a497873b50 msgr2=0x55a497873f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f1183e00640 1 --2- 172.21.15.32:0/434266528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a497873b50 0x55a497873f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f1183e00640 1 -- 172.21.15.32:0/434266528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4975fde80 con 0x55a497829cf0 2024-04-17T10:04:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f1183e00640 1 --2- 172.21.15.32:0/434266528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497829cf0 0x55a49782a0d0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f117c05ed00 tx=0x7f117c056340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7fbdac8cfc58723 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f1173e00640 1 -- 172.21.15.32:0/434266528 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c05aa00 con 0x55a497829cf0 2024-04-17T10:04:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f1173e00640 1 -- 172.21.15.32:0/434266528 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f117c05f230 con 0x55a497829cf0 2024-04-17T10:04:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f1173e00640 1 -- 172.21.15.32:0/434266528 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c052d70 con 0x55a497829cf0 2024-04-17T10:04:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f118d871000 1 -- 172.21.15.32:0/434266528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497829cf0 msgr2=0x55a49782a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f118d871000 1 --2- 172.21.15.32:0/434266528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497829cf0 0x55a49782a0d0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f117c05ed00 tx=0x7f117c056340 comp rx=0 tx=0).stop 2024-04-17T10:04:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f118d871000 1 -- 172.21.15.32:0/434266528 shutdown_connections 2024-04-17T10:04:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f118d871000 1 --2- 172.21.15.32:0/434266528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a497874470 0x55a4978297b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f118d871000 1 --2- 172.21.15.32:0/434266528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497829cf0 0x55a49782a0d0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f118d871000 1 --2- 172.21.15.32:0/434266528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a497873b50 0x55a497873f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f118d871000 1 -- 172.21.15.32:0/434266528 >> 172.21.15.32:0/434266528 conn(0x55a49782f470 msgr2=0x55a497828090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f118d871000 1 -- 172.21.15.32:0/434266528 shutdown_connections 2024-04-17T10:04:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f118d871000 1 -- 172.21.15.32:0/434266528 wait complete. 2024-04-17T10:04:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.643+0000 7f118d871000 1 -- start start 2024-04-17T10:04:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f118d871000 1 Processor -- start 2024-04-17T10:04:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f118d871000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a497829cf0 0x55a49782e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f118d871000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497873b50 0x55a497861440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f118d871000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a497874470 0x55a497861980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f118d871000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a497634a60 con 0x55a497873b50 2024-04-17T10:04:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f118d871000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a49761be80 con 0x55a497874470 2024-04-17T10:04:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f118d871000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55a497632cf0 con 0x55a497829cf0 2024-04-17T10:04:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f118d871000 10 client.0 _ll_register_callbacks cb 0x55a497814a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:04:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f117be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497873b50 0x55a497861440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f117be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497873b50 0x55a497861440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41142/0 (socket says 172.21.15.32:41142) 2024-04-17T10:04:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f117be00640 1 -- 172.21.15.32:0/3355861576 learned_addr learned my addr 172.21.15.32:0/3355861576 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f1183400640 1 --2- 172.21.15.32:0/3355861576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a497829cf0 0x55a49782e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f1183e00640 1 --2- 172.21.15.32:0/3355861576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a497874470 0x55a497861980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f117be00640 1 -- 172.21.15.32:0/3355861576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a497829cf0 msgr2=0x55a49782e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:08.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f117be00640 1 --2- 172.21.15.32:0/3355861576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a497829cf0 0x55a49782e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f117be00640 1 -- 172.21.15.32:0/3355861576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a497874470 msgr2=0x55a497861980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:08.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f117be00640 1 --2- 172.21.15.32:0/3355861576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a497874470 0x55a497861980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f117be00640 1 -- 172.21.15.32:0/3355861576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a497611e40 con 0x55a497873b50 2024-04-17T10:04:08.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f1183e00640 1 --2- 172.21.15.32:0/3355861576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a497874470 0x55a497861980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:08.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f117be00640 1 --2- 172.21.15.32:0/3355861576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497873b50 0x55a497861440 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f116c0b0040 tx=0x7f116c019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20266f04f5ed516 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:08.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f1170c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:04:08.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f1170c00640 1 -- 172.21.15.32:0/3355861576 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c0b2040 con 0x55a497873b50 2024-04-17T10:04:08.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f118d871000 1 -- 172.21.15.32:0/3355861576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f117c05f230 con 0x55a497873b50 2024-04-17T10:04:08.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f1170c00640 1 -- 172.21.15.32:0/3355861576 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7f116c0096b0 con 0x55a497873b50 2024-04-17T10:04:08.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f1170c00640 1 -- 172.21.15.32:0/3355861576 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f116c0c6050 con 0x55a497873b50 2024-04-17T10:04:08.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f118d871000 10 client.9663 fetch_fsmap learned FSMap version 104 2024-04-17T10:04:08.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f118d871000 1 -- 172.21.15.32:0/3355861576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x55a497642e00 con 0x55a497873b50 2024-04-17T10:04:08.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f1170c00640 1 -- 172.21.15.32:0/3355861576 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f117c05f230 con 0x55a497873b50 2024-04-17T10:04:08.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f1170c00640 1 -- 172.21.15.32:0/3355861576 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 104) ==== 3495+0+0 (secure 0 0 0) 0x7f116c0c9020 con 0x55a497873b50 2024-04-17T10:04:08.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f118d871000 10 client.9663 fetch_fsmap finished waiting for FSMap version 104 2024-04-17T10:04:08.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f118d871000 10 client.9663 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T10:04:08.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f118d871000 20 client.9663 populate_metadata read hostname 'smithi032' 2024-04-17T10:04:08.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f118d871000 1 --2- 172.21.15.32:0/3355861576 >> [v2:172.21.15.32:6836/578643390,v1:172.21.15.32:6837/578643390] conn(0x55a4978830b0 0x55a4978854a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f118d871000 4 client.9663 mds_command: new command op to 9615 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:04:08.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f118d871000 1 -- 172.21.15.32:0/3355861576 --> [v2:172.21.15.32:6836/578643390,v1:172.21.15.32:6837/578643390] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a497610bc0 con 0x55a4978830b0 2024-04-17T10:04:08.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f1183400640 1 --2- 172.21.15.32:0/3355861576 >> [v2:172.21.15.32:6836/578643390,v1:172.21.15.32:6837/578643390] conn(0x55a4978830b0 0x55a4978854a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f1183400640 1 --2- 172.21.15.32:0/3355861576 >> [v2:172.21.15.32:6836/578643390,v1:172.21.15.32:6837/578643390] conn(0x55a4978830b0 0x55a4978854a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5cacd9eb3dd14c server_cookie=c119668f50d628e5 in_seq=0 out_seq=0 2024-04-17T10:04:08.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.647+0000 7f1170c00640 10 client.9663 ms_handle_connect on v2:172.21.15.32:6836/578643390 2024-04-17T10:04:08.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.651+0000 7f1170c00640 1 -- 172.21.15.32:0/3355861576 <== mds.0 v2:172.21.15.32:6836/578643390 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a497610bc0 con 0x55a4978830b0 2024-04-17T10:04:08.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.651+0000 7f1170c00640 10 client.9663 handle_command_reply: tid=0 2024-04-17T10:04:08.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.651+0000 7f118d871000 1 client.9663 shutdown 2024-04-17T10:04:08.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.651+0000 7f118d871000 1 -- 172.21.15.32:0/3355861576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497873b50 msgr2=0x55a497861440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:08.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.651+0000 7f118d871000 1 --2- 172.21.15.32:0/3355861576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497873b50 0x55a497861440 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f116c0b0040 tx=0x7f116c019810 comp rx=0 tx=0).stop 2024-04-17T10:04:08.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.651+0000 7f118d871000 1 -- 172.21.15.32:0/3355861576 shutdown_connections 2024-04-17T10:04:08.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.651+0000 7f118d871000 1 --2- 172.21.15.32:0/3355861576 >> [v2:172.21.15.32:6836/578643390,v1:172.21.15.32:6837/578643390] conn(0x55a4978830b0 0x55a4978854a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.651+0000 7f118d871000 1 --2- 172.21.15.32:0/3355861576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a497874470 0x55a497861980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.651+0000 7f118d871000 1 --2- 172.21.15.32:0/3355861576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497873b50 0x55a497861440 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.651+0000 7f118d871000 1 --2- 172.21.15.32:0/3355861576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a497829cf0 0x55a49782e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.651+0000 7f1170c00640 0 client.9663 ms_handle_reset on v2:172.21.15.32:6836/578643390 2024-04-17T10:04:08.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.651+0000 7f118d871000 1 -- 172.21.15.32:0/3355861576 >> 172.21.15.32:0/3355861576 conn(0x55a49782f470 msgr2=0x55a4978287a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:08.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.651+0000 7f118d871000 1 -- 172.21.15.32:0/3355861576 shutdown_connections 2024-04-17T10:04:08.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.651+0000 7f118d871000 1 -- 172.21.15.32:0/3355861576 wait complete. 2024-04-17T10:04:08.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.651+0000 7f118d871000 20 client.9663 trim_cache size 0 max 16384 2024-04-17T10:04:08.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.667+0000 7f118d871000 1 Processor -- start 2024-04-17T10:04:08.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.667+0000 7f118d871000 1 -- start start 2024-04-17T10:04:08.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f118d871000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a497829cf0 0x55a497861020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f118d871000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497873b50 0x55a497861560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f118d871000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a497874470 0x55a4978844a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f118d871000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f11841fe450 con 0x55a497873b50 2024-04-17T10:04:08.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f118d871000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f11840ca7f0 con 0x55a497874470 2024-04-17T10:04:08.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f118d871000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f11840cae40 con 0x55a497829cf0 2024-04-17T10:04:08.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f117be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497873b50 0x55a497861560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f117be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497873b50 0x55a497861560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41156/0 (socket says 172.21.15.32:41156) 2024-04-17T10:04:08.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f117be00640 1 -- 172.21.15.32:0/2380691824 learned_addr learned my addr 172.21.15.32:0/2380691824 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:08.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f1183400640 1 --2- 172.21.15.32:0/2380691824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a497829cf0 0x55a497861020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f1183e00640 1 --2- 172.21.15.32:0/2380691824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a497874470 0x55a4978844a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f117be00640 1 -- 172.21.15.32:0/2380691824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a497829cf0 msgr2=0x55a497861020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:08.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f117be00640 1 --2- 172.21.15.32:0/2380691824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a497829cf0 0x55a497861020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f117be00640 1 -- 172.21.15.32:0/2380691824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a497874470 msgr2=0x55a4978844a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:08.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f117be00640 1 --2- 172.21.15.32:0/2380691824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a497874470 0x55a4978844a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f117be00640 1 -- 172.21.15.32:0/2380691824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4976f8d30 con 0x55a497873b50 2024-04-17T10:04:08.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f1183400640 1 --2- 172.21.15.32:0/2380691824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a497829cf0 0x55a497861020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:08.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f1183e00640 1 --2- 172.21.15.32:0/2380691824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a497874470 0x55a4978844a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:08.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f117be00640 1 --2- 172.21.15.32:0/2380691824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497873b50 0x55a497861560 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f116c01cf50 tx=0x7f116c0b8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559cdaf203627e5e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:08.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f1173400640 1 -- 172.21.15.32:0/2380691824 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c0b2040 con 0x55a497873b50 2024-04-17T10:04:08.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f1173400640 1 -- 172.21.15.32:0/2380691824 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f116c000e50 con 0x55a497873b50 2024-04-17T10:04:08.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f1173400640 1 -- 172.21.15.32:0/2380691824 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c0a58b0 con 0x55a497873b50 2024-04-17T10:04:08.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f118d871000 1 -- 172.21.15.32:0/2380691824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497873b50 msgr2=0x55a497861560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:08.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f118d871000 1 --2- 172.21.15.32:0/2380691824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497873b50 0x55a497861560 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f116c01cf50 tx=0x7f116c0b8000 comp rx=0 tx=0).stop 2024-04-17T10:04:08.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f118d871000 1 -- 172.21.15.32:0/2380691824 shutdown_connections 2024-04-17T10:04:08.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f118d871000 1 --2- 172.21.15.32:0/2380691824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a497874470 0x55a4978844a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f118d871000 1 --2- 172.21.15.32:0/2380691824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497873b50 0x55a497861560 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f118d871000 1 --2- 172.21.15.32:0/2380691824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a497829cf0 0x55a497861020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f118d871000 1 -- 172.21.15.32:0/2380691824 >> 172.21.15.32:0/2380691824 conn(0x55a49782f470 msgr2=0x55a497860910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:08.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f118d871000 1 -- 172.21.15.32:0/2380691824 shutdown_connections 2024-04-17T10:04:08.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f118d871000 1 -- 172.21.15.32:0/2380691824 wait complete. 2024-04-17T10:04:08.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f118d871000 1 -- start start 2024-04-17T10:04:08.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f118d871000 1 Processor -- start 2024-04-17T10:04:08.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f118d871000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a497829cf0 0x55a4978e26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f118d871000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497873b50 0x55a4978e2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f118d871000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a497874470 0x55a497887270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f118d871000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a49767bb00 con 0x55a497873b50 2024-04-17T10:04:08.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f118d871000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a4976767e0 con 0x55a497874470 2024-04-17T10:04:08.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.671+0000 7f118d871000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55a4976846e0 con 0x55a497829cf0 2024-04-17T10:04:08.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f118d871000 10 client.0 _ll_register_callbacks cb 0x7f116800b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:04:08.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f117be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497873b50 0x55a4978e2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f117be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497873b50 0x55a4978e2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41170/0 (socket says 172.21.15.32:41170) 2024-04-17T10:04:08.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f117be00640 1 -- 172.21.15.32:0/3121980326 learned_addr learned my addr 172.21.15.32:0/3121980326 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:08.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f1183e00640 1 --2- 172.21.15.32:0/3121980326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a497874470 0x55a497887270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f1183400640 1 --2- 172.21.15.32:0/3121980326 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a497829cf0 0x55a4978e26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f117be00640 1 -- 172.21.15.32:0/3121980326 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a497829cf0 msgr2=0x55a4978e26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:08.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f117be00640 1 --2- 172.21.15.32:0/3121980326 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a497829cf0 0x55a4978e26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f117be00640 1 -- 172.21.15.32:0/3121980326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a497874470 msgr2=0x55a497887270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:08.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f117be00640 1 --2- 172.21.15.32:0/3121980326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a497874470 0x55a497887270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f117be00640 1 -- 172.21.15.32:0/3121980326 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4976097c0 con 0x55a497873b50 2024-04-17T10:04:08.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f117be00640 1 --2- 172.21.15.32:0/3121980326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497873b50 0x55a4978e2bf0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f116c006b20 tx=0x7f116c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b66682e92eaeae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:08.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f1170c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:04:08.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f1170c00640 1 -- 172.21.15.32:0/3121980326 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c0b2070 con 0x55a497873b50 2024-04-17T10:04:08.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f1170c00640 1 -- 172.21.15.32:0/3121980326 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7f116c003820 con 0x55a497873b50 2024-04-17T10:04:08.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f118d871000 1 -- 172.21.15.32:0/3121980326 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f116c000e50 con 0x55a497873b50 2024-04-17T10:04:08.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f1170c00640 1 -- 172.21.15.32:0/3121980326 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f116c003a40 con 0x55a497873b50 2024-04-17T10:04:08.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f118d871000 10 client.9669 fetch_fsmap learned FSMap version 104 2024-04-17T10:04:08.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f118d871000 1 -- 172.21.15.32:0/3121980326 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x55a49761c650 con 0x55a497873b50 2024-04-17T10:04:08.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f1170c00640 1 -- 172.21.15.32:0/3121980326 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f116c000e50 con 0x55a497873b50 2024-04-17T10:04:08.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f1170c00640 1 -- 172.21.15.32:0/3121980326 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 104) ==== 3495+0+0 (secure 0 0 0) 0x7f116c0c9070 con 0x55a497873b50 2024-04-17T10:04:08.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f118d871000 10 client.9669 fetch_fsmap finished waiting for FSMap version 104 2024-04-17T10:04:08.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f118d871000 10 client.9669 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T10:04:08.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f118d871000 20 client.9669 populate_metadata read hostname 'smithi032' 2024-04-17T10:04:08.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f118d871000 1 --2- 172.21.15.32:0/3121980326 >> [v2:172.21.15.32:6836/578643390,v1:172.21.15.32:6837/578643390] conn(0x55a497887c30 0x55a497888010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f118d871000 4 client.9669 mds_command: new command op to 9615 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T10:04:08.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f118d871000 1 -- 172.21.15.32:0/3121980326 --> [v2:172.21.15.32:6836/578643390,v1:172.21.15.32:6837/578643390] -- command(tid 0: {"prefix": "status"}) -- 0x55a497684ef0 con 0x55a497887c30 2024-04-17T10:04:08.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f1183400640 1 --2- 172.21.15.32:0/3121980326 >> [v2:172.21.15.32:6836/578643390,v1:172.21.15.32:6837/578643390] conn(0x55a497887c30 0x55a497888010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f1183400640 1 --2- 172.21.15.32:0/3121980326 >> [v2:172.21.15.32:6836/578643390,v1:172.21.15.32:6837/578643390] conn(0x55a497887c30 0x55a497888010 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8fcef8a67b4ef12 server_cookie=a5278770f7beb043 in_seq=0 out_seq=0 2024-04-17T10:04:08.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f1170c00640 10 client.9669 ms_handle_connect on v2:172.21.15.32:6836/578643390 2024-04-17T10:04:08.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f1170c00640 1 -- 172.21.15.32:0/3121980326 <== mds.0 v2:172.21.15.32:6836/578643390 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55a497684ef0 con 0x55a497887c30 2024-04-17T10:04:08.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f1170c00640 10 client.9669 handle_command_reply: tid=0 2024-04-17T10:04:08.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f118d871000 1 client.9669 shutdown 2024-04-17T10:04:08.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f118d871000 1 -- 172.21.15.32:0/3121980326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497873b50 msgr2=0x55a4978e2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:08.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f118d871000 1 --2- 172.21.15.32:0/3121980326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497873b50 0x55a4978e2bf0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f116c006b20 tx=0x7f116c009190 comp rx=0 tx=0).stop 2024-04-17T10:04:08.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f118d871000 1 -- 172.21.15.32:0/3121980326 shutdown_connections 2024-04-17T10:04:08.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f118d871000 1 --2- 172.21.15.32:0/3121980326 >> [v2:172.21.15.32:6836/578643390,v1:172.21.15.32:6837/578643390] conn(0x55a497887c30 0x55a497888010 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f118d871000 1 --2- 172.21.15.32:0/3121980326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a497874470 0x55a497887270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f118d871000 1 --2- 172.21.15.32:0/3121980326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a497873b50 0x55a4978e2bf0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f118d871000 1 --2- 172.21.15.32:0/3121980326 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a497829cf0 0x55a4978e26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f118d871000 1 -- 172.21.15.32:0/3121980326 >> 172.21.15.32:0/3121980326 conn(0x55a49782f470 msgr2=0x55a49782b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:08.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f1170c00640 0 client.9669 ms_handle_reset on v2:172.21.15.32:6836/578643390 2024-04-17T10:04:08.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f118d871000 1 -- 172.21.15.32:0/3121980326 shutdown_connections 2024-04-17T10:04:08.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f118d871000 1 -- 172.21.15.32:0/3121980326 wait complete. 2024-04-17T10:04:08.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f118d871000 20 client.9669 trim_cache size 0 max 16384 2024-04-17T10:04:08.685 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:04:08.685 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "ff178df8-4c06-4367-a1e9-2e3a66d870d0", 2024-04-17T10:04:08.685 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-17T10:04:08.685 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 9615, 2024-04-17T10:04:08.685 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-17T10:04:08.685 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-17T10:04:08.686 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-17T10:04:08.686 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 4.5876028629999999, 2024-04-17T10:04:08.686 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 104, 2024-04-17T10:04:08.686 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 112, 2024-04-17T10:04:08.686 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 112, 2024-04-17T10:04:08.686 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 5.5287810300000002 2024-04-17T10:04:08.686 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:04:08.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f1188e00640 1 -- 172.21.15.32:0/744073961 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1160068380 msgr2=0x7f116006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:08.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f1188e00640 1 --2- 172.21.15.32:0/744073961 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1160068380 0x7f116006a840 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f1174007af0 tx=0x7f1174007290 comp rx=0 tx=0).stop 2024-04-17T10:04:08.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f1188e00640 1 -- 172.21.15.32:0/744073961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11840c7320 msgr2=0x7f1184157800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:08.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.675+0000 7f1188e00640 1 --2- 172.21.15.32:0/744073961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11840c7320 0x7f1184157800 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f11840b8330 tx=0x7f117c00e530 comp rx=0 tx=0).stop 2024-04-17T10:04:08.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.679+0000 7f1188e00640 1 -- 172.21.15.32:0/744073961 shutdown_connections 2024-04-17T10:04:08.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.679+0000 7f1188e00640 1 --2- 172.21.15.32:0/744073961 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f1160068380 0x7f116006a840 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.679+0000 7f1188e00640 1 --2- 172.21.15.32:0/744073961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11840c7320 0x7f1184157800 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.679+0000 7f1188e00640 1 --2- 172.21.15.32:0/744073961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11840c6ea0 0x7f11841572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.679+0000 7f1188e00640 1 --2- 172.21.15.32:0/744073961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11680040e0 0x7f118414e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.679+0000 7f1188e00640 1 -- 172.21.15.32:0/744073961 >> 172.21.15.32:0/744073961 conn(0x7f11840b3e10 msgr2=0x7f118415e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:08.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.679+0000 7f1188e00640 1 -- 172.21.15.32:0/744073961 shutdown_connections 2024-04-17T10:04:08.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.679+0000 7f1188e00640 1 -- 172.21.15.32:0/744073961 wait complete. 2024-04-17T10:04:08.695 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T10:04:08.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.839+0000 7f871e400640 1 Processor -- start 2024-04-17T10:04:08.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.839+0000 7f871e400640 1 -- start start 2024-04-17T10:04:08.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.839+0000 7f871e400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f87180cf990 0x7f871806dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.839+0000 7f871e400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f871806f250 0x7f87180aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.839+0000 7f871e400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f871806e1b0 con 0x7f87180cf5b0 2024-04-17T10:04:08.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.839+0000 7f871e400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f871806edc0 con 0x7f871806f250 2024-04-17T10:04:08.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.839+0000 7f871e400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f871806e490 con 0x7f87180cf990 2024-04-17T10:04:08.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.839+0000 7f871da00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f871806f250 0x7f87180aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.839+0000 7f871da00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f871806f250 0x7f87180aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41178/0 (socket says 172.21.15.32:41178) 2024-04-17T10:04:08.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.839+0000 7f871da00640 1 -- 172.21.15.32:0/1767480754 learned_addr learned my addr 172.21.15.32:0/1767480754 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:08.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.839+0000 7f8717e00640 1 --2- 172.21.15.32:0/1767480754 >> v2:172.21.15.144:3300/0 conn(0x7f87180cf990 0x7f871806dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.839+0000 7f871da00640 1 -- 172.21.15.32:0/1767480754 >> v2:172.21.15.144:3300/0 conn(0x7f87180cf990 msgr2=0x7f871806dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:08.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.839+0000 7f871da00640 1 --2- 172.21.15.32:0/1767480754 >> v2:172.21.15.144:3300/0 conn(0x7f87180cf990 0x7f871806dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.839+0000 7f871da00640 1 -- 172.21.15.32:0/1767480754 >> v1:172.21.15.144:6789/0 conn(0x7f87180cf5b0 legacy=0x7f871815d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:08.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.839+0000 7f871da00640 1 -- 172.21.15.32:0/1767480754 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871806e770 con 0x7f871806f250 2024-04-17T10:04:08.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.843+0000 7f871da00640 1 --2- 172.21.15.32:0/1767480754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f871806f250 0x7f87180aeef0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f871806fbe0 tx=0x7f871001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919bfd3be9af483f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:08.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.843+0000 7f8717400640 1 -- 172.21.15.32:0/1767480754 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87100a5810 con 0x7f871806f250 2024-04-17T10:04:08.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.843+0000 7f8717400640 1 -- 172.21.15.32:0/1767480754 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87100a59b0 con 0x7f871806f250 2024-04-17T10:04:08.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.843+0000 7f8717400640 1 -- 172.21.15.32:0/1767480754 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87100a5d80 con 0x7f871806f250 2024-04-17T10:04:08.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.843+0000 7f871e400640 1 -- 172.21.15.32:0/1767480754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f871806f250 msgr2=0x7f87180aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:08.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.843+0000 7f871e400640 1 --2- 172.21.15.32:0/1767480754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f871806f250 0x7f87180aeef0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f871806fbe0 tx=0x7f871001c930 comp rx=0 tx=0).stop 2024-04-17T10:04:08.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.843+0000 7f871e400640 1 -- 172.21.15.32:0/1767480754 shutdown_connections 2024-04-17T10:04:08.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.843+0000 7f871e400640 1 --2- 172.21.15.32:0/1767480754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f871806f250 0x7f87180aeef0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.843+0000 7f871e400640 1 --2- 172.21.15.32:0/1767480754 >> v2:172.21.15.144:3300/0 conn(0x7f87180cf990 0x7f871806dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.843+0000 7f871e400640 1 -- 172.21.15.32:0/1767480754 >> 172.21.15.32:0/1767480754 conn(0x7f87180d0180 msgr2=0x7f87180d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:08.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.843+0000 7f871e400640 1 -- 172.21.15.32:0/1767480754 shutdown_connections 2024-04-17T10:04:08.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.843+0000 7f871e400640 1 -- 172.21.15.32:0/1767480754 wait complete. 2024-04-17T10:04:08.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f871e400640 1 Processor -- start 2024-04-17T10:04:08.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f871e400640 1 -- start start 2024-04-17T10:04:08.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f871e400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f871806f250 0x7f8718099730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f871e400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87180cf5b0 0x7f8718098ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f871e400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87180cf990 0x7f871809aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f871e400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f871815e6e0 con 0x7f871806f250 2024-04-17T10:04:08.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f871e400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f871815e560 con 0x7f87180cf990 2024-04-17T10:04:08.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f871e400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f871815e860 con 0x7f87180cf5b0 2024-04-17T10:04:08.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f871d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f871806f250 0x7f8718099730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f871d000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f871806f250 0x7f8718099730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41180/0 (socket says 172.21.15.32:41180) 2024-04-17T10:04:08.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f871d000640 1 -- 172.21.15.32:0/714842116 learned_addr learned my addr 172.21.15.32:0/714842116 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:08.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f8717e00640 1 --2- 172.21.15.32:0/714842116 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87180cf5b0 0x7f8718098ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f871da00640 1 --2- 172.21.15.32:0/714842116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87180cf990 0x7f871809aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f871d000640 1 -- 172.21.15.32:0/714842116 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87180cf5b0 msgr2=0x7f8718098ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:08.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f871d000640 1 --2- 172.21.15.32:0/714842116 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87180cf5b0 0x7f8718098ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f871d000640 1 -- 172.21.15.32:0/714842116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87180cf990 msgr2=0x7f871809aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:08.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f871d000640 1 --2- 172.21.15.32:0/714842116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87180cf990 0x7f871809aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:08.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f871d000640 1 -- 172.21.15.32:0/714842116 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8718156e10 con 0x7f871806f250 2024-04-17T10:04:08.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f871d000640 1 --2- 172.21.15.32:0/714842116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f871806f250 0x7f8718099730 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f870c00c3f0 tx=0x7f870c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:08.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f8714c00640 1 -- 172.21.15.32:0/714842116 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c017070 con 0x7f871806f250 2024-04-17T10:04:08.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f8714c00640 1 -- 172.21.15.32:0/714842116 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f870c0052d0 con 0x7f871806f250 2024-04-17T10:04:08.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f8714c00640 1 -- 172.21.15.32:0/714842116 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c003840 con 0x7f871806f250 2024-04-17T10:04:08.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f871e400640 1 -- 172.21.15.32:0/714842116 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871815bb50 con 0x7f871806f250 2024-04-17T10:04:08.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f871e400640 1 -- 172.21.15.32:0/714842116 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8718099350 con 0x7f871806f250 2024-04-17T10:04:08.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.847+0000 7f871e400640 1 -- 172.21.15.32:0/714842116 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f871804fd00 con 0x7f871806f250 2024-04-17T10:04:08.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.851+0000 7f8714c00640 1 -- 172.21.15.32:0/714842116 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f870c01a070 con 0x7f871806f250 2024-04-17T10:04:08.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.851+0000 7f8714c00640 1 --2- 172.21.15.32:0/714842116 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f86f80683b0 0x7f86f806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:08.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.851+0000 7f8714c00640 1 -- 172.21.15.32:0/714842116 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7f870c085d30 con 0x7f871806f250 2024-04-17T10:04:08.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.851+0000 7f8717e00640 1 --2- 172.21.15.32:0/714842116 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f86f80683b0 0x7f86f806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:08.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.851+0000 7f8717e00640 1 --2- 172.21.15.32:0/714842116 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f86f80683b0 0x7f86f806a870 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f87181fb6c0 tx=0x7f8704009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:08.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:08.855+0000 7f8714c00640 1 -- 172.21.15.32:0/714842116 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f870c005480 con 0x7f871806f250 2024-04-17T10:04:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:09.055+0000 7f871e400640 1 -- 172.21.15.32:0/714842116 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f871815bd20 con 0x7f871806f250 2024-04-17T10:04:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:09.055+0000 7f8714c00640 1 -- 172.21.15.32:0/714842116 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v104) ==== 53+0+73 (secure 0 0 0) 0x7f870c055c10 con 0x7f871806f250 2024-04-17T10:04:09.053 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T10:04:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:09.055+0000 7f871e400640 1 -- 172.21.15.32:0/714842116 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f86f80683b0 msgr2=0x7f86f806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:09.055+0000 7f871e400640 1 --2- 172.21.15.32:0/714842116 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f86f80683b0 0x7f86f806a870 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f87181fb6c0 tx=0x7f8704009190 comp rx=0 tx=0).stop 2024-04-17T10:04:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:09.055+0000 7f871e400640 1 -- 172.21.15.32:0/714842116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f871806f250 msgr2=0x7f8718099730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:09.055+0000 7f871e400640 1 --2- 172.21.15.32:0/714842116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f871806f250 0x7f8718099730 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f870c00c3f0 tx=0x7f870c00c8b0 comp rx=0 tx=0).stop 2024-04-17T10:04:09.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:09.055+0000 7f871e400640 1 -- 172.21.15.32:0/714842116 shutdown_connections 2024-04-17T10:04:09.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:09.055+0000 7f871e400640 1 --2- 172.21.15.32:0/714842116 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f86f80683b0 0x7f86f806a870 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:09.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:09.055+0000 7f871e400640 1 --2- 172.21.15.32:0/714842116 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87180cf5b0 0x7f8718098ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:09.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:09.055+0000 7f871e400640 1 --2- 172.21.15.32:0/714842116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87180cf990 0x7f871809aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:09.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:09.055+0000 7f871e400640 1 --2- 172.21.15.32:0/714842116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f871806f250 0x7f8718099730 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:09.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:09.055+0000 7f871e400640 1 -- 172.21.15.32:0/714842116 >> 172.21.15.32:0/714842116 conn(0x7f87180d0180 msgr2=0x7f87180b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:09.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:09.055+0000 7f871e400640 1 -- 172.21.15.32:0/714842116 shutdown_connections 2024-04-17T10:04:09.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:09.055+0000 7f871e400640 1 -- 172.21.15.32:0/714842116 wait complete. 2024-04-17T10:04:09.075 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T10:04:09.075 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T10:04:09.075 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-17T10:04:09.075 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T10:04:09.075 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T10:04:09.075 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T10:04:09.075 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T10:04:09.075 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T10:04:09.075 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:04:09.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:09.076 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-17T10:04:09.085 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:04:09.085 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:04:09.085 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:04:09.085 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:04:09.085 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-17T10:04:09.085 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:04:09.085 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:04:09.085 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-17T10:04:09.085 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-17T10:04:09.085 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T10:04:09.085 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:04:09.085 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-17T10:04:09.085 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-17T10:04:09.086 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 41805sec preferred_lft 41805sec 2024-04-17T10:04:09.086 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2024-04-17T10:04:09.086 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:04:09.086 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:04:09.086 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-17T10:04:09.086 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-17T10:04:09.086 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:04:09.086 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:04:09.086 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-17T10:04:09.086 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T10:04:09.086 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:cd:f4:aa:5b brd ff:ff:ff:ff:ff:ff 2024-04-17T10:04:09.086 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T10:04:09.086 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:04:09.086 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T10:04:09.087 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T10:04:09.087 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T10:04:09.087 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:04:09.087 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T10:04:09.087 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:04:09.087 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:04:09.087 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:04:09.087 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T10:04:09.087 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:04:09.087 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:04:09.087 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:04:09.087 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T10:04:09.087 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:04:09.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:09.088 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T10:04:09.089 INFO:tasks.ceph.mds.a.smithi032.stderr:2024-04-17T10:04:09.091+0000 7f991b000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:04:09.107 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-17T10:04:09.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:09.109 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-17T10:04:09.161 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-17T10:04:09.161 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T10:04:09.161 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-17T10:04:09.161 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T10:04:09.161 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-17T10:04:09.161 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T10:04:09.162 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T10:04:09.162 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T10:04:09.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-17T10:04:09.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-17T10:04:09.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-17T10:04:09.162 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T10:04:09.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T10:04:09.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:04:09.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:09.305 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-17T10:04:09.314 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T10:04:09.315 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T10:04:09.315 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:04:09.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:09.315 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-17T10:04:09.380 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T10:04:09.380 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:04:09.380 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T10:04:09.431 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:04:09.431 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:04:09.431 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:04:09.486 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T10:04:09.547 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:04:09.600 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T10:04:09.600 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T10:04:09.600 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T10:04:09.601 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T10:04:09.601 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T10:04:09.601 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T10:04:09.601 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T10:04:09.601 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T10:04:09.601 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T10:04:09.601 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T10:04:09.601 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T10:04:09.601 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T10:04:09.601 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:04:09.601 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T10:04:09.601 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T10:04:09.601 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T10:04:09.601 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T10:04:09.602 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T10:04:09.602 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T10:04:09.602 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T10:04:09.602 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:04:09.602 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T10:04:09.602 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T10:04:09.602 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T10:04:09.602 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:04:09.602 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:04:09.602 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:04:09.602 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:04:09.602 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T10:04:09.602 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:04:09.602 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:04:09.603 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:04:09.603 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:04:09.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:09.604 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T10:04:09.659 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T10:04:09.659 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T10:04:09.710 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T10:04:09.731 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:04:09.785 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T10:04:09.785 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T10:04:09.785 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T10:04:09.785 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T10:04:09.785 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T10:04:09.785 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T10:04:09.785 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T10:04:09.785 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T10:04:09.785 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T10:04:09.785 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T10:04:09.785 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T10:04:09.785 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T10:04:09.786 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:04:09.786 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T10:04:09.786 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T10:04:09.786 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T10:04:09.786 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T10:04:09.786 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T10:04:09.786 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T10:04:09.786 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T10:04:09.786 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:04:09.786 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T10:04:09.786 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T10:04:09.786 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T10:04:09.786 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:04:09.787 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:04:09.787 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:04:09.787 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:04:09.787 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T10:04:09.787 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:04:09.787 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:04:09.787 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:04:09.787 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:04:09.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:09.788 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T10:04:09.806 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2024-04-17T10:04:09.807+0000 7f5e9e3be580 -1 init, newargv = 0x55e5424caf90 newargc=15 2024-04-17T10:04:09.806 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[52329]: starting ceph client 2024-04-17T10:04:09.813 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[52329]: starting fuse 2024-04-17T10:04:09.814 INFO:teuthology.orchestra.run.smithi032.stdout:44 2024-04-17T10:04:09.815 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-17T10:04:09.815 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2024-04-17T10:04:09.815 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2024-04-17T10:04:09.815 DEBUG:teuthology.orchestra.run.smithi032:> import re 2024-04-17T10:04:09.815 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:04:09.815 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2024-04-17T10:04:09.815 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:04:09.815 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2024-04-17T10:04:09.815 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T10:04:09.815 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2024-04-17T10:04:09.816 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T10:04:09.816 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:04:09.816 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2024-04-17T10:04:09.816 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2024-04-17T10:04:09.816 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2024-04-17T10:04:09.816 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2024-04-17T10:04:09.816 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:04:09.816 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2024-04-17T10:04:09.816 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T10:04:09.816 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T10:04:09.816 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T10:04:09.816 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2024-04-17T10:04:09.816 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2024-04-17T10:04:09.816 DEBUG:teuthology.orchestra.run.smithi032:> return f 2024-04-17T10:04:09.816 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T10:04:09.817 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:04:09.817 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2024-04-17T10:04:09.817 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:04:09.939 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.52329.asok 2024-04-17T10:04:09.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:04:09.950 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52329.asok 2024-04-17T10:04:09.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:09.951 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52329.asok status 2024-04-17T10:04:10.127 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:04:10.128 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2024-04-17T10:04:10.128 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-17T10:04:10.128 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T10:04:10.128 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2024-04-17T10:04:10.128 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2024-04-17T10:04:10.128 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T10:04:10.128 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "52329", 2024-04-17T10:04:10.128 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2024-04-17T10:04:10.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:04:10.128 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2024-04-17T10:04:10.128 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2024-04-17T10:04:10.128 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 9684, 2024-04-17T10:04:10.128 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2024-04-17T10:04:10.129 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2024-04-17T10:04:10.129 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2024-04-17T10:04:10.129 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 9684 2024-04-17T10:04:10.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:04:10.129 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T10:04:10.129 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T10:04:10.129 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:04:10.129 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3393830791 2024-04-17T10:04:10.129 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:04:10.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:04:10.129 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T10:04:10.129 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T10:04:10.129 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:04:10.129 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3393830791 2024-04-17T10:04:10.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:04:10.130 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.9684 192.168.0.1:0/3393830791", 2024-04-17T10:04:10.130 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/3393830791", 2024-04-17T10:04:10.130 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2024-04-17T10:04:10.130 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 104, 2024-04-17T10:04:10.130 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 112, 2024-04-17T10:04:10.130 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2024-04-17T10:04:10.130 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2024-04-17T10:04:10.130 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs" 2024-04-17T10:04:10.130 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:04:10.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:10.143 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2024-04-17T10:04:10.143 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T10:04:10.152 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2024-04-17T10:04:10.152 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T10:04:10.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:10.153 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:04:10.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:10.215 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-17T10:04:10.280 INFO:teuthology.orchestra.run.smithi032.stdout:./ 2024-04-17T10:04:10.280 INFO:teuthology.orchestra.run.smithi032.stdout:./parent 2024-04-17T10:04:10.280 INFO:teuthology.orchestra.run.smithi032.stdout:./parent/flushed 2024-04-17T10:04:10.281 INFO:teuthology.orchestra.run.smithi032.stdout:./parent/flushed/charlie 2024-04-17T10:04:10.281 INFO:teuthology.orchestra.run.smithi032.stdout:./parent/flushed/alpha 2024-04-17T10:04:10.281 INFO:teuthology.orchestra.run.smithi032.stdout:./parent/unflushed 2024-04-17T10:04:10.281 INFO:teuthology.orchestra.run.smithi032.stdout:./parent/unflushed/jfile 2024-04-17T10:04:10.282 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:04:10.282 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:04:10.282 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:04:10.282 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:04:10.282 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-17T10:04:10.282 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:04:10.393 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-17T10:04:10.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:04:10.403 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:04:10.403 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:04:10.403 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:04:10.403 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:04:10.403 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-17T10:04:10.403 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:04:10.509 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627776 2024-04-17T10:04:10.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:04:10.519 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:04:10.519 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:04:10.519 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:04:10.519 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:04:10.519 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-17T10:04:10.519 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:04:10.625 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627777 2024-04-17T10:04:10.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:04:10.635 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:04:10.635 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:04:10.635 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:04:10.635 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:04:10.635 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-17T10:04:10.635 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:04:10.746 INFO:teuthology.orchestra.run.smithi032.stdout:1099511628280 2024-04-17T10:04:10.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:04:10.755 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:04:10.755 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:04:10.755 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:04:10.755 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:04:10.755 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-17T10:04:10.755 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:04:10.866 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627778 2024-04-17T10:04:10.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:04:10.876 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:04:10.876 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:04:10.876 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:04:10.876 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:04:10.876 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-17T10:04:10.876 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:04:10.985 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627779 2024-04-17T10:04:10.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:04:10.994 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:04:10.995 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:04:10.995 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:04:10.995 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:04:10.995 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-17T10:04:10.995 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:04:11.105 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627780 2024-04-17T10:04:11.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:04:11.115 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:04:11.116 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:04:11.168 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:04:11.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.350+0000 7f37a2200640 1 Processor -- start 2024-04-17T10:04:11.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.350+0000 7f37a2200640 1 -- start start 2024-04-17T10:04:11.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.354+0000 7f37a2200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f379c144e90 0x7f379c0b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:11.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.354+0000 7f37a2200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f379c072bb0 con 0x7f379c0b4de0 2024-04-17T10:04:11.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.354+0000 7f37a2200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f379c0b3dc0 con 0x7f379c15c1b0 2024-04-17T10:04:11.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.354+0000 7f37a2200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f379c145270 con 0x7f379c144e90 2024-04-17T10:04:11.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.354+0000 7f379b400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f379c144e90 0x7f379c0b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:11.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.354+0000 7f379b400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f379c144e90 0x7f379c0b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50494/0 (socket says 172.21.15.32:50494) 2024-04-17T10:04:11.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.354+0000 7f379b400640 1 -- 172.21.15.32:0/2161205476 learned_addr learned my addr 172.21.15.32:0/2161205476 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:11.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.354+0000 7f3792a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f379c0b4de0 0x7f379c15baa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59060/0 (socket says 172.21.15.32:59060) 2024-04-17T10:04:11.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.354+0000 7f379b400640 1 -- 172.21.15.32:0/2161205476 >> v1:172.21.15.144:6789/0 conn(0x7f379c15c1b0 legacy=0x7f379c06ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:11.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.354+0000 7f379aa00640 1 -- 172.21.15.32:0/2161205476 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4031933390 0 0) 0x7f379c072bb0 con 0x7f379c0b4de0 2024-04-17T10:04:11.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.354+0000 7f379b400640 1 -- 172.21.15.32:0/2161205476 >> v1:172.21.15.32:6789/0 conn(0x7f379c0b4de0 legacy=0x7f379c15baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:11.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.354+0000 7f379b400640 1 -- 172.21.15.32:0/2161205476 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379c0b40a0 con 0x7f379c144e90 2024-04-17T10:04:11.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.358+0000 7f379b400640 1 --2- 172.21.15.32:0/2161205476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c144e90 0x7f379c0b48a0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f378c009870 tx=0x7f378c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdd249d77aaf553 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:11.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.358+0000 7f379aa00640 1 -- 172.21.15.32:0/2161205476 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c0a9610 con 0x7f379c144e90 2024-04-17T10:04:11.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.358+0000 7f379aa00640 1 -- 172.21.15.32:0/2161205476 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f378c0a97b0 con 0x7f379c144e90 2024-04-17T10:04:11.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.358+0000 7f379aa00640 1 -- 172.21.15.32:0/2161205476 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c0a9b80 con 0x7f379c144e90 2024-04-17T10:04:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.358+0000 7f37a2200640 1 -- 172.21.15.32:0/2161205476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c144e90 msgr2=0x7f379c0b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.358+0000 7f37a2200640 1 --2- 172.21.15.32:0/2161205476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c144e90 0x7f379c0b48a0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f378c009870 tx=0x7f378c01c530 comp rx=0 tx=0).stop 2024-04-17T10:04:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.358+0000 7f37a2200640 1 -- 172.21.15.32:0/2161205476 shutdown_connections 2024-04-17T10:04:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.358+0000 7f37a2200640 1 --2- 172.21.15.32:0/2161205476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c144e90 0x7f379c0b48a0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.358+0000 7f37a2200640 1 -- 172.21.15.32:0/2161205476 >> 172.21.15.32:0/2161205476 conn(0x7f379c06ba30 msgr2=0x7f379c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.358+0000 7f37a2200640 1 -- 172.21.15.32:0/2161205476 shutdown_connections 2024-04-17T10:04:11.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.358+0000 7f37a2200640 1 -- 172.21.15.32:0/2161205476 wait complete. 2024-04-17T10:04:11.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.358+0000 7f37a2200640 1 Processor -- start 2024-04-17T10:04:11.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.358+0000 7f37a2200640 1 -- start start 2024-04-17T10:04:11.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f37a2200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f379c0b4de0 0x7f379c0cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:11.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f37a2200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c144e90 0x7f379c13fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:11.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f37a2200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f379c15c1b0 0x7f379c0b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:11.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f37a2200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f379c169120 con 0x7f379c144e90 2024-04-17T10:04:11.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f37a2200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f379c072880 con 0x7f379c15c1b0 2024-04-17T10:04:11.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f37a2200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f379c1692a0 con 0x7f379c0b4de0 2024-04-17T10:04:11.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f3792a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c144e90 0x7f379c13fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:11.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f3792a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c144e90 0x7f379c13fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50508/0 (socket says 172.21.15.32:50508) 2024-04-17T10:04:11.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f3792a00640 1 -- 172.21.15.32:0/3568205215 learned_addr learned my addr 172.21.15.32:0/3568205215 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:11.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f379be00640 1 --2- 172.21.15.32:0/3568205215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f379c15c1b0 0x7f379c0b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:11.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f379b400640 1 --2- 172.21.15.32:0/3568205215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f379c0b4de0 0x7f379c0cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:11.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f3792a00640 1 -- 172.21.15.32:0/3568205215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f379c0b4de0 msgr2=0x7f379c0cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:11.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f3792a00640 1 --2- 172.21.15.32:0/3568205215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f379c0b4de0 0x7f379c0cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:11.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f3792a00640 1 -- 172.21.15.32:0/3568205215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f379c15c1b0 msgr2=0x7f379c0b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:11.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f3792a00640 1 --2- 172.21.15.32:0/3568205215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f379c15c1b0 0x7f379c0b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:11.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f3792a00640 1 -- 172.21.15.32:0/3568205215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379c0b3000 con 0x7f379c144e90 2024-04-17T10:04:11.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f3792a00640 1 --2- 172.21.15.32:0/3568205215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c144e90 0x7f379c13fd30 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f378400c480 tx=0x7f378400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:11.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f3793e00640 1 -- 172.21.15.32:0/3568205215 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784018070 con 0x7f379c144e90 2024-04-17T10:04:11.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f3793e00640 1 -- 172.21.15.32:0/3568205215 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37840074a0 con 0x7f379c144e90 2024-04-17T10:04:11.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f37a2200640 1 -- 172.21.15.32:0/3568205215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379c0b0630 con 0x7f379c144e90 2024-04-17T10:04:11.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f3793e00640 1 -- 172.21.15.32:0/3568205215 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784004a30 con 0x7f379c144e90 2024-04-17T10:04:11.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f37a2200640 1 -- 172.21.15.32:0/3568205215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f379c0b0bf0 con 0x7f379c144e90 2024-04-17T10:04:11.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f37a2200640 1 -- 172.21.15.32:0/3568205215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f379c068a90 con 0x7f379c144e90 2024-04-17T10:04:11.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.362+0000 7f3793e00640 1 -- 172.21.15.32:0/3568205215 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f378401b070 con 0x7f379c144e90 2024-04-17T10:04:11.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.366+0000 7f3793e00640 1 --2- 172.21.15.32:0/3568205215 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3778068380 0x7f377806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:11.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.366+0000 7f3793e00640 1 -- 172.21.15.32:0/3568205215 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7f3784085ce0 con 0x7f379c144e90 2024-04-17T10:04:11.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.366+0000 7f379b400640 1 --2- 172.21.15.32:0/3568205215 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3778068380 0x7f377806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:11.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.366+0000 7f379b400640 1 --2- 172.21.15.32:0/3568205215 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3778068380 0x7f377806a840 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f378c0ae000 tx=0x7f378c0b3040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:11.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.370+0000 7f3793e00640 1 -- 172.21.15.32:0/3568205215 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3784055bb0 con 0x7f379c144e90 2024-04-17T10:04:11.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.534+0000 7f37a2200640 1 -- 172.21.15.32:0/3568205215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f379c0b55b0 con 0x7f379c144e90 2024-04-17T10:04:11.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.534+0000 7f3793e00640 1 -- 172.21.15.32:0/3568205215 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+16863 (secure 0 0 0) 0x7f3784059eb0 con 0x7f379c144e90 2024-04-17T10:04:11.535 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:11.535 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":112,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:04:03.080842+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:03:36.021322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-17T10:03:36.581134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.5","mappings":[{"from":0,"to":5}]},{"pgid":"17.25","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6837/901447746":"2024-04-18T10:04:03.080821+0000","172.21.15.32:6836/901447746":"2024-04-18T10:04:03.080821+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:04:11.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.538+0000 7f37a2200640 1 -- 172.21.15.32:0/3568205215 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3778068380 msgr2=0x7f377806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:11.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.538+0000 7f37a2200640 1 --2- 172.21.15.32:0/3568205215 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3778068380 0x7f377806a840 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f378c0ae000 tx=0x7f378c0b3040 comp rx=0 tx=0).stop 2024-04-17T10:04:11.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.538+0000 7f37a2200640 1 -- 172.21.15.32:0/3568205215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c144e90 msgr2=0x7f379c13fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:11.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.538+0000 7f37a2200640 1 --2- 172.21.15.32:0/3568205215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c144e90 0x7f379c13fd30 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f378400c480 tx=0x7f378400c940 comp rx=0 tx=0).stop 2024-04-17T10:04:11.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.538+0000 7f37a2200640 1 -- 172.21.15.32:0/3568205215 shutdown_connections 2024-04-17T10:04:11.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.538+0000 7f37a2200640 1 --2- 172.21.15.32:0/3568205215 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3778068380 0x7f377806a840 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:11.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.538+0000 7f37a2200640 1 --2- 172.21.15.32:0/3568205215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f379c15c1b0 0x7f379c0b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:11.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.538+0000 7f37a2200640 1 --2- 172.21.15.32:0/3568205215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f379c144e90 0x7f379c13fd30 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:11.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.538+0000 7f37a2200640 1 --2- 172.21.15.32:0/3568205215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f379c0b4de0 0x7f379c0cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:11.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.538+0000 7f37a2200640 1 -- 172.21.15.32:0/3568205215 >> 172.21.15.32:0/3568205215 conn(0x7f379c06ba30 msgr2=0x7f379c06bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:11.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.538+0000 7f37a2200640 1 -- 172.21.15.32:0/3568205215 shutdown_connections 2024-04-17T10:04:11.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.538+0000 7f37a2200640 1 -- 172.21.15.32:0/3568205215 wait complete. 2024-04-17T10:04:11.557 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:03:36.021322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:03:36.581134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:04:11.557 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:04:11.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.702+0000 7fbae2400640 1 Processor -- start 2024-04-17T10:04:11.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.702+0000 7fbae2400640 1 -- start start 2024-04-17T10:04:11.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.702+0000 7fbae2400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbadc06ba30 0x7fbadc06be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:11.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.702+0000 7fbae2400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbadc071e60 0x7fbadc072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:11.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.702+0000 7fbae2400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbadc06c350 con 0x7fbadc0cf370 2024-04-17T10:04:11.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.702+0000 7fbae2400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fbadc0cfef0 con 0x7fbadc071e60 2024-04-17T10:04:11.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.702+0000 7fbae2400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fbadc0d0130 con 0x7fbadc06ba30 2024-04-17T10:04:11.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.702+0000 7fbadbe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbadc071e60 0x7fbadc072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:11.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.702+0000 7fbadbe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbadc071e60 0x7fbadc072240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50516/0 (socket says 172.21.15.32:50516) 2024-04-17T10:04:11.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.702+0000 7fbadbe00640 1 -- 172.21.15.32:0/1249576920 learned_addr learned my addr 172.21.15.32:0/1249576920 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:11.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.706+0000 7fbadbe00640 1 -- 172.21.15.32:0/1249576920 >> v2:172.21.15.144:3300/0 conn(0x7fbadc06ba30 msgr2=0x7fbadc06be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:11.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.706+0000 7fbadb400640 1 --2- 172.21.15.32:0/1249576920 >> v2:172.21.15.144:3300/0 conn(0x7fbadc06ba30 0x7fbadc06be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.706+0000 7fbadbe00640 1 --2- 172.21.15.32:0/1249576920 >> v2:172.21.15.144:3300/0 conn(0x7fbadc06ba30 0x7fbadc06be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.706+0000 7fbadbe00640 1 -- 172.21.15.32:0/1249576920 >> v1:172.21.15.144:6789/0 conn(0x7fbadc0cf370 legacy=0x7fbadc0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.706+0000 7fbadbe00640 1 -- 172.21.15.32:0/1249576920 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbadc15d090 con 0x7fbadc071e60 2024-04-17T10:04:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.706+0000 7fbadb400640 1 --2- 172.21.15.32:0/1249576920 >> v2:172.21.15.144:3300/0 conn(0x7fbadc06ba30 0x7fbadc06be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.706+0000 7fbadbe00640 1 --2- 172.21.15.32:0/1249576920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbadc071e60 0x7fbadc072240 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fbadc0b5730 tx=0x7fbad401c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8516a3030e43a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:11.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.706+0000 7fbadaa00640 1 -- 172.21.15.32:0/1249576920 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad40a56b0 con 0x7fbadc071e60 2024-04-17T10:04:11.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.706+0000 7fbadaa00640 1 -- 172.21.15.32:0/1249576920 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbad40a5850 con 0x7fbadc071e60 2024-04-17T10:04:11.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.706+0000 7fbadaa00640 1 -- 172.21.15.32:0/1249576920 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad40a5c20 con 0x7fbadc071e60 2024-04-17T10:04:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.706+0000 7fbae2400640 1 -- 172.21.15.32:0/1249576920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbadc071e60 msgr2=0x7fbadc072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.706+0000 7fbae2400640 1 --2- 172.21.15.32:0/1249576920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbadc071e60 0x7fbadc072240 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fbadc0b5730 tx=0x7fbad401c8c0 comp rx=0 tx=0).stop 2024-04-17T10:04:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.706+0000 7fbae2400640 1 -- 172.21.15.32:0/1249576920 shutdown_connections 2024-04-17T10:04:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.706+0000 7fbae2400640 1 --2- 172.21.15.32:0/1249576920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbadc071e60 0x7fbadc072240 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.706+0000 7fbae2400640 1 --2- 172.21.15.32:0/1249576920 >> v2:172.21.15.144:3300/0 conn(0x7fbadc06ba30 0x7fbadc06be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.706+0000 7fbae2400640 1 -- 172.21.15.32:0/1249576920 >> 172.21.15.32:0/1249576920 conn(0x7fbadc0b2120 msgr2=0x7fbadc0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.710+0000 7fbae2400640 1 -- 172.21.15.32:0/1249576920 shutdown_connections 2024-04-17T10:04:11.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.710+0000 7fbae2400640 1 -- 172.21.15.32:0/1249576920 wait complete. 2024-04-17T10:04:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.710+0000 7fbae2400640 1 Processor -- start 2024-04-17T10:04:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.710+0000 7fbae2400640 1 -- start start 2024-04-17T10:04:11.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.710+0000 7fbae2400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbadc06ba30 0x7fbadc144e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:11.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.710+0000 7fbae2400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbadc071e60 0x7fbadc0aae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:11.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.710+0000 7fbae2400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbadc0cf370 0x7fbadc0aa060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:11.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.710+0000 7fbae2400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc15dcf0 con 0x7fbadc06ba30 2024-04-17T10:04:11.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.710+0000 7fbae2400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc15db70 con 0x7fbadc0cf370 2024-04-17T10:04:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.710+0000 7fbae2400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbadc146230 con 0x7fbadc071e60 2024-04-17T10:04:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.710+0000 7fbadb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbadc06ba30 0x7fbadc144e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.710+0000 7fbadb400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbadc06ba30 0x7fbadc144e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50532/0 (socket says 172.21.15.32:50532) 2024-04-17T10:04:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.710+0000 7fbadb400640 1 -- 172.21.15.32:0/4197986952 learned_addr learned my addr 172.21.15.32:0/4197986952 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.710+0000 7fbadbe00640 1 --2- 172.21.15.32:0/4197986952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbadc0cf370 0x7fbadc0aa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.710+0000 7fbad2a00640 1 --2- 172.21.15.32:0/4197986952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbadc071e60 0x7fbadc0aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.710+0000 7fbadb400640 1 -- 172.21.15.32:0/4197986952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbadc071e60 msgr2=0x7fbadc0aae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.710+0000 7fbadb400640 1 --2- 172.21.15.32:0/4197986952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbadc071e60 0x7fbadc0aae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.710+0000 7fbadb400640 1 -- 172.21.15.32:0/4197986952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbadc0cf370 msgr2=0x7fbadc0aa060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.710+0000 7fbadb400640 1 --2- 172.21.15.32:0/4197986952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbadc0cf370 0x7fbadc0aa060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.710+0000 7fbadb400640 1 -- 172.21.15.32:0/4197986952 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbadc0ab890 con 0x7fbadc06ba30 2024-04-17T10:04:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.710+0000 7fbadb400640 1 --2- 172.21.15.32:0/4197986952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbadc06ba30 0x7fbadc144e00 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00c7c0 tx=0x7fbacc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.714+0000 7fbad3e00640 1 -- 172.21.15.32:0/4197986952 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc007a30 con 0x7fbadc06ba30 2024-04-17T10:04:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.714+0000 7fbad3e00640 1 -- 172.21.15.32:0/4197986952 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbacc007bd0 con 0x7fbadc06ba30 2024-04-17T10:04:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.714+0000 7fbad3e00640 1 -- 172.21.15.32:0/4197986952 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc0113f0 con 0x7fbadc06ba30 2024-04-17T10:04:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.714+0000 7fbae2400640 1 -- 172.21.15.32:0/4197986952 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbadc0a9580 con 0x7fbadc06ba30 2024-04-17T10:04:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.714+0000 7fbae2400640 1 -- 172.21.15.32:0/4197986952 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbadc0abb70 con 0x7fbadc06ba30 2024-04-17T10:04:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.714+0000 7fbae2400640 1 -- 172.21.15.32:0/4197986952 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbadc068a90 con 0x7fbadc06ba30 2024-04-17T10:04:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.714+0000 7fbad3e00640 1 -- 172.21.15.32:0/4197986952 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbacc011590 con 0x7fbadc06ba30 2024-04-17T10:04:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.714+0000 7fbad3e00640 1 --2- 172.21.15.32:0/4197986952 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbab8068380 0x7fbab806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.714+0000 7fbad3e00640 1 -- 172.21.15.32:0/4197986952 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7fbacc086c50 con 0x7fbadc06ba30 2024-04-17T10:04:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.714+0000 7fbad2a00640 1 --2- 172.21.15.32:0/4197986952 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbab8068380 0x7fbab806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.714+0000 7fbad2a00640 1 --2- 172.21.15.32:0/4197986952 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbab8068380 0x7fbab806a840 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fbadc13f0f0 tx=0x7fbac4009060 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.718+0000 7fbad3e00640 1 -- 172.21.15.32:0/4197986952 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fbacc015020 con 0x7fbadc06ba30 2024-04-17T10:04:11.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.894+0000 7fbae2400640 1 -- 172.21.15.32:0/4197986952 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbadc06c850 con 0x7fbadc06ba30 2024-04-17T10:04:11.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.894+0000 7fbad3e00640 1 -- 172.21.15.32:0/4197986952 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) ==== 93+0+10 (secure 0 0 0) 0x7fbacc056a90 con 0x7fbadc06ba30 2024-04-17T10:04:11.893 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T10:04:11.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.894+0000 7fbae2400640 1 -- 172.21.15.32:0/4197986952 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbab8068380 msgr2=0x7fbab806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:11.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.894+0000 7fbae2400640 1 --2- 172.21.15.32:0/4197986952 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbab8068380 0x7fbab806a840 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fbadc13f0f0 tx=0x7fbac4009060 comp rx=0 tx=0).stop 2024-04-17T10:04:11.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.894+0000 7fbae2400640 1 -- 172.21.15.32:0/4197986952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbadc06ba30 msgr2=0x7fbadc144e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:11.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.894+0000 7fbae2400640 1 --2- 172.21.15.32:0/4197986952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbadc06ba30 0x7fbadc144e00 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00c7c0 tx=0x7fbacc00cc80 comp rx=0 tx=0).stop 2024-04-17T10:04:11.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.894+0000 7fbae2400640 1 -- 172.21.15.32:0/4197986952 shutdown_connections 2024-04-17T10:04:11.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.894+0000 7fbae2400640 1 --2- 172.21.15.32:0/4197986952 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbab8068380 0x7fbab806a840 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:11.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.894+0000 7fbae2400640 1 --2- 172.21.15.32:0/4197986952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbadc071e60 0x7fbadc0aae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:11.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.894+0000 7fbae2400640 1 --2- 172.21.15.32:0/4197986952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbadc0cf370 0x7fbadc0aa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:11.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.894+0000 7fbae2400640 1 --2- 172.21.15.32:0/4197986952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbadc06ba30 0x7fbadc144e00 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:11.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.894+0000 7fbae2400640 1 -- 172.21.15.32:0/4197986952 >> 172.21.15.32:0/4197986952 conn(0x7fbadc0b2120 msgr2=0x7fbadc0b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:11.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.894+0000 7fbae2400640 1 -- 172.21.15.32:0/4197986952 shutdown_connections 2024-04-17T10:04:11.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:11.894+0000 7fbae2400640 1 -- 172.21.15.32:0/4197986952 wait complete. 2024-04-17T10:04:11.914 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:04:12.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.058+0000 7f9b19e00640 1 Processor -- start 2024-04-17T10:04:12.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.058+0000 7f9b19e00640 1 -- start start 2024-04-17T10:04:12.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.058+0000 7f9b19e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b140cf370 0x7f9b140cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:12.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.058+0000 7f9b19e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b14071e60 0x7f9b140722d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:12.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.058+0000 7f9b19e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b140d0130 con 0x7f9b1406ba30 2024-04-17T10:04:12.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.058+0000 7f9b19e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f9b1415d090 con 0x7f9b14071e60 2024-04-17T10:04:12.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.058+0000 7f9b19e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9b1415d370 con 0x7f9b140cf370 2024-04-17T10:04:12.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.058+0000 7f9b13400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b14071e60 0x7f9b140722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:12.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.058+0000 7f9b13400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b14071e60 0x7f9b140722d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50546/0 (socket says 172.21.15.32:50546) 2024-04-17T10:04:12.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.058+0000 7f9b13400640 1 -- 172.21.15.32:0/3457594908 learned_addr learned my addr 172.21.15.32:0/3457594908 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.058+0000 7f9b0a000640 1 --2- 172.21.15.32:0/3457594908 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b140cf370 0x7f9b140cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.058+0000 7f9b13400640 1 -- 172.21.15.32:0/3457594908 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b140cf370 msgr2=0x7f9b140cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.058+0000 7f9b13400640 1 --2- 172.21.15.32:0/3457594908 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b140cf370 0x7f9b140cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.058+0000 7f9b13400640 1 -- 172.21.15.32:0/3457594908 >> v1:172.21.15.144:6789/0 conn(0x7f9b1406ba30 legacy=0x7f9b1406be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.058+0000 7f9b13400640 1 -- 172.21.15.32:0/3457594908 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1415d650 con 0x7f9b14071e60 2024-04-17T10:04:12.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.062+0000 7f9b13400640 1 --2- 172.21.15.32:0/3457594908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b14071e60 0x7f9b140722d0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009870 tx=0x7f9b0c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a28567e941823b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:12.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.062+0000 7f9b12000640 1 -- 172.21.15.32:0/3457594908 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c0a9640 con 0x7f9b14071e60 2024-04-17T10:04:12.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.062+0000 7f9b12000640 1 -- 172.21.15.32:0/3457594908 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b0c0a97e0 con 0x7f9b14071e60 2024-04-17T10:04:12.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.062+0000 7f9b12000640 1 -- 172.21.15.32:0/3457594908 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c0a9bb0 con 0x7f9b14071e60 2024-04-17T10:04:12.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.062+0000 7f9b19e00640 1 -- 172.21.15.32:0/3457594908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b14071e60 msgr2=0x7f9b140722d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:12.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.062+0000 7f9b19e00640 1 --2- 172.21.15.32:0/3457594908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b14071e60 0x7f9b140722d0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009870 tx=0x7f9b0c01c560 comp rx=0 tx=0).stop 2024-04-17T10:04:12.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.062+0000 7f9b19e00640 1 -- 172.21.15.32:0/3457594908 shutdown_connections 2024-04-17T10:04:12.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.062+0000 7f9b19e00640 1 --2- 172.21.15.32:0/3457594908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b14071e60 0x7f9b140722d0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.062+0000 7f9b19e00640 1 --2- 172.21.15.32:0/3457594908 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b140cf370 0x7f9b140cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.062+0000 7f9b19e00640 1 -- 172.21.15.32:0/3457594908 >> 172.21.15.32:0/3457594908 conn(0x7f9b140b2120 msgr2=0x7f9b140b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:12.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.062+0000 7f9b19e00640 1 -- 172.21.15.32:0/3457594908 shutdown_connections 2024-04-17T10:04:12.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.062+0000 7f9b19e00640 1 -- 172.21.15.32:0/3457594908 wait complete. 2024-04-17T10:04:12.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b19e00640 1 Processor -- start 2024-04-17T10:04:12.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b19e00640 1 -- start start 2024-04-17T10:04:12.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b19e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b1406ba30 0x7f9b14144210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:12.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b19e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b14071e60 0x7f9b140b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:12.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b19e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b140cf370 0x7f9b140b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:12.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b19e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9b14146230 con 0x7f9b14071e60 2024-04-17T10:04:12.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b19e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1415dd80 con 0x7f9b1406ba30 2024-04-17T10:04:12.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b19e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9b141463b0 con 0x7f9b140cf370 2024-04-17T10:04:12.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b0a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b14071e60 0x7f9b140b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:12.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b0a000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b14071e60 0x7f9b140b4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50560/0 (socket says 172.21.15.32:50560) 2024-04-17T10:04:12.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b0a000640 1 -- 172.21.15.32:0/836576124 learned_addr learned my addr 172.21.15.32:0/836576124 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:12.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b12a00640 1 --2- 172.21.15.32:0/836576124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b1406ba30 0x7f9b14144210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:12.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b13400640 1 --2- 172.21.15.32:0/836576124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b140cf370 0x7f9b140b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:12.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b0a000640 1 -- 172.21.15.32:0/836576124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b140cf370 msgr2=0x7f9b140b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:12.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b0a000640 1 --2- 172.21.15.32:0/836576124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b140cf370 0x7f9b140b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b0a000640 1 -- 172.21.15.32:0/836576124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b1406ba30 msgr2=0x7f9b14144210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:12.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b0a000640 1 --2- 172.21.15.32:0/836576124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b1406ba30 0x7f9b14144210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b0a000640 1 -- 172.21.15.32:0/836576124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b141f7b50 con 0x7f9b14071e60 2024-04-17T10:04:12.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b0a000640 1 --2- 172.21.15.32:0/836576124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b14071e60 0x7f9b140b4260 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00e870 tx=0x7f9afc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:12.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b0be00640 1 -- 172.21.15.32:0/836576124 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc017070 con 0x7f9b14071e60 2024-04-17T10:04:12.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b0be00640 1 -- 172.21.15.32:0/836576124 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9afc00c940 con 0x7f9b14071e60 2024-04-17T10:04:12.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b0be00640 1 -- 172.21.15.32:0/836576124 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc00ccf0 con 0x7f9b14071e60 2024-04-17T10:04:12.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b19e00640 1 -- 172.21.15.32:0/836576124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b141f7e30 con 0x7f9b14071e60 2024-04-17T10:04:12.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.066+0000 7f9b19e00640 1 -- 172.21.15.32:0/836576124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b141f83f0 con 0x7f9b14071e60 2024-04-17T10:04:12.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.070+0000 7f9b19e00640 1 -- 172.21.15.32:0/836576124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b14068a90 con 0x7f9b14071e60 2024-04-17T10:04:12.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.070+0000 7f9b0be00640 1 -- 172.21.15.32:0/836576124 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9afc01a070 con 0x7f9b14071e60 2024-04-17T10:04:12.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.070+0000 7f9b0be00640 1 --2- 172.21.15.32:0/836576124 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9af0068380 0x7f9af006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:12.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.070+0000 7f9b0be00640 1 -- 172.21.15.32:0/836576124 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7f9afc086910 con 0x7f9b14071e60 2024-04-17T10:04:12.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.070+0000 7f9b12a00640 1 --2- 172.21.15.32:0/836576124 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9af0068380 0x7f9af006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:12.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.070+0000 7f9b12a00640 1 --2- 172.21.15.32:0/836576124 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9af0068380 0x7f9af006a840 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9b140b1830 tx=0x7f9b04009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:12.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.074+0000 7f9b0be00640 1 -- 172.21.15.32:0/836576124 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f9afc056750 con 0x7f9b14071e60 2024-04-17T10:04:12.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.242+0000 7f9b19e00640 1 -- 172.21.15.32:0/836576124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9b1406c850 con 0x7f9b14071e60 2024-04-17T10:04:12.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.242+0000 7f9b0be00640 1 -- 172.21.15.32:0/836576124 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) ==== 104+0+11 (secure 0 0 0) 0x7f9afc05aa50 con 0x7f9b14071e60 2024-04-17T10:04:12.243 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T10:04:12.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.246+0000 7f9b19e00640 1 -- 172.21.15.32:0/836576124 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9af0068380 msgr2=0x7f9af006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:12.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.246+0000 7f9b19e00640 1 --2- 172.21.15.32:0/836576124 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9af0068380 0x7f9af006a840 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9b140b1830 tx=0x7f9b04009040 comp rx=0 tx=0).stop 2024-04-17T10:04:12.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.246+0000 7f9b19e00640 1 -- 172.21.15.32:0/836576124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b14071e60 msgr2=0x7f9b140b4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:12.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.246+0000 7f9b19e00640 1 --2- 172.21.15.32:0/836576124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b14071e60 0x7f9b140b4260 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00e870 tx=0x7f9afc00ec70 comp rx=0 tx=0).stop 2024-04-17T10:04:12.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.246+0000 7f9b19e00640 1 -- 172.21.15.32:0/836576124 shutdown_connections 2024-04-17T10:04:12.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.246+0000 7f9b19e00640 1 --2- 172.21.15.32:0/836576124 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9af0068380 0x7f9af006a840 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.246+0000 7f9b19e00640 1 --2- 172.21.15.32:0/836576124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b140cf370 0x7f9b140b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.246+0000 7f9b19e00640 1 --2- 172.21.15.32:0/836576124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b14071e60 0x7f9b140b4260 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.246+0000 7f9b19e00640 1 --2- 172.21.15.32:0/836576124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b1406ba30 0x7f9b14144210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.246+0000 7f9b19e00640 1 -- 172.21.15.32:0/836576124 >> 172.21.15.32:0/836576124 conn(0x7f9b140b2120 msgr2=0x7f9b140b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:12.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.246+0000 7f9b19e00640 1 -- 172.21.15.32:0/836576124 shutdown_connections 2024-04-17T10:04:12.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.246+0000 7f9b19e00640 1 -- 172.21.15.32:0/836576124 wait complete. 2024-04-17T10:04:12.265 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:04:12.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.406+0000 7fcbc1000640 1 Processor -- start 2024-04-17T10:04:12.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.410+0000 7fcbc1000640 1 -- start start 2024-04-17T10:04:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.410+0000 7fcbc1000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbbc0d0650 0x7fcbbc13fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.410+0000 7fcbc1000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcbbc06d150 0x7fcbbc160560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.410+0000 7fcbc1000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbbc15d070 con 0x7fcbbc0af490 2024-04-17T10:04:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.410+0000 7fcbc1000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fcbbc15dc80 con 0x7fcbbc06d150 2024-04-17T10:04:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.410+0000 7fcbc1000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcbbc15d350 con 0x7fcbbc0d0650 2024-04-17T10:04:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.410+0000 7fcbba800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcbbc06d150 0x7fcbbc160560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.410+0000 7fcbba800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcbbc06d150 0x7fcbbc160560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50572/0 (socket says 172.21.15.32:50572) 2024-04-17T10:04:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.410+0000 7fcbb3e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcbbc0af490 0x7fcbbc06ca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59074/0 (socket says 172.21.15.32:59074) 2024-04-17T10:04:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.410+0000 7fcbba800640 1 -- 172.21.15.32:0/389614422 learned_addr learned my addr 172.21.15.32:0/389614422 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:12.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.410+0000 7fcbb9e00640 1 --2- 172.21.15.32:0/389614422 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbbc0d0650 0x7fcbbc13fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:12.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.410+0000 7fcbba800640 1 -- 172.21.15.32:0/389614422 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbbc0d0650 msgr2=0x7fcbbc13fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:12.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.410+0000 7fcbba800640 1 --2- 172.21.15.32:0/389614422 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbbc0d0650 0x7fcbbc13fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.410+0000 7fcbba800640 1 -- 172.21.15.32:0/389614422 >> v1:172.21.15.32:6789/0 conn(0x7fcbbc0af490 legacy=0x7fcbbc06ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:12.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.410+0000 7fcbba800640 1 -- 172.21.15.32:0/389614422 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbbc15d630 con 0x7fcbbc06d150 2024-04-17T10:04:12.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.414+0000 7fcbba800640 1 --2- 172.21.15.32:0/389614422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbbc06d150 0x7fcbbc160560 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009e50 tx=0x7fcbb401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b65eee1d4b67a26 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:12.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.414+0000 7fcbb9400640 1 -- 172.21.15.32:0/389614422 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40a58c0 con 0x7fcbbc06d150 2024-04-17T10:04:12.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.414+0000 7fcbb9400640 1 -- 172.21.15.32:0/389614422 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb40a5a60 con 0x7fcbbc06d150 2024-04-17T10:04:12.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.414+0000 7fcbb9400640 1 -- 172.21.15.32:0/389614422 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40a5e30 con 0x7fcbbc06d150 2024-04-17T10:04:12.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.414+0000 7fcbc1000640 1 -- 172.21.15.32:0/389614422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbbc06d150 msgr2=0x7fcbbc160560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:12.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.414+0000 7fcbc1000640 1 --2- 172.21.15.32:0/389614422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbbc06d150 0x7fcbbc160560 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009e50 tx=0x7fcbb401ca30 comp rx=0 tx=0).stop 2024-04-17T10:04:12.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.414+0000 7fcbc1000640 1 -- 172.21.15.32:0/389614422 shutdown_connections 2024-04-17T10:04:12.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.414+0000 7fcbc1000640 1 --2- 172.21.15.32:0/389614422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbbc06d150 0x7fcbbc160560 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.414+0000 7fcbc1000640 1 --2- 172.21.15.32:0/389614422 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbbc0d0650 0x7fcbbc13fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.414+0000 7fcbc1000640 1 -- 172.21.15.32:0/389614422 >> 172.21.15.32:0/389614422 conn(0x7fcbbc072670 msgr2=0x7fcbbc072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:12.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.414+0000 7fcbc1000640 1 -- 172.21.15.32:0/389614422 shutdown_connections 2024-04-17T10:04:12.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.414+0000 7fcbc1000640 1 -- 172.21.15.32:0/389614422 wait complete. 2024-04-17T10:04:12.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.414+0000 7fcbc1000640 1 Processor -- start 2024-04-17T10:04:12.415 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.414+0000 7fcbc1000640 1 -- start start 2024-04-17T10:04:12.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbc1000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbbc06d150 0x7fcbbc0b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:12.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbc1000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbbc0af490 0x7fcbbc0b5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:12.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbc1000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbbc0d0650 0x7fcbbc0cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:12.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbc1000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc15ce10 con 0x7fcbbc0d0650 2024-04-17T10:04:12.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbc1000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc15cc90 con 0x7fcbbc0af490 2024-04-17T10:04:12.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbc1000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcbbc169620 con 0x7fcbbc06d150 2024-04-17T10:04:12.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbba800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbbc0d0650 0x7fcbbc0cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:12.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbba800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbbc0d0650 0x7fcbbc0cdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50584/0 (socket says 172.21.15.32:50584) 2024-04-17T10:04:12.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbba800640 1 -- 172.21.15.32:0/3340374526 learned_addr learned my addr 172.21.15.32:0/3340374526 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:12.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbb3e00640 1 --2- 172.21.15.32:0/3340374526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbbc0af490 0x7fcbbc0b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:12.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbb9e00640 1 --2- 172.21.15.32:0/3340374526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbbc06d150 0x7fcbbc0b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:12.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbba800640 1 -- 172.21.15.32:0/3340374526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbbc06d150 msgr2=0x7fcbbc0b1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:12.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbba800640 1 --2- 172.21.15.32:0/3340374526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbbc06d150 0x7fcbbc0b1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbba800640 1 -- 172.21.15.32:0/3340374526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbbc0af490 msgr2=0x7fcbbc0b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:12.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbba800640 1 --2- 172.21.15.32:0/3340374526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbbc0af490 0x7fcbbc0b5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbba800640 1 -- 172.21.15.32:0/3340374526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbbc153960 con 0x7fcbbc0d0650 2024-04-17T10:04:12.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbba800640 1 --2- 172.21.15.32:0/3340374526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbbc0d0650 0x7fcbbc0cdea0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009e50 tx=0x7fcbb40197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:12.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbb2000640 1 -- 172.21.15.32:0/3340374526 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40bc020 con 0x7fcbbc0d0650 2024-04-17T10:04:12.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbb2000640 1 -- 172.21.15.32:0/3340374526 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb40b8470 con 0x7fcbbc0d0650 2024-04-17T10:04:12.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbc1000640 1 -- 172.21.15.32:0/3340374526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbbc0cf360 con 0x7fcbbc0d0650 2024-04-17T10:04:12.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbb2000640 1 -- 172.21.15.32:0/3340374526 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40a58c0 con 0x7fcbbc0d0650 2024-04-17T10:04:12.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbc1000640 1 -- 172.21.15.32:0/3340374526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbbc0cf920 con 0x7fcbbc0d0650 2024-04-17T10:04:12.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbc1000640 1 -- 172.21.15.32:0/3340374526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbbc068a90 con 0x7fcbbc0d0650 2024-04-17T10:04:12.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.418+0000 7fcbb2000640 1 -- 172.21.15.32:0/3340374526 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcbb40bf020 con 0x7fcbbc0d0650 2024-04-17T10:04:12.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.422+0000 7fcbb2000640 1 --2- 172.21.15.32:0/3340374526 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcb98068380 0x7fcb9806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:12.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.422+0000 7fcbb2000640 1 -- 172.21.15.32:0/3340374526 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7fcbb4121e60 con 0x7fcbbc0d0650 2024-04-17T10:04:12.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.422+0000 7fcbb9e00640 1 --2- 172.21.15.32:0/3340374526 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcb98068380 0x7fcb9806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:12.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.422+0000 7fcbb9e00640 1 --2- 172.21.15.32:0/3340374526 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcb98068380 0x7fcb9806a840 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00b930 tx=0x7fcbac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:12.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.426+0000 7fcbb2000640 1 -- 172.21.15.32:0/3340374526 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fcbb40c6030 con 0x7fcbbc0d0650 2024-04-17T10:04:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.590+0000 7fcbc1000640 1 -- 172.21.15.32:0/3340374526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcbbc0737d0 con 0x7fcbbc0d0650 2024-04-17T10:04:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.590+0000 7fcbb2000640 1 -- 172.21.15.32:0/3340374526 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) ==== 100+0+11 (secure 0 0 0) 0x7fcbb4016620 con 0x7fcbbc0d0650 2024-04-17T10:04:12.589 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T10:04:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.590+0000 7fcbc1000640 1 -- 172.21.15.32:0/3340374526 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcb98068380 msgr2=0x7fcb9806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.590+0000 7fcbc1000640 1 --2- 172.21.15.32:0/3340374526 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcb98068380 0x7fcb9806a840 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00b930 tx=0x7fcbac008040 comp rx=0 tx=0).stop 2024-04-17T10:04:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.590+0000 7fcbc1000640 1 -- 172.21.15.32:0/3340374526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbbc0d0650 msgr2=0x7fcbbc0cdea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.590+0000 7fcbc1000640 1 --2- 172.21.15.32:0/3340374526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbbc0d0650 0x7fcbbc0cdea0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009e50 tx=0x7fcbb40197d0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.590+0000 7fcbc1000640 1 -- 172.21.15.32:0/3340374526 shutdown_connections 2024-04-17T10:04:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.590+0000 7fcbc1000640 1 --2- 172.21.15.32:0/3340374526 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcb98068380 0x7fcb9806a840 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.590+0000 7fcbc1000640 1 --2- 172.21.15.32:0/3340374526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbbc0d0650 0x7fcbbc0cdea0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.590+0000 7fcbc1000640 1 --2- 172.21.15.32:0/3340374526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbbc0af490 0x7fcbbc0b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.590+0000 7fcbc1000640 1 --2- 172.21.15.32:0/3340374526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbbc06d150 0x7fcbbc0b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.590+0000 7fcbc1000640 1 -- 172.21.15.32:0/3340374526 >> 172.21.15.32:0/3340374526 conn(0x7fcbbc072670 msgr2=0x7fcbbc06f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.590+0000 7fcbc1000640 1 -- 172.21.15.32:0/3340374526 shutdown_connections 2024-04-17T10:04:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.590+0000 7fcbc1000640 1 -- 172.21.15.32:0/3340374526 wait complete. 2024-04-17T10:04:12.611 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:04:12.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c2aa00640 1 Processor -- start 2024-04-17T10:04:12.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c2aa00640 1 -- start start 2024-04-17T10:04:12.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c2aa00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7c24155a70 0x7f7c2415f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:12.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c2aa00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c2415f870 con 0x7f7c24155e50 2024-04-17T10:04:12.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c2aa00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c2415fb50 con 0x7f7c240b3340 2024-04-17T10:04:12.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c2aa00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f7c240b42a0 con 0x7f7c24155a70 2024-04-17T10:04:12.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c23400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7c24155e50 0x7f7c2415d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59078/0 (socket says 172.21.15.32:59078) 2024-04-17T10:04:12.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c23400640 1 -- 172.21.15.32:0/2343878642 learned_addr learned my addr 172.21.15.32:0/2343878642 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:12.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c23e00640 1 --2- 172.21.15.32:0/2343878642 >> v2:172.21.15.144:3300/0 conn(0x7f7c24155a70 0x7f7c2415f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:12.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c22a00640 1 -- 172.21.15.32:0/2343878642 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 994140219 0 0) 0x7f7c2415f870 con 0x7f7c24155e50 2024-04-17T10:04:12.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c22a00640 1 -- 172.21.15.32:0/2343878642 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c08003530 con 0x7f7c24155e50 2024-04-17T10:04:12.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c22a00640 1 -- 172.21.15.32:0/2343878642 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 631179931 0 0) 0x7f7c08003530 con 0x7f7c24155e50 2024-04-17T10:04:12.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c22a00640 1 -- 172.21.15.32:0/2343878642 >> v2:172.21.15.144:3300/0 conn(0x7f7c24155a70 msgr2=0x7f7c2415f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:12.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c22a00640 1 --2- 172.21.15.32:0/2343878642 >> v2:172.21.15.144:3300/0 conn(0x7f7c24155a70 0x7f7c2415f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c22a00640 1 -- 172.21.15.32:0/2343878642 >> v1:172.21.15.144:6789/0 conn(0x7f7c240b3340 legacy=0x7f7c2415c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:12.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c22a00640 1 -- 172.21.15.32:0/2343878642 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2415fe30 con 0x7f7c24155e50 2024-04-17T10:04:12.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c22a00640 1 -- 172.21.15.32:0/2343878642 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f7c10001680 con 0x7f7c24155e50 2024-04-17T10:04:12.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c22a00640 1 -- 172.21.15.32:0/2343878642 >> v1:172.21.15.32:6789/0 conn(0x7f7c24155e50 legacy=0x7f7c2415d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:12.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c22a00640 1 --2- 172.21.15.32:0/2343878642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c080039c0 0x7f7c08003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:12.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c22a00640 1 --2- 172.21.15.32:0/2343878642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c08004810 0x7f7c08004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:12.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c22a00640 1 --2- 172.21.15.32:0/2343878642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c080051d0 0x7f7c0800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:12.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c22a00640 1 -- 172.21.15.32:0/2343878642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7c240b42a0 con 0x7f7c080039c0 2024-04-17T10:04:12.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c22a00640 1 -- 172.21.15.32:0/2343878642 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0800c050 con 0x7f7c08004810 2024-04-17T10:04:12.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c22a00640 1 -- 172.21.15.32:0/2343878642 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7c0800c250 con 0x7f7c080051d0 2024-04-17T10:04:12.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c23e00640 1 --2- 172.21.15.32:0/2343878642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c080039c0 0x7f7c08003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:12.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c23400640 1 --2- 172.21.15.32:0/2343878642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c08004810 0x7f7c08004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:12.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c28a00640 1 --2- 172.21.15.32:0/2343878642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c080051d0 0x7f7c0800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:12.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c23e00640 1 -- 172.21.15.32:0/2343878642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c080051d0 msgr2=0x7f7c0800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:12.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c23e00640 1 --2- 172.21.15.32:0/2343878642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c080051d0 0x7f7c0800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c23e00640 1 -- 172.21.15.32:0/2343878642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c08004810 msgr2=0x7f7c08004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:12.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c23e00640 1 --2- 172.21.15.32:0/2343878642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c08004810 0x7f7c08004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c23e00640 1 -- 172.21.15.32:0/2343878642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c08003530 con 0x7f7c080039c0 2024-04-17T10:04:12.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.754+0000 7f7c28a00640 1 -- 172.21.15.32:0/2343878642 reap_dead start 2024-04-17T10:04:12.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.758+0000 7f7c23e00640 1 --2- 172.21.15.32:0/2343878642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c080039c0 0x7f7c08003dc0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f7c14009ae0 tx=0x7f7c1401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835b5b5c382acee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:12.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.758+0000 7f7c22a00640 1 -- 172.21.15.32:0/2343878642 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c140b2070 con 0x7f7c080039c0 2024-04-17T10:04:12.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.758+0000 7f7c22a00640 1 -- 172.21.15.32:0/2343878642 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c140a5660 con 0x7f7c080039c0 2024-04-17T10:04:12.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.758+0000 7f7c22a00640 1 -- 172.21.15.32:0/2343878642 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c140096b0 con 0x7f7c080039c0 2024-04-17T10:04:12.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.758+0000 7f7c2aa00640 1 -- 172.21.15.32:0/2343878642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c080039c0 msgr2=0x7f7c08003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:12.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.758+0000 7f7c2aa00640 1 --2- 172.21.15.32:0/2343878642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c080039c0 0x7f7c08003dc0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f7c14009ae0 tx=0x7f7c1401ca70 comp rx=0 tx=0).stop 2024-04-17T10:04:12.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.758+0000 7f7c2aa00640 1 -- 172.21.15.32:0/2343878642 shutdown_connections 2024-04-17T10:04:12.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.758+0000 7f7c2aa00640 1 --2- 172.21.15.32:0/2343878642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c080039c0 0x7f7c08003dc0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.758+0000 7f7c2aa00640 1 -- 172.21.15.32:0/2343878642 >> 172.21.15.32:0/2343878642 conn(0x7f7c2406ba30 msgr2=0x7f7c2406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:12.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.758+0000 7f7c2aa00640 1 -- 172.21.15.32:0/2343878642 shutdown_connections 2024-04-17T10:04:12.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.758+0000 7f7c2aa00640 1 -- 172.21.15.32:0/2343878642 wait complete. 2024-04-17T10:04:12.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c2aa00640 1 Processor -- start 2024-04-17T10:04:12.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c2aa00640 1 -- start start 2024-04-17T10:04:12.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c2aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c080039c0 0x7f7c24151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:12.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c2aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c24152cb0 0x7f7c241f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:12.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c2aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c241f5f90 0x7f7c24071d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:12.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c2aa00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7c24160110 con 0x7f7c24152cb0 2024-04-17T10:04:12.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c2aa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7c240d1250 con 0x7f7c080039c0 2024-04-17T10:04:12.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c2aa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7c2415f190 con 0x7f7c241f5f90 2024-04-17T10:04:12.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c23400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c24152cb0 0x7f7c241f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:12.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c23400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c24152cb0 0x7f7c241f5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50600/0 (socket says 172.21.15.32:50600) 2024-04-17T10:04:12.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c23400640 1 -- 172.21.15.32:0/349628377 learned_addr learned my addr 172.21.15.32:0/349628377 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:12.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c23e00640 1 --2- 172.21.15.32:0/349628377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c080039c0 0x7f7c24151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:12.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c28a00640 1 --2- 172.21.15.32:0/349628377 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c241f5f90 0x7f7c24071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:12.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c23400640 1 -- 172.21.15.32:0/349628377 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c241f5f90 msgr2=0x7f7c24071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:12.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c23400640 1 --2- 172.21.15.32:0/349628377 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c241f5f90 0x7f7c24071d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c23400640 1 -- 172.21.15.32:0/349628377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c080039c0 msgr2=0x7f7c24151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:12.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c23400640 1 --2- 172.21.15.32:0/349628377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c080039c0 0x7f7c24151f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c23400640 1 -- 172.21.15.32:0/349628377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c24153140 con 0x7f7c24152cb0 2024-04-17T10:04:12.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c23400640 1 --2- 172.21.15.32:0/349628377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c24152cb0 0x7f7c241f5a50 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000e700 tx=0x7f7c1000ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:12.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c1be00640 1 -- 172.21.15.32:0/349628377 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1000ee10 con 0x7f7c24152cb0 2024-04-17T10:04:12.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c1be00640 1 -- 172.21.15.32:0/349628377 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c10018070 con 0x7f7c24152cb0 2024-04-17T10:04:12.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c1be00640 1 -- 172.21.15.32:0/349628377 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1000cb10 con 0x7f7c24152cb0 2024-04-17T10:04:12.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c2aa00640 1 -- 172.21.15.32:0/349628377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c24153420 con 0x7f7c24152cb0 2024-04-17T10:04:12.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.762+0000 7f7c2aa00640 1 -- 172.21.15.32:0/349628377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c241539e0 con 0x7f7c24152cb0 2024-04-17T10:04:12.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.766+0000 7f7c2aa00640 1 -- 172.21.15.32:0/349628377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c241f63a0 con 0x7f7c24152cb0 2024-04-17T10:04:12.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.766+0000 7f7c1be00640 1 -- 172.21.15.32:0/349628377 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7c1000ccb0 con 0x7f7c24152cb0 2024-04-17T10:04:12.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.766+0000 7f7c1be00640 1 --2- 172.21.15.32:0/349628377 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7c04068380 0x7f7c0406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:12.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.766+0000 7f7c1be00640 1 -- 172.21.15.32:0/349628377 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7f7c100867f0 con 0x7f7c24152cb0 2024-04-17T10:04:12.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.766+0000 7f7c23e00640 1 --2- 172.21.15.32:0/349628377 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7c04068380 0x7f7c0406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:12.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.766+0000 7f7c23e00640 1 --2- 172.21.15.32:0/349628377 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7c04068380 0x7f7c0406a840 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7c140af000 tx=0x7f7c140046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:12.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.770+0000 7f7c1be00640 1 -- 172.21.15.32:0/349628377 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f7c10056630 con 0x7f7c24152cb0 2024-04-17T10:04:12.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.950+0000 7f7c2aa00640 1 -- 172.21.15.32:0/349628377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7c24068a90 con 0x7f7c24152cb0 2024-04-17T10:04:12.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.950+0000 7f7c1be00640 1 -- 172.21.15.32:0/349628377 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v112) ==== 80+0+120 (secure 0 0 0) 0x7f7c1005a930 con 0x7f7c24152cb0 2024-04-17T10:04:12.948 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6837/901447746 2024-04-18T10:04:03.080821+0000 2024-04-17T10:04:12.948 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6836/901447746 2024-04-18T10:04:03.080821+0000 2024-04-17T10:04:12.949 INFO:teuthology.orchestra.run.smithi032.stderr:listed 2 entries 2024-04-17T10:04:12.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.950+0000 7f7c2aa00640 1 -- 172.21.15.32:0/349628377 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7c04068380 msgr2=0x7f7c0406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:12.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.950+0000 7f7c2aa00640 1 --2- 172.21.15.32:0/349628377 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7c04068380 0x7f7c0406a840 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7c140af000 tx=0x7f7c140046d0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.950+0000 7f7c2aa00640 1 -- 172.21.15.32:0/349628377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c24152cb0 msgr2=0x7f7c241f5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:12.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.950+0000 7f7c2aa00640 1 --2- 172.21.15.32:0/349628377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c24152cb0 0x7f7c241f5a50 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000e700 tx=0x7f7c1000ebc0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.950+0000 7f7c2aa00640 1 -- 172.21.15.32:0/349628377 shutdown_connections 2024-04-17T10:04:12.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.950+0000 7f7c2aa00640 1 --2- 172.21.15.32:0/349628377 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7c04068380 0x7f7c0406a840 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.950+0000 7f7c2aa00640 1 --2- 172.21.15.32:0/349628377 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c241f5f90 0x7f7c24071d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.950+0000 7f7c2aa00640 1 --2- 172.21.15.32:0/349628377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c24152cb0 0x7f7c241f5a50 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.950+0000 7f7c2aa00640 1 --2- 172.21.15.32:0/349628377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c080039c0 0x7f7c24151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:12.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.950+0000 7f7c2aa00640 1 -- 172.21.15.32:0/349628377 >> 172.21.15.32:0/349628377 conn(0x7f7c2406ba30 msgr2=0x7f7c2415d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:12.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.950+0000 7f7c2aa00640 1 -- 172.21.15.32:0/349628377 shutdown_connections 2024-04-17T10:04:12.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:12.950+0000 7f7c2aa00640 1 -- 172.21.15.32:0/349628377 wait complete. 2024-04-17T10:04:12.970 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:04:12.970 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:04:12.980 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:04:13.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.162+0000 7feec1c00640 1 Processor -- start 2024-04-17T10:04:13.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.162+0000 7feec1c00640 1 -- start start 2024-04-17T10:04:13.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.166+0000 7feec1c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7feebc072d90 0x7feebc165940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:13.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.166+0000 7feec1c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feebc0b3490 con 0x7feebc06f630 2024-04-17T10:04:13.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.166+0000 7feec1c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feebc0b58d0 con 0x7feebc151db0 2024-04-17T10:04:13.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.166+0000 7feec1c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7feebc0b44f0 con 0x7feebc072d90 2024-04-17T10:04:13.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.166+0000 7feeb2000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7feebc06f630 0x7feebc072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59086/0 (socket says 172.21.15.32:59086) 2024-04-17T10:04:13.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.166+0000 7feeb2000640 1 -- 172.21.15.32:0/2032424904 learned_addr learned my addr 172.21.15.32:0/2032424904 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:13.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.166+0000 7feebb400640 1 --2- 172.21.15.32:0/2032424904 >> v2:172.21.15.144:3300/0 conn(0x7feebc072d90 0x7feebc165940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:13.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.166+0000 7feeba000640 1 -- 172.21.15.32:0/2032424904 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3029205144 0 0) 0x7feebc0b3490 con 0x7feebc06f630 2024-04-17T10:04:13.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.166+0000 7feeba000640 1 -- 172.21.15.32:0/2032424904 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feea0003450 con 0x7feebc06f630 2024-04-17T10:04:13.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.166+0000 7feeba000640 1 -- 172.21.15.32:0/2032424904 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 556769463 0 0) 0x7feebc0b58d0 con 0x7feebc151db0 2024-04-17T10:04:13.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.166+0000 7feeba000640 1 -- 172.21.15.32:0/2032424904 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feebc0b3490 con 0x7feebc151db0 2024-04-17T10:04:13.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.166+0000 7feebb400640 1 -- 172.21.15.32:0/2032424904 >> v1:172.21.15.144:6789/0 conn(0x7feebc151db0 legacy=0x7feebc0b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:13.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.166+0000 7feebb400640 1 -- 172.21.15.32:0/2032424904 >> v1:172.21.15.32:6789/0 conn(0x7feebc06f630 legacy=0x7feebc072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:13.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.166+0000 7feebb400640 1 -- 172.21.15.32:0/2032424904 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feebc0cea40 con 0x7feebc072d90 2024-04-17T10:04:13.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.170+0000 7feebb400640 1 --2- 172.21.15.32:0/2032424904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feebc072d90 0x7feebc165940 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009d90 tx=0x7feeb401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34c6a70de8e92d7b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:13.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.170+0000 7feeba000640 1 -- 172.21.15.32:0/2032424904 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb40b2070 con 0x7feebc072d90 2024-04-17T10:04:13.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.170+0000 7feeba000640 1 -- 172.21.15.32:0/2032424904 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb40a57b0 con 0x7feebc072d90 2024-04-17T10:04:13.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.170+0000 7feeba000640 1 -- 172.21.15.32:0/2032424904 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb40a5b80 con 0x7feebc072d90 2024-04-17T10:04:13.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.170+0000 7feec1c00640 1 -- 172.21.15.32:0/2032424904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feebc072d90 msgr2=0x7feebc165940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:13.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.170+0000 7feec1c00640 1 --2- 172.21.15.32:0/2032424904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feebc072d90 0x7feebc165940 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009d90 tx=0x7feeb401c970 comp rx=0 tx=0).stop 2024-04-17T10:04:13.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.170+0000 7feec1c00640 1 -- 172.21.15.32:0/2032424904 shutdown_connections 2024-04-17T10:04:13.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.170+0000 7feec1c00640 1 --2- 172.21.15.32:0/2032424904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feebc072d90 0x7feebc165940 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:13.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.170+0000 7feec1c00640 1 -- 172.21.15.32:0/2032424904 >> 172.21.15.32:0/2032424904 conn(0x7feebc06ba30 msgr2=0x7feebc06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:13.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.170+0000 7feec1c00640 1 -- 172.21.15.32:0/2032424904 shutdown_connections 2024-04-17T10:04:13.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.170+0000 7feec1c00640 1 -- 172.21.15.32:0/2032424904 wait complete. 2024-04-17T10:04:13.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.170+0000 7feec1c00640 1 Processor -- start 2024-04-17T10:04:13.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.170+0000 7feec1c00640 1 -- start start 2024-04-17T10:04:13.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feec1c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feebc06f630 0x7feebc165420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feec1c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feebc072d90 0x7feebc135590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feec1c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feebc151db0 0x7feebc150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feec1c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7feebc167e90 con 0x7feebc06f630 2024-04-17T10:04:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feec1c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feebc0b3fd0 con 0x7feebc151db0 2024-04-17T10:04:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feec1c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7feebc168010 con 0x7feebc072d90 2024-04-17T10:04:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feebaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feebc06f630 0x7feebc165420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feebaa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feebc06f630 0x7feebc165420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50616/0 (socket says 172.21.15.32:50616) 2024-04-17T10:04:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feebaa00640 1 -- 172.21.15.32:0/4059812574 learned_addr learned my addr 172.21.15.32:0/4059812574 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feeb2000640 1 --2- 172.21.15.32:0/4059812574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feebc072d90 0x7feebc135590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feebb400640 1 --2- 172.21.15.32:0/4059812574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feebc151db0 0x7feebc150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:13.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feebaa00640 1 -- 172.21.15.32:0/4059812574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feebc072d90 msgr2=0x7feebc135590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:13.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feebaa00640 1 --2- 172.21.15.32:0/4059812574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feebc072d90 0x7feebc135590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:13.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feebaa00640 1 -- 172.21.15.32:0/4059812574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feebc151db0 msgr2=0x7feebc150400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:13.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feebaa00640 1 --2- 172.21.15.32:0/4059812574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feebc151db0 0x7feebc150400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:13.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feebaa00640 1 -- 172.21.15.32:0/4059812574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feebc14f1c0 con 0x7feebc06f630 2024-04-17T10:04:13.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feebb400640 1 --2- 172.21.15.32:0/4059812574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feebc151db0 0x7feebc150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:13.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feeb2000640 1 --2- 172.21.15.32:0/4059812574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feebc072d90 0x7feebc135590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:13.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feebaa00640 1 --2- 172.21.15.32:0/4059812574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feebc06f630 0x7feebc165420 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7feeac00c490 tx=0x7feeac00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:13.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feeb3400640 1 -- 172.21.15.32:0/4059812574 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac0052d0 con 0x7feebc06f630 2024-04-17T10:04:13.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feeb3400640 1 -- 172.21.15.32:0/4059812574 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeac005470 con 0x7feebc06f630 2024-04-17T10:04:13.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feec1c00640 1 -- 172.21.15.32:0/4059812574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feebc14e5b0 con 0x7feebc06f630 2024-04-17T10:04:13.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feeb3400640 1 -- 172.21.15.32:0/4059812574 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac012870 con 0x7feebc06f630 2024-04-17T10:04:13.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feec1c00640 1 -- 172.21.15.32:0/4059812574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feebc14eb70 con 0x7feebc06f630 2024-04-17T10:04:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feec1c00640 1 -- 172.21.15.32:0/4059812574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feebc068a90 con 0x7feebc06f630 2024-04-17T10:04:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.174+0000 7feeb3400640 1 -- 172.21.15.32:0/4059812574 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7feeac012a10 con 0x7feebc06f630 2024-04-17T10:04:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.178+0000 7feeb3400640 1 --2- 172.21.15.32:0/4059812574 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee98068380 0x7fee9806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.178+0000 7feeb3400640 1 -- 172.21.15.32:0/4059812574 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7feeac086710 con 0x7feebc06f630 2024-04-17T10:04:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.178+0000 7feeb2000640 1 --2- 172.21.15.32:0/4059812574 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee98068380 0x7fee9806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.178+0000 7feeb2000640 1 --2- 172.21.15.32:0/4059812574 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee98068380 0x7fee9806a840 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7feebc1485a0 tx=0x7feea4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.182+0000 7feeb3400640 1 -- 172.21.15.32:0/4059812574 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7feeac056c40 con 0x7feebc06f630 2024-04-17T10:04:13.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.350+0000 7feec1c00640 1 -- 172.21.15.32:0/4059812574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feebc15e390 con 0x7feebc06f630 2024-04-17T10:04:13.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.350+0000 7feeb3400640 1 -- 172.21.15.32:0/4059812574 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+16863 (secure 0 0 0) 0x7feeac056540 con 0x7feebc06f630 2024-04-17T10:04:13.349 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:13.350 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":112,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:04:03.080842+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:03:36.021322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-17T10:03:36.581134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.5","mappings":[{"from":0,"to":5}]},{"pgid":"17.25","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6837/901447746":"2024-04-18T10:04:03.080821+0000","172.21.15.32:6836/901447746":"2024-04-18T10:04:03.080821+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:04:13.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.350+0000 7feec1c00640 1 -- 172.21.15.32:0/4059812574 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee98068380 msgr2=0x7fee9806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:13.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.350+0000 7feec1c00640 1 --2- 172.21.15.32:0/4059812574 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee98068380 0x7fee9806a840 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7feebc1485a0 tx=0x7feea4009040 comp rx=0 tx=0).stop 2024-04-17T10:04:13.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.350+0000 7feec1c00640 1 -- 172.21.15.32:0/4059812574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feebc06f630 msgr2=0x7feebc165420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:13.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.350+0000 7feec1c00640 1 --2- 172.21.15.32:0/4059812574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feebc06f630 0x7feebc165420 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7feeac00c490 tx=0x7feeac00c950 comp rx=0 tx=0).stop 2024-04-17T10:04:13.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.354+0000 7feec1c00640 1 -- 172.21.15.32:0/4059812574 shutdown_connections 2024-04-17T10:04:13.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.354+0000 7feec1c00640 1 --2- 172.21.15.32:0/4059812574 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fee98068380 0x7fee9806a840 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:13.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.354+0000 7feec1c00640 1 --2- 172.21.15.32:0/4059812574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feebc072d90 0x7feebc135590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:13.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.354+0000 7feec1c00640 1 --2- 172.21.15.32:0/4059812574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feebc151db0 0x7feebc150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:13.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.354+0000 7feec1c00640 1 --2- 172.21.15.32:0/4059812574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feebc06f630 0x7feebc165420 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:13.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.354+0000 7feec1c00640 1 -- 172.21.15.32:0/4059812574 >> 172.21.15.32:0/4059812574 conn(0x7feebc06ba30 msgr2=0x7feebc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:13.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.354+0000 7feec1c00640 1 -- 172.21.15.32:0/4059812574 shutdown_connections 2024-04-17T10:04:13.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.354+0000 7feec1c00640 1 -- 172.21.15.32:0/4059812574 wait complete. 2024-04-17T10:04:13.371 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:03:36.021322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:03:36.581134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:04:13.372 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:04:13.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.518+0000 7fdd1b200640 1 Processor -- start 2024-04-17T10:04:13.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.518+0000 7fdd1b200640 1 -- start start 2024-04-17T10:04:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.518+0000 7fdd1b200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdd14141700 0x7fdd1406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.518+0000 7fdd1b200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd140b0540 con 0x7fdd140afeb0 2024-04-17T10:04:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.518+0000 7fdd1b200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd140b3ef0 con 0x7fdd1406cf80 2024-04-17T10:04:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.518+0000 7fdd1b200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fdd140b0820 con 0x7fdd14141700 2024-04-17T10:04:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.518+0000 7fdd13e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdd14141700 0x7fdd1406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.518+0000 7fdd19200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdd140afeb0 0x7fdd140b49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59088/0 (socket says 172.21.15.32:59088) 2024-04-17T10:04:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.518+0000 7fdd19200640 1 -- 172.21.15.32:0/547268964 learned_addr learned my addr 172.21.15.32:0/547268964 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:13.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.518+0000 7fdd13e00640 1 -- 172.21.15.32:0/547268964 >> v1:172.21.15.144:6789/0 conn(0x7fdd1406cf80 legacy=0x7fdd140af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:13.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.518+0000 7fdd13e00640 1 -- 172.21.15.32:0/547268964 >> v1:172.21.15.32:6789/0 conn(0x7fdd140afeb0 legacy=0x7fdd140b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:13.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.518+0000 7fdd13e00640 1 -- 172.21.15.32:0/547268964 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd140b41d0 con 0x7fdd14141700 2024-04-17T10:04:13.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.518+0000 7fdd13400640 1 -- 172.21.15.32:0/547268964 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2138591632 0 0) 0x7fdd140b0540 con 0x7fdd140afeb0 2024-04-17T10:04:13.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.522+0000 7fdd13e00640 1 --2- 172.21.15.32:0/547268964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd14141700 0x7fdd1406ca40 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fdd040097c0 tx=0x7fdd0401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1530f48a2d73720 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:13.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.522+0000 7fdd13400640 1 -- 172.21.15.32:0/547268964 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd040b0070 con 0x7fdd14141700 2024-04-17T10:04:13.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.522+0000 7fdd13400640 1 -- 172.21.15.32:0/547268964 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd040ab600 con 0x7fdd14141700 2024-04-17T10:04:13.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.522+0000 7fdd13400640 1 -- 172.21.15.32:0/547268964 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd040ab9d0 con 0x7fdd14141700 2024-04-17T10:04:13.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.522+0000 7fdd1b200640 1 -- 172.21.15.32:0/547268964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd14141700 msgr2=0x7fdd1406ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:13.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.522+0000 7fdd1b200640 1 --2- 172.21.15.32:0/547268964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd14141700 0x7fdd1406ca40 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fdd040097c0 tx=0x7fdd0401c4a0 comp rx=0 tx=0).stop 2024-04-17T10:04:13.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.522+0000 7fdd1b200640 1 -- 172.21.15.32:0/547268964 shutdown_connections 2024-04-17T10:04:13.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.522+0000 7fdd1b200640 1 --2- 172.21.15.32:0/547268964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd14141700 0x7fdd1406ca40 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:13.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.522+0000 7fdd1b200640 1 -- 172.21.15.32:0/547268964 >> 172.21.15.32:0/547268964 conn(0x7fdd14072670 msgr2=0x7fdd14072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:13.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.522+0000 7fdd1b200640 1 -- 172.21.15.32:0/547268964 shutdown_connections 2024-04-17T10:04:13.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.522+0000 7fdd1b200640 1 -- 172.21.15.32:0/547268964 wait complete. 2024-04-17T10:04:13.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.522+0000 7fdd1b200640 1 Processor -- start 2024-04-17T10:04:13.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.522+0000 7fdd1b200640 1 -- start start 2024-04-17T10:04:13.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd1b200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd1406cf80 0x7fdd1415b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:13.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd1b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd140afeb0 0x7fdd1413d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:13.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd1b200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd14141700 0x7fdd1413b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:13.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd1b200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdd140b5cc0 con 0x7fdd1406cf80 2024-04-17T10:04:13.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd1b200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdd140b5b40 con 0x7fdd14141700 2024-04-17T10:04:13.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd1b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdd14141cf0 con 0x7fdd140afeb0 2024-04-17T10:04:13.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd13e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd1406cf80 0x7fdd1415b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:13.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd13e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd1406cf80 0x7fdd1415b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50640/0 (socket says 172.21.15.32:50640) 2024-04-17T10:04:13.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd13e00640 1 -- 172.21.15.32:0/2774736843 learned_addr learned my addr 172.21.15.32:0/2774736843 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:13.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd0b400640 1 --2- 172.21.15.32:0/2774736843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd140afeb0 0x7fdd1413d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:13.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd19200640 1 --2- 172.21.15.32:0/2774736843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd14141700 0x7fdd1413b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:13.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd13e00640 1 -- 172.21.15.32:0/2774736843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd140afeb0 msgr2=0x7fdd1413d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:13.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd13e00640 1 --2- 172.21.15.32:0/2774736843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd140afeb0 0x7fdd1413d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:13.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd13e00640 1 -- 172.21.15.32:0/2774736843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd14141700 msgr2=0x7fdd1413b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:13.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd13e00640 1 --2- 172.21.15.32:0/2774736843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd14141700 0x7fdd1413b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:13.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd13e00640 1 -- 172.21.15.32:0/2774736843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1413e030 con 0x7fdd1406cf80 2024-04-17T10:04:13.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd19200640 1 --2- 172.21.15.32:0/2774736843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd14141700 0x7fdd1413b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:13.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd13e00640 1 --2- 172.21.15.32:0/2774736843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd1406cf80 0x7fdd1415b290 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fdd040ae040 tx=0x7fdd0401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:13.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd10c00640 1 -- 172.21.15.32:0/2774736843 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd040b0050 con 0x7fdd1406cf80 2024-04-17T10:04:13.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd10c00640 1 -- 172.21.15.32:0/2774736843 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd040bc070 con 0x7fdd1406cf80 2024-04-17T10:04:13.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd10c00640 1 -- 172.21.15.32:0/2774736843 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd040b5a50 con 0x7fdd1406cf80 2024-04-17T10:04:13.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd1b200640 1 -- 172.21.15.32:0/2774736843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd1413e310 con 0x7fdd1406cf80 2024-04-17T10:04:13.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd1b200640 1 -- 172.21.15.32:0/2774736843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd1413e8d0 con 0x7fdd1406cf80 2024-04-17T10:04:13.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd1b200640 1 -- 172.21.15.32:0/2774736843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd14068a90 con 0x7fdd1406cf80 2024-04-17T10:04:13.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.526+0000 7fdd10c00640 1 -- 172.21.15.32:0/2774736843 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdd040ab4a0 con 0x7fdd1406cf80 2024-04-17T10:04:13.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.530+0000 7fdd10c00640 1 --2- 172.21.15.32:0/2774736843 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdcf00683d0 0x7fdcf006a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:13.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.530+0000 7fdd10c00640 1 -- 172.21.15.32:0/2774736843 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7fdd04121370 con 0x7fdd1406cf80 2024-04-17T10:04:13.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.530+0000 7fdd0b400640 1 --2- 172.21.15.32:0/2774736843 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdcf00683d0 0x7fdcf006a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:13.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.530+0000 7fdd0b400640 1 --2- 172.21.15.32:0/2774736843 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdcf00683d0 0x7fdcf006a890 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fdd14140920 tx=0x7fdcfc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:13.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.534+0000 7fdd10c00640 1 -- 172.21.15.32:0/2774736843 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fdd04016090 con 0x7fdd1406cf80 2024-04-17T10:04:13.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.698+0000 7fdd1b200640 1 -- 172.21.15.32:0/2774736843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdd140b4ae0 con 0x7fdd1406cf80 2024-04-17T10:04:13.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.698+0000 7fdd10c00640 1 -- 172.21.15.32:0/2774736843 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) ==== 93+0+10 (secure 0 0 0) 0x7fdd040f1880 con 0x7fdd1406cf80 2024-04-17T10:04:13.696 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T10:04:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.698+0000 7fdd1b200640 1 -- 172.21.15.32:0/2774736843 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdcf00683d0 msgr2=0x7fdcf006a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.698+0000 7fdd1b200640 1 --2- 172.21.15.32:0/2774736843 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdcf00683d0 0x7fdcf006a890 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fdd14140920 tx=0x7fdcfc009040 comp rx=0 tx=0).stop 2024-04-17T10:04:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.698+0000 7fdd1b200640 1 -- 172.21.15.32:0/2774736843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd1406cf80 msgr2=0x7fdd1415b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.698+0000 7fdd1b200640 1 --2- 172.21.15.32:0/2774736843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd1406cf80 0x7fdd1415b290 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fdd040ae040 tx=0x7fdd0401ace0 comp rx=0 tx=0).stop 2024-04-17T10:04:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.698+0000 7fdd1b200640 1 -- 172.21.15.32:0/2774736843 shutdown_connections 2024-04-17T10:04:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.698+0000 7fdd1b200640 1 --2- 172.21.15.32:0/2774736843 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdcf00683d0 0x7fdcf006a890 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.698+0000 7fdd1b200640 1 --2- 172.21.15.32:0/2774736843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd140afeb0 0x7fdd1413d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.698+0000 7fdd1b200640 1 --2- 172.21.15.32:0/2774736843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd14141700 0x7fdd1413b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.698+0000 7fdd1b200640 1 --2- 172.21.15.32:0/2774736843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd1406cf80 0x7fdd1415b290 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.698+0000 7fdd1b200640 1 -- 172.21.15.32:0/2774736843 >> 172.21.15.32:0/2774736843 conn(0x7fdd14072670 msgr2=0x7fdd1406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.698+0000 7fdd1b200640 1 -- 172.21.15.32:0/2774736843 shutdown_connections 2024-04-17T10:04:13.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.698+0000 7fdd1b200640 1 -- 172.21.15.32:0/2774736843 wait complete. 2024-04-17T10:04:13.717 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:04:13.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.866+0000 7f0cf4200640 1 Processor -- start 2024-04-17T10:04:13.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.866+0000 7f0cf4200640 1 -- start start 2024-04-17T10:04:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.866+0000 7f0cf4200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0cec0cf370 0x7f0cec0cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.866+0000 7f0cf4200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cec0d0130 con 0x7f0cec071e60 2024-04-17T10:04:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.866+0000 7f0cf4200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cec15d090 con 0x7f0cec06ba30 2024-04-17T10:04:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.866+0000 7f0cf4200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0cec15d370 con 0x7f0cec0cf370 2024-04-17T10:04:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.866+0000 7f0cf2200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0cec071e60 0x7f0cec072270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59100/0 (socket says 172.21.15.32:59100) 2024-04-17T10:04:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.866+0000 7f0cf2200640 1 -- 172.21.15.32:0/3876300109 learned_addr learned my addr 172.21.15.32:0/3876300109 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.866+0000 7f0cf0e00640 1 --2- 172.21.15.32:0/3876300109 >> v2:172.21.15.144:3300/0 conn(0x7f0cec0cf370 0x7f0cec0cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:13.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.866+0000 7f0cebe00640 1 -- 172.21.15.32:0/3876300109 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4209727252 0 0) 0x7f0cec0d0130 con 0x7f0cec071e60 2024-04-17T10:04:13.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.866+0000 7f0cebe00640 1 -- 172.21.15.32:0/3876300109 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0cd4003530 con 0x7f0cec071e60 2024-04-17T10:04:13.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.866+0000 7f0cebe00640 1 -- 172.21.15.32:0/3876300109 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 902578549 0 0) 0x7f0cec15d090 con 0x7f0cec06ba30 2024-04-17T10:04:13.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.866+0000 7f0cebe00640 1 -- 172.21.15.32:0/3876300109 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0cec0d0130 con 0x7f0cec06ba30 2024-04-17T10:04:13.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.866+0000 7f0cf0e00640 1 -- 172.21.15.32:0/3876300109 >> v1:172.21.15.144:6789/0 conn(0x7f0cec06ba30 legacy=0x7f0cec06be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:13.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.866+0000 7f0cf0e00640 1 -- 172.21.15.32:0/3876300109 >> v1:172.21.15.32:6789/0 conn(0x7f0cec071e60 legacy=0x7f0cec072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:13.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.866+0000 7f0cf0e00640 1 -- 172.21.15.32:0/3876300109 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cec15d650 con 0x7f0cec0cf370 2024-04-17T10:04:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.870+0000 7f0cf0e00640 1 --2- 172.21.15.32:0/3876300109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cec0cf370 0x7f0cec0cf7e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80097c0 tx=0x7f0cd801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f298850e2c306b06 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.870+0000 7f0cebe00640 1 -- 172.21.15.32:0/3876300109 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd80aa600 con 0x7f0cec0cf370 2024-04-17T10:04:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.870+0000 7f0cebe00640 1 -- 172.21.15.32:0/3876300109 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cd80af070 con 0x7f0cec0cf370 2024-04-17T10:04:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.870+0000 7f0cebe00640 1 -- 172.21.15.32:0/3876300109 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd80aa9c0 con 0x7f0cec0cf370 2024-04-17T10:04:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.870+0000 7f0cf4200640 1 -- 172.21.15.32:0/3876300109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cec0cf370 msgr2=0x7f0cec0cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.870+0000 7f0cf4200640 1 --2- 172.21.15.32:0/3876300109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cec0cf370 0x7f0cec0cf7e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80097c0 tx=0x7f0cd801c4a0 comp rx=0 tx=0).stop 2024-04-17T10:04:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.870+0000 7f0cf4200640 1 -- 172.21.15.32:0/3876300109 shutdown_connections 2024-04-17T10:04:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.870+0000 7f0cf4200640 1 --2- 172.21.15.32:0/3876300109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cec0cf370 0x7f0cec0cf7e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.870+0000 7f0cf4200640 1 -- 172.21.15.32:0/3876300109 >> 172.21.15.32:0/3876300109 conn(0x7f0cec0b2120 msgr2=0x7f0cec0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.870+0000 7f0cf4200640 1 -- 172.21.15.32:0/3876300109 shutdown_connections 2024-04-17T10:04:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.870+0000 7f0cf4200640 1 -- 172.21.15.32:0/3876300109 wait complete. 2024-04-17T10:04:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0cf4200640 1 Processor -- start 2024-04-17T10:04:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0cf4200640 1 -- start start 2024-04-17T10:04:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0cf4200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0cec06ba30 0x7f0cec141a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0cf4200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cec071e60 0x7f0cec0c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0cf4200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cec0cf370 0x7f0cec13bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0cf4200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec15dcf0 con 0x7f0cec0cf370 2024-04-17T10:04:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0cf4200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec15db70 con 0x7f0cec071e60 2024-04-17T10:04:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0cf4200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0cec146230 con 0x7f0cec06ba30 2024-04-17T10:04:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0cf2200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cec0cf370 0x7f0cec13bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0cf2200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cec0cf370 0x7f0cec13bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50650/0 (socket says 172.21.15.32:50650) 2024-04-17T10:04:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0cf2200640 1 -- 172.21.15.32:0/969605160 learned_addr learned my addr 172.21.15.32:0/969605160 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0cf0e00640 1 --2- 172.21.15.32:0/969605160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cec071e60 0x7f0cec0c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0cf1800640 1 --2- 172.21.15.32:0/969605160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0cec06ba30 0x7f0cec141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0cf2200640 1 -- 172.21.15.32:0/969605160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0cec06ba30 msgr2=0x7f0cec141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0cf2200640 1 --2- 172.21.15.32:0/969605160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0cec06ba30 0x7f0cec141a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0cf2200640 1 -- 172.21.15.32:0/969605160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cec071e60 msgr2=0x7f0cec0c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0cf2200640 1 --2- 172.21.15.32:0/969605160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cec071e60 0x7f0cec0c60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0cf2200640 1 -- 172.21.15.32:0/969605160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cec0c6600 con 0x7f0cec0cf370 2024-04-17T10:04:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0cf1800640 1 --2- 172.21.15.32:0/969605160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0cec06ba30 0x7f0cec141a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0cf2200640 1 --2- 172.21.15.32:0/969605160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cec0cf370 0x7f0cec13bef0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400cbd0 tx=0x7f0ce400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0ce9600640 1 -- 172.21.15.32:0/969605160 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4017070 con 0x7f0cec0cf370 2024-04-17T10:04:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0ce9600640 1 -- 172.21.15.32:0/969605160 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce400ace0 con 0x7f0cec0cf370 2024-04-17T10:04:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0ce9600640 1 -- 172.21.15.32:0/969605160 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4012620 con 0x7f0cec0cf370 2024-04-17T10:04:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0cf4200640 1 -- 172.21.15.32:0/969605160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cec142190 con 0x7f0cec0cf370 2024-04-17T10:04:13.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.874+0000 7f0cf4200640 1 -- 172.21.15.32:0/969605160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cec142750 con 0x7f0cec0cf370 2024-04-17T10:04:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.878+0000 7f0cf4200640 1 -- 172.21.15.32:0/969605160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cec068a90 con 0x7f0cec0cf370 2024-04-17T10:04:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.878+0000 7f0ce9600640 1 -- 172.21.15.32:0/969605160 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0ce401a070 con 0x7f0cec0cf370 2024-04-17T10:04:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.878+0000 7f0ce9600640 1 --2- 172.21.15.32:0/969605160 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0ccc068380 0x7f0ccc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.878+0000 7f0ce9600640 1 -- 172.21.15.32:0/969605160 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7f0ce4086a50 con 0x7f0cec0cf370 2024-04-17T10:04:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.878+0000 7f0cf1800640 1 --2- 172.21.15.32:0/969605160 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0ccc068380 0x7f0ccc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.878+0000 7f0cf1800640 1 --2- 172.21.15.32:0/969605160 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0ccc068380 0x7f0ccc06a840 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0001740 tx=0x7f0ce0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:13.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:13.882+0000 7f0ce9600640 1 -- 172.21.15.32:0/969605160 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0cec0c6600 con 0x7f0cec0cf370 2024-04-17T10:04:14.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.046+0000 7f0cf4200640 1 -- 172.21.15.32:0/969605160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0cec1684d0 con 0x7f0cec0cf370 2024-04-17T10:04:14.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.046+0000 7f0ce9600640 1 -- 172.21.15.32:0/969605160 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) ==== 104+0+11 (secure 0 0 0) 0x7f0ce4056890 con 0x7f0cec0cf370 2024-04-17T10:04:14.042 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T10:04:14.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.050+0000 7f0cf4200640 1 -- 172.21.15.32:0/969605160 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0ccc068380 msgr2=0x7f0ccc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:14.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.050+0000 7f0cf4200640 1 --2- 172.21.15.32:0/969605160 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0ccc068380 0x7f0ccc06a840 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0001740 tx=0x7f0ce0009040 comp rx=0 tx=0).stop 2024-04-17T10:04:14.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.050+0000 7f0cf4200640 1 -- 172.21.15.32:0/969605160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cec0cf370 msgr2=0x7f0cec13bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:14.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.050+0000 7f0cf4200640 1 --2- 172.21.15.32:0/969605160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cec0cf370 0x7f0cec13bef0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400cbd0 tx=0x7f0ce400a430 comp rx=0 tx=0).stop 2024-04-17T10:04:14.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.050+0000 7f0cf4200640 1 -- 172.21.15.32:0/969605160 shutdown_connections 2024-04-17T10:04:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.050+0000 7f0cf4200640 1 --2- 172.21.15.32:0/969605160 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0ccc068380 0x7f0ccc06a840 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.050+0000 7f0cf4200640 1 --2- 172.21.15.32:0/969605160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cec0cf370 0x7f0cec13bef0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.050+0000 7f0cf4200640 1 --2- 172.21.15.32:0/969605160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cec071e60 0x7f0cec0c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.050+0000 7f0cf4200640 1 --2- 172.21.15.32:0/969605160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0cec06ba30 0x7f0cec141a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.050+0000 7f0cf4200640 1 -- 172.21.15.32:0/969605160 >> 172.21.15.32:0/969605160 conn(0x7f0cec0b2120 msgr2=0x7f0cec0b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.050+0000 7f0cf4200640 1 -- 172.21.15.32:0/969605160 shutdown_connections 2024-04-17T10:04:14.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.050+0000 7f0cf4200640 1 -- 172.21.15.32:0/969605160 wait complete. 2024-04-17T10:04:14.063 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:04:14.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.210+0000 7fbd3c600640 1 Processor -- start 2024-04-17T10:04:14.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.210+0000 7fbd3c600640 1 -- start start 2024-04-17T10:04:14.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.214+0000 7fbd3c600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbd3406ba30 0x7fbd3406be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:14.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.214+0000 7fbd3c600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd340cf370 0x7fbd340cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:14.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.214+0000 7fbd3c600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd34160350 0x7fbd341607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:14.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.214+0000 7fbd3c600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fbd3406c350 con 0x7fbd34160350 2024-04-17T10:04:14.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.214+0000 7fbd3c600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fbd340d0130 con 0x7fbd3406ba30 2024-04-17T10:04:14.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.214+0000 7fbd3c600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbd3415d090 con 0x7fbd340cf370 2024-04-17T10:04:14.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.214+0000 7fbd3a600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd34160350 0x7fbd341607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:14.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.214+0000 7fbd3a600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd34160350 0x7fbd341607a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50658/0 (socket says 172.21.15.32:50658) 2024-04-17T10:04:14.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.214+0000 7fbd3a600640 1 -- 172.21.15.32:0/75762078 learned_addr learned my addr 172.21.15.32:0/75762078 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:14.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.214+0000 7fbd39200640 1 --2- 172.21.15.32:0/75762078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd340cf370 0x7fbd340cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.214+0000 7fbd39c00640 1 --2- 172.21.15.32:0/75762078 >> v2:172.21.15.144:3300/0 conn(0x7fbd3406ba30 0x7fbd3406be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.214+0000 7fbd3a600640 1 -- 172.21.15.32:0/75762078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd340cf370 msgr2=0x7fbd340cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.214+0000 7fbd3a600640 1 --2- 172.21.15.32:0/75762078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd340cf370 0x7fbd340cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.214+0000 7fbd3a600640 1 -- 172.21.15.32:0/75762078 >> v2:172.21.15.144:3300/0 conn(0x7fbd3406ba30 msgr2=0x7fbd3406be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.214+0000 7fbd3a600640 1 --2- 172.21.15.32:0/75762078 >> v2:172.21.15.144:3300/0 conn(0x7fbd3406ba30 0x7fbd3406be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.214+0000 7fbd3a600640 1 -- 172.21.15.32:0/75762078 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3415d370 con 0x7fbd34160350 2024-04-17T10:04:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.214+0000 7fbd39c00640 1 --2- 172.21.15.32:0/75762078 >> v2:172.21.15.144:3300/0 conn(0x7fbd3406ba30 0x7fbd3406be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.218+0000 7fbd3a600640 1 --2- 172.21.15.32:0/75762078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd34160350 0x7fbd341607a0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fbd340b5730 tx=0x7fbd2c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4288658b15fb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.218+0000 7fbd33e00640 1 -- 172.21.15.32:0/75762078 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c0a5800 con 0x7fbd34160350 2024-04-17T10:04:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.218+0000 7fbd33e00640 1 -- 172.21.15.32:0/75762078 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd2c0a59a0 con 0x7fbd34160350 2024-04-17T10:04:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.218+0000 7fbd33e00640 1 -- 172.21.15.32:0/75762078 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c0a5d70 con 0x7fbd34160350 2024-04-17T10:04:14.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.218+0000 7fbd3c600640 1 -- 172.21.15.32:0/75762078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd34160350 msgr2=0x7fbd341607a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:14.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.218+0000 7fbd3c600640 1 --2- 172.21.15.32:0/75762078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd34160350 0x7fbd341607a0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fbd340b5730 tx=0x7fbd2c01c9c0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.218+0000 7fbd3c600640 1 -- 172.21.15.32:0/75762078 shutdown_connections 2024-04-17T10:04:14.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.218+0000 7fbd3c600640 1 --2- 172.21.15.32:0/75762078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd34160350 0x7fbd341607a0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.218+0000 7fbd3c600640 1 --2- 172.21.15.32:0/75762078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd340cf370 0x7fbd340cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.218+0000 7fbd3c600640 1 --2- 172.21.15.32:0/75762078 >> v2:172.21.15.144:3300/0 conn(0x7fbd3406ba30 0x7fbd3406be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.218+0000 7fbd3c600640 1 -- 172.21.15.32:0/75762078 >> 172.21.15.32:0/75762078 conn(0x7fbd340b2120 msgr2=0x7fbd340b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:14.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.218+0000 7fbd3c600640 1 -- 172.21.15.32:0/75762078 shutdown_connections 2024-04-17T10:04:14.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.218+0000 7fbd3c600640 1 -- 172.21.15.32:0/75762078 wait complete. 2024-04-17T10:04:14.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.218+0000 7fbd3c600640 1 Processor -- start 2024-04-17T10:04:14.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.218+0000 7fbd3c600640 1 -- start start 2024-04-17T10:04:14.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd3c600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd3406ba30 0x7fbd340af650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:14.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd3c600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd340cf370 0x7fbd341f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:14.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd3c600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd34160350 0x7fbd340b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:14.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd3c600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbd340cfea0 con 0x7fbd340cf370 2024-04-17T10:04:14.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd3c600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbd340cfd20 con 0x7fbd3406ba30 2024-04-17T10:04:14.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd3c600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbd3415db70 con 0x7fbd34160350 2024-04-17T10:04:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd39200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd340cf370 0x7fbd341f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd39200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd340cf370 0x7fbd341f80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50668/0 (socket says 172.21.15.32:50668) 2024-04-17T10:04:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd39200640 1 -- 172.21.15.32:0/1244098769 learned_addr learned my addr 172.21.15.32:0/1244098769 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd39c00640 1 --2- 172.21.15.32:0/1244098769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd3406ba30 0x7fbd340af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd3a600640 1 --2- 172.21.15.32:0/1244098769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd34160350 0x7fbd340b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd39200640 1 -- 172.21.15.32:0/1244098769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd34160350 msgr2=0x7fbd340b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd39200640 1 --2- 172.21.15.32:0/1244098769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd34160350 0x7fbd340b0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd39200640 1 -- 172.21.15.32:0/1244098769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd3406ba30 msgr2=0x7fbd340af650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd39200640 1 --2- 172.21.15.32:0/1244098769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd3406ba30 0x7fbd340af650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd39200640 1 -- 172.21.15.32:0/1244098769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd340b37d0 con 0x7fbd340cf370 2024-04-17T10:04:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd3a600640 1 --2- 172.21.15.32:0/1244098769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd34160350 0x7fbd340b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd39c00640 1 --2- 172.21.15.32:0/1244098769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd3406ba30 0x7fbd340af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd39200640 1 --2- 172.21.15.32:0/1244098769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd340cf370 0x7fbd341f80d0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000e930 tx=0x7fbd2000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:14.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd31600640 1 -- 172.21.15.32:0/1244098769 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd20017070 con 0x7fbd340cf370 2024-04-17T10:04:14.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd31600640 1 -- 172.21.15.32:0/1244098769 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd2000ca00 con 0x7fbd340cf370 2024-04-17T10:04:14.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd31600640 1 -- 172.21.15.32:0/1244098769 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2000cdb0 con 0x7fbd340cf370 2024-04-17T10:04:14.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd3c600640 1 -- 172.21.15.32:0/1244098769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd340b3ab0 con 0x7fbd340cf370 2024-04-17T10:04:14.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd3c600640 1 -- 172.21.15.32:0/1244098769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd340b4070 con 0x7fbd340cf370 2024-04-17T10:04:14.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd3c600640 1 -- 172.21.15.32:0/1244098769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd34068a90 con 0x7fbd340cf370 2024-04-17T10:04:14.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.222+0000 7fbd31600640 1 -- 172.21.15.32:0/1244098769 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbd2001a070 con 0x7fbd340cf370 2024-04-17T10:04:14.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.226+0000 7fbd31600640 1 --2- 172.21.15.32:0/1244098769 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbd14068380 0x7fbd1406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:14.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.226+0000 7fbd31600640 1 -- 172.21.15.32:0/1244098769 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7fbd20086ea0 con 0x7fbd340cf370 2024-04-17T10:04:14.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.226+0000 7fbd39c00640 1 --2- 172.21.15.32:0/1244098769 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbd14068380 0x7fbd1406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:14.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.226+0000 7fbd39c00640 1 --2- 172.21.15.32:0/1244098769 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbd14068380 0x7fbd1406a840 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800c940 tx=0x7fbd28008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.230+0000 7fbd31600640 1 -- 172.21.15.32:0/1244098769 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fbd200573f0 con 0x7fbd340cf370 2024-04-17T10:04:14.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.390+0000 7fbd3c600640 1 -- 172.21.15.32:0/1244098769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbd3406c850 con 0x7fbd340cf370 2024-04-17T10:04:14.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.390+0000 7fbd31600640 1 -- 172.21.15.32:0/1244098769 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) ==== 100+0+11 (secure 0 0 0) 0x7fbd20056cf0 con 0x7fbd340cf370 2024-04-17T10:04:14.387 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T10:04:14.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.394+0000 7fbd3c600640 1 -- 172.21.15.32:0/1244098769 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbd14068380 msgr2=0x7fbd1406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:14.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.394+0000 7fbd3c600640 1 --2- 172.21.15.32:0/1244098769 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbd14068380 0x7fbd1406a840 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800c940 tx=0x7fbd28008040 comp rx=0 tx=0).stop 2024-04-17T10:04:14.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.394+0000 7fbd3c600640 1 -- 172.21.15.32:0/1244098769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd340cf370 msgr2=0x7fbd341f80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:14.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.394+0000 7fbd3c600640 1 --2- 172.21.15.32:0/1244098769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd340cf370 0x7fbd341f80d0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000e930 tx=0x7fbd2000edf0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.394+0000 7fbd3c600640 1 -- 172.21.15.32:0/1244098769 shutdown_connections 2024-04-17T10:04:14.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.394+0000 7fbd3c600640 1 --2- 172.21.15.32:0/1244098769 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbd14068380 0x7fbd1406a840 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.394+0000 7fbd3c600640 1 --2- 172.21.15.32:0/1244098769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd34160350 0x7fbd340b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.394+0000 7fbd3c600640 1 --2- 172.21.15.32:0/1244098769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd340cf370 0x7fbd341f80d0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.394+0000 7fbd3c600640 1 --2- 172.21.15.32:0/1244098769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd3406ba30 0x7fbd340af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.394+0000 7fbd3c600640 1 -- 172.21.15.32:0/1244098769 >> 172.21.15.32:0/1244098769 conn(0x7fbd340b2120 msgr2=0x7fbd3406f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:14.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.394+0000 7fbd3c600640 1 -- 172.21.15.32:0/1244098769 shutdown_connections 2024-04-17T10:04:14.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.394+0000 7fbd3c600640 1 -- 172.21.15.32:0/1244098769 wait complete. 2024-04-17T10:04:14.410 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:04:14.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.554+0000 7fba3ee00640 1 Processor -- start 2024-04-17T10:04:14.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.554+0000 7fba3ee00640 1 -- start start 2024-04-17T10:04:14.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.558+0000 7fba3ee00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fba3813f6e0 0x7fba380cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:14.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.558+0000 7fba3ee00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba380d09f0 0x7fba38071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:14.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.558+0000 7fba3ee00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba381601e0 con 0x7fba380cf370 2024-04-17T10:04:14.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.558+0000 7fba3ee00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fba3813fac0 con 0x7fba380d09f0 2024-04-17T10:04:14.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.558+0000 7fba3ee00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fba38160480 con 0x7fba3813f6e0 2024-04-17T10:04:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.558+0000 7fba3ce00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba380d09f0 0x7fba38071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.558+0000 7fba3ce00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba380d09f0 0x7fba38071d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50676/0 (socket says 172.21.15.32:50676) 2024-04-17T10:04:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.558+0000 7fba2f400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fba380cf370 0x7fba380d02e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59110/0 (socket says 172.21.15.32:59110) 2024-04-17T10:04:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.558+0000 7fba3ce00640 1 -- 172.21.15.32:0/1796681102 learned_addr learned my addr 172.21.15.32:0/1796681102 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.558+0000 7fba37e00640 1 --2- 172.21.15.32:0/1796681102 >> v2:172.21.15.144:3300/0 conn(0x7fba3813f6e0 0x7fba380cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.558+0000 7fba3ce00640 1 -- 172.21.15.32:0/1796681102 >> v2:172.21.15.144:3300/0 conn(0x7fba3813f6e0 msgr2=0x7fba380cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.558+0000 7fba3ce00640 1 --2- 172.21.15.32:0/1796681102 >> v2:172.21.15.144:3300/0 conn(0x7fba3813f6e0 0x7fba380cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.558+0000 7fba3ce00640 1 -- 172.21.15.32:0/1796681102 >> v1:172.21.15.32:6789/0 conn(0x7fba380cf370 legacy=0x7fba380d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.558+0000 7fba3ce00640 1 -- 172.21.15.32:0/1796681102 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba38160760 con 0x7fba380d09f0 2024-04-17T10:04:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.558+0000 7fba37e00640 1 --2- 172.21.15.32:0/1796681102 >> v2:172.21.15.144:3300/0 conn(0x7fba3813f6e0 0x7fba380cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:14.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.562+0000 7fba3ce00640 1 --2- 172.21.15.32:0/1796681102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba380d09f0 0x7fba38071d80 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fba380b3100 tx=0x7fba3001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7bdc63d39509689 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:14.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.562+0000 7fba37400640 1 -- 172.21.15.32:0/1796681102 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba300aa650 con 0x7fba380d09f0 2024-04-17T10:04:14.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.562+0000 7fba37400640 1 -- 172.21.15.32:0/1796681102 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba300af070 con 0x7fba380d09f0 2024-04-17T10:04:14.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.562+0000 7fba37400640 1 -- 172.21.15.32:0/1796681102 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba300aaa10 con 0x7fba380d09f0 2024-04-17T10:04:14.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.562+0000 7fba3ee00640 1 -- 172.21.15.32:0/1796681102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba380d09f0 msgr2=0x7fba38071d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:14.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.562+0000 7fba3ee00640 1 --2- 172.21.15.32:0/1796681102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba380d09f0 0x7fba38071d80 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fba380b3100 tx=0x7fba3001c4f0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.562+0000 7fba3ee00640 1 -- 172.21.15.32:0/1796681102 shutdown_connections 2024-04-17T10:04:14.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.562+0000 7fba3ee00640 1 --2- 172.21.15.32:0/1796681102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba380d09f0 0x7fba38071d80 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.562+0000 7fba3ee00640 1 --2- 172.21.15.32:0/1796681102 >> v2:172.21.15.144:3300/0 conn(0x7fba3813f6e0 0x7fba380cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.562+0000 7fba3ee00640 1 -- 172.21.15.32:0/1796681102 >> 172.21.15.32:0/1796681102 conn(0x7fba3806ba30 msgr2=0x7fba3806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:14.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.562+0000 7fba3ee00640 1 -- 172.21.15.32:0/1796681102 shutdown_connections 2024-04-17T10:04:14.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.562+0000 7fba3ee00640 1 -- 172.21.15.32:0/1796681102 wait complete. 2024-04-17T10:04:14.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.562+0000 7fba3ee00640 1 Processor -- start 2024-04-17T10:04:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.562+0000 7fba3ee00640 1 -- start start 2024-04-17T10:04:14.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.562+0000 7fba3ee00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba380cf370 0x7fba38138df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:14.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.562+0000 7fba3ee00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba380d09f0 0x7fba38141b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:14.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.566+0000 7fba3ee00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba3813f6e0 0x7fba3813e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:14.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.566+0000 7fba3ee00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fba380cff10 con 0x7fba380cf370 2024-04-17T10:04:14.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.566+0000 7fba3ee00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fba380cfd90 con 0x7fba3813f6e0 2024-04-17T10:04:14.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.566+0000 7fba3ee00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fba38072ad0 con 0x7fba380d09f0 2024-04-17T10:04:14.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.566+0000 7fba37e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba380cf370 0x7fba38138df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:14.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.566+0000 7fba37e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba380cf370 0x7fba38138df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50682/0 (socket says 172.21.15.32:50682) 2024-04-17T10:04:14.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.566+0000 7fba37e00640 1 -- 172.21.15.32:0/1738003119 learned_addr learned my addr 172.21.15.32:0/1738003119 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.566+0000 7fba3ce00640 1 --2- 172.21.15.32:0/1738003119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba3813f6e0 0x7fba3813e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.566+0000 7fba2f400640 1 --2- 172.21.15.32:0/1738003119 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba380d09f0 0x7fba38141b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.566+0000 7fba37e00640 1 -- 172.21.15.32:0/1738003119 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba380d09f0 msgr2=0x7fba38141b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.566+0000 7fba37e00640 1 --2- 172.21.15.32:0/1738003119 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba380d09f0 0x7fba38141b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.566+0000 7fba37e00640 1 -- 172.21.15.32:0/1738003119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba3813f6e0 msgr2=0x7fba3813e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.566+0000 7fba37e00640 1 --2- 172.21.15.32:0/1738003119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba3813f6e0 0x7fba3813e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.566+0000 7fba37e00640 1 -- 172.21.15.32:0/1738003119 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba380b2700 con 0x7fba380cf370 2024-04-17T10:04:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.566+0000 7fba3ce00640 1 --2- 172.21.15.32:0/1738003119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba3813f6e0 0x7fba3813e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.566+0000 7fba37e00640 1 --2- 172.21.15.32:0/1738003119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba380cf370 0x7fba38138df0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fba2800c7f0 tx=0x7fba2800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.566+0000 7fba34c00640 1 -- 172.21.15.32:0/1738003119 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba28015070 con 0x7fba380cf370 2024-04-17T10:04:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.566+0000 7fba34c00640 1 -- 172.21.15.32:0/1738003119 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba28007820 con 0x7fba380cf370 2024-04-17T10:04:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.566+0000 7fba34c00640 1 -- 172.21.15.32:0/1738003119 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba28007c10 con 0x7fba380cf370 2024-04-17T10:04:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.566+0000 7fba3ee00640 1 -- 172.21.15.32:0/1738003119 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba380b29e0 con 0x7fba380cf370 2024-04-17T10:04:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.566+0000 7fba3ee00640 1 -- 172.21.15.32:0/1738003119 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba38154cb0 con 0x7fba380cf370 2024-04-17T10:04:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.566+0000 7fba3ee00640 1 -- 172.21.15.32:0/1738003119 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba38068a90 con 0x7fba380cf370 2024-04-17T10:04:14.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.566+0000 7fba34c00640 1 -- 172.21.15.32:0/1738003119 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fba28018070 con 0x7fba380cf370 2024-04-17T10:04:14.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.570+0000 7fba34c00640 1 --2- 172.21.15.32:0/1738003119 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fba14068380 0x7fba1406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:14.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.570+0000 7fba34c00640 1 -- 172.21.15.32:0/1738003119 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7fba28086b80 con 0x7fba380cf370 2024-04-17T10:04:14.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.570+0000 7fba2f400640 1 --2- 172.21.15.32:0/1738003119 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fba14068380 0x7fba1406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:14.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.570+0000 7fba2f400640 1 --2- 172.21.15.32:0/1738003119 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fba14068380 0x7fba1406a840 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fba38156890 tx=0x7fba20005b80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:14.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.574+0000 7fba34c00640 1 -- 172.21.15.32:0/1738003119 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fba28057090 con 0x7fba380cf370 2024-04-17T10:04:14.573 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:04:14.576 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:04:14.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.750+0000 7fba3ee00640 1 -- 172.21.15.32:0/1738003119 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fba380d1030 con 0x7fba380cf370 2024-04-17T10:04:14.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.750+0000 7fba34c00640 1 -- 172.21.15.32:0/1738003119 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v112) ==== 80+0+120 (secure 0 0 0) 0x7fba28056990 con 0x7fba380cf370 2024-04-17T10:04:14.747 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6837/901447746 2024-04-18T10:04:03.080821+0000 2024-04-17T10:04:14.747 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6836/901447746 2024-04-18T10:04:03.080821+0000 2024-04-17T10:04:14.747 INFO:teuthology.orchestra.run.smithi032.stderr:listed 2 entries 2024-04-17T10:04:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.754+0000 7fba3ee00640 1 -- 172.21.15.32:0/1738003119 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fba14068380 msgr2=0x7fba1406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.754+0000 7fba3ee00640 1 --2- 172.21.15.32:0/1738003119 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fba14068380 0x7fba1406a840 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fba38156890 tx=0x7fba20005b80 comp rx=0 tx=0).stop 2024-04-17T10:04:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.754+0000 7fba3ee00640 1 -- 172.21.15.32:0/1738003119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba380cf370 msgr2=0x7fba38138df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.754+0000 7fba3ee00640 1 --2- 172.21.15.32:0/1738003119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba380cf370 0x7fba38138df0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fba2800c7f0 tx=0x7fba2800ccb0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.754+0000 7fba3ee00640 1 -- 172.21.15.32:0/1738003119 shutdown_connections 2024-04-17T10:04:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.754+0000 7fba3ee00640 1 --2- 172.21.15.32:0/1738003119 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fba14068380 0x7fba1406a840 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.754+0000 7fba3ee00640 1 --2- 172.21.15.32:0/1738003119 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba380d09f0 0x7fba38141b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.754+0000 7fba3ee00640 1 --2- 172.21.15.32:0/1738003119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba3813f6e0 0x7fba3813e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.754+0000 7fba3ee00640 1 --2- 172.21.15.32:0/1738003119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba380cf370 0x7fba38138df0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.754+0000 7fba3ee00640 1 -- 172.21.15.32:0/1738003119 >> 172.21.15.32:0/1738003119 conn(0x7fba3806ba30 msgr2=0x7fba3815de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.754+0000 7fba3ee00640 1 -- 172.21.15.32:0/1738003119 shutdown_connections 2024-04-17T10:04:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:14.754+0000 7fba3ee00640 1 -- 172.21.15.32:0/1738003119 wait complete. 2024-04-17T10:04:14.769 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2024-04-17T10:04:14.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:14.769 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T10:04:14.787 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[52329]: fuse finished with error 0 and tester_r 0 2024-04-17T10:04:14.790 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:04:20.891 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T10:04:20.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:20.891 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T10:04:20.902 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:04:21.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.102+0000 7f7377e00640 1 Processor -- start 2024-04-17T10:04:21.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.102+0000 7f7377e00640 1 -- start start 2024-04-17T10:04:21.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.102+0000 7f7377e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f737809a9f0 0x7f737806ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.102+0000 7f7377e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f73780d03f0 0x7f7378070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.102+0000 7f7377e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f737806eb40 con 0x7f73780d0870 2024-04-17T10:04:21.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.102+0000 7f7377e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f73780b4970 con 0x7f737809a9f0 2024-04-17T10:04:21.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.102+0000 7f7377e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f73780b4c50 con 0x7f73780d03f0 2024-04-17T10:04:21.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.102+0000 7f7376a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f737809a9f0 0x7f737806ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.102+0000 7f7377400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f73780d0870 0x7f737809add0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50322/0 (socket says 172.21.15.32:50322) 2024-04-17T10:04:21.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.102+0000 7f7377400640 1 -- 172.21.15.32:0/3359763214 learned_addr learned my addr 172.21.15.32:0/3359763214 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:21.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.102+0000 7f7376000640 1 -- 172.21.15.32:0/3359763214 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3542879640 0 0) 0x7f737806eb40 con 0x7f73780d0870 2024-04-17T10:04:21.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.102+0000 7f7376a00640 1 -- 172.21.15.32:0/3359763214 >> v2:172.21.15.144:3300/0 conn(0x7f73780d03f0 msgr2=0x7f7378070880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:21.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.102+0000 7f736e000640 1 --2- 172.21.15.32:0/3359763214 >> v2:172.21.15.144:3300/0 conn(0x7f73780d03f0 0x7f7378070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.102+0000 7f7376a00640 1 --2- 172.21.15.32:0/3359763214 >> v2:172.21.15.144:3300/0 conn(0x7f73780d03f0 0x7f7378070880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.102+0000 7f7376a00640 1 -- 172.21.15.32:0/3359763214 >> v1:172.21.15.32:6789/0 conn(0x7f73780d0870 legacy=0x7f737809add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.102+0000 7f7376a00640 1 -- 172.21.15.32:0/3359763214 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737806d550 con 0x7f737809a9f0 2024-04-17T10:04:21.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.102+0000 7f736e000640 1 --2- 172.21.15.32:0/3359763214 >> v2:172.21.15.144:3300/0 conn(0x7f73780d03f0 0x7f7378070880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:21.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.106+0000 7f7376a00640 1 --2- 172.21.15.32:0/3359763214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f737809a9f0 0x7f737806ddf0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f7378073150 tx=0x7f736801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b713c00706fbe001 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:21.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.106+0000 7f7376000640 1 -- 172.21.15.32:0/3359763214 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73680b2070 con 0x7f737809a9f0 2024-04-17T10:04:21.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.106+0000 7f7376000640 1 -- 172.21.15.32:0/3359763214 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73680a58e0 con 0x7f737809a9f0 2024-04-17T10:04:21.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.106+0000 7f7376000640 1 -- 172.21.15.32:0/3359763214 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73680a5cb0 con 0x7f737809a9f0 2024-04-17T10:04:21.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.106+0000 7f7377e00640 1 -- 172.21.15.32:0/3359763214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f737809a9f0 msgr2=0x7f737806ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.106+0000 7f7377e00640 1 --2- 172.21.15.32:0/3359763214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f737809a9f0 0x7f737806ddf0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f7378073150 tx=0x7f736801c930 comp rx=0 tx=0).stop 2024-04-17T10:04:21.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.106+0000 7f7377e00640 1 -- 172.21.15.32:0/3359763214 shutdown_connections 2024-04-17T10:04:21.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.106+0000 7f7377e00640 1 --2- 172.21.15.32:0/3359763214 >> v2:172.21.15.144:3300/0 conn(0x7f73780d03f0 0x7f7378070880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.106+0000 7f7377e00640 1 --2- 172.21.15.32:0/3359763214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f737809a9f0 0x7f737806ddf0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.106+0000 7f7377e00640 1 -- 172.21.15.32:0/3359763214 >> 172.21.15.32:0/3359763214 conn(0x7f73780b3e50 msgr2=0x7f73780b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:21.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.106+0000 7f7377e00640 1 -- 172.21.15.32:0/3359763214 shutdown_connections 2024-04-17T10:04:21.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.106+0000 7f7377e00640 1 -- 172.21.15.32:0/3359763214 wait complete. 2024-04-17T10:04:21.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f7377e00640 1 Processor -- start 2024-04-17T10:04:21.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f7377e00640 1 -- start start 2024-04-17T10:04:21.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f7377e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f737809a9f0 0x7f73781525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f7377e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73780d03f0 0x7f73780b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f7377e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73780d0870 0x7f737815d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f7377e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f737816c0d0 con 0x7f737809a9f0 2024-04-17T10:04:21.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f7377e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f737816bf50 con 0x7f73780d0870 2024-04-17T10:04:21.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f7377e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f737816c250 con 0x7f73780d03f0 2024-04-17T10:04:21.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f7376a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f737809a9f0 0x7f73781525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f7376a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f737809a9f0 0x7f73781525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60630/0 (socket says 172.21.15.32:60630) 2024-04-17T10:04:21.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f7376a00640 1 -- 172.21.15.32:0/600991431 learned_addr learned my addr 172.21.15.32:0/600991431 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:21.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f7377400640 1 --2- 172.21.15.32:0/600991431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73780d0870 0x7f737815d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f736e000640 1 --2- 172.21.15.32:0/600991431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73780d03f0 0x7f73780b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f7376a00640 1 -- 172.21.15.32:0/600991431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73780d03f0 msgr2=0x7f73780b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f7376a00640 1 --2- 172.21.15.32:0/600991431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73780d03f0 0x7f73780b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f7376a00640 1 -- 172.21.15.32:0/600991431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73780d0870 msgr2=0x7f737815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f7376a00640 1 --2- 172.21.15.32:0/600991431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73780d0870 0x7f737815d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f7376a00640 1 -- 172.21.15.32:0/600991431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7378157c40 con 0x7f737809a9f0 2024-04-17T10:04:21.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f736e000640 1 --2- 172.21.15.32:0/600991431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73780d03f0 0x7f73780b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:21.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f7376a00640 1 --2- 172.21.15.32:0/600991431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f737809a9f0 0x7f73781525b0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f73680b0040 tx=0x7f73680196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:21.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f736f400640 1 -- 172.21.15.32:0/600991431 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73680b2050 con 0x7f737809a9f0 2024-04-17T10:04:21.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f736f400640 1 -- 172.21.15.32:0/600991431 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73680be070 con 0x7f737809a9f0 2024-04-17T10:04:21.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f736f400640 1 -- 172.21.15.32:0/600991431 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73680ada50 con 0x7f737809a9f0 2024-04-17T10:04:21.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f7377e00640 1 -- 172.21.15.32:0/600991431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7378157f20 con 0x7f737809a9f0 2024-04-17T10:04:21.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.110+0000 7f7377e00640 1 -- 172.21.15.32:0/600991431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73781584e0 con 0x7f737809a9f0 2024-04-17T10:04:21.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.114+0000 7f7377e00640 1 -- 172.21.15.32:0/600991431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f737804fd00 con 0x7f737809a9f0 2024-04-17T10:04:21.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.114+0000 7f736f400640 1 -- 172.21.15.32:0/600991431 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f73680adbf0 con 0x7f737809a9f0 2024-04-17T10:04:21.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.114+0000 7f736f400640 1 --2- 172.21.15.32:0/600991431 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7354068380 0x7f735406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.114+0000 7f736f400640 1 -- 172.21.15.32:0/600991431 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7f7368121e00 con 0x7f737809a9f0 2024-04-17T10:04:21.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.114+0000 7f736e000640 1 --2- 172.21.15.32:0/600991431 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7354068380 0x7f735406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.114+0000 7f736e000640 1 --2- 172.21.15.32:0/600991431 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7354068380 0x7f735406a840 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7360008890 tx=0x7f7360008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:21.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.118+0000 7f736f400640 1 -- 172.21.15.32:0/600991431 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f7368016520 con 0x7f737809a9f0 2024-04-17T10:04:21.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.318+0000 7f7377e00640 1 -- 172.21.15.32:0/600991431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73780d20d0 con 0x7f737809a9f0 2024-04-17T10:04:21.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.318+0000 7f736f400640 1 -- 172.21.15.32:0/600991431 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5209 (secure 0 0 0) 0x7f73680ab340 con 0x7f737809a9f0 2024-04-17T10:04:21.315 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:21.315 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":104,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7586,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3601627312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3601627312},{"type":"v1","addr":"172.21.15.144:6834","nonce":3601627312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2382359917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2382359917},{"type":"v1","addr":"172.21.15.144:6835","nonce":2382359917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1857552160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1857552160},{"type":"v1","addr":"172.21.15.32:6835","nonce":1857552160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:03:37.619785+0000","modified":"2024-04-17T10:04:07.100059+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9615},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9615":{"gid":9615,"name":"a","rank":0,"incarnation":101,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6837/578643390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":578643390},{"type":"v1","addr":"172.21.15.32:6837","nonce":578643390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9615,"qdb_cluster":[9615]},"id":8}]} 2024-04-17T10:04:21.315 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 104 2024-04-17T10:04:21.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.322+0000 7f7377e00640 1 -- 172.21.15.32:0/600991431 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7354068380 msgr2=0x7f735406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.322+0000 7f7377e00640 1 --2- 172.21.15.32:0/600991431 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7354068380 0x7f735406a840 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7360008890 tx=0x7f7360008040 comp rx=0 tx=0).stop 2024-04-17T10:04:21.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.322+0000 7f7377e00640 1 -- 172.21.15.32:0/600991431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f737809a9f0 msgr2=0x7f73781525b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.322+0000 7f7377e00640 1 --2- 172.21.15.32:0/600991431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f737809a9f0 0x7f73781525b0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f73680b0040 tx=0x7f73680196d0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.322+0000 7f7377e00640 1 -- 172.21.15.32:0/600991431 shutdown_connections 2024-04-17T10:04:21.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.322+0000 7f7377e00640 1 --2- 172.21.15.32:0/600991431 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7354068380 0x7f735406a840 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.322+0000 7f7377e00640 1 --2- 172.21.15.32:0/600991431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73780d03f0 0x7f73780b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.322+0000 7f7377e00640 1 --2- 172.21.15.32:0/600991431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73780d0870 0x7f737815d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.322+0000 7f7377e00640 1 --2- 172.21.15.32:0/600991431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f737809a9f0 0x7f73781525b0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.322+0000 7f7377e00640 1 -- 172.21.15.32:0/600991431 >> 172.21.15.32:0/600991431 conn(0x7f73780b3e50 msgr2=0x7f73780af0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:21.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.322+0000 7f7377e00640 1 -- 172.21.15.32:0/600991431 shutdown_connections 2024-04-17T10:04:21.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.322+0000 7f7377e00640 1 -- 172.21.15.32:0/600991431 wait complete. 2024-04-17T10:04:21.337 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parent mytag123 2024-04-17T10:04:21.504 INFO:teuthology.orchestra.run.smithi032.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-04-17T10:04:21.504 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-04-17T10:04:21.504 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-17T10:04:21.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.554+0000 7fcbc7ad3080 1 Processor -- start 2024-04-17T10:04:21.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.554+0000 7fcbc7ad3080 1 -- start start 2024-04-17T10:04:21.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc7ad3080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560562337100 0x560562380610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc7ad3080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560562381230 0x5605623686d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc7ad3080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56056240e270 0x560562368c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc7ad3080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5605622ddf90 con 0x560562381230 2024-04-17T10:04:21.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc7ad3080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5605622e1fd0 con 0x56056240e270 2024-04-17T10:04:21.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc7ad3080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x56056240ed60 con 0x560562337100 2024-04-17T10:04:21.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbbfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560562381230 0x5605623686d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbbfe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560562381230 0x5605623686d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60648/0 (socket says 172.21.15.32:60648) 2024-04-17T10:04:21.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbbfe00640 1 -- 172.21.15.32:0/710700991 learned_addr learned my addr 172.21.15.32:0/710700991 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:21.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc5600640 1 --2- 172.21.15.32:0/710700991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56056240e270 0x560562368c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc4c00640 1 --2- 172.21.15.32:0/710700991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560562337100 0x560562380610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbbfe00640 1 -- 172.21.15.32:0/710700991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560562337100 msgr2=0x560562380610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:21.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbbfe00640 1 --2- 172.21.15.32:0/710700991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560562337100 0x560562380610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbbfe00640 1 -- 172.21.15.32:0/710700991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56056240e270 msgr2=0x560562368c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:21.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbbfe00640 1 --2- 172.21.15.32:0/710700991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56056240e270 0x560562368c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbbfe00640 1 -- 172.21.15.32:0/710700991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56056240a020 con 0x560562381230 2024-04-17T10:04:21.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc5600640 1 --2- 172.21.15.32:0/710700991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56056240e270 0x560562368c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:21.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbbfe00640 1 --2- 172.21.15.32:0/710700991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560562381230 0x5605623686d0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000b330 tx=0x7fcbb000b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96dd0d6b39c649de server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:21.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbbea00640 1 -- 172.21.15.32:0/710700991 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0012020 con 0x560562381230 2024-04-17T10:04:21.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbbea00640 1 -- 172.21.15.32:0/710700991 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb000e690 con 0x560562381230 2024-04-17T10:04:21.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbbea00640 1 -- 172.21.15.32:0/710700991 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb000ea60 con 0x560562381230 2024-04-17T10:04:21.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc7ad3080 1 -- 172.21.15.32:0/710700991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560562381230 msgr2=0x5605623686d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc7ad3080 1 --2- 172.21.15.32:0/710700991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560562381230 0x5605623686d0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000b330 tx=0x7fcbb000b7f0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc7ad3080 1 -- 172.21.15.32:0/710700991 shutdown_connections 2024-04-17T10:04:21.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc7ad3080 1 --2- 172.21.15.32:0/710700991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56056240e270 0x560562368c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc7ad3080 1 --2- 172.21.15.32:0/710700991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560562381230 0x5605623686d0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc7ad3080 1 --2- 172.21.15.32:0/710700991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560562337100 0x560562380610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc7ad3080 1 -- 172.21.15.32:0/710700991 >> 172.21.15.32:0/710700991 conn(0x560562337620 msgr2=0x56056240c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:21.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc7ad3080 1 -- 172.21.15.32:0/710700991 shutdown_connections 2024-04-17T10:04:21.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc7ad3080 1 -- 172.21.15.32:0/710700991 wait complete. 2024-04-17T10:04:21.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc7ad3080 1 Processor -- start 2024-04-17T10:04:21.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc7ad3080 1 -- start start 2024-04-17T10:04:21.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc7ad3080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560562337100 0x5605624098a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc7ad3080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560562381230 0x5605624038f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc7ad3080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56056240e270 0x56056237c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc7ad3080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56056240ad10 con 0x56056240e270 2024-04-17T10:04:21.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc7ad3080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56056240af70 con 0x560562381230 2024-04-17T10:04:21.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.558+0000 7fcbc7ad3080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x56056240b210 con 0x560562337100 2024-04-17T10:04:21.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.562+0000 7fcbc5600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56056240e270 0x56056237c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.562+0000 7fcbbfe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560562381230 0x5605624038f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.562+0000 7fcbc4c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560562337100 0x5605624098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.562+0000 7fcbc4c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560562337100 0x5605624098a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:44242/0 (socket says 172.21.15.32:44242) 2024-04-17T10:04:21.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.562+0000 7fcbc4c00640 1 -- 172.21.15.32:0/1097058772 learned_addr learned my addr 172.21.15.32:0/1097058772 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:21.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.562+0000 7fcbc5600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56056240e270 0x56056237c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60662/0 (socket says 172.21.15.32:60662) 2024-04-17T10:04:21.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.562+0000 7fcbbfe00640 1 --2- 172.21.15.32:0/1097058772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560562381230 0x5605624038f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.32:57250/0 (socket says 172.21.15.32:57250) 2024-04-17T10:04:21.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.562+0000 7fcbc5600640 1 -- 172.21.15.32:0/1097058772 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560562337100 msgr2=0x5605624098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.562+0000 7fcbc5600640 1 --2- 172.21.15.32:0/1097058772 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560562337100 0x5605624098a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.562+0000 7fcbc5600640 1 -- 172.21.15.32:0/1097058772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560562381230 msgr2=0x5605624038f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.562+0000 7fcbc5600640 1 --2- 172.21.15.32:0/1097058772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560562381230 0x5605624038f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.562+0000 7fcbc5600640 1 -- 172.21.15.32:0/1097058772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56056237cf10 con 0x56056240e270 2024-04-17T10:04:21.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.562+0000 7fcbc5600640 1 --2- 172.21.15.32:0/1097058772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56056240e270 0x56056237c6d0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00aa000 tx=0x7fcbc0017d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:21.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.562+0000 7fcbbd600640 1 -- 172.21.15.32:0/1097058772 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc00b8020 con 0x56056240e270 2024-04-17T10:04:21.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.562+0000 7fcbbd600640 1 -- 172.21.15.32:0/1097058772 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbc00b4610 con 0x56056240e270 2024-04-17T10:04:21.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.562+0000 7fcbbd600640 1 -- 172.21.15.32:0/1097058772 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc00b0040 con 0x56056240e270 2024-04-17T10:04:21.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.562+0000 7fcbc7ad3080 1 -- 172.21.15.32:0/1097058772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56056237a060 con 0x56056240e270 2024-04-17T10:04:21.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.562+0000 7fcbc7ad3080 1 -- 172.21.15.32:0/1097058772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56056237a5a0 con 0x56056240e270 2024-04-17T10:04:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.562+0000 7fcbbd600640 1 -- 172.21.15.32:0/1097058772 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcbc00bb020 con 0x56056240e270 2024-04-17T10:04:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.562+0000 7fcbbd600640 1 --2- 172.21.15.32:0/1097058772 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcba4066d60 0x7fcba4069220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.562+0000 7fcbbd600640 1 -- 172.21.15.32:0/1097058772 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7fcbc011fc70 con 0x56056240e270 2024-04-17T10:04:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.566+0000 7fcbc4c00640 1 --2- 172.21.15.32:0/1097058772 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcba4066d60 0x7fcba4069220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.566+0000 7fcbc7ad3080 1 --2- 172.21.15.32:0/1097058772 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x56056237aa40 0x56056237ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.566+0000 7fcbc7ad3080 1 -- 172.21.15.32:0/1097058772 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:1 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) -- 0x56056237b360 con 0x56056237aa40 2024-04-17T10:04:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.566+0000 7fcbbfe00640 1 --2- 172.21.15.32:0/1097058772 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x56056237aa40 0x56056237ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.566+0000 7fcbc4c00640 1 --2- 172.21.15.32:0/1097058772 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcba4066d60 0x7fcba4069220 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8003f20 tx=0x7fcbb80038a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.566+0000 7fcbbfe00640 1 --2- 172.21.15.32:0/1097058772 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x56056237aa40 0x56056237ae20 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.566+0000 7fcbbfe00640 1 -- 172.21.15.32:0/1097058772 <== osd.0 v2:172.21.15.32:6800/2392056658 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7fcbb0008030 con 0x56056237aa40 2024-04-17T10:04:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-04-17T10:04:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.566+0000 7fcbc7ad3080 1 -- 172.21.15.32:0/1097058772 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x56056237aa40 msgr2=0x56056237ae20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.566+0000 7fcbc7ad3080 1 --2- 172.21.15.32:0/1097058772 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x56056237aa40 0x56056237ae20 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.566+0000 7fcbc7ad3080 1 -- 172.21.15.32:0/1097058772 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcba4066d60 msgr2=0x7fcba4069220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.566+0000 7fcbc7ad3080 1 --2- 172.21.15.32:0/1097058772 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcba4066d60 0x7fcba4069220 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8003f20 tx=0x7fcbb80038a0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.566+0000 7fcbc7ad3080 1 -- 172.21.15.32:0/1097058772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56056240e270 msgr2=0x56056237c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.566+0000 7fcbc7ad3080 1 --2- 172.21.15.32:0/1097058772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56056240e270 0x56056237c6d0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00aa000 tx=0x7fcbc0017d80 comp rx=0 tx=0).stop 2024-04-17T10:04:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.566+0000 7fcbc5600640 1 -- 172.21.15.32:0/1097058772 reap_dead start 2024-04-17T10:04:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.566+0000 7fcbc7ad3080 1 -- 172.21.15.32:0/1097058772 shutdown_connections 2024-04-17T10:04:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.566+0000 7fcbc7ad3080 1 -- 172.21.15.32:0/1097058772 >> 172.21.15.32:0/1097058772 conn(0x560562337620 msgr2=0x56056240c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.566+0000 7fcbc7ad3080 1 -- 172.21.15.32:0/1097058772 shutdown_connections 2024-04-17T10:04:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.566+0000 7fcbc7ad3080 1 -- 172.21.15.32:0/1097058772 wait complete. 2024-04-17T10:04:21.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:04:21.566 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-17T10:04:21.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.662+0000 7fd329964080 1 Processor -- start 2024-04-17T10:04:21.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.662+0000 7fd329964080 1 -- start start 2024-04-17T10:04:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd329964080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd30c003100 0x55a363d4b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd329964080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a363d4c0c0 0x55a363d65080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd329964080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a363d4c4a0 0x55a363df1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd329964080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a363cc1f90 con 0x7fd30c003100 2024-04-17T10:04:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd329964080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a363cc5fd0 con 0x55a363d4c4a0 2024-04-17T10:04:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd329964080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55a363df4b20 con 0x55a363d4c0c0 2024-04-17T10:04:21.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd326c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd30c003100 0x55a363d4b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd326c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd30c003100 0x55a363d4b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60674/0 (socket says 172.21.15.32:60674) 2024-04-17T10:04:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd326c00640 1 -- 172.21.15.32:0/1591484080 learned_addr learned my addr 172.21.15.32:0/1591484080 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd326200640 1 --2- 172.21.15.32:0/1591484080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a363d4c0c0 0x55a363d65080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd327600640 1 --2- 172.21.15.32:0/1591484080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a363d4c4a0 0x55a363df1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd326c00640 1 -- 172.21.15.32:0/1591484080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a363d4c0c0 msgr2=0x55a363d65080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd326c00640 1 --2- 172.21.15.32:0/1591484080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a363d4c0c0 0x55a363d65080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd326c00640 1 -- 172.21.15.32:0/1591484080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a363d4c4a0 msgr2=0x55a363df1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd326c00640 1 --2- 172.21.15.32:0/1591484080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a363d4c4a0 0x55a363df1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd326c00640 1 -- 172.21.15.32:0/1591484080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a363d1c990 con 0x7fd30c003100 2024-04-17T10:04:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd327600640 1 --2- 172.21.15.32:0/1591484080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a363d4c4a0 0x55a363df1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:21.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd326c00640 1 --2- 172.21.15.32:0/1591484080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd30c003100 0x55a363d4b380 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fd318009720 tx=0x7fd318005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95472be492685b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd324e00640 1 -- 172.21.15.32:0/1591484080 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318008bf0 con 0x7fd30c003100 2024-04-17T10:04:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd324e00640 1 -- 172.21.15.32:0/1591484080 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd318008d90 con 0x7fd30c003100 2024-04-17T10:04:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd324e00640 1 -- 172.21.15.32:0/1591484080 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318015680 con 0x7fd30c003100 2024-04-17T10:04:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd329964080 1 -- 172.21.15.32:0/1591484080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd30c003100 msgr2=0x55a363d4b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd329964080 1 --2- 172.21.15.32:0/1591484080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd30c003100 0x55a363d4b380 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fd318009720 tx=0x7fd318005a10 comp rx=0 tx=0).stop 2024-04-17T10:04:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd329964080 1 -- 172.21.15.32:0/1591484080 shutdown_connections 2024-04-17T10:04:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd329964080 1 --2- 172.21.15.32:0/1591484080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a363d4c0c0 0x55a363d65080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd329964080 1 --2- 172.21.15.32:0/1591484080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a363d4c4a0 0x55a363df1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd329964080 1 --2- 172.21.15.32:0/1591484080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd30c003100 0x55a363d4b380 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd329964080 1 -- 172.21.15.32:0/1591484080 >> 172.21.15.32:0/1591484080 conn(0x55a363d47a00 msgr2=0x55a363df3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd329964080 1 -- 172.21.15.32:0/1591484080 shutdown_connections 2024-04-17T10:04:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd329964080 1 -- 172.21.15.32:0/1591484080 wait complete. 2024-04-17T10:04:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd329964080 1 Processor -- start 2024-04-17T10:04:21.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd329964080 1 -- start start 2024-04-17T10:04:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.666+0000 7fd329964080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a363d4c0c0 0x55a363e80ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd329964080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a363d4c4a0 0x55a363e81400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd329964080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd30c003100 0x55a363e7eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd329964080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a363d1ced0 con 0x7fd30c003100 2024-04-17T10:04:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd329964080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a363d1cc30 con 0x55a363d4c4a0 2024-04-17T10:04:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd329964080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55a363d1d170 con 0x55a363d4c0c0 2024-04-17T10:04:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd327600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd30c003100 0x55a363e7eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd327600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd30c003100 0x55a363e7eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60684/0 (socket says 172.21.15.32:60684) 2024-04-17T10:04:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd327600640 1 -- 172.21.15.32:0/1080257524 learned_addr learned my addr 172.21.15.32:0/1080257524 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd326c00640 1 --2- 172.21.15.32:0/1080257524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a363d4c0c0 0x55a363e80ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd326200640 1 --2- 172.21.15.32:0/1080257524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a363d4c4a0 0x55a363e81400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd327600640 1 -- 172.21.15.32:0/1080257524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a363d4c0c0 msgr2=0x55a363e80ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd327600640 1 --2- 172.21.15.32:0/1080257524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a363d4c0c0 0x55a363e80ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd327600640 1 -- 172.21.15.32:0/1080257524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a363d4c4a0 msgr2=0x55a363e81400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd327600640 1 --2- 172.21.15.32:0/1080257524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a363d4c4a0 0x55a363e81400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd327600640 1 -- 172.21.15.32:0/1080257524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a363e81b80 con 0x7fd30c003100 2024-04-17T10:04:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd326c00640 1 --2- 172.21.15.32:0/1080257524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a363d4c0c0 0x55a363e80ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd326200640 1 --2- 172.21.15.32:0/1080257524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a363d4c4a0 0x55a363e81400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd327600640 1 --2- 172.21.15.32:0/1080257524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd30c003100 0x55a363e7eda0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x55a363d48c70 tx=0x7fd32000e540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd31f400640 1 -- 172.21.15.32:0/1080257524 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32000c070 con 0x7fd30c003100 2024-04-17T10:04:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd31f400640 1 -- 172.21.15.32:0/1080257524 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3200133f0 con 0x7fd30c003100 2024-04-17T10:04:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd31f400640 1 -- 172.21.15.32:0/1080257524 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32000f040 con 0x7fd30c003100 2024-04-17T10:04:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd329964080 1 -- 172.21.15.32:0/1080257524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a363e81e20 con 0x7fd30c003100 2024-04-17T10:04:21.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd329964080 1 -- 172.21.15.32:0/1080257524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a363e82360 con 0x7fd30c003100 2024-04-17T10:04:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd31f400640 1 -- 172.21.15.32:0/1080257524 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd32001a020 con 0x7fd30c003100 2024-04-17T10:04:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.670+0000 7fd31f400640 1 --2- 172.21.15.32:0/1080257524 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd308066d90 0x7fd308069250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.674+0000 7fd31f400640 1 -- 172.21.15.32:0/1080257524 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7fd320084530 con 0x7fd30c003100 2024-04-17T10:04:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.674+0000 7fd326c00640 1 --2- 172.21.15.32:0/1080257524 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd308066d90 0x7fd308069250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.674+0000 7fd329964080 1 --2- 172.21.15.32:0/1080257524 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55a363e83ab0 0x55a363dde510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.674+0000 7fd329964080 1 -- 172.21.15.32:0/1080257524 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:1 17.12 17:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) -- 0x55a363ddea50 con 0x55a363e83ab0 2024-04-17T10:04:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.674+0000 7fd326200640 1 --2- 172.21.15.32:0/1080257524 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55a363e83ab0 0x55a363dde510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.674+0000 7fd326c00640 1 --2- 172.21.15.32:0/1080257524 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd308066d90 0x7fd308069250 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd318000c10 tx=0x7fd318014000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.674+0000 7fd326200640 1 --2- 172.21.15.32:0/1080257524 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55a363e83ab0 0x55a363dde510 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.674+0000 7fd326200640 1 -- 172.21.15.32:0/1080257524 <== osd.3 v2:172.21.15.32:6824/2182899298 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7fd3140ab3f0 con 0x55a363e83ab0 2024-04-17T10:04:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.674+0000 7fd329964080 1 -- 172.21.15.32:0/1080257524 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55a363e83ab0 msgr2=0x55a363dde510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.674+0000 7fd329964080 1 --2- 172.21.15.32:0/1080257524 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55a363e83ab0 0x55a363dde510 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.674+0000 7fd329964080 1 -- 172.21.15.32:0/1080257524 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd308066d90 msgr2=0x7fd308069250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.674+0000 7fd329964080 1 --2- 172.21.15.32:0/1080257524 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd308066d90 0x7fd308069250 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd318000c10 tx=0x7fd318014000 comp rx=0 tx=0).stop 2024-04-17T10:04:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.674+0000 7fd329964080 1 -- 172.21.15.32:0/1080257524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd30c003100 msgr2=0x55a363e7eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.674+0000 7fd329964080 1 --2- 172.21.15.32:0/1080257524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd30c003100 0x55a363e7eda0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x55a363d48c70 tx=0x7fd32000e540 comp rx=0 tx=0).stop 2024-04-17T10:04:21.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.674+0000 7fd327600640 1 -- 172.21.15.32:0/1080257524 reap_dead start 2024-04-17T10:04:21.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.674+0000 7fd329964080 1 -- 172.21.15.32:0/1080257524 shutdown_connections 2024-04-17T10:04:21.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.674+0000 7fd329964080 1 -- 172.21.15.32:0/1080257524 >> 172.21.15.32:0/1080257524 conn(0x55a363d47a00 msgr2=0x55a363d4b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:21.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.674+0000 7fd329964080 1 -- 172.21.15.32:0/1080257524 shutdown_connections 2024-04-17T10:04:21.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.674+0000 7fd329964080 1 -- 172.21.15.32:0/1080257524 wait complete. 2024-04-17T10:04:21.675 INFO:teuthology.orchestra.run.smithi032.stdout:mytag123 2024-04-17T10:04:21.675 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-17T10:04:21.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.726+0000 7f19ddb70080 1 Processor -- start 2024-04-17T10:04:21.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.726+0000 7f19ddb70080 1 -- start start 2024-04-17T10:04:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.726+0000 7f19ddb70080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19c0002440 0x55b5f9110370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.726+0000 7f19ddb70080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5f909cf20 0x55b5f90965c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.726+0000 7f19ddb70080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b5f9113830 0x55b5f910f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.726+0000 7f19ddb70080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b5f8ffcf90 con 0x7f19c0002440 2024-04-17T10:04:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.726+0000 7f19ddb70080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b5f9000fd0 con 0x55b5f909cf20 2024-04-17T10:04:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.726+0000 7f19ddb70080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b5f91148c0 con 0x55b5f9113830 2024-04-17T10:04:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19dae00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19c0002440 0x55b5f9110370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19dae00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19c0002440 0x55b5f9110370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60704/0 (socket says 172.21.15.32:60704) 2024-04-17T10:04:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19dae00640 1 -- 172.21.15.32:0/1268137416 learned_addr learned my addr 172.21.15.32:0/1268137416 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19db800640 1 --2- 172.21.15.32:0/1268137416 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b5f9113830 0x55b5f910f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19dae00640 1 -- 172.21.15.32:0/1268137416 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b5f9113830 msgr2=0x55b5f910f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19dae00640 1 --2- 172.21.15.32:0/1268137416 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b5f9113830 0x55b5f910f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19da400640 1 --2- 172.21.15.32:0/1268137416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5f909cf20 0x55b5f90965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19dae00640 1 -- 172.21.15.32:0/1268137416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5f909cf20 msgr2=0x55b5f90965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19dae00640 1 --2- 172.21.15.32:0/1268137416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5f909cf20 0x55b5f90965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19dae00640 1 -- 172.21.15.32:0/1268137416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5f9058040 con 0x7f19c0002440 2024-04-17T10:04:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19db800640 1 --2- 172.21.15.32:0/1268137416 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b5f9113830 0x55b5f910f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19da400640 1 --2- 172.21.15.32:0/1268137416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5f909cf20 0x55b5f90965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19dae00640 1 --2- 172.21.15.32:0/1268137416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19c0002440 0x55b5f9110370 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f19cc009900 tx=0x7f19cc019fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e1f1abdbfc4fd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19d9000640 1 -- 172.21.15.32:0/1268137416 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0a8a70 con 0x7f19c0002440 2024-04-17T10:04:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19d9000640 1 -- 172.21.15.32:0/1268137416 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19cc0a8c10 con 0x7f19c0002440 2024-04-17T10:04:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19d9000640 1 -- 172.21.15.32:0/1268137416 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0ba400 con 0x7f19c0002440 2024-04-17T10:04:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19ddb70080 1 -- 172.21.15.32:0/1268137416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19c0002440 msgr2=0x55b5f9110370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19ddb70080 1 --2- 172.21.15.32:0/1268137416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19c0002440 0x55b5f9110370 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f19cc009900 tx=0x7f19cc019fa0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19ddb70080 1 -- 172.21.15.32:0/1268137416 shutdown_connections 2024-04-17T10:04:21.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19ddb70080 1 --2- 172.21.15.32:0/1268137416 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b5f9113830 0x55b5f910f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19ddb70080 1 --2- 172.21.15.32:0/1268137416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5f909cf20 0x55b5f90965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19ddb70080 1 --2- 172.21.15.32:0/1268137416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19c0002440 0x55b5f9110370 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19ddb70080 1 -- 172.21.15.32:0/1268137416 >> 172.21.15.32:0/1268137416 conn(0x55b5f9082760 msgr2=0x55b5f9110990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:21.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19ddb70080 1 -- 172.21.15.32:0/1268137416 shutdown_connections 2024-04-17T10:04:21.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19ddb70080 1 -- 172.21.15.32:0/1268137416 wait complete. 2024-04-17T10:04:21.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19ddb70080 1 Processor -- start 2024-04-17T10:04:21.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19ddb70080 1 -- start start 2024-04-17T10:04:21.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19ddb70080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b5f909cf20 0x55b5f90965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19ddb70080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5f9113830 0x55b5f9055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19ddb70080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19c0002440 0x55b5f9097e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19ddb70080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b5f9055700 con 0x7f19c0002440 2024-04-17T10:04:21.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19ddb70080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b5f90582e0 con 0x55b5f9113830 2024-04-17T10:04:21.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19ddb70080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b5f90559a0 con 0x55b5f909cf20 2024-04-17T10:04:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19db800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19c0002440 0x55b5f9097e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19db800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19c0002440 0x55b5f9097e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60720/0 (socket says 172.21.15.32:60720) 2024-04-17T10:04:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19db800640 1 -- 172.21.15.32:0/1057846883 learned_addr learned my addr 172.21.15.32:0/1057846883 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19dae00640 1 --2- 172.21.15.32:0/1057846883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b5f909cf20 0x55b5f90965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.730+0000 7f19da400640 1 --2- 172.21.15.32:0/1057846883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5f9113830 0x55b5f9055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.734+0000 7f19db800640 1 -- 172.21.15.32:0/1057846883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b5f909cf20 msgr2=0x55b5f90965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.734+0000 7f19db800640 1 --2- 172.21.15.32:0/1057846883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b5f909cf20 0x55b5f90965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.734+0000 7f19db800640 1 -- 172.21.15.32:0/1057846883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5f9113830 msgr2=0x55b5f9055fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.734+0000 7f19db800640 1 --2- 172.21.15.32:0/1057846883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5f9113830 0x55b5f9055fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.734+0000 7f19db800640 1 -- 172.21.15.32:0/1057846883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5f9096f40 con 0x7f19c0002440 2024-04-17T10:04:21.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.734+0000 7f19da400640 1 --2- 172.21.15.32:0/1057846883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b5f9113830 0x55b5f9055fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:21.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.734+0000 7f19db800640 1 --2- 172.21.15.32:0/1057846883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19c0002440 0x55b5f9097e60 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x55b5f9043e10 tx=0x7f19d4012ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:21.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.734+0000 7f19d3400640 1 -- 172.21.15.32:0/1057846883 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4014680 con 0x7f19c0002440 2024-04-17T10:04:21.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.734+0000 7f19d3400640 1 -- 172.21.15.32:0/1057846883 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19d401a020 con 0x7f19c0002440 2024-04-17T10:04:21.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.734+0000 7f19ddb70080 1 -- 172.21.15.32:0/1057846883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b5f90971e0 con 0x7f19c0002440 2024-04-17T10:04:21.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.734+0000 7f19d3400640 1 -- 172.21.15.32:0/1057846883 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4014a60 con 0x7f19c0002440 2024-04-17T10:04:21.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.734+0000 7f19ddb70080 1 -- 172.21.15.32:0/1057846883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5f9097720 con 0x7f19c0002440 2024-04-17T10:04:21.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.734+0000 7f19d3400640 1 -- 172.21.15.32:0/1057846883 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f19d4014c00 con 0x7f19c0002440 2024-04-17T10:04:21.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.734+0000 7f19d3400640 1 --2- 172.21.15.32:0/1057846883 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f19bc066dc0 0x7f19bc069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.734+0000 7f19d3400640 1 -- 172.21.15.32:0/1057846883 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7f19d4088f70 con 0x7f19c0002440 2024-04-17T10:04:21.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.738+0000 7f19dae00640 1 --2- 172.21.15.32:0/1057846883 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f19bc066dc0 0x7f19bc069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.738+0000 7f19ddb70080 1 --2- 172.21.15.32:0/1057846883 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55b5f90564f0 0x55b5f9099f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.738+0000 7f19ddb70080 1 -- 172.21.15.32:0/1057846883 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:1 17.19 17:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) -- 0x55b5f9096b00 con 0x55b5f90564f0 2024-04-17T10:04:21.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.738+0000 7f19da400640 1 --2- 172.21.15.32:0/1057846883 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55b5f90564f0 0x55b5f9099f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.738+0000 7f19dae00640 1 --2- 172.21.15.32:0/1057846883 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f19bc066dc0 0x7f19bc069280 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x55b5f90598a0 tx=0x7f19cc0bc040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.738+0000 7f19da400640 1 --2- 172.21.15.32:0/1057846883 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55b5f90564f0 0x55b5f9099f40 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.738+0000 7f19da400640 1 -- 172.21.15.32:0/1057846883 <== osd.5 v2:172.21.15.144:6800/2379578648 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f19c800d040 con 0x55b5f90564f0 2024-04-17T10:04:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.738+0000 7f19ddb70080 1 -- 172.21.15.32:0/1057846883 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55b5f90564f0 msgr2=0x55b5f9099f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.738+0000 7f19ddb70080 1 --2- 172.21.15.32:0/1057846883 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55b5f90564f0 0x55b5f9099f40 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.738+0000 7f19ddb70080 1 -- 172.21.15.32:0/1057846883 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f19bc066dc0 msgr2=0x7f19bc069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.738+0000 7f19ddb70080 1 --2- 172.21.15.32:0/1057846883 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f19bc066dc0 0x7f19bc069280 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x55b5f90598a0 tx=0x7f19cc0bc040 comp rx=0 tx=0).stop 2024-04-17T10:04:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.738+0000 7f19ddb70080 1 -- 172.21.15.32:0/1057846883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19c0002440 msgr2=0x55b5f9097e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.738+0000 7f19ddb70080 1 --2- 172.21.15.32:0/1057846883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19c0002440 0x55b5f9097e60 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x55b5f9043e10 tx=0x7f19d4012ad0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.738+0000 7f19db800640 1 -- 172.21.15.32:0/1057846883 reap_dead start 2024-04-17T10:04:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.738+0000 7f19ddb70080 1 -- 172.21.15.32:0/1057846883 shutdown_connections 2024-04-17T10:04:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.738+0000 7f19ddb70080 1 -- 172.21.15.32:0/1057846883 >> 172.21.15.32:0/1057846883 conn(0x55b5f9082760 msgr2=0x55b5f9054860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.738+0000 7f19ddb70080 1 -- 172.21.15.32:0/1057846883 shutdown_connections 2024-04-17T10:04:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.738+0000 7f19ddb70080 1 -- 172.21.15.32:0/1057846883 wait complete. 2024-04-17T10:04:21.738 INFO:teuthology.orchestra.run.smithi032.stdout:mytag123 2024-04-17T10:04:21.739 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-04-17T10:04:21.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.830+0000 7f8fab048080 1 Processor -- start 2024-04-17T10:04:21.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.830+0000 7f8fab048080 1 -- start start 2024-04-17T10:04:21.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.830+0000 7f8fab048080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b0740860 0x55a1b0738230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.830+0000 7f8fab048080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1b07cf680 0x55a1b0727cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.830+0000 7f8fab048080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a1b07cfa60 0x55a1b07c7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.830+0000 7f8fab048080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a1b06fa5d0 con 0x55a1b0740860 2024-04-17T10:04:21.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.830+0000 7f8fab048080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a1b069df90 con 0x55a1b07cf680 2024-04-17T10:04:21.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.830+0000 7f8fab048080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55a1b06fa870 con 0x55a1b07cfa60 2024-04-17T10:04:21.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.830+0000 7f8fa7600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b0740860 0x55a1b0738230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.830+0000 7f8fa7600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b0740860 0x55a1b0738230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60730/0 (socket says 172.21.15.32:60730) 2024-04-17T10:04:21.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.830+0000 7f8fa7600640 1 -- 172.21.15.32:0/1088345159 learned_addr learned my addr 172.21.15.32:0/1088345159 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:21.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.830+0000 7f8fa8000640 1 --2- 172.21.15.32:0/1088345159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a1b07cfa60 0x55a1b07c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fa6c00640 1 --2- 172.21.15.32:0/1088345159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1b07cf680 0x55a1b0727cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fa7600640 1 -- 172.21.15.32:0/1088345159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a1b07cfa60 msgr2=0x55a1b07c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:21.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fa7600640 1 --2- 172.21.15.32:0/1088345159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a1b07cfa60 0x55a1b07c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fa7600640 1 -- 172.21.15.32:0/1088345159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1b07cf680 msgr2=0x55a1b0727cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:21.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fa7600640 1 --2- 172.21.15.32:0/1088345159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1b07cf680 0x55a1b0727cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fa7600640 1 -- 172.21.15.32:0/1088345159 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1b0622950 con 0x55a1b0740860 2024-04-17T10:04:21.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fa8000640 1 --2- 172.21.15.32:0/1088345159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a1b07cfa60 0x55a1b07c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:21.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fa6c00640 1 --2- 172.21.15.32:0/1088345159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1b07cf680 0x55a1b0727cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:21.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fa7600640 1 --2- 172.21.15.32:0/1088345159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b0740860 0x55a1b0738230 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009a40 tx=0x7f8f98005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417b8478a27f3e19 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:21.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fa5800640 1 -- 172.21.15.32:0/1088345159 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98008e60 con 0x55a1b0740860 2024-04-17T10:04:21.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fa5800640 1 -- 172.21.15.32:0/1088345159 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f98018070 con 0x55a1b0740860 2024-04-17T10:04:21.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fa5800640 1 -- 172.21.15.32:0/1088345159 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98013620 con 0x55a1b0740860 2024-04-17T10:04:21.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fab048080 1 -- 172.21.15.32:0/1088345159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b0740860 msgr2=0x55a1b0738230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fab048080 1 --2- 172.21.15.32:0/1088345159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b0740860 0x55a1b0738230 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009a40 tx=0x7f8f98005a10 comp rx=0 tx=0).stop 2024-04-17T10:04:21.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fab048080 1 -- 172.21.15.32:0/1088345159 shutdown_connections 2024-04-17T10:04:21.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fab048080 1 --2- 172.21.15.32:0/1088345159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a1b07cfa60 0x55a1b07c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fab048080 1 --2- 172.21.15.32:0/1088345159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1b07cf680 0x55a1b0727cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fab048080 1 --2- 172.21.15.32:0/1088345159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b0740860 0x55a1b0738230 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fab048080 1 -- 172.21.15.32:0/1088345159 >> 172.21.15.32:0/1088345159 conn(0x55a1b0723a00 msgr2=0x55a1b07ce820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:21.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fab048080 1 -- 172.21.15.32:0/1088345159 shutdown_connections 2024-04-17T10:04:21.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fab048080 1 -- 172.21.15.32:0/1088345159 wait complete. 2024-04-17T10:04:21.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fab048080 1 Processor -- start 2024-04-17T10:04:21.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fab048080 1 -- start start 2024-04-17T10:04:21.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fab048080 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b0740860 0x55a1b07c6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fab048080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1b07cf680 0x55a1b07c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fab048080 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a1b07cfa60 0x55a1b073d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fab048080 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55a1b07ce140 con 0x55a1b0740860 2024-04-17T10:04:21.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fab048080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a1b07ca980 con 0x55a1b07cf680 2024-04-17T10:04:21.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fab048080 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55a1b07ce3e0 con 0x55a1b07cfa60 2024-04-17T10:04:21.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fa7600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b0740860 0x55a1b07c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fa7600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b0740860 0x55a1b07c6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60740/0 (socket says 172.21.15.32:60740) 2024-04-17T10:04:21.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fa7600640 1 -- 172.21.15.32:0/4181419439 learned_addr learned my addr 172.21.15.32:0/4181419439 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:21.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fa6c00640 1 --2- 172.21.15.32:0/4181419439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1b07cf680 0x55a1b07c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fa8000640 1 --2- 172.21.15.32:0/4181419439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a1b07cfa60 0x55a1b073d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fa7600640 1 -- 172.21.15.32:0/4181419439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a1b07cfa60 msgr2=0x55a1b073d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fa7600640 1 --2- 172.21.15.32:0/4181419439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a1b07cfa60 0x55a1b073d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fa7600640 1 -- 172.21.15.32:0/4181419439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1b07cf680 msgr2=0x55a1b07c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fa7600640 1 --2- 172.21.15.32:0/4181419439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1b07cf680 0x55a1b07c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.834+0000 7f8fa7600640 1 -- 172.21.15.32:0/4181419439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1b07c6d80 con 0x55a1b0740860 2024-04-17T10:04:21.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.838+0000 7f8fa7600640 1 --2- 172.21.15.32:0/4181419439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b0740860 0x55a1b07c6400 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f8f98008780 tx=0x7f8f98003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:21.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.838+0000 7f8f9fe00640 1 -- 172.21.15.32:0/4181419439 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98013720 con 0x55a1b0740860 2024-04-17T10:04:21.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.838+0000 7f8f9fe00640 1 -- 172.21.15.32:0/4181419439 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f980138c0 con 0x55a1b0740860 2024-04-17T10:04:21.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.838+0000 7f8f9fe00640 1 -- 172.21.15.32:0/4181419439 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98013cb0 con 0x55a1b0740860 2024-04-17T10:04:21.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.838+0000 7f8fab048080 1 -- 172.21.15.32:0/4181419439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a1b07c7020 con 0x55a1b0740860 2024-04-17T10:04:21.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.838+0000 7f8fab048080 1 -- 172.21.15.32:0/4181419439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1b07c7560 con 0x55a1b0740860 2024-04-17T10:04:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.838+0000 7f8f9fe00640 1 -- 172.21.15.32:0/4181419439 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8f9802f440 con 0x55a1b0740860 2024-04-17T10:04:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.838+0000 7f8f9fe00640 1 --2- 172.21.15.32:0/4181419439 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8f88066d60 0x7f8f88069220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.838+0000 7f8f9fe00640 1 -- 172.21.15.32:0/4181419439 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7f8f980877d0 con 0x55a1b0740860 2024-04-17T10:04:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.838+0000 7f8fa6c00640 1 --2- 172.21.15.32:0/4181419439 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8f88066d60 0x7f8f88069220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.838+0000 7f8fab048080 1 --2- 172.21.15.32:0/4181419439 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55a1b07c44b0 0x55a1b07d0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.838+0000 7f8fab048080 1 -- 172.21.15.32:0/4181419439 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:1 17.5 17:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) -- 0x55a1b07c6940 con 0x55a1b07c44b0 2024-04-17T10:04:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.842+0000 7f8fa8000640 1 --2- 172.21.15.32:0/4181419439 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55a1b07c44b0 0x55a1b07d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.842+0000 7f8fa6c00640 1 --2- 172.21.15.32:0/4181419439 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8f88066d60 0x7f8f88069220 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8f940ac000 tx=0x7f8f940a5000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.842+0000 7f8fa8000640 1 --2- 172.21.15.32:0/4181419439 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55a1b07c44b0 0x55a1b07d0660 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.842+0000 7f8fa8000640 1 -- 172.21.15.32:0/4181419439 <== osd.6 v2:172.21.15.144:6808/3198738696 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f8fa00019b0 con 0x55a1b07c44b0 2024-04-17T10:04:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.842+0000 7f8fab048080 1 -- 172.21.15.32:0/4181419439 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55a1b07c44b0 msgr2=0x55a1b07d0660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.842+0000 7f8fab048080 1 --2- 172.21.15.32:0/4181419439 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55a1b07c44b0 0x55a1b07d0660 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.842+0000 7f8fab048080 1 -- 172.21.15.32:0/4181419439 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8f88066d60 msgr2=0x7f8f88069220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.842+0000 7f8fab048080 1 --2- 172.21.15.32:0/4181419439 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8f88066d60 0x7f8f88069220 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8f940ac000 tx=0x7f8f940a5000 comp rx=0 tx=0).stop 2024-04-17T10:04:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.842+0000 7f8fab048080 1 -- 172.21.15.32:0/4181419439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b0740860 msgr2=0x55a1b07c6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.842+0000 7f8fab048080 1 --2- 172.21.15.32:0/4181419439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a1b0740860 0x55a1b07c6400 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f8f98008780 tx=0x7f8f98003030 comp rx=0 tx=0).stop 2024-04-17T10:04:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.842+0000 7f8fa8000640 1 -- 172.21.15.32:0/4181419439 reap_dead start 2024-04-17T10:04:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.842+0000 7f8fab048080 1 -- 172.21.15.32:0/4181419439 shutdown_connections 2024-04-17T10:04:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.842+0000 7f8fab048080 1 -- 172.21.15.32:0/4181419439 >> 172.21.15.32:0/4181419439 conn(0x55a1b0723a00 msgr2=0x55a1b0742160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.842+0000 7f8fab048080 1 -- 172.21.15.32:0/4181419439 shutdown_connections 2024-04-17T10:04:21.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.842+0000 7f8fab048080 1 -- 172.21.15.32:0/4181419439 wait complete. 2024-04-17T10:04:21.841 INFO:teuthology.orchestra.run.smithi032.stdout:mytag123 2024-04-17T10:04:21.841 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-17T10:04:21.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.990+0000 7fc8a9800640 1 Processor -- start 2024-04-17T10:04:21.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.990+0000 7fc8a9800640 1 -- start start 2024-04-17T10:04:21.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.990+0000 7fc8a9800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8a406f7c0 0x7fc8a406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.990+0000 7fc8a9800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8a406f270 0x7fc8a406bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.990+0000 7fc8a9800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8a40cc940 con 0x7fc8a415b7d0 2024-04-17T10:04:21.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.990+0000 7fc8a9800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fc8a40ccc20 con 0x7fc8a406f7c0 2024-04-17T10:04:21.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.990+0000 7fc8a9800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc8a406e0b0 con 0x7fc8a406f270 2024-04-17T10:04:21.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.990+0000 7fc8a2000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8a406f7c0 0x7fc8a406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.990+0000 7fc8a2000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8a406f7c0 0x7fc8a406ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60752/0 (socket says 172.21.15.32:60752) 2024-04-17T10:04:21.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.990+0000 7fc8a2000640 1 -- 172.21.15.32:0/2032858897 learned_addr learned my addr 172.21.15.32:0/2032858897 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:21.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.990+0000 7fc8a3400640 1 --2- 172.21.15.32:0/2032858897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8a406f270 0x7fc8a406bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.990+0000 7fc8a2000640 1 -- 172.21.15.32:0/2032858897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8a406f270 msgr2=0x7fc8a406bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:21.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.990+0000 7fc8a2000640 1 --2- 172.21.15.32:0/2032858897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8a406f270 0x7fc8a406bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.990+0000 7fc8a2000640 1 -- 172.21.15.32:0/2032858897 >> v1:172.21.15.144:6789/0 conn(0x7fc8a415b7d0 legacy=0x7fc8a406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:21.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.990+0000 7fc8a2000640 1 -- 172.21.15.32:0/2032858897 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a406e390 con 0x7fc8a406f7c0 2024-04-17T10:04:21.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.994+0000 7fc8a2000640 1 --2- 172.21.15.32:0/2032858897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8a406f7c0 0x7fc8a406ba30 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fc890009870 tx=0x7fc89001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d053283ec62a9f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:21.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.994+0000 7fc8a1600640 1 -- 172.21.15.32:0/2032858897 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8900a7070 con 0x7fc8a406f7c0 2024-04-17T10:04:21.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.994+0000 7fc8a1600640 1 -- 172.21.15.32:0/2032858897 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8900b0070 con 0x7fc8a406f7c0 2024-04-17T10:04:21.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.994+0000 7fc8a1600640 1 -- 172.21.15.32:0/2032858897 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8900ab890 con 0x7fc8a406f7c0 2024-04-17T10:04:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.994+0000 7fc8a9800640 1 -- 172.21.15.32:0/2032858897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8a406f7c0 msgr2=0x7fc8a406ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.994+0000 7fc8a9800640 1 --2- 172.21.15.32:0/2032858897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8a406f7c0 0x7fc8a406ba30 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fc890009870 tx=0x7fc89001c490 comp rx=0 tx=0).stop 2024-04-17T10:04:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.994+0000 7fc8a9800640 1 -- 172.21.15.32:0/2032858897 shutdown_connections 2024-04-17T10:04:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.994+0000 7fc8a9800640 1 --2- 172.21.15.32:0/2032858897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8a406f270 0x7fc8a406bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.994+0000 7fc8a9800640 1 --2- 172.21.15.32:0/2032858897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8a406f7c0 0x7fc8a406ba30 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.994+0000 7fc8a9800640 1 -- 172.21.15.32:0/2032858897 >> 172.21.15.32:0/2032858897 conn(0x7fc8a40b2120 msgr2=0x7fc8a40b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.994+0000 7fc8a9800640 1 -- 172.21.15.32:0/2032858897 shutdown_connections 2024-04-17T10:04:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.994+0000 7fc8a9800640 1 -- 172.21.15.32:0/2032858897 wait complete. 2024-04-17T10:04:21.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.994+0000 7fc8a9800640 1 Processor -- start 2024-04-17T10:04:21.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.994+0000 7fc8a9800640 1 -- start start 2024-04-17T10:04:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.998+0000 7fc8a9800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8a406f270 0x7fc8a414caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.998+0000 7fc8a9800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8a406f7c0 0x7fc8a4151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.998+0000 7fc8a9800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8a415b7d0 0x7fc8a4156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.998+0000 7fc8a9800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a40cee70 con 0x7fc8a406f7c0 2024-04-17T10:04:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.998+0000 7fc8a9800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a40cecf0 con 0x7fc8a415b7d0 2024-04-17T10:04:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.998+0000 7fc8a9800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc8a4166290 con 0x7fc8a406f270 2024-04-17T10:04:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.998+0000 7fc8a2000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8a406f7c0 0x7fc8a4151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.998+0000 7fc8a2000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8a406f7c0 0x7fc8a4151310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60762/0 (socket says 172.21.15.32:60762) 2024-04-17T10:04:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.998+0000 7fc8a2000640 1 -- 172.21.15.32:0/2484901485 learned_addr learned my addr 172.21.15.32:0/2484901485 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.998+0000 7fc8a3400640 1 --2- 172.21.15.32:0/2484901485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8a415b7d0 0x7fc8a4156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.998+0000 7fc8a2a00640 1 --2- 172.21.15.32:0/2484901485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8a406f270 0x7fc8a414caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.998+0000 7fc8a2000640 1 -- 172.21.15.32:0/2484901485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8a406f270 msgr2=0x7fc8a414caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.998+0000 7fc8a2000640 1 --2- 172.21.15.32:0/2484901485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8a406f270 0x7fc8a414caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.998+0000 7fc8a2000640 1 -- 172.21.15.32:0/2484901485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8a415b7d0 msgr2=0x7fc8a4156860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.998+0000 7fc8a2000640 1 --2- 172.21.15.32:0/2484901485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8a415b7d0 0x7fc8a4156860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.998+0000 7fc8a2000640 1 -- 172.21.15.32:0/2484901485 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a4143370 con 0x7fc8a406f7c0 2024-04-17T10:04:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.998+0000 7fc8a2a00640 1 --2- 172.21.15.32:0/2484901485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8a406f270 0x7fc8a414caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.998+0000 7fc8a2000640 1 --2- 172.21.15.32:0/2484901485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8a406f7c0 0x7fc8a4151310 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fc8900ad040 tx=0x7fc89001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.998+0000 7fc89aa00640 1 -- 172.21.15.32:0/2484901485 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8900b4800 con 0x7fc8a406f7c0 2024-04-17T10:04:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.998+0000 7fc89aa00640 1 -- 172.21.15.32:0/2484901485 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8900b49a0 con 0x7fc8a406f7c0 2024-04-17T10:04:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.998+0000 7fc89aa00640 1 -- 172.21.15.32:0/2484901485 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8900b4d90 con 0x7fc8a406f7c0 2024-04-17T10:04:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.998+0000 7fc8a9800640 1 -- 172.21.15.32:0/2484901485 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a4143650 con 0x7fc8a406f7c0 2024-04-17T10:04:21.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:21.998+0000 7fc8a9800640 1 -- 172.21.15.32:0/2484901485 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a4143c10 con 0x7fc8a406f7c0 2024-04-17T10:04:21.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.002+0000 7fc89aa00640 1 -- 172.21.15.32:0/2484901485 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc8900c7390 con 0x7fc8a406f7c0 2024-04-17T10:04:21.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.002+0000 7fc89aa00640 1 --2- 172.21.15.32:0/2484901485 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc884068380 0x7fc88406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:21.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.002+0000 7fc89aa00640 1 -- 172.21.15.32:0/2484901485 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7fc890121430 con 0x7fc8a406f7c0 2024-04-17T10:04:21.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.002+0000 7fc8a2a00640 1 --2- 172.21.15.32:0/2484901485 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc884068380 0x7fc88406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:21.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.002+0000 7fc8acbcf000 1 Processor -- start 2024-04-17T10:04:21.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.002+0000 7fc8acbcf000 1 -- start start 2024-04-17T10:04:21.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.002+0000 7fc8a2a00640 1 --2- 172.21.15.32:0/2484901485 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc884068380 0x7fc88406a840 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc89400a850 tx=0x7fc894008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.002+0000 7fc8acbcf000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5613f323f260 0x5613f323f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.002+0000 7fc8acbcf000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f323fb80 0x5613f31f2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.002+0000 7fc8acbcf000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f31f29a0 0x5613f31f2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.002+0000 7fc8acbcf000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5613f30444f0 con 0x5613f31f29a0 2024-04-17T10:04:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.002+0000 7fc8acbcf000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5613f2ffd6c0 con 0x5613f323fb80 2024-04-17T10:04:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.002+0000 7fc8acbcf000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5613f30404b0 con 0x5613f323f260 2024-04-17T10:04:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.002+0000 7fc8a3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f31f29a0 0x5613f31f2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.002+0000 7fc8a3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f31f29a0 0x5613f31f2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60770/0 (socket says 172.21.15.32:60770) 2024-04-17T10:04:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.002+0000 7fc8a3400640 1 -- 172.21.15.32:0/1327258252 learned_addr learned my addr 172.21.15.32:0/1327258252 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.002+0000 7fc8a2a00640 1 --2- 172.21.15.32:0/1327258252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5613f323f260 0x5613f323f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.002+0000 7fc8a2000640 1 --2- 172.21.15.32:0/1327258252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f323fb80 0x5613f31f2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.002+0000 7fc8a3400640 1 -- 172.21.15.32:0/1327258252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5613f323f260 msgr2=0x5613f323f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.002+0000 7fc8a3400640 1 --2- 172.21.15.32:0/1327258252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5613f323f260 0x5613f323f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.002+0000 7fc8a3400640 1 -- 172.21.15.32:0/1327258252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f323fb80 msgr2=0x5613f31f2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.002+0000 7fc8a3400640 1 --2- 172.21.15.32:0/1327258252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f323fb80 0x5613f31f2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.002+0000 7fc8a3400640 1 -- 172.21.15.32:0/1327258252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613f30898e0 con 0x5613f31f29a0 2024-04-17T10:04:22.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8a3400640 1 --2- 172.21.15.32:0/1327258252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f31f29a0 0x5613f31f2d80 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fc89c007400 tx=0x7fc89c0078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1790aa89d75079f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc88f400640 1 -- 172.21.15.32:0/1327258252 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c0147e0 con 0x5613f31f29a0 2024-04-17T10:04:22.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc88f400640 1 -- 172.21.15.32:0/1327258252 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc89c01a070 con 0x5613f31f29a0 2024-04-17T10:04:22.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc88f400640 1 -- 172.21.15.32:0/1327258252 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c014b80 con 0x5613f31f29a0 2024-04-17T10:04:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8acbcf000 1 -- 172.21.15.32:0/1327258252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f31f29a0 msgr2=0x5613f31f2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8acbcf000 1 --2- 172.21.15.32:0/1327258252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f31f29a0 0x5613f31f2d80 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fc89c007400 tx=0x7fc89c0078c0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8acbcf000 1 -- 172.21.15.32:0/1327258252 shutdown_connections 2024-04-17T10:04:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8acbcf000 1 --2- 172.21.15.32:0/1327258252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f31f29a0 0x5613f31f2d80 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8acbcf000 1 --2- 172.21.15.32:0/1327258252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f323fb80 0x5613f31f2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8acbcf000 1 --2- 172.21.15.32:0/1327258252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5613f323f260 0x5613f323f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8acbcf000 1 -- 172.21.15.32:0/1327258252 >> 172.21.15.32:0/1327258252 conn(0x5613f3220000 msgr2=0x5613f32223f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:22.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8acbcf000 1 -- 172.21.15.32:0/1327258252 shutdown_connections 2024-04-17T10:04:22.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8acbcf000 1 -- 172.21.15.32:0/1327258252 wait complete. 2024-04-17T10:04:22.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8acbcf000 1 -- start start 2024-04-17T10:04:22.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8acbcf000 1 Processor -- start 2024-04-17T10:04:22.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8acbcf000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f31f29a0 0x5613f31f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8acbcf000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f323f260 0x5613f31f8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8acbcf000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5613f323fb80 0x5613f31f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8acbcf000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5613f30c57b0 con 0x5613f31f29a0 2024-04-17T10:04:22.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8acbcf000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5613f30ffd00 con 0x5613f323f260 2024-04-17T10:04:22.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8acbcf000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5613f30092c0 con 0x5613f323fb80 2024-04-17T10:04:22.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8acbcf000 10 client.0 _ll_register_callbacks cb 0x5613f3019030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:04:22.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8a2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f31f29a0 0x5613f31f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8a2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f31f29a0 0x5613f31f7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60772/0 (socket says 172.21.15.32:60772) 2024-04-17T10:04:22.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8a2a00640 1 -- 172.21.15.32:0/1202595349 learned_addr learned my addr 172.21.15.32:0/1202595349 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:22.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8a3400640 1 --2- 172.21.15.32:0/1202595349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5613f323fb80 0x5613f31f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8a2000640 1 --2- 172.21.15.32:0/1202595349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f323f260 0x5613f31f8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8a2a00640 1 -- 172.21.15.32:0/1202595349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5613f323fb80 msgr2=0x5613f31f8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8a2a00640 1 --2- 172.21.15.32:0/1202595349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5613f323fb80 0x5613f31f8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8a2a00640 1 -- 172.21.15.32:0/1202595349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f323f260 msgr2=0x5613f31f8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8a2a00640 1 --2- 172.21.15.32:0/1202595349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f323f260 0x5613f31f8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.006+0000 7fc8a2a00640 1 -- 172.21.15.32:0/1202595349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613f301c490 con 0x5613f31f29a0 2024-04-17T10:04:22.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8a2a00640 1 --2- 172.21.15.32:0/1202595349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f31f29a0 0x5613f31f7fe0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fc89400cfb0 tx=0x7fc894004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e85fabcc22ca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc883e00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:04:22.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc883e00640 1 -- 172.21.15.32:0/1202595349 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894023020 con 0x5613f31f29a0 2024-04-17T10:04:22.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc883e00640 1 -- 172.21.15.32:0/1202595349 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7fc89401f600 con 0x5613f31f29a0 2024-04-17T10:04:22.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8acbcf000 1 -- 172.21.15.32:0/1202595349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc89c01a070 con 0x5613f31f29a0 2024-04-17T10:04:22.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc883e00640 1 -- 172.21.15.32:0/1202595349 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7fc89401f800 con 0x5613f31f29a0 2024-04-17T10:04:22.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8acbcf000 10 client.9816 fetch_fsmap learned FSMap version 104 2024-04-17T10:04:22.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8acbcf000 1 -- 172.21.15.32:0/1202595349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x5613f300feb0 con 0x5613f31f29a0 2024-04-17T10:04:22.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc883e00640 1 -- 172.21.15.32:0/1202595349 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc89c01a070 con 0x5613f31f29a0 2024-04-17T10:04:22.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc883e00640 1 -- 172.21.15.32:0/1202595349 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 104) ==== 3495+0+0 (secure 0 0 0) 0x7fc894030020 con 0x5613f31f29a0 2024-04-17T10:04:22.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8acbcf000 10 client.9816 fetch_fsmap finished waiting for FSMap version 104 2024-04-17T10:04:22.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8acbcf000 10 client.9816 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-17T10:04:22.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8acbcf000 20 client.9816 populate_metadata read hostname 'smithi032' 2024-04-17T10:04:22.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8acbcf000 1 --2- 172.21.15.32:0/1202595349 >> [v2:172.21.15.32:6836/578643390,v1:172.21.15.32:6837/578643390] conn(0x5613f324e500 0x5613f32508f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8acbcf000 4 client.9816 mds_command: new command op to 9615 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:04:22.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8acbcf000 1 -- 172.21.15.32:0/1202595349 --> [v2:172.21.15.32:6836/578643390,v1:172.21.15.32:6837/578643390] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613f2fdddd0 con 0x5613f324e500 2024-04-17T10:04:22.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8a2000640 1 --2- 172.21.15.32:0/1202595349 >> [v2:172.21.15.32:6836/578643390,v1:172.21.15.32:6837/578643390] conn(0x5613f324e500 0x5613f32508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc883e00640 10 client.9816 ms_handle_connect on v2:172.21.15.32:6836/578643390 2024-04-17T10:04:22.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8a2000640 1 --2- 172.21.15.32:0/1202595349 >> [v2:172.21.15.32:6836/578643390,v1:172.21.15.32:6837/578643390] conn(0x5613f324e500 0x5613f32508f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1eef0a274381e4d server_cookie=eac297aeefe22887 in_seq=0 out_seq=0 2024-04-17T10:04:22.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc883e00640 1 -- 172.21.15.32:0/1202595349 <== mds.0 v2:172.21.15.32:6836/578643390 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5613f2fdddd0 con 0x5613f324e500 2024-04-17T10:04:22.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc883e00640 10 client.9816 handle_command_reply: tid=0 2024-04-17T10:04:22.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8acbcf000 1 client.9816 shutdown 2024-04-17T10:04:22.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8acbcf000 1 -- 172.21.15.32:0/1202595349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f31f29a0 msgr2=0x5613f31f7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8acbcf000 1 --2- 172.21.15.32:0/1202595349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f31f29a0 0x5613f31f7fe0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fc89400cfb0 tx=0x7fc894004700 comp rx=0 tx=0).stop 2024-04-17T10:04:22.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8acbcf000 1 -- 172.21.15.32:0/1202595349 shutdown_connections 2024-04-17T10:04:22.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8acbcf000 1 --2- 172.21.15.32:0/1202595349 >> [v2:172.21.15.32:6836/578643390,v1:172.21.15.32:6837/578643390] conn(0x5613f324e500 0x5613f32508f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8acbcf000 1 --2- 172.21.15.32:0/1202595349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5613f323fb80 0x5613f31f8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8acbcf000 1 --2- 172.21.15.32:0/1202595349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f323f260 0x5613f31f8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc883e00640 0 client.9816 ms_handle_reset on v2:172.21.15.32:6836/578643390 2024-04-17T10:04:22.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8acbcf000 1 --2- 172.21.15.32:0/1202595349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f31f29a0 0x5613f31f7fe0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8acbcf000 1 -- 172.21.15.32:0/1202595349 >> 172.21.15.32:0/1202595349 conn(0x5613f3220000 msgr2=0x5613f32223f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:22.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8acbcf000 1 -- 172.21.15.32:0/1202595349 shutdown_connections 2024-04-17T10:04:22.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8acbcf000 1 -- 172.21.15.32:0/1202595349 wait complete. 2024-04-17T10:04:22.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.010+0000 7fc8acbcf000 20 client.9816 trim_cache size 0 max 16384 2024-04-17T10:04:22.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc8acbcf000 1 Processor -- start 2024-04-17T10:04:22.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc8acbcf000 1 -- start start 2024-04-17T10:04:22.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc8acbcf000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f31f29a0 0x5613f323e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc8acbcf000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f323f260 0x5613f323e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc8acbcf000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5613f323fb80 0x5613f32508f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc8acbcf000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a41685a0 con 0x5613f323f260 2024-04-17T10:04:22.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc8acbcf000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a414c010 con 0x5613f31f29a0 2024-04-17T10:04:22.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc8acbcf000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc8a4167f50 con 0x5613f323fb80 2024-04-17T10:04:22.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc8a2000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f323f260 0x5613f323e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc8a2000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f323f260 0x5613f323e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60782/0 (socket says 172.21.15.32:60782) 2024-04-17T10:04:22.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc8a2000640 1 -- 172.21.15.32:0/3950800586 learned_addr learned my addr 172.21.15.32:0/3950800586 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:22.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc8a3400640 1 --2- 172.21.15.32:0/3950800586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5613f323fb80 0x5613f32508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc8a2a00640 1 --2- 172.21.15.32:0/3950800586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f31f29a0 0x5613f323e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc8a2000640 1 -- 172.21.15.32:0/3950800586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5613f323fb80 msgr2=0x5613f32508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc8a2000640 1 --2- 172.21.15.32:0/3950800586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5613f323fb80 0x5613f32508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc8a2000640 1 -- 172.21.15.32:0/3950800586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f31f29a0 msgr2=0x5613f323e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc8a2000640 1 --2- 172.21.15.32:0/3950800586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f31f29a0 0x5613f323e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc8a2000640 1 -- 172.21.15.32:0/3950800586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613f31fb050 con 0x5613f323f260 2024-04-17T10:04:22.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc8a3400640 1 --2- 172.21.15.32:0/3950800586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5613f323fb80 0x5613f32508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:22.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc8a2a00640 1 --2- 172.21.15.32:0/3950800586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f31f29a0 0x5613f323e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:22.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc8a2000640 1 --2- 172.21.15.32:0/3950800586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f323f260 0x5613f323e860 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fc8900f1270 tx=0x7fc890000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7f28802c7eec63 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc88ea00640 1 -- 172.21.15.32:0/3950800586 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8900f9ec0 con 0x5613f323f260 2024-04-17T10:04:22.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc88ea00640 1 -- 172.21.15.32:0/3950800586 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8900b0040 con 0x5613f323f260 2024-04-17T10:04:22.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.026+0000 7fc88ea00640 1 -- 172.21.15.32:0/3950800586 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8900ef5e0 con 0x5613f323f260 2024-04-17T10:04:22.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8acbcf000 1 -- 172.21.15.32:0/3950800586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f323f260 msgr2=0x5613f323e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8acbcf000 1 --2- 172.21.15.32:0/3950800586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f323f260 0x5613f323e860 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fc8900f1270 tx=0x7fc890000bd0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8acbcf000 1 -- 172.21.15.32:0/3950800586 shutdown_connections 2024-04-17T10:04:22.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8acbcf000 1 --2- 172.21.15.32:0/3950800586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5613f323fb80 0x5613f32508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8acbcf000 1 --2- 172.21.15.32:0/3950800586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f323f260 0x5613f323e860 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8acbcf000 1 --2- 172.21.15.32:0/3950800586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f31f29a0 0x5613f323e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8acbcf000 1 -- 172.21.15.32:0/3950800586 >> 172.21.15.32:0/3950800586 conn(0x5613f3220000 msgr2=0x5613f31f1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8acbcf000 1 -- 172.21.15.32:0/3950800586 shutdown_connections 2024-04-17T10:04:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8acbcf000 1 -- 172.21.15.32:0/3950800586 wait complete. 2024-04-17T10:04:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8acbcf000 1 -- start start 2024-04-17T10:04:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8acbcf000 1 Processor -- start 2024-04-17T10:04:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8acbcf000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f31f29a0 0x5613f3253970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8acbcf000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5613f323f260 0x5613f326a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8acbcf000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f323fb80 0x5613f326a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8acbcf000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5613f2fe8050 con 0x5613f323fb80 2024-04-17T10:04:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8acbcf000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5613f3006570 con 0x5613f31f29a0 2024-04-17T10:04:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8acbcf000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5613f3043800 con 0x5613f323f260 2024-04-17T10:04:22.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8acbcf000 10 client.0 _ll_register_callbacks cb 0x7fc89400c7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:04:22.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8a3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f323fb80 0x5613f326a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8a3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f323fb80 0x5613f326a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60784/0 (socket says 172.21.15.32:60784) 2024-04-17T10:04:22.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8a3400640 1 -- 172.21.15.32:0/1482697427 learned_addr learned my addr 172.21.15.32:0/1482697427 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:22.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8a2a00640 1 --2- 172.21.15.32:0/1482697427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f31f29a0 0x5613f3253970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8a2000640 1 --2- 172.21.15.32:0/1482697427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5613f323f260 0x5613f326a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8a3400640 1 -- 172.21.15.32:0/1482697427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5613f323f260 msgr2=0x5613f326a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8a3400640 1 --2- 172.21.15.32:0/1482697427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5613f323f260 0x5613f326a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8a3400640 1 -- 172.21.15.32:0/1482697427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f31f29a0 msgr2=0x5613f3253970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8a3400640 1 --2- 172.21.15.32:0/1482697427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f31f29a0 0x5613f3253970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8a3400640 1 -- 172.21.15.32:0/1482697427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613f2ffea20 con 0x5613f323fb80 2024-04-17T10:04:22.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8a3400640 1 --2- 172.21.15.32:0/1482697427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f323fb80 0x5613f326a9c0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fc89c007900 tx=0x7fc89c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331b11541fb1b600 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc883e00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:04:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc883e00640 1 -- 172.21.15.32:0/1482697427 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c014c00 con 0x5613f323fb80 2024-04-17T10:04:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc883e00640 1 -- 172.21.15.32:0/1482697427 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7fc89c014da0 con 0x5613f323fb80 2024-04-17T10:04:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc8acbcf000 1 -- 172.21.15.32:0/1482697427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8900b0040 con 0x5613f323fb80 2024-04-17T10:04:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.030+0000 7fc883e00640 1 -- 172.21.15.32:0/1482697427 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7fc89c033050 con 0x5613f323fb80 2024-04-17T10:04:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.034+0000 7fc8acbcf000 10 client.9822 fetch_fsmap learned FSMap version 104 2024-04-17T10:04:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.034+0000 7fc8acbcf000 1 -- 172.21.15.32:0/1482697427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x5613f30c5c40 con 0x5613f323fb80 2024-04-17T10:04:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.034+0000 7fc883e00640 1 -- 172.21.15.32:0/1482697427 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8900b0040 con 0x5613f323fb80 2024-04-17T10:04:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.034+0000 7fc883e00640 1 -- 172.21.15.32:0/1482697427 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 104) ==== 3495+0+0 (secure 0 0 0) 0x7fc89c036020 con 0x5613f323fb80 2024-04-17T10:04:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.034+0000 7fc8acbcf000 10 client.9822 fetch_fsmap finished waiting for FSMap version 104 2024-04-17T10:04:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.034+0000 7fc8acbcf000 10 client.9822 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-17T10:04:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.034+0000 7fc8acbcf000 20 client.9822 populate_metadata read hostname 'smithi032' 2024-04-17T10:04:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.034+0000 7fc8acbcf000 1 --2- 172.21.15.32:0/1482697427 >> [v2:172.21.15.32:6836/578643390,v1:172.21.15.32:6837/578643390] conn(0x5613f32ae300 0x5613f32ae710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.034+0000 7fc8acbcf000 4 client.9822 mds_command: new command op to 9615 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-17T10:04:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.034+0000 7fc8acbcf000 1 -- 172.21.15.32:0/1482697427 --> [v2:172.21.15.32:6836/578643390,v1:172.21.15.32:6837/578643390] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5613f3051f10 con 0x5613f32ae300 2024-04-17T10:04:22.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.034+0000 7fc8a2a00640 1 --2- 172.21.15.32:0/1482697427 >> [v2:172.21.15.32:6836/578643390,v1:172.21.15.32:6837/578643390] conn(0x5613f32ae300 0x5613f32ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.034+0000 7fc8a2a00640 1 --2- 172.21.15.32:0/1482697427 >> [v2:172.21.15.32:6836/578643390,v1:172.21.15.32:6837/578643390] conn(0x5613f32ae300 0x5613f32ae710 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce3f13ac65505969 server_cookie=504a2eff2b7f6baa in_seq=0 out_seq=0 2024-04-17T10:04:22.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.034+0000 7fc883e00640 10 client.9822 ms_handle_connect on v2:172.21.15.32:6836/578643390 2024-04-17T10:04:22.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc883e00640 1 -- 172.21.15.32:0/1482697427 <== mds.0 v2:172.21.15.32:6836/578643390 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5613f3051f10 con 0x5613f32ae300 2024-04-17T10:04:22.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc883e00640 10 client.9822 handle_command_reply: tid=0 2024-04-17T10:04:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8acbcf000 1 client.9822 shutdown 2024-04-17T10:04:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8acbcf000 1 -- 172.21.15.32:0/1482697427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f323fb80 msgr2=0x5613f326a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8acbcf000 1 --2- 172.21.15.32:0/1482697427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f323fb80 0x5613f326a9c0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fc89c007900 tx=0x7fc89c012000 comp rx=0 tx=0).stop 2024-04-17T10:04:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8acbcf000 1 -- 172.21.15.32:0/1482697427 shutdown_connections 2024-04-17T10:04:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8acbcf000 1 --2- 172.21.15.32:0/1482697427 >> [v2:172.21.15.32:6836/578643390,v1:172.21.15.32:6837/578643390] conn(0x5613f32ae300 0x5613f32ae710 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8acbcf000 1 --2- 172.21.15.32:0/1482697427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5613f323f260 0x5613f326a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8acbcf000 1 --2- 172.21.15.32:0/1482697427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5613f323fb80 0x5613f326a9c0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8acbcf000 1 --2- 172.21.15.32:0/1482697427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f31f29a0 0x5613f3253970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc883e00640 0 client.9822 ms_handle_reset on v2:172.21.15.32:6836/578643390 2024-04-17T10:04:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8acbcf000 1 -- 172.21.15.32:0/1482697427 >> 172.21.15.32:0/1482697427 conn(0x5613f3220000 msgr2=0x5613f31f1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8acbcf000 1 -- 172.21.15.32:0/1482697427 shutdown_connections 2024-04-17T10:04:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8acbcf000 1 -- 172.21.15.32:0/1482697427 wait complete. 2024-04-17T10:04:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8acbcf000 20 client.9822 trim_cache size 0 max 16384 2024-04-17T10:04:22.042 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:04:22.042 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "", 2024-04-17T10:04:22.042 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0 2024-04-17T10:04:22.042 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:04:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8a9800640 1 -- 172.21.15.32:0/2484901485 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc884068380 msgr2=0x7fc88406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8a9800640 1 --2- 172.21.15.32:0/2484901485 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc884068380 0x7fc88406a840 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc89400a850 tx=0x7fc894008040 comp rx=0 tx=0).stop 2024-04-17T10:04:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8a9800640 1 -- 172.21.15.32:0/2484901485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8a406f7c0 msgr2=0x7fc8a4151310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8a9800640 1 --2- 172.21.15.32:0/2484901485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8a406f7c0 0x7fc8a4151310 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fc8900ad040 tx=0x7fc89001acd0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8a9800640 1 -- 172.21.15.32:0/2484901485 shutdown_connections 2024-04-17T10:04:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8a9800640 1 --2- 172.21.15.32:0/2484901485 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fc884068380 0x7fc88406a840 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8a9800640 1 --2- 172.21.15.32:0/2484901485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8a415b7d0 0x7fc8a4156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8a9800640 1 --2- 172.21.15.32:0/2484901485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8a406f7c0 0x7fc8a4151310 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8a9800640 1 --2- 172.21.15.32:0/2484901485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8a406f270 0x7fc8a414caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8a9800640 1 -- 172.21.15.32:0/2484901485 >> 172.21.15.32:0/2484901485 conn(0x7fc8a40b2120 msgr2=0x7fc8a4072830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:22.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8a9800640 1 -- 172.21.15.32:0/2484901485 shutdown_connections 2024-04-17T10:04:22.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.046+0000 7fc8a9800640 1 -- 172.21.15.32:0/2484901485 wait complete. 2024-04-17T10:04:22.060 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T10:04:22.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.206+0000 7fe4f6800640 1 Processor -- start 2024-04-17T10:04:22.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.206+0000 7fe4f6800640 1 -- start start 2024-04-17T10:04:22.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.210+0000 7fe4f6800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4f015eba0 0x7fe4f00700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.210+0000 7fe4f6800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4f0067d60 con 0x7fe4f00ce950 2024-04-17T10:04:22.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.210+0000 7fe4f6800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4f006f320 con 0x7fe4f00cedd0 2024-04-17T10:04:22.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.210+0000 7fe4f6800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe4f006f0d0 con 0x7fe4f015eba0 2024-04-17T10:04:22.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.210+0000 7fe4e6a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe4f00ce950 0x7fe4f00add30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50360/0 (socket says 172.21.15.32:50360) 2024-04-17T10:04:22.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.210+0000 7fe4e6a00640 1 -- 172.21.15.32:0/2730554418 learned_addr learned my addr 172.21.15.32:0/2730554418 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:22.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.210+0000 7fe4ef400640 1 --2- 172.21.15.32:0/2730554418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4f015eba0 0x7fe4f00700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.210+0000 7fe4eea00640 1 -- 172.21.15.32:0/2730554418 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3253722003 0 0) 0x7fe4f0067d60 con 0x7fe4f00ce950 2024-04-17T10:04:22.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.210+0000 7fe4eea00640 1 -- 172.21.15.32:0/2730554418 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4d4003530 con 0x7fe4f00ce950 2024-04-17T10:04:22.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.210+0000 7fe4ef400640 1 -- 172.21.15.32:0/2730554418 >> v1:172.21.15.144:6789/0 conn(0x7fe4f00cedd0 legacy=0x7fe4f00ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.210+0000 7fe4eea00640 1 -- 172.21.15.32:0/2730554418 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2234669227 0 0) 0x7fe4f006f320 con 0x7fe4f00cedd0 2024-04-17T10:04:22.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.210+0000 7fe4ef400640 1 -- 172.21.15.32:0/2730554418 >> v1:172.21.15.32:6789/0 conn(0x7fe4f00ce950 legacy=0x7fe4f00add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.210+0000 7fe4ef400640 1 -- 172.21.15.32:0/2730554418 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f006f600 con 0x7fe4f015eba0 2024-04-17T10:04:22.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.214+0000 7fe4ef400640 1 --2- 172.21.15.32:0/2730554418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4f015eba0 0x7fe4f00700e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00097f0 tx=0x7fe4e001c260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1acc5da4d2feddd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.214+0000 7fe4eea00640 1 -- 172.21.15.32:0/2730554418 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e00b0070 con 0x7fe4f015eba0 2024-04-17T10:04:22.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.214+0000 7fe4eea00640 1 -- 172.21.15.32:0/2730554418 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4e00ab400 con 0x7fe4f015eba0 2024-04-17T10:04:22.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.214+0000 7fe4eea00640 1 -- 172.21.15.32:0/2730554418 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8001600 con 0x7fe4f015eba0 2024-04-17T10:04:22.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.214+0000 7fe4f6800640 1 -- 172.21.15.32:0/2730554418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4f015eba0 msgr2=0x7fe4f00700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.214+0000 7fe4f6800640 1 --2- 172.21.15.32:0/2730554418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4f015eba0 0x7fe4f00700e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00097f0 tx=0x7fe4e001c260 comp rx=0 tx=0).stop 2024-04-17T10:04:22.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.214+0000 7fe4f6800640 1 -- 172.21.15.32:0/2730554418 shutdown_connections 2024-04-17T10:04:22.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.214+0000 7fe4f6800640 1 --2- 172.21.15.32:0/2730554418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4f015eba0 0x7fe4f00700e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.214+0000 7fe4f6800640 1 -- 172.21.15.32:0/2730554418 >> 172.21.15.32:0/2730554418 conn(0x7fe4f00cd800 msgr2=0x7fe4f00cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:22.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.214+0000 7fe4f6800640 1 -- 172.21.15.32:0/2730554418 shutdown_connections 2024-04-17T10:04:22.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.214+0000 7fe4f6800640 1 -- 172.21.15.32:0/2730554418 wait complete. 2024-04-17T10:04:22.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.214+0000 7fe4f6800640 1 Processor -- start 2024-04-17T10:04:22.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.214+0000 7fe4f6800640 1 -- start start 2024-04-17T10:04:22.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.218+0000 7fe4f6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4f00ce950 0x7fe4f014f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.218+0000 7fe4f6800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4f00cedd0 0x7fe4f00c7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.218+0000 7fe4f6800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4f015eba0 0x7fe4f014b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.218+0000 7fe4f6800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f016c760 con 0x7fe4f00ce950 2024-04-17T10:04:22.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.218+0000 7fe4f6800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f016c5e0 con 0x7fe4f015eba0 2024-04-17T10:04:22.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.218+0000 7fe4f6800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe4f016c8e0 con 0x7fe4f00cedd0 2024-04-17T10:04:22.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.218+0000 7fe4ef400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4f00ce950 0x7fe4f014f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.218+0000 7fe4ef400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4f00ce950 0x7fe4f014f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60788/0 (socket says 172.21.15.32:60788) 2024-04-17T10:04:22.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.218+0000 7fe4ef400640 1 -- 172.21.15.32:0/1110561446 learned_addr learned my addr 172.21.15.32:0/1110561446 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:22.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.218+0000 7fe4ef400640 1 -- 172.21.15.32:0/1110561446 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4f00cedd0 msgr2=0x7fe4f00c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.218+0000 7fe4ef400640 1 --2- 172.21.15.32:0/1110561446 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4f00cedd0 0x7fe4f00c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.218+0000 7fe4ef400640 1 -- 172.21.15.32:0/1110561446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4f015eba0 msgr2=0x7fe4f014b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.218+0000 7fe4ef400640 1 --2- 172.21.15.32:0/1110561446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4f015eba0 0x7fe4f014b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.218+0000 7fe4ef400640 1 -- 172.21.15.32:0/1110561446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f014bc40 con 0x7fe4f00ce950 2024-04-17T10:04:22.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.218+0000 7fe4ef400640 1 --2- 172.21.15.32:0/1110561446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4f00ce950 0x7fe4f014f9f0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00ae040 tx=0x7fe4e001aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.218+0000 7fe4e7e00640 1 -- 172.21.15.32:0/1110561446 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e00bb020 con 0x7fe4f00ce950 2024-04-17T10:04:22.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.218+0000 7fe4e7e00640 1 -- 172.21.15.32:0/1110561446 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4e00b1040 con 0x7fe4f00ce950 2024-04-17T10:04:22.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.218+0000 7fe4e7e00640 1 -- 172.21.15.32:0/1110561446 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e00b0050 con 0x7fe4f00ce950 2024-04-17T10:04:22.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.218+0000 7fe4f6800640 1 -- 172.21.15.32:0/1110561446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f0152c90 con 0x7fe4f00ce950 2024-04-17T10:04:22.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.218+0000 7fe4f6800640 1 -- 172.21.15.32:0/1110561446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f0153250 con 0x7fe4f00ce950 2024-04-17T10:04:22.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.218+0000 7fe4f6800640 1 -- 172.21.15.32:0/1110561446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4f0068a90 con 0x7fe4f00ce950 2024-04-17T10:04:22.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.218+0000 7fe4e7e00640 1 -- 172.21.15.32:0/1110561446 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe4e00be070 con 0x7fe4f00ce950 2024-04-17T10:04:22.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.222+0000 7fe4e7e00640 1 --2- 172.21.15.32:0/1110561446 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe4cc068380 0x7fe4cc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.222+0000 7fe4e7e00640 1 -- 172.21.15.32:0/1110561446 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7fe4e0120140 con 0x7fe4f00ce950 2024-04-17T10:04:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.222+0000 7fe4e6a00640 1 --2- 172.21.15.32:0/1110561446 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe4cc068380 0x7fe4cc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.222+0000 7fe4e6a00640 1 --2- 172.21.15.32:0/1110561446 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe4cc068380 0x7fe4cc06a840 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8001770 tx=0x7fe4d8001480 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.226+0000 7fe4e7e00640 1 -- 172.21.15.32:0/1110561446 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fe4e0015fd0 con 0x7fe4f00ce950 2024-04-17T10:04:22.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.426+0000 7fe4f6800640 1 -- 172.21.15.32:0/1110561446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe4f0157bf0 con 0x7fe4f00ce950 2024-04-17T10:04:22.423 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-17T10:04:22.426+0000 7f93c9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T10:04:22.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.430+0000 7fe4e7e00640 1 -- 172.21.15.32:0/1110561446 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v105) ==== 161+0+0 (secure 0 0 0) 0x7fe4e00f05d0 con 0x7fe4f00ce950 2024-04-17T10:04:22.425 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T10:04:22.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.434+0000 7fe4f6800640 1 -- 172.21.15.32:0/1110561446 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe4cc068380 msgr2=0x7fe4cc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.434+0000 7fe4f6800640 1 --2- 172.21.15.32:0/1110561446 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe4cc068380 0x7fe4cc06a840 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8001770 tx=0x7fe4d8001480 comp rx=0 tx=0).stop 2024-04-17T10:04:22.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.434+0000 7fe4f6800640 1 -- 172.21.15.32:0/1110561446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4f00ce950 msgr2=0x7fe4f014f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.434+0000 7fe4f6800640 1 --2- 172.21.15.32:0/1110561446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4f00ce950 0x7fe4f014f9f0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00ae040 tx=0x7fe4e001aaa0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.434+0000 7fe4f6800640 1 -- 172.21.15.32:0/1110561446 shutdown_connections 2024-04-17T10:04:22.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.434+0000 7fe4f6800640 1 --2- 172.21.15.32:0/1110561446 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe4cc068380 0x7fe4cc06a840 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.434+0000 7fe4f6800640 1 --2- 172.21.15.32:0/1110561446 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4f00cedd0 0x7fe4f00c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.434+0000 7fe4f6800640 1 --2- 172.21.15.32:0/1110561446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4f015eba0 0x7fe4f014b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.434+0000 7fe4f6800640 1 --2- 172.21.15.32:0/1110561446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4f00ce950 0x7fe4f014f9f0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.434+0000 7fe4f6800640 1 -- 172.21.15.32:0/1110561446 >> 172.21.15.32:0/1110561446 conn(0x7fe4f00cd800 msgr2=0x7fe4f0072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:22.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.434+0000 7fe4f6800640 1 -- 172.21.15.32:0/1110561446 shutdown_connections 2024-04-17T10:04:22.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.434+0000 7fe4f6800640 1 -- 172.21.15.32:0/1110561446 wait complete. 2024-04-17T10:04:22.439 INFO:tasks.ceph.mds.a.smithi032.stderr: -2348> 2024-04-17T10:04:09.091+0000 7f991b000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:04:22.455 DEBUG:teuthology.orchestra.run.smithi032:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-04-17T10:04:22.492 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-17T10:04:22.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.506+0000 7f0ce684a400 1 -- start start 2024-04-17T10:04:22.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.506+0000 7f0ce684a400 1 Processor -- start 2024-04-17T10:04:22.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.506+0000 7f0ce684a400 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f811742000 0x55f81172a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.506+0000 7f0ce684a400 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f810961c00 0x55f81172ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.506+0000 7f0ce684a400 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f811743000 0x55f81172a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.506+0000 7f0ce684a400 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f81187c000 con 0x55f810961c00 2024-04-17T10:04:22.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.506+0000 7f0ce684a400 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f81187c180 con 0x55f811742000 2024-04-17T10:04:22.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.506+0000 7f0ce684a400 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f81187c300 con 0x55f811743000 2024-04-17T10:04:22.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce3200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f810961c00 0x55f81172ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce3200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f810961c00 0x55f81172ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60816/0 (socket says 172.21.15.32:60816) 2024-04-17T10:04:22.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce3200640 1 -- 172.21.15.32:0/4196183021 learned_addr learned my addr 172.21.15.32:0/4196183021 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:22.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce3c00640 1 --2- 172.21.15.32:0/4196183021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f811742000 0x55f81172a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce4600640 1 --2- 172.21.15.32:0/4196183021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f811743000 0x55f81172a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce3200640 1 -- 172.21.15.32:0/4196183021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f811743000 msgr2=0x55f81172a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce3200640 1 --2- 172.21.15.32:0/4196183021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f811743000 0x55f81172a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce3200640 1 -- 172.21.15.32:0/4196183021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f811742000 msgr2=0x55f81172a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce3200640 1 --2- 172.21.15.32:0/4196183021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f811742000 0x55f81172a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce3200640 1 -- 172.21.15.32:0/4196183021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8116d10e0 con 0x55f810961c00 2024-04-17T10:04:22.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce3200640 1 --2- 172.21.15.32:0/4196183021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f810961c00 0x55f81172ab00 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x55f8117d73e0 tx=0x55f810a3c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d7c03f8bc0da9f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce0a00640 1 -- 172.21.15.32:0/4196183021 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f8109969c0 con 0x55f810961c00 2024-04-17T10:04:22.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce0a00640 1 -- 172.21.15.32:0/4196183021 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55f8109acc40 con 0x55f810961c00 2024-04-17T10:04:22.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce0a00640 1 -- 172.21.15.32:0/4196183021 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f810996b60 con 0x55f810961c00 2024-04-17T10:04:22.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce684a400 1 -- 172.21.15.32:0/4196183021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8116d12c0 con 0x55f810961c00 2024-04-17T10:04:22.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce684a400 1 -- 172.21.15.32:0/4196183021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55f8116d14a0 con 0x55f810961c00 2024-04-17T10:04:22.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce684a400 4 waiting for MDS map... 2024-04-17T10:04:22.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce0a00640 1 -- 172.21.15.32:0/4196183021 <== mon.0 v2:172.21.15.32:3300/0 4 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x55f8117e4400 con 0x55f810961c00 2024-04-17T10:04:22.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce0a00640 1 -- 172.21.15.32:0/4196183021 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 105) ==== 2886+0+0 (secure 0 0 0) 0x55f8109b7180 con 0x55f810961c00 2024-04-17T10:04:22.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce684a400 4 Got MDS map 105 2024-04-17T10:04:22.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce684a400 10 main: JournalTool::main 2024-04-17T10:04:22.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce684a400 4 main: JournalTool: connecting to RADOS... 2024-04-17T10:04:22.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce684a400 1 Processor -- start 2024-04-17T10:04:22.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce684a400 1 -- start start 2024-04-17T10:04:22.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce684a400 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f811743800 0x55f81172b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce684a400 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f811743c00 0x55f81172bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce684a400 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f811895c00 0x55f81172c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce684a400 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f81187c480 con 0x55f811743c00 2024-04-17T10:04:22.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce684a400 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f81187c600 con 0x55f811895c00 2024-04-17T10:04:22.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce684a400 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f81187c780 con 0x55f811743800 2024-04-17T10:04:22.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce3c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f811743c00 0x55f81172bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce3c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f811743c00 0x55f81172bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60832/0 (socket says 172.21.15.32:60832) 2024-04-17T10:04:22.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce3c00640 1 -- 172.21.15.32:0/2243368370 learned_addr learned my addr 172.21.15.32:0/2243368370 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:22.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce3200640 1 --2- 172.21.15.32:0/2243368370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f811895c00 0x55f81172c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce4600640 1 --2- 172.21.15.32:0/2243368370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f811743800 0x55f81172b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce3c00640 1 -- 172.21.15.32:0/2243368370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f811743800 msgr2=0x55f81172b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce3c00640 1 --2- 172.21.15.32:0/2243368370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f811743800 0x55f81172b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce3c00640 1 -- 172.21.15.32:0/2243368370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f811895c00 msgr2=0x55f81172c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce3c00640 1 --2- 172.21.15.32:0/2243368370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f811895c00 0x55f81172c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce3c00640 1 -- 172.21.15.32:0/2243368370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8116d01e0 con 0x55f811743c00 2024-04-17T10:04:22.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.510+0000 7f0ce3200640 1 --2- 172.21.15.32:0/2243368370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f811895c00 0x55f81172c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:22.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce3c00640 1 --2- 172.21.15.32:0/2243368370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f811743c00 0x55f81172bb80 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x55f8117d6990 tx=0x55f810a3c120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f887b34347f92d14 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0cde200640 1 -- 172.21.15.32:0/2243368370 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f8109964e0 con 0x55f811743c00 2024-04-17T10:04:22.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0cde200640 1 -- 172.21.15.32:0/2243368370 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55f8109ac540 con 0x55f811743c00 2024-04-17T10:04:22.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0cde200640 1 -- 172.21.15.32:0/2243368370 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f8109969c0 con 0x55f811743c00 2024-04-17T10:04:22.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce684a400 1 -- 172.21.15.32:0/2243368370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f811743c00 msgr2=0x55f81172bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce684a400 1 --2- 172.21.15.32:0/2243368370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f811743c00 0x55f81172bb80 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x55f8117d6990 tx=0x55f810a3c120 comp rx=0 tx=0).stop 2024-04-17T10:04:22.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce684a400 1 -- 172.21.15.32:0/2243368370 shutdown_connections 2024-04-17T10:04:22.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce684a400 1 --2- 172.21.15.32:0/2243368370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f811895c00 0x55f81172c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce684a400 1 --2- 172.21.15.32:0/2243368370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f811743c00 0x55f81172bb80 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce684a400 1 --2- 172.21.15.32:0/2243368370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f811743800 0x55f81172b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce684a400 1 -- 172.21.15.32:0/2243368370 >> 172.21.15.32:0/2243368370 conn(0x55f811743400 msgr2=0x55f81098f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:22.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce684a400 1 -- 172.21.15.32:0/2243368370 shutdown_connections 2024-04-17T10:04:22.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce684a400 1 -- 172.21.15.32:0/2243368370 wait complete. 2024-04-17T10:04:22.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce684a400 1 Processor -- start 2024-04-17T10:04:22.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce684a400 1 -- start start 2024-04-17T10:04:22.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce684a400 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f811743800 0x55f81172b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce684a400 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f811743c00 0x55f81172bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce684a400 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f811895c00 0x55f81172c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce684a400 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f81187dc80 con 0x55f811743c00 2024-04-17T10:04:22.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce684a400 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f81187db00 con 0x55f811895c00 2024-04-17T10:04:22.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce684a400 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f81187d980 con 0x55f811743800 2024-04-17T10:04:22.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce3c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f811743c00 0x55f81172bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce3c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f811743c00 0x55f81172bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60842/0 (socket says 172.21.15.32:60842) 2024-04-17T10:04:22.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce3c00640 1 -- 172.21.15.32:0/1560703627 learned_addr learned my addr 172.21.15.32:0/1560703627 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:22.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce4600640 1 --2- 172.21.15.32:0/1560703627 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f811743800 0x55f81172b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce3200640 1 --2- 172.21.15.32:0/1560703627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f811895c00 0x55f81172c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce3c00640 1 -- 172.21.15.32:0/1560703627 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f811743800 msgr2=0x55f81172b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce3c00640 1 --2- 172.21.15.32:0/1560703627 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f811743800 0x55f81172b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce3c00640 1 -- 172.21.15.32:0/1560703627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f811895c00 msgr2=0x55f81172c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce3c00640 1 --2- 172.21.15.32:0/1560703627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f811895c00 0x55f81172c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce3c00640 1 -- 172.21.15.32:0/1560703627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8116d1e00 con 0x55f811743c00 2024-04-17T10:04:22.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce3200640 1 --2- 172.21.15.32:0/1560703627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f811895c00 0x55f81172c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:22.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce3c00640 1 --2- 172.21.15.32:0/1560703627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f811743c00 0x55f81172bb80 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x55f8118aa990 tx=0x55f810a3a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0cdc400640 1 -- 172.21.15.32:0/1560703627 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f810996b60 con 0x55f811743c00 2024-04-17T10:04:22.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0cdc400640 1 -- 172.21.15.32:0/1560703627 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55f8109ad340 con 0x55f811743c00 2024-04-17T10:04:22.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0cdc400640 1 -- 172.21.15.32:0/1560703627 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f810996340 con 0x55f811743c00 2024-04-17T10:04:22.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce684a400 1 -- 172.21.15.32:0/1560703627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f8116d0f00 con 0x55f811743c00 2024-04-17T10:04:22.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce684a400 1 -- 172.21.15.32:0/1560703627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8116d0960 con 0x55f811743c00 2024-04-17T10:04:22.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.514+0000 7f0ce684a400 4 main: JournalTool: resolving pool 16 2024-04-17T10:04:22.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.518+0000 7f0cdc400640 1 -- 172.21.15.32:0/1560703627 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x55f8118e0c00 con 0x55f811743c00 2024-04-17T10:04:22.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.518+0000 7f0cdc400640 1 --2- 172.21.15.32:0/1560703627 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55f810d44000 0x55f81172c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.518+0000 7f0cdc400640 1 -- 172.21.15.32:0/1560703627 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x55f810c62600 con 0x55f811743c00 2024-04-17T10:04:22.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.518+0000 7f0ce684a400 4 main: JournalTool: creating IoCtx.. 2024-04-17T10:04:22.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.518+0000 7f0ce684a400 4 main: Executing for rank 0 2024-04-17T10:04:22.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.518+0000 7f0ce4600640 1 --2- 172.21.15.32:0/1560703627 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55f810d44000 0x55f81172c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.518+0000 7f0ce4600640 1 --2- 172.21.15.32:0/1560703627 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55f810d44000 0x55f81172c680 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x55f810a7a810 tx=0x55f810d4d800 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.518+0000 7f0ce684a400 1 -- start start 2024-04-17T10:04:22.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.518+0000 7f0ce684a400 1 Processor -- start 2024-04-17T10:04:22.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.518+0000 7f0ce684a400 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f810d45400 0x55f81172cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.518+0000 7f0ce684a400 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f810d45800 0x55f81172d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.518+0000 7f0ce684a400 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f810d45c00 0x55f81172d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.518+0000 7f0ce684a400 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55f81187d680 con 0x55f810d45400 2024-04-17T10:04:22.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.518+0000 7f0ce684a400 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f81187d500 con 0x55f810d45800 2024-04-17T10:04:22.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.518+0000 7f0ce684a400 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f81187c480 con 0x55f810d45c00 2024-04-17T10:04:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.518+0000 7f0ce4600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f810d45400 0x55f81172cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.518+0000 7f0ce4600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f810d45400 0x55f81172cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60858/0 (socket says 172.21.15.32:60858) 2024-04-17T10:04:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.518+0000 7f0ce4600640 1 -- 172.21.15.32:0/1244121404 learned_addr learned my addr 172.21.15.32:0/1244121404 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.522+0000 7f0ce3c00640 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f810d45800 0x55f81172d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.522+0000 7f0ce3200640 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f810d45c00 0x55f81172d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.522+0000 7f0ce4600640 1 -- 172.21.15.32:0/1244121404 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f810d45c00 msgr2=0x55f81172d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.522+0000 7f0ce4600640 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f810d45c00 0x55f81172d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.522+0000 7f0ce4600640 1 -- 172.21.15.32:0/1244121404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f810d45800 msgr2=0x55f81172d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.522+0000 7f0ce4600640 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f810d45800 0x55f81172d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.522+0000 7f0ce4600640 1 -- 172.21.15.32:0/1244121404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f810d68780 con 0x55f810d45400 2024-04-17T10:04:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.522+0000 7f0ce3200640 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f810d45c00 0x55f81172d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.522+0000 7f0ce4600640 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f810d45400 0x55f81172cc00 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x55f810d03590 tx=0x55f810a3a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65dbdc2d733a9e61 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.522+0000 7f0cd8800640 1 -- 172.21.15.32:0/1244121404 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f810c94b60 con 0x55f810d45400 2024-04-17T10:04:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.522+0000 7f0cd8800640 1 -- 172.21.15.32:0/1244121404 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55f8109ad880 con 0x55f810d45400 2024-04-17T10:04:22.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.522+0000 7f0ce684a400 1 -- 172.21.15.32:0/1244121404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f810d68960 con 0x55f810d45400 2024-04-17T10:04:22.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.522+0000 7f0ce684a400 1 -- 172.21.15.32:0/1244121404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55f810d68b40 con 0x55f810d45400 2024-04-17T10:04:22.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.522+0000 7f0ce684a400 4 waiting for MDS map... 2024-04-17T10:04:22.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.522+0000 7f0cd8800640 1 -- 172.21.15.32:0/1244121404 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f810c949c0 con 0x55f810d45400 2024-04-17T10:04:22.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.522+0000 7f0cd8800640 1 -- 172.21.15.32:0/1244121404 <== mon.0 v2:172.21.15.32:3300/0 4 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x55f810dad000 con 0x55f810d45400 2024-04-17T10:04:22.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.522+0000 7f0cd8800640 1 -- 172.21.15.32:0/1244121404 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 105) ==== 2886+0+0 (secure 0 0 0) 0x55f8109b7500 con 0x55f810d45400 2024-04-17T10:04:22.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.522+0000 7f0ce684a400 4 Got MDS map 105 2024-04-17T10:04:22.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.522+0000 7f0ce684a400 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55f810daf800 0x55f810dca000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.522+0000 7f0ce684a400 1 -- 172.21.15.32:0/1244121404 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:1 16.f 16:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55f810dafc00 con 0x55f810daf800 2024-04-17T10:04:22.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.522+0000 7f0ce3c00640 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55f810daf800 0x55f810dca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.522+0000 7f0ce3c00640 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55f810daf800 0x55f810dca000 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=ded1396178daefba server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.526+0000 7f0ce3c00640 1 -- 172.21.15.32:0/1244121404 <== osd.0 v2:172.21.15.32:6800/2392056658 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v113'2 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55f8116e4000 con 0x55f810daf800 2024-04-17T10:04:22.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.526+0000 7f0ce684a400 1 -- 172.21.15.32:0/1244121404 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:2 16.14 16:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55f810dce000 con 0x55f810daf800 2024-04-17T10:04:22.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0ce3c00640 1 -- 172.21.15.32:0/1244121404 <== osd.0 v2:172.21.15.32:6800/2392056658 2 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v113'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55f8116e4000 con 0x55f810daf800 2024-04-17T10:04:22.524 INFO:teuthology.orchestra.run.smithi032.stdout:writing EResetJournal entry 2024-04-17T10:04:22.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0ce684a400 1 -- 172.21.15.32:0/1244121404 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:3 16.14 16:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55f810dce400 con 0x55f810daf800 2024-04-17T10:04:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0ce684a400 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55f810dce800 0x55f810dca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0ce684a400 1 -- 172.21.15.32:0/1244121404 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:4 16.1b 16:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55f810dcec00 con 0x55f810dce800 2024-04-17T10:04:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0ce684a400 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55f810dcf000 0x55f810dcab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0ce684a400 1 -- 172.21.15.32:0/1244121404 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:5 16.b 16:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55f810dcf400 con 0x55f810dcf000 2024-04-17T10:04:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0ce684a400 1 -- 172.21.15.32:0/1244121404 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:6 16.1f 16:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55f810dcf800 con 0x55f810dce800 2024-04-17T10:04:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0ce684a400 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55f810dcfc00 0x55f810dcb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0ce684a400 1 -- 172.21.15.32:0/1244121404 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:7 16.1a 16:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55f810ddc000 con 0x55f810dcfc00 2024-04-17T10:04:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0ce684a400 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55f810ddc400 0x55f810dcb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0ce684a400 1 -- 172.21.15.32:0/1244121404 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:8 16.1e 16:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55f810ddc800 con 0x55f810ddc400 2024-04-17T10:04:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0ce4600640 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55f810dcf000 0x55f810dcab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0ce3200640 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55f810dce800 0x55f810dca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0ce3200640 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55f810ddc400 0x55f810dcb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0ce3c00640 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55f810dcfc00 0x55f810dcb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0ce4600640 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55f810dcf000 0x55f810dcab00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=4294f3b2f7c455a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0ce3200640 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55f810dce800 0x55f810dca580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=72dd691100c5cffb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0ce3200640 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55f810ddc400 0x55f810dcb600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=e4028df83766adb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0ce3c00640 1 -- 172.21.15.32:0/1244121404 <== osd.0 v2:172.21.15.32:6800/2392056658 3 ==== osd_op_reply(3 200.00000001 [delete] v113'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55f8116e4000 con 0x55f810daf800 2024-04-17T10:04:22.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0ce3c00640 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55f810dcfc00 0x55f810dcb080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=d28cacecd92aabf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0ce3200640 1 -- 172.21.15.32:0/1244121404 <== osd.5 v2:172.21.15.144:6800/2379578648 1 ==== osd_op_reply(4 200.00000002 [delete] v113'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55f8116e4500 con 0x55f810dce800 2024-04-17T10:04:22.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0cd6a00640 1 -- 172.21.15.32:0/1244121404 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:9 16.14 16:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55f811742c00 con 0x55f810daf800 2024-04-17T10:04:22.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0ce3200640 1 -- 172.21.15.32:0/1244121404 <== osd.5 v2:172.21.15.144:6800/2379578648 2 ==== osd_op_reply(6 200.00000004 [delete] v113'9 uv8 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55f8116e4500 con 0x55f810dce800 2024-04-17T10:04:22.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.530+0000 7f0ce3c00640 1 -- 172.21.15.32:0/1244121404 <== osd.3 v2:172.21.15.32:6824/2182899298 1 ==== osd_op_reply(7 200.00000005 [delete] v113'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55f8116e4000 con 0x55f810dcfc00 2024-04-17T10:04:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce3c00640 1 -- 172.21.15.32:0/1244121404 <== osd.0 v2:172.21.15.32:6800/2392056658 4 ==== osd_op_reply(9 200.00000001 [write 0~48] v113'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55f8116e4000 con 0x55f810daf800 2024-04-17T10:04:22.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce4600640 1 -- 172.21.15.32:0/1244121404 <== osd.1 v2:172.21.15.32:6808/2454993455 1 ==== osd_op_reply(5 200.00000003 [delete] v113'6 uv5 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55f8116e4780 con 0x55f810dcf000 2024-04-17T10:04:22.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce3200640 1 -- 172.21.15.32:0/1244121404 <== osd.7 v2:172.21.15.144:6824/1746570140 1 ==== osd_op_reply(8 200.00000006 [delete] v113'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55f8116e4500 con 0x55f810ddc400 2024-04-17T10:04:22.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 4 Successfully wrote new journal pointer and header for rank 8:0 2024-04-17T10:04:22.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 -- 172.21.15.32:0/1244121404 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55f810daf800 msgr2=0x55f810dca000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55f810daf800 0x55f810dca000 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 -- 172.21.15.32:0/1244121404 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55f810dcf000 msgr2=0x55f810dcab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55f810dcf000 0x55f810dcab00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 -- 172.21.15.32:0/1244121404 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55f810dcfc00 msgr2=0x55f810dcb080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55f810dcfc00 0x55f810dcb080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 -- 172.21.15.32:0/1244121404 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55f810dce800 msgr2=0x55f810dca580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55f810dce800 0x55f810dca580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 -- 172.21.15.32:0/1244121404 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55f810ddc400 msgr2=0x55f810dcb600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce3200640 1 -- 172.21.15.32:0/1244121404 reap_dead start 2024-04-17T10:04:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55f810ddc400 0x55f810dcb600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce3200640 1 -- 172.21.15.32:0/1244121404 reap_dead start 2024-04-17T10:04:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 -- 172.21.15.32:0/1244121404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f810d45400 msgr2=0x55f81172cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f810d45400 0x55f81172cc00 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x55f810d03590 tx=0x55f810a3a9c0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 -- 172.21.15.32:0/1244121404 shutdown_connections 2024-04-17T10:04:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55f810ddc400 0x55f810dcb600 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 --2- 172.21.15.32:0/1244121404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f810d45400 0x55f81172cc00 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 -- 172.21.15.32:0/1244121404 >> 172.21.15.32:0/1244121404 conn(0x55f810d45000 msgr2=0x55f810d9a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 -- 172.21.15.32:0/1244121404 shutdown_connections 2024-04-17T10:04:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 -- 172.21.15.32:0/1244121404 wait complete. 2024-04-17T10:04:22.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 -- 172.21.15.32:0/1560703627 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55f810d44000 msgr2=0x55f81172c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 --2- 172.21.15.32:0/1560703627 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55f810d44000 0x55f81172c680 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x55f810a7a810 tx=0x55f810d4d800 comp rx=0 tx=0).stop 2024-04-17T10:04:22.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 -- 172.21.15.32:0/1560703627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f811743c00 msgr2=0x55f81172bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 --2- 172.21.15.32:0/1560703627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f811743c00 0x55f81172bb80 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x55f8118aa990 tx=0x55f810a3a540 comp rx=0 tx=0).stop 2024-04-17T10:04:22.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 -- 172.21.15.32:0/1560703627 shutdown_connections 2024-04-17T10:04:22.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 --2- 172.21.15.32:0/1560703627 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55f810d44000 0x55f81172c680 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 --2- 172.21.15.32:0/1560703627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f811895c00 0x55f81172c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 --2- 172.21.15.32:0/1560703627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f811743c00 0x55f81172bb80 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 --2- 172.21.15.32:0/1560703627 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f811743800 0x55f81172b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 -- 172.21.15.32:0/1560703627 >> 172.21.15.32:0/1560703627 conn(0x55f811743400 msgr2=0x55f81098f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:22.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 -- 172.21.15.32:0/1560703627 shutdown_connections 2024-04-17T10:04:22.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 -- 172.21.15.32:0/1560703627 wait complete. 2024-04-17T10:04:22.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 -- 172.21.15.32:0/4196183021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f810961c00 msgr2=0x55f81172ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 --2- 172.21.15.32:0/4196183021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f810961c00 0x55f81172ab00 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x55f8117d73e0 tx=0x55f810a3c180 comp rx=0 tx=0).stop 2024-04-17T10:04:22.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 -- 172.21.15.32:0/4196183021 shutdown_connections 2024-04-17T10:04:22.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 --2- 172.21.15.32:0/4196183021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f811743000 0x55f81172a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 --2- 172.21.15.32:0/4196183021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f810961c00 0x55f81172ab00 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 --2- 172.21.15.32:0/4196183021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f811742000 0x55f81172a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.534+0000 7f0ce684a400 1 -- 172.21.15.32:0/4196183021 >> 172.21.15.32:0/4196183021 conn(0x55f810960c00 msgr2=0x55f81098e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:22.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.538+0000 7f0ce684a400 1 -- 172.21.15.32:0/4196183021 shutdown_connections 2024-04-17T10:04:22.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.538+0000 7f0ce684a400 1 -- 172.21.15.32:0/4196183021 wait complete. 2024-04-17T10:04:22.537 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.081863, result: writing EResetJournal entry 2024-04-17T10:04:22.537 DEBUG:teuthology.orchestra.run.smithi032:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-17T10:04:22.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.626+0000 7f59c46c08c0 1 -- start start 2024-04-17T10:04:22.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.626+0000 7f59c46c08c0 1 Processor -- start 2024-04-17T10:04:22.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c46c08c0 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d72645b800 0x55d727224000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c46c08c0 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d72645b400 0x55d727224580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c46c08c0 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d72645b000 0x55d727224b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c46c08c0 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d7271ce780 con 0x55d72645b800 2024-04-17T10:04:22.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c46c08c0 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d7271ce600 con 0x55d72645b000 2024-04-17T10:04:22.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c46c08c0 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55d7271ce480 con 0x55d72645b400 2024-04-17T10:04:22.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d72645b800 0x55d727224000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d72645b800 0x55d727224000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60874/0 (socket says 172.21.15.32:60874) 2024-04-17T10:04:22.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c2800640 1 -- 172.21.15.32:0/3608182268 learned_addr learned my addr 172.21.15.32:0/3608182268 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:22.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c1e00640 1 --2- 172.21.15.32:0/3608182268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d72645b400 0x55d727224580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c3200640 1 --2- 172.21.15.32:0/3608182268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d72645b000 0x55d727224b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c2800640 1 -- 172.21.15.32:0/3608182268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d72645b400 msgr2=0x55d727224580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c2800640 1 --2- 172.21.15.32:0/3608182268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d72645b400 0x55d727224580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c2800640 1 -- 172.21.15.32:0/3608182268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d72645b000 msgr2=0x55d727224b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c2800640 1 --2- 172.21.15.32:0/3608182268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d72645b000 0x55d727224b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c2800640 1 -- 172.21.15.32:0/3608182268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7264801e0 con 0x55d72645b800 2024-04-17T10:04:22.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c1e00640 1 --2- 172.21.15.32:0/3608182268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d72645b400 0x55d727224580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:22.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c3200640 1 --2- 172.21.15.32:0/3608182268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d72645b000 0x55d727224b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:22.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c2800640 1 --2- 172.21.15.32:0/3608182268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d72645b800 0x55d727224000 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x55d7272b50e0 tx=0x55d726538660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27aaf707f3d8c37 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59bf600640 1 -- 172.21.15.32:0/3608182268 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d7264909c0 con 0x55d72645b800 2024-04-17T10:04:22.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59bf600640 1 -- 172.21.15.32:0/3608182268 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55d7264a6e00 con 0x55d72645b800 2024-04-17T10:04:22.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59bf600640 1 -- 172.21.15.32:0/3608182268 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d7264904e0 con 0x55d72645b800 2024-04-17T10:04:22.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c46c08c0 1 -- 172.21.15.32:0/3608182268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7264814a0 con 0x55d72645b800 2024-04-17T10:04:22.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c46c08c0 1 -- 172.21.15.32:0/3608182268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55d7264803c0 con 0x55d72645b800 2024-04-17T10:04:22.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c46c08c0 4 waiting for MDS map... 2024-04-17T10:04:22.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59bf600640 1 -- 172.21.15.32:0/3608182268 <== mon.0 v2:172.21.15.32:3300/0 4 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x55d727388200 con 0x55d72645b800 2024-04-17T10:04:22.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59bf600640 1 -- 172.21.15.32:0/3608182268 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 105) ==== 2886+0+0 (secure 0 0 0) 0x55d7264b1180 con 0x55d72645b800 2024-04-17T10:04:22.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c46c08c0 4 Got MDS map 105 2024-04-17T10:04:22.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c46c08c0 4 datascan.main: Using metadata pool output 2024-04-17T10:04:22.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c46c08c0 4 datascan.main: connecting to RADOS... 2024-04-17T10:04:22.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c46c08c0 1 Processor -- start 2024-04-17T10:04:22.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c46c08c0 1 -- start start 2024-04-17T10:04:22.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c46c08c0 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d727245c00 0x55d727225080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c46c08c0 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7273dc000 0x55d727225600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.630+0000 7f59c46c08c0 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7273dc400 0x55d727225b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c46c08c0 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d7271ce300 con 0x55d7273dc000 2024-04-17T10:04:22.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c46c08c0 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d7271ce000 con 0x55d727245c00 2024-04-17T10:04:22.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c46c08c0 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55d7264ba900 con 0x55d7273dc400 2024-04-17T10:04:22.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7273dc000 0x55d727225600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7273dc000 0x55d727225600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60888/0 (socket says 172.21.15.32:60888) 2024-04-17T10:04:22.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c2800640 1 -- 172.21.15.32:0/2326330090 learned_addr learned my addr 172.21.15.32:0/2326330090 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:22.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c1e00640 1 --2- 172.21.15.32:0/2326330090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7273dc400 0x55d727225b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c3200640 1 --2- 172.21.15.32:0/2326330090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d727245c00 0x55d727225080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c2800640 1 -- 172.21.15.32:0/2326330090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7273dc400 msgr2=0x55d727225b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c2800640 1 --2- 172.21.15.32:0/2326330090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7273dc400 0x55d727225b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c2800640 1 -- 172.21.15.32:0/2326330090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d727245c00 msgr2=0x55d727225080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c2800640 1 --2- 172.21.15.32:0/2326330090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d727245c00 0x55d727225080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c2800640 1 -- 172.21.15.32:0/2326330090 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7264801e0 con 0x55d7273dc000 2024-04-17T10:04:22.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c2800640 1 --2- 172.21.15.32:0/2326330090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7273dc000 0x55d727225600 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x55d72722ed50 tx=0x55d726538600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627042a2e68e17b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59bce00640 1 -- 172.21.15.32:0/2326330090 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d726490b60 con 0x55d7273dc000 2024-04-17T10:04:22.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59bce00640 1 -- 172.21.15.32:0/2326330090 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55d7264a6c40 con 0x55d7273dc000 2024-04-17T10:04:22.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59bce00640 1 -- 172.21.15.32:0/2326330090 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d726490340 con 0x55d7273dc000 2024-04-17T10:04:22.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c46c08c0 1 -- 172.21.15.32:0/2326330090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7273dc000 msgr2=0x55d727225600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c46c08c0 1 --2- 172.21.15.32:0/2326330090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7273dc000 0x55d727225600 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x55d72722ed50 tx=0x55d726538600 comp rx=0 tx=0).stop 2024-04-17T10:04:22.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c46c08c0 1 -- 172.21.15.32:0/2326330090 shutdown_connections 2024-04-17T10:04:22.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c46c08c0 1 --2- 172.21.15.32:0/2326330090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7273dc400 0x55d727225b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c46c08c0 1 --2- 172.21.15.32:0/2326330090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7273dc000 0x55d727225600 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c46c08c0 1 --2- 172.21.15.32:0/2326330090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d727245c00 0x55d727225080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c46c08c0 1 -- 172.21.15.32:0/2326330090 >> 172.21.15.32:0/2326330090 conn(0x55d72645ac00 msgr2=0x55d726489800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:22.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c46c08c0 1 -- 172.21.15.32:0/2326330090 shutdown_connections 2024-04-17T10:04:22.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c46c08c0 1 -- 172.21.15.32:0/2326330090 wait complete. 2024-04-17T10:04:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c46c08c0 1 Processor -- start 2024-04-17T10:04:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c46c08c0 1 -- start start 2024-04-17T10:04:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c46c08c0 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d727245c00 0x55d727225080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c46c08c0 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7273dd800 0x55d727225600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c46c08c0 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7273dd400 0x55d727225b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c46c08c0 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d7271cf500 con 0x55d7273dd800 2024-04-17T10:04:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c46c08c0 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d7271cf380 con 0x55d727245c00 2024-04-17T10:04:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c46c08c0 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55d7271cf680 con 0x55d7273dd400 2024-04-17T10:04:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7273dd800 0x55d727225600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c2800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7273dd800 0x55d727225600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60892/0 (socket says 172.21.15.32:60892) 2024-04-17T10:04:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c2800640 1 -- 172.21.15.32:0/1926751093 learned_addr learned my addr 172.21.15.32:0/1926751093 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c3200640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d727245c00 0x55d727225080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c1e00640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7273dd400 0x55d727225b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c2800640 1 -- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7273dd400 msgr2=0x55d727225b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c2800640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7273dd400 0x55d727225b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c2800640 1 -- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d727245c00 msgr2=0x55d727225080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c2800640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d727245c00 0x55d727225080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.634+0000 7f59c2800640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7264812c0 con 0x55d7273dd800 2024-04-17T10:04:22.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.638+0000 7f59c2800640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7273dd800 0x55d727225600 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x55d72722f080 tx=0x55d726538840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.638+0000 7f59bb000640 1 -- 172.21.15.32:0/1926751093 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d726490b60 con 0x55d7273dd800 2024-04-17T10:04:22.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.638+0000 7f59bb000640 1 -- 172.21.15.32:0/1926751093 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55d7264a6a80 con 0x55d7273dd800 2024-04-17T10:04:22.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.638+0000 7f59bb000640 1 -- 172.21.15.32:0/1926751093 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d726490340 con 0x55d7273dd800 2024-04-17T10:04:22.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.638+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d726481a40 con 0x55d7273dd800 2024-04-17T10:04:22.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.638+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d726481c20 con 0x55d7273dd800 2024-04-17T10:04:22.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.638+0000 7f59c46c08c0 4 datascan.init: resolving metadata pool 16 2024-04-17T10:04:22.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.638+0000 7f59bb000640 1 -- 172.21.15.32:0/1926751093 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x55d726762c00 con 0x55d7273dd800 2024-04-17T10:04:22.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.638+0000 7f59bb000640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55d72675b400 0x55d727226100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.638+0000 7f59bb000640 1 -- 172.21.15.32:0/1926751093 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x55d7273e2c00 con 0x55d7273dd800 2024-04-17T10:04:22.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.638+0000 7f59c46c08c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-17T10:04:22.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.638+0000 7f59c46c08c0 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-04-17T10:04:22.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.638+0000 7f59c46c08c0 4 datascan.main: opening data pool 'cephfs_data' 2024-04-17T10:04:22.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.638+0000 7f59c46c08c0 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55d7273dd000 0x55d727226680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.638+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d7273dcc00 con 0x55d7273dd000 2024-04-17T10:04:22.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.638+0000 7f59c3200640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55d72675b400 0x55d727226100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.638+0000 7f59c1e00640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55d7273dd000 0x55d727226680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.638+0000 7f59c3200640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55d72675b400 0x55d727226100 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x55d7272b5a70 tx=0x55d72723af60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.638+0000 7f59c1e00640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55d7273dd000 0x55d727226680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.642+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.7 v2:172.21.15.144:6824/1746570140 1 ==== osd_op_reply(1 [pgnls start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d7273dd000 2024-04-17T10:04:22.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.642+0000 7f59bc400640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55d726854000 0x55d72685a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.642+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:2 17.20 17:04000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d726854400 con 0x55d726854000 2024-04-17T10:04:22.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.642+0000 7f59c3200640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55d726854000 0x55d72685a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.642+0000 7f59c3200640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55d726854000 0x55d72685a000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.642+0000 7f59c3200640 1 -- 172.21.15.32:0/1926751093 <== osd.1 v2:172.21.15.32:6808/2454993455 1 ==== osd_op_reply(2 [pgnls start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528500 con 0x55d726854000 2024-04-17T10:04:22.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.642+0000 7f59bce00640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55d726854c00 0x55d72685a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.642+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:3 17.10 17:08000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d726855000 con 0x55d726854c00 2024-04-17T10:04:22.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.642+0000 7f59c2800640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55d726854c00 0x55d72685a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.642+0000 7f59c2800640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55d726854c00 0x55d72685a580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.642+0000 7f59c2800640 1 -- 172.21.15.32:0/1926751093 <== osd.3 v2:172.21.15.32:6824/2182899298 1 ==== osd_op_reply(3 [pgnls start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528780 con 0x55d726854c00 2024-04-17T10:04:22.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.642+0000 7f59bc400640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55d726854800 0x55d72685ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.642+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:4 17.30 17:0c000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d726855800 con 0x55d726854800 2024-04-17T10:04:22.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.642+0000 7f59c1e00640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55d726854800 0x55d72685ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.642+0000 7f59c1e00640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55d726854800 0x55d72685ab00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.642+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.0 v2:172.21.15.32:6800/2392056658 1 ==== osd_op_reply(4 [pgnls start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d726854800 2024-04-17T10:04:22.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.642+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:5 17.8 17:10000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d726855400 con 0x55d7273dd000 2024-04-17T10:04:22.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.642+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.7 v2:172.21.15.144:6824/1746570140 2 ==== osd_op_reply(5 [pgnls start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d7273dd000 2024-04-17T10:04:22.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.646+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:6 17.28 17:14000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d726855c00 con 0x55d726854c00 2024-04-17T10:04:22.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.646+0000 7f59c2800640 1 -- 172.21.15.32:0/1926751093 <== osd.3 v2:172.21.15.32:6824/2182899298 2 ==== osd_op_reply(6 [pgnls start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528780 con 0x55d726854c00 2024-04-17T10:04:22.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.646+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:7 17.18 17:18000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d726855800 con 0x55d726854800 2024-04-17T10:04:22.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.646+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.0 v2:172.21.15.32:6800/2392056658 2 ==== osd_op_reply(7 [pgnls start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d726854800 2024-04-17T10:04:22.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.646+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:8 17.38 17:1c000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72686c000 con 0x55d7273dd000 2024-04-17T10:04:22.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.646+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.7 v2:172.21.15.144:6824/1746570140 3 ==== osd_op_reply(8 [pgnls start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d7273dd000 2024-04-17T10:04:22.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.646+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:9 17.4 17:20000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d7273dcc00 con 0x55d726854800 2024-04-17T10:04:22.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.646+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.0 v2:172.21.15.32:6800/2392056658 3 ==== osd_op_reply(9 [pgnls start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d726854800 2024-04-17T10:04:22.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.646+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:10 17.24 17:24000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72686c400 con 0x55d726854c00 2024-04-17T10:04:22.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.646+0000 7f59c2800640 1 -- 172.21.15.32:0/1926751093 <== osd.3 v2:172.21.15.32:6824/2182899298 3 ==== osd_op_reply(10 [pgnls start_epoch 113 out=49b] v111'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528780 con 0x55d726854c00 2024-04-17T10:04:22.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.646+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:11 17.14 17:28000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72686c000 con 0x55d726854c00 2024-04-17T10:04:22.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.646+0000 7f59c2800640 1 -- 172.21.15.32:0/1926751093 <== osd.3 v2:172.21.15.32:6824/2182899298 4 ==== osd_op_reply(11 [pgnls start_epoch 113 out=49b] v111'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528780 con 0x55d726854c00 2024-04-17T10:04:22.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.646+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:12 17.34 17:2c000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72686c800 con 0x55d726854800 2024-04-17T10:04:22.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.646+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.0 v2:172.21.15.32:6800/2392056658 4 ==== osd_op_reply(12 [pgnls start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d726854800 2024-04-17T10:04:22.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.646+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:13 17.c 17:30000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72686d000 con 0x55d7273dd000 2024-04-17T10:04:22.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.646+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.7 v2:172.21.15.144:6824/1746570140 4 ==== osd_op_reply(13 [pgnls start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d7273dd000 2024-04-17T10:04:22.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.646+0000 7f59bc400640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55d72686cc00 0x55d72685b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.646+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:14 17.2c 17:34000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d726855000 con 0x55d72686cc00 2024-04-17T10:04:22.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.646+0000 7f59c3200640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55d72686cc00 0x55d72685b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.646+0000 7f59c3200640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55d72686cc00 0x55d72685b080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.650+0000 7f59c3200640 1 -- 172.21.15.32:0/1926751093 <== osd.4 v2:172.21.15.144:6809/4076371964 1 ==== osd_op_reply(14 [pgnls start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528500 con 0x55d72686cc00 2024-04-17T10:04:22.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.650+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:15 17.1c 17:38000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d726855400 con 0x55d726854000 2024-04-17T10:04:22.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.650+0000 7f59c3200640 1 -- 172.21.15.32:0/1926751093 <== osd.1 v2:172.21.15.32:6808/2454993455 2 ==== osd_op_reply(15 [pgnls start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528500 con 0x55d726854000 2024-04-17T10:04:22.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.650+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:16 17.3c 17:3c000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d726855c00 con 0x55d726854800 2024-04-17T10:04:22.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.650+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.0 v2:172.21.15.32:6800/2392056658 5 ==== osd_op_reply(16 [pgnls start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d726854800 2024-04-17T10:04:22.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.650+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:17 17.2 17:40000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d726855800 con 0x55d72686cc00 2024-04-17T10:04:22.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.650+0000 7f59c3200640 1 -- 172.21.15.32:0/1926751093 <== osd.4 v2:172.21.15.144:6809/4076371964 2 ==== osd_op_reply(17 [pgnls start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528500 con 0x55d72686cc00 2024-04-17T10:04:22.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.650+0000 7f59bc400640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55d72675b000 0x55d72685b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.650+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:18 17.22 17:44000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72675ac00 con 0x55d72675b000 2024-04-17T10:04:22.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.650+0000 7f59c2800640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55d72675b000 0x55d72685b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.650+0000 7f59c2800640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55d72675b000 0x55d72685b600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.650+0000 7f59c2800640 1 -- 172.21.15.32:0/1926751093 <== osd.5 v2:172.21.15.144:6800/2379578648 1 ==== osd_op_reply(18 [pgnls start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528780 con 0x55d72675b000 2024-04-17T10:04:22.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.650+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:19 17.12 17:48000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d726854400 con 0x55d726854c00 2024-04-17T10:04:22.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.650+0000 7f59c2800640 1 -- 172.21.15.32:0/1926751093 <== osd.3 v2:172.21.15.32:6824/2182899298 5 ==== osd_op_reply(19 [pgnls start_epoch 113 out=81b] v112'22 uv22 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55d726528780 con 0x55d726854c00 2024-04-17T10:04:22.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.650+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:20 17.12 17:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x55d7273dc800 con 0x55d726854c00 2024-04-17T10:04:22.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.650+0000 7f59c2800640 1 -- 172.21.15.32:0/1926751093 <== osd.3 v2:172.21.15.32:6824/2182899298 6 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55d726528780 con 0x55d726854c00 2024-04-17T10:04:22.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.650+0000 7f59c46c08c0 10 datascan.operator(): handling object 1099511627778.0 2024-04-17T10:04:22.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.650+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:21 17.12 17:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55d7273dc400 con 0x55d726854c00 2024-04-17T10:04:22.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.654+0000 7f59c2800640 1 -- 172.21.15.32:0/1926751093 <== osd.3 v2:172.21.15.32:6824/2182899298 7 ==== osd_op_reply(21 10000000002.00000000 [call] v113'23 uv23 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55d726528780 con 0x55d726854c00 2024-04-17T10:04:22.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.654+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:22 17.32 17:4c000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d7273dc000 con 0x55d7273dd000 2024-04-17T10:04:22.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.654+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.7 v2:172.21.15.144:6824/1746570140 5 ==== osd_op_reply(22 [pgnls start_epoch 113 out=49b] v111'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d7273dd000 2024-04-17T10:04:22.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.654+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:23 17.a 17:50000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d726855000 con 0x55d7273dd000 2024-04-17T10:04:22.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.654+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.7 v2:172.21.15.144:6824/1746570140 6 ==== osd_op_reply(23 [pgnls start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d7273dd000 2024-04-17T10:04:22.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.654+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:24 17.2a 17:54000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72675a800 con 0x55d726854c00 2024-04-17T10:04:22.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.654+0000 7f59c2800640 1 -- 172.21.15.32:0/1926751093 <== osd.3 v2:172.21.15.32:6824/2182899298 8 ==== osd_op_reply(24 [pgnls start_epoch 113 out=49b] v111'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528780 con 0x55d726854c00 2024-04-17T10:04:22.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.654+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:25 17.1a 17:58000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d726855400 con 0x55d726854800 2024-04-17T10:04:22.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.654+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.0 v2:172.21.15.32:6800/2392056658 6 ==== osd_op_reply(25 [pgnls start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d726854800 2024-04-17T10:04:22.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.654+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:26 17.3a 17:5c000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72686c800 con 0x55d726854000 2024-04-17T10:04:22.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.654+0000 7f59c3200640 1 -- 172.21.15.32:0/1926751093 <== osd.1 v2:172.21.15.32:6808/2454993455 3 ==== osd_op_reply(26 [pgnls start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528500 con 0x55d726854000 2024-04-17T10:04:22.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.654+0000 7f59bce00640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55d726855800 0x55d72685c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.654+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:27 17.6 17:60000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d7273dcc00 con 0x55d726855800 2024-04-17T10:04:22.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.654+0000 7f59c1e00640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55d726855800 0x55d72685c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.654+0000 7f59c1e00640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55d726855800 0x55d72685c680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.658+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.6 v2:172.21.15.144:6808/3198738696 1 ==== osd_op_reply(27 [pgnls start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d726855800 2024-04-17T10:04:22.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.658+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:28 17.26 17:64000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72686d000 con 0x55d726854800 2024-04-17T10:04:22.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.658+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.0 v2:172.21.15.32:6800/2392056658 7 ==== osd_op_reply(28 [pgnls start_epoch 113 out=49b] v111'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d726854800 2024-04-17T10:04:22.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.658+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:29 17.16 17:68000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72686d000 con 0x55d72675b000 2024-04-17T10:04:22.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.658+0000 7f59c2800640 1 -- 172.21.15.32:0/1926751093 <== osd.5 v2:172.21.15.144:6800/2379578648 2 ==== osd_op_reply(29 [pgnls start_epoch 113 out=49b] v111'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528780 con 0x55d72675b000 2024-04-17T10:04:22.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.658+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:30 17.36 17:6c000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72686c400 con 0x55d726854800 2024-04-17T10:04:22.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.658+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.0 v2:172.21.15.32:6800/2392056658 8 ==== osd_op_reply(30 [pgnls start_epoch 113 out=81b] v111'33 uv17 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55d726528000 con 0x55d726854800 2024-04-17T10:04:22.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.658+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:31 17.36 17:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x55d72686dc00 con 0x55d726854800 2024-04-17T10:04:22.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.658+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.0 v2:172.21.15.32:6800/2392056658 9 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55d726528000 con 0x55d726854800 2024-04-17T10:04:22.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.658+0000 7f59c46c08c0 10 datascan.operator(): handling object 1099511628279.0 2024-04-17T10:04:22.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.658+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:32 17.36 17:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55d72688a000 con 0x55d726854800 2024-04-17T10:04:22.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.658+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.0 v2:172.21.15.32:6800/2392056658 10 ==== osd_op_reply(32 100000001f7.00000000 [call] v113'34 uv34 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55d726528000 con 0x55d726854800 2024-04-17T10:04:22.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.658+0000 7f59c46c08c0 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55d72688a400 0x55d727226c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.658+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:33 17.e 17:70000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72688a800 con 0x55d72688a400 2024-04-17T10:04:22.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.658+0000 7f59c3200640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55d72688a400 0x55d727226c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.662+0000 7f59c3200640 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55d72688a400 0x55d727226c00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.662+0000 7f59c3200640 1 -- 172.21.15.32:0/1926751093 <== osd.2 v2:172.21.15.32:6809/1954833462 1 ==== osd_op_reply(33 [pgnls start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528500 con 0x55d72688a400 2024-04-17T10:04:22.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.662+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:34 17.2e 17:74000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72686c000 con 0x55d726855800 2024-04-17T10:04:22.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.662+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.6 v2:172.21.15.144:6808/3198738696 2 ==== osd_op_reply(34 [pgnls start_epoch 113 out=49b] v111'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d726855800 2024-04-17T10:04:22.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.662+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:35 17.1e 17:78000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72686d800 con 0x55d726854000 2024-04-17T10:04:22.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.662+0000 7f59c3200640 1 -- 172.21.15.32:0/1926751093 <== osd.1 v2:172.21.15.32:6808/2454993455 4 ==== osd_op_reply(35 [pgnls start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528500 con 0x55d726854000 2024-04-17T10:04:22.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.662+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:36 17.3e 17:7c000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72686d400 con 0x55d72686cc00 2024-04-17T10:04:22.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.662+0000 7f59c3200640 1 -- 172.21.15.32:0/1926751093 <== osd.4 v2:172.21.15.144:6809/4076371964 3 ==== osd_op_reply(36 [pgnls start_epoch 113 out=49b] v111'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528500 con 0x55d72686cc00 2024-04-17T10:04:22.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.662+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:37 17.1 17:80000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d7273dc000 con 0x55d726854800 2024-04-17T10:04:22.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.662+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.0 v2:172.21.15.32:6800/2392056658 11 ==== osd_op_reply(37 [pgnls start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d726854800 2024-04-17T10:04:22.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.662+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:38 17.21 17:84000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d726855c00 con 0x55d726854800 2024-04-17T10:04:22.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.662+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.0 v2:172.21.15.32:6800/2392056658 12 ==== osd_op_reply(38 [pgnls start_epoch 113 out=49b] v111'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d726854800 2024-04-17T10:04:22.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.662+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:39 17.11 17:88000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d7273dcc00 con 0x55d726854c00 2024-04-17T10:04:22.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.662+0000 7f59c2800640 1 -- 172.21.15.32:0/1926751093 <== osd.3 v2:172.21.15.32:6824/2182899298 9 ==== osd_op_reply(39 [pgnls start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528780 con 0x55d726854c00 2024-04-17T10:04:22.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.662+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:40 17.31 17:8c000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d726855c00 con 0x55d726854000 2024-04-17T10:04:22.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.662+0000 7f59c3200640 1 -- 172.21.15.32:0/1926751093 <== osd.1 v2:172.21.15.32:6808/2454993455 5 ==== osd_op_reply(40 [pgnls start_epoch 113 out=81b] v111'15 uv8 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55d726528500 con 0x55d726854000 2024-04-17T10:04:22.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.662+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:41 17.31 17:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x55d72688ac00 con 0x55d726854000 2024-04-17T10:04:22.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.662+0000 7f59c3200640 1 -- 172.21.15.32:0/1926751093 <== osd.1 v2:172.21.15.32:6808/2454993455 6 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55d726528500 con 0x55d726854000 2024-04-17T10:04:22.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.662+0000 7f59c46c08c0 10 datascan.operator(): handling object 1099511627780.1 2024-04-17T10:04:22.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.662+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:42 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55d72688b000 con 0x55d726855800 2024-04-17T10:04:22.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.666+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.6 v2:172.21.15.144:6808/3198738696 3 ==== osd_op_reply(42 10000000004.00000000 [call] v113'14 uv14 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55d726528000 con 0x55d726855800 2024-04-17T10:04:22.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.666+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:43 17.9 17:90000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72688b400 con 0x55d726854c00 2024-04-17T10:04:22.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.666+0000 7f59c2800640 1 -- 172.21.15.32:0/1926751093 <== osd.3 v2:172.21.15.32:6824/2182899298 10 ==== osd_op_reply(43 [pgnls start_epoch 113 out=49b] v111'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528780 con 0x55d726854c00 2024-04-17T10:04:22.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.666+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:44 17.29 17:94000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d7273dc000 con 0x55d72686cc00 2024-04-17T10:04:22.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.666+0000 7f59c3200640 1 -- 172.21.15.32:0/1926751093 <== osd.4 v2:172.21.15.144:6809/4076371964 4 ==== osd_op_reply(44 [pgnls start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528500 con 0x55d72686cc00 2024-04-17T10:04:22.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.666+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:45 17.19 17:98000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d726855000 con 0x55d72675b000 2024-04-17T10:04:22.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.666+0000 7f59c2800640 1 -- 172.21.15.32:0/1926751093 <== osd.5 v2:172.21.15.144:6800/2379578648 3 ==== osd_op_reply(45 [pgnls start_epoch 113 out=81b] v112'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55d726528780 con 0x55d72675b000 2024-04-17T10:04:22.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.666+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:46 17.19 17:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x55d72688b800 con 0x55d72675b000 2024-04-17T10:04:22.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.666+0000 7f59c2800640 1 -- 172.21.15.32:0/1926751093 <== osd.5 v2:172.21.15.144:6800/2379578648 4 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55d726528780 con 0x55d72675b000 2024-04-17T10:04:22.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.666+0000 7f59c46c08c0 10 datascan.operator(): handling object 1099511628280.0 2024-04-17T10:04:22.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.666+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:47 17.19 17:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55d72688bc00 con 0x55d72675b000 2024-04-17T10:04:22.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.670+0000 7f59c2800640 1 -- 172.21.15.32:0/1926751093 <== osd.5 v2:172.21.15.144:6800/2379578648 5 ==== osd_op_reply(47 100000001f8.00000000 [call] v113'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55d726528780 con 0x55d72675b000 2024-04-17T10:04:22.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.670+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:48 17.39 17:9c000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d726854400 con 0x55d72675b000 2024-04-17T10:04:22.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.670+0000 7f59c2800640 1 -- 172.21.15.32:0/1926751093 <== osd.5 v2:172.21.15.144:6800/2379578648 6 ==== osd_op_reply(48 [pgnls start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528780 con 0x55d72675b000 2024-04-17T10:04:22.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.670+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:49 17.5 17:a0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d726855400 con 0x55d726855800 2024-04-17T10:04:22.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.670+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.6 v2:172.21.15.144:6808/3198738696 4 ==== osd_op_reply(49 [pgnls start_epoch 113 out=81b] v113'14 uv14 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55d726528000 con 0x55d726855800 2024-04-17T10:04:22.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.670+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:50 17.5 17:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x55d72686d000 con 0x55d726855800 2024-04-17T10:04:22.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.670+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.6 v2:172.21.15.144:6808/3198738696 5 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55d726528000 con 0x55d726855800 2024-04-17T10:04:22.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.670+0000 7f59c46c08c0 10 datascan.operator(): handling object 1099511627780.0 2024-04-17T10:04:22.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.670+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:51 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55d72686c800 con 0x55d726855800 2024-04-17T10:04:22.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.670+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.6 v2:172.21.15.144:6808/3198738696 6 ==== osd_op_reply(51 10000000004.00000000 [call] v113'15 uv15 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55d726528000 con 0x55d726855800 2024-04-17T10:04:22.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.670+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:52 17.25 17:a4000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72686d800 con 0x55d726854000 2024-04-17T10:04:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.670+0000 7f59c3200640 1 -- 172.21.15.32:0/1926751093 <== osd.1 v2:172.21.15.32:6808/2454993455 7 ==== osd_op_reply(52 [pgnls start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528500 con 0x55d726854000 2024-04-17T10:04:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.670+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:53 17.15 17:a8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d726855c00 con 0x55d72675b000 2024-04-17T10:04:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.670+0000 7f59c2800640 1 -- 172.21.15.32:0/1926751093 <== osd.5 v2:172.21.15.144:6800/2379578648 7 ==== osd_op_reply(53 [pgnls start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528780 con 0x55d72675b000 2024-04-17T10:04:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.670+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:54 17.35 17:ac000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72686c400 con 0x55d72686cc00 2024-04-17T10:04:22.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.674+0000 7f59c3200640 1 -- 172.21.15.32:0/1926751093 <== osd.4 v2:172.21.15.144:6809/4076371964 5 ==== osd_op_reply(54 [pgnls start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528500 con 0x55d72686cc00 2024-04-17T10:04:22.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.674+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:55 17.d 17:b0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72689a000 con 0x55d7273dd000 2024-04-17T10:04:22.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.674+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.7 v2:172.21.15.144:6824/1746570140 7 ==== osd_op_reply(55 [pgnls start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d7273dd000 2024-04-17T10:04:22.668 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.674+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:56 17.2d 17:b4000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72686dc00 con 0x55d7273dd000 2024-04-17T10:04:22.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.674+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.7 v2:172.21.15.144:6824/1746570140 8 ==== osd_op_reply(56 [pgnls start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d7273dd000 2024-04-17T10:04:22.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.674+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:57 17.1d 17:b8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72689a400 con 0x55d7273dd000 2024-04-17T10:04:22.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.674+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.7 v2:172.21.15.144:6824/1746570140 9 ==== osd_op_reply(57 [pgnls start_epoch 113 out=49b] v111'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d7273dd000 2024-04-17T10:04:22.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.674+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:58 17.3d 17:bc000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72686c000 con 0x55d7273dd000 2024-04-17T10:04:22.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.674+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.7 v2:172.21.15.144:6824/1746570140 10 ==== osd_op_reply(58 [pgnls start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d7273dd000 2024-04-17T10:04:22.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.674+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:59 17.3 17:c0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72689a800 con 0x55d72686cc00 2024-04-17T10:04:22.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.674+0000 7f59c3200640 1 -- 172.21.15.32:0/1926751093 <== osd.4 v2:172.21.15.144:6809/4076371964 6 ==== osd_op_reply(59 [pgnls start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528500 con 0x55d72686cc00 2024-04-17T10:04:22.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.674+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:60 17.23 17:c4000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72688a000 con 0x55d72686cc00 2024-04-17T10:04:22.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.674+0000 7f59c3200640 1 -- 172.21.15.32:0/1926751093 <== osd.4 v2:172.21.15.144:6809/4076371964 7 ==== osd_op_reply(60 [pgnls start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528500 con 0x55d72686cc00 2024-04-17T10:04:22.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.674+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:61 17.13 17:c8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72689ac00 con 0x55d726854c00 2024-04-17T10:04:22.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.674+0000 7f59c2800640 1 -- 172.21.15.32:0/1926751093 <== osd.3 v2:172.21.15.32:6824/2182899298 11 ==== osd_op_reply(61 [pgnls start_epoch 113 out=49b] v111'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528780 con 0x55d726854c00 2024-04-17T10:04:22.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.674+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:62 17.33 17:cc000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72686d800 con 0x55d7273dd000 2024-04-17T10:04:22.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.674+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.7 v2:172.21.15.144:6824/1746570140 11 ==== osd_op_reply(62 [pgnls start_epoch 113 out=49b] v111'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d7273dd000 2024-04-17T10:04:22.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.674+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:63 17.b 17:d0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72689b000 con 0x55d726854800 2024-04-17T10:04:22.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.674+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.0 v2:172.21.15.32:6800/2392056658 13 ==== osd_op_reply(63 [pgnls start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d726854800 2024-04-17T10:04:22.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.674+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:64 17.2b 17:d4000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72686c400 con 0x55d72686cc00 2024-04-17T10:04:22.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.678+0000 7f59c3200640 1 -- 172.21.15.32:0/1926751093 <== osd.4 v2:172.21.15.144:6809/4076371964 8 ==== osd_op_reply(64 [pgnls start_epoch 113 out=49b] v111'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528500 con 0x55d72686cc00 2024-04-17T10:04:22.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.678+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:65 17.1b 17:d8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72689b400 con 0x55d72686cc00 2024-04-17T10:04:22.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.678+0000 7f59c3200640 1 -- 172.21.15.32:0/1926751093 <== osd.4 v2:172.21.15.144:6809/4076371964 9 ==== osd_op_reply(65 [pgnls start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528500 con 0x55d72686cc00 2024-04-17T10:04:22.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.678+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:66 17.3b 17:dc000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72686d000 con 0x55d72688a400 2024-04-17T10:04:22.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.678+0000 7f59c3200640 1 -- 172.21.15.32:0/1926751093 <== osd.2 v2:172.21.15.32:6809/1954833462 2 ==== osd_op_reply(66 [pgnls start_epoch 113 out=49b] v111'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528500 con 0x55d72688a400 2024-04-17T10:04:22.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.678+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:67 17.7 17:e0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72689b800 con 0x55d726854800 2024-04-17T10:04:22.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.678+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.0 v2:172.21.15.32:6800/2392056658 14 ==== osd_op_reply(67 [pgnls start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d726854800 2024-04-17T10:04:22.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.678+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:68 17.27 17:e4000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d726855400 con 0x55d72686cc00 2024-04-17T10:04:22.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.678+0000 7f59c3200640 1 -- 172.21.15.32:0/1926751093 <== osd.4 v2:172.21.15.144:6809/4076371964 10 ==== osd_op_reply(68 [pgnls start_epoch 113 out=49b] v111'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528500 con 0x55d72686cc00 2024-04-17T10:04:22.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.678+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:69 17.17 17:e8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72689bc00 con 0x55d726855800 2024-04-17T10:04:22.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.678+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.6 v2:172.21.15.144:6808/3198738696 7 ==== osd_op_reply(69 [pgnls start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d726855800 2024-04-17T10:04:22.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.678+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:70 17.37 17:ec000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72688ac00 con 0x55d726854800 2024-04-17T10:04:22.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.678+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.0 v2:172.21.15.32:6800/2392056658 15 ==== osd_op_reply(70 [pgnls start_epoch 113 out=49b] v111'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d726854800 2024-04-17T10:04:22.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.678+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:71 17.f 17:f0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72689a800 con 0x55d726854000 2024-04-17T10:04:22.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.678+0000 7f59c3200640 1 -- 172.21.15.32:0/1926751093 <== osd.1 v2:172.21.15.32:6808/2454993455 8 ==== osd_op_reply(71 [pgnls start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528500 con 0x55d726854000 2024-04-17T10:04:22.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.678+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:72 17.2f 17:f4000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72688a000 con 0x55d72686cc00 2024-04-17T10:04:22.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.678+0000 7f59c3200640 1 -- 172.21.15.32:0/1926751093 <== osd.4 v2:172.21.15.144:6809/4076371964 11 ==== osd_op_reply(72 [pgnls start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528500 con 0x55d72686cc00 2024-04-17T10:04:22.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.678+0000 7f59bce00640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:73 17.1f 17:f8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72689a000 con 0x55d726855800 2024-04-17T10:04:22.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.6 v2:172.21.15.144:6808/3198738696 8 ==== osd_op_reply(73 [pgnls start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d726855800 2024-04-17T10:04:22.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59bc400640 1 -- 172.21.15.32:0/1926751093 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:74 17.3f 17:fc000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x55d72688b000 con 0x55d726854800 2024-04-17T10:04:22.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 <== osd.0 v2:172.21.15.32:6800/2392056658 16 ==== osd_op_reply(74 [pgnls start_epoch 113 out=49b] v111'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d726528000 con 0x55d726854800 2024-04-17T10:04:22.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55d726854800 msgr2=0x55d72685ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x55d726854800 0x55d72685ab00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55d726854000 msgr2=0x55d72685a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x55d726854000 0x55d72685a000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55d72688a400 msgr2=0x55d727226c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x55d72688a400 0x55d727226c00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55d726854c00 msgr2=0x55d72685a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x55d726854c00 0x55d72685a580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55d72686cc00 msgr2=0x55d72685b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x55d72686cc00 0x55d72685b080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55d72675b000 msgr2=0x55d72685b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x55d72675b000 0x55d72685b600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.678 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55d726855800 msgr2=0x55d72685c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x55d726855800 0x55d72685c680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55d7273dd000 msgr2=0x55d727226680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x55d7273dd000 0x55d727226680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 reap_dead start 2024-04-17T10:04:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 reap_dead start 2024-04-17T10:04:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c1e00640 1 -- 172.21.15.32:0/1926751093 reap_dead start 2024-04-17T10:04:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55d72675b400 msgr2=0x55d727226100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55d72675b400 0x55d727226100 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x55d7272b5a70 tx=0x55d72723af60 comp rx=0 tx=0).stop 2024-04-17T10:04:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7273dd800 msgr2=0x55d727225600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7273dd800 0x55d727225600 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x55d72722f080 tx=0x55d726538840 comp rx=0 tx=0).stop 2024-04-17T10:04:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 shutdown_connections 2024-04-17T10:04:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x55d72675b400 0x55d727226100 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 --2- 172.21.15.32:0/1926751093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d7273dd800 0x55d727225600 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 >> 172.21.15.32:0/1926751093 conn(0x55d72645ac00 msgr2=0x55d726489800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:22.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 shutdown_connections 2024-04-17T10:04:22.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 -- 172.21.15.32:0/1926751093 wait complete. 2024-04-17T10:04:22.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 -- 172.21.15.32:0/3608182268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d72645b800 msgr2=0x55d727224000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 --2- 172.21.15.32:0/3608182268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d72645b800 0x55d727224000 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x55d7272b50e0 tx=0x55d726538660 comp rx=0 tx=0).stop 2024-04-17T10:04:22.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 -- 172.21.15.32:0/3608182268 shutdown_connections 2024-04-17T10:04:22.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 --2- 172.21.15.32:0/3608182268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d72645b400 0x55d727224580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 --2- 172.21.15.32:0/3608182268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d72645b000 0x55d727224b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 --2- 172.21.15.32:0/3608182268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d72645b800 0x55d727224000 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 -- 172.21.15.32:0/3608182268 >> 172.21.15.32:0/3608182268 conn(0x55d72645bc00 msgr2=0x55d726489000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:22.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 -- 172.21.15.32:0/3608182268 shutdown_connections 2024-04-17T10:04:22.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.682+0000 7f59c46c08c0 1 -- 172.21.15.32:0/3608182268 wait complete. 2024-04-17T10:04:22.682 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.144692, result: 2024-04-17T10:04:22.682 DEBUG:teuthology.orchestra.run.smithi032:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-04-17T10:04:22.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4904ec08c0 1 -- start start 2024-04-17T10:04:22.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4904ec08c0 1 Processor -- start 2024-04-17T10:04:22.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4904ec08c0 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634ad509000 0x5634ae2dc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4904ec08c0 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634ad509400 0x5634ae2dc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4904ec08c0 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5634ad509800 0x5634ae2dcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4904ec08c0 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5634ae300180 con 0x5634ad509000 2024-04-17T10:04:22.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4904ec08c0 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5634ad568900 con 0x5634ad509400 2024-04-17T10:04:22.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4904ec08c0 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5634ad568480 con 0x5634ad509800 2024-04-17T10:04:22.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4903000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634ad509000 0x5634ae2dc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4903000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634ad509000 0x5634ae2dc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60908/0 (socket says 172.21.15.32:60908) 2024-04-17T10:04:22.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4903000640 1 -- 172.21.15.32:0/1343628428 learned_addr learned my addr 172.21.15.32:0/1343628428 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:22.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4903a00640 1 --2- 172.21.15.32:0/1343628428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5634ad509800 0x5634ae2dcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4902600640 1 --2- 172.21.15.32:0/1343628428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634ad509400 0x5634ae2dc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4903000640 1 -- 172.21.15.32:0/1343628428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5634ad509800 msgr2=0x5634ae2dcb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4903000640 1 --2- 172.21.15.32:0/1343628428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5634ad509800 0x5634ae2dcb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4903000640 1 -- 172.21.15.32:0/1343628428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634ad509400 msgr2=0x5634ae2dc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4903000640 1 --2- 172.21.15.32:0/1343628428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634ad509400 0x5634ae2dc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4903000640 1 -- 172.21.15.32:0/1343628428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634ad52e1e0 con 0x5634ad509000 2024-04-17T10:04:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4903000640 1 --2- 172.21.15.32:0/1343628428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634ad509000 0x5634ae2dc000 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x5634ae371470 tx=0x5634ad5eafc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0a23709f13e1ca server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f48ffe00640 1 -- 172.21.15.32:0/1343628428 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5634ad53e4e0 con 0x5634ad509000 2024-04-17T10:04:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f48ffe00640 1 -- 172.21.15.32:0/1343628428 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5634ad5541c0 con 0x5634ad509000 2024-04-17T10:04:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f48ffe00640 1 -- 172.21.15.32:0/1343628428 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5634ad53eb60 con 0x5634ad509000 2024-04-17T10:04:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4904ec08c0 1 -- 172.21.15.32:0/1343628428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634ad52f2c0 con 0x5634ad509000 2024-04-17T10:04:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4904ec08c0 1 -- 172.21.15.32:0/1343628428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5634ad52fa40 con 0x5634ad509000 2024-04-17T10:04:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4904ec08c0 4 waiting for MDS map... 2024-04-17T10:04:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f48ffe00640 1 -- 172.21.15.32:0/1343628428 <== mon.0 v2:172.21.15.32:3300/0 4 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x5634ad60d000 con 0x5634ad509000 2024-04-17T10:04:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f48ffe00640 1 -- 172.21.15.32:0/1343628428 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 105) ==== 2886+0+0 (secure 0 0 0) 0x5634ad55f180 con 0x5634ad509000 2024-04-17T10:04:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4904ec08c0 4 Got MDS map 105 2024-04-17T10:04:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4904ec08c0 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-04-17T10:04:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4904ec08c0 4 datascan.main: Using metadata pool output 2024-04-17T10:04:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.778+0000 7f4904ec08c0 4 datascan.main: connecting to RADOS... 2024-04-17T10:04:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4904ec08c0 1 Processor -- start 2024-04-17T10:04:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4904ec08c0 1 -- start start 2024-04-17T10:04:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4904ec08c0 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634ae44b400 0x5634ae2dd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4904ec08c0 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5634ae44b800 0x5634ae2dd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4904ec08c0 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634ae44bc00 0x5634ae2ddb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4904ec08c0 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5634ad568180 con 0x5634ae44b400 2024-04-17T10:04:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4904ec08c0 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5634ae300d80 con 0x5634ae44bc00 2024-04-17T10:04:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4904ec08c0 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5634ae300f00 con 0x5634ae44b800 2024-04-17T10:04:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4903a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634ae44b400 0x5634ae2dd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4903a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634ae44b400 0x5634ae2dd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60916/0 (socket says 172.21.15.32:60916) 2024-04-17T10:04:22.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4903a00640 1 -- 172.21.15.32:0/4219043141 learned_addr learned my addr 172.21.15.32:0/4219043141 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:22.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4903000640 1 --2- 172.21.15.32:0/4219043141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5634ae44b800 0x5634ae2dd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4903a00640 1 -- 172.21.15.32:0/4219043141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5634ae44b800 msgr2=0x5634ae2dd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4902600640 1 --2- 172.21.15.32:0/4219043141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634ae44bc00 0x5634ae2ddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4903a00640 1 --2- 172.21.15.32:0/4219043141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5634ae44b800 0x5634ae2dd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4903a00640 1 -- 172.21.15.32:0/4219043141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634ae44bc00 msgr2=0x5634ae2ddb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4903a00640 1 --2- 172.21.15.32:0/4219043141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634ae44bc00 0x5634ae2ddb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4903a00640 1 -- 172.21.15.32:0/4219043141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634ad52fc20 con 0x5634ae44b400 2024-04-17T10:04:22.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4903a00640 1 --2- 172.21.15.32:0/4219043141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634ae44b400 0x5634ae2dd080 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x5634ae44e660 tx=0x5634ae458c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83cb5a9b2827d489 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f48fd600640 1 -- 172.21.15.32:0/4219043141 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5634ad53e4e0 con 0x5634ae44b400 2024-04-17T10:04:22.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f48fd600640 1 -- 172.21.15.32:0/4219043141 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5634ad554e00 con 0x5634ae44b400 2024-04-17T10:04:22.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f48fd600640 1 -- 172.21.15.32:0/4219043141 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5634ad53eb60 con 0x5634ae44b400 2024-04-17T10:04:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4904ec08c0 1 -- 172.21.15.32:0/4219043141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634ae44b400 msgr2=0x5634ae2dd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/4219043141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634ae44b400 0x5634ae2dd080 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x5634ae44e660 tx=0x5634ae458c00 comp rx=0 tx=0).stop 2024-04-17T10:04:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4904ec08c0 1 -- 172.21.15.32:0/4219043141 shutdown_connections 2024-04-17T10:04:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/4219043141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5634ae44b800 0x5634ae2dd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/4219043141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634ae44bc00 0x5634ae2ddb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/4219043141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634ae44b400 0x5634ae2dd080 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4904ec08c0 1 -- 172.21.15.32:0/4219043141 >> 172.21.15.32:0/4219043141 conn(0x5634ae44b000 msgr2=0x5634ad537800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:22.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4904ec08c0 1 -- 172.21.15.32:0/4219043141 shutdown_connections 2024-04-17T10:04:22.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4904ec08c0 1 -- 172.21.15.32:0/4219043141 wait complete. 2024-04-17T10:04:22.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4904ec08c0 1 Processor -- start 2024-04-17T10:04:22.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.782+0000 7f4904ec08c0 1 -- start start 2024-04-17T10:04:22.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f4904ec08c0 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634ae44b400 0x5634ae2dd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f4904ec08c0 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5634ae44b800 0x5634ae2dd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f4904ec08c0 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634ae44bc00 0x5634ae2ddb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f4904ec08c0 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5634ae301200 con 0x5634ae44bc00 2024-04-17T10:04:22.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f4904ec08c0 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5634ae301080 con 0x5634ae44b400 2024-04-17T10:04:22.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f4904ec08c0 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5634ae301380 con 0x5634ae44b800 2024-04-17T10:04:22.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f4902600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634ae44bc00 0x5634ae2ddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f4902600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634ae44bc00 0x5634ae2ddb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60918/0 (socket says 172.21.15.32:60918) 2024-04-17T10:04:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f4902600640 1 -- 172.21.15.32:0/3432519519 learned_addr learned my addr 172.21.15.32:0/3432519519 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f4903000640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5634ae44b800 0x5634ae2dd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f4903a00640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634ae44b400 0x5634ae2dd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f4902600640 1 -- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5634ae44b800 msgr2=0x5634ae2dd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f4902600640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5634ae44b800 0x5634ae2dd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f4902600640 1 -- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634ae44b400 msgr2=0x5634ae2dd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f4902600640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634ae44b400 0x5634ae2dd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f4902600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634ad52e1e0 con 0x5634ae44bc00 2024-04-17T10:04:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f4902600640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634ae44bc00 0x5634ae2ddb80 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x5634ae44f950 tx=0x5634ad5ea120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f48fb800640 1 -- 172.21.15.32:0/3432519519 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5634ad53e9c0 con 0x5634ae44bc00 2024-04-17T10:04:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f48fb800640 1 -- 172.21.15.32:0/3432519519 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5634ad554540 con 0x5634ae44bc00 2024-04-17T10:04:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f48fb800640 1 -- 172.21.15.32:0/3432519519 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5634ad53e4e0 con 0x5634ae44bc00 2024-04-17T10:04:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5634ad52f2c0 con 0x5634ae44bc00 2024-04-17T10:04:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634ad52fe00 con 0x5634ae44bc00 2024-04-17T10:04:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f4904ec08c0 4 datascan.init: resolving metadata pool 16 2024-04-17T10:04:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f48fb800640 1 -- 172.21.15.32:0/3432519519 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x5634ad810c00 con 0x5634ae44bc00 2024-04-17T10:04:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.786+0000 7f48fb800640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x5634ad806c00 0x5634ae2de100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.790+0000 7f48fb800640 1 -- 172.21.15.32:0/3432519519 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x5634ad808a00 con 0x5634ae44bc00 2024-04-17T10:04:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.790+0000 7f4904ec08c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-17T10:04:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.790+0000 7f4904ec08c0 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-04-17T10:04:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.790+0000 7f4904ec08c0 4 datascan.main: opening data pool 'cephfs_data' 2024-04-17T10:04:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.790+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x5634ad807000 0x5634ae2de680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.790+0000 7f4903a00640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x5634ad806c00 0x5634ae2de100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.790+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:1 16.14 16:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5634ad807400 con 0x5634ad807000 2024-04-17T10:04:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.790+0000 7f4903000640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x5634ad807000 0x5634ae2de680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.790+0000 7f4903a00640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x5634ad806c00 0x5634ae2de100 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x5634ae44e660 tx=0x5634ad5eacc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.790+0000 7f4903000640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x5634ad807000 0x5634ae2de680 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.790+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.0 v2:172.21.15.32:6800/2392056658 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv36 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807000 2024-04-17T10:04:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.790+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x5634ad807800 0x5634ae2dec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.790+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:2 16.1f 16:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5634ad807c00 con 0x5634ad807800 2024-04-17T10:04:22.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.790+0000 7f4903a00640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x5634ad807800 0x5634ae2dec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.790+0000 7f4903a00640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x5634ad807800 0x5634ae2dec00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.790+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.5 v2:172.21.15.144:6800/2379578648 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad807800 2024-04-17T10:04:22.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.790+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x5634ad904000 0x5634ae2df180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.790+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:3 17.0 17:00000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad904400 con 0x5634ad904000 2024-04-17T10:04:22.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.790+0000 7f4903000640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x5634ad904000 0x5634ae2df180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.790+0000 7f4903000640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x5634ad904000 0x5634ae2df180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.794+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.7 v2:172.21.15.144:6824/1746570140 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad904000 2024-04-17T10:04:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.794+0000 7f48fcc00640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x5634ad905800 0x5634ad910000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.794+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:4 17.20 17:04000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad905c00 con 0x5634ad905800 2024-04-17T10:04:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.794+0000 7f4902600640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x5634ad905800 0x5634ad910000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.794+0000 7f4902600640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x5634ad905800 0x5634ad910000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.794+0000 7f4902600640 1 -- 172.21.15.32:0/3432519519 <== osd.1 v2:172.21.15.32:6808/2454993455 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e780 con 0x5634ad905800 2024-04-17T10:04:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.794+0000 7f48fd600640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x5634ad926000 0x5634ad910580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.794+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:5 17.10 17:08000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad926400 con 0x5634ad926000 2024-04-17T10:04:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.794+0000 7f4903a00640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x5634ad926000 0x5634ad910580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.794+0000 7f4903a00640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x5634ad926000 0x5634ad910580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.794+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.3 v2:172.21.15.32:6824/2182899298 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad926000 2024-04-17T10:04:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.794+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:6 17.30 17:0c000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad926c00 con 0x5634ad807000 2024-04-17T10:04:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.794+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.0 v2:172.21.15.32:6800/2392056658 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807000 2024-04-17T10:04:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.794+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:7 17.8 17:10000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad926800 con 0x5634ad904000 2024-04-17T10:04:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.794+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.7 v2:172.21.15.144:6824/1746570140 2 ==== osd_op_reply(7 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad904000 2024-04-17T10:04:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.794+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:8 17.28 17:14000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad927000 con 0x5634ad926000 2024-04-17T10:04:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.794+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.3 v2:172.21.15.32:6824/2182899298 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad926000 2024-04-17T10:04:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.794+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:9 17.18 17:18000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad806800 con 0x5634ad807000 2024-04-17T10:04:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.794+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.0 v2:172.21.15.32:6800/2392056658 3 ==== osd_op_reply(9 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807000 2024-04-17T10:04:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.794+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:10 17.38 17:1c000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad927400 con 0x5634ad904000 2024-04-17T10:04:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.794+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.7 v2:172.21.15.144:6824/1746570140 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad904000 2024-04-17T10:04:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.794+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:11 17.4 17:20000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ae44a800 con 0x5634ad807000 2024-04-17T10:04:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.798+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.0 v2:172.21.15.32:6800/2392056658 4 ==== osd_op_reply(11 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807000 2024-04-17T10:04:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.798+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:12 17.24 17:24000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad926400 con 0x5634ad926000 2024-04-17T10:04:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.798+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.3 v2:172.21.15.32:6824/2182899298 3 ==== osd_op_reply(12 [pgnls-filter start_epoch 113 out=49b] v111'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad926000 2024-04-17T10:04:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.798+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:13 17.14 17:28000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ae44a400 con 0x5634ad926000 2024-04-17T10:04:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.798+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.3 v2:172.21.15.32:6824/2182899298 4 ==== osd_op_reply(13 [pgnls-filter start_epoch 113 out=49b] v111'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad926000 2024-04-17T10:04:22.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.798+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:14 17.34 17:2c000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad904400 con 0x5634ad807000 2024-04-17T10:04:22.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.798+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.0 v2:172.21.15.32:6800/2392056658 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807000 2024-04-17T10:04:22.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.798+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:15 17.c 17:30000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad927000 con 0x5634ad904000 2024-04-17T10:04:22.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.798+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.7 v2:172.21.15.144:6824/1746570140 4 ==== osd_op_reply(15 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad904000 2024-04-17T10:04:22.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.798+0000 7f48fcc00640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x5634ad807c00 0x5634ad910b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.798+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:16 17.2c 17:34000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad807400 con 0x5634ad807c00 2024-04-17T10:04:22.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.798+0000 7f4903000640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x5634ad807c00 0x5634ad910b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.798+0000 7f4903000640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x5634ad807c00 0x5634ad910b00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.798+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.4 v2:172.21.15.144:6809/4076371964 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807c00 2024-04-17T10:04:22.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.798+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:17 17.1c 17:38000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad926c00 con 0x5634ad905800 2024-04-17T10:04:22.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.798+0000 7f4902600640 1 -- 172.21.15.32:0/3432519519 <== osd.1 v2:172.21.15.32:6808/2454993455 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e780 con 0x5634ad905800 2024-04-17T10:04:22.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.798+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:18 17.3c 17:3c000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad926800 con 0x5634ad807000 2024-04-17T10:04:22.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.798+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.0 v2:172.21.15.32:6800/2392056658 6 ==== osd_op_reply(18 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807000 2024-04-17T10:04:22.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.798+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:19 17.2 17:40000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad927800 con 0x5634ad807c00 2024-04-17T10:04:22.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.798+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.4 v2:172.21.15.144:6809/4076371964 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807c00 2024-04-17T10:04:22.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.798+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:20 17.22 17:44000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad927400 con 0x5634ad807800 2024-04-17T10:04:22.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.802+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.5 v2:172.21.15.144:6800/2379578648 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad807800 2024-04-17T10:04:22.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.802+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:21 17.12 17:48000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad927c00 con 0x5634ad926000 2024-04-17T10:04:22.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.802+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.3 v2:172.21.15.32:6824/2182899298 5 ==== osd_op_reply(21 [pgnls-filter start_epoch 113 out=81b] v113'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad926000 2024-04-17T10:04:22.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.802+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:22 17.0 17:00000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad904800 con 0x5634ad904000 2024-04-17T10:04:22.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.802+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.7 v2:172.21.15.144:6824/1746570140 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad904000 2024-04-17T10:04:22.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.802+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:23 17.20 17:04000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad927000 con 0x5634ad905800 2024-04-17T10:04:22.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.802+0000 7f4902600640 1 -- 172.21.15.32:0/3432519519 <== osd.1 v2:172.21.15.32:6808/2454993455 3 ==== osd_op_reply(23 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e780 con 0x5634ad905800 2024-04-17T10:04:22.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.802+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:24 17.10 17:08000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad904400 con 0x5634ad926000 2024-04-17T10:04:22.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.802+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.3 v2:172.21.15.32:6824/2182899298 6 ==== osd_op_reply(24 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad926000 2024-04-17T10:04:22.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.802+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:25 17.30 17:0c000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad904400 con 0x5634ad807000 2024-04-17T10:04:22.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.802+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.0 v2:172.21.15.32:6800/2392056658 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807000 2024-04-17T10:04:22.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.802+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:26 17.8 17:10000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ae44a800 con 0x5634ad904000 2024-04-17T10:04:22.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.802+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.7 v2:172.21.15.144:6824/1746570140 6 ==== osd_op_reply(26 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad904000 2024-04-17T10:04:22.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.802+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:27 17.28 17:14000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad926400 con 0x5634ad926000 2024-04-17T10:04:22.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.802+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.3 v2:172.21.15.32:6824/2182899298 7 ==== osd_op_reply(27 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad926000 2024-04-17T10:04:22.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.802+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:28 17.18 17:18000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad806800 con 0x5634ad807000 2024-04-17T10:04:22.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.802+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.0 v2:172.21.15.32:6800/2392056658 8 ==== osd_op_reply(28 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807000 2024-04-17T10:04:22.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.802+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:29 17.38 17:1c000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad927400 con 0x5634ad904000 2024-04-17T10:04:22.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.802+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.7 v2:172.21.15.144:6824/1746570140 7 ==== osd_op_reply(29 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad904000 2024-04-17T10:04:22.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.802+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:30 17.4 17:20000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad807400 con 0x5634ad807000 2024-04-17T10:04:22.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.806+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.0 v2:172.21.15.32:6800/2392056658 9 ==== osd_op_reply(30 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807000 2024-04-17T10:04:22.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.806+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:31 17.24 17:24000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad927c00 con 0x5634ad926000 2024-04-17T10:04:22.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.806+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.3 v2:172.21.15.32:6824/2182899298 8 ==== osd_op_reply(31 [pgnls-filter start_epoch 113 out=49b] v111'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad926000 2024-04-17T10:04:22.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.806+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:32 17.14 17:28000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ae44a800 con 0x5634ad926000 2024-04-17T10:04:22.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.806+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.3 v2:172.21.15.32:6824/2182899298 9 ==== osd_op_reply(32 [pgnls-filter start_epoch 113 out=49b] v111'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad926000 2024-04-17T10:04:22.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.806+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:33 17.34 17:2c000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ae44a400 con 0x5634ad807000 2024-04-17T10:04:22.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.806+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.0 v2:172.21.15.32:6800/2392056658 10 ==== osd_op_reply(33 [pgnls-filter start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807000 2024-04-17T10:04:22.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.806+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:34 17.c 17:30000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad904800 con 0x5634ad904000 2024-04-17T10:04:22.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.806+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.7 v2:172.21.15.144:6824/1746570140 8 ==== osd_op_reply(34 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad904000 2024-04-17T10:04:22.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.806+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:35 17.2c 17:34000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad806800 con 0x5634ad807c00 2024-04-17T10:04:22.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.806+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.4 v2:172.21.15.144:6809/4076371964 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807c00 2024-04-17T10:04:22.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.806+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:36 17.1c 17:38000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad904400 con 0x5634ad905800 2024-04-17T10:04:22.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.806+0000 7f4902600640 1 -- 172.21.15.32:0/3432519519 <== osd.1 v2:172.21.15.32:6808/2454993455 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e780 con 0x5634ad905800 2024-04-17T10:04:22.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.806+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:37 17.3c 17:3c000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad905c00 con 0x5634ad807000 2024-04-17T10:04:22.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.806+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.0 v2:172.21.15.32:6800/2392056658 11 ==== osd_op_reply(37 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807000 2024-04-17T10:04:22.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.806+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:38 17.2 17:40000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad926800 con 0x5634ad807c00 2024-04-17T10:04:22.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.806+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.4 v2:172.21.15.144:6809/4076371964 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807c00 2024-04-17T10:04:22.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.806+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:39 17.22 17:44000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad904400 con 0x5634ad807800 2024-04-17T10:04:22.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.806+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.5 v2:172.21.15.144:6800/2379578648 3 ==== osd_op_reply(39 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad807800 2024-04-17T10:04:22.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.806+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:40 17.12 17:48000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad927800 con 0x5634ad926000 2024-04-17T10:04:22.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.806+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.3 v2:172.21.15.32:6824/2182899298 10 ==== osd_op_reply(40 [pgnls-filter start_epoch 113 out=49b] v113'23 uv23 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad926000 2024-04-17T10:04:22.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.806+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:41 17.32 17:4c000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad926c00 con 0x5634ad904000 2024-04-17T10:04:22.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.810+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.7 v2:172.21.15.144:6824/1746570140 9 ==== osd_op_reply(41 [pgnls-filter start_epoch 113 out=49b] v111'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad904000 2024-04-17T10:04:22.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.810+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:42 17.a 17:50000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad927400 con 0x5634ad904000 2024-04-17T10:04:22.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.810+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.7 v2:172.21.15.144:6824/1746570140 10 ==== osd_op_reply(42 [pgnls-filter start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad904000 2024-04-17T10:04:22.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.810+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:43 17.2a 17:54000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad927000 con 0x5634ad926000 2024-04-17T10:04:22.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.810+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.3 v2:172.21.15.32:6824/2182899298 11 ==== osd_op_reply(43 [pgnls-filter start_epoch 113 out=49b] v111'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad926000 2024-04-17T10:04:22.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.810+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:44 17.1a 17:58000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad926800 con 0x5634ad807000 2024-04-17T10:04:22.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.810+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.0 v2:172.21.15.32:6800/2392056658 12 ==== osd_op_reply(44 [pgnls-filter start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807000 2024-04-17T10:04:22.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.810+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:45 17.3a 17:5c000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ae305000 con 0x5634ad905800 2024-04-17T10:04:22.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.810+0000 7f4902600640 1 -- 172.21.15.32:0/3432519519 <== osd.1 v2:172.21.15.32:6808/2454993455 5 ==== osd_op_reply(45 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e780 con 0x5634ad905800 2024-04-17T10:04:22.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.810+0000 7f48fcc00640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x5634ad904800 0x5634ad911080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.810+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:46 17.6 17:60000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad905c00 con 0x5634ad904800 2024-04-17T10:04:22.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.810+0000 7f4902600640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x5634ad904800 0x5634ad911080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.810+0000 7f4902600640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x5634ad904800 0x5634ad911080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.810+0000 7f4902600640 1 -- 172.21.15.32:0/3432519519 <== osd.6 v2:172.21.15.144:6808/3198738696 1 ==== osd_op_reply(46 [pgnls-filter start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e780 con 0x5634ad904800 2024-04-17T10:04:22.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.810+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:47 17.26 17:64000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad926400 con 0x5634ad807000 2024-04-17T10:04:22.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.810+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.0 v2:172.21.15.32:6800/2392056658 13 ==== osd_op_reply(47 [pgnls-filter start_epoch 113 out=49b] v111'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807000 2024-04-17T10:04:22.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.810+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:48 17.16 17:68000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad807400 con 0x5634ad807800 2024-04-17T10:04:22.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.810+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.5 v2:172.21.15.144:6800/2379578648 4 ==== osd_op_reply(48 [pgnls-filter start_epoch 113 out=49b] v111'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad807800 2024-04-17T10:04:22.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.810+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:49 17.36 17:6c000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad927c00 con 0x5634ad807000 2024-04-17T10:04:22.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.0 v2:172.21.15.32:6800/2392056658 14 ==== osd_op_reply(49 [pgnls-filter start_epoch 113 out=81b] v113'34 uv34 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807000 2024-04-17T10:04:22.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4904ec08c0 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-04-17T10:04:22.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4904ec08c0 10 datascan.operator(): handling object 100000001f7.0 2024-04-17T10:04:22.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:50 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5634ad905000 con 0x5634ad807000 2024-04-17T10:04:22.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.0 v2:172.21.15.32:6800/2392056658 15 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) ==== 416+0+194 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807000 2024-04-17T10:04:22.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4904ec08c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-04-17T10:04:22.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4904ec08c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-04-17T10:04:22.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4904ec08c0 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-04-17T10:04:22.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:51 16.7 16:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5634ad905400 con 0x5634ad926000 2024-04-17T10:04:22.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.3 v2:172.21.15.32:6824/2182899298 12 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad926000 2024-04-17T10:04:22.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4904ec08c0 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-04-17T10:04:22.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x5634ad936000 0x5634ae2df700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:52 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5634ad936400 con 0x5634ad936000 2024-04-17T10:04:22.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4903a00640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x5634ad936000 0x5634ae2df700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4903a00640 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x5634ad936000 0x5634ae2df700 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.2 v2:172.21.15.32:6809/1954833462 1 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=524b] v0'0 uv12 ondisk = 0) ==== 164+0+524 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad936000 2024-04-17T10:04:22.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4904ec08c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-04-17T10:04:22.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4904ec08c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-04-17T10:04:22.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4904ec08c0 20 datascan.get_frag_of: frag is * 2024-04-17T10:04:22.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:53 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5634ad936800 con 0x5634ad926000 2024-04-17T10:04:22.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.3 v2:172.21.15.32:6824/2182899298 13 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad926000 2024-04-17T10:04:22.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4904ec08c0 10 datascan.find_or_create_dirfrag: read_version = 8 2024-04-17T10:04:22.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4904ec08c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-04-17T10:04:22.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:54 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5634ad936c00 con 0x5634ad926000 2024-04-17T10:04:22.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.3 v2:172.21.15.32:6824/2182899298 14 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad926000 2024-04-17T10:04:22.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4904ec08c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-17T10:04:22.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4904ec08c0 20 datascan.read_dentry: bravo_head not found in result 2024-04-17T10:04:22.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4904ec08c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-04-17T10:04:22.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:55 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5634ad937000 con 0x5634ad926000 2024-04-17T10:04:22.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.3 v2:172.21.15.32:6824/2182899298 15 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v113'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad926000 2024-04-17T10:04:22.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4904ec08c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-04-17T10:04:22.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.814+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:56 17.e 17:70000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad937400 con 0x5634ad936000 2024-04-17T10:04:22.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.818+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.2 v2:172.21.15.32:6809/1954833462 2 ==== osd_op_reply(56 [pgnls-filter start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad936000 2024-04-17T10:04:22.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.818+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:57 17.2e 17:74000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad927800 con 0x5634ad904800 2024-04-17T10:04:22.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.818+0000 7f4902600640 1 -- 172.21.15.32:0/3432519519 <== osd.6 v2:172.21.15.144:6808/3198738696 2 ==== osd_op_reply(57 [pgnls-filter start_epoch 113 out=49b] v111'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e780 con 0x5634ad904800 2024-04-17T10:04:22.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.818+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:58 17.1e 17:78000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad806800 con 0x5634ad905800 2024-04-17T10:04:22.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.818+0000 7f4902600640 1 -- 172.21.15.32:0/3432519519 <== osd.1 v2:172.21.15.32:6808/2454993455 6 ==== osd_op_reply(58 [pgnls-filter start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e780 con 0x5634ad905800 2024-04-17T10:04:22.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.818+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:59 17.3e 17:7c000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad927000 con 0x5634ad807c00 2024-04-17T10:04:22.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.818+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.4 v2:172.21.15.144:6809/4076371964 5 ==== osd_op_reply(59 [pgnls-filter start_epoch 113 out=49b] v111'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807c00 2024-04-17T10:04:22.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.818+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:60 17.1 17:80000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ae44a400 con 0x5634ad807000 2024-04-17T10:04:22.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.818+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.0 v2:172.21.15.32:6800/2392056658 16 ==== osd_op_reply(60 [pgnls-filter start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807000 2024-04-17T10:04:22.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.818+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:61 17.21 17:84000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad926c00 con 0x5634ad807000 2024-04-17T10:04:22.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.818+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.0 v2:172.21.15.32:6800/2392056658 17 ==== osd_op_reply(61 [pgnls-filter start_epoch 113 out=49b] v111'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807000 2024-04-17T10:04:22.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.818+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:62 17.11 17:88000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad807400 con 0x5634ad926000 2024-04-17T10:04:22.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.818+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.3 v2:172.21.15.32:6824/2182899298 16 ==== osd_op_reply(62 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad926000 2024-04-17T10:04:22.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.818+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:63 17.31 17:8c000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad904400 con 0x5634ad905800 2024-04-17T10:04:22.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.818+0000 7f4902600640 1 -- 172.21.15.32:0/3432519519 <== osd.1 v2:172.21.15.32:6808/2454993455 7 ==== osd_op_reply(63 [pgnls-filter start_epoch 113 out=49b] v111'15 uv8 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e780 con 0x5634ad905800 2024-04-17T10:04:22.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.818+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:64 17.9 17:90000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad927400 con 0x5634ad926000 2024-04-17T10:04:22.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.818+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.3 v2:172.21.15.32:6824/2182899298 17 ==== osd_op_reply(64 [pgnls-filter start_epoch 113 out=49b] v111'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad926000 2024-04-17T10:04:22.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.818+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:65 17.29 17:94000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ae44a800 con 0x5634ad807c00 2024-04-17T10:04:22.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.822+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.4 v2:172.21.15.144:6809/4076371964 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807c00 2024-04-17T10:04:22.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.822+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:66 17.19 17:98000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad926800 con 0x5634ad807800 2024-04-17T10:04:22.822 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.822+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.5 v2:172.21.15.144:6800/2379578648 5 ==== osd_op_reply(66 [pgnls-filter start_epoch 113 out=49b] v113'24 uv24 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad807800 2024-04-17T10:04:22.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.822+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:67 17.39 17:9c000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad936400 con 0x5634ad807800 2024-04-17T10:04:22.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.822+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.5 v2:172.21.15.144:6800/2379578648 6 ==== osd_op_reply(67 [pgnls-filter start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad807800 2024-04-17T10:04:22.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.822+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad926400 con 0x5634ad904800 2024-04-17T10:04:22.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.822+0000 7f4902600640 1 -- 172.21.15.32:0/3432519519 <== osd.6 v2:172.21.15.144:6808/3198738696 3 ==== osd_op_reply(68 [pgnls-filter start_epoch 113 out=49b] v113'15 uv15 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e780 con 0x5634ad904800 2024-04-17T10:04:22.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.822+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:69 17.25 17:a4000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad936800 con 0x5634ad905800 2024-04-17T10:04:22.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.822+0000 7f4902600640 1 -- 172.21.15.32:0/3432519519 <== osd.1 v2:172.21.15.32:6808/2454993455 8 ==== osd_op_reply(69 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e780 con 0x5634ad905800 2024-04-17T10:04:22.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.822+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:70 17.15 17:a8000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad927c00 con 0x5634ad807800 2024-04-17T10:04:22.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.822+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.5 v2:172.21.15.144:6800/2379578648 7 ==== osd_op_reply(70 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad807800 2024-04-17T10:04:22.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.822+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:71 17.35 17:ac000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad936c00 con 0x5634ad807c00 2024-04-17T10:04:22.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.822+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.4 v2:172.21.15.144:6809/4076371964 7 ==== osd_op_reply(71 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807c00 2024-04-17T10:04:22.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.822+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:72 17.d 17:b0000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad927000 con 0x5634ad904000 2024-04-17T10:04:22.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.822+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.7 v2:172.21.15.144:6824/1746570140 11 ==== osd_op_reply(72 [pgnls-filter start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad904000 2024-04-17T10:04:22.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.822+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:73 17.2d 17:b4000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad937000 con 0x5634ad904000 2024-04-17T10:04:22.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.822+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.7 v2:172.21.15.144:6824/1746570140 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad904000 2024-04-17T10:04:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.822+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:74 17.1d 17:b8000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad926c00 con 0x5634ad904000 2024-04-17T10:04:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.822+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.7 v2:172.21.15.144:6824/1746570140 13 ==== osd_op_reply(74 [pgnls-filter start_epoch 113 out=49b] v111'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad904000 2024-04-17T10:04:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.822+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:75 17.3d 17:bc000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad937400 con 0x5634ad904000 2024-04-17T10:04:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.826+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.7 v2:172.21.15.144:6824/1746570140 14 ==== osd_op_reply(75 [pgnls-filter start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad904000 2024-04-17T10:04:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.826+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:76 17.3 17:c0000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ae44a400 con 0x5634ad807c00 2024-04-17T10:04:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.826+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.4 v2:172.21.15.144:6809/4076371964 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807c00 2024-04-17T10:04:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.826+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:77 17.23 17:c4000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad937c00 con 0x5634ad807c00 2024-04-17T10:04:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.826+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.4 v2:172.21.15.144:6809/4076371964 9 ==== osd_op_reply(77 [pgnls-filter start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807c00 2024-04-17T10:04:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.826+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:78 17.13 17:c8000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad927800 con 0x5634ad926000 2024-04-17T10:04:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.826+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.3 v2:172.21.15.32:6824/2182899298 18 ==== osd_op_reply(78 [pgnls-filter start_epoch 113 out=49b] v111'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad926000 2024-04-17T10:04:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.826+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:79 17.33 17:cc000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad905000 con 0x5634ad904000 2024-04-17T10:04:22.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.826+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.7 v2:172.21.15.144:6824/1746570140 15 ==== osd_op_reply(79 [pgnls-filter start_epoch 113 out=49b] v111'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad904000 2024-04-17T10:04:22.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.826+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:80 17.b 17:d0000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad926400 con 0x5634ad807000 2024-04-17T10:04:22.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.826+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.0 v2:172.21.15.32:6800/2392056658 18 ==== osd_op_reply(80 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807000 2024-04-17T10:04:22.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.826+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:81 17.2b 17:d4000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad904c00 con 0x5634ad807c00 2024-04-17T10:04:22.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.826+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.4 v2:172.21.15.144:6809/4076371964 10 ==== osd_op_reply(81 [pgnls-filter start_epoch 113 out=49b] v111'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807c00 2024-04-17T10:04:22.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.826+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:82 17.1b 17:d8000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad905c00 con 0x5634ad807c00 2024-04-17T10:04:22.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.826+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.4 v2:172.21.15.144:6809/4076371964 11 ==== osd_op_reply(82 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807c00 2024-04-17T10:04:22.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.826+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:83 17.3b 17:dc000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad905400 con 0x5634ad936000 2024-04-17T10:04:22.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.826+0000 7f4903a00640 1 -- 172.21.15.32:0/3432519519 <== osd.2 v2:172.21.15.32:6809/1954833462 3 ==== osd_op_reply(83 [pgnls-filter start_epoch 113 out=49b] v111'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e500 con 0x5634ad936000 2024-04-17T10:04:22.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.826+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:84 17.7 17:e0000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad904400 con 0x5634ad807000 2024-04-17T10:04:22.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.826+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.0 v2:172.21.15.32:6800/2392056658 19 ==== osd_op_reply(84 [pgnls-filter start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807000 2024-04-17T10:04:22.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.826+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:85 17.27 17:e4000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad927000 con 0x5634ad807c00 2024-04-17T10:04:22.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.4 v2:172.21.15.144:6809/4076371964 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 113 out=49b] v111'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807c00 2024-04-17T10:04:22.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:86 17.17 17:e8000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad806800 con 0x5634ad904800 2024-04-17T10:04:22.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4902600640 1 -- 172.21.15.32:0/3432519519 <== osd.6 v2:172.21.15.144:6808/3198738696 4 ==== osd_op_reply(86 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e780 con 0x5634ad904800 2024-04-17T10:04:22.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:87 17.37 17:ec000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad926c00 con 0x5634ad807000 2024-04-17T10:04:22.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.0 v2:172.21.15.32:6800/2392056658 20 ==== osd_op_reply(87 [pgnls-filter start_epoch 113 out=49b] v111'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807000 2024-04-17T10:04:22.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:88 17.f 17:f0000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ae305000 con 0x5634ad905800 2024-04-17T10:04:22.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4902600640 1 -- 172.21.15.32:0/3432519519 <== osd.1 v2:172.21.15.32:6808/2454993455 9 ==== osd_op_reply(88 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e780 con 0x5634ad905800 2024-04-17T10:04:22.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:89 17.2f 17:f4000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad926400 con 0x5634ad807c00 2024-04-17T10:04:22.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.4 v2:172.21.15.144:6809/4076371964 13 ==== osd_op_reply(89 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807c00 2024-04-17T10:04:22.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f48fcc00640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:90 17.1f 17:f8000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad94a000 con 0x5634ad904800 2024-04-17T10:04:22.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4902600640 1 -- 172.21.15.32:0/3432519519 <== osd.6 v2:172.21.15.144:6808/3198738696 5 ==== osd_op_reply(90 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e780 con 0x5634ad904800 2024-04-17T10:04:22.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f48fd600640 1 -- 172.21.15.32:0/3432519519 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:91 17.3f 17:fc000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5634ad905000 con 0x5634ad807000 2024-04-17T10:04:22.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4903000640 1 -- 172.21.15.32:0/3432519519 <== osd.0 v2:172.21.15.32:6800/2392056658 21 ==== osd_op_reply(91 [pgnls-filter start_epoch 113 out=49b] v111'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5634ad61e000 con 0x5634ad807000 2024-04-17T10:04:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x5634ad807000 msgr2=0x5634ae2de680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x5634ad807000 0x5634ae2de680 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x5634ad905800 msgr2=0x5634ad910000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x5634ad905800 0x5634ad910000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x5634ad936000 msgr2=0x5634ae2df700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x5634ad936000 0x5634ae2df700 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x5634ad926000 msgr2=0x5634ad910580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x5634ad926000 0x5634ad910580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x5634ad807c00 msgr2=0x5634ad910b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x5634ad807c00 0x5634ad910b00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x5634ad807800 msgr2=0x5634ae2dec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x5634ad807800 0x5634ae2dec00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x5634ad904800 msgr2=0x5634ad911080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x5634ad904800 0x5634ad911080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x5634ad904000 msgr2=0x5634ae2df180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x5634ad904000 0x5634ae2df180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4902600640 1 -- 172.21.15.32:0/3432519519 reap_dead start 2024-04-17T10:04:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4902600640 1 -- 172.21.15.32:0/3432519519 reap_dead start 2024-04-17T10:04:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4902600640 1 -- 172.21.15.32:0/3432519519 reap_dead start 2024-04-17T10:04:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4902600640 1 -- 172.21.15.32:0/3432519519 reap_dead start 2024-04-17T10:04:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x5634ad806c00 msgr2=0x5634ae2de100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x5634ad806c00 0x5634ae2de100 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x5634ae44e660 tx=0x5634ad5eacc0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634ae44bc00 msgr2=0x5634ae2ddb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634ae44bc00 0x5634ae2ddb80 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x5634ae44f950 tx=0x5634ad5ea120 comp rx=0 tx=0).stop 2024-04-17T10:04:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 shutdown_connections 2024-04-17T10:04:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x5634ad806c00 0x5634ae2de100 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/3432519519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634ae44bc00 0x5634ae2ddb80 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 >> 172.21.15.32:0/3432519519 conn(0x5634ae44b000 msgr2=0x5634ad537800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 shutdown_connections 2024-04-17T10:04:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 -- 172.21.15.32:0/3432519519 wait complete. 2024-04-17T10:04:22.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 -- 172.21.15.32:0/1343628428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634ad509000 msgr2=0x5634ae2dc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/1343628428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634ad509000 0x5634ae2dc000 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x5634ae371470 tx=0x5634ad5eafc0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 -- 172.21.15.32:0/1343628428 shutdown_connections 2024-04-17T10:04:22.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/1343628428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5634ad509800 0x5634ae2dcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/1343628428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634ad509400 0x5634ae2dc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 --2- 172.21.15.32:0/1343628428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5634ad509000 0x5634ae2dc000 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 -- 172.21.15.32:0/1343628428 >> 172.21.15.32:0/1343628428 conn(0x5634ad508c00 msgr2=0x5634ad537000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:22.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 -- 172.21.15.32:0/1343628428 shutdown_connections 2024-04-17T10:04:22.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.830+0000 7f4904ec08c0 1 -- 172.21.15.32:0/1343628428 wait complete. 2024-04-17T10:04:22.834 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.152240, result: 2024-04-17T10:04:22.835 DEBUG:teuthology.orchestra.run.smithi032:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-17T10:04:22.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a7b0cd8c0 1 -- start start 2024-04-17T10:04:22.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a7b0cd8c0 1 Processor -- start 2024-04-17T10:04:22.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a7b0cd8c0 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5626b67bf800 0x5626b758c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a7b0cd8c0 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626b67bf400 0x5626b758c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a7b0cd8c0 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626b67bf000 0x5626b758cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a7b0cd8c0 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5626b7532780 con 0x5626b67bf000 2024-04-17T10:04:22.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a7b0cd8c0 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5626b7532600 con 0x5626b67bf400 2024-04-17T10:04:22.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a7b0cd8c0 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5626b7532480 con 0x5626b67bf800 2024-04-17T10:04:22.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a79c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626b67bf000 0x5626b758cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a79c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626b67bf000 0x5626b758cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60922/0 (socket says 172.21.15.32:60922) 2024-04-17T10:04:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a79c00640 1 -- 172.21.15.32:0/790985359 learned_addr learned my addr 172.21.15.32:0/790985359 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a79200640 1 --2- 172.21.15.32:0/790985359 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5626b67bf800 0x5626b758c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a78800640 1 --2- 172.21.15.32:0/790985359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626b67bf400 0x5626b758c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a79c00640 1 -- 172.21.15.32:0/790985359 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5626b67bf800 msgr2=0x5626b758c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a79c00640 1 --2- 172.21.15.32:0/790985359 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5626b67bf800 0x5626b758c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a79c00640 1 -- 172.21.15.32:0/790985359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626b67bf400 msgr2=0x5626b758c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a79c00640 1 --2- 172.21.15.32:0/790985359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626b67bf400 0x5626b758c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a79c00640 1 -- 172.21.15.32:0/790985359 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626b67e41e0 con 0x5626b67bf000 2024-04-17T10:04:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a79200640 1 --2- 172.21.15.32:0/790985359 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5626b67bf800 0x5626b758c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a79c00640 1 --2- 172.21.15.32:0/790985359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626b67bf000 0x5626b758cb00 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x5626b76190e0 tx=0x5626b689c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9767d749e731803d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a76000640 1 -- 172.21.15.32:0/790985359 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5626b67f49c0 con 0x5626b67bf000 2024-04-17T10:04:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a76000640 1 -- 172.21.15.32:0/790985359 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5626b680a1c0 con 0x5626b67bf000 2024-04-17T10:04:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a76000640 1 -- 172.21.15.32:0/790985359 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5626b67f44e0 con 0x5626b67bf000 2024-04-17T10:04:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/790985359 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626b67e52c0 con 0x5626b67bf000 2024-04-17T10:04:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/790985359 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5626b67e43c0 con 0x5626b67bf000 2024-04-17T10:04:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a7b0cd8c0 4 waiting for MDS map... 2024-04-17T10:04:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a76000640 1 -- 172.21.15.32:0/790985359 <== mon.0 v2:172.21.15.32:3300/0 4 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x5626b76e8200 con 0x5626b67bf000 2024-04-17T10:04:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a76000640 1 -- 172.21.15.32:0/790985359 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 105) ==== 2886+0+0 (secure 0 0 0) 0x5626b6815180 con 0x5626b67bf000 2024-04-17T10:04:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a7b0cd8c0 4 Got MDS map 105 2024-04-17T10:04:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a7b0cd8c0 4 datascan.main: Using metadata pool output 2024-04-17T10:04:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.926+0000 7f1a7b0cd8c0 4 datascan.main: connecting to RADOS... 2024-04-17T10:04:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a7b0cd8c0 1 Processor -- start 2024-04-17T10:04:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a7b0cd8c0 1 -- start start 2024-04-17T10:04:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a7b0cd8c0 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5626b75a9c00 0x5626b758d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a7b0cd8c0 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626b7740000 0x5626b758d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a7b0cd8c0 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626b7740400 0x5626b758db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a7b0cd8c0 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5626b7532300 con 0x5626b7740400 2024-04-17T10:04:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a7b0cd8c0 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5626b7532000 con 0x5626b7740000 2024-04-17T10:04:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a7b0cd8c0 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5626b681e900 con 0x5626b75a9c00 2024-04-17T10:04:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a78800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626b7740400 0x5626b758db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a78800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626b7740400 0x5626b758db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60924/0 (socket says 172.21.15.32:60924) 2024-04-17T10:04:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a78800640 1 -- 172.21.15.32:0/1036581359 learned_addr learned my addr 172.21.15.32:0/1036581359 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a79200640 1 --2- 172.21.15.32:0/1036581359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626b7740000 0x5626b758d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a78800640 1 --2- 172.21.15.32:0/1036581359 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5626b75a9c00 0x5626b758d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a78800640 1 -- 172.21.15.32:0/1036581359 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5626b75a9c00 msgr2=0x5626b758d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a78800640 1 --2- 172.21.15.32:0/1036581359 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5626b75a9c00 0x5626b758d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a78800640 1 -- 172.21.15.32:0/1036581359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626b7740000 msgr2=0x5626b758d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a78800640 1 --2- 172.21.15.32:0/1036581359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626b7740000 0x5626b758d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a78800640 1 -- 172.21.15.32:0/1036581359 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626b67e41e0 con 0x5626b7740400 2024-04-17T10:04:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a78800640 1 --2- 172.21.15.32:0/1036581359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626b7740400 0x5626b758db80 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x5626b76dec30 tx=0x5626b76e4d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ce459fc343cc08 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a73800640 1 -- 172.21.15.32:0/1036581359 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5626b67f49c0 con 0x5626b7740400 2024-04-17T10:04:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a73800640 1 -- 172.21.15.32:0/1036581359 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5626b680ac40 con 0x5626b7740400 2024-04-17T10:04:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a73800640 1 -- 172.21.15.32:0/1036581359 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5626b67f44e0 con 0x5626b7740400 2024-04-17T10:04:22.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/1036581359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626b7740400 msgr2=0x5626b758db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a7b0cd8c0 1 --2- 172.21.15.32:0/1036581359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626b7740400 0x5626b758db80 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x5626b76dec30 tx=0x5626b76e4d80 comp rx=0 tx=0).stop 2024-04-17T10:04:22.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/1036581359 shutdown_connections 2024-04-17T10:04:22.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a7b0cd8c0 1 --2- 172.21.15.32:0/1036581359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626b7740400 0x5626b758db80 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a7b0cd8c0 1 --2- 172.21.15.32:0/1036581359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626b7740000 0x5626b758d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a7b0cd8c0 1 --2- 172.21.15.32:0/1036581359 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5626b75a9c00 0x5626b758d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/1036581359 >> 172.21.15.32:0/1036581359 conn(0x5626b67bec00 msgr2=0x5626b67ed800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:22.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/1036581359 shutdown_connections 2024-04-17T10:04:22.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/1036581359 wait complete. 2024-04-17T10:04:22.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a7b0cd8c0 1 Processor -- start 2024-04-17T10:04:22.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.930+0000 7f1a7b0cd8c0 1 -- start start 2024-04-17T10:04:22.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a7b0cd8c0 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626b75a9c00 0x5626b758d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a7b0cd8c0 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5626b7741800 0x5626b758d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a7b0cd8c0 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626b7741400 0x5626b758db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a7b0cd8c0 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5626b7532000 con 0x5626b7741400 2024-04-17T10:04:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a7b0cd8c0 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5626b7533500 con 0x5626b75a9c00 2024-04-17T10:04:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a7b0cd8c0 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5626b7532300 con 0x5626b7741800 2024-04-17T10:04:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a78800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626b75a9c00 0x5626b758d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a78800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626b7741400 0x5626b758db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a79200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5626b7741800 0x5626b758d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a78800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626b7741400 0x5626b758db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60932/0 (socket says 172.21.15.32:60932) 2024-04-17T10:04:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 learned_addr learned my addr 172.21.15.32:0/4286105612 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5626b7741800 msgr2=0x5626b758d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a78800640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5626b7741800 0x5626b758d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626b75a9c00 msgr2=0x5626b758d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a78800640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626b75a9c00 0x5626b758d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626b67e54a0 con 0x5626b7741400 2024-04-17T10:04:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a78800640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626b7741400 0x5626b758db80 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x5626b76df8f0 tx=0x5626b689cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a71a00640 1 -- 172.21.15.32:0/4286105612 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5626b67f4000 con 0x5626b7741400 2024-04-17T10:04:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a71a00640 1 -- 172.21.15.32:0/4286105612 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5626b680aa80 con 0x5626b7741400 2024-04-17T10:04:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a71a00640 1 -- 172.21.15.32:0/4286105612 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5626b67f49c0 con 0x5626b7741400 2024-04-17T10:04:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5626b67e5a40 con 0x5626b7741400 2024-04-17T10:04:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626b67e5c20 con 0x5626b7741400 2024-04-17T10:04:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a7b0cd8c0 4 datascan.init: resolving metadata pool 16 2024-04-17T10:04:22.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a71a00640 1 -- 172.21.15.32:0/4286105612 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x5626b6aaac00 con 0x5626b7741400 2024-04-17T10:04:22.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a71a00640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x5626b7746800 0x5626b758e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.934+0000 7f1a71a00640 1 -- 172.21.15.32:0/4286105612 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x5626b6aa8a00 con 0x5626b7741400 2024-04-17T10:04:22.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.938+0000 7f1a7b0cd8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-17T10:04:22.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.938+0000 7f1a7b0cd8c0 4 datascan.main: resolving metadata pool 16 2024-04-17T10:04:22.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.938+0000 7f1a7b0cd8c0 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x5626b7741000 0x5626b758e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.938+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:1 16.0 16:00000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740c00 con 0x5626b7741000 2024-04-17T10:04:22.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.938+0000 7f1a78800640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x5626b7741000 0x5626b758e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.938+0000 7f1a79200640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x5626b7746800 0x5626b758e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.938+0000 7f1a78800640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x5626b7741000 0x5626b758e680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.938+0000 7f1a79200640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x5626b7746800 0x5626b758e100 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x5626b689f080 tx=0x5626b76e5020 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.938+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.0 v2:172.21.15.32:6800/2392056658 1 ==== osd_op_reply(1 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c000 con 0x5626b7741000 2024-04-17T10:04:22.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.938+0000 7f1a78800640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x5626b7740c00 0x5626b6bba000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.938+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:2 16.10 16:08000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7747800 con 0x5626b7740c00 2024-04-17T10:04:22.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.938+0000 7f1a79c00640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x5626b7740c00 0x5626b6bba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.938+0000 7f1a79c00640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x5626b7740c00 0x5626b6bba000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.938+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 1 ==== osd_op_reply(2 [pgnls start_epoch 113 out=177b] v112'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.938+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:3 16.8 16:10000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7747800 con 0x5626b7741000 2024-04-17T10:04:22.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.938+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.0 v2:172.21.15.32:6800/2392056658 2 ==== osd_op_reply(3 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c000 con 0x5626b7741000 2024-04-17T10:04:22.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.938+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:4 16.18 16:18000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7747800 con 0x5626b7741000 2024-04-17T10:04:22.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.938+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.0 v2:172.21.15.32:6800/2392056658 3 ==== osd_op_reply(4 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c000 con 0x5626b7741000 2024-04-17T10:04:22.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.938+0000 7f1a78800640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x5626b7747800 0x5626b6bba580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.938+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:5 16.4 16:20000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7747c00 con 0x5626b7747800 2024-04-17T10:04:22.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.938+0000 7f1a79200640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x5626b7747800 0x5626b6bba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.938+0000 7f1a79200640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x5626b7747800 0x5626b6bba580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.942+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 <== osd.3 v2:172.21.15.32:6824/2182899298 1 ==== osd_op_reply(5 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c780 con 0x5626b7747800 2024-04-17T10:04:22.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.942+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:6 16.14 16:28000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7747c00 con 0x5626b7741000 2024-04-17T10:04:22.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.942+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.0 v2:172.21.15.32:6800/2392056658 4 ==== osd_op_reply(6 [pgnls start_epoch 113 out=149b] v113'43 uv43 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x5626b688c000 con 0x5626b7741000 2024-04-17T10:04:22.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.942+0000 7f1a78800640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x5626b7747c00 0x5626b6bbab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.942+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:7 16.c 16:30000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc2000 con 0x5626b7747c00 2024-04-17T10:04:22.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.942+0000 7f1a78800640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x5626b7747c00 0x5626b6bbab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.942+0000 7f1a78800640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x5626b7747c00 0x5626b6bbab00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.942+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.5 v2:172.21.15.144:6800/2379578648 1 ==== osd_op_reply(7 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c000 con 0x5626b7747c00 2024-04-17T10:04:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.942+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:8 16.1c 16:38000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc2000 con 0x5626b7740c00 2024-04-17T10:04:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.942+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 2 ==== osd_op_reply(8 [pgnls start_epoch 113 out=73b] v112'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.942+0000 7f1a79c00640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x5626b6bc2000 0x5626b6bbb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.942+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:9 16.2 16:40000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc3000 con 0x5626b6bc2000 2024-04-17T10:04:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.942+0000 7f1a79c00640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x5626b6bc2000 0x5626b6bbb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.942+0000 7f1a79c00640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x5626b6bc2000 0x5626b6bbb080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.942+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.4 v2:172.21.15.144:6809/4076371964 1 ==== osd_op_reply(9 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c500 con 0x5626b6bc2000 2024-04-17T10:04:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.942+0000 7f1a79c00640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x5626b6bc3000 0x5626b6bbb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.942+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:10 16.12 16:48000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc3400 con 0x5626b6bc3000 2024-04-17T10:04:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.942+0000 7f1a79200640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x5626b6bc3000 0x5626b6bbb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.946+0000 7f1a79200640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x5626b6bc3000 0x5626b6bbb600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.946+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 <== osd.6 v2:172.21.15.144:6808/3198738696 1 ==== osd_op_reply(10 [pgnls start_epoch 113 out=81b] v112'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5626b688c780 con 0x5626b6bc3000 2024-04-17T10:04:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.946+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:11 16.a 16:50000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc3400 con 0x5626b7740c00 2024-04-17T10:04:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.946+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 3 ==== osd_op_reply(11 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.946+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:12 16.1a 16:58000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc3400 con 0x5626b7747800 2024-04-17T10:04:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.946+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 <== osd.3 v2:172.21.15.32:6824/2182899298 2 ==== osd_op_reply(12 [pgnls start_epoch 113 out=49b] v113'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c780 con 0x5626b7747800 2024-04-17T10:04:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.946+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:13 16.6 16:60000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc3400 con 0x5626b6bc3000 2024-04-17T10:04:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.946+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 <== osd.6 v2:172.21.15.144:6808/3198738696 2 ==== osd_op_reply(13 [pgnls start_epoch 113 out=77b] v112'9 uv9 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x5626b688c780 con 0x5626b6bc3000 2024-04-17T10:04:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.946+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:14 16.16 16:68000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc3400 con 0x5626b7747800 2024-04-17T10:04:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.946+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 <== osd.3 v2:172.21.15.32:6824/2182899298 3 ==== osd_op_reply(14 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c780 con 0x5626b7747800 2024-04-17T10:04:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.946+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:15 16.e 16:70000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc3400 con 0x5626b7740c00 2024-04-17T10:04:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.946+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 4 ==== osd_op_reply(15 [pgnls start_epoch 113 out=73b] v112'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.946+0000 7f1a79c00640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x5626b6bc3400 0x5626b6bbbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.946+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:16 16.1e 16:78000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc3800 con 0x5626b6bc3400 2024-04-17T10:04:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.946+0000 7f1a78800640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x5626b6bc3400 0x5626b6bbbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.946+0000 7f1a78800640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x5626b6bc3400 0x5626b6bbbb80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.946+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.7 v2:172.21.15.144:6824/1746570140 1 ==== osd_op_reply(16 [pgnls start_epoch 113 out=49b] v113'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c000 con 0x5626b6bc3400 2024-04-17T10:04:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.946+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:17 16.1 16:80000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc3800 con 0x5626b7740c00 2024-04-17T10:04:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.946+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 5 ==== osd_op_reply(17 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.950+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:18 16.11 16:88000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc3800 con 0x5626b7747c00 2024-04-17T10:04:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.950+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.5 v2:172.21.15.144:6800/2379578648 2 ==== osd_op_reply(18 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c000 con 0x5626b7747c00 2024-04-17T10:04:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.950+0000 7f1a78800640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x5626b6bc3800 0x5626b6bbc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.950+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:19 16.9 16:90000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc2400 con 0x5626b6bc3800 2024-04-17T10:04:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.950+0000 7f1a79c00640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x5626b6bc3800 0x5626b6bbc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.950+0000 7f1a79c00640 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x5626b6bc3800 0x5626b6bbc100 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.950+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.1 v2:172.21.15.32:6808/2454993455 1 ==== osd_op_reply(19 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c500 con 0x5626b6bc3800 2024-04-17T10:04:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.950+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:20 16.19 16:98000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc2400 con 0x5626b6bc2000 2024-04-17T10:04:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.950+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.4 v2:172.21.15.144:6809/4076371964 2 ==== osd_op_reply(20 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c500 con 0x5626b6bc2000 2024-04-17T10:04:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.950+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:21 16.5 16:a0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc2400 con 0x5626b7741000 2024-04-17T10:04:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.950+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.0 v2:172.21.15.32:6800/2392056658 5 ==== osd_op_reply(21 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c000 con 0x5626b7741000 2024-04-17T10:04:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.950+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:22 16.15 16:a8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc2400 con 0x5626b7741000 2024-04-17T10:04:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.950+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.0 v2:172.21.15.32:6800/2392056658 6 ==== osd_op_reply(22 [pgnls start_epoch 113 out=97b] v112'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x5626b688c000 con 0x5626b7741000 2024-04-17T10:04:22.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.950+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:23 16.d 16:b0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc2400 con 0x5626b6bc3000 2024-04-17T10:04:22.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.950+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 <== osd.6 v2:172.21.15.144:6808/3198738696 3 ==== osd_op_reply(23 [pgnls start_epoch 113 out=74b] v108'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x5626b688c780 con 0x5626b6bc3000 2024-04-17T10:04:22.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.950+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:24 16.1d 16:b8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc2400 con 0x5626b7740c00 2024-04-17T10:04:22.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.950+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 6 ==== osd_op_reply(24 [pgnls start_epoch 113 out=73b] v112'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.950+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:25 16.3 16:c0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc2400 con 0x5626b7747c00 2024-04-17T10:04:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.950+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.5 v2:172.21.15.144:6800/2379578648 3 ==== osd_op_reply(25 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c000 con 0x5626b7747c00 2024-04-17T10:04:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.950+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:26 16.13 16:c8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc2400 con 0x5626b6bc3800 2024-04-17T10:04:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.1 v2:172.21.15.32:6808/2454993455 2 ==== osd_op_reply(26 [pgnls start_epoch 113 out=98b] v112'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x5626b688c500 con 0x5626b6bc3800 2024-04-17T10:04:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:27 16.b 16:d0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc2400 con 0x5626b6bc3800 2024-04-17T10:04:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.1 v2:172.21.15.32:6808/2454993455 3 ==== osd_op_reply(27 [pgnls start_epoch 113 out=76b] v113'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x5626b688c500 con 0x5626b6bc3800 2024-04-17T10:04:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:28 16.1b 16:d8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc2400 con 0x5626b7747c00 2024-04-17T10:04:22.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.5 v2:172.21.15.144:6800/2379578648 4 ==== osd_op_reply(28 [pgnls start_epoch 113 out=49b] v113'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c000 con 0x5626b7747c00 2024-04-17T10:04:22.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:29 16.7 16:e0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc2400 con 0x5626b7747800 2024-04-17T10:04:22.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 <== osd.3 v2:172.21.15.32:6824/2182899298 4 ==== osd_op_reply(29 [pgnls start_epoch 113 out=105b] v113'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x5626b688c780 con 0x5626b7747800 2024-04-17T10:04:22.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:30 16.17 16:e8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc2400 con 0x5626b7740c00 2024-04-17T10:04:22.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 7 ==== osd_op_reply(30 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:31 16.f 16:f0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc2400 con 0x5626b7741000 2024-04-17T10:04:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.0 v2:172.21.15.32:6800/2392056658 7 ==== osd_op_reply(31 [pgnls start_epoch 113 out=73b] v113'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5626b688c000 con 0x5626b7741000 2024-04-17T10:04:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:32 16.1f 16:f8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc2400 con 0x5626b7747c00 2024-04-17T10:04:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.5 v2:172.21.15.144:6800/2379578648 5 ==== osd_op_reply(32 [pgnls start_epoch 113 out=101b] v113'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x5626b688c000 con 0x5626b7747c00 2024-04-17T10:04:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-17T10:04:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:33 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740800 con 0x5626b7740c00 2024-04-17T10:04:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 8 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-17T10:04:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:34 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740400 con 0x5626b7740c00 2024-04-17T10:04:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 9 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-17T10:04:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:35 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7740c00 2024-04-17T10:04:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 10 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) ==== 164+0+1047 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-17T10:04:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:36 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bdc000 con 0x5626b7740c00 2024-04-17T10:04:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 11 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-17T10:04:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.954+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:37 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bdc400 con 0x5626b7740c00 2024-04-17T10:04:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 12 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-17T10:04:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-17T10:04:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-17T10:04:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-17T10:04:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-17T10:04:22.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-17T10:04:22.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-17T10:04:22.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:38 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bdc800 con 0x5626b7741000 2024-04-17T10:04:22.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.0 v2:172.21.15.32:6800/2392056658 8 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv38 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5626b688c000 con 0x5626b7741000 2024-04-17T10:04:22.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-17T10:04:22.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:39 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bdcc00 con 0x5626b7740c00 2024-04-17T10:04:22.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 13 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-04-17T10:04:22.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:40 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bdd000 con 0x5626b6bc3000 2024-04-17T10:04:22.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 <== osd.6 v2:172.21.15.144:6808/3198738696 4 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) ==== 164+0+523 (crc 0 0 0) 0x5626b688c780 con 0x5626b6bc3000 2024-04-17T10:04:22.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-17T10:04:22.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-17T10:04:22.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-17T10:04:22.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:41 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bdd400 con 0x5626b7740c00 2024-04-17T10:04:22.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 14 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-17T10:04:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:42 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bdd800 con 0x5626b7741000 2024-04-17T10:04:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.0 v2:172.21.15.32:6800/2392056658 9 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5626b688c000 con 0x5626b7741000 2024-04-17T10:04:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-17T10:04:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-17T10:04:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-17T10:04:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-17T10:04:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-17T10:04:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:43 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bddc00 con 0x5626b7740c00 2024-04-17T10:04:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 15 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-17T10:04:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-17T10:04:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-17T10:04:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:44 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bdc000 con 0x5626b6bc3800 2024-04-17T10:04:22.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.1 v2:172.21.15.32:6808/2454993455 4 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x5626b688c500 con 0x5626b6bc3800 2024-04-17T10:04:22.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-17T10:04:22.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-17T10:04:22.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-17T10:04:22.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:45 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bdc400 con 0x5626b7747800 2024-04-17T10:04:22.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 <== osd.3 v2:172.21.15.32:6824/2182899298 5 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5626b688c780 con 0x5626b7747800 2024-04-17T10:04:22.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-04-17T10:04:22.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:46 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740800 con 0x5626b7747800 2024-04-17T10:04:22.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 <== osd.3 v2:172.21.15.32:6824/2182899298 6 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) ==== 164+0+1545 (crc 0 0 0) 0x5626b688c780 con 0x5626b7747800 2024-04-17T10:04:22.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 20 datascan.scan_links: injected ino detected 2024-04-17T10:04:22.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-17T10:04:22.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-17T10:04:22.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-17T10:04:22.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-17T10:04:22.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-17T10:04:22.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:47 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740400 con 0x5626b7747c00 2024-04-17T10:04:22.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.5 v2:172.21.15.144:6800/2379578648 6 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) ==== 154+0+524 (crc 0 0 0) 0x5626b688c000 con 0x5626b7747c00 2024-04-17T10:04:22.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.958+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:48 16.0 16:00000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7741000 2024-04-17T10:04:22.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.962+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.0 v2:172.21.15.32:6800/2392056658 10 ==== osd_op_reply(48 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c000 con 0x5626b7741000 2024-04-17T10:04:22.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.962+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:49 16.10 16:08000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7740c00 2024-04-17T10:04:22.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.962+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 16 ==== osd_op_reply(49 [pgnls start_epoch 113 out=177b] v112'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.962+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:50 16.8 16:10000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7741000 2024-04-17T10:04:22.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.962+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.0 v2:172.21.15.32:6800/2392056658 11 ==== osd_op_reply(50 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c000 con 0x5626b7741000 2024-04-17T10:04:22.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.962+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:51 16.18 16:18000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7741000 2024-04-17T10:04:22.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.962+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.0 v2:172.21.15.32:6800/2392056658 12 ==== osd_op_reply(51 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c000 con 0x5626b7741000 2024-04-17T10:04:22.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.962+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:52 16.4 16:20000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7747800 2024-04-17T10:04:22.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.962+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 <== osd.3 v2:172.21.15.32:6824/2182899298 7 ==== osd_op_reply(52 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c780 con 0x5626b7747800 2024-04-17T10:04:22.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.962+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:53 16.14 16:28000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7741000 2024-04-17T10:04:22.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.962+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.0 v2:172.21.15.32:6800/2392056658 13 ==== osd_op_reply(53 [pgnls start_epoch 113 out=149b] v113'43 uv43 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x5626b688c000 con 0x5626b7741000 2024-04-17T10:04:22.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.962+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:54 16.c 16:30000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7747c00 2024-04-17T10:04:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.962+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.5 v2:172.21.15.144:6800/2379578648 7 ==== osd_op_reply(54 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c000 con 0x5626b7747c00 2024-04-17T10:04:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.962+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:55 16.1c 16:38000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7740c00 2024-04-17T10:04:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.962+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 17 ==== osd_op_reply(55 [pgnls start_epoch 113 out=73b] v112'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.962+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:56 16.2 16:40000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b6bc2000 2024-04-17T10:04:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.962+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.4 v2:172.21.15.144:6809/4076371964 3 ==== osd_op_reply(56 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c500 con 0x5626b6bc2000 2024-04-17T10:04:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.962+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:57 16.12 16:48000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b6bc3000 2024-04-17T10:04:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.962+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 <== osd.6 v2:172.21.15.144:6808/3198738696 5 ==== osd_op_reply(57 [pgnls start_epoch 113 out=81b] v112'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5626b688c780 con 0x5626b6bc3000 2024-04-17T10:04:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.962+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:58 16.a 16:50000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7740c00 2024-04-17T10:04:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.962+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 18 ==== osd_op_reply(58 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.962+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:59 16.1a 16:58000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7747800 2024-04-17T10:04:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.966+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 <== osd.3 v2:172.21.15.32:6824/2182899298 8 ==== osd_op_reply(59 [pgnls start_epoch 113 out=49b] v113'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c780 con 0x5626b7747800 2024-04-17T10:04:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.966+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:60 16.6 16:60000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b6bc3000 2024-04-17T10:04:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.966+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 <== osd.6 v2:172.21.15.144:6808/3198738696 6 ==== osd_op_reply(60 [pgnls start_epoch 113 out=77b] v112'9 uv9 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x5626b688c780 con 0x5626b6bc3000 2024-04-17T10:04:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.966+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:61 16.16 16:68000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7747800 2024-04-17T10:04:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.966+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 <== osd.3 v2:172.21.15.32:6824/2182899298 9 ==== osd_op_reply(61 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c780 con 0x5626b7747800 2024-04-17T10:04:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.966+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:62 16.e 16:70000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7740c00 2024-04-17T10:04:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.966+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 19 ==== osd_op_reply(62 [pgnls start_epoch 113 out=73b] v112'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.966+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- osd_op(unknown.0.0:63 16.1e 16:78000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b6bc3400 2024-04-17T10:04:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.966+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.7 v2:172.21.15.144:6824/1746570140 2 ==== osd_op_reply(63 [pgnls start_epoch 113 out=49b] v113'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c000 con 0x5626b6bc3400 2024-04-17T10:04:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.966+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:64 16.1 16:80000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7740c00 2024-04-17T10:04:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.966+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 20 ==== osd_op_reply(64 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.966+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:65 16.11 16:88000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7747c00 2024-04-17T10:04:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.966+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.5 v2:172.21.15.144:6800/2379578648 8 ==== osd_op_reply(65 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c000 con 0x5626b7747c00 2024-04-17T10:04:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.966+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:66 16.9 16:90000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b6bc3800 2024-04-17T10:04:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.966+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.1 v2:172.21.15.32:6808/2454993455 5 ==== osd_op_reply(66 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c500 con 0x5626b6bc3800 2024-04-17T10:04:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.966+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- osd_op(unknown.0.0:67 16.19 16:98000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b6bc2000 2024-04-17T10:04:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.966+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.4 v2:172.21.15.144:6809/4076371964 4 ==== osd_op_reply(67 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c500 con 0x5626b6bc2000 2024-04-17T10:04:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.966+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:68 16.5 16:a0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7741000 2024-04-17T10:04:22.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.966+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.0 v2:172.21.15.32:6800/2392056658 14 ==== osd_op_reply(68 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c000 con 0x5626b7741000 2024-04-17T10:04:22.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.966+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:69 16.15 16:a8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7741000 2024-04-17T10:04:22.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.966+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.0 v2:172.21.15.32:6800/2392056658 15 ==== osd_op_reply(69 [pgnls start_epoch 113 out=97b] v112'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x5626b688c000 con 0x5626b7741000 2024-04-17T10:04:22.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.966+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:70 16.d 16:b0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b6bc3000 2024-04-17T10:04:22.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.970+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 <== osd.6 v2:172.21.15.144:6808/3198738696 7 ==== osd_op_reply(70 [pgnls start_epoch 113 out=74b] v108'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x5626b688c780 con 0x5626b6bc3000 2024-04-17T10:04:22.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.970+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:71 16.1d 16:b8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7740c00 2024-04-17T10:04:22.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.970+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 21 ==== osd_op_reply(71 [pgnls start_epoch 113 out=73b] v112'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.970+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:72 16.3 16:c0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7747c00 2024-04-17T10:04:22.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.970+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.5 v2:172.21.15.144:6800/2379578648 9 ==== osd_op_reply(72 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c000 con 0x5626b7747c00 2024-04-17T10:04:22.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.970+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:73 16.13 16:c8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b6bc3800 2024-04-17T10:04:22.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.970+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.1 v2:172.21.15.32:6808/2454993455 6 ==== osd_op_reply(73 [pgnls start_epoch 113 out=98b] v112'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x5626b688c500 con 0x5626b6bc3800 2024-04-17T10:04:22.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.970+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:74 16.b 16:d0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b6bc3800 2024-04-17T10:04:22.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.970+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.1 v2:172.21.15.32:6808/2454993455 7 ==== osd_op_reply(74 [pgnls start_epoch 113 out=76b] v113'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x5626b688c500 con 0x5626b6bc3800 2024-04-17T10:04:22.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.970+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:75 16.1b 16:d8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7747c00 2024-04-17T10:04:22.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.970+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.5 v2:172.21.15.144:6800/2379578648 10 ==== osd_op_reply(75 [pgnls start_epoch 113 out=49b] v113'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c000 con 0x5626b7747c00 2024-04-17T10:04:22.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.970+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:76 16.7 16:e0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7747800 2024-04-17T10:04:22.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.970+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 <== osd.3 v2:172.21.15.32:6824/2182899298 10 ==== osd_op_reply(76 [pgnls start_epoch 113 out=105b] v113'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x5626b688c780 con 0x5626b7747800 2024-04-17T10:04:22.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.970+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:77 16.17 16:e8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7740c00 2024-04-17T10:04:22.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.970+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 22 ==== osd_op_reply(77 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.970+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:78 16.f 16:f0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7741000 2024-04-17T10:04:22.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.970+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.0 v2:172.21.15.32:6800/2392056658 16 ==== osd_op_reply(78 [pgnls start_epoch 113 out=73b] v113'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5626b688c000 con 0x5626b7741000 2024-04-17T10:04:22.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.970+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:79 16.1f 16:f8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740000 con 0x5626b7747c00 2024-04-17T10:04:22.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.970+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.5 v2:172.21.15.144:6800/2379578648 11 ==== osd_op_reply(79 [pgnls start_epoch 113 out=101b] v113'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x5626b688c000 con 0x5626b7747c00 2024-04-17T10:04:22.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.970+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-17T10:04:22.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.970+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:80 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc3c00 con 0x5626b7740c00 2024-04-17T10:04:22.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 23 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-17T10:04:22.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:81 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740400 con 0x5626b7740c00 2024-04-17T10:04:22.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 24 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-17T10:04:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:82 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740800 con 0x5626b7740c00 2024-04-17T10:04:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 25 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) ==== 164+0+1047 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-17T10:04:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:83 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bdc800 con 0x5626b7740c00 2024-04-17T10:04:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 26 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-17T10:04:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:84 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bdd800 con 0x5626b7740c00 2024-04-17T10:04:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 27 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-17T10:04:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-17T10:04:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-17T10:04:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-17T10:04:22.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-17T10:04:22.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-17T10:04:22.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-17T10:04:22.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:85 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bdd000 con 0x5626b7741000 2024-04-17T10:04:22.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.0 v2:172.21.15.32:6800/2392056658 17 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv38 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5626b688c000 con 0x5626b7741000 2024-04-17T10:04:22.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-17T10:04:22.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:86 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bdc400 con 0x5626b7740c00 2024-04-17T10:04:22.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 28 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-04-17T10:04:22.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:87 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b7747400 con 0x5626b6bc3000 2024-04-17T10:04:22.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 <== osd.6 v2:172.21.15.144:6808/3198738696 8 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) ==== 164+0+523 (crc 0 0 0) 0x5626b688c780 con 0x5626b6bc3000 2024-04-17T10:04:22.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-17T10:04:22.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-17T10:04:22.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-17T10:04:22.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:88 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc2c00 con 0x5626b7740c00 2024-04-17T10:04:22.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 29 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-17T10:04:22.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- osd_op(unknown.0.0:89 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc2800 con 0x5626b7741000 2024-04-17T10:04:22.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.0 v2:172.21.15.32:6800/2392056658 18 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5626b688c000 con 0x5626b7741000 2024-04-17T10:04:22.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-17T10:04:22.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-17T10:04:22.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-17T10:04:22.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-17T10:04:22.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-17T10:04:22.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- osd_op(unknown.0.0:90 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc2400 con 0x5626b7740c00 2024-04-17T10:04:22.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.2 v2:172.21.15.32:6809/1954833462 30 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5626b688c500 con 0x5626b7740c00 2024-04-17T10:04:22.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-17T10:04:22.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-17T10:04:22.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-17T10:04:22.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.974+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:91 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740400 con 0x5626b6bc3800 2024-04-17T10:04:22.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.1 v2:172.21.15.32:6808/2454993455 8 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x5626b688c500 con 0x5626b6bc3800 2024-04-17T10:04:22.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-17T10:04:22.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-17T10:04:22.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-17T10:04:22.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:92 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b7740800 con 0x5626b7747800 2024-04-17T10:04:22.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 <== osd.3 v2:172.21.15.32:6824/2182899298 11 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5626b688c780 con 0x5626b7747800 2024-04-17T10:04:22.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-04-17T10:04:22.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:93 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bc3c00 con 0x5626b7747800 2024-04-17T10:04:22.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 <== osd.3 v2:172.21.15.32:6824/2182899298 12 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) ==== 164+0+1545 (crc 0 0 0) 0x5626b688c780 con 0x5626b7747800 2024-04-17T10:04:22.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 20 datascan.scan_links: injected ino detected 2024-04-17T10:04:22.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-04-17T10:04:22.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-17T10:04:22.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-17T10:04:22.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-17T10:04:22.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-17T10:04:22.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-17T10:04:22.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- osd_op(unknown.0.0:94 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bdcc00 con 0x5626b7747c00 2024-04-17T10:04:22.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a78800640 1 -- 172.21.15.32:0/4286105612 <== osd.5 v2:172.21.15.144:6800/2379578648 12 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) ==== 154+0+524 (crc 0 0 0) 0x5626b688c000 con 0x5626b7747c00 2024-04-17T10:04:22.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-04-17T10:04:22.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-17T10:04:22.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-04-17T10:04:22.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 10 datascan.scan_links: processing 1 injected_inos 2024-04-17T10:04:22.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-04-17T10:04:22.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-04-17T10:04:22.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:95 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bdd400 con 0x5626b7747800 2024-04-17T10:04:22.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 <== osd.3 v2:172.21.15.32:6824/2182899298 13 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x5626b688c780 con 0x5626b7747800 2024-04-17T10:04:22.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-17T10:04:22.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 20 datascan.scan_links: first is now 2 2024-04-17T10:04:22.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- osd_op(unknown.0.0:96 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bddc00 con 0x5626b7747800 2024-04-17T10:04:22.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 <== osd.3 v2:172.21.15.32:6824/2182899298 14 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v113'10 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5626b688c780 con 0x5626b7747800 2024-04-17T10:04:22.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-04-17T10:04:22.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 10 datascan.scan_links: updating inotable 2024-04-17T10:04:22.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:97 16.13 16:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bdc000 con 0x5626b6bc3800 2024-04-17T10:04:22.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.1 v2:172.21.15.32:6808/2454993455 9 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x5626b688c500 con 0x5626b6bc3800 2024-04-17T10:04:22.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-04-17T10:04:22.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-04-17T10:04:22.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.978+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- osd_op(unknown.0.0:98 16.13 16:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bdd400 con 0x5626b6bc3800 2024-04-17T10:04:22.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a79c00640 1 -- 172.21.15.32:0/4286105612 <== osd.1 v2:172.21.15.32:6808/2454993455 10 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v113'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x5626b688c500 con 0x5626b6bc3800 2024-04-17T10:04:22.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 10 datascan.scan_links: updating snaptable 2024-04-17T10:04:22.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- osd_op(unknown.0.0:99 16.d 16:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x5626b6bddc00 con 0x5626b6bc3000 2024-04-17T10:04:22.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 <== osd.6 v2:172.21.15.144:6808/3198738696 9 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x5626b688c780 con 0x5626b6bc3000 2024-04-17T10:04:22.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x5626b7741000 msgr2=0x5626b758e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x5626b7741000 0x5626b758e680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x5626b6bc3800 msgr2=0x5626b6bbc100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x5626b6bc3800 0x5626b6bbc100 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x5626b7740c00 msgr2=0x5626b6bba000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x5626b7740c00 0x5626b6bba000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x5626b7747800 msgr2=0x5626b6bba580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x5626b7747800 0x5626b6bba580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 reap_dead start 2024-04-17T10:04:22.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x5626b6bc2000 msgr2=0x5626b6bbb080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x5626b6bc2000 0x5626b6bbb080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x5626b7747c00 msgr2=0x5626b6bbab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x5626b7747c00 0x5626b6bbab00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x5626b6bc3000 msgr2=0x5626b6bbb600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x5626b6bc3000 0x5626b6bbb600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x5626b6bc3400 msgr2=0x5626b6bbbb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x5626b6bc3400 0x5626b6bbbb80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a79200640 1 -- 172.21.15.32:0/4286105612 reap_dead start 2024-04-17T10:04:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x5626b7746800 msgr2=0x5626b758e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x5626b7746800 0x5626b758e100 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x5626b689f080 tx=0x5626b76e5020 comp rx=0 tx=0).stop 2024-04-17T10:04:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626b7741400 msgr2=0x5626b758db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626b7741400 0x5626b758db80 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x5626b76df8f0 tx=0x5626b689cae0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 shutdown_connections 2024-04-17T10:04:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x5626b7746800 0x5626b758e100 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 --2- 172.21.15.32:0/4286105612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626b7741400 0x5626b758db80 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 >> 172.21.15.32:0/4286105612 conn(0x5626b67bec00 msgr2=0x5626b67ed800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:22.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 shutdown_connections 2024-04-17T10:04:22.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/4286105612 wait complete. 2024-04-17T10:04:22.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/790985359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626b67bf000 msgr2=0x5626b758cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:22.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 --2- 172.21.15.32:0/790985359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626b67bf000 0x5626b758cb00 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x5626b76190e0 tx=0x5626b689c720 comp rx=0 tx=0).stop 2024-04-17T10:04:22.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/790985359 shutdown_connections 2024-04-17T10:04:22.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 --2- 172.21.15.32:0/790985359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5626b67bf000 0x5626b758cb00 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 --2- 172.21.15.32:0/790985359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626b67bf400 0x5626b758c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 --2- 172.21.15.32:0/790985359 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5626b67bf800 0x5626b758c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:22.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/790985359 >> 172.21.15.32:0/790985359 conn(0x5626b67bfc00 msgr2=0x5626b67ed000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:22.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/790985359 shutdown_connections 2024-04-17T10:04:22.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:22.982+0000 7f1a7b0cd8c0 1 -- 172.21.15.32:0/790985359 wait complete. 2024-04-17T10:04:22.985 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.150598, result: 2024-04-17T10:04:22.986 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-17T10:04:23.034 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-17T10:04:23.075 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2024-04-17T10:04:23.075 INFO:teuthology.orchestra.run.smithi032.stdout: chdir = "" 2024-04-17T10:04:23.075 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-17T10:04:23.075 INFO:teuthology.orchestra.run.smithi032.stdout: auth supported = cephx 2024-04-17T10:04:23.075 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.075 INFO:teuthology.orchestra.run.smithi032.stdout: filestore xattr use omap = true 2024-04-17T10:04:23.075 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.076 INFO:teuthology.orchestra.run.smithi032.stdout: mon clock drift allowed = 1.000 2024-04-17T10:04:23.076 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.076 INFO:teuthology.orchestra.run.smithi032.stdout: osd crush chooseleaf type = 0 2024-04-17T10:04:23.076 INFO:teuthology.orchestra.run.smithi032.stdout: auth debug = true 2024-04-17T10:04:23.076 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.076 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on old message = true 2024-04-17T10:04:23.076 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on bug = true 2024-04-17T10:04:23.076 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.076 INFO:teuthology.orchestra.run.smithi032.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-17T10:04:23.077 INFO:teuthology.orchestra.run.smithi032.stdout: mon pg warn max object skew = 0 2024-04-17T10:04:23.077 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.077 INFO:teuthology.orchestra.run.smithi032.stdout: # disable pg_autoscaler by default for new pools 2024-04-17T10:04:23.077 INFO:teuthology.orchestra.run.smithi032.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-17T10:04:23.077 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.077 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default size = 2 2024-04-17T10:04:23.077 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.077 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow primary affinity = true 2024-04-17T10:04:23.078 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow pg remap = true 2024-04-17T10:04:23.078 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on legacy crush tunables = false 2024-04-17T10:04:23.078 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on crush straw calc version zero = false 2024-04-17T10:04:23.078 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on no sortbitwise = false 2024-04-17T10:04:23.078 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on osd down out interval zero = false 2024-04-17T10:04:23.078 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on too few osds = false 2024-04-17T10:04:23.078 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-17T10:04:23.079 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-17T10:04:23.079 INFO:teuthology.orchestra.run.smithi032.stdout: mon_allow_pool_size_one = true 2024-04-17T10:04:23.079 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.079 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-17T10:04:23.079 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.079 INFO:teuthology.orchestra.run.smithi032.stdout: osd default data pool replay window = 5 2024-04-17T10:04:23.079 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.079 INFO:teuthology.orchestra.run.smithi032.stdout: mon allow pool delete = true 2024-04-17T10:04:23.079 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.079 INFO:teuthology.orchestra.run.smithi032.stdout: mon cluster log file level = debug 2024-04-17T10:04:23.079 INFO:teuthology.orchestra.run.smithi032.stdout: debug asserts on shutdown = true 2024-04-17T10:04:23.079 INFO:teuthology.orchestra.run.smithi032.stdout: mon health detail to clog = false 2024-04-17T10:04:23.079 INFO:teuthology.orchestra.run.smithi032.stdout: mon host = "172.21.15.32,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2024-04-17T10:04:23.079 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = ff178df8-4c06-4367-a1e9-2e3a66d870d0 2024-04-17T10:04:23.080 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.080 INFO:teuthology.orchestra.run.smithi032.stdout:[osd] 2024-04-17T10:04:23.080 INFO:teuthology.orchestra.run.smithi032.stdout: osd journal size = 100 2024-04-17T10:04:23.080 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.080 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub load threshold = 5.0 2024-04-17T10:04:23.080 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub max interval = 600 2024-04-17T10:04:23.080 INFO:teuthology.orchestra.run.smithi032.stdout: osd mclock profile = high_recovery_ops 2024-04-17T10:04:23.080 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.080 INFO:teuthology.orchestra.run.smithi032.stdout: osd recover clone overlap = true 2024-04-17T10:04:23.080 INFO:teuthology.orchestra.run.smithi032.stdout: osd recovery max chunk = 1048576 2024-04-17T10:04:23.080 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.080 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug shutdown = true 2024-04-17T10:04:23.080 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug op order = true 2024-04-17T10:04:23.080 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug verify stray on activate = true 2024-04-17T10:04:23.080 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug trim objects = true 2024-04-17T10:04:23.080 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.081 INFO:teuthology.orchestra.run.smithi032.stdout: osd open classes on start = true 2024-04-17T10:04:23.081 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug pg log writeout = true 2024-04-17T10:04:23.081 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.081 INFO:teuthology.orchestra.run.smithi032.stdout: osd deep scrub update digest min age = 30 2024-04-17T10:04:23.081 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.081 INFO:teuthology.orchestra.run.smithi032.stdout: osd map max advance = 10 2024-04-17T10:04:23.081 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.081 INFO:teuthology.orchestra.run.smithi032.stdout: journal zero on create = true 2024-04-17T10:04:23.081 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.081 INFO:teuthology.orchestra.run.smithi032.stdout: filestore ondisk finisher threads = 3 2024-04-17T10:04:23.081 INFO:teuthology.orchestra.run.smithi032.stdout: filestore apply finisher threads = 3 2024-04-17T10:04:23.081 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.081 INFO:teuthology.orchestra.run.smithi032.stdout: bdev debug aio = true 2024-04-17T10:04:23.081 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug misdirected ops = true 2024-04-17T10:04:23.081 INFO:teuthology.orchestra.run.smithi032.stdout: bdev async discard = True 2024-04-17T10:04:23.082 INFO:teuthology.orchestra.run.smithi032.stdout: bdev enable discard = True 2024-04-17T10:04:23.082 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore allocator = bitmap 2024-04-17T10:04:23.082 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore block size = 96636764160 2024-04-17T10:04:23.082 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore fsck on mount = True 2024-04-17T10:04:23.082 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluefs = 1/20 2024-04-17T10:04:23.082 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluestore = 1/20 2024-04-17T10:04:23.082 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T10:04:23.082 INFO:teuthology.orchestra.run.smithi032.stdout: debug osd = 20 2024-04-17T10:04:23.082 INFO:teuthology.orchestra.run.smithi032.stdout: debug rocksdb = 4/10 2024-04-17T10:04:23.082 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd backfillfull_ratio = 0.85 2024-04-17T10:04:23.082 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd full ratio = 0.9 2024-04-17T10:04:23.082 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd nearfull ratio = 0.8 2024-04-17T10:04:23.082 INFO:teuthology.orchestra.run.smithi032.stdout: osd failsafe full ratio = 0.95 2024-04-17T10:04:23.083 INFO:teuthology.orchestra.run.smithi032.stdout: osd objectstore = bluestore 2024-04-17T10:04:23.083 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2024-04-17T10:04:23.083 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.083 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr] 2024-04-17T10:04:23.083 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T10:04:23.083 INFO:teuthology.orchestra.run.smithi032.stdout: debug mgr = 20 2024-04-17T10:04:23.083 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2024-04-17T10:04:23.083 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2024-04-17T10:04:23.083 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2024-04-17T10:04:23.083 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2024-04-17T10:04:23.083 INFO:teuthology.orchestra.run.smithi032.stdout: mgr/telemetry/nag = false 2024-04-17T10:04:23.083 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2024-04-17T10:04:23.083 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.083 INFO:teuthology.orchestra.run.smithi032.stdout:[mon] 2024-04-17T10:04:23.083 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T10:04:23.084 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2024-04-17T10:04:23.084 INFO:teuthology.orchestra.run.smithi032.stdout: debug paxos = 20 2024-04-17T10:04:23.084 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2024-04-17T10:04:23.084 INFO:teuthology.orchestra.run.smithi032.stdout: mon data avail warn = 5 2024-04-17T10:04:23.084 INFO:teuthology.orchestra.run.smithi032.stdout: mon mgr mkfs grace = 240 2024-04-17T10:04:23.084 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2024-04-17T10:04:23.084 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd reporter subtree level = osd 2024-04-17T10:04:23.084 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd prime pg temp = true 2024-04-17T10:04:23.084 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2024-04-17T10:04:23.084 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.084 INFO:teuthology.orchestra.run.smithi032.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-17T10:04:23.084 INFO:teuthology.orchestra.run.smithi032.stdout: auth mon ticket ttl = 660 # 11m 2024-04-17T10:04:23.084 INFO:teuthology.orchestra.run.smithi032.stdout: auth service ticket ttl = 240 # 4m 2024-04-17T10:04:23.084 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.085 INFO:teuthology.orchestra.run.smithi032.stdout: # don't complain about insecure global_id in the test suite 2024-04-17T10:04:23.085 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-17T10:04:23.085 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-17T10:04:23.085 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.085 INFO:teuthology.orchestra.run.smithi032.stdout: # 1m isn't quite enough 2024-04-17T10:04:23.085 INFO:teuthology.orchestra.run.smithi032.stdout: mon_down_mkfs_grace = 2m 2024-04-17T10:04:23.085 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.085 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_filestore_osds = false 2024-04-17T10:04:23.085 INFO:teuthology.orchestra.run.smithi032.stdout: mon down mkfs grace = 300 2024-04-17T10:04:23.085 INFO:teuthology.orchestra.run.smithi032.stdout: mon op complaint time = 120 2024-04-17T10:04:23.085 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.085 INFO:teuthology.orchestra.run.smithi032.stdout:[client] 2024-04-17T10:04:23.085 INFO:teuthology.orchestra.run.smithi032.stdout: rgw cache enabled = true 2024-04-17T10:04:23.086 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable ops log = true 2024-04-17T10:04:23.086 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable usage log = true 2024-04-17T10:04:23.086 INFO:teuthology.orchestra.run.smithi032.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-17T10:04:23.086 INFO:teuthology.orchestra.run.smithi032.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-17T10:04:23.086 INFO:teuthology.orchestra.run.smithi032.stdout: client mount timeout = 600 2024-04-17T10:04:23.086 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2024-04-17T10:04:23.086 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T10:04:23.086 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = "" 2024-04-17T10:04:23.086 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2024-04-17T10:04:23.086 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2024-04-17T10:04:23.086 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.a] 2024-04-17T10:04:23.086 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.b] 2024-04-17T10:04:23.086 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.c] 2024-04-17T10:04:23.086 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2024-04-17T10:04:23.086 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c] 2024-04-17T10:04:23.087 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2024-04-17T10:04:23.087 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.d] 2024-04-17T10:04:23.087 INFO:teuthology.orchestra.run.smithi032.stdout:[mds] 2024-04-17T10:04:23.087 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds = 20 2024-04-17T10:04:23.087 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds balancer = 20 2024-04-17T10:04:23.087 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T10:04:23.087 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug frag = True 2024-04-17T10:04:23.087 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug scatterstat = False 2024-04-17T10:04:23.087 INFO:teuthology.orchestra.run.smithi032.stdout: mds op complaint time = 180 2024-04-17T10:04:23.087 INFO:teuthology.orchestra.run.smithi032.stdout: mds verify scatter = True 2024-04-17T10:04:23.087 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2024-04-17T10:04:23.087 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2024-04-17T10:04:23.087 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2024-04-17T10:04:23.088 INFO:teuthology.orchestra.run.smithi144.stdout:[global] 2024-04-17T10:04:23.088 INFO:teuthology.orchestra.run.smithi144.stdout: chdir = "" 2024-04-17T10:04:23.088 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-17T10:04:23.088 INFO:teuthology.orchestra.run.smithi144.stdout: auth supported = cephx 2024-04-17T10:04:23.088 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.088 INFO:teuthology.orchestra.run.smithi144.stdout: filestore xattr use omap = true 2024-04-17T10:04:23.088 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.088 INFO:teuthology.orchestra.run.smithi144.stdout: mon clock drift allowed = 1.000 2024-04-17T10:04:23.088 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.088 INFO:teuthology.orchestra.run.smithi144.stdout: osd crush chooseleaf type = 0 2024-04-17T10:04:23.088 INFO:teuthology.orchestra.run.smithi144.stdout: auth debug = true 2024-04-17T10:04:23.088 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.088 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on old message = true 2024-04-17T10:04:23.089 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on bug = true 2024-04-17T10:04:23.089 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.089 INFO:teuthology.orchestra.run.smithi144.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-17T10:04:23.089 INFO:teuthology.orchestra.run.smithi144.stdout: mon pg warn max object skew = 0 2024-04-17T10:04:23.089 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.089 INFO:teuthology.orchestra.run.smithi144.stdout: # disable pg_autoscaler by default for new pools 2024-04-17T10:04:23.089 INFO:teuthology.orchestra.run.smithi144.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-17T10:04:23.089 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.089 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default size = 2 2024-04-17T10:04:23.089 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.089 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow primary affinity = true 2024-04-17T10:04:23.089 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow pg remap = true 2024-04-17T10:04:23.089 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on legacy crush tunables = false 2024-04-17T10:04:23.089 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on crush straw calc version zero = false 2024-04-17T10:04:23.089 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on no sortbitwise = false 2024-04-17T10:04:23.089 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on osd down out interval zero = false 2024-04-17T10:04:23.090 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on too few osds = false 2024-04-17T10:04:23.090 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-17T10:04:23.090 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-17T10:04:23.090 INFO:teuthology.orchestra.run.smithi144.stdout: mon_allow_pool_size_one = true 2024-04-17T10:04:23.090 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.090 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-17T10:04:23.090 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.090 INFO:teuthology.orchestra.run.smithi144.stdout: osd default data pool replay window = 5 2024-04-17T10:04:23.090 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.090 INFO:teuthology.orchestra.run.smithi144.stdout: mon allow pool delete = true 2024-04-17T10:04:23.090 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.090 INFO:teuthology.orchestra.run.smithi144.stdout: mon cluster log file level = debug 2024-04-17T10:04:23.090 INFO:teuthology.orchestra.run.smithi144.stdout: debug asserts on shutdown = true 2024-04-17T10:04:23.090 INFO:teuthology.orchestra.run.smithi144.stdout: mon health detail to clog = false 2024-04-17T10:04:23.090 INFO:teuthology.orchestra.run.smithi144.stdout: mon host = "172.21.15.32,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2024-04-17T10:04:23.091 INFO:teuthology.orchestra.run.smithi144.stdout: fsid = ff178df8-4c06-4367-a1e9-2e3a66d870d0 2024-04-17T10:04:23.091 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.091 INFO:teuthology.orchestra.run.smithi144.stdout:[osd] 2024-04-17T10:04:23.091 INFO:teuthology.orchestra.run.smithi144.stdout: osd journal size = 100 2024-04-17T10:04:23.091 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.091 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub load threshold = 5.0 2024-04-17T10:04:23.091 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub max interval = 600 2024-04-17T10:04:23.091 INFO:teuthology.orchestra.run.smithi144.stdout: osd mclock profile = high_recovery_ops 2024-04-17T10:04:23.091 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.091 INFO:teuthology.orchestra.run.smithi144.stdout: osd recover clone overlap = true 2024-04-17T10:04:23.091 INFO:teuthology.orchestra.run.smithi144.stdout: osd recovery max chunk = 1048576 2024-04-17T10:04:23.091 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.091 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug shutdown = true 2024-04-17T10:04:23.091 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug op order = true 2024-04-17T10:04:23.092 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug verify stray on activate = true 2024-04-17T10:04:23.092 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug trim objects = true 2024-04-17T10:04:23.092 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.092 INFO:teuthology.orchestra.run.smithi144.stdout: osd open classes on start = true 2024-04-17T10:04:23.092 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug pg log writeout = true 2024-04-17T10:04:23.092 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.092 INFO:teuthology.orchestra.run.smithi144.stdout: osd deep scrub update digest min age = 30 2024-04-17T10:04:23.092 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.092 INFO:teuthology.orchestra.run.smithi144.stdout: osd map max advance = 10 2024-04-17T10:04:23.092 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.092 INFO:teuthology.orchestra.run.smithi144.stdout: journal zero on create = true 2024-04-17T10:04:23.092 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.092 INFO:teuthology.orchestra.run.smithi144.stdout: filestore ondisk finisher threads = 3 2024-04-17T10:04:23.092 INFO:teuthology.orchestra.run.smithi144.stdout: filestore apply finisher threads = 3 2024-04-17T10:04:23.092 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.092 INFO:teuthology.orchestra.run.smithi144.stdout: bdev debug aio = true 2024-04-17T10:04:23.092 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug misdirected ops = true 2024-04-17T10:04:23.093 INFO:teuthology.orchestra.run.smithi144.stdout: bdev async discard = True 2024-04-17T10:04:23.093 INFO:teuthology.orchestra.run.smithi144.stdout: bdev enable discard = True 2024-04-17T10:04:23.093 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore allocator = bitmap 2024-04-17T10:04:23.093 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore block size = 96636764160 2024-04-17T10:04:23.093 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore fsck on mount = True 2024-04-17T10:04:23.093 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluefs = 1/20 2024-04-17T10:04:23.093 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluestore = 1/20 2024-04-17T10:04:23.093 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T10:04:23.093 INFO:teuthology.orchestra.run.smithi144.stdout: debug osd = 20 2024-04-17T10:04:23.093 INFO:teuthology.orchestra.run.smithi144.stdout: debug rocksdb = 4/10 2024-04-17T10:04:23.093 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd backfillfull_ratio = 0.85 2024-04-17T10:04:23.093 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd full ratio = 0.9 2024-04-17T10:04:23.093 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd nearfull ratio = 0.8 2024-04-17T10:04:23.093 INFO:teuthology.orchestra.run.smithi144.stdout: osd failsafe full ratio = 0.95 2024-04-17T10:04:23.094 INFO:teuthology.orchestra.run.smithi144.stdout: osd objectstore = bluestore 2024-04-17T10:04:23.094 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2024-04-17T10:04:23.094 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.094 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr] 2024-04-17T10:04:23.094 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T10:04:23.094 INFO:teuthology.orchestra.run.smithi144.stdout: debug mgr = 20 2024-04-17T10:04:23.094 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-04-17T10:04:23.094 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-04-17T10:04:23.094 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-04-17T10:04:23.094 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-04-17T10:04:23.094 INFO:teuthology.orchestra.run.smithi144.stdout: mgr/telemetry/nag = false 2024-04-17T10:04:23.094 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2024-04-17T10:04:23.094 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.094 INFO:teuthology.orchestra.run.smithi144.stdout:[mon] 2024-04-17T10:04:23.095 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T10:04:23.095 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-04-17T10:04:23.095 INFO:teuthology.orchestra.run.smithi144.stdout: debug paxos = 20 2024-04-17T10:04:23.095 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-04-17T10:04:23.095 INFO:teuthology.orchestra.run.smithi144.stdout: mon data avail warn = 5 2024-04-17T10:04:23.095 INFO:teuthology.orchestra.run.smithi144.stdout: mon mgr mkfs grace = 240 2024-04-17T10:04:23.095 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-04-17T10:04:23.095 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd reporter subtree level = osd 2024-04-17T10:04:23.095 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd prime pg temp = true 2024-04-17T10:04:23.095 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-04-17T10:04:23.095 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.095 INFO:teuthology.orchestra.run.smithi144.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-17T10:04:23.095 INFO:teuthology.orchestra.run.smithi144.stdout: auth mon ticket ttl = 660 # 11m 2024-04-17T10:04:23.095 INFO:teuthology.orchestra.run.smithi144.stdout: auth service ticket ttl = 240 # 4m 2024-04-17T10:04:23.096 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.096 INFO:teuthology.orchestra.run.smithi144.stdout: # don't complain about insecure global_id in the test suite 2024-04-17T10:04:23.096 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-17T10:04:23.096 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-17T10:04:23.096 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.096 INFO:teuthology.orchestra.run.smithi144.stdout: # 1m isn't quite enough 2024-04-17T10:04:23.096 INFO:teuthology.orchestra.run.smithi144.stdout: mon_down_mkfs_grace = 2m 2024-04-17T10:04:23.096 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.096 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_filestore_osds = false 2024-04-17T10:04:23.096 INFO:teuthology.orchestra.run.smithi144.stdout: mon down mkfs grace = 300 2024-04-17T10:04:23.096 INFO:teuthology.orchestra.run.smithi144.stdout: mon op complaint time = 120 2024-04-17T10:04:23.096 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.096 INFO:teuthology.orchestra.run.smithi144.stdout:[client] 2024-04-17T10:04:23.096 INFO:teuthology.orchestra.run.smithi144.stdout: rgw cache enabled = true 2024-04-17T10:04:23.096 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable ops log = true 2024-04-17T10:04:23.097 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable usage log = true 2024-04-17T10:04:23.097 INFO:teuthology.orchestra.run.smithi144.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-17T10:04:23.097 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-17T10:04:23.097 INFO:teuthology.orchestra.run.smithi144.stdout: client mount timeout = 600 2024-04-17T10:04:23.097 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2024-04-17T10:04:23.097 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T10:04:23.097 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = "" 2024-04-17T10:04:23.097 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2024-04-17T10:04:23.097 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2024-04-17T10:04:23.097 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.a] 2024-04-17T10:04:23.097 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.b] 2024-04-17T10:04:23.097 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.c] 2024-04-17T10:04:23.097 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.a] 2024-04-17T10:04:23.097 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.c] 2024-04-17T10:04:23.097 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.b] 2024-04-17T10:04:23.098 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.d] 2024-04-17T10:04:23.098 INFO:teuthology.orchestra.run.smithi144.stdout:[mds] 2024-04-17T10:04:23.098 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds = 20 2024-04-17T10:04:23.098 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds balancer = 20 2024-04-17T10:04:23.098 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T10:04:23.098 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug frag = True 2024-04-17T10:04:23.098 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug scatterstat = False 2024-04-17T10:04:23.098 INFO:teuthology.orchestra.run.smithi144.stdout: mds op complaint time = 180 2024-04-17T10:04:23.098 INFO:teuthology.orchestra.run.smithi144.stdout: mds verify scatter = True 2024-04-17T10:04:23.098 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2024-04-17T10:04:23.098 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2024-04-17T10:04:23.098 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2024-04-17T10:04:23.100 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-17T10:04:23.150 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-17T10:04:23.191 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2024-04-17T10:04:23.191 INFO:teuthology.orchestra.run.smithi032.stdout: chdir = "" 2024-04-17T10:04:23.191 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-17T10:04:23.191 INFO:teuthology.orchestra.run.smithi032.stdout: auth supported = cephx 2024-04-17T10:04:23.191 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.191 INFO:teuthology.orchestra.run.smithi032.stdout: filestore xattr use omap = true 2024-04-17T10:04:23.191 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.191 INFO:teuthology.orchestra.run.smithi032.stdout: mon clock drift allowed = 1.000 2024-04-17T10:04:23.191 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.191 INFO:teuthology.orchestra.run.smithi032.stdout: osd crush chooseleaf type = 0 2024-04-17T10:04:23.191 INFO:teuthology.orchestra.run.smithi032.stdout: auth debug = true 2024-04-17T10:04:23.191 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.192 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on old message = true 2024-04-17T10:04:23.192 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on bug = true 2024-04-17T10:04:23.192 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.192 INFO:teuthology.orchestra.run.smithi032.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-17T10:04:23.192 INFO:teuthology.orchestra.run.smithi032.stdout: mon pg warn max object skew = 0 2024-04-17T10:04:23.192 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.192 INFO:teuthology.orchestra.run.smithi032.stdout: # disable pg_autoscaler by default for new pools 2024-04-17T10:04:23.192 INFO:teuthology.orchestra.run.smithi032.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-17T10:04:23.192 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.192 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default size = 2 2024-04-17T10:04:23.192 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.192 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow primary affinity = true 2024-04-17T10:04:23.192 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow pg remap = true 2024-04-17T10:04:23.192 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on legacy crush tunables = false 2024-04-17T10:04:23.193 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on crush straw calc version zero = false 2024-04-17T10:04:23.193 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on no sortbitwise = false 2024-04-17T10:04:23.193 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on osd down out interval zero = false 2024-04-17T10:04:23.193 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on too few osds = false 2024-04-17T10:04:23.193 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-17T10:04:23.193 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-17T10:04:23.193 INFO:teuthology.orchestra.run.smithi032.stdout: mon_allow_pool_size_one = true 2024-04-17T10:04:23.193 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.193 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-17T10:04:23.193 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.193 INFO:teuthology.orchestra.run.smithi032.stdout: osd default data pool replay window = 5 2024-04-17T10:04:23.193 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.193 INFO:teuthology.orchestra.run.smithi032.stdout: mon allow pool delete = true 2024-04-17T10:04:23.193 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.193 INFO:teuthology.orchestra.run.smithi032.stdout: mon cluster log file level = debug 2024-04-17T10:04:23.194 INFO:teuthology.orchestra.run.smithi032.stdout: debug asserts on shutdown = true 2024-04-17T10:04:23.194 INFO:teuthology.orchestra.run.smithi032.stdout: mon health detail to clog = false 2024-04-17T10:04:23.194 INFO:teuthology.orchestra.run.smithi032.stdout: mon host = "172.21.15.32,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2024-04-17T10:04:23.194 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = ff178df8-4c06-4367-a1e9-2e3a66d870d0 2024-04-17T10:04:23.194 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.194 INFO:teuthology.orchestra.run.smithi032.stdout:[osd] 2024-04-17T10:04:23.194 INFO:teuthology.orchestra.run.smithi032.stdout: osd journal size = 100 2024-04-17T10:04:23.194 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.194 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub load threshold = 5.0 2024-04-17T10:04:23.194 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub max interval = 600 2024-04-17T10:04:23.194 INFO:teuthology.orchestra.run.smithi032.stdout: osd mclock profile = high_recovery_ops 2024-04-17T10:04:23.194 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.194 INFO:teuthology.orchestra.run.smithi032.stdout: osd recover clone overlap = true 2024-04-17T10:04:23.194 INFO:teuthology.orchestra.run.smithi032.stdout: osd recovery max chunk = 1048576 2024-04-17T10:04:23.195 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.195 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug shutdown = true 2024-04-17T10:04:23.195 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug op order = true 2024-04-17T10:04:23.195 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug verify stray on activate = true 2024-04-17T10:04:23.195 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug trim objects = true 2024-04-17T10:04:23.195 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.195 INFO:teuthology.orchestra.run.smithi032.stdout: osd open classes on start = true 2024-04-17T10:04:23.195 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug pg log writeout = true 2024-04-17T10:04:23.195 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.195 INFO:teuthology.orchestra.run.smithi032.stdout: osd deep scrub update digest min age = 30 2024-04-17T10:04:23.196 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.196 INFO:teuthology.orchestra.run.smithi032.stdout: osd map max advance = 10 2024-04-17T10:04:23.196 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.196 INFO:teuthology.orchestra.run.smithi032.stdout: journal zero on create = true 2024-04-17T10:04:23.196 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.196 INFO:teuthology.orchestra.run.smithi032.stdout: filestore ondisk finisher threads = 3 2024-04-17T10:04:23.196 INFO:teuthology.orchestra.run.smithi032.stdout: filestore apply finisher threads = 3 2024-04-17T10:04:23.196 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.196 INFO:teuthology.orchestra.run.smithi032.stdout: bdev debug aio = true 2024-04-17T10:04:23.196 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug misdirected ops = true 2024-04-17T10:04:23.196 INFO:teuthology.orchestra.run.smithi032.stdout: bdev async discard = True 2024-04-17T10:04:23.196 INFO:teuthology.orchestra.run.smithi032.stdout: bdev enable discard = True 2024-04-17T10:04:23.196 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore allocator = bitmap 2024-04-17T10:04:23.196 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore block size = 96636764160 2024-04-17T10:04:23.197 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore fsck on mount = True 2024-04-17T10:04:23.197 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluefs = 1/20 2024-04-17T10:04:23.197 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluestore = 1/20 2024-04-17T10:04:23.197 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T10:04:23.197 INFO:teuthology.orchestra.run.smithi032.stdout: debug osd = 20 2024-04-17T10:04:23.197 INFO:teuthology.orchestra.run.smithi032.stdout: debug rocksdb = 4/10 2024-04-17T10:04:23.197 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd backfillfull_ratio = 0.85 2024-04-17T10:04:23.197 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd full ratio = 0.9 2024-04-17T10:04:23.197 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd nearfull ratio = 0.8 2024-04-17T10:04:23.197 INFO:teuthology.orchestra.run.smithi032.stdout: osd failsafe full ratio = 0.95 2024-04-17T10:04:23.197 INFO:teuthology.orchestra.run.smithi032.stdout: osd objectstore = bluestore 2024-04-17T10:04:23.197 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2024-04-17T10:04:23.197 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.197 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr] 2024-04-17T10:04:23.197 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T10:04:23.198 INFO:teuthology.orchestra.run.smithi032.stdout: debug mgr = 20 2024-04-17T10:04:23.198 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2024-04-17T10:04:23.198 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2024-04-17T10:04:23.198 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2024-04-17T10:04:23.198 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2024-04-17T10:04:23.198 INFO:teuthology.orchestra.run.smithi032.stdout: mgr/telemetry/nag = false 2024-04-17T10:04:23.198 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2024-04-17T10:04:23.198 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.198 INFO:teuthology.orchestra.run.smithi032.stdout:[mon] 2024-04-17T10:04:23.198 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T10:04:23.198 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2024-04-17T10:04:23.198 INFO:teuthology.orchestra.run.smithi032.stdout: debug paxos = 20 2024-04-17T10:04:23.198 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2024-04-17T10:04:23.198 INFO:teuthology.orchestra.run.smithi032.stdout: mon data avail warn = 5 2024-04-17T10:04:23.199 INFO:teuthology.orchestra.run.smithi032.stdout: mon mgr mkfs grace = 240 2024-04-17T10:04:23.199 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2024-04-17T10:04:23.199 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd reporter subtree level = osd 2024-04-17T10:04:23.199 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd prime pg temp = true 2024-04-17T10:04:23.199 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2024-04-17T10:04:23.199 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.199 INFO:teuthology.orchestra.run.smithi032.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-17T10:04:23.199 INFO:teuthology.orchestra.run.smithi032.stdout: auth mon ticket ttl = 660 # 11m 2024-04-17T10:04:23.199 INFO:teuthology.orchestra.run.smithi032.stdout: auth service ticket ttl = 240 # 4m 2024-04-17T10:04:23.199 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.199 INFO:teuthology.orchestra.run.smithi032.stdout: # don't complain about insecure global_id in the test suite 2024-04-17T10:04:23.199 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-17T10:04:23.199 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-17T10:04:23.199 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.199 INFO:teuthology.orchestra.run.smithi032.stdout: # 1m isn't quite enough 2024-04-17T10:04:23.200 INFO:teuthology.orchestra.run.smithi032.stdout: mon_down_mkfs_grace = 2m 2024-04-17T10:04:23.200 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.200 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_filestore_osds = false 2024-04-17T10:04:23.200 INFO:teuthology.orchestra.run.smithi032.stdout: mon down mkfs grace = 300 2024-04-17T10:04:23.200 INFO:teuthology.orchestra.run.smithi032.stdout: mon op complaint time = 120 2024-04-17T10:04:23.200 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:23.200 INFO:teuthology.orchestra.run.smithi032.stdout:[client] 2024-04-17T10:04:23.200 INFO:teuthology.orchestra.run.smithi032.stdout: rgw cache enabled = true 2024-04-17T10:04:23.200 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable ops log = true 2024-04-17T10:04:23.200 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable usage log = true 2024-04-17T10:04:23.201 INFO:teuthology.orchestra.run.smithi032.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-17T10:04:23.201 INFO:teuthology.orchestra.run.smithi032.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-17T10:04:23.201 INFO:teuthology.orchestra.run.smithi032.stdout: client mount timeout = 600 2024-04-17T10:04:23.201 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2024-04-17T10:04:23.201 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T10:04:23.201 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = "" 2024-04-17T10:04:23.201 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2024-04-17T10:04:23.201 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2024-04-17T10:04:23.201 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.a] 2024-04-17T10:04:23.201 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.b] 2024-04-17T10:04:23.201 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.c] 2024-04-17T10:04:23.201 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2024-04-17T10:04:23.201 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c] 2024-04-17T10:04:23.201 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2024-04-17T10:04:23.202 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.d] 2024-04-17T10:04:23.202 INFO:teuthology.orchestra.run.smithi032.stdout:[mds] 2024-04-17T10:04:23.202 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds = 20 2024-04-17T10:04:23.202 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds balancer = 20 2024-04-17T10:04:23.202 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2024-04-17T10:04:23.202 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug frag = True 2024-04-17T10:04:23.202 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug scatterstat = True 2024-04-17T10:04:23.202 INFO:teuthology.orchestra.run.smithi032.stdout: mds op complaint time = 180 2024-04-17T10:04:23.202 INFO:teuthology.orchestra.run.smithi032.stdout: mds verify scatter = True 2024-04-17T10:04:23.202 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2024-04-17T10:04:23.202 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2024-04-17T10:04:23.202 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2024-04-17T10:04:23.203 INFO:teuthology.orchestra.run.smithi144.stdout:[global] 2024-04-17T10:04:23.203 INFO:teuthology.orchestra.run.smithi144.stdout: chdir = "" 2024-04-17T10:04:23.203 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-17T10:04:23.203 INFO:teuthology.orchestra.run.smithi144.stdout: auth supported = cephx 2024-04-17T10:04:23.203 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.203 INFO:teuthology.orchestra.run.smithi144.stdout: filestore xattr use omap = true 2024-04-17T10:04:23.203 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.203 INFO:teuthology.orchestra.run.smithi144.stdout: mon clock drift allowed = 1.000 2024-04-17T10:04:23.203 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: osd crush chooseleaf type = 0 2024-04-17T10:04:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: auth debug = true 2024-04-17T10:04:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on old message = true 2024-04-17T10:04:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on bug = true 2024-04-17T10:04:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-17T10:04:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: mon pg warn max object skew = 0 2024-04-17T10:04:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: # disable pg_autoscaler by default for new pools 2024-04-17T10:04:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-17T10:04:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default size = 2 2024-04-17T10:04:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow primary affinity = true 2024-04-17T10:04:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow pg remap = true 2024-04-17T10:04:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on legacy crush tunables = false 2024-04-17T10:04:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on crush straw calc version zero = false 2024-04-17T10:04:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on no sortbitwise = false 2024-04-17T10:04:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on osd down out interval zero = false 2024-04-17T10:04:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on too few osds = false 2024-04-17T10:04:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-17T10:04:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-17T10:04:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: mon_allow_pool_size_one = true 2024-04-17T10:04:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-17T10:04:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: osd default data pool replay window = 5 2024-04-17T10:04:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: mon allow pool delete = true 2024-04-17T10:04:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: mon cluster log file level = debug 2024-04-17T10:04:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: debug asserts on shutdown = true 2024-04-17T10:04:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: mon health detail to clog = false 2024-04-17T10:04:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: mon host = "172.21.15.32,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2024-04-17T10:04:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: fsid = ff178df8-4c06-4367-a1e9-2e3a66d870d0 2024-04-17T10:04:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.206 INFO:teuthology.orchestra.run.smithi144.stdout:[osd] 2024-04-17T10:04:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: osd journal size = 100 2024-04-17T10:04:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub load threshold = 5.0 2024-04-17T10:04:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub max interval = 600 2024-04-17T10:04:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: osd mclock profile = high_recovery_ops 2024-04-17T10:04:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: osd recover clone overlap = true 2024-04-17T10:04:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: osd recovery max chunk = 1048576 2024-04-17T10:04:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug shutdown = true 2024-04-17T10:04:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug op order = true 2024-04-17T10:04:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug verify stray on activate = true 2024-04-17T10:04:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug trim objects = true 2024-04-17T10:04:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: osd open classes on start = true 2024-04-17T10:04:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug pg log writeout = true 2024-04-17T10:04:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: osd deep scrub update digest min age = 30 2024-04-17T10:04:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: osd map max advance = 10 2024-04-17T10:04:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: journal zero on create = true 2024-04-17T10:04:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: filestore ondisk finisher threads = 3 2024-04-17T10:04:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: filestore apply finisher threads = 3 2024-04-17T10:04:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: bdev debug aio = true 2024-04-17T10:04:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug misdirected ops = true 2024-04-17T10:04:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: bdev async discard = True 2024-04-17T10:04:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: bdev enable discard = True 2024-04-17T10:04:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore allocator = bitmap 2024-04-17T10:04:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore block size = 96636764160 2024-04-17T10:04:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore fsck on mount = True 2024-04-17T10:04:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluefs = 1/20 2024-04-17T10:04:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluestore = 1/20 2024-04-17T10:04:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T10:04:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: debug osd = 20 2024-04-17T10:04:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: debug rocksdb = 4/10 2024-04-17T10:04:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd backfillfull_ratio = 0.85 2024-04-17T10:04:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd full ratio = 0.9 2024-04-17T10:04:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd nearfull ratio = 0.8 2024-04-17T10:04:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: osd failsafe full ratio = 0.95 2024-04-17T10:04:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: osd objectstore = bluestore 2024-04-17T10:04:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2024-04-17T10:04:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.209 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr] 2024-04-17T10:04:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T10:04:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: debug mgr = 20 2024-04-17T10:04:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-04-17T10:04:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-04-17T10:04:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-04-17T10:04:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-04-17T10:04:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: mgr/telemetry/nag = false 2024-04-17T10:04:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2024-04-17T10:04:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.210 INFO:teuthology.orchestra.run.smithi144.stdout:[mon] 2024-04-17T10:04:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T10:04:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-04-17T10:04:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: debug paxos = 20 2024-04-17T10:04:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-04-17T10:04:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: mon data avail warn = 5 2024-04-17T10:04:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: mon mgr mkfs grace = 240 2024-04-17T10:04:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-04-17T10:04:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd reporter subtree level = osd 2024-04-17T10:04:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd prime pg temp = true 2024-04-17T10:04:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-04-17T10:04:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-17T10:04:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: auth mon ticket ttl = 660 # 11m 2024-04-17T10:04:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: auth service ticket ttl = 240 # 4m 2024-04-17T10:04:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: # don't complain about insecure global_id in the test suite 2024-04-17T10:04:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-17T10:04:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-17T10:04:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: # 1m isn't quite enough 2024-04-17T10:04:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: mon_down_mkfs_grace = 2m 2024-04-17T10:04:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_filestore_osds = false 2024-04-17T10:04:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: mon down mkfs grace = 300 2024-04-17T10:04:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: mon op complaint time = 120 2024-04-17T10:04:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-17T10:04:23.212 INFO:teuthology.orchestra.run.smithi144.stdout:[client] 2024-04-17T10:04:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: rgw cache enabled = true 2024-04-17T10:04:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable ops log = true 2024-04-17T10:04:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable usage log = true 2024-04-17T10:04:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-17T10:04:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-17T10:04:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: client mount timeout = 600 2024-04-17T10:04:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2024-04-17T10:04:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T10:04:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = "" 2024-04-17T10:04:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2024-04-17T10:04:23.213 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2024-04-17T10:04:23.213 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.a] 2024-04-17T10:04:23.213 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.b] 2024-04-17T10:04:23.213 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.c] 2024-04-17T10:04:23.213 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.a] 2024-04-17T10:04:23.213 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.c] 2024-04-17T10:04:23.213 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.b] 2024-04-17T10:04:23.213 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.d] 2024-04-17T10:04:23.213 INFO:teuthology.orchestra.run.smithi144.stdout:[mds] 2024-04-17T10:04:23.213 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds = 20 2024-04-17T10:04:23.213 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds balancer = 20 2024-04-17T10:04:23.213 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-17T10:04:23.213 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug frag = True 2024-04-17T10:04:23.213 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug scatterstat = True 2024-04-17T10:04:23.213 INFO:teuthology.orchestra.run.smithi144.stdout: mds op complaint time = 180 2024-04-17T10:04:23.214 INFO:teuthology.orchestra.run.smithi144.stdout: mds verify scatter = True 2024-04-17T10:04:23.214 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2024-04-17T10:04:23.214 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2024-04-17T10:04:23.214 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2024-04-17T10:04:23.215 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-17T10:04:23.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.410+0000 7fd318c00640 1 Processor -- start 2024-04-17T10:04:23.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.410+0000 7fd318c00640 1 -- start start 2024-04-17T10:04:23.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.414+0000 7fd318c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd314160c50 0x7fd31406c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:23.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.414+0000 7fd318c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd3140b0eb0 0x7fd3140b1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:23.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.414+0000 7fd318c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3141611b0 con 0x7fd314145e70 2024-04-17T10:04:23.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.414+0000 7fd318c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd3140afba0 con 0x7fd3140b0eb0 2024-04-17T10:04:23.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.414+0000 7fd318c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd3140afdd0 con 0x7fd314160c50 2024-04-17T10:04:23.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.414+0000 7fd313400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd314145e70 0x7fd31406ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50384/0 (socket says 172.21.15.32:50384) 2024-04-17T10:04:23.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.414+0000 7fd313400640 1 -- 172.21.15.32:0/2903432426 learned_addr learned my addr 172.21.15.32:0/2903432426 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:23.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.414+0000 7fd313e00640 1 --2- 172.21.15.32:0/2903432426 >> v2:172.21.15.144:3300/0 conn(0x7fd3140b0eb0 0x7fd3140b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:23.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.414+0000 7fd30aa00640 1 --2- 172.21.15.32:0/2903432426 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd314160c50 0x7fd31406c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:23.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.414+0000 7fd312a00640 1 -- 172.21.15.32:0/2903432426 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3919030270 0 0) 0x7fd3141611b0 con 0x7fd314145e70 2024-04-17T10:04:23.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.414+0000 7fd312a00640 1 -- 172.21.15.32:0/2903432426 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2f8003530 con 0x7fd314145e70 2024-04-17T10:04:23.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.414+0000 7fd313e00640 1 -- 172.21.15.32:0/2903432426 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd314160c50 msgr2=0x7fd31406c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:23.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.414+0000 7fd313e00640 1 --2- 172.21.15.32:0/2903432426 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd314160c50 0x7fd31406c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:23.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.414+0000 7fd313e00640 1 -- 172.21.15.32:0/2903432426 >> v1:172.21.15.32:6789/0 conn(0x7fd314145e70 legacy=0x7fd31406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:23.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.414+0000 7fd313e00640 1 -- 172.21.15.32:0/2903432426 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3140b0070 con 0x7fd3140b0eb0 2024-04-17T10:04:23.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.418+0000 7fd313e00640 1 --2- 172.21.15.32:0/2903432426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3140b0eb0 0x7fd3140b1340 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009d90 tx=0x7fd30c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cf27a70d2c08d31 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:23.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.418+0000 7fd312a00640 1 -- 172.21.15.32:0/2903432426 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c0b2070 con 0x7fd3140b0eb0 2024-04-17T10:04:23.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.418+0000 7fd312a00640 1 -- 172.21.15.32:0/2903432426 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30c0a5920 con 0x7fd3140b0eb0 2024-04-17T10:04:23.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.418+0000 7fd312a00640 1 -- 172.21.15.32:0/2903432426 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c0a5cf0 con 0x7fd3140b0eb0 2024-04-17T10:04:23.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.418+0000 7fd318c00640 1 -- 172.21.15.32:0/2903432426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3140b0eb0 msgr2=0x7fd3140b1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:23.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.418+0000 7fd318c00640 1 --2- 172.21.15.32:0/2903432426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3140b0eb0 0x7fd3140b1340 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009d90 tx=0x7fd30c01c970 comp rx=0 tx=0).stop 2024-04-17T10:04:23.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.418+0000 7fd318c00640 1 -- 172.21.15.32:0/2903432426 shutdown_connections 2024-04-17T10:04:23.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.418+0000 7fd318c00640 1 --2- 172.21.15.32:0/2903432426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3140b0eb0 0x7fd3140b1340 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:23.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.418+0000 7fd318c00640 1 --2- 172.21.15.32:0/2903432426 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd314160c50 0x7fd31406c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:23.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.418+0000 7fd318c00640 1 -- 172.21.15.32:0/2903432426 >> 172.21.15.32:0/2903432426 conn(0x7fd3140732d0 msgr2=0x7fd31409a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:23.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.418+0000 7fd318c00640 1 -- 172.21.15.32:0/2903432426 shutdown_connections 2024-04-17T10:04:23.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.418+0000 7fd318c00640 1 -- 172.21.15.32:0/2903432426 wait complete. 2024-04-17T10:04:23.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.418+0000 7fd318c00640 1 Processor -- start 2024-04-17T10:04:23.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.418+0000 7fd318c00640 1 -- start start 2024-04-17T10:04:23.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.418+0000 7fd318c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3140b0eb0 0x7fd3140d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:23.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.418+0000 7fd318c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd314145e70 0x7fd3140cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:23.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.418+0000 7fd318c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd314160c50 0x7fd31415ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:23.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.418+0000 7fd318c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd3140b0b50 con 0x7fd314160c50 2024-04-17T10:04:23.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.418+0000 7fd318c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd3140ae570 con 0x7fd3140b0eb0 2024-04-17T10:04:23.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.418+0000 7fd318c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd3140cf700 con 0x7fd314145e70 2024-04-17T10:04:23.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.418+0000 7fd313e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd314160c50 0x7fd31415ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:23.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.422+0000 7fd313e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd314160c50 0x7fd31415ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60934/0 (socket says 172.21.15.32:60934) 2024-04-17T10:04:23.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.422+0000 7fd313e00640 1 -- 172.21.15.32:0/302285085 learned_addr learned my addr 172.21.15.32:0/302285085 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:23.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.422+0000 7fd313400640 1 --2- 172.21.15.32:0/302285085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3140b0eb0 0x7fd3140d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:23.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.422+0000 7fd30aa00640 1 --2- 172.21.15.32:0/302285085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd314145e70 0x7fd3140cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:23.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.422+0000 7fd313e00640 1 -- 172.21.15.32:0/302285085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd314145e70 msgr2=0x7fd3140cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:23.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.422+0000 7fd313e00640 1 --2- 172.21.15.32:0/302285085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd314145e70 0x7fd3140cf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:23.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.422+0000 7fd313e00640 1 -- 172.21.15.32:0/302285085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3140b0eb0 msgr2=0x7fd3140d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:23.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.422+0000 7fd313e00640 1 --2- 172.21.15.32:0/302285085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3140b0eb0 0x7fd3140d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:23.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.422+0000 7fd313e00640 1 -- 172.21.15.32:0/302285085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3140d08c0 con 0x7fd314160c50 2024-04-17T10:04:23.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.422+0000 7fd313e00640 1 --2- 172.21.15.32:0/302285085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd314160c50 0x7fd31415ce90 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0b0040 tx=0x7fd30c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:23.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.422+0000 7fd30be00640 1 -- 172.21.15.32:0/302285085 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c0b2050 con 0x7fd314160c50 2024-04-17T10:04:23.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.422+0000 7fd30be00640 1 -- 172.21.15.32:0/302285085 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30c0be070 con 0x7fd314160c50 2024-04-17T10:04:23.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.422+0000 7fd318c00640 1 -- 172.21.15.32:0/302285085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31413a3b0 con 0x7fd314160c50 2024-04-17T10:04:23.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.422+0000 7fd30be00640 1 -- 172.21.15.32:0/302285085 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c0ad850 con 0x7fd314160c50 2024-04-17T10:04:23.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.422+0000 7fd318c00640 1 -- 172.21.15.32:0/302285085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31413a970 con 0x7fd314160c50 2024-04-17T10:04:23.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.422+0000 7fd318c00640 1 -- 172.21.15.32:0/302285085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd31404fda0 con 0x7fd314160c50 2024-04-17T10:04:23.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.422+0000 7fd30be00640 1 -- 172.21.15.32:0/302285085 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd30c0ad9f0 con 0x7fd314160c50 2024-04-17T10:04:23.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.426+0000 7fd30be00640 1 --2- 172.21.15.32:0/302285085 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd2f0068380 0x7fd2f006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:23.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.426+0000 7fd30be00640 1 -- 172.21.15.32:0/302285085 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7fd30c121b40 con 0x7fd314160c50 2024-04-17T10:04:23.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.426+0000 7fd313400640 1 --2- 172.21.15.32:0/302285085 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd2f0068380 0x7fd2f006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:23.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.426+0000 7fd313400640 1 --2- 172.21.15.32:0/302285085 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd2f0068380 0x7fd2f006a840 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd304007fd0 tx=0x7fd304009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:23.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.430+0000 7fd30be00640 1 -- 172.21.15.32:0/302285085 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd30c016560 con 0x7fd314160c50 2024-04-17T10:04:23.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:23.638+0000 7fd318c00640 1 -- 172.21.15.32:0/302285085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fd31413f190 con 0x7fd314160c50 2024-04-17T10:04:24.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.438+0000 7fd30be00640 1 -- 172.21.15.32:0/302285085 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v107) ==== 162+0+0 (secure 0 0 0) 0x7fd30c0f1a40 con 0x7fd314160c50 2024-04-17T10:04:24.433 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-17T10:04:24.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.442+0000 7fd318c00640 1 -- 172.21.15.32:0/302285085 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd2f0068380 msgr2=0x7fd2f006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:24.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.442+0000 7fd318c00640 1 --2- 172.21.15.32:0/302285085 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd2f0068380 0x7fd2f006a840 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd304007fd0 tx=0x7fd304009040 comp rx=0 tx=0).stop 2024-04-17T10:04:24.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.442+0000 7fd318c00640 1 -- 172.21.15.32:0/302285085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd314160c50 msgr2=0x7fd31415ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:24.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.442+0000 7fd318c00640 1 --2- 172.21.15.32:0/302285085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd314160c50 0x7fd31415ce90 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0b0040 tx=0x7fd30c019710 comp rx=0 tx=0).stop 2024-04-17T10:04:24.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.442+0000 7fd318c00640 1 -- 172.21.15.32:0/302285085 shutdown_connections 2024-04-17T10:04:24.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.442+0000 7fd318c00640 1 --2- 172.21.15.32:0/302285085 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd2f0068380 0x7fd2f006a840 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:24.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.442+0000 7fd318c00640 1 --2- 172.21.15.32:0/302285085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd314145e70 0x7fd3140cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:24.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.442+0000 7fd318c00640 1 --2- 172.21.15.32:0/302285085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd314160c50 0x7fd31415ce90 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:24.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.442+0000 7fd318c00640 1 --2- 172.21.15.32:0/302285085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3140b0eb0 0x7fd3140d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:24.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.442+0000 7fd318c00640 1 -- 172.21.15.32:0/302285085 >> 172.21.15.32:0/302285085 conn(0x7fd3140732d0 msgr2=0x7fd31406ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:24.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.442+0000 7fd318c00640 1 -- 172.21.15.32:0/302285085 shutdown_connections 2024-04-17T10:04:24.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.442+0000 7fd318c00640 1 -- 172.21.15.32:0/302285085 wait complete. 2024-04-17T10:04:24.464 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:04:24.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.614+0000 7f3f2dc00640 1 Processor -- start 2024-04-17T10:04:24.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.614+0000 7f3f2dc00640 1 -- start start 2024-04-17T10:04:24.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.614+0000 7f3f2dc00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3f2815b860 0x7f3f280cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:24.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.614+0000 7f3f2dc00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3f280ccaf0 0x7f3f280ce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:24.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.614+0000 7f3f2dc00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f280cdb80 con 0x7f3f280ceb80 2024-04-17T10:04:24.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.614+0000 7f3f2dc00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f3f280ccfd0 con 0x7f3f280ccaf0 2024-04-17T10:04:24.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.614+0000 7f3f2dc00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f3f280cd2b0 con 0x7f3f2815b860 2024-04-17T10:04:24.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.614+0000 7f3f26000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3f280ccaf0 0x7f3f280ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:24.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.614+0000 7f3f26000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3f280ccaf0 0x7f3f280ce640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60940/0 (socket says 172.21.15.32:60940) 2024-04-17T10:04:24.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.614+0000 7f3f26000640 1 -- 172.21.15.32:0/3339723607 learned_addr learned my addr 172.21.15.32:0/3339723607 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:24.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.614+0000 7f3f26a00640 1 --2- 172.21.15.32:0/3339723607 >> v2:172.21.15.144:3300/0 conn(0x7f3f2815b860 0x7f3f280cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:24.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.614+0000 7f3f26000640 1 -- 172.21.15.32:0/3339723607 >> v2:172.21.15.144:3300/0 conn(0x7f3f2815b860 msgr2=0x7f3f280cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:24.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.614+0000 7f3f26000640 1 --2- 172.21.15.32:0/3339723607 >> v2:172.21.15.144:3300/0 conn(0x7f3f2815b860 0x7f3f280cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:24.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.614+0000 7f3f26000640 1 -- 172.21.15.32:0/3339723607 >> v1:172.21.15.144:6789/0 conn(0x7f3f280ceb80 legacy=0x7f3f280b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:24.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.614+0000 7f3f26000640 1 -- 172.21.15.32:0/3339723607 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f280cde40 con 0x7f3f280ccaf0 2024-04-17T10:04:24.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.618+0000 7f3f26000640 1 --2- 172.21.15.32:0/3339723607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f280ccaf0 0x7f3f280ce640 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f3f28072740 tx=0x7f3f1401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9d79dc44e28646 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:24.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.618+0000 7f3f25600640 1 -- 172.21.15.32:0/3339723607 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f140aa610 con 0x7f3f280ccaf0 2024-04-17T10:04:24.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.618+0000 7f3f25600640 1 -- 172.21.15.32:0/3339723607 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f140af070 con 0x7f3f280ccaf0 2024-04-17T10:04:24.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.618+0000 7f3f25600640 1 -- 172.21.15.32:0/3339723607 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f140aa9d0 con 0x7f3f280ccaf0 2024-04-17T10:04:24.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.618+0000 7f3f2dc00640 1 -- 172.21.15.32:0/3339723607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f280ccaf0 msgr2=0x7f3f280ce640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:24.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.618+0000 7f3f2dc00640 1 --2- 172.21.15.32:0/3339723607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f280ccaf0 0x7f3f280ce640 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f3f28072740 tx=0x7f3f1401c490 comp rx=0 tx=0).stop 2024-04-17T10:04:24.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.618+0000 7f3f2dc00640 1 -- 172.21.15.32:0/3339723607 shutdown_connections 2024-04-17T10:04:24.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.618+0000 7f3f2dc00640 1 --2- 172.21.15.32:0/3339723607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f280ccaf0 0x7f3f280ce640 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:24.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.618+0000 7f3f2dc00640 1 --2- 172.21.15.32:0/3339723607 >> v2:172.21.15.144:3300/0 conn(0x7f3f2815b860 0x7f3f280cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:24.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.618+0000 7f3f2dc00640 1 -- 172.21.15.32:0/3339723607 >> 172.21.15.32:0/3339723607 conn(0x7f3f2809a9f0 msgr2=0x7f3f280b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:24.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.618+0000 7f3f2dc00640 1 -- 172.21.15.32:0/3339723607 shutdown_connections 2024-04-17T10:04:24.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.618+0000 7f3f2dc00640 1 -- 172.21.15.32:0/3339723607 wait complete. 2024-04-17T10:04:24.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f2dc00640 1 Processor -- start 2024-04-17T10:04:24.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f2dc00640 1 -- start start 2024-04-17T10:04:24.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f2dc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f280ccaf0 0x7f3f2815cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:24.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f2dc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f280ceb80 0x7f3f280c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:24.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f2dc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f2815b860 0x7f3f280c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:24.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f2dc00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3f281685a0 con 0x7f3f280ceb80 2024-04-17T10:04:24.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f2dc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3f28168720 con 0x7f3f2815b860 2024-04-17T10:04:24.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f2dc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3f281688a0 con 0x7f3f280ccaf0 2024-04-17T10:04:24.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f26000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f280ceb80 0x7f3f280c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:24.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f26000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f280ceb80 0x7f3f280c85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60956/0 (socket says 172.21.15.32:60956) 2024-04-17T10:04:24.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f26000640 1 -- 172.21.15.32:0/3384351762 learned_addr learned my addr 172.21.15.32:0/3384351762 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:24.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f26a00640 1 --2- 172.21.15.32:0/3384351762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f280ccaf0 0x7f3f2815cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:24.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f27400640 1 --2- 172.21.15.32:0/3384351762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f2815b860 0x7f3f280c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:24.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f26000640 1 -- 172.21.15.32:0/3384351762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f280ccaf0 msgr2=0x7f3f2815cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:24.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f26000640 1 --2- 172.21.15.32:0/3384351762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f280ccaf0 0x7f3f2815cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:24.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f26000640 1 -- 172.21.15.32:0/3384351762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f2815b860 msgr2=0x7f3f280c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:24.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f26000640 1 --2- 172.21.15.32:0/3384351762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f2815b860 0x7f3f280c8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:24.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f26000640 1 -- 172.21.15.32:0/3384351762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2815d320 con 0x7f3f280ceb80 2024-04-17T10:04:24.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f26000640 1 --2- 172.21.15.32:0/3384351762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f280ceb80 0x7f3f280c85a0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f3f140190a0 tx=0x7f3f1401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:24.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f1ea00640 1 -- 172.21.15.32:0/3384351762 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f140bd5f0 con 0x7f3f280ceb80 2024-04-17T10:04:24.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f1ea00640 1 -- 172.21.15.32:0/3384351762 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f140bd790 con 0x7f3f280ceb80 2024-04-17T10:04:24.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f1ea00640 1 -- 172.21.15.32:0/3384351762 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f140bdb40 con 0x7f3f280ceb80 2024-04-17T10:04:24.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f2dc00640 1 -- 172.21.15.32:0/3384351762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f2815d600 con 0x7f3f280ceb80 2024-04-17T10:04:24.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.622+0000 7f3f2dc00640 1 -- 172.21.15.32:0/3384351762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f2815dbc0 con 0x7f3f280ceb80 2024-04-17T10:04:24.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.626+0000 7f3f2dc00640 1 -- 172.21.15.32:0/3384351762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f28068a90 con 0x7f3f280ceb80 2024-04-17T10:04:24.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.626+0000 7f3f1ea00640 1 -- 172.21.15.32:0/3384351762 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3f140b33f0 con 0x7f3f280ceb80 2024-04-17T10:04:24.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.626+0000 7f3f1ea00640 1 --2- 172.21.15.32:0/3384351762 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3f08068380 0x7f3f0806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:24.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.626+0000 7f3f1ea00640 1 -- 172.21.15.32:0/3384351762 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f3f14121720 con 0x7f3f280ceb80 2024-04-17T10:04:24.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.626+0000 7f3f26a00640 1 --2- 172.21.15.32:0/3384351762 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3f08068380 0x7f3f0806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:24.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.626+0000 7f3f26a00640 1 --2- 172.21.15.32:0/3384351762 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3f08068380 0x7f3f0806a840 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3f18009ac0 tx=0x7f3f18009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:24.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.630+0000 7f3f1ea00640 1 -- 172.21.15.32:0/3384351762 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f3f140c3060 con 0x7f3f280ceb80 2024-04-17T10:04:24.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.830+0000 7f3f2dc00640 1 -- 172.21.15.32:0/3384351762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f280b7450 con 0x7f3f280ceb80 2024-04-17T10:04:24.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.830+0000 7f3f1ea00640 1 -- 172.21.15.32:0/3384351762 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5205 (secure 0 0 0) 0x7f3f140a7070 con 0x7f3f280ceb80 2024-04-17T10:04:24.824 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:24.824 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":108,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7586,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3601627312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3601627312},{"type":"v1","addr":"172.21.15.144:6834","nonce":3601627312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2382359917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2382359917},{"type":"v1","addr":"172.21.15.144:6835","nonce":2382359917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1857552160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1857552160},{"type":"v1","addr":"172.21.15.32:6835","nonce":1857552160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:03:37.619785+0000","modified":"2024-04-17T10:04:24.443822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9831},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9831":{"gid":9831,"name":"a","rank":0,"incarnation":108,"state":"up:replay","state_seq":1,"addr":"172.21.15.32:6837/1204910660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1204910660},{"type":"v1","addr":"172.21.15.32:6837","nonce":1204910660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-17T10:04:24.825 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 108 2024-04-17T10:04:24.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.830+0000 7f3f2dc00640 1 -- 172.21.15.32:0/3384351762 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3f08068380 msgr2=0x7f3f0806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:24.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.830+0000 7f3f2dc00640 1 --2- 172.21.15.32:0/3384351762 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3f08068380 0x7f3f0806a840 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3f18009ac0 tx=0x7f3f18009190 comp rx=0 tx=0).stop 2024-04-17T10:04:24.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.830+0000 7f3f2dc00640 1 -- 172.21.15.32:0/3384351762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f280ceb80 msgr2=0x7f3f280c85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:24.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.830+0000 7f3f2dc00640 1 --2- 172.21.15.32:0/3384351762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f280ceb80 0x7f3f280c85a0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f3f140190a0 tx=0x7f3f1401acd0 comp rx=0 tx=0).stop 2024-04-17T10:04:24.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.830+0000 7f3f2dc00640 1 -- 172.21.15.32:0/3384351762 shutdown_connections 2024-04-17T10:04:24.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.830+0000 7f3f2dc00640 1 --2- 172.21.15.32:0/3384351762 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f3f08068380 0x7f3f0806a840 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:24.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.830+0000 7f3f2dc00640 1 --2- 172.21.15.32:0/3384351762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f2815b860 0x7f3f280c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:24.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.830+0000 7f3f2dc00640 1 --2- 172.21.15.32:0/3384351762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f280ceb80 0x7f3f280c85a0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:24.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.830+0000 7f3f2dc00640 1 --2- 172.21.15.32:0/3384351762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f280ccaf0 0x7f3f2815cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:24.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.830+0000 7f3f2dc00640 1 -- 172.21.15.32:0/3384351762 >> 172.21.15.32:0/3384351762 conn(0x7f3f2809a9f0 msgr2=0x7f3f280703b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:24.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.830+0000 7f3f2dc00640 1 -- 172.21.15.32:0/3384351762 shutdown_connections 2024-04-17T10:04:24.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:24.830+0000 7f3f2dc00640 1 -- 172.21.15.32:0/3384351762 wait complete. 2024-04-17T10:04:25.247 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 108, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:03:37.619785+0000', 'modified': '2024-04-17T10:04:24.443822+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 113, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9831}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9831': {'gid': 9831, 'name': 'a', 'rank': 0, 'incarnation': 108, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.32:6837/1204910660', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6836', 'nonce': 1204910660}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 1204910660}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-17T10:04:25.247 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9831:up:replay 2024-04-17T10:04:26.248 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:04:26.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.398+0000 7fbe12200640 1 Processor -- start 2024-04-17T10:04:26.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.398+0000 7fbe12200640 1 -- start start 2024-04-17T10:04:26.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.402+0000 7fbe12200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbe0c15baa0 0x7fbe0c15bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:26.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.402+0000 7fbe12200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe0c072bb0 con 0x7fbe0c06ef60 2024-04-17T10:04:26.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.402+0000 7fbe12200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe0c0b3dc0 con 0x7fbe0c144e90 2024-04-17T10:04:26.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.402+0000 7fbe12200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fbe0c145270 con 0x7fbe0c15baa0 2024-04-17T10:04:26.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.402+0000 7fbe02a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbe0c15baa0 0x7fbe0c15bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:26.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.402+0000 7fbe0be00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbe0c06ef60 0x7fbe0c071e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50400/0 (socket says 172.21.15.32:50400) 2024-04-17T10:04:26.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.402+0000 7fbe0be00640 1 -- 172.21.15.32:0/1161086062 learned_addr learned my addr 172.21.15.32:0/1161086062 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:26.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.402+0000 7fbe02a00640 1 -- 172.21.15.32:0/1161086062 >> v1:172.21.15.144:6789/0 conn(0x7fbe0c144e90 legacy=0x7fbe0c0b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:26.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.402+0000 7fbe02a00640 1 -- 172.21.15.32:0/1161086062 >> v1:172.21.15.32:6789/0 conn(0x7fbe0c06ef60 legacy=0x7fbe0c071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:26.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.402+0000 7fbe02a00640 1 -- 172.21.15.32:0/1161086062 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0c0b40a0 con 0x7fbe0c15baa0 2024-04-17T10:04:26.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.402+0000 7fbe0aa00640 1 -- 172.21.15.32:0/1161086062 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2587583726 0 0) 0x7fbe0c072bb0 con 0x7fbe0c06ef60 2024-04-17T10:04:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.402+0000 7fbe02a00640 1 --2- 172.21.15.32:0/1161086062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe0c15baa0 0x7fbe0c15bf30 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fbdf40097c0 tx=0x7fbdf401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1966eaa601d321b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.402+0000 7fbe0aa00640 1 -- 172.21.15.32:0/1161086062 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf40b0070 con 0x7fbe0c15baa0 2024-04-17T10:04:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.402+0000 7fbe0aa00640 1 -- 172.21.15.32:0/1161086062 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdf40ab600 con 0x7fbe0c15baa0 2024-04-17T10:04:26.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.402+0000 7fbe0aa00640 1 -- 172.21.15.32:0/1161086062 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf40ab9d0 con 0x7fbe0c15baa0 2024-04-17T10:04:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.406+0000 7fbe12200640 1 -- 172.21.15.32:0/1161086062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe0c15baa0 msgr2=0x7fbe0c15bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.406+0000 7fbe12200640 1 --2- 172.21.15.32:0/1161086062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe0c15baa0 0x7fbe0c15bf30 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fbdf40097c0 tx=0x7fbdf401c4a0 comp rx=0 tx=0).stop 2024-04-17T10:04:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.406+0000 7fbe12200640 1 -- 172.21.15.32:0/1161086062 shutdown_connections 2024-04-17T10:04:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.406+0000 7fbe12200640 1 --2- 172.21.15.32:0/1161086062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe0c15baa0 0x7fbe0c15bf30 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.406+0000 7fbe12200640 1 -- 172.21.15.32:0/1161086062 >> 172.21.15.32:0/1161086062 conn(0x7fbe0c06ba30 msgr2=0x7fbe0c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.406+0000 7fbe12200640 1 -- 172.21.15.32:0/1161086062 shutdown_connections 2024-04-17T10:04:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.406+0000 7fbe12200640 1 -- 172.21.15.32:0/1161086062 wait complete. 2024-04-17T10:04:26.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.406+0000 7fbe12200640 1 Processor -- start 2024-04-17T10:04:26.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.406+0000 7fbe12200640 1 -- start start 2024-04-17T10:04:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.406+0000 7fbe12200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe0c06ef60 0x7fbe0c15a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.406+0000 7fbe12200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe0c144e90 0x7fbe0c13e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.406+0000 7fbe12200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe0c15baa0 0x7fbe0c139660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.406+0000 7fbe12200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c169ab0 con 0x7fbe0c15baa0 2024-04-17T10:04:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.406+0000 7fbe12200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c169930 con 0x7fbe0c144e90 2024-04-17T10:04:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.406+0000 7fbe12200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbe0c169c30 con 0x7fbe0c06ef60 2024-04-17T10:04:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.406+0000 7fbe0be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe0c15baa0 0x7fbe0c139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.406+0000 7fbe0be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe0c15baa0 0x7fbe0c139660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60974/0 (socket says 172.21.15.32:60974) 2024-04-17T10:04:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.406+0000 7fbe0be00640 1 -- 172.21.15.32:0/2838634642 learned_addr learned my addr 172.21.15.32:0/2838634642 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.406+0000 7fbe02a00640 1 --2- 172.21.15.32:0/2838634642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe0c144e90 0x7fbe0c13e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.406+0000 7fbe0b400640 1 --2- 172.21.15.32:0/2838634642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe0c06ef60 0x7fbe0c15a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.410+0000 7fbe0be00640 1 -- 172.21.15.32:0/2838634642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe0c06ef60 msgr2=0x7fbe0c15a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.410+0000 7fbe0be00640 1 --2- 172.21.15.32:0/2838634642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe0c06ef60 0x7fbe0c15a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.410+0000 7fbe0be00640 1 -- 172.21.15.32:0/2838634642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe0c144e90 msgr2=0x7fbe0c13e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.410+0000 7fbe0be00640 1 --2- 172.21.15.32:0/2838634642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe0c144e90 0x7fbe0c13e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.410+0000 7fbe0be00640 1 -- 172.21.15.32:0/2838634642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0c13a2e0 con 0x7fbe0c15baa0 2024-04-17T10:04:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.410+0000 7fbe0b400640 1 --2- 172.21.15.32:0/2838634642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe0c06ef60 0x7fbe0c15a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.410+0000 7fbe0be00640 1 --2- 172.21.15.32:0/2838634642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe0c15baa0 0x7fbe0c139660 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fbe0400c7d0 tx=0x7fbe0400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.410+0000 7fbe03e00640 1 -- 172.21.15.32:0/2838634642 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe04018070 con 0x7fbe0c15baa0 2024-04-17T10:04:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.410+0000 7fbe03e00640 1 -- 172.21.15.32:0/2838634642 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe0400a640 con 0x7fbe0c15baa0 2024-04-17T10:04:26.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.410+0000 7fbe03e00640 1 -- 172.21.15.32:0/2838634642 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0400aa30 con 0x7fbe0c15baa0 2024-04-17T10:04:26.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.410+0000 7fbe12200640 1 -- 172.21.15.32:0/2838634642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe0c13a5c0 con 0x7fbe0c15baa0 2024-04-17T10:04:26.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.410+0000 7fbe12200640 1 -- 172.21.15.32:0/2838634642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe0c15da20 con 0x7fbe0c15baa0 2024-04-17T10:04:26.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.410+0000 7fbe12200640 1 -- 172.21.15.32:0/2838634642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe0c068a90 con 0x7fbe0c15baa0 2024-04-17T10:04:26.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.410+0000 7fbe03e00640 1 -- 172.21.15.32:0/2838634642 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbe0401b070 con 0x7fbe0c15baa0 2024-04-17T10:04:26.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.410+0000 7fbe03e00640 1 --2- 172.21.15.32:0/2838634642 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbde8068380 0x7fbde806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:26.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.410+0000 7fbe03e00640 1 -- 172.21.15.32:0/2838634642 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7fbe04086010 con 0x7fbe0c15baa0 2024-04-17T10:04:26.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.410+0000 7fbe0b400640 1 --2- 172.21.15.32:0/2838634642 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbde8068380 0x7fbde806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.414+0000 7fbe0b400640 1 --2- 172.21.15.32:0/2838634642 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbde8068380 0x7fbde806a840 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c0b0b60 tx=0x7fbdfc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:26.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.414+0000 7fbe03e00640 1 -- 172.21.15.32:0/2838634642 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fbe04055e90 con 0x7fbe0c15baa0 2024-04-17T10:04:26.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.614+0000 7fbe12200640 1 -- 172.21.15.32:0/2838634642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe0c06bfb0 con 0x7fbe0c15baa0 2024-04-17T10:04:26.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.614+0000 7fbe03e00640 1 -- 172.21.15.32:0/2838634642 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+5205 (secure 0 0 0) 0x7fbe0405a190 con 0x7fbe0c15baa0 2024-04-17T10:04:26.611 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:26.611 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":110,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7586,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3601627312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3601627312},{"type":"v1","addr":"172.21.15.144:6834","nonce":3601627312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2382359917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2382359917},{"type":"v1","addr":"172.21.15.144:6835","nonce":2382359917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1857552160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1857552160},{"type":"v1","addr":"172.21.15.32:6835","nonce":1857552160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:03:37.619785+0000","modified":"2024-04-17T10:04:25.448982+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9831},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9831":{"gid":9831,"name":"a","rank":0,"incarnation":108,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.32:6837/1204910660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1204910660},{"type":"v1","addr":"172.21.15.32:6837","nonce":1204910660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-17T10:04:26.611 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 110 2024-04-17T10:04:26.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.618+0000 7fbe12200640 1 -- 172.21.15.32:0/2838634642 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbde8068380 msgr2=0x7fbde806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:26.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.618+0000 7fbe12200640 1 --2- 172.21.15.32:0/2838634642 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbde8068380 0x7fbde806a840 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c0b0b60 tx=0x7fbdfc009040 comp rx=0 tx=0).stop 2024-04-17T10:04:26.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.618+0000 7fbe12200640 1 -- 172.21.15.32:0/2838634642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe0c15baa0 msgr2=0x7fbe0c139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:26.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.618+0000 7fbe12200640 1 --2- 172.21.15.32:0/2838634642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe0c15baa0 0x7fbe0c139660 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fbe0400c7d0 tx=0x7fbe0400cc90 comp rx=0 tx=0).stop 2024-04-17T10:04:26.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.618+0000 7fbe12200640 1 -- 172.21.15.32:0/2838634642 shutdown_connections 2024-04-17T10:04:26.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.618+0000 7fbe12200640 1 --2- 172.21.15.32:0/2838634642 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbde8068380 0x7fbde806a840 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:26.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.618+0000 7fbe12200640 1 --2- 172.21.15.32:0/2838634642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe0c15baa0 0x7fbe0c139660 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:26.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.618+0000 7fbe12200640 1 --2- 172.21.15.32:0/2838634642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe0c144e90 0x7fbe0c13e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:26.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.618+0000 7fbe12200640 1 --2- 172.21.15.32:0/2838634642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe0c06ef60 0x7fbe0c15a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:26.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.618+0000 7fbe12200640 1 -- 172.21.15.32:0/2838634642 >> 172.21.15.32:0/2838634642 conn(0x7fbe0c06ba30 msgr2=0x7fbe0c0ad9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:26.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.618+0000 7fbe12200640 1 -- 172.21.15.32:0/2838634642 shutdown_connections 2024-04-17T10:04:26.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:26.618+0000 7fbe12200640 1 -- 172.21.15.32:0/2838634642 wait complete. 2024-04-17T10:04:27.035 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 110, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:03:37.619785+0000', 'modified': '2024-04-17T10:04:25.448982+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 113, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9831}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9831': {'gid': 9831, 'name': 'a', 'rank': 0, 'incarnation': 108, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.32:6837/1204910660', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6836', 'nonce': 1204910660}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 1204910660}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-17T10:04:27.035 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9831:up:rejoin 2024-04-17T10:04:28.036 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:04:28.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.190+0000 7f7532600640 1 Processor -- start 2024-04-17T10:04:28.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.194+0000 7f7532600640 1 -- start start 2024-04-17T10:04:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.194+0000 7f7532600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f752c144e90 0x7f752c0b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.194+0000 7f7532600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f752c0b4de0 0x7f752c15baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.194+0000 7f7532600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f752c072bb0 con 0x7f752c15bfe0 2024-04-17T10:04:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.194+0000 7f7532600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f752c15c430 con 0x7f752c144e90 2024-04-17T10:04:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.194+0000 7f7532600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f752c145270 con 0x7f752c0b4de0 2024-04-17T10:04:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.194+0000 7f752b400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f752c144e90 0x7f752c0b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.194+0000 7f752b400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f752c144e90 0x7f752c0b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60990/0 (socket says 172.21.15.32:60990) 2024-04-17T10:04:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.194+0000 7f752b400640 1 -- 172.21.15.32:0/1968527321 learned_addr learned my addr 172.21.15.32:0/1968527321 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.194+0000 7f7522a00640 1 --2- 172.21.15.32:0/1968527321 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f752c0b4de0 0x7f752c15baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.194+0000 7f752b400640 1 -- 172.21.15.32:0/1968527321 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f752c0b4de0 msgr2=0x7f752c15baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.194+0000 7f752b400640 1 --2- 172.21.15.32:0/1968527321 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f752c0b4de0 0x7f752c15baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.194+0000 7f752b400640 1 -- 172.21.15.32:0/1968527321 >> v1:172.21.15.144:6789/0 conn(0x7f752c15bfe0 legacy=0x7f752c06ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.194+0000 7f752b400640 1 -- 172.21.15.32:0/1968527321 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752c0b3dc0 con 0x7f752c144e90 2024-04-17T10:04:28.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.194+0000 7f7522a00640 1 --2- 172.21.15.32:0/1968527321 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f752c0b4de0 0x7f752c15baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:28.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.198+0000 7f752b400640 1 --2- 172.21.15.32:0/1968527321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f752c144e90 0x7f752c0b48a0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f75140098a0 tx=0x7f751401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af0e00eec4acb66 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:28.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.198+0000 7f752aa00640 1 -- 172.21.15.32:0/1968527321 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75140aa610 con 0x7f752c144e90 2024-04-17T10:04:28.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.198+0000 7f752aa00640 1 -- 172.21.15.32:0/1968527321 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75140af070 con 0x7f752c144e90 2024-04-17T10:04:28.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.198+0000 7f752aa00640 1 -- 172.21.15.32:0/1968527321 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75140aa9d0 con 0x7f752c144e90 2024-04-17T10:04:28.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.198+0000 7f7532600640 1 -- 172.21.15.32:0/1968527321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f752c144e90 msgr2=0x7f752c0b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:28.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.198+0000 7f7532600640 1 --2- 172.21.15.32:0/1968527321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f752c144e90 0x7f752c0b48a0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f75140098a0 tx=0x7f751401c4d0 comp rx=0 tx=0).stop 2024-04-17T10:04:28.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.198+0000 7f7532600640 1 -- 172.21.15.32:0/1968527321 shutdown_connections 2024-04-17T10:04:28.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.198+0000 7f7532600640 1 --2- 172.21.15.32:0/1968527321 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f752c0b4de0 0x7f752c15baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:28.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.198+0000 7f7532600640 1 --2- 172.21.15.32:0/1968527321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f752c144e90 0x7f752c0b48a0 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:28.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.198+0000 7f7532600640 1 -- 172.21.15.32:0/1968527321 >> 172.21.15.32:0/1968527321 conn(0x7f752c06ba30 msgr2=0x7f752c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:28.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.198+0000 7f7532600640 1 -- 172.21.15.32:0/1968527321 shutdown_connections 2024-04-17T10:04:28.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.198+0000 7f7532600640 1 -- 172.21.15.32:0/1968527321 wait complete. 2024-04-17T10:04:28.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.198+0000 7f7532600640 1 Processor -- start 2024-04-17T10:04:28.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.198+0000 7f7532600640 1 -- start start 2024-04-17T10:04:28.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.202+0000 7f7532600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f752c0b4de0 0x7f752c13b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:28.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.202+0000 7f7532600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f752c144e90 0x7f752c1fab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:28.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.202+0000 7f7532600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f752c15bfe0 0x7f752c0cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:28.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.202+0000 7f7532600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f752c072670 con 0x7f752c144e90 2024-04-17T10:04:28.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.202+0000 7f7532600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f752c06f8c0 con 0x7f752c15bfe0 2024-04-17T10:04:28.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.202+0000 7f7532600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f752c0727f0 con 0x7f752c0b4de0 2024-04-17T10:04:28.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.202+0000 7f7522a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f752c144e90 0x7f752c1fab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:28.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.202+0000 7f7522a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f752c144e90 0x7f752c1fab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60994/0 (socket says 172.21.15.32:60994) 2024-04-17T10:04:28.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.202+0000 7f7522a00640 1 -- 172.21.15.32:0/2839557624 learned_addr learned my addr 172.21.15.32:0/2839557624 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:28.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.202+0000 7f7522a00640 1 -- 172.21.15.32:0/2839557624 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f752c0b4de0 msgr2=0x7f752c13b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:28.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.202+0000 7f7522a00640 1 --2- 172.21.15.32:0/2839557624 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f752c0b4de0 0x7f752c13b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:28.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.202+0000 7f7522a00640 1 -- 172.21.15.32:0/2839557624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f752c15bfe0 msgr2=0x7f752c0cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:28.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.202+0000 7f7522a00640 1 --2- 172.21.15.32:0/2839557624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f752c15bfe0 0x7f752c0cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:28.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.202+0000 7f7522a00640 1 -- 172.21.15.32:0/2839557624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752c13d590 con 0x7f752c144e90 2024-04-17T10:04:28.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.202+0000 7f7522a00640 1 --2- 172.21.15.32:0/2839557624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f752c144e90 0x7f752c1fab80 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f751c00c7c0 tx=0x7f751c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:28.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.202+0000 7f7523e00640 1 -- 172.21.15.32:0/2839557624 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c007a30 con 0x7f752c144e90 2024-04-17T10:04:28.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.202+0000 7f7523e00640 1 -- 172.21.15.32:0/2839557624 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f751c007bd0 con 0x7f752c144e90 2024-04-17T10:04:28.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.202+0000 7f7532600640 1 -- 172.21.15.32:0/2839557624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f752c13bd00 con 0x7f752c144e90 2024-04-17T10:04:28.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.202+0000 7f7523e00640 1 -- 172.21.15.32:0/2839557624 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c0113f0 con 0x7f752c144e90 2024-04-17T10:04:28.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.202+0000 7f7532600640 1 -- 172.21.15.32:0/2839557624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f752c13c2c0 con 0x7f752c144e90 2024-04-17T10:04:28.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.202+0000 7f7532600640 1 -- 172.21.15.32:0/2839557624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f752c068a90 con 0x7f752c144e90 2024-04-17T10:04:28.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.202+0000 7f7523e00640 1 -- 172.21.15.32:0/2839557624 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f751c011590 con 0x7f752c144e90 2024-04-17T10:04:28.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.206+0000 7f7523e00640 1 --2- 172.21.15.32:0/2839557624 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f75080683b0 0x7f750806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:28.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.206+0000 7f7523e00640 1 -- 172.21.15.32:0/2839557624 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f751c086c40 con 0x7f752c144e90 2024-04-17T10:04:28.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.206+0000 7f752b400640 1 --2- 172.21.15.32:0/2839557624 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f75080683b0 0x7f750806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:28.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.206+0000 7f752b400640 1 --2- 172.21.15.32:0/2839557624 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f75080683b0 0x7f750806a870 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f75140ae000 tx=0x7f7514004740 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:28.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.210+0000 7f7523e00640 1 -- 172.21.15.32:0/2839557624 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f751c015020 con 0x7f752c144e90 2024-04-17T10:04:28.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.414+0000 7f7532600640 1 -- 172.21.15.32:0/2839557624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f752c06cdd0 con 0x7f752c144e90 2024-04-17T10:04:28.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.414+0000 7f7523e00640 1 -- 172.21.15.32:0/2839557624 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5212 (secure 0 0 0) 0x7f751c057100 con 0x7f752c144e90 2024-04-17T10:04:28.408 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:28.408 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7586,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3601627312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3601627312},{"type":"v1","addr":"172.21.15.144:6834","nonce":3601627312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2382359917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2382359917},{"type":"v1","addr":"172.21.15.144:6835","nonce":2382359917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1857552160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1857552160},{"type":"v1","addr":"172.21.15.32:6835","nonce":1857552160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:03:37.619785+0000","modified":"2024-04-17T10:04:27.457010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9831},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9831":{"gid":9831,"name":"a","rank":0,"incarnation":108,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6837/1204910660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1204910660},{"type":"v1","addr":"172.21.15.32:6837","nonce":1204910660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9831,"qdb_cluster":[9831]},"id":8}]} 2024-04-17T10:04:28.409 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 111 2024-04-17T10:04:28.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.414+0000 7f7532600640 1 -- 172.21.15.32:0/2839557624 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f75080683b0 msgr2=0x7f750806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:28.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.414+0000 7f7532600640 1 --2- 172.21.15.32:0/2839557624 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f75080683b0 0x7f750806a870 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f75140ae000 tx=0x7f7514004740 comp rx=0 tx=0).stop 2024-04-17T10:04:28.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.414+0000 7f7532600640 1 -- 172.21.15.32:0/2839557624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f752c144e90 msgr2=0x7f752c1fab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:28.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.414+0000 7f7532600640 1 --2- 172.21.15.32:0/2839557624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f752c144e90 0x7f752c1fab80 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f751c00c7c0 tx=0x7f751c00cc80 comp rx=0 tx=0).stop 2024-04-17T10:04:28.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.414+0000 7f7532600640 1 -- 172.21.15.32:0/2839557624 shutdown_connections 2024-04-17T10:04:28.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.414+0000 7f7532600640 1 --2- 172.21.15.32:0/2839557624 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f75080683b0 0x7f750806a870 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:28.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.414+0000 7f7532600640 1 --2- 172.21.15.32:0/2839557624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f752c15bfe0 0x7f752c0cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:28.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.414+0000 7f7532600640 1 --2- 172.21.15.32:0/2839557624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f752c144e90 0x7f752c1fab80 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:28.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.414+0000 7f7532600640 1 --2- 172.21.15.32:0/2839557624 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f752c0b4de0 0x7f752c13b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:28.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.414+0000 7f7532600640 1 -- 172.21.15.32:0/2839557624 >> 172.21.15.32:0/2839557624 conn(0x7f752c06ba30 msgr2=0x7f752c15e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:28.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.414+0000 7f7532600640 1 -- 172.21.15.32:0/2839557624 shutdown_connections 2024-04-17T10:04:28.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.414+0000 7f7532600640 1 -- 172.21.15.32:0/2839557624 wait complete. 2024-04-17T10:04:28.434 INFO:tasks.ceph.mds.a.smithi032.stderr:2024-04-17T10:04:28.438+0000 7fd889600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:04:28.832 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 111, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T10:03:37.619785+0000', 'modified': '2024-04-17T10:04:27.457010+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 113, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9831}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9831': {'gid': 9831, 'name': 'a', 'rank': 0, 'incarnation': 108, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.32:6837/1204910660', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6836', 'nonce': 1204910660}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 1204910660}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9831, 'qdb_cluster': [9831]} 2024-04-17T10:04:28.832 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T10:04:28.832 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-17T10:04:28.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.978+0000 7f67aa000640 1 Processor -- start 2024-04-17T10:04:28.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.978+0000 7f67aa000640 1 -- start start 2024-04-17T10:04:28.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67aa000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67a40b3340 0x7f67a415f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:28.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67aa000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67a415f870 con 0x7f67a4155a70 2024-04-17T10:04:28.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67aa000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67a415fb50 con 0x7f67a4155e50 2024-04-17T10:04:28.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67aa000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f67a40b42a0 con 0x7f67a40b3340 2024-04-17T10:04:28.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a3400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f67a4155a70 0x7f67a415d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50404/0 (socket says 172.21.15.32:50404) 2024-04-17T10:04:28.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a3400640 1 -- 172.21.15.32:0/1035046398 learned_addr learned my addr 172.21.15.32:0/1035046398 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:28.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a3e00640 1 --2- 172.21.15.32:0/1035046398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67a40b3340 0x7f67a415f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:28.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a2a00640 1 -- 172.21.15.32:0/1035046398 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2626214661 0 0) 0x7f67a415f870 con 0x7f67a4155a70 2024-04-17T10:04:28.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a2a00640 1 -- 172.21.15.32:0/1035046398 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6788003530 con 0x7f67a4155a70 2024-04-17T10:04:28.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a2a00640 1 -- 172.21.15.32:0/1035046398 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3355063434 0 0) 0x7f67a415fb50 con 0x7f67a4155e50 2024-04-17T10:04:28.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a2a00640 1 -- 172.21.15.32:0/1035046398 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67a415f870 con 0x7f67a4155e50 2024-04-17T10:04:28.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a2a00640 1 -- 172.21.15.32:0/1035046398 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 531957500 0 0) 0x7f6788003530 con 0x7f67a4155a70 2024-04-17T10:04:28.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a2a00640 1 -- 172.21.15.32:0/1035046398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67a40b3340 msgr2=0x7f67a415f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:28.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a2a00640 1 --2- 172.21.15.32:0/1035046398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67a40b3340 0x7f67a415f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:28.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a2a00640 1 -- 172.21.15.32:0/1035046398 >> v1:172.21.15.144:6789/0 conn(0x7f67a4155e50 legacy=0x7f67a415c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:28.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a2a00640 1 -- 172.21.15.32:0/1035046398 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a415fe30 con 0x7f67a4155a70 2024-04-17T10:04:28.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a2a00640 1 -- 172.21.15.32:0/1035046398 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f6794001680 con 0x7f67a4155a70 2024-04-17T10:04:28.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a2a00640 1 -- 172.21.15.32:0/1035046398 >> v1:172.21.15.32:6789/0 conn(0x7f67a4155a70 legacy=0x7f67a415d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:28.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a2a00640 1 --2- 172.21.15.32:0/1035046398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6788003b80 0x7f6788003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:28.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a2a00640 1 --2- 172.21.15.32:0/1035046398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67a40b3340 0x7f6788004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:28.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a2a00640 1 --2- 172.21.15.32:0/1035046398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6788004d50 0x7f67880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:28.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a2a00640 1 -- 172.21.15.32:0/1035046398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f67a40b42a0 con 0x7f6788004d50 2024-04-17T10:04:28.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a2a00640 1 -- 172.21.15.32:0/1035046398 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f678800dc90 con 0x7f6788003b80 2024-04-17T10:04:28.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a2a00640 1 -- 172.21.15.32:0/1035046398 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f678800de40 con 0x7f67a40b3340 2024-04-17T10:04:28.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a3e00640 1 --2- 172.21.15.32:0/1035046398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6788004d50 0x7f67880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:28.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a3400640 1 --2- 172.21.15.32:0/1035046398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6788003b80 0x7f6788003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:28.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f679aa00640 1 --2- 172.21.15.32:0/1035046398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67a40b3340 0x7f6788004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:28.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a3e00640 1 -- 172.21.15.32:0/1035046398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67a40b3340 msgr2=0x7f6788004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:28.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a3e00640 1 --2- 172.21.15.32:0/1035046398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67a40b3340 0x7f6788004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:28.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a3e00640 1 -- 172.21.15.32:0/1035046398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6788003b80 msgr2=0x7f6788003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:28.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a3e00640 1 --2- 172.21.15.32:0/1035046398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6788003b80 0x7f6788003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:28.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f67a3e00640 1 -- 172.21.15.32:0/1035046398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6788003530 con 0x7f6788004d50 2024-04-17T10:04:28.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.982+0000 7f679aa00640 1 --2- 172.21.15.32:0/1035046398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67a40b3340 0x7f6788004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:28.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.986+0000 7f67a3e00640 1 --2- 172.21.15.32:0/1035046398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6788004d50 0x7f67880051d0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f679c009fc0 tx=0x7f679c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea306f0bfad6ffd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:28.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.986+0000 7f67a2a00640 1 -- 172.21.15.32:0/1035046398 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c0b2070 con 0x7f6788004d50 2024-04-17T10:04:28.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.986+0000 7f67a2a00640 1 -- 172.21.15.32:0/1035046398 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f679c0a5c90 con 0x7f6788004d50 2024-04-17T10:04:28.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.986+0000 7f67a2a00640 1 -- 172.21.15.32:0/1035046398 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c009c10 con 0x7f6788004d50 2024-04-17T10:04:28.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.986+0000 7f67aa000640 1 -- 172.21.15.32:0/1035046398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6788004d50 msgr2=0x7f67880051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:28.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.986+0000 7f67aa000640 1 --2- 172.21.15.32:0/1035046398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6788004d50 0x7f67880051d0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f679c009fc0 tx=0x7f679c01cc80 comp rx=0 tx=0).stop 2024-04-17T10:04:28.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.986+0000 7f67a3e00640 1 -- 172.21.15.32:0/1035046398 reap_dead start 2024-04-17T10:04:28.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.986+0000 7f67aa000640 1 -- 172.21.15.32:0/1035046398 shutdown_connections 2024-04-17T10:04:28.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.986+0000 7f67aa000640 1 -- 172.21.15.32:0/1035046398 >> 172.21.15.32:0/1035046398 conn(0x7f67a406ba30 msgr2=0x7f67a406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:28.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.986+0000 7f67aa000640 1 -- 172.21.15.32:0/1035046398 shutdown_connections 2024-04-17T10:04:28.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.986+0000 7f67aa000640 1 -- 172.21.15.32:0/1035046398 wait complete. 2024-04-17T10:04:28.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f67aa000640 1 Processor -- start 2024-04-17T10:04:28.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f67aa000640 1 -- start start 2024-04-17T10:04:28.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f67aa000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6788004d50 0x7f67a413ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:28.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f67aa000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67a4072510 0x7f67a414cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:28.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f67aa000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67a4072990 0x7f67a414d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:28.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f67aa000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f67a4160110 con 0x7f6788004d50 2024-04-17T10:04:28.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f67aa000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f67a40d1250 con 0x7f67a4072510 2024-04-17T10:04:28.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f67aa000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f67a415d7a0 con 0x7f67a4072990 2024-04-17T10:04:28.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f67a3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6788004d50 0x7f67a413ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:28.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f67a3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6788004d50 0x7f67a413ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32768/0 (socket says 172.21.15.32:32768) 2024-04-17T10:04:28.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f67a3400640 1 -- 172.21.15.32:0/2426447163 learned_addr learned my addr 172.21.15.32:0/2426447163 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:28.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f67a3e00640 1 --2- 172.21.15.32:0/2426447163 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67a4072990 0x7f67a414d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:28.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f679aa00640 1 --2- 172.21.15.32:0/2426447163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67a4072510 0x7f67a414cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:28.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f67a3400640 1 -- 172.21.15.32:0/2426447163 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67a4072990 msgr2=0x7f67a414d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:28.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f67a3400640 1 --2- 172.21.15.32:0/2426447163 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67a4072990 0x7f67a414d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:28.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f67a3400640 1 -- 172.21.15.32:0/2426447163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67a4072510 msgr2=0x7f67a414cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:28.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f67a3400640 1 --2- 172.21.15.32:0/2426447163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67a4072510 0x7f67a414cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:28.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f67a3400640 1 -- 172.21.15.32:0/2426447163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a4149460 con 0x7f6788004d50 2024-04-17T10:04:28.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f67a3400640 1 --2- 172.21.15.32:0/2426447163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6788004d50 0x7f67a413ddb0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f6794012510 tx=0x7f67940129d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:28.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f679be00640 1 -- 172.21.15.32:0/2426447163 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679401c070 con 0x7f6788004d50 2024-04-17T10:04:28.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f679be00640 1 -- 172.21.15.32:0/2426447163 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6794014810 con 0x7f6788004d50 2024-04-17T10:04:28.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f67aa000640 1 -- 172.21.15.32:0/2426447163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a40ce450 con 0x7f6788004d50 2024-04-17T10:04:28.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f679be00640 1 -- 172.21.15.32:0/2426447163 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6794014c00 con 0x7f6788004d50 2024-04-17T10:04:28.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.990+0000 7f67aa000640 1 -- 172.21.15.32:0/2426447163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a40cea10 con 0x7f6788004d50 2024-04-17T10:04:28.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.994+0000 7f679be00640 1 -- 172.21.15.32:0/2426447163 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f679401f070 con 0x7f6788004d50 2024-04-17T10:04:28.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.994+0000 7f679be00640 1 --2- 172.21.15.32:0/2426447163 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6780068380 0x7f678006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:28.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.994+0000 7f679be00640 1 -- 172.21.15.32:0/2426447163 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f6794089510 con 0x7f6788004d50 2024-04-17T10:04:28.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.994+0000 7f679aa00640 1 --2- 172.21.15.32:0/2426447163 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6780068380 0x7f678006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:28.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.994+0000 7f67ad3ec000 1 Processor -- start 2024-04-17T10:04:28.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.994+0000 7f67ad3ec000 1 -- start start 2024-04-17T10:04:28.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.994+0000 7f679aa00640 1 --2- 172.21.15.32:0/2426447163 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6780068380 0x7f678006a840 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f67a415ee40 tx=0x7f678c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:28.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.994+0000 7f67ad3ec000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c946e8fb50 0x55c946e8ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:28.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.994+0000 7f67ad3ec000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e90470 0x55c946e457b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:28.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.994+0000 7f67ad3ec000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c946e45cf0 0x55c946e460d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:28.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.994+0000 7f67ad3ec000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c946c934d0 con 0x55c946e90470 2024-04-17T10:04:28.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.994+0000 7f67ad3ec000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c946c2dbf0 con 0x55c946e45cf0 2024-04-17T10:04:28.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.994+0000 7f67ad3ec000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55c946c8f490 con 0x55c946e8fb50 2024-04-17T10:04:28.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.994+0000 7f679aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e90470 0x55c946e457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:28.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.994+0000 7f679aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e90470 0x55c946e457b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32784/0 (socket says 172.21.15.32:32784) 2024-04-17T10:04:28.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.994+0000 7f679aa00640 1 -- 172.21.15.32:0/3924112236 learned_addr learned my addr 172.21.15.32:0/3924112236 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:28.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.994+0000 7f67a3400640 1 --2- 172.21.15.32:0/3924112236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c946e8fb50 0x55c946e8ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:28.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.994+0000 7f67a3e00640 1 --2- 172.21.15.32:0/3924112236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c946e45cf0 0x55c946e460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:28.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f679aa00640 1 -- 172.21.15.32:0/3924112236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c946e8fb50 msgr2=0x55c946e8ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:28.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f679aa00640 1 --2- 172.21.15.32:0/3924112236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c946e8fb50 0x55c946e8ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:28.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f679aa00640 1 -- 172.21.15.32:0/3924112236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c946e45cf0 msgr2=0x55c946e460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:28.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f679aa00640 1 --2- 172.21.15.32:0/3924112236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c946e45cf0 0x55c946e460d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:28.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f679aa00640 1 -- 172.21.15.32:0/3924112236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c946c19e80 con 0x55c946e90470 2024-04-17T10:04:28.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f679aa00640 1 --2- 172.21.15.32:0/3924112236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e90470 0x55c946e457b0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f678c0016b0 tx=0x7f678c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1e7e0b9b51a680 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:28.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f6793e00640 1 -- 172.21.15.32:0/3924112236 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c027070 con 0x55c946e90470 2024-04-17T10:04:28.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f6793e00640 1 -- 172.21.15.32:0/3924112236 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f678c014840 con 0x55c946e90470 2024-04-17T10:04:28.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f6793e00640 1 -- 172.21.15.32:0/3924112236 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c014bf0 con 0x55c946e90470 2024-04-17T10:04:28.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f67ad3ec000 1 -- 172.21.15.32:0/3924112236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e90470 msgr2=0x55c946e457b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:28.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f67ad3ec000 1 --2- 172.21.15.32:0/3924112236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e90470 0x55c946e457b0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f678c0016b0 tx=0x7f678c013000 comp rx=0 tx=0).stop 2024-04-17T10:04:28.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f67ad3ec000 1 -- 172.21.15.32:0/3924112236 shutdown_connections 2024-04-17T10:04:28.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f67ad3ec000 1 --2- 172.21.15.32:0/3924112236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c946e45cf0 0x55c946e460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:28.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f67ad3ec000 1 --2- 172.21.15.32:0/3924112236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e90470 0x55c946e457b0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:28.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f67ad3ec000 1 --2- 172.21.15.32:0/3924112236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c946e8fb50 0x55c946e8ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:28.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f67ad3ec000 1 -- 172.21.15.32:0/3924112236 >> 172.21.15.32:0/3924112236 conn(0x55c946e4b470 msgr2=0x55c946e44090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:28.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f67ad3ec000 1 -- 172.21.15.32:0/3924112236 shutdown_connections 2024-04-17T10:04:28.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f67ad3ec000 1 -- 172.21.15.32:0/3924112236 wait complete. 2024-04-17T10:04:28.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f67ad3ec000 1 -- start start 2024-04-17T10:04:28.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f67ad3ec000 1 Processor -- start 2024-04-17T10:04:28.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f67ad3ec000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c946e45cf0 0x55c946e4a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:28.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f67ad3ec000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c946e8fb50 0x55c946e7d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:28.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f67ad3ec000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e90470 0x55c946e7d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:28.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f67ad3ec000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c946c50a60 con 0x55c946e90470 2024-04-17T10:04:28.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f67ad3ec000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c946c37e80 con 0x55c946e45cf0 2024-04-17T10:04:28.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f67ad3ec000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55c946c4ecf0 con 0x55c946e8fb50 2024-04-17T10:04:28.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f67ad3ec000 10 client.0 _ll_register_callbacks cb 0x55c946e30a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:04:28.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f67a3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e90470 0x55c946e7d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:28.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f67a3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e90470 0x55c946e7d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32786/0 (socket says 172.21.15.32:32786) 2024-04-17T10:04:28.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f67a3e00640 1 -- 172.21.15.32:0/740856947 learned_addr learned my addr 172.21.15.32:0/740856947 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:28.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f67a3400640 1 --2- 172.21.15.32:0/740856947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c946e45cf0 0x55c946e4a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:28.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:28.998+0000 7f679aa00640 1 --2- 172.21.15.32:0/740856947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c946e8fb50 0x55c946e7d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:28.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67a3e00640 1 -- 172.21.15.32:0/740856947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c946e8fb50 msgr2=0x55c946e7d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:28.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67a3e00640 1 --2- 172.21.15.32:0/740856947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c946e8fb50 0x55c946e7d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:28.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67a3e00640 1 -- 172.21.15.32:0/740856947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c946e45cf0 msgr2=0x55c946e4a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:28.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67a3e00640 1 --2- 172.21.15.32:0/740856947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c946e45cf0 0x55c946e4a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:28.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67a3e00640 1 -- 172.21.15.32:0/740856947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c946c2de40 con 0x55c946e90470 2024-04-17T10:04:28.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67a3e00640 1 --2- 172.21.15.32:0/740856947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e90470 0x55c946e7d980 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f679c0b0040 tx=0x7f6788005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc34119bac49274 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:28.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f6790c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:04:28.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f6790c00640 1 -- 172.21.15.32:0/740856947 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c0b2040 con 0x55c946e90470 2024-04-17T10:04:28.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f6790c00640 1 -- 172.21.15.32:0/740856947 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f679c0039a0 con 0x55c946e90470 2024-04-17T10:04:28.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67ad3ec000 1 -- 172.21.15.32:0/740856947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f678c014840 con 0x55c946e90470 2024-04-17T10:04:28.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f6790c00640 1 -- 172.21.15.32:0/740856947 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) ==== 24+0+0 (secure 0 0 0) 0x7f679c009cd0 con 0x55c946e90470 2024-04-17T10:04:28.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67ad3ec000 10 client.9921 fetch_fsmap learned FSMap version 111 2024-04-17T10:04:28.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67ad3ec000 1 -- 172.21.15.32:0/740856947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) -- 0x55c946c5ee00 con 0x55c946e90470 2024-04-17T10:04:28.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f6790c00640 1 -- 172.21.15.32:0/740856947 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f678c014840 con 0x55c946e90470 2024-04-17T10:04:28.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f6790c00640 1 -- 172.21.15.32:0/740856947 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 111) ==== 3495+0+0 (secure 0 0 0) 0x7f679c0c4480 con 0x55c946e90470 2024-04-17T10:04:28.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67ad3ec000 10 client.9921 fetch_fsmap finished waiting for FSMap version 111 2024-04-17T10:04:28.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67ad3ec000 10 client.9921 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T10:04:28.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67ad3ec000 20 client.9921 populate_metadata read hostname 'smithi032' 2024-04-17T10:04:28.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67ad3ec000 1 --2- 172.21.15.32:0/740856947 >> [v2:172.21.15.32:6836/1204910660,v1:172.21.15.32:6837/1204910660] conn(0x55c946e9f0b0 0x55c946ea14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:28.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67ad3ec000 4 client.9921 mds_command: new command op to 9831 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T10:04:29.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67ad3ec000 1 -- 172.21.15.32:0/740856947 --> [v2:172.21.15.32:6836/1204910660,v1:172.21.15.32:6837/1204910660] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c946c2cbc0 con 0x55c946e9f0b0 2024-04-17T10:04:29.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67a3400640 1 --2- 172.21.15.32:0/740856947 >> [v2:172.21.15.32:6836/1204910660,v1:172.21.15.32:6837/1204910660] conn(0x55c946e9f0b0 0x55c946ea14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:29.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67a3400640 1 --2- 172.21.15.32:0/740856947 >> [v2:172.21.15.32:6836/1204910660,v1:172.21.15.32:6837/1204910660] conn(0x55c946e9f0b0 0x55c946ea14a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1a756a01bb64161 server_cookie=ab29b9d254292b1c in_seq=0 out_seq=0 2024-04-17T10:04:29.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f6790c00640 10 client.9921 ms_handle_connect on v2:172.21.15.32:6836/1204910660 2024-04-17T10:04:29.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f6790c00640 1 -- 172.21.15.32:0/740856947 <== mds.0 v2:172.21.15.32:6836/1204910660 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c946c2cbc0 con 0x55c946e9f0b0 2024-04-17T10:04:29.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f6790c00640 10 client.9921 handle_command_reply: tid=0 2024-04-17T10:04:29.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67ad3ec000 1 client.9921 shutdown 2024-04-17T10:04:29.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67ad3ec000 1 -- 172.21.15.32:0/740856947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e90470 msgr2=0x55c946e7d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:29.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67ad3ec000 1 --2- 172.21.15.32:0/740856947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e90470 0x55c946e7d980 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f679c0b0040 tx=0x7f6788005840 comp rx=0 tx=0).stop 2024-04-17T10:04:29.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67ad3ec000 1 -- 172.21.15.32:0/740856947 shutdown_connections 2024-04-17T10:04:29.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67ad3ec000 1 --2- 172.21.15.32:0/740856947 >> [v2:172.21.15.32:6836/1204910660,v1:172.21.15.32:6837/1204910660] conn(0x55c946e9f0b0 0x55c946ea14a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67ad3ec000 1 --2- 172.21.15.32:0/740856947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c946e8fb50 0x55c946e7d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67ad3ec000 1 --2- 172.21.15.32:0/740856947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e90470 0x55c946e7d980 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67ad3ec000 1 --2- 172.21.15.32:0/740856947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c946e45cf0 0x55c946e4a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f6790c00640 0 client.9921 ms_handle_reset on v2:172.21.15.32:6836/1204910660 2024-04-17T10:04:29.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67ad3ec000 1 -- 172.21.15.32:0/740856947 >> 172.21.15.32:0/740856947 conn(0x55c946e4b470 msgr2=0x55c946e447a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:29.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67ad3ec000 1 -- 172.21.15.32:0/740856947 shutdown_connections 2024-04-17T10:04:29.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67ad3ec000 1 -- 172.21.15.32:0/740856947 wait complete. 2024-04-17T10:04:29.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.002+0000 7f67ad3ec000 20 client.9921 trim_cache size 0 max 16384 2024-04-17T10:04:29.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f67ad3ec000 1 Processor -- start 2024-04-17T10:04:29.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f67ad3ec000 1 -- start start 2024-04-17T10:04:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f67ad3ec000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e4b470 0x55c946ea04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f67ad3ec000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c946e45cf0 0x55c946ea09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f67ad3ec000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c946e8fb50 0x55c946ea0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f67ad3ec000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f67a41fc280 con 0x55c946e4b470 2024-04-17T10:04:29.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f67ad3ec000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f67a40d0260 con 0x55c946e8fb50 2024-04-17T10:04:29.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f67ad3ec000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f67a41fbc60 con 0x55c946e45cf0 2024-04-17T10:04:29.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f67a3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e4b470 0x55c946ea04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:29.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f67a3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e4b470 0x55c946ea04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32798/0 (socket says 172.21.15.32:32798) 2024-04-17T10:04:29.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f67a3400640 1 -- 172.21.15.32:0/3494284492 learned_addr learned my addr 172.21.15.32:0/3494284492 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:29.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f67a3400640 1 -- 172.21.15.32:0/3494284492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c946e45cf0 msgr2=0x55c946ea09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:29.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f67a3400640 1 --2- 172.21.15.32:0/3494284492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c946e45cf0 0x55c946ea09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f67a3400640 1 -- 172.21.15.32:0/3494284492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c946e8fb50 msgr2=0x55c946ea0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:29.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f67a3400640 1 --2- 172.21.15.32:0/3494284492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c946e8fb50 0x55c946ea0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f67a3400640 1 -- 172.21.15.32:0/3494284492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c946d14d30 con 0x55c946e4b470 2024-04-17T10:04:29.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f67a3400640 1 --2- 172.21.15.32:0/3494284492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e4b470 0x55c946ea04a0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f67940017f0 tx=0x7f6794059140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c92cb37c07110d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:29.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f6793400640 1 -- 172.21.15.32:0/3494284492 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6794057540 con 0x55c946e4b470 2024-04-17T10:04:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f6793400640 1 -- 172.21.15.32:0/3494284492 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6794014da0 con 0x55c946e4b470 2024-04-17T10:04:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f6793400640 1 -- 172.21.15.32:0/3494284492 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67940594a0 con 0x55c946e4b470 2024-04-17T10:04:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f67ad3ec000 1 -- 172.21.15.32:0/3494284492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e4b470 msgr2=0x55c946ea04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f67ad3ec000 1 --2- 172.21.15.32:0/3494284492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e4b470 0x55c946ea04a0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f67940017f0 tx=0x7f6794059140 comp rx=0 tx=0).stop 2024-04-17T10:04:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f67ad3ec000 1 -- 172.21.15.32:0/3494284492 shutdown_connections 2024-04-17T10:04:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f67ad3ec000 1 --2- 172.21.15.32:0/3494284492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c946e45cf0 0x55c946ea09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f67ad3ec000 1 --2- 172.21.15.32:0/3494284492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c946e8fb50 0x55c946ea0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.022+0000 7f67ad3ec000 1 --2- 172.21.15.32:0/3494284492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e4b470 0x55c946ea04a0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67ad3ec000 1 -- 172.21.15.32:0/3494284492 >> 172.21.15.32:0/3494284492 conn(0x55c946e8f3d0 msgr2=0x55c946e7cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67ad3ec000 1 -- 172.21.15.32:0/3494284492 shutdown_connections 2024-04-17T10:04:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67ad3ec000 1 -- 172.21.15.32:0/3494284492 wait complete. 2024-04-17T10:04:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67ad3ec000 1 -- start start 2024-04-17T10:04:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67ad3ec000 1 Processor -- start 2024-04-17T10:04:29.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67ad3ec000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c946e45cf0 0x55c946ea4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:29.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67ad3ec000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e4b470 0x55c946ebb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:29.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67ad3ec000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c946e8fb50 0x55c946ebb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:29.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67ad3ec000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55c946c97b00 con 0x55c946e4b470 2024-04-17T10:04:29.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67ad3ec000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c946c927e0 con 0x55c946e45cf0 2024-04-17T10:04:29.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67ad3ec000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55c946ca06e0 con 0x55c946e8fb50 2024-04-17T10:04:29.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67ad3ec000 10 client.0 _ll_register_callbacks cb 0x7f6788003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T10:04:29.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f679aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e4b470 0x55c946ebb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:29.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f679aa00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e4b470 0x55c946ebb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32806/0 (socket says 172.21.15.32:32806) 2024-04-17T10:04:29.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f679aa00640 1 -- 172.21.15.32:0/3006314878 learned_addr learned my addr 172.21.15.32:0/3006314878 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:29.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67a3e00640 1 --2- 172.21.15.32:0/3006314878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c946e8fb50 0x55c946ebb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:29.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67a3400640 1 --2- 172.21.15.32:0/3006314878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c946e45cf0 0x55c946ea4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:29.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f679aa00640 1 -- 172.21.15.32:0/3006314878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c946e8fb50 msgr2=0x55c946ebb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:29.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f679aa00640 1 --2- 172.21.15.32:0/3006314878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c946e8fb50 0x55c946ebb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f679aa00640 1 -- 172.21.15.32:0/3006314878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c946e45cf0 msgr2=0x55c946ea4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:29.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f679aa00640 1 --2- 172.21.15.32:0/3006314878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c946e45cf0 0x55c946ea4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f679aa00640 1 -- 172.21.15.32:0/3006314878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c946c257c0 con 0x55c946e4b470 2024-04-17T10:04:29.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67a3400640 1 --2- 172.21.15.32:0/3006314878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c946e45cf0 0x55c946ea4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:29.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f679aa00640 1 --2- 172.21.15.32:0/3006314878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e4b470 0x55c946ebb030 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f678c020000 tx=0x7f678c007a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a512b5384bbae7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f6790c00640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-04-17T10:04:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f6790c00640 1 -- 172.21.15.32:0/3006314878 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c027040 con 0x55c946e4b470 2024-04-17T10:04:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f6790c00640 1 -- 172.21.15.32:0/3006314878 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f678c02e430 con 0x55c946e4b470 2024-04-17T10:04:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67ad3ec000 1 -- 172.21.15.32:0/3006314878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6794014da0 con 0x55c946e4b470 2024-04-17T10:04:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f6790c00640 1 -- 172.21.15.32:0/3006314878 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) ==== 24+0+0 (secure 0 0 0) 0x7f678c03b050 con 0x55c946e4b470 2024-04-17T10:04:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67ad3ec000 10 client.9927 fetch_fsmap learned FSMap version 111 2024-04-17T10:04:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67ad3ec000 1 -- 172.21.15.32:0/3006314878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) -- 0x55c946c38650 con 0x55c946e4b470 2024-04-17T10:04:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f6790c00640 1 -- 172.21.15.32:0/3006314878 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6794014da0 con 0x55c946e4b470 2024-04-17T10:04:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f6790c00640 1 -- 172.21.15.32:0/3006314878 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 111) ==== 3495+0+0 (secure 0 0 0) 0x7f678c03e020 con 0x55c946e4b470 2024-04-17T10:04:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67ad3ec000 10 client.9927 fetch_fsmap finished waiting for FSMap version 111 2024-04-17T10:04:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67ad3ec000 10 client.9927 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T10:04:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67ad3ec000 20 client.9927 populate_metadata read hostname 'smithi032' 2024-04-17T10:04:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67ad3ec000 1 --2- 172.21.15.32:0/3006314878 >> [v2:172.21.15.32:6836/1204910660,v1:172.21.15.32:6837/1204910660] conn(0x55c946e90470 0x55c946efea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:29.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67ad3ec000 4 client.9927 mds_command: new command op to 9831 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T10:04:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.026+0000 7f67ad3ec000 1 -- 172.21.15.32:0/3006314878 --> [v2:172.21.15.32:6836/1204910660,v1:172.21.15.32:6837/1204910660] -- command(tid 0: {"prefix": "status"}) -- 0x55c946ca0ef0 con 0x55c946e90470 2024-04-17T10:04:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67a3400640 1 --2- 172.21.15.32:0/3006314878 >> [v2:172.21.15.32:6836/1204910660,v1:172.21.15.32:6837/1204910660] conn(0x55c946e90470 0x55c946efea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67a3400640 1 --2- 172.21.15.32:0/3006314878 >> [v2:172.21.15.32:6836/1204910660,v1:172.21.15.32:6837/1204910660] conn(0x55c946e90470 0x55c946efea90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=283fa8c235c2baa9 server_cookie=e8e50cf707dbe56d in_seq=0 out_seq=0 2024-04-17T10:04:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f6790c00640 10 client.9927 ms_handle_connect on v2:172.21.15.32:6836/1204910660 2024-04-17T10:04:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f6790c00640 1 -- 172.21.15.32:0/3006314878 <== mds.0 v2:172.21.15.32:6836/1204910660 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55c946ca0ef0 con 0x55c946e90470 2024-04-17T10:04:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f6790c00640 10 client.9927 handle_command_reply: tid=0 2024-04-17T10:04:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67ad3ec000 1 client.9927 shutdown 2024-04-17T10:04:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67ad3ec000 1 -- 172.21.15.32:0/3006314878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e4b470 msgr2=0x55c946ebb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67ad3ec000 1 --2- 172.21.15.32:0/3006314878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e4b470 0x55c946ebb030 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f678c020000 tx=0x7f678c007a60 comp rx=0 tx=0).stop 2024-04-17T10:04:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67ad3ec000 1 -- 172.21.15.32:0/3006314878 shutdown_connections 2024-04-17T10:04:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67ad3ec000 1 --2- 172.21.15.32:0/3006314878 >> [v2:172.21.15.32:6836/1204910660,v1:172.21.15.32:6837/1204910660] conn(0x55c946e90470 0x55c946efea90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67ad3ec000 1 --2- 172.21.15.32:0/3006314878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c946e8fb50 0x55c946ebb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67ad3ec000 1 --2- 172.21.15.32:0/3006314878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c946e4b470 0x55c946ebb030 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67ad3ec000 1 --2- 172.21.15.32:0/3006314878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c946e45cf0 0x55c946ea4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f6790c00640 0 client.9927 ms_handle_reset on v2:172.21.15.32:6836/1204910660 2024-04-17T10:04:29.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67ad3ec000 1 -- 172.21.15.32:0/3006314878 >> 172.21.15.32:0/3006314878 conn(0x55c946e8f3d0 msgr2=0x55c946e7d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:29.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67ad3ec000 1 -- 172.21.15.32:0/3006314878 shutdown_connections 2024-04-17T10:04:29.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67ad3ec000 1 -- 172.21.15.32:0/3006314878 wait complete. 2024-04-17T10:04:29.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67ad3ec000 20 client.9927 trim_cache size 0 max 16384 2024-04-17T10:04:29.029 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:04:29.029 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "ff178df8-4c06-4367-a1e9-2e3a66d870d0", 2024-04-17T10:04:29.029 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-04-17T10:04:29.029 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 9831, 2024-04-17T10:04:29.029 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-04-17T10:04:29.029 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-04-17T10:04:29.029 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-04-17T10:04:29.030 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 4.5856132540000001, 2024-04-17T10:04:29.030 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 111, 2024-04-17T10:04:29.030 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 113, 2024-04-17T10:04:29.030 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 113, 2024-04-17T10:04:29.030 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 6.5289943600000004 2024-04-17T10:04:29.030 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:04:29.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67aa000640 1 -- 172.21.15.32:0/2426447163 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6780068380 msgr2=0x7f678006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:29.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67aa000640 1 --2- 172.21.15.32:0/2426447163 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6780068380 0x7f678006a840 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f67a415ee40 tx=0x7f678c009040 comp rx=0 tx=0).stop 2024-04-17T10:04:29.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67aa000640 1 -- 172.21.15.32:0/2426447163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6788004d50 msgr2=0x7f67a413ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:29.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67aa000640 1 --2- 172.21.15.32:0/2426447163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6788004d50 0x7f67a413ddb0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f6794012510 tx=0x7f67940129d0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67aa000640 1 -- 172.21.15.32:0/2426447163 shutdown_connections 2024-04-17T10:04:29.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67aa000640 1 --2- 172.21.15.32:0/2426447163 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f6780068380 0x7f678006a840 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67aa000640 1 --2- 172.21.15.32:0/2426447163 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67a4072990 0x7f67a414d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67aa000640 1 --2- 172.21.15.32:0/2426447163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67a4072510 0x7f67a414cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67aa000640 1 --2- 172.21.15.32:0/2426447163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6788004d50 0x7f67a413ddb0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67aa000640 1 -- 172.21.15.32:0/2426447163 >> 172.21.15.32:0/2426447163 conn(0x7f67a406ba30 msgr2=0x7f67a406e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:29.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67aa000640 1 -- 172.21.15.32:0/2426447163 shutdown_connections 2024-04-17T10:04:29.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.030+0000 7f67aa000640 1 -- 172.21.15.32:0/2426447163 wait complete. 2024-04-17T10:04:29.045 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T10:04:29.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07fc200640 1 Processor -- start 2024-04-17T10:04:29.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07fc200640 1 -- start start 2024-04-17T10:04:29.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07fc200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f07f4144e90 0x7f07f40b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:29.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07fc200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07f4072bb0 con 0x7f07f415c1b0 2024-04-17T10:04:29.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07fc200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07f40b3dc0 con 0x7f07f40b4de0 2024-04-17T10:04:29.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07fc200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f07f4145270 con 0x7f07f4144e90 2024-04-17T10:04:29.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07fa200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f07f415c1b0 0x7f07f406ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50418/0 (socket says 172.21.15.32:50418) 2024-04-17T10:04:29.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07fa200640 1 -- 172.21.15.32:0/3502668011 learned_addr learned my addr 172.21.15.32:0/3502668011 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:29.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07f3e00640 1 -- 172.21.15.32:0/3502668011 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1781989104 0 0) 0x7f07f4072bb0 con 0x7f07f415c1b0 2024-04-17T10:04:29.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07f9800640 1 --2- 172.21.15.32:0/3502668011 >> v2:172.21.15.144:3300/0 conn(0x7f07f4144e90 0x7f07f40b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:29.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07f3e00640 1 -- 172.21.15.32:0/3502668011 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07dc003530 con 0x7f07f415c1b0 2024-04-17T10:04:29.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07f3e00640 1 -- 172.21.15.32:0/3502668011 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2540539190 0 0) 0x7f07dc003530 con 0x7f07f415c1b0 2024-04-17T10:04:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07f3e00640 1 -- 172.21.15.32:0/3502668011 >> v2:172.21.15.144:3300/0 conn(0x7f07f4144e90 msgr2=0x7f07f40b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07f3e00640 1 --2- 172.21.15.32:0/3502668011 >> v2:172.21.15.144:3300/0 conn(0x7f07f4144e90 0x7f07f40b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07f3e00640 1 -- 172.21.15.32:0/3502668011 >> v1:172.21.15.144:6789/0 conn(0x7f07f40b4de0 legacy=0x7f07f415baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07f3e00640 1 -- 172.21.15.32:0/3502668011 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f40b40a0 con 0x7f07f415c1b0 2024-04-17T10:04:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07f3e00640 1 -- 172.21.15.32:0/3502668011 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f07ec001a20 con 0x7f07f415c1b0 2024-04-17T10:04:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07f3e00640 1 -- 172.21.15.32:0/3502668011 >> v1:172.21.15.32:6789/0 conn(0x7f07f415c1b0 legacy=0x7f07f406ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07f3e00640 1 --2- 172.21.15.32:0/3502668011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07dc0039c0 0x7f07dc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07f3e00640 1 --2- 172.21.15.32:0/3502668011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07dc004810 0x7f07dc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07f3e00640 1 --2- 172.21.15.32:0/3502668011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07dc0051d0 0x7f07dc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07f3e00640 1 -- 172.21.15.32:0/3502668011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f07f4145270 con 0x7f07dc004810 2024-04-17T10:04:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07f3e00640 1 -- 172.21.15.32:0/3502668011 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc00c070 con 0x7f07dc0051d0 2024-04-17T10:04:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.190+0000 7f07f3e00640 1 -- 172.21.15.32:0/3502668011 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f07dc00c270 con 0x7f07dc0039c0 2024-04-17T10:04:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.194+0000 7f07f8e00640 1 --2- 172.21.15.32:0/3502668011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07dc004810 0x7f07dc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.194+0000 7f07f9800640 1 --2- 172.21.15.32:0/3502668011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07dc0039c0 0x7f07dc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.194+0000 7f07fa200640 1 --2- 172.21.15.32:0/3502668011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07dc0051d0 0x7f07dc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.194+0000 7f07f8e00640 1 -- 172.21.15.32:0/3502668011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07dc0039c0 msgr2=0x7f07dc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.194+0000 7f07f8e00640 1 --2- 172.21.15.32:0/3502668011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07dc0039c0 0x7f07dc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.194+0000 7f07f8e00640 1 -- 172.21.15.32:0/3502668011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07dc0051d0 msgr2=0x7f07dc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.194+0000 7f07f8e00640 1 --2- 172.21.15.32:0/3502668011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07dc0051d0 0x7f07dc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.194+0000 7f07f8e00640 1 -- 172.21.15.32:0/3502668011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07dc003530 con 0x7f07dc004810 2024-04-17T10:04:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.194+0000 7f07fa200640 1 -- 172.21.15.32:0/3502668011 reap_dead start 2024-04-17T10:04:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.194+0000 7f07f8e00640 1 --2- 172.21.15.32:0/3502668011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07dc004810 0x7f07dc004c90 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f07e0001780 tx=0x7f07e001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a8eee07d15b035 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:29.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.194+0000 7f07f3e00640 1 -- 172.21.15.32:0/3502668011 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e00013a0 con 0x7f07dc004810 2024-04-17T10:04:29.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.194+0000 7f07f3e00640 1 -- 172.21.15.32:0/3502668011 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07e00052f0 con 0x7f07dc004810 2024-04-17T10:04:29.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.194+0000 7f07f3e00640 1 -- 172.21.15.32:0/3502668011 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0003840 con 0x7f07dc004810 2024-04-17T10:04:29.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.194+0000 7f07fc200640 1 -- 172.21.15.32:0/3502668011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07dc004810 msgr2=0x7f07dc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:29.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.194+0000 7f07fc200640 1 --2- 172.21.15.32:0/3502668011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07dc004810 0x7f07dc004c90 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f07e0001780 tx=0x7f07e001c400 comp rx=0 tx=0).stop 2024-04-17T10:04:29.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07fc200640 1 -- 172.21.15.32:0/3502668011 shutdown_connections 2024-04-17T10:04:29.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07fc200640 1 --2- 172.21.15.32:0/3502668011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07dc004810 0x7f07dc004c90 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07fc200640 1 -- 172.21.15.32:0/3502668011 >> 172.21.15.32:0/3502668011 conn(0x7f07f406ba30 msgr2=0x7f07f406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:29.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07fc200640 1 -- 172.21.15.32:0/3502668011 shutdown_connections 2024-04-17T10:04:29.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07fc200640 1 -- 172.21.15.32:0/3502668011 wait complete. 2024-04-17T10:04:29.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07fc200640 1 Processor -- start 2024-04-17T10:04:29.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07fc200640 1 -- start start 2024-04-17T10:04:29.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07fc200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07dc004810 0x7f07f40b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:29.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07fc200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07f41f1ad0 0x7f07f4137600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:29.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07fc200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07f41f1f50 0x7f07f4137b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:29.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07fc200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f07f40b4a20 con 0x7f07f41f1ad0 2024-04-17T10:04:29.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07fc200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f07f40b48a0 con 0x7f07dc004810 2024-04-17T10:04:29.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07fc200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f07f40b4ba0 con 0x7f07f41f1f50 2024-04-17T10:04:29.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07f8e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07f41f1ad0 0x7f07f4137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:29.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07f8e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07f41f1ad0 0x7f07f4137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32826/0 (socket says 172.21.15.32:32826) 2024-04-17T10:04:29.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07f8e00640 1 -- 172.21.15.32:0/1416934106 learned_addr learned my addr 172.21.15.32:0/1416934106 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:29.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07f9800640 1 --2- 172.21.15.32:0/1416934106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07dc004810 0x7f07f40b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:29.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07fa200640 1 --2- 172.21.15.32:0/1416934106 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07f41f1f50 0x7f07f4137b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:29.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07f8e00640 1 -- 172.21.15.32:0/1416934106 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07f41f1f50 msgr2=0x7f07f4137b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:29.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07f8e00640 1 --2- 172.21.15.32:0/1416934106 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07f41f1f50 0x7f07f4137b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07f8e00640 1 -- 172.21.15.32:0/1416934106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07dc004810 msgr2=0x7f07f40b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:29.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07f8e00640 1 --2- 172.21.15.32:0/1416934106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07dc004810 0x7f07f40b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07f8e00640 1 -- 172.21.15.32:0/1416934106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f412fe90 con 0x7f07f41f1ad0 2024-04-17T10:04:29.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07fa200640 1 --2- 172.21.15.32:0/1416934106 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07f41f1f50 0x7f07f4137b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:29.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07f8e00640 1 --2- 172.21.15.32:0/1416934106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07f41f1ad0 0x7f07f4137600 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f07e001c230 tx=0x7f07dc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:29.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07f1600640 1 -- 172.21.15.32:0/1416934106 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e00013a0 con 0x7f07f41f1ad0 2024-04-17T10:04:29.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07f1600640 1 -- 172.21.15.32:0/1416934106 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07e00ba070 con 0x7f07f41f1ad0 2024-04-17T10:04:29.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.198+0000 7f07fc200640 1 -- 172.21.15.32:0/1416934106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f4130170 con 0x7f07f41f1ad0 2024-04-17T10:04:29.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.202+0000 7f07f1600640 1 -- 172.21.15.32:0/1416934106 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0007a40 con 0x7f07f41f1ad0 2024-04-17T10:04:29.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.202+0000 7f07fc200640 1 -- 172.21.15.32:0/1416934106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f4130730 con 0x7f07f41f1ad0 2024-04-17T10:04:29.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.202+0000 7f07fc200640 1 -- 172.21.15.32:0/1416934106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07f4068a90 con 0x7f07f41f1ad0 2024-04-17T10:04:29.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.202+0000 7f07f1600640 1 -- 172.21.15.32:0/1416934106 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f07e0007490 con 0x7f07f41f1ad0 2024-04-17T10:04:29.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.202+0000 7f07f1600640 1 --2- 172.21.15.32:0/1416934106 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07d4068380 0x7f07d406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:29.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.202+0000 7f07f1600640 1 -- 172.21.15.32:0/1416934106 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f07e0120da0 con 0x7f07f41f1ad0 2024-04-17T10:04:29.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.202+0000 7f07f9800640 1 --2- 172.21.15.32:0/1416934106 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07d4068380 0x7f07d406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:29.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.202+0000 7f07f9800640 1 --2- 172.21.15.32:0/1416934106 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07d4068380 0x7f07d406a840 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f07f406a1a0 tx=0x7f07e8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.206+0000 7f07f1600640 1 -- 172.21.15.32:0/1416934106 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f07e0016110 con 0x7f07f41f1ad0 2024-04-17T10:04:29.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.410+0000 7f07fc200640 1 -- 172.21.15.32:0/1416934106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f07f40cf6a0 con 0x7f07f41f1ad0 2024-04-17T10:04:29.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.410+0000 7f07f1600640 1 -- 172.21.15.32:0/1416934106 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v111) ==== 53+0+73 (secure 0 0 0) 0x7f07e00f1270 con 0x7f07f41f1ad0 2024-04-17T10:04:29.404 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T10:04:29.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.410+0000 7f07fc200640 1 -- 172.21.15.32:0/1416934106 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07d4068380 msgr2=0x7f07d406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:29.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.410+0000 7f07fc200640 1 --2- 172.21.15.32:0/1416934106 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07d4068380 0x7f07d406a840 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f07f406a1a0 tx=0x7f07e8008040 comp rx=0 tx=0).stop 2024-04-17T10:04:29.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.410+0000 7f07fc200640 1 -- 172.21.15.32:0/1416934106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07f41f1ad0 msgr2=0x7f07f4137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:29.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.410+0000 7f07fc200640 1 --2- 172.21.15.32:0/1416934106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07f41f1ad0 0x7f07f4137600 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f07e001c230 tx=0x7f07dc005840 comp rx=0 tx=0).stop 2024-04-17T10:04:29.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.410+0000 7f07fc200640 1 -- 172.21.15.32:0/1416934106 shutdown_connections 2024-04-17T10:04:29.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.410+0000 7f07fc200640 1 --2- 172.21.15.32:0/1416934106 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f07d4068380 0x7f07d406a840 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.410+0000 7f07fc200640 1 --2- 172.21.15.32:0/1416934106 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07f41f1f50 0x7f07f4137b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.410+0000 7f07fc200640 1 --2- 172.21.15.32:0/1416934106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07f41f1ad0 0x7f07f4137600 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.410+0000 7f07fc200640 1 --2- 172.21.15.32:0/1416934106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07dc004810 0x7f07f40b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:29.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.410+0000 7f07fc200640 1 -- 172.21.15.32:0/1416934106 >> 172.21.15.32:0/1416934106 conn(0x7f07f406ba30 msgr2=0x7f07f40b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:29.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.410+0000 7f07fc200640 1 -- 172.21.15.32:0/1416934106 shutdown_connections 2024-04-17T10:04:29.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29.410+0000 7f07fc200640 1 -- 172.21.15.32:0/1416934106 wait complete. 2024-04-17T10:04:29.427 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T10:04:29.427 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T10:04:29.427 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-04-17T10:04:29.427 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T10:04:29.427 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T10:04:29.427 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T10:04:29.427 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T10:04:29.427 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T10:04:29.427 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:04:29.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:29.428 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-04-17T10:04:29.437 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T10:04:29.437 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T10:04:29.437 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T10:04:29.437 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:04:29.437 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-04-17T10:04:29.437 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:04:29.437 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:04:29.437 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-04-17T10:04:29.437 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-04-17T10:04:29.437 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T10:04:29.437 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:04:29.437 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-04-17T10:04:29.437 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-17T10:04:29.438 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 41784sec preferred_lft 41784sec 2024-04-17T10:04:29.438 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2024-04-17T10:04:29.438 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:04:29.438 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:04:29.438 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-04-17T10:04:29.438 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-04-17T10:04:29.438 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-17T10:04:29.438 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-04-17T10:04:29.438 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-04-17T10:04:29.438 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-17T10:04:29.438 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:cd:f4:aa:5b brd ff:ff:ff:ff:ff:ff 2024-04-17T10:04:29.438 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-17T10:04:29.438 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:04:29.438 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T10:04:29.439 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-17T10:04:29.439 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T10:04:29.439 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:04:29.439 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-17T10:04:29.439 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:04:29.439 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:04:29.439 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:04:29.439 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-17T10:04:29.439 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:04:29.439 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T10:04:29.439 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:04:29.439 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-17T10:04:29.439 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-04-17T10:04:29.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:29.440 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T10:04:29.501 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-17T10:04:29.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:29.503 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-04-17T10:04:29.556 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-04-17T10:04:29.556 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T10:04:29.556 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-17T10:04:29.557 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-17T10:04:29.557 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-17T10:04:29.557 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T10:04:29.557 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T10:04:29.557 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-04-17T10:04:29.557 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-17T10:04:29.557 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-17T10:04:29.557 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-17T10:04:29.557 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-04-17T10:04:29.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T10:04:29.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:04:29.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:29.695 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-17T10:04:29.703 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T10:04:29.703 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T10:04:29.704 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:04:29.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:29.704 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-04-17T10:04:29.771 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-17T10:04:29.771 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:04:29.771 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T10:04:29.827 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T10:04:29.827 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T10:04:29.827 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:04:29.879 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T10:04:29.939 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:04:29.993 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T10:04:29.993 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T10:04:29.993 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T10:04:29.993 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T10:04:29.993 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T10:04:29.993 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T10:04:29.994 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T10:04:29.994 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T10:04:29.994 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T10:04:29.994 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T10:04:29.994 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T10:04:29.994 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T10:04:29.994 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:04:29.994 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T10:04:29.994 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T10:04:29.994 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T10:04:29.994 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T10:04:29.994 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T10:04:29.994 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T10:04:29.995 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T10:04:29.995 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:04:29.995 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T10:04:29.995 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T10:04:29.995 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T10:04:29.995 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:04:29.995 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:04:29.995 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:04:29.995 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:04:29.995 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T10:04:29.995 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:04:29.995 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:04:29.995 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:04:29.995 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:04:29.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:29.996 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T10:04:30.047 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T10:04:30.047 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-17T10:04:30.094 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-04-17T10:04:30.114 DEBUG:teuthology.orchestra.run.smithi032:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T10:04:30.164 INFO:teuthology.orchestra.run.smithi032.stdout:/sys 2024-04-17T10:04:30.165 INFO:teuthology.orchestra.run.smithi032.stdout:/proc 2024-04-17T10:04:30.165 INFO:teuthology.orchestra.run.smithi032.stdout:/dev 2024-04-17T10:04:30.165 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/pts 2024-04-17T10:04:30.165 INFO:teuthology.orchestra.run.smithi032.stdout:/run 2024-04-17T10:04:30.165 INFO:teuthology.orchestra.run.smithi032.stdout:/ 2024-04-17T10:04:30.165 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/security 2024-04-17T10:04:30.165 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/shm 2024-04-17T10:04:30.165 INFO:teuthology.orchestra.run.smithi032.stdout:/run/lock 2024-04-17T10:04:30.165 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/cgroup 2024-04-17T10:04:30.165 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/pstore 2024-04-17T10:04:30.165 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/bpf 2024-04-17T10:04:30.165 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T10:04:30.165 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/hugepages 2024-04-17T10:04:30.166 INFO:teuthology.orchestra.run.smithi032.stdout:/dev/mqueue 2024-04-17T10:04:30.166 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/debug 2024-04-17T10:04:30.166 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/tracing 2024-04-17T10:04:30.166 INFO:teuthology.orchestra.run.smithi032.stdout:/proc/fs/nfsd 2024-04-17T10:04:30.166 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/fs/fuse/connections 2024-04-17T10:04:30.166 INFO:teuthology.orchestra.run.smithi032.stdout:/sys/kernel/config 2024-04-17T10:04:30.166 INFO:teuthology.orchestra.run.smithi032.stdout:/run/credentials/systemd-sysusers.service 2024-04-17T10:04:30.166 INFO:teuthology.orchestra.run.smithi032.stdout:/run/rpc_pipefs 2024-04-17T10:04:30.166 INFO:teuthology.orchestra.run.smithi032.stdout:/run/user/1000 2024-04-17T10:04:30.166 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph 2024-04-17T10:04:30.166 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T10:04:30.166 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T10:04:30.166 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T10:04:30.166 INFO:teuthology.orchestra.run.smithi032.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T10:04:30.167 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns 2024-04-17T10:04:30.167 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:04:30.167 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:04:30.167 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:04:30.167 INFO:teuthology.orchestra.run.smithi032.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:04:30.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:30.168 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-04-17T10:04:30.191 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2024-04-17T10:04:30.194+0000 7f86cc120580 -1 init, newargv = 0x5649991d12c0 newargc=15 2024-04-17T10:04:30.191 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[53235]: starting ceph client 2024-04-17T10:04:30.198 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[53235]: starting fuse 2024-04-17T10:04:30.200 INFO:teuthology.orchestra.run.smithi032.stdout:44 2024-04-17T10:04:30.200 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-17T10:04:30.200 DEBUG:teuthology.orchestra.run.smithi032:> sudo stdin-killer -- python3 -c ' 2024-04-17T10:04:30.200 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2024-04-17T10:04:30.201 DEBUG:teuthology.orchestra.run.smithi032:> import re 2024-04-17T10:04:30.201 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:04:30.201 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2024-04-17T10:04:30.201 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:04:30.201 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2024-04-17T10:04:30.201 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T10:04:30.201 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2024-04-17T10:04:30.201 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-17T10:04:30.201 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:04:30.201 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2024-04-17T10:04:30.201 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2024-04-17T10:04:30.201 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2024-04-17T10:04:30.201 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2024-04-17T10:04:30.201 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:04:30.202 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2024-04-17T10:04:30.202 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T10:04:30.202 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T10:04:30.202 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T10:04:30.202 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2024-04-17T10:04:30.202 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2024-04-17T10:04:30.202 DEBUG:teuthology.orchestra.run.smithi032:> return f 2024-04-17T10:04:30.202 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T10:04:30.202 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:04:30.202 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2024-04-17T10:04:30.202 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:04:30.326 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.53235.asok 2024-04-17T10:04:30.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:04:30.337 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.53235.asok 2024-04-17T10:04:30.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:30.337 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.53235.asok status 2024-04-17T10:04:30.515 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:04:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2024-04-17T10:04:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-17T10:04:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T10:04:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2024-04-17T10:04:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2024-04-17T10:04:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-17T10:04:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "53235", 2024-04-17T10:04:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2024-04-17T10:04:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:04:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2024-04-17T10:04:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2024-04-17T10:04:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 9939, 2024-04-17T10:04:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2024-04-17T10:04:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2024-04-17T10:04:30.516 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2024-04-17T10:04:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 9939 2024-04-17T10:04:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:04:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T10:04:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T10:04:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:04:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2250510637 2024-04-17T10:04:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-04-17T10:04:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:04:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-04-17T10:04:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-04-17T10:04:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-04-17T10:04:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 2250510637 2024-04-17T10:04:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-04-17T10:04:30.517 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.9939 192.168.0.1:0/2250510637", 2024-04-17T10:04:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/2250510637", 2024-04-17T10:04:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2024-04-17T10:04:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 111, 2024-04-17T10:04:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 113, 2024-04-17T10:04:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2024-04-17T10:04:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2024-04-17T10:04:30.518 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs" 2024-04-17T10:04:30.518 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:04:30.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:30.531 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2024-04-17T10:04:30.532 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-17T10:04:30.540 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2024-04-17T10:04:30.541 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-17T10:04:30.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:30.541 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T10:04:30.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:30.606 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-17T10:04:30.672 INFO:teuthology.orchestra.run.smithi032.stdout:./ 2024-04-17T10:04:30.672 INFO:teuthology.orchestra.run.smithi032.stdout:./parent 2024-04-17T10:04:30.673 INFO:teuthology.orchestra.run.smithi032.stdout:./parent/flushed 2024-04-17T10:04:30.673 INFO:teuthology.orchestra.run.smithi032.stdout:./parent/flushed/charlie 2024-04-17T10:04:30.673 INFO:teuthology.orchestra.run.smithi032.stdout:./parent/flushed/alpha 2024-04-17T10:04:30.673 INFO:teuthology.orchestra.run.smithi032.stdout:./parent/flushed/bravo 2024-04-17T10:04:30.673 INFO:teuthology.orchestra.run.smithi032.stdout:./parent/unflushed 2024-04-17T10:04:30.673 INFO:teuthology.orchestra.run.smithi032.stdout:./parent/unflushed/jfile 2024-04-17T10:04:30.674 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:04:30.674 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:04:30.674 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:04:30.674 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:04:30.674 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-17T10:04:30.674 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:04:30.784 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-04-17T10:04:30.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:04:30.794 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:04:30.794 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:04:30.794 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:04:30.794 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:04:30.794 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-17T10:04:30.794 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:04:30.901 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627776 2024-04-17T10:04:30.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:04:30.910 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:04:30.910 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:04:30.910 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:04:30.910 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:04:30.911 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-17T10:04:30.911 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:04:31.017 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627777 2024-04-17T10:04:31.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:04:31.026 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:04:31.026 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:04:31.026 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:04:31.026 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:04:31.026 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-17T10:04:31.026 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:04:31.133 INFO:teuthology.orchestra.run.smithi032.stdout:1099511628280 2024-04-17T10:04:31.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:04:31.143 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:04:31.143 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:04:31.143 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:04:31.143 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:04:31.143 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-17T10:04:31.143 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:04:31.249 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627778 2024-04-17T10:04:31.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:04:31.258 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:04:31.258 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:04:31.259 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:04:31.259 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:04:31.259 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-17T10:04:31.259 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:04:31.369 INFO:teuthology.orchestra.run.smithi032.stdout:1099511628279 2024-04-17T10:04:31.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:04:31.379 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:04:31.379 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:04:31.379 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:04:31.379 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:04:31.379 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-17T10:04:31.380 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:04:31.490 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627779 2024-04-17T10:04:31.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:04:31.499 DEBUG:teuthology.orchestra.run.smithi032:> stdin-killer -- python3 -c ' 2024-04-17T10:04:31.499 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-04-17T10:04:31.499 DEBUG:teuthology.orchestra.run.smithi032:> import stat 2024-04-17T10:04:31.499 DEBUG:teuthology.orchestra.run.smithi032:> 2024-04-17T10:04:31.499 DEBUG:teuthology.orchestra.run.smithi032:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-17T10:04:31.500 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-04-17T10:04:31.609 INFO:teuthology.orchestra.run.smithi032.stdout:1099511627780 2024-04-17T10:04:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T10:04:31.618 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T10:04:31.618 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:04:31.778 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T10:04:31.886 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T10:04:31.886 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T10:04:31.886 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:04:31.886 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:04:31.895 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:04:32.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0deb600640 1 Processor -- start 2024-04-17T10:04:32.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0deb600640 1 -- start start 2024-04-17T10:04:32.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0deb600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0de4144e90 0x7f0de40b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:32.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0deb600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0de415c1b0 0x7f0de406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:32.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0deb600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0de4072bb0 con 0x7f0de40b4de0 2024-04-17T10:04:32.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0deb600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0de4145270 con 0x7f0de415c1b0 2024-04-17T10:04:32.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0deb600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0de40b3dc0 con 0x7f0de4144e90 2024-04-17T10:04:32.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0de40b4de0 0x7f0de415baa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44320/0 (socket says 172.21.15.32:44320) 2024-04-17T10:04:32.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3e00640 1 -- 172.21.15.32:0/2812611767 learned_addr learned my addr 172.21.15.32:0/2812611767 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:32.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de9600640 1 --2- 172.21.15.32:0/2812611767 >> v2:172.21.15.144:3300/0 conn(0x7f0de415c1b0 0x7f0de406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:32.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de8c00640 1 --2- 172.21.15.32:0/2812611767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0de4144e90 0x7f0de40b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:32.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3400640 1 -- 172.21.15.32:0/2812611767 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1458312964 0 0) 0x7f0de4072bb0 con 0x7f0de40b4de0 2024-04-17T10:04:32.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3400640 1 -- 172.21.15.32:0/2812611767 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0dcc003530 con 0x7f0de40b4de0 2024-04-17T10:04:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3400640 1 -- 172.21.15.32:0/2812611767 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 940787959 0 0) 0x7f0dcc003530 con 0x7f0de40b4de0 2024-04-17T10:04:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3400640 1 -- 172.21.15.32:0/2812611767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0de4144e90 msgr2=0x7f0de40b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3400640 1 --2- 172.21.15.32:0/2812611767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0de4144e90 0x7f0de40b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3400640 1 -- 172.21.15.32:0/2812611767 >> v2:172.21.15.144:3300/0 conn(0x7f0de415c1b0 msgr2=0x7f0de406ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3400640 1 --2- 172.21.15.32:0/2812611767 >> v2:172.21.15.144:3300/0 conn(0x7f0de415c1b0 0x7f0de406ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3400640 1 -- 172.21.15.32:0/2812611767 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de40b40a0 con 0x7f0de40b4de0 2024-04-17T10:04:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3400640 1 -- 172.21.15.32:0/2812611767 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7f0dd00016a0 con 0x7f0de40b4de0 2024-04-17T10:04:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3400640 1 -- 172.21.15.32:0/2812611767 >> v1:172.21.15.32:6789/0 conn(0x7f0de40b4de0 legacy=0x7f0de415baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3400640 1 --2- 172.21.15.32:0/2812611767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0de4144e90 0x7f0dcc0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3400640 1 --2- 172.21.15.32:0/2812611767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0dcc003f40 0x7f0dcc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3400640 1 --2- 172.21.15.32:0/2812611767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dcc004d50 0x7f0dcc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3400640 1 -- 172.21.15.32:0/2812611767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0de4145270 con 0x7f0dcc003f40 2024-04-17T10:04:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3400640 1 -- 172.21.15.32:0/2812611767 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0de40b3dc0 con 0x7f0dcc004d50 2024-04-17T10:04:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3400640 1 -- 172.21.15.32:0/2812611767 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0dcc00b9d0 con 0x7f0de4144e90 2024-04-17T10:04:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3e00640 1 --2- 172.21.15.32:0/2812611767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0dcc003f40 0x7f0dcc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de8c00640 1 --2- 172.21.15.32:0/2812611767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0de4144e90 0x7f0dcc0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de9600640 1 --2- 172.21.15.32:0/2812611767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dcc004d50 0x7f0dcc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3e00640 1 -- 172.21.15.32:0/2812611767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0de4144e90 msgr2=0x7f0dcc0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3e00640 1 --2- 172.21.15.32:0/2812611767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0de4144e90 0x7f0dcc0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3e00640 1 -- 172.21.15.32:0/2812611767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dcc004d50 msgr2=0x7f0dcc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3e00640 1 --2- 172.21.15.32:0/2812611767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dcc004d50 0x7f0dcc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.082+0000 7f0de3e00640 1 -- 172.21.15.32:0/2812611767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dcc003530 con 0x7f0dcc003f40 2024-04-17T10:04:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.086+0000 7f0de3e00640 1 --2- 172.21.15.32:0/2812611767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0dcc003f40 0x7f0dcc004810 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f0dd000c210 tx=0x7f0dd001f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b81f7521524bac server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.086+0000 7f0de3400640 1 -- 172.21.15.32:0/2812611767 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd0007d80 con 0x7f0dcc003f40 2024-04-17T10:04:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.086+0000 7f0de3400640 1 -- 172.21.15.32:0/2812611767 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dd0005ae0 con 0x7f0dcc003f40 2024-04-17T10:04:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.086+0000 7f0de3400640 1 -- 172.21.15.32:0/2812611767 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd0007070 con 0x7f0dcc003f40 2024-04-17T10:04:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.086+0000 7f0deb600640 1 -- 172.21.15.32:0/2812611767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0dcc003f40 msgr2=0x7f0dcc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.086+0000 7f0deb600640 1 --2- 172.21.15.32:0/2812611767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0dcc003f40 0x7f0dcc004810 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f0dd000c210 tx=0x7f0dd001f390 comp rx=0 tx=0).stop 2024-04-17T10:04:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.086+0000 7f0de9600640 1 -- 172.21.15.32:0/2812611767 reap_dead start 2024-04-17T10:04:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.086+0000 7f0deb600640 1 -- 172.21.15.32:0/2812611767 shutdown_connections 2024-04-17T10:04:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.086+0000 7f0deb600640 1 -- 172.21.15.32:0/2812611767 >> 172.21.15.32:0/2812611767 conn(0x7f0de406ba30 msgr2=0x7f0de406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.086+0000 7f0deb600640 1 -- 172.21.15.32:0/2812611767 shutdown_connections 2024-04-17T10:04:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.086+0000 7f0deb600640 1 -- 172.21.15.32:0/2812611767 wait complete. 2024-04-17T10:04:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0deb600640 1 Processor -- start 2024-04-17T10:04:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0deb600640 1 -- start start 2024-04-17T10:04:32.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0deb600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dcc003f40 0x7f0de413c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:32.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0deb600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0de413f3d0 0x7f0de413bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:32.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0deb600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0de4071df0 0x7f0de4072270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:32.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0deb600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0de40b4a20 con 0x7f0de4071df0 2024-04-17T10:04:32.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0deb600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0de40b48a0 con 0x7f0dcc003f40 2024-04-17T10:04:32.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0deb600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0de40b4ba0 con 0x7f0de413f3d0 2024-04-17T10:04:32.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0de9600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0de4071df0 0x7f0de4072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0de9600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0de4071df0 0x7f0de4072270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59642/0 (socket says 172.21.15.32:59642) 2024-04-17T10:04:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0de9600640 1 -- 172.21.15.32:0/3989001351 learned_addr learned my addr 172.21.15.32:0/3989001351 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0de3e00640 1 --2- 172.21.15.32:0/3989001351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0de413f3d0 0x7f0de413bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0de8c00640 1 --2- 172.21.15.32:0/3989001351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dcc003f40 0x7f0de413c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0de9600640 1 -- 172.21.15.32:0/3989001351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0de413f3d0 msgr2=0x7f0de413bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0de9600640 1 --2- 172.21.15.32:0/3989001351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0de413f3d0 0x7f0de413bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0de9600640 1 -- 172.21.15.32:0/3989001351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dcc003f40 msgr2=0x7f0de413c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0de9600640 1 --2- 172.21.15.32:0/3989001351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dcc003f40 0x7f0de413c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0de9600640 1 -- 172.21.15.32:0/3989001351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de413d0e0 con 0x7f0de4071df0 2024-04-17T10:04:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0de8c00640 1 --2- 172.21.15.32:0/3989001351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dcc003f40 0x7f0de413c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0de9600640 1 --2- 172.21.15.32:0/3989001351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0de4071df0 0x7f0de4072270 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f0de40cce80 tx=0x7f0ddc00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0de0c00640 1 -- 172.21.15.32:0/3989001351 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc00fdb0 con 0x7f0de4071df0 2024-04-17T10:04:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0de0c00640 1 -- 172.21.15.32:0/3989001351 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ddc01d070 con 0x7f0de4071df0 2024-04-17T10:04:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0de0c00640 1 -- 172.21.15.32:0/3989001351 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc015670 con 0x7f0de4071df0 2024-04-17T10:04:32.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0deb600640 1 -- 172.21.15.32:0/3989001351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de415d7a0 con 0x7f0de4071df0 2024-04-17T10:04:32.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.090+0000 7f0deb600640 1 -- 172.21.15.32:0/3989001351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de415da40 con 0x7f0de4071df0 2024-04-17T10:04:32.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.094+0000 7f0deb600640 1 -- 172.21.15.32:0/3989001351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0de40ada30 con 0x7f0de4071df0 2024-04-17T10:04:32.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.094+0000 7f0de0c00640 1 -- 172.21.15.32:0/3989001351 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0ddc015810 con 0x7f0de4071df0 2024-04-17T10:04:32.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.094+0000 7f0de0c00640 1 --2- 172.21.15.32:0/3989001351 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0dc40683d0 0x7f0dc406a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:32.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.094+0000 7f0de0c00640 1 -- 172.21.15.32:0/3989001351 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f0ddc087310 con 0x7f0de4071df0 2024-04-17T10:04:32.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.094+0000 7f0de8c00640 1 --2- 172.21.15.32:0/3989001351 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0dc40683d0 0x7f0dc406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:32.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.094+0000 7f0de8c00640 1 --2- 172.21.15.32:0/3989001351 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0dc40683d0 0x7f0dc406a890 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8009a80 tx=0x7f0dd8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:32.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.098+0000 7f0de0c00640 1 -- 172.21.15.32:0/3989001351 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0ddc010040 con 0x7f0de4071df0 2024-04-17T10:04:32.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.266+0000 7f0deb600640 1 -- 172.21.15.32:0/3989001351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0de4068a90 con 0x7f0de4071df0 2024-04-17T10:04:32.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.266+0000 7f0de0c00640 1 -- 172.21.15.32:0/3989001351 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16991 (secure 0 0 0) 0x7f0ddc0577c0 con 0x7f0de4071df0 2024-04-17T10:04:32.262 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:32.263 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":113,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:04:22.430002+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:03:36.021322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-17T10:03:36.581134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.5","mappings":[{"from":0,"to":5}]},{"pgid":"17.25","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6837/578643390":"2024-04-18T10:04:22.429980+0000","172.21.15.32:6836/578643390":"2024-04-18T10:04:22.429980+0000","172.21.15.32:6837/901447746":"2024-04-18T10:04:03.080821+0000","172.21.15.32:6836/901447746":"2024-04-18T10:04:03.080821+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:04:32.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.270+0000 7f0deb600640 1 -- 172.21.15.32:0/3989001351 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0dc40683d0 msgr2=0x7f0dc406a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:32.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.270+0000 7f0deb600640 1 --2- 172.21.15.32:0/3989001351 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0dc40683d0 0x7f0dc406a890 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8009a80 tx=0x7f0dd8008040 comp rx=0 tx=0).stop 2024-04-17T10:04:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.270+0000 7f0deb600640 1 -- 172.21.15.32:0/3989001351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0de4071df0 msgr2=0x7f0de4072270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.270+0000 7f0deb600640 1 --2- 172.21.15.32:0/3989001351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0de4071df0 0x7f0de4072270 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f0de40cce80 tx=0x7f0ddc00f580 comp rx=0 tx=0).stop 2024-04-17T10:04:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.270+0000 7f0deb600640 1 -- 172.21.15.32:0/3989001351 shutdown_connections 2024-04-17T10:04:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.270+0000 7f0deb600640 1 --2- 172.21.15.32:0/3989001351 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0dc40683d0 0x7f0dc406a890 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.270+0000 7f0deb600640 1 --2- 172.21.15.32:0/3989001351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0de413f3d0 0x7f0de413bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.270+0000 7f0deb600640 1 --2- 172.21.15.32:0/3989001351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0de4071df0 0x7f0de4072270 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.270+0000 7f0deb600640 1 --2- 172.21.15.32:0/3989001351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dcc003f40 0x7f0de413c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.270+0000 7f0deb600640 1 -- 172.21.15.32:0/3989001351 >> 172.21.15.32:0/3989001351 conn(0x7f0de406ba30 msgr2=0x7f0de40b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.270+0000 7f0deb600640 1 -- 172.21.15.32:0/3989001351 shutdown_connections 2024-04-17T10:04:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.270+0000 7f0deb600640 1 -- 172.21.15.32:0/3989001351 wait complete. 2024-04-17T10:04:32.284 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:03:36.021322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:03:36.581134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:04:32.284 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:04:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.434+0000 7f9640400640 1 Processor -- start 2024-04-17T10:04:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.434+0000 7f9640400640 1 -- start start 2024-04-17T10:04:32.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.434+0000 7f9640400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f963806f7c0 0x7f963806ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:32.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.434+0000 7f9640400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f963806f270 0x7f963806bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:32.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.434+0000 7f9640400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96380cc940 con 0x7f963815b7d0 2024-04-17T10:04:32.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.434+0000 7f9640400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f96380ccc20 con 0x7f963806f270 2024-04-17T10:04:32.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.434+0000 7f9640400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f963806e0b0 con 0x7f963806f7c0 2024-04-17T10:04:32.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.434+0000 7f963e400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f963806f270 0x7f963806bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:32.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.434+0000 7f963da00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f963815b7d0 0x7f963806e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44322/0 (socket says 172.21.15.32:44322) 2024-04-17T10:04:32.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.434+0000 7f963da00640 1 -- 172.21.15.32:0/3800268505 learned_addr learned my addr 172.21.15.32:0/3800268505 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.434+0000 7f963d000640 1 --2- 172.21.15.32:0/3800268505 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f963806f7c0 0x7f963806ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.434+0000 7f963e400640 1 -- 172.21.15.32:0/3800268505 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f963806f7c0 msgr2=0x7f963806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.434+0000 7f963e400640 1 --2- 172.21.15.32:0/3800268505 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f963806f7c0 0x7f963806ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.434+0000 7f963e400640 1 -- 172.21.15.32:0/3800268505 >> v1:172.21.15.32:6789/0 conn(0x7f963815b7d0 legacy=0x7f963806e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.434+0000 7f963e400640 1 -- 172.21.15.32:0/3800268505 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963806e390 con 0x7f963806f270 2024-04-17T10:04:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.434+0000 7f9637e00640 1 -- 172.21.15.32:0/3800268505 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3166996136 0 0) 0x7f96380cc940 con 0x7f963815b7d0 2024-04-17T10:04:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.434+0000 7f963d000640 1 --2- 172.21.15.32:0/3800268505 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f963806f7c0 0x7f963806ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.438+0000 7f963e400640 1 --2- 172.21.15.32:0/3800268505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963806f270 0x7f963806bf70 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f9630009d90 tx=0x7f963001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4300523a082cbe17 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.438+0000 7f9637e00640 1 -- 172.21.15.32:0/3800268505 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96300b2070 con 0x7f963806f270 2024-04-17T10:04:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.438+0000 7f9637e00640 1 -- 172.21.15.32:0/3800268505 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96300a58b0 con 0x7f963806f270 2024-04-17T10:04:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.438+0000 7f9637e00640 1 -- 172.21.15.32:0/3800268505 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96300a5c80 con 0x7f963806f270 2024-04-17T10:04:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.438+0000 7f9640400640 1 -- 172.21.15.32:0/3800268505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963806f270 msgr2=0x7f963806bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.438+0000 7f9640400640 1 --2- 172.21.15.32:0/3800268505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963806f270 0x7f963806bf70 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f9630009d90 tx=0x7f963001c8e0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.438+0000 7f9640400640 1 -- 172.21.15.32:0/3800268505 shutdown_connections 2024-04-17T10:04:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.438+0000 7f9640400640 1 --2- 172.21.15.32:0/3800268505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963806f270 0x7f963806bf70 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.438+0000 7f9640400640 1 --2- 172.21.15.32:0/3800268505 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f963806f7c0 0x7f963806ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.438+0000 7f9640400640 1 -- 172.21.15.32:0/3800268505 >> 172.21.15.32:0/3800268505 conn(0x7f96380b2120 msgr2=0x7f96380b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.438+0000 7f9640400640 1 -- 172.21.15.32:0/3800268505 shutdown_connections 2024-04-17T10:04:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.438+0000 7f9640400640 1 -- 172.21.15.32:0/3800268505 wait complete. 2024-04-17T10:04:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.438+0000 7f9640400640 1 Processor -- start 2024-04-17T10:04:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f9640400640 1 -- start start 2024-04-17T10:04:32.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f9640400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f963806f270 0x7f963813de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:32.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f9640400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f963806f7c0 0x7f963815d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:32.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f9640400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963815b7d0 0x7f963815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:32.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f9640400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f96380cee30 con 0x7f963815b7d0 2024-04-17T10:04:32.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f9640400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96380cecb0 con 0x7f963806f7c0 2024-04-17T10:04:32.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f9640400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9638166290 con 0x7f963806f270 2024-04-17T10:04:32.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f963e400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963815b7d0 0x7f963815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:32.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f963e400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963815b7d0 0x7f963815a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59650/0 (socket says 172.21.15.32:59650) 2024-04-17T10:04:32.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f963e400640 1 -- 172.21.15.32:0/1844527375 learned_addr learned my addr 172.21.15.32:0/1844527375 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:32.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f963d000640 1 --2- 172.21.15.32:0/1844527375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f963806f7c0 0x7f963815d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:32.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f963da00640 1 --2- 172.21.15.32:0/1844527375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f963806f270 0x7f963813de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:32.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f963e400640 1 -- 172.21.15.32:0/1844527375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f963806f270 msgr2=0x7f963813de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:32.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f963e400640 1 --2- 172.21.15.32:0/1844527375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f963806f270 0x7f963813de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f963e400640 1 -- 172.21.15.32:0/1844527375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f963806f7c0 msgr2=0x7f963815d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:32.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f963e400640 1 --2- 172.21.15.32:0/1844527375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f963806f7c0 0x7f963815d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f963e400640 1 -- 172.21.15.32:0/1844527375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963813e360 con 0x7f963815b7d0 2024-04-17T10:04:32.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f963d000640 1 --2- 172.21.15.32:0/1844527375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f963806f7c0 0x7f963815d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:32.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f963e400640 1 --2- 172.21.15.32:0/1844527375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963815b7d0 0x7f963815a5e0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f96300b0040 tx=0x7f9630019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:32.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f9635600640 1 -- 172.21.15.32:0/1844527375 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96300b2050 con 0x7f963815b7d0 2024-04-17T10:04:32.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f9635600640 1 -- 172.21.15.32:0/1844527375 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96300b3040 con 0x7f963815b7d0 2024-04-17T10:04:32.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f9635600640 1 -- 172.21.15.32:0/1844527375 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96300adca0 con 0x7f963815b7d0 2024-04-17T10:04:32.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f9640400640 1 -- 172.21.15.32:0/1844527375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963815dc70 con 0x7f963815b7d0 2024-04-17T10:04:32.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f9640400640 1 -- 172.21.15.32:0/1844527375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f963815e220 con 0x7f963815b7d0 2024-04-17T10:04:32.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.442+0000 7f9640400640 1 -- 172.21.15.32:0/1844527375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96380b4b10 con 0x7f963815b7d0 2024-04-17T10:04:32.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.446+0000 7f9635600640 1 -- 172.21.15.32:0/1844527375 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f96300039a0 con 0x7f963815b7d0 2024-04-17T10:04:32.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.446+0000 7f9635600640 1 --2- 172.21.15.32:0/1844527375 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9618068380 0x7f961806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:32.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.446+0000 7f9635600640 1 -- 172.21.15.32:0/1844527375 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f9630121600 con 0x7f963815b7d0 2024-04-17T10:04:32.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.446+0000 7f963da00640 1 --2- 172.21.15.32:0/1844527375 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9618068380 0x7f961806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:32.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.446+0000 7f963da00640 1 --2- 172.21.15.32:0/1844527375 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9618068380 0x7f961806a840 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f962c000f70 tx=0x7f962c007c10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.450+0000 7f9635600640 1 -- 172.21.15.32:0/1844527375 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f9630016560 con 0x7f963815b7d0 2024-04-17T10:04:32.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.614+0000 7f9640400640 1 -- 172.21.15.32:0/1844527375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9638068a90 con 0x7f963815b7d0 2024-04-17T10:04:32.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.614+0000 7f9635600640 1 -- 172.21.15.32:0/1844527375 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) ==== 93+0+10 (secure 0 0 0) 0x7f96300f19a0 con 0x7f963815b7d0 2024-04-17T10:04:32.610 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T10:04:32.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.618+0000 7f9640400640 1 -- 172.21.15.32:0/1844527375 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9618068380 msgr2=0x7f961806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:32.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.618+0000 7f9640400640 1 --2- 172.21.15.32:0/1844527375 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9618068380 0x7f961806a840 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f962c000f70 tx=0x7f962c007c10 comp rx=0 tx=0).stop 2024-04-17T10:04:32.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.618+0000 7f9640400640 1 -- 172.21.15.32:0/1844527375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963815b7d0 msgr2=0x7f963815a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:32.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.618+0000 7f9640400640 1 --2- 172.21.15.32:0/1844527375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963815b7d0 0x7f963815a5e0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f96300b0040 tx=0x7f9630019680 comp rx=0 tx=0).stop 2024-04-17T10:04:32.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.618+0000 7f9640400640 1 -- 172.21.15.32:0/1844527375 shutdown_connections 2024-04-17T10:04:32.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.618+0000 7f9640400640 1 --2- 172.21.15.32:0/1844527375 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9618068380 0x7f961806a840 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.618+0000 7f9640400640 1 --2- 172.21.15.32:0/1844527375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963815b7d0 0x7f963815a5e0 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.618+0000 7f9640400640 1 --2- 172.21.15.32:0/1844527375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f963806f7c0 0x7f963815d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.618+0000 7f9640400640 1 --2- 172.21.15.32:0/1844527375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f963806f270 0x7f963813de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.618+0000 7f9640400640 1 -- 172.21.15.32:0/1844527375 >> 172.21.15.32:0/1844527375 conn(0x7f96380b2120 msgr2=0x7f9638071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:32.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.618+0000 7f9640400640 1 -- 172.21.15.32:0/1844527375 shutdown_connections 2024-04-17T10:04:32.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.618+0000 7f9640400640 1 -- 172.21.15.32:0/1844527375 wait complete. 2024-04-17T10:04:32.631 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:04:32.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.778+0000 7fe0dd600640 1 Processor -- start 2024-04-17T10:04:32.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.778+0000 7fe0dd600640 1 -- start start 2024-04-17T10:04:32.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.778+0000 7fe0dd600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe0d806f7c0 0x7fe0d806ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:32.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.778+0000 7fe0dd600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0d80cc940 con 0x7fe0d815b7d0 2024-04-17T10:04:32.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.778+0000 7fe0dd600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0d80ccc20 con 0x7fe0d806f270 2024-04-17T10:04:32.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.778+0000 7fe0dd600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe0d806e0b0 con 0x7fe0d806f7c0 2024-04-17T10:04:32.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.778+0000 7fe0d6a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe0d815b7d0 0x7fe0d806e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44328/0 (socket says 172.21.15.32:44328) 2024-04-17T10:04:32.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.778+0000 7fe0d6a00640 1 -- 172.21.15.32:0/4124005178 learned_addr learned my addr 172.21.15.32:0/4124005178 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:32.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0ce000640 1 --2- 172.21.15.32:0/4124005178 >> v2:172.21.15.144:3300/0 conn(0x7fe0d806f7c0 0x7fe0d806ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:32.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0d6000640 1 -- 172.21.15.32:0/4124005178 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2705871984 0 0) 0x7fe0d80cc940 con 0x7fe0d815b7d0 2024-04-17T10:04:32.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0d6000640 1 -- 172.21.15.32:0/4124005178 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0bc003530 con 0x7fe0d815b7d0 2024-04-17T10:04:32.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0d6000640 1 -- 172.21.15.32:0/4124005178 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1166539491 0 0) 0x7fe0bc003530 con 0x7fe0d815b7d0 2024-04-17T10:04:32.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0d6000640 1 -- 172.21.15.32:0/4124005178 >> v2:172.21.15.144:3300/0 conn(0x7fe0d806f7c0 msgr2=0x7fe0d806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:32.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0d6000640 1 --2- 172.21.15.32:0/4124005178 >> v2:172.21.15.144:3300/0 conn(0x7fe0d806f7c0 0x7fe0d806ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0d6000640 1 -- 172.21.15.32:0/4124005178 >> v1:172.21.15.144:6789/0 conn(0x7fe0d806f270 legacy=0x7fe0d80ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:32.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0d6000640 1 -- 172.21.15.32:0/4124005178 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d806e390 con 0x7fe0d815b7d0 2024-04-17T10:04:32.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0d6000640 1 -- 172.21.15.32:0/4124005178 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7fe0c8001a00 con 0x7fe0d815b7d0 2024-04-17T10:04:32.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0d6000640 1 -- 172.21.15.32:0/4124005178 >> v1:172.21.15.32:6789/0 conn(0x7fe0d815b7d0 legacy=0x7fe0d806e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:32.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0d6000640 1 --2- 172.21.15.32:0/4124005178 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0bc0039c0 0x7fe0bc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:32.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0d6000640 1 --2- 172.21.15.32:0/4124005178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0bc004810 0x7fe0bc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:32.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0d6000640 1 --2- 172.21.15.32:0/4124005178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0bc0051d0 0x7fe0bc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:32.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0d6000640 1 -- 172.21.15.32:0/4124005178 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d806e0b0 con 0x7fe0bc004810 2024-04-17T10:04:32.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0d6000640 1 -- 172.21.15.32:0/4124005178 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc00c170 con 0x7fe0bc0051d0 2024-04-17T10:04:32.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0d6000640 1 -- 172.21.15.32:0/4124005178 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe0bc00c370 con 0x7fe0bc0039c0 2024-04-17T10:04:32.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0ce000640 1 --2- 172.21.15.32:0/4124005178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0bc004810 0x7fe0bc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:32.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0d7400640 1 --2- 172.21.15.32:0/4124005178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0bc0051d0 0x7fe0bc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:32.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0d6a00640 1 --2- 172.21.15.32:0/4124005178 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0bc0039c0 0x7fe0bc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:32.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0ce000640 1 -- 172.21.15.32:0/4124005178 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0bc0039c0 msgr2=0x7fe0bc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:32.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0ce000640 1 --2- 172.21.15.32:0/4124005178 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0bc0039c0 0x7fe0bc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0ce000640 1 -- 172.21.15.32:0/4124005178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0bc0051d0 msgr2=0x7fe0bc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:32.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0ce000640 1 --2- 172.21.15.32:0/4124005178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0bc0051d0 0x7fe0bc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0ce000640 1 -- 172.21.15.32:0/4124005178 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0bc003530 con 0x7fe0bc004810 2024-04-17T10:04:32.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.782+0000 7fe0d7400640 1 -- 172.21.15.32:0/4124005178 reap_dead start 2024-04-17T10:04:32.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.786+0000 7fe0ce000640 1 --2- 172.21.15.32:0/4124005178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0bc004810 0x7fe0bc004c90 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009ae0 tx=0x7fe0c001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967bba124ffda6c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:32.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.786+0000 7fe0d6000640 1 -- 172.21.15.32:0/4124005178 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c00b2070 con 0x7fe0bc004810 2024-04-17T10:04:32.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.786+0000 7fe0d6000640 1 -- 172.21.15.32:0/4124005178 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0c00a56f0 con 0x7fe0bc004810 2024-04-17T10:04:32.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.786+0000 7fe0d6000640 1 -- 172.21.15.32:0/4124005178 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c00096b0 con 0x7fe0bc004810 2024-04-17T10:04:32.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.786+0000 7fe0dd600640 1 -- 172.21.15.32:0/4124005178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0bc004810 msgr2=0x7fe0bc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:32.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.786+0000 7fe0dd600640 1 --2- 172.21.15.32:0/4124005178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0bc004810 0x7fe0bc004c90 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009ae0 tx=0x7fe0c001ca70 comp rx=0 tx=0).stop 2024-04-17T10:04:32.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.786+0000 7fe0dd600640 1 -- 172.21.15.32:0/4124005178 shutdown_connections 2024-04-17T10:04:32.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.786+0000 7fe0dd600640 1 --2- 172.21.15.32:0/4124005178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0bc004810 0x7fe0bc004c90 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.786+0000 7fe0dd600640 1 -- 172.21.15.32:0/4124005178 >> 172.21.15.32:0/4124005178 conn(0x7fe0d80b2120 msgr2=0x7fe0d80b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:32.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.786+0000 7fe0dd600640 1 -- 172.21.15.32:0/4124005178 shutdown_connections 2024-04-17T10:04:32.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.786+0000 7fe0dd600640 1 -- 172.21.15.32:0/4124005178 wait complete. 2024-04-17T10:04:32.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.786+0000 7fe0dd600640 1 Processor -- start 2024-04-17T10:04:32.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.786+0000 7fe0dd600640 1 -- start start 2024-04-17T10:04:32.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.786+0000 7fe0dd600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0bc004810 0x7fe0d815df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:32.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.786+0000 7fe0dd600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0d8144f00 0x7fe0d80a9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:32.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.786+0000 7fe0dd600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0d80cce40 0x7fe0d80cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.786+0000 7fe0dd600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d80ce9d0 con 0x7fe0d80cce40 2024-04-17T10:04:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.786+0000 7fe0dd600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d80ceb50 con 0x7fe0d8144f00 2024-04-17T10:04:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.786+0000 7fe0dd600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe0d80cecd0 con 0x7fe0bc004810 2024-04-17T10:04:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.786+0000 7fe0d7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0d80cce40 0x7fe0d80cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.786+0000 7fe0d7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0d80cce40 0x7fe0d80cd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59678/0 (socket says 172.21.15.32:59678) 2024-04-17T10:04:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.786+0000 7fe0d7400640 1 -- 172.21.15.32:0/2646269784 learned_addr learned my addr 172.21.15.32:0/2646269784 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.786+0000 7fe0d6a00640 1 --2- 172.21.15.32:0/2646269784 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0bc004810 0x7fe0d815df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.790+0000 7fe0ce000640 1 --2- 172.21.15.32:0/2646269784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0d8144f00 0x7fe0d80a9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.790+0000 7fe0d7400640 1 -- 172.21.15.32:0/2646269784 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0bc004810 msgr2=0x7fe0d815df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.790+0000 7fe0d7400640 1 --2- 172.21.15.32:0/2646269784 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0bc004810 0x7fe0d815df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.790+0000 7fe0d7400640 1 -- 172.21.15.32:0/2646269784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0d8144f00 msgr2=0x7fe0d80a9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.790+0000 7fe0d7400640 1 --2- 172.21.15.32:0/2646269784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0d8144f00 0x7fe0d80a9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.790+0000 7fe0d7400640 1 -- 172.21.15.32:0/2646269784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d80aa6b0 con 0x7fe0d80cce40 2024-04-17T10:04:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.790+0000 7fe0d7400640 1 --2- 172.21.15.32:0/2646269784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0d80cce40 0x7fe0d80cd2c0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fe0d000e1e0 tx=0x7fe0d000e6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.790+0000 7fe0cf400640 1 -- 172.21.15.32:0/2646269784 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0019070 con 0x7fe0d80cce40 2024-04-17T10:04:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.790+0000 7fe0cf400640 1 -- 172.21.15.32:0/2646269784 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0d0005320 con 0x7fe0d80cce40 2024-04-17T10:04:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.790+0000 7fe0cf400640 1 -- 172.21.15.32:0/2646269784 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0007270 con 0x7fe0d80cce40 2024-04-17T10:04:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.790+0000 7fe0dd600640 1 -- 172.21.15.32:0/2646269784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d80aa990 con 0x7fe0d80cce40 2024-04-17T10:04:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.790+0000 7fe0dd600640 1 -- 172.21.15.32:0/2646269784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d80aaf50 con 0x7fe0d80cce40 2024-04-17T10:04:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.790+0000 7fe0dd600640 1 -- 172.21.15.32:0/2646269784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0d8068a90 con 0x7fe0d80cce40 2024-04-17T10:04:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.790+0000 7fe0cf400640 1 -- 172.21.15.32:0/2646269784 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe0d0007410 con 0x7fe0d80cce40 2024-04-17T10:04:32.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.790+0000 7fe0cf400640 1 --2- 172.21.15.32:0/2646269784 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0b4068380 0x7fe0b406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:32.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.794+0000 7fe0cf400640 1 -- 172.21.15.32:0/2646269784 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7fe0d0085c50 con 0x7fe0d80cce40 2024-04-17T10:04:32.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.794+0000 7fe0d6a00640 1 --2- 172.21.15.32:0/2646269784 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0b4068380 0x7fe0b406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:32.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.794+0000 7fe0d6a00640 1 --2- 172.21.15.32:0/2646269784 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0b4068380 0x7fe0b406a840 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80013e0 tx=0x7fe0c8001ac0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.798+0000 7fe0cf400640 1 -- 172.21.15.32:0/2646269784 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fe0d008b020 con 0x7fe0d80cce40 2024-04-17T10:04:32.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.958+0000 7fe0dd600640 1 -- 172.21.15.32:0/2646269784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe0d80b0c40 con 0x7fe0d80cce40 2024-04-17T10:04:32.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.958+0000 7fe0cf400640 1 -- 172.21.15.32:0/2646269784 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) ==== 104+0+11 (secure 0 0 0) 0x7fe0d0055b60 con 0x7fe0d80cce40 2024-04-17T10:04:32.955 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T10:04:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.962+0000 7fe0dd600640 1 -- 172.21.15.32:0/2646269784 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0b4068380 msgr2=0x7fe0b406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.962+0000 7fe0dd600640 1 --2- 172.21.15.32:0/2646269784 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0b4068380 0x7fe0b406a840 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80013e0 tx=0x7fe0c8001ac0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.962+0000 7fe0dd600640 1 -- 172.21.15.32:0/2646269784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0d80cce40 msgr2=0x7fe0d80cd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.962+0000 7fe0dd600640 1 --2- 172.21.15.32:0/2646269784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0d80cce40 0x7fe0d80cd2c0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fe0d000e1e0 tx=0x7fe0d000e6a0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.962+0000 7fe0dd600640 1 -- 172.21.15.32:0/2646269784 shutdown_connections 2024-04-17T10:04:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.962+0000 7fe0dd600640 1 --2- 172.21.15.32:0/2646269784 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe0b4068380 0x7fe0b406a840 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.962+0000 7fe0dd600640 1 --2- 172.21.15.32:0/2646269784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0d80cce40 0x7fe0d80cd2c0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.962+0000 7fe0dd600640 1 --2- 172.21.15.32:0/2646269784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0d8144f00 0x7fe0d80a9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.962+0000 7fe0dd600640 1 --2- 172.21.15.32:0/2646269784 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0bc004810 0x7fe0d815df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.962+0000 7fe0dd600640 1 -- 172.21.15.32:0/2646269784 >> 172.21.15.32:0/2646269784 conn(0x7fe0d80b2120 msgr2=0x7fe0d815aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.962+0000 7fe0dd600640 1 -- 172.21.15.32:0/2646269784 shutdown_connections 2024-04-17T10:04:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:32.962+0000 7fe0dd600640 1 -- 172.21.15.32:0/2646269784 wait complete. 2024-04-17T10:04:32.977 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:04:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.126+0000 7f0d5c800640 1 Processor -- start 2024-04-17T10:04:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.126+0000 7f0d5c800640 1 -- start start 2024-04-17T10:04:33.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.126+0000 7f0d5c800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d5415b7d0 0x7f0d5406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.126+0000 7f0d5c800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d5406f7c0 0x7f0d5406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.126+0000 7f0d5c800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0d5406f270 0x7f0d5406bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.126+0000 7f0d5c800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0d540cc940 con 0x7f0d5406f7c0 2024-04-17T10:04:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.126+0000 7f0d5c800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0d540ccc20 con 0x7f0d5406f270 2024-04-17T10:04:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.126+0000 7f0d5c800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0d5406e0b0 con 0x7f0d5415b7d0 2024-04-17T10:04:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.126+0000 7f0d59400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d5406f7c0 0x7f0d5406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.126+0000 7f0d59400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d5406f7c0 0x7f0d5406ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59684/0 (socket says 172.21.15.32:59684) 2024-04-17T10:04:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.126+0000 7f0d59400640 1 -- 172.21.15.32:0/4144584847 learned_addr learned my addr 172.21.15.32:0/4144584847 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.126+0000 7f0d59e00640 1 --2- 172.21.15.32:0/4144584847 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d5415b7d0 0x7f0d5406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.126+0000 7f0d5a800640 1 --2- 172.21.15.32:0/4144584847 >> v2:172.21.15.144:3300/0 conn(0x7f0d5406f270 0x7f0d5406bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:33.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.126+0000 7f0d59400640 1 -- 172.21.15.32:0/4144584847 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d5415b7d0 msgr2=0x7f0d5406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:33.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.126+0000 7f0d59400640 1 --2- 172.21.15.32:0/4144584847 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d5415b7d0 0x7f0d5406e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:33.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.126+0000 7f0d59400640 1 -- 172.21.15.32:0/4144584847 >> v2:172.21.15.144:3300/0 conn(0x7f0d5406f270 msgr2=0x7f0d5406bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:33.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.126+0000 7f0d59400640 1 --2- 172.21.15.32:0/4144584847 >> v2:172.21.15.144:3300/0 conn(0x7f0d5406f270 0x7f0d5406bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:33.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.126+0000 7f0d59400640 1 -- 172.21.15.32:0/4144584847 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5406e390 con 0x7f0d5406f7c0 2024-04-17T10:04:33.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.130+0000 7f0d59400640 1 --2- 172.21.15.32:0/4144584847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d5406f7c0 0x7f0d5406ba30 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f0d540b44e0 tx=0x7f0d4401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76943af4213dd17 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:33.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.130+0000 7f0d58a00640 1 -- 172.21.15.32:0/4144584847 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d440a7070 con 0x7f0d5406f7c0 2024-04-17T10:04:33.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.130+0000 7f0d58a00640 1 -- 172.21.15.32:0/4144584847 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d440b0070 con 0x7f0d5406f7c0 2024-04-17T10:04:33.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.130+0000 7f0d58a00640 1 -- 172.21.15.32:0/4144584847 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d440ab890 con 0x7f0d5406f7c0 2024-04-17T10:04:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.130+0000 7f0d5c800640 1 -- 172.21.15.32:0/4144584847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d5406f7c0 msgr2=0x7f0d5406ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.130+0000 7f0d5c800640 1 --2- 172.21.15.32:0/4144584847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d5406f7c0 0x7f0d5406ba30 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f0d540b44e0 tx=0x7f0d4401c490 comp rx=0 tx=0).stop 2024-04-17T10:04:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.130+0000 7f0d5c800640 1 -- 172.21.15.32:0/4144584847 shutdown_connections 2024-04-17T10:04:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.130+0000 7f0d5c800640 1 --2- 172.21.15.32:0/4144584847 >> v2:172.21.15.144:3300/0 conn(0x7f0d5406f270 0x7f0d5406bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.130+0000 7f0d5c800640 1 --2- 172.21.15.32:0/4144584847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d5406f7c0 0x7f0d5406ba30 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.130+0000 7f0d5c800640 1 --2- 172.21.15.32:0/4144584847 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d5415b7d0 0x7f0d5406e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.130+0000 7f0d5c800640 1 -- 172.21.15.32:0/4144584847 >> 172.21.15.32:0/4144584847 conn(0x7f0d540b2120 msgr2=0x7f0d540b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.130+0000 7f0d5c800640 1 -- 172.21.15.32:0/4144584847 shutdown_connections 2024-04-17T10:04:33.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.130+0000 7f0d5c800640 1 -- 172.21.15.32:0/4144584847 wait complete. 2024-04-17T10:04:33.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d5c800640 1 Processor -- start 2024-04-17T10:04:33.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d5c800640 1 -- start start 2024-04-17T10:04:33.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d5c800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d5406f270 0x7f0d540af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d5c800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d5406f7c0 0x7f0d540b0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d5c800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d5415b7d0 0x7f0d54146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d5c800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0d540cee70 con 0x7f0d5406f270 2024-04-17T10:04:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d5c800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0d540cecf0 con 0x7f0d5406f7c0 2024-04-17T10:04:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d5c800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0d540b3420 con 0x7f0d5415b7d0 2024-04-17T10:04:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d59e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d5406f270 0x7f0d540af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d59e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d5406f270 0x7f0d540af1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59700/0 (socket says 172.21.15.32:59700) 2024-04-17T10:04:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d59e00640 1 -- 172.21.15.32:0/3346265574 learned_addr learned my addr 172.21.15.32:0/3346265574 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d59400640 1 --2- 172.21.15.32:0/3346265574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d5406f7c0 0x7f0d540b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:33.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d5a800640 1 --2- 172.21.15.32:0/3346265574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d5415b7d0 0x7f0d54146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:33.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d59e00640 1 -- 172.21.15.32:0/3346265574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d5415b7d0 msgr2=0x7f0d54146710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:33.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d59e00640 1 --2- 172.21.15.32:0/3346265574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d5415b7d0 0x7f0d54146710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:33.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d59e00640 1 -- 172.21.15.32:0/3346265574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d5406f7c0 msgr2=0x7f0d540b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:33.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d59e00640 1 --2- 172.21.15.32:0/3346265574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d5406f7c0 0x7f0d540b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:33.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d59e00640 1 -- 172.21.15.32:0/3346265574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5414e400 con 0x7f0d5406f270 2024-04-17T10:04:33.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d5a800640 1 --2- 172.21.15.32:0/3346265574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d5415b7d0 0x7f0d54146710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:33.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d59e00640 1 --2- 172.21.15.32:0/3346265574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d5406f270 0x7f0d540af1f0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800e8a0 tx=0x7f0d4800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:33.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d4e000640 1 -- 172.21.15.32:0/3346265574 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d48017070 con 0x7f0d5406f270 2024-04-17T10:04:33.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d4e000640 1 -- 172.21.15.32:0/3346265574 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d4800c940 con 0x7f0d5406f270 2024-04-17T10:04:33.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d4e000640 1 -- 172.21.15.32:0/3346265574 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4800ccf0 con 0x7f0d5406f270 2024-04-17T10:04:33.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d5c800640 1 -- 172.21.15.32:0/3346265574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d540b05d0 con 0x7f0d5406f270 2024-04-17T10:04:33.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.134+0000 7f0d5c800640 1 -- 172.21.15.32:0/3346265574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d54157550 con 0x7f0d5406f270 2024-04-17T10:04:33.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.138+0000 7f0d5c800640 1 -- 172.21.15.32:0/3346265574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d54068a90 con 0x7f0d5406f270 2024-04-17T10:04:33.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.138+0000 7f0d4e000640 1 -- 172.21.15.32:0/3346265574 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0d4801a070 con 0x7f0d5406f270 2024-04-17T10:04:33.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.138+0000 7f0d4e000640 1 --2- 172.21.15.32:0/3346265574 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0d38068380 0x7f0d3806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:33.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.138+0000 7f0d4e000640 1 -- 172.21.15.32:0/3346265574 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f0d480868e0 con 0x7f0d5406f270 2024-04-17T10:04:33.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.138+0000 7f0d59400640 1 --2- 172.21.15.32:0/3346265574 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0d38068380 0x7f0d3806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:33.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.138+0000 7f0d59400640 1 --2- 172.21.15.32:0/3346265574 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0d38068380 0x7f0d3806a840 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0d440af000 tx=0x7f0d440a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:33.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.142+0000 7f0d4e000640 1 -- 172.21.15.32:0/3346265574 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0d48056760 con 0x7f0d5406f270 2024-04-17T10:04:33.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.310+0000 7f0d5c800640 1 -- 172.21.15.32:0/3346265574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0d54159ba0 con 0x7f0d5406f270 2024-04-17T10:04:33.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.310+0000 7f0d4e000640 1 -- 172.21.15.32:0/3346265574 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) ==== 100+0+11 (secure 0 0 0) 0x7f0d4805aa60 con 0x7f0d5406f270 2024-04-17T10:04:33.307 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T10:04:33.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.314+0000 7f0d5c800640 1 -- 172.21.15.32:0/3346265574 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0d38068380 msgr2=0x7f0d3806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:33.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.314+0000 7f0d5c800640 1 --2- 172.21.15.32:0/3346265574 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0d38068380 0x7f0d3806a840 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0d440af000 tx=0x7f0d440a7000 comp rx=0 tx=0).stop 2024-04-17T10:04:33.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.314+0000 7f0d5c800640 1 -- 172.21.15.32:0/3346265574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d5406f270 msgr2=0x7f0d540af1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:33.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.314+0000 7f0d5c800640 1 --2- 172.21.15.32:0/3346265574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d5406f270 0x7f0d540af1f0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800e8a0 tx=0x7f0d4800eca0 comp rx=0 tx=0).stop 2024-04-17T10:04:33.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.314+0000 7f0d5c800640 1 -- 172.21.15.32:0/3346265574 shutdown_connections 2024-04-17T10:04:33.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.314+0000 7f0d5c800640 1 --2- 172.21.15.32:0/3346265574 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0d38068380 0x7f0d3806a840 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:33.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.314+0000 7f0d5c800640 1 --2- 172.21.15.32:0/3346265574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d5415b7d0 0x7f0d54146710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:33.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.314+0000 7f0d5c800640 1 --2- 172.21.15.32:0/3346265574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d5406f7c0 0x7f0d540b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:33.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.314+0000 7f0d5c800640 1 --2- 172.21.15.32:0/3346265574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d5406f270 0x7f0d540af1f0 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:33.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.314+0000 7f0d5c800640 1 -- 172.21.15.32:0/3346265574 >> 172.21.15.32:0/3346265574 conn(0x7f0d540b2120 msgr2=0x7f0d54158c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:33.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.314+0000 7f0d5c800640 1 -- 172.21.15.32:0/3346265574 shutdown_connections 2024-04-17T10:04:33.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.314+0000 7f0d5c800640 1 -- 172.21.15.32:0/3346265574 wait complete. 2024-04-17T10:04:33.329 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T10:04:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.478+0000 7f0e4ec00640 1 Processor -- start 2024-04-17T10:04:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.478+0000 7f0e4ec00640 1 -- start start 2024-04-17T10:04:33.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.478+0000 7f0e4ec00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0e480b3340 0x7f0e4815fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:33.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.478+0000 7f0e4ec00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0e4809a9f0 0x7f0e4809ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.478+0000 7f0e4ec00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e4806ce70 con 0x7f0e4815eeb0 2024-04-17T10:04:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.478+0000 7f0e4ec00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f0e4809b380 con 0x7f0e480b3340 2024-04-17T10:04:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.478+0000 7f0e4ec00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0e4815f290 con 0x7f0e4809a9f0 2024-04-17T10:04:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.478+0000 7f0e47e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0e4815eeb0 0x7f0e4806d910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44330/0 (socket says 172.21.15.32:44330) 2024-04-17T10:04:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.478+0000 7f0e47e00640 1 -- 172.21.15.32:0/928926201 learned_addr learned my addr 172.21.15.32:0/928926201 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.478+0000 7f0e47400640 1 --2- 172.21.15.32:0/928926201 >> v2:172.21.15.32:3300/0 conn(0x7f0e480b3340 0x7f0e4815fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.478+0000 7f0e4cc00640 1 --2- 172.21.15.32:0/928926201 >> v2:172.21.15.144:3300/0 conn(0x7f0e4809a9f0 0x7f0e4809ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.478+0000 7f0e46a00640 1 -- 172.21.15.32:0/928926201 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2297224205 0 0) 0x7f0e4806ce70 con 0x7f0e4815eeb0 2024-04-17T10:04:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.478+0000 7f0e47400640 1 -- 172.21.15.32:0/928926201 >> v2:172.21.15.144:3300/0 conn(0x7f0e4809a9f0 msgr2=0x7f0e4809ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.478+0000 7f0e47400640 1 --2- 172.21.15.32:0/928926201 >> v2:172.21.15.144:3300/0 conn(0x7f0e4809a9f0 0x7f0e4809ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.478+0000 7f0e47400640 1 -- 172.21.15.32:0/928926201 >> v1:172.21.15.32:6789/0 conn(0x7f0e4815eeb0 legacy=0x7f0e4806d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.478+0000 7f0e47400640 1 -- 172.21.15.32:0/928926201 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4809b9a0 con 0x7f0e480b3340 2024-04-17T10:04:33.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.482+0000 7f0e47400640 1 --2- 172.21.15.32:0/928926201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e480b3340 0x7f0e4815fec0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f0e340097c0 tx=0x7f0e3401ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317a8051fd13c3c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:33.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.482+0000 7f0e46a00640 1 -- 172.21.15.32:0/928926201 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e340b5070 con 0x7f0e480b3340 2024-04-17T10:04:33.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.482+0000 7f0e46a00640 1 -- 172.21.15.32:0/928926201 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e340a8950 con 0x7f0e480b3340 2024-04-17T10:04:33.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.482+0000 7f0e46a00640 1 -- 172.21.15.32:0/928926201 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e340a8d20 con 0x7f0e480b3340 2024-04-17T10:04:33.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.482+0000 7f0e4ec00640 1 -- 172.21.15.32:0/928926201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e480b3340 msgr2=0x7f0e4815fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:33.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.482+0000 7f0e4ec00640 1 --2- 172.21.15.32:0/928926201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e480b3340 0x7f0e4815fec0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f0e340097c0 tx=0x7f0e3401ec50 comp rx=0 tx=0).stop 2024-04-17T10:04:33.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.482+0000 7f0e4ec00640 1 -- 172.21.15.32:0/928926201 shutdown_connections 2024-04-17T10:04:33.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.482+0000 7f0e4ec00640 1 --2- 172.21.15.32:0/928926201 >> v2:172.21.15.144:3300/0 conn(0x7f0e4809a9f0 0x7f0e4809ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:33.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.482+0000 7f0e4ec00640 1 --2- 172.21.15.32:0/928926201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e480b3340 0x7f0e4815fec0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:33.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.482+0000 7f0e4ec00640 1 -- 172.21.15.32:0/928926201 >> 172.21.15.32:0/928926201 conn(0x7f0e480b40a0 msgr2=0x7f0e480b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:33.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.482+0000 7f0e4ec00640 1 -- 172.21.15.32:0/928926201 shutdown_connections 2024-04-17T10:04:33.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.482+0000 7f0e4ec00640 1 -- 172.21.15.32:0/928926201 wait complete. 2024-04-17T10:04:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.482+0000 7f0e4ec00640 1 Processor -- start 2024-04-17T10:04:33.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e4ec00640 1 -- start start 2024-04-17T10:04:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e4ec00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e4809a9f0 0x7f0e4814cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e4ec00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e480b3340 0x7f0e480b7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e4ec00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e4815eeb0 0x7f0e480c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e4ec00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4806d3f0 con 0x7f0e480b3340 2024-04-17T10:04:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e4ec00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4806d110 con 0x7f0e4809a9f0 2024-04-17T10:04:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e4ec00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0e480d09c0 con 0x7f0e4815eeb0 2024-04-17T10:04:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e47400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e480b3340 0x7f0e480b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e47400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e480b3340 0x7f0e480b7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59710/0 (socket says 172.21.15.32:59710) 2024-04-17T10:04:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e47400640 1 -- 172.21.15.32:0/145885797 learned_addr learned my addr 172.21.15.32:0/145885797 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e47e00640 1 --2- 172.21.15.32:0/145885797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e4809a9f0 0x7f0e4814cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e4cc00640 1 --2- 172.21.15.32:0/145885797 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e4815eeb0 0x7f0e480c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e47400640 1 -- 172.21.15.32:0/145885797 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e4815eeb0 msgr2=0x7f0e480c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e47400640 1 --2- 172.21.15.32:0/145885797 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e4815eeb0 0x7f0e480c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e47400640 1 -- 172.21.15.32:0/145885797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e4809a9f0 msgr2=0x7f0e4814cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e47400640 1 --2- 172.21.15.32:0/145885797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e4809a9f0 0x7f0e4814cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e47400640 1 -- 172.21.15.32:0/145885797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e480b7e40 con 0x7f0e480b3340 2024-04-17T10:04:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e47400640 1 --2- 172.21.15.32:0/145885797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e480b3340 0x7f0e480b7460 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f0e340b3040 tx=0x7f0e3401b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e3fe00640 1 -- 172.21.15.32:0/145885797 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e340bf070 con 0x7f0e480b3340 2024-04-17T10:04:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e3fe00640 1 -- 172.21.15.32:0/145885797 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e340b0720 con 0x7f0e480b3340 2024-04-17T10:04:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e3fe00640 1 -- 172.21.15.32:0/145885797 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e340b5040 con 0x7f0e480b3340 2024-04-17T10:04:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e4ec00640 1 -- 172.21.15.32:0/145885797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e480c62f0 con 0x7f0e480b3340 2024-04-17T10:04:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e4ec00640 1 -- 172.21.15.32:0/145885797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e480c68b0 con 0x7f0e480b3340 2024-04-17T10:04:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.486+0000 7f0e4ec00640 1 -- 172.21.15.32:0/145885797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e48068a90 con 0x7f0e480b3340 2024-04-17T10:04:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.490+0000 7f0e3fe00640 1 -- 172.21.15.32:0/145885797 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0e340b0b10 con 0x7f0e480b3340 2024-04-17T10:04:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.490+0000 7f0e3fe00640 1 --2- 172.21.15.32:0/145885797 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0e28068380 0x7f0e2806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.490+0000 7f0e3fe00640 1 -- 172.21.15.32:0/145885797 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f0e34123310 con 0x7f0e480b3340 2024-04-17T10:04:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.490+0000 7f0e47e00640 1 --2- 172.21.15.32:0/145885797 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0e28068380 0x7f0e2806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.490+0000 7f0e47e00640 1 --2- 172.21.15.32:0/145885797 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0e28068380 0x7f0e2806a840 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0e38000f70 tx=0x7f0e38007df0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.494+0000 7f0e3fe00640 1 -- 172.21.15.32:0/145885797 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0e340187b0 con 0x7f0e480b3340 2024-04-17T10:04:33.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.670+0000 7f0e4ec00640 1 -- 172.21.15.32:0/145885797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0e480b4fb0 con 0x7f0e480b3340 2024-04-17T10:04:33.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.670+0000 7f0e3fe00640 1 -- 172.21.15.32:0/145885797 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v113) ==== 80+0+240 (secure 0 0 0) 0x7f0e341230d0 con 0x7f0e480b3340 2024-04-17T10:04:33.665 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6837/578643390 2024-04-18T10:04:22.429980+0000 2024-04-17T10:04:33.665 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6836/578643390 2024-04-18T10:04:22.429980+0000 2024-04-17T10:04:33.665 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6837/901447746 2024-04-18T10:04:03.080821+0000 2024-04-17T10:04:33.665 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6836/901447746 2024-04-18T10:04:03.080821+0000 2024-04-17T10:04:33.665 INFO:teuthology.orchestra.run.smithi032.stderr:listed 4 entries 2024-04-17T10:04:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.670+0000 7f0e4ec00640 1 -- 172.21.15.32:0/145885797 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0e28068380 msgr2=0x7f0e2806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.670+0000 7f0e4ec00640 1 --2- 172.21.15.32:0/145885797 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0e28068380 0x7f0e2806a840 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0e38000f70 tx=0x7f0e38007df0 comp rx=0 tx=0).stop 2024-04-17T10:04:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.670+0000 7f0e4ec00640 1 -- 172.21.15.32:0/145885797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e480b3340 msgr2=0x7f0e480b7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.670+0000 7f0e4ec00640 1 --2- 172.21.15.32:0/145885797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e480b3340 0x7f0e480b7460 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f0e340b3040 tx=0x7f0e3401b9f0 comp rx=0 tx=0).stop 2024-04-17T10:04:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.670+0000 7f0e4ec00640 1 -- 172.21.15.32:0/145885797 shutdown_connections 2024-04-17T10:04:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.670+0000 7f0e4ec00640 1 --2- 172.21.15.32:0/145885797 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0e28068380 0x7f0e2806a840 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.670+0000 7f0e4ec00640 1 --2- 172.21.15.32:0/145885797 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e4815eeb0 0x7f0e480c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.670+0000 7f0e4ec00640 1 --2- 172.21.15.32:0/145885797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e480b3340 0x7f0e480b7460 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.670+0000 7f0e4ec00640 1 --2- 172.21.15.32:0/145885797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e4809a9f0 0x7f0e4814cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.670+0000 7f0e4ec00640 1 -- 172.21.15.32:0/145885797 >> 172.21.15.32:0/145885797 conn(0x7f0e480b40a0 msgr2=0x7f0e4806d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.670+0000 7f0e4ec00640 1 -- 172.21.15.32:0/145885797 shutdown_connections 2024-04-17T10:04:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:33.670+0000 7f0e4ec00640 1 -- 172.21.15.32:0/145885797 wait complete. 2024-04-17T10:04:33.686 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2024-04-17T10:04:33.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:33.686 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-17T10:04:33.704 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[53235]: fuse finished with error 0 and tester_r 0 2024-04-17T10:04:33.706 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T10:04:33.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:33.707 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T10:04:33.775 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T10:04:33.775 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T10:04:33.775 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:04:33.785 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:04:33.785 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T10:04:33.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:33.785 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-17T10:04:33.835 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-17T10:04:33.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:04:33.836 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T10:04:33.836 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:04:33.836 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:04:33.845 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:04:33.845 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T10:04:33.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:33.845 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-17T10:04:33.894 INFO:teuthology.orchestra.run.smithi032.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-17T10:04:33.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:04:33.895 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T10:04:33.895 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T10:04:33.895 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:04:33.904 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:04:33.904 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T10:04:33.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:33.904 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-17T10:04:33.955 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-17T10:04:33.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:04:33.956 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:04:34.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.102+0000 7fb5c4400640 1 Processor -- start 2024-04-17T10:04:34.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.102+0000 7fb5c4400640 1 -- start start 2024-04-17T10:04:34.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.102+0000 7fb5c4400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb5bc0d0890 0x7fb5bc0d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:34.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.102+0000 7fb5c4400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5bc0cdb10 0x7fb5bc0cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:34.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.102+0000 7fb5c4400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5bc162620 con 0x7fb5bc0b2430 2024-04-17T10:04:34.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.102+0000 7fb5c4400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fb5bc162900 con 0x7fb5bc0cdb10 2024-04-17T10:04:34.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.102+0000 7fb5c4400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb5bc162be0 con 0x7fb5bc0d0890 2024-04-17T10:04:34.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.102+0000 7fb5c2400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5bc0cdb10 0x7fb5bc0cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:34.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.102+0000 7fb5c2400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5bc0cdb10 0x7fb5bc0cdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59712/0 (socket says 172.21.15.32:59712) 2024-04-17T10:04:34.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.102+0000 7fb5c2400640 1 -- 172.21.15.32:0/1572726984 learned_addr learned my addr 172.21.15.32:0/1572726984 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:34.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.102+0000 7fb5c1000640 1 --2- 172.21.15.32:0/1572726984 >> v2:172.21.15.144:3300/0 conn(0x7fb5bc0d0890 0x7fb5bc0d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:34.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.102+0000 7fb5c2400640 1 -- 172.21.15.32:0/1572726984 >> v2:172.21.15.144:3300/0 conn(0x7fb5bc0d0890 msgr2=0x7fb5bc0d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:34.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.102+0000 7fb5c2400640 1 --2- 172.21.15.32:0/1572726984 >> v2:172.21.15.144:3300/0 conn(0x7fb5bc0d0890 0x7fb5bc0d0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:34.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.102+0000 7fb5c2400640 1 -- 172.21.15.32:0/1572726984 >> v1:172.21.15.144:6789/0 conn(0x7fb5bc0b2430 legacy=0x7fb5bc0b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:34.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.102+0000 7fb5c2400640 1 -- 172.21.15.32:0/1572726984 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5bc162ec0 con 0x7fb5bc0cdb10 2024-04-17T10:04:34.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.102+0000 7fb5c1000640 1 --2- 172.21.15.32:0/1572726984 >> v2:172.21.15.144:3300/0 conn(0x7fb5bc0d0890 0x7fb5bc0d0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:34.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.106+0000 7fb5c2400640 1 --2- 172.21.15.32:0/1572726984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5bc0cdb10 0x7fb5bc0cdf20 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc162430 tx=0x7fb5b401c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc0804c5b4e04d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:34.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.106+0000 7fb5bbe00640 1 -- 172.21.15.32:0/1572726984 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b40a56a0 con 0x7fb5bc0cdb10 2024-04-17T10:04:34.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.106+0000 7fb5bbe00640 1 -- 172.21.15.32:0/1572726984 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b40a5840 con 0x7fb5bc0cdb10 2024-04-17T10:04:34.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.106+0000 7fb5bbe00640 1 -- 172.21.15.32:0/1572726984 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b40a5c10 con 0x7fb5bc0cdb10 2024-04-17T10:04:34.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.106+0000 7fb5c4400640 1 -- 172.21.15.32:0/1572726984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5bc0cdb10 msgr2=0x7fb5bc0cdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:34.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.106+0000 7fb5c4400640 1 --2- 172.21.15.32:0/1572726984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5bc0cdb10 0x7fb5bc0cdf20 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc162430 tx=0x7fb5b401c8d0 comp rx=0 tx=0).stop 2024-04-17T10:04:34.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.106+0000 7fb5c4400640 1 -- 172.21.15.32:0/1572726984 shutdown_connections 2024-04-17T10:04:34.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.106+0000 7fb5c4400640 1 --2- 172.21.15.32:0/1572726984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5bc0cdb10 0x7fb5bc0cdf20 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:34.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.106+0000 7fb5c4400640 1 --2- 172.21.15.32:0/1572726984 >> v2:172.21.15.144:3300/0 conn(0x7fb5bc0d0890 0x7fb5bc0d0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:34.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.106+0000 7fb5c4400640 1 -- 172.21.15.32:0/1572726984 >> 172.21.15.32:0/1572726984 conn(0x7fb5bc09a9f0 msgr2=0x7fb5bc0b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:34.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.106+0000 7fb5c4400640 1 -- 172.21.15.32:0/1572726984 shutdown_connections 2024-04-17T10:04:34.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.106+0000 7fb5c4400640 1 -- 172.21.15.32:0/1572726984 wait complete. 2024-04-17T10:04:34.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5c4400640 1 Processor -- start 2024-04-17T10:04:34.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5c4400640 1 -- start start 2024-04-17T10:04:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5c4400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5bc0b2430 0x7fb5bc0b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5c4400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5bc0cdb10 0x7fb5bc15a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5c4400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5bc0d0890 0x7fb5bc14fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5c4400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc16bc20 con 0x7fb5bc0b2430 2024-04-17T10:04:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5c4400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc16baa0 con 0x7fb5bc0cdb10 2024-04-17T10:04:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5c4400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb5bc16bda0 con 0x7fb5bc0d0890 2024-04-17T10:04:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5c1a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5bc0b2430 0x7fb5bc0b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5c1a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5bc0b2430 0x7fb5bc0b6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59718/0 (socket says 172.21.15.32:59718) 2024-04-17T10:04:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5c1a00640 1 -- 172.21.15.32:0/340342501 learned_addr learned my addr 172.21.15.32:0/340342501 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5c1000640 1 --2- 172.21.15.32:0/340342501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5bc0cdb10 0x7fb5bc15a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5c2400640 1 --2- 172.21.15.32:0/340342501 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5bc0d0890 0x7fb5bc14fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5c1a00640 1 -- 172.21.15.32:0/340342501 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5bc0d0890 msgr2=0x7fb5bc14fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5c1a00640 1 --2- 172.21.15.32:0/340342501 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5bc0d0890 0x7fb5bc14fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5c1a00640 1 -- 172.21.15.32:0/340342501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5bc0cdb10 msgr2=0x7fb5bc15a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5c1a00640 1 --2- 172.21.15.32:0/340342501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5bc0cdb10 0x7fb5bc15a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5c1a00640 1 -- 172.21.15.32:0/340342501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5bc0b0050 con 0x7fb5bc0b2430 2024-04-17T10:04:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5c1a00640 1 --2- 172.21.15.32:0/340342501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5bc0b2430 0x7fb5bc0b6c30 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000c3a0 tx=0x7fb5b000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5b9600640 1 -- 172.21.15.32:0/340342501 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b00052d0 con 0x7fb5bc0b2430 2024-04-17T10:04:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5b9600640 1 -- 172.21.15.32:0/340342501 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b0005470 con 0x7fb5bc0b2430 2024-04-17T10:04:34.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5c4400640 1 -- 172.21.15.32:0/340342501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5bc15ac60 con 0x7fb5bc0b2430 2024-04-17T10:04:34.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5b9600640 1 -- 172.21.15.32:0/340342501 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0007260 con 0x7fb5bc0b2430 2024-04-17T10:04:34.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5c4400640 1 -- 172.21.15.32:0/340342501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5bc0b05d0 con 0x7fb5bc0b2430 2024-04-17T10:04:34.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.110+0000 7fb5c4400640 1 -- 172.21.15.32:0/340342501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5bc068a90 con 0x7fb5bc0b2430 2024-04-17T10:04:34.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.114+0000 7fb5b9600640 1 -- 172.21.15.32:0/340342501 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb5b0007400 con 0x7fb5bc0b2430 2024-04-17T10:04:34.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.114+0000 7fb5b9600640 1 --2- 172.21.15.32:0/340342501 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb59c068380 0x7fb59c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:34.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.114+0000 7fb5b9600640 1 -- 172.21.15.32:0/340342501 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7fb5b0085fe0 con 0x7fb5bc0b2430 2024-04-17T10:04:34.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.114+0000 7fb5c1000640 1 --2- 172.21.15.32:0/340342501 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb59c068380 0x7fb59c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:34.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.114+0000 7fb5c1000640 1 --2- 172.21.15.32:0/340342501 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb59c068380 0x7fb59c06a840 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc1fb090 tx=0x7fb5a8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:34.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.118+0000 7fb5b9600640 1 -- 172.21.15.32:0/340342501 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fb5b0056430 con 0x7fb5bc0b2430 2024-04-17T10:04:34.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.322+0000 7fb5c4400640 1 -- 172.21.15.32:0/340342501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5bc0b3ed0 con 0x7fb5bc0b2430 2024-04-17T10:04:34.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.322+0000 7fb5b9600640 1 -- 172.21.15.32:0/340342501 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5212 (secure 0 0 0) 0x7fb5b0055d30 con 0x7fb5bc0b2430 2024-04-17T10:04:34.317 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:34.317 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7586,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3601627312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3601627312},{"type":"v1","addr":"172.21.15.144:6834","nonce":3601627312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2382359917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2382359917},{"type":"v1","addr":"172.21.15.144:6835","nonce":2382359917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1857552160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1857552160},{"type":"v1","addr":"172.21.15.32:6835","nonce":1857552160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:03:37.619785+0000","modified":"2024-04-17T10:04:27.457010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9831},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9831":{"gid":9831,"name":"a","rank":0,"incarnation":108,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6837/1204910660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1204910660},{"type":"v1","addr":"172.21.15.32:6837","nonce":1204910660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9831,"qdb_cluster":[9831]},"id":8}]} 2024-04-17T10:04:34.317 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 111 2024-04-17T10:04:34.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.322+0000 7fb5c4400640 1 -- 172.21.15.32:0/340342501 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb59c068380 msgr2=0x7fb59c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:34.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.322+0000 7fb5c4400640 1 --2- 172.21.15.32:0/340342501 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb59c068380 0x7fb59c06a840 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc1fb090 tx=0x7fb5a8008040 comp rx=0 tx=0).stop 2024-04-17T10:04:34.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.322+0000 7fb5c4400640 1 -- 172.21.15.32:0/340342501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5bc0b2430 msgr2=0x7fb5bc0b6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:34.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.322+0000 7fb5c4400640 1 --2- 172.21.15.32:0/340342501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5bc0b2430 0x7fb5bc0b6c30 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000c3a0 tx=0x7fb5b000c860 comp rx=0 tx=0).stop 2024-04-17T10:04:34.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.322+0000 7fb5c4400640 1 -- 172.21.15.32:0/340342501 shutdown_connections 2024-04-17T10:04:34.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.322+0000 7fb5c4400640 1 --2- 172.21.15.32:0/340342501 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fb59c068380 0x7fb59c06a840 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:34.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.322+0000 7fb5c4400640 1 --2- 172.21.15.32:0/340342501 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5bc0d0890 0x7fb5bc14fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:34.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.322+0000 7fb5c4400640 1 --2- 172.21.15.32:0/340342501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5bc0cdb10 0x7fb5bc15a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:34.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.322+0000 7fb5c4400640 1 --2- 172.21.15.32:0/340342501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5bc0b2430 0x7fb5bc0b6c30 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:34.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.322+0000 7fb5c4400640 1 -- 172.21.15.32:0/340342501 >> 172.21.15.32:0/340342501 conn(0x7fb5bc09a9f0 msgr2=0x7fb5bc15c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:34.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.322+0000 7fb5c4400640 1 -- 172.21.15.32:0/340342501 shutdown_connections 2024-04-17T10:04:34.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.326+0000 7fb5c4400640 1 -- 172.21.15.32:0/340342501 wait complete. 2024-04-17T10:04:34.340 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:04:34.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.486+0000 7f8164a00640 1 Processor -- start 2024-04-17T10:04:34.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.486+0000 7f8164a00640 1 -- start start 2024-04-17T10:04:34.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.486+0000 7f8164a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f816015fb70 0x7f81600b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:34.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.486+0000 7f8164a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f816015ff50 0x7f816009a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:34.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.486+0000 7f8164a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81600b4d40 con 0x7f81600b12a0 2024-04-17T10:04:34.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.486+0000 7f8164a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f81600d0c70 con 0x7f816015fb70 2024-04-17T10:04:34.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.486+0000 7f8164a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f81600d0ec0 con 0x7f816015ff50 2024-04-17T10:04:34.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.486+0000 7f815f200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f81600b12a0 0x7f81600cd6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44332/0 (socket says 172.21.15.32:44332) 2024-04-17T10:04:34.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.486+0000 7f815f200640 1 -- 172.21.15.32:0/2308955694 learned_addr learned my addr 172.21.15.32:0/2308955694 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:34.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.486+0000 7f815e800640 1 --2- 172.21.15.32:0/2308955694 >> v2:172.21.15.144:3300/0 conn(0x7f816015fb70 0x7f81600b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:34.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.486+0000 7f8157e00640 1 --2- 172.21.15.32:0/2308955694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f816015ff50 0x7f816009a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:34.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.486+0000 7f815de00640 1 -- 172.21.15.32:0/2308955694 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 661110248 0 0) 0x7f81600b4d40 con 0x7f81600b12a0 2024-04-17T10:04:34.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.486+0000 7f815de00640 1 -- 172.21.15.32:0/2308955694 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8144003530 con 0x7f81600b12a0 2024-04-17T10:04:34.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.486+0000 7f815e800640 1 -- 172.21.15.32:0/2308955694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f816015ff50 msgr2=0x7f816009a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:34.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.486+0000 7f815e800640 1 --2- 172.21.15.32:0/2308955694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f816015ff50 0x7f816009a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:34.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.486+0000 7f815e800640 1 -- 172.21.15.32:0/2308955694 >> v1:172.21.15.32:6789/0 conn(0x7f81600b12a0 legacy=0x7f81600cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:34.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.486+0000 7f815e800640 1 -- 172.21.15.32:0/2308955694 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81600d11a0 con 0x7f816015fb70 2024-04-17T10:04:34.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.486+0000 7f8157e00640 1 --2- 172.21.15.32:0/2308955694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f816015ff50 0x7f816009a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:04:34.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.490+0000 7f815e800640 1 --2- 172.21.15.32:0/2308955694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f816015fb70 0x7f81600b1bb0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f81500097c0 tx=0x7f815001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3557621322a6164 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:34.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.490+0000 7f815de00640 1 -- 172.21.15.32:0/2308955694 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81500b0070 con 0x7f816015fb70 2024-04-17T10:04:34.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.490+0000 7f815de00640 1 -- 172.21.15.32:0/2308955694 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81500ab600 con 0x7f816015fb70 2024-04-17T10:04:34.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.490+0000 7f815de00640 1 -- 172.21.15.32:0/2308955694 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81500ab9d0 con 0x7f816015fb70 2024-04-17T10:04:34.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.490+0000 7f8164a00640 1 -- 172.21.15.32:0/2308955694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f816015fb70 msgr2=0x7f81600b1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:34.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.490+0000 7f8164a00640 1 --2- 172.21.15.32:0/2308955694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f816015fb70 0x7f81600b1bb0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f81500097c0 tx=0x7f815001c410 comp rx=0 tx=0).stop 2024-04-17T10:04:34.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.490+0000 7f8164a00640 1 -- 172.21.15.32:0/2308955694 shutdown_connections 2024-04-17T10:04:34.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.490+0000 7f8164a00640 1 --2- 172.21.15.32:0/2308955694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f816015ff50 0x7f816009a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:34.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.490+0000 7f8164a00640 1 --2- 172.21.15.32:0/2308955694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f816015fb70 0x7f81600b1bb0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:34.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.490+0000 7f8164a00640 1 -- 172.21.15.32:0/2308955694 >> 172.21.15.32:0/2308955694 conn(0x7f81600b3e50 msgr2=0x7f81600b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:34.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.490+0000 7f8164a00640 1 -- 172.21.15.32:0/2308955694 shutdown_connections 2024-04-17T10:04:34.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.490+0000 7f8164a00640 1 -- 172.21.15.32:0/2308955694 wait complete. 2024-04-17T10:04:34.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f8164a00640 1 Processor -- start 2024-04-17T10:04:34.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f8164a00640 1 -- start start 2024-04-17T10:04:34.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f8164a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81600b12a0 0x7f816006cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:34.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f8164a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f816015fb70 0x7f81600b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:34.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f8164a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f816015ff50 0x7f8160144730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:34.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f8164a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f81601699a0 con 0x7f81600b12a0 2024-04-17T10:04:34.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f8164a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f816009b340 con 0x7f816015fb70 2024-04-17T10:04:34.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f8164a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8160169b20 con 0x7f816015ff50 2024-04-17T10:04:34.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f815e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81600b12a0 0x7f816006cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:34.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f815e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81600b12a0 0x7f816006cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59724/0 (socket says 172.21.15.32:59724) 2024-04-17T10:04:34.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f815e800640 1 -- 172.21.15.32:0/1720512867 learned_addr learned my addr 172.21.15.32:0/1720512867 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:34.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f8157e00640 1 --2- 172.21.15.32:0/1720512867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f816015fb70 0x7f81600b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:34.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f815f200640 1 --2- 172.21.15.32:0/1720512867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f816015ff50 0x7f8160144730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:34.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f815e800640 1 -- 172.21.15.32:0/1720512867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f816015ff50 msgr2=0x7f8160144730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:34.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f815e800640 1 --2- 172.21.15.32:0/1720512867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f816015ff50 0x7f8160144730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:34.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f815e800640 1 -- 172.21.15.32:0/1720512867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f816015fb70 msgr2=0x7f81600b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:34.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f815e800640 1 --2- 172.21.15.32:0/1720512867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f816015fb70 0x7f81600b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:34.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f815e800640 1 -- 172.21.15.32:0/1720512867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816006d290 con 0x7f81600b12a0 2024-04-17T10:04:34.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f815f200640 1 --2- 172.21.15.32:0/1720512867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f816015ff50 0x7f8160144730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:34.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f815e800640 1 --2- 172.21.15.32:0/1720512867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81600b12a0 0x7f816006cb40 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f81500ae040 tx=0x7f815001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:34.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f8156a00640 1 -- 172.21.15.32:0/1720512867 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81500b0050 con 0x7f81600b12a0 2024-04-17T10:04:34.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f8156a00640 1 -- 172.21.15.32:0/1720512867 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81500bc070 con 0x7f81600b12a0 2024-04-17T10:04:34.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f8156a00640 1 -- 172.21.15.32:0/1720512867 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150006e20 con 0x7f81600b12a0 2024-04-17T10:04:34.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f8164a00640 1 -- 172.21.15.32:0/1720512867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816006d570 con 0x7f81600b12a0 2024-04-17T10:04:34.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.494+0000 7f8164a00640 1 -- 172.21.15.32:0/1720512867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f816006db30 con 0x7f81600b12a0 2024-04-17T10:04:34.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.498+0000 7f8164a00640 1 -- 172.21.15.32:0/1720512867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f816004fd00 con 0x7f81600b12a0 2024-04-17T10:04:34.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.498+0000 7f8156a00640 1 -- 172.21.15.32:0/1720512867 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f81500b5870 con 0x7f81600b12a0 2024-04-17T10:04:34.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.498+0000 7f8156a00640 1 --2- 172.21.15.32:0/1720512867 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f813c068380 0x7f813c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:34.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.498+0000 7f8156a00640 1 -- 172.21.15.32:0/1720512867 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f8150121090 con 0x7f81600b12a0 2024-04-17T10:04:34.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.498+0000 7f8157e00640 1 --2- 172.21.15.32:0/1720512867 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f813c068380 0x7f813c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:34.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.498+0000 7f8157e00640 1 --2- 172.21.15.32:0/1720512867 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f813c068380 0x7f813c06a840 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8148009580 tx=0x7f8148009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:34.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.502+0000 7f8156a00640 1 -- 172.21.15.32:0/1720512867 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f8150016090 con 0x7f81600b12a0 2024-04-17T10:04:34.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.666+0000 7f8164a00640 1 -- 172.21.15.32:0/1720512867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81600b0ad0 con 0x7f81600b12a0 2024-04-17T10:04:34.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.666+0000 7f8156a00640 1 -- 172.21.15.32:0/1720512867 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16991 (secure 0 0 0) 0x7f8150118e20 con 0x7f81600b12a0 2024-04-17T10:04:34.663 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:34.663 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":113,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:04:22.430002+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:03:36.021322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-17T10:03:36.581134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.5","mappings":[{"from":0,"to":5}]},{"pgid":"17.25","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6837/578643390":"2024-04-18T10:04:22.429980+0000","172.21.15.32:6836/578643390":"2024-04-18T10:04:22.429980+0000","172.21.15.32:6837/901447746":"2024-04-18T10:04:03.080821+0000","172.21.15.32:6836/901447746":"2024-04-18T10:04:03.080821+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:04:34.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.670+0000 7f8164a00640 1 -- 172.21.15.32:0/1720512867 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f813c068380 msgr2=0x7f813c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:34.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.670+0000 7f8164a00640 1 --2- 172.21.15.32:0/1720512867 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f813c068380 0x7f813c06a840 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8148009580 tx=0x7f8148009190 comp rx=0 tx=0).stop 2024-04-17T10:04:34.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.670+0000 7f8164a00640 1 -- 172.21.15.32:0/1720512867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81600b12a0 msgr2=0x7f816006cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:34.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.670+0000 7f8164a00640 1 --2- 172.21.15.32:0/1720512867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81600b12a0 0x7f816006cb40 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f81500ae040 tx=0x7f815001ac50 comp rx=0 tx=0).stop 2024-04-17T10:04:34.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.670+0000 7f8164a00640 1 -- 172.21.15.32:0/1720512867 shutdown_connections 2024-04-17T10:04:34.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.670+0000 7f8164a00640 1 --2- 172.21.15.32:0/1720512867 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f813c068380 0x7f813c06a840 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:34.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.670+0000 7f8164a00640 1 --2- 172.21.15.32:0/1720512867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f816015ff50 0x7f8160144730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:34.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.670+0000 7f8164a00640 1 --2- 172.21.15.32:0/1720512867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f816015fb70 0x7f81600b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:34.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.670+0000 7f8164a00640 1 --2- 172.21.15.32:0/1720512867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81600b12a0 0x7f816006cb40 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:34.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.670+0000 7f8164a00640 1 -- 172.21.15.32:0/1720512867 >> 172.21.15.32:0/1720512867 conn(0x7f81600b3e50 msgr2=0x7f8160071700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:34.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.670+0000 7f8164a00640 1 -- 172.21.15.32:0/1720512867 shutdown_connections 2024-04-17T10:04:34.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.670+0000 7f8164a00640 1 -- 172.21.15.32:0/1720512867 wait complete. 2024-04-17T10:04:34.685 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T09:52:59.309691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T10:03:36.021322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T10:03:36.581134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T10:04:34.685 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T10:04:34.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.834+0000 7f0311600640 1 Processor -- start 2024-04-17T10:04:34.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.834+0000 7f0311600640 1 -- start start 2024-04-17T10:04:34.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.834+0000 7f0311600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f030c06ba30 0x7f030c06be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:34.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.834+0000 7f0311600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f030c06c350 con 0x7f030c071e60 2024-04-17T10:04:34.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.834+0000 7f0311600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f030c0d0130 con 0x7f030c0cf370 2024-04-17T10:04:34.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.834+0000 7f0311600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f030c0cfef0 con 0x7f030c06ba30 2024-04-17T10:04:34.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.834+0000 7f030b400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f030c071e60 0x7f030c072240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44348/0 (socket says 172.21.15.32:44348) 2024-04-17T10:04:34.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.834+0000 7f030b400640 1 -- 172.21.15.32:0/1499177206 learned_addr learned my addr 172.21.15.32:0/1499177206 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.834+0000 7f030aa00640 1 --2- 172.21.15.32:0/1499177206 >> v2:172.21.15.144:3300/0 conn(0x7f030c06ba30 0x7f030c06be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.834+0000 7f030a000640 1 -- 172.21.15.32:0/1499177206 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 439884282 0 0) 0x7f030c06c350 con 0x7f030c071e60 2024-04-17T10:04:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.834+0000 7f030a000640 1 -- 172.21.15.32:0/1499177206 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02f0003530 con 0x7f030c071e60 2024-04-17T10:04:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.834+0000 7f030aa00640 1 -- 172.21.15.32:0/1499177206 >> v1:172.21.15.144:6789/0 conn(0x7f030c0cf370 legacy=0x7f030c0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.834+0000 7f030aa00640 1 -- 172.21.15.32:0/1499177206 >> v1:172.21.15.32:6789/0 conn(0x7f030c071e60 legacy=0x7f030c072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.834+0000 7f030aa00640 1 -- 172.21.15.32:0/1499177206 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030c15d090 con 0x7f030c06ba30 2024-04-17T10:04:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.838+0000 7f030aa00640 1 --2- 172.21.15.32:0/1499177206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f030c06ba30 0x7f030c06be10 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0097c0 tx=0x7f02fc01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc7068ae7e125c6c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:34.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.838+0000 7f030a000640 1 -- 172.21.15.32:0/1499177206 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0b1070 con 0x7f030c06ba30 2024-04-17T10:04:34.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.838+0000 7f030a000640 1 -- 172.21.15.32:0/1499177206 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02fc0a8050 con 0x7f030c06ba30 2024-04-17T10:04:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.838+0000 7f030a000640 1 -- 172.21.15.32:0/1499177206 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0ac860 con 0x7f030c06ba30 2024-04-17T10:04:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.838+0000 7f0311600640 1 -- 172.21.15.32:0/1499177206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f030c06ba30 msgr2=0x7f030c06be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.838+0000 7f0311600640 1 --2- 172.21.15.32:0/1499177206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f030c06ba30 0x7f030c06be10 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0097c0 tx=0x7f02fc01c3e0 comp rx=0 tx=0).stop 2024-04-17T10:04:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.838+0000 7f0311600640 1 -- 172.21.15.32:0/1499177206 shutdown_connections 2024-04-17T10:04:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.838+0000 7f0311600640 1 --2- 172.21.15.32:0/1499177206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f030c06ba30 0x7f030c06be10 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.838+0000 7f0311600640 1 -- 172.21.15.32:0/1499177206 >> 172.21.15.32:0/1499177206 conn(0x7f030c0b2120 msgr2=0x7f030c0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.838+0000 7f0311600640 1 -- 172.21.15.32:0/1499177206 shutdown_connections 2024-04-17T10:04:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.838+0000 7f0311600640 1 -- 172.21.15.32:0/1499177206 wait complete. 2024-04-17T10:04:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f0311600640 1 Processor -- start 2024-04-17T10:04:34.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f0311600640 1 -- start start 2024-04-17T10:04:34.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f0311600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f030c06ba30 0x7f030c153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f0311600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f030c071e60 0x7f030c13e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f0311600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f030c0cf370 0x7f030c0ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f0311600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f030c15db70 con 0x7f030c0cf370 2024-04-17T10:04:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f0311600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f030c15cd00 con 0x7f030c06ba30 2024-04-17T10:04:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f0311600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f030c15dcf0 con 0x7f030c071e60 2024-04-17T10:04:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f030b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f030c0cf370 0x7f030c0ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f030b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f030c0cf370 0x7f030c0ad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59726/0 (socket says 172.21.15.32:59726) 2024-04-17T10:04:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f030b400640 1 -- 172.21.15.32:0/3077343793 learned_addr learned my addr 172.21.15.32:0/3077343793 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f030aa00640 1 --2- 172.21.15.32:0/3077343793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f030c06ba30 0x7f030c153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f0302000640 1 --2- 172.21.15.32:0/3077343793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f030c071e60 0x7f030c13e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f030b400640 1 -- 172.21.15.32:0/3077343793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f030c071e60 msgr2=0x7f030c13e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f030b400640 1 --2- 172.21.15.32:0/3077343793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f030c071e60 0x7f030c13e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f030b400640 1 -- 172.21.15.32:0/3077343793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f030c06ba30 msgr2=0x7f030c153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:34.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f030b400640 1 --2- 172.21.15.32:0/3077343793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f030c06ba30 0x7f030c153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f030b400640 1 -- 172.21.15.32:0/3077343793 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030c154e20 con 0x7f030c0cf370 2024-04-17T10:04:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f030aa00640 1 --2- 172.21.15.32:0/3077343793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f030c06ba30 0x7f030c153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f030b400640 1 --2- 172.21.15.32:0/3077343793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f030c0cf370 0x7f030c0ad860 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f030400cb40 tx=0x7f030400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f0303400640 1 -- 172.21.15.32:0/3077343793 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304017070 con 0x7f030c0cf370 2024-04-17T10:04:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f0303400640 1 -- 172.21.15.32:0/3077343793 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f030400aca0 con 0x7f030c0cf370 2024-04-17T10:04:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f0303400640 1 -- 172.21.15.32:0/3077343793 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304012620 con 0x7f030c0cf370 2024-04-17T10:04:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f0311600640 1 -- 172.21.15.32:0/3077343793 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030c155100 con 0x7f030c0cf370 2024-04-17T10:04:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f0311600640 1 -- 172.21.15.32:0/3077343793 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f030c1582e0 con 0x7f030c0cf370 2024-04-17T10:04:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.842+0000 7f0311600640 1 -- 172.21.15.32:0/3077343793 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f030c068a90 con 0x7f030c0cf370 2024-04-17T10:04:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.846+0000 7f0303400640 1 -- 172.21.15.32:0/3077343793 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f030401a070 con 0x7f030c0cf370 2024-04-17T10:04:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.846+0000 7f0303400640 1 --2- 172.21.15.32:0/3077343793 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f02e8068380 0x7f02e806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.846+0000 7f0303400640 1 -- 172.21.15.32:0/3077343793 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f0304086980 con 0x7f030c0cf370 2024-04-17T10:04:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.846+0000 7f030aa00640 1 --2- 172.21.15.32:0/3077343793 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f02e8068380 0x7f02e806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:34.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.846+0000 7f030aa00640 1 --2- 172.21.15.32:0/3077343793 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f02e8068380 0x7f02e806a840 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0ae000 tx=0x7f02fc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:34.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:34.850+0000 7f0303400640 1 -- 172.21.15.32:0/3077343793 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0304056db0 con 0x7f030c0cf370 2024-04-17T10:04:35.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.014+0000 7f0311600640 1 -- 172.21.15.32:0/3077343793 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f030c06c850 con 0x7f030c0cf370 2024-04-17T10:04:35.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.014+0000 7f0303400640 1 -- 172.21.15.32:0/3077343793 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) ==== 93+0+10 (secure 0 0 0) 0x7f03040566b0 con 0x7f030c0cf370 2024-04-17T10:04:35.009 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-04-17T10:04:35.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.014+0000 7f0311600640 1 -- 172.21.15.32:0/3077343793 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f02e8068380 msgr2=0x7f02e806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:35.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.014+0000 7f0311600640 1 --2- 172.21.15.32:0/3077343793 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f02e8068380 0x7f02e806a840 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0ae000 tx=0x7f02fc008040 comp rx=0 tx=0).stop 2024-04-17T10:04:35.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.014+0000 7f0311600640 1 -- 172.21.15.32:0/3077343793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f030c0cf370 msgr2=0x7f030c0ad860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:35.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.014+0000 7f0311600640 1 --2- 172.21.15.32:0/3077343793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f030c0cf370 0x7f030c0ad860 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f030400cb40 tx=0x7f030400a060 comp rx=0 tx=0).stop 2024-04-17T10:04:35.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.014+0000 7f0311600640 1 -- 172.21.15.32:0/3077343793 shutdown_connections 2024-04-17T10:04:35.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.014+0000 7f0311600640 1 --2- 172.21.15.32:0/3077343793 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f02e8068380 0x7f02e806a840 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.014+0000 7f0311600640 1 --2- 172.21.15.32:0/3077343793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f030c071e60 0x7f030c13e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.014+0000 7f0311600640 1 --2- 172.21.15.32:0/3077343793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f030c0cf370 0x7f030c0ad860 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.014+0000 7f0311600640 1 --2- 172.21.15.32:0/3077343793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f030c06ba30 0x7f030c153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.014+0000 7f0311600640 1 -- 172.21.15.32:0/3077343793 >> 172.21.15.32:0/3077343793 conn(0x7f030c0b2120 msgr2=0x7f030c15c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:35.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.014+0000 7f0311600640 1 -- 172.21.15.32:0/3077343793 shutdown_connections 2024-04-17T10:04:35.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.014+0000 7f0311600640 1 -- 172.21.15.32:0/3077343793 wait complete. 2024-04-17T10:04:35.031 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T10:04:35.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.178+0000 7f523e800640 1 Processor -- start 2024-04-17T10:04:35.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.178+0000 7f523e800640 1 -- start start 2024-04-17T10:04:35.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.178+0000 7f523e800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52380cf370 0x7f52380cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:35.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.178+0000 7f523e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5238071e60 0x7f5238072270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:35.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.178+0000 7f523e800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52380d0130 con 0x7f523806ba30 2024-04-17T10:04:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.178+0000 7f523e800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f523815d090 con 0x7f52380cf370 2024-04-17T10:04:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.178+0000 7f523e800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f523815d370 con 0x7f5238071e60 2024-04-17T10:04:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.178+0000 7f5236a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52380cf370 0x7f52380cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.178+0000 7f5237400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f523806ba30 0x7f523806be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44350/0 (socket says 172.21.15.32:44350) 2024-04-17T10:04:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.178+0000 7f5237400640 1 -- 172.21.15.32:0/1710690651 learned_addr learned my addr 172.21.15.32:0/1710690651 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.182+0000 7f5237e00640 1 --2- 172.21.15.32:0/1710690651 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5238071e60 0x7f5238072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.182+0000 7f5236a00640 1 -- 172.21.15.32:0/1710690651 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5238071e60 msgr2=0x7f5238072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.182+0000 7f5236a00640 1 --2- 172.21.15.32:0/1710690651 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5238071e60 0x7f5238072270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.182+0000 7f5236a00640 1 -- 172.21.15.32:0/1710690651 >> v1:172.21.15.32:6789/0 conn(0x7f523806ba30 legacy=0x7f523806be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:35.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.182+0000 7f5236a00640 1 -- 172.21.15.32:0/1710690651 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523815d650 con 0x7f52380cf370 2024-04-17T10:04:35.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.182+0000 7f5236000640 1 -- 172.21.15.32:0/1710690651 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1857958737 0 0) 0x7f52380d0130 con 0x7f523806ba30 2024-04-17T10:04:35.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.182+0000 7f5236a00640 1 --2- 172.21.15.32:0/1710690651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52380cf370 0x7f52380cf7e0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f52240097c0 tx=0x7f522401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7aaddd8e725d2e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:35.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.182+0000 7f5236000640 1 -- 172.21.15.32:0/1710690651 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52240aa620 con 0x7f52380cf370 2024-04-17T10:04:35.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.182+0000 7f5236000640 1 -- 172.21.15.32:0/1710690651 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52240af070 con 0x7f52380cf370 2024-04-17T10:04:35.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.182+0000 7f5236000640 1 -- 172.21.15.32:0/1710690651 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52240aa9e0 con 0x7f52380cf370 2024-04-17T10:04:35.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.182+0000 7f523e800640 1 -- 172.21.15.32:0/1710690651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52380cf370 msgr2=0x7f52380cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:35.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.182+0000 7f523e800640 1 --2- 172.21.15.32:0/1710690651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52380cf370 0x7f52380cf7e0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f52240097c0 tx=0x7f522401c4a0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f523e800640 1 -- 172.21.15.32:0/1710690651 shutdown_connections 2024-04-17T10:04:35.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f523e800640 1 --2- 172.21.15.32:0/1710690651 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5238071e60 0x7f5238072270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f523e800640 1 --2- 172.21.15.32:0/1710690651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52380cf370 0x7f52380cf7e0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f523e800640 1 -- 172.21.15.32:0/1710690651 >> 172.21.15.32:0/1710690651 conn(0x7f52380b2120 msgr2=0x7f52380b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:35.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f523e800640 1 -- 172.21.15.32:0/1710690651 shutdown_connections 2024-04-17T10:04:35.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f523e800640 1 -- 172.21.15.32:0/1710690651 wait complete. 2024-04-17T10:04:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f523e800640 1 Processor -- start 2024-04-17T10:04:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f523e800640 1 -- start start 2024-04-17T10:04:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f523e800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f523806ba30 0x7f523815b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f523e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5238071e60 0x7f5238142400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f523e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52380cf370 0x7f5238142940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f523e800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5238146230 con 0x7f523806ba30 2024-04-17T10:04:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f523e800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f523815dd80 con 0x7f5238071e60 2024-04-17T10:04:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f523e800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f52381463b0 con 0x7f52380cf370 2024-04-17T10:04:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f5237400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f523806ba30 0x7f523815b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f5237400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f523806ba30 0x7f523815b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59742/0 (socket says 172.21.15.32:59742) 2024-04-17T10:04:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f5237400640 1 -- 172.21.15.32:0/1507569925 learned_addr learned my addr 172.21.15.32:0/1507569925 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f5237e00640 1 --2- 172.21.15.32:0/1507569925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52380cf370 0x7f5238142940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f5236a00640 1 --2- 172.21.15.32:0/1507569925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5238071e60 0x7f5238142400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f5237400640 1 -- 172.21.15.32:0/1507569925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52380cf370 msgr2=0x7f5238142940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f5237400640 1 --2- 172.21.15.32:0/1507569925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52380cf370 0x7f5238142940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f5237400640 1 -- 172.21.15.32:0/1507569925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5238071e60 msgr2=0x7f5238142400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f5237400640 1 --2- 172.21.15.32:0/1507569925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5238071e60 0x7f5238142400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f5237400640 1 -- 172.21.15.32:0/1507569925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52380b3890 con 0x7f523806ba30 2024-04-17T10:04:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f5237e00640 1 --2- 172.21.15.32:0/1507569925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52380cf370 0x7f5238142940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f5236a00640 1 --2- 172.21.15.32:0/1507569925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5238071e60 0x7f5238142400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f5237400640 1 --2- 172.21.15.32:0/1507569925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f523806ba30 0x7f523815b540 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f522800cc10 tx=0x7f522800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f522f400640 1 -- 172.21.15.32:0/1507569925 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5228017070 con 0x7f523806ba30 2024-04-17T10:04:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f522f400640 1 -- 172.21.15.32:0/1507569925 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f522800ce20 con 0x7f523806ba30 2024-04-17T10:04:35.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f522f400640 1 -- 172.21.15.32:0/1507569925 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5228012640 con 0x7f523806ba30 2024-04-17T10:04:35.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.186+0000 7f523e800640 1 -- 172.21.15.32:0/1507569925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52380b3b70 con 0x7f523806ba30 2024-04-17T10:04:35.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.190+0000 7f523e800640 1 -- 172.21.15.32:0/1507569925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52380b4130 con 0x7f523806ba30 2024-04-17T10:04:35.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.190+0000 7f523e800640 1 -- 172.21.15.32:0/1507569925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5238068a90 con 0x7f523806ba30 2024-04-17T10:04:35.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.190+0000 7f522f400640 1 -- 172.21.15.32:0/1507569925 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f522801a070 con 0x7f523806ba30 2024-04-17T10:04:35.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.190+0000 7f522f400640 1 --2- 172.21.15.32:0/1507569925 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5218068380 0x7f521806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:35.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.190+0000 7f522f400640 1 -- 172.21.15.32:0/1507569925 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f5228086880 con 0x7f523806ba30 2024-04-17T10:04:35.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.190+0000 7f5236a00640 1 --2- 172.21.15.32:0/1507569925 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5218068380 0x7f521806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:35.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.190+0000 7f5236a00640 1 --2- 172.21.15.32:0/1507569925 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5218068380 0x7f521806a840 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f52240ae000 tx=0x7f52240a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:35.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.194+0000 7f522f400640 1 -- 172.21.15.32:0/1507569925 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f5228056de0 con 0x7f523806ba30 2024-04-17T10:04:35.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.358+0000 7f523e800640 1 -- 172.21.15.32:0/1507569925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f523806c850 con 0x7f523806ba30 2024-04-17T10:04:35.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.362+0000 7f522f400640 1 -- 172.21.15.32:0/1507569925 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) ==== 104+0+11 (secure 0 0 0) 0x7f52280566e0 con 0x7f523806ba30 2024-04-17T10:04:35.356 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-04-17T10:04:35.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.362+0000 7f523e800640 1 -- 172.21.15.32:0/1507569925 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5218068380 msgr2=0x7f521806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:35.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.362+0000 7f523e800640 1 --2- 172.21.15.32:0/1507569925 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5218068380 0x7f521806a840 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f52240ae000 tx=0x7f52240a7000 comp rx=0 tx=0).stop 2024-04-17T10:04:35.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.362+0000 7f523e800640 1 -- 172.21.15.32:0/1507569925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f523806ba30 msgr2=0x7f523815b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:35.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.362+0000 7f523e800640 1 --2- 172.21.15.32:0/1507569925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f523806ba30 0x7f523815b540 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f522800cc10 tx=0x7f522800a430 comp rx=0 tx=0).stop 2024-04-17T10:04:35.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.362+0000 7f523e800640 1 -- 172.21.15.32:0/1507569925 shutdown_connections 2024-04-17T10:04:35.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.362+0000 7f523e800640 1 --2- 172.21.15.32:0/1507569925 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f5218068380 0x7f521806a840 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.362+0000 7f523e800640 1 --2- 172.21.15.32:0/1507569925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52380cf370 0x7f5238142940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.362+0000 7f523e800640 1 --2- 172.21.15.32:0/1507569925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5238071e60 0x7f5238142400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.362+0000 7f523e800640 1 --2- 172.21.15.32:0/1507569925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f523806ba30 0x7f523815b540 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.362+0000 7f523e800640 1 -- 172.21.15.32:0/1507569925 >> 172.21.15.32:0/1507569925 conn(0x7f52380b2120 msgr2=0x7f52380aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:35.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.362+0000 7f523e800640 1 -- 172.21.15.32:0/1507569925 shutdown_connections 2024-04-17T10:04:35.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.362+0000 7f523e800640 1 -- 172.21.15.32:0/1507569925 wait complete. 2024-04-17T10:04:35.377 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T10:04:35.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.526+0000 7fe989000640 1 Processor -- start 2024-04-17T10:04:35.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.526+0000 7fe989000640 1 -- start start 2024-04-17T10:04:35.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.526+0000 7fe989000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe98406ba30 0x7fe98406be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:35.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.526+0000 7fe989000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe9840cf370 0x7fe9840cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:35.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.526+0000 7fe989000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe984160350 0x7fe984160730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:35.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.526+0000 7fe989000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe98406c350 con 0x7fe98406ba30 2024-04-17T10:04:35.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.526+0000 7fe989000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe9840d0130 con 0x7fe9840cf370 2024-04-17T10:04:35.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.526+0000 7fe989000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe98415d090 con 0x7fe984160350 2024-04-17T10:04:35.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.526+0000 7fe981e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe98406ba30 0x7fe98406be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:35.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.526+0000 7fe981e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe98406ba30 0x7fe98406be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59744/0 (socket says 172.21.15.32:59744) 2024-04-17T10:04:35.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.526+0000 7fe981e00640 1 -- 172.21.15.32:0/1345124967 learned_addr learned my addr 172.21.15.32:0/1345124967 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:35.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.526+0000 7fe981e00640 1 -- 172.21.15.32:0/1345124967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe984160350 msgr2=0x7fe984160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:35.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.526+0000 7fe981e00640 1 --2- 172.21.15.32:0/1345124967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe984160350 0x7fe984160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.526+0000 7fe981e00640 1 -- 172.21.15.32:0/1345124967 >> v2:172.21.15.144:3300/0 conn(0x7fe9840cf370 msgr2=0x7fe9840cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:35.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.526+0000 7fe981e00640 1 --2- 172.21.15.32:0/1345124967 >> v2:172.21.15.144:3300/0 conn(0x7fe9840cf370 0x7fe9840cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.526+0000 7fe981e00640 1 -- 172.21.15.32:0/1345124967 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98415d370 con 0x7fe98406ba30 2024-04-17T10:04:35.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.530+0000 7fe981e00640 1 --2- 172.21.15.32:0/1345124967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe98406ba30 0x7fe98406be10 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fe9840afec0 tx=0x7fe97401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7ec2e30e5fd5bb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:35.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.530+0000 7fe981400640 1 -- 172.21.15.32:0/1345124967 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9740aa430 con 0x7fe98406ba30 2024-04-17T10:04:35.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.530+0000 7fe981400640 1 -- 172.21.15.32:0/1345124967 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9740af070 con 0x7fe98406ba30 2024-04-17T10:04:35.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.530+0000 7fe981400640 1 -- 172.21.15.32:0/1345124967 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9740aa7f0 con 0x7fe98406ba30 2024-04-17T10:04:35.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.530+0000 7fe989000640 1 -- 172.21.15.32:0/1345124967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe98406ba30 msgr2=0x7fe98406be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:35.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.530+0000 7fe989000640 1 --2- 172.21.15.32:0/1345124967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe98406ba30 0x7fe98406be10 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fe9840afec0 tx=0x7fe97401c4f0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.530+0000 7fe989000640 1 -- 172.21.15.32:0/1345124967 shutdown_connections 2024-04-17T10:04:35.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.530+0000 7fe989000640 1 --2- 172.21.15.32:0/1345124967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe984160350 0x7fe984160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.530+0000 7fe989000640 1 --2- 172.21.15.32:0/1345124967 >> v2:172.21.15.144:3300/0 conn(0x7fe9840cf370 0x7fe9840cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.530+0000 7fe989000640 1 --2- 172.21.15.32:0/1345124967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe98406ba30 0x7fe98406be10 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.530+0000 7fe989000640 1 -- 172.21.15.32:0/1345124967 >> 172.21.15.32:0/1345124967 conn(0x7fe9840b2120 msgr2=0x7fe9840b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:35.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.530+0000 7fe989000640 1 -- 172.21.15.32:0/1345124967 shutdown_connections 2024-04-17T10:04:35.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.530+0000 7fe989000640 1 -- 172.21.15.32:0/1345124967 wait complete. 2024-04-17T10:04:35.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe989000640 1 Processor -- start 2024-04-17T10:04:35.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe989000640 1 -- start start 2024-04-17T10:04:35.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe989000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe98406ba30 0x7fe9841fad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:35.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe989000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9840cf370 0x7fe984139be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:35.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe989000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe984160350 0x7fe984142830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:35.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe989000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe9840cfea0 con 0x7fe98406ba30 2024-04-17T10:04:35.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe989000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe9840cfd20 con 0x7fe9840cf370 2024-04-17T10:04:35.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe989000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe98415db70 con 0x7fe984160350 2024-04-17T10:04:35.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe981e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe98406ba30 0x7fe9841fad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:35.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe981e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe98406ba30 0x7fe9841fad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59756/0 (socket says 172.21.15.32:59756) 2024-04-17T10:04:35.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe981e00640 1 -- 172.21.15.32:0/2705290914 learned_addr learned my addr 172.21.15.32:0/2705290914 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:35.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe979400640 1 --2- 172.21.15.32:0/2705290914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9840cf370 0x7fe984139be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:35.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe981e00640 1 -- 172.21.15.32:0/2705290914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe984160350 msgr2=0x7fe984142830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:35.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe982800640 1 --2- 172.21.15.32:0/2705290914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe984160350 0x7fe984142830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:35.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe981e00640 1 --2- 172.21.15.32:0/2705290914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe984160350 0x7fe984142830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe981e00640 1 -- 172.21.15.32:0/2705290914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9840cf370 msgr2=0x7fe984139be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:35.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe981e00640 1 --2- 172.21.15.32:0/2705290914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9840cf370 0x7fe984139be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe981e00640 1 -- 172.21.15.32:0/2705290914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98415e700 con 0x7fe98406ba30 2024-04-17T10:04:35.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe979400640 1 --2- 172.21.15.32:0/2705290914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9840cf370 0x7fe984139be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:35.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe982800640 1 --2- 172.21.15.32:0/2705290914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe984160350 0x7fe984142830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:35.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe981e00640 1 --2- 172.21.15.32:0/2705290914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe98406ba30 0x7fe9841fad10 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fe97401cd90 tx=0x7fe97401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:35.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe97aa00640 1 -- 172.21.15.32:0/2705290914 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9740b3800 con 0x7fe98406ba30 2024-04-17T10:04:35.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe97aa00640 1 -- 172.21.15.32:0/2705290914 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9740b39c0 con 0x7fe98406ba30 2024-04-17T10:04:35.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe989000640 1 -- 172.21.15.32:0/2705290914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98415e9e0 con 0x7fe98406ba30 2024-04-17T10:04:35.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe97aa00640 1 -- 172.21.15.32:0/2705290914 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9740b3db0 con 0x7fe98406ba30 2024-04-17T10:04:35.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.534+0000 7fe989000640 1 -- 172.21.15.32:0/2705290914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe98415efa0 con 0x7fe98406ba30 2024-04-17T10:04:35.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.538+0000 7fe989000640 1 -- 172.21.15.32:0/2705290914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe984068a90 con 0x7fe98406ba30 2024-04-17T10:04:35.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.538+0000 7fe97aa00640 1 -- 172.21.15.32:0/2705290914 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe9740b2380 con 0x7fe98406ba30 2024-04-17T10:04:35.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.538+0000 7fe97aa00640 1 --2- 172.21.15.32:0/2705290914 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe9600683b0 0x7fe96006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:35.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.538+0000 7fe97aa00640 1 -- 172.21.15.32:0/2705290914 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7fe974120e90 con 0x7fe98406ba30 2024-04-17T10:04:35.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.538+0000 7fe979400640 1 --2- 172.21.15.32:0/2705290914 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe9600683b0 0x7fe96006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:35.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.538+0000 7fe979400640 1 --2- 172.21.15.32:0/2705290914 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe9600683b0 0x7fe96006a870 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe984140690 tx=0x7fe96c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:35.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.542+0000 7fe97aa00640 1 -- 172.21.15.32:0/2705290914 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fe9740c3060 con 0x7fe98406ba30 2024-04-17T10:04:35.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.706+0000 7fe989000640 1 -- 172.21.15.32:0/2705290914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe9840b27b0 con 0x7fe98406ba30 2024-04-17T10:04:35.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.706+0000 7fe97aa00640 1 -- 172.21.15.32:0/2705290914 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) ==== 100+0+11 (secure 0 0 0) 0x7fe974016110 con 0x7fe98406ba30 2024-04-17T10:04:35.702 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2024-04-17T10:04:35.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.710+0000 7fe989000640 1 -- 172.21.15.32:0/2705290914 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe9600683b0 msgr2=0x7fe96006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.710+0000 7fe989000640 1 --2- 172.21.15.32:0/2705290914 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe9600683b0 0x7fe96006a870 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe984140690 tx=0x7fe96c008040 comp rx=0 tx=0).stop 2024-04-17T10:04:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.710+0000 7fe989000640 1 -- 172.21.15.32:0/2705290914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe98406ba30 msgr2=0x7fe9841fad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.710+0000 7fe989000640 1 --2- 172.21.15.32:0/2705290914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe98406ba30 0x7fe9841fad10 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fe97401cd90 tx=0x7fe97401ad30 comp rx=0 tx=0).stop 2024-04-17T10:04:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.710+0000 7fe989000640 1 -- 172.21.15.32:0/2705290914 shutdown_connections 2024-04-17T10:04:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.710+0000 7fe989000640 1 --2- 172.21.15.32:0/2705290914 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe9600683b0 0x7fe96006a870 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.710+0000 7fe989000640 1 --2- 172.21.15.32:0/2705290914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe984160350 0x7fe984142830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.710+0000 7fe989000640 1 --2- 172.21.15.32:0/2705290914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9840cf370 0x7fe984139be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.710+0000 7fe989000640 1 --2- 172.21.15.32:0/2705290914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe98406ba30 0x7fe9841fad10 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.710+0000 7fe989000640 1 -- 172.21.15.32:0/2705290914 >> 172.21.15.32:0/2705290914 conn(0x7fe9840b2120 msgr2=0x7fe9840af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.710+0000 7fe989000640 1 -- 172.21.15.32:0/2705290914 shutdown_connections 2024-04-17T10:04:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.710+0000 7fe989000640 1 -- 172.21.15.32:0/2705290914 wait complete. 2024-04-17T10:04:35.723 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:04:35.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.866+0000 7fcae0200640 1 Processor -- start 2024-04-17T10:04:35.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.866+0000 7fcae0200640 1 -- start start 2024-04-17T10:04:35.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.870+0000 7fcae0200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcad8144e90 0x7fcad80b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:35.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.870+0000 7fcae0200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcad815c1b0 0x7fcad806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:35.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.870+0000 7fcae0200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcad8072bb0 con 0x7fcad80b4de0 2024-04-17T10:04:35.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.870+0000 7fcae0200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fcad8145270 con 0x7fcad815c1b0 2024-04-17T10:04:35.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.870+0000 7fcae0200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcad80b3dc0 con 0x7fcad8144e90 2024-04-17T10:04:35.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.870+0000 7fcade200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcad815c1b0 0x7fcad806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:35.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.870+0000 7fcade200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcad815c1b0 0x7fcad806ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59762/0 (socket says 172.21.15.32:59762) 2024-04-17T10:04:35.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.870+0000 7fcade200640 1 -- 172.21.15.32:0/95912584 learned_addr learned my addr 172.21.15.32:0/95912584 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:35.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.870+0000 7fcadd800640 1 --2- 172.21.15.32:0/95912584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcad8144e90 0x7fcad80b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:35.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.870+0000 7fcade200640 1 -- 172.21.15.32:0/95912584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcad8144e90 msgr2=0x7fcad80b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:35.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.870+0000 7fcade200640 1 --2- 172.21.15.32:0/95912584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcad8144e90 0x7fcad80b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.870+0000 7fcade200640 1 -- 172.21.15.32:0/95912584 >> v1:172.21.15.144:6789/0 conn(0x7fcad80b4de0 legacy=0x7fcad815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:35.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.870+0000 7fcade200640 1 -- 172.21.15.32:0/95912584 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad80b40a0 con 0x7fcad815c1b0 2024-04-17T10:04:35.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.874+0000 7fcade200640 1 --2- 172.21.15.32:0/95912584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcad815c1b0 0x7fcad806ef60 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009e80 tx=0x7fcad001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a0e5ead82c294a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:35.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.874+0000 7fcad7e00640 1 -- 172.21.15.32:0/95912584 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad00a58d0 con 0x7fcad815c1b0 2024-04-17T10:04:35.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.874+0000 7fcad7e00640 1 -- 172.21.15.32:0/95912584 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad00a5a70 con 0x7fcad815c1b0 2024-04-17T10:04:35.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.874+0000 7fcad7e00640 1 -- 172.21.15.32:0/95912584 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad00a5e40 con 0x7fcad815c1b0 2024-04-17T10:04:35.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.874+0000 7fcae0200640 1 -- 172.21.15.32:0/95912584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcad815c1b0 msgr2=0x7fcad806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:35.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.874+0000 7fcae0200640 1 --2- 172.21.15.32:0/95912584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcad815c1b0 0x7fcad806ef60 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009e80 tx=0x7fcad001c9e0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.874+0000 7fcae0200640 1 -- 172.21.15.32:0/95912584 shutdown_connections 2024-04-17T10:04:35.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.874+0000 7fcae0200640 1 --2- 172.21.15.32:0/95912584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcad815c1b0 0x7fcad806ef60 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.874+0000 7fcae0200640 1 --2- 172.21.15.32:0/95912584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcad8144e90 0x7fcad80b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.874+0000 7fcae0200640 1 -- 172.21.15.32:0/95912584 >> 172.21.15.32:0/95912584 conn(0x7fcad806ba30 msgr2=0x7fcad806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:35.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.874+0000 7fcae0200640 1 -- 172.21.15.32:0/95912584 shutdown_connections 2024-04-17T10:04:35.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.874+0000 7fcae0200640 1 -- 172.21.15.32:0/95912584 wait complete. 2024-04-17T10:04:35.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.874+0000 7fcae0200640 1 Processor -- start 2024-04-17T10:04:35.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.874+0000 7fcae0200640 1 -- start start 2024-04-17T10:04:35.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcae0200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcad80b4de0 0x7fcad8159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:35.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcae0200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcad8144e90 0x7fcad80ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:35.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcae0200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcad815c1b0 0x7fcad80adf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:35.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcae0200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcad806f8b0 con 0x7fcad80b4de0 2024-04-17T10:04:35.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcae0200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcad8072880 con 0x7fcad815c1b0 2024-04-17T10:04:35.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcae0200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcad8166100 con 0x7fcad8144e90 2024-04-17T10:04:35.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcadd800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcad80b4de0 0x7fcad8159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:35.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcadd800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcad80b4de0 0x7fcad8159790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59770/0 (socket says 172.21.15.32:59770) 2024-04-17T10:04:35.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcadd800640 1 -- 172.21.15.32:0/242684982 learned_addr learned my addr 172.21.15.32:0/242684982 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:35.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcade200640 1 --2- 172.21.15.32:0/242684982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcad815c1b0 0x7fcad80adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:35.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcadce00640 1 --2- 172.21.15.32:0/242684982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcad8144e90 0x7fcad80ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:35.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcadd800640 1 -- 172.21.15.32:0/242684982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcad8144e90 msgr2=0x7fcad80ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:35.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcadd800640 1 --2- 172.21.15.32:0/242684982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcad8144e90 0x7fcad80ad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcadd800640 1 -- 172.21.15.32:0/242684982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcad815c1b0 msgr2=0x7fcad80adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:35.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcadd800640 1 --2- 172.21.15.32:0/242684982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcad815c1b0 0x7fcad80adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:35.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcadd800640 1 -- 172.21.15.32:0/242684982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad80cda90 con 0x7fcad80b4de0 2024-04-17T10:04:35.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcadce00640 1 --2- 172.21.15.32:0/242684982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcad8144e90 0x7fcad80ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:35.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcade200640 1 --2- 172.21.15.32:0/242684982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcad815c1b0 0x7fcad80adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:35.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcadd800640 1 --2- 172.21.15.32:0/242684982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcad80b4de0 0x7fcad8159790 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00d830 tx=0x7fcacc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:35.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcad5600640 1 -- 172.21.15.32:0/242684982 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc00b6c0 con 0x7fcad80b4de0 2024-04-17T10:04:35.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcad5600640 1 -- 172.21.15.32:0/242684982 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcacc019070 con 0x7fcad80b4de0 2024-04-17T10:04:35.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcad5600640 1 -- 172.21.15.32:0/242684982 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc00ba60 con 0x7fcad80b4de0 2024-04-17T10:04:35.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcae0200640 1 -- 172.21.15.32:0/242684982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad815a9b0 con 0x7fcad80b4de0 2024-04-17T10:04:35.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcae0200640 1 -- 172.21.15.32:0/242684982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad815af70 con 0x7fcad80b4de0 2024-04-17T10:04:35.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcae0200640 1 -- 172.21.15.32:0/242684982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad8068a90 con 0x7fcad80b4de0 2024-04-17T10:04:35.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.878+0000 7fcad5600640 1 -- 172.21.15.32:0/242684982 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcacc01e460 con 0x7fcad80b4de0 2024-04-17T10:04:35.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.882+0000 7fcad5600640 1 --2- 172.21.15.32:0/242684982 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcab8068380 0x7fcab806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:35.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.882+0000 7fcad5600640 1 -- 172.21.15.32:0/242684982 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7fcacc0870e0 con 0x7fcad80b4de0 2024-04-17T10:04:35.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.882+0000 7fcadce00640 1 --2- 172.21.15.32:0/242684982 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcab8068380 0x7fcab806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:35.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.882+0000 7fcadce00640 1 --2- 172.21.15.32:0/242684982 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcab8068380 0x7fcab806a840 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fcad81fab50 tx=0x7fcac4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:35.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:35.886+0000 7fcad5600640 1 -- 172.21.15.32:0/242684982 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fcacc057640 con 0x7fcad80b4de0 2024-04-17T10:04:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.082+0000 7fcae0200640 1 -- 172.21.15.32:0/242684982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcad80afaf0 con 0x7fcad80b4de0 2024-04-17T10:04:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.086+0000 7fcad5600640 1 -- 172.21.15.32:0/242684982 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5212 (secure 0 0 0) 0x7fcacc056f40 con 0x7fcad80b4de0 2024-04-17T10:04:36.080 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:36.080 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7586,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3601627312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3601627312},{"type":"v1","addr":"172.21.15.144:6834","nonce":3601627312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2382359917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2382359917},{"type":"v1","addr":"172.21.15.144:6835","nonce":2382359917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1857552160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1857552160},{"type":"v1","addr":"172.21.15.32:6835","nonce":1857552160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:03:37.619785+0000","modified":"2024-04-17T10:04:27.457010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9831},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9831":{"gid":9831,"name":"a","rank":0,"incarnation":108,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6837/1204910660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1204910660},{"type":"v1","addr":"172.21.15.32:6837","nonce":1204910660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9831,"qdb_cluster":[9831]},"id":8}]} 2024-04-17T10:04:36.080 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 111 2024-04-17T10:04:36.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.086+0000 7fcae0200640 1 -- 172.21.15.32:0/242684982 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcab8068380 msgr2=0x7fcab806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:36.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.086+0000 7fcae0200640 1 --2- 172.21.15.32:0/242684982 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcab8068380 0x7fcab806a840 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fcad81fab50 tx=0x7fcac4009040 comp rx=0 tx=0).stop 2024-04-17T10:04:36.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.086+0000 7fcae0200640 1 -- 172.21.15.32:0/242684982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcad80b4de0 msgr2=0x7fcad8159790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:36.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.086+0000 7fcae0200640 1 --2- 172.21.15.32:0/242684982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcad80b4de0 0x7fcad8159790 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00d830 tx=0x7fcacc00dcf0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.086+0000 7fcae0200640 1 -- 172.21.15.32:0/242684982 shutdown_connections 2024-04-17T10:04:36.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.086+0000 7fcae0200640 1 --2- 172.21.15.32:0/242684982 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fcab8068380 0x7fcab806a840 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.086+0000 7fcae0200640 1 --2- 172.21.15.32:0/242684982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcad8144e90 0x7fcad80ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.086+0000 7fcae0200640 1 --2- 172.21.15.32:0/242684982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcad815c1b0 0x7fcad80adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.086+0000 7fcae0200640 1 --2- 172.21.15.32:0/242684982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcad80b4de0 0x7fcad8159790 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.086+0000 7fcae0200640 1 -- 172.21.15.32:0/242684982 >> 172.21.15.32:0/242684982 conn(0x7fcad806ba30 msgr2=0x7fcad80ccee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:36.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.086+0000 7fcae0200640 1 -- 172.21.15.32:0/242684982 shutdown_connections 2024-04-17T10:04:36.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.086+0000 7fcae0200640 1 -- 172.21.15.32:0/242684982 wait complete. 2024-04-17T10:04:36.101 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:04:36.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.246+0000 7f00b3800640 1 Processor -- start 2024-04-17T10:04:36.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.246+0000 7f00b3800640 1 -- start start 2024-04-17T10:04:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.250+0000 7f00b3800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00ac145f30 0x7f00ac071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.250+0000 7f00b3800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f00ac15d8d0 0x7f00ac072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.250+0000 7f00b3800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00ac0cdac0 con 0x7f00ac0cce70 2024-04-17T10:04:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.250+0000 7f00b3800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f00ac06f400 con 0x7f00ac15d8d0 2024-04-17T10:04:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.250+0000 7f00b3800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f00ac146310 con 0x7f00ac145f30 2024-04-17T10:04:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.250+0000 7f00b1800640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f00ac0cce70 0x7f00ac0cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44358/0 (socket says 172.21.15.32:44358) 2024-04-17T10:04:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.250+0000 7f00b1800640 1 -- 172.21.15.32:0/2552003011 learned_addr learned my addr 172.21.15.32:0/2552003011 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.250+0000 7f00a3e00640 1 --2- 172.21.15.32:0/2552003011 >> v2:172.21.15.144:3300/0 conn(0x7f00ac15d8d0 0x7f00ac072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.250+0000 7f00b0e00640 1 --2- 172.21.15.32:0/2552003011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00ac145f30 0x7f00ac071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.250+0000 7f00abe00640 1 -- 172.21.15.32:0/2552003011 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2743695031 0 0) 0x7f00ac0cdac0 con 0x7f00ac0cce70 2024-04-17T10:04:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.250+0000 7f00abe00640 1 -- 172.21.15.32:0/2552003011 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0090003530 con 0x7f00ac0cce70 2024-04-17T10:04:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.250+0000 7f00b0e00640 1 -- 172.21.15.32:0/2552003011 >> v2:172.21.15.144:3300/0 conn(0x7f00ac15d8d0 msgr2=0x7f00ac072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.250+0000 7f00b0e00640 1 --2- 172.21.15.32:0/2552003011 >> v2:172.21.15.144:3300/0 conn(0x7f00ac15d8d0 0x7f00ac072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.250+0000 7f00b0e00640 1 -- 172.21.15.32:0/2552003011 >> v1:172.21.15.32:6789/0 conn(0x7f00ac0cce70 legacy=0x7f00ac0cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:36.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.250+0000 7f00b0e00640 1 -- 172.21.15.32:0/2552003011 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ac15cdf0 con 0x7f00ac145f30 2024-04-17T10:04:36.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.250+0000 7f00a3e00640 1 --2- 172.21.15.32:0/2552003011 >> v2:172.21.15.144:3300/0 conn(0x7f00ac15d8d0 0x7f00ac072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:04:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.254+0000 7f00b0e00640 1 --2- 172.21.15.32:0/2552003011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00ac145f30 0x7f00ac071df0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f009c0097c0 tx=0x7f009c01ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b44e17f414ee1fc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:36.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.254+0000 7f00abe00640 1 -- 172.21.15.32:0/2552003011 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c0b5070 con 0x7f00ac145f30 2024-04-17T10:04:36.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.254+0000 7f00abe00640 1 -- 172.21.15.32:0/2552003011 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f009c0a8990 con 0x7f00ac145f30 2024-04-17T10:04:36.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.254+0000 7f00abe00640 1 -- 172.21.15.32:0/2552003011 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c0a8d40 con 0x7f00ac145f30 2024-04-17T10:04:36.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.254+0000 7f00b3800640 1 -- 172.21.15.32:0/2552003011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00ac145f30 msgr2=0x7f00ac071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:36.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.254+0000 7f00b3800640 1 --2- 172.21.15.32:0/2552003011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00ac145f30 0x7f00ac071df0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f009c0097c0 tx=0x7f009c01ec90 comp rx=0 tx=0).stop 2024-04-17T10:04:36.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.254+0000 7f00b3800640 1 -- 172.21.15.32:0/2552003011 shutdown_connections 2024-04-17T10:04:36.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.254+0000 7f00b3800640 1 --2- 172.21.15.32:0/2552003011 >> v2:172.21.15.144:3300/0 conn(0x7f00ac15d8d0 0x7f00ac072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.254+0000 7f00b3800640 1 --2- 172.21.15.32:0/2552003011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00ac145f30 0x7f00ac071df0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.254+0000 7f00b3800640 1 -- 172.21.15.32:0/2552003011 >> 172.21.15.32:0/2552003011 conn(0x7f00ac06ba30 msgr2=0x7f00ac06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:36.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.254+0000 7f00b3800640 1 -- 172.21.15.32:0/2552003011 shutdown_connections 2024-04-17T10:04:36.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.254+0000 7f00b3800640 1 -- 172.21.15.32:0/2552003011 wait complete. 2024-04-17T10:04:36.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.254+0000 7f00b3800640 1 Processor -- start 2024-04-17T10:04:36.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.254+0000 7f00b3800640 1 -- start start 2024-04-17T10:04:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.254+0000 7f00b3800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00ac0cce70 0x7f00ac1f70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.254+0000 7f00b3800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00ac145f30 0x7f00ac1e6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.254+0000 7f00b3800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00ac15d8d0 0x7f00ac1e67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.254+0000 7f00b3800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac15cb90 con 0x7f00ac145f30 2024-04-17T10:04:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.254+0000 7f00b3800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac15d440 con 0x7f00ac0cce70 2024-04-17T10:04:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.254+0000 7f00b3800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f00ac1601d0 con 0x7f00ac15d8d0 2024-04-17T10:04:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.258+0000 7f00a3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00ac145f30 0x7f00ac1e6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.258+0000 7f00a3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00ac145f30 0x7f00ac1e6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59782/0 (socket says 172.21.15.32:59782) 2024-04-17T10:04:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.258+0000 7f00a3e00640 1 -- 172.21.15.32:0/75741960 learned_addr learned my addr 172.21.15.32:0/75741960 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.258+0000 7f00b0e00640 1 --2- 172.21.15.32:0/75741960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00ac0cce70 0x7f00ac1f70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.258+0000 7f00b1800640 1 --2- 172.21.15.32:0/75741960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00ac15d8d0 0x7f00ac1e67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.258+0000 7f00a3e00640 1 -- 172.21.15.32:0/75741960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00ac15d8d0 msgr2=0x7f00ac1e67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.258+0000 7f00a3e00640 1 --2- 172.21.15.32:0/75741960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00ac15d8d0 0x7f00ac1e67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.258+0000 7f00a3e00640 1 -- 172.21.15.32:0/75741960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00ac0cce70 msgr2=0x7f00ac1f70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.258+0000 7f00a3e00640 1 --2- 172.21.15.32:0/75741960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00ac0cce70 0x7f00ac1f70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.258+0000 7f00a3e00640 1 -- 172.21.15.32:0/75741960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ac1f4480 con 0x7f00ac145f30 2024-04-17T10:04:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.258+0000 7f00b1800640 1 --2- 172.21.15.32:0/75741960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00ac15d8d0 0x7f00ac1e67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.258+0000 7f00a3e00640 1 --2- 172.21.15.32:0/75741960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00ac145f30 0x7f00ac1e6290 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f009400e980 tx=0x7f009400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.258+0000 7f00a9600640 1 -- 172.21.15.32:0/75741960 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094017070 con 0x7f00ac145f30 2024-04-17T10:04:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.258+0000 7f00a9600640 1 -- 172.21.15.32:0/75741960 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f009400ca00 con 0x7f00ac145f30 2024-04-17T10:04:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.258+0000 7f00b3800640 1 -- 172.21.15.32:0/75741960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00ac1f4760 con 0x7f00ac145f30 2024-04-17T10:04:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.258+0000 7f00a9600640 1 -- 172.21.15.32:0/75741960 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009400cdd0 con 0x7f00ac145f30 2024-04-17T10:04:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.258+0000 7f00b3800640 1 -- 172.21.15.32:0/75741960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00ac1f4d20 con 0x7f00ac145f30 2024-04-17T10:04:36.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.258+0000 7f00b3800640 1 -- 172.21.15.32:0/75741960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00ac068a90 con 0x7f00ac145f30 2024-04-17T10:04:36.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.258+0000 7f00a9600640 1 -- 172.21.15.32:0/75741960 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f009401a070 con 0x7f00ac145f30 2024-04-17T10:04:36.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.258+0000 7f00a9600640 1 --2- 172.21.15.32:0/75741960 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0088068380 0x7f008806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:36.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.262+0000 7f00a9600640 1 -- 172.21.15.32:0/75741960 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f0094086c30 con 0x7f00ac145f30 2024-04-17T10:04:36.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.262+0000 7f00b0e00640 1 --2- 172.21.15.32:0/75741960 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0088068380 0x7f008806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:36.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.262+0000 7f00b0e00640 1 --2- 172.21.15.32:0/75741960 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0088068380 0x7f008806a840 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f009c0b2000 tx=0x7f009c0aa070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:36.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.266+0000 7f00a9600640 1 -- 172.21.15.32:0/75741960 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f0094056b40 con 0x7f00ac145f30 2024-04-17T10:04:36.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.434+0000 7f00b3800640 1 -- 172.21.15.32:0/75741960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00ac072f90 con 0x7f00ac145f30 2024-04-17T10:04:36.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.438+0000 7f00a9600640 1 -- 172.21.15.32:0/75741960 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16991 (secure 0 0 0) 0x7f009405ae40 con 0x7f00ac145f30 2024-04-17T10:04:36.432 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:36.432 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":113,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:04:22.430002+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:03:36.021322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-17T10:03:36.581134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.5","mappings":[{"from":0,"to":5}]},{"pgid":"17.25","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6837/578643390":"2024-04-18T10:04:22.429980+0000","172.21.15.32:6836/578643390":"2024-04-18T10:04:22.429980+0000","172.21.15.32:6837/901447746":"2024-04-18T10:04:03.080821+0000","172.21.15.32:6836/901447746":"2024-04-18T10:04:03.080821+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:04:36.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.438+0000 7f00b3800640 1 -- 172.21.15.32:0/75741960 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0088068380 msgr2=0x7f008806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:36.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.438+0000 7f00b3800640 1 --2- 172.21.15.32:0/75741960 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0088068380 0x7f008806a840 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f009c0b2000 tx=0x7f009c0aa070 comp rx=0 tx=0).stop 2024-04-17T10:04:36.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.438+0000 7f00b3800640 1 -- 172.21.15.32:0/75741960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00ac145f30 msgr2=0x7f00ac1e6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:36.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.438+0000 7f00b3800640 1 --2- 172.21.15.32:0/75741960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00ac145f30 0x7f00ac1e6290 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f009400e980 tx=0x7f009400ee40 comp rx=0 tx=0).stop 2024-04-17T10:04:36.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.438+0000 7f00b3800640 1 -- 172.21.15.32:0/75741960 shutdown_connections 2024-04-17T10:04:36.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.438+0000 7f00b3800640 1 --2- 172.21.15.32:0/75741960 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f0088068380 0x7f008806a840 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.438+0000 7f00b3800640 1 --2- 172.21.15.32:0/75741960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f00ac15d8d0 0x7f00ac1e67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.438+0000 7f00b3800640 1 --2- 172.21.15.32:0/75741960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00ac145f30 0x7f00ac1e6290 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.438+0000 7f00b3800640 1 --2- 172.21.15.32:0/75741960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00ac0cce70 0x7f00ac1f70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.438+0000 7f00b3800640 1 -- 172.21.15.32:0/75741960 >> 172.21.15.32:0/75741960 conn(0x7f00ac06ba30 msgr2=0x7f00ac160560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:36.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.438+0000 7f00b3800640 1 -- 172.21.15.32:0/75741960 shutdown_connections 2024-04-17T10:04:36.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.438+0000 7f00b3800640 1 -- 172.21.15.32:0/75741960 wait complete. 2024-04-17T10:04:36.454 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T10:04:36.454 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:04:36.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.598+0000 7f83b6800640 1 Processor -- start 2024-04-17T10:04:36.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.598+0000 7f83b6800640 1 -- start start 2024-04-17T10:04:36.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.602+0000 7f83b6800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83b0144e90 0x7f83b00b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:36.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.602+0000 7f83b6800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83b0072bb0 con 0x7f83b015c1b0 2024-04-17T10:04:36.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.602+0000 7f83b6800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83b00b3dc0 con 0x7f83b00b4de0 2024-04-17T10:04:36.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.602+0000 7f83b6800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f83b0145270 con 0x7f83b0144e90 2024-04-17T10:04:36.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.602+0000 7f83af400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83b0144e90 0x7f83b00b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:36.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.602+0000 7f83afe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f83b015c1b0 0x7f83b006ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44374/0 (socket says 172.21.15.32:44374) 2024-04-17T10:04:36.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.602+0000 7f83afe00640 1 -- 172.21.15.32:0/1218719732 learned_addr learned my addr 172.21.15.32:0/1218719732 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:36.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.602+0000 7f83af400640 1 -- 172.21.15.32:0/1218719732 >> v1:172.21.15.144:6789/0 conn(0x7f83b00b4de0 legacy=0x7f83b015baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:36.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.602+0000 7f83af400640 1 -- 172.21.15.32:0/1218719732 >> v1:172.21.15.32:6789/0 conn(0x7f83b015c1b0 legacy=0x7f83b006ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:36.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.602+0000 7f83af400640 1 -- 172.21.15.32:0/1218719732 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b00b40a0 con 0x7f83b0144e90 2024-04-17T10:04:36.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.606+0000 7f83af400640 1 --2- 172.21.15.32:0/1218719732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b0144e90 0x7f83b00b48a0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f83a00097c0 tx=0x7f83a001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edae646571ccbbc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:36.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.606+0000 7f83aea00640 1 -- 172.21.15.32:0/1218719732 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a00aa5f0 con 0x7f83b0144e90 2024-04-17T10:04:36.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.606+0000 7f83aea00640 1 -- 172.21.15.32:0/1218719732 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a00af070 con 0x7f83b0144e90 2024-04-17T10:04:36.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.606+0000 7f83aea00640 1 -- 172.21.15.32:0/1218719732 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a00aa9b0 con 0x7f83b0144e90 2024-04-17T10:04:36.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.606+0000 7f83b6800640 1 -- 172.21.15.32:0/1218719732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b0144e90 msgr2=0x7f83b00b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:36.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.606+0000 7f83b6800640 1 --2- 172.21.15.32:0/1218719732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b0144e90 0x7f83b00b48a0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f83a00097c0 tx=0x7f83a001c470 comp rx=0 tx=0).stop 2024-04-17T10:04:36.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.606+0000 7f83b6800640 1 -- 172.21.15.32:0/1218719732 shutdown_connections 2024-04-17T10:04:36.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.606+0000 7f83b6800640 1 --2- 172.21.15.32:0/1218719732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b0144e90 0x7f83b00b48a0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.606+0000 7f83b6800640 1 -- 172.21.15.32:0/1218719732 >> 172.21.15.32:0/1218719732 conn(0x7f83b006ba30 msgr2=0x7f83b006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:36.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.606+0000 7f83b6800640 1 -- 172.21.15.32:0/1218719732 shutdown_connections 2024-04-17T10:04:36.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.606+0000 7f83b6800640 1 -- 172.21.15.32:0/1218719732 wait complete. 2024-04-17T10:04:36.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.606+0000 7f83b6800640 1 Processor -- start 2024-04-17T10:04:36.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.606+0000 7f83b6800640 1 -- start start 2024-04-17T10:04:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.606+0000 7f83b6800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b00b4de0 0x7f83b006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.606+0000 7f83b6800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b0144e90 0x7f83b0147860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.606+0000 7f83b6800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b015c1b0 0x7f83b0147da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.606+0000 7f83b6800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f83b0169120 con 0x7f83b00b4de0 2024-04-17T10:04:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.606+0000 7f83b6800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f83b0072880 con 0x7f83b015c1b0 2024-04-17T10:04:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.606+0000 7f83b6800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f83b01692a0 con 0x7f83b0144e90 2024-04-17T10:04:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.610+0000 7f83af400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b00b4de0 0x7f83b006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.610+0000 7f83af400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b00b4de0 0x7f83b006e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59806/0 (socket says 172.21.15.32:59806) 2024-04-17T10:04:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.610+0000 7f83af400640 1 -- 172.21.15.32:0/445068832 learned_addr learned my addr 172.21.15.32:0/445068832 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.610+0000 7f83a6a00640 1 --2- 172.21.15.32:0/445068832 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b0144e90 0x7f83b0147860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.610+0000 7f83afe00640 1 --2- 172.21.15.32:0/445068832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b015c1b0 0x7f83b0147da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:36.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.610+0000 7f83af400640 1 -- 172.21.15.32:0/445068832 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b0144e90 msgr2=0x7f83b0147860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:36.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.610+0000 7f83af400640 1 --2- 172.21.15.32:0/445068832 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b0144e90 0x7f83b0147860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.610+0000 7f83af400640 1 -- 172.21.15.32:0/445068832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b015c1b0 msgr2=0x7f83b0147da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:36.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.610+0000 7f83af400640 1 --2- 172.21.15.32:0/445068832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b015c1b0 0x7f83b0147da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.610+0000 7f83af400640 1 -- 172.21.15.32:0/445068832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b0148440 con 0x7f83b00b4de0 2024-04-17T10:04:36.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.610+0000 7f83af400640 1 --2- 172.21.15.32:0/445068832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b00b4de0 0x7f83b006e0b0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f83a0019080 tx=0x7f83a001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:36.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.610+0000 7f83a7e00640 1 -- 172.21.15.32:0/445068832 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a00be410 con 0x7f83b00b4de0 2024-04-17T10:04:36.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.610+0000 7f83a7e00640 1 -- 172.21.15.32:0/445068832 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a00be5d0 con 0x7f83b00b4de0 2024-04-17T10:04:36.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.610+0000 7f83a7e00640 1 -- 172.21.15.32:0/445068832 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a00be9c0 con 0x7f83b00b4de0 2024-04-17T10:04:36.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.610+0000 7f83b6800640 1 -- 172.21.15.32:0/445068832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b0148720 con 0x7f83b00b4de0 2024-04-17T10:04:36.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.610+0000 7f83b6800640 1 -- 172.21.15.32:0/445068832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b0148ce0 con 0x7f83b00b4de0 2024-04-17T10:04:36.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.610+0000 7f83b6800640 1 -- 172.21.15.32:0/445068832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83b0068a90 con 0x7f83b00b4de0 2024-04-17T10:04:36.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.610+0000 7f83a7e00640 1 -- 172.21.15.32:0/445068832 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f83a00bc020 con 0x7f83b00b4de0 2024-04-17T10:04:36.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.610+0000 7f83a7e00640 1 --2- 172.21.15.32:0/445068832 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f838c068380 0x7f838c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:36.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.614+0000 7f83a7e00640 1 -- 172.21.15.32:0/445068832 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f83a0120910 con 0x7f83b00b4de0 2024-04-17T10:04:36.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.614+0000 7f83a6a00640 1 --2- 172.21.15.32:0/445068832 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f838c068380 0x7f838c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:36.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.614+0000 7f83a6a00640 1 --2- 172.21.15.32:0/445068832 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f838c068380 0x7f838c06a840 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f83b01fa8a0 tx=0x7f8398009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:36.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.614+0000 7f83a7e00640 1 -- 172.21.15.32:0/445068832 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f83a00c5070 con 0x7f83b00b4de0 2024-04-17T10:04:36.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.814+0000 7f83b6800640 1 -- 172.21.15.32:0/445068832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83b0141830 con 0x7f83b00b4de0 2024-04-17T10:04:36.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.814+0000 7f83a7e00640 1 -- 172.21.15.32:0/445068832 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5212 (secure 0 0 0) 0x7f83a0016090 con 0x7f83b00b4de0 2024-04-17T10:04:36.811 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:36.811 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7586,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3601627312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3601627312},{"type":"v1","addr":"172.21.15.144:6834","nonce":3601627312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2382359917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2382359917},{"type":"v1","addr":"172.21.15.144:6835","nonce":2382359917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1857552160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1857552160},{"type":"v1","addr":"172.21.15.32:6835","nonce":1857552160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:03:37.619785+0000","modified":"2024-04-17T10:04:27.457010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9831},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9831":{"gid":9831,"name":"a","rank":0,"incarnation":108,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6837/1204910660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1204910660},{"type":"v1","addr":"172.21.15.32:6837","nonce":1204910660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9831,"qdb_cluster":[9831]},"id":8}]} 2024-04-17T10:04:36.811 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 111 2024-04-17T10:04:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.818+0000 7f83b6800640 1 -- 172.21.15.32:0/445068832 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f838c068380 msgr2=0x7f838c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.818+0000 7f83b6800640 1 --2- 172.21.15.32:0/445068832 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f838c068380 0x7f838c06a840 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f83b01fa8a0 tx=0x7f8398009040 comp rx=0 tx=0).stop 2024-04-17T10:04:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.818+0000 7f83b6800640 1 -- 172.21.15.32:0/445068832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b00b4de0 msgr2=0x7f83b006e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.818+0000 7f83b6800640 1 --2- 172.21.15.32:0/445068832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b00b4de0 0x7f83b006e0b0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f83a0019080 tx=0x7f83a001acb0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.818+0000 7f83b6800640 1 -- 172.21.15.32:0/445068832 shutdown_connections 2024-04-17T10:04:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.818+0000 7f83b6800640 1 --2- 172.21.15.32:0/445068832 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f838c068380 0x7f838c06a840 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.818+0000 7f83b6800640 1 --2- 172.21.15.32:0/445068832 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b0144e90 0x7f83b0147860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.818+0000 7f83b6800640 1 --2- 172.21.15.32:0/445068832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b015c1b0 0x7f83b0147da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.818+0000 7f83b6800640 1 --2- 172.21.15.32:0/445068832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b00b4de0 0x7f83b006e0b0 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.818+0000 7f83b6800640 1 -- 172.21.15.32:0/445068832 >> 172.21.15.32:0/445068832 conn(0x7f83b006ba30 msgr2=0x7f83b00cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.818+0000 7f83b6800640 1 -- 172.21.15.32:0/445068832 shutdown_connections 2024-04-17T10:04:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.818+0000 7f83b6800640 1 -- 172.21.15.32:0/445068832 wait complete. 2024-04-17T10:04:36.832 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:04:36.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.978+0000 7fa9a9800640 1 Processor -- start 2024-04-17T10:04:36.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.978+0000 7fa9a9800640 1 -- start start 2024-04-17T10:04:36.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.978+0000 7fa9a9800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9a4144e90 0x7fa9a40b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:36.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.978+0000 7fa9a9800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9a4072bb0 con 0x7fa9a40b4de0 2024-04-17T10:04:36.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.978+0000 7fa9a9800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9a40b3dc0 con 0x7fa9a415c1b0 2024-04-17T10:04:36.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.978+0000 7fa9a9800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a4145270 con 0x7fa9a4144e90 2024-04-17T10:04:36.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.978+0000 7fa99a000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa9a40b4de0 0x7fa9a415baa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44378/0 (socket says 172.21.15.32:44378) 2024-04-17T10:04:36.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.978+0000 7fa99a000640 1 -- 172.21.15.32:0/868561677 learned_addr learned my addr 172.21.15.32:0/868561677 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:36.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.978+0000 7fa9a2a00640 1 --2- 172.21.15.32:0/868561677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9a4144e90 0x7fa9a40b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:36.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.978+0000 7fa9a2000640 1 -- 172.21.15.32:0/868561677 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4276256981 0 0) 0x7fa9a4072bb0 con 0x7fa9a40b4de0 2024-04-17T10:04:36.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.978+0000 7fa9a2000640 1 -- 172.21.15.32:0/868561677 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa988003530 con 0x7fa9a40b4de0 2024-04-17T10:04:36.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.978+0000 7fa9a2000640 1 -- 172.21.15.32:0/868561677 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 451934484 0 0) 0x7fa9a40b3dc0 con 0x7fa9a415c1b0 2024-04-17T10:04:36.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.978+0000 7fa9a2a00640 1 -- 172.21.15.32:0/868561677 >> v1:172.21.15.144:6789/0 conn(0x7fa9a415c1b0 legacy=0x7fa9a406ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:36.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.978+0000 7fa9a2a00640 1 -- 172.21.15.32:0/868561677 >> v1:172.21.15.32:6789/0 conn(0x7fa9a40b4de0 legacy=0x7fa9a415baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:36.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.978+0000 7fa9a2a00640 1 -- 172.21.15.32:0/868561677 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a40b40a0 con 0x7fa9a4144e90 2024-04-17T10:04:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.982+0000 7fa9a2a00640 1 --2- 172.21.15.32:0/868561677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9a4144e90 0x7fa9a40b48a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa9940097f0 tx=0x7fa99401c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c0da1aa95ab9adc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:36.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.982+0000 7fa9a2000640 1 -- 172.21.15.32:0/868561677 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9940aa430 con 0x7fa9a4144e90 2024-04-17T10:04:36.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.982+0000 7fa9a2000640 1 -- 172.21.15.32:0/868561677 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9940af070 con 0x7fa9a4144e90 2024-04-17T10:04:36.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.982+0000 7fa9a2000640 1 -- 172.21.15.32:0/868561677 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9940aa7f0 con 0x7fa9a4144e90 2024-04-17T10:04:36.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.982+0000 7fa9a9800640 1 -- 172.21.15.32:0/868561677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9a4144e90 msgr2=0x7fa9a40b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:36.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.982+0000 7fa9a9800640 1 --2- 172.21.15.32:0/868561677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9a4144e90 0x7fa9a40b48a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa9940097f0 tx=0x7fa99401c4b0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.982+0000 7fa9a9800640 1 -- 172.21.15.32:0/868561677 shutdown_connections 2024-04-17T10:04:36.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.982+0000 7fa9a9800640 1 --2- 172.21.15.32:0/868561677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9a4144e90 0x7fa9a40b48a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.982+0000 7fa9a9800640 1 -- 172.21.15.32:0/868561677 >> 172.21.15.32:0/868561677 conn(0x7fa9a406ba30 msgr2=0x7fa9a406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:36.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.982+0000 7fa9a9800640 1 -- 172.21.15.32:0/868561677 shutdown_connections 2024-04-17T10:04:36.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.982+0000 7fa9a9800640 1 -- 172.21.15.32:0/868561677 wait complete. 2024-04-17T10:04:36.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa9a9800640 1 Processor -- start 2024-04-17T10:04:36.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa9a9800640 1 -- start start 2024-04-17T10:04:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa9a9800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9a40b4de0 0x7fa9a415a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa9a9800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9a4144e90 0x7fa9a40b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa9a9800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a415c1b0 0x7fa9a40cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa9a9800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a4072670 con 0x7fa9a415c1b0 2024-04-17T10:04:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa9a9800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a406f880 con 0x7fa9a40b4de0 2024-04-17T10:04:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa9a9800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a40727f0 con 0x7fa9a4144e90 2024-04-17T10:04:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa9a3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a415c1b0 0x7fa9a40cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa9a3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a415c1b0 0x7fa9a40cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59812/0 (socket says 172.21.15.32:59812) 2024-04-17T10:04:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa9a3400640 1 -- 172.21.15.32:0/3391025130 learned_addr learned my addr 172.21.15.32:0/3391025130 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:36.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa9a2a00640 1 --2- 172.21.15.32:0/3391025130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9a40b4de0 0x7fa9a415a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa99a000640 1 --2- 172.21.15.32:0/3391025130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9a4144e90 0x7fa9a40b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa9a3400640 1 -- 172.21.15.32:0/3391025130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9a4144e90 msgr2=0x7fa9a40b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa9a3400640 1 --2- 172.21.15.32:0/3391025130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9a4144e90 0x7fa9a40b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa9a3400640 1 -- 172.21.15.32:0/3391025130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9a40b4de0 msgr2=0x7fa9a415a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa9a3400640 1 --2- 172.21.15.32:0/3391025130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9a40b4de0 0x7fa9a415a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa9a3400640 1 -- 172.21.15.32:0/3391025130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a40b1ee0 con 0x7fa9a415c1b0 2024-04-17T10:04:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa9a3400640 1 --2- 172.21.15.32:0/3391025130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a415c1b0 0x7fa9a40cf4d0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00c780 tx=0x7fa99c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa99b400640 1 -- 172.21.15.32:0/3391025130 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c003db0 con 0x7fa9a415c1b0 2024-04-17T10:04:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa99b400640 1 -- 172.21.15.32:0/3391025130 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa99c00ce30 con 0x7fa9a415c1b0 2024-04-17T10:04:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa99b400640 1 -- 172.21.15.32:0/3391025130 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c005290 con 0x7fa9a415c1b0 2024-04-17T10:04:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa9a9800640 1 -- 172.21.15.32:0/3391025130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a41398d0 con 0x7fa9a415c1b0 2024-04-17T10:04:36.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.986+0000 7fa9a9800640 1 -- 172.21.15.32:0/3391025130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a4139e90 con 0x7fa9a415c1b0 2024-04-17T10:04:36.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.990+0000 7fa9a9800640 1 -- 172.21.15.32:0/3391025130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a4068a90 con 0x7fa9a415c1b0 2024-04-17T10:04:36.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.990+0000 7fa99b400640 1 -- 172.21.15.32:0/3391025130 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa99c005430 con 0x7fa9a415c1b0 2024-04-17T10:04:36.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.990+0000 7fa99b400640 1 --2- 172.21.15.32:0/3391025130 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa980068380 0x7fa98006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:36.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.990+0000 7fa99b400640 1 -- 172.21.15.32:0/3391025130 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7fa99c0868d0 con 0x7fa9a415c1b0 2024-04-17T10:04:36.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.990+0000 7fa9a2a00640 1 --2- 172.21.15.32:0/3391025130 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa980068380 0x7fa98006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:36.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.990+0000 7fa9a2a00640 1 --2- 172.21.15.32:0/3391025130 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa980068380 0x7fa98006a840 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fa9940ae000 tx=0x7fa9940a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:36.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:36.994+0000 7fa99b400640 1 -- 172.21.15.32:0/3391025130 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa99c0566b0 con 0x7fa9a415c1b0 2024-04-17T10:04:37.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.190+0000 7fa9a9800640 1 -- 172.21.15.32:0/3391025130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9a40af880 con 0x7fa9a415c1b0 2024-04-17T10:04:37.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.194+0000 7fa99b400640 1 -- 172.21.15.32:0/3391025130 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5212 (secure 0 0 0) 0x7fa99c05a9b0 con 0x7fa9a415c1b0 2024-04-17T10:04:37.188 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:37.188 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7586,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3601627312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3601627312},{"type":"v1","addr":"172.21.15.144:6834","nonce":3601627312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2382359917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2382359917},{"type":"v1","addr":"172.21.15.144:6835","nonce":2382359917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1857552160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1857552160},{"type":"v1","addr":"172.21.15.32:6835","nonce":1857552160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T10:03:37.619785+0000","modified":"2024-04-17T10:04:27.457010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9831},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9831":{"gid":9831,"name":"a","rank":0,"incarnation":108,"state":"up:active","state_seq":4,"addr":"172.21.15.32:6837/1204910660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1204910660},{"type":"v1","addr":"172.21.15.32:6837","nonce":1204910660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9831,"qdb_cluster":[9831]},"id":8}]} 2024-04-17T10:04:37.188 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 111 2024-04-17T10:04:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.194+0000 7fa9a9800640 1 -- 172.21.15.32:0/3391025130 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa980068380 msgr2=0x7fa98006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.194+0000 7fa9a9800640 1 --2- 172.21.15.32:0/3391025130 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa980068380 0x7fa98006a840 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fa9940ae000 tx=0x7fa9940a7000 comp rx=0 tx=0).stop 2024-04-17T10:04:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.194+0000 7fa9a9800640 1 -- 172.21.15.32:0/3391025130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a415c1b0 msgr2=0x7fa9a40cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.194+0000 7fa9a9800640 1 --2- 172.21.15.32:0/3391025130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a415c1b0 0x7fa9a40cf4d0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00c780 tx=0x7fa99c00cc40 comp rx=0 tx=0).stop 2024-04-17T10:04:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.194+0000 7fa9a9800640 1 -- 172.21.15.32:0/3391025130 shutdown_connections 2024-04-17T10:04:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.194+0000 7fa9a9800640 1 --2- 172.21.15.32:0/3391025130 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa980068380 0x7fa98006a840 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.194+0000 7fa9a9800640 1 --2- 172.21.15.32:0/3391025130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9a4144e90 0x7fa9a40b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:37.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.194+0000 7fa9a9800640 1 --2- 172.21.15.32:0/3391025130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a415c1b0 0x7fa9a40cf4d0 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:37.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.194+0000 7fa9a9800640 1 --2- 172.21.15.32:0/3391025130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9a40b4de0 0x7fa9a415a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:37.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.194+0000 7fa9a9800640 1 -- 172.21.15.32:0/3391025130 >> 172.21.15.32:0/3391025130 conn(0x7fa9a406ba30 msgr2=0x7fa9a406be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:37.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.194+0000 7fa9a9800640 1 -- 172.21.15.32:0/3391025130 shutdown_connections 2024-04-17T10:04:37.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.194+0000 7fa9a9800640 1 -- 172.21.15.32:0/3391025130 wait complete. 2024-04-17T10:04:37.210 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:04:37.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.354+0000 7f40ae200640 1 Processor -- start 2024-04-17T10:04:37.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.354+0000 7f40ae200640 1 -- start start 2024-04-17T10:04:37.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.354+0000 7f40ae200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40a815baa0 0x7f40a815bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:37.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.354+0000 7f40ae200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f40a806ef60 0x7f40a806f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:37.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.354+0000 7f40ae200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40a8072bb0 con 0x7f40a8144e90 2024-04-17T10:04:37.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.354+0000 7f40ae200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f40a806f8f0 con 0x7f40a815baa0 2024-04-17T10:04:37.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.354+0000 7f40ae200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f40a8145270 con 0x7f40a806ef60 2024-04-17T10:04:37.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.358+0000 7f409ea00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40a815baa0 0x7f40a815bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:37.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.358+0000 7f40a7400640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f40a8144e90 0x7f40a80b48a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:42794/0 (socket says 172.21.15.32:42794) 2024-04-17T10:04:37.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.358+0000 7f40a7400640 1 -- 172.21.15.32:0/2345667620 learned_addr learned my addr 172.21.15.32:0/2345667620 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:37.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.358+0000 7f40a7e00640 1 --2- 172.21.15.32:0/2345667620 >> v2:172.21.15.144:3300/0 conn(0x7f40a806ef60 0x7f40a806f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:37.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.358+0000 7f409ea00640 1 -- 172.21.15.32:0/2345667620 >> v2:172.21.15.144:3300/0 conn(0x7f40a806ef60 msgr2=0x7f40a806f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:37.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.358+0000 7f409ea00640 1 --2- 172.21.15.32:0/2345667620 >> v2:172.21.15.144:3300/0 conn(0x7f40a806ef60 0x7f40a806f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:37.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.358+0000 7f409ea00640 1 -- 172.21.15.32:0/2345667620 >> v1:172.21.15.144:6789/0 conn(0x7f40a8144e90 legacy=0x7f40a80b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:37.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.358+0000 7f409ea00640 1 -- 172.21.15.32:0/2345667620 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40a80b3dc0 con 0x7f40a815baa0 2024-04-17T10:04:37.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.358+0000 7f40a6a00640 1 -- 172.21.15.32:0/2345667620 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 379076161 0 0) 0x7f40a8072bb0 con 0x7f40a8144e90 2024-04-17T10:04:37.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.358+0000 7f409ea00640 1 --2- 172.21.15.32:0/2345667620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40a815baa0 0x7f40a815bf30 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f40a80b3a00 tx=0x7f409001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f698ecbd895d365c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:37.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.358+0000 7f40a6a00640 1 -- 172.21.15.32:0/2345667620 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40900b0070 con 0x7f40a815baa0 2024-04-17T10:04:37.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.358+0000 7f40a6a00640 1 -- 172.21.15.32:0/2345667620 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4090004230 con 0x7f40a815baa0 2024-04-17T10:04:37.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.358+0000 7f40a6a00640 1 -- 172.21.15.32:0/2345667620 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4090003620 con 0x7f40a815baa0 2024-04-17T10:04:37.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40ae200640 1 -- 172.21.15.32:0/2345667620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40a815baa0 msgr2=0x7f40a815bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:37.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40ae200640 1 --2- 172.21.15.32:0/2345667620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40a815baa0 0x7f40a815bf30 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f40a80b3a00 tx=0x7f409001c460 comp rx=0 tx=0).stop 2024-04-17T10:04:37.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40ae200640 1 -- 172.21.15.32:0/2345667620 shutdown_connections 2024-04-17T10:04:37.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40ae200640 1 --2- 172.21.15.32:0/2345667620 >> v2:172.21.15.144:3300/0 conn(0x7f40a806ef60 0x7f40a806f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:37.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40ae200640 1 --2- 172.21.15.32:0/2345667620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40a815baa0 0x7f40a815bf30 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:37.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40ae200640 1 -- 172.21.15.32:0/2345667620 >> 172.21.15.32:0/2345667620 conn(0x7f40a806ba30 msgr2=0x7f40a806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:37.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40ae200640 1 -- 172.21.15.32:0/2345667620 shutdown_connections 2024-04-17T10:04:37.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40ae200640 1 -- 172.21.15.32:0/2345667620 wait complete. 2024-04-17T10:04:37.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40ae200640 1 Processor -- start 2024-04-17T10:04:37.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40ae200640 1 -- start start 2024-04-17T10:04:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40ae200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40a806ef60 0x7f40a813cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40ae200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40a8144e90 0x7f40a815a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40ae200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40a815baa0 0x7f40a80cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40ae200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f40a80727f0 con 0x7f40a806ef60 2024-04-17T10:04:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40ae200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f40a8072670 con 0x7f40a815baa0 2024-04-17T10:04:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40ae200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f40a8072970 con 0x7f40a8144e90 2024-04-17T10:04:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40a7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40a806ef60 0x7f40a813cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40a7400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40a806ef60 0x7f40a813cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59832/0 (socket says 172.21.15.32:59832) 2024-04-17T10:04:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40a7400640 1 -- 172.21.15.32:0/989771403 learned_addr learned my addr 172.21.15.32:0/989771403 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40a7e00640 1 --2- 172.21.15.32:0/989771403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40a815baa0 0x7f40a80cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:37.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f409ea00640 1 --2- 172.21.15.32:0/989771403 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40a8144e90 0x7f40a815a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:37.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40a7400640 1 -- 172.21.15.32:0/989771403 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40a8144e90 msgr2=0x7f40a815a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:37.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40a7400640 1 --2- 172.21.15.32:0/989771403 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40a8144e90 0x7f40a815a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:37.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40a7400640 1 -- 172.21.15.32:0/989771403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40a815baa0 msgr2=0x7f40a80cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:37.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40a7400640 1 --2- 172.21.15.32:0/989771403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40a815baa0 0x7f40a80cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:37.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40a7400640 1 -- 172.21.15.32:0/989771403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40a815ad70 con 0x7f40a806ef60 2024-04-17T10:04:37.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40a7e00640 1 --2- 172.21.15.32:0/989771403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40a815baa0 0x7f40a80cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:37.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f409ea00640 1 --2- 172.21.15.32:0/989771403 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40a8144e90 0x7f40a815a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:37.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.362+0000 7f40a7400640 1 --2- 172.21.15.32:0/989771403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40a806ef60 0x7f40a813cd10 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f40980011f0 tx=0x7f409800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:37.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.366+0000 7f409fe00640 1 -- 172.21.15.32:0/989771403 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4098018070 con 0x7f40a806ef60 2024-04-17T10:04:37.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.366+0000 7f409fe00640 1 -- 172.21.15.32:0/989771403 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40980074e0 con 0x7f40a806ef60 2024-04-17T10:04:37.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.366+0000 7f409fe00640 1 -- 172.21.15.32:0/989771403 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4098004a10 con 0x7f40a806ef60 2024-04-17T10:04:37.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.366+0000 7f40ae200640 1 -- 172.21.15.32:0/989771403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40a80c5400 con 0x7f40a806ef60 2024-04-17T10:04:37.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.366+0000 7f40ae200640 1 -- 172.21.15.32:0/989771403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40a80c59c0 con 0x7f40a806ef60 2024-04-17T10:04:37.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.366+0000 7f40ae200640 1 -- 172.21.15.32:0/989771403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40a8068a90 con 0x7f40a806ef60 2024-04-17T10:04:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.366+0000 7f409fe00640 1 -- 172.21.15.32:0/989771403 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f409801b070 con 0x7f40a806ef60 2024-04-17T10:04:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.366+0000 7f409fe00640 1 --2- 172.21.15.32:0/989771403 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40840683d0 0x7f408406a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.366+0000 7f409fe00640 1 -- 172.21.15.32:0/989771403 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f4098085f20 con 0x7f40a806ef60 2024-04-17T10:04:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.366+0000 7f409ea00640 1 --2- 172.21.15.32:0/989771403 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40840683d0 0x7f408406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.366+0000 7f409ea00640 1 --2- 172.21.15.32:0/989771403 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40840683d0 0x7f408406a890 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f40900ad000 tx=0x7f40900a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:37.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.370+0000 7f409fe00640 1 -- 172.21.15.32:0/989771403 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f4098056460 con 0x7f40a806ef60 2024-04-17T10:04:37.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.538+0000 7f40ae200640 1 -- 172.21.15.32:0/989771403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f40a806c870 con 0x7f40a806ef60 2024-04-17T10:04:37.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.542+0000 7f409fe00640 1 -- 172.21.15.32:0/989771403 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16991 (secure 0 0 0) 0x7f4098055d60 con 0x7f40a806ef60 2024-04-17T10:04:37.537 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:37.537 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":113,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:04:22.430002+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-17T10:03:36.021322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-17T10:03:36.581134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.5","mappings":[{"from":0,"to":5}]},{"pgid":"17.25","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6837/578643390":"2024-04-18T10:04:22.429980+0000","172.21.15.32:6836/578643390":"2024-04-18T10:04:22.429980+0000","172.21.15.32:6837/901447746":"2024-04-18T10:04:03.080821+0000","172.21.15.32:6836/901447746":"2024-04-18T10:04:03.080821+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:04:37.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.542+0000 7f40ae200640 1 -- 172.21.15.32:0/989771403 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40840683d0 msgr2=0x7f408406a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:37.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.542+0000 7f40ae200640 1 --2- 172.21.15.32:0/989771403 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40840683d0 0x7f408406a890 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f40900ad000 tx=0x7f40900a6000 comp rx=0 tx=0).stop 2024-04-17T10:04:37.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.542+0000 7f40ae200640 1 -- 172.21.15.32:0/989771403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40a806ef60 msgr2=0x7f40a813cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:37.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.542+0000 7f40ae200640 1 --2- 172.21.15.32:0/989771403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40a806ef60 0x7f40a813cd10 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f40980011f0 tx=0x7f409800c9b0 comp rx=0 tx=0).stop 2024-04-17T10:04:37.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.542+0000 7f40ae200640 1 -- 172.21.15.32:0/989771403 shutdown_connections 2024-04-17T10:04:37.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.542+0000 7f40ae200640 1 --2- 172.21.15.32:0/989771403 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f40840683d0 0x7f408406a890 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:37.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.542+0000 7f40ae200640 1 --2- 172.21.15.32:0/989771403 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40a8144e90 0x7f40a815a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:37.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.542+0000 7f40ae200640 1 --2- 172.21.15.32:0/989771403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40a815baa0 0x7f40a80cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:37.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.542+0000 7f40ae200640 1 --2- 172.21.15.32:0/989771403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40a806ef60 0x7f40a813cd10 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:37.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.542+0000 7f40ae200640 1 -- 172.21.15.32:0/989771403 >> 172.21.15.32:0/989771403 conn(0x7f40a806ba30 msgr2=0x7f40a81434a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:37.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.542+0000 7f40ae200640 1 -- 172.21.15.32:0/989771403 shutdown_connections 2024-04-17T10:04:37.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.542+0000 7f40ae200640 1 -- 172.21.15.32:0/989771403 wait complete. 2024-04-17T10:04:37.558 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T10:04:37.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.702+0000 7f66c9800640 1 Processor -- start 2024-04-17T10:04:37.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.702+0000 7f66c9800640 1 -- start start 2024-04-17T10:04:37.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.706+0000 7f66c9800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f66c4155a70 0x7f66c415f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:37.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.706+0000 7f66c9800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66c415f870 con 0x7f66c4155e50 2024-04-17T10:04:37.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.706+0000 7f66c9800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66c415fb50 con 0x7f66c40b3340 2024-04-17T10:04:37.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.706+0000 7f66c9800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f66c40b42a0 con 0x7f66c4155a70 2024-04-17T10:04:37.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.706+0000 7f66c2000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f66c4155e50 0x7f66c415d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44388/0 (socket says 172.21.15.32:44388) 2024-04-17T10:04:37.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.706+0000 7f66c2000640 1 -- 172.21.15.32:0/1162371024 learned_addr learned my addr 172.21.15.32:0/1162371024 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:37.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.706+0000 7f66c2a00640 1 --2- 172.21.15.32:0/1162371024 >> v2:172.21.15.32:3300/0 conn(0x7f66c4155a70 0x7f66c415f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:37.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.706+0000 7f66c1600640 1 -- 172.21.15.32:0/1162371024 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2498667454 0 0) 0x7f66c415f870 con 0x7f66c4155e50 2024-04-17T10:04:37.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.706+0000 7f66c1600640 1 -- 172.21.15.32:0/1162371024 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66a8003530 con 0x7f66c4155e50 2024-04-17T10:04:37.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.706+0000 7f66c2a00640 1 -- 172.21.15.32:0/1162371024 >> v1:172.21.15.144:6789/0 conn(0x7f66c40b3340 legacy=0x7f66c415c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:37.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.706+0000 7f66c2a00640 1 -- 172.21.15.32:0/1162371024 >> v1:172.21.15.32:6789/0 conn(0x7f66c4155e50 legacy=0x7f66c415d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:37.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.706+0000 7f66c2a00640 1 -- 172.21.15.32:0/1162371024 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c415fe30 con 0x7f66c4155a70 2024-04-17T10:04:37.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.710+0000 7f66c2a00640 1 --2- 172.21.15.32:0/1162371024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66c4155a70 0x7f66c415f190 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f66c40b4420 tx=0x7f66b401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f60ec4aaaef1085 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:37.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.710+0000 7f66c1600640 1 -- 172.21.15.32:0/1162371024 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b40b0070 con 0x7f66c4155a70 2024-04-17T10:04:37.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.710+0000 7f66c1600640 1 -- 172.21.15.32:0/1162371024 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b40ab600 con 0x7f66c4155a70 2024-04-17T10:04:37.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.710+0000 7f66c1600640 1 -- 172.21.15.32:0/1162371024 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b40ab9d0 con 0x7f66c4155a70 2024-04-17T10:04:37.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.710+0000 7f66c9800640 1 -- 172.21.15.32:0/1162371024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66c4155a70 msgr2=0x7f66c415f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:37.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.710+0000 7f66c9800640 1 --2- 172.21.15.32:0/1162371024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66c4155a70 0x7f66c415f190 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f66c40b4420 tx=0x7f66b401c480 comp rx=0 tx=0).stop 2024-04-17T10:04:37.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.710+0000 7f66c9800640 1 -- 172.21.15.32:0/1162371024 shutdown_connections 2024-04-17T10:04:37.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.710+0000 7f66c9800640 1 --2- 172.21.15.32:0/1162371024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66c4155a70 0x7f66c415f190 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:37.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.710+0000 7f66c9800640 1 -- 172.21.15.32:0/1162371024 >> 172.21.15.32:0/1162371024 conn(0x7f66c406ba30 msgr2=0x7f66c406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:37.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.710+0000 7f66c9800640 1 -- 172.21.15.32:0/1162371024 shutdown_connections 2024-04-17T10:04:37.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.710+0000 7f66c9800640 1 -- 172.21.15.32:0/1162371024 wait complete. 2024-04-17T10:04:37.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.710+0000 7f66c9800640 1 Processor -- start 2024-04-17T10:04:37.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.710+0000 7f66c9800640 1 -- start start 2024-04-17T10:04:37.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.710+0000 7f66c9800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66c40b3340 0x7f66c4146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:37.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.710+0000 7f66c9800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66c4155a70 0x7f66c414bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:37.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.710+0000 7f66c9800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c4155e50 0x7f66c41424b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:37.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.710+0000 7f66c9800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f66c40d02f0 con 0x7f66c40b3340 2024-04-17T10:04:37.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.710+0000 7f66c9800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f66c40d0170 con 0x7f66c4155e50 2024-04-17T10:04:37.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.710+0000 7f66c9800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f66c40d0470 con 0x7f66c4155a70 2024-04-17T10:04:37.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.710+0000 7f66c2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66c40b3340 0x7f66c4146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:37.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.710+0000 7f66c2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66c40b3340 0x7f66c4146170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59848/0 (socket says 172.21.15.32:59848) 2024-04-17T10:04:37.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.710+0000 7f66c2a00640 1 -- 172.21.15.32:0/1263751310 learned_addr learned my addr 172.21.15.32:0/1263751310 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:37.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.714+0000 7f66c3400640 1 --2- 172.21.15.32:0/1263751310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c4155e50 0x7f66c41424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:37.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.714+0000 7f66c2000640 1 --2- 172.21.15.32:0/1263751310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66c4155a70 0x7f66c414bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:37.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.714+0000 7f66c2a00640 1 -- 172.21.15.32:0/1263751310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66c4155a70 msgr2=0x7f66c414bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:37.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.714+0000 7f66c2a00640 1 --2- 172.21.15.32:0/1263751310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66c4155a70 0x7f66c414bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:37.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.714+0000 7f66c2a00640 1 -- 172.21.15.32:0/1263751310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c4155e50 msgr2=0x7f66c41424b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:37.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.714+0000 7f66c2a00640 1 --2- 172.21.15.32:0/1263751310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c4155e50 0x7f66c41424b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:37.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.714+0000 7f66c2a00640 1 -- 172.21.15.32:0/1263751310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c414c350 con 0x7f66c40b3340 2024-04-17T10:04:37.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.714+0000 7f66c3400640 1 --2- 172.21.15.32:0/1263751310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c4155e50 0x7f66c41424b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:37.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.714+0000 7f66c2a00640 1 --2- 172.21.15.32:0/1263751310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66c40b3340 0x7f66c4146170 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f66b4019090 tx=0x7f66b401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:37.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.714+0000 7f66baa00640 1 -- 172.21.15.32:0/1263751310 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b40b0050 con 0x7f66c40b3340 2024-04-17T10:04:37.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.714+0000 7f66baa00640 1 -- 172.21.15.32:0/1263751310 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b40bc070 con 0x7f66c40b3340 2024-04-17T10:04:37.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.714+0000 7f66baa00640 1 -- 172.21.15.32:0/1263751310 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b40b5830 con 0x7f66c40b3340 2024-04-17T10:04:37.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.714+0000 7f66c9800640 1 -- 172.21.15.32:0/1263751310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c414c630 con 0x7f66c40b3340 2024-04-17T10:04:37.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.714+0000 7f66c9800640 1 -- 172.21.15.32:0/1263751310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c414cbf0 con 0x7f66c40b3340 2024-04-17T10:04:37.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.714+0000 7f66c9800640 1 -- 172.21.15.32:0/1263751310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66c4068a90 con 0x7f66c40b3340 2024-04-17T10:04:37.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.714+0000 7f66baa00640 1 -- 172.21.15.32:0/1263751310 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f66b40b59d0 con 0x7f66c40b3340 2024-04-17T10:04:37.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.714+0000 7f66baa00640 1 --2- 172.21.15.32:0/1263751310 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f66a40683b0 0x7f66a406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:37.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.718+0000 7f66baa00640 1 -- 172.21.15.32:0/1263751310 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f66b4121160 con 0x7f66c40b3340 2024-04-17T10:04:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.718+0000 7f66c2000640 1 --2- 172.21.15.32:0/1263751310 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f66a40683b0 0x7f66a406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.718+0000 7f66c2000640 1 --2- 172.21.15.32:0/1263751310 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f66a40683b0 0x7f66a406a870 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f66c4151b70 tx=0x7f66b0007d40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:37.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.718+0000 7f66baa00640 1 -- 172.21.15.32:0/1263751310 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f66b4016070 con 0x7f66c40b3340 2024-04-17T10:04:37.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.926+0000 7f66c9800640 1 -- 172.21.15.32:0/1263751310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f66c406cd60 con 0x7f66c40b3340 2024-04-17T10:04:37.921 INFO:tasks.ceph.mon.a.smithi032.stderr:2024-04-17T10:04:37.926+0000 7f93c9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T10:04:37.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.926+0000 7f66baa00640 1 -- 172.21.15.32:0/1263751310 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v112) ==== 161+0+0 (secure 0 0 0) 0x7f66b40f1620 con 0x7f66c40b3340 2024-04-17T10:04:37.923 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T10:04:37.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.930+0000 7f66c9800640 1 -- 172.21.15.32:0/1263751310 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f66a40683b0 msgr2=0x7f66a406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:37.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.930+0000 7f66c9800640 1 --2- 172.21.15.32:0/1263751310 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f66a40683b0 0x7f66a406a870 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f66c4151b70 tx=0x7f66b0007d40 comp rx=0 tx=0).stop 2024-04-17T10:04:37.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.930+0000 7f66c9800640 1 -- 172.21.15.32:0/1263751310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66c40b3340 msgr2=0x7f66c4146170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:37.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.930+0000 7f66c9800640 1 --2- 172.21.15.32:0/1263751310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66c40b3340 0x7f66c4146170 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f66b4019090 tx=0x7f66b401acc0 comp rx=0 tx=0).stop 2024-04-17T10:04:37.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.930+0000 7f66c9800640 1 -- 172.21.15.32:0/1263751310 shutdown_connections 2024-04-17T10:04:37.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.930+0000 7f66c9800640 1 --2- 172.21.15.32:0/1263751310 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f66a40683b0 0x7f66a406a870 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:37.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.930+0000 7f66c9800640 1 --2- 172.21.15.32:0/1263751310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66c4155a70 0x7f66c414bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:37.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.930+0000 7f66c9800640 1 --2- 172.21.15.32:0/1263751310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c4155e50 0x7f66c41424b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:37.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.930+0000 7f66c9800640 1 --2- 172.21.15.32:0/1263751310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66c40b3340 0x7f66c4146170 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:37.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.930+0000 7f66c9800640 1 -- 172.21.15.32:0/1263751310 >> 172.21.15.32:0/1263751310 conn(0x7f66c406ba30 msgr2=0x7f66c415c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:37.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.930+0000 7f66c9800640 1 -- 172.21.15.32:0/1263751310 shutdown_connections 2024-04-17T10:04:37.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:37.930+0000 7f66c9800640 1 -- 172.21.15.32:0/1263751310 wait complete. 2024-04-17T10:04:37.931 INFO:tasks.ceph.mds.a.smithi032.stderr: -1508> 2024-04-17T10:04:28.438+0000 7fd889600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T10:04:37.947 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-17T10:04:37.975 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2024-04-17T10:04:38.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.098+0000 7f85b9a00640 1 Processor -- start 2024-04-17T10:04:38.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.098+0000 7f85b9a00640 1 -- start start 2024-04-17T10:04:38.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.098+0000 7f85b9a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85b415c4c0 0x7f85b40adb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:38.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.098+0000 7f85b9a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f85b415f9c0 0x7f85b406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:38.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.098+0000 7f85b9a00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85b415fe50 con 0x7f85b406bf70 2024-04-17T10:04:38.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.098+0000 7f85b9a00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f85b406c370 con 0x7f85b415c4c0 2024-04-17T10:04:38.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.098+0000 7f85b9a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f85b41629a0 con 0x7f85b415f9c0 2024-04-17T10:04:38.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.098+0000 7f85b2a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85b415c4c0 0x7f85b40adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:38.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.098+0000 7f85b3400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f85b406bf70 0x7f85b4071e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44394/0 (socket says 172.21.15.32:44394) 2024-04-17T10:04:38.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.098+0000 7f85b3400640 1 -- 172.21.15.32:0/833386697 learned_addr learned my addr 172.21.15.32:0/833386697 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:38.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.098+0000 7f85aa000640 1 --2- 172.21.15.32:0/833386697 >> v2:172.21.15.144:3300/0 conn(0x7f85b415f9c0 0x7f85b406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:38.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.098+0000 7f85b2a00640 1 -- 172.21.15.32:0/833386697 >> v2:172.21.15.144:3300/0 conn(0x7f85b415f9c0 msgr2=0x7f85b406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:38.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.098+0000 7f85b2a00640 1 --2- 172.21.15.32:0/833386697 >> v2:172.21.15.144:3300/0 conn(0x7f85b415f9c0 0x7f85b406ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:38.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.098+0000 7f85b2a00640 1 -- 172.21.15.32:0/833386697 >> v1:172.21.15.32:6789/0 conn(0x7f85b406bf70 legacy=0x7f85b4071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:38.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.098+0000 7f85b2000640 1 -- 172.21.15.32:0/833386697 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1321467225 0 0) 0x7f85b415fe50 con 0x7f85b406bf70 2024-04-17T10:04:38.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.098+0000 7f85b2a00640 1 -- 172.21.15.32:0/833386697 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b4162c80 con 0x7f85b415c4c0 2024-04-17T10:04:38.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.098+0000 7f85aa000640 1 --2- 172.21.15.32:0/833386697 >> v2:172.21.15.144:3300/0 conn(0x7f85b415f9c0 0x7f85b406ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:38.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.102+0000 7f85b2a00640 1 --2- 172.21.15.32:0/833386697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b415c4c0 0x7f85b40adb40 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f85b406df10 tx=0x7f85a401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a440462c5e7f8263 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:38.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.102+0000 7f85b2000640 1 -- 172.21.15.32:0/833386697 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a40b0070 con 0x7f85b415c4c0 2024-04-17T10:04:38.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.102+0000 7f85b2000640 1 -- 172.21.15.32:0/833386697 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a40ab600 con 0x7f85b415c4c0 2024-04-17T10:04:38.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.102+0000 7f85b2000640 1 -- 172.21.15.32:0/833386697 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a40ab9d0 con 0x7f85b415c4c0 2024-04-17T10:04:38.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.102+0000 7f85b9a00640 1 -- 172.21.15.32:0/833386697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b415c4c0 msgr2=0x7f85b40adb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:38.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.102+0000 7f85b9a00640 1 --2- 172.21.15.32:0/833386697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b415c4c0 0x7f85b40adb40 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f85b406df10 tx=0x7f85a401c480 comp rx=0 tx=0).stop 2024-04-17T10:04:38.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.102+0000 7f85b9a00640 1 -- 172.21.15.32:0/833386697 shutdown_connections 2024-04-17T10:04:38.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.102+0000 7f85b9a00640 1 --2- 172.21.15.32:0/833386697 >> v2:172.21.15.144:3300/0 conn(0x7f85b415f9c0 0x7f85b406ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:38.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.102+0000 7f85b9a00640 1 --2- 172.21.15.32:0/833386697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b415c4c0 0x7f85b40adb40 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:38.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.102+0000 7f85b9a00640 1 -- 172.21.15.32:0/833386697 >> 172.21.15.32:0/833386697 conn(0x7f85b40b2120 msgr2=0x7f85b40b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:38.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.102+0000 7f85b9a00640 1 -- 172.21.15.32:0/833386697 shutdown_connections 2024-04-17T10:04:38.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.102+0000 7f85b9a00640 1 -- 172.21.15.32:0/833386697 wait complete. 2024-04-17T10:04:38.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.106+0000 7f85b9a00640 1 Processor -- start 2024-04-17T10:04:38.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.106+0000 7f85b9a00640 1 -- start start 2024-04-17T10:04:38.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.106+0000 7f85b9a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b406bf70 0x7f85b40cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:38.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.106+0000 7f85b9a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85b415c4c0 0x7f85b40cf140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:38.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.106+0000 7f85b9a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85b415f9c0 0x7f85b40afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:38.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.106+0000 7f85b9a00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f85b4162530 con 0x7f85b406bf70 2024-04-17T10:04:38.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.106+0000 7f85b9a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f85b40af670 con 0x7f85b415c4c0 2024-04-17T10:04:38.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.106+0000 7f85b9a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f85b41626b0 con 0x7f85b415f9c0 2024-04-17T10:04:38.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.106+0000 7f85b2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b406bf70 0x7f85b40cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:38.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.106+0000 7f85b2a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b406bf70 0x7f85b40cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59876/0 (socket says 172.21.15.32:59876) 2024-04-17T10:04:38.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.106+0000 7f85b2a00640 1 -- 172.21.15.32:0/4126847078 learned_addr learned my addr 172.21.15.32:0/4126847078 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:38.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.106+0000 7f85b2a00640 1 -- 172.21.15.32:0/4126847078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85b415f9c0 msgr2=0x7f85b40afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:38.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.106+0000 7f85b2a00640 1 --2- 172.21.15.32:0/4126847078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85b415f9c0 0x7f85b40afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:38.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.106+0000 7f85b2a00640 1 -- 172.21.15.32:0/4126847078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85b415c4c0 msgr2=0x7f85b40cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:38.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.106+0000 7f85b2a00640 1 --2- 172.21.15.32:0/4126847078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85b415c4c0 0x7f85b40cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:38.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.106+0000 7f85b2a00640 1 -- 172.21.15.32:0/4126847078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b40cc9e0 con 0x7f85b406bf70 2024-04-17T10:04:38.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.106+0000 7f85b2a00640 1 --2- 172.21.15.32:0/4126847078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b406bf70 0x7f85b40cd3f0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f85a401a0b0 tx=0x7f85a401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:38.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.106+0000 7f85ab400640 1 -- 172.21.15.32:0/4126847078 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a40b0050 con 0x7f85b406bf70 2024-04-17T10:04:38.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.106+0000 7f85ab400640 1 -- 172.21.15.32:0/4126847078 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a40b1040 con 0x7f85b406bf70 2024-04-17T10:04:38.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.106+0000 7f85ab400640 1 -- 172.21.15.32:0/4126847078 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4004220 con 0x7f85b406bf70 2024-04-17T10:04:38.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.106+0000 7f85b9a00640 1 -- 172.21.15.32:0/4126847078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b41532c0 con 0x7f85b406bf70 2024-04-17T10:04:38.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.106+0000 7f85b9a00640 1 -- 172.21.15.32:0/4126847078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b40ccf60 con 0x7f85b406bf70 2024-04-17T10:04:38.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.106+0000 7f85b9a00640 1 -- 172.21.15.32:0/4126847078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b4068a90 con 0x7f85b406bf70 2024-04-17T10:04:38.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.110+0000 7f85ab400640 1 -- 172.21.15.32:0/4126847078 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f85a40be060 con 0x7f85b406bf70 2024-04-17T10:04:38.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.110+0000 7f85ab400640 1 --2- 172.21.15.32:0/4126847078 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8590068380 0x7f859006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:38.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.110+0000 7f85ab400640 1 -- 172.21.15.32:0/4126847078 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6025+0+0 (secure 0 0 0) 0x7f85a4120520 con 0x7f85b406bf70 2024-04-17T10:04:38.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.110+0000 7f85aa000640 1 --2- 172.21.15.32:0/4126847078 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8590068380 0x7f859006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:38.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.110+0000 7f85aa000640 1 --2- 172.21.15.32:0/4126847078 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8590068380 0x7f859006a840 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f85b4152840 tx=0x7f859c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:38.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.114+0000 7f85ab400640 1 -- 172.21.15.32:0/4126847078 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f85a4016070 con 0x7f85b406bf70 2024-04-17T10:04:38.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.314+0000 7f85b9a00640 1 -- 172.21.15.32:0/4126847078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f85b4072100 con 0x7f85b406bf70 2024-04-17T10:04:38.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.930+0000 7f85ab400640 1 -- 172.21.15.32:0/4126847078 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v113) ==== 104+0+0 (secure 0 0 0) 0x7f85a40f09a0 con 0x7f85b406bf70 2024-04-17T10:04:38.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.934+0000 7f85b9a00640 1 -- 172.21.15.32:0/4126847078 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8590068380 msgr2=0x7f859006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:38.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.934+0000 7f85b9a00640 1 --2- 172.21.15.32:0/4126847078 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8590068380 0x7f859006a840 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f85b4152840 tx=0x7f859c008040 comp rx=0 tx=0).stop 2024-04-17T10:04:38.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.934+0000 7f85b9a00640 1 -- 172.21.15.32:0/4126847078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b406bf70 msgr2=0x7f85b40cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:38.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.934+0000 7f85b9a00640 1 --2- 172.21.15.32:0/4126847078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b406bf70 0x7f85b40cd3f0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f85a401a0b0 tx=0x7f85a401acc0 comp rx=0 tx=0).stop 2024-04-17T10:04:38.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.934+0000 7f85b9a00640 1 -- 172.21.15.32:0/4126847078 shutdown_connections 2024-04-17T10:04:38.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.934+0000 7f85b9a00640 1 --2- 172.21.15.32:0/4126847078 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8590068380 0x7f859006a840 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:38.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.934+0000 7f85b9a00640 1 --2- 172.21.15.32:0/4126847078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85b415f9c0 0x7f85b40afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:38.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.934+0000 7f85b9a00640 1 --2- 172.21.15.32:0/4126847078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85b415c4c0 0x7f85b40cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:38.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.934+0000 7f85b9a00640 1 --2- 172.21.15.32:0/4126847078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b406bf70 0x7f85b40cd3f0 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:38.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.934+0000 7f85b9a00640 1 -- 172.21.15.32:0/4126847078 >> 172.21.15.32:0/4126847078 conn(0x7f85b40b2120 msgr2=0x7f85b40b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:38.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.934+0000 7f85b9a00640 1 -- 172.21.15.32:0/4126847078 shutdown_connections 2024-04-17T10:04:38.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:38.934+0000 7f85b9a00640 1 -- 172.21.15.32:0/4126847078 wait complete. 2024-04-17T10:04:38.954 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-17T10:04:39.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.154+0000 7f41a6200640 1 Processor -- start 2024-04-17T10:04:39.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.154+0000 7f41a6200640 1 -- start start 2024-04-17T10:04:39.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.154+0000 7f41a6200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f41a015acc0 0x7f41a00cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:39.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.154+0000 7f41a6200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41a00ceb50 0x7f41a00cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:39.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.154+0000 7f41a6200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41a015b0a0 con 0x7f41a015e840 2024-04-17T10:04:39.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.154+0000 7f41a6200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f41a015f020 con 0x7f41a015acc0 2024-04-17T10:04:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.154+0000 7f41a6200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f41a015f300 con 0x7f41a00ceb50 2024-04-17T10:04:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.154+0000 7f4196a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f41a015e840 0x7f41a006e410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44410/0 (socket says 172.21.15.32:44410) 2024-04-17T10:04:39.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.154+0000 7f4196a00640 1 -- 172.21.15.32:0/2785825680 learned_addr learned my addr 172.21.15.32:0/2785825680 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:39.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.154+0000 7f419f400640 1 --2- 172.21.15.32:0/2785825680 >> v2:172.21.15.144:3300/0 conn(0x7f41a015acc0 0x7f41a00cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:39.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.154+0000 7f419fe00640 1 --2- 172.21.15.32:0/2785825680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41a00ceb50 0x7f41a00cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:39.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.154+0000 7f419ea00640 1 -- 172.21.15.32:0/2785825680 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2727002564 0 0) 0x7f41a015b0a0 con 0x7f41a015e840 2024-04-17T10:04:39.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.154+0000 7f419ea00640 1 -- 172.21.15.32:0/2785825680 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4184003530 con 0x7f41a015e840 2024-04-17T10:04:39.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.154+0000 7f419ea00640 1 -- 172.21.15.32:0/2785825680 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3669620898 0 0) 0x7f4184003530 con 0x7f41a015e840 2024-04-17T10:04:39.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.154+0000 7f419f400640 1 -- 172.21.15.32:0/2785825680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41a00ceb50 msgr2=0x7f41a00cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:39.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.154+0000 7f419f400640 1 --2- 172.21.15.32:0/2785825680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41a00ceb50 0x7f41a00cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:39.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.154+0000 7f419f400640 1 -- 172.21.15.32:0/2785825680 >> v1:172.21.15.32:6789/0 conn(0x7f41a015e840 legacy=0x7f41a006e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:39.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.154+0000 7f419f400640 1 -- 172.21.15.32:0/2785825680 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a015f5e0 con 0x7f41a015acc0 2024-04-17T10:04:39.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.158+0000 7f419f400640 1 --2- 172.21.15.32:0/2785825680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41a015acc0 0x7f41a00cd230 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f419000a960 tx=0x7f419001d700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bed66c62e374ce8f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:39.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.158+0000 7f419ea00640 1 -- 172.21.15.32:0/2785825680 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41900ae070 con 0x7f41a015acc0 2024-04-17T10:04:39.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.158+0000 7f419ea00640 1 -- 172.21.15.32:0/2785825680 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41900aa420 con 0x7f41a015acc0 2024-04-17T10:04:39.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.158+0000 7f419ea00640 1 -- 172.21.15.32:0/2785825680 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4188001800 con 0x7f41a015acc0 2024-04-17T10:04:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.158+0000 7f41a6200640 1 -- 172.21.15.32:0/2785825680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41a015acc0 msgr2=0x7f41a00cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.158+0000 7f41a6200640 1 --2- 172.21.15.32:0/2785825680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41a015acc0 0x7f41a00cd230 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f419000a960 tx=0x7f419001d700 comp rx=0 tx=0).stop 2024-04-17T10:04:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.158+0000 7f41a6200640 1 -- 172.21.15.32:0/2785825680 shutdown_connections 2024-04-17T10:04:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.158+0000 7f41a6200640 1 --2- 172.21.15.32:0/2785825680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41a00ceb50 0x7f41a00cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.158+0000 7f41a6200640 1 --2- 172.21.15.32:0/2785825680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41a015acc0 0x7f41a00cd230 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.158+0000 7f41a6200640 1 -- 172.21.15.32:0/2785825680 >> 172.21.15.32:0/2785825680 conn(0x7f41a00b2190 msgr2=0x7f41a00b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.158+0000 7f41a6200640 1 -- 172.21.15.32:0/2785825680 shutdown_connections 2024-04-17T10:04:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.158+0000 7f41a6200640 1 -- 172.21.15.32:0/2785825680 wait complete. 2024-04-17T10:04:39.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f41a6200640 1 Processor -- start 2024-04-17T10:04:39.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f41a6200640 1 -- start start 2024-04-17T10:04:39.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f41a6200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41a00ceb50 0x7f41a015de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:39.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f41a6200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41a015acc0 0x7f41a00b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:39.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f41a6200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a015e840 0x7f41a00b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:39.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f41a6200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f41a0072740 con 0x7f41a015e840 2024-04-17T10:04:39.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f41a6200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f41a00725c0 con 0x7f41a015acc0 2024-04-17T10:04:39.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f41a6200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f41a00728c0 con 0x7f41a00ceb50 2024-04-17T10:04:39.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f419fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a015e840 0x7f41a00b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:39.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f419fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a015e840 0x7f41a00b3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59882/0 (socket says 172.21.15.32:59882) 2024-04-17T10:04:39.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f419fe00640 1 -- 172.21.15.32:0/2681738003 learned_addr learned my addr 172.21.15.32:0/2681738003 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:39.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f4196a00640 1 --2- 172.21.15.32:0/2681738003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41a015acc0 0x7f41a00b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:39.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f419f400640 1 --2- 172.21.15.32:0/2681738003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41a00ceb50 0x7f41a015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:39.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f419fe00640 1 -- 172.21.15.32:0/2681738003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41a00ceb50 msgr2=0x7f41a015de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:39.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f419fe00640 1 --2- 172.21.15.32:0/2681738003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41a00ceb50 0x7f41a015de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:39.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f419fe00640 1 -- 172.21.15.32:0/2681738003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41a015acc0 msgr2=0x7f41a00b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:39.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f419fe00640 1 --2- 172.21.15.32:0/2681738003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41a015acc0 0x7f41a00b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:39.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f419fe00640 1 -- 172.21.15.32:0/2681738003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a00b0fa0 con 0x7f41a015e840 2024-04-17T10:04:39.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f419fe00640 1 --2- 172.21.15.32:0/2681738003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a015e840 0x7f41a00b3280 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f419800efc0 tx=0x7f419800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:39.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f4197e00640 1 -- 172.21.15.32:0/2681738003 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4198018020 con 0x7f41a015e840 2024-04-17T10:04:39.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f4197e00640 1 -- 172.21.15.32:0/2681738003 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4198014480 con 0x7f41a015e840 2024-04-17T10:04:39.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f4197e00640 1 -- 172.21.15.32:0/2681738003 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4198014870 con 0x7f41a015e840 2024-04-17T10:04:39.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f41a6200640 1 -- 172.21.15.32:0/2681738003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a00b1280 con 0x7f41a015e840 2024-04-17T10:04:39.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.162+0000 7f41a6200640 1 -- 172.21.15.32:0/2681738003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a01417b0 con 0x7f41a015e840 2024-04-17T10:04:39.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.166+0000 7f41a6200640 1 -- 172.21.15.32:0/2681738003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41a0068a90 con 0x7f41a015e840 2024-04-17T10:04:39.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.166+0000 7f4197e00640 1 -- 172.21.15.32:0/2681738003 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f419801b020 con 0x7f41a015e840 2024-04-17T10:04:39.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.166+0000 7f4197e00640 1 --2- 172.21.15.32:0/2681738003 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f417c068380 0x7f417c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:39.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.166+0000 7f4197e00640 1 -- 172.21.15.32:0/2681738003 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6025+0+0 (secure 0 0 0) 0x7f4198087560 con 0x7f41a015e840 2024-04-17T10:04:39.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.166+0000 7f419f400640 1 --2- 172.21.15.32:0/2681738003 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f417c068380 0x7f417c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:39.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.166+0000 7f419f400640 1 --2- 172.21.15.32:0/2681738003 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f417c068380 0x7f417c06a840 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f41900b0000 tx=0x7f4190008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:39.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.170+0000 7f4197e00640 1 -- 172.21.15.32:0/2681738003 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f4198022070 con 0x7f41a015e840 2024-04-17T10:04:39.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.334+0000 7f41a6200640 1 -- 172.21.15.32:0/2681738003 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f41a006f5e0 con 0x7f41a015e840 2024-04-17T10:04:39.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.934+0000 7f4197e00640 1 -- 172.21.15.32:0/2681738003 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v115) ==== 181+0+0 (secure 0 0 0) 0x7f4198010040 con 0x7f41a015e840 2024-04-17T10:04:39.931 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-04-17T10:04:39.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.938+0000 7f41a6200640 1 -- 172.21.15.32:0/2681738003 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f417c068380 msgr2=0x7f417c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:39.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.938+0000 7f41a6200640 1 --2- 172.21.15.32:0/2681738003 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f417c068380 0x7f417c06a840 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f41900b0000 tx=0x7f4190008040 comp rx=0 tx=0).stop 2024-04-17T10:04:39.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.938+0000 7f41a6200640 1 -- 172.21.15.32:0/2681738003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a015e840 msgr2=0x7f41a00b3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:39.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.938+0000 7f41a6200640 1 --2- 172.21.15.32:0/2681738003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a015e840 0x7f41a00b3280 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f419800efc0 tx=0x7f419800c490 comp rx=0 tx=0).stop 2024-04-17T10:04:39.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.938+0000 7f41a6200640 1 -- 172.21.15.32:0/2681738003 shutdown_connections 2024-04-17T10:04:39.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.938+0000 7f41a6200640 1 --2- 172.21.15.32:0/2681738003 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f417c068380 0x7f417c06a840 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:39.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.938+0000 7f41a6200640 1 --2- 172.21.15.32:0/2681738003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a015e840 0x7f41a00b3280 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:39.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.938+0000 7f41a6200640 1 --2- 172.21.15.32:0/2681738003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41a015acc0 0x7f41a00b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:39.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.938+0000 7f41a6200640 1 --2- 172.21.15.32:0/2681738003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41a00ceb50 0x7f41a015de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:39.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.938+0000 7f41a6200640 1 -- 172.21.15.32:0/2681738003 >> 172.21.15.32:0/2681738003 conn(0x7f41a00b2190 msgr2=0x7f41a0165400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:39.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.938+0000 7f41a6200640 1 -- 172.21.15.32:0/2681738003 shutdown_connections 2024-04-17T10:04:39.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:39.938+0000 7f41a6200640 1 -- 172.21.15.32:0/2681738003 wait complete. 2024-04-17T10:04:39.957 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-17T10:04:40.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.106+0000 7f9510c00640 1 Processor -- start 2024-04-17T10:04:40.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.106+0000 7f9510c00640 1 -- start start 2024-04-17T10:04:40.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.110+0000 7f9510c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f950c0cd870 0x7f950c1661a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.110+0000 7f9510c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f950c15f200 con 0x7f950c06f500 2024-04-17T10:04:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.110+0000 7f9510c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f950c15f4e0 con 0x7f950c0cfc70 2024-04-17T10:04:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.110+0000 7f9510c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f950c15fb30 con 0x7f950c0cd870 2024-04-17T10:04:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.110+0000 7f950a800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f950c0cd870 0x7f950c1661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.110+0000 7f9501400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f950c06f500 0x7f950c0cd160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44422/0 (socket says 172.21.15.32:44422) 2024-04-17T10:04:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.110+0000 7f9501400640 1 -- 172.21.15.32:0/113732885 learned_addr learned my addr 172.21.15.32:0/113732885 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.110+0000 7f950a800640 1 -- 172.21.15.32:0/113732885 >> v1:172.21.15.144:6789/0 conn(0x7f950c0cfc70 legacy=0x7f950c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:40.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.110+0000 7f950a800640 1 -- 172.21.15.32:0/113732885 >> v1:172.21.15.32:6789/0 conn(0x7f950c06f500 legacy=0x7f950c0cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:40.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.110+0000 7f950a800640 1 -- 172.21.15.32:0/113732885 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f950c15f7c0 con 0x7f950c0cd870 2024-04-17T10:04:40.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.110+0000 7f9509400640 1 -- 172.21.15.32:0/113732885 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1565749026 0 0) 0x7f950c15f200 con 0x7f950c06f500 2024-04-17T10:04:40.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.114+0000 7f950a800640 1 --2- 172.21.15.32:0/113732885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950c0cd870 0x7f950c1661a0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f95040097c0 tx=0x7f950401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0d691478a8490a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:40.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.114+0000 7f9509400640 1 -- 172.21.15.32:0/113732885 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95040b1070 con 0x7f950c0cd870 2024-04-17T10:04:40.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.114+0000 7f9509400640 1 -- 172.21.15.32:0/113732885 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95040ac410 con 0x7f950c0cd870 2024-04-17T10:04:40.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.114+0000 7f9509400640 1 -- 172.21.15.32:0/113732885 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95040ac7e0 con 0x7f950c0cd870 2024-04-17T10:04:40.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.114+0000 7f9510c00640 1 -- 172.21.15.32:0/113732885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950c0cd870 msgr2=0x7f950c1661a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:40.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.114+0000 7f9510c00640 1 --2- 172.21.15.32:0/113732885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950c0cd870 0x7f950c1661a0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f95040097c0 tx=0x7f950401c4a0 comp rx=0 tx=0).stop 2024-04-17T10:04:40.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.114+0000 7f9510c00640 1 -- 172.21.15.32:0/113732885 shutdown_connections 2024-04-17T10:04:40.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.114+0000 7f9510c00640 1 --2- 172.21.15.32:0/113732885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950c0cd870 0x7f950c1661a0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:40.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.114+0000 7f9510c00640 1 -- 172.21.15.32:0/113732885 >> 172.21.15.32:0/113732885 conn(0x7f950c0b2190 msgr2=0x7f950c0b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:40.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.114+0000 7f9510c00640 1 -- 172.21.15.32:0/113732885 shutdown_connections 2024-04-17T10:04:40.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.114+0000 7f9510c00640 1 -- 172.21.15.32:0/113732885 wait complete. 2024-04-17T10:04:40.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.114+0000 7f9510c00640 1 Processor -- start 2024-04-17T10:04:40.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.114+0000 7f9510c00640 1 -- start start 2024-04-17T10:04:40.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f9510c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f950c06f500 0x7f950c143c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:40.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f9510c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950c0cd870 0x7f950c15d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:40.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f9510c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f950c0cfc70 0x7f950c14fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:40.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f9510c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f950c169110 con 0x7f950c0cd870 2024-04-17T10:04:40.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f9510c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f950c0d0260 con 0x7f950c0cfc70 2024-04-17T10:04:40.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f9510c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f950c169290 con 0x7f950c06f500 2024-04-17T10:04:40.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f9501400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950c0cd870 0x7f950c15d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:40.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f9501400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950c0cd870 0x7f950c15d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59898/0 (socket says 172.21.15.32:59898) 2024-04-17T10:04:40.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f9501400640 1 -- 172.21.15.32:0/1482983938 learned_addr learned my addr 172.21.15.32:0/1482983938 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f9509e00640 1 --2- 172.21.15.32:0/1482983938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f950c06f500 0x7f950c143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f950a800640 1 --2- 172.21.15.32:0/1482983938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f950c0cfc70 0x7f950c14fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f9501400640 1 -- 172.21.15.32:0/1482983938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f950c06f500 msgr2=0x7f950c143c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f9501400640 1 --2- 172.21.15.32:0/1482983938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f950c06f500 0x7f950c143c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f9501400640 1 -- 172.21.15.32:0/1482983938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f950c0cfc70 msgr2=0x7f950c14fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f9501400640 1 --2- 172.21.15.32:0/1482983938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f950c0cfc70 0x7f950c14fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f9501400640 1 -- 172.21.15.32:0/1482983938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f950c1503f0 con 0x7f950c0cd870 2024-04-17T10:04:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f950a800640 1 --2- 172.21.15.32:0/1482983938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f950c0cfc70 0x7f950c14fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f9501400640 1 --2- 172.21.15.32:0/1482983938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950c0cd870 0x7f950c15d680 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f94f400c760 tx=0x7f94f400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f9502a00640 1 -- 172.21.15.32:0/1482983938 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f4007270 con 0x7f950c0cd870 2024-04-17T10:04:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f9502a00640 1 -- 172.21.15.32:0/1482983938 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94f4007430 con 0x7f950c0cd870 2024-04-17T10:04:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f9502a00640 1 -- 172.21.15.32:0/1482983938 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f40059f0 con 0x7f950c0cd870 2024-04-17T10:04:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f9510c00640 1 -- 172.21.15.32:0/1482983938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f950c1506d0 con 0x7f950c0cd870 2024-04-17T10:04:40.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f9510c00640 1 -- 172.21.15.32:0/1482983938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f950c15ce80 con 0x7f950c0cd870 2024-04-17T10:04:40.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f9510c00640 1 -- 172.21.15.32:0/1482983938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f950c068a90 con 0x7f950c0cd870 2024-04-17T10:04:40.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.118+0000 7f9502a00640 1 -- 172.21.15.32:0/1482983938 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f94f4005b90 con 0x7f950c0cd870 2024-04-17T10:04:40.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.122+0000 7f9502a00640 1 --2- 172.21.15.32:0/1482983938 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f94e8068380 0x7f94e806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:40.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.122+0000 7f9502a00640 1 -- 172.21.15.32:0/1482983938 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5587+0+0 (secure 0 0 0) 0x7f94f4086530 con 0x7f950c0cd870 2024-04-17T10:04:40.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.122+0000 7f9509e00640 1 --2- 172.21.15.32:0/1482983938 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f94e8068380 0x7f94e806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:40.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.122+0000 7f9509e00640 1 --2- 172.21.15.32:0/1482983938 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f94e8068380 0x7f94e806a840 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f94fc0012f0 tx=0x7f94fc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:40.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.126+0000 7f9502a00640 1 -- 172.21.15.32:0/1482983938 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f94f4001d10 con 0x7f950c0cd870 2024-04-17T10:04:40.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.298+0000 7f9510c00640 1 -- 172.21.15.32:0/1482983938 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f950c0b35b0 con 0x7f950c0cd870 2024-04-17T10:04:40.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.942+0000 7f9502a00640 1 -- 172.21.15.32:0/1482983938 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v116) ==== 169+0+0 (secure 0 0 0) 0x7f94f4056b50 con 0x7f950c0cd870 2024-04-17T10:04:40.937 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-04-17T10:04:40.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.946+0000 7f9510c00640 1 -- 172.21.15.32:0/1482983938 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f94e8068380 msgr2=0x7f94e806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:40.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.946+0000 7f9510c00640 1 --2- 172.21.15.32:0/1482983938 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f94e8068380 0x7f94e806a840 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f94fc0012f0 tx=0x7f94fc009040 comp rx=0 tx=0).stop 2024-04-17T10:04:40.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.946+0000 7f9510c00640 1 -- 172.21.15.32:0/1482983938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950c0cd870 msgr2=0x7f950c15d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:40.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.946+0000 7f9510c00640 1 --2- 172.21.15.32:0/1482983938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950c0cd870 0x7f950c15d680 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f94f400c760 tx=0x7f94f400cc20 comp rx=0 tx=0).stop 2024-04-17T10:04:40.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.946+0000 7f9510c00640 1 -- 172.21.15.32:0/1482983938 shutdown_connections 2024-04-17T10:04:40.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.946+0000 7f9510c00640 1 --2- 172.21.15.32:0/1482983938 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f94e8068380 0x7f94e806a840 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:40.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.946+0000 7f9510c00640 1 --2- 172.21.15.32:0/1482983938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f950c0cfc70 0x7f950c14fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:40.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.946+0000 7f9510c00640 1 --2- 172.21.15.32:0/1482983938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f950c0cd870 0x7f950c15d680 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:40.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.946+0000 7f9510c00640 1 --2- 172.21.15.32:0/1482983938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f950c06f500 0x7f950c143c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:40.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.946+0000 7f9510c00640 1 -- 172.21.15.32:0/1482983938 >> 172.21.15.32:0/1482983938 conn(0x7f950c0b2190 msgr2=0x7f950c0b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:40.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.946+0000 7f9510c00640 1 -- 172.21.15.32:0/1482983938 shutdown_connections 2024-04-17T10:04:40.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:40.946+0000 7f9510c00640 1 -- 172.21.15.32:0/1482983938 wait complete. 2024-04-17T10:04:40.967 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-17T10:04:41.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.118+0000 7fbf86400640 1 Processor -- start 2024-04-17T10:04:41.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.118+0000 7fbf86400640 1 -- start start 2024-04-17T10:04:41.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.118+0000 7fbf86400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf8015b7d0 0x7fbf8006e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:41.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.118+0000 7fbf86400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf8006f7c0 0x7fbf8006ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:41.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.118+0000 7fbf86400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf800cc940 con 0x7fbf8006f270 2024-04-17T10:04:41.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.118+0000 7fbf86400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fbf800ccc20 con 0x7fbf8006f7c0 2024-04-17T10:04:41.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.118+0000 7fbf86400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbf8006e0b0 con 0x7fbf8015b7d0 2024-04-17T10:04:41.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.118+0000 7fbf76a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf8006f7c0 0x7fbf8006ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:41.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.118+0000 7fbf76a00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf8006f7c0 0x7fbf8006ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44062/0 (socket says 172.21.15.32:44062) 2024-04-17T10:04:41.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.118+0000 7fbf76a00640 1 -- 172.21.15.32:0/3245640300 learned_addr learned my addr 172.21.15.32:0/3245640300 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:41.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.118+0000 7fbf76a00640 1 -- 172.21.15.32:0/3245640300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf8015b7d0 msgr2=0x7fbf8006e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:41.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.118+0000 7fbf7f400640 1 --2- 172.21.15.32:0/3245640300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf8015b7d0 0x7fbf8006e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:41.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.118+0000 7fbf76a00640 1 --2- 172.21.15.32:0/3245640300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf8015b7d0 0x7fbf8006e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:41.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.118+0000 7fbf76a00640 1 -- 172.21.15.32:0/3245640300 >> v1:172.21.15.144:6789/0 conn(0x7fbf8006f270 legacy=0x7fbf800ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:41.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.122+0000 7fbf76a00640 1 -- 172.21.15.32:0/3245640300 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8006e390 con 0x7fbf8006f7c0 2024-04-17T10:04:41.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.122+0000 7fbf7f400640 1 --2- 172.21.15.32:0/3245640300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf8015b7d0 0x7fbf8006e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:41.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.122+0000 7fbf76a00640 1 --2- 172.21.15.32:0/3245640300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8006f7c0 0x7fbf8006ba30 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009870 tx=0x7fbf6801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2806a5fc78dcdcd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:41.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.122+0000 7fbf7ea00640 1 -- 172.21.15.32:0/3245640300 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf680aa650 con 0x7fbf8006f7c0 2024-04-17T10:04:41.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.122+0000 7fbf7ea00640 1 -- 172.21.15.32:0/3245640300 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf680af070 con 0x7fbf8006f7c0 2024-04-17T10:04:41.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.122+0000 7fbf7ea00640 1 -- 172.21.15.32:0/3245640300 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf680aaa10 con 0x7fbf8006f7c0 2024-04-17T10:04:41.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.122+0000 7fbf86400640 1 -- 172.21.15.32:0/3245640300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8006f7c0 msgr2=0x7fbf8006ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:41.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.122+0000 7fbf86400640 1 --2- 172.21.15.32:0/3245640300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8006f7c0 0x7fbf8006ba30 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009870 tx=0x7fbf6801c4d0 comp rx=0 tx=0).stop 2024-04-17T10:04:41.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.122+0000 7fbf86400640 1 -- 172.21.15.32:0/3245640300 shutdown_connections 2024-04-17T10:04:41.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.122+0000 7fbf86400640 1 --2- 172.21.15.32:0/3245640300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8006f7c0 0x7fbf8006ba30 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:41.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.122+0000 7fbf86400640 1 --2- 172.21.15.32:0/3245640300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf8015b7d0 0x7fbf8006e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:41.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.122+0000 7fbf86400640 1 -- 172.21.15.32:0/3245640300 >> 172.21.15.32:0/3245640300 conn(0x7fbf800b2120 msgr2=0x7fbf800b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:41.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.122+0000 7fbf86400640 1 -- 172.21.15.32:0/3245640300 shutdown_connections 2024-04-17T10:04:41.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.122+0000 7fbf86400640 1 -- 172.21.15.32:0/3245640300 wait complete. 2024-04-17T10:04:41.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf86400640 1 Processor -- start 2024-04-17T10:04:41.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf86400640 1 -- start start 2024-04-17T10:04:41.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf86400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf8006f270 0x7fbf801445c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:41.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf86400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf8006f7c0 0x7fbf800b0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf86400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8015b7d0 0x7fbf80159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf86400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbf80166aa0 con 0x7fbf8015b7d0 2024-04-17T10:04:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf86400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8006c380 con 0x7fbf8006f7c0 2024-04-17T10:04:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf86400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbf80166c20 con 0x7fbf8006f270 2024-04-17T10:04:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf7fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8015b7d0 0x7fbf80159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf7f400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf8006f270 0x7fbf801445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf7f400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf8006f270 0x7fbf801445c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.32:43160/0 (socket says 172.21.15.32:43160) 2024-04-17T10:04:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf7fe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8015b7d0 0x7fbf80159de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44066/0 (socket says 172.21.15.32:44066) 2024-04-17T10:04:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf7f400640 1 -- 172.21.15.32:0/2885935285 learned_addr learned my addr 172.21.15.32:0/2885935285 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf76a00640 1 --2- 172.21.15.32:0/2885935285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf8006f7c0 0x7fbf800b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf7fe00640 1 -- 172.21.15.32:0/2885935285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf8006f270 msgr2=0x7fbf801445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf7fe00640 1 --2- 172.21.15.32:0/2885935285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf8006f270 0x7fbf801445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:41.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf7fe00640 1 -- 172.21.15.32:0/2885935285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf8006f7c0 msgr2=0x7fbf800b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:41.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf7fe00640 1 --2- 172.21.15.32:0/2885935285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf8006f7c0 0x7fbf800b0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:41.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf7fe00640 1 -- 172.21.15.32:0/2885935285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf800b13b0 con 0x7fbf8015b7d0 2024-04-17T10:04:41.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf7fe00640 1 --2- 172.21.15.32:0/2885935285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8015b7d0 0x7fbf80159de0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800c700 tx=0x7fbf7800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:41.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf77e00640 1 -- 172.21.15.32:0/2885935285 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78003b80 con 0x7fbf8015b7d0 2024-04-17T10:04:41.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf77e00640 1 -- 172.21.15.32:0/2885935285 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf78003d20 con 0x7fbf8015b7d0 2024-04-17T10:04:41.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf86400640 1 -- 172.21.15.32:0/2885935285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf800b1690 con 0x7fbf8015b7d0 2024-04-17T10:04:41.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf77e00640 1 -- 172.21.15.32:0/2885935285 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7800aa30 con 0x7fbf8015b7d0 2024-04-17T10:04:41.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.126+0000 7fbf86400640 1 -- 172.21.15.32:0/2885935285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf80144b00 con 0x7fbf8015b7d0 2024-04-17T10:04:41.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.130+0000 7fbf86400640 1 -- 172.21.15.32:0/2885935285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf80154d70 con 0x7fbf8015b7d0 2024-04-17T10:04:41.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.130+0000 7fbf77e00640 1 -- 172.21.15.32:0/2885935285 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbf7800abd0 con 0x7fbf8015b7d0 2024-04-17T10:04:41.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.130+0000 7fbf77e00640 1 --2- 172.21.15.32:0/2885935285 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbf5c0683b0 0x7fbf5c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:41.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.130+0000 7fbf77e00640 1 -- 172.21.15.32:0/2885935285 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7fbf78086500 con 0x7fbf8015b7d0 2024-04-17T10:04:41.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.130+0000 7fbf7f400640 1 --2- 172.21.15.32:0/2885935285 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbf5c0683b0 0x7fbf5c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:41.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.130+0000 7fbf7f400640 1 --2- 172.21.15.32:0/2885935285 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbf5c0683b0 0x7fbf5c06a870 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fbf70006fd0 tx=0x7fbf70008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:41.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.134+0000 7fbf77e00640 1 -- 172.21.15.32:0/2885935285 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fbf78056620 con 0x7fbf8015b7d0 2024-04-17T10:04:41.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.298+0000 7fbf86400640 1 -- 172.21.15.32:0/2885935285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fbf80068a90 con 0x7fbf8015b7d0 2024-04-17T10:04:41.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.302+0000 7fbf77e00640 1 -- 172.21.15.32:0/2885935285 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7fbf7805a920 con 0x7fbf8015b7d0 2024-04-17T10:04:41.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.306+0000 7fbf86400640 1 -- 172.21.15.32:0/2885935285 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbf5c0683b0 msgr2=0x7fbf5c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:41.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.306+0000 7fbf86400640 1 --2- 172.21.15.32:0/2885935285 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbf5c0683b0 0x7fbf5c06a870 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fbf70006fd0 tx=0x7fbf70008040 comp rx=0 tx=0).stop 2024-04-17T10:04:41.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.306+0000 7fbf86400640 1 -- 172.21.15.32:0/2885935285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8015b7d0 msgr2=0x7fbf80159de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:41.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.306+0000 7fbf86400640 1 --2- 172.21.15.32:0/2885935285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8015b7d0 0x7fbf80159de0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800c700 tx=0x7fbf7800cbc0 comp rx=0 tx=0).stop 2024-04-17T10:04:41.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.306+0000 7fbf86400640 1 -- 172.21.15.32:0/2885935285 shutdown_connections 2024-04-17T10:04:41.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.306+0000 7fbf86400640 1 --2- 172.21.15.32:0/2885935285 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fbf5c0683b0 0x7fbf5c06a870 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:41.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.306+0000 7fbf86400640 1 --2- 172.21.15.32:0/2885935285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf8015b7d0 0x7fbf80159de0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:41.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.306+0000 7fbf86400640 1 --2- 172.21.15.32:0/2885935285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf8006f7c0 0x7fbf800b0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:41.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.306+0000 7fbf86400640 1 --2- 172.21.15.32:0/2885935285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf8006f270 0x7fbf801445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:41.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.306+0000 7fbf86400640 1 -- 172.21.15.32:0/2885935285 >> 172.21.15.32:0/2885935285 conn(0x7fbf800b2120 msgr2=0x7fbf80146780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:41.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.306+0000 7fbf86400640 1 -- 172.21.15.32:0/2885935285 shutdown_connections 2024-04-17T10:04:41.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.306+0000 7fbf86400640 1 -- 172.21.15.32:0/2885935285 wait complete. 2024-04-17T10:04:41.321 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-17T10:04:41.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.474+0000 7f93dba00640 1 Processor -- start 2024-04-17T10:04:41.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.474+0000 7f93dba00640 1 -- start start 2024-04-17T10:04:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.478+0000 7f93dba00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d4145f30 0x7f93d4071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.478+0000 7f93dba00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f93d415d8d0 0x7f93d4072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.478+0000 7f93dba00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93d40cdac0 con 0x7f93d40cce70 2024-04-17T10:04:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.478+0000 7f93dba00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f93d406f400 con 0x7f93d415d8d0 2024-04-17T10:04:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.478+0000 7f93dba00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f93d4146310 con 0x7f93d4145f30 2024-04-17T10:04:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.478+0000 7f93d9a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f93d40cce70 0x7f93d40cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33958/0 (socket says 172.21.15.32:33958) 2024-04-17T10:04:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.478+0000 7f93d9a00640 1 -- 172.21.15.32:0/2796178594 learned_addr learned my addr 172.21.15.32:0/2796178594 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.478+0000 7f93d9000640 1 --2- 172.21.15.32:0/2796178594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d4145f30 0x7f93d4071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.478+0000 7f93cbe00640 1 --2- 172.21.15.32:0/2796178594 >> v2:172.21.15.144:3300/0 conn(0x7f93d415d8d0 0x7f93d4072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.478+0000 7f93d3e00640 1 -- 172.21.15.32:0/2796178594 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2363888868 0 0) 0x7f93d40cdac0 con 0x7f93d40cce70 2024-04-17T10:04:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.478+0000 7f93d3e00640 1 -- 172.21.15.32:0/2796178594 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93b8003530 con 0x7f93d40cce70 2024-04-17T10:04:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.478+0000 7f93d9000640 1 -- 172.21.15.32:0/2796178594 >> v2:172.21.15.144:3300/0 conn(0x7f93d415d8d0 msgr2=0x7f93d4072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.478+0000 7f93d9000640 1 --2- 172.21.15.32:0/2796178594 >> v2:172.21.15.144:3300/0 conn(0x7f93d415d8d0 0x7f93d4072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.478+0000 7f93d9000640 1 -- 172.21.15.32:0/2796178594 >> v1:172.21.15.32:6789/0 conn(0x7f93d40cce70 legacy=0x7f93d40cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:41.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.478+0000 7f93d9000640 1 -- 172.21.15.32:0/2796178594 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d415cdf0 con 0x7f93d4145f30 2024-04-17T10:04:41.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.482+0000 7f93d9000640 1 --2- 172.21.15.32:0/2796178594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d4145f30 0x7f93d4071df0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f93c40097c0 tx=0x7f93c401ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=944e9cd798fe0076 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:41.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.482+0000 7f93d3e00640 1 -- 172.21.15.32:0/2796178594 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c40b5070 con 0x7f93d4145f30 2024-04-17T10:04:41.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.482+0000 7f93d3e00640 1 -- 172.21.15.32:0/2796178594 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93c40a8990 con 0x7f93d4145f30 2024-04-17T10:04:41.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.482+0000 7f93d3e00640 1 -- 172.21.15.32:0/2796178594 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c40a8d60 con 0x7f93d4145f30 2024-04-17T10:04:41.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.482+0000 7f93dba00640 1 -- 172.21.15.32:0/2796178594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d4145f30 msgr2=0x7f93d4071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:41.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.482+0000 7f93dba00640 1 --2- 172.21.15.32:0/2796178594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d4145f30 0x7f93d4071df0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f93c40097c0 tx=0x7f93c401ec90 comp rx=0 tx=0).stop 2024-04-17T10:04:41.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.482+0000 7f93dba00640 1 -- 172.21.15.32:0/2796178594 shutdown_connections 2024-04-17T10:04:41.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.482+0000 7f93dba00640 1 --2- 172.21.15.32:0/2796178594 >> v2:172.21.15.144:3300/0 conn(0x7f93d415d8d0 0x7f93d4072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:41.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.482+0000 7f93dba00640 1 --2- 172.21.15.32:0/2796178594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d4145f30 0x7f93d4071df0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:41.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.482+0000 7f93dba00640 1 -- 172.21.15.32:0/2796178594 >> 172.21.15.32:0/2796178594 conn(0x7f93d406ba30 msgr2=0x7f93d406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:41.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.482+0000 7f93dba00640 1 -- 172.21.15.32:0/2796178594 shutdown_connections 2024-04-17T10:04:41.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.482+0000 7f93dba00640 1 -- 172.21.15.32:0/2796178594 wait complete. 2024-04-17T10:04:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.482+0000 7f93dba00640 1 Processor -- start 2024-04-17T10:04:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93dba00640 1 -- start start 2024-04-17T10:04:41.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93dba00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d40cce70 0x7f93d41f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:41.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93dba00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d4145f30 0x7f93d415e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:41.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93dba00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93d415d8d0 0x7f93d415ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:41.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93dba00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f93d415cb90 con 0x7f93d4145f30 2024-04-17T10:04:41.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93dba00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f93d415d440 con 0x7f93d415d8d0 2024-04-17T10:04:41.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93dba00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f93d41601d0 con 0x7f93d40cce70 2024-04-17T10:04:41.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93cbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d4145f30 0x7f93d415e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:41.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93d9000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d40cce70 0x7f93d41f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:41.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93cbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d4145f30 0x7f93d415e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44078/0 (socket says 172.21.15.32:44078) 2024-04-17T10:04:41.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93cbe00640 1 -- 172.21.15.32:0/2983676049 learned_addr learned my addr 172.21.15.32:0/2983676049 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:41.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93d9a00640 1 --2- 172.21.15.32:0/2983676049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93d415d8d0 0x7f93d415ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:41.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93cbe00640 1 -- 172.21.15.32:0/2983676049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d40cce70 msgr2=0x7f93d41f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:41.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93cbe00640 1 --2- 172.21.15.32:0/2983676049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d40cce70 0x7f93d41f62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:41.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93cbe00640 1 -- 172.21.15.32:0/2983676049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93d415d8d0 msgr2=0x7f93d415ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:41.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93cbe00640 1 --2- 172.21.15.32:0/2983676049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93d415d8d0 0x7f93d415ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:41.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93cbe00640 1 -- 172.21.15.32:0/2983676049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d41f67e0 con 0x7f93d4145f30 2024-04-17T10:04:41.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93d9000640 1 --2- 172.21.15.32:0/2983676049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d40cce70 0x7f93d41f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:04:41.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93d9a00640 1 --2- 172.21.15.32:0/2983676049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93d415d8d0 0x7f93d415ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:04:41.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93cbe00640 1 --2- 172.21.15.32:0/2983676049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d4145f30 0x7f93d415e940 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f93bc00e870 tx=0x7f93bc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:41.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93d1600640 1 -- 172.21.15.32:0/2983676049 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc017070 con 0x7f93d4145f30 2024-04-17T10:04:41.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93d1600640 1 -- 172.21.15.32:0/2983676049 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93bc00c940 con 0x7f93d4145f30 2024-04-17T10:04:41.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93d1600640 1 -- 172.21.15.32:0/2983676049 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc00ccf0 con 0x7f93d4145f30 2024-04-17T10:04:41.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93dba00640 1 -- 172.21.15.32:0/2983676049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d40c9800 con 0x7f93d4145f30 2024-04-17T10:04:41.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93dba00640 1 -- 172.21.15.32:0/2983676049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d41f2770 con 0x7f93d4145f30 2024-04-17T10:04:41.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.486+0000 7f93dba00640 1 -- 172.21.15.32:0/2983676049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d41f22a0 con 0x7f93d4145f30 2024-04-17T10:04:41.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.490+0000 7f93d1600640 1 -- 172.21.15.32:0/2983676049 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f93bc01a070 con 0x7f93d4145f30 2024-04-17T10:04:41.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.490+0000 7f93d1600640 1 --2- 172.21.15.32:0/2983676049 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f93b0068380 0x7f93b006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:41.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.490+0000 7f93d1600640 1 -- 172.21.15.32:0/2983676049 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f93bc0865a0 con 0x7f93d4145f30 2024-04-17T10:04:41.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.490+0000 7f93d9000640 1 --2- 172.21.15.32:0/2983676049 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f93b0068380 0x7f93b006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:41.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.490+0000 7f93d9000640 1 --2- 172.21.15.32:0/2983676049 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f93b0068380 0x7f93b006a840 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f93c40b2000 tx=0x7f93c4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:41.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.494+0000 7f93d1600640 1 -- 172.21.15.32:0/2983676049 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f93d41f67e0 con 0x7f93d4145f30 2024-04-17T10:04:41.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.714+0000 7f93dba00640 1 -- 172.21.15.32:0/2983676049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f93d4068a90 con 0x7f93d4145f30 2024-04-17T10:04:41.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.942+0000 7f93d1600640 1 -- 172.21.15.32:0/2983676049 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v540) ==== 144+0+0 (secure 0 0 0) 0x7f93bc056750 con 0x7f93d4145f30 2024-04-17T10:04:41.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.946+0000 7f93dba00640 1 -- 172.21.15.32:0/2983676049 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f93b0068380 msgr2=0x7f93b006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:41.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.946+0000 7f93dba00640 1 --2- 172.21.15.32:0/2983676049 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f93b0068380 0x7f93b006a840 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f93c40b2000 tx=0x7f93c4008040 comp rx=0 tx=0).stop 2024-04-17T10:04:41.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.946+0000 7f93dba00640 1 -- 172.21.15.32:0/2983676049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d4145f30 msgr2=0x7f93d415e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:41.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.946+0000 7f93dba00640 1 --2- 172.21.15.32:0/2983676049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d4145f30 0x7f93d415e940 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f93bc00e870 tx=0x7f93bc00ec70 comp rx=0 tx=0).stop 2024-04-17T10:04:41.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.946+0000 7f93dba00640 1 -- 172.21.15.32:0/2983676049 shutdown_connections 2024-04-17T10:04:41.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.946+0000 7f93dba00640 1 --2- 172.21.15.32:0/2983676049 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f93b0068380 0x7f93b006a840 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:41.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.946+0000 7f93dba00640 1 --2- 172.21.15.32:0/2983676049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93d415d8d0 0x7f93d415ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:41.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.946+0000 7f93dba00640 1 --2- 172.21.15.32:0/2983676049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d4145f30 0x7f93d415e940 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:41.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.946+0000 7f93dba00640 1 --2- 172.21.15.32:0/2983676049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d40cce70 0x7f93d41f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:41.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.946+0000 7f93dba00640 1 -- 172.21.15.32:0/2983676049 >> 172.21.15.32:0/2983676049 conn(0x7f93d406ba30 msgr2=0x7f93d41437c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:41.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.946+0000 7f93dba00640 1 -- 172.21.15.32:0/2983676049 shutdown_connections 2024-04-17T10:04:41.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:41.946+0000 7f93dba00640 1 -- 172.21.15.32:0/2983676049 wait complete. 2024-04-17T10:04:41.961 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-17T10:04:41.962 INFO:tasks.cephfs_test_runner: 2024-04-17T10:04:41.962 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-17T10:04:41.962 INFO:tasks.cephfs_test_runner:Ran 7 tests in 657.072s 2024-04-17T10:04:41.962 INFO:tasks.cephfs_test_runner: 2024-04-17T10:04:41.962 INFO:tasks.cephfs_test_runner:OK 2024-04-17T10:04:41.962 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-17T10:04:41.977 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-17T10:04:41.996 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-17T10:04:41.996 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:04:41.997 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:04:42.005 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:04:42.006 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-04-17T10:04:42.006 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:04:42.055 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353396k,nr_inodes=4088349,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2894 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-17T10:04:42.055 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T10:04:42.055 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:04:42.064 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:04:42.064 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-17T10:04:42.064 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T10:04:42.115 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5215 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-17T10:04:42.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:42.115 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-04-17T10:04:42.124 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T10:04:42.124 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T10:04:42.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:42.125 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T10:04:42.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:42.188 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T10:04:42.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:42.257 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2024-04-17T10:04:42.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:42.422 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2024-04-17T10:04:42.432 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-17T10:04:42.432 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-17T10:04:42.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:42.432 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T10:04:42.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:42.503 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-17T10:04:42.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T10:04:42.565 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link delete ceph-brx 2024-04-17T10:04:42.726 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-17T10:04:42.735 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-17T10:04:42.736 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:04:42.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.882+0000 7fa139c00640 1 Processor -- start 2024-04-17T10:04:42.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.882+0000 7fa139c00640 1 -- start start 2024-04-17T10:04:42.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.882+0000 7fa139c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa13415eba0 0x7fa13406cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:42.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.882+0000 7fa139c00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa13409b3c0 0x7fa13406d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:42.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.882+0000 7fa139c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa13406dea0 con 0x7fa13406da90 2024-04-17T10:04:42.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.882+0000 7fa139c00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa13409bd60 con 0x7fa13409b3c0 2024-04-17T10:04:42.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.882+0000 7fa139c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa1340b1e40 con 0x7fa13415eba0 2024-04-17T10:04:42.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.882+0000 7fa12be00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa13409b3c0 0x7fa13406d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:42.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.882+0000 7fa133400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa13406da90 0x7fa1340b2ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33960/0 (socket says 172.21.15.32:33960) 2024-04-17T10:04:42.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.882+0000 7fa133400640 1 -- 172.21.15.32:0/1156994871 learned_addr learned my addr 172.21.15.32:0/1156994871 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:42.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.882+0000 7fa132a00640 1 --2- 172.21.15.32:0/1156994871 >> v2:172.21.15.144:3300/0 conn(0x7fa13415eba0 0x7fa13406cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:42.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.882+0000 7fa12be00640 1 -- 172.21.15.32:0/1156994871 >> v2:172.21.15.144:3300/0 conn(0x7fa13415eba0 msgr2=0x7fa13406cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:42.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.882+0000 7fa12be00640 1 --2- 172.21.15.32:0/1156994871 >> v2:172.21.15.144:3300/0 conn(0x7fa13415eba0 0x7fa13406cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:42.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.882+0000 7fa12be00640 1 -- 172.21.15.32:0/1156994871 >> v1:172.21.15.32:6789/0 conn(0x7fa13406da90 legacy=0x7fa1340b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:42.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.882+0000 7fa12be00640 1 -- 172.21.15.32:0/1156994871 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1340b2120 con 0x7fa13409b3c0 2024-04-17T10:04:42.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.882+0000 7fa132000640 1 -- 172.21.15.32:0/1156994871 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2533617330 0 0) 0x7fa12c0014a0 con 0x7fa13406da90 2024-04-17T10:04:42.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.886+0000 7fa12be00640 1 --2- 172.21.15.32:0/1156994871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13409b3c0 0x7fa13406d550 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fa1340731e0 tx=0x7fa11c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66f63b1bc39afc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:42.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.886+0000 7fa132000640 1 -- 172.21.15.32:0/1156994871 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c0b1070 con 0x7fa13409b3c0 2024-04-17T10:04:42.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.886+0000 7fa132000640 1 -- 172.21.15.32:0/1156994871 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa11c0ac620 con 0x7fa13409b3c0 2024-04-17T10:04:42.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.886+0000 7fa132000640 1 -- 172.21.15.32:0/1156994871 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c0ac9f0 con 0x7fa13409b3c0 2024-04-17T10:04:42.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.886+0000 7fa139c00640 1 -- 172.21.15.32:0/1156994871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13409b3c0 msgr2=0x7fa13406d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:42.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.886+0000 7fa139c00640 1 --2- 172.21.15.32:0/1156994871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13409b3c0 0x7fa13406d550 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fa1340731e0 tx=0x7fa11c01c3c0 comp rx=0 tx=0).stop 2024-04-17T10:04:42.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.886+0000 7fa139c00640 1 -- 172.21.15.32:0/1156994871 shutdown_connections 2024-04-17T10:04:42.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.886+0000 7fa139c00640 1 --2- 172.21.15.32:0/1156994871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13409b3c0 0x7fa13406d550 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:42.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.886+0000 7fa139c00640 1 --2- 172.21.15.32:0/1156994871 >> v2:172.21.15.144:3300/0 conn(0x7fa13415eba0 0x7fa13406cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:42.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.886+0000 7fa139c00640 1 -- 172.21.15.32:0/1156994871 >> 172.21.15.32:0/1156994871 conn(0x7fa1340b3fd0 msgr2=0x7fa1340b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:42.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.886+0000 7fa139c00640 1 -- 172.21.15.32:0/1156994871 shutdown_connections 2024-04-17T10:04:42.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.886+0000 7fa139c00640 1 -- 172.21.15.32:0/1156994871 wait complete. 2024-04-17T10:04:42.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa139c00640 1 Processor -- start 2024-04-17T10:04:42.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa139c00640 1 -- start start 2024-04-17T10:04:42.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa139c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa13406da90 0x7fa134159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:42.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa139c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13409b3c0 0x7fa1340b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:42.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa139c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa13415eba0 0x7fa1340b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:42.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa139c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa1340cccb0 con 0x7fa13409b3c0 2024-04-17T10:04:42.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa139c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa1340ccb30 con 0x7fa13415eba0 2024-04-17T10:04:42.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa139c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa13416b580 con 0x7fa13406da90 2024-04-17T10:04:42.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa12be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13409b3c0 0x7fa1340b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:42.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa12be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13409b3c0 0x7fa1340b63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44104/0 (socket says 172.21.15.32:44104) 2024-04-17T10:04:42.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa12be00640 1 -- 172.21.15.32:0/1650867888 learned_addr learned my addr 172.21.15.32:0/1650867888 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:42.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa133400640 1 --2- 172.21.15.32:0/1650867888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa13415eba0 0x7fa1340b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:42.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa132a00640 1 --2- 172.21.15.32:0/1650867888 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa13406da90 0x7fa134159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:42.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa12be00640 1 -- 172.21.15.32:0/1650867888 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa13406da90 msgr2=0x7fa134159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:42.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa12be00640 1 --2- 172.21.15.32:0/1650867888 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa13406da90 0x7fa134159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:42.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa12be00640 1 -- 172.21.15.32:0/1650867888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa13415eba0 msgr2=0x7fa1340b7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:42.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa12be00640 1 --2- 172.21.15.32:0/1650867888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa13415eba0 0x7fa1340b7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:42.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa12be00640 1 -- 172.21.15.32:0/1650867888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1340b6a40 con 0x7fa13409b3c0 2024-04-17T10:04:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa12be00640 1 --2- 172.21.15.32:0/1650867888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13409b3c0 0x7fa1340b63b0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0af040 tx=0x7fa11c019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa12aa00640 1 -- 172.21.15.32:0/1650867888 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c0b9070 con 0x7fa13409b3c0 2024-04-17T10:04:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa12aa00640 1 -- 172.21.15.32:0/1650867888 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa11c0b5410 con 0x7fa13409b3c0 2024-04-17T10:04:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa139c00640 1 -- 172.21.15.32:0/1650867888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1340b6d20 con 0x7fa13409b3c0 2024-04-17T10:04:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa12aa00640 1 -- 172.21.15.32:0/1650867888 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c0b1040 con 0x7fa13409b3c0 2024-04-17T10:04:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa139c00640 1 -- 172.21.15.32:0/1650867888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1340b72e0 con 0x7fa13409b3c0 2024-04-17T10:04:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.890+0000 7fa139c00640 1 -- 172.21.15.32:0/1650867888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa134068a90 con 0x7fa13409b3c0 2024-04-17T10:04:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.894+0000 7fa12aa00640 1 -- 172.21.15.32:0/1650867888 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa11c0bc070 con 0x7fa13409b3c0 2024-04-17T10:04:42.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.894+0000 7fa12aa00640 1 --2- 172.21.15.32:0/1650867888 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa1100683b0 0x7fa11006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:42.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.894+0000 7fa12aa00640 1 -- 172.21.15.32:0/1650867888 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7fa11c120680 con 0x7fa13409b3c0 2024-04-17T10:04:42.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.894+0000 7fa132a00640 1 --2- 172.21.15.32:0/1650867888 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa1100683b0 0x7fa11006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:42.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.894+0000 7fa132a00640 1 --2- 172.21.15.32:0/1650867888 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa1100683b0 0x7fa11006a870 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa12400b930 tx=0x7fa124008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:42.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:42.898+0000 7fa12aa00640 1 -- 172.21.15.32:0/1650867888 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa11c016070 con 0x7fa13409b3c0 2024-04-17T10:04:43.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.062+0000 7fa139c00640 1 -- 172.21.15.32:0/1650867888 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa1340b4790 con 0x7fa1100683b0 2024-04-17T10:04:43.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.062+0000 7fa12aa00640 1 -- 172.21.15.32:0/1650867888 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13508 (secure 0 0 0) 0x7fa1340b4790 con 0x7fa1100683b0 2024-04-17T10:04:43.058 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:43.058 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T10:04:43.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.062+0000 7fa139c00640 1 -- 172.21.15.32:0/1650867888 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa1100683b0 msgr2=0x7fa11006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:43.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.062+0000 7fa139c00640 1 --2- 172.21.15.32:0/1650867888 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa1100683b0 0x7fa11006a870 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa12400b930 tx=0x7fa124008040 comp rx=0 tx=0).stop 2024-04-17T10:04:43.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.062+0000 7fa139c00640 1 -- 172.21.15.32:0/1650867888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13409b3c0 msgr2=0x7fa1340b63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:43.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.062+0000 7fa139c00640 1 --2- 172.21.15.32:0/1650867888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13409b3c0 0x7fa1340b63b0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0af040 tx=0x7fa11c019160 comp rx=0 tx=0).stop 2024-04-17T10:04:43.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.066+0000 7fa139c00640 1 -- 172.21.15.32:0/1650867888 shutdown_connections 2024-04-17T10:04:43.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.066+0000 7fa139c00640 1 --2- 172.21.15.32:0/1650867888 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa1100683b0 0x7fa11006a870 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.066+0000 7fa139c00640 1 --2- 172.21.15.32:0/1650867888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa13415eba0 0x7fa1340b7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.066+0000 7fa139c00640 1 --2- 172.21.15.32:0/1650867888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa13409b3c0 0x7fa1340b63b0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.066+0000 7fa139c00640 1 --2- 172.21.15.32:0/1650867888 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa13406da90 0x7fa134159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.066+0000 7fa139c00640 1 -- 172.21.15.32:0/1650867888 >> 172.21.15.32:0/1650867888 conn(0x7fa1340b3fd0 msgr2=0x7fa1340b0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:43.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.066+0000 7fa139c00640 1 -- 172.21.15.32:0/1650867888 shutdown_connections 2024-04-17T10:04:43.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.066+0000 7fa139c00640 1 -- 172.21.15.32:0/1650867888 wait complete. 2024-04-17T10:04:43.078 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-04-17T10:04:41.446803+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":286436,"kb_used_data":28676,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754688284,"statfs":{"total":773094113280,"available":772800802816,"internally_reserved":0,"allocated":29364224,"data_stored":22894129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392376,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-262,"num_write":-178,"num_write_kb":-8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001292"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":348,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-17T10:04:40.261907+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T10:04:40.261907+0000","last_peered":"2024-04-17T10:04:40.261907+0000","last_clean":"2024-04-17T10:04:40.261907+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T10:04:40.261907+0000","last_undegraded":"2024-04-17T10:04:40.261907+0000","last_fullsized":"2024-04-17T10:04:40.261907+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509583,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":2011136,"data_stored":1204811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509584,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36672,"kb_used_data":5492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335168,"statfs":{"total":96636764160,"available":96599212032,"internally_reserved":0,"allocated":5623808,"data_stored":4814371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509583,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36676,"kb_used_data":5496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335164,"statfs":{"total":96636764160,"available":96599207936,"internally_reserved":0,"allocated":5627904,"data_stored":4814376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509583,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40832,"kb_used_data":5492,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331008,"statfs":{"total":96636764160,"available":96594952192,"internally_reserved":0,"allocated":5623808,"data_stored":4811159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509584,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1429504,"data_stored":620065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509583,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36660,"kb_used_data":5480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335180,"statfs":{"total":96636764160,"available":96599224320,"internally_reserved":0,"allocated":5611520,"data_stored":4808255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542287,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1409024,"data_stored":613827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542287,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37320,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334520,"statfs":{"total":96636764160,"available":96598548480,"internally_reserved":0,"allocated":2027520,"data_stored":1207265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:04:43.079 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:04:43.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.226+0000 7fa0ca200640 1 Processor -- start 2024-04-17T10:04:43.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.226+0000 7fa0ca200640 1 -- start start 2024-04-17T10:04:43.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.226+0000 7fa0ca200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c415e9c0 0x7fa0c40d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:43.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.226+0000 7fa0ca200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0c40cdf00 con 0x7fa0c40d1ef0 2024-04-17T10:04:43.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.226+0000 7fa0ca200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0c40ce1e0 con 0x7fa0c40cffe0 2024-04-17T10:04:43.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.226+0000 7fa0ca200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa0c40cdcb0 con 0x7fa0c415e9c0 2024-04-17T10:04:43.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.226+0000 7fa0c3e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa0c40d1ef0 0x7fa0c40cf8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33972/0 (socket says 172.21.15.32:33972) 2024-04-17T10:04:43.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.226+0000 7fa0c3e00640 1 -- 172.21.15.32:0/3012854689 learned_addr learned my addr 172.21.15.32:0/3012854689 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:43.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.226+0000 7fa0c8e00640 1 --2- 172.21.15.32:0/3012854689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c415e9c0 0x7fa0c40d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:43.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.226+0000 7fa0c3400640 1 -- 172.21.15.32:0/3012854689 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 47675097 0 0) 0x7fa0c40cdf00 con 0x7fa0c40d1ef0 2024-04-17T10:04:43.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.226+0000 7fa0c3400640 1 -- 172.21.15.32:0/3012854689 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0ac003530 con 0x7fa0c40d1ef0 2024-04-17T10:04:43.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.226+0000 7fa0c3400640 1 -- 172.21.15.32:0/3012854689 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3320546989 0 0) 0x7fa0c40ce1e0 con 0x7fa0c40cffe0 2024-04-17T10:04:43.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.226+0000 7fa0c3400640 1 -- 172.21.15.32:0/3012854689 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0c40cdf00 con 0x7fa0c40cffe0 2024-04-17T10:04:43.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.226+0000 7fa0c8e00640 1 -- 172.21.15.32:0/3012854689 >> v1:172.21.15.144:6789/0 conn(0x7fa0c40cffe0 legacy=0x7fa0c406f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:43.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.226+0000 7fa0c8e00640 1 -- 172.21.15.32:0/3012854689 >> v1:172.21.15.32:6789/0 conn(0x7fa0c40d1ef0 legacy=0x7fa0c40cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:43.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.226+0000 7fa0c8e00640 1 -- 172.21.15.32:0/3012854689 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c40ce4c0 con 0x7fa0c415e9c0 2024-04-17T10:04:43.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.230+0000 7fa0c8e00640 1 --2- 172.21.15.32:0/3012854689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c415e9c0 0x7fa0c40d19b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80097f0 tx=0x7fa0b801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b59d79321498b51 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:43.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.230+0000 7fa0c3400640 1 -- 172.21.15.32:0/3012854689 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b80b0070 con 0x7fa0c415e9c0 2024-04-17T10:04:43.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.230+0000 7fa0c3400640 1 -- 172.21.15.32:0/3012854689 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0b80ab600 con 0x7fa0c415e9c0 2024-04-17T10:04:43.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.230+0000 7fa0c3400640 1 -- 172.21.15.32:0/3012854689 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b80ab9d0 con 0x7fa0c415e9c0 2024-04-17T10:04:43.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.230+0000 7fa0ca200640 1 -- 172.21.15.32:0/3012854689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c415e9c0 msgr2=0x7fa0c40d19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:43.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.230+0000 7fa0ca200640 1 --2- 172.21.15.32:0/3012854689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c415e9c0 0x7fa0c40d19b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80097f0 tx=0x7fa0b801c4e0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.230+0000 7fa0ca200640 1 -- 172.21.15.32:0/3012854689 shutdown_connections 2024-04-17T10:04:43.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.230+0000 7fa0ca200640 1 --2- 172.21.15.32:0/3012854689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c415e9c0 0x7fa0c40d19b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.230+0000 7fa0ca200640 1 -- 172.21.15.32:0/3012854689 >> 172.21.15.32:0/3012854689 conn(0x7fa0c40b67b0 msgr2=0x7fa0c40b6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:43.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.230+0000 7fa0ca200640 1 -- 172.21.15.32:0/3012854689 shutdown_connections 2024-04-17T10:04:43.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.230+0000 7fa0ca200640 1 -- 172.21.15.32:0/3012854689 wait complete. 2024-04-17T10:04:43.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.230+0000 7fa0ca200640 1 Processor -- start 2024-04-17T10:04:43.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0ca200640 1 -- start start 2024-04-17T10:04:43.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0ca200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c40cffe0 0x7fa0c4153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:43.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0ca200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c40d1ef0 0x7fa0c415ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:43.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0ca200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c415e9c0 0x7fa0c41480c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:43.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0ca200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c4070790 con 0x7fa0c40cffe0 2024-04-17T10:04:43.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0ca200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c4070610 con 0x7fa0c40d1ef0 2024-04-17T10:04:43.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0ca200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa0c40ceb10 con 0x7fa0c415e9c0 2024-04-17T10:04:43.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0c8e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c40cffe0 0x7fa0c4153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:43.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0c8e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c40cffe0 0x7fa0c4153360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44120/0 (socket says 172.21.15.32:44120) 2024-04-17T10:04:43.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0c8e00640 1 -- 172.21.15.32:0/3596137039 learned_addr learned my addr 172.21.15.32:0/3596137039 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:43.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0c3e00640 1 --2- 172.21.15.32:0/3596137039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c40d1ef0 0x7fa0c415ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:43.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0c9800640 1 --2- 172.21.15.32:0/3596137039 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c415e9c0 0x7fa0c41480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:43.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0c8e00640 1 -- 172.21.15.32:0/3596137039 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c415e9c0 msgr2=0x7fa0c41480c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:43.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0c8e00640 1 --2- 172.21.15.32:0/3596137039 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c415e9c0 0x7fa0c41480c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0c8e00640 1 -- 172.21.15.32:0/3596137039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c40d1ef0 msgr2=0x7fa0c415ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:43.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0c8e00640 1 --2- 172.21.15.32:0/3596137039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c40d1ef0 0x7fa0c415ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0c8e00640 1 -- 172.21.15.32:0/3596137039 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c415f950 con 0x7fa0c40cffe0 2024-04-17T10:04:43.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0c9800640 1 --2- 172.21.15.32:0/3596137039 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c415e9c0 0x7fa0c41480c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:43.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0c8e00640 1 --2- 172.21.15.32:0/3596137039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c40cffe0 0x7fa0c4153360 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80097f0 tx=0x7fa0b801ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:43.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0c0c00640 1 -- 172.21.15.32:0/3596137039 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b80b0050 con 0x7fa0c40cffe0 2024-04-17T10:04:43.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0c0c00640 1 -- 172.21.15.32:0/3596137039 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0b80bc070 con 0x7fa0c40cffe0 2024-04-17T10:04:43.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0c0c00640 1 -- 172.21.15.32:0/3596137039 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8006e20 con 0x7fa0c40cffe0 2024-04-17T10:04:43.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0ca200640 1 -- 172.21.15.32:0/3596137039 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c415fc30 con 0x7fa0c40cffe0 2024-04-17T10:04:43.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0ca200640 1 -- 172.21.15.32:0/3596137039 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c41601f0 con 0x7fa0c40cffe0 2024-04-17T10:04:43.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.234+0000 7fa0ca200640 1 -- 172.21.15.32:0/3596137039 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c404fd00 con 0x7fa0c40cffe0 2024-04-17T10:04:43.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.238+0000 7fa0c0c00640 1 -- 172.21.15.32:0/3596137039 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa0b80b5870 con 0x7fa0c40cffe0 2024-04-17T10:04:43.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.238+0000 7fa0c0c00640 1 --2- 172.21.15.32:0/3596137039 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa0a4068380 0x7fa0a406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:43.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.238+0000 7fa0c0c00640 1 -- 172.21.15.32:0/3596137039 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7fa0b8120de0 con 0x7fa0c40cffe0 2024-04-17T10:04:43.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.238+0000 7fa0c3e00640 1 --2- 172.21.15.32:0/3596137039 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa0a4068380 0x7fa0a406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:43.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.238+0000 7fa0c3e00640 1 --2- 172.21.15.32:0/3596137039 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa0a4068380 0x7fa0a406a840 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4156d10 tx=0x7fa0b0007d80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:43.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.242+0000 7fa0c0c00640 1 -- 172.21.15.32:0/3596137039 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa0b80160d0 con 0x7fa0c40cffe0 2024-04-17T10:04:43.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.406+0000 7fa0ca200640 1 -- 172.21.15.32:0/3596137039 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa0c40d1c80 con 0x7fa0a4068380 2024-04-17T10:04:43.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.406+0000 7fa0c0c00640 1 -- 172.21.15.32:0/3596137039 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13508 (secure 0 0 0) 0x7fa0c40d1c80 con 0x7fa0a4068380 2024-04-17T10:04:43.402 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:43.402 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T10:04:43.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.410+0000 7fa0ca200640 1 -- 172.21.15.32:0/3596137039 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa0a4068380 msgr2=0x7fa0a406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:43.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.410+0000 7fa0ca200640 1 --2- 172.21.15.32:0/3596137039 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa0a4068380 0x7fa0a406a840 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4156d10 tx=0x7fa0b0007d80 comp rx=0 tx=0).stop 2024-04-17T10:04:43.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.410+0000 7fa0ca200640 1 -- 172.21.15.32:0/3596137039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c40cffe0 msgr2=0x7fa0c4153360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:43.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.410+0000 7fa0ca200640 1 --2- 172.21.15.32:0/3596137039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c40cffe0 0x7fa0c4153360 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80097f0 tx=0x7fa0b801ad20 comp rx=0 tx=0).stop 2024-04-17T10:04:43.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.410+0000 7fa0ca200640 1 -- 172.21.15.32:0/3596137039 shutdown_connections 2024-04-17T10:04:43.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.410+0000 7fa0ca200640 1 --2- 172.21.15.32:0/3596137039 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa0a4068380 0x7fa0a406a840 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.410+0000 7fa0ca200640 1 --2- 172.21.15.32:0/3596137039 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0c415e9c0 0x7fa0c41480c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.410+0000 7fa0ca200640 1 --2- 172.21.15.32:0/3596137039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c40d1ef0 0x7fa0c415ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.410+0000 7fa0ca200640 1 --2- 172.21.15.32:0/3596137039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c40cffe0 0x7fa0c4153360 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.410+0000 7fa0ca200640 1 -- 172.21.15.32:0/3596137039 >> 172.21.15.32:0/3596137039 conn(0x7fa0c40b67b0 msgr2=0x7fa0c40d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:43.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.410+0000 7fa0ca200640 1 -- 172.21.15.32:0/3596137039 shutdown_connections 2024-04-17T10:04:43.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.410+0000 7fa0ca200640 1 -- 172.21.15.32:0/3596137039 wait complete. 2024-04-17T10:04:43.423 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-04-17T10:04:41.446803+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":286436,"kb_used_data":28676,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754688284,"statfs":{"total":773094113280,"available":772800802816,"internally_reserved":0,"allocated":29364224,"data_stored":22894129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392376,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-262,"num_write":-178,"num_write_kb":-8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001292"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":348,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-17T10:04:40.261907+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T10:04:40.261907+0000","last_peered":"2024-04-17T10:04:40.261907+0000","last_clean":"2024-04-17T10:04:40.261907+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T10:04:40.261907+0000","last_undegraded":"2024-04-17T10:04:40.261907+0000","last_fullsized":"2024-04-17T10:04:40.261907+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509583,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":2011136,"data_stored":1204811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509584,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36672,"kb_used_data":5492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335168,"statfs":{"total":96636764160,"available":96599212032,"internally_reserved":0,"allocated":5623808,"data_stored":4814371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509583,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36676,"kb_used_data":5496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335164,"statfs":{"total":96636764160,"available":96599207936,"internally_reserved":0,"allocated":5627904,"data_stored":4814376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509583,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40832,"kb_used_data":5492,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331008,"statfs":{"total":96636764160,"available":96594952192,"internally_reserved":0,"allocated":5623808,"data_stored":4811159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509584,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1429504,"data_stored":620065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509583,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36660,"kb_used_data":5480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335180,"statfs":{"total":96636764160,"available":96599224320,"internally_reserved":0,"allocated":5611520,"data_stored":4808255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542287,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1409024,"data_stored":613827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542287,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37320,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334520,"statfs":{"total":96636764160,"available":96598548480,"internally_reserved":0,"allocated":2027520,"data_stored":1207265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:04:43.424 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-17T10:04:43.424 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:04:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.570+0000 7f7c1f600640 1 Processor -- start 2024-04-17T10:04:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.570+0000 7f7c1f600640 1 -- start start 2024-04-17T10:04:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.570+0000 7f7c1f600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c180b2430 0x7f7c180b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.570+0000 7f7c1f600640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7c180b3900 0x7f7c1806ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.570+0000 7f7c1f600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c1806c800 con 0x7f7c1815d670 2024-04-17T10:04:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.570+0000 7f7c1f600640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f7c180b2d50 con 0x7f7c180b3900 2024-04-17T10:04:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.570+0000 7f7c1f600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7c1806bf70 con 0x7f7c180b2430 2024-04-17T10:04:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.570+0000 7f7c17e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7c180b3900 0x7f7c1806ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.570+0000 7f7c1cc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c180b2430 0x7f7c180b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.570+0000 7f7c1d600640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f7c1815d670 0x7f7c181602e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:36708/0 (socket says 172.21.15.32:36708) 2024-04-17T10:04:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.570+0000 7f7c1d600640 1 -- 172.21.15.32:0/3900425094 learned_addr learned my addr 172.21.15.32:0/3900425094 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.570+0000 7f7c17400640 1 -- 172.21.15.32:0/3900425094 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 41103448 0 0) 0x7f7c1806c800 con 0x7f7c1815d670 2024-04-17T10:04:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.570+0000 7f7c1cc00640 1 -- 172.21.15.32:0/3900425094 >> v2:172.21.15.32:3300/0 conn(0x7f7c180b3900 msgr2=0x7f7c1806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:43.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.570+0000 7f7c1cc00640 1 --2- 172.21.15.32:0/3900425094 >> v2:172.21.15.32:3300/0 conn(0x7f7c180b3900 0x7f7c1806ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.570+0000 7f7c1cc00640 1 -- 172.21.15.32:0/3900425094 >> v1:172.21.15.144:6789/0 conn(0x7f7c1815d670 legacy=0x7f7c181602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:43.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.570+0000 7f7c1cc00640 1 -- 172.21.15.32:0/3900425094 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1806c240 con 0x7f7c180b2430 2024-04-17T10:04:43.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.574+0000 7f7c1cc00640 1 --2- 172.21.15.32:0/3900425094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c180b2430 0x7f7c180b2810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c0097c0 tx=0x7f7c0c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a50af3684fc37f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:43.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.574+0000 7f7c17400640 1 -- 172.21.15.32:0/3900425094 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c0b0070 con 0x7f7c180b2430 2024-04-17T10:04:43.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.574+0000 7f7c17400640 1 -- 172.21.15.32:0/3900425094 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c0c0ab600 con 0x7f7c180b2430 2024-04-17T10:04:43.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.574+0000 7f7c17400640 1 -- 172.21.15.32:0/3900425094 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c0ab9d0 con 0x7f7c180b2430 2024-04-17T10:04:43.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.574+0000 7f7c1f600640 1 -- 172.21.15.32:0/3900425094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c180b2430 msgr2=0x7f7c180b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:43.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.574+0000 7f7c1f600640 1 --2- 172.21.15.32:0/3900425094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c180b2430 0x7f7c180b2810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c0097c0 tx=0x7f7c0c01c4a0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.574+0000 7f7c1f600640 1 -- 172.21.15.32:0/3900425094 shutdown_connections 2024-04-17T10:04:43.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.574+0000 7f7c1f600640 1 --2- 172.21.15.32:0/3900425094 >> v2:172.21.15.32:3300/0 conn(0x7f7c180b3900 0x7f7c1806ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.574+0000 7f7c1f600640 1 --2- 172.21.15.32:0/3900425094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c180b2430 0x7f7c180b2810 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.574+0000 7f7c1f600640 1 -- 172.21.15.32:0/3900425094 >> 172.21.15.32:0/3900425094 conn(0x7f7c1809a9f0 msgr2=0x7f7c180b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:43.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.574+0000 7f7c1f600640 1 -- 172.21.15.32:0/3900425094 shutdown_connections 2024-04-17T10:04:43.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.574+0000 7f7c1f600640 1 -- 172.21.15.32:0/3900425094 wait complete. 2024-04-17T10:04:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c1f600640 1 Processor -- start 2024-04-17T10:04:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c1f600640 1 -- start start 2024-04-17T10:04:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c1f600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c180b2430 0x7f7c180aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c1f600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c180b3900 0x7f7c181fae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c1f600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c1815d670 0x7f7c181669a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c1f600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7c180cdcb0 con 0x7f7c180b2430 2024-04-17T10:04:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c1f600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7c180cdb30 con 0x7f7c180b3900 2024-04-17T10:04:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c1f600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7c1815dac0 con 0x7f7c1815d670 2024-04-17T10:04:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c1cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c180b2430 0x7f7c180aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c1cc00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c180b2430 0x7f7c180aed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44138/0 (socket says 172.21.15.32:44138) 2024-04-17T10:04:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c1cc00640 1 -- 172.21.15.32:0/1631609988 learned_addr learned my addr 172.21.15.32:0/1631609988 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c1d600640 1 --2- 172.21.15.32:0/1631609988 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c1815d670 0x7f7c181669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c17e00640 1 --2- 172.21.15.32:0/1631609988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c180b3900 0x7f7c181fae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c1cc00640 1 -- 172.21.15.32:0/1631609988 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c1815d670 msgr2=0x7f7c181669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c1cc00640 1 --2- 172.21.15.32:0/1631609988 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c1815d670 0x7f7c181669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c1cc00640 1 -- 172.21.15.32:0/1631609988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c180b3900 msgr2=0x7f7c181fae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c1cc00640 1 --2- 172.21.15.32:0/1631609988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c180b3900 0x7f7c181fae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c1cc00640 1 -- 172.21.15.32:0/1631609988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c181fb670 con 0x7f7c180b2430 2024-04-17T10:04:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c1cc00640 1 --2- 172.21.15.32:0/1631609988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c180b2430 0x7f7c180aed40 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c01a0d0 tx=0x7f7c0c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c14c00640 1 -- 172.21.15.32:0/1631609988 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c0b0050 con 0x7f7c180b2430 2024-04-17T10:04:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c14c00640 1 -- 172.21.15.32:0/1631609988 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c0c0bc070 con 0x7f7c180b2430 2024-04-17T10:04:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c14c00640 1 -- 172.21.15.32:0/1631609988 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c0b5830 con 0x7f7c180b2430 2024-04-17T10:04:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c1f600640 1 -- 172.21.15.32:0/1631609988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c181fb950 con 0x7f7c180b2430 2024-04-17T10:04:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c1f600640 1 -- 172.21.15.32:0/1631609988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c181547d0 con 0x7f7c180b2430 2024-04-17T10:04:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.578+0000 7f7c1f600640 1 -- 172.21.15.32:0/1631609988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c18068a90 con 0x7f7c180b2430 2024-04-17T10:04:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.582+0000 7f7c14c00640 1 -- 172.21.15.32:0/1631609988 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7c0c0b59d0 con 0x7f7c180b2430 2024-04-17T10:04:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.582+0000 7f7c14c00640 1 --2- 172.21.15.32:0/1631609988 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7bf80683b0 0x7f7bf806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.582+0000 7f7c14c00640 1 -- 172.21.15.32:0/1631609988 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f7c0c120f50 con 0x7f7c180b2430 2024-04-17T10:04:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.582+0000 7f7c17e00640 1 --2- 172.21.15.32:0/1631609988 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7bf80683b0 0x7f7bf806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:43.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.582+0000 7f7c17e00640 1 --2- 172.21.15.32:0/1631609988 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7bf80683b0 0x7f7bf806a870 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f7c18159d20 tx=0x7f7c04008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.586+0000 7f7c14c00640 1 -- 172.21.15.32:0/1631609988 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f7c0c016090 con 0x7f7c180b2430 2024-04-17T10:04:43.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.746+0000 7f7c1f600640 1 -- 172.21.15.32:0/1631609988 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7c180b7450 con 0x7f7bf80683b0 2024-04-17T10:04:43.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.750+0000 7f7c14c00640 1 -- 172.21.15.32:0/1631609988 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13508 (secure 0 0 0) 0x7f7c180b7450 con 0x7f7bf80683b0 2024-04-17T10:04:43.744 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:43.744 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T10:04:43.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.750+0000 7f7c1f600640 1 -- 172.21.15.32:0/1631609988 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7bf80683b0 msgr2=0x7f7bf806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:43.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.750+0000 7f7c1f600640 1 --2- 172.21.15.32:0/1631609988 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7bf80683b0 0x7f7bf806a870 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f7c18159d20 tx=0x7f7c04008040 comp rx=0 tx=0).stop 2024-04-17T10:04:43.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.750+0000 7f7c1f600640 1 -- 172.21.15.32:0/1631609988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c180b2430 msgr2=0x7f7c180aed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:43.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.750+0000 7f7c1f600640 1 --2- 172.21.15.32:0/1631609988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c180b2430 0x7f7c180aed40 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c01a0d0 tx=0x7f7c0c01ace0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.750+0000 7f7c1f600640 1 -- 172.21.15.32:0/1631609988 shutdown_connections 2024-04-17T10:04:43.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.750+0000 7f7c1f600640 1 --2- 172.21.15.32:0/1631609988 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7bf80683b0 0x7f7bf806a870 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.750+0000 7f7c1f600640 1 --2- 172.21.15.32:0/1631609988 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c1815d670 0x7f7c181669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.750+0000 7f7c1f600640 1 --2- 172.21.15.32:0/1631609988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c180b3900 0x7f7c181fae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.750+0000 7f7c1f600640 1 --2- 172.21.15.32:0/1631609988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c180b2430 0x7f7c180aed40 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.750+0000 7f7c1f600640 1 -- 172.21.15.32:0/1631609988 >> 172.21.15.32:0/1631609988 conn(0x7f7c1809a9f0 msgr2=0x7f7c18070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:43.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.750+0000 7f7c1f600640 1 -- 172.21.15.32:0/1631609988 shutdown_connections 2024-04-17T10:04:43.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.750+0000 7f7c1f600640 1 -- 172.21.15.32:0/1631609988 wait complete. 2024-04-17T10:04:43.765 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-04-17T10:04:43.447069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":286436,"kb_used_data":28676,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754688284,"statfs":{"total":773094113280,"available":772800802816,"internally_reserved":0,"allocated":29364224,"data_stored":22894129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392376,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-262,"num_write":-178,"num_write_kb":-8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001071"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":348,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-17T10:04:40.261907+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T10:04:40.261907+0000","last_peered":"2024-04-17T10:04:40.261907+0000","last_clean":"2024-04-17T10:04:40.261907+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T10:04:40.261907+0000","last_undegraded":"2024-04-17T10:04:40.261907+0000","last_fullsized":"2024-04-17T10:04:40.261907+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509583,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":2011136,"data_stored":1204811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509584,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36672,"kb_used_data":5492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335168,"statfs":{"total":96636764160,"available":96599212032,"internally_reserved":0,"allocated":5623808,"data_stored":4814371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509583,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36676,"kb_used_data":5496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335164,"statfs":{"total":96636764160,"available":96599207936,"internally_reserved":0,"allocated":5627904,"data_stored":4814376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509583,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40832,"kb_used_data":5492,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331008,"statfs":{"total":96636764160,"available":96594952192,"internally_reserved":0,"allocated":5623808,"data_stored":4811159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509584,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1429504,"data_stored":620065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509583,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36660,"kb_used_data":5480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335180,"statfs":{"total":96636764160,"available":96599224320,"internally_reserved":0,"allocated":5611520,"data_stored":4808255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542287,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1409024,"data_stored":613827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542287,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37320,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334520,"statfs":{"total":96636764160,"available":96598548480,"internally_reserved":0,"allocated":2027520,"data_stored":1207265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:04:43.766 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T10:04:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.910+0000 7f7704e00640 1 Processor -- start 2024-04-17T10:04:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.910+0000 7f7704e00640 1 -- start start 2024-04-17T10:04:43.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.910+0000 7f7704e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770006ba30 0x7f77000d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:43.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.910+0000 7f7704e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f770006ce20 0x7f77000b5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:43.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.910+0000 7f7704e00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77000ce970 con 0x7f77000d1790 2024-04-17T10:04:43.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.910+0000 7f7704e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f77000cef30 con 0x7f770006ce20 2024-04-17T10:04:43.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.910+0000 7f7704e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f77000cec50 con 0x7f770006ba30 2024-04-17T10:04:43.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.910+0000 7f76f7e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f77000d1790 0x7f770006e370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33974/0 (socket says 172.21.15.32:33974) 2024-04-17T10:04:43.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.910+0000 7f76f7e00640 1 -- 172.21.15.32:0/283461538 learned_addr learned my addr 172.21.15.32:0/283461538 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:43.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.910+0000 7f76fde00640 1 --2- 172.21.15.32:0/283461538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770006ba30 0x7f77000d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:43.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.910+0000 7f76fe800640 1 --2- 172.21.15.32:0/283461538 >> v2:172.21.15.144:3300/0 conn(0x7f770006ce20 0x7f77000b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:43.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.910+0000 7f76fd400640 1 -- 172.21.15.32:0/283461538 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 920687327 0 0) 0x7f77000ce970 con 0x7f77000d1790 2024-04-17T10:04:43.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.910+0000 7f76fd400640 1 -- 172.21.15.32:0/283461538 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76e4003530 con 0x7f77000d1790 2024-04-17T10:04:43.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.910+0000 7f76fde00640 1 -- 172.21.15.32:0/283461538 >> v2:172.21.15.144:3300/0 conn(0x7f770006ce20 msgr2=0x7f77000b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:43.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.910+0000 7f76fde00640 1 --2- 172.21.15.32:0/283461538 >> v2:172.21.15.144:3300/0 conn(0x7f770006ce20 0x7f77000b5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.910+0000 7f76fde00640 1 -- 172.21.15.32:0/283461538 >> v1:172.21.15.32:6789/0 conn(0x7f77000d1790 legacy=0x7f770006e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:43.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.910+0000 7f76fde00640 1 -- 172.21.15.32:0/283461538 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77000b4800 con 0x7f770006ba30 2024-04-17T10:04:43.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.914+0000 7f76fde00640 1 --2- 172.21.15.32:0/283461538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770006ba30 0x7f77000d0360 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f76f00097c0 tx=0x7f76f001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceafc76e31e0544b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:43.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.914+0000 7f76fd400640 1 -- 172.21.15.32:0/283461538 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f00b0070 con 0x7f770006ba30 2024-04-17T10:04:43.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.914+0000 7f76fd400640 1 -- 172.21.15.32:0/283461538 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76f00ab600 con 0x7f770006ba30 2024-04-17T10:04:43.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.914+0000 7f76fd400640 1 -- 172.21.15.32:0/283461538 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f00ab9d0 con 0x7f770006ba30 2024-04-17T10:04:43.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.914+0000 7f7704e00640 1 -- 172.21.15.32:0/283461538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770006ba30 msgr2=0x7f77000d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:43.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.914+0000 7f7704e00640 1 --2- 172.21.15.32:0/283461538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770006ba30 0x7f77000d0360 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f76f00097c0 tx=0x7f76f001c470 comp rx=0 tx=0).stop 2024-04-17T10:04:43.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.914+0000 7f7704e00640 1 -- 172.21.15.32:0/283461538 shutdown_connections 2024-04-17T10:04:43.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.914+0000 7f7704e00640 1 --2- 172.21.15.32:0/283461538 >> v2:172.21.15.144:3300/0 conn(0x7f770006ce20 0x7f77000b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.914+0000 7f7704e00640 1 --2- 172.21.15.32:0/283461538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770006ba30 0x7f77000d0360 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.914+0000 7f7704e00640 1 -- 172.21.15.32:0/283461538 >> 172.21.15.32:0/283461538 conn(0x7f77000d0c00 msgr2=0x7f77000d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:43.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.914+0000 7f7704e00640 1 -- 172.21.15.32:0/283461538 shutdown_connections 2024-04-17T10:04:43.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.914+0000 7f7704e00640 1 -- 172.21.15.32:0/283461538 wait complete. 2024-04-17T10:04:43.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f7704e00640 1 Processor -- start 2024-04-17T10:04:43.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f7704e00640 1 -- start start 2024-04-17T10:04:43.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f7704e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770006ba30 0x7f77001f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:43.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f7704e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f770006ce20 0x7f7700158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:43.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f7704e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77000d1790 0x7f770015a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:43.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f7704e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f77000d0a20 con 0x7f77000d1790 2024-04-17T10:04:43.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f7704e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f77000d08a0 con 0x7f770006ce20 2024-04-17T10:04:43.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f7704e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7700144860 con 0x7f770006ba30 2024-04-17T10:04:43.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f76fe800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77000d1790 0x7f770015a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:43.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f76fe800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77000d1790 0x7f770015a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44144/0 (socket says 172.21.15.32:44144) 2024-04-17T10:04:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f76fe800640 1 -- 172.21.15.32:0/3137800895 learned_addr learned my addr 172.21.15.32:0/3137800895 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f76fde00640 1 --2- 172.21.15.32:0/3137800895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770006ba30 0x7f77001f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f76f7e00640 1 --2- 172.21.15.32:0/3137800895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f770006ce20 0x7f7700158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f76fe800640 1 -- 172.21.15.32:0/3137800895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770006ba30 msgr2=0x7f77001f80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f76fe800640 1 --2- 172.21.15.32:0/3137800895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770006ba30 0x7f77001f80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f76fe800640 1 -- 172.21.15.32:0/3137800895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f770006ce20 msgr2=0x7f7700158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f76fe800640 1 --2- 172.21.15.32:0/3137800895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f770006ce20 0x7f7700158d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f76fe800640 1 -- 172.21.15.32:0/3137800895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77000505b0 con 0x7f77000d1790 2024-04-17T10:04:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f76fe800640 1 --2- 172.21.15.32:0/3137800895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77000d1790 0x7f770015a820 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009dd0 tx=0x7f76f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f76f6000640 1 -- 172.21.15.32:0/3137800895 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f8018020 con 0x7f77000d1790 2024-04-17T10:04:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f76f6000640 1 -- 172.21.15.32:0/3137800895 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76f8014480 con 0x7f77000d1790 2024-04-17T10:04:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f76f6000640 1 -- 172.21.15.32:0/3137800895 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f8014870 con 0x7f77000d1790 2024-04-17T10:04:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f7704e00640 1 -- 172.21.15.32:0/3137800895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7700143960 con 0x7f77000d1790 2024-04-17T10:04:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.918+0000 7f7704e00640 1 -- 172.21.15.32:0/3137800895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7700143bd0 con 0x7f77000d1790 2024-04-17T10:04:43.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.922+0000 7f7704e00640 1 -- 172.21.15.32:0/3137800895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7700068a90 con 0x7f77000d1790 2024-04-17T10:04:43.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.922+0000 7f76f6000640 1 -- 172.21.15.32:0/3137800895 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f76f801b020 con 0x7f77000d1790 2024-04-17T10:04:43.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.922+0000 7f76f6000640 1 --2- 172.21.15.32:0/3137800895 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f76dc068380 0x7f76dc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:43.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.922+0000 7f76f6000640 1 -- 172.21.15.32:0/3137800895 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f76f8087130 con 0x7f77000d1790 2024-04-17T10:04:43.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.922+0000 7f76fde00640 1 --2- 172.21.15.32:0/3137800895 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f76dc068380 0x7f76dc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:43.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.922+0000 7f76fde00640 1 --2- 172.21.15.32:0/3137800895 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f76dc068380 0x7f76dc06a840 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f76f00ad000 tx=0x7f76f00a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:43.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:43.926+0000 7f76f6000640 1 -- 172.21.15.32:0/3137800895 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f76f8022070 con 0x7f77000d1790 2024-04-17T10:04:44.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:44.094+0000 7f7704e00640 1 -- 172.21.15.32:0/3137800895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77000b5250 con 0x7f77000d1790 2024-04-17T10:04:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:44.094+0000 7f76f6000640 1 -- 172.21.15.32:0/3137800895 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+13298 (secure 0 0 0) 0x7f76f8010040 con 0x7f77000d1790 2024-04-17T10:04:44.090 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:44.090 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":116,"fsid":"ff178df8-4c06-4367-a1e9-2e3a66d870d0","created":"2024-04-17T09:52:48.119114+0000","modified":"2024-04-17T10:04:40.938062+0000","last_up_change":"2024-04-17T09:52:59.700897+0000","last_in_change":"2024-04-17T09:52:50.671360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T09:52:59.309691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09a0453d-54e1-4672-89dc-466a9dc6a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6801","nonce":2392056658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6803","nonce":2392056658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6807","nonce":2392056658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":2392056658},{"type":"v1","addr":"172.21.15.32:6805","nonce":2392056658}]},"public_addr":"172.21.15.32:6801/2392056658","cluster_addr":"172.21.15.32:6803/2392056658","heartbeat_back_addr":"172.21.15.32:6807/2392056658","heartbeat_front_addr":"172.21.15.32:6805/2392056658","state":["exists","up"]},{"osd":1,"uuid":"0fb07290-ef1f-4ab6-99f8-12e4b399579f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6810","nonce":2454993455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6814","nonce":2454993455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6822","nonce":2454993455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2454993455},{"type":"v1","addr":"172.21.15.32:6818","nonce":2454993455}]},"public_addr":"172.21.15.32:6810/2454993455","cluster_addr":"172.21.15.32:6814/2454993455","heartbeat_back_addr":"172.21.15.32:6822/2454993455","heartbeat_front_addr":"172.21.15.32:6818/2454993455","state":["exists","up"]},{"osd":2,"uuid":"3e5c6a47-f005-4854-b351-a83b0bf1525a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6809","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6811","nonce":1954833462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6813","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6815","nonce":1954833462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6821","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6823","nonce":1954833462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6817","nonce":1954833462},{"type":"v1","addr":"172.21.15.32:6819","nonce":1954833462}]},"public_addr":"172.21.15.32:6811/1954833462","cluster_addr":"172.21.15.32:6815/1954833462","heartbeat_back_addr":"172.21.15.32:6823/1954833462","heartbeat_front_addr":"172.21.15.32:6819/1954833462","state":["exists","up"]},{"osd":3,"uuid":"0d5b4f04-83ef-4444-bb10-477c868e58a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6825","nonce":2182899298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6827","nonce":2182899298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6831","nonce":2182899298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":2182899298},{"type":"v1","addr":"172.21.15.32:6829","nonce":2182899298}]},"public_addr":"172.21.15.32:6825/2182899298","cluster_addr":"172.21.15.32:6827/2182899298","heartbeat_back_addr":"172.21.15.32:6831/2182899298","heartbeat_front_addr":"172.21.15.32:6829/2182899298","state":["exists","up"]},{"osd":4,"uuid":"db4af439-9a47-4c14-a6dc-6c4eef51ea45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6809","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6811","nonce":4076371964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6813","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6815","nonce":4076371964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6823","nonce":4076371964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6817","nonce":4076371964},{"type":"v1","addr":"172.21.15.144:6820","nonce":4076371964}]},"public_addr":"172.21.15.144:6811/4076371964","cluster_addr":"172.21.15.144:6815/4076371964","heartbeat_back_addr":"172.21.15.144:6823/4076371964","heartbeat_front_addr":"172.21.15.144:6820/4076371964","state":["exists","up"]},{"osd":5,"uuid":"c6b386a6-ee3e-4830-b841-8a0eccd10f52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6801","nonce":2379578648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6803","nonce":2379578648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6807","nonce":2379578648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2379578648},{"type":"v1","addr":"172.21.15.144:6805","nonce":2379578648}]},"public_addr":"172.21.15.144:6801/2379578648","cluster_addr":"172.21.15.144:6803/2379578648","heartbeat_back_addr":"172.21.15.144:6807/2379578648","heartbeat_front_addr":"172.21.15.144:6805/2379578648","state":["exists","up"]},{"osd":6,"uuid":"ed3e4cf6-cfa2-472b-a061-d9a5734f4169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6810","nonce":3198738696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6814","nonce":3198738696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6821","nonce":3198738696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3198738696},{"type":"v1","addr":"172.21.15.144:6818","nonce":3198738696}]},"public_addr":"172.21.15.144:6810/3198738696","cluster_addr":"172.21.15.144:6814/3198738696","heartbeat_back_addr":"172.21.15.144:6821/3198738696","heartbeat_front_addr":"172.21.15.144:6818/3198738696","state":["exists","up"]},{"osd":7,"uuid":"bc3f5d1f-f576-4fa5-85fe-c957d278349f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6825","nonce":1746570140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6827","nonce":1746570140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6831","nonce":1746570140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1746570140},{"type":"v1","addr":"172.21.15.144:6829","nonce":1746570140}]},"public_addr":"172.21.15.144:6825/1746570140","cluster_addr":"172.21.15.144:6827/1746570140","heartbeat_back_addr":"172.21.15.144:6831/1746570140","heartbeat_front_addr":"172.21.15.144:6829/1746570140","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.286221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.385442+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.384108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.466276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.252105+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.510940+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.283633+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T09:52:56.591300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6837/1204910660":"2024-04-18T10:04:37.928545+0000","172.21.15.32:6836/1204910660":"2024-04-18T10:04:37.928545+0000","172.21.15.32:6837/578643390":"2024-04-18T10:04:22.429980+0000","172.21.15.32:6836/578643390":"2024-04-18T10:04:22.429980+0000","172.21.15.32:6837/901447746":"2024-04-18T10:04:03.080821+0000","172.21.15.32:6836/901447746":"2024-04-18T10:04:03.080821+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T10:04:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:44.098+0000 7f7704e00640 1 -- 172.21.15.32:0/3137800895 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f76dc068380 msgr2=0x7f76dc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:44.098+0000 7f7704e00640 1 --2- 172.21.15.32:0/3137800895 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f76dc068380 0x7f76dc06a840 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f76f00ad000 tx=0x7f76f00a6000 comp rx=0 tx=0).stop 2024-04-17T10:04:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:44.098+0000 7f7704e00640 1 -- 172.21.15.32:0/3137800895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77000d1790 msgr2=0x7f770015a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:44.098+0000 7f7704e00640 1 --2- 172.21.15.32:0/3137800895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77000d1790 0x7f770015a820 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009dd0 tx=0x7f76f800c490 comp rx=0 tx=0).stop 2024-04-17T10:04:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:44.098+0000 7f7704e00640 1 -- 172.21.15.32:0/3137800895 shutdown_connections 2024-04-17T10:04:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:44.098+0000 7f7704e00640 1 --2- 172.21.15.32:0/3137800895 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f76dc068380 0x7f76dc06a840 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:44.098+0000 7f7704e00640 1 --2- 172.21.15.32:0/3137800895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77000d1790 0x7f770015a820 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:44.098+0000 7f7704e00640 1 --2- 172.21.15.32:0/3137800895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f770006ce20 0x7f7700158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:44.098+0000 7f7704e00640 1 --2- 172.21.15.32:0/3137800895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770006ba30 0x7f77001f80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:44.098+0000 7f7704e00640 1 -- 172.21.15.32:0/3137800895 >> 172.21.15.32:0/3137800895 conn(0x7f77000d0c00 msgr2=0x7f77000b17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:44.098+0000 7f7704e00640 1 -- 172.21.15.32:0/3137800895 shutdown_connections 2024-04-17T10:04:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:44.098+0000 7f7704e00640 1 -- 172.21.15.32:0/3137800895 wait complete. 2024-04-17T10:04:44.625 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:04:44.629 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:04:45.112 INFO:tasks.ceph:Scrubbing osd.0 2024-04-17T10:04:45.112 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-17T10:04:45.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.274+0000 7fadea800640 1 Processor -- start 2024-04-17T10:04:45.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.274+0000 7fadea800640 1 -- start start 2024-04-17T10:04:45.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.274+0000 7fadea800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fade40cfc70 0x7fade406edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:45.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.274+0000 7fadea800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fade406f330 0x7fade40cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:45.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.274+0000 7fadea800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fade40cd6a0 0x7fade4071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:45.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.274+0000 7fadea800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fade415fb30 con 0x7fade40cfc70 2024-04-17T10:04:45.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.274+0000 7fadea800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fade406f770 con 0x7fade40cd6a0 2024-04-17T10:04:45.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.274+0000 7fadea800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fade415f200 con 0x7fade406f330 2024-04-17T10:04:45.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.274+0000 7fade3400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fade40cfc70 0x7fade406edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:45.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.274+0000 7fade3400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fade40cfc70 0x7fade406edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44158/0 (socket says 172.21.15.32:44158) 2024-04-17T10:04:45.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.274+0000 7fade3400640 1 -- 172.21.15.32:0/3058969209 learned_addr learned my addr 172.21.15.32:0/3058969209 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:45.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.274+0000 7fade2a00640 1 --2- 172.21.15.32:0/3058969209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fade406f330 0x7fade40cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:45.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.274+0000 7fade3e00640 1 --2- 172.21.15.32:0/3058969209 >> v2:172.21.15.144:3300/0 conn(0x7fade40cd6a0 0x7fade4071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:45.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.274+0000 7fade3400640 1 -- 172.21.15.32:0/3058969209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fade406f330 msgr2=0x7fade40cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:45.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.274+0000 7fade3400640 1 --2- 172.21.15.32:0/3058969209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fade406f330 0x7fade40cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.274+0000 7fade3400640 1 -- 172.21.15.32:0/3058969209 >> v2:172.21.15.144:3300/0 conn(0x7fade40cd6a0 msgr2=0x7fade4071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:45.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.274+0000 7fade3400640 1 --2- 172.21.15.32:0/3058969209 >> v2:172.21.15.144:3300/0 conn(0x7fade40cd6a0 0x7fade4071db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.274+0000 7fade3400640 1 -- 172.21.15.32:0/3058969209 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade415f4e0 con 0x7fade40cfc70 2024-04-17T10:04:45.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.278+0000 7fade3400640 1 --2- 172.21.15.32:0/3058969209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade40cfc70 0x7fade406edf0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fade406fe10 tx=0x7fadd401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b3e5e222a65f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:45.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.278+0000 7fade2000640 1 -- 172.21.15.32:0/3058969209 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd40b0070 con 0x7fade40cfc70 2024-04-17T10:04:45.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.278+0000 7fade2000640 1 -- 172.21.15.32:0/3058969209 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadd40ab600 con 0x7fade40cfc70 2024-04-17T10:04:45.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.278+0000 7fade2000640 1 -- 172.21.15.32:0/3058969209 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd40ab9d0 con 0x7fade40cfc70 2024-04-17T10:04:45.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.278+0000 7fadea800640 1 -- 172.21.15.32:0/3058969209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade40cfc70 msgr2=0x7fade406edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:45.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.278+0000 7fadea800640 1 --2- 172.21.15.32:0/3058969209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade40cfc70 0x7fade406edf0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fade406fe10 tx=0x7fadd401c260 comp rx=0 tx=0).stop 2024-04-17T10:04:45.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.278+0000 7fadea800640 1 -- 172.21.15.32:0/3058969209 shutdown_connections 2024-04-17T10:04:45.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.278+0000 7fadea800640 1 --2- 172.21.15.32:0/3058969209 >> v2:172.21.15.144:3300/0 conn(0x7fade40cd6a0 0x7fade4071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.278+0000 7fadea800640 1 --2- 172.21.15.32:0/3058969209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fade406f330 0x7fade40cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.278+0000 7fadea800640 1 --2- 172.21.15.32:0/3058969209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade40cfc70 0x7fade406edf0 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.278+0000 7fadea800640 1 -- 172.21.15.32:0/3058969209 >> 172.21.15.32:0/3058969209 conn(0x7fade40b2190 msgr2=0x7fade40b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:45.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.278+0000 7fadea800640 1 -- 172.21.15.32:0/3058969209 shutdown_connections 2024-04-17T10:04:45.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.278+0000 7fadea800640 1 -- 172.21.15.32:0/3058969209 wait complete. 2024-04-17T10:04:45.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7fadea800640 1 Processor -- start 2024-04-17T10:04:45.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7fadea800640 1 -- start start 2024-04-17T10:04:45.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7fadea800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade406f330 0x7fade4158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:45.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7fadea800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fade40cd6a0 0x7fade4159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:45.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7fadea800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade40cfc70 0x7fade4150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:45.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7fadea800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fade40d01d0 con 0x7fade406f330 2024-04-17T10:04:45.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7fadea800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fade40d0050 con 0x7fade40cfc70 2024-04-17T10:04:45.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7fadea800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fade415ba70 con 0x7fade40cd6a0 2024-04-17T10:04:45.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7fade3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade406f330 0x7fade4158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:45.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7fade3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade406f330 0x7fade4158c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44166/0 (socket says 172.21.15.32:44166) 2024-04-17T10:04:45.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7fade3400640 1 -- 172.21.15.32:0/3178589588 learned_addr learned my addr 172.21.15.32:0/3178589588 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:45.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7fade2a00640 1 --2- 172.21.15.32:0/3178589588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fade40cd6a0 0x7fade4159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:45.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7fade3e00640 1 --2- 172.21.15.32:0/3178589588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade40cfc70 0x7fade4150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:45.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7fade3400640 1 -- 172.21.15.32:0/3178589588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fade40cd6a0 msgr2=0x7fade4159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:45.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7fade3400640 1 --2- 172.21.15.32:0/3178589588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fade40cd6a0 0x7fade4159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7fade3400640 1 -- 172.21.15.32:0/3178589588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade40cfc70 msgr2=0x7fade4150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:45.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7fade3400640 1 --2- 172.21.15.32:0/3178589588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade40cfc70 0x7fade4150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7fade3400640 1 -- 172.21.15.32:0/3178589588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade4158000 con 0x7fade406f330 2024-04-17T10:04:45.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7fade3400640 1 --2- 172.21.15.32:0/3178589588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade406f330 0x7fade4158c30 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fadd40ae040 tx=0x7fadd4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:45.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7faddb400640 1 -- 172.21.15.32:0/3178589588 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd40b0040 con 0x7fade406f330 2024-04-17T10:04:45.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7faddb400640 1 -- 172.21.15.32:0/3178589588 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadd40b9070 con 0x7fade406f330 2024-04-17T10:04:45.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7faddb400640 1 -- 172.21.15.32:0/3178589588 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd40b4680 con 0x7fade406f330 2024-04-17T10:04:45.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7fadea800640 1 -- 172.21.15.32:0/3178589588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade41582e0 con 0x7fade406f330 2024-04-17T10:04:45.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7fadea800640 1 -- 172.21.15.32:0/3178589588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade414f270 con 0x7fade406f330 2024-04-17T10:04:45.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.282+0000 7fadea800640 1 -- 172.21.15.32:0/3178589588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fade4158520 con 0x7fade406f330 2024-04-17T10:04:45.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.286+0000 7faddb400640 1 -- 172.21.15.32:0/3178589588 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fadd40ab4c0 con 0x7fade406f330 2024-04-17T10:04:45.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.286+0000 7faddb400640 1 --2- 172.21.15.32:0/3178589588 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fadc4068380 0x7fadc406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:45.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.286+0000 7faddb400640 1 -- 172.21.15.32:0/3178589588 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7fadd4120d10 con 0x7fade406f330 2024-04-17T10:04:45.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.286+0000 7faddb400640 1 --2- 172.21.15.32:0/3178589588 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x7fadc406e380 0x7fadc40707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:45.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.286+0000 7fade2a00640 1 --2- 172.21.15.32:0/3178589588 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fadc4068380 0x7fadc406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:45.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.286+0000 7faddb400640 1 -- 172.21.15.32:0/3178589588 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fadd40c7ec0 con 0x7fadc406e380 2024-04-17T10:04:45.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.286+0000 7faddb400640 1 -- 172.21.15.32:0/3178589588 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7fadd41210c0 con 0x7fade406f330 2024-04-17T10:04:45.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.286+0000 7fade3e00640 1 --2- 172.21.15.32:0/3178589588 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x7fadc406e380 0x7fadc40707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:45.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.286+0000 7fade2a00640 1 --2- 172.21.15.32:0/3178589588 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fadc4068380 0x7fadc406a840 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fade40b5580 tx=0x7fadd0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:45.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.286+0000 7fade3e00640 1 --2- 172.21.15.32:0/3178589588 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x7fadc406e380 0x7fadc40707a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:45.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.286+0000 7faddb400640 1 -- 172.21.15.32:0/3178589588 <== osd.0 v2:172.21.15.32:6800/2392056658 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fadd40c7ec0 con 0x7fadc406e380 2024-04-17T10:04:45.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.302+0000 7fadea800640 1 -- 172.21.15.32:0/3178589588 --> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fade4068a90 con 0x7fadc406e380 2024-04-17T10:04:45.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.302+0000 7faddb400640 1 -- 172.21.15.32:0/3178589588 <== osd.0 v2:172.21.15.32:6800/2392056658 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fade4068a90 con 0x7fadc406e380 2024-04-17T10:04:45.297 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:04:45.297 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T10:04:45.297 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:04:45.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.302+0000 7fadea800640 1 -- 172.21.15.32:0/3178589588 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x7fadc406e380 msgr2=0x7fadc40707a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:45.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.302+0000 7fadea800640 1 --2- 172.21.15.32:0/3178589588 >> [v2:172.21.15.32:6800/2392056658,v1:172.21.15.32:6801/2392056658] conn(0x7fadc406e380 0x7fadc40707a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.302+0000 7fadea800640 1 -- 172.21.15.32:0/3178589588 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fadc4068380 msgr2=0x7fadc406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:45.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.302+0000 7fadea800640 1 --2- 172.21.15.32:0/3178589588 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fadc4068380 0x7fadc406a840 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fade40b5580 tx=0x7fadd0008040 comp rx=0 tx=0).stop 2024-04-17T10:04:45.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.302+0000 7fadea800640 1 -- 172.21.15.32:0/3178589588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade406f330 msgr2=0x7fade4158c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:45.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.302+0000 7fadea800640 1 --2- 172.21.15.32:0/3178589588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade406f330 0x7fade4158c30 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fadd40ae040 tx=0x7fadd4019000 comp rx=0 tx=0).stop 2024-04-17T10:04:45.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.302+0000 7fade3e00640 1 -- 172.21.15.32:0/3178589588 reap_dead start 2024-04-17T10:04:45.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.302+0000 7fadea800640 1 -- 172.21.15.32:0/3178589588 shutdown_connections 2024-04-17T10:04:45.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.302+0000 7fadea800640 1 -- 172.21.15.32:0/3178589588 >> 172.21.15.32:0/3178589588 conn(0x7fade40b2190 msgr2=0x7fade40b0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:45.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.302+0000 7fadea800640 1 -- 172.21.15.32:0/3178589588 shutdown_connections 2024-04-17T10:04:45.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.302+0000 7fadea800640 1 -- 172.21.15.32:0/3178589588 wait complete. 2024-04-17T10:04:45.316 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-17T10:04:45.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.462+0000 7fe676000640 1 Processor -- start 2024-04-17T10:04:45.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.462+0000 7fe676000640 1 -- start start 2024-04-17T10:04:45.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.462+0000 7fe676000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe670155a70 0x7fe67015f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:45.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.462+0000 7fe676000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe670155e50 0x7fe67015d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:45.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.462+0000 7fe676000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe67015f870 con 0x7fe67015d5d0 2024-04-17T10:04:45.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.462+0000 7fe676000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fe6700b42a0 con 0x7fe670155e50 2024-04-17T10:04:45.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.462+0000 7fe676000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe6700d1250 con 0x7fe670155a70 2024-04-17T10:04:45.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.462+0000 7fe667e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe670155e50 0x7fe67015d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:45.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.462+0000 7fe66fe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe67015d5d0 0x7fe67015c260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33978/0 (socket says 172.21.15.32:33978) 2024-04-17T10:04:45.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.462+0000 7fe66fe00640 1 -- 172.21.15.32:0/2539616141 learned_addr learned my addr 172.21.15.32:0/2539616141 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:45.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.462+0000 7fe66f400640 1 --2- 172.21.15.32:0/2539616141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe670155a70 0x7fe67015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:45.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.466+0000 7fe667e00640 1 -- 172.21.15.32:0/2539616141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe670155a70 msgr2=0x7fe67015f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:45.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.466+0000 7fe667e00640 1 --2- 172.21.15.32:0/2539616141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe670155a70 0x7fe67015f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.466+0000 7fe667e00640 1 -- 172.21.15.32:0/2539616141 >> v1:172.21.15.32:6789/0 conn(0x7fe67015d5d0 legacy=0x7fe67015c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:45.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.466+0000 7fe667e00640 1 -- 172.21.15.32:0/2539616141 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe67015fb50 con 0x7fe670155e50 2024-04-17T10:04:45.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.466+0000 7fe667e00640 1 --2- 172.21.15.32:0/2539616141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe670155e50 0x7fe67015d090 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fe6580097c0 tx=0x7fe65801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f007c15a5a32fc02 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:45.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.466+0000 7fe66ea00640 1 -- 172.21.15.32:0/2539616141 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6580b2070 con 0x7fe670155e50 2024-04-17T10:04:45.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.466+0000 7fe66ea00640 1 -- 172.21.15.32:0/2539616141 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6580ad640 con 0x7fe670155e50 2024-04-17T10:04:45.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.466+0000 7fe66ea00640 1 -- 172.21.15.32:0/2539616141 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6580ada10 con 0x7fe670155e50 2024-04-17T10:04:45.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.466+0000 7fe676000640 1 -- 172.21.15.32:0/2539616141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe670155e50 msgr2=0x7fe67015d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:45.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.466+0000 7fe676000640 1 --2- 172.21.15.32:0/2539616141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe670155e50 0x7fe67015d090 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fe6580097c0 tx=0x7fe65801c4a0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.466+0000 7fe676000640 1 -- 172.21.15.32:0/2539616141 shutdown_connections 2024-04-17T10:04:45.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.466+0000 7fe676000640 1 --2- 172.21.15.32:0/2539616141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe670155e50 0x7fe67015d090 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.466+0000 7fe676000640 1 --2- 172.21.15.32:0/2539616141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe670155a70 0x7fe67015f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.466+0000 7fe676000640 1 -- 172.21.15.32:0/2539616141 >> 172.21.15.32:0/2539616141 conn(0x7fe67006ba30 msgr2=0x7fe67006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:45.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.466+0000 7fe676000640 1 -- 172.21.15.32:0/2539616141 shutdown_connections 2024-04-17T10:04:45.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.466+0000 7fe676000640 1 -- 172.21.15.32:0/2539616141 wait complete. 2024-04-17T10:04:45.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe676000640 1 Processor -- start 2024-04-17T10:04:45.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe676000640 1 -- start start 2024-04-17T10:04:45.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe676000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe670155a70 0x7fe6700aab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:45.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe676000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe670155e50 0x7fe670151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:45.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe676000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe67015d5d0 0x7fe6700aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:45.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe676000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe6700d02f0 con 0x7fe670155e50 2024-04-17T10:04:45.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe676000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe6700d0170 con 0x7fe670155a70 2024-04-17T10:04:45.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe676000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe6700d0470 con 0x7fe67015d5d0 2024-04-17T10:04:45.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe667e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe670155e50 0x7fe670151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:45.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe667e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe670155e50 0x7fe670151f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44188/0 (socket says 172.21.15.32:44188) 2024-04-17T10:04:45.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe667e00640 1 -- 172.21.15.32:0/2968524156 learned_addr learned my addr 172.21.15.32:0/2968524156 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:45.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe66fe00640 1 --2- 172.21.15.32:0/2968524156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe67015d5d0 0x7fe6700aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:45.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe667e00640 1 -- 172.21.15.32:0/2968524156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe67015d5d0 msgr2=0x7fe6700aeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:45.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe667e00640 1 --2- 172.21.15.32:0/2968524156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe67015d5d0 0x7fe6700aeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe66f400640 1 --2- 172.21.15.32:0/2968524156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe670155a70 0x7fe6700aab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:45.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe667e00640 1 -- 172.21.15.32:0/2968524156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe670155a70 msgr2=0x7fe6700aab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:45.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe667e00640 1 --2- 172.21.15.32:0/2968524156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe670155a70 0x7fe6700aab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe667e00640 1 -- 172.21.15.32:0/2968524156 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe670153470 con 0x7fe670155e50 2024-04-17T10:04:45.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe66fe00640 1 --2- 172.21.15.32:0/2968524156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe67015d5d0 0x7fe6700aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:45.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe66f400640 1 --2- 172.21.15.32:0/2968524156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe670155a70 0x7fe6700aab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:45.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe667e00640 1 --2- 172.21.15.32:0/2968524156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe670155e50 0x7fe670151f60 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fe65801a0d0 tx=0x7fe658019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:45.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe667400640 1 -- 172.21.15.32:0/2968524156 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6580ba020 con 0x7fe670155e50 2024-04-17T10:04:45.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe667400640 1 -- 172.21.15.32:0/2968524156 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6580b6410 con 0x7fe670155e50 2024-04-17T10:04:45.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe667400640 1 -- 172.21.15.32:0/2968524156 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6580b2040 con 0x7fe670155e50 2024-04-17T10:04:45.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe676000640 1 -- 172.21.15.32:0/2968524156 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6700aa0b0 con 0x7fe670155e50 2024-04-17T10:04:45.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.470+0000 7fe676000640 1 -- 172.21.15.32:0/2968524156 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6700aa670 con 0x7fe670155e50 2024-04-17T10:04:45.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.474+0000 7fe676000640 1 -- 172.21.15.32:0/2968524156 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe670068a90 con 0x7fe670155e50 2024-04-17T10:04:45.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.474+0000 7fe667400640 1 -- 172.21.15.32:0/2968524156 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe6580bd020 con 0x7fe670155e50 2024-04-17T10:04:45.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.474+0000 7fe667400640 1 --2- 172.21.15.32:0/2968524156 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe64c068380 0x7fe64c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:45.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.474+0000 7fe667400640 1 -- 172.21.15.32:0/2968524156 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7fe6581205d0 con 0x7fe670155e50 2024-04-17T10:04:45.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.474+0000 7fe66f400640 1 --2- 172.21.15.32:0/2968524156 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe64c068380 0x7fe64c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:45.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.478+0000 7fe66f400640 1 --2- 172.21.15.32:0/2968524156 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe64c068380 0x7fe64c06a840 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe660009ae0 tx=0x7fe660008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:45.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.478+0000 7fe667400640 1 -- 172.21.15.32:0/2968524156 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fe6580c4030 con 0x7fe670155e50 2024-04-17T10:04:45.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.646+0000 7fe676000640 1 -- 172.21.15.32:0/2968524156 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fe6700b17d0 con 0x7fe64c068380 2024-04-17T10:04:45.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.650+0000 7fe667400640 1 -- 172.21.15.32:0/2968524156 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe6700b17d0 con 0x7fe64c068380 2024-04-17T10:04:45.644 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 0 to deep-scrub 2024-04-17T10:04:45.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.650+0000 7fe676000640 1 -- 172.21.15.32:0/2968524156 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe64c068380 msgr2=0x7fe64c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:45.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.650+0000 7fe676000640 1 --2- 172.21.15.32:0/2968524156 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe64c068380 0x7fe64c06a840 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe660009ae0 tx=0x7fe660008040 comp rx=0 tx=0).stop 2024-04-17T10:04:45.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.650+0000 7fe676000640 1 -- 172.21.15.32:0/2968524156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe670155e50 msgr2=0x7fe670151f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:45.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.650+0000 7fe676000640 1 --2- 172.21.15.32:0/2968524156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe670155e50 0x7fe670151f60 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fe65801a0d0 tx=0x7fe658019240 comp rx=0 tx=0).stop 2024-04-17T10:04:45.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.650+0000 7fe676000640 1 -- 172.21.15.32:0/2968524156 shutdown_connections 2024-04-17T10:04:45.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.650+0000 7fe676000640 1 --2- 172.21.15.32:0/2968524156 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe64c068380 0x7fe64c06a840 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.650+0000 7fe676000640 1 --2- 172.21.15.32:0/2968524156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe67015d5d0 0x7fe6700aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.650+0000 7fe676000640 1 --2- 172.21.15.32:0/2968524156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe670155e50 0x7fe670151f60 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.650+0000 7fe676000640 1 --2- 172.21.15.32:0/2968524156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe670155a70 0x7fe6700aab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.650+0000 7fe676000640 1 -- 172.21.15.32:0/2968524156 >> 172.21.15.32:0/2968524156 conn(0x7fe67006ba30 msgr2=0x7fe67006f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:45.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.650+0000 7fe676000640 1 -- 172.21.15.32:0/2968524156 shutdown_connections 2024-04-17T10:04:45.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.650+0000 7fe676000640 1 -- 172.21.15.32:0/2968524156 wait complete. 2024-04-17T10:04:45.665 INFO:tasks.ceph:Scrubbing osd.1 2024-04-17T10:04:45.665 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-17T10:04:45.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.814+0000 7fe77b800640 1 Processor -- start 2024-04-17T10:04:45.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.814+0000 7fe77b800640 1 -- start start 2024-04-17T10:04:45.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.814+0000 7fe77b800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe77406f500 0x7fe7740cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:45.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.814+0000 7fe77b800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7740cd6a0 0x7fe77415ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:45.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.814+0000 7fe77b800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe77415f200 con 0x7fe7740cfc70 2024-04-17T10:04:45.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.814+0000 7fe77b800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe77415fb30 con 0x7fe77406f500 2024-04-17T10:04:45.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.814+0000 7fe77b800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe77415f4e0 con 0x7fe7740cd6a0 2024-04-17T10:04:45.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.814+0000 7fe778e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe7740cfc70 0x7fe77406edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33992/0 (socket says 172.21.15.32:33992) 2024-04-17T10:04:45.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.814+0000 7fe778e00640 1 -- 172.21.15.32:0/38695179 learned_addr learned my addr 172.21.15.32:0/38695179 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:45.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.814+0000 7fe76be00640 1 --2- 172.21.15.32:0/38695179 >> v2:172.21.15.144:3300/0 conn(0x7fe77406f500 0x7fe7740cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:45.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.814+0000 7fe779800640 1 --2- 172.21.15.32:0/38695179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7740cd6a0 0x7fe77415ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:45.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.814+0000 7fe773e00640 1 -- 172.21.15.32:0/38695179 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1840456709 0 0) 0x7fe77415f200 con 0x7fe7740cfc70 2024-04-17T10:04:45.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.814+0000 7fe773e00640 1 -- 172.21.15.32:0/38695179 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe758005740 con 0x7fe7740cfc70 2024-04-17T10:04:45.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.814+0000 7fe76be00640 1 -- 172.21.15.32:0/38695179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7740cd6a0 msgr2=0x7fe77415ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:45.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.814+0000 7fe76be00640 1 --2- 172.21.15.32:0/38695179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7740cd6a0 0x7fe77415ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.814+0000 7fe76be00640 1 -- 172.21.15.32:0/38695179 >> v1:172.21.15.32:6789/0 conn(0x7fe7740cfc70 legacy=0x7fe77406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:45.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.814+0000 7fe76be00640 1 -- 172.21.15.32:0/38695179 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe77415f7c0 con 0x7fe77406f500 2024-04-17T10:04:45.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.814+0000 7fe779800640 1 --2- 172.21.15.32:0/38695179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7740cd6a0 0x7fe77415ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:04:45.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.818+0000 7fe76be00640 1 --2- 172.21.15.32:0/38695179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe77406f500 0x7fe7740cd160 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0097a0 tx=0x7fe75c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d40941560721df6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:45.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.818+0000 7fe773e00640 1 -- 172.21.15.32:0/38695179 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0b0070 con 0x7fe77406f500 2024-04-17T10:04:45.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.818+0000 7fe773e00640 1 -- 172.21.15.32:0/38695179 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe75c0ab600 con 0x7fe77406f500 2024-04-17T10:04:45.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.818+0000 7fe773e00640 1 -- 172.21.15.32:0/38695179 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0ab9d0 con 0x7fe77406f500 2024-04-17T10:04:45.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.818+0000 7fe77b800640 1 -- 172.21.15.32:0/38695179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe77406f500 msgr2=0x7fe7740cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:45.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.818+0000 7fe77b800640 1 --2- 172.21.15.32:0/38695179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe77406f500 0x7fe7740cd160 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0097a0 tx=0x7fe75c01c480 comp rx=0 tx=0).stop 2024-04-17T10:04:45.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.818+0000 7fe77b800640 1 -- 172.21.15.32:0/38695179 shutdown_connections 2024-04-17T10:04:45.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.818+0000 7fe77b800640 1 --2- 172.21.15.32:0/38695179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7740cd6a0 0x7fe77415ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.818+0000 7fe77b800640 1 --2- 172.21.15.32:0/38695179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe77406f500 0x7fe7740cd160 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.818+0000 7fe77b800640 1 -- 172.21.15.32:0/38695179 >> 172.21.15.32:0/38695179 conn(0x7fe7740b2190 msgr2=0x7fe7740b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:45.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.818+0000 7fe77b800640 1 -- 172.21.15.32:0/38695179 shutdown_connections 2024-04-17T10:04:45.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.818+0000 7fe77b800640 1 -- 172.21.15.32:0/38695179 wait complete. 2024-04-17T10:04:45.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe77b800640 1 Processor -- start 2024-04-17T10:04:45.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe77b800640 1 -- start start 2024-04-17T10:04:45.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe77b800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe77406f500 0x7fe77413e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:45.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe77b800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7740cd6a0 0x7fe7741589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:45.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe77b800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7740cfc70 0x7fe7741f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:45.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe77b800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe77406de60 con 0x7fe7740cd6a0 2024-04-17T10:04:45.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe77b800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe77406dce0 con 0x7fe7740cfc70 2024-04-17T10:04:45.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe77b800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe77406dfe0 con 0x7fe77406f500 2024-04-17T10:04:45.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe76be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7740cd6a0 0x7fe7741589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:45.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe76be00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7740cd6a0 0x7fe7741589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44198/0 (socket says 172.21.15.32:44198) 2024-04-17T10:04:45.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe76be00640 1 -- 172.21.15.32:0/848774839 learned_addr learned my addr 172.21.15.32:0/848774839 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:45.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe778e00640 1 --2- 172.21.15.32:0/848774839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe77406f500 0x7fe77413e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:45.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe779800640 1 --2- 172.21.15.32:0/848774839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7740cfc70 0x7fe7741f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:45.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe76be00640 1 -- 172.21.15.32:0/848774839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe77406f500 msgr2=0x7fe77413e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:45.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe76be00640 1 --2- 172.21.15.32:0/848774839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe77406f500 0x7fe77413e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe76be00640 1 -- 172.21.15.32:0/848774839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7740cfc70 msgr2=0x7fe7741f73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:45.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe76be00640 1 --2- 172.21.15.32:0/848774839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7740cfc70 0x7fe7741f73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe76be00640 1 -- 172.21.15.32:0/848774839 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe77413c670 con 0x7fe7740cd6a0 2024-04-17T10:04:45.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe76be00640 1 --2- 172.21.15.32:0/848774839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7740cd6a0 0x7fe7741589c0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fe75c01a0b0 tx=0x7fe75c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:45.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe771600640 1 -- 172.21.15.32:0/848774839 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0b0040 con 0x7fe7740cd6a0 2024-04-17T10:04:45.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe771600640 1 -- 172.21.15.32:0/848774839 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe75c0b9070 con 0x7fe7740cd6a0 2024-04-17T10:04:45.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe771600640 1 -- 172.21.15.32:0/848774839 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0b4670 con 0x7fe7740cd6a0 2024-04-17T10:04:45.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe77b800640 1 -- 172.21.15.32:0/848774839 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7741563a0 con 0x7fe7740cd6a0 2024-04-17T10:04:45.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.822+0000 7fe77b800640 1 -- 172.21.15.32:0/848774839 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe77413ef20 con 0x7fe7740cd6a0 2024-04-17T10:04:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.826+0000 7fe77b800640 1 -- 172.21.15.32:0/848774839 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe774068a90 con 0x7fe7740cd6a0 2024-04-17T10:04:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.826+0000 7fe771600640 1 -- 172.21.15.32:0/848774839 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe75c0b4810 con 0x7fe7740cd6a0 2024-04-17T10:04:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.826+0000 7fe771600640 1 --2- 172.21.15.32:0/848774839 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe750068380 0x7fe75006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.826+0000 7fe771600640 1 -- 172.21.15.32:0/848774839 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7fe75c120460 con 0x7fe7740cd6a0 2024-04-17T10:04:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.826+0000 7fe778e00640 1 --2- 172.21.15.32:0/848774839 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe750068380 0x7fe75006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.826+0000 7fe771600640 1 --2- 172.21.15.32:0/848774839 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x7fe75006e380 0x7fe7500707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.826+0000 7fe771600640 1 -- 172.21.15.32:0/848774839 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe75c0ab600 con 0x7fe75006e380 2024-04-17T10:04:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.826+0000 7fe771600640 1 -- 172.21.15.32:0/848774839 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7fe75c127050 con 0x7fe7740cd6a0 2024-04-17T10:04:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.826+0000 7fe779800640 1 --2- 172.21.15.32:0/848774839 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x7fe75006e380 0x7fe7500707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.826+0000 7fe778e00640 1 --2- 172.21.15.32:0/848774839 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe750068380 0x7fe75006a840 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe764005890 tx=0x7fe764009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.826+0000 7fe779800640 1 --2- 172.21.15.32:0/848774839 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x7fe75006e380 0x7fe7500707a0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:45.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.830+0000 7fe771600640 1 -- 172.21.15.32:0/848774839 <== osd.1 v2:172.21.15.32:6808/2454993455 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe75c0ab600 con 0x7fe75006e380 2024-04-17T10:04:45.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.842+0000 7fe77b800640 1 -- 172.21.15.32:0/848774839 --> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe774072e50 con 0x7fe75006e380 2024-04-17T10:04:45.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.842+0000 7fe771600640 1 -- 172.21.15.32:0/848774839 <== osd.1 v2:172.21.15.32:6808/2454993455 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe774072e50 con 0x7fe75006e380 2024-04-17T10:04:45.838 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:04:45.838 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T10:04:45.838 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:04:45.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.842+0000 7fe77b800640 1 -- 172.21.15.32:0/848774839 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x7fe75006e380 msgr2=0x7fe7500707a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:45.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.842+0000 7fe77b800640 1 --2- 172.21.15.32:0/848774839 >> [v2:172.21.15.32:6808/2454993455,v1:172.21.15.32:6810/2454993455] conn(0x7fe75006e380 0x7fe7500707a0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:45.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.842+0000 7fe77b800640 1 -- 172.21.15.32:0/848774839 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe750068380 msgr2=0x7fe75006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:45.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.842+0000 7fe77b800640 1 --2- 172.21.15.32:0/848774839 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fe750068380 0x7fe75006a840 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe764005890 tx=0x7fe764009040 comp rx=0 tx=0).stop 2024-04-17T10:04:45.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.842+0000 7fe77b800640 1 -- 172.21.15.32:0/848774839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7740cd6a0 msgr2=0x7fe7741589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:45.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.842+0000 7fe77b800640 1 --2- 172.21.15.32:0/848774839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7740cd6a0 0x7fe7741589c0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fe75c01a0b0 tx=0x7fe75c019220 comp rx=0 tx=0).stop 2024-04-17T10:04:45.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.842+0000 7fe779800640 1 -- 172.21.15.32:0/848774839 reap_dead start 2024-04-17T10:04:45.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.842+0000 7fe77b800640 1 -- 172.21.15.32:0/848774839 shutdown_connections 2024-04-17T10:04:45.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.842+0000 7fe77b800640 1 -- 172.21.15.32:0/848774839 >> 172.21.15.32:0/848774839 conn(0x7fe7740b2190 msgr2=0x7fe774155830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:45.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.842+0000 7fe77b800640 1 -- 172.21.15.32:0/848774839 shutdown_connections 2024-04-17T10:04:45.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:45.842+0000 7fe77b800640 1 -- 172.21.15.32:0/848774839 wait complete. 2024-04-17T10:04:45.856 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-17T10:04:45.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.002+0000 7f52b5600640 1 Processor -- start 2024-04-17T10:04:45.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.002+0000 7f52b5600640 1 -- start start 2024-04-17T10:04:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.002+0000 7f52b5600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b00cd150 0x7f52b015d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.002+0000 7f52b5600640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b00cd9a0 con 0x7f52b00d04f0 2024-04-17T10:04:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.002+0000 7f52b5600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b00cdc80 con 0x7f52b009a9f0 2024-04-17T10:04:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.002+0000 7f52b5600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f52b00cdf60 con 0x7f52b00cd150 2024-04-17T10:04:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.002+0000 7f52af400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f52b00d04f0 0x7f52b00d0980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33994/0 (socket says 172.21.15.32:33994) 2024-04-17T10:04:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.002+0000 7f52af400640 1 -- 172.21.15.32:0/1955086676 learned_addr learned my addr 172.21.15.32:0/1955086676 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.002+0000 7f52a7e00640 1 --2- 172.21.15.32:0/1955086676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b00cd150 0x7f52b015d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.002+0000 7f52ae000640 1 -- 172.21.15.32:0/1955086676 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3616587604 0 0) 0x7f52b00cd9a0 con 0x7f52b00d04f0 2024-04-17T10:04:46.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.002+0000 7f52ae000640 1 -- 172.21.15.32:0/1955086676 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5294003530 con 0x7f52b00d04f0 2024-04-17T10:04:46.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.002+0000 7f52ae000640 1 -- 172.21.15.32:0/1955086676 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1003741976 0 0) 0x7f52b00cdc80 con 0x7f52b009a9f0 2024-04-17T10:04:46.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.002+0000 7f52a7e00640 1 -- 172.21.15.32:0/1955086676 >> v1:172.21.15.144:6789/0 conn(0x7f52b009a9f0 legacy=0x7f52b009add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.002+0000 7f52a7e00640 1 -- 172.21.15.32:0/1955086676 >> v1:172.21.15.32:6789/0 conn(0x7f52b00d04f0 legacy=0x7f52b00d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.002+0000 7f52a7e00640 1 -- 172.21.15.32:0/1955086676 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b00ce240 con 0x7f52b00cd150 2024-04-17T10:04:46.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.006+0000 7f52a7e00640 1 --2- 172.21.15.32:0/1955086676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b00cd150 0x7f52b015d5a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f52980097f0 tx=0x7f529801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ecf8f7ade6d34e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:46.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.006+0000 7f52ae000640 1 -- 172.21.15.32:0/1955086676 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52980ab5f0 con 0x7f52b00cd150 2024-04-17T10:04:46.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.006+0000 7f52ae000640 1 -- 172.21.15.32:0/1955086676 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52980b0070 con 0x7f52b00cd150 2024-04-17T10:04:46.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.006+0000 7f52ae000640 1 -- 172.21.15.32:0/1955086676 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52980ab9b0 con 0x7f52b00cd150 2024-04-17T10:04:46.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.006+0000 7f52b5600640 1 -- 172.21.15.32:0/1955086676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b00cd150 msgr2=0x7f52b015d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.006+0000 7f52b5600640 1 --2- 172.21.15.32:0/1955086676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b00cd150 0x7f52b015d5a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f52980097f0 tx=0x7f529801c4e0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.006+0000 7f52b5600640 1 -- 172.21.15.32:0/1955086676 shutdown_connections 2024-04-17T10:04:46.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.006+0000 7f52b5600640 1 --2- 172.21.15.32:0/1955086676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b00cd150 0x7f52b015d5a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.006+0000 7f52b5600640 1 -- 172.21.15.32:0/1955086676 >> 172.21.15.32:0/1955086676 conn(0x7f52b00b4030 msgr2=0x7f52b00b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:46.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.006+0000 7f52b5600640 1 -- 172.21.15.32:0/1955086676 shutdown_connections 2024-04-17T10:04:46.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.006+0000 7f52b5600640 1 -- 172.21.15.32:0/1955086676 wait complete. 2024-04-17T10:04:46.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52b5600640 1 Processor -- start 2024-04-17T10:04:46.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52b5600640 1 -- start start 2024-04-17T10:04:46.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52b5600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b009a9f0 0x7f52b0152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:46.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52b5600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52b00cd150 0x7f52b006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:46.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52b5600640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b00d04f0 0x7f52b00cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:46.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52b5600640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f52b015dc60 con 0x7f52b00d04f0 2024-04-17T10:04:46.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52b5600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f52b015dae0 con 0x7f52b00cd150 2024-04-17T10:04:46.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52b5600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f52b016ae10 con 0x7f52b009a9f0 2024-04-17T10:04:46.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52af400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b00d04f0 0x7f52b00cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:46.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52af400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b00d04f0 0x7f52b00cbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44206/0 (socket says 172.21.15.32:44206) 2024-04-17T10:04:46.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52af400640 1 -- 172.21.15.32:0/2586782801 learned_addr learned my addr 172.21.15.32:0/2586782801 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:46.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52aea00640 1 --2- 172.21.15.32:0/2586782801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b009a9f0 0x7f52b0152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:46.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52a7e00640 1 --2- 172.21.15.32:0/2586782801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52b00cd150 0x7f52b006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:46.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52af400640 1 -- 172.21.15.32:0/2586782801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b009a9f0 msgr2=0x7f52b0152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52af400640 1 --2- 172.21.15.32:0/2586782801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b009a9f0 0x7f52b0152030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52af400640 1 -- 172.21.15.32:0/2586782801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52b00cd150 msgr2=0x7f52b006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52af400640 1 --2- 172.21.15.32:0/2586782801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52b00cd150 0x7f52b006d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52af400640 1 -- 172.21.15.32:0/2586782801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b00cb070 con 0x7f52b00d04f0 2024-04-17T10:04:46.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52af400640 1 --2- 172.21.15.32:0/2586782801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b00d04f0 0x7f52b00cbc60 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f52a800cb70 tx=0x7f52a800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:46.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52a6a00640 1 -- 172.21.15.32:0/2586782801 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8017070 con 0x7f52b00d04f0 2024-04-17T10:04:46.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52a6a00640 1 -- 172.21.15.32:0/2586782801 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a800ad40 con 0x7f52b00d04f0 2024-04-17T10:04:46.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52a6a00640 1 -- 172.21.15.32:0/2586782801 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8012620 con 0x7f52b00d04f0 2024-04-17T10:04:46.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52b5600640 1 -- 172.21.15.32:0/2586782801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b00c9b10 con 0x7f52b00d04f0 2024-04-17T10:04:46.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.010+0000 7f52b5600640 1 -- 172.21.15.32:0/2586782801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b00c8d30 con 0x7f52b00d04f0 2024-04-17T10:04:46.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.014+0000 7f52b5600640 1 -- 172.21.15.32:0/2586782801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b0068a90 con 0x7f52b00d04f0 2024-04-17T10:04:46.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.014+0000 7f52a6a00640 1 -- 172.21.15.32:0/2586782801 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f52a801a070 con 0x7f52b00d04f0 2024-04-17T10:04:46.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.014+0000 7f52a6a00640 1 --2- 172.21.15.32:0/2586782801 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f528c068380 0x7f528c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:46.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.014+0000 7f52a6a00640 1 -- 172.21.15.32:0/2586782801 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f52a80865c0 con 0x7f52b00d04f0 2024-04-17T10:04:46.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.014+0000 7f52aea00640 1 --2- 172.21.15.32:0/2586782801 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f528c068380 0x7f528c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:46.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.014+0000 7f52aea00640 1 --2- 172.21.15.32:0/2586782801 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f528c068380 0x7f528c06a840 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f52a0000f70 tx=0x7f52a0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:46.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.018+0000 7f52a6a00640 1 -- 172.21.15.32:0/2586782801 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f52a8056dc0 con 0x7f52b00d04f0 2024-04-17T10:04:46.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.182+0000 7f52b5600640 1 -- 172.21.15.32:0/2586782801 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f52b00b4d00 con 0x7f528c068380 2024-04-17T10:04:46.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.182+0000 7f52a6a00640 1 -- 172.21.15.32:0/2586782801 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f52b00b4d00 con 0x7f528c068380 2024-04-17T10:04:46.178 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 1 to deep-scrub 2024-04-17T10:04:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.182+0000 7f52b5600640 1 -- 172.21.15.32:0/2586782801 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f528c068380 msgr2=0x7f528c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.182+0000 7f52b5600640 1 --2- 172.21.15.32:0/2586782801 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f528c068380 0x7f528c06a840 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f52a0000f70 tx=0x7f52a0009040 comp rx=0 tx=0).stop 2024-04-17T10:04:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.182+0000 7f52b5600640 1 -- 172.21.15.32:0/2586782801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b00d04f0 msgr2=0x7f52b00cbc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.182+0000 7f52b5600640 1 --2- 172.21.15.32:0/2586782801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b00d04f0 0x7f52b00cbc60 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f52a800cb70 tx=0x7f52a800a430 comp rx=0 tx=0).stop 2024-04-17T10:04:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.182+0000 7f52b5600640 1 -- 172.21.15.32:0/2586782801 shutdown_connections 2024-04-17T10:04:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.182+0000 7f52b5600640 1 --2- 172.21.15.32:0/2586782801 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f528c068380 0x7f528c06a840 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.182+0000 7f52b5600640 1 --2- 172.21.15.32:0/2586782801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b00d04f0 0x7f52b00cbc60 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.182+0000 7f52b5600640 1 --2- 172.21.15.32:0/2586782801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52b00cd150 0x7f52b006d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.182+0000 7f52b5600640 1 --2- 172.21.15.32:0/2586782801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b009a9f0 0x7f52b0152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.182+0000 7f52b5600640 1 -- 172.21.15.32:0/2586782801 >> 172.21.15.32:0/2586782801 conn(0x7f52b00b4030 msgr2=0x7f52b009add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:46.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.186+0000 7f52b5600640 1 -- 172.21.15.32:0/2586782801 shutdown_connections 2024-04-17T10:04:46.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.186+0000 7f52b5600640 1 -- 172.21.15.32:0/2586782801 wait complete. 2024-04-17T10:04:46.199 INFO:tasks.ceph:Scrubbing osd.2 2024-04-17T10:04:46.200 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-17T10:04:46.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.350+0000 7f2b9d200640 1 Processor -- start 2024-04-17T10:04:46.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.350+0000 7f2b9d200640 1 -- start start 2024-04-17T10:04:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.350+0000 7f2b9d200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2b9806f500 0x7f2b980cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.350+0000 7f2b9d200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b980cd6a0 0x7f2b9815ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.350+0000 7f2b9d200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b9815f200 con 0x7f2b980cfc70 2024-04-17T10:04:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.350+0000 7f2b9d200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f2b9815fb30 con 0x7f2b9806f500 2024-04-17T10:04:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.350+0000 7f2b9d200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2b9815f4e0 con 0x7f2b980cd6a0 2024-04-17T10:04:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.350+0000 7f2b95e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2b980cfc70 0x7f2b9806edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33998/0 (socket says 172.21.15.32:33998) 2024-04-17T10:04:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.350+0000 7f2b95e00640 1 -- 172.21.15.32:0/936827217 learned_addr learned my addr 172.21.15.32:0/936827217 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.350+0000 7f2b95400640 1 --2- 172.21.15.32:0/936827217 >> v2:172.21.15.32:3300/0 conn(0x7f2b9806f500 0x7f2b980cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.350+0000 7f2b94a00640 1 -- 172.21.15.32:0/936827217 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3653053936 0 0) 0x7f2b9815f200 con 0x7f2b980cfc70 2024-04-17T10:04:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.350+0000 7f2b96800640 1 --2- 172.21.15.32:0/936827217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b980cd6a0 0x7f2b9815ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.350+0000 7f2b94a00640 1 -- 172.21.15.32:0/936827217 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b7c003530 con 0x7f2b980cfc70 2024-04-17T10:04:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.350+0000 7f2b95400640 1 -- 172.21.15.32:0/936827217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b980cd6a0 msgr2=0x7f2b9815ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.350+0000 7f2b95400640 1 --2- 172.21.15.32:0/936827217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b980cd6a0 0x7f2b9815ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.350+0000 7f2b95400640 1 -- 172.21.15.32:0/936827217 >> v1:172.21.15.32:6789/0 conn(0x7f2b980cfc70 legacy=0x7f2b9806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.350+0000 7f2b95400640 1 -- 172.21.15.32:0/936827217 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b9815f7c0 con 0x7f2b9806f500 2024-04-17T10:04:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.354+0000 7f2b95400640 1 --2- 172.21.15.32:0/936827217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b9806f500 0x7f2b980cd160 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f2b840097a0 tx=0x7f2b8401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5e2e65dafa68de server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.354+0000 7f2b94a00640 1 -- 172.21.15.32:0/936827217 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b840b0070 con 0x7f2b9806f500 2024-04-17T10:04:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.354+0000 7f2b94a00640 1 -- 172.21.15.32:0/936827217 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b840ab600 con 0x7f2b9806f500 2024-04-17T10:04:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.354+0000 7f2b94a00640 1 -- 172.21.15.32:0/936827217 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b840ab9d0 con 0x7f2b9806f500 2024-04-17T10:04:46.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.354+0000 7f2b9d200640 1 -- 172.21.15.32:0/936827217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b9806f500 msgr2=0x7f2b980cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.354+0000 7f2b9d200640 1 --2- 172.21.15.32:0/936827217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b9806f500 0x7f2b980cd160 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f2b840097a0 tx=0x7f2b8401c480 comp rx=0 tx=0).stop 2024-04-17T10:04:46.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.354+0000 7f2b9d200640 1 -- 172.21.15.32:0/936827217 shutdown_connections 2024-04-17T10:04:46.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.354+0000 7f2b9d200640 1 --2- 172.21.15.32:0/936827217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b980cd6a0 0x7f2b9815ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.354+0000 7f2b9d200640 1 --2- 172.21.15.32:0/936827217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b9806f500 0x7f2b980cd160 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.354+0000 7f2b9d200640 1 -- 172.21.15.32:0/936827217 >> 172.21.15.32:0/936827217 conn(0x7f2b980b2190 msgr2=0x7f2b980b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:46.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.354+0000 7f2b9d200640 1 -- 172.21.15.32:0/936827217 shutdown_connections 2024-04-17T10:04:46.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.354+0000 7f2b9d200640 1 -- 172.21.15.32:0/936827217 wait complete. 2024-04-17T10:04:46.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b9d200640 1 Processor -- start 2024-04-17T10:04:46.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b9d200640 1 -- start start 2024-04-17T10:04:46.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b9d200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b9806f500 0x7f2b981560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:46.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b9d200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b980cd6a0 0x7f2b981550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:46.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b9d200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b980cfc70 0x7f2b980ac640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:46.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b9d200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9806de60 con 0x7f2b980cfc70 2024-04-17T10:04:46.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b9d200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9806dce0 con 0x7f2b980cd6a0 2024-04-17T10:04:46.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b9d200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2b9806dfe0 con 0x7f2b9806f500 2024-04-17T10:04:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b96800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b980cfc70 0x7f2b980ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b96800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b980cfc70 0x7f2b980ac640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44228/0 (socket says 172.21.15.32:44228) 2024-04-17T10:04:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b96800640 1 -- 172.21.15.32:0/3979553612 learned_addr learned my addr 172.21.15.32:0/3979553612 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b95400640 1 --2- 172.21.15.32:0/3979553612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b980cd6a0 0x7f2b981550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b95e00640 1 --2- 172.21.15.32:0/3979553612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b9806f500 0x7f2b981560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b96800640 1 -- 172.21.15.32:0/3979553612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b9806f500 msgr2=0x7f2b981560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b96800640 1 --2- 172.21.15.32:0/3979553612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b9806f500 0x7f2b981560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b96800640 1 -- 172.21.15.32:0/3979553612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b980cd6a0 msgr2=0x7f2b981550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b96800640 1 --2- 172.21.15.32:0/3979553612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b980cd6a0 0x7f2b981550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b96800640 1 -- 172.21.15.32:0/3979553612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b98156780 con 0x7f2b980cfc70 2024-04-17T10:04:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b96800640 1 --2- 172.21.15.32:0/3979553612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b980cfc70 0x7f2b980ac640 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f2b90007e40 tx=0x7f2b9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:46.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b8e000640 1 -- 172.21.15.32:0/3979553612 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b90014620 con 0x7f2b980cfc70 2024-04-17T10:04:46.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b8e000640 1 -- 172.21.15.32:0/3979553612 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b90019070 con 0x7f2b980cfc70 2024-04-17T10:04:46.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b8e000640 1 -- 172.21.15.32:0/3979553612 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b90014a20 con 0x7f2b980cfc70 2024-04-17T10:04:46.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b9d200640 1 -- 172.21.15.32:0/3979553612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b98156a60 con 0x7f2b980cfc70 2024-04-17T10:04:46.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b9d200640 1 -- 172.21.15.32:0/3979553612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b98157020 con 0x7f2b980cfc70 2024-04-17T10:04:46.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.358+0000 7f2b9d200640 1 -- 172.21.15.32:0/3979553612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2b981555f0 con 0x7f2b980cfc70 2024-04-17T10:04:46.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.362+0000 7f2b8e000640 1 -- 172.21.15.32:0/3979553612 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2b90014bc0 con 0x7f2b980cfc70 2024-04-17T10:04:46.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.362+0000 7f2b8e000640 1 --2- 172.21.15.32:0/3979553612 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2b78068380 0x7f2b7806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:46.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.362+0000 7f2b8e000640 1 -- 172.21.15.32:0/3979553612 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f2b90087e40 con 0x7f2b980cfc70 2024-04-17T10:04:46.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.362+0000 7f2b8e000640 1 --2- 172.21.15.32:0/3979553612 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x7f2b7806e380 0x7f2b780707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:46.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.362+0000 7f2b95e00640 1 --2- 172.21.15.32:0/3979553612 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2b78068380 0x7f2b7806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:46.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.362+0000 7f2b8e000640 1 -- 172.21.15.32:0/3979553612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2b9000df30 con 0x7f2b7806e380 2024-04-17T10:04:46.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.362+0000 7f2b8e000640 1 -- 172.21.15.32:0/3979553612 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7f2b900881f0 con 0x7f2b980cfc70 2024-04-17T10:04:46.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.362+0000 7f2b95400640 1 --2- 172.21.15.32:0/3979553612 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x7f2b7806e380 0x7f2b780707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:46.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.362+0000 7f2b95e00640 1 --2- 172.21.15.32:0/3979553612 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2b78068380 0x7f2b7806a840 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f2b88005890 tx=0x7f2b88009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:46.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.362+0000 7f2b95400640 1 --2- 172.21.15.32:0/3979553612 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x7f2b7806e380 0x7f2b780707a0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:46.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.362+0000 7f2b8e000640 1 -- 172.21.15.32:0/3979553612 <== osd.2 v2:172.21.15.32:6809/1954833462 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2b9000df30 con 0x7f2b7806e380 2024-04-17T10:04:46.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.378+0000 7f2b9d200640 1 -- 172.21.15.32:0/3979553612 --> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2b98068a90 con 0x7f2b7806e380 2024-04-17T10:04:46.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.378+0000 7f2b8e000640 1 -- 172.21.15.32:0/3979553612 <== osd.2 v2:172.21.15.32:6809/1954833462 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2b98068a90 con 0x7f2b7806e380 2024-04-17T10:04:46.372 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:04:46.372 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T10:04:46.372 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:04:46.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.378+0000 7f2b9d200640 1 -- 172.21.15.32:0/3979553612 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x7f2b7806e380 msgr2=0x7f2b780707a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.378+0000 7f2b9d200640 1 --2- 172.21.15.32:0/3979553612 >> [v2:172.21.15.32:6809/1954833462,v1:172.21.15.32:6811/1954833462] conn(0x7f2b7806e380 0x7f2b780707a0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.378+0000 7f2b9d200640 1 -- 172.21.15.32:0/3979553612 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2b78068380 msgr2=0x7f2b7806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.378+0000 7f2b9d200640 1 --2- 172.21.15.32:0/3979553612 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f2b78068380 0x7f2b7806a840 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f2b88005890 tx=0x7f2b88009040 comp rx=0 tx=0).stop 2024-04-17T10:04:46.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.378+0000 7f2b9d200640 1 -- 172.21.15.32:0/3979553612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b980cfc70 msgr2=0x7f2b980ac640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.378+0000 7f2b9d200640 1 --2- 172.21.15.32:0/3979553612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b980cfc70 0x7f2b980ac640 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f2b90007e40 tx=0x7f2b9000e4b0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.378+0000 7f2b96800640 1 -- 172.21.15.32:0/3979553612 reap_dead start 2024-04-17T10:04:46.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.378+0000 7f2b9d200640 1 -- 172.21.15.32:0/3979553612 shutdown_connections 2024-04-17T10:04:46.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.378+0000 7f2b9d200640 1 -- 172.21.15.32:0/3979553612 >> 172.21.15.32:0/3979553612 conn(0x7f2b980b2190 msgr2=0x7f2b9806ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:46.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.378+0000 7f2b9d200640 1 -- 172.21.15.32:0/3979553612 shutdown_connections 2024-04-17T10:04:46.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.378+0000 7f2b9d200640 1 -- 172.21.15.32:0/3979553612 wait complete. 2024-04-17T10:04:46.391 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-17T10:04:46.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.538+0000 7f25da000640 1 Processor -- start 2024-04-17T10:04:46.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.538+0000 7f25da000640 1 -- start start 2024-04-17T10:04:46.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.538+0000 7f25da000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f25d4072b00 0x7f25d40b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:46.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.538+0000 7f25da000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25d4072ee0 con 0x7f25d40d09f0 2024-04-17T10:04:46.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.538+0000 7f25da000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25d4166900 con 0x7f25d406f8f0 2024-04-17T10:04:46.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.538+0000 7f25da000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f25d4166be0 con 0x7f25d4072b00 2024-04-17T10:04:46.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.538+0000 7f25d3e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f25d40d09f0 0x7f25d4163200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34006/0 (socket says 172.21.15.32:34006) 2024-04-17T10:04:46.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.538+0000 7f25d3e00640 1 -- 172.21.15.32:0/3094863512 learned_addr learned my addr 172.21.15.32:0/3094863512 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:46.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.538+0000 7f25d3400640 1 --2- 172.21.15.32:0/3094863512 >> v2:172.21.15.144:3300/0 conn(0x7f25d4072b00 0x7f25d40b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:46.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.538+0000 7f25d2a00640 1 -- 172.21.15.32:0/3094863512 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 96352291 0 0) 0x7f25d4072ee0 con 0x7f25d40d09f0 2024-04-17T10:04:46.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.538+0000 7f25d2a00640 1 -- 172.21.15.32:0/3094863512 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25b8003530 con 0x7f25d40d09f0 2024-04-17T10:04:46.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.538+0000 7f25d2a00640 1 -- 172.21.15.32:0/3094863512 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 170880745 0 0) 0x7f25d4166900 con 0x7f25d406f8f0 2024-04-17T10:04:46.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.538+0000 7f25d2a00640 1 -- 172.21.15.32:0/3094863512 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25d4072ee0 con 0x7f25d406f8f0 2024-04-17T10:04:46.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.538+0000 7f25d3400640 1 -- 172.21.15.32:0/3094863512 >> v1:172.21.15.144:6789/0 conn(0x7f25d406f8f0 legacy=0x7f25d40d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.538+0000 7f25d2a00640 1 -- 172.21.15.32:0/3094863512 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1525761349 0 0) 0x7f25b8003530 con 0x7f25d40d09f0 2024-04-17T10:04:46.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.538+0000 7f25d3400640 1 -- 172.21.15.32:0/3094863512 >> v1:172.21.15.32:6789/0 conn(0x7f25d40d09f0 legacy=0x7f25d4163200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.538+0000 7f25d3400640 1 -- 172.21.15.32:0/3094863512 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d4166ec0 con 0x7f25d4072b00 2024-04-17T10:04:46.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.542+0000 7f25d3400640 1 --2- 172.21.15.32:0/3094863512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25d4072b00 0x7f25d40b3de0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f25c40097c0 tx=0x7f25c401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f52f05fa6faab78b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:46.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.542+0000 7f25d2a00640 1 -- 172.21.15.32:0/3094863512 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c40b1070 con 0x7f25d4072b00 2024-04-17T10:04:46.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.542+0000 7f25d2a00640 1 -- 172.21.15.32:0/3094863512 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25c40ac410 con 0x7f25d4072b00 2024-04-17T10:04:46.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.542+0000 7f25d2a00640 1 -- 172.21.15.32:0/3094863512 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c40ac7e0 con 0x7f25d4072b00 2024-04-17T10:04:46.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.542+0000 7f25da000640 1 -- 172.21.15.32:0/3094863512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25d4072b00 msgr2=0x7f25d40b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.542+0000 7f25da000640 1 --2- 172.21.15.32:0/3094863512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25d4072b00 0x7f25d40b3de0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f25c40097c0 tx=0x7f25c401c4a0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.542+0000 7f25da000640 1 -- 172.21.15.32:0/3094863512 shutdown_connections 2024-04-17T10:04:46.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.542+0000 7f25da000640 1 --2- 172.21.15.32:0/3094863512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25d4072b00 0x7f25d40b3de0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.542+0000 7f25da000640 1 -- 172.21.15.32:0/3094863512 >> 172.21.15.32:0/3094863512 conn(0x7f25d406ba30 msgr2=0x7f25d406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:46.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.542+0000 7f25da000640 1 -- 172.21.15.32:0/3094863512 shutdown_connections 2024-04-17T10:04:46.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.542+0000 7f25da000640 1 -- 172.21.15.32:0/3094863512 wait complete. 2024-04-17T10:04:46.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25da000640 1 Processor -- start 2024-04-17T10:04:46.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25da000640 1 -- start start 2024-04-17T10:04:46.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25da000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25d406f8f0 0x7f25d40cf190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:46.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25da000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25d4072b00 0x7f25d40cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:46.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25da000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25d40d09f0 0x7f25d4150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:46.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25da000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f25d4170d90 con 0x7f25d40d09f0 2024-04-17T10:04:46.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25da000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f25d4170c10 con 0x7f25d406f8f0 2024-04-17T10:04:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25da000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f25d4170f10 con 0x7f25d4072b00 2024-04-17T10:04:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25d3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25d40d09f0 0x7f25d4150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25d3e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25d40d09f0 0x7f25d4150ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44242/0 (socket says 172.21.15.32:44242) 2024-04-17T10:04:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25d3e00640 1 -- 172.21.15.32:0/192349053 learned_addr learned my addr 172.21.15.32:0/192349053 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25d3400640 1 --2- 172.21.15.32:0/192349053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25d406f8f0 0x7f25d40cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25cbe00640 1 --2- 172.21.15.32:0/192349053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25d4072b00 0x7f25d40cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25d3e00640 1 -- 172.21.15.32:0/192349053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25d4072b00 msgr2=0x7f25d40cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25d3e00640 1 --2- 172.21.15.32:0/192349053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25d4072b00 0x7f25d40cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25d3e00640 1 -- 172.21.15.32:0/192349053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25d406f8f0 msgr2=0x7f25d40cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25d3e00640 1 --2- 172.21.15.32:0/192349053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25d406f8f0 0x7f25d40cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25d3e00640 1 -- 172.21.15.32:0/192349053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d4157420 con 0x7f25d40d09f0 2024-04-17T10:04:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25d3e00640 1 --2- 172.21.15.32:0/192349053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25d40d09f0 0x7f25d4150ef0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00cb40 tx=0x7f25cc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25cb400640 1 -- 172.21.15.32:0/192349053 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc00aca0 con 0x7f25d40d09f0 2024-04-17T10:04:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25cb400640 1 -- 172.21.15.32:0/192349053 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25cc012400 con 0x7f25d40d09f0 2024-04-17T10:04:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25cb400640 1 -- 172.21.15.32:0/192349053 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc00ae60 con 0x7f25d40d09f0 2024-04-17T10:04:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25da000640 1 -- 172.21.15.32:0/192349053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d4157700 con 0x7f25d40d09f0 2024-04-17T10:04:46.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.546+0000 7f25da000640 1 -- 172.21.15.32:0/192349053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d4157cc0 con 0x7f25d40d09f0 2024-04-17T10:04:46.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.550+0000 7f25da000640 1 -- 172.21.15.32:0/192349053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25d4068a90 con 0x7f25d40d09f0 2024-04-17T10:04:46.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.550+0000 7f25cb400640 1 -- 172.21.15.32:0/192349053 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f25cc012830 con 0x7f25d40d09f0 2024-04-17T10:04:46.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.550+0000 7f25cb400640 1 --2- 172.21.15.32:0/192349053 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f25b0068380 0x7f25b006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:46.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.550+0000 7f25cb400640 1 -- 172.21.15.32:0/192349053 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f25cc086de0 con 0x7f25d40d09f0 2024-04-17T10:04:46.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.550+0000 7f25d3400640 1 --2- 172.21.15.32:0/192349053 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f25b0068380 0x7f25b006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:46.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.550+0000 7f25d3400640 1 --2- 172.21.15.32:0/192349053 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f25b0068380 0x7f25b006a840 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f25c40ae000 tx=0x7f25c4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:46.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.554+0000 7f25cb400640 1 -- 172.21.15.32:0/192349053 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f25cc0575e0 con 0x7f25d40d09f0 2024-04-17T10:04:46.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.718+0000 7f25da000640 1 -- 172.21.15.32:0/192349053 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f25d40cc810 con 0x7f25b0068380 2024-04-17T10:04:46.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.718+0000 7f25cb400640 1 -- 172.21.15.32:0/192349053 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f25d40cc810 con 0x7f25b0068380 2024-04-17T10:04:46.715 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 2 to deep-scrub 2024-04-17T10:04:46.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.722+0000 7f25da000640 1 -- 172.21.15.32:0/192349053 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f25b0068380 msgr2=0x7f25b006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.722+0000 7f25da000640 1 --2- 172.21.15.32:0/192349053 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f25b0068380 0x7f25b006a840 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f25c40ae000 tx=0x7f25c4008040 comp rx=0 tx=0).stop 2024-04-17T10:04:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.722+0000 7f25da000640 1 -- 172.21.15.32:0/192349053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25d40d09f0 msgr2=0x7f25d4150ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.722+0000 7f25da000640 1 --2- 172.21.15.32:0/192349053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25d40d09f0 0x7f25d4150ef0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00cb40 tx=0x7f25cc00a060 comp rx=0 tx=0).stop 2024-04-17T10:04:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.722+0000 7f25da000640 1 -- 172.21.15.32:0/192349053 shutdown_connections 2024-04-17T10:04:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.722+0000 7f25da000640 1 --2- 172.21.15.32:0/192349053 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f25b0068380 0x7f25b006a840 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.722+0000 7f25da000640 1 --2- 172.21.15.32:0/192349053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25d4072b00 0x7f25d40cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.722+0000 7f25da000640 1 --2- 172.21.15.32:0/192349053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25d40d09f0 0x7f25d4150ef0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.722+0000 7f25da000640 1 --2- 172.21.15.32:0/192349053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25d406f8f0 0x7f25d40cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.722+0000 7f25da000640 1 -- 172.21.15.32:0/192349053 >> 172.21.15.32:0/192349053 conn(0x7f25d406ba30 msgr2=0x7f25d41644a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.722+0000 7f25da000640 1 -- 172.21.15.32:0/192349053 shutdown_connections 2024-04-17T10:04:46.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.722+0000 7f25da000640 1 -- 172.21.15.32:0/192349053 wait complete. 2024-04-17T10:04:46.736 INFO:tasks.ceph:Scrubbing osd.3 2024-04-17T10:04:46.736 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-17T10:04:46.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.882+0000 7f98d3200640 1 Processor -- start 2024-04-17T10:04:46.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.886+0000 7f98d3200640 1 -- start start 2024-04-17T10:04:46.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.886+0000 7f98d3200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98cc162360 0x7f98cc06e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:46.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.886+0000 7f98d3200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f98cc06edd0 0x7f98cc06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:46.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.886+0000 7f98d3200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98cc06f6f0 con 0x7f98cc0b02e0 2024-04-17T10:04:46.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.886+0000 7f98d3200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f98cc0b06c0 con 0x7f98cc06edd0 2024-04-17T10:04:46.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.886+0000 7f98d3200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f98cc162820 con 0x7f98cc162360 2024-04-17T10:04:46.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.886+0000 7f98cbe00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f98cc0b02e0 0x7f98cc15f7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34014/0 (socket says 172.21.15.32:34014) 2024-04-17T10:04:46.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.886+0000 7f98cbe00640 1 -- 172.21.15.32:0/2153125225 learned_addr learned my addr 172.21.15.32:0/2153125225 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:46.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.886+0000 7f98d1200640 1 --2- 172.21.15.32:0/2153125225 >> v2:172.21.15.144:3300/0 conn(0x7f98cc06edd0 0x7f98cc06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:46.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.886+0000 7f98c3400640 1 --2- 172.21.15.32:0/2153125225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98cc162360 0x7f98cc06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:46.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.886+0000 7f98cb400640 1 -- 172.21.15.32:0/2153125225 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4113437531 0 0) 0x7f98cc06f6f0 con 0x7f98cc0b02e0 2024-04-17T10:04:46.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.886+0000 7f98cb400640 1 -- 172.21.15.32:0/2153125225 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98b0003530 con 0x7f98cc0b02e0 2024-04-17T10:04:46.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.886+0000 7f98d1200640 1 -- 172.21.15.32:0/2153125225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98cc162360 msgr2=0x7f98cc06e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:46.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.886+0000 7f98d1200640 1 --2- 172.21.15.32:0/2153125225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98cc162360 0x7f98cc06e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.886+0000 7f98d1200640 1 -- 172.21.15.32:0/2153125225 >> v1:172.21.15.32:6789/0 conn(0x7f98cc0b02e0 legacy=0x7f98cc15f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.886+0000 7f98d1200640 1 -- 172.21.15.32:0/2153125225 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98cc162b00 con 0x7f98cc06edd0 2024-04-17T10:04:46.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.890+0000 7f98d1200640 1 --2- 172.21.15.32:0/2153125225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98cc06edd0 0x7f98cc06bdf0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009d90 tx=0x7f98c401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2fbc12e90a367b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:46.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.890+0000 7f98cb400640 1 -- 172.21.15.32:0/2153125225 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c40b2070 con 0x7f98cc06edd0 2024-04-17T10:04:46.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.890+0000 7f98cb400640 1 -- 172.21.15.32:0/2153125225 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98c40a5830 con 0x7f98cc06edd0 2024-04-17T10:04:46.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.890+0000 7f98cb400640 1 -- 172.21.15.32:0/2153125225 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c40a5c00 con 0x7f98cc06edd0 2024-04-17T10:04:46.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.890+0000 7f98d3200640 1 -- 172.21.15.32:0/2153125225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98cc06edd0 msgr2=0x7f98cc06bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.890+0000 7f98d3200640 1 --2- 172.21.15.32:0/2153125225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98cc06edd0 0x7f98cc06bdf0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009d90 tx=0x7f98c401c970 comp rx=0 tx=0).stop 2024-04-17T10:04:46.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.890+0000 7f98d3200640 1 -- 172.21.15.32:0/2153125225 shutdown_connections 2024-04-17T10:04:46.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.890+0000 7f98d3200640 1 --2- 172.21.15.32:0/2153125225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98cc06edd0 0x7f98cc06bdf0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.890+0000 7f98d3200640 1 --2- 172.21.15.32:0/2153125225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98cc162360 0x7f98cc06e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.890+0000 7f98d3200640 1 -- 172.21.15.32:0/2153125225 >> 172.21.15.32:0/2153125225 conn(0x7f98cc0b3330 msgr2=0x7f98cc0d0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:46.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.890+0000 7f98d3200640 1 -- 172.21.15.32:0/2153125225 shutdown_connections 2024-04-17T10:04:46.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.890+0000 7f98d3200640 1 -- 172.21.15.32:0/2153125225 wait complete. 2024-04-17T10:04:46.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98d3200640 1 Processor -- start 2024-04-17T10:04:46.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98d3200640 1 -- start start 2024-04-17T10:04:46.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98d3200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98cc06edd0 0x7f98cc0af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:46.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98d3200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98cc0b02e0 0x7f98cc14daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:46.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98d3200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98cc162360 0x7f98cc15bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:46.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98d3200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc06ced0 con 0x7f98cc06edd0 2024-04-17T10:04:46.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98d3200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc06cd50 con 0x7f98cc0b02e0 2024-04-17T10:04:46.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98d3200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f98cc06d050 con 0x7f98cc162360 2024-04-17T10:04:46.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98cbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98cc06edd0 0x7f98cc0af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:46.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98cbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98cc06edd0 0x7f98cc0af540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44256/0 (socket says 172.21.15.32:44256) 2024-04-17T10:04:46.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98cbe00640 1 -- 172.21.15.32:0/3830904461 learned_addr learned my addr 172.21.15.32:0/3830904461 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:46.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98c3400640 1 --2- 172.21.15.32:0/3830904461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98cc0b02e0 0x7f98cc14daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:46.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98d1200640 1 --2- 172.21.15.32:0/3830904461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98cc162360 0x7f98cc15bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:46.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98cbe00640 1 -- 172.21.15.32:0/3830904461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98cc162360 msgr2=0x7f98cc15bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98cbe00640 1 --2- 172.21.15.32:0/3830904461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98cc162360 0x7f98cc15bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98cbe00640 1 -- 172.21.15.32:0/3830904461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98cc0b02e0 msgr2=0x7f98cc14daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98cbe00640 1 --2- 172.21.15.32:0/3830904461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98cc0b02e0 0x7f98cc14daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98cbe00640 1 -- 172.21.15.32:0/3830904461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98cc159ef0 con 0x7f98cc06edd0 2024-04-17T10:04:46.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98cbe00640 1 --2- 172.21.15.32:0/3830904461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98cc06edd0 0x7f98cc0af540 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00c3f0 tx=0x7f98bc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:46.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98c8c00640 1 -- 172.21.15.32:0/3830904461 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc017070 con 0x7f98cc06edd0 2024-04-17T10:04:46.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98c8c00640 1 -- 172.21.15.32:0/3830904461 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98bc005cf0 con 0x7f98cc06edd0 2024-04-17T10:04:46.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98c8c00640 1 -- 172.21.15.32:0/3830904461 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc0072f0 con 0x7f98cc06edd0 2024-04-17T10:04:46.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98d3200640 1 -- 172.21.15.32:0/3830904461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98cc15a1d0 con 0x7f98cc06edd0 2024-04-17T10:04:46.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98d3200640 1 -- 172.21.15.32:0/3830904461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98cc0af020 con 0x7f98cc06edd0 2024-04-17T10:04:46.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.894+0000 7f98d3200640 1 -- 172.21.15.32:0/3830904461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f98cc068a90 con 0x7f98cc06edd0 2024-04-17T10:04:46.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.898+0000 7f98c8c00640 1 -- 172.21.15.32:0/3830904461 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f98bc01a070 con 0x7f98cc06edd0 2024-04-17T10:04:46.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.898+0000 7f98c8c00640 1 --2- 172.21.15.32:0/3830904461 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f98a8068380 0x7f98a806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:46.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.898+0000 7f98c8c00640 1 -- 172.21.15.32:0/3830904461 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f98bc085b30 con 0x7f98cc06edd0 2024-04-17T10:04:46.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.898+0000 7f98c3400640 1 --2- 172.21.15.32:0/3830904461 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f98a8068380 0x7f98a806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:46.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.898+0000 7f98c8c00640 1 --2- 172.21.15.32:0/3830904461 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x7f98a806e380 0x7f98a80707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:46.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.898+0000 7f98c8c00640 1 -- 172.21.15.32:0/3830904461 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f98bc016ec0 con 0x7f98a806e380 2024-04-17T10:04:46.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.898+0000 7f98c8c00640 1 -- 172.21.15.32:0/3830904461 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7f98bc08a2a0 con 0x7f98cc06edd0 2024-04-17T10:04:46.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.898+0000 7f98d1200640 1 --2- 172.21.15.32:0/3830904461 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x7f98a806e380 0x7f98a80707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:46.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.898+0000 7f98c3400640 1 --2- 172.21.15.32:0/3830904461 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f98a8068380 0x7f98a806a840 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0d07e0 tx=0x7f98b4009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:46.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.898+0000 7f98d1200640 1 --2- 172.21.15.32:0/3830904461 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x7f98a806e380 0x7f98a80707a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:46.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.898+0000 7f98c8c00640 1 -- 172.21.15.32:0/3830904461 <== osd.3 v2:172.21.15.32:6824/2182899298 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98bc016ec0 con 0x7f98a806e380 2024-04-17T10:04:46.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.914+0000 7f98d3200640 1 -- 172.21.15.32:0/3830904461 --> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f98cc0d1b40 con 0x7f98a806e380 2024-04-17T10:04:46.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.914+0000 7f98c8c00640 1 -- 172.21.15.32:0/3830904461 <== osd.3 v2:172.21.15.32:6824/2182899298 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f98cc0d1b40 con 0x7f98a806e380 2024-04-17T10:04:46.909 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:04:46.909 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T10:04:46.909 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:04:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.914+0000 7f98d3200640 1 -- 172.21.15.32:0/3830904461 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x7f98a806e380 msgr2=0x7f98a80707a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.914+0000 7f98d3200640 1 --2- 172.21.15.32:0/3830904461 >> [v2:172.21.15.32:6824/2182899298,v1:172.21.15.32:6825/2182899298] conn(0x7f98a806e380 0x7f98a80707a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.914+0000 7f98d3200640 1 -- 172.21.15.32:0/3830904461 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f98a8068380 msgr2=0x7f98a806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.914+0000 7f98d3200640 1 --2- 172.21.15.32:0/3830904461 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f98a8068380 0x7f98a806a840 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0d07e0 tx=0x7f98b4009190 comp rx=0 tx=0).stop 2024-04-17T10:04:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.914+0000 7f98d3200640 1 -- 172.21.15.32:0/3830904461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98cc06edd0 msgr2=0x7f98cc0af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.914+0000 7f98d3200640 1 --2- 172.21.15.32:0/3830904461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98cc06edd0 0x7f98cc0af540 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00c3f0 tx=0x7f98bc00c8b0 comp rx=0 tx=0).stop 2024-04-17T10:04:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.914+0000 7f98d1200640 1 -- 172.21.15.32:0/3830904461 reap_dead start 2024-04-17T10:04:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.914+0000 7f98d3200640 1 -- 172.21.15.32:0/3830904461 shutdown_connections 2024-04-17T10:04:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.914+0000 7f98d3200640 1 -- 172.21.15.32:0/3830904461 >> 172.21.15.32:0/3830904461 conn(0x7f98cc0b3330 msgr2=0x7f98cc0b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.914+0000 7f98d3200640 1 -- 172.21.15.32:0/3830904461 shutdown_connections 2024-04-17T10:04:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:46.914+0000 7f98d3200640 1 -- 172.21.15.32:0/3830904461 wait complete. 2024-04-17T10:04:46.926 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-17T10:04:47.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.074+0000 7f1861000640 1 Processor -- start 2024-04-17T10:04:47.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.074+0000 7f1861000640 1 -- start start 2024-04-17T10:04:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.074+0000 7f1861000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f185c155a70 0x7f185c15f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.074+0000 7f1861000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f185c0b3340 0x7f185c15c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.074+0000 7f1861000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f185c15f870 con 0x7f185c155e50 2024-04-17T10:04:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.074+0000 7f1861000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f185c0b42a0 con 0x7f185c0b3340 2024-04-17T10:04:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.074+0000 7f1861000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f185c0d1250 con 0x7f185c155a70 2024-04-17T10:04:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.074+0000 7f185a800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f185c0b3340 0x7f185c15c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.074+0000 7f1859e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f185c155a70 0x7f185c15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.074+0000 7f1859400640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f185c155e50 0x7f185c15d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:36728/0 (socket says 172.21.15.32:36728) 2024-04-17T10:04:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.074+0000 7f1859400640 1 -- 172.21.15.32:0/3380011818 learned_addr learned my addr 172.21.15.32:0/3380011818 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.074+0000 7f185a800640 1 -- 172.21.15.32:0/3380011818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f185c155a70 msgr2=0x7f185c15f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.074+0000 7f185a800640 1 --2- 172.21.15.32:0/3380011818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f185c155a70 0x7f185c15f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.074+0000 7f185a800640 1 -- 172.21.15.32:0/3380011818 >> v1:172.21.15.144:6789/0 conn(0x7f185c155e50 legacy=0x7f185c15d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.074+0000 7f185a800640 1 -- 172.21.15.32:0/3380011818 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185c15fb50 con 0x7f185c0b3340 2024-04-17T10:04:47.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.074+0000 7f1859e00640 1 --2- 172.21.15.32:0/3380011818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f185c155a70 0x7f185c15f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:04:47.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.078+0000 7f185a800640 1 --2- 172.21.15.32:0/3380011818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c0b3340 0x7f185c15c260 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1854009d90 tx=0x7f185401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60efe4a04757a1ed server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:47.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.078+0000 7f1858a00640 1 -- 172.21.15.32:0/3380011818 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18540b2070 con 0x7f185c0b3340 2024-04-17T10:04:47.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.078+0000 7f1858a00640 1 -- 172.21.15.32:0/3380011818 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18540a5830 con 0x7f185c0b3340 2024-04-17T10:04:47.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.078+0000 7f1858a00640 1 -- 172.21.15.32:0/3380011818 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18540a5c00 con 0x7f185c0b3340 2024-04-17T10:04:47.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.078+0000 7f1861000640 1 -- 172.21.15.32:0/3380011818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c0b3340 msgr2=0x7f185c15c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.078+0000 7f1861000640 1 --2- 172.21.15.32:0/3380011818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c0b3340 0x7f185c15c260 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1854009d90 tx=0x7f185401c970 comp rx=0 tx=0).stop 2024-04-17T10:04:47.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.078+0000 7f1861000640 1 -- 172.21.15.32:0/3380011818 shutdown_connections 2024-04-17T10:04:47.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.078+0000 7f1861000640 1 --2- 172.21.15.32:0/3380011818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c0b3340 0x7f185c15c260 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.078+0000 7f1861000640 1 --2- 172.21.15.32:0/3380011818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f185c155a70 0x7f185c15f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.078+0000 7f1861000640 1 -- 172.21.15.32:0/3380011818 >> 172.21.15.32:0/3380011818 conn(0x7f185c06ba30 msgr2=0x7f185c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:47.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.078+0000 7f1861000640 1 -- 172.21.15.32:0/3380011818 shutdown_connections 2024-04-17T10:04:47.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.078+0000 7f1861000640 1 -- 172.21.15.32:0/3380011818 wait complete. 2024-04-17T10:04:47.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1861000640 1 Processor -- start 2024-04-17T10:04:47.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1861000640 1 -- start start 2024-04-17T10:04:47.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1861000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c0b3340 0x7f185c152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1861000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f185c155a70 0x7f185c1f8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1861000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f185c155e50 0x7f185c071d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1861000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f185c0d02f0 con 0x7f185c155a70 2024-04-17T10:04:47.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1861000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f185c0d0170 con 0x7f185c0b3340 2024-04-17T10:04:47.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1861000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f185c0d0470 con 0x7f185c155e50 2024-04-17T10:04:47.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1859400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f185c155a70 0x7f185c1f8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1859400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f185c155a70 0x7f185c1f8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44262/0 (socket says 172.21.15.32:44262) 2024-04-17T10:04:47.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1859400640 1 -- 172.21.15.32:0/492370931 learned_addr learned my addr 172.21.15.32:0/492370931 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:47.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1859e00640 1 --2- 172.21.15.32:0/492370931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c0b3340 0x7f185c152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f185a800640 1 --2- 172.21.15.32:0/492370931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f185c155e50 0x7f185c071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1859400640 1 -- 172.21.15.32:0/492370931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f185c155e50 msgr2=0x7f185c071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1859400640 1 --2- 172.21.15.32:0/492370931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f185c155e50 0x7f185c071d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1859400640 1 -- 172.21.15.32:0/492370931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c0b3340 msgr2=0x7f185c152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1859400640 1 --2- 172.21.15.32:0/492370931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c0b3340 0x7f185c152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1859400640 1 -- 172.21.15.32:0/492370931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185c15c360 con 0x7f185c155a70 2024-04-17T10:04:47.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f185a800640 1 --2- 172.21.15.32:0/492370931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f185c155e50 0x7f185c071d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:47.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1859400640 1 --2- 172.21.15.32:0/492370931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f185c155a70 0x7f185c1f8150 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f184800c340 tx=0x7f184800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:47.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1852000640 1 -- 172.21.15.32:0/492370931 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848017070 con 0x7f185c155a70 2024-04-17T10:04:47.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1852000640 1 -- 172.21.15.32:0/492370931 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1848005cf0 con 0x7f185c155a70 2024-04-17T10:04:47.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1852000640 1 -- 172.21.15.32:0/492370931 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18480072f0 con 0x7f185c155a70 2024-04-17T10:04:47.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1861000640 1 -- 172.21.15.32:0/492370931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185c153000 con 0x7f185c155a70 2024-04-17T10:04:47.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1861000640 1 -- 172.21.15.32:0/492370931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f185c14fda0 con 0x7f185c155a70 2024-04-17T10:04:47.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.082+0000 7f1861000640 1 -- 172.21.15.32:0/492370931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f185c14ffa0 con 0x7f185c155a70 2024-04-17T10:04:47.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.086+0000 7f1852000640 1 -- 172.21.15.32:0/492370931 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f184801a070 con 0x7f185c155a70 2024-04-17T10:04:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.086+0000 7f1852000640 1 --2- 172.21.15.32:0/492370931 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f183c0683b0 0x7f183c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.086+0000 7f1852000640 1 -- 172.21.15.32:0/492370931 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f18480858f0 con 0x7f185c155a70 2024-04-17T10:04:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.086+0000 7f1859e00640 1 --2- 172.21.15.32:0/492370931 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f183c0683b0 0x7f183c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.086+0000 7f1859e00640 1 --2- 172.21.15.32:0/492370931 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f183c0683b0 0x7f183c06a870 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f184c009580 tx=0x7f184c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:47.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.090+0000 7f1852000640 1 -- 172.21.15.32:0/492370931 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f185c14ffa0 con 0x7f185c155a70 2024-04-17T10:04:47.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.254+0000 7f1861000640 1 -- 172.21.15.32:0/492370931 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f185c068a90 con 0x7f183c0683b0 2024-04-17T10:04:47.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.254+0000 7f1852000640 1 -- 172.21.15.32:0/492370931 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f185c068a90 con 0x7f183c0683b0 2024-04-17T10:04:47.250 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 3 to deep-scrub 2024-04-17T10:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.258+0000 7f1861000640 1 -- 172.21.15.32:0/492370931 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f183c0683b0 msgr2=0x7f183c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.258+0000 7f1861000640 1 --2- 172.21.15.32:0/492370931 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f183c0683b0 0x7f183c06a870 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f184c009580 tx=0x7f184c009190 comp rx=0 tx=0).stop 2024-04-17T10:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.258+0000 7f1861000640 1 -- 172.21.15.32:0/492370931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f185c155a70 msgr2=0x7f185c1f8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.258+0000 7f1861000640 1 --2- 172.21.15.32:0/492370931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f185c155a70 0x7f185c1f8150 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f184800c340 tx=0x7f184800c800 comp rx=0 tx=0).stop 2024-04-17T10:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.258+0000 7f1861000640 1 -- 172.21.15.32:0/492370931 shutdown_connections 2024-04-17T10:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.258+0000 7f1861000640 1 --2- 172.21.15.32:0/492370931 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f183c0683b0 0x7f183c06a870 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.258+0000 7f1861000640 1 --2- 172.21.15.32:0/492370931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f185c155e50 0x7f185c071d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.258+0000 7f1861000640 1 --2- 172.21.15.32:0/492370931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f185c155a70 0x7f185c1f8150 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.258+0000 7f1861000640 1 --2- 172.21.15.32:0/492370931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c0b3340 0x7f185c152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.258+0000 7f1861000640 1 -- 172.21.15.32:0/492370931 >> 172.21.15.32:0/492370931 conn(0x7f185c06ba30 msgr2=0x7f185c15e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.258+0000 7f1861000640 1 -- 172.21.15.32:0/492370931 shutdown_connections 2024-04-17T10:04:47.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.258+0000 7f1861000640 1 -- 172.21.15.32:0/492370931 wait complete. 2024-04-17T10:04:47.272 INFO:tasks.ceph:Scrubbing osd.4 2024-04-17T10:04:47.272 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-17T10:04:47.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.426+0000 7f8b25000640 1 Processor -- start 2024-04-17T10:04:47.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.426+0000 7f8b25000640 1 -- start start 2024-04-17T10:04:47.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.426+0000 7f8b25000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8b200b1df0 0x7f8b2015cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.426+0000 7f8b25000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b200b28e0 0x7f8b200cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.426+0000 7f8b25000640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b200cd190 con 0x7f8b2006c4d0 2024-04-17T10:04:47.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.426+0000 7f8b25000640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f8b200cd490 con 0x7f8b200b1df0 2024-04-17T10:04:47.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.426+0000 7f8b25000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8b2015d590 con 0x7f8b200b28e0 2024-04-17T10:04:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.426+0000 7f8b15400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8b2006c4d0 0x7f8b200b21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34030/0 (socket says 172.21.15.32:34030) 2024-04-17T10:04:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.426+0000 7f8b15400640 1 -- 172.21.15.32:0/2398807354 learned_addr learned my addr 172.21.15.32:0/2398807354 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.426+0000 7f8b1de00640 1 --2- 172.21.15.32:0/2398807354 >> v2:172.21.15.32:3300/0 conn(0x7f8b200b1df0 0x7f8b2015cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.426+0000 7f8b1e800640 1 --2- 172.21.15.32:0/2398807354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b200b28e0 0x7f8b200cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.426+0000 7f8b1de00640 1 -- 172.21.15.32:0/2398807354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b200b28e0 msgr2=0x7f8b200cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.426+0000 7f8b1de00640 1 --2- 172.21.15.32:0/2398807354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b200b28e0 0x7f8b200cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.426+0000 7f8b1de00640 1 -- 172.21.15.32:0/2398807354 >> v1:172.21.15.32:6789/0 conn(0x7f8b2006c4d0 legacy=0x7f8b200b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.426+0000 7f8b1de00640 1 -- 172.21.15.32:0/2398807354 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2015d870 con 0x7f8b200b1df0 2024-04-17T10:04:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.426+0000 7f8b1d400640 1 -- 172.21.15.32:0/2398807354 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 903637962 0 0) 0x7f8b200cd190 con 0x7f8b2006c4d0 2024-04-17T10:04:47.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.430+0000 7f8b1de00640 1 --2- 172.21.15.32:0/2398807354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b200b1df0 0x7f8b2015cc60 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f8b100097c0 tx=0x7f8b1001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a74b819fffbec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:47.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.430+0000 7f8b1d400640 1 -- 172.21.15.32:0/2398807354 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b100aa620 con 0x7f8b200b1df0 2024-04-17T10:04:47.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.430+0000 7f8b1d400640 1 -- 172.21.15.32:0/2398807354 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b100af070 con 0x7f8b200b1df0 2024-04-17T10:04:47.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.430+0000 7f8b1d400640 1 -- 172.21.15.32:0/2398807354 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b100aa9e0 con 0x7f8b200b1df0 2024-04-17T10:04:47.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.430+0000 7f8b25000640 1 -- 172.21.15.32:0/2398807354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b200b1df0 msgr2=0x7f8b2015cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.430+0000 7f8b25000640 1 --2- 172.21.15.32:0/2398807354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b200b1df0 0x7f8b2015cc60 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f8b100097c0 tx=0x7f8b1001c4a0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.430+0000 7f8b25000640 1 -- 172.21.15.32:0/2398807354 shutdown_connections 2024-04-17T10:04:47.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.430+0000 7f8b25000640 1 --2- 172.21.15.32:0/2398807354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b200b28e0 0x7f8b200cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.430+0000 7f8b25000640 1 --2- 172.21.15.32:0/2398807354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b200b1df0 0x7f8b2015cc60 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.430+0000 7f8b25000640 1 -- 172.21.15.32:0/2398807354 >> 172.21.15.32:0/2398807354 conn(0x7f8b2009a9f0 msgr2=0x7f8b200b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:47.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.430+0000 7f8b25000640 1 -- 172.21.15.32:0/2398807354 shutdown_connections 2024-04-17T10:04:47.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.430+0000 7f8b25000640 1 -- 172.21.15.32:0/2398807354 wait complete. 2024-04-17T10:04:47.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b25000640 1 Processor -- start 2024-04-17T10:04:47.428 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b25000640 1 -- start start 2024-04-17T10:04:47.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b25000640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b2006c4d0 0x7f8b200b6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b25000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b200b1df0 0x7f8b201428c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b25000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b200b28e0 0x7f8b20142e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b25000640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8b200cea80 con 0x7f8b2006c4d0 2024-04-17T10:04:47.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b25000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8b200ce900 con 0x7f8b200b1df0 2024-04-17T10:04:47.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b25000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8b200cec00 con 0x7f8b200b28e0 2024-04-17T10:04:47.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b1de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b2006c4d0 0x7f8b200b6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b1de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b2006c4d0 0x7f8b200b6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44278/0 (socket says 172.21.15.32:44278) 2024-04-17T10:04:47.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b1de00640 1 -- 172.21.15.32:0/673768428 learned_addr learned my addr 172.21.15.32:0/673768428 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:47.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b15400640 1 --2- 172.21.15.32:0/673768428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b200b1df0 0x7f8b201428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b1e800640 1 --2- 172.21.15.32:0/673768428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b200b28e0 0x7f8b20142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b1de00640 1 -- 172.21.15.32:0/673768428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b200b28e0 msgr2=0x7f8b20142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b1de00640 1 --2- 172.21.15.32:0/673768428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b200b28e0 0x7f8b20142e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b1de00640 1 -- 172.21.15.32:0/673768428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b200b1df0 msgr2=0x7f8b201428c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b1de00640 1 --2- 172.21.15.32:0/673768428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b200b1df0 0x7f8b201428c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b1de00640 1 -- 172.21.15.32:0/673768428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b200cb6b0 con 0x7f8b2006c4d0 2024-04-17T10:04:47.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b15400640 1 --2- 172.21.15.32:0/673768428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b200b1df0 0x7f8b201428c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:47.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b1de00640 1 --2- 172.21.15.32:0/673768428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b2006c4d0 0x7f8b200b6a50 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f8b100ac040 tx=0x7f8b1001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:47.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b16a00640 1 -- 172.21.15.32:0/673768428 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b100aacd0 con 0x7f8b2006c4d0 2024-04-17T10:04:47.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b16a00640 1 -- 172.21.15.32:0/673768428 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b100be410 con 0x7f8b2006c4d0 2024-04-17T10:04:47.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b16a00640 1 -- 172.21.15.32:0/673768428 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b100be800 con 0x7f8b2006c4d0 2024-04-17T10:04:47.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b25000640 1 -- 172.21.15.32:0/673768428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b200c8dd0 con 0x7f8b2006c4d0 2024-04-17T10:04:47.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b25000640 1 -- 172.21.15.32:0/673768428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b200aff80 con 0x7f8b2006c4d0 2024-04-17T10:04:47.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.434+0000 7f8b25000640 1 -- 172.21.15.32:0/673768428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8b200afd90 con 0x7f8b2006c4d0 2024-04-17T10:04:47.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.438+0000 7f8b16a00640 1 -- 172.21.15.32:0/673768428 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8b100bc020 con 0x7f8b2006c4d0 2024-04-17T10:04:47.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.438+0000 7f8b16a00640 1 --2- 172.21.15.32:0/673768428 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8afc068380 0x7f8afc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.438+0000 7f8b16a00640 1 -- 172.21.15.32:0/673768428 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f8b10120740 con 0x7f8b2006c4d0 2024-04-17T10:04:47.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.438+0000 7f8b16a00640 1 --2- 172.21.15.32:0/673768428 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x7f8afc06e380 0x7f8afc0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.438+0000 7f8b15400640 1 --2- 172.21.15.32:0/673768428 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8afc068380 0x7f8afc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.438+0000 7f8b16a00640 1 -- 172.21.15.32:0/673768428 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8b100a8e50 con 0x7f8afc06e380 2024-04-17T10:04:47.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.438+0000 7f8b16a00640 1 -- 172.21.15.32:0/673768428 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7f8b10127050 con 0x7f8b2006c4d0 2024-04-17T10:04:47.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.438+0000 7f8b1e800640 1 --2- 172.21.15.32:0/673768428 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x7f8afc06e380 0x7f8afc0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.438+0000 7f8b15400640 1 --2- 172.21.15.32:0/673768428 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8afc068380 0x7f8afc06a840 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8b08007fd0 tx=0x7f8b08009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:47.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.438+0000 7f8b1e800640 1 --2- 172.21.15.32:0/673768428 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x7f8afc06e380 0x7f8afc0707a0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:47.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.438+0000 7f8b16a00640 1 -- 172.21.15.32:0/673768428 <== osd.4 v2:172.21.15.144:6809/4076371964 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b100a8e50 con 0x7f8afc06e380 2024-04-17T10:04:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.454+0000 7f8b25000640 1 -- 172.21.15.32:0/673768428 --> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8b200b06a0 con 0x7f8afc06e380 2024-04-17T10:04:47.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.454+0000 7f8b16a00640 1 -- 172.21.15.32:0/673768428 <== osd.4 v2:172.21.15.144:6809/4076371964 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8b200b06a0 con 0x7f8afc06e380 2024-04-17T10:04:47.450 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:04:47.450 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T10:04:47.450 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:04:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.454+0000 7f8b25000640 1 -- 172.21.15.32:0/673768428 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x7f8afc06e380 msgr2=0x7f8afc0707a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.454+0000 7f8b25000640 1 --2- 172.21.15.32:0/673768428 >> [v2:172.21.15.144:6809/4076371964,v1:172.21.15.144:6811/4076371964] conn(0x7f8afc06e380 0x7f8afc0707a0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.454+0000 7f8b25000640 1 -- 172.21.15.32:0/673768428 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8afc068380 msgr2=0x7f8afc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.454+0000 7f8b25000640 1 --2- 172.21.15.32:0/673768428 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8afc068380 0x7f8afc06a840 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8b08007fd0 tx=0x7f8b08009040 comp rx=0 tx=0).stop 2024-04-17T10:04:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.454+0000 7f8b25000640 1 -- 172.21.15.32:0/673768428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b2006c4d0 msgr2=0x7f8b200b6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.454+0000 7f8b25000640 1 --2- 172.21.15.32:0/673768428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b2006c4d0 0x7f8b200b6a50 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f8b100ac040 tx=0x7f8b1001ace0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.454+0000 7f8b1e800640 1 -- 172.21.15.32:0/673768428 reap_dead start 2024-04-17T10:04:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.454+0000 7f8b25000640 1 -- 172.21.15.32:0/673768428 shutdown_connections 2024-04-17T10:04:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.454+0000 7f8b25000640 1 -- 172.21.15.32:0/673768428 >> 172.21.15.32:0/673768428 conn(0x7f8b2009a9f0 msgr2=0x7f8b200d1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.454+0000 7f8b25000640 1 -- 172.21.15.32:0/673768428 shutdown_connections 2024-04-17T10:04:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.454+0000 7f8b25000640 1 -- 172.21.15.32:0/673768428 wait complete. 2024-04-17T10:04:47.468 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-17T10:04:47.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.614+0000 7fd9b3200640 1 Processor -- start 2024-04-17T10:04:47.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.614+0000 7fd9b3200640 1 -- start start 2024-04-17T10:04:47.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.614+0000 7fd9b3200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9ac06d550 0x7fd9ac06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.614+0000 7fd9b3200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd9ac09ba30 0x7fd9ac0d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.614+0000 7fd9b3200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9ac06de70 con 0x7fd9ac1624e0 2024-04-17T10:04:47.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.614+0000 7fd9b3200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd9ac0d1140 con 0x7fd9ac09ba30 2024-04-17T10:04:47.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.614+0000 7fd9b3200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd9ac0d1420 con 0x7fd9ac06d550 2024-04-17T10:04:47.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.614+0000 7fd9b1200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd9ac1624e0 0x7fd9ac162970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34046/0 (socket says 172.21.15.32:34046) 2024-04-17T10:04:47.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.614+0000 7fd9b1200640 1 -- 172.21.15.32:0/4179937262 learned_addr learned my addr 172.21.15.32:0/4179937262 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:47.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.614+0000 7fd9abe00640 1 --2- 172.21.15.32:0/4179937262 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9ac06d550 0x7fd9ac06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.614+0000 7fd9a3400640 1 --2- 172.21.15.32:0/4179937262 >> v2:172.21.15.144:3300/0 conn(0x7fd9ac09ba30 0x7fd9ac0d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.614+0000 7fd9ab400640 1 -- 172.21.15.32:0/4179937262 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 104670167 0 0) 0x7fd9ac06de70 con 0x7fd9ac1624e0 2024-04-17T10:04:47.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.614+0000 7fd9ab400640 1 -- 172.21.15.32:0/4179937262 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd990003530 con 0x7fd9ac1624e0 2024-04-17T10:04:47.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.614+0000 7fd9abe00640 1 -- 172.21.15.32:0/4179937262 >> v2:172.21.15.144:3300/0 conn(0x7fd9ac09ba30 msgr2=0x7fd9ac0d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:47.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.614+0000 7fd9abe00640 1 --2- 172.21.15.32:0/4179937262 >> v2:172.21.15.144:3300/0 conn(0x7fd9ac09ba30 0x7fd9ac0d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.614+0000 7fd9abe00640 1 -- 172.21.15.32:0/4179937262 >> v1:172.21.15.32:6789/0 conn(0x7fd9ac1624e0 legacy=0x7fd9ac162970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.614+0000 7fd9abe00640 1 -- 172.21.15.32:0/4179937262 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ac0d1700 con 0x7fd9ac06d550 2024-04-17T10:04:47.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.618+0000 7fd9abe00640 1 --2- 172.21.15.32:0/4179937262 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9ac06d550 0x7fd9ac06d930 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0097c0 tx=0x7fd99c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8b1f4c193b322c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:47.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.618+0000 7fd9ab400640 1 -- 172.21.15.32:0/4179937262 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0b0070 con 0x7fd9ac06d550 2024-04-17T10:04:47.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.618+0000 7fd9ab400640 1 -- 172.21.15.32:0/4179937262 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd99c0ab600 con 0x7fd9ac06d550 2024-04-17T10:04:47.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.618+0000 7fd9ab400640 1 -- 172.21.15.32:0/4179937262 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0ab9d0 con 0x7fd9ac06d550 2024-04-17T10:04:47.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.618+0000 7fd9b3200640 1 -- 172.21.15.32:0/4179937262 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9ac06d550 msgr2=0x7fd9ac06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.618+0000 7fd9b3200640 1 --2- 172.21.15.32:0/4179937262 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9ac06d550 0x7fd9ac06d930 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0097c0 tx=0x7fd99c01c4a0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.618+0000 7fd9b3200640 1 -- 172.21.15.32:0/4179937262 shutdown_connections 2024-04-17T10:04:47.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.618+0000 7fd9b3200640 1 --2- 172.21.15.32:0/4179937262 >> v2:172.21.15.144:3300/0 conn(0x7fd9ac09ba30 0x7fd9ac0d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.618+0000 7fd9b3200640 1 --2- 172.21.15.32:0/4179937262 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9ac06d550 0x7fd9ac06d930 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.618+0000 7fd9b3200640 1 -- 172.21.15.32:0/4179937262 >> 172.21.15.32:0/4179937262 conn(0x7fd9ac0b3ff0 msgr2=0x7fd9ac0b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:47.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.618+0000 7fd9b3200640 1 -- 172.21.15.32:0/4179937262 shutdown_connections 2024-04-17T10:04:47.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.618+0000 7fd9b3200640 1 -- 172.21.15.32:0/4179937262 wait complete. 2024-04-17T10:04:47.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9b3200640 1 Processor -- start 2024-04-17T10:04:47.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9b3200640 1 -- start start 2024-04-17T10:04:47.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9b3200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9ac06d550 0x7fd9ac09fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9b3200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9ac09ba30 0x7fd9ac156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9b3200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9ac1624e0 0x7fd9ac0b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9b3200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac16c1b0 con 0x7fd9ac09ba30 2024-04-17T10:04:47.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9b3200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac16c030 con 0x7fd9ac1624e0 2024-04-17T10:04:47.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9b3200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd9ac16c330 con 0x7fd9ac06d550 2024-04-17T10:04:47.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9a3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9ac09ba30 0x7fd9ac156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9a3400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9ac09ba30 0x7fd9ac156750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44292/0 (socket says 172.21.15.32:44292) 2024-04-17T10:04:47.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9a3400640 1 -- 172.21.15.32:0/2785021022 learned_addr learned my addr 172.21.15.32:0/2785021022 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9abe00640 1 --2- 172.21.15.32:0/2785021022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9ac06d550 0x7fd9ac09fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9b1200640 1 --2- 172.21.15.32:0/2785021022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9ac1624e0 0x7fd9ac0b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9a3400640 1 -- 172.21.15.32:0/2785021022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9ac06d550 msgr2=0x7fd9ac09fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9a3400640 1 --2- 172.21.15.32:0/2785021022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9ac06d550 0x7fd9ac09fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9a3400640 1 -- 172.21.15.32:0/2785021022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9ac1624e0 msgr2=0x7fd9ac0b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9a3400640 1 --2- 172.21.15.32:0/2785021022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9ac1624e0 0x7fd9ac0b6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9a3400640 1 -- 172.21.15.32:0/2785021022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ac157a40 con 0x7fd9ac09ba30 2024-04-17T10:04:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9a3400640 1 --2- 172.21.15.32:0/2785021022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9ac09ba30 0x7fd9ac156750 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fd99400efc0 tx=0x7fd99400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9a8c00640 1 -- 172.21.15.32:0/2785021022 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994018020 con 0x7fd9ac09ba30 2024-04-17T10:04:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9a8c00640 1 -- 172.21.15.32:0/2785021022 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd994014480 con 0x7fd9ac09ba30 2024-04-17T10:04:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9a8c00640 1 -- 172.21.15.32:0/2785021022 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994014850 con 0x7fd9ac09ba30 2024-04-17T10:04:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9b3200640 1 -- 172.21.15.32:0/2785021022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9ac1571a0 con 0x7fd9ac09ba30 2024-04-17T10:04:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.622+0000 7fd9b3200640 1 -- 172.21.15.32:0/2785021022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9ac157760 con 0x7fd9ac09ba30 2024-04-17T10:04:47.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.626+0000 7fd9b3200640 1 -- 172.21.15.32:0/2785021022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9ac068a90 con 0x7fd9ac09ba30 2024-04-17T10:04:47.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.626+0000 7fd9a8c00640 1 -- 172.21.15.32:0/2785021022 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd99401b020 con 0x7fd9ac09ba30 2024-04-17T10:04:47.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.626+0000 7fd9a8c00640 1 --2- 172.21.15.32:0/2785021022 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd988068380 0x7fd98806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.626+0000 7fd9a8c00640 1 -- 172.21.15.32:0/2785021022 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7fd994087570 con 0x7fd9ac09ba30 2024-04-17T10:04:47.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.626+0000 7fd9abe00640 1 --2- 172.21.15.32:0/2785021022 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd988068380 0x7fd98806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.626+0000 7fd9abe00640 1 --2- 172.21.15.32:0/2785021022 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd988068380 0x7fd98806a840 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0ad000 tx=0x7fd99c0a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:47.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.630+0000 7fd9a8c00640 1 -- 172.21.15.32:0/2785021022 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fd994022030 con 0x7fd9ac09ba30 2024-04-17T10:04:47.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.794+0000 7fd9b3200640 1 -- 172.21.15.32:0/2785021022 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fd9ac06d330 con 0x7fd988068380 2024-04-17T10:04:47.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.794+0000 7fd9a8c00640 1 -- 172.21.15.32:0/2785021022 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd9ac06d330 con 0x7fd988068380 2024-04-17T10:04:47.791 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 4 to deep-scrub 2024-04-17T10:04:47.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.798+0000 7fd9b3200640 1 -- 172.21.15.32:0/2785021022 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd988068380 msgr2=0x7fd98806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.798+0000 7fd9b3200640 1 --2- 172.21.15.32:0/2785021022 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd988068380 0x7fd98806a840 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0ad000 tx=0x7fd99c0a6000 comp rx=0 tx=0).stop 2024-04-17T10:04:47.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.798+0000 7fd9b3200640 1 -- 172.21.15.32:0/2785021022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9ac09ba30 msgr2=0x7fd9ac156750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.798+0000 7fd9b3200640 1 --2- 172.21.15.32:0/2785021022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9ac09ba30 0x7fd9ac156750 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fd99400efc0 tx=0x7fd99400c490 comp rx=0 tx=0).stop 2024-04-17T10:04:47.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.798+0000 7fd9b3200640 1 -- 172.21.15.32:0/2785021022 shutdown_connections 2024-04-17T10:04:47.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.798+0000 7fd9b3200640 1 --2- 172.21.15.32:0/2785021022 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fd988068380 0x7fd98806a840 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.798+0000 7fd9b3200640 1 --2- 172.21.15.32:0/2785021022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9ac1624e0 0x7fd9ac0b6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.798+0000 7fd9b3200640 1 --2- 172.21.15.32:0/2785021022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9ac09ba30 0x7fd9ac156750 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.798+0000 7fd9b3200640 1 --2- 172.21.15.32:0/2785021022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9ac06d550 0x7fd9ac09fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.798+0000 7fd9b3200640 1 -- 172.21.15.32:0/2785021022 >> 172.21.15.32:0/2785021022 conn(0x7fd9ac0b3ff0 msgr2=0x7fd9ac160ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:47.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.798+0000 7fd9b3200640 1 -- 172.21.15.32:0/2785021022 shutdown_connections 2024-04-17T10:04:47.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.798+0000 7fd9b3200640 1 -- 172.21.15.32:0/2785021022 wait complete. 2024-04-17T10:04:47.812 INFO:tasks.ceph:Scrubbing osd.5 2024-04-17T10:04:47.812 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-17T10:04:47.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.962+0000 7f9c1de00640 1 Processor -- start 2024-04-17T10:04:47.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.962+0000 7f9c1de00640 1 -- start start 2024-04-17T10:04:47.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.966+0000 7f9c1de00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9c180d03b0 0x7f9c180ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.966+0000 7f9c1de00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c1815d200 con 0x7f9c1806fda0 2024-04-17T10:04:47.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.966+0000 7f9c1de00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c1815d4e0 con 0x7f9c1815ccd0 2024-04-17T10:04:47.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.966+0000 7f9c1de00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f9c1815d7c0 con 0x7f9c180d03b0 2024-04-17T10:04:47.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.966+0000 7f9c16a00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9c1806fda0 0x7f9c180cfca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34054/0 (socket says 172.21.15.32:34054) 2024-04-17T10:04:47.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.966+0000 7f9c16a00640 1 -- 172.21.15.32:0/2671248402 learned_addr learned my addr 172.21.15.32:0/2671248402 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:47.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.966+0000 7f9c16000640 1 --2- 172.21.15.32:0/2671248402 >> v2:172.21.15.144:3300/0 conn(0x7f9c180d03b0 0x7f9c180ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.966+0000 7f9c15600640 1 -- 172.21.15.32:0/2671248402 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1207328016 0 0) 0x7f9c1815d200 con 0x7f9c1806fda0 2024-04-17T10:04:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.966+0000 7f9c15600640 1 -- 172.21.15.32:0/2671248402 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bfc003530 con 0x7f9c1806fda0 2024-04-17T10:04:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.966+0000 7f9c15600640 1 -- 172.21.15.32:0/2671248402 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2685606131 0 0) 0x7f9c1815d4e0 con 0x7f9c1815ccd0 2024-04-17T10:04:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.966+0000 7f9c15600640 1 -- 172.21.15.32:0/2671248402 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c1815d200 con 0x7f9c1815ccd0 2024-04-17T10:04:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.966+0000 7f9c15600640 1 -- 172.21.15.32:0/2671248402 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 689836444 0 0) 0x7f9bfc003530 con 0x7f9c1806fda0 2024-04-17T10:04:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.966+0000 7f9c16000640 1 -- 172.21.15.32:0/2671248402 >> v1:172.21.15.144:6789/0 conn(0x7f9c1815ccd0 legacy=0x7f9c1806be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.966+0000 7f9c16000640 1 -- 172.21.15.32:0/2671248402 >> v1:172.21.15.32:6789/0 conn(0x7f9c1806fda0 legacy=0x7f9c180cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.966+0000 7f9c16000640 1 -- 172.21.15.32:0/2671248402 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1815daa0 con 0x7f9c180d03b0 2024-04-17T10:04:47.963 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.970+0000 7f9c16000640 1 --2- 172.21.15.32:0/2671248402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c180d03b0 0x7f9c180ce410 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9c040097c0 tx=0x7f9c0401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94fcc41b696d26d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.970+0000 7f9c15600640 1 -- 172.21.15.32:0/2671248402 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c040b0070 con 0x7f9c180d03b0 2024-04-17T10:04:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.970+0000 7f9c15600640 1 -- 172.21.15.32:0/2671248402 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c040ab400 con 0x7f9c180d03b0 2024-04-17T10:04:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.970+0000 7f9c15600640 1 -- 172.21.15.32:0/2671248402 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08001680 con 0x7f9c180d03b0 2024-04-17T10:04:47.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.970+0000 7f9c1de00640 1 -- 172.21.15.32:0/2671248402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c180d03b0 msgr2=0x7f9c180ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.970+0000 7f9c1de00640 1 --2- 172.21.15.32:0/2671248402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c180d03b0 0x7f9c180ce410 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9c040097c0 tx=0x7f9c0401c3c0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.970+0000 7f9c1de00640 1 -- 172.21.15.32:0/2671248402 shutdown_connections 2024-04-17T10:04:47.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.970+0000 7f9c1de00640 1 --2- 172.21.15.32:0/2671248402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c180d03b0 0x7f9c180ce410 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.970+0000 7f9c1de00640 1 -- 172.21.15.32:0/2671248402 >> 172.21.15.32:0/2671248402 conn(0x7f9c180cd6e0 msgr2=0x7f9c180cdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:47.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.970+0000 7f9c1de00640 1 -- 172.21.15.32:0/2671248402 shutdown_connections 2024-04-17T10:04:47.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.970+0000 7f9c1de00640 1 -- 172.21.15.32:0/2671248402 wait complete. 2024-04-17T10:04:47.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.970+0000 7f9c1de00640 1 Processor -- start 2024-04-17T10:04:47.966 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.970+0000 7f9c1de00640 1 -- start start 2024-04-17T10:04:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.970+0000 7f9c1de00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1806fda0 0x7f9c1814f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.970+0000 7f9c1de00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c180d03b0 0x7f9c180b4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.970+0000 7f9c1de00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c1815ccd0 0x7f9c181520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.970+0000 7f9c1de00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1806d170 con 0x7f9c1806fda0 2024-04-17T10:04:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.970+0000 7f9c1de00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1806cff0 con 0x7f9c1815ccd0 2024-04-17T10:04:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.970+0000 7f9c1de00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9c1816a6c0 con 0x7f9c180d03b0 2024-04-17T10:04:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.974+0000 7f9c16a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1806fda0 0x7f9c1814f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.974+0000 7f9c16a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1806fda0 0x7f9c1814f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44294/0 (socket says 172.21.15.32:44294) 2024-04-17T10:04:47.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.974+0000 7f9c16a00640 1 -- 172.21.15.32:0/813054971 learned_addr learned my addr 172.21.15.32:0/813054971 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.974+0000 7f9c17400640 1 --2- 172.21.15.32:0/813054971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c1815ccd0 0x7f9c181520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.974+0000 7f9c16000640 1 --2- 172.21.15.32:0/813054971 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c180d03b0 0x7f9c180b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.974+0000 7f9c16a00640 1 -- 172.21.15.32:0/813054971 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c180d03b0 msgr2=0x7f9c180b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.974+0000 7f9c16a00640 1 --2- 172.21.15.32:0/813054971 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c180d03b0 0x7f9c180b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.974+0000 7f9c16a00640 1 -- 172.21.15.32:0/813054971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c1815ccd0 msgr2=0x7f9c181520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.974+0000 7f9c16a00640 1 --2- 172.21.15.32:0/813054971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c1815ccd0 0x7f9c181520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.974+0000 7f9c16a00640 1 -- 172.21.15.32:0/813054971 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c180b4cf0 con 0x7f9c1806fda0 2024-04-17T10:04:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.974+0000 7f9c17400640 1 --2- 172.21.15.32:0/813054971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c1815ccd0 0x7f9c181520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.974+0000 7f9c16000640 1 --2- 172.21.15.32:0/813054971 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c180d03b0 0x7f9c180b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:04:47.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.974+0000 7f9c16a00640 1 --2- 172.21.15.32:0/813054971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1806fda0 0x7f9c1814f310 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c750 tx=0x7f9c0800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:47.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.974+0000 7f9c0f400640 1 -- 172.21.15.32:0/813054971 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08005af0 con 0x7f9c1806fda0 2024-04-17T10:04:47.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.974+0000 7f9c0f400640 1 -- 172.21.15.32:0/813054971 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c08005c90 con 0x7f9c1806fda0 2024-04-17T10:04:47.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.974+0000 7f9c1de00640 1 -- 172.21.15.32:0/813054971 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c180b4fd0 con 0x7f9c1806fda0 2024-04-17T10:04:47.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.974+0000 7f9c0f400640 1 -- 172.21.15.32:0/813054971 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c080072b0 con 0x7f9c1806fda0 2024-04-17T10:04:47.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.974+0000 7f9c1de00640 1 -- 172.21.15.32:0/813054971 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c180b5590 con 0x7f9c1806fda0 2024-04-17T10:04:47.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.974+0000 7f9c1de00640 1 -- 172.21.15.32:0/813054971 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9c18068a90 con 0x7f9c1806fda0 2024-04-17T10:04:47.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.974+0000 7f9c0f400640 1 -- 172.21.15.32:0/813054971 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9c08007450 con 0x7f9c1806fda0 2024-04-17T10:04:47.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.978+0000 7f9c0f400640 1 --2- 172.21.15.32:0/813054971 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9bf8068380 0x7f9bf806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.978+0000 7f9c0f400640 1 -- 172.21.15.32:0/813054971 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f9c08086660 con 0x7f9c1806fda0 2024-04-17T10:04:47.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.978+0000 7f9c16000640 1 --2- 172.21.15.32:0/813054971 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9bf8068380 0x7f9bf806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.978+0000 7f9c0f400640 1 --2- 172.21.15.32:0/813054971 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x7f9bf806e380 0x7f9bf80707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:47.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.978+0000 7f9c0f400640 1 -- 172.21.15.32:0/813054971 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9c08014e50 con 0x7f9bf806e380 2024-04-17T10:04:47.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.978+0000 7f9c0f400640 1 -- 172.21.15.32:0/813054971 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7f9c0808d050 con 0x7f9c1806fda0 2024-04-17T10:04:47.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.978+0000 7f9c17400640 1 --2- 172.21.15.32:0/813054971 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x7f9bf806e380 0x7f9bf80707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:47.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.978+0000 7f9c16000640 1 --2- 172.21.15.32:0/813054971 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9bf8068380 0x7f9bf806a840 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9c040ad000 tx=0x7f9c04008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:47.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.978+0000 7f9c17400640 1 --2- 172.21.15.32:0/813054971 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x7f9bf806e380 0x7f9bf80707a0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:47.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.978+0000 7f9c0f400640 1 -- 172.21.15.32:0/813054971 <== osd.5 v2:172.21.15.144:6800/2379578648 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9c08014e50 con 0x7f9bf806e380 2024-04-17T10:04:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.990+0000 7f9c1de00640 1 -- 172.21.15.32:0/813054971 --> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9c1806c140 con 0x7f9bf806e380 2024-04-17T10:04:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.990+0000 7f9c0f400640 1 -- 172.21.15.32:0/813054971 <== osd.5 v2:172.21.15.144:6800/2379578648 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9c1806c140 con 0x7f9bf806e380 2024-04-17T10:04:47.988 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:04:47.988 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T10:04:47.988 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:04:47.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.994+0000 7f9c1de00640 1 -- 172.21.15.32:0/813054971 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x7f9bf806e380 msgr2=0x7f9bf80707a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.994+0000 7f9c1de00640 1 --2- 172.21.15.32:0/813054971 >> [v2:172.21.15.144:6800/2379578648,v1:172.21.15.144:6801/2379578648] conn(0x7f9bf806e380 0x7f9bf80707a0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:47.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.994+0000 7f9c1de00640 1 -- 172.21.15.32:0/813054971 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9bf8068380 msgr2=0x7f9bf806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.994+0000 7f9c1de00640 1 --2- 172.21.15.32:0/813054971 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f9bf8068380 0x7f9bf806a840 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9c040ad000 tx=0x7f9c04008040 comp rx=0 tx=0).stop 2024-04-17T10:04:47.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.994+0000 7f9c1de00640 1 -- 172.21.15.32:0/813054971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1806fda0 msgr2=0x7f9c1814f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:47.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.994+0000 7f9c1de00640 1 --2- 172.21.15.32:0/813054971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1806fda0 0x7f9c1814f310 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c750 tx=0x7f9c0800cc10 comp rx=0 tx=0).stop 2024-04-17T10:04:47.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.994+0000 7f9c17400640 1 -- 172.21.15.32:0/813054971 reap_dead start 2024-04-17T10:04:47.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.994+0000 7f9c1de00640 1 -- 172.21.15.32:0/813054971 shutdown_connections 2024-04-17T10:04:47.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.994+0000 7f9c1de00640 1 -- 172.21.15.32:0/813054971 >> 172.21.15.32:0/813054971 conn(0x7f9c180cd6e0 msgr2=0x7f9c180b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:47.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.994+0000 7f9c1de00640 1 -- 172.21.15.32:0/813054971 shutdown_connections 2024-04-17T10:04:47.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:47.994+0000 7f9c1de00640 1 -- 172.21.15.32:0/813054971 wait complete. 2024-04-17T10:04:48.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-17T10:04:48.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.150+0000 7ff393e00640 1 Processor -- start 2024-04-17T10:04:48.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.150+0000 7ff393e00640 1 -- start start 2024-04-17T10:04:48.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.154+0000 7ff393e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff38c13f6e0 0x7ff38c0cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:48.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.154+0000 7ff393e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff38c0d09f0 0x7ff38c071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:48.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.154+0000 7ff393e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff38c1601e0 con 0x7ff38c0cf370 2024-04-17T10:04:48.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.154+0000 7ff393e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff38c13fac0 con 0x7ff38c0d09f0 2024-04-17T10:04:48.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.154+0000 7ff393e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff38c160480 con 0x7ff38c13f6e0 2024-04-17T10:04:48.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.154+0000 7ff391400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff38c13f6e0 0x7ff38c0cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:48.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.154+0000 7ff390a00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7ff38c0cf370 0x7ff38c0d02e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.32:36750/0 (socket says 172.21.15.32:36750) 2024-04-17T10:04:48.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.154+0000 7ff391e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff38c0d09f0 0x7ff38c071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:48.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.154+0000 7ff390a00640 1 -- 172.21.15.32:0/1436723821 learned_addr learned my addr 172.21.15.32:0/1436723821 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:48.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.154+0000 7ff391400640 1 -- 172.21.15.32:0/1436723821 >> v2:172.21.15.144:3300/0 conn(0x7ff38c0d09f0 msgr2=0x7ff38c071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:48.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.154+0000 7ff391400640 1 --2- 172.21.15.32:0/1436723821 >> v2:172.21.15.144:3300/0 conn(0x7ff38c0d09f0 0x7ff38c071d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.154+0000 7ff391400640 1 -- 172.21.15.32:0/1436723821 >> v1:172.21.15.144:6789/0 conn(0x7ff38c0cf370 legacy=0x7ff38c0d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:48.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.154+0000 7ff391400640 1 -- 172.21.15.32:0/1436723821 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38c160760 con 0x7ff38c13f6e0 2024-04-17T10:04:48.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.158+0000 7ff391400640 1 --2- 172.21.15.32:0/1436723821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff38c13f6e0 0x7ff38c0cf850 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff3800097c0 tx=0x7ff38001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aa6a0125816169d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:48.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.158+0000 7ff38be00640 1 -- 172.21.15.32:0/1436723821 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3800a7070 con 0x7ff38c13f6e0 2024-04-17T10:04:48.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.158+0000 7ff38be00640 1 -- 172.21.15.32:0/1436723821 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3800b0070 con 0x7ff38c13f6e0 2024-04-17T10:04:48.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.158+0000 7ff38be00640 1 -- 172.21.15.32:0/1436723821 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3800ab890 con 0x7ff38c13f6e0 2024-04-17T10:04:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.158+0000 7ff393e00640 1 -- 172.21.15.32:0/1436723821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff38c13f6e0 msgr2=0x7ff38c0cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.158+0000 7ff393e00640 1 --2- 172.21.15.32:0/1436723821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff38c13f6e0 0x7ff38c0cf850 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff3800097c0 tx=0x7ff38001c470 comp rx=0 tx=0).stop 2024-04-17T10:04:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.158+0000 7ff393e00640 1 -- 172.21.15.32:0/1436723821 shutdown_connections 2024-04-17T10:04:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.158+0000 7ff393e00640 1 --2- 172.21.15.32:0/1436723821 >> v2:172.21.15.144:3300/0 conn(0x7ff38c0d09f0 0x7ff38c071d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.158+0000 7ff393e00640 1 --2- 172.21.15.32:0/1436723821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff38c13f6e0 0x7ff38c0cf850 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.158+0000 7ff393e00640 1 -- 172.21.15.32:0/1436723821 >> 172.21.15.32:0/1436723821 conn(0x7ff38c06ba30 msgr2=0x7ff38c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.158+0000 7ff393e00640 1 -- 172.21.15.32:0/1436723821 shutdown_connections 2024-04-17T10:04:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.158+0000 7ff393e00640 1 -- 172.21.15.32:0/1436723821 wait complete. 2024-04-17T10:04:48.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.158+0000 7ff393e00640 1 Processor -- start 2024-04-17T10:04:48.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.158+0000 7ff393e00640 1 -- start start 2024-04-17T10:04:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.162+0000 7ff393e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff38c0cf370 0x7ff38c13c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.162+0000 7ff393e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff38c0d09f0 0x7ff38c15f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.162+0000 7ff393e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff38c13f6e0 0x7ff38c138cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.162+0000 7ff393e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c0cff10 con 0x7ff38c13f6e0 2024-04-17T10:04:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.162+0000 7ff393e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c0cfd90 con 0x7ff38c0d09f0 2024-04-17T10:04:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.162+0000 7ff393e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff38c072ad0 con 0x7ff38c0cf370 2024-04-17T10:04:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.162+0000 7ff391e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff38c13f6e0 0x7ff38c138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.162+0000 7ff391400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff38c0cf370 0x7ff38c13c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.162+0000 7ff391e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff38c13f6e0 0x7ff38c138cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44308/0 (socket says 172.21.15.32:44308) 2024-04-17T10:04:48.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.162+0000 7ff391e00640 1 -- 172.21.15.32:0/1919759415 learned_addr learned my addr 172.21.15.32:0/1919759415 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:48.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.162+0000 7ff390a00640 1 --2- 172.21.15.32:0/1919759415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff38c0d09f0 0x7ff38c15f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:48.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.162+0000 7ff391e00640 1 -- 172.21.15.32:0/1919759415 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff38c0cf370 msgr2=0x7ff38c13c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:48.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.162+0000 7ff391e00640 1 --2- 172.21.15.32:0/1919759415 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff38c0cf370 0x7ff38c13c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.162+0000 7ff391e00640 1 -- 172.21.15.32:0/1919759415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff38c0d09f0 msgr2=0x7ff38c15f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:48.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.162+0000 7ff391e00640 1 --2- 172.21.15.32:0/1919759415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff38c0d09f0 0x7ff38c15f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.162+0000 7ff391e00640 1 -- 172.21.15.32:0/1919759415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38c15e810 con 0x7ff38c13f6e0 2024-04-17T10:04:48.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.162+0000 7ff391e00640 1 --2- 172.21.15.32:0/1919759415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff38c13f6e0 0x7ff38c138cf0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7ff38400efc0 tx=0x7ff38400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:48.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.162+0000 7ff389600640 1 -- 172.21.15.32:0/1919759415 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384018020 con 0x7ff38c13f6e0 2024-04-17T10:04:48.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.162+0000 7ff389600640 1 -- 172.21.15.32:0/1919759415 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff384014480 con 0x7ff38c13f6e0 2024-04-17T10:04:48.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.162+0000 7ff389600640 1 -- 172.21.15.32:0/1919759415 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384014870 con 0x7ff38c13f6e0 2024-04-17T10:04:48.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.162+0000 7ff393e00640 1 -- 172.21.15.32:0/1919759415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38c15eaf0 con 0x7ff38c13f6e0 2024-04-17T10:04:48.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.162+0000 7ff393e00640 1 -- 172.21.15.32:0/1919759415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff38c15f0b0 con 0x7ff38c13f6e0 2024-04-17T10:04:48.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.162+0000 7ff393e00640 1 -- 172.21.15.32:0/1919759415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff38c068a90 con 0x7ff38c13f6e0 2024-04-17T10:04:48.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.166+0000 7ff389600640 1 -- 172.21.15.32:0/1919759415 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff38401b020 con 0x7ff38c13f6e0 2024-04-17T10:04:48.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.166+0000 7ff389600640 1 --2- 172.21.15.32:0/1919759415 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff36c068380 0x7ff36c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:48.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.166+0000 7ff389600640 1 -- 172.21.15.32:0/1919759415 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7ff3840874f0 con 0x7ff38c13f6e0 2024-04-17T10:04:48.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.166+0000 7ff391400640 1 --2- 172.21.15.32:0/1919759415 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff36c068380 0x7ff36c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:48.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.166+0000 7ff391400640 1 --2- 172.21.15.32:0/1919759415 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff36c068380 0x7ff36c06a840 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff3800af000 tx=0x7ff3800a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:48.165 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.170+0000 7ff389600640 1 -- 172.21.15.32:0/1919759415 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7ff384022030 con 0x7ff38c13f6e0 2024-04-17T10:04:48.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.334+0000 7ff393e00640 1 -- 172.21.15.32:0/1919759415 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7ff38c0d1030 con 0x7ff36c068380 2024-04-17T10:04:48.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.334+0000 7ff389600640 1 -- 172.21.15.32:0/1919759415 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff38c0d1030 con 0x7ff36c068380 2024-04-17T10:04:48.329 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 5 to deep-scrub 2024-04-17T10:04:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.334+0000 7ff393e00640 1 -- 172.21.15.32:0/1919759415 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff36c068380 msgr2=0x7ff36c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.334+0000 7ff393e00640 1 --2- 172.21.15.32:0/1919759415 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff36c068380 0x7ff36c06a840 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff3800af000 tx=0x7ff3800a7000 comp rx=0 tx=0).stop 2024-04-17T10:04:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.334+0000 7ff393e00640 1 -- 172.21.15.32:0/1919759415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff38c13f6e0 msgr2=0x7ff38c138cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.334+0000 7ff393e00640 1 --2- 172.21.15.32:0/1919759415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff38c13f6e0 0x7ff38c138cf0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7ff38400efc0 tx=0x7ff38400c490 comp rx=0 tx=0).stop 2024-04-17T10:04:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.334+0000 7ff393e00640 1 -- 172.21.15.32:0/1919759415 shutdown_connections 2024-04-17T10:04:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.334+0000 7ff393e00640 1 --2- 172.21.15.32:0/1919759415 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7ff36c068380 0x7ff36c06a840 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.334+0000 7ff393e00640 1 --2- 172.21.15.32:0/1919759415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff38c13f6e0 0x7ff38c138cf0 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.334+0000 7ff393e00640 1 --2- 172.21.15.32:0/1919759415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff38c0d09f0 0x7ff38c15f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.334+0000 7ff393e00640 1 --2- 172.21.15.32:0/1919759415 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff38c0cf370 0x7ff38c13c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.334+0000 7ff393e00640 1 -- 172.21.15.32:0/1919759415 >> 172.21.15.32:0/1919759415 conn(0x7ff38c06ba30 msgr2=0x7ff38c141b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.334+0000 7ff393e00640 1 -- 172.21.15.32:0/1919759415 shutdown_connections 2024-04-17T10:04:48.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.338+0000 7ff393e00640 1 -- 172.21.15.32:0/1919759415 wait complete. 2024-04-17T10:04:48.350 INFO:tasks.ceph:Scrubbing osd.6 2024-04-17T10:04:48.350 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-17T10:04:48.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.498+0000 7fec22800640 1 Processor -- start 2024-04-17T10:04:48.495 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.498+0000 7fec22800640 1 -- start start 2024-04-17T10:04:48.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec22800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fec1c0cd870 0x7fec1c1661a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:48.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec22800640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec1c15f200 con 0x7fec1c06f500 2024-04-17T10:04:48.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec22800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec1c15f4e0 con 0x7fec1c0cfc70 2024-04-17T10:04:48.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec22800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fec1c15fb30 con 0x7fec1c0cd870 2024-04-17T10:04:48.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec13e00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fec1c06f500 0x7fec1c0cd160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34070/0 (socket says 172.21.15.32:34070) 2024-04-17T10:04:48.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec13e00640 1 -- 172.21.15.32:0/1293474564 learned_addr learned my addr 172.21.15.32:0/1293474564 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:48.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1be00640 1 --2- 172.21.15.32:0/1293474564 >> v2:172.21.15.144:3300/0 conn(0x7fec1c0cd870 0x7fec1c1661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:48.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1aa00640 1 -- 172.21.15.32:0/1293474564 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 855525468 0 0) 0x7fec1c15f200 con 0x7fec1c06f500 2024-04-17T10:04:48.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1aa00640 1 -- 172.21.15.32:0/1293474564 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec00005740 con 0x7fec1c06f500 2024-04-17T10:04:48.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1aa00640 1 -- 172.21.15.32:0/1293474564 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1633459526 0 0) 0x7fec00005740 con 0x7fec1c06f500 2024-04-17T10:04:48.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1aa00640 1 -- 172.21.15.32:0/1293474564 >> v2:172.21.15.144:3300/0 conn(0x7fec1c0cd870 msgr2=0x7fec1c1661a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:48.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1aa00640 1 --2- 172.21.15.32:0/1293474564 >> v2:172.21.15.144:3300/0 conn(0x7fec1c0cd870 0x7fec1c1661a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1aa00640 1 -- 172.21.15.32:0/1293474564 >> v1:172.21.15.144:6789/0 conn(0x7fec1c0cfc70 legacy=0x7fec1c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:48.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1aa00640 1 -- 172.21.15.32:0/1293474564 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1c15f7c0 con 0x7fec1c06f500 2024-04-17T10:04:48.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1aa00640 1 -- 172.21.15.32:0/1293474564 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1198833763 0 0) 0x7fec04001800 con 0x7fec1c06f500 2024-04-17T10:04:48.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1aa00640 1 -- 172.21.15.32:0/1293474564 >> v1:172.21.15.32:6789/0 conn(0x7fec1c06f500 legacy=0x7fec1c0cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:48.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1aa00640 1 --2- 172.21.15.32:0/1293474564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec00005c20 0x7fec00006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1aa00640 1 --2- 172.21.15.32:0/1293474564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec00006a20 0x7fec00006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1aa00640 1 --2- 172.21.15.32:0/1293474564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec000073e0 0x7fec0000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1aa00640 1 -- 172.21.15.32:0/1293474564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c15fb30 con 0x7fec00005c20 2024-04-17T10:04:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1aa00640 1 -- 172.21.15.32:0/1293474564 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec0000e2e0 con 0x7fec00006a20 2024-04-17T10:04:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1aa00640 1 -- 172.21.15.32:0/1293474564 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fec0000e4e0 con 0x7fec000073e0 2024-04-17T10:04:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1b400640 1 --2- 172.21.15.32:0/1293474564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec00005c20 0x7fec00006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec13e00640 1 --2- 172.21.15.32:0/1293474564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec00006a20 0x7fec00006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1be00640 1 --2- 172.21.15.32:0/1293474564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec000073e0 0x7fec0000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1b400640 1 -- 172.21.15.32:0/1293474564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec000073e0 msgr2=0x7fec0000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1b400640 1 --2- 172.21.15.32:0/1293474564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec000073e0 0x7fec0000dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1b400640 1 -- 172.21.15.32:0/1293474564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec00006a20 msgr2=0x7fec00006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1b400640 1 --2- 172.21.15.32:0/1293474564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec00006a20 0x7fec00006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1b400640 1 -- 172.21.15.32:0/1293474564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec00005740 con 0x7fec00005c20 2024-04-17T10:04:48.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.502+0000 7fec1be00640 1 -- 172.21.15.32:0/1293474564 reap_dead start 2024-04-17T10:04:48.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.506+0000 7fec1b400640 1 --2- 172.21.15.32:0/1293474564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec00005c20 0x7fec00006040 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fec0c001780 tx=0x7fec0c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e586f27e6a8477 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:48.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.506+0000 7fec1aa00640 1 -- 172.21.15.32:0/1293474564 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c0013a0 con 0x7fec00005c20 2024-04-17T10:04:48.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.506+0000 7fec1aa00640 1 -- 172.21.15.32:0/1293474564 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec0c0052f0 con 0x7fec00005c20 2024-04-17T10:04:48.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.506+0000 7fec1aa00640 1 -- 172.21.15.32:0/1293474564 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c003840 con 0x7fec00005c20 2024-04-17T10:04:48.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.506+0000 7fec22800640 1 -- 172.21.15.32:0/1293474564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec00005c20 msgr2=0x7fec00006040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:48.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.506+0000 7fec22800640 1 --2- 172.21.15.32:0/1293474564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec00005c20 0x7fec00006040 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fec0c001780 tx=0x7fec0c01c400 comp rx=0 tx=0).stop 2024-04-17T10:04:48.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.506+0000 7fec22800640 1 -- 172.21.15.32:0/1293474564 shutdown_connections 2024-04-17T10:04:48.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.506+0000 7fec22800640 1 --2- 172.21.15.32:0/1293474564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec00005c20 0x7fec00006040 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.506+0000 7fec22800640 1 -- 172.21.15.32:0/1293474564 >> 172.21.15.32:0/1293474564 conn(0x7fec1c0b2190 msgr2=0x7fec1c0b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:48.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.506+0000 7fec22800640 1 -- 172.21.15.32:0/1293474564 shutdown_connections 2024-04-17T10:04:48.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.506+0000 7fec22800640 1 -- 172.21.15.32:0/1293474564 wait complete. 2024-04-17T10:04:48.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.506+0000 7fec22800640 1 Processor -- start 2024-04-17T10:04:48.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.506+0000 7fec22800640 1 -- start start 2024-04-17T10:04:48.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec22800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec00005c20 0x7fec1c157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:48.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec22800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec1c14b1b0 0x7fec1c14ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:48.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec22800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec1c158570 0x7fec1c1589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:48.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec22800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c0725c0 con 0x7fec00005c20 2024-04-17T10:04:48.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec22800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c0d0260 con 0x7fec1c158570 2024-04-17T10:04:48.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec22800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fec1c072740 con 0x7fec1c14b1b0 2024-04-17T10:04:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec1b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec00005c20 0x7fec1c157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec1b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec00005c20 0x7fec1c157b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44336/0 (socket says 172.21.15.32:44336) 2024-04-17T10:04:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec1b400640 1 -- 172.21.15.32:0/367411866 learned_addr learned my addr 172.21.15.32:0/367411866 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec13e00640 1 --2- 172.21.15.32:0/367411866 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec1c14b1b0 0x7fec1c14ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec1be00640 1 --2- 172.21.15.32:0/367411866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec1c158570 0x7fec1c1589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec1b400640 1 -- 172.21.15.32:0/367411866 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec1c14b1b0 msgr2=0x7fec1c14ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec1b400640 1 --2- 172.21.15.32:0/367411866 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec1c14b1b0 0x7fec1c14ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec1b400640 1 -- 172.21.15.32:0/367411866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec1c158570 msgr2=0x7fec1c1589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec1b400640 1 --2- 172.21.15.32:0/367411866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec1c158570 0x7fec1c1589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec1b400640 1 -- 172.21.15.32:0/367411866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1c15d650 con 0x7fec00005c20 2024-04-17T10:04:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec13e00640 1 --2- 172.21.15.32:0/367411866 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec1c14b1b0 0x7fec1c14ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec1b400640 1 --2- 172.21.15.32:0/367411866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec00005c20 0x7fec1c157b90 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fec0c000c10 tx=0x7fec00007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec13400640 1 -- 172.21.15.32:0/367411866 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c007a40 con 0x7fec00005c20 2024-04-17T10:04:48.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec13400640 1 -- 172.21.15.32:0/367411866 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec0c0ba070 con 0x7fec00005c20 2024-04-17T10:04:48.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec13400640 1 -- 172.21.15.32:0/367411866 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c0054a0 con 0x7fec00005c20 2024-04-17T10:04:48.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec22800640 1 -- 172.21.15.32:0/367411866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec1c15d930 con 0x7fec00005c20 2024-04-17T10:04:48.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec22800640 1 -- 172.21.15.32:0/367411866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec1c15e410 con 0x7fec00005c20 2024-04-17T10:04:48.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec22800640 1 -- 172.21.15.32:0/367411866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fec1c068a90 con 0x7fec00005c20 2024-04-17T10:04:48.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.510+0000 7fec13400640 1 -- 172.21.15.32:0/367411866 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fec0c007030 con 0x7fec00005c20 2024-04-17T10:04:48.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.514+0000 7fec13400640 1 --2- 172.21.15.32:0/367411866 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7febf8068380 0x7febf806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:48.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.514+0000 7fec13400640 1 -- 172.21.15.32:0/367411866 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7fec0c120710 con 0x7fec00005c20 2024-04-17T10:04:48.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.514+0000 7fec13e00640 1 --2- 172.21.15.32:0/367411866 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7febf8068380 0x7febf806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:48.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.514+0000 7fec13400640 1 --2- 172.21.15.32:0/367411866 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x7febf806e380 0x7febf80707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:48.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.514+0000 7fec13400640 1 -- 172.21.15.32:0/367411866 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7febf8070e50 con 0x7febf806e380 2024-04-17T10:04:48.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.514+0000 7fec13400640 1 -- 172.21.15.32:0/367411866 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7fec0c127050 con 0x7fec00005c20 2024-04-17T10:04:48.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.514+0000 7fec1be00640 1 --2- 172.21.15.32:0/367411866 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x7febf806e380 0x7febf80707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:48.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.514+0000 7fec13e00640 1 --2- 172.21.15.32:0/367411866 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7febf8068380 0x7febf806a840 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fec1c1564c0 tx=0x7fec04009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:48.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.514+0000 7fec1be00640 1 --2- 172.21.15.32:0/367411866 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x7febf806e380 0x7febf80707a0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:48.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.514+0000 7fec13400640 1 -- 172.21.15.32:0/367411866 <== osd.6 v2:172.21.15.144:6808/3198738696 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7febf8070e50 con 0x7febf806e380 2024-04-17T10:04:48.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.530+0000 7fec22800640 1 -- 172.21.15.32:0/367411866 --> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fec1c072e50 con 0x7febf806e380 2024-04-17T10:04:48.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.530+0000 7fec13400640 1 -- 172.21.15.32:0/367411866 <== osd.6 v2:172.21.15.144:6808/3198738696 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fec1c072e50 con 0x7febf806e380 2024-04-17T10:04:48.524 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:04:48.524 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T10:04:48.524 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:04:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.530+0000 7fec22800640 1 -- 172.21.15.32:0/367411866 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x7febf806e380 msgr2=0x7febf80707a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.530+0000 7fec22800640 1 --2- 172.21.15.32:0/367411866 >> [v2:172.21.15.144:6808/3198738696,v1:172.21.15.144:6810/3198738696] conn(0x7febf806e380 0x7febf80707a0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.530+0000 7fec22800640 1 -- 172.21.15.32:0/367411866 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7febf8068380 msgr2=0x7febf806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.530+0000 7fec22800640 1 --2- 172.21.15.32:0/367411866 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7febf8068380 0x7febf806a840 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fec1c1564c0 tx=0x7fec04009040 comp rx=0 tx=0).stop 2024-04-17T10:04:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.530+0000 7fec22800640 1 -- 172.21.15.32:0/367411866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec00005c20 msgr2=0x7fec1c157b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.530+0000 7fec22800640 1 --2- 172.21.15.32:0/367411866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec00005c20 0x7fec1c157b90 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fec0c000c10 tx=0x7fec00007a50 comp rx=0 tx=0).stop 2024-04-17T10:04:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.530+0000 7fec1be00640 1 -- 172.21.15.32:0/367411866 reap_dead start 2024-04-17T10:04:48.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.530+0000 7fec22800640 1 -- 172.21.15.32:0/367411866 shutdown_connections 2024-04-17T10:04:48.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.530+0000 7fec22800640 1 -- 172.21.15.32:0/367411866 >> 172.21.15.32:0/367411866 conn(0x7fec1c0b2190 msgr2=0x7fec1c0b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:48.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.530+0000 7fec22800640 1 -- 172.21.15.32:0/367411866 shutdown_connections 2024-04-17T10:04:48.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.530+0000 7fec22800640 1 -- 172.21.15.32:0/367411866 wait complete. 2024-04-17T10:04:48.543 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-17T10:04:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.690+0000 7f7831200640 1 Processor -- start 2024-04-17T10:04:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.690+0000 7f7831200640 1 -- start start 2024-04-17T10:04:48.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.690+0000 7f7831200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f782c155a70 0x7f782c15f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:48.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.690+0000 7f7831200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f782c0b3340 0x7f782c15c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:48.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.690+0000 7f7831200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f782c15f870 con 0x7f782c155e50 2024-04-17T10:04:48.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.690+0000 7f7831200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f782c0b42a0 con 0x7f782c155a70 2024-04-17T10:04:48.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.690+0000 7f7831200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f782c0d1250 con 0x7f782c0b3340 2024-04-17T10:04:48.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.690+0000 7f7821400640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f782c155e50 0x7f782c15d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34084/0 (socket says 172.21.15.32:34084) 2024-04-17T10:04:48.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.690+0000 7f7821400640 1 -- 172.21.15.32:0/680037317 learned_addr learned my addr 172.21.15.32:0/680037317 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:48.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.690+0000 7f7829e00640 1 --2- 172.21.15.32:0/680037317 >> v2:172.21.15.32:3300/0 conn(0x7f782c155a70 0x7f782c15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:48.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.690+0000 7f782a800640 1 --2- 172.21.15.32:0/680037317 >> v2:172.21.15.144:3300/0 conn(0x7f782c0b3340 0x7f782c15c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:48.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.690+0000 7f7829e00640 1 -- 172.21.15.32:0/680037317 >> v2:172.21.15.144:3300/0 conn(0x7f782c0b3340 msgr2=0x7f782c15c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:48.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.690+0000 7f7829e00640 1 --2- 172.21.15.32:0/680037317 >> v2:172.21.15.144:3300/0 conn(0x7f782c0b3340 0x7f782c15c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.690+0000 7f7829e00640 1 -- 172.21.15.32:0/680037317 >> v1:172.21.15.32:6789/0 conn(0x7f782c155e50 legacy=0x7f782c15d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:48.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.690+0000 7f7829e00640 1 -- 172.21.15.32:0/680037317 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782c15fb50 con 0x7f782c155a70 2024-04-17T10:04:48.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.694+0000 7f7829e00640 1 --2- 172.21.15.32:0/680037317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f782c155a70 0x7f782c15f190 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f781c0097c0 tx=0x7f781c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3bf09615715017 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:48.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.694+0000 7f7829400640 1 -- 172.21.15.32:0/680037317 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c0b0070 con 0x7f782c155a70 2024-04-17T10:04:48.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.694+0000 7f7829400640 1 -- 172.21.15.32:0/680037317 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f781c0ab600 con 0x7f782c155a70 2024-04-17T10:04:48.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.694+0000 7f7829400640 1 -- 172.21.15.32:0/680037317 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c0ab9d0 con 0x7f782c155a70 2024-04-17T10:04:48.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.694+0000 7f7831200640 1 -- 172.21.15.32:0/680037317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f782c155a70 msgr2=0x7f782c15f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:48.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.694+0000 7f7831200640 1 --2- 172.21.15.32:0/680037317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f782c155a70 0x7f782c15f190 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f781c0097c0 tx=0x7f781c01c460 comp rx=0 tx=0).stop 2024-04-17T10:04:48.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.694+0000 7f7831200640 1 -- 172.21.15.32:0/680037317 shutdown_connections 2024-04-17T10:04:48.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.694+0000 7f7831200640 1 --2- 172.21.15.32:0/680037317 >> v2:172.21.15.144:3300/0 conn(0x7f782c0b3340 0x7f782c15c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.694+0000 7f7831200640 1 --2- 172.21.15.32:0/680037317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f782c155a70 0x7f782c15f190 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.694+0000 7f7831200640 1 -- 172.21.15.32:0/680037317 >> 172.21.15.32:0/680037317 conn(0x7f782c06ba30 msgr2=0x7f782c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:48.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.694+0000 7f7831200640 1 -- 172.21.15.32:0/680037317 shutdown_connections 2024-04-17T10:04:48.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.694+0000 7f7831200640 1 -- 172.21.15.32:0/680037317 wait complete. 2024-04-17T10:04:48.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.694+0000 7f7831200640 1 Processor -- start 2024-04-17T10:04:48.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.694+0000 7f7831200640 1 -- start start 2024-04-17T10:04:48.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f7831200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f782c0b3340 0x7f782c06dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:48.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f7831200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f782c155a70 0x7f782c0ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:48.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f7831200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f782c155e50 0x7f782c0aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:48.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f7831200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f782c0d02f0 con 0x7f782c155e50 2024-04-17T10:04:48.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f7831200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f782c0d0170 con 0x7f782c155a70 2024-04-17T10:04:48.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f7831200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f782c0d0470 con 0x7f782c0b3340 2024-04-17T10:04:48.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f782a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f782c155e50 0x7f782c0aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:48.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f782a800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f782c155e50 0x7f782c0aeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44354/0 (socket says 172.21.15.32:44354) 2024-04-17T10:04:48.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f782a800640 1 -- 172.21.15.32:0/3087987933 learned_addr learned my addr 172.21.15.32:0/3087987933 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:48.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f7829e00640 1 --2- 172.21.15.32:0/3087987933 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f782c0b3340 0x7f782c06dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:48.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f7821400640 1 --2- 172.21.15.32:0/3087987933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f782c155a70 0x7f782c0ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:48.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f782a800640 1 -- 172.21.15.32:0/3087987933 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f782c0b3340 msgr2=0x7f782c06dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:48.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f782a800640 1 --2- 172.21.15.32:0/3087987933 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f782c0b3340 0x7f782c06dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f782a800640 1 -- 172.21.15.32:0/3087987933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f782c155a70 msgr2=0x7f782c0ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:48.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f782a800640 1 --2- 172.21.15.32:0/3087987933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f782c155a70 0x7f782c0ae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f782a800640 1 -- 172.21.15.32:0/3087987933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782c1593b0 con 0x7f782c155e50 2024-04-17T10:04:48.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f7829e00640 1 --2- 172.21.15.32:0/3087987933 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f782c0b3340 0x7f782c06dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:48.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f7821400640 1 --2- 172.21.15.32:0/3087987933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f782c155a70 0x7f782c0ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:48.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f782a800640 1 --2- 172.21.15.32:0/3087987933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f782c155e50 0x7f782c0aeb20 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f7824007e40 tx=0x7f782400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:48.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f7822a00640 1 -- 172.21.15.32:0/3087987933 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7824014620 con 0x7f782c155e50 2024-04-17T10:04:48.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f7822a00640 1 -- 172.21.15.32:0/3087987933 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7824019070 con 0x7f782c155e50 2024-04-17T10:04:48.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f7831200640 1 -- 172.21.15.32:0/3087987933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f782c1585f0 con 0x7f782c155e50 2024-04-17T10:04:48.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f7822a00640 1 -- 172.21.15.32:0/3087987933 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7824014a20 con 0x7f782c155e50 2024-04-17T10:04:48.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f7831200640 1 -- 172.21.15.32:0/3087987933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f782c158bb0 con 0x7f782c155e50 2024-04-17T10:04:48.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.698+0000 7f7831200640 1 -- 172.21.15.32:0/3087987933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f782c068a90 con 0x7f782c155e50 2024-04-17T10:04:48.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.702+0000 7f7822a00640 1 -- 172.21.15.32:0/3087987933 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7824014bc0 con 0x7f782c155e50 2024-04-17T10:04:48.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.702+0000 7f7822a00640 1 --2- 172.21.15.32:0/3087987933 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7808068380 0x7f780806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:48.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.702+0000 7f7822a00640 1 -- 172.21.15.32:0/3087987933 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f7824087070 con 0x7f782c155e50 2024-04-17T10:04:48.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.702+0000 7f7829e00640 1 --2- 172.21.15.32:0/3087987933 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7808068380 0x7f780806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:48.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.702+0000 7f7829e00640 1 --2- 172.21.15.32:0/3087987933 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7808068380 0x7f780806a840 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f781c0ad000 tx=0x7f781c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:48.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.706+0000 7f7822a00640 1 -- 172.21.15.32:0/3087987933 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f7824057870 con 0x7f782c155e50 2024-04-17T10:04:48.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.870+0000 7f7831200640 1 -- 172.21.15.32:0/3087987933 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f782c0ccb40 con 0x7f7808068380 2024-04-17T10:04:48.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.870+0000 7f7822a00640 1 -- 172.21.15.32:0/3087987933 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f782c0ccb40 con 0x7f7808068380 2024-04-17T10:04:48.866 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 6 to deep-scrub 2024-04-17T10:04:48.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.870+0000 7f7831200640 1 -- 172.21.15.32:0/3087987933 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7808068380 msgr2=0x7f780806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:48.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.870+0000 7f7831200640 1 --2- 172.21.15.32:0/3087987933 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7808068380 0x7f780806a840 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f781c0ad000 tx=0x7f781c008040 comp rx=0 tx=0).stop 2024-04-17T10:04:48.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.870+0000 7f7831200640 1 -- 172.21.15.32:0/3087987933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f782c155e50 msgr2=0x7f782c0aeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:48.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.870+0000 7f7831200640 1 --2- 172.21.15.32:0/3087987933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f782c155e50 0x7f782c0aeb20 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f7824007e40 tx=0x7f782400e4b0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.874+0000 7f7831200640 1 -- 172.21.15.32:0/3087987933 shutdown_connections 2024-04-17T10:04:48.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.874+0000 7f7831200640 1 --2- 172.21.15.32:0/3087987933 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f7808068380 0x7f780806a840 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.874+0000 7f7831200640 1 --2- 172.21.15.32:0/3087987933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f782c155e50 0x7f782c0aeb20 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.874+0000 7f7831200640 1 --2- 172.21.15.32:0/3087987933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f782c155a70 0x7f782c0ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.874+0000 7f7831200640 1 --2- 172.21.15.32:0/3087987933 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f782c0b3340 0x7f782c06dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:48.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.874+0000 7f7831200640 1 -- 172.21.15.32:0/3087987933 >> 172.21.15.32:0/3087987933 conn(0x7f782c06ba30 msgr2=0x7f782c155100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:48.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.874+0000 7f7831200640 1 -- 172.21.15.32:0/3087987933 shutdown_connections 2024-04-17T10:04:48.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:48.874+0000 7f7831200640 1 -- 172.21.15.32:0/3087987933 wait complete. 2024-04-17T10:04:48.887 INFO:tasks.ceph:Scrubbing osd.7 2024-04-17T10:04:48.887 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-17T10:04:49.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.038+0000 7f57a3200640 1 Processor -- start 2024-04-17T10:04:49.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.038+0000 7f57a3200640 1 -- start start 2024-04-17T10:04:49.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.038+0000 7f57a3200640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f579c0cd870 0x7f579c1661a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:49.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.038+0000 7f57a3200640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f579c15f200 con 0x7f579c0cfc70 2024-04-17T10:04:49.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.038+0000 7f57a3200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f579c15f4e0 con 0x7f579c06f500 2024-04-17T10:04:49.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.038+0000 7f57a3200640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f579c15fb30 con 0x7f579c0cd870 2024-04-17T10:04:49.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.038+0000 7f579be00640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f579c0cfc70 0x7f579c06edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34100/0 (socket says 172.21.15.32:34100) 2024-04-17T10:04:49.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.038+0000 7f579be00640 1 -- 172.21.15.32:0/1713502345 learned_addr learned my addr 172.21.15.32:0/1713502345 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:04:49.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.038+0000 7f57a1200640 1 --2- 172.21.15.32:0/1713502345 >> v2:172.21.15.32:3300/0 conn(0x7f579c0cd870 0x7f579c1661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:49.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.038+0000 7f579aa00640 1 -- 172.21.15.32:0/1713502345 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 451636913 0 0) 0x7f579c15f200 con 0x7f579c0cfc70 2024-04-17T10:04:49.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.038+0000 7f579aa00640 1 -- 172.21.15.32:0/1713502345 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5780003530 con 0x7f579c0cfc70 2024-04-17T10:04:49.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.038+0000 7f57a1200640 1 -- 172.21.15.32:0/1713502345 >> v1:172.21.15.144:6789/0 conn(0x7f579c06f500 legacy=0x7f579c0cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:49.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.038+0000 7f579aa00640 1 -- 172.21.15.32:0/1713502345 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 632239846 0 0) 0x7f579c15f4e0 con 0x7f579c06f500 2024-04-17T10:04:49.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.038+0000 7f57a1200640 1 -- 172.21.15.32:0/1713502345 >> v1:172.21.15.32:6789/0 conn(0x7f579c0cfc70 legacy=0x7f579c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:49.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.038+0000 7f57a1200640 1 -- 172.21.15.32:0/1713502345 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579c15f7c0 con 0x7f579c0cd870 2024-04-17T10:04:49.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.042+0000 7f57a1200640 1 --2- 172.21.15.32:0/1713502345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f579c0cd870 0x7f579c1661a0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f5794009d90 tx=0x7f579401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8842b115840dff5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:49.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.042+0000 7f579aa00640 1 -- 172.21.15.32:0/1713502345 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57940b2070 con 0x7f579c0cd870 2024-04-17T10:04:49.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.042+0000 7f579aa00640 1 -- 172.21.15.32:0/1713502345 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57940a5940 con 0x7f579c0cd870 2024-04-17T10:04:49.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.042+0000 7f579aa00640 1 -- 172.21.15.32:0/1713502345 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57940a5d10 con 0x7f579c0cd870 2024-04-17T10:04:49.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.042+0000 7f57a3200640 1 -- 172.21.15.32:0/1713502345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f579c0cd870 msgr2=0x7f579c1661a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:49.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.042+0000 7f57a3200640 1 --2- 172.21.15.32:0/1713502345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f579c0cd870 0x7f579c1661a0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f5794009d90 tx=0x7f579401c970 comp rx=0 tx=0).stop 2024-04-17T10:04:49.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.042+0000 7f57a3200640 1 -- 172.21.15.32:0/1713502345 shutdown_connections 2024-04-17T10:04:49.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.042+0000 7f57a3200640 1 --2- 172.21.15.32:0/1713502345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f579c0cd870 0x7f579c1661a0 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.042+0000 7f57a3200640 1 -- 172.21.15.32:0/1713502345 >> 172.21.15.32:0/1713502345 conn(0x7f579c0b2190 msgr2=0x7f579c0b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:49.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.042+0000 7f57a3200640 1 -- 172.21.15.32:0/1713502345 shutdown_connections 2024-04-17T10:04:49.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.042+0000 7f57a3200640 1 -- 172.21.15.32:0/1713502345 wait complete. 2024-04-17T10:04:49.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.042+0000 7f57a3200640 1 Processor -- start 2024-04-17T10:04:49.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.042+0000 7f57a3200640 1 -- start start 2024-04-17T10:04:49.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f57a3200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f579c06f500 0x7f579c0aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:49.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f57a3200640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f579c0cd870 0x7f579c1560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:49.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f57a3200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f579c0cfc70 0x7f579c144aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:49.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f57a3200640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f579c169110 con 0x7f579c0cd870 2024-04-17T10:04:49.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f57a3200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f579c0d0260 con 0x7f579c06f500 2024-04-17T10:04:49.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f57a3200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f579c169290 con 0x7f579c0cfc70 2024-04-17T10:04:49.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f579b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f579c0cd870 0x7f579c1560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:49.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f579b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f579c0cd870 0x7f579c1560a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44372/0 (socket says 172.21.15.32:44372) 2024-04-17T10:04:49.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f579b400640 1 -- 172.21.15.32:0/2595994232 learned_addr learned my addr 172.21.15.32:0/2595994232 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:49.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f579be00640 1 --2- 172.21.15.32:0/2595994232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f579c06f500 0x7f579c0aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:49.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f57a1200640 1 --2- 172.21.15.32:0/2595994232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f579c0cfc70 0x7f579c144aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:49.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f579b400640 1 -- 172.21.15.32:0/2595994232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f579c0cfc70 msgr2=0x7f579c144aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:49.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f579b400640 1 --2- 172.21.15.32:0/2595994232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f579c0cfc70 0x7f579c144aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f579b400640 1 -- 172.21.15.32:0/2595994232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f579c06f500 msgr2=0x7f579c0aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:49.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f579b400640 1 --2- 172.21.15.32:0/2595994232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f579c06f500 0x7f579c0aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f579b400640 1 -- 172.21.15.32:0/2595994232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579c158bc0 con 0x7f579c0cd870 2024-04-17T10:04:49.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f57a1200640 1 --2- 172.21.15.32:0/2595994232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f579c0cfc70 0x7f579c144aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:49.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f579b400640 1 --2- 172.21.15.32:0/2595994232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f579c0cd870 0x7f579c1560a0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f578800c480 tx=0x7f578800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:49.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f5793e00640 1 -- 172.21.15.32:0/2595994232 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5788005480 con 0x7f579c0cd870 2024-04-17T10:04:49.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f5793e00640 1 -- 172.21.15.32:0/2595994232 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5788003650 con 0x7f579c0cd870 2024-04-17T10:04:49.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f5793e00640 1 -- 172.21.15.32:0/2595994232 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5788003a40 con 0x7f579c0cd870 2024-04-17T10:04:49.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f57a3200640 1 -- 172.21.15.32:0/2595994232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f579c153990 con 0x7f579c0cd870 2024-04-17T10:04:49.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f57a3200640 1 -- 172.21.15.32:0/2595994232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f579c158ea0 con 0x7f579c0cd870 2024-04-17T10:04:49.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.046+0000 7f57a3200640 1 -- 172.21.15.32:0/2595994232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f579c068a90 con 0x7f579c0cd870 2024-04-17T10:04:49.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.050+0000 7f5793e00640 1 -- 172.21.15.32:0/2595994232 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5788003be0 con 0x7f579c0cd870 2024-04-17T10:04:49.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.050+0000 7f5793e00640 1 --2- 172.21.15.32:0/2595994232 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f577c068380 0x7f577c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:49.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.050+0000 7f5793e00640 1 -- 172.21.15.32:0/2595994232 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f57880864d0 con 0x7f579c0cd870 2024-04-17T10:04:49.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.050+0000 7f579be00640 1 --2- 172.21.15.32:0/2595994232 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f577c068380 0x7f577c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:49.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.050+0000 7f5793e00640 1 --2- 172.21.15.32:0/2595994232 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x7f577c06e380 0x7f577c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:49.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.050+0000 7f5793e00640 1 -- 172.21.15.32:0/2595994232 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f578801ce50 con 0x7f577c06e380 2024-04-17T10:04:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.050+0000 7f5793e00640 1 -- 172.21.15.32:0/2595994232 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7f578808d050 con 0x7f579c0cd870 2024-04-17T10:04:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.050+0000 7f57a1200640 1 --2- 172.21.15.32:0/2595994232 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x7f577c06e380 0x7f577c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.050+0000 7f579be00640 1 --2- 172.21.15.32:0/2595994232 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f577c068380 0x7f577c06a840 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f578c005890 tx=0x7f578c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.050+0000 7f57a1200640 1 --2- 172.21.15.32:0/2595994232 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x7f577c06e380 0x7f577c0707a0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.050+0000 7f5793e00640 1 -- 172.21.15.32:0/2595994232 <== osd.7 v2:172.21.15.144:6824/1746570140 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f578801ce50 con 0x7f577c06e380 2024-04-17T10:04:49.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.066+0000 7f57a3200640 1 -- 172.21.15.32:0/2595994232 --> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f579c0cd620 con 0x7f577c06e380 2024-04-17T10:04:49.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.066+0000 7f5793e00640 1 -- 172.21.15.32:0/2595994232 <== osd.7 v2:172.21.15.144:6824/1746570140 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f579c0cd620 con 0x7f577c06e380 2024-04-17T10:04:49.061 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-04-17T10:04:49.061 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T10:04:49.061 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-04-17T10:04:49.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.066+0000 7f57a3200640 1 -- 172.21.15.32:0/2595994232 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x7f577c06e380 msgr2=0x7f577c0707a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:49.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.066+0000 7f57a3200640 1 --2- 172.21.15.32:0/2595994232 >> [v2:172.21.15.144:6824/1746570140,v1:172.21.15.144:6825/1746570140] conn(0x7f577c06e380 0x7f577c0707a0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.066+0000 7f57a3200640 1 -- 172.21.15.32:0/2595994232 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f577c068380 msgr2=0x7f577c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:49.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.066+0000 7f57a3200640 1 --2- 172.21.15.32:0/2595994232 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f577c068380 0x7f577c06a840 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f578c005890 tx=0x7f578c009040 comp rx=0 tx=0).stop 2024-04-17T10:04:49.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.066+0000 7f57a3200640 1 -- 172.21.15.32:0/2595994232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f579c0cd870 msgr2=0x7f579c1560a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.066+0000 7f57a3200640 1 --2- 172.21.15.32:0/2595994232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f579c0cd870 0x7f579c1560a0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f578800c480 tx=0x7f578800c940 comp rx=0 tx=0).stop 2024-04-17T10:04:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.066+0000 7f57a1200640 1 -- 172.21.15.32:0/2595994232 reap_dead start 2024-04-17T10:04:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.066+0000 7f57a3200640 1 -- 172.21.15.32:0/2595994232 shutdown_connections 2024-04-17T10:04:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.066+0000 7f57a3200640 1 -- 172.21.15.32:0/2595994232 >> 172.21.15.32:0/2595994232 conn(0x7f579c0b2190 msgr2=0x7f579c0b4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.066+0000 7f57a3200640 1 -- 172.21.15.32:0/2595994232 shutdown_connections 2024-04-17T10:04:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.066+0000 7f57a3200640 1 -- 172.21.15.32:0/2595994232 wait complete. 2024-04-17T10:04:49.080 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-17T10:04:49.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.226+0000 7f1321e00640 1 Processor -- start 2024-04-17T10:04:49.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.226+0000 7f1321e00640 1 -- start start 2024-04-17T10:04:49.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.226+0000 7f1321e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f131c15b420 0x7f131c15bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:49.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.226+0000 7f1321e00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f131c0b3b30 0x7f131c0d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:49.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.226+0000 7f1321e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f131c071d80 0x7f131c0721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:49.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.226+0000 7f1321e00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f131c15c2d0 con 0x7f131c0b3b30 2024-04-17T10:04:49.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.226+0000 7f1321e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f131c072710 con 0x7f131c071d80 2024-04-17T10:04:49.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.226+0000 7f1321e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f131c0b40f0 con 0x7f131c15b420 2024-04-17T10:04:49.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.226+0000 7f1312000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f131c0b3b30 0x7f131c0d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:49.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.226+0000 7f1312000640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f131c0b3b30 0x7f131c0d0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44386/0 (socket says 172.21.15.32:44386) 2024-04-17T10:04:49.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.226+0000 7f1312000640 1 -- 172.21.15.32:0/3652931499 learned_addr learned my addr 172.21.15.32:0/3652931499 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:49.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.226+0000 7f131aa00640 1 --2- 172.21.15.32:0/3652931499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f131c15b420 0x7f131c15bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:49.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.226+0000 7f131b400640 1 --2- 172.21.15.32:0/3652931499 >> v2:172.21.15.144:3300/0 conn(0x7f131c071d80 0x7f131c0721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:49.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.226+0000 7f1312000640 1 -- 172.21.15.32:0/3652931499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f131c15b420 msgr2=0x7f131c15bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:49.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.226+0000 7f1312000640 1 --2- 172.21.15.32:0/3652931499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f131c15b420 0x7f131c15bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.226+0000 7f1312000640 1 -- 172.21.15.32:0/3652931499 >> v2:172.21.15.144:3300/0 conn(0x7f131c071d80 msgr2=0x7f131c0721d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:49.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.226+0000 7f1312000640 1 --2- 172.21.15.32:0/3652931499 >> v2:172.21.15.144:3300/0 conn(0x7f131c071d80 0x7f131c0721d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.226+0000 7f1312000640 1 -- 172.21.15.32:0/3652931499 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131c0b43d0 con 0x7f131c0b3b30 2024-04-17T10:04:49.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.226+0000 7f131b400640 1 --2- 172.21.15.32:0/3652931499 >> v2:172.21.15.144:3300/0 conn(0x7f131c071d80 0x7f131c0721d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:04:49.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.230+0000 7f1312000640 1 --2- 172.21.15.32:0/3652931499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f131c0b3b30 0x7f131c0d0350 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f131c06fdf0 tx=0x7f130401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4749f6a27497c232 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:49.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.230+0000 7f131a000640 1 -- 172.21.15.32:0/3652931499 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13040aa610 con 0x7f131c0b3b30 2024-04-17T10:04:49.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.230+0000 7f131a000640 1 -- 172.21.15.32:0/3652931499 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13040af070 con 0x7f131c0b3b30 2024-04-17T10:04:49.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.230+0000 7f131a000640 1 -- 172.21.15.32:0/3652931499 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13040aa9d0 con 0x7f131c0b3b30 2024-04-17T10:04:49.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.230+0000 7f1321e00640 1 -- 172.21.15.32:0/3652931499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f131c0b3b30 msgr2=0x7f131c0d0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:49.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.230+0000 7f1321e00640 1 --2- 172.21.15.32:0/3652931499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f131c0b3b30 0x7f131c0d0350 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f131c06fdf0 tx=0x7f130401c4f0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.230+0000 7f1321e00640 1 -- 172.21.15.32:0/3652931499 shutdown_connections 2024-04-17T10:04:49.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.230+0000 7f1321e00640 1 --2- 172.21.15.32:0/3652931499 >> v2:172.21.15.144:3300/0 conn(0x7f131c071d80 0x7f131c0721d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.230+0000 7f1321e00640 1 --2- 172.21.15.32:0/3652931499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f131c0b3b30 0x7f131c0d0350 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.230+0000 7f1321e00640 1 --2- 172.21.15.32:0/3652931499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f131c15b420 0x7f131c15bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.230+0000 7f1321e00640 1 -- 172.21.15.32:0/3652931499 >> 172.21.15.32:0/3652931499 conn(0x7f131c06ba30 msgr2=0x7f131c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:49.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.230+0000 7f1321e00640 1 -- 172.21.15.32:0/3652931499 shutdown_connections 2024-04-17T10:04:49.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.230+0000 7f1321e00640 1 -- 172.21.15.32:0/3652931499 wait complete. 2024-04-17T10:04:49.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f1321e00640 1 Processor -- start 2024-04-17T10:04:49.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f1321e00640 1 -- start start 2024-04-17T10:04:49.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f1321e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f131c071d80 0x7f131c13b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:49.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f1321e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f131c0b3b30 0x7f131c154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:49.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f1321e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f131c15b420 0x7f131c154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:49.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f1321e00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f131c0d0890 con 0x7f131c15b420 2024-04-17T10:04:49.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f1321e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f131c072ed0 con 0x7f131c0b3b30 2024-04-17T10:04:49.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f1321e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f131c0d0a10 con 0x7f131c071d80 2024-04-17T10:04:49.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f131b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f131c15b420 0x7f131c154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:49.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f131b400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f131c15b420 0x7f131c154c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44394/0 (socket says 172.21.15.32:44394) 2024-04-17T10:04:49.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f131b400640 1 -- 172.21.15.32:0/4276637945 learned_addr learned my addr 172.21.15.32:0/4276637945 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:49.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f131aa00640 1 --2- 172.21.15.32:0/4276637945 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f131c071d80 0x7f131c13b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:49.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f1312000640 1 --2- 172.21.15.32:0/4276637945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f131c0b3b30 0x7f131c154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:49.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f131b400640 1 -- 172.21.15.32:0/4276637945 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f131c071d80 msgr2=0x7f131c13b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:49.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f131b400640 1 --2- 172.21.15.32:0/4276637945 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f131c071d80 0x7f131c13b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f131b400640 1 -- 172.21.15.32:0/4276637945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f131c0b3b30 msgr2=0x7f131c154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:49.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f131b400640 1 --2- 172.21.15.32:0/4276637945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f131c0b3b30 0x7f131c154700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f131b400640 1 -- 172.21.15.32:0/4276637945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131c13bed0 con 0x7f131c15b420 2024-04-17T10:04:49.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f131b400640 1 --2- 172.21.15.32:0/4276637945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f131c15b420 0x7f131c154c40 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f1314007de0 tx=0x7f131400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:49.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f1313e00640 1 -- 172.21.15.32:0/4276637945 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131401a070 con 0x7f131c15b420 2024-04-17T10:04:49.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f1313e00640 1 -- 172.21.15.32:0/4276637945 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13140039a0 con 0x7f131c15b420 2024-04-17T10:04:49.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f1313e00640 1 -- 172.21.15.32:0/4276637945 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314003d90 con 0x7f131c15b420 2024-04-17T10:04:49.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f1321e00640 1 -- 172.21.15.32:0/4276637945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f131c13c1b0 con 0x7f131c15b420 2024-04-17T10:04:49.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.234+0000 7f1321e00640 1 -- 172.21.15.32:0/4276637945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f131c153b40 con 0x7f131c15b420 2024-04-17T10:04:49.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.238+0000 7f1321e00640 1 -- 172.21.15.32:0/4276637945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f131c068a90 con 0x7f131c15b420 2024-04-17T10:04:49.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.238+0000 7f1313e00640 1 -- 172.21.15.32:0/4276637945 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f131401d070 con 0x7f131c15b420 2024-04-17T10:04:49.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.238+0000 7f1313e00640 1 --2- 172.21.15.32:0/4276637945 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f12f80683b0 0x7f12f806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:49.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.238+0000 7f1313e00640 1 -- 172.21.15.32:0/4276637945 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f13140869e0 con 0x7f131c15b420 2024-04-17T10:04:49.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.238+0000 7f131aa00640 1 --2- 172.21.15.32:0/4276637945 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f12f80683b0 0x7f12f806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:49.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.238+0000 7f131aa00640 1 --2- 172.21.15.32:0/4276637945 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f12f80683b0 0x7f12f806a870 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f131c0b5810 tx=0x7f130c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:49.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.242+0000 7f1313e00640 1 -- 172.21.15.32:0/4276637945 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f1314010070 con 0x7f131c15b420 2024-04-17T10:04:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.406+0000 7f1321e00640 1 -- 172.21.15.32:0/4276637945 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f131c06c1c0 con 0x7f12f80683b0 2024-04-17T10:04:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.406+0000 7f1313e00640 1 -- 172.21.15.32:0/4276637945 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f131c06c1c0 con 0x7f12f80683b0 2024-04-17T10:04:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 7 to deep-scrub 2024-04-17T10:04:49.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.410+0000 7f1321e00640 1 -- 172.21.15.32:0/4276637945 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f12f80683b0 msgr2=0x7f12f806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:49.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.410+0000 7f1321e00640 1 --2- 172.21.15.32:0/4276637945 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f12f80683b0 0x7f12f806a870 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f131c0b5810 tx=0x7f130c008040 comp rx=0 tx=0).stop 2024-04-17T10:04:49.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.410+0000 7f1321e00640 1 -- 172.21.15.32:0/4276637945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f131c15b420 msgr2=0x7f131c154c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:49.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.410+0000 7f1321e00640 1 --2- 172.21.15.32:0/4276637945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f131c15b420 0x7f131c154c40 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f1314007de0 tx=0x7f131400d4a0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.410+0000 7f1321e00640 1 -- 172.21.15.32:0/4276637945 shutdown_connections 2024-04-17T10:04:49.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.410+0000 7f1321e00640 1 --2- 172.21.15.32:0/4276637945 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f12f80683b0 0x7f12f806a870 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.410+0000 7f1321e00640 1 --2- 172.21.15.32:0/4276637945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f131c15b420 0x7f131c154c40 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.410+0000 7f1321e00640 1 --2- 172.21.15.32:0/4276637945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f131c0b3b30 0x7f131c154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.410+0000 7f1321e00640 1 --2- 172.21.15.32:0/4276637945 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f131c071d80 0x7f131c13b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.410+0000 7f1321e00640 1 -- 172.21.15.32:0/4276637945 >> 172.21.15.32:0/4276637945 conn(0x7f131c06ba30 msgr2=0x7f131c15c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:49.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.410+0000 7f1321e00640 1 -- 172.21.15.32:0/4276637945 shutdown_connections 2024-04-17T10:04:49.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.410+0000 7f1321e00640 1 -- 172.21.15.32:0/4276637945 wait complete. 2024-04-17T10:04:49.424 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:04:49.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.570+0000 7f85bc800640 1 Processor -- start 2024-04-17T10:04:49.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.570+0000 7f85bc800640 1 -- start start 2024-04-17T10:04:49.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.570+0000 7f85bc800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85b415e870 0x7f85b4162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:49.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.570+0000 7f85bc800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f85b40b0320 0x7f85b40d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:49.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.570+0000 7f85bc800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85b40d0780 0x7f85b406f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:49.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.570+0000 7f85bc800640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7f85b40cf810 con 0x7f85b40d0780 2024-04-17T10:04:49.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.570+0000 7f85bc800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f85b4067dd0 con 0x7f85b40b0320 2024-04-17T10:04:49.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.570+0000 7f85bc800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f85b40cfdb0 con 0x7f85b415e870 2024-04-17T10:04:49.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.570+0000 7f85ba800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85b40d0780 0x7f85b406f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:49.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.570+0000 7f85ba800640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85b40d0780 0x7f85b406f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44410/0 (socket says 172.21.15.32:44410) 2024-04-17T10:04:49.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.570+0000 7f85ba800640 1 -- 172.21.15.32:0/1772414540 learned_addr learned my addr 172.21.15.32:0/1772414540 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:49.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.570+0000 7f85b9e00640 1 --2- 172.21.15.32:0/1772414540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85b415e870 0x7f85b4162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:49.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.570+0000 7f85b9400640 1 --2- 172.21.15.32:0/1772414540 >> v2:172.21.15.144:3300/0 conn(0x7f85b40b0320 0x7f85b40d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:49.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.570+0000 7f85ba800640 1 -- 172.21.15.32:0/1772414540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85b415e870 msgr2=0x7f85b4162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:49.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.570+0000 7f85ba800640 1 --2- 172.21.15.32:0/1772414540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85b415e870 0x7f85b4162e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.570+0000 7f85ba800640 1 -- 172.21.15.32:0/1772414540 >> v2:172.21.15.144:3300/0 conn(0x7f85b40b0320 msgr2=0x7f85b40d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:04:49.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.570+0000 7f85ba800640 1 --2- 172.21.15.32:0/1772414540 >> v2:172.21.15.144:3300/0 conn(0x7f85b40b0320 0x7f85b40d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.570+0000 7f85ba800640 1 -- 172.21.15.32:0/1772414540 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b4163340 con 0x7f85b40d0780 2024-04-17T10:04:49.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.574+0000 7f85ba800640 1 --2- 172.21.15.32:0/1772414540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b40d0780 0x7f85b406f7c0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f85b406e130 tx=0x7f85b001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f8600428ef2ae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:49.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.574+0000 7f85b8a00640 1 -- 172.21.15.32:0/1772414540 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b00a5780 con 0x7f85b40d0780 2024-04-17T10:04:49.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.574+0000 7f85b8a00640 1 -- 172.21.15.32:0/1772414540 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85b00a5920 con 0x7f85b40d0780 2024-04-17T10:04:49.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.574+0000 7f85b8a00640 1 -- 172.21.15.32:0/1772414540 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b00a5cf0 con 0x7f85b40d0780 2024-04-17T10:04:49.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.574+0000 7f85bc800640 1 -- 172.21.15.32:0/1772414540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b40d0780 msgr2=0x7f85b406f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:49.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.574+0000 7f85bc800640 1 --2- 172.21.15.32:0/1772414540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b40d0780 0x7f85b406f7c0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f85b406e130 tx=0x7f85b001c9f0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.574+0000 7f85bc800640 1 -- 172.21.15.32:0/1772414540 shutdown_connections 2024-04-17T10:04:49.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.574+0000 7f85bc800640 1 --2- 172.21.15.32:0/1772414540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b40d0780 0x7f85b406f7c0 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.574+0000 7f85bc800640 1 --2- 172.21.15.32:0/1772414540 >> v2:172.21.15.144:3300/0 conn(0x7f85b40b0320 0x7f85b40d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.574+0000 7f85bc800640 1 --2- 172.21.15.32:0/1772414540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85b415e870 0x7f85b4162e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.574+0000 7f85bc800640 1 -- 172.21.15.32:0/1772414540 >> 172.21.15.32:0/1772414540 conn(0x7f85b40d0c00 msgr2=0x7f85b40d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:49.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.574+0000 7f85bc800640 1 -- 172.21.15.32:0/1772414540 shutdown_connections 2024-04-17T10:04:49.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.574+0000 7f85bc800640 1 -- 172.21.15.32:0/1772414540 wait complete. 2024-04-17T10:04:49.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85bc800640 1 Processor -- start 2024-04-17T10:04:49.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85bc800640 1 -- start start 2024-04-17T10:04:49.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85bc800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85b40b0320 0x7f85b413b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:49.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85bc800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85b415e870 0x7f85b414e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:49.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85bc800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b414ee70 0x7f85b413e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:49.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85bc800640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f85b406e370 con 0x7f85b414ee70 2024-04-17T10:04:49.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85bc800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f85b40cfa40 con 0x7f85b415e870 2024-04-17T10:04:49.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85bc800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f85b406e4f0 con 0x7f85b40b0320 2024-04-17T10:04:49.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85ba800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b414ee70 0x7f85b413e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:49.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85ba800640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b414ee70 0x7f85b413e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44418/0 (socket says 172.21.15.32:44418) 2024-04-17T10:04:49.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85b9e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85b40b0320 0x7f85b413b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:49.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85ba800640 1 -- 172.21.15.32:0/2155662104 learned_addr learned my addr 172.21.15.32:0/2155662104 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:04:49.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85b9400640 1 --2- 172.21.15.32:0/2155662104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85b415e870 0x7f85b414e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:49.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85ba800640 1 -- 172.21.15.32:0/2155662104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85b40b0320 msgr2=0x7f85b413b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:49.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85ba800640 1 --2- 172.21.15.32:0/2155662104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85b40b0320 0x7f85b413b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85ba800640 1 -- 172.21.15.32:0/2155662104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85b415e870 msgr2=0x7f85b414e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:49.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85ba800640 1 --2- 172.21.15.32:0/2155662104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85b415e870 0x7f85b414e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85ba800640 1 -- 172.21.15.32:0/2155662104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b40505b0 con 0x7f85b414ee70 2024-04-17T10:04:49.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85ba800640 1 --2- 172.21.15.32:0/2155662104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b414ee70 0x7f85b413e7c0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f85b00af040 tx=0x7f85b0019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:49.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85ae000640 1 -- 172.21.15.32:0/2155662104 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b00b8880 con 0x7f85b414ee70 2024-04-17T10:04:49.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85ae000640 1 -- 172.21.15.32:0/2155662104 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85b00be070 con 0x7f85b414ee70 2024-04-17T10:04:49.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85ae000640 1 -- 172.21.15.32:0/2155662104 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b00b8c80 con 0x7f85b414ee70 2024-04-17T10:04:49.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85bc800640 1 -- 172.21.15.32:0/2155662104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b41511e0 con 0x7f85b414ee70 2024-04-17T10:04:49.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85bc800640 1 -- 172.21.15.32:0/2155662104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b41517a0 con 0x7f85b414ee70 2024-04-17T10:04:49.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.578+0000 7f85bc800640 1 -- 172.21.15.32:0/2155662104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b4068a90 con 0x7f85b414ee70 2024-04-17T10:04:49.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.582+0000 7f85ae000640 1 -- 172.21.15.32:0/2155662104 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f85b00c3460 con 0x7f85b414ee70 2024-04-17T10:04:49.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.582+0000 7f85ae000640 1 --2- 172.21.15.32:0/2155662104 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8598068380 0x7f859806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:04:49.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.582+0000 7f85ae000640 1 -- 172.21.15.32:0/2155662104 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f85b0121b50 con 0x7f85b414ee70 2024-04-17T10:04:49.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.582+0000 7f85b9e00640 1 --2- 172.21.15.32:0/2155662104 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8598068380 0x7f859806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:04:49.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.582+0000 7f85b9e00640 1 --2- 172.21.15.32:0/2155662104 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8598068380 0x7f859806a840 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f85a80079c0 tx=0x7f85a8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:04:49.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.586+0000 7f85ae000640 1 -- 172.21.15.32:0/2155662104 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7f85b00165e0 con 0x7f85b414ee70 2024-04-17T10:04:49.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.746+0000 7f85bc800640 1 -- 172.21.15.32:0/2155662104 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f85b40cd210 con 0x7f8598068380 2024-04-17T10:04:49.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.750+0000 7f85ae000640 1 -- 172.21.15.32:0/2155662104 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13480 (secure 0 0 0) 0x7f85b40cd210 con 0x7f8598068380 2024-04-17T10:04:49.744 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:04:49.744 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T10:04:49.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.750+0000 7f85bc800640 1 -- 172.21.15.32:0/2155662104 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8598068380 msgr2=0x7f859806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:49.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.750+0000 7f85bc800640 1 --2- 172.21.15.32:0/2155662104 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8598068380 0x7f859806a840 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f85a80079c0 tx=0x7f85a8008040 comp rx=0 tx=0).stop 2024-04-17T10:04:49.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.750+0000 7f85bc800640 1 -- 172.21.15.32:0/2155662104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b414ee70 msgr2=0x7f85b413e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:04:49.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.750+0000 7f85bc800640 1 --2- 172.21.15.32:0/2155662104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b414ee70 0x7f85b413e7c0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f85b00af040 tx=0x7f85b0019790 comp rx=0 tx=0).stop 2024-04-17T10:04:49.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.750+0000 7f85bc800640 1 -- 172.21.15.32:0/2155662104 shutdown_connections 2024-04-17T10:04:49.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.750+0000 7f85bc800640 1 --2- 172.21.15.32:0/2155662104 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7f8598068380 0x7f859806a840 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.750+0000 7f85bc800640 1 --2- 172.21.15.32:0/2155662104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b414ee70 0x7f85b413e7c0 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.750+0000 7f85bc800640 1 --2- 172.21.15.32:0/2155662104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85b415e870 0x7f85b414e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.750+0000 7f85bc800640 1 --2- 172.21.15.32:0/2155662104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85b40b0320 0x7f85b413b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:04:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.750+0000 7f85bc800640 1 -- 172.21.15.32:0/2155662104 >> 172.21.15.32:0/2155662104 conn(0x7f85b40d0c00 msgr2=0x7f85b40b4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:04:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.750+0000 7f85bc800640 1 -- 172.21.15.32:0/2155662104 shutdown_connections 2024-04-17T10:04:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:04:49.750+0000 7f85bc800640 1 -- 172.21.15.32:0/2155662104 wait complete. 2024-04-17T10:04:49.765 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":464,"stamp":"2024-04-17T10:04:49.447895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":270212,"kb_used_data":12424,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754704508,"statfs":{"total":773094113280,"available":772817416192,"internally_reserved":0,"allocated":12722176,"data_stored":6187384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8388625,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":-32,"num_read_kb":-13,"num_write":-29,"num_write_kb":-8195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.509772"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":350,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-17T10:04:41.363299+0000","last_change":"2024-04-17T09:53:01.271526+0000","last_active":"2024-04-17T10:04:41.363299+0000","last_peered":"2024-04-17T10:04:41.363299+0000","last_clean":"2024-04-17T10:04:41.363299+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T10:04:41.363299+0000","last_undegraded":"2024-04-17T10:04:41.363299+0000","last_fullsized":"2024-04-17T10:04:41.363299+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T09:52:59.700897+0000","last_clean_scrub_stamp":"2024-04-17T09:52:59.700897+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:04.137292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509584,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33168,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338672,"statfs":{"total":96636764160,"available":96602800128,"internally_reserved":0,"allocated":2035712,"data_stored":1216199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509585,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32592,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1441792,"data_stored":625831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509584,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32592,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1441792,"data_stored":625831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509584,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36752,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335088,"statfs":{"total":96636764160,"available":96599130112,"internally_reserved":0,"allocated":1441792,"data_stored":625831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509585,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32592,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1441792,"data_stored":625831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509584,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32592,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1441792,"data_stored":625831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542289,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32592,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1441792,"data_stored":625831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542289,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37332,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334508,"statfs":{"total":96636764160,"available":96598536192,"internally_reserved":0,"allocated":2035712,"data_stored":1216199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:04:49.768 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-17T09:52:59.700897+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=9, tm_min=52, tm_sec=59, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=10, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T10:04:49.768 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-17T10:05:09.769 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T10:05:09.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.925+0000 7fed31c00640 1 Processor -- start 2024-04-17T10:05:09.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.925+0000 7fed31c00640 1 -- start start 2024-04-17T10:05:09.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.925+0000 7fed31c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed2c0ce500 0x7fed2c0b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:05:09.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.925+0000 7fed31c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fed2c0ce8e0 0x7fed2c0d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:05:09.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.925+0000 7fed31c00640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed2c070d10 con 0x7fed2c070850 2024-04-17T10:05:09.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.925+0000 7fed31c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fed2c070ff0 con 0x7fed2c0ce8e0 2024-04-17T10:05:09.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.925+0000 7fed31c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fed2c071340 con 0x7fed2c0ce500 2024-04-17T10:05:09.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.925+0000 7fed31200640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fed2c070850 0x7fed2c09a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36970/0 (socket says 172.21.15.32:36970) 2024-04-17T10:05:09.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.925+0000 7fed31200640 1 -- 172.21.15.32:0/1175374590 learned_addr learned my addr 172.21.15.32:0/1175374590 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:05:09.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.925+0000 7fed2be00640 1 --2- 172.21.15.32:0/1175374590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed2c0ce500 0x7fed2c0b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:05:09.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.925+0000 7fed23e00640 1 --2- 172.21.15.32:0/1175374590 >> v2:172.21.15.144:3300/0 conn(0x7fed2c0ce8e0 0x7fed2c0d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:05:09.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.925+0000 7fed2b400640 1 -- 172.21.15.32:0/1175374590 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1493168870 0 0) 0x7fed2c070d10 con 0x7fed2c070850 2024-04-17T10:05:09.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.925+0000 7fed2b400640 1 -- 172.21.15.32:0/1175374590 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed10003530 con 0x7fed2c070850 2024-04-17T10:05:09.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.925+0000 7fed2be00640 1 -- 172.21.15.32:0/1175374590 >> v2:172.21.15.144:3300/0 conn(0x7fed2c0ce8e0 msgr2=0x7fed2c0d0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:05:09.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.925+0000 7fed2be00640 1 --2- 172.21.15.32:0/1175374590 >> v2:172.21.15.144:3300/0 conn(0x7fed2c0ce8e0 0x7fed2c0d0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:09.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.925+0000 7fed2be00640 1 -- 172.21.15.32:0/1175374590 >> v1:172.21.15.32:6789/0 conn(0x7fed2c070850 legacy=0x7fed2c09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:05:09.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.925+0000 7fed2be00640 1 -- 172.21.15.32:0/1175374590 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c071620 con 0x7fed2c0ce500 2024-04-17T10:05:09.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.925+0000 7fed23e00640 1 --2- 172.21.15.32:0/1175374590 >> v2:172.21.15.144:3300/0 conn(0x7fed2c0ce8e0 0x7fed2c0d0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T10:05:09.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.929+0000 7fed2be00640 1 --2- 172.21.15.32:0/1175374590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed2c0ce500 0x7fed2c0b4970 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0097c0 tx=0x7fed1c01c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93d4aad18ee4f83d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:05:09.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.929+0000 7fed2b400640 1 -- 172.21.15.32:0/1175374590 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c0b0070 con 0x7fed2c0ce500 2024-04-17T10:05:09.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.929+0000 7fed2b400640 1 -- 172.21.15.32:0/1175374590 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed1c0ab600 con 0x7fed2c0ce500 2024-04-17T10:05:09.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.929+0000 7fed2b400640 1 -- 172.21.15.32:0/1175374590 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c0ab9b0 con 0x7fed2c0ce500 2024-04-17T10:05:09.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.929+0000 7fed31c00640 1 -- 172.21.15.32:0/1175374590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed2c0ce500 msgr2=0x7fed2c0b4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:05:09.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.929+0000 7fed31c00640 1 --2- 172.21.15.32:0/1175374590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed2c0ce500 0x7fed2c0b4970 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0097c0 tx=0x7fed1c01c380 comp rx=0 tx=0).stop 2024-04-17T10:05:09.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.929+0000 7fed31c00640 1 -- 172.21.15.32:0/1175374590 shutdown_connections 2024-04-17T10:05:09.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.929+0000 7fed31c00640 1 --2- 172.21.15.32:0/1175374590 >> v2:172.21.15.144:3300/0 conn(0x7fed2c0ce8e0 0x7fed2c0d0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:09.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.929+0000 7fed31c00640 1 --2- 172.21.15.32:0/1175374590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed2c0ce500 0x7fed2c0b4970 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:09.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.929+0000 7fed31c00640 1 -- 172.21.15.32:0/1175374590 >> 172.21.15.32:0/1175374590 conn(0x7fed2c0b3e50 msgr2=0x7fed2c0b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:05:09.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.929+0000 7fed31c00640 1 -- 172.21.15.32:0/1175374590 shutdown_connections 2024-04-17T10:05:09.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.929+0000 7fed31c00640 1 -- 172.21.15.32:0/1175374590 wait complete. 2024-04-17T10:05:09.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed31c00640 1 Processor -- start 2024-04-17T10:05:09.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed31c00640 1 -- start start 2024-04-17T10:05:09.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed31c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed2c070850 0x7fed2c0b7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:05:09.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed31c00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed2c0ce500 0x7fed2c1f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:05:09.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed31c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed2c0ce8e0 0x7fed2c153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:05:09.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed31c00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c0cd550 con 0x7fed2c0ce500 2024-04-17T10:05:09.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed31c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c09b310 con 0x7fed2c0ce8e0 2024-04-17T10:05:09.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed31c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fed2c0cd6d0 con 0x7fed2c070850 2024-04-17T10:05:09.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed23e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed2c0ce500 0x7fed2c1f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:05:09.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed23e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed2c0ce500 0x7fed2c1f69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42260/0 (socket says 172.21.15.32:42260) 2024-04-17T10:05:09.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed23e00640 1 -- 172.21.15.32:0/4104031702 learned_addr learned my addr 172.21.15.32:0/4104031702 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:05:09.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed2be00640 1 --2- 172.21.15.32:0/4104031702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed2c070850 0x7fed2c0b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:05:09.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed31200640 1 --2- 172.21.15.32:0/4104031702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed2c0ce8e0 0x7fed2c153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:05:09.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed23e00640 1 -- 172.21.15.32:0/4104031702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed2c070850 msgr2=0x7fed2c0b7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:05:09.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed23e00640 1 --2- 172.21.15.32:0/4104031702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed2c070850 0x7fed2c0b7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:09.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed23e00640 1 -- 172.21.15.32:0/4104031702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed2c0ce8e0 msgr2=0x7fed2c153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:05:09.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed23e00640 1 --2- 172.21.15.32:0/4104031702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed2c0ce8e0 0x7fed2c153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:09.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed23e00640 1 -- 172.21.15.32:0/4104031702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c06cb40 con 0x7fed2c0ce500 2024-04-17T10:05:09.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed31200640 1 --2- 172.21.15.32:0/4104031702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed2c0ce8e0 0x7fed2c153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T10:05:09.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed23e00640 1 --2- 172.21.15.32:0/4104031702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed2c0ce500 0x7fed2c1f69d0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fed1400ea00 tx=0x7fed1400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:05:09.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed28c00640 1 -- 172.21.15.32:0/4104031702 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed14017070 con 0x7fed2c0ce500 2024-04-17T10:05:09.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed28c00640 1 -- 172.21.15.32:0/4104031702 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed1400ca60 con 0x7fed2c0ce500 2024-04-17T10:05:09.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed28c00640 1 -- 172.21.15.32:0/4104031702 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1400ce50 con 0x7fed2c0ce500 2024-04-17T10:05:09.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed31c00640 1 -- 172.21.15.32:0/4104031702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2c0b6ad0 con 0x7fed2c0ce500 2024-04-17T10:05:09.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed31c00640 1 -- 172.21.15.32:0/4104031702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2c06d0e0 con 0x7fed2c0ce500 2024-04-17T10:05:09.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.933+0000 7fed31c00640 1 -- 172.21.15.32:0/4104031702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed2c04fd00 con 0x7fed2c0ce500 2024-04-17T10:05:09.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.937+0000 7fed28c00640 1 -- 172.21.15.32:0/4104031702 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fed1401a070 con 0x7fed2c0ce500 2024-04-17T10:05:09.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.937+0000 7fed28c00640 1 --2- 172.21.15.32:0/4104031702 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fed08068380 0x7fed0806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:05:09.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.937+0000 7fed28c00640 1 -- 172.21.15.32:0/4104031702 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7fed140862f0 con 0x7fed2c0ce500 2024-04-17T10:05:09.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.937+0000 7fed2be00640 1 --2- 172.21.15.32:0/4104031702 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fed08068380 0x7fed0806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:05:09.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.937+0000 7fed2be00640 1 --2- 172.21.15.32:0/4104031702 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fed08068380 0x7fed0806a840 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0ad000 tx=0x7fed1c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:05:09.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:09.941+0000 7fed28c00640 1 -- 172.21.15.32:0/4104031702 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fed140564a0 con 0x7fed2c0ce500 2024-04-17T10:05:10.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.105+0000 7fed31c00640 1 -- 172.21.15.32:0/4104031702 --> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fed2c0b82e0 con 0x7fed08068380 2024-04-17T10:05:10.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.105+0000 7fed28c00640 1 -- 172.21.15.32:0/4104031702 <== mgr.4106 v2:172.21.15.32:6832/1154261516 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13379 (secure 0 0 0) 0x7fed2c0b82e0 con 0x7fed08068380 2024-04-17T10:05:10.104 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:05:10.104 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-04-17T10:05:10.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.109+0000 7fed31c00640 1 -- 172.21.15.32:0/4104031702 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fed08068380 msgr2=0x7fed0806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:05:10.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.109+0000 7fed31c00640 1 --2- 172.21.15.32:0/4104031702 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fed08068380 0x7fed0806a840 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0ad000 tx=0x7fed1c008040 comp rx=0 tx=0).stop 2024-04-17T10:05:10.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.109+0000 7fed31c00640 1 -- 172.21.15.32:0/4104031702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed2c0ce500 msgr2=0x7fed2c1f69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:05:10.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.109+0000 7fed31c00640 1 --2- 172.21.15.32:0/4104031702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed2c0ce500 0x7fed2c1f69d0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fed1400ea00 tx=0x7fed1400eec0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.109+0000 7fed31c00640 1 -- 172.21.15.32:0/4104031702 shutdown_connections 2024-04-17T10:05:10.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.109+0000 7fed31c00640 1 --2- 172.21.15.32:0/4104031702 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fed08068380 0x7fed0806a840 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.109+0000 7fed31c00640 1 --2- 172.21.15.32:0/4104031702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed2c0ce8e0 0x7fed2c153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.109+0000 7fed31c00640 1 --2- 172.21.15.32:0/4104031702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed2c0ce500 0x7fed2c1f69d0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.109+0000 7fed31c00640 1 --2- 172.21.15.32:0/4104031702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed2c070850 0x7fed2c0b7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.109+0000 7fed31c00640 1 -- 172.21.15.32:0/4104031702 >> 172.21.15.32:0/4104031702 conn(0x7fed2c0b3e50 msgr2=0x7fed2c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:05:10.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.109+0000 7fed31c00640 1 -- 172.21.15.32:0/4104031702 shutdown_connections 2024-04-17T10:05:10.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.109+0000 7fed31c00640 1 -- 172.21.15.32:0/4104031702 wait complete. 2024-04-17T10:05:10.125 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":474,"stamp":"2024-04-17T10:05:09.452367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":270200,"kb_used_data":12424,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754704520,"statfs":{"total":773094113280,"available":772817428480,"internally_reserved":0,"allocated":12722176,"data_stored":6187384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002637"},"pg_stats":[{"pgid":"1.0","version":"116'119","reported_seq":362,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-17T10:04:49.639678+0000","last_change":"2024-04-17T10:04:49.639678+0000","last_active":"2024-04-17T10:04:49.639678+0000","last_peered":"2024-04-17T10:04:49.639678+0000","last_clean":"2024-04-17T10:04:49.639678+0000","last_became_active":"2024-04-17T09:53:01.270606+0000","last_became_peered":"2024-04-17T09:53:01.270606+0000","last_unstale":"2024-04-17T10:04:49.639678+0000","last_undegraded":"2024-04-17T10:04:49.639678+0000","last_fullsized":"2024-04-17T10:04:49.639678+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'119","last_scrub_stamp":"2024-04-17T10:04:49.639525+0000","last_deep_scrub":"116'119","last_deep_scrub_stamp":"2024-04-17T10:04:49.639525+0000","last_clean_scrub_stamp":"2024-04-17T10:04:49.639525+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:49:55.591806+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509588,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33164,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338676,"statfs":{"total":96636764160,"available":96602804224,"internally_reserved":0,"allocated":2035712,"data_stored":1216199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509589,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32592,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1441792,"data_stored":625831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509588,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32592,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1441792,"data_stored":625831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509588,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36752,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335088,"statfs":{"total":96636764160,"available":96599130112,"internally_reserved":0,"allocated":1441792,"data_stored":625831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509589,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32592,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1441792,"data_stored":625831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509588,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32592,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1441792,"data_stored":625831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542293,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32592,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1441792,"data_stored":625831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542293,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37324,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334516,"statfs":{"total":96636764160,"available":96598544384,"internally_reserved":0,"allocated":2035712,"data_stored":1216199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T10:05:10.126 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-17T10:05:10.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.269+0000 7fdcfbe00640 1 Processor -- start 2024-04-17T10:05:10.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.269+0000 7fdcfbe00640 1 -- start start 2024-04-17T10:05:10.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.269+0000 7fdcfbe00640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdcf415b7d0 0x7fdcf406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:05:10.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.269+0000 7fdcfbe00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdcf406f7c0 0x7fdcf406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:05:10.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.269+0000 7fdcfbe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcf406f270 0x7fdcf406bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:05:10.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.269+0000 7fdcfbe00640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fdcf40cc940 con 0x7fdcf415b7d0 2024-04-17T10:05:10.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.269+0000 7fdcfbe00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fdcf40ccc20 con 0x7fdcf406f7c0 2024-04-17T10:05:10.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.269+0000 7fdcfbe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf406e0b0 con 0x7fdcf406f270 2024-04-17T10:05:10.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.269+0000 7fdcf9400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdcf415b7d0 0x7fdcf406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:05:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.269+0000 7fdcf9400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdcf415b7d0 0x7fdcf406e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42264/0 (socket says 172.21.15.32:42264) 2024-04-17T10:05:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.269+0000 7fdcf9400640 1 -- 172.21.15.32:0/848968688 learned_addr learned my addr 172.21.15.32:0/848968688 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:05:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.269+0000 7fdcf8a00640 1 --2- 172.21.15.32:0/848968688 >> v2:172.21.15.144:3300/0 conn(0x7fdcf406f7c0 0x7fdcf406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:05:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.269+0000 7fdcf9e00640 1 --2- 172.21.15.32:0/848968688 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcf406f270 0x7fdcf406bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:05:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.269+0000 7fdcf9400640 1 -- 172.21.15.32:0/848968688 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcf406f270 msgr2=0x7fdcf406bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:05:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.273+0000 7fdcf9400640 1 --2- 172.21.15.32:0/848968688 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcf406f270 0x7fdcf406bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.273+0000 7fdcf9400640 1 -- 172.21.15.32:0/848968688 >> v2:172.21.15.144:3300/0 conn(0x7fdcf406f7c0 msgr2=0x7fdcf406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:05:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.273+0000 7fdcf9400640 1 --2- 172.21.15.32:0/848968688 >> v2:172.21.15.144:3300/0 conn(0x7fdcf406f7c0 0x7fdcf406ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.273+0000 7fdcf9400640 1 -- 172.21.15.32:0/848968688 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf406e390 con 0x7fdcf415b7d0 2024-04-17T10:05:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.273+0000 7fdcf8a00640 1 --2- 172.21.15.32:0/848968688 >> v2:172.21.15.144:3300/0 conn(0x7fdcf406f7c0 0x7fdcf406ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:05:10.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.273+0000 7fdcf9e00640 1 --2- 172.21.15.32:0/848968688 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcf406f270 0x7fdcf406bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:05:10.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.273+0000 7fdcf9400640 1 --2- 172.21.15.32:0/848968688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcf415b7d0 0x7fdcf406e8b0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40b58f0 tx=0x7fdce801eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15533d8142896376 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:05:10.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.273+0000 7fdcf3e00640 1 -- 172.21.15.32:0/848968688 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce80b4070 con 0x7fdcf415b7d0 2024-04-17T10:05:10.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.273+0000 7fdcf3e00640 1 -- 172.21.15.32:0/848968688 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdce80a7c10 con 0x7fdcf415b7d0 2024-04-17T10:05:10.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.273+0000 7fdcf3e00640 1 -- 172.21.15.32:0/848968688 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce80af440 con 0x7fdcf415b7d0 2024-04-17T10:05:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.273+0000 7fdcfbe00640 1 -- 172.21.15.32:0/848968688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcf415b7d0 msgr2=0x7fdcf406e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:05:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.273+0000 7fdcfbe00640 1 --2- 172.21.15.32:0/848968688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcf415b7d0 0x7fdcf406e8b0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40b58f0 tx=0x7fdce801eb60 comp rx=0 tx=0).stop 2024-04-17T10:05:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcfbe00640 1 -- 172.21.15.32:0/848968688 shutdown_connections 2024-04-17T10:05:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcfbe00640 1 --2- 172.21.15.32:0/848968688 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcf406f270 0x7fdcf406bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcfbe00640 1 --2- 172.21.15.32:0/848968688 >> v2:172.21.15.144:3300/0 conn(0x7fdcf406f7c0 0x7fdcf406ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcfbe00640 1 --2- 172.21.15.32:0/848968688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcf415b7d0 0x7fdcf406e8b0 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcfbe00640 1 -- 172.21.15.32:0/848968688 >> 172.21.15.32:0/848968688 conn(0x7fdcf40b2120 msgr2=0x7fdcf40b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:05:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcfbe00640 1 -- 172.21.15.32:0/848968688 shutdown_connections 2024-04-17T10:05:10.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcfbe00640 1 -- 172.21.15.32:0/848968688 wait complete. 2024-04-17T10:05:10.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcfbe00640 1 Processor -- start 2024-04-17T10:05:10.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcfbe00640 1 -- start start 2024-04-17T10:05:10.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcfbe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcf406f270 0x7fdcf414b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:05:10.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcfbe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcf406f7c0 0x7fdcf4141bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:05:10.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcfbe00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcf415b7d0 0x7fdcf413f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:05:10.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcfbe00640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf40cee70 con 0x7fdcf415b7d0 2024-04-17T10:05:10.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcfbe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf40cecf0 con 0x7fdcf406f7c0 2024-04-17T10:05:10.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcfbe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf40b3420 con 0x7fdcf406f270 2024-04-17T10:05:10.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcf9e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcf415b7d0 0x7fdcf413f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:05:10.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcf9400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcf406f270 0x7fdcf414b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:05:10.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcf9e00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcf415b7d0 0x7fdcf413f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42276/0 (socket says 172.21.15.32:42276) 2024-04-17T10:05:10.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcf9e00640 1 -- 172.21.15.32:0/2639873177 learned_addr learned my addr 172.21.15.32:0/2639873177 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:05:10.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcf8a00640 1 --2- 172.21.15.32:0/2639873177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcf406f7c0 0x7fdcf4141bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:05:10.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcf9e00640 1 -- 172.21.15.32:0/2639873177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcf406f270 msgr2=0x7fdcf414b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:05:10.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcf9e00640 1 --2- 172.21.15.32:0/2639873177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcf406f270 0x7fdcf414b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcf9e00640 1 -- 172.21.15.32:0/2639873177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcf406f7c0 msgr2=0x7fdcf4141bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:05:10.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcf9e00640 1 --2- 172.21.15.32:0/2639873177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcf406f7c0 0x7fdcf4141bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.276 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.277+0000 7fdcf9e00640 1 -- 172.21.15.32:0/2639873177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf41530c0 con 0x7fdcf415b7d0 2024-04-17T10:05:10.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.281+0000 7fdcf9e00640 1 --2- 172.21.15.32:0/2639873177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcf415b7d0 0x7fdcf413f9b0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00cef0 tx=0x7fdcec00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:05:10.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.281+0000 7fdcf1600640 1 -- 172.21.15.32:0/2639873177 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec01a070 con 0x7fdcf415b7d0 2024-04-17T10:05:10.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.281+0000 7fdcf1600640 1 -- 172.21.15.32:0/2639873177 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcec00d840 con 0x7fdcf415b7d0 2024-04-17T10:05:10.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.281+0000 7fdcfbe00640 1 -- 172.21.15.32:0/2639873177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf4153360 con 0x7fdcf415b7d0 2024-04-17T10:05:10.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.281+0000 7fdcf1600640 1 -- 172.21.15.32:0/2639873177 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec00dc30 con 0x7fdcf415b7d0 2024-04-17T10:05:10.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.281+0000 7fdcfbe00640 1 -- 172.21.15.32:0/2639873177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf4153920 con 0x7fdcf415b7d0 2024-04-17T10:05:10.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.281+0000 7fdcfbe00640 1 -- 172.21.15.32:0/2639873177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf4068a90 con 0x7fdcf415b7d0 2024-04-17T10:05:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.281+0000 7fdcf1600640 1 -- 172.21.15.32:0/2639873177 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdcec01d070 con 0x7fdcf415b7d0 2024-04-17T10:05:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.281+0000 7fdcf1600640 1 --2- 172.21.15.32:0/2639873177 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdcd4068380 0x7fdcd406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:05:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.281+0000 7fdcf1600640 1 -- 172.21.15.32:0/2639873177 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7fdcec086f10 con 0x7fdcf415b7d0 2024-04-17T10:05:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.281+0000 7fdcf9400640 1 --2- 172.21.15.32:0/2639873177 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdcd4068380 0x7fdcd406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:05:10.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.285+0000 7fdcf9400640 1 --2- 172.21.15.32:0/2639873177 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdcd4068380 0x7fdcd406a840 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fdce80b1000 tx=0x7fdce80a9070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:05:10.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.285+0000 7fdcf1600640 1 -- 172.21.15.32:0/2639873177 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fdcec0570c0 con 0x7fdcf415b7d0 2024-04-17T10:05:10.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.453+0000 7fdcfbe00640 1 -- 172.21.15.32:0/2639873177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fdcf40721e0 con 0x7fdcf415b7d0 2024-04-17T10:05:10.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.457+0000 7fdcf1600640 1 -- 172.21.15.32:0/2639873177 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) ==== 123+0+0 (secure 0 0 0) 0x7fdcec05b3c0 con 0x7fdcf415b7d0 2024-04-17T10:05:10.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.457+0000 7fdcf1600640 1 -- 172.21.15.32:0/2639873177 <== mon.0 v2:172.21.15.32:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fdcec05fbf0 con 0x7fdcf415b7d0 2024-04-17T10:05:10.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.457+0000 7fdcfbe00640 1 -- 172.21.15.32:0/2639873177 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdcd4068380 msgr2=0x7fdcd406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:05:10.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.457+0000 7fdcfbe00640 1 --2- 172.21.15.32:0/2639873177 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdcd4068380 0x7fdcd406a840 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fdce80b1000 tx=0x7fdce80a9070 comp rx=0 tx=0).stop 2024-04-17T10:05:10.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.457+0000 7fdcfbe00640 1 -- 172.21.15.32:0/2639873177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcf415b7d0 msgr2=0x7fdcf413f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:05:10.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.457+0000 7fdcfbe00640 1 --2- 172.21.15.32:0/2639873177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcf415b7d0 0x7fdcf413f9b0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00cef0 tx=0x7fdcec00a710 comp rx=0 tx=0).stop 2024-04-17T10:05:10.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.457+0000 7fdcfbe00640 1 -- 172.21.15.32:0/2639873177 shutdown_connections 2024-04-17T10:05:10.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.457+0000 7fdcfbe00640 1 --2- 172.21.15.32:0/2639873177 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fdcd4068380 0x7fdcd406a840 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.457+0000 7fdcfbe00640 1 --2- 172.21.15.32:0/2639873177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcf415b7d0 0x7fdcf413f9b0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.457+0000 7fdcfbe00640 1 --2- 172.21.15.32:0/2639873177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcf406f7c0 0x7fdcf4141bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.457+0000 7fdcfbe00640 1 --2- 172.21.15.32:0/2639873177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcf406f270 0x7fdcf414b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.457+0000 7fdcfbe00640 1 -- 172.21.15.32:0/2639873177 >> 172.21.15.32:0/2639873177 conn(0x7fdcf40b2120 msgr2=0x7fdcf40b0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:05:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.457+0000 7fdcfbe00640 1 -- 172.21.15.32:0/2639873177 shutdown_connections 2024-04-17T10:05:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.457+0000 7fdcfbe00640 1 -- 172.21.15.32:0/2639873177 wait complete. 2024-04-17T10:05:10.475 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T10:05:10.475 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T10:05:10.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.617+0000 7fa6f4400640 1 Processor -- start 2024-04-17T10:05:10.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.617+0000 7fa6f4400640 1 -- start start 2024-04-17T10:05:10.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.621+0000 7fa6f4400640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa6ec145f30 0x7fa6ec071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:05:10.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.621+0000 7fa6f4400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6ec0cd580 0x7fa6ec072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:05:10.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.621+0000 7fa6f4400640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6ec0cdac0 con 0x7fa6ec15d8d0 2024-04-17T10:05:10.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.621+0000 7fa6f4400640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 -- 0x7fa6ec06f400 con 0x7fa6ec145f30 2024-04-17T10:05:10.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.621+0000 7fa6f4400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa6ec146310 con 0x7fa6ec0cd580 2024-04-17T10:05:10.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.621+0000 7fa6f1000640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa6ec15d8d0 0x7fa6ec0cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50520/0 (socket says 172.21.15.32:50520) 2024-04-17T10:05:10.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.621+0000 7fa6f1000640 1 -- 172.21.15.32:0/1506802943 learned_addr learned my addr 172.21.15.32:0/1506802943 (peer_addr_for_me v1:172.21.15.32:0/0) 2024-04-17T10:05:10.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.621+0000 7fa6f1a00640 1 --2- 172.21.15.32:0/1506802943 >> v2:172.21.15.32:3300/0 conn(0x7fa6ec145f30 0x7fa6ec071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:05:10.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.621+0000 7fa6f2400640 1 --2- 172.21.15.32:0/1506802943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6ec0cd580 0x7fa6ec072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:05:10.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.621+0000 7fa6ebe00640 1 -- 172.21.15.32:0/1506802943 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 643210878 0 0) 0x7fa6ec0cdac0 con 0x7fa6ec15d8d0 2024-04-17T10:05:10.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.621+0000 7fa6ebe00640 1 -- 172.21.15.32:0/1506802943 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6d4003530 con 0x7fa6ec15d8d0 2024-04-17T10:05:10.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.621+0000 7fa6f1a00640 1 -- 172.21.15.32:0/1506802943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6ec0cd580 msgr2=0x7fa6ec072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T10:05:10.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.621+0000 7fa6f1a00640 1 --2- 172.21.15.32:0/1506802943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6ec0cd580 0x7fa6ec072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.621+0000 7fa6f1a00640 1 -- 172.21.15.32:0/1506802943 >> v1:172.21.15.32:6789/0 conn(0x7fa6ec15d8d0 legacy=0x7fa6ec0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:05:10.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.621+0000 7fa6f1a00640 1 -- 172.21.15.32:0/1506802943 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ec15cdf0 con 0x7fa6ec145f30 2024-04-17T10:05:10.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.621+0000 7fa6f2400640 1 --2- 172.21.15.32:0/1506802943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6ec0cd580 0x7fa6ec072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:05:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.625+0000 7fa6f1a00640 1 --2- 172.21.15.32:0/1506802943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6ec145f30 0x7fa6ec071df0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00097c0 tx=0x7fa6e001ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e615313c5a55ad7c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:05:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.625+0000 7fa6ebe00640 1 -- 172.21.15.32:0/1506802943 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00b5070 con 0x7fa6ec145f30 2024-04-17T10:05:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.625+0000 7fa6ebe00640 1 -- 172.21.15.32:0/1506802943 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa6e00a8990 con 0x7fa6ec145f30 2024-04-17T10:05:10.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.625+0000 7fa6ebe00640 1 -- 172.21.15.32:0/1506802943 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00a8d80 con 0x7fa6ec145f30 2024-04-17T10:05:10.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.625+0000 7fa6f4400640 1 -- 172.21.15.32:0/1506802943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6ec145f30 msgr2=0x7fa6ec071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:05:10.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.625+0000 7fa6f4400640 1 --2- 172.21.15.32:0/1506802943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6ec145f30 0x7fa6ec071df0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00097c0 tx=0x7fa6e001ec90 comp rx=0 tx=0).stop 2024-04-17T10:05:10.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.625+0000 7fa6f4400640 1 -- 172.21.15.32:0/1506802943 shutdown_connections 2024-04-17T10:05:10.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.625+0000 7fa6f4400640 1 --2- 172.21.15.32:0/1506802943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6ec0cd580 0x7fa6ec072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.625+0000 7fa6f4400640 1 --2- 172.21.15.32:0/1506802943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6ec145f30 0x7fa6ec071df0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.625+0000 7fa6f4400640 1 -- 172.21.15.32:0/1506802943 >> 172.21.15.32:0/1506802943 conn(0x7fa6ec06ba30 msgr2=0x7fa6ec06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:05:10.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.625+0000 7fa6f4400640 1 -- 172.21.15.32:0/1506802943 shutdown_connections 2024-04-17T10:05:10.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.625+0000 7fa6f4400640 1 -- 172.21.15.32:0/1506802943 wait complete. 2024-04-17T10:05:10.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.625+0000 7fa6f4400640 1 Processor -- start 2024-04-17T10:05:10.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.625+0000 7fa6f4400640 1 -- start start 2024-04-17T10:05:10.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.625+0000 7fa6f4400640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6ec0cd580 0x7fa6ec143690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:05:10.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.625+0000 7fa6f4400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6ec145f30 0x7fa6ec140a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:05:10.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.625+0000 7fa6f4400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6ec15d8d0 0x7fa6ec142150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:05:10.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.625+0000 7fa6f4400640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec15cb90 con 0x7fa6ec0cd580 2024-04-17T10:05:10.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.625+0000 7fa6f4400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec15d440 con 0x7fa6ec15d8d0 2024-04-17T10:05:10.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.625+0000 7fa6f4400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa6ec1601d0 con 0x7fa6ec145f30 2024-04-17T10:05:10.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.629+0000 7fa6f1a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6ec0cd580 0x7fa6ec143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:05:10.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.629+0000 7fa6f1a00640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6ec0cd580 0x7fa6ec143690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54452/0 (socket says 172.21.15.32:54452) 2024-04-17T10:05:10.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.629+0000 7fa6f1a00640 1 -- 172.21.15.32:0/3546493603 learned_addr learned my addr 172.21.15.32:0/3546493603 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-04-17T10:05:10.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.629+0000 7fa6f2400640 1 --2- 172.21.15.32:0/3546493603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6ec15d8d0 0x7fa6ec142150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:05:10.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.629+0000 7fa6f1000640 1 --2- 172.21.15.32:0/3546493603 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6ec145f30 0x7fa6ec140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:05:10.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.629+0000 7fa6f1a00640 1 -- 172.21.15.32:0/3546493603 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6ec145f30 msgr2=0x7fa6ec140a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:05:10.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.629+0000 7fa6f1a00640 1 --2- 172.21.15.32:0/3546493603 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6ec145f30 0x7fa6ec140a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.629+0000 7fa6f1a00640 1 -- 172.21.15.32:0/3546493603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6ec15d8d0 msgr2=0x7fa6ec142150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:05:10.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.629+0000 7fa6f1a00640 1 --2- 172.21.15.32:0/3546493603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6ec15d8d0 0x7fa6ec142150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.629+0000 7fa6f1a00640 1 -- 172.21.15.32:0/3546493603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ec144bd0 con 0x7fa6ec0cd580 2024-04-17T10:05:10.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.629+0000 7fa6f1000640 1 --2- 172.21.15.32:0/3546493603 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6ec145f30 0x7fa6ec140a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T10:05:10.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.629+0000 7fa6f1a00640 1 --2- 172.21.15.32:0/3546493603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6ec0cd580 0x7fa6ec143690 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fa6e001c8c0 tx=0x7fa6e001ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:05:10.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.629+0000 7fa6e9600640 1 -- 172.21.15.32:0/3546493603 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00bf070 con 0x7fa6ec0cd580 2024-04-17T10:05:10.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.629+0000 7fa6e9600640 1 -- 172.21.15.32:0/3546493603 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa6e00b0820 con 0x7fa6ec0cd580 2024-04-17T10:05:10.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.629+0000 7fa6f4400640 1 -- 172.21.15.32:0/3546493603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6ec144610 con 0x7fa6ec0cd580 2024-04-17T10:05:10.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.629+0000 7fa6e9600640 1 -- 172.21.15.32:0/3546493603 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00b5040 con 0x7fa6ec0cd580 2024-04-17T10:05:10.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.629+0000 7fa6f4400640 1 -- 172.21.15.32:0/3546493603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6ec141950 con 0x7fa6ec0cd580 2024-04-17T10:05:10.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.629+0000 7fa6f4400640 1 -- 172.21.15.32:0/3546493603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6ec068a90 con 0x7fa6ec0cd580 2024-04-17T10:05:10.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.629+0000 7fa6e9600640 1 -- 172.21.15.32:0/3546493603 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa6e00c2070 con 0x7fa6ec0cd580 2024-04-17T10:05:10.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.633+0000 7fa6e9600640 1 --2- 172.21.15.32:0/3546493603 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa6cc068380 0x7fa6cc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T10:05:10.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.633+0000 7fa6e9600640 1 -- 172.21.15.32:0/3546493603 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7fa6e0122fe0 con 0x7fa6ec0cd580 2024-04-17T10:05:10.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.633+0000 7fa6f1000640 1 --2- 172.21.15.32:0/3546493603 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa6cc068380 0x7fa6cc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T10:05:10.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.633+0000 7fa6f1000640 1 --2- 172.21.15.32:0/3546493603 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa6cc068380 0x7fa6cc06a840 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8005890 tx=0x7fa6d8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T10:05:10.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.637+0000 7fa6e9600640 1 -- 172.21.15.32:0/3546493603 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179810 (secure 0 0 0) 0x7fa6e00187f0 con 0x7fa6ec0cd580 2024-04-17T10:05:10.829 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.833+0000 7fa6f4400640 1 -- 172.21.15.32:0/3546493603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6ec072f90 con 0x7fa6ec0cd580 2024-04-17T10:05:10.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.833+0000 7fa6e9600640 1 -- 172.21.15.32:0/3546493603 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+3878 (secure 0 0 0) 0x7fa6e00f37c0 con 0x7fa6ec0cd580 2024-04-17T10:05:10.830 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-04-17T10:05:10.830 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":113,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7586,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3601627312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3601627312},{"type":"v1","addr":"172.21.15.144:6834","nonce":3601627312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7589,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2382359917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2382359917},{"type":"v1","addr":"172.21.15.144:6835","nonce":2382359917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/1857552160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1857552160},{"type":"v1","addr":"172.21.15.32:6835","nonce":1857552160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":10056,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2444744355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2444744355},{"type":"v1","addr":"172.21.15.32:6837","nonce":2444744355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113}],"filesystems":[]} 2024-04-17T10:05:10.830 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 113 2024-04-17T10:05:10.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.833+0000 7fa6f4400640 1 -- 172.21.15.32:0/3546493603 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa6cc068380 msgr2=0x7fa6cc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:05:10.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.833+0000 7fa6f4400640 1 --2- 172.21.15.32:0/3546493603 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa6cc068380 0x7fa6cc06a840 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8005890 tx=0x7fa6d8009040 comp rx=0 tx=0).stop 2024-04-17T10:05:10.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.833+0000 7fa6f4400640 1 -- 172.21.15.32:0/3546493603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6ec0cd580 msgr2=0x7fa6ec143690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T10:05:10.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.833+0000 7fa6f4400640 1 --2- 172.21.15.32:0/3546493603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6ec0cd580 0x7fa6ec143690 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fa6e001c8c0 tx=0x7fa6e001ba30 comp rx=0 tx=0).stop 2024-04-17T10:05:10.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.833+0000 7fa6f4400640 1 -- 172.21.15.32:0/3546493603 shutdown_connections 2024-04-17T10:05:10.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.833+0000 7fa6f4400640 1 --2- 172.21.15.32:0/3546493603 >> [v2:172.21.15.32:6832/1154261516,v1:172.21.15.32:6833/1154261516] conn(0x7fa6cc068380 0x7fa6cc06a840 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.833+0000 7fa6f4400640 1 --2- 172.21.15.32:0/3546493603 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6ec145f30 0x7fa6ec140a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.833+0000 7fa6f4400640 1 --2- 172.21.15.32:0/3546493603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6ec15d8d0 0x7fa6ec142150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.833+0000 7fa6f4400640 1 --2- 172.21.15.32:0/3546493603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6ec0cd580 0x7fa6ec143690 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T10:05:10.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.833+0000 7fa6f4400640 1 -- 172.21.15.32:0/3546493603 >> 172.21.15.32:0/3546493603 conn(0x7fa6ec06ba30 msgr2=0x7fa6ec15e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T10:05:10.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.837+0000 7fa6f4400640 1 -- 172.21.15.32:0/3546493603 shutdown_connections 2024-04-17T10:05:10.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-04-17T10:05:10.837+0000 7fa6f4400640 1 -- 172.21.15.32:0/3546493603 wait complete. 2024-04-17T10:05:10.851 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-17T10:05:10.851 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-17T10:05:10.851 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-17T10:05:10.852 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:05:10.863 INFO:tasks.ceph.mds.a:Stopped 2024-04-17T10:05:10.864 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-17T10:05:10.864 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:05:10.923 INFO:tasks.ceph.mds.c:Stopped 2024-04-17T10:05:10.924 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-17T10:05:10.924 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:05:10.939 INFO:tasks.ceph.mds.b:Stopped 2024-04-17T10:05:10.939 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-17T10:05:10.939 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:05:11.000 INFO:tasks.ceph.mds.d:Stopped 2024-04-17T10:05:11.001 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-17T10:05:11.001 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-17T10:05:11.001 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:05:11.080 INFO:tasks.ceph.osd.0:Stopped 2024-04-17T10:05:11.081 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-17T10:05:11.081 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:05:14.676 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:05:14.680 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:05:17.183 INFO:tasks.ceph.osd.1:Stopped 2024-04-17T10:05:17.183 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-17T10:05:17.183 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:05:17.255 INFO:tasks.ceph.osd.2:Stopped 2024-04-17T10:05:17.255 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-17T10:05:17.255 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:05:17.566 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-04-17T10:05:23.357 INFO:tasks.ceph.osd.3:Stopped 2024-04-17T10:05:23.357 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-17T10:05:23.357 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:05:23.445 INFO:tasks.ceph.osd.4:Stopped 2024-04-17T10:05:23.446 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-17T10:05:23.446 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:05:23.474 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-04-17T10:05:23.474 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-04-17T10:05:23.544 INFO:tasks.ceph.osd.5:Stopped 2024-04-17T10:05:23.545 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-17T10:05:23.545 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:05:29.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-04-17T10:05:29.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-04-17T10:05:29.646 INFO:tasks.ceph.osd.6:Stopped 2024-04-17T10:05:29.647 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-17T10:05:29.647 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:05:29.717 INFO:tasks.ceph.osd.7:Stopped 2024-04-17T10:05:29.717 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-17T10:05:29.717 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-17T10:05:29.717 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:05:29.779 INFO:tasks.ceph.mgr.y:Stopped 2024-04-17T10:05:29.780 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-17T10:05:29.780 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:05:29.835 INFO:tasks.ceph.mgr.x:Stopped 2024-04-17T10:05:29.835 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-17T10:05:29.836 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-17T10:05:29.836 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:05:29.856 INFO:tasks.ceph.mon.a:Stopped 2024-04-17T10:05:29.856 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-17T10:05:29.856 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:05:29.872 INFO:tasks.ceph.mon.b:Stopped 2024-04-17T10:05:29.872 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-17T10:05:29.872 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T10:05:29.935 INFO:tasks.ceph.mon.c:Stopped 2024-04-17T10:05:29.935 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-17T10:05:29.936 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-17T10:05:29.972 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T09:57:37.948235+0000 mds.a (mds.0) 1 : cluster [WRN] Corrupt dentry 'file_to_be_damaged' in dir frag 0x10000000000: Invalid tag char 'd' pos 0: Malformed input [buffer:3](/subdir) 2024-04-17T10:05:29.973 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-04-17T10:05:29.973 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-17T10:05:30.055 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-17T10:05:30.133 INFO:teuthology.orchestra.run.smithi032.stdout:2024-04-17T09:59:36.271169+0000 mds.c (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-17T10:05:30.134 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T10:05:30.134 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-17T10:05:30.358 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T10:05:30.359 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-17T10:05:30.692 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T10:05:30.693 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-17T10:05:30.883 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T10:05:30.883 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-17T10:05:31.050 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T10:05:31.051 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-17T10:05:31.269 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T10:05:31.269 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-17T10:05:31.509 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T10:05:31.509 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-17T10:05:31.747 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T10:05:31.748 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-17T10:05:31.947 INFO:tasks.ceph:Archiving mon data... 2024-04-17T10:05:31.948 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660029/data/mon.a.tgz 2024-04-17T10:05:31.948 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2024-04-17T10:05:31.957 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.wz0gZ2kzaf 2024-04-17T10:05:31.958 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.wz0gZ2kzaf 2024-04-17T10:05:32.109 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.wz0gZ2kzaf 2024-04-17T10:05:32.188 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.wz0gZ2kzaf is 435KB 2024-04-17T10:05:32.258 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.wz0gZ2kzaf 2024-04-17T10:05:32.266 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660029/data/mon.b.tgz 2024-04-17T10:05:32.266 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2024-04-17T10:05:32.275 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.kpZN3EhWaW 2024-04-17T10:05:32.276 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.kpZN3EhWaW 2024-04-17T10:05:32.425 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.kpZN3EhWaW 2024-04-17T10:05:32.503 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.kpZN3EhWaW is 448KB 2024-04-17T10:05:32.565 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.kpZN3EhWaW 2024-04-17T10:05:32.573 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660029/data/mon.c.tgz 2024-04-17T10:05:32.574 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2024-04-17T10:05:32.622 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.dlR6C5YU46 2024-04-17T10:05:32.622 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.dlR6C5YU46 2024-04-17T10:05:32.765 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.dlR6C5YU46 2024-04-17T10:05:32.839 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.dlR6C5YU46 is 448KB 2024-04-17T10:05:32.905 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.dlR6C5YU46 2024-04-17T10:05:32.913 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-17T10:05:32.913 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-17T10:05:32.917 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-17T10:05:32.992 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-17T10:05:33.023 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-17T10:05:33.023 INFO:tasks.ceph:Archiving crash dumps... 2024-04-17T10:05:33.024 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660029/remote/smithi032/crash 2024-04-17T10:05:33.025 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-17T10:05:33.056 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660029/remote/smithi144/crash 2024-04-17T10:05:33.057 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-17T10:05:33.088 INFO:tasks.ceph:Shutting down logrotate 2024-04-17T10:05:33.088 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:05:33.102 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T10:05:33.179 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-17T10:05:33.200 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-17T10:05:33.219 INFO:tasks.ceph:Compressing logs... 2024-04-17T10:05:33.219 DEBUG:teuthology.orchestra.run.smithi032:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-17T10:05:33.246 DEBUG:teuthology.orchestra.run.smithi144:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-17T10:05:33.269 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32008.log 2024-04-17T10:05:33.270 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53549.log 2024-04-17T10:05:33.271 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53497.log 2024-04-17T10:05:33.271 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32008.log.gz 2024-04-17T10:05:33.272 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34770.log 2024-04-17T10:05:33.272 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53549.log.gz 2024-04-17T10:05:33.273 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32840.log 2024-04-17T10:05:33.273 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53497.log.gz 2024-04-17T10:05:33.273 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38280.log 2024-04-17T10:05:33.273 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34770.log.gz 2024-04-17T10:05:33.274 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48274.log 2024-04-17T10:05:33.274 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32840.log.gz 2024-04-17T10:05:33.275 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52474.log 2024-04-17T10:05:33.275 INFO:teuthology.orchestra.run.smithi032.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38280.log.gz 2024-04-17T10:05:33.276 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47298.log 2024-04-17T10:05:33.276 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48274.log.gz 2024-04-17T10:05:33.276 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50319.log 2024-04-17T10:05:33.277 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52474.log.gz 2024-04-17T10:05:33.277 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54376.log 2024-04-17T10:05:33.278 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47298.log.gz 2024-04-17T10:05:33.278 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43609.log 2024-04-17T10:05:33.279 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50319.log.gz 2024-04-17T10:05:33.279 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46381.log 2024-04-17T10:05:33.279 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54376.log.gz 2024-04-17T10:05:33.280 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50963.log 2024-04-17T10:05:33.280 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43609.log.gz 2024-04-17T10:05:33.281 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33576.log 2024-04-17T10:05:33.281 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46381.log.gz 2024-04-17T10:05:33.281 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52329.log 2024-04-17T10:05:33.282 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50963.log.gz 2024-04-17T10:05:33.282 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51119.log 2024-04-17T10:05:33.283 INFO:teuthology.orchestra.run.smithi032.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33576.log.gz 2024-04-17T10:05:33.283 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.52329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50846.log 2024-04-17T10:05:33.284 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47002.log 2024-04-17T10:05:33.284 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51119.log.gz 2024-04-17T10:05:33.285 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49506.log 2024-04-17T10:05:33.285 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50846.log.gz 2024-04-17T10:05:33.286 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47002.log: 93.5% -- replaced with /var/log/ceph/ceph-client.0.52329.log.gz 2024-04-17T10:05:33.286 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34242.log 2024-04-17T10:05:33.287 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49506.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.47002.log.gz 2024-04-17T10:05:33.287 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53008.log 2024-04-17T10:05:33.287 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49506.log.gz 2024-04-17T10:05:33.288 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38884.log 2024-04-17T10:05:33.288 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34242.log.gz 2024-04-17T10:05:33.289 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39948.log 2024-04-17T10:05:33.289 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53008.log.gz 2024-04-17T10:05:33.289 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33437.log 2024-04-17T10:05:33.289 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38884.log.gz 2024-04-17T10:05:33.290 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.39948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42616.log 2024-04-17T10:05:33.291 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31956.log 2024-04-17T10:05:33.292 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27679.log 2024-04-17T10:05:33.292 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33437.log.gz 2024-04-17T10:05:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38728.log 2024-04-17T10:05:33.292 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42616.log.gz 2024-04-17T10:05:33.293 INFO:teuthology.orchestra.run.smithi032.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.0.39948.log.gz 2024-04-17T10:05:33.295 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45200.log 2024-04-17T10:05:33.295 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31956.log.gz 2024-04-17T10:05:33.295 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38990.log 2024-04-17T10:05:33.295 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38728.log.gz 2024-04-17T10:05:33.295 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42269.log 2024-04-17T10:05:33.295 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45200.log.gz 2024-04-17T10:05:33.296 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27775.log 2024-04-17T10:05:33.296 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.27679.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-17T10:05:33.296 INFO:teuthology.orchestra.run.smithi144.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.27679.log.gz 2024-04-17T10:05:33.297 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.27775.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-17T10:05:33.297 INFO:teuthology.orchestra.run.smithi144.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.27775.log.gz 2024-04-17T10:05:33.297 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-17T10:05:33.297 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27751.log 2024-04-17T10:05:33.297 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27799.log 2024-04-17T10:05:33.299 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.27751.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-17T10:05:33.299 INFO:teuthology.orchestra.run.smithi144.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27751.log.gz 2024-04-17T10:05:33.299 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.27799.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-17T10:05:33.300 INFO:teuthology.orchestra.run.smithi144.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27799.log.gz 2024-04-17T10:05:33.300 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.32854.log 2024-04-17T10:05:33.301 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50737.log 2024-04-17T10:05:33.301 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38990.log.gz 2024-04-17T10:05:33.301 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42269.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-17T10:05:33.301 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42269.log.gz 2024-04-17T10:05:33.301 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37648.log 2024-04-17T10:05:33.302 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50737.log.gz 2024-04-17T10:05:33.302 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43954.log 2024-04-17T10:05:33.302 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32218.log 2024-04-17T10:05:33.302 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37648.log.gz 2024-04-17T10:05:33.302 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50620.log 2024-04-17T10:05:33.302 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43954.log.gz 2024-04-17T10:05:33.302 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47760.log 2024-04-17T10:05:33.302 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32218.log.gz 2024-04-17T10:05:33.305 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50215.log 2024-04-17T10:05:33.306 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50620.log.gz 2024-04-17T10:05:33.306 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41596.log 2024-04-17T10:05:33.306 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47760.log.gz 2024-04-17T10:05:33.306 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38910.log 2024-04-17T10:05:33.306 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50215.log.gz 2024-04-17T10:05:33.306 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31663.log 2024-04-17T10:05:33.306 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41596.log.gz 2024-04-17T10:05:33.306 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36712.log 2024-04-17T10:05:33.306 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38910.log.gz 2024-04-17T10:05:33.306 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27727.log 2024-04-17T10:05:33.306 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.3.32854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27655.log 2024-04-17T10:05:33.306 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.27727.log: 90.9% -- replaced with /var/log/ceph/ceph-client.3.32854.log.gz 2024-04-17T10:05:33.306 INFO:teuthology.orchestra.run.smithi144.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.27727.log.gz 2024-04-17T10:05:33.307 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-17T10:05:33.307 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.27655.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27655.log.gz 2024-04-17T10:05:33.307 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.32735.log 2024-04-17T10:05:33.307 INFO:teuthology.orchestra.run.smithi144.stderr: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-17T10:05:33.307 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-17T10:05:33.308 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.1.32735.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-17T10:05:33.309 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph.audit.log: 89.9% -- replaced with /var/log/ceph/ceph-client.1.32735.log.gz 2024-04-17T10:05:33.309 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-17T10:05:33.310 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35582.log 2024-04-17T10:05:33.310 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31663.log.gz 2024-04-17T10:05:33.310 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52605.log 2024-04-17T10:05:33.310 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36712.log.gz 2024-04-17T10:05:33.310 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43876.log 2024-04-17T10:05:33.310 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35582.log.gz 2024-04-17T10:05:33.310 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31848.log 2024-04-17T10:05:33.310 INFO:teuthology.orchestra.run.smithi032.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.52605.log.gz 2024-04-17T10:05:33.311 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43137.log 2024-04-17T10:05:33.311 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43876.log.gz 2024-04-17T10:05:33.311 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-17T10:05:33.312 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34026.log 2024-04-17T10:05:33.312 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31848.log.gz 2024-04-17T10:05:33.312 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39120.log 2024-04-17T10:05:33.312 INFO:teuthology.orchestra.run.smithi032.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.43137.log.gz 2024-04-17T10:05:33.313 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-17T10:05:33.313 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-mds.b.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-17T10:05:33.314 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37213.log 2024-04-17T10:05:33.314 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34026.log.gz 2024-04-17T10:05:33.314 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35810.log 2024-04-17T10:05:33.314 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39120.log.gz 2024-04-17T10:05:33.315 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44642.log 2024-04-17T10:05:33.315 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37213.log.gz 2024-04-17T10:05:33.315 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51478.log 2024-04-17T10:05:33.316 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35810.log.gz 2024-04-17T10:05:33.316 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44642.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44642.log.gz 2024-04-17T10:05:33.316 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36738.log 2024-04-17T10:05:33.317 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41544.log 2024-04-17T10:05:33.317 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51478.log.gz 2024-04-17T10:05:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47376.log 2024-04-17T10:05:33.318 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36738.log.gz 2024-04-17T10:05:33.319 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-17T10:05:33.319 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45764.log 2024-04-17T10:05:33.319 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41544.log.gz 2024-04-17T10:05:33.320 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40991.log 2024-04-17T10:05:33.320 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47376.log.gz 2024-04-17T10:05:33.321 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42104.log 2024-04-17T10:05:33.321 INFO:teuthology.orchestra.run.smithi032.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.45764.log.gz 2024-04-17T10:05:33.322 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52855.log 2024-04-17T10:05:33.322 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40991.log.gz 2024-04-17T10:05:33.323 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32892.log 2024-04-17T10:05:33.323 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27703.log 2024-04-17T10:05:33.323 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27823.log 2024-04-17T10:05:33.323 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.27703.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27703.log.gz 2024-04-17T10:05:33.324 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42104.log.gz 2024-04-17T10:05:33.324 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38511.log 2024-04-17T10:05:33.324 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32892.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.52855.log.gz 2024-04-17T10:05:33.324 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50475.log 2024-04-17T10:05:33.325 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32892.log.gz 2024-04-17T10:05:33.325 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40256.log 2024-04-17T10:05:33.326 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.27823.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27823.log.gz 2024-04-17T10:05:33.326 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38511.log.gz 2024-04-17T10:05:33.326 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45069.log 2024-04-17T10:05:33.326 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50475.log.gz 2024-04-17T10:05:33.327 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32199.log 2024-04-17T10:05:33.328 INFO:teuthology.orchestra.run.smithi032.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.40256.log.gz 2024-04-17T10:05:33.328 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41036.log 2024-04-17T10:05:33.328 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45069.log.gz 2024-04-17T10:05:33.329 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46949.log 2024-04-17T10:05:33.329 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32199.log.gz 2024-04-17T10:05:33.330 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51425.log 2024-04-17T10:05:33.330 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41036.log.gz 2024-04-17T10:05:33.331 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42455.log 2024-04-17T10:05:33.331 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51425.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.46949.log.gz 2024-04-17T10:05:33.332 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54350.log 2024-04-17T10:05:33.332 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51425.log.gz 2024-04-17T10:05:33.332 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53828.log 2024-04-17T10:05:33.333 INFO:teuthology.orchestra.run.smithi032.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.42455.log.gz 2024-04-17T10:05:33.333 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54350.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-17T10:05:33.333 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54350.log.gz 2024-04-17T10:05:33.334 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37942.log 2024-04-17T10:05:33.335 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53828.log.gz 2024-04-17T10:05:33.335 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36634.log 2024-04-17T10:05:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51587.log 2024-04-17T10:05:33.336 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37942.log.gz 2024-04-17T10:05:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34690.log 2024-04-17T10:05:33.337 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36634.log.gz 2024-04-17T10:05:33.338 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39355.log 2024-04-17T10:05:33.338 INFO:teuthology.orchestra.run.smithi032.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51587.log.gz 2024-04-17T10:05:33.339 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32406.log 2024-04-17T10:05:33.339 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34690.log.gz 2024-04-17T10:05:33.340 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35244.log 2024-04-17T10:05:33.340 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39355.log.gz 2024-04-17T10:05:33.341 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54298.log 2024-04-17T10:05:33.341 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32406.log.gz 2024-04-17T10:05:33.342 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35244.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35244.log.gz 2024-04-17T10:05:33.342 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35088.log 2024-04-17T10:05:33.343 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35530.log 2024-04-17T10:05:33.344 INFO:teuthology.orchestra.run.smithi144.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-17T10:05:33.344 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54298.log.gz 2024-04-17T10:05:33.344 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35088.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35088.log.gz 2024-04-17T10:05:33.344 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46433.log 2024-04-17T10:05:33.345 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53128.log 2024-04-17T10:05:33.345 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35530.log.gz 2024-04-17T10:05:33.346 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46433.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46433.log.gz 2024-04-17T10:05:33.346 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39328.log 2024-04-17T10:05:33.347 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48860.log 2024-04-17T10:05:33.348 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39328.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53128.log.gz 2024-04-17T10:05:33.348 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39328.log.gz 2024-04-17T10:05:33.348 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33974.log 2024-04-17T10:05:33.349 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33204.log 2024-04-17T10:05:33.349 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48860.log.gz 2024-04-17T10:05:33.350 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42051.log 2024-04-17T10:05:33.350 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33974.log.gz 2024-04-17T10:05:33.351 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36581.log 2024-04-17T10:05:33.351 INFO:teuthology.orchestra.run.smithi032.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.33204.log.gz 2024-04-17T10:05:33.352 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40939.log 2024-04-17T10:05:33.352 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42051.log.gz 2024-04-17T10:05:33.353 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36581.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36581.log.gz 2024-04-17T10:05:33.353 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43531.log 2024-04-17T10:05:33.354 INFO:teuthology.orchestra.run.smithi144.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-17T10:05:33.354 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46224.log 2024-04-17T10:05:33.355 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40939.log.gz 2024-04-17T10:05:33.355 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43531.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43531.log.gz 2024-04-17T10:05:33.355 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43499.log 2024-04-17T10:05:33.356 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44534.log 2024-04-17T10:05:33.356 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46224.log.gz 2024-04-17T10:05:33.357 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49057.log 2024-04-17T10:05:33.357 INFO:teuthology.orchestra.run.smithi032.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.43499.log.gz 2024-04-17T10:05:33.358 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38964.log 2024-04-17T10:05:33.358 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44534.log.gz 2024-04-17T10:05:33.359 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49057.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49057.log.gz 2024-04-17T10:05:33.359 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54037.log 2024-04-17T10:05:33.360 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47734.log 2024-04-17T10:05:33.361 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38964.log.gz 2024-04-17T10:05:33.361 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45438.log 2024-04-17T10:05:33.361 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54037.log.gz 2024-04-17T10:05:33.362 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34796.log 2024-04-17T10:05:33.363 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47734.log.gz 2024-04-17T10:05:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41654.log 2024-04-17T10:05:33.363 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45438.log.gz 2024-04-17T10:05:33.364 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50241.log 2024-04-17T10:05:33.364 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34796.log.gz 2024-04-17T10:05:33.365 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41654.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41654.log.gz 2024-04-17T10:05:33.365 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37109.log 2024-04-17T10:05:33.366 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50163.log 2024-04-17T10:05:33.367 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50241.log.gz 2024-04-17T10:05:33.367 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37109.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37109.log.gz 2024-04-17T10:05:33.367 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50764.log 2024-04-17T10:05:33.368 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54011.log 2024-04-17T10:05:33.369 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50163.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50163.log.gz 2024-04-17T10:05:33.369 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53601.log 2024-04-17T10:05:33.369 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50764.log.gz 2024-04-17T10:05:33.370 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46621.log 2024-04-17T10:05:33.370 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54011.log.gz 2024-04-17T10:05:33.371 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32212.log 2024-04-17T10:05:33.371 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53601.log.gz 2024-04-17T10:05:33.372 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51253.log 2024-04-17T10:05:33.372 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46621.log.gz 2024-04-17T10:05:33.373 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36555.log 2024-04-17T10:05:33.373 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32212.log.gz 2024-04-17T10:05:33.374 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.51253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52527.log 2024-04-17T10:05:33.375 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36555.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36555.log.gz 2024-04-17T10:05:33.375 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35836.log 2024-04-17T10:05:33.376 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36271.log 2024-04-17T10:05:33.376 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52527.log.gz 2024-04-17T10:05:33.377 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42825.log 2024-04-17T10:05:33.377 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35836.log.gz 2024-04-17T10:05:33.378 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36271.log: gzip -5 --verbose 88.0% -- -- replaced with /var/log/ceph/ceph-client.admin.36271.log.gz /var/log/ceph/ceph-client.admin.48563.log 2024-04-17T10:05:33.378 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-17T10:05:33.379 INFO:teuthology.orchestra.run.smithi032.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.51253.log.gz 2024-04-17T10:05:33.379 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48908.log 2024-04-17T10:05:33.380 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42825.log.gz 2024-04-17T10:05:33.380 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46121.log 2024-04-17T10:05:33.380 INFO:teuthology.orchestra.run.smithi032.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.48563.log.gz 2024-04-17T10:05:33.381 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48908.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48908.log.gz 2024-04-17T10:05:33.381 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53386.log 2024-04-17T10:05:33.382 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51776.log 2024-04-17T10:05:33.382 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46121.log.gz 2024-04-17T10:05:33.383 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39762.log 2024-04-17T10:05:33.383 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53386.log.gz 2024-04-17T10:05:33.384 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53957.log 2024-04-17T10:05:33.384 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51776.log.gz 2024-04-17T10:05:33.385 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40433.log 2024-04-17T10:05:33.385 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39762.log.gz 2024-04-17T10:05:33.386 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34612.log 2024-04-17T10:05:33.386 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53957.log.gz 2024-04-17T10:05:33.388 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40433.log: gzip -5 --verbose 87.9% -- -- replaced with /var/log/ceph/ceph-client.admin.40433.log.gz /var/log/ceph/ceph-client.admin.34979.log 2024-04-17T10:05:33.388 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-17T10:05:33.388 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40180.log 2024-04-17T10:05:33.388 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34612.log.gz 2024-04-17T10:05:33.389 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36193.log 2024-04-17T10:05:33.389 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34979.log.gz 2024-04-17T10:05:33.389 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40783.log 2024-04-17T10:05:33.390 INFO:teuthology.orchestra.run.smithi032.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.40180.log.gz 2024-04-17T10:05:33.391 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36193.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36193.log.gz 2024-04-17T10:05:33.391 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42642.log 2024-04-17T10:05:33.392 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48746.log 2024-04-17T10:05:33.392 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40783.log.gz 2024-04-17T10:05:33.393 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39788.log 2024-04-17T10:05:33.393 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42642.log.gz 2024-04-17T10:05:33.394 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48746.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48746.log.gz 2024-04-17T10:05:33.394 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53983.log 2024-04-17T10:05:33.394 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53627.log 2024-04-17T10:05:33.395 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53983.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39788.log.gz 2024-04-17T10:05:33.395 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43772.log 2024-04-17T10:05:33.396 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53983.log.gz 2024-04-17T10:05:33.396 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50345.log 2024-04-17T10:05:33.396 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53627.log.gz 2024-04-17T10:05:33.397 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52019.log 2024-04-17T10:05:33.397 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43772.log.gz 2024-04-17T10:05:33.399 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48511.log 2024-04-17T10:05:33.399 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50345.log.gz 2024-04-17T10:05:33.399 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41218.log 2024-04-17T10:05:33.400 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52019.log.gz 2024-04-17T10:05:33.400 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.48511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42373.log 2024-04-17T10:05:33.401 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41218.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41218.log.gz 2024-04-17T10:05:33.401 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27451.log 2024-04-17T10:05:33.402 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33922.log 2024-04-17T10:05:33.403 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.27451.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42373.log.gz 2024-04-17T10:05:33.404 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50790.log 2024-04-17T10:05:33.404 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.27451.log.gz 2024-04-17T10:05:33.404 INFO:teuthology.orchestra.run.smithi032.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.0.48511.log.gz 2024-04-17T10:05:33.404 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42851.log 2024-04-17T10:05:33.405 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50790.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33922.log.gz 2024-04-17T10:05:33.405 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49083.log 2024-04-17T10:05:33.406 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50790.log.gz 2024-04-17T10:05:33.406 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45121.log 2024-04-17T10:05:33.406 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42851.log.gz 2024-04-17T10:05:33.407 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40887.log 2024-04-17T10:05:33.407 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49083.log.gz 2024-04-17T10:05:33.408 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38152.log 2024-04-17T10:05:33.408 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45121.log.gz 2024-04-17T10:05:33.409 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31820.log 2024-04-17T10:05:33.409 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40887.log.gz 2024-04-17T10:05:33.410 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.38152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45867.log 2024-04-17T10:05:33.411 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31820.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.31820.log.gz 2024-04-17T10:05:33.411 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35942.log 2024-04-17T10:05:33.412 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51452.log 2024-04-17T10:05:33.412 INFO:teuthology.orchestra.run.smithi032.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.38152.log.gz 2024-04-17T10:05:33.412 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45867.log.gz 2024-04-17T10:05:33.413 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51067.log 2024-04-17T10:05:33.413 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35942.log.gz 2024-04-17T10:05:33.414 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32217.log 2024-04-17T10:05:33.414 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51452.log.gz 2024-04-17T10:05:33.415 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51067.log: gzip -5 --verbose -- 89.4% /var/log/ceph/ceph-client.admin.32086.log -- replaced with /var/log/ceph/ceph-client.admin.51067.log.gz 2024-04-17T10:05:33.415 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-17T10:05:33.416 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52500.log 2024-04-17T10:05:33.416 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32217.log.gz 2024-04-17T10:05:33.417 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45608.log 2024-04-17T10:05:33.417 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32086.log.gz 2024-04-17T10:05:33.418 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52500.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-17T10:05:33.418 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52500.log.gz 2024-04-17T10:05:33.419 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37968.log 2024-04-17T10:05:33.419 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45608.log.gz 2024-04-17T10:05:33.419 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32409.log 2024-04-17T10:05:33.420 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37968.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37968.log.gz 2024-04-17T10:05:33.421 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41192.log 2024-04-17T10:05:33.421 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33634.log 2024-04-17T10:05:33.422 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32409.log.gz 2024-04-17T10:05:33.423 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31904.log 2024-04-17T10:05:33.423 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41192.log.gz 2024-04-17T10:05:33.424 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32215.log 2024-04-17T10:05:33.424 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33634.log.gz 2024-04-17T10:05:33.425 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41509.log 2024-04-17T10:05:33.425 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31904.log.gz 2024-04-17T10:05:33.426 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44590.log 2024-04-17T10:05:33.426 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32215.log.gz 2024-04-17T10:05:33.427 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41453.log 2024-04-17T10:05:33.427 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41509.log.gz 2024-04-17T10:05:33.428 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42747.log 2024-04-17T10:05:33.428 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44590.log: /var/log/ceph/ceph-client.admin.41453.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44590.log.gz 2024-04-17T10:05:33.429 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36851.log 2024-04-17T10:05:33.429 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41453.log.gz 2024-04-17T10:05:33.430 INFO:teuthology.orchestra.run.smithi032.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-17T10:05:33.430 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32418.log 2024-04-17T10:05:33.430 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42747.log.gz 2024-04-17T10:05:33.431 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36686.log 2024-04-17T10:05:33.432 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36851.log: /var/log/ceph/ceph-client.admin.32418.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36851.log.gz 2024-04-17T10:05:33.432 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42292.log 2024-04-17T10:05:33.432 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32418.log.gz 2024-04-17T10:05:33.433 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36686.log: gzip -5 --verbose 88.9% -- -- replaced with /var/log/ceph/ceph-client.admin.36686.log.gz /var/log/ceph/ceph-client.admin.54272.log 2024-04-17T10:05:33.433 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-17T10:05:33.434 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48720.log 2024-04-17T10:05:33.434 INFO:teuthology.orchestra.run.smithi032.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.42292.log.gz 2024-04-17T10:05:33.435 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39606.log 87.8% 2024-04-17T10:05:33.435 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54272.log.gz 2024-04-17T10:05:33.436 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40535.log 2024-04-17T10:05:33.436 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48720.log.gz 2024-04-17T10:05:33.437 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40509.log 2024-04-17T10:05:33.437 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39606.log.gz 2024-04-17T10:05:33.438 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36953.log 2024-04-17T10:05:33.438 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40535.log.gz 2024-04-17T10:05:33.439 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49214.log 2024-04-17T10:05:33.439 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40509.log.gz 2024-04-17T10:05:33.439 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35732.log 2024-04-17T10:05:33.440 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36953.log.gz 2024-04-17T10:05:33.440 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44826.log 2024-04-17T10:05:33.441 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49214.log.gz 2024-04-17T10:05:33.441 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35864.log 2024-04-17T10:05:33.441 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35732.log.gz 2024-04-17T10:05:33.442 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34425.log 2024-04-17T10:05:33.442 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44826.log.gz 2024-04-17T10:05:33.443 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41916.log 2024-04-17T10:05:33.443 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35864.log.gz 2024-04-17T10:05:33.444 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43635.log 2024-04-17T10:05:33.444 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34425.log.gz 2024-04-17T10:05:33.445 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.41916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49628.log 2024-04-17T10:05:33.446 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32944.log 2024-04-17T10:05:33.446 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43635.log.gz 2024-04-17T10:05:33.447 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49628.log: 92.8% -- replaced with /var/log/ceph/ceph-client.0.41916.log.gz 2024-04-17T10:05:33.447 INFO:teuthology.orchestra.run.smithi032.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49628.log.gz 2024-04-17T10:05:33.447 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39068.log 2024-04-17T10:05:33.448 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35374.log 2024-04-17T10:05:33.448 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32944.log.gz 2024-04-17T10:05:33.449 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39068.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-17T10:05:33.449 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39068.log.gz 2024-04-17T10:05:33.450 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46743.log 2024-04-17T10:05:33.450 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35374.log.gz 2024-04-17T10:05:33.451 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32060.log 2024-04-17T10:05:33.452 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35426.log 2024-04-17T10:05:33.453 INFO:teuthology.orchestra.run.smithi032.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.46743.log.gz 2024-04-17T10:05:33.453 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36660.log 2024-04-17T10:05:33.453 INFO:teuthology.orchestra.run.smithi032.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.32060.log.gz 2024-04-17T10:05:33.454 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45226.log 2024-04-17T10:05:33.454 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35426.log.gz 2024-04-17T10:05:33.455 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40705.log 2024-04-17T10:05:33.455 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36660.log.gz 2024-04-17T10:05:33.456 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49135.log 2024-04-17T10:05:33.456 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45226.log.gz 2024-04-17T10:05:33.457 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39554.log 2024-04-17T10:05:33.458 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40705.log.gz 2024-04-17T10:05:33.458 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49135.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.49135.log.gz 2024-04-17T10:05:33.458 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38754.log 2024-04-17T10:05:33.459 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5/var/log/ceph/ceph-client.admin.39554.log: --verbose -- /var/log/ceph/ceph-client.admin.54194.log 2024-04-17T10:05:33.460 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39554.log.gz 2024-04-17T10:05:33.460 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39528.log 2024-04-17T10:05:33.461 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38754.log.gz 2024-04-17T10:05:33.462 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35036.log 2024-04-17T10:05:33.462 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54194.log.gz 2024-04-17T10:05:33.462 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43902.log 2024-04-17T10:05:33.463 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39528.log.gz 2024-04-17T10:05:33.464 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36024.log 2024-04-17T10:05:33.464 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35036.log.gz 2024-04-17T10:05:33.465 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50907.log 2024-04-17T10:05:33.465 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43902.log.gz 2024-04-17T10:05:33.466 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50058.log 2024-04-17T10:05:33.466 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36024.log.gz 2024-04-17T10:05:33.467 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39146.log 2024-04-17T10:05:33.467 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50907.log.gz 2024-04-17T10:05:33.468 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46276.log 2024-04-17T10:05:33.468 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50058.log.gz 2024-04-17T10:05:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35192.log 2024-04-17T10:05:33.469 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46276.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39146.log.gz 2024-04-17T10:05:33.470 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46276.log.gz 2024-04-17T10:05:33.470 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43583.log 2024-04-17T10:05:33.471 INFO:teuthology.orchestra.run.smithi032.stderr:gzip/var/log/ceph/ceph-client.admin.35192.log: -5 --verbose -- /var/log/ceph/ceph-client.0.42561.log 2024-04-17T10:05:33.471 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35192.log.gz 2024-04-17T10:05:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42799.log 2024-04-17T10:05:33.472 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43583.log.gz 2024-04-17T10:05:33.473 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.42561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45664.log 2024-04-17T10:05:33.474 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39890.log 2024-04-17T10:05:33.474 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42799.log.gz 2024-04-17T10:05:33.475 INFO:teuthology.orchestra.run.smithi032.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.0.42561.log.gz 2024-04-17T10:05:33.475 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.45664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45943.log 2024-04-17T10:05:33.476 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51985.log 2024-04-17T10:05:33.476 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39890.log.gz 2024-04-17T10:05:33.477 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45943.log: 93.9% -- replaced with /var/log/ceph/ceph-client.0.45664.log.gz 2024-04-17T10:05:33.477 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51802.log 2024-04-17T10:05:33.478 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45943.log.gz 2024-04-17T10:05:33.478 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41680.log 2024-04-17T10:05:33.479 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51985.log.gz 2024-04-17T10:05:33.480 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35556.log 2024-04-17T10:05:33.480 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51802.log.gz 2024-04-17T10:05:33.481 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44746.log 2024-04-17T10:05:33.481 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41680.log.gz 2024-04-17T10:05:33.482 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37466.log 2024-04-17T10:05:33.482 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35556.log.gz 2024-04-17T10:05:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47402.log 2024-04-17T10:05:33.483 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44746.log.gz 2024-04-17T10:05:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36245.log 2024-04-17T10:05:33.484 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37466.log.gz 2024-04-17T10:05:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42694.log 2024-04-17T10:05:33.486 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47402.log.gz 2024-04-17T10:05:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36245.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36245.log.gz 2024-04-17T10:05:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51041.log 2024-04-17T10:05:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44445.log 2024-04-17T10:05:33.488 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42694.log.gz 2024-04-17T10:05:33.488 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43395.log 2024-04-17T10:05:33.488 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51041.log.gz 2024-04-17T10:05:33.489 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32866.log 2024-04-17T10:05:33.490 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44445.log.gz 2024-04-17T10:05:33.490 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49801.log 2024-04-17T10:05:33.491 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43395.log.gz 2024-04-17T10:05:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31565.log 2024-04-17T10:05:33.492 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32866.log.gz 2024-04-17T10:05:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53072.log 2024-04-17T10:05:33.493 INFO:teuthology.orchestra.run.smithi032.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49801.log.gz 2024-04-17T10:05:33.494 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32407.log 2024-04-17T10:05:33.494 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31565.log.gz 2024-04-17T10:05:33.495 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33790.log 2024-04-17T10:05:33.495 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53072.log: /var/log/ceph/ceph-client.admin.32407.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53072.log.gz 2024-04-17T10:05:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47967.log 2024-04-17T10:05:33.496 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32407.log.gz 2024-04-17T10:05:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38454.log 2024-04-17T10:05:33.497 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33790.log.gz 2024-04-17T10:05:33.498 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47967.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47967.log.gz -5 2024-04-17T10:05:33.498 INFO:teuthology.orchestra.run.smithi032.stderr: --verbose -- /var/log/ceph/ceph-client.admin.33712.log 2024-04-17T10:05:33.499 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50685.log 2024-04-17T10:05:33.499 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38454.log.gz 2024-04-17T10:05:33.500 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40593.log 2024-04-17T10:05:33.500 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33712.log.gz 2024-04-17T10:05:33.501 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52223.log 2024-04-17T10:05:33.501 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50685.log.gz 2024-04-17T10:05:33.502 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46849.log 2024-04-17T10:05:33.502 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40593.log.gz 2024-04-17T10:05:33.503 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47480.log 2024-04-17T10:05:33.504 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.46849.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.52223.log.gz 2024-04-17T10:05:33.504 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49601.log 2024-04-17T10:05:33.505 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47480.log: 92.2% -- replaced with /var/log/ceph/ceph-client.0.46849.log.gz 2024-04-17T10:05:33.505 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38702.log 2024-04-17T10:05:33.506 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47480.log.gz 2024-04-17T10:05:33.506 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54457.log 2024-04-17T10:05:33.507 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49601.log: /var/log/ceph/ceph-client.admin.38702.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.49601.log.gz 2024-04-17T10:05:33.507 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32614.log 2024-04-17T10:05:33.507 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.38702.log.gz 2024-04-17T10:05:33.509 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36607.log 2024-04-17T10:05:33.509 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54457.log.gz 2024-04-17T10:05:33.510 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32614.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32614.log.gz 2024-04-17T10:05:33.510 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37890.log 2024-04-17T10:05:33.511 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32417.log/var/log/ceph/ceph-client.admin.36607.log: 2024-04-17T10:05:33.511 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37890.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36607.log.gz 2024-04-17T10:05:33.512 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48138.log 2024-04-17T10:05:33.512 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37890.log.gz 2024-04-17T10:05:33.513 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37187.log 2024-04-17T10:05:33.513 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32417.log.gz 2024-04-17T10:05:33.514 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36085.log 2024-04-17T10:05:33.514 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48138.log.gz 2024-04-17T10:05:33.515 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33868.log 2024-04-17T10:05:33.515 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37187.log.gz 2024-04-17T10:05:33.516 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42882.log 2024-04-17T10:05:33.516 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36085.log.gz 2024-04-17T10:05:33.517 INFO:teuthology.orchestra.run.smithi032.stderr:gzip/var/log/ceph/ceph-client.admin.33868.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34399.log 2024-04-17T10:05:33.517 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7%/var/log/ceph/ceph-client.admin.42882.log: -- replaced with /var/log/ceph/ceph-client.admin.33868.log.gz 2024-04-17T10:05:33.518 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52273.log 2024-04-17T10:05:33.519 INFO:teuthology.orchestra.run.smithi032.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.42882.log.gz 2024-04-17T10:05:33.519 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35062.log 2024-04-17T10:05:33.519 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34399.log.gz 2024-04-17T10:05:33.520 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48378.log 2024-04-17T10:05:33.521 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52273.log.gz 2024-04-17T10:05:33.521 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34744.log 2024-04-17T10:05:33.521 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35062.log.gz 2024-04-17T10:05:33.522 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53046.log 2024-04-17T10:05:33.522 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48378.log.gz 2024-04-17T10:05:33.523 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51347.log 2024-04-17T10:05:33.524 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34744.log.gz 2024-04-17T10:05:33.524 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53046.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53046.log.gz 2024-04-17T10:05:33.525 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38936.log 2024-04-17T10:05:33.525 INFO:teuthology.orchestra.run.smithi032.stderr:gzip/var/log/ceph/ceph-client.admin.51347.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33764.log 2024-04-17T10:05:33.526 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38936.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51347.log.gz 2024-04-17T10:05:33.527 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32634.log 2024-04-17T10:05:33.527 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38936.log.gz 2024-04-17T10:05:33.528 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48883.log 2024-04-17T10:05:33.528 INFO:teuthology.orchestra.run.smithi032.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.33764.log.gz 2024-04-17T10:05:33.528 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5/var/log/ceph/ceph-client.admin.32634.log: --verbose -- /var/log/ceph/ceph-client.admin.42025.log 2024-04-17T10:05:33.529 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48883.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32634.log.gz 2024-04-17T10:05:33.530 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32633.log 2024-04-17T10:05:33.530 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48883.log.gz 2024-04-17T10:05:33.531 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.33387.log 2024-04-17T10:05:33.531 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42025.log.gz 2024-04-17T10:05:33.532 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44051.log 2024-04-17T10:05:33.532 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32633.log.gz 2024-04-17T10:05:33.533 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.33387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39437.log 2024-04-17T10:05:33.534 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44051.log: 90.7% -- replaced with /var/log/ceph/ceph-client.0.33387.log.gz 2024-04-17T10:05:33.534 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52448.log 2024-04-17T10:05:33.534 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44051.log.gz 2024-04-17T10:05:33.535 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43557.log 2024-04-17T10:05:33.535 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52448.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39437.log.gz 2024-04-17T10:05:33.536 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43189.log 2024-04-17T10:05:33.536 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52448.log.gz 2024-04-17T10:05:33.537 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37336.log 2024-04-17T10:05:33.537 INFO:teuthology.orchestra.run.smithi032.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.43557.log.gz 2024-04-17T10:05:33.538 INFO:teuthology.orchestra.run.smithi032.stderr:gzip/var/log/ceph/ceph-client.admin.43189.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.52141.log 2024-04-17T10:05:33.539 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37336.log: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.43189.log.gz 2024-04-17T10:05:33.539 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40809.log 2024-04-17T10:05:33.540 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37336.log.gz 2024-04-17T10:05:33.540 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45095.log 2024-04-17T10:05:33.540 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52141.log.gz 2024-04-17T10:05:33.541 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45402.log 2024-04-17T10:05:33.542 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5%/var/log/ceph/ceph-client.admin.45095.log: -- replaced with /var/log/ceph/ceph-client.admin.40809.log.gz 2024-04-17T10:05:33.542 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33842.log 2024-04-17T10:05:33.543 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45095.log.gz 2024-04-17T10:05:33.543 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35218.log 2024-04-17T10:05:33.544 INFO:teuthology.orchestra.run.smithi032.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.45402.log.gz 2024-04-17T10:05:33.544 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34451.log 2024-04-17T10:05:33.545 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33842.log.gz 2024-04-17T10:05:33.546 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32613.log 2024-04-17T10:05:33.546 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35218.log.gz 2024-04-17T10:05:33.547 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34451.log: gzip -5 88.4% --verbose -- replaced with /var/log/ceph/ceph-client.admin.34451.log.gz -- 2024-04-17T10:05:33.547 INFO:teuthology.orchestra.run.smithi032.stderr: /var/log/ceph/ceph-client.admin.48589.log 2024-04-17T10:05:33.548 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5/var/log/ceph/ceph-client.admin.32613.log: --verbose -- /var/log/ceph/ceph-client.admin.53102.log 2024-04-17T10:05:33.548 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48589.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32613.log.gz 2024-04-17T10:05:33.549 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40757.log 2024-04-17T10:05:33.549 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48589.log.gz 2024-04-17T10:05:33.550 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51399.log 2024-04-17T10:05:33.550 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53102.log.gz 2024-04-17T10:05:33.551 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34347.log 2024-04-17T10:05:33.552 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40757.log.gz 2024-04-17T10:05:33.552 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41481.log 2024-04-17T10:05:33.553 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51399.log.gz 2024-04-17T10:05:33.553 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47708.log 2024-04-17T10:05:33.553 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34347.log.gz 2024-04-17T10:05:33.554 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37622.log 2024-04-17T10:05:33.555 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41481.log.gz 2024-04-17T10:05:33.555 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42773.log 2024-04-17T10:05:33.556 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47708.log.gz 2024-04-17T10:05:33.556 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39840.log 2024-04-17T10:05:33.557 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37622.log.gz 2024-04-17T10:05:33.557 INFO:teuthology.orchestra.run.smithi032.stderr:gzip/var/log/ceph/ceph-client.admin.42773.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44364.log 2024-04-17T10:05:33.558 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9%/var/log/ceph/ceph-client.admin.39840.log: -- replaced with /var/log/ceph/ceph-client.admin.42773.log.gz 2024-04-17T10:05:33.559 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35400.log 2024-04-17T10:05:33.559 INFO:teuthology.orchestra.run.smithi032.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.39840.log.gz 2024-04-17T10:05:33.560 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48103.log 2024-04-17T10:05:33.560 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44364.log.gz 2024-04-17T10:05:33.561 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35996.log 2024-04-17T10:05:33.561 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35400.log.gz 2024-04-17T10:05:33.562 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47682.log 2024-04-17T10:05:33.562 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48103.log.gz 2024-04-17T10:05:33.563 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49976.log 2024-04-17T10:05:33.563 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35996.log.gz 2024-04-17T10:05:33.564 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43319.log 2024-04-17T10:05:33.564 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47682.log: /var/log/ceph/ceph-client.admin.49976.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47682.log.gz 2024-04-17T10:05:33.565 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40965.log 2024-04-17T10:05:33.565 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49976.log.gz 2024-04-17T10:05:33.566 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43035.log 2024-04-17T10:05:33.566 INFO:teuthology.orchestra.run.smithi144.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-17T10:05:33.567 INFO:teuthology.orchestra.run.smithi032.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.43319.log.gz 2024-04-17T10:05:33.567 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31689.log 2024-04-17T10:05:33.567 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40965.log.gz 2024-04-17T10:05:33.568 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49031.log 2024-04-17T10:05:33.569 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43035.log.gz 2024-04-17T10:05:33.569 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40078.log 2024-04-17T10:05:33.570 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31689.log.gz 2024-04-17T10:05:33.570 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52396.log 2024-04-17T10:05:33.571 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49031.log.gz 2024-04-17T10:05:33.572 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53776.log 2024-04-17T10:05:33.572 INFO:teuthology.orchestra.run.smithi032.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.40078.log.gz 2024-04-17T10:05:33.573 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50397.log 2024-04-17T10:05:33.573 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52396.log.gz 2024-04-17T10:05:33.573 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53776.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-17T10:05:33.574 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53776.log.gz 2024-04-17T10:05:33.575 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50818.log 2024-04-17T10:05:33.575 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50397.log.gz 2024-04-17T10:05:33.576 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40861.log 2024-04-17T10:05:33.577 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53334.log 2024-04-17T10:05:33.577 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50818.log.gz 2024-04-17T10:05:33.578 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44878.log 2024-04-17T10:05:33.579 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40861.log.gz 2024-04-17T10:05:33.579 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39684.log 2024-04-17T10:05:33.580 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53334.log.gz 2024-04-17T10:05:33.580 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53905.log 2024-04-17T10:05:33.581 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39684.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44878.log.gz 2024-04-17T10:05:33.582 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42156.log 2024-04-17T10:05:33.582 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39684.log.gz 2024-04-17T10:05:33.583 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40913.log 2024-04-17T10:05:33.583 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53905.log.gz 2024-04-17T10:05:33.584 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37499.log 2024-04-17T10:05:33.584 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42156.log.gz 2024-04-17T10:05:33.585 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.33532.log 2024-04-17T10:05:33.585 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40913.log.gz 2024-04-17T10:05:33.586 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33074.log 2024-04-17T10:05:33.586 INFO:teuthology.orchestra.run.smithi032.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.37499.log.gz 2024-04-17T10:05:33.587 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.2.33532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40567.log 2024-04-17T10:05:33.589 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37760.log 2024-04-17T10:05:33.589 INFO:teuthology.orchestra.run.smithi032.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.2.33532.log.gz 2024-04-17T10:05:33.589 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33074.log.gz 2024-04-17T10:05:33.589 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31598.log 2024-04-17T10:05:33.590 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40567.log.gz 2024-04-17T10:05:33.591 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37760.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37760.log.gz 2024-04-17T10:05:33.591 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44694.log 2024-04-17T10:05:33.592 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31598.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.31598.log.gz 2024-04-17T10:05:33.592 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38858.log 2024-04-17T10:05:33.593 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40333.log 2024-04-17T10:05:33.594 INFO:teuthology.orchestra.run.smithi032.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44694.log.gz 2024-04-17T10:05:33.595 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51828.log 2024-04-17T10:05:33.595 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38858.log.gz 2024-04-17T10:05:33.596 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47558.log 2024-04-17T10:05:33.597 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51828.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.40333.log.gz 2024-04-17T10:05:33.597 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42335.log 2024-04-17T10:05:33.597 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51828.log.gz 2024-04-17T10:05:33.598 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51321.log 2024-04-17T10:05:33.598 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47558.log.gz 2024-04-17T10:05:33.599 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41973.log 2024-04-17T10:05:33.599 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42335.log.gz 2024-04-17T10:05:33.600 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40459.log 2024-04-17T10:05:33.600 INFO:teuthology.orchestra.run.smithi032.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51321.log.gz 2024-04-17T10:05:33.602 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41973.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41973.log.gz 2024-04-17T10:05:33.602 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34104.log 2024-04-17T10:05:33.603 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39580.log 2024-04-17T10:05:33.604 INFO:teuthology.orchestra.run.smithi032.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.40459.log.gz 2024-04-17T10:05:33.604 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45893.log 2024-04-17T10:05:33.605 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34104.log.gz 2024-04-17T10:05:33.605 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38279.log 2024-04-17T10:05:33.606 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39580.log.gz 2024-04-17T10:05:33.606 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44852.log 2024-04-17T10:05:33.607 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38279.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.45893.log.gz 2024-04-17T10:05:33.608 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35640.log 2024-04-17T10:05:33.608 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44852.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.38279.log.gz 2024-04-17T10:05:33.609 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39381.log 2024-04-17T10:05:33.609 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44852.log.gz 2024-04-17T10:05:33.610 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53445.log 2024-04-17T10:05:33.610 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35640.log.gz 2024-04-17T10:05:33.611 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37701.log 2024-04-17T10:05:33.611 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39381.log.gz 2024-04-17T10:05:33.612 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53471.log 2024-04-17T10:05:33.613 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53445.log.gz 2024-04-17T10:05:33.613 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36503.log 2024-04-17T10:05:33.613 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37701.log.gz 2024-04-17T10:05:33.615 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35114.log 2024-04-17T10:05:33.615 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53471.log.gz 2024-04-17T10:05:33.616 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34716.log 2024-04-17T10:05:33.616 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36503.log.gz 2024-04-17T10:05:33.617 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49802.log 2024-04-17T10:05:33.617 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35114.log.gz 2024-04-17T10:05:33.618 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38563.log 2024-04-17T10:05:33.618 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34716.log.gz 2024-04-17T10:05:33.619 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39016.log 2024-04-17T10:05:33.619 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49802.log.gz 2024-04-17T10:05:33.620 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49749.log 2024-04-17T10:05:33.621 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38563.log.gz 2024-04-17T10:05:33.622 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34822.log 2024-04-17T10:05:33.622 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39016.log.gz 2024-04-17T10:05:33.623 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45278.log 2024-04-17T10:05:33.624 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34822.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49749.log.gz 2024-04-17T10:05:33.624 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36059.log 2024-04-17T10:05:33.624 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34822.log.gz 2024-04-17T10:05:33.625 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37440.log 2024-04-17T10:05:33.625 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45278.log.gz 2024-04-17T10:05:33.626 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52735.log 2024-04-17T10:05:33.626 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36059.log.gz 2024-04-17T10:05:33.627 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39736.log 2024-04-17T10:05:33.628 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37440.log.gz 2024-04-17T10:05:33.629 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32625.log 2024-04-17T10:05:33.629 INFO:teuthology.orchestra.run.smithi032.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.52735.log.gz 2024-04-17T10:05:33.630 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50881.log 2024-04-17T10:05:33.630 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39736.log.gz 2024-04-17T10:05:33.631 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32208.log 2024-04-17T10:05:33.631 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32625.log.gz 2024-04-17T10:05:33.632 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50711.log 2024-04-17T10:05:33.632 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50881.log.gz 2024-04-17T10:05:33.633 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36167.log 2024-04-17T10:05:33.633 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32208.log.gz 2024-04-17T10:05:33.634 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41140.log 2024-04-17T10:05:33.634 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50711.log.gz 2024-04-17T10:05:33.636 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47454.log 2024-04-17T10:05:33.636 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36167.log.gz 2024-04-17T10:05:33.637 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51693.log 2024-04-17T10:05:33.637 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41140.log.gz 2024-04-17T10:05:33.638 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47454.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-17T10:05:33.638 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47454.log.gz 2024-04-17T10:05:33.639 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.51693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54089.log 2024-04-17T10:05:33.640 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44560.log 2024-04-17T10:05:33.642 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54089.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54089.log.gz 2024-04-17T10:05:33.643 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44560.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44560.log.gz 2024-04-17T10:05:33.643 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53523.log 2024-04-17T10:05:33.644 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51015.log 2024-04-17T10:05:33.644 INFO:teuthology.orchestra.run.smithi032.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.0.51693.log.gz 2024-04-17T10:05:33.645 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53523.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53523.log.gz 2024-04-17T10:05:33.646 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38096.log 2024-04-17T10:05:33.647 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32611.log 2024-04-17T10:05:33.647 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51015.log.gz 2024-04-17T10:05:33.648 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52631.log 2024-04-17T10:05:33.649 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38096.log.gz 2024-04-17T10:05:33.649 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32611.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32611.log.gz 2024-04-17T10:05:33.650 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52631.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52631.log.gz 2024-04-17T10:05:33.651 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37864.log 2024-04-17T10:05:33.653 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45252.log 2024-04-17T10:05:33.654 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46897.log 2024-04-17T10:05:33.654 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37864.log.gz 2024-04-17T10:05:33.655 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45252.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45252.log.gz 2024-04-17T10:05:33.656 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46897.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.46897.log.gz 2024-04-17T10:05:33.657 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54428.log 2024-04-17T10:05:33.658 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37812.log 2024-04-17T10:05:33.659 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48164.log 2024-04-17T10:05:33.660 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54428.log.gz 2024-04-17T10:05:33.660 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37812.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37812.log.gz 2024-04-17T10:05:33.661 INFO:teuthology.orchestra.run.smithi144.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-17T10:05:33.662 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48164.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48164.log.gz 2024-04-17T10:05:33.662 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54167.log 2024-04-17T10:05:33.663 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44616.log 2024-04-17T10:05:33.665 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46302.log 2024-04-17T10:05:33.665 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54167.log.gz 2024-04-17T10:05:33.666 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44616.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44616.log.gz 2024-04-17T10:05:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46302.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46302.log.gz 2024-04-17T10:05:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45476.log 2024-04-17T10:05:33.668 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51373.log 2024-04-17T10:05:33.670 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41784.log 2024-04-17T10:05:33.670 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45476.log.gz 2024-04-17T10:05:33.671 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37135.log 2024-04-17T10:05:33.672 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51373.log.gz 2024-04-17T10:05:33.672 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41784.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41784.log.gz 2024-04-17T10:05:33.673 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35296.log 2024-04-17T10:05:33.674 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37135.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37135.log.gz 2024-04-17T10:05:33.674 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40645.log 2024-04-17T10:05:33.675 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47052.log 2024-04-17T10:05:33.675 INFO:teuthology.orchestra.run.smithi032.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35296.log.gz 2024-04-17T10:05:33.677 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34268.log 2024-04-17T10:05:33.677 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40645.log.gz 2024-04-17T10:05:33.678 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32203.log 2024-04-17T10:05:33.678 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47052.log.gz 2024-04-17T10:05:33.679 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34268.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34268.log.gz 2024-04-17T10:05:33.680 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.32203.log: /var/log/ceph/ceph-client.admin.39211.log 2024-04-17T10:05:33.680 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32203.log.gz 2024-04-17T10:05:33.682 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39211.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39211.log.gz 2024-04-17T10:05:33.682 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33152.log 2024-04-17T10:05:33.683 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49999.log 2024-04-17T10:05:33.685 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47506.log 2024-04-17T10:05:33.685 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33152.log.gz 2024-04-17T10:05:33.687 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49999.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.49999.log.gz 2024-04-17T10:05:33.687 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47506.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47506.log.gz 2024-04-17T10:05:33.688 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48300.log 2024-04-17T10:05:33.689 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32424.log 2024-04-17T10:05:33.690 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35140.log 2024-04-17T10:05:33.690 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48300.log.gz 2024-04-17T10:05:33.692 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32424.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32424.log.gz 2024-04-17T10:05:33.692 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35140.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35140.log.gz 2024-04-17T10:05:33.693 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50111.log 2024-04-17T10:05:33.694 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48190.log 2024-04-17T10:05:33.695 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44247.log 2024-04-17T10:05:33.696 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50111.log.gz 2024-04-17T10:05:33.697 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48190.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48190.log.gz 2024-04-17T10:05:33.698 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44247.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44247.log.gz 2024-04-17T10:05:33.698 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49404.log 2024-04-17T10:05:33.700 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47238.log 2024-04-17T10:05:33.701 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53931.log 2024-04-17T10:05:33.701 INFO:teuthology.orchestra.run.smithi032.stderr: 91.6%/var/log/ceph/ceph-client.admin.47238.log: -- replaced with /var/log/ceph/ceph-client.admin.49404.log.gz 2024-04-17T10:05:33.702 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47238.log.gz 2024-04-17T10:05:33.703 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53931.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53931.log.gz 2024-04-17T10:05:33.704 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37596.log 2024-04-17T10:05:33.705 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45969.log 2024-04-17T10:05:33.706 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44928.log 2024-04-17T10:05:33.707 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37596.log.gz 2024-04-17T10:05:33.707 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54220.log 2024-04-17T10:05:33.708 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44928.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.45969.log.gz 2024-04-17T10:05:33.708 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44928.log.gz 2024-04-17T10:05:33.709 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45714.log 2024-04-17T10:05:33.710 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54220.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54220.log.gz 2024-04-17T10:05:33.710 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37786.log 2024-04-17T10:05:33.711 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49723.log 2024-04-17T10:05:33.712 INFO:teuthology.orchestra.run.smithi032.stderr: 91.1%/var/log/ceph/ceph-client.admin.37786.log: -- replaced with /var/log/ceph/ceph-client.admin.45714.log.gz 2024-04-17T10:05:33.712 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37786.log.gz 2024-04-17T10:05:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49723.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49723.log.gz 2024-04-17T10:05:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48667.log 2024-04-17T10:05:33.715 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34156.log 2024-04-17T10:05:33.717 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46407.log 2024-04-17T10:05:33.717 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48667.log.gz 2024-04-17T10:05:33.718 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34156.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34156.log.gz 2024-04-17T10:05:33.719 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46407.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46407.log.gz 2024-04-17T10:05:33.719 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34927.log 2024-04-17T10:05:33.720 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46071.log 2024-04-17T10:05:33.722 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39179.log 2024-04-17T10:05:33.722 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34927.log.gz 2024-04-17T10:05:33.723 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49332.log 2024-04-17T10:05:33.724 INFO:teuthology.orchestra.run.smithi032.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46071.log.gz/var/log/ceph/ceph-client.admin.39179.log: 2024-04-17T10:05:33.724 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39179.log.gz 2024-04-17T10:05:33.726 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49332.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.49332.log.gz 2024-04-17T10:05:33.727 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43061.log 2024-04-17T10:05:33.727 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43850.log 2024-04-17T10:05:33.730 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53360.log 2024-04-17T10:05:33.730 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43850.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.43061.log.gz 2024-04-17T10:05:33.730 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43850.log.gz 2024-04-17T10:05:33.731 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53360.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53360.log.gz 2024-04-17T10:05:33.732 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34078.log 2024-04-17T10:05:33.733 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50501.log 2024-04-17T10:05:33.735 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44025.log 2024-04-17T10:05:33.735 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34078.log.gz 2024-04-17T10:05:33.735 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51504.log 2024-04-17T10:05:33.736 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50501.log.gz 2024-04-17T10:05:33.737 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44025.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44025.log.gz 2024-04-17T10:05:33.737 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46147.log 2024-04-17T10:05:33.738 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51504.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51504.log.gz 2024-04-17T10:05:33.738 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44473.log 2024-04-17T10:05:33.739 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31716.log 2024-04-17T10:05:33.740 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44473.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46147.log.gz 2024-04-17T10:05:33.741 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44473.log.gz 2024-04-17T10:05:33.742 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31716.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31716.log.gz 2024-04-17T10:05:33.742 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52713.log 2024-04-17T10:05:33.743 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50084.log 2024-04-17T10:05:33.745 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35322.log 2024-04-17T10:05:33.745 INFO:teuthology.orchestra.run.smithi032.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.52713.log.gz 2024-04-17T10:05:33.746 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50084.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50084.log.gz 2024-04-17T10:05:33.747 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35322.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35322.log.gz 2024-04-17T10:05:33.747 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34482.log 2024-04-17T10:05:33.749 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32814.log 2024-04-17T10:05:33.750 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47186.log 2024-04-17T10:05:33.750 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34482.log.gz 2024-04-17T10:05:33.751 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52422.log 2024-04-17T10:05:33.751 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32814.log.gz 2024-04-17T10:05:33.752 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47186.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47186.log.gz 2024-04-17T10:05:33.753 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42932.log 2024-04-17T10:05:33.754 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52422.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52422.log.gz 2024-04-17T10:05:33.754 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38046.log 2024-04-17T10:05:33.755 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37161.log 2024-04-17T10:05:33.756 INFO:teuthology.orchestra.run.smithi032.stderr: 90.9%/var/log/ceph/ceph-client.admin.38046.log: -- replaced with /var/log/ceph/ceph-client.admin.42932.log.gz 2024-04-17T10:05:33.757 INFO:teuthology.orchestra.run.smithi032.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.38046.log.gz 2024-04-17T10:05:33.757 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37161.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37161.log.gz 2024-04-17T10:05:33.759 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52782.log 2024-04-17T10:05:33.760 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49109.log 2024-04-17T10:05:33.761 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52960.log 2024-04-17T10:05:33.762 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49109.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.52782.log.gz 2024-04-17T10:05:33.763 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49109.log.gz 2024-04-17T10:05:33.765 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52960.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.52960.log.gz 2024-04-17T10:05:33.765 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50371.log 2024-04-17T10:05:33.766 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34294.log 2024-04-17T10:05:33.768 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50371.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-17T10:05:33.768 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50371.log.gz 2024-04-17T10:05:33.769 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34294.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34294.log.gz 2024-04-17T10:05:33.771 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49188.log 2024-04-17T10:05:33.773 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33259.log 2024-04-17T10:05:33.774 INFO:teuthology.orchestra.run.smithi032.stderr: 87.1%/var/log/ceph/ceph-client.admin.49188.log: -- replaced with /var/log/ceph/ceph.log.gz 2024-04-17T10:05:33.774 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49873.log 2024-04-17T10:05:33.774 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49188.log.gz 2024-04-17T10:05:33.775 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33259.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33259.log.gz 2024-04-17T10:05:33.776 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41758.log 2024-04-17T10:05:33.777 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32112.log 2024-04-17T10:05:33.778 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41758.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49873.log.gz 2024-04-17T10:05:33.778 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44508.log 2024-04-17T10:05:33.779 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41758.log.gz 2024-04-17T10:05:33.779 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32112.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32112.log.gz 2024-04-17T10:05:33.781 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44508.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44508.log.gz 2024-04-17T10:05:33.781 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53705.log 2024-04-17T10:05:33.782 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45817.log 2024-04-17T10:05:33.784 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54507.log 2024-04-17T10:05:33.784 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53705.log.gz 2024-04-17T10:05:33.786 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45817.log: /var/log/ceph/ceph-client.admin.54507.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.45817.log.gz 2024-04-17T10:05:33.786 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54507.log.gz 2024-04-17T10:05:33.786 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32970.log 2024-04-17T10:05:33.788 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53854.log 2024-04-17T10:05:33.789 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54402.log 2024-04-17T10:05:33.789 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32970.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32970.log.gz/var/log/ceph/ceph-client.admin.53854.log: 2024-04-17T10:05:33.790 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53854.log.gz 2024-04-17T10:05:33.791 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54402.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54402.log.gz 2024-04-17T10:05:33.792 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48694.log 2024-04-17T10:05:33.793 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48326.log 2024-04-17T10:05:33.795 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49574.log 2024-04-17T10:05:33.795 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48694.log.gz 2024-04-17T10:05:33.795 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48326.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48326.log.gz 2024-04-17T10:05:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49574.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49574.log.gz 2024-04-17T10:05:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36219.log 2024-04-17T10:05:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36373.log 2024-04-17T10:05:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36429.log 2024-04-17T10:05:33.800 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36219.log.gz 2024-04-17T10:05:33.801 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36373.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36373.log.gz 2024-04-17T10:05:33.803 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.36429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36873.log 2024-04-17T10:05:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50553.log 2024-04-17T10:05:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40731.log 2024-04-17T10:05:33.806 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36873.log.gz 2024-04-17T10:05:33.810 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47584.log 2024-04-17T10:05:33.810 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50553.log.gz 2024-04-17T10:05:33.810 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41810.log 2024-04-17T10:05:33.810 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50189.log 2024-04-17T10:05:33.810 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40731.log.gz 2024-04-17T10:05:33.810 INFO:teuthology.orchestra.run.smithi032.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.0.36429.log.gz 2024-04-17T10:05:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47584.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47584.log.gz 2024-04-17T10:05:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50189.log: /var/log/ceph/ceph-client.admin.41810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40619.log 2024-04-17T10:05:33.814 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50189.log.gz 2024-04-17T10:05:33.814 INFO:teuthology.orchestra.run.smithi032.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.41810.log.gz 2024-04-17T10:05:33.814 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40619.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40619.log.gz 2024-04-17T10:05:33.815 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38351.log 2024-04-17T10:05:33.816 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35504.log 2024-04-17T10:05:33.816 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33894.log 2024-04-17T10:05:33.818 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35504.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.38351.log.gz 2024-04-17T10:05:33.818 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35504.log.gz 2024-04-17T10:05:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33894.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33894.log.gz 2024-04-17T10:05:33.819 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43293.log 2024-04-17T10:05:33.821 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51556.log 2024-04-17T10:05:33.822 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50137.log 2024-04-17T10:05:33.822 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43293.log.gz 2024-04-17T10:05:33.823 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51556.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.51556.log.gz 2024-04-17T10:05:33.824 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50137.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50137.log.gz 2024-04-17T10:05:33.825 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22644.log 2024-04-17T10:05:33.825 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53802.log 2024-04-17T10:05:33.827 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph.tmp-client.admin.22644.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.53575.log -- replaced with /var/log/ceph/ceph.tmp-client.admin.22644.log.gz 2024-04-17T10:05:33.827 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-17T10:05:33.828 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53802.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53802.log.gz 2024-04-17T10:05:33.829 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53575.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53575.log.gz 2024-04-17T10:05:33.830 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34953.log 2024-04-17T10:05:33.831 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33100.log 2024-04-17T10:05:33.833 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54115.log 2024-04-17T10:05:33.833 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34953.log.gz 2024-04-17T10:05:33.834 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33100.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33100.log.gz 2024-04-17T10:05:33.835 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54115.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54115.log.gz 2024-04-17T10:05:33.835 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45043.log 2024-04-17T10:05:33.836 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40053.log 2024-04-17T10:05:33.838 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48772.log 2024-04-17T10:05:33.838 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45043.log.gz 2024-04-17T10:05:33.839 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46517.log 2024-04-17T10:05:33.839 INFO:teuthology.orchestra.run.smithi032.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.40053.log.gz 2024-04-17T10:05:33.840 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48772.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48772.log.gz 2024-04-17T10:05:33.840 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34875.log 2024-04-17T10:05:33.842 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51881.log 2024-04-17T10:05:33.842 INFO:teuthology.orchestra.run.smithi032.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.46517.log.gz 2024-04-17T10:05:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34875.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34875.log.gz 2024-04-17T10:05:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47078.log 2024-04-17T10:05:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51881.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51881.log.gz 2024-04-17T10:05:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52832.log 2024-04-17T10:05:33.846 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41347.log 2024-04-17T10:05:33.847 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52832.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.47078.log.gz 2024-04-17T10:05:33.848 INFO:teuthology.orchestra.run.smithi032.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52832.log.gz 2024-04-17T10:05:33.848 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41347.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41347.log.gz 2024-04-17T10:05:33.849 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35784.log 2024-04-17T10:05:33.850 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33048.log 2024-04-17T10:05:33.851 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53679.log 2024-04-17T10:05:33.852 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35784.log.gz 2024-04-17T10:05:33.853 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33048.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33048.log.gz 2024-04-17T10:05:33.854 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53679.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53679.log.gz 2024-04-17T10:05:33.854 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43746.log 2024-04-17T10:05:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33660.log 2024-04-17T10:05:33.857 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33126.log 2024-04-17T10:05:33.857 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43746.log.gz 2024-04-17T10:05:33.858 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51637.log 2024-04-17T10:05:33.858 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33660.log.gz 2024-04-17T10:05:33.860 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33126.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.33126.log.gz 2024-04-17T10:05:33.860 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50527.log 2024-04-17T10:05:33.860 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51637.log.gz 2024-04-17T10:05:33.861 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43243.log 2024-04-17T10:05:33.863 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32632.log 2024-04-17T10:05:33.863 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50527.log.gz 2024-04-17T10:05:33.865 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43243.log: /var/log/ceph/ceph-client.admin.32632.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.43243.log.gz 2024-04-17T10:05:33.865 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32632.log.gz 2024-04-17T10:05:33.865 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50267.log 2024-04-17T10:05:33.866 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38252.log 2024-04-17T10:05:33.868 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38020.log 2024-04-17T10:05:33.868 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50267.log.gz 2024-04-17T10:05:33.869 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38252.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.38252.log.gz 2024-04-17T10:05:33.870 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38020.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38020.log.gz 2024-04-17T10:05:33.871 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42239.log 2024-04-17T10:05:33.872 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52906.log 2024-04-17T10:05:33.874 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47324.log 2024-04-17T10:05:33.874 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42239.log.gz 2024-04-17T10:05:33.876 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52906.log: /var/log/ceph/ceph-client.admin.47324.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.52906.log.gz 2024-04-17T10:05:33.876 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47324.log.gz 2024-04-17T10:05:33.876 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40383.log 2024-04-17T10:05:33.877 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48455.log 2024-04-17T10:05:33.879 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39409.log 2024-04-17T10:05:33.880 INFO:teuthology.orchestra.run.smithi032.stderr: 91.2%/var/log/ceph/ceph-client.admin.48455.log: -- replaced with /var/log/ceph/ceph-client.admin.40383.log.gz 2024-04-17T10:05:33.880 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48455.log.gz 2024-04-17T10:05:33.881 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39409.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39409.log.gz 2024-04-17T10:05:33.881 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54324.log 2024-04-17T10:05:33.882 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37675.log 2024-04-17T10:05:33.884 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46685.log 2024-04-17T10:05:33.884 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54324.log.gz 2024-04-17T10:05:33.885 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37675.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37675.log.gz 2024-04-17T10:05:33.886 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46685.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46685.log.gz 2024-04-17T10:05:33.887 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52664.log 2024-04-17T10:05:33.887 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38479.log 2024-04-17T10:05:33.889 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51933.log 2024-04-17T10:05:33.889 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52664.log.gz 2024-04-17T10:05:33.891 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38479.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38479.log.gz 2024-04-17T10:05:33.892 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51933.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51933.log.gz 2024-04-17T10:05:33.892 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38780.log 2024-04-17T10:05:33.892 INFO:teuthology.orchestra.run.smithi144.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-17T10:05:33.893 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54141.log 2024-04-17T10:05:33.894 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40230.log 2024-04-17T10:05:33.895 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38780.log.gz 2024-04-17T10:05:33.896 INFO:teuthology.orchestra.run.smithi144.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-17T10:05:33.896 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54141.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54141.log.gz 2024-04-17T10:05:33.896 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32426.log 2024-04-17T10:05:33.897 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40230.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40230.log.gz 2024-04-17T10:05:33.898 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31794.log 2024-04-17T10:05:33.898 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32426.log.gz 2024-04-17T10:05:33.899 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33816.log 2024-04-17T10:05:33.900 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35890.log 2024-04-17T10:05:33.901 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31794.log.gz 2024-04-17T10:05:33.901 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38206.log 2024-04-17T10:05:33.901 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33816.log.gz 2024-04-17T10:05:33.903 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35890.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35890.log.gz 2024-04-17T10:05:33.903 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36141.log 2024-04-17T10:05:33.903 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38206.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38206.log.gz 2024-04-17T10:05:33.904 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41732.log 2024-04-17T10:05:33.906 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32996.log 2024-04-17T10:05:33.906 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36141.log.gz 2024-04-17T10:05:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41732.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41732.log.gz 2024-04-17T10:05:33.908 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32211.log 2024-04-17T10:05:33.908 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32996.log.gz 2024-04-17T10:05:33.909 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34130.log 2024-04-17T10:05:33.910 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34638.log 2024-04-17T10:05:33.911 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32211.log.gz 2024-04-17T10:05:33.912 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34130.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34130.log.gz 2024-04-17T10:05:33.912 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39814.log 2024-04-17T10:05:33.913 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34638.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34638.log.gz 2024-04-17T10:05:33.913 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27568.log 2024-04-17T10:05:33.914 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44155.log 2024-04-17T10:05:33.914 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39814.log.gz 2024-04-17T10:05:33.916 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.27568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53653.log 2024-04-17T10:05:33.916 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27568.log.gz 2024-04-17T10:05:33.917 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47786.log 2024-04-17T10:05:33.917 INFO:teuthology.orchestra.run.smithi032.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44155.log.gz 2024-04-17T10:05:33.918 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44103.log 2024-04-17T10:05:33.918 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53653.log.gz 2024-04-17T10:05:33.919 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47786.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47786.log.gz 2024-04-17T10:05:33.920 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44103.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44103.log.gz 2024-04-17T10:05:33.921 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36115.log 2024-04-17T10:05:33.922 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43798.log 2024-04-17T10:05:33.923 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42668.log 2024-04-17T10:05:33.924 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36115.log.gz 2024-04-17T10:05:33.924 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43798.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43798.log.gz 2024-04-17T10:05:33.925 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42668.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42668.log.gz 2024-04-17T10:05:33.926 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34320.log 2024-04-17T10:05:33.927 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39498.log 2024-04-17T10:05:33.929 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52691.log 2024-04-17T10:05:33.929 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34320.log.gz 2024-04-17T10:05:33.929 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39498.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39498.log.gz 2024-04-17T10:05:33.931 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52691.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52691.log.gz 2024-04-17T10:05:33.931 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46717.log 2024-04-17T10:05:33.933 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48405.log 2024-04-17T10:05:33.934 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47941.log 2024-04-17T10:05:33.935 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46717.log.gz 2024-04-17T10:05:33.935 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43824.log 2024-04-17T10:05:33.936 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47941.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.48405.log.gz 2024-04-17T10:05:33.937 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47941.log.gz 2024-04-17T10:05:33.937 INFO:teuthology.orchestra.run.smithi032.stderr:gzip/var/log/ceph/ceph-client.admin.43824.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34560.log 2024-04-17T10:05:33.937 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43824.log.gz 2024-04-17T10:05:33.938 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34373.log 2024-04-17T10:05:33.939 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44181.log 2024-04-17T10:05:33.939 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34560.log.gz 2024-04-17T10:05:33.941 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34373.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34373.log.gz 2024-04-17T10:05:33.942 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44181.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44181.log.gz 2024-04-17T10:05:33.942 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40671.log 2024-04-17T10:05:33.943 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46198.log 2024-04-17T10:05:33.945 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39042.log 2024-04-17T10:05:33.945 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40671.log.gz 2024-04-17T10:05:33.946 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46198.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46198.log.gz 2024-04-17T10:05:33.947 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50293.log 2024-04-17T10:05:33.947 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39042.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39042.log.gz 2024-04-17T10:05:33.948 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34000.log 2024-04-17T10:05:33.949 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37531.log 2024-04-17T10:05:33.949 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50293.log.gz 2024-04-17T10:05:33.951 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34000.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34000.log.gz 2024-04-17T10:05:33.951 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37531.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37531.log.gz 2024-04-17T10:05:33.952 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54063.log 2024-04-17T10:05:33.953 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44312.log 2024-04-17T10:05:33.955 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49266.log 2024-04-17T10:05:33.955 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54063.log.gz 2024-04-17T10:05:33.956 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44312.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44312.log.gz 2024-04-17T10:05:33.957 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49266.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49266.log.gz 2024-04-17T10:05:33.957 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45532.log 2024-04-17T10:05:33.958 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37362.log 2024-04-17T10:05:33.960 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38589.log 2024-04-17T10:05:33.960 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45532.log.gz 2024-04-17T10:05:33.961 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37362.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37362.log.gz 2024-04-17T10:05:33.962 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38589.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38589.log.gz 2024-04-17T10:05:33.963 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35452.log 2024-04-17T10:05:33.964 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38615.log 2024-04-17T10:05:33.965 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47630.log 2024-04-17T10:05:33.966 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35452.log.gz 2024-04-17T10:05:33.966 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47128.log 2024-04-17T10:05:33.966 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38615.log.gz 2024-04-17T10:05:33.968 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47630.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47630.log.gz 2024-04-17T10:05:33.968 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36769.log 2024-04-17T10:05:33.969 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47128.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47128.log.gz 2024-04-17T10:05:33.969 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41999.log 2024-04-17T10:05:33.971 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50588.log 2024-04-17T10:05:33.971 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36769.log.gz 2024-04-17T10:05:33.972 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41999.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41999.log.gz 2024-04-17T10:05:33.973 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50588.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50588.log.gz 2024-04-17T10:05:33.973 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34534.log 2024-04-17T10:05:33.975 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44214.log 2024-04-17T10:05:33.977 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49297.log 2024-04-17T10:05:33.977 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34534.log.gz 2024-04-17T10:05:33.977 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44214.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44214.log.gz 2024-04-17T10:05:33.978 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49297.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49297.log.gz 2024-04-17T10:05:33.979 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32034.log 2024-04-17T10:05:33.980 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52103.log 2024-04-17T10:05:33.981 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32034.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-17T10:05:33.981 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32034.log.gz 2024-04-17T10:05:33.983 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52103.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52103.log.gz 2024-04-17T10:05:33.987 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph.audit.log: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-17T10:05:33.987 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-17T10:05:33.988 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51959.log 2024-04-17T10:05:33.989 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47212.log 2024-04-17T10:05:33.991 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51959.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51959.log.gz 2024-04-17T10:05:33.992 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36297.log 2024-04-17T10:05:33.993 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47212.log.gz 2024-04-17T10:05:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48248.log 2024-04-17T10:05:33.995 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36297.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36297.log.gz 2024-04-17T10:05:33.996 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48248.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.48248.log.gz 2024-04-17T10:05:33.996 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-17T10:05:33.997 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45502.log 2024-04-17T10:05:34.000 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.45502.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45502.log.gz 2024-04-17T10:05:34.000 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43720.log 2024-04-17T10:05:34.007 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43720.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43720.log.gz 2024-04-17T10:05:34.007 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51145.log 2024-04-17T10:05:34.007 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51145.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.51145.log.gz 2024-04-17T10:05:34.007 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47350.log 2024-04-17T10:05:34.010 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47350.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47350.log.gz 2024-04-17T10:05:34.010 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35706.log 2024-04-17T10:05:34.014 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36917.log 2024-04-17T10:05:34.014 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35706.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35706.log.gz 2024-04-17T10:05:34.014 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36979.log 2024-04-17T10:05:34.014 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36917.log.gz 2024-04-17T10:05:34.015 INFO:teuthology.orchestra.run.smithi144.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-17T10:05:34.017 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36979.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36979.log.gz 2024-04-17T10:05:34.017 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48352.log 2024-04-17T10:05:34.020 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48352.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48352.log.gz 2024-04-17T10:05:34.020 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37057.log 2024-04-17T10:05:34.023 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40307.log 2024-04-17T10:05:34.023 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37057.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37057.log.gz 2024-04-17T10:05:34.026 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40307.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40307.log.gz 2024-04-17T10:05:34.026 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42720.log 2024-04-17T10:05:34.028 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31982.log 2024-04-17T10:05:34.029 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42720.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42720.log.gz 2024-04-17T10:05:34.032 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31982.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31982.log.gz 2024-04-17T10:05:34.033 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53731.log 2024-04-17T10:05:34.039 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53731.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53731.log.gz 2024-04-17T10:05:34.039 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54481.log 2024-04-17T10:05:34.039 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54481.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.54481.log.gz 2024-04-17T10:05:34.039 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49240.log 2024-04-17T10:05:34.039 INFO:teuthology.orchestra.run.smithi144.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-17T10:05:34.040 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-04-17T10:05:34.040 INFO:teuthology.orchestra.run.smithi144.stderr:real 0m0.772s 2024-04-17T10:05:34.040 INFO:teuthology.orchestra.run.smithi144.stderr:user 0m3.128s 2024-04-17T10:05:34.041 INFO:teuthology.orchestra.run.smithi144.stderr:sys 0m0.278s 2024-04-17T10:05:34.042 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49240.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49240.log.gz 2024-04-17T10:05:34.042 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36895.log 2024-04-17T10:05:34.043 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49926.log 2024-04-17T10:05:34.045 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36895.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36895.log.gz 2024-04-17T10:05:34.048 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49926.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49926.log.gz 2024-04-17T10:05:34.048 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34586.log 2024-04-17T10:05:34.051 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34586.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34586.log.gz 2024-04-17T10:05:34.051 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51530.log 2024-04-17T10:05:34.054 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51530.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51530.log.gz 2024-04-17T10:05:34.054 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37994.log 2024-04-17T10:05:34.057 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37994.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37994.log.gz 2024-04-17T10:05:34.057 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45174.log 2024-04-17T10:05:34.061 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40002.log 2024-04-17T10:05:34.061 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45174.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45174.log.gz 2024-04-17T10:05:34.061 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39658.log 2024-04-17T10:05:34.061 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40002.log.gz 2024-04-17T10:05:34.064 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39658.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39658.log.gz 2024-04-17T10:05:34.064 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48615.log 2024-04-17T10:05:34.067 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48615.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48615.log.gz 2024-04-17T10:05:34.068 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47428.log 2024-04-17T10:05:34.071 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47428.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47428.log.gz 2024-04-17T10:05:34.071 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39472.log 2024-04-17T10:05:34.075 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39472.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39472.log.gz 2024-04-17T10:05:34.075 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48641.log 2024-04-17T10:05:34.076 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48641.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48641.log.gz 2024-04-17T10:05:34.077 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33602.log 2024-04-17T10:05:34.079 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35348.log 2024-04-17T10:05:34.081 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-04-17T10:05:34.082 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4%/var/log/ceph/ceph-client.admin.35348.log: -- replaced with /var/log/ceph/ceph-client.admin.33602.log.gz 2024-04-17T10:05:34.082 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34508.log 2024-04-17T10:05:34.084 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47264.log 2024-04-17T10:05:34.084 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35348.log.gz 2024-04-17T10:05:34.085 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34508.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34508.log.gz 2024-04-17T10:05:34.088 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53412.log 2024-04-17T10:05:34.089 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51907.log 2024-04-17T10:05:34.089 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47264.log.gz 2024-04-17T10:05:34.091 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53412.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53412.log.gz 2024-04-17T10:05:34.092 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36799.log 2024-04-17T10:05:34.095 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51907.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51907.log.gz 2024-04-17T10:05:34.095 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36799.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36799.log.gz 2024-04-17T10:05:34.095 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37083.log 2024-04-17T10:05:34.098 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37083.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37083.log.gz 2024-04-17T10:05:34.098 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31742.log 2024-04-17T10:05:34.099 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39710.log 2024-04-17T10:05:34.101 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31742.log: /var/log/ceph/ceph-client.admin.39710.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31742.log.gz 2024-04-17T10:05:34.101 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34052.log 2024-04-17T10:05:34.102 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39710.log.gz 2024-04-17T10:05:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34052.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34052.log.gz 2024-04-17T10:05:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47845.log 2024-04-17T10:05:34.108 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47845.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47845.log.gz 2024-04-17T10:05:34.108 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47532.log 2024-04-17T10:05:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47532.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47532.log.gz 2024-04-17T10:05:34.111 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35478.log 2024-04-17T10:05:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47160.log 2024-04-17T10:05:34.116 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35478.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35478.log.gz 2024-04-17T10:05:34.116 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47160.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47160.log.gz 2024-04-17T10:05:34.116 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36529.log 2024-04-17T10:05:34.119 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36529.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36529.log.gz 2024-04-17T10:05:34.119 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33178.log 2024-04-17T10:05:34.122 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33178.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33178.log.gz 2024-04-17T10:05:34.122 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43369.log 2024-04-17T10:05:34.125 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38229.log 2024-04-17T10:05:34.126 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43369.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43369.log.gz 2024-04-17T10:05:34.128 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38229.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.38229.log.gz 2024-04-17T10:05:34.129 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43928.log 2024-04-17T10:05:34.132 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43928.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43928.log.gz 2024-04-17T10:05:34.132 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44984.log 2024-04-17T10:05:34.136 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.44984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48830.log 2024-04-17T10:05:34.138 INFO:teuthology.orchestra.run.smithi032.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.0.44984.log.gz 2024-04-17T10:05:34.139 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42985.log 2024-04-17T10:05:34.141 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48830.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48830.log.gz 2024-04-17T10:05:34.143 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42985.log: /var/log/ceph/ceph-client.admin.46494.log 2024-04-17T10:05:34.145 INFO:teuthology.orchestra.run.smithi032.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.42985.log.gz 2024-04-17T10:05:34.147 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45558.log 2024-04-17T10:05:34.148 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39632.log 2024-04-17T10:05:34.152 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46494.log: /var/log/ceph/ceph-client.admin.45558.log: /var/log/ceph/ceph-client.admin.39632.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.45558.log.gz 2024-04-17T10:05:34.152 INFO:teuthology.orchestra.run.smithi032.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-17T10:05:34.152 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39632.log.gz 2024-04-17T10:05:34.152 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35916.log 2024-04-17T10:05:34.152 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46494.log.gz 2024-04-17T10:05:34.154 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35916.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35916.log.gz 2024-04-17T10:05:34.155 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39094.log 2024-04-17T10:05:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50989.log 2024-04-17T10:05:34.158 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39094.log: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-17T10:05:34.158 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39094.log.gz 2024-04-17T10:05:34.158 INFO:teuthology.orchestra.run.smithi032.stderr:gzip/var/log/ceph/ceph-client.admin.50989.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41088.log 2024-04-17T10:05:34.159 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50989.log.gz 2024-04-17T10:05:34.159 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46355.log 2024-04-17T10:05:34.161 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37729.log 2024-04-17T10:05:34.161 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41088.log.gz 2024-04-17T10:05:34.162 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46355.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46355.log.gz 2024-04-17T10:05:34.162 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41570.log 2024-04-17T10:05:34.163 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51854.log 2024-04-17T10:05:34.163 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37729.log.gz 2024-04-17T10:05:34.164 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42505.log 2024-04-17T10:05:34.165 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41570.log.gz 2024-04-17T10:05:34.166 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51854.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51854.log.gz 2024-04-17T10:05:34.167 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42505.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42505.log.gz 2024-04-17T10:05:34.167 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37388.log 2024-04-17T10:05:34.169 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33022.log 2024-04-17T10:05:34.171 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44800.log 2024-04-17T10:05:34.171 INFO:teuthology.orchestra.run.smithi032.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-17T10:05:34.171 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37388.log.gz 2024-04-17T10:05:34.171 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40835.log 2024-04-17T10:05:34.171 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33022.log.gz 2024-04-17T10:05:34.172 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47993.log 88.2% 2024-04-17T10:05:34.172 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44800.log.gz 2024-04-17T10:05:34.173 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32138.log 2024-04-17T10:05:34.173 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40835.log.gz 2024-04-17T10:05:34.174 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44417.log 2024-04-17T10:05:34.174 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47993.log.gz 2024-04-17T10:05:34.175 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52579.log 2024-04-17T10:05:34.176 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32138.log.gz 2024-04-17T10:05:34.176 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53308.log 2024-04-17T10:05:34.176 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44417.log.gz 2024-04-17T10:05:34.177 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34901.log 2024-04-17T10:05:34.177 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52579.log.gz 2024-04-17T10:05:34.178 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35270.log 2024-04-17T10:05:34.178 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53308.log.gz 2024-04-17T10:05:34.179 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39276.log 2024-04-17T10:05:34.180 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34901.log.gz 2024-04-17T10:05:34.180 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38649.log 2024-04-17T10:05:34.181 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35270.log.gz 2024-04-17T10:05:34.182 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45362.log 2024-04-17T10:05:34.182 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39276.log.gz 2024-04-17T10:05:34.183 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46793.log 2024-04-17T10:05:34.183 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38649.log.gz 2024-04-17T10:05:34.184 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38806.log 2024-04-17T10:05:34.185 INFO:teuthology.orchestra.run.smithi032.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.45362.log.gz 2024-04-17T10:05:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37031.log 2024-04-17T10:05:34.185 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46793.log.gz 2024-04-17T10:05:34.186 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52167.log 2024-04-17T10:05:34.187 INFO:teuthology.orchestra.run.smithi032.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38806.log.gz 2024-04-17T10:05:34.188 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37031.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37031.log.gz 2024-04-17T10:05:34.188 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41427.log 2024-04-17T10:05:34.189 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50933.log 2024-04-17T10:05:34.189 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52167.log.gz 2024-04-17T10:05:34.190 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44391.log 2024-04-17T10:05:34.190 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41427.log.gz 2024-04-17T10:05:34.191 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32918.log 2024-04-17T10:05:34.191 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50933.log.gz 2024-04-17T10:05:34.192 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36323.log 2024-04-17T10:05:34.192 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44391.log.gz 2024-04-17T10:05:34.193 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38537.log 2024-04-17T10:05:34.193 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32918.log.gz 2024-04-17T10:05:34.194 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.36323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41373.log 2024-04-17T10:05:34.195 INFO:teuthology.orchestra.run.smithi032.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.36323.log.gz 2024-04-17T10:05:34.195 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52933.log 2024-04-17T10:05:34.195 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38537.log.gz 2024-04-17T10:05:34.196 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41399.log 2024-04-17T10:05:34.197 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41373.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41373.log.gz 2024-04-17T10:05:34.198 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43694.log 2024-04-17T10:05:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41399.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.52933.log.gz 2024-04-17T10:05:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42399.log 2024-04-17T10:05:34.199 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41399.log.gz 2024-04-17T10:05:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47877.log 2024-04-17T10:05:34.200 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43694.log.gz 2024-04-17T10:05:34.201 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45309.log 2024-04-17T10:05:34.201 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42399.log.gz 2024-04-17T10:05:34.201 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37838.log 2024-04-17T10:05:34.202 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47877.log.gz 2024-04-17T10:05:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39302.log 2024-04-17T10:05:34.203 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45309.log.gz 2024-04-17T10:05:34.204 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33738.log 2024-04-17T10:05:34.204 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37838.log.gz 2024-04-17T10:05:34.205 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.39302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41628.log 2024-04-17T10:05:34.205 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39302.log.gz 2024-04-17T10:05:34.206 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37310.log 2024-04-17T10:05:34.206 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33738.log.gz 2024-04-17T10:05:34.207 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35166.log 2024-04-17T10:05:34.207 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41628.log.gz 2024-04-17T10:05:34.208 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46329.log 2024-04-17T10:05:34.208 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37310.log.gz 2024-04-17T10:05:34.209 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38832.log 2024-04-17T10:05:34.209 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35166.log.gz 2024-04-17T10:05:34.210 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50423.log 2024-04-17T10:05:34.210 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46329.log.gz 2024-04-17T10:05:34.211 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50449.log 2024-04-17T10:05:34.211 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38832.log.gz 2024-04-17T10:05:34.212 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48021.log 2024-04-17T10:05:34.213 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50423.log.gz 2024-04-17T10:05:34.213 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.50449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51093.log 2024-04-17T10:05:34.214 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50449.log.gz 2024-04-17T10:05:34.214 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53178.log 2024-04-17T10:05:34.215 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48021.log.gz 2024-04-17T10:05:34.216 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51093.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51093.log.gz 2024-04-17T10:05:34.216 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45339.log 2024-04-17T10:05:34.217 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.53178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43451.log 2024-04-17T10:05:34.217 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2%/var/log/ceph/ceph-client.admin.45339.log: -- replaced with /var/log/ceph/ceph-client.admin.53178.log.gz 2024-04-17T10:05:34.218 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43668.log 2024-04-17T10:05:34.218 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45339.log.gz 2024-04-17T10:05:34.219 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.43451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44668.log 2024-04-17T10:05:34.220 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37916.log 2024-04-17T10:05:34.220 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43668.log.gz 2024-04-17T10:05:34.220 INFO:teuthology.orchestra.run.smithi032.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.0.43451.log.gz 2024-04-17T10:05:34.221 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31876.log 2024-04-17T10:05:34.222 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44668.log.gz 2024-04-17T10:05:34.222 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44077.log 2024-04-17T10:05:34.222 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37916.log.gz 2024-04-17T10:05:34.223 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32629.log 2024-04-17T10:05:34.224 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31876.log.gz 2024-04-17T10:05:34.224 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41860.log 2024-04-17T10:05:34.225 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44077.log.gz 2024-04-17T10:05:34.225 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34664.log 2024-04-17T10:05:34.226 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32629.log.gz 2024-04-17T10:05:34.227 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45147.log 2024-04-17T10:05:34.227 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41860.log.gz 2024-04-17T10:05:34.228 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35968.log 2024-04-17T10:05:34.228 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34664.log.gz 2024-04-17T10:05:34.229 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.45147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52078.log 2024-04-17T10:05:34.229 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45147.log.gz 2024-04-17T10:05:34.230 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41283.log 2024-04-17T10:05:34.230 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35968.log.gz 2024-04-17T10:05:34.231 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52553.log 2024-04-17T10:05:34.231 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52078.log.gz 2024-04-17T10:05:34.232 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31498.log 2024-04-17T10:05:34.233 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41283.log.gz 2024-04-17T10:05:34.233 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52553.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52553.log.gz 2024-04-17T10:05:34.233 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41114.log 2024-04-17T10:05:34.234 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49693.log 2024-04-17T10:05:34.235 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31498.log: /var/log/ceph/ceph-client.admin.41114.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31498.log.gz 2024-04-17T10:05:34.235 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41062.log 2024-04-17T10:05:34.236 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41114.log.gz 2024-04-17T10:05:34.237 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49333.log 2024-04-17T10:05:34.237 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49693.log.gz 2024-04-17T10:05:34.238 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51195.log 2024-04-17T10:05:34.238 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41062.log.gz 2024-04-17T10:05:34.238 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32164.log 2024-04-17T10:05:34.239 INFO:teuthology.orchestra.run.smithi032.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.49333.log.gz 2024-04-17T10:05:34.240 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51195.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51195.log.gz 2024-04-17T10:05:34.240 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43999.log 2024-04-17T10:05:34.242 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31930.log 2024-04-17T10:05:34.242 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32164.log.gz 2024-04-17T10:05:34.242 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.43999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46464.log 2024-04-17T10:05:34.242 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43999.log.gz 2024-04-17T10:05:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37284.log 2024-04-17T10:05:34.243 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31930.log.gz 2024-04-17T10:05:34.244 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47656.log 2024-04-17T10:05:34.244 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46464.log.gz 2024-04-17T10:05:34.245 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51750.log 2024-04-17T10:05:34.245 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37284.log.gz 2024-04-17T10:05:34.246 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38676.log 2024-04-17T10:05:34.246 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47656.log.gz 2024-04-17T10:05:34.247 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.51750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41251.log 2024-04-17T10:05:34.248 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51750.log.gz 2024-04-17T10:05:34.248 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42077.log 2024-04-17T10:05:34.248 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38676.log.gz 2024-04-17T10:05:34.249 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48798.log 2024-04-17T10:05:34.249 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41251.log.gz 2024-04-17T10:05:34.250 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35758.log 2024-04-17T10:05:34.250 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42077.log.gz 2024-04-17T10:05:34.252 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44720.log 2024-04-17T10:05:34.252 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48798.log.gz 2024-04-17T10:05:34.253 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37239.log 2024-04-17T10:05:34.253 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35758.log.gz 2024-04-17T10:05:34.254 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42182.log 2024-04-17T10:05:34.254 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44720.log.gz 2024-04-17T10:05:34.255 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52197.log 2024-04-17T10:05:34.255 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37239.log.gz 2024-04-17T10:05:34.257 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48047.log 2024-04-17T10:05:34.257 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42182.log.gz 2024-04-17T10:05:34.257 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32413.log 2024-04-17T10:05:34.258 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52197.log.gz 2024-04-17T10:05:34.258 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41166.log 2024-04-17T10:05:34.259 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48047.log.gz 2024-04-17T10:05:34.259 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.32413.log: gzip -5 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32413.log.gz 2024-04-17T10:05:34.260 INFO:teuthology.orchestra.run.smithi032.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37005.log 2024-04-17T10:05:34.260 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42130.log 2024-04-17T10:05:34.261 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41166.log.gz 2024-04-17T10:05:34.262 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52757.log 2024-04-17T10:05:34.262 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37005.log.gz 2024-04-17T10:05:34.263 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53235.log 2024-04-17T10:05:34.265 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42130.log.gz 2024-04-17T10:05:34.266 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.52757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46019.log 2024-04-17T10:05:34.266 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.52757.log.gz 2024-04-17T10:05:34.266 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.53235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31768.log 2024-04-17T10:05:34.266 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46659.log 2024-04-17T10:05:34.267 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.31768.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.46019.log.gz 2024-04-17T10:05:34.267 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49666.log 2024-04-17T10:05:34.268 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.31768.log.gz 2024-04-17T10:05:34.269 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34183.log 2024-04-17T10:05:34.269 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46659.log.gz 2024-04-17T10:05:34.269 INFO:teuthology.orchestra.run.smithi032.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.0.53235.log.gz 2024-04-17T10:05:34.270 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44772.log 2024-04-17T10:05:34.270 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49666.log.gz 2024-04-17T10:05:34.271 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46562.log 2024-04-17T10:05:34.271 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34183.log.gz 2024-04-17T10:05:34.272 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33686.log 2024-04-17T10:05:34.272 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44772.log.gz 2024-04-17T10:05:34.273 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35010.log 2024-04-17T10:05:34.275 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44129.log 2024-04-17T10:05:34.275 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33686.log.gz 2024-04-17T10:05:34.275 INFO:teuthology.orchestra.run.smithi032.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.46562.log.gz 2024-04-17T10:05:34.275 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54246.log 2024-04-17T10:05:34.276 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35010.log.gz 2024-04-17T10:05:34.277 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48075.log 2024-04-17T10:05:34.277 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44129.log.gz 2024-04-17T10:05:34.278 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.54246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34848.log 2024-04-17T10:05:34.278 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54246.log.gz 2024-04-17T10:05:34.279 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40128.log 2024-04-17T10:05:34.279 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48075.log.gz 2024-04-17T10:05:34.280 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34848.log: 91.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-17T10:05:34.280 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48220.log 2024-04-17T10:05:34.280 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34848.log.gz 2024-04-17T10:05:34.280 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35608.log 2024-04-17T10:05:34.282 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.48220.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40128.log.gz 2024-04-17T10:05:34.282 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43111.log 2024-04-17T10:05:34.282 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48220.log.gz 2024-04-17T10:05:34.283 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.35608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44338.log 2024-04-17T10:05:34.283 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35608.log.gz 2024-04-17T10:05:34.284 INFO:teuthology.orchestra.run.smithi032.stderr:gzip/var/log/ceph/ceph-client.admin.43111.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41706.log 2024-04-17T10:05:34.284 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43111.log.gz 2024-04-17T10:05:34.285 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.44338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38404.log 2024-04-17T10:05:34.285 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44338.log.gz 2024-04-17T10:05:34.286 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.41706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36829.log 2024-04-17T10:05:34.286 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41706.log.gz 2024-04-17T10:05:34.287 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47812.log 2024-04-17T10:05:34.288 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.38404.log: /var/log/ceph/ceph-client.admin.36829.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.38404.log.gz 2024-04-17T10:05:34.288 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49529.log 2024-04-17T10:05:34.288 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36829.log.gz 2024-04-17T10:05:34.289 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.47812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40030.log 2024-04-17T10:05:34.289 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47812.log.gz 2024-04-17T10:05:34.290 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48965.log 2024-04-17T10:05:34.291 INFO:teuthology.orchestra.run.smithi032.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.49529.log.gz 2024-04-17T10:05:34.291 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.40030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37414.log 2024-04-17T10:05:34.292 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.0.48965.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40030.log.gz 2024-04-17T10:05:34.293 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42429.log 2024-04-17T10:05:34.294 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.37414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33948.log 2024-04-17T10:05:34.294 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37414.log.gz 2024-04-17T10:05:34.295 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27670.log 2024-04-17T10:05:34.295 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42429.log.gz 2024-04-17T10:05:34.296 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.33948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34216.log 2024-04-17T10:05:34.296 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33948.log.gz 2024-04-17T10:05:34.297 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.27670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46591.log 2024-04-17T10:05:34.297 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27670.log.gz 2024-04-17T10:05:34.299 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.34216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49456.log 2024-04-17T10:05:34.299 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34216.log.gz 2024-04-17T10:05:34.299 INFO:teuthology.orchestra.run.smithi032.stderr: 94.5% -- replaced with /var/log/ceph/ceph-client.0.48965.log.gz 2024-04-17T10:05:34.299 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.46591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46250.log 2024-04-17T10:05:34.300 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49162.log 2024-04-17T10:05:34.301 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49456.log: /var/log/ceph/ceph-client.admin.46250.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.46591.log.gz 2024-04-17T10:05:34.302 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42208.log 2024-04-17T10:05:34.302 INFO:teuthology.orchestra.run.smithi032.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.49456.log.gz 2024-04-17T10:05:34.302 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46250.log.gz 2024-04-17T10:05:34.303 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.49162.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49162.log.gz 2024-04-17T10:05:34.303 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.42208.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42208.log.gz 2024-04-17T10:05:34.363 INFO:teuthology.orchestra.run.smithi032.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-17T10:05:34.383 INFO:teuthology.orchestra.run.smithi032.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-17T10:05:34.410 INFO:teuthology.orchestra.run.smithi032.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-17T10:05:34.412 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-04-17T10:05:34.412 INFO:teuthology.orchestra.run.smithi032.stderr:real 0m1.162s 2024-04-17T10:05:34.412 INFO:teuthology.orchestra.run.smithi032.stderr:user 0m5.227s 2024-04-17T10:05:34.412 INFO:teuthology.orchestra.run.smithi032.stderr:sys 0m1.617s 2024-04-17T10:05:34.413 INFO:tasks.ceph:Archiving logs... 2024-04-17T10:05:34.413 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660029/remote/smithi032/log 2024-04-17T10:05:34.414 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-17T10:05:35.555 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660029/remote/smithi144/log 2024-04-17T10:05:35.556 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-17T10:05:35.923 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-17T10:05:35.933 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-17T10:05:35.933 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-17T10:05:35.937 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-17T10:05:36.021 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-17T10:05:36.030 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-17T10:05:36.030 DEBUG:teuthology.orchestra.run.smithi032:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T10:05:36.033 DEBUG:teuthology.orchestra.run.smithi144:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T10:05:36.046 INFO:teuthology.orchestra.run.smithi032.stdout: remote refid st t when poll reach delay offset jitter 2024-04-17T10:05:36.046 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================== 2024-04-17T10:05:36.046 INFO:teuthology.orchestra.run.smithi032.stdout:+hv01.front.sepi 67.205.162.81 3 u 63 64 377 0.093 -5.219 1.461 2024-04-17T10:05:36.046 INFO:teuthology.orchestra.run.smithi032.stdout:*hv02.front.sepi 63.231.80.2 3 u 65 64 377 0.076 -4.301 2.446 2024-04-17T10:05:36.046 INFO:teuthology.orchestra.run.smithi032.stdout:+hv03.front.sepi 74.6.168.72 3 u 57 64 377 0.099 -5.669 2.066 2024-04-17T10:05:36.047 INFO:teuthology.orchestra.run.smithi032.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-04-17T10:05:36.049 INFO:teuthology.orchestra.run.smithi144.stdout: remote refid st t when poll reach delay offset jitter 2024-04-17T10:05:36.049 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================== 2024-04-17T10:05:36.049 INFO:teuthology.orchestra.run.smithi144.stdout:+hv01.front.sepi 67.205.162.81 3 u 59 64 377 0.075 -3.508 1.304 2024-04-17T10:05:36.049 INFO:teuthology.orchestra.run.smithi144.stdout:+hv02.front.sepi 63.231.80.2 3 u 65 64 377 0.118 +0.073 1.705 2024-04-17T10:05:36.049 INFO:teuthology.orchestra.run.smithi144.stdout:*hv03.front.sepi 74.6.168.72 3 u 57 64 377 0.105 -4.035 1.665 2024-04-17T10:05:36.049 INFO:teuthology.orchestra.run.smithi144.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-04-17T10:05:36.050 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-17T10:05:36.060 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-17T10:05:36.060 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-17T10:05:36.069 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-17T10:05:36.078 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-17T10:05:36.086 INFO:teuthology.task.internal:Duration was 1451.946399 seconds 2024-04-17T10:05:36.087 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-17T10:05:36.096 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-17T10:05:36.096 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-17T10:05:36.101 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-17T10:05:36.193 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-17T10:05:36.194 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi032.front.sepia.ceph.com 2024-04-17T10:05:36.194 DEBUG:teuthology.orchestra.run.smithi032:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-17T10:05:36.214 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi144.front.sepia.ceph.com 2024-04-17T10:05:36.214 DEBUG:teuthology.orchestra.run.smithi144:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-17T10:05:36.235 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-17T10:05:36.235 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-17T10:05:36.262 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-17T10:05:36.316 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-17T10:05:36.316 DEBUG:teuthology.orchestra.run.smithi032:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-17T10:05:36.319 DEBUG:teuthology.orchestra.run.smithi144:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-17T10:05:36.427 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-17T10:05:36.438 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-17T10:05:36.438 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-17T10:05:36.458 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-17T10:05:36.478 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-17T10:05:36.488 DEBUG:teuthology.orchestra.run.smithi032:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-17T10:05:36.502 DEBUG:teuthology.orchestra.run.smithi144:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-17T10:05:36.516 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = core 2024-04-17T10:05:36.535 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = core 2024-04-17T10:05:36.553 DEBUG:teuthology.orchestra.run.smithi032:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-17T10:05:36.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:05:36.582 DEBUG:teuthology.orchestra.run.smithi144:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-17T10:05:36.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T10:05:36.605 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-17T10:05:36.616 INFO:teuthology.task.internal:Transferring archived files... 2024-04-17T10:05:36.616 DEBUG:teuthology.misc:Transferring archived files from smithi032:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660029/remote/smithi032 2024-04-17T10:05:36.617 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-17T10:05:36.650 DEBUG:teuthology.misc:Transferring archived files from smithi144:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660029/remote/smithi144 2024-04-17T10:05:36.650 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-17T10:05:36.676 INFO:teuthology.task.internal:Removing archive directory... 2024-04-17T10:05:36.676 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-17T10:05:36.689 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-17T10:05:36.725 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-17T10:05:36.737 INFO:teuthology.task.internal:Not uploading archives. 2024-04-17T10:05:36.737 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-17T10:05:36.747 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-17T10:05:36.747 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-17T10:05:36.750 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-17T10:05:36.756 INFO:teuthology.orchestra.run.smithi032.stdout: 658120 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 17 10:05 /home/ubuntu/cephtest 2024-04-17T10:05:36.778 INFO:teuthology.orchestra.run.smithi144.stdout: 658121 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 17 10:05 /home/ubuntu/cephtest 2024-04-17T10:05:36.780 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-17T10:05:36.790 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-17T10:05:36.811 INFO:teuthology.nuke:Checking targets against current locks 2024-04-17T10:05:36.838 DEBUG:teuthology.nuke:shortname: smithi032 2024-04-17T10:05:36.838 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-17T10:05:36.862 DEBUG:teuthology.nuke:shortname: smithi144 2024-04-17T10:05:36.862 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-17T10:05:36.866 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660029', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 09:31:06.962818', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAQNVG61I8ge12Sn5QATesek5EUC5OJJA2Lfl5bn1TFZolZGwJ5+ic2yXdRNKlEZ2eFM7ed+K/Dl/RatKvM26U8='} 2024-04-17T10:05:36.884 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660029', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 09:31:06.964098', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDmzWqi31jF9Vw1PLRKqPj3ams9Rzq8QnAwo7r+CIq6CX4JoYwuACajkXfQHwmdK6OwV3MpfHFw9u4d/frZwoZg='} 2024-04-17T10:05:36.913 INFO:teuthology.orchestra.console.smithi032:Power off 2024-04-17T10:05:36.914 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-17T10:05:36.942 DEBUG:teuthology.orchestra.console.smithi032:power off output: Chassis Power Control: Down/Off 2024-04-17T10:05:36.942 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:05:36.956 INFO:teuthology.orchestra.console.smithi144:Power off 2024-04-17T10:05:36.956 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-17T10:05:36.965 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2024-04-17T10:05:36.977 DEBUG:teuthology.orchestra.console.smithi144:power off output: Chassis Power Control: Down/Off 2024-04-17T10:05:36.977 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:05:36.999 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-04-17T10:05:40.967 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:05:40.999 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:05:41.077 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2024-04-17T10:05:41.107 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-04-17T10:05:45.079 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:05:45.108 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:05:45.189 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is off 2024-04-17T10:05:45.217 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-04-17T10:05:45.290 INFO:teuthology.orchestra.console.smithi032:Power off completed 2024-04-17T10:05:49.217 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:05:49.327 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2024-04-17T10:05:49.428 INFO:teuthology.orchestra.console.smithi144:Power off completed 2024-04-17T10:05:49.532 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} duration: 1451.946398973465 failure_reason: '"2024-04-17T09:59:36.271169+0000 mds.c (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir)" in cluster log' flavor: default mon.a-kernel-sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 mon.b-kernel-sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 owner: scheduled_rishabh@teuthology success: false 2024-04-17T10:05:49.533 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T10:05:49.609 INFO:teuthology.run:FAIL